2023-11-28T18:03:55.657 INFO:root:teuthology version: 0.0.1.dev209+g5ae09a5 2023-11-28T18:03:55.657 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_vshankar@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/exports} --archive /home/teuthworker/archive/vshankar-2023-11-28_06:23:42-fs-wip-vshankar-testing-20231127.102654-testing-default-smithi/7469459 --verbose --name vshankar-2023-11-28_06:23:42-fs-wip-vshankar-testing-20231127.102654-testing-default-smithi -- /home/teuthworker/archive/vshankar-2023-11-28_06:23:42-fs-wip-vshankar-testing-20231127.102654-testing-default-smithi/7469459/orig.config.yaml 2023-11-28T18:03:55.699 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T18:03:55.821 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2023-11-28_06:23:42-fs-wip-vshankar-testing-20231127.102654-testing-default-smithi/7469459 branch: wip-vshankar-testing-20231127.102654 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/exports} email: vshankar@redhat.com first_in_suite: false job_id: '7469459' kernel: flavor: default kdb: true sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2023-11-28_06:23:42-fs-wip-vshankar-testing-20231127.102654-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-vshankar-testing-20231127.102654 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Replacing daemon mds sha1: 47c1c547372ed5bff7984647dc200bac0e019caf ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 47c1c547372ed5bff7984647dc200bac0e019caf thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20231127.102654 sha1: 47c1c547372ed5bff7984647dc200bac0e019caf owner: scheduled_vshankar@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1098 sha1: 47c1c547372ed5bff7984647dc200bac0e019caf sleep_before_teardown: 0 subset: 854/1024 suite: fs suite_branch: wip-vshankar-testing-20231127.102654 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 47c1c547372ed5bff7984647dc200bac0e019caf syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi105.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDZ22c2VmsOKangy/tmCrNzMEGGlu/JeHoweWe97u2UH0HlxTHil6t5VqDGxukuHSHtFDA2TXg5f2HYYjyPSJjY= smithi148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGxinxnOSbx5MBENicFt1dCm1DSP36mcR2g5+fKgfl6EeU5vDQ38py62JnF5x9dPKj7SFH1W5pzPm1nMdnSnfbY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_exports teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 5ae09a5cc7064ca2bf994df19f3006b79794789a timestamp: 2023-11-28_06:23:42 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-28T18:03:55.821 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa; will attempt to use it 2023-11-28T18:03:55.822 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks 2023-11-28T18:03:55.822 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-28T18:03:55.823 INFO:teuthology.task.internal:Checking packages... 2023-11-28T18:03:55.846 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '47c1c547372ed5bff7984647dc200bac0e019caf' 2023-11-28T18:03:55.847 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-28T18:03:55.847 INFO:teuthology.packaging:ref: None 2023-11-28T18:03:55.847 INFO:teuthology.packaging:tag: None 2023-11-28T18:03:55.847 INFO:teuthology.packaging:branch: wip-vshankar-testing-20231127.102654 2023-11-28T18:03:55.847 INFO:teuthology.packaging:sha1: 47c1c547372ed5bff7984647dc200bac0e019caf 2023-11-28T18:03:55.847 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-vshankar-testing-20231127.102654 2023-11-28T18:03:56.145 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7531-g47c1c547-1jammy 2023-11-28T18:03:56.145 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-28T18:03:56.151 INFO:teuthology.task.internal:no buildpackages task found 2023-11-28T18:03:56.151 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-28T18:03:56.166 INFO:teuthology.task.internal:Saving configuration 2023-11-28T18:03:56.183 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-28T18:03:56.189 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T18:03:56.217 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-11-28_06:23:42-fs-wip-vshankar-testing-20231127.102654-testing-default-smithi/7469459', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 17:58:07.576741', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIP5cSYeHgsq/68X4ln+0glpGC3o2QZ0mIPBgiW9boN8T'} 2023-11-28T18:03:56.239 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-11-28_06:23:42-fs-wip-vshankar-testing-20231127.102654-testing-default-smithi/7469459', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 17:58:07.578703', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAnV7goBLgPOizdlgdfYm41doEP9h5PLyXKHoko1FDPb'} 2023-11-28T18:03:56.239 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-28T18:03:56.245 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-28T18:03:56.246 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-28T18:03:56.246 INFO:teuthology.run_tasks:Running task console_log... 2023-11-28T18:03:56.317 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2d841f3040>, signals=[15]) 2023-11-28T18:03:56.318 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-28T18:03:56.324 INFO:teuthology.task.internal:Opening connections... 2023-11-28T18:03:56.324 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:03:56.327 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T18:03:56.402 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:03:56.403 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T18:03:56.478 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-28T18:03:56.485 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2023-11-28T18:03:56.490 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2023-11-28T18:03:56.491 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2023-11-28T18:03:56.536 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-11-28T18:03:56.537 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="Ubuntu" 2023-11-28T18:03:56.537 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="22.04" 2023-11-28T18:03:56.537 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-11-28T18:03:56.537 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_CODENAME=jammy 2023-11-28T18:03:56.537 INFO:teuthology.orchestra.run.smithi105.stdout:ID=ubuntu 2023-11-28T18:03:56.537 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE=debian 2023-11-28T18:03:56.537 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-28T18:03:56.537 INFO:teuthology.orchestra.run.smithi105.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-28T18:03:56.537 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-28T18:03:56.537 INFO:teuthology.orchestra.run.smithi105.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-28T18:03:56.537 INFO:teuthology.orchestra.run.smithi105.stdout:UBUNTU_CODENAME=jammy 2023-11-28T18:03:56.538 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2023-11-28T18:03:56.559 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2023-11-28T18:03:56.564 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2023-11-28T18:03:56.565 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2023-11-28T18:03:56.615 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-11-28T18:03:56.616 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="Ubuntu" 2023-11-28T18:03:56.616 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="22.04" 2023-11-28T18:03:56.616 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-11-28T18:03:56.616 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_CODENAME=jammy 2023-11-28T18:03:56.616 INFO:teuthology.orchestra.run.smithi148.stdout:ID=ubuntu 2023-11-28T18:03:56.616 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE=debian 2023-11-28T18:03:56.616 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-28T18:03:56.616 INFO:teuthology.orchestra.run.smithi148.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-28T18:03:56.616 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-28T18:03:56.616 INFO:teuthology.orchestra.run.smithi148.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-28T18:03:56.617 INFO:teuthology.orchestra.run.smithi148.stdout:UBUNTU_CODENAME=jammy 2023-11-28T18:03:56.617 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2023-11-28T18:03:56.643 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-28T18:03:56.652 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-28T18:03:56.659 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-28T18:03:56.659 DEBUG:teuthology.orchestra.run.smithi105:> test '!' -e /home/ubuntu/cephtest 2023-11-28T18:03:56.661 DEBUG:teuthology.orchestra.run.smithi148:> test '!' -e /home/ubuntu/cephtest 2023-11-28T18:03:56.665 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-28T18:03:56.671 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-28T18:03:56.671 DEBUG:teuthology.orchestra.run.smithi105:> test -z $(ls -A /var/lib/ceph) 2023-11-28T18:03:56.706 DEBUG:teuthology.orchestra.run.smithi148:> test -z $(ls -A /var/lib/ceph) 2023-11-28T18:03:56.714 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-28T18:03:56.767 INFO:teuthology.run_tasks:Running task kernel... 2023-11-28T18:03:56.784 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'} 2023-11-28T18:03:56.784 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}}, timeout 300 2023-11-28T18:03:56.784 DEBUG:teuthology.orchestra.run.smithi105:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T18:03:56.785 DEBUG:teuthology.orchestra.run.smithi148:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T18:03:56.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:03:56.790 DEBUG:teuthology.task.kernel:sha1 for mon.a is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T18:03:56.790 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-28T18:03:56.790 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2023-11-28T18:03:56.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:03:56.791 DEBUG:teuthology.task.kernel:sha1 for mon.b is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T18:03:56.792 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-28T18:03:56.792 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2023-11-28T18:03:56.839 INFO:teuthology.orchestra.run.smithi105.stdout:5.15.0-56-generic 2023-11-28T18:03:56.839 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T18:03:56.839 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-28T18:03:56.840 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T18:03:56.843 INFO:teuthology.orchestra.run.smithi148.stdout:5.15.0-56-generic 2023-11-28T18:03:56.843 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T18:03:56.843 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-28T18:03:56.844 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T18:03:56.980 INFO:teuthology.task.kernel:Installing linux-firmware on mon.a... 2023-11-28T18:03:56.981 DEBUG:teuthology.orchestra.run.smithi105:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-11-28T18:03:56.995 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2023-11-28T18:03:57.127 INFO:teuthology.task.kernel:Installing linux-firmware on mon.b... 2023-11-28T18:03:57.127 DEBUG:teuthology.orchestra.run.smithi148:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-11-28T18:03:57.140 INFO:teuthology.orchestra.run.smithi105.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2023-11-28T18:03:57.140 INFO:teuthology.orchestra.run.smithi105.stderr:hint: is subject to change. To configure the initial branch name to use in all 2023-11-28T18:03:57.140 INFO:teuthology.orchestra.run.smithi105.stderr:hint: of your new repositories, which will suppress this warning, call: 2023-11-28T18:03:57.140 INFO:teuthology.orchestra.run.smithi105.stderr:hint: 2023-11-28T18:03:57.140 INFO:teuthology.orchestra.run.smithi105.stderr:hint: git config --global init.defaultBranch 2023-11-28T18:03:57.141 INFO:teuthology.orchestra.run.smithi105.stderr:hint: 2023-11-28T18:03:57.141 INFO:teuthology.orchestra.run.smithi105.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2023-11-28T18:03:57.141 INFO:teuthology.orchestra.run.smithi105.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2023-11-28T18:03:57.141 INFO:teuthology.orchestra.run.smithi105.stderr:hint: 2023-11-28T18:03:57.141 INFO:teuthology.orchestra.run.smithi105.stderr:hint: git branch -m 2023-11-28T18:03:57.141 INFO:teuthology.orchestra.run.smithi105.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-11-28T18:03:57.143 DEBUG:teuthology.orchestra.run.smithi105:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-11-28T18:03:57.150 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2023-11-28T18:03:57.215 DEBUG:teuthology.orchestra.run.smithi105:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-11-28T18:03:57.269 DEBUG:teuthology.orchestra.run.smithi105:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-11-28T18:03:57.284 INFO:teuthology.orchestra.run.smithi148.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2023-11-28T18:03:57.284 INFO:teuthology.orchestra.run.smithi148.stderr:hint: is subject to change. To configure the initial branch name to use in all 2023-11-28T18:03:57.284 INFO:teuthology.orchestra.run.smithi148.stderr:hint: of your new repositories, which will suppress this warning, call: 2023-11-28T18:03:57.284 INFO:teuthology.orchestra.run.smithi148.stderr:hint: 2023-11-28T18:03:57.284 INFO:teuthology.orchestra.run.smithi148.stderr:hint: git config --global init.defaultBranch 2023-11-28T18:03:57.284 INFO:teuthology.orchestra.run.smithi148.stderr:hint: 2023-11-28T18:03:57.284 INFO:teuthology.orchestra.run.smithi148.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2023-11-28T18:03:57.284 INFO:teuthology.orchestra.run.smithi148.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2023-11-28T18:03:57.284 INFO:teuthology.orchestra.run.smithi148.stderr:hint: 2023-11-28T18:03:57.284 INFO:teuthology.orchestra.run.smithi148.stderr:hint: git branch -m 2023-11-28T18:03:57.285 INFO:teuthology.orchestra.run.smithi148.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-11-28T18:03:57.286 DEBUG:teuthology.orchestra.run.smithi148:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-11-28T18:03:57.351 DEBUG:teuthology.orchestra.run.smithi148:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-11-28T18:03:57.404 DEBUG:teuthology.orchestra.run.smithi148:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-11-28T18:07:53.539 INFO:teuthology.orchestra.run.smithi148.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-11-28T18:07:53.540 INFO:teuthology.orchestra.run.smithi148.stderr: * [new branch] main -> origin/main 2023-11-28T18:07:53.570 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20190312 -> 20190312 2023-11-28T18:07:53.571 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20190416 -> 20190416 2023-11-28T18:07:53.571 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20190514 -> 20190514 2023-11-28T18:07:53.571 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20190618 -> 20190618 2023-11-28T18:07:53.571 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20190717 -> 20190717 2023-11-28T18:07:53.571 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20190815 -> 20190815 2023-11-28T18:07:53.571 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20190923 -> 20190923 2023-11-28T18:07:53.571 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20191022 -> 20191022 2023-11-28T18:07:53.571 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20191215 -> 20191215 2023-11-28T18:07:53.571 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20200122 -> 20200122 2023-11-28T18:07:53.573 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20200316 -> 20200316 2023-11-28T18:07:53.573 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20200421 -> 20200421 2023-11-28T18:07:53.574 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20200519 -> 20200519 2023-11-28T18:07:53.574 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20200619 -> 20200619 2023-11-28T18:07:53.574 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20200721 -> 20200721 2023-11-28T18:07:53.574 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20200817 -> 20200817 2023-11-28T18:07:53.574 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20200918 -> 20200918 2023-11-28T18:07:53.574 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20201022 -> 20201022 2023-11-28T18:07:53.574 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20201118 -> 20201118 2023-11-28T18:07:53.574 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20201218 -> 20201218 2023-11-28T18:07:53.574 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20210208 -> 20210208 2023-11-28T18:07:53.574 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20210315 -> 20210315 2023-11-28T18:07:53.574 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20210511 -> 20210511 2023-11-28T18:07:53.574 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20210716 -> 20210716 2023-11-28T18:07:53.575 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20210818 -> 20210818 2023-11-28T18:07:53.575 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20210919 -> 20210919 2023-11-28T18:07:53.575 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20211027 -> 20211027 2023-11-28T18:07:53.575 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20211216 -> 20211216 2023-11-28T18:07:53.575 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20220209 -> 20220209 2023-11-28T18:07:53.577 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20220310 -> 20220310 2023-11-28T18:07:53.577 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20220411 -> 20220411 2023-11-28T18:07:53.577 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20220509 -> 20220509 2023-11-28T18:07:53.577 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20220610 -> 20220610 2023-11-28T18:07:53.578 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20220708 -> 20220708 2023-11-28T18:07:53.578 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20220815 -> 20220815 2023-11-28T18:07:53.578 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20220913 -> 20220913 2023-11-28T18:07:53.578 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20221012 -> 20221012 2023-11-28T18:07:53.578 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20221109 -> 20221109 2023-11-28T18:07:53.578 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20221214 -> 20221214 2023-11-28T18:07:53.578 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20230117 -> 20230117 2023-11-28T18:07:53.578 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20230210 -> 20230210 2023-11-28T18:07:53.578 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20230310 -> 20230310 2023-11-28T18:07:53.578 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20230404 -> 20230404 2023-11-28T18:07:53.578 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20230515 -> 20230515 2023-11-28T18:07:53.578 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20230625 -> 20230625 2023-11-28T18:07:53.579 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20230804 -> 20230804 2023-11-28T18:07:53.579 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20230919 -> 20230919 2023-11-28T18:07:53.579 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20231030 -> 20231030 2023-11-28T18:07:53.579 INFO:teuthology.orchestra.run.smithi148.stderr: * [new tag] 20231111 -> 20231111 2023-11-28T18:07:57.560 INFO:teuthology.orchestra.run.smithi148.stderr:Updating files: 51% (1669/3250) Updating files: 52% (1690/3250) Updating files: 53% (1723/3250) Updating files: 54% (1755/3250) Updating files: 55% (1788/3250) Updating files: 56% (1820/3250) Updating files: 57% (1853/3250) Updating files: 58% (1885/3250) Updating files: 59% (1918/3250) Updating files: 60% (1950/3250) Updating files: 61% (1983/3250) Updating files: 62% (2015/3250) Updating files: 63% (2048/3250) Updating files: 64% (2080/3250) Updating files: 65% (2113/3250) Updating files: 65% (2116/3250) Updating files: 66% (2145/3250) Updating files: 67% (2178/3250) Updating files: 68% (2210/3250) Updating files: 69% (2243/3250) Updating files: 70% (2275/3250) Updating files: 71% (2308/3250) Updating files: 72% (2340/3250) Updating files: 73% (2373/3250) Updating files: 74% (2405/3250) Updating files: 75% (2438/3250) Updating files: 76% (2470/3250) Updating files: 77% (2503/3250) Updating files: 78% (2535/3250) Updating files: 79% (2568/3250) Updating files: 80% (2600/3250) Updating files: 81% (2633/3250) Updating files: 81% (2650/3250) Updating files: 82% (2665/3250) Updating files: 83% (2698/3250) Updating files: 84% (2730/3250) Updating files: 85% (2763/3250) Updating files: 86% (2795/3250) Updating files: 87% (2828/3250) Updating files: 88% (2860/3250) Updating files: 89% (2893/3250) Updating files: 90% (2925/3250) Updating files: 91% (2958/3250) Updating files: 92% (2990/3250) Updating files: 93% (3023/3250) Updating files: 94% (3055/3250) Updating files: 95% (3088/3250) Updating files: 96% (3120/3250) Updating files: 97% (3153/3250) Updating files: 98% (3185/3250) Updating files: 99% (3218/3250) Updating files: 100% (3250/3250) Updating files: 100% (3250/3250), done. 2023-11-28T18:07:57.565 INFO:teuthology.orchestra.run.smithi148.stdout:HEAD is now at aae60524 Merge branch 'make-opts' into 'main' 2023-11-28T18:07:57.586 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.b... 2023-11-28T18:07:57.586 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T18:07:57.805 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g8247bbb5b470/ 2023-11-28T18:07:57.805 DEBUG:teuthology.orchestra.run.smithi148:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-g8247bbb5b470_6.6.0-g8247bbb5b470-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g8247bbb5b470/ --input-file=- 2023-11-28T18:07:57.808 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi148.front.sepia.ceph.com to complete... 2023-11-28T18:07:59.560 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28 18:07:59 URL:https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g8247bbb5b470/linux-image-6.6.0-g8247bbb5b470_6.6.0-g8247bbb5b470-1_amd64.deb [38938726/38938726] -> "/tmp/linux-image.deb" [1] 2023-11-28T18:07:59.560 INFO:teuthology.orchestra.run.smithi148.stderr:FINISHED --2023-11-28 18:07:59-- 2023-11-28T18:07:59.560 INFO:teuthology.orchestra.run.smithi148.stderr:Total wall clock time: 1.7s 2023-11-28T18:07:59.560 INFO:teuthology.orchestra.run.smithi148.stderr:Downloaded: 1 files, 37M in 1.5s (25.5 MB/s) 2023-11-28T18:07:59.563 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.b... 2023-11-28T18:07:59.563 DEBUG:teuthology.orchestra.run.smithi148:> sudo dpkg -i /tmp/linux-image.deb 2023-11-28T18:07:59.779 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package linux-image-6.6.0-g8247bbb5b470. 2023-11-28T18:08:00.638 INFO:teuthology.orchestra.run.smithi105.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-11-28T18:08:00.638 INFO:teuthology.orchestra.run.smithi105.stderr: * [new branch] main -> origin/main 2023-11-28T18:08:00.662 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20190312 -> 20190312 2023-11-28T18:08:00.662 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20190416 -> 20190416 2023-11-28T18:08:00.662 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20190514 -> 20190514 2023-11-28T18:08:00.662 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20190618 -> 20190618 2023-11-28T18:08:00.662 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20190717 -> 20190717 2023-11-28T18:08:00.663 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20190815 -> 20190815 2023-11-28T18:08:00.665 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20190923 -> 20190923 2023-11-28T18:08:00.665 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20191022 -> 20191022 2023-11-28T18:08:00.665 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20191215 -> 20191215 2023-11-28T18:08:00.665 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20200122 -> 20200122 2023-11-28T18:08:00.666 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20200316 -> 20200316 2023-11-28T18:08:00.666 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20200421 -> 20200421 2023-11-28T18:08:00.666 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20200519 -> 20200519 2023-11-28T18:08:00.667 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20200619 -> 20200619 2023-11-28T18:08:00.667 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20200721 -> 20200721 2023-11-28T18:08:00.667 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20200817 -> 20200817 2023-11-28T18:08:00.667 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20200918 -> 20200918 2023-11-28T18:08:00.667 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20201022 -> 20201022 2023-11-28T18:08:00.667 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20201118 -> 20201118 2023-11-28T18:08:00.667 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20201218 -> 20201218 2023-11-28T18:08:00.667 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20210208 -> 20210208 2023-11-28T18:08:00.667 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20210315 -> 20210315 2023-11-28T18:08:00.667 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20210511 -> 20210511 2023-11-28T18:08:00.667 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20210716 -> 20210716 2023-11-28T18:08:00.667 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20210818 -> 20210818 2023-11-28T18:08:00.668 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20210919 -> 20210919 2023-11-28T18:08:00.668 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20211027 -> 20211027 2023-11-28T18:08:00.670 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20211216 -> 20211216 2023-11-28T18:08:00.670 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20220209 -> 20220209 2023-11-28T18:08:00.670 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20220310 -> 20220310 2023-11-28T18:08:00.670 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20220411 -> 20220411 2023-11-28T18:08:00.670 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20220509 -> 20220509 2023-11-28T18:08:00.670 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20220610 -> 20220610 2023-11-28T18:08:00.670 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20220708 -> 20220708 2023-11-28T18:08:00.671 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20220815 -> 20220815 2023-11-28T18:08:00.671 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20220913 -> 20220913 2023-11-28T18:08:00.671 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20221012 -> 20221012 2023-11-28T18:08:00.671 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20221109 -> 20221109 2023-11-28T18:08:00.671 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20221214 -> 20221214 2023-11-28T18:08:00.671 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20230117 -> 20230117 2023-11-28T18:08:00.671 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20230210 -> 20230210 2023-11-28T18:08:00.671 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20230310 -> 20230310 2023-11-28T18:08:00.671 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20230404 -> 20230404 2023-11-28T18:08:00.671 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20230515 -> 20230515 2023-11-28T18:08:00.671 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20230625 -> 20230625 2023-11-28T18:08:00.671 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20230804 -> 20230804 2023-11-28T18:08:00.672 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20230919 -> 20230919 2023-11-28T18:08:00.672 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20231030 -> 20231030 2023-11-28T18:08:00.672 INFO:teuthology.orchestra.run.smithi105.stderr: * [new tag] 20231111 -> 20231111 2023-11-28T18:08:01.751 INFO:teuthology.orchestra.run.smithi148.stdout:(Reading database ... 129542 files and directories currently installed.) 2023-11-28T18:08:01.755 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-11-28T18:08:01.856 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking linux-image-6.6.0-g8247bbb5b470 (6.6.0-g8247bbb5b470-1) ... 2023-11-28T18:08:04.652 INFO:teuthology.orchestra.run.smithi105.stderr:Updating files: 51% (1669/3250) Updating files: 52% (1690/3250) Updating files: 53% (1723/3250) Updating files: 54% (1755/3250) Updating files: 55% (1788/3250) Updating files: 56% (1820/3250) Updating files: 57% (1853/3250) Updating files: 58% (1885/3250) Updating files: 59% (1918/3250) Updating files: 60% (1950/3250) Updating files: 61% (1983/3250) Updating files: 62% (2015/3250) Updating files: 63% (2048/3250) Updating files: 64% (2080/3250) Updating files: 65% (2113/3250) Updating files: 65% (2116/3250) Updating files: 66% (2145/3250) Updating files: 67% (2178/3250) Updating files: 68% (2210/3250) Updating files: 69% (2243/3250) Updating files: 70% (2275/3250) Updating files: 71% (2308/3250) Updating files: 72% (2340/3250) Updating files: 73% (2373/3250) Updating files: 74% (2405/3250) Updating files: 75% (2438/3250) Updating files: 76% (2470/3250) Updating files: 77% (2503/3250) Updating files: 78% (2535/3250) Updating files: 79% (2568/3250) Updating files: 80% (2600/3250) Updating files: 81% (2633/3250) Updating files: 81% (2650/3250) Updating files: 82% (2665/3250) Updating files: 83% (2698/3250) Updating files: 84% (2730/3250) Updating files: 85% (2763/3250) Updating files: 86% (2795/3250) Updating files: 87% (2828/3250) Updating files: 88% (2860/3250) Updating files: 89% (2893/3250) Updating files: 90% (2925/3250) Updating files: 91% (2958/3250) Updating files: 92% (2990/3250) Updating files: 93% (3023/3250) Updating files: 94% (3055/3250) Updating files: 95% (3088/3250) Updating files: 96% (3120/3250) Updating files: 97% (3153/3250) Updating files: 98% (3185/3250) Updating files: 99% (3218/3250) Updating files: 100% (3250/3250) Updating files: 100% (3250/3250), done. 2023-11-28T18:08:04.657 INFO:teuthology.orchestra.run.smithi105.stdout:HEAD is now at aae60524 Merge branch 'make-opts' into 'main' 2023-11-28T18:08:04.677 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.a... 2023-11-28T18:08:04.677 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T18:08:04.905 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g8247bbb5b470/ 2023-11-28T18:08:04.906 DEBUG:teuthology.orchestra.run.smithi105:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-g8247bbb5b470_6.6.0-g8247bbb5b470-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g8247bbb5b470/ --input-file=- 2023-11-28T18:08:04.908 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi105.front.sepia.ceph.com to complete... 2023-11-28T18:08:05.122 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up linux-image-6.6.0-g8247bbb5b470 (6.6.0-g8247bbb5b470-1) ... 2023-11-28T18:08:05.237 INFO:teuthology.orchestra.run.smithi148.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-g8247bbb5b470 2023-11-28T18:08:06.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28 18:08:06 URL:https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g8247bbb5b470/linux-image-6.6.0-g8247bbb5b470_6.6.0-g8247bbb5b470-1_amd64.deb [38938726/38938726] -> "/tmp/linux-image.deb" [1] 2023-11-28T18:08:06.468 INFO:teuthology.orchestra.run.smithi105.stderr:FINISHED --2023-11-28 18:08:06-- 2023-11-28T18:08:06.468 INFO:teuthology.orchestra.run.smithi105.stderr:Total wall clock time: 1.5s 2023-11-28T18:08:06.468 INFO:teuthology.orchestra.run.smithi105.stderr:Downloaded: 1 files, 37M in 1.4s (27.4 MB/s) 2023-11-28T18:08:06.471 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.a... 2023-11-28T18:08:06.471 DEBUG:teuthology.orchestra.run.smithi105:> sudo dpkg -i /tmp/linux-image.deb 2023-11-28T18:08:06.716 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package linux-image-6.6.0-g8247bbb5b470. 2023-11-28T18:08:08.690 INFO:teuthology.orchestra.run.smithi105.stdout:(Reading database ... 129542 files and directories currently installed.) 2023-11-28T18:08:08.694 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-11-28T18:08:08.803 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking linux-image-6.6.0-g8247bbb5b470 (6.6.0-g8247bbb5b470-1) ... 2023-11-28T18:08:12.407 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up linux-image-6.6.0-g8247bbb5b470 (6.6.0-g8247bbb5b470-1) ... 2023-11-28T18:08:12.519 INFO:teuthology.orchestra.run.smithi105.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-g8247bbb5b470 2023-11-28T18:08:25.693 INFO:teuthology.orchestra.run.smithi148.stderr:Sourcing file `/etc/default/grub' 2023-11-28T18:08:25.707 INFO:teuthology.orchestra.run.smithi148.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-28T18:08:25.745 INFO:teuthology.orchestra.run.smithi148.stderr:Generating grub configuration file ... 2023-11-28T18:08:26.049 INFO:teuthology.orchestra.run.smithi148.stderr:Found linux image: /boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-28T18:08:26.063 INFO:teuthology.orchestra.run.smithi148.stderr:Found initrd image: /boot/initrd.img-6.6.0-g8247bbb5b470 2023-11-28T18:08:26.356 INFO:teuthology.orchestra.run.smithi148.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-11-28T18:08:26.367 INFO:teuthology.orchestra.run.smithi148.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-11-28T18:08:26.432 INFO:teuthology.orchestra.run.smithi148.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-11-28T18:08:26.437 INFO:teuthology.orchestra.run.smithi148.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-11-28T18:08:26.582 INFO:teuthology.orchestra.run.smithi148.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-11-28T18:08:26.582 INFO:teuthology.orchestra.run.smithi148.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-11-28T18:08:26.583 INFO:teuthology.orchestra.run.smithi148.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-11-28T18:08:26.624 INFO:teuthology.orchestra.run.smithi148.stderr:done 2023-11-28T18:08:26.779 DEBUG:teuthology.orchestra.run.smithi148:> dpkg-deb -c /tmp/linux-image.deb 2023-11-28T18:08:26.787 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./ 2023-11-28T18:08:26.787 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./boot/ 2023-11-28T18:08:26.787 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 4415617 2023-11-23 03:47 ./boot/System.map-6.6.0-g8247bbb5b470 2023-11-28T18:08:26.799 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 169214 2023-11-23 03:47 ./boot/config-6.6.0-g8247bbb5b470 2023-11-28T18:08:26.799 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11352832 2023-11-23 03:47 ./boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-28T18:08:26.821 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/ 2023-11-28T18:08:26.821 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/ 2023-11-28T18:08:26.821 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/postinst.d/ 2023-11-28T18:08:26.821 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/postrm.d/ 2023-11-28T18:08:26.821 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/preinst.d/ 2023-11-28T18:08:26.821 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/prerm.d/ 2023-11-28T18:08:26.821 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/ 2023-11-28T18:08:26.821 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/ 2023-11-28T18:08:26.821 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/ 2023-11-28T18:08:26.822 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/ 2023-11-28T18:08:26.822 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/ 2023-11-28T18:08:26.822 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/ 2023-11-28T18:08:26.822 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ 2023-11-28T18:08:26.822 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 490233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/aesni-intel.ko 2023-11-28T18:08:26.822 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-28T18:08:26.822 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 29529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-28T18:08:26.822 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-28T18:08:26.822 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 93113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-28T18:08:26.822 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 106729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-28T18:08:26.823 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 152241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-28T18:08:26.823 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-28T18:08:26.823 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-28T18:08:26.823 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 83889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-28T18:08:26.823 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-28T18:08:26.823 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 47225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-28T18:08:26.823 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 47209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-28T18:08:26.823 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 51801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-28T18:08:26.823 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 38937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-28T18:08:26.823 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 40529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-28T18:08:26.823 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 58585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-28T18:08:26.823 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 50497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-28T18:08:26.824 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 32065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-28T18:08:26.824 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-28T18:08:26.824 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/ 2023-11-28T18:08:26.824 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/ 2023-11-28T18:08:26.824 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/ 2023-11-28T18:08:26.824 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 40113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-28T18:08:26.824 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpuid.ko 2023-11-28T18:08:26.824 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/msr.ko 2023-11-28T18:08:26.824 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/ 2023-11-28T18:08:26.824 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 354433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-28T18:08:26.825 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 881625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-28T18:08:26.825 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 2275865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-28T18:08:26.865 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ 2023-11-28T18:08:26.866 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 53697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/af_alg.ko 2023-11-28T18:08:26.866 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/algif_hash.ko 2023-11-28T18:08:26.866 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/algif_skcipher.ko 2023-11-28T18:08:26.866 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-28T18:08:26.866 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-28T18:08:26.866 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-28T18:08:26.866 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/asymmetric_keys/ 2023-11-28T18:08:26.866 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-11-28T18:08:26.866 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/ 2023-11-28T18:08:26.867 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-28T18:08:26.867 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-28T18:08:26.867 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-28T18:08:26.867 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-28T18:08:26.867 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-28T18:08:26.867 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-28T18:08:26.867 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-28T18:08:26.867 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-28T18:08:26.867 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-28T18:08:26.867 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-28T18:08:26.867 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-28T18:08:26.867 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 38809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-28T18:08:26.868 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 27321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-28T18:08:26.868 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-28T18:08:26.868 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-28T18:08:26.868 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 35097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-28T18:08:26.868 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-28T18:08:26.868 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-28T18:08:26.868 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 47505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cryptd.ko 2023-11-28T18:08:26.868 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 39505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-28T18:08:26.868 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_simd.ko 2023-11-28T18:08:26.868 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-28T18:08:26.868 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-28T18:08:26.869 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-28T18:08:26.869 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-28T18:08:26.869 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-28T18:08:26.869 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-28T18:08:26.869 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-28T18:08:26.869 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-28T18:08:26.869 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lrw.ko 2023-11-28T18:08:26.869 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-28T18:08:26.869 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-28T18:08:26.869 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-28T18:08:26.869 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-28T18:08:26.869 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-28T18:08:26.870 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-28T18:08:26.870 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-28T18:08:26.870 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 29393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-28T18:08:26.870 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 27081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-28T18:08:26.870 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 149193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-28T18:08:26.870 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-28T18:08:26.870 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-28T18:08:26.870 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-28T18:08:26.870 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-28T18:08:26.870 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 36185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-28T18:08:26.870 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-28T18:08:26.870 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-28T18:08:26.871 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xts.ko 2023-11-28T18:08:26.871 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-28T18:08:26.871 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ 2023-11-28T18:08:26.871 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ 2023-11-28T18:08:26.871 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_extlog.ko 2023-11-28T18:08:26.871 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-28T18:08:26.871 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-28T18:08:26.871 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/apei/ 2023-11-28T18:08:26.871 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 31241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/apei/einj.ko 2023-11-28T18:08:26.871 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-28T18:08:26.871 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-28T18:08:26.872 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-28T18:08:26.872 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-28T18:08:26.872 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 103025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-28T18:08:26.872 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ 2023-11-28T18:08:26.872 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-28T18:08:26.872 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 88865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci.ko 2023-11-28T18:08:26.872 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-28T18:08:26.872 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 88289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci.ko 2023-11-28T18:08:26.872 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 34985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-28T18:08:26.872 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-28T18:08:26.872 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-28T18:08:26.873 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 33505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-28T18:08:26.873 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-28T18:08:26.873 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-28T18:08:26.873 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-28T18:08:26.873 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-28T18:08:26.873 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-28T18:08:26.873 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-28T18:08:26.873 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-28T18:08:26.873 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-28T18:08:26.873 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 33977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-28T18:08:26.873 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-28T18:08:26.873 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-28T18:08:26.874 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-28T18:08:26.874 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 35273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-28T18:08:26.874 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-28T18:08:26.874 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 51481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_legacy.ko 2023-11-28T18:08:26.874 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-28T18:08:26.874 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-28T18:08:26.874 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-28T18:08:26.874 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-28T18:08:26.874 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-28T18:08:26.874 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-28T18:08:26.874 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-28T18:08:26.875 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-28T18:08:26.875 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-28T18:08:26.875 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 32585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-28T18:08:26.875 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 37793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-28T18:08:26.875 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-28T18:08:26.875 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-28T18:08:26.875 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_radisys.ko 2023-11-28T18:08:26.875 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-28T18:08:26.875 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rz1000.ko 2023-11-28T18:08:26.875 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-28T18:08:26.875 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-28T18:08:26.876 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-28T18:08:26.876 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-28T18:08:26.876 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-28T18:08:26.876 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-28T18:08:26.876 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-28T18:08:26.876 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 29121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-28T18:08:26.876 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 83345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-28T18:08:26.876 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 70393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-28T18:08:26.876 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 37377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-28T18:08:26.876 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-28T18:08:26.876 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 29273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-28T18:08:26.876 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 38337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-28T18:08:26.877 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-28T18:08:26.877 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-28T18:08:26.877 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 44793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-28T18:08:26.877 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-28T18:08:26.877 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 35065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-28T18:08:26.877 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-28T18:08:26.877 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ 2023-11-28T18:08:26.877 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-28T18:08:26.877 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/ 2023-11-28T18:08:26.877 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 124473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-28T18:08:26.877 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/ 2023-11-28T18:08:26.878 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/ 2023-11-28T18:08:26.878 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 113665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-28T18:08:26.878 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/ 2023-11-28T18:08:26.878 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 965585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-28T18:08:26.878 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 173225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-28T18:08:26.878 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/ 2023-11-28T18:08:26.878 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 94297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-28T18:08:26.878 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 115793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-28T18:08:26.878 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/ 2023-11-28T18:08:26.878 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 110385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-28T18:08:26.878 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 98193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-28T18:08:26.879 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 263769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-28T18:08:26.879 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/ 2023-11-28T18:08:26.879 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 98145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-28T18:08:26.879 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/ 2023-11-28T18:08:26.879 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 60217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-28T18:08:26.879 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ 2023-11-28T18:08:26.879 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/applicom.ko 2023-11-28T18:08:26.879 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-28T18:08:26.879 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/ 2023-11-28T18:08:26.879 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-28T18:08:26.879 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-28T18:08:26.879 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-28T18:08:26.880 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-28T18:08:26.880 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-28T18:08:26.880 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ 2023-11-28T18:08:26.880 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-28T18:08:26.880 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 148601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-28T18:08:26.880 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-28T18:08:26.880 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 139313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-28T18:08:26.880 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 70921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-28T18:08:26.880 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 53513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-28T18:08:26.880 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 36321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-28T18:08:26.881 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/ 2023-11-28T18:08:26.881 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 84097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-28T18:08:26.881 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/nvram.ko 2023-11-28T18:08:26.881 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 32169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-28T18:08:26.881 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 51113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-28T18:08:26.881 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/ 2023-11-28T18:08:26.881 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-28T18:08:26.881 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-11-28T18:08:26.881 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-11-28T18:08:26.881 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 32985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-11-28T18:08:26.881 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-28T18:08:26.881 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-28T18:08:26.882 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-11-28T18:08:26.882 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/ 2023-11-28T18:08:26.882 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/xillybus_class.ko 2023-11-28T18:08:26.882 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 50945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/xillybus_core.ko 2023-11-28T18:08:26.882 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-11-28T18:08:26.882 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/ 2023-11-28T18:08:26.882 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-palmas.ko 2023-11-28T18:08:26.882 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 47849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-si5351.ko 2023-11-28T18:08:26.882 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-twl6040.ko 2023-11-28T18:08:26.882 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-wm831x.ko 2023-11-28T18:08:26.882 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/ 2023-11-28T18:08:26.885 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-28T18:08:26.885 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-28T18:08:26.885 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-28T18:08:26.885 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ 2023-11-28T18:08:26.886 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ 2023-11-28T18:08:26.886 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 70801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-28T18:08:26.886 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 243025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-28T18:08:26.886 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/ 2023-11-28T18:08:26.886 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/ 2023-11-28T18:08:26.886 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/ 2023-11-28T18:08:26.886 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 438761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-28T18:08:26.886 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-11-28T18:08:26.886 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-28T18:08:26.886 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-28T18:08:26.886 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-28T18:08:26.886 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/ 2023-11-28T18:08:26.887 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 70273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-28T18:08:26.887 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/ 2023-11-28T18:08:26.887 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 29785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-28T18:08:26.887 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ 2023-11-28T18:08:26.887 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/ 2023-11-28T18:08:26.887 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-28T18:08:26.887 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ 2023-11-28T18:08:26.887 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 122569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-28T18:08:26.887 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ 2023-11-28T18:08:26.887 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 79953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-28T18:08:26.887 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 35769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-28T18:08:26.888 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 63617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-28T18:08:26.888 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-28T18:08:26.888 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-28T18:08:26.888 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-28T18:08:26.888 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-28T18:08:26.888 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-28T18:08:26.888 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 52041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-28T18:08:26.888 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-28T18:08:26.888 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-28T18:08:26.888 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 56825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-28T18:08:26.888 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-28T18:08:26.889 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/ 2023-11-28T18:08:26.889 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-gpio.ko 2023-11-28T18:08:26.889 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 27185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-max14577.ko 2023-11-28T18:08:26.889 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 29569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-max77693.ko 2023-11-28T18:08:26.889 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-max8997.ko 2023-11-28T18:08:26.889 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-palmas.ko 2023-11-28T18:08:26.889 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-rt8973a.ko 2023-11-28T18:08:26.889 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 27785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-sm5502.ko 2023-11-28T18:08:26.889 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/ 2023-11-28T18:08:26.889 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 34977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/dmi-sysfs.ko 2023-11-28T18:08:26.889 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/efi/ 2023-11-28T18:08:26.889 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/efi/efi-pstore.ko 2023-11-28T18:08:26.890 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-28T18:08:26.890 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/ 2023-11-28T18:08:26.890 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-adp5520.ko 2023-11-28T18:08:26.890 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-amd8111.ko 2023-11-28T18:08:26.890 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-arizona.ko 2023-11-28T18:08:26.890 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-da9052.ko 2023-11-28T18:08:26.890 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-da9055.ko 2023-11-28T18:08:26.890 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-dln2.ko 2023-11-28T18:08:26.890 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 77849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-f7188x.ko 2023-11-28T18:08:26.890 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-generic.ko 2023-11-28T18:08:26.890 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-28T18:08:26.891 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-11-28T18:08:26.891 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-kempld.ko 2023-11-28T18:08:26.891 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-lp3943.ko 2023-11-28T18:08:26.891 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-max7300.ko 2023-11-28T18:08:26.891 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-max730x.ko 2023-11-28T18:08:26.891 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-max732x.ko 2023-11-28T18:08:26.891 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-11-28T18:08:26.891 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 47073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-pca953x.ko 2023-11-28T18:08:26.891 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-pcf857x.ko 2023-11-28T18:08:26.891 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-rdc321x.ko 2023-11-28T18:08:26.891 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-sch.ko 2023-11-28T18:08:26.891 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-sch311x.ko 2023-11-28T18:08:26.892 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-tps65912.ko 2023-11-28T18:08:26.892 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-twl4030.ko 2023-11-28T18:08:26.892 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-twl6040.ko 2023-11-28T18:08:26.892 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-28T18:08:26.892 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-vx855.ko 2023-11-28T18:08:26.892 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-wm831x.ko 2023-11-28T18:08:26.892 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-wm8350.ko 2023-11-28T18:08:26.892 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-wm8994.ko 2023-11-28T18:08:26.892 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/ 2023-11-28T18:08:26.892 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-a4tech.ko 2023-11-28T18:08:26.892 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 39169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-apple.ko 2023-11-28T18:08:26.892 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-28T18:08:26.893 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-28T18:08:26.893 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-28T18:08:26.893 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-belkin.ko 2023-11-28T18:08:26.893 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-cherry.ko 2023-11-28T18:08:26.893 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-chicony.ko 2023-11-28T18:08:26.893 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 60649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-cp2112.ko 2023-11-28T18:08:26.893 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-cypress.ko 2023-11-28T18:08:26.893 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-28T18:08:26.893 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-28T18:08:26.893 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-28T18:08:26.893 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-28T18:08:26.894 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ezkey.ko 2023-11-28T18:08:26.894 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-28T18:08:26.894 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-generic.ko 2023-11-28T18:08:26.894 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-28T18:08:26.894 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-28T18:08:26.894 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-28T18:08:26.894 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-28T18:08:26.894 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-28T18:08:26.894 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-28T18:08:26.894 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-28T18:08:26.894 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kensington.ko 2023-11-28T18:08:26.895 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-28T18:08:26.895 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-28T18:08:26.895 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-28T18:08:26.895 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-28T18:08:26.895 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 42297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-28T18:08:26.895 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lg-g15.ko 2023-11-28T18:08:26.895 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 58185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-28T18:08:26.895 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 101481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-28T18:08:26.895 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 80993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech.ko 2023-11-28T18:08:26.895 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-magicmouse.ko 2023-11-28T18:08:26.895 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-microsoft.ko 2023-11-28T18:08:26.895 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-monterey.ko 2023-11-28T18:08:26.896 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 50161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-28T18:08:26.896 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 32593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ntrig.ko 2023-11-28T18:08:26.896 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-28T18:08:26.896 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-28T18:08:26.896 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-28T18:08:26.896 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 72937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-28T18:08:26.896 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-28T18:08:26.896 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-28T18:08:26.896 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-28T18:08:26.896 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 35393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-28T18:08:26.896 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-28T18:08:26.897 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-28T18:08:26.897 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 31425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-28T18:08:26.897 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 29729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-28T18:08:26.897 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 31129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-28T18:08:26.897 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-28T18:08:26.897 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 29489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-28T18:08:26.897 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-28T18:08:26.897 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 29313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-28T18:08:26.897 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 27937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-28T18:08:26.897 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-28T18:08:26.897 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-28T18:08:26.897 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-28T18:08:26.898 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-28T18:08:26.898 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 36817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-28T18:08:26.898 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-28T18:08:26.898 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 51249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-28T18:08:26.898 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-28T18:08:26.898 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-28T18:08:26.898 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-28T18:08:26.898 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-28T18:08:26.898 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-28T18:08:26.898 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-28T18:08:26.898 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-28T18:08:26.899 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-28T18:08:26.899 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 71073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-28T18:08:26.899 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-28T18:08:26.899 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 143473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-28T18:08:26.899 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-28T18:08:26.899 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-28T18:08:26.899 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-28T18:08:26.899 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 267825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid.ko 2023-11-28T18:08:26.899 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 32321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-28T18:08:26.899 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/ 2023-11-28T18:08:26.900 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 125721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/usbhid.ko 2023-11-28T18:08:26.900 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/usbkbd.ko 2023-11-28T18:08:26.900 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/usbmouse.ko 2023-11-28T18:08:26.900 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 217873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-28T18:08:26.900 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/ 2023-11-28T18:08:26.900 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 64201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-28T18:08:26.900 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 89625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-28T18:08:26.900 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 305465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-28T18:08:26.900 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ 2023-11-28T18:08:26.900 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 66521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-28T18:08:26.900 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 61393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-28T18:08:26.900 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 34745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-28T18:08:26.901 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-28T18:08:26.901 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-28T18:08:26.901 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-28T18:08:26.901 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 31729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-28T18:08:26.901 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 97977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-28T18:08:26.901 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-28T18:08:26.901 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 45553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-28T18:08:26.901 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-28T18:08:26.901 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-28T18:08:26.902 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-28T18:08:26.902 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-28T18:08:26.902 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 74433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-28T18:08:26.902 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 39585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-28T18:08:26.902 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 67929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-28T18:08:26.902 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-28T18:08:26.902 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 39073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-28T18:08:26.902 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 55201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-28T18:08:26.902 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 49913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-28T18:08:26.902 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 58617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-28T18:08:26.902 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 40825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-28T18:08:26.903 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-28T18:08:26.903 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-28T18:08:26.903 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/da9052-hwmon.ko 2023-11-28T18:08:26.903 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/da9055-hwmon.ko 2023-11-28T18:08:26.903 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 44465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-28T18:08:26.903 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 99601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-28T18:08:26.903 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-28T18:08:26.903 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-28T18:08:26.903 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 29777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-28T18:08:26.903 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 33185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc2103.ko 2023-11-28T18:08:26.903 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-28T18:08:26.904 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 66985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-28T18:08:26.904 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 94457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-28T18:08:26.904 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 41401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-28T18:08:26.904 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-28T18:08:26.904 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 55449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-28T18:08:26.904 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-28T18:08:26.904 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-28T18:08:26.904 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 46801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-28T18:08:26.904 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 45529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-28T18:08:26.904 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hih6130.ko 2023-11-28T18:08:26.904 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-28T18:08:26.904 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-28T18:08:26.905 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-28T18:08:26.905 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 34529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-28T18:08:26.905 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-28T18:08:26.905 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-28T18:08:26.905 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-28T18:08:26.905 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 130449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-28T18:08:26.905 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/jc42.ko 2023-11-28T18:08:26.905 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-28T18:08:26.905 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-28T18:08:26.905 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-28T18:08:26.905 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 48601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-28T18:08:26.905 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-28T18:08:26.906 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 36777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-28T18:08:26.906 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-28T18:08:26.906 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 34593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-28T18:08:26.906 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 31745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-28T18:08:26.906 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-28T18:08:26.906 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 72833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-28T18:08:26.906 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 46153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-28T18:08:26.906 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 37913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-28T18:08:26.906 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-28T18:08:26.906 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 108841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-28T18:08:26.906 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 33393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-28T18:08:26.907 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-28T18:08:26.907 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-28T18:08:26.907 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-28T18:08:26.907 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-28T18:08:26.907 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-28T18:08:26.907 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-28T18:08:26.907 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-28T18:08:26.907 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-28T18:08:26.907 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-28T18:08:26.907 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 33489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-28T18:08:26.907 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-28T18:08:26.907 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-28T18:08:26.908 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-28T18:08:26.908 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 27129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-28T18:08:26.908 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-28T18:08:26.908 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 35777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-28T18:08:26.908 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mc13783-adc.ko 2023-11-28T18:08:26.908 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-28T18:08:26.908 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-11-28T18:08:26.908 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 44129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-28T18:08:26.908 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 119817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-28T18:08:26.908 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 60705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-28T18:08:26.908 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 59737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-28T18:08:26.908 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 82337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-28T18:08:26.909 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 54433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-28T18:08:26.909 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-28T18:08:26.909 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ 2023-11-28T18:08:26.909 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-28T18:08:26.909 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-28T18:08:26.909 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-28T18:08:26.909 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-28T18:08:26.909 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-28T18:08:26.909 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-28T18:08:26.909 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-28T18:08:26.909 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 82577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-28T18:08:26.909 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-28T18:08:26.910 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-28T18:08:26.910 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-28T18:08:26.910 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-28T18:08:26.910 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-28T18:08:26.910 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-28T18:08:26.910 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-28T18:08:26.910 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 27145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-28T18:08:26.910 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 32137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-28T18:08:26.910 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-28T18:08:26.910 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-28T18:08:26.910 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 40105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-28T18:08:26.911 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-28T18:08:26.911 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 39193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-28T18:08:26.911 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 37385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-28T18:08:26.911 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-28T18:08:26.911 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-28T18:08:26.911 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-28T18:08:26.911 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-28T18:08:26.911 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-28T18:08:26.911 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-28T18:08:26.911 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 41945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-28T18:08:26.911 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 52457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-28T18:08:26.912 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 51545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-28T18:08:26.912 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 57449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-28T18:08:26.912 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 78873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-28T18:08:26.912 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 71345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-28T18:08:26.912 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 63225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-28T18:08:26.912 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 71321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-28T18:08:26.912 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 101473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-28T18:08:26.912 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 119505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-28T18:08:26.912 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-28T18:08:26.912 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 35625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-28T18:08:26.912 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-11-28T18:08:26.912 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-11-28T18:08:26.913 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/ 2023-11-28T18:08:26.913 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/ 2023-11-28T18:08:26.913 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-28T18:08:26.913 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-28T18:08:26.913 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/ 2023-11-28T18:08:26.913 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 27969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-11-28T18:08:26.913 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-11-28T18:08:26.913 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 27777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-11-28T18:08:26.913 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-28T18:08:26.913 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-28T18:08:26.913 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-28T18:08:26.914 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-11-28T18:08:26.914 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 6929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-28T18:08:26.914 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-11-28T18:08:26.914 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 50201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-28T18:08:26.914 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-28T18:08:26.914 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-28T18:08:26.914 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-28T18:08:26.914 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-11-28T18:08:26.914 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-11-28T18:08:26.914 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 62817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-28T18:08:26.914 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-28T18:08:26.914 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 47889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-28T18:08:26.915 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-11-28T18:08:26.915 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-28T18:08:26.915 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-28T18:08:26.915 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-11-28T18:08:26.915 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-28T18:08:26.915 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-28T18:08:26.915 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 47241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-28T18:08:26.915 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-11-28T18:08:26.915 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-28T18:08:26.915 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-28T18:08:26.915 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-11-28T18:08:26.916 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 29361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-11-28T18:08:26.916 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-28T18:08:26.916 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-11-28T18:08:26.916 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-28T18:08:26.916 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-28T18:08:26.916 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-28T18:08:26.916 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-28T18:08:26.916 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 47657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-11-28T18:08:26.916 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-28T18:08:26.916 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-28T18:08:26.916 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 27073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-28T18:08:26.916 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/ 2023-11-28T18:08:26.917 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-11-28T18:08:26.917 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-11-28T18:08:26.917 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 29737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-11-28T18:08:26.917 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ 2023-11-28T18:08:26.917 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ 2023-11-28T18:08:26.917 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 273609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-28T18:08:26.917 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 879969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-28T18:08:26.917 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 69273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-28T18:08:26.917 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 324217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-28T18:08:26.917 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 97985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-28T18:08:26.917 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 282273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-28T18:08:26.972 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 66201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-28T18:08:26.972 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/ 2023-11-28T18:08:26.972 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/ 2023-11-28T18:08:26.972 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 493073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-28T18:08:26.972 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/ 2023-11-28T18:08:26.972 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 419737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-28T18:08:26.972 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/ 2023-11-28T18:08:26.972 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 739025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-28T18:08:26.972 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ 2023-11-28T18:08:26.973 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 296625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-28T18:08:26.973 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/ocrdma/ 2023-11-28T18:08:26.973 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 153697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-11-28T18:08:26.973 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/usnic/ 2023-11-28T18:08:26.973 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 168969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-11-28T18:08:26.973 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ 2023-11-28T18:08:26.973 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ 2023-11-28T18:08:26.973 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 263081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-28T18:08:26.973 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ 2023-11-28T18:08:26.973 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 100241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-28T18:08:26.973 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ 2023-11-28T18:08:26.974 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 111425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-28T18:08:26.974 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ 2023-11-28T18:08:26.974 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 128753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-28T18:08:26.974 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ 2023-11-28T18:08:26.974 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 129529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-28T18:08:26.974 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/ 2023-11-28T18:08:26.974 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/evbug.ko 2023-11-28T18:08:26.974 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/ff-memless.ko 2023-11-28T18:08:26.974 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/ 2023-11-28T18:08:26.974 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-28T18:08:26.974 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-28T18:08:26.975 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 41241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-28T18:08:26.975 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 41801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-28T18:08:26.975 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/ 2023-11-28T18:08:26.975 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-11-28T18:08:26.975 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-11-28T18:08:26.975 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-11-28T18:08:26.975 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-11-28T18:08:26.975 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 36665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-28T18:08:26.975 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-11-28T18:08:26.975 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/lkkbd.ko 2023-11-28T18:08:26.975 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/lm8323.ko 2023-11-28T18:08:26.975 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/lm8333.ko 2023-11-28T18:08:26.976 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-11-28T18:08:26.976 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-11-28T18:08:26.976 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-11-28T18:08:26.976 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-11-28T18:08:26.976 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/newtonkbd.ko 2023-11-28T18:08:26.976 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-11-28T18:08:26.976 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/qt1070.ko 2023-11-28T18:08:26.976 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/qt2160.ko 2023-11-28T18:08:26.976 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-11-28T18:08:26.976 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/stowaway.ko 2023-11-28T18:08:26.976 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/sunkbd.ko 2023-11-28T18:08:26.976 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-11-28T18:08:26.977 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-11-28T18:08:26.977 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-11-28T18:08:26.977 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/xtkbd.ko 2023-11-28T18:08:26.977 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/matrix-keymap.ko 2023-11-28T18:08:26.977 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/ 2023-11-28T18:08:26.977 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 34329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-28T18:08:26.977 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 34177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-28T18:08:26.977 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 123681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-28T18:08:26.977 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 89449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-28T18:08:26.977 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/gpio_mouse.ko 2023-11-28T18:08:26.977 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 335929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/psmouse.ko 2023-11-28T18:08:26.978 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-28T18:08:26.978 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-28T18:08:26.978 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-28T18:08:26.978 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-28T18:08:26.978 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/ 2023-11-28T18:08:26.978 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 109873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-28T18:08:26.978 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/ 2023-11-28T18:08:26.978 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-28T18:08:26.978 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-28T18:08:26.978 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/ct82c710.ko 2023-11-28T18:08:26.978 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-28T18:08:26.978 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/parkbd.ko 2023-11-28T18:08:26.979 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/pcips2.ko 2023-11-28T18:08:26.979 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/ps2mult.ko 2023-11-28T18:08:26.979 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-28T18:08:26.979 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serport.ko 2023-11-28T18:08:26.979 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-28T18:08:26.979 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/ 2023-11-28T18:08:26.979 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/ 2023-11-28T18:08:26.979 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 39289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-28T18:08:26.979 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/ 2023-11-28T18:08:26.979 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-88pm860x.ko 2023-11-28T18:08:26.979 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-adp5520.ko 2023-11-28T18:08:26.979 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 59753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-bd2802.ko 2023-11-28T18:08:26.980 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-28T18:08:26.980 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-da903x.ko 2023-11-28T18:08:26.980 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-da9052.ko 2023-11-28T18:08:26.980 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-gpio.ko 2023-11-28T18:08:26.980 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-28T18:08:26.980 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3533.ko 2023-11-28T18:08:26.980 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm355x.ko 2023-11-28T18:08:26.980 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3642.ko 2023-11-28T18:08:26.980 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-28T18:08:26.980 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp8788.ko 2023-11-28T18:08:26.980 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-max8997.ko 2023-11-28T18:08:26.981 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-mc13783.ko 2023-11-28T18:08:26.981 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-menf21bmc.ko 2023-11-28T18:08:26.981 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-pca9532.ko 2023-11-28T18:08:26.981 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-pca955x.ko 2023-11-28T18:08:26.981 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-pca963x.ko 2023-11-28T18:08:26.981 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 34433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-28T18:08:26.981 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-tca6507.ko 2023-11-28T18:08:26.982 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-wm831x-status.ko 2023-11-28T18:08:26.982 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-wm8350.ko 2023-11-28T18:08:26.982 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ 2023-11-28T18:08:26.982 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-28T18:08:26.983 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-28T18:08:26.983 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-28T18:08:26.983 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-28T18:08:26.983 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-28T18:08:26.983 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-28T18:08:26.983 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-28T18:08:26.983 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-28T18:08:26.983 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/ 2023-11-28T18:08:26.983 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/ 2023-11-28T18:08:26.983 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 654969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-28T18:08:26.983 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 40577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-28T18:08:26.983 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 89601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bufio.ko 2023-11-28T18:08:26.984 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 47529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-28T18:08:26.984 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 156009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-28T18:08:26.984 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 106897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-28T18:08:26.984 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-28T18:08:26.984 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 53857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-era.ko 2023-11-28T18:08:26.984 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-28T18:08:26.984 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 41529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-28T18:08:26.984 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 32737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log.ko 2023-11-28T18:08:26.984 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 49545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-mirror.ko 2023-11-28T18:08:26.984 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 78033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-28T18:08:26.984 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-28T18:08:26.984 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 70825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-28T18:08:26.985 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 36297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-region-hash.ko 2023-11-28T18:08:26.985 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-28T18:08:26.985 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-28T18:08:26.985 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 109529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-snapshot.ko 2023-11-28T18:08:26.985 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-28T18:08:26.985 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 181865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-28T18:08:26.985 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 51737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-28T18:08:26.985 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-zero.ko 2023-11-28T18:08:26.985 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-28T18:08:26.985 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-28T18:08:26.985 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 32417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-28T18:08:26.986 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/ 2023-11-28T18:08:26.986 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 199937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-28T18:08:26.986 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 37337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-28T18:08:26.986 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 113073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-28T18:08:26.986 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 158681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-28T18:08:26.986 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 395721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-28T18:08:26.986 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/ 2023-11-28T18:08:26.986 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/ 2023-11-28T18:08:26.986 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 178137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-28T18:08:26.986 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 63385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-28T18:08:26.986 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 44665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-28T18:08:26.986 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 37225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-28T18:08:26.987 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 123321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-28T18:08:26.987 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 79041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-28T18:08:26.987 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 53017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-28T18:08:26.987 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/ 2023-11-28T18:08:26.987 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/88pm800.ko 2023-11-28T18:08:26.987 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/88pm805.ko 2023-11-28T18:08:26.987 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/88pm80x.ko 2023-11-28T18:08:26.987 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/arizona-i2c.ko 2023-11-28T18:08:26.987 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 123721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/arizona.ko 2023-11-28T18:08:26.987 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/bcm590xx.ko 2023-11-28T18:08:26.987 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/cros_ec_dev.ko 2023-11-28T18:08:26.987 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 34961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/dln2.ko 2023-11-28T18:08:26.988 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/janz-cmodio.ko 2023-11-28T18:08:26.988 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 53585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/kempld-core.ko 2023-11-28T18:08:26.988 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 27529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lm3533-core.ko 2023-11-28T18:08:26.988 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-11-28T18:08:26.988 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lp3943.ko 2023-11-28T18:08:26.988 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 54569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-28T18:08:26.988 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-28T18:08:26.988 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/max8907.ko 2023-11-28T18:08:26.988 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 29857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mc13xxx-core.ko 2023-11-28T18:08:26.988 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-11-28T18:08:26.988 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/menf21bmc.ko 2023-11-28T18:08:26.989 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/pcf50633-adc.ko 2023-11-28T18:08:26.989 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/pcf50633-gpio.ko 2023-11-28T18:08:26.989 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 34873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/pcf50633.ko 2023-11-28T18:08:26.989 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-11-28T18:08:26.989 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/retu-mfd.ko 2023-11-28T18:08:26.989 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 80425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/si476x-core.ko 2023-11-28T18:08:26.989 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 48457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-28T18:08:26.989 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-11-28T18:08:26.989 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 51137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/tps65010.ko 2023-11-28T18:08:26.989 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/tps6507x.ko 2023-11-28T18:08:26.989 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-28T18:08:26.989 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-28T18:08:26.990 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-28T18:08:26.990 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 44345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wm8994.ko 2023-11-28T18:08:26.990 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ 2023-11-28T18:08:26.990 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-11-28T18:08:26.990 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 48489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ad525x_dpot.ko 2023-11-28T18:08:26.990 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/ 2023-11-28T18:08:26.990 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 46585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-28T18:08:26.990 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-28T18:08:26.990 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 36729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-28T18:08:26.990 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 45593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-28T18:08:26.990 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/c2port/ 2023-11-28T18:08:26.990 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-11-28T18:08:26.991 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 35281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/c2port/core.ko 2023-11-28T18:08:26.991 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/ 2023-11-28T18:08:26.991 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-28T18:08:26.991 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ds1682.ko 2023-11-28T18:08:26.991 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/dummy-irq.ko 2023-11-28T18:08:26.991 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/echo/ 2023-11-28T18:08:26.991 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/echo/echo.ko 2023-11-28T18:08:26.991 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/ 2023-11-28T18:08:26.991 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 37849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-28T18:08:26.991 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-28T18:08:26.991 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-28T18:08:26.992 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-28T18:08:26.992 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 33009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-28T18:08:26.992 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/genwqe/ 2023-11-28T18:08:26.992 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 152201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-11-28T18:08:26.992 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hmc6352.ko 2023-11-28T18:08:26.992 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 31257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-28T18:08:26.992 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ibmasm/ 2023-11-28T18:08:26.992 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 72129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-11-28T18:08:26.992 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ics932s401.ko 2023-11-28T18:08:26.992 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-28T18:08:26.992 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-28T18:08:26.992 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/ 2023-11-28T18:08:26.993 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 42513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-28T18:08:26.993 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-28T18:08:26.993 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/ 2023-11-28T18:08:26.993 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 125209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-28T18:08:26.993 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 56521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-28T18:08:26.993 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 310113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-28T18:08:26.993 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 31329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/phantom.ko 2023-11-28T18:08:26.993 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ti-st/ 2023-11-28T18:08:26.993 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 74265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ti-st/st_drv.ko 2023-11-28T18:08:26.993 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-28T18:08:26.993 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-28T18:08:26.993 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-28T18:08:26.994 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 48697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-28T18:08:26.994 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/ 2023-11-28T18:08:26.994 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 183609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-28T18:08:26.994 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ 2023-11-28T18:08:26.994 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/ 2023-11-28T18:08:26.994 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 490129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-28T18:08:26.994 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/ 2023-11-28T18:08:26.994 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/ 2023-11-28T18:08:26.994 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 106033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-28T18:08:26.994 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 40417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-28T18:08:26.994 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-28T18:08:26.995 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/ 2023-11-28T18:08:26.995 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 453737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-28T18:08:26.995 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-28T18:08:26.995 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-28T18:08:26.995 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ 2023-11-28T18:08:26.996 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/ 2023-11-28T18:08:26.996 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 45785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-28T18:08:26.996 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 42105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-28T18:08:26.996 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 130553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-28T18:08:26.996 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 68849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-28T18:08:26.996 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ 2023-11-28T18:08:26.996 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 37649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-28T18:08:26.996 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 59689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-28T18:08:26.996 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 27201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-28T18:08:26.996 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 99049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-28T18:08:26.996 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/ 2023-11-28T18:08:26.996 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 54521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-28T18:08:26.997 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/ 2023-11-28T18:08:26.997 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 69137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-28T18:08:26.997 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/ 2023-11-28T18:08:26.997 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 59585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-28T18:08:26.997 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/ 2023-11-28T18:08:26.997 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 41409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-28T18:08:26.997 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/ 2023-11-28T18:08:26.997 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 54609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-28T18:08:26.997 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 49289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-28T18:08:26.997 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 87297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-28T18:08:26.998 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/ 2023-11-28T18:08:26.998 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/ 2023-11-28T18:08:26.998 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 97337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-28T18:08:26.998 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-11-28T18:08:26.998 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 104673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-28T18:08:26.998 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-11-28T18:08:26.998 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 90257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-28T18:08:26.998 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/ 2023-11-28T18:08:26.998 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 81569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-28T18:08:26.998 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 69729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-28T18:08:26.998 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/ 2023-11-28T18:08:26.998 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 78401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-28T18:08:26.999 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 176761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-28T18:08:26.999 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-11-28T18:08:26.999 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 1421497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-28T18:08:26.999 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 128569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-28T18:08:26.999 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/ 2023-11-28T18:08:26.999 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 132089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-28T18:08:26.999 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 312081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-28T18:08:26.999 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/ 2023-11-28T18:08:27.000 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/ 2023-11-28T18:08:27.000 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 323513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-28T18:08:27.000 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/ 2023-11-28T18:08:27.000 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-11-28T18:08:27.000 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 193497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-28T18:08:27.000 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-11-28T18:08:27.000 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 347417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-28T18:08:27.000 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-11-28T18:08:27.000 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 1044497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-28T18:08:27.000 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-11-28T18:08:27.000 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 166193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-28T18:08:27.000 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-11-28T18:08:27.001 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 34881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-28T18:08:27.001 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/ 2023-11-28T18:08:27.001 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/ 2023-11-28T18:08:27.001 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 204345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-28T18:08:27.001 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/ 2023-11-28T18:08:27.001 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/ 2023-11-28T18:08:27.001 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 69449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-28T18:08:27.001 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 61489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-28T18:08:27.001 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 141225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-28T18:08:27.001 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 51497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-28T18:08:27.001 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 62817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-28T18:08:27.002 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 31369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-28T18:08:27.002 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/ 2023-11-28T18:08:27.002 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 60873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-28T18:08:27.002 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 62833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-28T18:08:27.002 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 45721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-28T18:08:27.002 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ec_bhf.ko 2023-11-28T18:08:27.002 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/ 2023-11-28T18:08:27.002 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/ 2023-11-28T18:08:27.002 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 281137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-28T18:08:27.003 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 49457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-28T18:08:27.003 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 51737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-28T18:08:27.003 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fujitsu/ 2023-11-28T18:08:27.003 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 45345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-11-28T18:08:27.003 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ 2023-11-28T18:08:27.003 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 90465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-28T18:08:27.003 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/ 2023-11-28T18:08:27.003 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 295577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-28T18:08:27.003 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/ 2023-11-28T18:08:27.003 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 584497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-28T18:08:27.003 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/ 2023-11-28T18:08:27.004 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 246209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-28T18:08:27.004 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/ 2023-11-28T18:08:27.004 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 925825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-28T18:08:27.004 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/ 2023-11-28T18:08:27.004 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 302937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-28T18:08:27.004 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/ 2023-11-28T18:08:27.004 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 514641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-28T18:08:27.004 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/ 2023-11-28T18:08:27.004 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 93593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-28T18:08:27.004 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-11-28T18:08:27.004 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 734305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-28T18:08:27.004 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-11-28T18:08:27.005 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 151769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-28T18:08:27.005 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 86921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-28T18:08:27.005 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/ 2023-11-28T18:08:27.005 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-28T18:08:27.005 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 102289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-28T18:08:27.005 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 122649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-28T18:08:27.005 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/ 2023-11-28T18:08:27.005 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-11-28T18:08:27.005 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 727601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-28T18:08:27.005 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 278937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-28T18:08:27.005 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-11-28T18:08:27.005 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-11-28T18:08:27.006 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 1113681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-28T18:08:27.006 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ 2023-11-28T18:08:27.006 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 53041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-11-28T18:08:27.006 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 40993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-11-28T18:08:27.006 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-11-28T18:08:27.006 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 95665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-28T18:08:27.006 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/ 2023-11-28T18:08:27.006 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-11-28T18:08:27.006 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 108649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-28T18:08:27.006 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ 2023-11-28T18:08:27.006 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 73441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-28T18:08:27.007 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 47777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-28T18:08:27.007 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/ 2023-11-28T18:08:27.007 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 150905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-28T18:08:27.007 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/ 2023-11-28T18:08:27.007 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 136721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-28T18:08:27.007 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/ 2023-11-28T18:08:27.007 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 50481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-28T18:08:27.007 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 43145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-28T18:08:27.007 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/ 2023-11-28T18:08:27.007 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-11-28T18:08:27.007 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 227897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-28T18:08:27.007 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 87081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-28T18:08:27.008 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-11-28T18:08:27.008 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 672049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-28T18:08:27.008 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/ 2023-11-28T18:08:27.008 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 43017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-28T18:08:27.008 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/ 2023-11-28T18:08:27.008 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 67385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-28T18:08:27.008 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 83985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-28T18:08:27.008 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 29321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-28T18:08:27.008 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 189801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-28T18:08:27.009 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/samsung/ 2023-11-28T18:08:27.009 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-11-28T18:08:27.009 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 104241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-11-28T18:08:27.009 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/ 2023-11-28T18:08:27.009 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 1052497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-28T18:08:27.009 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/ 2023-11-28T18:08:27.009 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 47097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-28T18:08:27.009 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/ 2023-11-28T18:08:27.009 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 54817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-28T18:08:27.009 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 63337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-28T18:08:27.009 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/ 2023-11-28T18:08:27.010 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 61537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-28T18:08:27.010 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 67193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-28T18:08:27.010 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 73409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-28T18:08:27.010 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 68177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-28T18:08:27.010 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/ 2023-11-28T18:08:27.010 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-11-28T18:08:27.010 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-11-28T18:08:27.010 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 39905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-28T18:08:27.010 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-11-28T18:08:27.010 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 487025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-28T18:08:27.010 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/ 2023-11-28T18:08:27.010 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 103337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-28T18:08:27.011 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 166097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-28T18:08:27.011 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 66329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-28T18:08:27.011 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 78801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-28T18:08:27.011 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/ 2023-11-28T18:08:27.011 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 50057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-28T18:08:27.011 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/ 2023-11-28T18:08:27.011 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 86529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-28T18:08:27.011 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/ 2023-11-28T18:08:27.011 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 80273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-28T18:08:27.011 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 83633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-28T18:08:27.011 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/ 2023-11-28T18:08:27.012 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 46241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-28T18:08:27.012 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-28T18:08:27.012 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/ 2023-11-28T18:08:27.012 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 59993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-28T18:08:27.012 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/ 2023-11-28T18:08:27.012 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 41001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/defxx.ko 2023-11-28T18:08:27.012 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/skfp/ 2023-11-28T18:08:27.012 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 216985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/skfp/skfp.ko 2023-11-28T18:08:27.012 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 77345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-28T18:08:27.012 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/ 2023-11-28T18:08:27.012 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 235521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-28T18:08:27.012 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/ 2023-11-28T18:08:27.013 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-28T18:08:27.013 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-28T18:08:27.013 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ 2023-11-28T18:08:27.013 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 83769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-28T18:08:27.013 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 62145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-28T18:08:27.013 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-28T18:08:27.013 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/ 2023-11-28T18:08:27.013 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-28T18:08:27.013 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-28T18:08:27.013 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-gpio.ko 2023-11-28T18:08:27.013 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-28T18:08:27.013 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-28T18:08:27.014 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-28T18:08:27.014 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 43953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-28T18:08:27.014 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-28T18:08:27.014 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/ 2023-11-28T18:08:27.014 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-28T18:08:27.014 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 38977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-28T18:08:27.014 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ 2023-11-28T18:08:27.014 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-28T18:08:27.014 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 50033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-28T18:08:27.014 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 44721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-28T18:08:27.014 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-28T18:08:27.015 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 42241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-28T18:08:27.015 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-28T18:08:27.015 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-28T18:08:27.015 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-28T18:08:27.015 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-28T18:08:27.015 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-28T18:08:27.015 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 82049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-28T18:08:27.015 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 76113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-28T18:08:27.015 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-28T18:08:27.015 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 97857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-28T18:08:27.015 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-28T18:08:27.016 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 50065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-28T18:08:27.016 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 29985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-28T18:08:27.016 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-28T18:08:27.016 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-28T18:08:27.016 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/plip/ 2023-11-28T18:08:27.016 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 43505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/plip/plip.ko 2023-11-28T18:08:27.016 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ 2023-11-28T18:08:27.016 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-28T18:08:27.016 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 29713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-28T18:08:27.016 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-28T18:08:27.016 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-28T18:08:27.016 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-28T18:08:27.017 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 42545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-28T18:08:27.017 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-28T18:08:27.017 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 32689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-28T18:08:27.017 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sb1000.ko 2023-11-28T18:08:27.017 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/ 2023-11-28T18:08:27.017 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 42465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-28T18:08:27.017 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-28T18:08:27.017 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 64449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-28T18:08:27.017 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/ 2023-11-28T18:08:27.017 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 106169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-28T18:08:27.017 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-28T18:08:27.018 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-28T18:08:27.018 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-28T18:08:27.018 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-28T18:08:27.018 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-28T18:08:27.018 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 87017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-28T18:08:27.018 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/ 2023-11-28T18:08:27.018 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 144297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-28T18:08:27.018 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/ 2023-11-28T18:08:27.018 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 297329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-28T18:08:27.018 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/ 2023-11-28T18:08:27.018 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 144585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-28T18:08:27.018 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ 2023-11-28T18:08:27.019 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-28T18:08:27.019 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/ 2023-11-28T18:08:27.019 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/ 2023-11-28T18:08:27.019 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 265121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-28T18:08:27.019 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 102209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-28T18:08:27.019 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/ 2023-11-28T18:08:27.019 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 133841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-28T18:08:27.019 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-28T18:08:27.019 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 84465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-28T18:08:27.019 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-28T18:08:27.019 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/ 2023-11-28T18:08:27.020 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/ 2023-11-28T18:08:27.020 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-28T18:08:27.020 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-11-28T18:08:27.020 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-11-28T18:08:27.020 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pci-stub.ko 2023-11-28T18:08:27.020 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 46465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-28T18:08:27.020 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/ 2023-11-28T18:08:27.020 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-28T18:08:27.020 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 145809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pcmcia.ko 2023-11-28T18:08:27.020 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 62137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pcmcia_core.ko 2023-11-28T18:08:27.020 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 34345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-11-28T18:08:27.020 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 34265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-28T18:08:27.021 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 112889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-28T18:08:27.021 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/ 2023-11-28T18:08:27.021 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/broadcom/ 2023-11-28T18:08:27.021 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-11-28T18:08:27.021 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/samsung/ 2023-11-28T18:08:27.021 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-11-28T18:08:27.021 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/ 2023-11-28T18:08:27.021 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/ 2023-11-28T18:08:27.021 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 70153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-28T18:08:27.021 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/ 2023-11-28T18:08:27.021 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/ 2023-11-28T18:08:27.021 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 50889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-11-28T18:08:27.022 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec.ko 2023-11-28T18:08:27.022 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-11-28T18:08:27.022 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-11-28T18:08:27.022 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-11-28T18:08:27.022 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-11-28T18:08:27.022 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-11-28T18:08:27.022 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/ 2023-11-28T18:08:27.022 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 67097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-28T18:08:27.022 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 39145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-28T18:08:27.022 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 37025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-28T18:08:27.022 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 59665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-28T18:08:27.022 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 31161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-28T18:08:27.023 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 127441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-28T18:08:27.023 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 36849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-28T18:08:27.023 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 49121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-28T18:08:27.023 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-28T18:08:27.023 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 40265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-28T18:08:27.023 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-28T18:08:27.023 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 36593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-28T18:08:27.023 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/ibm_rtl.ko 2023-11-28T18:08:27.023 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/ 2023-11-28T18:08:27.023 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-28T18:08:27.023 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-11-28T18:08:27.024 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 43497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-28T18:08:27.024 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-28T18:08:27.024 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-28T18:08:27.024 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 34801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-28T18:08:27.024 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 42329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-28T18:08:27.024 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-28T18:08:27.024 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-28T18:08:27.024 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-28T18:08:27.024 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-28T18:08:27.024 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-28T18:08:27.024 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 57569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-28T18:08:27.024 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/ 2023-11-28T18:08:27.025 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ 2023-11-28T18:08:27.025 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/88pm860x_battery.ko 2023-11-28T18:08:27.025 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 33985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/88pm860x_charger.ko 2023-11-28T18:08:27.025 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 55057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/bq2415x_charger.ko 2023-11-28T18:08:27.025 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 54081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/bq24190_charger.ko 2023-11-28T18:08:27.025 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/bq24735-charger.ko 2023-11-28T18:08:27.025 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-11-28T18:08:27.025 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/da9030_battery.ko 2023-11-28T18:08:27.025 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/da9052-battery.ko 2023-11-28T18:08:27.025 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 29281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2760_battery.ko 2023-11-28T18:08:27.025 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2780_battery.ko 2023-11-28T18:08:27.026 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2781_battery.ko 2023-11-28T18:08:27.026 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2782_battery.ko 2023-11-28T18:08:27.026 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/gpio-charger.ko 2023-11-28T18:08:27.026 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/isp1704_charger.ko 2023-11-28T18:08:27.026 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/lp8727_charger.ko 2023-11-28T18:08:27.026 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max14577_charger.ko 2023-11-28T18:08:27.026 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max17040_battery.ko 2023-11-28T18:08:27.026 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max17042_battery.ko 2023-11-28T18:08:27.026 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max8903_charger.ko 2023-11-28T18:08:27.026 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max8925_power.ko 2023-11-28T18:08:27.026 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/pcf50633-charger.ko 2023-11-28T18:08:27.026 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 38009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/sbs-battery.ko 2023-11-28T18:08:27.027 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/test_power.ko 2023-11-28T18:08:27.027 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/tps65090-charger.ko 2023-11-28T18:08:27.027 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/wm831x_backup.ko 2023-11-28T18:08:27.027 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/wm831x_power.ko 2023-11-28T18:08:27.027 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/wm8350_power.ko 2023-11-28T18:08:27.027 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/ 2023-11-28T18:08:27.027 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 57729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-28T18:08:27.027 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-28T18:08:27.027 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/ 2023-11-28T18:08:27.027 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/ 2023-11-28T18:08:27.027 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-28T18:08:27.028 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-28T18:08:27.028 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-28T18:08:27.028 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 41873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-28T18:08:27.028 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ 2023-11-28T18:08:27.028 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 66385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-28T18:08:27.028 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-28T18:08:27.028 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/ 2023-11-28T18:08:27.028 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-88pm80x.ko 2023-11-28T18:08:27.028 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-88pm860x.ko 2023-11-28T18:08:27.028 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-28T18:08:27.028 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-da9052.ko 2023-11-28T18:08:27.028 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-da9055.ko 2023-11-28T18:08:27.029 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-da9063.ko 2023-11-28T18:08:27.029 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-28T18:08:27.029 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 68577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-28T18:08:27.029 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-28T18:08:27.029 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-28T18:08:27.029 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-28T18:08:27.029 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-28T18:08:27.030 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-28T18:08:27.030 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-28T18:08:27.030 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-28T18:08:27.030 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-28T18:08:27.030 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-28T18:08:27.030 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-28T18:08:27.030 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 35969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-28T18:08:27.030 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-lp8788.ko 2023-11-28T18:08:27.030 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 38097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-28T18:08:27.030 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-28T18:08:27.030 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-28T18:08:27.031 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t86.ko 2023-11-28T18:08:27.031 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-28T18:08:27.031 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8907.ko 2023-11-28T18:08:27.031 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8925.ko 2023-11-28T18:08:27.031 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8997.ko 2023-11-28T18:08:27.031 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8998.ko 2023-11-28T18:08:27.031 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-11-28T18:08:27.031 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-28T18:08:27.031 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-palmas.ko 2023-11-28T18:08:27.031 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 46633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-28T18:08:27.031 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf50633.ko 2023-11-28T18:08:27.031 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-28T18:08:27.032 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-28T18:08:27.032 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 29817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-28T18:08:27.032 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-28T18:08:27.032 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rc5t583.ko 2023-11-28T18:08:27.032 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-28T18:08:27.032 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 33753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-28T18:08:27.032 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 27833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-28T18:08:27.032 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-28T18:08:27.032 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-28T18:08:27.032 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-s35390a.ko 2023-11-28T18:08:27.032 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-28T18:08:27.032 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-tps6586x.ko 2023-11-28T18:08:27.033 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-tps65910.ko 2023-11-28T18:08:27.033 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-wm831x.ko 2023-11-28T18:08:27.033 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-wm8350.ko 2023-11-28T18:08:27.033 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 27873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-28T18:08:27.033 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ 2023-11-28T18:08:27.033 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 77617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-28T18:08:27.033 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 57665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-28T18:08:27.033 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 55105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-28T18:08:27.033 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 135185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-28T18:08:27.033 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-28T18:08:27.033 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/ 2023-11-28T18:08:27.034 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 272305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-28T18:08:27.034 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 100617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-28T18:08:27.034 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/ 2023-11-28T18:08:27.034 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 356201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-28T18:08:27.034 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 295801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-28T18:08:27.034 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic94xx/ 2023-11-28T18:08:27.034 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 189185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-11-28T18:08:27.034 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-28T18:08:27.034 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/ 2023-11-28T18:08:27.034 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 106881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-28T18:08:27.034 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 38185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-28T18:08:27.034 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/ 2023-11-28T18:08:27.035 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 246985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-28T18:08:27.035 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/ 2023-11-28T18:08:27.035 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 1171945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-28T18:08:27.035 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/ 2023-11-28T18:08:27.035 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 237273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-28T18:08:27.035 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/ 2023-11-28T18:08:27.035 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 115369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-28T18:08:27.035 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 40585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-28T18:08:27.035 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/ 2023-11-28T18:08:27.035 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 308921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-28T18:08:27.035 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/ 2023-11-28T18:08:27.036 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-11-28T18:08:27.036 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 76145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-28T18:08:27.036 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-11-28T18:08:27.036 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 101449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-28T18:08:27.036 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 146689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-28T18:08:27.036 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 75553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-28T18:08:27.036 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 38073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-28T18:08:27.036 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/ 2023-11-28T18:08:27.036 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 212425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-28T18:08:27.036 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 60505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-28T18:08:27.036 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/ 2023-11-28T18:08:27.036 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 78209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-28T18:08:27.037 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 158505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-28T18:08:27.037 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fdomain.ko 2023-11-28T18:08:27.037 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/ 2023-11-28T18:08:27.037 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 263489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-28T18:08:27.037 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 201817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-28T18:08:27.037 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 51209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-28T18:08:27.037 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 49041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-28T18:08:27.037 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 35337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/imm.ko 2023-11-28T18:08:27.037 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 44057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-28T18:08:27.037 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 266537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-28T18:08:27.037 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 101881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-28T18:08:27.037 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/ 2023-11-28T18:08:27.038 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 317153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-28T18:08:27.038 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-28T18:08:27.038 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 52961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-28T18:08:27.038 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/ 2023-11-28T18:08:27.038 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 314865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-28T18:08:27.038 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 145585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-28T18:08:27.038 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 53521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-28T18:08:27.038 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/ 2023-11-28T18:08:27.038 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 216665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-28T18:08:27.038 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/ 2023-11-28T18:08:27.038 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 2197249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-28T18:08:27.039 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/ 2023-11-28T18:08:27.039 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 73569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-28T18:08:27.039 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 33529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-28T18:08:27.039 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 310617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-28T18:08:27.039 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 84721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-28T18:08:27.039 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/ 2023-11-28T18:08:27.039 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 659825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-28T18:08:27.039 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/ 2023-11-28T18:08:27.039 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 119561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-28T18:08:27.039 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 78129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-28T18:08:27.039 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/ 2023-11-28T18:08:27.039 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 81625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-11-28T18:08:27.040 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-11-28T18:08:27.040 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-11-28T18:08:27.040 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-11-28T18:08:27.040 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/ 2023-11-28T18:08:27.040 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 364105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-28T18:08:27.040 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 123617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-28T18:08:27.040 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 34513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ppa.ko 2023-11-28T18:08:27.040 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 55361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-28T18:08:27.040 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/ 2023-11-28T18:08:27.040 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 1876689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-28T18:08:27.040 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 85289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-28T18:08:27.040 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/ 2023-11-28T18:08:27.041 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 559033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-28T18:08:27.041 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qlogicfas408.ko 2023-11-28T18:08:27.041 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-28T18:08:27.041 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 238761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-28T18:08:27.041 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 173937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-28T18:08:27.041 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 279313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-28T18:08:27.041 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 97729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-28T18:08:27.041 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 73881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-28T18:08:27.041 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 44993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-28T18:08:27.041 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-28T18:08:27.041 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 115889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-28T18:08:27.041 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 46825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-28T18:08:27.114 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/ 2023-11-28T18:08:27.114 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 162121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-28T18:08:27.115 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 42297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-28T18:08:27.115 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 50321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-28T18:08:27.115 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 39961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-28T18:08:27.115 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 50025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/xen-scsifront.ko 2023-11-28T18:08:27.115 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ 2023-11-28T18:08:27.115 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 129441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-28T18:08:27.115 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/ 2023-11-28T18:08:27.115 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/ 2023-11-28T18:08:27.115 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 736601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-28T18:08:27.115 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/ 2023-11-28T18:08:27.115 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 49985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-28T18:08:27.115 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 39713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-28T18:08:27.116 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 43025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-28T18:08:27.116 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 930257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-28T18:08:27.116 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 44305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-28T18:08:27.116 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 122521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-28T18:08:27.116 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/ 2023-11-28T18:08:27.116 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 75593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-28T18:08:27.116 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/ 2023-11-28T18:08:27.116 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/ 2023-11-28T18:08:27.116 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/ 2023-11-28T18:08:27.116 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-28T18:08:27.117 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 29321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-28T18:08:27.117 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-28T18:08:27.117 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-28T18:08:27.117 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-28T18:08:27.117 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-28T18:08:27.117 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-28T18:08:27.117 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-28T18:08:27.117 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-28T18:08:27.117 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-28T18:08:27.117 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-28T18:08:27.117 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 37065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-28T18:08:27.118 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-11-28T18:08:27.118 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-28T18:08:27.118 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-28T18:08:27.118 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-28T18:08:27.118 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ 2023-11-28T18:08:27.118 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ 2023-11-28T18:08:27.118 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 78425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-28T18:08:27.118 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 57865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/moxa.ko 2023-11-28T18:08:27.118 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 50953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/mxser.ko 2023-11-28T18:08:27.118 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 108601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-28T18:08:27.118 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 31961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-28T18:08:27.118 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 67929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-28T18:08:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/ 2023-11-28T18:08:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/ 2023-11-28T18:08:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 35753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-11-28T18:08:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 59497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-28T18:08:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-11-28T18:08:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/altera_uart.ko 2023-11-28T18:08:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-28T18:08:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 72425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-11-28T18:08:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/ 2023-11-28T18:08:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 103073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-28T18:08:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/rp2.ko 2023-11-28T18:08:27.119 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 52817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/sc16is7xx.ko 2023-11-28T18:08:27.120 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/ 2023-11-28T18:08:27.120 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ 2023-11-28T18:08:27.120 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 531265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-28T18:08:27.120 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ 2023-11-28T18:08:27.120 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-28T18:08:27.120 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-11-28T18:08:27.120 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/ 2023-11-28T18:08:27.120 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 44513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-28T18:08:27.120 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-28T18:08:27.120 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-28T18:08:27.120 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_dmem_genirq.ko 2023-11-28T18:08:27.121 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_mf624.ko 2023-11-28T18:08:27.121 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_netx.ko 2023-11-28T18:08:27.121 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-28T18:08:27.121 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-11-28T18:08:27.121 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-28T18:08:27.121 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/ 2023-11-28T18:08:27.121 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/c67x00/ 2023-11-28T18:08:27.121 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 73297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/c67x00/c67x00.ko 2023-11-28T18:08:27.121 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ 2023-11-28T18:08:27.121 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 144377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-11-28T18:08:27.121 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-11-28T18:08:27.121 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-11-28T18:08:27.122 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-11-28T18:08:27.122 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/ 2023-11-28T18:08:27.122 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 87089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-28T18:08:27.122 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 51345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-28T18:08:27.122 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 47305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-28T18:08:27.122 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 89857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-28T18:08:27.122 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/common/ 2023-11-28T18:08:27.122 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/common/ulpi.ko 2023-11-28T18:08:27.122 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc2/ 2023-11-28T18:08:27.122 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-11-28T18:08:27.122 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/ 2023-11-28T18:08:27.122 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-11-28T18:08:27.123 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-11-28T18:08:27.123 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 448505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/dwc3.ko 2023-11-28T18:08:27.123 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/ 2023-11-28T18:08:27.123 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/ 2023-11-28T18:08:27.123 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 60177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/u_ether.ko 2023-11-28T18:08:27.123 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 51881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/u_serial.ko 2023-11-28T18:08:27.123 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-11-28T18:08:27.123 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 42449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-11-28T18:08:27.123 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 27953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-11-28T18:08:27.123 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 36513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-11-28T18:08:27.123 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 92489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-11-28T18:08:27.123 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 51049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-11-28T18:08:27.124 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 122081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-11-28T18:08:27.124 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 57681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-11-28T18:08:27.124 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-11-28T18:08:27.124 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 31265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-11-28T18:08:27.124 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 57729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-11-28T18:08:27.124 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 85609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-11-28T18:08:27.124 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-11-28T18:08:27.124 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 58113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-11-28T18:08:27.124 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 67337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-11-28T18:08:27.124 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/ 2023-11-28T18:08:27.124 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-11-28T18:08:27.124 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-11-28T18:08:27.125 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-11-28T18:08:27.125 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-11-28T18:08:27.125 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-11-28T18:08:27.125 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-11-28T18:08:27.125 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-11-28T18:08:27.125 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-11-28T18:08:27.125 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 32073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-11-28T18:08:27.125 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-11-28T18:08:27.125 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-11-28T18:08:27.125 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 27201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-11-28T18:08:27.125 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 62185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-11-28T18:08:27.126 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-11-28T18:08:27.126 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 161225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/libcomposite.ko 2023-11-28T18:08:27.126 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/ 2023-11-28T18:08:27.126 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-11-28T18:08:27.126 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/bdc/ 2023-11-28T18:08:27.126 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 157681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-11-28T18:08:27.126 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 36089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-11-28T18:08:27.126 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 52721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-11-28T18:08:27.126 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 69705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-11-28T18:08:27.126 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 56033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-11-28T18:08:27.126 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 76057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/net2272.ko 2023-11-28T18:08:27.127 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 75073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/net2280.ko 2023-11-28T18:08:27.127 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 73169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-11-28T18:08:27.127 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 65809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-11-28T18:08:27.127 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 66097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-11-28T18:08:27.127 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 81505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-11-28T18:08:27.127 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 156073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-11-28T18:08:27.127 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/ 2023-11-28T18:08:27.127 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/bcma-hcd.ko 2023-11-28T18:08:27.127 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 80113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/isp116x-hcd.ko 2023-11-28T18:08:27.127 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 87353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-11-28T18:08:27.127 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 78873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-11-28T18:08:27.127 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 51329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-28T18:08:27.128 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811_cs.ko 2023-11-28T18:08:27.128 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/ssb-hcd.ko 2023-11-28T18:08:27.128 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-11-28T18:08:27.128 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/ 2023-11-28T18:08:27.128 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 34585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-28T18:08:27.128 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-28T18:08:27.128 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ 2023-11-28T18:08:27.128 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 41721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-28T18:08:27.128 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-28T18:08:27.128 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-11-28T18:08:27.128 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/cytherm.ko 2023-11-28T18:08:27.128 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ehset.ko 2023-11-28T18:08:27.129 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-28T18:08:27.129 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-28T18:08:27.129 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-28T18:08:27.129 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-28T18:08:27.129 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 36553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-28T18:08:27.129 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-28T18:08:27.129 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 36529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-28T18:08:27.129 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 36265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-28T18:08:27.129 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 27793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/lvstest.ko 2023-11-28T18:08:27.129 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/ 2023-11-28T18:08:27.129 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 64841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-28T18:08:27.129 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-28T18:08:27.130 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-28T18:08:27.130 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-28T18:08:27.130 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-28T18:08:27.130 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 70809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbtest.ko 2023-11-28T18:08:27.130 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 41905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-28T18:08:27.130 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 27617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-28T18:08:27.130 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/mon/ 2023-11-28T18:08:27.130 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 72953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/mon/usbmon.ko 2023-11-28T18:08:27.130 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/musb/ 2023-11-28T18:08:27.130 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 301697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/musb/musb_hdrc.ko 2023-11-28T18:08:27.130 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/ 2023-11-28T18:08:27.130 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-28T18:08:27.131 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-11-28T18:08:27.131 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-isp1301.ko 2023-11-28T18:08:27.131 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-tahvo.ko 2023-11-28T18:08:27.131 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ 2023-11-28T18:08:27.131 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-28T18:08:27.131 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 33329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-28T18:08:27.131 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-28T18:08:27.131 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 38105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-28T18:08:27.131 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 70257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-28T18:08:27.131 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 31057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-28T18:08:27.131 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 58217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-28T18:08:27.131 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 49817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-28T18:08:27.132 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-28T18:08:27.132 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 44273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/f81232.ko 2023-11-28T18:08:27.132 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 175137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-28T18:08:27.132 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 53113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-28T18:08:27.132 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 137081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-28T18:08:27.132 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 124833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-28T18:08:27.132 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 67769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-28T18:08:27.132 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-28T18:08:27.132 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-28T18:08:27.132 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 58233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-28T18:08:27.132 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 107521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-28T18:08:27.133 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 35617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-28T18:08:27.133 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-28T18:08:27.133 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 33689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-28T18:08:27.133 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 32953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-28T18:08:27.133 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/metro-usb.ko 2023-11-28T18:08:27.133 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 66449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-28T18:08:27.133 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 78217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-28T18:08:27.133 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 56337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mxuport.ko 2023-11-28T18:08:27.133 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-28T18:08:27.133 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-28T18:08:27.133 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-28T18:08:27.133 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 191137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-28T18:08:27.134 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 39073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-28T18:08:27.134 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 54353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-28T18:08:27.134 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-28T18:08:27.134 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 37825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-28T18:08:27.134 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 34321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-28T18:08:27.134 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-28T18:08:27.134 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 51625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-28T18:08:27.134 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-28T18:08:27.134 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 29993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-28T18:08:27.134 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-28T18:08:27.134 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 61489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-28T18:08:27.135 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-28T18:08:27.135 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-28T18:08:27.135 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 33201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-28T18:08:27.135 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 125609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usbserial.ko 2023-11-28T18:08:27.135 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-28T18:08:27.135 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 37409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-28T18:08:27.135 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/wishbone-serial.ko 2023-11-28T18:08:27.135 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-28T18:08:27.135 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ 2023-11-28T18:08:27.135 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 52913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-28T18:08:27.135 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-28T18:08:27.135 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-28T18:08:27.136 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-28T18:08:27.136 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 33497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-28T18:08:27.136 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-28T18:08:27.136 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-28T18:08:27.136 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-28T18:08:27.136 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-28T18:08:27.136 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-28T18:08:27.136 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-28T18:08:27.136 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-28T18:08:27.136 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-28T18:08:27.136 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 35337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-28T18:08:27.136 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 179969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-28T18:08:27.137 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/ 2023-11-28T18:08:27.137 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 67481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-28T18:08:27.137 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 72177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-28T18:08:27.137 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 107001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-28T18:08:27.137 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/ 2023-11-28T18:08:27.137 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/ 2023-11-28T18:08:27.137 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 154297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-28T18:08:27.137 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 29537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-28T18:08:27.137 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 108737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-28T18:08:27.137 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 77609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-28T18:08:27.137 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/ 2023-11-28T18:08:27.137 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 113785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-28T18:08:27.138 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-28T18:08:27.138 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 58089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-28T18:08:27.138 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 89249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-28T18:08:27.138 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/ 2023-11-28T18:08:27.138 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/ 2023-11-28T18:08:27.138 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-11-28T18:08:27.138 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/aat2870_bl.ko 2023-11-28T18:08:27.138 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/adp5520_bl.ko 2023-11-28T18:08:27.138 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 32609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/adp8860_bl.ko 2023-11-28T18:08:27.138 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 39993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/adp8870_bl.ko 2023-11-28T18:08:27.138 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-28T18:08:27.139 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/as3711_bl.ko 2023-11-28T18:08:27.139 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/bd6107.ko 2023-11-28T18:08:27.139 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/da903x_bl.ko 2023-11-28T18:08:27.139 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/da9052_bl.ko 2023-11-28T18:08:27.139 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/gpio_backlight.ko 2023-11-28T18:08:27.139 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/kb3886_bl.ko 2023-11-28T18:08:27.139 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-28T18:08:27.139 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lm3533_bl.ko 2023-11-28T18:08:27.139 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lm3639_bl.ko 2023-11-28T18:08:27.139 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lv5207lp.ko 2023-11-28T18:08:27.139 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/max8925_bl.ko 2023-11-28T18:08:27.139 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/pandora_bl.ko 2023-11-28T18:08:27.140 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-11-28T18:08:27.140 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-28T18:08:27.140 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/wm831x_bl.ko 2023-11-28T18:08:27.140 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/ 2023-11-28T18:08:27.140 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ 2023-11-28T18:08:27.140 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-28T18:08:27.140 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 38985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-28T18:08:27.140 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/matrox_w1.ko 2023-11-28T18:08:27.140 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/w1-gpio.ko 2023-11-28T18:08:27.140 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/ 2023-11-28T18:08:27.140 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-28T18:08:27.141 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-28T18:08:27.141 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-28T18:08:27.141 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-28T18:08:27.141 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-28T18:08:27.141 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-28T18:08:27.141 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-28T18:08:27.141 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-28T18:08:27.141 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-28T18:08:27.141 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-28T18:08:27.141 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 56425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-28T18:08:27.141 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 102673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-28T18:08:27.141 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ 2023-11-28T18:08:27.142 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/acquirewdt.ko 2023-11-28T18:08:27.142 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/advantechwdt.ko 2023-11-28T18:08:27.142 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-28T18:08:27.142 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-28T18:08:27.142 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/cpu5wdt.ko 2023-11-28T18:08:27.142 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/da9052_wdt.ko 2023-11-28T18:08:27.142 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/da9055_wdt.ko 2023-11-28T18:08:27.142 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/da9063_wdt.ko 2023-11-28T18:08:27.142 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/dw_wdt.ko 2023-11-28T18:08:27.142 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/eurotechwdt.ko 2023-11-28T18:08:27.142 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-28T18:08:27.142 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 27401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-28T18:08:27.143 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-28T18:08:27.143 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-28T18:08:27.143 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-28T18:08:27.143 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-28T18:08:27.143 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-28T18:08:27.143 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-28T18:08:27.143 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-28T18:08:27.143 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-28T18:08:27.143 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/kempld_wdt.ko 2023-11-28T18:08:27.143 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-28T18:08:27.143 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/mena21_wdt.ko 2023-11-28T18:08:27.144 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-11-28T18:08:27.144 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-28T18:08:27.144 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-11-28T18:08:27.144 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pc87413_wdt.ko 2023-11-28T18:08:27.144 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 38657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-28T18:08:27.144 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 31825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-28T18:08:27.144 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/retu_wdt.ko 2023-11-28T18:08:27.144 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-11-28T18:08:27.144 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-11-28T18:08:27.144 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-28T18:08:27.144 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sc1200wdt.ko 2023-11-28T18:08:27.144 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-28T18:08:27.145 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-11-28T18:08:27.145 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-28T18:08:27.145 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-28T18:08:27.145 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/twl4030_wdt.ko 2023-11-28T18:08:27.145 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-28T18:08:27.145 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-28T18:08:27.145 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-28T18:08:27.145 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-28T18:08:27.145 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wafer5823wdt.ko 2023-11-28T18:08:27.145 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-28T18:08:27.145 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wm831x_wdt.ko 2023-11-28T18:08:27.145 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wm8350_wdt.ko 2023-11-28T18:08:27.146 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-28T18:08:27.146 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/ 2023-11-28T18:08:27.146 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-28T18:08:27.146 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-28T18:08:27.146 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 47865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-28T18:08:27.146 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/ 2023-11-28T18:08:27.146 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 176377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-28T18:08:27.146 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 40121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-28T18:08:27.146 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 71249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-scsiback.ko 2023-11-28T18:08:27.146 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/ 2023-11-28T18:08:27.146 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-28T18:08:27.146 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ 2023-11-28T18:08:27.147 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/ 2023-11-28T18:08:27.147 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 166425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-28T18:08:27.147 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/adfs/ 2023-11-28T18:08:27.147 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 61073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/adfs/adfs.ko 2023-11-28T18:08:27.147 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/ 2023-11-28T18:08:27.147 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 163593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-28T18:08:27.147 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/afs/ 2023-11-28T18:08:27.147 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 932817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/afs/kafs.ko 2023-11-28T18:08:27.147 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/autofs/ 2023-11-28T18:08:27.147 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 101057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/autofs/autofs4.ko 2023-11-28T18:08:27.147 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/ 2023-11-28T18:08:27.147 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 64545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-28T18:08:27.148 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/bfs/ 2023-11-28T18:08:27.148 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 41257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/bfs/bfs.ko 2023-11-28T18:08:27.148 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 34585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-28T18:08:27.148 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/ 2023-11-28T18:08:27.148 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 3402729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-28T18:08:27.148 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/ 2023-11-28T18:08:27.148 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 306009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-28T18:08:27.148 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ 2023-11-28T18:08:27.148 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 1333017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-28T18:08:27.148 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/ 2023-11-28T18:08:27.148 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 100801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-28T18:08:27.148 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/configfs/ 2023-11-28T18:08:27.149 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 108393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/configfs/configfs.ko 2023-11-28T18:08:27.149 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/ 2023-11-28T18:08:27.149 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 29625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-28T18:08:27.149 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/ 2023-11-28T18:08:27.149 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 642513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-28T18:08:27.149 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/efs/ 2023-11-28T18:08:27.149 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 36649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/efs/efs.ko 2023-11-28T18:08:27.149 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/ 2023-11-28T18:08:27.149 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 1505025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-28T18:08:27.149 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/ 2023-11-28T18:08:27.149 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 27929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-28T18:08:27.149 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/freevxfs/ 2023-11-28T18:08:27.150 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 42545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/freevxfs/freevxfs.ko 2023-11-28T18:08:27.150 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/ 2023-11-28T18:08:27.150 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 226481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-28T18:08:27.150 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/ 2023-11-28T18:08:27.150 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-28T18:08:27.150 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/ 2023-11-28T18:08:27.150 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 1231529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-28T18:08:27.150 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/ 2023-11-28T18:08:27.150 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 141209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-28T18:08:27.150 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/ 2023-11-28T18:08:27.150 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 222817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-28T18:08:27.151 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hpfs/ 2023-11-28T18:08:27.151 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 192329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hpfs/hpfs.ko 2023-11-28T18:08:27.151 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/ 2023-11-28T18:08:27.151 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 74065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-28T18:08:27.151 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/ 2023-11-28T18:08:27.151 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 419497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-28T18:08:27.151 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/ 2023-11-28T18:08:27.151 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 280313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-28T18:08:27.151 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/ 2023-11-28T18:08:27.151 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 89193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-28T18:08:27.151 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/ 2023-11-28T18:08:27.151 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 124649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-28T18:08:27.152 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/ 2023-11-28T18:08:27.152 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/ 2023-11-28T18:08:27.152 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 81449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-28T18:08:27.152 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/ 2023-11-28T18:08:27.152 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 59449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-28T18:08:27.152 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/ 2023-11-28T18:08:27.152 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 117465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-28T18:08:27.152 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 1055721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-28T18:08:27.152 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 66441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv2.ko 2023-11-28T18:08:27.152 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 120177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-28T18:08:27.152 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 2042137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-28T18:08:27.152 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/ 2023-11-28T18:08:27.153 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-28T18:08:27.153 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-28T18:08:27.153 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/ 2023-11-28T18:08:27.153 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 1516873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-28T18:08:27.153 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/ 2023-11-28T18:08:27.153 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 592833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-28T18:08:27.153 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/ 2023-11-28T18:08:27.153 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-28T18:08:27.153 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-28T18:08:27.153 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-28T18:08:27.153 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-28T18:08:27.153 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-28T18:08:27.154 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-28T18:08:27.154 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-28T18:08:27.154 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-28T18:08:27.154 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-28T18:08:27.154 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-28T18:08:27.154 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-28T18:08:27.154 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ascii.ko 2023-11-28T18:08:27.154 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-28T18:08:27.154 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-28T18:08:27.154 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-28T18:08:27.154 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-28T18:08:27.154 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-28T18:08:27.155 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-28T18:08:27.155 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-28T18:08:27.155 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-28T18:08:27.155 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-28T18:08:27.155 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-28T18:08:27.155 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-28T18:08:27.155 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-28T18:08:27.155 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-28T18:08:27.155 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-28T18:08:27.155 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-28T18:08:27.155 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-28T18:08:27.155 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-28T18:08:27.156 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-28T18:08:27.156 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 94137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-28T18:08:27.156 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 142673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-28T18:08:27.156 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 165745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-28T18:08:27.156 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 118121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-28T18:08:27.156 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-28T18:08:27.156 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-28T18:08:27.156 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-28T18:08:27.156 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-28T18:08:27.156 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-28T18:08:27.156 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-28T18:08:27.156 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-28T18:08:27.157 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-28T18:08:27.157 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-28T18:08:27.157 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-28T18:08:27.157 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-28T18:08:27.157 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-28T18:08:27.157 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-28T18:08:27.157 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-28T18:08:27.157 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-28T18:08:27.157 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-28T18:08:27.157 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-28T18:08:27.157 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ntfs/ 2023-11-28T18:08:27.157 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 219417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ntfs/ntfs.ko 2023-11-28T18:08:27.158 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ 2023-11-28T18:08:27.158 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ 2023-11-28T18:08:27.158 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 244409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-28T18:08:27.158 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ 2023-11-28T18:08:27.158 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 440737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-28T18:08:27.158 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ 2023-11-28T18:08:27.158 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 54049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-28T18:08:27.158 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 2592433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-28T18:08:27.158 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-28T18:08:27.158 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 31729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-28T18:08:27.158 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 35897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-28T18:08:27.158 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/omfs/ 2023-11-28T18:08:27.159 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 53433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/omfs/omfs.ko 2023-11-28T18:08:27.159 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/ 2023-11-28T18:08:27.159 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 352161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-28T18:08:27.159 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ 2023-11-28T18:08:27.159 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 54809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-28T18:08:27.159 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx4/ 2023-11-28T18:08:27.159 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx4/qnx4.ko 2023-11-28T18:08:27.159 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx6/ 2023-11-28T18:08:27.159 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 46689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx6/qnx6.ko 2023-11-28T18:08:27.159 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/ 2023-11-28T18:08:27.159 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/quota_tree.ko 2023-11-28T18:08:27.160 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/quota_v1.ko 2023-11-28T18:08:27.160 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/quota_v2.ko 2023-11-28T18:08:27.160 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/ 2023-11-28T18:08:27.160 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 532673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-28T18:08:27.160 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/ 2023-11-28T18:08:27.160 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-28T18:08:27.160 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/ 2023-11-28T18:08:27.160 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/ 2023-11-28T18:08:27.160 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 2668401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-28T18:08:27.160 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/ 2023-11-28T18:08:27.160 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-28T18:08:27.160 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-28T18:08:27.160 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/ 2023-11-28T18:08:27.161 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 134641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-28T18:08:27.161 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/ 2023-11-28T18:08:27.161 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 90721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-28T18:08:27.161 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/ 2023-11-28T18:08:27.161 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 247737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-28T18:08:27.161 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ 2023-11-28T18:08:27.161 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 162497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-28T18:08:27.161 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/ 2023-11-28T18:08:27.161 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 4011017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-28T18:08:27.161 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/ 2023-11-28T18:08:27.161 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/locking/ 2023-11-28T18:08:27.162 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 75489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/locking/locktorture.ko 2023-11-28T18:08:27.162 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/time/ 2023-11-28T18:08:27.162 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/time/test_udelay.ko 2023-11-28T18:08:27.162 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 60553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-28T18:08:27.162 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ 2023-11-28T18:08:27.162 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 6833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-28T18:08:27.162 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 6529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc7.ko 2023-11-28T18:08:27.162 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-28T18:08:27.162 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/ 2023-11-28T18:08:27.162 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-28T18:08:27.162 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 37081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-28T18:08:27.162 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/interval_tree_test.ko 2023-11-28T18:08:27.163 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-28T18:08:27.163 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-28T18:08:27.163 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/ 2023-11-28T18:08:27.163 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 33801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-28T18:08:27.163 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-28T18:08:27.163 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/ 2023-11-28T18:08:27.163 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 6617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-28T18:08:27.163 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/memory-notifier-error-inject.ko 2023-11-28T18:08:27.163 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/notifier-error-inject.ko 2023-11-28T18:08:27.163 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/percpu_test.ko 2023-11-28T18:08:27.163 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/pm-notifier-error-inject.ko 2023-11-28T18:08:27.164 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/ 2023-11-28T18:08:27.164 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 143433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-28T18:08:27.164 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/rbtree_test.ko 2023-11-28T18:08:27.164 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/ 2023-11-28T18:08:27.164 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-28T18:08:27.164 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 38841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test-kstrtox.ko 2023-11-28T18:08:27.164 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test-string_helpers.ko 2023-11-28T18:08:27.164 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 4760425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_bpf.ko 2023-11-28T18:08:27.164 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 49089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_firmware.ko 2023-11-28T18:08:27.164 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 6369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_module.ko 2023-11-28T18:08:27.164 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_user_copy.ko 2023-11-28T18:08:27.164 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-28T18:08:27.165 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-28T18:08:27.165 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-28T18:08:27.165 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/xz/ 2023-11-28T18:08:27.165 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/xz/xz_dec_test.ko 2023-11-28T18:08:27.165 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/mm/ 2023-11-28T18:08:27.165 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-28T18:08:27.165 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ 2023-11-28T18:08:27.165 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/ 2023-11-28T18:08:27.165 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 55737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-28T18:08:27.165 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-28T18:08:27.165 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-28T18:08:27.165 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-28T18:08:27.166 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 6985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-28T18:08:27.166 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-28T18:08:27.166 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-28T18:08:27.166 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-28T18:08:27.166 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/ 2023-11-28T18:08:27.166 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-28T18:08:27.166 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 34809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-28T18:08:27.166 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-28T18:08:27.166 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-28T18:08:27.166 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-28T18:08:27.166 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/ 2023-11-28T18:08:27.166 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 81737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-28T18:08:27.167 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/ 2023-11-28T18:08:27.167 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 198337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-28T18:08:27.167 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 35977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-28T18:08:27.167 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-28T18:08:27.167 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 32337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-28T18:08:27.167 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/ 2023-11-28T18:08:27.167 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 649297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-28T18:08:27.167 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/ 2023-11-28T18:08:27.167 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 63809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-28T18:08:27.167 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 777689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-28T18:08:27.167 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ 2023-11-28T18:08:27.167 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-28T18:08:27.168 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-28T18:08:27.168 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-28T18:08:27.168 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-28T18:08:27.168 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-28T18:08:27.168 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-28T18:08:27.168 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-28T18:08:27.168 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-28T18:08:27.168 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-28T18:08:27.168 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-28T18:08:27.168 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-28T18:08:27.168 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-28T18:08:27.169 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-28T18:08:27.169 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-28T18:08:27.169 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-28T18:08:27.169 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-28T18:08:27.169 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-28T18:08:27.169 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-28T18:08:27.169 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-28T18:08:27.169 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-28T18:08:27.169 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 55929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-28T18:08:27.169 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-28T18:08:27.169 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-28T18:08:27.169 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/ 2023-11-28T18:08:27.170 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 1046833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-28T18:08:27.170 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/ 2023-11-28T18:08:27.170 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 125993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-28T18:08:27.170 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/ 2023-11-28T18:08:27.170 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 194209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-28T18:08:27.170 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-28T18:08:27.170 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 57881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-28T18:08:27.170 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 60337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-28T18:08:27.170 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/ 2023-11-28T18:08:27.170 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 390473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-28T18:08:27.170 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-28T18:08:27.170 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-28T18:08:27.171 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-28T18:08:27.171 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-28T18:08:27.171 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/hsr/ 2023-11-28T18:08:27.171 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 99729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/hsr/hsr.ko 2023-11-28T18:08:27.171 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ 2023-11-28T18:08:27.171 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ 2023-11-28T18:08:27.171 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 45233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-28T18:08:27.171 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 254625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-28T18:08:27.171 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 47289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-28T18:08:27.171 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ 2023-11-28T18:08:27.171 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-28T18:08:27.171 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 38905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-28T18:08:27.172 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 48337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-28T18:08:27.172 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-28T18:08:27.172 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 48657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-28T18:08:27.172 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 60633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-28T18:08:27.172 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 61225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-28T18:08:27.172 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 34505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-28T18:08:27.172 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-28T18:08:27.172 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-28T18:08:27.172 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ 2023-11-28T18:08:27.172 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 37641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-28T18:08:27.172 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-28T18:08:27.173 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-28T18:08:27.173 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 41609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ip_tables.ko 2023-11-28T18:08:27.173 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-28T18:08:27.173 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-11-28T18:08:27.173 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-28T18:08:27.173 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-28T18:08:27.173 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-28T18:08:27.173 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-11-28T18:08:27.173 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-28T18:08:27.173 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-28T18:08:27.173 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-28T18:08:27.173 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-28T18:08:27.174 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-28T18:08:27.174 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-28T18:08:27.174 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 27625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-28T18:08:27.174 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-28T18:08:27.174 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-28T18:08:27.174 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-11-28T18:08:27.174 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-28T18:08:27.174 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-28T18:08:27.174 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-28T18:08:27.174 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-28T18:08:27.174 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-28T18:08:27.174 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-28T18:08:27.175 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-28T18:08:27.175 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-28T18:08:27.175 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-28T18:08:27.175 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-28T18:08:27.175 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-28T18:08:27.175 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-28T18:08:27.175 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-28T18:08:27.175 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-28T18:08:27.175 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-28T18:08:27.175 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-28T18:08:27.175 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-28T18:08:27.175 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-28T18:08:27.176 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-28T18:08:27.176 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-28T18:08:27.176 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 41665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-28T18:08:27.176 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-28T18:08:27.176 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ 2023-11-28T18:08:27.176 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-28T18:08:27.176 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 39537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-28T18:08:27.176 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-28T18:08:27.176 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 80897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_gre.ko 2023-11-28T18:08:27.176 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 93577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-28T18:08:27.176 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-28T18:08:27.176 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 53273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-28T18:08:27.177 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-28T18:08:27.177 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/mip6.ko 2023-11-28T18:08:27.177 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ 2023-11-28T18:08:27.177 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 42169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-28T18:08:27.177 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-11-28T18:08:27.177 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-28T18:08:27.177 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-28T18:08:27.177 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-28T18:08:27.177 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-28T18:08:27.177 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-28T18:08:27.177 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-28T18:08:27.177 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-28T18:08:27.178 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-28T18:08:27.178 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-28T18:08:27.178 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-28T18:08:27.178 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-28T18:08:27.178 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-28T18:08:27.178 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-28T18:08:27.178 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-28T18:08:27.178 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-28T18:08:27.178 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 39113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-28T18:08:27.178 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-28T18:08:27.178 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-28T18:08:27.179 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-28T18:08:27.179 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-28T18:08:27.179 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-28T18:08:27.179 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-28T18:08:27.179 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-28T18:08:27.179 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 74073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-28T18:08:27.179 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-28T18:08:27.179 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-28T18:08:27.179 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/ 2023-11-28T18:08:27.179 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 82833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-28T18:08:27.179 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/ 2023-11-28T18:08:27.179 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 125281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-28T18:08:27.180 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-28T18:08:27.180 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 25953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-28T18:08:27.180 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 35569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-28T18:08:27.180 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 36713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-28T18:08:27.180 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 33001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-28T18:08:27.180 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 53241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-28T18:08:27.180 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/lapb/ 2023-11-28T18:08:27.180 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 47977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/lapb/lapb.ko 2023-11-28T18:08:27.180 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/ 2023-11-28T18:08:27.180 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-28T18:08:27.180 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 267913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc2.ko 2023-11-28T18:08:27.180 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/ 2023-11-28T18:08:27.181 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 290417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-28T18:08:27.181 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/ 2023-11-28T18:08:27.181 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-28T18:08:27.181 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ 2023-11-28T18:08:27.181 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ 2023-11-28T18:08:27.181 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 119585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-28T18:08:27.181 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-28T18:08:27.182 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-28T18:08:27.182 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-28T18:08:27.182 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 100297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-28T18:08:27.182 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 101441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-28T18:08:27.182 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 102817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-28T18:08:27.183 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 103417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-28T18:08:27.183 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 112849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-28T18:08:27.183 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 60329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-28T18:08:27.183 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 109593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-28T18:08:27.183 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 113129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-28T18:08:27.183 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 112585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-28T18:08:27.183 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 111585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-28T18:08:27.183 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 114465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-28T18:08:27.183 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-28T18:08:27.183 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ 2023-11-28T18:08:27.183 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 411729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-28T18:08:27.184 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-28T18:08:27.184 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-28T18:08:27.184 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-28T18:08:27.184 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-28T18:08:27.184 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-28T18:08:27.184 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-28T18:08:27.184 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-28T18:08:27.184 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-28T18:08:27.184 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-28T18:08:27.184 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-28T18:08:27.184 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-28T18:08:27.184 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-28T18:08:27.185 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-28T18:08:27.185 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 49257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-28T18:08:27.185 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 408257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-28T18:08:27.185 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-28T18:08:27.185 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-28T18:08:27.185 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 33761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-28T18:08:27.185 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 118145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-28T18:08:27.185 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-28T18:08:27.185 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-28T18:08:27.185 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 107233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-28T18:08:27.185 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 33633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-28T18:08:27.185 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-28T18:08:27.186 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 66641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-28T18:08:27.186 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-28T18:08:27.186 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-28T18:08:27.186 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-28T18:08:27.186 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 89425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-28T18:08:27.186 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-28T18:08:27.186 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 37129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-28T18:08:27.186 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 95353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-28T18:08:27.186 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-28T18:08:27.186 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-28T18:08:27.186 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-28T18:08:27.187 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-28T18:08:27.187 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-28T18:08:27.187 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 44065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-28T18:08:27.187 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 615649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-28T18:08:27.187 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 42561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-28T18:08:27.187 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-28T18:08:27.187 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-11-28T18:08:27.187 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 39785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-28T18:08:27.187 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-28T18:08:27.187 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 49377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-28T18:08:27.187 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-28T18:08:27.187 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 33209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-28T18:08:27.188 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 40721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-28T18:08:27.188 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-28T18:08:27.188 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-28T18:08:27.188 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-28T18:08:27.188 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-28T18:08:27.188 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-28T18:08:27.188 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-28T18:08:27.188 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-28T18:08:27.188 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-28T18:08:27.188 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-28T18:08:27.188 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_osf.ko 2023-11-28T18:08:27.189 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-28T18:08:27.189 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-28T18:08:27.189 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-28T18:08:27.189 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-28T18:08:27.189 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-28T18:08:27.189 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_synproxy.ko 2023-11-28T18:08:27.189 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tproxy.ko 2023-11-28T18:08:27.189 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 89313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/x_tables.ko 2023-11-28T18:08:27.189 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-28T18:08:27.189 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-28T18:08:27.189 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-28T18:08:27.189 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-28T18:08:27.190 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 18169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-28T18:08:27.190 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-28T18:08:27.190 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-28T18:08:27.190 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-28T18:08:27.190 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 38873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-28T18:08:27.190 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-28T18:08:27.190 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-28T18:08:27.190 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-28T18:08:27.190 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-28T18:08:27.190 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-28T18:08:27.190 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-28T18:08:27.190 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-28T18:08:27.191 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-28T18:08:27.191 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 14337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-28T18:08:27.191 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-28T18:08:27.191 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-28T18:08:27.191 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-28T18:08:27.191 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-28T18:08:27.191 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-28T18:08:27.191 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-28T18:08:27.191 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-28T18:08:27.191 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-28T18:08:27.191 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-28T18:08:27.192 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-28T18:08:27.192 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-28T18:08:27.192 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-28T18:08:27.192 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-28T18:08:27.192 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-28T18:08:27.192 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-28T18:08:27.192 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-28T18:08:27.192 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-28T18:08:27.192 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-28T18:08:27.192 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-28T18:08:27.192 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-28T18:08:27.192 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-28T18:08:27.193 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 43289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-28T18:08:27.193 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-28T18:08:27.193 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-28T18:08:27.193 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-28T18:08:27.193 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-28T18:08:27.193 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-28T18:08:27.193 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-28T18:08:27.193 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-28T18:08:27.193 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-28T18:08:27.193 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-28T18:08:27.193 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-28T18:08:27.193 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-28T18:08:27.194 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-28T18:08:27.194 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-28T18:08:27.194 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-28T18:08:27.194 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-28T18:08:27.194 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-28T18:08:27.194 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-28T18:08:27.194 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-28T18:08:27.194 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-28T18:08:27.194 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-28T18:08:27.194 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-28T18:08:27.194 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 36433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-28T18:08:27.195 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-28T18:08:27.195 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-28T18:08:27.195 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 16089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-28T18:08:27.195 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-28T18:08:27.195 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-28T18:08:27.195 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-28T18:08:27.195 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 8553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-28T18:08:27.195 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpudp.ko 2023-11-28T18:08:27.195 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-28T18:08:27.195 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-28T18:08:27.195 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/ 2023-11-28T18:08:27.195 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-28T18:08:27.196 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/ 2023-11-28T18:08:27.196 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 12425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-28T18:08:27.196 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/ 2023-11-28T18:08:27.196 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 380033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-28T18:08:27.196 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-28T18:08:27.196 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-28T18:08:27.196 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 11793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-28T18:08:27.196 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/ 2023-11-28T18:08:27.196 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-28T18:08:27.196 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/phonet/ 2023-11-28T18:08:27.196 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 110201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/phonet/phonet.ko 2023-11-28T18:08:27.196 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 54721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/phonet/pn_pep.ko 2023-11-28T18:08:27.197 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/ 2023-11-28T18:08:27.197 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 251481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-28T18:08:27.197 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 164937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-28T18:08:27.197 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 67377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-28T18:08:27.197 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rxrpc/ 2023-11-28T18:08:27.197 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 819657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rxrpc/rxrpc.ko 2023-11-28T18:08:27.197 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/ 2023-11-28T18:08:27.197 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 32241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-28T18:08:27.197 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-28T18:08:27.197 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-28T18:08:27.197 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 34097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-28T18:08:27.197 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-28T18:08:27.198 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 43449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-28T18:08:27.198 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 31961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-28T18:08:27.198 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-28T18:08:27.198 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 35297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-28T18:08:27.198 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 36665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-28T18:08:27.198 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-28T18:08:27.198 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 33961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-28T18:08:27.198 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_cgroup.ko 2023-11-28T18:08:27.198 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 31401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-28T18:08:27.198 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 31769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-28T18:08:27.198 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 38425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-28T18:08:27.198 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 61409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-28T18:08:27.199 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-28T18:08:27.199 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 13409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-28T18:08:27.199 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 33289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-28T18:08:27.199 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-28T18:08:27.199 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-28T18:08:27.199 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-28T18:08:27.199 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 21649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-28T18:08:27.199 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-28T18:08:27.199 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 34633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-28T18:08:27.199 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 33609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-28T18:08:27.199 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 31617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq_codel.ko 2023-11-28T18:08:27.199 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 35433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-28T18:08:27.200 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 49817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-28T18:08:27.200 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-28T18:08:27.200 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 66393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-28T18:08:27.200 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-28T18:08:27.200 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 30273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-28T18:08:27.200 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-28T18:08:27.200 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-28T18:08:27.200 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 36137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-28T18:08:27.200 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 27497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-28T18:08:27.200 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-28T18:08:27.200 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 24729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-28T18:08:27.201 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 48729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-28T18:08:27.201 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 32905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-28T18:08:27.201 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 26881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-28T18:08:27.201 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 31353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-28T18:08:27.201 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-28T18:08:27.201 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 28105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-28T18:08:27.201 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/ 2023-11-28T18:08:27.201 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 929745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-28T18:08:27.201 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 19033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-28T18:08:27.201 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/ 2023-11-28T18:08:27.201 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/ 2023-11-28T18:08:27.201 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 336529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-28T18:08:27.202 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 54737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-28T18:08:27.202 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 1520097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-28T18:08:27.202 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/ 2023-11-28T18:08:27.202 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 707657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-28T18:08:27.202 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/ 2023-11-28T18:08:27.202 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-28T18:08:27.202 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 862665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-28T18:08:27.202 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/ 2023-11-28T18:08:27.202 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-28T18:08:27.203 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/ 2023-11-28T18:08:27.203 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 85409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-28T18:08:27.203 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 68049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-28T18:08:27.203 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 96193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-28T18:08:27.203 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 10705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-28T18:08:27.203 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 17521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-28T18:08:27.203 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/x25/ 2023-11-28T18:08:27.203 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 125177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/x25/x25.ko 2023-11-28T18:08:27.203 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/ 2023-11-28T18:08:27.203 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 22657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_algo.ko 2023-11-28T18:08:27.203 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 20649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-28T18:08:27.204 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 102257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_user.ko 2023-11-28T18:08:27.204 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/virt/ 2023-11-28T18:08:27.204 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/ 2023-11-28T18:08:27.204 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 15537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-28T18:08:27.204 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 640081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-28T18:08:27.204 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 624107 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-28T18:08:27.204 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 7663 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-28T18:08:27.204 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 23428 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin.alias.bin 2023-11-28T18:08:27.204 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 9821 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-28T18:08:27.204 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 70004 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-28T18:08:27.204 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 146313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-28T18:08:27.204 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 232548 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-28T18:08:27.205 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 210 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-28T18:08:27.205 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 72606 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-28T18:08:27.205 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 879 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-28T18:08:27.205 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 254438 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-28T18:08:27.205 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 309363 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-28T18:08:27.205 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/ 2023-11-28T18:08:27.205 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/share/ 2023-11-28T18:08:27.205 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/share/doc/ 2023-11-28T18:08:27.205 INFO:teuthology.orchestra.run.smithi148.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/share/doc/linux-image-6.6.0-g8247bbb5b470/ 2023-11-28T18:08:27.205 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 172 2023-11-23 03:47 ./usr/share/doc/linux-image-6.6.0-g8247bbb5b470/changelog.Debian.gz 2023-11-28T18:08:27.205 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r-- root/root 691 2023-11-23 03:47 ./usr/share/doc/linux-image-6.6.0-g8247bbb5b470/copyright 2023-11-28T18:08:27.206 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-28T18:08:27.206 INFO:teuthology.task.kernel:searching for kernel 6.6.0-g8247bbb5b470 2023-11-28T18:08:27.206 DEBUG:teuthology.orchestra.run.smithi148:> egrep '(submenu|menuentry.*6.6.0-g8247bbb5b470).*{' /boot/grub/grub.cfg 2023-11-28T18:08:27.213 INFO:teuthology.orchestra.run.smithi148.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-11-28T18:08:27.214 INFO:teuthology.orchestra.run.smithi148.stdout: menuentry 'Ubuntu, with Linux 6.6.0-g8247bbb5b470' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-g8247bbb5b470-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-11-28T18:08:27.214 INFO:teuthology.orchestra.run.smithi148.stdout: menuentry 'Ubuntu, with Linux 6.6.0-g8247bbb5b470 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-g8247bbb5b470-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-11-28T18:08:27.214 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-11-28T18:08:27.214 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-g8247bbb5b470 2023-11-28T18:08:27.215 DEBUG:teuthology.orchestra.run.smithi148:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-11-28T18:08:27.258 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi148.front.sepia.ceph.com to complete... 2023-11-28T18:08:27.672 INFO:teuthology.orchestra.run.smithi148.stderr:Sourcing file `/etc/default/grub' 2023-11-28T18:08:27.673 INFO:teuthology.orchestra.run.smithi148.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-28T18:08:27.701 INFO:teuthology.orchestra.run.smithi148.stderr:Generating grub configuration file ... 2023-11-28T18:08:27.883 INFO:teuthology.orchestra.run.smithi148.stderr:Found linux image: /boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-28T18:08:27.889 INFO:teuthology.orchestra.run.smithi148.stderr:Found initrd image: /boot/initrd.img-6.6.0-g8247bbb5b470 2023-11-28T18:08:28.118 INFO:teuthology.orchestra.run.smithi148.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-11-28T18:08:28.123 INFO:teuthology.orchestra.run.smithi148.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-11-28T18:08:28.161 INFO:teuthology.orchestra.run.smithi148.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-11-28T18:08:28.166 INFO:teuthology.orchestra.run.smithi148.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-11-28T18:08:28.314 INFO:teuthology.orchestra.run.smithi148.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-11-28T18:08:28.314 INFO:teuthology.orchestra.run.smithi148.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-11-28T18:08:28.315 INFO:teuthology.orchestra.run.smithi148.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-11-28T18:08:28.343 INFO:teuthology.orchestra.run.smithi148.stderr:done 2023-11-28T18:08:33.100 INFO:teuthology.orchestra.run.smithi105.stderr:Sourcing file `/etc/default/grub' 2023-11-28T18:08:33.117 INFO:teuthology.orchestra.run.smithi105.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-28T18:08:33.147 INFO:teuthology.orchestra.run.smithi105.stderr:Generating grub configuration file ... 2023-11-28T18:08:33.448 INFO:teuthology.orchestra.run.smithi105.stderr:Found linux image: /boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-28T18:08:33.491 INFO:teuthology.orchestra.run.smithi105.stderr:Found initrd image: /boot/initrd.img-6.6.0-g8247bbb5b470 2023-11-28T18:08:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-11-28T18:08:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-11-28T18:08:33.825 INFO:teuthology.orchestra.run.smithi105.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-11-28T18:08:33.830 INFO:teuthology.orchestra.run.smithi105.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-11-28T18:08:33.972 INFO:teuthology.orchestra.run.smithi105.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-11-28T18:08:33.972 INFO:teuthology.orchestra.run.smithi105.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-11-28T18:08:33.973 INFO:teuthology.orchestra.run.smithi105.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-11-28T18:08:34.017 INFO:teuthology.orchestra.run.smithi105.stderr:done 2023-11-28T18:08:34.205 DEBUG:teuthology.orchestra.run.smithi105:> dpkg-deb -c /tmp/linux-image.deb 2023-11-28T18:08:34.214 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./ 2023-11-28T18:08:34.214 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./boot/ 2023-11-28T18:08:34.214 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 4415617 2023-11-23 03:47 ./boot/System.map-6.6.0-g8247bbb5b470 2023-11-28T18:08:34.224 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 169214 2023-11-23 03:47 ./boot/config-6.6.0-g8247bbb5b470 2023-11-28T18:08:34.224 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11352832 2023-11-23 03:47 ./boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-28T18:08:34.244 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/ 2023-11-28T18:08:34.244 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/ 2023-11-28T18:08:34.245 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/postinst.d/ 2023-11-28T18:08:34.245 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/postrm.d/ 2023-11-28T18:08:34.245 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/preinst.d/ 2023-11-28T18:08:34.245 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/prerm.d/ 2023-11-28T18:08:34.245 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/ 2023-11-28T18:08:34.245 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/ 2023-11-28T18:08:34.245 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/ 2023-11-28T18:08:34.245 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/ 2023-11-28T18:08:34.245 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/ 2023-11-28T18:08:34.245 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/ 2023-11-28T18:08:34.245 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ 2023-11-28T18:08:34.245 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 490233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/aesni-intel.ko 2023-11-28T18:08:34.246 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-28T18:08:34.246 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 29529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-28T18:08:34.246 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-28T18:08:34.246 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 93113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-28T18:08:34.246 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 106729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-28T18:08:34.246 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 152241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-28T18:08:34.246 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-28T18:08:34.246 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-28T18:08:34.246 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 83889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-28T18:08:34.246 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-28T18:08:34.246 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 47225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-28T18:08:34.247 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 47209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-28T18:08:34.247 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 51801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-28T18:08:34.247 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 38937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-28T18:08:34.247 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 40529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-28T18:08:34.247 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 58585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-28T18:08:34.247 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 50497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-28T18:08:34.247 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 32065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-28T18:08:34.247 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-28T18:08:34.247 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/ 2023-11-28T18:08:34.248 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/ 2023-11-28T18:08:34.248 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/ 2023-11-28T18:08:34.248 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 40113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-28T18:08:34.248 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpuid.ko 2023-11-28T18:08:34.248 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/msr.ko 2023-11-28T18:08:34.249 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/ 2023-11-28T18:08:34.249 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 354433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-28T18:08:34.249 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 881625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-28T18:08:34.428 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 2275865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-28T18:08:34.428 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ 2023-11-28T18:08:34.428 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 53697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/af_alg.ko 2023-11-28T18:08:34.428 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/algif_hash.ko 2023-11-28T18:08:34.428 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/algif_skcipher.ko 2023-11-28T18:08:34.428 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-28T18:08:34.428 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-28T18:08:34.428 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-28T18:08:34.428 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/asymmetric_keys/ 2023-11-28T18:08:34.429 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-11-28T18:08:34.429 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/ 2023-11-28T18:08:34.429 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-28T18:08:34.429 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-28T18:08:34.429 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-28T18:08:34.429 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-28T18:08:34.429 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-28T18:08:34.429 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-28T18:08:34.429 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-28T18:08:34.429 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-28T18:08:34.429 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-28T18:08:34.430 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-28T18:08:34.430 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-28T18:08:34.430 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 38809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-28T18:08:34.430 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 27321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-28T18:08:34.430 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-28T18:08:34.430 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-28T18:08:34.430 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 35097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-28T18:08:34.430 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-28T18:08:34.430 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-28T18:08:34.430 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 47505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cryptd.ko 2023-11-28T18:08:34.430 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 39505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-28T18:08:34.430 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_simd.ko 2023-11-28T18:08:34.431 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-28T18:08:34.431 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-28T18:08:34.431 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-28T18:08:34.431 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-28T18:08:34.431 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-28T18:08:34.431 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-28T18:08:34.431 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-28T18:08:34.431 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-28T18:08:34.431 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lrw.ko 2023-11-28T18:08:34.431 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-28T18:08:34.431 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-28T18:08:34.432 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-28T18:08:34.432 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-28T18:08:34.432 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-28T18:08:34.432 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-28T18:08:34.432 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-28T18:08:34.432 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 29393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-28T18:08:34.432 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 27081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-28T18:08:34.432 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 149193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-28T18:08:34.432 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-28T18:08:34.432 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-28T18:08:34.432 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-28T18:08:34.432 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-28T18:08:34.432 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 36185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-28T18:08:34.433 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-28T18:08:34.433 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-28T18:08:34.433 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xts.ko 2023-11-28T18:08:34.433 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-28T18:08:34.433 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ 2023-11-28T18:08:34.433 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ 2023-11-28T18:08:34.433 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_extlog.ko 2023-11-28T18:08:34.433 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-28T18:08:34.433 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-28T18:08:34.433 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/apei/ 2023-11-28T18:08:34.433 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 31241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/apei/einj.ko 2023-11-28T18:08:34.433 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-28T18:08:34.434 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-28T18:08:34.434 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-28T18:08:34.434 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-28T18:08:34.434 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 103025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-28T18:08:34.434 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ 2023-11-28T18:08:34.434 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-28T18:08:34.434 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 88865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci.ko 2023-11-28T18:08:34.434 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-28T18:08:34.434 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 88289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci.ko 2023-11-28T18:08:34.434 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 34985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-28T18:08:34.435 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-28T18:08:34.435 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-28T18:08:34.435 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 33505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-28T18:08:34.435 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-28T18:08:34.435 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-28T18:08:34.435 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-28T18:08:34.435 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-28T18:08:34.435 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-28T18:08:34.435 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-28T18:08:34.435 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-28T18:08:34.435 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-28T18:08:34.435 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 33977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-28T18:08:34.436 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-28T18:08:34.436 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-28T18:08:34.436 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-28T18:08:34.436 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 35273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-28T18:08:34.436 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-28T18:08:34.436 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 51481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_legacy.ko 2023-11-28T18:08:34.436 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-28T18:08:34.436 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-28T18:08:34.436 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-28T18:08:34.436 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-28T18:08:34.436 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-28T18:08:34.436 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-28T18:08:34.437 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-28T18:08:34.437 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-28T18:08:34.437 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-28T18:08:34.437 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 32585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-28T18:08:34.437 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 37793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-28T18:08:34.437 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-28T18:08:34.437 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-28T18:08:34.437 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_radisys.ko 2023-11-28T18:08:34.437 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-28T18:08:34.437 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rz1000.ko 2023-11-28T18:08:34.437 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-28T18:08:34.437 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-28T18:08:34.438 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-28T18:08:34.438 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-28T18:08:34.438 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-28T18:08:34.438 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-28T18:08:34.438 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-28T18:08:34.438 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 29121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-28T18:08:34.438 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 83345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-28T18:08:34.438 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 70393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-28T18:08:34.438 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 37377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-28T18:08:34.438 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-28T18:08:34.438 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 29273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-28T18:08:34.438 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 38337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-28T18:08:34.439 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-28T18:08:34.439 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-28T18:08:34.439 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 44793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-28T18:08:34.439 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-28T18:08:34.439 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 35065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-28T18:08:34.439 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-28T18:08:34.439 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ 2023-11-28T18:08:34.439 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-28T18:08:34.439 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/ 2023-11-28T18:08:34.439 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 124473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-28T18:08:34.439 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/ 2023-11-28T18:08:34.439 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/ 2023-11-28T18:08:34.440 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 113665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-28T18:08:34.440 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/ 2023-11-28T18:08:34.440 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 965585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-28T18:08:34.440 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 173225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-28T18:08:34.440 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/ 2023-11-28T18:08:34.440 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 94297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-28T18:08:34.440 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 115793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-28T18:08:34.440 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/ 2023-11-28T18:08:34.440 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 110385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-28T18:08:34.440 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 98193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-28T18:08:34.440 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 263769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-28T18:08:34.441 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/ 2023-11-28T18:08:34.441 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 98145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-28T18:08:34.441 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/ 2023-11-28T18:08:34.441 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 60217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-28T18:08:34.441 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ 2023-11-28T18:08:34.441 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/applicom.ko 2023-11-28T18:08:34.441 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-28T18:08:34.441 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/ 2023-11-28T18:08:34.441 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-28T18:08:34.441 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-28T18:08:34.441 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-28T18:08:34.441 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-28T18:08:34.442 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-28T18:08:34.442 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ 2023-11-28T18:08:34.442 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-28T18:08:34.442 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 148601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-28T18:08:34.442 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-28T18:08:34.442 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 139313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-28T18:08:34.442 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 70921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-28T18:08:34.442 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 53513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-28T18:08:34.442 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 36321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-28T18:08:34.442 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/ 2023-11-28T18:08:34.442 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 84097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-28T18:08:34.443 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/nvram.ko 2023-11-28T18:08:34.443 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 32169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-28T18:08:34.443 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 51113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-28T18:08:34.443 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/ 2023-11-28T18:08:34.443 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-28T18:08:34.443 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-11-28T18:08:34.443 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-11-28T18:08:34.443 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 32985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-11-28T18:08:34.443 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-28T18:08:34.443 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-28T18:08:34.443 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-11-28T18:08:34.443 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/ 2023-11-28T18:08:34.444 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/xillybus_class.ko 2023-11-28T18:08:34.444 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 50945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/xillybus_core.ko 2023-11-28T18:08:34.444 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-11-28T18:08:34.444 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/ 2023-11-28T18:08:34.444 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-palmas.ko 2023-11-28T18:08:34.444 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 47849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-si5351.ko 2023-11-28T18:08:34.444 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-twl6040.ko 2023-11-28T18:08:34.444 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-wm831x.ko 2023-11-28T18:08:34.444 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/ 2023-11-28T18:08:34.444 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-28T18:08:34.444 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-28T18:08:34.444 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-28T18:08:34.445 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ 2023-11-28T18:08:34.445 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ 2023-11-28T18:08:34.445 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 70801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-28T18:08:34.445 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 243025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-28T18:08:34.445 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/ 2023-11-28T18:08:34.445 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/ 2023-11-28T18:08:34.445 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/ 2023-11-28T18:08:34.445 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 438761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-28T18:08:34.445 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-11-28T18:08:34.445 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-28T18:08:34.445 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-28T18:08:34.445 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-28T18:08:34.446 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/ 2023-11-28T18:08:34.446 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 70273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-28T18:08:34.446 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/ 2023-11-28T18:08:34.446 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 29785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-28T18:08:34.446 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ 2023-11-28T18:08:34.446 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/ 2023-11-28T18:08:34.446 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-28T18:08:34.446 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ 2023-11-28T18:08:34.446 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 122569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-28T18:08:34.446 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ 2023-11-28T18:08:34.446 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 79953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-28T18:08:34.447 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 35769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-28T18:08:34.447 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 63617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-28T18:08:34.447 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-28T18:08:34.447 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-28T18:08:34.447 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-28T18:08:34.447 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-28T18:08:34.447 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-28T18:08:34.447 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 52041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-28T18:08:34.447 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-28T18:08:34.447 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-28T18:08:34.447 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 56825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-28T18:08:34.447 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-28T18:08:34.448 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/ 2023-11-28T18:08:34.448 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-gpio.ko 2023-11-28T18:08:34.448 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 27185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-max14577.ko 2023-11-28T18:08:34.448 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 29569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-max77693.ko 2023-11-28T18:08:34.448 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-max8997.ko 2023-11-28T18:08:34.448 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-palmas.ko 2023-11-28T18:08:34.448 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-rt8973a.ko 2023-11-28T18:08:34.448 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 27785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-sm5502.ko 2023-11-28T18:08:34.448 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/ 2023-11-28T18:08:34.448 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 34977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/dmi-sysfs.ko 2023-11-28T18:08:34.448 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/efi/ 2023-11-28T18:08:34.448 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/efi/efi-pstore.ko 2023-11-28T18:08:34.449 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-28T18:08:34.449 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/ 2023-11-28T18:08:34.449 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-adp5520.ko 2023-11-28T18:08:34.449 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-amd8111.ko 2023-11-28T18:08:34.449 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-arizona.ko 2023-11-28T18:08:34.449 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-da9052.ko 2023-11-28T18:08:34.449 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-da9055.ko 2023-11-28T18:08:34.449 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-dln2.ko 2023-11-28T18:08:34.449 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 77849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-f7188x.ko 2023-11-28T18:08:34.449 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-generic.ko 2023-11-28T18:08:34.449 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-28T18:08:34.449 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-11-28T18:08:34.450 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-kempld.ko 2023-11-28T18:08:34.450 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-lp3943.ko 2023-11-28T18:08:34.450 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-max7300.ko 2023-11-28T18:08:34.450 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-max730x.ko 2023-11-28T18:08:34.450 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-max732x.ko 2023-11-28T18:08:34.450 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-11-28T18:08:34.450 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 47073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-pca953x.ko 2023-11-28T18:08:34.450 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-pcf857x.ko 2023-11-28T18:08:34.450 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-rdc321x.ko 2023-11-28T18:08:34.450 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-sch.ko 2023-11-28T18:08:34.450 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-sch311x.ko 2023-11-28T18:08:34.451 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-tps65912.ko 2023-11-28T18:08:34.451 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-twl4030.ko 2023-11-28T18:08:34.451 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-twl6040.ko 2023-11-28T18:08:34.451 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-28T18:08:34.451 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-vx855.ko 2023-11-28T18:08:34.451 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-wm831x.ko 2023-11-28T18:08:34.451 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-wm8350.ko 2023-11-28T18:08:34.451 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-wm8994.ko 2023-11-28T18:08:34.451 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/ 2023-11-28T18:08:34.451 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-a4tech.ko 2023-11-28T18:08:34.451 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 39169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-apple.ko 2023-11-28T18:08:34.451 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-28T18:08:34.452 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-28T18:08:34.452 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-28T18:08:34.452 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-belkin.ko 2023-11-28T18:08:34.452 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-cherry.ko 2023-11-28T18:08:34.452 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-chicony.ko 2023-11-28T18:08:34.452 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 60649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-cp2112.ko 2023-11-28T18:08:34.452 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-cypress.ko 2023-11-28T18:08:34.452 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-28T18:08:34.452 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-28T18:08:34.452 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-28T18:08:34.452 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-28T18:08:34.452 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ezkey.ko 2023-11-28T18:08:34.453 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-28T18:08:34.453 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-generic.ko 2023-11-28T18:08:34.453 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-28T18:08:34.453 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-28T18:08:34.453 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-28T18:08:34.453 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-28T18:08:34.453 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-28T18:08:34.453 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-28T18:08:34.453 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-28T18:08:34.453 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kensington.ko 2023-11-28T18:08:34.453 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-28T18:08:34.454 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-28T18:08:34.454 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-28T18:08:34.454 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-28T18:08:34.454 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 42297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-28T18:08:34.454 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lg-g15.ko 2023-11-28T18:08:34.454 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 58185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-28T18:08:34.454 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 101481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-28T18:08:34.455 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 80993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech.ko 2023-11-28T18:08:34.455 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-magicmouse.ko 2023-11-28T18:08:34.455 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-microsoft.ko 2023-11-28T18:08:34.455 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-monterey.ko 2023-11-28T18:08:34.455 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 50161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-28T18:08:34.455 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 32593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ntrig.ko 2023-11-28T18:08:34.456 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-28T18:08:34.456 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-28T18:08:34.456 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-28T18:08:34.456 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 72937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-28T18:08:34.456 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-28T18:08:34.456 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-28T18:08:34.457 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-28T18:08:34.457 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 35393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-28T18:08:34.457 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-28T18:08:34.457 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-28T18:08:34.457 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 31425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-28T18:08:34.458 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 29729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-28T18:08:34.458 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 31129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-28T18:08:34.458 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-28T18:08:34.458 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 29489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-28T18:08:34.458 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-28T18:08:34.458 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 29313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-28T18:08:34.458 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 27937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-28T18:08:34.458 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-28T18:08:34.458 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-28T18:08:34.458 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-28T18:08:34.459 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-28T18:08:34.459 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 36817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-28T18:08:34.459 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-28T18:08:34.461 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 51249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-28T18:08:34.461 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-28T18:08:34.461 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-28T18:08:34.461 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-28T18:08:34.461 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-28T18:08:34.461 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-28T18:08:34.461 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-28T18:08:34.462 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-28T18:08:34.462 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-28T18:08:34.472 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 71073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-28T18:08:34.472 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-28T18:08:34.472 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 143473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-28T18:08:34.473 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-28T18:08:34.473 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-28T18:08:34.473 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-28T18:08:34.474 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 267825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid.ko 2023-11-28T18:08:34.474 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 32321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-28T18:08:34.474 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/ 2023-11-28T18:08:34.474 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 125721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/usbhid.ko 2023-11-28T18:08:34.474 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/usbkbd.ko 2023-11-28T18:08:34.475 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/usbmouse.ko 2023-11-28T18:08:34.475 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 217873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-28T18:08:34.475 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/ 2023-11-28T18:08:34.475 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 64201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-28T18:08:34.475 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 89625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-28T18:08:34.476 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 305465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-28T18:08:34.477 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ 2023-11-28T18:08:34.477 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 66521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-28T18:08:34.477 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 61393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-28T18:08:34.477 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 34745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-28T18:08:34.477 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-28T18:08:34.478 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-28T18:08:34.478 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-28T18:08:34.478 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 31729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-28T18:08:34.479 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 97977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-28T18:08:34.479 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-28T18:08:34.479 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 45553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-28T18:08:34.479 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-28T18:08:34.479 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-28T18:08:34.479 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-28T18:08:34.479 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-28T18:08:34.479 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 74433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-28T18:08:34.479 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 39585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-28T18:08:34.479 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 67929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-28T18:08:34.479 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-28T18:08:34.480 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 39073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-28T18:08:34.480 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 55201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-28T18:08:34.480 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 49913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-28T18:08:34.480 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 58617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-28T18:08:34.480 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 40825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-28T18:08:34.480 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-28T18:08:34.480 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-28T18:08:34.481 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/da9052-hwmon.ko 2023-11-28T18:08:34.481 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/da9055-hwmon.ko 2023-11-28T18:08:34.481 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 44465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-28T18:08:34.481 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 99601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-28T18:08:34.481 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-28T18:08:34.481 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-28T18:08:34.481 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 29777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-28T18:08:34.481 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 33185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc2103.ko 2023-11-28T18:08:34.481 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-28T18:08:34.481 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 66985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-28T18:08:34.482 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 94457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-28T18:08:34.482 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 41401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-28T18:08:34.482 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-28T18:08:34.482 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 55449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-28T18:08:34.482 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-28T18:08:34.482 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-28T18:08:34.482 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 46801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-28T18:08:34.482 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 45529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-28T18:08:34.482 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hih6130.ko 2023-11-28T18:08:34.482 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-28T18:08:34.482 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-28T18:08:34.483 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-28T18:08:34.483 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 34529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-28T18:08:34.483 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-28T18:08:34.483 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-28T18:08:34.483 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-28T18:08:34.483 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 130449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-28T18:08:34.483 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/jc42.ko 2023-11-28T18:08:34.483 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-28T18:08:34.483 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-28T18:08:34.483 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-28T18:08:34.483 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 48601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-28T18:08:34.483 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-28T18:08:34.484 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 36777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-28T18:08:34.484 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-28T18:08:34.484 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 34593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-28T18:08:34.484 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 31745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-28T18:08:34.484 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-28T18:08:34.484 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 72833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-28T18:08:34.484 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 46153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-28T18:08:34.484 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 37913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-28T18:08:34.484 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-28T18:08:34.484 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 108841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-28T18:08:34.484 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 33393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-28T18:08:34.484 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-28T18:08:34.485 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-28T18:08:34.485 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-28T18:08:34.485 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-28T18:08:34.485 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-28T18:08:34.485 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-28T18:08:34.485 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-28T18:08:34.485 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-28T18:08:34.485 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-28T18:08:34.485 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 33489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-28T18:08:34.485 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-28T18:08:34.485 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-28T18:08:34.485 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-28T18:08:34.486 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 27129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-28T18:08:34.486 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-28T18:08:34.486 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 35777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-28T18:08:34.486 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mc13783-adc.ko 2023-11-28T18:08:34.486 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-28T18:08:34.486 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-11-28T18:08:34.486 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 44129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-28T18:08:34.486 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 119817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-28T18:08:34.486 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 60705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-28T18:08:34.486 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 59737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-28T18:08:34.487 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 82337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-28T18:08:34.487 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 54433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-28T18:08:34.487 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-28T18:08:34.487 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ 2023-11-28T18:08:34.487 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-28T18:08:34.487 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-28T18:08:34.487 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-28T18:08:34.487 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-28T18:08:34.487 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-28T18:08:34.487 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-28T18:08:34.487 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-28T18:08:34.487 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 82577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-28T18:08:34.488 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-28T18:08:34.488 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-28T18:08:34.488 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-28T18:08:34.488 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-28T18:08:34.488 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-28T18:08:34.488 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-28T18:08:34.488 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-28T18:08:34.488 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 27145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-28T18:08:34.488 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 32137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-28T18:08:34.488 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-28T18:08:34.488 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-28T18:08:34.488 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 40105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-28T18:08:34.489 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-28T18:08:34.489 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 39193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-28T18:08:34.489 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 37385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-28T18:08:34.489 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-28T18:08:34.489 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-28T18:08:34.489 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-28T18:08:34.489 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-28T18:08:34.489 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-28T18:08:34.489 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-28T18:08:34.489 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 41945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-28T18:08:34.489 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 52457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-28T18:08:34.490 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 51545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-28T18:08:34.490 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 57449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-28T18:08:34.490 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 78873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-28T18:08:34.490 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 71345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-28T18:08:34.490 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 63225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-28T18:08:34.490 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 71321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-28T18:08:34.490 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 101473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-28T18:08:34.491 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 119505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-28T18:08:34.491 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-28T18:08:34.491 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 35625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-28T18:08:34.491 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-11-28T18:08:34.491 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-11-28T18:08:34.491 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/ 2023-11-28T18:08:34.491 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/ 2023-11-28T18:08:34.491 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-28T18:08:34.492 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-28T18:08:34.492 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/ 2023-11-28T18:08:34.492 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 27969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-11-28T18:08:34.492 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-11-28T18:08:34.492 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 27777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-11-28T18:08:34.492 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-28T18:08:34.492 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-28T18:08:34.492 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-28T18:08:34.492 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-11-28T18:08:34.493 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 6929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-28T18:08:34.493 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-11-28T18:08:34.493 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 50201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-28T18:08:34.493 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-28T18:08:34.494 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-28T18:08:34.494 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-28T18:08:34.494 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-11-28T18:08:34.494 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-11-28T18:08:34.494 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 62817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-28T18:08:34.494 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-28T18:08:34.494 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 47889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-28T18:08:34.494 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-11-28T18:08:34.494 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-28T18:08:34.494 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-28T18:08:34.495 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-11-28T18:08:34.495 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-28T18:08:34.495 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-28T18:08:34.495 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 47241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-28T18:08:34.495 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-11-28T18:08:34.495 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-28T18:08:34.495 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-28T18:08:34.495 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-11-28T18:08:34.495 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 29361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-11-28T18:08:34.495 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-28T18:08:34.495 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-11-28T18:08:34.495 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-28T18:08:34.496 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-28T18:08:34.496 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-28T18:08:34.496 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-28T18:08:34.496 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 47657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-11-28T18:08:34.496 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-28T18:08:34.496 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-28T18:08:34.496 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 27073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-28T18:08:34.496 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/ 2023-11-28T18:08:34.497 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-11-28T18:08:34.497 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-11-28T18:08:34.498 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 29737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-11-28T18:08:34.498 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ 2023-11-28T18:08:34.498 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ 2023-11-28T18:08:34.498 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 273609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-28T18:08:34.498 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 879969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-28T18:08:34.498 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 69273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-28T18:08:34.498 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 324217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-28T18:08:34.498 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 97985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-28T18:08:34.498 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 282273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-28T18:08:34.498 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 66201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-28T18:08:34.498 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/ 2023-11-28T18:08:34.498 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/ 2023-11-28T18:08:34.499 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 493073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-28T18:08:34.499 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/ 2023-11-28T18:08:34.499 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 419737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-28T18:08:34.499 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/ 2023-11-28T18:08:34.499 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 739025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-28T18:08:34.499 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ 2023-11-28T18:08:34.499 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 296625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-28T18:08:34.499 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/ocrdma/ 2023-11-28T18:08:34.499 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 153697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-11-28T18:08:34.499 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/usnic/ 2023-11-28T18:08:34.499 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 168969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-11-28T18:08:34.500 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ 2023-11-28T18:08:34.500 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ 2023-11-28T18:08:34.500 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 263081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-28T18:08:34.500 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ 2023-11-28T18:08:34.500 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 100241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-28T18:08:34.500 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ 2023-11-28T18:08:34.500 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 111425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-28T18:08:34.500 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ 2023-11-28T18:08:34.500 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 128753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-28T18:08:34.500 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ 2023-11-28T18:08:34.500 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 129529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-28T18:08:34.501 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/ 2023-11-28T18:08:34.501 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/evbug.ko 2023-11-28T18:08:34.501 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/ff-memless.ko 2023-11-28T18:08:34.501 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/ 2023-11-28T18:08:34.501 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-28T18:08:34.501 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-28T18:08:34.501 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 41241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-28T18:08:34.501 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 41801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-28T18:08:34.501 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/ 2023-11-28T18:08:34.502 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-11-28T18:08:34.502 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-11-28T18:08:34.502 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-11-28T18:08:34.502 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-11-28T18:08:34.502 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 36665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-28T18:08:34.502 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-11-28T18:08:34.502 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/lkkbd.ko 2023-11-28T18:08:34.502 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/lm8323.ko 2023-11-28T18:08:34.502 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/lm8333.ko 2023-11-28T18:08:34.503 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-11-28T18:08:34.503 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-11-28T18:08:34.503 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-11-28T18:08:34.503 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-11-28T18:08:34.503 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/newtonkbd.ko 2023-11-28T18:08:34.503 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-11-28T18:08:34.503 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/qt1070.ko 2023-11-28T18:08:34.503 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/qt2160.ko 2023-11-28T18:08:34.503 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-11-28T18:08:34.503 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/stowaway.ko 2023-11-28T18:08:34.504 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/sunkbd.ko 2023-11-28T18:08:34.504 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-11-28T18:08:34.504 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-11-28T18:08:34.504 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-11-28T18:08:34.504 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/xtkbd.ko 2023-11-28T18:08:34.504 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/matrix-keymap.ko 2023-11-28T18:08:34.504 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/ 2023-11-28T18:08:34.504 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 34329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-28T18:08:34.504 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 34177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-28T18:08:34.504 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 123681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-28T18:08:34.504 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 89449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-28T18:08:34.504 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/gpio_mouse.ko 2023-11-28T18:08:34.505 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 335929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/psmouse.ko 2023-11-28T18:08:34.505 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-28T18:08:34.505 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-28T18:08:34.505 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-28T18:08:34.505 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-28T18:08:34.505 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/ 2023-11-28T18:08:34.505 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 109873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-28T18:08:34.505 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/ 2023-11-28T18:08:34.505 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-28T18:08:34.505 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-28T18:08:34.505 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/ct82c710.ko 2023-11-28T18:08:34.506 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-28T18:08:34.506 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/parkbd.ko 2023-11-28T18:08:34.506 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/pcips2.ko 2023-11-28T18:08:34.506 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/ps2mult.ko 2023-11-28T18:08:34.506 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-28T18:08:34.506 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serport.ko 2023-11-28T18:08:34.506 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-28T18:08:34.506 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/ 2023-11-28T18:08:34.506 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/ 2023-11-28T18:08:34.506 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 39289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-28T18:08:34.506 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/ 2023-11-28T18:08:34.506 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-88pm860x.ko 2023-11-28T18:08:34.507 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-adp5520.ko 2023-11-28T18:08:34.507 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 59753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-bd2802.ko 2023-11-28T18:08:34.507 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-28T18:08:34.507 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-da903x.ko 2023-11-28T18:08:34.507 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-da9052.ko 2023-11-28T18:08:34.507 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-gpio.ko 2023-11-28T18:08:34.507 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-28T18:08:34.507 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3533.ko 2023-11-28T18:08:34.507 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm355x.ko 2023-11-28T18:08:34.507 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3642.ko 2023-11-28T18:08:34.507 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-28T18:08:34.507 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp8788.ko 2023-11-28T18:08:34.508 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-max8997.ko 2023-11-28T18:08:34.508 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-mc13783.ko 2023-11-28T18:08:34.508 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-menf21bmc.ko 2023-11-28T18:08:34.508 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-pca9532.ko 2023-11-28T18:08:34.508 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-pca955x.ko 2023-11-28T18:08:34.508 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-pca963x.ko 2023-11-28T18:08:34.508 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 34433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-28T18:08:34.508 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-tca6507.ko 2023-11-28T18:08:34.508 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-wm831x-status.ko 2023-11-28T18:08:34.509 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-wm8350.ko 2023-11-28T18:08:34.509 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ 2023-11-28T18:08:34.509 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-28T18:08:34.509 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-28T18:08:34.509 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-28T18:08:34.509 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-28T18:08:34.509 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-28T18:08:34.509 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-28T18:08:34.509 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-28T18:08:34.509 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-28T18:08:34.509 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/ 2023-11-28T18:08:34.510 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/ 2023-11-28T18:08:34.510 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 654969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-28T18:08:34.510 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 40577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-28T18:08:34.510 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 89601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bufio.ko 2023-11-28T18:08:34.510 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 47529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-28T18:08:34.510 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 156009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-28T18:08:34.510 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 106897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-28T18:08:34.510 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-28T18:08:34.510 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 53857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-era.ko 2023-11-28T18:08:34.510 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-28T18:08:34.510 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 41529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-28T18:08:34.511 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 32737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log.ko 2023-11-28T18:08:34.511 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 49545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-mirror.ko 2023-11-28T18:08:34.511 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 78033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-28T18:08:34.511 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-28T18:08:34.511 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 70825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-28T18:08:34.511 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 36297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-region-hash.ko 2023-11-28T18:08:34.511 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-28T18:08:34.511 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-28T18:08:34.511 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 109529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-snapshot.ko 2023-11-28T18:08:34.511 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-28T18:08:34.511 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 181865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-28T18:08:34.511 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 51737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-28T18:08:34.512 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-zero.ko 2023-11-28T18:08:34.512 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-28T18:08:34.512 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-28T18:08:34.512 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 32417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-28T18:08:34.512 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/ 2023-11-28T18:08:34.512 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 199937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-28T18:08:34.512 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 37337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-28T18:08:34.512 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 113073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-28T18:08:34.512 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 158681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-28T18:08:34.512 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 395721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-28T18:08:34.512 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/ 2023-11-28T18:08:34.513 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/ 2023-11-28T18:08:34.513 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 178137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-28T18:08:34.513 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 63385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-28T18:08:34.513 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 44665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-28T18:08:34.513 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 37225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-28T18:08:34.513 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 123321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-28T18:08:34.513 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 79041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-28T18:08:34.513 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 53017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-28T18:08:34.513 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/ 2023-11-28T18:08:34.513 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/88pm800.ko 2023-11-28T18:08:34.514 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/88pm805.ko 2023-11-28T18:08:34.514 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/88pm80x.ko 2023-11-28T18:08:34.514 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/arizona-i2c.ko 2023-11-28T18:08:34.514 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 123721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/arizona.ko 2023-11-28T18:08:34.514 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/bcm590xx.ko 2023-11-28T18:08:34.514 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/cros_ec_dev.ko 2023-11-28T18:08:34.514 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 34961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/dln2.ko 2023-11-28T18:08:34.515 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/janz-cmodio.ko 2023-11-28T18:08:34.515 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 53585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/kempld-core.ko 2023-11-28T18:08:34.515 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 27529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lm3533-core.ko 2023-11-28T18:08:34.515 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-11-28T18:08:34.515 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lp3943.ko 2023-11-28T18:08:34.515 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 54569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-28T18:08:34.515 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-28T18:08:34.515 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/max8907.ko 2023-11-28T18:08:34.515 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 29857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mc13xxx-core.ko 2023-11-28T18:08:34.515 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-11-28T18:08:34.515 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/menf21bmc.ko 2023-11-28T18:08:34.515 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/pcf50633-adc.ko 2023-11-28T18:08:34.516 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/pcf50633-gpio.ko 2023-11-28T18:08:34.516 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 34873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/pcf50633.ko 2023-11-28T18:08:34.516 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-11-28T18:08:34.516 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/retu-mfd.ko 2023-11-28T18:08:34.516 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 80425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/si476x-core.ko 2023-11-28T18:08:34.516 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 48457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-28T18:08:34.516 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-11-28T18:08:34.516 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 51137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/tps65010.ko 2023-11-28T18:08:34.516 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/tps6507x.ko 2023-11-28T18:08:34.517 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-28T18:08:34.517 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-28T18:08:34.517 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-28T18:08:34.517 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 44345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wm8994.ko 2023-11-28T18:08:34.517 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ 2023-11-28T18:08:34.517 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-11-28T18:08:34.517 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 48489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ad525x_dpot.ko 2023-11-28T18:08:34.517 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/ 2023-11-28T18:08:34.517 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 46585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-28T18:08:34.517 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-28T18:08:34.518 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 36729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-28T18:08:34.518 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 45593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-28T18:08:34.518 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/c2port/ 2023-11-28T18:08:34.518 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-11-28T18:08:34.518 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 35281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/c2port/core.ko 2023-11-28T18:08:34.518 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/ 2023-11-28T18:08:34.518 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-28T18:08:34.518 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ds1682.ko 2023-11-28T18:08:34.518 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/dummy-irq.ko 2023-11-28T18:08:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/echo/ 2023-11-28T18:08:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/echo/echo.ko 2023-11-28T18:08:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/ 2023-11-28T18:08:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 37849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-28T18:08:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-28T18:08:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-28T18:08:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-28T18:08:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 33009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-28T18:08:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/genwqe/ 2023-11-28T18:08:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 152201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-11-28T18:08:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hmc6352.ko 2023-11-28T18:08:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 31257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-28T18:08:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ibmasm/ 2023-11-28T18:08:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 72129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-11-28T18:08:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ics932s401.ko 2023-11-28T18:08:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-28T18:08:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-28T18:08:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/ 2023-11-28T18:08:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 42513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-28T18:08:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-28T18:08:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/ 2023-11-28T18:08:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 125209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-28T18:08:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 56521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-28T18:08:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 310113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-28T18:08:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 31329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/phantom.ko 2023-11-28T18:08:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ti-st/ 2023-11-28T18:08:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 74265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ti-st/st_drv.ko 2023-11-28T18:08:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-28T18:08:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-28T18:08:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-28T18:08:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 48697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-28T18:08:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/ 2023-11-28T18:08:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 183609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-28T18:08:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ 2023-11-28T18:08:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/ 2023-11-28T18:08:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 490129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-28T18:08:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/ 2023-11-28T18:08:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/ 2023-11-28T18:08:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 106033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-28T18:08:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 40417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-28T18:08:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-28T18:08:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/ 2023-11-28T18:08:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 453737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-28T18:08:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-28T18:08:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-28T18:08:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ 2023-11-28T18:08:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/ 2023-11-28T18:08:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 45785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-28T18:08:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 42105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-28T18:08:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 130553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-28T18:08:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 68849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-28T18:08:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ 2023-11-28T18:08:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 37649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-28T18:08:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 59689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-28T18:08:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 27201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-28T18:08:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 99049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-28T18:08:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/ 2023-11-28T18:08:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 54521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-28T18:08:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/ 2023-11-28T18:08:34.524 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 69137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-28T18:08:34.524 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/ 2023-11-28T18:08:34.524 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 59585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-28T18:08:34.524 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/ 2023-11-28T18:08:34.524 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 41409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-28T18:08:34.524 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/ 2023-11-28T18:08:34.524 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 54609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-28T18:08:34.524 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 49289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-28T18:08:34.524 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 87297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-28T18:08:34.524 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/ 2023-11-28T18:08:34.524 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/ 2023-11-28T18:08:34.524 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 97337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-28T18:08:34.525 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-11-28T18:08:34.525 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 104673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-28T18:08:34.525 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-11-28T18:08:34.525 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 90257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-28T18:08:34.525 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/ 2023-11-28T18:08:34.525 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 81569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-28T18:08:34.525 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 69729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-28T18:08:34.525 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/ 2023-11-28T18:08:34.525 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 78401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-28T18:08:34.525 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 176761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-28T18:08:34.525 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-11-28T18:08:34.525 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 1421497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-28T18:08:34.526 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 128569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-28T18:08:34.526 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/ 2023-11-28T18:08:34.526 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 132089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-28T18:08:34.526 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 312081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-28T18:08:34.526 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/ 2023-11-28T18:08:34.526 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/ 2023-11-28T18:08:34.526 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 323513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-28T18:08:34.526 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/ 2023-11-28T18:08:34.526 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-11-28T18:08:34.526 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 193497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-28T18:08:34.526 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-11-28T18:08:34.527 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 347417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-28T18:08:34.527 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-11-28T18:08:34.527 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 1044497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-28T18:08:34.527 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-11-28T18:08:34.527 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 166193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-28T18:08:34.527 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-11-28T18:08:34.527 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 34881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-28T18:08:34.528 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/ 2023-11-28T18:08:34.528 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/ 2023-11-28T18:08:34.528 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 204345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-28T18:08:34.528 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/ 2023-11-28T18:08:34.528 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/ 2023-11-28T18:08:34.528 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 69449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-28T18:08:34.528 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 61489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-28T18:08:34.528 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 141225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-28T18:08:34.528 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 51497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-28T18:08:34.528 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 62817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-28T18:08:34.528 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 31369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-28T18:08:34.529 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/ 2023-11-28T18:08:34.529 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 60873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-28T18:08:34.529 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 62833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-28T18:08:34.529 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 45721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-28T18:08:34.529 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ec_bhf.ko 2023-11-28T18:08:34.529 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/ 2023-11-28T18:08:34.530 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/ 2023-11-28T18:08:34.530 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 281137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-28T18:08:34.530 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 49457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-28T18:08:34.530 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 51737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-28T18:08:34.530 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fujitsu/ 2023-11-28T18:08:34.530 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 45345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-11-28T18:08:34.530 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ 2023-11-28T18:08:34.531 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 90465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-28T18:08:34.531 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/ 2023-11-28T18:08:34.533 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 295577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-28T18:08:34.534 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/ 2023-11-28T18:08:34.534 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 584497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-28T18:08:34.534 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/ 2023-11-28T18:08:34.534 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 246209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-28T18:08:34.534 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/ 2023-11-28T18:08:34.534 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 925825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-28T18:08:34.535 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/ 2023-11-28T18:08:34.535 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 302937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-28T18:08:34.535 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/ 2023-11-28T18:08:34.535 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 514641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-28T18:08:34.535 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/ 2023-11-28T18:08:34.535 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 93593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-28T18:08:34.535 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-11-28T18:08:34.535 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 734305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-28T18:08:34.535 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-11-28T18:08:34.535 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 151769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-28T18:08:34.535 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 86921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-28T18:08:34.536 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/ 2023-11-28T18:08:34.536 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-28T18:08:34.536 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 102289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-28T18:08:34.536 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 122649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-28T18:08:34.536 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/ 2023-11-28T18:08:34.536 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-11-28T18:08:34.536 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 727601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-28T18:08:34.536 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 278937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-28T18:08:34.536 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-11-28T18:08:34.536 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-11-28T18:08:34.536 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 1113681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-28T18:08:34.537 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ 2023-11-28T18:08:34.537 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 53041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-11-28T18:08:34.537 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 40993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-11-28T18:08:34.537 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-11-28T18:08:34.537 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 95665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-28T18:08:34.537 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/ 2023-11-28T18:08:34.537 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-11-28T18:08:34.537 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 108649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-28T18:08:34.537 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ 2023-11-28T18:08:34.537 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 73441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-28T18:08:34.537 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 47777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-28T18:08:34.537 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/ 2023-11-28T18:08:34.538 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 150905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-28T18:08:34.538 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/ 2023-11-28T18:08:34.538 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 136721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-28T18:08:34.538 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/ 2023-11-28T18:08:34.538 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 50481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-28T18:08:34.538 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 43145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-28T18:08:34.538 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/ 2023-11-28T18:08:34.538 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-11-28T18:08:34.538 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 227897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-28T18:08:34.538 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 87081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-28T18:08:34.538 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-11-28T18:08:34.539 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 672049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-28T18:08:34.539 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/ 2023-11-28T18:08:34.539 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 43017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-28T18:08:34.539 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/ 2023-11-28T18:08:34.539 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 67385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-28T18:08:34.539 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 83985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-28T18:08:34.539 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 29321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-28T18:08:34.539 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 189801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-28T18:08:34.539 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/samsung/ 2023-11-28T18:08:34.539 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-11-28T18:08:34.539 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 104241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-11-28T18:08:34.539 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/ 2023-11-28T18:08:34.540 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 1052497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-28T18:08:34.540 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/ 2023-11-28T18:08:34.540 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 47097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-28T18:08:34.540 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/ 2023-11-28T18:08:34.540 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 54817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-28T18:08:34.540 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 63337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-28T18:08:34.540 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/ 2023-11-28T18:08:34.540 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 61537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-28T18:08:34.540 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 67193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-28T18:08:34.540 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 73409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-28T18:08:34.540 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 68177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-28T18:08:34.540 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/ 2023-11-28T18:08:34.541 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-11-28T18:08:34.541 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-11-28T18:08:34.541 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 39905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-28T18:08:34.541 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-11-28T18:08:34.541 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 487025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-28T18:08:34.541 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/ 2023-11-28T18:08:34.541 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 103337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-28T18:08:34.541 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 166097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-28T18:08:34.541 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 66329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-28T18:08:34.541 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 78801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-28T18:08:34.541 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/ 2023-11-28T18:08:34.542 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 50057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-28T18:08:34.542 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/ 2023-11-28T18:08:34.542 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 86529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-28T18:08:34.542 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/ 2023-11-28T18:08:34.542 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 80273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-28T18:08:34.542 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 83633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-28T18:08:34.542 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/ 2023-11-28T18:08:34.542 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 46241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-28T18:08:34.543 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-28T18:08:34.543 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/ 2023-11-28T18:08:34.543 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 59993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-28T18:08:34.545 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/ 2023-11-28T18:08:34.545 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 41001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/defxx.ko 2023-11-28T18:08:34.545 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/skfp/ 2023-11-28T18:08:34.545 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 216985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/skfp/skfp.ko 2023-11-28T18:08:34.545 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 77345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-28T18:08:34.546 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/ 2023-11-28T18:08:34.546 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 235521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-28T18:08:34.547 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/ 2023-11-28T18:08:34.547 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-28T18:08:34.547 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-28T18:08:34.547 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ 2023-11-28T18:08:34.547 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 83769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-28T18:08:34.547 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 62145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-28T18:08:34.547 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-28T18:08:34.547 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/ 2023-11-28T18:08:34.548 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-28T18:08:34.548 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-28T18:08:34.548 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-gpio.ko 2023-11-28T18:08:34.549 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-28T18:08:34.549 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-28T18:08:34.549 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-28T18:08:34.549 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 43953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-28T18:08:34.549 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-28T18:08:34.549 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/ 2023-11-28T18:08:34.550 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-28T18:08:34.550 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 38977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-28T18:08:34.550 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ 2023-11-28T18:08:34.550 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-28T18:08:34.550 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 50033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-28T18:08:34.551 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 44721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-28T18:08:34.551 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-28T18:08:34.551 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 42241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-28T18:08:34.551 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-28T18:08:34.551 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-28T18:08:34.551 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-28T18:08:34.552 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-28T18:08:34.552 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-28T18:08:34.552 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 82049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-28T18:08:34.552 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 76113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-28T18:08:34.552 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-28T18:08:34.552 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 97857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-28T18:08:34.553 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-28T18:08:34.553 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 50065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-28T18:08:34.553 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 29985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-28T18:08:34.553 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-28T18:08:34.553 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-28T18:08:34.553 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/plip/ 2023-11-28T18:08:34.553 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 43505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/plip/plip.ko 2023-11-28T18:08:34.554 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ 2023-11-28T18:08:34.554 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-28T18:08:34.554 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 29713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-28T18:08:34.554 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-28T18:08:34.555 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-28T18:08:34.555 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-28T18:08:34.555 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 42545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-28T18:08:34.555 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-28T18:08:34.555 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 32689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-28T18:08:34.556 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sb1000.ko 2023-11-28T18:08:34.556 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/ 2023-11-28T18:08:34.557 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 42465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-28T18:08:34.557 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-28T18:08:34.557 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 64449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-28T18:08:34.558 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/ 2023-11-28T18:08:34.558 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 106169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-28T18:08:34.558 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-28T18:08:34.558 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-28T18:08:34.559 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-28T18:08:34.559 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-28T18:08:34.559 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-28T18:08:34.560 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 87017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-28T18:08:34.560 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/ 2023-11-28T18:08:34.560 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 144297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-28T18:08:34.560 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/ 2023-11-28T18:08:34.560 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 297329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-28T18:08:34.560 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/ 2023-11-28T18:08:34.561 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 144585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-28T18:08:34.561 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ 2023-11-28T18:08:34.561 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-28T18:08:34.561 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/ 2023-11-28T18:08:34.561 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/ 2023-11-28T18:08:34.561 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 265121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-28T18:08:34.562 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 102209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-28T18:08:34.562 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/ 2023-11-28T18:08:34.562 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 133841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-28T18:08:34.562 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-28T18:08:34.562 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 84465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-28T18:08:34.563 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-28T18:08:34.563 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/ 2023-11-28T18:08:34.563 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/ 2023-11-28T18:08:34.563 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-28T18:08:34.563 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-11-28T18:08:34.564 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-11-28T18:08:34.564 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pci-stub.ko 2023-11-28T18:08:34.564 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 46465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-28T18:08:34.564 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/ 2023-11-28T18:08:34.564 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-28T18:08:34.564 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 145809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pcmcia.ko 2023-11-28T18:08:34.565 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 62137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pcmcia_core.ko 2023-11-28T18:08:34.565 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 34345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-11-28T18:08:34.565 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 34265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-28T18:08:34.565 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 112889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-28T18:08:34.565 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/ 2023-11-28T18:08:34.565 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/broadcom/ 2023-11-28T18:08:34.566 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-11-28T18:08:34.566 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/samsung/ 2023-11-28T18:08:34.566 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-11-28T18:08:34.566 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/ 2023-11-28T18:08:34.567 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/ 2023-11-28T18:08:34.567 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 70153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-28T18:08:34.567 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/ 2023-11-28T18:08:34.568 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/ 2023-11-28T18:08:34.568 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 50889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-11-28T18:08:34.568 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec.ko 2023-11-28T18:08:34.568 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-11-28T18:08:34.568 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-11-28T18:08:34.569 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-11-28T18:08:34.569 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-11-28T18:08:34.569 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-11-28T18:08:34.569 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/ 2023-11-28T18:08:34.570 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 67097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-28T18:08:34.570 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 39145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-28T18:08:34.570 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 37025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-28T18:08:34.570 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 59665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-28T18:08:34.571 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 31161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-28T18:08:34.571 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 127441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-28T18:08:34.571 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 36849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-28T18:08:34.571 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 49121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-28T18:08:34.571 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-28T18:08:34.572 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 40265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-28T18:08:34.572 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-28T18:08:34.572 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 36593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-28T18:08:34.572 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/ibm_rtl.ko 2023-11-28T18:08:34.573 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/ 2023-11-28T18:08:34.573 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-28T18:08:34.573 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-11-28T18:08:34.573 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 43497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-28T18:08:34.573 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-28T18:08:34.574 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-28T18:08:34.574 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 34801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-28T18:08:34.574 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 42329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-28T18:08:34.574 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-28T18:08:34.574 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-28T18:08:34.575 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-28T18:08:34.575 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-28T18:08:34.575 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-28T18:08:34.575 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 57569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-28T18:08:34.575 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/ 2023-11-28T18:08:34.576 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ 2023-11-28T18:08:34.576 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/88pm860x_battery.ko 2023-11-28T18:08:34.576 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 33985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/88pm860x_charger.ko 2023-11-28T18:08:34.576 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 55057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/bq2415x_charger.ko 2023-11-28T18:08:34.577 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 54081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/bq24190_charger.ko 2023-11-28T18:08:34.577 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/bq24735-charger.ko 2023-11-28T18:08:34.577 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-11-28T18:08:34.577 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/da9030_battery.ko 2023-11-28T18:08:34.577 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/da9052-battery.ko 2023-11-28T18:08:34.578 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 29281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2760_battery.ko 2023-11-28T18:08:34.579 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2780_battery.ko 2023-11-28T18:08:34.580 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2781_battery.ko 2023-11-28T18:08:34.581 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2782_battery.ko 2023-11-28T18:08:34.581 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/gpio-charger.ko 2023-11-28T18:08:34.581 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/isp1704_charger.ko 2023-11-28T18:08:34.581 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/lp8727_charger.ko 2023-11-28T18:08:34.582 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max14577_charger.ko 2023-11-28T18:08:34.582 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max17040_battery.ko 2023-11-28T18:08:34.583 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max17042_battery.ko 2023-11-28T18:08:34.583 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max8903_charger.ko 2023-11-28T18:08:34.583 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max8925_power.ko 2023-11-28T18:08:34.583 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/pcf50633-charger.ko 2023-11-28T18:08:34.583 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 38009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/sbs-battery.ko 2023-11-28T18:08:34.583 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/test_power.ko 2023-11-28T18:08:34.584 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/tps65090-charger.ko 2023-11-28T18:08:34.584 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/wm831x_backup.ko 2023-11-28T18:08:34.584 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/wm831x_power.ko 2023-11-28T18:08:34.584 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/wm8350_power.ko 2023-11-28T18:08:34.584 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/ 2023-11-28T18:08:34.585 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 57729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-28T18:08:34.585 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-28T18:08:34.585 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/ 2023-11-28T18:08:34.585 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/ 2023-11-28T18:08:34.585 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-28T18:08:34.585 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-28T18:08:34.586 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-28T18:08:34.586 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 41873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-28T18:08:34.586 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ 2023-11-28T18:08:34.586 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 66385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-28T18:08:34.587 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-28T18:08:34.587 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/ 2023-11-28T18:08:34.587 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-88pm80x.ko 2023-11-28T18:08:34.587 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-88pm860x.ko 2023-11-28T18:08:34.587 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-28T18:08:34.587 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-da9052.ko 2023-11-28T18:08:34.588 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-da9055.ko 2023-11-28T18:08:34.588 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-da9063.ko 2023-11-28T18:08:34.588 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-28T18:08:34.588 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 68577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-28T18:08:34.589 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-28T18:08:34.589 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-28T18:08:34.590 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-28T18:08:34.590 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-28T18:08:34.590 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-28T18:08:34.590 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-28T18:08:34.590 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-28T18:08:34.591 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-28T18:08:34.591 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-28T18:08:34.591 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-28T18:08:34.592 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 35969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-28T18:08:34.592 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-lp8788.ko 2023-11-28T18:08:34.592 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 38097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-28T18:08:34.592 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-28T18:08:34.592 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-28T18:08:34.593 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t86.ko 2023-11-28T18:08:34.593 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-28T18:08:34.593 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8907.ko 2023-11-28T18:08:34.593 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8925.ko 2023-11-28T18:08:34.593 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8997.ko 2023-11-28T18:08:34.594 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8998.ko 2023-11-28T18:08:34.594 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-11-28T18:08:34.594 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-28T18:08:34.594 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-palmas.ko 2023-11-28T18:08:34.594 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 46633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-28T18:08:34.594 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf50633.ko 2023-11-28T18:08:34.595 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-28T18:08:34.595 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-28T18:08:34.595 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 29817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-28T18:08:34.595 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-28T18:08:34.596 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rc5t583.ko 2023-11-28T18:08:34.596 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-28T18:08:34.596 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 33753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-28T18:08:34.596 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 27833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-28T18:08:34.596 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-28T18:08:34.597 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-28T18:08:34.597 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-s35390a.ko 2023-11-28T18:08:34.597 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-28T18:08:34.597 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-tps6586x.ko 2023-11-28T18:08:34.598 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-tps65910.ko 2023-11-28T18:08:34.598 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-wm831x.ko 2023-11-28T18:08:34.598 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-wm8350.ko 2023-11-28T18:08:34.598 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 27873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-28T18:08:34.598 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ 2023-11-28T18:08:34.598 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 77617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-28T18:08:34.598 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 57665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-28T18:08:34.599 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 55105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-28T18:08:34.599 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 135185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-28T18:08:34.599 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-28T18:08:34.599 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/ 2023-11-28T18:08:34.599 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 272305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-28T18:08:34.599 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 100617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-28T18:08:34.600 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/ 2023-11-28T18:08:34.600 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 356201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-28T18:08:34.600 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 295801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-28T18:08:34.600 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic94xx/ 2023-11-28T18:08:34.600 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 189185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-11-28T18:08:34.600 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-28T18:08:34.600 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/ 2023-11-28T18:08:34.600 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 106881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-28T18:08:34.600 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 38185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-28T18:08:34.600 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/ 2023-11-28T18:08:34.600 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 246985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-28T18:08:34.601 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/ 2023-11-28T18:08:34.601 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 1171945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-28T18:08:34.601 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/ 2023-11-28T18:08:34.601 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 237273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-28T18:08:34.601 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/ 2023-11-28T18:08:34.601 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 115369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-28T18:08:34.602 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 40585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-28T18:08:34.602 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/ 2023-11-28T18:08:34.602 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 308921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-28T18:08:34.602 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/ 2023-11-28T18:08:34.602 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-11-28T18:08:34.602 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 76145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-28T18:08:34.602 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-11-28T18:08:34.602 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 101449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-28T18:08:34.603 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 146689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-28T18:08:34.603 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 75553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-28T18:08:34.603 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 38073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-28T18:08:34.603 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/ 2023-11-28T18:08:34.603 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 212425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-28T18:08:34.604 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 60505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-28T18:08:34.604 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/ 2023-11-28T18:08:34.604 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 78209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-28T18:08:34.604 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 158505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-28T18:08:34.604 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fdomain.ko 2023-11-28T18:08:34.604 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/ 2023-11-28T18:08:34.605 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 263489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-28T18:08:34.605 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 201817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-28T18:08:34.605 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 51209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-28T18:08:34.605 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 49041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-28T18:08:34.605 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 35337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/imm.ko 2023-11-28T18:08:34.606 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 44057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-28T18:08:34.606 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 266537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-28T18:08:34.606 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 101881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-28T18:08:34.607 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/ 2023-11-28T18:08:34.607 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 317153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-28T18:08:34.607 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-28T18:08:34.607 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 52961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-28T18:08:34.608 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/ 2023-11-28T18:08:34.608 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 314865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-28T18:08:34.608 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 145585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-28T18:08:34.608 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 53521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-28T18:08:34.608 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/ 2023-11-28T18:08:34.609 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 216665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-28T18:08:34.609 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/ 2023-11-28T18:08:34.609 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 2197249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-28T18:08:34.609 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/ 2023-11-28T18:08:34.609 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 73569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-28T18:08:34.609 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 33529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-28T18:08:34.609 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 310617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-28T18:08:34.609 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 84721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-28T18:08:34.609 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/ 2023-11-28T18:08:34.610 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 659825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-28T18:08:34.610 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/ 2023-11-28T18:08:34.610 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 119561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-28T18:08:34.610 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 78129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-28T18:08:34.610 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/ 2023-11-28T18:08:34.610 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 81625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-11-28T18:08:34.611 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-11-28T18:08:34.611 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-11-28T18:08:34.611 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-11-28T18:08:34.611 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/ 2023-11-28T18:08:34.611 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 364105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-28T18:08:34.611 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 123617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-28T18:08:34.612 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 34513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ppa.ko 2023-11-28T18:08:34.612 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 55361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-28T18:08:34.613 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/ 2023-11-28T18:08:34.613 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 1876689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-28T18:08:34.613 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 85289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-28T18:08:34.613 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/ 2023-11-28T18:08:34.613 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 559033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-28T18:08:34.614 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qlogicfas408.ko 2023-11-28T18:08:34.614 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-28T18:08:34.614 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 238761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-28T18:08:34.614 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 173937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-28T18:08:34.614 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 279313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-28T18:08:34.614 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 97729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-28T18:08:34.615 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 73881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-28T18:08:34.615 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 44993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-28T18:08:34.615 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-28T18:08:34.615 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 115889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-28T18:08:34.615 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 46825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-28T18:08:34.615 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/ 2023-11-28T18:08:34.616 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 162121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-28T18:08:34.616 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 42297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-28T18:08:34.616 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 50321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-28T18:08:34.616 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 39961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-28T18:08:34.616 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 50025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/xen-scsifront.ko 2023-11-28T18:08:34.616 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ 2023-11-28T18:08:34.617 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 129441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-28T18:08:34.617 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/ 2023-11-28T18:08:34.617 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/ 2023-11-28T18:08:34.617 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 736601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-28T18:08:34.617 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/ 2023-11-28T18:08:34.617 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 49985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-28T18:08:34.618 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 39713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-28T18:08:34.618 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 43025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-28T18:08:34.618 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 930257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-28T18:08:34.618 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 44305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-28T18:08:34.618 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 122521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-28T18:08:34.619 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/ 2023-11-28T18:08:34.619 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 75593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-28T18:08:34.619 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/ 2023-11-28T18:08:34.619 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/ 2023-11-28T18:08:34.620 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/ 2023-11-28T18:08:34.621 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-28T18:08:34.621 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 29321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-28T18:08:34.621 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-28T18:08:34.622 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-28T18:08:34.622 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-28T18:08:34.622 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-28T18:08:34.623 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-28T18:08:34.623 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-28T18:08:34.623 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-28T18:08:34.623 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-28T18:08:34.623 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-28T18:08:34.624 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 37065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-28T18:08:34.624 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-11-28T18:08:34.624 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-28T18:08:34.624 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-28T18:08:34.624 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-28T18:08:34.624 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ 2023-11-28T18:08:34.624 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ 2023-11-28T18:08:34.624 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 78425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-28T18:08:34.625 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 57865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/moxa.ko 2023-11-28T18:08:34.625 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 50953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/mxser.ko 2023-11-28T18:08:34.625 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 108601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-28T18:08:34.625 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 31961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-28T18:08:34.625 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 67929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-28T18:08:34.626 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/ 2023-11-28T18:08:34.626 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/ 2023-11-28T18:08:34.626 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 35753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-11-28T18:08:34.626 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 59497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-28T18:08:34.626 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-11-28T18:08:34.626 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/altera_uart.ko 2023-11-28T18:08:34.626 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-28T18:08:34.627 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 72425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-11-28T18:08:34.627 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/ 2023-11-28T18:08:34.627 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 103073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-28T18:08:34.627 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/rp2.ko 2023-11-28T18:08:34.628 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 52817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/sc16is7xx.ko 2023-11-28T18:08:34.628 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/ 2023-11-28T18:08:34.628 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ 2023-11-28T18:08:34.629 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 531265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-28T18:08:34.629 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ 2023-11-28T18:08:34.629 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-28T18:08:34.629 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-11-28T18:08:34.630 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/ 2023-11-28T18:08:34.630 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 44513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-28T18:08:34.630 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-28T18:08:34.631 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-28T18:08:34.632 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_dmem_genirq.ko 2023-11-28T18:08:34.632 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_mf624.ko 2023-11-28T18:08:34.632 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_netx.ko 2023-11-28T18:08:34.633 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-28T18:08:34.634 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-11-28T18:08:34.634 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-28T18:08:34.634 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/ 2023-11-28T18:08:34.635 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/c67x00/ 2023-11-28T18:08:34.635 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 73297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/c67x00/c67x00.ko 2023-11-28T18:08:34.635 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ 2023-11-28T18:08:34.636 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 144377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-11-28T18:08:34.636 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-11-28T18:08:34.636 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-11-28T18:08:34.636 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-11-28T18:08:34.637 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/ 2023-11-28T18:08:34.637 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 87089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-28T18:08:34.639 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 51345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-28T18:08:34.640 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 47305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-28T18:08:34.640 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 89857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-28T18:08:34.640 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/common/ 2023-11-28T18:08:34.640 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/common/ulpi.ko 2023-11-28T18:08:34.641 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc2/ 2023-11-28T18:08:34.641 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-11-28T18:08:34.641 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/ 2023-11-28T18:08:34.641 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-11-28T18:08:34.642 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-11-28T18:08:34.643 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 448505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/dwc3.ko 2023-11-28T18:08:34.643 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/ 2023-11-28T18:08:34.643 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/ 2023-11-28T18:08:34.644 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 60177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/u_ether.ko 2023-11-28T18:08:34.644 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 51881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/u_serial.ko 2023-11-28T18:08:34.645 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-11-28T18:08:34.645 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 42449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-11-28T18:08:34.645 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 27953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-11-28T18:08:34.646 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 36513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-11-28T18:08:34.646 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 92489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-11-28T18:08:34.646 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 51049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-11-28T18:08:34.647 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 122081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-11-28T18:08:34.647 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 57681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-11-28T18:08:34.647 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-11-28T18:08:34.648 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 31265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-11-28T18:08:34.648 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 57729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-11-28T18:08:34.648 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 85609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-11-28T18:08:34.649 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-11-28T18:08:34.649 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 58113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-11-28T18:08:34.649 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 67337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-11-28T18:08:34.650 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/ 2023-11-28T18:08:34.650 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-11-28T18:08:34.650 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-11-28T18:08:34.651 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-11-28T18:08:34.651 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-11-28T18:08:34.651 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-11-28T18:08:34.652 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-11-28T18:08:34.652 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-11-28T18:08:34.652 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-11-28T18:08:34.653 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 32073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-11-28T18:08:34.653 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-11-28T18:08:34.653 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-11-28T18:08:34.654 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 27201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-11-28T18:08:34.654 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 62185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-11-28T18:08:34.654 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-11-28T18:08:34.654 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 161225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/libcomposite.ko 2023-11-28T18:08:34.654 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/ 2023-11-28T18:08:34.655 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-11-28T18:08:34.655 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/bdc/ 2023-11-28T18:08:34.655 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 157681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-11-28T18:08:34.655 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 36089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-11-28T18:08:34.656 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 52721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-11-28T18:08:34.656 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 69705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-11-28T18:08:34.656 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 56033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-11-28T18:08:34.656 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 76057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/net2272.ko 2023-11-28T18:08:34.656 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 75073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/net2280.ko 2023-11-28T18:08:34.659 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 73169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-11-28T18:08:34.659 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 65809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-11-28T18:08:34.659 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 66097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-11-28T18:08:34.660 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 81505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-11-28T18:08:34.660 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 156073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-11-28T18:08:34.660 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/ 2023-11-28T18:08:34.660 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/bcma-hcd.ko 2023-11-28T18:08:34.660 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 80113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/isp116x-hcd.ko 2023-11-28T18:08:34.660 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 87353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-11-28T18:08:34.660 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 78873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-11-28T18:08:34.661 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 51329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-28T18:08:34.661 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811_cs.ko 2023-11-28T18:08:34.661 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/ssb-hcd.ko 2023-11-28T18:08:34.661 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-11-28T18:08:34.661 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/ 2023-11-28T18:08:34.661 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 34585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-28T18:08:34.661 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-28T18:08:34.662 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ 2023-11-28T18:08:34.662 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 41721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-28T18:08:34.662 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-28T18:08:34.667 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-11-28T18:08:34.667 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/cytherm.ko 2023-11-28T18:08:34.667 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ehset.ko 2023-11-28T18:08:34.667 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-28T18:08:34.667 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-28T18:08:34.667 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-28T18:08:34.667 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-28T18:08:34.667 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 36553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-28T18:08:34.667 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-28T18:08:34.668 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 36529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-28T18:08:34.668 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 36265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-28T18:08:34.668 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 27793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/lvstest.ko 2023-11-28T18:08:34.668 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/ 2023-11-28T18:08:34.668 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 64841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-28T18:08:34.668 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-28T18:08:34.669 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-28T18:08:34.669 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-28T18:08:34.669 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-28T18:08:34.669 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 70809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbtest.ko 2023-11-28T18:08:34.669 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 41905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-28T18:08:34.669 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 27617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-28T18:08:34.670 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/mon/ 2023-11-28T18:08:34.670 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 72953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/mon/usbmon.ko 2023-11-28T18:08:34.670 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/musb/ 2023-11-28T18:08:34.670 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 301697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/musb/musb_hdrc.ko 2023-11-28T18:08:34.670 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/ 2023-11-28T18:08:34.676 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-28T18:08:34.676 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-11-28T18:08:34.677 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-isp1301.ko 2023-11-28T18:08:34.677 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-tahvo.ko 2023-11-28T18:08:34.677 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ 2023-11-28T18:08:34.677 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-28T18:08:34.677 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 33329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-28T18:08:34.677 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-28T18:08:34.677 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 38105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-28T18:08:34.677 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 70257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-28T18:08:34.678 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 31057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-28T18:08:34.678 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 58217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-28T18:08:34.678 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 49817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-28T18:08:34.678 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-28T18:08:34.678 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 44273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/f81232.ko 2023-11-28T18:08:34.678 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 175137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-28T18:08:34.679 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 53113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-28T18:08:34.679 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 137081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-28T18:08:34.679 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 124833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-28T18:08:34.679 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 67769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-28T18:08:34.680 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-28T18:08:34.680 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-28T18:08:34.680 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 58233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-28T18:08:34.681 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 107521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-28T18:08:34.681 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 35617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-28T18:08:34.681 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-28T18:08:34.681 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 33689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-28T18:08:34.681 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 32953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-28T18:08:34.681 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/metro-usb.ko 2023-11-28T18:08:34.682 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 66449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-28T18:08:34.682 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 78217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-28T18:08:34.682 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 56337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mxuport.ko 2023-11-28T18:08:34.682 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-28T18:08:34.682 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-28T18:08:34.682 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-28T18:08:34.683 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 191137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-28T18:08:34.683 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 39073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-28T18:08:34.683 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 54353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-28T18:08:34.683 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-28T18:08:34.683 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 37825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-28T18:08:34.683 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 34321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-28T18:08:34.683 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-28T18:08:34.684 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 51625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-28T18:08:34.684 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-28T18:08:34.684 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 29993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-28T18:08:34.684 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-28T18:08:34.684 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 61489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-28T18:08:34.684 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-28T18:08:34.684 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-28T18:08:34.684 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 33201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-28T18:08:34.685 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 125609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usbserial.ko 2023-11-28T18:08:34.685 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-28T18:08:34.685 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 37409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-28T18:08:34.685 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/wishbone-serial.ko 2023-11-28T18:08:34.685 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-28T18:08:34.685 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ 2023-11-28T18:08:34.685 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 52913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-28T18:08:34.686 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-28T18:08:34.686 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-28T18:08:34.686 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-28T18:08:34.686 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 33497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-28T18:08:34.686 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-28T18:08:34.687 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-28T18:08:34.687 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-28T18:08:34.687 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-28T18:08:34.687 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-28T18:08:34.688 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-28T18:08:34.688 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-28T18:08:34.688 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-28T18:08:34.688 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 35337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-28T18:08:34.688 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 179969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-28T18:08:34.689 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/ 2023-11-28T18:08:34.689 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 67481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-28T18:08:34.689 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 72177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-28T18:08:34.690 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 107001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-28T18:08:34.690 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/ 2023-11-28T18:08:34.690 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/ 2023-11-28T18:08:34.690 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 154297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-28T18:08:34.690 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 29537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-28T18:08:34.690 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 108737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-28T18:08:34.690 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 77609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-28T18:08:34.691 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/ 2023-11-28T18:08:34.691 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 113785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-28T18:08:34.691 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-28T18:08:34.691 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 58089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-28T18:08:34.692 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 89249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-28T18:08:34.692 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/ 2023-11-28T18:08:34.692 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/ 2023-11-28T18:08:34.692 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-11-28T18:08:34.692 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/aat2870_bl.ko 2023-11-28T18:08:34.692 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/adp5520_bl.ko 2023-11-28T18:08:34.692 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 32609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/adp8860_bl.ko 2023-11-28T18:08:34.692 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 39993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/adp8870_bl.ko 2023-11-28T18:08:34.692 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-28T18:08:34.693 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/as3711_bl.ko 2023-11-28T18:08:34.693 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/bd6107.ko 2023-11-28T18:08:34.693 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/da903x_bl.ko 2023-11-28T18:08:34.693 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/da9052_bl.ko 2023-11-28T18:08:34.693 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/gpio_backlight.ko 2023-11-28T18:08:34.693 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/kb3886_bl.ko 2023-11-28T18:08:34.693 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-28T18:08:34.693 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lm3533_bl.ko 2023-11-28T18:08:34.693 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lm3639_bl.ko 2023-11-28T18:08:34.693 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lv5207lp.ko 2023-11-28T18:08:34.694 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/max8925_bl.ko 2023-11-28T18:08:34.694 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/pandora_bl.ko 2023-11-28T18:08:34.694 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-11-28T18:08:34.694 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-28T18:08:34.694 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/wm831x_bl.ko 2023-11-28T18:08:34.694 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/ 2023-11-28T18:08:34.694 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ 2023-11-28T18:08:34.694 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-28T18:08:34.694 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 38985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-28T18:08:34.695 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/matrox_w1.ko 2023-11-28T18:08:34.695 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/w1-gpio.ko 2023-11-28T18:08:34.695 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/ 2023-11-28T18:08:34.695 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-28T18:08:34.695 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-28T18:08:34.695 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-28T18:08:34.695 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-28T18:08:34.695 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-28T18:08:34.695 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-28T18:08:34.695 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-28T18:08:34.695 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-28T18:08:34.695 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-28T18:08:34.696 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-28T18:08:34.696 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 56425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-28T18:08:34.696 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 102673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-28T18:08:34.696 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ 2023-11-28T18:08:34.696 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/acquirewdt.ko 2023-11-28T18:08:34.696 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/advantechwdt.ko 2023-11-28T18:08:34.696 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-28T18:08:34.696 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-28T18:08:34.696 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/cpu5wdt.ko 2023-11-28T18:08:34.696 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/da9052_wdt.ko 2023-11-28T18:08:34.696 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/da9055_wdt.ko 2023-11-28T18:08:34.697 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/da9063_wdt.ko 2023-11-28T18:08:34.697 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/dw_wdt.ko 2023-11-28T18:08:34.697 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/eurotechwdt.ko 2023-11-28T18:08:34.697 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-28T18:08:34.697 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 27401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-28T18:08:34.697 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-28T18:08:34.697 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-28T18:08:34.698 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-28T18:08:34.698 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-28T18:08:34.698 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-28T18:08:34.698 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-28T18:08:34.698 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-28T18:08:34.698 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-28T18:08:34.698 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/kempld_wdt.ko 2023-11-28T18:08:34.698 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-28T18:08:34.698 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/mena21_wdt.ko 2023-11-28T18:08:34.698 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-11-28T18:08:34.699 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-28T18:08:34.699 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-11-28T18:08:34.699 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pc87413_wdt.ko 2023-11-28T18:08:34.704 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 38657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-28T18:08:34.705 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 31825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-28T18:08:34.705 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/retu_wdt.ko 2023-11-28T18:08:34.705 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-11-28T18:08:34.705 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-11-28T18:08:34.705 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-28T18:08:34.705 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sc1200wdt.ko 2023-11-28T18:08:34.705 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-28T18:08:34.706 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-11-28T18:08:34.706 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-28T18:08:34.706 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-28T18:08:34.706 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/twl4030_wdt.ko 2023-11-28T18:08:34.706 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-28T18:08:34.706 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-28T18:08:34.706 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-28T18:08:34.707 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-28T18:08:34.707 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wafer5823wdt.ko 2023-11-28T18:08:34.707 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-28T18:08:34.707 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wm831x_wdt.ko 2023-11-28T18:08:34.707 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wm8350_wdt.ko 2023-11-28T18:08:34.707 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-28T18:08:34.707 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/ 2023-11-28T18:08:34.708 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-28T18:08:34.708 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-28T18:08:34.708 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 47865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-28T18:08:34.708 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/ 2023-11-28T18:08:34.708 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 176377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-28T18:08:34.708 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 40121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-28T18:08:34.708 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 71249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-scsiback.ko 2023-11-28T18:08:34.708 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/ 2023-11-28T18:08:34.708 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-28T18:08:34.709 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ 2023-11-28T18:08:34.709 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/ 2023-11-28T18:08:34.709 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 166425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-28T18:08:34.709 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/adfs/ 2023-11-28T18:08:34.709 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 61073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/adfs/adfs.ko 2023-11-28T18:08:34.709 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/ 2023-11-28T18:08:34.709 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 163593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-28T18:08:34.709 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/afs/ 2023-11-28T18:08:34.710 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 932817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/afs/kafs.ko 2023-11-28T18:08:34.710 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/autofs/ 2023-11-28T18:08:34.710 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 101057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/autofs/autofs4.ko 2023-11-28T18:08:34.710 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/ 2023-11-28T18:08:34.710 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 64545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-28T18:08:34.710 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/bfs/ 2023-11-28T18:08:34.710 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 41257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/bfs/bfs.ko 2023-11-28T18:08:34.710 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 34585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-28T18:08:34.711 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/ 2023-11-28T18:08:34.711 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 3402729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-28T18:08:34.711 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/ 2023-11-28T18:08:34.711 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 306009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-28T18:08:34.711 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ 2023-11-28T18:08:34.712 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 1333017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-28T18:08:34.712 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/ 2023-11-28T18:08:34.713 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 100801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-28T18:08:34.713 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/configfs/ 2023-11-28T18:08:34.713 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 108393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/configfs/configfs.ko 2023-11-28T18:08:34.713 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/ 2023-11-28T18:08:34.713 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 29625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-28T18:08:34.713 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/ 2023-11-28T18:08:34.713 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 642513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-28T18:08:34.713 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/efs/ 2023-11-28T18:08:34.714 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 36649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/efs/efs.ko 2023-11-28T18:08:34.714 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/ 2023-11-28T18:08:34.714 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 1505025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-28T18:08:34.779 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/ 2023-11-28T18:08:34.780 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 27929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-28T18:08:34.780 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/freevxfs/ 2023-11-28T18:08:34.780 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 42545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/freevxfs/freevxfs.ko 2023-11-28T18:08:34.780 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/ 2023-11-28T18:08:34.780 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 226481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-28T18:08:34.780 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/ 2023-11-28T18:08:34.780 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-28T18:08:34.780 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/ 2023-11-28T18:08:34.780 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 1231529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-28T18:08:34.780 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/ 2023-11-28T18:08:34.780 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 141209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-28T18:08:34.781 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/ 2023-11-28T18:08:34.781 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 222817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-28T18:08:34.781 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hpfs/ 2023-11-28T18:08:34.781 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 192329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hpfs/hpfs.ko 2023-11-28T18:08:34.781 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/ 2023-11-28T18:08:34.781 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 74065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-28T18:08:34.781 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/ 2023-11-28T18:08:34.781 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 419497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-28T18:08:34.781 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/ 2023-11-28T18:08:34.781 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 280313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-28T18:08:34.781 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/ 2023-11-28T18:08:34.781 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 89193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-28T18:08:34.782 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/ 2023-11-28T18:08:34.782 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 124649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-28T18:08:34.782 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/ 2023-11-28T18:08:34.782 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/ 2023-11-28T18:08:34.782 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 81449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-28T18:08:34.782 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/ 2023-11-28T18:08:34.782 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 59449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-28T18:08:34.782 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/ 2023-11-28T18:08:34.782 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 117465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-28T18:08:34.782 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 1055721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-28T18:08:34.782 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 66441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv2.ko 2023-11-28T18:08:34.782 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 120177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-28T18:08:34.783 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 2042137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-28T18:08:34.783 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/ 2023-11-28T18:08:34.783 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-28T18:08:34.783 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-28T18:08:34.783 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/ 2023-11-28T18:08:34.783 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 1516873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-28T18:08:34.783 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/ 2023-11-28T18:08:34.783 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 592833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-28T18:08:34.784 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/ 2023-11-28T18:08:34.784 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-28T18:08:34.784 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-28T18:08:34.784 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-28T18:08:34.784 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-28T18:08:34.784 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-28T18:08:34.784 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-28T18:08:34.784 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-28T18:08:34.784 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-28T18:08:34.784 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-28T18:08:34.784 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-28T18:08:34.784 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-28T18:08:34.785 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ascii.ko 2023-11-28T18:08:34.785 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-28T18:08:34.785 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-28T18:08:34.785 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-28T18:08:34.785 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-28T18:08:34.785 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-28T18:08:34.785 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-28T18:08:34.785 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-28T18:08:34.785 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-28T18:08:34.785 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-28T18:08:34.785 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-28T18:08:34.785 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-28T18:08:34.786 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-28T18:08:34.786 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-28T18:08:34.786 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-28T18:08:34.786 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-28T18:08:34.786 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-28T18:08:34.786 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-28T18:08:34.786 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-28T18:08:34.786 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 94137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-28T18:08:34.786 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 142673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-28T18:08:34.786 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 165745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-28T18:08:34.787 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 118121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-28T18:08:34.787 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-28T18:08:34.787 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-28T18:08:34.787 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-28T18:08:34.787 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-28T18:08:34.787 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-28T18:08:34.787 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-28T18:08:34.787 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-28T18:08:34.788 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-28T18:08:34.788 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-28T18:08:34.788 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-28T18:08:34.788 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-28T18:08:34.788 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-28T18:08:34.788 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-28T18:08:34.788 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-28T18:08:34.788 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-28T18:08:34.788 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-28T18:08:34.788 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-28T18:08:34.789 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ntfs/ 2023-11-28T18:08:34.789 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 219417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ntfs/ntfs.ko 2023-11-28T18:08:34.789 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ 2023-11-28T18:08:34.789 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ 2023-11-28T18:08:34.789 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 244409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-28T18:08:34.790 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ 2023-11-28T18:08:34.790 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 440737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-28T18:08:34.790 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ 2023-11-28T18:08:34.790 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 54049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-28T18:08:34.791 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 2592433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-28T18:08:34.791 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-28T18:08:34.791 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 31729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-28T18:08:34.791 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 35897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-28T18:08:34.791 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/omfs/ 2023-11-28T18:08:34.791 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 53433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/omfs/omfs.ko 2023-11-28T18:08:34.791 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/ 2023-11-28T18:08:34.791 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 352161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-28T18:08:34.791 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ 2023-11-28T18:08:34.791 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 54809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-28T18:08:34.791 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx4/ 2023-11-28T18:08:34.792 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx4/qnx4.ko 2023-11-28T18:08:34.792 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx6/ 2023-11-28T18:08:34.792 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 46689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx6/qnx6.ko 2023-11-28T18:08:34.793 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/ 2023-11-28T18:08:34.793 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/quota_tree.ko 2023-11-28T18:08:34.793 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/quota_v1.ko 2023-11-28T18:08:34.793 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/quota_v2.ko 2023-11-28T18:08:34.793 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/ 2023-11-28T18:08:34.793 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 532673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-28T18:08:34.793 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/ 2023-11-28T18:08:34.793 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-28T18:08:34.793 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/ 2023-11-28T18:08:34.793 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/ 2023-11-28T18:08:34.793 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 2668401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-28T18:08:34.793 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/ 2023-11-28T18:08:34.794 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-28T18:08:34.794 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-28T18:08:34.794 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/ 2023-11-28T18:08:34.794 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 134641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-28T18:08:34.794 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/ 2023-11-28T18:08:34.794 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 90721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-28T18:08:34.794 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/ 2023-11-28T18:08:34.794 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 247737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-28T18:08:34.794 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ 2023-11-28T18:08:34.794 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 162497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-28T18:08:34.794 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/ 2023-11-28T18:08:34.795 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 4011017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-28T18:08:34.795 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/ 2023-11-28T18:08:34.795 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/locking/ 2023-11-28T18:08:34.795 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 75489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/locking/locktorture.ko 2023-11-28T18:08:34.795 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/time/ 2023-11-28T18:08:34.795 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/time/test_udelay.ko 2023-11-28T18:08:34.795 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 60553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-28T18:08:34.795 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ 2023-11-28T18:08:34.795 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 6833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-28T18:08:34.795 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 6529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc7.ko 2023-11-28T18:08:34.795 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-28T18:08:34.795 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/ 2023-11-28T18:08:34.796 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-28T18:08:34.796 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 37081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-28T18:08:34.796 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/interval_tree_test.ko 2023-11-28T18:08:34.796 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-28T18:08:34.796 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-28T18:08:34.796 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/ 2023-11-28T18:08:34.796 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 33801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-28T18:08:34.797 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-28T18:08:34.797 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/ 2023-11-28T18:08:34.797 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 6617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-28T18:08:34.797 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/memory-notifier-error-inject.ko 2023-11-28T18:08:34.797 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/notifier-error-inject.ko 2023-11-28T18:08:34.797 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/percpu_test.ko 2023-11-28T18:08:34.797 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/pm-notifier-error-inject.ko 2023-11-28T18:08:34.797 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/ 2023-11-28T18:08:34.797 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 143433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-28T18:08:34.798 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/rbtree_test.ko 2023-11-28T18:08:34.798 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/ 2023-11-28T18:08:34.798 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-28T18:08:34.798 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 38841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test-kstrtox.ko 2023-11-28T18:08:34.798 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test-string_helpers.ko 2023-11-28T18:08:34.798 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 4760425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_bpf.ko 2023-11-28T18:08:34.798 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 49089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_firmware.ko 2023-11-28T18:08:34.798 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 6369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_module.ko 2023-11-28T18:08:34.798 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_user_copy.ko 2023-11-28T18:08:34.798 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-28T18:08:34.798 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-28T18:08:34.799 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-28T18:08:34.799 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/xz/ 2023-11-28T18:08:34.799 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/xz/xz_dec_test.ko 2023-11-28T18:08:34.799 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/mm/ 2023-11-28T18:08:34.799 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-28T18:08:34.799 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ 2023-11-28T18:08:34.799 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/ 2023-11-28T18:08:34.799 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 55737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-28T18:08:34.799 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-28T18:08:34.799 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-28T18:08:34.799 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-28T18:08:34.799 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 6985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-28T18:08:34.800 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-28T18:08:34.800 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-28T18:08:34.800 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-28T18:08:34.800 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/ 2023-11-28T18:08:34.800 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-28T18:08:34.800 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 34809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-28T18:08:34.800 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-28T18:08:34.800 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-28T18:08:34.801 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-28T18:08:34.801 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/ 2023-11-28T18:08:34.801 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 81737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-28T18:08:34.801 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/ 2023-11-28T18:08:34.801 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 198337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-28T18:08:34.801 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 35977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-28T18:08:34.801 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-28T18:08:34.801 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 32337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-28T18:08:34.801 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/ 2023-11-28T18:08:34.801 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 649297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-28T18:08:34.801 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/ 2023-11-28T18:08:34.802 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 63809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-28T18:08:34.802 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 777689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-28T18:08:34.802 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ 2023-11-28T18:08:34.802 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-28T18:08:34.803 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-28T18:08:34.803 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-28T18:08:34.803 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-28T18:08:34.803 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-28T18:08:34.803 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-28T18:08:34.803 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-28T18:08:34.803 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-28T18:08:34.803 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-28T18:08:34.803 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-28T18:08:34.803 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-28T18:08:34.803 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-28T18:08:34.804 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-28T18:08:34.804 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-28T18:08:34.804 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-28T18:08:34.804 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-28T18:08:34.804 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-28T18:08:34.804 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-28T18:08:34.804 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-28T18:08:34.804 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-28T18:08:34.804 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 55929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-28T18:08:34.804 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-28T18:08:34.804 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-28T18:08:34.804 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/ 2023-11-28T18:08:34.805 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 1046833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-28T18:08:34.805 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/ 2023-11-28T18:08:34.805 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 125993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-28T18:08:34.805 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/ 2023-11-28T18:08:34.805 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 194209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-28T18:08:34.805 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-28T18:08:34.805 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 57881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-28T18:08:34.805 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 60337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-28T18:08:34.805 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/ 2023-11-28T18:08:34.805 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 390473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-28T18:08:34.805 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-28T18:08:34.805 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-28T18:08:34.806 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-28T18:08:34.806 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-28T18:08:34.806 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/hsr/ 2023-11-28T18:08:34.806 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 99729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/hsr/hsr.ko 2023-11-28T18:08:34.806 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ 2023-11-28T18:08:34.806 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ 2023-11-28T18:08:34.806 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 45233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-28T18:08:34.806 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 254625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-28T18:08:34.807 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 47289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-28T18:08:34.807 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ 2023-11-28T18:08:34.807 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-28T18:08:34.807 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 38905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-28T18:08:34.807 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 48337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-28T18:08:34.807 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-28T18:08:34.807 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 48657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-28T18:08:34.807 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 60633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-28T18:08:34.808 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 61225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-28T18:08:34.808 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 34505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-28T18:08:34.808 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-28T18:08:34.808 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-28T18:08:34.808 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ 2023-11-28T18:08:34.808 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 37641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-28T18:08:34.808 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-28T18:08:34.808 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-28T18:08:34.809 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 41609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ip_tables.ko 2023-11-28T18:08:34.809 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-28T18:08:34.809 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-11-28T18:08:34.809 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-28T18:08:34.809 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-28T18:08:34.809 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-28T18:08:34.809 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-11-28T18:08:34.809 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-28T18:08:34.809 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-28T18:08:34.809 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-28T18:08:34.809 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-28T18:08:34.809 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-28T18:08:34.810 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-28T18:08:34.810 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 27625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-28T18:08:34.810 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-28T18:08:34.810 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-28T18:08:34.810 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-11-28T18:08:34.810 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-28T18:08:34.810 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-28T18:08:34.810 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-28T18:08:34.810 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-28T18:08:34.810 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-28T18:08:34.811 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-28T18:08:34.811 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-28T18:08:34.811 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-28T18:08:34.811 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-28T18:08:34.811 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-28T18:08:34.811 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-28T18:08:34.811 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-28T18:08:34.811 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-28T18:08:34.811 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-28T18:08:34.811 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-28T18:08:34.812 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-28T18:08:34.812 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-28T18:08:34.812 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-28T18:08:34.812 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-28T18:08:34.812 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-28T18:08:34.812 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 41665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-28T18:08:34.812 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-28T18:08:34.812 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ 2023-11-28T18:08:34.812 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-28T18:08:34.812 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 39537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-28T18:08:34.812 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-28T18:08:34.813 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 80897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_gre.ko 2023-11-28T18:08:34.813 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 93577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-28T18:08:34.813 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-28T18:08:34.814 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 53273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-28T18:08:34.814 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-28T18:08:34.814 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/mip6.ko 2023-11-28T18:08:34.814 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ 2023-11-28T18:08:34.814 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 42169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-28T18:08:34.814 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-11-28T18:08:34.814 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-28T18:08:34.814 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-28T18:08:34.814 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-28T18:08:34.815 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-28T18:08:34.815 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-28T18:08:34.815 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-28T18:08:34.815 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-28T18:08:34.815 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-28T18:08:34.815 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-28T18:08:34.815 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-28T18:08:34.815 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-28T18:08:34.815 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-28T18:08:34.815 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-28T18:08:34.815 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-28T18:08:34.816 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-28T18:08:34.816 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 39113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-28T18:08:34.816 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-28T18:08:34.816 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-28T18:08:34.816 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-28T18:08:34.816 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-28T18:08:34.816 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-28T18:08:34.816 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-28T18:08:34.816 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-28T18:08:34.816 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 74073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-28T18:08:34.816 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-28T18:08:34.816 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-28T18:08:34.817 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/ 2023-11-28T18:08:34.817 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 82833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-28T18:08:34.817 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/ 2023-11-28T18:08:34.817 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 125281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-28T18:08:34.817 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-28T18:08:34.817 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 25953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-28T18:08:34.817 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 35569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-28T18:08:34.818 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 36713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-28T18:08:34.818 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 33001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-28T18:08:34.818 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 53241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-28T18:08:34.818 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/lapb/ 2023-11-28T18:08:34.818 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 47977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/lapb/lapb.ko 2023-11-28T18:08:34.818 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/ 2023-11-28T18:08:34.818 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-28T18:08:34.818 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 267913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc2.ko 2023-11-28T18:08:34.818 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/ 2023-11-28T18:08:34.818 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 290417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-28T18:08:34.819 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/ 2023-11-28T18:08:34.819 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-28T18:08:34.819 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ 2023-11-28T18:08:34.819 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ 2023-11-28T18:08:34.819 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 119585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-28T18:08:34.819 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-28T18:08:34.819 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-28T18:08:34.819 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-28T18:08:34.819 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 100297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-28T18:08:34.819 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 101441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-28T18:08:34.819 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 102817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-28T18:08:34.820 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 103417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-28T18:08:34.820 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 112849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-28T18:08:34.820 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 60329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-28T18:08:34.820 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 109593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-28T18:08:34.820 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 113129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-28T18:08:34.820 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 112585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-28T18:08:34.820 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 111585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-28T18:08:34.820 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 114465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-28T18:08:34.820 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-28T18:08:34.820 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ 2023-11-28T18:08:34.820 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 411729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-28T18:08:34.821 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-28T18:08:34.821 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-28T18:08:34.821 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-28T18:08:34.821 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-28T18:08:34.822 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-28T18:08:34.822 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-28T18:08:34.822 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-28T18:08:34.822 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-28T18:08:34.822 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-28T18:08:34.822 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-28T18:08:34.822 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-28T18:08:34.822 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-28T18:08:34.822 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-28T18:08:34.822 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 49257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-28T18:08:34.823 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 408257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-28T18:08:34.823 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-28T18:08:34.824 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-28T18:08:34.824 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 33761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-28T18:08:34.824 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 118145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-28T18:08:34.824 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-28T18:08:34.824 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-28T18:08:34.824 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 107233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-28T18:08:34.824 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 33633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-28T18:08:34.824 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-28T18:08:34.824 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 66641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-28T18:08:34.824 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-28T18:08:34.825 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-28T18:08:34.825 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-28T18:08:34.825 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 89425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-28T18:08:34.825 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-28T18:08:34.825 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 37129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-28T18:08:34.825 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 95353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-28T18:08:34.825 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-28T18:08:34.825 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-28T18:08:34.825 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-28T18:08:34.825 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-28T18:08:34.826 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-28T18:08:34.826 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 44065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-28T18:08:34.826 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 615649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-28T18:08:34.826 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 42561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-28T18:08:34.826 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-28T18:08:34.826 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-11-28T18:08:34.827 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 39785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-28T18:08:34.827 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-28T18:08:34.827 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 49377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-28T18:08:34.827 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-28T18:08:34.827 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 33209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-28T18:08:34.827 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 40721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-28T18:08:34.827 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-28T18:08:34.828 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-28T18:08:34.828 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-28T18:08:34.828 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-28T18:08:34.828 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-28T18:08:34.828 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-28T18:08:34.828 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-28T18:08:34.828 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-28T18:08:34.828 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-28T18:08:34.828 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_osf.ko 2023-11-28T18:08:34.828 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-28T18:08:34.828 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-28T18:08:34.829 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-28T18:08:34.829 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-28T18:08:34.829 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-28T18:08:34.829 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_synproxy.ko 2023-11-28T18:08:34.829 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tproxy.ko 2023-11-28T18:08:34.829 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 89313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/x_tables.ko 2023-11-28T18:08:34.829 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-28T18:08:34.829 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-28T18:08:34.829 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-28T18:08:34.829 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-28T18:08:34.829 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 18169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-28T18:08:34.829 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-28T18:08:34.830 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-28T18:08:34.830 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-28T18:08:34.830 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 38873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-28T18:08:34.830 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-28T18:08:34.830 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-28T18:08:34.830 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-28T18:08:34.830 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-28T18:08:34.830 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-28T18:08:34.830 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-28T18:08:34.830 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-28T18:08:34.830 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-28T18:08:34.831 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 14337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-28T18:08:34.831 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-28T18:08:34.831 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-28T18:08:34.831 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-28T18:08:34.831 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-28T18:08:34.831 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-28T18:08:34.831 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-28T18:08:34.831 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-28T18:08:34.832 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-28T18:08:34.832 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-28T18:08:34.832 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-28T18:08:34.832 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-28T18:08:34.832 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-28T18:08:34.832 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-28T18:08:34.832 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-28T18:08:34.832 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-28T18:08:34.832 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-28T18:08:34.832 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-28T18:08:34.832 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-28T18:08:34.833 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-28T18:08:34.833 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-28T18:08:34.833 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-28T18:08:34.833 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 43289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-28T18:08:34.833 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-28T18:08:34.833 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-28T18:08:34.833 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-28T18:08:34.833 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-28T18:08:34.833 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-28T18:08:34.833 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-28T18:08:34.834 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-28T18:08:34.834 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-28T18:08:34.834 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-28T18:08:34.834 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-28T18:08:34.835 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-28T18:08:34.835 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-28T18:08:34.835 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-28T18:08:34.835 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-28T18:08:34.835 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-28T18:08:34.835 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-28T18:08:34.835 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-28T18:08:34.835 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-28T18:08:34.835 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-28T18:08:34.835 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-28T18:08:34.835 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-28T18:08:34.836 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 36433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-28T18:08:34.836 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-28T18:08:34.836 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-28T18:08:34.836 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 16089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-28T18:08:34.836 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-28T18:08:34.836 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-28T18:08:34.836 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-28T18:08:34.837 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 8553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-28T18:08:34.837 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpudp.ko 2023-11-28T18:08:34.837 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-28T18:08:34.837 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-28T18:08:34.837 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/ 2023-11-28T18:08:34.837 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-28T18:08:34.837 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/ 2023-11-28T18:08:34.837 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 12425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-28T18:08:34.838 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/ 2023-11-28T18:08:34.838 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 380033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-28T18:08:34.838 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-28T18:08:34.838 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-28T18:08:34.838 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 11793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-28T18:08:34.838 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/ 2023-11-28T18:08:34.838 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-28T18:08:34.838 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/phonet/ 2023-11-28T18:08:34.839 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 110201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/phonet/phonet.ko 2023-11-28T18:08:34.839 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 54721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/phonet/pn_pep.ko 2023-11-28T18:08:34.839 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/ 2023-11-28T18:08:34.839 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 251481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-28T18:08:34.839 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 164937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-28T18:08:34.839 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 67377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-28T18:08:34.839 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rxrpc/ 2023-11-28T18:08:34.839 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 819657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rxrpc/rxrpc.ko 2023-11-28T18:08:34.839 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/ 2023-11-28T18:08:34.839 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 32241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-28T18:08:34.839 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-28T18:08:34.839 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-28T18:08:34.840 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 34097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-28T18:08:34.840 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-28T18:08:34.840 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 43449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-28T18:08:34.840 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 31961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-28T18:08:34.840 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-28T18:08:34.840 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 35297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-28T18:08:34.840 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 36665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-28T18:08:34.840 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-28T18:08:34.841 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 33961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-28T18:08:34.841 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_cgroup.ko 2023-11-28T18:08:34.841 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 31401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-28T18:08:34.841 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 31769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-28T18:08:34.841 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 38425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-28T18:08:34.841 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 61409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-28T18:08:34.841 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-28T18:08:34.841 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 13409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-28T18:08:34.841 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 33289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-28T18:08:34.841 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-28T18:08:34.841 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-28T18:08:34.842 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-28T18:08:34.842 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 21649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-28T18:08:34.842 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-28T18:08:34.842 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 34633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-28T18:08:34.842 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 33609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-28T18:08:34.842 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 31617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq_codel.ko 2023-11-28T18:08:34.842 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 35433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-28T18:08:34.842 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 49817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-28T18:08:34.843 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-28T18:08:34.843 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 66393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-28T18:08:34.843 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-28T18:08:34.843 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 30273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-28T18:08:34.843 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-28T18:08:34.843 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-28T18:08:34.843 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 36137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-28T18:08:34.843 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 27497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-28T18:08:34.843 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-28T18:08:34.844 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 24729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-28T18:08:34.844 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 48729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-28T18:08:34.844 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 32905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-28T18:08:34.845 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 26881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-28T18:08:34.845 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 31353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-28T18:08:34.845 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-28T18:08:34.845 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 28105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-28T18:08:34.845 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/ 2023-11-28T18:08:34.846 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 929745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-28T18:08:34.846 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 19033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-28T18:08:34.846 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/ 2023-11-28T18:08:34.846 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/ 2023-11-28T18:08:34.846 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 336529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-28T18:08:34.846 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 54737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-28T18:08:34.853 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 1520097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-28T18:08:34.853 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/ 2023-11-28T18:08:34.854 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 707657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-28T18:08:34.854 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/ 2023-11-28T18:08:34.854 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-28T18:08:34.854 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 862665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-28T18:08:34.854 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/ 2023-11-28T18:08:34.854 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-28T18:08:34.854 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/ 2023-11-28T18:08:34.854 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 85409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-28T18:08:34.854 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 68049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-28T18:08:34.854 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 96193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-28T18:08:34.855 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 10705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-28T18:08:34.855 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 17521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-28T18:08:34.855 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/x25/ 2023-11-28T18:08:34.855 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 125177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/x25/x25.ko 2023-11-28T18:08:34.855 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/ 2023-11-28T18:08:34.855 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 22657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_algo.ko 2023-11-28T18:08:34.855 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 20649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-28T18:08:34.855 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 102257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_user.ko 2023-11-28T18:08:34.855 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/virt/ 2023-11-28T18:08:34.855 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/ 2023-11-28T18:08:34.855 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 15537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-28T18:08:34.856 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 640081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-28T18:08:34.856 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 624107 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-28T18:08:34.856 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 7663 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-28T18:08:34.856 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 23428 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin.alias.bin 2023-11-28T18:08:34.856 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 9821 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-28T18:08:34.856 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 70004 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-28T18:08:34.856 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 146313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-28T18:08:34.856 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 232548 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-28T18:08:34.856 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 210 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-28T18:08:34.856 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 72606 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-28T18:08:34.856 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 879 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-28T18:08:34.857 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 254438 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-28T18:08:34.857 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 309363 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-28T18:08:34.857 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/ 2023-11-28T18:08:34.857 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/share/ 2023-11-28T18:08:34.857 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/share/doc/ 2023-11-28T18:08:34.857 INFO:teuthology.orchestra.run.smithi105.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/share/doc/linux-image-6.6.0-g8247bbb5b470/ 2023-11-28T18:08:34.857 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 172 2023-11-23 03:47 ./usr/share/doc/linux-image-6.6.0-g8247bbb5b470/changelog.Debian.gz 2023-11-28T18:08:34.857 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r-- root/root 691 2023-11-23 03:47 ./usr/share/doc/linux-image-6.6.0-g8247bbb5b470/copyright 2023-11-28T18:08:34.858 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-28T18:08:34.858 INFO:teuthology.task.kernel:searching for kernel 6.6.0-g8247bbb5b470 2023-11-28T18:08:34.859 DEBUG:teuthology.orchestra.run.smithi105:> egrep '(submenu|menuentry.*6.6.0-g8247bbb5b470).*{' /boot/grub/grub.cfg 2023-11-28T18:08:34.866 INFO:teuthology.orchestra.run.smithi105.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-11-28T18:08:34.866 INFO:teuthology.orchestra.run.smithi105.stdout: menuentry 'Ubuntu, with Linux 6.6.0-g8247bbb5b470' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-g8247bbb5b470-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-11-28T18:08:34.866 INFO:teuthology.orchestra.run.smithi105.stdout: menuentry 'Ubuntu, with Linux 6.6.0-g8247bbb5b470 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-g8247bbb5b470-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-11-28T18:08:34.869 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-11-28T18:08:34.869 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-g8247bbb5b470 2023-11-28T18:08:34.870 DEBUG:teuthology.orchestra.run.smithi105:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-11-28T18:08:34.911 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi105.front.sepia.ceph.com to complete... 2023-11-28T18:08:35.382 INFO:teuthology.orchestra.run.smithi105.stderr:Sourcing file `/etc/default/grub' 2023-11-28T18:08:35.383 INFO:teuthology.orchestra.run.smithi105.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-28T18:08:35.399 INFO:teuthology.orchestra.run.smithi105.stderr:Generating grub configuration file ... 2023-11-28T18:08:35.566 INFO:teuthology.orchestra.run.smithi105.stderr:Found linux image: /boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-28T18:08:35.572 INFO:teuthology.orchestra.run.smithi105.stderr:Found initrd image: /boot/initrd.img-6.6.0-g8247bbb5b470 2023-11-28T18:08:35.788 INFO:teuthology.orchestra.run.smithi105.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-11-28T18:08:35.793 INFO:teuthology.orchestra.run.smithi105.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-11-28T18:08:35.835 INFO:teuthology.orchestra.run.smithi105.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-11-28T18:08:35.840 INFO:teuthology.orchestra.run.smithi105.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-11-28T18:08:35.975 INFO:teuthology.orchestra.run.smithi105.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-11-28T18:08:35.975 INFO:teuthology.orchestra.run.smithi105.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-11-28T18:08:35.975 INFO:teuthology.orchestra.run.smithi105.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-11-28T18:08:36.000 INFO:teuthology.orchestra.run.smithi105.stderr:done 2023-11-28T18:08:59.514 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-11-28T18:08:59.515 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-11-28T18:08:59.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T18:09:07.114 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-11-28T18:09:07.115 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2023-11-28T18:09:07.115 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T18:09:34.050 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-11-28T18:09:41.734 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-11-28T18:09:43.053 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-11-28T18:09:43.054 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T18:09:50.742 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2023-11-28T18:09:50.743 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T18:10:01.442 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-11-28T18:10:09.126 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-11-28T18:10:13.446 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-11-28T18:10:13.447 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T18:10:16.514 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-11-28T18:10:21.130 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2023-11-28T18:10:21.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T18:10:21.132 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-11-28T18:10:31.522 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-11-28T18:10:31.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T18:10:31.837 DEBUG:teuthology.orchestra.run.smithi148:> true 2023-11-28T18:10:32.510 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-11-28T18:10:32.511 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.6.0-g8247bbb5b470"... 2023-11-28T18:10:32.511 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2023-11-28T18:10:32.565 INFO:teuthology.orchestra.run.smithi148.stdout:6.6.0-g8247bbb5b470 2023-11-28T18:10:32.565 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-28T18:10:32.565 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-28T18:10:32.565 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-11-28T18:10:33.567 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-11-28T18:10:33.568 DEBUG:teuthology.orchestra.run.smithi148:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-28T18:10:33.664 INFO:teuthology.orchestra.run.smithi148.stdout:ttyS1 2023-11-28T18:10:33.720 DEBUG:teuthology.parallel:result is None 2023-11-28T18:10:36.134 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2023-11-28T18:10:36.134 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T18:10:36.424 DEBUG:teuthology.orchestra.run.smithi105:> true 2023-11-28T18:10:37.053 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2023-11-28T18:10:37.053 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.6.0-g8247bbb5b470"... 2023-11-28T18:10:37.053 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2023-11-28T18:10:37.103 INFO:teuthology.orchestra.run.smithi105.stdout:6.6.0-g8247bbb5b470 2023-11-28T18:10:37.103 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-28T18:10:37.103 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-28T18:10:37.103 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-11-28T18:10:38.105 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-11-28T18:10:38.105 DEBUG:teuthology.orchestra.run.smithi105:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-28T18:10:38.201 INFO:teuthology.orchestra.run.smithi105.stdout:ttyS1 2023-11-28T18:10:38.267 DEBUG:teuthology.parallel:result is None 2023-11-28T18:10:38.267 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-28T18:10:38.275 INFO:teuthology.task.internal:Creating test directory... 2023-11-28T18:10:38.275 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T18:10:38.279 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T18:10:38.289 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-28T18:10:38.295 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-28T18:10:38.302 INFO:teuthology.task.internal:Creating archive directory... 2023-11-28T18:10:38.302 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T18:10:38.335 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T18:10:38.364 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-28T18:10:38.371 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-28T18:10:38.372 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T18:10:38.387 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T18:10:38.408 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T18:10:38.421 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T18:10:38.426 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T18:10:38.440 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T18:10:38.442 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-28T18:10:38.449 INFO:teuthology.task.internal:Configuring sudo... 2023-11-28T18:10:38.449 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T18:10:38.471 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T18:10:38.510 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-28T18:10:38.519 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-28T18:10:38.519 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T18:10:38.535 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T18:10:38.561 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T18:10:38.592 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T18:10:38.644 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:10:38.645 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T18:10:38.716 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T18:10:38.725 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T18:10:38.778 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:10:38.778 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T18:10:38.849 DEBUG:teuthology.orchestra.run.smithi105:> sudo service rsyslog restart 2023-11-28T18:10:38.855 DEBUG:teuthology.orchestra.run.smithi148:> sudo service rsyslog restart 2023-11-28T18:10:38.979 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-28T18:10:38.987 INFO:teuthology.task.internal:Starting timer... 2023-11-28T18:10:38.987 INFO:teuthology.run_tasks:Running task pcp... 2023-11-28T18:10:38.999 INFO:teuthology.run_tasks:Running task selinux... 2023-11-28T18:10:39.008 DEBUG:teuthology.task.selinux:Excluding smithi105: OS 'ubuntu' does not support SELinux 2023-11-28T18:10:39.008 DEBUG:teuthology.task.selinux:Excluding smithi148: OS 'ubuntu' does not support SELinux 2023-11-28T18:10:39.009 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-28T18:10:39.009 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-11-28T18:10:39.010 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-28T18:10:39.010 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-28T18:10:39.019 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-28T18:10:39.019 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-28T18:10:39.040 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-28T18:10:39.041 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi105.front.sepia.ceph.com,smithi148.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-28T18:19:28.238 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi105.front.sepia.ceph.com'), Remote(name='ubuntu@smithi148.front.sepia.ceph.com')] 2023-11-28T18:19:28.239 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2023-11-28T18:19:28.240 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T18:19:28.322 DEBUG:teuthology.orchestra.run.smithi105:> true 2023-11-28T18:19:28.457 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2023-11-28T18:19:28.458 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-11-28T18:19:28.458 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T18:19:28.536 DEBUG:teuthology.orchestra.run.smithi148:> true 2023-11-28T18:19:28.677 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-11-28T18:19:28.677 INFO:teuthology.run_tasks:Running task clock... 2023-11-28T18:19:28.689 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-28T18:19:28.689 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T18:19:28.690 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T18:19:28.695 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T18:19:28.695 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T18:19:28.731 INFO:teuthology.orchestra.run.smithi105.stdout:28 Nov 18:19:28 ntpd[27993]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-11-28T18:19:28.731 INFO:teuthology.orchestra.run.smithi105.stdout:28 Nov 18:19:28 ntpd[27993]: Command line: ntpd -gq 2023-11-28T18:19:28.731 INFO:teuthology.orchestra.run.smithi105.stdout:28 Nov 18:19:28 ntpd[27993]: ---------------------------------------------------- 2023-11-28T18:19:28.731 INFO:teuthology.orchestra.run.smithi105.stdout:28 Nov 18:19:28 ntpd[27993]: ntp-4 is maintained by Network Time Foundation, 2023-11-28T18:19:28.731 INFO:teuthology.orchestra.run.smithi105.stdout:28 Nov 18:19:28 ntpd[27993]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-11-28T18:19:28.731 INFO:teuthology.orchestra.run.smithi105.stdout:28 Nov 18:19:28 ntpd[27993]: corporation. Support and training for ntp-4 are 2023-11-28T18:19:28.732 INFO:teuthology.orchestra.run.smithi105.stdout:28 Nov 18:19:28 ntpd[27993]: available at https://www.nwtime.org/support 2023-11-28T18:19:28.732 INFO:teuthology.orchestra.run.smithi105.stdout:28 Nov 18:19:28 ntpd[27993]: ---------------------------------------------------- 2023-11-28T18:19:28.732 INFO:teuthology.orchestra.run.smithi105.stdout:28 Nov 18:19:28 ntpd[27993]: proto: precision = 0.042 usec (-24) 2023-11-28T18:19:28.732 INFO:teuthology.orchestra.run.smithi105.stdout:28 Nov 18:19:28 ntpd[27993]: basedate set to 2022-02-04 2023-11-28T18:19:28.732 INFO:teuthology.orchestra.run.smithi105.stdout:28 Nov 18:19:28 ntpd[27993]: gps base set to 2022-02-06 (week 2196) 2023-11-28T18:19:28.733 INFO:teuthology.orchestra.run.smithi105.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-28T18:19:28.733 INFO:teuthology.orchestra.run.smithi105.stdout:28 Nov 18:19:28 ntpd[27993]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-28T18:19:28.733 INFO:teuthology.orchestra.run.smithi105.stdout:28 Nov 18:19:28 ntpd[27993]: restrict ::: KOD does nothing without LIMITED. 2023-11-28T18:19:28.734 INFO:teuthology.orchestra.run.smithi105.stderr:restrict ::: KOD does nothing without LIMITED. 2023-11-28T18:19:28.734 INFO:teuthology.orchestra.run.smithi105.stdout:28 Nov 18:19:28 ntpd[27993]: Listen and drop on 0 v6wildcard [::]:123 2023-11-28T18:19:28.735 INFO:teuthology.orchestra.run.smithi105.stdout:28 Nov 18:19:28 ntpd[27993]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-11-28T18:19:28.735 INFO:teuthology.orchestra.run.smithi105.stdout:28 Nov 18:19:28 ntpd[27993]: Listen normally on 2 lo 127.0.0.1:123 2023-11-28T18:19:28.735 INFO:teuthology.orchestra.run.smithi105.stdout:28 Nov 18:19:28 ntpd[27993]: Listen normally on 3 enp3s0f1 172.21.15.105:123 2023-11-28T18:19:28.735 INFO:teuthology.orchestra.run.smithi105.stdout:28 Nov 18:19:28 ntpd[27993]: Listen normally on 4 lo [::1]:123 2023-11-28T18:19:28.735 INFO:teuthology.orchestra.run.smithi105.stdout:28 Nov 18:19:28 ntpd[27993]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d429%5]:123 2023-11-28T18:19:28.735 INFO:teuthology.orchestra.run.smithi105.stdout:28 Nov 18:19:28 ntpd[27993]: Listening on routing socket on fd #22 for interface updates 2023-11-28T18:19:28.756 INFO:teuthology.orchestra.run.smithi148.stdout:28 Nov 18:19:28 ntpd[27995]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-11-28T18:19:28.756 INFO:teuthology.orchestra.run.smithi148.stdout:28 Nov 18:19:28 ntpd[27995]: Command line: ntpd -gq 2023-11-28T18:19:28.756 INFO:teuthology.orchestra.run.smithi148.stdout:28 Nov 18:19:28 ntpd[27995]: ---------------------------------------------------- 2023-11-28T18:19:28.756 INFO:teuthology.orchestra.run.smithi148.stdout:28 Nov 18:19:28 ntpd[27995]: ntp-4 is maintained by Network Time Foundation, 2023-11-28T18:19:28.756 INFO:teuthology.orchestra.run.smithi148.stdout:28 Nov 18:19:28 ntpd[27995]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-11-28T18:19:28.756 INFO:teuthology.orchestra.run.smithi148.stdout:28 Nov 18:19:28 ntpd[27995]: corporation. Support and training for ntp-4 are 2023-11-28T18:19:28.756 INFO:teuthology.orchestra.run.smithi148.stdout:28 Nov 18:19:28 ntpd[27995]: available at https://www.nwtime.org/support 2023-11-28T18:19:28.756 INFO:teuthology.orchestra.run.smithi148.stdout:28 Nov 18:19:28 ntpd[27995]: ---------------------------------------------------- 2023-11-28T18:19:28.757 INFO:teuthology.orchestra.run.smithi148.stdout:28 Nov 18:19:28 ntpd[27995]: proto: precision = 0.041 usec (-24) 2023-11-28T18:19:28.757 INFO:teuthology.orchestra.run.smithi148.stdout:28 Nov 18:19:28 ntpd[27995]: basedate set to 2022-02-04 2023-11-28T18:19:28.757 INFO:teuthology.orchestra.run.smithi148.stdout:28 Nov 18:19:28 ntpd[27995]: gps base set to 2022-02-06 (week 2196) 2023-11-28T18:19:28.757 INFO:teuthology.orchestra.run.smithi148.stdout:28 Nov 18:19:28 ntpd[27995]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-28T18:19:28.758 INFO:teuthology.orchestra.run.smithi148.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-28T18:19:28.758 INFO:teuthology.orchestra.run.smithi148.stderr:restrict ::: KOD does nothing without LIMITED. 2023-11-28T18:19:28.759 INFO:teuthology.orchestra.run.smithi148.stdout:28 Nov 18:19:28 ntpd[27995]: restrict ::: KOD does nothing without LIMITED. 2023-11-28T18:19:28.759 INFO:teuthology.orchestra.run.smithi148.stdout:28 Nov 18:19:28 ntpd[27995]: Listen and drop on 0 v6wildcard [::]:123 2023-11-28T18:19:28.759 INFO:teuthology.orchestra.run.smithi148.stdout:28 Nov 18:19:28 ntpd[27995]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-11-28T18:19:28.759 INFO:teuthology.orchestra.run.smithi148.stdout:28 Nov 18:19:28 ntpd[27995]: Listen normally on 2 lo 127.0.0.1:123 2023-11-28T18:19:28.759 INFO:teuthology.orchestra.run.smithi148.stdout:28 Nov 18:19:28 ntpd[27995]: Listen normally on 3 enp3s0f1 172.21.15.148:123 2023-11-28T18:19:28.759 INFO:teuthology.orchestra.run.smithi148.stdout:28 Nov 18:19:28 ntpd[27995]: Listen normally on 4 lo [::1]:123 2023-11-28T18:19:28.759 INFO:teuthology.orchestra.run.smithi148.stdout:28 Nov 18:19:28 ntpd[27995]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7257%5]:123 2023-11-28T18:19:28.760 INFO:teuthology.orchestra.run.smithi148.stdout:28 Nov 18:19:28 ntpd[27995]: Listening on routing socket on fd #22 for interface updates 2023-11-28T18:19:29.756 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:29 ntpd[27995]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:29.756 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:29 ntpd[27995]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:31.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:31 ntpd[27993]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:31.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:31 ntpd[27993]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:31.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:31 ntpd[27993]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:31.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:31 ntpd[27993]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:31.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:31 ntpd[27993]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:31.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:31 ntpd[27993]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:31.735 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:31 ntpd[27993]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T18:19:31.735 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:31 ntpd[27993]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T18:19:31.735 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:31 ntpd[27993]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:31.735 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:31 ntpd[27993]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:31.735 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:31 ntpd[27993]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T18:19:31.735 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:31 ntpd[27993]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T18:19:32.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:32 ntpd[27995]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:32.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:32 ntpd[27995]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:32.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:32 ntpd[27995]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:32.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:32 ntpd[27995]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:32.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:32 ntpd[27995]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:32.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:32 ntpd[27995]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:32.758 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:32 ntpd[27995]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T18:19:32.758 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:32 ntpd[27995]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T18:19:32.758 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:32 ntpd[27995]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:32.758 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:32 ntpd[27995]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:32.758 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:32 ntpd[27995]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T18:19:32.758 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:32 ntpd[27995]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T18:19:33.733 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:33 ntpd[27993]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:33.733 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:33 ntpd[27993]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:33.733 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:33 ntpd[27993]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:33.733 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:33 ntpd[27993]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:33.733 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:33 ntpd[27993]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:33.733 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:33 ntpd[27993]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:33.733 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:33 ntpd[27993]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:33.733 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:33 ntpd[27993]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:33.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:33 ntpd[27993]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:33.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:33 ntpd[27993]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:33.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:33 ntpd[27993]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T18:19:33.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:33 ntpd[27993]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T18:19:33.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:33 ntpd[27993]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:33.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:33 ntpd[27993]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:33.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:33 ntpd[27993]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T18:19:33.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:33 ntpd[27993]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T18:19:33.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:33 ntpd[27993]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:33.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:33 ntpd[27993]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:33.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:33 ntpd[27993]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T18:19:33.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:33 ntpd[27993]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T18:19:33.756 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:33 ntpd[27995]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:33.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:33 ntpd[27995]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:33.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:33 ntpd[27995]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:33.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:33 ntpd[27995]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:33.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:33 ntpd[27995]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T18:19:33.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:33 ntpd[27995]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T18:19:34.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:34 ntpd[27995]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:34.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:34 ntpd[27995]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:34.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:34 ntpd[27995]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:34.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:34 ntpd[27995]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:34.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:34 ntpd[27995]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:34.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:34 ntpd[27995]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:34.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:34 ntpd[27995]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T18:19:34.758 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:34 ntpd[27995]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T18:19:34.758 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:34 ntpd[27995]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:34.758 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:34 ntpd[27995]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:34.758 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:34 ntpd[27995]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T18:19:34.758 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:34 ntpd[27995]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T18:19:35.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:35 ntpd[27993]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:35.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:35 ntpd[27993]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:35.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:35 ntpd[27993]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:35.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:35 ntpd[27993]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:35.735 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:35 ntpd[27993]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:35.735 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:35 ntpd[27993]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:35.735 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:35 ntpd[27993]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:35.736 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:35 ntpd[27993]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:35.736 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:35 ntpd[27993]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T18:19:35.736 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:35 ntpd[27993]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T18:19:35.736 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:35 ntpd[27993]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:35.736 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:35 ntpd[27993]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:35.737 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:35 ntpd[27993]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T18:19:35.737 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:35 ntpd[27993]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T18:19:35.737 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:35 ntpd[27993]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:35.737 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:35 ntpd[27993]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:35.737 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:35 ntpd[27993]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T18:19:35.738 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:35 ntpd[27993]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T18:19:35.756 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:35 ntpd[27995]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:35.756 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:35 ntpd[27995]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:35.756 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:35 ntpd[27995]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:35.756 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:35 ntpd[27995]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:35.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:35 ntpd[27995]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T18:19:35.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:35 ntpd[27995]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T18:19:36.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:36 ntpd[27995]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:36.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:36 ntpd[27995]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:36.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:36 ntpd[27995]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:36.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:36 ntpd[27995]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:36.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:36 ntpd[27995]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:36.758 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:36 ntpd[27995]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:36.758 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:36 ntpd[27995]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T18:19:36.758 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:36 ntpd[27995]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T18:19:36.758 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:36 ntpd[27995]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:36.758 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:36 ntpd[27995]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:36.758 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:36 ntpd[27995]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T18:19:36.758 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:36 ntpd[27995]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T18:19:37.732 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:37 ntpd[27993]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:37.732 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:37 ntpd[27993]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:37.732 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:37 ntpd[27993]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:37.732 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:37 ntpd[27993]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:37.732 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:37 ntpd[27993]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:37.732 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:37 ntpd[27993]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:37.733 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:37 ntpd[27993]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:37.733 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:37 ntpd[27993]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:37.733 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:37 ntpd[27993]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T18:19:37.733 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:37 ntpd[27993]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T18:19:37.733 INFO:teuthology.orchestra.run.smithi105.stdout:28 Nov 18:19:37 ntpd[27993]: ntpd: time slew -0.000750 s 2023-11-28T18:19:37.733 INFO:teuthology.orchestra.run.smithi105.stdout:ntpd: time slew -0.000750s 2023-11-28T18:19:37.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:37 ntpd[27993]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-11-28T18:19:37.734 INFO:teuthology.orchestra.run.smithi105.stderr:28 Nov 18:19:37 ntpd[27993]: can't open /var/log/ntpstats/loopstats.20231128: Permission denied 2023-11-28T18:19:37.756 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:37 ntpd[27995]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:37.756 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:37 ntpd[27995]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:37.756 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:37 ntpd[27995]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:37.756 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:37 ntpd[27995]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:37.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:37 ntpd[27995]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T18:19:37.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:37 ntpd[27995]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T18:19:37.834 INFO:teuthology.orchestra.run.smithi105.stdout: remote refid st t when poll reach delay offset jitter 2023-11-28T18:19:37.834 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================== 2023-11-28T18:19:37.834 INFO:teuthology.orchestra.run.smithi105.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-28T18:19:37.834 INFO:teuthology.orchestra.run.smithi105.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-28T18:19:37.834 INFO:teuthology.orchestra.run.smithi105.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-28T18:19:37.834 INFO:teuthology.orchestra.run.smithi105.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-28T18:19:38.756 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:38 ntpd[27995]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:38.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:38 ntpd[27995]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:38.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:38 ntpd[27995]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:38.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:38 ntpd[27995]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:38.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:38 ntpd[27995]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T18:19:38.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:38 ntpd[27995]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T18:19:38.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:38 ntpd[27995]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T18:19:38.757 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:38 ntpd[27995]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T18:19:38.758 INFO:teuthology.orchestra.run.smithi148.stdout:28 Nov 18:19:38 ntpd[27995]: ntpd: time slew -0.001165 s 2023-11-28T18:19:38.758 INFO:teuthology.orchestra.run.smithi148.stdout:ntpd: time slew -0.001165s 2023-11-28T18:19:38.758 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:38 ntpd[27995]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-11-28T18:19:38.758 INFO:teuthology.orchestra.run.smithi148.stderr:28 Nov 18:19:38 ntpd[27995]: can't open /var/log/ntpstats/loopstats.20231128: Permission denied 2023-11-28T18:19:38.854 INFO:teuthology.orchestra.run.smithi148.stdout: remote refid st t when poll reach delay offset jitter 2023-11-28T18:19:38.854 INFO:teuthology.orchestra.run.smithi148.stdout:============================================================================== 2023-11-28T18:19:38.854 INFO:teuthology.orchestra.run.smithi148.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-28T18:19:38.854 INFO:teuthology.orchestra.run.smithi148.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-28T18:19:38.854 INFO:teuthology.orchestra.run.smithi148.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-28T18:19:38.854 INFO:teuthology.orchestra.run.smithi148.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-28T18:19:38.855 INFO:teuthology.run_tasks:Running task install... 2023-11-28T18:19:38.864 DEBUG:teuthology.task.install:project ceph 2023-11-28T18:19:38.865 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '47c1c547372ed5bff7984647dc200bac0e019caf'}} 2023-11-28T18:19:38.865 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '47c1c547372ed5bff7984647dc200bac0e019caf'} 2023-11-28T18:19:38.865 INFO:teuthology.task.install:Using flavor: default 2023-11-28T18:19:38.874 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-28T18:19:38.874 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-28T18:19:38.875 DEBUG:teuthology.orchestra.run.smithi105:> sudo apt-key list | grep Ceph 2023-11-28T18:19:38.876 DEBUG:teuthology.orchestra.run.smithi148:> sudo apt-key list | grep Ceph 2023-11-28T18:19:38.983 INFO:teuthology.orchestra.run.smithi105.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-11-28T18:19:39.003 INFO:teuthology.orchestra.run.smithi148.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-11-28T18:19:39.036 INFO:teuthology.orchestra.run.smithi105.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-11-28T18:19:39.036 INFO:teuthology.orchestra.run.smithi105.stdout:uid [ unknown] Ceph.com (release key) 2023-11-28T18:19:39.037 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-11-28T18:19:39.038 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-11-28T18:19:39.038 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=47c1c547372ed5bff7984647dc200bac0e019caf 2023-11-28T18:19:39.056 INFO:teuthology.orchestra.run.smithi148.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-11-28T18:19:39.056 INFO:teuthology.orchestra.run.smithi148.stdout:uid [ unknown] Ceph.com (release key) 2023-11-28T18:19:39.057 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-11-28T18:19:39.057 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-11-28T18:19:39.057 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=47c1c547372ed5bff7984647dc200bac0e019caf 2023-11-28T18:19:39.182 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default/ 2023-11-28T18:19:39.182 INFO:teuthology.task.install.deb:Package version is 18.0.0-7531-g47c1c547-1jammy 2023-11-28T18:19:39.194 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default/ 2023-11-28T18:19:39.194 INFO:teuthology.task.install.deb:Package version is 18.0.0-7531-g47c1c547-1jammy 2023-11-28T18:19:39.320 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:19:39.320 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-11-28T18:19:39.343 DEBUG:teuthology.orchestra.run.smithi148:> sudo apt-get update 2023-11-28T18:19:39.365 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:19:39.365 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-11-28T18:19:39.385 DEBUG:teuthology.orchestra.run.smithi105:> sudo apt-get update 2023-11-28T18:19:39.574 INFO:teuthology.orchestra.run.smithi105.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-11-28T18:19:39.574 INFO:teuthology.orchestra.run.smithi105.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-11-28T18:19:39.578 INFO:teuthology.orchestra.run.smithi148.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy InRelease 2023-11-28T18:19:39.585 INFO:teuthology.orchestra.run.smithi105.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-11-28T18:19:39.600 INFO:teuthology.orchestra.run.smithi148.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-11-28T18:19:39.605 INFO:teuthology.orchestra.run.smithi105.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-11-28T18:19:39.621 INFO:teuthology.orchestra.run.smithi105.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy InRelease 2023-11-28T18:19:39.622 INFO:teuthology.orchestra.run.smithi148.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy Release.gpg 2023-11-28T18:19:39.646 INFO:teuthology.orchestra.run.smithi148.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-11-28T18:19:39.646 INFO:teuthology.orchestra.run.smithi105.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-11-28T18:19:39.665 INFO:teuthology.orchestra.run.smithi105.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy Release.gpg 2023-11-28T18:19:39.674 INFO:teuthology.orchestra.run.smithi148.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-11-28T18:19:39.675 INFO:teuthology.orchestra.run.smithi148.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-11-28T18:19:39.675 INFO:teuthology.orchestra.run.smithi148.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,403 B] 2023-11-28T18:19:39.686 INFO:teuthology.orchestra.run.smithi105.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,403 B] 2023-11-28T18:19:39.721 INFO:teuthology.orchestra.run.smithi105.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-11-28T18:19:39.745 INFO:teuthology.orchestra.run.smithi148.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-11-28T18:19:39.827 INFO:teuthology.orchestra.run.smithi148.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-11-28T18:19:40.182 INFO:teuthology.orchestra.run.smithi105.stdout:Fetched 28.6 kB in 1s (49.8 kB/s) 2023-11-28T18:19:40.278 INFO:teuthology.orchestra.run.smithi148.stdout:Fetched 28.6 kB in 1s (40.0 kB/s) 2023-11-28T18:19:41.110 INFO:teuthology.orchestra.run.smithi105.stdout:Reading package lists... 2023-11-28T18:19:41.125 DEBUG:teuthology.orchestra.run.smithi105:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-7531-g47c1c547-1jammy cephadm=18.0.0-7531-g47c1c547-1jammy ceph-mds=18.0.0-7531-g47c1c547-1jammy ceph-mgr=18.0.0-7531-g47c1c547-1jammy ceph-common=18.0.0-7531-g47c1c547-1jammy ceph-fuse=18.0.0-7531-g47c1c547-1jammy ceph-test=18.0.0-7531-g47c1c547-1jammy ceph-volume=18.0.0-7531-g47c1c547-1jammy radosgw=18.0.0-7531-g47c1c547-1jammy python3-rados=18.0.0-7531-g47c1c547-1jammy python3-rgw=18.0.0-7531-g47c1c547-1jammy python3-cephfs=18.0.0-7531-g47c1c547-1jammy python3-rbd=18.0.0-7531-g47c1c547-1jammy libcephfs2=18.0.0-7531-g47c1c547-1jammy libcephfs-dev=18.0.0-7531-g47c1c547-1jammy librados2=18.0.0-7531-g47c1c547-1jammy librbd1=18.0.0-7531-g47c1c547-1jammy rbd-fuse=18.0.0-7531-g47c1c547-1jammy python3-cephfs=18.0.0-7531-g47c1c547-1jammy cephfs-shell=18.0.0-7531-g47c1c547-1jammy cephfs-top=18.0.0-7531-g47c1c547-1jammy cephfs-mirror=18.0.0-7531-g47c1c547-1jammy 2023-11-28T18:19:41.200 INFO:teuthology.orchestra.run.smithi105.stdout:Reading package lists... 2023-11-28T18:19:41.202 INFO:teuthology.orchestra.run.smithi148.stdout:Reading package lists... 2023-11-28T18:19:41.217 DEBUG:teuthology.orchestra.run.smithi148:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-7531-g47c1c547-1jammy cephadm=18.0.0-7531-g47c1c547-1jammy ceph-mds=18.0.0-7531-g47c1c547-1jammy ceph-mgr=18.0.0-7531-g47c1c547-1jammy ceph-common=18.0.0-7531-g47c1c547-1jammy ceph-fuse=18.0.0-7531-g47c1c547-1jammy ceph-test=18.0.0-7531-g47c1c547-1jammy ceph-volume=18.0.0-7531-g47c1c547-1jammy radosgw=18.0.0-7531-g47c1c547-1jammy python3-rados=18.0.0-7531-g47c1c547-1jammy python3-rgw=18.0.0-7531-g47c1c547-1jammy python3-cephfs=18.0.0-7531-g47c1c547-1jammy python3-rbd=18.0.0-7531-g47c1c547-1jammy libcephfs2=18.0.0-7531-g47c1c547-1jammy libcephfs-dev=18.0.0-7531-g47c1c547-1jammy librados2=18.0.0-7531-g47c1c547-1jammy librbd1=18.0.0-7531-g47c1c547-1jammy rbd-fuse=18.0.0-7531-g47c1c547-1jammy python3-cephfs=18.0.0-7531-g47c1c547-1jammy cephfs-shell=18.0.0-7531-g47c1c547-1jammy cephfs-top=18.0.0-7531-g47c1c547-1jammy cephfs-mirror=18.0.0-7531-g47c1c547-1jammy 2023-11-28T18:19:41.293 INFO:teuthology.orchestra.run.smithi148.stdout:Reading package lists... 2023-11-28T18:19:41.365 INFO:teuthology.orchestra.run.smithi105.stdout:Building dependency tree... 2023-11-28T18:19:41.366 INFO:teuthology.orchestra.run.smithi105.stdout:Reading state information... 2023-11-28T18:19:41.465 INFO:teuthology.orchestra.run.smithi148.stdout:Building dependency tree... 2023-11-28T18:19:41.466 INFO:teuthology.orchestra.run.smithi148.stdout:Reading state information... 2023-11-28T18:19:41.516 INFO:teuthology.orchestra.run.smithi105.stdout:The following packages were automatically installed and are no longer required: 2023-11-28T18:19:41.516 INFO:teuthology.orchestra.run.smithi105.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-28T18:19:41.516 INFO:teuthology.orchestra.run.smithi105.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-28T18:19:41.516 INFO:teuthology.orchestra.run.smithi105.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-28T18:19:41.516 INFO:teuthology.orchestra.run.smithi105.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-28T18:19:41.517 INFO:teuthology.orchestra.run.smithi105.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-28T18:19:41.517 INFO:teuthology.orchestra.run.smithi105.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-28T18:19:41.517 INFO:teuthology.orchestra.run.smithi105.stdout:The following additional packages will be installed: 2023-11-28T18:19:41.517 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-11-28T18:19:41.518 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-11-28T18:19:41.518 INFO:teuthology.orchestra.run.smithi105.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2023-11-28T18:19:41.518 INFO:teuthology.orchestra.run.smithi105.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-11-28T18:19:41.519 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2023-11-28T18:19:41.520 INFO:teuthology.orchestra.run.smithi105.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2023-11-28T18:19:41.520 INFO:teuthology.orchestra.run.smithi105.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-11-28T18:19:41.520 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-11-28T18:19:41.520 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-11-28T18:19:41.520 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-11-28T18:19:41.520 INFO:teuthology.orchestra.run.smithi105.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-11-28T18:19:41.520 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-11-28T18:19:41.520 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-11-28T18:19:41.520 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-11-28T18:19:41.521 INFO:teuthology.orchestra.run.smithi105.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-11-28T18:19:41.521 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-11-28T18:19:41.521 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-11-28T18:19:41.521 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-11-28T18:19:41.521 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-11-28T18:19:41.521 INFO:teuthology.orchestra.run.smithi105.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-11-28T18:19:41.521 INFO:teuthology.orchestra.run.smithi105.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-11-28T18:19:41.521 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-11-28T18:19:41.521 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-11-28T18:19:41.521 INFO:teuthology.orchestra.run.smithi105.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2023-11-28T18:19:41.521 INFO:teuthology.orchestra.run.smithi105.stdout:Suggested packages: 2023-11-28T18:19:41.521 INFO:teuthology.orchestra.run.smithi105.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2023-11-28T18:19:41.521 INFO:teuthology.orchestra.run.smithi105.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2023-11-28T18:19:41.522 INFO:teuthology.orchestra.run.smithi105.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2023-11-28T18:19:41.522 INFO:teuthology.orchestra.run.smithi105.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2023-11-28T18:19:41.522 INFO:teuthology.orchestra.run.smithi105.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2023-11-28T18:19:41.522 INFO:teuthology.orchestra.run.smithi105.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-11-28T18:19:41.522 INFO:teuthology.orchestra.run.smithi105.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2023-11-28T18:19:41.522 INFO:teuthology.orchestra.run.smithi105.stdout:Recommended packages: 2023-11-28T18:19:41.522 INFO:teuthology.orchestra.run.smithi105.stdout: btrfs-tools 2023-11-28T18:19:41.624 INFO:teuthology.orchestra.run.smithi148.stdout:The following packages were automatically installed and are no longer required: 2023-11-28T18:19:41.625 INFO:teuthology.orchestra.run.smithi148.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-28T18:19:41.625 INFO:teuthology.orchestra.run.smithi148.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-28T18:19:41.625 INFO:teuthology.orchestra.run.smithi148.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-28T18:19:41.625 INFO:teuthology.orchestra.run.smithi148.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-28T18:19:41.625 INFO:teuthology.orchestra.run.smithi148.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-28T18:19:41.625 INFO:teuthology.orchestra.run.smithi148.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-28T18:19:41.626 INFO:teuthology.orchestra.run.smithi148.stdout:The following additional packages will be installed: 2023-11-28T18:19:41.626 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-11-28T18:19:41.626 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-11-28T18:19:41.627 INFO:teuthology.orchestra.run.smithi148.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2023-11-28T18:19:41.627 INFO:teuthology.orchestra.run.smithi148.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-11-28T18:19:41.628 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2023-11-28T18:19:41.628 INFO:teuthology.orchestra.run.smithi148.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2023-11-28T18:19:41.629 INFO:teuthology.orchestra.run.smithi148.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-11-28T18:19:41.629 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-11-28T18:19:41.629 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-11-28T18:19:41.629 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-11-28T18:19:41.629 INFO:teuthology.orchestra.run.smithi148.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-11-28T18:19:41.629 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-11-28T18:19:41.629 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-11-28T18:19:41.629 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-11-28T18:19:41.629 INFO:teuthology.orchestra.run.smithi148.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-11-28T18:19:41.630 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-11-28T18:19:41.630 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-11-28T18:19:41.630 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-11-28T18:19:41.630 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-11-28T18:19:41.630 INFO:teuthology.orchestra.run.smithi148.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-11-28T18:19:41.630 INFO:teuthology.orchestra.run.smithi148.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-11-28T18:19:41.630 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-11-28T18:19:41.630 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-11-28T18:19:41.630 INFO:teuthology.orchestra.run.smithi148.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2023-11-28T18:19:41.630 INFO:teuthology.orchestra.run.smithi148.stdout:Suggested packages: 2023-11-28T18:19:41.631 INFO:teuthology.orchestra.run.smithi148.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2023-11-28T18:19:41.631 INFO:teuthology.orchestra.run.smithi148.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2023-11-28T18:19:41.631 INFO:teuthology.orchestra.run.smithi148.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2023-11-28T18:19:41.631 INFO:teuthology.orchestra.run.smithi148.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2023-11-28T18:19:41.631 INFO:teuthology.orchestra.run.smithi148.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2023-11-28T18:19:41.631 INFO:teuthology.orchestra.run.smithi148.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-11-28T18:19:41.631 INFO:teuthology.orchestra.run.smithi148.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2023-11-28T18:19:41.631 INFO:teuthology.orchestra.run.smithi148.stdout:Recommended packages: 2023-11-28T18:19:41.631 INFO:teuthology.orchestra.run.smithi148.stdout: btrfs-tools 2023-11-28T18:19:41.665 INFO:teuthology.orchestra.run.smithi105.stdout:The following NEW packages will be installed: 2023-11-28T18:19:41.665 INFO:teuthology.orchestra.run.smithi105.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-11-28T18:19:41.665 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-11-28T18:19:41.665 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-11-28T18:19:41.665 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-11-28T18:19:41.667 INFO:teuthology.orchestra.run.smithi105.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2023-11-28T18:19:41.667 INFO:teuthology.orchestra.run.smithi105.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-11-28T18:19:41.667 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2023-11-28T18:19:41.667 INFO:teuthology.orchestra.run.smithi105.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2023-11-28T18:19:41.667 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2023-11-28T18:19:41.668 INFO:teuthology.orchestra.run.smithi105.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-11-28T18:19:41.668 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-11-28T18:19:41.668 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-11-28T18:19:41.668 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-11-28T18:19:41.668 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-11-28T18:19:41.668 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-11-28T18:19:41.668 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-11-28T18:19:41.668 INFO:teuthology.orchestra.run.smithi105.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-11-28T18:19:41.668 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-11-28T18:19:41.668 INFO:teuthology.orchestra.run.smithi105.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-11-28T18:19:41.668 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-11-28T18:19:41.668 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-11-28T18:19:41.668 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-11-28T18:19:41.669 INFO:teuthology.orchestra.run.smithi105.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-11-28T18:19:41.669 INFO:teuthology.orchestra.run.smithi105.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-11-28T18:19:41.669 INFO:teuthology.orchestra.run.smithi105.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-11-28T18:19:41.669 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-11-28T18:19:41.669 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-11-28T18:19:41.669 INFO:teuthology.orchestra.run.smithi105.stdout: socat unzip xmlstarlet zip 2023-11-28T18:19:41.734 INFO:teuthology.orchestra.run.smithi105.stdout:0 upgraded, 124 newly installed, 0 to remove and 264 not upgraded. 2023-11-28T18:19:41.734 INFO:teuthology.orchestra.run.smithi105.stdout:Need to get 187 MB of archives. 2023-11-28T18:19:41.734 INFO:teuthology.orchestra.run.smithi105.stdout:After this operation, 835 MB of additional disk space will be used. 2023-11-28T18:19:41.734 INFO:teuthology.orchestra.run.smithi105.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-11-28T18:19:41.751 INFO:teuthology.orchestra.run.smithi105.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-11-28T18:19:41.760 INFO:teuthology.orchestra.run.smithi105.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-11-28T18:19:41.779 INFO:teuthology.orchestra.run.smithi148.stdout:The following NEW packages will be installed: 2023-11-28T18:19:41.779 INFO:teuthology.orchestra.run.smithi148.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-11-28T18:19:41.779 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-11-28T18:19:41.779 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-11-28T18:19:41.779 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-11-28T18:19:41.781 INFO:teuthology.orchestra.run.smithi148.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2023-11-28T18:19:41.781 INFO:teuthology.orchestra.run.smithi148.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-11-28T18:19:41.781 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2023-11-28T18:19:41.781 INFO:teuthology.orchestra.run.smithi148.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2023-11-28T18:19:41.781 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2023-11-28T18:19:41.781 INFO:teuthology.orchestra.run.smithi148.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-11-28T18:19:41.782 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-11-28T18:19:41.782 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-11-28T18:19:41.782 INFO:teuthology.orchestra.run.smithi148.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-11-28T18:19:41.782 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-11-28T18:19:41.782 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-11-28T18:19:41.782 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-11-28T18:19:41.782 INFO:teuthology.orchestra.run.smithi148.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-11-28T18:19:41.783 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-11-28T18:19:41.783 INFO:teuthology.orchestra.run.smithi148.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-11-28T18:19:41.783 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-11-28T18:19:41.783 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-11-28T18:19:41.783 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-11-28T18:19:41.783 INFO:teuthology.orchestra.run.smithi148.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-11-28T18:19:41.783 INFO:teuthology.orchestra.run.smithi148.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-11-28T18:19:41.784 INFO:teuthology.orchestra.run.smithi148.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-11-28T18:19:41.784 INFO:teuthology.orchestra.run.smithi148.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-11-28T18:19:41.784 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-11-28T18:19:41.784 INFO:teuthology.orchestra.run.smithi148.stdout: socat unzip xmlstarlet zip 2023-11-28T18:19:41.804 INFO:teuthology.orchestra.run.smithi105.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-11-28T18:19:41.817 INFO:teuthology.orchestra.run.smithi105.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-7531-g47c1c547-1jammy [3,466 kB] 2023-11-28T18:19:41.825 INFO:teuthology.orchestra.run.smithi105.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-11-28T18:19:41.827 INFO:teuthology.orchestra.run.smithi105.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-11-28T18:19:41.839 INFO:teuthology.orchestra.run.smithi105.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-11-28T18:19:41.850 INFO:teuthology.orchestra.run.smithi148.stdout:0 upgraded, 124 newly installed, 0 to remove and 264 not upgraded. 2023-11-28T18:19:41.850 INFO:teuthology.orchestra.run.smithi148.stdout:Need to get 187 MB of archives. 2023-11-28T18:19:41.850 INFO:teuthology.orchestra.run.smithi148.stdout:After this operation, 835 MB of additional disk space will be used. 2023-11-28T18:19:41.850 INFO:teuthology.orchestra.run.smithi148.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-11-28T18:19:41.867 INFO:teuthology.orchestra.run.smithi148.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-11-28T18:19:41.877 INFO:teuthology.orchestra.run.smithi105.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-11-28T18:19:41.878 INFO:teuthology.orchestra.run.smithi148.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-11-28T18:19:41.879 INFO:teuthology.orchestra.run.smithi105.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-11-28T18:19:41.887 INFO:teuthology.orchestra.run.smithi105.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-11-28T18:19:41.890 INFO:teuthology.orchestra.run.smithi105.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-11-28T18:19:41.891 INFO:teuthology.orchestra.run.smithi105.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-11-28T18:19:41.897 INFO:teuthology.orchestra.run.smithi105.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2023-11-28T18:19:41.899 INFO:teuthology.orchestra.run.smithi105.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2023-11-28T18:19:41.901 INFO:teuthology.orchestra.run.smithi105.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2023-11-28T18:19:41.902 INFO:teuthology.orchestra.run.smithi105.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2023-11-28T18:19:41.902 INFO:teuthology.orchestra.run.smithi105.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2023-11-28T18:19:41.912 INFO:teuthology.orchestra.run.smithi105.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.1 [174 kB] 2023-11-28T18:19:41.914 INFO:teuthology.orchestra.run.smithi105.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2023-11-28T18:19:41.916 INFO:teuthology.orchestra.run.smithi105.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-11-28T18:19:41.916 INFO:teuthology.orchestra.run.smithi105.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-11-28T18:19:41.920 INFO:teuthology.orchestra.run.smithi148.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-11-28T18:19:41.930 INFO:teuthology.orchestra.run.smithi105.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-11-28T18:19:41.931 INFO:teuthology.orchestra.run.smithi105.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-11-28T18:19:41.931 INFO:teuthology.orchestra.run.smithi105.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-11-28T18:19:41.932 INFO:teuthology.orchestra.run.smithi105.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-11-28T18:19:41.932 INFO:teuthology.orchestra.run.smithi105.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-11-28T18:19:41.932 INFO:teuthology.orchestra.run.smithi105.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-11-28T18:19:41.941 INFO:teuthology.orchestra.run.smithi148.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-11-28T18:19:41.943 INFO:teuthology.orchestra.run.smithi148.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-11-28T18:19:41.947 INFO:teuthology.orchestra.run.smithi105.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-11-28T18:19:41.947 INFO:teuthology.orchestra.run.smithi105.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-11-28T18:19:41.949 INFO:teuthology.orchestra.run.smithi105.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-11-28T18:19:41.950 INFO:teuthology.orchestra.run.smithi105.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-11-28T18:19:41.955 INFO:teuthology.orchestra.run.smithi148.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-11-28T18:19:41.964 INFO:teuthology.orchestra.run.smithi105.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-11-28T18:19:41.965 INFO:teuthology.orchestra.run.smithi105.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-11-28T18:19:41.966 INFO:teuthology.orchestra.run.smithi105.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-11-28T18:19:41.966 INFO:teuthology.orchestra.run.smithi105.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-11-28T18:19:41.966 INFO:teuthology.orchestra.run.smithi105.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-11-28T18:19:41.968 INFO:teuthology.orchestra.run.smithi105.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-11-28T18:19:41.981 INFO:teuthology.orchestra.run.smithi105.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-11-28T18:19:41.982 INFO:teuthology.orchestra.run.smithi105.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-11-28T18:19:41.986 INFO:teuthology.orchestra.run.smithi105.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-11-28T18:19:41.987 INFO:teuthology.orchestra.run.smithi105.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-11-28T18:19:41.994 INFO:teuthology.orchestra.run.smithi148.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-11-28T18:19:41.997 INFO:teuthology.orchestra.run.smithi148.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-11-28T18:19:41.998 INFO:teuthology.orchestra.run.smithi105.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-11-28T18:19:41.999 INFO:teuthology.orchestra.run.smithi105.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-11-28T18:19:42.000 INFO:teuthology.orchestra.run.smithi105.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-11-28T18:19:42.002 INFO:teuthology.orchestra.run.smithi105.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-11-28T18:19:42.002 INFO:teuthology.orchestra.run.smithi105.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2023-11-28T18:19:42.004 INFO:teuthology.orchestra.run.smithi148.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-11-28T18:19:42.004 INFO:teuthology.orchestra.run.smithi105.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-11-28T18:19:42.005 INFO:teuthology.orchestra.run.smithi148.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-7531-g47c1c547-1jammy [3,466 kB] 2023-11-28T18:19:42.006 INFO:teuthology.orchestra.run.smithi148.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-11-28T18:19:42.007 INFO:teuthology.orchestra.run.smithi148.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-11-28T18:19:42.013 INFO:teuthology.orchestra.run.smithi148.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2023-11-28T18:19:42.014 INFO:teuthology.orchestra.run.smithi148.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2023-11-28T18:19:42.015 INFO:teuthology.orchestra.run.smithi105.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-11-28T18:19:42.016 INFO:teuthology.orchestra.run.smithi148.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2023-11-28T18:19:42.017 INFO:teuthology.orchestra.run.smithi105.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-11-28T18:19:42.017 INFO:teuthology.orchestra.run.smithi148.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2023-11-28T18:19:42.018 INFO:teuthology.orchestra.run.smithi105.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-11-28T18:19:42.018 INFO:teuthology.orchestra.run.smithi148.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2023-11-28T18:19:42.018 INFO:teuthology.orchestra.run.smithi105.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-11-28T18:19:42.031 INFO:teuthology.orchestra.run.smithi148.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.1 [174 kB] 2023-11-28T18:19:42.032 INFO:teuthology.orchestra.run.smithi105.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-11-28T18:19:42.032 INFO:teuthology.orchestra.run.smithi148.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2023-11-28T18:19:42.034 INFO:teuthology.orchestra.run.smithi148.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-11-28T18:19:42.034 INFO:teuthology.orchestra.run.smithi148.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-11-28T18:19:42.048 INFO:teuthology.orchestra.run.smithi148.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-11-28T18:19:42.048 INFO:teuthology.orchestra.run.smithi148.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-11-28T18:19:42.049 INFO:teuthology.orchestra.run.smithi148.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-11-28T18:19:42.049 INFO:teuthology.orchestra.run.smithi148.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-11-28T18:19:42.050 INFO:teuthology.orchestra.run.smithi148.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-11-28T18:19:42.050 INFO:teuthology.orchestra.run.smithi148.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-11-28T18:19:42.065 INFO:teuthology.orchestra.run.smithi148.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-11-28T18:19:42.065 INFO:teuthology.orchestra.run.smithi148.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-11-28T18:19:42.067 INFO:teuthology.orchestra.run.smithi148.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-11-28T18:19:42.068 INFO:teuthology.orchestra.run.smithi148.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-11-28T18:19:42.072 INFO:teuthology.orchestra.run.smithi105.stdout:Get:54 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-7531-g47c1c547-1jammy [3,111 kB] 2023-11-28T18:19:42.080 INFO:teuthology.orchestra.run.smithi105.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-11-28T18:19:42.081 INFO:teuthology.orchestra.run.smithi105.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-11-28T18:19:42.081 INFO:teuthology.orchestra.run.smithi148.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-11-28T18:19:42.082 INFO:teuthology.orchestra.run.smithi148.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-11-28T18:19:42.082 INFO:teuthology.orchestra.run.smithi148.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-11-28T18:19:42.083 INFO:teuthology.orchestra.run.smithi105.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-11-28T18:19:42.084 INFO:teuthology.orchestra.run.smithi105.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-11-28T18:19:42.084 INFO:teuthology.orchestra.run.smithi148.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-11-28T18:19:42.084 INFO:teuthology.orchestra.run.smithi148.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-11-28T18:19:42.084 INFO:teuthology.orchestra.run.smithi148.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-11-28T18:19:42.085 INFO:teuthology.orchestra.run.smithi105.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-11-28T18:19:42.098 INFO:teuthology.orchestra.run.smithi148.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-11-28T18:19:42.099 INFO:teuthology.orchestra.run.smithi148.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-11-28T18:19:42.103 INFO:teuthology.orchestra.run.smithi148.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-11-28T18:19:42.103 INFO:teuthology.orchestra.run.smithi148.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-11-28T18:19:42.105 INFO:teuthology.orchestra.run.smithi105.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-11-28T18:19:42.107 INFO:teuthology.orchestra.run.smithi105.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-11-28T18:19:42.107 INFO:teuthology.orchestra.run.smithi105.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-11-28T18:19:42.115 INFO:teuthology.orchestra.run.smithi148.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-11-28T18:19:42.116 INFO:teuthology.orchestra.run.smithi148.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-11-28T18:19:42.117 INFO:teuthology.orchestra.run.smithi148.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-11-28T18:19:42.119 INFO:teuthology.orchestra.run.smithi148.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-11-28T18:19:42.119 INFO:teuthology.orchestra.run.smithi148.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2023-11-28T18:19:42.121 INFO:teuthology.orchestra.run.smithi148.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-11-28T18:19:42.124 INFO:teuthology.orchestra.run.smithi105.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-11-28T18:19:42.125 INFO:teuthology.orchestra.run.smithi105.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-11-28T18:19:42.125 INFO:teuthology.orchestra.run.smithi105.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-11-28T18:19:42.126 INFO:teuthology.orchestra.run.smithi105.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-11-28T18:19:42.127 INFO:teuthology.orchestra.run.smithi105.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-11-28T18:19:42.127 INFO:teuthology.orchestra.run.smithi105.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-11-28T18:19:42.132 INFO:teuthology.orchestra.run.smithi148.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-11-28T18:19:42.134 INFO:teuthology.orchestra.run.smithi105.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-11-28T18:19:42.134 INFO:teuthology.orchestra.run.smithi148.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-11-28T18:19:42.135 INFO:teuthology.orchestra.run.smithi105.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-11-28T18:19:42.135 INFO:teuthology.orchestra.run.smithi148.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-11-28T18:19:42.135 INFO:teuthology.orchestra.run.smithi148.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-11-28T18:19:42.138 INFO:teuthology.orchestra.run.smithi105.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-11-28T18:19:42.149 INFO:teuthology.orchestra.run.smithi148.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-11-28T18:19:42.151 INFO:teuthology.orchestra.run.smithi105.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-11-28T18:19:42.153 INFO:teuthology.orchestra.run.smithi105.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-11-28T18:19:42.154 INFO:teuthology.orchestra.run.smithi105.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-11-28T18:19:42.157 INFO:teuthology.orchestra.run.smithi105.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-11-28T18:19:42.160 INFO:teuthology.orchestra.run.smithi105.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-11-28T18:19:42.160 INFO:teuthology.orchestra.run.smithi105.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2023-11-28T18:19:42.161 INFO:teuthology.orchestra.run.smithi105.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2023-11-28T18:19:42.169 INFO:teuthology.orchestra.run.smithi105.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-11-28T18:19:42.173 INFO:teuthology.orchestra.run.smithi105.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2023-11-28T18:19:42.174 INFO:teuthology.orchestra.run.smithi105.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-11-28T18:19:42.186 INFO:teuthology.orchestra.run.smithi105.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-11-28T18:19:42.186 INFO:teuthology.orchestra.run.smithi105.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-11-28T18:19:42.187 INFO:teuthology.orchestra.run.smithi105.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-11-28T18:19:42.188 INFO:teuthology.orchestra.run.smithi105.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-11-28T18:19:42.189 INFO:teuthology.orchestra.run.smithi105.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-11-28T18:19:42.190 INFO:teuthology.orchestra.run.smithi105.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-11-28T18:19:42.190 INFO:teuthology.orchestra.run.smithi105.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-7531-g47c1c547-1jammy [690 kB] 2023-11-28T18:19:42.194 INFO:teuthology.orchestra.run.smithi148.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-11-28T18:19:42.195 INFO:teuthology.orchestra.run.smithi148.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-11-28T18:19:42.196 INFO:teuthology.orchestra.run.smithi148.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-11-28T18:19:42.197 INFO:teuthology.orchestra.run.smithi148.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-11-28T18:19:42.197 INFO:teuthology.orchestra.run.smithi105.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-11-28T18:19:42.198 INFO:teuthology.orchestra.run.smithi148.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-11-28T18:19:42.203 INFO:teuthology.orchestra.run.smithi105.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-11-28T18:19:42.203 INFO:teuthology.orchestra.run.smithi105.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-11-28T18:19:42.205 INFO:teuthology.orchestra.run.smithi105.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-11-28T18:19:42.216 INFO:teuthology.orchestra.run.smithi148.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-11-28T18:19:42.217 INFO:teuthology.orchestra.run.smithi105.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-7531-g47c1c547-1jammy [356 kB] 2023-11-28T18:19:42.219 INFO:teuthology.orchestra.run.smithi148.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-11-28T18:19:42.219 INFO:teuthology.orchestra.run.smithi148.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-11-28T18:19:42.220 INFO:teuthology.orchestra.run.smithi105.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-11-28T18:19:42.229 INFO:teuthology.orchestra.run.smithi105.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-7531-g47c1c547-1jammy [32.8 kB] 2023-11-28T18:19:42.230 INFO:teuthology.orchestra.run.smithi105.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-7531-g47c1c547-1jammy [180 kB] 2023-11-28T18:19:42.236 INFO:teuthology.orchestra.run.smithi148.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-11-28T18:19:42.236 INFO:teuthology.orchestra.run.smithi148.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-11-28T18:19:42.236 INFO:teuthology.orchestra.run.smithi105.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-7531-g47c1c547-1jammy [63.5 kB] 2023-11-28T18:19:42.237 INFO:teuthology.orchestra.run.smithi148.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-11-28T18:19:42.237 INFO:teuthology.orchestra.run.smithi148.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-11-28T18:19:42.238 INFO:teuthology.orchestra.run.smithi148.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-11-28T18:19:42.238 INFO:teuthology.orchestra.run.smithi148.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-11-28T18:19:42.239 INFO:teuthology.orchestra.run.smithi105.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-7531-g47c1c547-1jammy [330 kB] 2023-11-28T18:19:42.247 INFO:teuthology.orchestra.run.smithi148.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-11-28T18:19:42.247 INFO:teuthology.orchestra.run.smithi148.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-11-28T18:19:42.251 INFO:teuthology.orchestra.run.smithi148.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-11-28T18:19:42.255 INFO:teuthology.orchestra.run.smithi105.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-7531-g47c1c547-1jammy [6,233 kB] 2023-11-28T18:19:42.263 INFO:teuthology.orchestra.run.smithi148.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-11-28T18:19:42.265 INFO:teuthology.orchestra.run.smithi148.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-11-28T18:19:42.266 INFO:teuthology.orchestra.run.smithi148.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-11-28T18:19:42.269 INFO:teuthology.orchestra.run.smithi148.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-11-28T18:19:42.272 INFO:teuthology.orchestra.run.smithi148.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-11-28T18:19:42.272 INFO:teuthology.orchestra.run.smithi148.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2023-11-28T18:19:42.273 INFO:teuthology.orchestra.run.smithi148.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2023-11-28T18:19:42.280 INFO:teuthology.orchestra.run.smithi148.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-11-28T18:19:42.285 INFO:teuthology.orchestra.run.smithi148.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2023-11-28T18:19:42.286 INFO:teuthology.orchestra.run.smithi148.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-11-28T18:19:42.298 INFO:teuthology.orchestra.run.smithi148.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-11-28T18:19:42.298 INFO:teuthology.orchestra.run.smithi148.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-11-28T18:19:42.298 INFO:teuthology.orchestra.run.smithi148.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-11-28T18:19:42.299 INFO:teuthology.orchestra.run.smithi148.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-11-28T18:19:42.300 INFO:teuthology.orchestra.run.smithi148.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-11-28T18:19:42.301 INFO:teuthology.orchestra.run.smithi148.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-11-28T18:19:42.308 INFO:teuthology.orchestra.run.smithi148.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-11-28T18:19:42.314 INFO:teuthology.orchestra.run.smithi148.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-11-28T18:19:42.315 INFO:teuthology.orchestra.run.smithi148.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-11-28T18:19:42.317 INFO:teuthology.orchestra.run.smithi148.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-11-28T18:19:42.331 INFO:teuthology.orchestra.run.smithi148.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-11-28T18:19:42.505 INFO:teuthology.orchestra.run.smithi105.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-7531-g47c1c547-1jammy [112 kB] 2023-11-28T18:19:42.506 INFO:teuthology.orchestra.run.smithi105.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-7531-g47c1c547-1jammy [439 kB] 2023-11-28T18:19:42.525 INFO:teuthology.orchestra.run.smithi105.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-7531-g47c1c547-1jammy [22.6 MB] 2023-11-28T18:19:42.845 INFO:teuthology.orchestra.run.smithi148.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-7531-g47c1c547-1jammy [3,111 kB] 2023-11-28T18:19:43.365 INFO:teuthology.orchestra.run.smithi148.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-7531-g47c1c547-1jammy [690 kB] 2023-11-28T18:19:43.424 INFO:teuthology.orchestra.run.smithi105.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-7531-g47c1c547-1jammy [6,900 kB] 2023-11-28T18:19:43.460 INFO:teuthology.orchestra.run.smithi148.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-7531-g47c1c547-1jammy [356 kB] 2023-11-28T18:19:43.487 INFO:teuthology.orchestra.run.smithi148.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-7531-g47c1c547-1jammy [32.8 kB] 2023-11-28T18:19:43.494 INFO:teuthology.orchestra.run.smithi148.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-7531-g47c1c547-1jammy [180 kB] 2023-11-28T18:19:43.519 INFO:teuthology.orchestra.run.smithi148.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-7531-g47c1c547-1jammy [63.5 kB] 2023-11-28T18:19:43.536 INFO:teuthology.orchestra.run.smithi148.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-7531-g47c1c547-1jammy [330 kB] 2023-11-28T18:19:43.563 INFO:teuthology.orchestra.run.smithi148.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-7531-g47c1c547-1jammy [6,233 kB] 2023-11-28T18:19:43.732 INFO:teuthology.orchestra.run.smithi105.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-7531-g47c1c547-1jammy [238 kB] 2023-11-28T18:19:43.739 INFO:teuthology.orchestra.run.smithi105.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-7531-g47c1c547-1jammy [124 kB] 2023-11-28T18:19:43.745 INFO:teuthology.orchestra.run.smithi105.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-7531-g47c1c547-1jammy [1,496 kB] 2023-11-28T18:19:43.816 INFO:teuthology.orchestra.run.smithi105.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-7531-g47c1c547-1jammy [9,957 kB] 2023-11-28T18:19:44.223 INFO:teuthology.orchestra.run.smithi148.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-7531-g47c1c547-1jammy [112 kB] 2023-11-28T18:19:44.232 INFO:teuthology.orchestra.run.smithi148.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-7531-g47c1c547-1jammy [439 kB] 2023-11-28T18:19:44.276 INFO:teuthology.orchestra.run.smithi148.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-7531-g47c1c547-1jammy [22.6 MB] 2023-11-28T18:19:44.311 INFO:teuthology.orchestra.run.smithi105.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-7531-g47c1c547-1jammy [31.9 MB] 2023-11-28T18:19:45.976 INFO:teuthology.orchestra.run.smithi105.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-7531-g47c1c547-1jammy [14.0 kB] 2023-11-28T18:19:45.977 INFO:teuthology.orchestra.run.smithi105.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-7531-g47c1c547-1jammy [839 kB] 2023-11-28T18:19:46.011 INFO:teuthology.orchestra.run.smithi105.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-7531-g47c1c547-1jammy [2,144 kB] 2023-11-28T18:19:46.124 INFO:teuthology.orchestra.run.smithi105.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-7531-g47c1c547-1jammy [802 kB] 2023-11-28T18:19:46.167 INFO:teuthology.orchestra.run.smithi105.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-7531-g47c1c547-1jammy [139 kB] 2023-11-28T18:19:46.176 INFO:teuthology.orchestra.run.smithi105.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-7531-g47c1c547-1jammy [1,900 kB] 2023-11-28T18:19:46.277 INFO:teuthology.orchestra.run.smithi105.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-7531-g47c1c547-1jammy [8,625 kB] 2023-11-28T18:19:46.278 INFO:teuthology.orchestra.run.smithi148.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-7531-g47c1c547-1jammy [6,900 kB] 2023-11-28T18:19:46.808 INFO:teuthology.orchestra.run.smithi105.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-7531-g47c1c547-1jammy [14.2 kB] 2023-11-28T18:19:46.808 INFO:teuthology.orchestra.run.smithi105.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-7531-g47c1c547-1jammy [48.7 MB] 2023-11-28T18:19:46.983 INFO:teuthology.orchestra.run.smithi148.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-7531-g47c1c547-1jammy [238 kB] 2023-11-28T18:19:47.002 INFO:teuthology.orchestra.run.smithi148.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-7531-g47c1c547-1jammy [124 kB] 2023-11-28T18:19:47.010 INFO:teuthology.orchestra.run.smithi148.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-7531-g47c1c547-1jammy [1,496 kB] 2023-11-28T18:19:47.172 INFO:teuthology.orchestra.run.smithi148.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-7531-g47c1c547-1jammy [9,957 kB] 2023-11-28T18:19:48.271 INFO:teuthology.orchestra.run.smithi148.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-7531-g47c1c547-1jammy [31.9 MB] 2023-11-28T18:19:50.209 INFO:teuthology.orchestra.run.smithi105.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-7531-g47c1c547-1jammy [121 kB] 2023-11-28T18:19:50.210 INFO:teuthology.orchestra.run.smithi105.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-7531-g47c1c547-1jammy [225 kB] 2023-11-28T18:19:50.217 INFO:teuthology.orchestra.run.smithi105.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-7531-g47c1c547-1jammy [34.0 kB] 2023-11-28T18:19:50.219 INFO:teuthology.orchestra.run.smithi105.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-7531-g47c1c547-1jammy [23.4 kB] 2023-11-28T18:19:50.221 INFO:teuthology.orchestra.run.smithi105.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-7531-g47c1c547-1jammy [38.3 kB] 2023-11-28T18:19:50.224 INFO:teuthology.orchestra.run.smithi105.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-7531-g47c1c547-1jammy [12.3 MB] 2023-11-28T18:19:51.175 INFO:teuthology.orchestra.run.smithi105.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-7531-g47c1c547-1jammy [91.4 kB] 2023-11-28T18:19:51.814 INFO:teuthology.orchestra.run.smithi105.stdout:Fetched 187 MB in 9s (19.7 MB/s) 2023-11-28T18:19:51.855 INFO:teuthology.orchestra.run.smithi148.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-7531-g47c1c547-1jammy [14.0 kB] 2023-11-28T18:19:51.856 INFO:teuthology.orchestra.run.smithi148.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-7531-g47c1c547-1jammy [839 kB] 2023-11-28T18:19:51.937 INFO:teuthology.orchestra.run.smithi148.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-7531-g47c1c547-1jammy [2,144 kB] 2023-11-28T18:19:51.995 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pyperclip. 2023-11-28T18:19:52.064 INFO:teuthology.orchestra.run.smithi105.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132930 files and directories currently installed.) 2023-11-28T18:19:52.067 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-11-28T18:19:52.119 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-11-28T18:19:52.138 INFO:teuthology.orchestra.run.smithi148.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-7531-g47c1c547-1jammy [802 kB] 2023-11-28T18:19:52.219 INFO:teuthology.orchestra.run.smithi148.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-7531-g47c1c547-1jammy [139 kB] 2023-11-28T18:19:52.242 INFO:teuthology.orchestra.run.smithi148.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-7531-g47c1c547-1jammy [1,900 kB] 2023-11-28T18:19:52.441 INFO:teuthology.orchestra.run.smithi148.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-7531-g47c1c547-1jammy [8,625 kB] 2023-11-28T18:19:52.488 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-wcwidth. 2023-11-28T18:19:52.502 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-11-28T18:19:52.547 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-28T18:19:52.933 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-cmd2. 2023-11-28T18:19:52.947 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-11-28T18:19:52.999 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-11-28T18:19:53.444 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-11-28T18:19:53.459 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-11-28T18:19:53.476 INFO:teuthology.orchestra.run.smithi148.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-7531-g47c1c547-1jammy [14.2 kB] 2023-11-28T18:19:53.476 INFO:teuthology.orchestra.run.smithi148.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-7531-g47c1c547-1jammy [48.7 MB] 2023-11-28T18:19:53.561 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-28T18:19:53.915 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-11-28T18:19:53.930 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-11-28T18:19:53.990 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-28T18:19:54.426 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-11-28T18:19:54.441 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-11-28T18:19:54.493 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-28T18:19:55.038 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-11-28T18:19:55.053 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-28T18:19:55.113 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T18:19:55.533 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-11-28T18:19:55.547 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-28T18:19:55.599 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T18:19:56.011 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-11-28T18:19:56.026 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-28T18:19:56.078 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T18:19:56.506 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-11-28T18:19:56.520 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-11-28T18:19:56.572 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-28T18:19:57.176 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package librados2. 2023-11-28T18:19:57.190 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../010-librados2_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:19:57.242 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking librados2 (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:19:57.720 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package librbd1. 2023-11-28T18:19:57.734 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../011-librbd1_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:19:57.786 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking librbd1 (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:19:58.290 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libcephfs2. 2023-11-28T18:19:58.305 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../012-libcephfs2_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:19:58.356 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libcephfs2 (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:19:58.642 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-rados. 2023-11-28T18:19:58.657 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../013-python3-rados_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:19:58.700 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-rados (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:19:58.953 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-11-28T18:19:58.971 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.0.0-7531-g47c1c547-1jammy_all.deb ... 2023-11-28T18:19:59.003 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-ceph-argparse (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:19:59.255 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-cephfs. 2023-11-28T18:19:59.270 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../015-python3-cephfs_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:19:59.314 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-cephfs (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:19:59.617 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-ceph-common. 2023-11-28T18:19:59.631 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../016-python3-ceph-common_18.0.0-7531-g47c1c547-1jammy_all.deb ... 2023-11-28T18:19:59.675 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-ceph-common (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:00.003 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-prettytable. 2023-11-28T18:20:00.018 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2023-11-28T18:20:00.061 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-11-28T18:20:00.289 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-rbd. 2023-11-28T18:20:00.303 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../018-python3-rbd_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:00.359 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-rbd (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:00.683 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-11-28T18:20:00.697 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-11-28T18:20:00.741 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-28T18:20:01.027 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libreadline-dev:amd64. 2023-11-28T18:20:01.042 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2023-11-28T18:20:01.086 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2023-11-28T18:20:01.372 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2023-11-28T18:20:01.387 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2023-11-28T18:20:01.422 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2023-11-28T18:20:01.725 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package lua5.1. 2023-11-28T18:20:01.740 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2023-11-28T18:20:01.783 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2023-11-28T18:20:02.078 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package lua-any. 2023-11-28T18:20:02.093 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2023-11-28T18:20:02.136 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking lua-any (27ubuntu1) ... 2023-11-28T18:20:02.389 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package zip. 2023-11-28T18:20:02.404 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2023-11-28T18:20:02.447 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking zip (3.0-12build2) ... 2023-11-28T18:20:02.717 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package unzip. 2023-11-28T18:20:02.732 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.1_amd64.deb ... 2023-11-28T18:20:02.767 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking unzip (6.0-26ubuntu3.1) ... 2023-11-28T18:20:02.797 INFO:teuthology.orchestra.run.smithi148.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-7531-g47c1c547-1jammy [121 kB] 2023-11-28T18:20:02.817 INFO:teuthology.orchestra.run.smithi148.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-7531-g47c1c547-1jammy [225 kB] 2023-11-28T18:20:02.882 INFO:teuthology.orchestra.run.smithi148.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-7531-g47c1c547-1jammy [34.0 kB] 2023-11-28T18:20:02.884 INFO:teuthology.orchestra.run.smithi148.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-7531-g47c1c547-1jammy [23.4 kB] 2023-11-28T18:20:02.898 INFO:teuthology.orchestra.run.smithi148.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-7531-g47c1c547-1jammy [38.3 kB] 2023-11-28T18:20:02.902 INFO:teuthology.orchestra.run.smithi148.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-7531-g47c1c547-1jammy [12.3 MB] 2023-11-28T18:20:03.179 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package luarocks. 2023-11-28T18:20:03.194 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2023-11-28T18:20:03.238 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2023-11-28T18:20:03.645 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package librgw2. 2023-11-28T18:20:03.659 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../027-librgw2_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:03.704 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking librgw2 (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:04.181 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-rgw. 2023-11-28T18:20:04.196 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../028-python3-rgw_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:04.240 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-rgw (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:04.525 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package liboath0:amd64. 2023-11-28T18:20:04.540 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2023-11-28T18:20:04.584 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-11-28T18:20:04.887 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libradosstriper1. 2023-11-28T18:20:04.902 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../030-libradosstriper1_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:04.945 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libradosstriper1 (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:05.390 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-common. 2023-11-28T18:20:05.405 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../031-ceph-common_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:05.443 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-common (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:05.880 INFO:teuthology.orchestra.run.smithi148.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-7531-g47c1c547-1jammy [91.4 kB] 2023-11-28T18:20:06.487 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-base. 2023-11-28T18:20:06.502 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../032-ceph-base_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:06.551 INFO:teuthology.orchestra.run.smithi148.stdout:Fetched 187 MB in 24s (7,751 kB/s) 2023-11-28T18:20:06.554 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-base (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:06.698 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-pyperclip. 2023-11-28T18:20:06.767 INFO:teuthology.orchestra.run.smithi148.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132930 files and directories currently installed.) 2023-11-28T18:20:06.771 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-11-28T18:20:06.814 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-11-28T18:20:07.117 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-wcwidth. 2023-11-28T18:20:07.130 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-11-28T18:20:07.175 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-28T18:20:07.216 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-11-28T18:20:07.230 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-11-28T18:20:07.274 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-11-28T18:20:07.478 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-cmd2. 2023-11-28T18:20:07.491 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-11-28T18:20:07.527 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-11-28T18:20:07.754 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-cheroot. 2023-11-28T18:20:07.769 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-11-28T18:20:07.811 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-28T18:20:07.813 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-11-28T18:20:07.828 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-11-28T18:20:07.914 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-28T18:20:08.106 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-11-28T18:20:08.120 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-11-28T18:20:08.155 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-11-28T18:20:08.301 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-11-28T18:20:08.316 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-11-28T18:20:08.359 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-28T18:20:08.425 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-jaraco.text. 2023-11-28T18:20:08.440 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2023-11-28T18:20:08.483 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-11-28T18:20:08.687 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-11-28T18:20:08.702 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-11-28T18:20:08.745 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-28T18:20:08.796 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-11-28T18:20:08.809 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-11-28T18:20:08.884 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-11-28T18:20:09.149 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-11-28T18:20:09.164 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-28T18:20:09.189 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-tempora. 2023-11-28T18:20:09.204 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2023-11-28T18:20:09.206 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T18:20:09.247 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-11-28T18:20:09.500 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-portend. 2023-11-28T18:20:09.515 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2023-11-28T18:20:09.550 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-11-28T18:20:09.585 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-11-28T18:20:09.600 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-28T18:20:09.644 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T18:20:09.853 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-11-28T18:20:09.868 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2023-11-28T18:20:09.911 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-11-28T18:20:09.980 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-11-28T18:20:09.994 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-28T18:20:10.029 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T18:20:10.214 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-cherrypy3. 2023-11-28T18:20:10.229 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2023-11-28T18:20:10.273 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-11-28T18:20:10.366 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-11-28T18:20:10.381 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-11-28T18:20:10.433 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-28T18:20:10.634 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-natsort. 2023-11-28T18:20:10.650 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2023-11-28T18:20:10.692 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-11-28T18:20:10.744 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package librados2. 2023-11-28T18:20:10.759 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../010-librados2_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:10.794 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking librados2 (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:10.979 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-logutils. 2023-11-28T18:20:10.994 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2023-11-28T18:20:11.029 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-11-28T18:20:11.324 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-markupsafe. 2023-11-28T18:20:11.339 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-11-28T18:20:11.372 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package librbd1. 2023-11-28T18:20:11.382 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-11-28T18:20:11.386 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../011-librbd1_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:11.446 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking librbd1 (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:11.718 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-mako. 2023-11-28T18:20:11.733 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-11-28T18:20:11.776 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-28T18:20:11.832 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libcephfs2. 2023-11-28T18:20:11.846 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../012-libcephfs2_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:11.890 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libcephfs2 (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:12.105 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-simplegeneric. 2023-11-28T18:20:12.120 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2023-11-28T18:20:12.163 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-11-28T18:20:12.168 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-rados. 2023-11-28T18:20:12.183 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../013-python3-rados_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:12.243 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-rados (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:12.441 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-singledispatch. 2023-11-28T18:20:12.456 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-11-28T18:20:12.491 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-11-28T18:20:12.496 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-11-28T18:20:12.510 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.0.0-7531-g47c1c547-1jammy_all.deb ... 2023-11-28T18:20:12.554 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-ceph-argparse (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:12.786 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-webob. 2023-11-28T18:20:12.790 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-cephfs. 2023-11-28T18:20:12.800 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-11-28T18:20:12.804 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../015-python3-cephfs_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:12.843 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-11-28T18:20:12.848 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-cephfs (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:13.122 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-waitress. 2023-11-28T18:20:13.136 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-11-28T18:20:13.174 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-28T18:20:13.176 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-ceph-common. 2023-11-28T18:20:13.191 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../016-python3-ceph-common_18.0.0-7531-g47c1c547-1jammy_all.deb ... 2023-11-28T18:20:13.234 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-ceph-common (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:13.474 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-tempita. 2023-11-28T18:20:13.489 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-11-28T18:20:13.533 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-11-28T18:20:13.553 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-prettytable. 2023-11-28T18:20:13.568 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2023-11-28T18:20:13.612 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-11-28T18:20:13.864 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-rbd. 2023-11-28T18:20:13.879 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../018-python3-rbd_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:13.914 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-rbd (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:14.242 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-11-28T18:20:14.257 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-11-28T18:20:14.300 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-28T18:20:14.502 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-paste. 2023-11-28T18:20:14.517 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-11-28T18:20:14.561 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libreadline-dev:amd64. 2023-11-28T18:20:14.569 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-11-28T18:20:14.576 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2023-11-28T18:20:14.645 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2023-11-28T18:20:14.864 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-11-28T18:20:14.879 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-11-28T18:20:14.922 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-11-28T18:20:15.225 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pastedeploy. 2023-11-28T18:20:15.240 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2023-11-28T18:20:15.283 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-11-28T18:20:15.547 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2023-11-28T18:20:15.562 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2023-11-28T18:20:15.597 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2023-11-28T18:20:15.612 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-webtest. 2023-11-28T18:20:15.627 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2023-11-28T18:20:15.669 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-11-28T18:20:15.918 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package lua5.1. 2023-11-28T18:20:15.933 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2023-11-28T18:20:15.947 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pecan. 2023-11-28T18:20:15.962 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-11-28T18:20:15.976 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2023-11-28T18:20:15.997 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-11-28T18:20:16.303 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package lua-any. 2023-11-28T18:20:16.318 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2023-11-28T18:20:16.319 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-certifi. 2023-11-28T18:20:16.333 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2023-11-28T18:20:16.362 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking lua-any (27ubuntu1) ... 2023-11-28T18:20:16.368 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-11-28T18:20:16.581 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package zip. 2023-11-28T18:20:16.597 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2023-11-28T18:20:16.631 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking zip (3.0-12build2) ... 2023-11-28T18:20:16.638 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-idna. 2023-11-28T18:20:16.652 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2023-11-28T18:20:16.713 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-idna (3.3-1) ... 2023-11-28T18:20:16.928 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package unzip. 2023-11-28T18:20:16.943 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.1_amd64.deb ... 2023-11-28T18:20:16.986 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking unzip (6.0-26ubuntu3.1) ... 2023-11-28T18:20:17.009 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-urllib3. 2023-11-28T18:20:17.022 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2023-11-28T18:20:17.066 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-11-28T18:20:17.386 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-requests. 2023-11-28T18:20:17.398 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package luarocks. 2023-11-28T18:20:17.401 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-11-28T18:20:17.412 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2023-11-28T18:20:17.444 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-28T18:20:17.456 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2023-11-28T18:20:17.822 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package librgw2. 2023-11-28T18:20:17.836 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../027-librgw2_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:17.888 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking librgw2 (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:17.922 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-werkzeug. 2023-11-28T18:20:17.937 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-11-28T18:20:17.980 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-28T18:20:18.267 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-11-28T18:20:18.282 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_18.0.0-7531-g47c1c547-1jammy_all.deb ... 2023-11-28T18:20:18.325 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mgr-modules-core (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:18.374 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-rgw. 2023-11-28T18:20:18.388 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../028-python3-rgw_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:18.432 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-rgw (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:18.738 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-11-28T18:20:18.753 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:18.794 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package liboath0:amd64. 2023-11-28T18:20:18.796 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:18.808 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2023-11-28T18:20:18.852 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-11-28T18:20:19.066 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-bcrypt. 2023-11-28T18:20:19.081 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-11-28T18:20:19.130 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libradosstriper1. 2023-11-28T18:20:19.141 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-11-28T18:20:19.144 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../030-libradosstriper1_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:19.180 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libradosstriper1 (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:19.385 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-openssl. 2023-11-28T18:20:19.400 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2023-11-28T18:20:19.443 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-11-28T18:20:19.591 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-common. 2023-11-28T18:20:19.606 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../031-ceph-common_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:19.649 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-common (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:19.772 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mgr. 2023-11-28T18:20:19.786 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../065-ceph-mgr_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:19.830 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mgr (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:20.208 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mon. 2023-11-28T18:20:20.223 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../066-ceph-mon_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:20.266 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mon (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:20.705 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-base. 2023-11-28T18:20:20.719 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../032-ceph-base_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:20.772 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-base (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:21.003 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-osd. 2023-11-28T18:20:21.018 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../067-ceph-osd_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:21.053 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-osd (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:21.667 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-11-28T18:20:21.682 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-11-28T18:20:21.725 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-11-28T18:20:22.053 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-cheroot. 2023-11-28T18:20:22.068 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-11-28T18:20:22.103 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-28T18:20:22.256 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph. 2023-11-28T18:20:22.271 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../068-ceph_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:22.315 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:22.389 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-11-28T18:20:22.404 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-11-28T18:20:22.439 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-11-28T18:20:22.642 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-fuse. 2023-11-28T18:20:22.657 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../069-ceph-fuse_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:22.700 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-fuse (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:22.725 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-jaraco.text. 2023-11-28T18:20:22.740 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2023-11-28T18:20:22.801 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-11-28T18:20:23.062 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mds. 2023-11-28T18:20:23.077 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../070-ceph-mds_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:23.112 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-11-28T18:20:23.112 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mds (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:23.126 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-11-28T18:20:23.170 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-11-28T18:20:23.431 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-tempora. 2023-11-28T18:20:23.446 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2023-11-28T18:20:23.456 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package cephadm. 2023-11-28T18:20:23.471 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../071-cephadm_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:23.481 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-11-28T18:20:23.515 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking cephadm (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:23.775 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-portend. 2023-11-28T18:20:23.790 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2023-11-28T18:20:23.826 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-asyncssh. 2023-11-28T18:20:23.833 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-11-28T18:20:23.841 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2023-11-28T18:20:23.884 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-11-28T18:20:24.120 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-11-28T18:20:24.135 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2023-11-28T18:20:24.178 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-11-28T18:20:24.212 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python-babel-localedata. 2023-11-28T18:20:24.227 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-11-28T18:20:24.272 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-28T18:20:24.506 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-cherrypy3. 2023-11-28T18:20:24.521 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2023-11-28T18:20:24.564 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-11-28T18:20:24.951 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-natsort. 2023-11-28T18:20:24.966 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2023-11-28T18:20:25.017 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-11-28T18:20:25.276 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-babel. 2023-11-28T18:20:25.291 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-11-28T18:20:25.335 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-11-28T18:20:25.429 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-logutils. 2023-11-28T18:20:25.444 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2023-11-28T18:20:25.495 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-11-28T18:20:25.654 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-jinja2. 2023-11-28T18:20:25.668 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1_all.deb ... 2023-11-28T18:20:25.712 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-11-28T18:20:25.856 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-markupsafe. 2023-11-28T18:20:25.871 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-11-28T18:20:25.915 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-11-28T18:20:26.032 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-11-28T18:20:26.046 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_18.0.0-7531-g47c1c547-1jammy_all.deb ... 2023-11-28T18:20:26.090 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mgr-cephadm (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:26.285 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-mako. 2023-11-28T18:20:26.299 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-11-28T18:20:26.351 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-28T18:20:26.437 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-repoze.lru. 2023-11-28T18:20:26.450 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2023-11-28T18:20:26.493 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-11-28T18:20:26.763 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-simplegeneric. 2023-11-28T18:20:26.778 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2023-11-28T18:20:26.779 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-routes. 2023-11-28T18:20:26.793 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-11-28T18:20:26.838 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-11-28T18:20:26.854 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-11-28T18:20:27.174 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-11-28T18:20:27.188 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_18.0.0-7531-g47c1c547-1jammy_all.deb ... 2023-11-28T18:20:27.232 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-singledispatch. 2023-11-28T18:20:27.232 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mgr-dashboard (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:27.247 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-11-28T18:20:27.298 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-11-28T18:20:27.656 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-11-28T18:20:27.669 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-11-28T18:20:27.676 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-webob. 2023-11-28T18:20:27.691 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-11-28T18:20:27.711 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-28T18:20:27.734 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-11-28T18:20:28.154 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-waitress. 2023-11-28T18:20:28.170 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-11-28T18:20:28.215 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-28T18:20:28.431 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-joblib. 2023-11-28T18:20:28.445 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-11-28T18:20:28.490 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-11-28T18:20:28.599 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-tempita. 2023-11-28T18:20:28.614 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-11-28T18:20:28.666 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-11-28T18:20:28.875 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-11-28T18:20:28.890 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-11-28T18:20:29.021 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-11-28T18:20:29.036 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-paste. 2023-11-28T18:20:29.050 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-11-28T18:20:29.101 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-11-28T18:20:29.315 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-sklearn. 2023-11-28T18:20:29.330 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-11-28T18:20:29.364 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-28T18:20:29.463 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-11-28T18:20:29.543 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-11-28T18:20:29.543 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-11-28T18:20:29.808 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-pastedeploy. 2023-11-28T18:20:29.823 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2023-11-28T18:20:29.865 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-11-28T18:20:30.023 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-11-28T18:20:30.037 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_18.0.0-7531-g47c1c547-1jammy_all.deb ... 2023-11-28T18:20:30.107 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:30.169 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-webtest. 2023-11-28T18:20:30.184 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2023-11-28T18:20:30.218 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-11-28T18:20:30.505 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-pecan. 2023-11-28T18:20:30.520 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-11-28T18:20:30.563 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-11-28T18:20:30.959 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-certifi. 2023-11-28T18:20:30.975 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2023-11-28T18:20:31.017 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-11-28T18:20:31.052 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-cachetools. 2023-11-28T18:20:31.067 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2023-11-28T18:20:31.110 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-11-28T18:20:31.354 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-idna. 2023-11-28T18:20:31.369 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2023-11-28T18:20:31.412 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-idna (3.3-1) ... 2023-11-28T18:20:31.421 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pyasn1. 2023-11-28T18:20:31.436 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2023-11-28T18:20:31.479 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-11-28T18:20:31.791 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-11-28T18:20:31.806 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-11-28T18:20:31.840 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-urllib3. 2023-11-28T18:20:31.849 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-11-28T18:20:31.855 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2023-11-28T18:20:31.890 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-11-28T18:20:32.159 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-requests. 2023-11-28T18:20:32.175 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-11-28T18:20:32.277 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-rsa. 2023-11-28T18:20:32.291 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2023-11-28T18:20:32.293 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-28T18:20:32.335 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-rsa (4.8-1) ... 2023-11-28T18:20:32.621 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-werkzeug. 2023-11-28T18:20:32.636 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-11-28T18:20:32.638 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-google-auth. 2023-11-28T18:20:32.652 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2023-11-28T18:20:32.679 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-28T18:20:32.696 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-11-28T18:20:32.949 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-11-28T18:20:32.963 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_18.0.0-7531-g47c1c547-1jammy_all.deb ... 2023-11-28T18:20:32.998 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-mgr-modules-core (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:33.017 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-11-28T18:20:33.031 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-11-28T18:20:33.075 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-28T18:20:33.386 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-websocket. 2023-11-28T18:20:33.395 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-11-28T18:20:33.401 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2023-11-28T18:20:33.409 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:33.444 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-11-28T18:20:33.453 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:33.731 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-bcrypt. 2023-11-28T18:20:33.745 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-11-28T18:20:33.781 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-11-28T18:20:33.781 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-kubernetes. 2023-11-28T18:20:33.797 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-11-28T18:20:33.848 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-28T18:20:34.076 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-openssl. 2023-11-28T18:20:34.089 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2023-11-28T18:20:34.133 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-11-28T18:20:34.462 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-mgr. 2023-11-28T18:20:34.475 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../065-ceph-mgr_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:34.486 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-11-28T18:20:34.501 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_18.0.0-7531-g47c1c547-1jammy_all.deb ... 2023-11-28T18:20:34.519 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-mgr (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:34.536 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:34.814 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libonig5:amd64. 2023-11-28T18:20:34.828 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-11-28T18:20:34.872 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-28T18:20:34.939 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-mon. 2023-11-28T18:20:34.953 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../066-ceph-mon_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:34.998 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-mon (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:35.242 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libjq1:amd64. 2023-11-28T18:20:35.256 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-11-28T18:20:35.301 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-28T18:20:35.561 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package jq. 2023-11-28T18:20:35.575 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-11-28T18:20:35.619 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-11-28T18:20:35.743 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-osd. 2023-11-28T18:20:35.757 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../067-ceph-osd_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:35.793 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-osd (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:35.847 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package socat. 2023-11-28T18:20:35.862 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-11-28T18:20:35.897 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-11-28T18:20:36.209 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package xmlstarlet. 2023-11-28T18:20:36.223 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-11-28T18:20:36.267 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-11-28T18:20:36.588 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-test. 2023-11-28T18:20:36.602 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../099-ceph-test_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:36.637 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-test (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:36.997 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph. 2023-11-28T18:20:37.011 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../068-ceph_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:37.054 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:37.356 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-fuse. 2023-11-28T18:20:37.372 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../069-ceph-fuse_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:37.431 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-fuse (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:37.743 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-mds. 2023-11-28T18:20:37.758 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../070-ceph-mds_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:37.793 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-mds (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:38.138 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package cephadm. 2023-11-28T18:20:38.152 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../071-cephadm_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:38.229 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking cephadm (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:38.532 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-asyncssh. 2023-11-28T18:20:38.547 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2023-11-28T18:20:38.590 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-11-28T18:20:38.692 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-volume. 2023-11-28T18:20:38.707 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../100-ceph-volume_18.0.0-7531-g47c1c547-1jammy_all.deb ... 2023-11-28T18:20:38.750 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-volume (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:38.892 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python-babel-localedata. 2023-11-28T18:20:38.908 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-11-28T18:20:38.950 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-28T18:20:39.129 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package cephfs-mirror. 2023-11-28T18:20:39.145 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../101-cephfs-mirror_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:39.187 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking cephfs-mirror (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:39.507 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-colorama. 2023-11-28T18:20:39.523 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2023-11-28T18:20:39.565 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-11-28T18:20:39.819 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package cephfs-shell. 2023-11-28T18:20:39.833 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../103-cephfs-shell_18.0.0-7531-g47c1c547-1jammy_all.deb ... 2023-11-28T18:20:39.893 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking cephfs-shell (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:39.906 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-babel. 2023-11-28T18:20:39.920 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-11-28T18:20:39.958 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-11-28T18:20:40.103 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package cephfs-top. 2023-11-28T18:20:40.119 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../104-cephfs-top_18.0.0-7531-g47c1c547-1jammy_all.deb ... 2023-11-28T18:20:40.162 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking cephfs-top (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:40.233 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-jinja2. 2023-11-28T18:20:40.247 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1_all.deb ... 2023-11-28T18:20:40.291 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-11-28T18:20:40.414 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libcephfs-dev. 2023-11-28T18:20:40.430 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../105-libcephfs-dev_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:40.472 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libcephfs-dev (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:40.627 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-11-28T18:20:40.642 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_18.0.0-7531-g47c1c547-1jammy_all.deb ... 2023-11-28T18:20:40.686 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-mgr-cephadm (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:40.767 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package lua-socket:amd64. 2023-11-28T18:20:40.782 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2023-11-28T18:20:40.817 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2023-11-28T18:20:40.972 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-repoze.lru. 2023-11-28T18:20:40.986 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2023-11-28T18:20:41.031 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-11-28T18:20:41.138 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package lua-sec:amd64. 2023-11-28T18:20:41.154 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2023-11-28T18:20:41.196 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2023-11-28T18:20:41.333 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-routes. 2023-11-28T18:20:41.347 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-11-28T18:20:41.425 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-11-28T18:20:41.558 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package nvme-cli. 2023-11-28T18:20:41.574 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-11-28T18:20:41.617 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-11-28T18:20:41.745 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-11-28T18:20:41.759 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_18.0.0-7531-g47c1c547-1jammy_all.deb ... 2023-11-28T18:20:41.803 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-mgr-dashboard (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:42.030 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package pkg-config. 2023-11-28T18:20:42.045 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2023-11-28T18:20:42.088 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2023-11-28T18:20:42.383 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-11-28T18:20:42.399 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-11-28T18:20:42.441 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-11-28T18:20:42.448 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-11-28T18:20:42.462 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-11-28T18:20:42.498 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-28T18:20:42.886 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-iniconfig. 2023-11-28T18:20:42.901 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2023-11-28T18:20:42.934 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-joblib. 2023-11-28T18:20:42.944 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-11-28T18:20:42.949 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-11-28T18:20:42.993 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-11-28T18:20:43.247 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pastescript. 2023-11-28T18:20:43.262 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2023-11-28T18:20:43.305 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-11-28T18:20:43.363 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-11-28T18:20:43.377 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-11-28T18:20:43.506 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-11-28T18:20:43.667 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pluggy. 2023-11-28T18:20:43.682 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2023-11-28T18:20:43.725 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-11-28T18:20:43.776 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-sklearn. 2023-11-28T18:20:43.790 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-11-28T18:20:43.867 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-28T18:20:44.011 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-psutil. 2023-11-28T18:20:44.027 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-11-28T18:20:44.062 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-11-28T18:20:44.406 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-py. 2023-11-28T18:20:44.421 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2023-11-28T18:20:44.463 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-py (1.10.0-1) ... 2023-11-28T18:20:44.518 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-11-28T18:20:44.532 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_18.0.0-7531-g47c1c547-1jammy_all.deb ... 2023-11-28T18:20:44.568 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:44.793 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pygments. 2023-11-28T18:20:44.808 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-11-28T18:20:44.850 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-11-28T18:20:45.313 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pyinotify. 2023-11-28T18:20:45.327 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-11-28T18:20:45.370 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-11-28T18:20:45.571 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-cachetools. 2023-11-28T18:20:45.586 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2023-11-28T18:20:45.629 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-11-28T18:20:45.673 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-toml. 2023-11-28T18:20:45.689 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2023-11-28T18:20:45.731 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-11-28T18:20:45.916 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-pyasn1. 2023-11-28T18:20:45.931 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2023-11-28T18:20:45.974 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-11-28T18:20:46.026 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pytest. 2023-11-28T18:20:46.041 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-11-28T18:20:46.084 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-11-28T18:20:46.327 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-11-28T18:20:46.343 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-11-28T18:20:46.385 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-11-28T18:20:46.421 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-simplejson. 2023-11-28T18:20:46.436 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-11-28T18:20:46.479 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-11-28T18:20:46.696 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-rsa. 2023-11-28T18:20:46.712 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2023-11-28T18:20:46.749 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-11-28T18:20:46.754 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-rsa (4.8-1) ... 2023-11-28T18:20:46.763 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-11-28T18:20:46.815 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-11-28T18:20:47.075 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-google-auth. 2023-11-28T18:20:47.091 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2023-11-28T18:20:47.125 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-11-28T18:20:47.411 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-11-28T18:20:47.427 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-11-28T18:20:47.452 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package radosgw. 2023-11-28T18:20:47.467 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../122-radosgw_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:47.469 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-28T18:20:47.511 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking radosgw (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:47.781 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-websocket. 2023-11-28T18:20:47.796 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2023-11-28T18:20:47.864 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-11-28T18:20:48.097 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package rbd-fuse. 2023-11-28T18:20:48.111 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../123-rbd-fuse_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:48.155 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking rbd-fuse (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:48.194 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-kubernetes. 2023-11-28T18:20:48.208 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-11-28T18:20:48.250 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-28T18:20:48.456 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-11-28T18:20:48.827 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-28T18:20:48.914 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-11-28T18:20:48.929 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_18.0.0-7531-g47c1c547-1jammy_all.deb ... 2023-11-28T18:20:48.953 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-11-28T18:20:48.972 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:49.184 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-11-28T18:20:49.291 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libonig5:amd64. 2023-11-28T18:20:49.306 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-11-28T18:20:49.350 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-28T18:20:49.589 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-11-28T18:20:49.703 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libjq1:amd64. 2023-11-28T18:20:49.717 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-11-28T18:20:49.761 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-28T18:20:50.005 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package jq. 2023-11-28T18:20:50.020 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-11-28T18:20:50.055 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-11-28T18:20:50.197 INFO:teuthology.orchestra.run.smithi105.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-11-28T18:20:50.239 INFO:teuthology.orchestra.run.smithi105.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2023-11-28T18:20:50.280 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up cephadm (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:50.333 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package socat. 2023-11-28T18:20:50.348 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-11-28T18:20:50.391 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-11-28T18:20:50.694 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package xmlstarlet. 2023-11-28T18:20:50.710 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-11-28T18:20:50.752 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-11-28T18:20:50.858 INFO:teuthology.orchestra.run.smithi105.stdout:Adding system user cephadm....done 2023-11-28T18:20:51.011 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-28T18:20:51.132 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-test. 2023-11-28T18:20:51.146 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../099-ceph-test_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:51.190 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-test (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:51.279 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-11-28T18:20:51.530 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-11-28T18:20:51.656 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-11-28T18:20:51.924 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-11-28T18:20:52.183 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-11-28T18:20:52.284 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-py (1.10.0-1) ... 2023-11-28T18:20:52.586 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-11-28T18:20:52.938 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-11-28T18:20:53.061 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ceph-volume. 2023-11-28T18:20:53.076 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../100-ceph-volume_18.0.0-7531-g47c1c547-1jammy_all.deb ... 2023-11-28T18:20:53.110 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ceph-volume (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:53.206 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-11-28T18:20:53.506 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package cephfs-mirror. 2023-11-28T18:20:53.507 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-11-28T18:20:53.522 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../101-cephfs-mirror_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:53.564 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking cephfs-mirror (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:53.783 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up unzip (6.0-26ubuntu3.1) ... 2023-11-28T18:20:53.885 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-colorama. 2023-11-28T18:20:53.900 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2023-11-28T18:20:53.934 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-11-28T18:20:54.060 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-11-28T18:20:54.161 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package cephfs-shell. 2023-11-28T18:20:54.177 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../103-cephfs-shell_18.0.0-7531-g47c1c547-1jammy_all.deb ... 2023-11-28T18:20:54.219 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking cephfs-shell (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:54.402 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-11-28T18:20:54.501 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package cephfs-top. 2023-11-28T18:20:54.515 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../104-cephfs-top_18.0.0-7531-g47c1c547-1jammy_all.deb ... 2023-11-28T18:20:54.557 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking cephfs-top (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:54.696 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-11-28T18:20:54.793 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libcephfs-dev. 2023-11-28T18:20:54.809 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../105-libcephfs-dev_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:20:54.852 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libcephfs-dev (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:54.988 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-ceph-argparse (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:20:55.162 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package lua-socket:amd64. 2023-11-28T18:20:55.178 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2023-11-28T18:20:55.237 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2023-11-28T18:20:55.256 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-28T18:20:55.382 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2023-11-28T18:20:55.516 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-11-28T18:20:55.542 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package lua-sec:amd64. 2023-11-28T18:20:55.558 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2023-11-28T18:20:55.591 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2023-11-28T18:20:55.776 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2023-11-28T18:20:55.885 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-11-28T18:20:55.938 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package nvme-cli. 2023-11-28T18:20:55.953 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-11-28T18:20:56.020 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-11-28T18:20:56.136 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2023-11-28T18:20:56.425 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package pkg-config. 2023-11-28T18:20:56.440 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2023-11-28T18:20:56.483 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2023-11-28T18:20:56.525 INFO:teuthology.orchestra.run.smithi105.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2023-11-28T18:20:56.558 INFO:teuthology.orchestra.run.smithi105.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2023-11-28T18:20:56.631 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-28T18:20:56.744 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-11-28T18:20:56.757 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-11-28T18:20:56.760 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-11-28T18:20:56.828 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-11-28T18:20:57.141 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-28T18:20:57.247 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-iniconfig. 2023-11-28T18:20:57.263 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2023-11-28T18:20:57.267 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-11-28T18:20:57.306 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-11-28T18:20:57.544 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-11-28T18:20:57.634 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-pastescript. 2023-11-28T18:20:57.650 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2023-11-28T18:20:57.692 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-11-28T18:20:57.811 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-11-28T18:20:58.003 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-pluggy. 2023-11-28T18:20:58.019 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2023-11-28T18:20:58.079 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-11-28T18:20:58.137 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up zip (3.0-12build2) ... 2023-11-28T18:20:58.263 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-11-28T18:20:58.348 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-psutil. 2023-11-28T18:20:58.365 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-11-28T18:20:58.423 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-11-28T18:20:58.717 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-py. 2023-11-28T18:20:58.733 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2023-11-28T18:20:58.767 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-py (1.10.0-1) ... 2023-11-28T18:20:58.949 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-11-28T18:20:59.037 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-pygments. 2023-11-28T18:20:59.052 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-11-28T18:20:59.095 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-11-28T18:20:59.217 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-11-28T18:20:59.337 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-11-28T18:20:59.460 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-28T18:20:59.524 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-pyinotify. 2023-11-28T18:20:59.539 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-11-28T18:20:59.582 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-11-28T18:20:59.769 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-11-28T18:20:59.876 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-toml. 2023-11-28T18:20:59.891 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2023-11-28T18:20:59.935 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-11-28T18:21:00.154 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-11-28T18:21:00.262 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-pytest. 2023-11-28T18:21:00.277 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-11-28T18:21:00.312 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-11-28T18:21:00.413 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-11-28T18:21:00.657 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package python3-simplejson. 2023-11-28T18:21:00.672 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-11-28T18:21:00.715 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-11-28T18:21:00.798 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-28T18:21:01.018 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-11-28T18:21:01.033 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-11-28T18:21:01.076 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-11-28T18:21:01.092 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-11-28T18:21:01.418 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-28T18:21:01.755 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package radosgw. 2023-11-28T18:21:01.769 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../122-radosgw_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:21:01.794 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-11-28T18:21:01.813 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking radosgw (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:02.037 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-11-28T18:21:02.162 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-ceph-common (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:02.433 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package rbd-fuse. 2023-11-28T18:21:02.448 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../123-rbd-fuse_18.0.0-7531-g47c1c547-1jammy_amd64.deb ... 2023-11-28T18:21:02.464 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-idna (3.3-1) ... 2023-11-28T18:21:02.483 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking rbd-fuse (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:02.760 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-11-28T18:21:02.806 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-28T18:21:03.030 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-28T18:21:03.155 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-11-28T18:21:03.436 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-11-28T18:21:03.847 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-11-28T18:21:04.271 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2023-11-28T18:21:04.501 INFO:teuthology.orchestra.run.smithi148.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-11-28T18:21:04.514 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T18:21:04.556 INFO:teuthology.orchestra.run.smithi148.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2023-11-28T18:21:04.591 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up cephadm (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:04.649 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-toml (0.10.2-1) ... 2023-11-28T18:21:04.917 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-28T18:21:05.017 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-11-28T18:21:05.143 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-11-28T18:21:05.221 INFO:teuthology.orchestra.run.smithi148.stdout:Adding system user cephadm....done 2023-11-28T18:21:05.467 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-28T18:21:05.470 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-11-28T18:21:05.721 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-11-28T18:21:05.828 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-11-28T18:21:05.980 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T18:21:06.106 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-11-28T18:21:06.191 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-11-28T18:21:06.416 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-11-28T18:21:06.438 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-11-28T18:21:06.684 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-11-28T18:21:06.720 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-11-28T18:21:06.952 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-11-28T18:21:06.997 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-11-28T18:21:07.122 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-py (1.10.0-1) ... 2023-11-28T18:21:07.211 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-11-28T18:21:07.454 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-11-28T18:21:07.482 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-11-28T18:21:07.721 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-28T18:21:07.843 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-11-28T18:21:07.847 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-11-28T18:21:08.094 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-11-28T18:21:08.165 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-28T18:21:08.291 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-28T18:21:08.353 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-11-28T18:21:08.584 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-11-28T18:21:08.654 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up unzip (6.0-26ubuntu3.1) ... 2023-11-28T18:21:08.894 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-11-28T18:21:08.939 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-11-28T18:21:09.153 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2023-11-28T18:21:09.207 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-11-28T18:21:09.254 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2023-11-28T18:21:09.380 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-11-28T18:21:09.468 INFO:teuthology.orchestra.run.smithi105.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-11-28T18:21:09.475 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-11-28T18:21:09.715 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-28T18:21:09.726 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-ceph-argparse (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:09.843 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-11-28T18:21:09.986 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-28T18:21:10.111 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2023-11-28T18:21:10.228 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-11-28T18:21:10.237 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-11-28T18:21:10.505 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2023-11-28T18:21:10.520 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up lua-any (27ubuntu1) ... 2023-11-28T18:21:10.630 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-11-28T18:21:10.646 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-portend (3.0.0-1) ... 2023-11-28T18:21:10.890 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2023-11-28T18:21:10.906 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T18:21:11.031 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-11-28T18:21:11.281 INFO:teuthology.orchestra.run.smithi148.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2023-11-28T18:21:11.329 INFO:teuthology.orchestra.run.smithi148.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2023-11-28T18:21:11.358 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-11-28T18:21:11.402 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-28T18:21:11.528 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-11-28T18:21:11.650 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-28T18:21:11.895 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-28T18:21:11.927 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-11-28T18:21:12.054 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-11-28T18:21:12.146 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-rsa (4.8-1) ... 2023-11-28T18:21:12.339 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-11-28T18:21:12.423 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-11-28T18:21:12.624 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-11-28T18:21:12.699 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-28T18:21:12.992 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-11-28T18:21:13.308 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up zip (3.0-12build2) ... 2023-11-28T18:21:13.377 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-11-28T18:21:13.451 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-11-28T18:21:13.653 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-11-28T18:21:13.997 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-28T18:21:14.123 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up librados2 (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:14.144 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-11-28T18:21:14.249 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libsqlite3-mod-ceph (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:14.375 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2023-11-28T18:21:14.420 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-11-28T18:21:14.546 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-11-28T18:21:14.687 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libcephfs2 (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:14.697 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-28T18:21:14.796 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libradosstriper1 (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:14.922 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-11-28T18:21:15.015 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-11-28T18:21:15.223 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up librbd1 (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:15.324 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mgr-modules-core (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:15.391 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-11-28T18:21:15.433 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-fuse (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:15.651 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-28T18:21:15.651 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-28T18:21:15.676 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-11-28T18:21:16.061 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-28T18:21:16.344 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libcephfs-dev (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:16.370 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-11-28T18:21:16.503 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-rados (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:16.629 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up librgw2 (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:16.696 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-28T18:21:16.755 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up cephfs-top (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:16.880 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-28T18:21:17.082 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-11-28T18:21:17.349 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-11-28T18:21:17.458 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-ceph-common (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:17.743 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-idna (3.3-1) ... 2023-11-28T18:21:18.085 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-28T18:21:18.168 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-rbd (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:18.294 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up rbd-fuse (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:18.419 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-rgw (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:18.545 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-cephfs (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:18.671 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up cephfs-shell (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:18.797 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-common (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:19.316 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2023-11-28T18:21:19.343 INFO:teuthology.orchestra.run.smithi105.stdout:Adding group ceph....done 2023-11-28T18:21:19.551 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T18:21:19.685 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-toml (0.10.2-1) ... 2023-11-28T18:21:19.718 INFO:teuthology.orchestra.run.smithi105.stdout:Adding system user ceph....done 2023-11-28T18:21:19.905 INFO:teuthology.orchestra.run.smithi105.stdout:Setting system user ceph properties....done 2023-11-28T18:21:19.922 INFO:teuthology.orchestra.run.smithi105.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-11-28T18:21:19.962 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-28T18:21:20.064 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-11-28T18:21:20.087 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-11-28T18:21:20.213 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-11-28T18:21:20.433 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-11-28T18:21:20.557 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-11-28T18:21:20.799 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-11-28T18:21:21.034 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T18:21:21.115 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up cephfs-mirror (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:21.176 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-11-28T18:21:21.324 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-28T18:21:21.324 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-28T18:21:21.486 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-11-28T18:21:21.746 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-11-28T18:21:21.958 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-test (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:22.021 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-11-28T18:21:22.084 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up radosgw (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:22.298 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-11-28T18:21:22.558 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-11-28T18:21:22.631 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-28T18:21:22.632 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-28T18:21:22.817 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-28T18:21:22.917 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-11-28T18:21:23.210 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-28T18:21:23.261 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-base (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:23.336 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-28T18:21:23.606 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-11-28T18:21:23.629 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-11-28T18:21:23.947 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-11-28T18:21:24.206 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2023-11-28T18:21:24.279 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mds (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:24.349 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2023-11-28T18:21:24.500 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-11-28T18:21:24.511 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-28T18:21:24.511 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-28T18:21:24.572 INFO:teuthology.orchestra.run.smithi148.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-11-28T18:21:24.810 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-28T18:21:24.936 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-11-28T18:21:25.148 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mgr (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:25.338 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-11-28T18:21:25.360 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-28T18:21:25.361 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-28T18:21:25.614 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up lua-any (27ubuntu1) ... 2023-11-28T18:21:25.740 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-portend (3.0.0-1) ... 2023-11-28T18:21:25.991 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-osd (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:25.999 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T18:21:26.108 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-11-28T18:21:26.344 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-28T18:21:26.344 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-28T18:21:26.418 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-11-28T18:21:26.744 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-28T18:21:26.985 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mgr-k8sevents (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:27.028 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-11-28T18:21:27.085 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:27.203 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mon (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:27.231 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-rsa (4.8-1) ... 2023-11-28T18:21:27.412 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-28T18:21:27.413 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-28T18:21:27.524 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-11-28T18:21:27.793 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-28T18:21:28.071 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mgr-cephadm (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:28.085 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-11-28T18:21:28.205 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:28.331 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mgr-dashboard (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:28.465 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-volume (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:28.470 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-11-28T18:21:28.772 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-11-28T18:21:28.835 INFO:teuthology.orchestra.run.smithi105.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2023-11-28T18:21:28.944 INFO:teuthology.orchestra.run.smithi105.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-28T18:21:29.121 INFO:teuthology.orchestra.run.smithi105.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-28T18:21:29.157 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-28T18:21:29.275 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up librados2 (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:29.376 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libsqlite3-mod-ceph (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:29.502 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2023-11-28T18:21:29.847 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libcephfs2 (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:30.014 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libradosstriper1 (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:30.141 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-11-28T18:21:30.297 INFO:teuthology.orchestra.run.smithi105.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-11-28T18:21:30.417 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up librbd1 (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:30.518 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-mgr-modules-core (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:30.644 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-fuse (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:30.879 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-28T18:21:30.879 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-28T18:21:30.972 INFO:teuthology.orchestra.run.smithi105.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-28T18:21:30.977 DEBUG:teuthology.orchestra.run.smithi105:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-11-28T18:21:31.053 INFO:teuthology.orchestra.run.smithi105.stdout:Reading package lists... 2023-11-28T18:21:31.218 INFO:teuthology.orchestra.run.smithi105.stdout:Building dependency tree... 2023-11-28T18:21:31.219 INFO:teuthology.orchestra.run.smithi105.stdout:Reading state information... 2023-11-28T18:21:31.368 INFO:teuthology.orchestra.run.smithi105.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-11-28T18:21:31.368 INFO:teuthology.orchestra.run.smithi105.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-11-28T18:21:31.368 INFO:teuthology.orchestra.run.smithi105.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-11-28T18:21:31.368 INFO:teuthology.orchestra.run.smithi105.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-11-28T18:21:31.368 INFO:teuthology.orchestra.run.smithi105.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-11-28T18:21:31.368 INFO:teuthology.orchestra.run.smithi105.stdout:The following packages were automatically installed and are no longer required: 2023-11-28T18:21:31.368 INFO:teuthology.orchestra.run.smithi105.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-28T18:21:31.368 INFO:teuthology.orchestra.run.smithi105.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-28T18:21:31.368 INFO:teuthology.orchestra.run.smithi105.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-28T18:21:31.368 INFO:teuthology.orchestra.run.smithi105.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-28T18:21:31.369 INFO:teuthology.orchestra.run.smithi105.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-28T18:21:31.369 INFO:teuthology.orchestra.run.smithi105.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-28T18:21:31.370 INFO:teuthology.orchestra.run.smithi105.stdout:The following additional packages will be installed: 2023-11-28T18:21:31.370 INFO:teuthology.orchestra.run.smithi105.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-11-28T18:21:31.370 INFO:teuthology.orchestra.run.smithi105.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-11-28T18:21:31.370 INFO:teuthology.orchestra.run.smithi105.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-11-28T18:21:31.370 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-11-28T18:21:31.370 INFO:teuthology.orchestra.run.smithi105.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-28T18:21:31.371 INFO:teuthology.orchestra.run.smithi105.stdout:Suggested packages: 2023-11-28T18:21:31.371 INFO:teuthology.orchestra.run.smithi105.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-11-28T18:21:31.371 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-11-28T18:21:31.371 INFO:teuthology.orchestra.run.smithi105.stdout: libengine-pkcs11-openssl 2023-11-28T18:21:31.419 INFO:teuthology.orchestra.run.smithi105.stdout:The following NEW packages will be installed: 2023-11-28T18:21:31.420 INFO:teuthology.orchestra.run.smithi105.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-11-28T18:21:31.420 INFO:teuthology.orchestra.run.smithi105.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-11-28T18:21:31.421 INFO:teuthology.orchestra.run.smithi105.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-11-28T18:21:31.421 INFO:teuthology.orchestra.run.smithi105.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-11-28T18:21:31.421 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-11-28T18:21:31.421 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-11-28T18:21:31.421 INFO:teuthology.orchestra.run.smithi105.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-28T18:21:31.421 INFO:teuthology.orchestra.run.smithi105.stdout:The following packages will be upgraded: 2023-11-28T18:21:31.422 INFO:teuthology.orchestra.run.smithi105.stdout: libssl3 2023-11-28T18:21:31.537 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libcephfs-dev (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:31.607 INFO:teuthology.orchestra.run.smithi105.stdout:1 upgraded, 39 newly installed, 0 to remove and 263 not upgraded. 2023-11-28T18:21:31.607 INFO:teuthology.orchestra.run.smithi105.stdout:Need to get 54.6 MB of archives. 2023-11-28T18:21:31.607 INFO:teuthology.orchestra.run.smithi105.stdout:After this operation, 202 MB of additional disk space will be used. 2023-11-28T18:21:31.607 INFO:teuthology.orchestra.run.smithi105.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-11-28T18:21:31.663 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-rados (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:31.789 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up librgw2 (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:31.916 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up cephfs-top (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:32.041 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-28T18:21:32.103 INFO:teuthology.orchestra.run.smithi105.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-11-28T18:21:32.361 INFO:teuthology.orchestra.run.smithi105.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-11-28T18:21:32.395 INFO:teuthology.orchestra.run.smithi105.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-11-28T18:21:32.397 INFO:teuthology.orchestra.run.smithi105.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-11-28T18:21:32.402 INFO:teuthology.orchestra.run.smithi105.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-11-28T18:21:32.405 INFO:teuthology.orchestra.run.smithi105.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-11-28T18:21:32.406 INFO:teuthology.orchestra.run.smithi105.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-11-28T18:21:32.410 INFO:teuthology.orchestra.run.smithi105.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-11-28T18:21:32.412 INFO:teuthology.orchestra.run.smithi105.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-11-28T18:21:32.578 INFO:teuthology.orchestra.run.smithi105.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-11-28T18:21:32.578 INFO:teuthology.orchestra.run.smithi105.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-11-28T18:21:32.830 INFO:teuthology.orchestra.run.smithi105.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-11-28T18:21:33.336 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-rbd (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:33.462 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up rbd-fuse (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:33.588 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-rgw (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:33.714 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up python3-cephfs (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:33.837 INFO:teuthology.orchestra.run.smithi105.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-11-28T18:21:33.839 INFO:teuthology.orchestra.run.smithi105.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-11-28T18:21:33.839 INFO:teuthology.orchestra.run.smithi105.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-11-28T18:21:33.848 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up cephfs-shell (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:33.866 INFO:teuthology.orchestra.run.smithi105.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-11-28T18:21:33.866 INFO:teuthology.orchestra.run.smithi105.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-11-28T18:21:33.871 INFO:teuthology.orchestra.run.smithi105.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-11-28T18:21:33.873 INFO:teuthology.orchestra.run.smithi105.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-11-28T18:21:33.879 INFO:teuthology.orchestra.run.smithi105.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-11-28T18:21:33.999 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-common (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:34.043 INFO:teuthology.orchestra.run.smithi105.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-11-28T18:21:34.520 INFO:teuthology.orchestra.run.smithi148.stdout:Adding group ceph....done 2023-11-28T18:21:34.787 INFO:teuthology.orchestra.run.smithi105.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-11-28T18:21:34.788 INFO:teuthology.orchestra.run.smithi105.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-11-28T18:21:34.814 INFO:teuthology.orchestra.run.smithi105.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-11-28T18:21:34.891 INFO:teuthology.orchestra.run.smithi105.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-11-28T18:21:34.895 INFO:teuthology.orchestra.run.smithi148.stdout:Adding system user ceph....done 2023-11-28T18:21:34.899 INFO:teuthology.orchestra.run.smithi105.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-11-28T18:21:34.900 INFO:teuthology.orchestra.run.smithi105.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-11-28T18:21:34.901 INFO:teuthology.orchestra.run.smithi105.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-11-28T18:21:34.902 INFO:teuthology.orchestra.run.smithi105.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-11-28T18:21:34.930 INFO:teuthology.orchestra.run.smithi105.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-11-28T18:21:34.933 INFO:teuthology.orchestra.run.smithi105.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-11-28T18:21:35.065 INFO:teuthology.orchestra.run.smithi148.stdout:Setting system user ceph properties....done 2023-11-28T18:21:35.083 INFO:teuthology.orchestra.run.smithi148.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-11-28T18:21:35.223 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-11-28T18:21:35.558 INFO:teuthology.orchestra.run.smithi105.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-11-28T18:21:35.599 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-11-28T18:21:35.641 INFO:teuthology.orchestra.run.smithi105.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-11-28T18:21:35.641 INFO:teuthology.orchestra.run.smithi105.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-11-28T18:21:35.641 INFO:teuthology.orchestra.run.smithi105.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-11-28T18:21:35.730 INFO:teuthology.orchestra.run.smithi105.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-11-28T18:21:35.741 INFO:teuthology.orchestra.run.smithi105.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-11-28T18:21:36.201 INFO:teuthology.orchestra.run.smithi105.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-11-28T18:21:36.202 INFO:teuthology.orchestra.run.smithi105.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-11-28T18:21:36.267 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up cephfs-mirror (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:36.501 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-28T18:21:36.502 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-28T18:21:36.593 INFO:teuthology.orchestra.run.smithi105.stdout:Preconfiguring packages ... 2023-11-28T18:21:36.747 INFO:teuthology.orchestra.run.smithi105.stdout:Fetched 54.6 MB in 5s (11.4 MB/s) 2023-11-28T18:21:36.897 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package flex. 2023-11-28T18:21:36.970 INFO:teuthology.orchestra.run.smithi105.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140566 files and directories currently installed.) 2023-11-28T18:21:36.974 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-11-28T18:21:37.012 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking flex (2.6.4-8build2) ... 2023-11-28T18:21:37.210 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-test (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:37.311 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up radosgw (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:37.565 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-28T18:21:37.743 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-11-28T18:21:37.795 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-28T18:21:37.795 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-28T18:21:38.117 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-28T18:21:38.405 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-base (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:38.621 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package bison. 2023-11-28T18:21:38.689 INFO:teuthology.orchestra.run.smithi105.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140639 files and directories currently installed.) 2023-11-28T18:21:38.693 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-11-28T18:21:38.738 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-11-28T18:21:38.776 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-11-28T18:21:39.121 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-11-28T18:21:39.135 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-11-28T18:21:39.179 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-11-28T18:21:39.449 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package dump. 2023-11-28T18:21:39.457 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-mds (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:39.463 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-11-28T18:21:39.532 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking dump (0.4b47-3) ... 2023-11-28T18:21:39.704 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-28T18:21:39.704 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-28T18:21:39.811 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package indent. 2023-11-28T18:21:39.826 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-11-28T18:21:39.869 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-28T18:21:40.180 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-11-28T18:21:40.196 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-11-28T18:21:40.239 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-28T18:21:40.343 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-mgr (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:40.550 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-11-28T18:21:40.555 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-28T18:21:40.555 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-28T18:21:40.566 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-11-28T18:21:40.620 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-28T18:21:40.886 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-11-28T18:21:40.902 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-11-28T18:21:40.936 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-11-28T18:21:41.144 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-osd (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:41.256 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libfl2:amd64. 2023-11-28T18:21:41.271 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-11-28T18:21:41.314 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-11-28T18:21:41.443 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-28T18:21:41.443 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-28T18:21:41.567 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-11-28T18:21:41.583 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-11-28T18:21:41.625 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-28T18:21:41.895 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libjson-perl. 2023-11-28T18:21:41.910 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-11-28T18:21:41.978 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-11-28T18:21:42.062 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-mgr-k8sevents (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:42.188 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:42.256 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libllvm14:amd64. 2023-11-28T18:21:42.271 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-11-28T18:21:42.305 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-28T18:21:42.348 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-mon (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:42.556 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-28T18:21:42.557 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-28T18:21:43.182 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-mgr-cephadm (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:43.325 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:43.442 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-mgr-dashboard (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:43.602 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ceph-volume (18.0.0-7531-g47c1c547-1jammy) ... 2023-11-28T18:21:43.683 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-11-28T18:21:43.699 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-28T18:21:43.741 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-28T18:21:43.938 INFO:teuthology.orchestra.run.smithi148.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2023-11-28T18:21:44.011 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libmbim-proxy. 2023-11-28T18:21:44.027 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-28T18:21:44.038 INFO:teuthology.orchestra.run.smithi148.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-28T18:21:44.069 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-28T18:21:44.207 INFO:teuthology.orchestra.run.smithi148.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-28T18:21:44.396 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-11-28T18:21:44.412 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-28T18:21:44.455 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-28T18:21:44.766 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libndp0:amd64. 2023-11-28T18:21:44.781 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-11-28T18:21:44.824 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-28T18:21:45.152 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libnm0:amd64. 2023-11-28T18:21:45.167 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-11-28T18:21:45.273 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-28T18:21:45.614 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libpq5:amd64. 2023-11-28T18:21:45.629 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-28T18:21:45.660 INFO:teuthology.orchestra.run.smithi148.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-11-28T18:21:45.672 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T18:21:46.000 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-11-28T18:21:46.015 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-28T18:21:46.058 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-28T18:21:46.336 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libqmi-proxy. 2023-11-28T18:21:46.351 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-28T18:21:46.360 INFO:teuthology.orchestra.run.smithi148.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-28T18:21:46.365 DEBUG:teuthology.orchestra.run.smithi148:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-11-28T18:21:46.394 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-28T18:21:46.440 INFO:teuthology.orchestra.run.smithi148.stdout:Reading package lists... 2023-11-28T18:21:46.614 INFO:teuthology.orchestra.run.smithi148.stdout:Building dependency tree... 2023-11-28T18:21:46.617 INFO:teuthology.orchestra.run.smithi148.stdout:Reading state information... 2023-11-28T18:21:46.638 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-11-28T18:21:46.653 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-28T18:21:46.696 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-28T18:21:46.767 INFO:teuthology.orchestra.run.smithi148.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-11-28T18:21:46.767 INFO:teuthology.orchestra.run.smithi148.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-11-28T18:21:46.767 INFO:teuthology.orchestra.run.smithi148.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-11-28T18:21:46.767 INFO:teuthology.orchestra.run.smithi148.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-11-28T18:21:46.767 INFO:teuthology.orchestra.run.smithi148.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-11-28T18:21:46.767 INFO:teuthology.orchestra.run.smithi148.stdout:The following packages were automatically installed and are no longer required: 2023-11-28T18:21:46.767 INFO:teuthology.orchestra.run.smithi148.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-28T18:21:46.767 INFO:teuthology.orchestra.run.smithi148.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-28T18:21:46.768 INFO:teuthology.orchestra.run.smithi148.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-28T18:21:46.768 INFO:teuthology.orchestra.run.smithi148.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-28T18:21:46.769 INFO:teuthology.orchestra.run.smithi148.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-28T18:21:46.769 INFO:teuthology.orchestra.run.smithi148.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-28T18:21:46.769 INFO:teuthology.orchestra.run.smithi148.stdout:The following additional packages will be installed: 2023-11-28T18:21:46.769 INFO:teuthology.orchestra.run.smithi148.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-11-28T18:21:46.769 INFO:teuthology.orchestra.run.smithi148.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-11-28T18:21:46.769 INFO:teuthology.orchestra.run.smithi148.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-11-28T18:21:46.769 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-11-28T18:21:46.769 INFO:teuthology.orchestra.run.smithi148.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-28T18:21:46.770 INFO:teuthology.orchestra.run.smithi148.stdout:Suggested packages: 2023-11-28T18:21:46.770 INFO:teuthology.orchestra.run.smithi148.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-11-28T18:21:46.770 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-11-28T18:21:46.770 INFO:teuthology.orchestra.run.smithi148.stdout: libengine-pkcs11-openssl 2023-11-28T18:21:46.819 INFO:teuthology.orchestra.run.smithi148.stdout:The following NEW packages will be installed: 2023-11-28T18:21:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-11-28T18:21:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-11-28T18:21:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-11-28T18:21:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-11-28T18:21:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-11-28T18:21:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-11-28T18:21:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-28T18:21:46.822 INFO:teuthology.orchestra.run.smithi148.stdout:The following packages will be upgraded: 2023-11-28T18:21:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: libssl3 2023-11-28T18:21:46.878 INFO:teuthology.orchestra.run.smithi148.stdout:1 upgraded, 39 newly installed, 0 to remove and 263 not upgraded. 2023-11-28T18:21:46.878 INFO:teuthology.orchestra.run.smithi148.stdout:Need to get 54.6 MB of archives. 2023-11-28T18:21:46.878 INFO:teuthology.orchestra.run.smithi148.stdout:After this operation, 202 MB of additional disk space will be used. 2023-11-28T18:21:46.878 INFO:teuthology.orchestra.run.smithi148.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-11-28T18:21:46.985 INFO:teuthology.orchestra.run.smithi148.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-11-28T18:21:47.041 INFO:teuthology.orchestra.run.smithi148.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-11-28T18:21:47.049 INFO:teuthology.orchestra.run.smithi148.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-11-28T18:21:47.049 INFO:teuthology.orchestra.run.smithi148.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-11-28T18:21:47.051 INFO:teuthology.orchestra.run.smithi148.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-11-28T18:21:47.052 INFO:teuthology.orchestra.run.smithi148.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-11-28T18:21:47.052 INFO:teuthology.orchestra.run.smithi148.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-11-28T18:21:47.054 INFO:teuthology.orchestra.run.smithi148.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-11-28T18:21:47.054 INFO:teuthology.orchestra.run.smithi148.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-11-28T18:21:47.055 INFO:teuthology.orchestra.run.smithi148.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-11-28T18:21:47.060 INFO:teuthology.orchestra.run.smithi148.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-11-28T18:21:47.072 INFO:teuthology.orchestra.run.smithi148.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-11-28T18:21:47.200 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-11-28T18:21:47.214 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-11-28T18:21:47.258 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-11-28T18:21:47.284 INFO:teuthology.orchestra.run.smithi148.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-11-28T18:21:47.286 INFO:teuthology.orchestra.run.smithi148.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-11-28T18:21:47.286 INFO:teuthology.orchestra.run.smithi148.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-11-28T18:21:47.289 INFO:teuthology.orchestra.run.smithi148.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-11-28T18:21:47.289 INFO:teuthology.orchestra.run.smithi148.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-11-28T18:21:47.294 INFO:teuthology.orchestra.run.smithi148.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-11-28T18:21:47.295 INFO:teuthology.orchestra.run.smithi148.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-11-28T18:21:47.302 INFO:teuthology.orchestra.run.smithi148.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-11-28T18:21:47.303 INFO:teuthology.orchestra.run.smithi148.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-11-28T18:21:47.324 INFO:teuthology.orchestra.run.smithi148.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-11-28T18:21:47.324 INFO:teuthology.orchestra.run.smithi148.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-11-28T18:21:47.335 INFO:teuthology.orchestra.run.smithi148.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-11-28T18:21:47.355 INFO:teuthology.orchestra.run.smithi148.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-11-28T18:21:47.359 INFO:teuthology.orchestra.run.smithi148.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-11-28T18:21:47.359 INFO:teuthology.orchestra.run.smithi148.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-11-28T18:21:47.360 INFO:teuthology.orchestra.run.smithi148.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-11-28T18:21:47.360 INFO:teuthology.orchestra.run.smithi148.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-11-28T18:21:47.372 INFO:teuthology.orchestra.run.smithi148.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-11-28T18:21:47.374 INFO:teuthology.orchestra.run.smithi148.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-11-28T18:21:47.521 INFO:teuthology.orchestra.run.smithi148.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-11-28T18:21:47.521 INFO:teuthology.orchestra.run.smithi148.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-11-28T18:21:47.521 INFO:teuthology.orchestra.run.smithi148.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-11-28T18:21:47.521 INFO:teuthology.orchestra.run.smithi148.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-11-28T18:21:47.522 INFO:teuthology.orchestra.run.smithi148.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-11-28T18:21:47.522 INFO:teuthology.orchestra.run.smithi148.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-11-28T18:21:47.535 INFO:teuthology.orchestra.run.smithi148.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-11-28T18:21:47.535 INFO:teuthology.orchestra.run.smithi148.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-11-28T18:21:47.561 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package modemmanager. 2023-11-28T18:21:47.576 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-28T18:21:47.619 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-28T18:21:47.917 INFO:teuthology.orchestra.run.smithi148.stdout:Preconfiguring packages ... 2023-11-28T18:21:48.074 INFO:teuthology.orchestra.run.smithi148.stdout:Fetched 54.6 MB in 1s (77.4 MB/s) 2023-11-28T18:21:48.233 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package flex. 2023-11-28T18:21:48.258 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package network-manager. 2023-11-28T18:21:48.274 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-11-28T18:21:48.304 INFO:teuthology.orchestra.run.smithi148.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140566 files and directories currently installed.) 2023-11-28T18:21:48.307 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-11-28T18:21:48.324 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-11-28T18:21:48.356 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking flex (2.6.4-8build2) ... 2023-11-28T18:21:48.915 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ppp. 2023-11-28T18:21:48.930 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-11-28T18:21:49.001 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-28T18:21:49.204 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-11-28T18:21:49.277 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-11-28T18:21:49.586 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-28T18:21:49.603 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package pptp-linux. 2023-11-28T18:21:49.619 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-11-28T18:21:49.654 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-11-28T18:21:49.880 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package network-manager-pptp. 2023-11-28T18:21:49.895 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-11-28T18:21:49.939 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-11-28T18:21:50.163 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package bison. 2023-11-28T18:21:50.236 INFO:teuthology.orchestra.run.smithi148.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140639 files and directories currently installed.) 2023-11-28T18:21:50.240 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-11-28T18:21:50.241 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package postgresql-client-common. 2023-11-28T18:21:50.256 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-11-28T18:21:50.299 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-11-28T18:21:50.299 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking postgresql-client-common (238) ... 2023-11-28T18:21:50.813 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package postgresql-client-14. 2023-11-28T18:21:50.829 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-28T18:21:50.871 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T18:21:50.876 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-11-28T18:21:50.890 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-11-28T18:21:51.010 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-11-28T18:21:51.346 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package dump. 2023-11-28T18:21:51.360 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-11-28T18:21:51.404 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking dump (0.4b47-3) ... 2023-11-28T18:21:51.451 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package postgresql-common. 2023-11-28T18:21:51.467 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-11-28T18:21:51.527 INFO:teuthology.orchestra.run.smithi105.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-11-28T18:21:51.559 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking postgresql-common (238) ... 2023-11-28T18:21:51.799 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package indent. 2023-11-28T18:21:51.814 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-11-28T18:21:51.849 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-28T18:21:52.077 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-11-28T18:21:52.083 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package postgresql-14. 2023-11-28T18:21:52.092 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-11-28T18:21:52.099 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-28T18:21:52.135 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-28T18:21:52.136 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T18:21:52.488 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-11-28T18:21:52.503 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-11-28T18:21:52.547 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-28T18:21:52.808 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-11-28T18:21:52.822 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-11-28T18:21:52.866 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-11-28T18:21:53.186 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libfl2:amd64. 2023-11-28T18:21:53.200 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-11-28T18:21:53.236 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-11-28T18:21:53.244 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package postgresql. 2023-11-28T18:21:53.259 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-11-28T18:21:53.302 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking postgresql (14+238) ... 2023-11-28T18:21:53.488 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-11-28T18:21:53.502 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-11-28T18:21:53.546 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package postgresql-client. 2023-11-28T18:21:53.547 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-28T18:21:53.561 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-11-28T18:21:53.604 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking postgresql-client (14+238) ... 2023-11-28T18:21:53.799 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libjson-perl. 2023-11-28T18:21:53.813 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-11-28T18:21:53.840 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package postgresql-contrib. 2023-11-28T18:21:53.855 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-11-28T18:21:53.857 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-11-28T18:21:53.890 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking postgresql-contrib (14+238) ... 2023-11-28T18:21:54.159 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package tcl8.6. 2023-11-28T18:21:54.175 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-11-28T18:21:54.185 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libllvm14:amd64. 2023-11-28T18:21:54.199 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-11-28T18:21:54.217 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-28T18:21:54.243 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-28T18:21:54.520 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package tcl. 2023-11-28T18:21:54.536 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-11-28T18:21:54.583 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-11-28T18:21:54.931 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package wpasupplicant. 2023-11-28T18:21:54.947 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-11-28T18:21:54.989 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-28T18:21:55.318 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-11-28T18:21:55.334 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-11-28T18:21:55.368 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-11-28T18:21:55.571 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-11-28T18:21:55.587 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-28T18:21:55.630 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-28T18:21:55.747 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package usb-modeswitch. 2023-11-28T18:21:55.762 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-11-28T18:21:55.813 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-28T18:21:55.924 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libmbim-proxy. 2023-11-28T18:21:55.940 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-28T18:21:56.008 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-28T18:21:56.091 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up postgresql-client-common (238) ... 2023-11-28T18:21:56.277 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up flex (2.6.4-8build2) ... 2023-11-28T18:21:56.327 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-11-28T18:21:56.343 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-28T18:21:56.378 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-28T18:21:56.385 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-28T18:21:56.496 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T18:21:56.622 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-11-28T18:21:56.671 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libndp0:amd64. 2023-11-28T18:21:56.687 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-11-28T18:21:56.722 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-28T18:21:56.773 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-28T18:21:56.899 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T18:21:57.066 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libnm0:amd64. 2023-11-28T18:21:57.081 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-11-28T18:21:57.124 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-28T18:21:57.469 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libpq5:amd64. 2023-11-28T18:21:57.485 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-28T18:21:57.527 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T18:21:57.863 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-11-28T18:21:57.879 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-28T18:21:57.908 INFO:teuthology.orchestra.run.smithi105.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-11-28T18:21:57.913 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-28T18:21:58.020 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-28T18:21:58.191 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libqmi-proxy. 2023-11-28T18:21:58.206 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-28T18:21:58.249 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-28T18:21:58.400 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-28T18:21:58.400 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-28T18:21:58.526 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-11-28T18:21:58.542 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-28T18:21:58.585 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-28T18:21:59.064 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-11-28T18:21:59.073 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-11-28T18:21:59.086 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-11-28T18:21:59.130 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-11-28T18:21:59.215 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-28T18:21:59.341 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-11-28T18:21:59.469 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-11-28T18:21:59.491 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package modemmanager. 2023-11-28T18:21:59.506 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-28T18:21:59.550 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-28T18:21:59.593 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-11-28T18:21:59.713 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-28T18:21:59.839 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-11-28T18:22:00.189 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package network-manager. 2023-11-28T18:22:00.205 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-11-28T18:22:00.255 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-11-28T18:22:00.627 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-28T18:22:00.728 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up dump (0.4b47-3) ... 2023-11-28T18:22:00.807 INFO:teuthology.orchestra.run.smithi105.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-11-28T18:22:00.897 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-28T18:22:00.930 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package ppp. 2023-11-28T18:22:00.946 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-11-28T18:22:01.023 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-28T18:22:01.148 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up tcl (8.6.11+1build2) ... 2023-11-28T18:22:01.275 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libjson-perl (4.04000-1) ... 2023-11-28T18:22:01.305 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-11-28T18:22:01.401 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-11-28T18:22:01.490 INFO:teuthology.orchestra.run.smithi105.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-11-28T18:22:01.560 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-28T18:22:01.676 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package pptp-linux. 2023-11-28T18:22:01.691 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-11-28T18:22:01.694 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-28T18:22:01.743 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-11-28T18:22:01.846 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-28T18:22:01.972 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-28T18:22:02.046 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package network-manager-pptp. 2023-11-28T18:22:02.060 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-11-28T18:22:02.098 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-11-28T18:22:02.112 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-11-28T18:22:02.266 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up postgresql-client (14+238) ... 2023-11-28T18:22:02.392 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-11-28T18:22:02.499 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package postgresql-client-common. 2023-11-28T18:22:02.513 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-11-28T18:22:02.566 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking postgresql-client-common (238) ... 2023-11-28T18:22:02.886 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package postgresql-client-14. 2023-11-28T18:22:02.900 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-28T18:22:02.944 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T18:22:02.994 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:02.995 INFO:teuthology.orchestra.run.smithi105.stdout:The following network interfaces were found in /etc/network/interfaces 2023-11-28T18:22:02.995 INFO:teuthology.orchestra.run.smithi105.stdout:which means they are currently configured by ifupdown: 2023-11-28T18:22:02.995 INFO:teuthology.orchestra.run.smithi105.stdout:- enp3s0f1 2023-11-28T18:22:02.995 INFO:teuthology.orchestra.run.smithi105.stdout:If you want to manage those interfaces with NetworkManager instead 2023-11-28T18:22:02.995 INFO:teuthology.orchestra.run.smithi105.stdout:remove their configuration from /etc/network/interfaces. 2023-11-28T18:22:02.995 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:03.127 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-11-28T18:22:03.545 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-11-28T18:22:03.591 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package postgresql-common. 2023-11-28T18:22:03.605 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-11-28T18:22:03.667 INFO:teuthology.orchestra.run.smithi148.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-11-28T18:22:03.698 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking postgresql-common (238) ... 2023-11-28T18:22:03.947 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-11-28T18:22:04.215 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package postgresql-14. 2023-11-28T18:22:04.229 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-28T18:22:04.276 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T18:22:04.636 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-28T18:22:04.796 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-28T18:22:04.922 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-11-28T18:22:05.048 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up postgresql-common (238) ... 2023-11-28T18:22:05.401 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package postgresql. 2023-11-28T18:22:05.416 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-11-28T18:22:05.459 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking postgresql (14+238) ... 2023-11-28T18:22:05.703 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package postgresql-client. 2023-11-28T18:22:05.718 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-11-28T18:22:05.753 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking postgresql-client (14+238) ... 2023-11-28T18:22:06.148 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package postgresql-contrib. 2023-11-28T18:22:06.163 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-11-28T18:22:06.207 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking postgresql-contrib (14+238) ... 2023-11-28T18:22:06.265 INFO:teuthology.orchestra.run.smithi105.stdout:Adding user postgres to group ssl-cert 2023-11-28T18:22:06.499 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:06.499 INFO:teuthology.orchestra.run.smithi105.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-11-28T18:22:06.526 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package tcl8.6. 2023-11-28T18:22:06.541 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-11-28T18:22:06.609 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-28T18:22:06.750 INFO:teuthology.orchestra.run.smithi105.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-11-28T18:22:06.751 INFO:teuthology.orchestra.run.smithi105.stdout:Removing obsolete dictionary files: 2023-11-28T18:22:06.878 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package tcl. 2023-11-28T18:22:06.893 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-11-28T18:22:06.933 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-11-28T18:22:07.281 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package wpasupplicant. 2023-11-28T18:22:07.296 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-11-28T18:22:07.340 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-28T18:22:07.478 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-11-28T18:22:07.686 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-11-28T18:22:07.700 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-11-28T18:22:07.735 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-11-28T18:22:08.017 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T18:22:08.080 INFO:teuthology.orchestra.run.smithi148.stdout:Selecting previously unselected package usb-modeswitch. 2023-11-28T18:22:08.095 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-11-28T18:22:08.146 INFO:teuthology.orchestra.run.smithi148.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-28T18:22:08.337 INFO:teuthology.orchestra.run.smithi105.stdout:Creating new PostgreSQL cluster 14/main ... 2023-11-28T18:22:08.352 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-11-28T18:22:08.411 INFO:teuthology.orchestra.run.smithi105.stdout:The files belonging to this database system will be owned by user "postgres". 2023-11-28T18:22:08.411 INFO:teuthology.orchestra.run.smithi105.stdout:This user must also own the server process. 2023-11-28T18:22:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:08.411 INFO:teuthology.orchestra.run.smithi105.stdout:The database cluster will be initialized with locale "en_US". 2023-11-28T18:22:08.411 INFO:teuthology.orchestra.run.smithi105.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-11-28T18:22:08.411 INFO:teuthology.orchestra.run.smithi105.stdout:The default text search configuration will be set to "english". 2023-11-28T18:22:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:08.411 INFO:teuthology.orchestra.run.smithi105.stdout:Data page checksums are disabled. 2023-11-28T18:22:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:08.411 INFO:teuthology.orchestra.run.smithi105.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-11-28T18:22:08.412 INFO:teuthology.orchestra.run.smithi105.stdout:creating subdirectories ... ok 2023-11-28T18:22:08.412 INFO:teuthology.orchestra.run.smithi105.stdout:selecting dynamic shared memory implementation ... posix 2023-11-28T18:22:08.436 INFO:teuthology.orchestra.run.smithi105.stdout:selecting default max_connections ... 100 2023-11-28T18:22:08.455 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up postgresql-client-common (238) ... 2023-11-28T18:22:08.471 INFO:teuthology.orchestra.run.smithi105.stdout:selecting default shared_buffers ... 128MB 2023-11-28T18:22:08.485 INFO:teuthology.orchestra.run.smithi105.stdout:selecting default time zone ... Etc/UTC 2023-11-28T18:22:08.485 INFO:teuthology.orchestra.run.smithi105.stdout:creating configuration files ... ok 2023-11-28T18:22:08.653 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up flex (2.6.4-8build2) ... 2023-11-28T18:22:08.654 INFO:teuthology.orchestra.run.smithi105.stdout:running bootstrap script ... ok 2023-11-28T18:22:08.788 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-28T18:22:08.913 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T18:22:09.064 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-11-28T18:22:09.191 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-28T18:22:09.271 INFO:teuthology.orchestra.run.smithi105.stdout:performing post-bootstrap initialization ... ok 2023-11-28T18:22:09.316 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T18:22:09.798 INFO:teuthology.orchestra.run.smithi105.stdout:syncing data to disk ... ok 2023-11-28T18:22:10.317 INFO:teuthology.orchestra.run.smithi148.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-11-28T18:22:10.325 INFO:teuthology.orchestra.run.smithi105.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-11-28T18:22:10.405 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-28T18:22:10.814 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-28T18:22:10.815 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-28T18:22:11.474 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-11-28T18:22:11.593 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-28T18:22:11.710 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-11-28T18:22:11.811 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-11-28T18:22:11.937 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-11-28T18:22:12.063 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-28T18:22:12.214 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-11-28T18:22:12.894 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-28T18:22:13.045 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up dump (0.4b47-3) ... 2023-11-28T18:22:13.140 INFO:teuthology.orchestra.run.smithi148.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-11-28T18:22:13.205 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-28T18:22:13.347 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-28T18:22:13.474 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up tcl (8.6.11+1build2) ... 2023-11-28T18:22:13.583 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libjson-perl (4.04000-1) ... 2023-11-28T18:22:13.683 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-28T18:22:13.684 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-11-28T18:22:13.757 INFO:teuthology.orchestra.run.smithi148.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-11-28T18:22:13.843 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up postgresql-contrib (14+238) ... 2023-11-28T18:22:13.844 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-28T18:22:13.970 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-28T18:22:13.977 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up postgresql (14+238) ... 2023-11-28T18:22:14.103 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-28T18:22:14.146 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-28T18:22:14.203 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-28T18:22:14.272 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-28T18:22:14.398 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-11-28T18:22:14.465 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-11-28T18:22:14.466 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-11-28T18:22:14.549 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up postgresql-client (14+238) ... 2023-11-28T18:22:14.700 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-11-28T18:22:15.214 INFO:teuthology.orchestra.run.smithi105.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-11-28T18:22:15.282 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:15.282 INFO:teuthology.orchestra.run.smithi148.stdout:The following network interfaces were found in /etc/network/interfaces 2023-11-28T18:22:15.282 INFO:teuthology.orchestra.run.smithi148.stdout:which means they are currently configured by ifupdown: 2023-11-28T18:22:15.283 INFO:teuthology.orchestra.run.smithi148.stdout:- enp3s0f1 2023-11-28T18:22:15.283 INFO:teuthology.orchestra.run.smithi148.stdout:If you want to manage those interfaces with NetworkManager instead 2023-11-28T18:22:15.283 INFO:teuthology.orchestra.run.smithi148.stdout:remove their configuration from /etc/network/interfaces. 2023-11-28T18:22:15.283 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:15.415 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-11-28T18:22:15.533 INFO:teuthology.orchestra.run.smithi105.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-28T18:22:15.676 INFO:teuthology.orchestra.run.smithi105.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-28T18:22:15.808 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-11-28T18:22:16.201 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-11-28T18:22:16.910 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-28T18:22:17.062 INFO:teuthology.orchestra.run.smithi105.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-11-28T18:22:17.063 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-28T18:22:17.162 INFO:teuthology.orchestra.run.smithi105.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-11-28T18:22:17.214 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-11-28T18:22:17.340 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up postgresql-common (238) ... 2023-11-28T18:22:17.736 INFO:teuthology.orchestra.run.smithi105.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-28T18:22:17.741 DEBUG:teuthology.parallel:result is None 2023-11-28T18:22:18.287 INFO:teuthology.orchestra.run.smithi148.stdout:Adding user postgres to group ssl-cert 2023-11-28T18:22:18.548 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:18.548 INFO:teuthology.orchestra.run.smithi148.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-11-28T18:22:18.798 INFO:teuthology.orchestra.run.smithi148.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-11-28T18:22:18.798 INFO:teuthology.orchestra.run.smithi148.stdout:Removing obsolete dictionary files: 2023-11-28T18:22:19.503 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-11-28T18:22:19.958 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T18:22:20.381 INFO:teuthology.orchestra.run.smithi148.stdout:Creating new PostgreSQL cluster 14/main ... 2023-11-28T18:22:20.402 INFO:teuthology.orchestra.run.smithi148.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-11-28T18:22:20.452 INFO:teuthology.orchestra.run.smithi148.stdout:The files belonging to this database system will be owned by user "postgres". 2023-11-28T18:22:20.452 INFO:teuthology.orchestra.run.smithi148.stdout:This user must also own the server process. 2023-11-28T18:22:20.452 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:20.452 INFO:teuthology.orchestra.run.smithi148.stdout:The database cluster will be initialized with locale "en_US". 2023-11-28T18:22:20.452 INFO:teuthology.orchestra.run.smithi148.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-11-28T18:22:20.452 INFO:teuthology.orchestra.run.smithi148.stdout:The default text search configuration will be set to "english". 2023-11-28T18:22:20.452 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:20.452 INFO:teuthology.orchestra.run.smithi148.stdout:Data page checksums are disabled. 2023-11-28T18:22:20.452 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:20.453 INFO:teuthology.orchestra.run.smithi148.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-11-28T18:22:20.453 INFO:teuthology.orchestra.run.smithi148.stdout:creating subdirectories ... ok 2023-11-28T18:22:20.453 INFO:teuthology.orchestra.run.smithi148.stdout:selecting dynamic shared memory implementation ... posix 2023-11-28T18:22:20.478 INFO:teuthology.orchestra.run.smithi148.stdout:selecting default max_connections ... 100 2023-11-28T18:22:20.514 INFO:teuthology.orchestra.run.smithi148.stdout:selecting default shared_buffers ... 128MB 2023-11-28T18:22:20.527 INFO:teuthology.orchestra.run.smithi148.stdout:selecting default time zone ... Etc/UTC 2023-11-28T18:22:20.528 INFO:teuthology.orchestra.run.smithi148.stdout:creating configuration files ... ok 2023-11-28T18:22:20.698 INFO:teuthology.orchestra.run.smithi148.stdout:running bootstrap script ... ok 2023-11-28T18:22:21.351 INFO:teuthology.orchestra.run.smithi148.stdout:performing post-bootstrap initialization ... ok 2023-11-28T18:22:21.847 INFO:teuthology.orchestra.run.smithi148.stdout:syncing data to disk ... ok 2023-11-28T18:22:22.348 INFO:teuthology.orchestra.run.smithi148.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-11-28T18:22:25.290 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-28T18:22:25.408 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up postgresql-contrib (14+238) ... 2023-11-28T18:22:25.518 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up postgresql (14+238) ... 2023-11-28T18:22:25.645 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-28T18:22:25.769 INFO:teuthology.orchestra.run.smithi148.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-28T18:22:26.056 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-11-28T18:22:26.057 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-11-28T18:22:26.846 INFO:teuthology.orchestra.run.smithi148.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-11-28T18:22:27.165 INFO:teuthology.orchestra.run.smithi148.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-28T18:22:27.309 INFO:teuthology.orchestra.run.smithi148.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-28T18:22:28.770 INFO:teuthology.orchestra.run.smithi148.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-11-28T18:22:28.861 INFO:teuthology.orchestra.run.smithi148.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-11-28T18:22:29.448 INFO:teuthology.orchestra.run.smithi148.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-28T18:22:29.453 DEBUG:teuthology.parallel:result is None 2023-11-28T18:22:29.453 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=47c1c547372ed5bff7984647dc200bac0e019caf 2023-11-28T18:22:29.595 DEBUG:teuthology.orchestra.run.smithi105:> dpkg-query -W -f '${Version}' ceph 2023-11-28T18:22:29.619 INFO:teuthology.orchestra.run.smithi105.stdout:18.0.0-7531-g47c1c547-1jammy 2023-11-28T18:22:29.620 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7531-g47c1c547-1jammy 2023-11-28T18:22:29.620 INFO:teuthology.task.install:The correct ceph version 18.0.0-7531-g47c1c547-1jammy is installed. 2023-11-28T18:22:29.620 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=47c1c547372ed5bff7984647dc200bac0e019caf 2023-11-28T18:22:29.842 DEBUG:teuthology.orchestra.run.smithi148:> dpkg-query -W -f '${Version}' ceph 2023-11-28T18:22:29.864 INFO:teuthology.orchestra.run.smithi148.stdout:18.0.0-7531-g47c1c547-1jammy 2023-11-28T18:22:29.864 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7531-g47c1c547-1jammy 2023-11-28T18:22:29.865 INFO:teuthology.task.install:The correct ceph version 18.0.0-7531-g47c1c547-1jammy is installed. 2023-11-28T18:22:29.865 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-28T18:22:29.865 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:22:29.865 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T18:22:29.888 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:22:29.888 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T18:22:29.932 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-28T18:22:29.933 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:22:29.933 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T18:22:29.955 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T18:22:30.020 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:22:30.020 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T18:22:30.041 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T18:22:30.108 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-28T18:22:30.108 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:22:30.108 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T18:22:30.129 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T18:22:30.192 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:22:30.192 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T18:22:30.212 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T18:22:30.276 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-28T18:22:30.276 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:22:30.276 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T18:22:30.298 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T18:22:30.360 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:22:30.360 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T18:22:30.380 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T18:22:30.440 INFO:teuthology.run_tasks:Running task ceph... 2023-11-28T18:22:30.508 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-28T18:22:30.508 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 777 /var/log/ceph 2023-11-28T18:22:30.512 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 777 /var/log/ceph 2023-11-28T18:22:30.531 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-28T18:22:30.532 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T18:22:30.574 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T18:22:30.594 INFO:tasks.ceph:Creating extra log directories... 2023-11-28T18:22:30.594 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T18:22:30.639 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T18:22:30.664 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-28T18:22:30.664 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-28T18:22:30.665 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-28T18:22:30.665 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:22:30.665 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T18:22:30.708 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T18:22:30.802 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:22:30.802 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T18:22:30.812 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T18:22:30.911 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-28T18:22:30.912 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-28T18:22:30.912 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2023-11-28_06:23:42-fs-wip-vshankar-testing-20231127.102654-testing-default-smithi/7469459', 'branch': 'wip-vshankar-testing-20231127.102654', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/exports}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7469459', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2023-11-28_06:23:42-fs-wip-vshankar-testing-20231127.102654-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-20231127.102654'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds'], 'sha1': '47c1c547372ed5bff7984647dc200bac0e019caf'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '47c1c547372ed5bff7984647dc200bac0e019caf'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20231127.102654', 'sha1': '47c1c547372ed5bff7984647dc200bac0e019caf'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1098, 'sha1': '47c1c547372ed5bff7984647dc200bac0e019caf', 'sleep_before_teardown': 0, 'subset': '854/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20231127.102654', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '47c1c547372ed5bff7984647dc200bac0e019caf', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi105.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDZ22c2VmsOKangy/tmCrNzMEGGlu/JeHoweWe97u2UH0HlxTHil6t5VqDGxukuHSHtFDA2TXg5f2HYYjyPSJjY=', 'smithi148.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGxinxnOSbx5MBENicFt1dCm1DSP36mcR2g5+fKgfl6EeU5vDQ38py62JnF5x9dPKj7SFH1W5pzPm1nMdnSnfbY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '47c1c547372ed5bff7984647dc200bac0e019caf'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds'], 'sha1': '47c1c547372ed5bff7984647dc200bac0e019caf', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_exports']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '5ae09a5cc7064ca2bf994df19f3006b79794789a', 'timestamp': '2023-11-28_06:23:42', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-28T18:22:30.913 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T18:22:30.917 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T18:22:30.926 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T18:22:30.966 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T18:22:30.991 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:22:30.992 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T18:22:31.036 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T18:22:31.036 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_1 2023-11-28T18:22:31.090 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T18:22:31.090 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T18:22:31.090 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 887 Links: 1 2023-11-28T18:22:31.090 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T18:22:31.090 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2023-11-28 18:15:54.049746156 +0000 2023-11-28T18:22:31.090 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2023-11-28 18:15:53.657749659 +0000 2023-11-28T18:22:31.090 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2023-11-28 18:15:53.657749659 +0000 2023-11-28T18:22:31.091 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2023-11-28 18:15:53.657749659 +0000 2023-11-28T18:22:31.091 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T18:22:31.151 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2023-11-28T18:22:31.151 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2023-11-28T18:22:31.152 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000395993 s, 1.3 MB/s 2023-11-28T18:22:31.153 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T18:22:31.206 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_2 2023-11-28T18:22:31.257 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T18:22:31.257 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T18:22:31.257 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 903 Links: 1 2023-11-28T18:22:31.257 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T18:22:31.257 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2023-11-28 18:15:54.745739941 +0000 2023-11-28T18:22:31.257 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2023-11-28 18:15:54.337743585 +0000 2023-11-28T18:22:31.257 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2023-11-28 18:15:54.337743585 +0000 2023-11-28T18:22:31.257 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2023-11-28 18:15:54.337743585 +0000 2023-11-28T18:22:31.258 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T18:22:31.323 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2023-11-28T18:22:31.323 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2023-11-28T18:22:31.323 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000395773 s, 1.3 MB/s 2023-11-28T18:22:31.325 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T18:22:31.382 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_3 2023-11-28T18:22:31.432 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T18:22:31.432 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T18:22:31.432 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 921 Links: 1 2023-11-28T18:22:31.432 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T18:22:31.433 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2023-11-28 18:15:55.385734224 +0000 2023-11-28T18:22:31.433 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2023-11-28 18:15:54.997737689 +0000 2023-11-28T18:22:31.433 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2023-11-28 18:15:54.997737689 +0000 2023-11-28T18:22:31.433 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2023-11-28 18:15:54.997737689 +0000 2023-11-28T18:22:31.433 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T18:22:31.498 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2023-11-28T18:22:31.498 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2023-11-28T18:22:31.498 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000635338 s, 806 kB/s 2023-11-28T18:22:31.500 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T18:22:31.554 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_4 2023-11-28T18:22:31.605 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T18:22:31.605 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T18:22:31.605 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 940 Links: 1 2023-11-28T18:22:31.605 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T18:22:31.605 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2023-11-28 18:15:56.073728078 +0000 2023-11-28T18:22:31.606 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2023-11-28 18:15:55.649731865 +0000 2023-11-28T18:22:31.606 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2023-11-28 18:15:55.649731865 +0000 2023-11-28T18:22:31.606 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2023-11-28 18:15:55.649731865 +0000 2023-11-28T18:22:31.606 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T18:22:31.671 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2023-11-28T18:22:31.671 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2023-11-28T18:22:31.671 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000380833 s, 1.3 MB/s 2023-11-28T18:22:31.673 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T18:22:31.730 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T18:22:31.730 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:22:31.731 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T18:22:31.740 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T18:22:31.740 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_1 2023-11-28T18:22:31.797 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T18:22:31.797 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T18:22:31.798 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 903 Links: 1 2023-11-28T18:22:31.798 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T18:22:31.798 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2023-11-28 18:15:52.824668092 +0000 2023-11-28T18:22:31.798 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2023-11-28 18:15:52.396671915 +0000 2023-11-28T18:22:31.798 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2023-11-28 18:15:52.396671915 +0000 2023-11-28T18:22:31.798 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2023-11-28 18:15:52.396671915 +0000 2023-11-28T18:22:31.798 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T18:22:31.863 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2023-11-28T18:22:31.864 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2023-11-28T18:22:31.864 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000379759 s, 1.3 MB/s 2023-11-28T18:22:31.865 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T18:22:31.923 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_2 2023-11-28T18:22:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T18:22:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T18:22:31.973 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 918 Links: 1 2023-11-28T18:22:31.973 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T18:22:31.973 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2023-11-28 18:15:53.484662197 +0000 2023-11-28T18:22:31.974 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2023-11-28 18:15:53.096665663 +0000 2023-11-28T18:22:31.974 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2023-11-28 18:15:53.096665663 +0000 2023-11-28T18:22:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2023-11-28 18:15:53.096665663 +0000 2023-11-28T18:22:31.974 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T18:22:32.035 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2023-11-28T18:22:32.035 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2023-11-28T18:22:32.035 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000370066 s, 1.4 MB/s 2023-11-28T18:22:32.037 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T18:22:32.090 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_3 2023-11-28T18:22:32.141 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T18:22:32.141 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T18:22:32.141 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 935 Links: 1 2023-11-28T18:22:32.141 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T18:22:32.142 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2023-11-28 18:15:54.168656088 +0000 2023-11-28T18:22:32.142 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2023-11-28 18:15:53.764659695 +0000 2023-11-28T18:22:32.142 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2023-11-28 18:15:53.764659695 +0000 2023-11-28T18:22:32.142 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2023-11-28 18:15:53.764659695 +0000 2023-11-28T18:22:32.143 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T18:22:32.203 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2023-11-28T18:22:32.203 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2023-11-28T18:22:32.203 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000370868 s, 1.4 MB/s 2023-11-28T18:22:32.205 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T18:22:32.259 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_4 2023-11-28T18:22:32.309 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T18:22:32.309 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T18:22:32.309 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 952 Links: 1 2023-11-28T18:22:32.309 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T18:22:32.309 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2023-11-28 18:15:54.852649978 +0000 2023-11-28T18:22:32.309 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2023-11-28 18:15:54.448653586 +0000 2023-11-28T18:22:32.309 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2023-11-28 18:15:54.448653586 +0000 2023-11-28T18:22:32.309 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2023-11-28 18:15:54.448653586 +0000 2023-11-28T18:22:32.310 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T18:22:32.371 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2023-11-28T18:22:32.371 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2023-11-28T18:22:32.371 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000651276 s, 786 kB/s 2023-11-28T18:22:32.373 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T18:22:32.427 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T18:22:32.427 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi148.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T18:22:32.427 INFO:tasks.ceph:Generating config... 2023-11-28T18:22:32.428 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-28T18:22:32.429 INFO:tasks.ceph:[client] debug client = 20 2023-11-28T18:22:32.429 INFO:tasks.ceph:[client] debug ms = 1 2023-11-28T18:22:32.429 INFO:tasks.ceph:[client] pid file = 2023-11-28T18:22:32.429 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-28T18:22:32.429 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-28T18:22:32.430 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-28T18:22:32.430 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-28T18:22:32.431 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-28T18:22:32.431 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-28T18:22:32.431 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-28T18:22:32.432 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-28T18:22:32.432 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-28T18:22:32.432 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-28T18:22:32.432 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-28T18:22:32.432 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-28T18:22:32.432 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-28T18:22:32.433 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-28T18:22:32.433 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-28T18:22:32.433 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-28T18:22:32.433 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-28T18:22:32.433 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-28T18:22:32.433 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-28T18:22:32.434 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-28T18:22:32.434 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-11-28T18:22:32.434 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-11-28T18:22:32.434 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-28T18:22:32.434 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-28T18:22:32.434 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-11-28T18:22:32.434 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-28T18:22:32.435 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-28T18:22:32.435 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-28T18:22:32.435 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-28T18:22:32.435 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-28T18:22:32.435 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-28T18:22:32.436 INFO:tasks.ceph:Setting up mon.a... 2023-11-28T18:22:32.437 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-28T18:22:32.483 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.keyring 2023-11-28T18:22:32.500 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-28T18:22:32.592 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T18:22:32.613 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.105'), ('mon.b', '172.21.15.148'), ('mon.c', '[v2:172.21.15.148:3301,v1:172.21.15.148:6790]')] 2023-11-28T18:22:32.613 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.105,172.21.15.148,[v2:172.21.15.148:3301,v1:172.21.15.148:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-28T18:22:32.614 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:22:32.614 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T18:22:32.668 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.105 --add b 172.21.15.148 --addv c '[v2:172.21.15.148:3301,v1:172.21.15.148:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-28T18:22:32.758 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-28T18:22:32.758 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: generated fsid b302bec5-10e5-4c91-9026-ecb69250e81c 2023-11-28T18:22:32.758 INFO:teuthology.orchestra.run.smithi105.stdout:setting min_mon_release = pacific 2023-11-28T18:22:32.759 INFO:teuthology.orchestra.run.smithi105.stdout:epoch 0 2023-11-28T18:22:32.759 INFO:teuthology.orchestra.run.smithi105.stdout:fsid b302bec5-10e5-4c91-9026-ecb69250e81c 2023-11-28T18:22:32.759 INFO:teuthology.orchestra.run.smithi105.stdout:last_changed 2023-11-28T18:22:32.759820+0000 2023-11-28T18:22:32.759 INFO:teuthology.orchestra.run.smithi105.stdout:created 2023-11-28T18:22:32.759820+0000 2023-11-28T18:22:32.759 INFO:teuthology.orchestra.run.smithi105.stdout:min_mon_release 16 (pacific) 2023-11-28T18:22:32.759 INFO:teuthology.orchestra.run.smithi105.stdout:election_strategy: 1 2023-11-28T18:22:32.759 INFO:teuthology.orchestra.run.smithi105.stdout:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2023-11-28T18:22:32.760 INFO:teuthology.orchestra.run.smithi105.stdout:1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2023-11-28T18:22:32.760 INFO:teuthology.orchestra.run.smithi105.stdout:2: [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] mon.c 2023-11-28T18:22:32.760 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-28T18:22:32.761 DEBUG:teuthology.orchestra.run.smithi105:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T18:22:32.770 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b302bec5-10e5-4c91-9026-ecb69250e81c... 2023-11-28T18:22:32.771 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T18:22:32.814 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T18:22:32.855 INFO:teuthology.orchestra.run.smithi105.stdout:[global] 2023-11-28T18:22:32.855 INFO:teuthology.orchestra.run.smithi105.stdout: chdir = "" 2023-11-28T18:22:32.855 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T18:22:32.856 INFO:teuthology.orchestra.run.smithi105.stdout: auth supported = cephx 2023-11-28T18:22:32.856 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.856 INFO:teuthology.orchestra.run.smithi105.stdout: filestore xattr use omap = true 2023-11-28T18:22:32.856 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.856 INFO:teuthology.orchestra.run.smithi105.stdout: mon clock drift allowed = 1.000 2023-11-28T18:22:32.857 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.857 INFO:teuthology.orchestra.run.smithi105.stdout: osd crush chooseleaf type = 0 2023-11-28T18:22:32.857 INFO:teuthology.orchestra.run.smithi105.stdout: auth debug = true 2023-11-28T18:22:32.857 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.857 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on old message = true 2023-11-28T18:22:32.858 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on bug = true 2023-11-28T18:22:32.858 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.858 INFO:teuthology.orchestra.run.smithi105.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T18:22:32.858 INFO:teuthology.orchestra.run.smithi105.stdout: mon pg warn max object skew = 0 2023-11-28T18:22:32.858 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.859 INFO:teuthology.orchestra.run.smithi105.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T18:22:32.859 INFO:teuthology.orchestra.run.smithi105.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T18:22:32.859 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.859 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default size = 2 2023-11-28T18:22:32.859 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.859 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow primary affinity = true 2023-11-28T18:22:32.860 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow pg remap = true 2023-11-28T18:22:32.860 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on legacy crush tunables = false 2023-11-28T18:22:32.860 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on crush straw calc version zero = false 2023-11-28T18:22:32.860 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on no sortbitwise = false 2023-11-28T18:22:32.860 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on osd down out interval zero = false 2023-11-28T18:22:32.861 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on too few osds = false 2023-11-28T18:22:32.861 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T18:22:32.861 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T18:22:32.862 INFO:teuthology.orchestra.run.smithi105.stdout: mon_allow_pool_size_one = true 2023-11-28T18:22:32.862 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.862 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T18:22:32.862 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.862 INFO:teuthology.orchestra.run.smithi105.stdout: osd default data pool replay window = 5 2023-11-28T18:22:32.863 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.863 INFO:teuthology.orchestra.run.smithi105.stdout: mon allow pool delete = true 2023-11-28T18:22:32.863 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.863 INFO:teuthology.orchestra.run.smithi105.stdout: mon cluster log file level = debug 2023-11-28T18:22:32.863 INFO:teuthology.orchestra.run.smithi105.stdout: debug asserts on shutdown = true 2023-11-28T18:22:32.864 INFO:teuthology.orchestra.run.smithi105.stdout: mon health detail to clog = false 2023-11-28T18:22:32.864 INFO:teuthology.orchestra.run.smithi105.stdout: mon host = "172.21.15.105,172.21.15.148,[v2:172.21.15.148:3301,v1:172.21.15.148:6790]" 2023-11-28T18:22:32.864 INFO:teuthology.orchestra.run.smithi105.stdout: fsid = b302bec5-10e5-4c91-9026-ecb69250e81c 2023-11-28T18:22:32.864 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.864 INFO:teuthology.orchestra.run.smithi105.stdout:[osd] 2023-11-28T18:22:32.864 INFO:teuthology.orchestra.run.smithi105.stdout: osd journal size = 100 2023-11-28T18:22:32.864 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.864 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub load threshold = 5.0 2023-11-28T18:22:32.864 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub max interval = 600 2023-11-28T18:22:32.864 INFO:teuthology.orchestra.run.smithi105.stdout: osd mclock profile = high_recovery_ops 2023-11-28T18:22:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: osd recover clone overlap = true 2023-11-28T18:22:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: osd recovery max chunk = 1048576 2023-11-28T18:22:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug shutdown = true 2023-11-28T18:22:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug op order = true 2023-11-28T18:22:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug verify stray on activate = true 2023-11-28T18:22:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug trim objects = true 2023-11-28T18:22:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: osd open classes on start = true 2023-11-28T18:22:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug pg log writeout = true 2023-11-28T18:22:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: osd deep scrub update digest min age = 30 2023-11-28T18:22:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: osd map max advance = 10 2023-11-28T18:22:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: journal zero on create = true 2023-11-28T18:22:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: filestore ondisk finisher threads = 3 2023-11-28T18:22:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: filestore apply finisher threads = 3 2023-11-28T18:22:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: bdev debug aio = true 2023-11-28T18:22:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug misdirected ops = true 2023-11-28T18:22:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore block size = 96636764160 2023-11-28T18:22:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore fsck on mount = True 2023-11-28T18:22:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluefs = 20 2023-11-28T18:22:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluestore = 20 2023-11-28T18:22:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-11-28T18:22:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: debug osd = 20 2023-11-28T18:22:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: debug rocksdb = 10 2023-11-28T18:22:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T18:22:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd full ratio = 0.9 2023-11-28T18:22:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd nearfull ratio = 0.8 2023-11-28T18:22:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: osd failsafe full ratio = 0.95 2023-11-28T18:22:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: osd objectstore = bluestore 2023-11-28T18:22:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2023-11-28T18:22:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.868 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr] 2023-11-28T18:22:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-11-28T18:22:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: debug mgr = 20 2023-11-28T18:22:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2023-11-28T18:22:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2023-11-28T18:22:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2023-11-28T18:22:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2023-11-28T18:22:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: mgr/telemetry/nag = false 2023-11-28T18:22:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.869 INFO:teuthology.orchestra.run.smithi105.stdout:[mon] 2023-11-28T18:22:32.869 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-11-28T18:22:32.869 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2023-11-28T18:22:32.869 INFO:teuthology.orchestra.run.smithi105.stdout: debug paxos = 20 2023-11-28T18:22:32.869 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2023-11-28T18:22:32.869 INFO:teuthology.orchestra.run.smithi105.stdout: mon data avail warn = 5 2023-11-28T18:22:32.869 INFO:teuthology.orchestra.run.smithi105.stdout: mon mgr mkfs grace = 240 2023-11-28T18:22:32.869 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2023-11-28T18:22:32.869 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd reporter subtree level = osd 2023-11-28T18:22:32.869 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd prime pg temp = true 2023-11-28T18:22:32.869 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2023-11-28T18:22:32.869 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.870 INFO:teuthology.orchestra.run.smithi105.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T18:22:32.870 INFO:teuthology.orchestra.run.smithi105.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T18:22:32.870 INFO:teuthology.orchestra.run.smithi105.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T18:22:32.870 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.870 INFO:teuthology.orchestra.run.smithi105.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T18:22:32.870 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T18:22:32.870 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T18:22:32.870 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.870 INFO:teuthology.orchestra.run.smithi105.stdout: # 1m isn't quite enough 2023-11-28T18:22:32.870 INFO:teuthology.orchestra.run.smithi105.stdout: mon_down_mkfs_grace = 2m 2023-11-28T18:22:32.871 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.871 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_filestore_osds = false 2023-11-28T18:22:32.871 INFO:teuthology.orchestra.run.smithi105.stdout: mon op complaint time = 120 2023-11-28T18:22:32.871 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:22:32.871 INFO:teuthology.orchestra.run.smithi105.stdout:[client] 2023-11-28T18:22:32.871 INFO:teuthology.orchestra.run.smithi105.stdout: rgw cache enabled = true 2023-11-28T18:22:32.871 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable ops log = true 2023-11-28T18:22:32.871 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable usage log = true 2023-11-28T18:22:32.871 INFO:teuthology.orchestra.run.smithi105.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T18:22:32.871 INFO:teuthology.orchestra.run.smithi105.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T18:22:32.871 INFO:teuthology.orchestra.run.smithi105.stdout: client mount timeout = 600 2023-11-28T18:22:32.871 INFO:teuthology.orchestra.run.smithi105.stdout: debug client = 20 2023-11-28T18:22:32.872 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-11-28T18:22:32.872 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = "" 2023-11-28T18:22:32.872 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2023-11-28T18:22:32.872 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2023-11-28T18:22:32.872 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.a] 2023-11-28T18:22:32.872 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.b] 2023-11-28T18:22:32.872 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.c] 2023-11-28T18:22:32.872 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2023-11-28T18:22:32.872 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2023-11-28T18:22:32.872 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2023-11-28T18:22:32.872 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2023-11-28T18:22:32.872 INFO:teuthology.orchestra.run.smithi105.stdout:[mds] 2023-11-28T18:22:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds = 20 2023-11-28T18:22:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds balancer = 20 2023-11-28T18:22:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-11-28T18:22:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug frag = True 2023-11-28T18:22:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug scatterstat = True 2023-11-28T18:22:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: mds op complaint time = 180 2023-11-28T18:22:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: mds verify scatter = True 2023-11-28T18:22:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2023-11-28T18:22:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2023-11-28T18:22:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2023-11-28T18:22:32.874 INFO:teuthology.orchestra.run.smithi148.stdout:[global] 2023-11-28T18:22:32.874 INFO:teuthology.orchestra.run.smithi148.stdout: chdir = "" 2023-11-28T18:22:32.874 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T18:22:32.874 INFO:teuthology.orchestra.run.smithi148.stdout: auth supported = cephx 2023-11-28T18:22:32.874 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.874 INFO:teuthology.orchestra.run.smithi148.stdout: filestore xattr use omap = true 2023-11-28T18:22:32.874 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.875 INFO:teuthology.orchestra.run.smithi148.stdout: mon clock drift allowed = 1.000 2023-11-28T18:22:32.875 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.875 INFO:teuthology.orchestra.run.smithi148.stdout: osd crush chooseleaf type = 0 2023-11-28T18:22:32.875 INFO:teuthology.orchestra.run.smithi148.stdout: auth debug = true 2023-11-28T18:22:32.875 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.875 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on old message = true 2023-11-28T18:22:32.875 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on bug = true 2023-11-28T18:22:32.875 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.875 INFO:teuthology.orchestra.run.smithi148.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T18:22:32.875 INFO:teuthology.orchestra.run.smithi148.stdout: mon pg warn max object skew = 0 2023-11-28T18:22:32.875 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.876 INFO:teuthology.orchestra.run.smithi148.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T18:22:32.876 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T18:22:32.876 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.876 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default size = 2 2023-11-28T18:22:32.876 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.876 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow primary affinity = true 2023-11-28T18:22:32.876 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow pg remap = true 2023-11-28T18:22:32.876 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on legacy crush tunables = false 2023-11-28T18:22:32.876 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on crush straw calc version zero = false 2023-11-28T18:22:32.877 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on no sortbitwise = false 2023-11-28T18:22:32.877 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on osd down out interval zero = false 2023-11-28T18:22:32.877 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on too few osds = false 2023-11-28T18:22:32.877 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T18:22:32.877 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T18:22:32.877 INFO:teuthology.orchestra.run.smithi148.stdout: mon_allow_pool_size_one = true 2023-11-28T18:22:32.877 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.878 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T18:22:32.878 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.878 INFO:teuthology.orchestra.run.smithi148.stdout: osd default data pool replay window = 5 2023-11-28T18:22:32.878 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.878 INFO:teuthology.orchestra.run.smithi148.stdout: mon allow pool delete = true 2023-11-28T18:22:32.878 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.878 INFO:teuthology.orchestra.run.smithi148.stdout: mon cluster log file level = debug 2023-11-28T18:22:32.878 INFO:teuthology.orchestra.run.smithi148.stdout: debug asserts on shutdown = true 2023-11-28T18:22:32.878 INFO:teuthology.orchestra.run.smithi148.stdout: mon health detail to clog = false 2023-11-28T18:22:32.878 INFO:teuthology.orchestra.run.smithi148.stdout: mon host = "172.21.15.105,172.21.15.148,[v2:172.21.15.148:3301,v1:172.21.15.148:6790]" 2023-11-28T18:22:32.878 INFO:teuthology.orchestra.run.smithi148.stdout: fsid = b302bec5-10e5-4c91-9026-ecb69250e81c 2023-11-28T18:22:32.879 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.879 INFO:teuthology.orchestra.run.smithi148.stdout:[osd] 2023-11-28T18:22:32.879 INFO:teuthology.orchestra.run.smithi148.stdout: osd journal size = 100 2023-11-28T18:22:32.879 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.879 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub load threshold = 5.0 2023-11-28T18:22:32.879 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub max interval = 600 2023-11-28T18:22:32.879 INFO:teuthology.orchestra.run.smithi148.stdout: osd mclock profile = high_recovery_ops 2023-11-28T18:22:32.879 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.879 INFO:teuthology.orchestra.run.smithi148.stdout: osd recover clone overlap = true 2023-11-28T18:22:32.880 INFO:teuthology.orchestra.run.smithi148.stdout: osd recovery max chunk = 1048576 2023-11-28T18:22:32.880 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.880 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug shutdown = true 2023-11-28T18:22:32.880 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug op order = true 2023-11-28T18:22:32.880 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug verify stray on activate = true 2023-11-28T18:22:32.880 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug trim objects = true 2023-11-28T18:22:32.880 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.880 INFO:teuthology.orchestra.run.smithi148.stdout: osd open classes on start = true 2023-11-28T18:22:32.880 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug pg log writeout = true 2023-11-28T18:22:32.881 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.881 INFO:teuthology.orchestra.run.smithi148.stdout: osd deep scrub update digest min age = 30 2023-11-28T18:22:32.881 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.881 INFO:teuthology.orchestra.run.smithi148.stdout: osd map max advance = 10 2023-11-28T18:22:32.881 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.881 INFO:teuthology.orchestra.run.smithi148.stdout: journal zero on create = true 2023-11-28T18:22:32.881 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.881 INFO:teuthology.orchestra.run.smithi148.stdout: filestore ondisk finisher threads = 3 2023-11-28T18:22:32.881 INFO:teuthology.orchestra.run.smithi148.stdout: filestore apply finisher threads = 3 2023-11-28T18:22:32.881 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.881 INFO:teuthology.orchestra.run.smithi148.stdout: bdev debug aio = true 2023-11-28T18:22:32.882 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug misdirected ops = true 2023-11-28T18:22:32.882 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore block size = 96636764160 2023-11-28T18:22:32.882 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore fsck on mount = True 2023-11-28T18:22:32.882 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluefs = 20 2023-11-28T18:22:32.882 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluestore = 20 2023-11-28T18:22:32.882 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2023-11-28T18:22:32.882 INFO:teuthology.orchestra.run.smithi148.stdout: debug osd = 20 2023-11-28T18:22:32.882 INFO:teuthology.orchestra.run.smithi148.stdout: debug rocksdb = 10 2023-11-28T18:22:32.882 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T18:22:32.882 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd full ratio = 0.9 2023-11-28T18:22:32.883 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd nearfull ratio = 0.8 2023-11-28T18:22:32.883 INFO:teuthology.orchestra.run.smithi148.stdout: osd failsafe full ratio = 0.95 2023-11-28T18:22:32.883 INFO:teuthology.orchestra.run.smithi148.stdout: osd objectstore = bluestore 2023-11-28T18:22:32.883 INFO:teuthology.orchestra.run.smithi148.stdout: osd op complaint time = 180 2023-11-28T18:22:32.883 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.883 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr] 2023-11-28T18:22:32.883 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2023-11-28T18:22:32.883 INFO:teuthology.orchestra.run.smithi148.stdout: debug mgr = 20 2023-11-28T18:22:32.883 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2023-11-28T18:22:32.883 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2023-11-28T18:22:32.884 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2023-11-28T18:22:32.884 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2023-11-28T18:22:32.884 INFO:teuthology.orchestra.run.smithi148.stdout: mgr/telemetry/nag = false 2023-11-28T18:22:32.884 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.884 INFO:teuthology.orchestra.run.smithi148.stdout:[mon] 2023-11-28T18:22:32.884 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2023-11-28T18:22:32.884 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2023-11-28T18:22:32.884 INFO:teuthology.orchestra.run.smithi148.stdout: debug paxos = 20 2023-11-28T18:22:32.884 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2023-11-28T18:22:32.884 INFO:teuthology.orchestra.run.smithi148.stdout: mon data avail warn = 5 2023-11-28T18:22:32.884 INFO:teuthology.orchestra.run.smithi148.stdout: mon mgr mkfs grace = 240 2023-11-28T18:22:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2023-11-28T18:22:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd reporter subtree level = osd 2023-11-28T18:22:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd prime pg temp = true 2023-11-28T18:22:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2023-11-28T18:22:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T18:22:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T18:22:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T18:22:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T18:22:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T18:22:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T18:22:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: # 1m isn't quite enough 2023-11-28T18:22:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: mon_down_mkfs_grace = 2m 2023-11-28T18:22:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_filestore_osds = false 2023-11-28T18:22:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: mon op complaint time = 120 2023-11-28T18:22:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-28T18:22:32.886 INFO:teuthology.orchestra.run.smithi148.stdout:[client] 2023-11-28T18:22:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: rgw cache enabled = true 2023-11-28T18:22:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable ops log = true 2023-11-28T18:22:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable usage log = true 2023-11-28T18:22:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T18:22:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T18:22:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: client mount timeout = 600 2023-11-28T18:22:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: debug client = 20 2023-11-28T18:22:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2023-11-28T18:22:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = "" 2023-11-28T18:22:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: rados mon op timeout = 900 2023-11-28T18:22:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: rados osd op timeout = 900 2023-11-28T18:22:32.887 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.a] 2023-11-28T18:22:32.887 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.b] 2023-11-28T18:22:32.888 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.c] 2023-11-28T18:22:32.888 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.a] 2023-11-28T18:22:32.888 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.c] 2023-11-28T18:22:32.888 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.b] 2023-11-28T18:22:32.888 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.d] 2023-11-28T18:22:32.888 INFO:teuthology.orchestra.run.smithi148.stdout:[mds] 2023-11-28T18:22:32.888 INFO:teuthology.orchestra.run.smithi148.stdout: debug mds = 20 2023-11-28T18:22:32.888 INFO:teuthology.orchestra.run.smithi148.stdout: debug mds balancer = 20 2023-11-28T18:22:32.888 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2023-11-28T18:22:32.889 INFO:teuthology.orchestra.run.smithi148.stdout: mds debug frag = True 2023-11-28T18:22:32.889 INFO:teuthology.orchestra.run.smithi148.stdout: mds debug scatterstat = True 2023-11-28T18:22:32.889 INFO:teuthology.orchestra.run.smithi148.stdout: mds op complaint time = 180 2023-11-28T18:22:32.889 INFO:teuthology.orchestra.run.smithi148.stdout: mds verify scatter = True 2023-11-28T18:22:32.889 INFO:teuthology.orchestra.run.smithi148.stdout: osd op complaint time = 180 2023-11-28T18:22:32.889 INFO:teuthology.orchestra.run.smithi148.stdout: rados mon op timeout = 900 2023-11-28T18:22:32.889 INFO:teuthology.orchestra.run.smithi148.stdout: rados osd op timeout = 900 2023-11-28T18:22:32.891 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-28T18:22:32.891 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-28T18:22:32.991 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-28T18:22:32.992 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:22:32.992 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-28T18:22:33.003 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:22:33.003 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-28T18:22:33.051 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:22:33.051 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:22:33.052 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T18:22:33.052 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T18:22:33.130 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:22:33.130 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T18:22:33.179 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:22:33.179 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:22:33.179 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T18:22:33.180 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T18:22:33.215 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:22:33.215 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T18:22:33.265 INFO:tasks.ceph:Setting up mon nodes... 2023-11-28T18:22:33.265 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-28T18:22:33.265 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T18:22:33.322 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T18:22:33.327 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T18:22:33.387 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T18:22:33.404 INFO:tasks.ceph:Setting up mds nodes... 2023-11-28T18:22:33.404 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T18:22:33.460 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T18:22:33.465 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-28T18:22:33.485 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T18:22:33.581 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T18:22:33.585 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-28T18:22:33.606 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T18:22:33.666 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T18:22:33.670 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-28T18:22:33.691 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T18:22:33.788 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T18:22:33.792 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-28T18:22:33.812 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-28T18:22:33.813 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-28T18:22:33.855 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-28T18:22:33.872 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-28T18:22:33.962 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-28T18:22:33.979 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-28T18:22:34.022 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-28T18:22:34.039 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-28T18:22:34.122 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-28T18:22:34.139 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-28T18:22:34.139 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi148.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T18:22:34.139 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-28T18:22:34.159 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T18:22:34.159 INFO:tasks.ceph:role: osd.0 2023-11-28T18:22:34.159 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:22:34.160 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T18:22:34.234 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T18:22:34.234 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T18:22:34.235 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T18:22:34.235 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T18:22:34.235 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T18:22:34.235 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2023-11-28T18:22:34.236 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T18:22:34.236 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T18:22:34.236 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T18:22:34.236 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T18:22:34.704 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2023-11-28T18:22:34.707 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2023-11-28T18:22:34.707 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-28T18:22:34.850 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-28T18:22:34.866 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: /sbin/restorecon: command not found 2023-11-28T18:22:34.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:22:34.867 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-28T18:22:34.935 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T18:22:34.935 INFO:tasks.ceph:role: osd.1 2023-11-28T18:22:34.935 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:22:34.935 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T18:22:35.006 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T18:22:35.006 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T18:22:35.006 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T18:22:35.007 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T18:22:35.007 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T18:22:35.007 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2023-11-28T18:22:35.007 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T18:22:35.007 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T18:22:35.007 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T18:22:35.007 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T18:22:35.457 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2023-11-28T18:22:35.459 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2023-11-28T18:22:35.459 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-28T18:22:35.564 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-28T18:22:35.580 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: /sbin/restorecon: command not found 2023-11-28T18:22:35.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:22:35.581 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-28T18:22:35.644 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T18:22:35.644 INFO:tasks.ceph:role: osd.2 2023-11-28T18:22:35.644 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:22:35.644 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T18:22:35.715 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T18:22:35.715 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T18:22:35.715 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T18:22:35.716 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T18:22:35.716 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T18:22:35.716 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2023-11-28T18:22:35.716 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T18:22:35.716 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T18:22:35.717 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T18:22:35.717 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T18:22:36.176 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2023-11-28T18:22:36.180 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2023-11-28T18:22:36.180 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-28T18:22:36.334 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-28T18:22:36.351 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: /sbin/restorecon: command not found 2023-11-28T18:22:36.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:22:36.353 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-28T18:22:36.420 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T18:22:36.420 INFO:tasks.ceph:role: osd.3 2023-11-28T18:22:36.420 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:22:36.421 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T18:22:36.492 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T18:22:36.492 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T18:22:36.492 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T18:22:36.492 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T18:22:36.493 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T18:22:36.493 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2023-11-28T18:22:36.493 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T18:22:36.493 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T18:22:36.493 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T18:22:36.493 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T18:22:36.987 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2023-11-28T18:22:36.990 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2023-11-28T18:22:36.991 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-28T18:22:37.127 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-28T18:22:37.145 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: /sbin/restorecon: command not found 2023-11-28T18:22:37.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:22:37.146 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T18:22:37.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:37.266+0000 7feacb1c6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-28T18:22:37.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:37.270+0000 7feacb1c6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-28T18:22:37.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:37.270+0000 7feacb1c6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T18:22:37.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:37.270+0000 7feacb1c6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T18:22:37.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:37.270+0000 7feacb1c6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T18:22:37.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:37.274+0000 7feacb1c6640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-28T18:22:40.272 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-28T18:22:40.293 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T18:22:40.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:40.390+0000 7f6692cda640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-28T18:22:40.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:40.390+0000 7f6692cda640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-28T18:22:40.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:40.390+0000 7f6692cda640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T18:22:40.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:40.390+0000 7f6692cda640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T18:22:40.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:40.390+0000 7f6692cda640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T18:22:40.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:40.394+0000 7f6692cda640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-28T18:22:43.395 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-28T18:22:43.417 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T18:22:43.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:43.514+0000 7f7f01848640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-28T18:22:43.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:43.518+0000 7f7f01848640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-28T18:22:43.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:43.518+0000 7f7f01848640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T18:22:43.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:43.518+0000 7f7f01848640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T18:22:43.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:43.518+0000 7f7f01848640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T18:22:43.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:43.518+0000 7f7f01848640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-28T18:22:46.509 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-28T18:22:46.531 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T18:22:46.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:46.626+0000 7f540c22b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-28T18:22:46.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:46.626+0000 7f540c22b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-28T18:22:46.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:46.626+0000 7f540c22b640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T18:22:46.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:46.626+0000 7f540c22b640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T18:22:46.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:46.626+0000 7f540c22b640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T18:22:46.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:22:46.630+0000 7f540c22b640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-28T18:22:49.629 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-28T18:22:49.651 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-28T18:22:49.675 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T18:22:49.675 INFO:tasks.ceph:role: osd.4 2023-11-28T18:22:49.675 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:22:49.675 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T18:22:49.745 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T18:22:49.745 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T18:22:49.745 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T18:22:49.745 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T18:22:49.745 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T18:22:49.746 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2023-11-28T18:22:49.746 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T18:22:49.746 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T18:22:49.746 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T18:22:49.746 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T18:22:50.215 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2023-11-28T18:22:50.219 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2023-11-28T18:22:50.219 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-28T18:22:50.358 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-28T18:22:50.375 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: /sbin/restorecon: command not found 2023-11-28T18:22:50.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:22:50.376 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-28T18:22:50.444 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T18:22:50.444 INFO:tasks.ceph:role: osd.5 2023-11-28T18:22:50.444 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:22:50.444 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T18:22:50.515 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T18:22:50.516 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T18:22:50.516 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T18:22:50.516 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T18:22:50.516 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T18:22:50.516 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2023-11-28T18:22:50.516 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T18:22:50.516 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T18:22:50.516 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T18:22:50.516 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T18:22:50.956 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2023-11-28T18:22:50.959 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2023-11-28T18:22:50.959 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-28T18:22:51.053 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-28T18:22:51.070 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: /sbin/restorecon: command not found 2023-11-28T18:22:51.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:22:51.071 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-28T18:22:51.140 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T18:22:51.140 INFO:tasks.ceph:role: osd.6 2023-11-28T18:22:51.140 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:22:51.140 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T18:22:51.215 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T18:22:51.216 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T18:22:51.216 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T18:22:51.216 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T18:22:51.216 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T18:22:51.216 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2023-11-28T18:22:51.216 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T18:22:51.216 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T18:22:51.216 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T18:22:51.216 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T18:22:51.675 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2023-11-28T18:22:51.678 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2023-11-28T18:22:51.678 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-28T18:22:51.838 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-28T18:22:51.856 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: /sbin/restorecon: command not found 2023-11-28T18:22:51.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:22:51.857 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-28T18:22:51.925 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T18:22:51.926 INFO:tasks.ceph:role: osd.7 2023-11-28T18:22:51.926 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:22:51.926 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T18:22:51.997 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T18:22:51.997 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T18:22:51.997 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T18:22:51.998 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T18:22:51.998 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T18:22:51.998 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2023-11-28T18:22:51.998 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T18:22:51.998 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T18:22:51.998 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T18:22:51.998 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T18:22:52.481 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2023-11-28T18:22:52.485 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2023-11-28T18:22:52.486 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-28T18:22:52.649 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-28T18:22:52.667 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: /sbin/restorecon: command not found 2023-11-28T18:22:52.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:22:52.668 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T18:22:52.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:22:52.791+0000 7f98dbb3c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-28T18:22:52.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:22:52.795+0000 7f98dbb3c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-28T18:22:52.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:22:52.795+0000 7f98dbb3c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T18:22:52.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:22:52.795+0000 7f98dbb3c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T18:22:52.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:22:52.795+0000 7f98dbb3c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T18:22:52.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:22:52.799+0000 7f98dbb3c640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-28T18:22:55.807 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-28T18:22:55.829 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T18:22:55.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:22:55.927+0000 7fa9d30c7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-28T18:22:55.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:22:55.927+0000 7fa9d30c7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-28T18:22:55.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:22:55.927+0000 7fa9d30c7640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T18:22:55.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:22:55.927+0000 7fa9d30c7640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T18:22:55.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:22:55.927+0000 7fa9d30c7640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T18:22:55.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:22:55.931+0000 7fa9d30c7640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-28T18:22:58.931 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-28T18:22:58.953 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T18:22:59.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:22:59.054+0000 7f5c067a7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-28T18:22:59.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:22:59.054+0000 7f5c067a7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-28T18:22:59.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:22:59.054+0000 7f5c067a7640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T18:22:59.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:22:59.054+0000 7f5c067a7640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T18:22:59.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:22:59.054+0000 7f5c067a7640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T18:22:59.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:22:59.054+0000 7f5c067a7640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-28T18:23:00.914 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:23:00.919 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:23:02.058 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-28T18:23:02.079 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T18:23:02.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:02.178+0000 7fe0d3be1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-28T18:23:02.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:02.178+0000 7fe0d3be1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-28T18:23:02.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:02.178+0000 7fe0d3be1640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T18:23:02.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:02.178+0000 7fe0d3be1640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T18:23:02.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:02.178+0000 7fe0d3be1640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T18:23:02.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:02.178+0000 7fe0d3be1640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-28T18:23:05.176 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-28T18:23:05.199 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-28T18:23:05.199 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:23:05.200 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-28T18:23:05.223 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:23:05.223 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-28T18:23:05.285 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:23:05.285 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-28T18:23:05.350 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:23:05.350 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-28T18:23:05.412 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:23:05.413 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-28T18:23:05.477 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:23:05.477 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-28T18:23:05.543 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:23:05.544 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-28T18:23:05.610 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:23:05.610 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-28T18:23:05.632 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:23:05.632 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-28T18:23:05.699 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:23:05.699 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-28T18:23:05.762 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:23:05.762 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-28T18:23:05.829 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:23:05.829 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-28T18:23:05.893 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:23:05.893 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-28T18:23:05.957 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:23:05.957 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-28T18:23:06.023 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:23:06.023 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-28T18:23:06.033 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:23:06.033 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-28T18:23:06.084 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:23:06.084 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-28T18:23:06.094 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:23:06.095 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-28T18:23:06.144 INFO:tasks.ceph:Adding keys to all mons... 2023-11-28T18:23:06.145 DEBUG:teuthology.orchestra.run.smithi105:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T18:23:06.148 DEBUG:teuthology.orchestra.run.smithi148:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T18:23:06.194 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr.y] 2023-11-28T18:23:06.194 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDpL2ZlU89RExAAwMJp+8zZ6aFEt0KbiB6toA== 2023-11-28T18:23:06.195 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2023-11-28T18:23:06.195 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDpL2ZlOXSTGxAAU5kGNB3tznYsDNG51vCIZA== 2023-11-28T18:23:06.195 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2023-11-28T18:23:06.195 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDpL2Zl5Y7GIhAA5OvgkNNFaI7LnzSsQVdFsQ== 2023-11-28T18:23:06.195 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.0] 2023-11-28T18:23:06.195 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDtL2ZlphxQEBAAQ280OXh1zeaotIdVU8oZng== 2023-11-28T18:23:06.195 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.1] 2023-11-28T18:23:06.195 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDwL2ZlezmOFxAARRBIOTkVJYfquIdcpW7sQQ== 2023-11-28T18:23:06.195 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.2] 2023-11-28T18:23:06.195 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDzL2Zl4YcSHxAA2ww2F4TAD/0H/ZJ+aVMGEQ== 2023-11-28T18:23:06.195 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.3] 2023-11-28T18:23:06.195 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQD2L2ZlNxWxJRAAImp7TjTF18KwFTZ14DM6EA== 2023-11-28T18:23:06.196 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr.x] 2023-11-28T18:23:06.196 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDpL2ZlEG49FxAAespsG+QK1WTpbmpV+DIwiw== 2023-11-28T18:23:06.196 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2023-11-28T18:23:06.196 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDpL2ZlrnbWJxAAKefl0YI2MhobnAOIj6WeaQ== 2023-11-28T18:23:06.196 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2023-11-28T18:23:06.196 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDpL2ZlmfYdLxAAreI7UE1YCjzkj6yl3s5Fwg== 2023-11-28T18:23:06.196 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.4] 2023-11-28T18:23:06.196 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQD8L2ZlOAieLxAAz8g+anH8uJRfsttUS41GGQ== 2023-11-28T18:23:06.196 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.5] 2023-11-28T18:23:06.196 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQD/L2Zl5PKTNxAAPk+uIodLPjSd0EEMVT6iEw== 2023-11-28T18:23:06.196 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.6] 2023-11-28T18:23:06.196 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQADMGZll+FUAxAAHlym5Z8rd/OF1911ZE2OUg== 2023-11-28T18:23:06.196 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.7] 2023-11-28T18:23:06.197 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQAGMGZl8GO4ChAAVn3N6DnxUcs+ew09QvGo2w== 2023-11-28T18:23:06.197 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2023-11-28T18:23:06.197 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDpL2ZlSt0VMxAAQ8+j1BgBPMJ2cuiZj8aG0w== 2023-11-28T18:23:06.197 INFO:teuthology.orchestra.run.smithi105.stdout:[client.2] 2023-11-28T18:23:06.197 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDpL2ZlPLtyORAAU2GST6TJzlTfh6YqEb3U3w== 2023-11-28T18:23:06.197 INFO:teuthology.orchestra.run.smithi105.stdout:[client.1] 2023-11-28T18:23:06.197 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDqL2ZlfZ1/ARAAhnJB6oj4Cc6p/oCjM6hTCw== 2023-11-28T18:23:06.197 INFO:teuthology.orchestra.run.smithi105.stdout:[client.3] 2023-11-28T18:23:06.197 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDqL2ZlhiFuBxAAKQRxCdtqWpAHy1AVvdj9Sg== 2023-11-28T18:23:06.209 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr.y] 2023-11-28T18:23:06.209 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDpL2ZlU89RExAAwMJp+8zZ6aFEt0KbiB6toA== 2023-11-28T18:23:06.209 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.a] 2023-11-28T18:23:06.209 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDpL2ZlOXSTGxAAU5kGNB3tznYsDNG51vCIZA== 2023-11-28T18:23:06.209 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.c] 2023-11-28T18:23:06.209 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDpL2Zl5Y7GIhAA5OvgkNNFaI7LnzSsQVdFsQ== 2023-11-28T18:23:06.209 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.0] 2023-11-28T18:23:06.209 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDtL2ZlphxQEBAAQ280OXh1zeaotIdVU8oZng== 2023-11-28T18:23:06.209 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.1] 2023-11-28T18:23:06.209 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDwL2ZlezmOFxAARRBIOTkVJYfquIdcpW7sQQ== 2023-11-28T18:23:06.210 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.2] 2023-11-28T18:23:06.210 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDzL2Zl4YcSHxAA2ww2F4TAD/0H/ZJ+aVMGEQ== 2023-11-28T18:23:06.210 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.3] 2023-11-28T18:23:06.210 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQD2L2ZlNxWxJRAAImp7TjTF18KwFTZ14DM6EA== 2023-11-28T18:23:06.210 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr.x] 2023-11-28T18:23:06.210 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDpL2ZlEG49FxAAespsG+QK1WTpbmpV+DIwiw== 2023-11-28T18:23:06.210 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.b] 2023-11-28T18:23:06.210 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDpL2ZlrnbWJxAAKefl0YI2MhobnAOIj6WeaQ== 2023-11-28T18:23:06.210 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.d] 2023-11-28T18:23:06.211 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDpL2ZlmfYdLxAAreI7UE1YCjzkj6yl3s5Fwg== 2023-11-28T18:23:06.211 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.4] 2023-11-28T18:23:06.211 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQD8L2ZlOAieLxAAz8g+anH8uJRfsttUS41GGQ== 2023-11-28T18:23:06.211 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.5] 2023-11-28T18:23:06.211 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQD/L2Zl5PKTNxAAPk+uIodLPjSd0EEMVT6iEw== 2023-11-28T18:23:06.211 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.6] 2023-11-28T18:23:06.211 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQADMGZll+FUAxAAHlym5Z8rd/OF1911ZE2OUg== 2023-11-28T18:23:06.211 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.7] 2023-11-28T18:23:06.211 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAGMGZl8GO4ChAAVn3N6DnxUcs+ew09QvGo2w== 2023-11-28T18:23:06.211 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2023-11-28T18:23:06.211 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDpL2ZlSt0VMxAAQ8+j1BgBPMJ2cuiZj8aG0w== 2023-11-28T18:23:06.211 INFO:teuthology.orchestra.run.smithi148.stdout:[client.2] 2023-11-28T18:23:06.211 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDpL2ZlPLtyORAAU2GST6TJzlTfh6YqEb3U3w== 2023-11-28T18:23:06.212 INFO:teuthology.orchestra.run.smithi148.stdout:[client.1] 2023-11-28T18:23:06.212 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDqL2ZlfZ1/ARAAhnJB6oj4Cc6p/oCjM6hTCw== 2023-11-28T18:23:06.212 INFO:teuthology.orchestra.run.smithi148.stdout:[client.3] 2023-11-28T18:23:06.212 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDqL2ZlhiFuBxAAKQRxCdtqWpAHy1AVvdj9Sg== 2023-11-28T18:23:06.213 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T18:23:06.243 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T18:23:06.313 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T18:23:06.316 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T18:23:06.375 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T18:23:06.379 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T18:23:06.435 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T18:23:06.438 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T18:23:06.492 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T18:23:06.495 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T18:23:06.549 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T18:23:06.553 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T18:23:06.607 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T18:23:06.611 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T18:23:06.664 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T18:23:06.669 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T18:23:06.723 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T18:23:06.726 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T18:23:06.783 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T18:23:06.787 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T18:23:06.844 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T18:23:06.848 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T18:23:06.903 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T18:23:06.906 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T18:23:06.961 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T18:23:06.964 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T18:23:07.020 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T18:23:07.024 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T18:23:07.079 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T18:23:07.083 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T18:23:07.141 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T18:23:07.145 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T18:23:07.202 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T18:23:07.206 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T18:23:07.263 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T18:23:07.267 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T18:23:07.325 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-28T18:23:07.325 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-28T18:23:07.345 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T18:23:07.461 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-28T18:23:07.482 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-28T18:23:07.502 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T18:23:07.622 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-28T18:23:07.642 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-28T18:23:07.703 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T18:23:07.821 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-28T18:23:07.841 DEBUG:teuthology.orchestra.run.smithi105:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T18:23:07.844 DEBUG:teuthology.orchestra.run.smithi148:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T18:23:07.892 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-28T18:23:07.893 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-28T18:23:07.893 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-28T18:23:07.896 INFO:tasks.ceph.mon.a:Started 2023-11-28T18:23:07.897 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-28T18:23:07.897 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-28T18:23:07.935 INFO:tasks.ceph.mon.b:Started 2023-11-28T18:23:07.936 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-28T18:23:07.936 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-28T18:23:07.940 INFO:tasks.ceph.mon.c:Started 2023-11-28T18:23:07.940 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-28T18:23:07.940 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-28T18:23:07.940 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-28T18:23:07.945 INFO:tasks.ceph.mgr.y:Started 2023-11-28T18:23:07.945 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-28T18:23:07.945 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-28T18:23:07.948 INFO:tasks.ceph.mgr.x:Started 2023-11-28T18:23:07.949 INFO:tasks.ceph:Setting crush tunables to default 2023-11-28T18:23:07.949 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd crush tunables default 2023-11-28T18:23:08.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.089+0000 7f329da9a640 1 Processor -- start 2023-11-28T18:23:08.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f329da9a640 1 -- start start 2023-11-28T18:23:08.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f329da9a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f329815d4f0 0x7f32980d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:08.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f329da9a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32980d3c30 0x7f32980d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:08.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f329da9a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f32980d1890 0x7f32980cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:08.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f329da9a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32980b2900 con 0x7f329815d4f0 2023-11-28T18:23:08.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f329da9a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32980b25b0 con 0x7f32980d1890 2023-11-28T18:23:08.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f329da9a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32980cea30 con 0x7f32980d3c30 2023-11-28T18:23:08.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f32977fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f329815d4f0 0x7f32980d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:08.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f32977fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f329815d4f0 0x7f32980d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47742/0 (socket says 172.21.15.105:47742) 2023-11-28T18:23:08.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f32977fe640 1 -- 172.21.15.105:0/3495248058 learned_addr learned my addr 172.21.15.105:0/3495248058 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:08.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f3297fff640 1 --2- 172.21.15.105:0/3495248058 >> v2:172.21.15.148:3300/0 conn(0x7f32980d1890 0x7f32980cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:08.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f328ffff640 1 --2- 172.21.15.105:0/3495248058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32980d3c30 0x7f32980d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:08.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f3297fff640 1 -- 172.21.15.105:0/3495248058 >> v2:172.21.15.148:3300/0 conn(0x7f32980d1890 msgr2=0x7f32980cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-28T18:23:08.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f3297fff640 1 -- 172.21.15.105:0/3495248058 >> v2:172.21.15.148:3300/0 conn(0x7f32980d1890 msgr2=0x7f32980cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T18:23:08.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f3297fff640 1 --2- 172.21.15.105:0/3495248058 >> v2:172.21.15.148:3300/0 conn(0x7f32980d1890 0x7f32980cfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T18:23:08.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f3297fff640 1 --2- 172.21.15.105:0/3495248058 >> v2:172.21.15.148:3300/0 conn(0x7f32980d1890 0x7f32980cfa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:23:08.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f32977fe640 1 -- 172.21.15.105:0/3495248058 >> v2:172.21.15.105:3300/0 conn(0x7f329815d4f0 msgr2=0x7f32980d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-28T18:23:08.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f32977fe640 1 -- 172.21.15.105:0/3495248058 >> v2:172.21.15.105:3300/0 conn(0x7f329815d4f0 msgr2=0x7f32980d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T18:23:08.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f32977fe640 1 --2- 172.21.15.105:0/3495248058 >> v2:172.21.15.105:3300/0 conn(0x7f329815d4f0 0x7f32980d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T18:23:08.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f32977fe640 1 --2- 172.21.15.105:0/3495248058 >> v2:172.21.15.105:3300/0 conn(0x7f329815d4f0 0x7f32980d0e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:23:08.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f328ffff640 1 -- 172.21.15.105:0/3495248058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32980d3c30 msgr2=0x7f32980d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-28T18:23:08.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f328ffff640 1 -- 172.21.15.105:0/3495248058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32980d3c30 msgr2=0x7f32980d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T18:23:08.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f328ffff640 1 --2- 172.21.15.105:0/3495248058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32980d3c30 0x7f32980d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T18:23:08.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.093+0000 7f328ffff640 1 --2- 172.21.15.105:0/3495248058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32980d3c30 0x7f32980d1350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:23:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.297+0000 7f32977fe640 1 --2- 172.21.15.105:0/3495248058 >> v2:172.21.15.105:3300/0 conn(0x7f329815d4f0 0x7f32980d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.297+0000 7f3297fff640 1 --2- 172.21.15.105:0/3495248058 >> v2:172.21.15.148:3300/0 conn(0x7f32980d1890 0x7f32980cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.297+0000 7f328ffff640 1 --2- 172.21.15.105:0/3495248058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32980d3c30 0x7f32980d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.297+0000 7f32977fe640 1 -- 172.21.15.105:0/3495248058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32980d3c30 msgr2=0x7f32980d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.297+0000 7f32977fe640 1 --2- 172.21.15.105:0/3495248058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32980d3c30 0x7f32980d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.297+0000 7f32977fe640 1 -- 172.21.15.105:0/3495248058 >> v2:172.21.15.148:3300/0 conn(0x7f32980d1890 msgr2=0x7f32980cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.297+0000 7f32977fe640 1 --2- 172.21.15.105:0/3495248058 >> v2:172.21.15.148:3300/0 conn(0x7f32980d1890 0x7f32980cfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.297+0000 7f32977fe640 1 -- 172.21.15.105:0/3495248058 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32980ced10 con 0x7f329815d4f0 2023-11-28T18:23:08.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.297+0000 7f3297fff640 1 --2- 172.21.15.105:0/3495248058 >> v2:172.21.15.148:3300/0 conn(0x7f32980d1890 0x7f32980cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:08.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.297+0000 7f32977fe640 1 --2- 172.21.15.105:0/3495248058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f329815d4f0 0x7f32980d0e10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3298160fa0 tx=0x7f328801c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d817a12691f25c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:08.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.297+0000 7f3296ffd640 1 -- 172.21.15.105:0/3495248058 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32880a6670 con 0x7f329815d4f0 2023-11-28T18:23:08.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.301+0000 7f3296ffd640 1 -- 172.21.15.105:0/3495248058 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32880a67d0 con 0x7f329815d4f0 2023-11-28T18:23:08.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.301+0000 7f3296ffd640 1 -- 172.21.15.105:0/3495248058 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32880a6b80 con 0x7f329815d4f0 2023-11-28T18:23:08.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.301+0000 7f329da9a640 1 -- 172.21.15.105:0/3495248058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f329815d4f0 msgr2=0x7f32980d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:08.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.301+0000 7f329da9a640 1 --2- 172.21.15.105:0/3495248058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f329815d4f0 0x7f32980d0e10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3298160fa0 tx=0x7f328801c6d0 comp rx=0 tx=0).stop 2023-11-28T18:23:08.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.301+0000 7f329da9a640 1 -- 172.21.15.105:0/3495248058 shutdown_connections 2023-11-28T18:23:08.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.301+0000 7f329da9a640 1 --2- 172.21.15.105:0/3495248058 >> v2:172.21.15.148:3300/0 conn(0x7f32980d1890 0x7f32980cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:08.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.301+0000 7f329da9a640 1 --2- 172.21.15.105:0/3495248058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32980d3c30 0x7f32980d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:08.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.301+0000 7f329da9a640 1 --2- 172.21.15.105:0/3495248058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f329815d4f0 0x7f32980d0e10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:08.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.301+0000 7f329da9a640 1 -- 172.21.15.105:0/3495248058 >> 172.21.15.105:0/3495248058 conn(0x7f32980d3030 msgr2=0x7f32980d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:08.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.301+0000 7f329da9a640 1 -- 172.21.15.105:0/3495248058 shutdown_connections 2023-11-28T18:23:08.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.301+0000 7f329da9a640 1 -- 172.21.15.105:0/3495248058 wait complete. 2023-11-28T18:23:08.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.301+0000 7f329da9a640 1 Processor -- start 2023-11-28T18:23:08.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.301+0000 7f329da9a640 1 -- start start 2023-11-28T18:23:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.301+0000 7f329da9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32980d1890 0x7f329814b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.301+0000 7f329da9a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32980d3c30 0x7f3298154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.301+0000 7f329da9a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f329815d4f0 0x7f32980cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.301+0000 7f329da9a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3298157970 con 0x7f32980d1890 2023-11-28T18:23:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.301+0000 7f329da9a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3298154bc0 con 0x7f329815d4f0 2023-11-28T18:23:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.301+0000 7f329da9a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3298154ea0 con 0x7f32980d3c30 2023-11-28T18:23:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.301+0000 7f32977fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32980d1890 0x7f329814b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.301+0000 7f32977fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32980d1890 0x7f329814b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47772/0 (socket says 172.21.15.105:47772) 2023-11-28T18:23:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.301+0000 7f32977fe640 1 -- 172.21.15.105:0/2086949537 learned_addr learned my addr 172.21.15.105:0/2086949537 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.305+0000 7f3297fff640 1 --2- 172.21.15.105:0/2086949537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f329815d4f0 0x7f32980cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.305+0000 7f328ffff640 1 --2- 172.21.15.105:0/2086949537 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32980d3c30 0x7f3298154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.305+0000 7f32977fe640 1 -- 172.21.15.105:0/2086949537 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32980d3c30 msgr2=0x7f3298154590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.305+0000 7f32977fe640 1 --2- 172.21.15.105:0/2086949537 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32980d3c30 0x7f3298154590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.305+0000 7f32977fe640 1 -- 172.21.15.105:0/2086949537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f329815d4f0 msgr2=0x7f32980cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.305+0000 7f32977fe640 1 --2- 172.21.15.105:0/2086949537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f329815d4f0 0x7f32980cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.305+0000 7f32977fe640 1 -- 172.21.15.105:0/2086949537 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3288009530 con 0x7f32980d1890 2023-11-28T18:23:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.305+0000 7f3297fff640 1 --2- 172.21.15.105:0/2086949537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f329815d4f0 0x7f32980cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.305+0000 7f32977fe640 1 --2- 172.21.15.105:0/2086949537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32980d1890 0x7f329814b4b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f328801c710 tx=0x7f3288019470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.305+0000 7f3294ff9640 1 -- 172.21.15.105:0/2086949537 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32880b6070 con 0x7f32980d1890 2023-11-28T18:23:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.305+0000 7f3294ff9640 1 -- 172.21.15.105:0/2086949537 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32880a6b50 con 0x7f32980d1890 2023-11-28T18:23:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.305+0000 7f329da9a640 1 -- 172.21.15.105:0/2086949537 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3298155180 con 0x7f32980d1890 2023-11-28T18:23:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.305+0000 7f3294ff9640 1 -- 172.21.15.105:0/2086949537 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3288003620 con 0x7f32980d1890 2023-11-28T18:23:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.305+0000 7f329da9a640 1 -- 172.21.15.105:0/2086949537 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3298155460 con 0x7f32980d1890 2023-11-28T18:23:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.305+0000 7f3294ff9640 1 -- 172.21.15.105:0/2086949537 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3288003780 con 0x7f32980d1890 2023-11-28T18:23:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.305+0000 7f3294ff9640 1 -- 172.21.15.105:0/2086949537 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f32880b1950 con 0x7f32980d1890 2023-11-28T18:23:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.305+0000 7f329da9a640 1 -- 172.21.15.105:0/2086949537 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3298157b40 con 0x7f32980d1890 2023-11-28T18:23:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.317+0000 7f3294ff9640 1 -- 172.21.15.105:0/2086949537 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f32880ac070 con 0x7f32980d1890 2023-11-28T18:23:08.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.377+0000 7f329da9a640 1 -- 172.21.15.105:0/2086949537 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f3298067570 con 0x7f32980d1890 2023-11-28T18:23:08.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.381+0000 7f3294ff9640 1 -- 172.21.15.105:0/2086949537 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f328800f040 con 0x7f32980d1890 2023-11-28T18:23:08.382 INFO:teuthology.orchestra.run.smithi105.stderr:adjusted tunables profile to default 2023-11-28T18:23:08.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.381+0000 7f329da9a640 1 -- 172.21.15.105:0/2086949537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32980d1890 msgr2=0x7f329814b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:08.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.381+0000 7f329da9a640 1 --2- 172.21.15.105:0/2086949537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32980d1890 0x7f329814b4b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f328801c710 tx=0x7f3288019470 comp rx=0 tx=0).stop 2023-11-28T18:23:08.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.381+0000 7f329da9a640 1 -- 172.21.15.105:0/2086949537 shutdown_connections 2023-11-28T18:23:08.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.381+0000 7f329da9a640 1 --2- 172.21.15.105:0/2086949537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f329815d4f0 0x7f32980cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:08.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.381+0000 7f329da9a640 1 --2- 172.21.15.105:0/2086949537 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32980d3c30 0x7f3298154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:08.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.381+0000 7f329da9a640 1 --2- 172.21.15.105:0/2086949537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32980d1890 0x7f329814b4b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:08.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.381+0000 7f329da9a640 1 -- 172.21.15.105:0/2086949537 >> 172.21.15.105:0/2086949537 conn(0x7f32980d3030 msgr2=0x7f32980cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:08.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.381+0000 7f329da9a640 1 -- 172.21.15.105:0/2086949537 shutdown_connections 2023-11-28T18:23:08.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:08.381+0000 7f329da9a640 1 -- 172.21.15.105:0/2086949537 wait complete. 2023-11-28T18:23:08.402 INFO:tasks.ceph:check_enable_crimson: False 2023-11-28T18:23:08.402 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-28T18:23:08.402 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:23:08.402 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-28T18:23:08.423 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:23:08.424 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-28T18:23:08.488 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:23:08.488 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-28T18:23:08.491 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:08.489+0000 7fe468e51180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T18:23:08.513 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:23:08.514 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-28T18:23:08.525 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:08.526+0000 7ffa104f0180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T18:23:08.577 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:23:08.577 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-28T18:23:08.599 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:23:08.599 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-28T18:23:08.661 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:23:08.661 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-28T18:23:08.725 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:23:08.725 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-28T18:23:08.788 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new ef4e9b9f-3655-47a8-89b8-a34b1dedbe70 0 2023-11-28T18:23:08.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.965+0000 7fb4c770f640 1 Processor -- start 2023-11-28T18:23:08.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.969+0000 7fb4c770f640 1 -- start start 2023-11-28T18:23:08.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.969+0000 7fb4c770f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb4c00d10e0 0x7fb4c00d3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:08.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.969+0000 7fb4c770f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4c00b4140 0x7fb4c00d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:08.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.969+0000 7fb4c770f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c00d1d20 con 0x7fb4c0160530 2023-11-28T18:23:08.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.969+0000 7fb4c770f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4c00d01d0 con 0x7fb4c00d10e0 2023-11-28T18:23:08.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.969+0000 7fb4c770f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c00d04b0 con 0x7fb4c00b4140 2023-11-28T18:23:08.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.969+0000 7fb4c4caa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4c00b4140 0x7fb4c00d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:08.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.969+0000 7fb4c54ab640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb4c00d10e0 0x7fb4c00d3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:08.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.969+0000 7fb4c54ab640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb4c00d10e0 0x7fb4c00d3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53916/0 (socket says 172.21.15.148:53916) 2023-11-28T18:23:08.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.969+0000 7fb4c4caa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4c00b4140 0x7fb4c00d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.148:35410/0 (socket says 172.21.15.148:35410) 2023-11-28T18:23:08.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.969+0000 7fb4c54ab640 1 -- 172.21.15.148:0/665566448 learned_addr learned my addr 172.21.15.148:0/665566448 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-11-28T18:23:08.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.969+0000 7fb4c5cac640 1 --1- 172.21.15.148:0/665566448 >> v1:172.21.15.148:6789/0 conn(0x7fb4c0160530 0x7fb4c0160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39398/0 (socket says 172.21.15.148:39398) 2023-11-28T18:23:08.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.969+0000 7fb4c4caa640 1 -- 172.21.15.148:0/665566448 >> v2:172.21.15.148:3300/0 conn(0x7fb4c00d10e0 msgr2=0x7fb4c00d3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:08.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.969+0000 7fb4c4caa640 1 --2- 172.21.15.148:0/665566448 >> v2:172.21.15.148:3300/0 conn(0x7fb4c00d10e0 0x7fb4c00d3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:08.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.969+0000 7fb4c4caa640 1 -- 172.21.15.148:0/665566448 >> v1:172.21.15.148:6789/0 conn(0x7fb4c0160530 legacy=0x7fb4c0160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:08.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.969+0000 7fb4c4caa640 1 -- 172.21.15.148:0/665566448 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c00d0790 con 0x7fb4c00b4140 2023-11-28T18:23:08.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.969+0000 7fb4b7fff640 1 -- 172.21.15.148:0/665566448 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2932307730 0 0) 0x7fb4c00d1d20 con 0x7fb4c0160530 2023-11-28T18:23:08.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.969+0000 7fb4c54ab640 1 --2- 172.21.15.148:0/665566448 >> v2:172.21.15.148:3300/0 conn(0x7fb4c00d10e0 0x7fb4c00d3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:08.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.973+0000 7fb4c4caa640 1 --2- 172.21.15.148:0/665566448 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4c00b4140 0x7fb4c00d14c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb4b00097c0 tx=0x7fb4b001ce10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6abfba0af7d9ca4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:08.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.973+0000 7fb4b7fff640 1 -- 172.21.15.148:0/665566448 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b00ab610 con 0x7fb4c00b4140 2023-11-28T18:23:08.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.973+0000 7fb4b7fff640 1 -- 172.21.15.148:0/665566448 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b00ab770 con 0x7fb4c00b4140 2023-11-28T18:23:08.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.973+0000 7fb4b7fff640 1 -- 172.21.15.148:0/665566448 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b00abb00 con 0x7fb4c00b4140 2023-11-28T18:23:08.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.973+0000 7fb4c770f640 1 -- 172.21.15.148:0/665566448 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4c00b4140 msgr2=0x7fb4c00d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:08.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.973+0000 7fb4c770f640 1 --2- 172.21.15.148:0/665566448 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4c00b4140 0x7fb4c00d14c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb4b00097c0 tx=0x7fb4b001ce10 comp rx=0 tx=0).stop 2023-11-28T18:23:08.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.973+0000 7fb4c770f640 1 -- 172.21.15.148:0/665566448 shutdown_connections 2023-11-28T18:23:08.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.973+0000 7fb4c770f640 1 --2- 172.21.15.148:0/665566448 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4c00b4140 0x7fb4c00d14c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:08.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.973+0000 7fb4c770f640 1 --2- 172.21.15.148:0/665566448 >> v2:172.21.15.148:3300/0 conn(0x7fb4c00d10e0 0x7fb4c00d3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:08.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.973+0000 7fb4c770f640 1 -- 172.21.15.148:0/665566448 >> 172.21.15.148:0/665566448 conn(0x7fb4c00d30a0 msgr2=0x7fb4c00d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:08.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.973+0000 7fb4c770f640 1 -- 172.21.15.148:0/665566448 shutdown_connections 2023-11-28T18:23:08.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.973+0000 7fb4c770f640 1 -- 172.21.15.148:0/665566448 wait complete. 2023-11-28T18:23:08.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4c770f640 1 Processor -- start 2023-11-28T18:23:08.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4c770f640 1 -- start start 2023-11-28T18:23:08.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4c770f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4c00b4140 0x7fb4c0159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:08.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4c770f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c00d10e0 0x7fb4c00d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:08.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4c770f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4c0160530 0x7fb4c0154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:08.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4c770f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c00cb2f0 con 0x7fb4c0160530 2023-11-28T18:23:08.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4c770f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c00cbde0 con 0x7fb4c00d10e0 2023-11-28T18:23:08.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4c770f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c00ca6e0 con 0x7fb4c00b4140 2023-11-28T18:23:08.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4c4caa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c00d10e0 0x7fb4c00d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:08.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4c54ab640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4c00b4140 0x7fb4c0159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:08.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4c4caa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c00d10e0 0x7fb4c00d36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53926/0 (socket says 172.21.15.148:53926) 2023-11-28T18:23:08.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4c4caa640 1 -- 172.21.15.148:0/1881572907 learned_addr learned my addr 172.21.15.148:0/1881572907 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-11-28T18:23:08.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4c4caa640 1 -- 172.21.15.148:0/1881572907 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4c00b4140 msgr2=0x7fb4c0159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:08.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4c4caa640 1 --2- 172.21.15.148:0/1881572907 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4c00b4140 0x7fb4c0159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:08.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4c4caa640 1 -- 172.21.15.148:0/1881572907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4c0160530 msgr2=0x7fb4c0154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:08.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4c4caa640 1 --2- 172.21.15.148:0/1881572907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4c0160530 0x7fb4c0154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:08.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4c4caa640 1 -- 172.21.15.148:0/1881572907 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4bc0094d0 con 0x7fb4c00d10e0 2023-11-28T18:23:08.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4c4caa640 1 --2- 172.21.15.148:0/1881572907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c00d10e0 0x7fb4c00d36e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb4b00ad040 tx=0x7fb4b001abf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:08.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4b5ffb640 1 -- 172.21.15.148:0/1881572907 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b00bf610 con 0x7fb4c00d10e0 2023-11-28T18:23:08.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4b5ffb640 1 -- 172.21.15.148:0/1881572907 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b00bf770 con 0x7fb4c00d10e0 2023-11-28T18:23:08.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4b5ffb640 1 -- 172.21.15.148:0/1881572907 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b00bfb20 con 0x7fb4c00d10e0 2023-11-28T18:23:08.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4c770f640 1 -- 172.21.15.148:0/1881572907 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b0009420 con 0x7fb4c00d10e0 2023-11-28T18:23:08.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4c770f640 1 -- 172.21.15.148:0/1881572907 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c00caca0 con 0x7fb4c00d10e0 2023-11-28T18:23:08.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4b5ffb640 1 -- 172.21.15.148:0/1881572907 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb4b00bfc80 con 0x7fb4c00d10e0 2023-11-28T18:23:08.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4b5ffb640 1 -- 172.21.15.148:0/1881572907 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb4b00b5ca0 con 0x7fb4c00d10e0 2023-11-28T18:23:08.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.977+0000 7fb4c770f640 1 -- 172.21.15.148:0/1881572907 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c00cb460 con 0x7fb4c00d10e0 2023-11-28T18:23:08.990 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:08.989+0000 7fb4b5ffb640 1 -- 172.21.15.148:0/1881572907 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fb4b00c4020 con 0x7fb4c00d10e0 2023-11-28T18:23:09.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.053+0000 7fb4c770f640 1 -- 172.21.15.148:0/1881572907 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ef4e9b9f-3655-47a8-89b8-a34b1dedbe70", "id": 0} v 0) v1 -- 0x7fb4c0068650 con 0x7fb4c00d10e0 2023-11-28T18:23:09.059 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:09.057+0000 7fe468e51180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T18:23:09.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.061+0000 7fb4b5ffb640 1 -- 172.21.15.148:0/1881572907 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ef4e9b9f-3655-47a8-89b8-a34b1dedbe70", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb4b00b5400 con 0x7fb4c00d10e0 2023-11-28T18:23:09.064 INFO:teuthology.orchestra.run.smithi148.stdout:0 2023-11-28T18:23:09.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.061+0000 7fb4c770f640 1 -- 172.21.15.148:0/1881572907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c00d10e0 msgr2=0x7fb4c00d36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:09.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.061+0000 7fb4c770f640 1 --2- 172.21.15.148:0/1881572907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c00d10e0 0x7fb4c00d36e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb4b00ad040 tx=0x7fb4b001abf0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.061+0000 7fb4c770f640 1 -- 172.21.15.148:0/1881572907 shutdown_connections 2023-11-28T18:23:09.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.061+0000 7fb4c770f640 1 --2- 172.21.15.148:0/1881572907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4c0160530 0x7fb4c0154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.061+0000 7fb4c770f640 1 --2- 172.21.15.148:0/1881572907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c00d10e0 0x7fb4c00d36e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.061+0000 7fb4c770f640 1 --2- 172.21.15.148:0/1881572907 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4c00b4140 0x7fb4c0159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.061+0000 7fb4c770f640 1 -- 172.21.15.148:0/1881572907 >> 172.21.15.148:0/1881572907 conn(0x7fb4c00d30a0 msgr2=0x7fb4c00c82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:09.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.065+0000 7fb4c770f640 1 -- 172.21.15.148:0/1881572907 shutdown_connections 2023-11-28T18:23:09.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.065+0000 7fb4c770f640 1 -- 172.21.15.148:0/1881572907 wait complete. 2023-11-28T18:23:09.083 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 2738deef-83f7-4af6-871c-3e947cdc6954 1 2023-11-28T18:23:09.087 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:09.085+0000 7ffa104f0180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T18:23:09.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.217+0000 7f5788a17640 1 Processor -- start 2023-11-28T18:23:09.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.217+0000 7f5788a17640 1 -- start start 2023-11-28T18:23:09.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.221+0000 7f5788a17640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f578415d4f0 0x7f57840d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:09.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.221+0000 7f5788a17640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f57840b2d20 0x7f57840b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:09.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.221+0000 7f5788a17640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57840b25b0 con 0x7f57840d3c30 2023-11-28T18:23:09.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.221+0000 7f5788a17640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57840b2900 con 0x7f578415d4f0 2023-11-28T18:23:09.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.221+0000 7f5788a17640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57840d0a20 con 0x7f57840b2d20 2023-11-28T18:23:09.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.221+0000 7f5782d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f57840b2d20 0x7f57840b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:09.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.221+0000 7f5782d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f57840b2d20 0x7f57840b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53928/0 (socket says 172.21.15.148:53928) 2023-11-28T18:23:09.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.221+0000 7f5782d9d640 1 -- 172.21.15.148:0/1840705322 learned_addr learned my addr 172.21.15.148:0/1840705322 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-11-28T18:23:09.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.221+0000 7f578259c640 1 --2- 172.21.15.148:0/1840705322 >> v2:172.21.15.105:3300/0 conn(0x7f578415d4f0 0x7f57840d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:09.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.221+0000 7f5782d9d640 1 -- 172.21.15.148:0/1840705322 >> v2:172.21.15.105:3300/0 conn(0x7f578415d4f0 msgr2=0x7f57840d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:09.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.221+0000 7f5782d9d640 1 --2- 172.21.15.148:0/1840705322 >> v2:172.21.15.105:3300/0 conn(0x7f578415d4f0 0x7f57840d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.221+0000 7f5782d9d640 1 -- 172.21.15.148:0/1840705322 >> v1:172.21.15.105:6789/0 conn(0x7f57840d3c30 legacy=0x7f57840d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:09.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.221+0000 7f5782d9d640 1 -- 172.21.15.148:0/1840705322 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57840cea30 con 0x7f57840b2d20 2023-11-28T18:23:09.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.221+0000 7f5782d9d640 1 --2- 172.21.15.148:0/1840705322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57840b2d20 0x7f57840b3150 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5784160fa0 tx=0x7f577c01c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7d002d8939865d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:09.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.221+0000 7f578159a640 1 -- 172.21.15.148:0/1840705322 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c0a57d0 con 0x7f57840b2d20 2023-11-28T18:23:09.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.225+0000 7f578159a640 1 -- 172.21.15.148:0/1840705322 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577c0a5930 con 0x7f57840b2d20 2023-11-28T18:23:09.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.225+0000 7f578159a640 1 -- 172.21.15.148:0/1840705322 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c0a5cc0 con 0x7f57840b2d20 2023-11-28T18:23:09.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.225+0000 7f5788a17640 1 -- 172.21.15.148:0/1840705322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57840b2d20 msgr2=0x7f57840b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:09.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.225+0000 7f5788a17640 1 --2- 172.21.15.148:0/1840705322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57840b2d20 0x7f57840b3150 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5784160fa0 tx=0x7f577c01c910 comp rx=0 tx=0).stop 2023-11-28T18:23:09.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.225+0000 7f5788a17640 1 -- 172.21.15.148:0/1840705322 shutdown_connections 2023-11-28T18:23:09.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.225+0000 7f5788a17640 1 --2- 172.21.15.148:0/1840705322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57840b2d20 0x7f57840b3150 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.225+0000 7f5788a17640 1 --2- 172.21.15.148:0/1840705322 >> v2:172.21.15.105:3300/0 conn(0x7f578415d4f0 0x7f57840d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.225+0000 7f5788a17640 1 -- 172.21.15.148:0/1840705322 >> 172.21.15.148:0/1840705322 conn(0x7f57840d3030 msgr2=0x7f57840d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:09.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.225+0000 7f5788a17640 1 -- 172.21.15.148:0/1840705322 shutdown_connections 2023-11-28T18:23:09.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.225+0000 7f5788a17640 1 -- 172.21.15.148:0/1840705322 wait complete. 2023-11-28T18:23:09.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.225+0000 7f5788a17640 1 Processor -- start 2023-11-28T18:23:09.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.225+0000 7f5788a17640 1 -- start start 2023-11-28T18:23:09.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.225+0000 7f5788a17640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57840b2d20 0x7f578414b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:09.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.225+0000 7f5788a17640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f57840d3c30 0x7f5784149f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:09.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.225+0000 7f5788a17640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f578415d4f0 0x7f578414a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:09.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.225+0000 7f5788a17640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57840c8bb0 con 0x7f57840b2d20 2023-11-28T18:23:09.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.225+0000 7f5788a17640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578414a9b0 con 0x7f578415d4f0 2023-11-28T18:23:09.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.225+0000 7f5788a17640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57840c90a0 con 0x7f57840d3c30 2023-11-28T18:23:09.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.225+0000 7f5782d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f578415d4f0 0x7f578414a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:09.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.229+0000 7f5782d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f578415d4f0 0x7f578414a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53942/0 (socket says 172.21.15.148:53942) 2023-11-28T18:23:09.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.229+0000 7f5782d9d640 1 -- 172.21.15.148:0/2352513553 learned_addr learned my addr 172.21.15.148:0/2352513553 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-11-28T18:23:09.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.229+0000 7f578259c640 1 --2- 172.21.15.148:0/2352513553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57840b2d20 0x7f578414b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:09.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.229+0000 7f5781d9b640 1 --2- 172.21.15.148:0/2352513553 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f57840d3c30 0x7f5784149f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:09.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.229+0000 7f5782d9d640 1 -- 172.21.15.148:0/2352513553 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f57840d3c30 msgr2=0x7f5784149f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:09.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.229+0000 7f5782d9d640 1 --2- 172.21.15.148:0/2352513553 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f57840d3c30 0x7f5784149f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.229+0000 7f5782d9d640 1 -- 172.21.15.148:0/2352513553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57840b2d20 msgr2=0x7f578414b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:09.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.229+0000 7f5782d9d640 1 --2- 172.21.15.148:0/2352513553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57840b2d20 0x7f578414b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.229+0000 7f5782d9d640 1 -- 172.21.15.148:0/2352513553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c009b30 con 0x7f578415d4f0 2023-11-28T18:23:09.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.229+0000 7f5782d9d640 1 --2- 172.21.15.148:0/2352513553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f578415d4f0 0x7f578414a470 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f577c0af040 tx=0x7f577c0196b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:09.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.229+0000 7f5772ffd640 1 -- 172.21.15.148:0/2352513553 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c0b8670 con 0x7f578415d4f0 2023-11-28T18:23:09.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.229+0000 7f5772ffd640 1 -- 172.21.15.148:0/2352513553 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577c0b87d0 con 0x7f578415d4f0 2023-11-28T18:23:09.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.229+0000 7f5772ffd640 1 -- 172.21.15.148:0/2352513553 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c0b8b60 con 0x7f578415d4f0 2023-11-28T18:23:09.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.229+0000 7f5788a17640 1 -- 172.21.15.148:0/2352513553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57840c8dc0 con 0x7f578415d4f0 2023-11-28T18:23:09.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.229+0000 7f5788a17640 1 -- 172.21.15.148:0/2352513553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578415def0 con 0x7f578415d4f0 2023-11-28T18:23:09.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.229+0000 7f5772ffd640 1 -- 172.21.15.148:0/2352513553 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f577c0c1400 con 0x7f578415d4f0 2023-11-28T18:23:09.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.229+0000 7f5772ffd640 1 -- 172.21.15.148:0/2352513553 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f577c0b6890 con 0x7f578415d4f0 2023-11-28T18:23:09.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.229+0000 7f5788a17640 1 -- 172.21.15.148:0/2352513553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5784067570 con 0x7f578415d4f0 2023-11-28T18:23:09.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.233+0000 7f5772ffd640 1 -- 172.21.15.148:0/2352513553 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f577c0dfce0 con 0x7f578415d4f0 2023-11-28T18:23:09.263 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:09.261+0000 7fe468e51180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T18:23:09.295 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:09.293+0000 7ffa104f0180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T18:23:09.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.301+0000 7f5788a17640 1 -- 172.21.15.148:0/2352513553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2738deef-83f7-4af6-871c-3e947cdc6954", "id": 1} v 0) v1 -- 0x7f57840cf930 con 0x7f578415d4f0 2023-11-28T18:23:09.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.309+0000 7f5772ffd640 1 -- 172.21.15.148:0/2352513553 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2738deef-83f7-4af6-871c-3e947cdc6954", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f577c0ce030 con 0x7f578415d4f0 2023-11-28T18:23:09.311 INFO:teuthology.orchestra.run.smithi148.stdout:1 2023-11-28T18:23:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.309+0000 7f5788a17640 1 -- 172.21.15.148:0/2352513553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f578415d4f0 msgr2=0x7f578414a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.309+0000 7f5788a17640 1 --2- 172.21.15.148:0/2352513553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f578415d4f0 0x7f578414a470 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f577c0af040 tx=0x7f577c0196b0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.309+0000 7f5788a17640 1 -- 172.21.15.148:0/2352513553 shutdown_connections 2023-11-28T18:23:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.309+0000 7f5788a17640 1 --2- 172.21.15.148:0/2352513553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f578415d4f0 0x7f578414a470 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.309+0000 7f5788a17640 1 --2- 172.21.15.148:0/2352513553 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f57840d3c30 0x7f5784149f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.309+0000 7f5788a17640 1 --2- 172.21.15.148:0/2352513553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57840b2d20 0x7f578414b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.309+0000 7f5788a17640 1 -- 172.21.15.148:0/2352513553 >> 172.21.15.148:0/2352513553 conn(0x7f57840d3030 msgr2=0x7f57840d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:09.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.309+0000 7f5788a17640 1 -- 172.21.15.148:0/2352513553 shutdown_connections 2023-11-28T18:23:09.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.309+0000 7f5788a17640 1 -- 172.21.15.148:0/2352513553 wait complete. 2023-11-28T18:23:09.331 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 83d7788f-b5c7-407b-8879-b456e53016c4 2 2023-11-28T18:23:09.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.509+0000 7f9fa9054640 1 Processor -- start 2023-11-28T18:23:09.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.509+0000 7f9fa9054640 1 -- start start 2023-11-28T18:23:09.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.509+0000 7f9fa9054640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9fa415d4f0 0x7f9fa40d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:09.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.509+0000 7f9fa9054640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9fa40d3c30 0x7f9fa40d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:09.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.509+0000 7f9fa9054640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa40b25b0 con 0x7f9fa40d1890 2023-11-28T18:23:09.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.509+0000 7f9fa9054640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa40b2900 con 0x7f9fa415d4f0 2023-11-28T18:23:09.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.509+0000 7f9fa9054640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa40d1cc0 con 0x7f9fa40d3c30 2023-11-28T18:23:09.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.509+0000 7f9f9bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9fa40d3c30 0x7f9fa40d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:09.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.509+0000 7f9fa359e640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9fa40d1890 0x7f9fa40cfa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39404/0 (socket says 172.21.15.148:39404) 2023-11-28T18:23:09.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.509+0000 7f9f9bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9fa40d3c30 0x7f9fa40d1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53958/0 (socket says 172.21.15.148:53958) 2023-11-28T18:23:09.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.509+0000 7f9fa359e640 1 -- 172.21.15.148:0/2748765273 learned_addr learned my addr 172.21.15.148:0/2748765273 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-11-28T18:23:09.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.509+0000 7f9fa2d9d640 1 --2- 172.21.15.148:0/2748765273 >> v2:172.21.15.105:3300/0 conn(0x7f9fa415d4f0 0x7f9fa40d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:09.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.509+0000 7f9f9bfff640 1 -- 172.21.15.148:0/2748765273 >> v2:172.21.15.105:3300/0 conn(0x7f9fa415d4f0 msgr2=0x7f9fa40d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:09.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.509+0000 7f9f9bfff640 1 --2- 172.21.15.148:0/2748765273 >> v2:172.21.15.105:3300/0 conn(0x7f9fa415d4f0 0x7f9fa40d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.509+0000 7f9f9bfff640 1 -- 172.21.15.148:0/2748765273 >> v1:172.21.15.148:6789/0 conn(0x7f9fa40d1890 legacy=0x7f9fa40cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:09.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.509+0000 7f9f9bfff640 1 -- 172.21.15.148:0/2748765273 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa40cea30 con 0x7f9fa40d3c30 2023-11-28T18:23:09.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.509+0000 7f9fa2d9d640 1 --2- 172.21.15.148:0/2748765273 >> v2:172.21.15.105:3300/0 conn(0x7f9fa415d4f0 0x7f9fa40d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:09.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.513+0000 7f9f9bfff640 1 --2- 172.21.15.148:0/2748765273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fa40d3c30 0x7f9fa40d1350 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9fa408c430 tx=0x7f9f8c01c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b324537bdf3b5aca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:09.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.513+0000 7f9fa259c640 1 -- 172.21.15.148:0/2748765273 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0ac600 con 0x7f9fa40d3c30 2023-11-28T18:23:09.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.513+0000 7f9fa259c640 1 -- 172.21.15.148:0/2748765273 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8c0a8050 con 0x7f9fa40d3c30 2023-11-28T18:23:09.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.513+0000 7f9fa259c640 1 -- 172.21.15.148:0/2748765273 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0ac980 con 0x7f9fa40d3c30 2023-11-28T18:23:09.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.513+0000 7f9fa9054640 1 -- 172.21.15.148:0/2748765273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fa40d3c30 msgr2=0x7f9fa40d1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:09.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.513+0000 7f9fa9054640 1 --2- 172.21.15.148:0/2748765273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fa40d3c30 0x7f9fa40d1350 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9fa408c430 tx=0x7f9f8c01c3a0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.513+0000 7f9fa9054640 1 -- 172.21.15.148:0/2748765273 shutdown_connections 2023-11-28T18:23:09.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.513+0000 7f9fa9054640 1 --2- 172.21.15.148:0/2748765273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fa40d3c30 0x7f9fa40d1350 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.513+0000 7f9fa9054640 1 --2- 172.21.15.148:0/2748765273 >> v2:172.21.15.105:3300/0 conn(0x7f9fa415d4f0 0x7f9fa40d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.513+0000 7f9fa9054640 1 -- 172.21.15.148:0/2748765273 >> 172.21.15.148:0/2748765273 conn(0x7f9fa40d3030 msgr2=0x7f9fa40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:09.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.513+0000 7f9fa9054640 1 -- 172.21.15.148:0/2748765273 shutdown_connections 2023-11-28T18:23:09.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.513+0000 7f9fa9054640 1 -- 172.21.15.148:0/2748765273 wait complete. 2023-11-28T18:23:09.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9fa9054640 1 Processor -- start 2023-11-28T18:23:09.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9fa9054640 1 -- start start 2023-11-28T18:23:09.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9fa9054640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fa40d1890 0x7f9fa415db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:09.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9fa9054640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fa40d3c30 0x7f9fa40c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:09.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9fa9054640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9fa415d4f0 0x7f9fa406b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:09.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9fa9054640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa40cafd0 con 0x7f9fa40d1890 2023-11-28T18:23:09.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9fa9054640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa415e080 con 0x7f9fa40d3c30 2023-11-28T18:23:09.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9fa9054640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa40ca100 con 0x7f9fa415d4f0 2023-11-28T18:23:09.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9f9bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fa40d3c30 0x7f9fa40c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:09.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9f9bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fa40d3c30 0x7f9fa40c8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53960/0 (socket says 172.21.15.148:53960) 2023-11-28T18:23:09.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9f9bfff640 1 -- 172.21.15.148:0/499231592 learned_addr learned my addr 172.21.15.148:0/499231592 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-11-28T18:23:09.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9fa2d9d640 1 --2- 172.21.15.148:0/499231592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fa40d1890 0x7f9fa415db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:09.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9fa359e640 1 --2- 172.21.15.148:0/499231592 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9fa415d4f0 0x7f9fa406b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:09.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9f9bfff640 1 -- 172.21.15.148:0/499231592 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9fa415d4f0 msgr2=0x7f9fa406b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:09.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9f9bfff640 1 --2- 172.21.15.148:0/499231592 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9fa415d4f0 0x7f9fa406b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9f9bfff640 1 -- 172.21.15.148:0/499231592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fa40d1890 msgr2=0x7f9fa415db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:09.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9f9bfff640 1 --2- 172.21.15.148:0/499231592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fa40d1890 0x7f9fa415db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9f9bfff640 1 -- 172.21.15.148:0/499231592 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c009420 con 0x7f9fa40d3c30 2023-11-28T18:23:09.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9f9bfff640 1 --2- 172.21.15.148:0/499231592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fa40d3c30 0x7f9fa40c8ba0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c019fd0 tx=0x7f9f8c01abe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:09.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9f9b7fe640 1 -- 172.21.15.148:0/499231592 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0b5830 con 0x7f9fa40d3c30 2023-11-28T18:23:09.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9f9b7fe640 1 -- 172.21.15.148:0/499231592 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8c0b5990 con 0x7f9fa40d3c30 2023-11-28T18:23:09.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9f9b7fe640 1 -- 172.21.15.148:0/499231592 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0b5d00 con 0x7f9fa40d3c30 2023-11-28T18:23:09.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9fa9054640 1 -- 172.21.15.148:0/499231592 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa40ca350 con 0x7f9fa40d3c30 2023-11-28T18:23:09.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9fa9054640 1 -- 172.21.15.148:0/499231592 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa40ca910 con 0x7f9fa40d3c30 2023-11-28T18:23:09.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9f9b7fe640 1 -- 172.21.15.148:0/499231592 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9f8c0b3070 con 0x7f9fa40d3c30 2023-11-28T18:23:09.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.517+0000 7f9f9b7fe640 1 -- 172.21.15.148:0/499231592 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f9f8c0b7b90 con 0x7f9fa40d3c30 2023-11-28T18:23:09.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.521+0000 7f9fa9054640 1 -- 172.21.15.148:0/499231592 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa40cb110 con 0x7f9fa40d3c30 2023-11-28T18:23:09.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.521+0000 7f9f9b7fe640 1 -- 172.21.15.148:0/499231592 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f9fa40cb110 con 0x7f9fa40d3c30 2023-11-28T18:23:09.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.585+0000 7f9fa9054640 1 -- 172.21.15.148:0/499231592 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd new", "uuid": "83d7788f-b5c7-407b-8879-b456e53016c4", "id": 2} v 0) v1 -- 0x7f9fa4067570 con 0x7f9fa40d3c30 2023-11-28T18:23:09.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.597+0000 7f9f9b7fe640 1 -- 172.21.15.148:0/499231592 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "83d7788f-b5c7-407b-8879-b456e53016c4", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9f8c0b1070 con 0x7f9fa40d3c30 2023-11-28T18:23:09.597 INFO:teuthology.orchestra.run.smithi148.stdout:2 2023-11-28T18:23:09.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.597+0000 7f9fa9054640 1 -- 172.21.15.148:0/499231592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fa40d3c30 msgr2=0x7f9fa40c8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:09.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.597+0000 7f9fa9054640 1 --2- 172.21.15.148:0/499231592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fa40d3c30 0x7f9fa40c8ba0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c019fd0 tx=0x7f9f8c01abe0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.597+0000 7f9fa9054640 1 -- 172.21.15.148:0/499231592 shutdown_connections 2023-11-28T18:23:09.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.597+0000 7f9fa9054640 1 --2- 172.21.15.148:0/499231592 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9fa415d4f0 0x7f9fa406b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.597+0000 7f9fa9054640 1 --2- 172.21.15.148:0/499231592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fa40d3c30 0x7f9fa40c8ba0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.597+0000 7f9fa9054640 1 --2- 172.21.15.148:0/499231592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fa40d1890 0x7f9fa415db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.597+0000 7f9fa9054640 1 -- 172.21.15.148:0/499231592 >> 172.21.15.148:0/499231592 conn(0x7f9fa40d3030 msgr2=0x7f9fa406c830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:09.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.597+0000 7f9fa9054640 1 -- 172.21.15.148:0/499231592 shutdown_connections 2023-11-28T18:23:09.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.597+0000 7f9fa9054640 1 -- 172.21.15.148:0/499231592 wait complete. 2023-11-28T18:23:09.618 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 2996213e-81df-4eec-94d2-6c7dd7f4f556 3 2023-11-28T18:23:09.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.753+0000 7efc2ddd8640 1 Processor -- start 2023-11-28T18:23:09.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.753+0000 7efc2ddd8640 1 -- start start 2023-11-28T18:23:09.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.753+0000 7efc2ddd8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc2815d4f0 0x7efc280d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:09.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.753+0000 7efc2ddd8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efc280d3c30 0x7efc280d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:09.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.753+0000 7efc2ddd8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc280b25b0 con 0x7efc280d1890 2023-11-28T18:23:09.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.753+0000 7efc2ddd8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc280b2900 con 0x7efc280d3c30 2023-11-28T18:23:09.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.753+0000 7efc2ddd8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc280cea30 con 0x7efc2815d4f0 2023-11-28T18:23:09.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.753+0000 7efc277fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc2815d4f0 0x7efc280d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:09.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.753+0000 7efc27fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efc280d1890 0x7efc280cfa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39416/0 (socket says 172.21.15.148:39416) 2023-11-28T18:23:09.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.753+0000 7efc27fff640 1 -- 172.21.15.148:0/2479854134 learned_addr learned my addr 172.21.15.148:0/2479854134 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-11-28T18:23:09.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.753+0000 7efc26ffd640 1 --2- 172.21.15.148:0/2479854134 >> v2:172.21.15.105:3300/0 conn(0x7efc280d3c30 0x7efc280d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:09.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.753+0000 7efc277fe640 1 -- 172.21.15.148:0/2479854134 >> v2:172.21.15.105:3300/0 conn(0x7efc280d3c30 msgr2=0x7efc280d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:09.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.753+0000 7efc277fe640 1 --2- 172.21.15.148:0/2479854134 >> v2:172.21.15.105:3300/0 conn(0x7efc280d3c30 0x7efc280d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.753+0000 7efc277fe640 1 -- 172.21.15.148:0/2479854134 >> v1:172.21.15.148:6789/0 conn(0x7efc280d1890 legacy=0x7efc280cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:09.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.753+0000 7efc267fc640 1 -- 172.21.15.148:0/2479854134 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1064496701 0 0) 0x7efc280b25b0 con 0x7efc280d1890 2023-11-28T18:23:09.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.753+0000 7efc277fe640 1 -- 172.21.15.148:0/2479854134 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc280ced10 con 0x7efc2815d4f0 2023-11-28T18:23:09.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.757+0000 7efc277fe640 1 --2- 172.21.15.148:0/2479854134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc2815d4f0 0x7efc280d0e10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efc180097c0 tx=0x7efc1801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7600a9bb730dc3fc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:09.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.757+0000 7efc267fc640 1 -- 172.21.15.148:0/2479854134 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc180ac640 con 0x7efc2815d4f0 2023-11-28T18:23:09.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.757+0000 7efc267fc640 1 -- 172.21.15.148:0/2479854134 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc180ac7a0 con 0x7efc2815d4f0 2023-11-28T18:23:09.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.757+0000 7efc267fc640 1 -- 172.21.15.148:0/2479854134 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc180acb30 con 0x7efc2815d4f0 2023-11-28T18:23:09.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.757+0000 7efc2ddd8640 1 -- 172.21.15.148:0/2479854134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc2815d4f0 msgr2=0x7efc280d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:09.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.757+0000 7efc2ddd8640 1 --2- 172.21.15.148:0/2479854134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc2815d4f0 0x7efc280d0e10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efc180097c0 tx=0x7efc1801c3c0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.757+0000 7efc2ddd8640 1 -- 172.21.15.148:0/2479854134 shutdown_connections 2023-11-28T18:23:09.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.757+0000 7efc2ddd8640 1 --2- 172.21.15.148:0/2479854134 >> v2:172.21.15.105:3300/0 conn(0x7efc280d3c30 0x7efc280d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.757+0000 7efc2ddd8640 1 --2- 172.21.15.148:0/2479854134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc2815d4f0 0x7efc280d0e10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.757+0000 7efc2ddd8640 1 -- 172.21.15.148:0/2479854134 >> 172.21.15.148:0/2479854134 conn(0x7efc280d3030 msgr2=0x7efc280d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:09.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.757+0000 7efc2ddd8640 1 -- 172.21.15.148:0/2479854134 shutdown_connections 2023-11-28T18:23:09.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.757+0000 7efc2ddd8640 1 -- 172.21.15.148:0/2479854134 wait complete. 2023-11-28T18:23:09.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc2ddd8640 1 Processor -- start 2023-11-28T18:23:09.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc2ddd8640 1 -- start start 2023-11-28T18:23:09.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc2ddd8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc280d1890 0x7efc280cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:09.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc2ddd8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc280d3c30 0x7efc281f9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:09.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc2ddd8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc2815d4f0 0x7efc281f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:09.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc2ddd8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc280b88e0 con 0x7efc280d3c30 2023-11-28T18:23:09.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc2ddd8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc280b7fb0 con 0x7efc280d1890 2023-11-28T18:23:09.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc2ddd8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc280b8290 con 0x7efc2815d4f0 2023-11-28T18:23:09.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc277fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc280d1890 0x7efc280cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:09.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc27fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc2815d4f0 0x7efc281f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:09.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc27fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc2815d4f0 0x7efc281f96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.148:35440/0 (socket says 172.21.15.148:35440) 2023-11-28T18:23:09.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc277fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc280d1890 0x7efc280cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53974/0 (socket says 172.21.15.148:53974) 2023-11-28T18:23:09.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc27fff640 1 -- 172.21.15.148:0/194976116 learned_addr learned my addr 172.21.15.148:0/194976116 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-11-28T18:23:09.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc26ffd640 1 --2- 172.21.15.148:0/194976116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc280d3c30 0x7efc281f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:09.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc277fe640 1 -- 172.21.15.148:0/194976116 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc2815d4f0 msgr2=0x7efc281f96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:09.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc277fe640 1 --2- 172.21.15.148:0/194976116 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc2815d4f0 0x7efc281f96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc277fe640 1 -- 172.21.15.148:0/194976116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc280d3c30 msgr2=0x7efc281f9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:09.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc277fe640 1 --2- 172.21.15.148:0/194976116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc280d3c30 0x7efc281f9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc277fe640 1 -- 172.21.15.148:0/194976116 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc18009420 con 0x7efc280d1890 2023-11-28T18:23:09.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc27fff640 1 --2- 172.21.15.148:0/194976116 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc2815d4f0 0x7efc281f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:09.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc277fe640 1 --2- 172.21.15.148:0/194976116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc280d1890 0x7efc280cc6e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efc180ac000 tx=0x7efc18019160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:09.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc0ffff640 1 -- 172.21.15.148:0/194976116 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc180b6670 con 0x7efc280d1890 2023-11-28T18:23:09.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc0ffff640 1 -- 172.21.15.148:0/194976116 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc180b6810 con 0x7efc280d1890 2023-11-28T18:23:09.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc2ddd8640 1 -- 172.21.15.148:0/194976116 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc280b8570 con 0x7efc280d1890 2023-11-28T18:23:09.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc0ffff640 1 -- 172.21.15.148:0/194976116 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc180b6bc0 con 0x7efc280d1890 2023-11-28T18:23:09.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc2ddd8640 1 -- 172.21.15.148:0/194976116 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc28146550 con 0x7efc280d1890 2023-11-28T18:23:09.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc0ffff640 1 -- 172.21.15.148:0/194976116 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7efc180b3040 con 0x7efc280d1890 2023-11-28T18:23:09.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc0ffff640 1 -- 172.21.15.148:0/194976116 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7efc180b6d20 con 0x7efc280d1890 2023-11-28T18:23:09.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.761+0000 7efc2ddd8640 1 -- 172.21.15.148:0/194976116 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc28067570 con 0x7efc280d1890 2023-11-28T18:23:09.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.765+0000 7efc0ffff640 1 -- 172.21.15.148:0/194976116 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7efc180b1070 con 0x7efc280d1890 2023-11-28T18:23:09.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.833+0000 7efc2ddd8640 1 -- 172.21.15.148:0/194976116 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2996213e-81df-4eec-94d2-6c7dd7f4f556", "id": 3} v 0) v1 -- 0x7efc280b2500 con 0x7efc280d1890 2023-11-28T18:23:09.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.841+0000 7efc0ffff640 1 -- 172.21.15.148:0/194976116 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2996213e-81df-4eec-94d2-6c7dd7f4f556", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7efc180c4920 con 0x7efc280d1890 2023-11-28T18:23:09.842 INFO:teuthology.orchestra.run.smithi148.stdout:3 2023-11-28T18:23:09.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.841+0000 7efc2ddd8640 1 -- 172.21.15.148:0/194976116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc280d1890 msgr2=0x7efc280cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:09.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.841+0000 7efc2ddd8640 1 --2- 172.21.15.148:0/194976116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc280d1890 0x7efc280cc6e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efc180ac000 tx=0x7efc18019160 comp rx=0 tx=0).stop 2023-11-28T18:23:09.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.841+0000 7efc2ddd8640 1 -- 172.21.15.148:0/194976116 shutdown_connections 2023-11-28T18:23:09.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.841+0000 7efc2ddd8640 1 --2- 172.21.15.148:0/194976116 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc2815d4f0 0x7efc281f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.841+0000 7efc2ddd8640 1 --2- 172.21.15.148:0/194976116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc280d3c30 0x7efc281f9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.841+0000 7efc2ddd8640 1 --2- 172.21.15.148:0/194976116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc280d1890 0x7efc280cc6e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:09.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.841+0000 7efc2ddd8640 1 -- 172.21.15.148:0/194976116 >> 172.21.15.148:0/194976116 conn(0x7efc280d3030 msgr2=0x7efc2806bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:09.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.841+0000 7efc2ddd8640 1 -- 172.21.15.148:0/194976116 shutdown_connections 2023-11-28T18:23:09.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.841+0000 7efc2ddd8640 1 -- 172.21.15.148:0/194976116 wait complete. 2023-11-28T18:23:09.862 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 4aae5372-22fc-423c-8ed2-e8ab05881be5 4 2023-11-28T18:23:09.896 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:09.897+0000 7fe468e51180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T18:23:09.957 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:09.957+0000 7fe468e51180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T18:23:09.973 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:09.969+0000 7ffa104f0180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T18:23:09.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.997+0000 7f1ac1cfe640 1 Processor -- start 2023-11-28T18:23:09.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.997+0000 7f1ac1cfe640 1 -- start start 2023-11-28T18:23:10.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.997+0000 7f1ac1cfe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1abc15d4f0 0x7f1abc0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:10.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.997+0000 7f1ac1cfe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1abc0d3c30 0x7f1abc0d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:10.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.997+0000 7f1ac1cfe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1abc0d1890 0x7f1abc0cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:10.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.997+0000 7f1ac1cfe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1abc0b2900 con 0x7f1abc0d1890 2023-11-28T18:23:10.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.997+0000 7f1ac1cfe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1abc0b25b0 con 0x7f1abc0d3c30 2023-11-28T18:23:10.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.997+0000 7f1ac1cfe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc0cea30 con 0x7f1abc15d4f0 2023-11-28T18:23:10.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.997+0000 7f1abb7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1abc15d4f0 0x7f1abc0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:10.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.997+0000 7f1abb7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1abc15d4f0 0x7f1abc0d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.148:35450/0 (socket says 172.21.15.148:35450) 2023-11-28T18:23:10.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.997+0000 7f1abb7fe640 1 -- 172.21.15.148:0/1297617123 learned_addr learned my addr 172.21.15.148:0/1297617123 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-11-28T18:23:10.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.997+0000 7f1abaffd640 1 --2- 172.21.15.148:0/1297617123 >> v2:172.21.15.148:3300/0 conn(0x7f1abc0d3c30 0x7f1abc0d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:10.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.997+0000 7f1abbfff640 1 --2- 172.21.15.148:0/1297617123 >> v2:172.21.15.105:3300/0 conn(0x7f1abc0d1890 0x7f1abc0cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:10.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.997+0000 7f1abaffd640 1 -- 172.21.15.148:0/1297617123 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1abc15d4f0 msgr2=0x7f1abc0d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:10.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.997+0000 7f1abaffd640 1 --2- 172.21.15.148:0/1297617123 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1abc15d4f0 0x7f1abc0d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.997+0000 7f1abaffd640 1 -- 172.21.15.148:0/1297617123 >> v2:172.21.15.105:3300/0 conn(0x7f1abc0d1890 msgr2=0x7f1abc0cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:10.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.997+0000 7f1abaffd640 1 --2- 172.21.15.148:0/1297617123 >> v2:172.21.15.105:3300/0 conn(0x7f1abc0d1890 0x7f1abc0cfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.997+0000 7f1abaffd640 1 -- 172.21.15.148:0/1297617123 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1abc0ced10 con 0x7f1abc0d3c30 2023-11-28T18:23:10.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:09.997+0000 7f1abb7fe640 1 --2- 172.21.15.148:0/1297617123 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1abc15d4f0 0x7f1abc0d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:10.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.001+0000 7f1abaffd640 1 --2- 172.21.15.148:0/1297617123 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1abc0d3c30 0x7f1abc0d1350 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1abc08b2f0 tx=0x7f1ab001c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ca75aaecbc8d72a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:10.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.001+0000 7f1aba7fc640 1 -- 172.21.15.148:0/1297617123 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab00ab600 con 0x7f1abc0d3c30 2023-11-28T18:23:10.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.001+0000 7f1aba7fc640 1 -- 172.21.15.148:0/1297617123 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab00a7050 con 0x7f1abc0d3c30 2023-11-28T18:23:10.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.001+0000 7f1aba7fc640 1 -- 172.21.15.148:0/1297617123 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab00ab980 con 0x7f1abc0d3c30 2023-11-28T18:23:10.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.001+0000 7f1ac1cfe640 1 -- 172.21.15.148:0/1297617123 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1abc0d3c30 msgr2=0x7f1abc0d1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:10.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.001+0000 7f1ac1cfe640 1 --2- 172.21.15.148:0/1297617123 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1abc0d3c30 0x7f1abc0d1350 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1abc08b2f0 tx=0x7f1ab001c3a0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.001+0000 7f1ac1cfe640 1 -- 172.21.15.148:0/1297617123 shutdown_connections 2023-11-28T18:23:10.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.001+0000 7f1ac1cfe640 1 --2- 172.21.15.148:0/1297617123 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1abc0d3c30 0x7f1abc0d1350 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.001+0000 7f1ac1cfe640 1 --2- 172.21.15.148:0/1297617123 >> v2:172.21.15.105:3300/0 conn(0x7f1abc0d1890 0x7f1abc0cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.001+0000 7f1ac1cfe640 1 --2- 172.21.15.148:0/1297617123 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1abc15d4f0 0x7f1abc0d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.001+0000 7f1ac1cfe640 1 -- 172.21.15.148:0/1297617123 >> 172.21.15.148:0/1297617123 conn(0x7f1abc0d3030 msgr2=0x7f1abc0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:10.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.001+0000 7f1ac1cfe640 1 -- 172.21.15.148:0/1297617123 shutdown_connections 2023-11-28T18:23:10.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.001+0000 7f1ac1cfe640 1 -- 172.21.15.148:0/1297617123 wait complete. 2023-11-28T18:23:10.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.005+0000 7f1ac1cfe640 1 Processor -- start 2023-11-28T18:23:10.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.005+0000 7f1ac1cfe640 1 -- start start 2023-11-28T18:23:10.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.005+0000 7f1ac1cfe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1abc0d1890 0x7f1abc0cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:10.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.005+0000 7f1ac1cfe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1abc0d3c30 0x7f1abc0ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:10.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.005+0000 7f1ac1cfe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1abc15d4f0 0x7f1abc0b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:10.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.005+0000 7f1ac1cfe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc14b180 con 0x7f1abc0d1890 2023-11-28T18:23:10.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.005+0000 7f1ac1cfe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc15a790 con 0x7f1abc15d4f0 2023-11-28T18:23:10.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.005+0000 7f1ac1cfe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc15e8f0 con 0x7f1abc0d3c30 2023-11-28T18:23:10.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.005+0000 7f1abbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1abc15d4f0 0x7f1abc0b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:10.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.005+0000 7f1abaffd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1abc0d3c30 0x7f1abc0ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:10.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.005+0000 7f1abaffd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1abc0d3c30 0x7f1abc0ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.148:35460/0 (socket says 172.21.15.148:35460) 2023-11-28T18:23:10.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.005+0000 7f1abaffd640 1 -- 172.21.15.148:0/1094699477 learned_addr learned my addr 172.21.15.148:0/1094699477 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-11-28T18:23:10.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.005+0000 7f1abb7fe640 1 --2- 172.21.15.148:0/1094699477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1abc0d1890 0x7f1abc0cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:10.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.005+0000 7f1abaffd640 1 -- 172.21.15.148:0/1094699477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1abc15d4f0 msgr2=0x7f1abc0b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:10.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.005+0000 7f1abaffd640 1 --2- 172.21.15.148:0/1094699477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1abc15d4f0 0x7f1abc0b7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.005+0000 7f1abaffd640 1 -- 172.21.15.148:0/1094699477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1abc0d1890 msgr2=0x7f1abc0cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:10.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.005+0000 7f1abaffd640 1 --2- 172.21.15.148:0/1094699477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1abc0d1890 0x7f1abc0cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.005+0000 7f1abaffd640 1 -- 172.21.15.148:0/1094699477 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aac0094d0 con 0x7f1abc0d3c30 2023-11-28T18:23:10.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.005+0000 7f1abbfff640 1 --2- 172.21.15.148:0/1094699477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1abc15d4f0 0x7f1abc0b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:10.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.005+0000 7f1abaffd640 1 --2- 172.21.15.148:0/1094699477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1abc0d3c30 0x7f1abc0ccb30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00ae040 tx=0x7f1ab001abe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:10.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.009+0000 7f1aa3fff640 1 -- 172.21.15.148:0/1094699477 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab00abe70 con 0x7f1abc0d3c30 2023-11-28T18:23:10.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.009+0000 7f1aa3fff640 1 -- 172.21.15.148:0/1094699477 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab00b7450 con 0x7f1abc0d3c30 2023-11-28T18:23:10.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.009+0000 7f1aa3fff640 1 -- 172.21.15.148:0/1094699477 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab00b7820 con 0x7f1abc0d3c30 2023-11-28T18:23:10.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.009+0000 7f1ac1cfe640 1 -- 172.21.15.148:0/1094699477 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab0009420 con 0x7f1abc0d3c30 2023-11-28T18:23:10.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.009+0000 7f1ac1cfe640 1 -- 172.21.15.148:0/1094699477 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1abc15acc0 con 0x7f1abc0d3c30 2023-11-28T18:23:10.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.009+0000 7f1aa3fff640 1 -- 172.21.15.148:0/1094699477 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1ab00b7c80 con 0x7f1abc0d3c30 2023-11-28T18:23:10.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.009+0000 7f1aa3fff640 1 -- 172.21.15.148:0/1094699477 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f1ab00c0910 con 0x7f1abc0d3c30 2023-11-28T18:23:10.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.009+0000 7f1ac1cfe640 1 -- 172.21.15.148:0/1094699477 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1abc067570 con 0x7f1abc0d3c30 2023-11-28T18:23:10.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.013+0000 7f1aa3fff640 1 -- 172.21.15.148:0/1094699477 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f1ab00bb020 con 0x7f1abc0d3c30 2023-11-28T18:23:10.018 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:10.017+0000 7fe468e51180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T18:23:10.039 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:10.037+0000 7ffa104f0180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T18:23:10.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.077+0000 7f1ac1cfe640 1 -- 172.21.15.148:0/1094699477 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4aae5372-22fc-423c-8ed2-e8ab05881be5", "id": 4} v 0) v1 -- 0x7f1abc1567d0 con 0x7f1abc0d3c30 2023-11-28T18:23:10.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.085+0000 7f1aa3fff640 1 -- 172.21.15.148:0/1094699477 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4aae5372-22fc-423c-8ed2-e8ab05881be5", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1ab00b0070 con 0x7f1abc0d3c30 2023-11-28T18:23:10.086 INFO:teuthology.orchestra.run.smithi148.stdout:4 2023-11-28T18:23:10.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.085+0000 7f1ac1cfe640 1 -- 172.21.15.148:0/1094699477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1abc0d3c30 msgr2=0x7f1abc0ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:10.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.085+0000 7f1ac1cfe640 1 --2- 172.21.15.148:0/1094699477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1abc0d3c30 0x7f1abc0ccb30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00ae040 tx=0x7f1ab001abe0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.085+0000 7f1ac1cfe640 1 -- 172.21.15.148:0/1094699477 shutdown_connections 2023-11-28T18:23:10.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.085+0000 7f1ac1cfe640 1 --2- 172.21.15.148:0/1094699477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1abc15d4f0 0x7f1abc0b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.085+0000 7f1ac1cfe640 1 --2- 172.21.15.148:0/1094699477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1abc0d3c30 0x7f1abc0ccb30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.085+0000 7f1ac1cfe640 1 --2- 172.21.15.148:0/1094699477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1abc0d1890 0x7f1abc0cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.085+0000 7f1ac1cfe640 1 -- 172.21.15.148:0/1094699477 >> 172.21.15.148:0/1094699477 conn(0x7f1abc0d3030 msgr2=0x7f1abc06c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:10.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.085+0000 7f1ac1cfe640 1 -- 172.21.15.148:0/1094699477 shutdown_connections 2023-11-28T18:23:10.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.085+0000 7f1ac1cfe640 1 -- 172.21.15.148:0/1094699477 wait complete. 2023-11-28T18:23:10.102 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:10.101+0000 7ffa104f0180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T18:23:10.106 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 2b95412d-1a2a-48b3-a7be-25495361720f 5 2023-11-28T18:23:10.106 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:10.105+0000 7fe468e51180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T18:23:10.166 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:10.165+0000 7fe468e51180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T18:23:10.194 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:10.193+0000 7ffa104f0180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T18:23:10.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.241+0000 7fc06bafe640 1 Processor -- start 2023-11-28T18:23:10.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.241+0000 7fc06bafe640 1 -- start start 2023-11-28T18:23:10.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.241+0000 7fc06bafe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0640d3c30 0x7fc0640d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:10.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.241+0000 7fc06bafe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc0640b2d20 0x7fc0640b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:10.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.241+0000 7fc06bafe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0640b25b0 con 0x7fc06415d4f0 2023-11-28T18:23:10.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.241+0000 7fc06bafe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0640b2900 con 0x7fc0640b2d20 2023-11-28T18:23:10.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.241+0000 7fc06bafe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0640d0a20 con 0x7fc0640d3c30 2023-11-28T18:23:10.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.241+0000 7fc069099640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0640d3c30 0x7fc0640d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:10.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.241+0000 7fc06a09b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc0640b2d20 0x7fc0640b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:10.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.241+0000 7fc069099640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0640d3c30 0x7fc0640d1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.148:35466/0 (socket says 172.21.15.148:35466) 2023-11-28T18:23:10.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.241+0000 7fc06a09b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc0640b2d20 0x7fc0640b31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53986/0 (socket says 172.21.15.148:53986) 2023-11-28T18:23:10.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.241+0000 7fc069099640 1 -- 172.21.15.148:0/3071431357 learned_addr learned my addr 172.21.15.148:0/3071431357 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-11-28T18:23:10.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.245+0000 7fc069099640 1 -- 172.21.15.148:0/3071431357 >> v2:172.21.15.148:3300/0 conn(0x7fc0640b2d20 msgr2=0x7fc0640b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:10.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.245+0000 7fc069099640 1 --2- 172.21.15.148:0/3071431357 >> v2:172.21.15.148:3300/0 conn(0x7fc0640b2d20 0x7fc0640b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.245+0000 7fc069099640 1 -- 172.21.15.148:0/3071431357 >> v1:172.21.15.148:6789/0 conn(0x7fc06415d4f0 legacy=0x7fc0640d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:10.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.245+0000 7fc069099640 1 -- 172.21.15.148:0/3071431357 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0640cea30 con 0x7fc0640d3c30 2023-11-28T18:23:10.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.245+0000 7fc06a09b640 1 --2- 172.21.15.148:0/3071431357 >> v2:172.21.15.148:3300/0 conn(0x7fc0640b2d20 0x7fc0640b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:10.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.245+0000 7fc069099640 1 --2- 172.21.15.148:0/3071431357 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0640d3c30 0x7fc0640d1520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc054009870 tx=0x7fc05401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c129adab29ee0b0e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:10.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.245+0000 7fc068898640 1 -- 172.21.15.148:0/3071431357 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0540ab680 con 0x7fc0640d3c30 2023-11-28T18:23:10.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.245+0000 7fc068898640 1 -- 172.21.15.148:0/3071431357 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0540ab7e0 con 0x7fc0640d3c30 2023-11-28T18:23:10.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.245+0000 7fc068898640 1 -- 172.21.15.148:0/3071431357 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0540abb70 con 0x7fc0640d3c30 2023-11-28T18:23:10.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06bafe640 1 -- 172.21.15.148:0/3071431357 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0640d3c30 msgr2=0x7fc0640d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:10.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06bafe640 1 --2- 172.21.15.148:0/3071431357 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0640d3c30 0x7fc0640d1520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc054009870 tx=0x7fc05401c480 comp rx=0 tx=0).stop 2023-11-28T18:23:10.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06bafe640 1 -- 172.21.15.148:0/3071431357 shutdown_connections 2023-11-28T18:23:10.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06bafe640 1 --2- 172.21.15.148:0/3071431357 >> v2:172.21.15.148:3300/0 conn(0x7fc0640b2d20 0x7fc0640b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06bafe640 1 --2- 172.21.15.148:0/3071431357 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0640d3c30 0x7fc0640d1520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06bafe640 1 -- 172.21.15.148:0/3071431357 >> 172.21.15.148:0/3071431357 conn(0x7fc0640d3030 msgr2=0x7fc0640d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:10.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06bafe640 1 -- 172.21.15.148:0/3071431357 shutdown_connections 2023-11-28T18:23:10.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06bafe640 1 -- 172.21.15.148:0/3071431357 wait complete. 2023-11-28T18:23:10.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06bafe640 1 Processor -- start 2023-11-28T18:23:10.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06bafe640 1 -- start start 2023-11-28T18:23:10.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06bafe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0640b2d20 0x7fc0641538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:10.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06bafe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0640d3c30 0x7fc06415be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:10.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06bafe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc06415d4f0 0x7fc06414dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:10.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06bafe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0640caf10 con 0x7fc0640d3c30 2023-11-28T18:23:10.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06bafe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06414e5a0 con 0x7fc06415d4f0 2023-11-28T18:23:10.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06bafe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06414e880 con 0x7fc0640b2d20 2023-11-28T18:23:10.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06989a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0640b2d20 0x7fc0641538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:10.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06a09b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc06415d4f0 0x7fc06414dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:10.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06989a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0640b2d20 0x7fc0641538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.148:35480/0 (socket says 172.21.15.148:35480) 2023-11-28T18:23:10.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06989a640 1 -- 172.21.15.148:0/1097919815 learned_addr learned my addr 172.21.15.148:0/1097919815 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-11-28T18:23:10.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc069099640 1 --2- 172.21.15.148:0/1097919815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0640d3c30 0x7fc06415be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:10.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06989a640 1 -- 172.21.15.148:0/1097919815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc06415d4f0 msgr2=0x7fc06414dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:10.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06989a640 1 --2- 172.21.15.148:0/1097919815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc06415d4f0 0x7fc06414dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06989a640 1 -- 172.21.15.148:0/1097919815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0640d3c30 msgr2=0x7fc06415be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:10.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06989a640 1 --2- 172.21.15.148:0/1097919815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0640d3c30 0x7fc06415be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06989a640 1 -- 172.21.15.148:0/1097919815 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05c009a70 con 0x7fc0640b2d20 2023-11-28T18:23:10.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.249+0000 7fc06a09b640 1 --2- 172.21.15.148:0/1097919815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc06415d4f0 0x7fc06414dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:10.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.253+0000 7fc06989a640 1 --2- 172.21.15.148:0/1097919815 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0640b2d20 0x7fc0641538f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc06000c500 tx=0x7fc06000c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:10.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.253+0000 7fc05a7fc640 1 -- 172.21.15.148:0/1097919815 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0600052d0 con 0x7fc0640b2d20 2023-11-28T18:23:10.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.253+0000 7fc05a7fc640 1 -- 172.21.15.148:0/1097919815 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc060005430 con 0x7fc0640b2d20 2023-11-28T18:23:10.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.253+0000 7fc06bafe640 1 -- 172.21.15.148:0/1097919815 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0540094d0 con 0x7fc0640b2d20 2023-11-28T18:23:10.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.253+0000 7fc05a7fc640 1 -- 172.21.15.148:0/1097919815 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc060003840 con 0x7fc0640b2d20 2023-11-28T18:23:10.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.253+0000 7fc06bafe640 1 -- 172.21.15.148:0/1097919815 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0640ca670 con 0x7fc0640b2d20 2023-11-28T18:23:10.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.253+0000 7fc05a7fc640 1 -- 172.21.15.148:0/1097919815 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc060003ca0 con 0x7fc0640b2d20 2023-11-28T18:23:10.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.253+0000 7fc05a7fc640 1 -- 172.21.15.148:0/1097919815 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fc060011e00 con 0x7fc0640b2d20 2023-11-28T18:23:10.256 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:10.253+0000 7fe468e51180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T18:23:10.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.253+0000 7fc06bafe640 1 -- 172.21.15.148:0/1097919815 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06415c380 con 0x7fc0640b2d20 2023-11-28T18:23:10.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.257+0000 7fc05a7fc640 1 -- 172.21.15.148:0/1097919815 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fc060017070 con 0x7fc0640b2d20 2023-11-28T18:23:10.259 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:10.257+0000 7ffa104f0180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T18:23:10.314 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:10.313+0000 7fe468e51180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T18:23:10.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.321+0000 7fc06bafe640 1 -- 172.21.15.148:0/1097919815 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2b95412d-1a2a-48b3-a7be-25495361720f", "id": 5} v 0) v1 -- 0x7fc064067570 con 0x7fc0640b2d20 2023-11-28T18:23:10.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.329+0000 7fc05a7fc640 1 -- 172.21.15.148:0/1097919815 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2b95412d-1a2a-48b3-a7be-25495361720f", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc060012420 con 0x7fc0640b2d20 2023-11-28T18:23:10.332 INFO:teuthology.orchestra.run.smithi148.stdout:5 2023-11-28T18:23:10.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.329+0000 7fc06bafe640 1 -- 172.21.15.148:0/1097919815 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0640b2d20 msgr2=0x7fc0641538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:10.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.329+0000 7fc06bafe640 1 --2- 172.21.15.148:0/1097919815 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0640b2d20 0x7fc0641538f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc06000c500 tx=0x7fc06000c900 comp rx=0 tx=0).stop 2023-11-28T18:23:10.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.329+0000 7fc06bafe640 1 -- 172.21.15.148:0/1097919815 shutdown_connections 2023-11-28T18:23:10.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.329+0000 7fc06bafe640 1 --2- 172.21.15.148:0/1097919815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc06415d4f0 0x7fc06414dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.329+0000 7fc06bafe640 1 --2- 172.21.15.148:0/1097919815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0640d3c30 0x7fc06415be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.329+0000 7fc06bafe640 1 --2- 172.21.15.148:0/1097919815 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0640b2d20 0x7fc0641538f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.329+0000 7fc06bafe640 1 -- 172.21.15.148:0/1097919815 >> 172.21.15.148:0/1097919815 conn(0x7fc0640d3030 msgr2=0x7fc0640cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:10.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.333+0000 7fc06bafe640 1 -- 172.21.15.148:0/1097919815 shutdown_connections 2023-11-28T18:23:10.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.333+0000 7fc06bafe640 1 -- 172.21.15.148:0/1097919815 wait complete. 2023-11-28T18:23:10.359 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 83fe816e-9810-46fa-a430-8670f32c7940 6 2023-11-28T18:23:10.360 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:10.357+0000 7ffa104f0180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T18:23:10.424 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:10.421+0000 7ffa104f0180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T18:23:10.469 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:10.469+0000 7fe468e51180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T18:23:10.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.497+0000 7fce0a557640 1 Processor -- start 2023-11-28T18:23:10.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.497+0000 7fce0a557640 1 -- start start 2023-11-28T18:23:10.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.497+0000 7fce0a557640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fce040d10e0 0x7fce040d3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:10.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.497+0000 7fce0a557640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fce04160530 0x7fce04160950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:10.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.497+0000 7fce0a557640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce04160e90 con 0x7fce040b4140 2023-11-28T18:23:10.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.497+0000 7fce0a557640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce040d1d20 con 0x7fce04160530 2023-11-28T18:23:10.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.497+0000 7fce0a557640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce040d01d0 con 0x7fce040d10e0 2023-11-28T18:23:10.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.497+0000 7fce03fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fce040d10e0 0x7fce040d3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:10.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.497+0000 7fce03fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fce040d10e0 0x7fce040d3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53998/0 (socket says 172.21.15.148:53998) 2023-11-28T18:23:10.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.497+0000 7fce03fff640 1 -- 172.21.15.148:0/3829839438 learned_addr learned my addr 172.21.15.148:0/3829839438 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-11-28T18:23:10.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.497+0000 7fce08af4640 1 --2- 172.21.15.148:0/3829839438 >> v2:172.21.15.105:3300/0 conn(0x7fce04160530 0x7fce04160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:10.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.497+0000 7fce03fff640 1 -- 172.21.15.148:0/3829839438 >> v2:172.21.15.105:3300/0 conn(0x7fce04160530 msgr2=0x7fce04160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:10.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.497+0000 7fce03fff640 1 --2- 172.21.15.148:0/3829839438 >> v2:172.21.15.105:3300/0 conn(0x7fce04160530 0x7fce04160950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.497+0000 7fce03fff640 1 -- 172.21.15.148:0/3829839438 >> v1:172.21.15.105:6789/0 conn(0x7fce040b4140 legacy=0x7fce040d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:10.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.497+0000 7fce03fff640 1 -- 172.21.15.148:0/3829839438 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce040d04b0 con 0x7fce040d10e0 2023-11-28T18:23:10.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.497+0000 7fce08af4640 1 --2- 172.21.15.148:0/3829839438 >> v2:172.21.15.105:3300/0 conn(0x7fce04160530 0x7fce04160950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:10.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.501+0000 7fce03fff640 1 --2- 172.21.15.148:0/3829839438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce040d10e0 0x7fce040d3bc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fce0408d810 tx=0x7fcdf401c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35bb76a275e62ce3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:10.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.501+0000 7fce02ffd640 1 -- 172.21.15.148:0/3829839438 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf40ab620 con 0x7fce040d10e0 2023-11-28T18:23:10.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.501+0000 7fce02ffd640 1 -- 172.21.15.148:0/3829839438 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf40ab780 con 0x7fce040d10e0 2023-11-28T18:23:10.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.501+0000 7fce02ffd640 1 -- 172.21.15.148:0/3829839438 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf40abb10 con 0x7fce040d10e0 2023-11-28T18:23:10.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.501+0000 7fce0a557640 1 -- 172.21.15.148:0/3829839438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce040d10e0 msgr2=0x7fce040d3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:10.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.501+0000 7fce0a557640 1 --2- 172.21.15.148:0/3829839438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce040d10e0 0x7fce040d3bc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fce0408d810 tx=0x7fcdf401c440 comp rx=0 tx=0).stop 2023-11-28T18:23:10.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.501+0000 7fce0a557640 1 -- 172.21.15.148:0/3829839438 shutdown_connections 2023-11-28T18:23:10.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.501+0000 7fce0a557640 1 --2- 172.21.15.148:0/3829839438 >> v2:172.21.15.105:3300/0 conn(0x7fce04160530 0x7fce04160950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.501+0000 7fce0a557640 1 --2- 172.21.15.148:0/3829839438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce040d10e0 0x7fce040d3bc0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.501+0000 7fce0a557640 1 -- 172.21.15.148:0/3829839438 >> 172.21.15.148:0/3829839438 conn(0x7fce040d30a0 msgr2=0x7fce040d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:10.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.501+0000 7fce0a557640 1 -- 172.21.15.148:0/3829839438 shutdown_connections 2023-11-28T18:23:10.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.501+0000 7fce0a557640 1 -- 172.21.15.148:0/3829839438 wait complete. 2023-11-28T18:23:10.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce0a557640 1 Processor -- start 2023-11-28T18:23:10.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce0a557640 1 -- start start 2023-11-28T18:23:10.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce0a557640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce040b4140 0x7fce040cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:10.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce0a557640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce040d10e0 0x7fce04154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:10.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce0a557640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce04160530 0x7fce04154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:10.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce0a557640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce04155310 con 0x7fce04160530 2023-11-28T18:23:10.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce0a557640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce040cd040 con 0x7fce040b4140 2023-11-28T18:23:10.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce0a557640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0415e1a0 con 0x7fce040d10e0 2023-11-28T18:23:10.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce037fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce040d10e0 0x7fce04154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:10.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce03fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce040b4140 0x7fce040cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:10.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce03fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce040b4140 0x7fce040cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54000/0 (socket says 172.21.15.148:54000) 2023-11-28T18:23:10.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce03fff640 1 -- 172.21.15.148:0/367552892 learned_addr learned my addr 172.21.15.148:0/367552892 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-11-28T18:23:10.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce08af4640 1 --2- 172.21.15.148:0/367552892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce04160530 0x7fce04154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:10.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce03fff640 1 -- 172.21.15.148:0/367552892 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce040d10e0 msgr2=0x7fce04154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:10.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce03fff640 1 --2- 172.21.15.148:0/367552892 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce040d10e0 0x7fce04154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce03fff640 1 -- 172.21.15.148:0/367552892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce04160530 msgr2=0x7fce04154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:10.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce03fff640 1 --2- 172.21.15.148:0/367552892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce04160530 0x7fce04154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce03fff640 1 -- 172.21.15.148:0/367552892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc009a70 con 0x7fce040b4140 2023-11-28T18:23:10.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce03fff640 1 --2- 172.21.15.148:0/367552892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce040b4140 0x7fce040cdd90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4019050 tx=0x7fcdf401ac80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:10.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce00ff9640 1 -- 172.21.15.148:0/367552892 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf40abd00 con 0x7fce040b4140 2023-11-28T18:23:10.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce00ff9640 1 -- 172.21.15.148:0/367552892 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf40abe60 con 0x7fce040b4140 2023-11-28T18:23:10.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce0a557640 1 -- 172.21.15.148:0/367552892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf40094d0 con 0x7fce040b4140 2023-11-28T18:23:10.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce00ff9640 1 -- 172.21.15.148:0/367552892 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf40c4630 con 0x7fce040b4140 2023-11-28T18:23:10.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.505+0000 7fce0a557640 1 -- 172.21.15.148:0/367552892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0415db50 con 0x7fce040b4140 2023-11-28T18:23:10.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.509+0000 7fce00ff9640 1 -- 172.21.15.148:0/367552892 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcdf4005ca0 con 0x7fce040b4140 2023-11-28T18:23:10.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.509+0000 7fce00ff9640 1 -- 172.21.15.148:0/367552892 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fcdf40060c0 con 0x7fce040b4140 2023-11-28T18:23:10.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.509+0000 7fce0a557640 1 -- 172.21.15.148:0/367552892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce04068650 con 0x7fce040b4140 2023-11-28T18:23:10.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.509+0000 7fce00ff9640 1 -- 172.21.15.148:0/367552892 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fcdf40c4790 con 0x7fce040b4140 2023-11-28T18:23:10.538 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:10.537+0000 7fe468e51180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T18:23:10.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.577+0000 7fce0a557640 1 -- 172.21.15.148:0/367552892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd new", "uuid": "83fe816e-9810-46fa-a430-8670f32c7940", "id": 6} v 0) v1 -- 0x7fce0406c3e0 con 0x7fce040b4140 2023-11-28T18:23:10.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.581+0000 7fce00ff9640 1 -- 172.21.15.148:0/367552892 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "83fe816e-9810-46fa-a430-8670f32c7940", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcdf40c4930 con 0x7fce040b4140 2023-11-28T18:23:10.584 INFO:teuthology.orchestra.run.smithi148.stdout:6 2023-11-28T18:23:10.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.585+0000 7fce0a557640 1 -- 172.21.15.148:0/367552892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce040b4140 msgr2=0x7fce040cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:10.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.585+0000 7fce0a557640 1 --2- 172.21.15.148:0/367552892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce040b4140 0x7fce040cdd90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4019050 tx=0x7fcdf401ac80 comp rx=0 tx=0).stop 2023-11-28T18:23:10.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.585+0000 7fce0a557640 1 -- 172.21.15.148:0/367552892 shutdown_connections 2023-11-28T18:23:10.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.585+0000 7fce0a557640 1 --2- 172.21.15.148:0/367552892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce04160530 0x7fce04154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.585+0000 7fce0a557640 1 --2- 172.21.15.148:0/367552892 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce040d10e0 0x7fce04154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.585+0000 7fce0a557640 1 --2- 172.21.15.148:0/367552892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce040b4140 0x7fce040cdd90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.585+0000 7fce0a557640 1 -- 172.21.15.148:0/367552892 >> 172.21.15.148:0/367552892 conn(0x7fce040d30a0 msgr2=0x7fce0406c900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:10.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.585+0000 7fce0a557640 1 -- 172.21.15.148:0/367552892 shutdown_connections 2023-11-28T18:23:10.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.585+0000 7fce0a557640 1 -- 172.21.15.148:0/367552892 wait complete. 2023-11-28T18:23:10.586 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:10.585+0000 7ffa104f0180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T18:23:10.604 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 33b8b090-d66f-4149-983a-da294c435c85 7 2023-11-28T18:23:10.658 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:10.657+0000 7ffa104f0180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T18:23:10.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.741+0000 7f43f7f3e640 1 Processor -- start 2023-11-28T18:23:10.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.741+0000 7f43f7f3e640 1 -- start start 2023-11-28T18:23:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.741+0000 7f43f7f3e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43f00d10e0 0x7f43f00d3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.741+0000 7f43f7f3e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f43f00b4140 0x7f43f00d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.741+0000 7f43f7f3e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f00d1d20 con 0x7f43f0160530 2023-11-28T18:23:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.741+0000 7f43f7f3e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f00d01d0 con 0x7f43f00b4140 2023-11-28T18:23:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.741+0000 7f43f7f3e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f00d04b0 con 0x7f43f00d10e0 2023-11-28T18:23:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.741+0000 7f43f5cda640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43f00d10e0 0x7f43f00d3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.741+0000 7f43f5cda640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43f00d10e0 0x7f43f00d3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.148:35500/0 (socket says 172.21.15.148:35500) 2023-11-28T18:23:10.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.741+0000 7f43f5cda640 1 -- 172.21.15.148:0/3079502530 learned_addr learned my addr 172.21.15.148:0/3079502530 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-11-28T18:23:10.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.741+0000 7f43f54d9640 1 --2- 172.21.15.148:0/3079502530 >> v2:172.21.15.105:3300/0 conn(0x7f43f00b4140 0x7f43f00d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:10.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.741+0000 7f43f5cda640 1 -- 172.21.15.148:0/3079502530 >> v2:172.21.15.105:3300/0 conn(0x7f43f00b4140 msgr2=0x7f43f00d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:10.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.741+0000 7f43f5cda640 1 --2- 172.21.15.148:0/3079502530 >> v2:172.21.15.105:3300/0 conn(0x7f43f00b4140 0x7f43f00d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.741+0000 7f43f5cda640 1 -- 172.21.15.148:0/3079502530 >> v1:172.21.15.105:6789/0 conn(0x7f43f0160530 legacy=0x7f43f0160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:10.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.741+0000 7f43f5cda640 1 -- 172.21.15.148:0/3079502530 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f00d0790 con 0x7f43f00d10e0 2023-11-28T18:23:10.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.745+0000 7f43f5cda640 1 --2- 172.21.15.148:0/3079502530 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43f00d10e0 0x7f43f00d3bc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f43ec009870 tx=0x7f43ec01ced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbcf09065b7a50ed server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:10.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.745+0000 7f43f4cd8640 1 -- 172.21.15.148:0/3079502530 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec0aa430 con 0x7f43f00d10e0 2023-11-28T18:23:10.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.745+0000 7f43f4cd8640 1 -- 172.21.15.148:0/3079502530 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ec0aa590 con 0x7f43f00d10e0 2023-11-28T18:23:10.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.745+0000 7f43f4cd8640 1 -- 172.21.15.148:0/3079502530 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec0aa920 con 0x7f43f00d10e0 2023-11-28T18:23:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.745+0000 7f43f7f3e640 1 -- 172.21.15.148:0/3079502530 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43f00d10e0 msgr2=0x7f43f00d3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.745+0000 7f43f7f3e640 1 --2- 172.21.15.148:0/3079502530 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43f00d10e0 0x7f43f00d3bc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f43ec009870 tx=0x7f43ec01ced0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.745+0000 7f43f7f3e640 1 -- 172.21.15.148:0/3079502530 shutdown_connections 2023-11-28T18:23:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.745+0000 7f43f7f3e640 1 --2- 172.21.15.148:0/3079502530 >> v2:172.21.15.105:3300/0 conn(0x7f43f00b4140 0x7f43f00d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.745+0000 7f43f7f3e640 1 --2- 172.21.15.148:0/3079502530 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43f00d10e0 0x7f43f00d3bc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.745+0000 7f43f7f3e640 1 -- 172.21.15.148:0/3079502530 >> 172.21.15.148:0/3079502530 conn(0x7f43f00d30a0 msgr2=0x7f43f00d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.745+0000 7f43f7f3e640 1 -- 172.21.15.148:0/3079502530 shutdown_connections 2023-11-28T18:23:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.745+0000 7f43f7f3e640 1 -- 172.21.15.148:0/3079502530 wait complete. 2023-11-28T18:23:10.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43f7f3e640 1 Processor -- start 2023-11-28T18:23:10.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43f7f3e640 1 -- start start 2023-11-28T18:23:10.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43f7f3e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43f00b4140 0x7f43f014d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:10.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43f7f3e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43f00d10e0 0x7f43f015e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:10.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43f7f3e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43f0160530 0x7f43f00b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:10.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43f7f3e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f00b7280 con 0x7f43f0160530 2023-11-28T18:23:10.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43f7f3e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f015e3f0 con 0x7f43f00b4140 2023-11-28T18:23:10.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43f7f3e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f00b8b50 con 0x7f43f00d10e0 2023-11-28T18:23:10.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43f5cda640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43f00b4140 0x7f43f014d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:10.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43f54d9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43f00d10e0 0x7f43f015e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:10.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43f5cda640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43f00b4140 0x7f43f014d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54008/0 (socket says 172.21.15.148:54008) 2023-11-28T18:23:10.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43f5cda640 1 -- 172.21.15.148:0/2821377149 learned_addr learned my addr 172.21.15.148:0/2821377149 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-11-28T18:23:10.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43f64db640 1 --2- 172.21.15.148:0/2821377149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43f0160530 0x7f43f00b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:10.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43f5cda640 1 -- 172.21.15.148:0/2821377149 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43f00d10e0 msgr2=0x7f43f015e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:10.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43f5cda640 1 --2- 172.21.15.148:0/2821377149 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43f00d10e0 0x7f43f015e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43f5cda640 1 -- 172.21.15.148:0/2821377149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43f0160530 msgr2=0x7f43f00b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:10.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43f5cda640 1 --2- 172.21.15.148:0/2821377149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43f0160530 0x7f43f00b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43f5cda640 1 -- 172.21.15.148:0/2821377149 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ec0094d0 con 0x7f43f00b4140 2023-11-28T18:23:10.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43f54d9640 1 --2- 172.21.15.148:0/2821377149 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43f00d10e0 0x7f43f015e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:10.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43f5cda640 1 --2- 172.21.15.148:0/2821377149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43f00b4140 0x7f43f014d380 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f43ec0ac040 tx=0x7f43ec01acb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:10.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43e67fc640 1 -- 172.21.15.148:0/2821377149 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec0aa430 con 0x7f43f00b4140 2023-11-28T18:23:10.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43e67fc640 1 -- 172.21.15.148:0/2821377149 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ec0b3820 con 0x7f43f00b4140 2023-11-28T18:23:10.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43f7f3e640 1 -- 172.21.15.148:0/2821377149 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f00b6cc0 con 0x7f43f00b4140 2023-11-28T18:23:10.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43e67fc640 1 -- 172.21.15.148:0/2821377149 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec0b3bd0 con 0x7f43f00b4140 2023-11-28T18:23:10.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43f7f3e640 1 -- 172.21.15.148:0/2821377149 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f015e080 con 0x7f43f00b4140 2023-11-28T18:23:10.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43e67fc640 1 -- 172.21.15.148:0/2821377149 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f43ec0c6740 con 0x7f43f00b4140 2023-11-28T18:23:10.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.749+0000 7f43e67fc640 1 -- 172.21.15.148:0/2821377149 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f43ec0bdb40 con 0x7f43f00b4140 2023-11-28T18:23:10.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.753+0000 7f43f7f3e640 1 -- 172.21.15.148:0/2821377149 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f0068650 con 0x7f43f00b4140 2023-11-28T18:23:10.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.753+0000 7f43e67fc640 1 -- 172.21.15.148:0/2821377149 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f43ec0c6b30 con 0x7f43f00b4140 2023-11-28T18:23:10.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.821+0000 7f43f7f3e640 1 -- 172.21.15.148:0/2821377149 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd new", "uuid": "33b8b090-d66f-4149-983a-da294c435c85", "id": 7} v 0) v1 -- 0x7f43f00cdf90 con 0x7f43f00b4140 2023-11-28T18:23:10.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.825+0000 7f43e67fc640 1 -- 172.21.15.148:0/2821377149 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "33b8b090-d66f-4149-983a-da294c435c85", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f43ec0c6cd0 con 0x7f43f00b4140 2023-11-28T18:23:10.828 INFO:teuthology.orchestra.run.smithi148.stdout:7 2023-11-28T18:23:10.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.829+0000 7f43f7f3e640 1 -- 172.21.15.148:0/2821377149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43f00b4140 msgr2=0x7f43f014d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:10.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.829+0000 7f43f7f3e640 1 --2- 172.21.15.148:0/2821377149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43f00b4140 0x7f43f014d380 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f43ec0ac040 tx=0x7f43ec01acb0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.829+0000 7f43f7f3e640 1 -- 172.21.15.148:0/2821377149 shutdown_connections 2023-11-28T18:23:10.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.829+0000 7f43f7f3e640 1 --2- 172.21.15.148:0/2821377149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43f0160530 0x7f43f00b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.829+0000 7f43f7f3e640 1 --2- 172.21.15.148:0/2821377149 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43f00d10e0 0x7f43f015e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.829+0000 7f43f7f3e640 1 --2- 172.21.15.148:0/2821377149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43f00b4140 0x7f43f014d380 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:10.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.829+0000 7f43f7f3e640 1 -- 172.21.15.148:0/2821377149 >> 172.21.15.148:0/2821377149 conn(0x7f43f00d30a0 msgr2=0x7f43f00ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:10.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.829+0000 7f43f7f3e640 1 -- 172.21.15.148:0/2821377149 shutdown_connections 2023-11-28T18:23:10.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:23:10.829+0000 7f43f7f3e640 1 -- 172.21.15.148:0/2821377149 wait complete. 2023-11-28T18:23:10.847 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-28T18:23:10.847 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-28T18:23:10.860 INFO:tasks.ceph.osd.0:Started 2023-11-28T18:23:10.860 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-28T18:23:10.861 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-28T18:23:10.866 INFO:tasks.ceph.osd.1:Started 2023-11-28T18:23:10.866 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-28T18:23:10.866 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-28T18:23:10.876 INFO:tasks.ceph.osd.2:Started 2023-11-28T18:23:10.879 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-28T18:23:10.879 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-28T18:23:10.882 INFO:tasks.ceph.osd.3:Started 2023-11-28T18:23:10.882 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-28T18:23:10.883 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-28T18:23:10.886 INFO:tasks.ceph.osd.4:Started 2023-11-28T18:23:10.886 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-28T18:23:10.886 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-28T18:23:10.889 INFO:tasks.ceph.osd.5:Started 2023-11-28T18:23:10.889 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-28T18:23:10.890 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-28T18:23:10.893 INFO:tasks.ceph.osd.6:Started 2023-11-28T18:23:10.893 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-28T18:23:10.893 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-28T18:23:10.897 INFO:tasks.ceph.osd.7:Started 2023-11-28T18:23:10.897 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:23:11.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.077+0000 7f9dac646640 1 Processor -- start 2023-11-28T18:23:11.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.077+0000 7f9dac646640 1 -- start start 2023-11-28T18:23:11.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.081+0000 7f9dac646640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9da40ce2c0 0x7f9da406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:11.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.081+0000 7f9dac646640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9da406cd20 0x7f9da40abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:11.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.081+0000 7f9dac646640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da40cde30 con 0x7f9da4159610 2023-11-28T18:23:11.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.081+0000 7f9dac646640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da406d190 con 0x7f9da406cd20 2023-11-28T18:23:11.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.081+0000 7f9dac646640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da41599f0 con 0x7f9da40ce2c0 2023-11-28T18:23:11.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.081+0000 7f9daa3e2640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9da4159610 0x7f9da40cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33084/0 (socket says 172.21.15.105:33084) 2023-11-28T18:23:11.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.081+0000 7f9daa3e2640 1 -- 172.21.15.105:0/1547446912 learned_addr learned my addr 172.21.15.105:0/1547446912 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:11.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.081+0000 7f9daabe3640 1 --2- 172.21.15.105:0/1547446912 >> v2:172.21.15.105:3300/0 conn(0x7f9da406cd20 0x7f9da40abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:11.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.081+0000 7f9da9be1640 1 --2- 172.21.15.105:0/1547446912 >> v2:172.21.15.148:3300/0 conn(0x7f9da40ce2c0 0x7f9da406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:11.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.081+0000 7f9da93e0640 1 -- 172.21.15.105:0/1547446912 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4207835343 0 0) 0x7f9da40cde30 con 0x7f9da4159610 2023-11-28T18:23:11.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.081+0000 7f9da93e0640 1 -- 172.21.15.105:0/1547446912 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d90005740 con 0x7f9da4159610 2023-11-28T18:23:11.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.081+0000 7f9daabe3640 1 -- 172.21.15.105:0/1547446912 >> v2:172.21.15.148:3300/0 conn(0x7f9da40ce2c0 msgr2=0x7f9da406c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:11.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.081+0000 7f9daabe3640 1 --2- 172.21.15.105:0/1547446912 >> v2:172.21.15.148:3300/0 conn(0x7f9da40ce2c0 0x7f9da406c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:11.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.081+0000 7f9daabe3640 1 -- 172.21.15.105:0/1547446912 >> v1:172.21.15.105:6789/0 conn(0x7f9da4159610 legacy=0x7f9da40cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:11.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.081+0000 7f9daabe3640 1 -- 172.21.15.105:0/1547446912 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da406de20 con 0x7f9da406cd20 2023-11-28T18:23:11.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.085+0000 7f9daabe3640 1 --2- 172.21.15.105:0/1547446912 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da406cd20 0x7f9da40abd80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9da40cfc90 tx=0x7f9d9c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212de0c8914a38ca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:11.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.085+0000 7f9da93e0640 1 -- 172.21.15.105:0/1547446912 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c00a2d0 con 0x7f9da406cd20 2023-11-28T18:23:11.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.085+0000 7f9da93e0640 1 -- 172.21.15.105:0/1547446912 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9c0a5650 con 0x7f9da406cd20 2023-11-28T18:23:11.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.085+0000 7f9da93e0640 1 -- 172.21.15.105:0/1547446912 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c0a59e0 con 0x7f9da406cd20 2023-11-28T18:23:11.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.085+0000 7f9dac646640 1 -- 172.21.15.105:0/1547446912 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da406cd20 msgr2=0x7f9da40abd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:11.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.085+0000 7f9dac646640 1 --2- 172.21.15.105:0/1547446912 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da406cd20 0x7f9da40abd80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9da40cfc90 tx=0x7f9d9c01c900 comp rx=0 tx=0).stop 2023-11-28T18:23:11.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.085+0000 7f9dac646640 1 -- 172.21.15.105:0/1547446912 shutdown_connections 2023-11-28T18:23:11.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.085+0000 7f9dac646640 1 --2- 172.21.15.105:0/1547446912 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da406cd20 0x7f9da40abd80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:11.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.085+0000 7f9dac646640 1 --2- 172.21.15.105:0/1547446912 >> v2:172.21.15.148:3300/0 conn(0x7f9da40ce2c0 0x7f9da406c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:11.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.085+0000 7f9dac646640 1 -- 172.21.15.105:0/1547446912 >> 172.21.15.105:0/1547446912 conn(0x7f9da40ceb90 msgr2=0x7f9da40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:11.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.085+0000 7f9dac646640 1 -- 172.21.15.105:0/1547446912 shutdown_connections 2023-11-28T18:23:11.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.085+0000 7f9dac646640 1 -- 172.21.15.105:0/1547446912 wait complete. 2023-11-28T18:23:11.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.085+0000 7f9dac646640 1 Processor -- start 2023-11-28T18:23:11.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9dac646640 1 -- start start 2023-11-28T18:23:11.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9dac646640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da406cd20 0x7f9da414fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:11.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9dac646640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da40ce2c0 0x7f9da414e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:11.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9dac646640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9da4159610 0x7f9da40b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:11.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9dac646640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da4155b50 con 0x7f9da406cd20 2023-11-28T18:23:11.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9dac646640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da41502b0 con 0x7f9da40ce2c0 2023-11-28T18:23:11.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9dac646640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da4150590 con 0x7f9da4159610 2023-11-28T18:23:11.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9daa3e2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da406cd20 0x7f9da414fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:11.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9daa3e2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da406cd20 0x7f9da414fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46474/0 (socket says 172.21.15.105:46474) 2023-11-28T18:23:11.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9daa3e2640 1 -- 172.21.15.105:0/498714375 learned_addr learned my addr 172.21.15.105:0/498714375 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:11.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9da9be1640 1 --2- 172.21.15.105:0/498714375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da40ce2c0 0x7f9da414e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:11.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9daabe3640 1 --2- 172.21.15.105:0/498714375 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9da4159610 0x7f9da40b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:11.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9daa3e2640 1 -- 172.21.15.105:0/498714375 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9da4159610 msgr2=0x7f9da40b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:11.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9daa3e2640 1 --2- 172.21.15.105:0/498714375 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9da4159610 0x7f9da40b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:11.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9daa3e2640 1 -- 172.21.15.105:0/498714375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da40ce2c0 msgr2=0x7f9da414e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:11.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9daa3e2640 1 --2- 172.21.15.105:0/498714375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da40ce2c0 0x7f9da414e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:11.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9daa3e2640 1 -- 172.21.15.105:0/498714375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d9c00ab80 con 0x7f9da406cd20 2023-11-28T18:23:11.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9daa3e2640 1 --2- 172.21.15.105:0/498714375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da406cd20 0x7f9da414fa60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9da000c420 tx=0x7f9da000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:11.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9d9affd640 1 -- 172.21.15.105:0/498714375 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da0005cf0 con 0x7f9da406cd20 2023-11-28T18:23:11.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9d9affd640 1 -- 172.21.15.105:0/498714375 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da0005e50 con 0x7f9da406cd20 2023-11-28T18:23:11.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9dac646640 1 -- 172.21.15.105:0/498714375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da40ccff0 con 0x7f9da406cd20 2023-11-28T18:23:11.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9d9affd640 1 -- 172.21.15.105:0/498714375 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da00072d0 con 0x7f9da406cd20 2023-11-28T18:23:11.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9dac646640 1 -- 172.21.15.105:0/498714375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da4155df0 con 0x7f9da406cd20 2023-11-28T18:23:11.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9d9affd640 1 -- 172.21.15.105:0/498714375 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9da001c440 con 0x7f9da406cd20 2023-11-28T18:23:11.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9d9affd640 1 -- 172.21.15.105:0/498714375 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f9da0012ba0 con 0x7f9da406cd20 2023-11-28T18:23:11.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.089+0000 7f9dac646640 1 -- 172.21.15.105:0/498714375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da4067570 con 0x7f9da406cd20 2023-11-28T18:23:11.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.093+0000 7f9d9affd640 1 -- 172.21.15.105:0/498714375 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f9da0021060 con 0x7f9da406cd20 2023-11-28T18:23:11.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.149+0000 7f9dac646640 1 -- 172.21.15.105:0/498714375 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9da40cf950 con 0x7f9da406cd20 2023-11-28T18:23:11.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.149+0000 7f9d9affd640 1 -- 172.21.15.105:0/498714375 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f9da0017070 con 0x7f9da406cd20 2023-11-28T18:23:11.150 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:11.150 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":10,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:23:10.823816+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:23:11.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.149+0000 7f9dac646640 1 -- 172.21.15.105:0/498714375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da406cd20 msgr2=0x7f9da414fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:11.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.149+0000 7f9dac646640 1 --2- 172.21.15.105:0/498714375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da406cd20 0x7f9da414fa60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9da000c420 tx=0x7f9da000c8e0 comp rx=0 tx=0).stop 2023-11-28T18:23:11.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.149+0000 7f9dac646640 1 -- 172.21.15.105:0/498714375 shutdown_connections 2023-11-28T18:23:11.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.149+0000 7f9dac646640 1 --2- 172.21.15.105:0/498714375 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9da4159610 0x7f9da40b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:11.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.149+0000 7f9dac646640 1 --2- 172.21.15.105:0/498714375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da40ce2c0 0x7f9da414e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:11.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.149+0000 7f9dac646640 1 --2- 172.21.15.105:0/498714375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da406cd20 0x7f9da414fa60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:11.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.149+0000 7f9dac646640 1 -- 172.21.15.105:0/498714375 >> 172.21.15.105:0/498714375 conn(0x7f9da40ceb90 msgr2=0x7f9da415bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:11.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.149+0000 7f9dac646640 1 -- 172.21.15.105:0/498714375 shutdown_connections 2023-11-28T18:23:11.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:11.149+0000 7f9dac646640 1 -- 172.21.15.105:0/498714375 wait complete. 2023-11-28T18:23:11.169 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-28T18:23:11.169 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-28T18:23:11.193 INFO:tasks.ceph.mgr.y.smithi105.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-11-28T18:23:11.193 INFO:tasks.ceph.mgr.y.smithi105.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-11-28T18:23:11.194 INFO:tasks.ceph.mgr.y.smithi105.stderr: from numpy import show_config as show_numpy_config 2023-11-28T18:23:11.246 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:11.245+0000 7fe468e51180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T18:23:11.296 INFO:tasks.ceph.mgr.x.smithi148.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-11-28T18:23:11.296 INFO:tasks.ceph.mgr.x.smithi148.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-11-28T18:23:11.296 INFO:tasks.ceph.mgr.x.smithi148.stderr: from numpy import show_config as show_numpy_config 2023-11-28T18:23:11.335 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:11.333+0000 7ffa104f0180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T18:23:11.383 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:11.381+0000 7fe468e51180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T18:23:11.470 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:11.469+0000 7ffa104f0180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T18:23:11.529 INFO:tasks.ceph.osd.3.smithi105.stderr:2023-11-28T18:23:11.529+0000 7fb5b3cff640 -1 Falling back to public interface 2023-11-28T18:23:11.537 INFO:tasks.ceph.osd.1.smithi105.stderr:2023-11-28T18:23:11.537+0000 7f7d69784640 -1 Falling back to public interface 2023-11-28T18:23:11.538 INFO:tasks.ceph.osd.4.smithi148.stderr:2023-11-28T18:23:11.537+0000 7f1b38cba640 -1 Falling back to public interface 2023-11-28T18:23:11.549 INFO:tasks.ceph.osd.2.smithi105.stderr:2023-11-28T18:23:11.549+0000 7f74eee28640 -1 Falling back to public interface 2023-11-28T18:23:11.570 INFO:tasks.ceph.osd.7.smithi148.stderr:2023-11-28T18:23:11.569+0000 7fb84989e640 -1 Falling back to public interface 2023-11-28T18:23:11.570 INFO:tasks.ceph.osd.6.smithi148.stderr:2023-11-28T18:23:11.569+0000 7f8785d20640 -1 Falling back to public interface 2023-11-28T18:23:11.577 INFO:tasks.ceph.osd.0.smithi105.stderr:2023-11-28T18:23:11.577+0000 7fd867d6c640 -1 Falling back to public interface 2023-11-28T18:23:11.582 INFO:tasks.ceph.osd.5.smithi148.stderr:2023-11-28T18:23:11.581+0000 7f481b02f640 -1 Falling back to public interface 2023-11-28T18:23:11.613 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:11.613+0000 7fe468e51180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T18:23:11.657 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:11.657+0000 7ffa104f0180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T18:23:11.685 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:11.685+0000 7fe468e51180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T18:23:11.730 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:11.729+0000 7ffa104f0180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T18:23:11.823 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:11.821+0000 7fe468e51180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T18:23:11.868 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:11.865+0000 7ffa104f0180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T18:23:11.976 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T18:23:12.107 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.101+0000 7fa6cc18d640 1 Processor -- start 2023-11-28T18:23:12.107 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.101+0000 7fa6cc18d640 1 -- start start 2023-11-28T18:23:12.107 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.105+0000 7fa6cc18d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6c40b13d0 0x7fa6c40ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:12.107 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.105+0000 7fa6cc18d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c40d04b0 con 0x7fa6c415d400 2023-11-28T18:23:12.107 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.105+0000 7fa6cc18d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c40d3bb0 con 0x7fa6c40d1bd0 2023-11-28T18:23:12.107 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.105+0000 7fa6cc18d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40afc50 con 0x7fa6c40b13d0 2023-11-28T18:23:12.108 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.105+0000 7fa6c9f29640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa6c415d400 0x7fa6c40d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33092/0 (socket says 172.21.15.105:33092) 2023-11-28T18:23:12.108 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.105+0000 7fa6c9f29640 1 -- 172.21.15.105:0/1187312179 learned_addr learned my addr 172.21.15.105:0/1187312179 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:12.108 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.105+0000 7fa6ca72a640 1 --2- 172.21.15.105:0/1187312179 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6c40b13d0 0x7fa6c40ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:12.108 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.105+0000 7fa6c8f27640 1 -- 172.21.15.105:0/1187312179 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1812508721 0 0) 0x7fa6c40d04b0 con 0x7fa6c415d400 2023-11-28T18:23:12.108 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.105+0000 7fa6c8f27640 1 -- 172.21.15.105:0/1187312179 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6b0003530 con 0x7fa6c415d400 2023-11-28T18:23:12.108 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.105+0000 7fa6c8f27640 1 -- 172.21.15.105:0/1187312179 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2776635995 0 0) 0x7fa6c40d3bb0 con 0x7fa6c40d1bd0 2023-11-28T18:23:12.108 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.105+0000 7fa6c8f27640 1 -- 172.21.15.105:0/1187312179 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6c40d04b0 con 0x7fa6c40d1bd0 2023-11-28T18:23:12.108 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.105+0000 7fa6ca72a640 1 -- 172.21.15.105:0/1187312179 >> v1:172.21.15.148:6789/0 conn(0x7fa6c40d1bd0 legacy=0x7fa6c40cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:12.108 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.105+0000 7fa6ca72a640 1 -- 172.21.15.105:0/1187312179 >> v1:172.21.15.105:6789/0 conn(0x7fa6c415d400 legacy=0x7fa6c40d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:12.108 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.105+0000 7fa6ca72a640 1 -- 172.21.15.105:0/1187312179 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c40d3e00 con 0x7fa6c40b13d0 2023-11-28T18:23:12.109 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.109+0000 7fa6ca72a640 1 --2- 172.21.15.105:0/1187312179 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6c40b13d0 0x7fa6c40ce340 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009dc0 tx=0x7fa6bc01c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a11b6fc19c37d7a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:12.110 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.109+0000 7fa6c8f27640 1 -- 172.21.15.105:0/1187312179 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc0a58e0 con 0x7fa6c40b13d0 2023-11-28T18:23:12.110 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.109+0000 7fa6c8f27640 1 -- 172.21.15.105:0/1187312179 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c40cf090 con 0x7fa6c40b13d0 2023-11-28T18:23:12.110 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.109+0000 7fa6c8f27640 1 -- 172.21.15.105:0/1187312179 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc0a5c40 con 0x7fa6c40b13d0 2023-11-28T18:23:12.110 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.109+0000 7fa6cc18d640 1 -- 172.21.15.105:0/1187312179 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6c40b13d0 msgr2=0x7fa6c40ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:12.110 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.109+0000 7fa6cc18d640 1 --2- 172.21.15.105:0/1187312179 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6c40b13d0 0x7fa6c40ce340 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009dc0 tx=0x7fa6bc01c820 comp rx=0 tx=0).stop 2023-11-28T18:23:12.110 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.109+0000 7fa6cc18d640 1 -- 172.21.15.105:0/1187312179 shutdown_connections 2023-11-28T18:23:12.110 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.109+0000 7fa6cc18d640 1 --2- 172.21.15.105:0/1187312179 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6c40b13d0 0x7fa6c40ce340 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:12.110 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.109+0000 7fa6cc18d640 1 -- 172.21.15.105:0/1187312179 >> 172.21.15.105:0/1187312179 conn(0x7fa6c40d3090 msgr2=0x7fa6c40d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:12.110 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.109+0000 7fa6cc18d640 1 -- 172.21.15.105:0/1187312179 shutdown_connections 2023-11-28T18:23:12.110 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.109+0000 7fa6cc18d640 1 -- 172.21.15.105:0/1187312179 wait complete. 2023-11-28T18:23:12.111 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.109+0000 7fa6cc18d640 1 Processor -- start 2023-11-28T18:23:12.111 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.109+0000 7fa6cc18d640 1 -- start start 2023-11-28T18:23:12.113 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6cc18d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6c40b13d0 0x7fa6c4159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:12.113 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6cc18d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c40d1bd0 0x7fa6c4155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:12.113 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6cc18d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6c415d400 0x7fa6c40cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:12.114 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6cc18d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c4156180 con 0x7fa6c40b13d0 2023-11-28T18:23:12.114 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6cc18d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40cbb20 con 0x7fa6c40d1bd0 2023-11-28T18:23:12.114 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6cc18d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40cbe00 con 0x7fa6c415d400 2023-11-28T18:23:12.114 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6ca72a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6c415d400 0x7fa6c40cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:12.114 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6ca72a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6c415d400 0x7fa6c40cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:52670/0 (socket says 172.21.15.105:52670) 2023-11-28T18:23:12.114 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6ca72a640 1 -- 172.21.15.105:0/1726097200 learned_addr learned my addr 172.21.15.105:0/1726097200 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:12.114 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6c9f29640 1 --2- 172.21.15.105:0/1726097200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6c40b13d0 0x7fa6c4159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:12.114 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6c9728640 1 --2- 172.21.15.105:0/1726097200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c40d1bd0 0x7fa6c4155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:12.114 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6ca72a640 1 -- 172.21.15.105:0/1726097200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c40d1bd0 msgr2=0x7fa6c4155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:12.114 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6ca72a640 1 --2- 172.21.15.105:0/1726097200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c40d1bd0 0x7fa6c4155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:12.114 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6ca72a640 1 -- 172.21.15.105:0/1726097200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6c40b13d0 msgr2=0x7fa6c4159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:12.114 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6ca72a640 1 --2- 172.21.15.105:0/1726097200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6c40b13d0 0x7fa6c4159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:12.115 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6ca72a640 1 -- 172.21.15.105:0/1726097200 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc009a70 con 0x7fa6c415d400 2023-11-28T18:23:12.115 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6c9728640 1 --2- 172.21.15.105:0/1726097200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c40d1bd0 0x7fa6c4155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:12.115 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6ca72a640 1 --2- 172.21.15.105:0/1726097200 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6c415d400 0x7fa6c40cb1b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc01a450 tx=0x7fa6bc0195c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:12.116 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6ba7fc640 1 -- 172.21.15.105:0/1726097200 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc007000 con 0x7fa6c415d400 2023-11-28T18:23:12.116 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6ba7fc640 1 -- 172.21.15.105:0/1726097200 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc0ad640 con 0x7fa6c415d400 2023-11-28T18:23:12.116 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6ba7fc640 1 -- 172.21.15.105:0/1726097200 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc0ad9d0 con 0x7fa6c415d400 2023-11-28T18:23:12.116 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6cc18d640 1 -- 172.21.15.105:0/1726097200 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c40cc0e0 con 0x7fa6c415d400 2023-11-28T18:23:12.116 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6cc18d640 1 -- 172.21.15.105:0/1726097200 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c415a040 con 0x7fa6c415d400 2023-11-28T18:23:12.117 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6ba7fc640 1 -- 172.21.15.105:0/1726097200 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa6bc0b4070 con 0x7fa6c415d400 2023-11-28T18:23:12.117 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6ba7fc640 1 -- 172.21.15.105:0/1726097200 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fa6bc0ade30 con 0x7fa6c415d400 2023-11-28T18:23:12.117 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.113+0000 7fa6cc18d640 1 -- 172.21.15.105:0/1726097200 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c41562c0 con 0x7fa6c415d400 2023-11-28T18:23:12.117 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.117+0000 7fa6ba7fc640 1 -- 172.21.15.105:0/1726097200 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fa6c41562c0 con 0x7fa6c415d400 2023-11-28T18:23:12.174 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.173+0000 7fa6cc18d640 1 -- 172.21.15.105:0/1726097200 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6c4067570 con 0x7fa6c415d400 2023-11-28T18:23:12.174 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.173+0000 7fa6ba7fc640 1 -- 172.21.15.105:0/1726097200 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fa6bc0c5060 con 0x7fa6c415d400 2023-11-28T18:23:12.175 INFO:teuthology.misc.health.smithi105.stdout: 2023-11-28T18:23:12.175 INFO:teuthology.misc.health.smithi105.stdout:{"epoch":10,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:23:10.823816+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:23:12.176 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.173+0000 7fa6cc18d640 1 -- 172.21.15.105:0/1726097200 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6c415d400 msgr2=0x7fa6c40cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:12.176 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.173+0000 7fa6cc18d640 1 --2- 172.21.15.105:0/1726097200 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6c415d400 0x7fa6c40cb1b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc01a450 tx=0x7fa6bc0195c0 comp rx=0 tx=0).stop 2023-11-28T18:23:12.176 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.173+0000 7fa6cc18d640 1 -- 172.21.15.105:0/1726097200 shutdown_connections 2023-11-28T18:23:12.176 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.173+0000 7fa6cc18d640 1 --2- 172.21.15.105:0/1726097200 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6c415d400 0x7fa6c40cb1b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:12.176 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.173+0000 7fa6cc18d640 1 --2- 172.21.15.105:0/1726097200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c40d1bd0 0x7fa6c4155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:12.176 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.173+0000 7fa6cc18d640 1 --2- 172.21.15.105:0/1726097200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6c40b13d0 0x7fa6c4159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:12.176 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.173+0000 7fa6cc18d640 1 -- 172.21.15.105:0/1726097200 >> 172.21.15.105:0/1726097200 conn(0x7fa6c40d3090 msgr2=0x7fa6c415fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:12.176 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.173+0000 7fa6cc18d640 1 -- 172.21.15.105:0/1726097200 shutdown_connections 2023-11-28T18:23:12.176 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:12.173+0000 7fa6cc18d640 1 -- 172.21.15.105:0/1726097200 wait complete. 2023-11-28T18:23:12.193 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-28T18:23:12.498 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:12.497+0000 7fe468e51180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T18:23:12.575 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:12.573+0000 7ffa104f0180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T18:23:12.585 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:12.585+0000 7fe468e51180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T18:23:12.661 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:12.661+0000 7ffa104f0180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T18:23:12.774 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:12.773+0000 7fe468e51180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T18:23:12.853 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:12.849+0000 7ffa104f0180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T18:23:12.859 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:12.857+0000 7fe468e51180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T18:23:12.937 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:12.937+0000 7ffa104f0180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T18:23:13.186 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:13.185+0000 7fe468e51180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T18:23:13.254 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:13.253+0000 7fe468e51180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T18:23:13.268 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:13.265+0000 7ffa104f0180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T18:23:13.340 INFO:tasks.ceph.mgr.x.smithi148.stderr:2023-11-28T18:23:13.337+0000 7ffa104f0180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T18:23:14.275 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:14.273+0000 7fe40fa3d640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-28T18:23:14.275 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:23:14.273+0000 7fe40fa3d640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-28T18:23:15.306 INFO:tasks.ceph.osd.4.smithi148.stderr:2023-11-28T18:23:15.305+0000 7f1b38cba640 -1 osd.4 0 log_to_monitors true 2023-11-28T18:23:15.399 INFO:tasks.ceph.osd.3.smithi105.stderr:2023-11-28T18:23:15.397+0000 7fb5b3cff640 -1 osd.3 0 log_to_monitors true 2023-11-28T18:23:15.428 INFO:tasks.ceph.osd.6.smithi148.stderr:2023-11-28T18:23:15.425+0000 7f8785d20640 -1 osd.6 0 log_to_monitors true 2023-11-28T18:23:15.434 INFO:tasks.ceph.osd.5.smithi148.stderr:2023-11-28T18:23:15.433+0000 7f481b02f640 -1 osd.5 0 log_to_monitors true 2023-11-28T18:23:15.486 INFO:tasks.ceph.osd.0.smithi105.stderr:2023-11-28T18:23:15.485+0000 7fd867d6c640 -1 osd.0 0 log_to_monitors true 2023-11-28T18:23:15.487 INFO:tasks.ceph.osd.1.smithi105.stderr:2023-11-28T18:23:15.485+0000 7f7d69784640 -1 osd.1 0 log_to_monitors true 2023-11-28T18:23:15.523 INFO:tasks.ceph.osd.2.smithi105.stderr:2023-11-28T18:23:15.521+0000 7f74eee28640 -1 osd.2 0 log_to_monitors true 2023-11-28T18:23:15.549 INFO:tasks.ceph.osd.7.smithi148.stderr:2023-11-28T18:23:15.545+0000 7fb84989e640 -1 osd.7 0 log_to_monitors true 2023-11-28T18:23:19.000 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T18:23:19.212 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.208+0000 7f69a52f5640 1 Processor -- start 2023-11-28T18:23:19.212 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.208+0000 7f69a52f5640 1 -- start start 2023-11-28T18:23:19.213 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.212+0000 7f69a52f5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f69a00cf920 0x7f69a00ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:19.216 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.212+0000 7f69a52f5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69a00b3c90 con 0x7f69a00b4200 2023-11-28T18:23:19.216 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.212+0000 7f69a52f5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69a00d39b0 con 0x7f69a00cfd00 2023-11-28T18:23:19.216 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.212+0000 7f69a52f5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69a00d2c70 con 0x7f69a00cf920 2023-11-28T18:23:19.217 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.212+0000 7f699f7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f69a00b4200 0x7f69a00b2600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33108/0 (socket says 172.21.15.105:33108) 2023-11-28T18:23:19.217 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.212+0000 7f699f7fe640 1 -- 172.21.15.105:0/1058901458 learned_addr learned my addr 172.21.15.105:0/1058901458 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:19.217 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.212+0000 7f699effd640 1 -- 172.21.15.105:0/1058901458 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2926617000 0 0) 0x7f69a00b3c90 con 0x7f69a00b4200 2023-11-28T18:23:19.217 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.212+0000 7f699effd640 1 -- 172.21.15.105:0/1058901458 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6988003530 con 0x7f69a00b4200 2023-11-28T18:23:19.217 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.212+0000 7f699effd640 1 -- 172.21.15.105:0/1058901458 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1116501761 0 0) 0x7f6988003530 con 0x7f69a00b4200 2023-11-28T18:23:19.217 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.212+0000 7f699effd640 1 -- 172.21.15.105:0/1058901458 >> v2:172.21.15.105:3300/0 conn(0x7f69a00cf920 msgr2=0x7f69a00ce9b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:23:19.217 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.212+0000 7f699effd640 1 --2- 172.21.15.105:0/1058901458 >> v2:172.21.15.105:3300/0 conn(0x7f69a00cf920 0x7f69a00ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:19.217 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.212+0000 7f699effd640 1 -- 172.21.15.105:0/1058901458 >> v1:172.21.15.148:6789/0 conn(0x7f69a00cfd00 legacy=0x7f69a00b2d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:23:19.218 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.212+0000 7f699effd640 1 -- 172.21.15.105:0/1058901458 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a00d3c90 con 0x7f69a00b4200 2023-11-28T18:23:19.218 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.212+0000 7f699effd640 1 -- 172.21.15.105:0/1058901458 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f6994001680 con 0x7f69a00b4200 2023-11-28T18:23:19.218 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.212+0000 7f699effd640 1 -- 172.21.15.105:0/1058901458 >> v1:172.21.15.105:6789/0 conn(0x7f69a00b4200 legacy=0x7f69a00b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:19.218 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.212+0000 7f699effd640 1 --2- 172.21.15.105:0/1058901458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6988003980 0x7f6988003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:19.218 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.212+0000 7f699effd640 1 --2- 172.21.15.105:0/1058901458 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69880047d0 0x7f6988004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:19.218 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.212+0000 7f699effd640 1 --2- 172.21.15.105:0/1058901458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6988005390 0x7f698800b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:19.218 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.212+0000 7f699effd640 1 -- 172.21.15.105:0/1058901458 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a00d2c70 con 0x7f6988005390 2023-11-28T18:23:19.218 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.212+0000 7f699effd640 1 -- 172.21.15.105:0/1058901458 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698800c2d0 con 0x7f6988003980 2023-11-28T18:23:19.219 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.212+0000 7f699effd640 1 -- 172.21.15.105:0/1058901458 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f698800c490 con 0x7f69880047d0 2023-11-28T18:23:19.219 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.216+0000 7f699f7fe640 1 --2- 172.21.15.105:0/1058901458 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69880047d0 0x7f6988004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:19.219 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.216+0000 7f69a4af4640 1 --2- 172.21.15.105:0/1058901458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6988005390 0x7f698800b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:19.220 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.216+0000 7f699f7fe640 1 -- 172.21.15.105:0/1058901458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6988003980 msgr2=0x7f6988003d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:23:19.220 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.216+0000 7f699f7fe640 1 --2- 172.21.15.105:0/1058901458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6988003980 0x7f6988003d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:19.220 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.216+0000 7f699f7fe640 1 -- 172.21.15.105:0/1058901458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6988005390 msgr2=0x7f698800b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:19.220 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.216+0000 7f699f7fe640 1 --2- 172.21.15.105:0/1058901458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6988005390 0x7f698800b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:19.220 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.216+0000 7f699f7fe640 1 -- 172.21.15.105:0/1058901458 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a00d3c90 con 0x7f69880047d0 2023-11-28T18:23:19.220 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.216+0000 7f69a4af4640 1 -- 172.21.15.105:0/1058901458 reap_dead start 2023-11-28T18:23:19.226 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.224+0000 7f699f7fe640 1 --2- 172.21.15.105:0/1058901458 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69880047d0 0x7f6988004c50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f699400c290 tx=0x7f699401cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53e50c42757cef60 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:19.229 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.228+0000 7f699effd640 1 -- 172.21.15.105:0/1058901458 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6994005ae0 con 0x7f69880047d0 2023-11-28T18:23:19.229 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.228+0000 7f699effd640 1 -- 172.21.15.105:0/1058901458 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6994005c40 con 0x7f69880047d0 2023-11-28T18:23:19.229 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.228+0000 7f699effd640 1 -- 172.21.15.105:0/1058901458 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69940a5890 con 0x7f69880047d0 2023-11-28T18:23:19.230 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.228+0000 7f69a52f5640 1 -- 172.21.15.105:0/1058901458 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69880047d0 msgr2=0x7f6988004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:19.230 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.228+0000 7f69a52f5640 1 --2- 172.21.15.105:0/1058901458 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69880047d0 0x7f6988004c50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f699400c290 tx=0x7f699401cca0 comp rx=0 tx=0).stop 2023-11-28T18:23:19.231 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.228+0000 7f69a52f5640 1 -- 172.21.15.105:0/1058901458 shutdown_connections 2023-11-28T18:23:19.231 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.228+0000 7f69a52f5640 1 --2- 172.21.15.105:0/1058901458 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69880047d0 0x7f6988004c50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:19.231 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.228+0000 7f69a52f5640 1 -- 172.21.15.105:0/1058901458 >> 172.21.15.105:0/1058901458 conn(0x7f69a00d2df0 msgr2=0x7f69a00d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:19.231 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.228+0000 7f69a52f5640 1 -- 172.21.15.105:0/1058901458 shutdown_connections 2023-11-28T18:23:19.231 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.228+0000 7f69a52f5640 1 -- 172.21.15.105:0/1058901458 wait complete. 2023-11-28T18:23:19.232 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.232+0000 7f69a52f5640 1 Processor -- start 2023-11-28T18:23:19.233 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.232+0000 7f69a52f5640 1 -- start start 2023-11-28T18:23:19.235 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.232+0000 7f69a52f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69880047d0 0x7f69a00c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:19.235 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.232+0000 7f69a52f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a00b7380 0x7f69a01f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:19.235 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.232+0000 7f69a52f5640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69a01f58b0 0x7f69a00cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:19.235 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.232+0000 7f69a52f5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a00c9250 con 0x7f69880047d0 2023-11-28T18:23:19.235 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.232+0000 7f69a52f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a00c8890 con 0x7f69a00b7380 2023-11-28T18:23:19.235 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.232+0000 7f69a52f5640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a00b78d0 con 0x7f69a01f58b0 2023-11-28T18:23:19.235 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.232+0000 7f69a4af4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69a01f58b0 0x7f69a00cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:19.235 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.232+0000 7f69a4af4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69a01f58b0 0x7f69a00cbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:52786/0 (socket says 172.21.15.105:52786) 2023-11-28T18:23:19.236 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.232+0000 7f69a4af4640 1 -- 172.21.15.105:0/2170677795 learned_addr learned my addr 172.21.15.105:0/2170677795 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:19.236 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.232+0000 7f699f7fe640 1 --2- 172.21.15.105:0/2170677795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a00b7380 0x7f69a01f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:19.236 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.232+0000 7f699ffff640 1 --2- 172.21.15.105:0/2170677795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69880047d0 0x7f69a00c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:19.238 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.236+0000 7f69a4af4640 1 -- 172.21.15.105:0/2170677795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a00b7380 msgr2=0x7f69a01f5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:19.238 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.236+0000 7f69a4af4640 1 --2- 172.21.15.105:0/2170677795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a00b7380 0x7f69a01f5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:19.239 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.236+0000 7f69a4af4640 1 -- 172.21.15.105:0/2170677795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69880047d0 msgr2=0x7f69a00c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:19.239 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.236+0000 7f69a4af4640 1 --2- 172.21.15.105:0/2170677795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69880047d0 0x7f69a00c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:19.239 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.236+0000 7f69a4af4640 1 -- 172.21.15.105:0/2170677795 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a00b7bb0 con 0x7f69a01f58b0 2023-11-28T18:23:19.239 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.236+0000 7f69a4af4640 1 --2- 172.21.15.105:0/2170677795 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69a01f58b0 0x7f69a00cbed0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f69a008c260 tx=0x7f699800b700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:19.239 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.236+0000 7f699cff9640 1 -- 172.21.15.105:0/2170677795 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699800e420 con 0x7f69a01f58b0 2023-11-28T18:23:19.239 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.236+0000 7f699cff9640 1 -- 172.21.15.105:0/2170677795 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6998012070 con 0x7f69a01f58b0 2023-11-28T18:23:19.239 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.236+0000 7f699cff9640 1 -- 172.21.15.105:0/2170677795 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699800e7e0 con 0x7f69a01f58b0 2023-11-28T18:23:19.239 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.236+0000 7f69a52f5640 1 -- 172.21.15.105:0/2170677795 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69a00b7e90 con 0x7f69a01f58b0 2023-11-28T18:23:19.239 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.236+0000 7f69a52f5640 1 -- 172.21.15.105:0/2170677795 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69a00cc410 con 0x7f69a01f58b0 2023-11-28T18:23:19.240 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.236+0000 7f69a52f5640 1 -- 172.21.15.105:0/2170677795 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69a0074580 con 0x7f69a01f58b0 2023-11-28T18:23:19.242 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.240+0000 7f699cff9640 1 -- 172.21.15.105:0/2170677795 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 4) v1 ==== 86114+0+0 (secure 0 0 0) 0x7f699800e940 con 0x7f69a01f58b0 2023-11-28T18:23:19.244 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.240+0000 7f699cff9640 1 --2- 172.21.15.105:0/2170677795 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f698c066840 0x7f698c068d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:19.244 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.240+0000 7f699cff9640 1 -- 172.21.15.105:0/2170677795 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f6998083ca0 con 0x7f69a01f58b0 2023-11-28T18:23:19.245 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.244+0000 7f699ffff640 1 --2- 172.21.15.105:0/2170677795 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f698c066840 0x7f698c068d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:19.246 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.244+0000 7f699ffff640 1 --2- 172.21.15.105:0/2170677795 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f698c066840 0x7f698c068d00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6990006fd0 tx=0x7f69900090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:19.255 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.252+0000 7f699cff9640 1 -- 172.21.15.105:0/2170677795 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6998051e70 con 0x7f69a01f58b0 2023-11-28T18:23:19.561 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.560+0000 7f69a52f5640 1 -- 172.21.15.105:0/2170677795 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69a00b1000 con 0x7f69a01f58b0 2023-11-28T18:23:19.563 INFO:teuthology.misc.health.smithi105.stdout: 2023-11-28T18:23:19.563 INFO:teuthology.misc.health.smithi105.stdout:{"epoch":12,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:23:17.276180+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:23:19.563 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.560+0000 7f699cff9640 1 -- 172.21.15.105:0/2170677795 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f6998026020 con 0x7f69a01f58b0 2023-11-28T18:23:19.566 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.564+0000 7f69a52f5640 1 -- 172.21.15.105:0/2170677795 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f698c066840 msgr2=0x7f698c068d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:19.566 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.564+0000 7f69a52f5640 1 --2- 172.21.15.105:0/2170677795 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f698c066840 0x7f698c068d00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6990006fd0 tx=0x7f69900090e0 comp rx=0 tx=0).stop 2023-11-28T18:23:19.566 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.564+0000 7f69a52f5640 1 -- 172.21.15.105:0/2170677795 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69a01f58b0 msgr2=0x7f69a00cbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:19.566 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.564+0000 7f69a52f5640 1 --2- 172.21.15.105:0/2170677795 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69a01f58b0 0x7f69a00cbed0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f69a008c260 tx=0x7f699800b700 comp rx=0 tx=0).stop 2023-11-28T18:23:19.567 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.564+0000 7f69a52f5640 1 -- 172.21.15.105:0/2170677795 shutdown_connections 2023-11-28T18:23:19.567 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.564+0000 7f69a52f5640 1 --2- 172.21.15.105:0/2170677795 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69a01f58b0 0x7f69a00cbed0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:19.567 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.564+0000 7f69a52f5640 1 --2- 172.21.15.105:0/2170677795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a00b7380 0x7f69a01f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:19.567 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.564+0000 7f69a52f5640 1 --2- 172.21.15.105:0/2170677795 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f698c066840 0x7f698c068d00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:19.567 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.564+0000 7f69a52f5640 1 --2- 172.21.15.105:0/2170677795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69880047d0 0x7f69a00c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:19.567 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.564+0000 7f69a52f5640 1 -- 172.21.15.105:0/2170677795 >> 172.21.15.105:0/2170677795 conn(0x7f69a00d2df0 msgr2=0x7f69a015d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:19.568 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.564+0000 7f69a52f5640 1 -- 172.21.15.105:0/2170677795 shutdown_connections 2023-11-28T18:23:19.568 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:19.564+0000 7f69a52f5640 1 -- 172.21.15.105:0/2170677795 wait complete. 2023-11-28T18:23:19.601 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-28T18:23:19.709 INFO:tasks.ceph.osd.4.smithi148.stderr:2023-11-28T18:23:19.705+0000 7f1b35477640 -1 osd.4 0 waiting for initial osdmap 2023-11-28T18:23:19.743 INFO:tasks.ceph.osd.2.smithi105.stderr:2023-11-28T18:23:19.740+0000 7f74eadd1640 -1 osd.2 0 waiting for initial osdmap 2023-11-28T18:23:19.744 INFO:tasks.ceph.osd.5.smithi148.stderr:2023-11-28T18:23:19.741+0000 7f4816fd8640 -1 osd.5 0 waiting for initial osdmap 2023-11-28T18:23:19.749 INFO:tasks.ceph.osd.4.smithi148.stderr:2023-11-28T18:23:19.745+0000 7f1b2caa0640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T18:23:19.763 INFO:tasks.ceph.osd.6.smithi148.stderr:2023-11-28T18:23:19.761+0000 7f8781ccb640 -1 osd.6 0 waiting for initial osdmap 2023-11-28T18:23:19.764 INFO:tasks.ceph.osd.7.smithi148.stderr:2023-11-28T18:23:19.761+0000 7fb846059640 -1 osd.7 0 waiting for initial osdmap 2023-11-28T18:23:19.786 INFO:tasks.ceph.osd.2.smithi105.stderr:2023-11-28T18:23:19.784+0000 7f74e2c0c640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T18:23:19.791 INFO:tasks.ceph.osd.5.smithi148.stderr:2023-11-28T18:23:19.789+0000 7f480ee13640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T18:23:19.799 INFO:tasks.ceph.osd.7.smithi148.stderr:2023-11-28T18:23:19.797+0000 7fb83d682640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T18:23:19.799 INFO:tasks.ceph.osd.6.smithi148.stderr:2023-11-28T18:23:19.797+0000 7f8779b06640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T18:23:19.842 INFO:tasks.ceph.osd.3.smithi105.stderr:2023-11-28T18:23:19.840+0000 7fb5afca8640 -1 osd.3 0 waiting for initial osdmap 2023-11-28T18:23:19.847 INFO:tasks.ceph.osd.1.smithi105.stderr:2023-11-28T18:23:19.844+0000 7f7d6572d640 -1 osd.1 0 waiting for initial osdmap 2023-11-28T18:23:19.867 INFO:tasks.ceph.osd.0.smithi105.stderr:2023-11-28T18:23:19.864+0000 7fd864527640 -1 osd.0 0 waiting for initial osdmap 2023-11-28T18:23:19.870 INFO:tasks.ceph.osd.3.smithi105.stderr:2023-11-28T18:23:19.868+0000 7fb5a7ae3640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T18:23:19.886 INFO:tasks.ceph.osd.1.smithi105.stderr:2023-11-28T18:23:19.884+0000 7f7d5d568640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T18:23:19.893 INFO:tasks.ceph.osd.0.smithi105.stderr:2023-11-28T18:23:19.892+0000 7fd85bb50640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T18:23:26.409 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T18:23:26.545 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.544+0000 7fc956f80640 1 Processor -- start 2023-11-28T18:23:26.545 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.544+0000 7fc956f80640 1 -- start start 2023-11-28T18:23:26.547 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.544+0000 7fc956f80640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc95015d400 0x7fc9500d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:26.547 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.544+0000 7fc956f80640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc9500b2050 0x7fc9500d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:26.547 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.544+0000 7fc956f80640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9500d04b0 con 0x7fc9500b03c0 2023-11-28T18:23:26.547 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.544+0000 7fc956f80640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9500d3bb0 con 0x7fc9500b2050 2023-11-28T18:23:26.547 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.544+0000 7fc956f80640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9500afc50 con 0x7fc95015d400 2023-11-28T18:23:26.547 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.544+0000 7fc954d1c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc95015d400 0x7fc9500d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:26.547 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.544+0000 7fc947fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc9500b2050 0x7fc9500d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:26.547 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.544+0000 7fc954d1c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc95015d400 0x7fc9500d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:41658/0 (socket says 172.21.15.105:41658) 2023-11-28T18:23:26.548 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.544+0000 7fc95551d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc9500b03c0 0x7fc9500b0850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:41916/0 (socket says 172.21.15.105:41916) 2023-11-28T18:23:26.548 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.544+0000 7fc954d1c640 1 -- 172.21.15.105:0/1604597108 learned_addr learned my addr 172.21.15.105:0/1604597108 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:26.548 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.544+0000 7fc947fff640 1 --2- 172.21.15.105:0/1604597108 >> v2:172.21.15.148:3300/0 conn(0x7fc9500b2050 0x7fc9500d1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:55556/0 (socket says 172.21.15.105:55556) 2023-11-28T18:23:26.548 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.544+0000 7fc947fff640 1 -- 172.21.15.105:0/1604597108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc95015d400 msgr2=0x7fc9500d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:26.548 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.544+0000 7fc947fff640 1 --2- 172.21.15.105:0/1604597108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc95015d400 0x7fc9500d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:26.548 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.544+0000 7fc947fff640 1 -- 172.21.15.105:0/1604597108 >> v1:172.21.15.148:6789/0 conn(0x7fc9500b03c0 legacy=0x7fc9500b0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:26.549 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.544+0000 7fc947fff640 1 -- 172.21.15.105:0/1604597108 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9500d3e00 con 0x7fc9500b2050 2023-11-28T18:23:26.550 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.548+0000 7fc947fff640 1 --2- 172.21.15.105:0/1604597108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9500b2050 0x7fc9500d1a00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc9400097c0 tx=0x7fc94001c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da53f15a10c84274 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:26.551 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.548+0000 7fc9477fe640 1 -- 172.21.15.105:0/1604597108 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9400ab600 con 0x7fc9500b2050 2023-11-28T18:23:26.551 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.548+0000 7fc9477fe640 1 -- 172.21.15.105:0/1604597108 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9400a7050 con 0x7fc9500b2050 2023-11-28T18:23:26.551 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.548+0000 7fc9477fe640 1 -- 172.21.15.105:0/1604597108 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9400ab980 con 0x7fc9500b2050 2023-11-28T18:23:26.551 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.548+0000 7fc956f80640 1 -- 172.21.15.105:0/1604597108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9500b2050 msgr2=0x7fc9500d1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:26.552 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.548+0000 7fc956f80640 1 --2- 172.21.15.105:0/1604597108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9500b2050 0x7fc9500d1a00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc9400097c0 tx=0x7fc94001c330 comp rx=0 tx=0).stop 2023-11-28T18:23:26.552 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.548+0000 7fc956f80640 1 -- 172.21.15.105:0/1604597108 shutdown_connections 2023-11-28T18:23:26.552 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.548+0000 7fc956f80640 1 --2- 172.21.15.105:0/1604597108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9500b2050 0x7fc9500d1a00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:26.552 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.548+0000 7fc956f80640 1 --2- 172.21.15.105:0/1604597108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc95015d400 0x7fc9500d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:26.552 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.548+0000 7fc956f80640 1 -- 172.21.15.105:0/1604597108 >> 172.21.15.105:0/1604597108 conn(0x7fc9500d3090 msgr2=0x7fc9500d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:26.552 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.548+0000 7fc956f80640 1 -- 172.21.15.105:0/1604597108 shutdown_connections 2023-11-28T18:23:26.553 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.548+0000 7fc956f80640 1 -- 172.21.15.105:0/1604597108 wait complete. 2023-11-28T18:23:26.553 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc956f80640 1 Processor -- start 2023-11-28T18:23:26.553 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc956f80640 1 -- start start 2023-11-28T18:23:26.555 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc956f80640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9500b03c0 0x7fc9500cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:26.555 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc956f80640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc9500b2050 0x7fc950158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:26.555 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc956f80640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc95015d400 0x7fc95006c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:26.555 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc956f80640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9500cbee0 con 0x7fc9500b03c0 2023-11-28T18:23:26.555 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc956f80640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9500cc1c0 con 0x7fc95015d400 2023-11-28T18:23:26.555 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc956f80640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9500cc4a0 con 0x7fc9500b2050 2023-11-28T18:23:26.555 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc954d1c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9500b03c0 0x7fc9500cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:26.555 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc954d1c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9500b03c0 0x7fc9500cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54822/0 (socket says 172.21.15.105:54822) 2023-11-28T18:23:26.555 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc954d1c640 1 -- 172.21.15.105:0/131090484 learned_addr learned my addr 172.21.15.105:0/131090484 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:26.555 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc947fff640 1 --2- 172.21.15.105:0/131090484 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc9500b2050 0x7fc950158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:26.556 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc95551d640 1 --2- 172.21.15.105:0/131090484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc95015d400 0x7fc95006c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:26.556 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc954d1c640 1 -- 172.21.15.105:0/131090484 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc9500b2050 msgr2=0x7fc950158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:26.556 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc954d1c640 1 --2- 172.21.15.105:0/131090484 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc9500b2050 0x7fc950158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:26.556 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc954d1c640 1 -- 172.21.15.105:0/131090484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc95015d400 msgr2=0x7fc95006c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:26.556 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc954d1c640 1 --2- 172.21.15.105:0/131090484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc95015d400 0x7fc95006c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:26.556 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc954d1c640 1 -- 172.21.15.105:0/131090484 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc940009420 con 0x7fc9500b03c0 2023-11-28T18:23:26.557 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc954d1c640 1 --2- 172.21.15.105:0/131090484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9500b03c0 0x7fc9500cb890 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00e890 tx=0x7fc94c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:26.557 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc9457fa640 1 -- 172.21.15.105:0/131090484 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c009620 con 0x7fc9500b03c0 2023-11-28T18:23:26.557 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc9457fa640 1 -- 172.21.15.105:0/131090484 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94c00c940 con 0x7fc9500b03c0 2023-11-28T18:23:26.557 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc956f80640 1 -- 172.21.15.105:0/131090484 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9500cc780 con 0x7fc9500b03c0 2023-11-28T18:23:26.557 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc9457fa640 1 -- 172.21.15.105:0/131090484 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c00ccf0 con 0x7fc9500b03c0 2023-11-28T18:23:26.557 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.552+0000 7fc956f80640 1 -- 172.21.15.105:0/131090484 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc950158570 con 0x7fc9500b03c0 2023-11-28T18:23:26.558 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.556+0000 7fc956f80640 1 -- 172.21.15.105:0/131090484 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc950067570 con 0x7fc9500b03c0 2023-11-28T18:23:26.558 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.556+0000 7fc9457fa640 1 -- 172.21.15.105:0/131090484 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc94c013470 con 0x7fc9500b03c0 2023-11-28T18:23:26.558 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.556+0000 7fc9457fa640 1 --2- 172.21.15.105:0/131090484 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc93c0668d0 0x7fc93c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:26.558 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.556+0000 7fc9457fa640 1 -- 172.21.15.105:0/131090484 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc94c084220 con 0x7fc9500b03c0 2023-11-28T18:23:26.558 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.556+0000 7fc947fff640 1 --2- 172.21.15.105:0/131090484 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc93c0668d0 0x7fc93c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:26.559 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.556+0000 7fc947fff640 1 --2- 172.21.15.105:0/131090484 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc93c0668d0 0x7fc93c068d90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc9400ad000 tx=0x7fc940008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:26.563 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.560+0000 7fc9457fa640 1 -- 172.21.15.105:0/131090484 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc94c051860 con 0x7fc9500b03c0 2023-11-28T18:23:26.713 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.712+0000 7fc956f80640 1 -- 172.21.15.105:0/131090484 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc950072eb0 con 0x7fc9500b03c0 2023-11-28T18:23:26.714 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.712+0000 7fc9457fa640 1 -- 172.21.15.105:0/131090484 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12910 (secure 0 0 0) 0x7fc94c054e80 con 0x7fc9500b03c0 2023-11-28T18:23:26.714 INFO:teuthology.misc.health.smithi105.stdout: 2023-11-28T18:23:26.715 INFO:teuthology.misc.health.smithi105.stdout:{"epoch":16,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:23:24.027932+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:23:26.716 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.712+0000 7fc956f80640 1 -- 172.21.15.105:0/131090484 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc93c0668d0 msgr2=0x7fc93c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:26.716 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.712+0000 7fc956f80640 1 --2- 172.21.15.105:0/131090484 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc93c0668d0 0x7fc93c068d90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc9400ad000 tx=0x7fc940008040 comp rx=0 tx=0).stop 2023-11-28T18:23:26.717 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.712+0000 7fc956f80640 1 -- 172.21.15.105:0/131090484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9500b03c0 msgr2=0x7fc9500cb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:26.717 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.712+0000 7fc956f80640 1 --2- 172.21.15.105:0/131090484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9500b03c0 0x7fc9500cb890 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00e890 tx=0x7fc94c00ed50 comp rx=0 tx=0).stop 2023-11-28T18:23:26.717 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.712+0000 7fc956f80640 1 -- 172.21.15.105:0/131090484 shutdown_connections 2023-11-28T18:23:26.717 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.712+0000 7fc956f80640 1 --2- 172.21.15.105:0/131090484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc95015d400 0x7fc95006c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:26.717 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.712+0000 7fc956f80640 1 --2- 172.21.15.105:0/131090484 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc9500b2050 0x7fc950158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:26.717 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.712+0000 7fc956f80640 1 --2- 172.21.15.105:0/131090484 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc93c0668d0 0x7fc93c068d90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:26.717 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.712+0000 7fc956f80640 1 --2- 172.21.15.105:0/131090484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9500b03c0 0x7fc9500cb890 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:26.717 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.712+0000 7fc956f80640 1 -- 172.21.15.105:0/131090484 >> 172.21.15.105:0/131090484 conn(0x7fc9500d3090 msgr2=0x7fc95015e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:26.717 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.716+0000 7fc956f80640 1 -- 172.21.15.105:0/131090484 shutdown_connections 2023-11-28T18:23:26.717 INFO:teuthology.misc.health.smithi105.stderr:2023-11-28T18:23:26.716+0000 7fc956f80640 1 -- 172.21.15.105:0/131090484 wait complete. 2023-11-28T18:23:26.733 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-28T18:23:26.733 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-28T18:23:26.734 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T18:23:26.734 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T18:23:26.737 INFO:tasks.ceph.mds.a:Started 2023-11-28T18:23:26.738 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T18:23:26.738 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T18:23:26.741 INFO:tasks.ceph.mds.c:Started 2023-11-28T18:23:26.741 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T18:23:26.741 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T18:23:26.745 INFO:tasks.ceph.mds.b:Started 2023-11-28T18:23:26.745 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T18:23:26.746 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T18:23:26.749 INFO:tasks.ceph.mds.d:Started 2023-11-28T18:23:26.749 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-28T18:23:26.750 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:23:26.847 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2023-11-28T18:23:26.849 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2023-11-28T18:23:26.852 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2023-11-28T18:23:26.854 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2023-11-28T18:23:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.896+0000 7f8ddc6bb640 1 Processor -- start 2023-11-28T18:23:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.896+0000 7f8ddc6bb640 1 -- start start 2023-11-28T18:23:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.896+0000 7f8ddc6bb640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8dd415d4f0 0x7f8dd40d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:26.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.896+0000 7f8ddc6bb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8dd40d3c30 0x7f8dd40d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:26.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.896+0000 7f8ddc6bb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8dd40d1890 0x7f8dd40cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:26.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.896+0000 7f8ddc6bb640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dd40b2900 con 0x7f8dd40d1890 2023-11-28T18:23:26.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.896+0000 7f8ddc6bb640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dd40b25b0 con 0x7f8dd40d3c30 2023-11-28T18:23:26.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.896+0000 7f8ddc6bb640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd40cea30 con 0x7f8dd415d4f0 2023-11-28T18:23:26.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.900+0000 7f8ddac58640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8dd40d1890 0x7f8dd40cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:26.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.900+0000 7f8ddac58640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8dd40d1890 0x7f8dd40cfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54880/0 (socket says 172.21.15.105:54880) 2023-11-28T18:23:26.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.900+0000 7f8ddac58640 1 -- 172.21.15.105:0/55319433 learned_addr learned my addr 172.21.15.105:0/55319433 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.900+0000 7f8ddac58640 1 -- 172.21.15.105:0/55319433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8dd415d4f0 msgr2=0x7f8dd40d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.900+0000 7f8dd9c56640 1 --2- 172.21.15.105:0/55319433 >> v2:172.21.15.148:3300/0 conn(0x7f8dd40d3c30 0x7f8dd40d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.900+0000 7f8dda457640 1 --2- 172.21.15.105:0/55319433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8dd415d4f0 0x7f8dd40d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.900+0000 7f8ddac58640 1 --2- 172.21.15.105:0/55319433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8dd415d4f0 0x7f8dd40d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.900+0000 7f8ddac58640 1 -- 172.21.15.105:0/55319433 >> v2:172.21.15.148:3300/0 conn(0x7f8dd40d3c30 msgr2=0x7f8dd40d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.900+0000 7f8ddac58640 1 --2- 172.21.15.105:0/55319433 >> v2:172.21.15.148:3300/0 conn(0x7f8dd40d3c30 0x7f8dd40d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.900+0000 7f8ddac58640 1 -- 172.21.15.105:0/55319433 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd40ced10 con 0x7f8dd40d1890 2023-11-28T18:23:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.900+0000 7f8dda457640 1 --2- 172.21.15.105:0/55319433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8dd415d4f0 0x7f8dd40d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.900+0000 7f8dd9c56640 1 --2- 172.21.15.105:0/55319433 >> v2:172.21.15.148:3300/0 conn(0x7f8dd40d3c30 0x7f8dd40d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.900+0000 7f8ddac58640 1 --2- 172.21.15.105:0/55319433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dd40d1890 0x7f8dd40cfa10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8dd408b2f0 tx=0x7f8dcc01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fe7891b342ff6c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.900+0000 7f8dd9455640 1 -- 172.21.15.105:0/55319433 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc0a5730 con 0x7f8dd40d1890 2023-11-28T18:23:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.900+0000 7f8dd9455640 1 -- 172.21.15.105:0/55319433 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dcc0a5890 con 0x7f8dd40d1890 2023-11-28T18:23:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.900+0000 7f8dd9455640 1 -- 172.21.15.105:0/55319433 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc0a5c20 con 0x7f8dd40d1890 2023-11-28T18:23:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8ddc6bb640 1 -- 172.21.15.105:0/55319433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dd40d1890 msgr2=0x7f8dd40cfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8ddc6bb640 1 --2- 172.21.15.105:0/55319433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dd40d1890 0x7f8dd40cfa10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8dd408b2f0 tx=0x7f8dcc01c910 comp rx=0 tx=0).stop 2023-11-28T18:23:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8ddc6bb640 1 -- 172.21.15.105:0/55319433 shutdown_connections 2023-11-28T18:23:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8ddc6bb640 1 --2- 172.21.15.105:0/55319433 >> v2:172.21.15.148:3300/0 conn(0x7f8dd40d3c30 0x7f8dd40d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8ddc6bb640 1 --2- 172.21.15.105:0/55319433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dd40d1890 0x7f8dd40cfa10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8ddc6bb640 1 --2- 172.21.15.105:0/55319433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8dd415d4f0 0x7f8dd40d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8ddc6bb640 1 -- 172.21.15.105:0/55319433 >> 172.21.15.105:0/55319433 conn(0x7f8dd40d3030 msgr2=0x7f8dd40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8ddc6bb640 1 -- 172.21.15.105:0/55319433 shutdown_connections 2023-11-28T18:23:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8ddc6bb640 1 -- 172.21.15.105:0/55319433 wait complete. 2023-11-28T18:23:26.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8ddc6bb640 1 Processor -- start 2023-11-28T18:23:26.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8ddc6bb640 1 -- start start 2023-11-28T18:23:26.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8ddc6bb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dd40d1890 0x7f8dd406bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:26.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8ddc6bb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dd40d3c30 0x7f8dd406c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:26.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8ddc6bb640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8dd415d4f0 0x7f8dd40cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:26.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8ddc6bb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd4158220 con 0x7f8dd40d1890 2023-11-28T18:23:26.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8ddc6bb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd4158500 con 0x7f8dd40d3c30 2023-11-28T18:23:26.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8ddc6bb640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd41587e0 con 0x7f8dd415d4f0 2023-11-28T18:23:26.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8dda457640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dd40d1890 0x7f8dd406bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:26.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8dda457640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dd40d1890 0x7f8dd406bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54884/0 (socket says 172.21.15.105:54884) 2023-11-28T18:23:26.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8dda457640 1 -- 172.21.15.105:0/2224527346 learned_addr learned my addr 172.21.15.105:0/2224527346 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:26.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8dd9c56640 1 --2- 172.21.15.105:0/2224527346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dd40d3c30 0x7f8dd406c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:26.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8dda457640 1 -- 172.21.15.105:0/2224527346 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8dd415d4f0 msgr2=0x7f8dd40cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:26.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8ddac58640 1 --2- 172.21.15.105:0/2224527346 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8dd415d4f0 0x7f8dd40cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:26.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.904+0000 7f8dda457640 1 --2- 172.21.15.105:0/2224527346 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8dd415d4f0 0x7f8dd40cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:26.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.908+0000 7f8dda457640 1 -- 172.21.15.105:0/2224527346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dd40d3c30 msgr2=0x7f8dd406c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:26.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.908+0000 7f8dda457640 1 --2- 172.21.15.105:0/2224527346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dd40d3c30 0x7f8dd406c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:26.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.908+0000 7f8dda457640 1 -- 172.21.15.105:0/2224527346 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dcc009b30 con 0x7f8dd40d1890 2023-11-28T18:23:26.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.908+0000 7f8dd9c56640 1 --2- 172.21.15.105:0/2224527346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dd40d3c30 0x7f8dd406c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:26.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.908+0000 7f8ddac58640 1 --2- 172.21.15.105:0/2224527346 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8dd415d4f0 0x7f8dd40cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:26.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.908+0000 7f8dda457640 1 --2- 172.21.15.105:0/2224527346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dd40d1890 0x7f8dd406bf70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000c7f0 tx=0x7f8dd000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:26.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.908+0000 7f8dcaffd640 1 -- 172.21.15.105:0/2224527346 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0007820 con 0x7f8dd40d1890 2023-11-28T18:23:26.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.908+0000 7f8dcaffd640 1 -- 172.21.15.105:0/2224527346 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd0007980 con 0x7f8dd40d1890 2023-11-28T18:23:26.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.908+0000 7f8dcaffd640 1 -- 172.21.15.105:0/2224527346 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0007d30 con 0x7f8dd40d1890 2023-11-28T18:23:26.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.908+0000 7f8ddc6bb640 1 -- 172.21.15.105:0/2224527346 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd4158ac0 con 0x7f8dd40d1890 2023-11-28T18:23:26.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.908+0000 7f8ddc6bb640 1 -- 172.21.15.105:0/2224527346 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd4153830 con 0x7f8dd40d1890 2023-11-28T18:23:26.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.908+0000 7f8ddc6bb640 1 -- 172.21.15.105:0/2224527346 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dd4067570 con 0x7f8dd40d1890 2023-11-28T18:23:26.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.908+0000 7f8dcaffd640 1 -- 172.21.15.105:0/2224527346 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8dd00113f0 con 0x7f8dd40d1890 2023-11-28T18:23:26.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.908+0000 7f8dcaffd640 1 --2- 172.21.15.105:0/2224527346 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8db8066900 0x7f8db8068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:26.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.908+0000 7f8dcaffd640 1 -- 172.21.15.105:0/2224527346 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8dd0084350 con 0x7f8dd40d1890 2023-11-28T18:23:26.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.908+0000 7f8dd9c56640 1 --2- 172.21.15.105:0/2224527346 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8db8066900 0x7f8db8068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:26.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.912+0000 7f8dd9c56640 1 --2- 172.21.15.105:0/2224527346 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8db8066900 0x7f8db8068dc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4006fd0 tx=0x7f8dc4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:26.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:26.912+0000 7f8dcaffd640 1 -- 172.21.15.105:0/2224527346 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8dd0051950 con 0x7f8dd40d1890 2023-11-28T18:23:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.064+0000 7f8ddc6bb640 1 -- 172.21.15.105:0/2224527346 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8dd4155830 con 0x7f8dd40d1890 2023-11-28T18:23:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.064+0000 7f8dcaffd640 1 -- 172.21.15.105:0/2224527346 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12910 (secure 0 0 0) 0x7f8dd0054f70 con 0x7f8dd40d1890 2023-11-28T18:23:27.067 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:27.067 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":16,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:23:24.027932+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:23:27.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.064+0000 7f8ddc6bb640 1 -- 172.21.15.105:0/2224527346 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8db8066900 msgr2=0x7f8db8068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:27.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.064+0000 7f8ddc6bb640 1 --2- 172.21.15.105:0/2224527346 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8db8066900 0x7f8db8068dc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4006fd0 tx=0x7f8dc4008040 comp rx=0 tx=0).stop 2023-11-28T18:23:27.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.064+0000 7f8ddc6bb640 1 -- 172.21.15.105:0/2224527346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dd40d1890 msgr2=0x7f8dd406bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:27.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.064+0000 7f8ddc6bb640 1 --2- 172.21.15.105:0/2224527346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dd40d1890 0x7f8dd406bf70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000c7f0 tx=0x7f8dd000ccb0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.064+0000 7f8ddc6bb640 1 -- 172.21.15.105:0/2224527346 shutdown_connections 2023-11-28T18:23:27.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.064+0000 7f8ddc6bb640 1 --2- 172.21.15.105:0/2224527346 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8dd415d4f0 0x7f8dd40cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.064+0000 7f8ddc6bb640 1 --2- 172.21.15.105:0/2224527346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dd40d3c30 0x7f8dd406c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.064+0000 7f8ddc6bb640 1 --2- 172.21.15.105:0/2224527346 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8db8066900 0x7f8db8068dc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.064+0000 7f8ddc6bb640 1 --2- 172.21.15.105:0/2224527346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dd40d1890 0x7f8dd406bf70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.064+0000 7f8ddc6bb640 1 -- 172.21.15.105:0/2224527346 >> 172.21.15.105:0/2224527346 conn(0x7f8dd40d3030 msgr2=0x7f8dd40d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:27.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.064+0000 7f8ddc6bb640 1 -- 172.21.15.105:0/2224527346 shutdown_connections 2023-11-28T18:23:27.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.064+0000 7f8ddc6bb640 1 -- 172.21.15.105:0/2224527346 wait complete. 2023-11-28T18:23:27.087 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:23:27.087 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:23:27.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.224+0000 7f155b2c9640 1 Processor -- start 2023-11-28T18:23:27.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.224+0000 7f155b2c9640 1 -- start start 2023-11-28T18:23:27.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.228+0000 7f155b2c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f155406bf50 0x7f155415d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:27.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.228+0000 7f155b2c9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f15540adb80 0x7f15540adff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:27.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.228+0000 7f155b2c9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15540cc7f0 con 0x7f1554154070 2023-11-28T18:23:27.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.228+0000 7f155b2c9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1554072540 con 0x7f15540adb80 2023-11-28T18:23:27.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.228+0000 7f155b2c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15540ae530 con 0x7f155406bf50 2023-11-28T18:23:27.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.228+0000 7f1559866640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f15540adb80 0x7f15540adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:27.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.228+0000 7f1558864640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f155406bf50 0x7f155415d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:27.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.228+0000 7f1558864640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f155406bf50 0x7f155415d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:41740/0 (socket says 172.21.15.105:41740) 2023-11-28T18:23:27.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.228+0000 7f1558864640 1 -- 172.21.15.105:0/375824008 learned_addr learned my addr 172.21.15.105:0/375824008 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:27.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.228+0000 7f1559866640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f15540adb80 0x7f15540adff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:55644/0 (socket says 172.21.15.105:55644) 2023-11-28T18:23:27.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.228+0000 7f154bfff640 1 -- 172.21.15.105:0/375824008 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1116631955 0 0) 0x7f15540cc7f0 con 0x7f1554154070 2023-11-28T18:23:27.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.228+0000 7f154bfff640 1 -- 172.21.15.105:0/375824008 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1540003530 con 0x7f1554154070 2023-11-28T18:23:27.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.228+0000 7f1558864640 1 -- 172.21.15.105:0/375824008 >> v2:172.21.15.148:3300/0 conn(0x7f15540adb80 msgr2=0x7f15540adff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:27.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.228+0000 7f1558864640 1 --2- 172.21.15.105:0/375824008 >> v2:172.21.15.148:3300/0 conn(0x7f15540adb80 0x7f15540adff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.228+0000 7f1558864640 1 -- 172.21.15.105:0/375824008 >> v1:172.21.15.148:6789/0 conn(0x7f1554154070 legacy=0x7f15540ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:27.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.228+0000 7f1558864640 1 -- 172.21.15.105:0/375824008 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155415dd10 con 0x7f155406bf50 2023-11-28T18:23:27.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.228+0000 7f1559866640 1 --2- 172.21.15.105:0/375824008 >> v2:172.21.15.148:3300/0 conn(0x7f15540adb80 0x7f15540adff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:27.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.228+0000 7f1558864640 1 --2- 172.21.15.105:0/375824008 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f155406bf50 0x7f155415d700 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1544009870 tx=0x7f154401c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23d6f094593deed7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:27.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f154bfff640 1 -- 172.21.15.105:0/375824008 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15440ab600 con 0x7f155406bf50 2023-11-28T18:23:27.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f154bfff640 1 -- 172.21.15.105:0/375824008 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15440a7050 con 0x7f155406bf50 2023-11-28T18:23:27.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f154bfff640 1 -- 172.21.15.105:0/375824008 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15440ab980 con 0x7f155406bf50 2023-11-28T18:23:27.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f155b2c9640 1 -- 172.21.15.105:0/375824008 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f155406bf50 msgr2=0x7f155415d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:27.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f155b2c9640 1 --2- 172.21.15.105:0/375824008 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f155406bf50 0x7f155415d700 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1544009870 tx=0x7f154401c510 comp rx=0 tx=0).stop 2023-11-28T18:23:27.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f155b2c9640 1 -- 172.21.15.105:0/375824008 shutdown_connections 2023-11-28T18:23:27.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f155b2c9640 1 --2- 172.21.15.105:0/375824008 >> v2:172.21.15.148:3300/0 conn(0x7f15540adb80 0x7f15540adff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f155b2c9640 1 --2- 172.21.15.105:0/375824008 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f155406bf50 0x7f155415d700 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f155b2c9640 1 -- 172.21.15.105:0/375824008 >> 172.21.15.105:0/375824008 conn(0x7f15540ceb90 msgr2=0x7f15540cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:27.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f155b2c9640 1 -- 172.21.15.105:0/375824008 shutdown_connections 2023-11-28T18:23:27.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f155b2c9640 1 -- 172.21.15.105:0/375824008 wait complete. 2023-11-28T18:23:27.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f155b2c9640 1 Processor -- start 2023-11-28T18:23:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f155b2c9640 1 -- start start 2023-11-28T18:23:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f155b2c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f155406bf50 0x7f1554152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f155b2c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15540adb80 0x7f155414a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f155b2c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1554154070 0x7f15540b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f155b2c9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155414ab90 con 0x7f1554154070 2023-11-28T18:23:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f155b2c9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15540b3b40 con 0x7f155406bf50 2023-11-28T18:23:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f155b2c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15540b3de0 con 0x7f15540adb80 2023-11-28T18:23:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f1559866640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1554154070 0x7f15540b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f1559065640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f155406bf50 0x7f1554152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f1558864640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15540adb80 0x7f155414a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:27.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f1559866640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1554154070 0x7f15540b3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54888/0 (socket says 172.21.15.105:54888) 2023-11-28T18:23:27.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.232+0000 7f1559866640 1 -- 172.21.15.105:0/3931182614 learned_addr learned my addr 172.21.15.105:0/3931182614 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:27.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.236+0000 7f1559866640 1 -- 172.21.15.105:0/3931182614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15540adb80 msgr2=0x7f155414a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:27.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.236+0000 7f1559866640 1 --2- 172.21.15.105:0/3931182614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15540adb80 0x7f155414a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.236+0000 7f1559866640 1 -- 172.21.15.105:0/3931182614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f155406bf50 msgr2=0x7f1554152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:27.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.236+0000 7f1559866640 1 --2- 172.21.15.105:0/3931182614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f155406bf50 0x7f1554152b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.236+0000 7f1559866640 1 -- 172.21.15.105:0/3931182614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c009a70 con 0x7f1554154070 2023-11-28T18:23:27.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.236+0000 7f1559866640 1 --2- 172.21.15.105:0/3931182614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1554154070 0x7f15540b3600 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f154c00efc0 tx=0x7f154c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:27.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.236+0000 7f1549ffb640 1 -- 172.21.15.105:0/3931182614 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c009d70 con 0x7f1554154070 2023-11-28T18:23:27.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.236+0000 7f1549ffb640 1 -- 172.21.15.105:0/3931182614 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154c00f040 con 0x7f1554154070 2023-11-28T18:23:27.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.236+0000 7f1549ffb640 1 -- 172.21.15.105:0/3931182614 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c0039a0 con 0x7f1554154070 2023-11-28T18:23:27.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.236+0000 7f155b2c9640 1 -- 172.21.15.105:0/3931182614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15440094d0 con 0x7f1554154070 2023-11-28T18:23:27.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.236+0000 7f155b2c9640 1 -- 172.21.15.105:0/3931182614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15540b43a0 con 0x7f1554154070 2023-11-28T18:23:27.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.236+0000 7f155b2c9640 1 -- 172.21.15.105:0/3931182614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1554153050 con 0x7f1554154070 2023-11-28T18:23:27.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.236+0000 7f1549ffb640 1 -- 172.21.15.105:0/3931182614 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f154c0144a0 con 0x7f1554154070 2023-11-28T18:23:27.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.236+0000 7f1549ffb640 1 --2- 172.21.15.105:0/3931182614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15380668d0 0x7f1538068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:27.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.236+0000 7f1549ffb640 1 -- 172.21.15.105:0/3931182614 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f154c084d40 con 0x7f1554154070 2023-11-28T18:23:27.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.240+0000 7f1559065640 1 --2- 172.21.15.105:0/3931182614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15380668d0 0x7f1538068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:27.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.240+0000 7f1559065640 1 --2- 172.21.15.105:0/3931182614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15380668d0 0x7f1538068d90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f15500011f0 tx=0x7f1550009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:27.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.240+0000 7f1549ffb640 1 -- 172.21.15.105:0/3931182614 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f154c0522e0 con 0x7f1554154070 2023-11-28T18:23:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.392+0000 7f155b2c9640 1 -- 172.21.15.105:0/3931182614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1554067570 con 0x7f1554154070 2023-11-28T18:23:27.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.392+0000 7f1549ffb640 1 -- 172.21.15.105:0/3931182614 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f154c055900 con 0x7f1554154070 2023-11-28T18:23:27.396 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:23:27.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.396+0000 7f155b2c9640 1 -- 172.21.15.105:0/3931182614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15380668d0 msgr2=0x7f1538068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:27.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.396+0000 7f155b2c9640 1 --2- 172.21.15.105:0/3931182614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15380668d0 0x7f1538068d90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f15500011f0 tx=0x7f1550009040 comp rx=0 tx=0).stop 2023-11-28T18:23:27.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.396+0000 7f155b2c9640 1 -- 172.21.15.105:0/3931182614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1554154070 msgr2=0x7f15540b3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:27.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.396+0000 7f155b2c9640 1 --2- 172.21.15.105:0/3931182614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1554154070 0x7f15540b3600 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f154c00efc0 tx=0x7f154c009f90 comp rx=0 tx=0).stop 2023-11-28T18:23:27.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.396+0000 7f155b2c9640 1 -- 172.21.15.105:0/3931182614 shutdown_connections 2023-11-28T18:23:27.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.396+0000 7f155b2c9640 1 --2- 172.21.15.105:0/3931182614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15380668d0 0x7f1538068d90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.396+0000 7f155b2c9640 1 --2- 172.21.15.105:0/3931182614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1554154070 0x7f15540b3600 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.396+0000 7f155b2c9640 1 --2- 172.21.15.105:0/3931182614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15540adb80 0x7f155414a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.396+0000 7f155b2c9640 1 --2- 172.21.15.105:0/3931182614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f155406bf50 0x7f1554152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.396+0000 7f155b2c9640 1 -- 172.21.15.105:0/3931182614 >> 172.21.15.105:0/3931182614 conn(0x7f15540ceb90 msgr2=0x7f15541604a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:27.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.396+0000 7f155b2c9640 1 -- 172.21.15.105:0/3931182614 shutdown_connections 2023-11-28T18:23:27.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.396+0000 7f155b2c9640 1 -- 172.21.15.105:0/3931182614 wait complete. 2023-11-28T18:23:27.416 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:23:27.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f13ed6640 1 Processor -- start 2023-11-28T18:23:27.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f13ed6640 1 -- start start 2023-11-28T18:23:27.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f13ed6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f0c160a80 0x7f9f0c0d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f13ed6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f0c0d3b30 con 0x7f9f0c0b1f50 2023-11-28T18:23:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f13ed6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f0c0d3e10 con 0x7f9f0c0d1620 2023-11-28T18:23:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f13ed6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c0cdc50 con 0x7f9f0c160a80 2023-11-28T18:23:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f11471640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9f0c0b1f50 0x7f9f0c0cea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48114/0 (socket says 172.21.15.105:48114) 2023-11-28T18:23:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f11471640 1 -- 172.21.15.105:0/1359167854 learned_addr learned my addr 172.21.15.105:0/1359167854 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:27.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f11c72640 1 --2- 172.21.15.105:0/1359167854 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f0c160a80 0x7f9f0c0d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:27.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f10c70640 1 -- 172.21.15.105:0/1359167854 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4260197443 0 0) 0x7f9f0c0d3b30 con 0x7f9f0c0b1f50 2023-11-28T18:23:27.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f10c70640 1 -- 172.21.15.105:0/1359167854 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ef8003530 con 0x7f9f0c0b1f50 2023-11-28T18:23:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f10c70640 1 -- 172.21.15.105:0/1359167854 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4177957670 0 0) 0x7f9ef8003530 con 0x7f9f0c0b1f50 2023-11-28T18:23:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f10c70640 1 -- 172.21.15.105:0/1359167854 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f0c160a80 msgr2=0x7f9f0c0d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f10c70640 1 --2- 172.21.15.105:0/1359167854 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f0c160a80 0x7f9f0c0d0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f10c70640 1 -- 172.21.15.105:0/1359167854 >> v1:172.21.15.148:6789/0 conn(0x7f9f0c0d1620 legacy=0x7f9f0c0afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f10c70640 1 -- 172.21.15.105:0/1359167854 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0c0cdf30 con 0x7f9f0c0b1f50 2023-11-28T18:23:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f10c70640 1 -- 172.21.15.105:0/1359167854 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f9efc001680 con 0x7f9f0c0b1f50 2023-11-28T18:23:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f10c70640 1 -- 172.21.15.105:0/1359167854 >> v1:172.21.15.105:6789/0 conn(0x7f9f0c0b1f50 legacy=0x7f9f0c0cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f10c70640 1 --2- 172.21.15.105:0/1359167854 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f0c160a80 0x7f9ef80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f10c70640 1 --2- 172.21.15.105:0/1359167854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef8003ef0 0x7f9ef80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f10c70640 1 --2- 172.21.15.105:0/1359167854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ef8004d10 0x7f9ef8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f10c70640 1 -- 172.21.15.105:0/1359167854 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c0cdc50 con 0x7f9ef8003ef0 2023-11-28T18:23:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f10c70640 1 -- 172.21.15.105:0/1359167854 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef800baa0 con 0x7f9ef8004d10 2023-11-28T18:23:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f10c70640 1 -- 172.21.15.105:0/1359167854 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef800bc60 con 0x7f9f0c160a80 2023-11-28T18:23:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f11471640 1 --2- 172.21.15.105:0/1359167854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef8003ef0 0x7f9ef80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f11c72640 1 --2- 172.21.15.105:0/1359167854 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f0c160a80 0x7f9ef80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f12473640 1 --2- 172.21.15.105:0/1359167854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ef8004d10 0x7f9ef8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f11471640 1 -- 172.21.15.105:0/1359167854 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f0c160a80 msgr2=0x7f9ef80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f11471640 1 --2- 172.21.15.105:0/1359167854 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f0c160a80 0x7f9ef80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f11471640 1 -- 172.21.15.105:0/1359167854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ef8004d10 msgr2=0x7f9ef8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f11471640 1 --2- 172.21.15.105:0/1359167854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ef8004d10 0x7f9ef8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f11471640 1 -- 172.21.15.105:0/1359167854 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef8003530 con 0x7f9ef8003ef0 2023-11-28T18:23:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.552+0000 7f9f12473640 1 --2- 172.21.15.105:0/1359167854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ef8004d10 0x7f9ef8005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:27.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.556+0000 7f9f11471640 1 --2- 172.21.15.105:0/1359167854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef8003ef0 0x7f9ef80047d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00c190 tx=0x7f9efc01cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c79a733bbbc3c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:27.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.556+0000 7f9f10c70640 1 -- 172.21.15.105:0/1359167854 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc005ae0 con 0x7f9ef8003ef0 2023-11-28T18:23:27.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.556+0000 7f9f10c70640 1 -- 172.21.15.105:0/1359167854 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9efc005c60 con 0x7f9ef8003ef0 2023-11-28T18:23:27.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.556+0000 7f9f10c70640 1 -- 172.21.15.105:0/1359167854 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc0a5b00 con 0x7f9ef8003ef0 2023-11-28T18:23:27.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.556+0000 7f9f13ed6640 1 -- 172.21.15.105:0/1359167854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef8003ef0 msgr2=0x7f9ef80047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:27.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.556+0000 7f9f13ed6640 1 --2- 172.21.15.105:0/1359167854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef8003ef0 0x7f9ef80047d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00c190 tx=0x7f9efc01cbc0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.556+0000 7f9f12473640 1 -- 172.21.15.105:0/1359167854 reap_dead start 2023-11-28T18:23:27.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.556+0000 7f9f13ed6640 1 -- 172.21.15.105:0/1359167854 shutdown_connections 2023-11-28T18:23:27.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.556+0000 7f9f13ed6640 1 -- 172.21.15.105:0/1359167854 >> 172.21.15.105:0/1359167854 conn(0x7f9f0c0d3040 msgr2=0x7f9f0c0d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:27.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.556+0000 7f9f13ed6640 1 -- 172.21.15.105:0/1359167854 shutdown_connections 2023-11-28T18:23:27.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.556+0000 7f9f13ed6640 1 -- 172.21.15.105:0/1359167854 wait complete. 2023-11-28T18:23:27.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.560+0000 7f9f13ed6640 1 Processor -- start 2023-11-28T18:23:27.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.560+0000 7f9f13ed6640 1 -- start start 2023-11-28T18:23:27.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.560+0000 7f9f13ed6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef8003ef0 0x7f9f0c158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:27.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.560+0000 7f9f13ed6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f0c152250 0x7f9f0c1505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:27.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.560+0000 7f9f13ed6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f0c0ca110 0x7f9f0c0ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:27.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.560+0000 7f9f13ed6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c0cab00 con 0x7f9ef8003ef0 2023-11-28T18:23:27.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.560+0000 7f9f13ed6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c150ae0 con 0x7f9f0c0ca110 2023-11-28T18:23:27.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.560+0000 7f9f13ed6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c1593d0 con 0x7f9f0c152250 2023-11-28T18:23:27.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.560+0000 7f9f11c72640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef8003ef0 0x7f9f0c158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:27.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.560+0000 7f9f11c72640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef8003ef0 0x7f9f0c158df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54900/0 (socket says 172.21.15.105:54900) 2023-11-28T18:23:27.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.560+0000 7f9f11c72640 1 -- 172.21.15.105:0/3040692306 learned_addr learned my addr 172.21.15.105:0/3040692306 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:27.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.560+0000 7f9f11c72640 1 -- 172.21.15.105:0/3040692306 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f0c152250 msgr2=0x7f9f0c1505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:27.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.560+0000 7f9f11c72640 1 --2- 172.21.15.105:0/3040692306 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f0c152250 0x7f9f0c1505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.560+0000 7f9f11c72640 1 -- 172.21.15.105:0/3040692306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f0c0ca110 msgr2=0x7f9f0c0ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:27.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.560+0000 7f9f11c72640 1 --2- 172.21.15.105:0/3040692306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f0c0ca110 0x7f9f0c0ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.560+0000 7f9f11c72640 1 -- 172.21.15.105:0/3040692306 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0c0cbff0 con 0x7f9ef8003ef0 2023-11-28T18:23:27.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.560+0000 7f9f11c72640 1 --2- 172.21.15.105:0/3040692306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef8003ef0 0x7f9f0c158df0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9f080119d0 tx=0x7f9f08011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:27.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.560+0000 7f9f027fc640 1 -- 172.21.15.105:0/3040692306 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0800fb50 con 0x7f9ef8003ef0 2023-11-28T18:23:27.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.560+0000 7f9f027fc640 1 -- 172.21.15.105:0/3040692306 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0800fcb0 con 0x7f9ef8003ef0 2023-11-28T18:23:27.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.560+0000 7f9f027fc640 1 -- 172.21.15.105:0/3040692306 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f08014440 con 0x7f9ef8003ef0 2023-11-28T18:23:27.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.560+0000 7f9f13ed6640 1 -- 172.21.15.105:0/3040692306 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f0c0cc2d0 con 0x7f9ef8003ef0 2023-11-28T18:23:27.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.560+0000 7f9f13ed6640 1 -- 172.21.15.105:0/3040692306 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f0c0cc890 con 0x7f9ef8003ef0 2023-11-28T18:23:27.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.564+0000 7f9f13ed6640 1 -- 172.21.15.105:0/3040692306 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f0c0745f0 con 0x7f9ef8003ef0 2023-11-28T18:23:27.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.564+0000 7f9f027fc640 1 -- 172.21.15.105:0/3040692306 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f9f080145a0 con 0x7f9ef8003ef0 2023-11-28T18:23:27.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.564+0000 7f9f027fc640 1 --2- 172.21.15.105:0/3040692306 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9ef00668d0 0x7f9ef0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:27.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.564+0000 7f9f027fc640 1 -- 172.21.15.105:0/3040692306 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9f08084a60 con 0x7f9ef8003ef0 2023-11-28T18:23:27.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.564+0000 7f9f11471640 1 --2- 172.21.15.105:0/3040692306 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9ef00668d0 0x7f9ef0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:27.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.564+0000 7f9f11471640 1 --2- 172.21.15.105:0/3040692306 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9ef00668d0 0x7f9ef0068d90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0ab000 tx=0x7f9efc001990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.568+0000 7f9f027fc640 1 -- 172.21.15.105:0/3040692306 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9f08052010 con 0x7f9ef8003ef0 2023-11-28T18:23:27.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.748+0000 7f9f13ed6640 1 -- 172.21.15.105:0/3040692306 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f0c0c99a0 con 0x7f9ef8003ef0 2023-11-28T18:23:27.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.748+0000 7f9f027fc640 1 -- 172.21.15.105:0/3040692306 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1327 (secure 0 0 0) 0x7f9f08022090 con 0x7f9ef8003ef0 2023-11-28T18:23:27.752 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:27.752 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3271971772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3271971772},{"type":"v1","addr":"172.21.15.105:6835","nonce":3271971772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-11-28T18:23:27.752 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 2 2023-11-28T18:23:27.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.752+0000 7f9f13ed6640 1 -- 172.21.15.105:0/3040692306 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9ef00668d0 msgr2=0x7f9ef0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:27.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.752+0000 7f9f13ed6640 1 --2- 172.21.15.105:0/3040692306 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9ef00668d0 0x7f9ef0068d90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0ab000 tx=0x7f9efc001990 comp rx=0 tx=0).stop 2023-11-28T18:23:27.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.752+0000 7f9f13ed6640 1 -- 172.21.15.105:0/3040692306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef8003ef0 msgr2=0x7f9f0c158df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:27.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.752+0000 7f9f13ed6640 1 --2- 172.21.15.105:0/3040692306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef8003ef0 0x7f9f0c158df0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9f080119d0 tx=0x7f9f08011e90 comp rx=0 tx=0).stop 2023-11-28T18:23:27.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.752+0000 7f9f13ed6640 1 -- 172.21.15.105:0/3040692306 shutdown_connections 2023-11-28T18:23:27.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.752+0000 7f9f13ed6640 1 --2- 172.21.15.105:0/3040692306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f0c0ca110 0x7f9f0c0ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.752+0000 7f9f13ed6640 1 --2- 172.21.15.105:0/3040692306 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f0c152250 0x7f9f0c1505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.752+0000 7f9f13ed6640 1 --2- 172.21.15.105:0/3040692306 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9ef00668d0 0x7f9ef0068d90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.752+0000 7f9f13ed6640 1 --2- 172.21.15.105:0/3040692306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef8003ef0 0x7f9f0c158df0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:27.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.752+0000 7f9f13ed6640 1 -- 172.21.15.105:0/3040692306 >> 172.21.15.105:0/3040692306 conn(0x7f9f0c0d3040 msgr2=0x7f9f0c0afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:27.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.752+0000 7f9f13ed6640 1 -- 172.21.15.105:0/3040692306 shutdown_connections 2023-11-28T18:23:27.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:27.752+0000 7f9f13ed6640 1 -- 172.21.15.105:0/3040692306 wait complete. 2023-11-28T18:23:29.773 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:23:29.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.924+0000 7f48de6bb640 1 Processor -- start 2023-11-28T18:23:29.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.924+0000 7f48de6bb640 1 -- start start 2023-11-28T18:23:29.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.924+0000 7f48de6bb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f48d815e6e0 0x7f48d80cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:29.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.924+0000 7f48de6bb640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d80b1b60 con 0x7f48d80b2300 2023-11-28T18:23:29.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.924+0000 7f48de6bb640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d80b12c0 con 0x7f48d80ae180 2023-11-28T18:23:29.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.924+0000 7f48de6bb640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d80b15a0 con 0x7f48d815e6e0 2023-11-28T18:23:29.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.924+0000 7f48d77fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f48d80b2300 0x7f48d80cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48124/0 (socket says 172.21.15.105:48124) 2023-11-28T18:23:29.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.924+0000 7f48d7fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f48d815e6e0 0x7f48d80cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:29.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.924+0000 7f48d77fe640 1 -- 172.21.15.105:0/1738511027 learned_addr learned my addr 172.21.15.105:0/1738511027 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:29.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.928+0000 7f48d6ffd640 1 -- 172.21.15.105:0/1738511027 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1193917099 0 0) 0x7f48d80b1b60 con 0x7f48d80b2300 2023-11-28T18:23:29.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.928+0000 7f48d6ffd640 1 -- 172.21.15.105:0/1738511027 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48c0003530 con 0x7f48d80b2300 2023-11-28T18:23:29.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.928+0000 7f48d7fff640 1 -- 172.21.15.105:0/1738511027 >> v1:172.21.15.148:6789/0 conn(0x7f48d80ae180 legacy=0x7f48d80ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:29.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.928+0000 7f48d6ffd640 1 -- 172.21.15.105:0/1738511027 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 478305849 0 0) 0x7f48d80b12c0 con 0x7f48d80ae180 2023-11-28T18:23:29.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.928+0000 7f48d7fff640 1 -- 172.21.15.105:0/1738511027 >> v1:172.21.15.105:6789/0 conn(0x7f48d80b2300 legacy=0x7f48d80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:29.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.928+0000 7f48d7fff640 1 -- 172.21.15.105:0/1738511027 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d80b1880 con 0x7f48d815e6e0 2023-11-28T18:23:29.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.928+0000 7f48d7fff640 1 --2- 172.21.15.105:0/1738511027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48d815e6e0 0x7f48d80cedf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f48c80097c0 tx=0x7f48c801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b698a82a66312446 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:29.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.928+0000 7f48d6ffd640 1 -- 172.21.15.105:0/1738511027 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c80a8970 con 0x7f48d815e6e0 2023-11-28T18:23:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.928+0000 7f48d6ffd640 1 -- 172.21.15.105:0/1738511027 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c80a8ad0 con 0x7f48d815e6e0 2023-11-28T18:23:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.928+0000 7f48d6ffd640 1 -- 172.21.15.105:0/1738511027 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c80a8e60 con 0x7f48d815e6e0 2023-11-28T18:23:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.928+0000 7f48de6bb640 1 -- 172.21.15.105:0/1738511027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48d815e6e0 msgr2=0x7f48d80cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.928+0000 7f48de6bb640 1 --2- 172.21.15.105:0/1738511027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48d815e6e0 0x7f48d80cedf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f48c80097c0 tx=0x7f48c801ec70 comp rx=0 tx=0).stop 2023-11-28T18:23:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.928+0000 7f48de6bb640 1 -- 172.21.15.105:0/1738511027 shutdown_connections 2023-11-28T18:23:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.928+0000 7f48de6bb640 1 --2- 172.21.15.105:0/1738511027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48d815e6e0 0x7f48d80cedf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.928+0000 7f48de6bb640 1 -- 172.21.15.105:0/1738511027 >> 172.21.15.105:0/1738511027 conn(0x7f48d80d0df0 msgr2=0x7f48d80d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.932+0000 7f48de6bb640 1 -- 172.21.15.105:0/1738511027 shutdown_connections 2023-11-28T18:23:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.932+0000 7f48de6bb640 1 -- 172.21.15.105:0/1738511027 wait complete. 2023-11-28T18:23:29.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.932+0000 7f48de6bb640 1 Processor -- start 2023-11-28T18:23:29.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.932+0000 7f48de6bb640 1 -- start start 2023-11-28T18:23:29.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.932+0000 7f48de6bb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48d80ae180 0x7f48d814f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.932+0000 7f48de6bb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d80b2300 0x7f48d80c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.932+0000 7f48de6bb640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48d815e6e0 0x7f48d80af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.932+0000 7f48de6bb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d80c9eb0 con 0x7f48d80ae180 2023-11-28T18:23:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.932+0000 7f48de6bb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d80ca190 con 0x7f48d80b2300 2023-11-28T18:23:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.932+0000 7f48de6bb640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d80ca470 con 0x7f48d815e6e0 2023-11-28T18:23:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.932+0000 7f48d7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48d80ae180 0x7f48d814f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.932+0000 7f48d7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48d80ae180 0x7f48d814f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54910/0 (socket says 172.21.15.105:54910) 2023-11-28T18:23:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.932+0000 7f48d7fff640 1 -- 172.21.15.105:0/2544215073 learned_addr learned my addr 172.21.15.105:0/2544215073 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.932+0000 7f48d77fe640 1 --2- 172.21.15.105:0/2544215073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d80b2300 0x7f48d80c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.932+0000 7f48dcc58640 1 --2- 172.21.15.105:0/2544215073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48d815e6e0 0x7f48d80af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.932+0000 7f48d7fff640 1 -- 172.21.15.105:0/2544215073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48d815e6e0 msgr2=0x7f48d80af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.932+0000 7f48d7fff640 1 --2- 172.21.15.105:0/2544215073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48d815e6e0 0x7f48d80af690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.932+0000 7f48d7fff640 1 -- 172.21.15.105:0/2544215073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d80b2300 msgr2=0x7f48d80c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.932+0000 7f48d7fff640 1 --2- 172.21.15.105:0/2544215073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d80b2300 0x7f48d80c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.932+0000 7f48d7fff640 1 -- 172.21.15.105:0/2544215073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c8009420 con 0x7f48d80ae180 2023-11-28T18:23:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.932+0000 7f48d77fe640 1 --2- 172.21.15.105:0/2544215073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d80b2300 0x7f48d80c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.932+0000 7f48d7fff640 1 --2- 172.21.15.105:0/2544215073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48d80ae180 0x7f48d814f1f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f48c80b3040 tx=0x7f48c801ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.936+0000 7f48d4ff9640 1 -- 172.21.15.105:0/2544215073 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c80bf070 con 0x7f48d80ae180 2023-11-28T18:23:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.936+0000 7f48d4ff9640 1 -- 172.21.15.105:0/2544215073 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c80b0bf0 con 0x7f48d80ae180 2023-11-28T18:23:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.936+0000 7f48d4ff9640 1 -- 172.21.15.105:0/2544215073 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c80b9410 con 0x7f48d80ae180 2023-11-28T18:23:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.936+0000 7f48de6bb640 1 -- 172.21.15.105:0/2544215073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d80ca750 con 0x7f48d80ae180 2023-11-28T18:23:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.936+0000 7f48de6bb640 1 -- 172.21.15.105:0/2544215073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d806cbe0 con 0x7f48d80ae180 2023-11-28T18:23:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.936+0000 7f48de6bb640 1 -- 172.21.15.105:0/2544215073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d80cac70 con 0x7f48d80ae180 2023-11-28T18:23:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.936+0000 7f48d4ff9640 1 -- 172.21.15.105:0/2544215073 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f48c80b7040 con 0x7f48d80ae180 2023-11-28T18:23:29.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.936+0000 7f48d4ff9640 1 --2- 172.21.15.105:0/2544215073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48c4066900 0x7f48c4068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:29.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.936+0000 7f48d4ff9640 1 -- 172.21.15.105:0/2544215073 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f48c8121380 con 0x7f48d80ae180 2023-11-28T18:23:29.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.936+0000 7f48d77fe640 1 --2- 172.21.15.105:0/2544215073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48c4066900 0x7f48c4068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:29.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.940+0000 7f48d77fe640 1 --2- 172.21.15.105:0/2544215073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48c4066900 0x7f48c4068dc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f48d81f6870 tx=0x7f48cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:29.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:29.940+0000 7f48d4ff9640 1 -- 172.21.15.105:0/2544215073 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f48d80cac70 con 0x7f48d80ae180 2023-11-28T18:23:30.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.120+0000 7f48de6bb640 1 -- 172.21.15.105:0/2544215073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48d80cac70 con 0x7f48d80ae180 2023-11-28T18:23:30.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.120+0000 7f48d4ff9640 1 -- 172.21.15.105:0/2544215073 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3703 (secure 0 0 0) 0x7f48d80cac70 con 0x7f48d80ae180 2023-11-28T18:23:30.127 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:30.127 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/390529893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":390529893},{"type":"v1","addr":"172.21.15.148:6835","nonce":390529893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3924571887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3924571887},{"type":"v1","addr":"172.21.15.148:6833","nonce":3924571887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3271971772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3271971772},{"type":"v1","addr":"172.21.15.105:6835","nonce":3271971772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2504263238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2504263238},{"type":"v1","addr":"172.21.15.105:6837","nonce":2504263238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-11-28T18:23:30.128 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 3 2023-11-28T18:23:30.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.124+0000 7f48de6bb640 1 -- 172.21.15.105:0/2544215073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48c4066900 msgr2=0x7f48c4068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:30.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.124+0000 7f48de6bb640 1 --2- 172.21.15.105:0/2544215073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48c4066900 0x7f48c4068dc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f48d81f6870 tx=0x7f48cc009040 comp rx=0 tx=0).stop 2023-11-28T18:23:30.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.124+0000 7f48de6bb640 1 -- 172.21.15.105:0/2544215073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48d80ae180 msgr2=0x7f48d814f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:30.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.124+0000 7f48de6bb640 1 --2- 172.21.15.105:0/2544215073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48d80ae180 0x7f48d814f1f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f48c80b3040 tx=0x7f48c801ba10 comp rx=0 tx=0).stop 2023-11-28T18:23:30.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.124+0000 7f48de6bb640 1 -- 172.21.15.105:0/2544215073 shutdown_connections 2023-11-28T18:23:30.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.124+0000 7f48de6bb640 1 --2- 172.21.15.105:0/2544215073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48d815e6e0 0x7f48d80af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.124+0000 7f48de6bb640 1 --2- 172.21.15.105:0/2544215073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d80b2300 0x7f48d80c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.124+0000 7f48de6bb640 1 --2- 172.21.15.105:0/2544215073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48c4066900 0x7f48c4068dc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.124+0000 7f48de6bb640 1 --2- 172.21.15.105:0/2544215073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48d80ae180 0x7f48d814f1f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.124+0000 7f48de6bb640 1 -- 172.21.15.105:0/2544215073 >> 172.21.15.105:0/2544215073 conn(0x7f48d80d0df0 msgr2=0x7f48d80b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:30.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.124+0000 7f48de6bb640 1 -- 172.21.15.105:0/2544215073 shutdown_connections 2023-11-28T18:23:30.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.124+0000 7f48de6bb640 1 -- 172.21.15.105:0/2544215073 wait complete. 2023-11-28T18:23:30.146 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:23:30.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.280+0000 7faa93f01640 1 Processor -- start 2023-11-28T18:23:30.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.280+0000 7faa93f01640 1 -- start start 2023-11-28T18:23:30.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.284+0000 7faa93f01640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa8c0ce2c0 0x7faa8c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:30.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.284+0000 7faa93f01640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7faa8c06cd20 0x7faa8c0abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:30.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.284+0000 7faa93f01640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa8c0cde30 con 0x7faa8c159610 2023-11-28T18:23:30.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.284+0000 7faa93f01640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa8c06d1b0 con 0x7faa8c06cd20 2023-11-28T18:23:30.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.284+0000 7faa93f01640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c1599f0 con 0x7faa8c0ce2c0 2023-11-28T18:23:30.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.284+0000 7faa9249e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7faa8c06cd20 0x7faa8c0abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:30.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.284+0000 7faa9149c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa8c0ce2c0 0x7faa8c06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:30.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.284+0000 7faa9249e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7faa8c06cd20 0x7faa8c0abd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:55702/0 (socket says 172.21.15.105:55702) 2023-11-28T18:23:30.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.284+0000 7faa9249e640 1 -- 172.21.15.105:0/236965934 learned_addr learned my addr 172.21.15.105:0/236965934 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:30.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.284+0000 7faa9249e640 1 -- 172.21.15.105:0/236965934 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa8c0ce2c0 msgr2=0x7faa8c06c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:30.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.284+0000 7faa9249e640 1 --2- 172.21.15.105:0/236965934 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa8c0ce2c0 0x7faa8c06c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.284+0000 7faa9249e640 1 -- 172.21.15.105:0/236965934 >> v1:172.21.15.148:6789/0 conn(0x7faa8c159610 legacy=0x7faa8c0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:30.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.284+0000 7faa9249e640 1 -- 172.21.15.105:0/236965934 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8c06de20 con 0x7faa8c06cd20 2023-11-28T18:23:30.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.284+0000 7faa9149c640 1 --2- 172.21.15.105:0/236965934 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa8c0ce2c0 0x7faa8c06c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:30.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.288+0000 7faa9249e640 1 --2- 172.21.15.105:0/236965934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa8c06cd20 0x7faa8c0abd80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faa84009e50 tx=0x7faa8401c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c836fbd402c5a524 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:30.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.288+0000 7faa90c9b640 1 -- 172.21.15.105:0/236965934 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa840a5740 con 0x7faa8c06cd20 2023-11-28T18:23:30.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.288+0000 7faa90c9b640 1 -- 172.21.15.105:0/236965934 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa840a58a0 con 0x7faa8c06cd20 2023-11-28T18:23:30.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.288+0000 7faa90c9b640 1 -- 172.21.15.105:0/236965934 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa840a5c30 con 0x7faa8c06cd20 2023-11-28T18:23:30.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.288+0000 7faa93f01640 1 -- 172.21.15.105:0/236965934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa8c06cd20 msgr2=0x7faa8c0abd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:30.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.288+0000 7faa93f01640 1 --2- 172.21.15.105:0/236965934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa8c06cd20 0x7faa8c0abd80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faa84009e50 tx=0x7faa8401c9e0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.288+0000 7faa93f01640 1 -- 172.21.15.105:0/236965934 shutdown_connections 2023-11-28T18:23:30.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.288+0000 7faa93f01640 1 --2- 172.21.15.105:0/236965934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa8c06cd20 0x7faa8c0abd80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.288+0000 7faa93f01640 1 --2- 172.21.15.105:0/236965934 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa8c0ce2c0 0x7faa8c06c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.288+0000 7faa93f01640 1 -- 172.21.15.105:0/236965934 >> 172.21.15.105:0/236965934 conn(0x7faa8c0ceb90 msgr2=0x7faa8c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:30.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.288+0000 7faa93f01640 1 -- 172.21.15.105:0/236965934 shutdown_connections 2023-11-28T18:23:30.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.288+0000 7faa93f01640 1 -- 172.21.15.105:0/236965934 wait complete. 2023-11-28T18:23:30.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.288+0000 7faa93f01640 1 Processor -- start 2023-11-28T18:23:30.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.288+0000 7faa93f01640 1 -- start start 2023-11-28T18:23:30.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa93f01640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa8c06cd20 0x7faa8c147670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:30.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa93f01640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa8c0ce2c0 0x7faa8c158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:30.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa93f01640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa8c159610 0x7faa8c152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:30.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa93f01640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c156ec0 con 0x7faa8c159610 2023-11-28T18:23:30.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa93f01640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c147bb0 con 0x7faa8c06cd20 2023-11-28T18:23:30.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa93f01640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c156be0 con 0x7faa8c0ce2c0 2023-11-28T18:23:30.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa9249e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa8c159610 0x7faa8c152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:30.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa9249e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa8c159610 0x7faa8c152d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54926/0 (socket says 172.21.15.105:54926) 2023-11-28T18:23:30.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa9249e640 1 -- 172.21.15.105:0/1331611080 learned_addr learned my addr 172.21.15.105:0/1331611080 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:30.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa9149c640 1 --2- 172.21.15.105:0/1331611080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa8c0ce2c0 0x7faa8c158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:30.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa91c9d640 1 --2- 172.21.15.105:0/1331611080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa8c06cd20 0x7faa8c147670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:30.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa9249e640 1 -- 172.21.15.105:0/1331611080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa8c0ce2c0 msgr2=0x7faa8c158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:30.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa9249e640 1 --2- 172.21.15.105:0/1331611080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa8c0ce2c0 0x7faa8c158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa9249e640 1 -- 172.21.15.105:0/1331611080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa8c06cd20 msgr2=0x7faa8c147670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:30.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa9249e640 1 --2- 172.21.15.105:0/1331611080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa8c06cd20 0x7faa8c147670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa9249e640 1 -- 172.21.15.105:0/1331611080 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c00a5c0 con 0x7faa8c159610 2023-11-28T18:23:30.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa9249e640 1 --2- 172.21.15.105:0/1331611080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa8c159610 0x7faa8c152d70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faa840af040 tx=0x7faa84019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:30.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa827fc640 1 -- 172.21.15.105:0/1331611080 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa840b8670 con 0x7faa8c159610 2023-11-28T18:23:30.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa827fc640 1 -- 172.21.15.105:0/1331611080 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa840b87f0 con 0x7faa8c159610 2023-11-28T18:23:30.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa827fc640 1 -- 172.21.15.105:0/1331611080 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa840b8ba0 con 0x7faa8c159610 2023-11-28T18:23:30.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa93f01640 1 -- 172.21.15.105:0/1331611080 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa84009b30 con 0x7faa8c159610 2023-11-28T18:23:30.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa93f01640 1 -- 172.21.15.105:0/1331611080 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa8c0ccff0 con 0x7faa8c159610 2023-11-28T18:23:30.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.292+0000 7faa93f01640 1 -- 172.21.15.105:0/1331611080 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa8c067570 con 0x7faa8c159610 2023-11-28T18:23:30.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.296+0000 7faa827fc640 1 -- 172.21.15.105:0/1331611080 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7faa840b8d00 con 0x7faa8c159610 2023-11-28T18:23:30.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.296+0000 7faa827fc640 1 --2- 172.21.15.105:0/1331611080 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faa70066900 0x7faa70068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:30.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.296+0000 7faa827fc640 1 -- 172.21.15.105:0/1331611080 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7faa8411ee20 con 0x7faa8c159610 2023-11-28T18:23:30.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.296+0000 7faa91c9d640 1 --2- 172.21.15.105:0/1331611080 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faa70066900 0x7faa70068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:30.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.296+0000 7faa91c9d640 1 --2- 172.21.15.105:0/1331611080 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faa70066900 0x7faa70068dc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faa88007fd0 tx=0x7faa88009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:30.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.300+0000 7faa827fc640 1 -- 172.21.15.105:0/1331611080 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faa840ec3c0 con 0x7faa8c159610 2023-11-28T18:23:30.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.448+0000 7faa93f01640 1 -- 172.21.15.105:0/1331611080 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa8c0cf950 con 0x7faa8c159610 2023-11-28T18:23:30.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.448+0000 7faa827fc640 1 -- 172.21.15.105:0/1331611080 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12910 (secure 0 0 0) 0x7faa840f4e60 con 0x7faa8c159610 2023-11-28T18:23:30.452 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:30.453 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":16,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:23:24.027932+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:23:30.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.452+0000 7faa93f01640 1 -- 172.21.15.105:0/1331611080 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faa70066900 msgr2=0x7faa70068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:30.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.452+0000 7faa93f01640 1 --2- 172.21.15.105:0/1331611080 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faa70066900 0x7faa70068dc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faa88007fd0 tx=0x7faa88009040 comp rx=0 tx=0).stop 2023-11-28T18:23:30.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.452+0000 7faa93f01640 1 -- 172.21.15.105:0/1331611080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa8c159610 msgr2=0x7faa8c152d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:30.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.452+0000 7faa93f01640 1 --2- 172.21.15.105:0/1331611080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa8c159610 0x7faa8c152d70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faa840af040 tx=0x7faa84019780 comp rx=0 tx=0).stop 2023-11-28T18:23:30.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.452+0000 7faa93f01640 1 -- 172.21.15.105:0/1331611080 shutdown_connections 2023-11-28T18:23:30.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.452+0000 7faa93f01640 1 --2- 172.21.15.105:0/1331611080 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faa70066900 0x7faa70068dc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.452+0000 7faa93f01640 1 --2- 172.21.15.105:0/1331611080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa8c159610 0x7faa8c152d70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.452+0000 7faa93f01640 1 --2- 172.21.15.105:0/1331611080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa8c0ce2c0 0x7faa8c158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.452+0000 7faa93f01640 1 --2- 172.21.15.105:0/1331611080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa8c06cd20 0x7faa8c147670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.452+0000 7faa93f01640 1 -- 172.21.15.105:0/1331611080 >> 172.21.15.105:0/1331611080 conn(0x7faa8c0ceb90 msgr2=0x7faa8c155410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:30.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.452+0000 7faa93f01640 1 -- 172.21.15.105:0/1331611080 shutdown_connections 2023-11-28T18:23:30.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.452+0000 7faa93f01640 1 -- 172.21.15.105:0/1331611080 wait complete. 2023-11-28T18:23:30.473 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:23:30.473 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:23:30.613 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.612+0000 7f5fa19f8640 1 Processor -- start 2023-11-28T18:23:30.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.612+0000 7f5fa19f8640 1 -- start start 2023-11-28T18:23:30.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.612+0000 7f5fa19f8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5f9c160680 0x7f5f9c0cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:30.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.612+0000 7f5fa19f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f9c0d1a80 0x7f5f9c0c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:30.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.612+0000 7f5fa19f8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f9c0ced00 con 0x7f5f9c160a60 2023-11-28T18:23:30.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.612+0000 7f5fa19f8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f9c0d38b0 con 0x7f5f9c160680 2023-11-28T18:23:30.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.612+0000 7f5fa19f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c0d3ac0 con 0x7f5f9c0d1a80 2023-11-28T18:23:30.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.612+0000 7f5f927fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5f9c160a60 0x7f5f9c0d1370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48130/0 (socket says 172.21.15.105:48130) 2023-11-28T18:23:30.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.612+0000 7f5f927fc640 1 -- 172.21.15.105:0/2797973283 learned_addr learned my addr 172.21.15.105:0/2797973283 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:30.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.612+0000 7f5f9affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5f9c160680 0x7f5f9c0cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:30.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.612+0000 7f5f9b7fe640 1 --2- 172.21.15.105:0/2797973283 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f9c0d1a80 0x7f5f9c0c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:30.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.612+0000 7f5f9affd640 1 -- 172.21.15.105:0/2797973283 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f9c0d1a80 msgr2=0x7f5f9c0c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:30.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.612+0000 7f5f9affd640 1 --2- 172.21.15.105:0/2797973283 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f9c0d1a80 0x7f5f9c0c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.612+0000 7f5f9affd640 1 -- 172.21.15.105:0/2797973283 >> v1:172.21.15.105:6789/0 conn(0x7f5f9c160a60 legacy=0x7f5f9c0d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:30.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.612+0000 7f5f9a7fc640 1 -- 172.21.15.105:0/2797973283 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3275785585 0 0) 0x7f5f9c0ced00 con 0x7f5f9c160a60 2023-11-28T18:23:30.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.612+0000 7f5f9affd640 1 -- 172.21.15.105:0/2797973283 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c0d0cf0 con 0x7f5f9c160680 2023-11-28T18:23:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.616+0000 7f5f9affd640 1 --2- 172.21.15.105:0/2797973283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f9c160680 0x7f5f9c0cfda0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0097c0 tx=0x7f5f8c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452e146d13c6fc11 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.616+0000 7f5f9a7fc640 1 -- 172.21.15.105:0/2797973283 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c0a6040 con 0x7f5f9c160680 2023-11-28T18:23:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.616+0000 7f5f9a7fc640 1 -- 172.21.15.105:0/2797973283 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8c0aa600 con 0x7f5f9c160680 2023-11-28T18:23:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.616+0000 7f5f9a7fc640 1 -- 172.21.15.105:0/2797973283 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c0aa990 con 0x7f5f9c160680 2023-11-28T18:23:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.616+0000 7f5fa19f8640 1 -- 172.21.15.105:0/2797973283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f9c160680 msgr2=0x7f5f9c0cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.616+0000 7f5fa19f8640 1 --2- 172.21.15.105:0/2797973283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f9c160680 0x7f5f9c0cfda0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0097c0 tx=0x7f5f8c01c480 comp rx=0 tx=0).stop 2023-11-28T18:23:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.616+0000 7f5fa19f8640 1 -- 172.21.15.105:0/2797973283 shutdown_connections 2023-11-28T18:23:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.616+0000 7f5fa19f8640 1 --2- 172.21.15.105:0/2797973283 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f9c0d1a80 0x7f5f9c0c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.616+0000 7f5fa19f8640 1 --2- 172.21.15.105:0/2797973283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f9c160680 0x7f5f9c0cfda0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.616+0000 7f5fa19f8640 1 -- 172.21.15.105:0/2797973283 >> 172.21.15.105:0/2797973283 conn(0x7f5f9c0d2d90 msgr2=0x7f5f9c0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:30.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.616+0000 7f5fa19f8640 1 -- 172.21.15.105:0/2797973283 shutdown_connections 2023-11-28T18:23:30.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.616+0000 7f5fa19f8640 1 -- 172.21.15.105:0/2797973283 wait complete. 2023-11-28T18:23:30.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5fa19f8640 1 Processor -- start 2023-11-28T18:23:30.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5fa19f8640 1 -- start start 2023-11-28T18:23:30.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5fa19f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f9c0d1a80 0x7f5f9c1554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:30.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5fa19f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f9c160680 0x7f5f9c0b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:30.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5fa19f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f9c160a60 0x7f5f9c0c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:30.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5fa19f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c156090 con 0x7f5f9c0d1a80 2023-11-28T18:23:30.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5fa19f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c0b6d70 con 0x7f5f9c160a60 2023-11-28T18:23:30.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5fa19f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c0b7200 con 0x7f5f9c160680 2023-11-28T18:23:30.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5f9affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f9c0d1a80 0x7f5f9c1554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:30.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5f9affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f9c0d1a80 0x7f5f9c1554d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54952/0 (socket says 172.21.15.105:54952) 2023-11-28T18:23:30.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5f9affd640 1 -- 172.21.15.105:0/4186449170 learned_addr learned my addr 172.21.15.105:0/4186449170 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:30.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5f9affd640 1 -- 172.21.15.105:0/4186449170 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f9c160680 msgr2=0x7f5f9c0b5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:30.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5f9affd640 1 --2- 172.21.15.105:0/4186449170 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f9c160680 0x7f5f9c0b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5f9affd640 1 -- 172.21.15.105:0/4186449170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f9c160a60 msgr2=0x7f5f9c0c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:30.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5f9affd640 1 --2- 172.21.15.105:0/4186449170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f9c160a60 0x7f5f9c0c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5f9affd640 1 -- 172.21.15.105:0/4186449170 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8c009420 con 0x7f5f9c0d1a80 2023-11-28T18:23:30.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5f9affd640 1 --2- 172.21.15.105:0/4186449170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f9c0d1a80 0x7f5f9c1554d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c01a0b0 tx=0x7f5f8c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:30.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5f98ff9640 1 -- 172.21.15.105:0/4186449170 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c006e20 con 0x7f5f9c0d1a80 2023-11-28T18:23:30.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5f98ff9640 1 -- 172.21.15.105:0/4186449170 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8c0b1950 con 0x7f5f9c0d1a80 2023-11-28T18:23:30.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5f98ff9640 1 -- 172.21.15.105:0/4186449170 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c0b1d20 con 0x7f5f9c0d1a80 2023-11-28T18:23:30.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5fa19f8640 1 -- 172.21.15.105:0/4186449170 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9c0c7b40 con 0x7f5f9c0d1a80 2023-11-28T18:23:30.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5fa19f8640 1 -- 172.21.15.105:0/4186449170 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9c0b6720 con 0x7f5f9c0d1a80 2023-11-28T18:23:30.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.620+0000 7f5fa19f8640 1 -- 172.21.15.105:0/4186449170 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9c074580 con 0x7f5f9c0d1a80 2023-11-28T18:23:30.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.624+0000 7f5f98ff9640 1 -- 172.21.15.105:0/4186449170 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5f8c0b33f0 con 0x7f5f9c0d1a80 2023-11-28T18:23:30.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.624+0000 7f5f98ff9640 1 --2- 172.21.15.105:0/4186449170 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f7c0668d0 0x7f5f7c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:30.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.624+0000 7f5f98ff9640 1 -- 172.21.15.105:0/4186449170 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5f8c11e600 con 0x7f5f9c0d1a80 2023-11-28T18:23:30.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.624+0000 7f5f927fc640 1 --2- 172.21.15.105:0/4186449170 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f7c0668d0 0x7f5f7c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:30.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.624+0000 7f5f927fc640 1 --2- 172.21.15.105:0/4186449170 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f7c0668d0 0x7f5f7c068d90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0a0d30 tx=0x7f5f84007cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:30.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.628+0000 7f5f98ff9640 1 -- 172.21.15.105:0/4186449170 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5f8c0ebcd0 con 0x7f5f9c0d1a80 2023-11-28T18:23:30.780 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.776+0000 7f5fa19f8640 1 -- 172.21.15.105:0/4186449170 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f9c0cd270 con 0x7f5f9c0d1a80 2023-11-28T18:23:30.780 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.776+0000 7f5f98ff9640 1 -- 172.21.15.105:0/4186449170 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f8c0c30a0 con 0x7f5f9c0d1a80 2023-11-28T18:23:30.780 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:23:30.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.780+0000 7f5fa19f8640 1 -- 172.21.15.105:0/4186449170 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f7c0668d0 msgr2=0x7f5f7c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:30.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.780+0000 7f5fa19f8640 1 --2- 172.21.15.105:0/4186449170 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f7c0668d0 0x7f5f7c068d90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0a0d30 tx=0x7f5f84007cf0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.780+0000 7f5fa19f8640 1 -- 172.21.15.105:0/4186449170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f9c0d1a80 msgr2=0x7f5f9c1554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:30.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.780+0000 7f5fa19f8640 1 --2- 172.21.15.105:0/4186449170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f9c0d1a80 0x7f5f9c1554d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c01a0b0 tx=0x7f5f8c01acc0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.780+0000 7f5fa19f8640 1 -- 172.21.15.105:0/4186449170 shutdown_connections 2023-11-28T18:23:30.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.780+0000 7f5fa19f8640 1 --2- 172.21.15.105:0/4186449170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f9c160a60 0x7f5f9c0c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.780+0000 7f5fa19f8640 1 --2- 172.21.15.105:0/4186449170 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f9c160680 0x7f5f9c0b5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.780+0000 7f5fa19f8640 1 --2- 172.21.15.105:0/4186449170 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f7c0668d0 0x7f5f7c068d90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.780+0000 7f5fa19f8640 1 --2- 172.21.15.105:0/4186449170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f9c0d1a80 0x7f5f9c1554d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.780+0000 7f5fa19f8640 1 -- 172.21.15.105:0/4186449170 >> 172.21.15.105:0/4186449170 conn(0x7f5f9c0d2d90 msgr2=0x7f5f9c0d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:30.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.780+0000 7f5fa19f8640 1 -- 172.21.15.105:0/4186449170 shutdown_connections 2023-11-28T18:23:30.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.780+0000 7f5fa19f8640 1 -- 172.21.15.105:0/4186449170 wait complete. 2023-11-28T18:23:30.801 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T18:23:30.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.936+0000 7ff9cadaa640 1 Processor -- start 2023-11-28T18:23:30.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.940+0000 7ff9cadaa640 1 -- start start 2023-11-28T18:23:30.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.940+0000 7ff9cadaa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff9c4071960 0x7ff9c415f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.940+0000 7ff9cadaa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff9c415b030 0x7ff9c40709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.940+0000 7ff9cadaa640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9c40701d0 con 0x7ff9c40b4290 2023-11-28T18:23:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.940+0000 7ff9cadaa640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9c406df80 con 0x7ff9c4071960 2023-11-28T18:23:30.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.940+0000 7ff9cadaa640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9c4071df0 con 0x7ff9c415b030 2023-11-28T18:23:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.940+0000 7ff9c8b46640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff9c40b4290 0x7ff9c40b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48134/0 (socket says 172.21.15.105:48134) 2023-11-28T18:23:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.940+0000 7ff9c8b46640 1 -- 172.21.15.105:0/2164818842 learned_addr learned my addr 172.21.15.105:0/2164818842 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.940+0000 7ff9bbfff640 1 --2- 172.21.15.105:0/2164818842 >> v2:172.21.15.105:3300/0 conn(0x7ff9c4071960 0x7ff9c415f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.940+0000 7ff9c9347640 1 --2- 172.21.15.105:0/2164818842 >> v2:172.21.15.148:3300/0 conn(0x7ff9c415b030 0x7ff9c40709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.940+0000 7ff9bb7fe640 1 -- 172.21.15.105:0/2164818842 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987321449 0 0) 0x7ff9c40701d0 con 0x7ff9c40b4290 2023-11-28T18:23:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.940+0000 7ff9bb7fe640 1 -- 172.21.15.105:0/2164818842 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9ac003530 con 0x7ff9c40b4290 2023-11-28T18:23:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.940+0000 7ff9bbfff640 1 -- 172.21.15.105:0/2164818842 >> v2:172.21.15.148:3300/0 conn(0x7ff9c415b030 msgr2=0x7ff9c40709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.940+0000 7ff9bbfff640 1 --2- 172.21.15.105:0/2164818842 >> v2:172.21.15.148:3300/0 conn(0x7ff9c415b030 0x7ff9c40709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.940+0000 7ff9bbfff640 1 -- 172.21.15.105:0/2164818842 >> v1:172.21.15.105:6789/0 conn(0x7ff9c40b4290 legacy=0x7ff9c40b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.940+0000 7ff9bbfff640 1 -- 172.21.15.105:0/2164818842 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c40720d0 con 0x7ff9c4071960 2023-11-28T18:23:30.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.944+0000 7ff9bbfff640 1 --2- 172.21.15.105:0/2164818842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9c4071960 0x7ff9c415f3b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4067c50 tx=0x7ff9b401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec7b50ed763d438 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:30.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.944+0000 7ff9bb7fe640 1 -- 172.21.15.105:0/2164818842 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b40ab600 con 0x7ff9c4071960 2023-11-28T18:23:30.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.944+0000 7ff9bb7fe640 1 -- 172.21.15.105:0/2164818842 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b40a7050 con 0x7ff9c4071960 2023-11-28T18:23:30.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.944+0000 7ff9bb7fe640 1 -- 172.21.15.105:0/2164818842 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b40ab960 con 0x7ff9c4071960 2023-11-28T18:23:30.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.944+0000 7ff9cadaa640 1 -- 172.21.15.105:0/2164818842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9c4071960 msgr2=0x7ff9c415f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:30.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.944+0000 7ff9cadaa640 1 --2- 172.21.15.105:0/2164818842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9c4071960 0x7ff9c415f3b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4067c50 tx=0x7ff9b401c430 comp rx=0 tx=0).stop 2023-11-28T18:23:30.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.944+0000 7ff9cadaa640 1 -- 172.21.15.105:0/2164818842 shutdown_connections 2023-11-28T18:23:30.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.944+0000 7ff9cadaa640 1 --2- 172.21.15.105:0/2164818842 >> v2:172.21.15.148:3300/0 conn(0x7ff9c415b030 0x7ff9c40709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.944+0000 7ff9cadaa640 1 --2- 172.21.15.105:0/2164818842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9c4071960 0x7ff9c415f3b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.944+0000 7ff9cadaa640 1 -- 172.21.15.105:0/2164818842 >> 172.21.15.105:0/2164818842 conn(0x7ff9c40ceb90 msgr2=0x7ff9c40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:30.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.944+0000 7ff9cadaa640 1 -- 172.21.15.105:0/2164818842 shutdown_connections 2023-11-28T18:23:30.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.944+0000 7ff9cadaa640 1 -- 172.21.15.105:0/2164818842 wait complete. 2023-11-28T18:23:30.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.944+0000 7ff9cadaa640 1 Processor -- start 2023-11-28T18:23:30.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.944+0000 7ff9cadaa640 1 -- start start 2023-11-28T18:23:30.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9cadaa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9c40b4290 0x7ff9c40cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:30.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9cadaa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9c415b030 0x7ff9c41431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:30.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9cadaa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9c4143700 0x7ff9c40cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:30.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9cadaa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c40ce830 con 0x7ff9c415b030 2023-11-28T18:23:30.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9cadaa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c4143b90 con 0x7ff9c40b4290 2023-11-28T18:23:30.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9cadaa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c4152850 con 0x7ff9c4143700 2023-11-28T18:23:30.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9bbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9c415b030 0x7ff9c41431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:30.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9bbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9c415b030 0x7ff9c41431c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54968/0 (socket says 172.21.15.105:54968) 2023-11-28T18:23:30.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9bbfff640 1 -- 172.21.15.105:0/2725605356 learned_addr learned my addr 172.21.15.105:0/2725605356 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:30.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9c8b46640 1 --2- 172.21.15.105:0/2725605356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9c40b4290 0x7ff9c40cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:30.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9c9347640 1 --2- 172.21.15.105:0/2725605356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9c4143700 0x7ff9c40cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:30.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9bbfff640 1 -- 172.21.15.105:0/2725605356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9c4143700 msgr2=0x7ff9c40cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:30.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9bbfff640 1 --2- 172.21.15.105:0/2725605356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9c4143700 0x7ff9c40cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9bbfff640 1 -- 172.21.15.105:0/2725605356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9c40b4290 msgr2=0x7ff9c40cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:30.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9bbfff640 1 --2- 172.21.15.105:0/2725605356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9c40b4290 0x7ff9c40cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:30.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9bbfff640 1 -- 172.21.15.105:0/2725605356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b4009420 con 0x7ff9c415b030 2023-11-28T18:23:30.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9bbfff640 1 --2- 172.21.15.105:0/2725605356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9c415b030 0x7ff9c41431c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4097e10 tx=0x7ff9b40191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:30.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9b97fa640 1 -- 172.21.15.105:0/2725605356 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b40b4400 con 0x7ff9c415b030 2023-11-28T18:23:30.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9b97fa640 1 -- 172.21.15.105:0/2725605356 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b40b4560 con 0x7ff9c415b030 2023-11-28T18:23:30.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9cadaa640 1 -- 172.21.15.105:0/2725605356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c40cd3e0 con 0x7ff9c415b030 2023-11-28T18:23:30.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9b97fa640 1 -- 172.21.15.105:0/2725605356 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b40b48f0 con 0x7ff9c415b030 2023-11-28T18:23:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9cadaa640 1 -- 172.21.15.105:0/2725605356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c4152aa0 con 0x7ff9c415b030 2023-11-28T18:23:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.948+0000 7ff9cadaa640 1 -- 172.21.15.105:0/2725605356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c4067570 con 0x7ff9c415b030 2023-11-28T18:23:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.952+0000 7ff9b97fa640 1 -- 172.21.15.105:0/2725605356 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff9b40b2040 con 0x7ff9c415b030 2023-11-28T18:23:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.952+0000 7ff9b97fa640 1 --2- 172.21.15.105:0/2725605356 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff9b00668d0 0x7ff9b0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.952+0000 7ff9b97fa640 1 -- 172.21.15.105:0/2725605356 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff9b411de60 con 0x7ff9c415b030 2023-11-28T18:23:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.952+0000 7ff9c8b46640 1 --2- 172.21.15.105:0/2725605356 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff9b00668d0 0x7ff9b0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.952+0000 7ff9c8b46640 1 --2- 172.21.15.105:0/2725605356 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff9b00668d0 0x7ff9b0068d90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0007fd0 tx=0x7ff9c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:30.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:30.956+0000 7ff9b97fa640 1 -- 172.21.15.105:0/2725605356 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff9b40eb400 con 0x7ff9c415b030 2023-11-28T18:23:30.998 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:23:31.002 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:23:31.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.108+0000 7ff9cadaa640 1 -- 172.21.15.105:0/2725605356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff9c40cf7f0 con 0x7ff9c415b030 2023-11-28T18:23:31.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.108+0000 7ff9b97fa640 1 -- 172.21.15.105:0/2725605356 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff9b40b0090 con 0x7ff9c415b030 2023-11-28T18:23:31.112 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:31.112 INFO:teuthology.orchestra.run.smithi105.stdout:[ 2023-11-28T18:23:31.112 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:23:31.113 INFO:teuthology.orchestra.run.smithi105.stdout: "poolnum": 1, 2023-11-28T18:23:31.113 INFO:teuthology.orchestra.run.smithi105.stdout: "poolname": ".mgr" 2023-11-28T18:23:31.113 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:23:31.113 INFO:teuthology.orchestra.run.smithi105.stdout:] 2023-11-28T18:23:31.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.112+0000 7ff9cadaa640 1 -- 172.21.15.105:0/2725605356 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff9b00668d0 msgr2=0x7ff9b0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:31.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.112+0000 7ff9cadaa640 1 --2- 172.21.15.105:0/2725605356 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff9b00668d0 0x7ff9b0068d90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0007fd0 tx=0x7ff9c0009040 comp rx=0 tx=0).stop 2023-11-28T18:23:31.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.112+0000 7ff9cadaa640 1 -- 172.21.15.105:0/2725605356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9c415b030 msgr2=0x7ff9c41431c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:31.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.112+0000 7ff9cadaa640 1 --2- 172.21.15.105:0/2725605356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9c415b030 0x7ff9c41431c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4097e10 tx=0x7ff9b40191d0 comp rx=0 tx=0).stop 2023-11-28T18:23:31.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.112+0000 7ff9cadaa640 1 -- 172.21.15.105:0/2725605356 shutdown_connections 2023-11-28T18:23:31.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.112+0000 7ff9cadaa640 1 --2- 172.21.15.105:0/2725605356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9c4143700 0x7ff9c40cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:31.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.112+0000 7ff9cadaa640 1 --2- 172.21.15.105:0/2725605356 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff9b00668d0 0x7ff9b0068d90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:31.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.112+0000 7ff9cadaa640 1 --2- 172.21.15.105:0/2725605356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9c415b030 0x7ff9c41431c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:31.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.112+0000 7ff9cadaa640 1 --2- 172.21.15.105:0/2725605356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9c40b4290 0x7ff9c40cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:31.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.112+0000 7ff9cadaa640 1 -- 172.21.15.105:0/2725605356 >> 172.21.15.105:0/2725605356 conn(0x7ff9c40ceb90 msgr2=0x7ff9c40b4670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:31.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.112+0000 7ff9cadaa640 1 -- 172.21.15.105:0/2725605356 shutdown_connections 2023-11-28T18:23:31.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.112+0000 7ff9cadaa640 1 -- 172.21.15.105:0/2725605356 wait complete. 2023-11-28T18:23:31.133 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T18:23:31.133 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T18:23:31.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.272+0000 7ff55ac09640 1 Processor -- start 2023-11-28T18:23:31.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.272+0000 7ff55ac09640 1 -- start start 2023-11-28T18:23:31.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.272+0000 7ff55ac09640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff5540b4290 0x7ff5540b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:31.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.272+0000 7ff55ac09640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff554071960 0x7ff55415f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:31.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.272+0000 7ff55ac09640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5540b4bb0 con 0x7ff55415b030 2023-11-28T18:23:31.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.272+0000 7ff55ac09640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5540701d0 con 0x7ff554071960 2023-11-28T18:23:31.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.272+0000 7ff55ac09640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff55406df80 con 0x7ff5540b4290 2023-11-28T18:23:31.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.272+0000 7ff54bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff554071960 0x7ff55415f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:31.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.272+0000 7ff5591a6640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff55415b030 0x7ff5540709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51398/0 (socket says 172.21.15.105:51398) 2023-11-28T18:23:31.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.276+0000 7ff5591a6640 1 -- 172.21.15.105:0/3533101639 learned_addr learned my addr 172.21.15.105:0/3533101639 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:31.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.276+0000 7ff5589a5640 1 --2- 172.21.15.105:0/3533101639 >> v2:172.21.15.148:3300/0 conn(0x7ff5540b4290 0x7ff5540b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:31.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.276+0000 7ff54bfff640 1 -- 172.21.15.105:0/3533101639 >> v2:172.21.15.148:3300/0 conn(0x7ff5540b4290 msgr2=0x7ff5540b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:31.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.276+0000 7ff54bfff640 1 --2- 172.21.15.105:0/3533101639 >> v2:172.21.15.148:3300/0 conn(0x7ff5540b4290 0x7ff5540b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:31.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.276+0000 7ff54bfff640 1 -- 172.21.15.105:0/3533101639 >> v1:172.21.15.105:6789/0 conn(0x7ff55415b030 legacy=0x7ff5540709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:31.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.276+0000 7ff54bfff640 1 -- 172.21.15.105:0/3533101639 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff554071df0 con 0x7ff554071960 2023-11-28T18:23:31.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.276+0000 7ff5589a5640 1 --2- 172.21.15.105:0/3533101639 >> v2:172.21.15.148:3300/0 conn(0x7ff5540b4290 0x7ff5540b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:31.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.276+0000 7ff54bfff640 1 --2- 172.21.15.105:0/3533101639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff554071960 0x7ff55415f3b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff554073a70 tx=0x7ff54401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2318a3e040fba60c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:31.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.276+0000 7ff54b7fe640 1 -- 172.21.15.105:0/3533101639 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5440a7070 con 0x7ff554071960 2023-11-28T18:23:31.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.276+0000 7ff54b7fe640 1 -- 172.21.15.105:0/3533101639 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5440ab430 con 0x7ff554071960 2023-11-28T18:23:31.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.276+0000 7ff54b7fe640 1 -- 172.21.15.105:0/3533101639 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5440ab7c0 con 0x7ff554071960 2023-11-28T18:23:31.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff55ac09640 1 -- 172.21.15.105:0/3533101639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff554071960 msgr2=0x7ff55415f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:31.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff55ac09640 1 --2- 172.21.15.105:0/3533101639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff554071960 0x7ff55415f3b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff554073a70 tx=0x7ff54401c430 comp rx=0 tx=0).stop 2023-11-28T18:23:31.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff55ac09640 1 -- 172.21.15.105:0/3533101639 shutdown_connections 2023-11-28T18:23:31.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff55ac09640 1 --2- 172.21.15.105:0/3533101639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff554071960 0x7ff55415f3b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:31.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff55ac09640 1 --2- 172.21.15.105:0/3533101639 >> v2:172.21.15.148:3300/0 conn(0x7ff5540b4290 0x7ff5540b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff55ac09640 1 -- 172.21.15.105:0/3533101639 >> 172.21.15.105:0/3533101639 conn(0x7ff5540ceb90 msgr2=0x7ff5540cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff55ac09640 1 -- 172.21.15.105:0/3533101639 shutdown_connections 2023-11-28T18:23:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff55ac09640 1 -- 172.21.15.105:0/3533101639 wait complete. 2023-11-28T18:23:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff55ac09640 1 Processor -- start 2023-11-28T18:23:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff55ac09640 1 -- start start 2023-11-28T18:23:31.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff55ac09640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff5540b4290 0x7ff5540cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:31.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff55ac09640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff55415b030 0x7ff554165580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:31.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff55ac09640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff554151df0 0x7ff554156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:31.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff55ac09640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5541f60c0 con 0x7ff55415b030 2023-11-28T18:23:31.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff55ac09640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff554152920 con 0x7ff554151df0 2023-11-28T18:23:31.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff55ac09640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff55409bf20 con 0x7ff5540b4290 2023-11-28T18:23:31.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff54bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff55415b030 0x7ff554165580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:31.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff54bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff55415b030 0x7ff554165580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59016/0 (socket says 172.21.15.105:59016) 2023-11-28T18:23:31.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff54bfff640 1 -- 172.21.15.105:0/2088935583 learned_addr learned my addr 172.21.15.105:0/2088935583 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:31.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff5589a5640 1 --2- 172.21.15.105:0/2088935583 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff5540b4290 0x7ff5540cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:31.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff5591a6640 1 --2- 172.21.15.105:0/2088935583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff554151df0 0x7ff554156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:31.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff54bfff640 1 -- 172.21.15.105:0/2088935583 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff5540b4290 msgr2=0x7ff5540cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:31.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff54bfff640 1 --2- 172.21.15.105:0/2088935583 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff5540b4290 0x7ff5540cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:31.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff54bfff640 1 -- 172.21.15.105:0/2088935583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff554151df0 msgr2=0x7ff554156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:31.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff54bfff640 1 --2- 172.21.15.105:0/2088935583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff554151df0 0x7ff554156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:31.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff54bfff640 1 -- 172.21.15.105:0/2088935583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff544009420 con 0x7ff55415b030 2023-11-28T18:23:31.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.280+0000 7ff5589a5640 1 --2- 172.21.15.105:0/2088935583 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff5540b4290 0x7ff5540cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:31.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.284+0000 7ff54bfff640 1 --2- 172.21.15.105:0/2088935583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff55415b030 0x7ff554165580 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff544019040 tx=0x7ff54401ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:31.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.284+0000 7ff5497fa640 1 -- 172.21.15.105:0/2088935583 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5440b5810 con 0x7ff55415b030 2023-11-28T18:23:31.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.284+0000 7ff5497fa640 1 -- 172.21.15.105:0/2088935583 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5440b5990 con 0x7ff55415b030 2023-11-28T18:23:31.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.284+0000 7ff55ac09640 1 -- 172.21.15.105:0/2088935583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55409c200 con 0x7ff55415b030 2023-11-28T18:23:31.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.284+0000 7ff5497fa640 1 -- 172.21.15.105:0/2088935583 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5440b5d40 con 0x7ff55415b030 2023-11-28T18:23:31.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.284+0000 7ff55ac09640 1 -- 172.21.15.105:0/2088935583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5540ccca0 con 0x7ff55415b030 2023-11-28T18:23:31.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.284+0000 7ff55ac09640 1 -- 172.21.15.105:0/2088935583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5541f6200 con 0x7ff55415b030 2023-11-28T18:23:31.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.284+0000 7ff5497fa640 1 -- 172.21.15.105:0/2088935583 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff5440c7390 con 0x7ff55415b030 2023-11-28T18:23:31.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.284+0000 7ff5497fa640 1 --2- 172.21.15.105:0/2088935583 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff5400668d0 0x7ff540068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:31.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.284+0000 7ff5497fa640 1 -- 172.21.15.105:0/2088935583 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff54411eea0 con 0x7ff55415b030 2023-11-28T18:23:31.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.284+0000 7ff5589a5640 1 --2- 172.21.15.105:0/2088935583 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff5400668d0 0x7ff540068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:31.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.288+0000 7ff5589a5640 1 --2- 172.21.15.105:0/2088935583 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff5400668d0 0x7ff540068d90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff550006fd0 tx=0x7ff550008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:31.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.288+0000 7ff5497fa640 1 -- 172.21.15.105:0/2088935583 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff5541f6200 con 0x7ff55415b030 2023-11-28T18:23:31.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.440+0000 7ff55ac09640 1 -- 172.21.15.105:0/2088935583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff5541f6200 con 0x7ff55415b030 2023-11-28T18:23:31.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.888+0000 7ff5497fa640 1 -- 172.21.15.105:0/2088935583 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff5541f6200 con 0x7ff55415b030 2023-11-28T18:23:31.892 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' created 2023-11-28T18:23:31.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.892+0000 7ff55ac09640 1 -- 172.21.15.105:0/2088935583 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff5400668d0 msgr2=0x7ff540068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:31.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.892+0000 7ff55ac09640 1 --2- 172.21.15.105:0/2088935583 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff5400668d0 0x7ff540068d90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff550006fd0 tx=0x7ff550008040 comp rx=0 tx=0).stop 2023-11-28T18:23:31.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.892+0000 7ff55ac09640 1 -- 172.21.15.105:0/2088935583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff55415b030 msgr2=0x7ff554165580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:31.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.892+0000 7ff55ac09640 1 --2- 172.21.15.105:0/2088935583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff55415b030 0x7ff554165580 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff544019040 tx=0x7ff54401ac70 comp rx=0 tx=0).stop 2023-11-28T18:23:31.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.892+0000 7ff55ac09640 1 -- 172.21.15.105:0/2088935583 shutdown_connections 2023-11-28T18:23:31.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.892+0000 7ff55ac09640 1 --2- 172.21.15.105:0/2088935583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff554151df0 0x7ff554156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:31.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.892+0000 7ff55ac09640 1 --2- 172.21.15.105:0/2088935583 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff5400668d0 0x7ff540068d90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:31.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.892+0000 7ff55ac09640 1 --2- 172.21.15.105:0/2088935583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff55415b030 0x7ff554165580 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:31.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.892+0000 7ff55ac09640 1 --2- 172.21.15.105:0/2088935583 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff5540b4290 0x7ff5540cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:31.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.892+0000 7ff55ac09640 1 -- 172.21.15.105:0/2088935583 >> 172.21.15.105:0/2088935583 conn(0x7ff5540ceb90 msgr2=0x7ff5540b39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:31.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.892+0000 7ff55ac09640 1 -- 172.21.15.105:0/2088935583 shutdown_connections 2023-11-28T18:23:31.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:31.892+0000 7ff55ac09640 1 -- 172.21.15.105:0/2088935583 wait complete. 2023-11-28T18:23:31.920 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-11-28T18:23:32.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.060+0000 7fe3d451d640 1 Processor -- start 2023-11-28T18:23:32.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.060+0000 7fe3d451d640 1 -- start start 2023-11-28T18:23:32.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.064+0000 7fe3d451d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe3cc15d3c0 0x7fe3cc071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:32.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.064+0000 7fe3d451d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe3cc161870 0x7fe3cc0ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:32.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.064+0000 7fe3d451d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3cc0cefa0 con 0x7fe3cc070a10 2023-11-28T18:23:32.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.064+0000 7fe3d451d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3cc15d7a0 con 0x7fe3cc15d3c0 2023-11-28T18:23:32.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.064+0000 7fe3d451d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3cc0ce9e0 con 0x7fe3cc161870 2023-11-28T18:23:32.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.064+0000 7fe3d2aba640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe3cc070a10 0x7fe3cc070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51408/0 (socket says 172.21.15.105:51408) 2023-11-28T18:23:32.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.064+0000 7fe3d2aba640 1 -- 172.21.15.105:0/2611410280 learned_addr learned my addr 172.21.15.105:0/2611410280 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:32.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.064+0000 7fe3d22b9640 1 --2- 172.21.15.105:0/2611410280 >> v2:172.21.15.105:3300/0 conn(0x7fe3cc15d3c0 0x7fe3cc071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:32.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.064+0000 7fe3d1ab8640 1 --2- 172.21.15.105:0/2611410280 >> v2:172.21.15.148:3300/0 conn(0x7fe3cc161870 0x7fe3cc0ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:32.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.064+0000 7fe3d12b7640 1 -- 172.21.15.105:0/2611410280 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3420731350 0 0) 0x7fe3cc0cefa0 con 0x7fe3cc070a10 2023-11-28T18:23:32.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.064+0000 7fe3d12b7640 1 -- 172.21.15.105:0/2611410280 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3b8003530 con 0x7fe3cc070a10 2023-11-28T18:23:32.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.064+0000 7fe3d22b9640 1 -- 172.21.15.105:0/2611410280 >> v2:172.21.15.148:3300/0 conn(0x7fe3cc161870 msgr2=0x7fe3cc0ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:32.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.064+0000 7fe3d22b9640 1 --2- 172.21.15.105:0/2611410280 >> v2:172.21.15.148:3300/0 conn(0x7fe3cc161870 0x7fe3cc0ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:32.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.064+0000 7fe3d22b9640 1 -- 172.21.15.105:0/2611410280 >> v1:172.21.15.105:6789/0 conn(0x7fe3cc070a10 legacy=0x7fe3cc070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:32.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.064+0000 7fe3d22b9640 1 -- 172.21.15.105:0/2611410280 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc0cecc0 con 0x7fe3cc15d3c0 2023-11-28T18:23:32.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.068+0000 7fe3d22b9640 1 --2- 172.21.15.105:0/2611410280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3cc15d3c0 0x7fe3cc071a10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc160e30 tx=0x7fe3c801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293160baacc612f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:32.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.068+0000 7fe3d12b7640 1 -- 172.21.15.105:0/2611410280 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80a7070 con 0x7fe3cc15d3c0 2023-11-28T18:23:32.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.068+0000 7fe3d12b7640 1 -- 172.21.15.105:0/2611410280 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c80ab430 con 0x7fe3cc15d3c0 2023-11-28T18:23:32.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.068+0000 7fe3d12b7640 1 -- 172.21.15.105:0/2611410280 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80ab7a0 con 0x7fe3cc15d3c0 2023-11-28T18:23:32.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.068+0000 7fe3d451d640 1 -- 172.21.15.105:0/2611410280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3cc15d3c0 msgr2=0x7fe3cc071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:32.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.068+0000 7fe3d451d640 1 --2- 172.21.15.105:0/2611410280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3cc15d3c0 0x7fe3cc071a10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc160e30 tx=0x7fe3c801c430 comp rx=0 tx=0).stop 2023-11-28T18:23:32.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.068+0000 7fe3d451d640 1 -- 172.21.15.105:0/2611410280 shutdown_connections 2023-11-28T18:23:32.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.068+0000 7fe3d451d640 1 --2- 172.21.15.105:0/2611410280 >> v2:172.21.15.148:3300/0 conn(0x7fe3cc161870 0x7fe3cc0ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:32.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.068+0000 7fe3d451d640 1 --2- 172.21.15.105:0/2611410280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3cc15d3c0 0x7fe3cc071a10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:32.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.068+0000 7fe3d451d640 1 -- 172.21.15.105:0/2611410280 >> 172.21.15.105:0/2611410280 conn(0x7fe3cc0cf540 msgr2=0x7fe3cc0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:32.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.068+0000 7fe3d451d640 1 -- 172.21.15.105:0/2611410280 shutdown_connections 2023-11-28T18:23:32.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.068+0000 7fe3d451d640 1 -- 172.21.15.105:0/2611410280 wait complete. 2023-11-28T18:23:32.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.068+0000 7fe3d451d640 1 Processor -- start 2023-11-28T18:23:32.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.068+0000 7fe3d451d640 1 -- start start 2023-11-28T18:23:32.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3d451d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3cc070a10 0x7fe3cc154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:32.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3d451d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3cc15d3c0 0x7fe3cc150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:32.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3d451d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3cc161870 0x7fe3cc150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:32.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3d451d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc14f7c0 con 0x7fe3cc161870 2023-11-28T18:23:32.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3d451d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc14f290 con 0x7fe3cc070a10 2023-11-28T18:23:32.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3d451d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc14e0a0 con 0x7fe3cc15d3c0 2023-11-28T18:23:32.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3d2aba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3cc161870 0x7fe3cc150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:32.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3d2aba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3cc161870 0x7fe3cc150fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59034/0 (socket says 172.21.15.105:59034) 2023-11-28T18:23:32.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3d2aba640 1 -- 172.21.15.105:0/3728044434 learned_addr learned my addr 172.21.15.105:0/3728044434 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:32.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3d22b9640 1 --2- 172.21.15.105:0/3728044434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3cc070a10 0x7fe3cc154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:32.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3d1ab8640 1 --2- 172.21.15.105:0/3728044434 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3cc15d3c0 0x7fe3cc150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:32.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3d2aba640 1 -- 172.21.15.105:0/3728044434 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3cc15d3c0 msgr2=0x7fe3cc150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:32.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3d2aba640 1 --2- 172.21.15.105:0/3728044434 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3cc15d3c0 0x7fe3cc150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:32.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3d2aba640 1 -- 172.21.15.105:0/3728044434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3cc070a10 msgr2=0x7fe3cc154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:32.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3d2aba640 1 --2- 172.21.15.105:0/3728044434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3cc070a10 0x7fe3cc154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:32.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3d2aba640 1 -- 172.21.15.105:0/3728044434 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c8009420 con 0x7fe3cc161870 2023-11-28T18:23:32.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3d2aba640 1 --2- 172.21.15.105:0/3728044434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3cc161870 0x7fe3cc150fb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400cb30 tx=0x7fe3c400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:32.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3c2ffd640 1 -- 172.21.15.105:0/3728044434 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c400aca0 con 0x7fe3cc161870 2023-11-28T18:23:32.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3c2ffd640 1 -- 172.21.15.105:0/3728044434 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c400ae00 con 0x7fe3cc161870 2023-11-28T18:23:32.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3d451d640 1 -- 172.21.15.105:0/3728044434 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3cc14f4e0 con 0x7fe3cc161870 2023-11-28T18:23:32.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3c2ffd640 1 -- 172.21.15.105:0/3728044434 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4012600 con 0x7fe3cc161870 2023-11-28T18:23:32.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3d451d640 1 -- 172.21.15.105:0/3728044434 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3cc145e50 con 0x7fe3cc161870 2023-11-28T18:23:32.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.072+0000 7fe3d451d640 1 -- 172.21.15.105:0/3728044434 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3cc14df00 con 0x7fe3cc161870 2023-11-28T18:23:32.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.076+0000 7fe3c2ffd640 1 -- 172.21.15.105:0/3728044434 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fe3c4012760 con 0x7fe3cc161870 2023-11-28T18:23:32.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.076+0000 7fe3c2ffd640 1 --2- 172.21.15.105:0/3728044434 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3b0066900 0x7fe3b0068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:32.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.076+0000 7fe3c2ffd640 1 -- 172.21.15.105:0/3728044434 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fe3c4084820 con 0x7fe3cc161870 2023-11-28T18:23:32.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.076+0000 7fe3d22b9640 1 --2- 172.21.15.105:0/3728044434 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3b0066900 0x7fe3b0068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:32.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.076+0000 7fe3d22b9640 1 --2- 172.21.15.105:0/3728044434 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3b0066900 0x7fe3b0068dc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80af000 tx=0x7fe3c80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:32.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.080+0000 7fe3c2ffd640 1 -- 172.21.15.105:0/3728044434 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe3c4051cd0 con 0x7fe3cc161870 2023-11-28T18:23:32.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.228+0000 7fe3d451d640 1 -- 172.21.15.105:0/3728044434 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fe3cc14e580 con 0x7fe3cc161870 2023-11-28T18:23:32.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.892+0000 7fe3c2ffd640 1 -- 172.21.15.105:0/3728044434 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7fe3cc14e580 con 0x7fe3cc161870 2023-11-28T18:23:32.896 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' created 2023-11-28T18:23:32.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.896+0000 7fe3d451d640 1 -- 172.21.15.105:0/3728044434 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3b0066900 msgr2=0x7fe3b0068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:32.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.896+0000 7fe3d451d640 1 --2- 172.21.15.105:0/3728044434 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3b0066900 0x7fe3b0068dc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80af000 tx=0x7fe3c80a7000 comp rx=0 tx=0).stop 2023-11-28T18:23:32.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.896+0000 7fe3d451d640 1 -- 172.21.15.105:0/3728044434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3cc161870 msgr2=0x7fe3cc150fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:32.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.896+0000 7fe3d451d640 1 --2- 172.21.15.105:0/3728044434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3cc161870 0x7fe3cc150fb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400cb30 tx=0x7fe3c400a060 comp rx=0 tx=0).stop 2023-11-28T18:23:32.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.896+0000 7fe3d451d640 1 -- 172.21.15.105:0/3728044434 shutdown_connections 2023-11-28T18:23:32.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.896+0000 7fe3d451d640 1 --2- 172.21.15.105:0/3728044434 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3b0066900 0x7fe3b0068dc0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:32.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.896+0000 7fe3d451d640 1 --2- 172.21.15.105:0/3728044434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3cc161870 0x7fe3cc150fb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:32.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.896+0000 7fe3d451d640 1 --2- 172.21.15.105:0/3728044434 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3cc15d3c0 0x7fe3cc150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:32.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.896+0000 7fe3d451d640 1 --2- 172.21.15.105:0/3728044434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3cc070a10 0x7fe3cc154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:32.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.896+0000 7fe3d451d640 1 -- 172.21.15.105:0/3728044434 >> 172.21.15.105:0/3728044434 conn(0x7fe3cc0cf540 msgr2=0x7fe3cc0cf920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:32.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.900+0000 7fe3d451d640 1 -- 172.21.15.105:0/3728044434 shutdown_connections 2023-11-28T18:23:32.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:32.900+0000 7fe3d451d640 1 -- 172.21.15.105:0/3728044434 wait complete. 2023-11-28T18:23:32.946 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T18:23:33.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.084+0000 7f546d26b640 1 Processor -- start 2023-11-28T18:23:33.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.084+0000 7f546d26b640 1 -- start start 2023-11-28T18:23:33.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.088+0000 7f546d26b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f546815d4f0 0x7f54680d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:33.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.088+0000 7f546d26b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f54680b2d20 0x7f54680b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:33.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.088+0000 7f546d26b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54680b25b0 con 0x7f54680d3c30 2023-11-28T18:23:33.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.088+0000 7f546d26b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54680b2900 con 0x7f54680b2d20 2023-11-28T18:23:33.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.088+0000 7f546d26b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54680d0a20 con 0x7f546815d4f0 2023-11-28T18:23:33.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.088+0000 7f54677fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f54680b2d20 0x7f54680b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:33.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.088+0000 7f545e7fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f54680d3c30 0x7f54680d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51416/0 (socket says 172.21.15.105:51416) 2023-11-28T18:23:33.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.088+0000 7f545e7fc640 1 -- 172.21.15.105:0/3852887336 learned_addr learned my addr 172.21.15.105:0/3852887336 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:33.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.088+0000 7f5466ffd640 1 --2- 172.21.15.105:0/3852887336 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f546815d4f0 0x7f54680d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:33.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.088+0000 7f54667fc640 1 -- 172.21.15.105:0/3852887336 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 933769176 0 0) 0x7f54680b25b0 con 0x7f54680d3c30 2023-11-28T18:23:33.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.088+0000 7f54667fc640 1 -- 172.21.15.105:0/3852887336 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5454003530 con 0x7f54680d3c30 2023-11-28T18:23:33.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.088+0000 7f54677fe640 1 -- 172.21.15.105:0/3852887336 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f546815d4f0 msgr2=0x7f54680d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:33.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.088+0000 7f54677fe640 1 --2- 172.21.15.105:0/3852887336 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f546815d4f0 0x7f54680d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:33.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.088+0000 7f54677fe640 1 -- 172.21.15.105:0/3852887336 >> v1:172.21.15.105:6789/0 conn(0x7f54680d3c30 legacy=0x7f54680d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:33.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.088+0000 7f54677fe640 1 -- 172.21.15.105:0/3852887336 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54680cea30 con 0x7f54680b2d20 2023-11-28T18:23:33.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.092+0000 7f54677fe640 1 --2- 172.21.15.105:0/3852887336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54680b2d20 0x7f54680b31d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f54600097c0 tx=0x7f546001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3978222496334a4c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:33.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.092+0000 7f54667fc640 1 -- 172.21.15.105:0/3852887336 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54600ac620 con 0x7f54680b2d20 2023-11-28T18:23:33.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.092+0000 7f54667fc640 1 -- 172.21.15.105:0/3852887336 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54600ac780 con 0x7f54680b2d20 2023-11-28T18:23:33.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.092+0000 7f54667fc640 1 -- 172.21.15.105:0/3852887336 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54600acb10 con 0x7f54680b2d20 2023-11-28T18:23:33.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.092+0000 7f546d26b640 1 -- 172.21.15.105:0/3852887336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54680b2d20 msgr2=0x7f54680b31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:33.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.092+0000 7f546d26b640 1 --2- 172.21.15.105:0/3852887336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54680b2d20 0x7f54680b31d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f54600097c0 tx=0x7f546001c3c0 comp rx=0 tx=0).stop 2023-11-28T18:23:33.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.092+0000 7f546d26b640 1 -- 172.21.15.105:0/3852887336 shutdown_connections 2023-11-28T18:23:33.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.092+0000 7f546d26b640 1 --2- 172.21.15.105:0/3852887336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54680b2d20 0x7f54680b31d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:33.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.092+0000 7f546d26b640 1 --2- 172.21.15.105:0/3852887336 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f546815d4f0 0x7f54680d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:33.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.092+0000 7f546d26b640 1 -- 172.21.15.105:0/3852887336 >> 172.21.15.105:0/3852887336 conn(0x7f54680d3030 msgr2=0x7f54680d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:33.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.092+0000 7f546d26b640 1 -- 172.21.15.105:0/3852887336 shutdown_connections 2023-11-28T18:23:33.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.092+0000 7f546d26b640 1 -- 172.21.15.105:0/3852887336 wait complete. 2023-11-28T18:23:33.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.092+0000 7f546d26b640 1 Processor -- start 2023-11-28T18:23:33.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.092+0000 7f546d26b640 1 -- start start 2023-11-28T18:23:33.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f546d26b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54680b2d20 0x7f5468154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:33.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f546d26b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54680d3c30 0x7f54680ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:33.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f546d26b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f546815d4f0 0x7f54681602e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:33.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f546d26b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546814c7a0 con 0x7f546815d4f0 2023-11-28T18:23:33.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f546d26b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5468153e90 con 0x7f54680b2d20 2023-11-28T18:23:33.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f546d26b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546815c010 con 0x7f54680d3c30 2023-11-28T18:23:33.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f54677fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f546815d4f0 0x7f54681602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:33.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f54677fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f546815d4f0 0x7f54681602e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59054/0 (socket says 172.21.15.105:59054) 2023-11-28T18:23:33.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f54677fe640 1 -- 172.21.15.105:0/4030873964 learned_addr learned my addr 172.21.15.105:0/4030873964 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:33.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f5466ffd640 1 --2- 172.21.15.105:0/4030873964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54680b2d20 0x7f5468154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:33.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f545e7fc640 1 --2- 172.21.15.105:0/4030873964 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54680d3c30 0x7f54680ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:33.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f54677fe640 1 -- 172.21.15.105:0/4030873964 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54680d3c30 msgr2=0x7f54680ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:33.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f54677fe640 1 --2- 172.21.15.105:0/4030873964 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54680d3c30 0x7f54680ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:33.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f54677fe640 1 -- 172.21.15.105:0/4030873964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54680b2d20 msgr2=0x7f5468154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:33.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f54677fe640 1 --2- 172.21.15.105:0/4030873964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54680b2d20 0x7f5468154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:33.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f54677fe640 1 -- 172.21.15.105:0/4030873964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5460009420 con 0x7f546815d4f0 2023-11-28T18:23:33.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f545e7fc640 1 --2- 172.21.15.105:0/4030873964 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54680d3c30 0x7f54680ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:33.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f54677fe640 1 --2- 172.21.15.105:0/4030873964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f546815d4f0 0x7f54681602e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5460019ff0 tx=0x7f546001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:33.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f545ffff640 1 -- 172.21.15.105:0/4030873964 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54600ac620 con 0x7f546815d4f0 2023-11-28T18:23:33.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f545ffff640 1 -- 172.21.15.105:0/4030873964 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54600b8450 con 0x7f546815d4f0 2023-11-28T18:23:33.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f545ffff640 1 -- 172.21.15.105:0/4030873964 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54600b87e0 con 0x7f546815d4f0 2023-11-28T18:23:33.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f546d26b640 1 -- 172.21.15.105:0/4030873964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f546815c2f0 con 0x7f546815d4f0 2023-11-28T18:23:33.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f546d26b640 1 -- 172.21.15.105:0/4030873964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5468153b20 con 0x7f546815d4f0 2023-11-28T18:23:33.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f546d26b640 1 -- 172.21.15.105:0/4030873964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5468154000 con 0x7f546815d4f0 2023-11-28T18:23:33.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.096+0000 7f545ffff640 1 -- 172.21.15.105:0/4030873964 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f54600aa070 con 0x7f546815d4f0 2023-11-28T18:23:33.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.100+0000 7f545ffff640 1 --2- 172.21.15.105:0/4030873964 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f54480668d0 0x7f5448068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:33.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.100+0000 7f545ffff640 1 -- 172.21.15.105:0/4030873964 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f546011eda0 con 0x7f546815d4f0 2023-11-28T18:23:33.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.100+0000 7f5466ffd640 1 --2- 172.21.15.105:0/4030873964 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f54480668d0 0x7f5448068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:33.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.100+0000 7f5466ffd640 1 --2- 172.21.15.105:0/4030873964 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f54480668d0 0x7f5448068d90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f54680cd100 tx=0x7f54580098d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:33.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.104+0000 7f545ffff640 1 -- 172.21.15.105:0/4030873964 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f54600ec020 con 0x7f546815d4f0 2023-11-28T18:23:33.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:33.284+0000 7f546d26b640 1 -- 172.21.15.105:0/4030873964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5468067570 con 0x7f546815d4f0 2023-11-28T18:23:34.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.032+0000 7f545ffff640 1 -- 172.21.15.105:0/4030873964 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f54600ef640 con 0x7f546815d4f0 2023-11-28T18:23:34.035 INFO:teuthology.orchestra.run.smithi105.stderr:new fs with metadata pool 2 and data pool 3 2023-11-28T18:23:34.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.032+0000 7f546d26b640 1 -- 172.21.15.105:0/4030873964 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f54480668d0 msgr2=0x7f5448068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:34.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.032+0000 7f546d26b640 1 --2- 172.21.15.105:0/4030873964 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f54480668d0 0x7f5448068d90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f54680cd100 tx=0x7f54580098d0 comp rx=0 tx=0).stop 2023-11-28T18:23:34.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.032+0000 7f546d26b640 1 -- 172.21.15.105:0/4030873964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f546815d4f0 msgr2=0x7f54681602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:34.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.032+0000 7f546d26b640 1 --2- 172.21.15.105:0/4030873964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f546815d4f0 0x7f54681602e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5460019ff0 tx=0x7f546001ac00 comp rx=0 tx=0).stop 2023-11-28T18:23:34.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.032+0000 7f546d26b640 1 -- 172.21.15.105:0/4030873964 shutdown_connections 2023-11-28T18:23:34.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.032+0000 7f546d26b640 1 --2- 172.21.15.105:0/4030873964 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f54480668d0 0x7f5448068d90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:34.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.032+0000 7f546d26b640 1 --2- 172.21.15.105:0/4030873964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f546815d4f0 0x7f54681602e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:34.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.032+0000 7f546d26b640 1 --2- 172.21.15.105:0/4030873964 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54680d3c30 0x7f54680ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:34.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.032+0000 7f546d26b640 1 --2- 172.21.15.105:0/4030873964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54680b2d20 0x7f5468154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:34.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.032+0000 7f546d26b640 1 -- 172.21.15.105:0/4030873964 >> 172.21.15.105:0/4030873964 conn(0x7f54680d3030 msgr2=0x7f54680c9940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:34.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.036+0000 7f546d26b640 1 -- 172.21.15.105:0/4030873964 shutdown_connections 2023-11-28T18:23:34.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.036+0000 7f546d26b640 1 -- 172.21.15.105:0/4030873964 wait complete. 2023-11-28T18:23:34.064 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-11-28T18:23:34.064 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-11-28T18:23:34.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.208+0000 7f988c29a640 1 Processor -- start 2023-11-28T18:23:34.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.208+0000 7f988c29a640 1 -- start start 2023-11-28T18:23:34.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.208+0000 7f988c29a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f988415d320 0x7f988406c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:34.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.208+0000 7f988c29a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f98840cdd80 0x7f98840cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:34.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.208+0000 7f988c29a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98840b3c40 con 0x7f98840ce230 2023-11-28T18:23:34.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.208+0000 7f988c29a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988406ca80 con 0x7f98840cdd80 2023-11-28T18:23:34.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.208+0000 7f988c29a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988415d700 con 0x7f988415d320 2023-11-28T18:23:34.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.208+0000 7f988a837640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f98840cdd80 0x7f98840cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:34.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.208+0000 7f988a837640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f98840cdd80 0x7f98840cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59070/0 (socket says 172.21.15.105:59070) 2023-11-28T18:23:34.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.208+0000 7f988a837640 1 -- 172.21.15.105:0/3558845313 learned_addr learned my addr 172.21.15.105:0/3558845313 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:34.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.208+0000 7f988a036640 1 --2- 172.21.15.105:0/3558845313 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f988415d320 0x7f988406c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:34.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.208+0000 7f988a837640 1 -- 172.21.15.105:0/3558845313 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f988415d320 msgr2=0x7f988406c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:34.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.208+0000 7f988a837640 1 --2- 172.21.15.105:0/3558845313 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f988415d320 0x7f988406c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:34.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.208+0000 7f988a837640 1 -- 172.21.15.105:0/3558845313 >> v1:172.21.15.148:6789/0 conn(0x7f98840ce230 legacy=0x7f98840cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:34.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.208+0000 7f988a837640 1 -- 172.21.15.105:0/3558845313 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988406c7a0 con 0x7f98840cdd80 2023-11-28T18:23:34.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.212+0000 7f988a837640 1 --2- 172.21.15.105:0/3558845313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98840cdd80 0x7f98840cc0c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f987c009e50 tx=0x7f987c01c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc25fba552c33c86 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:34.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.212+0000 7f9889034640 1 -- 172.21.15.105:0/3558845313 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c0a5840 con 0x7f98840cdd80 2023-11-28T18:23:34.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.212+0000 7f9889034640 1 -- 172.21.15.105:0/3558845313 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987c0a59a0 con 0x7f98840cdd80 2023-11-28T18:23:34.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.212+0000 7f9889034640 1 -- 172.21.15.105:0/3558845313 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c0a5d30 con 0x7f98840cdd80 2023-11-28T18:23:34.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.212+0000 7f988c29a640 1 -- 172.21.15.105:0/3558845313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98840cdd80 msgr2=0x7f98840cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:34.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.212+0000 7f988c29a640 1 --2- 172.21.15.105:0/3558845313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98840cdd80 0x7f98840cc0c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f987c009e50 tx=0x7f987c01c980 comp rx=0 tx=0).stop 2023-11-28T18:23:34.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.212+0000 7f988c29a640 1 -- 172.21.15.105:0/3558845313 shutdown_connections 2023-11-28T18:23:34.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.212+0000 7f988c29a640 1 --2- 172.21.15.105:0/3558845313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98840cdd80 0x7f98840cc0c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:34.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.212+0000 7f988c29a640 1 --2- 172.21.15.105:0/3558845313 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f988415d320 0x7f988406c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:34.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.212+0000 7f988c29a640 1 -- 172.21.15.105:0/3558845313 >> 172.21.15.105:0/3558845313 conn(0x7f98840d0df0 msgr2=0x7f98840d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:34.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.212+0000 7f988c29a640 1 -- 172.21.15.105:0/3558845313 shutdown_connections 2023-11-28T18:23:34.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.212+0000 7f988c29a640 1 -- 172.21.15.105:0/3558845313 wait complete. 2023-11-28T18:23:34.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f988c29a640 1 Processor -- start 2023-11-28T18:23:34.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f988c29a640 1 -- start start 2023-11-28T18:23:34.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f988c29a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98840cdd80 0x7f9884154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:34.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f988c29a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98840ce230 0x7f9884156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:34.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f988c29a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f988415d320 0x7f98840b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:34.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f988c29a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98841584b0 con 0x7f98840cdd80 2023-11-28T18:23:34.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f988c29a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9884158940 con 0x7f98840ce230 2023-11-28T18:23:34.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f988c29a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98841578a0 con 0x7f988415d320 2023-11-28T18:23:34.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f988a036640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98840cdd80 0x7f9884154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:34.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f988a036640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98840cdd80 0x7f9884154dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59078/0 (socket says 172.21.15.105:59078) 2023-11-28T18:23:34.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f988a036640 1 -- 172.21.15.105:0/80678899 learned_addr learned my addr 172.21.15.105:0/80678899 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:34.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f9889835640 1 --2- 172.21.15.105:0/80678899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98840ce230 0x7f9884156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:34.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f988a837640 1 --2- 172.21.15.105:0/80678899 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f988415d320 0x7f98840b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:34.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f988a036640 1 -- 172.21.15.105:0/80678899 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f988415d320 msgr2=0x7f98840b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:34.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f988a036640 1 --2- 172.21.15.105:0/80678899 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f988415d320 0x7f98840b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:34.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f988a036640 1 -- 172.21.15.105:0/80678899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98840ce230 msgr2=0x7f9884156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:34.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f988a036640 1 --2- 172.21.15.105:0/80678899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98840ce230 0x7f9884156f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:34.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f988a036640 1 -- 172.21.15.105:0/80678899 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987c009b30 con 0x7f98840cdd80 2023-11-28T18:23:34.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f988a837640 1 --2- 172.21.15.105:0/80678899 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f988415d320 0x7f98840b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:34.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f988a036640 1 --2- 172.21.15.105:0/80678899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98840cdd80 0x7f9884154dc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f988000e870 tx=0x7f988000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:34.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f987affd640 1 -- 172.21.15.105:0/80678899 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880009510 con 0x7f98840cdd80 2023-11-28T18:23:34.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f987affd640 1 -- 172.21.15.105:0/80678899 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988000eec0 con 0x7f98840cdd80 2023-11-28T18:23:34.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f987affd640 1 -- 172.21.15.105:0/80678899 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988000cb40 con 0x7f98840cdd80 2023-11-28T18:23:34.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f988c29a640 1 -- 172.21.15.105:0/80678899 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9884157b80 con 0x7f98840cdd80 2023-11-28T18:23:34.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f988c29a640 1 -- 172.21.15.105:0/80678899 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9884158140 con 0x7f98840cdd80 2023-11-28T18:23:34.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.216+0000 7f988c29a640 1 -- 172.21.15.105:0/80678899 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9884067570 con 0x7f98840cdd80 2023-11-28T18:23:34.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.220+0000 7f987affd640 1 -- 172.21.15.105:0/80678899 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f988000ccc0 con 0x7f98840cdd80 2023-11-28T18:23:34.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.220+0000 7f987affd640 1 --2- 172.21.15.105:0/80678899 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f98680668d0 0x7f9868068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:34.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.220+0000 7f987affd640 1 -- 172.21.15.105:0/80678899 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9880084930 con 0x7f98840cdd80 2023-11-28T18:23:34.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.220+0000 7f9889835640 1 --2- 172.21.15.105:0/80678899 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f98680668d0 0x7f9868068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:34.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.220+0000 7f9889835640 1 --2- 172.21.15.105:0/80678899 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f98680668d0 0x7f9868068d90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f988406b240 tx=0x7f9874009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:34.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.224+0000 7f987affd640 1 -- 172.21.15.105:0/80678899 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9880051af0 con 0x7f98840cdd80 2023-11-28T18:23:34.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.380+0000 7f988c29a640 1 -- 172.21.15.105:0/80678899 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f98840d0910 con 0x7f98840cdd80 2023-11-28T18:23:34.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.912+0000 7f987affd640 1 -- 172.21.15.105:0/80678899 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7f98800210e0 con 0x7f98840cdd80 2023-11-28T18:23:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.912+0000 7f988c29a640 1 -- 172.21.15.105:0/80678899 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f98680668d0 msgr2=0x7f9868068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.912+0000 7f988c29a640 1 --2- 172.21.15.105:0/80678899 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f98680668d0 0x7f9868068d90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f988406b240 tx=0x7f9874009040 comp rx=0 tx=0).stop 2023-11-28T18:23:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.912+0000 7f988c29a640 1 -- 172.21.15.105:0/80678899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98840cdd80 msgr2=0x7f9884154dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.912+0000 7f988c29a640 1 --2- 172.21.15.105:0/80678899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98840cdd80 0x7f9884154dc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f988000e870 tx=0x7f988000ec70 comp rx=0 tx=0).stop 2023-11-28T18:23:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.916+0000 7f988c29a640 1 -- 172.21.15.105:0/80678899 shutdown_connections 2023-11-28T18:23:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.916+0000 7f988c29a640 1 --2- 172.21.15.105:0/80678899 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f988415d320 0x7f98840b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.916+0000 7f988c29a640 1 --2- 172.21.15.105:0/80678899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98840ce230 0x7f9884156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.916+0000 7f988c29a640 1 --2- 172.21.15.105:0/80678899 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f98680668d0 0x7f9868068d90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:34.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.916+0000 7f988c29a640 1 --2- 172.21.15.105:0/80678899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98840cdd80 0x7f9884154dc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:34.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.916+0000 7f988c29a640 1 -- 172.21.15.105:0/80678899 >> 172.21.15.105:0/80678899 conn(0x7f98840d0df0 msgr2=0x7f98840d11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:34.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.916+0000 7f988c29a640 1 -- 172.21.15.105:0/80678899 shutdown_connections 2023-11-28T18:23:34.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:34.916+0000 7f988c29a640 1 -- 172.21.15.105:0/80678899 wait complete. 2023-11-28T18:23:34.939 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T18:23:35.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.084+0000 7fc6c1ff8640 1 Processor -- start 2023-11-28T18:23:35.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.084+0000 7fc6c1ff8640 1 -- start start 2023-11-28T18:23:35.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.084+0000 7fc6c1ff8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc6bc1546c0 0x7fc6bc071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:35.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.084+0000 7fc6c1ff8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6bc0b4e90 0x7fc6bc06df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:35.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.084+0000 7fc6c1ff8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6bc071d40 con 0x7fc6bc071890 2023-11-28T18:23:35.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.084+0000 7fc6c1ff8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6bc0b5300 con 0x7fc6bc1546c0 2023-11-28T18:23:35.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.084+0000 7fc6c1ff8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc0b3e80 con 0x7fc6bc0b4e90 2023-11-28T18:23:35.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.084+0000 7fc6bbfff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6bc0b4e90 0x7fc6bc06df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:35.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.084+0000 7fc6bb7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc6bc1546c0 0x7fc6bc071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:35.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.084+0000 7fc6baffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc6bc071890 0x7fc6bc0b4780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:44152/0 (socket says 172.21.15.105:44152) 2023-11-28T18:23:35.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.084+0000 7fc6baffd640 1 -- 172.21.15.105:0/1975254149 learned_addr learned my addr 172.21.15.105:0/1975254149 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:35.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.088+0000 7fc6bbfff640 1 -- 172.21.15.105:0/1975254149 >> v2:172.21.15.148:3300/0 conn(0x7fc6bc1546c0 msgr2=0x7fc6bc071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:35.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.088+0000 7fc6bbfff640 1 --2- 172.21.15.105:0/1975254149 >> v2:172.21.15.148:3300/0 conn(0x7fc6bc1546c0 0x7fc6bc071350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:35.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.088+0000 7fc6bbfff640 1 -- 172.21.15.105:0/1975254149 >> v1:172.21.15.148:6789/0 conn(0x7fc6bc071890 legacy=0x7fc6bc0b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:35.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.088+0000 7fc6bbfff640 1 -- 172.21.15.105:0/1975254149 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc154aa0 con 0x7fc6bc0b4e90 2023-11-28T18:23:35.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.088+0000 7fc6ba7fc640 1 -- 172.21.15.105:0/1975254149 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2415812131 0 0) 0x7fc6bc071d40 con 0x7fc6bc071890 2023-11-28T18:23:35.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.088+0000 7fc6bb7fe640 1 --2- 172.21.15.105:0/1975254149 >> v2:172.21.15.148:3300/0 conn(0x7fc6bc1546c0 0x7fc6bc071350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:35.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.088+0000 7fc6bbfff640 1 --2- 172.21.15.105:0/1975254149 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6bc0b4e90 0x7fc6bc06df70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc6b400bfa0 tx=0x7fc6b401eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d15ab82ff0a098a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:35.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.088+0000 7fc6ba7fc640 1 -- 172.21.15.105:0/1975254149 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b400b2d0 con 0x7fc6bc0b4e90 2023-11-28T18:23:35.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.088+0000 7fc6ba7fc640 1 -- 172.21.15.105:0/1975254149 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b40a79f0 con 0x7fc6bc0b4e90 2023-11-28T18:23:35.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.088+0000 7fc6ba7fc640 1 -- 172.21.15.105:0/1975254149 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b40a7d80 con 0x7fc6bc0b4e90 2023-11-28T18:23:35.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6c1ff8640 1 -- 172.21.15.105:0/1975254149 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6bc0b4e90 msgr2=0x7fc6bc06df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:35.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6c1ff8640 1 --2- 172.21.15.105:0/1975254149 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6bc0b4e90 0x7fc6bc06df70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc6b400bfa0 tx=0x7fc6b401eb20 comp rx=0 tx=0).stop 2023-11-28T18:23:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6c1ff8640 1 -- 172.21.15.105:0/1975254149 shutdown_connections 2023-11-28T18:23:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6c1ff8640 1 --2- 172.21.15.105:0/1975254149 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6bc0b4e90 0x7fc6bc06df70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6c1ff8640 1 --2- 172.21.15.105:0/1975254149 >> v2:172.21.15.148:3300/0 conn(0x7fc6bc1546c0 0x7fc6bc071350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6c1ff8640 1 -- 172.21.15.105:0/1975254149 >> 172.21.15.105:0/1975254149 conn(0x7fc6bc0b1a40 msgr2=0x7fc6bc0b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6c1ff8640 1 -- 172.21.15.105:0/1975254149 shutdown_connections 2023-11-28T18:23:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6c1ff8640 1 -- 172.21.15.105:0/1975254149 wait complete. 2023-11-28T18:23:35.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6c1ff8640 1 Processor -- start 2023-11-28T18:23:35.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6c1ff8640 1 -- start start 2023-11-28T18:23:35.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6c1ff8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6bc071890 0x7fc6bc14d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:35.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6c1ff8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6bc0b4e90 0x7fc6bc0ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:35.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6c1ff8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6bc1546c0 0x7fc6bc1f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:35.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6c1ff8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc09ce10 con 0x7fc6bc0b4e90 2023-11-28T18:23:35.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6c1ff8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc09d0f0 con 0x7fc6bc071890 2023-11-28T18:23:35.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6c1ff8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc09d3d0 con 0x7fc6bc1546c0 2023-11-28T18:23:35.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6baffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6bc0b4e90 0x7fc6bc0ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:35.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6baffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6bc0b4e90 0x7fc6bc0ae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59082/0 (socket says 172.21.15.105:59082) 2023-11-28T18:23:35.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6baffd640 1 -- 172.21.15.105:0/2921785999 learned_addr learned my addr 172.21.15.105:0/2921785999 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:35.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6bb7fe640 1 --2- 172.21.15.105:0/2921785999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6bc071890 0x7fc6bc14d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:35.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6bbfff640 1 --2- 172.21.15.105:0/2921785999 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6bc1546c0 0x7fc6bc1f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:35.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6baffd640 1 -- 172.21.15.105:0/2921785999 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6bc1546c0 msgr2=0x7fc6bc1f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:35.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6baffd640 1 --2- 172.21.15.105:0/2921785999 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6bc1546c0 0x7fc6bc1f4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:35.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6baffd640 1 -- 172.21.15.105:0/2921785999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6bc071890 msgr2=0x7fc6bc14d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:35.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6baffd640 1 --2- 172.21.15.105:0/2921785999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6bc071890 0x7fc6bc14d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:35.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.092+0000 7fc6baffd640 1 -- 172.21.15.105:0/2921785999 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac009420 con 0x7fc6bc0b4e90 2023-11-28T18:23:35.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.096+0000 7fc6baffd640 1 --2- 172.21.15.105:0/2921785999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6bc0b4e90 0x7fc6bc0ae170 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00011f0 tx=0x7fc6b000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:35.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.096+0000 7fc6a3fff640 1 -- 172.21.15.105:0/2921785999 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b00074e0 con 0x7fc6bc0b4e90 2023-11-28T18:23:35.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.096+0000 7fc6a3fff640 1 -- 172.21.15.105:0/2921785999 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b0007640 con 0x7fc6bc0b4e90 2023-11-28T18:23:35.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.096+0000 7fc6a3fff640 1 -- 172.21.15.105:0/2921785999 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0004a10 con 0x7fc6bc0b4e90 2023-11-28T18:23:35.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.096+0000 7fc6c1ff8640 1 -- 172.21.15.105:0/2921785999 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b400bc80 con 0x7fc6bc0b4e90 2023-11-28T18:23:35.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.096+0000 7fc6c1ff8640 1 -- 172.21.15.105:0/2921785999 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6bc14e490 con 0x7fc6bc0b4e90 2023-11-28T18:23:35.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.096+0000 7fc6c1ff8640 1 -- 172.21.15.105:0/2921785999 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6bc067570 con 0x7fc6bc0b4e90 2023-11-28T18:23:35.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.096+0000 7fc6a3fff640 1 -- 172.21.15.105:0/2921785999 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc6b0004b70 con 0x7fc6bc0b4e90 2023-11-28T18:23:35.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.096+0000 7fc6a3fff640 1 --2- 172.21.15.105:0/2921785999 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6a80668d0 0x7fc6a8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:35.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.096+0000 7fc6a3fff640 1 -- 172.21.15.105:0/2921785999 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5935+0+0 (secure 0 0 0) 0x7fc6b00836e0 con 0x7fc6bc0b4e90 2023-11-28T18:23:35.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.096+0000 7fc6bb7fe640 1 --2- 172.21.15.105:0/2921785999 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6a80668d0 0x7fc6a8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:35.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.100+0000 7fc6bb7fe640 1 --2- 172.21.15.105:0/2921785999 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6a80668d0 0x7fc6a8068d90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc06d3f0 tx=0x7fc6ac009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:35.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.100+0000 7fc6a3fff640 1 -- 172.21.15.105:0/2921785999 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc6b0050980 con 0x7fc6bc0b4e90 2023-11-28T18:23:35.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:35.252+0000 7fc6c1ff8640 1 -- 172.21.15.105:0/2921785999 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc6bc0b4560 con 0x7fc6bc0b4e90 2023-11-28T18:23:37.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.052+0000 7fc6a3fff640 1 -- 172.21.15.105:0/2921785999 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7fc6b0018100 con 0x7fc6bc0b4e90 2023-11-28T18:23:37.055 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data_ec' created 2023-11-28T18:23:37.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.056+0000 7fc6c1ff8640 1 -- 172.21.15.105:0/2921785999 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6a80668d0 msgr2=0x7fc6a8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:37.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.056+0000 7fc6c1ff8640 1 --2- 172.21.15.105:0/2921785999 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6a80668d0 0x7fc6a8068d90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc06d3f0 tx=0x7fc6ac009190 comp rx=0 tx=0).stop 2023-11-28T18:23:37.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.056+0000 7fc6c1ff8640 1 -- 172.21.15.105:0/2921785999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6bc0b4e90 msgr2=0x7fc6bc0ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:37.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.056+0000 7fc6c1ff8640 1 --2- 172.21.15.105:0/2921785999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6bc0b4e90 0x7fc6bc0ae170 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00011f0 tx=0x7fc6b000c9b0 comp rx=0 tx=0).stop 2023-11-28T18:23:37.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.056+0000 7fc6c1ff8640 1 -- 172.21.15.105:0/2921785999 shutdown_connections 2023-11-28T18:23:37.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.056+0000 7fc6c1ff8640 1 --2- 172.21.15.105:0/2921785999 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6bc1546c0 0x7fc6bc1f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:37.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.056+0000 7fc6c1ff8640 1 --2- 172.21.15.105:0/2921785999 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6a80668d0 0x7fc6a8068d90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:37.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.056+0000 7fc6c1ff8640 1 --2- 172.21.15.105:0/2921785999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6bc0b4e90 0x7fc6bc0ae170 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:37.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.056+0000 7fc6c1ff8640 1 --2- 172.21.15.105:0/2921785999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6bc071890 0x7fc6bc14d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:37.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.056+0000 7fc6c1ff8640 1 -- 172.21.15.105:0/2921785999 >> 172.21.15.105:0/2921785999 conn(0x7fc6bc0b1a40 msgr2=0x7fc6bc151ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:37.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.056+0000 7fc6c1ff8640 1 -- 172.21.15.105:0/2921785999 shutdown_connections 2023-11-28T18:23:37.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.056+0000 7fc6c1ff8640 1 -- 172.21.15.105:0/2921785999 wait complete. 2023-11-28T18:23:37.090 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-11-28T18:23:37.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.232+0000 7f2bcbdc4640 1 Processor -- start 2023-11-28T18:23:37.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.232+0000 7f2bcbdc4640 1 -- start start 2023-11-28T18:23:37.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.232+0000 7f2bcbdc4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2bc4159610 0x7f2bc40ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:37.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.232+0000 7f2bcbdc4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2bc40cc760 0x7f2bc40ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:37.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.232+0000 7f2bcbdc4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc40cde30 con 0x7f2bc406c7e0 2023-11-28T18:23:37.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.232+0000 7f2bcbdc4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc41599f0 con 0x7f2bc40cc760 2023-11-28T18:23:37.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.232+0000 7f2bcbdc4640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc406de20 con 0x7f2bc4159610 2023-11-28T18:23:37.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.232+0000 7f2bc935f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2bc40cc760 0x7f2bc40ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:37.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.232+0000 7f2bc9b60640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2bc4159610 0x7f2bc40ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:37.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.232+0000 7f2bc935f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2bc40cc760 0x7f2bc40ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:43220/0 (socket says 172.21.15.105:43220) 2023-11-28T18:23:37.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.232+0000 7f2bc935f640 1 -- 172.21.15.105:0/1746084743 learned_addr learned my addr 172.21.15.105:0/1746084743 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:37.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.232+0000 7f2bca361640 1 --1- 172.21.15.105:0/1746084743 >> v1:172.21.15.148:6789/0 conn(0x7f2bc406c7e0 0x7f2bc40abd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:44160/0 (socket says 172.21.15.105:44160) 2023-11-28T18:23:37.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.232+0000 7f2bc9b60640 1 -- 172.21.15.105:0/1746084743 >> v2:172.21.15.148:3300/0 conn(0x7f2bc40cc760 msgr2=0x7f2bc40ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:37.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.232+0000 7f2bc9b60640 1 --2- 172.21.15.105:0/1746084743 >> v2:172.21.15.148:3300/0 conn(0x7f2bc40cc760 0x7f2bc40ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:37.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.232+0000 7f2bc9b60640 1 -- 172.21.15.105:0/1746084743 >> v1:172.21.15.148:6789/0 conn(0x7f2bc406c7e0 legacy=0x7f2bc40abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:37.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.232+0000 7f2bc9b60640 1 -- 172.21.15.105:0/1746084743 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc40cd220 con 0x7f2bc4159610 2023-11-28T18:23:37.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.236+0000 7f2bc9b60640 1 --2- 172.21.15.105:0/1746084743 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2bc4159610 0x7f2bc40ce2c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000a960 tx=0x7f2bc40acca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e79a2206d1fcf65 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:37.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.236+0000 7f2bc8b5e640 1 -- 172.21.15.105:0/1746084743 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc00a68d0 con 0x7f2bc4159610 2023-11-28T18:23:37.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.236+0000 7f2bc8b5e640 1 -- 172.21.15.105:0/1746084743 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc00a6a30 con 0x7f2bc4159610 2023-11-28T18:23:37.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.236+0000 7f2bc8b5e640 1 -- 172.21.15.105:0/1746084743 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc00a6dc0 con 0x7f2bc4159610 2023-11-28T18:23:37.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.236+0000 7f2bcbdc4640 1 -- 172.21.15.105:0/1746084743 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2bc4159610 msgr2=0x7f2bc40ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:37.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.236+0000 7f2bcbdc4640 1 --2- 172.21.15.105:0/1746084743 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2bc4159610 0x7f2bc40ce2c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000a960 tx=0x7f2bc40acca0 comp rx=0 tx=0).stop 2023-11-28T18:23:37.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.236+0000 7f2bcbdc4640 1 -- 172.21.15.105:0/1746084743 shutdown_connections 2023-11-28T18:23:37.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.236+0000 7f2bcbdc4640 1 --2- 172.21.15.105:0/1746084743 >> v2:172.21.15.148:3300/0 conn(0x7f2bc40cc760 0x7f2bc40ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:37.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.236+0000 7f2bcbdc4640 1 --2- 172.21.15.105:0/1746084743 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2bc4159610 0x7f2bc40ce2c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:37.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.236+0000 7f2bcbdc4640 1 -- 172.21.15.105:0/1746084743 >> 172.21.15.105:0/1746084743 conn(0x7f2bc40ceb90 msgr2=0x7f2bc40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:37.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.236+0000 7f2bcbdc4640 1 -- 172.21.15.105:0/1746084743 shutdown_connections 2023-11-28T18:23:37.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.236+0000 7f2bcbdc4640 1 -- 172.21.15.105:0/1746084743 wait complete. 2023-11-28T18:23:37.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bcbdc4640 1 Processor -- start 2023-11-28T18:23:37.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bcbdc4640 1 -- start start 2023-11-28T18:23:37.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bcbdc4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2bc406c7e0 0x7f2bc4148500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:37.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bcbdc4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2bc40cc760 0x7f2bc4155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:37.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bcbdc4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bc4159610 0x7f2bc40afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:37.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bcbdc4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4148a40 con 0x7f2bc4159610 2023-11-28T18:23:37.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bcbdc4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4155fc0 con 0x7f2bc406c7e0 2023-11-28T18:23:37.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bcbdc4640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc414c630 con 0x7f2bc40cc760 2023-11-28T18:23:37.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bca361640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bc4159610 0x7f2bc40afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:37.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bca361640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bc4159610 0x7f2bc40afc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59092/0 (socket says 172.21.15.105:59092) 2023-11-28T18:23:37.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bca361640 1 -- 172.21.15.105:0/1127664141 learned_addr learned my addr 172.21.15.105:0/1127664141 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:37.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bc935f640 1 --2- 172.21.15.105:0/1127664141 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2bc40cc760 0x7f2bc4155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:37.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bca361640 1 -- 172.21.15.105:0/1127664141 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2bc40cc760 msgr2=0x7f2bc4155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:37.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bc9b60640 1 --2- 172.21.15.105:0/1127664141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2bc406c7e0 0x7f2bc4148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:37.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bca361640 1 --2- 172.21.15.105:0/1127664141 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2bc40cc760 0x7f2bc4155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:37.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bca361640 1 -- 172.21.15.105:0/1127664141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2bc406c7e0 msgr2=0x7f2bc4148500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:37.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bca361640 1 --2- 172.21.15.105:0/1127664141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2bc406c7e0 0x7f2bc4148500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:37.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bca361640 1 -- 172.21.15.105:0/1127664141 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc000a5c0 con 0x7f2bc4159610 2023-11-28T18:23:37.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bc935f640 1 --2- 172.21.15.105:0/1127664141 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2bc40cc760 0x7f2bc4155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:37.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bc9b60640 1 --2- 172.21.15.105:0/1127664141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2bc406c7e0 0x7f2bc4148500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:37.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bca361640 1 --2- 172.21.15.105:0/1127664141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bc4159610 0x7f2bc40afc20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00c7a0 tx=0x7f2bbc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:37.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bba7fc640 1 -- 172.21.15.105:0/1127664141 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc00a640 con 0x7f2bc4159610 2023-11-28T18:23:37.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bba7fc640 1 -- 172.21.15.105:0/1127664141 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bbc00a7a0 con 0x7f2bc4159610 2023-11-28T18:23:37.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bba7fc640 1 -- 172.21.15.105:0/1127664141 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc00ab10 con 0x7f2bc4159610 2023-11-28T18:23:37.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bcbdc4640 1 -- 172.21.15.105:0/1127664141 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc406d180 con 0x7f2bc4159610 2023-11-28T18:23:37.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.240+0000 7f2bcbdc4640 1 -- 172.21.15.105:0/1127664141 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc414cbf0 con 0x7f2bc4159610 2023-11-28T18:23:37.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.244+0000 7f2bcbdc4640 1 -- 172.21.15.105:0/1127664141 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc4067570 con 0x7f2bc4159610 2023-11-28T18:23:37.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.244+0000 7f2bba7fc640 1 -- 172.21.15.105:0/1127664141 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f2bbc00ac70 con 0x7f2bc4159610 2023-11-28T18:23:37.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.244+0000 7f2bba7fc640 1 --2- 172.21.15.105:0/1127664141 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2ba8066900 0x7f2ba8068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:37.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.244+0000 7f2bba7fc640 1 -- 172.21.15.105:0/1127664141 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f2bbc084300 con 0x7f2bc4159610 2023-11-28T18:23:37.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.244+0000 7f2bc9b60640 1 --2- 172.21.15.105:0/1127664141 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2ba8066900 0x7f2ba8068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:37.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.244+0000 7f2bc9b60640 1 --2- 172.21.15.105:0/1127664141 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2ba8066900 0x7f2ba8068dc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00b0000 tx=0x7f2bc0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.248+0000 7f2bba7fc640 1 -- 172.21.15.105:0/1127664141 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2bbc051390 con 0x7f2bc4159610 2023-11-28T18:23:37.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:37.400+0000 7f2bcbdc4640 1 -- 172.21.15.105:0/1127664141 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f2bc40cf950 con 0x7f2bc4159610 2023-11-28T18:23:37.877 INFO:tasks.ceph.mds.a.smithi105.stderr:2023-11-28T18:23:37.876+0000 7f30d6928640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T18:23:38.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.060+0000 7f2bba7fc640 1 -- 172.21.15.105:0/1127664141 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2bbc0549b0 con 0x7f2bc4159610 2023-11-28T18:23:38.061 INFO:teuthology.orchestra.run.smithi105.stderr:set pool 4 allow_ec_overwrites to true 2023-11-28T18:23:38.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.060+0000 7f2bcbdc4640 1 -- 172.21.15.105:0/1127664141 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2ba8066900 msgr2=0x7f2ba8068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:38.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.060+0000 7f2bcbdc4640 1 --2- 172.21.15.105:0/1127664141 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2ba8066900 0x7f2ba8068dc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00b0000 tx=0x7f2bc0008040 comp rx=0 tx=0).stop 2023-11-28T18:23:38.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.060+0000 7f2bcbdc4640 1 -- 172.21.15.105:0/1127664141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bc4159610 msgr2=0x7f2bc40afc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:38.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.060+0000 7f2bcbdc4640 1 --2- 172.21.15.105:0/1127664141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bc4159610 0x7f2bc40afc20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00c7a0 tx=0x7f2bbc00cc60 comp rx=0 tx=0).stop 2023-11-28T18:23:38.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.064+0000 7f2bcbdc4640 1 -- 172.21.15.105:0/1127664141 shutdown_connections 2023-11-28T18:23:38.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.064+0000 7f2bcbdc4640 1 --2- 172.21.15.105:0/1127664141 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2ba8066900 0x7f2ba8068dc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:38.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.064+0000 7f2bcbdc4640 1 --2- 172.21.15.105:0/1127664141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bc4159610 0x7f2bc40afc20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:38.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.064+0000 7f2bcbdc4640 1 --2- 172.21.15.105:0/1127664141 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2bc40cc760 0x7f2bc4155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:38.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.064+0000 7f2bcbdc4640 1 --2- 172.21.15.105:0/1127664141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2bc406c7e0 0x7f2bc4148500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:38.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.064+0000 7f2bcbdc4640 1 -- 172.21.15.105:0/1127664141 >> 172.21.15.105:0/1127664141 conn(0x7f2bc40ceb90 msgr2=0x7f2bc40af4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:38.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.064+0000 7f2bcbdc4640 1 -- 172.21.15.105:0/1127664141 shutdown_connections 2023-11-28T18:23:38.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.064+0000 7f2bcbdc4640 1 -- 172.21.15.105:0/1127664141 wait complete. 2023-11-28T18:23:38.111 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-11-28T18:23:38.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.248+0000 7f5496d2a640 1 Processor -- start 2023-11-28T18:23:38.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.248+0000 7f5496d2a640 1 -- start start 2023-11-28T18:23:38.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.248+0000 7f5496d2a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f54900cc760 0x7f5490164d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:38.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.248+0000 7f5496d2a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54900ccc10 con 0x7f54900caf60 2023-11-28T18:23:38.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.248+0000 7f5496d2a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549006bf50 con 0x7f5490159f40 2023-11-28T18:23:38.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.248+0000 7f5496d2a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54900cb3c0 con 0x7f54900cc760 2023-11-28T18:23:38.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.252+0000 7f54952c7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f54900cc760 0x7f5490164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:38.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.252+0000 7f5487fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f54900caf60 0x7f54900ad850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51430/0 (socket says 172.21.15.105:51430) 2023-11-28T18:23:38.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.252+0000 7f5487fff640 1 -- 172.21.15.105:0/4202760848 learned_addr learned my addr 172.21.15.105:0/4202760848 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:38.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.252+0000 7f54952c7640 1 -- 172.21.15.105:0/4202760848 >> v1:172.21.15.148:6789/0 conn(0x7f5490159f40 legacy=0x7f54900ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:38.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.252+0000 7f54952c7640 1 -- 172.21.15.105:0/4202760848 >> v1:172.21.15.105:6789/0 conn(0x7f54900caf60 legacy=0x7f54900ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:38.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.252+0000 7f54877fe640 1 -- 172.21.15.105:0/4202760848 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791003364 0 0) 0x7f54900ccc10 con 0x7f54900caf60 2023-11-28T18:23:38.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.252+0000 7f54952c7640 1 -- 172.21.15.105:0/4202760848 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549015a320 con 0x7f54900cc760 2023-11-28T18:23:38.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.252+0000 7f54952c7640 1 --2- 172.21.15.105:0/4202760848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54900cc760 0x7f5490164d90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5488009d90 tx=0x7f548801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911512cd38b8b69e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:38.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.252+0000 7f54877fe640 1 -- 172.21.15.105:0/4202760848 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54880a5950 con 0x7f54900cc760 2023-11-28T18:23:38.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.252+0000 7f54877fe640 1 -- 172.21.15.105:0/4202760848 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54880a5ab0 con 0x7f54900cc760 2023-11-28T18:23:38.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.252+0000 7f54877fe640 1 -- 172.21.15.105:0/4202760848 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54880a5e40 con 0x7f54900cc760 2023-11-28T18:23:38.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.256+0000 7f5496d2a640 1 -- 172.21.15.105:0/4202760848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54900cc760 msgr2=0x7f5490164d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:38.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.256+0000 7f5496d2a640 1 --2- 172.21.15.105:0/4202760848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54900cc760 0x7f5490164d90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5488009d90 tx=0x7f548801c890 comp rx=0 tx=0).stop 2023-11-28T18:23:38.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.256+0000 7f5496d2a640 1 -- 172.21.15.105:0/4202760848 shutdown_connections 2023-11-28T18:23:38.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.256+0000 7f5496d2a640 1 --2- 172.21.15.105:0/4202760848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54900cc760 0x7f5490164d90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:38.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.256+0000 7f5496d2a640 1 -- 172.21.15.105:0/4202760848 >> 172.21.15.105:0/4202760848 conn(0x7f54900ceb90 msgr2=0x7f54900cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:38.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.256+0000 7f5496d2a640 1 -- 172.21.15.105:0/4202760848 shutdown_connections 2023-11-28T18:23:38.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.256+0000 7f5496d2a640 1 -- 172.21.15.105:0/4202760848 wait complete. 2023-11-28T18:23:38.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.256+0000 7f5496d2a640 1 Processor -- start 2023-11-28T18:23:38.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.256+0000 7f5496d2a640 1 -- start start 2023-11-28T18:23:38.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.256+0000 7f5496d2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54900caf60 0x7f5490154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:38.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.256+0000 7f5496d2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54900cc760 0x7f54900cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:38.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.256+0000 7f5496d2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5490159f40 0x7f54900cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:38.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.256+0000 7f5496d2a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54901570c0 con 0x7f54900cc760 2023-11-28T18:23:38.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.256+0000 7f5496d2a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549014a8c0 con 0x7f54900caf60 2023-11-28T18:23:38.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.256+0000 7f5496d2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549014a390 con 0x7f5490159f40 2023-11-28T18:23:38.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.256+0000 7f5487fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54900cc760 0x7f54900cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:38.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.256+0000 7f5487fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54900cc760 0x7f54900cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59104/0 (socket says 172.21.15.105:59104) 2023-11-28T18:23:38.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.256+0000 7f5487fff640 1 -- 172.21.15.105:0/3787334331 learned_addr learned my addr 172.21.15.105:0/3787334331 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:38.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.256+0000 7f54952c7640 1 --2- 172.21.15.105:0/3787334331 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5490159f40 0x7f54900cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:38.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.256+0000 7f5494ac6640 1 --2- 172.21.15.105:0/3787334331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54900caf60 0x7f5490154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:38.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.256+0000 7f5487fff640 1 -- 172.21.15.105:0/3787334331 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5490159f40 msgr2=0x7f54900cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:38.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.256+0000 7f5487fff640 1 --2- 172.21.15.105:0/3787334331 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5490159f40 0x7f54900cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:38.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.260+0000 7f5487fff640 1 -- 172.21.15.105:0/3787334331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54900caf60 msgr2=0x7f5490154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:38.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.260+0000 7f5487fff640 1 --2- 172.21.15.105:0/3787334331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54900caf60 0x7f5490154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:38.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.260+0000 7f5487fff640 1 -- 172.21.15.105:0/3787334331 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5488009a70 con 0x7f54900cc760 2023-11-28T18:23:38.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.260+0000 7f5487fff640 1 --2- 172.21.15.105:0/3787334331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54900cc760 0x7f54900cb5a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f548000c4a0 tx=0x7f548000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:38.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.260+0000 7f54857fa640 1 -- 172.21.15.105:0/3787334331 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54800074e0 con 0x7f54900cc760 2023-11-28T18:23:38.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.260+0000 7f54857fa640 1 -- 172.21.15.105:0/3787334331 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5480007640 con 0x7f54900cc760 2023-11-28T18:23:38.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.260+0000 7f54857fa640 1 -- 172.21.15.105:0/3787334331 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480004a30 con 0x7f54900cc760 2023-11-28T18:23:38.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.260+0000 7f5496d2a640 1 -- 172.21.15.105:0/3787334331 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5490156de0 con 0x7f54900cc760 2023-11-28T18:23:38.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.260+0000 7f5496d2a640 1 -- 172.21.15.105:0/3787334331 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54901f9b30 con 0x7f54900cc760 2023-11-28T18:23:38.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.260+0000 7f5496d2a640 1 -- 172.21.15.105:0/3787334331 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54901f9690 con 0x7f54900cc760 2023-11-28T18:23:38.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.260+0000 7f54857fa640 1 -- 172.21.15.105:0/3787334331 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5480004b90 con 0x7f54900cc760 2023-11-28T18:23:38.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.260+0000 7f54857fa640 1 --2- 172.21.15.105:0/3787334331 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f547c0668d0 0x7f547c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:38.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.260+0000 7f54857fa640 1 -- 172.21.15.105:0/3787334331 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f54800839a0 con 0x7f54900cc760 2023-11-28T18:23:38.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.264+0000 7f5494ac6640 1 --2- 172.21.15.105:0/3787334331 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f547c0668d0 0x7f547c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:38.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.264+0000 7f5494ac6640 1 --2- 172.21.15.105:0/3787334331 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f547c0668d0 0x7f547c068d90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f548c000f70 tx=0x7f548c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:38.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.268+0000 7f54857fa640 1 -- 172.21.15.105:0/3787334331 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5480050910 con 0x7f54900cc760 2023-11-28T18:23:38.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:38.456+0000 7f5496d2a640 1 -- 172.21.15.105:0/3787334331 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f54901f9d50 con 0x7f54900cc760 2023-11-28T18:23:39.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.060+0000 7f54857fa640 1 -- 172.21.15.105:0/3787334331 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f5480018070 con 0x7f54900cc760 2023-11-28T18:23:39.064 INFO:teuthology.orchestra.run.smithi105.stderr:added data pool 4 to fsmap 2023-11-28T18:23:39.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.064+0000 7f5496d2a640 1 -- 172.21.15.105:0/3787334331 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f547c0668d0 msgr2=0x7f547c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.064+0000 7f5496d2a640 1 --2- 172.21.15.105:0/3787334331 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f547c0668d0 0x7f547c068d90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f548c000f70 tx=0x7f548c009040 comp rx=0 tx=0).stop 2023-11-28T18:23:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.064+0000 7f5496d2a640 1 -- 172.21.15.105:0/3787334331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54900cc760 msgr2=0x7f54900cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.064+0000 7f5496d2a640 1 --2- 172.21.15.105:0/3787334331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54900cc760 0x7f54900cb5a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f548000c4a0 tx=0x7f548000c960 comp rx=0 tx=0).stop 2023-11-28T18:23:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.064+0000 7f5496d2a640 1 -- 172.21.15.105:0/3787334331 shutdown_connections 2023-11-28T18:23:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.064+0000 7f5496d2a640 1 --2- 172.21.15.105:0/3787334331 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5490159f40 0x7f54900cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.064+0000 7f5496d2a640 1 --2- 172.21.15.105:0/3787334331 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f547c0668d0 0x7f547c068d90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.064+0000 7f5496d2a640 1 --2- 172.21.15.105:0/3787334331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54900cc760 0x7f54900cb5a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.064+0000 7f5496d2a640 1 --2- 172.21.15.105:0/3787334331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54900caf60 0x7f5490154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.064+0000 7f5496d2a640 1 -- 172.21.15.105:0/3787334331 >> 172.21.15.105:0/3787334331 conn(0x7f54900ceb90 msgr2=0x7f5490070ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.068+0000 7f5496d2a640 1 -- 172.21.15.105:0/3787334331 shutdown_connections 2023-11-28T18:23:39.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.068+0000 7f5496d2a640 1 -- 172.21.15.105:0/3787334331 wait complete. 2023-11-28T18:23:39.091 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:23:39.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e62b01640 1 Processor -- start 2023-11-28T18:23:39.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e62b01640 1 -- start start 2023-11-28T18:23:39.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e62b01640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3e5c0cdd80 0x7f3e5c06c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:39.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e62b01640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e5c0ce230 0x7f3e5c0cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:39.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e62b01640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e5c0b3c40 con 0x7f3e5c15d320 2023-11-28T18:23:39.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e62b01640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e5c06ca80 con 0x7f3e5c0cdd80 2023-11-28T18:23:39.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e62b01640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c15d700 con 0x7f3e5c0ce230 2023-11-28T18:23:39.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e6089d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3e5c15d320 0x7f3e5c0cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51434/0 (socket says 172.21.15.105:51434) 2023-11-28T18:23:39.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e6089d640 1 -- 172.21.15.105:0/136343452 learned_addr learned my addr 172.21.15.105:0/136343452 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:39.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e537fe640 1 -- 172.21.15.105:0/136343452 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2952512156 0 0) 0x7f3e5c0b3c40 con 0x7f3e5c15d320 2023-11-28T18:23:39.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e6109e640 1 --2- 172.21.15.105:0/136343452 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e5c0ce230 0x7f3e5c0cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e53fff640 1 --2- 172.21.15.105:0/136343452 >> v2:172.21.15.148:3300/0 conn(0x7f3e5c0cdd80 0x7f3e5c06c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e537fe640 1 -- 172.21.15.105:0/136343452 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e44003530 con 0x7f3e5c15d320 2023-11-28T18:23:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e537fe640 1 -- 172.21.15.105:0/136343452 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1433949138 0 0) 0x7f3e44003530 con 0x7f3e5c15d320 2023-11-28T18:23:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e537fe640 1 -- 172.21.15.105:0/136343452 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e5c0ce230 msgr2=0x7f3e5c0cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e537fe640 1 --2- 172.21.15.105:0/136343452 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e5c0ce230 0x7f3e5c0cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e537fe640 1 -- 172.21.15.105:0/136343452 >> v2:172.21.15.148:3300/0 conn(0x7f3e5c0cdd80 msgr2=0x7f3e5c06c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e537fe640 1 --2- 172.21.15.105:0/136343452 >> v2:172.21.15.148:3300/0 conn(0x7f3e5c0cdd80 0x7f3e5c06c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e537fe640 1 -- 172.21.15.105:0/136343452 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5c06c7a0 con 0x7f3e5c15d320 2023-11-28T18:23:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e537fe640 1 -- 172.21.15.105:0/136343452 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f3e58001680 con 0x7f3e5c15d320 2023-11-28T18:23:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e537fe640 1 -- 172.21.15.105:0/136343452 >> v1:172.21.15.105:6789/0 conn(0x7f3e5c15d320 legacy=0x7f3e5c0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e537fe640 1 --2- 172.21.15.105:0/136343452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e44003980 0x7f3e44003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e537fe640 1 --2- 172.21.15.105:0/136343452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e440047d0 0x7f3e44004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e537fe640 1 --2- 172.21.15.105:0/136343452 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e5c0ce230 0x7f3e44005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e537fe640 1 -- 172.21.15.105:0/136343452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c06ca80 con 0x7f3e44003980 2023-11-28T18:23:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e537fe640 1 -- 172.21.15.105:0/136343452 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c15d700 con 0x7f3e440047d0 2023-11-28T18:23:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e537fe640 1 -- 172.21.15.105:0/136343452 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4400b9e0 con 0x7f3e5c0ce230 2023-11-28T18:23:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e6089d640 1 --2- 172.21.15.105:0/136343452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e44003980 0x7f3e44003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e6109e640 1 --2- 172.21.15.105:0/136343452 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e5c0ce230 0x7f3e44005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e53fff640 1 --2- 172.21.15.105:0/136343452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e440047d0 0x7f3e44004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e6089d640 1 -- 172.21.15.105:0/136343452 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e5c0ce230 msgr2=0x7f3e44005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.228+0000 7f3e6089d640 1 --2- 172.21.15.105:0/136343452 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e5c0ce230 0x7f3e44005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.232+0000 7f3e6089d640 1 -- 172.21.15.105:0/136343452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e440047d0 msgr2=0x7f3e44004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.232+0000 7f3e6089d640 1 --2- 172.21.15.105:0/136343452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e440047d0 0x7f3e44004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.232+0000 7f3e6089d640 1 -- 172.21.15.105:0/136343452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e44003530 con 0x7f3e44003980 2023-11-28T18:23:39.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.232+0000 7f3e53fff640 1 --2- 172.21.15.105:0/136343452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e440047d0 0x7f3e44004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:39.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.232+0000 7f3e6089d640 1 --2- 172.21.15.105:0/136343452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e44003980 0x7f3e44003da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800c140 tx=0x7f3e5801cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2751e8e24aa621 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:39.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.232+0000 7f3e537fe640 1 -- 172.21.15.105:0/136343452 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e580058e0 con 0x7f3e44003980 2023-11-28T18:23:39.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.232+0000 7f3e537fe640 1 -- 172.21.15.105:0/136343452 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e58005a60 con 0x7f3e44003980 2023-11-28T18:23:39.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.232+0000 7f3e537fe640 1 -- 172.21.15.105:0/136343452 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e58005e10 con 0x7f3e44003980 2023-11-28T18:23:39.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.232+0000 7f3e62b01640 1 -- 172.21.15.105:0/136343452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e44003980 msgr2=0x7f3e44003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:39.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.232+0000 7f3e62b01640 1 --2- 172.21.15.105:0/136343452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e44003980 0x7f3e44003da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800c140 tx=0x7f3e5801cb80 comp rx=0 tx=0).stop 2023-11-28T18:23:39.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.232+0000 7f3e6109e640 1 -- 172.21.15.105:0/136343452 reap_dead start 2023-11-28T18:23:39.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.232+0000 7f3e62b01640 1 -- 172.21.15.105:0/136343452 shutdown_connections 2023-11-28T18:23:39.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.232+0000 7f3e62b01640 1 -- 172.21.15.105:0/136343452 >> 172.21.15.105:0/136343452 conn(0x7f3e5c0d0df0 msgr2=0x7f3e5c0d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:39.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.232+0000 7f3e62b01640 1 -- 172.21.15.105:0/136343452 shutdown_connections 2023-11-28T18:23:39.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.232+0000 7f3e62b01640 1 -- 172.21.15.105:0/136343452 wait complete. 2023-11-28T18:23:39.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e62b01640 1 Processor -- start 2023-11-28T18:23:39.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e62b01640 1 -- start start 2023-11-28T18:23:39.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e62b01640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e44003980 0x7f3e5c160150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:39.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e62b01640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e5c15a4d0 0x7f3e5c15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:39.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e62b01640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e5c148410 0x7f3e5c15b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:39.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e62b01640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0b6830 con 0x7f3e5c15a4d0 2023-11-28T18:23:39.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e62b01640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0b63a0 con 0x7f3e44003980 2023-11-28T18:23:39.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e62b01640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0b57d0 con 0x7f3e5c148410 2023-11-28T18:23:39.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e5c15a4d0 0x7f3e5c15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:39.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e5c15a4d0 0x7f3e5c15afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59130/0 (socket says 172.21.15.105:59130) 2023-11-28T18:23:39.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e53fff640 1 -- 172.21.15.105:0/428673035 learned_addr learned my addr 172.21.15.105:0/428673035 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:39.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e6109e640 1 --2- 172.21.15.105:0/428673035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e5c148410 0x7f3e5c15b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:39.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e6089d640 1 --2- 172.21.15.105:0/428673035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e44003980 0x7f3e5c160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:39.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e53fff640 1 -- 172.21.15.105:0/428673035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e5c148410 msgr2=0x7f3e5c15b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:39.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e53fff640 1 --2- 172.21.15.105:0/428673035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e5c148410 0x7f3e5c15b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e53fff640 1 -- 172.21.15.105:0/428673035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e44003980 msgr2=0x7f3e5c160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:39.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e53fff640 1 --2- 172.21.15.105:0/428673035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e44003980 0x7f3e5c160150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e53fff640 1 -- 172.21.15.105:0/428673035 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5c0b5a70 con 0x7f3e5c15a4d0 2023-11-28T18:23:39.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e53fff640 1 --2- 172.21.15.105:0/428673035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e5c15a4d0 0x7f3e5c15afe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c010870 tx=0x7f3e4c010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:39.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e517fa640 1 -- 172.21.15.105:0/428673035 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c004220 con 0x7f3e5c15a4d0 2023-11-28T18:23:39.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e517fa640 1 -- 172.21.15.105:0/428673035 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4c004380 con 0x7f3e5c15a4d0 2023-11-28T18:23:39.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e517fa640 1 -- 172.21.15.105:0/428673035 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c00eb60 con 0x7f3e5c15a4d0 2023-11-28T18:23:39.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.236+0000 7f3e62b01640 1 -- 172.21.15.105:0/428673035 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e5c0b5d50 con 0x7f3e5c15a4d0 2023-11-28T18:23:39.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.240+0000 7f3e62b01640 1 -- 172.21.15.105:0/428673035 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5c160690 con 0x7f3e5c15a4d0 2023-11-28T18:23:39.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.240+0000 7f3e62b01640 1 -- 172.21.15.105:0/428673035 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e5c06cf50 con 0x7f3e5c15a4d0 2023-11-28T18:23:39.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.240+0000 7f3e517fa640 1 -- 172.21.15.105:0/428673035 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3e4c00ecc0 con 0x7f3e5c15a4d0 2023-11-28T18:23:39.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.240+0000 7f3e517fa640 1 --2- 172.21.15.105:0/428673035 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3e48066900 0x7f3e48068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:39.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.240+0000 7f3e517fa640 1 -- 172.21.15.105:0/428673035 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3e4c084520 con 0x7f3e5c15a4d0 2023-11-28T18:23:39.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.240+0000 7f3e6089d640 1 --2- 172.21.15.105:0/428673035 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3e48066900 0x7f3e48068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:39.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.240+0000 7f3e6089d640 1 --2- 172.21.15.105:0/428673035 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3e48066900 0x7f3e48068dc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3e580aa000 tx=0x7f3e580019b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:39.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.244+0000 7f3e517fa640 1 -- 172.21.15.105:0/428673035 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3e4c051470 con 0x7f3e5c15a4d0 2023-11-28T18:23:39.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.424+0000 7f3e62b01640 1 -- 172.21.15.105:0/428673035 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e5c067570 con 0x7f3e5c15a4d0 2023-11-28T18:23:39.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.424+0000 7f3e517fa640 1 -- 172.21.15.105:0/428673035 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4918 (secure 0 0 0) 0x7f3e4c0044f0 con 0x7f3e5c15a4d0 2023-11-28T18:23:39.428 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:39.428 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/390529893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":390529893},{"type":"v1","addr":"172.21.15.148:6835","nonce":390529893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3924571887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3924571887},{"type":"v1","addr":"172.21.15.148:6833","nonce":3924571887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3271971772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3271971772},{"type":"v1","addr":"172.21.15.105:6835","nonce":3271971772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:23:33.288743+0000","modified":"2023-11-28T18:23:38.459146+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2504263238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2504263238},{"type":"v1","addr":"172.21.15.105:6837","nonce":2504263238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-28T18:23:39.428 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 7 2023-11-28T18:23:39.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.428+0000 7f3e62b01640 1 -- 172.21.15.105:0/428673035 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3e48066900 msgr2=0x7f3e48068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:39.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.428+0000 7f3e62b01640 1 --2- 172.21.15.105:0/428673035 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3e48066900 0x7f3e48068dc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3e580aa000 tx=0x7f3e580019b0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.428+0000 7f3e62b01640 1 -- 172.21.15.105:0/428673035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e5c15a4d0 msgr2=0x7f3e5c15afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:39.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.428+0000 7f3e62b01640 1 --2- 172.21.15.105:0/428673035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e5c15a4d0 0x7f3e5c15afe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c010870 tx=0x7f3e4c010d30 comp rx=0 tx=0).stop 2023-11-28T18:23:39.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.428+0000 7f3e62b01640 1 -- 172.21.15.105:0/428673035 shutdown_connections 2023-11-28T18:23:39.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.428+0000 7f3e62b01640 1 --2- 172.21.15.105:0/428673035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e5c148410 0x7f3e5c15b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.428+0000 7f3e62b01640 1 --2- 172.21.15.105:0/428673035 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3e48066900 0x7f3e48068dc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.428+0000 7f3e62b01640 1 --2- 172.21.15.105:0/428673035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e5c15a4d0 0x7f3e5c15afe0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.428+0000 7f3e62b01640 1 --2- 172.21.15.105:0/428673035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e44003980 0x7f3e5c160150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.428+0000 7f3e62b01640 1 -- 172.21.15.105:0/428673035 >> 172.21.15.105:0/428673035 conn(0x7f3e5c0d0df0 msgr2=0x7f3e5c06eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:39.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.428+0000 7f3e62b01640 1 -- 172.21.15.105:0/428673035 shutdown_connections 2023-11-28T18:23:39.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.428+0000 7f3e62b01640 1 -- 172.21.15.105:0/428673035 wait complete. 2023-11-28T18:23:39.448 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:23:39.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.580+0000 7f53cc31d640 1 Processor -- start 2023-11-28T18:23:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.580+0000 7f53cc31d640 1 -- start start 2023-11-28T18:23:39.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.584+0000 7f53cc31d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53c4159610 0x7f53c40ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:39.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.584+0000 7f53cc31d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f53c40cc760 0x7f53c40ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:39.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.584+0000 7f53cc31d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c40cde30 con 0x7f53c406c7e0 2023-11-28T18:23:39.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.584+0000 7f53cc31d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c41599f0 con 0x7f53c40cc760 2023-11-28T18:23:39.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.584+0000 7f53cc31d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c406de20 con 0x7f53c4159610 2023-11-28T18:23:39.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.584+0000 7f53c98b8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f53c40cc760 0x7f53c40ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:39.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.584+0000 7f53ca0b9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53c4159610 0x7f53c40ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:39.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.584+0000 7f53c98b8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f53c40cc760 0x7f53c40ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:43274/0 (socket says 172.21.15.105:43274) 2023-11-28T18:23:39.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.584+0000 7f53c98b8640 1 -- 172.21.15.105:0/3290976100 learned_addr learned my addr 172.21.15.105:0/3290976100 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:39.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.584+0000 7f53ca0b9640 1 -- 172.21.15.105:0/3290976100 >> v2:172.21.15.148:3300/0 conn(0x7f53c40cc760 msgr2=0x7f53c40ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:39.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.584+0000 7f53ca0b9640 1 --2- 172.21.15.105:0/3290976100 >> v2:172.21.15.148:3300/0 conn(0x7f53c40cc760 0x7f53c40ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.584+0000 7f53ca0b9640 1 -- 172.21.15.105:0/3290976100 >> v1:172.21.15.148:6789/0 conn(0x7f53c406c7e0 legacy=0x7f53c40abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:39.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.584+0000 7f53ca0b9640 1 -- 172.21.15.105:0/3290976100 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c40cd220 con 0x7f53c4159610 2023-11-28T18:23:39.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.588+0000 7f53ca0b9640 1 --2- 172.21.15.105:0/3290976100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53c4159610 0x7f53c40ce2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f53c000a960 tx=0x7f53c40acca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eb2af5baf64e0dd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:39.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.588+0000 7f53c90b7640 1 -- 172.21.15.105:0/3290976100 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c00a68d0 con 0x7f53c4159610 2023-11-28T18:23:39.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.588+0000 7f53c90b7640 1 -- 172.21.15.105:0/3290976100 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c00a6a30 con 0x7f53c4159610 2023-11-28T18:23:39.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.588+0000 7f53c90b7640 1 -- 172.21.15.105:0/3290976100 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c00a6dc0 con 0x7f53c4159610 2023-11-28T18:23:39.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.588+0000 7f53cc31d640 1 -- 172.21.15.105:0/3290976100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53c4159610 msgr2=0x7f53c40ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:39.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.588+0000 7f53cc31d640 1 --2- 172.21.15.105:0/3290976100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53c4159610 0x7f53c40ce2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f53c000a960 tx=0x7f53c40acca0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.588+0000 7f53cc31d640 1 -- 172.21.15.105:0/3290976100 shutdown_connections 2023-11-28T18:23:39.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.588+0000 7f53cc31d640 1 --2- 172.21.15.105:0/3290976100 >> v2:172.21.15.148:3300/0 conn(0x7f53c40cc760 0x7f53c40ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.588+0000 7f53cc31d640 1 --2- 172.21.15.105:0/3290976100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53c4159610 0x7f53c40ce2c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.588+0000 7f53cc31d640 1 -- 172.21.15.105:0/3290976100 >> 172.21.15.105:0/3290976100 conn(0x7f53c40ceb90 msgr2=0x7f53c40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:39.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.588+0000 7f53cc31d640 1 -- 172.21.15.105:0/3290976100 shutdown_connections 2023-11-28T18:23:39.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.588+0000 7f53cc31d640 1 -- 172.21.15.105:0/3290976100 wait complete. 2023-11-28T18:23:39.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53cc31d640 1 Processor -- start 2023-11-28T18:23:39.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53cc31d640 1 -- start start 2023-11-28T18:23:39.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53cc31d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53c406c7e0 0x7f53c4159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53cc31d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53c40cc760 0x7f53c41517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53cc31d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53c4159610 0x7f53c4148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53cc31d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c415a1e0 con 0x7f53c406c7e0 2023-11-28T18:23:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53cc31d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c4151cf0 con 0x7f53c4159610 2023-11-28T18:23:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53cc31d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c4155100 con 0x7f53c40cc760 2023-11-28T18:23:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53ca0b9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53c406c7e0 0x7f53c4159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53ca0b9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53c406c7e0 0x7f53c4159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59138/0 (socket says 172.21.15.105:59138) 2023-11-28T18:23:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53ca0b9640 1 -- 172.21.15.105:0/2908192379 learned_addr learned my addr 172.21.15.105:0/2908192379 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53c98b8640 1 --2- 172.21.15.105:0/2908192379 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53c40cc760 0x7f53c41517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53ca8ba640 1 --2- 172.21.15.105:0/2908192379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53c4159610 0x7f53c4148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53ca0b9640 1 -- 172.21.15.105:0/2908192379 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53c40cc760 msgr2=0x7f53c41517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:39.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53ca0b9640 1 --2- 172.21.15.105:0/2908192379 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53c40cc760 0x7f53c41517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53ca0b9640 1 -- 172.21.15.105:0/2908192379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53c4159610 msgr2=0x7f53c4148950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:39.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53ca0b9640 1 --2- 172.21.15.105:0/2908192379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53c4159610 0x7f53c4148950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53ca0b9640 1 -- 172.21.15.105:0/2908192379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c000a5c0 con 0x7f53c406c7e0 2023-11-28T18:23:39.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53ca0b9640 1 --2- 172.21.15.105:0/2908192379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53c406c7e0 0x7f53c4159ca0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f53c001b450 tx=0x7f53c001a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:39.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53baffd640 1 -- 172.21.15.105:0/2908192379 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c00ae850 con 0x7f53c406c7e0 2023-11-28T18:23:39.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53baffd640 1 -- 172.21.15.105:0/2908192379 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c00ae9d0 con 0x7f53c406c7e0 2023-11-28T18:23:39.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53baffd640 1 -- 172.21.15.105:0/2908192379 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c00aed80 con 0x7f53c406c7e0 2023-11-28T18:23:39.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53cc31d640 1 -- 172.21.15.105:0/2908192379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c406d180 con 0x7f53c406c7e0 2023-11-28T18:23:39.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53cc31d640 1 -- 172.21.15.105:0/2908192379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c41547d0 con 0x7f53c406c7e0 2023-11-28T18:23:39.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.592+0000 7f53cc31d640 1 -- 172.21.15.105:0/2908192379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c4067570 con 0x7f53c406c7e0 2023-11-28T18:23:39.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.596+0000 7f53baffd640 1 -- 172.21.15.105:0/2908192379 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f53c00b9480 con 0x7f53c406c7e0 2023-11-28T18:23:39.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.596+0000 7f53baffd640 1 --2- 172.21.15.105:0/2908192379 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f53a80668d0 0x7f53a8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:39.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.596+0000 7f53baffd640 1 -- 172.21.15.105:0/2908192379 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f53c011f5c0 con 0x7f53c406c7e0 2023-11-28T18:23:39.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.596+0000 7f53c98b8640 1 --2- 172.21.15.105:0/2908192379 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f53a80668d0 0x7f53a8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:39.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.596+0000 7f53c98b8640 1 --2- 172.21.15.105:0/2908192379 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f53a80668d0 0x7f53a8068d90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f53c4157d30 tx=0x7f53b4009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:39.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.600+0000 7f53baffd640 1 -- 172.21.15.105:0/2908192379 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f53c00ec640 con 0x7f53c406c7e0 2023-11-28T18:23:39.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.748+0000 7f53cc31d640 1 -- 172.21.15.105:0/2908192379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53c4158e40 con 0x7f53c406c7e0 2023-11-28T18:23:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.748+0000 7f53baffd640 1 -- 172.21.15.105:0/2908192379 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7f53c00efc60 con 0x7f53c406c7e0 2023-11-28T18:23:39.752 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:39.753 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":24,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:23:38.458954+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:23:39.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.752+0000 7f53cc31d640 1 -- 172.21.15.105:0/2908192379 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f53a80668d0 msgr2=0x7f53a8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:39.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.752+0000 7f53cc31d640 1 --2- 172.21.15.105:0/2908192379 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f53a80668d0 0x7f53a8068d90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f53c4157d30 tx=0x7f53b4009240 comp rx=0 tx=0).stop 2023-11-28T18:23:39.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.752+0000 7f53cc31d640 1 -- 172.21.15.105:0/2908192379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53c406c7e0 msgr2=0x7f53c4159ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:39.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.752+0000 7f53cc31d640 1 --2- 172.21.15.105:0/2908192379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53c406c7e0 0x7f53c4159ca0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f53c001b450 tx=0x7f53c001a5c0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.752+0000 7f53cc31d640 1 -- 172.21.15.105:0/2908192379 shutdown_connections 2023-11-28T18:23:39.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.752+0000 7f53cc31d640 1 --2- 172.21.15.105:0/2908192379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53c4159610 0x7f53c4148950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.752+0000 7f53cc31d640 1 --2- 172.21.15.105:0/2908192379 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53c40cc760 0x7f53c41517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.752+0000 7f53cc31d640 1 --2- 172.21.15.105:0/2908192379 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f53a80668d0 0x7f53a8068d90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.752+0000 7f53cc31d640 1 --2- 172.21.15.105:0/2908192379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53c406c7e0 0x7f53c4159ca0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.752+0000 7f53cc31d640 1 -- 172.21.15.105:0/2908192379 >> 172.21.15.105:0/2908192379 conn(0x7f53c40ceb90 msgr2=0x7f53c4162690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.752+0000 7f53cc31d640 1 -- 172.21.15.105:0/2908192379 shutdown_connections 2023-11-28T18:23:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.752+0000 7f53cc31d640 1 -- 172.21.15.105:0/2908192379 wait complete. 2023-11-28T18:23:39.774 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:23:39.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.908+0000 7fd4928c9640 1 Processor -- start 2023-11-28T18:23:39.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.908+0000 7fd4928c9640 1 -- start start 2023-11-28T18:23:39.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.908+0000 7fd4928c9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd48c15d4f0 0x7fd48c0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:39.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.908+0000 7fd4928c9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd48c0d3c30 0x7fd48c0d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:39.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.908+0000 7fd4928c9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd48c0b25b0 con 0x7fd48c0d1890 2023-11-28T18:23:39.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.908+0000 7fd4928c9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd48c0b2900 con 0x7fd48c15d4f0 2023-11-28T18:23:39.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.908+0000 7fd4928c9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd48c0d1cc0 con 0x7fd48c0d3c30 2023-11-28T18:23:39.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.908+0000 7fd48bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd48c15d4f0 0x7fd48c0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:39.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.908+0000 7fd48bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd48c15d4f0 0x7fd48c0d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59144/0 (socket says 172.21.15.105:59144) 2023-11-28T18:23:39.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.908+0000 7fd48bfff640 1 -- 172.21.15.105:0/3434128210 learned_addr learned my addr 172.21.15.105:0/3434128210 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:39.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.908+0000 7fd48b7fe640 1 --2- 172.21.15.105:0/3434128210 >> v2:172.21.15.148:3300/0 conn(0x7fd48c0d3c30 0x7fd48c0d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:39.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.908+0000 7fd48bfff640 1 -- 172.21.15.105:0/3434128210 >> v2:172.21.15.148:3300/0 conn(0x7fd48c0d3c30 msgr2=0x7fd48c0d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:39.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.908+0000 7fd48bfff640 1 --2- 172.21.15.105:0/3434128210 >> v2:172.21.15.148:3300/0 conn(0x7fd48c0d3c30 0x7fd48c0d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.908+0000 7fd48bfff640 1 -- 172.21.15.105:0/3434128210 >> v1:172.21.15.148:6789/0 conn(0x7fd48c0d1890 legacy=0x7fd48c0cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:39.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.908+0000 7fd48bfff640 1 -- 172.21.15.105:0/3434128210 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48c0cea30 con 0x7fd48c15d4f0 2023-11-28T18:23:39.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.912+0000 7fd48bfff640 1 --2- 172.21.15.105:0/3434128210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd48c15d4f0 0x7fd48c0d0e10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd48c08c430 tx=0x7fd47c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1337500adbf3198b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:39.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.912+0000 7fd48affd640 1 -- 172.21.15.105:0/3434128210 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c0ab640 con 0x7fd48c15d4f0 2023-11-28T18:23:39.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.912+0000 7fd48affd640 1 -- 172.21.15.105:0/3434128210 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd47c0a7070 con 0x7fd48c15d4f0 2023-11-28T18:23:39.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.912+0000 7fd48affd640 1 -- 172.21.15.105:0/3434128210 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c0ab9c0 con 0x7fd48c15d4f0 2023-11-28T18:23:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.912+0000 7fd4928c9640 1 -- 172.21.15.105:0/3434128210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd48c15d4f0 msgr2=0x7fd48c0d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.912+0000 7fd4928c9640 1 --2- 172.21.15.105:0/3434128210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd48c15d4f0 0x7fd48c0d0e10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd48c08c430 tx=0x7fd47c01c440 comp rx=0 tx=0).stop 2023-11-28T18:23:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.912+0000 7fd4928c9640 1 -- 172.21.15.105:0/3434128210 shutdown_connections 2023-11-28T18:23:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.912+0000 7fd4928c9640 1 --2- 172.21.15.105:0/3434128210 >> v2:172.21.15.148:3300/0 conn(0x7fd48c0d3c30 0x7fd48c0d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.912+0000 7fd4928c9640 1 --2- 172.21.15.105:0/3434128210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd48c15d4f0 0x7fd48c0d0e10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.912+0000 7fd4928c9640 1 -- 172.21.15.105:0/3434128210 >> 172.21.15.105:0/3434128210 conn(0x7fd48c0d3030 msgr2=0x7fd48c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.912+0000 7fd4928c9640 1 -- 172.21.15.105:0/3434128210 shutdown_connections 2023-11-28T18:23:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.912+0000 7fd4928c9640 1 -- 172.21.15.105:0/3434128210 wait complete. 2023-11-28T18:23:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd4928c9640 1 Processor -- start 2023-11-28T18:23:39.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd4928c9640 1 -- start start 2023-11-28T18:23:39.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd4928c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48c0d1890 0x7fd48c14c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:39.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd4928c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd48c0d3c30 0x7fd48c0c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:39.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd4928c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd48c15d4f0 0x7fd48c0c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:39.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd4928c9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c1585a0 con 0x7fd48c0d3c30 2023-11-28T18:23:39.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd4928c9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c1579b0 con 0x7fd48c0d1890 2023-11-28T18:23:39.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd4928c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c15dd00 con 0x7fd48c15d4f0 2023-11-28T18:23:39.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd48b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd48c0d3c30 0x7fd48c0c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:39.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd48b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd48c0d3c30 0x7fd48c0c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59150/0 (socket says 172.21.15.105:59150) 2023-11-28T18:23:39.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd48b7fe640 1 -- 172.21.15.105:0/3336505493 learned_addr learned my addr 172.21.15.105:0/3336505493 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:39.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd48bfff640 1 --2- 172.21.15.105:0/3336505493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48c0d1890 0x7fd48c14c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:39.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd490e66640 1 --2- 172.21.15.105:0/3336505493 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd48c15d4f0 0x7fd48c0c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:39.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd48b7fe640 1 -- 172.21.15.105:0/3336505493 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd48c15d4f0 msgr2=0x7fd48c0c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:39.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd48b7fe640 1 --2- 172.21.15.105:0/3336505493 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd48c15d4f0 0x7fd48c0c9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd48b7fe640 1 -- 172.21.15.105:0/3336505493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48c0d1890 msgr2=0x7fd48c14c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:39.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd48b7fe640 1 --2- 172.21.15.105:0/3336505493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48c0d1890 0x7fd48c14c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:39.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd48b7fe640 1 -- 172.21.15.105:0/3336505493 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c0094d0 con 0x7fd48c0d3c30 2023-11-28T18:23:39.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd48b7fe640 1 --2- 172.21.15.105:0/3336505493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd48c0d3c30 0x7fd48c0c94e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd48000e8a0 tx=0x7fd48000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:39.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd488ff9640 1 -- 172.21.15.105:0/3336505493 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480009510 con 0x7fd48c0d3c30 2023-11-28T18:23:39.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd488ff9640 1 -- 172.21.15.105:0/3336505493 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48000c940 con 0x7fd48c0d3c30 2023-11-28T18:23:39.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd488ff9640 1 -- 172.21.15.105:0/3336505493 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48000ccb0 con 0x7fd48c0d3c30 2023-11-28T18:23:39.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.916+0000 7fd4928c9640 1 -- 172.21.15.105:0/3336505493 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48c14ca20 con 0x7fd48c0d3c30 2023-11-28T18:23:39.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.920+0000 7fd4928c9640 1 -- 172.21.15.105:0/3336505493 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48c157ee0 con 0x7fd48c0d3c30 2023-11-28T18:23:39.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.920+0000 7fd4928c9640 1 -- 172.21.15.105:0/3336505493 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd48c158810 con 0x7fd48c0d3c30 2023-11-28T18:23:39.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.920+0000 7fd488ff9640 1 -- 172.21.15.105:0/3336505493 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd480013470 con 0x7fd48c0d3c30 2023-11-28T18:23:39.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.920+0000 7fd488ff9640 1 --2- 172.21.15.105:0/3336505493 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd4780668d0 0x7fd478068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:39.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.920+0000 7fd488ff9640 1 -- 172.21.15.105:0/3336505493 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd4800847b0 con 0x7fd48c0d3c30 2023-11-28T18:23:39.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.920+0000 7fd48bfff640 1 --2- 172.21.15.105:0/3336505493 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd4780668d0 0x7fd478068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:39.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.920+0000 7fd48bfff640 1 --2- 172.21.15.105:0/3336505493 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd4780668d0 0x7fd478068d90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0af000 tx=0x7fd47c0b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:39.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:39.924+0000 7fd488ff9640 1 -- 172.21.15.105:0/3336505493 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd4800517a0 con 0x7fd48c0d3c30 2023-11-28T18:23:40.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.080+0000 7fd4928c9640 1 -- 172.21.15.105:0/3336505493 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd48c067570 con 0x7fd48c0d3c30 2023-11-28T18:23:40.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.080+0000 7fd488ff9640 1 -- 172.21.15.105:0/3336505493 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7fd480054dc0 con 0x7fd48c0d3c30 2023-11-28T18:23:40.083 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:40.083 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":24,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:23:38.458954+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:23:40.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.084+0000 7fd4928c9640 1 -- 172.21.15.105:0/3336505493 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd4780668d0 msgr2=0x7fd478068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:40.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.084+0000 7fd4928c9640 1 --2- 172.21.15.105:0/3336505493 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd4780668d0 0x7fd478068d90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0af000 tx=0x7fd47c0b6040 comp rx=0 tx=0).stop 2023-11-28T18:23:40.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.084+0000 7fd4928c9640 1 -- 172.21.15.105:0/3336505493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd48c0d3c30 msgr2=0x7fd48c0c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:40.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.084+0000 7fd4928c9640 1 --2- 172.21.15.105:0/3336505493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd48c0d3c30 0x7fd48c0c94e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd48000e8a0 tx=0x7fd48000eca0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.084+0000 7fd4928c9640 1 -- 172.21.15.105:0/3336505493 shutdown_connections 2023-11-28T18:23:40.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.084+0000 7fd4928c9640 1 --2- 172.21.15.105:0/3336505493 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd48c15d4f0 0x7fd48c0c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.084+0000 7fd4928c9640 1 --2- 172.21.15.105:0/3336505493 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd4780668d0 0x7fd478068d90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.084+0000 7fd4928c9640 1 --2- 172.21.15.105:0/3336505493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd48c0d3c30 0x7fd48c0c94e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.084+0000 7fd4928c9640 1 --2- 172.21.15.105:0/3336505493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48c0d1890 0x7fd48c14c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.084+0000 7fd4928c9640 1 -- 172.21.15.105:0/3336505493 >> 172.21.15.105:0/3336505493 conn(0x7fd48c0d3030 msgr2=0x7fd48c156720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:40.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.084+0000 7fd4928c9640 1 -- 172.21.15.105:0/3336505493 shutdown_connections 2023-11-28T18:23:40.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.084+0000 7fd4928c9640 1 -- 172.21.15.105:0/3336505493 wait complete. 2023-11-28T18:23:40.118 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:23:40.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.252+0000 7f5e005aa640 1 Processor -- start 2023-11-28T18:23:40.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.252+0000 7f5e005aa640 1 -- start start 2023-11-28T18:23:40.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.256+0000 7f5e005aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5df80caf60 0x7f5df80cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:40.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.256+0000 7f5e005aa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5df80ad850 0x7f5df80701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:40.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.256+0000 7f5e005aa640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5df806bf50 con 0x7f5df8159f40 2023-11-28T18:23:40.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.256+0000 7f5e005aa640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df80cb3c0 con 0x7f5df80ad850 2023-11-28T18:23:40.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.256+0000 7f5e005aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df815a320 con 0x7f5df80caf60 2023-11-28T18:23:40.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.256+0000 7f5dfeb47640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5df80ad850 0x7f5df80701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:40.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.256+0000 7f5dfdb45640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5df80caf60 0x7f5df80cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:40.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.256+0000 7f5dfeb47640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5df80ad850 0x7f5df80701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:43314/0 (socket says 172.21.15.105:43314) 2023-11-28T18:23:40.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.256+0000 7f5dfdb45640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5df80caf60 0x7f5df80cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:38636/0 (socket says 172.21.15.105:38636) 2023-11-28T18:23:40.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.256+0000 7f5dfeb47640 1 -- 172.21.15.105:0/1232879952 learned_addr learned my addr 172.21.15.105:0/1232879952 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:40.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.256+0000 7f5dfe346640 1 --1- 172.21.15.105:0/1232879952 >> v1:172.21.15.148:6789/0 conn(0x7f5df8159f40 0x7f5df80ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:44190/0 (socket says 172.21.15.105:44190) 2023-11-28T18:23:40.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.256+0000 7f5dfd344640 1 -- 172.21.15.105:0/1232879952 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058854104 0 0) 0x7f5df806bf50 con 0x7f5df8159f40 2023-11-28T18:23:40.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.256+0000 7f5dfd344640 1 -- 172.21.15.105:0/1232879952 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5de4003530 con 0x7f5df8159f40 2023-11-28T18:23:40.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.256+0000 7f5dfdb45640 1 -- 172.21.15.105:0/1232879952 >> v2:172.21.15.148:3300/0 conn(0x7f5df80ad850 msgr2=0x7f5df80701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:40.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.256+0000 7f5dfdb45640 1 --2- 172.21.15.105:0/1232879952 >> v2:172.21.15.148:3300/0 conn(0x7f5df80ad850 0x7f5df80701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.256+0000 7f5dfdb45640 1 -- 172.21.15.105:0/1232879952 >> v1:172.21.15.148:6789/0 conn(0x7f5df8159f40 legacy=0x7f5df80ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:40.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.256+0000 7f5dfdb45640 1 -- 172.21.15.105:0/1232879952 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df8071380 con 0x7f5df80caf60 2023-11-28T18:23:40.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.256+0000 7f5dfeb47640 1 --2- 172.21.15.105:0/1232879952 >> v2:172.21.15.148:3300/0 conn(0x7f5df80ad850 0x7f5df80701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:40.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.256+0000 7f5dfdb45640 1 --2- 172.21.15.105:0/1232879952 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5df80caf60 0x7f5df80cc760 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5de80097c0 tx=0x7f5de801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebec3e9e1658b7ce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:40.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.260+0000 7f5dfd344640 1 -- 172.21.15.105:0/1232879952 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80ac620 con 0x7f5df80caf60 2023-11-28T18:23:40.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.260+0000 7f5dfd344640 1 -- 172.21.15.105:0/1232879952 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de80ac780 con 0x7f5df80caf60 2023-11-28T18:23:40.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.260+0000 7f5dfd344640 1 -- 172.21.15.105:0/1232879952 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80acb10 con 0x7f5df80caf60 2023-11-28T18:23:40.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.260+0000 7f5e005aa640 1 -- 172.21.15.105:0/1232879952 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5df80caf60 msgr2=0x7f5df80cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:40.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.260+0000 7f5e005aa640 1 --2- 172.21.15.105:0/1232879952 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5df80caf60 0x7f5df80cc760 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5de80097c0 tx=0x7f5de801c3c0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.260+0000 7f5e005aa640 1 -- 172.21.15.105:0/1232879952 shutdown_connections 2023-11-28T18:23:40.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.260+0000 7f5e005aa640 1 --2- 172.21.15.105:0/1232879952 >> v2:172.21.15.148:3300/0 conn(0x7f5df80ad850 0x7f5df80701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.260+0000 7f5e005aa640 1 --2- 172.21.15.105:0/1232879952 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5df80caf60 0x7f5df80cc760 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.260+0000 7f5e005aa640 1 -- 172.21.15.105:0/1232879952 >> 172.21.15.105:0/1232879952 conn(0x7f5df80ceb90 msgr2=0x7f5df80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:40.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.260+0000 7f5e005aa640 1 -- 172.21.15.105:0/1232879952 shutdown_connections 2023-11-28T18:23:40.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.260+0000 7f5e005aa640 1 -- 172.21.15.105:0/1232879952 wait complete. 2023-11-28T18:23:40.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.260+0000 7f5e005aa640 1 Processor -- start 2023-11-28T18:23:40.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.260+0000 7f5e005aa640 1 -- start start 2023-11-28T18:23:40.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.260+0000 7f5e005aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5df80ad850 0x7f5df814aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:40.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.260+0000 7f5e005aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5df80caf60 0x7f5df815a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:40.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.260+0000 7f5e005aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5df8159f40 0x7f5df80ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:40.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.264+0000 7f5e005aa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df814bfb0 con 0x7f5df80caf60 2023-11-28T18:23:40.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.264+0000 7f5e005aa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df815aa60 con 0x7f5df8159f40 2023-11-28T18:23:40.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.264+0000 7f5e005aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df814b870 con 0x7f5df80ad850 2023-11-28T18:23:40.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.264+0000 7f5dfdb45640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5df80caf60 0x7f5df815a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:40.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.264+0000 7f5dfdb45640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5df80caf60 0x7f5df815a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59160/0 (socket says 172.21.15.105:59160) 2023-11-28T18:23:40.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.264+0000 7f5dfdb45640 1 -- 172.21.15.105:0/2296776378 learned_addr learned my addr 172.21.15.105:0/2296776378 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:40.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.264+0000 7f5dfeb47640 1 --2- 172.21.15.105:0/2296776378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5df8159f40 0x7f5df80ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:40.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.264+0000 7f5dfe346640 1 --2- 172.21.15.105:0/2296776378 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5df80ad850 0x7f5df814aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:40.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.264+0000 7f5dfdb45640 1 -- 172.21.15.105:0/2296776378 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5df80ad850 msgr2=0x7f5df814aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:40.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.264+0000 7f5dfdb45640 1 --2- 172.21.15.105:0/2296776378 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5df80ad850 0x7f5df814aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.264+0000 7f5dfdb45640 1 -- 172.21.15.105:0/2296776378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5df8159f40 msgr2=0x7f5df80ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.264+0000 7f5dfdb45640 1 --2- 172.21.15.105:0/2296776378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5df8159f40 0x7f5df80ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.264+0000 7f5dfdb45640 1 -- 172.21.15.105:0/2296776378 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df0009b30 con 0x7f5df80caf60 2023-11-28T18:23:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.264+0000 7f5dfdb45640 1 --2- 172.21.15.105:0/2296776378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5df80caf60 0x7f5df815a520 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5de8019ff0 tx=0x7f5de8019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:40.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.264+0000 7f5deeffd640 1 -- 172.21.15.105:0/2296776378 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80b5400 con 0x7f5df80caf60 2023-11-28T18:23:40.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.264+0000 7f5deeffd640 1 -- 172.21.15.105:0/2296776378 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de80b5560 con 0x7f5df80caf60 2023-11-28T18:23:40.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.264+0000 7f5e005aa640 1 -- 172.21.15.105:0/2296776378 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de8009420 con 0x7f5df80caf60 2023-11-28T18:23:40.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.264+0000 7f5deeffd640 1 -- 172.21.15.105:0/2296776378 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80b5910 con 0x7f5df80caf60 2023-11-28T18:23:40.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.264+0000 7f5e005aa640 1 -- 172.21.15.105:0/2296776378 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df814c200 con 0x7f5df80caf60 2023-11-28T18:23:40.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.264+0000 7f5e005aa640 1 -- 172.21.15.105:0/2296776378 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df8067570 con 0x7f5df80caf60 2023-11-28T18:23:40.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.264+0000 7f5deeffd640 1 -- 172.21.15.105:0/2296776378 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5de80b3040 con 0x7f5df80caf60 2023-11-28T18:23:40.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.268+0000 7f5deeffd640 1 --2- 172.21.15.105:0/2296776378 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5ddc0668d0 0x7f5ddc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:40.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.268+0000 7f5deeffd640 1 -- 172.21.15.105:0/2296776378 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5de811e730 con 0x7f5df80caf60 2023-11-28T18:23:40.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.268+0000 7f5dfe346640 1 --2- 172.21.15.105:0/2296776378 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5ddc0668d0 0x7f5ddc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:40.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.268+0000 7f5dfe346640 1 --2- 172.21.15.105:0/2296776378 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5ddc0668d0 0x7f5ddc068d90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5df4000f70 tx=0x7f5df4007d00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:40.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.272+0000 7f5deeffd640 1 -- 172.21.15.105:0/2296776378 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5de80eb7c0 con 0x7f5df80caf60 2023-11-28T18:23:40.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.452+0000 7f5e005aa640 1 -- 172.21.15.105:0/2296776378 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5df80cf950 con 0x7f5df80caf60 2023-11-28T18:23:40.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.452+0000 7f5deeffd640 1 -- 172.21.15.105:0/2296776378 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4918 (secure 0 0 0) 0x7f5de80eede0 con 0x7f5df80caf60 2023-11-28T18:23:40.454 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:40.454 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/390529893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":390529893},{"type":"v1","addr":"172.21.15.148:6835","nonce":390529893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3924571887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3924571887},{"type":"v1","addr":"172.21.15.148:6833","nonce":3924571887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3271971772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3271971772},{"type":"v1","addr":"172.21.15.105:6835","nonce":3271971772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:23:33.288743+0000","modified":"2023-11-28T18:23:38.459146+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2504263238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2504263238},{"type":"v1","addr":"172.21.15.105:6837","nonce":2504263238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-28T18:23:40.455 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 7 2023-11-28T18:23:40.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.452+0000 7f5e005aa640 1 -- 172.21.15.105:0/2296776378 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5ddc0668d0 msgr2=0x7f5ddc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:40.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.452+0000 7f5e005aa640 1 --2- 172.21.15.105:0/2296776378 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5ddc0668d0 0x7f5ddc068d90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5df4000f70 tx=0x7f5df4007d00 comp rx=0 tx=0).stop 2023-11-28T18:23:40.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.452+0000 7f5e005aa640 1 -- 172.21.15.105:0/2296776378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5df80caf60 msgr2=0x7f5df815a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:40.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.452+0000 7f5e005aa640 1 --2- 172.21.15.105:0/2296776378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5df80caf60 0x7f5df815a520 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5de8019ff0 tx=0x7f5de8019160 comp rx=0 tx=0).stop 2023-11-28T18:23:40.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.452+0000 7f5e005aa640 1 -- 172.21.15.105:0/2296776378 shutdown_connections 2023-11-28T18:23:40.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.452+0000 7f5e005aa640 1 --2- 172.21.15.105:0/2296776378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5df8159f40 0x7f5df80ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.452+0000 7f5e005aa640 1 --2- 172.21.15.105:0/2296776378 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5ddc0668d0 0x7f5ddc068d90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.452+0000 7f5e005aa640 1 --2- 172.21.15.105:0/2296776378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5df80caf60 0x7f5df815a520 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.452+0000 7f5e005aa640 1 --2- 172.21.15.105:0/2296776378 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5df80ad850 0x7f5df814aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.452+0000 7f5e005aa640 1 -- 172.21.15.105:0/2296776378 >> 172.21.15.105:0/2296776378 conn(0x7f5df80ceb90 msgr2=0x7f5df806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:40.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.452+0000 7f5e005aa640 1 -- 172.21.15.105:0/2296776378 shutdown_connections 2023-11-28T18:23:40.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.452+0000 7f5e005aa640 1 -- 172.21.15.105:0/2296776378 wait complete. 2023-11-28T18:23:40.475 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:23:40.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.608+0000 7f1fffdd1640 1 Processor -- start 2023-11-28T18:23:40.613 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.612+0000 7f1fffdd1640 1 -- start start 2023-11-28T18:23:40.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.612+0000 7f1fffdd1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1ff80caf60 0x7f1ff80cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:40.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.612+0000 7f1fffdd1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1ff80ad850 0x7f1ff80701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:40.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.612+0000 7f1fffdd1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff806bf50 con 0x7f1ff8159f40 2023-11-28T18:23:40.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.612+0000 7f1fffdd1640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff80cb3c0 con 0x7f1ff80caf60 2023-11-28T18:23:40.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.612+0000 7f1fffdd1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff815a320 con 0x7f1ff80ad850 2023-11-28T18:23:40.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.612+0000 7f1ffd36c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1ff80caf60 0x7f1ff80cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:40.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.612+0000 7f1ffd36c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1ff80caf60 0x7f1ff80cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59162/0 (socket says 172.21.15.105:59162) 2023-11-28T18:23:40.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.612+0000 7f1ffd36c640 1 -- 172.21.15.105:0/136518241 learned_addr learned my addr 172.21.15.105:0/136518241 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:40.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.612+0000 7f1ffd36c640 1 -- 172.21.15.105:0/136518241 >> v2:172.21.15.148:3300/0 conn(0x7f1ff80ad850 msgr2=0x7f1ff80701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:40.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.612+0000 7f1ffd36c640 1 --2- 172.21.15.105:0/136518241 >> v2:172.21.15.148:3300/0 conn(0x7f1ff80ad850 0x7f1ff80701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.612+0000 7f1ffd36c640 1 -- 172.21.15.105:0/136518241 >> v1:172.21.15.148:6789/0 conn(0x7f1ff8159f40 legacy=0x7f1ff80ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:40.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.612+0000 7f1ffd36c640 1 -- 172.21.15.105:0/136518241 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff8071380 con 0x7f1ff80caf60 2023-11-28T18:23:40.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.616+0000 7f1ffd36c640 1 --2- 172.21.15.105:0/136518241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ff80caf60 0x7f1ff80cc760 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80b0850 tx=0x7f1fe801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6a371ae6e8753b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:40.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.616+0000 7f1ffcb6b640 1 -- 172.21.15.105:0/136518241 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80ab600 con 0x7f1ff80caf60 2023-11-28T18:23:40.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.616+0000 7f1ffcb6b640 1 -- 172.21.15.105:0/136518241 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe80ab760 con 0x7f1ff80caf60 2023-11-28T18:23:40.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.616+0000 7f1ffcb6b640 1 -- 172.21.15.105:0/136518241 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80abaf0 con 0x7f1ff80caf60 2023-11-28T18:23:40.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.616+0000 7f1fffdd1640 1 -- 172.21.15.105:0/136518241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ff80caf60 msgr2=0x7f1ff80cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:40.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.616+0000 7f1fffdd1640 1 --2- 172.21.15.105:0/136518241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ff80caf60 0x7f1ff80cc760 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80b0850 tx=0x7f1fe801c400 comp rx=0 tx=0).stop 2023-11-28T18:23:40.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.616+0000 7f1fffdd1640 1 -- 172.21.15.105:0/136518241 shutdown_connections 2023-11-28T18:23:40.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.616+0000 7f1fffdd1640 1 --2- 172.21.15.105:0/136518241 >> v2:172.21.15.148:3300/0 conn(0x7f1ff80ad850 0x7f1ff80701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.616+0000 7f1fffdd1640 1 --2- 172.21.15.105:0/136518241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ff80caf60 0x7f1ff80cc760 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.616+0000 7f1fffdd1640 1 -- 172.21.15.105:0/136518241 >> 172.21.15.105:0/136518241 conn(0x7f1ff80ceb90 msgr2=0x7f1ff80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:40.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.616+0000 7f1fffdd1640 1 -- 172.21.15.105:0/136518241 shutdown_connections 2023-11-28T18:23:40.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.616+0000 7f1fffdd1640 1 -- 172.21.15.105:0/136518241 wait complete. 2023-11-28T18:23:40.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.616+0000 7f1fffdd1640 1 Processor -- start 2023-11-28T18:23:40.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.616+0000 7f1fffdd1640 1 -- start start 2023-11-28T18:23:40.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1fffdd1640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1ff80ad850 0x7f1ff8143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:40.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1fffdd1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ff80caf60 0x7f1ff815ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:40.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1fffdd1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ff8159f40 0x7f1ff809acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:40.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1fffdd1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff8144c10 con 0x7f1ff80caf60 2023-11-28T18:23:40.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1fffdd1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff809cb10 con 0x7f1ff8159f40 2023-11-28T18:23:40.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1fffdd1640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff809be10 con 0x7f1ff80ad850 2023-11-28T18:23:40.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1ffd36c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ff80caf60 0x7f1ff815ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:40.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1ffd36c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ff80caf60 0x7f1ff815ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59164/0 (socket says 172.21.15.105:59164) 2023-11-28T18:23:40.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1ffd36c640 1 -- 172.21.15.105:0/706025579 learned_addr learned my addr 172.21.15.105:0/706025579 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:40.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1ffe36e640 1 --2- 172.21.15.105:0/706025579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ff8159f40 0x7f1ff809acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:40.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1ffdb6d640 1 --2- 172.21.15.105:0/706025579 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1ff80ad850 0x7f1ff8143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:40.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1ffd36c640 1 -- 172.21.15.105:0/706025579 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1ff80ad850 msgr2=0x7f1ff8143cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:40.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1ffd36c640 1 --2- 172.21.15.105:0/706025579 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1ff80ad850 0x7f1ff8143cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1ffd36c640 1 -- 172.21.15.105:0/706025579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ff8159f40 msgr2=0x7f1ff809acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:40.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1ffd36c640 1 --2- 172.21.15.105:0/706025579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ff8159f40 0x7f1ff809acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1ffd36c640 1 -- 172.21.15.105:0/706025579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe80094d0 con 0x7f1ff80caf60 2023-11-28T18:23:40.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1ffd36c640 1 --2- 172.21.15.105:0/706025579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ff80caf60 0x7f1ff815ce20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1fe801a030 tx=0x7f1fe801ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:40.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1fee7fc640 1 -- 172.21.15.105:0/706025579 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80b4800 con 0x7f1ff80caf60 2023-11-28T18:23:40.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1fee7fc640 1 -- 172.21.15.105:0/706025579 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe80b4980 con 0x7f1ff80caf60 2023-11-28T18:23:40.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1fffdd1640 1 -- 172.21.15.105:0/706025579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff8144200 con 0x7f1ff80caf60 2023-11-28T18:23:40.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1fee7fc640 1 -- 172.21.15.105:0/706025579 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80b4d30 con 0x7f1ff80caf60 2023-11-28T18:23:40.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1fffdd1640 1 -- 172.21.15.105:0/706025579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff809c550 con 0x7f1ff80caf60 2023-11-28T18:23:40.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.620+0000 7f1fffdd1640 1 -- 172.21.15.105:0/706025579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff809bfa0 con 0x7f1ff80caf60 2023-11-28T18:23:40.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.624+0000 7f1fee7fc640 1 -- 172.21.15.105:0/706025579 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f1fe80c7440 con 0x7f1ff80caf60 2023-11-28T18:23:40.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.624+0000 7f1fee7fc640 1 --2- 172.21.15.105:0/706025579 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1fdc0668d0 0x7f1fdc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:40.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.624+0000 7f1fee7fc640 1 -- 172.21.15.105:0/706025579 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1fe811f620 con 0x7f1ff80caf60 2023-11-28T18:23:40.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.624+0000 7f1ffdb6d640 1 --2- 172.21.15.105:0/706025579 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1fdc0668d0 0x7f1fdc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:40.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.624+0000 7f1ffdb6d640 1 --2- 172.21.15.105:0/706025579 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1fdc0668d0 0x7f1fdc068d90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400b810 tx=0x7f1ff4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:40.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.624+0000 7f1fee7fc640 1 -- 172.21.15.105:0/706025579 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1fe80ec5e0 con 0x7f1ff80caf60 2023-11-28T18:23:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.780+0000 7f1fffdd1640 1 -- 172.21.15.105:0/706025579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ff8067570 con 0x7f1ff80caf60 2023-11-28T18:23:40.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.780+0000 7f1fee7fc640 1 -- 172.21.15.105:0/706025579 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7f1fe80efc00 con 0x7f1ff80caf60 2023-11-28T18:23:40.784 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:40.785 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":24,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:23:38.458954+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:23:40.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.784+0000 7f1fffdd1640 1 -- 172.21.15.105:0/706025579 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1fdc0668d0 msgr2=0x7f1fdc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.784+0000 7f1fffdd1640 1 --2- 172.21.15.105:0/706025579 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1fdc0668d0 0x7f1fdc068d90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400b810 tx=0x7f1ff4008040 comp rx=0 tx=0).stop 2023-11-28T18:23:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.784+0000 7f1fffdd1640 1 -- 172.21.15.105:0/706025579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ff80caf60 msgr2=0x7f1ff815ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.784+0000 7f1fffdd1640 1 --2- 172.21.15.105:0/706025579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ff80caf60 0x7f1ff815ce20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1fe801a030 tx=0x7f1fe801ac40 comp rx=0 tx=0).stop 2023-11-28T18:23:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.784+0000 7f1fffdd1640 1 -- 172.21.15.105:0/706025579 shutdown_connections 2023-11-28T18:23:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.784+0000 7f1fffdd1640 1 --2- 172.21.15.105:0/706025579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ff8159f40 0x7f1ff809acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.784+0000 7f1fffdd1640 1 --2- 172.21.15.105:0/706025579 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1fdc0668d0 0x7f1fdc068d90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.784+0000 7f1fffdd1640 1 --2- 172.21.15.105:0/706025579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ff80caf60 0x7f1ff815ce20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.784+0000 7f1fffdd1640 1 --2- 172.21.15.105:0/706025579 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1ff80ad850 0x7f1ff8143cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.784+0000 7f1fffdd1640 1 -- 172.21.15.105:0/706025579 >> 172.21.15.105:0/706025579 conn(0x7f1ff80ceb90 msgr2=0x7f1ff80cd960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.784+0000 7f1fffdd1640 1 -- 172.21.15.105:0/706025579 shutdown_connections 2023-11-28T18:23:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:40.784+0000 7f1fffdd1640 1 -- 172.21.15.105:0/706025579 wait complete. 2023-11-28T18:23:41.208 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:23:33.288743+0000', 'modified': '2023-11-28T18:23:38.459146+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6837/2504263238', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 2504263238}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 2504263238}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-11-28T18:23:41.208 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T18:23:41.208 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T18:23:41.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.344+0000 7f96b11b8640 1 Processor -- start 2023-11-28T18:23:41.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.344+0000 7f96b11b8640 1 -- start start 2023-11-28T18:23:41.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.348+0000 7f96b11b8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f96ac15d3c0 0x7f96ac071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.348+0000 7f96b11b8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f96ac070a10 0x7f96ac070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.348+0000 7f96b11b8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96ac0713a0 con 0x7f96ac161870 2023-11-28T18:23:41.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.348+0000 7f96b11b8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96ac0cefa0 con 0x7f96ac15d3c0 2023-11-28T18:23:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.348+0000 7f96b11b8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac15d7a0 con 0x7f96ac070a10 2023-11-28T18:23:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.348+0000 7f96a259c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f96ac161870 0x7f96ac0ce190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:60148/0 (socket says 172.21.15.105:60148) 2023-11-28T18:23:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.348+0000 7f96a259c640 1 -- 172.21.15.105:0/3282571148 learned_addr learned my addr 172.21.15.105:0/3282571148 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.348+0000 7f96aad9d640 1 --2- 172.21.15.105:0/3282571148 >> v2:172.21.15.105:3300/0 conn(0x7f96ac15d3c0 0x7f96ac071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.348+0000 7f96ab59e640 1 --2- 172.21.15.105:0/3282571148 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f96ac070a10 0x7f96ac070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.348+0000 7f96aad9d640 1 -- 172.21.15.105:0/3282571148 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f96ac070a10 msgr2=0x7f96ac070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.348+0000 7f96aa59c640 1 -- 172.21.15.105:0/3282571148 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3340625178 0 0) 0x7f9694001120 con 0x7f96ac161870 2023-11-28T18:23:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.348+0000 7f96aad9d640 1 --2- 172.21.15.105:0/3282571148 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f96ac070a10 0x7f96ac070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.348+0000 7f96aad9d640 1 -- 172.21.15.105:0/3282571148 >> v1:172.21.15.105:6789/0 conn(0x7f96ac161870 legacy=0x7f96ac0ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.348+0000 7f96aad9d640 1 -- 172.21.15.105:0/3282571148 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ac0ce9e0 con 0x7f96ac15d3c0 2023-11-28T18:23:41.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.352+0000 7f96aad9d640 1 --2- 172.21.15.105:0/3282571148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96ac15d3c0 0x7f96ac071a10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f969c0097c0 tx=0x7f969c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67ba4cd651648cf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:41.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.352+0000 7f96aa59c640 1 -- 172.21.15.105:0/3282571148 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c0ab650 con 0x7f96ac15d3c0 2023-11-28T18:23:41.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.352+0000 7f96aa59c640 1 -- 172.21.15.105:0/3282571148 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969c0ab7b0 con 0x7f96ac15d3c0 2023-11-28T18:23:41.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.352+0000 7f96aa59c640 1 -- 172.21.15.105:0/3282571148 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c0abb40 con 0x7f96ac15d3c0 2023-11-28T18:23:41.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.352+0000 7f96b11b8640 1 -- 172.21.15.105:0/3282571148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96ac15d3c0 msgr2=0x7f96ac071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:41.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.352+0000 7f96b11b8640 1 --2- 172.21.15.105:0/3282571148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96ac15d3c0 0x7f96ac071a10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f969c0097c0 tx=0x7f969c01c450 comp rx=0 tx=0).stop 2023-11-28T18:23:41.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.352+0000 7f96b11b8640 1 -- 172.21.15.105:0/3282571148 shutdown_connections 2023-11-28T18:23:41.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.352+0000 7f96b11b8640 1 --2- 172.21.15.105:0/3282571148 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f96ac070a10 0x7f96ac070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.352+0000 7f96b11b8640 1 --2- 172.21.15.105:0/3282571148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96ac15d3c0 0x7f96ac071a10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.352+0000 7f96b11b8640 1 -- 172.21.15.105:0/3282571148 >> 172.21.15.105:0/3282571148 conn(0x7f96ac0cf540 msgr2=0x7f96ac0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:41.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.352+0000 7f96b11b8640 1 -- 172.21.15.105:0/3282571148 shutdown_connections 2023-11-28T18:23:41.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.352+0000 7f96b11b8640 1 -- 172.21.15.105:0/3282571148 wait complete. 2023-11-28T18:23:41.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.352+0000 7f96b11b8640 1 Processor -- start 2023-11-28T18:23:41.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.352+0000 7f96b11b8640 1 -- start start 2023-11-28T18:23:41.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96b11b8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f96ac070a10 0x7f96ac138d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96b11b8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96ac15d3c0 0x7f96ac0afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96b11b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96ac161870 0x7f96ac0b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96b11b8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac13fa60 con 0x7f96ac15d3c0 2023-11-28T18:23:41.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96b11b8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac152930 con 0x7f96ac161870 2023-11-28T18:23:41.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96b11b8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac152c10 con 0x7f96ac070a10 2023-11-28T18:23:41.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96a259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96ac15d3c0 0x7f96ac0afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96a259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96ac15d3c0 0x7f96ac0afd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38168/0 (socket says 172.21.15.105:38168) 2023-11-28T18:23:41.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96a259c640 1 -- 172.21.15.105:0/813956801 learned_addr learned my addr 172.21.15.105:0/813956801 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:41.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96ab59e640 1 --2- 172.21.15.105:0/813956801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96ac161870 0x7f96ac0b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96aad9d640 1 --2- 172.21.15.105:0/813956801 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f96ac070a10 0x7f96ac138d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96a259c640 1 -- 172.21.15.105:0/813956801 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f96ac070a10 msgr2=0x7f96ac138d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:41.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96a259c640 1 --2- 172.21.15.105:0/813956801 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f96ac070a10 0x7f96ac138d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96a259c640 1 -- 172.21.15.105:0/813956801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96ac161870 msgr2=0x7f96ac0b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:41.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96a259c640 1 --2- 172.21.15.105:0/813956801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96ac161870 0x7f96ac0b02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96a259c640 1 -- 172.21.15.105:0/813956801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c009420 con 0x7f96ac15d3c0 2023-11-28T18:23:41.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96ab59e640 1 --2- 172.21.15.105:0/813956801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96ac161870 0x7f96ac0b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:41.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96aad9d640 1 --2- 172.21.15.105:0/813956801 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f96ac070a10 0x7f96ac138d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:41.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96a259c640 1 --2- 172.21.15.105:0/813956801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96ac15d3c0 0x7f96ac0afd90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f969400c480 tx=0x7f969400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:41.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96a3fff640 1 -- 172.21.15.105:0/813956801 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96940074e0 con 0x7f96ac15d3c0 2023-11-28T18:23:41.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96a3fff640 1 -- 172.21.15.105:0/813956801 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9694007660 con 0x7f96ac15d3c0 2023-11-28T18:23:41.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96a3fff640 1 -- 172.21.15.105:0/813956801 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9694004a30 con 0x7f96ac15d3c0 2023-11-28T18:23:41.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96b11b8640 1 -- 172.21.15.105:0/813956801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ac152ef0 con 0x7f96ac15d3c0 2023-11-28T18:23:41.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96b11b8640 1 -- 172.21.15.105:0/813956801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ac139990 con 0x7f96ac15d3c0 2023-11-28T18:23:41.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.356+0000 7f96a3fff640 1 -- 172.21.15.105:0/813956801 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f9694004b90 con 0x7f96ac15d3c0 2023-11-28T18:23:41.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96a3fff640 1 --2- 172.21.15.105:0/813956801 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f968c0668d0 0x7f968c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96a3fff640 1 -- 172.21.15.105:0/813956801 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f96940837f0 con 0x7f96ac15d3c0 2023-11-28T18:23:41.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96aad9d640 1 --2- 172.21.15.105:0/813956801 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f968c0668d0 0x7f968c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96b451d000 1 Processor -- start 2023-11-28T18:23:41.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96b451d000 1 -- start start 2023-11-28T18:23:41.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96aad9d640 1 --2- 172.21.15.105:0/813956801 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f968c0668d0 0x7f968c068d90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f969c0af000 tx=0x7f969c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:41.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96b451d000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0dd30 0x5560a5f0e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96b451d000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5f0e650 0x5560a5ec0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96b451d000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5ec1500 0x5560a5ec18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96b451d000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560a5d8a110 con 0x5560a5ec1500 2023-11-28T18:23:41.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96b451d000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560a5d1a9a0 con 0x5560a5f0dd30 2023-11-28T18:23:41.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96b451d000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5560a5d17ed0 con 0x5560a5f0e650 2023-11-28T18:23:41.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96ab59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5ec1500 0x5560a5ec18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96ab59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5ec1500 0x5560a5ec18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38182/0 (socket says 172.21.15.105:38182) 2023-11-28T18:23:41.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96ab59e640 1 -- 172.21.15.105:0/578623354 learned_addr learned my addr 172.21.15.105:0/578623354 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:41.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96a259c640 1 --2- 172.21.15.105:0/578623354 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5f0e650 0x5560a5ec0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96aad9d640 1 --2- 172.21.15.105:0/578623354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0dd30 0x5560a5f0e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96ab59e640 1 -- 172.21.15.105:0/578623354 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5f0e650 msgr2=0x5560a5ec0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:41.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96ab59e640 1 --2- 172.21.15.105:0/578623354 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5f0e650 0x5560a5ec0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96ab59e640 1 -- 172.21.15.105:0/578623354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0dd30 msgr2=0x5560a5f0e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:41.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96ab59e640 1 --2- 172.21.15.105:0/578623354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0dd30 0x5560a5f0e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96ab59e640 1 -- 172.21.15.105:0/578623354 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560a5a0b890 con 0x5560a5ec1500 2023-11-28T18:23:41.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96a259c640 1 --2- 172.21.15.105:0/578623354 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5f0e650 0x5560a5ec0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:41.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96aad9d640 1 --2- 172.21.15.105:0/578623354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0dd30 0x5560a5f0e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:41.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96ab59e640 1 --2- 172.21.15.105:0/578623354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5ec1500 0x5560a5ec18e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f96a40074e0 tx=0x7f96a40079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d0340c66d2ca9b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:41.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96a0d99640 1 -- 172.21.15.105:0/578623354 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a40147e0 con 0x5560a5ec1500 2023-11-28T18:23:41.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96a0d99640 1 -- 172.21.15.105:0/578623354 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a4014940 con 0x5560a5ec1500 2023-11-28T18:23:41.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.360+0000 7f96a0d99640 1 -- 172.21.15.105:0/578623354 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a4014cf0 con 0x5560a5ec1500 2023-11-28T18:23:41.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96b451d000 1 -- 172.21.15.105:0/578623354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5ec1500 msgr2=0x5560a5ec18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:41.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96b451d000 1 --2- 172.21.15.105:0/578623354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5ec1500 0x5560a5ec18e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f96a40074e0 tx=0x7f96a40079a0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96b451d000 1 -- 172.21.15.105:0/578623354 shutdown_connections 2023-11-28T18:23:41.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96b451d000 1 --2- 172.21.15.105:0/578623354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5ec1500 0x5560a5ec18e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96b451d000 1 --2- 172.21.15.105:0/578623354 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5f0e650 0x5560a5ec0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96b451d000 1 --2- 172.21.15.105:0/578623354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0dd30 0x5560a5f0e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96b451d000 1 -- 172.21.15.105:0/578623354 >> 172.21.15.105:0/578623354 conn(0x5560a5eee8c0 msgr2=0x5560a5ef0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:41.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96b451d000 1 -- 172.21.15.105:0/578623354 shutdown_connections 2023-11-28T18:23:41.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96b451d000 1 -- 172.21.15.105:0/578623354 wait complete. 2023-11-28T18:23:41.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96b451d000 1 -- start start 2023-11-28T18:23:41.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96b451d000 1 Processor -- start 2023-11-28T18:23:41.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96b451d000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5ec1500 0x5560a5ec74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96b451d000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0dd30 0x5560a5ec7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96b451d000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5f0e650 0x5560a5ec7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96b451d000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560a5d137c0 con 0x5560a5f0e650 2023-11-28T18:23:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96b451d000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560a5ccbbf0 con 0x5560a5f0dd30 2023-11-28T18:23:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96b451d000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5560a5cd1a90 con 0x5560a5ec1500 2023-11-28T18:23:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96b451d000 10 client.0 _ll_register_callbacks cb 0x5560a5eaee20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:23:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96ab59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5f0e650 0x5560a5ec7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96ab59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5f0e650 0x5560a5ec7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38196/0 (socket says 172.21.15.105:38196) 2023-11-28T18:23:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96ab59e640 1 -- 172.21.15.105:0/12609135 learned_addr learned my addr 172.21.15.105:0/12609135 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96a259c640 1 --2- 172.21.15.105:0/12609135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0dd30 0x5560a5ec7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96aad9d640 1 --2- 172.21.15.105:0/12609135 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5ec1500 0x5560a5ec74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96ab59e640 1 -- 172.21.15.105:0/12609135 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5ec1500 msgr2=0x5560a5ec74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96ab59e640 1 --2- 172.21.15.105:0/12609135 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5ec1500 0x5560a5ec74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96ab59e640 1 -- 172.21.15.105:0/12609135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0dd30 msgr2=0x5560a5ec7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96ab59e640 1 --2- 172.21.15.105:0/12609135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0dd30 0x5560a5ec7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96ab59e640 1 -- 172.21.15.105:0/12609135 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a4009d80 con 0x5560a5f0e650 2023-11-28T18:23:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96aad9d640 1 --2- 172.21.15.105:0/12609135 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5ec1500 0x5560a5ec74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96ab59e640 1 --2- 172.21.15.105:0/12609135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5f0e650 0x5560a5ec7f60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f96a4018040 tx=0x7f96a4004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273d26a82275d01c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f9689ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:23:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f9689ffb640 1 -- 172.21.15.105:0/12609135 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a400d040 con 0x5560a5f0e650 2023-11-28T18:23:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96b451d000 1 -- 172.21.15.105:0/12609135 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5560a5cd5470 con 0x5560a5f0e650 2023-11-28T18:23:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f9689ffb640 1 -- 172.21.15.105:0/12609135 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f96a4014da0 con 0x5560a5f0e650 2023-11-28T18:23:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f9689ffb640 1 -- 172.21.15.105:0/12609135 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96a4016420 con 0x5560a5f0e650 2023-11-28T18:23:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96b451d000 10 client.4425 fetch_fsmap learned FSMap version 7 2023-11-28T18:23:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.364+0000 7f96b451d000 1 -- 172.21.15.105:0/12609135 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5560a5cd4190 con 0x5560a5f0e650 2023-11-28T18:23:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f9689ffb640 1 -- 172.21.15.105:0/12609135 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5560a5cd5470 con 0x5560a5f0e650 2023-11-28T18:23:41.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f9689ffb640 1 -- 172.21.15.105:0/12609135 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f96a4005d20 con 0x5560a5f0e650 2023-11-28T18:23:41.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f96b451d000 10 client.4425 fetch_fsmap finished waiting for FSMap version 7 2023-11-28T18:23:41.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f96b451d000 10 client.4425 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:23:41.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f96b451d000 20 client.4425 populate_metadata read hostname 'smithi105' 2023-11-28T18:23:41.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f96b451d000 1 --2- 172.21.15.105:0/12609135 >> [v2:172.21.15.105:6836/2504263238,v1:172.21.15.105:6837/2504263238] conn(0x5560a5ec8970 0x5560a5f1fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f96b451d000 4 client.4425 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:23:41.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f96b451d000 1 -- 172.21.15.105:0/12609135 --> [v2:172.21.15.105:6836/2504263238,v1:172.21.15.105:6837/2504263238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5560a5cd8b50 con 0x5560a5ec8970 2023-11-28T18:23:41.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f96aad9d640 1 --2- 172.21.15.105:0/12609135 >> [v2:172.21.15.105:6836/2504263238,v1:172.21.15.105:6837/2504263238] conn(0x5560a5ec8970 0x5560a5f1fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f9689ffb640 10 client.4425 ms_handle_connect on v2:172.21.15.105:6836/2504263238 2023-11-28T18:23:41.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f96aad9d640 1 --2- 172.21.15.105:0/12609135 >> [v2:172.21.15.105:6836/2504263238,v1:172.21.15.105:6837/2504263238] conn(0x5560a5ec8970 0x5560a5f1fff0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=222d1c0f76666fb0 server_cookie=6dd40469bd8cfbe5 in_seq=0 out_seq=0 2023-11-28T18:23:41.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f9689ffb640 1 -- 172.21.15.105:0/12609135 <== mds.0 v2:172.21.15.105:6836/2504263238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5560a5cd8b50 con 0x5560a5ec8970 2023-11-28T18:23:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f9689ffb640 10 client.4425 handle_command_reply: tid=0 2023-11-28T18:23:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f96b451d000 1 client.4425 shutdown 2023-11-28T18:23:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f96b451d000 1 -- 172.21.15.105:0/12609135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5f0e650 msgr2=0x5560a5ec7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f96b451d000 1 --2- 172.21.15.105:0/12609135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5f0e650 0x5560a5ec7f60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f96a4018040 tx=0x7f96a4004ac0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f96b451d000 1 -- 172.21.15.105:0/12609135 shutdown_connections 2023-11-28T18:23:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f96b451d000 1 --2- 172.21.15.105:0/12609135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5f0e650 0x5560a5ec7f60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f96b451d000 1 --2- 172.21.15.105:0/12609135 >> [v2:172.21.15.105:6836/2504263238,v1:172.21.15.105:6837/2504263238] conn(0x5560a5ec8970 0x5560a5f1fff0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f96b451d000 1 --2- 172.21.15.105:0/12609135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0dd30 0x5560a5ec7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f96b451d000 1 --2- 172.21.15.105:0/12609135 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5ec1500 0x5560a5ec74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f9689ffb640 0 client.4425 ms_handle_reset on v2:172.21.15.105:6836/2504263238 2023-11-28T18:23:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f96b451d000 1 -- 172.21.15.105:0/12609135 >> 172.21.15.105:0/12609135 conn(0x5560a5eee8c0 msgr2=0x5560a5ec06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f96b451d000 1 -- 172.21.15.105:0/12609135 shutdown_connections 2023-11-28T18:23:41.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f96b451d000 1 -- 172.21.15.105:0/12609135 wait complete. 2023-11-28T18:23:41.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.368+0000 7f96b451d000 20 client.4425 trim_cache size 0 max 16384 2023-11-28T18:23:41.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96b451d000 1 Processor -- start 2023-11-28T18:23:41.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96b451d000 1 -- start start 2023-11-28T18:23:41.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96b451d000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5ec1500 0x5560a5eed3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96b451d000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0dd30 0x5560a5eed8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96b451d000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5f0e650 0x5560a5f1fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96b451d000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560a59cc940 con 0x5560a5ec1500 2023-11-28T18:23:41.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96b451d000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560a5cdc910 con 0x5560a5f0dd30 2023-11-28T18:23:41.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96b451d000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5560a5d0f130 con 0x5560a5f0e650 2023-11-28T18:23:41.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96aad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5ec1500 0x5560a5eed3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96aad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5ec1500 0x5560a5eed3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38198/0 (socket says 172.21.15.105:38198) 2023-11-28T18:23:41.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96aad9d640 1 -- 172.21.15.105:0/2009617698 learned_addr learned my addr 172.21.15.105:0/2009617698 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:41.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96a259c640 1 --2- 172.21.15.105:0/2009617698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0dd30 0x5560a5eed8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96ab59e640 1 --2- 172.21.15.105:0/2009617698 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5f0e650 0x5560a5f1fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96aad9d640 1 -- 172.21.15.105:0/2009617698 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5f0e650 msgr2=0x5560a5f1fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:41.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96aad9d640 1 --2- 172.21.15.105:0/2009617698 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5f0e650 0x5560a5f1fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96aad9d640 1 -- 172.21.15.105:0/2009617698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0dd30 msgr2=0x5560a5eed8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:41.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96aad9d640 1 --2- 172.21.15.105:0/2009617698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0dd30 0x5560a5eed8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96aad9d640 1 -- 172.21.15.105:0/2009617698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560a5cba490 con 0x5560a5ec1500 2023-11-28T18:23:41.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96a259c640 1 --2- 172.21.15.105:0/2009617698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0dd30 0x5560a5eed8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:41.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96ab59e640 1 --2- 172.21.15.105:0/2009617698 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5f0e650 0x5560a5f1fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:41.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96aad9d640 1 --2- 172.21.15.105:0/2009617698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5ec1500 0x5560a5eed3a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f969c0ad040 tx=0x7f969c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff3f2a5f1b50c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:41.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96a159a640 1 -- 172.21.15.105:0/2009617698 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c0c5040 con 0x5560a5ec1500 2023-11-28T18:23:41.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96a159a640 1 -- 172.21.15.105:0/2009617698 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969c0b5e20 con 0x5560a5ec1500 2023-11-28T18:23:41.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96a159a640 1 -- 172.21.15.105:0/2009617698 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c0ca620 con 0x5560a5ec1500 2023-11-28T18:23:41.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96b451d000 1 -- 172.21.15.105:0/2009617698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5ec1500 msgr2=0x5560a5eed3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:41.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96b451d000 1 --2- 172.21.15.105:0/2009617698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5ec1500 0x5560a5eed3a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f969c0ad040 tx=0x7f969c009190 comp rx=0 tx=0).stop 2023-11-28T18:23:41.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96b451d000 1 -- 172.21.15.105:0/2009617698 shutdown_connections 2023-11-28T18:23:41.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96b451d000 1 --2- 172.21.15.105:0/2009617698 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5f0e650 0x5560a5f1fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96b451d000 1 --2- 172.21.15.105:0/2009617698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0dd30 0x5560a5eed8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96b451d000 1 --2- 172.21.15.105:0/2009617698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5ec1500 0x5560a5eed3a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.384+0000 7f96b451d000 1 -- 172.21.15.105:0/2009617698 >> 172.21.15.105:0/2009617698 conn(0x5560a5eee8c0 msgr2=0x5560a5ebf720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:41.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96b451d000 1 -- 172.21.15.105:0/2009617698 shutdown_connections 2023-11-28T18:23:41.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96b451d000 1 -- 172.21.15.105:0/2009617698 wait complete. 2023-11-28T18:23:41.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96b451d000 1 -- start start 2023-11-28T18:23:41.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96b451d000 1 Processor -- start 2023-11-28T18:23:41.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96b451d000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5ec1500 0x5560a5f6e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96b451d000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5f0dd30 0x5560a5f6ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96b451d000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0e650 0x5560a5f29a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96b451d000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560a5cd28f0 con 0x5560a5ec1500 2023-11-28T18:23:41.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96b451d000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560a5ebb430 con 0x5560a5f0e650 2023-11-28T18:23:41.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96b451d000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5560a5cdd2e0 con 0x5560a5f0dd30 2023-11-28T18:23:41.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96b451d000 10 client.0 _ll_register_callbacks cb 0x5560a5d8a060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:23:41.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96aad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5ec1500 0x5560a5f6e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96aad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5ec1500 0x5560a5f6e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38200/0 (socket says 172.21.15.105:38200) 2023-11-28T18:23:41.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96aad9d640 1 -- 172.21.15.105:0/1900063166 learned_addr learned my addr 172.21.15.105:0/1900063166 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:41.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96a259c640 1 --2- 172.21.15.105:0/1900063166 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5f0dd30 0x5560a5f6ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96ab59e640 1 --2- 172.21.15.105:0/1900063166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0e650 0x5560a5f29a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96aad9d640 1 -- 172.21.15.105:0/1900063166 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5f0dd30 msgr2=0x5560a5f6ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:41.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96aad9d640 1 --2- 172.21.15.105:0/1900063166 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5f0dd30 0x5560a5f6ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96aad9d640 1 -- 172.21.15.105:0/1900063166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0e650 msgr2=0x5560a5f29a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:41.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96aad9d640 1 --2- 172.21.15.105:0/1900063166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0e650 0x5560a5f29a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96aad9d640 1 -- 172.21.15.105:0/1900063166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969c0bb070 con 0x5560a5ec1500 2023-11-28T18:23:41.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96ab59e640 1 --2- 172.21.15.105:0/1900063166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0e650 0x5560a5f29a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:41.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96aad9d640 1 --2- 172.21.15.105:0/1900063166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5ec1500 0x5560a5f6e810 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f969c0cc040 tx=0x7f969c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50e7f91c17b5d1d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:41.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f9689ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:23:41.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f9689ffb640 1 -- 172.21.15.105:0/1900063166 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c0c5040 con 0x5560a5ec1500 2023-11-28T18:23:41.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96b451d000 1 -- 172.21.15.105:0/1900063166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5560a5ec12f0 con 0x5560a5ec1500 2023-11-28T18:23:41.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f9689ffb640 1 -- 172.21.15.105:0/1900063166 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f969c0ca7e0 con 0x5560a5ec1500 2023-11-28T18:23:41.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f9689ffb640 1 -- 172.21.15.105:0/1900063166 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f969c0d5460 con 0x5560a5ec1500 2023-11-28T18:23:41.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96b451d000 10 client.4431 fetch_fsmap learned FSMap version 7 2023-11-28T18:23:41.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96b451d000 1 -- 172.21.15.105:0/1900063166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5560a5a0bcc0 con 0x5560a5ec1500 2023-11-28T18:23:41.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f9689ffb640 1 -- 172.21.15.105:0/1900063166 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5560a5ec12f0 con 0x5560a5ec1500 2023-11-28T18:23:41.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f9689ffb640 1 -- 172.21.15.105:0/1900063166 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f969c0d55c0 con 0x5560a5ec1500 2023-11-28T18:23:41.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96b451d000 10 client.4431 fetch_fsmap finished waiting for FSMap version 7 2023-11-28T18:23:41.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96b451d000 10 client.4431 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:23:41.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96b451d000 20 client.4431 populate_metadata read hostname 'smithi105' 2023-11-28T18:23:41.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96b451d000 1 --2- 172.21.15.105:0/1900063166 >> [v2:172.21.15.105:6836/2504263238,v1:172.21.15.105:6837/2504263238] conn(0x5560a5f75670 0x5560a5f77a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96b451d000 4 client.4431 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:23:41.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96b451d000 1 -- 172.21.15.105:0/1900063166 --> [v2:172.21.15.105:6836/2504263238,v1:172.21.15.105:6837/2504263238] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5560a5d12110 con 0x5560a5f75670 2023-11-28T18:23:41.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.388+0000 7f96a259c640 1 --2- 172.21.15.105:0/1900063166 >> [v2:172.21.15.105:6836/2504263238,v1:172.21.15.105:6837/2504263238] conn(0x5560a5f75670 0x5560a5f77a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f9689ffb640 10 client.4431 ms_handle_connect on v2:172.21.15.105:6836/2504263238 2023-11-28T18:23:41.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96a259c640 1 --2- 172.21.15.105:0/1900063166 >> [v2:172.21.15.105:6836/2504263238,v1:172.21.15.105:6837/2504263238] conn(0x5560a5f75670 0x5560a5f77a60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ac06ade1250d80a server_cookie=c7e0ec44dfeeb6d0 in_seq=0 out_seq=0 2023-11-28T18:23:41.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f9689ffb640 1 -- 172.21.15.105:0/1900063166 <== mds.0 v2:172.21.15.105:6836/2504263238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5560a5d12110 con 0x5560a5f75670 2023-11-28T18:23:41.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f9689ffb640 10 client.4431 handle_command_reply: tid=0 2023-11-28T18:23:41.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b451d000 1 client.4431 shutdown 2023-11-28T18:23:41.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b451d000 1 -- 172.21.15.105:0/1900063166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5ec1500 msgr2=0x5560a5f6e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:41.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b451d000 1 --2- 172.21.15.105:0/1900063166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5ec1500 0x5560a5f6e810 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f969c0cc040 tx=0x7f969c0046d0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b451d000 1 -- 172.21.15.105:0/1900063166 shutdown_connections 2023-11-28T18:23:41.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b451d000 1 --2- 172.21.15.105:0/1900063166 >> [v2:172.21.15.105:6836/2504263238,v1:172.21.15.105:6837/2504263238] conn(0x5560a5f75670 0x5560a5f77a60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b451d000 1 --2- 172.21.15.105:0/1900063166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560a5f0e650 0x5560a5f29a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b451d000 1 --2- 172.21.15.105:0/1900063166 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5560a5f0dd30 0x5560a5f6ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f9689ffb640 0 client.4431 ms_handle_reset on v2:172.21.15.105:6836/2504263238 2023-11-28T18:23:41.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b451d000 1 --2- 172.21.15.105:0/1900063166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560a5ec1500 0x5560a5f6e810 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b451d000 1 -- 172.21.15.105:0/1900063166 >> 172.21.15.105:0/1900063166 conn(0x5560a5eee8c0 msgr2=0x5560a5ebffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:41.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b451d000 1 -- 172.21.15.105:0/1900063166 shutdown_connections 2023-11-28T18:23:41.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b451d000 1 -- 172.21.15.105:0/1900063166 wait complete. 2023-11-28T18:23:41.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b451d000 20 client.4431 trim_cache size 0 max 16384 2023-11-28T18:23:41.400 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:23:41.400 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:23:41.400 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2023-11-28T18:23:41.400 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 4287, 2023-11-28T18:23:41.400 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:23:41.401 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:23:41.401 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:23:41.401 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 7.3582289210000003, 2023-11-28T18:23:41.401 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 7, 2023-11-28T18:23:41.401 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 24, 2023-11-28T18:23:41.401 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 23, 2023-11-28T18:23:41.401 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 14.539298442 2023-11-28T18:23:41.401 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:23:41.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b11b8640 1 -- 172.21.15.105:0/813956801 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f968c0668d0 msgr2=0x7f968c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:41.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b11b8640 1 --2- 172.21.15.105:0/813956801 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f968c0668d0 0x7f968c068d90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f969c0af000 tx=0x7f969c0a7000 comp rx=0 tx=0).stop 2023-11-28T18:23:41.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b11b8640 1 -- 172.21.15.105:0/813956801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96ac15d3c0 msgr2=0x7f96ac0afd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:41.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b11b8640 1 --2- 172.21.15.105:0/813956801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96ac15d3c0 0x7f96ac0afd90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f969400c480 tx=0x7f969400c940 comp rx=0 tx=0).stop 2023-11-28T18:23:41.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b11b8640 1 -- 172.21.15.105:0/813956801 shutdown_connections 2023-11-28T18:23:41.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b11b8640 1 --2- 172.21.15.105:0/813956801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96ac161870 0x7f96ac0b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b11b8640 1 --2- 172.21.15.105:0/813956801 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f968c0668d0 0x7f968c068d90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b11b8640 1 --2- 172.21.15.105:0/813956801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96ac15d3c0 0x7f96ac0afd90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b11b8640 1 --2- 172.21.15.105:0/813956801 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f96ac070a10 0x7f96ac138d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b11b8640 1 -- 172.21.15.105:0/813956801 >> 172.21.15.105:0/813956801 conn(0x7f96ac0cf540 msgr2=0x7f96ac0cb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:41.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b11b8640 1 -- 172.21.15.105:0/813956801 shutdown_connections 2023-11-28T18:23:41.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.392+0000 7f96b11b8640 1 -- 172.21.15.105:0/813956801 wait complete. 2023-11-28T18:23:41.414 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T18:23:41.414 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-28T18:23:41.415 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T18:23:41.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.548+0000 7fcab3217640 1 Processor -- start 2023-11-28T18:23:41.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.548+0000 7fcab3217640 1 -- start start 2023-11-28T18:23:41.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.552+0000 7fcab3217640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcaac159610 0x7fcaac0ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.552+0000 7fcab3217640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcaac0cc760 0x7fcaac0ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.552+0000 7fcab3217640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaac0cde30 con 0x7fcaac06c7e0 2023-11-28T18:23:41.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.552+0000 7fcab3217640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaac1599f0 con 0x7fcaac159610 2023-11-28T18:23:41.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.552+0000 7fcab3217640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaac06de20 con 0x7fcaac0cc760 2023-11-28T18:23:41.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.552+0000 7fcab0fb3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcaac159610 0x7fcaac0ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.552+0000 7fcab0fb3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcaac159610 0x7fcaac0ce2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38202/0 (socket says 172.21.15.105:38202) 2023-11-28T18:23:41.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.552+0000 7fcab0fb3640 1 -- 172.21.15.105:0/2814756799 learned_addr learned my addr 172.21.15.105:0/2814756799 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:41.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.552+0000 7fcaa3fff640 1 --2- 172.21.15.105:0/2814756799 >> v2:172.21.15.148:3300/0 conn(0x7fcaac0cc760 0x7fcaac0ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.552+0000 7fcab0fb3640 1 -- 172.21.15.105:0/2814756799 >> v2:172.21.15.148:3300/0 conn(0x7fcaac0cc760 msgr2=0x7fcaac0ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:41.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.552+0000 7fcab0fb3640 1 --2- 172.21.15.105:0/2814756799 >> v2:172.21.15.148:3300/0 conn(0x7fcaac0cc760 0x7fcaac0ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.552+0000 7fcab0fb3640 1 -- 172.21.15.105:0/2814756799 >> v1:172.21.15.148:6789/0 conn(0x7fcaac06c7e0 legacy=0x7fcaac0abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:41.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.552+0000 7fcab0fb3640 1 -- 172.21.15.105:0/2814756799 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaac0cd220 con 0x7fcaac159610 2023-11-28T18:23:41.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.556+0000 7fcab0fb3640 1 --2- 172.21.15.105:0/2814756799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcaac159610 0x7fcaac0ce2c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcaac0cfc90 tx=0x7fcaac0acca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb394d90f17c953 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:41.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.556+0000 7fcaa37fe640 1 -- 172.21.15.105:0/2814756799 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa80aa5f0 con 0x7fcaac159610 2023-11-28T18:23:41.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.556+0000 7fcaa37fe640 1 -- 172.21.15.105:0/2814756799 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa80aa750 con 0x7fcaac159610 2023-11-28T18:23:41.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.556+0000 7fcaa37fe640 1 -- 172.21.15.105:0/2814756799 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa80aaae0 con 0x7fcaac159610 2023-11-28T18:23:41.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.556+0000 7fcab3217640 1 -- 172.21.15.105:0/2814756799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcaac159610 msgr2=0x7fcaac0ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:41.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.556+0000 7fcab3217640 1 --2- 172.21.15.105:0/2814756799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcaac159610 0x7fcaac0ce2c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcaac0cfc90 tx=0x7fcaac0acca0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.556+0000 7fcab3217640 1 -- 172.21.15.105:0/2814756799 shutdown_connections 2023-11-28T18:23:41.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.556+0000 7fcab3217640 1 --2- 172.21.15.105:0/2814756799 >> v2:172.21.15.148:3300/0 conn(0x7fcaac0cc760 0x7fcaac0ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.556+0000 7fcab3217640 1 --2- 172.21.15.105:0/2814756799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcaac159610 0x7fcaac0ce2c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.556+0000 7fcab3217640 1 -- 172.21.15.105:0/2814756799 >> 172.21.15.105:0/2814756799 conn(0x7fcaac0ceb90 msgr2=0x7fcaac0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:41.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.556+0000 7fcab3217640 1 -- 172.21.15.105:0/2814756799 shutdown_connections 2023-11-28T18:23:41.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.556+0000 7fcab3217640 1 -- 172.21.15.105:0/2814756799 wait complete. 2023-11-28T18:23:41.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.556+0000 7fcab3217640 1 Processor -- start 2023-11-28T18:23:41.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.556+0000 7fcab3217640 1 -- start start 2023-11-28T18:23:41.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcab3217640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcaac06c7e0 0x7fcaac155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcab3217640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcaac0cc760 0x7fcaac167b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcab3217640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaac159610 0x7fcaac0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcab3217640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac156490 con 0x7fcaac0cc760 2023-11-28T18:23:41.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcab3217640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac15ac20 con 0x7fcaac159610 2023-11-28T18:23:41.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcab3217640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac167670 con 0x7fcaac06c7e0 2023-11-28T18:23:41.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcaa3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcaac0cc760 0x7fcaac167b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcaa3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcaac0cc760 0x7fcaac167b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38214/0 (socket says 172.21.15.105:38214) 2023-11-28T18:23:41.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcaa3fff640 1 -- 172.21.15.105:0/1996968383 learned_addr learned my addr 172.21.15.105:0/1996968383 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:41.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcab0fb3640 1 --2- 172.21.15.105:0/1996968383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcaac06c7e0 0x7fcaac155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcab17b4640 1 --2- 172.21.15.105:0/1996968383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaac159610 0x7fcaac0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcaa3fff640 1 -- 172.21.15.105:0/1996968383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcaac06c7e0 msgr2=0x7fcaac155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:41.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcaa3fff640 1 --2- 172.21.15.105:0/1996968383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcaac06c7e0 0x7fcaac155f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcaa3fff640 1 -- 172.21.15.105:0/1996968383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaac159610 msgr2=0x7fcaac0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:41.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcaa3fff640 1 --2- 172.21.15.105:0/1996968383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaac159610 0x7fcaac0b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcaa3fff640 1 -- 172.21.15.105:0/1996968383 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa80094d0 con 0x7fcaac0cc760 2023-11-28T18:23:41.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcab17b4640 1 --2- 172.21.15.105:0/1996968383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaac159610 0x7fcaac0b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:41.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcaa3fff640 1 --2- 172.21.15.105:0/1996968383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcaac0cc760 0x7fcaac167b00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00d800 tx=0x7fca9c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:41.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcaa17fa640 1 -- 172.21.15.105:0/1996968383 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c00b6c0 con 0x7fcaac0cc760 2023-11-28T18:23:41.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcaa17fa640 1 -- 172.21.15.105:0/1996968383 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c00b820 con 0x7fcaac0cc760 2023-11-28T18:23:41.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcab3217640 1 -- 172.21.15.105:0/1996968383 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaac166a60 con 0x7fcaac0cc760 2023-11-28T18:23:41.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcaa17fa640 1 -- 172.21.15.105:0/1996968383 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c00bb90 con 0x7fcaac0cc760 2023-11-28T18:23:41.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcab3217640 1 -- 172.21.15.105:0/1996968383 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaac167020 con 0x7fcaac0cc760 2023-11-28T18:23:41.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcab3217640 1 -- 172.21.15.105:0/1996968383 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaac067570 con 0x7fcaac0cc760 2023-11-28T18:23:41.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.560+0000 7fcaa17fa640 1 -- 172.21.15.105:0/1996968383 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fca9c00bcf0 con 0x7fcaac0cc760 2023-11-28T18:23:41.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.564+0000 7fcaa17fa640 1 --2- 172.21.15.105:0/1996968383 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fca980668d0 0x7fca98068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:41.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.564+0000 7fcaa17fa640 1 -- 172.21.15.105:0/1996968383 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fca9c084bb0 con 0x7fcaac0cc760 2023-11-28T18:23:41.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.564+0000 7fcab0fb3640 1 --2- 172.21.15.105:0/1996968383 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fca980668d0 0x7fca98068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:41.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.564+0000 7fcab0fb3640 1 --2- 172.21.15.105:0/1996968383 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fca980668d0 0x7fca98068d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80ae000 tx=0x7fcaa80b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:41.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.568+0000 7fcaa17fa640 1 -- 172.21.15.105:0/1996968383 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fca9c051b00 con 0x7fcaac0cc760 2023-11-28T18:23:41.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.748+0000 7fcab3217640 1 -- 172.21.15.105:0/1996968383 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcaac1677b0 con 0x7fcaac0cc760 2023-11-28T18:23:41.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.748+0000 7fcaa17fa640 1 -- 172.21.15.105:0/1996968383 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7fca9c055120 con 0x7fcaac0cc760 2023-11-28T18:23:41.753 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T18:23:41.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.752+0000 7fcab3217640 1 -- 172.21.15.105:0/1996968383 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fca980668d0 msgr2=0x7fca98068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:41.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.752+0000 7fcab3217640 1 --2- 172.21.15.105:0/1996968383 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fca980668d0 0x7fca98068d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80ae000 tx=0x7fcaa80b5040 comp rx=0 tx=0).stop 2023-11-28T18:23:41.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.752+0000 7fcab3217640 1 -- 172.21.15.105:0/1996968383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcaac0cc760 msgr2=0x7fcaac167b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:41.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.752+0000 7fcab3217640 1 --2- 172.21.15.105:0/1996968383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcaac0cc760 0x7fcaac167b00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00d800 tx=0x7fca9c00dcc0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.752+0000 7fcab3217640 1 -- 172.21.15.105:0/1996968383 shutdown_connections 2023-11-28T18:23:41.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.752+0000 7fcab3217640 1 --2- 172.21.15.105:0/1996968383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaac159610 0x7fcaac0b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.752+0000 7fcab3217640 1 --2- 172.21.15.105:0/1996968383 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fca980668d0 0x7fca98068d90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.752+0000 7fcab3217640 1 --2- 172.21.15.105:0/1996968383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcaac0cc760 0x7fcaac167b00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.752+0000 7fcab3217640 1 --2- 172.21.15.105:0/1996968383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcaac06c7e0 0x7fcaac155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:41.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.752+0000 7fcab3217640 1 -- 172.21.15.105:0/1996968383 >> 172.21.15.105:0/1996968383 conn(0x7fcaac0ceb90 msgr2=0x7fcaac0adfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:41.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.752+0000 7fcab3217640 1 -- 172.21.15.105:0/1996968383 shutdown_connections 2023-11-28T18:23:41.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:41.752+0000 7fcab3217640 1 -- 172.21.15.105:0/1996968383 wait complete. 2023-11-28T18:23:41.773 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T18:23:41.773 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T18:23:41.773 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2023-11-28T18:23:41.773 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-11-28T18:23:41.774 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-28T18:23:41.774 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T18:23:41.774 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T18:23:41.774 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T18:23:41.774 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-11-28T18:23:41.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:23:41.774 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2023-11-28T18:23:41.784 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T18:23:41.784 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T18:23:41.784 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T18:23:41.784 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:23:41.784 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-11-28T18:23:41.785 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:23:41.785 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:23:41.785 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:23:41.785 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2023-11-28T18:23:41.785 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:23:41.785 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:23:41.785 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2023-11-28T18:23:41.785 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:23:41.785 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:23:41.785 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T18:23:41.785 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:23:41.785 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-28T18:23:41.785 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42400sec preferred_lft 42400sec 2023-11-28T18:23:41.786 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2023-11-28T18:23:41.786 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:23:41.786 INFO:teuthology.orchestra.run.smithi105.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-28T18:23:41.786 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 02:42:cc:d4:87:d2 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:23:41.786 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-28T18:23:41.786 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:23:41.787 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T18:23:41.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:23:41.787 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-11-28T18:23:41.787 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:23:41.787 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add name ceph-brx type bridge 2023-11-28T18:23:41.787 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr flush dev ceph-brx 2023-11-28T18:23:41.787 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set ceph-brx up 2023-11-28T18:23:41.787 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T18:23:41.787 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:23:41.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:23:41.902 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T18:23:41.921 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-11-28T18:23:41.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:23:41.924 DEBUG:teuthology.orchestra.run.smithi105:> route 2023-11-28T18:23:42.001 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2023-11-28T18:23:42.001 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T18:23:42.001 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-28T18:23:42.001 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T18:23:42.001 INFO:teuthology.orchestra.run.smithi105.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-28T18:23:42.001 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T18:23:42.001 INFO:teuthology.orchestra.run.smithi105.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:23:42.001 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:23:42.001 INFO:teuthology.orchestra.run.smithi105.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:23:42.001 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T18:23:42.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:23:42.002 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-11-28T18:23:42.002 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:23:42.002 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T18:23:42.002 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T18:23:42.002 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T18:23:42.002 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:23:42.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:23:42.097 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-11-28T18:23:42.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:23:42.107 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list-id 2023-11-28T18:23:42.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:23:42.157 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-11-28T18:23:42.157 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:23:42.157 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T18:23:42.157 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-11-28T18:23:42.157 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:23:42.240 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-11-28T18:23:42.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:23:42.240 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-11-28T18:23:42.240 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:23:42.240 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-11-28T18:23:42.240 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T18:23:42.240 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-11-28T18:23:42.240 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-11-28T18:23:42.241 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-11-28T18:23:42.241 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:23:42.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:23:42.405 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-11-28T18:23:42.405 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:23:42.405 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2023-11-28T18:23:42.406 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T18:23:42.406 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:23:42.446 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-11-28T18:23:42.446 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:23:42.446 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-11-28T18:23:42.501 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-11-28T18:23:42.502 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:23:42.502 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-11-28T18:23:42.556 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:23:42.617 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:23:42.670 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:23:42.670 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:23:42.670 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:23:42.670 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:23:42.670 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:23:42.670 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:23:42.670 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:23:42.671 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:23:42.671 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:23:42.671 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:23:42.671 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:23:42.671 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:23:42.671 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:23:42.671 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:23:42.671 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:23:42.671 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:23:42.671 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:23:42.671 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:23:42.671 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:23:42.671 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:23:42.672 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:23:42.672 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:23:42.672 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:23:42.672 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:23:42.672 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:23:42.672 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:23:42.672 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:23:42.672 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:23:42.672 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:23:42.672 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T18:23:42.672 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T18:23:42.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:23:42.673 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:23:42.725 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T18:23:42.725 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-11-28T18:23:42.772 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:23:42.793 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:23:42.850 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:23:42.850 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:23:42.850 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:23:42.850 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:23:42.851 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:23:42.851 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:23:42.851 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:23:42.851 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:23:42.851 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:23:42.851 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:23:42.851 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:23:42.851 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:23:42.851 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:23:42.851 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:23:42.851 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:23:42.851 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:23:42.851 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:23:42.852 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:23:42.852 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:23:42.852 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:23:42.852 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:23:42.852 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:23:42.852 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:23:42.852 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:23:42.852 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:23:42.852 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:23:42.852 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:23:42.852 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:23:42.852 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:23:42.853 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T18:23:42.853 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T18:23:42.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:23:42.854 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:23:42.880 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi105.stderr:2023-11-28T18:23:42.876+0000 7f06a537e580 -1 init, newargv = 0x5585b052ef60 newargc=15 2023-11-28T18:23:42.880 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi105.stderr:ceph-fuse[44820]: starting ceph client 2023-11-28T18:23:42.889 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi105.stderr:ceph-fuse[44820]: starting fuse 2023-11-28T18:23:43.990 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:23:44.013 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:23:44.067 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:23:44.067 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:23:44.067 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:23:44.067 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:23:44.067 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:23:44.067 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:23:44.067 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:23:44.067 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:23:44.068 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:23:44.068 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:23:44.068 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:23:44.068 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:23:44.068 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:23:44.068 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:23:44.068 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:23:44.068 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:23:44.068 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:23:44.068 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:23:44.068 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:23:44.068 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:23:44.068 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:23:44.069 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:23:44.069 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:23:44.069 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:23:44.069 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:23:44.069 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:23:44.069 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:23:44.069 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:23:44.069 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:23:44.069 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T18:23:44.069 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T18:23:44.069 INFO:teuthology.orchestra.run.smithi105.stdout:/home/ubuntu/cephtest/mnt.admin 2023-11-28T18:23:44.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:23:44.071 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:23:44.120 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-11-28T18:23:44.121 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-28T18:23:44.121 DEBUG:teuthology.orchestra.run.smithi105:> sudo stdin-killer -- python3 -c ' 2023-11-28T18:23:44.121 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2023-11-28T18:23:44.121 DEBUG:teuthology.orchestra.run.smithi105:> import re 2023-11-28T18:23:44.121 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-11-28T18:23:44.121 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2023-11-28T18:23:44.121 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:23:44.121 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2023-11-28T18:23:44.121 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-11-28T18:23:44.122 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2023-11-28T18:23:44.122 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-11-28T18:23:44.122 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:23:44.122 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2023-11-28T18:23:44.122 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2023-11-28T18:23:44.122 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2023-11-28T18:23:44.122 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2023-11-28T18:23:44.122 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:23:44.122 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2023-11-28T18:23:44.122 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T18:23:44.122 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T18:23:44.122 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T18:23:44.122 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2023-11-28T18:23:44.122 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2023-11-28T18:23:44.123 DEBUG:teuthology.orchestra.run.smithi105:> return f 2023-11-28T18:23:44.123 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T18:23:44.123 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:23:44.123 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.admin")) 2023-11-28T18:23:44.123 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-11-28T18:23:44.248 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.admin.44820.asok 2023-11-28T18:23:44.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-28T18:23:44.259 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.44820.asok 2023-11-28T18:23:44.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:23:44.260 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.44820.asok status 2023-11-28T18:23:44.432 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:23:44.432 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2023-11-28T18:23:44.432 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "47c1c547372ed5bff7984647dc200bac0e019caf", 2023-11-28T18:23:44.432 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev)", 2023-11-28T18:23:44.432 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "admin", 2023-11-28T18:23:44.432 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2023-11-28T18:23:44.432 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-11-28T18:23:44.432 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "44820", 2023-11-28T18:23:44.432 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2023-11-28T18:23:44.432 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:23:44.432 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2023-11-28T18:23:44.433 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2023-11-28T18:23:44.433 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 4446, 2023-11-28T18:23:44.433 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2023-11-28T18:23:44.433 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2023-11-28T18:23:44.433 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2023-11-28T18:23:44.433 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 4446 2023-11-28T18:23:44.433 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:23:44.433 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-11-28T18:23:44.433 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-11-28T18:23:44.433 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-11-28T18:23:44.433 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 279271540 2023-11-28T18:23:44.433 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:23:44.433 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:23:44.434 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-11-28T18:23:44.434 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-11-28T18:23:44.434 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-11-28T18:23:44.434 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 279271540 2023-11-28T18:23:44.434 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:23:44.434 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.4446 192.168.0.1:0/279271540", 2023-11-28T18:23:44.434 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/279271540", 2023-11-28T18:23:44.434 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2023-11-28T18:23:44.434 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 7, 2023-11-28T18:23:44.434 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 24, 2023-11-28T18:23:44.434 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2023-11-28T18:23:44.434 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2023-11-28T18:23:44.434 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2023-11-28T18:23:44.434 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:23:44.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:23:44.446 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2023-11-28T18:23:44.447 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-11-28T18:23:44.456 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2023-11-28T18:23:44.456 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-11-28T18:23:44.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:23:44.456 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-11-28T18:23:44.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:23:44.525 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-11-28T18:23:44.603 INFO:teuthology.orchestra.run.smithi105.stdout:# file: . 2023-11-28T18:23:44.603 INFO:teuthology.orchestra.run.smithi105.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-11-28T18:23:44.603 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:44.604 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:23:44.605 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:23:44.657 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:23:44.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f974d901640 1 Processor -- start 2023-11-28T18:23:44.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f974d901640 1 -- start start 2023-11-28T18:23:44.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f974d901640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f974815d4f0 0x7f97480d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:44.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f974d901640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97480b25b0 con 0x7f97480b2d20 2023-11-28T18:23:44.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f974d901640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97480cea30 con 0x7f97480d3c30 2023-11-28T18:23:44.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f974d901640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97480b2900 con 0x7f974815d4f0 2023-11-28T18:23:44.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f97477fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f97480b2d20 0x7f97480b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:60156/0 (socket says 172.21.15.105:60156) 2023-11-28T18:23:44.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f97477fe640 1 -- 172.21.15.105:0/3028634977 learned_addr learned my addr 172.21.15.105:0/3028634977 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:44.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f9746ffd640 1 --2- 172.21.15.105:0/3028634977 >> v2:172.21.15.148:3300/0 conn(0x7f974815d4f0 0x7f97480d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:44.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f97467fc640 1 -- 172.21.15.105:0/3028634977 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1446158863 0 0) 0x7f97480b25b0 con 0x7f97480b2d20 2023-11-28T18:23:44.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f97467fc640 1 -- 172.21.15.105:0/3028634977 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9734003530 con 0x7f97480b2d20 2023-11-28T18:23:44.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f97467fc640 1 -- 172.21.15.105:0/3028634977 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2193340478 0 0) 0x7f9734003530 con 0x7f97480b2d20 2023-11-28T18:23:44.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f97467fc640 1 -- 172.21.15.105:0/3028634977 >> v2:172.21.15.148:3300/0 conn(0x7f974815d4f0 msgr2=0x7f97480d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:44.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f97467fc640 1 --2- 172.21.15.105:0/3028634977 >> v2:172.21.15.148:3300/0 conn(0x7f974815d4f0 0x7f97480d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:44.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f97467fc640 1 -- 172.21.15.105:0/3028634977 >> v1:172.21.15.148:6789/0 conn(0x7f97480d3c30 legacy=0x7f97480d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:44.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f97467fc640 1 -- 172.21.15.105:0/3028634977 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97480ced10 con 0x7f97480b2d20 2023-11-28T18:23:44.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f97467fc640 1 -- 172.21.15.105:0/3028634977 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f9740001680 con 0x7f97480b2d20 2023-11-28T18:23:44.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f97467fc640 1 -- 172.21.15.105:0/3028634977 >> v1:172.21.15.105:6789/0 conn(0x7f97480b2d20 legacy=0x7f97480b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:44.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f97467fc640 1 --2- 172.21.15.105:0/3028634977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9734003980 0x7f9734003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:44.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f97467fc640 1 --2- 172.21.15.105:0/3028634977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97340047d0 0x7f9734004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:44.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f97467fc640 1 --2- 172.21.15.105:0/3028634977 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9734005190 0x7f973400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:44.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f97467fc640 1 -- 172.21.15.105:0/3028634977 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97480b2900 con 0x7f9734003980 2023-11-28T18:23:44.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f97467fc640 1 -- 172.21.15.105:0/3028634977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973400c130 con 0x7f97340047d0 2023-11-28T18:23:44.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f97467fc640 1 -- 172.21.15.105:0/3028634977 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973400c2f0 con 0x7f9734005190 2023-11-28T18:23:44.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f9746ffd640 1 --2- 172.21.15.105:0/3028634977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9734003980 0x7f9734003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:44.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f973e7fc640 1 --2- 172.21.15.105:0/3028634977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97340047d0 0x7f9734004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:44.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.836+0000 7f97477fe640 1 --2- 172.21.15.105:0/3028634977 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9734005190 0x7f973400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:44.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.840+0000 7f9746ffd640 1 -- 172.21.15.105:0/3028634977 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9734005190 msgr2=0x7f973400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:44.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.840+0000 7f9746ffd640 1 --2- 172.21.15.105:0/3028634977 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9734005190 0x7f973400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:44.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.840+0000 7f9746ffd640 1 -- 172.21.15.105:0/3028634977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97340047d0 msgr2=0x7f9734004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:44.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.840+0000 7f9746ffd640 1 --2- 172.21.15.105:0/3028634977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97340047d0 0x7f9734004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:44.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.840+0000 7f9746ffd640 1 -- 172.21.15.105:0/3028634977 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9734003530 con 0x7f9734003980 2023-11-28T18:23:44.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.840+0000 7f97477fe640 1 --2- 172.21.15.105:0/3028634977 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9734005190 0x7f973400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:44.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.840+0000 7f97477fe640 1 -- 172.21.15.105:0/3028634977 reap_dead start 2023-11-28T18:23:44.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.840+0000 7f9746ffd640 1 --2- 172.21.15.105:0/3028634977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9734003980 0x7f9734003da0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9738009ae0 tx=0x7f973801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3084588fc1e566 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:44.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.840+0000 7f97467fc640 1 -- 172.21.15.105:0/3028634977 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97380a6510 con 0x7f9734003980 2023-11-28T18:23:44.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.840+0000 7f97467fc640 1 -- 172.21.15.105:0/3028634977 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97380a6670 con 0x7f9734003980 2023-11-28T18:23:44.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.840+0000 7f97467fc640 1 -- 172.21.15.105:0/3028634977 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97380a6a20 con 0x7f9734003980 2023-11-28T18:23:44.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.840+0000 7f974d901640 1 -- 172.21.15.105:0/3028634977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9734003980 msgr2=0x7f9734003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:44.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.840+0000 7f974d901640 1 --2- 172.21.15.105:0/3028634977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9734003980 0x7f9734003da0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9738009ae0 tx=0x7f973801ca20 comp rx=0 tx=0).stop 2023-11-28T18:23:44.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.840+0000 7f974d901640 1 -- 172.21.15.105:0/3028634977 shutdown_connections 2023-11-28T18:23:44.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.840+0000 7f974d901640 1 --2- 172.21.15.105:0/3028634977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9734003980 0x7f9734003da0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:44.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.840+0000 7f974d901640 1 -- 172.21.15.105:0/3028634977 >> 172.21.15.105:0/3028634977 conn(0x7f97480d3030 msgr2=0x7f97480d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:44.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.840+0000 7f974d901640 1 -- 172.21.15.105:0/3028634977 shutdown_connections 2023-11-28T18:23:44.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.840+0000 7f974d901640 1 -- 172.21.15.105:0/3028634977 wait complete. 2023-11-28T18:23:44.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f974d901640 1 Processor -- start 2023-11-28T18:23:44.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f974d901640 1 -- start start 2023-11-28T18:23:44.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f974d901640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9734003980 0x7f9748152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:44.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f974d901640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97480cbf10 0x7f97480cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:44.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f974d901640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f974815f2e0 0x7f974815f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:44.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f974d901640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97480d0a20 con 0x7f9734003980 2023-11-28T18:23:44.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f974d901640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9748153260 con 0x7f97480cbf10 2023-11-28T18:23:44.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f974d901640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97480cc8d0 con 0x7f974815f2e0 2023-11-28T18:23:44.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f9746ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9734003980 0x7f9748152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:44.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f9746ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9734003980 0x7f9748152d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38230/0 (socket says 172.21.15.105:38230) 2023-11-28T18:23:44.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f9746ffd640 1 -- 172.21.15.105:0/1159572853 learned_addr learned my addr 172.21.15.105:0/1159572853 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:44.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f973e7fc640 1 --2- 172.21.15.105:0/1159572853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97480cbf10 0x7f97480cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:44.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f97477fe640 1 --2- 172.21.15.105:0/1159572853 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f974815f2e0 0x7f974815f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:44.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f9746ffd640 1 -- 172.21.15.105:0/1159572853 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f974815f2e0 msgr2=0x7f974815f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:44.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f9746ffd640 1 --2- 172.21.15.105:0/1159572853 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f974815f2e0 0x7f974815f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:44.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f9746ffd640 1 -- 172.21.15.105:0/1159572853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97480cbf10 msgr2=0x7f97480cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:44.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f9746ffd640 1 --2- 172.21.15.105:0/1159572853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97480cbf10 0x7f97480cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:44.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f9746ffd640 1 -- 172.21.15.105:0/1159572853 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9748155ca0 con 0x7f9734003980 2023-11-28T18:23:44.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f97477fe640 1 --2- 172.21.15.105:0/1159572853 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f974815f2e0 0x7f974815f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:44.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f9746ffd640 1 --2- 172.21.15.105:0/1159572853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9734003980 0x7f9748152d20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f973801c850 tx=0x7f97380197c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:44.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f9744ff9640 1 -- 172.21.15.105:0/1159572853 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97380a67e0 con 0x7f9734003980 2023-11-28T18:23:44.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f9744ff9640 1 -- 172.21.15.105:0/1159572853 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97380a6940 con 0x7f9734003980 2023-11-28T18:23:44.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f9744ff9640 1 -- 172.21.15.105:0/1159572853 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97380a6cb0 con 0x7f9734003980 2023-11-28T18:23:44.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.844+0000 7f974d901640 1 -- 172.21.15.105:0/1159572853 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974815fec0 con 0x7f9734003980 2023-11-28T18:23:44.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.848+0000 7f974d901640 1 -- 172.21.15.105:0/1159572853 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97481f8ff0 con 0x7f9734003980 2023-11-28T18:23:44.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.848+0000 7f974d901640 1 -- 172.21.15.105:0/1159572853 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9748067570 con 0x7f9734003980 2023-11-28T18:23:44.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.848+0000 7f9744ff9640 1 -- 172.21.15.105:0/1159572853 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f9738006040 con 0x7f9734003980 2023-11-28T18:23:44.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.848+0000 7f9744ff9640 1 --2- 172.21.15.105:0/1159572853 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9728066900 0x7f9728068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:44.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.848+0000 7f9744ff9640 1 -- 172.21.15.105:0/1159572853 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f97380aa070 con 0x7f9734003980 2023-11-28T18:23:44.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.848+0000 7f973e7fc640 1 --2- 172.21.15.105:0/1159572853 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9728066900 0x7f9728068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:44.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.848+0000 7f973e7fc640 1 --2- 172.21.15.105:0/1159572853 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9728066900 0x7f9728068dc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f97480cbea0 tx=0x7f9730009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:44.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:44.852+0000 7f9744ff9640 1 -- 172.21.15.105:0/1159572853 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f97380ebe40 con 0x7f9734003980 2023-11-28T18:23:45.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.004+0000 7f974d901640 1 -- 172.21.15.105:0/1159572853 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97480b32e0 con 0x7f9734003980 2023-11-28T18:23:45.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.004+0000 7f9744ff9640 1 -- 172.21.15.105:0/1159572853 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7f97380ef460 con 0x7f9734003980 2023-11-28T18:23:45.007 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:45.007 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":24,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:23:38.458954+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:23:45.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.008+0000 7f974d901640 1 -- 172.21.15.105:0/1159572853 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9728066900 msgr2=0x7f9728068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:45.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.008+0000 7f974d901640 1 --2- 172.21.15.105:0/1159572853 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9728066900 0x7f9728068dc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f97480cbea0 tx=0x7f9730009040 comp rx=0 tx=0).stop 2023-11-28T18:23:45.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.008+0000 7f974d901640 1 -- 172.21.15.105:0/1159572853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9734003980 msgr2=0x7f9748152d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:45.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.008+0000 7f974d901640 1 --2- 172.21.15.105:0/1159572853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9734003980 0x7f9748152d20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f973801c850 tx=0x7f97380197c0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.008+0000 7f974d901640 1 -- 172.21.15.105:0/1159572853 shutdown_connections 2023-11-28T18:23:45.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.008+0000 7f974d901640 1 --2- 172.21.15.105:0/1159572853 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f974815f2e0 0x7f974815f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.008+0000 7f974d901640 1 --2- 172.21.15.105:0/1159572853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97480cbf10 0x7f97480cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.008+0000 7f974d901640 1 --2- 172.21.15.105:0/1159572853 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9728066900 0x7f9728068dc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.008+0000 7f974d901640 1 --2- 172.21.15.105:0/1159572853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9734003980 0x7f9748152d20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.008+0000 7f974d901640 1 -- 172.21.15.105:0/1159572853 >> 172.21.15.105:0/1159572853 conn(0x7f97480d3030 msgr2=0x7f97480b3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:45.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.008+0000 7f974d901640 1 -- 172.21.15.105:0/1159572853 shutdown_connections 2023-11-28T18:23:45.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.008+0000 7f974d901640 1 -- 172.21.15.105:0/1159572853 wait complete. 2023-11-28T18:23:45.028 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:23:31.485422+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:23:32.274378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T18:23:36.100071+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T18:23:45.028 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:23:45.170 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.168+0000 7fdcff279640 1 Processor -- start 2023-11-28T18:23:45.170 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.168+0000 7fdcff279640 1 -- start start 2023-11-28T18:23:45.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.168+0000 7fdcff279640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcf80ad9f0 0x7fdcf80ade50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:45.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.168+0000 7fdcff279640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf80ae390 con 0x7fdcf80cc7d0 2023-11-28T18:23:45.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.168+0000 7fdcff279640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf8162820 con 0x7fdcf806d610 2023-11-28T18:23:45.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.168+0000 7fdcff279640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf8161c10 con 0x7fdcf80ad9f0 2023-11-28T18:23:45.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.168+0000 7fdcfd816640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fdcf80cc7d0 0x7fdcf80ccc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:60158/0 (socket says 172.21.15.105:60158) 2023-11-28T18:23:45.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.168+0000 7fdcfd816640 1 -- 172.21.15.105:0/2515353069 learned_addr learned my addr 172.21.15.105:0/2515353069 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:45.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.168+0000 7fdcfc814640 1 --2- 172.21.15.105:0/2515353069 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcf80ad9f0 0x7fdcf80ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:45.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.168+0000 7fdceffff640 1 -- 172.21.15.105:0/2515353069 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243572980 0 0) 0x7fdcf80ae390 con 0x7fdcf80cc7d0 2023-11-28T18:23:45.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.168+0000 7fdceffff640 1 -- 172.21.15.105:0/2515353069 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdce4003530 con 0x7fdcf80cc7d0 2023-11-28T18:23:45.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.168+0000 7fdceffff640 1 -- 172.21.15.105:0/2515353069 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2887788458 0 0) 0x7fdcf8162820 con 0x7fdcf806d610 2023-11-28T18:23:45.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.168+0000 7fdceffff640 1 -- 172.21.15.105:0/2515353069 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcf80ae390 con 0x7fdcf806d610 2023-11-28T18:23:45.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.168+0000 7fdcfc814640 1 -- 172.21.15.105:0/2515353069 >> v1:172.21.15.148:6789/0 conn(0x7fdcf806d610 legacy=0x7fdcf80cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:45.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.168+0000 7fdcfc814640 1 -- 172.21.15.105:0/2515353069 >> v1:172.21.15.105:6789/0 conn(0x7fdcf80cc7d0 legacy=0x7fdcf80ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:45.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.168+0000 7fdcfc814640 1 -- 172.21.15.105:0/2515353069 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf8161ef0 con 0x7fdcf80ad9f0 2023-11-28T18:23:45.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.172+0000 7fdcfc814640 1 --2- 172.21.15.105:0/2515353069 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcf80ad9f0 0x7fdcf80ade50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdce80097f0 tx=0x7fdce801c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12b834daac4b6b83 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:45.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.172+0000 7fdceffff640 1 -- 172.21.15.105:0/2515353069 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce80aa5f0 con 0x7fdcf80ad9f0 2023-11-28T18:23:45.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.172+0000 7fdceffff640 1 -- 172.21.15.105:0/2515353069 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce80aa750 con 0x7fdcf80ad9f0 2023-11-28T18:23:45.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.172+0000 7fdceffff640 1 -- 172.21.15.105:0/2515353069 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce80aaac0 con 0x7fdcf80ad9f0 2023-11-28T18:23:45.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.172+0000 7fdcff279640 1 -- 172.21.15.105:0/2515353069 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcf80ad9f0 msgr2=0x7fdcf80ade50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:45.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.172+0000 7fdcff279640 1 --2- 172.21.15.105:0/2515353069 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcf80ad9f0 0x7fdcf80ade50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdce80097f0 tx=0x7fdce801c490 comp rx=0 tx=0).stop 2023-11-28T18:23:45.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.172+0000 7fdcff279640 1 -- 172.21.15.105:0/2515353069 shutdown_connections 2023-11-28T18:23:45.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.172+0000 7fdcff279640 1 --2- 172.21.15.105:0/2515353069 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcf80ad9f0 0x7fdcf80ade50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.172+0000 7fdcff279640 1 -- 172.21.15.105:0/2515353069 >> 172.21.15.105:0/2515353069 conn(0x7fdcf80ceb90 msgr2=0x7fdcf80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:45.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.176+0000 7fdcff279640 1 -- 172.21.15.105:0/2515353069 shutdown_connections 2023-11-28T18:23:45.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.176+0000 7fdcff279640 1 -- 172.21.15.105:0/2515353069 wait complete. 2023-11-28T18:23:45.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.176+0000 7fdcff279640 1 Processor -- start 2023-11-28T18:23:45.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.176+0000 7fdcff279640 1 -- start start 2023-11-28T18:23:45.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.176+0000 7fdcff279640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcf806d610 0x7fdcf80aca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:45.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.176+0000 7fdcff279640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcf80ad9f0 0x7fdcf81448f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:45.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.176+0000 7fdcff279640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcf80cc7d0 0x7fdcf8144e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:45.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.176+0000 7fdcff279640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf8145370 con 0x7fdcf80ad9f0 2023-11-28T18:23:45.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.176+0000 7fdcff279640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf815fe00 con 0x7fdcf80cc7d0 2023-11-28T18:23:45.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.176+0000 7fdcff279640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf81600e0 con 0x7fdcf806d610 2023-11-28T18:23:45.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.176+0000 7fdcfc814640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcf80ad9f0 0x7fdcf81448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:45.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.176+0000 7fdcfc814640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcf80ad9f0 0x7fdcf81448f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38232/0 (socket says 172.21.15.105:38232) 2023-11-28T18:23:45.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.176+0000 7fdcfc814640 1 -- 172.21.15.105:0/1197299732 learned_addr learned my addr 172.21.15.105:0/1197299732 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:45.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.176+0000 7fdcfd015640 1 --2- 172.21.15.105:0/1197299732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcf806d610 0x7fdcf80aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:45.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.176+0000 7fdcfd816640 1 --2- 172.21.15.105:0/1197299732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcf80cc7d0 0x7fdcf8144e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:45.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.176+0000 7fdcfc814640 1 -- 172.21.15.105:0/1197299732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcf806d610 msgr2=0x7fdcf80aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:45.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.176+0000 7fdcfc814640 1 --2- 172.21.15.105:0/1197299732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcf806d610 0x7fdcf80aca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.176+0000 7fdcfc814640 1 -- 172.21.15.105:0/1197299732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcf80cc7d0 msgr2=0x7fdcf8144e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:45.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.176+0000 7fdcfc814640 1 --2- 172.21.15.105:0/1197299732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcf80cc7d0 0x7fdcf8144e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.176+0000 7fdcfc814640 1 -- 172.21.15.105:0/1197299732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce8009420 con 0x7fdcf80ad9f0 2023-11-28T18:23:45.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.176+0000 7fdcfc814640 1 --2- 172.21.15.105:0/1197299732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcf80ad9f0 0x7fdcf81448f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdce80190a0 tx=0x7fdce8019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:45.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.180+0000 7fdcedffb640 1 -- 172.21.15.105:0/1197299732 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce8004510 con 0x7fdcf80ad9f0 2023-11-28T18:23:45.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.180+0000 7fdcedffb640 1 -- 172.21.15.105:0/1197299732 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce80aa8c0 con 0x7fdcf80ad9f0 2023-11-28T18:23:45.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.180+0000 7fdcedffb640 1 -- 172.21.15.105:0/1197299732 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce80aac30 con 0x7fdcf80ad9f0 2023-11-28T18:23:45.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.180+0000 7fdcff279640 1 -- 172.21.15.105:0/1197299732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf81603c0 con 0x7fdcf80ad9f0 2023-11-28T18:23:45.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.180+0000 7fdcff279640 1 -- 172.21.15.105:0/1197299732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf80acfc0 con 0x7fdcf80ad9f0 2023-11-28T18:23:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.180+0000 7fdcff279640 1 -- 172.21.15.105:0/1197299732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf8156520 con 0x7fdcf80ad9f0 2023-11-28T18:23:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.180+0000 7fdcedffb640 1 -- 172.21.15.105:0/1197299732 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fdce8003620 con 0x7fdcf80ad9f0 2023-11-28T18:23:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.180+0000 7fdcedffb640 1 --2- 172.21.15.105:0/1197299732 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdcdc0668d0 0x7fdcdc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:45.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.180+0000 7fdcedffb640 1 -- 172.21.15.105:0/1197299732 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdce80a7070 con 0x7fdcf80ad9f0 2023-11-28T18:23:45.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.180+0000 7fdcfd015640 1 --2- 172.21.15.105:0/1197299732 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdcdc0668d0 0x7fdcdc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:45.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.184+0000 7fdcfd015640 1 --2- 172.21.15.105:0/1197299732 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdcdc0668d0 0x7fdcdc068d90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40011f0 tx=0x7fdcf4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.184+0000 7fdcedffb640 1 -- 172.21.15.105:0/1197299732 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdcf8156520 con 0x7fdcf80ad9f0 2023-11-28T18:23:45.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.336+0000 7fdcff279640 1 -- 172.21.15.105:0/1197299732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdcf8156520 con 0x7fdcf80ad9f0 2023-11-28T18:23:45.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.336+0000 7fdcedffb640 1 -- 172.21.15.105:0/1197299732 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdcf8156520 con 0x7fdcf80ad9f0 2023-11-28T18:23:45.338 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:23:45.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.336+0000 7fdcff279640 1 -- 172.21.15.105:0/1197299732 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdcdc0668d0 msgr2=0x7fdcdc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:45.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.336+0000 7fdcff279640 1 --2- 172.21.15.105:0/1197299732 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdcdc0668d0 0x7fdcdc068d90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40011f0 tx=0x7fdcf4009040 comp rx=0 tx=0).stop 2023-11-28T18:23:45.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.336+0000 7fdcff279640 1 -- 172.21.15.105:0/1197299732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcf80ad9f0 msgr2=0x7fdcf81448f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:45.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.336+0000 7fdcff279640 1 --2- 172.21.15.105:0/1197299732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcf80ad9f0 0x7fdcf81448f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdce80190a0 tx=0x7fdce8019230 comp rx=0 tx=0).stop 2023-11-28T18:23:45.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.336+0000 7fdcff279640 1 -- 172.21.15.105:0/1197299732 shutdown_connections 2023-11-28T18:23:45.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.336+0000 7fdcff279640 1 --2- 172.21.15.105:0/1197299732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcf80cc7d0 0x7fdcf8144e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.336+0000 7fdcff279640 1 --2- 172.21.15.105:0/1197299732 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdcdc0668d0 0x7fdcdc068d90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.336+0000 7fdcff279640 1 --2- 172.21.15.105:0/1197299732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcf80ad9f0 0x7fdcf81448f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.336+0000 7fdcff279640 1 --2- 172.21.15.105:0/1197299732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcf806d610 0x7fdcf80aca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.336+0000 7fdcff279640 1 -- 172.21.15.105:0/1197299732 >> 172.21.15.105:0/1197299732 conn(0x7fdcf80ceb90 msgr2=0x7fdcf8071f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:45.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.336+0000 7fdcff279640 1 -- 172.21.15.105:0/1197299732 shutdown_connections 2023-11-28T18:23:45.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.336+0000 7fdcff279640 1 -- 172.21.15.105:0/1197299732 wait complete. 2023-11-28T18:23:45.359 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:23:45.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.496+0000 7fad6cf34640 1 Processor -- start 2023-11-28T18:23:45.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.496+0000 7fad6cf34640 1 -- start start 2023-11-28T18:23:45.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.500+0000 7fad6cf34640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad6815d1a0 0x7fad680ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:45.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.500+0000 7fad6cf34640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fad680d0d40 0x7fad680d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:45.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.500+0000 7fad6cf34640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fad680ce970 0x7fad680d1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:45.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.500+0000 7fad6cf34640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad680cf440 con 0x7fad680ce970 2023-11-28T18:23:45.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.500+0000 7fad6cf34640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad680cffc0 con 0x7fad680d0d40 2023-11-28T18:23:45.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.500+0000 7fad6cf34640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad680cf690 con 0x7fad6815d1a0 2023-11-28T18:23:45.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.500+0000 7fad67fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fad680ce970 0x7fad680d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:45.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.500+0000 7fad67fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fad680ce970 0x7fad680d1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38240/0 (socket says 172.21.15.105:38240) 2023-11-28T18:23:45.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.500+0000 7fad67fff640 1 -- 172.21.15.105:0/916066769 learned_addr learned my addr 172.21.15.105:0/916066769 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:45.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.500+0000 7fad67fff640 1 -- 172.21.15.105:0/916066769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad6815d1a0 msgr2=0x7fad680ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:45.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.500+0000 7fad67fff640 1 --2- 172.21.15.105:0/916066769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad6815d1a0 0x7fad680ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.500+0000 7fad67fff640 1 -- 172.21.15.105:0/916066769 >> v2:172.21.15.148:3300/0 conn(0x7fad680d0d40 msgr2=0x7fad680d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:45.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.500+0000 7fad67fff640 1 --2- 172.21.15.105:0/916066769 >> v2:172.21.15.148:3300/0 conn(0x7fad680d0d40 0x7fad680d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.500+0000 7fad67fff640 1 -- 172.21.15.105:0/916066769 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad680cf970 con 0x7fad680ce970 2023-11-28T18:23:45.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.504+0000 7fad67fff640 1 --2- 172.21.15.105:0/916066769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad680ce970 0x7fad680d1930 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fad6808d0f0 tx=0x7fad6001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62263a816efe9935 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:45.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.504+0000 7fad667fc640 1 -- 172.21.15.105:0/916066769 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad600a57b0 con 0x7fad680ce970 2023-11-28T18:23:45.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.504+0000 7fad667fc640 1 -- 172.21.15.105:0/916066769 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad600a5910 con 0x7fad680ce970 2023-11-28T18:23:45.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.504+0000 7fad667fc640 1 -- 172.21.15.105:0/916066769 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad600a5ca0 con 0x7fad680ce970 2023-11-28T18:23:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.504+0000 7fad6cf34640 1 -- 172.21.15.105:0/916066769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad680ce970 msgr2=0x7fad680d1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.504+0000 7fad6cf34640 1 --2- 172.21.15.105:0/916066769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad680ce970 0x7fad680d1930 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fad6808d0f0 tx=0x7fad6001c910 comp rx=0 tx=0).stop 2023-11-28T18:23:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.504+0000 7fad6cf34640 1 -- 172.21.15.105:0/916066769 shutdown_connections 2023-11-28T18:23:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.504+0000 7fad6cf34640 1 --2- 172.21.15.105:0/916066769 >> v2:172.21.15.148:3300/0 conn(0x7fad680d0d40 0x7fad680d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.504+0000 7fad6cf34640 1 --2- 172.21.15.105:0/916066769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad680ce970 0x7fad680d1930 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.504+0000 7fad6cf34640 1 --2- 172.21.15.105:0/916066769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad6815d1a0 0x7fad680ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.504+0000 7fad6cf34640 1 -- 172.21.15.105:0/916066769 >> 172.21.15.105:0/916066769 conn(0x7fad680d2e10 msgr2=0x7fad680d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.504+0000 7fad6cf34640 1 -- 172.21.15.105:0/916066769 shutdown_connections 2023-11-28T18:23:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.504+0000 7fad6cf34640 1 -- 172.21.15.105:0/916066769 wait complete. 2023-11-28T18:23:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.504+0000 7fad6cf34640 1 Processor -- start 2023-11-28T18:23:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.504+0000 7fad6cf34640 1 -- start start 2023-11-28T18:23:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.504+0000 7fad6cf34640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad680ce970 0x7fad68153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.504+0000 7fad6cf34640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad680d0d40 0x7fad680b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.504+0000 7fad6cf34640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad6815d1a0 0x7fad680c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.504+0000 7fad6cf34640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad68155550 con 0x7fad680d0d40 2023-11-28T18:23:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.504+0000 7fad6cf34640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad681557f0 con 0x7fad6815d1a0 2023-11-28T18:23:45.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.504+0000 7fad6cf34640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad68155ad0 con 0x7fad680ce970 2023-11-28T18:23:45.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.508+0000 7fad66ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad680d0d40 0x7fad680b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:45.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.508+0000 7fad66ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad680d0d40 0x7fad680b6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38250/0 (socket says 172.21.15.105:38250) 2023-11-28T18:23:45.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.508+0000 7fad66ffd640 1 -- 172.21.15.105:0/2929986900 learned_addr learned my addr 172.21.15.105:0/2929986900 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:45.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.508+0000 7fad677fe640 1 --2- 172.21.15.105:0/2929986900 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad680ce970 0x7fad68153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:45.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.508+0000 7fad67fff640 1 --2- 172.21.15.105:0/2929986900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad6815d1a0 0x7fad680c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:45.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.508+0000 7fad66ffd640 1 -- 172.21.15.105:0/2929986900 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad680ce970 msgr2=0x7fad68153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:45.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.508+0000 7fad66ffd640 1 --2- 172.21.15.105:0/2929986900 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad680ce970 0x7fad68153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.508+0000 7fad66ffd640 1 -- 172.21.15.105:0/2929986900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad6815d1a0 msgr2=0x7fad680c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:45.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.508+0000 7fad66ffd640 1 --2- 172.21.15.105:0/2929986900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad6815d1a0 0x7fad680c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.508+0000 7fad66ffd640 1 -- 172.21.15.105:0/2929986900 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad60009b30 con 0x7fad680d0d40 2023-11-28T18:23:45.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.508+0000 7fad66ffd640 1 --2- 172.21.15.105:0/2929986900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad680d0d40 0x7fad680b6b40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00b550 tx=0x7fad5c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:45.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.508+0000 7fad4ffff640 1 -- 172.21.15.105:0/2929986900 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c010870 con 0x7fad680d0d40 2023-11-28T18:23:45.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.508+0000 7fad4ffff640 1 -- 172.21.15.105:0/2929986900 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5c0109f0 con 0x7fad680d0d40 2023-11-28T18:23:45.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.508+0000 7fad4ffff640 1 -- 172.21.15.105:0/2929986900 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c010da0 con 0x7fad680d0d40 2023-11-28T18:23:45.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.508+0000 7fad6cf34640 1 -- 172.21.15.105:0/2929986900 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad68155db0 con 0x7fad680d0d40 2023-11-28T18:23:45.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.508+0000 7fad6cf34640 1 -- 172.21.15.105:0/2929986900 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad68154170 con 0x7fad680d0d40 2023-11-28T18:23:45.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.508+0000 7fad6cf34640 1 -- 172.21.15.105:0/2929986900 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad68074580 con 0x7fad680d0d40 2023-11-28T18:23:45.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.508+0000 7fad4ffff640 1 -- 172.21.15.105:0/2929986900 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fad5c00f3c0 con 0x7fad680d0d40 2023-11-28T18:23:45.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.508+0000 7fad4ffff640 1 --2- 172.21.15.105:0/2929986900 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fad540668d0 0x7fad54068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:45.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.512+0000 7fad4ffff640 1 -- 172.21.15.105:0/2929986900 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fad5c084760 con 0x7fad680d0d40 2023-11-28T18:23:45.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.512+0000 7fad677fe640 1 --2- 172.21.15.105:0/2929986900 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fad540668d0 0x7fad54068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:45.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.512+0000 7fad677fe640 1 --2- 172.21.15.105:0/2929986900 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fad540668d0 0x7fad54068d90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fad58009990 tx=0x7fad58009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:45.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.512+0000 7fad4ffff640 1 -- 172.21.15.105:0/2929986900 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fad5c0516c0 con 0x7fad680d0d40 2023-11-28T18:23:45.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.664+0000 7fad6cf34640 1 -- 172.21.15.105:0/2929986900 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fad68154ac0 con 0x7fad680d0d40 2023-11-28T18:23:45.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.668+0000 7fad4ffff640 1 -- 172.21.15.105:0/2929986900 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fad5c054ce0 con 0x7fad680d0d40 2023-11-28T18:23:45.669 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:23:45.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.668+0000 7fad6cf34640 1 -- 172.21.15.105:0/2929986900 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fad540668d0 msgr2=0x7fad54068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:45.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.668+0000 7fad6cf34640 1 --2- 172.21.15.105:0/2929986900 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fad540668d0 0x7fad54068d90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fad58009990 tx=0x7fad58009110 comp rx=0 tx=0).stop 2023-11-28T18:23:45.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.668+0000 7fad6cf34640 1 -- 172.21.15.105:0/2929986900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad680d0d40 msgr2=0x7fad680b6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:45.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.668+0000 7fad6cf34640 1 --2- 172.21.15.105:0/2929986900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad680d0d40 0x7fad680b6b40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00b550 tx=0x7fad5c00ba10 comp rx=0 tx=0).stop 2023-11-28T18:23:45.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.668+0000 7fad6cf34640 1 -- 172.21.15.105:0/2929986900 shutdown_connections 2023-11-28T18:23:45.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.668+0000 7fad6cf34640 1 --2- 172.21.15.105:0/2929986900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad6815d1a0 0x7fad680c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.668+0000 7fad6cf34640 1 --2- 172.21.15.105:0/2929986900 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fad540668d0 0x7fad54068d90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.668+0000 7fad6cf34640 1 --2- 172.21.15.105:0/2929986900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad680d0d40 0x7fad680b6b40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.668+0000 7fad6cf34640 1 --2- 172.21.15.105:0/2929986900 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad680ce970 0x7fad68153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.668+0000 7fad6cf34640 1 -- 172.21.15.105:0/2929986900 >> 172.21.15.105:0/2929986900 conn(0x7fad680d2e10 msgr2=0x7fad6815dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:45.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.668+0000 7fad6cf34640 1 -- 172.21.15.105:0/2929986900 shutdown_connections 2023-11-28T18:23:45.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.668+0000 7fad6cf34640 1 -- 172.21.15.105:0/2929986900 wait complete. 2023-11-28T18:23:45.690 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:23:45.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.828+0000 7fcca891d640 1 Processor -- start 2023-11-28T18:23:45.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.828+0000 7fcca891d640 1 -- start start 2023-11-28T18:23:45.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.828+0000 7fcca891d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcca406bf50 0x7fcca415d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:45.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.828+0000 7fcca891d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcca40adb80 0x7fcca40adff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:45.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.828+0000 7fcca891d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcca40cc7f0 con 0x7fcca4154070 2023-11-28T18:23:45.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.828+0000 7fcca891d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca4072540 con 0x7fcca40adb80 2023-11-28T18:23:45.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.828+0000 7fcca891d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca40ae530 con 0x7fcca406bf50 2023-11-28T18:23:45.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.828+0000 7fcca2d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcca40adb80 0x7fcca40adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:45.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.828+0000 7fcca1d9b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcca406bf50 0x7fcca415d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:45.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.828+0000 7fcca2d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcca40adb80 0x7fcca40adff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:60750/0 (socket says 172.21.15.105:60750) 2023-11-28T18:23:45.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.828+0000 7fcca2d9d640 1 -- 172.21.15.105:0/168535229 learned_addr learned my addr 172.21.15.105:0/168535229 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:45.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.832+0000 7fcca1d9b640 1 -- 172.21.15.105:0/168535229 >> v2:172.21.15.148:3300/0 conn(0x7fcca40adb80 msgr2=0x7fcca40adff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:45.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.832+0000 7fcca1d9b640 1 --2- 172.21.15.105:0/168535229 >> v2:172.21.15.148:3300/0 conn(0x7fcca40adb80 0x7fcca40adff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.832+0000 7fcca1d9b640 1 -- 172.21.15.105:0/168535229 >> v1:172.21.15.148:6789/0 conn(0x7fcca4154070 legacy=0x7fcca40ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:45.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.832+0000 7fcca1d9b640 1 -- 172.21.15.105:0/168535229 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca415dd10 con 0x7fcca406bf50 2023-11-28T18:23:45.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.832+0000 7fcca1d9b640 1 --2- 172.21.15.105:0/168535229 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcca406bf50 0x7fcca415d700 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcc980097c0 tx=0x7fcc9801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c534a70072b0e26b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:45.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.832+0000 7fcca159a640 1 -- 172.21.15.105:0/168535229 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc980ab600 con 0x7fcca406bf50 2023-11-28T18:23:45.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.832+0000 7fcca159a640 1 -- 172.21.15.105:0/168535229 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc980a7050 con 0x7fcca406bf50 2023-11-28T18:23:45.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.832+0000 7fcca159a640 1 -- 172.21.15.105:0/168535229 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc980ab980 con 0x7fcca406bf50 2023-11-28T18:23:45.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.832+0000 7fcca891d640 1 -- 172.21.15.105:0/168535229 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcca406bf50 msgr2=0x7fcca415d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:45.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.832+0000 7fcca891d640 1 --2- 172.21.15.105:0/168535229 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcca406bf50 0x7fcca415d700 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcc980097c0 tx=0x7fcc9801c450 comp rx=0 tx=0).stop 2023-11-28T18:23:45.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.836+0000 7fcca891d640 1 -- 172.21.15.105:0/168535229 shutdown_connections 2023-11-28T18:23:45.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.836+0000 7fcca891d640 1 --2- 172.21.15.105:0/168535229 >> v2:172.21.15.148:3300/0 conn(0x7fcca40adb80 0x7fcca40adff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.836+0000 7fcca891d640 1 --2- 172.21.15.105:0/168535229 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcca406bf50 0x7fcca415d700 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.836+0000 7fcca891d640 1 -- 172.21.15.105:0/168535229 >> 172.21.15.105:0/168535229 conn(0x7fcca40ceb90 msgr2=0x7fcca40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:45.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.836+0000 7fcca891d640 1 -- 172.21.15.105:0/168535229 shutdown_connections 2023-11-28T18:23:45.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.836+0000 7fcca891d640 1 -- 172.21.15.105:0/168535229 wait complete. 2023-11-28T18:23:45.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.836+0000 7fcca891d640 1 Processor -- start 2023-11-28T18:23:45.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.836+0000 7fcca891d640 1 -- start start 2023-11-28T18:23:45.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.836+0000 7fcca891d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca406bf50 0x7fcca41376b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:45.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.836+0000 7fcca891d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcca40adb80 0x7fcca414c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:45.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.836+0000 7fcca891d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca4154070 0x7fcca4150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:45.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.836+0000 7fcca891d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca4153a40 con 0x7fcca406bf50 2023-11-28T18:23:45.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.836+0000 7fcca891d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca4152e30 con 0x7fcca4154070 2023-11-28T18:23:45.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.836+0000 7fcca891d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca4153110 con 0x7fcca40adb80 2023-11-28T18:23:45.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.836+0000 7fcca259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca406bf50 0x7fcca41376b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:45.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.836+0000 7fcca259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca406bf50 0x7fcca41376b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38260/0 (socket says 172.21.15.105:38260) 2023-11-28T18:23:45.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.836+0000 7fcca259c640 1 -- 172.21.15.105:0/3926008286 learned_addr learned my addr 172.21.15.105:0/3926008286 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:45.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.836+0000 7fcca1d9b640 1 --2- 172.21.15.105:0/3926008286 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcca40adb80 0x7fcca414c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:45.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.836+0000 7fcca2d9d640 1 --2- 172.21.15.105:0/3926008286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca4154070 0x7fcca4150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:45.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.836+0000 7fcca259c640 1 -- 172.21.15.105:0/3926008286 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcca40adb80 msgr2=0x7fcca414c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:45.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.836+0000 7fcca259c640 1 --2- 172.21.15.105:0/3926008286 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcca40adb80 0x7fcca414c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.840+0000 7fcca259c640 1 -- 172.21.15.105:0/3926008286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca4154070 msgr2=0x7fcca4150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:45.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.840+0000 7fcca259c640 1 --2- 172.21.15.105:0/3926008286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca4154070 0x7fcca4150d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:45.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.840+0000 7fcca259c640 1 -- 172.21.15.105:0/3926008286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc98009420 con 0x7fcca406bf50 2023-11-28T18:23:45.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.840+0000 7fcca259c640 1 --2- 172.21.15.105:0/3926008286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca406bf50 0x7fcca41376b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400c3c0 tx=0x7fcc9400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:45.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.840+0000 7fcc92ffd640 1 -- 172.21.15.105:0/3926008286 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc940052d0 con 0x7fcca406bf50 2023-11-28T18:23:45.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.840+0000 7fcc92ffd640 1 -- 172.21.15.105:0/3926008286 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc94005430 con 0x7fcca406bf50 2023-11-28T18:23:45.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.840+0000 7fcca891d640 1 -- 172.21.15.105:0/3926008286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca41533f0 con 0x7fcca406bf50 2023-11-28T18:23:45.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.840+0000 7fcc92ffd640 1 -- 172.21.15.105:0/3926008286 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc94003840 con 0x7fcca406bf50 2023-11-28T18:23:45.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.840+0000 7fcca891d640 1 -- 172.21.15.105:0/3926008286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca414c9b0 con 0x7fcca406bf50 2023-11-28T18:23:45.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.840+0000 7fcca891d640 1 -- 172.21.15.105:0/3926008286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca4067570 con 0x7fcca406bf50 2023-11-28T18:23:45.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.840+0000 7fcc92ffd640 1 -- 172.21.15.105:0/3926008286 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fcc940039a0 con 0x7fcca406bf50 2023-11-28T18:23:45.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.840+0000 7fcc92ffd640 1 --2- 172.21.15.105:0/3926008286 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcc840668d0 0x7fcc84068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:45.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.840+0000 7fcc92ffd640 1 -- 172.21.15.105:0/3926008286 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcc94083bc0 con 0x7fcca406bf50 2023-11-28T18:23:45.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.844+0000 7fcca1d9b640 1 --2- 172.21.15.105:0/3926008286 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcc840668d0 0x7fcc84068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:45.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.844+0000 7fcca1d9b640 1 --2- 172.21.15.105:0/3926008286 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcc840668d0 0x7fcc84068d90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcc980ad000 tx=0x7fcc98008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:45.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:45.844+0000 7fcc92ffd640 1 -- 172.21.15.105:0/3926008286 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcc94050bc0 con 0x7fcca406bf50 2023-11-28T18:23:46.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.000+0000 7fcca891d640 1 -- 172.21.15.105:0/3926008286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcca40cf950 con 0x7fcca406bf50 2023-11-28T18:23:46.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.000+0000 7fcc92ffd640 1 -- 172.21.15.105:0/3926008286 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcc940170a0 con 0x7fcca406bf50 2023-11-28T18:23:46.002 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:23:46.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.000+0000 7fcca891d640 1 -- 172.21.15.105:0/3926008286 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcc840668d0 msgr2=0x7fcc84068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.000+0000 7fcca891d640 1 --2- 172.21.15.105:0/3926008286 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcc840668d0 0x7fcc84068d90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcc980ad000 tx=0x7fcc98008040 comp rx=0 tx=0).stop 2023-11-28T18:23:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.000+0000 7fcca891d640 1 -- 172.21.15.105:0/3926008286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca406bf50 msgr2=0x7fcca41376b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.000+0000 7fcca891d640 1 --2- 172.21.15.105:0/3926008286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca406bf50 0x7fcca41376b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400c3c0 tx=0x7fcc9400c880 comp rx=0 tx=0).stop 2023-11-28T18:23:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.000+0000 7fcca891d640 1 -- 172.21.15.105:0/3926008286 shutdown_connections 2023-11-28T18:23:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.000+0000 7fcca891d640 1 --2- 172.21.15.105:0/3926008286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca4154070 0x7fcca4150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.000+0000 7fcca891d640 1 --2- 172.21.15.105:0/3926008286 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcca40adb80 0x7fcca414c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.000+0000 7fcca891d640 1 --2- 172.21.15.105:0/3926008286 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcc840668d0 0x7fcc84068d90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.000+0000 7fcca891d640 1 --2- 172.21.15.105:0/3926008286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca406bf50 0x7fcca41376b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.000+0000 7fcca891d640 1 -- 172.21.15.105:0/3926008286 >> 172.21.15.105:0/3926008286 conn(0x7fcca40ceb90 msgr2=0x7fcca40aeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.000+0000 7fcca891d640 1 -- 172.21.15.105:0/3926008286 shutdown_connections 2023-11-28T18:23:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.000+0000 7fcca891d640 1 -- 172.21.15.105:0/3926008286 wait complete. 2023-11-28T18:23:46.022 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T18:23:46.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.164+0000 7fdac00d6640 1 Processor -- start 2023-11-28T18:23:46.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.164+0000 7fdac00d6640 1 -- start start 2023-11-28T18:23:46.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.164+0000 7fdac00d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdab815d4f0 0x7fdab80d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:46.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.164+0000 7fdac00d6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab80b25b0 con 0x7fdab80b2d20 2023-11-28T18:23:46.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.164+0000 7fdac00d6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab80cea30 con 0x7fdab80d3c30 2023-11-28T18:23:46.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.164+0000 7fdac00d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab80b2900 con 0x7fdab815d4f0 2023-11-28T18:23:46.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.164+0000 7fdabe673640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fdab80b2d20 0x7fdab80b31d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:60172/0 (socket says 172.21.15.105:60172) 2023-11-28T18:23:46.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.164+0000 7fdabe673640 1 -- 172.21.15.105:0/2921791012 learned_addr learned my addr 172.21.15.105:0/2921791012 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:46.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.164+0000 7fdabde72640 1 --2- 172.21.15.105:0/2921791012 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdab815d4f0 0x7fdab80d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:46.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.164+0000 7fdabce70640 1 -- 172.21.15.105:0/2921791012 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2107152895 0 0) 0x7fdab80b25b0 con 0x7fdab80b2d20 2023-11-28T18:23:46.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.164+0000 7fdabce70640 1 -- 172.21.15.105:0/2921791012 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaa4003530 con 0x7fdab80b2d20 2023-11-28T18:23:46.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.164+0000 7fdabce70640 1 -- 172.21.15.105:0/2921791012 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3077590000 0 0) 0x7fdab80cea30 con 0x7fdab80d3c30 2023-11-28T18:23:46.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.164+0000 7fdabce70640 1 -- 172.21.15.105:0/2921791012 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdab80b25b0 con 0x7fdab80d3c30 2023-11-28T18:23:46.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.164+0000 7fdabde72640 1 -- 172.21.15.105:0/2921791012 >> v1:172.21.15.148:6789/0 conn(0x7fdab80d3c30 legacy=0x7fdab80d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:46.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.164+0000 7fdabde72640 1 -- 172.21.15.105:0/2921791012 >> v1:172.21.15.105:6789/0 conn(0x7fdab80b2d20 legacy=0x7fdab80b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:46.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.164+0000 7fdabde72640 1 -- 172.21.15.105:0/2921791012 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab80ced10 con 0x7fdab815d4f0 2023-11-28T18:23:46.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.168+0000 7fdabde72640 1 --2- 172.21.15.105:0/2921791012 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdab815d4f0 0x7fdab80d0e10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdab40097f0 tx=0x7fdab401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce94d2a5ded33a60 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:46.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.168+0000 7fdabce70640 1 -- 172.21.15.105:0/2921791012 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab40ab600 con 0x7fdab815d4f0 2023-11-28T18:23:46.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.168+0000 7fdabce70640 1 -- 172.21.15.105:0/2921791012 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab40a7050 con 0x7fdab815d4f0 2023-11-28T18:23:46.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.168+0000 7fdabce70640 1 -- 172.21.15.105:0/2921791012 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab40ab980 con 0x7fdab815d4f0 2023-11-28T18:23:46.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.168+0000 7fdac00d6640 1 -- 172.21.15.105:0/2921791012 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdab815d4f0 msgr2=0x7fdab80d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:46.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.168+0000 7fdac00d6640 1 --2- 172.21.15.105:0/2921791012 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdab815d4f0 0x7fdab80d0e10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdab40097f0 tx=0x7fdab401c400 comp rx=0 tx=0).stop 2023-11-28T18:23:46.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.168+0000 7fdac00d6640 1 -- 172.21.15.105:0/2921791012 shutdown_connections 2023-11-28T18:23:46.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.168+0000 7fdac00d6640 1 --2- 172.21.15.105:0/2921791012 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdab815d4f0 0x7fdab80d0e10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.168+0000 7fdac00d6640 1 -- 172.21.15.105:0/2921791012 >> 172.21.15.105:0/2921791012 conn(0x7fdab80d3030 msgr2=0x7fdab80d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:46.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.168+0000 7fdac00d6640 1 -- 172.21.15.105:0/2921791012 shutdown_connections 2023-11-28T18:23:46.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.172+0000 7fdac00d6640 1 -- 172.21.15.105:0/2921791012 wait complete. 2023-11-28T18:23:46.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.172+0000 7fdac00d6640 1 Processor -- start 2023-11-28T18:23:46.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.172+0000 7fdac00d6640 1 -- start start 2023-11-28T18:23:46.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.172+0000 7fdac00d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab80b2d20 0x7fdab8153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:46.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.172+0000 7fdac00d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdab80d3c30 0x7fdab8160010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:46.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.172+0000 7fdac00d6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdab815d4f0 0x7fdab81542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:46.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.172+0000 7fdac00d6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab80d0a20 con 0x7fdab815d4f0 2023-11-28T18:23:46.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.172+0000 7fdac00d6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab81565f0 con 0x7fdab80b2d20 2023-11-28T18:23:46.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.172+0000 7fdac00d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab80b6f40 con 0x7fdab80d3c30 2023-11-28T18:23:46.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.172+0000 7fdabe673640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdab815d4f0 0x7fdab81542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:46.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.172+0000 7fdabe673640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdab815d4f0 0x7fdab81542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38272/0 (socket says 172.21.15.105:38272) 2023-11-28T18:23:46.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.172+0000 7fdabe673640 1 -- 172.21.15.105:0/972292499 learned_addr learned my addr 172.21.15.105:0/972292499 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:46.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.172+0000 7fdabde72640 1 --2- 172.21.15.105:0/972292499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab80b2d20 0x7fdab8153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:46.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.172+0000 7fdabd671640 1 --2- 172.21.15.105:0/972292499 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdab80d3c30 0x7fdab8160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:46.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.172+0000 7fdabe673640 1 -- 172.21.15.105:0/972292499 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdab80d3c30 msgr2=0x7fdab8160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:46.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.172+0000 7fdabe673640 1 --2- 172.21.15.105:0/972292499 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdab80d3c30 0x7fdab8160010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.172+0000 7fdabe673640 1 -- 172.21.15.105:0/972292499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab80b2d20 msgr2=0x7fdab8153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:46.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.172+0000 7fdabe673640 1 --2- 172.21.15.105:0/972292499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab80b2d20 0x7fdab8153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.172+0000 7fdabe673640 1 -- 172.21.15.105:0/972292499 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab4009420 con 0x7fdab815d4f0 2023-11-28T18:23:46.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.172+0000 7fdabe673640 1 --2- 172.21.15.105:0/972292499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdab815d4f0 0x7fdab81542b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdab000cb20 tx=0x7fdab000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:46.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.176+0000 7fdaae7fc640 1 -- 172.21.15.105:0/972292499 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab000aca0 con 0x7fdab815d4f0 2023-11-28T18:23:46.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.176+0000 7fdaae7fc640 1 -- 172.21.15.105:0/972292499 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab000ae00 con 0x7fdab815d4f0 2023-11-28T18:23:46.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.176+0000 7fdaae7fc640 1 -- 172.21.15.105:0/972292499 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab0012600 con 0x7fdab815d4f0 2023-11-28T18:23:46.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.176+0000 7fdac00d6640 1 -- 172.21.15.105:0/972292499 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab8160550 con 0x7fdab815d4f0 2023-11-28T18:23:46.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.176+0000 7fdac00d6640 1 -- 172.21.15.105:0/972292499 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab815f5c0 con 0x7fdab815d4f0 2023-11-28T18:23:46.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.176+0000 7fdac00d6640 1 -- 172.21.15.105:0/972292499 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab8067570 con 0x7fdab815d4f0 2023-11-28T18:23:46.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.176+0000 7fdaae7fc640 1 -- 172.21.15.105:0/972292499 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fdab0007070 con 0x7fdab815d4f0 2023-11-28T18:23:46.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.176+0000 7fdaae7fc640 1 --2- 172.21.15.105:0/972292499 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fda9c0668d0 0x7fda9c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:46.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.176+0000 7fdaae7fc640 1 -- 172.21.15.105:0/972292499 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdab0084460 con 0x7fdab815d4f0 2023-11-28T18:23:46.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.176+0000 7fdabde72640 1 --2- 172.21.15.105:0/972292499 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fda9c0668d0 0x7fda9c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:46.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.180+0000 7fdabde72640 1 --2- 172.21.15.105:0/972292499 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fda9c0668d0 0x7fda9c068d90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdab40ad000 tx=0x7fdab40a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:46.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.180+0000 7fdaae7fc640 1 -- 172.21.15.105:0/972292499 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdab00513c0 con 0x7fdab815d4f0 2023-11-28T18:23:46.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.332+0000 7fdac00d6640 1 -- 172.21.15.105:0/972292499 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdab8160550 con 0x7fdab815d4f0 2023-11-28T18:23:46.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.332+0000 7fdaae7fc640 1 -- 172.21.15.105:0/972292499 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdab8160550 con 0x7fdab815d4f0 2023-11-28T18:23:46.336 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:23:46.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.336+0000 7fdac00d6640 1 -- 172.21.15.105:0/972292499 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fda9c0668d0 msgr2=0x7fda9c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:46.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.336+0000 7fdac00d6640 1 --2- 172.21.15.105:0/972292499 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fda9c0668d0 0x7fda9c068d90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdab40ad000 tx=0x7fdab40a6000 comp rx=0 tx=0).stop 2023-11-28T18:23:46.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.336+0000 7fdac00d6640 1 -- 172.21.15.105:0/972292499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdab815d4f0 msgr2=0x7fdab81542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:46.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.336+0000 7fdac00d6640 1 --2- 172.21.15.105:0/972292499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdab815d4f0 0x7fdab81542b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdab000cb20 tx=0x7fdab000a060 comp rx=0 tx=0).stop 2023-11-28T18:23:46.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.336+0000 7fdac00d6640 1 -- 172.21.15.105:0/972292499 shutdown_connections 2023-11-28T18:23:46.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.336+0000 7fdac00d6640 1 --2- 172.21.15.105:0/972292499 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fda9c0668d0 0x7fda9c068d90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.336+0000 7fdac00d6640 1 --2- 172.21.15.105:0/972292499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdab815d4f0 0x7fdab81542b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.336+0000 7fdac00d6640 1 --2- 172.21.15.105:0/972292499 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdab80d3c30 0x7fdab8160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.336+0000 7fdac00d6640 1 --2- 172.21.15.105:0/972292499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab80b2d20 0x7fdab8153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.336+0000 7fdac00d6640 1 -- 172.21.15.105:0/972292499 >> 172.21.15.105:0/972292499 conn(0x7fdab80d3030 msgr2=0x7fdab80d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:46.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.336+0000 7fdac00d6640 1 -- 172.21.15.105:0/972292499 shutdown_connections 2023-11-28T18:23:46.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.336+0000 7fdac00d6640 1 -- 172.21.15.105:0/972292499 wait complete. 2023-11-28T18:23:46.357 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T18:23:46.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.496+0000 7f4a2ca17640 1 Processor -- start 2023-11-28T18:23:46.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.496+0000 7f4a2ca17640 1 -- start start 2023-11-28T18:23:46.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.496+0000 7f4a2ca17640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4a280b4290 0x7f4a280b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:46.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.496+0000 7f4a2ca17640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4a28071960 0x7f4a2815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:46.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.496+0000 7f4a2ca17640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a2815b030 0x7f4a280709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:46.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.496+0000 7f4a2ca17640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a280b4bb0 con 0x7f4a280b4290 2023-11-28T18:23:46.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.496+0000 7f4a2ca17640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a280701d0 con 0x7f4a28071960 2023-11-28T18:23:46.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.496+0000 7f4a2ca17640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2806df80 con 0x7f4a2815b030 2023-11-28T18:23:46.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.496+0000 7f4a2659c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4a280b4290 0x7f4a280b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:46.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.496+0000 7f4a2659c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4a280b4290 0x7f4a280b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38280/0 (socket says 172.21.15.105:38280) 2023-11-28T18:23:46.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.496+0000 7f4a2659c640 1 -- 172.21.15.105:0/102970013 learned_addr learned my addr 172.21.15.105:0/102970013 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:46.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.496+0000 7f4a2659c640 1 -- 172.21.15.105:0/102970013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a2815b030 msgr2=0x7f4a280709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:46.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.496+0000 7f4a2659c640 1 --2- 172.21.15.105:0/102970013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a2815b030 0x7f4a280709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.496+0000 7f4a2659c640 1 -- 172.21.15.105:0/102970013 >> v2:172.21.15.148:3300/0 conn(0x7f4a28071960 msgr2=0x7f4a2815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:46.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.496+0000 7f4a25d9b640 1 --2- 172.21.15.105:0/102970013 >> v2:172.21.15.148:3300/0 conn(0x7f4a28071960 0x7f4a2815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:46.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.496+0000 7f4a2659c640 1 --2- 172.21.15.105:0/102970013 >> v2:172.21.15.148:3300/0 conn(0x7f4a28071960 0x7f4a2815f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.496+0000 7f4a2659c640 1 -- 172.21.15.105:0/102970013 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a28071df0 con 0x7f4a280b4290 2023-11-28T18:23:46.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.496+0000 7f4a25d9b640 1 --2- 172.21.15.105:0/102970013 >> v2:172.21.15.148:3300/0 conn(0x7f4a28071960 0x7f4a2815f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:46.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.500+0000 7f4a2659c640 1 --2- 172.21.15.105:0/102970013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a280b4290 0x7f4a280b4670 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4a2806e520 tx=0x7f4a1801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b8e7520fd53008 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:46.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.500+0000 7f4a2559a640 1 -- 172.21.15.105:0/102970013 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a180ac600 con 0x7f4a280b4290 2023-11-28T18:23:46.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.500+0000 7f4a2559a640 1 -- 172.21.15.105:0/102970013 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a180a8050 con 0x7f4a280b4290 2023-11-28T18:23:46.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.500+0000 7f4a2559a640 1 -- 172.21.15.105:0/102970013 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a180ac980 con 0x7f4a280b4290 2023-11-28T18:23:46.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.500+0000 7f4a2ca17640 1 -- 172.21.15.105:0/102970013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a280b4290 msgr2=0x7f4a280b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:46.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.500+0000 7f4a2ca17640 1 --2- 172.21.15.105:0/102970013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a280b4290 0x7f4a280b4670 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4a2806e520 tx=0x7f4a1801c4d0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.500+0000 7f4a2ca17640 1 -- 172.21.15.105:0/102970013 shutdown_connections 2023-11-28T18:23:46.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.500+0000 7f4a2ca17640 1 --2- 172.21.15.105:0/102970013 >> v2:172.21.15.148:3300/0 conn(0x7f4a28071960 0x7f4a2815f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.500+0000 7f4a2ca17640 1 --2- 172.21.15.105:0/102970013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a2815b030 0x7f4a280709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.500+0000 7f4a2ca17640 1 --2- 172.21.15.105:0/102970013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a280b4290 0x7f4a280b4670 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.500+0000 7f4a2ca17640 1 -- 172.21.15.105:0/102970013 >> 172.21.15.105:0/102970013 conn(0x7f4a280ceb90 msgr2=0x7f4a280cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:46.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.500+0000 7f4a2ca17640 1 -- 172.21.15.105:0/102970013 shutdown_connections 2023-11-28T18:23:46.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.500+0000 7f4a2ca17640 1 -- 172.21.15.105:0/102970013 wait complete. 2023-11-28T18:23:46.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a2ca17640 1 Processor -- start 2023-11-28T18:23:46.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a2ca17640 1 -- start start 2023-11-28T18:23:46.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a2ca17640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a280b4290 0x7f4a281464e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:46.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a2ca17640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a2815b030 0x7f4a280ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:46.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a2ca17640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a280aee00 0x7f4a280cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:46.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a2ca17640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2809a600 con 0x7f4a280b4290 2023-11-28T18:23:46.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a2ca17640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a280af290 con 0x7f4a280aee00 2023-11-28T18:23:46.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a2ca17640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a28099a50 con 0x7f4a2815b030 2023-11-28T18:23:46.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a2659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a280b4290 0x7f4a281464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:46.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a2659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a280b4290 0x7f4a281464e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38290/0 (socket says 172.21.15.105:38290) 2023-11-28T18:23:46.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a2659c640 1 -- 172.21.15.105:0/2252375052 learned_addr learned my addr 172.21.15.105:0/2252375052 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:46.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a25d9b640 1 --2- 172.21.15.105:0/2252375052 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a2815b030 0x7f4a280ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:46.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a26d9d640 1 --2- 172.21.15.105:0/2252375052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a280aee00 0x7f4a280cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:46.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a2659c640 1 -- 172.21.15.105:0/2252375052 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a2815b030 msgr2=0x7f4a280ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:46.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a2659c640 1 --2- 172.21.15.105:0/2252375052 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a2815b030 0x7f4a280ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a2659c640 1 -- 172.21.15.105:0/2252375052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a280aee00 msgr2=0x7f4a280cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:46.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a2659c640 1 --2- 172.21.15.105:0/2252375052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a280aee00 0x7f4a280cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a2659c640 1 -- 172.21.15.105:0/2252375052 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a180094d0 con 0x7f4a280b4290 2023-11-28T18:23:46.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a26d9d640 1 --2- 172.21.15.105:0/2252375052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a280aee00 0x7f4a280cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:46.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a2659c640 1 --2- 172.21.15.105:0/2252375052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a280b4290 0x7f4a281464e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4a1801ab80 tx=0x7f4a1801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:46.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a16ffd640 1 -- 172.21.15.105:0/2252375052 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a180b7620 con 0x7f4a280b4290 2023-11-28T18:23:46.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a16ffd640 1 -- 172.21.15.105:0/2252375052 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a180b7780 con 0x7f4a280b4290 2023-11-28T18:23:46.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a16ffd640 1 -- 172.21.15.105:0/2252375052 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a180b7af0 con 0x7f4a280b4290 2023-11-28T18:23:46.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a2ca17640 1 -- 172.21.15.105:0/2252375052 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a28099c60 con 0x7f4a280b4290 2023-11-28T18:23:46.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a2ca17640 1 -- 172.21.15.105:0/2252375052 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a2809a220 con 0x7f4a280b4290 2023-11-28T18:23:46.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.504+0000 7f4a2ca17640 1 -- 172.21.15.105:0/2252375052 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a28067570 con 0x7f4a280b4290 2023-11-28T18:23:46.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.508+0000 7f4a16ffd640 1 -- 172.21.15.105:0/2252375052 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f4a180b3040 con 0x7f4a280b4290 2023-11-28T18:23:46.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.508+0000 7f4a16ffd640 1 --2- 172.21.15.105:0/2252375052 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4a080668d0 0x7f4a08068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:46.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.508+0000 7f4a16ffd640 1 -- 172.21.15.105:0/2252375052 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4a1811e930 con 0x7f4a280b4290 2023-11-28T18:23:46.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.508+0000 7f4a25d9b640 1 --2- 172.21.15.105:0/2252375052 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4a080668d0 0x7f4a08068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:46.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.508+0000 7f4a25d9b640 1 --2- 172.21.15.105:0/2252375052 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4a080668d0 0x7f4a08068d90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4a281f9750 tx=0x7f4a1c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:46.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.512+0000 7f4a16ffd640 1 -- 172.21.15.105:0/2252375052 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4a180eb870 con 0x7f4a280b4290 2023-11-28T18:23:46.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.672+0000 7f4a2ca17640 1 -- 172.21.15.105:0/2252375052 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4a280cf7f0 con 0x7f4a280b4290 2023-11-28T18:23:46.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.672+0000 7f4a16ffd640 1 -- 172.21.15.105:0/2252375052 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4a180eee90 con 0x7f4a280b4290 2023-11-28T18:23:46.676 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2023-11-28T18:23:46.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.676+0000 7f4a2ca17640 1 -- 172.21.15.105:0/2252375052 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4a080668d0 msgr2=0x7f4a08068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:46.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.676+0000 7f4a2ca17640 1 --2- 172.21.15.105:0/2252375052 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4a080668d0 0x7f4a08068d90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4a281f9750 tx=0x7f4a1c008040 comp rx=0 tx=0).stop 2023-11-28T18:23:46.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.676+0000 7f4a2ca17640 1 -- 172.21.15.105:0/2252375052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a280b4290 msgr2=0x7f4a281464e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:46.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.676+0000 7f4a2ca17640 1 --2- 172.21.15.105:0/2252375052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a280b4290 0x7f4a281464e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4a1801ab80 tx=0x7f4a1801ad10 comp rx=0 tx=0).stop 2023-11-28T18:23:46.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.676+0000 7f4a2ca17640 1 -- 172.21.15.105:0/2252375052 shutdown_connections 2023-11-28T18:23:46.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.676+0000 7f4a2ca17640 1 --2- 172.21.15.105:0/2252375052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a280aee00 0x7f4a280cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.676+0000 7f4a2ca17640 1 --2- 172.21.15.105:0/2252375052 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a2815b030 0x7f4a280ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.676+0000 7f4a2ca17640 1 --2- 172.21.15.105:0/2252375052 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4a080668d0 0x7f4a08068d90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.676+0000 7f4a2ca17640 1 --2- 172.21.15.105:0/2252375052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a280b4290 0x7f4a281464e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.676+0000 7f4a2ca17640 1 -- 172.21.15.105:0/2252375052 >> 172.21.15.105:0/2252375052 conn(0x7f4a280ceb90 msgr2=0x7f4a280afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:46.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.676+0000 7f4a2ca17640 1 -- 172.21.15.105:0/2252375052 shutdown_connections 2023-11-28T18:23:46.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.676+0000 7f4a2ca17640 1 -- 172.21.15.105:0/2252375052 wait complete. 2023-11-28T18:23:46.697 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:23:46.697 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:23:46.706 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:23:46.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.884+0000 7f3663f1d640 1 Processor -- start 2023-11-28T18:23:46.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.884+0000 7f3663f1d640 1 -- start start 2023-11-28T18:23:46.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.884+0000 7f3663f1d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f365c15d320 0x7f365c06c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:46.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.884+0000 7f3663f1d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f365c0ce230 0x7f365c0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:46.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.884+0000 7f3663f1d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f365c0cdbb0 0x7f365c0cef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:46.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.884+0000 7f3663f1d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f365c0b3c40 con 0x7f365c15d320 2023-11-28T18:23:46.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.884+0000 7f3663f1d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f365c06ca80 con 0x7f365c0cdbb0 2023-11-28T18:23:46.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.884+0000 7f3663f1d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365c15d700 con 0x7f365c0ce230 2023-11-28T18:23:46.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.884+0000 7f3661cb9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f365c15d320 0x7f365c06c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:46.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.884+0000 7f3661cb9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f365c15d320 0x7f365c06c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38304/0 (socket says 172.21.15.105:38304) 2023-11-28T18:23:46.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.884+0000 7f3661cb9640 1 -- 172.21.15.105:0/3969031329 learned_addr learned my addr 172.21.15.105:0/3969031329 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:46.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.884+0000 7f36624ba640 1 --2- 172.21.15.105:0/3969031329 >> v2:172.21.15.148:3300/0 conn(0x7f365c0cdbb0 0x7f365c0cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:46.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.884+0000 7f36614b8640 1 --2- 172.21.15.105:0/3969031329 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f365c0ce230 0x7f365c0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:46.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.884+0000 7f3661cb9640 1 -- 172.21.15.105:0/3969031329 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f365c0ce230 msgr2=0x7f365c0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:46.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.884+0000 7f3661cb9640 1 --2- 172.21.15.105:0/3969031329 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f365c0ce230 0x7f365c0cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.884+0000 7f3661cb9640 1 -- 172.21.15.105:0/3969031329 >> v2:172.21.15.148:3300/0 conn(0x7f365c0cdbb0 msgr2=0x7f365c0cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:46.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.884+0000 7f3661cb9640 1 --2- 172.21.15.105:0/3969031329 >> v2:172.21.15.148:3300/0 conn(0x7f365c0cdbb0 0x7f365c0cef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.884+0000 7f3661cb9640 1 -- 172.21.15.105:0/3969031329 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c0cdfc0 con 0x7f365c15d320 2023-11-28T18:23:46.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.888+0000 7f3661cb9640 1 --2- 172.21.15.105:0/3969031329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f365c15d320 0x7f365c06c0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f365c06d850 tx=0x7f365801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457131397ee9f4f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:46.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.888+0000 7f3660cb7640 1 -- 172.21.15.105:0/3969031329 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36580a7070 con 0x7f365c15d320 2023-11-28T18:23:46.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.888+0000 7f3660cb7640 1 -- 172.21.15.105:0/3969031329 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36580ab430 con 0x7f365c15d320 2023-11-28T18:23:46.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.888+0000 7f3660cb7640 1 -- 172.21.15.105:0/3969031329 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36580ab7c0 con 0x7f365c15d320 2023-11-28T18:23:46.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.888+0000 7f3663f1d640 1 -- 172.21.15.105:0/3969031329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f365c15d320 msgr2=0x7f365c06c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:46.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.888+0000 7f3663f1d640 1 --2- 172.21.15.105:0/3969031329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f365c15d320 0x7f365c06c0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f365c06d850 tx=0x7f365801c470 comp rx=0 tx=0).stop 2023-11-28T18:23:46.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.888+0000 7f3663f1d640 1 -- 172.21.15.105:0/3969031329 shutdown_connections 2023-11-28T18:23:46.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.888+0000 7f3663f1d640 1 --2- 172.21.15.105:0/3969031329 >> v2:172.21.15.148:3300/0 conn(0x7f365c0cdbb0 0x7f365c0cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.888+0000 7f3663f1d640 1 --2- 172.21.15.105:0/3969031329 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f365c0ce230 0x7f365c0cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.888+0000 7f3663f1d640 1 --2- 172.21.15.105:0/3969031329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f365c15d320 0x7f365c06c0a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.888+0000 7f3663f1d640 1 -- 172.21.15.105:0/3969031329 >> 172.21.15.105:0/3969031329 conn(0x7f365c0d0df0 msgr2=0x7f365c0d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:46.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.888+0000 7f3663f1d640 1 -- 172.21.15.105:0/3969031329 shutdown_connections 2023-11-28T18:23:46.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.888+0000 7f3663f1d640 1 -- 172.21.15.105:0/3969031329 wait complete. 2023-11-28T18:23:46.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f3663f1d640 1 Processor -- start 2023-11-28T18:23:46.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f3663f1d640 1 -- start start 2023-11-28T18:23:46.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f3663f1d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f365c0cdbb0 0x7f365c14edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f3663f1d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f365c0ce230 0x7f365c15a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f3663f1d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f365c15d320 0x7f365c0c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f3663f1d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365c15bca0 con 0x7f365c15d320 2023-11-28T18:23:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f3663f1d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365c14f310 con 0x7f365c0ce230 2023-11-28T18:23:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f3663f1d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365c15adf0 con 0x7f365c0cdbb0 2023-11-28T18:23:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f36624ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f365c15d320 0x7f365c0c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f36624ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f365c15d320 0x7f365c0c7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38310/0 (socket says 172.21.15.105:38310) 2023-11-28T18:23:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f36624ba640 1 -- 172.21.15.105:0/132157815 learned_addr learned my addr 172.21.15.105:0/132157815 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f36614b8640 1 --2- 172.21.15.105:0/132157815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f365c0ce230 0x7f365c15a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f3661cb9640 1 --2- 172.21.15.105:0/132157815 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f365c0cdbb0 0x7f365c14edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:46.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f36624ba640 1 -- 172.21.15.105:0/132157815 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f365c0cdbb0 msgr2=0x7f365c14edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:46.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f36624ba640 1 --2- 172.21.15.105:0/132157815 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f365c0cdbb0 0x7f365c14edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f36624ba640 1 -- 172.21.15.105:0/132157815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f365c0ce230 msgr2=0x7f365c15a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:46.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f36624ba640 1 --2- 172.21.15.105:0/132157815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f365c0ce230 0x7f365c15a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:46.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f36624ba640 1 -- 172.21.15.105:0/132157815 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36580094d0 con 0x7f365c15d320 2023-11-28T18:23:46.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f36624ba640 1 --2- 172.21.15.105:0/132157815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f365c15d320 0x7f365c0c7ec0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3654007e70 tx=0x7f365400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:46.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f36527fc640 1 -- 172.21.15.105:0/132157815 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3654014620 con 0x7f365c15d320 2023-11-28T18:23:46.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f36527fc640 1 -- 172.21.15.105:0/132157815 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36540147a0 con 0x7f365c15d320 2023-11-28T18:23:46.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f36527fc640 1 -- 172.21.15.105:0/132157815 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3654014b50 con 0x7f365c15d320 2023-11-28T18:23:46.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f3663f1d640 1 -- 172.21.15.105:0/132157815 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f365c15b090 con 0x7f365c15d320 2023-11-28T18:23:46.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f3663f1d640 1 -- 172.21.15.105:0/132157815 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f365c15b650 con 0x7f365c15d320 2023-11-28T18:23:46.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.892+0000 7f3663f1d640 1 -- 172.21.15.105:0/132157815 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f365c15be50 con 0x7f365c15d320 2023-11-28T18:23:46.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.896+0000 7f36527fc640 1 -- 172.21.15.105:0/132157815 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3654014cb0 con 0x7f365c15d320 2023-11-28T18:23:46.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.896+0000 7f36527fc640 1 --2- 172.21.15.105:0/132157815 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f36400668d0 0x7f3640068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:46.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.896+0000 7f36527fc640 1 -- 172.21.15.105:0/132157815 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3654084f30 con 0x7f365c15d320 2023-11-28T18:23:46.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.896+0000 7f3661cb9640 1 --2- 172.21.15.105:0/132157815 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f36400668d0 0x7f3640068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:46.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.896+0000 7f3661cb9640 1 --2- 172.21.15.105:0/132157815 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f36400668d0 0x7f3640068d90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f36580af000 tx=0x7f3658008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:46.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:46.900+0000 7f36527fc640 1 -- 172.21.15.105:0/132157815 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f365c15be50 con 0x7f365c15d320 2023-11-28T18:23:47.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.048+0000 7f3663f1d640 1 -- 172.21.15.105:0/132157815 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f365c15be50 con 0x7f365c15d320 2023-11-28T18:23:47.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.052+0000 7f36527fc640 1 -- 172.21.15.105:0/132157815 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7f365c15be50 con 0x7f365c15d320 2023-11-28T18:23:47.053 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:47.053 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":24,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:23:38.458954+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:23:47.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.052+0000 7f3663f1d640 1 -- 172.21.15.105:0/132157815 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f36400668d0 msgr2=0x7f3640068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:47.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.052+0000 7f3663f1d640 1 --2- 172.21.15.105:0/132157815 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f36400668d0 0x7f3640068d90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f36580af000 tx=0x7f3658008040 comp rx=0 tx=0).stop 2023-11-28T18:23:47.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.052+0000 7f3663f1d640 1 -- 172.21.15.105:0/132157815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f365c15d320 msgr2=0x7f365c0c7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:47.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.052+0000 7f3663f1d640 1 --2- 172.21.15.105:0/132157815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f365c15d320 0x7f365c0c7ec0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3654007e70 tx=0x7f365400e4b0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.052+0000 7f3663f1d640 1 -- 172.21.15.105:0/132157815 shutdown_connections 2023-11-28T18:23:47.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.052+0000 7f3663f1d640 1 --2- 172.21.15.105:0/132157815 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f36400668d0 0x7f3640068d90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.052+0000 7f3663f1d640 1 --2- 172.21.15.105:0/132157815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f365c15d320 0x7f365c0c7ec0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.052+0000 7f3663f1d640 1 --2- 172.21.15.105:0/132157815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f365c0ce230 0x7f365c15a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.052+0000 7f3663f1d640 1 --2- 172.21.15.105:0/132157815 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f365c0cdbb0 0x7f365c14edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.052+0000 7f3663f1d640 1 -- 172.21.15.105:0/132157815 >> 172.21.15.105:0/132157815 conn(0x7f365c0d0df0 msgr2=0x7f365c155410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:47.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.052+0000 7f3663f1d640 1 -- 172.21.15.105:0/132157815 shutdown_connections 2023-11-28T18:23:47.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.052+0000 7f3663f1d640 1 -- 172.21.15.105:0/132157815 wait complete. 2023-11-28T18:23:47.074 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:23:31.485422+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:23:32.274378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T18:23:36.100071+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T18:23:47.074 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:23:47.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.212+0000 7f7b6b5aa640 1 Processor -- start 2023-11-28T18:23:47.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.212+0000 7f7b6b5aa640 1 -- start start 2023-11-28T18:23:47.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.212+0000 7f7b6b5aa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7b640d3c30 0x7f7b640d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:47.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.212+0000 7f7b6b5aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b640b2d20 0x7f7b640b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:47.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.212+0000 7f7b6b5aa640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b640b25b0 con 0x7f7b6415d4f0 2023-11-28T18:23:47.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.212+0000 7f7b6b5aa640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b640b2900 con 0x7f7b640d3c30 2023-11-28T18:23:47.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.212+0000 7f7b6b5aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b640d0a20 con 0x7f7b640b2d20 2023-11-28T18:23:47.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.212+0000 7f7b69b47640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b640b2d20 0x7f7b640b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:47.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.212+0000 7f7b69b47640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b640b2d20 0x7f7b640b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:36990/0 (socket says 172.21.15.105:36990) 2023-11-28T18:23:47.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.212+0000 7f7b69b47640 1 -- 172.21.15.105:0/3411105421 learned_addr learned my addr 172.21.15.105:0/3411105421 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:47.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.212+0000 7f7b68b45640 1 --2- 172.21.15.105:0/3411105421 >> v2:172.21.15.148:3300/0 conn(0x7f7b640d3c30 0x7f7b640d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:47.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.212+0000 7f7b69b47640 1 -- 172.21.15.105:0/3411105421 >> v2:172.21.15.148:3300/0 conn(0x7f7b640d3c30 msgr2=0x7f7b640d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:47.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.212+0000 7f7b69b47640 1 --2- 172.21.15.105:0/3411105421 >> v2:172.21.15.148:3300/0 conn(0x7f7b640d3c30 0x7f7b640d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.212+0000 7f7b69b47640 1 -- 172.21.15.105:0/3411105421 >> v1:172.21.15.148:6789/0 conn(0x7f7b6415d4f0 legacy=0x7f7b640d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:47.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.212+0000 7f7b69b47640 1 -- 172.21.15.105:0/3411105421 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b640cea30 con 0x7f7b640b2d20 2023-11-28T18:23:47.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.216+0000 7f7b69b47640 1 --2- 172.21.15.105:0/3411105421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b640b2d20 0x7f7b640b3190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009e50 tx=0x7f7b5c01c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b034e86d823dfa16 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.216+0000 7f7b5bfff640 1 -- 172.21.15.105:0/3411105421 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c0a5810 con 0x7f7b640b2d20 2023-11-28T18:23:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.216+0000 7f7b5bfff640 1 -- 172.21.15.105:0/3411105421 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5c0a5970 con 0x7f7b640b2d20 2023-11-28T18:23:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.216+0000 7f7b5bfff640 1 -- 172.21.15.105:0/3411105421 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c0a5d00 con 0x7f7b640b2d20 2023-11-28T18:23:47.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.216+0000 7f7b6b5aa640 1 -- 172.21.15.105:0/3411105421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b640b2d20 msgr2=0x7f7b640b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:47.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.216+0000 7f7b6b5aa640 1 --2- 172.21.15.105:0/3411105421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b640b2d20 0x7f7b640b3190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009e50 tx=0x7f7b5c01c950 comp rx=0 tx=0).stop 2023-11-28T18:23:47.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.216+0000 7f7b6b5aa640 1 -- 172.21.15.105:0/3411105421 shutdown_connections 2023-11-28T18:23:47.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.216+0000 7f7b6b5aa640 1 --2- 172.21.15.105:0/3411105421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b640b2d20 0x7f7b640b3190 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.216+0000 7f7b6b5aa640 1 --2- 172.21.15.105:0/3411105421 >> v2:172.21.15.148:3300/0 conn(0x7f7b640d3c30 0x7f7b640d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.216+0000 7f7b6b5aa640 1 -- 172.21.15.105:0/3411105421 >> 172.21.15.105:0/3411105421 conn(0x7f7b640d3030 msgr2=0x7f7b640d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:47.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b6b5aa640 1 -- 172.21.15.105:0/3411105421 shutdown_connections 2023-11-28T18:23:47.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b6b5aa640 1 -- 172.21.15.105:0/3411105421 wait complete. 2023-11-28T18:23:47.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b6b5aa640 1 Processor -- start 2023-11-28T18:23:47.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b6b5aa640 1 -- start start 2023-11-28T18:23:47.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b6b5aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b640b2d20 0x7f7b640c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:47.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b6b5aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b640d3c30 0x7f7b64155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:47.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b6b5aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b6415d4f0 0x7f7b64153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:47.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b6b5aa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b640b7a30 con 0x7f7b640d3c30 2023-11-28T18:23:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b6b5aa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b640b6e60 con 0x7f7b6415d4f0 2023-11-28T18:23:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b6b5aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b640cba60 con 0x7f7b640b2d20 2023-11-28T18:23:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b68b45640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b640d3c30 0x7f7b64155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b68b45640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b640d3c30 0x7f7b64155f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38320/0 (socket says 172.21.15.105:38320) 2023-11-28T18:23:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b68b45640 1 -- 172.21.15.105:0/1355248763 learned_addr learned my addr 172.21.15.105:0/1355248763 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b69346640 1 --2- 172.21.15.105:0/1355248763 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b640b2d20 0x7f7b640c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b69b47640 1 --2- 172.21.15.105:0/1355248763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b6415d4f0 0x7f7b64153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b68b45640 1 -- 172.21.15.105:0/1355248763 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b640b2d20 msgr2=0x7f7b640c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b68b45640 1 --2- 172.21.15.105:0/1355248763 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b640b2d20 0x7f7b640c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b68b45640 1 -- 172.21.15.105:0/1355248763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b6415d4f0 msgr2=0x7f7b64153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:47.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b68b45640 1 --2- 172.21.15.105:0/1355248763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b6415d4f0 0x7f7b64153e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b68b45640 1 -- 172.21.15.105:0/1355248763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5c009b30 con 0x7f7b640d3c30 2023-11-28T18:23:47.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b69b47640 1 --2- 172.21.15.105:0/1355248763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b6415d4f0 0x7f7b64153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:47.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b68b45640 1 --2- 172.21.15.105:0/1355248763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b640d3c30 0x7f7b64155f40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7b5400e870 tx=0x7f7b5400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:47.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b59ffb640 1 -- 172.21.15.105:0/1355248763 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b54009510 con 0x7f7b640d3c30 2023-11-28T18:23:47.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b59ffb640 1 -- 172.21.15.105:0/1355248763 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5400eec0 con 0x7f7b640d3c30 2023-11-28T18:23:47.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b6b5aa640 1 -- 172.21.15.105:0/1355248763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b640c9500 con 0x7f7b640d3c30 2023-11-28T18:23:47.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.220+0000 7f7b59ffb640 1 -- 172.21.15.105:0/1355248763 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5400cb40 con 0x7f7b640d3c30 2023-11-28T18:23:47.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.224+0000 7f7b6b5aa640 1 -- 172.21.15.105:0/1355248763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b640b7370 con 0x7f7b640d3c30 2023-11-28T18:23:47.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.224+0000 7f7b6b5aa640 1 -- 172.21.15.105:0/1355248763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b64067570 con 0x7f7b640d3c30 2023-11-28T18:23:47.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.224+0000 7f7b59ffb640 1 -- 172.21.15.105:0/1355248763 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f7b5400cca0 con 0x7f7b640d3c30 2023-11-28T18:23:47.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.224+0000 7f7b59ffb640 1 --2- 172.21.15.105:0/1355248763 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7b480668d0 0x7f7b48068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:47.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.224+0000 7f7b59ffb640 1 -- 172.21.15.105:0/1355248763 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7b54084700 con 0x7f7b640d3c30 2023-11-28T18:23:47.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.224+0000 7f7b69346640 1 --2- 172.21.15.105:0/1355248763 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7b480668d0 0x7f7b48068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:47.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.224+0000 7f7b69346640 1 --2- 172.21.15.105:0/1355248763 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7b480668d0 0x7f7b48068d90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7b60007fd0 tx=0x7f7b60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.228+0000 7f7b59ffb640 1 -- 172.21.15.105:0/1355248763 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7b54051730 con 0x7f7b640d3c30 2023-11-28T18:23:47.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.380+0000 7f7b6b5aa640 1 -- 172.21.15.105:0/1355248763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b64156480 con 0x7f7b640d3c30 2023-11-28T18:23:47.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.380+0000 7f7b59ffb640 1 -- 172.21.15.105:0/1355248763 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7b54054d50 con 0x7f7b640d3c30 2023-11-28T18:23:47.385 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:23:47.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.384+0000 7f7b6b5aa640 1 -- 172.21.15.105:0/1355248763 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7b480668d0 msgr2=0x7f7b48068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:47.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.384+0000 7f7b6b5aa640 1 --2- 172.21.15.105:0/1355248763 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7b480668d0 0x7f7b48068d90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7b60007fd0 tx=0x7f7b60009040 comp rx=0 tx=0).stop 2023-11-28T18:23:47.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.384+0000 7f7b6b5aa640 1 -- 172.21.15.105:0/1355248763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b640d3c30 msgr2=0x7f7b64155f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:47.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.384+0000 7f7b6b5aa640 1 --2- 172.21.15.105:0/1355248763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b640d3c30 0x7f7b64155f40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7b5400e870 tx=0x7f7b5400ec70 comp rx=0 tx=0).stop 2023-11-28T18:23:47.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.384+0000 7f7b6b5aa640 1 -- 172.21.15.105:0/1355248763 shutdown_connections 2023-11-28T18:23:47.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.384+0000 7f7b6b5aa640 1 --2- 172.21.15.105:0/1355248763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b6415d4f0 0x7f7b64153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.384+0000 7f7b6b5aa640 1 --2- 172.21.15.105:0/1355248763 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7b480668d0 0x7f7b48068d90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.384+0000 7f7b6b5aa640 1 --2- 172.21.15.105:0/1355248763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b640d3c30 0x7f7b64155f40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.384+0000 7f7b6b5aa640 1 --2- 172.21.15.105:0/1355248763 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b640b2d20 0x7f7b640c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.384+0000 7f7b6b5aa640 1 -- 172.21.15.105:0/1355248763 >> 172.21.15.105:0/1355248763 conn(0x7f7b640d3030 msgr2=0x7f7b640d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:47.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.384+0000 7f7b6b5aa640 1 -- 172.21.15.105:0/1355248763 shutdown_connections 2023-11-28T18:23:47.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.384+0000 7f7b6b5aa640 1 -- 172.21.15.105:0/1355248763 wait complete. 2023-11-28T18:23:47.406 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:23:47.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.544+0000 7f9986c32640 1 Processor -- start 2023-11-28T18:23:47.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.544+0000 7f9986c32640 1 -- start start 2023-11-28T18:23:47.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.544+0000 7f9986c32640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f99800caf60 0x7f99800cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:47.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.544+0000 7f9986c32640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f99800ad850 0x7f99800701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:47.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.544+0000 7f9986c32640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f998006bf50 con 0x7f9980159f40 2023-11-28T18:23:47.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.544+0000 7f9986c32640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99800cb3c0 con 0x7f99800caf60 2023-11-28T18:23:47.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.544+0000 7f9986c32640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f998015a320 con 0x7f99800ad850 2023-11-28T18:23:47.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.544+0000 7f99849ce640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9980159f40 0x7f99800ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:60174/0 (socket says 172.21.15.105:60174) 2023-11-28T18:23:47.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.544+0000 7f99849ce640 1 -- 172.21.15.105:0/1781732342 learned_addr learned my addr 172.21.15.105:0/1781732342 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:47.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.544+0000 7f997ffff640 1 --2- 172.21.15.105:0/1781732342 >> v2:172.21.15.105:3300/0 conn(0x7f99800caf60 0x7f99800cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:47.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.544+0000 7f99851cf640 1 --2- 172.21.15.105:0/1781732342 >> v2:172.21.15.148:3300/0 conn(0x7f99800ad850 0x7f99800701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:47.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.544+0000 7f997f7fe640 1 -- 172.21.15.105:0/1781732342 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2815793666 0 0) 0x7f998006bf50 con 0x7f9980159f40 2023-11-28T18:23:47.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.544+0000 7f997f7fe640 1 -- 172.21.15.105:0/1781732342 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9970005740 con 0x7f9980159f40 2023-11-28T18:23:47.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.544+0000 7f997ffff640 1 -- 172.21.15.105:0/1781732342 >> v2:172.21.15.148:3300/0 conn(0x7f99800ad850 msgr2=0x7f99800701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:47.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.544+0000 7f997ffff640 1 --2- 172.21.15.105:0/1781732342 >> v2:172.21.15.148:3300/0 conn(0x7f99800ad850 0x7f99800701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.544+0000 7f997ffff640 1 -- 172.21.15.105:0/1781732342 >> v1:172.21.15.105:6789/0 conn(0x7f9980159f40 legacy=0x7f99800ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:47.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.548+0000 7f997ffff640 1 -- 172.21.15.105:0/1781732342 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9980071380 con 0x7f99800caf60 2023-11-28T18:23:47.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.548+0000 7f997ffff640 1 --2- 172.21.15.105:0/1781732342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99800caf60 0x7f99800cc760 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f99800b0850 tx=0x7f996c01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6e4ae5fc382902 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:47.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.548+0000 7f997f7fe640 1 -- 172.21.15.105:0/1781732342 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c0ac600 con 0x7f99800caf60 2023-11-28T18:23:47.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.548+0000 7f997f7fe640 1 -- 172.21.15.105:0/1781732342 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c0a8050 con 0x7f99800caf60 2023-11-28T18:23:47.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.548+0000 7f997f7fe640 1 -- 172.21.15.105:0/1781732342 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c0ac980 con 0x7f99800caf60 2023-11-28T18:23:47.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.548+0000 7f9986c32640 1 -- 172.21.15.105:0/1781732342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99800caf60 msgr2=0x7f99800cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:47.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.548+0000 7f9986c32640 1 --2- 172.21.15.105:0/1781732342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99800caf60 0x7f99800cc760 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f99800b0850 tx=0x7f996c01c3a0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.548+0000 7f9986c32640 1 -- 172.21.15.105:0/1781732342 shutdown_connections 2023-11-28T18:23:47.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.548+0000 7f9986c32640 1 --2- 172.21.15.105:0/1781732342 >> v2:172.21.15.148:3300/0 conn(0x7f99800ad850 0x7f99800701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.548+0000 7f9986c32640 1 --2- 172.21.15.105:0/1781732342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99800caf60 0x7f99800cc760 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.548+0000 7f9986c32640 1 -- 172.21.15.105:0/1781732342 >> 172.21.15.105:0/1781732342 conn(0x7f99800ceb90 msgr2=0x7f99800cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:47.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.548+0000 7f9986c32640 1 -- 172.21.15.105:0/1781732342 shutdown_connections 2023-11-28T18:23:47.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.548+0000 7f9986c32640 1 -- 172.21.15.105:0/1781732342 wait complete. 2023-11-28T18:23:47.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f9986c32640 1 Processor -- start 2023-11-28T18:23:47.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f9986c32640 1 -- start start 2023-11-28T18:23:47.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f9986c32640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99800ad850 0x7f998014b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:47.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f9986c32640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99800caf60 0x7f998014bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:47.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f9986c32640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9980159f40 0x7f99800b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:47.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f9986c32640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998014c110 con 0x7f99800ad850 2023-11-28T18:23:47.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f9986c32640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99800c9920 con 0x7f99800caf60 2023-11-28T18:23:47.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f9986c32640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998014caf0 con 0x7f9980159f40 2023-11-28T18:23:47.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f99849ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99800ad850 0x7f998014b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:47.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f99849ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99800ad850 0x7f998014b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38338/0 (socket says 172.21.15.105:38338) 2023-11-28T18:23:47.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f99849ce640 1 -- 172.21.15.105:0/2269788380 learned_addr learned my addr 172.21.15.105:0/2269788380 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:47.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f99851cf640 1 --2- 172.21.15.105:0/2269788380 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9980159f40 0x7f99800b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:47.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f997ffff640 1 --2- 172.21.15.105:0/2269788380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99800caf60 0x7f998014bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:47.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f99849ce640 1 -- 172.21.15.105:0/2269788380 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9980159f40 msgr2=0x7f99800b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:47.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f99849ce640 1 --2- 172.21.15.105:0/2269788380 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9980159f40 0x7f99800b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f99849ce640 1 -- 172.21.15.105:0/2269788380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99800caf60 msgr2=0x7f998014bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:47.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f99849ce640 1 --2- 172.21.15.105:0/2269788380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99800caf60 0x7f998014bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f99849ce640 1 -- 172.21.15.105:0/2269788380 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c009420 con 0x7f99800ad850 2023-11-28T18:23:47.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f99851cf640 1 --2- 172.21.15.105:0/2269788380 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9980159f40 0x7f99800b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:47.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f99849ce640 1 --2- 172.21.15.105:0/2269788380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99800ad850 0x7f998014b690 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f997400c420 tx=0x7f997400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:47.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f997d7fa640 1 -- 172.21.15.105:0/2269788380 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974005cf0 con 0x7f99800ad850 2023-11-28T18:23:47.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f997d7fa640 1 -- 172.21.15.105:0/2269788380 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9974005e50 con 0x7f99800ad850 2023-11-28T18:23:47.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f997d7fa640 1 -- 172.21.15.105:0/2269788380 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99740072d0 con 0x7f99800ad850 2023-11-28T18:23:47.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.552+0000 7f9986c32640 1 -- 172.21.15.105:0/2269788380 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99800b3bf0 con 0x7f99800ad850 2023-11-28T18:23:47.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.556+0000 7f9986c32640 1 -- 172.21.15.105:0/2269788380 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99800c92d0 con 0x7f99800ad850 2023-11-28T18:23:47.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.556+0000 7f9986c32640 1 -- 172.21.15.105:0/2269788380 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9980067570 con 0x7f99800ad850 2023-11-28T18:23:47.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.556+0000 7f997d7fa640 1 -- 172.21.15.105:0/2269788380 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f9974007430 con 0x7f99800ad850 2023-11-28T18:23:47.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.556+0000 7f997d7fa640 1 --2- 172.21.15.105:0/2269788380 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9964066900 0x7f9964068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:47.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.556+0000 7f997d7fa640 1 -- 172.21.15.105:0/2269788380 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9974083d00 con 0x7f99800ad850 2023-11-28T18:23:47.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.556+0000 7f997ffff640 1 --2- 172.21.15.105:0/2269788380 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9964066900 0x7f9964068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:47.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.556+0000 7f997ffff640 1 --2- 172.21.15.105:0/2269788380 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9964066900 0x7f9964068dc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f996c0ae000 tx=0x7f996c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:47.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.560+0000 7f997d7fa640 1 -- 172.21.15.105:0/2269788380 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9974050bd0 con 0x7f99800ad850 2023-11-28T18:23:47.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.712+0000 7f9986c32640 1 -- 172.21.15.105:0/2269788380 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f99800cf950 con 0x7f99800ad850 2023-11-28T18:23:47.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.712+0000 7f997d7fa640 1 -- 172.21.15.105:0/2269788380 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9974017020 con 0x7f99800ad850 2023-11-28T18:23:47.715 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:23:47.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.712+0000 7f9986c32640 1 -- 172.21.15.105:0/2269788380 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9964066900 msgr2=0x7f9964068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:47.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.712+0000 7f9986c32640 1 --2- 172.21.15.105:0/2269788380 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9964066900 0x7f9964068dc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f996c0ae000 tx=0x7f996c008040 comp rx=0 tx=0).stop 2023-11-28T18:23:47.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.712+0000 7f9986c32640 1 -- 172.21.15.105:0/2269788380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99800ad850 msgr2=0x7f998014b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:47.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.712+0000 7f9986c32640 1 --2- 172.21.15.105:0/2269788380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99800ad850 0x7f998014b690 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f997400c420 tx=0x7f997400c8e0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.712+0000 7f9986c32640 1 -- 172.21.15.105:0/2269788380 shutdown_connections 2023-11-28T18:23:47.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.712+0000 7f9986c32640 1 --2- 172.21.15.105:0/2269788380 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9980159f40 0x7f99800b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.712+0000 7f9986c32640 1 --2- 172.21.15.105:0/2269788380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99800caf60 0x7f998014bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.712+0000 7f9986c32640 1 --2- 172.21.15.105:0/2269788380 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9964066900 0x7f9964068dc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.712+0000 7f9986c32640 1 --2- 172.21.15.105:0/2269788380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99800ad850 0x7f998014b690 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.712+0000 7f9986c32640 1 -- 172.21.15.105:0/2269788380 >> 172.21.15.105:0/2269788380 conn(0x7f99800ceb90 msgr2=0x7f99800cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:47.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.712+0000 7f9986c32640 1 -- 172.21.15.105:0/2269788380 shutdown_connections 2023-11-28T18:23:47.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.712+0000 7f9986c32640 1 -- 172.21.15.105:0/2269788380 wait complete. 2023-11-28T18:23:47.736 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:23:47.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.872+0000 7f447ebb8640 1 Processor -- start 2023-11-28T18:23:47.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.872+0000 7f447ebb8640 1 -- start start 2023-11-28T18:23:47.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.876+0000 7f447ebb8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4478154070 0x7f447815d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:47.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.876+0000 7f447ebb8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f447806bf50 0x7f44780ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:47.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.876+0000 7f447ebb8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44780cc7f0 con 0x7f44780adb80 2023-11-28T18:23:47.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.876+0000 7f447ebb8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4478072540 con 0x7f4478154070 2023-11-28T18:23:47.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.876+0000 7f447ebb8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f447815dd10 con 0x7f447806bf50 2023-11-28T18:23:47.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.876+0000 7f447c954640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4478154070 0x7f447815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:47.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.876+0000 7f447c954640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4478154070 0x7f447815d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38352/0 (socket says 172.21.15.105:38352) 2023-11-28T18:23:47.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.876+0000 7f447c954640 1 -- 172.21.15.105:0/788809462 learned_addr learned my addr 172.21.15.105:0/788809462 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:47.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.876+0000 7f4477fff640 1 --2- 172.21.15.105:0/788809462 >> v2:172.21.15.148:3300/0 conn(0x7f447806bf50 0x7f44780ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:47.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.876+0000 7f447c954640 1 -- 172.21.15.105:0/788809462 >> v2:172.21.15.148:3300/0 conn(0x7f447806bf50 msgr2=0x7f44780ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:47.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.876+0000 7f447c954640 1 --2- 172.21.15.105:0/788809462 >> v2:172.21.15.148:3300/0 conn(0x7f447806bf50 0x7f44780ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.876+0000 7f447c954640 1 -- 172.21.15.105:0/788809462 >> v1:172.21.15.148:6789/0 conn(0x7f44780adb80 legacy=0x7f44780acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:47.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.876+0000 7f447c954640 1 -- 172.21.15.105:0/788809462 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447815dff0 con 0x7f4478154070 2023-11-28T18:23:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.880+0000 7f447c954640 1 --2- 172.21.15.105:0/788809462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4478154070 0x7f447815d700 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f447815dc40 tx=0x7f446c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73a09bf29c79f96 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:47.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.880+0000 7f44777fe640 1 -- 172.21.15.105:0/788809462 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c0ac630 con 0x7f4478154070 2023-11-28T18:23:47.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.880+0000 7f44777fe640 1 -- 172.21.15.105:0/788809462 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c0ac790 con 0x7f4478154070 2023-11-28T18:23:47.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.880+0000 7f44777fe640 1 -- 172.21.15.105:0/788809462 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c0acb20 con 0x7f4478154070 2023-11-28T18:23:47.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.880+0000 7f447ebb8640 1 -- 172.21.15.105:0/788809462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4478154070 msgr2=0x7f447815d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:47.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.880+0000 7f447ebb8640 1 --2- 172.21.15.105:0/788809462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4478154070 0x7f447815d700 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f447815dc40 tx=0x7f446c01c3d0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.880+0000 7f447ebb8640 1 -- 172.21.15.105:0/788809462 shutdown_connections 2023-11-28T18:23:47.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.880+0000 7f447ebb8640 1 --2- 172.21.15.105:0/788809462 >> v2:172.21.15.148:3300/0 conn(0x7f447806bf50 0x7f44780ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.880+0000 7f447ebb8640 1 --2- 172.21.15.105:0/788809462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4478154070 0x7f447815d700 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.880+0000 7f447ebb8640 1 -- 172.21.15.105:0/788809462 >> 172.21.15.105:0/788809462 conn(0x7f44780ceb90 msgr2=0x7f44780cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:47.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.880+0000 7f447ebb8640 1 -- 172.21.15.105:0/788809462 shutdown_connections 2023-11-28T18:23:47.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.880+0000 7f447ebb8640 1 -- 172.21.15.105:0/788809462 wait complete. 2023-11-28T18:23:47.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.880+0000 7f447ebb8640 1 Processor -- start 2023-11-28T18:23:47.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.880+0000 7f447ebb8640 1 -- start start 2023-11-28T18:23:47.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f447ebb8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f447806bf50 0x7f447813d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:47.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f447ebb8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44780adb80 0x7f4478152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:47.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f447ebb8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4478154070 0x7f4478152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:47.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f447ebb8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447809c4b0 con 0x7f44780adb80 2023-11-28T18:23:47.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f447ebb8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44780c8050 con 0x7f447806bf50 2023-11-28T18:23:47.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f447ebb8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44780c8330 con 0x7f4478154070 2023-11-28T18:23:47.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f4477fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44780adb80 0x7f4478152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:47.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f4477fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44780adb80 0x7f4478152ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38356/0 (socket says 172.21.15.105:38356) 2023-11-28T18:23:47.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f4477fff640 1 -- 172.21.15.105:0/4095195938 learned_addr learned my addr 172.21.15.105:0/4095195938 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:47.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f447c954640 1 --2- 172.21.15.105:0/4095195938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f447806bf50 0x7f447813d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:47.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f447d155640 1 --2- 172.21.15.105:0/4095195938 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4478154070 0x7f4478152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:47.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f4477fff640 1 -- 172.21.15.105:0/4095195938 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4478154070 msgr2=0x7f4478152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:47.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f4477fff640 1 --2- 172.21.15.105:0/4095195938 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4478154070 0x7f4478152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f4477fff640 1 -- 172.21.15.105:0/4095195938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f447806bf50 msgr2=0x7f447813d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:47.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f4477fff640 1 --2- 172.21.15.105:0/4095195938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f447806bf50 0x7f447813d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:47.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f4477fff640 1 -- 172.21.15.105:0/4095195938 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c0094d0 con 0x7f44780adb80 2023-11-28T18:23:47.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f4477fff640 1 --2- 172.21.15.105:0/4095195938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44780adb80 0x7f4478152ab0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f446400d800 tx=0x7f446400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:47.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f44757fa640 1 -- 172.21.15.105:0/4095195938 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446400b6c0 con 0x7f44780adb80 2023-11-28T18:23:47.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f44757fa640 1 -- 172.21.15.105:0/4095195938 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446400b820 con 0x7f44780adb80 2023-11-28T18:23:47.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f447ebb8640 1 -- 172.21.15.105:0/4095195938 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44780c8610 con 0x7f44780adb80 2023-11-28T18:23:47.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f44757fa640 1 -- 172.21.15.105:0/4095195938 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446400bb90 con 0x7f44780adb80 2023-11-28T18:23:47.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f447ebb8640 1 -- 172.21.15.105:0/4095195938 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447814ea90 con 0x7f44780adb80 2023-11-28T18:23:47.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f447ebb8640 1 -- 172.21.15.105:0/4095195938 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4478067570 con 0x7f44780adb80 2023-11-28T18:23:47.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.884+0000 7f44757fa640 1 -- 172.21.15.105:0/4095195938 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f446400bcf0 con 0x7f44780adb80 2023-11-28T18:23:47.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.888+0000 7f44757fa640 1 --2- 172.21.15.105:0/4095195938 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f445c0668d0 0x7f445c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:47.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.888+0000 7f44757fa640 1 -- 172.21.15.105:0/4095195938 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4464084aa0 con 0x7f44780adb80 2023-11-28T18:23:47.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.888+0000 7f447c954640 1 --2- 172.21.15.105:0/4095195938 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f445c0668d0 0x7f445c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:47.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.888+0000 7f447c954640 1 --2- 172.21.15.105:0/4095195938 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f445c0668d0 0x7f445c068d90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f446c0ae000 tx=0x7f446c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:47.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:47.892+0000 7f44757fa640 1 -- 172.21.15.105:0/4095195938 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4464051b30 con 0x7f44780adb80 2023-11-28T18:23:48.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.044+0000 7f447ebb8640 1 -- 172.21.15.105:0/4095195938 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f44780cf950 con 0x7f44780adb80 2023-11-28T18:23:48.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.044+0000 7f44757fa640 1 -- 172.21.15.105:0/4095195938 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4464055150 con 0x7f44780adb80 2023-11-28T18:23:48.046 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:23:48.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.044+0000 7f447ebb8640 1 -- 172.21.15.105:0/4095195938 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f445c0668d0 msgr2=0x7f445c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:48.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.044+0000 7f447ebb8640 1 --2- 172.21.15.105:0/4095195938 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f445c0668d0 0x7f445c068d90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f446c0ae000 tx=0x7f446c008040 comp rx=0 tx=0).stop 2023-11-28T18:23:48.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.044+0000 7f447ebb8640 1 -- 172.21.15.105:0/4095195938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44780adb80 msgr2=0x7f4478152ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:48.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.044+0000 7f447ebb8640 1 --2- 172.21.15.105:0/4095195938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44780adb80 0x7f4478152ab0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f446400d800 tx=0x7f446400dcc0 comp rx=0 tx=0).stop 2023-11-28T18:23:48.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.044+0000 7f447ebb8640 1 -- 172.21.15.105:0/4095195938 shutdown_connections 2023-11-28T18:23:48.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.044+0000 7f447ebb8640 1 --2- 172.21.15.105:0/4095195938 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4478154070 0x7f4478152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:48.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.044+0000 7f447ebb8640 1 --2- 172.21.15.105:0/4095195938 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f445c0668d0 0x7f445c068d90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:48.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.044+0000 7f447ebb8640 1 --2- 172.21.15.105:0/4095195938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44780adb80 0x7f4478152ab0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:48.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.044+0000 7f447ebb8640 1 --2- 172.21.15.105:0/4095195938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f447806bf50 0x7f447813d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:48.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.044+0000 7f447ebb8640 1 -- 172.21.15.105:0/4095195938 >> 172.21.15.105:0/4095195938 conn(0x7f44780ceb90 msgr2=0x7f447806ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:48.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.044+0000 7f447ebb8640 1 -- 172.21.15.105:0/4095195938 shutdown_connections 2023-11-28T18:23:48.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.044+0000 7f447ebb8640 1 -- 172.21.15.105:0/4095195938 wait complete. 2023-11-28T18:23:48.069 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T18:23:48.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.208+0000 7f2505f8d640 1 Processor -- start 2023-11-28T18:23:48.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.208+0000 7f2505f8d640 1 -- start start 2023-11-28T18:23:48.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.208+0000 7f2505f8d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2500159f40 0x7f25000caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:48.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.208+0000 7f2505f8d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f25000ad850 0x7f25000701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:48.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.208+0000 7f2505f8d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f250006bf50 con 0x7f25000cc760 2023-11-28T18:23:48.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.208+0000 7f2505f8d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25000ccbc0 con 0x7f25000ad850 2023-11-28T18:23:48.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.208+0000 7f2505f8d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f250015a320 con 0x7f2500159f40 2023-11-28T18:23:48.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.208+0000 7f24fffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f25000ad850 0x7f25000701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:48.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.208+0000 7f24fffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f25000ad850 0x7f25000701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38372/0 (socket says 172.21.15.105:38372) 2023-11-28T18:23:48.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.208+0000 7f24fffff640 1 -- 172.21.15.105:0/3230088921 learned_addr learned my addr 172.21.15.105:0/3230088921 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:48.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.208+0000 7f24ff7fe640 1 --2- 172.21.15.105:0/3230088921 >> v2:172.21.15.148:3300/0 conn(0x7f2500159f40 0x7f25000caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:48.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.208+0000 7f24fffff640 1 -- 172.21.15.105:0/3230088921 >> v2:172.21.15.148:3300/0 conn(0x7f2500159f40 msgr2=0x7f25000caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:48.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.208+0000 7f24fffff640 1 --2- 172.21.15.105:0/3230088921 >> v2:172.21.15.148:3300/0 conn(0x7f2500159f40 0x7f25000caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:48.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.208+0000 7f24fffff640 1 -- 172.21.15.105:0/3230088921 >> v1:172.21.15.148:6789/0 conn(0x7f25000cc760 legacy=0x7f25000ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:48.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.208+0000 7f24fffff640 1 -- 172.21.15.105:0/3230088921 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2500071380 con 0x7f25000ad850 2023-11-28T18:23:48.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.212+0000 7f24fffff640 1 --2- 172.21.15.105:0/3230088921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25000ad850 0x7f25000701d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f250006e500 tx=0x7f24f801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f189f9916d38d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:48.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.212+0000 7f24fe7fc640 1 -- 172.21.15.105:0/3230088921 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f80a57d0 con 0x7f25000ad850 2023-11-28T18:23:48.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.212+0000 7f24fe7fc640 1 -- 172.21.15.105:0/3230088921 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f80a5930 con 0x7f25000ad850 2023-11-28T18:23:48.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.212+0000 7f24fe7fc640 1 -- 172.21.15.105:0/3230088921 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f80a5cc0 con 0x7f25000ad850 2023-11-28T18:23:48.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.212+0000 7f2505f8d640 1 -- 172.21.15.105:0/3230088921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25000ad850 msgr2=0x7f25000701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:48.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.212+0000 7f2505f8d640 1 --2- 172.21.15.105:0/3230088921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25000ad850 0x7f25000701d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f250006e500 tx=0x7f24f801c910 comp rx=0 tx=0).stop 2023-11-28T18:23:48.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.212+0000 7f2505f8d640 1 -- 172.21.15.105:0/3230088921 shutdown_connections 2023-11-28T18:23:48.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.212+0000 7f2505f8d640 1 --2- 172.21.15.105:0/3230088921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25000ad850 0x7f25000701d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:48.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.212+0000 7f2505f8d640 1 --2- 172.21.15.105:0/3230088921 >> v2:172.21.15.148:3300/0 conn(0x7f2500159f40 0x7f25000caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:48.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.212+0000 7f2505f8d640 1 -- 172.21.15.105:0/3230088921 >> 172.21.15.105:0/3230088921 conn(0x7f25000ceb90 msgr2=0x7f25000cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:48.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.212+0000 7f2505f8d640 1 -- 172.21.15.105:0/3230088921 shutdown_connections 2023-11-28T18:23:48.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.212+0000 7f2505f8d640 1 -- 172.21.15.105:0/3230088921 wait complete. 2023-11-28T18:23:48.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.212+0000 7f2505f8d640 1 Processor -- start 2023-11-28T18:23:48.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f2505f8d640 1 -- start start 2023-11-28T18:23:48.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f2505f8d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25000ad850 0x7f25001fa040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:48.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f2505f8d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25000cc760 0x7f25001525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:48.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f2505f8d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2500159f40 0x7f2500152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:48.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f2505f8d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2500158fa0 con 0x7f25000cc760 2023-11-28T18:23:48.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f2505f8d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2500158390 con 0x7f25000ad850 2023-11-28T18:23:48.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f2505f8d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2500158670 con 0x7f2500159f40 2023-11-28T18:23:48.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f24feffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25000cc760 0x7f25001525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:48.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f24feffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25000cc760 0x7f25001525d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38386/0 (socket says 172.21.15.105:38386) 2023-11-28T18:23:48.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f24ff7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25000ad850 0x7f25001fa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:48.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f24feffd640 1 -- 172.21.15.105:0/3348257054 learned_addr learned my addr 172.21.15.105:0/3348257054 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:48.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f24fffff640 1 --2- 172.21.15.105:0/3348257054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2500159f40 0x7f2500152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:48.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f24feffd640 1 -- 172.21.15.105:0/3348257054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2500159f40 msgr2=0x7f2500152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:48.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f24feffd640 1 --2- 172.21.15.105:0/3348257054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2500159f40 0x7f2500152b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:48.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f24feffd640 1 -- 172.21.15.105:0/3348257054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25000ad850 msgr2=0x7f25001fa040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:48.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f24feffd640 1 --2- 172.21.15.105:0/3348257054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25000ad850 0x7f25001fa040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:48.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f24feffd640 1 -- 172.21.15.105:0/3348257054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f8009b30 con 0x7f25000cc760 2023-11-28T18:23:48.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f24feffd640 1 --2- 172.21.15.105:0/3348257054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25000cc760 0x7f25001525d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f24f400c3c0 tx=0x7f24f400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:48.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f24e7fff640 1 -- 172.21.15.105:0/3348257054 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f40052d0 con 0x7f25000cc760 2023-11-28T18:23:48.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f24e7fff640 1 -- 172.21.15.105:0/3348257054 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f4005430 con 0x7f25000cc760 2023-11-28T18:23:48.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f24e7fff640 1 -- 172.21.15.105:0/3348257054 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f4003840 con 0x7f25000cc760 2023-11-28T18:23:48.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f2505f8d640 1 -- 172.21.15.105:0/3348257054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2500158950 con 0x7f25000cc760 2023-11-28T18:23:48.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f2505f8d640 1 -- 172.21.15.105:0/3348257054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2500159430 con 0x7f25000cc760 2023-11-28T18:23:48.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.216+0000 7f2505f8d640 1 -- 172.21.15.105:0/3348257054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2500159110 con 0x7f25000cc760 2023-11-28T18:23:48.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.220+0000 7f24e7fff640 1 -- 172.21.15.105:0/3348257054 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f24f40039a0 con 0x7f25000cc760 2023-11-28T18:23:48.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.220+0000 7f24e7fff640 1 --2- 172.21.15.105:0/3348257054 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f24ec0668d0 0x7f24ec068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:48.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.220+0000 7f24e7fff640 1 -- 172.21.15.105:0/3348257054 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f24f4083b40 con 0x7f25000cc760 2023-11-28T18:23:48.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.220+0000 7f24ff7fe640 1 --2- 172.21.15.105:0/3348257054 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f24ec0668d0 0x7f24ec068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:48.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.220+0000 7f24ff7fe640 1 --2- 172.21.15.105:0/3348257054 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f24ec0668d0 0x7f24ec068d90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f24f00079c0 tx=0x7f24f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:48.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.224+0000 7f24e7fff640 1 -- 172.21.15.105:0/3348257054 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f24f4050bd0 con 0x7f25000cc760 2023-11-28T18:23:48.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.372+0000 7f2505f8d640 1 -- 172.21.15.105:0/3348257054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2500067570 con 0x7f25000cc760 2023-11-28T18:23:48.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.372+0000 7f24e7fff640 1 -- 172.21.15.105:0/3348257054 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f24f40170a0 con 0x7f25000cc760 2023-11-28T18:23:48.376 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:23:48.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.376+0000 7f2505f8d640 1 -- 172.21.15.105:0/3348257054 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f24ec0668d0 msgr2=0x7f24ec068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:48.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.376+0000 7f2505f8d640 1 --2- 172.21.15.105:0/3348257054 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f24ec0668d0 0x7f24ec068d90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f24f00079c0 tx=0x7f24f0008040 comp rx=0 tx=0).stop 2023-11-28T18:23:48.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.376+0000 7f2505f8d640 1 -- 172.21.15.105:0/3348257054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25000cc760 msgr2=0x7f25001525d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:48.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.376+0000 7f2505f8d640 1 --2- 172.21.15.105:0/3348257054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25000cc760 0x7f25001525d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f24f400c3c0 tx=0x7f24f400c880 comp rx=0 tx=0).stop 2023-11-28T18:23:48.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.376+0000 7f2505f8d640 1 -- 172.21.15.105:0/3348257054 shutdown_connections 2023-11-28T18:23:48.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.376+0000 7f2505f8d640 1 --2- 172.21.15.105:0/3348257054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2500159f40 0x7f2500152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:48.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.376+0000 7f2505f8d640 1 --2- 172.21.15.105:0/3348257054 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f24ec0668d0 0x7f24ec068d90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:48.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.376+0000 7f2505f8d640 1 --2- 172.21.15.105:0/3348257054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25000cc760 0x7f25001525d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:48.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.376+0000 7f2505f8d640 1 --2- 172.21.15.105:0/3348257054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25000ad850 0x7f25001fa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:48.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.376+0000 7f2505f8d640 1 -- 172.21.15.105:0/3348257054 >> 172.21.15.105:0/3348257054 conn(0x7f25000ceb90 msgr2=0x7f2500073080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:48.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.376+0000 7f2505f8d640 1 -- 172.21.15.105:0/3348257054 shutdown_connections 2023-11-28T18:23:48.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.376+0000 7f2505f8d640 1 -- 172.21.15.105:0/3348257054 wait complete. 2023-11-28T18:23:48.398 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T18:23:48.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.532+0000 7fc6a1de6640 1 Processor -- start 2023-11-28T18:23:48.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.532+0000 7fc6a1de6640 1 -- start start 2023-11-28T18:23:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.536+0000 7fc6a1de6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc69c0b4290 0x7fc69c0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.536+0000 7fc6a1de6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc69c0b4bb0 con 0x7fc69c15b030 2023-11-28T18:23:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.536+0000 7fc6a1de6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc69c0701d0 con 0x7fc69c071960 2023-11-28T18:23:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.536+0000 7fc6a1de6640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc69c06df80 con 0x7fc69c0b4290 2023-11-28T18:23:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.536+0000 7fc69b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc69c0b4290 0x7fc69c0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.536+0000 7fc69b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc69c0b4290 0x7fc69c0b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38388/0 (socket says 172.21.15.105:38388) 2023-11-28T18:23:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.536+0000 7fc69b7fe640 1 -- 172.21.15.105:0/505566316 learned_addr learned my addr 172.21.15.105:0/505566316 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.536+0000 7fc69bfff640 1 --1- 172.21.15.105:0/505566316 >> v1:172.21.15.105:6789/0 conn(0x7fc69c15b030 0x7fc69c0709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:60184/0 (socket says 172.21.15.105:60184) 2023-11-28T18:23:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.536+0000 7fc69b7fe640 1 -- 172.21.15.105:0/505566316 >> v1:172.21.15.148:6789/0 conn(0x7fc69c071960 legacy=0x7fc69c15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.536+0000 7fc69b7fe640 1 -- 172.21.15.105:0/505566316 >> v1:172.21.15.105:6789/0 conn(0x7fc69c15b030 legacy=0x7fc69c0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.536+0000 7fc69b7fe640 1 -- 172.21.15.105:0/505566316 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69c071df0 con 0x7fc69c0b4290 2023-11-28T18:23:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.540+0000 7fc69b7fe640 1 --2- 172.21.15.105:0/505566316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc69c0b4290 0x7fc69c0b4670 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009870 tx=0x7fc68c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6bf86ab354926a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.540+0000 7fc69a7fc640 1 -- 172.21.15.105:0/505566316 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0aa430 con 0x7fc69c0b4290 2023-11-28T18:23:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.540+0000 7fc69a7fc640 1 -- 172.21.15.105:0/505566316 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c0aa590 con 0x7fc69c0b4290 2023-11-28T18:23:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.540+0000 7fc69a7fc640 1 -- 172.21.15.105:0/505566316 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0aa920 con 0x7fc69c0b4290 2023-11-28T18:23:48.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.540+0000 7fc6a1de6640 1 -- 172.21.15.105:0/505566316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc69c0b4290 msgr2=0x7fc69c0b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:48.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.540+0000 7fc6a1de6640 1 --2- 172.21.15.105:0/505566316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc69c0b4290 0x7fc69c0b4670 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009870 tx=0x7fc68c01c4d0 comp rx=0 tx=0).stop 2023-11-28T18:23:48.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.540+0000 7fc6a1de6640 1 -- 172.21.15.105:0/505566316 shutdown_connections 2023-11-28T18:23:48.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.540+0000 7fc6a1de6640 1 --2- 172.21.15.105:0/505566316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc69c0b4290 0x7fc69c0b4670 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:48.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.540+0000 7fc6a1de6640 1 -- 172.21.15.105:0/505566316 >> 172.21.15.105:0/505566316 conn(0x7fc69c0ceb90 msgr2=0x7fc69c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:48.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.540+0000 7fc6a1de6640 1 -- 172.21.15.105:0/505566316 shutdown_connections 2023-11-28T18:23:48.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.540+0000 7fc6a1de6640 1 -- 172.21.15.105:0/505566316 wait complete. 2023-11-28T18:23:48.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.540+0000 7fc6a1de6640 1 Processor -- start 2023-11-28T18:23:48.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.540+0000 7fc6a1de6640 1 -- start start 2023-11-28T18:23:48.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc6a1de6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc69c0b4290 0x7fc69c1486a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:48.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc6a1de6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc69c15b030 0x7fc69c164fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:48.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc6a1de6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc69c1654e0 0x7fc69c15c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:48.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc6a1de6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c165970 con 0x7fc69c15b030 2023-11-28T18:23:48.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc6a1de6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c1491b0 con 0x7fc69c0b4290 2023-11-28T18:23:48.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc6a1de6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c14b400 con 0x7fc69c1654e0 2023-11-28T18:23:48.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc69affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc69c15b030 0x7fc69c164fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:48.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc69affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc69c15b030 0x7fc69c164fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38404/0 (socket says 172.21.15.105:38404) 2023-11-28T18:23:48.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc69affd640 1 -- 172.21.15.105:0/626395388 learned_addr learned my addr 172.21.15.105:0/626395388 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:48.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc69b7fe640 1 --2- 172.21.15.105:0/626395388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc69c0b4290 0x7fc69c1486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:48.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc69bfff640 1 --2- 172.21.15.105:0/626395388 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc69c1654e0 0x7fc69c15c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:48.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc69affd640 1 -- 172.21.15.105:0/626395388 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc69c1654e0 msgr2=0x7fc69c15c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:48.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc69affd640 1 --2- 172.21.15.105:0/626395388 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc69c1654e0 0x7fc69c15c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:48.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc69affd640 1 -- 172.21.15.105:0/626395388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc69c0b4290 msgr2=0x7fc69c1486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:48.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc69affd640 1 --2- 172.21.15.105:0/626395388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc69c0b4290 0x7fc69c1486a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:48.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc69affd640 1 -- 172.21.15.105:0/626395388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68c0094d0 con 0x7fc69c15b030 2023-11-28T18:23:48.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc69bfff640 1 --2- 172.21.15.105:0/626395388 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc69c1654e0 0x7fc69c15c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:48.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc69affd640 1 --2- 172.21.15.105:0/626395388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc69c15b030 0x7fc69c164fa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc69000c450 tx=0x7fc69000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc683fff640 1 -- 172.21.15.105:0/626395388 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc690012830 con 0x7fc69c15b030 2023-11-28T18:23:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc683fff640 1 -- 172.21.15.105:0/626395388 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6900129b0 con 0x7fc69c15b030 2023-11-28T18:23:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc6a1de6640 1 -- 172.21.15.105:0/626395388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc69c14b6e0 con 0x7fc69c15b030 2023-11-28T18:23:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc683fff640 1 -- 172.21.15.105:0/626395388 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc690012d60 con 0x7fc69c15b030 2023-11-28T18:23:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc6a1de6640 1 -- 172.21.15.105:0/626395388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69c14bca0 con 0x7fc69c15b030 2023-11-28T18:23:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc6a1de6640 1 -- 172.21.15.105:0/626395388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc69c067570 con 0x7fc69c15b030 2023-11-28T18:23:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.544+0000 7fc683fff640 1 -- 172.21.15.105:0/626395388 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc69001c460 con 0x7fc69c15b030 2023-11-28T18:23:48.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.548+0000 7fc683fff640 1 --2- 172.21.15.105:0/626395388 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6880668d0 0x7fc688068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:48.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.548+0000 7fc683fff640 1 -- 172.21.15.105:0/626395388 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc6900844f0 con 0x7fc69c15b030 2023-11-28T18:23:48.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.548+0000 7fc69b7fe640 1 --2- 172.21.15.105:0/626395388 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6880668d0 0x7fc688068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:48.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.548+0000 7fc69b7fe640 1 --2- 172.21.15.105:0/626395388 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6880668d0 0x7fc688068d90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0ae000 tx=0x7fc68c0b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.552+0000 7fc683fff640 1 -- 172.21.15.105:0/626395388 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc6900513b0 con 0x7fc69c15b030 2023-11-28T18:23:48.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.712+0000 7fc6a1de6640 1 -- 172.21.15.105:0/626395388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc69c0cf7f0 con 0x7fc69c15b030 2023-11-28T18:23:48.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.712+0000 7fc683fff640 1 -- 172.21.15.105:0/626395388 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc6900549d0 con 0x7fc69c15b030 2023-11-28T18:23:48.716 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2023-11-28T18:23:48.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.716+0000 7fc6a1de6640 1 -- 172.21.15.105:0/626395388 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6880668d0 msgr2=0x7fc688068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:48.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.716+0000 7fc6a1de6640 1 --2- 172.21.15.105:0/626395388 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6880668d0 0x7fc688068d90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0ae000 tx=0x7fc68c0b5040 comp rx=0 tx=0).stop 2023-11-28T18:23:48.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.716+0000 7fc6a1de6640 1 -- 172.21.15.105:0/626395388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc69c15b030 msgr2=0x7fc69c164fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:48.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.716+0000 7fc6a1de6640 1 --2- 172.21.15.105:0/626395388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc69c15b030 0x7fc69c164fa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc69000c450 tx=0x7fc69000c910 comp rx=0 tx=0).stop 2023-11-28T18:23:48.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.716+0000 7fc6a1de6640 1 -- 172.21.15.105:0/626395388 shutdown_connections 2023-11-28T18:23:48.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.716+0000 7fc6a1de6640 1 --2- 172.21.15.105:0/626395388 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc69c1654e0 0x7fc69c15c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:48.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.716+0000 7fc6a1de6640 1 --2- 172.21.15.105:0/626395388 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6880668d0 0x7fc688068d90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:48.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.716+0000 7fc6a1de6640 1 --2- 172.21.15.105:0/626395388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc69c15b030 0x7fc69c164fa0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:48.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.716+0000 7fc6a1de6640 1 --2- 172.21.15.105:0/626395388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc69c0b4290 0x7fc69c1486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:48.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.716+0000 7fc6a1de6640 1 -- 172.21.15.105:0/626395388 >> 172.21.15.105:0/626395388 conn(0x7fc69c0ceb90 msgr2=0x7fc69c06c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:48.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.716+0000 7fc6a1de6640 1 -- 172.21.15.105:0/626395388 shutdown_connections 2023-11-28T18:23:48.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:48.716+0000 7fc6a1de6640 1 -- 172.21.15.105:0/626395388 wait complete. 2023-11-28T18:23:48.736 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2023-11-28T18:23:48.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:23:48.737 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-11-28T18:23:48.765 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi105.stderr:ceph-fuse[44820]: fuse finished with error 0 and tester_r 0 2023-11-28T18:23:48.768 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:23:48.866 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:23:48.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:23:48.867 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-11-28T18:23:48.884 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:23:49.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.064+0000 7fa4fb8d6640 1 Processor -- start 2023-11-28T18:23:49.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.064+0000 7fa4fb8d6640 1 -- start start 2023-11-28T18:23:49.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.064+0000 7fa4fb8d6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa4f415d4f0 0x7fa4f40d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:49.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.064+0000 7fa4fb8d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4f40d3c30 0x7fa4f40d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.064+0000 7fa4fb8d6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa4f40d1890 0x7fa4f40cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.064+0000 7fa4fb8d6640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f40b2900 con 0x7fa4f415d4f0 2023-11-28T18:23:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.064+0000 7fa4fb8d6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f40b25b0 con 0x7fa4f40d1890 2023-11-28T18:23:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.064+0000 7fa4fb8d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f40cea30 con 0x7fa4f40d3c30 2023-11-28T18:23:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.064+0000 7fa4f9672640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa4f415d4f0 0x7fa4f40d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.064+0000 7fa4f9672640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa4f415d4f0 0x7fa4f40d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38408/0 (socket says 172.21.15.105:38408) 2023-11-28T18:23:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.064+0000 7fa4f9672640 1 -- 172.21.15.105:0/1861829312 learned_addr learned my addr 172.21.15.105:0/1861829312 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.064+0000 7fa4f8e71640 1 --2- 172.21.15.105:0/1861829312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4f40d3c30 0x7fa4f40d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.064+0000 7fa4f9e73640 1 --2- 172.21.15.105:0/1861829312 >> v2:172.21.15.148:3300/0 conn(0x7fa4f40d1890 0x7fa4f40cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.064+0000 7fa4f9672640 1 -- 172.21.15.105:0/1861829312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4f40d3c30 msgr2=0x7fa4f40d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.064+0000 7fa4f9672640 1 --2- 172.21.15.105:0/1861829312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4f40d3c30 0x7fa4f40d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.064+0000 7fa4f9672640 1 -- 172.21.15.105:0/1861829312 >> v2:172.21.15.148:3300/0 conn(0x7fa4f40d1890 msgr2=0x7fa4f40cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:49.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.064+0000 7fa4f9672640 1 --2- 172.21.15.105:0/1861829312 >> v2:172.21.15.148:3300/0 conn(0x7fa4f40d1890 0x7fa4f40cfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.064+0000 7fa4f9672640 1 -- 172.21.15.105:0/1861829312 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f40ced10 con 0x7fa4f415d4f0 2023-11-28T18:23:49.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.068+0000 7fa4f9672640 1 --2- 172.21.15.105:0/1861829312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f415d4f0 0x7fa4f40d0e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4160fa0 tx=0x7fa4f001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625de61ffbe89265 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:49.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.068+0000 7fa4ebfff640 1 -- 172.21.15.105:0/1861829312 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f00ac640 con 0x7fa4f415d4f0 2023-11-28T18:23:49.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.068+0000 7fa4ebfff640 1 -- 172.21.15.105:0/1861829312 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f00ac7a0 con 0x7fa4f415d4f0 2023-11-28T18:23:49.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.068+0000 7fa4ebfff640 1 -- 172.21.15.105:0/1861829312 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f00acb30 con 0x7fa4f415d4f0 2023-11-28T18:23:49.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.068+0000 7fa4fb8d6640 1 -- 172.21.15.105:0/1861829312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f415d4f0 msgr2=0x7fa4f40d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:49.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.068+0000 7fa4fb8d6640 1 --2- 172.21.15.105:0/1861829312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f415d4f0 0x7fa4f40d0e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4160fa0 tx=0x7fa4f001c440 comp rx=0 tx=0).stop 2023-11-28T18:23:49.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.068+0000 7fa4fb8d6640 1 -- 172.21.15.105:0/1861829312 shutdown_connections 2023-11-28T18:23:49.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.068+0000 7fa4fb8d6640 1 --2- 172.21.15.105:0/1861829312 >> v2:172.21.15.148:3300/0 conn(0x7fa4f40d1890 0x7fa4f40cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.068+0000 7fa4fb8d6640 1 --2- 172.21.15.105:0/1861829312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4f40d3c30 0x7fa4f40d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.068+0000 7fa4fb8d6640 1 --2- 172.21.15.105:0/1861829312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f415d4f0 0x7fa4f40d0e10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.068+0000 7fa4fb8d6640 1 -- 172.21.15.105:0/1861829312 >> 172.21.15.105:0/1861829312 conn(0x7fa4f40d3030 msgr2=0x7fa4f40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:49.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.068+0000 7fa4fb8d6640 1 -- 172.21.15.105:0/1861829312 shutdown_connections 2023-11-28T18:23:49.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.068+0000 7fa4fb8d6640 1 -- 172.21.15.105:0/1861829312 wait complete. 2023-11-28T18:23:49.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4fb8d6640 1 Processor -- start 2023-11-28T18:23:49.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4fb8d6640 1 -- start start 2023-11-28T18:23:49.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4fb8d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4f40d1890 0x7fa4f414c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:49.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4fb8d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4f40d3c30 0x7fa4f40cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:49.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4fb8d6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f415d4f0 0x7fa4f41552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:49.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4fb8d6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f40cbb80 con 0x7fa4f415d4f0 2023-11-28T18:23:49.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4fb8d6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f415c5b0 con 0x7fa4f40d1890 2023-11-28T18:23:49.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4fb8d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f414ca90 con 0x7fa4f40d3c30 2023-11-28T18:23:49.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4f9e73640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f415d4f0 0x7fa4f41552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:49.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4f9e73640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f415d4f0 0x7fa4f41552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38422/0 (socket says 172.21.15.105:38422) 2023-11-28T18:23:49.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4f9e73640 1 -- 172.21.15.105:0/3457156241 learned_addr learned my addr 172.21.15.105:0/3457156241 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:49.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4f8e71640 1 --2- 172.21.15.105:0/3457156241 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4f40d3c30 0x7fa4f40cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:49.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4f9672640 1 --2- 172.21.15.105:0/3457156241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4f40d1890 0x7fa4f414c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:49.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4f9e73640 1 -- 172.21.15.105:0/3457156241 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4f40d3c30 msgr2=0x7fa4f40cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:49.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4f9e73640 1 --2- 172.21.15.105:0/3457156241 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4f40d3c30 0x7fa4f40cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4f9e73640 1 -- 172.21.15.105:0/3457156241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4f40d1890 msgr2=0x7fa4f414c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:49.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4f9e73640 1 --2- 172.21.15.105:0/3457156241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4f40d1890 0x7fa4f414c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4f9e73640 1 -- 172.21.15.105:0/3457156241 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f00094d0 con 0x7fa4f415d4f0 2023-11-28T18:23:49.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4f9e73640 1 --2- 172.21.15.105:0/3457156241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f415d4f0 0x7fa4f41552a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec007e90 tx=0x7fa4ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:49.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4e9ffb640 1 -- 172.21.15.105:0/3457156241 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec014420 con 0x7fa4f415d4f0 2023-11-28T18:23:49.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4e9ffb640 1 -- 172.21.15.105:0/3457156241 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ec0145a0 con 0x7fa4f415d4f0 2023-11-28T18:23:49.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4e9ffb640 1 -- 172.21.15.105:0/3457156241 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec014950 con 0x7fa4f415d4f0 2023-11-28T18:23:49.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4fb8d6640 1 -- 172.21.15.105:0/3457156241 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f415c2d0 con 0x7fa4f415d4f0 2023-11-28T18:23:49.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4fb8d6640 1 -- 172.21.15.105:0/3457156241 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f40cc090 con 0x7fa4f415d4f0 2023-11-28T18:23:49.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.072+0000 7fa4fb8d6640 1 -- 172.21.15.105:0/3457156241 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f4067570 con 0x7fa4f415d4f0 2023-11-28T18:23:49.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.076+0000 7fa4e9ffb640 1 -- 172.21.15.105:0/3457156241 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa4ec014ab0 con 0x7fa4f415d4f0 2023-11-28T18:23:49.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.076+0000 7fa4e9ffb640 1 --2- 172.21.15.105:0/3457156241 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa4d80668d0 0x7fa4d8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.076+0000 7fa4e9ffb640 1 -- 172.21.15.105:0/3457156241 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa4ec085030 con 0x7fa4f415d4f0 2023-11-28T18:23:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.076+0000 7fa4f9672640 1 --2- 172.21.15.105:0/3457156241 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa4d80668d0 0x7fa4d8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.076+0000 7fa4f9672640 1 --2- 172.21.15.105:0/3457156241 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa4d80668d0 0x7fa4d8068d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa4f00ae000 tx=0x7fa4f00a6070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:49.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.080+0000 7fa4e9ffb640 1 -- 172.21.15.105:0/3457156241 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa4ec0b6280 con 0x7fa4f415d4f0 2023-11-28T18:23:49.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.232+0000 7fa4fb8d6640 1 -- 172.21.15.105:0/3457156241 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4f4158d10 con 0x7fa4f415d4f0 2023-11-28T18:23:49.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.232+0000 7fa4e9ffb640 1 -- 172.21.15.105:0/3457156241 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7fa4ec051f00 con 0x7fa4f415d4f0 2023-11-28T18:23:49.236 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:49.236 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":24,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:23:38.458954+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:23:49.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.232+0000 7fa4fb8d6640 1 -- 172.21.15.105:0/3457156241 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa4d80668d0 msgr2=0x7fa4d8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:49.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.232+0000 7fa4fb8d6640 1 --2- 172.21.15.105:0/3457156241 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa4d80668d0 0x7fa4d8068d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa4f00ae000 tx=0x7fa4f00a6070 comp rx=0 tx=0).stop 2023-11-28T18:23:49.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.232+0000 7fa4fb8d6640 1 -- 172.21.15.105:0/3457156241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f415d4f0 msgr2=0x7fa4f41552a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:49.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.232+0000 7fa4fb8d6640 1 --2- 172.21.15.105:0/3457156241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f415d4f0 0x7fa4f41552a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec007e90 tx=0x7fa4ec00e4b0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.232+0000 7fa4fb8d6640 1 -- 172.21.15.105:0/3457156241 shutdown_connections 2023-11-28T18:23:49.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.232+0000 7fa4fb8d6640 1 --2- 172.21.15.105:0/3457156241 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa4d80668d0 0x7fa4d8068d90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.232+0000 7fa4fb8d6640 1 --2- 172.21.15.105:0/3457156241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f415d4f0 0x7fa4f41552a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.232+0000 7fa4fb8d6640 1 --2- 172.21.15.105:0/3457156241 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4f40d3c30 0x7fa4f40cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.232+0000 7fa4fb8d6640 1 --2- 172.21.15.105:0/3457156241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4f40d1890 0x7fa4f414c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.232+0000 7fa4fb8d6640 1 -- 172.21.15.105:0/3457156241 >> 172.21.15.105:0/3457156241 conn(0x7fa4f40d3030 msgr2=0x7fa4f406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:49.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.232+0000 7fa4fb8d6640 1 -- 172.21.15.105:0/3457156241 shutdown_connections 2023-11-28T18:23:49.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.232+0000 7fa4fb8d6640 1 -- 172.21.15.105:0/3457156241 wait complete. 2023-11-28T18:23:49.255 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:23:49.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.392+0000 7f913072a640 1 Processor -- start 2023-11-28T18:23:49.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.392+0000 7f913072a640 1 -- start start 2023-11-28T18:23:49.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.392+0000 7f913072a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f912815d4f0 0x7f91280d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:49.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.392+0000 7f913072a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91280d3c30 0x7f91280d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:49.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.392+0000 7f913072a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91280b25b0 con 0x7f91280d1890 2023-11-28T18:23:49.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.392+0000 7f913072a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91280b2900 con 0x7f912815d4f0 2023-11-28T18:23:49.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.392+0000 7f913072a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91280cea30 con 0x7f91280d3c30 2023-11-28T18:23:49.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.392+0000 7f912e4c6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f912815d4f0 0x7f91280d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:49.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.392+0000 7f912dcc5640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91280d3c30 0x7f91280d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:49.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.392+0000 7f912ecc7640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f91280d1890 0x7f91280cfa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:55068/0 (socket says 172.21.15.105:55068) 2023-11-28T18:23:49.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.392+0000 7f912ecc7640 1 -- 172.21.15.105:0/3273799697 learned_addr learned my addr 172.21.15.105:0/3273799697 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:49.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.392+0000 7f912e4c6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f912815d4f0 0x7f91280d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:60928/0 (socket says 172.21.15.105:60928) 2023-11-28T18:23:49.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.392+0000 7f912e4c6640 1 -- 172.21.15.105:0/3273799697 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91280d3c30 msgr2=0x7f91280d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:49.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.392+0000 7f912e4c6640 1 --2- 172.21.15.105:0/3273799697 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91280d3c30 0x7f91280d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.392+0000 7f912e4c6640 1 -- 172.21.15.105:0/3273799697 >> v1:172.21.15.148:6789/0 conn(0x7f91280d1890 legacy=0x7f91280cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:49.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.392+0000 7f912e4c6640 1 -- 172.21.15.105:0/3273799697 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91280ced10 con 0x7f912815d4f0 2023-11-28T18:23:49.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.392+0000 7f912d4c4640 1 -- 172.21.15.105:0/3273799697 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735919956 0 0) 0x7f91280b25b0 con 0x7f91280d1890 2023-11-28T18:23:49.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.396+0000 7f912e4c6640 1 --2- 172.21.15.105:0/3273799697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f912815d4f0 0x7f91280d0e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f91240097c0 tx=0x7f912401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2f4891d9cdd23e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:49.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.396+0000 7f912d4c4640 1 -- 172.21.15.105:0/3273799697 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91240ac640 con 0x7f912815d4f0 2023-11-28T18:23:49.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.396+0000 7f912d4c4640 1 -- 172.21.15.105:0/3273799697 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91240ac7a0 con 0x7f912815d4f0 2023-11-28T18:23:49.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.396+0000 7f912d4c4640 1 -- 172.21.15.105:0/3273799697 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91240acb30 con 0x7f912815d4f0 2023-11-28T18:23:49.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.396+0000 7f913072a640 1 -- 172.21.15.105:0/3273799697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f912815d4f0 msgr2=0x7f91280d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:49.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.396+0000 7f913072a640 1 --2- 172.21.15.105:0/3273799697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f912815d4f0 0x7f91280d0e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f91240097c0 tx=0x7f912401c3c0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.396+0000 7f913072a640 1 -- 172.21.15.105:0/3273799697 shutdown_connections 2023-11-28T18:23:49.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.396+0000 7f913072a640 1 --2- 172.21.15.105:0/3273799697 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91280d3c30 0x7f91280d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.396+0000 7f913072a640 1 --2- 172.21.15.105:0/3273799697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f912815d4f0 0x7f91280d0e10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.396+0000 7f913072a640 1 -- 172.21.15.105:0/3273799697 >> 172.21.15.105:0/3273799697 conn(0x7f91280d3030 msgr2=0x7f91280d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:49.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.396+0000 7f913072a640 1 -- 172.21.15.105:0/3273799697 shutdown_connections 2023-11-28T18:23:49.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.396+0000 7f913072a640 1 -- 172.21.15.105:0/3273799697 wait complete. 2023-11-28T18:23:49.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.400+0000 7f913072a640 1 Processor -- start 2023-11-28T18:23:49.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.400+0000 7f913072a640 1 -- start start 2023-11-28T18:23:49.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.400+0000 7f913072a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91280d1890 0x7f91280b8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:49.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.400+0000 7f913072a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91280d3c30 0x7f91280cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:49.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.400+0000 7f913072a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f912815d4f0 0x7f91281f9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:49.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.400+0000 7f913072a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91280b6580 con 0x7f912815d4f0 2023-11-28T18:23:49.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.400+0000 7f913072a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91280b8970 con 0x7f91280d1890 2023-11-28T18:23:49.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.400+0000 7f913072a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91280b5c50 con 0x7f91280d3c30 2023-11-28T18:23:49.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.400+0000 7f912ecc7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f912815d4f0 0x7f91281f9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:49.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.400+0000 7f912ecc7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f912815d4f0 0x7f91281f9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38436/0 (socket says 172.21.15.105:38436) 2023-11-28T18:23:49.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.400+0000 7f912ecc7640 1 -- 172.21.15.105:0/1736980508 learned_addr learned my addr 172.21.15.105:0/1736980508 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:49.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.400+0000 7f912e4c6640 1 --2- 172.21.15.105:0/1736980508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91280d1890 0x7f91280b8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:49.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.400+0000 7f912dcc5640 1 --2- 172.21.15.105:0/1736980508 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91280d3c30 0x7f91280cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:49.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.400+0000 7f912ecc7640 1 -- 172.21.15.105:0/1736980508 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91280d3c30 msgr2=0x7f91280cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:49.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.400+0000 7f912ecc7640 1 --2- 172.21.15.105:0/1736980508 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91280d3c30 0x7f91280cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.400+0000 7f912ecc7640 1 -- 172.21.15.105:0/1736980508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91280d1890 msgr2=0x7f91280b8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:49.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.400+0000 7f912ecc7640 1 --2- 172.21.15.105:0/1736980508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91280d1890 0x7f91280b8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.400+0000 7f912ecc7640 1 -- 172.21.15.105:0/1736980508 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9124009420 con 0x7f912815d4f0 2023-11-28T18:23:49.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.400+0000 7f912ecc7640 1 --2- 172.21.15.105:0/1736980508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f912815d4f0 0x7f91281f9d60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f912000c780 tx=0x7f912000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:49.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.404+0000 7f911effd640 1 -- 172.21.15.105:0/1736980508 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912000ce90 con 0x7f912815d4f0 2023-11-28T18:23:49.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.404+0000 7f911effd640 1 -- 172.21.15.105:0/1736980508 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912000a640 con 0x7f912815d4f0 2023-11-28T18:23:49.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.404+0000 7f911effd640 1 -- 172.21.15.105:0/1736980508 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912000a9d0 con 0x7f912815d4f0 2023-11-28T18:23:49.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.404+0000 7f913072a640 1 -- 172.21.15.105:0/1736980508 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91280b5f30 con 0x7f912815d4f0 2023-11-28T18:23:49.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.404+0000 7f913072a640 1 -- 172.21.15.105:0/1736980508 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91280b8680 con 0x7f912815d4f0 2023-11-28T18:23:49.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.404+0000 7f913072a640 1 -- 172.21.15.105:0/1736980508 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91280b66f0 con 0x7f912815d4f0 2023-11-28T18:23:49.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.404+0000 7f911effd640 1 -- 172.21.15.105:0/1736980508 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f912000ab30 con 0x7f912815d4f0 2023-11-28T18:23:49.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.404+0000 7f911effd640 1 --2- 172.21.15.105:0/1736980508 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f910c0668d0 0x7f910c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:49.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.404+0000 7f911effd640 1 -- 172.21.15.105:0/1736980508 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9120083fe0 con 0x7f912815d4f0 2023-11-28T18:23:49.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.404+0000 7f912e4c6640 1 --2- 172.21.15.105:0/1736980508 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f910c0668d0 0x7f910c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:49.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.408+0000 7f912e4c6640 1 --2- 172.21.15.105:0/1736980508 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f910c0668d0 0x7f910c068d90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f91240ae000 tx=0x7f91240a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:49.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.408+0000 7f911effd640 1 -- 172.21.15.105:0/1736980508 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9120050fe0 con 0x7f912815d4f0 2023-11-28T18:23:49.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.560+0000 7f913072a640 1 -- 172.21.15.105:0/1736980508 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9128067570 con 0x7f912815d4f0 2023-11-28T18:23:49.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.560+0000 7f911effd640 1 -- 172.21.15.105:0/1736980508 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7f9120018070 con 0x7f912815d4f0 2023-11-28T18:23:49.562 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:49.563 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":24,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:23:38.458954+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:23:49.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.560+0000 7f913072a640 1 -- 172.21.15.105:0/1736980508 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f910c0668d0 msgr2=0x7f910c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:49.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.560+0000 7f913072a640 1 --2- 172.21.15.105:0/1736980508 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f910c0668d0 0x7f910c068d90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f91240ae000 tx=0x7f91240a7000 comp rx=0 tx=0).stop 2023-11-28T18:23:49.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.560+0000 7f913072a640 1 -- 172.21.15.105:0/1736980508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f912815d4f0 msgr2=0x7f91281f9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:49.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.560+0000 7f913072a640 1 --2- 172.21.15.105:0/1736980508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f912815d4f0 0x7f91281f9d60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f912000c780 tx=0x7f912000cc40 comp rx=0 tx=0).stop 2023-11-28T18:23:49.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.560+0000 7f913072a640 1 -- 172.21.15.105:0/1736980508 shutdown_connections 2023-11-28T18:23:49.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.560+0000 7f913072a640 1 --2- 172.21.15.105:0/1736980508 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f910c0668d0 0x7f910c068d90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.560+0000 7f913072a640 1 --2- 172.21.15.105:0/1736980508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f912815d4f0 0x7f91281f9d60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.560+0000 7f913072a640 1 --2- 172.21.15.105:0/1736980508 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91280d3c30 0x7f91280cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.560+0000 7f913072a640 1 --2- 172.21.15.105:0/1736980508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91280d1890 0x7f91280b8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.560+0000 7f913072a640 1 -- 172.21.15.105:0/1736980508 >> 172.21.15.105:0/1736980508 conn(0x7f91280d3030 msgr2=0x7f9128156520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:49.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.560+0000 7f913072a640 1 -- 172.21.15.105:0/1736980508 shutdown_connections 2023-11-28T18:23:49.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.560+0000 7f913072a640 1 -- 172.21.15.105:0/1736980508 wait complete. 2023-11-28T18:23:49.584 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T18:23:49.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.720+0000 7fae582c9640 1 Processor -- start 2023-11-28T18:23:49.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.720+0000 7fae582c9640 1 -- start start 2023-11-28T18:23:49.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.720+0000 7fae582c9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fae50159f40 0x7fae500caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:49.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.720+0000 7fae582c9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fae500cc760 0x7fae500ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:49.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.720+0000 7fae582c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae500ad680 0x7fae5015da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:49.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.720+0000 7fae582c9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae5006bf50 con 0x7fae500cc760 2023-11-28T18:23:49.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.720+0000 7fae582c9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae500ccbc0 con 0x7fae50159f40 2023-11-28T18:23:49.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.720+0000 7fae582c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5015a320 con 0x7fae500ad680 2023-11-28T18:23:49.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.724+0000 7fae55864640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fae500cc760 0x7fae500ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:49.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.724+0000 7fae55864640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fae500cc760 0x7fae500ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38450/0 (socket says 172.21.15.105:38450) 2023-11-28T18:23:49.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.724+0000 7fae55864640 1 -- 172.21.15.105:0/1688380180 learned_addr learned my addr 172.21.15.105:0/1688380180 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:49.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.724+0000 7fae56065640 1 --2- 172.21.15.105:0/1688380180 >> v2:172.21.15.148:3300/0 conn(0x7fae50159f40 0x7fae500caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:49.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.724+0000 7fae56866640 1 --2- 172.21.15.105:0/1688380180 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae500ad680 0x7fae5015da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:49.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.724+0000 7fae55864640 1 -- 172.21.15.105:0/1688380180 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae500ad680 msgr2=0x7fae5015da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:49.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.724+0000 7fae55864640 1 --2- 172.21.15.105:0/1688380180 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae500ad680 0x7fae5015da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.724+0000 7fae55864640 1 -- 172.21.15.105:0/1688380180 >> v2:172.21.15.148:3300/0 conn(0x7fae50159f40 msgr2=0x7fae500caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:49.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.724+0000 7fae55864640 1 --2- 172.21.15.105:0/1688380180 >> v2:172.21.15.148:3300/0 conn(0x7fae50159f40 0x7fae500caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.724+0000 7fae55864640 1 -- 172.21.15.105:0/1688380180 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae50071380 con 0x7fae500cc760 2023-11-28T18:23:49.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.724+0000 7fae55864640 1 --2- 172.21.15.105:0/1688380180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae500cc760 0x7fae500ad140 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fae5006e500 tx=0x7fae4001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58e12756e89b43 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:49.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.724+0000 7fae55063640 1 -- 172.21.15.105:0/1688380180 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae400ab600 con 0x7fae500cc760 2023-11-28T18:23:49.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.724+0000 7fae55063640 1 -- 172.21.15.105:0/1688380180 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae400ab760 con 0x7fae500cc760 2023-11-28T18:23:49.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.724+0000 7fae55063640 1 -- 172.21.15.105:0/1688380180 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae400abaf0 con 0x7fae500cc760 2023-11-28T18:23:49.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.728+0000 7fae582c9640 1 -- 172.21.15.105:0/1688380180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae500cc760 msgr2=0x7fae500ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:49.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.728+0000 7fae582c9640 1 --2- 172.21.15.105:0/1688380180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae500cc760 0x7fae500ad140 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fae5006e500 tx=0x7fae4001c440 comp rx=0 tx=0).stop 2023-11-28T18:23:49.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.728+0000 7fae582c9640 1 -- 172.21.15.105:0/1688380180 shutdown_connections 2023-11-28T18:23:49.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.728+0000 7fae582c9640 1 --2- 172.21.15.105:0/1688380180 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae500ad680 0x7fae5015da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.728+0000 7fae582c9640 1 --2- 172.21.15.105:0/1688380180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae500cc760 0x7fae500ad140 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.728+0000 7fae582c9640 1 --2- 172.21.15.105:0/1688380180 >> v2:172.21.15.148:3300/0 conn(0x7fae50159f40 0x7fae500caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.728+0000 7fae582c9640 1 -- 172.21.15.105:0/1688380180 >> 172.21.15.105:0/1688380180 conn(0x7fae500ceb90 msgr2=0x7fae500cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:49.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.728+0000 7fae582c9640 1 -- 172.21.15.105:0/1688380180 shutdown_connections 2023-11-28T18:23:49.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.728+0000 7fae582c9640 1 -- 172.21.15.105:0/1688380180 wait complete. 2023-11-28T18:23:49.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.728+0000 7fae582c9640 1 Processor -- start 2023-11-28T18:23:49.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.728+0000 7fae582c9640 1 -- start start 2023-11-28T18:23:49.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.728+0000 7fae582c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae500ad680 0x7fae50147500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:49.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.728+0000 7fae582c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae500cc760 0x7fae5014c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:49.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.728+0000 7fae582c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae50159f40 0x7fae5014c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:49.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.728+0000 7fae582c9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae50153b30 con 0x7fae500cc760 2023-11-28T18:23:49.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.728+0000 7fae582c9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae50147a40 con 0x7fae50159f40 2023-11-28T18:23:49.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.728+0000 7fae582c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae50152f20 con 0x7fae500ad680 2023-11-28T18:23:49.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.728+0000 7fae55864640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae500cc760 0x7fae5014c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:49.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.728+0000 7fae55864640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae500cc760 0x7fae5014c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38456/0 (socket says 172.21.15.105:38456) 2023-11-28T18:23:49.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.728+0000 7fae55864640 1 -- 172.21.15.105:0/827634940 learned_addr learned my addr 172.21.15.105:0/827634940 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:49.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.728+0000 7fae56065640 1 --2- 172.21.15.105:0/827634940 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae500ad680 0x7fae50147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:49.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.728+0000 7fae56866640 1 --2- 172.21.15.105:0/827634940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae50159f40 0x7fae5014c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:49.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.732+0000 7fae55864640 1 -- 172.21.15.105:0/827634940 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae500ad680 msgr2=0x7fae50147500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:49.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.732+0000 7fae55864640 1 --2- 172.21.15.105:0/827634940 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae500ad680 0x7fae50147500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.732+0000 7fae55864640 1 -- 172.21.15.105:0/827634940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae50159f40 msgr2=0x7fae5014c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:49.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.732+0000 7fae55864640 1 --2- 172.21.15.105:0/827634940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae50159f40 0x7fae5014c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:49.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.732+0000 7fae55864640 1 -- 172.21.15.105:0/827634940 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae400094d0 con 0x7fae500cc760 2023-11-28T18:23:49.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.732+0000 7fae55864640 1 --2- 172.21.15.105:0/827634940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae500cc760 0x7fae5014c190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fae4001c920 tx=0x7fae4001ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:49.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.732+0000 7fae46ffd640 1 -- 172.21.15.105:0/827634940 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae400be5f0 con 0x7fae500cc760 2023-11-28T18:23:49.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.732+0000 7fae46ffd640 1 -- 172.21.15.105:0/827634940 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae400be750 con 0x7fae500cc760 2023-11-28T18:23:49.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.732+0000 7fae46ffd640 1 -- 172.21.15.105:0/827634940 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae400beb00 con 0x7fae500cc760 2023-11-28T18:23:49.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.732+0000 7fae582c9640 1 -- 172.21.15.105:0/827634940 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae50153200 con 0x7fae500cc760 2023-11-28T18:23:49.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.732+0000 7fae582c9640 1 -- 172.21.15.105:0/827634940 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae501537c0 con 0x7fae500cc760 2023-11-28T18:23:49.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.732+0000 7fae582c9640 1 -- 172.21.15.105:0/827634940 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae50153980 con 0x7fae500cc760 2023-11-28T18:23:49.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.732+0000 7fae46ffd640 1 -- 172.21.15.105:0/827634940 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fae400b43f0 con 0x7fae500cc760 2023-11-28T18:23:49.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.732+0000 7fae46ffd640 1 --2- 172.21.15.105:0/827634940 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae340668d0 0x7fae34068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:49.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.732+0000 7fae46ffd640 1 -- 172.21.15.105:0/827634940 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fae4011f300 con 0x7fae500cc760 2023-11-28T18:23:49.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.732+0000 7fae56065640 1 --2- 172.21.15.105:0/827634940 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae340668d0 0x7fae34068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:49.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.736+0000 7fae56065640 1 --2- 172.21.15.105:0/827634940 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae340668d0 0x7fae34068d90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fae5015cdc0 tx=0x7fae4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:49.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.736+0000 7fae46ffd640 1 -- 172.21.15.105:0/827634940 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fae400ec260 con 0x7fae500cc760 2023-11-28T18:23:49.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:49.920+0000 7fae582c9640 1 -- 172.21.15.105:0/827634940 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fae50067570 con 0x7fae500cc760 2023-11-28T18:23:50.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.060+0000 7fae46ffd640 1 -- 172.21.15.105:0/827634940 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7fae400ef880 con 0x7fae500cc760 2023-11-28T18:23:50.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.064+0000 7fae582c9640 1 -- 172.21.15.105:0/827634940 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae340668d0 msgr2=0x7fae34068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:50.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.064+0000 7fae582c9640 1 --2- 172.21.15.105:0/827634940 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae340668d0 0x7fae34068d90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fae5015cdc0 tx=0x7fae4c008040 comp rx=0 tx=0).stop 2023-11-28T18:23:50.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.064+0000 7fae582c9640 1 -- 172.21.15.105:0/827634940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae500cc760 msgr2=0x7fae5014c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:50.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.064+0000 7fae582c9640 1 --2- 172.21.15.105:0/827634940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae500cc760 0x7fae5014c190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fae4001c920 tx=0x7fae4001ac80 comp rx=0 tx=0).stop 2023-11-28T18:23:50.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.064+0000 7fae582c9640 1 -- 172.21.15.105:0/827634940 shutdown_connections 2023-11-28T18:23:50.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.064+0000 7fae582c9640 1 --2- 172.21.15.105:0/827634940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae50159f40 0x7fae5014c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:50.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.064+0000 7fae582c9640 1 --2- 172.21.15.105:0/827634940 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae340668d0 0x7fae34068d90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:50.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.064+0000 7fae582c9640 1 --2- 172.21.15.105:0/827634940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae500cc760 0x7fae5014c190 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:50.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.064+0000 7fae582c9640 1 --2- 172.21.15.105:0/827634940 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae500ad680 0x7fae50147500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:50.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.064+0000 7fae582c9640 1 -- 172.21.15.105:0/827634940 >> 172.21.15.105:0/827634940 conn(0x7fae500ceb90 msgr2=0x7fae500af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:50.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.064+0000 7fae582c9640 1 -- 172.21.15.105:0/827634940 shutdown_connections 2023-11-28T18:23:50.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.064+0000 7fae582c9640 1 -- 172.21.15.105:0/827634940 wait complete. 2023-11-28T18:23:50.085 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-11-28T18:23:50.086 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T18:23:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.224+0000 7f56c8db8640 1 Processor -- start 2023-11-28T18:23:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.224+0000 7f56c8db8640 1 -- start start 2023-11-28T18:23:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.224+0000 7f56c8db8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56c415d300 0x7f56c40d37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.224+0000 7f56c8db8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f56c40d3ce0 0x7f56c40d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.224+0000 7f56c8db8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f56c40d1b90 0x7f56c40cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.224+0000 7f56c8db8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c40d1470 con 0x7f56c40d1b90 2023-11-28T18:23:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.224+0000 7f56c8db8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c40d2040 con 0x7f56c40d3ce0 2023-11-28T18:23:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.224+0000 7f56c8db8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c40cefa0 con 0x7f56c415d300 2023-11-28T18:23:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.224+0000 7f56c2d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f56c40d1b90 0x7f56c40cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.224+0000 7f56c2d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f56c40d1b90 0x7f56c40cdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38466/0 (socket says 172.21.15.105:38466) 2023-11-28T18:23:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.224+0000 7f56c2d9d640 1 -- 172.21.15.105:0/3805273989 learned_addr learned my addr 172.21.15.105:0/3805273989 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.224+0000 7f56c259c640 1 --2- 172.21.15.105:0/3805273989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56c415d300 0x7f56c40d37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:50.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.224+0000 7f56bbfff640 1 --2- 172.21.15.105:0/3805273989 >> v2:172.21.15.148:3300/0 conn(0x7f56c40d3ce0 0x7f56c40d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:50.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.224+0000 7f56c2d9d640 1 -- 172.21.15.105:0/3805273989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56c415d300 msgr2=0x7f56c40d37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:50.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.224+0000 7f56c2d9d640 1 --2- 172.21.15.105:0/3805273989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56c415d300 0x7f56c40d37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:50.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.224+0000 7f56c2d9d640 1 -- 172.21.15.105:0/3805273989 >> v2:172.21.15.148:3300/0 conn(0x7f56c40d3ce0 msgr2=0x7f56c40d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:50.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.224+0000 7f56c2d9d640 1 --2- 172.21.15.105:0/3805273989 >> v2:172.21.15.148:3300/0 conn(0x7f56c40d3ce0 0x7f56c40d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:50.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.224+0000 7f56c2d9d640 1 -- 172.21.15.105:0/3805273989 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c40d0c60 con 0x7f56c40d1b90 2023-11-28T18:23:50.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.228+0000 7f56c2d9d640 1 --2- 172.21.15.105:0/3805273989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56c40d1b90 0x7f56c40cdb60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f56c408c9d0 tx=0x7f56bc01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9117ec74aeec578c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:50.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.228+0000 7f56c1d9b640 1 -- 172.21.15.105:0/3805273989 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc0a5790 con 0x7f56c40d1b90 2023-11-28T18:23:50.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.228+0000 7f56c1d9b640 1 -- 172.21.15.105:0/3805273989 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc0a58f0 con 0x7f56c40d1b90 2023-11-28T18:23:50.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.228+0000 7f56c1d9b640 1 -- 172.21.15.105:0/3805273989 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc0a5c80 con 0x7f56c40d1b90 2023-11-28T18:23:50.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.228+0000 7f56c8db8640 1 -- 172.21.15.105:0/3805273989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56c40d1b90 msgr2=0x7f56c40cdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:50.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.228+0000 7f56c8db8640 1 --2- 172.21.15.105:0/3805273989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56c40d1b90 0x7f56c40cdb60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f56c408c9d0 tx=0x7f56bc01c910 comp rx=0 tx=0).stop 2023-11-28T18:23:50.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.228+0000 7f56c8db8640 1 -- 172.21.15.105:0/3805273989 shutdown_connections 2023-11-28T18:23:50.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.228+0000 7f56c8db8640 1 --2- 172.21.15.105:0/3805273989 >> v2:172.21.15.148:3300/0 conn(0x7f56c40d3ce0 0x7f56c40d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:50.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.228+0000 7f56c8db8640 1 --2- 172.21.15.105:0/3805273989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56c40d1b90 0x7f56c40cdb60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:50.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.228+0000 7f56c8db8640 1 --2- 172.21.15.105:0/3805273989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56c415d300 0x7f56c40d37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:50.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.228+0000 7f56c8db8640 1 -- 172.21.15.105:0/3805273989 >> 172.21.15.105:0/3805273989 conn(0x7f56c409c360 msgr2=0x7f56c40d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:50.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.228+0000 7f56c8db8640 1 -- 172.21.15.105:0/3805273989 shutdown_connections 2023-11-28T18:23:50.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.228+0000 7f56c8db8640 1 -- 172.21.15.105:0/3805273989 wait complete. 2023-11-28T18:23:50.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56c8db8640 1 Processor -- start 2023-11-28T18:23:50.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56c8db8640 1 -- start start 2023-11-28T18:23:50.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56c8db8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56c40d1b90 0x7f56c40cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:50.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56c8db8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56c40d3ce0 0x7f56c41603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:50.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56c8db8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56c415d300 0x7f56c4154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:50.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56c8db8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c4157740 con 0x7f56c40d1b90 2023-11-28T18:23:50.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56c8db8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c4158e20 con 0x7f56c415d300 2023-11-28T18:23:50.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56c8db8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c4158b10 con 0x7f56c40d3ce0 2023-11-28T18:23:50.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56c259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56c40d1b90 0x7f56c40cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:50.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56c259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56c40d1b90 0x7f56c40cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38478/0 (socket says 172.21.15.105:38478) 2023-11-28T18:23:50.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56c259c640 1 -- 172.21.15.105:0/974479355 learned_addr learned my addr 172.21.15.105:0/974479355 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:50.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56bbfff640 1 --2- 172.21.15.105:0/974479355 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56c40d3ce0 0x7f56c41603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:50.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56c2d9d640 1 --2- 172.21.15.105:0/974479355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56c415d300 0x7f56c4154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:50.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56c259c640 1 -- 172.21.15.105:0/974479355 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56c40d3ce0 msgr2=0x7f56c41603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:50.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56c259c640 1 --2- 172.21.15.105:0/974479355 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56c40d3ce0 0x7f56c41603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:50.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56c259c640 1 -- 172.21.15.105:0/974479355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56c415d300 msgr2=0x7f56c4154f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:50.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56c259c640 1 --2- 172.21.15.105:0/974479355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56c415d300 0x7f56c4154f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:50.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56c259c640 1 -- 172.21.15.105:0/974479355 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc009b30 con 0x7f56c40d1b90 2023-11-28T18:23:50.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56c259c640 1 --2- 172.21.15.105:0/974479355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56c40d1b90 0x7f56c40cb1b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f56b400e8a0 tx=0x7f56b400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:50.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56baffd640 1 -- 172.21.15.105:0/974479355 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4009510 con 0x7f56c40d1b90 2023-11-28T18:23:50.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56baffd640 1 -- 172.21.15.105:0/974479355 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b400c940 con 0x7f56c40d1b90 2023-11-28T18:23:50.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56c8db8640 1 -- 172.21.15.105:0/974479355 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c4155ad0 con 0x7f56c40d1b90 2023-11-28T18:23:50.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.232+0000 7f56baffd640 1 -- 172.21.15.105:0/974479355 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b400ccb0 con 0x7f56c40d1b90 2023-11-28T18:23:50.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.236+0000 7f56c8db8640 1 -- 172.21.15.105:0/974479355 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c4156090 con 0x7f56c40d1b90 2023-11-28T18:23:50.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.236+0000 7f56c8db8640 1 -- 172.21.15.105:0/974479355 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c41562b0 con 0x7f56c40d1b90 2023-11-28T18:23:50.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.236+0000 7f56baffd640 1 -- 172.21.15.105:0/974479355 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f56b4013470 con 0x7f56c40d1b90 2023-11-28T18:23:50.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.236+0000 7f56baffd640 1 --2- 172.21.15.105:0/974479355 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56a40668d0 0x7f56a4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:50.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.236+0000 7f56baffd640 1 -- 172.21.15.105:0/974479355 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f56b40847f0 con 0x7f56c40d1b90 2023-11-28T18:23:50.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.236+0000 7f56bbfff640 1 --2- 172.21.15.105:0/974479355 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56a40668d0 0x7f56a4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:50.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.236+0000 7f56bbfff640 1 --2- 172.21.15.105:0/974479355 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56a40668d0 0x7f56a4068d90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f56c4140d20 tx=0x7f56ac009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:50.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.240+0000 7f56baffd640 1 -- 172.21.15.105:0/974479355 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f56b4051750 con 0x7f56c40d1b90 2023-11-28T18:23:50.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:50.424+0000 7f56c8db8640 1 -- 172.21.15.105:0/974479355 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f56c4067570 con 0x7f56c40d1b90 2023-11-28T18:23:51.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.068+0000 7f56baffd640 1 -- 172.21.15.105:0/974479355 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f56b4054d70 con 0x7f56c40d1b90 2023-11-28T18:23:51.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.068+0000 7f56c8db8640 1 -- 172.21.15.105:0/974479355 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56a40668d0 msgr2=0x7f56a4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:51.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.068+0000 7f56c8db8640 1 --2- 172.21.15.105:0/974479355 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56a40668d0 0x7f56a4068d90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f56c4140d20 tx=0x7f56ac009190 comp rx=0 tx=0).stop 2023-11-28T18:23:51.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.068+0000 7f56c8db8640 1 -- 172.21.15.105:0/974479355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56c40d1b90 msgr2=0x7f56c40cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:51.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.068+0000 7f56c8db8640 1 --2- 172.21.15.105:0/974479355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56c40d1b90 0x7f56c40cb1b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f56b400e8a0 tx=0x7f56b400eca0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.068+0000 7f56c8db8640 1 -- 172.21.15.105:0/974479355 shutdown_connections 2023-11-28T18:23:51.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.068+0000 7f56c8db8640 1 --2- 172.21.15.105:0/974479355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56c415d300 0x7f56c4154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.068+0000 7f56c8db8640 1 --2- 172.21.15.105:0/974479355 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56c40d3ce0 0x7f56c41603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.068+0000 7f56c8db8640 1 --2- 172.21.15.105:0/974479355 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56a40668d0 0x7f56a4068d90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.068+0000 7f56c8db8640 1 --2- 172.21.15.105:0/974479355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56c40d1b90 0x7f56c40cb1b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.068+0000 7f56c8db8640 1 -- 172.21.15.105:0/974479355 >> 172.21.15.105:0/974479355 conn(0x7f56c409c360 msgr2=0x7f56c40c93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:51.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.068+0000 7f56c8db8640 1 -- 172.21.15.105:0/974479355 shutdown_connections 2023-11-28T18:23:51.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.068+0000 7f56c8db8640 1 -- 172.21.15.105:0/974479355 wait complete. 2023-11-28T18:23:51.092 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T18:23:51.092 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:23:51.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.223+0000 7f7137565640 1 Processor -- start 2023-11-28T18:23:51.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.227+0000 7f7137565640 1 -- start start 2023-11-28T18:23:51.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.227+0000 7f7137565640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f713015d4f0 0x7f71300d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:51.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.227+0000 7f7137565640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f71300b2d20 0x7f71300b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:51.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.227+0000 7f7137565640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71300b25b0 con 0x7f71300d3c30 2023-11-28T18:23:51.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.227+0000 7f7137565640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71300b2900 con 0x7f713015d4f0 2023-11-28T18:23:51.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.227+0000 7f7137565640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71300d0a20 con 0x7f71300b2d20 2023-11-28T18:23:51.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.227+0000 7f7135301640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f713015d4f0 0x7f71300d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:51.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.227+0000 7f7135301640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f713015d4f0 0x7f71300d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54818/0 (socket says 172.21.15.105:54818) 2023-11-28T18:23:51.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.227+0000 7f7135301640 1 -- 172.21.15.105:0/127043230 learned_addr learned my addr 172.21.15.105:0/127043230 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:51.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.227+0000 7f7135b02640 1 --2- 172.21.15.105:0/127043230 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f71300b2d20 0x7f71300b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:51.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.227+0000 7f7135301640 1 -- 172.21.15.105:0/127043230 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f71300b2d20 msgr2=0x7f71300b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:51.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.227+0000 7f7135301640 1 --2- 172.21.15.105:0/127043230 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f71300b2d20 0x7f71300b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.227+0000 7f7135301640 1 -- 172.21.15.105:0/127043230 >> v1:172.21.15.148:6789/0 conn(0x7f71300d3c30 legacy=0x7f71300d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:51.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.227+0000 7f7135301640 1 -- 172.21.15.105:0/127043230 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71300cea30 con 0x7f713015d4f0 2023-11-28T18:23:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.231+0000 7f7135301640 1 --2- 172.21.15.105:0/127043230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f713015d4f0 0x7f71300d0e10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f712c009870 tx=0x7f712c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ac6df5de1e0caa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.231+0000 7f7127fff640 1 -- 172.21.15.105:0/127043230 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c0ab680 con 0x7f713015d4f0 2023-11-28T18:23:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.231+0000 7f7127fff640 1 -- 172.21.15.105:0/127043230 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712c0ab7e0 con 0x7f713015d4f0 2023-11-28T18:23:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.231+0000 7f7127fff640 1 -- 172.21.15.105:0/127043230 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c0abb70 con 0x7f713015d4f0 2023-11-28T18:23:51.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.231+0000 7f7137565640 1 -- 172.21.15.105:0/127043230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f713015d4f0 msgr2=0x7f71300d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:51.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.231+0000 7f7137565640 1 --2- 172.21.15.105:0/127043230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f713015d4f0 0x7f71300d0e10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f712c009870 tx=0x7f712c01c480 comp rx=0 tx=0).stop 2023-11-28T18:23:51.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.231+0000 7f7137565640 1 -- 172.21.15.105:0/127043230 shutdown_connections 2023-11-28T18:23:51.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.231+0000 7f7137565640 1 --2- 172.21.15.105:0/127043230 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f71300b2d20 0x7f71300b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.231+0000 7f7137565640 1 --2- 172.21.15.105:0/127043230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f713015d4f0 0x7f71300d0e10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.231+0000 7f7137565640 1 -- 172.21.15.105:0/127043230 >> 172.21.15.105:0/127043230 conn(0x7f71300d3030 msgr2=0x7f71300d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:51.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.231+0000 7f7137565640 1 -- 172.21.15.105:0/127043230 shutdown_connections 2023-11-28T18:23:51.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.231+0000 7f7137565640 1 -- 172.21.15.105:0/127043230 wait complete. 2023-11-28T18:23:51.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.231+0000 7f7137565640 1 Processor -- start 2023-11-28T18:23:51.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.231+0000 7f7137565640 1 -- start start 2023-11-28T18:23:51.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7137565640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71300b2d20 0x7f713014bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:51.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7137565640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71300d3c30 0x7f71300cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:51.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7137565640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f713015d4f0 0x7f7130159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:51.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7137565640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71300ccf10 con 0x7f71300b2d20 2023-11-28T18:23:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7137565640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71300cc950 con 0x7f71300d3c30 2023-11-28T18:23:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7137565640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71300ccc30 con 0x7f713015d4f0 2023-11-28T18:23:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7135301640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71300b2d20 0x7f713014bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7135301640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71300b2d20 0x7f713014bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54820/0 (socket says 172.21.15.105:54820) 2023-11-28T18:23:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7135301640 1 -- 172.21.15.105:0/1962753942 learned_addr learned my addr 172.21.15.105:0/1962753942 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7135b02640 1 --2- 172.21.15.105:0/1962753942 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f713015d4f0 0x7f7130159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7134b00640 1 --2- 172.21.15.105:0/1962753942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71300d3c30 0x7f71300cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7135301640 1 -- 172.21.15.105:0/1962753942 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f713015d4f0 msgr2=0x7f7130159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7135301640 1 --2- 172.21.15.105:0/1962753942 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f713015d4f0 0x7f7130159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7135301640 1 -- 172.21.15.105:0/1962753942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71300d3c30 msgr2=0x7f71300cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7135301640 1 --2- 172.21.15.105:0/1962753942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71300d3c30 0x7f71300cbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7135301640 1 -- 172.21.15.105:0/1962753942 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712c0094d0 con 0x7f71300b2d20 2023-11-28T18:23:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7135301640 1 --2- 172.21.15.105:0/1962753942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71300b2d20 0x7f713014bda0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f712c0ad040 tx=0x7f712c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7125ffb640 1 -- 172.21.15.105:0/1962753942 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c0b4800 con 0x7f71300b2d20 2023-11-28T18:23:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7125ffb640 1 -- 172.21.15.105:0/1962753942 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712c0b4980 con 0x7f71300b2d20 2023-11-28T18:23:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7125ffb640 1 -- 172.21.15.105:0/1962753942 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c0b4d30 con 0x7f71300b2d20 2023-11-28T18:23:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7137565640 1 -- 172.21.15.105:0/1962753942 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7130156ec0 con 0x7f71300b2d20 2023-11-28T18:23:51.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7137565640 1 -- 172.21.15.105:0/1962753942 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713014c2e0 con 0x7f71300b2d20 2023-11-28T18:23:51.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.235+0000 7f7137565640 1 -- 172.21.15.105:0/1962753942 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7130067570 con 0x7f71300b2d20 2023-11-28T18:23:51.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.239+0000 7f7125ffb640 1 -- 172.21.15.105:0/1962753942 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f712c0c7440 con 0x7f71300b2d20 2023-11-28T18:23:51.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.239+0000 7f7125ffb640 1 --2- 172.21.15.105:0/1962753942 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f71140668d0 0x7f7114068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:51.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.239+0000 7f7125ffb640 1 -- 172.21.15.105:0/1962753942 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f712c11f5d0 con 0x7f71300b2d20 2023-11-28T18:23:51.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.239+0000 7f7134b00640 1 --2- 172.21.15.105:0/1962753942 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f71140668d0 0x7f7114068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:51.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.239+0000 7f7134b00640 1 --2- 172.21.15.105:0/1962753942 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f71140668d0 0x7f7114068d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f71300c9e40 tx=0x7f7120009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:51.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.243+0000 7f7125ffb640 1 -- 172.21.15.105:0/1962753942 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f712c0ec5b0 con 0x7f71300b2d20 2023-11-28T18:23:51.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.427+0000 7f7137565640 1 -- 172.21.15.105:0/1962753942 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7130157370 con 0x7f71300b2d20 2023-11-28T18:23:51.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.427+0000 7f7125ffb640 1 -- 172.21.15.105:0/1962753942 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7f712c0efbd0 con 0x7f71300b2d20 2023-11-28T18:23:51.431 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:51.431 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/390529893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":390529893},{"type":"v1","addr":"172.21.15.148:6835","nonce":390529893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3924571887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3924571887},{"type":"v1","addr":"172.21.15.148:6833","nonce":3924571887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3271971772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3271971772},{"type":"v1","addr":"172.21.15.105:6835","nonce":3271971772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:23:33.288743+0000","modified":"2023-11-28T18:23:50.425976+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2504263238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2504263238},{"type":"v1","addr":"172.21.15.105:6837","nonce":2504263238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T18:23:51.431 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 9 2023-11-28T18:23:51.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.427+0000 7f7137565640 1 -- 172.21.15.105:0/1962753942 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f71140668d0 msgr2=0x7f7114068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:51.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.427+0000 7f7137565640 1 --2- 172.21.15.105:0/1962753942 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f71140668d0 0x7f7114068d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f71300c9e40 tx=0x7f7120009040 comp rx=0 tx=0).stop 2023-11-28T18:23:51.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.427+0000 7f7137565640 1 -- 172.21.15.105:0/1962753942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71300b2d20 msgr2=0x7f713014bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:51.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.427+0000 7f7137565640 1 --2- 172.21.15.105:0/1962753942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71300b2d20 0x7f713014bda0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f712c0ad040 tx=0x7f712c01acc0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.431+0000 7f7137565640 1 -- 172.21.15.105:0/1962753942 shutdown_connections 2023-11-28T18:23:51.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.431+0000 7f7137565640 1 --2- 172.21.15.105:0/1962753942 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f713015d4f0 0x7f7130159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.431+0000 7f7137565640 1 --2- 172.21.15.105:0/1962753942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71300d3c30 0x7f71300cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.431+0000 7f7137565640 1 --2- 172.21.15.105:0/1962753942 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f71140668d0 0x7f7114068d90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.431+0000 7f7137565640 1 --2- 172.21.15.105:0/1962753942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71300b2d20 0x7f713014bda0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.431+0000 7f7137565640 1 -- 172.21.15.105:0/1962753942 >> 172.21.15.105:0/1962753942 conn(0x7f71300d3030 msgr2=0x7f71300d1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:51.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.431+0000 7f7137565640 1 -- 172.21.15.105:0/1962753942 shutdown_connections 2023-11-28T18:23:51.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.431+0000 7f7137565640 1 -- 172.21.15.105:0/1962753942 wait complete. 2023-11-28T18:23:51.453 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:23:51.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.587+0000 7f42345d1640 1 Processor -- start 2023-11-28T18:23:51.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.587+0000 7f42345d1640 1 -- start start 2023-11-28T18:23:51.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.587+0000 7f42345d1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f422c071960 0x7f422c15f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:51.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.587+0000 7f42345d1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f422c15b030 0x7f422c0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:51.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.587+0000 7f42345d1640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422c0701d0 con 0x7f422c0b4290 2023-11-28T18:23:51.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.587+0000 7f42345d1640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f422c06df80 con 0x7f422c071960 2023-11-28T18:23:51.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.587+0000 7f42345d1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f422c071df0 con 0x7f422c15b030 2023-11-28T18:23:51.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.587+0000 7f4231b6c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f422c071960 0x7f422c15f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:51.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.587+0000 7f423236d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f422c0b4290 0x7f422c0b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39222/0 (socket says 172.21.15.105:39222) 2023-11-28T18:23:51.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.587+0000 7f423236d640 1 -- 172.21.15.105:0/2182675341 learned_addr learned my addr 172.21.15.105:0/2182675341 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:51.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.587+0000 7f4232b6e640 1 --2- 172.21.15.105:0/2182675341 >> v2:172.21.15.148:3300/0 conn(0x7f422c15b030 0x7f422c0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:51.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.587+0000 7f4231b6c640 1 -- 172.21.15.105:0/2182675341 >> v2:172.21.15.148:3300/0 conn(0x7f422c15b030 msgr2=0x7f422c0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:51.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.587+0000 7f4231b6c640 1 --2- 172.21.15.105:0/2182675341 >> v2:172.21.15.148:3300/0 conn(0x7f422c15b030 0x7f422c0709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.587+0000 7f4231b6c640 1 -- 172.21.15.105:0/2182675341 >> v1:172.21.15.105:6789/0 conn(0x7f422c0b4290 legacy=0x7f422c0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:51.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.587+0000 7f4231b6c640 1 -- 172.21.15.105:0/2182675341 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c0720d0 con 0x7f422c071960 2023-11-28T18:23:51.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.591+0000 7f4231b6c640 1 --2- 172.21.15.105:0/2182675341 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f422c071960 0x7f422c15f3b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f422c067c50 tx=0x7f421c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4df0814e9d2f11 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:51.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.591+0000 7f423136b640 1 -- 172.21.15.105:0/2182675341 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c0ab600 con 0x7f422c071960 2023-11-28T18:23:51.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.591+0000 7f423136b640 1 -- 172.21.15.105:0/2182675341 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421c0a7050 con 0x7f422c071960 2023-11-28T18:23:51.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.591+0000 7f423136b640 1 -- 172.21.15.105:0/2182675341 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c0ab980 con 0x7f422c071960 2023-11-28T18:23:51.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.591+0000 7f42345d1640 1 -- 172.21.15.105:0/2182675341 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f422c071960 msgr2=0x7f422c15f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:51.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.591+0000 7f42345d1640 1 --2- 172.21.15.105:0/2182675341 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f422c071960 0x7f422c15f3b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f422c067c50 tx=0x7f421c01c430 comp rx=0 tx=0).stop 2023-11-28T18:23:51.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.591+0000 7f42345d1640 1 -- 172.21.15.105:0/2182675341 shutdown_connections 2023-11-28T18:23:51.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.591+0000 7f42345d1640 1 --2- 172.21.15.105:0/2182675341 >> v2:172.21.15.148:3300/0 conn(0x7f422c15b030 0x7f422c0709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.591+0000 7f42345d1640 1 --2- 172.21.15.105:0/2182675341 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f422c071960 0x7f422c15f3b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.591+0000 7f42345d1640 1 -- 172.21.15.105:0/2182675341 >> 172.21.15.105:0/2182675341 conn(0x7f422c0ceb90 msgr2=0x7f422c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:51.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.591+0000 7f42345d1640 1 -- 172.21.15.105:0/2182675341 shutdown_connections 2023-11-28T18:23:51.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.591+0000 7f42345d1640 1 -- 172.21.15.105:0/2182675341 wait complete. 2023-11-28T18:23:51.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f42345d1640 1 Processor -- start 2023-11-28T18:23:51.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f42345d1640 1 -- start start 2023-11-28T18:23:51.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f42345d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f422c0b4290 0x7f422c158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:51.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f42345d1640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f422c15b030 0x7f422c153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:51.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f42345d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f422c1539c0 0x7f422c154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:51.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f42345d1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c15a530 con 0x7f422c0b4290 2023-11-28T18:23:51.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f42345d1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c09ad30 con 0x7f422c1539c0 2023-11-28T18:23:51.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f42345d1640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c09b010 con 0x7f422c15b030 2023-11-28T18:23:51.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f423236d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f422c0b4290 0x7f422c158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:51.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f423236d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f422c0b4290 0x7f422c158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54832/0 (socket says 172.21.15.105:54832) 2023-11-28T18:23:51.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f423236d640 1 -- 172.21.15.105:0/1610241413 learned_addr learned my addr 172.21.15.105:0/1610241413 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:51.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f4231b6c640 1 --2- 172.21.15.105:0/1610241413 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f422c15b030 0x7f422c153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:51.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f4232b6e640 1 --2- 172.21.15.105:0/1610241413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f422c1539c0 0x7f422c154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:51.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f423236d640 1 -- 172.21.15.105:0/1610241413 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f422c15b030 msgr2=0x7f422c153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f423236d640 1 --2- 172.21.15.105:0/1610241413 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f422c15b030 0x7f422c153f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f423236d640 1 -- 172.21.15.105:0/1610241413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f422c1539c0 msgr2=0x7f422c154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f423236d640 1 --2- 172.21.15.105:0/1610241413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f422c1539c0 0x7f422c154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f423236d640 1 -- 172.21.15.105:0/1610241413 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421c009420 con 0x7f422c0b4290 2023-11-28T18:23:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f423236d640 1 --2- 172.21.15.105:0/1610241413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f422c0b4290 0x7f422c158630 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f422800c500 tx=0x7f422800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f4222ffd640 1 -- 172.21.15.105:0/1610241413 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228005cf0 con 0x7f422c0b4290 2023-11-28T18:23:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f4222ffd640 1 -- 172.21.15.105:0/1610241413 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4228005e50 con 0x7f422c0b4290 2023-11-28T18:23:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f42345d1640 1 -- 172.21.15.105:0/1610241413 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422c09b2f0 con 0x7f422c0b4290 2023-11-28T18:23:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f4222ffd640 1 -- 172.21.15.105:0/1610241413 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42280072f0 con 0x7f422c0b4290 2023-11-28T18:23:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f42345d1640 1 -- 172.21.15.105:0/1610241413 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422c0b2bc0 con 0x7f422c0b4290 2023-11-28T18:23:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.595+0000 7f42345d1640 1 -- 172.21.15.105:0/1610241413 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f422c09ab90 con 0x7f422c0b4290 2023-11-28T18:23:51.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.599+0000 7f4222ffd640 1 -- 172.21.15.105:0/1610241413 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f4228007450 con 0x7f422c0b4290 2023-11-28T18:23:51.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.599+0000 7f4222ffd640 1 --2- 172.21.15.105:0/1610241413 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f42100668d0 0x7f4210068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:51.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.599+0000 7f4222ffd640 1 -- 172.21.15.105:0/1610241413 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4228083be0 con 0x7f422c0b4290 2023-11-28T18:23:51.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.599+0000 7f4231b6c640 1 --2- 172.21.15.105:0/1610241413 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f42100668d0 0x7f4210068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:51.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.599+0000 7f4231b6c640 1 --2- 172.21.15.105:0/1610241413 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f42100668d0 0x7f4210068d90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f422c073a70 tx=0x7f421c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:51.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.603+0000 7f4222ffd640 1 -- 172.21.15.105:0/1610241413 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f422c09ab90 con 0x7f422c0b4290 2023-11-28T18:23:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.755+0000 7f42345d1640 1 -- 172.21.15.105:0/1610241413 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f422c09ab90 con 0x7f422c0b4290 2023-11-28T18:23:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.755+0000 7f4222ffd640 1 -- 172.21.15.105:0/1610241413 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7f422c09ab90 con 0x7f422c0b4290 2023-11-28T18:23:51.758 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:51.758 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":24,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:23:38.458954+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:23:51.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.755+0000 7f42345d1640 1 -- 172.21.15.105:0/1610241413 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f42100668d0 msgr2=0x7f4210068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:51.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.755+0000 7f42345d1640 1 --2- 172.21.15.105:0/1610241413 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f42100668d0 0x7f4210068d90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f422c073a70 tx=0x7f421c008040 comp rx=0 tx=0).stop 2023-11-28T18:23:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.755+0000 7f42345d1640 1 -- 172.21.15.105:0/1610241413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f422c0b4290 msgr2=0x7f422c158630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.755+0000 7f42345d1640 1 --2- 172.21.15.105:0/1610241413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f422c0b4290 0x7f422c158630 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f422800c500 tx=0x7f422800c9c0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.755+0000 7f42345d1640 1 -- 172.21.15.105:0/1610241413 shutdown_connections 2023-11-28T18:23:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.755+0000 7f42345d1640 1 --2- 172.21.15.105:0/1610241413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f422c1539c0 0x7f422c154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.755+0000 7f42345d1640 1 --2- 172.21.15.105:0/1610241413 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f422c15b030 0x7f422c153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.755+0000 7f42345d1640 1 --2- 172.21.15.105:0/1610241413 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f42100668d0 0x7f4210068d90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.755+0000 7f42345d1640 1 --2- 172.21.15.105:0/1610241413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f422c0b4290 0x7f422c158630 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.755+0000 7f42345d1640 1 -- 172.21.15.105:0/1610241413 >> 172.21.15.105:0/1610241413 conn(0x7f422c0ceb90 msgr2=0x7f422c15f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.755+0000 7f42345d1640 1 -- 172.21.15.105:0/1610241413 shutdown_connections 2023-11-28T18:23:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.755+0000 7f42345d1640 1 -- 172.21.15.105:0/1610241413 wait complete. 2023-11-28T18:23:51.780 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T18:23:51.780 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:23:51.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.915+0000 7f70048fd640 1 Processor -- start 2023-11-28T18:23:51.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.915+0000 7f70048fd640 1 -- start start 2023-11-28T18:23:51.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.915+0000 7f70048fd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f700015cef0 0x7f70000d1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:51.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.915+0000 7f70048fd640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70000ce840 con 0x7f70000d1b40 2023-11-28T18:23:51.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.915+0000 7f70048fd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70000cd990 con 0x7f70000cfb30 2023-11-28T18:23:51.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.915+0000 7f70048fd640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70000b4780 con 0x7f700015cef0 2023-11-28T18:23:51.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.915+0000 7f6ff7fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f70000d1b40 0x7f70000cf420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39234/0 (socket says 172.21.15.105:39234) 2023-11-28T18:23:51.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.915+0000 7f6ff7fff640 1 -- 172.21.15.105:0/1186545977 learned_addr learned my addr 172.21.15.105:0/1186545977 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:51.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.915+0000 7f6ffed9d640 1 --2- 172.21.15.105:0/1186545977 >> v2:172.21.15.105:3300/0 conn(0x7f700015cef0 0x7f70000d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:51.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.915+0000 7f6ffe59c640 1 -- 172.21.15.105:0/1186545977 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3030999847 0 0) 0x7f70000ce840 con 0x7f70000d1b40 2023-11-28T18:23:51.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.915+0000 7f6ffe59c640 1 -- 172.21.15.105:0/1186545977 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fec003530 con 0x7f70000d1b40 2023-11-28T18:23:51.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.915+0000 7f6ffed9d640 1 -- 172.21.15.105:0/1186545977 >> v1:172.21.15.148:6789/0 conn(0x7f70000cfb30 legacy=0x7f7000166fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:51.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.915+0000 7f6ffed9d640 1 -- 172.21.15.105:0/1186545977 >> v1:172.21.15.105:6789/0 conn(0x7f70000d1b40 legacy=0x7f70000cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:51.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.915+0000 7f6ffed9d640 1 -- 172.21.15.105:0/1186545977 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70000cecd0 con 0x7f700015cef0 2023-11-28T18:23:51.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.919+0000 7f6ffed9d640 1 --2- 172.21.15.105:0/1186545977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f700015cef0 0x7f70000d1600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00097c0 tx=0x7f6ff001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1713dcf6d5d09db9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:51.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.919+0000 7f6ffe59c640 1 -- 172.21.15.105:0/1186545977 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff00ac620 con 0x7f700015cef0 2023-11-28T18:23:51.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.919+0000 7f6ffe59c640 1 -- 172.21.15.105:0/1186545977 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff00ac780 con 0x7f700015cef0 2023-11-28T18:23:51.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.919+0000 7f6ffe59c640 1 -- 172.21.15.105:0/1186545977 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff00acaf0 con 0x7f700015cef0 2023-11-28T18:23:51.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.919+0000 7f70048fd640 1 -- 172.21.15.105:0/1186545977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f700015cef0 msgr2=0x7f70000d1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:51.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.919+0000 7f70048fd640 1 --2- 172.21.15.105:0/1186545977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f700015cef0 0x7f70000d1600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00097c0 tx=0x7f6ff001c3c0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.919+0000 7f70048fd640 1 -- 172.21.15.105:0/1186545977 shutdown_connections 2023-11-28T18:23:51.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.919+0000 7f70048fd640 1 --2- 172.21.15.105:0/1186545977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f700015cef0 0x7f70000d1600 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.919+0000 7f70048fd640 1 -- 172.21.15.105:0/1186545977 >> 172.21.15.105:0/1186545977 conn(0x7f70000d2e10 msgr2=0x7f70000d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:51.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.919+0000 7f70048fd640 1 -- 172.21.15.105:0/1186545977 shutdown_connections 2023-11-28T18:23:51.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.919+0000 7f70048fd640 1 -- 172.21.15.105:0/1186545977 wait complete. 2023-11-28T18:23:51.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.919+0000 7f70048fd640 1 Processor -- start 2023-11-28T18:23:51.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f70048fd640 1 -- start start 2023-11-28T18:23:51.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f70048fd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70000cfb30 0x7f7000152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:51.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f70048fd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70000d1b40 0x7f70000cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:51.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f70048fd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f700015cef0 0x7f70000c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f70048fd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70000cbae0 con 0x7f70000cfb30 2023-11-28T18:23:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f70048fd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70001538d0 con 0x7f700015cef0 2023-11-28T18:23:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f70048fd640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70000cc360 con 0x7f70000d1b40 2023-11-28T18:23:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f6ffed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70000cfb30 0x7f7000152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f6ffed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70000cfb30 0x7f7000152a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54856/0 (socket says 172.21.15.105:54856) 2023-11-28T18:23:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f6ffed9d640 1 -- 172.21.15.105:0/3638379032 learned_addr learned my addr 172.21.15.105:0/3638379032 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f6ff7fff640 1 --2- 172.21.15.105:0/3638379032 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70000d1b40 0x7f70000cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:51.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f6fff59e640 1 --2- 172.21.15.105:0/3638379032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f700015cef0 0x7f70000c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:51.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f6ffed9d640 1 -- 172.21.15.105:0/3638379032 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70000d1b40 msgr2=0x7f70000cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:51.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f6ffed9d640 1 --2- 172.21.15.105:0/3638379032 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70000d1b40 0x7f70000cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f6ffed9d640 1 -- 172.21.15.105:0/3638379032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f700015cef0 msgr2=0x7f70000c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:51.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f6ffed9d640 1 --2- 172.21.15.105:0/3638379032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f700015cef0 0x7f70000c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:51.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f6ffed9d640 1 -- 172.21.15.105:0/3638379032 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff0009420 con 0x7f70000cfb30 2023-11-28T18:23:51.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f6fff59e640 1 --2- 172.21.15.105:0/3638379032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f700015cef0 0x7f70000c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:51.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f6ffed9d640 1 --2- 172.21.15.105:0/3638379032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70000cfb30 0x7f7000152a70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0019ff0 tx=0x7f6ff0019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:51.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f6ff77fe640 1 -- 172.21.15.105:0/3638379032 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff00ac620 con 0x7f70000cfb30 2023-11-28T18:23:51.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f6ff77fe640 1 -- 172.21.15.105:0/3638379032 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff0004030 con 0x7f70000cfb30 2023-11-28T18:23:51.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f6ff77fe640 1 -- 172.21.15.105:0/3638379032 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff00043a0 con 0x7f70000cfb30 2023-11-28T18:23:51.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f70048fd640 1 -- 172.21.15.105:0/3638379032 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7000152fb0 con 0x7f70000cfb30 2023-11-28T18:23:51.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f70048fd640 1 -- 172.21.15.105:0/3638379032 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70001535f0 con 0x7f70000cfb30 2023-11-28T18:23:51.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.923+0000 7f70048fd640 1 -- 172.21.15.105:0/3638379032 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70000d03c0 con 0x7f70000cfb30 2023-11-28T18:23:51.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.927+0000 7f6ff77fe640 1 -- 172.21.15.105:0/3638379032 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f6ff00b3040 con 0x7f70000cfb30 2023-11-28T18:23:51.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.927+0000 7f6ff77fe640 1 --2- 172.21.15.105:0/3638379032 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6fe00668d0 0x7f6fe0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:51.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.927+0000 7f6ff77fe640 1 -- 172.21.15.105:0/3638379032 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6ff011e530 con 0x7f70000cfb30 2023-11-28T18:23:51.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.927+0000 7f6ff7fff640 1 --2- 172.21.15.105:0/3638379032 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6fe00668d0 0x7f6fe0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:51.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.927+0000 7f6ff7fff640 1 --2- 172.21.15.105:0/3638379032 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6fe00668d0 0x7f6fe0068d90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7000168080 tx=0x7f6fe8007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:51.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:51.931+0000 7f6ff77fe640 1 -- 172.21.15.105:0/3638379032 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6ff00eb520 con 0x7f70000cfb30 2023-11-28T18:23:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.079+0000 7f70048fd640 1 -- 172.21.15.105:0/3638379032 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f70000d3bc0 con 0x7f6fe00668d0 2023-11-28T18:23:52.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.091+0000 7f6ff77fe640 1 -- 172.21.15.105:0/3638379032 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345294 (secure 0 0 0) 0x7f6fe8001510 con 0x7f6fe00668d0 2023-11-28T18:23:52.098 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:52.100 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:23:52.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.095+0000 7f70048fd640 1 -- 172.21.15.105:0/3638379032 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6fe00668d0 msgr2=0x7f6fe0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:52.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.095+0000 7f70048fd640 1 --2- 172.21.15.105:0/3638379032 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6fe00668d0 0x7f6fe0068d90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7000168080 tx=0x7f6fe8007d40 comp rx=0 tx=0).stop 2023-11-28T18:23:52.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.095+0000 7f70048fd640 1 -- 172.21.15.105:0/3638379032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70000cfb30 msgr2=0x7f7000152a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:52.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.095+0000 7f70048fd640 1 --2- 172.21.15.105:0/3638379032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70000cfb30 0x7f7000152a70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0019ff0 tx=0x7f6ff0019160 comp rx=0 tx=0).stop 2023-11-28T18:23:52.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.095+0000 7f70048fd640 1 -- 172.21.15.105:0/3638379032 shutdown_connections 2023-11-28T18:23:52.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.095+0000 7f70048fd640 1 --2- 172.21.15.105:0/3638379032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f700015cef0 0x7f70000c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.095+0000 7f70048fd640 1 --2- 172.21.15.105:0/3638379032 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70000d1b40 0x7f70000cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.095+0000 7f70048fd640 1 --2- 172.21.15.105:0/3638379032 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6fe00668d0 0x7f6fe0068d90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.095+0000 7f70048fd640 1 --2- 172.21.15.105:0/3638379032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70000cfb30 0x7f7000152a70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.095+0000 7f70048fd640 1 -- 172.21.15.105:0/3638379032 >> 172.21.15.105:0/3638379032 conn(0x7f70000d2e10 msgr2=0x7f700006c350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:52.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.095+0000 7f70048fd640 1 -- 172.21.15.105:0/3638379032 shutdown_connections 2023-11-28T18:23:52.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.095+0000 7f70048fd640 1 -- 172.21.15.105:0/3638379032 wait complete. 2023-11-28T18:23:52.117 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2023-11-28T18:23:51.272635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594866,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219380,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755340,"statfs":{"total":773094113280,"available":772869468160,"internally_reserved":0,"allocated":3284992,"data_stored":1988068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2088,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002299"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.076931+0000","last_change":"2023-11-28T18:23:32.930693+0000","last_active":"2023-11-28T18:23:38.076931+0000","last_peered":"2023-11-28T18:23:38.076931+0000","last_clean":"2023-11-28T18:23:38.076931+0000","last_became_active":"2023-11-28T18:23:32.930410+0000","last_became_peered":"2023-11-28T18:23:32.930410+0000","last_unstale":"2023-11-28T18:23:38.076931+0000","last_undegraded":"2023-11-28T18:23:38.076931+0000","last_fullsized":"2023-11-28T18:23:38.076931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:19.143213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.066696+0000","last_change":"2023-11-28T18:23:33.947291+0000","last_active":"2023-11-28T18:23:38.066696+0000","last_peered":"2023-11-28T18:23:38.066696+0000","last_clean":"2023-11-28T18:23:38.066696+0000","last_became_active":"2023-11-28T18:23:33.946963+0000","last_became_peered":"2023-11-28T18:23:33.946963+0000","last_unstale":"2023-11-28T18:23:38.066696+0000","last_undegraded":"2023-11-28T18:23:38.066696+0000","last_fullsized":"2023-11-28T18:23:38.066696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:27.506820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.067797+0000","last_change":"2023-11-28T18:23:38.425549+0000","last_active":"2023-11-28T18:23:39.067797+0000","last_peered":"2023-11-28T18:23:39.067797+0000","last_clean":"2023-11-28T18:23:39.067797+0000","last_became_active":"2023-11-28T18:23:38.425130+0000","last_became_peered":"2023-11-28T18:23:38.425130+0000","last_unstale":"2023-11-28T18:23:39.067797+0000","last_undegraded":"2023-11-28T18:23:39.067797+0000","last_fullsized":"2023-11-28T18:23:39.067797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:19:51.686256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.066904+0000","last_change":"2023-11-28T18:23:32.921328+0000","last_active":"2023-11-28T18:23:38.066904+0000","last_peered":"2023-11-28T18:23:38.066904+0000","last_clean":"2023-11-28T18:23:38.066904+0000","last_became_active":"2023-11-28T18:23:32.920775+0000","last_became_peered":"2023-11-28T18:23:32.920775+0000","last_unstale":"2023-11-28T18:23:38.066904+0000","last_undegraded":"2023-11-28T18:23:38.066904+0000","last_fullsized":"2023-11-28T18:23:38.066904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:05:45.779946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075403+0000","last_change":"2023-11-28T18:23:33.948349+0000","last_active":"2023-11-28T18:23:38.075403+0000","last_peered":"2023-11-28T18:23:38.075403+0000","last_clean":"2023-11-28T18:23:38.075403+0000","last_became_active":"2023-11-28T18:23:33.947987+0000","last_became_peered":"2023-11-28T18:23:33.947987+0000","last_unstale":"2023-11-28T18:23:38.075403+0000","last_undegraded":"2023-11-28T18:23:38.075403+0000","last_fullsized":"2023-11-28T18:23:38.075403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:35.484199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068226+0000","last_change":"2023-11-28T18:23:38.091059+0000","last_active":"2023-11-28T18:23:39.068226+0000","last_peered":"2023-11-28T18:23:39.068226+0000","last_clean":"2023-11-28T18:23:39.068226+0000","last_became_active":"2023-11-28T18:23:38.090259+0000","last_became_peered":"2023-11-28T18:23:38.090259+0000","last_unstale":"2023-11-28T18:23:39.068226+0000","last_undegraded":"2023-11-28T18:23:39.068226+0000","last_fullsized":"2023-11-28T18:23:39.068226+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:05:00.278938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420356+0000","last_change":"2023-11-28T18:23:32.937363+0000","last_active":"2023-11-28T18:23:38.420356+0000","last_peered":"2023-11-28T18:23:38.420356+0000","last_clean":"2023-11-28T18:23:38.420356+0000","last_became_active":"2023-11-28T18:23:32.937125+0000","last_became_peered":"2023-11-28T18:23:32.937125+0000","last_unstale":"2023-11-28T18:23:38.420356+0000","last_undegraded":"2023-11-28T18:23:38.420356+0000","last_fullsized":"2023-11-28T18:23:38.420356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:45.143373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067755+0000","last_change":"2023-11-28T18:23:33.935572+0000","last_active":"2023-11-28T18:23:38.067755+0000","last_peered":"2023-11-28T18:23:38.067755+0000","last_clean":"2023-11-28T18:23:38.067755+0000","last_became_active":"2023-11-28T18:23:33.935204+0000","last_became_peered":"2023-11-28T18:23:33.935204+0000","last_unstale":"2023-11-28T18:23:38.067755+0000","last_undegraded":"2023-11-28T18:23:38.067755+0000","last_fullsized":"2023-11-28T18:23:38.067755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:02.989816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068674+0000","last_change":"2023-11-28T18:23:38.096313+0000","last_active":"2023-11-28T18:23:39.068674+0000","last_peered":"2023-11-28T18:23:39.068674+0000","last_clean":"2023-11-28T18:23:39.068674+0000","last_became_active":"2023-11-28T18:23:38.095957+0000","last_became_peered":"2023-11-28T18:23:38.095957+0000","last_unstale":"2023-11-28T18:23:39.068674+0000","last_undegraded":"2023-11-28T18:23:39.068674+0000","last_fullsized":"2023-11-28T18:23:39.068674+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:07.150212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419858+0000","last_change":"2023-11-28T18:23:32.939798+0000","last_active":"2023-11-28T18:23:38.419858+0000","last_peered":"2023-11-28T18:23:38.419858+0000","last_clean":"2023-11-28T18:23:38.419858+0000","last_became_active":"2023-11-28T18:23:32.939450+0000","last_became_peered":"2023-11-28T18:23:32.939450+0000","last_unstale":"2023-11-28T18:23:38.419858+0000","last_undegraded":"2023-11-28T18:23:38.419858+0000","last_fullsized":"2023-11-28T18:23:38.419858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:55:26.908976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067829+0000","last_change":"2023-11-28T18:23:33.933973+0000","last_active":"2023-11-28T18:23:38.067829+0000","last_peered":"2023-11-28T18:23:38.067829+0000","last_clean":"2023-11-28T18:23:38.067829+0000","last_became_active":"2023-11-28T18:23:33.933692+0000","last_became_peered":"2023-11-28T18:23:33.933692+0000","last_unstale":"2023-11-28T18:23:38.067829+0000","last_undegraded":"2023-11-28T18:23:38.067829+0000","last_fullsized":"2023-11-28T18:23:38.067829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:38.277787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068388+0000","last_change":"2023-11-28T18:23:38.089272+0000","last_active":"2023-11-28T18:23:39.068388+0000","last_peered":"2023-11-28T18:23:39.068388+0000","last_clean":"2023-11-28T18:23:39.068388+0000","last_became_active":"2023-11-28T18:23:38.088843+0000","last_became_peered":"2023-11-28T18:23:38.088843+0000","last_unstale":"2023-11-28T18:23:39.068388+0000","last_undegraded":"2023-11-28T18:23:39.068388+0000","last_fullsized":"2023-11-28T18:23:39.068388+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:50.581936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.073981+0000","last_change":"2023-11-28T18:23:33.945002+0000","last_active":"2023-11-28T18:23:38.073981+0000","last_peered":"2023-11-28T18:23:38.073981+0000","last_clean":"2023-11-28T18:23:38.073981+0000","last_became_active":"2023-11-28T18:23:33.942488+0000","last_became_peered":"2023-11-28T18:23:33.942488+0000","last_unstale":"2023-11-28T18:23:38.073981+0000","last_undegraded":"2023-11-28T18:23:38.073981+0000","last_fullsized":"2023-11-28T18:23:38.073981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:04:56.118542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419933+0000","last_change":"2023-11-28T18:23:32.938369+0000","last_active":"2023-11-28T18:23:38.419933+0000","last_peered":"2023-11-28T18:23:38.419933+0000","last_clean":"2023-11-28T18:23:38.419933+0000","last_became_active":"2023-11-28T18:23:32.938126+0000","last_became_peered":"2023-11-28T18:23:32.938126+0000","last_unstale":"2023-11-28T18:23:38.419933+0000","last_undegraded":"2023-11-28T18:23:38.419933+0000","last_fullsized":"2023-11-28T18:23:38.419933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:50.646527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.069187+0000","last_change":"2023-11-28T18:23:38.093734+0000","last_active":"2023-11-28T18:23:39.069187+0000","last_peered":"2023-11-28T18:23:39.069187+0000","last_clean":"2023-11-28T18:23:39.069187+0000","last_became_active":"2023-11-28T18:23:38.093298+0000","last_became_peered":"2023-11-28T18:23:38.093298+0000","last_unstale":"2023-11-28T18:23:39.069187+0000","last_undegraded":"2023-11-28T18:23:39.069187+0000","last_fullsized":"2023-11-28T18:23:39.069187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:22:41.524911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065575+0000","last_change":"2023-11-28T18:23:33.938245+0000","last_active":"2023-11-28T18:23:38.065575+0000","last_peered":"2023-11-28T18:23:38.065575+0000","last_clean":"2023-11-28T18:23:38.065575+0000","last_became_active":"2023-11-28T18:23:33.937832+0000","last_became_peered":"2023-11-28T18:23:33.937832+0000","last_unstale":"2023-11-28T18:23:38.065575+0000","last_undegraded":"2023-11-28T18:23:38.065575+0000","last_fullsized":"2023-11-28T18:23:38.065575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:04:56.118542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.093039+0000","last_change":"2023-11-28T18:23:32.924880+0000","last_active":"2023-11-28T18:23:38.093039+0000","last_peered":"2023-11-28T18:23:38.093039+0000","last_clean":"2023-11-28T18:23:38.093039+0000","last_became_active":"2023-11-28T18:23:32.923804+0000","last_became_peered":"2023-11-28T18:23:32.923804+0000","last_unstale":"2023-11-28T18:23:38.093039+0000","last_undegraded":"2023-11-28T18:23:38.093039+0000","last_fullsized":"2023-11-28T18:23:38.093039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:10.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068797+0000","last_change":"2023-11-28T18:23:38.091628+0000","last_active":"2023-11-28T18:23:39.068797+0000","last_peered":"2023-11-28T18:23:39.068797+0000","last_clean":"2023-11-28T18:23:39.068797+0000","last_became_active":"2023-11-28T18:23:38.090900+0000","last_became_peered":"2023-11-28T18:23:38.090900+0000","last_unstale":"2023-11-28T18:23:39.068797+0000","last_undegraded":"2023-11-28T18:23:39.068797+0000","last_fullsized":"2023-11-28T18:23:39.068797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:47.152718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065511+0000","last_change":"2023-11-28T18:23:33.933391+0000","last_active":"2023-11-28T18:23:38.065511+0000","last_peered":"2023-11-28T18:23:38.065511+0000","last_clean":"2023-11-28T18:23:38.065511+0000","last_became_active":"2023-11-28T18:23:33.932767+0000","last_became_peered":"2023-11-28T18:23:33.932767+0000","last_unstale":"2023-11-28T18:23:38.065511+0000","last_undegraded":"2023-11-28T18:23:38.065511+0000","last_fullsized":"2023-11-28T18:23:38.065511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:21.035422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.092828+0000","last_change":"2023-11-28T18:23:32.929281+0000","last_active":"2023-11-28T18:23:38.092828+0000","last_peered":"2023-11-28T18:23:38.092828+0000","last_clean":"2023-11-28T18:23:38.092828+0000","last_became_active":"2023-11-28T18:23:32.928997+0000","last_became_peered":"2023-11-28T18:23:32.928997+0000","last_unstale":"2023-11-28T18:23:38.092828+0000","last_undegraded":"2023-11-28T18:23:38.092828+0000","last_fullsized":"2023-11-28T18:23:38.092828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:45.143373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.718883+0000","last_change":"2023-11-28T18:23:38.426592+0000","last_active":"2023-11-28T18:23:40.718883+0000","last_peered":"2023-11-28T18:23:40.718883+0000","last_clean":"2023-11-28T18:23:40.718883+0000","last_became_active":"2023-11-28T18:23:38.426089+0000","last_became_peered":"2023-11-28T18:23:38.426089+0000","last_unstale":"2023-11-28T18:23:40.718883+0000","last_undegraded":"2023-11-28T18:23:40.718883+0000","last_fullsized":"2023-11-28T18:23:40.718883+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:15.599832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075007+0000","last_change":"2023-11-28T18:23:33.935238+0000","last_active":"2023-11-28T18:23:38.075007+0000","last_peered":"2023-11-28T18:23:38.075007+0000","last_clean":"2023-11-28T18:23:38.075007+0000","last_became_active":"2023-11-28T18:23:33.934884+0000","last_became_peered":"2023-11-28T18:23:33.934884+0000","last_unstale":"2023-11-28T18:23:38.075007+0000","last_undegraded":"2023-11-28T18:23:38.075007+0000","last_fullsized":"2023-11-28T18:23:38.075007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:58.099391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419489+0000","last_change":"2023-11-28T18:23:32.933485+0000","last_active":"2023-11-28T18:23:38.419489+0000","last_peered":"2023-11-28T18:23:38.419489+0000","last_clean":"2023-11-28T18:23:38.419489+0000","last_became_active":"2023-11-28T18:23:32.932807+0000","last_became_peered":"2023-11-28T18:23:32.932807+0000","last_unstale":"2023-11-28T18:23:38.419489+0000","last_undegraded":"2023-11-28T18:23:38.419489+0000","last_fullsized":"2023-11-28T18:23:38.419489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:00.325734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.069240+0000","last_change":"2023-11-28T18:23:38.091553+0000","last_active":"2023-11-28T18:23:39.069240+0000","last_peered":"2023-11-28T18:23:39.069240+0000","last_clean":"2023-11-28T18:23:39.069240+0000","last_became_active":"2023-11-28T18:23:38.091133+0000","last_became_peered":"2023-11-28T18:23:38.091133+0000","last_unstale":"2023-11-28T18:23:39.069240+0000","last_undegraded":"2023-11-28T18:23:39.069240+0000","last_fullsized":"2023-11-28T18:23:39.069240+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:53.274366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.080431+0000","last_change":"2023-11-28T18:23:33.941025+0000","last_active":"2023-11-28T18:23:38.080431+0000","last_peered":"2023-11-28T18:23:38.080431+0000","last_clean":"2023-11-28T18:23:38.080431+0000","last_became_active":"2023-11-28T18:23:33.940714+0000","last_became_peered":"2023-11-28T18:23:33.940714+0000","last_unstale":"2023-11-28T18:23:38.080431+0000","last_undegraded":"2023-11-28T18:23:38.080431+0000","last_fullsized":"2023-11-28T18:23:38.080431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:01.912695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077789+0000","last_change":"2023-11-28T18:23:32.933239+0000","last_active":"2023-11-28T18:23:38.077789+0000","last_peered":"2023-11-28T18:23:38.077789+0000","last_clean":"2023-11-28T18:23:38.077789+0000","last_became_active":"2023-11-28T18:23:32.932935+0000","last_became_peered":"2023-11-28T18:23:32.932935+0000","last_unstale":"2023-11-28T18:23:38.077789+0000","last_undegraded":"2023-11-28T18:23:38.077789+0000","last_fullsized":"2023-11-28T18:23:38.077789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:36:09.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068440+0000","last_change":"2023-11-28T18:23:38.421858+0000","last_active":"2023-11-28T18:23:39.068440+0000","last_peered":"2023-11-28T18:23:39.068440+0000","last_clean":"2023-11-28T18:23:39.068440+0000","last_became_active":"2023-11-28T18:23:38.421416+0000","last_became_peered":"2023-11-28T18:23:38.421416+0000","last_unstale":"2023-11-28T18:23:39.068440+0000","last_undegraded":"2023-11-28T18:23:39.068440+0000","last_fullsized":"2023-11-28T18:23:39.068440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:31.667216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067068+0000","last_change":"2023-11-28T18:23:33.940147+0000","last_active":"2023-11-28T18:23:38.067068+0000","last_peered":"2023-11-28T18:23:38.067068+0000","last_clean":"2023-11-28T18:23:38.067068+0000","last_became_active":"2023-11-28T18:23:33.939882+0000","last_became_peered":"2023-11-28T18:23:33.939882+0000","last_unstale":"2023-11-28T18:23:38.067068+0000","last_undegraded":"2023-11-28T18:23:38.067068+0000","last_fullsized":"2023-11-28T18:23:38.067068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:02.989816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.096365+0000","last_change":"2023-11-28T18:23:32.925741+0000","last_active":"2023-11-28T18:23:38.096365+0000","last_peered":"2023-11-28T18:23:38.096365+0000","last_clean":"2023-11-28T18:23:38.096365+0000","last_became_active":"2023-11-28T18:23:32.925344+0000","last_became_peered":"2023-11-28T18:23:32.925344+0000","last_unstale":"2023-11-28T18:23:38.096365+0000","last_undegraded":"2023-11-28T18:23:38.096365+0000","last_fullsized":"2023-11-28T18:23:38.096365+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:41.820325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.067516+0000","last_change":"2023-11-28T18:23:38.421716+0000","last_active":"2023-11-28T18:23:39.067516+0000","last_peered":"2023-11-28T18:23:39.067516+0000","last_clean":"2023-11-28T18:23:39.067516+0000","last_became_active":"2023-11-28T18:23:38.421313+0000","last_became_peered":"2023-11-28T18:23:38.421313+0000","last_unstale":"2023-11-28T18:23:39.067516+0000","last_undegraded":"2023-11-28T18:23:39.067516+0000","last_fullsized":"2023-11-28T18:23:39.067516+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:12:36.678267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.094519+0000","last_change":"2023-11-28T18:23:33.934187+0000","last_active":"2023-11-28T18:23:38.094519+0000","last_peered":"2023-11-28T18:23:38.094519+0000","last_clean":"2023-11-28T18:23:38.094519+0000","last_became_active":"2023-11-28T18:23:33.933885+0000","last_became_peered":"2023-11-28T18:23:33.933885+0000","last_unstale":"2023-11-28T18:23:38.094519+0000","last_undegraded":"2023-11-28T18:23:38.094519+0000","last_fullsized":"2023-11-28T18:23:38.094519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:21.035422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420442+0000","last_change":"2023-11-28T18:23:32.933434+0000","last_active":"2023-11-28T18:23:38.420442+0000","last_peered":"2023-11-28T18:23:38.420442+0000","last_clean":"2023-11-28T18:23:38.420442+0000","last_became_active":"2023-11-28T18:23:32.933143+0000","last_became_peered":"2023-11-28T18:23:32.933143+0000","last_unstale":"2023-11-28T18:23:38.420442+0000","last_undegraded":"2023-11-28T18:23:38.420442+0000","last_fullsized":"2023-11-28T18:23:38.420442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:11.551517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.297189+0000","last_change":"2023-11-28T18:23:38.422070+0000","last_active":"2023-11-28T18:23:40.297189+0000","last_peered":"2023-11-28T18:23:40.297189+0000","last_clean":"2023-11-28T18:23:40.297189+0000","last_became_active":"2023-11-28T18:23:38.421240+0000","last_became_peered":"2023-11-28T18:23:38.421240+0000","last_unstale":"2023-11-28T18:23:40.297189+0000","last_undegraded":"2023-11-28T18:23:40.297189+0000","last_fullsized":"2023-11-28T18:23:40.297189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:42.438182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067926+0000","last_change":"2023-11-28T18:23:33.932685+0000","last_active":"2023-11-28T18:23:38.067926+0000","last_peered":"2023-11-28T18:23:38.067926+0000","last_clean":"2023-11-28T18:23:38.067926+0000","last_became_active":"2023-11-28T18:23:33.932342+0000","last_became_peered":"2023-11-28T18:23:33.932342+0000","last_unstale":"2023-11-28T18:23:38.067926+0000","last_undegraded":"2023-11-28T18:23:38.067926+0000","last_fullsized":"2023-11-28T18:23:38.067926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:49.113970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.094608+0000","last_change":"2023-11-28T18:23:32.923234+0000","last_active":"2023-11-28T18:23:38.094608+0000","last_peered":"2023-11-28T18:23:38.094608+0000","last_clean":"2023-11-28T18:23:38.094608+0000","last_became_active":"2023-11-28T18:23:32.922051+0000","last_became_peered":"2023-11-28T18:23:32.922051+0000","last_unstale":"2023-11-28T18:23:38.094608+0000","last_undegraded":"2023-11-28T18:23:38.094608+0000","last_fullsized":"2023-11-28T18:23:38.094608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:36:09.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.718350+0000","last_change":"2023-11-28T18:23:38.102307+0000","last_active":"2023-11-28T18:23:40.718350+0000","last_peered":"2023-11-28T18:23:40.718350+0000","last_clean":"2023-11-28T18:23:40.718350+0000","last_became_active":"2023-11-28T18:23:38.101987+0000","last_became_peered":"2023-11-28T18:23:38.101987+0000","last_unstale":"2023-11-28T18:23:40.718350+0000","last_undegraded":"2023-11-28T18:23:40.718350+0000","last_fullsized":"2023-11-28T18:23:40.718350+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:22:41.524911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.076427+0000","last_change":"2023-11-28T18:23:33.941492+0000","last_active":"2023-11-28T18:23:38.076427+0000","last_peered":"2023-11-28T18:23:38.076427+0000","last_clean":"2023-11-28T18:23:38.076427+0000","last_became_active":"2023-11-28T18:23:33.941148+0000","last_became_peered":"2023-11-28T18:23:33.941148+0000","last_unstale":"2023-11-28T18:23:38.076427+0000","last_undegraded":"2023-11-28T18:23:38.076427+0000","last_fullsized":"2023-11-28T18:23:38.076427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:29:23.426289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068690+0000","last_change":"2023-11-28T18:23:32.922032+0000","last_active":"2023-11-28T18:23:38.068690+0000","last_peered":"2023-11-28T18:23:38.068690+0000","last_clean":"2023-11-28T18:23:38.068690+0000","last_became_active":"2023-11-28T18:23:32.921635+0000","last_became_peered":"2023-11-28T18:23:32.921635+0000","last_unstale":"2023-11-28T18:23:38.068690+0000","last_undegraded":"2023-11-28T18:23:38.068690+0000","last_fullsized":"2023-11-28T18:23:38.068690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:36:09.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.718745+0000","last_change":"2023-11-28T18:23:38.096609+0000","last_active":"2023-11-28T18:23:40.718745+0000","last_peered":"2023-11-28T18:23:40.718745+0000","last_clean":"2023-11-28T18:23:40.718745+0000","last_became_active":"2023-11-28T18:23:38.095758+0000","last_became_peered":"2023-11-28T18:23:38.095758+0000","last_unstale":"2023-11-28T18:23:40.718745+0000","last_undegraded":"2023-11-28T18:23:40.718745+0000","last_fullsized":"2023-11-28T18:23:40.718745+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:42.438182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.090061+0000","last_change":"2023-11-28T18:23:33.937266+0000","last_active":"2023-11-28T18:23:38.090061+0000","last_peered":"2023-11-28T18:23:38.090061+0000","last_clean":"2023-11-28T18:23:38.090061+0000","last_became_active":"2023-11-28T18:23:33.936994+0000","last_became_peered":"2023-11-28T18:23:33.936994+0000","last_unstale":"2023-11-28T18:23:38.090061+0000","last_undegraded":"2023-11-28T18:23:38.090061+0000","last_fullsized":"2023-11-28T18:23:38.090061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:12.557669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.083701+0000","last_change":"2023-11-28T18:23:32.918591+0000","last_active":"2023-11-28T18:23:38.083701+0000","last_peered":"2023-11-28T18:23:38.083701+0000","last_clean":"2023-11-28T18:23:38.083701+0000","last_became_active":"2023-11-28T18:23:32.915381+0000","last_became_peered":"2023-11-28T18:23:32.915381+0000","last_unstale":"2023-11-28T18:23:38.083701+0000","last_undegraded":"2023-11-28T18:23:38.083701+0000","last_fullsized":"2023-11-28T18:23:38.083701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:57.093239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.718851+0000","last_change":"2023-11-28T18:23:38.100344+0000","last_active":"2023-11-28T18:23:40.718851+0000","last_peered":"2023-11-28T18:23:40.718851+0000","last_clean":"2023-11-28T18:23:40.718851+0000","last_became_active":"2023-11-28T18:23:38.099934+0000","last_became_peered":"2023-11-28T18:23:38.099934+0000","last_unstale":"2023-11-28T18:23:40.718851+0000","last_undegraded":"2023-11-28T18:23:40.718851+0000","last_fullsized":"2023-11-28T18:23:40.718851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:39.149571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.074486+0000","last_change":"2023-11-28T18:23:33.944704+0000","last_active":"2023-11-28T18:23:38.074486+0000","last_peered":"2023-11-28T18:23:38.074486+0000","last_clean":"2023-11-28T18:23:38.074486+0000","last_became_active":"2023-11-28T18:23:33.944484+0000","last_became_peered":"2023-11-28T18:23:33.944484+0000","last_unstale":"2023-11-28T18:23:38.074486+0000","last_undegraded":"2023-11-28T18:23:38.074486+0000","last_fullsized":"2023-11-28T18:23:38.074486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:55:27.915128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419489+0000","last_change":"2023-11-28T18:23:32.933677+0000","last_active":"2023-11-28T18:23:38.419489+0000","last_peered":"2023-11-28T18:23:38.419489+0000","last_clean":"2023-11-28T18:23:38.419489+0000","last_became_active":"2023-11-28T18:23:32.933168+0000","last_became_peered":"2023-11-28T18:23:32.933168+0000","last_unstale":"2023-11-28T18:23:38.419489+0000","last_undegraded":"2023-11-28T18:23:38.419489+0000","last_fullsized":"2023-11-28T18:23:38.419489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:29:22.420137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.297576+0000","last_change":"2023-11-28T18:23:38.427376+0000","last_active":"2023-11-28T18:23:40.297576+0000","last_peered":"2023-11-28T18:23:40.297576+0000","last_clean":"2023-11-28T18:23:40.297576+0000","last_became_active":"2023-11-28T18:23:38.426436+0000","last_became_peered":"2023-11-28T18:23:38.426436+0000","last_unstale":"2023-11-28T18:23:40.297576+0000","last_undegraded":"2023-11-28T18:23:40.297576+0000","last_fullsized":"2023-11-28T18:23:40.297576+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:19:23.246961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065098+0000","last_change":"2023-11-28T18:23:33.932836+0000","last_active":"2023-11-28T18:23:38.065098+0000","last_peered":"2023-11-28T18:23:38.065098+0000","last_clean":"2023-11-28T18:23:38.065098+0000","last_became_active":"2023-11-28T18:23:33.932534+0000","last_became_peered":"2023-11-28T18:23:33.932534+0000","last_unstale":"2023-11-28T18:23:38.065098+0000","last_undegraded":"2023-11-28T18:23:38.065098+0000","last_fullsized":"2023-11-28T18:23:38.065098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:46.421541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067421+0000","last_change":"2023-11-28T18:23:32.922842+0000","last_active":"2023-11-28T18:23:38.067421+0000","last_peered":"2023-11-28T18:23:38.067421+0000","last_clean":"2023-11-28T18:23:38.067421+0000","last_became_active":"2023-11-28T18:23:32.922541+0000","last_became_peered":"2023-11-28T18:23:32.922541+0000","last_unstale":"2023-11-28T18:23:38.067421+0000","last_undegraded":"2023-11-28T18:23:38.067421+0000","last_fullsized":"2023-11-28T18:23:38.067421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:39:52.964875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419571+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:23:38.419571+0000","last_peered":"2023-11-28T18:23:38.419571+0000","last_clean":"2023-11-28T18:23:38.419571+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:23:38.419571+0000","last_undegraded":"2023-11-28T18:23:38.419571+0000","last_fullsized":"2023-11-28T18:23:38.419571+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.719264+0000","last_change":"2023-11-28T18:23:38.427043+0000","last_active":"2023-11-28T18:23:40.719264+0000","last_peered":"2023-11-28T18:23:40.719264+0000","last_clean":"2023-11-28T18:23:40.719264+0000","last_became_active":"2023-11-28T18:23:38.426613+0000","last_became_peered":"2023-11-28T18:23:38.426613+0000","last_unstale":"2023-11-28T18:23:40.719264+0000","last_undegraded":"2023-11-28T18:23:40.719264+0000","last_fullsized":"2023-11-28T18:23:40.719264+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:47.152718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419993+0000","last_change":"2023-11-28T18:23:33.944370+0000","last_active":"2023-11-28T18:23:38.419993+0000","last_peered":"2023-11-28T18:23:38.419993+0000","last_clean":"2023-11-28T18:23:38.419993+0000","last_became_active":"2023-11-28T18:23:33.943698+0000","last_became_peered":"2023-11-28T18:23:33.943698+0000","last_unstale":"2023-11-28T18:23:38.419993+0000","last_undegraded":"2023-11-28T18:23:38.419993+0000","last_fullsized":"2023-11-28T18:23:38.419993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:49.113970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.083176+0000","last_change":"2023-11-28T18:23:32.917731+0000","last_active":"2023-11-28T18:23:38.083176+0000","last_peered":"2023-11-28T18:23:38.083176+0000","last_clean":"2023-11-28T18:23:38.083176+0000","last_became_active":"2023-11-28T18:23:32.917207+0000","last_became_peered":"2023-11-28T18:23:32.917207+0000","last_unstale":"2023-11-28T18:23:38.083176+0000","last_undegraded":"2023-11-28T18:23:38.083176+0000","last_fullsized":"2023-11-28T18:23:38.083176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:38:50.132097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.719164+0000","last_change":"2023-11-28T18:23:38.425300+0000","last_active":"2023-11-28T18:23:40.719164+0000","last_peered":"2023-11-28T18:23:40.719164+0000","last_clean":"2023-11-28T18:23:40.719164+0000","last_became_active":"2023-11-28T18:23:38.424708+0000","last_became_peered":"2023-11-28T18:23:38.424708+0000","last_unstale":"2023-11-28T18:23:40.719164+0000","last_undegraded":"2023-11-28T18:23:40.719164+0000","last_fullsized":"2023-11-28T18:23:40.719164+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:07.150212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067866+0000","last_change":"2023-11-28T18:23:33.932266+0000","last_active":"2023-11-28T18:23:38.067866+0000","last_peered":"2023-11-28T18:23:38.067866+0000","last_clean":"2023-11-28T18:23:38.067866+0000","last_became_active":"2023-11-28T18:23:33.931923+0000","last_became_peered":"2023-11-28T18:23:33.931923+0000","last_unstale":"2023-11-28T18:23:38.067866+0000","last_undegraded":"2023-11-28T18:23:38.067866+0000","last_fullsized":"2023-11-28T18:23:38.067866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:34.989175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.094421+0000","last_change":"2023-11-28T18:23:32.924719+0000","last_active":"2023-11-28T18:23:38.094421+0000","last_peered":"2023-11-28T18:23:38.094421+0000","last_clean":"2023-11-28T18:23:38.094421+0000","last_became_active":"2023-11-28T18:23:32.924462+0000","last_became_peered":"2023-11-28T18:23:32.924462+0000","last_unstale":"2023-11-28T18:23:38.094421+0000","last_undegraded":"2023-11-28T18:23:38.094421+0000","last_fullsized":"2023-11-28T18:23:38.094421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:55:26.908976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.297354+0000","last_change":"2023-11-28T18:23:38.093163+0000","last_active":"2023-11-28T18:23:40.297354+0000","last_peered":"2023-11-28T18:23:40.297354+0000","last_clean":"2023-11-28T18:23:40.297354+0000","last_became_active":"2023-11-28T18:23:38.092754+0000","last_became_peered":"2023-11-28T18:23:38.092754+0000","last_unstale":"2023-11-28T18:23:40.297354+0000","last_undegraded":"2023-11-28T18:23:40.297354+0000","last_fullsized":"2023-11-28T18:23:40.297354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:39.149571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420505+0000","last_change":"2023-11-28T18:23:33.954652+0000","last_active":"2023-11-28T18:23:38.420505+0000","last_peered":"2023-11-28T18:23:38.420505+0000","last_clean":"2023-11-28T18:23:38.420505+0000","last_became_active":"2023-11-28T18:23:33.954369+0000","last_became_peered":"2023-11-28T18:23:33.954369+0000","last_unstale":"2023-11-28T18:23:38.420505+0000","last_undegraded":"2023-11-28T18:23:38.420505+0000","last_fullsized":"2023-11-28T18:23:38.420505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:30:56.523444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.084061+0000","last_change":"2023-11-28T18:23:32.906348+0000","last_active":"2023-11-28T18:23:38.084061+0000","last_peered":"2023-11-28T18:23:38.084061+0000","last_clean":"2023-11-28T18:23:38.084061+0000","last_became_active":"2023-11-28T18:23:32.906007+0000","last_became_peered":"2023-11-28T18:23:32.906007+0000","last_unstale":"2023-11-28T18:23:38.084061+0000","last_undegraded":"2023-11-28T18:23:38.084061+0000","last_fullsized":"2023-11-28T18:23:38.084061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:00.325734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.719440+0000","last_change":"2023-11-28T18:23:38.426316+0000","last_active":"2023-11-28T18:23:40.719440+0000","last_peered":"2023-11-28T18:23:40.719440+0000","last_clean":"2023-11-28T18:23:40.719440+0000","last_became_active":"2023-11-28T18:23:38.425328+0000","last_became_peered":"2023-11-28T18:23:38.425328+0000","last_unstale":"2023-11-28T18:23:40.719440+0000","last_undegraded":"2023-11-28T18:23:40.719440+0000","last_fullsized":"2023-11-28T18:23:40.719440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:19:51.686256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420058+0000","last_change":"2023-11-28T18:23:33.954315+0000","last_active":"2023-11-28T18:23:38.420058+0000","last_peered":"2023-11-28T18:23:38.420058+0000","last_clean":"2023-11-28T18:23:38.420058+0000","last_became_active":"2023-11-28T18:23:33.950270+0000","last_became_peered":"2023-11-28T18:23:33.950270+0000","last_unstale":"2023-11-28T18:23:38.420058+0000","last_undegraded":"2023-11-28T18:23:38.420058+0000","last_fullsized":"2023-11-28T18:23:38.420058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:22:37.364515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077169+0000","last_change":"2023-11-28T18:23:32.921145+0000","last_active":"2023-11-28T18:23:38.077169+0000","last_peered":"2023-11-28T18:23:38.077169+0000","last_clean":"2023-11-28T18:23:38.077169+0000","last_became_active":"2023-11-28T18:23:32.920706+0000","last_became_peered":"2023-11-28T18:23:32.920706+0000","last_unstale":"2023-11-28T18:23:38.077169+0000","last_undegraded":"2023-11-28T18:23:38.077169+0000","last_fullsized":"2023-11-28T18:23:38.077169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:39:52.964875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068294+0000","last_change":"2023-11-28T18:23:38.096089+0000","last_active":"2023-11-28T18:23:39.068294+0000","last_peered":"2023-11-28T18:23:39.068294+0000","last_clean":"2023-11-28T18:23:39.068294+0000","last_became_active":"2023-11-28T18:23:38.095672+0000","last_became_peered":"2023-11-28T18:23:38.095672+0000","last_unstale":"2023-11-28T18:23:39.068294+0000","last_undegraded":"2023-11-28T18:23:39.068294+0000","last_fullsized":"2023-11-28T18:23:39.068294+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:09.933678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419670+0000","last_change":"2023-11-28T18:23:33.954352+0000","last_active":"2023-11-28T18:23:38.419670+0000","last_peered":"2023-11-28T18:23:38.419670+0000","last_clean":"2023-11-28T18:23:38.419670+0000","last_became_active":"2023-11-28T18:23:33.950344+0000","last_became_peered":"2023-11-28T18:23:33.950344+0000","last_unstale":"2023-11-28T18:23:38.419670+0000","last_undegraded":"2023-11-28T18:23:38.419670+0000","last_fullsized":"2023-11-28T18:23:38.419670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:08.033209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.078193+0000","last_change":"2023-11-28T18:23:32.919741+0000","last_active":"2023-11-28T18:23:38.078193+0000","last_peered":"2023-11-28T18:23:38.078193+0000","last_clean":"2023-11-28T18:23:38.078193+0000","last_became_active":"2023-11-28T18:23:32.919438+0000","last_became_peered":"2023-11-28T18:23:32.919438+0000","last_unstale":"2023-11-28T18:23:38.078193+0000","last_undegraded":"2023-11-28T18:23:38.078193+0000","last_fullsized":"2023-11-28T18:23:38.078193+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:39:52.964875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.297438+0000","last_change":"2023-11-28T18:23:38.084984+0000","last_active":"2023-11-28T18:23:40.297438+0000","last_peered":"2023-11-28T18:23:40.297438+0000","last_clean":"2023-11-28T18:23:40.297438+0000","last_became_active":"2023-11-28T18:23:38.084490+0000","last_became_peered":"2023-11-28T18:23:38.084490+0000","last_unstale":"2023-11-28T18:23:40.297438+0000","last_undegraded":"2023-11-28T18:23:40.297438+0000","last_fullsized":"2023-11-28T18:23:40.297438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:00.683839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.073493+0000","last_change":"2023-11-28T18:23:33.939191+0000","last_active":"2023-11-28T18:23:38.073493+0000","last_peered":"2023-11-28T18:23:38.073493+0000","last_clean":"2023-11-28T18:23:38.073493+0000","last_became_active":"2023-11-28T18:23:33.938295+0000","last_became_peered":"2023-11-28T18:23:33.938295+0000","last_unstale":"2023-11-28T18:23:38.073493+0000","last_undegraded":"2023-11-28T18:23:38.073493+0000","last_fullsized":"2023-11-28T18:23:38.073493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:12.557669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.073721+0000","last_change":"2023-11-28T18:23:32.930354+0000","last_active":"2023-11-28T18:23:38.073721+0000","last_peered":"2023-11-28T18:23:38.073721+0000","last_clean":"2023-11-28T18:23:38.073721+0000","last_became_active":"2023-11-28T18:23:32.930059+0000","last_became_peered":"2023-11-28T18:23:32.930059+0000","last_unstale":"2023-11-28T18:23:38.073721+0000","last_undegraded":"2023-11-28T18:23:38.073721+0000","last_fullsized":"2023-11-28T18:23:38.073721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:45.143373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.069210+0000","last_change":"2023-11-28T18:23:38.427077+0000","last_active":"2023-11-28T18:23:39.069210+0000","last_peered":"2023-11-28T18:23:39.069210+0000","last_clean":"2023-11-28T18:23:39.069210+0000","last_became_active":"2023-11-28T18:23:38.426613+0000","last_became_peered":"2023-11-28T18:23:38.426613+0000","last_unstale":"2023-11-28T18:23:39.069210+0000","last_undegraded":"2023-11-28T18:23:39.069210+0000","last_fullsized":"2023-11-28T18:23:39.069210+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:24:11.818911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419843+0000","last_change":"2023-11-28T18:23:33.955340+0000","last_active":"2023-11-28T18:23:38.419843+0000","last_peered":"2023-11-28T18:23:38.419843+0000","last_clean":"2023-11-28T18:23:38.419843+0000","last_became_active":"2023-11-28T18:23:33.955080+0000","last_became_peered":"2023-11-28T18:23:33.955080+0000","last_unstale":"2023-11-28T18:23:38.419843+0000","last_undegraded":"2023-11-28T18:23:38.419843+0000","last_fullsized":"2023-11-28T18:23:38.419843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:42.992322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068633+0000","last_change":"2023-11-28T18:23:32.935530+0000","last_active":"2023-11-28T18:23:38.068633+0000","last_peered":"2023-11-28T18:23:38.068633+0000","last_clean":"2023-11-28T18:23:38.068633+0000","last_became_active":"2023-11-28T18:23:32.934513+0000","last_became_peered":"2023-11-28T18:23:32.934513+0000","last_unstale":"2023-11-28T18:23:38.068633+0000","last_undegraded":"2023-11-28T18:23:38.068633+0000","last_fullsized":"2023-11-28T18:23:38.068633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:10.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068541+0000","last_change":"2023-11-28T18:23:38.421813+0000","last_active":"2023-11-28T18:23:39.068541+0000","last_peered":"2023-11-28T18:23:39.068541+0000","last_clean":"2023-11-28T18:23:39.068541+0000","last_became_active":"2023-11-28T18:23:38.421280+0000","last_became_peered":"2023-11-28T18:23:38.421280+0000","last_unstale":"2023-11-28T18:23:39.068541+0000","last_undegraded":"2023-11-28T18:23:39.068541+0000","last_fullsized":"2023-11-28T18:23:39.068541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:19:05.111022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.080161+0000","last_change":"2023-11-28T18:23:33.946537+0000","last_active":"2023-11-28T18:23:38.080161+0000","last_peered":"2023-11-28T18:23:38.080161+0000","last_clean":"2023-11-28T18:23:38.080161+0000","last_became_active":"2023-11-28T18:23:33.946220+0000","last_became_peered":"2023-11-28T18:23:33.946220+0000","last_unstale":"2023-11-28T18:23:38.080161+0000","last_undegraded":"2023-11-28T18:23:38.080161+0000","last_fullsized":"2023-11-28T18:23:38.080161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:51.652679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077792+0000","last_change":"2023-11-28T18:23:32.922020+0000","last_active":"2023-11-28T18:23:38.077792+0000","last_peered":"2023-11-28T18:23:38.077792+0000","last_clean":"2023-11-28T18:23:38.077792+0000","last_became_active":"2023-11-28T18:23:32.921656+0000","last_became_peered":"2023-11-28T18:23:32.921656+0000","last_unstale":"2023-11-28T18:23:38.077792+0000","last_undegraded":"2023-11-28T18:23:38.077792+0000","last_fullsized":"2023-11-28T18:23:38.077792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:10.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.721071+0000","last_change":"2023-11-28T18:23:38.421823+0000","last_active":"2023-11-28T18:23:40.721071+0000","last_peered":"2023-11-28T18:23:40.721071+0000","last_clean":"2023-11-28T18:23:40.721071+0000","last_became_active":"2023-11-28T18:23:38.421407+0000","last_became_peered":"2023-11-28T18:23:38.421407+0000","last_unstale":"2023-11-28T18:23:40.721071+0000","last_undegraded":"2023-11-28T18:23:40.721071+0000","last_fullsized":"2023-11-28T18:23:40.721071+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:01.885046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419892+0000","last_change":"2023-11-28T18:23:33.939629+0000","last_active":"2023-11-28T18:23:38.419892+0000","last_peered":"2023-11-28T18:23:38.419892+0000","last_clean":"2023-11-28T18:23:38.419892+0000","last_became_active":"2023-11-28T18:23:33.938662+0000","last_became_peered":"2023-11-28T18:23:33.938662+0000","last_unstale":"2023-11-28T18:23:38.419892+0000","last_undegraded":"2023-11-28T18:23:38.419892+0000","last_fullsized":"2023-11-28T18:23:38.419892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:02.989816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'8","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:42.888210+0000","last_change":"2023-11-28T18:23:32.923661+0000","last_active":"2023-11-28T18:23:42.888210+0000","last_peered":"2023-11-28T18:23:42.888210+0000","last_clean":"2023-11-28T18:23:42.888210+0000","last_became_active":"2023-11-28T18:23:32.922407+0000","last_became_peered":"2023-11-28T18:23:32.922407+0000","last_unstale":"2023-11-28T18:23:42.888210+0000","last_undegraded":"2023-11-28T18:23:42.888210+0000","last_fullsized":"2023-11-28T18:23:42.888210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:10.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3700,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.718701+0000","last_change":"2023-11-28T18:23:38.098761+0000","last_active":"2023-11-28T18:23:40.718701+0000","last_peered":"2023-11-28T18:23:40.718701+0000","last_clean":"2023-11-28T18:23:40.718701+0000","last_became_active":"2023-11-28T18:23:38.098374+0000","last_became_peered":"2023-11-28T18:23:38.098374+0000","last_unstale":"2023-11-28T18:23:40.718701+0000","last_undegraded":"2023-11-28T18:23:40.718701+0000","last_fullsized":"2023-11-28T18:23:40.718701+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:53.274366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075827+0000","last_change":"2023-11-28T18:23:33.945418+0000","last_active":"2023-11-28T18:23:38.075827+0000","last_peered":"2023-11-28T18:23:38.075827+0000","last_clean":"2023-11-28T18:23:38.075827+0000","last_became_active":"2023-11-28T18:23:33.944949+0000","last_became_peered":"2023-11-28T18:23:33.944949+0000","last_unstale":"2023-11-28T18:23:38.075827+0000","last_undegraded":"2023-11-28T18:23:38.075827+0000","last_fullsized":"2023-11-28T18:23:38.075827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:42.826477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.094316+0000","last_change":"2023-11-28T18:23:32.925933+0000","last_active":"2023-11-28T18:23:38.094316+0000","last_peered":"2023-11-28T18:23:38.094316+0000","last_clean":"2023-11-28T18:23:38.094316+0000","last_became_active":"2023-11-28T18:23:32.925619+0000","last_became_peered":"2023-11-28T18:23:32.925619+0000","last_unstale":"2023-11-28T18:23:38.094316+0000","last_undegraded":"2023-11-28T18:23:38.094316+0000","last_fullsized":"2023-11-28T18:23:38.094316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:19.143213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068423+0000","last_change":"2023-11-28T18:23:38.091049+0000","last_active":"2023-11-28T18:23:39.068423+0000","last_peered":"2023-11-28T18:23:39.068423+0000","last_clean":"2023-11-28T18:23:39.068423+0000","last_became_active":"2023-11-28T18:23:38.090236+0000","last_became_peered":"2023-11-28T18:23:38.090236+0000","last_unstale":"2023-11-28T18:23:39.068423+0000","last_undegraded":"2023-11-28T18:23:39.068423+0000","last_fullsized":"2023-11-28T18:23:39.068423+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:12.193604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.076836+0000","last_change":"2023-11-28T18:23:33.948346+0000","last_active":"2023-11-28T18:23:38.076836+0000","last_peered":"2023-11-28T18:23:38.076836+0000","last_clean":"2023-11-28T18:23:38.076836+0000","last_became_active":"2023-11-28T18:23:33.947983+0000","last_became_peered":"2023-11-28T18:23:33.947983+0000","last_unstale":"2023-11-28T18:23:38.076836+0000","last_undegraded":"2023-11-28T18:23:38.076836+0000","last_fullsized":"2023-11-28T18:23:38.076836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:12.557669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067715+0000","last_change":"2023-11-28T18:23:32.935430+0000","last_active":"2023-11-28T18:23:38.067715+0000","last_peered":"2023-11-28T18:23:38.067715+0000","last_clean":"2023-11-28T18:23:38.067715+0000","last_became_active":"2023-11-28T18:23:32.930282+0000","last_became_peered":"2023-11-28T18:23:32.930282+0000","last_unstale":"2023-11-28T18:23:38.067715+0000","last_undegraded":"2023-11-28T18:23:38.067715+0000","last_fullsized":"2023-11-28T18:23:38.067715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:50.646527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.067746+0000","last_change":"2023-11-28T18:23:38.093851+0000","last_active":"2023-11-28T18:23:39.067746+0000","last_peered":"2023-11-28T18:23:39.067746+0000","last_clean":"2023-11-28T18:23:39.067746+0000","last_became_active":"2023-11-28T18:23:38.091779+0000","last_became_peered":"2023-11-28T18:23:38.091779+0000","last_unstale":"2023-11-28T18:23:39.067746+0000","last_undegraded":"2023-11-28T18:23:39.067746+0000","last_fullsized":"2023-11-28T18:23:39.067746+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:13:14.577345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067617+0000","last_change":"2023-11-28T18:23:33.952137+0000","last_active":"2023-11-28T18:23:38.067617+0000","last_peered":"2023-11-28T18:23:38.067617+0000","last_clean":"2023-11-28T18:23:38.067617+0000","last_became_active":"2023-11-28T18:23:33.950857+0000","last_became_peered":"2023-11-28T18:23:33.950857+0000","last_unstale":"2023-11-28T18:23:38.067617+0000","last_undegraded":"2023-11-28T18:23:38.067617+0000","last_fullsized":"2023-11-28T18:23:38.067617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:42.992322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077679+0000","last_change":"2023-11-28T18:23:32.940089+0000","last_active":"2023-11-28T18:23:38.077679+0000","last_peered":"2023-11-28T18:23:38.077679+0000","last_clean":"2023-11-28T18:23:38.077679+0000","last_became_active":"2023-11-28T18:23:32.939524+0000","last_became_peered":"2023-11-28T18:23:32.939524+0000","last_unstale":"2023-11-28T18:23:38.077679+0000","last_undegraded":"2023-11-28T18:23:38.077679+0000","last_fullsized":"2023-11-28T18:23:38.077679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:19.143213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.067936+0000","last_change":"2023-11-28T18:23:38.424675+0000","last_active":"2023-11-28T18:23:39.067936+0000","last_peered":"2023-11-28T18:23:39.067936+0000","last_clean":"2023-11-28T18:23:39.067936+0000","last_became_active":"2023-11-28T18:23:38.424284+0000","last_became_peered":"2023-11-28T18:23:38.424284+0000","last_unstale":"2023-11-28T18:23:39.067936+0000","last_undegraded":"2023-11-28T18:23:39.067936+0000","last_fullsized":"2023-11-28T18:23:39.067936+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:01.885046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.297416+0000","last_change":"2023-11-28T18:23:38.425611+0000","last_active":"2023-11-28T18:23:40.297416+0000","last_peered":"2023-11-28T18:23:40.297416+0000","last_clean":"2023-11-28T18:23:40.297416+0000","last_became_active":"2023-11-28T18:23:38.424826+0000","last_became_peered":"2023-11-28T18:23:38.424826+0000","last_unstale":"2023-11-28T18:23:40.297416+0000","last_undegraded":"2023-11-28T18:23:40.297416+0000","last_fullsized":"2023-11-28T18:23:40.297416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:15.599832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.092623+0000","last_change":"2023-11-28T18:23:33.937161+0000","last_active":"2023-11-28T18:23:38.092623+0000","last_peered":"2023-11-28T18:23:38.092623+0000","last_clean":"2023-11-28T18:23:38.092623+0000","last_became_active":"2023-11-28T18:23:33.936724+0000","last_became_peered":"2023-11-28T18:23:33.936724+0000","last_unstale":"2023-11-28T18:23:38.092623+0000","last_undegraded":"2023-11-28T18:23:38.092623+0000","last_fullsized":"2023-11-28T18:23:38.092623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:17:14.412292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.070788+0000","last_change":"2023-11-28T18:23:32.921502+0000","last_active":"2023-11-28T18:23:38.070788+0000","last_peered":"2023-11-28T18:23:38.070788+0000","last_clean":"2023-11-28T18:23:38.070788+0000","last_became_active":"2023-11-28T18:23:32.920979+0000","last_became_peered":"2023-11-28T18:23:32.920979+0000","last_unstale":"2023-11-28T18:23:38.070788+0000","last_undegraded":"2023-11-28T18:23:38.070788+0000","last_fullsized":"2023-11-28T18:23:38.070788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:41.820325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.719783+0000","last_change":"2023-11-28T18:23:38.095346+0000","last_active":"2023-11-28T18:23:40.719783+0000","last_peered":"2023-11-28T18:23:40.719783+0000","last_clean":"2023-11-28T18:23:40.719783+0000","last_became_active":"2023-11-28T18:23:38.094985+0000","last_became_peered":"2023-11-28T18:23:38.094985+0000","last_unstale":"2023-11-28T18:23:40.719783+0000","last_undegraded":"2023-11-28T18:23:40.719783+0000","last_fullsized":"2023-11-28T18:23:40.719783+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:22:41.524911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065411+0000","last_change":"2023-11-28T18:23:33.934122+0000","last_active":"2023-11-28T18:23:38.065411+0000","last_peered":"2023-11-28T18:23:38.065411+0000","last_clean":"2023-11-28T18:23:38.065411+0000","last_became_active":"2023-11-28T18:23:33.933687+0000","last_became_peered":"2023-11-28T18:23:33.933687+0000","last_unstale":"2023-11-28T18:23:38.065411+0000","last_undegraded":"2023-11-28T18:23:38.065411+0000","last_fullsized":"2023-11-28T18:23:38.065411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:17:14.412292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065550+0000","last_change":"2023-11-28T18:23:32.931872+0000","last_active":"2023-11-28T18:23:38.065550+0000","last_peered":"2023-11-28T18:23:38.065550+0000","last_clean":"2023-11-28T18:23:38.065550+0000","last_became_active":"2023-11-28T18:23:32.931111+0000","last_became_peered":"2023-11-28T18:23:32.931111+0000","last_unstale":"2023-11-28T18:23:38.065550+0000","last_undegraded":"2023-11-28T18:23:38.065550+0000","last_fullsized":"2023-11-28T18:23:38.065550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:41.820325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.074084+0000","last_change":"2023-11-28T18:23:32.934258+0000","last_active":"2023-11-28T18:23:38.074084+0000","last_peered":"2023-11-28T18:23:38.074084+0000","last_clean":"2023-11-28T18:23:38.074084+0000","last_became_active":"2023-11-28T18:23:32.933956+0000","last_became_peered":"2023-11-28T18:23:32.933956+0000","last_unstale":"2023-11-28T18:23:38.074084+0000","last_undegraded":"2023-11-28T18:23:38.074084+0000","last_fullsized":"2023-11-28T18:23:38.074084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:41.820325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077709+0000","last_change":"2023-11-28T18:23:33.949218+0000","last_active":"2023-11-28T18:23:38.077709+0000","last_peered":"2023-11-28T18:23:38.077709+0000","last_clean":"2023-11-28T18:23:38.077709+0000","last_became_active":"2023-11-28T18:23:33.948930+0000","last_became_peered":"2023-11-28T18:23:33.948930+0000","last_unstale":"2023-11-28T18:23:38.077709+0000","last_undegraded":"2023-11-28T18:23:38.077709+0000","last_fullsized":"2023-11-28T18:23:38.077709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:46.421541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.067937+0000","last_change":"2023-11-28T18:23:38.095800+0000","last_active":"2023-11-28T18:23:39.067937+0000","last_peered":"2023-11-28T18:23:39.067937+0000","last_clean":"2023-11-28T18:23:39.067937+0000","last_became_active":"2023-11-28T18:23:38.095437+0000","last_became_peered":"2023-11-28T18:23:38.095437+0000","last_unstale":"2023-11-28T18:23:39.067937+0000","last_undegraded":"2023-11-28T18:23:39.067937+0000","last_fullsized":"2023-11-28T18:23:39.067937+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:39.644595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065704+0000","last_change":"2023-11-28T18:23:32.921448+0000","last_active":"2023-11-28T18:23:38.065704+0000","last_peered":"2023-11-28T18:23:38.065704+0000","last_clean":"2023-11-28T18:23:38.065704+0000","last_became_active":"2023-11-28T18:23:32.921183+0000","last_became_peered":"2023-11-28T18:23:32.921183+0000","last_unstale":"2023-11-28T18:23:38.065704+0000","last_undegraded":"2023-11-28T18:23:38.065704+0000","last_fullsized":"2023-11-28T18:23:38.065704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:36:09.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075654+0000","last_change":"2023-11-28T18:23:33.937197+0000","last_active":"2023-11-28T18:23:38.075654+0000","last_peered":"2023-11-28T18:23:38.075654+0000","last_clean":"2023-11-28T18:23:38.075654+0000","last_became_active":"2023-11-28T18:23:33.936922+0000","last_became_peered":"2023-11-28T18:23:33.936922+0000","last_unstale":"2023-11-28T18:23:38.075654+0000","last_undegraded":"2023-11-28T18:23:38.075654+0000","last_fullsized":"2023-11-28T18:23:38.075654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:01.331886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068617+0000","last_change":"2023-11-28T18:23:38.099275+0000","last_active":"2023-11-28T18:23:39.068617+0000","last_peered":"2023-11-28T18:23:39.068617+0000","last_clean":"2023-11-28T18:23:39.068617+0000","last_became_active":"2023-11-28T18:23:38.095544+0000","last_became_peered":"2023-11-28T18:23:38.095544+0000","last_unstale":"2023-11-28T18:23:39.068617+0000","last_undegraded":"2023-11-28T18:23:39.068617+0000","last_fullsized":"2023-11-28T18:23:39.068617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:04.943189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.073386+0000","last_change":"2023-11-28T18:23:33.943034+0000","last_active":"2023-11-28T18:23:38.073386+0000","last_peered":"2023-11-28T18:23:38.073386+0000","last_clean":"2023-11-28T18:23:38.073386+0000","last_became_active":"2023-11-28T18:23:33.942687+0000","last_became_peered":"2023-11-28T18:23:33.942687+0000","last_unstale":"2023-11-28T18:23:38.073386+0000","last_undegraded":"2023-11-28T18:23:38.073386+0000","last_fullsized":"2023-11-28T18:23:38.073386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:35.484199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.083621+0000","last_change":"2023-11-28T18:23:33.942958+0000","last_active":"2023-11-28T18:23:38.083621+0000","last_peered":"2023-11-28T18:23:38.083621+0000","last_clean":"2023-11-28T18:23:38.083621+0000","last_became_active":"2023-11-28T18:23:33.942696+0000","last_became_peered":"2023-11-28T18:23:33.942696+0000","last_unstale":"2023-11-28T18:23:38.083621+0000","last_undegraded":"2023-11-28T18:23:38.083621+0000","last_fullsized":"2023-11-28T18:23:38.083621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:42.826477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068550+0000","last_change":"2023-11-28T18:23:33.927110+0000","last_active":"2023-11-28T18:23:38.068550+0000","last_peered":"2023-11-28T18:23:38.068550+0000","last_clean":"2023-11-28T18:23:38.068550+0000","last_became_active":"2023-11-28T18:23:33.926812+0000","last_became_peered":"2023-11-28T18:23:33.926812+0000","last_unstale":"2023-11-28T18:23:38.068550+0000","last_undegraded":"2023-11-28T18:23:38.068550+0000","last_fullsized":"2023-11-28T18:23:38.068550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:22:37.364515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077616+0000","last_change":"2023-11-28T18:23:33.941141+0000","last_active":"2023-11-28T18:23:38.077616+0000","last_peered":"2023-11-28T18:23:38.077616+0000","last_clean":"2023-11-28T18:23:38.077616+0000","last_became_active":"2023-11-28T18:23:33.940832+0000","last_became_peered":"2023-11-28T18:23:33.940832+0000","last_unstale":"2023-11-28T18:23:38.077616+0000","last_undegraded":"2023-11-28T18:23:38.077616+0000","last_fullsized":"2023-11-28T18:23:38.077616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:04:56.118542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.066054+0000","last_change":"2023-11-28T18:23:33.924784+0000","last_active":"2023-11-28T18:23:38.066054+0000","last_peered":"2023-11-28T18:23:38.066054+0000","last_clean":"2023-11-28T18:23:38.066054+0000","last_became_active":"2023-11-28T18:23:33.924475+0000","last_became_peered":"2023-11-28T18:23:33.924475+0000","last_unstale":"2023-11-28T18:23:38.066054+0000","last_undegraded":"2023-11-28T18:23:38.066054+0000","last_fullsized":"2023-11-28T18:23:38.066054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:05.773283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.094179+0000","last_change":"2023-11-28T18:23:33.932579+0000","last_active":"2023-11-28T18:23:38.094179+0000","last_peered":"2023-11-28T18:23:38.094179+0000","last_clean":"2023-11-28T18:23:38.094179+0000","last_became_active":"2023-11-28T18:23:33.932201+0000","last_became_peered":"2023-11-28T18:23:33.932201+0000","last_unstale":"2023-11-28T18:23:38.094179+0000","last_undegraded":"2023-11-28T18:23:38.094179+0000","last_fullsized":"2023-11-28T18:23:38.094179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:01.912695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077881+0000","last_change":"2023-11-28T18:23:33.954178+0000","last_active":"2023-11-28T18:23:38.077881+0000","last_peered":"2023-11-28T18:23:38.077881+0000","last_clean":"2023-11-28T18:23:38.077881+0000","last_became_active":"2023-11-28T18:23:33.954020+0000","last_became_peered":"2023-11-28T18:23:33.954020+0000","last_unstale":"2023-11-28T18:23:38.077881+0000","last_undegraded":"2023-11-28T18:23:38.077881+0000","last_fullsized":"2023-11-28T18:23:38.077881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:01.912695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.066974+0000","last_change":"2023-11-28T18:23:33.939474+0000","last_active":"2023-11-28T18:23:38.066974+0000","last_peered":"2023-11-28T18:23:38.066974+0000","last_clean":"2023-11-28T18:23:38.066974+0000","last_became_active":"2023-11-28T18:23:33.939189+0000","last_became_peered":"2023-11-28T18:23:33.939189+0000","last_unstale":"2023-11-28T18:23:38.066974+0000","last_undegraded":"2023-11-28T18:23:38.066974+0000","last_fullsized":"2023-11-28T18:23:38.066974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:08.033209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.070751+0000","last_change":"2023-11-28T18:23:33.929306+0000","last_active":"2023-11-28T18:23:38.070751+0000","last_peered":"2023-11-28T18:23:38.070751+0000","last_clean":"2023-11-28T18:23:38.070751+0000","last_became_active":"2023-11-28T18:23:33.929029+0000","last_became_peered":"2023-11-28T18:23:33.929029+0000","last_unstale":"2023-11-28T18:23:38.070751+0000","last_undegraded":"2023-11-28T18:23:38.070751+0000","last_fullsized":"2023-11-28T18:23:38.070751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:46.421541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065454+0000","last_change":"2023-11-28T18:23:33.922789+0000","last_active":"2023-11-28T18:23:38.065454+0000","last_peered":"2023-11-28T18:23:38.065454+0000","last_clean":"2023-11-28T18:23:38.065454+0000","last_became_active":"2023-11-28T18:23:33.922458+0000","last_became_peered":"2023-11-28T18:23:33.922458+0000","last_unstale":"2023-11-28T18:23:38.065454+0000","last_undegraded":"2023-11-28T18:23:38.065454+0000","last_fullsized":"2023-11-28T18:23:38.065454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:51.652679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068465+0000","last_change":"2023-11-28T18:23:33.935519+0000","last_active":"2023-11-28T18:23:38.068465+0000","last_peered":"2023-11-28T18:23:38.068465+0000","last_clean":"2023-11-28T18:23:38.068465+0000","last_became_active":"2023-11-28T18:23:33.934779+0000","last_became_peered":"2023-11-28T18:23:33.934779+0000","last_unstale":"2023-11-28T18:23:38.068465+0000","last_undegraded":"2023-11-28T18:23:38.068465+0000","last_fullsized":"2023-11-28T18:23:38.068465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:27.506820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077522+0000","last_change":"2023-11-28T18:23:33.946486+0000","last_active":"2023-11-28T18:23:38.077522+0000","last_peered":"2023-11-28T18:23:38.077522+0000","last_clean":"2023-11-28T18:23:38.077522+0000","last_became_active":"2023-11-28T18:23:33.946212+0000","last_became_peered":"2023-11-28T18:23:33.946212+0000","last_unstale":"2023-11-28T18:23:38.077522+0000","last_undegraded":"2023-11-28T18:23:38.077522+0000","last_fullsized":"2023-11-28T18:23:38.077522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:17:14.412292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.076300+0000","last_change":"2023-11-28T18:23:33.949577+0000","last_active":"2023-11-28T18:23:38.076300+0000","last_peered":"2023-11-28T18:23:38.076300+0000","last_clean":"2023-11-28T18:23:38.076300+0000","last_became_active":"2023-11-28T18:23:33.949275+0000","last_became_peered":"2023-11-28T18:23:33.949275+0000","last_unstale":"2023-11-28T18:23:38.076300+0000","last_undegraded":"2023-11-28T18:23:38.076300+0000","last_fullsized":"2023-11-28T18:23:38.076300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:17:14.412292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420401+0000","last_change":"2023-11-28T18:23:33.939143+0000","last_active":"2023-11-28T18:23:38.420401+0000","last_peered":"2023-11-28T18:23:38.420401+0000","last_clean":"2023-11-28T18:23:38.420401+0000","last_became_active":"2023-11-28T18:23:33.938673+0000","last_became_peered":"2023-11-28T18:23:33.938673+0000","last_unstale":"2023-11-28T18:23:38.420401+0000","last_undegraded":"2023-11-28T18:23:38.420401+0000","last_fullsized":"2023-11-28T18:23:38.420401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:11.439436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.083975+0000","last_change":"2023-11-28T18:23:33.946701+0000","last_active":"2023-11-28T18:23:38.083975+0000","last_peered":"2023-11-28T18:23:38.083975+0000","last_clean":"2023-11-28T18:23:38.083975+0000","last_became_active":"2023-11-28T18:23:33.946460+0000","last_became_peered":"2023-11-28T18:23:33.946460+0000","last_unstale":"2023-11-28T18:23:38.083975+0000","last_undegraded":"2023-11-28T18:23:38.083975+0000","last_fullsized":"2023-11-28T18:23:38.083975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:11.792676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068495+0000","last_change":"2023-11-28T18:23:33.940275+0000","last_active":"2023-11-28T18:23:38.068495+0000","last_peered":"2023-11-28T18:23:38.068495+0000","last_clean":"2023-11-28T18:23:38.068495+0000","last_became_active":"2023-11-28T18:23:33.939937+0000","last_became_peered":"2023-11-28T18:23:33.939937+0000","last_unstale":"2023-11-28T18:23:38.068495+0000","last_undegraded":"2023-11-28T18:23:38.068495+0000","last_fullsized":"2023-11-28T18:23:38.068495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:11.439436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.070691+0000","last_change":"2023-11-28T18:23:33.937355+0000","last_active":"2023-11-28T18:23:38.070691+0000","last_peered":"2023-11-28T18:23:38.070691+0000","last_clean":"2023-11-28T18:23:38.070691+0000","last_became_active":"2023-11-28T18:23:33.937069+0000","last_became_peered":"2023-11-28T18:23:33.937069+0000","last_unstale":"2023-11-28T18:23:38.070691+0000","last_undegraded":"2023-11-28T18:23:38.070691+0000","last_fullsized":"2023-11-28T18:23:38.070691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:05.773283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.092623+0000","last_change":"2023-11-28T18:23:33.938555+0000","last_active":"2023-11-28T18:23:38.092623+0000","last_peered":"2023-11-28T18:23:38.092623+0000","last_clean":"2023-11-28T18:23:38.092623+0000","last_became_active":"2023-11-28T18:23:33.938297+0000","last_became_peered":"2023-11-28T18:23:33.938297+0000","last_unstale":"2023-11-28T18:23:38.092623+0000","last_undegraded":"2023-11-28T18:23:38.092623+0000","last_fullsized":"2023-11-28T18:23:38.092623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:51.652679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.080082+0000","last_change":"2023-11-28T18:23:33.942250+0000","last_active":"2023-11-28T18:23:38.080082+0000","last_peered":"2023-11-28T18:23:38.080082+0000","last_clean":"2023-11-28T18:23:38.080082+0000","last_became_active":"2023-11-28T18:23:33.941935+0000","last_became_peered":"2023-11-28T18:23:33.941935+0000","last_unstale":"2023-11-28T18:23:38.080082+0000","last_undegraded":"2023-11-28T18:23:38.080082+0000","last_fullsized":"2023-11-28T18:23:38.080082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:21.035422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065409+0000","last_change":"2023-11-28T18:23:33.944983+0000","last_active":"2023-11-28T18:23:38.065409+0000","last_peered":"2023-11-28T18:23:38.065409+0000","last_clean":"2023-11-28T18:23:38.065409+0000","last_became_active":"2023-11-28T18:23:33.944535+0000","last_became_peered":"2023-11-28T18:23:33.944535+0000","last_unstale":"2023-11-28T18:23:38.065409+0000","last_undegraded":"2023-11-28T18:23:38.065409+0000","last_fullsized":"2023-11-28T18:23:38.065409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:01.912695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075790+0000","last_change":"2023-11-28T18:23:33.945213+0000","last_active":"2023-11-28T18:23:38.075790+0000","last_peered":"2023-11-28T18:23:38.075790+0000","last_clean":"2023-11-28T18:23:38.075790+0000","last_became_active":"2023-11-28T18:23:33.944605+0000","last_became_peered":"2023-11-28T18:23:33.944605+0000","last_unstale":"2023-11-28T18:23:38.075790+0000","last_undegraded":"2023-11-28T18:23:38.075790+0000","last_fullsized":"2023-11-28T18:23:38.075790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:46.149525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.076131+0000","last_change":"2023-11-28T18:23:33.948453+0000","last_active":"2023-11-28T18:23:38.076131+0000","last_peered":"2023-11-28T18:23:38.076131+0000","last_clean":"2023-11-28T18:23:38.076131+0000","last_became_active":"2023-11-28T18:23:33.948203+0000","last_became_peered":"2023-11-28T18:23:33.948203+0000","last_unstale":"2023-11-28T18:23:38.076131+0000","last_undegraded":"2023-11-28T18:23:38.076131+0000","last_fullsized":"2023-11-28T18:23:38.076131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:51.652679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077793+0000","last_change":"2023-11-28T18:23:33.945564+0000","last_active":"2023-11-28T18:23:38.077793+0000","last_peered":"2023-11-28T18:23:38.077793+0000","last_clean":"2023-11-28T18:23:38.077793+0000","last_became_active":"2023-11-28T18:23:33.943276+0000","last_became_peered":"2023-11-28T18:23:33.943276+0000","last_unstale":"2023-11-28T18:23:38.077793+0000","last_undegraded":"2023-11-28T18:23:38.077793+0000","last_fullsized":"2023-11-28T18:23:38.077793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:21.035422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420502+0000","last_change":"2023-11-28T18:23:33.942649+0000","last_active":"2023-11-28T18:23:38.420502+0000","last_peered":"2023-11-28T18:23:38.420502+0000","last_clean":"2023-11-28T18:23:38.420502+0000","last_became_active":"2023-11-28T18:23:33.942109+0000","last_became_peered":"2023-11-28T18:23:33.942109+0000","last_unstale":"2023-11-28T18:23:38.420502+0000","last_undegraded":"2023-11-28T18:23:38.420502+0000","last_fullsized":"2023-11-28T18:23:38.420502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:34.989175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.064784+0000","last_change":"2023-11-28T18:23:33.933929+0000","last_active":"2023-11-28T18:23:38.064784+0000","last_peered":"2023-11-28T18:23:38.064784+0000","last_clean":"2023-11-28T18:23:38.064784+0000","last_became_active":"2023-11-28T18:23:33.933087+0000","last_became_peered":"2023-11-28T18:23:33.933087+0000","last_unstale":"2023-11-28T18:23:38.064784+0000","last_undegraded":"2023-11-28T18:23:38.064784+0000","last_fullsized":"2023-11-28T18:23:38.064784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:12.557669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419632+0000","last_change":"2023-11-28T18:23:33.950276+0000","last_active":"2023-11-28T18:23:38.419632+0000","last_peered":"2023-11-28T18:23:38.419632+0000","last_clean":"2023-11-28T18:23:38.419632+0000","last_became_active":"2023-11-28T18:23:33.946777+0000","last_became_peered":"2023-11-28T18:23:33.946777+0000","last_unstale":"2023-11-28T18:23:38.419632+0000","last_undegraded":"2023-11-28T18:23:38.419632+0000","last_fullsized":"2023-11-28T18:23:38.419632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:46.421541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068159+0000","last_change":"2023-11-28T18:23:33.943666+0000","last_active":"2023-11-28T18:23:38.068159+0000","last_peered":"2023-11-28T18:23:38.068159+0000","last_clean":"2023-11-28T18:23:38.068159+0000","last_became_active":"2023-11-28T18:23:33.940798+0000","last_became_peered":"2023-11-28T18:23:33.940798+0000","last_unstale":"2023-11-28T18:23:38.068159+0000","last_undegraded":"2023-11-28T18:23:38.068159+0000","last_fullsized":"2023-11-28T18:23:38.068159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:04:56.118542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419726+0000","last_change":"2023-11-28T18:23:33.955086+0000","last_active":"2023-11-28T18:23:38.419726+0000","last_peered":"2023-11-28T18:23:38.419726+0000","last_clean":"2023-11-28T18:23:38.419726+0000","last_became_active":"2023-11-28T18:23:33.950311+0000","last_became_peered":"2023-11-28T18:23:33.950311+0000","last_unstale":"2023-11-28T18:23:38.419726+0000","last_undegraded":"2023-11-28T18:23:38.419726+0000","last_fullsized":"2023-11-28T18:23:38.419726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:27.506820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067660+0000","last_change":"2023-11-28T18:23:33.932833+0000","last_active":"2023-11-28T18:23:38.067660+0000","last_peered":"2023-11-28T18:23:38.067660+0000","last_clean":"2023-11-28T18:23:38.067660+0000","last_became_active":"2023-11-28T18:23:33.932554+0000","last_became_peered":"2023-11-28T18:23:33.932554+0000","last_unstale":"2023-11-28T18:23:38.067660+0000","last_undegraded":"2023-11-28T18:23:38.067660+0000","last_fullsized":"2023-11-28T18:23:38.067660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:08.033209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075353+0000","last_change":"2023-11-28T18:23:33.939136+0000","last_active":"2023-11-28T18:23:38.075353+0000","last_peered":"2023-11-28T18:23:38.075353+0000","last_clean":"2023-11-28T18:23:38.075353+0000","last_became_active":"2023-11-28T18:23:33.938870+0000","last_became_peered":"2023-11-28T18:23:33.938870+0000","last_unstale":"2023-11-28T18:23:38.075353+0000","last_undegraded":"2023-11-28T18:23:38.075353+0000","last_fullsized":"2023-11-28T18:23:38.075353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:05.773283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.096250+0000","last_change":"2023-11-28T18:23:33.936563+0000","last_active":"2023-11-28T18:23:38.096250+0000","last_peered":"2023-11-28T18:23:38.096250+0000","last_clean":"2023-11-28T18:23:38.096250+0000","last_became_active":"2023-11-28T18:23:33.936272+0000","last_became_peered":"2023-11-28T18:23:33.936272+0000","last_unstale":"2023-11-28T18:23:38.096250+0000","last_undegraded":"2023-11-28T18:23:38.096250+0000","last_fullsized":"2023-11-28T18:23:38.096250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:04:56.118542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420445+0000","last_change":"2023-11-28T18:23:33.954685+0000","last_active":"2023-11-28T18:23:38.420445+0000","last_peered":"2023-11-28T18:23:38.420445+0000","last_clean":"2023-11-28T18:23:38.420445+0000","last_became_active":"2023-11-28T18:23:33.954434+0000","last_became_peered":"2023-11-28T18:23:33.954434+0000","last_unstale":"2023-11-28T18:23:38.420445+0000","last_undegraded":"2023-11-28T18:23:38.420445+0000","last_fullsized":"2023-11-28T18:23:38.420445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:38.277787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4498,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":690742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":99882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":99918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":99792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":99952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":690798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:23:52.123 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:23:52.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.291+0000 7fe9fcc9a640 1 Processor -- start 2023-11-28T18:23:52.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.295+0000 7fe9fcc9a640 1 -- start start 2023-11-28T18:23:52.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.295+0000 7fe9fcc9a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe9f815d4f0 0x7fe9f80d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:52.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.295+0000 7fe9fcc9a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9f80b2d20 0x7fe9f80b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:52.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.295+0000 7fe9fcc9a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f80b25b0 con 0x7fe9f80d3c30 2023-11-28T18:23:52.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.295+0000 7fe9fcc9a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f80b2900 con 0x7fe9f815d4f0 2023-11-28T18:23:52.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.295+0000 7fe9fcc9a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f80d0a20 con 0x7fe9f80b2d20 2023-11-28T18:23:52.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.295+0000 7fe9edd9b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe9f80d3c30 0x7fe9f80d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39238/0 (socket says 172.21.15.105:39238) 2023-11-28T18:23:52.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.295+0000 7fe9edd9b640 1 -- 172.21.15.105:0/3418884199 learned_addr learned my addr 172.21.15.105:0/3418884199 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:52.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.295+0000 7fe9f659c640 1 --2- 172.21.15.105:0/3418884199 >> v2:172.21.15.105:3300/0 conn(0x7fe9f815d4f0 0x7fe9f80d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:52.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.295+0000 7fe9f6d9d640 1 --2- 172.21.15.105:0/3418884199 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9f80b2d20 0x7fe9f80b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:52.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.295+0000 7fe9f5d9b640 1 -- 172.21.15.105:0/3418884199 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3097496497 0 0) 0x7fe9f80b25b0 con 0x7fe9f80d3c30 2023-11-28T18:23:52.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.295+0000 7fe9f5d9b640 1 -- 172.21.15.105:0/3418884199 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9e4003530 con 0x7fe9f80d3c30 2023-11-28T18:23:52.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.295+0000 7fe9f659c640 1 -- 172.21.15.105:0/3418884199 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9f80b2d20 msgr2=0x7fe9f80b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:52.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.295+0000 7fe9f659c640 1 --2- 172.21.15.105:0/3418884199 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9f80b2d20 0x7fe9f80b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.295+0000 7fe9f659c640 1 -- 172.21.15.105:0/3418884199 >> v1:172.21.15.105:6789/0 conn(0x7fe9f80d3c30 legacy=0x7fe9f80d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:52.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.295+0000 7fe9f659c640 1 -- 172.21.15.105:0/3418884199 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f80cea30 con 0x7fe9f815d4f0 2023-11-28T18:23:52.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.299+0000 7fe9f659c640 1 --2- 172.21.15.105:0/3418884199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9f815d4f0 0x7fe9f80d0e10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80097c0 tx=0x7fe9e801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2923e30f44392 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:52.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.299+0000 7fe9f5d9b640 1 -- 172.21.15.105:0/3418884199 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e80ac620 con 0x7fe9f815d4f0 2023-11-28T18:23:52.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.299+0000 7fe9f5d9b640 1 -- 172.21.15.105:0/3418884199 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e80ac780 con 0x7fe9f815d4f0 2023-11-28T18:23:52.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.299+0000 7fe9f5d9b640 1 -- 172.21.15.105:0/3418884199 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e80acb10 con 0x7fe9f815d4f0 2023-11-28T18:23:52.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.299+0000 7fe9fcc9a640 1 -- 172.21.15.105:0/3418884199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9f815d4f0 msgr2=0x7fe9f80d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:52.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.299+0000 7fe9fcc9a640 1 --2- 172.21.15.105:0/3418884199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9f815d4f0 0x7fe9f80d0e10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80097c0 tx=0x7fe9e801c3c0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.299+0000 7fe9fcc9a640 1 -- 172.21.15.105:0/3418884199 shutdown_connections 2023-11-28T18:23:52.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.299+0000 7fe9fcc9a640 1 --2- 172.21.15.105:0/3418884199 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9f80b2d20 0x7fe9f80b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.299+0000 7fe9fcc9a640 1 --2- 172.21.15.105:0/3418884199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9f815d4f0 0x7fe9f80d0e10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.299+0000 7fe9fcc9a640 1 -- 172.21.15.105:0/3418884199 >> 172.21.15.105:0/3418884199 conn(0x7fe9f80d3030 msgr2=0x7fe9f80d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:52.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.299+0000 7fe9fcc9a640 1 -- 172.21.15.105:0/3418884199 shutdown_connections 2023-11-28T18:23:52.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.299+0000 7fe9fcc9a640 1 -- 172.21.15.105:0/3418884199 wait complete. 2023-11-28T18:23:52.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.299+0000 7fe9fcc9a640 1 Processor -- start 2023-11-28T18:23:52.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9fcc9a640 1 -- start start 2023-11-28T18:23:52.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9fcc9a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9f80b2d20 0x7fe9f80cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:52.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9fcc9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9f80d3c30 0x7fe9f813f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:52.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9fcc9a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9f815d4f0 0x7fe9f80cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:52.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9fcc9a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f80ccfb0 con 0x7fe9f80d3c30 2023-11-28T18:23:52.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9fcc9a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f80b8700 con 0x7fe9f815d4f0 2023-11-28T18:23:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9fcc9a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f813fe20 con 0x7fe9f80b2d20 2023-11-28T18:23:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9edd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9f80d3c30 0x7fe9f813f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9edd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9f80d3c30 0x7fe9f813f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54882/0 (socket says 172.21.15.105:54882) 2023-11-28T18:23:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9edd9b640 1 -- 172.21.15.105:0/1381300026 learned_addr learned my addr 172.21.15.105:0/1381300026 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9f6d9d640 1 --2- 172.21.15.105:0/1381300026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9f815d4f0 0x7fe9f80cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9f659c640 1 --2- 172.21.15.105:0/1381300026 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9f80b2d20 0x7fe9f80cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9edd9b640 1 -- 172.21.15.105:0/1381300026 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9f80b2d20 msgr2=0x7fe9f80cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9edd9b640 1 --2- 172.21.15.105:0/1381300026 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9f80b2d20 0x7fe9f80cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9edd9b640 1 -- 172.21.15.105:0/1381300026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9f815d4f0 msgr2=0x7fe9f80cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:52.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9edd9b640 1 --2- 172.21.15.105:0/1381300026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9f815d4f0 0x7fe9f80cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9edd9b640 1 -- 172.21.15.105:0/1381300026 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e8009420 con 0x7fe9f80d3c30 2023-11-28T18:23:52.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9edd9b640 1 --2- 172.21.15.105:0/1381300026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9f80d3c30 0x7fe9f813f8e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000c420 tx=0x7fe9e000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:52.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9ef7fe640 1 -- 172.21.15.105:0/1381300026 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0005cf0 con 0x7fe9f80d3c30 2023-11-28T18:23:52.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9ef7fe640 1 -- 172.21.15.105:0/1381300026 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e0005e50 con 0x7fe9f80d3c30 2023-11-28T18:23:52.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9fcc9a640 1 -- 172.21.15.105:0/1381300026 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f80b8b90 con 0x7fe9f80d3c30 2023-11-28T18:23:52.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9ef7fe640 1 -- 172.21.15.105:0/1381300026 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e00072d0 con 0x7fe9f80d3c30 2023-11-28T18:23:52.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9fcc9a640 1 -- 172.21.15.105:0/1381300026 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f80b7dd0 con 0x7fe9f80d3c30 2023-11-28T18:23:52.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.303+0000 7fe9fcc9a640 1 -- 172.21.15.105:0/1381300026 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f8067570 con 0x7fe9f80d3c30 2023-11-28T18:23:52.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.307+0000 7fe9ef7fe640 1 -- 172.21.15.105:0/1381300026 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fe9e0007430 con 0x7fe9f80d3c30 2023-11-28T18:23:52.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.307+0000 7fe9ef7fe640 1 --2- 172.21.15.105:0/1381300026 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe9d8066900 0x7fe9d8068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:52.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.307+0000 7fe9ef7fe640 1 -- 172.21.15.105:0/1381300026 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe9e0083b40 con 0x7fe9f80d3c30 2023-11-28T18:23:52.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.307+0000 7fe9f659c640 1 --2- 172.21.15.105:0/1381300026 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe9d8066900 0x7fe9d8068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:52.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.307+0000 7fe9f659c640 1 --2- 172.21.15.105:0/1381300026 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe9d8066900 0x7fe9d8068dc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80ae000 tx=0x7fe9e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:52.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.311+0000 7fe9ef7fe640 1 -- 172.21.15.105:0/1381300026 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe9e0050b40 con 0x7fe9f80d3c30 2023-11-28T18:23:52.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.455+0000 7fe9fcc9a640 1 -- 172.21.15.105:0/1381300026 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe9f80d1a60 con 0x7fe9d8066900 2023-11-28T18:23:52.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.467+0000 7fe9ef7fe640 1 -- 172.21.15.105:0/1381300026 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345294 (secure 0 0 0) 0x7fe9e80ac8f0 con 0x7fe9d8066900 2023-11-28T18:23:52.473 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:52.475 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:23:52.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.471+0000 7fe9fcc9a640 1 -- 172.21.15.105:0/1381300026 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe9d8066900 msgr2=0x7fe9d8068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:52.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.471+0000 7fe9fcc9a640 1 --2- 172.21.15.105:0/1381300026 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe9d8066900 0x7fe9d8068dc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80ae000 tx=0x7fe9e8008040 comp rx=0 tx=0).stop 2023-11-28T18:23:52.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.471+0000 7fe9fcc9a640 1 -- 172.21.15.105:0/1381300026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9f80d3c30 msgr2=0x7fe9f813f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:52.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.471+0000 7fe9fcc9a640 1 --2- 172.21.15.105:0/1381300026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9f80d3c30 0x7fe9f813f8e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000c420 tx=0x7fe9e000c8e0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.471+0000 7fe9fcc9a640 1 -- 172.21.15.105:0/1381300026 shutdown_connections 2023-11-28T18:23:52.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.471+0000 7fe9fcc9a640 1 --2- 172.21.15.105:0/1381300026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9f815d4f0 0x7fe9f80cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.471+0000 7fe9fcc9a640 1 --2- 172.21.15.105:0/1381300026 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe9d8066900 0x7fe9d8068dc0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.471+0000 7fe9fcc9a640 1 --2- 172.21.15.105:0/1381300026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9f80d3c30 0x7fe9f813f8e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.471+0000 7fe9fcc9a640 1 --2- 172.21.15.105:0/1381300026 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9f80b2d20 0x7fe9f80cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.471+0000 7fe9fcc9a640 1 -- 172.21.15.105:0/1381300026 >> 172.21.15.105:0/1381300026 conn(0x7fe9f80d3030 msgr2=0x7fe9f806c990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:52.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.471+0000 7fe9fcc9a640 1 -- 172.21.15.105:0/1381300026 shutdown_connections 2023-11-28T18:23:52.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.471+0000 7fe9fcc9a640 1 -- 172.21.15.105:0/1381300026 wait complete. 2023-11-28T18:23:52.494 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2023-11-28T18:23:51.272635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594866,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219380,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755340,"statfs":{"total":773094113280,"available":772869468160,"internally_reserved":0,"allocated":3284992,"data_stored":1988068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2088,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002299"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.076931+0000","last_change":"2023-11-28T18:23:32.930693+0000","last_active":"2023-11-28T18:23:38.076931+0000","last_peered":"2023-11-28T18:23:38.076931+0000","last_clean":"2023-11-28T18:23:38.076931+0000","last_became_active":"2023-11-28T18:23:32.930410+0000","last_became_peered":"2023-11-28T18:23:32.930410+0000","last_unstale":"2023-11-28T18:23:38.076931+0000","last_undegraded":"2023-11-28T18:23:38.076931+0000","last_fullsized":"2023-11-28T18:23:38.076931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:19.143213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.066696+0000","last_change":"2023-11-28T18:23:33.947291+0000","last_active":"2023-11-28T18:23:38.066696+0000","last_peered":"2023-11-28T18:23:38.066696+0000","last_clean":"2023-11-28T18:23:38.066696+0000","last_became_active":"2023-11-28T18:23:33.946963+0000","last_became_peered":"2023-11-28T18:23:33.946963+0000","last_unstale":"2023-11-28T18:23:38.066696+0000","last_undegraded":"2023-11-28T18:23:38.066696+0000","last_fullsized":"2023-11-28T18:23:38.066696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:27.506820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.067797+0000","last_change":"2023-11-28T18:23:38.425549+0000","last_active":"2023-11-28T18:23:39.067797+0000","last_peered":"2023-11-28T18:23:39.067797+0000","last_clean":"2023-11-28T18:23:39.067797+0000","last_became_active":"2023-11-28T18:23:38.425130+0000","last_became_peered":"2023-11-28T18:23:38.425130+0000","last_unstale":"2023-11-28T18:23:39.067797+0000","last_undegraded":"2023-11-28T18:23:39.067797+0000","last_fullsized":"2023-11-28T18:23:39.067797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:19:51.686256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.066904+0000","last_change":"2023-11-28T18:23:32.921328+0000","last_active":"2023-11-28T18:23:38.066904+0000","last_peered":"2023-11-28T18:23:38.066904+0000","last_clean":"2023-11-28T18:23:38.066904+0000","last_became_active":"2023-11-28T18:23:32.920775+0000","last_became_peered":"2023-11-28T18:23:32.920775+0000","last_unstale":"2023-11-28T18:23:38.066904+0000","last_undegraded":"2023-11-28T18:23:38.066904+0000","last_fullsized":"2023-11-28T18:23:38.066904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:05:45.779946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075403+0000","last_change":"2023-11-28T18:23:33.948349+0000","last_active":"2023-11-28T18:23:38.075403+0000","last_peered":"2023-11-28T18:23:38.075403+0000","last_clean":"2023-11-28T18:23:38.075403+0000","last_became_active":"2023-11-28T18:23:33.947987+0000","last_became_peered":"2023-11-28T18:23:33.947987+0000","last_unstale":"2023-11-28T18:23:38.075403+0000","last_undegraded":"2023-11-28T18:23:38.075403+0000","last_fullsized":"2023-11-28T18:23:38.075403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:35.484199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068226+0000","last_change":"2023-11-28T18:23:38.091059+0000","last_active":"2023-11-28T18:23:39.068226+0000","last_peered":"2023-11-28T18:23:39.068226+0000","last_clean":"2023-11-28T18:23:39.068226+0000","last_became_active":"2023-11-28T18:23:38.090259+0000","last_became_peered":"2023-11-28T18:23:38.090259+0000","last_unstale":"2023-11-28T18:23:39.068226+0000","last_undegraded":"2023-11-28T18:23:39.068226+0000","last_fullsized":"2023-11-28T18:23:39.068226+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:05:00.278938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420356+0000","last_change":"2023-11-28T18:23:32.937363+0000","last_active":"2023-11-28T18:23:38.420356+0000","last_peered":"2023-11-28T18:23:38.420356+0000","last_clean":"2023-11-28T18:23:38.420356+0000","last_became_active":"2023-11-28T18:23:32.937125+0000","last_became_peered":"2023-11-28T18:23:32.937125+0000","last_unstale":"2023-11-28T18:23:38.420356+0000","last_undegraded":"2023-11-28T18:23:38.420356+0000","last_fullsized":"2023-11-28T18:23:38.420356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:45.143373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067755+0000","last_change":"2023-11-28T18:23:33.935572+0000","last_active":"2023-11-28T18:23:38.067755+0000","last_peered":"2023-11-28T18:23:38.067755+0000","last_clean":"2023-11-28T18:23:38.067755+0000","last_became_active":"2023-11-28T18:23:33.935204+0000","last_became_peered":"2023-11-28T18:23:33.935204+0000","last_unstale":"2023-11-28T18:23:38.067755+0000","last_undegraded":"2023-11-28T18:23:38.067755+0000","last_fullsized":"2023-11-28T18:23:38.067755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:02.989816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068674+0000","last_change":"2023-11-28T18:23:38.096313+0000","last_active":"2023-11-28T18:23:39.068674+0000","last_peered":"2023-11-28T18:23:39.068674+0000","last_clean":"2023-11-28T18:23:39.068674+0000","last_became_active":"2023-11-28T18:23:38.095957+0000","last_became_peered":"2023-11-28T18:23:38.095957+0000","last_unstale":"2023-11-28T18:23:39.068674+0000","last_undegraded":"2023-11-28T18:23:39.068674+0000","last_fullsized":"2023-11-28T18:23:39.068674+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:07.150212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419858+0000","last_change":"2023-11-28T18:23:32.939798+0000","last_active":"2023-11-28T18:23:38.419858+0000","last_peered":"2023-11-28T18:23:38.419858+0000","last_clean":"2023-11-28T18:23:38.419858+0000","last_became_active":"2023-11-28T18:23:32.939450+0000","last_became_peered":"2023-11-28T18:23:32.939450+0000","last_unstale":"2023-11-28T18:23:38.419858+0000","last_undegraded":"2023-11-28T18:23:38.419858+0000","last_fullsized":"2023-11-28T18:23:38.419858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:55:26.908976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067829+0000","last_change":"2023-11-28T18:23:33.933973+0000","last_active":"2023-11-28T18:23:38.067829+0000","last_peered":"2023-11-28T18:23:38.067829+0000","last_clean":"2023-11-28T18:23:38.067829+0000","last_became_active":"2023-11-28T18:23:33.933692+0000","last_became_peered":"2023-11-28T18:23:33.933692+0000","last_unstale":"2023-11-28T18:23:38.067829+0000","last_undegraded":"2023-11-28T18:23:38.067829+0000","last_fullsized":"2023-11-28T18:23:38.067829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:38.277787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068388+0000","last_change":"2023-11-28T18:23:38.089272+0000","last_active":"2023-11-28T18:23:39.068388+0000","last_peered":"2023-11-28T18:23:39.068388+0000","last_clean":"2023-11-28T18:23:39.068388+0000","last_became_active":"2023-11-28T18:23:38.088843+0000","last_became_peered":"2023-11-28T18:23:38.088843+0000","last_unstale":"2023-11-28T18:23:39.068388+0000","last_undegraded":"2023-11-28T18:23:39.068388+0000","last_fullsized":"2023-11-28T18:23:39.068388+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:50.581936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.073981+0000","last_change":"2023-11-28T18:23:33.945002+0000","last_active":"2023-11-28T18:23:38.073981+0000","last_peered":"2023-11-28T18:23:38.073981+0000","last_clean":"2023-11-28T18:23:38.073981+0000","last_became_active":"2023-11-28T18:23:33.942488+0000","last_became_peered":"2023-11-28T18:23:33.942488+0000","last_unstale":"2023-11-28T18:23:38.073981+0000","last_undegraded":"2023-11-28T18:23:38.073981+0000","last_fullsized":"2023-11-28T18:23:38.073981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:04:56.118542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419933+0000","last_change":"2023-11-28T18:23:32.938369+0000","last_active":"2023-11-28T18:23:38.419933+0000","last_peered":"2023-11-28T18:23:38.419933+0000","last_clean":"2023-11-28T18:23:38.419933+0000","last_became_active":"2023-11-28T18:23:32.938126+0000","last_became_peered":"2023-11-28T18:23:32.938126+0000","last_unstale":"2023-11-28T18:23:38.419933+0000","last_undegraded":"2023-11-28T18:23:38.419933+0000","last_fullsized":"2023-11-28T18:23:38.419933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:50.646527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.069187+0000","last_change":"2023-11-28T18:23:38.093734+0000","last_active":"2023-11-28T18:23:39.069187+0000","last_peered":"2023-11-28T18:23:39.069187+0000","last_clean":"2023-11-28T18:23:39.069187+0000","last_became_active":"2023-11-28T18:23:38.093298+0000","last_became_peered":"2023-11-28T18:23:38.093298+0000","last_unstale":"2023-11-28T18:23:39.069187+0000","last_undegraded":"2023-11-28T18:23:39.069187+0000","last_fullsized":"2023-11-28T18:23:39.069187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:22:41.524911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065575+0000","last_change":"2023-11-28T18:23:33.938245+0000","last_active":"2023-11-28T18:23:38.065575+0000","last_peered":"2023-11-28T18:23:38.065575+0000","last_clean":"2023-11-28T18:23:38.065575+0000","last_became_active":"2023-11-28T18:23:33.937832+0000","last_became_peered":"2023-11-28T18:23:33.937832+0000","last_unstale":"2023-11-28T18:23:38.065575+0000","last_undegraded":"2023-11-28T18:23:38.065575+0000","last_fullsized":"2023-11-28T18:23:38.065575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:04:56.118542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.093039+0000","last_change":"2023-11-28T18:23:32.924880+0000","last_active":"2023-11-28T18:23:38.093039+0000","last_peered":"2023-11-28T18:23:38.093039+0000","last_clean":"2023-11-28T18:23:38.093039+0000","last_became_active":"2023-11-28T18:23:32.923804+0000","last_became_peered":"2023-11-28T18:23:32.923804+0000","last_unstale":"2023-11-28T18:23:38.093039+0000","last_undegraded":"2023-11-28T18:23:38.093039+0000","last_fullsized":"2023-11-28T18:23:38.093039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:10.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068797+0000","last_change":"2023-11-28T18:23:38.091628+0000","last_active":"2023-11-28T18:23:39.068797+0000","last_peered":"2023-11-28T18:23:39.068797+0000","last_clean":"2023-11-28T18:23:39.068797+0000","last_became_active":"2023-11-28T18:23:38.090900+0000","last_became_peered":"2023-11-28T18:23:38.090900+0000","last_unstale":"2023-11-28T18:23:39.068797+0000","last_undegraded":"2023-11-28T18:23:39.068797+0000","last_fullsized":"2023-11-28T18:23:39.068797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:47.152718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065511+0000","last_change":"2023-11-28T18:23:33.933391+0000","last_active":"2023-11-28T18:23:38.065511+0000","last_peered":"2023-11-28T18:23:38.065511+0000","last_clean":"2023-11-28T18:23:38.065511+0000","last_became_active":"2023-11-28T18:23:33.932767+0000","last_became_peered":"2023-11-28T18:23:33.932767+0000","last_unstale":"2023-11-28T18:23:38.065511+0000","last_undegraded":"2023-11-28T18:23:38.065511+0000","last_fullsized":"2023-11-28T18:23:38.065511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:21.035422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.092828+0000","last_change":"2023-11-28T18:23:32.929281+0000","last_active":"2023-11-28T18:23:38.092828+0000","last_peered":"2023-11-28T18:23:38.092828+0000","last_clean":"2023-11-28T18:23:38.092828+0000","last_became_active":"2023-11-28T18:23:32.928997+0000","last_became_peered":"2023-11-28T18:23:32.928997+0000","last_unstale":"2023-11-28T18:23:38.092828+0000","last_undegraded":"2023-11-28T18:23:38.092828+0000","last_fullsized":"2023-11-28T18:23:38.092828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:45.143373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.718883+0000","last_change":"2023-11-28T18:23:38.426592+0000","last_active":"2023-11-28T18:23:40.718883+0000","last_peered":"2023-11-28T18:23:40.718883+0000","last_clean":"2023-11-28T18:23:40.718883+0000","last_became_active":"2023-11-28T18:23:38.426089+0000","last_became_peered":"2023-11-28T18:23:38.426089+0000","last_unstale":"2023-11-28T18:23:40.718883+0000","last_undegraded":"2023-11-28T18:23:40.718883+0000","last_fullsized":"2023-11-28T18:23:40.718883+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:15.599832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075007+0000","last_change":"2023-11-28T18:23:33.935238+0000","last_active":"2023-11-28T18:23:38.075007+0000","last_peered":"2023-11-28T18:23:38.075007+0000","last_clean":"2023-11-28T18:23:38.075007+0000","last_became_active":"2023-11-28T18:23:33.934884+0000","last_became_peered":"2023-11-28T18:23:33.934884+0000","last_unstale":"2023-11-28T18:23:38.075007+0000","last_undegraded":"2023-11-28T18:23:38.075007+0000","last_fullsized":"2023-11-28T18:23:38.075007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:58.099391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419489+0000","last_change":"2023-11-28T18:23:32.933485+0000","last_active":"2023-11-28T18:23:38.419489+0000","last_peered":"2023-11-28T18:23:38.419489+0000","last_clean":"2023-11-28T18:23:38.419489+0000","last_became_active":"2023-11-28T18:23:32.932807+0000","last_became_peered":"2023-11-28T18:23:32.932807+0000","last_unstale":"2023-11-28T18:23:38.419489+0000","last_undegraded":"2023-11-28T18:23:38.419489+0000","last_fullsized":"2023-11-28T18:23:38.419489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:00.325734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.069240+0000","last_change":"2023-11-28T18:23:38.091553+0000","last_active":"2023-11-28T18:23:39.069240+0000","last_peered":"2023-11-28T18:23:39.069240+0000","last_clean":"2023-11-28T18:23:39.069240+0000","last_became_active":"2023-11-28T18:23:38.091133+0000","last_became_peered":"2023-11-28T18:23:38.091133+0000","last_unstale":"2023-11-28T18:23:39.069240+0000","last_undegraded":"2023-11-28T18:23:39.069240+0000","last_fullsized":"2023-11-28T18:23:39.069240+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:53.274366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.080431+0000","last_change":"2023-11-28T18:23:33.941025+0000","last_active":"2023-11-28T18:23:38.080431+0000","last_peered":"2023-11-28T18:23:38.080431+0000","last_clean":"2023-11-28T18:23:38.080431+0000","last_became_active":"2023-11-28T18:23:33.940714+0000","last_became_peered":"2023-11-28T18:23:33.940714+0000","last_unstale":"2023-11-28T18:23:38.080431+0000","last_undegraded":"2023-11-28T18:23:38.080431+0000","last_fullsized":"2023-11-28T18:23:38.080431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:01.912695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077789+0000","last_change":"2023-11-28T18:23:32.933239+0000","last_active":"2023-11-28T18:23:38.077789+0000","last_peered":"2023-11-28T18:23:38.077789+0000","last_clean":"2023-11-28T18:23:38.077789+0000","last_became_active":"2023-11-28T18:23:32.932935+0000","last_became_peered":"2023-11-28T18:23:32.932935+0000","last_unstale":"2023-11-28T18:23:38.077789+0000","last_undegraded":"2023-11-28T18:23:38.077789+0000","last_fullsized":"2023-11-28T18:23:38.077789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:36:09.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068440+0000","last_change":"2023-11-28T18:23:38.421858+0000","last_active":"2023-11-28T18:23:39.068440+0000","last_peered":"2023-11-28T18:23:39.068440+0000","last_clean":"2023-11-28T18:23:39.068440+0000","last_became_active":"2023-11-28T18:23:38.421416+0000","last_became_peered":"2023-11-28T18:23:38.421416+0000","last_unstale":"2023-11-28T18:23:39.068440+0000","last_undegraded":"2023-11-28T18:23:39.068440+0000","last_fullsized":"2023-11-28T18:23:39.068440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:31.667216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067068+0000","last_change":"2023-11-28T18:23:33.940147+0000","last_active":"2023-11-28T18:23:38.067068+0000","last_peered":"2023-11-28T18:23:38.067068+0000","last_clean":"2023-11-28T18:23:38.067068+0000","last_became_active":"2023-11-28T18:23:33.939882+0000","last_became_peered":"2023-11-28T18:23:33.939882+0000","last_unstale":"2023-11-28T18:23:38.067068+0000","last_undegraded":"2023-11-28T18:23:38.067068+0000","last_fullsized":"2023-11-28T18:23:38.067068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:02.989816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.096365+0000","last_change":"2023-11-28T18:23:32.925741+0000","last_active":"2023-11-28T18:23:38.096365+0000","last_peered":"2023-11-28T18:23:38.096365+0000","last_clean":"2023-11-28T18:23:38.096365+0000","last_became_active":"2023-11-28T18:23:32.925344+0000","last_became_peered":"2023-11-28T18:23:32.925344+0000","last_unstale":"2023-11-28T18:23:38.096365+0000","last_undegraded":"2023-11-28T18:23:38.096365+0000","last_fullsized":"2023-11-28T18:23:38.096365+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:41.820325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.067516+0000","last_change":"2023-11-28T18:23:38.421716+0000","last_active":"2023-11-28T18:23:39.067516+0000","last_peered":"2023-11-28T18:23:39.067516+0000","last_clean":"2023-11-28T18:23:39.067516+0000","last_became_active":"2023-11-28T18:23:38.421313+0000","last_became_peered":"2023-11-28T18:23:38.421313+0000","last_unstale":"2023-11-28T18:23:39.067516+0000","last_undegraded":"2023-11-28T18:23:39.067516+0000","last_fullsized":"2023-11-28T18:23:39.067516+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:12:36.678267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.094519+0000","last_change":"2023-11-28T18:23:33.934187+0000","last_active":"2023-11-28T18:23:38.094519+0000","last_peered":"2023-11-28T18:23:38.094519+0000","last_clean":"2023-11-28T18:23:38.094519+0000","last_became_active":"2023-11-28T18:23:33.933885+0000","last_became_peered":"2023-11-28T18:23:33.933885+0000","last_unstale":"2023-11-28T18:23:38.094519+0000","last_undegraded":"2023-11-28T18:23:38.094519+0000","last_fullsized":"2023-11-28T18:23:38.094519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:21.035422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420442+0000","last_change":"2023-11-28T18:23:32.933434+0000","last_active":"2023-11-28T18:23:38.420442+0000","last_peered":"2023-11-28T18:23:38.420442+0000","last_clean":"2023-11-28T18:23:38.420442+0000","last_became_active":"2023-11-28T18:23:32.933143+0000","last_became_peered":"2023-11-28T18:23:32.933143+0000","last_unstale":"2023-11-28T18:23:38.420442+0000","last_undegraded":"2023-11-28T18:23:38.420442+0000","last_fullsized":"2023-11-28T18:23:38.420442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:11.551517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.297189+0000","last_change":"2023-11-28T18:23:38.422070+0000","last_active":"2023-11-28T18:23:40.297189+0000","last_peered":"2023-11-28T18:23:40.297189+0000","last_clean":"2023-11-28T18:23:40.297189+0000","last_became_active":"2023-11-28T18:23:38.421240+0000","last_became_peered":"2023-11-28T18:23:38.421240+0000","last_unstale":"2023-11-28T18:23:40.297189+0000","last_undegraded":"2023-11-28T18:23:40.297189+0000","last_fullsized":"2023-11-28T18:23:40.297189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:42.438182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067926+0000","last_change":"2023-11-28T18:23:33.932685+0000","last_active":"2023-11-28T18:23:38.067926+0000","last_peered":"2023-11-28T18:23:38.067926+0000","last_clean":"2023-11-28T18:23:38.067926+0000","last_became_active":"2023-11-28T18:23:33.932342+0000","last_became_peered":"2023-11-28T18:23:33.932342+0000","last_unstale":"2023-11-28T18:23:38.067926+0000","last_undegraded":"2023-11-28T18:23:38.067926+0000","last_fullsized":"2023-11-28T18:23:38.067926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:49.113970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.094608+0000","last_change":"2023-11-28T18:23:32.923234+0000","last_active":"2023-11-28T18:23:38.094608+0000","last_peered":"2023-11-28T18:23:38.094608+0000","last_clean":"2023-11-28T18:23:38.094608+0000","last_became_active":"2023-11-28T18:23:32.922051+0000","last_became_peered":"2023-11-28T18:23:32.922051+0000","last_unstale":"2023-11-28T18:23:38.094608+0000","last_undegraded":"2023-11-28T18:23:38.094608+0000","last_fullsized":"2023-11-28T18:23:38.094608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:36:09.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.718350+0000","last_change":"2023-11-28T18:23:38.102307+0000","last_active":"2023-11-28T18:23:40.718350+0000","last_peered":"2023-11-28T18:23:40.718350+0000","last_clean":"2023-11-28T18:23:40.718350+0000","last_became_active":"2023-11-28T18:23:38.101987+0000","last_became_peered":"2023-11-28T18:23:38.101987+0000","last_unstale":"2023-11-28T18:23:40.718350+0000","last_undegraded":"2023-11-28T18:23:40.718350+0000","last_fullsized":"2023-11-28T18:23:40.718350+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:22:41.524911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.076427+0000","last_change":"2023-11-28T18:23:33.941492+0000","last_active":"2023-11-28T18:23:38.076427+0000","last_peered":"2023-11-28T18:23:38.076427+0000","last_clean":"2023-11-28T18:23:38.076427+0000","last_became_active":"2023-11-28T18:23:33.941148+0000","last_became_peered":"2023-11-28T18:23:33.941148+0000","last_unstale":"2023-11-28T18:23:38.076427+0000","last_undegraded":"2023-11-28T18:23:38.076427+0000","last_fullsized":"2023-11-28T18:23:38.076427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:29:23.426289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068690+0000","last_change":"2023-11-28T18:23:32.922032+0000","last_active":"2023-11-28T18:23:38.068690+0000","last_peered":"2023-11-28T18:23:38.068690+0000","last_clean":"2023-11-28T18:23:38.068690+0000","last_became_active":"2023-11-28T18:23:32.921635+0000","last_became_peered":"2023-11-28T18:23:32.921635+0000","last_unstale":"2023-11-28T18:23:38.068690+0000","last_undegraded":"2023-11-28T18:23:38.068690+0000","last_fullsized":"2023-11-28T18:23:38.068690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:36:09.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.718745+0000","last_change":"2023-11-28T18:23:38.096609+0000","last_active":"2023-11-28T18:23:40.718745+0000","last_peered":"2023-11-28T18:23:40.718745+0000","last_clean":"2023-11-28T18:23:40.718745+0000","last_became_active":"2023-11-28T18:23:38.095758+0000","last_became_peered":"2023-11-28T18:23:38.095758+0000","last_unstale":"2023-11-28T18:23:40.718745+0000","last_undegraded":"2023-11-28T18:23:40.718745+0000","last_fullsized":"2023-11-28T18:23:40.718745+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:42.438182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.090061+0000","last_change":"2023-11-28T18:23:33.937266+0000","last_active":"2023-11-28T18:23:38.090061+0000","last_peered":"2023-11-28T18:23:38.090061+0000","last_clean":"2023-11-28T18:23:38.090061+0000","last_became_active":"2023-11-28T18:23:33.936994+0000","last_became_peered":"2023-11-28T18:23:33.936994+0000","last_unstale":"2023-11-28T18:23:38.090061+0000","last_undegraded":"2023-11-28T18:23:38.090061+0000","last_fullsized":"2023-11-28T18:23:38.090061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:12.557669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.083701+0000","last_change":"2023-11-28T18:23:32.918591+0000","last_active":"2023-11-28T18:23:38.083701+0000","last_peered":"2023-11-28T18:23:38.083701+0000","last_clean":"2023-11-28T18:23:38.083701+0000","last_became_active":"2023-11-28T18:23:32.915381+0000","last_became_peered":"2023-11-28T18:23:32.915381+0000","last_unstale":"2023-11-28T18:23:38.083701+0000","last_undegraded":"2023-11-28T18:23:38.083701+0000","last_fullsized":"2023-11-28T18:23:38.083701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:57.093239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.718851+0000","last_change":"2023-11-28T18:23:38.100344+0000","last_active":"2023-11-28T18:23:40.718851+0000","last_peered":"2023-11-28T18:23:40.718851+0000","last_clean":"2023-11-28T18:23:40.718851+0000","last_became_active":"2023-11-28T18:23:38.099934+0000","last_became_peered":"2023-11-28T18:23:38.099934+0000","last_unstale":"2023-11-28T18:23:40.718851+0000","last_undegraded":"2023-11-28T18:23:40.718851+0000","last_fullsized":"2023-11-28T18:23:40.718851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:39.149571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.074486+0000","last_change":"2023-11-28T18:23:33.944704+0000","last_active":"2023-11-28T18:23:38.074486+0000","last_peered":"2023-11-28T18:23:38.074486+0000","last_clean":"2023-11-28T18:23:38.074486+0000","last_became_active":"2023-11-28T18:23:33.944484+0000","last_became_peered":"2023-11-28T18:23:33.944484+0000","last_unstale":"2023-11-28T18:23:38.074486+0000","last_undegraded":"2023-11-28T18:23:38.074486+0000","last_fullsized":"2023-11-28T18:23:38.074486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:55:27.915128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419489+0000","last_change":"2023-11-28T18:23:32.933677+0000","last_active":"2023-11-28T18:23:38.419489+0000","last_peered":"2023-11-28T18:23:38.419489+0000","last_clean":"2023-11-28T18:23:38.419489+0000","last_became_active":"2023-11-28T18:23:32.933168+0000","last_became_peered":"2023-11-28T18:23:32.933168+0000","last_unstale":"2023-11-28T18:23:38.419489+0000","last_undegraded":"2023-11-28T18:23:38.419489+0000","last_fullsized":"2023-11-28T18:23:38.419489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:29:22.420137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.297576+0000","last_change":"2023-11-28T18:23:38.427376+0000","last_active":"2023-11-28T18:23:40.297576+0000","last_peered":"2023-11-28T18:23:40.297576+0000","last_clean":"2023-11-28T18:23:40.297576+0000","last_became_active":"2023-11-28T18:23:38.426436+0000","last_became_peered":"2023-11-28T18:23:38.426436+0000","last_unstale":"2023-11-28T18:23:40.297576+0000","last_undegraded":"2023-11-28T18:23:40.297576+0000","last_fullsized":"2023-11-28T18:23:40.297576+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:19:23.246961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065098+0000","last_change":"2023-11-28T18:23:33.932836+0000","last_active":"2023-11-28T18:23:38.065098+0000","last_peered":"2023-11-28T18:23:38.065098+0000","last_clean":"2023-11-28T18:23:38.065098+0000","last_became_active":"2023-11-28T18:23:33.932534+0000","last_became_peered":"2023-11-28T18:23:33.932534+0000","last_unstale":"2023-11-28T18:23:38.065098+0000","last_undegraded":"2023-11-28T18:23:38.065098+0000","last_fullsized":"2023-11-28T18:23:38.065098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:46.421541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067421+0000","last_change":"2023-11-28T18:23:32.922842+0000","last_active":"2023-11-28T18:23:38.067421+0000","last_peered":"2023-11-28T18:23:38.067421+0000","last_clean":"2023-11-28T18:23:38.067421+0000","last_became_active":"2023-11-28T18:23:32.922541+0000","last_became_peered":"2023-11-28T18:23:32.922541+0000","last_unstale":"2023-11-28T18:23:38.067421+0000","last_undegraded":"2023-11-28T18:23:38.067421+0000","last_fullsized":"2023-11-28T18:23:38.067421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:39:52.964875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419571+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:23:38.419571+0000","last_peered":"2023-11-28T18:23:38.419571+0000","last_clean":"2023-11-28T18:23:38.419571+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:23:38.419571+0000","last_undegraded":"2023-11-28T18:23:38.419571+0000","last_fullsized":"2023-11-28T18:23:38.419571+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.719264+0000","last_change":"2023-11-28T18:23:38.427043+0000","last_active":"2023-11-28T18:23:40.719264+0000","last_peered":"2023-11-28T18:23:40.719264+0000","last_clean":"2023-11-28T18:23:40.719264+0000","last_became_active":"2023-11-28T18:23:38.426613+0000","last_became_peered":"2023-11-28T18:23:38.426613+0000","last_unstale":"2023-11-28T18:23:40.719264+0000","last_undegraded":"2023-11-28T18:23:40.719264+0000","last_fullsized":"2023-11-28T18:23:40.719264+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:47.152718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419993+0000","last_change":"2023-11-28T18:23:33.944370+0000","last_active":"2023-11-28T18:23:38.419993+0000","last_peered":"2023-11-28T18:23:38.419993+0000","last_clean":"2023-11-28T18:23:38.419993+0000","last_became_active":"2023-11-28T18:23:33.943698+0000","last_became_peered":"2023-11-28T18:23:33.943698+0000","last_unstale":"2023-11-28T18:23:38.419993+0000","last_undegraded":"2023-11-28T18:23:38.419993+0000","last_fullsized":"2023-11-28T18:23:38.419993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:49.113970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.083176+0000","last_change":"2023-11-28T18:23:32.917731+0000","last_active":"2023-11-28T18:23:38.083176+0000","last_peered":"2023-11-28T18:23:38.083176+0000","last_clean":"2023-11-28T18:23:38.083176+0000","last_became_active":"2023-11-28T18:23:32.917207+0000","last_became_peered":"2023-11-28T18:23:32.917207+0000","last_unstale":"2023-11-28T18:23:38.083176+0000","last_undegraded":"2023-11-28T18:23:38.083176+0000","last_fullsized":"2023-11-28T18:23:38.083176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:38:50.132097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.719164+0000","last_change":"2023-11-28T18:23:38.425300+0000","last_active":"2023-11-28T18:23:40.719164+0000","last_peered":"2023-11-28T18:23:40.719164+0000","last_clean":"2023-11-28T18:23:40.719164+0000","last_became_active":"2023-11-28T18:23:38.424708+0000","last_became_peered":"2023-11-28T18:23:38.424708+0000","last_unstale":"2023-11-28T18:23:40.719164+0000","last_undegraded":"2023-11-28T18:23:40.719164+0000","last_fullsized":"2023-11-28T18:23:40.719164+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:07.150212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067866+0000","last_change":"2023-11-28T18:23:33.932266+0000","last_active":"2023-11-28T18:23:38.067866+0000","last_peered":"2023-11-28T18:23:38.067866+0000","last_clean":"2023-11-28T18:23:38.067866+0000","last_became_active":"2023-11-28T18:23:33.931923+0000","last_became_peered":"2023-11-28T18:23:33.931923+0000","last_unstale":"2023-11-28T18:23:38.067866+0000","last_undegraded":"2023-11-28T18:23:38.067866+0000","last_fullsized":"2023-11-28T18:23:38.067866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:34.989175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.094421+0000","last_change":"2023-11-28T18:23:32.924719+0000","last_active":"2023-11-28T18:23:38.094421+0000","last_peered":"2023-11-28T18:23:38.094421+0000","last_clean":"2023-11-28T18:23:38.094421+0000","last_became_active":"2023-11-28T18:23:32.924462+0000","last_became_peered":"2023-11-28T18:23:32.924462+0000","last_unstale":"2023-11-28T18:23:38.094421+0000","last_undegraded":"2023-11-28T18:23:38.094421+0000","last_fullsized":"2023-11-28T18:23:38.094421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:55:26.908976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.297354+0000","last_change":"2023-11-28T18:23:38.093163+0000","last_active":"2023-11-28T18:23:40.297354+0000","last_peered":"2023-11-28T18:23:40.297354+0000","last_clean":"2023-11-28T18:23:40.297354+0000","last_became_active":"2023-11-28T18:23:38.092754+0000","last_became_peered":"2023-11-28T18:23:38.092754+0000","last_unstale":"2023-11-28T18:23:40.297354+0000","last_undegraded":"2023-11-28T18:23:40.297354+0000","last_fullsized":"2023-11-28T18:23:40.297354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:39.149571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420505+0000","last_change":"2023-11-28T18:23:33.954652+0000","last_active":"2023-11-28T18:23:38.420505+0000","last_peered":"2023-11-28T18:23:38.420505+0000","last_clean":"2023-11-28T18:23:38.420505+0000","last_became_active":"2023-11-28T18:23:33.954369+0000","last_became_peered":"2023-11-28T18:23:33.954369+0000","last_unstale":"2023-11-28T18:23:38.420505+0000","last_undegraded":"2023-11-28T18:23:38.420505+0000","last_fullsized":"2023-11-28T18:23:38.420505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:30:56.523444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.084061+0000","last_change":"2023-11-28T18:23:32.906348+0000","last_active":"2023-11-28T18:23:38.084061+0000","last_peered":"2023-11-28T18:23:38.084061+0000","last_clean":"2023-11-28T18:23:38.084061+0000","last_became_active":"2023-11-28T18:23:32.906007+0000","last_became_peered":"2023-11-28T18:23:32.906007+0000","last_unstale":"2023-11-28T18:23:38.084061+0000","last_undegraded":"2023-11-28T18:23:38.084061+0000","last_fullsized":"2023-11-28T18:23:38.084061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:00.325734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.719440+0000","last_change":"2023-11-28T18:23:38.426316+0000","last_active":"2023-11-28T18:23:40.719440+0000","last_peered":"2023-11-28T18:23:40.719440+0000","last_clean":"2023-11-28T18:23:40.719440+0000","last_became_active":"2023-11-28T18:23:38.425328+0000","last_became_peered":"2023-11-28T18:23:38.425328+0000","last_unstale":"2023-11-28T18:23:40.719440+0000","last_undegraded":"2023-11-28T18:23:40.719440+0000","last_fullsized":"2023-11-28T18:23:40.719440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:19:51.686256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420058+0000","last_change":"2023-11-28T18:23:33.954315+0000","last_active":"2023-11-28T18:23:38.420058+0000","last_peered":"2023-11-28T18:23:38.420058+0000","last_clean":"2023-11-28T18:23:38.420058+0000","last_became_active":"2023-11-28T18:23:33.950270+0000","last_became_peered":"2023-11-28T18:23:33.950270+0000","last_unstale":"2023-11-28T18:23:38.420058+0000","last_undegraded":"2023-11-28T18:23:38.420058+0000","last_fullsized":"2023-11-28T18:23:38.420058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:22:37.364515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077169+0000","last_change":"2023-11-28T18:23:32.921145+0000","last_active":"2023-11-28T18:23:38.077169+0000","last_peered":"2023-11-28T18:23:38.077169+0000","last_clean":"2023-11-28T18:23:38.077169+0000","last_became_active":"2023-11-28T18:23:32.920706+0000","last_became_peered":"2023-11-28T18:23:32.920706+0000","last_unstale":"2023-11-28T18:23:38.077169+0000","last_undegraded":"2023-11-28T18:23:38.077169+0000","last_fullsized":"2023-11-28T18:23:38.077169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:39:52.964875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068294+0000","last_change":"2023-11-28T18:23:38.096089+0000","last_active":"2023-11-28T18:23:39.068294+0000","last_peered":"2023-11-28T18:23:39.068294+0000","last_clean":"2023-11-28T18:23:39.068294+0000","last_became_active":"2023-11-28T18:23:38.095672+0000","last_became_peered":"2023-11-28T18:23:38.095672+0000","last_unstale":"2023-11-28T18:23:39.068294+0000","last_undegraded":"2023-11-28T18:23:39.068294+0000","last_fullsized":"2023-11-28T18:23:39.068294+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:09.933678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419670+0000","last_change":"2023-11-28T18:23:33.954352+0000","last_active":"2023-11-28T18:23:38.419670+0000","last_peered":"2023-11-28T18:23:38.419670+0000","last_clean":"2023-11-28T18:23:38.419670+0000","last_became_active":"2023-11-28T18:23:33.950344+0000","last_became_peered":"2023-11-28T18:23:33.950344+0000","last_unstale":"2023-11-28T18:23:38.419670+0000","last_undegraded":"2023-11-28T18:23:38.419670+0000","last_fullsized":"2023-11-28T18:23:38.419670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:08.033209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.078193+0000","last_change":"2023-11-28T18:23:32.919741+0000","last_active":"2023-11-28T18:23:38.078193+0000","last_peered":"2023-11-28T18:23:38.078193+0000","last_clean":"2023-11-28T18:23:38.078193+0000","last_became_active":"2023-11-28T18:23:32.919438+0000","last_became_peered":"2023-11-28T18:23:32.919438+0000","last_unstale":"2023-11-28T18:23:38.078193+0000","last_undegraded":"2023-11-28T18:23:38.078193+0000","last_fullsized":"2023-11-28T18:23:38.078193+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:39:52.964875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.297438+0000","last_change":"2023-11-28T18:23:38.084984+0000","last_active":"2023-11-28T18:23:40.297438+0000","last_peered":"2023-11-28T18:23:40.297438+0000","last_clean":"2023-11-28T18:23:40.297438+0000","last_became_active":"2023-11-28T18:23:38.084490+0000","last_became_peered":"2023-11-28T18:23:38.084490+0000","last_unstale":"2023-11-28T18:23:40.297438+0000","last_undegraded":"2023-11-28T18:23:40.297438+0000","last_fullsized":"2023-11-28T18:23:40.297438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:00.683839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.073493+0000","last_change":"2023-11-28T18:23:33.939191+0000","last_active":"2023-11-28T18:23:38.073493+0000","last_peered":"2023-11-28T18:23:38.073493+0000","last_clean":"2023-11-28T18:23:38.073493+0000","last_became_active":"2023-11-28T18:23:33.938295+0000","last_became_peered":"2023-11-28T18:23:33.938295+0000","last_unstale":"2023-11-28T18:23:38.073493+0000","last_undegraded":"2023-11-28T18:23:38.073493+0000","last_fullsized":"2023-11-28T18:23:38.073493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:12.557669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.073721+0000","last_change":"2023-11-28T18:23:32.930354+0000","last_active":"2023-11-28T18:23:38.073721+0000","last_peered":"2023-11-28T18:23:38.073721+0000","last_clean":"2023-11-28T18:23:38.073721+0000","last_became_active":"2023-11-28T18:23:32.930059+0000","last_became_peered":"2023-11-28T18:23:32.930059+0000","last_unstale":"2023-11-28T18:23:38.073721+0000","last_undegraded":"2023-11-28T18:23:38.073721+0000","last_fullsized":"2023-11-28T18:23:38.073721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:45.143373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.069210+0000","last_change":"2023-11-28T18:23:38.427077+0000","last_active":"2023-11-28T18:23:39.069210+0000","last_peered":"2023-11-28T18:23:39.069210+0000","last_clean":"2023-11-28T18:23:39.069210+0000","last_became_active":"2023-11-28T18:23:38.426613+0000","last_became_peered":"2023-11-28T18:23:38.426613+0000","last_unstale":"2023-11-28T18:23:39.069210+0000","last_undegraded":"2023-11-28T18:23:39.069210+0000","last_fullsized":"2023-11-28T18:23:39.069210+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:24:11.818911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419843+0000","last_change":"2023-11-28T18:23:33.955340+0000","last_active":"2023-11-28T18:23:38.419843+0000","last_peered":"2023-11-28T18:23:38.419843+0000","last_clean":"2023-11-28T18:23:38.419843+0000","last_became_active":"2023-11-28T18:23:33.955080+0000","last_became_peered":"2023-11-28T18:23:33.955080+0000","last_unstale":"2023-11-28T18:23:38.419843+0000","last_undegraded":"2023-11-28T18:23:38.419843+0000","last_fullsized":"2023-11-28T18:23:38.419843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:42.992322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068633+0000","last_change":"2023-11-28T18:23:32.935530+0000","last_active":"2023-11-28T18:23:38.068633+0000","last_peered":"2023-11-28T18:23:38.068633+0000","last_clean":"2023-11-28T18:23:38.068633+0000","last_became_active":"2023-11-28T18:23:32.934513+0000","last_became_peered":"2023-11-28T18:23:32.934513+0000","last_unstale":"2023-11-28T18:23:38.068633+0000","last_undegraded":"2023-11-28T18:23:38.068633+0000","last_fullsized":"2023-11-28T18:23:38.068633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:10.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068541+0000","last_change":"2023-11-28T18:23:38.421813+0000","last_active":"2023-11-28T18:23:39.068541+0000","last_peered":"2023-11-28T18:23:39.068541+0000","last_clean":"2023-11-28T18:23:39.068541+0000","last_became_active":"2023-11-28T18:23:38.421280+0000","last_became_peered":"2023-11-28T18:23:38.421280+0000","last_unstale":"2023-11-28T18:23:39.068541+0000","last_undegraded":"2023-11-28T18:23:39.068541+0000","last_fullsized":"2023-11-28T18:23:39.068541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:19:05.111022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.080161+0000","last_change":"2023-11-28T18:23:33.946537+0000","last_active":"2023-11-28T18:23:38.080161+0000","last_peered":"2023-11-28T18:23:38.080161+0000","last_clean":"2023-11-28T18:23:38.080161+0000","last_became_active":"2023-11-28T18:23:33.946220+0000","last_became_peered":"2023-11-28T18:23:33.946220+0000","last_unstale":"2023-11-28T18:23:38.080161+0000","last_undegraded":"2023-11-28T18:23:38.080161+0000","last_fullsized":"2023-11-28T18:23:38.080161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:51.652679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077792+0000","last_change":"2023-11-28T18:23:32.922020+0000","last_active":"2023-11-28T18:23:38.077792+0000","last_peered":"2023-11-28T18:23:38.077792+0000","last_clean":"2023-11-28T18:23:38.077792+0000","last_became_active":"2023-11-28T18:23:32.921656+0000","last_became_peered":"2023-11-28T18:23:32.921656+0000","last_unstale":"2023-11-28T18:23:38.077792+0000","last_undegraded":"2023-11-28T18:23:38.077792+0000","last_fullsized":"2023-11-28T18:23:38.077792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:10.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.721071+0000","last_change":"2023-11-28T18:23:38.421823+0000","last_active":"2023-11-28T18:23:40.721071+0000","last_peered":"2023-11-28T18:23:40.721071+0000","last_clean":"2023-11-28T18:23:40.721071+0000","last_became_active":"2023-11-28T18:23:38.421407+0000","last_became_peered":"2023-11-28T18:23:38.421407+0000","last_unstale":"2023-11-28T18:23:40.721071+0000","last_undegraded":"2023-11-28T18:23:40.721071+0000","last_fullsized":"2023-11-28T18:23:40.721071+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:01.885046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419892+0000","last_change":"2023-11-28T18:23:33.939629+0000","last_active":"2023-11-28T18:23:38.419892+0000","last_peered":"2023-11-28T18:23:38.419892+0000","last_clean":"2023-11-28T18:23:38.419892+0000","last_became_active":"2023-11-28T18:23:33.938662+0000","last_became_peered":"2023-11-28T18:23:33.938662+0000","last_unstale":"2023-11-28T18:23:38.419892+0000","last_undegraded":"2023-11-28T18:23:38.419892+0000","last_fullsized":"2023-11-28T18:23:38.419892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:02.989816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'8","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:42.888210+0000","last_change":"2023-11-28T18:23:32.923661+0000","last_active":"2023-11-28T18:23:42.888210+0000","last_peered":"2023-11-28T18:23:42.888210+0000","last_clean":"2023-11-28T18:23:42.888210+0000","last_became_active":"2023-11-28T18:23:32.922407+0000","last_became_peered":"2023-11-28T18:23:32.922407+0000","last_unstale":"2023-11-28T18:23:42.888210+0000","last_undegraded":"2023-11-28T18:23:42.888210+0000","last_fullsized":"2023-11-28T18:23:42.888210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:10.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3700,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.718701+0000","last_change":"2023-11-28T18:23:38.098761+0000","last_active":"2023-11-28T18:23:40.718701+0000","last_peered":"2023-11-28T18:23:40.718701+0000","last_clean":"2023-11-28T18:23:40.718701+0000","last_became_active":"2023-11-28T18:23:38.098374+0000","last_became_peered":"2023-11-28T18:23:38.098374+0000","last_unstale":"2023-11-28T18:23:40.718701+0000","last_undegraded":"2023-11-28T18:23:40.718701+0000","last_fullsized":"2023-11-28T18:23:40.718701+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:53.274366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075827+0000","last_change":"2023-11-28T18:23:33.945418+0000","last_active":"2023-11-28T18:23:38.075827+0000","last_peered":"2023-11-28T18:23:38.075827+0000","last_clean":"2023-11-28T18:23:38.075827+0000","last_became_active":"2023-11-28T18:23:33.944949+0000","last_became_peered":"2023-11-28T18:23:33.944949+0000","last_unstale":"2023-11-28T18:23:38.075827+0000","last_undegraded":"2023-11-28T18:23:38.075827+0000","last_fullsized":"2023-11-28T18:23:38.075827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:42.826477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.094316+0000","last_change":"2023-11-28T18:23:32.925933+0000","last_active":"2023-11-28T18:23:38.094316+0000","last_peered":"2023-11-28T18:23:38.094316+0000","last_clean":"2023-11-28T18:23:38.094316+0000","last_became_active":"2023-11-28T18:23:32.925619+0000","last_became_peered":"2023-11-28T18:23:32.925619+0000","last_unstale":"2023-11-28T18:23:38.094316+0000","last_undegraded":"2023-11-28T18:23:38.094316+0000","last_fullsized":"2023-11-28T18:23:38.094316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:19.143213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068423+0000","last_change":"2023-11-28T18:23:38.091049+0000","last_active":"2023-11-28T18:23:39.068423+0000","last_peered":"2023-11-28T18:23:39.068423+0000","last_clean":"2023-11-28T18:23:39.068423+0000","last_became_active":"2023-11-28T18:23:38.090236+0000","last_became_peered":"2023-11-28T18:23:38.090236+0000","last_unstale":"2023-11-28T18:23:39.068423+0000","last_undegraded":"2023-11-28T18:23:39.068423+0000","last_fullsized":"2023-11-28T18:23:39.068423+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:12.193604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.076836+0000","last_change":"2023-11-28T18:23:33.948346+0000","last_active":"2023-11-28T18:23:38.076836+0000","last_peered":"2023-11-28T18:23:38.076836+0000","last_clean":"2023-11-28T18:23:38.076836+0000","last_became_active":"2023-11-28T18:23:33.947983+0000","last_became_peered":"2023-11-28T18:23:33.947983+0000","last_unstale":"2023-11-28T18:23:38.076836+0000","last_undegraded":"2023-11-28T18:23:38.076836+0000","last_fullsized":"2023-11-28T18:23:38.076836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:12.557669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067715+0000","last_change":"2023-11-28T18:23:32.935430+0000","last_active":"2023-11-28T18:23:38.067715+0000","last_peered":"2023-11-28T18:23:38.067715+0000","last_clean":"2023-11-28T18:23:38.067715+0000","last_became_active":"2023-11-28T18:23:32.930282+0000","last_became_peered":"2023-11-28T18:23:32.930282+0000","last_unstale":"2023-11-28T18:23:38.067715+0000","last_undegraded":"2023-11-28T18:23:38.067715+0000","last_fullsized":"2023-11-28T18:23:38.067715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:50.646527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.067746+0000","last_change":"2023-11-28T18:23:38.093851+0000","last_active":"2023-11-28T18:23:39.067746+0000","last_peered":"2023-11-28T18:23:39.067746+0000","last_clean":"2023-11-28T18:23:39.067746+0000","last_became_active":"2023-11-28T18:23:38.091779+0000","last_became_peered":"2023-11-28T18:23:38.091779+0000","last_unstale":"2023-11-28T18:23:39.067746+0000","last_undegraded":"2023-11-28T18:23:39.067746+0000","last_fullsized":"2023-11-28T18:23:39.067746+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:13:14.577345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067617+0000","last_change":"2023-11-28T18:23:33.952137+0000","last_active":"2023-11-28T18:23:38.067617+0000","last_peered":"2023-11-28T18:23:38.067617+0000","last_clean":"2023-11-28T18:23:38.067617+0000","last_became_active":"2023-11-28T18:23:33.950857+0000","last_became_peered":"2023-11-28T18:23:33.950857+0000","last_unstale":"2023-11-28T18:23:38.067617+0000","last_undegraded":"2023-11-28T18:23:38.067617+0000","last_fullsized":"2023-11-28T18:23:38.067617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:42.992322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077679+0000","last_change":"2023-11-28T18:23:32.940089+0000","last_active":"2023-11-28T18:23:38.077679+0000","last_peered":"2023-11-28T18:23:38.077679+0000","last_clean":"2023-11-28T18:23:38.077679+0000","last_became_active":"2023-11-28T18:23:32.939524+0000","last_became_peered":"2023-11-28T18:23:32.939524+0000","last_unstale":"2023-11-28T18:23:38.077679+0000","last_undegraded":"2023-11-28T18:23:38.077679+0000","last_fullsized":"2023-11-28T18:23:38.077679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:19.143213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.067936+0000","last_change":"2023-11-28T18:23:38.424675+0000","last_active":"2023-11-28T18:23:39.067936+0000","last_peered":"2023-11-28T18:23:39.067936+0000","last_clean":"2023-11-28T18:23:39.067936+0000","last_became_active":"2023-11-28T18:23:38.424284+0000","last_became_peered":"2023-11-28T18:23:38.424284+0000","last_unstale":"2023-11-28T18:23:39.067936+0000","last_undegraded":"2023-11-28T18:23:39.067936+0000","last_fullsized":"2023-11-28T18:23:39.067936+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:01.885046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.297416+0000","last_change":"2023-11-28T18:23:38.425611+0000","last_active":"2023-11-28T18:23:40.297416+0000","last_peered":"2023-11-28T18:23:40.297416+0000","last_clean":"2023-11-28T18:23:40.297416+0000","last_became_active":"2023-11-28T18:23:38.424826+0000","last_became_peered":"2023-11-28T18:23:38.424826+0000","last_unstale":"2023-11-28T18:23:40.297416+0000","last_undegraded":"2023-11-28T18:23:40.297416+0000","last_fullsized":"2023-11-28T18:23:40.297416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:15.599832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.092623+0000","last_change":"2023-11-28T18:23:33.937161+0000","last_active":"2023-11-28T18:23:38.092623+0000","last_peered":"2023-11-28T18:23:38.092623+0000","last_clean":"2023-11-28T18:23:38.092623+0000","last_became_active":"2023-11-28T18:23:33.936724+0000","last_became_peered":"2023-11-28T18:23:33.936724+0000","last_unstale":"2023-11-28T18:23:38.092623+0000","last_undegraded":"2023-11-28T18:23:38.092623+0000","last_fullsized":"2023-11-28T18:23:38.092623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:17:14.412292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.070788+0000","last_change":"2023-11-28T18:23:32.921502+0000","last_active":"2023-11-28T18:23:38.070788+0000","last_peered":"2023-11-28T18:23:38.070788+0000","last_clean":"2023-11-28T18:23:38.070788+0000","last_became_active":"2023-11-28T18:23:32.920979+0000","last_became_peered":"2023-11-28T18:23:32.920979+0000","last_unstale":"2023-11-28T18:23:38.070788+0000","last_undegraded":"2023-11-28T18:23:38.070788+0000","last_fullsized":"2023-11-28T18:23:38.070788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:41.820325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.719783+0000","last_change":"2023-11-28T18:23:38.095346+0000","last_active":"2023-11-28T18:23:40.719783+0000","last_peered":"2023-11-28T18:23:40.719783+0000","last_clean":"2023-11-28T18:23:40.719783+0000","last_became_active":"2023-11-28T18:23:38.094985+0000","last_became_peered":"2023-11-28T18:23:38.094985+0000","last_unstale":"2023-11-28T18:23:40.719783+0000","last_undegraded":"2023-11-28T18:23:40.719783+0000","last_fullsized":"2023-11-28T18:23:40.719783+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:22:41.524911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065411+0000","last_change":"2023-11-28T18:23:33.934122+0000","last_active":"2023-11-28T18:23:38.065411+0000","last_peered":"2023-11-28T18:23:38.065411+0000","last_clean":"2023-11-28T18:23:38.065411+0000","last_became_active":"2023-11-28T18:23:33.933687+0000","last_became_peered":"2023-11-28T18:23:33.933687+0000","last_unstale":"2023-11-28T18:23:38.065411+0000","last_undegraded":"2023-11-28T18:23:38.065411+0000","last_fullsized":"2023-11-28T18:23:38.065411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:17:14.412292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065550+0000","last_change":"2023-11-28T18:23:32.931872+0000","last_active":"2023-11-28T18:23:38.065550+0000","last_peered":"2023-11-28T18:23:38.065550+0000","last_clean":"2023-11-28T18:23:38.065550+0000","last_became_active":"2023-11-28T18:23:32.931111+0000","last_became_peered":"2023-11-28T18:23:32.931111+0000","last_unstale":"2023-11-28T18:23:38.065550+0000","last_undegraded":"2023-11-28T18:23:38.065550+0000","last_fullsized":"2023-11-28T18:23:38.065550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:41.820325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.074084+0000","last_change":"2023-11-28T18:23:32.934258+0000","last_active":"2023-11-28T18:23:38.074084+0000","last_peered":"2023-11-28T18:23:38.074084+0000","last_clean":"2023-11-28T18:23:38.074084+0000","last_became_active":"2023-11-28T18:23:32.933956+0000","last_became_peered":"2023-11-28T18:23:32.933956+0000","last_unstale":"2023-11-28T18:23:38.074084+0000","last_undegraded":"2023-11-28T18:23:38.074084+0000","last_fullsized":"2023-11-28T18:23:38.074084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:41.820325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077709+0000","last_change":"2023-11-28T18:23:33.949218+0000","last_active":"2023-11-28T18:23:38.077709+0000","last_peered":"2023-11-28T18:23:38.077709+0000","last_clean":"2023-11-28T18:23:38.077709+0000","last_became_active":"2023-11-28T18:23:33.948930+0000","last_became_peered":"2023-11-28T18:23:33.948930+0000","last_unstale":"2023-11-28T18:23:38.077709+0000","last_undegraded":"2023-11-28T18:23:38.077709+0000","last_fullsized":"2023-11-28T18:23:38.077709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:46.421541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.067937+0000","last_change":"2023-11-28T18:23:38.095800+0000","last_active":"2023-11-28T18:23:39.067937+0000","last_peered":"2023-11-28T18:23:39.067937+0000","last_clean":"2023-11-28T18:23:39.067937+0000","last_became_active":"2023-11-28T18:23:38.095437+0000","last_became_peered":"2023-11-28T18:23:38.095437+0000","last_unstale":"2023-11-28T18:23:39.067937+0000","last_undegraded":"2023-11-28T18:23:39.067937+0000","last_fullsized":"2023-11-28T18:23:39.067937+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:39.644595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065704+0000","last_change":"2023-11-28T18:23:32.921448+0000","last_active":"2023-11-28T18:23:38.065704+0000","last_peered":"2023-11-28T18:23:38.065704+0000","last_clean":"2023-11-28T18:23:38.065704+0000","last_became_active":"2023-11-28T18:23:32.921183+0000","last_became_peered":"2023-11-28T18:23:32.921183+0000","last_unstale":"2023-11-28T18:23:38.065704+0000","last_undegraded":"2023-11-28T18:23:38.065704+0000","last_fullsized":"2023-11-28T18:23:38.065704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:36:09.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075654+0000","last_change":"2023-11-28T18:23:33.937197+0000","last_active":"2023-11-28T18:23:38.075654+0000","last_peered":"2023-11-28T18:23:38.075654+0000","last_clean":"2023-11-28T18:23:38.075654+0000","last_became_active":"2023-11-28T18:23:33.936922+0000","last_became_peered":"2023-11-28T18:23:33.936922+0000","last_unstale":"2023-11-28T18:23:38.075654+0000","last_undegraded":"2023-11-28T18:23:38.075654+0000","last_fullsized":"2023-11-28T18:23:38.075654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:01.331886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068617+0000","last_change":"2023-11-28T18:23:38.099275+0000","last_active":"2023-11-28T18:23:39.068617+0000","last_peered":"2023-11-28T18:23:39.068617+0000","last_clean":"2023-11-28T18:23:39.068617+0000","last_became_active":"2023-11-28T18:23:38.095544+0000","last_became_peered":"2023-11-28T18:23:38.095544+0000","last_unstale":"2023-11-28T18:23:39.068617+0000","last_undegraded":"2023-11-28T18:23:39.068617+0000","last_fullsized":"2023-11-28T18:23:39.068617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:04.943189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.073386+0000","last_change":"2023-11-28T18:23:33.943034+0000","last_active":"2023-11-28T18:23:38.073386+0000","last_peered":"2023-11-28T18:23:38.073386+0000","last_clean":"2023-11-28T18:23:38.073386+0000","last_became_active":"2023-11-28T18:23:33.942687+0000","last_became_peered":"2023-11-28T18:23:33.942687+0000","last_unstale":"2023-11-28T18:23:38.073386+0000","last_undegraded":"2023-11-28T18:23:38.073386+0000","last_fullsized":"2023-11-28T18:23:38.073386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:35.484199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.083621+0000","last_change":"2023-11-28T18:23:33.942958+0000","last_active":"2023-11-28T18:23:38.083621+0000","last_peered":"2023-11-28T18:23:38.083621+0000","last_clean":"2023-11-28T18:23:38.083621+0000","last_became_active":"2023-11-28T18:23:33.942696+0000","last_became_peered":"2023-11-28T18:23:33.942696+0000","last_unstale":"2023-11-28T18:23:38.083621+0000","last_undegraded":"2023-11-28T18:23:38.083621+0000","last_fullsized":"2023-11-28T18:23:38.083621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:42.826477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068550+0000","last_change":"2023-11-28T18:23:33.927110+0000","last_active":"2023-11-28T18:23:38.068550+0000","last_peered":"2023-11-28T18:23:38.068550+0000","last_clean":"2023-11-28T18:23:38.068550+0000","last_became_active":"2023-11-28T18:23:33.926812+0000","last_became_peered":"2023-11-28T18:23:33.926812+0000","last_unstale":"2023-11-28T18:23:38.068550+0000","last_undegraded":"2023-11-28T18:23:38.068550+0000","last_fullsized":"2023-11-28T18:23:38.068550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:22:37.364515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077616+0000","last_change":"2023-11-28T18:23:33.941141+0000","last_active":"2023-11-28T18:23:38.077616+0000","last_peered":"2023-11-28T18:23:38.077616+0000","last_clean":"2023-11-28T18:23:38.077616+0000","last_became_active":"2023-11-28T18:23:33.940832+0000","last_became_peered":"2023-11-28T18:23:33.940832+0000","last_unstale":"2023-11-28T18:23:38.077616+0000","last_undegraded":"2023-11-28T18:23:38.077616+0000","last_fullsized":"2023-11-28T18:23:38.077616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:04:56.118542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.066054+0000","last_change":"2023-11-28T18:23:33.924784+0000","last_active":"2023-11-28T18:23:38.066054+0000","last_peered":"2023-11-28T18:23:38.066054+0000","last_clean":"2023-11-28T18:23:38.066054+0000","last_became_active":"2023-11-28T18:23:33.924475+0000","last_became_peered":"2023-11-28T18:23:33.924475+0000","last_unstale":"2023-11-28T18:23:38.066054+0000","last_undegraded":"2023-11-28T18:23:38.066054+0000","last_fullsized":"2023-11-28T18:23:38.066054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:05.773283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.094179+0000","last_change":"2023-11-28T18:23:33.932579+0000","last_active":"2023-11-28T18:23:38.094179+0000","last_peered":"2023-11-28T18:23:38.094179+0000","last_clean":"2023-11-28T18:23:38.094179+0000","last_became_active":"2023-11-28T18:23:33.932201+0000","last_became_peered":"2023-11-28T18:23:33.932201+0000","last_unstale":"2023-11-28T18:23:38.094179+0000","last_undegraded":"2023-11-28T18:23:38.094179+0000","last_fullsized":"2023-11-28T18:23:38.094179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:01.912695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077881+0000","last_change":"2023-11-28T18:23:33.954178+0000","last_active":"2023-11-28T18:23:38.077881+0000","last_peered":"2023-11-28T18:23:38.077881+0000","last_clean":"2023-11-28T18:23:38.077881+0000","last_became_active":"2023-11-28T18:23:33.954020+0000","last_became_peered":"2023-11-28T18:23:33.954020+0000","last_unstale":"2023-11-28T18:23:38.077881+0000","last_undegraded":"2023-11-28T18:23:38.077881+0000","last_fullsized":"2023-11-28T18:23:38.077881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:01.912695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.066974+0000","last_change":"2023-11-28T18:23:33.939474+0000","last_active":"2023-11-28T18:23:38.066974+0000","last_peered":"2023-11-28T18:23:38.066974+0000","last_clean":"2023-11-28T18:23:38.066974+0000","last_became_active":"2023-11-28T18:23:33.939189+0000","last_became_peered":"2023-11-28T18:23:33.939189+0000","last_unstale":"2023-11-28T18:23:38.066974+0000","last_undegraded":"2023-11-28T18:23:38.066974+0000","last_fullsized":"2023-11-28T18:23:38.066974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:08.033209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.070751+0000","last_change":"2023-11-28T18:23:33.929306+0000","last_active":"2023-11-28T18:23:38.070751+0000","last_peered":"2023-11-28T18:23:38.070751+0000","last_clean":"2023-11-28T18:23:38.070751+0000","last_became_active":"2023-11-28T18:23:33.929029+0000","last_became_peered":"2023-11-28T18:23:33.929029+0000","last_unstale":"2023-11-28T18:23:38.070751+0000","last_undegraded":"2023-11-28T18:23:38.070751+0000","last_fullsized":"2023-11-28T18:23:38.070751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:46.421541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065454+0000","last_change":"2023-11-28T18:23:33.922789+0000","last_active":"2023-11-28T18:23:38.065454+0000","last_peered":"2023-11-28T18:23:38.065454+0000","last_clean":"2023-11-28T18:23:38.065454+0000","last_became_active":"2023-11-28T18:23:33.922458+0000","last_became_peered":"2023-11-28T18:23:33.922458+0000","last_unstale":"2023-11-28T18:23:38.065454+0000","last_undegraded":"2023-11-28T18:23:38.065454+0000","last_fullsized":"2023-11-28T18:23:38.065454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:51.652679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068465+0000","last_change":"2023-11-28T18:23:33.935519+0000","last_active":"2023-11-28T18:23:38.068465+0000","last_peered":"2023-11-28T18:23:38.068465+0000","last_clean":"2023-11-28T18:23:38.068465+0000","last_became_active":"2023-11-28T18:23:33.934779+0000","last_became_peered":"2023-11-28T18:23:33.934779+0000","last_unstale":"2023-11-28T18:23:38.068465+0000","last_undegraded":"2023-11-28T18:23:38.068465+0000","last_fullsized":"2023-11-28T18:23:38.068465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:27.506820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077522+0000","last_change":"2023-11-28T18:23:33.946486+0000","last_active":"2023-11-28T18:23:38.077522+0000","last_peered":"2023-11-28T18:23:38.077522+0000","last_clean":"2023-11-28T18:23:38.077522+0000","last_became_active":"2023-11-28T18:23:33.946212+0000","last_became_peered":"2023-11-28T18:23:33.946212+0000","last_unstale":"2023-11-28T18:23:38.077522+0000","last_undegraded":"2023-11-28T18:23:38.077522+0000","last_fullsized":"2023-11-28T18:23:38.077522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:17:14.412292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.076300+0000","last_change":"2023-11-28T18:23:33.949577+0000","last_active":"2023-11-28T18:23:38.076300+0000","last_peered":"2023-11-28T18:23:38.076300+0000","last_clean":"2023-11-28T18:23:38.076300+0000","last_became_active":"2023-11-28T18:23:33.949275+0000","last_became_peered":"2023-11-28T18:23:33.949275+0000","last_unstale":"2023-11-28T18:23:38.076300+0000","last_undegraded":"2023-11-28T18:23:38.076300+0000","last_fullsized":"2023-11-28T18:23:38.076300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:17:14.412292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420401+0000","last_change":"2023-11-28T18:23:33.939143+0000","last_active":"2023-11-28T18:23:38.420401+0000","last_peered":"2023-11-28T18:23:38.420401+0000","last_clean":"2023-11-28T18:23:38.420401+0000","last_became_active":"2023-11-28T18:23:33.938673+0000","last_became_peered":"2023-11-28T18:23:33.938673+0000","last_unstale":"2023-11-28T18:23:38.420401+0000","last_undegraded":"2023-11-28T18:23:38.420401+0000","last_fullsized":"2023-11-28T18:23:38.420401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:11.439436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.083975+0000","last_change":"2023-11-28T18:23:33.946701+0000","last_active":"2023-11-28T18:23:38.083975+0000","last_peered":"2023-11-28T18:23:38.083975+0000","last_clean":"2023-11-28T18:23:38.083975+0000","last_became_active":"2023-11-28T18:23:33.946460+0000","last_became_peered":"2023-11-28T18:23:33.946460+0000","last_unstale":"2023-11-28T18:23:38.083975+0000","last_undegraded":"2023-11-28T18:23:38.083975+0000","last_fullsized":"2023-11-28T18:23:38.083975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:11.792676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068495+0000","last_change":"2023-11-28T18:23:33.940275+0000","last_active":"2023-11-28T18:23:38.068495+0000","last_peered":"2023-11-28T18:23:38.068495+0000","last_clean":"2023-11-28T18:23:38.068495+0000","last_became_active":"2023-11-28T18:23:33.939937+0000","last_became_peered":"2023-11-28T18:23:33.939937+0000","last_unstale":"2023-11-28T18:23:38.068495+0000","last_undegraded":"2023-11-28T18:23:38.068495+0000","last_fullsized":"2023-11-28T18:23:38.068495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:11.439436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.070691+0000","last_change":"2023-11-28T18:23:33.937355+0000","last_active":"2023-11-28T18:23:38.070691+0000","last_peered":"2023-11-28T18:23:38.070691+0000","last_clean":"2023-11-28T18:23:38.070691+0000","last_became_active":"2023-11-28T18:23:33.937069+0000","last_became_peered":"2023-11-28T18:23:33.937069+0000","last_unstale":"2023-11-28T18:23:38.070691+0000","last_undegraded":"2023-11-28T18:23:38.070691+0000","last_fullsized":"2023-11-28T18:23:38.070691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:05.773283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.092623+0000","last_change":"2023-11-28T18:23:33.938555+0000","last_active":"2023-11-28T18:23:38.092623+0000","last_peered":"2023-11-28T18:23:38.092623+0000","last_clean":"2023-11-28T18:23:38.092623+0000","last_became_active":"2023-11-28T18:23:33.938297+0000","last_became_peered":"2023-11-28T18:23:33.938297+0000","last_unstale":"2023-11-28T18:23:38.092623+0000","last_undegraded":"2023-11-28T18:23:38.092623+0000","last_fullsized":"2023-11-28T18:23:38.092623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:51.652679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.080082+0000","last_change":"2023-11-28T18:23:33.942250+0000","last_active":"2023-11-28T18:23:38.080082+0000","last_peered":"2023-11-28T18:23:38.080082+0000","last_clean":"2023-11-28T18:23:38.080082+0000","last_became_active":"2023-11-28T18:23:33.941935+0000","last_became_peered":"2023-11-28T18:23:33.941935+0000","last_unstale":"2023-11-28T18:23:38.080082+0000","last_undegraded":"2023-11-28T18:23:38.080082+0000","last_fullsized":"2023-11-28T18:23:38.080082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:21.035422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065409+0000","last_change":"2023-11-28T18:23:33.944983+0000","last_active":"2023-11-28T18:23:38.065409+0000","last_peered":"2023-11-28T18:23:38.065409+0000","last_clean":"2023-11-28T18:23:38.065409+0000","last_became_active":"2023-11-28T18:23:33.944535+0000","last_became_peered":"2023-11-28T18:23:33.944535+0000","last_unstale":"2023-11-28T18:23:38.065409+0000","last_undegraded":"2023-11-28T18:23:38.065409+0000","last_fullsized":"2023-11-28T18:23:38.065409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:01.912695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075790+0000","last_change":"2023-11-28T18:23:33.945213+0000","last_active":"2023-11-28T18:23:38.075790+0000","last_peered":"2023-11-28T18:23:38.075790+0000","last_clean":"2023-11-28T18:23:38.075790+0000","last_became_active":"2023-11-28T18:23:33.944605+0000","last_became_peered":"2023-11-28T18:23:33.944605+0000","last_unstale":"2023-11-28T18:23:38.075790+0000","last_undegraded":"2023-11-28T18:23:38.075790+0000","last_fullsized":"2023-11-28T18:23:38.075790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:46.149525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.076131+0000","last_change":"2023-11-28T18:23:33.948453+0000","last_active":"2023-11-28T18:23:38.076131+0000","last_peered":"2023-11-28T18:23:38.076131+0000","last_clean":"2023-11-28T18:23:38.076131+0000","last_became_active":"2023-11-28T18:23:33.948203+0000","last_became_peered":"2023-11-28T18:23:33.948203+0000","last_unstale":"2023-11-28T18:23:38.076131+0000","last_undegraded":"2023-11-28T18:23:38.076131+0000","last_fullsized":"2023-11-28T18:23:38.076131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:51.652679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077793+0000","last_change":"2023-11-28T18:23:33.945564+0000","last_active":"2023-11-28T18:23:38.077793+0000","last_peered":"2023-11-28T18:23:38.077793+0000","last_clean":"2023-11-28T18:23:38.077793+0000","last_became_active":"2023-11-28T18:23:33.943276+0000","last_became_peered":"2023-11-28T18:23:33.943276+0000","last_unstale":"2023-11-28T18:23:38.077793+0000","last_undegraded":"2023-11-28T18:23:38.077793+0000","last_fullsized":"2023-11-28T18:23:38.077793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:21.035422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420502+0000","last_change":"2023-11-28T18:23:33.942649+0000","last_active":"2023-11-28T18:23:38.420502+0000","last_peered":"2023-11-28T18:23:38.420502+0000","last_clean":"2023-11-28T18:23:38.420502+0000","last_became_active":"2023-11-28T18:23:33.942109+0000","last_became_peered":"2023-11-28T18:23:33.942109+0000","last_unstale":"2023-11-28T18:23:38.420502+0000","last_undegraded":"2023-11-28T18:23:38.420502+0000","last_fullsized":"2023-11-28T18:23:38.420502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:34.989175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.064784+0000","last_change":"2023-11-28T18:23:33.933929+0000","last_active":"2023-11-28T18:23:38.064784+0000","last_peered":"2023-11-28T18:23:38.064784+0000","last_clean":"2023-11-28T18:23:38.064784+0000","last_became_active":"2023-11-28T18:23:33.933087+0000","last_became_peered":"2023-11-28T18:23:33.933087+0000","last_unstale":"2023-11-28T18:23:38.064784+0000","last_undegraded":"2023-11-28T18:23:38.064784+0000","last_fullsized":"2023-11-28T18:23:38.064784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:12.557669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419632+0000","last_change":"2023-11-28T18:23:33.950276+0000","last_active":"2023-11-28T18:23:38.419632+0000","last_peered":"2023-11-28T18:23:38.419632+0000","last_clean":"2023-11-28T18:23:38.419632+0000","last_became_active":"2023-11-28T18:23:33.946777+0000","last_became_peered":"2023-11-28T18:23:33.946777+0000","last_unstale":"2023-11-28T18:23:38.419632+0000","last_undegraded":"2023-11-28T18:23:38.419632+0000","last_fullsized":"2023-11-28T18:23:38.419632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:46.421541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068159+0000","last_change":"2023-11-28T18:23:33.943666+0000","last_active":"2023-11-28T18:23:38.068159+0000","last_peered":"2023-11-28T18:23:38.068159+0000","last_clean":"2023-11-28T18:23:38.068159+0000","last_became_active":"2023-11-28T18:23:33.940798+0000","last_became_peered":"2023-11-28T18:23:33.940798+0000","last_unstale":"2023-11-28T18:23:38.068159+0000","last_undegraded":"2023-11-28T18:23:38.068159+0000","last_fullsized":"2023-11-28T18:23:38.068159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:04:56.118542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419726+0000","last_change":"2023-11-28T18:23:33.955086+0000","last_active":"2023-11-28T18:23:38.419726+0000","last_peered":"2023-11-28T18:23:38.419726+0000","last_clean":"2023-11-28T18:23:38.419726+0000","last_became_active":"2023-11-28T18:23:33.950311+0000","last_became_peered":"2023-11-28T18:23:33.950311+0000","last_unstale":"2023-11-28T18:23:38.419726+0000","last_undegraded":"2023-11-28T18:23:38.419726+0000","last_fullsized":"2023-11-28T18:23:38.419726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:27.506820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067660+0000","last_change":"2023-11-28T18:23:33.932833+0000","last_active":"2023-11-28T18:23:38.067660+0000","last_peered":"2023-11-28T18:23:38.067660+0000","last_clean":"2023-11-28T18:23:38.067660+0000","last_became_active":"2023-11-28T18:23:33.932554+0000","last_became_peered":"2023-11-28T18:23:33.932554+0000","last_unstale":"2023-11-28T18:23:38.067660+0000","last_undegraded":"2023-11-28T18:23:38.067660+0000","last_fullsized":"2023-11-28T18:23:38.067660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:08.033209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075353+0000","last_change":"2023-11-28T18:23:33.939136+0000","last_active":"2023-11-28T18:23:38.075353+0000","last_peered":"2023-11-28T18:23:38.075353+0000","last_clean":"2023-11-28T18:23:38.075353+0000","last_became_active":"2023-11-28T18:23:33.938870+0000","last_became_peered":"2023-11-28T18:23:33.938870+0000","last_unstale":"2023-11-28T18:23:38.075353+0000","last_undegraded":"2023-11-28T18:23:38.075353+0000","last_fullsized":"2023-11-28T18:23:38.075353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:05.773283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.096250+0000","last_change":"2023-11-28T18:23:33.936563+0000","last_active":"2023-11-28T18:23:38.096250+0000","last_peered":"2023-11-28T18:23:38.096250+0000","last_clean":"2023-11-28T18:23:38.096250+0000","last_became_active":"2023-11-28T18:23:33.936272+0000","last_became_peered":"2023-11-28T18:23:33.936272+0000","last_unstale":"2023-11-28T18:23:38.096250+0000","last_undegraded":"2023-11-28T18:23:38.096250+0000","last_fullsized":"2023-11-28T18:23:38.096250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:04:56.118542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420445+0000","last_change":"2023-11-28T18:23:33.954685+0000","last_active":"2023-11-28T18:23:38.420445+0000","last_peered":"2023-11-28T18:23:38.420445+0000","last_clean":"2023-11-28T18:23:38.420445+0000","last_became_active":"2023-11-28T18:23:33.954434+0000","last_became_peered":"2023-11-28T18:23:33.954434+0000","last_unstale":"2023-11-28T18:23:38.420445+0000","last_undegraded":"2023-11-28T18:23:38.420445+0000","last_fullsized":"2023-11-28T18:23:38.420445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:38.277787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4498,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":690742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":99882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":99918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":99792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":99952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":690798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:23:52.499 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T18:23:52.500 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-28T18:23:52.500 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-28T18:23:52.500 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-28T18:23:52.501 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-28T18:23:52.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.667+0000 7f23b769a640 1 Processor -- start 2023-11-28T18:23:52.673 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.671+0000 7f23b769a640 1 -- start start 2023-11-28T18:23:52.674 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.671+0000 7f23b769a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f23b00caf60 0x7f23b00cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:52.674 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.671+0000 7f23b769a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23b00ad850 0x7f23b00701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:52.674 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.671+0000 7f23b769a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b006bf50 con 0x7f23b0159f40 2023-11-28T18:23:52.674 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.671+0000 7f23b769a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b00cb3c0 con 0x7f23b00caf60 2023-11-28T18:23:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.671+0000 7f23b769a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b015a320 con 0x7f23b00ad850 2023-11-28T18:23:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.671+0000 7f23b4c35640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f23b00caf60 0x7f23b00cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.671+0000 7f23b4c35640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f23b00caf60 0x7f23b00cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54894/0 (socket says 172.21.15.105:54894) 2023-11-28T18:23:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.671+0000 7f23b4c35640 1 -- 172.21.15.105:0/667226229 learned_addr learned my addr 172.21.15.105:0/667226229 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:52.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.671+0000 7f23b5c37640 1 --2- 172.21.15.105:0/667226229 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23b00ad850 0x7f23b00701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:52.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.671+0000 7f23b4c35640 1 -- 172.21.15.105:0/667226229 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23b00ad850 msgr2=0x7f23b00701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:52.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.671+0000 7f23b4c35640 1 --2- 172.21.15.105:0/667226229 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23b00ad850 0x7f23b00701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.671+0000 7f23b4c35640 1 -- 172.21.15.105:0/667226229 >> v1:172.21.15.148:6789/0 conn(0x7f23b0159f40 legacy=0x7f23b00ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:52.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.671+0000 7f23b4c35640 1 -- 172.21.15.105:0/667226229 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b0071380 con 0x7f23b00caf60 2023-11-28T18:23:52.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.671+0000 7f23b5c37640 1 --2- 172.21.15.105:0/667226229 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23b00ad850 0x7f23b00701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:52.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.675+0000 7f23b4c35640 1 --2- 172.21.15.105:0/667226229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23b00caf60 0x7f23b00cc760 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009870 tx=0x7f23a001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497cfb1c32a13c12 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:52.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.675+0000 7f23a7fff640 1 -- 172.21.15.105:0/667226229 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a00ab680 con 0x7f23b00caf60 2023-11-28T18:23:52.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.675+0000 7f23a7fff640 1 -- 172.21.15.105:0/667226229 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a00ab7e0 con 0x7f23b00caf60 2023-11-28T18:23:52.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.675+0000 7f23a7fff640 1 -- 172.21.15.105:0/667226229 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a00abb70 con 0x7f23b00caf60 2023-11-28T18:23:52.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.675+0000 7f23b769a640 1 -- 172.21.15.105:0/667226229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23b00caf60 msgr2=0x7f23b00cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:52.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.675+0000 7f23b769a640 1 --2- 172.21.15.105:0/667226229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23b00caf60 0x7f23b00cc760 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009870 tx=0x7f23a001c480 comp rx=0 tx=0).stop 2023-11-28T18:23:52.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.675+0000 7f23b769a640 1 -- 172.21.15.105:0/667226229 shutdown_connections 2023-11-28T18:23:52.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.675+0000 7f23b769a640 1 --2- 172.21.15.105:0/667226229 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23b00ad850 0x7f23b00701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.675+0000 7f23b769a640 1 --2- 172.21.15.105:0/667226229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23b00caf60 0x7f23b00cc760 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.675+0000 7f23b769a640 1 -- 172.21.15.105:0/667226229 >> 172.21.15.105:0/667226229 conn(0x7f23b00ceb90 msgr2=0x7f23b00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:52.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.675+0000 7f23b769a640 1 -- 172.21.15.105:0/667226229 shutdown_connections 2023-11-28T18:23:52.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.675+0000 7f23b769a640 1 -- 172.21.15.105:0/667226229 wait complete. 2023-11-28T18:23:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.675+0000 7f23b769a640 1 Processor -- start 2023-11-28T18:23:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23b769a640 1 -- start start 2023-11-28T18:23:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23b769a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23b00ad850 0x7f23b014a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23b769a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23b00caf60 0x7f23b015b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23b769a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23b0159f40 0x7f23b014b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23b769a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b0157540 con 0x7f23b00ad850 2023-11-28T18:23:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23b769a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b014aa70 con 0x7f23b00caf60 2023-11-28T18:23:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23b769a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b015c030 con 0x7f23b0159f40 2023-11-28T18:23:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23b5436640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23b00ad850 0x7f23b014a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23b5436640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23b00ad850 0x7f23b014a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54898/0 (socket says 172.21.15.105:54898) 2023-11-28T18:23:52.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23b5436640 1 -- 172.21.15.105:0/332790554 learned_addr learned my addr 172.21.15.105:0/332790554 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:52.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23b4c35640 1 --2- 172.21.15.105:0/332790554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23b00caf60 0x7f23b015b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:52.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23b5c37640 1 --2- 172.21.15.105:0/332790554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23b0159f40 0x7f23b014b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:52.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23b5436640 1 -- 172.21.15.105:0/332790554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23b0159f40 msgr2=0x7f23b014b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:52.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23b5436640 1 --2- 172.21.15.105:0/332790554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23b0159f40 0x7f23b014b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23b5436640 1 -- 172.21.15.105:0/332790554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23b00caf60 msgr2=0x7f23b015b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:52.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23b5436640 1 --2- 172.21.15.105:0/332790554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23b00caf60 0x7f23b015b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23b5436640 1 -- 172.21.15.105:0/332790554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a00094d0 con 0x7f23b00ad850 2023-11-28T18:23:52.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23b5c37640 1 --2- 172.21.15.105:0/332790554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23b0159f40 0x7f23b014b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:52.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23b5436640 1 --2- 172.21.15.105:0/332790554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23b00ad850 0x7f23b014a530 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f23ac00c3f0 tx=0x7f23ac00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:52.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23a5ffb640 1 -- 172.21.15.105:0/332790554 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac0052d0 con 0x7f23b00ad850 2023-11-28T18:23:52.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23a5ffb640 1 -- 172.21.15.105:0/332790554 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ac005430 con 0x7f23b00ad850 2023-11-28T18:23:52.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23a5ffb640 1 -- 172.21.15.105:0/332790554 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac003840 con 0x7f23b00ad850 2023-11-28T18:23:52.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23b769a640 1 -- 172.21.15.105:0/332790554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b015c310 con 0x7f23b00ad850 2023-11-28T18:23:52.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23b769a640 1 -- 172.21.15.105:0/332790554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b0157a70 con 0x7f23b00ad850 2023-11-28T18:23:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.679+0000 7f23b769a640 1 -- 172.21.15.105:0/332790554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b0067570 con 0x7f23b00ad850 2023-11-28T18:23:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.683+0000 7f23a5ffb640 1 -- 172.21.15.105:0/332790554 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f23ac0039a0 con 0x7f23b00ad850 2023-11-28T18:23:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.683+0000 7f23a5ffb640 1 --2- 172.21.15.105:0/332790554 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f23940668d0 0x7f2394068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.683+0000 7f23a5ffb640 1 -- 172.21.15.105:0/332790554 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f23ac083bf0 con 0x7f23b00ad850 2023-11-28T18:23:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.683+0000 7f23b4c35640 1 --2- 172.21.15.105:0/332790554 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f23940668d0 0x7f2394068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.683+0000 7f23b4c35640 1 --2- 172.21.15.105:0/332790554 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f23940668d0 0x7f2394068d90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f23a00af000 tx=0x7f23a00b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:52.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.687+0000 7f23a5ffb640 1 -- 172.21.15.105:0/332790554 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f23ac050c80 con 0x7f23b00ad850 2023-11-28T18:23:52.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.867+0000 7f23b769a640 1 -- 172.21.15.105:0/332790554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f23b0157c60 con 0x7f23b00ad850 2023-11-28T18:23:52.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.871+0000 7f23a5ffb640 1 -- 172.21.15.105:0/332790554 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164976 (secure 0 0 0) 0x7f23ac017090 con 0x7f23b00ad850 2023-11-28T18:23:52.877 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:52.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.875+0000 7f23b769a640 1 -- 172.21.15.105:0/332790554 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f23940668d0 msgr2=0x7f2394068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:52.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.875+0000 7f23b769a640 1 --2- 172.21.15.105:0/332790554 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f23940668d0 0x7f2394068d90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f23a00af000 tx=0x7f23a00b6040 comp rx=0 tx=0).stop 2023-11-28T18:23:52.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.875+0000 7f23b769a640 1 -- 172.21.15.105:0/332790554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23b00ad850 msgr2=0x7f23b014a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:52.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.875+0000 7f23b769a640 1 --2- 172.21.15.105:0/332790554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23b00ad850 0x7f23b014a530 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f23ac00c3f0 tx=0x7f23ac00c8b0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.875+0000 7f23b769a640 1 -- 172.21.15.105:0/332790554 shutdown_connections 2023-11-28T18:23:52.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.875+0000 7f23b769a640 1 --2- 172.21.15.105:0/332790554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23b0159f40 0x7f23b014b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.875+0000 7f23b769a640 1 --2- 172.21.15.105:0/332790554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23b00caf60 0x7f23b015b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.875+0000 7f23b769a640 1 --2- 172.21.15.105:0/332790554 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f23940668d0 0x7f2394068d90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.875+0000 7f23b769a640 1 --2- 172.21.15.105:0/332790554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23b00ad850 0x7f23b014a530 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:52.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.875+0000 7f23b769a640 1 -- 172.21.15.105:0/332790554 >> 172.21.15.105:0/332790554 conn(0x7f23b00ceb90 msgr2=0x7f23b014e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:52.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.875+0000 7f23b769a640 1 -- 172.21.15.105:0/332790554 shutdown_connections 2023-11-28T18:23:52.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:52.875+0000 7f23b769a640 1 -- 172.21.15.105:0/332790554 wait complete. 2023-11-28T18:23:52.897 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":5,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1892510418},{"type":"v1","addr":"172.21.15.105:6833","nonce":1892510418}]},"active_addr":"172.21.15.105:6833/1892510418","active_change":"2023-11-28T18:23:13.257520+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4103,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":2069443173}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":3938835281}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":2072697631}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":281412488}]}]} 2023-11-28T18:23:52.900 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-28T18:23:52.900 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-28T18:23:52.901 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:23:53.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.035+0000 7f2fa0efe640 1 Processor -- start 2023-11-28T18:23:53.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.035+0000 7f2fa0efe640 1 -- start start 2023-11-28T18:23:53.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.035+0000 7f2fa0efe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f9c0d00b0 0x7f2f9c0cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.035+0000 7f2fa0efe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2f9c0ceb40 0x7f2f9c0b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.035+0000 7f2fa0efe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f9c0d3e60 con 0x7f2f9c0b8a20 2023-11-28T18:23:53.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.035+0000 7f2fa0efe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f9c0b5fe0 con 0x7f2f9c0ceb40 2023-11-28T18:23:53.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.035+0000 7f2fa0efe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c0b62c0 con 0x7f2f9c0d00b0 2023-11-28T18:23:53.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.035+0000 7f2f9ad9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2f9c0ceb40 0x7f2f9c0b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.035+0000 7f2f93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f9c0d00b0 0x7f2f9c0cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.035+0000 7f2f9ad9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2f9c0ceb40 0x7f2f9c0b6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34050/0 (socket says 172.21.15.105:34050) 2023-11-28T18:23:53.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.035+0000 7f2f9ad9d640 1 -- 172.21.15.105:0/474663416 learned_addr learned my addr 172.21.15.105:0/474663416 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:53.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.039+0000 7f2f9ad9d640 1 -- 172.21.15.105:0/474663416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f9c0d00b0 msgr2=0x7f2f9c0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:53.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.039+0000 7f2f9ad9d640 1 --2- 172.21.15.105:0/474663416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f9c0d00b0 0x7f2f9c0cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.039+0000 7f2f9ad9d640 1 -- 172.21.15.105:0/474663416 >> v1:172.21.15.148:6789/0 conn(0x7f2f9c0b8a20 legacy=0x7f2f9c0cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.039+0000 7f2f9ad9d640 1 -- 172.21.15.105:0/474663416 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9c0b65a0 con 0x7f2f9c0ceb40 2023-11-28T18:23:53.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.039+0000 7f2f9ad9d640 1 --2- 172.21.15.105:0/474663416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f9c0ceb40 0x7f2f9c0b6da0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2f94009e50 tx=0x7f2f9401c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4891682e625939fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.039+0000 7f2f99d9b640 1 -- 172.21.15.105:0/474663416 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f940a5870 con 0x7f2f9c0ceb40 2023-11-28T18:23:53.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.039+0000 7f2f99d9b640 1 -- 172.21.15.105:0/474663416 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f940a59d0 con 0x7f2f9c0ceb40 2023-11-28T18:23:53.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.039+0000 7f2f99d9b640 1 -- 172.21.15.105:0/474663416 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f940a5d60 con 0x7f2f9c0ceb40 2023-11-28T18:23:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.039+0000 7f2fa0efe640 1 -- 172.21.15.105:0/474663416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f9c0ceb40 msgr2=0x7f2f9c0b6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.039+0000 7f2fa0efe640 1 --2- 172.21.15.105:0/474663416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f9c0ceb40 0x7f2f9c0b6da0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2f94009e50 tx=0x7f2f9401c9e0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2fa0efe640 1 -- 172.21.15.105:0/474663416 shutdown_connections 2023-11-28T18:23:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2fa0efe640 1 --2- 172.21.15.105:0/474663416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f9c0ceb40 0x7f2f9c0b6da0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2fa0efe640 1 --2- 172.21.15.105:0/474663416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f9c0d00b0 0x7f2f9c0cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2fa0efe640 1 -- 172.21.15.105:0/474663416 >> 172.21.15.105:0/474663416 conn(0x7f2f9c09c340 msgr2=0x7f2f9c0d2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2fa0efe640 1 -- 172.21.15.105:0/474663416 shutdown_connections 2023-11-28T18:23:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2fa0efe640 1 -- 172.21.15.105:0/474663416 wait complete. 2023-11-28T18:23:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2fa0efe640 1 Processor -- start 2023-11-28T18:23:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2fa0efe640 1 -- start start 2023-11-28T18:23:53.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2fa0efe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f9c0b8a20 0x7f2f9c1607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2fa0efe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f9c0ceb40 0x7f2f9c150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2fa0efe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f9c0d00b0 0x7f2f9c1577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2fa0efe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c1505f0 con 0x7f2f9c0ceb40 2023-11-28T18:23:53.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2fa0efe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c14fc70 con 0x7f2f9c0b8a20 2023-11-28T18:23:53.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2fa0efe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c15b1c0 con 0x7f2f9c0d00b0 2023-11-28T18:23:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2f93fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f9c0ceb40 0x7f2f9c150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2f93fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f9c0ceb40 0x7f2f9c150a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54904/0 (socket says 172.21.15.105:54904) 2023-11-28T18:23:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2f93fff640 1 -- 172.21.15.105:0/666097904 learned_addr learned my addr 172.21.15.105:0/666097904 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2f9a59c640 1 --2- 172.21.15.105:0/666097904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f9c0b8a20 0x7f2f9c1607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2f9ad9d640 1 --2- 172.21.15.105:0/666097904 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f9c0d00b0 0x7f2f9c1577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2f93fff640 1 -- 172.21.15.105:0/666097904 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f9c0d00b0 msgr2=0x7f2f9c1577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2f93fff640 1 --2- 172.21.15.105:0/666097904 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f9c0d00b0 0x7f2f9c1577e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2f93fff640 1 -- 172.21.15.105:0/666097904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f9c0b8a20 msgr2=0x7f2f9c1607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2f93fff640 1 --2- 172.21.15.105:0/666097904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f9c0b8a20 0x7f2f9c1607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2f93fff640 1 -- 172.21.15.105:0/666097904 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f94009b30 con 0x7f2f9c0ceb40 2023-11-28T18:23:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.043+0000 7f2f93fff640 1 --2- 172.21.15.105:0/666097904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f9c0ceb40 0x7f2f9c150a10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2f8400e870 tx=0x7f2f8400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.047+0000 7f2f92ffd640 1 -- 172.21.15.105:0/666097904 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f84009510 con 0x7f2f9c0ceb40 2023-11-28T18:23:53.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.047+0000 7f2f92ffd640 1 -- 172.21.15.105:0/666097904 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8400eec0 con 0x7f2f9c0ceb40 2023-11-28T18:23:53.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.047+0000 7f2fa0efe640 1 -- 172.21.15.105:0/666097904 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f9c14ff30 con 0x7f2f9c0ceb40 2023-11-28T18:23:53.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.047+0000 7f2f92ffd640 1 -- 172.21.15.105:0/666097904 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8400cb40 con 0x7f2f9c0ceb40 2023-11-28T18:23:53.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.047+0000 7f2fa0efe640 1 -- 172.21.15.105:0/666097904 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f9c150f50 con 0x7f2f9c0ceb40 2023-11-28T18:23:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.047+0000 7f2fa0efe640 1 -- 172.21.15.105:0/666097904 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f9c067570 con 0x7f2f9c0ceb40 2023-11-28T18:23:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.047+0000 7f2f92ffd640 1 -- 172.21.15.105:0/666097904 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f2f8400cca0 con 0x7f2f9c0ceb40 2023-11-28T18:23:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.047+0000 7f2f92ffd640 1 --2- 172.21.15.105:0/666097904 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2f7c0668d0 0x7f2f7c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.047+0000 7f2f92ffd640 1 -- 172.21.15.105:0/666097904 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2f84084830 con 0x7f2f9c0ceb40 2023-11-28T18:23:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.047+0000 7f2f9a59c640 1 --2- 172.21.15.105:0/666097904 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2f7c0668d0 0x7f2f7c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.047+0000 7f2f9a59c640 1 --2- 172.21.15.105:0/666097904 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2f7c0668d0 0x7f2f7c068d90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c007fd0 tx=0x7f2f8c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.051+0000 7f2f92ffd640 1 -- 172.21.15.105:0/666097904 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2f840517a0 con 0x7f2f9c0ceb40 2023-11-28T18:23:53.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.203+0000 7f2fa0efe640 1 -- 172.21.15.105:0/666097904 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f9c0b43d0 con 0x7f2f9c0ceb40 2023-11-28T18:23:53.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.203+0000 7f2f92ffd640 1 -- 172.21.15.105:0/666097904 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7f2f84054dc0 con 0x7f2f9c0ceb40 2023-11-28T18:23:53.207 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:53.208 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":24,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:23:38.458954+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:23:53.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.203+0000 7f2fa0efe640 1 -- 172.21.15.105:0/666097904 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2f7c0668d0 msgr2=0x7f2f7c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.203+0000 7f2fa0efe640 1 --2- 172.21.15.105:0/666097904 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2f7c0668d0 0x7f2f7c068d90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c007fd0 tx=0x7f2f8c009040 comp rx=0 tx=0).stop 2023-11-28T18:23:53.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.203+0000 7f2fa0efe640 1 -- 172.21.15.105:0/666097904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f9c0ceb40 msgr2=0x7f2f9c150a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.203+0000 7f2fa0efe640 1 --2- 172.21.15.105:0/666097904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f9c0ceb40 0x7f2f9c150a10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2f8400e870 tx=0x7f2f8400ec70 comp rx=0 tx=0).stop 2023-11-28T18:23:53.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.203+0000 7f2fa0efe640 1 -- 172.21.15.105:0/666097904 shutdown_connections 2023-11-28T18:23:53.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.203+0000 7f2fa0efe640 1 --2- 172.21.15.105:0/666097904 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f9c0d00b0 0x7f2f9c1577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.203+0000 7f2fa0efe640 1 --2- 172.21.15.105:0/666097904 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2f7c0668d0 0x7f2f7c068d90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.203+0000 7f2fa0efe640 1 --2- 172.21.15.105:0/666097904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f9c0ceb40 0x7f2f9c150a10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.203+0000 7f2fa0efe640 1 --2- 172.21.15.105:0/666097904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f9c0b8a20 0x7f2f9c1607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.203+0000 7f2fa0efe640 1 -- 172.21.15.105:0/666097904 >> 172.21.15.105:0/666097904 conn(0x7f2f9c09c340 msgr2=0x7f2f9c15d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:53.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.203+0000 7f2fa0efe640 1 -- 172.21.15.105:0/666097904 shutdown_connections 2023-11-28T18:23:53.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.203+0000 7f2fa0efe640 1 -- 172.21.15.105:0/666097904 wait complete. 2023-11-28T18:23:53.228 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-28T18:23:53.229 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:23:53.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.363+0000 7f3ebbb2a640 1 Processor -- start 2023-11-28T18:23:53.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.363+0000 7f3ebbb2a640 1 -- start start 2023-11-28T18:23:53.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.367+0000 7f3ebbb2a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3eb415d4f0 0x7f3eb40d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.367+0000 7f3ebbb2a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3eb40d3c30 0x7f3eb40d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.367+0000 7f3ebbb2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3eb40d1890 0x7f3eb40cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.367+0000 7f3ebbb2a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb40b2900 con 0x7f3eb415d4f0 2023-11-28T18:23:53.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.367+0000 7f3ebbb2a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb40b25b0 con 0x7f3eb40d3c30 2023-11-28T18:23:53.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.367+0000 7f3ebbb2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb40cea30 con 0x7f3eb40d1890 2023-11-28T18:23:53.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.367+0000 7f3eb98c6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3eb415d4f0 0x7f3eb40d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.367+0000 7f3eb98c6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3eb415d4f0 0x7f3eb40d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54906/0 (socket says 172.21.15.105:54906) 2023-11-28T18:23:53.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.367+0000 7f3eb98c6640 1 -- 172.21.15.105:0/381832279 learned_addr learned my addr 172.21.15.105:0/381832279 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:53.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.367+0000 7f3eb90c5640 1 --2- 172.21.15.105:0/381832279 >> v2:172.21.15.148:3300/0 conn(0x7f3eb40d3c30 0x7f3eb40d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.367+0000 7f3eba0c7640 1 --2- 172.21.15.105:0/381832279 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3eb40d1890 0x7f3eb40cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.367+0000 7f3eb98c6640 1 -- 172.21.15.105:0/381832279 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3eb40d1890 msgr2=0x7f3eb40cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:53.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.367+0000 7f3eb98c6640 1 --2- 172.21.15.105:0/381832279 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3eb40d1890 0x7f3eb40cfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.367+0000 7f3eb98c6640 1 -- 172.21.15.105:0/381832279 >> v2:172.21.15.148:3300/0 conn(0x7f3eb40d3c30 msgr2=0x7f3eb40d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:53.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.367+0000 7f3eb98c6640 1 --2- 172.21.15.105:0/381832279 >> v2:172.21.15.148:3300/0 conn(0x7f3eb40d3c30 0x7f3eb40d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.367+0000 7f3eb98c6640 1 -- 172.21.15.105:0/381832279 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb40ced10 con 0x7f3eb415d4f0 2023-11-28T18:23:53.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.371+0000 7f3eb98c6640 1 --2- 172.21.15.105:0/381832279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eb415d4f0 0x7f3eb40d0e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3eb408c430 tx=0x7f3eb001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cffc1f7e231b1bd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.371+0000 7f3eb88c4640 1 -- 172.21.15.105:0/381832279 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb00ac630 con 0x7f3eb415d4f0 2023-11-28T18:23:53.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.371+0000 7f3eb88c4640 1 -- 172.21.15.105:0/381832279 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb00ac790 con 0x7f3eb415d4f0 2023-11-28T18:23:53.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.371+0000 7f3eb88c4640 1 -- 172.21.15.105:0/381832279 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb00acb20 con 0x7f3eb415d4f0 2023-11-28T18:23:53.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.371+0000 7f3ebbb2a640 1 -- 172.21.15.105:0/381832279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eb415d4f0 msgr2=0x7f3eb40d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.371+0000 7f3ebbb2a640 1 --2- 172.21.15.105:0/381832279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eb415d4f0 0x7f3eb40d0e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3eb408c430 tx=0x7f3eb001c470 comp rx=0 tx=0).stop 2023-11-28T18:23:53.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.371+0000 7f3ebbb2a640 1 -- 172.21.15.105:0/381832279 shutdown_connections 2023-11-28T18:23:53.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.371+0000 7f3ebbb2a640 1 --2- 172.21.15.105:0/381832279 >> v2:172.21.15.148:3300/0 conn(0x7f3eb40d3c30 0x7f3eb40d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.371+0000 7f3ebbb2a640 1 --2- 172.21.15.105:0/381832279 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3eb40d1890 0x7f3eb40cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.371+0000 7f3ebbb2a640 1 --2- 172.21.15.105:0/381832279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eb415d4f0 0x7f3eb40d0e10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.371+0000 7f3ebbb2a640 1 -- 172.21.15.105:0/381832279 >> 172.21.15.105:0/381832279 conn(0x7f3eb40d3030 msgr2=0x7f3eb40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:53.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.371+0000 7f3ebbb2a640 1 -- 172.21.15.105:0/381832279 shutdown_connections 2023-11-28T18:23:53.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.371+0000 7f3ebbb2a640 1 -- 172.21.15.105:0/381832279 wait complete. 2023-11-28T18:23:53.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.371+0000 7f3ebbb2a640 1 Processor -- start 2023-11-28T18:23:53.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.371+0000 7f3ebbb2a640 1 -- start start 2023-11-28T18:23:53.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3ebbb2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eb40d1890 0x7f3eb414c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3ebbb2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3eb40d3c30 0x7f3eb40ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3ebbb2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3eb415d4f0 0x7f3eb40ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3ebbb2a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb41543f0 con 0x7f3eb40d1890 2023-11-28T18:23:53.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3ebbb2a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb415b3d0 con 0x7f3eb40d3c30 2023-11-28T18:23:53.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3ebbb2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb4153ac0 con 0x7f3eb415d4f0 2023-11-28T18:23:53.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3eb98c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eb40d1890 0x7f3eb414c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3eb98c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eb40d1890 0x7f3eb414c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54920/0 (socket says 172.21.15.105:54920) 2023-11-28T18:23:53.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3eb98c6640 1 -- 172.21.15.105:0/3574759033 learned_addr learned my addr 172.21.15.105:0/3574759033 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:53.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3eb90c5640 1 --2- 172.21.15.105:0/3574759033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3eb40d3c30 0x7f3eb40ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3eba0c7640 1 --2- 172.21.15.105:0/3574759033 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3eb415d4f0 0x7f3eb40ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3eb98c6640 1 -- 172.21.15.105:0/3574759033 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3eb415d4f0 msgr2=0x7f3eb40ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3eb98c6640 1 --2- 172.21.15.105:0/3574759033 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3eb415d4f0 0x7f3eb40ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3eb98c6640 1 -- 172.21.15.105:0/3574759033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3eb40d3c30 msgr2=0x7f3eb40ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3eb98c6640 1 --2- 172.21.15.105:0/3574759033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3eb40d3c30 0x7f3eb40ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3eb98c6640 1 -- 172.21.15.105:0/3574759033 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb00094d0 con 0x7f3eb40d1890 2023-11-28T18:23:53.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3eb98c6640 1 --2- 172.21.15.105:0/3574759033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eb40d1890 0x7f3eb414c3e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3eb00af040 tx=0x7f3eb0019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3eaa7fc640 1 -- 172.21.15.105:0/3574759033 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb00b7670 con 0x7f3eb40d1890 2023-11-28T18:23:53.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3eaa7fc640 1 -- 172.21.15.105:0/3574759033 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb00b77f0 con 0x7f3eb40d1890 2023-11-28T18:23:53.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3ebbb2a640 1 -- 172.21.15.105:0/3574759033 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb4153da0 con 0x7f3eb40d1890 2023-11-28T18:23:53.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3eaa7fc640 1 -- 172.21.15.105:0/3574759033 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb00b7ba0 con 0x7f3eb40d1890 2023-11-28T18:23:53.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3ebbb2a640 1 -- 172.21.15.105:0/3574759033 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb414c920 con 0x7f3eb40d1890 2023-11-28T18:23:53.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3ebbb2a640 1 -- 172.21.15.105:0/3574759033 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb4067570 con 0x7f3eb40d1890 2023-11-28T18:23:53.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.375+0000 7f3eaa7fc640 1 -- 172.21.15.105:0/3574759033 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3eb00b3070 con 0x7f3eb40d1890 2023-11-28T18:23:53.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.379+0000 7f3eaa7fc640 1 --2- 172.21.15.105:0/3574759033 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3e980668d0 0x7f3e98068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.379+0000 7f3eaa7fc640 1 -- 172.21.15.105:0/3574759033 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3eb011ed00 con 0x7f3eb40d1890 2023-11-28T18:23:53.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.379+0000 7f3eb90c5640 1 --2- 172.21.15.105:0/3574759033 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3e980668d0 0x7f3e98068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.379+0000 7f3eb90c5640 1 --2- 172.21.15.105:0/3574759033 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3e980668d0 0x7f3e98068d90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3eb41f9760 tx=0x7f3ea4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.383+0000 7f3eaa7fc640 1 -- 172.21.15.105:0/3574759033 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3eb00ebd90 con 0x7f3eb40d1890 2023-11-28T18:23:53.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.531+0000 7f3ebbb2a640 1 -- 172.21.15.105:0/3574759033 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3eb40b1950 con 0x7f3eb40d1890 2023-11-28T18:23:53.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.535+0000 7f3eaa7fc640 1 -- 172.21.15.105:0/3574759033 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7f3eb00a8020 con 0x7f3eb40d1890 2023-11-28T18:23:53.537 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:53.537 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":24,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:23:38.458954+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:23:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.535+0000 7f3ebbb2a640 1 -- 172.21.15.105:0/3574759033 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3e980668d0 msgr2=0x7f3e98068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.535+0000 7f3ebbb2a640 1 --2- 172.21.15.105:0/3574759033 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3e980668d0 0x7f3e98068d90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3eb41f9760 tx=0x7f3ea4008040 comp rx=0 tx=0).stop 2023-11-28T18:23:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.535+0000 7f3ebbb2a640 1 -- 172.21.15.105:0/3574759033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eb40d1890 msgr2=0x7f3eb414c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.535+0000 7f3ebbb2a640 1 --2- 172.21.15.105:0/3574759033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eb40d1890 0x7f3eb414c3e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3eb00af040 tx=0x7f3eb0019210 comp rx=0 tx=0).stop 2023-11-28T18:23:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.535+0000 7f3ebbb2a640 1 -- 172.21.15.105:0/3574759033 shutdown_connections 2023-11-28T18:23:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.535+0000 7f3ebbb2a640 1 --2- 172.21.15.105:0/3574759033 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3eb415d4f0 0x7f3eb40ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.535+0000 7f3ebbb2a640 1 --2- 172.21.15.105:0/3574759033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3eb40d3c30 0x7f3eb40ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.535+0000 7f3ebbb2a640 1 --2- 172.21.15.105:0/3574759033 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3e980668d0 0x7f3e98068d90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.535+0000 7f3ebbb2a640 1 --2- 172.21.15.105:0/3574759033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eb40d1890 0x7f3eb414c3e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.535+0000 7f3ebbb2a640 1 -- 172.21.15.105:0/3574759033 >> 172.21.15.105:0/3574759033 conn(0x7f3eb40d3030 msgr2=0x7f3eb4156790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:53.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.535+0000 7f3ebbb2a640 1 -- 172.21.15.105:0/3574759033 shutdown_connections 2023-11-28T18:23:53.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.535+0000 7f3ebbb2a640 1 -- 172.21.15.105:0/3574759033 wait complete. 2023-11-28T18:23:53.558 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-28T18:23:53.559 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-28T18:23:53.559 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-28T18:23:53.560 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-28T18:23:53.560 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-28T18:23:53.560 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-28T18:23:53.561 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-28T18:23:53.561 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-28T18:23:53.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.795+0000 7f82ecc9a640 1 Processor -- start 2023-11-28T18:23:53.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.799+0000 7f82ecc9a640 1 -- start start 2023-11-28T18:23:53.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.799+0000 7f82ecc9a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e0038260 0x7f82e00eb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.799+0000 7f82ecc9a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e00ebf30 con 0x7f82e0006180 2023-11-28T18:23:53.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.799+0000 7f82ecc9a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e00e7bd0 con 0x7f82e00e77f0 2023-11-28T18:23:53.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.799+0000 7f82e659c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f82e00e77f0 0x7f82e00eb2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:47614/0 (socket says 172.21.15.105:47614) 2023-11-28T18:23:53.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.799+0000 7f82e659c640 1 -- 172.21.15.105:0/4131763390 learned_addr learned my addr 172.21.15.105:0/4131763390 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:53.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.799+0000 7f82e5d9b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e0038260 0x7f82e00eb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.799+0000 7f82ecc9a640 1 -- 172.21.15.105:0/4131763390 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e0057f00 con 0x7f82e0038260 2023-11-28T18:23:53.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.803+0000 7f82e559a640 1 -- 172.21.15.105:0/4131763390 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299994356 0 0) 0x7f82e00ebf30 con 0x7f82e0006180 2023-11-28T18:23:53.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.803+0000 7f82e559a640 1 -- 172.21.15.105:0/4131763390 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82d0003530 con 0x7f82e0006180 2023-11-28T18:23:53.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.803+0000 7f82e559a640 1 -- 172.21.15.105:0/4131763390 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 505469586 0 0) 0x7f82e00e7bd0 con 0x7f82e00e77f0 2023-11-28T18:23:53.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.803+0000 7f82e5d9b640 1 -- 172.21.15.105:0/4131763390 >> v1:172.21.15.148:6789/0 conn(0x7f82e00e77f0 legacy=0x7f82e00eb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.803+0000 7f82e5d9b640 1 -- 172.21.15.105:0/4131763390 >> v1:172.21.15.105:6789/0 conn(0x7f82e0006180 legacy=0x7f82e0006610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.803+0000 7f82e5d9b640 1 -- 172.21.15.105:0/4131763390 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e00581e0 con 0x7f82e0038260 2023-11-28T18:23:53.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.803+0000 7f5a02fe6640 1 Processor -- start 2023-11-28T18:23:53.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.803+0000 7f5a02fe6640 1 -- start start 2023-11-28T18:23:53.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.803+0000 7f8a20efe640 1 Processor -- start 2023-11-28T18:23:53.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.803+0000 7f8a20efe640 1 -- start start 2023-11-28T18:23:53.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f5a02fe6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59fc06c0e0 0x7f59fc0cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f82e5d9b640 1 --2- 172.21.15.105:0/4131763390 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e0038260 0x7f82e00eb9f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0097f0 tx=0x7f82dc01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=208c8d30d4e14634 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f59f3fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59fc06c0e0 0x7f59fc0cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f59f3fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59fc06c0e0 0x7f59fc0cdf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:37686/0 (socket says 172.21.15.105:37686) 2023-11-28T18:23:53.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f5a00d82640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f59fc06d9c0 0x7f59fc15b550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:47626/0 (socket says 172.21.15.105:47626) 2023-11-28T18:23:53.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f59f3fff640 1 -- 172.21.15.105:0/2805446340 learned_addr learned my addr 172.21.15.105:0/2805446340 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:53.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f5a02fe6640 1 --2- 172.21.15.105:0/2805446340 >> v2:172.21.15.105:3300/0 conn(0x7f59fc0ce4a0 0x7f59fc0b2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f5a02fe6640 1 -- 172.21.15.105:0/2805446340 --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59fc15c4f0 con 0x7f59fc06d9c0 2023-11-28T18:23:53.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f5a02fe6640 1 -- 172.21.15.105:0/2805446340 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59fc06dda0 con 0x7f59fc0ce4a0 2023-11-28T18:23:53.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f5a02fe6640 1 -- 172.21.15.105:0/2805446340 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc15bc60 con 0x7f59fc06c0e0 2023-11-28T18:23:53.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f8a20efe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8a1c0ad9f0 0x7f8a1c0ade80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f8a20efe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a1c0cc7d0 0x7f8a1c0ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f8a20efe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a1c162820 con 0x7f8a1c06d610 2023-11-28T18:23:53.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f8a20efe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a1c0ae3c0 con 0x7f8a1c0cc7d0 2023-11-28T18:23:53.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f8a20efe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a1c161c10 con 0x7f8a1c0ad9f0 2023-11-28T18:23:53.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f82e559a640 1 -- 172.21.15.105:0/4131763390 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc0ab650 con 0x7f82e0038260 2023-11-28T18:23:53.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f82e559a640 1 -- 172.21.15.105:0/4131763390 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82dc0ab7b0 con 0x7f82e0038260 2023-11-28T18:23:53.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f82e559a640 1 -- 172.21.15.105:0/4131763390 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc0abb40 con 0x7f82e0038260 2023-11-28T18:23:53.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f82ecc9a640 1 -- 172.21.15.105:0/4131763390 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e0038260 msgr2=0x7f82e00eb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f82ecc9a640 1 --2- 172.21.15.105:0/4131763390 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e0038260 0x7f82e00eb9f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0097f0 tx=0x7f82dc01c450 comp rx=0 tx=0).stop 2023-11-28T18:23:53.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f82ecc9a640 1 -- 172.21.15.105:0/4131763390 shutdown_connections 2023-11-28T18:23:53.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f82ecc9a640 1 --2- 172.21.15.105:0/4131763390 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e0038260 0x7f82e00eb9f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f82ecc9a640 1 -- 172.21.15.105:0/4131763390 >> 172.21.15.105:0/4131763390 conn(0x7f82e00559c0 msgr2=0x7f82e0055dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:53.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f8a19d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8a1c0ad9f0 0x7f8a1c0ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f8a1a59c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8a1c06d610 0x7f8a1c0cdff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:47636/0 (socket says 172.21.15.105:47636) 2023-11-28T18:23:53.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f8a1a59c640 1 -- 172.21.15.105:0/631757519 learned_addr learned my addr 172.21.15.105:0/631757519 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:53.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f8a1ad9d640 1 --2- 172.21.15.105:0/631757519 >> v2:172.21.15.105:3300/0 conn(0x7f8a1c0cc7d0 0x7f8a1c0ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f5a01583640 1 --2- 172.21.15.105:0/2805446340 >> v2:172.21.15.105:3300/0 conn(0x7f59fc0ce4a0 0x7f59fc0b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.807+0000 7f59f37fe640 1 -- 172.21.15.105:0/2805446340 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 483949713 0 0) 0x7f59fc15c4f0 con 0x7f59fc06d9c0 2023-11-28T18:23:53.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.811+0000 7f59f37fe640 1 -- 172.21.15.105:0/2805446340 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59e4003530 con 0x7f59fc06d9c0 2023-11-28T18:23:53.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.811+0000 7f59f3fff640 1 -- 172.21.15.105:0/2805446340 >> v2:172.21.15.105:3300/0 conn(0x7f59fc0ce4a0 msgr2=0x7f59fc0b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:53.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.811+0000 7f59f3fff640 1 --2- 172.21.15.105:0/2805446340 >> v2:172.21.15.105:3300/0 conn(0x7f59fc0ce4a0 0x7f59fc0b2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.811+0000 7f59f3fff640 1 -- 172.21.15.105:0/2805446340 >> v1:172.21.15.148:6789/0 conn(0x7f59fc06d9c0 legacy=0x7f59fc15b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.811+0000 7f59f3fff640 1 -- 172.21.15.105:0/2805446340 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59fc15bf30 con 0x7f59fc06c0e0 2023-11-28T18:23:53.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.811+0000 7f5a01583640 1 --2- 172.21.15.105:0/2805446340 >> v2:172.21.15.105:3300/0 conn(0x7f59fc0ce4a0 0x7f59fc0b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:53.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.811+0000 7f5be550f640 1 Processor -- start 2023-11-28T18:23:53.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.811+0000 7f8a19d9b640 1 -- 172.21.15.105:0/631757519 >> v2:172.21.15.105:3300/0 conn(0x7f8a1c0cc7d0 msgr2=0x7f8a1c0ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:53.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.811+0000 7f8a1959a640 1 -- 172.21.15.105:0/631757519 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 766498211 0 0) 0x7f8a1c162820 con 0x7f8a1c06d610 2023-11-28T18:23:53.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.811+0000 7f8a19d9b640 1 --2- 172.21.15.105:0/631757519 >> v2:172.21.15.105:3300/0 conn(0x7f8a1c0cc7d0 0x7f8a1c0ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.811+0000 7f8a19d9b640 1 -- 172.21.15.105:0/631757519 >> v1:172.21.15.148:6789/0 conn(0x7f8a1c06d610 legacy=0x7f8a1c0cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.811+0000 7f8a19d9b640 1 -- 172.21.15.105:0/631757519 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1c161ef0 con 0x7f8a1c0ad9f0 2023-11-28T18:23:53.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.811+0000 7f8a1ad9d640 1 --2- 172.21.15.105:0/631757519 >> v2:172.21.15.105:3300/0 conn(0x7f8a1c0cc7d0 0x7f8a1c0ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:53.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.811+0000 7f5be550f640 1 -- start start 2023-11-28T18:23:53.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.811+0000 7f82ecc9a640 1 -- 172.21.15.105:0/4131763390 shutdown_connections 2023-11-28T18:23:53.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.811+0000 7f82ecc9a640 1 -- 172.21.15.105:0/4131763390 wait complete. 2023-11-28T18:23:53.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.811+0000 7f5be550f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5be006d610 0x7f5be00cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.811+0000 7f5be550f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be00ce530 0x7f5be00ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.811+0000 7f5be550f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be00ae380 con 0x7f5be00adf30 2023-11-28T18:23:53.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.811+0000 7f5be550f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be0162820 con 0x7f5be006d610 2023-11-28T18:23:53.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.811+0000 7f5be550f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be0161c10 con 0x7f5be00ce530 2023-11-28T18:23:53.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f5bdeffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5be006d610 0x7f5be00cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f5bdf7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5be00adf30 0x7f5be01650f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:47638/0 (socket says 172.21.15.105:47638) 2023-11-28T18:23:53.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f5bdf7fe640 1 -- 172.21.15.105:0/3957196133 learned_addr learned my addr 172.21.15.105:0/3957196133 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:53.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f5bde7fc640 1 --2- 172.21.15.105:0/3957196133 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be00ce530 0x7f5be00ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f82ecc9a640 1 Processor -- start 2023-11-28T18:23:53.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f5bdeffd640 1 -- 172.21.15.105:0/3957196133 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be00ce530 msgr2=0x7f5be00ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:53.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f5bdeffd640 1 --2- 172.21.15.105:0/3957196133 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be00ce530 0x7f5be00ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f5bdeffd640 1 -- 172.21.15.105:0/3957196133 >> v1:172.21.15.148:6789/0 conn(0x7f5be00adf30 legacy=0x7f5be01650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f5bdeffd640 1 -- 172.21.15.105:0/3957196133 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be0161ef0 con 0x7f5be006d610 2023-11-28T18:23:53.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f8e3c9c4640 1 Processor -- start 2023-11-28T18:23:53.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f8e3c9c4640 1 -- start start 2023-11-28T18:23:53.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f59f3fff640 1 --2- 172.21.15.105:0/2805446340 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59fc06c0e0 0x7f59fc0cdf60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009870 tx=0x7f59ec01c2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e3d68e06a6bf589 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f59f37fe640 1 -- 172.21.15.105:0/2805446340 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8001680 con 0x7f59fc06c0e0 2023-11-28T18:23:53.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f59f37fe640 1 -- 172.21.15.105:0/2805446340 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ec0a7050 con 0x7f59fc06c0e0 2023-11-28T18:23:53.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f59f37fe640 1 -- 172.21.15.105:0/2805446340 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ec0ab620 con 0x7f59fc06c0e0 2023-11-28T18:23:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f8a19d9b640 1 --2- 172.21.15.105:0/631757519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a1c0ad9f0 0x7f8a1c0ade80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c073a50 tx=0x7f8a1001c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9435e83a6d9fb72 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f8a1959a640 1 -- 172.21.15.105:0/631757519 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a100ab600 con 0x7f8a1c0ad9f0 2023-11-28T18:23:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f8a1959a640 1 -- 172.21.15.105:0/631757519 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a100a7050 con 0x7f8a1c0ad9f0 2023-11-28T18:23:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f8a1959a640 1 -- 172.21.15.105:0/631757519 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a100ab980 con 0x7f8a1c0ad9f0 2023-11-28T18:23:53.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f5a02fe6640 1 -- 172.21.15.105:0/2805446340 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59fc06c0e0 msgr2=0x7f59fc0cdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f5a02fe6640 1 --2- 172.21.15.105:0/2805446340 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59fc06c0e0 0x7f59fc0cdf60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009870 tx=0x7f59ec01c2c0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f5a02fe6640 1 -- 172.21.15.105:0/2805446340 shutdown_connections 2023-11-28T18:23:53.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f5a02fe6640 1 --2- 172.21.15.105:0/2805446340 >> v2:172.21.15.105:3300/0 conn(0x7f59fc0ce4a0 0x7f59fc0b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f5a02fe6640 1 --2- 172.21.15.105:0/2805446340 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59fc06c0e0 0x7f59fc0cdf60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f5a02fe6640 1 -- 172.21.15.105:0/2805446340 >> 172.21.15.105:0/2805446340 conn(0x7f59fc06a750 msgr2=0x7f59fc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:53.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f82ecc9a640 1 -- start start 2023-11-28T18:23:53.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f5a02fe6640 1 -- 172.21.15.105:0/2805446340 shutdown_connections 2023-11-28T18:23:53.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f5a02fe6640 1 -- 172.21.15.105:0/2805446340 wait complete. 2023-11-28T18:23:53.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f8a20efe640 1 -- 172.21.15.105:0/631757519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a1c0ad9f0 msgr2=0x7f8a1c0ade80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.815+0000 7f8a20efe640 1 --2- 172.21.15.105:0/631757519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a1c0ad9f0 0x7f8a1c0ade80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c073a50 tx=0x7f8a1001c430 comp rx=0 tx=0).stop 2023-11-28T18:23:53.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f8a20efe640 1 -- 172.21.15.105:0/631757519 shutdown_connections 2023-11-28T18:23:53.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f8a20efe640 1 --2- 172.21.15.105:0/631757519 >> v2:172.21.15.105:3300/0 conn(0x7f8a1c0cc7d0 0x7f8a1c0ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f8e3c9c4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8e380ad130 0x7f8e38161780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f8e3c9c4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8e380ad510 0x7f8e3806dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f8e3c9c4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e380cb5a0 0x7f8e380cba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f8e3c9c4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e380cbf90 con 0x7f8e380ad130 2023-11-28T18:23:53.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f8e3c9c4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e38161cc0 con 0x7f8e380ad510 2023-11-28T18:23:53.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f8e3c9c4640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3806e020 con 0x7f8e380cb5a0 2023-11-28T18:23:53.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f8a20efe640 1 --2- 172.21.15.105:0/631757519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a1c0ad9f0 0x7f8a1c0ade80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f8a20efe640 1 -- 172.21.15.105:0/631757519 >> 172.21.15.105:0/631757519 conn(0x7f8a1c0ceb90 msgr2=0x7f8a1c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:53.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f8e36d9d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e380cb5a0 0x7f8e380cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f82ecc9a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e0006180 0x7f82e004f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f8e36d9d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e380cb5a0 0x7f8e380cba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:37702/0 (socket says 172.21.15.105:37702) 2023-11-28T18:23:53.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f8e36d9d640 1 -- 172.21.15.105:0/2653506422 learned_addr learned my addr 172.21.15.105:0/2653506422 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:53.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f8e35d9b640 1 --2- 172.21.15.105:0/2653506422 >> v2:172.21.15.148:3300/0 conn(0x7f8e380ad510 0x7f8e3806dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f8a20efe640 1 -- 172.21.15.105:0/631757519 shutdown_connections 2023-11-28T18:23:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f8a20efe640 1 -- 172.21.15.105:0/631757519 wait complete. 2023-11-28T18:23:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f82ecc9a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82e0038260 0x7f82e00d9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f82e659c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e0006180 0x7f82e004f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f82e659c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e0006180 0x7f82e004f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:37706/0 (socket says 172.21.15.105:37706) 2023-11-28T18:23:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f8e35d9b640 1 -- 172.21.15.105:0/2653506422 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e380cb5a0 msgr2=0x7f8e380cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f8e35d9b640 1 --2- 172.21.15.105:0/2653506422 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e380cb5a0 0x7f8e380cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f8e35d9b640 1 -- 172.21.15.105:0/2653506422 >> v2:172.21.15.105:3300/0 conn(0x7f8e380ad130 msgr2=0x7f8e38161780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:23:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f8e35d9b640 1 --2- 172.21.15.105:0/2653506422 >> v2:172.21.15.105:3300/0 conn(0x7f8e380ad130 0x7f8e38161780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f8e35d9b640 1 -- 172.21.15.105:0/2653506422 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e38160cc0 con 0x7f8e380ad510 2023-11-28T18:23:53.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7fe9d48d6640 1 Processor -- start 2023-11-28T18:23:53.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f82e659c640 1 -- 172.21.15.105:0/2149546953 learned_addr learned my addr 172.21.15.105:0/2149546953 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:53.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f82ecc9a640 1 --2- 172.21.15.105:0/2149546953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82e00e77f0 0x7f82e00da060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f5bdeffd640 1 --2- 172.21.15.105:0/3957196133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be006d610 0x7f5be00cdff0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5bd00097c0 tx=0x7f5bd001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9183c8367fb9390 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f5a02fe6640 1 Processor -- start 2023-11-28T18:23:53.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7fe9d48d6640 1 -- start start 2023-11-28T18:23:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f4e68c5f640 1 Processor -- start 2023-11-28T18:23:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f4e68c5f640 1 -- start start 2023-11-28T18:23:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f82e5d9b640 1 --2- 172.21.15.105:0/2149546953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82e0038260 0x7f82e00d9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f5bddffb640 1 -- 172.21.15.105:0/3957196133 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd00ab600 con 0x7f5be006d610 2023-11-28T18:23:53.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f5bddffb640 1 -- 172.21.15.105:0/3957196133 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd00a7050 con 0x7f5be006d610 2023-11-28T18:23:53.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f5bddffb640 1 -- 172.21.15.105:0/3957196133 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd00ab600 con 0x7f5be006d610 2023-11-28T18:23:53.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f82e6d9d640 1 --2- 172.21.15.105:0/2149546953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82e00e77f0 0x7f82e00da060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f82ecc9a640 1 -- 172.21.15.105:0/2149546953 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e004f8e0 con 0x7f82e00e77f0 2023-11-28T18:23:53.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f82ecc9a640 1 -- 172.21.15.105:0/2149546953 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e00d7f20 con 0x7f82e0038260 2023-11-28T18:23:53.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5be550f640 1 -- 172.21.15.105:0/3957196133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be006d610 msgr2=0x7f5be00cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5be550f640 1 --2- 172.21.15.105:0/3957196133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be006d610 0x7f5be00cdff0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5bd00097c0 tx=0x7f5bd001c470 comp rx=0 tx=0).stop 2023-11-28T18:23:53.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5be550f640 1 -- 172.21.15.105:0/3957196133 shutdown_connections 2023-11-28T18:23:53.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f82ecc9a640 1 -- 172.21.15.105:0/2149546953 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e00dc4f0 con 0x7f82e0006180 2023-11-28T18:23:53.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f5a02fe6640 1 -- start start 2023-11-28T18:23:53.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5a02fe6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59fc06c0e0 0x7f59fc144020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5a02fe6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59fc06d9c0 0x7f59fc13e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5a02fe6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59fc0ce4a0 0x7f59fc13da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5be550f640 1 --2- 172.21.15.105:0/3957196133 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be00ce530 0x7f5be00ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5be550f640 1 --2- 172.21.15.105:0/3957196133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be006d610 0x7f5be00cdff0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5be550f640 1 -- 172.21.15.105:0/3957196133 >> 172.21.15.105:0/3957196133 conn(0x7f5be00ceb90 msgr2=0x7f5be00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:53.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7fe9d48d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9c8033e40 0x7fe9c80eb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7fe9d48d6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe9c8047c20 0x7fe9c80eb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f4e68c5f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4e64071960 0x7f4e6415f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f4e68c5f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4e6415b030 0x7f4e640709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f4e68c5f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e640701d0 con 0x7f4e640b4290 2023-11-28T18:23:53.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f4e68c5f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e6406df80 con 0x7f4e6415b030 2023-11-28T18:23:53.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f4e68c5f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e64071df0 con 0x7f4e64071960 2023-11-28T18:23:53.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.819+0000 7f8a20efe640 1 Processor -- start 2023-11-28T18:23:53.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f8a20efe640 1 -- start start 2023-11-28T18:23:53.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7fe9d48d6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe9c80ebc00 0x7fe9c8052f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7fe9d48d6640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c8050210 con 0x7fe9c80ebc00 2023-11-28T18:23:53.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7fe9d48d6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c8037bc0 con 0x7fe9c8047c20 2023-11-28T18:23:53.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7fe9d48d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c804f090 con 0x7fe9c8033e40 2023-11-28T18:23:53.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5a02fe6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc144560 con 0x7f59fc06d9c0 2023-11-28T18:23:53.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5a02fe6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc13ccd0 con 0x7f59fc06c0e0 2023-11-28T18:23:53.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5a02fe6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc13cfb0 con 0x7f59fc0ce4a0 2023-11-28T18:23:53.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5a00d82640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59fc06c0e0 0x7f59fc144020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f59f3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59fc06d9c0 0x7f59fc13e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f59f3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59fc06d9c0 0x7f59fc13e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54964/0 (socket says 172.21.15.105:54964) 2023-11-28T18:23:53.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f59f3fff640 1 -- 172.21.15.105:0/2941538861 learned_addr learned my addr 172.21.15.105:0/2941538861 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:53.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5a00d82640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59fc06c0e0 0x7f59fc144020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34102/0 (socket says 172.21.15.105:34102) 2023-11-28T18:23:53.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5a01583640 1 --2- 172.21.15.105:0/2941538861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59fc0ce4a0 0x7f59fc13da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f8a20efe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a1c06d610 0x7f8a1c0b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f8a20efe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a1c0ad9f0 0x7f8a1c165060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f4e61d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4e64071960 0x7f4e6415f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f4e6259c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4e640b4290 0x7f4e640b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:47654/0 (socket says 172.21.15.105:47654) 2023-11-28T18:23:53.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f4e6259c640 1 -- 172.21.15.105:0/4081529877 learned_addr learned my addr 172.21.15.105:0/4081529877 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:53.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f82e659c640 1 -- 172.21.15.105:0/2149546953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82e0038260 msgr2=0x7f82e00d9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f82e659c640 1 --2- 172.21.15.105:0/2149546953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82e0038260 0x7f82e00d9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f82e659c640 1 -- 172.21.15.105:0/2149546953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82e00e77f0 msgr2=0x7f82e00da060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f82e659c640 1 --2- 172.21.15.105:0/2149546953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82e00e77f0 0x7f82e00da060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f8a20efe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a1c0cc7d0 0x7f8a1c147e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f82e659c640 1 -- 172.21.15.105:0/2149546953 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc009420 con 0x7f82e0006180 2023-11-28T18:23:53.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f8a1a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a1c06d610 0x7f8a1c0b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f8a1a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a1c06d610 0x7f8a1c0b19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34124/0 (socket says 172.21.15.105:34124) 2023-11-28T18:23:53.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f8a1a59c640 1 -- 172.21.15.105:0/1917019844 learned_addr learned my addr 172.21.15.105:0/1917019844 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:53.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5be550f640 1 -- 172.21.15.105:0/3957196133 shutdown_connections 2023-11-28T18:23:53.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5be550f640 1 -- 172.21.15.105:0/3957196133 wait complete. 2023-11-28T18:23:53.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5a01583640 1 -- 172.21.15.105:0/2941538861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59fc06c0e0 msgr2=0x7f59fc144020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5a01583640 1 --2- 172.21.15.105:0/2941538861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59fc06c0e0 0x7f59fc144020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f8a20efe640 1 -- 172.21.15.105:0/1917019844 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c1655a0 con 0x7f8a1c0cc7d0 2023-11-28T18:23:53.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5a01583640 1 -- 172.21.15.105:0/2941538861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59fc06d9c0 msgr2=0x7f59fc13e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5a01583640 1 --2- 172.21.15.105:0/2941538861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59fc06d9c0 0x7f59fc13e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f8a20efe640 1 -- 172.21.15.105:0/1917019844 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c148360 con 0x7f8a1c06d610 2023-11-28T18:23:53.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f4e6159a640 1 -- 172.21.15.105:0/4081529877 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2026960279 0 0) 0x7f4e640701d0 con 0x7f4e640b4290 2023-11-28T18:23:53.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f4e6159a640 1 -- 172.21.15.105:0/4081529877 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e4c003530 con 0x7f4e640b4290 2023-11-28T18:23:53.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5a01583640 1 -- 172.21.15.105:0/2941538861 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ec0094d0 con 0x7f59fc0ce4a0 2023-11-28T18:23:53.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f8a19d9b640 1 --2- 172.21.15.105:0/1917019844 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a1c0ad9f0 0x7f8a1c165060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f8a20efe640 1 -- 172.21.15.105:0/1917019844 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c0b2040 con 0x7f8a1c0ad9f0 2023-11-28T18:23:53.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7fe9cdd9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe9c8047c20 0x7fe9c80eb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f4e62d9d640 1 --2- 172.21.15.105:0/4081529877 >> v2:172.21.15.105:3300/0 conn(0x7f4e6415b030 0x7f4e640709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f4e61d9b640 1 -- 172.21.15.105:0/4081529877 >> v2:172.21.15.105:3300/0 conn(0x7f4e6415b030 msgr2=0x7f4e640709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:53.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7fe9cdd9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe9c8047c20 0x7fe9c80eb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34112/0 (socket says 172.21.15.105:34112) 2023-11-28T18:23:53.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7fe9ce59c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9c8033e40 0x7fe9c80eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7fe9cdd9b640 1 -- 172.21.15.105:0/2161688968 learned_addr learned my addr 172.21.15.105:0/2161688968 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:53.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f82e659c640 1 --2- 172.21.15.105:0/2149546953 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e0006180 0x7f82e004f3a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f82e804f6e0 tx=0x7f82e8067f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f4e61d9b640 1 --2- 172.21.15.105:0/4081529877 >> v2:172.21.15.105:3300/0 conn(0x7f4e6415b030 0x7f4e640709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f4e61d9b640 1 -- 172.21.15.105:0/4081529877 >> v1:172.21.15.148:6789/0 conn(0x7f4e640b4290 legacy=0x7f4e640b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f4e61d9b640 1 -- 172.21.15.105:0/4081529877 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e640720d0 con 0x7f4e64071960 2023-11-28T18:23:53.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f4e62d9d640 1 --2- 172.21.15.105:0/4081529877 >> v2:172.21.15.105:3300/0 conn(0x7f4e6415b030 0x7f4e640709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:53.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f5a01583640 1 --2- 172.21.15.105:0/2941538861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59fc0ce4a0 0x7f59fc13da90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f59f4007d40 tx=0x7f59f400d4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f8e35d9b640 1 --2- 172.21.15.105:0/2653506422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e380ad510 0x7f8e3806dae0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8e38071610 tx=0x7f8e2c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8a35d5cf8025e77 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f82d6ffd640 1 -- 172.21.15.105:0/2149546953 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8072eb0 con 0x7f82e0006180 2023-11-28T18:23:53.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7fe9ce59c640 1 -- 172.21.15.105:0/2161688968 >> v2:172.21.15.148:3300/0 conn(0x7fe9c8047c20 msgr2=0x7fe9c80eb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:53.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7fe9ce59c640 1 --2- 172.21.15.105:0/2161688968 >> v2:172.21.15.148:3300/0 conn(0x7fe9c8047c20 0x7fe9c80eb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7fe9ce59c640 1 -- 172.21.15.105:0/2161688968 >> v2:172.21.15.105:3300/0 conn(0x7fe9c80ebc00 msgr2=0x7fe9c8052f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:53.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7fe9ce59c640 1 --2- 172.21.15.105:0/2161688968 >> v2:172.21.15.105:3300/0 conn(0x7fe9c80ebc00 0x7fe9c8052f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7fe9ce59c640 1 -- 172.21.15.105:0/2161688968 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c804fd80 con 0x7fe9c8033e40 2023-11-28T18:23:53.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f82d6ffd640 1 -- 172.21.15.105:0/2149546953 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e808e070 con 0x7f82e0006180 2023-11-28T18:23:53.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f82d6ffd640 1 -- 172.21.15.105:0/2149546953 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8073210 con 0x7f82e0006180 2023-11-28T18:23:53.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7fe9cdd9b640 1 --2- 172.21.15.105:0/2161688968 >> v2:172.21.15.148:3300/0 conn(0x7fe9c8047c20 0x7fe9c80eb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:53.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f82ecc9a640 1 -- 172.21.15.105:0/2149546953 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e00dbc50 con 0x7f82e0006180 2023-11-28T18:23:53.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f8e3559a640 1 -- 172.21.15.105:0/2653506422 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c0ab610 con 0x7f8e380ad510 2023-11-28T18:23:53.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f82ecc9a640 1 -- 172.21.15.105:0/2149546953 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e00dc210 con 0x7f82e0006180 2023-11-28T18:23:53.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f82ecc9a640 1 -- 172.21.15.105:0/2149546953 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f82cc000fd0 con 0x7f82e0006180 2023-11-28T18:23:53.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.823+0000 7f8a1a59c640 1 -- 172.21.15.105:0/1917019844 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a1c0ad9f0 msgr2=0x7f8a1c165060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f8a1a59c640 1 --2- 172.21.15.105:0/1917019844 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a1c0ad9f0 0x7f8a1c165060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f8a1a59c640 1 -- 172.21.15.105:0/1917019844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a1c0cc7d0 msgr2=0x7f8a1c147e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f8a1a59c640 1 --2- 172.21.15.105:0/1917019844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a1c0cc7d0 0x7f8a1c147e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f8a1a59c640 1 -- 172.21.15.105:0/1917019844 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c009420 con 0x7f8a1c06d610 2023-11-28T18:23:53.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f8a19d9b640 1 --2- 172.21.15.105:0/1917019844 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a1c0ad9f0 0x7f8a1c165060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:53.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f5be550f640 1 Processor -- start 2023-11-28T18:23:53.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f5be550f640 1 -- start start 2023-11-28T18:23:53.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f8e3559a640 1 -- 172.21.15.105:0/2653506422 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2c0ab770 con 0x7f8e380ad510 2023-11-28T18:23:53.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f8e3559a640 1 -- 172.21.15.105:0/2653506422 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c0abb00 con 0x7f8e380ad510 2023-11-28T18:23:53.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f8e3c9c4640 1 -- 172.21.15.105:0/2653506422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e380ad510 msgr2=0x7f8e3806dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f8e3c9c4640 1 --2- 172.21.15.105:0/2653506422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e380ad510 0x7f8e3806dae0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8e38071610 tx=0x7f8e2c01c450 comp rx=0 tx=0).stop 2023-11-28T18:23:53.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f8a1a59c640 1 --2- 172.21.15.105:0/1917019844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a1c06d610 0x7f8a1c0b19b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400c780 tx=0x7f8a1400cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f8a0affd640 1 -- 172.21.15.105:0/1917019844 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1400ce90 con 0x7f8a1c06d610 2023-11-28T18:23:53.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f5be550f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be006d610 0x7f5be0164840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f5be550f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be00adf30 0x7f5be0152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f8a0affd640 1 -- 172.21.15.105:0/1917019844 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1400ce90 con 0x7f8a1c06d610 2023-11-28T18:23:53.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f8a20efe640 1 -- 172.21.15.105:0/1917019844 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a10009420 con 0x7f8a1c06d610 2023-11-28T18:23:53.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f8a20efe640 1 -- 172.21.15.105:0/1917019844 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a1c0b2600 con 0x7f8a1c06d610 2023-11-28T18:23:53.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f5be550f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be00ce530 0x7f5be00af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f8a0affd640 1 -- 172.21.15.105:0/1917019844 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1400a640 con 0x7f8a1c06d610 2023-11-28T18:23:53.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f59f17fa640 1 -- 172.21.15.105:0/2941538861 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f40039a0 con 0x7f59fc0ce4a0 2023-11-28T18:23:53.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f59f17fa640 1 -- 172.21.15.105:0/2941538861 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f400f040 con 0x7f59fc0ce4a0 2023-11-28T18:23:53.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f59f17fa640 1 -- 172.21.15.105:0/2941538861 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f4003d40 con 0x7f59fc0ce4a0 2023-11-28T18:23:53.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f5a02fe6640 1 -- 172.21.15.105:0/2941538861 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59fc13d290 con 0x7f59fc0ce4a0 2023-11-28T18:23:53.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f5bde7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be00adf30 0x7f5be0152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f5bde7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be00adf30 0x7f5be0152d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34140/0 (socket says 172.21.15.105:34140) 2023-11-28T18:23:53.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f5a02fe6640 1 -- 172.21.15.105:0/2941538861 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59fc13c9c0 con 0x7f59fc0ce4a0 2023-11-28T18:23:53.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f5bde7fc640 1 -- 172.21.15.105:0/2237642042 learned_addr learned my addr 172.21.15.105:0/2237642042 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:53.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f5be550f640 1 -- 172.21.15.105:0/2237642042 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be0162c00 con 0x7f5be00ce530 2023-11-28T18:23:53.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f5be550f640 1 -- 172.21.15.105:0/2237642042 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be0160900 con 0x7f5be00adf30 2023-11-28T18:23:53.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f5be550f640 1 -- 172.21.15.105:0/2237642042 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be0157420 con 0x7f5be006d610 2023-11-28T18:23:53.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f5bdf7fe640 1 --2- 172.21.15.105:0/2237642042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be00ce530 0x7f5be00af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f5bdeffd640 1 --2- 172.21.15.105:0/2237642042 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be006d610 0x7f5be0164840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f8e3c9c4640 1 -- 172.21.15.105:0/2653506422 shutdown_connections 2023-11-28T18:23:53.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f8e3c9c4640 1 --2- 172.21.15.105:0/2653506422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e380ad510 0x7f8e3806dae0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f8e3c9c4640 1 --2- 172.21.15.105:0/2653506422 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e380cb5a0 0x7f8e380cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f8e3c9c4640 1 --2- 172.21.15.105:0/2653506422 >> v2:172.21.15.105:3300/0 conn(0x7f8e380ad130 0x7f8e38161780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.827+0000 7f8e3c9c4640 1 -- 172.21.15.105:0/2653506422 >> 172.21.15.105:0/2653506422 conn(0x7f8e380b08c0 msgr2=0x7f8e380b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:53.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f8a20efe640 1 -- 172.21.15.105:0/1917019844 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8a00000fd0 con 0x7f8a1c06d610 2023-11-28T18:23:53.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f5bdeffd640 1 -- 172.21.15.105:0/2237642042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be00adf30 msgr2=0x7f5be0152d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f5bdeffd640 1 --2- 172.21.15.105:0/2237642042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be00adf30 0x7f5be0152d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f5bdeffd640 1 -- 172.21.15.105:0/2237642042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be00ce530 msgr2=0x7f5be00af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f5bdeffd640 1 --2- 172.21.15.105:0/2237642042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be00ce530 0x7f5be00af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f5bdeffd640 1 -- 172.21.15.105:0/2237642042 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd0009420 con 0x7f5be006d610 2023-11-28T18:23:53.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f5bdf7fe640 1 --2- 172.21.15.105:0/2237642042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be00ce530 0x7f5be00af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:53.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f5bdeffd640 1 --2- 172.21.15.105:0/2237642042 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be006d610 0x7f5be0164840 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5be006fac0 tx=0x7f5bd0019210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f5a02fe6640 1 -- 172.21.15.105:0/2941538861 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f59fc0b3060 con 0x7f59fc0ce4a0 2023-11-28T18:23:53.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f82d6ffd640 1 -- 172.21.15.105:0/2149546953 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f82e8073370 con 0x7f82e0006180 2023-11-28T18:23:53.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f8e3c9c4640 1 -- 172.21.15.105:0/2653506422 shutdown_connections 2023-11-28T18:23:53.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f8e3c9c4640 1 -- 172.21.15.105:0/2653506422 wait complete. 2023-11-28T18:23:53.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f5bc77fe640 1 -- 172.21.15.105:0/2237642042 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0003de0 con 0x7f5be006d610 2023-11-28T18:23:53.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f5bc77fe640 1 -- 172.21.15.105:0/2237642042 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd00b4430 con 0x7f5be006d610 2023-11-28T18:23:53.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f5bc77fe640 1 -- 172.21.15.105:0/2237642042 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd00b47e0 con 0x7f5be006d610 2023-11-28T18:23:53.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f5be550f640 1 -- 172.21.15.105:0/2237642042 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be015ffd0 con 0x7f5be006d610 2023-11-28T18:23:53.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f5be550f640 1 -- 172.21.15.105:0/2237642042 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be0160590 con 0x7f5be006d610 2023-11-28T18:23:53.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f8a0affd640 1 -- 172.21.15.105:0/1917019844 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8a1400a7a0 con 0x7f8a1c06d610 2023-11-28T18:23:53.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f4e61d9b640 1 --2- 172.21.15.105:0/4081529877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e64071960 0x7f4e6415f3b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4e6406e520 tx=0x7f4e5801c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4afe2924bc86e592 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f82d6ffd640 1 --2- 172.21.15.105:0/2149546953 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82c8066950 0x7f82c8068e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f4e6159a640 1 -- 172.21.15.105:0/4081529877 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e580ab600 con 0x7f4e64071960 2023-11-28T18:23:53.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f4e6159a640 1 -- 172.21.15.105:0/4081529877 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e580a7050 con 0x7f4e64071960 2023-11-28T18:23:53.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f4e6159a640 1 -- 172.21.15.105:0/4081529877 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e580ab980 con 0x7f4e64071960 2023-11-28T18:23:53.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f82d6ffd640 1 -- 172.21.15.105:0/2149546953 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f82e80fdde0 con 0x7f82e0006180 2023-11-28T18:23:53.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f82d6ffd640 1 --2- 172.21.15.105:0/2149546953 >> [v2:172.21.15.105:6824/2669077690,v1:172.21.15.105:6825/2669077690] conn(0x7f82c806ff90 0x7f82c80723d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f82d6ffd640 1 -- 172.21.15.105:0/2149546953 --> [v2:172.21.15.105:6824/2669077690,v1:172.21.15.105:6825/2669077690] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f82c8072aa0 con 0x7f82c806ff90 2023-11-28T18:23:53.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f82d6ffd640 1 -- 172.21.15.105:0/2149546953 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82e81003c0 con 0x7f82e0006180 2023-11-28T18:23:53.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7fe9ce59c640 1 --2- 172.21.15.105:0/2161688968 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9c8033e40 0x7fe9c80eb180 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe9c80524e0 tx=0x7fe9c001eb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ea820c9256260e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7fe9cd59a640 1 -- 172.21.15.105:0/2161688968 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00a8830 con 0x7fe9c8033e40 2023-11-28T18:23:53.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7fe9cd59a640 1 -- 172.21.15.105:0/2161688968 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c00a8990 con 0x7fe9c8033e40 2023-11-28T18:23:53.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7fe9cd59a640 1 -- 172.21.15.105:0/2161688968 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00a8d20 con 0x7fe9c8033e40 2023-11-28T18:23:53.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f8a0affd640 1 --2- 172.21.15.105:0/1917019844 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f89fc066930 0x7f89fc068df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f82e6d9d640 1 --2- 172.21.15.105:0/2149546953 >> [v2:172.21.15.105:6824/2669077690,v1:172.21.15.105:6825/2669077690] conn(0x7f82c806ff90 0x7f82c80723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f8a0affd640 1 -- 172.21.15.105:0/1917019844 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8a140841f0 con 0x7f8a1c06d610 2023-11-28T18:23:53.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f8a0affd640 1 --2- 172.21.15.105:0/1917019844 >> [v2:172.21.15.105:6816/2303306184,v1:172.21.15.105:6817/2303306184] conn(0x7f89fc06ff70 0x7f89fc0723b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f8a0affd640 1 -- 172.21.15.105:0/1917019844 --> [v2:172.21.15.105:6816/2303306184,v1:172.21.15.105:6817/2303306184] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f89fc072a80 con 0x7f89fc06ff70 2023-11-28T18:23:53.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f8e3c9c4640 1 Processor -- start 2023-11-28T18:23:53.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f8a0affd640 1 -- 172.21.15.105:0/1917019844 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a140845a0 con 0x7f8a1c06d610 2023-11-28T18:23:53.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.831+0000 7f59f17fa640 1 -- 172.21.15.105:0/2941538861 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f59f4004030 con 0x7f59fc0ce4a0 2023-11-28T18:23:53.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f59f17fa640 1 --2- 172.21.15.105:0/2941538861 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59e8066950 0x7f59e8068e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f59f17fa640 1 -- 172.21.15.105:0/2941538861 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f59f4084780 con 0x7f59fc0ce4a0 2023-11-28T18:23:53.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f59f17fa640 1 --2- 172.21.15.105:0/2941538861 >> [v2:172.21.15.105:6808/3491434256,v1:172.21.15.105:6809/3491434256] conn(0x7f59e806ff90 0x7f59e80723d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f59f17fa640 1 -- 172.21.15.105:0/2941538861 --> [v2:172.21.15.105:6808/3491434256,v1:172.21.15.105:6809/3491434256] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f59e8072aa0 con 0x7f59e806ff90 2023-11-28T18:23:53.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f59f17fa640 1 -- 172.21.15.105:0/2941538861 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59f40892c0 con 0x7f59fc0ce4a0 2023-11-28T18:23:53.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f8a1ad9d640 1 --2- 172.21.15.105:0/1917019844 >> [v2:172.21.15.105:6816/2303306184,v1:172.21.15.105:6817/2303306184] conn(0x7f89fc06ff70 0x7f89fc0723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f5be550f640 1 -- 172.21.15.105:0/2237642042 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5be00abd80 con 0x7f5be006d610 2023-11-28T18:23:53.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f8e3c9c4640 1 -- start start 2023-11-28T18:23:53.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f82e6d9d640 1 --2- 172.21.15.105:0/2149546953 >> [v2:172.21.15.105:6824/2669077690,v1:172.21.15.105:6825/2669077690] conn(0x7f82c806ff90 0x7f82c80723d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f5a00d82640 1 --2- 172.21.15.105:0/2941538861 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59e8066950 0x7f59e8068e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7fe9d48d6640 1 -- 172.21.15.105:0/2161688968 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9c8033e40 msgr2=0x7fe9c80eb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7fe9d48d6640 1 --2- 172.21.15.105:0/2161688968 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9c8033e40 0x7fe9c80eb180 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe9c80524e0 tx=0x7fe9c001eb90 comp rx=0 tx=0).stop 2023-11-28T18:23:53.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f82e5d9b640 1 --2- 172.21.15.105:0/2149546953 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82c8066950 0x7f82c8068e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7fab77501640 1 Processor -- start 2023-11-28T18:23:53.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f4e68c5f640 1 -- 172.21.15.105:0/4081529877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e64071960 msgr2=0x7f4e6415f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f4e68c5f640 1 --2- 172.21.15.105:0/4081529877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e64071960 0x7f4e6415f3b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4e6406e520 tx=0x7f4e5801c430 comp rx=0 tx=0).stop 2023-11-28T18:23:53.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f8a19d9b640 1 --2- 172.21.15.105:0/1917019844 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f89fc066930 0x7f89fc068df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f5bc77fe640 1 -- 172.21.15.105:0/2237642042 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5bd00b2040 con 0x7f5be006d610 2023-11-28T18:23:53.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7fe9d48d6640 1 -- 172.21.15.105:0/2161688968 shutdown_connections 2023-11-28T18:23:53.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7fe9d48d6640 1 --2- 172.21.15.105:0/2161688968 >> v2:172.21.15.148:3300/0 conn(0x7fe9c8047c20 0x7fe9c80eb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7fe9d48d6640 1 --2- 172.21.15.105:0/2161688968 >> v2:172.21.15.105:3300/0 conn(0x7fe9c80ebc00 0x7fe9c8052f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7fe9d48d6640 1 --2- 172.21.15.105:0/2161688968 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9c8033e40 0x7fe9c80eb180 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7fe9d48d6640 1 -- 172.21.15.105:0/2161688968 >> 172.21.15.105:0/2161688968 conn(0x7fe9c8009fc0 msgr2=0x7fe9c800a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:53.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f59f3fff640 1 --2- 172.21.15.105:0/2941538861 >> [v2:172.21.15.105:6808/3491434256,v1:172.21.15.105:6809/3491434256] conn(0x7f59e806ff90 0x7f59e80723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f4e68c5f640 1 -- 172.21.15.105:0/4081529877 shutdown_connections 2023-11-28T18:23:53.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f4e68c5f640 1 --2- 172.21.15.105:0/4081529877 >> v2:172.21.15.105:3300/0 conn(0x7f4e6415b030 0x7f4e640709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f4e68c5f640 1 --2- 172.21.15.105:0/4081529877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e64071960 0x7f4e6415f3b0 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4e6406e520 tx=0x7f4e5801c430 comp rx=0 tx=0).stop 2023-11-28T18:23:53.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f4e68c5f640 1 -- 172.21.15.105:0/4081529877 >> 172.21.15.105:0/4081529877 conn(0x7f4e640ceb90 msgr2=0x7f4e640cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:53.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7fab77501640 1 -- start start 2023-11-28T18:23:53.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f8a1ad9d640 1 --2- 172.21.15.105:0/1917019844 >> [v2:172.21.15.105:6816/2303306184,v1:172.21.15.105:6817/2303306184] conn(0x7f89fc06ff70 0x7f89fc0723b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f5bc77fe640 1 --2- 172.21.15.105:0/2237642042 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5bcc0669a0 0x7f5bcc068e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f5bc77fe640 1 -- 172.21.15.105:0/2237642042 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5bd011e820 con 0x7f5be006d610 2023-11-28T18:23:53.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f82d6ffd640 1 -- 172.21.15.105:0/2149546953 <== osd.0 v2:172.21.15.105:6824/2669077690 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f82c8072aa0 con 0x7f82c806ff90 2023-11-28T18:23:53.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f8a19d9b640 1 --2- 172.21.15.105:0/1917019844 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f89fc066930 0x7f89fc068df0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8a100ad000 tx=0x7f8a100a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f5bc77fe640 1 --2- 172.21.15.105:0/2237642042 >> [v2:172.21.15.105:6800/3228549411,v1:172.21.15.105:6801/3228549411] conn(0x7f5bcc06ffe0 0x7f5bcc072420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f5bc77fe640 1 -- 172.21.15.105:0/2237642042 --> [v2:172.21.15.105:6800/3228549411,v1:172.21.15.105:6801/3228549411] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5bcc072af0 con 0x7f5bcc06ffe0 2023-11-28T18:23:53.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f5bc77fe640 1 -- 172.21.15.105:0/2237642042 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bd0120d80 con 0x7f5be006d610 2023-11-28T18:23:53.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f5a00d82640 1 --2- 172.21.15.105:0/2941538861 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59e8066950 0x7f59e8068e10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f59f8000f70 tx=0x7f59f8001b20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7fe9d48d6640 1 -- 172.21.15.105:0/2161688968 shutdown_connections 2023-11-28T18:23:53.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f4e68c5f640 1 -- 172.21.15.105:0/4081529877 shutdown_connections 2023-11-28T18:23:53.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f59f3fff640 1 --2- 172.21.15.105:0/2941538861 >> [v2:172.21.15.105:6808/3491434256,v1:172.21.15.105:6809/3491434256] conn(0x7f59e806ff90 0x7f59e80723d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7f4e68c5f640 1 -- 172.21.15.105:0/4081529877 wait complete. 2023-11-28T18:23:53.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7fab77501640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab7006d610 0x7fab700cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7fab77501640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fab700ce530 0x7fab700ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f8a0affd640 1 -- 172.21.15.105:0/1917019844 <== osd.2 v2:172.21.15.105:6816/2303306184 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f89fc072a80 con 0x7f89fc06ff70 2023-11-28T18:23:53.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.835+0000 7fe9d48d6640 1 -- 172.21.15.105:0/2161688968 wait complete. 2023-11-28T18:23:53.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7fab77501640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fab700adf30 0x7fab701650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7fab77501640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab700ae380 con 0x7fab700ce530 2023-11-28T18:23:53.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7fab77501640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab70162820 con 0x7fab700adf30 2023-11-28T18:23:53.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7fab77501640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab70161c10 con 0x7fab7006d610 2023-11-28T18:23:53.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7fab7529d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab7006d610 0x7fab700cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f5bdf7fe640 1 --2- 172.21.15.105:0/2237642042 >> [v2:172.21.15.105:6800/3228549411,v1:172.21.15.105:6801/3228549411] conn(0x7f5bcc06ffe0 0x7f5bcc072420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7fab75a9e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fab700adf30 0x7fab701650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7fab7529d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab7006d610 0x7fab700cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:37752/0 (socket says 172.21.15.105:37752) 2023-11-28T18:23:53.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7fab7529d640 1 -- 172.21.15.105:0/2559653675 learned_addr learned my addr 172.21.15.105:0/2559653675 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:53.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7fab75a9e640 1 -- 172.21.15.105:0/2559653675 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab7006d610 msgr2=0x7fab700cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:53.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7fab75a9e640 1 --2- 172.21.15.105:0/2559653675 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab7006d610 0x7fab700cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7fab75a9e640 1 -- 172.21.15.105:0/2559653675 >> v2:172.21.15.105:3300/0 conn(0x7fab700ce530 msgr2=0x7fab700ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:53.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7fab74a9c640 1 --2- 172.21.15.105:0/2559653675 >> v2:172.21.15.105:3300/0 conn(0x7fab700ce530 0x7fab700ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7fab75a9e640 1 --2- 172.21.15.105:0/2559653675 >> v2:172.21.15.105:3300/0 conn(0x7fab700ce530 0x7fab700ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7fab75a9e640 1 -- 172.21.15.105:0/2559653675 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab70161ef0 con 0x7fab700adf30 2023-11-28T18:23:53.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f8e3c9c4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e380ad130 0x7f8e38146610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7fab74a9c640 1 --2- 172.21.15.105:0/2559653675 >> v2:172.21.15.105:3300/0 conn(0x7fab700ce530 0x7fab700ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:53.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f8e3c9c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e380ad510 0x7f8e38157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f8e3c9c4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e380cb5a0 0x7f8e381581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f8e3c9c4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e380b37f0 con 0x7f8e380ad130 2023-11-28T18:23:53.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f8e3c9c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3815d720 con 0x7f8e380ad510 2023-11-28T18:23:53.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f8e3c9c4640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e38146b50 con 0x7f8e380cb5a0 2023-11-28T18:23:53.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f8e35d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e380ad510 0x7f8e38157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f8e35d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e380ad510 0x7f8e38157c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34156/0 (socket says 172.21.15.105:34156) 2023-11-28T18:23:53.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f8e35d9b640 1 -- 172.21.15.105:0/1401582162 learned_addr learned my addr 172.21.15.105:0/1401582162 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:53.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f8e36d9d640 1 --2- 172.21.15.105:0/1401582162 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e380cb5a0 0x7f8e381581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f8e36d9d640 1 -- 172.21.15.105:0/1401582162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e380ad510 msgr2=0x7f8e38157c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f8e36d9d640 1 --2- 172.21.15.105:0/1401582162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e380ad510 0x7f8e38157c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f8e36d9d640 1 -- 172.21.15.105:0/1401582162 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e380ad130 msgr2=0x7f8e38146610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f8e36d9d640 1 --2- 172.21.15.105:0/1401582162 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e380ad130 0x7f8e38146610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f8e36d9d640 1 -- 172.21.15.105:0/1401582162 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2c009420 con 0x7f8e380cb5a0 2023-11-28T18:23:53.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f8e36d9d640 1 --2- 172.21.15.105:0/1401582162 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e380cb5a0 0x7f8e381581d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000eba0 tx=0x7f8e3000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f5bdf7fe640 1 --2- 172.21.15.105:0/2237642042 >> [v2:172.21.15.105:6800/3228549411,v1:172.21.15.105:6801/3228549411] conn(0x7f5bcc06ffe0 0x7f5bcc072420 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f4e68c5f640 1 Processor -- start 2023-11-28T18:23:53.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f8e26ffd640 1 -- 172.21.15.105:0/1401582162 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3000cb20 con 0x7f8e380cb5a0 2023-11-28T18:23:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f4e68c5f640 1 -- start start 2023-11-28T18:23:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f8e26ffd640 1 -- 172.21.15.105:0/1401582162 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3000f040 con 0x7f8e380cb5a0 2023-11-28T18:23:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f8e26ffd640 1 -- 172.21.15.105:0/1401582162 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3000cea0 con 0x7f8e380cb5a0 2023-11-28T18:23:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f8e3c9c4640 1 -- 172.21.15.105:0/1401582162 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e380b3230 con 0x7f8e380cb5a0 2023-11-28T18:23:53.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f59f17fa640 1 -- 172.21.15.105:0/2941538861 <== osd.1 v2:172.21.15.105:6808/3491434256 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f59e8072aa0 con 0x7f59e806ff90 2023-11-28T18:23:53.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7f8e3c9c4640 1 -- 172.21.15.105:0/1401582162 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3815d930 con 0x7f8e380cb5a0 2023-11-28T18:23:53.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.839+0000 7fe9d48d6640 1 Processor -- start 2023-11-28T18:23:53.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f5bde7fc640 1 --2- 172.21.15.105:0/2237642042 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5bcc0669a0 0x7f5bcc068e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f4e68c5f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e640b4290 0x7f4e641474f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f4e68c5f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e6415b030 0x7f4e640cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f4e68c5f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4e640cdcf0 0x7f4e641567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f4e68c5f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e640ce170 con 0x7f4e640b4290 2023-11-28T18:23:53.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f4e68c5f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e641586d0 con 0x7f4e6415b030 2023-11-28T18:23:53.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f4e68c5f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e641589b0 con 0x7f4e640cdcf0 2023-11-28T18:23:53.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f8e3c9c4640 1 -- 172.21.15.105:0/1401582162 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8e1c000fd0 con 0x7f8e380cb5a0 2023-11-28T18:23:53.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f5bc77fe640 1 -- 172.21.15.105:0/2237642042 <== osd.3 v2:172.21.15.105:6800/3228549411 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f5bcc072af0 con 0x7f5bcc06ffe0 2023-11-28T18:23:53.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7fe9d48d6640 1 -- start start 2023-11-28T18:23:53.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f82e5d9b640 1 --2- 172.21.15.105:0/2149546953 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82c8066950 0x7f82c8068e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0af000 tx=0x7f82dc0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f5bde7fc640 1 --2- 172.21.15.105:0/2237642042 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5bcc0669a0 0x7f5bcc068e60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5be0156700 tx=0x7f5bd4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7fe9d48d6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9c8033e40 0x7fe9c8036800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7fe9d48d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9c8047c20 0x7fe9c80de380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7fe9d48d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9c80ebc00 0x7fe9c80e4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7fe9d48d6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80e7940 con 0x7fe9c8033e40 2023-11-28T18:23:53.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7fe9d48d6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80de8c0 con 0x7fe9c8047c20 2023-11-28T18:23:53.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7fe9d48d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c8036e30 con 0x7fe9c80ebc00 2023-11-28T18:23:53.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f4e62d9d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4e640cdcf0 0x7f4e641567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f4e62d9d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4e640cdcf0 0x7f4e641567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:37770/0 (socket says 172.21.15.105:37770) 2023-11-28T18:23:53.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f4e62d9d640 1 -- 172.21.15.105:0/3729788372 learned_addr learned my addr 172.21.15.105:0/3729788372 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:53.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f4e61d9b640 1 --2- 172.21.15.105:0/3729788372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e6415b030 0x7f4e640cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f4e62d9d640 1 -- 172.21.15.105:0/3729788372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e6415b030 msgr2=0x7f4e640cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f4e62d9d640 1 --2- 172.21.15.105:0/3729788372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e6415b030 0x7f4e640cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f4e62d9d640 1 -- 172.21.15.105:0/3729788372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e640b4290 msgr2=0x7f4e641474f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f4e62d9d640 1 --2- 172.21.15.105:0/3729788372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e640b4290 0x7f4e641474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f4e62d9d640 1 -- 172.21.15.105:0/3729788372 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e58009420 con 0x7f4e640cdcf0 2023-11-28T18:23:53.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f4e61d9b640 1 --2- 172.21.15.105:0/3729788372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e6415b030 0x7f4e640cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:53.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f4e62d9d640 1 --2- 172.21.15.105:0/3729788372 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4e640cdcf0 0x7f4e641567e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00ccf0 tx=0x7f4e5c00a650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7fab75a9e640 1 --2- 172.21.15.105:0/2559653675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab700adf30 0x7fab701650f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fab701661b0 tx=0x7fab6801c620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2caa0611ea41a851 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7f8e26ffd640 1 -- 172.21.15.105:0/1401582162 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8e300039a0 con 0x7f8e380cb5a0 2023-11-28T18:23:53.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7fe9cdd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9c8047c20 0x7fe9c80de380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7fe9cdd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9c8047c20 0x7fe9c80de380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34182/0 (socket says 172.21.15.105:34182) 2023-11-28T18:23:53.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7fe9cdd9b640 1 -- 172.21.15.105:0/3473462336 learned_addr learned my addr 172.21.15.105:0/3473462336 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:53.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7fe9ced9d640 1 --2- 172.21.15.105:0/3473462336 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9c80ebc00 0x7fe9c80e4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7fab67fff640 1 -- 172.21.15.105:0/2559653675 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab680a66b0 con 0x7fab700adf30 2023-11-28T18:23:53.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7fab67fff640 1 -- 172.21.15.105:0/2559653675 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab680a6810 con 0x7fab700adf30 2023-11-28T18:23:53.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7fab67fff640 1 -- 172.21.15.105:0/2559653675 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab680a6ba0 con 0x7fab700adf30 2023-11-28T18:23:53.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.843+0000 7fab77501640 1 -- 172.21.15.105:0/2559653675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab700adf30 msgr2=0x7fab701650f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7fab77501640 1 --2- 172.21.15.105:0/2559653675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab700adf30 0x7fab701650f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fab701661b0 tx=0x7fab6801c620 comp rx=0 tx=0).stop 2023-11-28T18:23:53.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7fe9cdd9b640 1 -- 172.21.15.105:0/3473462336 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9c80ebc00 msgr2=0x7fe9c80e4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7fe9cdd9b640 1 --2- 172.21.15.105:0/3473462336 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9c80ebc00 0x7fe9c80e4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7fe9cdd9b640 1 -- 172.21.15.105:0/3473462336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9c8033e40 msgr2=0x7fe9c8036800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7fe9cdd9b640 1 --2- 172.21.15.105:0/3473462336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9c8033e40 0x7fe9c8036800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7fe9cdd9b640 1 -- 172.21.15.105:0/3473462336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d004f820 con 0x7fe9c8047c20 2023-11-28T18:23:53.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7fab77501640 1 -- 172.21.15.105:0/2559653675 shutdown_connections 2023-11-28T18:23:53.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7fab77501640 1 --2- 172.21.15.105:0/2559653675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab700adf30 0x7fab701650f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7fab77501640 1 --2- 172.21.15.105:0/2559653675 >> v2:172.21.15.105:3300/0 conn(0x7fab700ce530 0x7fab700ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7fab77501640 1 --2- 172.21.15.105:0/2559653675 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab7006d610 0x7fab700cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7fab77501640 1 -- 172.21.15.105:0/2559653675 >> 172.21.15.105:0/2559653675 conn(0x7fab700ceb90 msgr2=0x7fab700cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:53.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7f4e52ffd640 1 -- 172.21.15.105:0/3729788372 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c00f680 con 0x7f4e640cdcf0 2023-11-28T18:23:53.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7f4e52ffd640 1 -- 172.21.15.105:0/3729788372 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5c00f800 con 0x7f4e640cdcf0 2023-11-28T18:23:53.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7f4e52ffd640 1 -- 172.21.15.105:0/3729788372 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c00fbb0 con 0x7f4e640cdcf0 2023-11-28T18:23:53.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7f8e26ffd640 1 --2- 172.21.15.105:0/1401582162 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8e180668d0 0x7f8e18068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7f8e26ffd640 1 -- 172.21.15.105:0/1401582162 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8e30095bb0 con 0x7f8e380cb5a0 2023-11-28T18:23:53.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7f8e26ffd640 1 --2- 172.21.15.105:0/1401582162 >> [v2:172.21.15.148:6800/1517755666,v1:172.21.15.148:6801/1517755666] conn(0x7f8e1806ff10 0x7f8e18072350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7f8e26ffd640 1 -- 172.21.15.105:0/1401582162 --> [v2:172.21.15.148:6800/1517755666,v1:172.21.15.148:6801/1517755666] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8e18072a20 con 0x7f8e1806ff10 2023-11-28T18:23:53.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7f8e26ffd640 1 -- 172.21.15.105:0/1401582162 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e3009a2a0 con 0x7f8e380cb5a0 2023-11-28T18:23:53.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7f8e3659c640 1 --2- 172.21.15.105:0/1401582162 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8e180668d0 0x7f8e18068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7fe9cdd9b640 1 --2- 172.21.15.105:0/3473462336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9c8047c20 0x7fe9c80de380 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0079a10 tx=0x7fe9d0067f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7fe9beffd640 1 -- 172.21.15.105:0/3473462336 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d008f070 con 0x7fe9c8047c20 2023-11-28T18:23:53.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7fe9d48d6640 1 -- 172.21.15.105:0/3473462336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c0009420 con 0x7fe9c8047c20 2023-11-28T18:23:53.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7fe9d48d6640 1 -- 172.21.15.105:0/3473462336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c80e7b20 con 0x7fe9c8047c20 2023-11-28T18:23:53.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7fe9beffd640 1 -- 172.21.15.105:0/3473462336 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d0074000 con 0x7fe9c8047c20 2023-11-28T18:23:53.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7fe9beffd640 1 -- 172.21.15.105:0/3473462336 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d008f070 con 0x7fe9c8047c20 2023-11-28T18:23:53.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7fab77501640 1 -- 172.21.15.105:0/2559653675 shutdown_connections 2023-11-28T18:23:53.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7fab77501640 1 -- 172.21.15.105:0/2559653675 wait complete. 2023-11-28T18:23:53.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7fe9d48d6640 1 -- 172.21.15.105:0/3473462336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe9b4000fd0 con 0x7fe9c8047c20 2023-11-28T18:23:53.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7f8e35d9b640 1 --2- 172.21.15.105:0/1401582162 >> [v2:172.21.15.148:6800/1517755666,v1:172.21.15.148:6801/1517755666] conn(0x7f8e1806ff10 0x7f8e18072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7f8e35d9b640 1 --2- 172.21.15.105:0/1401582162 >> [v2:172.21.15.148:6800/1517755666,v1:172.21.15.148:6801/1517755666] conn(0x7f8e1806ff10 0x7f8e18072350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7f4e68c5f640 1 -- 172.21.15.105:0/3729788372 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e64158c90 con 0x7f4e640cdcf0 2023-11-28T18:23:53.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.847+0000 7f4e68c5f640 1 -- 172.21.15.105:0/3729788372 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e64159250 con 0x7f4e640cdcf0 2023-11-28T18:23:53.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.851+0000 7f8e3659c640 1 --2- 172.21.15.105:0/1401582162 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8e180668d0 0x7f8e18068d90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8e280096f0 tx=0x7f8e28006c50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.851+0000 7f8e26ffd640 1 -- 172.21.15.105:0/1401582162 <== osd.4 v2:172.21.15.148:6800/1517755666 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f8e18072a20 con 0x7f8e1806ff10 2023-11-28T18:23:53.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.851+0000 7fe9beffd640 1 -- 172.21.15.105:0/3473462336 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fe9d0072eb0 con 0x7fe9c8047c20 2023-11-28T18:23:53.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.851+0000 7fab77501640 1 Processor -- start 2023-11-28T18:23:53.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.851+0000 7fab77501640 1 -- start start 2023-11-28T18:23:53.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.851+0000 7f4e52ffd640 1 -- 172.21.15.105:0/3729788372 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f4e5c00b040 con 0x7f4e640cdcf0 2023-11-28T18:23:53.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.851+0000 7fe9beffd640 1 --2- 172.21.15.105:0/3473462336 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe9b0066930 0x7fe9b0068df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.851+0000 7fe9ce59c640 1 --2- 172.21.15.105:0/3473462336 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe9b0066930 0x7fe9b0068df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.851+0000 7fe9beffd640 1 -- 172.21.15.105:0/3473462336 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe9d00fe070 con 0x7fe9c8047c20 2023-11-28T18:23:53.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.851+0000 7fe9beffd640 1 --2- 172.21.15.105:0/3473462336 >> [v2:172.21.15.148:6824/3981624325,v1:172.21.15.148:6825/3981624325] conn(0x7fe9b006ff70 0x7fe9b00723b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.851+0000 7fe9beffd640 1 -- 172.21.15.105:0/3473462336 --> [v2:172.21.15.148:6824/3981624325,v1:172.21.15.148:6825/3981624325] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe9b0072a80 con 0x7fe9b006ff70 2023-11-28T18:23:53.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.851+0000 7fe9beffd640 1 -- 172.21.15.105:0/3473462336 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9d00fe420 con 0x7fe9c8047c20 2023-11-28T18:23:53.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.851+0000 7fab77501640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab7006d610 0x7fab7015af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.851+0000 7fab77501640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab700adf30 0x7fab700b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.851+0000 7fab77501640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab700ce530 0x7fab7009b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7f4e68c5f640 1 -- 172.21.15.105:0/3729788372 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4e64067570 con 0x7f4e640cdcf0 2023-11-28T18:23:53.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fab77501640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab700b2780 con 0x7fab700adf30 2023-11-28T18:23:53.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fab7529d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab7006d610 0x7fab7015af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fab74a9c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab700adf30 0x7fab700b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fab7529d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab7006d610 0x7fab7015af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:37798/0 (socket says 172.21.15.105:37798) 2023-11-28T18:23:53.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fab7529d640 1 -- 172.21.15.105:0/1209795070 learned_addr learned my addr 172.21.15.105:0/1209795070 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:53.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fab75a9e640 1 --2- 172.21.15.105:0/1209795070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab700ce530 0x7fab7009b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fab77501640 1 -- 172.21.15.105:0/1209795070 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7009c0a0 con 0x7fab700ce530 2023-11-28T18:23:53.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fe9ced9d640 1 --2- 172.21.15.105:0/3473462336 >> [v2:172.21.15.148:6824/3981624325,v1:172.21.15.148:6825/3981624325] conn(0x7fe9b006ff70 0x7fe9b00723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fab77501640 1 -- 172.21.15.105:0/1209795070 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab700b29b0 con 0x7fab7006d610 2023-11-28T18:23:53.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7f4e52ffd640 1 --2- 172.21.15.105:0/3729788372 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4e44066950 0x7f4e44068e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fab7529d640 1 -- 172.21.15.105:0/1209795070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab700ce530 msgr2=0x7fab7009b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fab7529d640 1 --2- 172.21.15.105:0/1209795070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab700ce530 0x7fab7009b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fab7529d640 1 -- 172.21.15.105:0/1209795070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab700adf30 msgr2=0x7fab700b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fab7529d640 1 --2- 172.21.15.105:0/1209795070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab700adf30 0x7fab700b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fe9ce59c640 1 --2- 172.21.15.105:0/3473462336 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe9b0066930 0x7fe9b0068df0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00b2000 tx=0x7fe9c0004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fab7529d640 1 -- 172.21.15.105:0/1209795070 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab68009a70 con 0x7fab7006d610 2023-11-28T18:23:53.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fab74a9c640 1 --2- 172.21.15.105:0/1209795070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab700adf30 0x7fab700b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:53.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fab7529d640 1 --2- 172.21.15.105:0/1209795070 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab7006d610 0x7fab7015af70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00e910 tx=0x7fab6c00edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fab65ffb640 1 -- 172.21.15.105:0/1209795070 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c009640 con 0x7fab7006d610 2023-11-28T18:23:53.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fab65ffb640 1 -- 172.21.15.105:0/1209795070 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6c00ca00 con 0x7fab7006d610 2023-11-28T18:23:53.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fab65ffb640 1 -- 172.21.15.105:0/1209795070 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c00cd70 con 0x7fab7006d610 2023-11-28T18:23:53.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fab77501640 1 -- 172.21.15.105:0/1209795070 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab700b2c90 con 0x7fab7006d610 2023-11-28T18:23:53.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fab77501640 1 -- 172.21.15.105:0/1209795070 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7009c380 con 0x7fab7006d610 2023-11-28T18:23:53.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7fab77501640 1 -- 172.21.15.105:0/1209795070 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fab50000fd0 con 0x7fab7006d610 2023-11-28T18:23:53.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.855+0000 7f4e52ffd640 1 -- 172.21.15.105:0/3729788372 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4e5c085180 con 0x7f4e640cdcf0 2023-11-28T18:23:53.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.859+0000 7f4e52ffd640 1 --2- 172.21.15.105:0/3729788372 >> [v2:172.21.15.148:6809/4279905987,v1:172.21.15.148:6811/4279905987] conn(0x7f4e4406ff10 0x7f4e44072350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.859+0000 7f4e52ffd640 1 -- 172.21.15.105:0/3729788372 --> [v2:172.21.15.148:6809/4279905987,v1:172.21.15.148:6811/4279905987] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e44072a20 con 0x7f4e4406ff10 2023-11-28T18:23:53.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.859+0000 7f4e52ffd640 1 -- 172.21.15.105:0/3729788372 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e5c007900 con 0x7f4e640cdcf0 2023-11-28T18:23:53.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.859+0000 7f4e6259c640 1 --2- 172.21.15.105:0/3729788372 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4e44066950 0x7f4e44068e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.859+0000 7f4e61d9b640 1 --2- 172.21.15.105:0/3729788372 >> [v2:172.21.15.148:6809/4279905987,v1:172.21.15.148:6811/4279905987] conn(0x7f4e4406ff10 0x7f4e44072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.859+0000 7fe9ced9d640 1 --2- 172.21.15.105:0/3473462336 >> [v2:172.21.15.148:6824/3981624325,v1:172.21.15.148:6825/3981624325] conn(0x7fe9b006ff70 0x7fe9b00723b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.859+0000 7f4e61d9b640 1 --2- 172.21.15.105:0/3729788372 >> [v2:172.21.15.148:6809/4279905987,v1:172.21.15.148:6811/4279905987] conn(0x7f4e4406ff10 0x7f4e44072350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.859+0000 7fe9beffd640 1 -- 172.21.15.105:0/3473462336 <== osd.5 v2:172.21.15.148:6824/3981624325 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fe9b0072a80 con 0x7fe9b006ff70 2023-11-28T18:23:53.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.859+0000 7f4e6259c640 1 --2- 172.21.15.105:0/3729788372 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4e44066950 0x7f4e44068e10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4e540011f0 tx=0x7f4e540078a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.859+0000 7f4e52ffd640 1 -- 172.21.15.105:0/3729788372 <== osd.6 v2:172.21.15.148:6809/4279905987 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f4e44072a20 con 0x7f4e4406ff10 2023-11-28T18:23:53.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.863+0000 7fab65ffb640 1 -- 172.21.15.105:0/1209795070 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fab6c013470 con 0x7fab7006d610 2023-11-28T18:23:53.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.863+0000 7f82ecc9a640 1 -- 172.21.15.105:0/2149546953 --> [v2:172.21.15.105:6824/2669077690,v1:172.21.15.105:6825/2669077690] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f82cc002cc0 con 0x7f82c806ff90 2023-11-28T18:23:53.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.863+0000 7fab65ffb640 1 --2- 172.21.15.105:0/1209795070 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fab540669a0 0x7fab54068e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.863+0000 7fab65ffb640 1 -- 172.21.15.105:0/1209795070 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fab6c084b90 con 0x7fab7006d610 2023-11-28T18:23:53.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.863+0000 7fab65ffb640 1 --2- 172.21.15.105:0/1209795070 >> [v2:172.21.15.148:6808/3280605225,v1:172.21.15.148:6810/3280605225] conn(0x7fab5406ffe0 0x7fab54072420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:53.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.863+0000 7fab65ffb640 1 -- 172.21.15.105:0/1209795070 --> [v2:172.21.15.148:6808/3280605225,v1:172.21.15.148:6810/3280605225] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab54072af0 con 0x7fab5406ffe0 2023-11-28T18:23:53.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.863+0000 7fab65ffb640 1 -- 172.21.15.105:0/1209795070 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab6c084f40 con 0x7fab7006d610 2023-11-28T18:23:53.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.863+0000 7fab74a9c640 1 --2- 172.21.15.105:0/1209795070 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fab540669a0 0x7fab54068e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.863+0000 7fab75a9e640 1 --2- 172.21.15.105:0/1209795070 >> [v2:172.21.15.148:6808/3280605225,v1:172.21.15.148:6810/3280605225] conn(0x7fab5406ffe0 0x7fab54072420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:53.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.863+0000 7fab75a9e640 1 --2- 172.21.15.105:0/1209795070 >> [v2:172.21.15.148:6808/3280605225,v1:172.21.15.148:6810/3280605225] conn(0x7fab5406ffe0 0x7fab54072420 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.867+0000 7f8a20efe640 1 -- 172.21.15.105:0/1917019844 --> [v2:172.21.15.105:6816/2303306184,v1:172.21.15.105:6817/2303306184] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8a00002cc0 con 0x7f89fc06ff70 2023-11-28T18:23:53.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.867+0000 7f82d6ffd640 1 -- 172.21.15.105:0/2149546953 <== osd.0 v2:172.21.15.105:6824/2669077690 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f82cc002cc0 con 0x7f82c806ff90 2023-11-28T18:23:53.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.867+0000 7fab65ffb640 1 -- 172.21.15.105:0/1209795070 <== osd.7 v2:172.21.15.148:6808/3280605225 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fab54072af0 con 0x7fab5406ffe0 2023-11-28T18:23:53.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.867+0000 7fab74a9c640 1 --2- 172.21.15.105:0/1209795070 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fab540669a0 0x7fab54068e60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fab70164920 tx=0x7fab60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:53.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.867+0000 7f82ecc9a640 1 -- 172.21.15.105:0/2149546953 >> [v2:172.21.15.105:6824/2669077690,v1:172.21.15.105:6825/2669077690] conn(0x7f82c806ff90 msgr2=0x7f82c80723d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.867+0000 7f82ecc9a640 1 --2- 172.21.15.105:0/2149546953 >> [v2:172.21.15.105:6824/2669077690,v1:172.21.15.105:6825/2669077690] conn(0x7f82c806ff90 0x7f82c80723d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.867+0000 7f82ecc9a640 1 -- 172.21.15.105:0/2149546953 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82c8066950 msgr2=0x7f82c8068e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.867+0000 7f82ecc9a640 1 --2- 172.21.15.105:0/2149546953 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82c8066950 0x7f82c8068e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0af000 tx=0x7f82dc0a7000 comp rx=0 tx=0).stop 2023-11-28T18:23:53.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.867+0000 7f82ecc9a640 1 -- 172.21.15.105:0/2149546953 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e0006180 msgr2=0x7f82e004f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.867+0000 7f82ecc9a640 1 --2- 172.21.15.105:0/2149546953 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e0006180 0x7f82e004f3a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f82e804f6e0 tx=0x7f82e8067f90 comp rx=0 tx=0).stop 2023-11-28T18:23:53.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.867+0000 7f82e6d9d640 1 -- 172.21.15.105:0/2149546953 reap_dead start 2023-11-28T18:23:53.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.867+0000 7f82ecc9a640 1 -- 172.21.15.105:0/2149546953 shutdown_connections 2023-11-28T18:23:53.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.867+0000 7f82ecc9a640 1 -- 172.21.15.105:0/2149546953 >> 172.21.15.105:0/2149546953 conn(0x7f82e00559c0 msgr2=0x7f82e003a190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:53.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.871+0000 7f82ecc9a640 1 -- 172.21.15.105:0/2149546953 shutdown_connections 2023-11-28T18:23:53.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.871+0000 7f82ecc9a640 1 -- 172.21.15.105:0/2149546953 wait complete. 2023-11-28T18:23:53.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.871+0000 7f8a0affd640 1 -- 172.21.15.105:0/1917019844 <== osd.2 v2:172.21.15.105:6816/2303306184 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8a00002cc0 con 0x7f89fc06ff70 2023-11-28T18:23:53.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.871+0000 7f8e3c9c4640 1 -- 172.21.15.105:0/1401582162 --> [v2:172.21.15.148:6800/1517755666,v1:172.21.15.148:6801/1517755666] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8e1c002cc0 con 0x7f8e1806ff10 2023-11-28T18:23:53.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.871+0000 7f8a20efe640 1 -- 172.21.15.105:0/1917019844 >> [v2:172.21.15.105:6816/2303306184,v1:172.21.15.105:6817/2303306184] conn(0x7f89fc06ff70 msgr2=0x7f89fc0723b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.871+0000 7f8a20efe640 1 --2- 172.21.15.105:0/1917019844 >> [v2:172.21.15.105:6816/2303306184,v1:172.21.15.105:6817/2303306184] conn(0x7f89fc06ff70 0x7f89fc0723b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.871+0000 7f8a20efe640 1 -- 172.21.15.105:0/1917019844 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f89fc066930 msgr2=0x7f89fc068df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.871+0000 7f8a20efe640 1 --2- 172.21.15.105:0/1917019844 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f89fc066930 0x7f89fc068df0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8a100ad000 tx=0x7f8a100a6000 comp rx=0 tx=0).stop 2023-11-28T18:23:53.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.871+0000 7f8a20efe640 1 -- 172.21.15.105:0/1917019844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a1c06d610 msgr2=0x7f8a1c0b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.871+0000 7f8a20efe640 1 --2- 172.21.15.105:0/1917019844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a1c06d610 0x7f8a1c0b19b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400c780 tx=0x7f8a1400cc40 comp rx=0 tx=0).stop 2023-11-28T18:23:53.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.871+0000 7f8a1ad9d640 1 -- 172.21.15.105:0/1917019844 reap_dead start 2023-11-28T18:23:53.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.871+0000 7f8e26ffd640 1 -- 172.21.15.105:0/1401582162 <== osd.4 v2:172.21.15.148:6800/1517755666 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8e1c002cc0 con 0x7f8e1806ff10 2023-11-28T18:23:53.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.875+0000 7f5be550f640 1 -- 172.21.15.105:0/2237642042 --> [v2:172.21.15.105:6800/3228549411,v1:172.21.15.105:6801/3228549411] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5be0067570 con 0x7f5bcc06ffe0 2023-11-28T18:23:53.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.875+0000 7f8e3c9c4640 1 -- 172.21.15.105:0/1401582162 >> [v2:172.21.15.148:6800/1517755666,v1:172.21.15.148:6801/1517755666] conn(0x7f8e1806ff10 msgr2=0x7f8e18072350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.875+0000 7f8e3c9c4640 1 --2- 172.21.15.105:0/1401582162 >> [v2:172.21.15.148:6800/1517755666,v1:172.21.15.148:6801/1517755666] conn(0x7f8e1806ff10 0x7f8e18072350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.875+0000 7f8a20efe640 1 -- 172.21.15.105:0/1917019844 shutdown_connections 2023-11-28T18:23:53.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.875+0000 7f8a20efe640 1 -- 172.21.15.105:0/1917019844 >> 172.21.15.105:0/1917019844 conn(0x7f8a1c0ceb90 msgr2=0x7f8a1c0af130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:53.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.875+0000 7f8a20efe640 1 -- 172.21.15.105:0/1917019844 shutdown_connections 2023-11-28T18:23:53.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.875+0000 7f8a20efe640 1 -- 172.21.15.105:0/1917019844 wait complete. 2023-11-28T18:23:53.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.875+0000 7f8e3c9c4640 1 -- 172.21.15.105:0/1401582162 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8e180668d0 msgr2=0x7f8e18068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.875+0000 7f8e3c9c4640 1 --2- 172.21.15.105:0/1401582162 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8e180668d0 0x7f8e18068d90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8e280096f0 tx=0x7f8e28006c50 comp rx=0 tx=0).stop 2023-11-28T18:23:53.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.875+0000 7f8e3c9c4640 1 -- 172.21.15.105:0/1401582162 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e380cb5a0 msgr2=0x7f8e381581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.875+0000 7f8e3c9c4640 1 --2- 172.21.15.105:0/1401582162 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e380cb5a0 0x7f8e381581d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000eba0 tx=0x7f8e3000c490 comp rx=0 tx=0).stop 2023-11-28T18:23:53.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.875+0000 7f8e36d9d640 1 -- 172.21.15.105:0/1401582162 reap_dead start 2023-11-28T18:23:53.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.875+0000 7f8e3c9c4640 1 -- 172.21.15.105:0/1401582162 shutdown_connections 2023-11-28T18:23:53.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.875+0000 7f8e3c9c4640 1 -- 172.21.15.105:0/1401582162 >> 172.21.15.105:0/1401582162 conn(0x7f8e380b08c0 msgr2=0x7f8e380b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:53.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.875+0000 7f8e3c9c4640 1 -- 172.21.15.105:0/1401582162 shutdown_connections 2023-11-28T18:23:53.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.875+0000 7f5a02fe6640 1 -- 172.21.15.105:0/2941538861 --> [v2:172.21.15.105:6808/3491434256,v1:172.21.15.105:6809/3491434256] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f59fc13d7b0 con 0x7f59e806ff90 2023-11-28T18:23:53.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.875+0000 7f5bc77fe640 1 -- 172.21.15.105:0/2237642042 <== osd.3 v2:172.21.15.105:6800/3228549411 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5be0067570 con 0x7f5bcc06ffe0 2023-11-28T18:23:53.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.875+0000 7f8e3c9c4640 1 -- 172.21.15.105:0/1401582162 wait complete. 2023-11-28T18:23:53.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.879+0000 7f5be550f640 1 -- 172.21.15.105:0/2237642042 >> [v2:172.21.15.105:6800/3228549411,v1:172.21.15.105:6801/3228549411] conn(0x7f5bcc06ffe0 msgr2=0x7f5bcc072420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.879+0000 7f5be550f640 1 --2- 172.21.15.105:0/2237642042 >> [v2:172.21.15.105:6800/3228549411,v1:172.21.15.105:6801/3228549411] conn(0x7f5bcc06ffe0 0x7f5bcc072420 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.879+0000 7f59f17fa640 1 -- 172.21.15.105:0/2941538861 <== osd.1 v2:172.21.15.105:6808/3491434256 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f59fc13d7b0 con 0x7f59e806ff90 2023-11-28T18:23:53.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.879+0000 7f5be550f640 1 -- 172.21.15.105:0/2237642042 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5bcc0669a0 msgr2=0x7f5bcc068e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.879+0000 7f5be550f640 1 --2- 172.21.15.105:0/2237642042 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5bcc0669a0 0x7f5bcc068e60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5be0156700 tx=0x7f5bd4008040 comp rx=0 tx=0).stop 2023-11-28T18:23:53.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.879+0000 7f5be550f640 1 -- 172.21.15.105:0/2237642042 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be006d610 msgr2=0x7f5be0164840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.879+0000 7f5be550f640 1 --2- 172.21.15.105:0/2237642042 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be006d610 0x7f5be0164840 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5be006fac0 tx=0x7f5bd0019210 comp rx=0 tx=0).stop 2023-11-28T18:23:53.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.879+0000 7f5bdf7fe640 1 -- 172.21.15.105:0/2237642042 reap_dead start 2023-11-28T18:23:53.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.879+0000 7f5be550f640 1 -- 172.21.15.105:0/2237642042 shutdown_connections 2023-11-28T18:23:53.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.879+0000 7f5be550f640 1 -- 172.21.15.105:0/2237642042 >> 172.21.15.105:0/2237642042 conn(0x7f5be00ceb90 msgr2=0x7f5be00ac930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:53.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.879+0000 7f5be550f640 1 -- 172.21.15.105:0/2237642042 shutdown_connections 2023-11-28T18:23:53.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.879+0000 7f5be550f640 1 -- 172.21.15.105:0/2237642042 wait complete. 2023-11-28T18:23:53.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.883+0000 7f5a02fe6640 1 -- 172.21.15.105:0/2941538861 >> [v2:172.21.15.105:6808/3491434256,v1:172.21.15.105:6809/3491434256] conn(0x7f59e806ff90 msgr2=0x7f59e80723d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.883+0000 7f5a02fe6640 1 --2- 172.21.15.105:0/2941538861 >> [v2:172.21.15.105:6808/3491434256,v1:172.21.15.105:6809/3491434256] conn(0x7f59e806ff90 0x7f59e80723d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.883+0000 7f5a02fe6640 1 -- 172.21.15.105:0/2941538861 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59e8066950 msgr2=0x7f59e8068e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.883+0000 7f5a02fe6640 1 --2- 172.21.15.105:0/2941538861 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59e8066950 0x7f59e8068e10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f59f8000f70 tx=0x7f59f8001b20 comp rx=0 tx=0).stop 2023-11-28T18:23:53.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.883+0000 7f5a02fe6640 1 -- 172.21.15.105:0/2941538861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59fc0ce4a0 msgr2=0x7f59fc13da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.883+0000 7f5a02fe6640 1 --2- 172.21.15.105:0/2941538861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59fc0ce4a0 0x7f59fc13da90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f59f4007d40 tx=0x7f59f400d4a0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.883+0000 7f5a01583640 1 -- 172.21.15.105:0/2941538861 reap_dead start 2023-11-28T18:23:53.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.883+0000 7f5a02fe6640 1 -- 172.21.15.105:0/2941538861 shutdown_connections 2023-11-28T18:23:53.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.883+0000 7f5a02fe6640 1 -- 172.21.15.105:0/2941538861 >> 172.21.15.105:0/2941538861 conn(0x7f59fc06a750 msgr2=0x7f59fc13f340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:53.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.883+0000 7f5a02fe6640 1 -- 172.21.15.105:0/2941538861 shutdown_connections 2023-11-28T18:23:53.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.883+0000 7f5a02fe6640 1 -- 172.21.15.105:0/2941538861 wait complete. 2023-11-28T18:23:53.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.887+0000 7fe9d48d6640 1 -- 172.21.15.105:0/3473462336 --> [v2:172.21.15.148:6824/3981624325,v1:172.21.15.148:6825/3981624325] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe9b4002cc0 con 0x7fe9b006ff70 2023-11-28T18:23:53.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.891+0000 7f4e68c5f640 1 -- 172.21.15.105:0/3729788372 --> [v2:172.21.15.148:6809/4279905987,v1:172.21.15.148:6811/4279905987] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4e64159440 con 0x7f4e4406ff10 2023-11-28T18:23:53.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.891+0000 7fe9beffd640 1 -- 172.21.15.105:0/3473462336 <== osd.5 v2:172.21.15.148:6824/3981624325 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe9b4002cc0 con 0x7fe9b006ff70 2023-11-28T18:23:53.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.891+0000 7f4e52ffd640 1 -- 172.21.15.105:0/3729788372 <== osd.6 v2:172.21.15.148:6809/4279905987 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4e64159440 con 0x7f4e4406ff10 2023-11-28T18:23:53.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.891+0000 7fe9d48d6640 1 -- 172.21.15.105:0/3473462336 >> [v2:172.21.15.148:6824/3981624325,v1:172.21.15.148:6825/3981624325] conn(0x7fe9b006ff70 msgr2=0x7fe9b00723b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.891+0000 7fe9d48d6640 1 --2- 172.21.15.105:0/3473462336 >> [v2:172.21.15.148:6824/3981624325,v1:172.21.15.148:6825/3981624325] conn(0x7fe9b006ff70 0x7fe9b00723b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.891+0000 7fe9d48d6640 1 -- 172.21.15.105:0/3473462336 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe9b0066930 msgr2=0x7fe9b0068df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.891+0000 7fe9d48d6640 1 --2- 172.21.15.105:0/3473462336 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe9b0066930 0x7fe9b0068df0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00b2000 tx=0x7fe9c0004740 comp rx=0 tx=0).stop 2023-11-28T18:23:53.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.891+0000 7fe9d48d6640 1 -- 172.21.15.105:0/3473462336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9c8047c20 msgr2=0x7fe9c80de380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.891+0000 7fe9d48d6640 1 --2- 172.21.15.105:0/3473462336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9c8047c20 0x7fe9c80de380 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0079a10 tx=0x7fe9d0067f90 comp rx=0 tx=0).stop 2023-11-28T18:23:53.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.891+0000 7fe9ced9d640 1 -- 172.21.15.105:0/3473462336 reap_dead start 2023-11-28T18:23:53.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.891+0000 7fe9d48d6640 1 -- 172.21.15.105:0/3473462336 shutdown_connections 2023-11-28T18:23:53.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.891+0000 7fe9d48d6640 1 -- 172.21.15.105:0/3473462336 >> 172.21.15.105:0/3473462336 conn(0x7fe9c8009fc0 msgr2=0x7fe9c803a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:53.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.891+0000 7f4e68c5f640 1 -- 172.21.15.105:0/3729788372 >> [v2:172.21.15.148:6809/4279905987,v1:172.21.15.148:6811/4279905987] conn(0x7f4e4406ff10 msgr2=0x7f4e44072350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.891+0000 7f4e68c5f640 1 --2- 172.21.15.105:0/3729788372 >> [v2:172.21.15.148:6809/4279905987,v1:172.21.15.148:6811/4279905987] conn(0x7f4e4406ff10 0x7f4e44072350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.891+0000 7f4e68c5f640 1 -- 172.21.15.105:0/3729788372 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4e44066950 msgr2=0x7f4e44068e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.891+0000 7f4e68c5f640 1 --2- 172.21.15.105:0/3729788372 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4e44066950 0x7f4e44068e10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4e540011f0 tx=0x7f4e540078a0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.891+0000 7f4e68c5f640 1 -- 172.21.15.105:0/3729788372 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4e640cdcf0 msgr2=0x7f4e641567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.891+0000 7f4e68c5f640 1 --2- 172.21.15.105:0/3729788372 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4e640cdcf0 0x7f4e641567e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00ccf0 tx=0x7f4e5c00a650 comp rx=0 tx=0).stop 2023-11-28T18:23:53.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.891+0000 7f4e62d9d640 1 -- 172.21.15.105:0/3729788372 reap_dead start 2023-11-28T18:23:53.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.891+0000 7f4e68c5f640 1 -- 172.21.15.105:0/3729788372 shutdown_connections 2023-11-28T18:23:53.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.891+0000 7f4e68c5f640 1 -- 172.21.15.105:0/3729788372 >> 172.21.15.105:0/3729788372 conn(0x7f4e640ceb90 msgr2=0x7f4e640cb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:53.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.895+0000 7fe9d48d6640 1 -- 172.21.15.105:0/3473462336 shutdown_connections 2023-11-28T18:23:53.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.895+0000 7fe9d48d6640 1 -- 172.21.15.105:0/3473462336 wait complete. 2023-11-28T18:23:53.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.895+0000 7f4e68c5f640 1 -- 172.21.15.105:0/3729788372 shutdown_connections 2023-11-28T18:23:53.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.895+0000 7f4e68c5f640 1 -- 172.21.15.105:0/3729788372 wait complete. 2023-11-28T18:23:53.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.899+0000 7fab77501640 1 -- 172.21.15.105:0/1209795070 --> [v2:172.21.15.148:6808/3280605225,v1:172.21.15.148:6810/3280605225] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fab50002cc0 con 0x7fab5406ffe0 2023-11-28T18:23:53.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.899+0000 7fab65ffb640 1 -- 172.21.15.105:0/1209795070 <== osd.7 v2:172.21.15.148:6808/3280605225 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fab50002cc0 con 0x7fab5406ffe0 2023-11-28T18:23:53.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.903+0000 7fab5b7fe640 1 -- 172.21.15.105:0/1209795070 >> [v2:172.21.15.148:6808/3280605225,v1:172.21.15.148:6810/3280605225] conn(0x7fab5406ffe0 msgr2=0x7fab54072420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.903+0000 7fab5b7fe640 1 --2- 172.21.15.105:0/1209795070 >> [v2:172.21.15.148:6808/3280605225,v1:172.21.15.148:6810/3280605225] conn(0x7fab5406ffe0 0x7fab54072420 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.965 INFO:teuthology.orchestra.run.smithi105.stdout:55834574857 2023-11-28T18:23:53.965 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T18:23:53.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.907+0000 7fab5b7fe640 1 -- 172.21.15.105:0/1209795070 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fab540669a0 msgr2=0x7fab54068e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.907+0000 7fab5b7fe640 1 --2- 172.21.15.105:0/1209795070 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fab540669a0 0x7fab54068e60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fab70164920 tx=0x7fab60008040 comp rx=0 tx=0).stop 2023-11-28T18:23:53.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.907+0000 7fab5b7fe640 1 -- 172.21.15.105:0/1209795070 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab7006d610 msgr2=0x7fab7015af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:53.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.907+0000 7fab5b7fe640 1 --2- 172.21.15.105:0/1209795070 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab7006d610 0x7fab7015af70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00e910 tx=0x7fab6c00edd0 comp rx=0 tx=0).stop 2023-11-28T18:23:53.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.907+0000 7fab75a9e640 1 -- 172.21.15.105:0/1209795070 reap_dead start 2023-11-28T18:23:53.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.911+0000 7fab5b7fe640 1 -- 172.21.15.105:0/1209795070 shutdown_connections 2023-11-28T18:23:53.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.911+0000 7fab5b7fe640 1 -- 172.21.15.105:0/1209795070 >> 172.21.15.105:0/1209795070 conn(0x7fab700ceb90 msgr2=0x7fab7015e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:53.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.915+0000 7fab5b7fe640 1 -- 172.21.15.105:0/1209795070 shutdown_connections 2023-11-28T18:23:53.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:53.915+0000 7fab5b7fe640 1 -- 172.21.15.105:0/1209795070 wait complete. 2023-11-28T18:23:53.967 INFO:teuthology.orchestra.run.smithi105.stdout:55834574858 2023-11-28T18:23:53.968 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-28T18:23:53.969 INFO:teuthology.orchestra.run.smithi105.stdout:55834574858 2023-11-28T18:23:53.969 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T18:23:53.970 INFO:teuthology.orchestra.run.smithi105.stdout:55834574857 2023-11-28T18:23:53.970 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-28T18:23:53.971 INFO:teuthology.orchestra.run.smithi105.stdout:55834574858 2023-11-28T18:23:53.971 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-28T18:23:53.972 INFO:teuthology.orchestra.run.smithi105.stdout:55834574857 2023-11-28T18:23:53.972 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-28T18:23:53.973 INFO:teuthology.orchestra.run.smithi105.stdout:55834574857 2023-11-28T18:23:53.973 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-28T18:23:53.974 INFO:teuthology.orchestra.run.smithi105.stdout:55834574858 2023-11-28T18:23:53.974 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-28T18:23:54.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.247+0000 7fa13359e640 1 Processor -- start 2023-11-28T18:23:54.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.247+0000 7fa13359e640 1 -- start start 2023-11-28T18:23:54.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.251+0000 7fa13359e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa1340b0fa0 0x7fa13406bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.251+0000 7fa13359e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1340cba90 0x7fa1340705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.251+0000 7fa132d9d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1340cba90 0x7fa1340705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.251+0000 7fa132d9d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1340cba90 0x7fa1340705f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:37802/0 (socket says 172.21.15.105:37802) 2023-11-28T18:23:54.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.251+0000 7fa13259c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa1340b0fa0 0x7fa13406bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.251+0000 7fa13259c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa1340b0fa0 0x7fa13406bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34206/0 (socket says 172.21.15.105:34206) 2023-11-28T18:23:54.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.251+0000 7fa131d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa1340adf20 0x7fa1340cb380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:47660/0 (socket says 172.21.15.105:47660) 2023-11-28T18:23:54.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.251+0000 7fa132d9d640 1 -- 172.21.15.105:0/1476134151 learned_addr learned my addr 172.21.15.105:0/1476134151 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:54.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.251+0000 7fa13359e640 1 -- 172.21.15.105:0/1476134151 --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa13406fe80 con 0x7fa1340adf20 2023-11-28T18:23:54.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.251+0000 7fa13359e640 1 -- 172.21.15.105:0/1476134151 --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa13406c7c0 con 0x7fa1340b0fa0 2023-11-28T18:23:54.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.251+0000 7fa13359e640 1 -- 172.21.15.105:0/1476134151 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13406caa0 con 0x7fa1340cba90 2023-11-28T18:23:54.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.251+0000 7fa13159a640 1 -- 172.21.15.105:0/1476134151 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 785330106 0 0) 0x7fa13406fe80 con 0x7fa1340adf20 2023-11-28T18:23:54.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.251+0000 7fa13159a640 1 -- 172.21.15.105:0/1476134151 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa11c005740 con 0x7fa1340adf20 2023-11-28T18:23:54.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.255+0000 7fa13159a640 1 -- 172.21.15.105:0/1476134151 <== mon.1 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3246126088 0 0) 0x7fa11c005740 con 0x7fa1340adf20 2023-11-28T18:23:54.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.255+0000 7fa132d9d640 1 -- 172.21.15.105:0/1476134151 >> v2:172.21.15.148:3300/0 conn(0x7fa1340b0fa0 msgr2=0x7fa13406bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:54.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.255+0000 7fa132d9d640 1 --2- 172.21.15.105:0/1476134151 >> v2:172.21.15.148:3300/0 conn(0x7fa1340b0fa0 0x7fa13406bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.255+0000 7fa132d9d640 1 -- 172.21.15.105:0/1476134151 >> v1:172.21.15.148:6789/0 conn(0x7fa1340adf20 legacy=0x7fa1340cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.255+0000 7fa132d9d640 1 -- 172.21.15.105:0/1476134151 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13406cd80 con 0x7fa1340cba90 2023-11-28T18:23:54.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.255+0000 7fa13259c640 1 --2- 172.21.15.105:0/1476134151 >> v2:172.21.15.148:3300/0 conn(0x7fa1340b0fa0 0x7fa13406bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:54.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.255+0000 7f1899d1e640 1 Processor -- start 2023-11-28T18:23:54.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.255+0000 7f1899d1e640 1 -- start start 2023-11-28T18:23:54.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.255+0000 7f1899d1e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1894161f30 0x7f18940ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.255+0000 7f1899d1e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f18940ccc50 0x7f189406bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.255+0000 7f1899d1e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18940cde90 con 0x7f18940c1950 2023-11-28T18:23:54.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.255+0000 7f1899d1e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1894162860 con 0x7f1894161f30 2023-11-28T18:23:54.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.255+0000 7f1899d1e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18940ce780 con 0x7f18940ccc50 2023-11-28T18:23:54.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.255+0000 7f189951d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f18940ccc50 0x7f189406bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.255+0000 7f189951d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f18940ccc50 0x7f189406bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34212/0 (socket says 172.21.15.105:34212) 2023-11-28T18:23:54.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.255+0000 7f189951d640 1 -- 172.21.15.105:0/72748266 learned_addr learned my addr 172.21.15.105:0/72748266 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:54.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.255+0000 7f189951d640 1 -- 172.21.15.105:0/72748266 >> v2:172.21.15.105:3300/0 conn(0x7f1894161f30 msgr2=0x7f18940ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:54.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.255+0000 7f189951d640 1 --2- 172.21.15.105:0/72748266 >> v2:172.21.15.105:3300/0 conn(0x7f1894161f30 0x7f18940ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.255+0000 7f189951d640 1 -- 172.21.15.105:0/72748266 >> v1:172.21.15.105:6789/0 conn(0x7f18940c1950 legacy=0x7f18940cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.255+0000 7f189951d640 1 -- 172.21.15.105:0/72748266 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18940c90f0 con 0x7f18940ccc50 2023-11-28T18:23:54.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.259+0000 7fa132d9d640 1 --2- 172.21.15.105:0/1476134151 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1340cba90 0x7fa1340705f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009e50 tx=0x7fa12c01c870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f990d1513efd0830 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.263+0000 7fa13159a640 1 -- 172.21.15.105:0/1476134151 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa128004fe0 con 0x7fa1340cba90 2023-11-28T18:23:54.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.263+0000 7fa13159a640 1 -- 172.21.15.105:0/1476134151 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12c0090d0 con 0x7fa1340cba90 2023-11-28T18:23:54.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.263+0000 7fa13159a640 1 -- 172.21.15.105:0/1476134151 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c0a57c0 con 0x7fa1340cba90 2023-11-28T18:23:54.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.263+0000 7fa13359e640 1 -- 172.21.15.105:0/1476134151 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1340cba90 msgr2=0x7fa1340705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.263+0000 7fa13359e640 1 --2- 172.21.15.105:0/1476134151 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1340cba90 0x7fa1340705f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009e50 tx=0x7fa12c01c870 comp rx=0 tx=0).stop 2023-11-28T18:23:54.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.263+0000 7fa13359e640 1 -- 172.21.15.105:0/1476134151 shutdown_connections 2023-11-28T18:23:54.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.263+0000 7fa13359e640 1 --2- 172.21.15.105:0/1476134151 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1340cba90 0x7fa1340705f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.263+0000 7fa13359e640 1 --2- 172.21.15.105:0/1476134151 >> v2:172.21.15.148:3300/0 conn(0x7fa1340b0fa0 0x7fa13406bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.263+0000 7fa13359e640 1 -- 172.21.15.105:0/1476134151 >> 172.21.15.105:0/1476134151 conn(0x7fa1340b0480 msgr2=0x7fa1340b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:54.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.263+0000 7f189951d640 1 --2- 172.21.15.105:0/72748266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18940ccc50 0x7f189406bee0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f189406d150 tx=0x7f188c01c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a7eb8ce81358135 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.263+0000 7f188b7fe640 1 -- 172.21.15.105:0/72748266 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c0a57f0 con 0x7f18940ccc50 2023-11-28T18:23:54.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.263+0000 7f188b7fe640 1 -- 172.21.15.105:0/72748266 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c0a5950 con 0x7f18940ccc50 2023-11-28T18:23:54.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.263+0000 7f188b7fe640 1 -- 172.21.15.105:0/72748266 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c0a5ce0 con 0x7f18940ccc50 2023-11-28T18:23:54.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.263+0000 7f1899d1e640 1 -- 172.21.15.105:0/72748266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18940ccc50 msgr2=0x7f189406bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.263+0000 7f1899d1e640 1 --2- 172.21.15.105:0/72748266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18940ccc50 0x7f189406bee0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f189406d150 tx=0x7f188c01c910 comp rx=0 tx=0).stop 2023-11-28T18:23:54.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.263+0000 7f1899d1e640 1 -- 172.21.15.105:0/72748266 shutdown_connections 2023-11-28T18:23:54.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.263+0000 7f1899d1e640 1 --2- 172.21.15.105:0/72748266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18940ccc50 0x7f189406bee0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.263+0000 7f1899d1e640 1 --2- 172.21.15.105:0/72748266 >> v2:172.21.15.105:3300/0 conn(0x7f1894161f30 0x7f18940ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.263+0000 7f1899d1e640 1 -- 172.21.15.105:0/72748266 >> 172.21.15.105:0/72748266 conn(0x7f18940b0480 msgr2=0x7f18940b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:54.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.263+0000 7f1899d1e640 1 -- 172.21.15.105:0/72748266 shutdown_connections 2023-11-28T18:23:54.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.263+0000 7f1899d1e640 1 -- 172.21.15.105:0/72748266 wait complete. 2023-11-28T18:23:54.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.267+0000 7fa13359e640 1 -- 172.21.15.105:0/1476134151 shutdown_connections 2023-11-28T18:23:54.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.267+0000 7fa13359e640 1 -- 172.21.15.105:0/1476134151 wait complete. 2023-11-28T18:23:54.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.267+0000 7f1899d1e640 1 Processor -- start 2023-11-28T18:23:54.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.267+0000 7f1899d1e640 1 -- start start 2023-11-28T18:23:54.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.267+0000 7f1899d1e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18940c1950 0x7f18941563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.267+0000 7fa13359e640 1 Processor -- start 2023-11-28T18:23:54.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.267+0000 7f1899d1e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18940ccc50 0x7f18940c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.267+0000 7f1899d1e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1894161f30 0x7f18940c2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.267+0000 7f1899d1e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18940c6680 con 0x7f18940ccc50 2023-11-28T18:23:54.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.267+0000 7f1899d1e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18940c5ae0 con 0x7f1894161f30 2023-11-28T18:23:54.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.267+0000 7f1899d1e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18940c5d50 con 0x7f18940c1950 2023-11-28T18:23:54.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f9293cfe640 1 Processor -- start 2023-11-28T18:23:54.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.267+0000 7f188bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18940ccc50 0x7f18940c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f189951d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1894161f30 0x7f18940c2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f189951d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1894161f30 0x7f18940c2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34224/0 (socket says 172.21.15.105:34224) 2023-11-28T18:23:54.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f189951d640 1 -- 172.21.15.105:0/3036556406 learned_addr learned my addr 172.21.15.105:0/3036556406 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:54.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f1898d1c640 1 --2- 172.21.15.105:0/3036556406 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18940c1950 0x7f18941563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f189951d640 1 -- 172.21.15.105:0/3036556406 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18940c1950 msgr2=0x7f18941563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f189951d640 1 --2- 172.21.15.105:0/3036556406 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18940c1950 0x7f18941563a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f189951d640 1 -- 172.21.15.105:0/3036556406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18940ccc50 msgr2=0x7f18940c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f189951d640 1 --2- 172.21.15.105:0/3036556406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18940ccc50 0x7f18940c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f189951d640 1 -- 172.21.15.105:0/3036556406 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c009b30 con 0x7f1894161f30 2023-11-28T18:23:54.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f9293cfe640 1 -- start start 2023-11-28T18:23:54.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.267+0000 7fa13359e640 1 -- start start 2023-11-28T18:23:54.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f189951d640 1 --2- 172.21.15.105:0/3036556406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1894161f30 0x7f18940c2d30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f188c019520 tx=0x7f188c0196b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7fa13359e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1340adf20 0x7fa1340c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7fa13359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1340b0fa0 0x7fa1340ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7fa13359e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1340cba90 0x7fa1340ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7fa13359e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1340c4480 con 0x7fa1340b0fa0 2023-11-28T18:23:54.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f18897fa640 1 -- 172.21.15.105:0/3036556406 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c0b8670 con 0x7f1894161f30 2023-11-28T18:23:54.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7fa13359e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1340c2c10 con 0x7fa1340adf20 2023-11-28T18:23:54.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7fa13259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1340adf20 0x7fa1340c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7fa13359e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1340c7100 con 0x7fa1340cba90 2023-11-28T18:23:54.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7fa13259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1340adf20 0x7fa1340c6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34232/0 (socket says 172.21.15.105:34232) 2023-11-28T18:23:54.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7fa13259c640 1 -- 172.21.15.105:0/4100023186 learned_addr learned my addr 172.21.15.105:0/4100023186 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:54.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7fa131d9b640 1 --2- 172.21.15.105:0/4100023186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1340b0fa0 0x7fa1340ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7fa6749f8640 1 Processor -- start 2023-11-28T18:23:54.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f18897fa640 1 -- 172.21.15.105:0/3036556406 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c0b87d0 con 0x7f1894161f30 2023-11-28T18:23:54.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f18897fa640 1 -- 172.21.15.105:0/3036556406 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c0b8b60 con 0x7f1894161f30 2023-11-28T18:23:54.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f1899d1e640 1 -- 172.21.15.105:0/3036556406 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18940c6030 con 0x7f1894161f30 2023-11-28T18:23:54.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f1899d1e640 1 -- 172.21.15.105:0/3036556406 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18941568e0 con 0x7f1894161f30 2023-11-28T18:23:54.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7fa132d9d640 1 --2- 172.21.15.105:0/4100023186 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1340cba90 0x7fa1340ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7fa131d9b640 1 -- 172.21.15.105:0/4100023186 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1340cba90 msgr2=0x7fa1340ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7fa131d9b640 1 --2- 172.21.15.105:0/4100023186 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1340cba90 0x7fa1340ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7fa131d9b640 1 -- 172.21.15.105:0/4100023186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1340adf20 msgr2=0x7fa1340c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7fa131d9b640 1 --2- 172.21.15.105:0/4100023186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1340adf20 0x7fa1340c6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7fa131d9b640 1 -- 172.21.15.105:0/4100023186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12400a4d0 con 0x7fa1340b0fa0 2023-11-28T18:23:54.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fa131d9b640 1 --2- 172.21.15.105:0/4100023186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1340b0fa0 0x7fa1340ac8d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa12800c800 tx=0x7fa12800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7f1899d1e640 1 -- 172.21.15.105:0/3036556406 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18940c6830 con 0x7f1894161f30 2023-11-28T18:23:54.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fa122ffd640 1 -- 172.21.15.105:0/4100023186 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa128007270 con 0x7fa1340b0fa0 2023-11-28T18:23:54.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fa122ffd640 1 -- 172.21.15.105:0/4100023186 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1280073f0 con 0x7fa1340b0fa0 2023-11-28T18:23:54.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fa122ffd640 1 -- 172.21.15.105:0/4100023186 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa128005820 con 0x7fa1340b0fa0 2023-11-28T18:23:54.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7fa6749f8640 1 -- start start 2023-11-28T18:23:54.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fa6749f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa6700d10e0 0x7fa6700d3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fa6749f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6700b4140 0x7fa6700d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fa6749f8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa670160530 0x7fa670160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fa6749f8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa670160ec0 con 0x7fa670160530 2023-11-28T18:23:54.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fa6749f8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6700d1d20 con 0x7fa6700d10e0 2023-11-28T18:23:54.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fa6749f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700d01d0 con 0x7fa6700b4140 2023-11-28T18:23:54.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7fae0caea640 1 Processor -- start 2023-11-28T18:23:54.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7fae0caea640 1 -- start start 2023-11-28T18:23:54.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fae0caea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae080abd80 0x7fae080cb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fae0caea640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fae080cba90 0x7fae080cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fae0caea640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae08070520 con 0x7fae08161520 2023-11-28T18:23:54.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fae0caea640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae080cbf20 con 0x7fae080cba90 2023-11-28T18:23:54.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fae0caea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae08161900 con 0x7fae080abd80 2023-11-28T18:23:54.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f9293cfe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f928c0ad130 0x7f928c161780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f9293cfe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f928c0cbcb0 0x7f928c06dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f9293cfe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f928c160cc0 con 0x7f928c0ad510 2023-11-28T18:23:54.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f9293cfe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f928c161cc0 con 0x7f928c0ad130 2023-11-28T18:23:54.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f9293cfe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f928c06e020 con 0x7f928c0cbcb0 2023-11-28T18:23:54.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f929229b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f928c0cbcb0 0x7f928c06dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f929229b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f928c0cbcb0 0x7f928c06dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34246/0 (socket says 172.21.15.105:34246) 2023-11-28T18:23:54.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.271+0000 7f929229b640 1 -- 172.21.15.105:0/1463538509 learned_addr learned my addr 172.21.15.105:0/1463538509 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:54.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fa66ed9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa670160530 0x7fa670160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7f929229b640 1 -- 172.21.15.105:0/1463538509 >> v2:172.21.15.105:3300/0 conn(0x7f928c0ad130 msgr2=0x7f928c161780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:54.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7f929229b640 1 --2- 172.21.15.105:0/1463538509 >> v2:172.21.15.105:3300/0 conn(0x7f928c0ad130 0x7f928c161780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7f929229b640 1 -- 172.21.15.105:0/1463538509 >> v1:172.21.15.148:6789/0 conn(0x7f928c0ad510 legacy=0x7f928c0cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:54.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7f929229b640 1 -- 172.21.15.105:0/1463538509 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928c0ce5b0 con 0x7f928c0cbcb0 2023-11-28T18:23:54.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fa66ed9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa670160530 0x7fa670160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55092/0 (socket says 172.21.15.105:55092) 2023-11-28T18:23:54.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fa66ed9d640 1 -- 172.21.15.105:0/1145906792 learned_addr learned my addr 172.21.15.105:0/1145906792 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:54.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fae0659c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fae08161520 0x7fae080ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39262/0 (socket says 172.21.15.105:39262) 2023-11-28T18:23:54.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fa13359e640 1 -- 172.21.15.105:0/4100023186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa12c009b30 con 0x7fa1340b0fa0 2023-11-28T18:23:54.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fa13359e640 1 -- 172.21.15.105:0/4100023186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa134147040 con 0x7fa1340b0fa0 2023-11-28T18:23:54.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7f18897fa640 1 -- 172.21.15.105:0/3036556406 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f188c0b8cc0 con 0x7f1894161f30 2023-11-28T18:23:54.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7f18897fa640 1 --2- 172.21.15.105:0/3036556406 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f18800669a0 0x7f1880068e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7f18897fa640 1 -- 172.21.15.105:0/3036556406 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f188c11fd50 con 0x7f1894161f30 2023-11-28T18:23:54.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7f1898d1c640 1 --2- 172.21.15.105:0/3036556406 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f18800669a0 0x7f1880068e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7f3bad3f5640 1 Processor -- start 2023-11-28T18:23:54.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7f3bad3f5640 1 -- start start 2023-11-28T18:23:54.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7efcc1c5a640 1 Processor -- start 2023-11-28T18:23:54.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7efcc1c5a640 1 -- start start 2023-11-28T18:23:54.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fae06d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fae080cba90 0x7fae080cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fae0659c640 1 -- 172.21.15.105:0/3973771223 learned_addr learned my addr 172.21.15.105:0/3973771223 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:54.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fae05d9b640 1 --2- 172.21.15.105:0/3973771223 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae080abd80 0x7fae080cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fae05d9b640 1 -- 172.21.15.105:0/3973771223 >> v2:172.21.15.105:3300/0 conn(0x7fae080cba90 msgr2=0x7fae080cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:54.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fae05d9b640 1 --2- 172.21.15.105:0/3973771223 >> v2:172.21.15.105:3300/0 conn(0x7fae080cba90 0x7fae080cc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fae05d9b640 1 -- 172.21.15.105:0/3973771223 >> v1:172.21.15.105:6789/0 conn(0x7fae08161520 legacy=0x7fae080ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:54.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fae05d9b640 1 -- 172.21.15.105:0/3973771223 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae080ce9a0 con 0x7fae080abd80 2023-11-28T18:23:54.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fa66ed9d640 1 -- 172.21.15.105:0/1145906792 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6700b4140 msgr2=0x7fa6700d14c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T18:23:54.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fa66ed9d640 1 --2- 172.21.15.105:0/1145906792 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6700b4140 0x7fa6700d14c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fa66ed9d640 1 -- 172.21.15.105:0/1145906792 >> v2:172.21.15.148:3300/0 conn(0x7fa6700d10e0 msgr2=0x7fa6700d3bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T18:23:54.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7f3bad3f5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3ba80ad130 0x7f3ba8161780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7f3bad3f5640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ba80ad510 0x7f3ba806dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7fa13359e640 1 -- 172.21.15.105:0/4100023186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa134074580 con 0x7fa1340b0fa0 2023-11-28T18:23:54.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7efcc1c5a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efcbc0d3c30 0x7efcbc0d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7efcc1c5a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcbc0b25b0 con 0x7efcbc15d4f0 2023-11-28T18:23:54.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7efcc1c5a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcbc0cea30 con 0x7efcbc0b2d20 2023-11-28T18:23:54.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7efcc1c5a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcbc0b2900 con 0x7efcbc0d3c30 2023-11-28T18:23:54.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7efcbbfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efcbc0b2d20 0x7efcbc0b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:47684/0 (socket says 172.21.15.105:47684) 2023-11-28T18:23:54.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7efcbbfff640 1 -- 172.21.15.105:0/1064249919 learned_addr learned my addr 172.21.15.105:0/1064249919 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:54.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7f3ba67fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ba80ad510 0x7f3ba806dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7f3ba67fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ba80ad510 0x7f3ba806dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:37860/0 (socket says 172.21.15.105:37860) 2023-11-28T18:23:54.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7f3ba67fc640 1 -- 172.21.15.105:0/315971033 learned_addr learned my addr 172.21.15.105:0/315971033 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:54.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7f1898d1c640 1 --2- 172.21.15.105:0/3036556406 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f18800669a0 0x7f1880068e60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f189000a790 tx=0x7f1890008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7f3bad3f5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba8160cc0 con 0x7f3ba80cb5a0 2023-11-28T18:23:54.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7f3bad3f5640 1 -- 172.21.15.105:0/315971033 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba8161cc0 con 0x7f3ba80ad130 2023-11-28T18:23:54.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7f3bad3f5640 1 -- 172.21.15.105:0/315971033 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba806e020 con 0x7f3ba80ad510 2023-11-28T18:23:54.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7f3ba6ffd640 1 --2- 172.21.15.105:0/315971033 >> v2:172.21.15.105:3300/0 conn(0x7f3ba80ad130 0x7f3ba8161780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fa66ed9d640 1 --2- 172.21.15.105:0/1145906792 >> v2:172.21.15.148:3300/0 conn(0x7fa6700d10e0 0x7fa6700d3bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.275+0000 7fa66ed9d640 1 -- 172.21.15.105:0/1145906792 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6700d04b0 con 0x7fa670160530 2023-11-28T18:23:54.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7f929229b640 1 --2- 172.21.15.105:0/1463538509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928c0cbcb0 0x7f928c06dae0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f928c071020 tx=0x7f927c01c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f0a47a39ba2b76f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7f3ba6ffd640 1 -- 172.21.15.105:0/315971033 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ba80ad510 msgr2=0x7f3ba806dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:54.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7f3ba6ffd640 1 --2- 172.21.15.105:0/315971033 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ba80ad510 0x7f3ba806dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7f3ba6ffd640 1 -- 172.21.15.105:0/315971033 >> v1:172.21.15.148:6789/0 conn(0x7f3ba80cb5a0 legacy=0x7f3ba80cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7f3ba6ffd640 1 -- 172.21.15.105:0/315971033 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba80ce5b0 con 0x7f3ba80ad130 2023-11-28T18:23:54.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7f3ba67fc640 1 --2- 172.21.15.105:0/315971033 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ba80ad510 0x7f3ba806dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:54.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7fa122ffd640 1 -- 172.21.15.105:0/4100023186 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa128005980 con 0x7fa1340b0fa0 2023-11-28T18:23:54.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7f78fb59e640 1 Processor -- start 2023-11-28T18:23:54.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7efcbaffd640 1 --2- 172.21.15.105:0/1064249919 >> v2:172.21.15.148:3300/0 conn(0x7efcbc0d3c30 0x7efcbc0d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7fa66ed9d640 1 --2- 172.21.15.105:0/1145906792 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa670160530 0x7fa670160980 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa67008d810 tx=0x7fa66801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f2f8164131a169 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.279+0000 7f78fb59e640 1 -- start start 2023-11-28T18:23:54.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7efcbaffd640 1 -- 172.21.15.105:0/1064249919 >> v1:172.21.15.148:6789/0 conn(0x7efcbc0b2d20 legacy=0x7efcbc0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7efcbaffd640 1 -- 172.21.15.105:0/1064249919 >> v1:172.21.15.105:6789/0 conn(0x7efcbc15d4f0 legacy=0x7efcbc0d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7efcbaffd640 1 -- 172.21.15.105:0/1064249919 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc0ced10 con 0x7efcbc0d3c30 2023-11-28T18:23:54.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7fa122ffd640 1 --2- 172.21.15.105:0/4100023186 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1140669a0 0x7fa114068e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7f9290a98640 1 -- 172.21.15.105:0/1463538509 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c0ab600 con 0x7f928c0cbcb0 2023-11-28T18:23:54.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7f9290a98640 1 -- 172.21.15.105:0/1463538509 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c0ab760 con 0x7f928c0cbcb0 2023-11-28T18:23:54.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7f9290a98640 1 -- 172.21.15.105:0/1463538509 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c0abaf0 con 0x7f928c0cbcb0 2023-11-28T18:23:54.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7fae05d9b640 1 --2- 172.21.15.105:0/3973771223 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae080abd80 0x7fae080cb550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fadf80097c0 tx=0x7fadf801c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e675bdf45e40830f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7fa122ffd640 1 -- 172.21.15.105:0/4100023186 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa128084340 con 0x7fa1340b0fa0 2023-11-28T18:23:54.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7fae0559a640 1 -- 172.21.15.105:0/3973771223 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf80ab400 con 0x7fae080abd80 2023-11-28T18:23:54.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7fae0559a640 1 -- 172.21.15.105:0/3973771223 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf80ab560 con 0x7fae080abd80 2023-11-28T18:23:54.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7fa13259c640 1 --2- 172.21.15.105:0/4100023186 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1140669a0 0x7fa114068e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7f78fb59e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78fc06bee0 0x7f78fc06fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7f78fb59e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f78fc0ccf20 0x7f78fc0ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7f78fb59e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78fc0ce7e0 con 0x7f78fc06cb30 2023-11-28T18:23:54.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7f78fb59e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78fc0cd450 con 0x7f78fc0ccf20 2023-11-28T18:23:54.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7f78fb59e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0ca8e0 con 0x7f78fc06bee0 2023-11-28T18:23:54.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7fae0559a640 1 -- 172.21.15.105:0/3973771223 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf80ab400 con 0x7fae080abd80 2023-11-28T18:23:54.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7f78f3fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f78fc06cb30 0x7f78fc06c2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:47694/0 (socket says 172.21.15.105:47694) 2023-11-28T18:23:54.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7f78f3fff640 1 -- 172.21.15.105:0/4191613769 learned_addr learned my addr 172.21.15.105:0/4191613769 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:54.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7f78fad9d640 1 --2- 172.21.15.105:0/4191613769 >> v2:172.21.15.148:3300/0 conn(0x7f78fc0ccf20 0x7f78fc0ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7fae0caea640 1 -- 172.21.15.105:0/3973771223 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae080abd80 msgr2=0x7fae080cb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7fae0caea640 1 --2- 172.21.15.105:0/3973771223 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae080abd80 0x7fae080cb550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fadf80097c0 tx=0x7fadf801c440 comp rx=0 tx=0).stop 2023-11-28T18:23:54.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7f18897fa640 1 -- 172.21.15.105:0/3036556406 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f18940c6830 con 0x7f1894161f30 2023-11-28T18:23:54.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7f78f9d9b640 1 -- 172.21.15.105:0/4191613769 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4021164014 0 0) 0x7f78fc0ce7e0 con 0x7f78fc06cb30 2023-11-28T18:23:54.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7f78f9d9b640 1 -- 172.21.15.105:0/4191613769 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78e8003530 con 0x7f78fc06cb30 2023-11-28T18:23:54.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7f78fad9d640 1 -- 172.21.15.105:0/4191613769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78fc06bee0 msgr2=0x7f78fc06fe80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T18:23:54.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7f78fad9d640 1 --2- 172.21.15.105:0/4191613769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78fc06bee0 0x7f78fc06fe80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7f78fad9d640 1 -- 172.21.15.105:0/4191613769 >> v1:172.21.15.148:6789/0 conn(0x7f78fc06cb30 legacy=0x7f78fc06c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7fa13259c640 1 --2- 172.21.15.105:0/4100023186 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1140669a0 0x7fa114068e60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa12400af30 tx=0x7fa12400a7f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7f78fad9d640 1 -- 172.21.15.105:0/4191613769 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc0ce220 con 0x7f78fc0ccf20 2023-11-28T18:23:54.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.283+0000 7f3ba6ffd640 1 --2- 172.21.15.105:0/315971033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba80ad130 0x7f3ba8161780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3b980097c0 tx=0x7f3b9801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a902b2ae3c396ad9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7fae0caea640 1 -- 172.21.15.105:0/3973771223 shutdown_connections 2023-11-28T18:23:54.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7fae0caea640 1 --2- 172.21.15.105:0/3973771223 >> v2:172.21.15.105:3300/0 conn(0x7fae080cba90 0x7fae080cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7fae0caea640 1 --2- 172.21.15.105:0/3973771223 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae080abd80 0x7fae080cb550 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7fae0caea640 1 -- 172.21.15.105:0/3973771223 >> 172.21.15.105:0/3973771223 conn(0x7fae080b08c0 msgr2=0x7fae080b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:54.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7fae0caea640 1 -- 172.21.15.105:0/3973771223 shutdown_connections 2023-11-28T18:23:54.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7fae0caea640 1 -- 172.21.15.105:0/3973771223 wait complete. 2023-11-28T18:23:54.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7f9293cfe640 1 -- 172.21.15.105:0/1463538509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928c0cbcb0 msgr2=0x7f928c06dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7f9293cfe640 1 --2- 172.21.15.105:0/1463538509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928c0cbcb0 0x7f928c06dae0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f928c071020 tx=0x7f927c01c420 comp rx=0 tx=0).stop 2023-11-28T18:23:54.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7f9293cfe640 1 -- 172.21.15.105:0/1463538509 shutdown_connections 2023-11-28T18:23:54.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7f9293cfe640 1 --2- 172.21.15.105:0/1463538509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928c0cbcb0 0x7f928c06dae0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7f9293cfe640 1 --2- 172.21.15.105:0/1463538509 >> v2:172.21.15.105:3300/0 conn(0x7f928c0ad130 0x7f928c161780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7f9293cfe640 1 -- 172.21.15.105:0/1463538509 >> 172.21.15.105:0/1463538509 conn(0x7f928c0b08c0 msgr2=0x7f928c0b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:54.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7f9293cfe640 1 -- 172.21.15.105:0/1463538509 shutdown_connections 2023-11-28T18:23:54.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7f9293cfe640 1 -- 172.21.15.105:0/1463538509 wait complete. 2023-11-28T18:23:54.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7efcbaffd640 1 --2- 172.21.15.105:0/1064249919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcbc0d3c30 0x7efcbc0d1520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efcac009d90 tx=0x7efcac01c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c38cf181a678050 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7efcba7fc640 1 -- 172.21.15.105:0/1064249919 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac0a58f0 con 0x7efcbc0d3c30 2023-11-28T18:23:54.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7efcba7fc640 1 -- 172.21.15.105:0/1064249919 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcac0a5a50 con 0x7efcbc0d3c30 2023-11-28T18:23:54.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7efcba7fc640 1 -- 172.21.15.105:0/1064249919 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac0a5de0 con 0x7efcbc0d3c30 2023-11-28T18:23:54.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7efcc1c5a640 1 -- 172.21.15.105:0/1064249919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcbc0d3c30 msgr2=0x7efcbc0d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7efcc1c5a640 1 --2- 172.21.15.105:0/1064249919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcbc0d3c30 0x7efcbc0d1520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efcac009d90 tx=0x7efcac01c8d0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7efcc1c5a640 1 -- 172.21.15.105:0/1064249919 shutdown_connections 2023-11-28T18:23:54.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7efcc1c5a640 1 --2- 172.21.15.105:0/1064249919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcbc0d3c30 0x7efcbc0d1520 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7efcc1c5a640 1 -- 172.21.15.105:0/1064249919 >> 172.21.15.105:0/1064249919 conn(0x7efcbc0d3030 msgr2=0x7efcbc0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:54.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7efcc1c5a640 1 -- 172.21.15.105:0/1064249919 shutdown_connections 2023-11-28T18:23:54.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7efcc1c5a640 1 -- 172.21.15.105:0/1064249919 wait complete. 2023-11-28T18:23:54.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7fae0caea640 1 Processor -- start 2023-11-28T18:23:54.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7fae0caea640 1 -- start start 2023-11-28T18:23:54.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fa122ffd640 1 -- 172.21.15.105:0/4100023186 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa128051330 con 0x7fa1340b0fa0 2023-11-28T18:23:54.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f3ba5ffb640 1 -- 172.21.15.105:0/315971033 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b980ab650 con 0x7f3ba80ad130 2023-11-28T18:23:54.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f3ba5ffb640 1 -- 172.21.15.105:0/315971033 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b980ab7b0 con 0x7f3ba80ad130 2023-11-28T18:23:54.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f3ba5ffb640 1 -- 172.21.15.105:0/315971033 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b980abb40 con 0x7f3ba80ad130 2023-11-28T18:23:54.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f3bad3f5640 1 -- 172.21.15.105:0/315971033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba80ad130 msgr2=0x7f3ba8161780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f3bad3f5640 1 --2- 172.21.15.105:0/315971033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba80ad130 0x7f3ba8161780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3b980097c0 tx=0x7f3b9801c450 comp rx=0 tx=0).stop 2023-11-28T18:23:54.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.287+0000 7f9293cfe640 1 Processor -- start 2023-11-28T18:23:54.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f9293cfe640 1 -- start start 2023-11-28T18:23:54.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fa66d59a640 1 -- 172.21.15.105:0/1145906792 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6680a56b0 con 0x7fa670160530 2023-11-28T18:23:54.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fa66d59a640 1 -- 172.21.15.105:0/1145906792 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6680a5810 con 0x7fa670160530 2023-11-28T18:23:54.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fa66d59a640 1 -- 172.21.15.105:0/1145906792 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6680a5ba0 con 0x7fa670160530 2023-11-28T18:23:54.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f78fad9d640 1 --2- 172.21.15.105:0/4191613769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78fc0ccf20 0x7f78fc0ad510 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009d90 tx=0x7f78f401c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdbb29669e4d832d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f78f9d9b640 1 -- 172.21.15.105:0/4191613769 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f40a5970 con 0x7f78fc0ccf20 2023-11-28T18:23:54.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f78f9d9b640 1 -- 172.21.15.105:0/4191613769 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f40a5ad0 con 0x7f78fc0ccf20 2023-11-28T18:23:54.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f78f9d9b640 1 -- 172.21.15.105:0/4191613769 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f40a5e60 con 0x7f78fc0ccf20 2023-11-28T18:23:54.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f3bad3f5640 1 -- 172.21.15.105:0/315971033 shutdown_connections 2023-11-28T18:23:54.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f3bad3f5640 1 --2- 172.21.15.105:0/315971033 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ba80ad510 0x7f3ba806dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f3bad3f5640 1 --2- 172.21.15.105:0/315971033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba80ad130 0x7f3ba8161780 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f3bad3f5640 1 -- 172.21.15.105:0/315971033 >> 172.21.15.105:0/315971033 conn(0x7f3ba80b08c0 msgr2=0x7f3ba80b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:54.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fae0caea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae080abd80 0x7fae08158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fae0caea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae080cba90 0x7fae0806bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fae0caea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae08161520 0x7fae0806c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fae0caea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae08158940 con 0x7fae080cba90 2023-11-28T18:23:54.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fae0caea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae08161b60 con 0x7fae080abd80 2023-11-28T18:23:54.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fae0caea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae08161e00 con 0x7fae08161520 2023-11-28T18:23:54.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fae0659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae080abd80 0x7fae08158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fae0659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae080abd80 0x7fae08158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34270/0 (socket says 172.21.15.105:34270) 2023-11-28T18:23:54.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fae0659c640 1 -- 172.21.15.105:0/2701565927 learned_addr learned my addr 172.21.15.105:0/2701565927 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:54.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f9293cfe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f928c0ad130 0x7f928c159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f9293cfe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928c0ad510 0x7f928c15e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f9293cfe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f928c0cbcb0 0x7f928c0ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f9293cfe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c099b60 con 0x7f928c0ad130 2023-11-28T18:23:54.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f9293cfe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c1599d0 con 0x7f928c0ad510 2023-11-28T18:23:54.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f9293cfe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f928c15e6e0 con 0x7f928c0cbcb0 2023-11-28T18:23:54.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7efcc1c5a640 1 Processor -- start 2023-11-28T18:23:54.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fa6749f8640 1 -- 172.21.15.105:0/1145906792 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa670160530 msgr2=0x7fa670160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fa6749f8640 1 --2- 172.21.15.105:0/1145906792 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa670160530 0x7fa670160980 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa67008d810 tx=0x7fa66801c950 comp rx=0 tx=0).stop 2023-11-28T18:23:54.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f78fb59e640 1 -- 172.21.15.105:0/4191613769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78fc0ccf20 msgr2=0x7f78fc0ad510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f78fb59e640 1 --2- 172.21.15.105:0/4191613769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78fc0ccf20 0x7f78fc0ad510 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009d90 tx=0x7f78f401c8b0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7efcc1c5a640 1 -- start start 2023-11-28T18:23:54.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f3bad3f5640 1 -- 172.21.15.105:0/315971033 shutdown_connections 2023-11-28T18:23:54.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f3bad3f5640 1 -- 172.21.15.105:0/315971033 wait complete. 2023-11-28T18:23:54.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fae0659c640 1 -- 172.21.15.105:0/2701565927 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae08161520 msgr2=0x7fae0806c490 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T18:23:54.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fae0659c640 1 --2- 172.21.15.105:0/2701565927 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae08161520 0x7fae0806c490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fae0659c640 1 -- 172.21.15.105:0/2701565927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae080cba90 msgr2=0x7fae0806bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fae0659c640 1 --2- 172.21.15.105:0/2701565927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae080cba90 0x7fae0806bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fae0659c640 1 -- 172.21.15.105:0/2701565927 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf8009420 con 0x7fae080abd80 2023-11-28T18:23:54.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fa6749f8640 1 -- 172.21.15.105:0/1145906792 shutdown_connections 2023-11-28T18:23:54.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fa6749f8640 1 --2- 172.21.15.105:0/1145906792 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa670160530 0x7fa670160980 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fa6749f8640 1 --2- 172.21.15.105:0/1145906792 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6700b4140 0x7fa6700d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fa6749f8640 1 --2- 172.21.15.105:0/1145906792 >> v2:172.21.15.148:3300/0 conn(0x7fa6700d10e0 0x7fa6700d3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fa6749f8640 1 -- 172.21.15.105:0/1145906792 >> 172.21.15.105:0/1145906792 conn(0x7fa6700d30a0 msgr2=0x7fa6700d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:54.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fa6749f8640 1 -- 172.21.15.105:0/1145906792 shutdown_connections 2023-11-28T18:23:54.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f78fb59e640 1 -- 172.21.15.105:0/4191613769 shutdown_connections 2023-11-28T18:23:54.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f78fb59e640 1 --2- 172.21.15.105:0/4191613769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78fc0ccf20 0x7f78fc0ad510 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f78fb59e640 1 --2- 172.21.15.105:0/4191613769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78fc06bee0 0x7f78fc06fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f78fb59e640 1 -- 172.21.15.105:0/4191613769 >> 172.21.15.105:0/4191613769 conn(0x7f78fc0b0480 msgr2=0x7f78fc0b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:54.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f78fb59e640 1 -- 172.21.15.105:0/4191613769 shutdown_connections 2023-11-28T18:23:54.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fae0659c640 1 --2- 172.21.15.105:0/2701565927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae080abd80 0x7fae08158400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fadf000b430 tx=0x7fadf000b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f929229b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f928c0cbcb0 0x7f928c0ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f9291299640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928c0ad510 0x7f928c15e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f929229b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f928c0cbcb0 0x7f928c0ae890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:37890/0 (socket says 172.21.15.105:37890) 2023-11-28T18:23:54.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f929229b640 1 -- 172.21.15.105:0/3897498887 learned_addr learned my addr 172.21.15.105:0/3897498887 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:54.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7f929229b640 1 -- 172.21.15.105:0/3897498887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928c0ad510 msgr2=0x7f928c15e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7efcc1c5a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcbc0b2d20 0x7efcbc15b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7efcc1c5a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcbc0d3c30 0x7efcbc0cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7efcc1c5a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efcbc15d4f0 0x7efcbc151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7efcc1c5a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc0d0a20 con 0x7efcbc0d3c30 2023-11-28T18:23:54.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7efcc1c5a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc15bc30 con 0x7efcbc0b2d20 2023-11-28T18:23:54.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7efcc1c5a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc0cbac0 con 0x7efcbc15d4f0 2023-11-28T18:23:54.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fadfeffd640 1 -- 172.21.15.105:0/2701565927 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0011830 con 0x7fae080abd80 2023-11-28T18:23:54.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fadfeffd640 1 -- 172.21.15.105:0/2701565927 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf0011990 con 0x7fae080abd80 2023-11-28T18:23:54.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7fadfeffd640 1 -- 172.21.15.105:0/2701565927 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0011830 con 0x7fae080abd80 2023-11-28T18:23:54.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7f9291a9a640 1 --2- 172.21.15.105:0/3897498887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f928c0ad130 0x7f928c159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7f929229b640 1 --2- 172.21.15.105:0/3897498887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928c0ad510 0x7f928c15e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7efcbb7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcbc0b2d20 0x7efcbc15b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7efcbb7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcbc0b2d20 0x7efcbc15b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34282/0 (socket says 172.21.15.105:34282) 2023-11-28T18:23:54.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7efcbb7fe640 1 -- 172.21.15.105:0/255902372 learned_addr learned my addr 172.21.15.105:0/255902372 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:54.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7efcbaffd640 1 --2- 172.21.15.105:0/255902372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcbc0d3c30 0x7efcbc0cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7efcbb7fe640 1 -- 172.21.15.105:0/255902372 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efcbc15d4f0 msgr2=0x7efcbc151460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T18:23:54.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7efcbb7fe640 1 --2- 172.21.15.105:0/255902372 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efcbc15d4f0 0x7efcbc151460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7efcbb7fe640 1 -- 172.21.15.105:0/255902372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcbc0d3c30 msgr2=0x7efcbc0cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7efcbb7fe640 1 --2- 172.21.15.105:0/255902372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcbc0d3c30 0x7efcbc0cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7efcbb7fe640 1 -- 172.21.15.105:0/255902372 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac009a70 con 0x7efcbc0b2d20 2023-11-28T18:23:54.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7f929229b640 1 -- 172.21.15.105:0/3897498887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f928c0ad130 msgr2=0x7f928c159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7f929229b640 1 --2- 172.21.15.105:0/3897498887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f928c0ad130 0x7f928c159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7f929229b640 1 -- 172.21.15.105:0/3897498887 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c0094d0 con 0x7f928c0cbcb0 2023-11-28T18:23:54.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7efcbb7fe640 1 --2- 172.21.15.105:0/255902372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcbc0b2d20 0x7efcbc15b6f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efcb400c370 tx=0x7efcb400c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7f78fb59e640 1 -- 172.21.15.105:0/4191613769 wait complete. 2023-11-28T18:23:54.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7f3bad3f5640 1 Processor -- start 2023-11-28T18:23:54.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7f3bad3f5640 1 -- start start 2023-11-28T18:23:54.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7f9291a9a640 1 --2- 172.21.15.105:0/3897498887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f928c0ad130 0x7f928c159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:54.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7f9291299640 1 --2- 172.21.15.105:0/3897498887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928c0ad510 0x7f928c15e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:54.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7f929229b640 1 --2- 172.21.15.105:0/3897498887 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f928c0cbcb0 0x7f928c0ae890 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f927c0ad040 tx=0x7f927c01ac60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7efcb3fff640 1 -- 172.21.15.105:0/255902372 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb40052d0 con 0x7efcbc0b2d20 2023-11-28T18:23:54.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7efcb3fff640 1 -- 172.21.15.105:0/255902372 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb4005430 con 0x7efcbc0b2d20 2023-11-28T18:23:54.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7efcb3fff640 1 -- 172.21.15.105:0/255902372 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb4007220 con 0x7efcbc0b2d20 2023-11-28T18:23:54.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7efcc1c5a640 1 -- 172.21.15.105:0/255902372 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcbc0ca7c0 con 0x7efcbc0b2d20 2023-11-28T18:23:54.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.291+0000 7fa6749f8640 1 -- 172.21.15.105:0/1145906792 wait complete. 2023-11-28T18:23:54.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7fa6749f8640 1 Processor -- start 2023-11-28T18:23:54.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7fa6749f8640 1 -- start start 2023-11-28T18:23:54.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7f78fb59e640 1 Processor -- start 2023-11-28T18:23:54.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7f3bad3f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ba80ad130 0x7f3ba81590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7f3bad3f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba80ad510 0x7f3ba80b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7f3bad3f5640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ba80cb5a0 0x7f3ba80ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7f3bad3f5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba815dd30 con 0x7f3ba80ad510 2023-11-28T18:23:54.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7f3bad3f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba8159c90 con 0x7f3ba80ad130 2023-11-28T18:23:54.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7f3bad3f5640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba809c860 con 0x7f3ba80cb5a0 2023-11-28T18:23:54.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7f78fb59e640 1 -- start start 2023-11-28T18:23:54.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7f92827fc640 1 -- 172.21.15.105:0/3897498887 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c0b4800 con 0x7f928c0cbcb0 2023-11-28T18:23:54.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7f92827fc640 1 -- 172.21.15.105:0/3897498887 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c0b4960 con 0x7f928c0cbcb0 2023-11-28T18:23:54.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f92827fc640 1 -- 172.21.15.105:0/3897498887 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c0b4cd0 con 0x7f928c0cbcb0 2023-11-28T18:23:54.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7fa6749f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6700b4140 0x7fa670157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f3ba77fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ba80cb5a0 0x7f3ba80ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7fae0caea640 1 -- 172.21.15.105:0/2701565927 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae081620e0 con 0x7fae080abd80 2023-11-28T18:23:54.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7fae0caea640 1 -- 172.21.15.105:0/2701565927 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae081626a0 con 0x7fae080abd80 2023-11-28T18:23:54.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f3ba77fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ba80cb5a0 0x7f3ba80ae830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:37904/0 (socket says 172.21.15.105:37904) 2023-11-28T18:23:54.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f3ba77fe640 1 -- 172.21.15.105:0/1418693299 learned_addr learned my addr 172.21.15.105:0/1418693299 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:54.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7efcc1c5a640 1 -- 172.21.15.105:0/255902372 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcbc0cad80 con 0x7efcbc0b2d20 2023-11-28T18:23:54.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7fa6749f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6700d10e0 0x7fa67015c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.295+0000 7efcc1c5a640 1 -- 172.21.15.105:0/255902372 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcbc067570 con 0x7efcbc0b2d20 2023-11-28T18:23:54.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f3ba6ffd640 1 --2- 172.21.15.105:0/1418693299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ba80ad130 0x7f3ba81590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f3ba77fe640 1 -- 172.21.15.105:0/1418693299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ba80ad130 msgr2=0x7f3ba81590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f3ba77fe640 1 --2- 172.21.15.105:0/1418693299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ba80ad130 0x7f3ba81590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f3ba77fe640 1 -- 172.21.15.105:0/1418693299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba80ad510 msgr2=0x7f3ba80b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f3ba77fe640 1 --2- 172.21.15.105:0/1418693299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba80ad510 0x7f3ba80b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f3ba77fe640 1 -- 172.21.15.105:0/1418693299 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9c0094d0 con 0x7f3ba80cb5a0 2023-11-28T18:23:54.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7fa6749f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa670160530 0x7fa6700cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f9293cfe640 1 -- 172.21.15.105:0/3897498887 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928c15e9c0 con 0x7f928c0cbcb0 2023-11-28T18:23:54.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7fa66dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6700d10e0 0x7fa67015c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7fa66dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6700d10e0 0x7fa67015c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34300/0 (socket says 172.21.15.105:34300) 2023-11-28T18:23:54.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f3ba6ffd640 1 --2- 172.21.15.105:0/1418693299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ba80ad130 0x7f3ba81590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:54.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f9293cfe640 1 -- 172.21.15.105:0/3897498887 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928c15ef80 con 0x7f928c0cbcb0 2023-11-28T18:23:54.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f3ba77fe640 1 --2- 172.21.15.105:0/1418693299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ba80cb5a0 0x7f3ba80ae830 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000c760 tx=0x7f3ba000cb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7fa66e59c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6700b4140 0x7fa670157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7fa66e59c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6700b4140 0x7fa670157d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:37912/0 (socket says 172.21.15.105:37912) 2023-11-28T18:23:54.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f78fb59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc06bee0 0x7f78fc157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f78fb59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78fc06cb30 0x7f78fc15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f78fa59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc06bee0 0x7f78fc157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f78fa59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc06bee0 0x7f78fc157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55174/0 (socket says 172.21.15.105:55174) 2023-11-28T18:23:54.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7fa66ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa670160530 0x7fa6700cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7fa66ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa670160530 0x7fa6700cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55172/0 (socket says 172.21.15.105:55172) 2023-11-28T18:23:54.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f78fa59c640 1 -- 172.21.15.105:0/2953044953 learned_addr learned my addr 172.21.15.105:0/2953044953 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:54.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7fa66dd9b640 1 -- 172.21.15.105:0/2616888575 learned_addr learned my addr 172.21.15.105:0/2616888575 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:54.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f78fb59e640 1 --2- 172.21.15.105:0/2953044953 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78fc0ccf20 0x7f78fc0c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7fa6749f8640 1 -- 172.21.15.105:0/2616888575 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700b8230 con 0x7fa670160530 2023-11-28T18:23:54.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7fa6749f8640 1 -- 172.21.15.105:0/2616888575 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700b75b0 con 0x7fa6700d10e0 2023-11-28T18:23:54.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7fa6749f8640 1 -- 172.21.15.105:0/2616888575 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700b7890 con 0x7fa6700b4140 2023-11-28T18:23:54.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f78fb59e640 1 -- 172.21.15.105:0/2953044953 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc158010 con 0x7f78fc06bee0 2023-11-28T18:23:54.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f78f3fff640 1 --2- 172.21.15.105:0/2953044953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78fc06cb30 0x7f78fc15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f78fb59e640 1 -- 172.21.15.105:0/2953044953 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0c2db0 con 0x7f78fc06cb30 2023-11-28T18:23:54.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f78fb59e640 1 -- 172.21.15.105:0/2953044953 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0c21e0 con 0x7f78fc0ccf20 2023-11-28T18:23:54.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f78fad9d640 1 --2- 172.21.15.105:0/2953044953 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78fc0ccf20 0x7f78fc0c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7fa66ed9d640 1 -- 172.21.15.105:0/2616888575 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6700b4140 msgr2=0x7fa670157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7fa66ed9d640 1 --2- 172.21.15.105:0/2616888575 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6700b4140 0x7fa670157d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7fa66ed9d640 1 -- 172.21.15.105:0/2616888575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6700d10e0 msgr2=0x7fa67015c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7fa66ed9d640 1 --2- 172.21.15.105:0/2616888575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6700d10e0 0x7fa67015c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7fa66ed9d640 1 -- 172.21.15.105:0/2616888575 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa668009b30 con 0x7fa670160530 2023-11-28T18:23:54.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f78fa59c640 1 -- 172.21.15.105:0/2953044953 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78fc0ccf20 msgr2=0x7f78fc0c37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f78fa59c640 1 --2- 172.21.15.105:0/2953044953 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78fc0ccf20 0x7f78fc0c37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f78fa59c640 1 -- 172.21.15.105:0/2953044953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78fc06cb30 msgr2=0x7f78fc15cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f78fa59c640 1 --2- 172.21.15.105:0/2953044953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78fc06cb30 0x7f78fc15cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f78fa59c640 1 -- 172.21.15.105:0/2953044953 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f4009a70 con 0x7f78fc06bee0 2023-11-28T18:23:54.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f78f3fff640 1 --2- 172.21.15.105:0/2953044953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78fc06cb30 0x7f78fc15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:54.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7fae0caea640 1 -- 172.21.15.105:0/2701565927 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae08067570 con 0x7fae080abd80 2023-11-28T18:23:54.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7fadfeffd640 1 -- 172.21.15.105:0/2701565927 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fadf0011b00 con 0x7fae080abd80 2023-11-28T18:23:54.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7f78fa59c640 1 --2- 172.21.15.105:0/2953044953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc06bee0 0x7f78fc157ad0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00b4a0 tx=0x7f78ec00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7fa66ed9d640 1 --2- 172.21.15.105:0/2616888575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa670160530 0x7fa6700cb0d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa6680ab000 tx=0x7fa6680196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7f3b8f7fe640 1 -- 172.21.15.105:0/1418693299 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0006d60 con 0x7f3ba80cb5a0 2023-11-28T18:23:54.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7f3b8f7fe640 1 -- 172.21.15.105:0/1418693299 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba0003980 con 0x7f3ba80cb5a0 2023-11-28T18:23:54.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7f3b8f7fe640 1 -- 172.21.15.105:0/1418693299 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0003d30 con 0x7f3ba80cb5a0 2023-11-28T18:23:54.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7efcb3fff640 1 -- 172.21.15.105:0/255902372 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7efcb4007380 con 0x7efcbc0b2d20 2023-11-28T18:23:54.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7fa66e59c640 1 --2- 172.21.15.105:0/2616888575 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6700b4140 0x7fa670157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2023-11-28T18:23:54.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7f78fad9d640 1 --2- 172.21.15.105:0/2953044953 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78fc0ccf20 0x7f78fc0c37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:54.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7f78f2ffd640 1 -- 172.21.15.105:0/2953044953 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec010610 con 0x7f78fc06bee0 2023-11-28T18:23:54.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7f78f2ffd640 1 -- 172.21.15.105:0/2953044953 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec010790 con 0x7f78fc06bee0 2023-11-28T18:23:54.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7f78f2ffd640 1 -- 172.21.15.105:0/2953044953 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec010b40 con 0x7f78fc06bee0 2023-11-28T18:23:54.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.299+0000 7f92827fc640 1 -- 172.21.15.105:0/3897498887 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f927c0c7390 con 0x7f928c0cbcb0 2023-11-28T18:23:54.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7f92827fc640 1 --2- 172.21.15.105:0/3897498887 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9270066920 0x7f9270068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7f92827fc640 1 -- 172.21.15.105:0/3897498887 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f927c11f140 con 0x7f928c0cbcb0 2023-11-28T18:23:54.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7f9293cfe640 1 -- 172.21.15.105:0/3897498887 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f928c067570 con 0x7f928c0cbcb0 2023-11-28T18:23:54.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7fa65effd640 1 -- 172.21.15.105:0/2616888575 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6680b8670 con 0x7fa670160530 2023-11-28T18:23:54.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7f78fb59e640 1 -- 172.21.15.105:0/2953044953 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78fc0c3240 con 0x7f78fc06bee0 2023-11-28T18:23:54.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7f78fb59e640 1 -- 172.21.15.105:0/2953044953 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78fc0c2760 con 0x7f78fc06bee0 2023-11-28T18:23:54.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7fadfeffd640 1 --2- 172.21.15.105:0/2701565927 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fade8066930 0x7fade8068df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7fadfeffd640 1 -- 172.21.15.105:0/2701565927 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fadf0084620 con 0x7fae080abd80 2023-11-28T18:23:54.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7efcb3fff640 1 --2- 172.21.15.105:0/255902372 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc9c0668d0 0x7efc9c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7efcb3fff640 1 -- 172.21.15.105:0/255902372 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efcb4083b40 con 0x7efcbc0b2d20 2023-11-28T18:23:54.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7f9291a9a640 1 --2- 172.21.15.105:0/3897498887 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9270066920 0x7f9270068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7efcbaffd640 1 --2- 172.21.15.105:0/255902372 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc9c0668d0 0x7efc9c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7fae05d9b640 1 --2- 172.21.15.105:0/2701565927 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fade8066930 0x7fade8068df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7fa65effd640 1 -- 172.21.15.105:0/2616888575 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6680b87d0 con 0x7fa670160530 2023-11-28T18:23:54.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7fa65effd640 1 -- 172.21.15.105:0/2616888575 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6680b8b40 con 0x7fa670160530 2023-11-28T18:23:54.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7fa6749f8640 1 -- 172.21.15.105:0/2616888575 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6700b7b70 con 0x7fa670160530 2023-11-28T18:23:54.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7f78fb59e640 1 -- 172.21.15.105:0/2953044953 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78fc074580 con 0x7f78fc06bee0 2023-11-28T18:23:54.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7fa6749f8640 1 -- 172.21.15.105:0/2616888575 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6701582b0 con 0x7fa670160530 2023-11-28T18:23:54.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.303+0000 7f9291a9a640 1 --2- 172.21.15.105:0/3897498887 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9270066920 0x7f9270068de0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f928c0b3c10 tx=0x7f92880073e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.307+0000 7fae05d9b640 1 --2- 172.21.15.105:0/2701565927 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fade8066930 0x7fade8068df0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fadf80af000 tx=0x7fadf8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.307+0000 7fa6749f8640 1 -- 172.21.15.105:0/2616888575 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6700b8370 con 0x7fa670160530 2023-11-28T18:23:54.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.307+0000 7efcbaffd640 1 --2- 172.21.15.105:0/255902372 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc9c0668d0 0x7efc9c068d90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efcac0af000 tx=0x7efcac0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.307+0000 7f3bad3f5640 1 -- 172.21.15.105:0/1418693299 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b98009420 con 0x7f3ba80cb5a0 2023-11-28T18:23:54.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.307+0000 7f3bad3f5640 1 -- 172.21.15.105:0/1418693299 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba815d6e0 con 0x7f3ba80cb5a0 2023-11-28T18:23:54.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.307+0000 7f78f2ffd640 1 -- 172.21.15.105:0/2953044953 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f78ec010ca0 con 0x7f78fc06bee0 2023-11-28T18:23:54.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.307+0000 7f78f2ffd640 1 --2- 172.21.15.105:0/2953044953 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78dc066950 0x7f78dc068e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.307+0000 7f78f2ffd640 1 -- 172.21.15.105:0/2953044953 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f78ec084220 con 0x7f78fc06bee0 2023-11-28T18:23:54.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.307+0000 7f3bad3f5640 1 -- 172.21.15.105:0/1418693299 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba815dee0 con 0x7f3ba80cb5a0 2023-11-28T18:23:54.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.307+0000 7f78f3fff640 1 --2- 172.21.15.105:0/2953044953 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78dc066950 0x7f78dc068e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.307+0000 7f78f3fff640 1 --2- 172.21.15.105:0/2953044953 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78dc066950 0x7f78dc068e10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f78fc0c8f90 tx=0x7f78e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.307+0000 7fa65effd640 1 -- 172.21.15.105:0/2616888575 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa6680b8ca0 con 0x7fa670160530 2023-11-28T18:23:54.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.311+0000 7efcb3fff640 1 -- 172.21.15.105:0/255902372 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efcb4050a00 con 0x7efcbc0b2d20 2023-11-28T18:23:54.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.311+0000 7fa65effd640 1 --2- 172.21.15.105:0/2616888575 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6500668d0 0x7fa650068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.311+0000 7fa65effd640 1 -- 172.21.15.105:0/2616888575 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa66811fed0 con 0x7fa670160530 2023-11-28T18:23:54.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.311+0000 7fa66e59c640 1 --2- 172.21.15.105:0/2616888575 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6500668d0 0x7fa650068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.311+0000 7fa66e59c640 1 --2- 172.21.15.105:0/2616888575 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6500668d0 0x7fa650068d90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa660006fd0 tx=0x7fa660008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.311+0000 7fa65effd640 1 -- 172.21.15.105:0/2616888575 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa6680ed0b0 con 0x7fa670160530 2023-11-28T18:23:54.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.315+0000 7f92827fc640 1 -- 172.21.15.105:0/3897498887 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f927c0ec120 con 0x7f928c0cbcb0 2023-11-28T18:23:54.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.315+0000 7f3b8f7fe640 1 -- 172.21.15.105:0/1418693299 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3ba00050d0 con 0x7f3ba80cb5a0 2023-11-28T18:23:54.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.315+0000 7f3b8f7fe640 1 --2- 172.21.15.105:0/1418693299 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3b94066920 0x7f3b94068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:54.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.315+0000 7f3b8f7fe640 1 -- 172.21.15.105:0/1418693299 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3ba00561c0 con 0x7f3ba80cb5a0 2023-11-28T18:23:54.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.319+0000 7f3ba6ffd640 1 --2- 172.21.15.105:0/1418693299 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3b94066920 0x7f3b94068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:54.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.319+0000 7f3ba6ffd640 1 --2- 172.21.15.105:0/1418693299 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3b94066920 0x7f3b94068de0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3b980af000 tx=0x7f3b98008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:54.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.319+0000 7fadfeffd640 1 -- 172.21.15.105:0/2701565927 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fadf00514d0 con 0x7fae080abd80 2023-11-28T18:23:54.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.323+0000 7f3b8f7fe640 1 -- 172.21.15.105:0/1418693299 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3ba0052590 con 0x7f3ba80cb5a0 2023-11-28T18:23:54.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.327+0000 7f78f2ffd640 1 -- 172.21.15.105:0/2953044953 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f78ec0512e0 con 0x7f78fc06bee0 2023-11-28T18:23:54.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.611+0000 7f1899d1e640 1 -- 172.21.15.105:0/3036556406 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f18940c6830 con 0x7f1894161f30 2023-11-28T18:23:54.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.615+0000 7f18897fa640 1 -- 172.21.15.105:0/3036556406 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f18940c6830 con 0x7f1894161f30 2023-11-28T18:23:54.617 INFO:teuthology.orchestra.run.smithi105.stdout:55834574856 2023-11-28T18:23:54.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.619+0000 7f187affd640 1 -- 172.21.15.105:0/3036556406 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f18800669a0 msgr2=0x7f1880068e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.619+0000 7f187affd640 1 --2- 172.21.15.105:0/3036556406 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f18800669a0 0x7f1880068e60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f189000a790 tx=0x7f1890008040 comp rx=0 tx=0).stop 2023-11-28T18:23:54.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.619+0000 7f187affd640 1 -- 172.21.15.105:0/3036556406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1894161f30 msgr2=0x7f18940c2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.619+0000 7f187affd640 1 --2- 172.21.15.105:0/3036556406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1894161f30 0x7f18940c2d30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f188c019520 tx=0x7f188c0196b0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.619+0000 7f187affd640 1 -- 172.21.15.105:0/3036556406 shutdown_connections 2023-11-28T18:23:54.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.619+0000 7f187affd640 1 --2- 172.21.15.105:0/3036556406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1894161f30 0x7f18940c2d30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.619+0000 7f187affd640 1 --2- 172.21.15.105:0/3036556406 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f18800669a0 0x7f1880068e60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.619+0000 7fa13359e640 1 -- 172.21.15.105:0/4100023186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa13406d800 con 0x7fa1340b0fa0 2023-11-28T18:23:54.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.619+0000 7f187affd640 1 --2- 172.21.15.105:0/3036556406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18940ccc50 0x7f18940c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.619+0000 7f187affd640 1 --2- 172.21.15.105:0/3036556406 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18940c1950 0x7f18941563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.619+0000 7f187affd640 1 -- 172.21.15.105:0/3036556406 >> 172.21.15.105:0/3036556406 conn(0x7f18940b0480 msgr2=0x7f18940afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:54.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.619+0000 7fa122ffd640 1 -- 172.21.15.105:0/4100023186 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa128054950 con 0x7fa1340b0fa0 2023-11-28T18:23:54.624 INFO:teuthology.orchestra.run.smithi105.stdout:55834574857 2023-11-28T18:23:54.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.619+0000 7f187affd640 1 -- 172.21.15.105:0/3036556406 shutdown_connections 2023-11-28T18:23:54.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.619+0000 7f187affd640 1 -- 172.21.15.105:0/3036556406 wait complete. 2023-11-28T18:23:54.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.623+0000 7fa13359e640 1 -- 172.21.15.105:0/4100023186 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1140669a0 msgr2=0x7fa114068e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.623+0000 7fa13359e640 1 --2- 172.21.15.105:0/4100023186 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1140669a0 0x7fa114068e60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa12400af30 tx=0x7fa12400a7f0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.623+0000 7fa13359e640 1 -- 172.21.15.105:0/4100023186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1340b0fa0 msgr2=0x7fa1340ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.623+0000 7fa13359e640 1 --2- 172.21.15.105:0/4100023186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1340b0fa0 0x7fa1340ac8d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa12800c800 tx=0x7fa12800ccc0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.623+0000 7fa13359e640 1 -- 172.21.15.105:0/4100023186 shutdown_connections 2023-11-28T18:23:54.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.623+0000 7fa13359e640 1 --2- 172.21.15.105:0/4100023186 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1340cba90 0x7fa1340ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.623+0000 7fa13359e640 1 --2- 172.21.15.105:0/4100023186 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1140669a0 0x7fa114068e60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.623+0000 7fa13359e640 1 --2- 172.21.15.105:0/4100023186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1340b0fa0 0x7fa1340ac8d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.623+0000 7fa13359e640 1 --2- 172.21.15.105:0/4100023186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1340adf20 0x7fa1340c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.623+0000 7fa13359e640 1 -- 172.21.15.105:0/4100023186 >> 172.21.15.105:0/4100023186 conn(0x7fa1340b0480 msgr2=0x7fa1340aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:54.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.623+0000 7fa13359e640 1 -- 172.21.15.105:0/4100023186 shutdown_connections 2023-11-28T18:23:54.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.623+0000 7fa13359e640 1 -- 172.21.15.105:0/4100023186 wait complete. 2023-11-28T18:23:54.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.631+0000 7efcc1c5a640 1 -- 172.21.15.105:0/255902372 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7efcbc073260 con 0x7efcbc0b2d20 2023-11-28T18:23:54.640 INFO:teuthology.orchestra.run.smithi105.stdout:55834574857 2023-11-28T18:23:54.640 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.635+0000 7efcb3fff640 1 -- 172.21.15.105:0/255902372 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efcb4054020 con 0x7efcbc0b2d20 2023-11-28T18:23:54.640 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.635+0000 7efcb1ffb640 1 -- 172.21.15.105:0/255902372 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc9c0668d0 msgr2=0x7efc9c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.635+0000 7efcb1ffb640 1 --2- 172.21.15.105:0/255902372 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc9c0668d0 0x7efc9c068d90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efcac0af000 tx=0x7efcac0a7070 comp rx=0 tx=0).stop 2023-11-28T18:23:54.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.635+0000 7efcb1ffb640 1 -- 172.21.15.105:0/255902372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcbc0b2d20 msgr2=0x7efcbc15b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.635+0000 7efcb1ffb640 1 --2- 172.21.15.105:0/255902372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcbc0b2d20 0x7efcbc15b6f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efcb400c370 tx=0x7efcb400c830 comp rx=0 tx=0).stop 2023-11-28T18:23:54.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.639+0000 7efcb1ffb640 1 -- 172.21.15.105:0/255902372 shutdown_connections 2023-11-28T18:23:54.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.639+0000 7efcb1ffb640 1 --2- 172.21.15.105:0/255902372 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efcbc15d4f0 0x7efcbc151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.639+0000 7efcb1ffb640 1 --2- 172.21.15.105:0/255902372 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc9c0668d0 0x7efc9c068d90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.639+0000 7efcb1ffb640 1 --2- 172.21.15.105:0/255902372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcbc0d3c30 0x7efcbc0cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.643 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.639+0000 7efcb1ffb640 1 --2- 172.21.15.105:0/255902372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcbc0b2d20 0x7efcbc15b6f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.643 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.639+0000 7efcb1ffb640 1 -- 172.21.15.105:0/255902372 >> 172.21.15.105:0/255902372 conn(0x7efcbc0d3030 msgr2=0x7efcbc0afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:54.643 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.639+0000 7efcb1ffb640 1 -- 172.21.15.105:0/255902372 shutdown_connections 2023-11-28T18:23:54.643 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.639+0000 7efcb1ffb640 1 -- 172.21.15.105:0/255902372 wait complete. 2023-11-28T18:23:54.643 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.639+0000 7fae0caea640 1 -- 172.21.15.105:0/2701565927 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fae080ae280 con 0x7fae080abd80 2023-11-28T18:23:54.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.639+0000 7fadfeffd640 1 -- 172.21.15.105:0/2701565927 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fadf0015020 con 0x7fae080abd80 2023-11-28T18:23:54.644 INFO:teuthology.orchestra.run.smithi105.stdout:55834574856 2023-11-28T18:23:54.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.639+0000 7fa6749f8640 1 -- 172.21.15.105:0/2616888575 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa6700b8370 con 0x7fa670160530 2023-11-28T18:23:54.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.643+0000 7f9293cfe640 1 -- 172.21.15.105:0/3897498887 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f928c09cc20 con 0x7f928c0cbcb0 2023-11-28T18:23:54.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.643+0000 7fa65effd640 1 -- 172.21.15.105:0/2616888575 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa6700b8370 con 0x7fa670160530 2023-11-28T18:23:54.649 INFO:teuthology.orchestra.run.smithi105.stdout:55834574856 2023-11-28T18:23:54.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.643+0000 7f92827fc640 1 -- 172.21.15.105:0/3897498887 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f927c0ef740 con 0x7f928c0cbcb0 2023-11-28T18:23:54.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.643+0000 7fae0caea640 1 -- 172.21.15.105:0/2701565927 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fade8066930 msgr2=0x7fade8068df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.643+0000 7fae0caea640 1 --2- 172.21.15.105:0/2701565927 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fade8066930 0x7fade8068df0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fadf80af000 tx=0x7fadf8008040 comp rx=0 tx=0).stop 2023-11-28T18:23:54.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.643+0000 7fae0caea640 1 -- 172.21.15.105:0/2701565927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae080abd80 msgr2=0x7fae08158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.643+0000 7fae0caea640 1 --2- 172.21.15.105:0/2701565927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae080abd80 0x7fae08158400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fadf000b430 tx=0x7fadf000b8f0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.643+0000 7fae0caea640 1 -- 172.21.15.105:0/2701565927 shutdown_connections 2023-11-28T18:23:54.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.643+0000 7fae0caea640 1 --2- 172.21.15.105:0/2701565927 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae08161520 0x7fae0806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.643+0000 7fae0caea640 1 --2- 172.21.15.105:0/2701565927 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fade8066930 0x7fade8068df0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.643+0000 7fae0caea640 1 --2- 172.21.15.105:0/2701565927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae080cba90 0x7fae0806bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.643+0000 7fae0caea640 1 --2- 172.21.15.105:0/2701565927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae080abd80 0x7fae08158400 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.643+0000 7fae0caea640 1 -- 172.21.15.105:0/2701565927 >> 172.21.15.105:0/2701565927 conn(0x7fae080b08c0 msgr2=0x7fae080afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:54.651 INFO:teuthology.orchestra.run.smithi105.stdout:55834574856 2023-11-28T18:23:54.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.647+0000 7fae0caea640 1 -- 172.21.15.105:0/2701565927 shutdown_connections 2023-11-28T18:23:54.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.647+0000 7fae0caea640 1 -- 172.21.15.105:0/2701565927 wait complete. 2023-11-28T18:23:54.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.647+0000 7fa6749f8640 1 -- 172.21.15.105:0/2616888575 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6500668d0 msgr2=0x7fa650068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.647+0000 7fa6749f8640 1 --2- 172.21.15.105:0/2616888575 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6500668d0 0x7fa650068d90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa660006fd0 tx=0x7fa660008040 comp rx=0 tx=0).stop 2023-11-28T18:23:54.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.647+0000 7fa6749f8640 1 -- 172.21.15.105:0/2616888575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa670160530 msgr2=0x7fa6700cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.647+0000 7fa6749f8640 1 --2- 172.21.15.105:0/2616888575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa670160530 0x7fa6700cb0d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa6680ab000 tx=0x7fa6680196f0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.647+0000 7fa6749f8640 1 -- 172.21.15.105:0/2616888575 shutdown_connections 2023-11-28T18:23:54.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.647+0000 7fa6749f8640 1 --2- 172.21.15.105:0/2616888575 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6500668d0 0x7fa650068d90 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa660006fd0 tx=0x7fa660008040 comp rx=0 tx=0).stop 2023-11-28T18:23:54.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.647+0000 7fa6749f8640 1 --2- 172.21.15.105:0/2616888575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa670160530 0x7fa6700cb0d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.647+0000 7fa6749f8640 1 --2- 172.21.15.105:0/2616888575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6700d10e0 0x7fa67015c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.647+0000 7fa6749f8640 1 --2- 172.21.15.105:0/2616888575 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6700b4140 0x7fa670157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.647+0000 7fa6749f8640 1 -- 172.21.15.105:0/2616888575 >> 172.21.15.105:0/2616888575 conn(0x7fa6700d30a0 msgr2=0x7fa67006c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:54.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.647+0000 7f9293cfe640 1 -- 172.21.15.105:0/3897498887 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9270066920 msgr2=0x7f9270068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.647+0000 7f9293cfe640 1 --2- 172.21.15.105:0/3897498887 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9270066920 0x7f9270068de0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f928c0b3c10 tx=0x7f92880073e0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.647+0000 7fa6749f8640 1 -- 172.21.15.105:0/2616888575 shutdown_connections 2023-11-28T18:23:54.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.647+0000 7fa6749f8640 1 -- 172.21.15.105:0/2616888575 wait complete. 2023-11-28T18:23:54.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.647+0000 7f9293cfe640 1 -- 172.21.15.105:0/3897498887 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f928c0cbcb0 msgr2=0x7f928c0ae890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.647+0000 7f9293cfe640 1 --2- 172.21.15.105:0/3897498887 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f928c0cbcb0 0x7f928c0ae890 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f927c0ad040 tx=0x7f927c01ac60 comp rx=0 tx=0).stop 2023-11-28T18:23:54.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.651+0000 7f9293cfe640 1 -- 172.21.15.105:0/3897498887 shutdown_connections 2023-11-28T18:23:54.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.651+0000 7f9293cfe640 1 --2- 172.21.15.105:0/3897498887 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f928c0cbcb0 0x7f928c0ae890 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.651+0000 7f9293cfe640 1 --2- 172.21.15.105:0/3897498887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928c0ad510 0x7f928c15e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.651+0000 7f9293cfe640 1 --2- 172.21.15.105:0/3897498887 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9270066920 0x7f9270068de0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.651+0000 7f9293cfe640 1 --2- 172.21.15.105:0/3897498887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f928c0ad130 0x7f928c159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.651+0000 7f9293cfe640 1 -- 172.21.15.105:0/3897498887 >> 172.21.15.105:0/3897498887 conn(0x7f928c0b08c0 msgr2=0x7f928c06cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:54.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.651+0000 7f3bad3f5640 1 -- 172.21.15.105:0/1418693299 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f3ba815dee0 con 0x7f3ba80cb5a0 2023-11-28T18:23:54.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.651+0000 7f9293cfe640 1 -- 172.21.15.105:0/3897498887 shutdown_connections 2023-11-28T18:23:54.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.651+0000 7f9293cfe640 1 -- 172.21.15.105:0/3897498887 wait complete. 2023-11-28T18:23:54.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.655+0000 7f3b8f7fe640 1 -- 172.21.15.105:0/1418693299 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3ba815dee0 con 0x7f3ba80cb5a0 2023-11-28T18:23:54.657 INFO:teuthology.orchestra.run.smithi105.stdout:55834574857 2023-11-28T18:23:54.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.655+0000 7f78fb59e640 1 -- 172.21.15.105:0/2953044953 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f78fc06d3a0 con 0x7f78fc06bee0 2023-11-28T18:23:54.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.655+0000 7f78f2ffd640 1 -- 172.21.15.105:0/2953044953 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f78ec054900 con 0x7f78fc06bee0 2023-11-28T18:23:54.661 INFO:teuthology.orchestra.run.smithi105.stdout:55834574857 2023-11-28T18:23:54.662 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.2 2023-11-28T18:23:54.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.655+0000 7f3bad3f5640 1 -- 172.21.15.105:0/1418693299 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3b94066920 msgr2=0x7f3b94068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.655+0000 7f3bad3f5640 1 --2- 172.21.15.105:0/1418693299 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3b94066920 0x7f3b94068de0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3b980af000 tx=0x7f3b98008040 comp rx=0 tx=0).stop 2023-11-28T18:23:54.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.655+0000 7f3bad3f5640 1 -- 172.21.15.105:0/1418693299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ba80cb5a0 msgr2=0x7f3ba80ae830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.655+0000 7f3bad3f5640 1 --2- 172.21.15.105:0/1418693299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ba80cb5a0 0x7f3ba80ae830 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000c760 tx=0x7f3ba000cb60 comp rx=0 tx=0).stop 2023-11-28T18:23:54.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.659+0000 7f3bad3f5640 1 -- 172.21.15.105:0/1418693299 shutdown_connections 2023-11-28T18:23:54.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.659+0000 7f3bad3f5640 1 --2- 172.21.15.105:0/1418693299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ba80cb5a0 0x7f3ba80ae830 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.659+0000 7f3bad3f5640 1 --2- 172.21.15.105:0/1418693299 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3b94066920 0x7f3b94068de0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.659+0000 7f3bad3f5640 1 --2- 172.21.15.105:0/1418693299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba80ad510 0x7f3ba80b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.659+0000 7f3bad3f5640 1 --2- 172.21.15.105:0/1418693299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ba80ad130 0x7f3ba81590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.659+0000 7f3bad3f5640 1 -- 172.21.15.105:0/1418693299 >> 172.21.15.105:0/1418693299 conn(0x7f3ba80b08c0 msgr2=0x7f3ba80afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:54.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.659+0000 7f3bad3f5640 1 -- 172.21.15.105:0/1418693299 shutdown_connections 2023-11-28T18:23:54.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.659+0000 7f3bad3f5640 1 -- 172.21.15.105:0/1418693299 wait complete. 2023-11-28T18:23:54.664 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2023-11-28T18:23:54.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.659+0000 7f78fb59e640 1 -- 172.21.15.105:0/2953044953 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78dc066950 msgr2=0x7f78dc068e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.659+0000 7f78fb59e640 1 --2- 172.21.15.105:0/2953044953 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78dc066950 0x7f78dc068e10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f78fc0c8f90 tx=0x7f78e4009040 comp rx=0 tx=0).stop 2023-11-28T18:23:54.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.659+0000 7f78fb59e640 1 -- 172.21.15.105:0/2953044953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc06bee0 msgr2=0x7f78fc157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:54.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.659+0000 7f78fb59e640 1 --2- 172.21.15.105:0/2953044953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc06bee0 0x7f78fc157ad0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00b4a0 tx=0x7f78ec00b960 comp rx=0 tx=0).stop 2023-11-28T18:23:54.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.659+0000 7f78fb59e640 1 -- 172.21.15.105:0/2953044953 shutdown_connections 2023-11-28T18:23:54.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.659+0000 7f78fb59e640 1 --2- 172.21.15.105:0/2953044953 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78fc0ccf20 0x7f78fc0c37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.659+0000 7f78fb59e640 1 --2- 172.21.15.105:0/2953044953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78fc06cb30 0x7f78fc15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.659+0000 7f78fb59e640 1 --2- 172.21.15.105:0/2953044953 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78dc066950 0x7f78dc068e10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.659+0000 7f78fb59e640 1 --2- 172.21.15.105:0/2953044953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc06bee0 0x7f78fc157ad0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:54.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.659+0000 7f78fb59e640 1 -- 172.21.15.105:0/2953044953 >> 172.21.15.105:0/2953044953 conn(0x7f78fc0b0480 msgr2=0x7f78fc0af870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:54.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.659+0000 7f78fb59e640 1 -- 172.21.15.105:0/2953044953 shutdown_connections 2023-11-28T18:23:54.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:54.659+0000 7f78fb59e640 1 -- 172.21.15.105:0/2953044953 wait complete. 2023-11-28T18:23:54.683 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.1 2023-11-28T18:23:54.687 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.5 2023-11-28T18:23:54.688 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.3 2023-11-28T18:23:54.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.6 2023-11-28T18:23:54.702 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.4 2023-11-28T18:23:54.703 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.7 2023-11-28T18:23:55.662 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-28T18:23:55.665 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T18:23:55.684 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-28T18:23:55.687 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-28T18:23:55.689 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-28T18:23:55.690 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-28T18:23:55.703 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T18:23:55.704 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-28T18:23:55.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.879+0000 7f241a0d6640 1 Processor -- start 2023-11-28T18:23:55.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.879+0000 7f241a0d6640 1 -- start start 2023-11-28T18:23:55.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.879+0000 7f241a0d6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f241413f6c0 0x7f2414071c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:55.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.879+0000 7f241a0d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f241413fe70 0x7f24140cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:55.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.883+0000 7f241a0d6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24140cbd20 con 0x7f24140cb8e0 2023-11-28T18:23:55.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.883+0000 7f241a0d6640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24140b3fd0 con 0x7f241413f6c0 2023-11-28T18:23:55.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.883+0000 7f241a0d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24140ce110 con 0x7f241413fe70 2023-11-28T18:23:55.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.883+0000 7f2412ffd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f241413fe70 0x7f24140cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:55.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.883+0000 7f2412ffd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f241413fe70 0x7f24140cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:37940/0 (socket says 172.21.15.105:37940) 2023-11-28T18:23:55.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.883+0000 7f2412ffd640 1 -- 172.21.15.105:0/366830335 learned_addr learned my addr 172.21.15.105:0/366830335 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:55.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.883+0000 7f24137fe640 1 --2- 172.21.15.105:0/366830335 >> v2:172.21.15.105:3300/0 conn(0x7f241413f6c0 0x7f2414071c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:55.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.883+0000 7f2412ffd640 1 -- 172.21.15.105:0/366830335 >> v2:172.21.15.105:3300/0 conn(0x7f241413f6c0 msgr2=0x7f2414071c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:55.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.883+0000 7f2412ffd640 1 --2- 172.21.15.105:0/366830335 >> v2:172.21.15.105:3300/0 conn(0x7f241413f6c0 0x7f2414071c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:55.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.883+0000 7f2412ffd640 1 -- 172.21.15.105:0/366830335 >> v1:172.21.15.105:6789/0 conn(0x7f24140cb8e0 legacy=0x7f24140cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:55.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.883+0000 7f2412ffd640 1 -- 172.21.15.105:0/366830335 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24140b3a10 con 0x7f241413fe70 2023-11-28T18:23:55.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.887+0000 7f2412ffd640 1 --2- 172.21.15.105:0/366830335 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f241413fe70 0x7f24140cb3a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f240c00c2e0 tx=0x7f240c01dd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6424dd55d022c7b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:55.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.887+0000 7f24127fc640 1 -- 172.21.15.105:0/366830335 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c0a6db0 con 0x7f241413fe70 2023-11-28T18:23:55.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.887+0000 7f24127fc640 1 -- 172.21.15.105:0/366830335 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240c00a0d0 con 0x7f241413fe70 2023-11-28T18:23:55.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.887+0000 7f24127fc640 1 -- 172.21.15.105:0/366830335 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c00a460 con 0x7f241413fe70 2023-11-28T18:23:55.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.891+0000 7f241a0d6640 1 -- 172.21.15.105:0/366830335 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f241413fe70 msgr2=0x7f24140cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:55.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.891+0000 7f241a0d6640 1 --2- 172.21.15.105:0/366830335 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f241413fe70 0x7f24140cb3a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f240c00c2e0 tx=0x7f240c01dd30 comp rx=0 tx=0).stop 2023-11-28T18:23:55.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.891+0000 7f241a0d6640 1 -- 172.21.15.105:0/366830335 shutdown_connections 2023-11-28T18:23:55.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.891+0000 7f241a0d6640 1 --2- 172.21.15.105:0/366830335 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f241413fe70 0x7f24140cb3a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:55.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.891+0000 7f241a0d6640 1 --2- 172.21.15.105:0/366830335 >> v2:172.21.15.105:3300/0 conn(0x7f241413f6c0 0x7f2414071c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:55.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.891+0000 7f241a0d6640 1 -- 172.21.15.105:0/366830335 >> 172.21.15.105:0/366830335 conn(0x7f24140b0720 msgr2=0x7f24140b0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:55.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.891+0000 7f241a0d6640 1 -- 172.21.15.105:0/366830335 shutdown_connections 2023-11-28T18:23:55.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.891+0000 7f241a0d6640 1 -- 172.21.15.105:0/366830335 wait complete. 2023-11-28T18:23:55.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.895+0000 7f241a0d6640 1 Processor -- start 2023-11-28T18:23:55.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.895+0000 7f241a0d6640 1 -- start start 2023-11-28T18:23:55.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.895+0000 7f241a0d6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24140cb8e0 0x7f24140c4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:55.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.895+0000 7f241a0d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f241413f6c0 0x7f241413d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:55.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.895+0000 7f241a0d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f241413fe70 0x7f24141f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:55.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.895+0000 7f241a0d6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24140c4970 con 0x7f24140cb8e0 2023-11-28T18:23:55.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.895+0000 7f241a0d6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241406c1e0 con 0x7f241413fe70 2023-11-28T18:23:55.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.895+0000 7f241a0d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241413d570 con 0x7f241413f6c0 2023-11-28T18:23:55.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.895+0000 7f2413fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f241413fe70 0x7f24141f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:55.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.895+0000 7f2413fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f241413fe70 0x7f24141f4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34316/0 (socket says 172.21.15.105:34316) 2023-11-28T18:23:55.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.895+0000 7f2413fff640 1 -- 172.21.15.105:0/2055950018 learned_addr learned my addr 172.21.15.105:0/2055950018 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:55.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.895+0000 7f2412ffd640 1 --2- 172.21.15.105:0/2055950018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f241413f6c0 0x7f241413d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:55.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.899+0000 7f24137fe640 1 --2- 172.21.15.105:0/2055950018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24140cb8e0 0x7f24140c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:55.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.899+0000 7f2413fff640 1 -- 172.21.15.105:0/2055950018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f241413f6c0 msgr2=0x7f241413d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:55.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.899+0000 7f2413fff640 1 --2- 172.21.15.105:0/2055950018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f241413f6c0 0x7f241413d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:55.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.899+0000 7f2413fff640 1 -- 172.21.15.105:0/2055950018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24140cb8e0 msgr2=0x7f24140c4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:55.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.899+0000 7f2413fff640 1 --2- 172.21.15.105:0/2055950018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24140cb8e0 0x7f24140c4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:55.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.899+0000 7f2413fff640 1 -- 172.21.15.105:0/2055950018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240c00c050 con 0x7f241413fe70 2023-11-28T18:23:55.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.899+0000 7f2413fff640 1 --2- 172.21.15.105:0/2055950018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f241413fe70 0x7f24141f4fe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f240800c3f0 tx=0x7f240800c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:55.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.899+0000 7f23fbfff640 1 -- 172.21.15.105:0/2055950018 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2408011790 con 0x7f241413fe70 2023-11-28T18:23:55.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.899+0000 7f241a0d6640 1 -- 172.21.15.105:0/2055950018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f241413d7b0 con 0x7f241413fe70 2023-11-28T18:23:55.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.899+0000 7f241a0d6640 1 -- 172.21.15.105:0/2055950018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241406c6d0 con 0x7f241413fe70 2023-11-28T18:23:55.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.899+0000 7f23fbfff640 1 -- 172.21.15.105:0/2055950018 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24080118f0 con 0x7f241413fe70 2023-11-28T18:23:55.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.899+0000 7f23fbfff640 1 -- 172.21.15.105:0/2055950018 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2408011790 con 0x7f241413fe70 2023-11-28T18:23:55.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.899+0000 7f241a0d6640 1 -- 172.21.15.105:0/2055950018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23f4005260 con 0x7f241413fe70 2023-11-28T18:23:55.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.903+0000 7f23fbfff640 1 -- 172.21.15.105:0/2055950018 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f2408011a60 con 0x7f241413fe70 2023-11-28T18:23:55.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.903+0000 7f23fbfff640 1 --2- 172.21.15.105:0/2055950018 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2400066990 0x7f2400068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:55.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.903+0000 7f23fbfff640 1 -- 172.21.15.105:0/2055950018 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2408055ce0 con 0x7f241413fe70 2023-11-28T18:23:55.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.907+0000 7f23fbfff640 1 -- 172.21.15.105:0/2055950018 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f24080520b0 con 0x7f241413fe70 2023-11-28T18:23:55.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.907+0000 7f24137fe640 1 --2- 172.21.15.105:0/2055950018 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2400066990 0x7f2400068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:55.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.907+0000 7f24137fe640 1 --2- 172.21.15.105:0/2055950018 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2400066990 0x7f2400068e50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2404009790 tx=0x7f2404008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.951+0000 7fa8db59e640 1 Processor -- start 2023-11-28T18:23:55.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.955+0000 7fa8db59e640 1 -- start start 2023-11-28T18:23:55.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.955+0000 7fa8db59e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa8dc15d000 0x7fa8dc0d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:55.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.955+0000 7fa8db59e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8dc0b2a80 0x7fa8dc0b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:55.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.955+0000 7fa8db59e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8dc0ce4c0 con 0x7fa8dc0cf6d0 2023-11-28T18:23:55.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.955+0000 7fa8db59e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8dc0cedf0 con 0x7fa8dc15d000 2023-11-28T18:23:55.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.955+0000 7fa8db59e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8dc0ce7a0 con 0x7fa8dc0b2a80 2023-11-28T18:23:55.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.955+0000 7fa8d1d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa8dc0cf6d0 0x7fa8dc0cfb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:47700/0 (socket says 172.21.15.105:47700) 2023-11-28T18:23:55.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.955+0000 7fa8dad9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8dc0b2a80 0x7fa8dc0b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:55.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.955+0000 7fa8dad9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8dc0b2a80 0x7fa8dc0b2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34332/0 (socket says 172.21.15.105:34332) 2023-11-28T18:23:55.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.955+0000 7fa8dad9d640 1 -- 172.21.15.105:0/2412264985 learned_addr learned my addr 172.21.15.105:0/2412264985 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:55.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.955+0000 7fa8da59c640 1 --2- 172.21.15.105:0/2412264985 >> v2:172.21.15.105:3300/0 conn(0x7fa8dc15d000 0x7fa8dc0d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:55.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.959+0000 7f580a955640 1 Processor -- start 2023-11-28T18:23:55.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.959+0000 7f580a955640 1 -- start start 2023-11-28T18:23:55.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.959+0000 7fa8dad9d640 1 -- 172.21.15.105:0/2412264985 >> v2:172.21.15.105:3300/0 conn(0x7fa8dc15d000 msgr2=0x7fa8dc0d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:55.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.959+0000 7fa8dad9d640 1 --2- 172.21.15.105:0/2412264985 >> v2:172.21.15.105:3300/0 conn(0x7fa8dc15d000 0x7fa8dc0d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:55.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.959+0000 7fa8dad9d640 1 -- 172.21.15.105:0/2412264985 >> v1:172.21.15.148:6789/0 conn(0x7fa8dc0cf6d0 legacy=0x7fa8dc0cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:55.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.959+0000 7fa8dad9d640 1 -- 172.21.15.105:0/2412264985 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc0cea80 con 0x7fa8dc0b2a80 2023-11-28T18:23:55.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.959+0000 7f580a955640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f58040b13e0 0x7f5804158e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:55.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.959+0000 7f580a955640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f58040abd80 0x7f58040ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:55.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.959+0000 7f580a955640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f580415d060 con 0x7f58040cb5a0 2023-11-28T18:23:55.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.959+0000 7f58037fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f58040abd80 0x7f58040ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:55.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.959+0000 7f58037fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f58040abd80 0x7f58040ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34340/0 (socket says 172.21.15.105:34340) 2023-11-28T18:23:55.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.959+0000 7f58037fe640 1 -- 172.21.15.105:0/98219077 learned_addr learned my addr 172.21.15.105:0/98219077 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:55.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.959+0000 7f580a955640 1 -- 172.21.15.105:0/98219077 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580415d340 con 0x7f58040b13e0 2023-11-28T18:23:55.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.959+0000 7f580a955640 1 -- 172.21.15.105:0/98219077 --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580415d620 con 0x7f58040abd80 2023-11-28T18:23:55.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.959+0000 7fa8da59c640 1 --2- 172.21.15.105:0/2412264985 >> v2:172.21.15.105:3300/0 conn(0x7fa8dc15d000 0x7fa8dc0d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:55.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.963+0000 7f5803fff640 1 --2- 172.21.15.105:0/98219077 >> v2:172.21.15.105:3300/0 conn(0x7f58040b13e0 0x7f5804158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:55.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.963+0000 7f58037fe640 1 -- 172.21.15.105:0/98219077 >> v2:172.21.15.105:3300/0 conn(0x7f58040b13e0 msgr2=0x7f5804158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:55.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.963+0000 7f58037fe640 1 --2- 172.21.15.105:0/98219077 >> v2:172.21.15.105:3300/0 conn(0x7f58040b13e0 0x7f5804158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:55.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.963+0000 7f58037fe640 1 -- 172.21.15.105:0/98219077 >> v1:172.21.15.105:6789/0 conn(0x7f58040cb5a0 legacy=0x7f58040cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:55.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.963+0000 7f58037fe640 1 -- 172.21.15.105:0/98219077 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580415d900 con 0x7f58040abd80 2023-11-28T18:23:55.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.963+0000 7fa8dad9d640 1 --2- 172.21.15.105:0/2412264985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8dc0b2a80 0x7fa8dc0b2eb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0b2050 tx=0x7fa8d401c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cc0458c732bc1ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:55.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.963+0000 7fa8d9d9b640 1 -- 172.21.15.105:0/2412264985 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc0b2490 con 0x7fa8dc0b2a80 2023-11-28T18:23:55.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.963+0000 7fa8d9d9b640 1 -- 172.21.15.105:0/2412264985 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d40a4a80 con 0x7fa8dc0b2a80 2023-11-28T18:23:55.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.963+0000 7f27d4017640 1 Processor -- start 2023-11-28T18:23:55.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.963+0000 7fa8d9d9b640 1 -- 172.21.15.105:0/2412264985 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40a4e10 con 0x7fa8dc0b2a80 2023-11-28T18:23:55.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f27d4017640 1 -- start start 2023-11-28T18:23:55.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f3923580640 1 Processor -- start 2023-11-28T18:23:55.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7fa8db59e640 1 -- 172.21.15.105:0/2412264985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8dc0b2a80 msgr2=0x7fa8dc0b2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:55.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7fa8db59e640 1 --2- 172.21.15.105:0/2412264985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8dc0b2a80 0x7fa8dc0b2eb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0b2050 tx=0x7fa8d401c4d0 comp rx=0 tx=0).stop 2023-11-28T18:23:55.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f3923580640 1 -- start start 2023-11-28T18:23:55.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7fa8db59e640 1 -- 172.21.15.105:0/2412264985 shutdown_connections 2023-11-28T18:23:55.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7fa8db59e640 1 --2- 172.21.15.105:0/2412264985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8dc0b2a80 0x7fa8dc0b2eb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:55.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7fa8db59e640 1 --2- 172.21.15.105:0/2412264985 >> v2:172.21.15.105:3300/0 conn(0x7fa8dc15d000 0x7fa8dc0d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:55.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7fa8db59e640 1 -- 172.21.15.105:0/2412264985 >> 172.21.15.105:0/2412264985 conn(0x7fa8dc0d2e00 msgr2=0x7fa8dc0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:55.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f58037fe640 1 --2- 172.21.15.105:0/98219077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58040abd80 0x7f58040ce2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f58040cc1d0 tx=0x7f57f801c4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c63fc878f20939d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:55.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f27d4017640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f27cc06d610 0x7f27cc0cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:55.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f27d4017640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f27cc0ae100 0x7f27cc0cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:55.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f27d4017640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27cc162820 con 0x7f27cc0ce530 2023-11-28T18:23:55.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7fa8db59e640 1 -- 172.21.15.105:0/2412264985 shutdown_connections 2023-11-28T18:23:55.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7fa8db59e640 1 -- 172.21.15.105:0/2412264985 wait complete. 2023-11-28T18:23:55.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f27d4017640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27cc161c10 con 0x7f27cc0ae100 2023-11-28T18:23:55.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f27d4017640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27cc161ef0 con 0x7f27cc06d610 2023-11-28T18:23:55.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f3923580640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f391c0d10e0 0x7f391c0d3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:55.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f3923580640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f391c0b4140 0x7f391c0d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:55.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f5802ffd640 1 -- 172.21.15.105:0/98219077 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f80ab600 con 0x7f58040abd80 2023-11-28T18:23:55.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f5802ffd640 1 -- 172.21.15.105:0/98219077 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f80a7050 con 0x7f58040abd80 2023-11-28T18:23:55.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f5802ffd640 1 -- 172.21.15.105:0/98219077 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f80ab980 con 0x7f58040abd80 2023-11-28T18:23:55.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f3923580640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f391c160530 0x7f391c160950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:55.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f3923580640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f391c160e90 con 0x7f391c0d10e0 2023-11-28T18:23:55.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f3923580640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f391c0d1d20 con 0x7f391c0b4140 2023-11-28T18:23:55.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f27d1db3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f27cc06d610 0x7f27cc0cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:55.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f27d15b2640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f27cc0ce530 0x7f27cc0ad9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:47704/0 (socket says 172.21.15.105:47704) 2023-11-28T18:23:55.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f27d15b2640 1 -- 172.21.15.105:0/2001003263 learned_addr learned my addr 172.21.15.105:0/2001003263 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:55.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f3923580640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f391c0d01d0 con 0x7f391c160530 2023-11-28T18:23:55.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f27d25b4640 1 --2- 172.21.15.105:0/2001003263 >> v2:172.21.15.105:3300/0 conn(0x7f27cc0ae100 0x7f27cc0cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:55.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f580a955640 1 -- 172.21.15.105:0/98219077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58040abd80 msgr2=0x7f58040ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:55.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.967+0000 7f580a955640 1 --2- 172.21.15.105:0/98219077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58040abd80 0x7f58040ce2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f58040cc1d0 tx=0x7f57f801c4c0 comp rx=0 tx=0).stop 2023-11-28T18:23:55.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f27d0db1640 1 -- 172.21.15.105:0/2001003263 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4096466602 0 0) 0x7f27cc162820 con 0x7f27cc0ce530 2023-11-28T18:23:55.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f27d0db1640 1 -- 172.21.15.105:0/2001003263 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27b8003530 con 0x7f27cc0ce530 2023-11-28T18:23:55.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f3920b1b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f391c0b4140 0x7f391c0d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:55.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f3920b1b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f391c0b4140 0x7f391c0d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34354/0 (socket says 172.21.15.105:34354) 2023-11-28T18:23:55.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f3920b1b640 1 -- 172.21.15.105:0/3220517206 learned_addr learned my addr 172.21.15.105:0/3220517206 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:55.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f580a955640 1 -- 172.21.15.105:0/98219077 shutdown_connections 2023-11-28T18:23:55.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f580a955640 1 --2- 172.21.15.105:0/98219077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58040abd80 0x7f58040ce2f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:55.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f580a955640 1 --2- 172.21.15.105:0/98219077 >> v2:172.21.15.105:3300/0 conn(0x7f58040b13e0 0x7f5804158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:55.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f580a955640 1 -- 172.21.15.105:0/98219077 >> 172.21.15.105:0/98219077 conn(0x7f58040b08c0 msgr2=0x7f58040b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:55.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f580a955640 1 -- 172.21.15.105:0/98219077 shutdown_connections 2023-11-28T18:23:55.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f0299580640 1 Processor -- start 2023-11-28T18:23:55.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f392131c640 1 --2- 172.21.15.105:0/3220517206 >> v2:172.21.15.105:3300/0 conn(0x7f391c0d10e0 0x7f391c0d3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:55.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f27d1db3640 1 -- 172.21.15.105:0/2001003263 >> v2:172.21.15.105:3300/0 conn(0x7f27cc0ae100 msgr2=0x7f27cc0cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:55.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f27d1db3640 1 --2- 172.21.15.105:0/2001003263 >> v2:172.21.15.105:3300/0 conn(0x7f27cc0ae100 0x7f27cc0cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:55.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f27d1db3640 1 -- 172.21.15.105:0/2001003263 >> v1:172.21.15.148:6789/0 conn(0x7f27cc0ce530 legacy=0x7f27cc0ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:55.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f27d1db3640 1 -- 172.21.15.105:0/2001003263 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc1621d0 con 0x7f27cc06d610 2023-11-28T18:23:55.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f580a955640 1 -- 172.21.15.105:0/98219077 wait complete. 2023-11-28T18:23:55.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f3920b1b640 1 -- 172.21.15.105:0/3220517206 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f391c160530 msgr2=0x7f391c160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:55.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f3921b1d640 1 --2- 172.21.15.105:0/3220517206 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f391c160530 0x7f391c160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:55.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f3920b1b640 1 --2- 172.21.15.105:0/3220517206 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f391c160530 0x7f391c160950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:55.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f3920b1b640 1 -- 172.21.15.105:0/3220517206 >> v2:172.21.15.105:3300/0 conn(0x7f391c0d10e0 msgr2=0x7f391c0d3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:55.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f3920b1b640 1 --2- 172.21.15.105:0/3220517206 >> v2:172.21.15.105:3300/0 conn(0x7f391c0d10e0 0x7f391c0d3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:55.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f3920b1b640 1 -- 172.21.15.105:0/3220517206 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391c0d04b0 con 0x7f391c0b4140 2023-11-28T18:23:55.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f0299580640 1 -- start start 2023-11-28T18:23:55.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f392131c640 1 --2- 172.21.15.105:0/3220517206 >> v2:172.21.15.105:3300/0 conn(0x7f391c0d10e0 0x7f391c0d3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:55.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f3921b1d640 1 --2- 172.21.15.105:0/3220517206 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f391c160530 0x7f391c160950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:55.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7fa8db59e640 1 Processor -- start 2023-11-28T18:23:55.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7fa8db59e640 1 -- start start 2023-11-28T18:23:55.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f0299580640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f02940cbcb0 0x7f0294161780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:55.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f0299580640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0294160cc0 con 0x7f02940ad130 2023-11-28T18:23:55.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f0299580640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02940ce5b0 con 0x7f02940ad510 2023-11-28T18:23:55.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f0299580640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0294161cc0 con 0x7f02940cbcb0 2023-11-28T18:23:55.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.971+0000 7f580a955640 1 Processor -- start 2023-11-28T18:23:55.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f02937fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f02940ad510 0x7f02940cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:47716/0 (socket says 172.21.15.105:47716) 2023-11-28T18:23:55.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f02937fe640 1 -- 172.21.15.105:0/2565503338 learned_addr learned my addr 172.21.15.105:0/2565503338 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:55.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f580a955640 1 -- start start 2023-11-28T18:23:55.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f02927fc640 1 --2- 172.21.15.105:0/2565503338 >> v2:172.21.15.105:3300/0 conn(0x7f02940cbcb0 0x7f0294161780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:55.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f0291ffb640 1 -- 172.21.15.105:0/2565503338 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1754697032 0 0) 0x7f02940ce5b0 con 0x7f02940ad510 2023-11-28T18:23:55.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f0291ffb640 1 -- 172.21.15.105:0/2565503338 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f027c003530 con 0x7f02940ad510 2023-11-28T18:23:55.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7fa8db59e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8dc0b2a80 0x7fa8dc15ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:55.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7fa8db59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8dc0cf6d0 0x7fa8dc156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:55.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7fa8db59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8dc15d000 0x7fa8dc0b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:55.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7fa8db59e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc150b40 con 0x7fa8dc0cf6d0 2023-11-28T18:23:55.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7fa8db59e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc0cbec0 con 0x7fa8dc15d000 2023-11-28T18:23:55.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7fa8db59e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc15bf70 con 0x7fa8dc0b2a80 2023-11-28T18:23:55.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f27d1db3640 1 --2- 172.21.15.105:0/2001003263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27cc06d610 0x7f27cc0cdff0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f27cc06fa90 tx=0x7f27c801c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76287e0fe0386212 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:55.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7fa8da59c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8dc0b2a80 0x7fa8dc15ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:55.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7fa8da59c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8dc0b2a80 0x7fa8dc15ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:37958/0 (socket says 172.21.15.105:37958) 2023-11-28T18:23:55.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7fa8da59c640 1 -- 172.21.15.105:0/3233716316 learned_addr learned my addr 172.21.15.105:0/3233716316 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:55.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7fa8dad9d640 1 --2- 172.21.15.105:0/3233716316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8dc15d000 0x7fa8dc0b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:55.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f580a955640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58040abd80 0x7f580414c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:55.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f27d0db1640 1 -- 172.21.15.105:0/2001003263 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c80ab650 con 0x7f27cc06d610 2023-11-28T18:23:55.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f27d0db1640 1 -- 172.21.15.105:0/2001003263 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c80ab7b0 con 0x7f27cc06d610 2023-11-28T18:23:55.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f27d0db1640 1 -- 172.21.15.105:0/2001003263 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c80ab650 con 0x7f27cc06d610 2023-11-28T18:23:55.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f3920b1b640 1 --2- 172.21.15.105:0/3220517206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391c0b4140 0x7f391c0d14c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f391c0d0b70 tx=0x7f390c01c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dcbbf121fac7589 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:55.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f3913fff640 1 -- 172.21.15.105:0/3220517206 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c0ab600 con 0x7f391c0b4140 2023-11-28T18:23:55.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f3913fff640 1 -- 172.21.15.105:0/3220517206 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390c0a7050 con 0x7f391c0b4140 2023-11-28T18:23:55.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f3913fff640 1 -- 172.21.15.105:0/3220517206 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c0ab980 con 0x7f391c0b4140 2023-11-28T18:23:55.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f02927fc640 1 -- 172.21.15.105:0/2565503338 >> v1:172.21.15.148:6789/0 conn(0x7f02940ad510 legacy=0x7f02940cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:55.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f580a955640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58040b13e0 0x7f58041514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:55.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f580a955640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58040cb5a0 0x7f58041547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:55.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f580a955640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580415b510 con 0x7f58040abd80 2023-11-28T18:23:55.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f580a955640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580415b7f0 con 0x7f58040cb5a0 2023-11-28T18:23:55.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f580a955640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580415bad0 con 0x7f58040b13e0 2023-11-28T18:23:55.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f5808ef2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58040cb5a0 0x7f58041547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:55.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f5808ef2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58040cb5a0 0x7f58041547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34380/0 (socket says 172.21.15.105:34380) 2023-11-28T18:23:55.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f5808ef2640 1 -- 172.21.15.105:0/1693616941 learned_addr learned my addr 172.21.15.105:0/1693616941 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:55.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f58037fe640 1 --2- 172.21.15.105:0/1693616941 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58040b13e0 0x7f58041514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7fa8da59c640 1 -- 172.21.15.105:0/3233716316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8dc15d000 msgr2=0x7fa8dc0b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7fa8da59c640 1 --2- 172.21.15.105:0/3233716316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8dc15d000 0x7fa8dc0b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f27d4017640 1 -- 172.21.15.105:0/2001003263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27cc06d610 msgr2=0x7f27cc0cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f27d4017640 1 --2- 172.21.15.105:0/2001003263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27cc06d610 0x7f27cc0cdff0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f27cc06fa90 tx=0x7f27c801c430 comp rx=0 tx=0).stop 2023-11-28T18:23:56.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7fa8da59c640 1 -- 172.21.15.105:0/3233716316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8dc0cf6d0 msgr2=0x7fa8dc156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f27d4017640 1 -- 172.21.15.105:0/2001003263 shutdown_connections 2023-11-28T18:23:56.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f27d4017640 1 --2- 172.21.15.105:0/2001003263 >> v2:172.21.15.105:3300/0 conn(0x7f27cc0ae100 0x7f27cc0cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f27d4017640 1 --2- 172.21.15.105:0/2001003263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27cc06d610 0x7f27cc0cdff0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f27d4017640 1 -- 172.21.15.105:0/2001003263 >> 172.21.15.105:0/2001003263 conn(0x7f27cc0ceb90 msgr2=0x7f27cc0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:56.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f5803fff640 1 --2- 172.21.15.105:0/1693616941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58040abd80 0x7f580414c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7fa8da59c640 1 --2- 172.21.15.105:0/3233716316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8dc0cf6d0 0x7fa8dc156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f5808ef2640 1 -- 172.21.15.105:0/1693616941 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58040b13e0 msgr2=0x7f58041514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f5808ef2640 1 --2- 172.21.15.105:0/1693616941 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58040b13e0 0x7f58041514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f5808ef2640 1 -- 172.21.15.105:0/1693616941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58040abd80 msgr2=0x7f580414c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f5808ef2640 1 --2- 172.21.15.105:0/1693616941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58040abd80 0x7f580414c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f5808ef2640 1 -- 172.21.15.105:0/1693616941 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f80094d0 con 0x7f58040cb5a0 2023-11-28T18:23:56.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f3923580640 1 -- 172.21.15.105:0/3220517206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391c0b4140 msgr2=0x7f391c0d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f3923580640 1 --2- 172.21.15.105:0/3220517206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391c0b4140 0x7f391c0d14c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f391c0d0b70 tx=0x7f390c01c440 comp rx=0 tx=0).stop 2023-11-28T18:23:56.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7f5803fff640 1 --2- 172.21.15.105:0/1693616941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58040abd80 0x7f580414c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:56.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7f27d4017640 1 -- 172.21.15.105:0/2001003263 shutdown_connections 2023-11-28T18:23:56.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7f3923580640 1 -- 172.21.15.105:0/3220517206 shutdown_connections 2023-11-28T18:23:56.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7f3923580640 1 --2- 172.21.15.105:0/3220517206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391c0b4140 0x7f391c0d14c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7f3923580640 1 --2- 172.21.15.105:0/3220517206 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f391c160530 0x7f391c160950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7f3923580640 1 --2- 172.21.15.105:0/3220517206 >> v2:172.21.15.105:3300/0 conn(0x7f391c0d10e0 0x7f391c0d3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7f3923580640 1 -- 172.21.15.105:0/3220517206 >> 172.21.15.105:0/3220517206 conn(0x7f391c0d30a0 msgr2=0x7f391c0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:56.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7fa8da59c640 1 -- 172.21.15.105:0/3233716316 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8cc009420 con 0x7fa8dc0b2a80 2023-11-28T18:23:56.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7f5808ef2640 1 --2- 172.21.15.105:0/1693616941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58040cb5a0 0x7f58041547f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f57f4013b90 tx=0x7f57f4013c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7f5800ff9640 1 -- 172.21.15.105:0/1693616941 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4014d90 con 0x7f58040cb5a0 2023-11-28T18:23:56.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f02927fc640 1 -- 172.21.15.105:0/2565503338 >> v1:172.21.15.105:6789/0 conn(0x7f02940ad130 legacy=0x7f02940cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f02927fc640 1 -- 172.21.15.105:0/2565503338 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02940ce890 con 0x7f02940cbcb0 2023-11-28T18:23:56.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7fa8da59c640 1 --2- 172.21.15.105:0/3233716316 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8dc0b2a80 0x7fa8dc15ba30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00e980 tx=0x7fa8cc00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7f5800ff9640 1 -- 172.21.15.105:0/1693616941 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f40110d0 con 0x7f58040cb5a0 2023-11-28T18:23:56.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7f5800ff9640 1 -- 172.21.15.105:0/1693616941 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4011480 con 0x7f58040cb5a0 2023-11-28T18:23:56.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7f580a955640 1 -- 172.21.15.105:0/1693616941 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f580415bdb0 con 0x7f58040cb5a0 2023-11-28T18:23:56.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7f580a955640 1 -- 172.21.15.105:0/1693616941 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f580414cdd0 con 0x7f58040cb5a0 2023-11-28T18:23:56.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7fa8d37fe640 1 -- 172.21.15.105:0/3233716316 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc00ca00 con 0x7fa8dc0b2a80 2023-11-28T18:23:56.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7fa8d37fe640 1 -- 172.21.15.105:0/3233716316 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8cc00cb60 con 0x7fa8dc0b2a80 2023-11-28T18:23:56.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7fa8d37fe640 1 -- 172.21.15.105:0/3233716316 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc00ced0 con 0x7fa8dc0b2a80 2023-11-28T18:23:56.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7f27d4017640 1 -- 172.21.15.105:0/2001003263 wait complete. 2023-11-28T18:23:56.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7f27d4017640 1 Processor -- start 2023-11-28T18:23:56.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7f3923580640 1 -- 172.21.15.105:0/3220517206 shutdown_connections 2023-11-28T18:23:56.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.975+0000 7f0291ffb640 1 -- 172.21.15.105:0/2565503338 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570650421 0 0) 0x7f0294160cc0 con 0x7f02940ad130 2023-11-28T18:23:56.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7f6088f2a640 1 Processor -- start 2023-11-28T18:23:56.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7f6088f2a640 1 -- start start 2023-11-28T18:23:56.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7f27d4017640 1 -- start start 2023-11-28T18:23:56.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7f02927fc640 1 --2- 172.21.15.105:0/2565503338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02940cbcb0 0x7f0294161780 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f02880097c0 tx=0x7f028801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d083dc9fc008969 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7fa8db59e640 1 -- 172.21.15.105:0/3233716316 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d4009b30 con 0x7fa8dc0b2a80 2023-11-28T18:23:56.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f0291ffb640 1 -- 172.21.15.105:0/2565503338 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02880ac620 con 0x7f02940cbcb0 2023-11-28T18:23:56.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7f3923580640 1 -- 172.21.15.105:0/3220517206 wait complete. 2023-11-28T18:23:56.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f0291ffb640 1 -- 172.21.15.105:0/2565503338 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02880ac780 con 0x7f02940cbcb0 2023-11-28T18:23:56.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f3923580640 1 Processor -- start 2023-11-28T18:23:56.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f0291ffb640 1 -- 172.21.15.105:0/2565503338 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02880ac620 con 0x7f02940cbcb0 2023-11-28T18:23:56.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f6088f2a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f608406e2c0 0x7f60840abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.979+0000 7f580a955640 1 -- 172.21.15.105:0/1693616941 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f580415c2d0 con 0x7f58040cb5a0 2023-11-28T18:23:56.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7fa8db59e640 1 -- 172.21.15.105:0/3233716316 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8dc0b84c0 con 0x7fa8dc0b2a80 2023-11-28T18:23:56.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f3923580640 1 -- start start 2023-11-28T18:23:56.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f5800ff9640 1 -- 172.21.15.105:0/1693616941 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f57f400f850 con 0x7f58040cb5a0 2023-11-28T18:23:56.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f6088f2a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f608406bf50 con 0x7f608415d900 2023-11-28T18:23:56.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f27d4017640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27cc06d610 0x7f27cc156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f0299580640 1 -- 172.21.15.105:0/2565503338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02940cbcb0 msgr2=0x7f0294161780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f6082d9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f60841592f0 0x7f608406d390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:47720/0 (socket says 172.21.15.105:47720) 2023-11-28T18:23:56.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f27d4017640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27cc0ae100 0x7f27cc0b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f0299580640 1 --2- 172.21.15.105:0/2565503338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02940cbcb0 0x7f0294161780 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f02880097c0 tx=0x7f028801c3c0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f27d4017640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27cc0ce530 0x7f27cc202760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f6088f2a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f608406c7f0 con 0x7f60841592f0 2023-11-28T18:23:56.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f6088f2a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60840cdae0 con 0x7f608406e2c0 2023-11-28T18:23:56.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f6082d9d640 1 -- 172.21.15.105:0/3289971808 learned_addr learned my addr 172.21.15.105:0/3289971808 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:56.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f0299580640 1 -- 172.21.15.105:0/2565503338 shutdown_connections 2023-11-28T18:23:56.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f0299580640 1 --2- 172.21.15.105:0/2565503338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02940cbcb0 0x7f0294161780 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f0299580640 1 -- 172.21.15.105:0/2565503338 >> 172.21.15.105:0/2565503338 conn(0x7f02940b08c0 msgr2=0x7f02940b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:56.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f27d1db3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27cc06d610 0x7f27cc156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7fa8db59e640 1 -- 172.21.15.105:0/3233716316 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8dc074580 con 0x7fa8dc0b2a80 2023-11-28T18:23:56.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f27d4017640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc099dc0 con 0x7f27cc0ce530 2023-11-28T18:23:56.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f27d4017640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc09a0a0 con 0x7f27cc0ae100 2023-11-28T18:23:56.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f27d4017640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc09a380 con 0x7f27cc06d610 2023-11-28T18:23:56.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f27d15b2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27cc0ae100 0x7f27cc0b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f3923580640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391c0b4140 0x7f391c15ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f27d15b2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27cc0ae100 0x7f27cc0b3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34394/0 (socket says 172.21.15.105:34394) 2023-11-28T18:23:56.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f3923580640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f391c0d10e0 0x7f391c0c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f3923580640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f391c160530 0x7f391c0c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f5800ff9640 1 --2- 172.21.15.105:0/1693616941 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f57f00668d0 0x7f57f0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f5800ff9640 1 -- 172.21.15.105:0/1693616941 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f57f408c2b0 con 0x7f58040cb5a0 2023-11-28T18:23:56.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f27d15b2640 1 -- 172.21.15.105:0/1333172192 learned_addr learned my addr 172.21.15.105:0/1333172192 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:56.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f27d25b4640 1 --2- 172.21.15.105:0/1333172192 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27cc0ce530 0x7f27cc202760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f3920b1b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f391c0d10e0 0x7f391c0c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f392131c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391c0b4140 0x7f391c15ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f392131c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391c0b4140 0x7f391c15ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34396/0 (socket says 172.21.15.105:34396) 2023-11-28T18:23:56.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f3920b1b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f391c0d10e0 0x7f391c0c60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:37996/0 (socket says 172.21.15.105:37996) 2023-11-28T18:23:56.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f3921b1d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f391c160530 0x7f391c0c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f3921b1d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f391c160530 0x7f391c0c6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55264/0 (socket says 172.21.15.105:55264) 2023-11-28T18:23:56.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f608259c640 1 --2- 172.21.15.105:0/3289971808 >> v2:172.21.15.105:3300/0 conn(0x7f608406e2c0 0x7f60840abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f5803fff640 1 --2- 172.21.15.105:0/1693616941 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f57f00668d0 0x7f57f0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f27d15b2640 1 -- 172.21.15.105:0/1333172192 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27cc06d610 msgr2=0x7f27cc156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f27d15b2640 1 --2- 172.21.15.105:0/1333172192 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27cc06d610 0x7f27cc156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f27d15b2640 1 -- 172.21.15.105:0/1333172192 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27cc0ce530 msgr2=0x7f27cc202760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f27d15b2640 1 --2- 172.21.15.105:0/1333172192 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27cc0ce530 0x7f27cc202760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f27d15b2640 1 -- 172.21.15.105:0/1333172192 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c8009420 con 0x7f27cc0ae100 2023-11-28T18:23:56.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f392131c640 1 -- 172.21.15.105:0/1589563894 learned_addr learned my addr 172.21.15.105:0/1589563894 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:56.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.983+0000 7f029edb8640 1 Processor -- start 2023-11-28T18:23:56.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f27d15b2640 1 --2- 172.21.15.105:0/1333172192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27cc0ae100 0x7f27cc0b3d90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00c420 tx=0x7f27bc00c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f3923580640 1 -- 172.21.15.105:0/1589563894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c0cbeb0 con 0x7f391c160530 2023-11-28T18:23:56.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f3923580640 1 -- 172.21.15.105:0/1589563894 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c0cc190 con 0x7f391c0b4140 2023-11-28T18:23:56.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f3923580640 1 -- 172.21.15.105:0/1589563894 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f391c0cc470 con 0x7f391c0d10e0 2023-11-28T18:23:56.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f029edb8640 1 -- start start 2023-11-28T18:23:56.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f6081d9b640 1 -- 172.21.15.105:0/3289971808 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 60715847 0 0) 0x7f608406bf50 con 0x7f608415d900 2023-11-28T18:23:56.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f3921b1d640 1 -- 172.21.15.105:0/1589563894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f391c0d10e0 msgr2=0x7f391c0c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f3921b1d640 1 --2- 172.21.15.105:0/1589563894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f391c0d10e0 0x7f391c0c60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f3921b1d640 1 -- 172.21.15.105:0/1589563894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391c0b4140 msgr2=0x7f391c15ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f3921b1d640 1 --2- 172.21.15.105:0/1589563894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391c0b4140 0x7f391c15ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f3921b1d640 1 -- 172.21.15.105:0/1589563894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c0094d0 con 0x7f391c160530 2023-11-28T18:23:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f5803fff640 1 --2- 172.21.15.105:0/1693616941 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f57f00668d0 0x7f57f0068d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0074c0 tx=0x7f57fc00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f6081d9b640 1 -- 172.21.15.105:0/3289971808 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6070003530 con 0x7f608415d900 2023-11-28T18:23:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f3921b1d640 1 --2- 172.21.15.105:0/1589563894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f391c160530 0x7f391c0c6610 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f391400cc90 tx=0x7f391400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7fa8d37fe640 1 -- 172.21.15.105:0/3233716316 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa8cc013470 con 0x7fa8dc0b2a80 2023-11-28T18:23:56.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f3911ffb640 1 -- 172.21.15.105:0/1589563894 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391400cea0 con 0x7f391c160530 2023-11-28T18:23:56.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f6081d9b640 1 -- 172.21.15.105:0/3289971808 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1073744052 0 0) 0x7f6070003530 con 0x7f608415d900 2023-11-28T18:23:56.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7fa8d37fe640 1 --2- 172.21.15.105:0/3233716316 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa8bc066920 0x7fa8bc068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f3911ffb640 1 -- 172.21.15.105:0/1589563894 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391400f6a0 con 0x7f391c160530 2023-11-28T18:23:56.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f3911ffb640 1 -- 172.21.15.105:0/1589563894 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391400fa50 con 0x7f391c160530 2023-11-28T18:23:56.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f3923580640 1 -- 172.21.15.105:0/1589563894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391c0cc750 con 0x7f391c160530 2023-11-28T18:23:56.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7fa8d37fe640 1 -- 172.21.15.105:0/3233716316 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa8cc084830 con 0x7fa8dc0b2a80 2023-11-28T18:23:56.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f608259c640 1 -- 172.21.15.105:0/3289971808 >> v1:172.21.15.148:6789/0 conn(0x7f60841592f0 legacy=0x7f608406d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f608259c640 1 -- 172.21.15.105:0/3289971808 >> v1:172.21.15.105:6789/0 conn(0x7f608415d900 legacy=0x7f608406cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f608259c640 1 -- 172.21.15.105:0/3289971808 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608406c1a0 con 0x7f608406e2c0 2023-11-28T18:23:56.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7fa8d1d9b640 1 --2- 172.21.15.105:0/3233716316 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa8bc066920 0x7fa8bc068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f3923580640 1 -- 172.21.15.105:0/1589563894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391c15b370 con 0x7f391c160530 2023-11-28T18:23:56.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f5800ff9640 1 -- 172.21.15.105:0/1693616941 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f580415c2d0 con 0x7f58040cb5a0 2023-11-28T18:23:56.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f029edb8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f02980abd80 0x7f02980cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f029edb8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02980cbae0 0x7f029806c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f029edb8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02981594a0 con 0x7f02981590c0 2023-11-28T18:23:56.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f029edb8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02980cbf90 con 0x7f02980abd80 2023-11-28T18:23:56.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f029edb8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02980cea00 con 0x7f02980cbae0 2023-11-28T18:23:56.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f029d355640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02980cbae0 0x7f029806c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f029d355640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02980cbae0 0x7f029806c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:38012/0 (socket says 172.21.15.105:38012) 2023-11-28T18:23:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f029d355640 1 -- 172.21.15.105:0/2297629210 learned_addr learned my addr 172.21.15.105:0/2297629210 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7fa8d1d9b640 1 --2- 172.21.15.105:0/3233716316 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa8bc066920 0x7fa8bc068de0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc1606e0 tx=0x7fa8c4007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f029d355640 1 -- 172.21.15.105:0/2297629210 >> v2:172.21.15.105:3300/0 conn(0x7f02980abd80 msgr2=0x7f02980cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f029d355640 1 --2- 172.21.15.105:0/2297629210 >> v2:172.21.15.105:3300/0 conn(0x7f02980abd80 0x7f02980cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f029d355640 1 -- 172.21.15.105:0/2297629210 >> v1:172.21.15.148:6789/0 conn(0x7f02981590c0 legacy=0x7f02980ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f029d355640 1 -- 172.21.15.105:0/2297629210 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029815cd80 con 0x7f02980cbae0 2023-11-28T18:23:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f0299580640 1 -- 172.21.15.105:0/2565503338 shutdown_connections 2023-11-28T18:23:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f0299580640 1 -- 172.21.15.105:0/2565503338 wait complete. 2023-11-28T18:23:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f0299580640 1 Processor -- start 2023-11-28T18:23:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.987+0000 7f0299580640 1 -- start start 2023-11-28T18:23:56.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.991+0000 7f27c27fc640 1 -- 172.21.15.105:0/1333172192 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc005cf0 con 0x7f27cc0ae100 2023-11-28T18:23:56.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.991+0000 7f27c27fc640 1 -- 172.21.15.105:0/1333172192 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27bc005e50 con 0x7f27cc0ae100 2023-11-28T18:23:56.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.991+0000 7f27c27fc640 1 -- 172.21.15.105:0/1333172192 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc0072d0 con 0x7f27cc0ae100 2023-11-28T18:23:56.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.991+0000 7f27d4017640 1 -- 172.21.15.105:0/1333172192 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27cc09a660 con 0x7f27cc0ae100 2023-11-28T18:23:56.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.991+0000 7f27d4017640 1 -- 172.21.15.105:0/1333172192 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27cc0b42d0 con 0x7f27cc0ae100 2023-11-28T18:23:56.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.991+0000 7f0299580640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02940ad130 0x7f029409b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.991+0000 7f608259c640 1 --2- 172.21.15.105:0/3289971808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f608406e2c0 0x7f60840abd80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f607400a8f0 tx=0x7f607401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5347212778bdc08 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.991+0000 7f0299580640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02940ad510 0x7f0294153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.991+0000 7f3923580640 1 -- 172.21.15.105:0/1589563894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391c0d2b10 con 0x7f391c160530 2023-11-28T18:23:56.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.991+0000 7f0299580640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02940cbcb0 0x7f029420a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.991+0000 7f6081d9b640 1 -- 172.21.15.105:0/3289971808 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60740a6040 con 0x7f608406e2c0 2023-11-28T18:23:56.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.991+0000 7f02927fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02940ad510 0x7f0294153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.991+0000 7f02927fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02940ad510 0x7f0294153e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:34408/0 (socket says 172.21.15.105:34408) 2023-11-28T18:23:56.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.991+0000 7fa8d37fe640 1 -- 172.21.15.105:0/3233716316 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa8cc0519a0 con 0x7fa8dc0b2a80 2023-11-28T18:23:56.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.991+0000 7f3911ffb640 1 -- 172.21.15.105:0/1589563894 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f391400b040 con 0x7f391c160530 2023-11-28T18:23:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.991+0000 7f0299580640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029409c340 con 0x7f02940cbcb0 2023-11-28T18:23:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.991+0000 7f0299580640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029409b6c0 con 0x7f02940ad510 2023-11-28T18:23:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.991+0000 7f0299580640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f029409b9a0 con 0x7f02940ad130 2023-11-28T18:23:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.991+0000 7f02927fc640 1 -- 172.21.15.105:0/1649893005 learned_addr learned my addr 172.21.15.105:0/1649893005 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:56.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.991+0000 7f6081d9b640 1 -- 172.21.15.105:0/3289971808 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60740aa4c0 con 0x7f608406e2c0 2023-11-28T18:23:56.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f02927fc640 1 -- 172.21.15.105:0/1649893005 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02940ad130 msgr2=0x7f029409b070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T18:23:56.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f02927fc640 1 --2- 172.21.15.105:0/1649893005 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02940ad130 0x7f029409b070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f02927fc640 1 -- 172.21.15.105:0/1649893005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02940cbcb0 msgr2=0x7f029420a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f02937fe640 1 --2- 172.21.15.105:0/1649893005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02940cbcb0 0x7f029420a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f02927fc640 1 --2- 172.21.15.105:0/1649893005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02940cbcb0 0x7f029420a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f029d355640 1 --2- 172.21.15.105:0/2297629210 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02980cbae0 0x7f029806c050 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f02880091e0 tx=0x7f028801d6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d6ac86a656e372d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f6081d9b640 1 -- 172.21.15.105:0/3289971808 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c001680 con 0x7f608406e2c0 2023-11-28T18:23:56.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f3911ffb640 1 --2- 172.21.15.105:0/1589563894 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f39000668d0 0x7f3900068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f3911ffb640 1 -- 172.21.15.105:0/1589563894 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3914084f80 con 0x7f391c160530 2023-11-28T18:23:56.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f6088f2a640 1 -- 172.21.15.105:0/3289971808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f608406e2c0 msgr2=0x7f60840abd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f6088f2a640 1 --2- 172.21.15.105:0/3289971808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f608406e2c0 0x7f60840abd80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f607400a8f0 tx=0x7f607401c470 comp rx=0 tx=0).stop 2023-11-28T18:23:56.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f02977fe640 1 -- 172.21.15.105:0/2297629210 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02880a7690 con 0x7f02980cbae0 2023-11-28T18:23:56.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f392131c640 1 --2- 172.21.15.105:0/1589563894 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f39000668d0 0x7f3900068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f02977fe640 1 -- 172.21.15.105:0/2297629210 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02880a77f0 con 0x7f02980cbae0 2023-11-28T18:23:56.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f02977fe640 1 -- 172.21.15.105:0/2297629210 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02880a7b80 con 0x7f02980cbae0 2023-11-28T18:23:56.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f02927fc640 1 -- 172.21.15.105:0/1649893005 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0288009420 con 0x7f02940ad510 2023-11-28T18:23:56.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f02937fe640 1 --2- 172.21.15.105:0/1649893005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02940cbcb0 0x7f029420a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:56.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f27c27fc640 1 -- 172.21.15.105:0/1333172192 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f27bc007430 con 0x7f27cc0ae100 2023-11-28T18:23:56.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f27d4017640 1 -- 172.21.15.105:0/1333172192 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b4005260 con 0x7f27cc0ae100 2023-11-28T18:23:56.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f02927fc640 1 --2- 172.21.15.105:0/1649893005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02940ad510 0x7f0294153e60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0288019ff0 tx=0x7f0288019160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f027b7fe640 1 -- 172.21.15.105:0/1649893005 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02880ac620 con 0x7f02940ad510 2023-11-28T18:23:56.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f27c27fc640 1 --2- 172.21.15.105:0/1333172192 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27b00669a0 0x7f27b0068e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f392131c640 1 --2- 172.21.15.105:0/1589563894 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f39000668d0 0x7f3900068d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3918009990 tx=0x7f3918008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f0299580640 1 -- 172.21.15.105:0/1649893005 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029409bc80 con 0x7f02940ad510 2023-11-28T18:23:56.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f0299580640 1 -- 172.21.15.105:0/1649893005 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02941543a0 con 0x7f02940ad510 2023-11-28T18:23:56.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f029edb8640 1 -- 172.21.15.105:0/2297629210 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02980cbae0 msgr2=0x7f029806c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f029edb8640 1 --2- 172.21.15.105:0/2297629210 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02980cbae0 0x7f029806c050 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f02880091e0 tx=0x7f028801d6e0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f029edb8640 1 -- 172.21.15.105:0/2297629210 shutdown_connections 2023-11-28T18:23:56.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f029edb8640 1 --2- 172.21.15.105:0/2297629210 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02980cbae0 0x7f029806c050 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f029edb8640 1 --2- 172.21.15.105:0/2297629210 >> v2:172.21.15.105:3300/0 conn(0x7f02980abd80 0x7f02980cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f029edb8640 1 -- 172.21.15.105:0/2297629210 >> 172.21.15.105:0/2297629210 conn(0x7f02980b08c0 msgr2=0x7f02980b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:56.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f6088f2a640 1 -- 172.21.15.105:0/3289971808 shutdown_connections 2023-11-28T18:23:56.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f6088f2a640 1 --2- 172.21.15.105:0/3289971808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f608406e2c0 0x7f60840abd80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f6088f2a640 1 -- 172.21.15.105:0/3289971808 >> 172.21.15.105:0/3289971808 conn(0x7f60840ceb90 msgr2=0x7f60840cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:56.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f27c27fc640 1 -- 172.21.15.105:0/1333172192 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f27bc083c20 con 0x7f27cc0ae100 2023-11-28T18:23:56.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f27d1db3640 1 --2- 172.21.15.105:0/1333172192 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27b00669a0 0x7f27b0068e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f027b7fe640 1 -- 172.21.15.105:0/1649893005 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02880ac8f0 con 0x7f02940ad510 2023-11-28T18:23:56.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.995+0000 7f027b7fe640 1 -- 172.21.15.105:0/1649893005 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02880acc60 con 0x7f02940ad510 2023-11-28T18:23:56.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.999+0000 7f6088f2a640 1 -- 172.21.15.105:0/3289971808 shutdown_connections 2023-11-28T18:23:56.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.999+0000 7f6088f2a640 1 -- 172.21.15.105:0/3289971808 wait complete. 2023-11-28T18:23:56.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.999+0000 7f029edb8640 1 -- 172.21.15.105:0/2297629210 shutdown_connections 2023-11-28T18:23:56.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.999+0000 7f029edb8640 1 -- 172.21.15.105:0/2297629210 wait complete. 2023-11-28T18:23:56.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.999+0000 7f02797fa640 1 -- 172.21.15.105:0/1649893005 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f029409c480 con 0x7f02940ad510 2023-11-28T18:23:56.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.999+0000 7f27d1db3640 1 --2- 172.21.15.105:0/1333172192 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27b00669a0 0x7f27b0068e60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f27c80af000 tx=0x7f27c80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.999+0000 7f6088f2a640 1 Processor -- start 2023-11-28T18:23:56.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.999+0000 7f029edb8640 1 Processor -- start 2023-11-28T18:23:56.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.999+0000 7f27c27fc640 1 -- 172.21.15.105:0/1333172192 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f27bc050c00 con 0x7f27cc0ae100 2023-11-28T18:23:56.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.999+0000 7f029edb8640 1 -- start start 2023-11-28T18:23:56.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:55.999+0000 7f6088f2a640 1 -- start start 2023-11-28T18:23:56.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f6088f2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f608406e2c0 0x7f60841f7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f6088f2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60841592f0 0x7f60841fa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f6088f2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f608415d900 0x7f60841fabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f6088f2a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60841f8150 con 0x7f608406e2c0 2023-11-28T18:23:56.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f6088f2a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6084099a50 con 0x7f608415d900 2023-11-28T18:23:56.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f6088f2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60841fb1b0 con 0x7f60841592f0 2023-11-28T18:23:56.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f029edb8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02980abd80 0x7f02980b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f608259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f608406e2c0 0x7f60841f7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f6079d9b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60841592f0 0x7f60841fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f6079d9b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60841592f0 0x7f60841fa680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:38022/0 (socket says 172.21.15.105:38022) 2023-11-28T18:23:56.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f6079d9b640 1 -- 172.21.15.105:0/3666138437 learned_addr learned my addr 172.21.15.105:0/3666138437 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:56.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f029edb8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02980cbae0 0x7f029814ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f029edb8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02981590c0 0x7f02981582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f029edb8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029815a590 con 0x7f02980cbae0 2023-11-28T18:23:56.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f029edb8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029815a870 con 0x7f02981590c0 2023-11-28T18:23:56.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f029edb8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f029815ab50 con 0x7f02980abd80 2023-11-28T18:23:56.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f3911ffb640 1 -- 172.21.15.105:0/1589563894 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3914051ee0 con 0x7f391c160530 2023-11-28T18:23:56.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f6082d9d640 1 --2- 172.21.15.105:0/3666138437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f608415d900 0x7f60841fabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f027b7fe640 1 -- 172.21.15.105:0/1649893005 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f02880b3050 con 0x7f02940ad510 2023-11-28T18:23:56.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f6079d9b640 1 -- 172.21.15.105:0/3666138437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f608415d900 msgr2=0x7f60841fabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f6079d9b640 1 --2- 172.21.15.105:0/3666138437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f608415d900 0x7f60841fabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f6079d9b640 1 -- 172.21.15.105:0/3666138437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f608406e2c0 msgr2=0x7f60841f7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f6079d9b640 1 --2- 172.21.15.105:0/3666138437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f608406e2c0 0x7f60841f7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f6079d9b640 1 -- 172.21.15.105:0/3666138437 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607400a520 con 0x7f60841592f0 2023-11-28T18:23:56.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f6082d9d640 1 --2- 172.21.15.105:0/3666138437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f608415d900 0x7f60841fabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:56.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f608259c640 1 --2- 172.21.15.105:0/3666138437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f608406e2c0 0x7f60841f7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:56.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f029cb54640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02980abd80 0x7f02980b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f029cb54640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02980abd80 0x7f02980b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:38024/0 (socket says 172.21.15.105:38024) 2023-11-28T18:23:56.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f029cb54640 1 -- 172.21.15.105:0/4285430364 learned_addr learned my addr 172.21.15.105:0/4285430364 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:56.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f029d355640 1 --2- 172.21.15.105:0/4285430364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02981590c0 0x7f02981582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f029cb54640 1 -- 172.21.15.105:0/4285430364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02981590c0 msgr2=0x7f02981582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f029cb54640 1 --2- 172.21.15.105:0/4285430364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02981590c0 0x7f02981582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f029cb54640 1 -- 172.21.15.105:0/4285430364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02980cbae0 msgr2=0x7f029814ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f0297fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02980cbae0 0x7f029814ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.007+0000 7f029cb54640 1 --2- 172.21.15.105:0/4285430364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02980cbae0 0x7f029814ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.007+0000 7f029cb54640 1 -- 172.21.15.105:0/4285430364 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0288009d60 con 0x7f02980abd80 2023-11-28T18:23:56.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.007+0000 7f0297fff640 1 --2- 172.21.15.105:0/4285430364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02980cbae0 0x7f029814ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:23:56.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.003+0000 7f027b7fe640 1 --2- 172.21.15.105:0/1649893005 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f02800668d0 0x7f0280068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.007+0000 7f027b7fe640 1 -- 172.21.15.105:0/1649893005 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f028811ed20 con 0x7f02940ad510 2023-11-28T18:23:56.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.007+0000 7f6079d9b640 1 --2- 172.21.15.105:0/3666138437 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60841592f0 0x7f60841fa680 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f606c00c6d0 tx=0x7f606c00cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.007+0000 7f0292ffd640 1 --2- 172.21.15.105:0/1649893005 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f02800668d0 0x7f0280068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.007+0000 7f607b7fe640 1 -- 172.21.15.105:0/3666138437 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c00a840 con 0x7f60841592f0 2023-11-28T18:23:56.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.007+0000 7f607b7fe640 1 -- 172.21.15.105:0/3666138437 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606c00a9a0 con 0x7f60841592f0 2023-11-28T18:23:56.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.007+0000 7f607b7fe640 1 -- 172.21.15.105:0/3666138437 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c00ad90 con 0x7f60841592f0 2023-11-28T18:23:56.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.007+0000 7f6088f2a640 1 -- 172.21.15.105:0/3666138437 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60841fb490 con 0x7f60841592f0 2023-11-28T18:23:56.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.007+0000 7f6088f2a640 1 -- 172.21.15.105:0/3666138437 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6084099f40 con 0x7f60841592f0 2023-11-28T18:23:56.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.007+0000 7f0292ffd640 1 --2- 172.21.15.105:0/1649893005 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f02800668d0 0x7f0280068d90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0284000f70 tx=0x7f02840078b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.007+0000 7f027b7fe640 1 -- 172.21.15.105:0/1649893005 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f02880ebce0 con 0x7f02940ad510 2023-11-28T18:23:56.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.007+0000 7f029cb54640 1 --2- 172.21.15.105:0/4285430364 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02980abd80 0x7f02980b3dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f028c00c3c0 tx=0x7f028c00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.007+0000 7f6088f2a640 1 -- 172.21.15.105:0/3666138437 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6084067570 con 0x7f60841592f0 2023-11-28T18:23:56.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.007+0000 7f02957fa640 1 -- 172.21.15.105:0/4285430364 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c0052d0 con 0x7f02980abd80 2023-11-28T18:23:56.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.007+0000 7f02957fa640 1 -- 172.21.15.105:0/4285430364 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028c005430 con 0x7f02980abd80 2023-11-28T18:23:56.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.007+0000 7f029edb8640 1 -- 172.21.15.105:0/4285430364 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029815ae30 con 0x7f02980abd80 2023-11-28T18:23:56.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.007+0000 7f02957fa640 1 -- 172.21.15.105:0/4285430364 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c003840 con 0x7f02980abd80 2023-11-28T18:23:56.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.007+0000 7f029edb8640 1 -- 172.21.15.105:0/4285430364 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029815b630 con 0x7f02980abd80 2023-11-28T18:23:56.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.011+0000 7f027effd640 1 -- 172.21.15.105:0/4285430364 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f029815b350 con 0x7f02980abd80 2023-11-28T18:23:56.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.011+0000 7f607b7fe640 1 -- 172.21.15.105:0/3666138437 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f606c012420 con 0x7f60841592f0 2023-11-28T18:23:56.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.011+0000 7f607b7fe640 1 --2- 172.21.15.105:0/3666138437 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6064066920 0x7f6064068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.011+0000 7f607b7fe640 1 -- 172.21.15.105:0/3666138437 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f606c055bd0 con 0x7f60841592f0 2023-11-28T18:23:56.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.015+0000 7f608259c640 1 --2- 172.21.15.105:0/3666138437 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6064066920 0x7f6064068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.015+0000 7f607b7fe640 1 -- 172.21.15.105:0/3666138437 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f606c0526a0 con 0x7f60841592f0 2023-11-28T18:23:56.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.015+0000 7f608259c640 1 --2- 172.21.15.105:0/3666138437 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6064066920 0x7f6064068de0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f60740ae000 tx=0x7f60740b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.019+0000 7f02957fa640 1 -- 172.21.15.105:0/4285430364 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f028c0039a0 con 0x7f02980abd80 2023-11-28T18:23:56.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.027+0000 7f02957fa640 1 --2- 172.21.15.105:0/4285430364 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0284066920 0x7f0284068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.027+0000 7f02957fa640 1 -- 172.21.15.105:0/4285430364 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f028c083c90 con 0x7f02980abd80 2023-11-28T18:23:56.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.027+0000 7f02957fa640 1 -- 172.21.15.105:0/4285430364 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f028c0ae320 con 0x7f02980abd80 2023-11-28T18:23:56.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.027+0000 7f0297fff640 1 --2- 172.21.15.105:0/4285430364 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0284066920 0x7f0284068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.035+0000 7f0297fff640 1 --2- 172.21.15.105:0/4285430364 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0284066920 0x7f0284068de0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0298155ee0 tx=0x7f02900073c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.251+0000 7f241a0d6640 1 -- 172.21.15.105:0/2055950018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f23f40057e0 con 0x7f241413fe70 2023-11-28T18:23:56.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.255+0000 7f23fbfff640 1 -- 172.21.15.105:0/2055950018 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2408052250 con 0x7f241413fe70 2023-11-28T18:23:56.260 INFO:teuthology.orchestra.run.smithi105.stdout:55834574858 2023-11-28T18:23:56.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.259+0000 7f241a0d6640 1 -- 172.21.15.105:0/2055950018 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2400066990 msgr2=0x7f2400068e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.259+0000 7f241a0d6640 1 --2- 172.21.15.105:0/2055950018 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2400066990 0x7f2400068e50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2404009790 tx=0x7f2404008040 comp rx=0 tx=0).stop 2023-11-28T18:23:56.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.259+0000 7f241a0d6640 1 -- 172.21.15.105:0/2055950018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f241413fe70 msgr2=0x7f24141f4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.259+0000 7f241a0d6640 1 --2- 172.21.15.105:0/2055950018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f241413fe70 0x7f24141f4fe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f240800c3f0 tx=0x7f240800c8b0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.263+0000 7f241a0d6640 1 -- 172.21.15.105:0/2055950018 shutdown_connections 2023-11-28T18:23:56.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.263+0000 7f241a0d6640 1 --2- 172.21.15.105:0/2055950018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f241413fe70 0x7f24141f4fe0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.263+0000 7f241a0d6640 1 --2- 172.21.15.105:0/2055950018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f241413f6c0 0x7f241413d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.263+0000 7f241a0d6640 1 --2- 172.21.15.105:0/2055950018 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2400066990 0x7f2400068e50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.263+0000 7f241a0d6640 1 --2- 172.21.15.105:0/2055950018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24140cb8e0 0x7f24140c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.263+0000 7f241a0d6640 1 -- 172.21.15.105:0/2055950018 >> 172.21.15.105:0/2055950018 conn(0x7f24140b0720 msgr2=0x7f24140b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:56.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.263+0000 7f241a0d6640 1 -- 172.21.15.105:0/2055950018 shutdown_connections 2023-11-28T18:23:56.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.263+0000 7f241a0d6640 1 -- 172.21.15.105:0/2055950018 wait complete. 2023-11-28T18:23:56.299 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2023-11-28T18:23:56.300 DEBUG:teuthology.parallel:result is None 2023-11-28T18:23:56.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.307+0000 7f580a955640 1 -- 172.21.15.105:0/1693616941 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f580415c2d0 con 0x7f58040cb5a0 2023-11-28T18:23:56.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.307+0000 7f5800ff9640 1 -- 172.21.15.105:0/1693616941 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f580415c2d0 con 0x7f58040cb5a0 2023-11-28T18:23:56.312 INFO:teuthology.orchestra.run.smithi105.stdout:55834574858 2023-11-28T18:23:56.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.311+0000 7f580a955640 1 -- 172.21.15.105:0/1693616941 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f57f00668d0 msgr2=0x7f57f0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.311+0000 7f580a955640 1 --2- 172.21.15.105:0/1693616941 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f57f00668d0 0x7f57f0068d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0074c0 tx=0x7f57fc00b040 comp rx=0 tx=0).stop 2023-11-28T18:23:56.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.311+0000 7f580a955640 1 -- 172.21.15.105:0/1693616941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58040cb5a0 msgr2=0x7f58041547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.311+0000 7f580a955640 1 --2- 172.21.15.105:0/1693616941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58040cb5a0 0x7f58041547f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f57f4013b90 tx=0x7f57f4013c80 comp rx=0 tx=0).stop 2023-11-28T18:23:56.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.311+0000 7f580a955640 1 -- 172.21.15.105:0/1693616941 shutdown_connections 2023-11-28T18:23:56.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.311+0000 7f580a955640 1 --2- 172.21.15.105:0/1693616941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58040cb5a0 0x7f58041547f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.311+0000 7f580a955640 1 --2- 172.21.15.105:0/1693616941 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58040b13e0 0x7f58041514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.311+0000 7f580a955640 1 --2- 172.21.15.105:0/1693616941 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f57f00668d0 0x7f57f0068d90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.311+0000 7f580a955640 1 --2- 172.21.15.105:0/1693616941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58040abd80 0x7f580414c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.311+0000 7f580a955640 1 -- 172.21.15.105:0/1693616941 >> 172.21.15.105:0/1693616941 conn(0x7f58040b08c0 msgr2=0x7f58040cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:56.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.311+0000 7f580a955640 1 -- 172.21.15.105:0/1693616941 shutdown_connections 2023-11-28T18:23:56.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.311+0000 7f580a955640 1 -- 172.21.15.105:0/1693616941 wait complete. 2023-11-28T18:23:56.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.315+0000 7fa8db59e640 1 -- 172.21.15.105:0/3233716316 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa8dc0d2830 con 0x7fa8dc0b2a80 2023-11-28T18:23:56.319 INFO:teuthology.orchestra.run.smithi105.stdout:55834574857 2023-11-28T18:23:56.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.315+0000 7fa8d37fe640 1 -- 172.21.15.105:0/3233716316 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa8cc054fc0 con 0x7fa8dc0b2a80 2023-11-28T18:23:56.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.319+0000 7f27d4017640 1 -- 172.21.15.105:0/1333172192 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f27b4005510 con 0x7f27cc0ae100 2023-11-28T18:23:56.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.319+0000 7f27c27fc640 1 -- 172.21.15.105:0/1333172192 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f27bc017090 con 0x7f27cc0ae100 2023-11-28T18:23:56.324 INFO:teuthology.orchestra.run.smithi105.stdout:55834574857 2023-11-28T18:23:56.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7fa8db59e640 1 -- 172.21.15.105:0/3233716316 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa8bc066920 msgr2=0x7fa8bc068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7fa8db59e640 1 --2- 172.21.15.105:0/3233716316 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa8bc066920 0x7fa8bc068de0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc1606e0 tx=0x7fa8c4007990 comp rx=0 tx=0).stop 2023-11-28T18:23:56.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7f27d4017640 1 -- 172.21.15.105:0/1333172192 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27b00669a0 msgr2=0x7f27b0068e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7f27d4017640 1 --2- 172.21.15.105:0/1333172192 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27b00669a0 0x7f27b0068e60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f27c80af000 tx=0x7f27c80a7000 comp rx=0 tx=0).stop 2023-11-28T18:23:56.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7f27d4017640 1 -- 172.21.15.105:0/1333172192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27cc0ae100 msgr2=0x7f27cc0b3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7f27d4017640 1 --2- 172.21.15.105:0/1333172192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27cc0ae100 0x7f27cc0b3d90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00c420 tx=0x7f27bc00c8e0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7fa8db59e640 1 -- 172.21.15.105:0/3233716316 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8dc0b2a80 msgr2=0x7fa8dc15ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7fa8db59e640 1 --2- 172.21.15.105:0/3233716316 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8dc0b2a80 0x7fa8dc15ba30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00e980 tx=0x7fa8cc00ee40 comp rx=0 tx=0).stop 2023-11-28T18:23:56.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7fa8db59e640 1 -- 172.21.15.105:0/3233716316 shutdown_connections 2023-11-28T18:23:56.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7f27d4017640 1 -- 172.21.15.105:0/1333172192 shutdown_connections 2023-11-28T18:23:56.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7f27d4017640 1 --2- 172.21.15.105:0/1333172192 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27b00669a0 0x7f27b0068e60 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f27c80af000 tx=0x7f27c80a7000 comp rx=0 tx=0).stop 2023-11-28T18:23:56.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7f27d4017640 1 --2- 172.21.15.105:0/1333172192 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27cc0ce530 0x7f27cc202760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7f27d4017640 1 --2- 172.21.15.105:0/1333172192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27cc0ae100 0x7f27cc0b3d90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7f27d4017640 1 --2- 172.21.15.105:0/1333172192 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27cc06d610 0x7f27cc156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7f27d4017640 1 -- 172.21.15.105:0/1333172192 >> 172.21.15.105:0/1333172192 conn(0x7f27cc0ceb90 msgr2=0x7f27cc0730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:56.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7fa8db59e640 1 --2- 172.21.15.105:0/3233716316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8dc15d000 0x7fa8dc0b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7fa8db59e640 1 --2- 172.21.15.105:0/3233716316 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa8bc066920 0x7fa8bc068de0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7fa8db59e640 1 --2- 172.21.15.105:0/3233716316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8dc0cf6d0 0x7fa8dc156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7fa8db59e640 1 --2- 172.21.15.105:0/3233716316 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8dc0b2a80 0x7fa8dc15ba30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7fa8db59e640 1 -- 172.21.15.105:0/3233716316 >> 172.21.15.105:0/3233716316 conn(0x7fa8dc0d2e00 msgr2=0x7fa8dc0d31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:56.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7f27d4017640 1 -- 172.21.15.105:0/1333172192 shutdown_connections 2023-11-28T18:23:56.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7f27d4017640 1 -- 172.21.15.105:0/1333172192 wait complete. 2023-11-28T18:23:56.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7fa8db59e640 1 -- 172.21.15.105:0/3233716316 shutdown_connections 2023-11-28T18:23:56.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.323+0000 7fa8db59e640 1 -- 172.21.15.105:0/3233716316 wait complete. 2023-11-28T18:23:56.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.331+0000 7f3923580640 1 -- 172.21.15.105:0/1589563894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f391c068650 con 0x7f391c160530 2023-11-28T18:23:56.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.331+0000 7f3911ffb640 1 -- 172.21.15.105:0/1589563894 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f391401d020 con 0x7f391c160530 2023-11-28T18:23:56.335 INFO:teuthology.orchestra.run.smithi105.stdout:55834574857 2023-11-28T18:23:56.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.331+0000 7f02797fa640 1 -- 172.21.15.105:0/1649893005 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f029409c480 con 0x7f02940ad510 2023-11-28T18:23:56.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.331+0000 7f027b7fe640 1 -- 172.21.15.105:0/1649893005 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f029409c480 con 0x7f02940ad510 2023-11-28T18:23:56.336 INFO:teuthology.orchestra.run.smithi105.stdout:55834574857 2023-11-28T18:23:56.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.335+0000 7f3923580640 1 -- 172.21.15.105:0/1589563894 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f39000668d0 msgr2=0x7f3900068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.335+0000 7f3923580640 1 --2- 172.21.15.105:0/1589563894 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f39000668d0 0x7f3900068d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3918009990 tx=0x7f3918008040 comp rx=0 tx=0).stop 2023-11-28T18:23:56.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.335+0000 7f3923580640 1 -- 172.21.15.105:0/1589563894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f391c160530 msgr2=0x7f391c0c6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.335+0000 7f3923580640 1 --2- 172.21.15.105:0/1589563894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f391c160530 0x7f391c0c6610 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f391400cc90 tx=0x7f391400a650 comp rx=0 tx=0).stop 2023-11-28T18:23:56.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.335+0000 7f3923580640 1 -- 172.21.15.105:0/1589563894 shutdown_connections 2023-11-28T18:23:56.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.335+0000 7f3923580640 1 --2- 172.21.15.105:0/1589563894 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f39000668d0 0x7f3900068d90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.335+0000 7f3923580640 1 --2- 172.21.15.105:0/1589563894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f391c160530 0x7f391c0c6610 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.335+0000 7f3923580640 1 --2- 172.21.15.105:0/1589563894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f391c0d10e0 0x7f391c0c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.335+0000 7f3923580640 1 --2- 172.21.15.105:0/1589563894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391c0b4140 0x7f391c15ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.335+0000 7f3923580640 1 -- 172.21.15.105:0/1589563894 >> 172.21.15.105:0/1589563894 conn(0x7f391c0d30a0 msgr2=0x7f391c15e010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:56.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.335+0000 7f3923580640 1 -- 172.21.15.105:0/1589563894 shutdown_connections 2023-11-28T18:23:56.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.335+0000 7f3923580640 1 -- 172.21.15.105:0/1589563894 wait complete. 2023-11-28T18:23:56.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.335+0000 7f02797fa640 1 -- 172.21.15.105:0/1649893005 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f02800668d0 msgr2=0x7f0280068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.335+0000 7f02797fa640 1 --2- 172.21.15.105:0/1649893005 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f02800668d0 0x7f0280068d90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0284000f70 tx=0x7f02840078b0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.335+0000 7f02797fa640 1 -- 172.21.15.105:0/1649893005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02940ad510 msgr2=0x7f0294153e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.335+0000 7f02797fa640 1 --2- 172.21.15.105:0/1649893005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02940ad510 0x7f0294153e60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0288019ff0 tx=0x7f0288019160 comp rx=0 tx=0).stop 2023-11-28T18:23:56.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.335+0000 7f02797fa640 1 -- 172.21.15.105:0/1649893005 shutdown_connections 2023-11-28T18:23:56.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.335+0000 7f02797fa640 1 --2- 172.21.15.105:0/1649893005 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f02800668d0 0x7f0280068d90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.335+0000 7f02797fa640 1 --2- 172.21.15.105:0/1649893005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02940cbcb0 0x7f029420a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.335+0000 7f02797fa640 1 --2- 172.21.15.105:0/1649893005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02940ad510 0x7f0294153e60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.335+0000 7f02797fa640 1 --2- 172.21.15.105:0/1649893005 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02940ad130 0x7f029409b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.335+0000 7f02797fa640 1 -- 172.21.15.105:0/1649893005 >> 172.21.15.105:0/1649893005 conn(0x7f02940b08c0 msgr2=0x7f02940cb430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:56.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.339+0000 7f02797fa640 1 -- 172.21.15.105:0/1649893005 shutdown_connections 2023-11-28T18:23:56.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.339+0000 7f02797fa640 1 -- 172.21.15.105:0/1649893005 wait complete. 2023-11-28T18:23:56.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.343+0000 7f6088f2a640 1 -- 172.21.15.105:0/3666138437 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f608406db60 con 0x7f60841592f0 2023-11-28T18:23:56.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.347+0000 7f027effd640 1 -- 172.21.15.105:0/4285430364 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f029815b350 con 0x7f02980abd80 2023-11-28T18:23:56.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.347+0000 7f607b7fe640 1 -- 172.21.15.105:0/3666138437 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f606c052840 con 0x7f60841592f0 2023-11-28T18:23:56.349 INFO:teuthology.orchestra.run.smithi105.stdout:55834574858 2023-11-28T18:23:56.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.347+0000 7f02957fa640 1 -- 172.21.15.105:0/4285430364 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f029815b350 con 0x7f02980abd80 2023-11-28T18:23:56.350 INFO:teuthology.orchestra.run.smithi105.stdout:55834574858 2023-11-28T18:23:56.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.347+0000 7f6088f2a640 1 -- 172.21.15.105:0/3666138437 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6064066920 msgr2=0x7f6064068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.347+0000 7f6088f2a640 1 --2- 172.21.15.105:0/3666138437 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6064066920 0x7f6064068de0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f60740ae000 tx=0x7f60740b5040 comp rx=0 tx=0).stop 2023-11-28T18:23:56.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.347+0000 7f6088f2a640 1 -- 172.21.15.105:0/3666138437 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60841592f0 msgr2=0x7f60841fa680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.347+0000 7f6088f2a640 1 --2- 172.21.15.105:0/3666138437 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60841592f0 0x7f60841fa680 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f606c00c6d0 tx=0x7f606c00cb90 comp rx=0 tx=0).stop 2023-11-28T18:23:56.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.347+0000 7f6088f2a640 1 -- 172.21.15.105:0/3666138437 shutdown_connections 2023-11-28T18:23:56.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.347+0000 7f6088f2a640 1 --2- 172.21.15.105:0/3666138437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f608415d900 0x7f60841fabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.347+0000 7f6088f2a640 1 --2- 172.21.15.105:0/3666138437 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60841592f0 0x7f60841fa680 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.347+0000 7f6088f2a640 1 --2- 172.21.15.105:0/3666138437 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6064066920 0x7f6064068de0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.347+0000 7f6088f2a640 1 --2- 172.21.15.105:0/3666138437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f608406e2c0 0x7f60841f7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.347+0000 7f6088f2a640 1 -- 172.21.15.105:0/3666138437 >> 172.21.15.105:0/3666138437 conn(0x7f60840ceb90 msgr2=0x7f60840afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:56.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.347+0000 7f6088f2a640 1 -- 172.21.15.105:0/3666138437 shutdown_connections 2023-11-28T18:23:56.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.347+0000 7f027effd640 1 -- 172.21.15.105:0/4285430364 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0284066920 msgr2=0x7f0284068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.347+0000 7f027effd640 1 --2- 172.21.15.105:0/4285430364 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0284066920 0x7f0284068de0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0298155ee0 tx=0x7f02900073c0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.347+0000 7f6088f2a640 1 -- 172.21.15.105:0/3666138437 wait complete. 2023-11-28T18:23:56.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.347+0000 7f027effd640 1 -- 172.21.15.105:0/4285430364 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02980abd80 msgr2=0x7f02980b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.347+0000 7f027effd640 1 --2- 172.21.15.105:0/4285430364 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02980abd80 0x7f02980b3dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f028c00c3c0 tx=0x7f028c00c880 comp rx=0 tx=0).stop 2023-11-28T18:23:56.355 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.1 2023-11-28T18:23:56.356 DEBUG:teuthology.parallel:result is None 2023-11-28T18:23:56.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.351+0000 7f027effd640 1 -- 172.21.15.105:0/4285430364 shutdown_connections 2023-11-28T18:23:56.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.351+0000 7f027effd640 1 --2- 172.21.15.105:0/4285430364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02981590c0 0x7f02981582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.351+0000 7f027effd640 1 --2- 172.21.15.105:0/4285430364 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0284066920 0x7f0284068de0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.351+0000 7f027effd640 1 --2- 172.21.15.105:0/4285430364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02980cbae0 0x7f029814ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.351+0000 7f027effd640 1 --2- 172.21.15.105:0/4285430364 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02980abd80 0x7f02980b3dd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.351+0000 7f027effd640 1 -- 172.21.15.105:0/4285430364 >> 172.21.15.105:0/4285430364 conn(0x7f02980b08c0 msgr2=0x7f0298155210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:56.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.351+0000 7f027effd640 1 -- 172.21.15.105:0/4285430364 shutdown_connections 2023-11-28T18:23:56.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.351+0000 7f027effd640 1 -- 172.21.15.105:0/4285430364 wait complete. 2023-11-28T18:23:56.361 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2023-11-28T18:23:56.361 DEBUG:teuthology.parallel:result is None 2023-11-28T18:23:56.364 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2023-11-28T18:23:56.365 DEBUG:teuthology.parallel:result is None 2023-11-28T18:23:56.376 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.3 2023-11-28T18:23:56.376 DEBUG:teuthology.parallel:result is None 2023-11-28T18:23:56.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.6 2023-11-28T18:23:56.381 DEBUG:teuthology.parallel:result is None 2023-11-28T18:23:56.381 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2023-11-28T18:23:56.381 DEBUG:teuthology.parallel:result is None 2023-11-28T18:23:56.383 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.7 2023-11-28T18:23:56.383 DEBUG:teuthology.parallel:result is None 2023-11-28T18:23:56.384 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T18:23:56.384 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:23:56.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.563+0000 7fdd03017640 1 Processor -- start 2023-11-28T18:23:56.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.563+0000 7fdd03017640 1 -- start start 2023-11-28T18:23:56.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.563+0000 7fdd03017640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdcfc0b4290 0x7fdcfc0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.563+0000 7fdd03017640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdcfc071960 0x7fdcfc15f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.563+0000 7fdd03017640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcfc0b4bb0 con 0x7fdcfc15b030 2023-11-28T18:23:56.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.563+0000 7fdd03017640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcfc0701d0 con 0x7fdcfc0b4290 2023-11-28T18:23:56.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.563+0000 7fdd03017640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcfc06df80 con 0x7fdcfc071960 2023-11-28T18:23:56.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.563+0000 7fdd00db3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdcfc0b4290 0x7fdcfc0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.563+0000 7fdd00db3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdcfc0b4290 0x7fdcfc0b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55330/0 (socket says 172.21.15.105:55330) 2023-11-28T18:23:56.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.563+0000 7fdd00db3640 1 -- 172.21.15.105:0/2585771640 learned_addr learned my addr 172.21.15.105:0/2585771640 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:56.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.563+0000 7fdcfbfff640 1 --2- 172.21.15.105:0/2585771640 >> v2:172.21.15.148:3300/0 conn(0x7fdcfc071960 0x7fdcfc15f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.563+0000 7fdd00db3640 1 -- 172.21.15.105:0/2585771640 >> v2:172.21.15.148:3300/0 conn(0x7fdcfc071960 msgr2=0x7fdcfc15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:56.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.563+0000 7fdd00db3640 1 --2- 172.21.15.105:0/2585771640 >> v2:172.21.15.148:3300/0 conn(0x7fdcfc071960 0x7fdcfc15f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.563+0000 7fdd00db3640 1 -- 172.21.15.105:0/2585771640 >> v1:172.21.15.105:6789/0 conn(0x7fdcfc15b030 legacy=0x7fdcfc0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.563+0000 7fdd00db3640 1 -- 172.21.15.105:0/2585771640 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc071df0 con 0x7fdcfc0b4290 2023-11-28T18:23:56.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.567+0000 7fdd00db3640 1 --2- 172.21.15.105:0/2585771640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcfc0b4290 0x7fdcfc0b4670 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc06e520 tx=0x7fdce401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76bd6165bb971b87 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.567+0000 7fdcfb7fe640 1 -- 172.21.15.105:0/2585771640 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce40aa430 con 0x7fdcfc0b4290 2023-11-28T18:23:56.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.567+0000 7fdcfb7fe640 1 -- 172.21.15.105:0/2585771640 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce40aa590 con 0x7fdcfc0b4290 2023-11-28T18:23:56.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.567+0000 7fdcfb7fe640 1 -- 172.21.15.105:0/2585771640 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce40aa920 con 0x7fdcfc0b4290 2023-11-28T18:23:56.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.567+0000 7fdd03017640 1 -- 172.21.15.105:0/2585771640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcfc0b4290 msgr2=0x7fdcfc0b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.567+0000 7fdd03017640 1 --2- 172.21.15.105:0/2585771640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcfc0b4290 0x7fdcfc0b4670 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc06e520 tx=0x7fdce401c4d0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.567+0000 7fdd03017640 1 -- 172.21.15.105:0/2585771640 shutdown_connections 2023-11-28T18:23:56.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.567+0000 7fdd03017640 1 --2- 172.21.15.105:0/2585771640 >> v2:172.21.15.148:3300/0 conn(0x7fdcfc071960 0x7fdcfc15f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.567+0000 7fdd03017640 1 --2- 172.21.15.105:0/2585771640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcfc0b4290 0x7fdcfc0b4670 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.567+0000 7fdd03017640 1 -- 172.21.15.105:0/2585771640 >> 172.21.15.105:0/2585771640 conn(0x7fdcfc0ceb90 msgr2=0x7fdcfc0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:56.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.567+0000 7fdd03017640 1 -- 172.21.15.105:0/2585771640 shutdown_connections 2023-11-28T18:23:56.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.567+0000 7fdd03017640 1 -- 172.21.15.105:0/2585771640 wait complete. 2023-11-28T18:23:56.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdd03017640 1 Processor -- start 2023-11-28T18:23:56.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdd03017640 1 -- start start 2023-11-28T18:23:56.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdd03017640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcfc0b4290 0x7fdcfc14ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdd03017640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcfc15b030 0x7fdcfc0ad650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdd03017640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcfc14ec50 0x7fdcfc14f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdd03017640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc14b960 con 0x7fdcfc14ec50 2023-11-28T18:23:56.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdd03017640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc0cd650 con 0x7fdcfc15b030 2023-11-28T18:23:56.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdd03017640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc0cc760 con 0x7fdcfc0b4290 2023-11-28T18:23:56.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdd015b4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcfc14ec50 0x7fdcfc14f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdd015b4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcfc14ec50 0x7fdcfc14f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55336/0 (socket says 172.21.15.105:55336) 2023-11-28T18:23:56.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdd015b4640 1 -- 172.21.15.105:0/2453625011 learned_addr learned my addr 172.21.15.105:0/2453625011 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:56.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdd00db3640 1 --2- 172.21.15.105:0/2453625011 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcfc0b4290 0x7fdcfc14ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdcfbfff640 1 --2- 172.21.15.105:0/2453625011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcfc15b030 0x7fdcfc0ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdd015b4640 1 -- 172.21.15.105:0/2453625011 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcfc0b4290 msgr2=0x7fdcfc14ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdd015b4640 1 --2- 172.21.15.105:0/2453625011 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcfc0b4290 0x7fdcfc14ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdd015b4640 1 -- 172.21.15.105:0/2453625011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcfc15b030 msgr2=0x7fdcfc0ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdd015b4640 1 --2- 172.21.15.105:0/2453625011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcfc15b030 0x7fdcfc0ad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdd015b4640 1 -- 172.21.15.105:0/2453625011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce40094d0 con 0x7fdcfc14ec50 2023-11-28T18:23:56.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdd015b4640 1 --2- 172.21.15.105:0/2453625011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcfc14ec50 0x7fdcfc14f0d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdcf400c730 tx=0x7fdcf400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdcf97fa640 1 -- 172.21.15.105:0/2453625011 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4003b80 con 0x7fdcfc14ec50 2023-11-28T18:23:56.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdcf97fa640 1 -- 172.21.15.105:0/2453625011 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf4003ce0 con 0x7fdcfc14ec50 2023-11-28T18:23:56.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdcf97fa640 1 -- 172.21.15.105:0/2453625011 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4005310 con 0x7fdcfc14ec50 2023-11-28T18:23:56.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdd03017640 1 -- 172.21.15.105:0/2453625011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcfc0cca40 con 0x7fdcfc14ec50 2023-11-28T18:23:56.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.571+0000 7fdd03017640 1 -- 172.21.15.105:0/2453625011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcfc0cd000 con 0x7fdcfc14ec50 2023-11-28T18:23:56.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.575+0000 7fdd03017640 1 -- 172.21.15.105:0/2453625011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcfc067570 con 0x7fdcfc14ec50 2023-11-28T18:23:56.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.575+0000 7fdcf97fa640 1 -- 172.21.15.105:0/2453625011 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fdcf4005680 con 0x7fdcfc14ec50 2023-11-28T18:23:56.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.575+0000 7fdcf97fa640 1 --2- 172.21.15.105:0/2453625011 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdcdc0668d0 0x7fdcdc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.575+0000 7fdcf97fa640 1 -- 172.21.15.105:0/2453625011 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdcf40845b0 con 0x7fdcfc14ec50 2023-11-28T18:23:56.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.575+0000 7fdd00db3640 1 --2- 172.21.15.105:0/2453625011 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdcdc0668d0 0x7fdcdc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.575+0000 7fdd00db3640 1 --2- 172.21.15.105:0/2453625011 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdcdc0668d0 0x7fdcdc068d90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdce40ae000 tx=0x7fdce40b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.579+0000 7fdcf97fa640 1 -- 172.21.15.105:0/2453625011 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdcf4051510 con 0x7fdcfc14ec50 2023-11-28T18:23:56.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.727+0000 7fdd03017640 1 -- 172.21.15.105:0/2453625011 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdcfc15f8f0 con 0x7fdcdc0668d0 2023-11-28T18:23:56.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.735+0000 7fdcf97fa640 1 -- 172.21.15.105:0/2453625011 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345252 (secure 0 0 0) 0x7fdce40aa880 con 0x7fdcdc0668d0 2023-11-28T18:23:56.743 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:56.745 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:23:56.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.739+0000 7fdd03017640 1 -- 172.21.15.105:0/2453625011 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdcdc0668d0 msgr2=0x7fdcdc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.739+0000 7fdd03017640 1 --2- 172.21.15.105:0/2453625011 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdcdc0668d0 0x7fdcdc068d90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdce40ae000 tx=0x7fdce40b5040 comp rx=0 tx=0).stop 2023-11-28T18:23:56.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.739+0000 7fdd03017640 1 -- 172.21.15.105:0/2453625011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcfc14ec50 msgr2=0x7fdcfc14f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.739+0000 7fdd03017640 1 --2- 172.21.15.105:0/2453625011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcfc14ec50 0x7fdcfc14f0d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdcf400c730 tx=0x7fdcf400cbf0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.739+0000 7fdd03017640 1 -- 172.21.15.105:0/2453625011 shutdown_connections 2023-11-28T18:23:56.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.739+0000 7fdd03017640 1 --2- 172.21.15.105:0/2453625011 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdcdc0668d0 0x7fdcdc068d90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.739+0000 7fdd03017640 1 --2- 172.21.15.105:0/2453625011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcfc14ec50 0x7fdcfc14f0d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.739+0000 7fdd03017640 1 --2- 172.21.15.105:0/2453625011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcfc15b030 0x7fdcfc0ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.739+0000 7fdd03017640 1 --2- 172.21.15.105:0/2453625011 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcfc0b4290 0x7fdcfc14ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.739+0000 7fdd03017640 1 -- 172.21.15.105:0/2453625011 >> 172.21.15.105:0/2453625011 conn(0x7fdcfc0ceb90 msgr2=0x7fdcfc09a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:56.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.739+0000 7fdd03017640 1 -- 172.21.15.105:0/2453625011 shutdown_connections 2023-11-28T18:23:56.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.739+0000 7fdd03017640 1 -- 172.21.15.105:0/2453625011 wait complete. 2023-11-28T18:23:56.763 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2023-11-28T18:23:55.273505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219380,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755340,"statfs":{"total":773094113280,"available":772869468160,"internally_reserved":0,"allocated":3284992,"data_stored":1988350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2229,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002522"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.076931+0000","last_change":"2023-11-28T18:23:32.930693+0000","last_active":"2023-11-28T18:23:38.076931+0000","last_peered":"2023-11-28T18:23:38.076931+0000","last_clean":"2023-11-28T18:23:38.076931+0000","last_became_active":"2023-11-28T18:23:32.930410+0000","last_became_peered":"2023-11-28T18:23:32.930410+0000","last_unstale":"2023-11-28T18:23:38.076931+0000","last_undegraded":"2023-11-28T18:23:38.076931+0000","last_fullsized":"2023-11-28T18:23:38.076931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:19.143213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.066696+0000","last_change":"2023-11-28T18:23:33.947291+0000","last_active":"2023-11-28T18:23:38.066696+0000","last_peered":"2023-11-28T18:23:38.066696+0000","last_clean":"2023-11-28T18:23:38.066696+0000","last_became_active":"2023-11-28T18:23:33.946963+0000","last_became_peered":"2023-11-28T18:23:33.946963+0000","last_unstale":"2023-11-28T18:23:38.066696+0000","last_undegraded":"2023-11-28T18:23:38.066696+0000","last_fullsized":"2023-11-28T18:23:38.066696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:27.506820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.067797+0000","last_change":"2023-11-28T18:23:38.425549+0000","last_active":"2023-11-28T18:23:39.067797+0000","last_peered":"2023-11-28T18:23:39.067797+0000","last_clean":"2023-11-28T18:23:39.067797+0000","last_became_active":"2023-11-28T18:23:38.425130+0000","last_became_peered":"2023-11-28T18:23:38.425130+0000","last_unstale":"2023-11-28T18:23:39.067797+0000","last_undegraded":"2023-11-28T18:23:39.067797+0000","last_fullsized":"2023-11-28T18:23:39.067797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:19:51.686256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.066904+0000","last_change":"2023-11-28T18:23:32.921328+0000","last_active":"2023-11-28T18:23:38.066904+0000","last_peered":"2023-11-28T18:23:38.066904+0000","last_clean":"2023-11-28T18:23:38.066904+0000","last_became_active":"2023-11-28T18:23:32.920775+0000","last_became_peered":"2023-11-28T18:23:32.920775+0000","last_unstale":"2023-11-28T18:23:38.066904+0000","last_undegraded":"2023-11-28T18:23:38.066904+0000","last_fullsized":"2023-11-28T18:23:38.066904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:05:45.779946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075403+0000","last_change":"2023-11-28T18:23:33.948349+0000","last_active":"2023-11-28T18:23:38.075403+0000","last_peered":"2023-11-28T18:23:38.075403+0000","last_clean":"2023-11-28T18:23:38.075403+0000","last_became_active":"2023-11-28T18:23:33.947987+0000","last_became_peered":"2023-11-28T18:23:33.947987+0000","last_unstale":"2023-11-28T18:23:38.075403+0000","last_undegraded":"2023-11-28T18:23:38.075403+0000","last_fullsized":"2023-11-28T18:23:38.075403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:35.484199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068226+0000","last_change":"2023-11-28T18:23:38.091059+0000","last_active":"2023-11-28T18:23:39.068226+0000","last_peered":"2023-11-28T18:23:39.068226+0000","last_clean":"2023-11-28T18:23:39.068226+0000","last_became_active":"2023-11-28T18:23:38.090259+0000","last_became_peered":"2023-11-28T18:23:38.090259+0000","last_unstale":"2023-11-28T18:23:39.068226+0000","last_undegraded":"2023-11-28T18:23:39.068226+0000","last_fullsized":"2023-11-28T18:23:39.068226+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:05:00.278938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420356+0000","last_change":"2023-11-28T18:23:32.937363+0000","last_active":"2023-11-28T18:23:38.420356+0000","last_peered":"2023-11-28T18:23:38.420356+0000","last_clean":"2023-11-28T18:23:38.420356+0000","last_became_active":"2023-11-28T18:23:32.937125+0000","last_became_peered":"2023-11-28T18:23:32.937125+0000","last_unstale":"2023-11-28T18:23:38.420356+0000","last_undegraded":"2023-11-28T18:23:38.420356+0000","last_fullsized":"2023-11-28T18:23:38.420356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:45.143373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067755+0000","last_change":"2023-11-28T18:23:33.935572+0000","last_active":"2023-11-28T18:23:38.067755+0000","last_peered":"2023-11-28T18:23:38.067755+0000","last_clean":"2023-11-28T18:23:38.067755+0000","last_became_active":"2023-11-28T18:23:33.935204+0000","last_became_peered":"2023-11-28T18:23:33.935204+0000","last_unstale":"2023-11-28T18:23:38.067755+0000","last_undegraded":"2023-11-28T18:23:38.067755+0000","last_fullsized":"2023-11-28T18:23:38.067755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:02.989816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068674+0000","last_change":"2023-11-28T18:23:38.096313+0000","last_active":"2023-11-28T18:23:39.068674+0000","last_peered":"2023-11-28T18:23:39.068674+0000","last_clean":"2023-11-28T18:23:39.068674+0000","last_became_active":"2023-11-28T18:23:38.095957+0000","last_became_peered":"2023-11-28T18:23:38.095957+0000","last_unstale":"2023-11-28T18:23:39.068674+0000","last_undegraded":"2023-11-28T18:23:39.068674+0000","last_fullsized":"2023-11-28T18:23:39.068674+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:07.150212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419858+0000","last_change":"2023-11-28T18:23:32.939798+0000","last_active":"2023-11-28T18:23:38.419858+0000","last_peered":"2023-11-28T18:23:38.419858+0000","last_clean":"2023-11-28T18:23:38.419858+0000","last_became_active":"2023-11-28T18:23:32.939450+0000","last_became_peered":"2023-11-28T18:23:32.939450+0000","last_unstale":"2023-11-28T18:23:38.419858+0000","last_undegraded":"2023-11-28T18:23:38.419858+0000","last_fullsized":"2023-11-28T18:23:38.419858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:55:26.908976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067829+0000","last_change":"2023-11-28T18:23:33.933973+0000","last_active":"2023-11-28T18:23:38.067829+0000","last_peered":"2023-11-28T18:23:38.067829+0000","last_clean":"2023-11-28T18:23:38.067829+0000","last_became_active":"2023-11-28T18:23:33.933692+0000","last_became_peered":"2023-11-28T18:23:33.933692+0000","last_unstale":"2023-11-28T18:23:38.067829+0000","last_undegraded":"2023-11-28T18:23:38.067829+0000","last_fullsized":"2023-11-28T18:23:38.067829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:38.277787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068388+0000","last_change":"2023-11-28T18:23:38.089272+0000","last_active":"2023-11-28T18:23:39.068388+0000","last_peered":"2023-11-28T18:23:39.068388+0000","last_clean":"2023-11-28T18:23:39.068388+0000","last_became_active":"2023-11-28T18:23:38.088843+0000","last_became_peered":"2023-11-28T18:23:38.088843+0000","last_unstale":"2023-11-28T18:23:39.068388+0000","last_undegraded":"2023-11-28T18:23:39.068388+0000","last_fullsized":"2023-11-28T18:23:39.068388+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:50.581936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.073981+0000","last_change":"2023-11-28T18:23:33.945002+0000","last_active":"2023-11-28T18:23:38.073981+0000","last_peered":"2023-11-28T18:23:38.073981+0000","last_clean":"2023-11-28T18:23:38.073981+0000","last_became_active":"2023-11-28T18:23:33.942488+0000","last_became_peered":"2023-11-28T18:23:33.942488+0000","last_unstale":"2023-11-28T18:23:38.073981+0000","last_undegraded":"2023-11-28T18:23:38.073981+0000","last_fullsized":"2023-11-28T18:23:38.073981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:04:56.118542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419933+0000","last_change":"2023-11-28T18:23:32.938369+0000","last_active":"2023-11-28T18:23:38.419933+0000","last_peered":"2023-11-28T18:23:38.419933+0000","last_clean":"2023-11-28T18:23:38.419933+0000","last_became_active":"2023-11-28T18:23:32.938126+0000","last_became_peered":"2023-11-28T18:23:32.938126+0000","last_unstale":"2023-11-28T18:23:38.419933+0000","last_undegraded":"2023-11-28T18:23:38.419933+0000","last_fullsized":"2023-11-28T18:23:38.419933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:50.646527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.069187+0000","last_change":"2023-11-28T18:23:38.093734+0000","last_active":"2023-11-28T18:23:39.069187+0000","last_peered":"2023-11-28T18:23:39.069187+0000","last_clean":"2023-11-28T18:23:39.069187+0000","last_became_active":"2023-11-28T18:23:38.093298+0000","last_became_peered":"2023-11-28T18:23:38.093298+0000","last_unstale":"2023-11-28T18:23:39.069187+0000","last_undegraded":"2023-11-28T18:23:39.069187+0000","last_fullsized":"2023-11-28T18:23:39.069187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:22:41.524911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065575+0000","last_change":"2023-11-28T18:23:33.938245+0000","last_active":"2023-11-28T18:23:38.065575+0000","last_peered":"2023-11-28T18:23:38.065575+0000","last_clean":"2023-11-28T18:23:38.065575+0000","last_became_active":"2023-11-28T18:23:33.937832+0000","last_became_peered":"2023-11-28T18:23:33.937832+0000","last_unstale":"2023-11-28T18:23:38.065575+0000","last_undegraded":"2023-11-28T18:23:38.065575+0000","last_fullsized":"2023-11-28T18:23:38.065575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:04:56.118542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.093039+0000","last_change":"2023-11-28T18:23:32.924880+0000","last_active":"2023-11-28T18:23:38.093039+0000","last_peered":"2023-11-28T18:23:38.093039+0000","last_clean":"2023-11-28T18:23:38.093039+0000","last_became_active":"2023-11-28T18:23:32.923804+0000","last_became_peered":"2023-11-28T18:23:32.923804+0000","last_unstale":"2023-11-28T18:23:38.093039+0000","last_undegraded":"2023-11-28T18:23:38.093039+0000","last_fullsized":"2023-11-28T18:23:38.093039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:10.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068797+0000","last_change":"2023-11-28T18:23:38.091628+0000","last_active":"2023-11-28T18:23:39.068797+0000","last_peered":"2023-11-28T18:23:39.068797+0000","last_clean":"2023-11-28T18:23:39.068797+0000","last_became_active":"2023-11-28T18:23:38.090900+0000","last_became_peered":"2023-11-28T18:23:38.090900+0000","last_unstale":"2023-11-28T18:23:39.068797+0000","last_undegraded":"2023-11-28T18:23:39.068797+0000","last_fullsized":"2023-11-28T18:23:39.068797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:47.152718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065511+0000","last_change":"2023-11-28T18:23:33.933391+0000","last_active":"2023-11-28T18:23:38.065511+0000","last_peered":"2023-11-28T18:23:38.065511+0000","last_clean":"2023-11-28T18:23:38.065511+0000","last_became_active":"2023-11-28T18:23:33.932767+0000","last_became_peered":"2023-11-28T18:23:33.932767+0000","last_unstale":"2023-11-28T18:23:38.065511+0000","last_undegraded":"2023-11-28T18:23:38.065511+0000","last_fullsized":"2023-11-28T18:23:38.065511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:21.035422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.092828+0000","last_change":"2023-11-28T18:23:32.929281+0000","last_active":"2023-11-28T18:23:38.092828+0000","last_peered":"2023-11-28T18:23:38.092828+0000","last_clean":"2023-11-28T18:23:38.092828+0000","last_became_active":"2023-11-28T18:23:32.928997+0000","last_became_peered":"2023-11-28T18:23:32.928997+0000","last_unstale":"2023-11-28T18:23:38.092828+0000","last_undegraded":"2023-11-28T18:23:38.092828+0000","last_fullsized":"2023-11-28T18:23:38.092828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:45.143373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.718883+0000","last_change":"2023-11-28T18:23:38.426592+0000","last_active":"2023-11-28T18:23:40.718883+0000","last_peered":"2023-11-28T18:23:40.718883+0000","last_clean":"2023-11-28T18:23:40.718883+0000","last_became_active":"2023-11-28T18:23:38.426089+0000","last_became_peered":"2023-11-28T18:23:38.426089+0000","last_unstale":"2023-11-28T18:23:40.718883+0000","last_undegraded":"2023-11-28T18:23:40.718883+0000","last_fullsized":"2023-11-28T18:23:40.718883+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:15.599832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075007+0000","last_change":"2023-11-28T18:23:33.935238+0000","last_active":"2023-11-28T18:23:38.075007+0000","last_peered":"2023-11-28T18:23:38.075007+0000","last_clean":"2023-11-28T18:23:38.075007+0000","last_became_active":"2023-11-28T18:23:33.934884+0000","last_became_peered":"2023-11-28T18:23:33.934884+0000","last_unstale":"2023-11-28T18:23:38.075007+0000","last_undegraded":"2023-11-28T18:23:38.075007+0000","last_fullsized":"2023-11-28T18:23:38.075007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:58.099391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419489+0000","last_change":"2023-11-28T18:23:32.933485+0000","last_active":"2023-11-28T18:23:38.419489+0000","last_peered":"2023-11-28T18:23:38.419489+0000","last_clean":"2023-11-28T18:23:38.419489+0000","last_became_active":"2023-11-28T18:23:32.932807+0000","last_became_peered":"2023-11-28T18:23:32.932807+0000","last_unstale":"2023-11-28T18:23:38.419489+0000","last_undegraded":"2023-11-28T18:23:38.419489+0000","last_fullsized":"2023-11-28T18:23:38.419489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:00.325734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.069240+0000","last_change":"2023-11-28T18:23:38.091553+0000","last_active":"2023-11-28T18:23:39.069240+0000","last_peered":"2023-11-28T18:23:39.069240+0000","last_clean":"2023-11-28T18:23:39.069240+0000","last_became_active":"2023-11-28T18:23:38.091133+0000","last_became_peered":"2023-11-28T18:23:38.091133+0000","last_unstale":"2023-11-28T18:23:39.069240+0000","last_undegraded":"2023-11-28T18:23:39.069240+0000","last_fullsized":"2023-11-28T18:23:39.069240+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:53.274366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.080431+0000","last_change":"2023-11-28T18:23:33.941025+0000","last_active":"2023-11-28T18:23:38.080431+0000","last_peered":"2023-11-28T18:23:38.080431+0000","last_clean":"2023-11-28T18:23:38.080431+0000","last_became_active":"2023-11-28T18:23:33.940714+0000","last_became_peered":"2023-11-28T18:23:33.940714+0000","last_unstale":"2023-11-28T18:23:38.080431+0000","last_undegraded":"2023-11-28T18:23:38.080431+0000","last_fullsized":"2023-11-28T18:23:38.080431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:01.912695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077789+0000","last_change":"2023-11-28T18:23:32.933239+0000","last_active":"2023-11-28T18:23:38.077789+0000","last_peered":"2023-11-28T18:23:38.077789+0000","last_clean":"2023-11-28T18:23:38.077789+0000","last_became_active":"2023-11-28T18:23:32.932935+0000","last_became_peered":"2023-11-28T18:23:32.932935+0000","last_unstale":"2023-11-28T18:23:38.077789+0000","last_undegraded":"2023-11-28T18:23:38.077789+0000","last_fullsized":"2023-11-28T18:23:38.077789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:36:09.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068440+0000","last_change":"2023-11-28T18:23:38.421858+0000","last_active":"2023-11-28T18:23:39.068440+0000","last_peered":"2023-11-28T18:23:39.068440+0000","last_clean":"2023-11-28T18:23:39.068440+0000","last_became_active":"2023-11-28T18:23:38.421416+0000","last_became_peered":"2023-11-28T18:23:38.421416+0000","last_unstale":"2023-11-28T18:23:39.068440+0000","last_undegraded":"2023-11-28T18:23:39.068440+0000","last_fullsized":"2023-11-28T18:23:39.068440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:31.667216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067068+0000","last_change":"2023-11-28T18:23:33.940147+0000","last_active":"2023-11-28T18:23:38.067068+0000","last_peered":"2023-11-28T18:23:38.067068+0000","last_clean":"2023-11-28T18:23:38.067068+0000","last_became_active":"2023-11-28T18:23:33.939882+0000","last_became_peered":"2023-11-28T18:23:33.939882+0000","last_unstale":"2023-11-28T18:23:38.067068+0000","last_undegraded":"2023-11-28T18:23:38.067068+0000","last_fullsized":"2023-11-28T18:23:38.067068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:02.989816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.096365+0000","last_change":"2023-11-28T18:23:32.925741+0000","last_active":"2023-11-28T18:23:38.096365+0000","last_peered":"2023-11-28T18:23:38.096365+0000","last_clean":"2023-11-28T18:23:38.096365+0000","last_became_active":"2023-11-28T18:23:32.925344+0000","last_became_peered":"2023-11-28T18:23:32.925344+0000","last_unstale":"2023-11-28T18:23:38.096365+0000","last_undegraded":"2023-11-28T18:23:38.096365+0000","last_fullsized":"2023-11-28T18:23:38.096365+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:41.820325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.067516+0000","last_change":"2023-11-28T18:23:38.421716+0000","last_active":"2023-11-28T18:23:39.067516+0000","last_peered":"2023-11-28T18:23:39.067516+0000","last_clean":"2023-11-28T18:23:39.067516+0000","last_became_active":"2023-11-28T18:23:38.421313+0000","last_became_peered":"2023-11-28T18:23:38.421313+0000","last_unstale":"2023-11-28T18:23:39.067516+0000","last_undegraded":"2023-11-28T18:23:39.067516+0000","last_fullsized":"2023-11-28T18:23:39.067516+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:12:36.678267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.094519+0000","last_change":"2023-11-28T18:23:33.934187+0000","last_active":"2023-11-28T18:23:38.094519+0000","last_peered":"2023-11-28T18:23:38.094519+0000","last_clean":"2023-11-28T18:23:38.094519+0000","last_became_active":"2023-11-28T18:23:33.933885+0000","last_became_peered":"2023-11-28T18:23:33.933885+0000","last_unstale":"2023-11-28T18:23:38.094519+0000","last_undegraded":"2023-11-28T18:23:38.094519+0000","last_fullsized":"2023-11-28T18:23:38.094519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:21.035422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420442+0000","last_change":"2023-11-28T18:23:32.933434+0000","last_active":"2023-11-28T18:23:38.420442+0000","last_peered":"2023-11-28T18:23:38.420442+0000","last_clean":"2023-11-28T18:23:38.420442+0000","last_became_active":"2023-11-28T18:23:32.933143+0000","last_became_peered":"2023-11-28T18:23:32.933143+0000","last_unstale":"2023-11-28T18:23:38.420442+0000","last_undegraded":"2023-11-28T18:23:38.420442+0000","last_fullsized":"2023-11-28T18:23:38.420442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:11.551517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.297189+0000","last_change":"2023-11-28T18:23:38.422070+0000","last_active":"2023-11-28T18:23:40.297189+0000","last_peered":"2023-11-28T18:23:40.297189+0000","last_clean":"2023-11-28T18:23:40.297189+0000","last_became_active":"2023-11-28T18:23:38.421240+0000","last_became_peered":"2023-11-28T18:23:38.421240+0000","last_unstale":"2023-11-28T18:23:40.297189+0000","last_undegraded":"2023-11-28T18:23:40.297189+0000","last_fullsized":"2023-11-28T18:23:40.297189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:42.438182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067926+0000","last_change":"2023-11-28T18:23:33.932685+0000","last_active":"2023-11-28T18:23:38.067926+0000","last_peered":"2023-11-28T18:23:38.067926+0000","last_clean":"2023-11-28T18:23:38.067926+0000","last_became_active":"2023-11-28T18:23:33.932342+0000","last_became_peered":"2023-11-28T18:23:33.932342+0000","last_unstale":"2023-11-28T18:23:38.067926+0000","last_undegraded":"2023-11-28T18:23:38.067926+0000","last_fullsized":"2023-11-28T18:23:38.067926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:49.113970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.094608+0000","last_change":"2023-11-28T18:23:32.923234+0000","last_active":"2023-11-28T18:23:38.094608+0000","last_peered":"2023-11-28T18:23:38.094608+0000","last_clean":"2023-11-28T18:23:38.094608+0000","last_became_active":"2023-11-28T18:23:32.922051+0000","last_became_peered":"2023-11-28T18:23:32.922051+0000","last_unstale":"2023-11-28T18:23:38.094608+0000","last_undegraded":"2023-11-28T18:23:38.094608+0000","last_fullsized":"2023-11-28T18:23:38.094608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:36:09.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.718350+0000","last_change":"2023-11-28T18:23:38.102307+0000","last_active":"2023-11-28T18:23:40.718350+0000","last_peered":"2023-11-28T18:23:40.718350+0000","last_clean":"2023-11-28T18:23:40.718350+0000","last_became_active":"2023-11-28T18:23:38.101987+0000","last_became_peered":"2023-11-28T18:23:38.101987+0000","last_unstale":"2023-11-28T18:23:40.718350+0000","last_undegraded":"2023-11-28T18:23:40.718350+0000","last_fullsized":"2023-11-28T18:23:40.718350+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:22:41.524911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.076427+0000","last_change":"2023-11-28T18:23:33.941492+0000","last_active":"2023-11-28T18:23:38.076427+0000","last_peered":"2023-11-28T18:23:38.076427+0000","last_clean":"2023-11-28T18:23:38.076427+0000","last_became_active":"2023-11-28T18:23:33.941148+0000","last_became_peered":"2023-11-28T18:23:33.941148+0000","last_unstale":"2023-11-28T18:23:38.076427+0000","last_undegraded":"2023-11-28T18:23:38.076427+0000","last_fullsized":"2023-11-28T18:23:38.076427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:29:23.426289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068690+0000","last_change":"2023-11-28T18:23:32.922032+0000","last_active":"2023-11-28T18:23:38.068690+0000","last_peered":"2023-11-28T18:23:38.068690+0000","last_clean":"2023-11-28T18:23:38.068690+0000","last_became_active":"2023-11-28T18:23:32.921635+0000","last_became_peered":"2023-11-28T18:23:32.921635+0000","last_unstale":"2023-11-28T18:23:38.068690+0000","last_undegraded":"2023-11-28T18:23:38.068690+0000","last_fullsized":"2023-11-28T18:23:38.068690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:36:09.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.718745+0000","last_change":"2023-11-28T18:23:38.096609+0000","last_active":"2023-11-28T18:23:40.718745+0000","last_peered":"2023-11-28T18:23:40.718745+0000","last_clean":"2023-11-28T18:23:40.718745+0000","last_became_active":"2023-11-28T18:23:38.095758+0000","last_became_peered":"2023-11-28T18:23:38.095758+0000","last_unstale":"2023-11-28T18:23:40.718745+0000","last_undegraded":"2023-11-28T18:23:40.718745+0000","last_fullsized":"2023-11-28T18:23:40.718745+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:42.438182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.090061+0000","last_change":"2023-11-28T18:23:33.937266+0000","last_active":"2023-11-28T18:23:38.090061+0000","last_peered":"2023-11-28T18:23:38.090061+0000","last_clean":"2023-11-28T18:23:38.090061+0000","last_became_active":"2023-11-28T18:23:33.936994+0000","last_became_peered":"2023-11-28T18:23:33.936994+0000","last_unstale":"2023-11-28T18:23:38.090061+0000","last_undegraded":"2023-11-28T18:23:38.090061+0000","last_fullsized":"2023-11-28T18:23:38.090061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:12.557669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.083701+0000","last_change":"2023-11-28T18:23:32.918591+0000","last_active":"2023-11-28T18:23:38.083701+0000","last_peered":"2023-11-28T18:23:38.083701+0000","last_clean":"2023-11-28T18:23:38.083701+0000","last_became_active":"2023-11-28T18:23:32.915381+0000","last_became_peered":"2023-11-28T18:23:32.915381+0000","last_unstale":"2023-11-28T18:23:38.083701+0000","last_undegraded":"2023-11-28T18:23:38.083701+0000","last_fullsized":"2023-11-28T18:23:38.083701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:57.093239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.718851+0000","last_change":"2023-11-28T18:23:38.100344+0000","last_active":"2023-11-28T18:23:40.718851+0000","last_peered":"2023-11-28T18:23:40.718851+0000","last_clean":"2023-11-28T18:23:40.718851+0000","last_became_active":"2023-11-28T18:23:38.099934+0000","last_became_peered":"2023-11-28T18:23:38.099934+0000","last_unstale":"2023-11-28T18:23:40.718851+0000","last_undegraded":"2023-11-28T18:23:40.718851+0000","last_fullsized":"2023-11-28T18:23:40.718851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:39.149571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.074486+0000","last_change":"2023-11-28T18:23:33.944704+0000","last_active":"2023-11-28T18:23:38.074486+0000","last_peered":"2023-11-28T18:23:38.074486+0000","last_clean":"2023-11-28T18:23:38.074486+0000","last_became_active":"2023-11-28T18:23:33.944484+0000","last_became_peered":"2023-11-28T18:23:33.944484+0000","last_unstale":"2023-11-28T18:23:38.074486+0000","last_undegraded":"2023-11-28T18:23:38.074486+0000","last_fullsized":"2023-11-28T18:23:38.074486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:55:27.915128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419489+0000","last_change":"2023-11-28T18:23:32.933677+0000","last_active":"2023-11-28T18:23:38.419489+0000","last_peered":"2023-11-28T18:23:38.419489+0000","last_clean":"2023-11-28T18:23:38.419489+0000","last_became_active":"2023-11-28T18:23:32.933168+0000","last_became_peered":"2023-11-28T18:23:32.933168+0000","last_unstale":"2023-11-28T18:23:38.419489+0000","last_undegraded":"2023-11-28T18:23:38.419489+0000","last_fullsized":"2023-11-28T18:23:38.419489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:29:22.420137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.297576+0000","last_change":"2023-11-28T18:23:38.427376+0000","last_active":"2023-11-28T18:23:40.297576+0000","last_peered":"2023-11-28T18:23:40.297576+0000","last_clean":"2023-11-28T18:23:40.297576+0000","last_became_active":"2023-11-28T18:23:38.426436+0000","last_became_peered":"2023-11-28T18:23:38.426436+0000","last_unstale":"2023-11-28T18:23:40.297576+0000","last_undegraded":"2023-11-28T18:23:40.297576+0000","last_fullsized":"2023-11-28T18:23:40.297576+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:19:23.246961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065098+0000","last_change":"2023-11-28T18:23:33.932836+0000","last_active":"2023-11-28T18:23:38.065098+0000","last_peered":"2023-11-28T18:23:38.065098+0000","last_clean":"2023-11-28T18:23:38.065098+0000","last_became_active":"2023-11-28T18:23:33.932534+0000","last_became_peered":"2023-11-28T18:23:33.932534+0000","last_unstale":"2023-11-28T18:23:38.065098+0000","last_undegraded":"2023-11-28T18:23:38.065098+0000","last_fullsized":"2023-11-28T18:23:38.065098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:46.421541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067421+0000","last_change":"2023-11-28T18:23:32.922842+0000","last_active":"2023-11-28T18:23:38.067421+0000","last_peered":"2023-11-28T18:23:38.067421+0000","last_clean":"2023-11-28T18:23:38.067421+0000","last_became_active":"2023-11-28T18:23:32.922541+0000","last_became_peered":"2023-11-28T18:23:32.922541+0000","last_unstale":"2023-11-28T18:23:38.067421+0000","last_undegraded":"2023-11-28T18:23:38.067421+0000","last_fullsized":"2023-11-28T18:23:38.067421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:39:52.964875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419571+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:23:38.419571+0000","last_peered":"2023-11-28T18:23:38.419571+0000","last_clean":"2023-11-28T18:23:38.419571+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:23:38.419571+0000","last_undegraded":"2023-11-28T18:23:38.419571+0000","last_fullsized":"2023-11-28T18:23:38.419571+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.719264+0000","last_change":"2023-11-28T18:23:38.427043+0000","last_active":"2023-11-28T18:23:40.719264+0000","last_peered":"2023-11-28T18:23:40.719264+0000","last_clean":"2023-11-28T18:23:40.719264+0000","last_became_active":"2023-11-28T18:23:38.426613+0000","last_became_peered":"2023-11-28T18:23:38.426613+0000","last_unstale":"2023-11-28T18:23:40.719264+0000","last_undegraded":"2023-11-28T18:23:40.719264+0000","last_fullsized":"2023-11-28T18:23:40.719264+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:47.152718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419993+0000","last_change":"2023-11-28T18:23:33.944370+0000","last_active":"2023-11-28T18:23:38.419993+0000","last_peered":"2023-11-28T18:23:38.419993+0000","last_clean":"2023-11-28T18:23:38.419993+0000","last_became_active":"2023-11-28T18:23:33.943698+0000","last_became_peered":"2023-11-28T18:23:33.943698+0000","last_unstale":"2023-11-28T18:23:38.419993+0000","last_undegraded":"2023-11-28T18:23:38.419993+0000","last_fullsized":"2023-11-28T18:23:38.419993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:49.113970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.083176+0000","last_change":"2023-11-28T18:23:32.917731+0000","last_active":"2023-11-28T18:23:38.083176+0000","last_peered":"2023-11-28T18:23:38.083176+0000","last_clean":"2023-11-28T18:23:38.083176+0000","last_became_active":"2023-11-28T18:23:32.917207+0000","last_became_peered":"2023-11-28T18:23:32.917207+0000","last_unstale":"2023-11-28T18:23:38.083176+0000","last_undegraded":"2023-11-28T18:23:38.083176+0000","last_fullsized":"2023-11-28T18:23:38.083176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:38:50.132097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.719164+0000","last_change":"2023-11-28T18:23:38.425300+0000","last_active":"2023-11-28T18:23:40.719164+0000","last_peered":"2023-11-28T18:23:40.719164+0000","last_clean":"2023-11-28T18:23:40.719164+0000","last_became_active":"2023-11-28T18:23:38.424708+0000","last_became_peered":"2023-11-28T18:23:38.424708+0000","last_unstale":"2023-11-28T18:23:40.719164+0000","last_undegraded":"2023-11-28T18:23:40.719164+0000","last_fullsized":"2023-11-28T18:23:40.719164+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:07.150212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067866+0000","last_change":"2023-11-28T18:23:33.932266+0000","last_active":"2023-11-28T18:23:38.067866+0000","last_peered":"2023-11-28T18:23:38.067866+0000","last_clean":"2023-11-28T18:23:38.067866+0000","last_became_active":"2023-11-28T18:23:33.931923+0000","last_became_peered":"2023-11-28T18:23:33.931923+0000","last_unstale":"2023-11-28T18:23:38.067866+0000","last_undegraded":"2023-11-28T18:23:38.067866+0000","last_fullsized":"2023-11-28T18:23:38.067866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:34.989175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.094421+0000","last_change":"2023-11-28T18:23:32.924719+0000","last_active":"2023-11-28T18:23:38.094421+0000","last_peered":"2023-11-28T18:23:38.094421+0000","last_clean":"2023-11-28T18:23:38.094421+0000","last_became_active":"2023-11-28T18:23:32.924462+0000","last_became_peered":"2023-11-28T18:23:32.924462+0000","last_unstale":"2023-11-28T18:23:38.094421+0000","last_undegraded":"2023-11-28T18:23:38.094421+0000","last_fullsized":"2023-11-28T18:23:38.094421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:55:26.908976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.297354+0000","last_change":"2023-11-28T18:23:38.093163+0000","last_active":"2023-11-28T18:23:40.297354+0000","last_peered":"2023-11-28T18:23:40.297354+0000","last_clean":"2023-11-28T18:23:40.297354+0000","last_became_active":"2023-11-28T18:23:38.092754+0000","last_became_peered":"2023-11-28T18:23:38.092754+0000","last_unstale":"2023-11-28T18:23:40.297354+0000","last_undegraded":"2023-11-28T18:23:40.297354+0000","last_fullsized":"2023-11-28T18:23:40.297354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:39.149571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420505+0000","last_change":"2023-11-28T18:23:33.954652+0000","last_active":"2023-11-28T18:23:38.420505+0000","last_peered":"2023-11-28T18:23:38.420505+0000","last_clean":"2023-11-28T18:23:38.420505+0000","last_became_active":"2023-11-28T18:23:33.954369+0000","last_became_peered":"2023-11-28T18:23:33.954369+0000","last_unstale":"2023-11-28T18:23:38.420505+0000","last_undegraded":"2023-11-28T18:23:38.420505+0000","last_fullsized":"2023-11-28T18:23:38.420505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:30:56.523444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.084061+0000","last_change":"2023-11-28T18:23:32.906348+0000","last_active":"2023-11-28T18:23:38.084061+0000","last_peered":"2023-11-28T18:23:38.084061+0000","last_clean":"2023-11-28T18:23:38.084061+0000","last_became_active":"2023-11-28T18:23:32.906007+0000","last_became_peered":"2023-11-28T18:23:32.906007+0000","last_unstale":"2023-11-28T18:23:38.084061+0000","last_undegraded":"2023-11-28T18:23:38.084061+0000","last_fullsized":"2023-11-28T18:23:38.084061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:00.325734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.719440+0000","last_change":"2023-11-28T18:23:38.426316+0000","last_active":"2023-11-28T18:23:40.719440+0000","last_peered":"2023-11-28T18:23:40.719440+0000","last_clean":"2023-11-28T18:23:40.719440+0000","last_became_active":"2023-11-28T18:23:38.425328+0000","last_became_peered":"2023-11-28T18:23:38.425328+0000","last_unstale":"2023-11-28T18:23:40.719440+0000","last_undegraded":"2023-11-28T18:23:40.719440+0000","last_fullsized":"2023-11-28T18:23:40.719440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:19:51.686256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420058+0000","last_change":"2023-11-28T18:23:33.954315+0000","last_active":"2023-11-28T18:23:38.420058+0000","last_peered":"2023-11-28T18:23:38.420058+0000","last_clean":"2023-11-28T18:23:38.420058+0000","last_became_active":"2023-11-28T18:23:33.950270+0000","last_became_peered":"2023-11-28T18:23:33.950270+0000","last_unstale":"2023-11-28T18:23:38.420058+0000","last_undegraded":"2023-11-28T18:23:38.420058+0000","last_fullsized":"2023-11-28T18:23:38.420058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:22:37.364515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077169+0000","last_change":"2023-11-28T18:23:32.921145+0000","last_active":"2023-11-28T18:23:38.077169+0000","last_peered":"2023-11-28T18:23:38.077169+0000","last_clean":"2023-11-28T18:23:38.077169+0000","last_became_active":"2023-11-28T18:23:32.920706+0000","last_became_peered":"2023-11-28T18:23:32.920706+0000","last_unstale":"2023-11-28T18:23:38.077169+0000","last_undegraded":"2023-11-28T18:23:38.077169+0000","last_fullsized":"2023-11-28T18:23:38.077169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:39:52.964875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068294+0000","last_change":"2023-11-28T18:23:38.096089+0000","last_active":"2023-11-28T18:23:39.068294+0000","last_peered":"2023-11-28T18:23:39.068294+0000","last_clean":"2023-11-28T18:23:39.068294+0000","last_became_active":"2023-11-28T18:23:38.095672+0000","last_became_peered":"2023-11-28T18:23:38.095672+0000","last_unstale":"2023-11-28T18:23:39.068294+0000","last_undegraded":"2023-11-28T18:23:39.068294+0000","last_fullsized":"2023-11-28T18:23:39.068294+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:09.933678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419670+0000","last_change":"2023-11-28T18:23:33.954352+0000","last_active":"2023-11-28T18:23:38.419670+0000","last_peered":"2023-11-28T18:23:38.419670+0000","last_clean":"2023-11-28T18:23:38.419670+0000","last_became_active":"2023-11-28T18:23:33.950344+0000","last_became_peered":"2023-11-28T18:23:33.950344+0000","last_unstale":"2023-11-28T18:23:38.419670+0000","last_undegraded":"2023-11-28T18:23:38.419670+0000","last_fullsized":"2023-11-28T18:23:38.419670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:08.033209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.078193+0000","last_change":"2023-11-28T18:23:32.919741+0000","last_active":"2023-11-28T18:23:38.078193+0000","last_peered":"2023-11-28T18:23:38.078193+0000","last_clean":"2023-11-28T18:23:38.078193+0000","last_became_active":"2023-11-28T18:23:32.919438+0000","last_became_peered":"2023-11-28T18:23:32.919438+0000","last_unstale":"2023-11-28T18:23:38.078193+0000","last_undegraded":"2023-11-28T18:23:38.078193+0000","last_fullsized":"2023-11-28T18:23:38.078193+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:39:52.964875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.297438+0000","last_change":"2023-11-28T18:23:38.084984+0000","last_active":"2023-11-28T18:23:40.297438+0000","last_peered":"2023-11-28T18:23:40.297438+0000","last_clean":"2023-11-28T18:23:40.297438+0000","last_became_active":"2023-11-28T18:23:38.084490+0000","last_became_peered":"2023-11-28T18:23:38.084490+0000","last_unstale":"2023-11-28T18:23:40.297438+0000","last_undegraded":"2023-11-28T18:23:40.297438+0000","last_fullsized":"2023-11-28T18:23:40.297438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:00.683839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.073493+0000","last_change":"2023-11-28T18:23:33.939191+0000","last_active":"2023-11-28T18:23:38.073493+0000","last_peered":"2023-11-28T18:23:38.073493+0000","last_clean":"2023-11-28T18:23:38.073493+0000","last_became_active":"2023-11-28T18:23:33.938295+0000","last_became_peered":"2023-11-28T18:23:33.938295+0000","last_unstale":"2023-11-28T18:23:38.073493+0000","last_undegraded":"2023-11-28T18:23:38.073493+0000","last_fullsized":"2023-11-28T18:23:38.073493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:12.557669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.073721+0000","last_change":"2023-11-28T18:23:32.930354+0000","last_active":"2023-11-28T18:23:38.073721+0000","last_peered":"2023-11-28T18:23:38.073721+0000","last_clean":"2023-11-28T18:23:38.073721+0000","last_became_active":"2023-11-28T18:23:32.930059+0000","last_became_peered":"2023-11-28T18:23:32.930059+0000","last_unstale":"2023-11-28T18:23:38.073721+0000","last_undegraded":"2023-11-28T18:23:38.073721+0000","last_fullsized":"2023-11-28T18:23:38.073721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:45.143373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.069210+0000","last_change":"2023-11-28T18:23:38.427077+0000","last_active":"2023-11-28T18:23:39.069210+0000","last_peered":"2023-11-28T18:23:39.069210+0000","last_clean":"2023-11-28T18:23:39.069210+0000","last_became_active":"2023-11-28T18:23:38.426613+0000","last_became_peered":"2023-11-28T18:23:38.426613+0000","last_unstale":"2023-11-28T18:23:39.069210+0000","last_undegraded":"2023-11-28T18:23:39.069210+0000","last_fullsized":"2023-11-28T18:23:39.069210+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:24:11.818911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419843+0000","last_change":"2023-11-28T18:23:33.955340+0000","last_active":"2023-11-28T18:23:38.419843+0000","last_peered":"2023-11-28T18:23:38.419843+0000","last_clean":"2023-11-28T18:23:38.419843+0000","last_became_active":"2023-11-28T18:23:33.955080+0000","last_became_peered":"2023-11-28T18:23:33.955080+0000","last_unstale":"2023-11-28T18:23:38.419843+0000","last_undegraded":"2023-11-28T18:23:38.419843+0000","last_fullsized":"2023-11-28T18:23:38.419843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:42.992322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068633+0000","last_change":"2023-11-28T18:23:32.935530+0000","last_active":"2023-11-28T18:23:38.068633+0000","last_peered":"2023-11-28T18:23:38.068633+0000","last_clean":"2023-11-28T18:23:38.068633+0000","last_became_active":"2023-11-28T18:23:32.934513+0000","last_became_peered":"2023-11-28T18:23:32.934513+0000","last_unstale":"2023-11-28T18:23:38.068633+0000","last_undegraded":"2023-11-28T18:23:38.068633+0000","last_fullsized":"2023-11-28T18:23:38.068633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:10.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068541+0000","last_change":"2023-11-28T18:23:38.421813+0000","last_active":"2023-11-28T18:23:39.068541+0000","last_peered":"2023-11-28T18:23:39.068541+0000","last_clean":"2023-11-28T18:23:39.068541+0000","last_became_active":"2023-11-28T18:23:38.421280+0000","last_became_peered":"2023-11-28T18:23:38.421280+0000","last_unstale":"2023-11-28T18:23:39.068541+0000","last_undegraded":"2023-11-28T18:23:39.068541+0000","last_fullsized":"2023-11-28T18:23:39.068541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:19:05.111022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.080161+0000","last_change":"2023-11-28T18:23:33.946537+0000","last_active":"2023-11-28T18:23:38.080161+0000","last_peered":"2023-11-28T18:23:38.080161+0000","last_clean":"2023-11-28T18:23:38.080161+0000","last_became_active":"2023-11-28T18:23:33.946220+0000","last_became_peered":"2023-11-28T18:23:33.946220+0000","last_unstale":"2023-11-28T18:23:38.080161+0000","last_undegraded":"2023-11-28T18:23:38.080161+0000","last_fullsized":"2023-11-28T18:23:38.080161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:51.652679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077792+0000","last_change":"2023-11-28T18:23:32.922020+0000","last_active":"2023-11-28T18:23:38.077792+0000","last_peered":"2023-11-28T18:23:38.077792+0000","last_clean":"2023-11-28T18:23:38.077792+0000","last_became_active":"2023-11-28T18:23:32.921656+0000","last_became_peered":"2023-11-28T18:23:32.921656+0000","last_unstale":"2023-11-28T18:23:38.077792+0000","last_undegraded":"2023-11-28T18:23:38.077792+0000","last_fullsized":"2023-11-28T18:23:38.077792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:10.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.721071+0000","last_change":"2023-11-28T18:23:38.421823+0000","last_active":"2023-11-28T18:23:40.721071+0000","last_peered":"2023-11-28T18:23:40.721071+0000","last_clean":"2023-11-28T18:23:40.721071+0000","last_became_active":"2023-11-28T18:23:38.421407+0000","last_became_peered":"2023-11-28T18:23:38.421407+0000","last_unstale":"2023-11-28T18:23:40.721071+0000","last_undegraded":"2023-11-28T18:23:40.721071+0000","last_fullsized":"2023-11-28T18:23:40.721071+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:01.885046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419892+0000","last_change":"2023-11-28T18:23:33.939629+0000","last_active":"2023-11-28T18:23:38.419892+0000","last_peered":"2023-11-28T18:23:38.419892+0000","last_clean":"2023-11-28T18:23:38.419892+0000","last_became_active":"2023-11-28T18:23:33.938662+0000","last_became_peered":"2023-11-28T18:23:33.938662+0000","last_unstale":"2023-11-28T18:23:38.419892+0000","last_undegraded":"2023-11-28T18:23:38.419892+0000","last_fullsized":"2023-11-28T18:23:38.419892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:02.989816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:44.598424+0000","last_change":"2023-11-28T18:23:32.923661+0000","last_active":"2023-11-28T18:23:44.598424+0000","last_peered":"2023-11-28T18:23:44.598424+0000","last_clean":"2023-11-28T18:23:44.598424+0000","last_became_active":"2023-11-28T18:23:32.922407+0000","last_became_peered":"2023-11-28T18:23:32.922407+0000","last_unstale":"2023-11-28T18:23:44.598424+0000","last_undegraded":"2023-11-28T18:23:44.598424+0000","last_fullsized":"2023-11-28T18:23:44.598424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:10.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.718701+0000","last_change":"2023-11-28T18:23:38.098761+0000","last_active":"2023-11-28T18:23:40.718701+0000","last_peered":"2023-11-28T18:23:40.718701+0000","last_clean":"2023-11-28T18:23:40.718701+0000","last_became_active":"2023-11-28T18:23:38.098374+0000","last_became_peered":"2023-11-28T18:23:38.098374+0000","last_unstale":"2023-11-28T18:23:40.718701+0000","last_undegraded":"2023-11-28T18:23:40.718701+0000","last_fullsized":"2023-11-28T18:23:40.718701+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:53.274366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075827+0000","last_change":"2023-11-28T18:23:33.945418+0000","last_active":"2023-11-28T18:23:38.075827+0000","last_peered":"2023-11-28T18:23:38.075827+0000","last_clean":"2023-11-28T18:23:38.075827+0000","last_became_active":"2023-11-28T18:23:33.944949+0000","last_became_peered":"2023-11-28T18:23:33.944949+0000","last_unstale":"2023-11-28T18:23:38.075827+0000","last_undegraded":"2023-11-28T18:23:38.075827+0000","last_fullsized":"2023-11-28T18:23:38.075827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:42.826477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.094316+0000","last_change":"2023-11-28T18:23:32.925933+0000","last_active":"2023-11-28T18:23:38.094316+0000","last_peered":"2023-11-28T18:23:38.094316+0000","last_clean":"2023-11-28T18:23:38.094316+0000","last_became_active":"2023-11-28T18:23:32.925619+0000","last_became_peered":"2023-11-28T18:23:32.925619+0000","last_unstale":"2023-11-28T18:23:38.094316+0000","last_undegraded":"2023-11-28T18:23:38.094316+0000","last_fullsized":"2023-11-28T18:23:38.094316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:19.143213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068423+0000","last_change":"2023-11-28T18:23:38.091049+0000","last_active":"2023-11-28T18:23:39.068423+0000","last_peered":"2023-11-28T18:23:39.068423+0000","last_clean":"2023-11-28T18:23:39.068423+0000","last_became_active":"2023-11-28T18:23:38.090236+0000","last_became_peered":"2023-11-28T18:23:38.090236+0000","last_unstale":"2023-11-28T18:23:39.068423+0000","last_undegraded":"2023-11-28T18:23:39.068423+0000","last_fullsized":"2023-11-28T18:23:39.068423+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:12.193604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.076836+0000","last_change":"2023-11-28T18:23:33.948346+0000","last_active":"2023-11-28T18:23:38.076836+0000","last_peered":"2023-11-28T18:23:38.076836+0000","last_clean":"2023-11-28T18:23:38.076836+0000","last_became_active":"2023-11-28T18:23:33.947983+0000","last_became_peered":"2023-11-28T18:23:33.947983+0000","last_unstale":"2023-11-28T18:23:38.076836+0000","last_undegraded":"2023-11-28T18:23:38.076836+0000","last_fullsized":"2023-11-28T18:23:38.076836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:12.557669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067715+0000","last_change":"2023-11-28T18:23:32.935430+0000","last_active":"2023-11-28T18:23:38.067715+0000","last_peered":"2023-11-28T18:23:38.067715+0000","last_clean":"2023-11-28T18:23:38.067715+0000","last_became_active":"2023-11-28T18:23:32.930282+0000","last_became_peered":"2023-11-28T18:23:32.930282+0000","last_unstale":"2023-11-28T18:23:38.067715+0000","last_undegraded":"2023-11-28T18:23:38.067715+0000","last_fullsized":"2023-11-28T18:23:38.067715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:50.646527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.067746+0000","last_change":"2023-11-28T18:23:38.093851+0000","last_active":"2023-11-28T18:23:39.067746+0000","last_peered":"2023-11-28T18:23:39.067746+0000","last_clean":"2023-11-28T18:23:39.067746+0000","last_became_active":"2023-11-28T18:23:38.091779+0000","last_became_peered":"2023-11-28T18:23:38.091779+0000","last_unstale":"2023-11-28T18:23:39.067746+0000","last_undegraded":"2023-11-28T18:23:39.067746+0000","last_fullsized":"2023-11-28T18:23:39.067746+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:13:14.577345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067617+0000","last_change":"2023-11-28T18:23:33.952137+0000","last_active":"2023-11-28T18:23:38.067617+0000","last_peered":"2023-11-28T18:23:38.067617+0000","last_clean":"2023-11-28T18:23:38.067617+0000","last_became_active":"2023-11-28T18:23:33.950857+0000","last_became_peered":"2023-11-28T18:23:33.950857+0000","last_unstale":"2023-11-28T18:23:38.067617+0000","last_undegraded":"2023-11-28T18:23:38.067617+0000","last_fullsized":"2023-11-28T18:23:38.067617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:42.992322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077679+0000","last_change":"2023-11-28T18:23:32.940089+0000","last_active":"2023-11-28T18:23:38.077679+0000","last_peered":"2023-11-28T18:23:38.077679+0000","last_clean":"2023-11-28T18:23:38.077679+0000","last_became_active":"2023-11-28T18:23:32.939524+0000","last_became_peered":"2023-11-28T18:23:32.939524+0000","last_unstale":"2023-11-28T18:23:38.077679+0000","last_undegraded":"2023-11-28T18:23:38.077679+0000","last_fullsized":"2023-11-28T18:23:38.077679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:19.143213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.067936+0000","last_change":"2023-11-28T18:23:38.424675+0000","last_active":"2023-11-28T18:23:39.067936+0000","last_peered":"2023-11-28T18:23:39.067936+0000","last_clean":"2023-11-28T18:23:39.067936+0000","last_became_active":"2023-11-28T18:23:38.424284+0000","last_became_peered":"2023-11-28T18:23:38.424284+0000","last_unstale":"2023-11-28T18:23:39.067936+0000","last_undegraded":"2023-11-28T18:23:39.067936+0000","last_fullsized":"2023-11-28T18:23:39.067936+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:01.885046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.297416+0000","last_change":"2023-11-28T18:23:38.425611+0000","last_active":"2023-11-28T18:23:40.297416+0000","last_peered":"2023-11-28T18:23:40.297416+0000","last_clean":"2023-11-28T18:23:40.297416+0000","last_became_active":"2023-11-28T18:23:38.424826+0000","last_became_peered":"2023-11-28T18:23:38.424826+0000","last_unstale":"2023-11-28T18:23:40.297416+0000","last_undegraded":"2023-11-28T18:23:40.297416+0000","last_fullsized":"2023-11-28T18:23:40.297416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:15.599832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.092623+0000","last_change":"2023-11-28T18:23:33.937161+0000","last_active":"2023-11-28T18:23:38.092623+0000","last_peered":"2023-11-28T18:23:38.092623+0000","last_clean":"2023-11-28T18:23:38.092623+0000","last_became_active":"2023-11-28T18:23:33.936724+0000","last_became_peered":"2023-11-28T18:23:33.936724+0000","last_unstale":"2023-11-28T18:23:38.092623+0000","last_undegraded":"2023-11-28T18:23:38.092623+0000","last_fullsized":"2023-11-28T18:23:38.092623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:17:14.412292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.070788+0000","last_change":"2023-11-28T18:23:32.921502+0000","last_active":"2023-11-28T18:23:38.070788+0000","last_peered":"2023-11-28T18:23:38.070788+0000","last_clean":"2023-11-28T18:23:38.070788+0000","last_became_active":"2023-11-28T18:23:32.920979+0000","last_became_peered":"2023-11-28T18:23:32.920979+0000","last_unstale":"2023-11-28T18:23:38.070788+0000","last_undegraded":"2023-11-28T18:23:38.070788+0000","last_fullsized":"2023-11-28T18:23:38.070788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:41.820325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.719783+0000","last_change":"2023-11-28T18:23:38.095346+0000","last_active":"2023-11-28T18:23:40.719783+0000","last_peered":"2023-11-28T18:23:40.719783+0000","last_clean":"2023-11-28T18:23:40.719783+0000","last_became_active":"2023-11-28T18:23:38.094985+0000","last_became_peered":"2023-11-28T18:23:38.094985+0000","last_unstale":"2023-11-28T18:23:40.719783+0000","last_undegraded":"2023-11-28T18:23:40.719783+0000","last_fullsized":"2023-11-28T18:23:40.719783+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:22:41.524911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065411+0000","last_change":"2023-11-28T18:23:33.934122+0000","last_active":"2023-11-28T18:23:38.065411+0000","last_peered":"2023-11-28T18:23:38.065411+0000","last_clean":"2023-11-28T18:23:38.065411+0000","last_became_active":"2023-11-28T18:23:33.933687+0000","last_became_peered":"2023-11-28T18:23:33.933687+0000","last_unstale":"2023-11-28T18:23:38.065411+0000","last_undegraded":"2023-11-28T18:23:38.065411+0000","last_fullsized":"2023-11-28T18:23:38.065411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:17:14.412292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065550+0000","last_change":"2023-11-28T18:23:32.931872+0000","last_active":"2023-11-28T18:23:38.065550+0000","last_peered":"2023-11-28T18:23:38.065550+0000","last_clean":"2023-11-28T18:23:38.065550+0000","last_became_active":"2023-11-28T18:23:32.931111+0000","last_became_peered":"2023-11-28T18:23:32.931111+0000","last_unstale":"2023-11-28T18:23:38.065550+0000","last_undegraded":"2023-11-28T18:23:38.065550+0000","last_fullsized":"2023-11-28T18:23:38.065550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:41.820325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.074084+0000","last_change":"2023-11-28T18:23:32.934258+0000","last_active":"2023-11-28T18:23:38.074084+0000","last_peered":"2023-11-28T18:23:38.074084+0000","last_clean":"2023-11-28T18:23:38.074084+0000","last_became_active":"2023-11-28T18:23:32.933956+0000","last_became_peered":"2023-11-28T18:23:32.933956+0000","last_unstale":"2023-11-28T18:23:38.074084+0000","last_undegraded":"2023-11-28T18:23:38.074084+0000","last_fullsized":"2023-11-28T18:23:38.074084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:41.820325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077709+0000","last_change":"2023-11-28T18:23:33.949218+0000","last_active":"2023-11-28T18:23:38.077709+0000","last_peered":"2023-11-28T18:23:38.077709+0000","last_clean":"2023-11-28T18:23:38.077709+0000","last_became_active":"2023-11-28T18:23:33.948930+0000","last_became_peered":"2023-11-28T18:23:33.948930+0000","last_unstale":"2023-11-28T18:23:38.077709+0000","last_undegraded":"2023-11-28T18:23:38.077709+0000","last_fullsized":"2023-11-28T18:23:38.077709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:46.421541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.067937+0000","last_change":"2023-11-28T18:23:38.095800+0000","last_active":"2023-11-28T18:23:39.067937+0000","last_peered":"2023-11-28T18:23:39.067937+0000","last_clean":"2023-11-28T18:23:39.067937+0000","last_became_active":"2023-11-28T18:23:38.095437+0000","last_became_peered":"2023-11-28T18:23:38.095437+0000","last_unstale":"2023-11-28T18:23:39.067937+0000","last_undegraded":"2023-11-28T18:23:39.067937+0000","last_fullsized":"2023-11-28T18:23:39.067937+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:39.644595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065704+0000","last_change":"2023-11-28T18:23:32.921448+0000","last_active":"2023-11-28T18:23:38.065704+0000","last_peered":"2023-11-28T18:23:38.065704+0000","last_clean":"2023-11-28T18:23:38.065704+0000","last_became_active":"2023-11-28T18:23:32.921183+0000","last_became_peered":"2023-11-28T18:23:32.921183+0000","last_unstale":"2023-11-28T18:23:38.065704+0000","last_undegraded":"2023-11-28T18:23:38.065704+0000","last_fullsized":"2023-11-28T18:23:38.065704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:36:09.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075654+0000","last_change":"2023-11-28T18:23:33.937197+0000","last_active":"2023-11-28T18:23:38.075654+0000","last_peered":"2023-11-28T18:23:38.075654+0000","last_clean":"2023-11-28T18:23:38.075654+0000","last_became_active":"2023-11-28T18:23:33.936922+0000","last_became_peered":"2023-11-28T18:23:33.936922+0000","last_unstale":"2023-11-28T18:23:38.075654+0000","last_undegraded":"2023-11-28T18:23:38.075654+0000","last_fullsized":"2023-11-28T18:23:38.075654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:01.331886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068617+0000","last_change":"2023-11-28T18:23:38.099275+0000","last_active":"2023-11-28T18:23:39.068617+0000","last_peered":"2023-11-28T18:23:39.068617+0000","last_clean":"2023-11-28T18:23:39.068617+0000","last_became_active":"2023-11-28T18:23:38.095544+0000","last_became_peered":"2023-11-28T18:23:38.095544+0000","last_unstale":"2023-11-28T18:23:39.068617+0000","last_undegraded":"2023-11-28T18:23:39.068617+0000","last_fullsized":"2023-11-28T18:23:39.068617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:04.943189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.073386+0000","last_change":"2023-11-28T18:23:33.943034+0000","last_active":"2023-11-28T18:23:38.073386+0000","last_peered":"2023-11-28T18:23:38.073386+0000","last_clean":"2023-11-28T18:23:38.073386+0000","last_became_active":"2023-11-28T18:23:33.942687+0000","last_became_peered":"2023-11-28T18:23:33.942687+0000","last_unstale":"2023-11-28T18:23:38.073386+0000","last_undegraded":"2023-11-28T18:23:38.073386+0000","last_fullsized":"2023-11-28T18:23:38.073386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:35.484199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.083621+0000","last_change":"2023-11-28T18:23:33.942958+0000","last_active":"2023-11-28T18:23:38.083621+0000","last_peered":"2023-11-28T18:23:38.083621+0000","last_clean":"2023-11-28T18:23:38.083621+0000","last_became_active":"2023-11-28T18:23:33.942696+0000","last_became_peered":"2023-11-28T18:23:33.942696+0000","last_unstale":"2023-11-28T18:23:38.083621+0000","last_undegraded":"2023-11-28T18:23:38.083621+0000","last_fullsized":"2023-11-28T18:23:38.083621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:42.826477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068550+0000","last_change":"2023-11-28T18:23:33.927110+0000","last_active":"2023-11-28T18:23:38.068550+0000","last_peered":"2023-11-28T18:23:38.068550+0000","last_clean":"2023-11-28T18:23:38.068550+0000","last_became_active":"2023-11-28T18:23:33.926812+0000","last_became_peered":"2023-11-28T18:23:33.926812+0000","last_unstale":"2023-11-28T18:23:38.068550+0000","last_undegraded":"2023-11-28T18:23:38.068550+0000","last_fullsized":"2023-11-28T18:23:38.068550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:22:37.364515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077616+0000","last_change":"2023-11-28T18:23:33.941141+0000","last_active":"2023-11-28T18:23:38.077616+0000","last_peered":"2023-11-28T18:23:38.077616+0000","last_clean":"2023-11-28T18:23:38.077616+0000","last_became_active":"2023-11-28T18:23:33.940832+0000","last_became_peered":"2023-11-28T18:23:33.940832+0000","last_unstale":"2023-11-28T18:23:38.077616+0000","last_undegraded":"2023-11-28T18:23:38.077616+0000","last_fullsized":"2023-11-28T18:23:38.077616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:04:56.118542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.066054+0000","last_change":"2023-11-28T18:23:33.924784+0000","last_active":"2023-11-28T18:23:38.066054+0000","last_peered":"2023-11-28T18:23:38.066054+0000","last_clean":"2023-11-28T18:23:38.066054+0000","last_became_active":"2023-11-28T18:23:33.924475+0000","last_became_peered":"2023-11-28T18:23:33.924475+0000","last_unstale":"2023-11-28T18:23:38.066054+0000","last_undegraded":"2023-11-28T18:23:38.066054+0000","last_fullsized":"2023-11-28T18:23:38.066054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:05.773283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.094179+0000","last_change":"2023-11-28T18:23:33.932579+0000","last_active":"2023-11-28T18:23:38.094179+0000","last_peered":"2023-11-28T18:23:38.094179+0000","last_clean":"2023-11-28T18:23:38.094179+0000","last_became_active":"2023-11-28T18:23:33.932201+0000","last_became_peered":"2023-11-28T18:23:33.932201+0000","last_unstale":"2023-11-28T18:23:38.094179+0000","last_undegraded":"2023-11-28T18:23:38.094179+0000","last_fullsized":"2023-11-28T18:23:38.094179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:01.912695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077881+0000","last_change":"2023-11-28T18:23:33.954178+0000","last_active":"2023-11-28T18:23:38.077881+0000","last_peered":"2023-11-28T18:23:38.077881+0000","last_clean":"2023-11-28T18:23:38.077881+0000","last_became_active":"2023-11-28T18:23:33.954020+0000","last_became_peered":"2023-11-28T18:23:33.954020+0000","last_unstale":"2023-11-28T18:23:38.077881+0000","last_undegraded":"2023-11-28T18:23:38.077881+0000","last_fullsized":"2023-11-28T18:23:38.077881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:01.912695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.066974+0000","last_change":"2023-11-28T18:23:33.939474+0000","last_active":"2023-11-28T18:23:38.066974+0000","last_peered":"2023-11-28T18:23:38.066974+0000","last_clean":"2023-11-28T18:23:38.066974+0000","last_became_active":"2023-11-28T18:23:33.939189+0000","last_became_peered":"2023-11-28T18:23:33.939189+0000","last_unstale":"2023-11-28T18:23:38.066974+0000","last_undegraded":"2023-11-28T18:23:38.066974+0000","last_fullsized":"2023-11-28T18:23:38.066974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:08.033209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.070751+0000","last_change":"2023-11-28T18:23:33.929306+0000","last_active":"2023-11-28T18:23:38.070751+0000","last_peered":"2023-11-28T18:23:38.070751+0000","last_clean":"2023-11-28T18:23:38.070751+0000","last_became_active":"2023-11-28T18:23:33.929029+0000","last_became_peered":"2023-11-28T18:23:33.929029+0000","last_unstale":"2023-11-28T18:23:38.070751+0000","last_undegraded":"2023-11-28T18:23:38.070751+0000","last_fullsized":"2023-11-28T18:23:38.070751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:46.421541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065454+0000","last_change":"2023-11-28T18:23:33.922789+0000","last_active":"2023-11-28T18:23:38.065454+0000","last_peered":"2023-11-28T18:23:38.065454+0000","last_clean":"2023-11-28T18:23:38.065454+0000","last_became_active":"2023-11-28T18:23:33.922458+0000","last_became_peered":"2023-11-28T18:23:33.922458+0000","last_unstale":"2023-11-28T18:23:38.065454+0000","last_undegraded":"2023-11-28T18:23:38.065454+0000","last_fullsized":"2023-11-28T18:23:38.065454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:51.652679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068465+0000","last_change":"2023-11-28T18:23:33.935519+0000","last_active":"2023-11-28T18:23:38.068465+0000","last_peered":"2023-11-28T18:23:38.068465+0000","last_clean":"2023-11-28T18:23:38.068465+0000","last_became_active":"2023-11-28T18:23:33.934779+0000","last_became_peered":"2023-11-28T18:23:33.934779+0000","last_unstale":"2023-11-28T18:23:38.068465+0000","last_undegraded":"2023-11-28T18:23:38.068465+0000","last_fullsized":"2023-11-28T18:23:38.068465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:27.506820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077522+0000","last_change":"2023-11-28T18:23:33.946486+0000","last_active":"2023-11-28T18:23:38.077522+0000","last_peered":"2023-11-28T18:23:38.077522+0000","last_clean":"2023-11-28T18:23:38.077522+0000","last_became_active":"2023-11-28T18:23:33.946212+0000","last_became_peered":"2023-11-28T18:23:33.946212+0000","last_unstale":"2023-11-28T18:23:38.077522+0000","last_undegraded":"2023-11-28T18:23:38.077522+0000","last_fullsized":"2023-11-28T18:23:38.077522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:17:14.412292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.076300+0000","last_change":"2023-11-28T18:23:33.949577+0000","last_active":"2023-11-28T18:23:38.076300+0000","last_peered":"2023-11-28T18:23:38.076300+0000","last_clean":"2023-11-28T18:23:38.076300+0000","last_became_active":"2023-11-28T18:23:33.949275+0000","last_became_peered":"2023-11-28T18:23:33.949275+0000","last_unstale":"2023-11-28T18:23:38.076300+0000","last_undegraded":"2023-11-28T18:23:38.076300+0000","last_fullsized":"2023-11-28T18:23:38.076300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:17:14.412292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420401+0000","last_change":"2023-11-28T18:23:33.939143+0000","last_active":"2023-11-28T18:23:38.420401+0000","last_peered":"2023-11-28T18:23:38.420401+0000","last_clean":"2023-11-28T18:23:38.420401+0000","last_became_active":"2023-11-28T18:23:33.938673+0000","last_became_peered":"2023-11-28T18:23:33.938673+0000","last_unstale":"2023-11-28T18:23:38.420401+0000","last_undegraded":"2023-11-28T18:23:38.420401+0000","last_fullsized":"2023-11-28T18:23:38.420401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:11.439436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.083975+0000","last_change":"2023-11-28T18:23:33.946701+0000","last_active":"2023-11-28T18:23:38.083975+0000","last_peered":"2023-11-28T18:23:38.083975+0000","last_clean":"2023-11-28T18:23:38.083975+0000","last_became_active":"2023-11-28T18:23:33.946460+0000","last_became_peered":"2023-11-28T18:23:33.946460+0000","last_unstale":"2023-11-28T18:23:38.083975+0000","last_undegraded":"2023-11-28T18:23:38.083975+0000","last_fullsized":"2023-11-28T18:23:38.083975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:11.792676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068495+0000","last_change":"2023-11-28T18:23:33.940275+0000","last_active":"2023-11-28T18:23:38.068495+0000","last_peered":"2023-11-28T18:23:38.068495+0000","last_clean":"2023-11-28T18:23:38.068495+0000","last_became_active":"2023-11-28T18:23:33.939937+0000","last_became_peered":"2023-11-28T18:23:33.939937+0000","last_unstale":"2023-11-28T18:23:38.068495+0000","last_undegraded":"2023-11-28T18:23:38.068495+0000","last_fullsized":"2023-11-28T18:23:38.068495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:11.439436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.070691+0000","last_change":"2023-11-28T18:23:33.937355+0000","last_active":"2023-11-28T18:23:38.070691+0000","last_peered":"2023-11-28T18:23:38.070691+0000","last_clean":"2023-11-28T18:23:38.070691+0000","last_became_active":"2023-11-28T18:23:33.937069+0000","last_became_peered":"2023-11-28T18:23:33.937069+0000","last_unstale":"2023-11-28T18:23:38.070691+0000","last_undegraded":"2023-11-28T18:23:38.070691+0000","last_fullsized":"2023-11-28T18:23:38.070691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:05.773283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.092623+0000","last_change":"2023-11-28T18:23:33.938555+0000","last_active":"2023-11-28T18:23:38.092623+0000","last_peered":"2023-11-28T18:23:38.092623+0000","last_clean":"2023-11-28T18:23:38.092623+0000","last_became_active":"2023-11-28T18:23:33.938297+0000","last_became_peered":"2023-11-28T18:23:33.938297+0000","last_unstale":"2023-11-28T18:23:38.092623+0000","last_undegraded":"2023-11-28T18:23:38.092623+0000","last_fullsized":"2023-11-28T18:23:38.092623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:51.652679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.080082+0000","last_change":"2023-11-28T18:23:33.942250+0000","last_active":"2023-11-28T18:23:38.080082+0000","last_peered":"2023-11-28T18:23:38.080082+0000","last_clean":"2023-11-28T18:23:38.080082+0000","last_became_active":"2023-11-28T18:23:33.941935+0000","last_became_peered":"2023-11-28T18:23:33.941935+0000","last_unstale":"2023-11-28T18:23:38.080082+0000","last_undegraded":"2023-11-28T18:23:38.080082+0000","last_fullsized":"2023-11-28T18:23:38.080082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:21.035422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065409+0000","last_change":"2023-11-28T18:23:33.944983+0000","last_active":"2023-11-28T18:23:38.065409+0000","last_peered":"2023-11-28T18:23:38.065409+0000","last_clean":"2023-11-28T18:23:38.065409+0000","last_became_active":"2023-11-28T18:23:33.944535+0000","last_became_peered":"2023-11-28T18:23:33.944535+0000","last_unstale":"2023-11-28T18:23:38.065409+0000","last_undegraded":"2023-11-28T18:23:38.065409+0000","last_fullsized":"2023-11-28T18:23:38.065409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:01.912695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075790+0000","last_change":"2023-11-28T18:23:33.945213+0000","last_active":"2023-11-28T18:23:38.075790+0000","last_peered":"2023-11-28T18:23:38.075790+0000","last_clean":"2023-11-28T18:23:38.075790+0000","last_became_active":"2023-11-28T18:23:33.944605+0000","last_became_peered":"2023-11-28T18:23:33.944605+0000","last_unstale":"2023-11-28T18:23:38.075790+0000","last_undegraded":"2023-11-28T18:23:38.075790+0000","last_fullsized":"2023-11-28T18:23:38.075790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:46.149525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.076131+0000","last_change":"2023-11-28T18:23:33.948453+0000","last_active":"2023-11-28T18:23:38.076131+0000","last_peered":"2023-11-28T18:23:38.076131+0000","last_clean":"2023-11-28T18:23:38.076131+0000","last_became_active":"2023-11-28T18:23:33.948203+0000","last_became_peered":"2023-11-28T18:23:33.948203+0000","last_unstale":"2023-11-28T18:23:38.076131+0000","last_undegraded":"2023-11-28T18:23:38.076131+0000","last_fullsized":"2023-11-28T18:23:38.076131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:51.652679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077793+0000","last_change":"2023-11-28T18:23:33.945564+0000","last_active":"2023-11-28T18:23:38.077793+0000","last_peered":"2023-11-28T18:23:38.077793+0000","last_clean":"2023-11-28T18:23:38.077793+0000","last_became_active":"2023-11-28T18:23:33.943276+0000","last_became_peered":"2023-11-28T18:23:33.943276+0000","last_unstale":"2023-11-28T18:23:38.077793+0000","last_undegraded":"2023-11-28T18:23:38.077793+0000","last_fullsized":"2023-11-28T18:23:38.077793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:21.035422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420502+0000","last_change":"2023-11-28T18:23:33.942649+0000","last_active":"2023-11-28T18:23:38.420502+0000","last_peered":"2023-11-28T18:23:38.420502+0000","last_clean":"2023-11-28T18:23:38.420502+0000","last_became_active":"2023-11-28T18:23:33.942109+0000","last_became_peered":"2023-11-28T18:23:33.942109+0000","last_unstale":"2023-11-28T18:23:38.420502+0000","last_undegraded":"2023-11-28T18:23:38.420502+0000","last_fullsized":"2023-11-28T18:23:38.420502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:34.989175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.064784+0000","last_change":"2023-11-28T18:23:33.933929+0000","last_active":"2023-11-28T18:23:38.064784+0000","last_peered":"2023-11-28T18:23:38.064784+0000","last_clean":"2023-11-28T18:23:38.064784+0000","last_became_active":"2023-11-28T18:23:33.933087+0000","last_became_peered":"2023-11-28T18:23:33.933087+0000","last_unstale":"2023-11-28T18:23:38.064784+0000","last_undegraded":"2023-11-28T18:23:38.064784+0000","last_fullsized":"2023-11-28T18:23:38.064784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:12.557669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419632+0000","last_change":"2023-11-28T18:23:33.950276+0000","last_active":"2023-11-28T18:23:38.419632+0000","last_peered":"2023-11-28T18:23:38.419632+0000","last_clean":"2023-11-28T18:23:38.419632+0000","last_became_active":"2023-11-28T18:23:33.946777+0000","last_became_peered":"2023-11-28T18:23:33.946777+0000","last_unstale":"2023-11-28T18:23:38.419632+0000","last_undegraded":"2023-11-28T18:23:38.419632+0000","last_fullsized":"2023-11-28T18:23:38.419632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:46.421541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068159+0000","last_change":"2023-11-28T18:23:33.943666+0000","last_active":"2023-11-28T18:23:38.068159+0000","last_peered":"2023-11-28T18:23:38.068159+0000","last_clean":"2023-11-28T18:23:38.068159+0000","last_became_active":"2023-11-28T18:23:33.940798+0000","last_became_peered":"2023-11-28T18:23:33.940798+0000","last_unstale":"2023-11-28T18:23:38.068159+0000","last_undegraded":"2023-11-28T18:23:38.068159+0000","last_fullsized":"2023-11-28T18:23:38.068159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:04:56.118542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419726+0000","last_change":"2023-11-28T18:23:33.955086+0000","last_active":"2023-11-28T18:23:38.419726+0000","last_peered":"2023-11-28T18:23:38.419726+0000","last_clean":"2023-11-28T18:23:38.419726+0000","last_became_active":"2023-11-28T18:23:33.950311+0000","last_became_peered":"2023-11-28T18:23:33.950311+0000","last_unstale":"2023-11-28T18:23:38.419726+0000","last_undegraded":"2023-11-28T18:23:38.419726+0000","last_fullsized":"2023-11-28T18:23:38.419726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:27.506820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067660+0000","last_change":"2023-11-28T18:23:33.932833+0000","last_active":"2023-11-28T18:23:38.067660+0000","last_peered":"2023-11-28T18:23:38.067660+0000","last_clean":"2023-11-28T18:23:38.067660+0000","last_became_active":"2023-11-28T18:23:33.932554+0000","last_became_peered":"2023-11-28T18:23:33.932554+0000","last_unstale":"2023-11-28T18:23:38.067660+0000","last_undegraded":"2023-11-28T18:23:38.067660+0000","last_fullsized":"2023-11-28T18:23:38.067660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:08.033209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075353+0000","last_change":"2023-11-28T18:23:33.939136+0000","last_active":"2023-11-28T18:23:38.075353+0000","last_peered":"2023-11-28T18:23:38.075353+0000","last_clean":"2023-11-28T18:23:38.075353+0000","last_became_active":"2023-11-28T18:23:33.938870+0000","last_became_peered":"2023-11-28T18:23:33.938870+0000","last_unstale":"2023-11-28T18:23:38.075353+0000","last_undegraded":"2023-11-28T18:23:38.075353+0000","last_fullsized":"2023-11-28T18:23:38.075353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:05.773283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.096250+0000","last_change":"2023-11-28T18:23:33.936563+0000","last_active":"2023-11-28T18:23:38.096250+0000","last_peered":"2023-11-28T18:23:38.096250+0000","last_clean":"2023-11-28T18:23:38.096250+0000","last_became_active":"2023-11-28T18:23:33.936272+0000","last_became_peered":"2023-11-28T18:23:33.936272+0000","last_unstale":"2023-11-28T18:23:38.096250+0000","last_undegraded":"2023-11-28T18:23:38.096250+0000","last_fullsized":"2023-11-28T18:23:38.096250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:04:56.118542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420445+0000","last_change":"2023-11-28T18:23:33.954685+0000","last_active":"2023-11-28T18:23:38.420445+0000","last_peered":"2023-11-28T18:23:38.420445+0000","last_clean":"2023-11-28T18:23:38.420445+0000","last_became_active":"2023-11-28T18:23:33.954434+0000","last_became_peered":"2023-11-28T18:23:33.954434+0000","last_unstale":"2023-11-28T18:23:38.420445+0000","last_undegraded":"2023-11-28T18:23:38.420445+0000","last_fullsized":"2023-11-28T18:23:38.420445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:38.277787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":690742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.427}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3849999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4590000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2730000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":99882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":99918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":99792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":99952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":690798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:23:56.769 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:23:56.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.943+0000 7fb748c17640 1 Processor -- start 2023-11-28T18:23:56.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.943+0000 7fb748c17640 1 -- start start 2023-11-28T18:23:56.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.947+0000 7fb748c17640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb74415d320 0x7fb74406c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.947+0000 7fb748c17640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7440b3c40 con 0x7fb7440ce230 2023-11-28T18:23:56.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.947+0000 7fb748c17640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb74406ca80 con 0x7fb7440cdd80 2023-11-28T18:23:56.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.947+0000 7fb748c17640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb74415d700 con 0x7fb74415d320 2023-11-28T18:23:56.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.947+0000 7fb739d9b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb7440ce230 0x7fb7440cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39328/0 (socket says 172.21.15.105:39328) 2023-11-28T18:23:56.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.947+0000 7fb739d9b640 1 -- 172.21.15.105:0/54928452 learned_addr learned my addr 172.21.15.105:0/54928452 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:56.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.947+0000 7fb74259c640 1 --2- 172.21.15.105:0/54928452 >> v2:172.21.15.148:3300/0 conn(0x7fb74415d320 0x7fb74406c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.947+0000 7fb741d9b640 1 -- 172.21.15.105:0/54928452 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3744931187 0 0) 0x7fb7440b3c40 con 0x7fb7440ce230 2023-11-28T18:23:56.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.947+0000 7fb741d9b640 1 -- 172.21.15.105:0/54928452 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb730003530 con 0x7fb7440ce230 2023-11-28T18:23:56.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.947+0000 7fb74259c640 1 -- 172.21.15.105:0/54928452 >> v1:172.21.15.148:6789/0 conn(0x7fb7440cdd80 legacy=0x7fb7440cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.947+0000 7fb74259c640 1 -- 172.21.15.105:0/54928452 >> v1:172.21.15.105:6789/0 conn(0x7fb7440ce230 legacy=0x7fb7440cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.947+0000 7fb74259c640 1 -- 172.21.15.105:0/54928452 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7440ce6c0 con 0x7fb74415d320 2023-11-28T18:23:56.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.951+0000 7fb74259c640 1 --2- 172.21.15.105:0/54928452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb74415d320 0x7fb74406c0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb7340097c0 tx=0x7fb73401c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc79b6dfc3c64b45 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.951+0000 7fb741d9b640 1 -- 172.21.15.105:0/54928452 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7340ab5f0 con 0x7fb74415d320 2023-11-28T18:23:56.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.951+0000 7fb741d9b640 1 -- 172.21.15.105:0/54928452 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7340ab750 con 0x7fb74415d320 2023-11-28T18:23:56.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.951+0000 7fb741d9b640 1 -- 172.21.15.105:0/54928452 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7340abae0 con 0x7fb74415d320 2023-11-28T18:23:56.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.951+0000 7fb748c17640 1 -- 172.21.15.105:0/54928452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb74415d320 msgr2=0x7fb74406c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.951+0000 7fb748c17640 1 --2- 172.21.15.105:0/54928452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb74415d320 0x7fb74406c0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb7340097c0 tx=0x7fb73401c3f0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.951+0000 7fb748c17640 1 -- 172.21.15.105:0/54928452 shutdown_connections 2023-11-28T18:23:56.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.951+0000 7fb748c17640 1 --2- 172.21.15.105:0/54928452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb74415d320 0x7fb74406c0a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.951+0000 7fb748c17640 1 -- 172.21.15.105:0/54928452 >> 172.21.15.105:0/54928452 conn(0x7fb7440d0df0 msgr2=0x7fb7440d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:56.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.951+0000 7fb748c17640 1 -- 172.21.15.105:0/54928452 shutdown_connections 2023-11-28T18:23:56.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.951+0000 7fb748c17640 1 -- 172.21.15.105:0/54928452 wait complete. 2023-11-28T18:23:56.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.951+0000 7fb748c17640 1 Processor -- start 2023-11-28T18:23:56.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.951+0000 7fb748c17640 1 -- start start 2023-11-28T18:23:56.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb748c17640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7440cdd80 0x7fb7441602a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb748c17640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7440ce230 0x7fb7441519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb748c17640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb74415d320 0x7fb744154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb748c17640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7440b43a0 con 0x7fb7440ce230 2023-11-28T18:23:56.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb748c17640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7441607e0 con 0x7fb74415d320 2023-11-28T18:23:56.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb748c17640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7440ca420 con 0x7fb7440cdd80 2023-11-28T18:23:56.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb739d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7440ce230 0x7fb7441519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb739d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7440ce230 0x7fb7441519b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55350/0 (socket says 172.21.15.105:55350) 2023-11-28T18:23:56.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb74259c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7440cdd80 0x7fb7441602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb739d9b640 1 -- 172.21.15.105:0/1869785059 learned_addr learned my addr 172.21.15.105:0/1869785059 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:56.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb742d9d640 1 --2- 172.21.15.105:0/1869785059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb74415d320 0x7fb744154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb739d9b640 1 -- 172.21.15.105:0/1869785059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7440cdd80 msgr2=0x7fb7441602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb739d9b640 1 --2- 172.21.15.105:0/1869785059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7440cdd80 0x7fb7441602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb739d9b640 1 -- 172.21.15.105:0/1869785059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb74415d320 msgr2=0x7fb744154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:56.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb739d9b640 1 --2- 172.21.15.105:0/1869785059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb74415d320 0x7fb744154200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:56.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb739d9b640 1 -- 172.21.15.105:0/1869785059 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb734009420 con 0x7fb7440ce230 2023-11-28T18:23:56.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb739d9b640 1 --2- 172.21.15.105:0/1869785059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7440ce230 0x7fb7441519b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00cbc0 tx=0x7fb72c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb73b7fe640 1 -- 172.21.15.105:0/1869785059 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c00ad40 con 0x7fb7440ce230 2023-11-28T18:23:56.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb73b7fe640 1 -- 172.21.15.105:0/1869785059 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72c012400 con 0x7fb7440ce230 2023-11-28T18:23:56.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb748c17640 1 -- 172.21.15.105:0/1869785059 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7440ca140 con 0x7fb7440ce230 2023-11-28T18:23:56.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb73b7fe640 1 -- 172.21.15.105:0/1869785059 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c00aea0 con 0x7fb7440ce230 2023-11-28T18:23:56.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb748c17640 1 -- 172.21.15.105:0/1869785059 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7440b3fc0 con 0x7fb7440ce230 2023-11-28T18:23:56.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb748c17640 1 -- 172.21.15.105:0/1869785059 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7440b44e0 con 0x7fb7440ce230 2023-11-28T18:23:56.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.955+0000 7fb73b7fe640 1 -- 172.21.15.105:0/1869785059 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fb72c007070 con 0x7fb7440ce230 2023-11-28T18:23:56.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.959+0000 7fb73b7fe640 1 --2- 172.21.15.105:0/1869785059 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb7240668d0 0x7fb724068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:56.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.959+0000 7fb73b7fe640 1 -- 172.21.15.105:0/1869785059 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb72c0846b0 con 0x7fb7440ce230 2023-11-28T18:23:56.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.959+0000 7fb74259c640 1 --2- 172.21.15.105:0/1869785059 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb7240668d0 0x7fb724068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:56.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.959+0000 7fb74259c640 1 --2- 172.21.15.105:0/1869785059 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb7240668d0 0x7fb724068d90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb7340af000 tx=0x7fb7340a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:56.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:56.963+0000 7fb73b7fe640 1 -- 172.21.15.105:0/1869785059 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb7440b44e0 con 0x7fb7440ce230 2023-11-28T18:23:57.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.111+0000 7fb748c17640 1 -- 172.21.15.105:0/1869785059 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb7440cbba0 con 0x7fb7240668d0 2023-11-28T18:23:57.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.119+0000 7fb73b7fe640 1 -- 172.21.15.105:0/1869785059 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345252 (secure 0 0 0) 0x7fb734006e20 con 0x7fb7240668d0 2023-11-28T18:23:57.125 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:57.127 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:23:57.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.123+0000 7fb748c17640 1 -- 172.21.15.105:0/1869785059 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb7240668d0 msgr2=0x7fb724068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:57.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.123+0000 7fb748c17640 1 --2- 172.21.15.105:0/1869785059 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb7240668d0 0x7fb724068d90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb7340af000 tx=0x7fb7340a7000 comp rx=0 tx=0).stop 2023-11-28T18:23:57.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.123+0000 7fb748c17640 1 -- 172.21.15.105:0/1869785059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7440ce230 msgr2=0x7fb7441519b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:57.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.123+0000 7fb748c17640 1 --2- 172.21.15.105:0/1869785059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7440ce230 0x7fb7441519b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00cbc0 tx=0x7fb72c00a430 comp rx=0 tx=0).stop 2023-11-28T18:23:57.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.123+0000 7fb748c17640 1 -- 172.21.15.105:0/1869785059 shutdown_connections 2023-11-28T18:23:57.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.123+0000 7fb748c17640 1 --2- 172.21.15.105:0/1869785059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb74415d320 0x7fb744154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:57.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.123+0000 7fb748c17640 1 --2- 172.21.15.105:0/1869785059 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb7240668d0 0x7fb724068d90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:57.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.123+0000 7fb748c17640 1 --2- 172.21.15.105:0/1869785059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7440ce230 0x7fb7441519b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:57.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.123+0000 7fb748c17640 1 --2- 172.21.15.105:0/1869785059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7440cdd80 0x7fb7441602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:57.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.123+0000 7fb748c17640 1 -- 172.21.15.105:0/1869785059 >> 172.21.15.105:0/1869785059 conn(0x7fb7440d0df0 msgr2=0x7fb744158930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:57.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.123+0000 7fb748c17640 1 -- 172.21.15.105:0/1869785059 shutdown_connections 2023-11-28T18:23:57.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.123+0000 7fb748c17640 1 -- 172.21.15.105:0/1869785059 wait complete. 2023-11-28T18:23:57.145 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2023-11-28T18:23:55.273505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219380,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755340,"statfs":{"total":773094113280,"available":772869468160,"internally_reserved":0,"allocated":3284992,"data_stored":1988350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2229,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002522"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.076931+0000","last_change":"2023-11-28T18:23:32.930693+0000","last_active":"2023-11-28T18:23:38.076931+0000","last_peered":"2023-11-28T18:23:38.076931+0000","last_clean":"2023-11-28T18:23:38.076931+0000","last_became_active":"2023-11-28T18:23:32.930410+0000","last_became_peered":"2023-11-28T18:23:32.930410+0000","last_unstale":"2023-11-28T18:23:38.076931+0000","last_undegraded":"2023-11-28T18:23:38.076931+0000","last_fullsized":"2023-11-28T18:23:38.076931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:19.143213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.066696+0000","last_change":"2023-11-28T18:23:33.947291+0000","last_active":"2023-11-28T18:23:38.066696+0000","last_peered":"2023-11-28T18:23:38.066696+0000","last_clean":"2023-11-28T18:23:38.066696+0000","last_became_active":"2023-11-28T18:23:33.946963+0000","last_became_peered":"2023-11-28T18:23:33.946963+0000","last_unstale":"2023-11-28T18:23:38.066696+0000","last_undegraded":"2023-11-28T18:23:38.066696+0000","last_fullsized":"2023-11-28T18:23:38.066696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:27.506820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.067797+0000","last_change":"2023-11-28T18:23:38.425549+0000","last_active":"2023-11-28T18:23:39.067797+0000","last_peered":"2023-11-28T18:23:39.067797+0000","last_clean":"2023-11-28T18:23:39.067797+0000","last_became_active":"2023-11-28T18:23:38.425130+0000","last_became_peered":"2023-11-28T18:23:38.425130+0000","last_unstale":"2023-11-28T18:23:39.067797+0000","last_undegraded":"2023-11-28T18:23:39.067797+0000","last_fullsized":"2023-11-28T18:23:39.067797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:19:51.686256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.066904+0000","last_change":"2023-11-28T18:23:32.921328+0000","last_active":"2023-11-28T18:23:38.066904+0000","last_peered":"2023-11-28T18:23:38.066904+0000","last_clean":"2023-11-28T18:23:38.066904+0000","last_became_active":"2023-11-28T18:23:32.920775+0000","last_became_peered":"2023-11-28T18:23:32.920775+0000","last_unstale":"2023-11-28T18:23:38.066904+0000","last_undegraded":"2023-11-28T18:23:38.066904+0000","last_fullsized":"2023-11-28T18:23:38.066904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:05:45.779946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075403+0000","last_change":"2023-11-28T18:23:33.948349+0000","last_active":"2023-11-28T18:23:38.075403+0000","last_peered":"2023-11-28T18:23:38.075403+0000","last_clean":"2023-11-28T18:23:38.075403+0000","last_became_active":"2023-11-28T18:23:33.947987+0000","last_became_peered":"2023-11-28T18:23:33.947987+0000","last_unstale":"2023-11-28T18:23:38.075403+0000","last_undegraded":"2023-11-28T18:23:38.075403+0000","last_fullsized":"2023-11-28T18:23:38.075403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:35.484199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068226+0000","last_change":"2023-11-28T18:23:38.091059+0000","last_active":"2023-11-28T18:23:39.068226+0000","last_peered":"2023-11-28T18:23:39.068226+0000","last_clean":"2023-11-28T18:23:39.068226+0000","last_became_active":"2023-11-28T18:23:38.090259+0000","last_became_peered":"2023-11-28T18:23:38.090259+0000","last_unstale":"2023-11-28T18:23:39.068226+0000","last_undegraded":"2023-11-28T18:23:39.068226+0000","last_fullsized":"2023-11-28T18:23:39.068226+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:05:00.278938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420356+0000","last_change":"2023-11-28T18:23:32.937363+0000","last_active":"2023-11-28T18:23:38.420356+0000","last_peered":"2023-11-28T18:23:38.420356+0000","last_clean":"2023-11-28T18:23:38.420356+0000","last_became_active":"2023-11-28T18:23:32.937125+0000","last_became_peered":"2023-11-28T18:23:32.937125+0000","last_unstale":"2023-11-28T18:23:38.420356+0000","last_undegraded":"2023-11-28T18:23:38.420356+0000","last_fullsized":"2023-11-28T18:23:38.420356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:45.143373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067755+0000","last_change":"2023-11-28T18:23:33.935572+0000","last_active":"2023-11-28T18:23:38.067755+0000","last_peered":"2023-11-28T18:23:38.067755+0000","last_clean":"2023-11-28T18:23:38.067755+0000","last_became_active":"2023-11-28T18:23:33.935204+0000","last_became_peered":"2023-11-28T18:23:33.935204+0000","last_unstale":"2023-11-28T18:23:38.067755+0000","last_undegraded":"2023-11-28T18:23:38.067755+0000","last_fullsized":"2023-11-28T18:23:38.067755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:02.989816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068674+0000","last_change":"2023-11-28T18:23:38.096313+0000","last_active":"2023-11-28T18:23:39.068674+0000","last_peered":"2023-11-28T18:23:39.068674+0000","last_clean":"2023-11-28T18:23:39.068674+0000","last_became_active":"2023-11-28T18:23:38.095957+0000","last_became_peered":"2023-11-28T18:23:38.095957+0000","last_unstale":"2023-11-28T18:23:39.068674+0000","last_undegraded":"2023-11-28T18:23:39.068674+0000","last_fullsized":"2023-11-28T18:23:39.068674+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:07.150212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419858+0000","last_change":"2023-11-28T18:23:32.939798+0000","last_active":"2023-11-28T18:23:38.419858+0000","last_peered":"2023-11-28T18:23:38.419858+0000","last_clean":"2023-11-28T18:23:38.419858+0000","last_became_active":"2023-11-28T18:23:32.939450+0000","last_became_peered":"2023-11-28T18:23:32.939450+0000","last_unstale":"2023-11-28T18:23:38.419858+0000","last_undegraded":"2023-11-28T18:23:38.419858+0000","last_fullsized":"2023-11-28T18:23:38.419858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:55:26.908976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067829+0000","last_change":"2023-11-28T18:23:33.933973+0000","last_active":"2023-11-28T18:23:38.067829+0000","last_peered":"2023-11-28T18:23:38.067829+0000","last_clean":"2023-11-28T18:23:38.067829+0000","last_became_active":"2023-11-28T18:23:33.933692+0000","last_became_peered":"2023-11-28T18:23:33.933692+0000","last_unstale":"2023-11-28T18:23:38.067829+0000","last_undegraded":"2023-11-28T18:23:38.067829+0000","last_fullsized":"2023-11-28T18:23:38.067829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:38.277787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068388+0000","last_change":"2023-11-28T18:23:38.089272+0000","last_active":"2023-11-28T18:23:39.068388+0000","last_peered":"2023-11-28T18:23:39.068388+0000","last_clean":"2023-11-28T18:23:39.068388+0000","last_became_active":"2023-11-28T18:23:38.088843+0000","last_became_peered":"2023-11-28T18:23:38.088843+0000","last_unstale":"2023-11-28T18:23:39.068388+0000","last_undegraded":"2023-11-28T18:23:39.068388+0000","last_fullsized":"2023-11-28T18:23:39.068388+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:50.581936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.073981+0000","last_change":"2023-11-28T18:23:33.945002+0000","last_active":"2023-11-28T18:23:38.073981+0000","last_peered":"2023-11-28T18:23:38.073981+0000","last_clean":"2023-11-28T18:23:38.073981+0000","last_became_active":"2023-11-28T18:23:33.942488+0000","last_became_peered":"2023-11-28T18:23:33.942488+0000","last_unstale":"2023-11-28T18:23:38.073981+0000","last_undegraded":"2023-11-28T18:23:38.073981+0000","last_fullsized":"2023-11-28T18:23:38.073981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:04:56.118542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419933+0000","last_change":"2023-11-28T18:23:32.938369+0000","last_active":"2023-11-28T18:23:38.419933+0000","last_peered":"2023-11-28T18:23:38.419933+0000","last_clean":"2023-11-28T18:23:38.419933+0000","last_became_active":"2023-11-28T18:23:32.938126+0000","last_became_peered":"2023-11-28T18:23:32.938126+0000","last_unstale":"2023-11-28T18:23:38.419933+0000","last_undegraded":"2023-11-28T18:23:38.419933+0000","last_fullsized":"2023-11-28T18:23:38.419933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:50.646527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.069187+0000","last_change":"2023-11-28T18:23:38.093734+0000","last_active":"2023-11-28T18:23:39.069187+0000","last_peered":"2023-11-28T18:23:39.069187+0000","last_clean":"2023-11-28T18:23:39.069187+0000","last_became_active":"2023-11-28T18:23:38.093298+0000","last_became_peered":"2023-11-28T18:23:38.093298+0000","last_unstale":"2023-11-28T18:23:39.069187+0000","last_undegraded":"2023-11-28T18:23:39.069187+0000","last_fullsized":"2023-11-28T18:23:39.069187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:22:41.524911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065575+0000","last_change":"2023-11-28T18:23:33.938245+0000","last_active":"2023-11-28T18:23:38.065575+0000","last_peered":"2023-11-28T18:23:38.065575+0000","last_clean":"2023-11-28T18:23:38.065575+0000","last_became_active":"2023-11-28T18:23:33.937832+0000","last_became_peered":"2023-11-28T18:23:33.937832+0000","last_unstale":"2023-11-28T18:23:38.065575+0000","last_undegraded":"2023-11-28T18:23:38.065575+0000","last_fullsized":"2023-11-28T18:23:38.065575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:04:56.118542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.093039+0000","last_change":"2023-11-28T18:23:32.924880+0000","last_active":"2023-11-28T18:23:38.093039+0000","last_peered":"2023-11-28T18:23:38.093039+0000","last_clean":"2023-11-28T18:23:38.093039+0000","last_became_active":"2023-11-28T18:23:32.923804+0000","last_became_peered":"2023-11-28T18:23:32.923804+0000","last_unstale":"2023-11-28T18:23:38.093039+0000","last_undegraded":"2023-11-28T18:23:38.093039+0000","last_fullsized":"2023-11-28T18:23:38.093039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:10.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068797+0000","last_change":"2023-11-28T18:23:38.091628+0000","last_active":"2023-11-28T18:23:39.068797+0000","last_peered":"2023-11-28T18:23:39.068797+0000","last_clean":"2023-11-28T18:23:39.068797+0000","last_became_active":"2023-11-28T18:23:38.090900+0000","last_became_peered":"2023-11-28T18:23:38.090900+0000","last_unstale":"2023-11-28T18:23:39.068797+0000","last_undegraded":"2023-11-28T18:23:39.068797+0000","last_fullsized":"2023-11-28T18:23:39.068797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:47.152718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065511+0000","last_change":"2023-11-28T18:23:33.933391+0000","last_active":"2023-11-28T18:23:38.065511+0000","last_peered":"2023-11-28T18:23:38.065511+0000","last_clean":"2023-11-28T18:23:38.065511+0000","last_became_active":"2023-11-28T18:23:33.932767+0000","last_became_peered":"2023-11-28T18:23:33.932767+0000","last_unstale":"2023-11-28T18:23:38.065511+0000","last_undegraded":"2023-11-28T18:23:38.065511+0000","last_fullsized":"2023-11-28T18:23:38.065511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:21.035422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.092828+0000","last_change":"2023-11-28T18:23:32.929281+0000","last_active":"2023-11-28T18:23:38.092828+0000","last_peered":"2023-11-28T18:23:38.092828+0000","last_clean":"2023-11-28T18:23:38.092828+0000","last_became_active":"2023-11-28T18:23:32.928997+0000","last_became_peered":"2023-11-28T18:23:32.928997+0000","last_unstale":"2023-11-28T18:23:38.092828+0000","last_undegraded":"2023-11-28T18:23:38.092828+0000","last_fullsized":"2023-11-28T18:23:38.092828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:45.143373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.718883+0000","last_change":"2023-11-28T18:23:38.426592+0000","last_active":"2023-11-28T18:23:40.718883+0000","last_peered":"2023-11-28T18:23:40.718883+0000","last_clean":"2023-11-28T18:23:40.718883+0000","last_became_active":"2023-11-28T18:23:38.426089+0000","last_became_peered":"2023-11-28T18:23:38.426089+0000","last_unstale":"2023-11-28T18:23:40.718883+0000","last_undegraded":"2023-11-28T18:23:40.718883+0000","last_fullsized":"2023-11-28T18:23:40.718883+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:15.599832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075007+0000","last_change":"2023-11-28T18:23:33.935238+0000","last_active":"2023-11-28T18:23:38.075007+0000","last_peered":"2023-11-28T18:23:38.075007+0000","last_clean":"2023-11-28T18:23:38.075007+0000","last_became_active":"2023-11-28T18:23:33.934884+0000","last_became_peered":"2023-11-28T18:23:33.934884+0000","last_unstale":"2023-11-28T18:23:38.075007+0000","last_undegraded":"2023-11-28T18:23:38.075007+0000","last_fullsized":"2023-11-28T18:23:38.075007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:58.099391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419489+0000","last_change":"2023-11-28T18:23:32.933485+0000","last_active":"2023-11-28T18:23:38.419489+0000","last_peered":"2023-11-28T18:23:38.419489+0000","last_clean":"2023-11-28T18:23:38.419489+0000","last_became_active":"2023-11-28T18:23:32.932807+0000","last_became_peered":"2023-11-28T18:23:32.932807+0000","last_unstale":"2023-11-28T18:23:38.419489+0000","last_undegraded":"2023-11-28T18:23:38.419489+0000","last_fullsized":"2023-11-28T18:23:38.419489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:00.325734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.069240+0000","last_change":"2023-11-28T18:23:38.091553+0000","last_active":"2023-11-28T18:23:39.069240+0000","last_peered":"2023-11-28T18:23:39.069240+0000","last_clean":"2023-11-28T18:23:39.069240+0000","last_became_active":"2023-11-28T18:23:38.091133+0000","last_became_peered":"2023-11-28T18:23:38.091133+0000","last_unstale":"2023-11-28T18:23:39.069240+0000","last_undegraded":"2023-11-28T18:23:39.069240+0000","last_fullsized":"2023-11-28T18:23:39.069240+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:53.274366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.080431+0000","last_change":"2023-11-28T18:23:33.941025+0000","last_active":"2023-11-28T18:23:38.080431+0000","last_peered":"2023-11-28T18:23:38.080431+0000","last_clean":"2023-11-28T18:23:38.080431+0000","last_became_active":"2023-11-28T18:23:33.940714+0000","last_became_peered":"2023-11-28T18:23:33.940714+0000","last_unstale":"2023-11-28T18:23:38.080431+0000","last_undegraded":"2023-11-28T18:23:38.080431+0000","last_fullsized":"2023-11-28T18:23:38.080431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:01.912695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077789+0000","last_change":"2023-11-28T18:23:32.933239+0000","last_active":"2023-11-28T18:23:38.077789+0000","last_peered":"2023-11-28T18:23:38.077789+0000","last_clean":"2023-11-28T18:23:38.077789+0000","last_became_active":"2023-11-28T18:23:32.932935+0000","last_became_peered":"2023-11-28T18:23:32.932935+0000","last_unstale":"2023-11-28T18:23:38.077789+0000","last_undegraded":"2023-11-28T18:23:38.077789+0000","last_fullsized":"2023-11-28T18:23:38.077789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:36:09.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068440+0000","last_change":"2023-11-28T18:23:38.421858+0000","last_active":"2023-11-28T18:23:39.068440+0000","last_peered":"2023-11-28T18:23:39.068440+0000","last_clean":"2023-11-28T18:23:39.068440+0000","last_became_active":"2023-11-28T18:23:38.421416+0000","last_became_peered":"2023-11-28T18:23:38.421416+0000","last_unstale":"2023-11-28T18:23:39.068440+0000","last_undegraded":"2023-11-28T18:23:39.068440+0000","last_fullsized":"2023-11-28T18:23:39.068440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:31.667216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067068+0000","last_change":"2023-11-28T18:23:33.940147+0000","last_active":"2023-11-28T18:23:38.067068+0000","last_peered":"2023-11-28T18:23:38.067068+0000","last_clean":"2023-11-28T18:23:38.067068+0000","last_became_active":"2023-11-28T18:23:33.939882+0000","last_became_peered":"2023-11-28T18:23:33.939882+0000","last_unstale":"2023-11-28T18:23:38.067068+0000","last_undegraded":"2023-11-28T18:23:38.067068+0000","last_fullsized":"2023-11-28T18:23:38.067068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:02.989816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.096365+0000","last_change":"2023-11-28T18:23:32.925741+0000","last_active":"2023-11-28T18:23:38.096365+0000","last_peered":"2023-11-28T18:23:38.096365+0000","last_clean":"2023-11-28T18:23:38.096365+0000","last_became_active":"2023-11-28T18:23:32.925344+0000","last_became_peered":"2023-11-28T18:23:32.925344+0000","last_unstale":"2023-11-28T18:23:38.096365+0000","last_undegraded":"2023-11-28T18:23:38.096365+0000","last_fullsized":"2023-11-28T18:23:38.096365+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:41.820325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.067516+0000","last_change":"2023-11-28T18:23:38.421716+0000","last_active":"2023-11-28T18:23:39.067516+0000","last_peered":"2023-11-28T18:23:39.067516+0000","last_clean":"2023-11-28T18:23:39.067516+0000","last_became_active":"2023-11-28T18:23:38.421313+0000","last_became_peered":"2023-11-28T18:23:38.421313+0000","last_unstale":"2023-11-28T18:23:39.067516+0000","last_undegraded":"2023-11-28T18:23:39.067516+0000","last_fullsized":"2023-11-28T18:23:39.067516+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:12:36.678267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.094519+0000","last_change":"2023-11-28T18:23:33.934187+0000","last_active":"2023-11-28T18:23:38.094519+0000","last_peered":"2023-11-28T18:23:38.094519+0000","last_clean":"2023-11-28T18:23:38.094519+0000","last_became_active":"2023-11-28T18:23:33.933885+0000","last_became_peered":"2023-11-28T18:23:33.933885+0000","last_unstale":"2023-11-28T18:23:38.094519+0000","last_undegraded":"2023-11-28T18:23:38.094519+0000","last_fullsized":"2023-11-28T18:23:38.094519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:21.035422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420442+0000","last_change":"2023-11-28T18:23:32.933434+0000","last_active":"2023-11-28T18:23:38.420442+0000","last_peered":"2023-11-28T18:23:38.420442+0000","last_clean":"2023-11-28T18:23:38.420442+0000","last_became_active":"2023-11-28T18:23:32.933143+0000","last_became_peered":"2023-11-28T18:23:32.933143+0000","last_unstale":"2023-11-28T18:23:38.420442+0000","last_undegraded":"2023-11-28T18:23:38.420442+0000","last_fullsized":"2023-11-28T18:23:38.420442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:11.551517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.297189+0000","last_change":"2023-11-28T18:23:38.422070+0000","last_active":"2023-11-28T18:23:40.297189+0000","last_peered":"2023-11-28T18:23:40.297189+0000","last_clean":"2023-11-28T18:23:40.297189+0000","last_became_active":"2023-11-28T18:23:38.421240+0000","last_became_peered":"2023-11-28T18:23:38.421240+0000","last_unstale":"2023-11-28T18:23:40.297189+0000","last_undegraded":"2023-11-28T18:23:40.297189+0000","last_fullsized":"2023-11-28T18:23:40.297189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:42.438182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067926+0000","last_change":"2023-11-28T18:23:33.932685+0000","last_active":"2023-11-28T18:23:38.067926+0000","last_peered":"2023-11-28T18:23:38.067926+0000","last_clean":"2023-11-28T18:23:38.067926+0000","last_became_active":"2023-11-28T18:23:33.932342+0000","last_became_peered":"2023-11-28T18:23:33.932342+0000","last_unstale":"2023-11-28T18:23:38.067926+0000","last_undegraded":"2023-11-28T18:23:38.067926+0000","last_fullsized":"2023-11-28T18:23:38.067926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:49.113970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.094608+0000","last_change":"2023-11-28T18:23:32.923234+0000","last_active":"2023-11-28T18:23:38.094608+0000","last_peered":"2023-11-28T18:23:38.094608+0000","last_clean":"2023-11-28T18:23:38.094608+0000","last_became_active":"2023-11-28T18:23:32.922051+0000","last_became_peered":"2023-11-28T18:23:32.922051+0000","last_unstale":"2023-11-28T18:23:38.094608+0000","last_undegraded":"2023-11-28T18:23:38.094608+0000","last_fullsized":"2023-11-28T18:23:38.094608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:36:09.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.718350+0000","last_change":"2023-11-28T18:23:38.102307+0000","last_active":"2023-11-28T18:23:40.718350+0000","last_peered":"2023-11-28T18:23:40.718350+0000","last_clean":"2023-11-28T18:23:40.718350+0000","last_became_active":"2023-11-28T18:23:38.101987+0000","last_became_peered":"2023-11-28T18:23:38.101987+0000","last_unstale":"2023-11-28T18:23:40.718350+0000","last_undegraded":"2023-11-28T18:23:40.718350+0000","last_fullsized":"2023-11-28T18:23:40.718350+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:22:41.524911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.076427+0000","last_change":"2023-11-28T18:23:33.941492+0000","last_active":"2023-11-28T18:23:38.076427+0000","last_peered":"2023-11-28T18:23:38.076427+0000","last_clean":"2023-11-28T18:23:38.076427+0000","last_became_active":"2023-11-28T18:23:33.941148+0000","last_became_peered":"2023-11-28T18:23:33.941148+0000","last_unstale":"2023-11-28T18:23:38.076427+0000","last_undegraded":"2023-11-28T18:23:38.076427+0000","last_fullsized":"2023-11-28T18:23:38.076427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:29:23.426289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068690+0000","last_change":"2023-11-28T18:23:32.922032+0000","last_active":"2023-11-28T18:23:38.068690+0000","last_peered":"2023-11-28T18:23:38.068690+0000","last_clean":"2023-11-28T18:23:38.068690+0000","last_became_active":"2023-11-28T18:23:32.921635+0000","last_became_peered":"2023-11-28T18:23:32.921635+0000","last_unstale":"2023-11-28T18:23:38.068690+0000","last_undegraded":"2023-11-28T18:23:38.068690+0000","last_fullsized":"2023-11-28T18:23:38.068690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:36:09.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.718745+0000","last_change":"2023-11-28T18:23:38.096609+0000","last_active":"2023-11-28T18:23:40.718745+0000","last_peered":"2023-11-28T18:23:40.718745+0000","last_clean":"2023-11-28T18:23:40.718745+0000","last_became_active":"2023-11-28T18:23:38.095758+0000","last_became_peered":"2023-11-28T18:23:38.095758+0000","last_unstale":"2023-11-28T18:23:40.718745+0000","last_undegraded":"2023-11-28T18:23:40.718745+0000","last_fullsized":"2023-11-28T18:23:40.718745+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:42.438182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.090061+0000","last_change":"2023-11-28T18:23:33.937266+0000","last_active":"2023-11-28T18:23:38.090061+0000","last_peered":"2023-11-28T18:23:38.090061+0000","last_clean":"2023-11-28T18:23:38.090061+0000","last_became_active":"2023-11-28T18:23:33.936994+0000","last_became_peered":"2023-11-28T18:23:33.936994+0000","last_unstale":"2023-11-28T18:23:38.090061+0000","last_undegraded":"2023-11-28T18:23:38.090061+0000","last_fullsized":"2023-11-28T18:23:38.090061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:12.557669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.083701+0000","last_change":"2023-11-28T18:23:32.918591+0000","last_active":"2023-11-28T18:23:38.083701+0000","last_peered":"2023-11-28T18:23:38.083701+0000","last_clean":"2023-11-28T18:23:38.083701+0000","last_became_active":"2023-11-28T18:23:32.915381+0000","last_became_peered":"2023-11-28T18:23:32.915381+0000","last_unstale":"2023-11-28T18:23:38.083701+0000","last_undegraded":"2023-11-28T18:23:38.083701+0000","last_fullsized":"2023-11-28T18:23:38.083701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:57.093239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.718851+0000","last_change":"2023-11-28T18:23:38.100344+0000","last_active":"2023-11-28T18:23:40.718851+0000","last_peered":"2023-11-28T18:23:40.718851+0000","last_clean":"2023-11-28T18:23:40.718851+0000","last_became_active":"2023-11-28T18:23:38.099934+0000","last_became_peered":"2023-11-28T18:23:38.099934+0000","last_unstale":"2023-11-28T18:23:40.718851+0000","last_undegraded":"2023-11-28T18:23:40.718851+0000","last_fullsized":"2023-11-28T18:23:40.718851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:39.149571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.074486+0000","last_change":"2023-11-28T18:23:33.944704+0000","last_active":"2023-11-28T18:23:38.074486+0000","last_peered":"2023-11-28T18:23:38.074486+0000","last_clean":"2023-11-28T18:23:38.074486+0000","last_became_active":"2023-11-28T18:23:33.944484+0000","last_became_peered":"2023-11-28T18:23:33.944484+0000","last_unstale":"2023-11-28T18:23:38.074486+0000","last_undegraded":"2023-11-28T18:23:38.074486+0000","last_fullsized":"2023-11-28T18:23:38.074486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:55:27.915128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419489+0000","last_change":"2023-11-28T18:23:32.933677+0000","last_active":"2023-11-28T18:23:38.419489+0000","last_peered":"2023-11-28T18:23:38.419489+0000","last_clean":"2023-11-28T18:23:38.419489+0000","last_became_active":"2023-11-28T18:23:32.933168+0000","last_became_peered":"2023-11-28T18:23:32.933168+0000","last_unstale":"2023-11-28T18:23:38.419489+0000","last_undegraded":"2023-11-28T18:23:38.419489+0000","last_fullsized":"2023-11-28T18:23:38.419489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:29:22.420137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.297576+0000","last_change":"2023-11-28T18:23:38.427376+0000","last_active":"2023-11-28T18:23:40.297576+0000","last_peered":"2023-11-28T18:23:40.297576+0000","last_clean":"2023-11-28T18:23:40.297576+0000","last_became_active":"2023-11-28T18:23:38.426436+0000","last_became_peered":"2023-11-28T18:23:38.426436+0000","last_unstale":"2023-11-28T18:23:40.297576+0000","last_undegraded":"2023-11-28T18:23:40.297576+0000","last_fullsized":"2023-11-28T18:23:40.297576+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:19:23.246961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065098+0000","last_change":"2023-11-28T18:23:33.932836+0000","last_active":"2023-11-28T18:23:38.065098+0000","last_peered":"2023-11-28T18:23:38.065098+0000","last_clean":"2023-11-28T18:23:38.065098+0000","last_became_active":"2023-11-28T18:23:33.932534+0000","last_became_peered":"2023-11-28T18:23:33.932534+0000","last_unstale":"2023-11-28T18:23:38.065098+0000","last_undegraded":"2023-11-28T18:23:38.065098+0000","last_fullsized":"2023-11-28T18:23:38.065098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:46.421541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067421+0000","last_change":"2023-11-28T18:23:32.922842+0000","last_active":"2023-11-28T18:23:38.067421+0000","last_peered":"2023-11-28T18:23:38.067421+0000","last_clean":"2023-11-28T18:23:38.067421+0000","last_became_active":"2023-11-28T18:23:32.922541+0000","last_became_peered":"2023-11-28T18:23:32.922541+0000","last_unstale":"2023-11-28T18:23:38.067421+0000","last_undegraded":"2023-11-28T18:23:38.067421+0000","last_fullsized":"2023-11-28T18:23:38.067421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:39:52.964875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419571+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:23:38.419571+0000","last_peered":"2023-11-28T18:23:38.419571+0000","last_clean":"2023-11-28T18:23:38.419571+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:23:38.419571+0000","last_undegraded":"2023-11-28T18:23:38.419571+0000","last_fullsized":"2023-11-28T18:23:38.419571+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.719264+0000","last_change":"2023-11-28T18:23:38.427043+0000","last_active":"2023-11-28T18:23:40.719264+0000","last_peered":"2023-11-28T18:23:40.719264+0000","last_clean":"2023-11-28T18:23:40.719264+0000","last_became_active":"2023-11-28T18:23:38.426613+0000","last_became_peered":"2023-11-28T18:23:38.426613+0000","last_unstale":"2023-11-28T18:23:40.719264+0000","last_undegraded":"2023-11-28T18:23:40.719264+0000","last_fullsized":"2023-11-28T18:23:40.719264+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:47.152718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419993+0000","last_change":"2023-11-28T18:23:33.944370+0000","last_active":"2023-11-28T18:23:38.419993+0000","last_peered":"2023-11-28T18:23:38.419993+0000","last_clean":"2023-11-28T18:23:38.419993+0000","last_became_active":"2023-11-28T18:23:33.943698+0000","last_became_peered":"2023-11-28T18:23:33.943698+0000","last_unstale":"2023-11-28T18:23:38.419993+0000","last_undegraded":"2023-11-28T18:23:38.419993+0000","last_fullsized":"2023-11-28T18:23:38.419993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:49.113970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.083176+0000","last_change":"2023-11-28T18:23:32.917731+0000","last_active":"2023-11-28T18:23:38.083176+0000","last_peered":"2023-11-28T18:23:38.083176+0000","last_clean":"2023-11-28T18:23:38.083176+0000","last_became_active":"2023-11-28T18:23:32.917207+0000","last_became_peered":"2023-11-28T18:23:32.917207+0000","last_unstale":"2023-11-28T18:23:38.083176+0000","last_undegraded":"2023-11-28T18:23:38.083176+0000","last_fullsized":"2023-11-28T18:23:38.083176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:38:50.132097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.719164+0000","last_change":"2023-11-28T18:23:38.425300+0000","last_active":"2023-11-28T18:23:40.719164+0000","last_peered":"2023-11-28T18:23:40.719164+0000","last_clean":"2023-11-28T18:23:40.719164+0000","last_became_active":"2023-11-28T18:23:38.424708+0000","last_became_peered":"2023-11-28T18:23:38.424708+0000","last_unstale":"2023-11-28T18:23:40.719164+0000","last_undegraded":"2023-11-28T18:23:40.719164+0000","last_fullsized":"2023-11-28T18:23:40.719164+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:07.150212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067866+0000","last_change":"2023-11-28T18:23:33.932266+0000","last_active":"2023-11-28T18:23:38.067866+0000","last_peered":"2023-11-28T18:23:38.067866+0000","last_clean":"2023-11-28T18:23:38.067866+0000","last_became_active":"2023-11-28T18:23:33.931923+0000","last_became_peered":"2023-11-28T18:23:33.931923+0000","last_unstale":"2023-11-28T18:23:38.067866+0000","last_undegraded":"2023-11-28T18:23:38.067866+0000","last_fullsized":"2023-11-28T18:23:38.067866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:34.989175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.094421+0000","last_change":"2023-11-28T18:23:32.924719+0000","last_active":"2023-11-28T18:23:38.094421+0000","last_peered":"2023-11-28T18:23:38.094421+0000","last_clean":"2023-11-28T18:23:38.094421+0000","last_became_active":"2023-11-28T18:23:32.924462+0000","last_became_peered":"2023-11-28T18:23:32.924462+0000","last_unstale":"2023-11-28T18:23:38.094421+0000","last_undegraded":"2023-11-28T18:23:38.094421+0000","last_fullsized":"2023-11-28T18:23:38.094421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:55:26.908976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.297354+0000","last_change":"2023-11-28T18:23:38.093163+0000","last_active":"2023-11-28T18:23:40.297354+0000","last_peered":"2023-11-28T18:23:40.297354+0000","last_clean":"2023-11-28T18:23:40.297354+0000","last_became_active":"2023-11-28T18:23:38.092754+0000","last_became_peered":"2023-11-28T18:23:38.092754+0000","last_unstale":"2023-11-28T18:23:40.297354+0000","last_undegraded":"2023-11-28T18:23:40.297354+0000","last_fullsized":"2023-11-28T18:23:40.297354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:39.149571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420505+0000","last_change":"2023-11-28T18:23:33.954652+0000","last_active":"2023-11-28T18:23:38.420505+0000","last_peered":"2023-11-28T18:23:38.420505+0000","last_clean":"2023-11-28T18:23:38.420505+0000","last_became_active":"2023-11-28T18:23:33.954369+0000","last_became_peered":"2023-11-28T18:23:33.954369+0000","last_unstale":"2023-11-28T18:23:38.420505+0000","last_undegraded":"2023-11-28T18:23:38.420505+0000","last_fullsized":"2023-11-28T18:23:38.420505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:30:56.523444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.084061+0000","last_change":"2023-11-28T18:23:32.906348+0000","last_active":"2023-11-28T18:23:38.084061+0000","last_peered":"2023-11-28T18:23:38.084061+0000","last_clean":"2023-11-28T18:23:38.084061+0000","last_became_active":"2023-11-28T18:23:32.906007+0000","last_became_peered":"2023-11-28T18:23:32.906007+0000","last_unstale":"2023-11-28T18:23:38.084061+0000","last_undegraded":"2023-11-28T18:23:38.084061+0000","last_fullsized":"2023-11-28T18:23:38.084061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:00.325734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.719440+0000","last_change":"2023-11-28T18:23:38.426316+0000","last_active":"2023-11-28T18:23:40.719440+0000","last_peered":"2023-11-28T18:23:40.719440+0000","last_clean":"2023-11-28T18:23:40.719440+0000","last_became_active":"2023-11-28T18:23:38.425328+0000","last_became_peered":"2023-11-28T18:23:38.425328+0000","last_unstale":"2023-11-28T18:23:40.719440+0000","last_undegraded":"2023-11-28T18:23:40.719440+0000","last_fullsized":"2023-11-28T18:23:40.719440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:19:51.686256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420058+0000","last_change":"2023-11-28T18:23:33.954315+0000","last_active":"2023-11-28T18:23:38.420058+0000","last_peered":"2023-11-28T18:23:38.420058+0000","last_clean":"2023-11-28T18:23:38.420058+0000","last_became_active":"2023-11-28T18:23:33.950270+0000","last_became_peered":"2023-11-28T18:23:33.950270+0000","last_unstale":"2023-11-28T18:23:38.420058+0000","last_undegraded":"2023-11-28T18:23:38.420058+0000","last_fullsized":"2023-11-28T18:23:38.420058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:22:37.364515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077169+0000","last_change":"2023-11-28T18:23:32.921145+0000","last_active":"2023-11-28T18:23:38.077169+0000","last_peered":"2023-11-28T18:23:38.077169+0000","last_clean":"2023-11-28T18:23:38.077169+0000","last_became_active":"2023-11-28T18:23:32.920706+0000","last_became_peered":"2023-11-28T18:23:32.920706+0000","last_unstale":"2023-11-28T18:23:38.077169+0000","last_undegraded":"2023-11-28T18:23:38.077169+0000","last_fullsized":"2023-11-28T18:23:38.077169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:39:52.964875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068294+0000","last_change":"2023-11-28T18:23:38.096089+0000","last_active":"2023-11-28T18:23:39.068294+0000","last_peered":"2023-11-28T18:23:39.068294+0000","last_clean":"2023-11-28T18:23:39.068294+0000","last_became_active":"2023-11-28T18:23:38.095672+0000","last_became_peered":"2023-11-28T18:23:38.095672+0000","last_unstale":"2023-11-28T18:23:39.068294+0000","last_undegraded":"2023-11-28T18:23:39.068294+0000","last_fullsized":"2023-11-28T18:23:39.068294+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:09.933678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419670+0000","last_change":"2023-11-28T18:23:33.954352+0000","last_active":"2023-11-28T18:23:38.419670+0000","last_peered":"2023-11-28T18:23:38.419670+0000","last_clean":"2023-11-28T18:23:38.419670+0000","last_became_active":"2023-11-28T18:23:33.950344+0000","last_became_peered":"2023-11-28T18:23:33.950344+0000","last_unstale":"2023-11-28T18:23:38.419670+0000","last_undegraded":"2023-11-28T18:23:38.419670+0000","last_fullsized":"2023-11-28T18:23:38.419670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:08.033209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.078193+0000","last_change":"2023-11-28T18:23:32.919741+0000","last_active":"2023-11-28T18:23:38.078193+0000","last_peered":"2023-11-28T18:23:38.078193+0000","last_clean":"2023-11-28T18:23:38.078193+0000","last_became_active":"2023-11-28T18:23:32.919438+0000","last_became_peered":"2023-11-28T18:23:32.919438+0000","last_unstale":"2023-11-28T18:23:38.078193+0000","last_undegraded":"2023-11-28T18:23:38.078193+0000","last_fullsized":"2023-11-28T18:23:38.078193+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:39:52.964875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.297438+0000","last_change":"2023-11-28T18:23:38.084984+0000","last_active":"2023-11-28T18:23:40.297438+0000","last_peered":"2023-11-28T18:23:40.297438+0000","last_clean":"2023-11-28T18:23:40.297438+0000","last_became_active":"2023-11-28T18:23:38.084490+0000","last_became_peered":"2023-11-28T18:23:38.084490+0000","last_unstale":"2023-11-28T18:23:40.297438+0000","last_undegraded":"2023-11-28T18:23:40.297438+0000","last_fullsized":"2023-11-28T18:23:40.297438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:00.683839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.073493+0000","last_change":"2023-11-28T18:23:33.939191+0000","last_active":"2023-11-28T18:23:38.073493+0000","last_peered":"2023-11-28T18:23:38.073493+0000","last_clean":"2023-11-28T18:23:38.073493+0000","last_became_active":"2023-11-28T18:23:33.938295+0000","last_became_peered":"2023-11-28T18:23:33.938295+0000","last_unstale":"2023-11-28T18:23:38.073493+0000","last_undegraded":"2023-11-28T18:23:38.073493+0000","last_fullsized":"2023-11-28T18:23:38.073493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:12.557669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.073721+0000","last_change":"2023-11-28T18:23:32.930354+0000","last_active":"2023-11-28T18:23:38.073721+0000","last_peered":"2023-11-28T18:23:38.073721+0000","last_clean":"2023-11-28T18:23:38.073721+0000","last_became_active":"2023-11-28T18:23:32.930059+0000","last_became_peered":"2023-11-28T18:23:32.930059+0000","last_unstale":"2023-11-28T18:23:38.073721+0000","last_undegraded":"2023-11-28T18:23:38.073721+0000","last_fullsized":"2023-11-28T18:23:38.073721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:45.143373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.069210+0000","last_change":"2023-11-28T18:23:38.427077+0000","last_active":"2023-11-28T18:23:39.069210+0000","last_peered":"2023-11-28T18:23:39.069210+0000","last_clean":"2023-11-28T18:23:39.069210+0000","last_became_active":"2023-11-28T18:23:38.426613+0000","last_became_peered":"2023-11-28T18:23:38.426613+0000","last_unstale":"2023-11-28T18:23:39.069210+0000","last_undegraded":"2023-11-28T18:23:39.069210+0000","last_fullsized":"2023-11-28T18:23:39.069210+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:24:11.818911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419843+0000","last_change":"2023-11-28T18:23:33.955340+0000","last_active":"2023-11-28T18:23:38.419843+0000","last_peered":"2023-11-28T18:23:38.419843+0000","last_clean":"2023-11-28T18:23:38.419843+0000","last_became_active":"2023-11-28T18:23:33.955080+0000","last_became_peered":"2023-11-28T18:23:33.955080+0000","last_unstale":"2023-11-28T18:23:38.419843+0000","last_undegraded":"2023-11-28T18:23:38.419843+0000","last_fullsized":"2023-11-28T18:23:38.419843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:42.992322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068633+0000","last_change":"2023-11-28T18:23:32.935530+0000","last_active":"2023-11-28T18:23:38.068633+0000","last_peered":"2023-11-28T18:23:38.068633+0000","last_clean":"2023-11-28T18:23:38.068633+0000","last_became_active":"2023-11-28T18:23:32.934513+0000","last_became_peered":"2023-11-28T18:23:32.934513+0000","last_unstale":"2023-11-28T18:23:38.068633+0000","last_undegraded":"2023-11-28T18:23:38.068633+0000","last_fullsized":"2023-11-28T18:23:38.068633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:10.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068541+0000","last_change":"2023-11-28T18:23:38.421813+0000","last_active":"2023-11-28T18:23:39.068541+0000","last_peered":"2023-11-28T18:23:39.068541+0000","last_clean":"2023-11-28T18:23:39.068541+0000","last_became_active":"2023-11-28T18:23:38.421280+0000","last_became_peered":"2023-11-28T18:23:38.421280+0000","last_unstale":"2023-11-28T18:23:39.068541+0000","last_undegraded":"2023-11-28T18:23:39.068541+0000","last_fullsized":"2023-11-28T18:23:39.068541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:19:05.111022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.080161+0000","last_change":"2023-11-28T18:23:33.946537+0000","last_active":"2023-11-28T18:23:38.080161+0000","last_peered":"2023-11-28T18:23:38.080161+0000","last_clean":"2023-11-28T18:23:38.080161+0000","last_became_active":"2023-11-28T18:23:33.946220+0000","last_became_peered":"2023-11-28T18:23:33.946220+0000","last_unstale":"2023-11-28T18:23:38.080161+0000","last_undegraded":"2023-11-28T18:23:38.080161+0000","last_fullsized":"2023-11-28T18:23:38.080161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:51.652679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077792+0000","last_change":"2023-11-28T18:23:32.922020+0000","last_active":"2023-11-28T18:23:38.077792+0000","last_peered":"2023-11-28T18:23:38.077792+0000","last_clean":"2023-11-28T18:23:38.077792+0000","last_became_active":"2023-11-28T18:23:32.921656+0000","last_became_peered":"2023-11-28T18:23:32.921656+0000","last_unstale":"2023-11-28T18:23:38.077792+0000","last_undegraded":"2023-11-28T18:23:38.077792+0000","last_fullsized":"2023-11-28T18:23:38.077792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:10.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.721071+0000","last_change":"2023-11-28T18:23:38.421823+0000","last_active":"2023-11-28T18:23:40.721071+0000","last_peered":"2023-11-28T18:23:40.721071+0000","last_clean":"2023-11-28T18:23:40.721071+0000","last_became_active":"2023-11-28T18:23:38.421407+0000","last_became_peered":"2023-11-28T18:23:38.421407+0000","last_unstale":"2023-11-28T18:23:40.721071+0000","last_undegraded":"2023-11-28T18:23:40.721071+0000","last_fullsized":"2023-11-28T18:23:40.721071+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:01.885046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419892+0000","last_change":"2023-11-28T18:23:33.939629+0000","last_active":"2023-11-28T18:23:38.419892+0000","last_peered":"2023-11-28T18:23:38.419892+0000","last_clean":"2023-11-28T18:23:38.419892+0000","last_became_active":"2023-11-28T18:23:33.938662+0000","last_became_peered":"2023-11-28T18:23:33.938662+0000","last_unstale":"2023-11-28T18:23:38.419892+0000","last_undegraded":"2023-11-28T18:23:38.419892+0000","last_fullsized":"2023-11-28T18:23:38.419892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:02.989816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:44.598424+0000","last_change":"2023-11-28T18:23:32.923661+0000","last_active":"2023-11-28T18:23:44.598424+0000","last_peered":"2023-11-28T18:23:44.598424+0000","last_clean":"2023-11-28T18:23:44.598424+0000","last_became_active":"2023-11-28T18:23:32.922407+0000","last_became_peered":"2023-11-28T18:23:32.922407+0000","last_unstale":"2023-11-28T18:23:44.598424+0000","last_undegraded":"2023-11-28T18:23:44.598424+0000","last_fullsized":"2023-11-28T18:23:44.598424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:10.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.718701+0000","last_change":"2023-11-28T18:23:38.098761+0000","last_active":"2023-11-28T18:23:40.718701+0000","last_peered":"2023-11-28T18:23:40.718701+0000","last_clean":"2023-11-28T18:23:40.718701+0000","last_became_active":"2023-11-28T18:23:38.098374+0000","last_became_peered":"2023-11-28T18:23:38.098374+0000","last_unstale":"2023-11-28T18:23:40.718701+0000","last_undegraded":"2023-11-28T18:23:40.718701+0000","last_fullsized":"2023-11-28T18:23:40.718701+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:53.274366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075827+0000","last_change":"2023-11-28T18:23:33.945418+0000","last_active":"2023-11-28T18:23:38.075827+0000","last_peered":"2023-11-28T18:23:38.075827+0000","last_clean":"2023-11-28T18:23:38.075827+0000","last_became_active":"2023-11-28T18:23:33.944949+0000","last_became_peered":"2023-11-28T18:23:33.944949+0000","last_unstale":"2023-11-28T18:23:38.075827+0000","last_undegraded":"2023-11-28T18:23:38.075827+0000","last_fullsized":"2023-11-28T18:23:38.075827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:42.826477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.094316+0000","last_change":"2023-11-28T18:23:32.925933+0000","last_active":"2023-11-28T18:23:38.094316+0000","last_peered":"2023-11-28T18:23:38.094316+0000","last_clean":"2023-11-28T18:23:38.094316+0000","last_became_active":"2023-11-28T18:23:32.925619+0000","last_became_peered":"2023-11-28T18:23:32.925619+0000","last_unstale":"2023-11-28T18:23:38.094316+0000","last_undegraded":"2023-11-28T18:23:38.094316+0000","last_fullsized":"2023-11-28T18:23:38.094316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:19.143213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068423+0000","last_change":"2023-11-28T18:23:38.091049+0000","last_active":"2023-11-28T18:23:39.068423+0000","last_peered":"2023-11-28T18:23:39.068423+0000","last_clean":"2023-11-28T18:23:39.068423+0000","last_became_active":"2023-11-28T18:23:38.090236+0000","last_became_peered":"2023-11-28T18:23:38.090236+0000","last_unstale":"2023-11-28T18:23:39.068423+0000","last_undegraded":"2023-11-28T18:23:39.068423+0000","last_fullsized":"2023-11-28T18:23:39.068423+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:12.193604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.076836+0000","last_change":"2023-11-28T18:23:33.948346+0000","last_active":"2023-11-28T18:23:38.076836+0000","last_peered":"2023-11-28T18:23:38.076836+0000","last_clean":"2023-11-28T18:23:38.076836+0000","last_became_active":"2023-11-28T18:23:33.947983+0000","last_became_peered":"2023-11-28T18:23:33.947983+0000","last_unstale":"2023-11-28T18:23:38.076836+0000","last_undegraded":"2023-11-28T18:23:38.076836+0000","last_fullsized":"2023-11-28T18:23:38.076836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:12.557669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067715+0000","last_change":"2023-11-28T18:23:32.935430+0000","last_active":"2023-11-28T18:23:38.067715+0000","last_peered":"2023-11-28T18:23:38.067715+0000","last_clean":"2023-11-28T18:23:38.067715+0000","last_became_active":"2023-11-28T18:23:32.930282+0000","last_became_peered":"2023-11-28T18:23:32.930282+0000","last_unstale":"2023-11-28T18:23:38.067715+0000","last_undegraded":"2023-11-28T18:23:38.067715+0000","last_fullsized":"2023-11-28T18:23:38.067715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:50.646527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.067746+0000","last_change":"2023-11-28T18:23:38.093851+0000","last_active":"2023-11-28T18:23:39.067746+0000","last_peered":"2023-11-28T18:23:39.067746+0000","last_clean":"2023-11-28T18:23:39.067746+0000","last_became_active":"2023-11-28T18:23:38.091779+0000","last_became_peered":"2023-11-28T18:23:38.091779+0000","last_unstale":"2023-11-28T18:23:39.067746+0000","last_undegraded":"2023-11-28T18:23:39.067746+0000","last_fullsized":"2023-11-28T18:23:39.067746+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:13:14.577345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067617+0000","last_change":"2023-11-28T18:23:33.952137+0000","last_active":"2023-11-28T18:23:38.067617+0000","last_peered":"2023-11-28T18:23:38.067617+0000","last_clean":"2023-11-28T18:23:38.067617+0000","last_became_active":"2023-11-28T18:23:33.950857+0000","last_became_peered":"2023-11-28T18:23:33.950857+0000","last_unstale":"2023-11-28T18:23:38.067617+0000","last_undegraded":"2023-11-28T18:23:38.067617+0000","last_fullsized":"2023-11-28T18:23:38.067617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:42.992322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077679+0000","last_change":"2023-11-28T18:23:32.940089+0000","last_active":"2023-11-28T18:23:38.077679+0000","last_peered":"2023-11-28T18:23:38.077679+0000","last_clean":"2023-11-28T18:23:38.077679+0000","last_became_active":"2023-11-28T18:23:32.939524+0000","last_became_peered":"2023-11-28T18:23:32.939524+0000","last_unstale":"2023-11-28T18:23:38.077679+0000","last_undegraded":"2023-11-28T18:23:38.077679+0000","last_fullsized":"2023-11-28T18:23:38.077679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:19.143213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.067936+0000","last_change":"2023-11-28T18:23:38.424675+0000","last_active":"2023-11-28T18:23:39.067936+0000","last_peered":"2023-11-28T18:23:39.067936+0000","last_clean":"2023-11-28T18:23:39.067936+0000","last_became_active":"2023-11-28T18:23:38.424284+0000","last_became_peered":"2023-11-28T18:23:38.424284+0000","last_unstale":"2023-11-28T18:23:39.067936+0000","last_undegraded":"2023-11-28T18:23:39.067936+0000","last_fullsized":"2023-11-28T18:23:39.067936+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:01.885046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.297416+0000","last_change":"2023-11-28T18:23:38.425611+0000","last_active":"2023-11-28T18:23:40.297416+0000","last_peered":"2023-11-28T18:23:40.297416+0000","last_clean":"2023-11-28T18:23:40.297416+0000","last_became_active":"2023-11-28T18:23:38.424826+0000","last_became_peered":"2023-11-28T18:23:38.424826+0000","last_unstale":"2023-11-28T18:23:40.297416+0000","last_undegraded":"2023-11-28T18:23:40.297416+0000","last_fullsized":"2023-11-28T18:23:40.297416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:15.599832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.092623+0000","last_change":"2023-11-28T18:23:33.937161+0000","last_active":"2023-11-28T18:23:38.092623+0000","last_peered":"2023-11-28T18:23:38.092623+0000","last_clean":"2023-11-28T18:23:38.092623+0000","last_became_active":"2023-11-28T18:23:33.936724+0000","last_became_peered":"2023-11-28T18:23:33.936724+0000","last_unstale":"2023-11-28T18:23:38.092623+0000","last_undegraded":"2023-11-28T18:23:38.092623+0000","last_fullsized":"2023-11-28T18:23:38.092623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:17:14.412292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.070788+0000","last_change":"2023-11-28T18:23:32.921502+0000","last_active":"2023-11-28T18:23:38.070788+0000","last_peered":"2023-11-28T18:23:38.070788+0000","last_clean":"2023-11-28T18:23:38.070788+0000","last_became_active":"2023-11-28T18:23:32.920979+0000","last_became_peered":"2023-11-28T18:23:32.920979+0000","last_unstale":"2023-11-28T18:23:38.070788+0000","last_undegraded":"2023-11-28T18:23:38.070788+0000","last_fullsized":"2023-11-28T18:23:38.070788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:41.820325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:40.719783+0000","last_change":"2023-11-28T18:23:38.095346+0000","last_active":"2023-11-28T18:23:40.719783+0000","last_peered":"2023-11-28T18:23:40.719783+0000","last_clean":"2023-11-28T18:23:40.719783+0000","last_became_active":"2023-11-28T18:23:38.094985+0000","last_became_peered":"2023-11-28T18:23:38.094985+0000","last_unstale":"2023-11-28T18:23:40.719783+0000","last_undegraded":"2023-11-28T18:23:40.719783+0000","last_fullsized":"2023-11-28T18:23:40.719783+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:22:41.524911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065411+0000","last_change":"2023-11-28T18:23:33.934122+0000","last_active":"2023-11-28T18:23:38.065411+0000","last_peered":"2023-11-28T18:23:38.065411+0000","last_clean":"2023-11-28T18:23:38.065411+0000","last_became_active":"2023-11-28T18:23:33.933687+0000","last_became_peered":"2023-11-28T18:23:33.933687+0000","last_unstale":"2023-11-28T18:23:38.065411+0000","last_undegraded":"2023-11-28T18:23:38.065411+0000","last_fullsized":"2023-11-28T18:23:38.065411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:17:14.412292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065550+0000","last_change":"2023-11-28T18:23:32.931872+0000","last_active":"2023-11-28T18:23:38.065550+0000","last_peered":"2023-11-28T18:23:38.065550+0000","last_clean":"2023-11-28T18:23:38.065550+0000","last_became_active":"2023-11-28T18:23:32.931111+0000","last_became_peered":"2023-11-28T18:23:32.931111+0000","last_unstale":"2023-11-28T18:23:38.065550+0000","last_undegraded":"2023-11-28T18:23:38.065550+0000","last_fullsized":"2023-11-28T18:23:38.065550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:41.820325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.074084+0000","last_change":"2023-11-28T18:23:32.934258+0000","last_active":"2023-11-28T18:23:38.074084+0000","last_peered":"2023-11-28T18:23:38.074084+0000","last_clean":"2023-11-28T18:23:38.074084+0000","last_became_active":"2023-11-28T18:23:32.933956+0000","last_became_peered":"2023-11-28T18:23:32.933956+0000","last_unstale":"2023-11-28T18:23:38.074084+0000","last_undegraded":"2023-11-28T18:23:38.074084+0000","last_fullsized":"2023-11-28T18:23:38.074084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:41.820325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077709+0000","last_change":"2023-11-28T18:23:33.949218+0000","last_active":"2023-11-28T18:23:38.077709+0000","last_peered":"2023-11-28T18:23:38.077709+0000","last_clean":"2023-11-28T18:23:38.077709+0000","last_became_active":"2023-11-28T18:23:33.948930+0000","last_became_peered":"2023-11-28T18:23:33.948930+0000","last_unstale":"2023-11-28T18:23:38.077709+0000","last_undegraded":"2023-11-28T18:23:38.077709+0000","last_fullsized":"2023-11-28T18:23:38.077709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:46.421541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.067937+0000","last_change":"2023-11-28T18:23:38.095800+0000","last_active":"2023-11-28T18:23:39.067937+0000","last_peered":"2023-11-28T18:23:39.067937+0000","last_clean":"2023-11-28T18:23:39.067937+0000","last_became_active":"2023-11-28T18:23:38.095437+0000","last_became_peered":"2023-11-28T18:23:38.095437+0000","last_unstale":"2023-11-28T18:23:39.067937+0000","last_undegraded":"2023-11-28T18:23:39.067937+0000","last_fullsized":"2023-11-28T18:23:39.067937+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:39.644595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065704+0000","last_change":"2023-11-28T18:23:32.921448+0000","last_active":"2023-11-28T18:23:38.065704+0000","last_peered":"2023-11-28T18:23:38.065704+0000","last_clean":"2023-11-28T18:23:38.065704+0000","last_became_active":"2023-11-28T18:23:32.921183+0000","last_became_peered":"2023-11-28T18:23:32.921183+0000","last_unstale":"2023-11-28T18:23:38.065704+0000","last_undegraded":"2023-11-28T18:23:38.065704+0000","last_fullsized":"2023-11-28T18:23:38.065704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:31.886440+0000","last_clean_scrub_stamp":"2023-11-28T18:23:31.886440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:36:09.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075654+0000","last_change":"2023-11-28T18:23:33.937197+0000","last_active":"2023-11-28T18:23:38.075654+0000","last_peered":"2023-11-28T18:23:38.075654+0000","last_clean":"2023-11-28T18:23:38.075654+0000","last_became_active":"2023-11-28T18:23:33.936922+0000","last_became_peered":"2023-11-28T18:23:33.936922+0000","last_unstale":"2023-11-28T18:23:38.075654+0000","last_undegraded":"2023-11-28T18:23:38.075654+0000","last_fullsized":"2023-11-28T18:23:38.075654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:01.331886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T18:23:39.068617+0000","last_change":"2023-11-28T18:23:38.099275+0000","last_active":"2023-11-28T18:23:39.068617+0000","last_peered":"2023-11-28T18:23:39.068617+0000","last_clean":"2023-11-28T18:23:39.068617+0000","last_became_active":"2023-11-28T18:23:38.095544+0000","last_became_peered":"2023-11-28T18:23:38.095544+0000","last_unstale":"2023-11-28T18:23:39.068617+0000","last_undegraded":"2023-11-28T18:23:39.068617+0000","last_fullsized":"2023-11-28T18:23:39.068617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:37.052987+0000","last_clean_scrub_stamp":"2023-11-28T18:23:37.052987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:04.943189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.073386+0000","last_change":"2023-11-28T18:23:33.943034+0000","last_active":"2023-11-28T18:23:38.073386+0000","last_peered":"2023-11-28T18:23:38.073386+0000","last_clean":"2023-11-28T18:23:38.073386+0000","last_became_active":"2023-11-28T18:23:33.942687+0000","last_became_peered":"2023-11-28T18:23:33.942687+0000","last_unstale":"2023-11-28T18:23:38.073386+0000","last_undegraded":"2023-11-28T18:23:38.073386+0000","last_fullsized":"2023-11-28T18:23:38.073386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:35.484199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.083621+0000","last_change":"2023-11-28T18:23:33.942958+0000","last_active":"2023-11-28T18:23:38.083621+0000","last_peered":"2023-11-28T18:23:38.083621+0000","last_clean":"2023-11-28T18:23:38.083621+0000","last_became_active":"2023-11-28T18:23:33.942696+0000","last_became_peered":"2023-11-28T18:23:33.942696+0000","last_unstale":"2023-11-28T18:23:38.083621+0000","last_undegraded":"2023-11-28T18:23:38.083621+0000","last_fullsized":"2023-11-28T18:23:38.083621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:42.826477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068550+0000","last_change":"2023-11-28T18:23:33.927110+0000","last_active":"2023-11-28T18:23:38.068550+0000","last_peered":"2023-11-28T18:23:38.068550+0000","last_clean":"2023-11-28T18:23:38.068550+0000","last_became_active":"2023-11-28T18:23:33.926812+0000","last_became_peered":"2023-11-28T18:23:33.926812+0000","last_unstale":"2023-11-28T18:23:38.068550+0000","last_undegraded":"2023-11-28T18:23:38.068550+0000","last_fullsized":"2023-11-28T18:23:38.068550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:22:37.364515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077616+0000","last_change":"2023-11-28T18:23:33.941141+0000","last_active":"2023-11-28T18:23:38.077616+0000","last_peered":"2023-11-28T18:23:38.077616+0000","last_clean":"2023-11-28T18:23:38.077616+0000","last_became_active":"2023-11-28T18:23:33.940832+0000","last_became_peered":"2023-11-28T18:23:33.940832+0000","last_unstale":"2023-11-28T18:23:38.077616+0000","last_undegraded":"2023-11-28T18:23:38.077616+0000","last_fullsized":"2023-11-28T18:23:38.077616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:04:56.118542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.066054+0000","last_change":"2023-11-28T18:23:33.924784+0000","last_active":"2023-11-28T18:23:38.066054+0000","last_peered":"2023-11-28T18:23:38.066054+0000","last_clean":"2023-11-28T18:23:38.066054+0000","last_became_active":"2023-11-28T18:23:33.924475+0000","last_became_peered":"2023-11-28T18:23:33.924475+0000","last_unstale":"2023-11-28T18:23:38.066054+0000","last_undegraded":"2023-11-28T18:23:38.066054+0000","last_fullsized":"2023-11-28T18:23:38.066054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:05.773283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.094179+0000","last_change":"2023-11-28T18:23:33.932579+0000","last_active":"2023-11-28T18:23:38.094179+0000","last_peered":"2023-11-28T18:23:38.094179+0000","last_clean":"2023-11-28T18:23:38.094179+0000","last_became_active":"2023-11-28T18:23:33.932201+0000","last_became_peered":"2023-11-28T18:23:33.932201+0000","last_unstale":"2023-11-28T18:23:38.094179+0000","last_undegraded":"2023-11-28T18:23:38.094179+0000","last_fullsized":"2023-11-28T18:23:38.094179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:01.912695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077881+0000","last_change":"2023-11-28T18:23:33.954178+0000","last_active":"2023-11-28T18:23:38.077881+0000","last_peered":"2023-11-28T18:23:38.077881+0000","last_clean":"2023-11-28T18:23:38.077881+0000","last_became_active":"2023-11-28T18:23:33.954020+0000","last_became_peered":"2023-11-28T18:23:33.954020+0000","last_unstale":"2023-11-28T18:23:38.077881+0000","last_undegraded":"2023-11-28T18:23:38.077881+0000","last_fullsized":"2023-11-28T18:23:38.077881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:01.912695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.066974+0000","last_change":"2023-11-28T18:23:33.939474+0000","last_active":"2023-11-28T18:23:38.066974+0000","last_peered":"2023-11-28T18:23:38.066974+0000","last_clean":"2023-11-28T18:23:38.066974+0000","last_became_active":"2023-11-28T18:23:33.939189+0000","last_became_peered":"2023-11-28T18:23:33.939189+0000","last_unstale":"2023-11-28T18:23:38.066974+0000","last_undegraded":"2023-11-28T18:23:38.066974+0000","last_fullsized":"2023-11-28T18:23:38.066974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:08.033209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.070751+0000","last_change":"2023-11-28T18:23:33.929306+0000","last_active":"2023-11-28T18:23:38.070751+0000","last_peered":"2023-11-28T18:23:38.070751+0000","last_clean":"2023-11-28T18:23:38.070751+0000","last_became_active":"2023-11-28T18:23:33.929029+0000","last_became_peered":"2023-11-28T18:23:33.929029+0000","last_unstale":"2023-11-28T18:23:38.070751+0000","last_undegraded":"2023-11-28T18:23:38.070751+0000","last_fullsized":"2023-11-28T18:23:38.070751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:46.421541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065454+0000","last_change":"2023-11-28T18:23:33.922789+0000","last_active":"2023-11-28T18:23:38.065454+0000","last_peered":"2023-11-28T18:23:38.065454+0000","last_clean":"2023-11-28T18:23:38.065454+0000","last_became_active":"2023-11-28T18:23:33.922458+0000","last_became_peered":"2023-11-28T18:23:33.922458+0000","last_unstale":"2023-11-28T18:23:38.065454+0000","last_undegraded":"2023-11-28T18:23:38.065454+0000","last_fullsized":"2023-11-28T18:23:38.065454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:51.652679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068465+0000","last_change":"2023-11-28T18:23:33.935519+0000","last_active":"2023-11-28T18:23:38.068465+0000","last_peered":"2023-11-28T18:23:38.068465+0000","last_clean":"2023-11-28T18:23:38.068465+0000","last_became_active":"2023-11-28T18:23:33.934779+0000","last_became_peered":"2023-11-28T18:23:33.934779+0000","last_unstale":"2023-11-28T18:23:38.068465+0000","last_undegraded":"2023-11-28T18:23:38.068465+0000","last_fullsized":"2023-11-28T18:23:38.068465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:27.506820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077522+0000","last_change":"2023-11-28T18:23:33.946486+0000","last_active":"2023-11-28T18:23:38.077522+0000","last_peered":"2023-11-28T18:23:38.077522+0000","last_clean":"2023-11-28T18:23:38.077522+0000","last_became_active":"2023-11-28T18:23:33.946212+0000","last_became_peered":"2023-11-28T18:23:33.946212+0000","last_unstale":"2023-11-28T18:23:38.077522+0000","last_undegraded":"2023-11-28T18:23:38.077522+0000","last_fullsized":"2023-11-28T18:23:38.077522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:17:14.412292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.076300+0000","last_change":"2023-11-28T18:23:33.949577+0000","last_active":"2023-11-28T18:23:38.076300+0000","last_peered":"2023-11-28T18:23:38.076300+0000","last_clean":"2023-11-28T18:23:38.076300+0000","last_became_active":"2023-11-28T18:23:33.949275+0000","last_became_peered":"2023-11-28T18:23:33.949275+0000","last_unstale":"2023-11-28T18:23:38.076300+0000","last_undegraded":"2023-11-28T18:23:38.076300+0000","last_fullsized":"2023-11-28T18:23:38.076300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:17:14.412292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420401+0000","last_change":"2023-11-28T18:23:33.939143+0000","last_active":"2023-11-28T18:23:38.420401+0000","last_peered":"2023-11-28T18:23:38.420401+0000","last_clean":"2023-11-28T18:23:38.420401+0000","last_became_active":"2023-11-28T18:23:33.938673+0000","last_became_peered":"2023-11-28T18:23:33.938673+0000","last_unstale":"2023-11-28T18:23:38.420401+0000","last_undegraded":"2023-11-28T18:23:38.420401+0000","last_fullsized":"2023-11-28T18:23:38.420401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:11.439436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.083975+0000","last_change":"2023-11-28T18:23:33.946701+0000","last_active":"2023-11-28T18:23:38.083975+0000","last_peered":"2023-11-28T18:23:38.083975+0000","last_clean":"2023-11-28T18:23:38.083975+0000","last_became_active":"2023-11-28T18:23:33.946460+0000","last_became_peered":"2023-11-28T18:23:33.946460+0000","last_unstale":"2023-11-28T18:23:38.083975+0000","last_undegraded":"2023-11-28T18:23:38.083975+0000","last_fullsized":"2023-11-28T18:23:38.083975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:11.792676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068495+0000","last_change":"2023-11-28T18:23:33.940275+0000","last_active":"2023-11-28T18:23:38.068495+0000","last_peered":"2023-11-28T18:23:38.068495+0000","last_clean":"2023-11-28T18:23:38.068495+0000","last_became_active":"2023-11-28T18:23:33.939937+0000","last_became_peered":"2023-11-28T18:23:33.939937+0000","last_unstale":"2023-11-28T18:23:38.068495+0000","last_undegraded":"2023-11-28T18:23:38.068495+0000","last_fullsized":"2023-11-28T18:23:38.068495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:11.439436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.070691+0000","last_change":"2023-11-28T18:23:33.937355+0000","last_active":"2023-11-28T18:23:38.070691+0000","last_peered":"2023-11-28T18:23:38.070691+0000","last_clean":"2023-11-28T18:23:38.070691+0000","last_became_active":"2023-11-28T18:23:33.937069+0000","last_became_peered":"2023-11-28T18:23:33.937069+0000","last_unstale":"2023-11-28T18:23:38.070691+0000","last_undegraded":"2023-11-28T18:23:38.070691+0000","last_fullsized":"2023-11-28T18:23:38.070691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:05.773283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.092623+0000","last_change":"2023-11-28T18:23:33.938555+0000","last_active":"2023-11-28T18:23:38.092623+0000","last_peered":"2023-11-28T18:23:38.092623+0000","last_clean":"2023-11-28T18:23:38.092623+0000","last_became_active":"2023-11-28T18:23:33.938297+0000","last_became_peered":"2023-11-28T18:23:33.938297+0000","last_unstale":"2023-11-28T18:23:38.092623+0000","last_undegraded":"2023-11-28T18:23:38.092623+0000","last_fullsized":"2023-11-28T18:23:38.092623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:51.652679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.080082+0000","last_change":"2023-11-28T18:23:33.942250+0000","last_active":"2023-11-28T18:23:38.080082+0000","last_peered":"2023-11-28T18:23:38.080082+0000","last_clean":"2023-11-28T18:23:38.080082+0000","last_became_active":"2023-11-28T18:23:33.941935+0000","last_became_peered":"2023-11-28T18:23:33.941935+0000","last_unstale":"2023-11-28T18:23:38.080082+0000","last_undegraded":"2023-11-28T18:23:38.080082+0000","last_fullsized":"2023-11-28T18:23:38.080082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:21.035422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.065409+0000","last_change":"2023-11-28T18:23:33.944983+0000","last_active":"2023-11-28T18:23:38.065409+0000","last_peered":"2023-11-28T18:23:38.065409+0000","last_clean":"2023-11-28T18:23:38.065409+0000","last_became_active":"2023-11-28T18:23:33.944535+0000","last_became_peered":"2023-11-28T18:23:33.944535+0000","last_unstale":"2023-11-28T18:23:38.065409+0000","last_undegraded":"2023-11-28T18:23:38.065409+0000","last_fullsized":"2023-11-28T18:23:38.065409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:01.912695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075790+0000","last_change":"2023-11-28T18:23:33.945213+0000","last_active":"2023-11-28T18:23:38.075790+0000","last_peered":"2023-11-28T18:23:38.075790+0000","last_clean":"2023-11-28T18:23:38.075790+0000","last_became_active":"2023-11-28T18:23:33.944605+0000","last_became_peered":"2023-11-28T18:23:33.944605+0000","last_unstale":"2023-11-28T18:23:38.075790+0000","last_undegraded":"2023-11-28T18:23:38.075790+0000","last_fullsized":"2023-11-28T18:23:38.075790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:46.149525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.076131+0000","last_change":"2023-11-28T18:23:33.948453+0000","last_active":"2023-11-28T18:23:38.076131+0000","last_peered":"2023-11-28T18:23:38.076131+0000","last_clean":"2023-11-28T18:23:38.076131+0000","last_became_active":"2023-11-28T18:23:33.948203+0000","last_became_peered":"2023-11-28T18:23:33.948203+0000","last_unstale":"2023-11-28T18:23:38.076131+0000","last_undegraded":"2023-11-28T18:23:38.076131+0000","last_fullsized":"2023-11-28T18:23:38.076131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:51.652679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.077793+0000","last_change":"2023-11-28T18:23:33.945564+0000","last_active":"2023-11-28T18:23:38.077793+0000","last_peered":"2023-11-28T18:23:38.077793+0000","last_clean":"2023-11-28T18:23:38.077793+0000","last_became_active":"2023-11-28T18:23:33.943276+0000","last_became_peered":"2023-11-28T18:23:33.943276+0000","last_unstale":"2023-11-28T18:23:38.077793+0000","last_undegraded":"2023-11-28T18:23:38.077793+0000","last_fullsized":"2023-11-28T18:23:38.077793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:21.035422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420502+0000","last_change":"2023-11-28T18:23:33.942649+0000","last_active":"2023-11-28T18:23:38.420502+0000","last_peered":"2023-11-28T18:23:38.420502+0000","last_clean":"2023-11-28T18:23:38.420502+0000","last_became_active":"2023-11-28T18:23:33.942109+0000","last_became_peered":"2023-11-28T18:23:33.942109+0000","last_unstale":"2023-11-28T18:23:38.420502+0000","last_undegraded":"2023-11-28T18:23:38.420502+0000","last_fullsized":"2023-11-28T18:23:38.420502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:34.989175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.064784+0000","last_change":"2023-11-28T18:23:33.933929+0000","last_active":"2023-11-28T18:23:38.064784+0000","last_peered":"2023-11-28T18:23:38.064784+0000","last_clean":"2023-11-28T18:23:38.064784+0000","last_became_active":"2023-11-28T18:23:33.933087+0000","last_became_peered":"2023-11-28T18:23:33.933087+0000","last_unstale":"2023-11-28T18:23:38.064784+0000","last_undegraded":"2023-11-28T18:23:38.064784+0000","last_fullsized":"2023-11-28T18:23:38.064784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:12.557669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419632+0000","last_change":"2023-11-28T18:23:33.950276+0000","last_active":"2023-11-28T18:23:38.419632+0000","last_peered":"2023-11-28T18:23:38.419632+0000","last_clean":"2023-11-28T18:23:38.419632+0000","last_became_active":"2023-11-28T18:23:33.946777+0000","last_became_peered":"2023-11-28T18:23:33.946777+0000","last_unstale":"2023-11-28T18:23:38.419632+0000","last_undegraded":"2023-11-28T18:23:38.419632+0000","last_fullsized":"2023-11-28T18:23:38.419632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:46.421541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.068159+0000","last_change":"2023-11-28T18:23:33.943666+0000","last_active":"2023-11-28T18:23:38.068159+0000","last_peered":"2023-11-28T18:23:38.068159+0000","last_clean":"2023-11-28T18:23:38.068159+0000","last_became_active":"2023-11-28T18:23:33.940798+0000","last_became_peered":"2023-11-28T18:23:33.940798+0000","last_unstale":"2023-11-28T18:23:38.068159+0000","last_undegraded":"2023-11-28T18:23:38.068159+0000","last_fullsized":"2023-11-28T18:23:38.068159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:04:56.118542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.419726+0000","last_change":"2023-11-28T18:23:33.955086+0000","last_active":"2023-11-28T18:23:38.419726+0000","last_peered":"2023-11-28T18:23:38.419726+0000","last_clean":"2023-11-28T18:23:38.419726+0000","last_became_active":"2023-11-28T18:23:33.950311+0000","last_became_peered":"2023-11-28T18:23:33.950311+0000","last_unstale":"2023-11-28T18:23:38.419726+0000","last_undegraded":"2023-11-28T18:23:38.419726+0000","last_fullsized":"2023-11-28T18:23:38.419726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:27.506820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.067660+0000","last_change":"2023-11-28T18:23:33.932833+0000","last_active":"2023-11-28T18:23:38.067660+0000","last_peered":"2023-11-28T18:23:38.067660+0000","last_clean":"2023-11-28T18:23:38.067660+0000","last_became_active":"2023-11-28T18:23:33.932554+0000","last_became_peered":"2023-11-28T18:23:33.932554+0000","last_unstale":"2023-11-28T18:23:38.067660+0000","last_undegraded":"2023-11-28T18:23:38.067660+0000","last_fullsized":"2023-11-28T18:23:38.067660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:08.033209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.075353+0000","last_change":"2023-11-28T18:23:33.939136+0000","last_active":"2023-11-28T18:23:38.075353+0000","last_peered":"2023-11-28T18:23:38.075353+0000","last_clean":"2023-11-28T18:23:38.075353+0000","last_became_active":"2023-11-28T18:23:33.938870+0000","last_became_peered":"2023-11-28T18:23:33.938870+0000","last_unstale":"2023-11-28T18:23:38.075353+0000","last_undegraded":"2023-11-28T18:23:38.075353+0000","last_fullsized":"2023-11-28T18:23:38.075353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:05.773283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.096250+0000","last_change":"2023-11-28T18:23:33.936563+0000","last_active":"2023-11-28T18:23:38.096250+0000","last_peered":"2023-11-28T18:23:38.096250+0000","last_clean":"2023-11-28T18:23:38.096250+0000","last_became_active":"2023-11-28T18:23:33.936272+0000","last_became_peered":"2023-11-28T18:23:33.936272+0000","last_unstale":"2023-11-28T18:23:38.096250+0000","last_undegraded":"2023-11-28T18:23:38.096250+0000","last_fullsized":"2023-11-28T18:23:38.096250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:04:56.118542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T18:23:38.420445+0000","last_change":"2023-11-28T18:23:33.954685+0000","last_active":"2023-11-28T18:23:38.420445+0000","last_peered":"2023-11-28T18:23:38.420445+0000","last_clean":"2023-11-28T18:23:38.420445+0000","last_became_active":"2023-11-28T18:23:33.954434+0000","last_became_peered":"2023-11-28T18:23:33.954434+0000","last_unstale":"2023-11-28T18:23:38.420445+0000","last_undegraded":"2023-11-28T18:23:38.420445+0000","last_fullsized":"2023-11-28T18:23:38.420445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:32.892592+0000","last_clean_scrub_stamp":"2023-11-28T18:23:32.892592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:38.277787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":690742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.427}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3849999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4590000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2730000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":99882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":99918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":99792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":99952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":690798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:23:57.151 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T18:23:57.151 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-28T18:23:57.151 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-28T18:23:57.151 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T18:23:57.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.327+0000 7f893aff8640 1 Processor -- start 2023-11-28T18:23:57.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.327+0000 7f893aff8640 1 -- start start 2023-11-28T18:23:57.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.327+0000 7f893aff8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f89340b3f50 0x7f89340cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:57.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.327+0000 7f893aff8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8934160eb0 con 0x7f8934146440 2023-11-28T18:23:57.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.327+0000 7f893aff8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8934161190 con 0x7f89340b3b70 2023-11-28T18:23:57.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.327+0000 7f893aff8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8934161b30 con 0x7f89340b3f50 2023-11-28T18:23:57.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.327+0000 7f8933fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f89340b3f50 0x7f89340cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:57.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.327+0000 7f8933fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f89340b3f50 0x7f89340cefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55362/0 (socket says 172.21.15.105:55362) 2023-11-28T18:23:57.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.327+0000 7f8939595640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8934146440 0x7f89340b2060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39340/0 (socket says 172.21.15.105:39340) 2023-11-28T18:23:57.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.327+0000 7f8933fff640 1 -- 172.21.15.105:0/2797342947 learned_addr learned my addr 172.21.15.105:0/2797342947 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:57.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.327+0000 7f8933fff640 1 -- 172.21.15.105:0/2797342947 >> v1:172.21.15.148:6789/0 conn(0x7f89340b3b70 legacy=0x7f893406dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:57.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.327+0000 7f8933fff640 1 -- 172.21.15.105:0/2797342947 >> v1:172.21.15.105:6789/0 conn(0x7f8934146440 legacy=0x7f89340b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:57.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.327+0000 7f8933fff640 1 -- 172.21.15.105:0/2797342947 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8934161470 con 0x7f89340b3f50 2023-11-28T18:23:57.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.331+0000 7f8933fff640 1 --2- 172.21.15.105:0/2797342947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89340b3f50 0x7f89340cefa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8920009870 tx=0x7f892001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367074fac85efcb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:57.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.331+0000 7f89337fe640 1 -- 172.21.15.105:0/2797342947 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89200ab600 con 0x7f89340b3f50 2023-11-28T18:23:57.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.331+0000 7f89337fe640 1 -- 172.21.15.105:0/2797342947 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89200a7050 con 0x7f89340b3f50 2023-11-28T18:23:57.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.331+0000 7f89337fe640 1 -- 172.21.15.105:0/2797342947 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89200ab980 con 0x7f89340b3f50 2023-11-28T18:23:57.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.331+0000 7f893aff8640 1 -- 172.21.15.105:0/2797342947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89340b3f50 msgr2=0x7f89340cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:57.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.331+0000 7f893aff8640 1 --2- 172.21.15.105:0/2797342947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89340b3f50 0x7f89340cefa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8920009870 tx=0x7f892001c540 comp rx=0 tx=0).stop 2023-11-28T18:23:57.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.331+0000 7f893aff8640 1 -- 172.21.15.105:0/2797342947 shutdown_connections 2023-11-28T18:23:57.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.331+0000 7f893aff8640 1 --2- 172.21.15.105:0/2797342947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89340b3f50 0x7f89340cefa0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:57.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.331+0000 7f893aff8640 1 -- 172.21.15.105:0/2797342947 >> 172.21.15.105:0/2797342947 conn(0x7f893406a980 msgr2=0x7f893406ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:57.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.331+0000 7f893aff8640 1 -- 172.21.15.105:0/2797342947 shutdown_connections 2023-11-28T18:23:57.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.331+0000 7f893aff8640 1 -- 172.21.15.105:0/2797342947 wait complete. 2023-11-28T18:23:57.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.331+0000 7f893aff8640 1 Processor -- start 2023-11-28T18:23:57.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f893aff8640 1 -- start start 2023-11-28T18:23:57.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f893aff8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89340b3b70 0x7f893412bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:57.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f893aff8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f89340b3f50 0x7f8934127b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:57.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f893aff8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8934146440 0x7f89340ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:57.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f893aff8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89341280a0 con 0x7f89340b3b70 2023-11-28T18:23:57.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f893aff8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8934128350 con 0x7f8934146440 2023-11-28T18:23:57.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f893aff8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8934128630 con 0x7f89340b3f50 2023-11-28T18:23:57.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f8938d94640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89340b3b70 0x7f893412bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:57.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f8938d94640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89340b3b70 0x7f893412bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55368/0 (socket says 172.21.15.105:55368) 2023-11-28T18:23:57.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f8938d94640 1 -- 172.21.15.105:0/3154012463 learned_addr learned my addr 172.21.15.105:0/3154012463 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:57.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f8939595640 1 --2- 172.21.15.105:0/3154012463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8934146440 0x7f89340ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:57.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f8933fff640 1 --2- 172.21.15.105:0/3154012463 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f89340b3f50 0x7f8934127b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:57.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f8938d94640 1 -- 172.21.15.105:0/3154012463 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f89340b3f50 msgr2=0x7f8934127b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:57.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f8938d94640 1 --2- 172.21.15.105:0/3154012463 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f89340b3f50 0x7f8934127b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:57.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f8938d94640 1 -- 172.21.15.105:0/3154012463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8934146440 msgr2=0x7f89340ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:57.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f8938d94640 1 --2- 172.21.15.105:0/3154012463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8934146440 0x7f89340ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:57.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f8938d94640 1 -- 172.21.15.105:0/3154012463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89200094d0 con 0x7f89340b3b70 2023-11-28T18:23:57.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f8939595640 1 --2- 172.21.15.105:0/3154012463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8934146440 0x7f89340ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:57.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f8938d94640 1 --2- 172.21.15.105:0/3154012463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89340b3b70 0x7f893412bb50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f892800c3c0 tx=0x7f892800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:57.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f89317fa640 1 -- 172.21.15.105:0/3154012463 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89280052d0 con 0x7f89340b3b70 2023-11-28T18:23:57.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f89317fa640 1 -- 172.21.15.105:0/3154012463 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8928005430 con 0x7f89340b3b70 2023-11-28T18:23:57.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f89317fa640 1 -- 172.21.15.105:0/3154012463 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8928003840 con 0x7f89340b3b70 2023-11-28T18:23:57.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f893aff8640 1 -- 172.21.15.105:0/3154012463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8934128910 con 0x7f89340b3b70 2023-11-28T18:23:57.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f893aff8640 1 -- 172.21.15.105:0/3154012463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8934140c90 con 0x7f89340b3b70 2023-11-28T18:23:57.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.335+0000 7f893aff8640 1 -- 172.21.15.105:0/3154012463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8934067570 con 0x7f89340b3b70 2023-11-28T18:23:57.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.339+0000 7f89317fa640 1 -- 172.21.15.105:0/3154012463 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f89280039a0 con 0x7f89340b3b70 2023-11-28T18:23:57.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.339+0000 7f89317fa640 1 --2- 172.21.15.105:0/3154012463 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f89180668d0 0x7f8918068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:57.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.339+0000 7f89317fa640 1 -- 172.21.15.105:0/3154012463 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8928083c90 con 0x7f89340b3b70 2023-11-28T18:23:57.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.339+0000 7f8933fff640 1 --2- 172.21.15.105:0/3154012463 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f89180668d0 0x7f8918068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:57.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.339+0000 7f8933fff640 1 --2- 172.21.15.105:0/3154012463 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f89180668d0 0x7f8918068d90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8934071ed0 tx=0x7f8920008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:57.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.343+0000 7f89317fa640 1 -- 172.21.15.105:0/3154012463 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8928050bd0 con 0x7f89340b3b70 2023-11-28T18:23:57.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.523+0000 7f893aff8640 1 -- 172.21.15.105:0/3154012463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f89340723d0 con 0x7f89340b3b70 2023-11-28T18:23:57.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.523+0000 7f89317fa640 1 -- 172.21.15.105:0/3154012463 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f8928017040 con 0x7f89340b3b70 2023-11-28T18:23:57.527 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:57.528 INFO:teuthology.orchestra.run.smithi105.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-28T18:23:57.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.527+0000 7f893aff8640 1 -- 172.21.15.105:0/3154012463 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f89180668d0 msgr2=0x7f8918068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:57.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.527+0000 7f893aff8640 1 --2- 172.21.15.105:0/3154012463 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f89180668d0 0x7f8918068d90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8934071ed0 tx=0x7f8920008040 comp rx=0 tx=0).stop 2023-11-28T18:23:57.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.527+0000 7f893aff8640 1 -- 172.21.15.105:0/3154012463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89340b3b70 msgr2=0x7f893412bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:57.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.527+0000 7f893aff8640 1 --2- 172.21.15.105:0/3154012463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89340b3b70 0x7f893412bb50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f892800c3c0 tx=0x7f892800c880 comp rx=0 tx=0).stop 2023-11-28T18:23:57.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.527+0000 7f893aff8640 1 -- 172.21.15.105:0/3154012463 shutdown_connections 2023-11-28T18:23:57.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.527+0000 7f893aff8640 1 --2- 172.21.15.105:0/3154012463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8934146440 0x7f89340ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:57.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.527+0000 7f893aff8640 1 --2- 172.21.15.105:0/3154012463 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f89340b3f50 0x7f8934127b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:57.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.527+0000 7f893aff8640 1 --2- 172.21.15.105:0/3154012463 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f89180668d0 0x7f8918068d90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:57.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.527+0000 7f893aff8640 1 --2- 172.21.15.105:0/3154012463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89340b3b70 0x7f893412bb50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:57.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.527+0000 7f893aff8640 1 -- 172.21.15.105:0/3154012463 >> 172.21.15.105:0/3154012463 conn(0x7f893406a980 msgr2=0x7f8934099cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:57.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.527+0000 7f893aff8640 1 -- 172.21.15.105:0/3154012463 shutdown_connections 2023-11-28T18:23:57.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.527+0000 7f893aff8640 1 -- 172.21.15.105:0/3154012463 wait complete. 2023-11-28T18:23:57.550 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-28T18:23:57.550 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:23:57.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.683+0000 7ffbe791d640 1 Processor -- start 2023-11-28T18:23:57.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.683+0000 7ffbe791d640 1 -- start start 2023-11-28T18:23:57.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.683+0000 7ffbe791d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffbe00ad9f0 0x7ffbe00ade80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:57.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.683+0000 7ffbe791d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbe00cc7d0 0x7ffbe00ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:57.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.683+0000 7ffbe791d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe0162820 con 0x7ffbe006d610 2023-11-28T18:23:57.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.683+0000 7ffbe791d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe00ae3c0 con 0x7ffbe00ad9f0 2023-11-28T18:23:57.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.683+0000 7ffbe791d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe0161c10 con 0x7ffbe00cc7d0 2023-11-28T18:23:57.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.683+0000 7ffbe56b9640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ffbe006d610 0x7ffbe00cdff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39346/0 (socket says 172.21.15.105:39346) 2023-11-28T18:23:57.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.683+0000 7ffbe56b9640 1 -- 172.21.15.105:0/1078936262 learned_addr learned my addr 172.21.15.105:0/1078936262 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:57.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.683+0000 7ffbe4eb8640 1 --2- 172.21.15.105:0/1078936262 >> v2:172.21.15.105:3300/0 conn(0x7ffbe00ad9f0 0x7ffbe00ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:57.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.683+0000 7ffbe5eba640 1 --2- 172.21.15.105:0/1078936262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbe00cc7d0 0x7ffbe00ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:57.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.687+0000 7ffbe4eb8640 1 -- 172.21.15.105:0/1078936262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbe00cc7d0 msgr2=0x7ffbe00ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:57.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.687+0000 7ffbe4eb8640 1 --2- 172.21.15.105:0/1078936262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbe00cc7d0 0x7ffbe00ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:57.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.687+0000 7ffbe4eb8640 1 -- 172.21.15.105:0/1078936262 >> v1:172.21.15.105:6789/0 conn(0x7ffbe006d610 legacy=0x7ffbe00cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:57.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.687+0000 7ffbe4eb8640 1 -- 172.21.15.105:0/1078936262 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe0161ef0 con 0x7ffbe00ad9f0 2023-11-28T18:23:57.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.687+0000 7ffbe5eba640 1 --2- 172.21.15.105:0/1078936262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbe00cc7d0 0x7ffbe00ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:57.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.687+0000 7ffbe4eb8640 1 --2- 172.21.15.105:0/1078936262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbe00ad9f0 0x7ffbe00ade80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00097c0 tx=0x7ffbd001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f8ea5ae5e20e82 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:57.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.687+0000 7ffbd7fff640 1 -- 172.21.15.105:0/1078936262 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd00ab650 con 0x7ffbe00ad9f0 2023-11-28T18:23:57.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.687+0000 7ffbd7fff640 1 -- 172.21.15.105:0/1078936262 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd00ab7b0 con 0x7ffbe00ad9f0 2023-11-28T18:23:57.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.687+0000 7ffbd7fff640 1 -- 172.21.15.105:0/1078936262 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd00abb40 con 0x7ffbe00ad9f0 2023-11-28T18:23:57.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.687+0000 7ffbe791d640 1 -- 172.21.15.105:0/1078936262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbe00ad9f0 msgr2=0x7ffbe00ade80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:57.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.687+0000 7ffbe791d640 1 --2- 172.21.15.105:0/1078936262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbe00ad9f0 0x7ffbe00ade80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00097c0 tx=0x7ffbd001c450 comp rx=0 tx=0).stop 2023-11-28T18:23:57.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe791d640 1 -- 172.21.15.105:0/1078936262 shutdown_connections 2023-11-28T18:23:57.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe791d640 1 --2- 172.21.15.105:0/1078936262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbe00cc7d0 0x7ffbe00ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:57.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe791d640 1 --2- 172.21.15.105:0/1078936262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbe00ad9f0 0x7ffbe00ade80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:57.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe791d640 1 -- 172.21.15.105:0/1078936262 >> 172.21.15.105:0/1078936262 conn(0x7ffbe00ceb90 msgr2=0x7ffbe00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:57.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe791d640 1 -- 172.21.15.105:0/1078936262 shutdown_connections 2023-11-28T18:23:57.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe791d640 1 -- 172.21.15.105:0/1078936262 wait complete. 2023-11-28T18:23:57.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe791d640 1 Processor -- start 2023-11-28T18:23:57.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe791d640 1 -- start start 2023-11-28T18:23:57.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe791d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbe006d610 0x7ffbe01655b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:57.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe791d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbe00ad9f0 0x7ffbe0156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:57.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe791d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbe00cc7d0 0x7ffbe01505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:57.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe791d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe015ee20 con 0x7ffbe00cc7d0 2023-11-28T18:23:57.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe791d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe0164570 con 0x7ffbe006d610 2023-11-28T18:23:57.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe791d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe0164850 con 0x7ffbe00ad9f0 2023-11-28T18:23:57.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe5eba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbe00cc7d0 0x7ffbe01505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:57.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe5eba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbe00cc7d0 0x7ffbe01505d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55386/0 (socket says 172.21.15.105:55386) 2023-11-28T18:23:57.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe5eba640 1 -- 172.21.15.105:0/1212804803 learned_addr learned my addr 172.21.15.105:0/1212804803 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:57.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe56b9640 1 --2- 172.21.15.105:0/1212804803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbe006d610 0x7ffbe01655b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:57.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe4eb8640 1 --2- 172.21.15.105:0/1212804803 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbe00ad9f0 0x7ffbe0156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:57.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe5eba640 1 -- 172.21.15.105:0/1212804803 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbe00ad9f0 msgr2=0x7ffbe0156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:57.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe5eba640 1 --2- 172.21.15.105:0/1212804803 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbe00ad9f0 0x7ffbe0156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:57.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe5eba640 1 -- 172.21.15.105:0/1212804803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbe006d610 msgr2=0x7ffbe01655b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:57.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe5eba640 1 --2- 172.21.15.105:0/1212804803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbe006d610 0x7ffbe01655b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:57.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe5eba640 1 -- 172.21.15.105:0/1212804803 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd0009420 con 0x7ffbe00cc7d0 2023-11-28T18:23:57.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.691+0000 7ffbe56b9640 1 --2- 172.21.15.105:0/1212804803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbe006d610 0x7ffbe01655b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:57.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.695+0000 7ffbe5eba640 1 --2- 172.21.15.105:0/1212804803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbe00cc7d0 0x7ffbe01505d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8007ef0 tx=0x7ffbd800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:57.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.695+0000 7ffbd5ffb640 1 -- 172.21.15.105:0/1212804803 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8003ba0 con 0x7ffbe00cc7d0 2023-11-28T18:23:57.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.695+0000 7ffbd5ffb640 1 -- 172.21.15.105:0/1212804803 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd800f040 con 0x7ffbe00cc7d0 2023-11-28T18:23:57.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.695+0000 7ffbd5ffb640 1 -- 172.21.15.105:0/1212804803 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd800de80 con 0x7ffbe00cc7d0 2023-11-28T18:23:57.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.695+0000 7ffbe791d640 1 -- 172.21.15.105:0/1212804803 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe0164b30 con 0x7ffbe00cc7d0 2023-11-28T18:23:57.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.695+0000 7ffbe791d640 1 -- 172.21.15.105:0/1212804803 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe01641d0 con 0x7ffbe00cc7d0 2023-11-28T18:23:57.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.695+0000 7ffbe791d640 1 -- 172.21.15.105:0/1212804803 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe0067570 con 0x7ffbe00cc7d0 2023-11-28T18:23:57.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.695+0000 7ffbd5ffb640 1 -- 172.21.15.105:0/1212804803 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ffbd8004030 con 0x7ffbe00cc7d0 2023-11-28T18:23:57.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.695+0000 7ffbd5ffb640 1 --2- 172.21.15.105:0/1212804803 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffbc40668d0 0x7ffbc4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:57.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.695+0000 7ffbd5ffb640 1 -- 172.21.15.105:0/1212804803 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ffbd8084ab0 con 0x7ffbe00cc7d0 2023-11-28T18:23:57.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.695+0000 7ffbe56b9640 1 --2- 172.21.15.105:0/1212804803 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffbc40668d0 0x7ffbc4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:57.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.699+0000 7ffbe56b9640 1 --2- 172.21.15.105:0/1212804803 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffbc40668d0 0x7ffbc4068d90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc005890 tx=0x7ffbdc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:57.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.699+0000 7ffbd5ffb640 1 -- 172.21.15.105:0/1212804803 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ffbd8051a10 con 0x7ffbe00cc7d0 2023-11-28T18:23:57.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.851+0000 7ffbe791d640 1 -- 172.21.15.105:0/1212804803 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffbe015f290 con 0x7ffbe00cc7d0 2023-11-28T18:23:57.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.851+0000 7ffbd5ffb640 1 -- 172.21.15.105:0/1212804803 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7ffbd801a090 con 0x7ffbe00cc7d0 2023-11-28T18:23:57.856 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:57.856 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":24,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:23:38.458954+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:23:57.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.855+0000 7ffbe791d640 1 -- 172.21.15.105:0/1212804803 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffbc40668d0 msgr2=0x7ffbc4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.855+0000 7ffbe791d640 1 --2- 172.21.15.105:0/1212804803 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffbc40668d0 0x7ffbc4068d90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc005890 tx=0x7ffbdc009040 comp rx=0 tx=0).stop 2023-11-28T18:23:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.855+0000 7ffbe791d640 1 -- 172.21.15.105:0/1212804803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbe00cc7d0 msgr2=0x7ffbe01505d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.855+0000 7ffbe791d640 1 --2- 172.21.15.105:0/1212804803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbe00cc7d0 0x7ffbe01505d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8007ef0 tx=0x7ffbd800d4a0 comp rx=0 tx=0).stop 2023-11-28T18:23:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.855+0000 7ffbe791d640 1 -- 172.21.15.105:0/1212804803 shutdown_connections 2023-11-28T18:23:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.855+0000 7ffbe791d640 1 --2- 172.21.15.105:0/1212804803 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffbc40668d0 0x7ffbc4068d90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.855+0000 7ffbe791d640 1 --2- 172.21.15.105:0/1212804803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbe00cc7d0 0x7ffbe01505d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.855+0000 7ffbe791d640 1 --2- 172.21.15.105:0/1212804803 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbe00ad9f0 0x7ffbe0156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.855+0000 7ffbe791d640 1 --2- 172.21.15.105:0/1212804803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbe006d610 0x7ffbe01655b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.855+0000 7ffbe791d640 1 -- 172.21.15.105:0/1212804803 >> 172.21.15.105:0/1212804803 conn(0x7ffbe00ceb90 msgr2=0x7ffbe00cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.855+0000 7ffbe791d640 1 -- 172.21.15.105:0/1212804803 shutdown_connections 2023-11-28T18:23:57.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:57.855+0000 7ffbe791d640 1 -- 172.21.15.105:0/1212804803 wait complete. 2023-11-28T18:23:57.877 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:23:31.485422+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:23:32.274378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T18:23:36.100071+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T18:23:57.877 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:23:58.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.015+0000 7fb1db757640 1 Processor -- start 2023-11-28T18:23:58.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.015+0000 7fb1db757640 1 -- start start 2023-11-28T18:23:58.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.019+0000 7fb1db757640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb1d4159f40 0x7fb1d40caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:58.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.019+0000 7fb1db757640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb1d40ad850 0x7fb1d40701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:58.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.019+0000 7fb1db757640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d406bf50 con 0x7fb1d40cc760 2023-11-28T18:23:58.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.019+0000 7fb1db757640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d40ccbc0 con 0x7fb1d40ad850 2023-11-28T18:23:58.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.019+0000 7fb1db757640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d415a320 con 0x7fb1d4159f40 2023-11-28T18:23:58.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.019+0000 7fb1d9cf4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb1d40ad850 0x7fb1d40701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:58.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.019+0000 7fb1d9cf4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb1d40ad850 0x7fb1d40701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55394/0 (socket says 172.21.15.105:55394) 2023-11-28T18:23:58.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.019+0000 7fb1d9cf4640 1 -- 172.21.15.105:0/76034142 learned_addr learned my addr 172.21.15.105:0/76034142 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:58.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.019+0000 7fb1d94f3640 1 --2- 172.21.15.105:0/76034142 >> v2:172.21.15.148:3300/0 conn(0x7fb1d4159f40 0x7fb1d40caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:58.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.019+0000 7fb1d9cf4640 1 -- 172.21.15.105:0/76034142 >> v2:172.21.15.148:3300/0 conn(0x7fb1d4159f40 msgr2=0x7fb1d40caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:58.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.019+0000 7fb1d9cf4640 1 --2- 172.21.15.105:0/76034142 >> v2:172.21.15.148:3300/0 conn(0x7fb1d4159f40 0x7fb1d40caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.019+0000 7fb1d9cf4640 1 -- 172.21.15.105:0/76034142 >> v1:172.21.15.148:6789/0 conn(0x7fb1d40cc760 legacy=0x7fb1d40ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:58.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.019+0000 7fb1d9cf4640 1 -- 172.21.15.105:0/76034142 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d4071380 con 0x7fb1d40ad850 2023-11-28T18:23:58.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.023+0000 7fb1d9cf4640 1 --2- 172.21.15.105:0/76034142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d40ad850 0x7fb1d40701d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb1d406e500 tx=0x7fb1cc01c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d30fa9c806e97d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:58.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.023+0000 7fb1cbfff640 1 -- 172.21.15.105:0/76034142 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0a5790 con 0x7fb1d40ad850 2023-11-28T18:23:58.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.023+0000 7fb1cbfff640 1 -- 172.21.15.105:0/76034142 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1cc0a58f0 con 0x7fb1d40ad850 2023-11-28T18:23:58.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.023+0000 7fb1cbfff640 1 -- 172.21.15.105:0/76034142 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0a5c80 con 0x7fb1d40ad850 2023-11-28T18:23:58.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.023+0000 7fb1db757640 1 -- 172.21.15.105:0/76034142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d40ad850 msgr2=0x7fb1d40701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:58.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.023+0000 7fb1db757640 1 --2- 172.21.15.105:0/76034142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d40ad850 0x7fb1d40701d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb1d406e500 tx=0x7fb1cc01c8d0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.023+0000 7fb1db757640 1 -- 172.21.15.105:0/76034142 shutdown_connections 2023-11-28T18:23:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.023+0000 7fb1db757640 1 --2- 172.21.15.105:0/76034142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d40ad850 0x7fb1d40701d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.023+0000 7fb1db757640 1 --2- 172.21.15.105:0/76034142 >> v2:172.21.15.148:3300/0 conn(0x7fb1d4159f40 0x7fb1d40caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.023+0000 7fb1db757640 1 -- 172.21.15.105:0/76034142 >> 172.21.15.105:0/76034142 conn(0x7fb1d40ceb90 msgr2=0x7fb1d40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.023+0000 7fb1db757640 1 -- 172.21.15.105:0/76034142 shutdown_connections 2023-11-28T18:23:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.023+0000 7fb1db757640 1 -- 172.21.15.105:0/76034142 wait complete. 2023-11-28T18:23:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.023+0000 7fb1db757640 1 Processor -- start 2023-11-28T18:23:58.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.023+0000 7fb1db757640 1 -- start start 2023-11-28T18:23:58.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1db757640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1d40ad850 0x7fb1d40b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:58.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1db757640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1d40cc760 0x7fb1d41569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:58.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1db757640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d4159f40 0x7fb1d414f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:58.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1db757640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d40b4130 con 0x7fb1d4159f40 2023-11-28T18:23:58.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1db757640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d413e090 con 0x7fb1d40ad850 2023-11-28T18:23:58.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1db757640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d40b3ae0 con 0x7fb1d40cc760 2023-11-28T18:23:58.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1d9cf4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d4159f40 0x7fb1d414f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:58.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1d94f3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1d40ad850 0x7fb1d40b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:58.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1d9cf4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d4159f40 0x7fb1d414f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55406/0 (socket says 172.21.15.105:55406) 2023-11-28T18:23:58.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1d9cf4640 1 -- 172.21.15.105:0/3308979648 learned_addr learned my addr 172.21.15.105:0/3308979648 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:58.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1d8cf2640 1 --2- 172.21.15.105:0/3308979648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1d40cc760 0x7fb1d41569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:58.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1d9cf4640 1 -- 172.21.15.105:0/3308979648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1d40cc760 msgr2=0x7fb1d41569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:58.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1d9cf4640 1 --2- 172.21.15.105:0/3308979648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1d40cc760 0x7fb1d41569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1d9cf4640 1 -- 172.21.15.105:0/3308979648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1d40ad850 msgr2=0x7fb1d40b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:58.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1d9cf4640 1 --2- 172.21.15.105:0/3308979648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1d40ad850 0x7fb1d40b45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1d9cf4640 1 -- 172.21.15.105:0/3308979648 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc009b30 con 0x7fb1d4159f40 2023-11-28T18:23:58.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1d9cf4640 1 --2- 172.21.15.105:0/3308979648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d4159f40 0x7fb1d414f730 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0af040 tx=0x7fb1cc019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:58.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1c9ffb640 1 -- 172.21.15.105:0/3308979648 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0b8670 con 0x7fb1d4159f40 2023-11-28T18:23:58.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1c9ffb640 1 -- 172.21.15.105:0/3308979648 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1cc0b87d0 con 0x7fb1d4159f40 2023-11-28T18:23:58.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1c9ffb640 1 -- 172.21.15.105:0/3308979648 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0b8b60 con 0x7fb1d4159f40 2023-11-28T18:23:58.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1db757640 1 -- 172.21.15.105:0/3308979648 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d40b3dc0 con 0x7fb1d4159f40 2023-11-28T18:23:58.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1db757640 1 -- 172.21.15.105:0/3308979648 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d414fc70 con 0x7fb1d4159f40 2023-11-28T18:23:58.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.027+0000 7fb1db757640 1 -- 172.21.15.105:0/3308979648 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d40b3f50 con 0x7fb1d4159f40 2023-11-28T18:23:58.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.031+0000 7fb1c9ffb640 1 -- 172.21.15.105:0/3308979648 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fb1cc0b8cc0 con 0x7fb1d4159f40 2023-11-28T18:23:58.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.031+0000 7fb1c9ffb640 1 --2- 172.21.15.105:0/3308979648 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb1b80668d0 0x7fb1b8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:58.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.031+0000 7fb1c9ffb640 1 -- 172.21.15.105:0/3308979648 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb1cc11fc00 con 0x7fb1d4159f40 2023-11-28T18:23:58.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.031+0000 7fb1d94f3640 1 --2- 172.21.15.105:0/3308979648 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb1b80668d0 0x7fb1b8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:58.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.031+0000 7fb1d94f3640 1 --2- 172.21.15.105:0/3308979648 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb1b80668d0 0x7fb1b8068d90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4157890 tx=0x7fb1d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:58.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.035+0000 7fb1c9ffb640 1 -- 172.21.15.105:0/3308979648 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb1cc126080 con 0x7fb1d4159f40 2023-11-28T18:23:58.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.187+0000 7fb1db757640 1 -- 172.21.15.105:0/3308979648 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb1d40b3910 con 0x7fb1d4159f40 2023-11-28T18:23:58.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.187+0000 7fb1c9ffb640 1 -- 172.21.15.105:0/3308979648 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb1cc0bc020 con 0x7fb1d4159f40 2023-11-28T18:23:58.192 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:23:58.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.191+0000 7fb1db757640 1 -- 172.21.15.105:0/3308979648 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb1b80668d0 msgr2=0x7fb1b8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:58.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.191+0000 7fb1db757640 1 --2- 172.21.15.105:0/3308979648 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb1b80668d0 0x7fb1b8068d90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4157890 tx=0x7fb1d0008040 comp rx=0 tx=0).stop 2023-11-28T18:23:58.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.191+0000 7fb1db757640 1 -- 172.21.15.105:0/3308979648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d4159f40 msgr2=0x7fb1d414f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:58.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.191+0000 7fb1db757640 1 --2- 172.21.15.105:0/3308979648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d4159f40 0x7fb1d414f730 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0af040 tx=0x7fb1cc019670 comp rx=0 tx=0).stop 2023-11-28T18:23:58.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.191+0000 7fb1db757640 1 -- 172.21.15.105:0/3308979648 shutdown_connections 2023-11-28T18:23:58.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.191+0000 7fb1db757640 1 --2- 172.21.15.105:0/3308979648 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb1b80668d0 0x7fb1b8068d90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.191+0000 7fb1db757640 1 --2- 172.21.15.105:0/3308979648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d4159f40 0x7fb1d414f730 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.191+0000 7fb1db757640 1 --2- 172.21.15.105:0/3308979648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1d40cc760 0x7fb1d41569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.191+0000 7fb1db757640 1 --2- 172.21.15.105:0/3308979648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1d40ad850 0x7fb1d40b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.191+0000 7fb1db757640 1 -- 172.21.15.105:0/3308979648 >> 172.21.15.105:0/3308979648 conn(0x7fb1d40ceb90 msgr2=0x7fb1d415c530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:58.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.191+0000 7fb1db757640 1 -- 172.21.15.105:0/3308979648 shutdown_connections 2023-11-28T18:23:58.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.191+0000 7fb1db757640 1 -- 172.21.15.105:0/3308979648 wait complete. 2023-11-28T18:23:58.213 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:23:58.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.351+0000 7fd48e2d6640 1 Processor -- start 2023-11-28T18:23:58.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.351+0000 7fd48e2d6640 1 -- start start 2023-11-28T18:23:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.351+0000 7fd48e2d6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd48815d3c0 0x7fd488071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.351+0000 7fd48e2d6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4880cefa0 con 0x7fd488161870 2023-11-28T18:23:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.351+0000 7fd48e2d6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd48815d7a0 con 0x7fd488070a10 2023-11-28T18:23:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.351+0000 7fd48e2d6640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4880ce9e0 con 0x7fd48815d3c0 2023-11-28T18:23:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.351+0000 7fd487fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd48815d3c0 0x7fd488071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.351+0000 7fd487fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd48815d3c0 0x7fd488071a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55412/0 (socket says 172.21.15.105:55412) 2023-11-28T18:23:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.351+0000 7fd487fff640 1 -- 172.21.15.105:0/4247620743 learned_addr learned my addr 172.21.15.105:0/4247620743 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.351+0000 7fd487fff640 1 -- 172.21.15.105:0/4247620743 >> v1:172.21.15.148:6789/0 conn(0x7fd488070a10 legacy=0x7fd488070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.351+0000 7fd487fff640 1 -- 172.21.15.105:0/4247620743 >> v1:172.21.15.105:6789/0 conn(0x7fd488161870 legacy=0x7fd4880ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.351+0000 7fd487fff640 1 -- 172.21.15.105:0/4247620743 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4880cecc0 con 0x7fd48815d3c0 2023-11-28T18:23:58.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.355+0000 7fd487fff640 1 --2- 172.21.15.105:0/4247620743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd48815d3c0 0x7fd488071a10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd478009870 tx=0x7fd47801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a564b02ed5456fc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:58.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.355+0000 7fd486ffd640 1 -- 172.21.15.105:0/4247620743 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4780aa670 con 0x7fd48815d3c0 2023-11-28T18:23:58.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.355+0000 7fd486ffd640 1 -- 172.21.15.105:0/4247620743 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4780aa7d0 con 0x7fd48815d3c0 2023-11-28T18:23:58.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.355+0000 7fd486ffd640 1 -- 172.21.15.105:0/4247620743 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4780aab60 con 0x7fd48815d3c0 2023-11-28T18:23:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.355+0000 7fd48e2d6640 1 -- 172.21.15.105:0/4247620743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd48815d3c0 msgr2=0x7fd488071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.355+0000 7fd48e2d6640 1 --2- 172.21.15.105:0/4247620743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd48815d3c0 0x7fd488071a10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd478009870 tx=0x7fd47801c510 comp rx=0 tx=0).stop 2023-11-28T18:23:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.355+0000 7fd48e2d6640 1 -- 172.21.15.105:0/4247620743 shutdown_connections 2023-11-28T18:23:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.355+0000 7fd48e2d6640 1 --2- 172.21.15.105:0/4247620743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd48815d3c0 0x7fd488071a10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.355+0000 7fd48e2d6640 1 -- 172.21.15.105:0/4247620743 >> 172.21.15.105:0/4247620743 conn(0x7fd4880cf540 msgr2=0x7fd4880cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.355+0000 7fd48e2d6640 1 -- 172.21.15.105:0/4247620743 shutdown_connections 2023-11-28T18:23:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.355+0000 7fd48e2d6640 1 -- 172.21.15.105:0/4247620743 wait complete. 2023-11-28T18:23:58.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd48e2d6640 1 Processor -- start 2023-11-28T18:23:58.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd48e2d6640 1 -- start start 2023-11-28T18:23:58.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd48e2d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd488070a10 0x7fd4881593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:58.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd48e2d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48815d3c0 0x7fd4881525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:58.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd48e2d6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd488161870 0x7fd488154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:58.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd48e2d6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4881546d0 con 0x7fd488161870 2023-11-28T18:23:58.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd48e2d6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd488159fb0 con 0x7fd48815d3c0 2023-11-28T18:23:58.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd48e2d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd488159cd0 con 0x7fd488070a10 2023-11-28T18:23:58.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd48c873640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd488161870 0x7fd488154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:58.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd48c873640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd488161870 0x7fd488154af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55414/0 (socket says 172.21.15.105:55414) 2023-11-28T18:23:58.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd48c873640 1 -- 172.21.15.105:0/3790658438 learned_addr learned my addr 172.21.15.105:0/3790658438 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:58.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd487fff640 1 --2- 172.21.15.105:0/3790658438 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd488070a10 0x7fd4881593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:58.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd4877fe640 1 --2- 172.21.15.105:0/3790658438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48815d3c0 0x7fd4881525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:58.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd48c873640 1 -- 172.21.15.105:0/3790658438 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd488070a10 msgr2=0x7fd4881593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:58.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd48c873640 1 --2- 172.21.15.105:0/3790658438 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd488070a10 0x7fd4881593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd48c873640 1 -- 172.21.15.105:0/3790658438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48815d3c0 msgr2=0x7fd4881525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:58.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd48c873640 1 --2- 172.21.15.105:0/3790658438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48815d3c0 0x7fd4881525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd48c873640 1 -- 172.21.15.105:0/3790658438 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4780094d0 con 0x7fd488161870 2023-11-28T18:23:58.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd48c873640 1 --2- 172.21.15.105:0/3790658438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd488161870 0x7fd488154af0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd48000c730 tx=0x7fd48000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:58.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd484ff9640 1 -- 172.21.15.105:0/3790658438 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480003b80 con 0x7fd488161870 2023-11-28T18:23:58.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd484ff9640 1 -- 172.21.15.105:0/3790658438 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd480003ce0 con 0x7fd488161870 2023-11-28T18:23:58.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd484ff9640 1 -- 172.21.15.105:0/3790658438 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48000aa30 con 0x7fd488161870 2023-11-28T18:23:58.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd48e2d6640 1 -- 172.21.15.105:0/3790658438 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd488154010 con 0x7fd488161870 2023-11-28T18:23:58.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.359+0000 7fd48e2d6640 1 -- 172.21.15.105:0/3790658438 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd488159910 con 0x7fd488161870 2023-11-28T18:23:58.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.363+0000 7fd48e2d6640 1 -- 172.21.15.105:0/3790658438 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd488154810 con 0x7fd488161870 2023-11-28T18:23:58.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.363+0000 7fd484ff9640 1 -- 172.21.15.105:0/3790658438 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd48000ab90 con 0x7fd488161870 2023-11-28T18:23:58.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.363+0000 7fd484ff9640 1 --2- 172.21.15.105:0/3790658438 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd4740668d0 0x7fd474068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:58.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.363+0000 7fd484ff9640 1 -- 172.21.15.105:0/3790658438 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd480084300 con 0x7fd488161870 2023-11-28T18:23:58.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.363+0000 7fd487fff640 1 --2- 172.21.15.105:0/3790658438 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd4740668d0 0x7fd474068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:58.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.363+0000 7fd487fff640 1 --2- 172.21.15.105:0/3790658438 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd4740668d0 0x7fd474068d90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd4780ae000 tx=0x7fd4780b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:58.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.367+0000 7fd484ff9640 1 -- 172.21.15.105:0/3790658438 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd488154810 con 0x7fd488161870 2023-11-28T18:23:58.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.515+0000 7fd48e2d6640 1 -- 172.21.15.105:0/3790658438 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd488154810 con 0x7fd488161870 2023-11-28T18:23:58.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.515+0000 7fd484ff9640 1 -- 172.21.15.105:0/3790658438 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd488154810 con 0x7fd488161870 2023-11-28T18:23:58.521 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:23:58.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.519+0000 7fd48e2d6640 1 -- 172.21.15.105:0/3790658438 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd4740668d0 msgr2=0x7fd474068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:58.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.519+0000 7fd48e2d6640 1 --2- 172.21.15.105:0/3790658438 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd4740668d0 0x7fd474068d90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd4780ae000 tx=0x7fd4780b5040 comp rx=0 tx=0).stop 2023-11-28T18:23:58.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.519+0000 7fd48e2d6640 1 -- 172.21.15.105:0/3790658438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd488161870 msgr2=0x7fd488154af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:58.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.519+0000 7fd48e2d6640 1 --2- 172.21.15.105:0/3790658438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd488161870 0x7fd488154af0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd48000c730 tx=0x7fd48000cbf0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.519+0000 7fd48e2d6640 1 -- 172.21.15.105:0/3790658438 shutdown_connections 2023-11-28T18:23:58.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.519+0000 7fd48e2d6640 1 --2- 172.21.15.105:0/3790658438 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd4740668d0 0x7fd474068d90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.519+0000 7fd48e2d6640 1 --2- 172.21.15.105:0/3790658438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd488161870 0x7fd488154af0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.519+0000 7fd48e2d6640 1 --2- 172.21.15.105:0/3790658438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48815d3c0 0x7fd4881525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.519+0000 7fd48e2d6640 1 --2- 172.21.15.105:0/3790658438 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd488070a10 0x7fd4881593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.519+0000 7fd48e2d6640 1 -- 172.21.15.105:0/3790658438 >> 172.21.15.105:0/3790658438 conn(0x7fd4880cf540 msgr2=0x7fd4880b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:58.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.519+0000 7fd48e2d6640 1 -- 172.21.15.105:0/3790658438 shutdown_connections 2023-11-28T18:23:58.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.519+0000 7fd48e2d6640 1 -- 172.21.15.105:0/3790658438 wait complete. 2023-11-28T18:23:58.541 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:23:58.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.679+0000 7f6b2d101640 1 Processor -- start 2023-11-28T18:23:58.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.679+0000 7f6b2d101640 1 -- start start 2023-11-28T18:23:58.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.679+0000 7f6b2d101640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b280af340 0x7f6b280721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:58.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.679+0000 7f6b2d101640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6b2806ed00 0x7f6b280d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.679+0000 7f6b2d101640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2806f160 con 0x7f6b280d08b0 2023-11-28T18:23:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.679+0000 7f6b2d101640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b280b34a0 con 0x7f6b2806ed00 2023-11-28T18:23:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.679+0000 7f6b2d101640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b280b2ee0 con 0x7f6b280af340 2023-11-28T18:23:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.679+0000 7f6b2759e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f6b280d08b0 0x7f6b2806a580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39370/0 (socket says 172.21.15.105:39370) 2023-11-28T18:23:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.679+0000 7f6b2759e640 1 -- 172.21.15.105:0/2524329883 learned_addr learned my addr 172.21.15.105:0/2524329883 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.679+0000 7f6b26d9d640 1 --2- 172.21.15.105:0/2524329883 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b280af340 0x7f6b280721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.679+0000 7f6b1e59c640 1 --2- 172.21.15.105:0/2524329883 >> v2:172.21.15.148:3300/0 conn(0x7f6b2806ed00 0x7f6b280d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.679+0000 7f6b2659c640 1 -- 172.21.15.105:0/2524329883 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181965494 0 0) 0x7f6b2806f160 con 0x7f6b280d08b0 2023-11-28T18:23:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.679+0000 7f6b2659c640 1 -- 172.21.15.105:0/2524329883 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b14003530 con 0x7f6b280d08b0 2023-11-28T18:23:58.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.679+0000 7f6b1e59c640 1 -- 172.21.15.105:0/2524329883 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b280af340 msgr2=0x7f6b280721d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:58.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.679+0000 7f6b1e59c640 1 --2- 172.21.15.105:0/2524329883 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b280af340 0x7f6b280721d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.679+0000 7f6b1e59c640 1 -- 172.21.15.105:0/2524329883 >> v1:172.21.15.105:6789/0 conn(0x7f6b280d08b0 legacy=0x7f6b2806a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:58.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.679+0000 7f6b1e59c640 1 -- 172.21.15.105:0/2524329883 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b280b31c0 con 0x7f6b2806ed00 2023-11-28T18:23:58.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.679+0000 7f6b26d9d640 1 --2- 172.21.15.105:0/2524329883 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b280af340 0x7f6b280721d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:58.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.683+0000 7f6b1e59c640 1 --2- 172.21.15.105:0/2524329883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b2806ed00 0x7f6b280d0370 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009d90 tx=0x7f6b1001c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ec6ad8d703ff166 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:58.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.683+0000 7f6b2659c640 1 -- 172.21.15.105:0/2524329883 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b100a59e0 con 0x7f6b2806ed00 2023-11-28T18:23:58.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.683+0000 7f6b2659c640 1 -- 172.21.15.105:0/2524329883 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b100a5b40 con 0x7f6b2806ed00 2023-11-28T18:23:58.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.683+0000 7f6b2659c640 1 -- 172.21.15.105:0/2524329883 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b100ad440 con 0x7f6b2806ed00 2023-11-28T18:23:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.683+0000 7f6b2d101640 1 -- 172.21.15.105:0/2524329883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b2806ed00 msgr2=0x7f6b280d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.683+0000 7f6b2d101640 1 --2- 172.21.15.105:0/2524329883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b2806ed00 0x7f6b280d0370 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009d90 tx=0x7f6b1001c920 comp rx=0 tx=0).stop 2023-11-28T18:23:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.683+0000 7f6b2d101640 1 -- 172.21.15.105:0/2524329883 shutdown_connections 2023-11-28T18:23:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.683+0000 7f6b2d101640 1 --2- 172.21.15.105:0/2524329883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b2806ed00 0x7f6b280d0370 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.683+0000 7f6b2d101640 1 --2- 172.21.15.105:0/2524329883 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b280af340 0x7f6b280721d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.683+0000 7f6b2d101640 1 -- 172.21.15.105:0/2524329883 >> 172.21.15.105:0/2524329883 conn(0x7f6b2806b860 msgr2=0x7f6b2806bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.683+0000 7f6b2d101640 1 -- 172.21.15.105:0/2524329883 shutdown_connections 2023-11-28T18:23:58.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.683+0000 7f6b2d101640 1 -- 172.21.15.105:0/2524329883 wait complete. 2023-11-28T18:23:58.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b2d101640 1 Processor -- start 2023-11-28T18:23:58.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b2d101640 1 -- start start 2023-11-28T18:23:58.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b2d101640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b2806ed00 0x7f6b2815cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b2d101640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b280af340 0x7f6b280ad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b2d101640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b280d08b0 0x7f6b280af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b2d101640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b280aeac0 con 0x7f6b280af340 2023-11-28T18:23:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b2d101640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2815d070 con 0x7f6b2806ed00 2023-11-28T18:23:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b2d101640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b280ae220 con 0x7f6b280d08b0 2023-11-28T18:23:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b1e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b280af340 0x7f6b280ad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b1e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b280af340 0x7f6b280ad7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55426/0 (socket says 172.21.15.105:55426) 2023-11-28T18:23:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b1e59c640 1 -- 172.21.15.105:0/240786102 learned_addr learned my addr 172.21.15.105:0/240786102 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b26d9d640 1 --2- 172.21.15.105:0/240786102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b2806ed00 0x7f6b2815cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b2759e640 1 --2- 172.21.15.105:0/240786102 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b280d08b0 0x7f6b280af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b1e59c640 1 -- 172.21.15.105:0/240786102 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b280d08b0 msgr2=0x7f6b280af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:58.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b1e59c640 1 --2- 172.21.15.105:0/240786102 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b280d08b0 0x7f6b280af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b1e59c640 1 -- 172.21.15.105:0/240786102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b2806ed00 msgr2=0x7f6b2815cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:58.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b1e59c640 1 --2- 172.21.15.105:0/240786102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b2806ed00 0x7f6b2815cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b1e59c640 1 -- 172.21.15.105:0/240786102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b18009420 con 0x7f6b280af340 2023-11-28T18:23:58.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b26d9d640 1 --2- 172.21.15.105:0/240786102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b2806ed00 0x7f6b2815cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:23:58.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b1e59c640 1 --2- 172.21.15.105:0/240786102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b280af340 0x7f6b280ad7a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6b1001a550 tx=0x7f6b100196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:58.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b1ffff640 1 -- 172.21.15.105:0/240786102 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b100add80 con 0x7f6b280af340 2023-11-28T18:23:58.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b1ffff640 1 -- 172.21.15.105:0/240786102 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b10009450 con 0x7f6b280af340 2023-11-28T18:23:58.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b1ffff640 1 -- 172.21.15.105:0/240786102 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b100b9690 con 0x7f6b280af340 2023-11-28T18:23:58.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b2d101640 1 -- 172.21.15.105:0/240786102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b10009a70 con 0x7f6b280af340 2023-11-28T18:23:58.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.687+0000 7f6b2d101640 1 -- 172.21.15.105:0/240786102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b280ae7e0 con 0x7f6b280af340 2023-11-28T18:23:58.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.691+0000 7f6b2d101640 1 -- 172.21.15.105:0/240786102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b28067570 con 0x7f6b280af340 2023-11-28T18:23:58.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.691+0000 7f6b1ffff640 1 -- 172.21.15.105:0/240786102 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f6b100b97f0 con 0x7f6b280af340 2023-11-28T18:23:58.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.691+0000 7f6b1ffff640 1 --2- 172.21.15.105:0/240786102 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b08066900 0x7f6b08068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:58.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.691+0000 7f6b1ffff640 1 -- 172.21.15.105:0/240786102 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6b1011fa90 con 0x7f6b280af340 2023-11-28T18:23:58.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.691+0000 7f6b26d9d640 1 --2- 172.21.15.105:0/240786102 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b08066900 0x7f6b08068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:58.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.691+0000 7f6b26d9d640 1 --2- 172.21.15.105:0/240786102 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b08066900 0x7f6b08068dc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009d40 tx=0x7f6b18008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:58.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.695+0000 7f6b1ffff640 1 -- 172.21.15.105:0/240786102 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6b100ec940 con 0x7f6b280af340 2023-11-28T18:23:58.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.847+0000 7f6b2d101640 1 -- 172.21.15.105:0/240786102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6b28073e00 con 0x7f6b280af340 2023-11-28T18:23:58.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.847+0000 7f6b1ffff640 1 -- 172.21.15.105:0/240786102 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6b100eff60 con 0x7f6b280af340 2023-11-28T18:23:58.850 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:23:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.847+0000 7f6b2d101640 1 -- 172.21.15.105:0/240786102 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b08066900 msgr2=0x7f6b08068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.847+0000 7f6b2d101640 1 --2- 172.21.15.105:0/240786102 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b08066900 0x7f6b08068dc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009d40 tx=0x7f6b18008040 comp rx=0 tx=0).stop 2023-11-28T18:23:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.847+0000 7f6b2d101640 1 -- 172.21.15.105:0/240786102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b280af340 msgr2=0x7f6b280ad7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.847+0000 7f6b2d101640 1 --2- 172.21.15.105:0/240786102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b280af340 0x7f6b280ad7a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6b1001a550 tx=0x7f6b100196c0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.847+0000 7f6b2d101640 1 -- 172.21.15.105:0/240786102 shutdown_connections 2023-11-28T18:23:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.847+0000 7f6b2d101640 1 --2- 172.21.15.105:0/240786102 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b280d08b0 0x7f6b280af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.847+0000 7f6b2d101640 1 --2- 172.21.15.105:0/240786102 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b08066900 0x7f6b08068dc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.847+0000 7f6b2d101640 1 --2- 172.21.15.105:0/240786102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b280af340 0x7f6b280ad7a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.847+0000 7f6b2d101640 1 --2- 172.21.15.105:0/240786102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b2806ed00 0x7f6b2815cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.847+0000 7f6b2d101640 1 -- 172.21.15.105:0/240786102 >> 172.21.15.105:0/240786102 conn(0x7f6b2806b860 msgr2=0x7f6b280b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.847+0000 7f6b2d101640 1 -- 172.21.15.105:0/240786102 shutdown_connections 2023-11-28T18:23:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:58.847+0000 7f6b2d101640 1 -- 172.21.15.105:0/240786102 wait complete. 2023-11-28T18:23:58.871 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T18:23:59.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.007+0000 7f094130f640 1 Processor -- start 2023-11-28T18:23:59.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.007+0000 7f094130f640 1 -- start start 2023-11-28T18:23:59.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.011+0000 7f094130f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f093c143470 0x7f093c0b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:59.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.011+0000 7f094130f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f093c0cf350 0x7f093c06c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:59.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.011+0000 7f094130f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093c0b1f60 con 0x7f093c06c050 2023-11-28T18:23:59.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.011+0000 7f094130f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093c070300 con 0x7f093c143470 2023-11-28T18:23:59.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.011+0000 7f094130f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093c0705a0 con 0x7f093c0cf350 2023-11-28T18:23:59.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.011+0000 7f093affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f093c143470 0x7f093c0b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:59.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.011+0000 7f093affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f093c143470 0x7f093c0b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55440/0 (socket says 172.21.15.105:55440) 2023-11-28T18:23:59.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.011+0000 7f093affd640 1 -- 172.21.15.105:0/1283313471 learned_addr learned my addr 172.21.15.105:0/1283313471 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:59.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.011+0000 7f093b7fe640 1 --2- 172.21.15.105:0/1283313471 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f093c0cf350 0x7f093c06c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:59.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.011+0000 7f093affd640 1 -- 172.21.15.105:0/1283313471 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f093c0cf350 msgr2=0x7f093c06c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:59.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.011+0000 7f093affd640 1 --2- 172.21.15.105:0/1283313471 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f093c0cf350 0x7f093c06c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.011+0000 7f093affd640 1 -- 172.21.15.105:0/1283313471 >> v1:172.21.15.148:6789/0 conn(0x7f093c06c050 legacy=0x7f093c0cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:59.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.011+0000 7f093affd640 1 -- 172.21.15.105:0/1283313471 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093c070880 con 0x7f093c143470 2023-11-28T18:23:59.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.015+0000 7f093affd640 1 --2- 172.21.15.105:0/1283313471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f093c143470 0x7f093c0b1a20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f092c009870 tx=0x7f092c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608eaf796d0b4f43 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:59.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.015+0000 7f0939ffb640 1 -- 172.21.15.105:0/1283313471 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c0aa610 con 0x7f093c143470 2023-11-28T18:23:59.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.015+0000 7f0939ffb640 1 -- 172.21.15.105:0/1283313471 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092c0a6040 con 0x7f093c143470 2023-11-28T18:23:59.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.015+0000 7f0939ffb640 1 -- 172.21.15.105:0/1283313471 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c0aa990 con 0x7f093c143470 2023-11-28T18:23:59.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.015+0000 7f094130f640 1 -- 172.21.15.105:0/1283313471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f093c143470 msgr2=0x7f093c0b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:59.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.015+0000 7f094130f640 1 --2- 172.21.15.105:0/1283313471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f093c143470 0x7f093c0b1a20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f092c009870 tx=0x7f092c01c4d0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.015+0000 7f094130f640 1 -- 172.21.15.105:0/1283313471 shutdown_connections 2023-11-28T18:23:59.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.015+0000 7f094130f640 1 --2- 172.21.15.105:0/1283313471 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f093c0cf350 0x7f093c06c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.015+0000 7f094130f640 1 --2- 172.21.15.105:0/1283313471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f093c143470 0x7f093c0b1a20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.015+0000 7f094130f640 1 -- 172.21.15.105:0/1283313471 >> 172.21.15.105:0/1283313471 conn(0x7f093c06a750 msgr2=0x7f093c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:59.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.015+0000 7f094130f640 1 -- 172.21.15.105:0/1283313471 shutdown_connections 2023-11-28T18:23:59.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.015+0000 7f094130f640 1 -- 172.21.15.105:0/1283313471 wait complete. 2023-11-28T18:23:59.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.015+0000 7f094130f640 1 Processor -- start 2023-11-28T18:23:59.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.015+0000 7f094130f640 1 -- start start 2023-11-28T18:23:59.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.015+0000 7f094130f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f093c06c050 0x7f093c14d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:59.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.015+0000 7f094130f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f093c0cf350 0x7f093c142150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:59.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.015+0000 7f094130f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f093c143470 0x7f093c13f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:59.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.019+0000 7f094130f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c1416f0 con 0x7f093c0cf350 2023-11-28T18:23:59.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.019+0000 7f094130f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c14d950 con 0x7f093c06c050 2023-11-28T18:23:59.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.019+0000 7f094130f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093c140a70 con 0x7f093c143470 2023-11-28T18:23:59.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.019+0000 7f093a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f093c0cf350 0x7f093c142150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:59.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.019+0000 7f093affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f093c06c050 0x7f093c14d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:59.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.019+0000 7f093a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f093c0cf350 0x7f093c142150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55456/0 (socket says 172.21.15.105:55456) 2023-11-28T18:23:59.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.019+0000 7f093a7fc640 1 -- 172.21.15.105:0/1952733174 learned_addr learned my addr 172.21.15.105:0/1952733174 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:59.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.019+0000 7f093b7fe640 1 --2- 172.21.15.105:0/1952733174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f093c143470 0x7f093c13f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:59.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.019+0000 7f093a7fc640 1 -- 172.21.15.105:0/1952733174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f093c143470 msgr2=0x7f093c13f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:59.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.019+0000 7f093a7fc640 1 --2- 172.21.15.105:0/1952733174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f093c143470 0x7f093c13f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.019+0000 7f093a7fc640 1 -- 172.21.15.105:0/1952733174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f093c06c050 msgr2=0x7f093c14d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:59.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.019+0000 7f093a7fc640 1 --2- 172.21.15.105:0/1952733174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f093c06c050 0x7f093c14d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.019+0000 7f093a7fc640 1 -- 172.21.15.105:0/1952733174 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c0094d0 con 0x7f093c0cf350 2023-11-28T18:23:59.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.019+0000 7f093a7fc640 1 --2- 172.21.15.105:0/1952733174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f093c0cf350 0x7f093c142150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f093000c480 tx=0x7f093000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:59.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.019+0000 7f09237fe640 1 -- 172.21.15.105:0/1952733174 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930012830 con 0x7f093c0cf350 2023-11-28T18:23:59.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.019+0000 7f09237fe640 1 -- 172.21.15.105:0/1952733174 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09300129b0 con 0x7f093c0cf350 2023-11-28T18:23:59.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.019+0000 7f094130f640 1 -- 172.21.15.105:0/1952733174 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093c140d50 con 0x7f093c0cf350 2023-11-28T18:23:59.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.019+0000 7f09237fe640 1 -- 172.21.15.105:0/1952733174 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930012d60 con 0x7f093c0cf350 2023-11-28T18:23:59.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.019+0000 7f094130f640 1 -- 172.21.15.105:0/1952733174 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093c141310 con 0x7f093c0cf350 2023-11-28T18:23:59.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.019+0000 7f094130f640 1 -- 172.21.15.105:0/1952733174 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f093c074580 con 0x7f093c0cf350 2023-11-28T18:23:59.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.019+0000 7f09237fe640 1 -- 172.21.15.105:0/1952733174 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f093001c460 con 0x7f093c0cf350 2023-11-28T18:23:59.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.023+0000 7f09237fe640 1 --2- 172.21.15.105:0/1952733174 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f09280668d0 0x7f0928068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:59.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.023+0000 7f09237fe640 1 -- 172.21.15.105:0/1952733174 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f09300843f0 con 0x7f093c0cf350 2023-11-28T18:23:59.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.023+0000 7f093affd640 1 --2- 172.21.15.105:0/1952733174 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f09280668d0 0x7f0928068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:59.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.023+0000 7f093affd640 1 --2- 172.21.15.105:0/1952733174 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f09280668d0 0x7f0928068d90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f092c0ae000 tx=0x7f092c0b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:59.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.027+0000 7f09237fe640 1 -- 172.21.15.105:0/1952733174 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0930051480 con 0x7f093c0cf350 2023-11-28T18:23:59.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.179+0000 7f094130f640 1 -- 172.21.15.105:0/1952733174 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f093c0cb270 con 0x7f093c0cf350 2023-11-28T18:23:59.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.179+0000 7f09237fe640 1 -- 172.21.15.105:0/1952733174 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0930054aa0 con 0x7f093c0cf350 2023-11-28T18:23:59.182 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:23:59.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.179+0000 7f094130f640 1 -- 172.21.15.105:0/1952733174 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f09280668d0 msgr2=0x7f0928068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:59.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.179+0000 7f094130f640 1 --2- 172.21.15.105:0/1952733174 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f09280668d0 0x7f0928068d90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f092c0ae000 tx=0x7f092c0b5040 comp rx=0 tx=0).stop 2023-11-28T18:23:59.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.179+0000 7f094130f640 1 -- 172.21.15.105:0/1952733174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f093c0cf350 msgr2=0x7f093c142150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:59.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.179+0000 7f094130f640 1 --2- 172.21.15.105:0/1952733174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f093c0cf350 0x7f093c142150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f093000c480 tx=0x7f093000c940 comp rx=0 tx=0).stop 2023-11-28T18:23:59.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.179+0000 7f094130f640 1 -- 172.21.15.105:0/1952733174 shutdown_connections 2023-11-28T18:23:59.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.179+0000 7f094130f640 1 --2- 172.21.15.105:0/1952733174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f093c143470 0x7f093c13f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.179+0000 7f094130f640 1 --2- 172.21.15.105:0/1952733174 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f09280668d0 0x7f0928068d90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.179+0000 7f094130f640 1 --2- 172.21.15.105:0/1952733174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f093c0cf350 0x7f093c142150 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.179+0000 7f094130f640 1 --2- 172.21.15.105:0/1952733174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f093c06c050 0x7f093c14d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.179+0000 7f094130f640 1 -- 172.21.15.105:0/1952733174 >> 172.21.15.105:0/1952733174 conn(0x7f093c06a750 msgr2=0x7f093c0b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:59.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.179+0000 7f094130f640 1 -- 172.21.15.105:0/1952733174 shutdown_connections 2023-11-28T18:23:59.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.179+0000 7f094130f640 1 -- 172.21.15.105:0/1952733174 wait complete. 2023-11-28T18:23:59.203 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:23:59.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.339+0000 7fa23ab72640 1 Processor -- start 2023-11-28T18:23:59.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.339+0000 7fa23ab72640 1 -- start start 2023-11-28T18:23:59.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.339+0000 7fa23ab72640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa2340cefd0 0x7fa234168250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:59.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.339+0000 7fa23ab72640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2340c4a00 con 0x7fa2340c7490 2023-11-28T18:23:59.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.339+0000 7fa23ab72640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2340d0400 con 0x7fa23406c790 2023-11-28T18:23:59.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.339+0000 7fa23ab72640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2340cf440 con 0x7fa2340cefd0 2023-11-28T18:23:59.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.339+0000 7fa23890e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa2340c7490 0x7fa23406b690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39374/0 (socket says 172.21.15.105:39374) 2023-11-28T18:23:59.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.339+0000 7fa23890e640 1 -- 172.21.15.105:0/2431800957 learned_addr learned my addr 172.21.15.105:0/2431800957 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:59.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.339+0000 7fa23910f640 1 --2- 172.21.15.105:0/2431800957 >> v2:172.21.15.148:3300/0 conn(0x7fa2340cefd0 0x7fa234168250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:59.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.339+0000 7fa22b7fe640 1 -- 172.21.15.105:0/2431800957 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2530675655 0 0) 0x7fa2340c4a00 con 0x7fa2340c7490 2023-11-28T18:23:59.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.339+0000 7fa22b7fe640 1 -- 172.21.15.105:0/2431800957 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa21c003530 con 0x7fa2340c7490 2023-11-28T18:23:59.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.339+0000 7fa23910f640 1 -- 172.21.15.105:0/2431800957 >> v1:172.21.15.148:6789/0 conn(0x7fa23406c790 legacy=0x7fa23406bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:59.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.339+0000 7fa23910f640 1 -- 172.21.15.105:0/2431800957 >> v1:172.21.15.105:6789/0 conn(0x7fa2340c7490 legacy=0x7fa23406b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:59.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.339+0000 7fa23910f640 1 -- 172.21.15.105:0/2431800957 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2340c7870 con 0x7fa2340cefd0 2023-11-28T18:23:59.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.343+0000 7fa23910f640 1 --2- 172.21.15.105:0/2431800957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2340cefd0 0x7fa234168250 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009d90 tx=0x7fa22c01c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=870b59bb5834f71e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:59.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.343+0000 7fa22b7fe640 1 -- 172.21.15.105:0/2431800957 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c0a5900 con 0x7fa2340cefd0 2023-11-28T18:23:59.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.343+0000 7fa22b7fe640 1 -- 172.21.15.105:0/2431800957 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22c0a5a60 con 0x7fa2340cefd0 2023-11-28T18:23:59.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.343+0000 7fa22b7fe640 1 -- 172.21.15.105:0/2431800957 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c0a5df0 con 0x7fa2340cefd0 2023-11-28T18:23:59.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.343+0000 7fa23ab72640 1 -- 172.21.15.105:0/2431800957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2340cefd0 msgr2=0x7fa234168250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:59.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.343+0000 7fa23ab72640 1 --2- 172.21.15.105:0/2431800957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2340cefd0 0x7fa234168250 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009d90 tx=0x7fa22c01c950 comp rx=0 tx=0).stop 2023-11-28T18:23:59.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.343+0000 7fa23ab72640 1 -- 172.21.15.105:0/2431800957 shutdown_connections 2023-11-28T18:23:59.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.343+0000 7fa23ab72640 1 --2- 172.21.15.105:0/2431800957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2340cefd0 0x7fa234168250 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.343+0000 7fa23ab72640 1 -- 172.21.15.105:0/2431800957 >> 172.21.15.105:0/2431800957 conn(0x7fa2340d0eb0 msgr2=0x7fa2340d12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:59.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.343+0000 7fa23ab72640 1 -- 172.21.15.105:0/2431800957 shutdown_connections 2023-11-28T18:23:59.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.343+0000 7fa23ab72640 1 -- 172.21.15.105:0/2431800957 wait complete. 2023-11-28T18:23:59.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa23ab72640 1 Processor -- start 2023-11-28T18:23:59.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa23ab72640 1 -- start start 2023-11-28T18:23:59.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa23ab72640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa23406c790 0x7fa234154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:59.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa23ab72640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2340c7490 0x7fa2340b1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:59.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa23ab72640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2340cefd0 0x7fa234152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:59.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa23ab72640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa234152760 con 0x7fa2340c7490 2023-11-28T18:23:59.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa23ab72640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2340b0230 con 0x7fa23406c790 2023-11-28T18:23:59.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa23ab72640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2340caf20 con 0x7fa2340cefd0 2023-11-28T18:23:59.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa22bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2340c7490 0x7fa2340b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:59.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa22bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2340c7490 0x7fa2340b1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55462/0 (socket says 172.21.15.105:55462) 2023-11-28T18:23:59.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa22bfff640 1 -- 172.21.15.105:0/251179357 learned_addr learned my addr 172.21.15.105:0/251179357 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:59.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa23890e640 1 --2- 172.21.15.105:0/251179357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa23406c790 0x7fa234154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:59.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa23910f640 1 --2- 172.21.15.105:0/251179357 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2340cefd0 0x7fa234152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:59.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa22bfff640 1 -- 172.21.15.105:0/251179357 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2340cefd0 msgr2=0x7fa234152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:59.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa22bfff640 1 --2- 172.21.15.105:0/251179357 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2340cefd0 0x7fa234152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa22bfff640 1 -- 172.21.15.105:0/251179357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa23406c790 msgr2=0x7fa234154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:59.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa22bfff640 1 --2- 172.21.15.105:0/251179357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa23406c790 0x7fa234154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa22bfff640 1 -- 172.21.15.105:0/251179357 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22c009a70 con 0x7fa2340c7490 2023-11-28T18:23:59.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa22bfff640 1 --2- 172.21.15.105:0/251179357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2340c7490 0x7fa2340b1730 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa22400c530 tx=0x7fa22400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:59.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa2297fa640 1 -- 172.21.15.105:0/251179357 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224005220 con 0x7fa2340c7490 2023-11-28T18:23:59.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa2297fa640 1 -- 172.21.15.105:0/251179357 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa224005380 con 0x7fa2340c7490 2023-11-28T18:23:59.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa23ab72640 1 -- 172.21.15.105:0/251179357 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2340ca8c0 con 0x7fa2340c7490 2023-11-28T18:23:59.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa2297fa640 1 -- 172.21.15.105:0/251179357 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224007270 con 0x7fa2340c7490 2023-11-28T18:23:59.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.347+0000 7fa23ab72640 1 -- 172.21.15.105:0/251179357 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2340ae8f0 con 0x7fa2340c7490 2023-11-28T18:23:59.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.351+0000 7fa23ab72640 1 -- 172.21.15.105:0/251179357 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa234067570 con 0x7fa2340c7490 2023-11-28T18:23:59.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.351+0000 7fa2297fa640 1 -- 172.21.15.105:0/251179357 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa2240073d0 con 0x7fa2340c7490 2023-11-28T18:23:59.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.351+0000 7fa2297fa640 1 --2- 172.21.15.105:0/251179357 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa2200668d0 0x7fa220068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:59.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.351+0000 7fa2297fa640 1 -- 172.21.15.105:0/251179357 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa224083ed0 con 0x7fa2340c7490 2023-11-28T18:23:59.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.351+0000 7fa23890e640 1 --2- 172.21.15.105:0/251179357 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa2200668d0 0x7fa220068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:59.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.351+0000 7fa23890e640 1 --2- 172.21.15.105:0/251179357 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa2200668d0 0x7fa220068d90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa2300011f0 tx=0x7fa230009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:59.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.355+0000 7fa2297fa640 1 -- 172.21.15.105:0/251179357 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa224050ec0 con 0x7fa2340c7490 2023-11-28T18:23:59.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.539+0000 7fa23ab72640 1 -- 172.21.15.105:0/251179357 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2340d0400 con 0x7fa2340c7490 2023-11-28T18:23:59.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.539+0000 7fa2297fa640 1 -- 172.21.15.105:0/251179357 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7fa2340d0400 con 0x7fa2340c7490 2023-11-28T18:23:59.542 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:59.542 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/390529893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":390529893},{"type":"v1","addr":"172.21.15.148:6835","nonce":390529893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3924571887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3924571887},{"type":"v1","addr":"172.21.15.148:6833","nonce":3924571887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3271971772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3271971772},{"type":"v1","addr":"172.21.15.105:6835","nonce":3271971772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:23:33.288743+0000","modified":"2023-11-28T18:23:50.425976+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2504263238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2504263238},{"type":"v1","addr":"172.21.15.105:6837","nonce":2504263238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T18:23:59.542 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 9 2023-11-28T18:23:59.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.539+0000 7fa23ab72640 1 -- 172.21.15.105:0/251179357 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa2200668d0 msgr2=0x7fa220068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:59.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.539+0000 7fa23ab72640 1 --2- 172.21.15.105:0/251179357 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa2200668d0 0x7fa220068d90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa2300011f0 tx=0x7fa230009040 comp rx=0 tx=0).stop 2023-11-28T18:23:59.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.539+0000 7fa23ab72640 1 -- 172.21.15.105:0/251179357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2340c7490 msgr2=0x7fa2340b1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:59.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.539+0000 7fa23ab72640 1 --2- 172.21.15.105:0/251179357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2340c7490 0x7fa2340b1730 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa22400c530 tx=0x7fa22400c9f0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.539+0000 7fa23ab72640 1 -- 172.21.15.105:0/251179357 shutdown_connections 2023-11-28T18:23:59.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.539+0000 7fa23ab72640 1 --2- 172.21.15.105:0/251179357 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2340cefd0 0x7fa234152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.539+0000 7fa23ab72640 1 --2- 172.21.15.105:0/251179357 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa2200668d0 0x7fa220068d90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.539+0000 7fa23ab72640 1 --2- 172.21.15.105:0/251179357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2340c7490 0x7fa2340b1730 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.539+0000 7fa23ab72640 1 --2- 172.21.15.105:0/251179357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa23406c790 0x7fa234154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.539+0000 7fa23ab72640 1 -- 172.21.15.105:0/251179357 >> 172.21.15.105:0/251179357 conn(0x7fa2340d0eb0 msgr2=0x7fa2340d1390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:59.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.539+0000 7fa23ab72640 1 -- 172.21.15.105:0/251179357 shutdown_connections 2023-11-28T18:23:59.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.539+0000 7fa23ab72640 1 -- 172.21.15.105:0/251179357 wait complete. 2023-11-28T18:23:59.563 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:23:59.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.695+0000 7f040bc6b640 1 Processor -- start 2023-11-28T18:23:59.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.695+0000 7f040bc6b640 1 -- start start 2023-11-28T18:23:59.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f040bc6b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f04040caf60 0x7f04040cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:59.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f040bc6b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f04040ad850 0x7f04040701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:59.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f040bc6b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f040406bf50 con 0x7f0404159f40 2023-11-28T18:23:59.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f040bc6b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04040cb3c0 con 0x7f04040ad850 2023-11-28T18:23:59.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f040bc6b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f040415a320 con 0x7f04040caf60 2023-11-28T18:23:59.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0409a07640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0404159f40 0x7f04040ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39386/0 (socket says 172.21.15.105:39386) 2023-11-28T18:23:59.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0409a07640 1 -- 172.21.15.105:0/441971978 learned_addr learned my addr 172.21.15.105:0/441971978 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:23:59.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f040a208640 1 --2- 172.21.15.105:0/441971978 >> v2:172.21.15.148:3300/0 conn(0x7f04040ad850 0x7f04040701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:59.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0409206640 1 --2- 172.21.15.105:0/441971978 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f04040caf60 0x7f04040cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:59.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0408a05640 1 -- 172.21.15.105:0/441971978 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1887009313 0 0) 0x7f040406bf50 con 0x7f0404159f40 2023-11-28T18:23:59.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0408a05640 1 -- 172.21.15.105:0/441971978 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03f0003530 con 0x7f0404159f40 2023-11-28T18:23:59.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0408a05640 1 -- 172.21.15.105:0/441971978 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3616258038 0 0) 0x7f03f0003530 con 0x7f0404159f40 2023-11-28T18:23:59.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0408a05640 1 -- 172.21.15.105:0/441971978 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f04040caf60 msgr2=0x7f04040cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:59.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0408a05640 1 --2- 172.21.15.105:0/441971978 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f04040caf60 0x7f04040cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0408a05640 1 -- 172.21.15.105:0/441971978 >> v2:172.21.15.148:3300/0 conn(0x7f04040ad850 msgr2=0x7f04040701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:59.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0408a05640 1 --2- 172.21.15.105:0/441971978 >> v2:172.21.15.148:3300/0 conn(0x7f04040ad850 0x7f04040701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0408a05640 1 -- 172.21.15.105:0/441971978 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0404071380 con 0x7f0404159f40 2023-11-28T18:23:59.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0408a05640 1 -- 172.21.15.105:0/441971978 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f0400001680 con 0x7f0404159f40 2023-11-28T18:23:59.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0408a05640 1 -- 172.21.15.105:0/441971978 >> v1:172.21.15.105:6789/0 conn(0x7f0404159f40 legacy=0x7f04040ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:59.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0408a05640 1 --2- 172.21.15.105:0/441971978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f0003980 0x7f03f0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:59.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0408a05640 1 --2- 172.21.15.105:0/441971978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f00047d0 0x7f03f0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:59.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0408a05640 1 --2- 172.21.15.105:0/441971978 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03f0005190 0x7f03f000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:59.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0408a05640 1 -- 172.21.15.105:0/441971978 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04040cb3c0 con 0x7f03f0003980 2023-11-28T18:23:59.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0408a05640 1 -- 172.21.15.105:0/441971978 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040415a320 con 0x7f03f00047d0 2023-11-28T18:23:59.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0408a05640 1 -- 172.21.15.105:0/441971978 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f000bf00 con 0x7f03f0005190 2023-11-28T18:23:59.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0409206640 1 --2- 172.21.15.105:0/441971978 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f04040caf60 0x7f04040cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:59.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f040a208640 1 --2- 172.21.15.105:0/441971978 >> v2:172.21.15.148:3300/0 conn(0x7f04040ad850 0x7f04040701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:23:59.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0409a07640 1 --2- 172.21.15.105:0/441971978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f0003980 0x7f03f0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:59.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0409206640 1 --2- 172.21.15.105:0/441971978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f00047d0 0x7f03f0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:59.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f040a208640 1 --2- 172.21.15.105:0/441971978 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03f0005190 0x7f03f000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:59.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0409a07640 1 -- 172.21.15.105:0/441971978 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03f0005190 msgr2=0x7f03f000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:59.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0409a07640 1 --2- 172.21.15.105:0/441971978 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03f0005190 0x7f03f000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0409a07640 1 -- 172.21.15.105:0/441971978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f00047d0 msgr2=0x7f03f0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:23:59.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0409a07640 1 --2- 172.21.15.105:0/441971978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f00047d0 0x7f03f0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.699+0000 7f0409a07640 1 -- 172.21.15.105:0/441971978 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f0003530 con 0x7f03f0003980 2023-11-28T18:23:59.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.703+0000 7f0409a07640 1 --2- 172.21.15.105:0/441971978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f0003980 0x7f03f0003da0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0400007ce0 tx=0x7f040001ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ae3780c2c468ac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:59.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.703+0000 7f0408a05640 1 -- 172.21.15.105:0/441971978 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04000a7b40 con 0x7f03f0003980 2023-11-28T18:23:59.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.703+0000 7f0408a05640 1 -- 172.21.15.105:0/441971978 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04000a7ca0 con 0x7f03f0003980 2023-11-28T18:23:59.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.703+0000 7f0408a05640 1 -- 172.21.15.105:0/441971978 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0400005b00 con 0x7f03f0003980 2023-11-28T18:23:59.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.703+0000 7f040bc6b640 1 -- 172.21.15.105:0/441971978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f0003980 msgr2=0x7f03f0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:59.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.703+0000 7f040bc6b640 1 --2- 172.21.15.105:0/441971978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f0003980 0x7f03f0003da0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0400007ce0 tx=0x7f040001ee00 comp rx=0 tx=0).stop 2023-11-28T18:23:59.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.703+0000 7f040a208640 1 -- 172.21.15.105:0/441971978 reap_dead start 2023-11-28T18:23:59.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.703+0000 7f040bc6b640 1 -- 172.21.15.105:0/441971978 shutdown_connections 2023-11-28T18:23:59.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.703+0000 7f040bc6b640 1 -- 172.21.15.105:0/441971978 >> 172.21.15.105:0/441971978 conn(0x7f04040ceb90 msgr2=0x7f04040cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:59.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.703+0000 7f040bc6b640 1 -- 172.21.15.105:0/441971978 shutdown_connections 2023-11-28T18:23:59.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.703+0000 7f040bc6b640 1 -- 172.21.15.105:0/441971978 wait complete. 2023-11-28T18:23:59.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f040bc6b640 1 Processor -- start 2023-11-28T18:23:59.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f040bc6b640 1 -- start start 2023-11-28T18:23:59.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f040bc6b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f0003980 0x7f0404148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:59.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f040bc6b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f04041520c0 0x7f0404158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:59.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f040bc6b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04040ac3b0 0x7f04040ac830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:59.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f040bc6b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0404157540 con 0x7f03f0003980 2023-11-28T18:23:59.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f040bc6b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04040acd70 con 0x7f04040ac3b0 2023-11-28T18:23:59.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f040bc6b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04041f64e0 con 0x7f04041520c0 2023-11-28T18:23:59.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f0409a07640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f0003980 0x7f0404148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:59.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f0409a07640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f0003980 0x7f0404148490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55478/0 (socket says 172.21.15.105:55478) 2023-11-28T18:23:59.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f0409a07640 1 -- 172.21.15.105:0/4161279784 learned_addr learned my addr 172.21.15.105:0/4161279784 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:23:59.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f040a208640 1 --2- 172.21.15.105:0/4161279784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04040ac3b0 0x7f04040ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:59.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f0409206640 1 --2- 172.21.15.105:0/4161279784 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f04041520c0 0x7f0404158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:59.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f0409a07640 1 -- 172.21.15.105:0/4161279784 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f04041520c0 msgr2=0x7f0404158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:59.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f0409a07640 1 --2- 172.21.15.105:0/4161279784 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f04041520c0 0x7f0404158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f0409a07640 1 -- 172.21.15.105:0/4161279784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04040ac3b0 msgr2=0x7f04040ac830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:59.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f0409a07640 1 --2- 172.21.15.105:0/4161279784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04040ac3b0 0x7f04040ac830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f0409a07640 1 -- 172.21.15.105:0/4161279784 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04041f6780 con 0x7f03f0003980 2023-11-28T18:23:59.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f0409a07640 1 --2- 172.21.15.105:0/4161279784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f0003980 0x7f0404148490 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f04000ad040 tx=0x7f03f0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:59.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f03fa7fc640 1 -- 172.21.15.105:0/4161279784 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04000049f0 con 0x7f03f0003980 2023-11-28T18:23:59.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f03fa7fc640 1 -- 172.21.15.105:0/4161279784 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0400004b50 con 0x7f03f0003980 2023-11-28T18:23:59.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f040bc6b640 1 -- 172.21.15.105:0/4161279784 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04041f6a60 con 0x7f03f0003980 2023-11-28T18:23:59.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f03fa7fc640 1 -- 172.21.15.105:0/4161279784 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04000070d0 con 0x7f03f0003980 2023-11-28T18:23:59.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f040bc6b640 1 -- 172.21.15.105:0/4161279784 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04041f7020 con 0x7f03f0003980 2023-11-28T18:23:59.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.707+0000 7f040bc6b640 1 -- 172.21.15.105:0/4161279784 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0404067570 con 0x7f03f0003980 2023-11-28T18:23:59.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.711+0000 7f03fa7fc640 1 -- 172.21.15.105:0/4161279784 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f0400007230 con 0x7f03f0003980 2023-11-28T18:23:59.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.711+0000 7f03fa7fc640 1 --2- 172.21.15.105:0/4161279784 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f03e80668d0 0x7f03e8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:23:59.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.711+0000 7f03fa7fc640 1 -- 172.21.15.105:0/4161279784 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f040011e4c0 con 0x7f03f0003980 2023-11-28T18:23:59.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.711+0000 7f0409206640 1 --2- 172.21.15.105:0/4161279784 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f03e80668d0 0x7f03e8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:23:59.727 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.711+0000 7f0409206640 1 --2- 172.21.15.105:0/4161279784 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f03e80668d0 0x7f03e8068d90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f040415d500 tx=0x7f03f4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:23:59.727 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.715+0000 7f03fa7fc640 1 -- 172.21.15.105:0/4161279784 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f04000eb420 con 0x7f03f0003980 2023-11-28T18:23:59.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.867+0000 7f040bc6b640 1 -- 172.21.15.105:0/4161279784 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04040cd590 con 0x7f03f0003980 2023-11-28T18:23:59.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.867+0000 7f03fa7fc640 1 -- 172.21.15.105:0/4161279784 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7f04000eea40 con 0x7f03f0003980 2023-11-28T18:23:59.871 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:23:59.871 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":24,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:23:38.458954+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:23:59.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.867+0000 7f040bc6b640 1 -- 172.21.15.105:0/4161279784 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f03e80668d0 msgr2=0x7f03e8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:59.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.867+0000 7f040bc6b640 1 --2- 172.21.15.105:0/4161279784 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f03e80668d0 0x7f03e8068d90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f040415d500 tx=0x7f03f4009190 comp rx=0 tx=0).stop 2023-11-28T18:23:59.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.867+0000 7f040bc6b640 1 -- 172.21.15.105:0/4161279784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f0003980 msgr2=0x7f0404148490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:23:59.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.867+0000 7f040bc6b640 1 --2- 172.21.15.105:0/4161279784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f0003980 0x7f0404148490 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f04000ad040 tx=0x7f03f0005800 comp rx=0 tx=0).stop 2023-11-28T18:23:59.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.867+0000 7f040bc6b640 1 -- 172.21.15.105:0/4161279784 shutdown_connections 2023-11-28T18:23:59.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.871+0000 7f040bc6b640 1 --2- 172.21.15.105:0/4161279784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04040ac3b0 0x7f04040ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.871+0000 7f040bc6b640 1 --2- 172.21.15.105:0/4161279784 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f04041520c0 0x7f0404158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.871+0000 7f040bc6b640 1 --2- 172.21.15.105:0/4161279784 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f03e80668d0 0x7f03e8068d90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.871+0000 7f040bc6b640 1 --2- 172.21.15.105:0/4161279784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f0003980 0x7f0404148490 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:23:59.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.871+0000 7f040bc6b640 1 -- 172.21.15.105:0/4161279784 >> 172.21.15.105:0/4161279784 conn(0x7f04040ceb90 msgr2=0x7f04040cd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:23:59.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.871+0000 7f040bc6b640 1 -- 172.21.15.105:0/4161279784 shutdown_connections 2023-11-28T18:23:59.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:23:59.871+0000 7f040bc6b640 1 -- 172.21.15.105:0/4161279784 wait complete. 2023-11-28T18:23:59.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:23:31.485422+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:23:32.274378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T18:23:36.100071+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T18:23:59.893 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:24:00.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.043+0000 7f72c1832640 1 Processor -- start 2023-11-28T18:24:00.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.043+0000 7f72c1832640 1 -- start start 2023-11-28T18:24:00.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.043+0000 7f72c1832640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72bc15d3c0 0x7f72bc071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:00.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.043+0000 7f72c1832640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f72bc161870 0x7f72bc0ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:00.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.043+0000 7f72c1832640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f72bc070a10 0x7f72bc070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:00.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.043+0000 7f72c1832640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72bc0713c0 con 0x7f72bc161870 2023-11-28T18:24:00.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.043+0000 7f72c1832640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72bc0cefa0 con 0x7f72bc070a10 2023-11-28T18:24:00.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.043+0000 7f72c1832640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc15d7a0 con 0x7f72bc15d3c0 2023-11-28T18:24:00.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.043+0000 7f72b3fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f72bc161870 0x7f72bc0ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:00.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.043+0000 7f72b3fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f72bc161870 0x7f72bc0ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55484/0 (socket says 172.21.15.105:55484) 2023-11-28T18:24:00.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.043+0000 7f72b3fff640 1 -- 172.21.15.105:0/414492656 learned_addr learned my addr 172.21.15.105:0/414492656 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:00.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.043+0000 7f72baffd640 1 --2- 172.21.15.105:0/414492656 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72bc15d3c0 0x7f72bc071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:00.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.043+0000 7f72bb7fe640 1 --2- 172.21.15.105:0/414492656 >> v2:172.21.15.148:3300/0 conn(0x7f72bc070a10 0x7f72bc070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:00.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.043+0000 7f72b3fff640 1 -- 172.21.15.105:0/414492656 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72bc15d3c0 msgr2=0x7f72bc071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:00.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.043+0000 7f72b3fff640 1 --2- 172.21.15.105:0/414492656 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72bc15d3c0 0x7f72bc071a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.043+0000 7f72b3fff640 1 -- 172.21.15.105:0/414492656 >> v2:172.21.15.148:3300/0 conn(0x7f72bc070a10 msgr2=0x7f72bc070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:00.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.043+0000 7f72b3fff640 1 --2- 172.21.15.105:0/414492656 >> v2:172.21.15.148:3300/0 conn(0x7f72bc070a10 0x7f72bc070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.043+0000 7f72b3fff640 1 -- 172.21.15.105:0/414492656 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72bc0ce9e0 con 0x7f72bc161870 2023-11-28T18:24:00.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.051+0000 7f72b3fff640 1 --2- 172.21.15.105:0/414492656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72bc161870 0x7f72bc0ce190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0cf4d0 tx=0x7f72a401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a011e6556fdefa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:00.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.051+0000 7f72ba7fc640 1 -- 172.21.15.105:0/414492656 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a40aa5f0 con 0x7f72bc161870 2023-11-28T18:24:00.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.051+0000 7f72ba7fc640 1 -- 172.21.15.105:0/414492656 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a40aa750 con 0x7f72bc161870 2023-11-28T18:24:00.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.051+0000 7f72ba7fc640 1 -- 172.21.15.105:0/414492656 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a40aaae0 con 0x7f72bc161870 2023-11-28T18:24:00.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.051+0000 7f72c1832640 1 -- 172.21.15.105:0/414492656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72bc161870 msgr2=0x7f72bc0ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:00.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.051+0000 7f72c1832640 1 --2- 172.21.15.105:0/414492656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72bc161870 0x7f72bc0ce190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0cf4d0 tx=0x7f72a401c4d0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.051+0000 7f72c1832640 1 -- 172.21.15.105:0/414492656 shutdown_connections 2023-11-28T18:24:00.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.051+0000 7f72c1832640 1 --2- 172.21.15.105:0/414492656 >> v2:172.21.15.148:3300/0 conn(0x7f72bc070a10 0x7f72bc070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.051+0000 7f72c1832640 1 --2- 172.21.15.105:0/414492656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72bc161870 0x7f72bc0ce190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.051+0000 7f72c1832640 1 --2- 172.21.15.105:0/414492656 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72bc15d3c0 0x7f72bc071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.051+0000 7f72c1832640 1 -- 172.21.15.105:0/414492656 >> 172.21.15.105:0/414492656 conn(0x7f72bc0cf540 msgr2=0x7f72bc0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:00.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.051+0000 7f72c1832640 1 -- 172.21.15.105:0/414492656 shutdown_connections 2023-11-28T18:24:00.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.051+0000 7f72c1832640 1 -- 172.21.15.105:0/414492656 wait complete. 2023-11-28T18:24:00.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.051+0000 7f72c1832640 1 Processor -- start 2023-11-28T18:24:00.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.051+0000 7f72c1832640 1 -- start start 2023-11-28T18:24:00.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.051+0000 7f72c1832640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72bc070a10 0x7f72bc143a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:00.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72c1832640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72bc15d3c0 0x7f72bc156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:00.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72c1832640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72bc161870 0x7f72bc143000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:00.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72c1832640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc15f640 con 0x7f72bc161870 2023-11-28T18:24:00.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72c1832640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc1594d0 con 0x7f72bc15d3c0 2023-11-28T18:24:00.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72c1832640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc1575b0 con 0x7f72bc070a10 2023-11-28T18:24:00.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72bb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72bc161870 0x7f72bc143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:00.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72b3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72bc15d3c0 0x7f72bc156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:00.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72bb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72bc161870 0x7f72bc143000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55494/0 (socket says 172.21.15.105:55494) 2023-11-28T18:24:00.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72baffd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72bc070a10 0x7f72bc143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:00.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72bb7fe640 1 -- 172.21.15.105:0/2948805208 learned_addr learned my addr 172.21.15.105:0/2948805208 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:00.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72bb7fe640 1 -- 172.21.15.105:0/2948805208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72bc070a10 msgr2=0x7f72bc143a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:00.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72bb7fe640 1 --2- 172.21.15.105:0/2948805208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72bc070a10 0x7f72bc143a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72bb7fe640 1 -- 172.21.15.105:0/2948805208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72bc15d3c0 msgr2=0x7f72bc156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:00.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72bb7fe640 1 --2- 172.21.15.105:0/2948805208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72bc15d3c0 0x7f72bc156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72bb7fe640 1 -- 172.21.15.105:0/2948805208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a40094d0 con 0x7f72bc161870 2023-11-28T18:24:00.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72bb7fe640 1 --2- 172.21.15.105:0/2948805208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72bc161870 0x7f72bc143000 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f72b400eed0 tx=0x7f72b400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:00.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72b8ff9640 1 -- 172.21.15.105:0/2948805208 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b4010040 con 0x7f72bc161870 2023-11-28T18:24:00.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72b8ff9640 1 -- 172.21.15.105:0/2948805208 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b400bd20 con 0x7f72bc161870 2023-11-28T18:24:00.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72b8ff9640 1 -- 172.21.15.105:0/2948805208 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b4014640 con 0x7f72bc161870 2023-11-28T18:24:00.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72c1832640 1 -- 172.21.15.105:0/2948805208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72bc157310 con 0x7f72bc161870 2023-11-28T18:24:00.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72c1832640 1 -- 172.21.15.105:0/2948805208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72bc15eca0 con 0x7f72bc161870 2023-11-28T18:24:00.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72c1832640 1 -- 172.21.15.105:0/2948805208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72bc15f780 con 0x7f72bc161870 2023-11-28T18:24:00.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.055+0000 7f72b8ff9640 1 -- 172.21.15.105:0/2948805208 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f72b40147a0 con 0x7f72bc161870 2023-11-28T18:24:00.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.059+0000 7f72b8ff9640 1 --2- 172.21.15.105:0/2948805208 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f729c0668d0 0x7f729c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:00.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.059+0000 7f72b8ff9640 1 -- 172.21.15.105:0/2948805208 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f72b4084c30 con 0x7f72bc161870 2023-11-28T18:24:00.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.059+0000 7f72baffd640 1 --2- 172.21.15.105:0/2948805208 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f729c0668d0 0x7f729c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:00.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.059+0000 7f72baffd640 1 --2- 172.21.15.105:0/2948805208 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f729c0668d0 0x7f729c068d90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0cdbf0 tx=0x7f72ac009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:00.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.063+0000 7f72b8ff9640 1 -- 172.21.15.105:0/2948805208 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f72bc15f780 con 0x7f72bc161870 2023-11-28T18:24:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.219+0000 7f72c1832640 1 -- 172.21.15.105:0/2948805208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f72bc15f780 con 0x7f72bc161870 2023-11-28T18:24:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.219+0000 7f72b8ff9640 1 -- 172.21.15.105:0/2948805208 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f72bc15f780 con 0x7f72bc161870 2023-11-28T18:24:00.223 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:24:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.219+0000 7f72c1832640 1 -- 172.21.15.105:0/2948805208 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f729c0668d0 msgr2=0x7f729c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.219+0000 7f72c1832640 1 --2- 172.21.15.105:0/2948805208 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f729c0668d0 0x7f729c068d90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0cdbf0 tx=0x7f72ac009190 comp rx=0 tx=0).stop 2023-11-28T18:24:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.219+0000 7f72c1832640 1 -- 172.21.15.105:0/2948805208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72bc161870 msgr2=0x7f72bc143000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.219+0000 7f72c1832640 1 --2- 172.21.15.105:0/2948805208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72bc161870 0x7f72bc143000 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f72b400eed0 tx=0x7f72b400b480 comp rx=0 tx=0).stop 2023-11-28T18:24:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.223+0000 7f72c1832640 1 -- 172.21.15.105:0/2948805208 shutdown_connections 2023-11-28T18:24:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.223+0000 7f72c1832640 1 --2- 172.21.15.105:0/2948805208 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f729c0668d0 0x7f729c068d90 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.223+0000 7f72c1832640 1 --2- 172.21.15.105:0/2948805208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72bc161870 0x7f72bc143000 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.223+0000 7f72c1832640 1 --2- 172.21.15.105:0/2948805208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72bc15d3c0 0x7f72bc156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.223+0000 7f72c1832640 1 --2- 172.21.15.105:0/2948805208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72bc070a10 0x7f72bc143a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.223+0000 7f72c1832640 1 -- 172.21.15.105:0/2948805208 >> 172.21.15.105:0/2948805208 conn(0x7f72bc0cf540 msgr2=0x7f72bc0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.223+0000 7f72c1832640 1 -- 172.21.15.105:0/2948805208 shutdown_connections 2023-11-28T18:24:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.223+0000 7f72c1832640 1 -- 172.21.15.105:0/2948805208 wait complete. 2023-11-28T18:24:00.252 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:24:00.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.391+0000 7fe54d8ad640 1 Processor -- start 2023-11-28T18:24:00.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.391+0000 7fe54d8ad640 1 -- start start 2023-11-28T18:24:00.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.391+0000 7fe54d8ad640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe5480b3a90 0x7fe548167c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:00.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.391+0000 7fe54d8ad640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5480ce4c0 con 0x7fe5480d3920 2023-11-28T18:24:00.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.391+0000 7fe54d8ad640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5480ce7a0 con 0x7fe54815d000 2023-11-28T18:24:00.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.391+0000 7fe54d8ad640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5480d3d80 con 0x7fe5480b3a90 2023-11-28T18:24:00.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.391+0000 7fe547fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe5480d3920 0x7fe5480cfde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39394/0 (socket says 172.21.15.105:39394) 2023-11-28T18:24:00.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.391+0000 7fe547fff640 1 -- 172.21.15.105:0/424908042 learned_addr learned my addr 172.21.15.105:0/424908042 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:00.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.391+0000 7fe54d0ac640 1 --2- 172.21.15.105:0/424908042 >> v2:172.21.15.105:3300/0 conn(0x7fe5480b3a90 0x7fe548167c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:00.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.395+0000 7fe54d0ac640 1 -- 172.21.15.105:0/424908042 >> v1:172.21.15.148:6789/0 conn(0x7fe54815d000 legacy=0x7fe5480cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:00.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.395+0000 7fe54d0ac640 1 -- 172.21.15.105:0/424908042 >> v1:172.21.15.105:6789/0 conn(0x7fe5480d3920 legacy=0x7fe5480cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:00.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.395+0000 7fe54d0ac640 1 -- 172.21.15.105:0/424908042 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5480cea80 con 0x7fe5480b3a90 2023-11-28T18:24:00.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.399+0000 7fe54d0ac640 1 --2- 172.21.15.105:0/424908042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5480b3a90 0x7fe548167c20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe540009d90 tx=0x7fe54001c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b497580fa60ef1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:00.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.399+0000 7fe5477fe640 1 -- 172.21.15.105:0/424908042 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5400a57a0 con 0x7fe5480b3a90 2023-11-28T18:24:00.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.399+0000 7fe5477fe640 1 -- 172.21.15.105:0/424908042 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5400a5900 con 0x7fe5480b3a90 2023-11-28T18:24:00.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.399+0000 7fe5477fe640 1 -- 172.21.15.105:0/424908042 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5400a5c90 con 0x7fe5480b3a90 2023-11-28T18:24:00.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.399+0000 7fe54d8ad640 1 -- 172.21.15.105:0/424908042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5480b3a90 msgr2=0x7fe548167c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:00.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.399+0000 7fe54d8ad640 1 --2- 172.21.15.105:0/424908042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5480b3a90 0x7fe548167c20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe540009d90 tx=0x7fe54001c8c0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.399+0000 7fe54d8ad640 1 -- 172.21.15.105:0/424908042 shutdown_connections 2023-11-28T18:24:00.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.399+0000 7fe54d8ad640 1 --2- 172.21.15.105:0/424908042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5480b3a90 0x7fe548167c20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.399+0000 7fe54d8ad640 1 -- 172.21.15.105:0/424908042 >> 172.21.15.105:0/424908042 conn(0x7fe5480d2e00 msgr2=0x7fe5480d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:00.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.399+0000 7fe54d8ad640 1 -- 172.21.15.105:0/424908042 shutdown_connections 2023-11-28T18:24:00.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.399+0000 7fe54d8ad640 1 -- 172.21.15.105:0/424908042 wait complete. 2023-11-28T18:24:00.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.403+0000 7fe54d8ad640 1 Processor -- start 2023-11-28T18:24:00.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.403+0000 7fe54d8ad640 1 -- start start 2023-11-28T18:24:00.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.403+0000 7fe54d8ad640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5480b3a90 0x7fe5480b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:00.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.403+0000 7fe54d8ad640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5480d3920 0x7fe54815d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:00.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.403+0000 7fe54d8ad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe54815d000 0x7fe54815dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:00.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.403+0000 7fe54d8ad640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5480b59c0 con 0x7fe5480b3a90 2023-11-28T18:24:00.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.403+0000 7fe54d8ad640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5480cb210 con 0x7fe54815d000 2023-11-28T18:24:00.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.403+0000 7fe54d8ad640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5480cb4f0 con 0x7fe5480d3920 2023-11-28T18:24:00.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.403+0000 7fe54c8ab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5480b3a90 0x7fe5480b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:00.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.403+0000 7fe54c8ab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5480b3a90 0x7fe5480b0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55512/0 (socket says 172.21.15.105:55512) 2023-11-28T18:24:00.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.403+0000 7fe54c8ab640 1 -- 172.21.15.105:0/2482064871 learned_addr learned my addr 172.21.15.105:0/2482064871 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:00.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.403+0000 7fe54d0ac640 1 --2- 172.21.15.105:0/2482064871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe54815d000 0x7fe54815dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:00.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.403+0000 7fe547fff640 1 --2- 172.21.15.105:0/2482064871 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5480d3920 0x7fe54815d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:00.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.403+0000 7fe54c8ab640 1 -- 172.21.15.105:0/2482064871 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5480d3920 msgr2=0x7fe54815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:00.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.403+0000 7fe54c8ab640 1 --2- 172.21.15.105:0/2482064871 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5480d3920 0x7fe54815d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.403+0000 7fe54c8ab640 1 -- 172.21.15.105:0/2482064871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe54815d000 msgr2=0x7fe54815dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:00.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.403+0000 7fe54c8ab640 1 --2- 172.21.15.105:0/2482064871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe54815d000 0x7fe54815dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.403+0000 7fe54c8ab640 1 -- 172.21.15.105:0/2482064871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe540009a70 con 0x7fe5480b3a90 2023-11-28T18:24:00.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.403+0000 7fe547fff640 1 --2- 172.21.15.105:0/2482064871 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5480d3920 0x7fe54815d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:00.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.407+0000 7fe54c8ab640 1 --2- 172.21.15.105:0/2482064871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5480b3a90 0x7fe5480b0120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00c500 tx=0x7fe53c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:00.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.407+0000 7fe5457fa640 1 -- 172.21.15.105:0/2482064871 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53c005220 con 0x7fe5480b3a90 2023-11-28T18:24:00.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.407+0000 7fe5457fa640 1 -- 172.21.15.105:0/2482064871 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53c005380 con 0x7fe5480b3a90 2023-11-28T18:24:00.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.407+0000 7fe5457fa640 1 -- 172.21.15.105:0/2482064871 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53c007270 con 0x7fe5480b3a90 2023-11-28T18:24:00.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.407+0000 7fe54d8ad640 1 -- 172.21.15.105:0/2482064871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe54815e930 con 0x7fe5480b3a90 2023-11-28T18:24:00.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.407+0000 7fe54d8ad640 1 -- 172.21.15.105:0/2482064871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe54815eef0 con 0x7fe5480b3a90 2023-11-28T18:24:00.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.407+0000 7fe54d8ad640 1 -- 172.21.15.105:0/2482064871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe548074580 con 0x7fe5480b3a90 2023-11-28T18:24:00.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.407+0000 7fe5457fa640 1 -- 172.21.15.105:0/2482064871 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fe53c0073d0 con 0x7fe5480b3a90 2023-11-28T18:24:00.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.407+0000 7fe5457fa640 1 --2- 172.21.15.105:0/2482064871 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe52c0668d0 0x7fe52c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:00.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.407+0000 7fe5457fa640 1 -- 172.21.15.105:0/2482064871 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe53c083f90 con 0x7fe5480b3a90 2023-11-28T18:24:00.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.411+0000 7fe547fff640 1 --2- 172.21.15.105:0/2482064871 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe52c0668d0 0x7fe52c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:00.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.411+0000 7fe547fff640 1 --2- 172.21.15.105:0/2482064871 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe52c0668d0 0x7fe52c068d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe548158130 tx=0x7fe534009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:00.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.411+0000 7fe5457fa640 1 -- 172.21.15.105:0/2482064871 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe53c0b5280 con 0x7fe5480b3a90 2023-11-28T18:24:00.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.567+0000 7fe54d8ad640 1 -- 172.21.15.105:0/2482064871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe54815f110 con 0x7fe5480b3a90 2023-11-28T18:24:00.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.567+0000 7fe5457fa640 1 -- 172.21.15.105:0/2482064871 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe53c050ef0 con 0x7fe5480b3a90 2023-11-28T18:24:00.570 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:00.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.567+0000 7fe54d8ad640 1 -- 172.21.15.105:0/2482064871 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe52c0668d0 msgr2=0x7fe52c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:00.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.567+0000 7fe54d8ad640 1 --2- 172.21.15.105:0/2482064871 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe52c0668d0 0x7fe52c068d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe548158130 tx=0x7fe534009040 comp rx=0 tx=0).stop 2023-11-28T18:24:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.567+0000 7fe54d8ad640 1 -- 172.21.15.105:0/2482064871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5480b3a90 msgr2=0x7fe5480b0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.567+0000 7fe54d8ad640 1 --2- 172.21.15.105:0/2482064871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5480b3a90 0x7fe5480b0120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00c500 tx=0x7fe53c00c9c0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.567+0000 7fe54d8ad640 1 -- 172.21.15.105:0/2482064871 shutdown_connections 2023-11-28T18:24:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.567+0000 7fe54d8ad640 1 --2- 172.21.15.105:0/2482064871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe54815d000 0x7fe54815dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.567+0000 7fe54d8ad640 1 --2- 172.21.15.105:0/2482064871 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5480d3920 0x7fe54815d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.567+0000 7fe54d8ad640 1 --2- 172.21.15.105:0/2482064871 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe52c0668d0 0x7fe52c068d90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.567+0000 7fe54d8ad640 1 --2- 172.21.15.105:0/2482064871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5480b3a90 0x7fe5480b0120 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.567+0000 7fe54d8ad640 1 -- 172.21.15.105:0/2482064871 >> 172.21.15.105:0/2482064871 conn(0x7fe5480d2e00 msgr2=0x7fe548158180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:00.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.567+0000 7fe54d8ad640 1 -- 172.21.15.105:0/2482064871 shutdown_connections 2023-11-28T18:24:00.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.567+0000 7fe54d8ad640 1 -- 172.21.15.105:0/2482064871 wait complete. 2023-11-28T18:24:00.591 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:24:00.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.731+0000 7f8a78c9c640 1 Processor -- start 2023-11-28T18:24:00.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.731+0000 7f8a78c9c640 1 -- start start 2023-11-28T18:24:00.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.731+0000 7f8a78c9c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8a7415d4f0 0x7f8a740d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.731+0000 7f8a78c9c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a740b2d20 0x7f8a740b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.731+0000 7f8a78c9c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a740b25b0 con 0x7f8a740d3c30 2023-11-28T18:24:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.731+0000 7f8a78c9c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a740b2900 con 0x7f8a740b2d20 2023-11-28T18:24:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.731+0000 7f8a78c9c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a740d0a20 con 0x7f8a7415d4f0 2023-11-28T18:24:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.731+0000 7f8a72d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a740b2d20 0x7f8a740b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.731+0000 7f8a72d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a740b2d20 0x7f8a740b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55520/0 (socket says 172.21.15.105:55520) 2023-11-28T18:24:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.731+0000 7f8a72d9d640 1 -- 172.21.15.105:0/3463920090 learned_addr learned my addr 172.21.15.105:0/3463920090 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.731+0000 7f8a7259c640 1 --2- 172.21.15.105:0/3463920090 >> v2:172.21.15.148:3300/0 conn(0x7f8a7415d4f0 0x7f8a740d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:00.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.731+0000 7f8a72d9d640 1 -- 172.21.15.105:0/3463920090 >> v2:172.21.15.148:3300/0 conn(0x7f8a7415d4f0 msgr2=0x7f8a740d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:00.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.731+0000 7f8a72d9d640 1 --2- 172.21.15.105:0/3463920090 >> v2:172.21.15.148:3300/0 conn(0x7f8a7415d4f0 0x7f8a740d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.731+0000 7f8a72d9d640 1 -- 172.21.15.105:0/3463920090 >> v1:172.21.15.148:6789/0 conn(0x7f8a740d3c30 legacy=0x7f8a740d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:00.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.731+0000 7f8a72d9d640 1 -- 172.21.15.105:0/3463920090 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a740cea30 con 0x7f8a740b2d20 2023-11-28T18:24:00.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.735+0000 7f8a72d9d640 1 --2- 172.21.15.105:0/3463920090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a740b2d20 0x7f8a740b3150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8a7408bbf0 tx=0x7f8a6c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24cc0ce90dccf65 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:00.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.735+0000 7f8a7159a640 1 -- 172.21.15.105:0/3463920090 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c0a57d0 con 0x7f8a740b2d20 2023-11-28T18:24:00.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.735+0000 7f8a7159a640 1 -- 172.21.15.105:0/3463920090 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6c0a5930 con 0x7f8a740b2d20 2023-11-28T18:24:00.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.735+0000 7f8a7159a640 1 -- 172.21.15.105:0/3463920090 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c0a5cc0 con 0x7f8a740b2d20 2023-11-28T18:24:00.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.735+0000 7f8a78c9c640 1 -- 172.21.15.105:0/3463920090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a740b2d20 msgr2=0x7f8a740b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:00.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.735+0000 7f8a78c9c640 1 --2- 172.21.15.105:0/3463920090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a740b2d20 0x7f8a740b3150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8a7408bbf0 tx=0x7f8a6c01c910 comp rx=0 tx=0).stop 2023-11-28T18:24:00.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.735+0000 7f8a78c9c640 1 -- 172.21.15.105:0/3463920090 shutdown_connections 2023-11-28T18:24:00.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.735+0000 7f8a78c9c640 1 --2- 172.21.15.105:0/3463920090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a740b2d20 0x7f8a740b3150 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.735+0000 7f8a78c9c640 1 --2- 172.21.15.105:0/3463920090 >> v2:172.21.15.148:3300/0 conn(0x7f8a7415d4f0 0x7f8a740d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.735+0000 7f8a78c9c640 1 -- 172.21.15.105:0/3463920090 >> 172.21.15.105:0/3463920090 conn(0x7f8a740d3030 msgr2=0x7f8a740d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:00.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.735+0000 7f8a78c9c640 1 -- 172.21.15.105:0/3463920090 shutdown_connections 2023-11-28T18:24:00.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.735+0000 7f8a78c9c640 1 -- 172.21.15.105:0/3463920090 wait complete. 2023-11-28T18:24:00.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.739+0000 7f8a78c9c640 1 Processor -- start 2023-11-28T18:24:00.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.739+0000 7f8a78c9c640 1 -- start start 2023-11-28T18:24:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.739+0000 7f8a78c9c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a740b2d20 0x7f8a7415b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.739+0000 7f8a78c9c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a740d3c30 0x7f8a740c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.739+0000 7f8a78c9c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a7415d4f0 0x7f8a740ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.739+0000 7f8a78c9c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a740c9ea0 con 0x7f8a740d3c30 2023-11-28T18:24:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.739+0000 7f8a78c9c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a740cab70 con 0x7f8a740b2d20 2023-11-28T18:24:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.739+0000 7f8a78c9c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a74158a80 con 0x7f8a7415d4f0 2023-11-28T18:24:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.739+0000 7f8a71d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a740d3c30 0x7f8a740c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.739+0000 7f8a71d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a740d3c30 0x7f8a740c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55536/0 (socket says 172.21.15.105:55536) 2023-11-28T18:24:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.739+0000 7f8a71d9b640 1 -- 172.21.15.105:0/324458701 learned_addr learned my addr 172.21.15.105:0/324458701 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.739+0000 7f8a7259c640 1 --2- 172.21.15.105:0/324458701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a740b2d20 0x7f8a7415b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.739+0000 7f8a72d9d640 1 --2- 172.21.15.105:0/324458701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a7415d4f0 0x7f8a740ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.739+0000 7f8a71d9b640 1 -- 172.21.15.105:0/324458701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a7415d4f0 msgr2=0x7f8a740ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.739+0000 7f8a71d9b640 1 --2- 172.21.15.105:0/324458701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a7415d4f0 0x7f8a740ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.739+0000 7f8a71d9b640 1 -- 172.21.15.105:0/324458701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a740b2d20 msgr2=0x7f8a7415b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.739+0000 7f8a71d9b640 1 --2- 172.21.15.105:0/324458701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a740b2d20 0x7f8a7415b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.739+0000 7f8a71d9b640 1 -- 172.21.15.105:0/324458701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c009b30 con 0x7f8a740d3c30 2023-11-28T18:24:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.739+0000 7f8a71d9b640 1 --2- 172.21.15.105:0/324458701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a740d3c30 0x7f8a740c9960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8a6800c500 tx=0x7f8a6800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.739+0000 7f8a62ffd640 1 -- 172.21.15.105:0/324458701 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a68005220 con 0x7f8a740d3c30 2023-11-28T18:24:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.743+0000 7f8a62ffd640 1 -- 172.21.15.105:0/324458701 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a68005380 con 0x7f8a740d3c30 2023-11-28T18:24:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.743+0000 7f8a62ffd640 1 -- 172.21.15.105:0/324458701 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a68007270 con 0x7f8a740d3c30 2023-11-28T18:24:00.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.743+0000 7f8a78c9c640 1 -- 172.21.15.105:0/324458701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a74158d60 con 0x7f8a740d3c30 2023-11-28T18:24:00.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.743+0000 7f8a78c9c640 1 -- 172.21.15.105:0/324458701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7415b030 con 0x7f8a740d3c30 2023-11-28T18:24:00.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.743+0000 7f8a78c9c640 1 -- 172.21.15.105:0/324458701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a74067570 con 0x7f8a740d3c30 2023-11-28T18:24:00.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.743+0000 7f8a62ffd640 1 -- 172.21.15.105:0/324458701 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8a680073d0 con 0x7f8a740d3c30 2023-11-28T18:24:00.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.743+0000 7f8a62ffd640 1 --2- 172.21.15.105:0/324458701 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a54066900 0x7f8a54068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:00.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.743+0000 7f8a62ffd640 1 -- 172.21.15.105:0/324458701 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8a68083f30 con 0x7f8a740d3c30 2023-11-28T18:24:00.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.743+0000 7f8a7259c640 1 --2- 172.21.15.105:0/324458701 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a54066900 0x7f8a54068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:00.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.743+0000 7f8a7259c640 1 --2- 172.21.15.105:0/324458701 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a54066900 0x7f8a54068dc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8a64009ac0 tx=0x7f8a64009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:00.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.747+0000 7f8a62ffd640 1 -- 172.21.15.105:0/324458701 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8a68050ea0 con 0x7f8a740d3c30 2023-11-28T18:24:00.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.907+0000 7f8a78c9c640 1 -- 172.21.15.105:0/324458701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8a740b0070 con 0x7f8a740d3c30 2023-11-28T18:24:00.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.911+0000 7f8a62ffd640 1 -- 172.21.15.105:0/324458701 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8a680544c0 con 0x7f8a740d3c30 2023-11-28T18:24:00.913 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:24:00.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.911+0000 7f8a78c9c640 1 -- 172.21.15.105:0/324458701 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a54066900 msgr2=0x7f8a54068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:00.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.911+0000 7f8a78c9c640 1 --2- 172.21.15.105:0/324458701 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a54066900 0x7f8a54068dc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8a64009ac0 tx=0x7f8a64009190 comp rx=0 tx=0).stop 2023-11-28T18:24:00.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.911+0000 7f8a78c9c640 1 -- 172.21.15.105:0/324458701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a740d3c30 msgr2=0x7f8a740c9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:00.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.911+0000 7f8a78c9c640 1 --2- 172.21.15.105:0/324458701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a740d3c30 0x7f8a740c9960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8a6800c500 tx=0x7f8a6800c9c0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.911+0000 7f8a78c9c640 1 -- 172.21.15.105:0/324458701 shutdown_connections 2023-11-28T18:24:00.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.911+0000 7f8a78c9c640 1 --2- 172.21.15.105:0/324458701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a7415d4f0 0x7f8a740ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.911+0000 7f8a78c9c640 1 --2- 172.21.15.105:0/324458701 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a54066900 0x7f8a54068dc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.911+0000 7f8a78c9c640 1 --2- 172.21.15.105:0/324458701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a740d3c30 0x7f8a740c9960 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.911+0000 7f8a78c9c640 1 --2- 172.21.15.105:0/324458701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a740b2d20 0x7f8a7415b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:00.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.911+0000 7f8a78c9c640 1 -- 172.21.15.105:0/324458701 >> 172.21.15.105:0/324458701 conn(0x7f8a740d3030 msgr2=0x7f8a7406c850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:00.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.911+0000 7f8a78c9c640 1 -- 172.21.15.105:0/324458701 shutdown_connections 2023-11-28T18:24:00.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:00.911+0000 7f8a78c9c640 1 -- 172.21.15.105:0/324458701 wait complete. 2023-11-28T18:24:00.934 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T18:24:01.055 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:24:01.058 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:24:01.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.071+0000 7fefd3a5f640 1 Processor -- start 2023-11-28T18:24:01.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.071+0000 7fefd3a5f640 1 -- start start 2023-11-28T18:24:01.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.071+0000 7fefd3a5f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc0b4140 0x7fefcc0d3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.071+0000 7fefd3a5f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fefcc160530 0x7fefcc160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.071+0000 7fefd3a5f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefcc160ec0 con 0x7fefcc0d10e0 2023-11-28T18:24:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.071+0000 7fefd3a5f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefcc0d1d20 con 0x7fefcc160530 2023-11-28T18:24:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.071+0000 7fefd3a5f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc0d01d0 con 0x7fefcc0b4140 2023-11-28T18:24:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.071+0000 7fefd17fb640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fefcc0d10e0 0x7fefcc0d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:35768/0 (socket says 172.21.15.105:35768) 2023-11-28T18:24:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.071+0000 7fefd17fb640 1 -- 172.21.15.105:0/4270551376 learned_addr learned my addr 172.21.15.105:0/4270551376 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.071+0000 7fefd0ffa640 1 --2- 172.21.15.105:0/4270551376 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc0b4140 0x7fefcc0d3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.071+0000 7fefd1ffc640 1 --2- 172.21.15.105:0/4270551376 >> v2:172.21.15.148:3300/0 conn(0x7fefcc160530 0x7fefcc160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:01.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.071+0000 7fefc3fff640 1 -- 172.21.15.105:0/4270551376 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1937601943 0 0) 0x7fefcc160ec0 con 0x7fefcc0d10e0 2023-11-28T18:24:01.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.071+0000 7fefc3fff640 1 -- 172.21.15.105:0/4270551376 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefb8003530 con 0x7fefcc0d10e0 2023-11-28T18:24:01.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.071+0000 7fefd1ffc640 1 -- 172.21.15.105:0/4270551376 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc0b4140 msgr2=0x7fefcc0d3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:01.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.071+0000 7fefd1ffc640 1 --2- 172.21.15.105:0/4270551376 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc0b4140 0x7fefcc0d3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.071+0000 7fefd1ffc640 1 -- 172.21.15.105:0/4270551376 >> v1:172.21.15.148:6789/0 conn(0x7fefcc0d10e0 legacy=0x7fefcc0d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:01.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.071+0000 7fefd1ffc640 1 -- 172.21.15.105:0/4270551376 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefcc0d04b0 con 0x7fefcc160530 2023-11-28T18:24:01.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.075+0000 7fefd1ffc640 1 --2- 172.21.15.105:0/4270551376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefcc160530 0x7fefcc160980 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009d90 tx=0x7fefc401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8802eb2a4f30cf32 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:01.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.075+0000 7fefc3fff640 1 -- 172.21.15.105:0/4270551376 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc40a5930 con 0x7fefcc160530 2023-11-28T18:24:01.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.075+0000 7fefc3fff640 1 -- 172.21.15.105:0/4270551376 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc40a5a90 con 0x7fefcc160530 2023-11-28T18:24:01.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.075+0000 7fefc3fff640 1 -- 172.21.15.105:0/4270551376 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc40a5e20 con 0x7fefcc160530 2023-11-28T18:24:01.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.075+0000 7fefd3a5f640 1 -- 172.21.15.105:0/4270551376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefcc160530 msgr2=0x7fefcc160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.075+0000 7fefd3a5f640 1 --2- 172.21.15.105:0/4270551376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefcc160530 0x7fefcc160980 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009d90 tx=0x7fefc401c890 comp rx=0 tx=0).stop 2023-11-28T18:24:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.075+0000 7fefd3a5f640 1 -- 172.21.15.105:0/4270551376 shutdown_connections 2023-11-28T18:24:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.075+0000 7fefd3a5f640 1 --2- 172.21.15.105:0/4270551376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefcc160530 0x7fefcc160980 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.075+0000 7fefd3a5f640 1 --2- 172.21.15.105:0/4270551376 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc0b4140 0x7fefcc0d3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.075+0000 7fefd3a5f640 1 -- 172.21.15.105:0/4270551376 >> 172.21.15.105:0/4270551376 conn(0x7fefcc0d30a0 msgr2=0x7fefcc0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd3a5f640 1 -- 172.21.15.105:0/4270551376 shutdown_connections 2023-11-28T18:24:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd3a5f640 1 -- 172.21.15.105:0/4270551376 wait complete. 2023-11-28T18:24:01.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd3a5f640 1 Processor -- start 2023-11-28T18:24:01.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd3a5f640 1 -- start start 2023-11-28T18:24:01.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd3a5f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefcc0b4140 0x7fefcc154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:01.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd3a5f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc0d10e0 0x7fefcc15b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:01.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd3a5f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefcc160530 0x7fefcc0b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:01.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd3a5f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc0b8b50 con 0x7fefcc0b4140 2023-11-28T18:24:01.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd3a5f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc0b7fd0 con 0x7fefcc160530 2023-11-28T18:24:01.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd3a5f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc0b82b0 con 0x7fefcc0d10e0 2023-11-28T18:24:01.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd17fb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefcc0b4140 0x7fefcc154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:01.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd17fb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefcc0b4140 0x7fefcc154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36280/0 (socket says 172.21.15.105:36280) 2023-11-28T18:24:01.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd17fb640 1 -- 172.21.15.105:0/1194258707 learned_addr learned my addr 172.21.15.105:0/1194258707 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:01.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd0ffa640 1 --2- 172.21.15.105:0/1194258707 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc0d10e0 0x7fefcc15b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:01.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd1ffc640 1 --2- 172.21.15.105:0/1194258707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefcc160530 0x7fefcc0b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:01.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd17fb640 1 -- 172.21.15.105:0/1194258707 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc0d10e0 msgr2=0x7fefcc15b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:01.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd17fb640 1 --2- 172.21.15.105:0/1194258707 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc0d10e0 0x7fefcc15b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd17fb640 1 -- 172.21.15.105:0/1194258707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefcc160530 msgr2=0x7fefcc0b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd17fb640 1 --2- 172.21.15.105:0/1194258707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefcc160530 0x7fefcc0b5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd17fb640 1 -- 172.21.15.105:0/1194258707 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc4009a70 con 0x7fefcc0b4140 2023-11-28T18:24:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd1ffc640 1 --2- 172.21.15.105:0/1194258707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefcc160530 0x7fefcc0b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd0ffa640 1 --2- 172.21.15.105:0/1194258707 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc0d10e0 0x7fefcc15b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.079+0000 7fefd17fb640 1 --2- 172.21.15.105:0/1194258707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefcc0b4140 0x7fefcc154e50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fefc800c4d0 tx=0x7fefc800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.083+0000 7fefc1ffb640 1 -- 172.21.15.105:0/1194258707 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc8005cf0 con 0x7fefcc0b4140 2023-11-28T18:24:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.083+0000 7fefc1ffb640 1 -- 172.21.15.105:0/1194258707 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc8005e50 con 0x7fefcc0b4140 2023-11-28T18:24:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.083+0000 7fefd3a5f640 1 -- 172.21.15.105:0/1194258707 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefcc0b8590 con 0x7fefcc0b4140 2023-11-28T18:24:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.083+0000 7fefc1ffb640 1 -- 172.21.15.105:0/1194258707 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc8007270 con 0x7fefcc0b4140 2023-11-28T18:24:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.083+0000 7fefd3a5f640 1 -- 172.21.15.105:0/1194258707 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefcc0b7c80 con 0x7fefcc0b4140 2023-11-28T18:24:01.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.083+0000 7fefd3a5f640 1 -- 172.21.15.105:0/1194258707 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefcc068650 con 0x7fefcc0b4140 2023-11-28T18:24:01.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.083+0000 7fefc1ffb640 1 -- 172.21.15.105:0/1194258707 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fefc80073d0 con 0x7fefcc0b4140 2023-11-28T18:24:01.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.083+0000 7fefc1ffb640 1 --2- 172.21.15.105:0/1194258707 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fefb00668d0 0x7fefb0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:01.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.083+0000 7fefc1ffb640 1 -- 172.21.15.105:0/1194258707 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fefc8083c90 con 0x7fefcc0b4140 2023-11-28T18:24:01.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.083+0000 7fefd0ffa640 1 --2- 172.21.15.105:0/1194258707 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fefb00668d0 0x7fefb0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:01.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.083+0000 7fefd0ffa640 1 --2- 172.21.15.105:0/1194258707 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fefb00668d0 0x7fefb0068d90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fefcc08d310 tx=0x7fefbc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:01.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.087+0000 7fefc1ffb640 1 -- 172.21.15.105:0/1194258707 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fefc8050c00 con 0x7fefcc0b4140 2023-11-28T18:24:01.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.239+0000 7fefd3a5f640 1 -- 172.21.15.105:0/1194258707 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fefcc0ceaf0 con 0x7fefcc0b4140 2023-11-28T18:24:01.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.239+0000 7fefc1ffb640 1 -- 172.21.15.105:0/1194258707 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fefc80170a0 con 0x7fefcc0b4140 2023-11-28T18:24:01.244 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:01.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.243+0000 7fefd3a5f640 1 -- 172.21.15.105:0/1194258707 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fefb00668d0 msgr2=0x7fefb0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:01.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.243+0000 7fefd3a5f640 1 --2- 172.21.15.105:0/1194258707 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fefb00668d0 0x7fefb0068d90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fefcc08d310 tx=0x7fefbc008040 comp rx=0 tx=0).stop 2023-11-28T18:24:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.243+0000 7fefd3a5f640 1 -- 172.21.15.105:0/1194258707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefcc0b4140 msgr2=0x7fefcc154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.243+0000 7fefd3a5f640 1 --2- 172.21.15.105:0/1194258707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefcc0b4140 0x7fefcc154e50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fefc800c4d0 tx=0x7fefc800c990 comp rx=0 tx=0).stop 2023-11-28T18:24:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.243+0000 7fefd3a5f640 1 -- 172.21.15.105:0/1194258707 shutdown_connections 2023-11-28T18:24:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.243+0000 7fefd3a5f640 1 --2- 172.21.15.105:0/1194258707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefcc160530 0x7fefcc0b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.243+0000 7fefd3a5f640 1 --2- 172.21.15.105:0/1194258707 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc0d10e0 0x7fefcc15b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.243+0000 7fefd3a5f640 1 --2- 172.21.15.105:0/1194258707 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fefb00668d0 0x7fefb0068d90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.243+0000 7fefd3a5f640 1 --2- 172.21.15.105:0/1194258707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefcc0b4140 0x7fefcc154e50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.243+0000 7fefd3a5f640 1 -- 172.21.15.105:0/1194258707 >> 172.21.15.105:0/1194258707 conn(0x7fefcc0d30a0 msgr2=0x7fefcc0d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.243+0000 7fefd3a5f640 1 -- 172.21.15.105:0/1194258707 shutdown_connections 2023-11-28T18:24:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.243+0000 7fefd3a5f640 1 -- 172.21.15.105:0/1194258707 wait complete. 2023-11-28T18:24:01.266 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:24:01.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.399+0000 7f9a5a8ea640 1 Processor -- start 2023-11-28T18:24:01.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.399+0000 7f9a5a8ea640 1 -- start start 2023-11-28T18:24:01.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.403+0000 7f9a5a8ea640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9a5406bf50 0x7f9a5415d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:01.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.403+0000 7f9a5a8ea640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9a540adb80 0x7f9a540adf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:01.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.403+0000 7f9a5a8ea640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a540cc7f0 con 0x7f9a54154070 2023-11-28T18:24:01.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.403+0000 7f9a5a8ea640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a54072540 con 0x7f9a540adb80 2023-11-28T18:24:01.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.403+0000 7f9a5a8ea640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a540ae4d0 con 0x7f9a5406bf50 2023-11-28T18:24:01.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.403+0000 7f9a53fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9a54154070 0x7f9a540ce310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41320/0 (socket says 172.21.15.105:41320) 2023-11-28T18:24:01.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.403+0000 7f9a53fff640 1 -- 172.21.15.105:0/4067905178 learned_addr learned my addr 172.21.15.105:0/4067905178 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:01.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.403+0000 7f9a58e87640 1 --2- 172.21.15.105:0/4067905178 >> v2:172.21.15.105:3300/0 conn(0x7f9a540adb80 0x7f9a540adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:01.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.403+0000 7f9a537fe640 1 --2- 172.21.15.105:0/4067905178 >> v2:172.21.15.148:3300/0 conn(0x7f9a5406bf50 0x7f9a5415d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:01.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.403+0000 7f9a58e87640 1 -- 172.21.15.105:0/4067905178 >> v2:172.21.15.148:3300/0 conn(0x7f9a5406bf50 msgr2=0x7f9a5415d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:01.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.403+0000 7f9a58e87640 1 --2- 172.21.15.105:0/4067905178 >> v2:172.21.15.148:3300/0 conn(0x7f9a5406bf50 0x7f9a5415d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.403+0000 7f9a58e87640 1 -- 172.21.15.105:0/4067905178 >> v1:172.21.15.105:6789/0 conn(0x7f9a54154070 legacy=0x7f9a540ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:01.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.403+0000 7f9a58e87640 1 -- 172.21.15.105:0/4067905178 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5415dd10 con 0x7f9a540adb80 2023-11-28T18:24:01.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.403+0000 7f9a537fe640 1 --2- 172.21.15.105:0/4067905178 >> v2:172.21.15.148:3300/0 conn(0x7f9a5406bf50 0x7f9a5415d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:01.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.403+0000 7f9a58e87640 1 --2- 172.21.15.105:0/4067905178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a540adb80 0x7f9a540adf90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9a54161d90 tx=0x7f9a4c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd948b0a5056ebd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:01.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.407+0000 7f9a52ffd640 1 -- 172.21.15.105:0/4067905178 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0a8640 con 0x7f9a540adb80 2023-11-28T18:24:01.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.407+0000 7f9a52ffd640 1 -- 172.21.15.105:0/4067905178 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4c0a87a0 con 0x7f9a540adb80 2023-11-28T18:24:01.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.407+0000 7f9a52ffd640 1 -- 172.21.15.105:0/4067905178 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0a8b30 con 0x7f9a540adb80 2023-11-28T18:24:01.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.407+0000 7f9a5a8ea640 1 -- 172.21.15.105:0/4067905178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a540adb80 msgr2=0x7f9a540adf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:01.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.407+0000 7f9a5a8ea640 1 --2- 172.21.15.105:0/4067905178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a540adb80 0x7f9a540adf90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9a54161d90 tx=0x7f9a4c01c8e0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.407+0000 7f9a5a8ea640 1 -- 172.21.15.105:0/4067905178 shutdown_connections 2023-11-28T18:24:01.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.407+0000 7f9a5a8ea640 1 --2- 172.21.15.105:0/4067905178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a540adb80 0x7f9a540adf90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.407+0000 7f9a5a8ea640 1 --2- 172.21.15.105:0/4067905178 >> v2:172.21.15.148:3300/0 conn(0x7f9a5406bf50 0x7f9a5415d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.407+0000 7f9a5a8ea640 1 -- 172.21.15.105:0/4067905178 >> 172.21.15.105:0/4067905178 conn(0x7f9a540ceb90 msgr2=0x7f9a540cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:01.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.407+0000 7f9a5a8ea640 1 -- 172.21.15.105:0/4067905178 shutdown_connections 2023-11-28T18:24:01.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.407+0000 7f9a5a8ea640 1 -- 172.21.15.105:0/4067905178 wait complete. 2023-11-28T18:24:01.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.407+0000 7f9a5a8ea640 1 Processor -- start 2023-11-28T18:24:01.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.407+0000 7f9a5a8ea640 1 -- start start 2023-11-28T18:24:01.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.407+0000 7f9a5a8ea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a5406bf50 0x7f9a540c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:01.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.407+0000 7f9a5a8ea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a540adb80 0x7f9a54143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:01.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.407+0000 7f9a5a8ea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a54154070 0x7f9a5414c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:01.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.407+0000 7f9a5a8ea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a54143b90 con 0x7f9a54154070 2023-11-28T18:24:01.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.407+0000 7f9a5a8ea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a54153b20 con 0x7f9a5406bf50 2023-11-28T18:24:01.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.407+0000 7f9a5a8ea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a541531f0 con 0x7f9a540adb80 2023-11-28T18:24:01.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.411+0000 7f9a58e87640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a54154070 0x7f9a5414c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:01.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.411+0000 7f9a58e87640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a54154070 0x7f9a5414c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36296/0 (socket says 172.21.15.105:36296) 2023-11-28T18:24:01.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.411+0000 7f9a58e87640 1 -- 172.21.15.105:0/842450837 learned_addr learned my addr 172.21.15.105:0/842450837 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:01.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.411+0000 7f9a53fff640 1 --2- 172.21.15.105:0/842450837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a5406bf50 0x7f9a540c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:01.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.411+0000 7f9a537fe640 1 --2- 172.21.15.105:0/842450837 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a540adb80 0x7f9a54143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:01.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.411+0000 7f9a58e87640 1 -- 172.21.15.105:0/842450837 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a540adb80 msgr2=0x7f9a54143650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:01.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.411+0000 7f9a58e87640 1 --2- 172.21.15.105:0/842450837 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a540adb80 0x7f9a54143650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.411+0000 7f9a58e87640 1 -- 172.21.15.105:0/842450837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a5406bf50 msgr2=0x7f9a540c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:01.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.411+0000 7f9a58e87640 1 --2- 172.21.15.105:0/842450837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a5406bf50 0x7f9a540c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.411+0000 7f9a58e87640 1 -- 172.21.15.105:0/842450837 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4c009a70 con 0x7f9a54154070 2023-11-28T18:24:01.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.411+0000 7f9a58e87640 1 --2- 172.21.15.105:0/842450837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a54154070 0x7f9a5414c4d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c01a510 tx=0x7f9a4c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:01.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.411+0000 7f9a50ff9640 1 -- 172.21.15.105:0/842450837 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0b0600 con 0x7f9a54154070 2023-11-28T18:24:01.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.411+0000 7f9a50ff9640 1 -- 172.21.15.105:0/842450837 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4c0b6040 con 0x7f9a54154070 2023-11-28T18:24:01.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.411+0000 7f9a5a8ea640 1 -- 172.21.15.105:0/842450837 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a541534d0 con 0x7f9a54154070 2023-11-28T18:24:01.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.411+0000 7f9a50ff9640 1 -- 172.21.15.105:0/842450837 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0b0980 con 0x7f9a54154070 2023-11-28T18:24:01.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.411+0000 7f9a5a8ea640 1 -- 172.21.15.105:0/842450837 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a54152e10 con 0x7f9a54154070 2023-11-28T18:24:01.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.411+0000 7f9a5a8ea640 1 -- 172.21.15.105:0/842450837 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a54067570 con 0x7f9a54154070 2023-11-28T18:24:01.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.411+0000 7f9a50ff9640 1 -- 172.21.15.105:0/842450837 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f9a4c0b0ae0 con 0x7f9a54154070 2023-11-28T18:24:01.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.411+0000 7f9a50ff9640 1 --2- 172.21.15.105:0/842450837 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9a400668d0 0x7f9a40068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:01.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.415+0000 7f9a50ff9640 1 -- 172.21.15.105:0/842450837 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9a4c11fcf0 con 0x7f9a54154070 2023-11-28T18:24:01.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.415+0000 7f9a53fff640 1 --2- 172.21.15.105:0/842450837 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9a400668d0 0x7f9a40068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:01.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.415+0000 7f9a53fff640 1 --2- 172.21.15.105:0/842450837 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9a400668d0 0x7f9a40068d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9a44005890 tx=0x7f9a44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:01.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.419+0000 7f9a50ff9640 1 -- 172.21.15.105:0/842450837 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9a4c0ecb10 con 0x7f9a54154070 2023-11-28T18:24:01.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.595+0000 7f9a5a8ea640 1 -- 172.21.15.105:0/842450837 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a541609e0 con 0x7f9a54154070 2023-11-28T18:24:01.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.595+0000 7f9a50ff9640 1 -- 172.21.15.105:0/842450837 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7f9a4c0f0130 con 0x7f9a54154070 2023-11-28T18:24:01.600 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:01.601 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/390529893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":390529893},{"type":"v1","addr":"172.21.15.148:6835","nonce":390529893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3924571887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3924571887},{"type":"v1","addr":"172.21.15.148:6833","nonce":3924571887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3271971772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3271971772},{"type":"v1","addr":"172.21.15.105:6835","nonce":3271971772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:23:33.288743+0000","modified":"2023-11-28T18:23:50.425976+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2504263238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2504263238},{"type":"v1","addr":"172.21.15.105:6837","nonce":2504263238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T18:24:01.602 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 9 2023-11-28T18:24:01.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.599+0000 7f9a5a8ea640 1 -- 172.21.15.105:0/842450837 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9a400668d0 msgr2=0x7f9a40068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:01.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.599+0000 7f9a5a8ea640 1 --2- 172.21.15.105:0/842450837 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9a400668d0 0x7f9a40068d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9a44005890 tx=0x7f9a44009040 comp rx=0 tx=0).stop 2023-11-28T18:24:01.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.599+0000 7f9a5a8ea640 1 -- 172.21.15.105:0/842450837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a54154070 msgr2=0x7f9a5414c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:01.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.599+0000 7f9a5a8ea640 1 --2- 172.21.15.105:0/842450837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a54154070 0x7f9a5414c4d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c01a510 tx=0x7f9a4c019680 comp rx=0 tx=0).stop 2023-11-28T18:24:01.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.599+0000 7f9a5a8ea640 1 -- 172.21.15.105:0/842450837 shutdown_connections 2023-11-28T18:24:01.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.599+0000 7f9a5a8ea640 1 --2- 172.21.15.105:0/842450837 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9a400668d0 0x7f9a40068d90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.599+0000 7f9a5a8ea640 1 --2- 172.21.15.105:0/842450837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a54154070 0x7f9a5414c4d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.599+0000 7f9a5a8ea640 1 --2- 172.21.15.105:0/842450837 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a540adb80 0x7f9a54143650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.599+0000 7f9a5a8ea640 1 --2- 172.21.15.105:0/842450837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a5406bf50 0x7f9a540c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.599+0000 7f9a5a8ea640 1 -- 172.21.15.105:0/842450837 >> 172.21.15.105:0/842450837 conn(0x7f9a540ceb90 msgr2=0x7f9a5406ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:01.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.599+0000 7f9a5a8ea640 1 -- 172.21.15.105:0/842450837 shutdown_connections 2023-11-28T18:24:01.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.599+0000 7f9a5a8ea640 1 -- 172.21.15.105:0/842450837 wait complete. 2023-11-28T18:24:01.621 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:24:01.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.755+0000 7f463f65a640 1 Processor -- start 2023-11-28T18:24:01.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.755+0000 7f463f65a640 1 -- start start 2023-11-28T18:24:01.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.759+0000 7f463f65a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f46380b4290 0x7f46380b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:01.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.759+0000 7f463f65a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4638071960 0x7f463815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:01.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.759+0000 7f463f65a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46380b4bb0 con 0x7f463815b030 2023-11-28T18:24:01.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.759+0000 7f463f65a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46380701d0 con 0x7f46380b4290 2023-11-28T18:24:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.759+0000 7f463f65a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463806df80 con 0x7f4638071960 2023-11-28T18:24:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.759+0000 7f463dbf7640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f463815b030 0x7f46380709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41324/0 (socket says 172.21.15.105:41324) 2023-11-28T18:24:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.759+0000 7f463dbf7640 1 -- 172.21.15.105:0/30496582 learned_addr learned my addr 172.21.15.105:0/30496582 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.759+0000 7f463d3f6640 1 --2- 172.21.15.105:0/30496582 >> v2:172.21.15.148:3300/0 conn(0x7f46380b4290 0x7f46380b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.759+0000 7f463cbf5640 1 --2- 172.21.15.105:0/30496582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4638071960 0x7f463815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.759+0000 7f462ffff640 1 -- 172.21.15.105:0/30496582 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 770599932 0 0) 0x7f46380b4bb0 con 0x7f463815b030 2023-11-28T18:24:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.759+0000 7f462ffff640 1 -- 172.21.15.105:0/30496582 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4624003530 con 0x7f463815b030 2023-11-28T18:24:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.759+0000 7f463d3f6640 1 -- 172.21.15.105:0/30496582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4638071960 msgr2=0x7f463815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.759+0000 7f463d3f6640 1 --2- 172.21.15.105:0/30496582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4638071960 0x7f463815f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.759+0000 7f463d3f6640 1 -- 172.21.15.105:0/30496582 >> v1:172.21.15.105:6789/0 conn(0x7f463815b030 legacy=0x7f46380709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.759+0000 7f463d3f6640 1 -- 172.21.15.105:0/30496582 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4638071df0 con 0x7f46380b4290 2023-11-28T18:24:01.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.763+0000 7f463d3f6640 1 --2- 172.21.15.105:0/30496582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46380b4290 0x7f46380b4670 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f46340097c0 tx=0x7f463401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c79e763cca52803c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:01.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.763+0000 7f462ffff640 1 -- 172.21.15.105:0/30496582 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46340ab650 con 0x7f46380b4290 2023-11-28T18:24:01.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.763+0000 7f462ffff640 1 -- 172.21.15.105:0/30496582 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46340ab7b0 con 0x7f46380b4290 2023-11-28T18:24:01.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.763+0000 7f462ffff640 1 -- 172.21.15.105:0/30496582 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46340abb40 con 0x7f46380b4290 2023-11-28T18:24:01.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.763+0000 7f463f65a640 1 -- 172.21.15.105:0/30496582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46380b4290 msgr2=0x7f46380b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:01.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.763+0000 7f463f65a640 1 --2- 172.21.15.105:0/30496582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46380b4290 0x7f46380b4670 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f46340097c0 tx=0x7f463401c450 comp rx=0 tx=0).stop 2023-11-28T18:24:01.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.763+0000 7f463f65a640 1 -- 172.21.15.105:0/30496582 shutdown_connections 2023-11-28T18:24:01.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.763+0000 7f463f65a640 1 --2- 172.21.15.105:0/30496582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4638071960 0x7f463815f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.763+0000 7f463f65a640 1 --2- 172.21.15.105:0/30496582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46380b4290 0x7f46380b4670 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.763+0000 7f463f65a640 1 -- 172.21.15.105:0/30496582 >> 172.21.15.105:0/30496582 conn(0x7f46380ceb90 msgr2=0x7f46380cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:01.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.763+0000 7f463f65a640 1 -- 172.21.15.105:0/30496582 shutdown_connections 2023-11-28T18:24:01.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.763+0000 7f463f65a640 1 -- 172.21.15.105:0/30496582 wait complete. 2023-11-28T18:24:01.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.763+0000 7f463f65a640 1 Processor -- start 2023-11-28T18:24:01.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.763+0000 7f463f65a640 1 -- start start 2023-11-28T18:24:01.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f463f65a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46380b4290 0x7f46380aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:01.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f463f65a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f463815b030 0x7f463814d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:01.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f463f65a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4638156630 0x7f4638158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:01.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f463f65a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4638099a50 con 0x7f4638156630 2023-11-28T18:24:01.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f463f65a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46380af470 con 0x7f46380b4290 2023-11-28T18:24:01.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f463f65a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46380af6d0 con 0x7f463815b030 2023-11-28T18:24:01.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f463dbf7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4638156630 0x7f4638158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:01.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f463dbf7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4638156630 0x7f4638158fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36300/0 (socket says 172.21.15.105:36300) 2023-11-28T18:24:01.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f463dbf7640 1 -- 172.21.15.105:0/240312086 learned_addr learned my addr 172.21.15.105:0/240312086 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:01.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f463d3f6640 1 --2- 172.21.15.105:0/240312086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46380b4290 0x7f46380aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:01.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f463cbf5640 1 --2- 172.21.15.105:0/240312086 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f463815b030 0x7f463814d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:01.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f463dbf7640 1 -- 172.21.15.105:0/240312086 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f463815b030 msgr2=0x7f463814d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:01.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f463dbf7640 1 --2- 172.21.15.105:0/240312086 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f463815b030 0x7f463814d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f463dbf7640 1 -- 172.21.15.105:0/240312086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46380b4290 msgr2=0x7f46380aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:01.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f463dbf7640 1 --2- 172.21.15.105:0/240312086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46380b4290 0x7f46380aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f463dbf7640 1 -- 172.21.15.105:0/240312086 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4634009420 con 0x7f4638156630 2023-11-28T18:24:01.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f463cbf5640 1 --2- 172.21.15.105:0/240312086 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f463815b030 0x7f463814d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:01.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f463dbf7640 1 --2- 172.21.15.105:0/240312086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4638156630 0x7f4638158fa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f463000ca70 tx=0x7f463000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:01.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f462dffb640 1 -- 172.21.15.105:0/240312086 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463000abd0 con 0x7f4638156630 2023-11-28T18:24:01.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f462dffb640 1 -- 172.21.15.105:0/240312086 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463000ad30 con 0x7f4638156630 2023-11-28T18:24:01.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f462dffb640 1 -- 172.21.15.105:0/240312086 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630012600 con 0x7f4638156630 2023-11-28T18:24:01.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f463f65a640 1 -- 172.21.15.105:0/240312086 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4638099ca0 con 0x7f4638156630 2023-11-28T18:24:01.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f463f65a640 1 -- 172.21.15.105:0/240312086 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463809a260 con 0x7f4638156630 2023-11-28T18:24:01.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f463f65a640 1 -- 172.21.15.105:0/240312086 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f463809a450 con 0x7f4638156630 2023-11-28T18:24:01.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.767+0000 7f462dffb640 1 -- 172.21.15.105:0/240312086 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f4630007070 con 0x7f4638156630 2023-11-28T18:24:01.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.771+0000 7f462dffb640 1 --2- 172.21.15.105:0/240312086 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f461c0668d0 0x7f461c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:01.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.771+0000 7f462dffb640 1 -- 172.21.15.105:0/240312086 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f46300845d0 con 0x7f4638156630 2023-11-28T18:24:01.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.771+0000 7f463d3f6640 1 --2- 172.21.15.105:0/240312086 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f461c0668d0 0x7f461c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:01.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.771+0000 7f463d3f6640 1 --2- 172.21.15.105:0/240312086 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f461c0668d0 0x7f461c068d90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f46340af000 tx=0x7f46340a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:01.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.775+0000 7f462dffb640 1 -- 172.21.15.105:0/240312086 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4630051540 con 0x7f4638156630 2023-11-28T18:24:01.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.927+0000 7f463f65a640 1 -- 172.21.15.105:0/240312086 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4638067570 con 0x7f4638156630 2023-11-28T18:24:01.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.927+0000 7f462dffb640 1 -- 172.21.15.105:0/240312086 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7f4630017090 con 0x7f4638156630 2023-11-28T18:24:01.931 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:01.931 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":24,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:23:38.458954+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:24:01.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.927+0000 7f463f65a640 1 -- 172.21.15.105:0/240312086 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f461c0668d0 msgr2=0x7f461c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:01.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.927+0000 7f463f65a640 1 --2- 172.21.15.105:0/240312086 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f461c0668d0 0x7f461c068d90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f46340af000 tx=0x7f46340a7000 comp rx=0 tx=0).stop 2023-11-28T18:24:01.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.927+0000 7f463f65a640 1 -- 172.21.15.105:0/240312086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4638156630 msgr2=0x7f4638158fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:01.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.927+0000 7f463f65a640 1 --2- 172.21.15.105:0/240312086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4638156630 0x7f4638158fa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f463000ca70 tx=0x7f463000cf30 comp rx=0 tx=0).stop 2023-11-28T18:24:01.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.927+0000 7f463f65a640 1 -- 172.21.15.105:0/240312086 shutdown_connections 2023-11-28T18:24:01.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.927+0000 7f463f65a640 1 --2- 172.21.15.105:0/240312086 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f461c0668d0 0x7f461c068d90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.927+0000 7f463f65a640 1 --2- 172.21.15.105:0/240312086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4638156630 0x7f4638158fa0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.927+0000 7f463f65a640 1 --2- 172.21.15.105:0/240312086 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f463815b030 0x7f463814d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.927+0000 7f463f65a640 1 --2- 172.21.15.105:0/240312086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46380b4290 0x7f46380aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:01.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.927+0000 7f463f65a640 1 -- 172.21.15.105:0/240312086 >> 172.21.15.105:0/240312086 conn(0x7f46380ceb90 msgr2=0x7f46380709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:01.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.931+0000 7f463f65a640 1 -- 172.21.15.105:0/240312086 shutdown_connections 2023-11-28T18:24:01.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:01.931+0000 7f463f65a640 1 -- 172.21.15.105:0/240312086 wait complete. 2023-11-28T18:24:01.952 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:24:02.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.087+0000 7f2799ead640 1 Processor -- start 2023-11-28T18:24:02.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.087+0000 7f2799ead640 1 -- start start 2023-11-28T18:24:02.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.087+0000 7f2799ead640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f279415cef0 0x7f27940d1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.087+0000 7f2799ead640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f27940cfb30 0x7f2794166fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.087+0000 7f2799ead640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27940ce840 con 0x7f27940d1b40 2023-11-28T18:24:02.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.087+0000 7f2799ead640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27940b4780 con 0x7f27940cfb30 2023-11-28T18:24:02.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.087+0000 7f2799ead640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27940cd990 con 0x7f279415cef0 2023-11-28T18:24:02.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.087+0000 7f27996ac640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f27940cfb30 0x7f2794166fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.087+0000 7f27996ac640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f27940cfb30 0x7f2794166fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36316/0 (socket says 172.21.15.105:36316) 2023-11-28T18:24:02.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.087+0000 7f2798eab640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f279415cef0 0x7f27940d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.087+0000 7f278bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f27940d1b40 0x7f27940cf420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:35776/0 (socket says 172.21.15.105:35776) 2023-11-28T18:24:02.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.087+0000 7f27996ac640 1 -- 172.21.15.105:0/3733135624 learned_addr learned my addr 172.21.15.105:0/3733135624 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.087+0000 7f27996ac640 1 -- 172.21.15.105:0/3733135624 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f279415cef0 msgr2=0x7f27940d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.087+0000 7f27996ac640 1 --2- 172.21.15.105:0/3733135624 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f279415cef0 0x7f27940d1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.087+0000 7f27996ac640 1 -- 172.21.15.105:0/3733135624 >> v1:172.21.15.148:6789/0 conn(0x7f27940d1b40 legacy=0x7f27940cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.087+0000 7f27996ac640 1 -- 172.21.15.105:0/3733135624 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27940cecd0 con 0x7f27940cfb30 2023-11-28T18:24:02.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.091+0000 7f27996ac640 1 --2- 172.21.15.105:0/3733135624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27940cfb30 0x7f2794166fd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f278c009e50 tx=0x7f278c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d1b5db5caef398 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:02.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.091+0000 7f278b7fe640 1 -- 172.21.15.105:0/3733135624 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c0a5810 con 0x7f27940cfb30 2023-11-28T18:24:02.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.091+0000 7f278b7fe640 1 -- 172.21.15.105:0/3733135624 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278c0a5970 con 0x7f27940cfb30 2023-11-28T18:24:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.091+0000 7f278b7fe640 1 -- 172.21.15.105:0/3733135624 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c0a5d00 con 0x7f27940cfb30 2023-11-28T18:24:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.091+0000 7f2799ead640 1 -- 172.21.15.105:0/3733135624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27940cfb30 msgr2=0x7f2794166fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.091+0000 7f2799ead640 1 --2- 172.21.15.105:0/3733135624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27940cfb30 0x7f2794166fd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f278c009e50 tx=0x7f278c01c950 comp rx=0 tx=0).stop 2023-11-28T18:24:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.091+0000 7f2799ead640 1 -- 172.21.15.105:0/3733135624 shutdown_connections 2023-11-28T18:24:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.091+0000 7f2799ead640 1 --2- 172.21.15.105:0/3733135624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27940cfb30 0x7f2794166fd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.091+0000 7f2799ead640 1 --2- 172.21.15.105:0/3733135624 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f279415cef0 0x7f27940d1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.091+0000 7f2799ead640 1 -- 172.21.15.105:0/3733135624 >> 172.21.15.105:0/3733135624 conn(0x7f27940d2e10 msgr2=0x7f27940d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.091+0000 7f2799ead640 1 -- 172.21.15.105:0/3733135624 shutdown_connections 2023-11-28T18:24:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.091+0000 7f2799ead640 1 -- 172.21.15.105:0/3733135624 wait complete. 2023-11-28T18:24:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f2799ead640 1 Processor -- start 2023-11-28T18:24:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f2799ead640 1 -- start start 2023-11-28T18:24:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f2799ead640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27940cfb30 0x7f2794153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f2799ead640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27940d1b40 0x7f2794154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f2799ead640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f279415cef0 0x7f279415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f2799ead640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27940b44a0 con 0x7f27940d1b40 2023-11-28T18:24:02.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f2799ead640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279415d780 con 0x7f279415cef0 2023-11-28T18:24:02.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f2799ead640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279415da60 con 0x7f27940cfb30 2023-11-28T18:24:02.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f278bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27940d1b40 0x7f2794154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f278bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27940d1b40 0x7f2794154260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36326/0 (socket says 172.21.15.105:36326) 2023-11-28T18:24:02.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f278bfff640 1 -- 172.21.15.105:0/1891055344 learned_addr learned my addr 172.21.15.105:0/1891055344 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:02.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f27996ac640 1 --2- 172.21.15.105:0/1891055344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f279415cef0 0x7f279415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f2798eab640 1 --2- 172.21.15.105:0/1891055344 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27940cfb30 0x7f2794153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f278bfff640 1 -- 172.21.15.105:0/1891055344 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27940cfb30 msgr2=0x7f2794153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:02.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f278bfff640 1 --2- 172.21.15.105:0/1891055344 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27940cfb30 0x7f2794153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f278bfff640 1 -- 172.21.15.105:0/1891055344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f279415cef0 msgr2=0x7f279415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:02.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f278bfff640 1 --2- 172.21.15.105:0/1891055344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f279415cef0 0x7f279415ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f278bfff640 1 -- 172.21.15.105:0/1891055344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278c009b30 con 0x7f27940d1b40 2023-11-28T18:24:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f2798eab640 1 --2- 172.21.15.105:0/1891055344 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27940cfb30 0x7f2794153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f278bfff640 1 --2- 172.21.15.105:0/1891055344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27940d1b40 0x7f2794154260 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f278400c500 tx=0x7f278400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f27897fa640 1 -- 172.21.15.105:0/1891055344 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2784007500 con 0x7f27940d1b40 2023-11-28T18:24:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f27897fa640 1 -- 172.21.15.105:0/1891055344 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2784007660 con 0x7f27940d1b40 2023-11-28T18:24:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f2799ead640 1 -- 172.21.15.105:0/1891055344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f279415dd40 con 0x7f27940d1b40 2023-11-28T18:24:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f27897fa640 1 -- 172.21.15.105:0/1891055344 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2784004a10 con 0x7f27940d1b40 2023-11-28T18:24:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.095+0000 7f2799ead640 1 -- 172.21.15.105:0/1891055344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f279415e300 con 0x7f27940d1b40 2023-11-28T18:24:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.099+0000 7f2799ead640 1 -- 172.21.15.105:0/1891055344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27940d03c0 con 0x7f27940d1b40 2023-11-28T18:24:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.099+0000 7f27897fa640 1 -- 172.21.15.105:0/1891055344 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f2784004b70 con 0x7f27940d1b40 2023-11-28T18:24:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.099+0000 7f27897fa640 1 --2- 172.21.15.105:0/1891055344 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27800668d0 0x7f2780068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.099+0000 7f27897fa640 1 -- 172.21.15.105:0/1891055344 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f27840838e0 con 0x7f27940d1b40 2023-11-28T18:24:02.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.099+0000 7f2798eab640 1 --2- 172.21.15.105:0/1891055344 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27800668d0 0x7f2780068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.099+0000 7f2798eab640 1 --2- 172.21.15.105:0/1891055344 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27800668d0 0x7f2780068d90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2790009bb0 tx=0x7f2790008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:02.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.103+0000 7f27897fa640 1 -- 172.21.15.105:0/1891055344 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f27840508d0 con 0x7f27940d1b40 2023-11-28T18:24:02.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.283+0000 7f2799ead640 1 -- 172.21.15.105:0/1891055344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f279415e4c0 con 0x7f27940d1b40 2023-11-28T18:24:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.287+0000 7f27897fa640 1 -- 172.21.15.105:0/1891055344 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7f2784018090 con 0x7f27940d1b40 2023-11-28T18:24:02.289 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:02.289 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/390529893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":390529893},{"type":"v1","addr":"172.21.15.148:6835","nonce":390529893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3924571887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3924571887},{"type":"v1","addr":"172.21.15.148:6833","nonce":3924571887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3271971772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3271971772},{"type":"v1","addr":"172.21.15.105:6835","nonce":3271971772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:23:33.288743+0000","modified":"2023-11-28T18:23:50.425976+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2504263238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2504263238},{"type":"v1","addr":"172.21.15.105:6837","nonce":2504263238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T18:24:02.290 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 9 2023-11-28T18:24:02.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.287+0000 7f2799ead640 1 -- 172.21.15.105:0/1891055344 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27800668d0 msgr2=0x7f2780068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:02.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.287+0000 7f2799ead640 1 --2- 172.21.15.105:0/1891055344 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27800668d0 0x7f2780068d90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2790009bb0 tx=0x7f2790008040 comp rx=0 tx=0).stop 2023-11-28T18:24:02.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.287+0000 7f2799ead640 1 -- 172.21.15.105:0/1891055344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27940d1b40 msgr2=0x7f2794154260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:02.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.287+0000 7f2799ead640 1 --2- 172.21.15.105:0/1891055344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27940d1b40 0x7f2794154260 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f278400c500 tx=0x7f278400c9c0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.287+0000 7f2799ead640 1 -- 172.21.15.105:0/1891055344 shutdown_connections 2023-11-28T18:24:02.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.287+0000 7f2799ead640 1 --2- 172.21.15.105:0/1891055344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f279415cef0 0x7f279415ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.287+0000 7f2799ead640 1 --2- 172.21.15.105:0/1891055344 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27800668d0 0x7f2780068d90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.287+0000 7f2799ead640 1 --2- 172.21.15.105:0/1891055344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27940d1b40 0x7f2794154260 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.287+0000 7f2799ead640 1 --2- 172.21.15.105:0/1891055344 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27940cfb30 0x7f2794153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.287+0000 7f2799ead640 1 -- 172.21.15.105:0/1891055344 >> 172.21.15.105:0/1891055344 conn(0x7f27940d2e10 msgr2=0x7f27940afa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:02.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.287+0000 7f2799ead640 1 -- 172.21.15.105:0/1891055344 shutdown_connections 2023-11-28T18:24:02.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.287+0000 7f2799ead640 1 -- 172.21.15.105:0/1891055344 wait complete. 2023-11-28T18:24:02.713 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:23:33.288743+0000', 'modified': '2023-11-28T18:23:50.425976+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6837/2504263238', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 2504263238}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 2504263238}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:24:02.714 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T18:24:02.714 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T18:24:02.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.851+0000 7fd9259c4640 1 Processor -- start 2023-11-28T18:24:02.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.851+0000 7fd9259c4640 1 -- start start 2023-11-28T18:24:02.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.851+0000 7fd9259c4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd920159f40 0x7fd9200caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.851+0000 7fd9259c4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd9200cc760 0x7fd9200ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.851+0000 7fd9259c4640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd92006bf50 con 0x7fd9200ad680 2023-11-28T18:24:02.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.851+0000 7fd9259c4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9200ccbc0 con 0x7fd9200cc760 2023-11-28T18:24:02.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.851+0000 7fd9259c4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd92015a320 con 0x7fd920159f40 2023-11-28T18:24:02.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.851+0000 7fd9167fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd9200cc760 0x7fd9200ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.851+0000 7fd9167fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd9200cc760 0x7fd9200ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36342/0 (socket says 172.21.15.105:36342) 2023-11-28T18:24:02.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.851+0000 7fd9167fc640 1 -- 172.21.15.105:0/3350330961 learned_addr learned my addr 172.21.15.105:0/3350330961 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:02.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.851+0000 7fd91effd640 1 --2- 172.21.15.105:0/3350330961 >> v2:172.21.15.148:3300/0 conn(0x7fd920159f40 0x7fd9200caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.851+0000 7fd9167fc640 1 -- 172.21.15.105:0/3350330961 >> v2:172.21.15.148:3300/0 conn(0x7fd920159f40 msgr2=0x7fd9200caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:02.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.851+0000 7fd9167fc640 1 --2- 172.21.15.105:0/3350330961 >> v2:172.21.15.148:3300/0 conn(0x7fd920159f40 0x7fd9200caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.851+0000 7fd9167fc640 1 -- 172.21.15.105:0/3350330961 >> v1:172.21.15.105:6789/0 conn(0x7fd9200ad680 legacy=0x7fd92015da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:02.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.851+0000 7fd9167fc640 1 -- 172.21.15.105:0/3350330961 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd920071380 con 0x7fd9200cc760 2023-11-28T18:24:02.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.855+0000 7fd9167fc640 1 --2- 172.21.15.105:0/3350330961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9200cc760 0x7fd9200ad140 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd92006e500 tx=0x7fd90801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9716ba5451d6b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:02.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.855+0000 7fd91e7fc640 1 -- 172.21.15.105:0/3350330961 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9080a57d0 con 0x7fd9200cc760 2023-11-28T18:24:02.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.855+0000 7fd91e7fc640 1 -- 172.21.15.105:0/3350330961 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9080a5930 con 0x7fd9200cc760 2023-11-28T18:24:02.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.855+0000 7fd91e7fc640 1 -- 172.21.15.105:0/3350330961 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9080a5cc0 con 0x7fd9200cc760 2023-11-28T18:24:02.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.855+0000 7fd9259c4640 1 -- 172.21.15.105:0/3350330961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9200cc760 msgr2=0x7fd9200ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:02.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.855+0000 7fd9259c4640 1 --2- 172.21.15.105:0/3350330961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9200cc760 0x7fd9200ad140 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd92006e500 tx=0x7fd90801c910 comp rx=0 tx=0).stop 2023-11-28T18:24:02.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.855+0000 7fd9259c4640 1 -- 172.21.15.105:0/3350330961 shutdown_connections 2023-11-28T18:24:02.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.855+0000 7fd9259c4640 1 --2- 172.21.15.105:0/3350330961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9200cc760 0x7fd9200ad140 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.855+0000 7fd9259c4640 1 --2- 172.21.15.105:0/3350330961 >> v2:172.21.15.148:3300/0 conn(0x7fd920159f40 0x7fd9200caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.855+0000 7fd9259c4640 1 -- 172.21.15.105:0/3350330961 >> 172.21.15.105:0/3350330961 conn(0x7fd9200ceb90 msgr2=0x7fd9200cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:02.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.855+0000 7fd9259c4640 1 -- 172.21.15.105:0/3350330961 shutdown_connections 2023-11-28T18:24:02.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.855+0000 7fd9259c4640 1 -- 172.21.15.105:0/3350330961 wait complete. 2023-11-28T18:24:02.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd9259c4640 1 Processor -- start 2023-11-28T18:24:02.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd9259c4640 1 -- start start 2023-11-28T18:24:02.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd9259c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9200ad680 0x7fd9200afae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd9259c4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9200cc760 0x7fd9200af460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd9259c4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd920159f40 0x7fd9200ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd9259c4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd920157670 con 0x7fd9200cc760 2023-11-28T18:24:02.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd9259c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92015d330 con 0x7fd9200ad680 2023-11-28T18:24:02.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd9259c4640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd920144d80 con 0x7fd920159f40 2023-11-28T18:24:02.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd9167fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9200cc760 0x7fd9200af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd9167fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9200cc760 0x7fd9200af460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36352/0 (socket says 172.21.15.105:36352) 2023-11-28T18:24:02.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd9167fc640 1 -- 172.21.15.105:0/2055491996 learned_addr learned my addr 172.21.15.105:0/2055491996 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:02.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd91effd640 1 --2- 172.21.15.105:0/2055491996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9200ad680 0x7fd9200afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd91f7fe640 1 --2- 172.21.15.105:0/2055491996 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd920159f40 0x7fd9200ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd9167fc640 1 -- 172.21.15.105:0/2055491996 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd920159f40 msgr2=0x7fd9200ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:02.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd9167fc640 1 --2- 172.21.15.105:0/2055491996 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd920159f40 0x7fd9200ae700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd9167fc640 1 -- 172.21.15.105:0/2055491996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9200ad680 msgr2=0x7fd9200afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:02.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd9167fc640 1 --2- 172.21.15.105:0/2055491996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9200ad680 0x7fd9200afae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd9167fc640 1 -- 172.21.15.105:0/2055491996 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd908009b30 con 0x7fd9200cc760 2023-11-28T18:24:02.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd9167fc640 1 --2- 172.21.15.105:0/2055491996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9200cc760 0x7fd9200af460 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd908019520 tx=0x7fd9080196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:02.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd91cff9640 1 -- 172.21.15.105:0/2055491996 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9080b8670 con 0x7fd9200cc760 2023-11-28T18:24:02.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd91cff9640 1 -- 172.21.15.105:0/2055491996 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9080b87f0 con 0x7fd9200cc760 2023-11-28T18:24:02.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd9259c4640 1 -- 172.21.15.105:0/2055491996 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd92015cd70 con 0x7fd9200cc760 2023-11-28T18:24:02.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd91cff9640 1 -- 172.21.15.105:0/2055491996 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9080b8ba0 con 0x7fd9200cc760 2023-11-28T18:24:02.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.859+0000 7fd9259c4640 1 -- 172.21.15.105:0/2055491996 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd920157b00 con 0x7fd9200cc760 2023-11-28T18:24:02.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.863+0000 7fd91cff9640 1 -- 172.21.15.105:0/2055491996 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd9080b8d00 con 0x7fd9200cc760 2023-11-28T18:24:02.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.863+0000 7fd91cff9640 1 --2- 172.21.15.105:0/2055491996 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd900066900 0x7fd900068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.863+0000 7fd91cff9640 1 -- 172.21.15.105:0/2055491996 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd90811fc20 con 0x7fd9200cc760 2023-11-28T18:24:02.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.863+0000 7fd91effd640 1 --2- 172.21.15.105:0/2055491996 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd900066900 0x7fd900068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.863+0000 7fd928d36000 1 Processor -- start 2023-11-28T18:24:02.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.863+0000 7fd928d36000 1 -- start start 2023-11-28T18:24:02.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd91effd640 1 --2- 172.21.15.105:0/2055491996 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd900066900 0x7fd900068dc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd920164800 tx=0x7fd910008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:02.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd928d36000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55932d208d30 0x55932d209110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd928d36000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55932d209650 0x55932d1bbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd928d36000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d1bc500 0x55932d1bc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd928d36000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932d085110 con 0x55932d1bc500 2023-11-28T18:24:02.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd928d36000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932d0159a0 con 0x55932d208d30 2023-11-28T18:24:02.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd928d36000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55932d012ed0 con 0x55932d209650 2023-11-28T18:24:02.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd91f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d1bc500 0x55932d1bc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd91f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d1bc500 0x55932d1bc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36354/0 (socket says 172.21.15.105:36354) 2023-11-28T18:24:02.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd91f7fe640 1 -- 172.21.15.105:0/1431102891 learned_addr learned my addr 172.21.15.105:0/1431102891 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:02.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd91effd640 1 --2- 172.21.15.105:0/1431102891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55932d208d30 0x55932d209110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd9167fc640 1 --2- 172.21.15.105:0/1431102891 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55932d209650 0x55932d1bbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd91f7fe640 1 -- 172.21.15.105:0/1431102891 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55932d209650 msgr2=0x55932d1bbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:02.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd91f7fe640 1 --2- 172.21.15.105:0/1431102891 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55932d209650 0x55932d1bbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd91f7fe640 1 -- 172.21.15.105:0/1431102891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55932d208d30 msgr2=0x55932d209110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:02.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd91f7fe640 1 --2- 172.21.15.105:0/1431102891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55932d208d30 0x55932d209110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd91f7fe640 1 -- 172.21.15.105:0/1431102891 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55932cd06890 con 0x55932d1bc500 2023-11-28T18:24:02.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd9167fc640 1 --2- 172.21.15.105:0/1431102891 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55932d209650 0x55932d1bbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:02.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd91f7fe640 1 --2- 172.21.15.105:0/1431102891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d1bc500 0x55932d1bc8e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd9180014d0 tx=0x7fd918001990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a383bdc02e30ce6a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:02.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd9157fa640 1 -- 172.21.15.105:0/1431102891 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9180052e0 con 0x55932d1bc500 2023-11-28T18:24:02.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd9157fa640 1 -- 172.21.15.105:0/1431102891 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd918005460 con 0x55932d1bc500 2023-11-28T18:24:02.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd9157fa640 1 -- 172.21.15.105:0/1431102891 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918003840 con 0x55932d1bc500 2023-11-28T18:24:02.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd928d36000 1 -- 172.21.15.105:0/1431102891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d1bc500 msgr2=0x55932d1bc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:02.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd928d36000 1 --2- 172.21.15.105:0/1431102891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d1bc500 0x55932d1bc8e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd9180014d0 tx=0x7fd918001990 comp rx=0 tx=0).stop 2023-11-28T18:24:02.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd928d36000 1 -- 172.21.15.105:0/1431102891 shutdown_connections 2023-11-28T18:24:02.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd928d36000 1 --2- 172.21.15.105:0/1431102891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d1bc500 0x55932d1bc8e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd928d36000 1 --2- 172.21.15.105:0/1431102891 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55932d209650 0x55932d1bbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd928d36000 1 --2- 172.21.15.105:0/1431102891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55932d208d30 0x55932d209110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd928d36000 1 -- 172.21.15.105:0/1431102891 >> 172.21.15.105:0/1431102891 conn(0x55932d1e98c0 msgr2=0x55932d1ebcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:02.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd928d36000 1 -- 172.21.15.105:0/1431102891 shutdown_connections 2023-11-28T18:24:02.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd928d36000 1 -- 172.21.15.105:0/1431102891 wait complete. 2023-11-28T18:24:02.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd928d36000 1 -- start start 2023-11-28T18:24:02.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.867+0000 7fd928d36000 1 Processor -- start 2023-11-28T18:24:02.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd928d36000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d1bc500 0x55932d1c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd928d36000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55932d208d30 0x55932d1c2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd928d36000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55932d209650 0x55932d1c2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd928d36000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932d00e7c0 con 0x55932d1bc500 2023-11-28T18:24:02.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd928d36000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932cfc6bf0 con 0x55932d208d30 2023-11-28T18:24:02.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd928d36000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55932cfcca90 con 0x55932d209650 2023-11-28T18:24:02.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd928d36000 10 client.0 _ll_register_callbacks cb 0x55932d1a9e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:24:02.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd91effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d1bc500 0x55932d1c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd91effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d1bc500 0x55932d1c24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36366/0 (socket says 172.21.15.105:36366) 2023-11-28T18:24:02.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd91effd640 1 -- 172.21.15.105:0/3156423020 learned_addr learned my addr 172.21.15.105:0/3156423020 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:02.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd9167fc640 1 --2- 172.21.15.105:0/3156423020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55932d208d30 0x55932d1c2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd91f7fe640 1 --2- 172.21.15.105:0/3156423020 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55932d209650 0x55932d1c2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd91effd640 1 -- 172.21.15.105:0/3156423020 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55932d209650 msgr2=0x55932d1c2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:02.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd91effd640 1 --2- 172.21.15.105:0/3156423020 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55932d209650 0x55932d1c2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd91effd640 1 -- 172.21.15.105:0/3156423020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55932d208d30 msgr2=0x55932d1c2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:02.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd91effd640 1 --2- 172.21.15.105:0/3156423020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55932d208d30 0x55932d1c2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd91effd640 1 -- 172.21.15.105:0/3156423020 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91800f040 con 0x55932d1bc500 2023-11-28T18:24:02.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd91effd640 1 --2- 172.21.15.105:0/3156423020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d1bc500 0x55932d1c24e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd9100156a0 tx=0x7fd9100046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f236dc36168f1432 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:02.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd8fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:24:02.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd8fe7fc640 1 -- 172.21.15.105:0/3156423020 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910014660 con 0x55932d1bc500 2023-11-28T18:24:02.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd8fe7fc640 1 -- 172.21.15.105:0/3156423020 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd9100147c0 con 0x55932d1bc500 2023-11-28T18:24:02.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd928d36000 1 -- 172.21.15.105:0/3156423020 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55932cfd0470 con 0x55932d1bc500 2023-11-28T18:24:02.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd8fe7fc640 1 -- 172.21.15.105:0/3156423020 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd910014980 con 0x55932d1bc500 2023-11-28T18:24:02.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd928d36000 10 client.4794 fetch_fsmap learned FSMap version 9 2023-11-28T18:24:02.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd928d36000 1 -- 172.21.15.105:0/3156423020 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55932cfcf190 con 0x55932d1bc500 2023-11-28T18:24:02.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd8fe7fc640 1 -- 172.21.15.105:0/3156423020 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55932cfd0470 con 0x55932d1bc500 2023-11-28T18:24:02.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd8fe7fc640 1 -- 172.21.15.105:0/3156423020 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fd910028020 con 0x55932d1bc500 2023-11-28T18:24:02.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd928d36000 10 client.4794 fetch_fsmap finished waiting for FSMap version 9 2023-11-28T18:24:02.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd928d36000 10 client.4794 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:24:02.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd928d36000 20 client.4794 populate_metadata read hostname 'smithi105' 2023-11-28T18:24:02.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd928d36000 1 --2- 172.21.15.105:0/3156423020 >> [v2:172.21.15.105:6836/2504263238,v1:172.21.15.105:6837/2504263238] conn(0x55932d1c3970 0x55932d21aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd928d36000 4 client.4794 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:24:02.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd928d36000 1 -- 172.21.15.105:0/3156423020 --> [v2:172.21.15.105:6836/2504263238,v1:172.21.15.105:6837/2504263238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55932cfd3b50 con 0x55932d1c3970 2023-11-28T18:24:02.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd9167fc640 1 --2- 172.21.15.105:0/3156423020 >> [v2:172.21.15.105:6836/2504263238,v1:172.21.15.105:6837/2504263238] conn(0x55932d1c3970 0x55932d21aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd9167fc640 1 --2- 172.21.15.105:0/3156423020 >> [v2:172.21.15.105:6836/2504263238,v1:172.21.15.105:6837/2504263238] conn(0x55932d1c3970 0x55932d21aff0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19c8d9b93b793839 server_cookie=fe94d2ffbf031d9 in_seq=0 out_seq=0 2023-11-28T18:24:02.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.871+0000 7fd8fe7fc640 10 client.4794 ms_handle_connect on v2:172.21.15.105:6836/2504263238 2023-11-28T18:24:02.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.875+0000 7fd8fe7fc640 1 -- 172.21.15.105:0/3156423020 <== mds.0 v2:172.21.15.105:6836/2504263238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55932cfd3b50 con 0x55932d1c3970 2023-11-28T18:24:02.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.875+0000 7fd8fe7fc640 10 client.4794 handle_command_reply: tid=0 2023-11-28T18:24:02.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.875+0000 7fd928d36000 1 client.4794 shutdown 2023-11-28T18:24:02.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.875+0000 7fd928d36000 1 -- 172.21.15.105:0/3156423020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d1bc500 msgr2=0x55932d1c24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:02.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.875+0000 7fd928d36000 1 --2- 172.21.15.105:0/3156423020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d1bc500 0x55932d1c24e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd9100156a0 tx=0x7fd9100046d0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.875+0000 7fd928d36000 1 -- 172.21.15.105:0/3156423020 shutdown_connections 2023-11-28T18:24:02.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.875+0000 7fd928d36000 1 --2- 172.21.15.105:0/3156423020 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55932d209650 0x55932d1c2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.875+0000 7fd928d36000 1 --2- 172.21.15.105:0/3156423020 >> [v2:172.21.15.105:6836/2504263238,v1:172.21.15.105:6837/2504263238] conn(0x55932d1c3970 0x55932d21aff0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.875+0000 7fd928d36000 1 --2- 172.21.15.105:0/3156423020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55932d208d30 0x55932d1c2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.875+0000 7fd928d36000 1 --2- 172.21.15.105:0/3156423020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d1bc500 0x55932d1c24e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.875+0000 7fd928d36000 1 -- 172.21.15.105:0/3156423020 >> 172.21.15.105:0/3156423020 conn(0x55932d1e98c0 msgr2=0x55932d1bb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:02.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.875+0000 7fd8fe7fc640 0 client.4794 ms_handle_reset on v2:172.21.15.105:6836/2504263238 2023-11-28T18:24:02.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.875+0000 7fd928d36000 1 -- 172.21.15.105:0/3156423020 shutdown_connections 2023-11-28T18:24:02.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.875+0000 7fd928d36000 1 -- 172.21.15.105:0/3156423020 wait complete. 2023-11-28T18:24:02.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.875+0000 7fd928d36000 20 client.4794 trim_cache size 0 max 16384 2023-11-28T18:24:02.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd928d36000 1 Processor -- start 2023-11-28T18:24:02.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd928d36000 1 -- start start 2023-11-28T18:24:02.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd928d36000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55932d1bc500 0x55932d1e83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd928d36000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d208d30 0x55932d1e88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd928d36000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55932d209650 0x55932d21aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd928d36000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932ccc7940 con 0x55932d208d30 2023-11-28T18:24:02.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd928d36000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932cfe3b70 con 0x55932d1bc500 2023-11-28T18:24:02.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd928d36000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55932d011320 con 0x55932d209650 2023-11-28T18:24:02.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd9167fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d208d30 0x55932d1e88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd9167fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d208d30 0x55932d1e88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36380/0 (socket says 172.21.15.105:36380) 2023-11-28T18:24:02.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd9167fc640 1 -- 172.21.15.105:0/3220600919 learned_addr learned my addr 172.21.15.105:0/3220600919 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:02.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd91effd640 1 --2- 172.21.15.105:0/3220600919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55932d1bc500 0x55932d1e83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd91f7fe640 1 --2- 172.21.15.105:0/3220600919 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55932d209650 0x55932d21aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd9167fc640 1 -- 172.21.15.105:0/3220600919 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55932d209650 msgr2=0x55932d21aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:02.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd9167fc640 1 --2- 172.21.15.105:0/3220600919 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55932d209650 0x55932d21aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd9167fc640 1 -- 172.21.15.105:0/3220600919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55932d1bc500 msgr2=0x55932d1e83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:02.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd9167fc640 1 --2- 172.21.15.105:0/3220600919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55932d1bc500 0x55932d1e83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd9167fc640 1 -- 172.21.15.105:0/3220600919 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55932d1bc370 con 0x55932d208d30 2023-11-28T18:24:02.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd9167fc640 1 --2- 172.21.15.105:0/3220600919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d208d30 0x55932d1e88e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd9080f07c0 tx=0x7fd9080ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6653ce69f6db99d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:02.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd915ffb640 1 -- 172.21.15.105:0/3220600919 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd908108170 con 0x55932d208d30 2023-11-28T18:24:02.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd915ffb640 1 -- 172.21.15.105:0/3220600919 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9080f8b30 con 0x55932d208d30 2023-11-28T18:24:02.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd915ffb640 1 -- 172.21.15.105:0/3220600919 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9080eebd0 con 0x55932d208d30 2023-11-28T18:24:02.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd928d36000 1 -- 172.21.15.105:0/3220600919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d208d30 msgr2=0x55932d1e88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:02.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd928d36000 1 --2- 172.21.15.105:0/3220600919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d208d30 0x55932d1e88e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd9080f07c0 tx=0x7fd9080ba040 comp rx=0 tx=0).stop 2023-11-28T18:24:02.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd928d36000 1 -- 172.21.15.105:0/3220600919 shutdown_connections 2023-11-28T18:24:02.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd928d36000 1 --2- 172.21.15.105:0/3220600919 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55932d209650 0x55932d21aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd928d36000 1 --2- 172.21.15.105:0/3220600919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d208d30 0x55932d1e88e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd928d36000 1 --2- 172.21.15.105:0/3220600919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55932d1bc500 0x55932d1e83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd928d36000 1 -- 172.21.15.105:0/3220600919 >> 172.21.15.105:0/3220600919 conn(0x55932d1e98c0 msgr2=0x55932d1ba720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:02.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd928d36000 1 -- 172.21.15.105:0/3220600919 shutdown_connections 2023-11-28T18:24:02.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.891+0000 7fd928d36000 1 -- 172.21.15.105:0/3220600919 wait complete. 2023-11-28T18:24:02.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd928d36000 1 -- start start 2023-11-28T18:24:02.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd928d36000 1 Processor -- start 2023-11-28T18:24:02.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd928d36000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55932d1bc500 0x55932d269630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd928d36000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d208d30 0x55932d269b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd928d36000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55932d209650 0x55932d224a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd928d36000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932d00bfd0 con 0x55932d208d30 2023-11-28T18:24:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd928d36000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932cfcd8f0 con 0x55932d209650 2023-11-28T18:24:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd928d36000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55932d1bfdd0 con 0x55932d1bc500 2023-11-28T18:24:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd928d36000 10 client.0 _ll_register_callbacks cb 0x55932d085060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:24:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd9167fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d208d30 0x55932d269b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd9167fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d208d30 0x55932d269b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36382/0 (socket says 172.21.15.105:36382) 2023-11-28T18:24:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd9167fc640 1 -- 172.21.15.105:0/3712001244 learned_addr learned my addr 172.21.15.105:0/3712001244 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd91effd640 1 --2- 172.21.15.105:0/3712001244 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55932d1bc500 0x55932d269630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd91f7fe640 1 --2- 172.21.15.105:0/3712001244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55932d209650 0x55932d224a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd9167fc640 1 -- 172.21.15.105:0/3712001244 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55932d1bc500 msgr2=0x55932d269630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd9167fc640 1 --2- 172.21.15.105:0/3712001244 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55932d1bc500 0x55932d269630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd9167fc640 1 -- 172.21.15.105:0/3712001244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55932d209650 msgr2=0x55932d224a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd9167fc640 1 --2- 172.21.15.105:0/3712001244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55932d209650 0x55932d224a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd9167fc640 1 -- 172.21.15.105:0/3712001244 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd920157b00 con 0x55932d208d30 2023-11-28T18:24:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd91f7fe640 1 --2- 172.21.15.105:0/3712001244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55932d209650 0x55932d224a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd9167fc640 1 --2- 172.21.15.105:0/3712001244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d208d30 0x55932d269b70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd9080f0530 tx=0x7fd908009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ee88bdc4f02fee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd8fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:24:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd8fe7fc640 1 -- 172.21.15.105:0/3712001244 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd908108170 con 0x55932d208d30 2023-11-28T18:24:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd8fe7fc640 1 -- 172.21.15.105:0/3712001244 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd9080ecf40 con 0x55932d208d30 2023-11-28T18:24:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd928d36000 1 -- 172.21.15.105:0/3712001244 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55932d00d110 con 0x55932d208d30 2023-11-28T18:24:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd8fe7fc640 1 -- 172.21.15.105:0/3712001244 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9080ed100 con 0x55932d208d30 2023-11-28T18:24:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd928d36000 10 client.4800 fetch_fsmap learned FSMap version 9 2023-11-28T18:24:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd928d36000 1 -- 172.21.15.105:0/3712001244 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55932cfb5490 con 0x55932d208d30 2023-11-28T18:24:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd8fe7fc640 1 -- 172.21.15.105:0/3712001244 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd908113020 con 0x55932d208d30 2023-11-28T18:24:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd8fe7fc640 1 -- 172.21.15.105:0/3712001244 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fd908005d20 con 0x55932d208d30 2023-11-28T18:24:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd928d36000 10 client.4800 fetch_fsmap finished waiting for FSMap version 9 2023-11-28T18:24:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd928d36000 10 client.4800 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:24:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd928d36000 20 client.4800 populate_metadata read hostname 'smithi105' 2023-11-28T18:24:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd928d36000 1 --2- 172.21.15.105:0/3712001244 >> [v2:172.21.15.105:6836/2504263238,v1:172.21.15.105:6837/2504263238] conn(0x55932d224fb0 0x55932d225390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd928d36000 4 client.4800 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:24:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd928d36000 1 -- 172.21.15.105:0/3712001244 --> [v2:172.21.15.105:6836/2504263238,v1:172.21.15.105:6837/2504263238] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55932d03b650 con 0x55932d224fb0 2023-11-28T18:24:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd91effd640 1 --2- 172.21.15.105:0/3712001244 >> [v2:172.21.15.105:6836/2504263238,v1:172.21.15.105:6837/2504263238] conn(0x55932d224fb0 0x55932d225390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd91effd640 1 --2- 172.21.15.105:0/3712001244 >> [v2:172.21.15.105:6836/2504263238,v1:172.21.15.105:6837/2504263238] conn(0x55932d224fb0 0x55932d225390 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4ce199b9ab0afb0 server_cookie=e6505302adb6049a in_seq=0 out_seq=0 2023-11-28T18:24:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.895+0000 7fd8fe7fc640 10 client.4800 ms_handle_connect on v2:172.21.15.105:6836/2504263238 2023-11-28T18:24:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd8fe7fc640 1 -- 172.21.15.105:0/3712001244 <== mds.0 v2:172.21.15.105:6836/2504263238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55932d03b650 con 0x55932d224fb0 2023-11-28T18:24:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd8fe7fc640 10 client.4800 handle_command_reply: tid=0 2023-11-28T18:24:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd928d36000 1 client.4800 shutdown 2023-11-28T18:24:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd928d36000 1 -- 172.21.15.105:0/3712001244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d208d30 msgr2=0x55932d269b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd928d36000 1 --2- 172.21.15.105:0/3712001244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d208d30 0x55932d269b70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd9080f0530 tx=0x7fd908009040 comp rx=0 tx=0).stop 2023-11-28T18:24:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd928d36000 1 -- 172.21.15.105:0/3712001244 shutdown_connections 2023-11-28T18:24:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd928d36000 1 --2- 172.21.15.105:0/3712001244 >> [v2:172.21.15.105:6836/2504263238,v1:172.21.15.105:6837/2504263238] conn(0x55932d224fb0 0x55932d225390 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd928d36000 1 --2- 172.21.15.105:0/3712001244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55932d209650 0x55932d224a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd928d36000 1 --2- 172.21.15.105:0/3712001244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55932d208d30 0x55932d269b70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd928d36000 1 --2- 172.21.15.105:0/3712001244 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55932d1bc500 0x55932d269630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd8fe7fc640 0 client.4800 ms_handle_reset on v2:172.21.15.105:6836/2504263238 2023-11-28T18:24:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd928d36000 1 -- 172.21.15.105:0/3712001244 >> 172.21.15.105:0/3712001244 conn(0x55932d1e98c0 msgr2=0x55932d1bafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd928d36000 1 -- 172.21.15.105:0/3712001244 shutdown_connections 2023-11-28T18:24:02.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd928d36000 1 -- 172.21.15.105:0/3712001244 wait complete. 2023-11-28T18:24:02.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd928d36000 20 client.4800 trim_cache size 0 max 16384 2023-11-28T18:24:02.907 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:24:02.907 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:24:02.907 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2023-11-28T18:24:02.907 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 4287, 2023-11-28T18:24:02.907 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:24:02.907 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:24:02.908 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:24:02.908 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 28.865675583000002, 2023-11-28T18:24:02.908 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 9, 2023-11-28T18:24:02.908 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 24, 2023-11-28T18:24:02.908 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 24, 2023-11-28T18:24:02.908 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 36.046746808999998 2023-11-28T18:24:02.908 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:24:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd9259c4640 1 -- 172.21.15.105:0/2055491996 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd900066900 msgr2=0x7fd900068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd9259c4640 1 --2- 172.21.15.105:0/2055491996 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd900066900 0x7fd900068dc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd920164800 tx=0x7fd910008040 comp rx=0 tx=0).stop 2023-11-28T18:24:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd9259c4640 1 -- 172.21.15.105:0/2055491996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9200cc760 msgr2=0x7fd9200af460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd9259c4640 1 --2- 172.21.15.105:0/2055491996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9200cc760 0x7fd9200af460 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd908019520 tx=0x7fd9080196b0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd9259c4640 1 -- 172.21.15.105:0/2055491996 shutdown_connections 2023-11-28T18:24:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd9259c4640 1 --2- 172.21.15.105:0/2055491996 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd920159f40 0x7fd9200ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd9259c4640 1 --2- 172.21.15.105:0/2055491996 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd900066900 0x7fd900068dc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd9259c4640 1 --2- 172.21.15.105:0/2055491996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9200cc760 0x7fd9200af460 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd9259c4640 1 --2- 172.21.15.105:0/2055491996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9200ad680 0x7fd9200afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd9259c4640 1 -- 172.21.15.105:0/2055491996 >> 172.21.15.105:0/2055491996 conn(0x7fd9200ceb90 msgr2=0x7fd920151ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd9259c4640 1 -- 172.21.15.105:0/2055491996 shutdown_connections 2023-11-28T18:24:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:02.899+0000 7fd9259c4640 1 -- 172.21.15.105:0/2055491996 wait complete. 2023-11-28T18:24:02.921 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-11-28T18:24:02.931 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-11-28T18:24:02.931 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-28T18:24:02.931 INFO:tasks.ceph_fuse:client.0 config is {} 2023-11-28T18:24:02.932 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T18:24:02.932 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T18:24:02.932 INFO:tasks.ceph_fuse:client.2 config is {} 2023-11-28T18:24:02.932 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T18:24:02.932 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T18:24:02.932 INFO:tasks.ceph_fuse:client.1 config is {} 2023-11-28T18:24:02.932 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T18:24:02.932 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T18:24:02.932 INFO:tasks.ceph_fuse:client.3 config is {} 2023-11-28T18:24:02.932 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T18:24:02.933 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T18:24:02.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:02.933 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-11-28T18:24:02.942 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-11-28T18:24:02.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:02.943 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T18:24:03.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:03.009 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2023-11-28T18:24:03.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:03.240 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2023-11-28T18:24:03.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:03.251 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2023-11-28T18:24:03.319 INFO:teuthology.orchestra.run.smithi148.stderr:Cannot find device "ceph-brx" 2023-11-28T18:24:03.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:24:03.322 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-11-28T18:24:03.322 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T18:24:03.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.455+0000 7fe433432640 1 Processor -- start 2023-11-28T18:24:03.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.455+0000 7fe433432640 1 -- start start 2023-11-28T18:24:03.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.459+0000 7fe433432640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe42c071a10 0x7fe42c0ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:03.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.459+0000 7fe433432640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe42c070a10 0x7fe42c070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:03.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.459+0000 7fe433432640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe42c0cefa0 con 0x7fe42c15d3c0 2023-11-28T18:24:03.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.459+0000 7fe433432640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe42c0713c0 con 0x7fe42c070a10 2023-11-28T18:24:03.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.459+0000 7fe433432640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c15d7a0 con 0x7fe42c071a10 2023-11-28T18:24:03.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.459+0000 7fe4319cf640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe42c070a10 0x7fe42c070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:03.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.459+0000 7fe4319cf640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe42c070a10 0x7fe42c070e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36384/0 (socket says 172.21.15.105:36384) 2023-11-28T18:24:03.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.459+0000 7fe4319cf640 1 -- 172.21.15.105:0/3186315966 learned_addr learned my addr 172.21.15.105:0/3186315966 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:03.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.459+0000 7fe4309cd640 1 --2- 172.21.15.105:0/3186315966 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe42c071a10 0x7fe42c0ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:03.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.459+0000 7fe4319cf640 1 -- 172.21.15.105:0/3186315966 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe42c071a10 msgr2=0x7fe42c0ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:03.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.459+0000 7fe4319cf640 1 --2- 172.21.15.105:0/3186315966 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe42c071a10 0x7fe42c0ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:03.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.459+0000 7fe4319cf640 1 -- 172.21.15.105:0/3186315966 >> v1:172.21.15.148:6789/0 conn(0x7fe42c15d3c0 legacy=0x7fe42c161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:03.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.459+0000 7fe4319cf640 1 -- 172.21.15.105:0/3186315966 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c0ce9e0 con 0x7fe42c070a10 2023-11-28T18:24:03.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.459+0000 7fe4319cf640 1 --2- 172.21.15.105:0/3186315966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe42c070a10 0x7fe42c070e80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe424009e50 tx=0x7fe42401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a837176ad107750 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:03.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.459+0000 7fe423fff640 1 -- 172.21.15.105:0/3186315966 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4240a5720 con 0x7fe42c070a10 2023-11-28T18:24:03.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe423fff640 1 -- 172.21.15.105:0/3186315966 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4240a5880 con 0x7fe42c070a10 2023-11-28T18:24:03.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe423fff640 1 -- 172.21.15.105:0/3186315966 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4240a5c10 con 0x7fe42c070a10 2023-11-28T18:24:03.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe433432640 1 -- 172.21.15.105:0/3186315966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe42c070a10 msgr2=0x7fe42c070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:03.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe433432640 1 --2- 172.21.15.105:0/3186315966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe42c070a10 0x7fe42c070e80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe424009e50 tx=0x7fe42401c950 comp rx=0 tx=0).stop 2023-11-28T18:24:03.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe433432640 1 -- 172.21.15.105:0/3186315966 shutdown_connections 2023-11-28T18:24:03.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe433432640 1 --2- 172.21.15.105:0/3186315966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe42c070a10 0x7fe42c070e80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:03.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe433432640 1 --2- 172.21.15.105:0/3186315966 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe42c071a10 0x7fe42c0ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:03.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe433432640 1 -- 172.21.15.105:0/3186315966 >> 172.21.15.105:0/3186315966 conn(0x7fe42c0cf540 msgr2=0x7fe42c0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:03.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe433432640 1 -- 172.21.15.105:0/3186315966 shutdown_connections 2023-11-28T18:24:03.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe433432640 1 -- 172.21.15.105:0/3186315966 wait complete. 2023-11-28T18:24:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe433432640 1 Processor -- start 2023-11-28T18:24:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe433432640 1 -- start start 2023-11-28T18:24:03.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe433432640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe42c070a10 0x7fe42c1601e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:03.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe433432640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe42c071a10 0x7fe42c158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:03.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe433432640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe42c15d3c0 0x7fe42c156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:03.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe433432640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c15eac0 con 0x7fe42c15d3c0 2023-11-28T18:24:03.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe433432640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c1565c0 con 0x7fe42c070a10 2023-11-28T18:24:03.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe433432640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c15e470 con 0x7fe42c071a10 2023-11-28T18:24:03.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe4319cf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe42c15d3c0 0x7fe42c156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:03.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe4311ce640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe42c070a10 0x7fe42c1601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:03.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe4311ce640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe42c070a10 0x7fe42c1601e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:37116/0 (socket says 172.21.15.105:37116) 2023-11-28T18:24:03.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe4311ce640 1 -- 172.21.15.105:0/3850330806 learned_addr learned my addr 172.21.15.105:0/3850330806 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:03.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe4319cf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe42c15d3c0 0x7fe42c156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36388/0 (socket says 172.21.15.105:36388) 2023-11-28T18:24:03.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.463+0000 7fe4309cd640 1 --2- 172.21.15.105:0/3850330806 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe42c071a10 0x7fe42c158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.467+0000 7fe4319cf640 1 -- 172.21.15.105:0/3850330806 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe42c071a10 msgr2=0x7fe42c158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.467+0000 7fe4319cf640 1 --2- 172.21.15.105:0/3850330806 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe42c071a10 0x7fe42c158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.467+0000 7fe4319cf640 1 -- 172.21.15.105:0/3850330806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe42c070a10 msgr2=0x7fe42c1601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.467+0000 7fe4319cf640 1 --2- 172.21.15.105:0/3850330806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe42c070a10 0x7fe42c1601e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.467+0000 7fe4319cf640 1 -- 172.21.15.105:0/3850330806 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe424009b30 con 0x7fe42c15d3c0 2023-11-28T18:24:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.467+0000 7fe4319cf640 1 --2- 172.21.15.105:0/3850330806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe42c15d3c0 0x7fe42c156bb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe4240ab000 tx=0x7fe4240196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.467+0000 7fe421ffb640 1 -- 172.21.15.105:0/3850330806 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4240b8470 con 0x7fe42c15d3c0 2023-11-28T18:24:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.467+0000 7fe421ffb640 1 -- 172.21.15.105:0/3850330806 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4240b85d0 con 0x7fe42c15d3c0 2023-11-28T18:24:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.467+0000 7fe433432640 1 -- 172.21.15.105:0/3850330806 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe42c15e750 con 0x7fe42c15d3c0 2023-11-28T18:24:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.467+0000 7fe421ffb640 1 -- 172.21.15.105:0/3850330806 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4240b8980 con 0x7fe42c15d3c0 2023-11-28T18:24:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.467+0000 7fe433432640 1 -- 172.21.15.105:0/3850330806 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe42c1591f0 con 0x7fe42c15d3c0 2023-11-28T18:24:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.467+0000 7fe433432640 1 -- 172.21.15.105:0/3850330806 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe42c1593e0 con 0x7fe42c15d3c0 2023-11-28T18:24:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.467+0000 7fe421ffb640 1 -- 172.21.15.105:0/3850330806 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fe4240b8ae0 con 0x7fe42c15d3c0 2023-11-28T18:24:03.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.471+0000 7fe421ffb640 1 --2- 172.21.15.105:0/3850330806 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe4100668d0 0x7fe410068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:03.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.471+0000 7fe421ffb640 1 -- 172.21.15.105:0/3850330806 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe42411f6a0 con 0x7fe42c15d3c0 2023-11-28T18:24:03.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.471+0000 7fe4311ce640 1 --2- 172.21.15.105:0/3850330806 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe4100668d0 0x7fe410068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:03.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.471+0000 7fe4311ce640 1 --2- 172.21.15.105:0/3850330806 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe4100668d0 0x7fe410068d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe428001810 tx=0x7fe428009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:03.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.475+0000 7fe421ffb640 1 -- 172.21.15.105:0/3850330806 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe42c1593e0 con 0x7fe42c15d3c0 2023-11-28T18:24:03.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.655+0000 7fe433432640 1 -- 172.21.15.105:0/3850330806 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe42c1593e0 con 0x7fe42c15d3c0 2023-11-28T18:24:03.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.655+0000 7fe421ffb640 1 -- 172.21.15.105:0/3850330806 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fe42c1593e0 con 0x7fe42c15d3c0 2023-11-28T18:24:03.659 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T18:24:03.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.659+0000 7fe433432640 1 -- 172.21.15.105:0/3850330806 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe4100668d0 msgr2=0x7fe410068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:03.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.659+0000 7fe433432640 1 --2- 172.21.15.105:0/3850330806 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe4100668d0 0x7fe410068d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe428001810 tx=0x7fe428009040 comp rx=0 tx=0).stop 2023-11-28T18:24:03.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.659+0000 7fe433432640 1 -- 172.21.15.105:0/3850330806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe42c15d3c0 msgr2=0x7fe42c156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:03.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.659+0000 7fe433432640 1 --2- 172.21.15.105:0/3850330806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe42c15d3c0 0x7fe42c156bb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe4240ab000 tx=0x7fe4240196f0 comp rx=0 tx=0).stop 2023-11-28T18:24:03.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.659+0000 7fe433432640 1 -- 172.21.15.105:0/3850330806 shutdown_connections 2023-11-28T18:24:03.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.659+0000 7fe433432640 1 --2- 172.21.15.105:0/3850330806 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe4100668d0 0x7fe410068d90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:03.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.659+0000 7fe433432640 1 --2- 172.21.15.105:0/3850330806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe42c15d3c0 0x7fe42c156bb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:03.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.659+0000 7fe433432640 1 --2- 172.21.15.105:0/3850330806 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe42c071a10 0x7fe42c158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:03.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.659+0000 7fe433432640 1 --2- 172.21.15.105:0/3850330806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe42c070a10 0x7fe42c1601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:03.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.659+0000 7fe433432640 1 -- 172.21.15.105:0/3850330806 >> 172.21.15.105:0/3850330806 conn(0x7fe42c0cf540 msgr2=0x7fe42c06af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:03.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.659+0000 7fe433432640 1 -- 172.21.15.105:0/3850330806 shutdown_connections 2023-11-28T18:24:03.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:03.659+0000 7fe433432640 1 -- 172.21.15.105:0/3850330806 wait complete. 2023-11-28T18:24:03.681 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T18:24:03.681 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T18:24:03.681 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2023-11-28T18:24:03.681 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T18:24:03.682 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T18:24:03.682 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T18:24:03.682 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T18:24:03.682 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T18:24:03.682 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:24:03.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:03.683 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2023-11-28T18:24:03.692 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T18:24:03.692 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T18:24:03.692 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T18:24:03.692 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:03.692 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-11-28T18:24:03.692 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:03.693 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:24:03.693 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:24:03.693 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2023-11-28T18:24:03.693 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:24:03.693 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:24:03.693 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2023-11-28T18:24:03.693 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:24:03.693 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:24:03.693 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T18:24:03.693 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:24:03.693 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-28T18:24:03.694 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42378sec preferred_lft 42378sec 2023-11-28T18:24:03.694 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2023-11-28T18:24:03.694 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:03.694 INFO:teuthology.orchestra.run.smithi105.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-28T18:24:03.694 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 02:42:cc:d4:87:d2 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:24:03.694 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-28T18:24:03.694 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:03.695 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T18:24:03.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:03.696 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-11-28T18:24:03.696 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:24:03.696 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add name ceph-brx type bridge 2023-11-28T18:24:03.696 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr flush dev ceph-brx 2023-11-28T18:24:03.696 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set ceph-brx up 2023-11-28T18:24:03.697 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T18:24:03.697 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:24:03.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:03.806 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T18:24:03.825 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-11-28T18:24:03.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:03.828 DEBUG:teuthology.orchestra.run.smithi105:> route 2023-11-28T18:24:03.879 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2023-11-28T18:24:03.879 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T18:24:03.879 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-28T18:24:03.879 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T18:24:03.880 INFO:teuthology.orchestra.run.smithi105.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-28T18:24:03.880 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T18:24:03.880 INFO:teuthology.orchestra.run.smithi105.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:24:03.880 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:24:03.880 INFO:teuthology.orchestra.run.smithi105.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:24:03.880 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T18:24:03.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:03.880 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-11-28T18:24:03.880 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:24:03.881 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T18:24:03.881 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T18:24:03.881 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T18:24:03.881 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:24:03.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:03.972 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-11-28T18:24:03.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:03.982 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list-id 2023-11-28T18:24:04.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:04.038 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-11-28T18:24:04.038 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:24:04.038 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:24:04.038 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-28T18:24:04.038 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:24:04.120 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-28T18:24:04.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:04.121 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-11-28T18:24:04.121 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:24:04.121 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-28T18:24:04.122 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T18:24:04.122 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-28T18:24:04.122 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-28T18:24:04.122 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-28T18:24:04.123 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:24:04.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:04.258 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-11-28T18:24:04.258 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:24:04.258 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2023-11-28T18:24:04.258 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T18:24:04.258 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:24:04.299 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-28T18:24:04.299 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:24:04.299 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T18:24:04.353 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:24:04.353 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:24:04.353 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T18:24:04.404 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:24:04.469 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:24:04.522 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:24:04.522 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:24:04.522 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:24:04.522 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:24:04.523 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:24:04.523 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:24:04.523 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:24:04.523 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:24:04.523 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:24:04.523 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:24:04.523 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:24:04.523 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:24:04.523 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:24:04.523 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:24:04.523 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:24:04.524 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:24:04.524 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:24:04.524 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:24:04.524 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:24:04.524 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:24:04.524 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:24:04.524 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:24:04.524 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:24:04.524 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:24:04.524 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:24:04.524 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:24:04.524 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:24:04.525 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:24:04.525 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:24:04.525 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:24:04.525 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:24:04.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:04.526 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:24:04.576 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T18:24:04.577 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-28T18:24:04.623 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:24:04.645 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:24:04.700 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:24:04.700 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:24:04.700 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:24:04.700 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:24:04.700 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:24:04.700 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:24:04.700 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:24:04.700 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:24:04.701 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:24:04.701 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:24:04.701 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:24:04.701 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:24:04.701 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:24:04.701 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:24:04.701 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:24:04.701 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:24:04.701 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:24:04.701 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:24:04.701 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:24:04.701 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:24:04.702 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:24:04.702 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:24:04.702 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:24:04.702 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:24:04.702 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:24:04.702 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:24:04.702 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:24:04.702 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:24:04.702 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:24:04.702 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:24:04.702 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:24:04.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:04.703 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:24:04.720 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:2023-11-28T18:24:04.715+0000 7f15f4a46580 -1 init, newargv = 0x55a34b68ee90 newargc=15 2023-11-28T18:24:04.720 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[46729]: starting ceph client 2023-11-28T18:24:04.727 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[46729]: starting fuse 2023-11-28T18:24:04.729 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-11-28T18:24:04.730 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-28T18:24:04.730 DEBUG:teuthology.orchestra.run.smithi105:> sudo stdin-killer -- python3 -c ' 2023-11-28T18:24:04.730 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2023-11-28T18:24:04.731 DEBUG:teuthology.orchestra.run.smithi105:> import re 2023-11-28T18:24:04.731 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-11-28T18:24:04.731 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2023-11-28T18:24:04.731 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:24:04.732 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2023-11-28T18:24:04.732 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-28T18:24:04.732 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2023-11-28T18:24:04.732 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-28T18:24:04.733 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:24:04.733 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2023-11-28T18:24:04.733 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2023-11-28T18:24:04.733 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2023-11-28T18:24:04.734 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2023-11-28T18:24:04.734 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:24:04.734 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2023-11-28T18:24:04.735 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T18:24:04.736 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T18:24:04.736 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T18:24:04.736 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2023-11-28T18:24:04.736 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2023-11-28T18:24:04.736 DEBUG:teuthology.orchestra.run.smithi105:> return f 2023-11-28T18:24:04.736 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T18:24:04.736 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:24:04.736 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.0")) 2023-11-28T18:24:04.737 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-11-28T18:24:04.857 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.0.46729.asok 2023-11-28T18:24:04.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-28T18:24:04.868 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46729.asok 2023-11-28T18:24:04.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:04.869 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46729.asok status 2023-11-28T18:24:05.042 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:24:05.042 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2023-11-28T18:24:05.042 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "47c1c547372ed5bff7984647dc200bac0e019caf", 2023-11-28T18:24:05.042 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev)", 2023-11-28T18:24:05.043 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "0", 2023-11-28T18:24:05.043 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2023-11-28T18:24:05.043 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-28T18:24:05.043 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "46729", 2023-11-28T18:24:05.043 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2023-11-28T18:24:05.043 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:05.043 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2023-11-28T18:24:05.044 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2023-11-28T18:24:05.044 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 4812, 2023-11-28T18:24:05.044 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2023-11-28T18:24:05.044 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2023-11-28T18:24:05.044 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2023-11-28T18:24:05.044 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 4812 2023-11-28T18:24:05.044 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:05.044 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-11-28T18:24:05.045 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-11-28T18:24:05.045 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-11-28T18:24:05.045 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 3680514718 2023-11-28T18:24:05.045 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:05.045 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:05.045 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-11-28T18:24:05.045 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-11-28T18:24:05.045 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-11-28T18:24:05.046 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 3680514718 2023-11-28T18:24:05.046 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:05.046 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.4812 192.168.0.1:0/3680514718", 2023-11-28T18:24:05.046 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/3680514718", 2023-11-28T18:24:05.046 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2023-11-28T18:24:05.046 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 9, 2023-11-28T18:24:05.046 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 24, 2023-11-28T18:24:05.046 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2023-11-28T18:24:05.047 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2023-11-28T18:24:05.047 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2023-11-28T18:24:05.047 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:24:05.057 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T18:24:05.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.191+0000 7f8d0cb1d640 1 Processor -- start 2023-11-28T18:24:05.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.191+0000 7f8d0cb1d640 1 -- start start 2023-11-28T18:24:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.191+0000 7f8d0cb1d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d0815d320 0x7f8d0806c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.191+0000 7f8d0cb1d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8d080ce230 0x7f8d080cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.191+0000 7f8d0cb1d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8d080cdbb0 0x7f8d080cef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.191+0000 7f8d0cb1d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d080b3c40 con 0x7f8d080cdbb0 2023-11-28T18:24:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.191+0000 7f8d0cb1d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d0806ca80 con 0x7f8d080ce230 2023-11-28T18:24:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.191+0000 7f8d0cb1d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0815d700 con 0x7f8d0815d320 2023-11-28T18:24:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.191+0000 7f8d06d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8d080cdbb0 0x7f8d080cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.191+0000 7f8d06d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8d080cdbb0 0x7f8d080cef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36390/0 (socket says 172.21.15.105:36390) 2023-11-28T18:24:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.191+0000 7f8d06d9d640 1 -- 172.21.15.105:0/852608230 learned_addr learned my addr 172.21.15.105:0/852608230 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.191+0000 7f8cfdd9b640 1 --2- 172.21.15.105:0/852608230 >> v2:172.21.15.148:3300/0 conn(0x7f8d080ce230 0x7f8d080cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.191+0000 7f8d0659c640 1 --2- 172.21.15.105:0/852608230 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d0815d320 0x7f8d0806c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.191+0000 7f8d06d9d640 1 -- 172.21.15.105:0/852608230 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d0815d320 msgr2=0x7f8d0806c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.191+0000 7f8d06d9d640 1 --2- 172.21.15.105:0/852608230 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d0815d320 0x7f8d0806c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.191+0000 7f8d06d9d640 1 -- 172.21.15.105:0/852608230 >> v2:172.21.15.148:3300/0 conn(0x7f8d080ce230 msgr2=0x7f8d080cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.191+0000 7f8d06d9d640 1 --2- 172.21.15.105:0/852608230 >> v2:172.21.15.148:3300/0 conn(0x7f8d080ce230 0x7f8d080cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.191+0000 7f8d06d9d640 1 -- 172.21.15.105:0/852608230 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d080cdfc0 con 0x7f8d080cdbb0 2023-11-28T18:24:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.191+0000 7f8d0659c640 1 --2- 172.21.15.105:0/852608230 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d0815d320 0x7f8d0806c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:05.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.195+0000 7f8d06d9d640 1 --2- 172.21.15.105:0/852608230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d080cdbb0 0x7f8d080cef80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8d08067310 tx=0x7f8d0001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbdbe7f4555b829 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.195+0000 7f8d05d9b640 1 -- 172.21.15.105:0/852608230 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d000a7070 con 0x7f8d080cdbb0 2023-11-28T18:24:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.195+0000 7f8d05d9b640 1 -- 172.21.15.105:0/852608230 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d000ab430 con 0x7f8d080cdbb0 2023-11-28T18:24:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.195+0000 7f8d05d9b640 1 -- 172.21.15.105:0/852608230 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d000ab7c0 con 0x7f8d080cdbb0 2023-11-28T18:24:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.195+0000 7f8d0cb1d640 1 -- 172.21.15.105:0/852608230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d080cdbb0 msgr2=0x7f8d080cef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.195+0000 7f8d0cb1d640 1 --2- 172.21.15.105:0/852608230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d080cdbb0 0x7f8d080cef80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8d08067310 tx=0x7f8d0001c470 comp rx=0 tx=0).stop 2023-11-28T18:24:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.195+0000 7f8d0cb1d640 1 -- 172.21.15.105:0/852608230 shutdown_connections 2023-11-28T18:24:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.195+0000 7f8d0cb1d640 1 --2- 172.21.15.105:0/852608230 >> v2:172.21.15.148:3300/0 conn(0x7f8d080ce230 0x7f8d080cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.195+0000 7f8d0cb1d640 1 --2- 172.21.15.105:0/852608230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d080cdbb0 0x7f8d080cef80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.195+0000 7f8d0cb1d640 1 --2- 172.21.15.105:0/852608230 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d0815d320 0x7f8d0806c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.195+0000 7f8d0cb1d640 1 -- 172.21.15.105:0/852608230 >> 172.21.15.105:0/852608230 conn(0x7f8d080d0df0 msgr2=0x7f8d080d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.195+0000 7f8d0cb1d640 1 -- 172.21.15.105:0/852608230 shutdown_connections 2023-11-28T18:24:05.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.195+0000 7f8d0cb1d640 1 -- 172.21.15.105:0/852608230 wait complete. 2023-11-28T18:24:05.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8d0cb1d640 1 Processor -- start 2023-11-28T18:24:05.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8d0cb1d640 1 -- start start 2023-11-28T18:24:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8d0cb1d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d080cdbb0 0x7f8d08156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8d0cb1d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d080ce230 0x7f8d0814f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8d0cb1d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d0815d320 0x7f8d080b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8d0cb1d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d08138820 con 0x7f8d0815d320 2023-11-28T18:24:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8d0cb1d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d08138b00 con 0x7f8d080cdbb0 2023-11-28T18:24:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8d0cb1d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d08138de0 con 0x7f8d080ce230 2023-11-28T18:24:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8d06d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d0815d320 0x7f8d080b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:05.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8d06d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d0815d320 0x7f8d080b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36396/0 (socket says 172.21.15.105:36396) 2023-11-28T18:24:05.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8d06d9d640 1 -- 172.21.15.105:0/3869024797 learned_addr learned my addr 172.21.15.105:0/3869024797 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:05.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8cfdd9b640 1 --2- 172.21.15.105:0/3869024797 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d080ce230 0x7f8d0814f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:05.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8d0659c640 1 --2- 172.21.15.105:0/3869024797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d080cdbb0 0x7f8d08156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:05.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8d06d9d640 1 -- 172.21.15.105:0/3869024797 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d080ce230 msgr2=0x7f8d0814f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:05.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8d06d9d640 1 --2- 172.21.15.105:0/3869024797 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d080ce230 0x7f8d0814f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:05.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8d06d9d640 1 -- 172.21.15.105:0/3869024797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d080cdbb0 msgr2=0x7f8d08156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:05.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8d06d9d640 1 --2- 172.21.15.105:0/3869024797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d080cdbb0 0x7f8d08156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:05.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8d06d9d640 1 -- 172.21.15.105:0/3869024797 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d000094d0 con 0x7f8d0815d320 2023-11-28T18:24:05.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8d06d9d640 1 --2- 172.21.15.105:0/3869024797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d0815d320 0x7f8d080b3dd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8d0001ab20 tx=0x7f8d00019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:05.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8cff7fe640 1 -- 172.21.15.105:0/3869024797 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d000b4620 con 0x7f8d0815d320 2023-11-28T18:24:05.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8cff7fe640 1 -- 172.21.15.105:0/3869024797 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d000b4780 con 0x7f8d0815d320 2023-11-28T18:24:05.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8d0cb1d640 1 -- 172.21.15.105:0/3869024797 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d081390c0 con 0x7f8d0815d320 2023-11-28T18:24:05.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8cff7fe640 1 -- 172.21.15.105:0/3869024797 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d000b4b30 con 0x7f8d0815d320 2023-11-28T18:24:05.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.199+0000 7f8d0cb1d640 1 -- 172.21.15.105:0/3869024797 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0814fcb0 con 0x7f8d0815d320 2023-11-28T18:24:05.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.203+0000 7f8d0cb1d640 1 -- 172.21.15.105:0/3869024797 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d081395e0 con 0x7f8d0815d320 2023-11-28T18:24:05.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.203+0000 7f8cff7fe640 1 -- 172.21.15.105:0/3869024797 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8d000b4c90 con 0x7f8d0815d320 2023-11-28T18:24:05.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.203+0000 7f8cff7fe640 1 --2- 172.21.15.105:0/3869024797 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8ce80668d0 0x7f8ce8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:05.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.203+0000 7f8cff7fe640 1 -- 172.21.15.105:0/3869024797 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8d0011e890 con 0x7f8d0815d320 2023-11-28T18:24:05.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.203+0000 7f8d0659c640 1 --2- 172.21.15.105:0/3869024797 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8ce80668d0 0x7f8ce8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:05.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.203+0000 7f8d0659c640 1 --2- 172.21.15.105:0/3869024797 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8ce80668d0 0x7f8ce8068d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8006e90 tx=0x7f8cf8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:05.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.207+0000 7f8cff7fe640 1 -- 172.21.15.105:0/3869024797 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8d000eb910 con 0x7f8d0815d320 2023-11-28T18:24:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.391+0000 7f8d0cb1d640 1 -- 172.21.15.105:0/3869024797 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8d08067570 con 0x7f8d0815d320 2023-11-28T18:24:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.391+0000 7f8cff7fe640 1 -- 172.21.15.105:0/3869024797 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f8d000eef30 con 0x7f8d0815d320 2023-11-28T18:24:05.396 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T18:24:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.395+0000 7f8d0cb1d640 1 -- 172.21.15.105:0/3869024797 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8ce80668d0 msgr2=0x7f8ce8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.395+0000 7f8d0cb1d640 1 --2- 172.21.15.105:0/3869024797 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8ce80668d0 0x7f8ce8068d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8006e90 tx=0x7f8cf8008040 comp rx=0 tx=0).stop 2023-11-28T18:24:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.395+0000 7f8d0cb1d640 1 -- 172.21.15.105:0/3869024797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d0815d320 msgr2=0x7f8d080b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.395+0000 7f8d0cb1d640 1 --2- 172.21.15.105:0/3869024797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d0815d320 0x7f8d080b3dd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8d0001ab20 tx=0x7f8d00019210 comp rx=0 tx=0).stop 2023-11-28T18:24:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.395+0000 7f8d0cb1d640 1 -- 172.21.15.105:0/3869024797 shutdown_connections 2023-11-28T18:24:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.395+0000 7f8d0cb1d640 1 --2- 172.21.15.105:0/3869024797 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8ce80668d0 0x7f8ce8068d90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.395+0000 7f8d0cb1d640 1 --2- 172.21.15.105:0/3869024797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d0815d320 0x7f8d080b3dd0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.395+0000 7f8d0cb1d640 1 --2- 172.21.15.105:0/3869024797 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d080ce230 0x7f8d0814f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.395+0000 7f8d0cb1d640 1 --2- 172.21.15.105:0/3869024797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d080cdbb0 0x7f8d08156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.395+0000 7f8d0cb1d640 1 -- 172.21.15.105:0/3869024797 >> 172.21.15.105:0/3869024797 conn(0x7f8d080d0df0 msgr2=0x7f8d080b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:05.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.395+0000 7f8d0cb1d640 1 -- 172.21.15.105:0/3869024797 shutdown_connections 2023-11-28T18:24:05.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:05.395+0000 7f8d0cb1d640 1 -- 172.21.15.105:0/3869024797 wait complete. 2023-11-28T18:24:05.418 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T18:24:05.418 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T18:24:05.419 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2023-11-28T18:24:05.419 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-28T18:24:05.419 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T18:24:05.419 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T18:24:05.419 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T18:24:05.419 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T18:24:05.419 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-28T18:24:05.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:05.420 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2023-11-28T18:24:05.429 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T18:24:05.430 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T18:24:05.430 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T18:24:05.430 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:05.431 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-11-28T18:24:05.431 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:05.431 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:24:05.431 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:24:05.431 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2023-11-28T18:24:05.431 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:24:05.431 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:24:05.432 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2023-11-28T18:24:05.432 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:24:05.432 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:24:05.432 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T18:24:05.432 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:24:05.432 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-28T18:24:05.432 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42376sec preferred_lft 42376sec 2023-11-28T18:24:05.432 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2023-11-28T18:24:05.433 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:05.433 INFO:teuthology.orchestra.run.smithi105.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-28T18:24:05.433 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 02:42:cc:d4:87:d2 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:24:05.433 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-28T18:24:05.433 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:05.433 INFO:teuthology.orchestra.run.smithi105.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T18:24:05.434 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-28T18:24:05.434 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T18:24:05.434 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:05.434 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2023-11-28T18:24:05.434 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:05.434 INFO:teuthology.orchestra.run.smithi105.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T18:24:05.435 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:24:05.435 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2023-11-28T18:24:05.435 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:05.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:05.435 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T18:24:05.491 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-11-28T18:24:05.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:05.493 DEBUG:teuthology.orchestra.run.smithi105:> route 2023-11-28T18:24:05.548 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2023-11-28T18:24:05.548 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T18:24:05.549 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-28T18:24:05.549 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T18:24:05.549 INFO:teuthology.orchestra.run.smithi105.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-28T18:24:05.549 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T18:24:05.549 INFO:teuthology.orchestra.run.smithi105.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:24:05.549 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:24:05.550 INFO:teuthology.orchestra.run.smithi105.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:24:05.550 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T18:24:05.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:05.550 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-11-28T18:24:05.551 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:24:05.551 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T18:24:05.551 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T18:24:05.551 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T18:24:05.551 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:24:05.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:05.639 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-11-28T18:24:05.648 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T18:24:05.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:05.649 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list-id 2023-11-28T18:24:05.701 INFO:teuthology.orchestra.run.smithi105.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-28T18:24:05.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:05.703 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-11-28T18:24:05.703 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:24:05.703 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:24:05.703 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-28T18:24:05.703 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:24:05.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:05.783 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T18:24:05.845 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T18:24:05.845 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T18:24:05.845 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T18:24:05.845 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:05.845 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-11-28T18:24:05.846 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:05.846 INFO:teuthology.orchestra.run.smithi105.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T18:24:05.846 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether a6:0e:4b:42:c0:94 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T18:24:05.846 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T18:24:05.846 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:05.846 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::a40e:4bff:fe42:c094/64 scope link tentative 2023-11-28T18:24:05.846 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:05.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:05.847 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T18:24:05.913 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T18:24:05.913 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T18:24:05.913 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T18:24:05.913 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:05.913 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-11-28T18:24:05.914 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:05.914 INFO:teuthology.orchestra.run.smithi105.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T18:24:05.914 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether a6:0e:4b:42:c0:94 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T18:24:05.914 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T18:24:05.914 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:05.914 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::a40e:4bff:fe42:c094/64 scope link tentative 2023-11-28T18:24:05.914 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:05.915 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-28T18:24:05.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:05.916 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-11-28T18:24:05.916 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:24:05.916 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-28T18:24:05.916 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T18:24:05.916 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-28T18:24:05.916 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-28T18:24:05.916 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-28T18:24:05.916 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:24:06.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:06.052 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-11-28T18:24:06.052 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:24:06.052 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.1 up 2023-11-28T18:24:06.052 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T18:24:06.052 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:24:06.092 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-11-28T18:24:06.092 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:24:06.093 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-28T18:24:06.142 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-28T18:24:06.142 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:24:06.142 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-28T18:24:06.193 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:24:06.258 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:24:06.310 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:24:06.310 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:24:06.310 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:24:06.310 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:24:06.311 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:24:06.311 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:24:06.311 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:24:06.311 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:24:06.311 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:24:06.311 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:24:06.311 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:24:06.311 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:24:06.311 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:24:06.311 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:24:06.311 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:24:06.312 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:24:06.312 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:24:06.312 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:24:06.312 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:24:06.312 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:24:06.312 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:24:06.312 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:24:06.312 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:24:06.312 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:24:06.312 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:24:06.312 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:24:06.313 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:24:06.313 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:24:06.313 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:24:06.313 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:24:06.313 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:24:06.313 INFO:teuthology.orchestra.run.smithi105.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-28T18:24:06.313 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:24:06.313 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:24:06.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:06.314 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:24:06.364 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-11-28T18:24:06.365 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2023-11-28T18:24:06.365 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-11-28T18:24:06.411 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:24:06.433 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:24:06.488 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:24:06.488 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:24:06.489 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:24:06.489 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:24:06.489 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:24:06.489 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:24:06.489 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:24:06.489 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:24:06.489 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:24:06.489 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:24:06.490 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:24:06.490 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:24:06.490 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:24:06.490 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:24:06.490 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:24:06.490 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:24:06.490 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:24:06.491 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:24:06.491 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:24:06.491 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:24:06.492 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:24:06.492 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:24:06.492 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:24:06.492 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:24:06.492 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:24:06.492 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:24:06.492 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:24:06.492 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:24:06.493 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:24:06.493 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:24:06.493 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:24:06.493 INFO:teuthology.orchestra.run.smithi105.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-28T18:24:06.493 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:24:06.493 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:24:06.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:06.494 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:24:06.508 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi105.stderr:2023-11-28T18:24:06.503+0000 7f212bdd7580 -1 init, newargv = 0x560ca1e7fd70 newargc=15 2023-11-28T18:24:06.509 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi105.stderr:ceph-fuse[46864]: starting ceph client 2023-11-28T18:24:06.516 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi105.stderr:ceph-fuse[46864]: starting fuse 2023-11-28T18:24:06.518 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-11-28T18:24:06.518 INFO:teuthology.orchestra.run.smithi105.stdout:44 2023-11-28T18:24:06.518 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2023-11-28T18:24:06.518 DEBUG:teuthology.orchestra.run.smithi105:> sudo stdin-killer -- python3 -c ' 2023-11-28T18:24:06.519 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2023-11-28T18:24:06.519 DEBUG:teuthology.orchestra.run.smithi105:> import re 2023-11-28T18:24:06.519 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-11-28T18:24:06.519 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2023-11-28T18:24:06.519 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:24:06.519 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2023-11-28T18:24:06.519 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-11-28T18:24:06.520 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2023-11-28T18:24:06.520 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-11-28T18:24:06.520 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:24:06.520 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2023-11-28T18:24:06.520 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2023-11-28T18:24:06.520 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2023-11-28T18:24:06.520 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2023-11-28T18:24:06.520 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:24:06.521 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2023-11-28T18:24:06.521 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T18:24:06.521 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T18:24:06.521 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T18:24:06.521 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2023-11-28T18:24:06.521 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2023-11-28T18:24:06.521 DEBUG:teuthology.orchestra.run.smithi105:> return f 2023-11-28T18:24:06.521 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T18:24:06.522 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:24:06.522 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.2")) 2023-11-28T18:24:06.522 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-11-28T18:24:06.641 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.2.46864.asok 2023-11-28T18:24:06.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-28T18:24:06.652 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.46864.asok 2023-11-28T18:24:06.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:06.652 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.46864.asok status 2023-11-28T18:24:06.820 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:24:06.821 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2023-11-28T18:24:06.821 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "47c1c547372ed5bff7984647dc200bac0e019caf", 2023-11-28T18:24:06.821 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev)", 2023-11-28T18:24:06.821 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "2", 2023-11-28T18:24:06.821 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2023-11-28T18:24:06.821 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-11-28T18:24:06.821 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "46864", 2023-11-28T18:24:06.821 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2023-11-28T18:24:06.822 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:06.822 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2023-11-28T18:24:06.822 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2023-11-28T18:24:06.822 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 4827, 2023-11-28T18:24:06.822 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2023-11-28T18:24:06.822 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2023-11-28T18:24:06.822 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2023-11-28T18:24:06.822 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 4827 2023-11-28T18:24:06.822 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:06.822 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-11-28T18:24:06.822 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-11-28T18:24:06.823 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:0", 2023-11-28T18:24:06.823 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 3563842909 2023-11-28T18:24:06.823 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:06.823 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:06.823 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-11-28T18:24:06.823 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-11-28T18:24:06.823 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:0", 2023-11-28T18:24:06.823 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 3563842909 2023-11-28T18:24:06.823 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:06.823 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.4827 172.21.15.105:0/3563842909", 2023-11-28T18:24:06.823 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "172.21.15.105:0/3563842909", 2023-11-28T18:24:06.823 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2023-11-28T18:24:06.824 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 9, 2023-11-28T18:24:06.824 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 24, 2023-11-28T18:24:06.824 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2023-11-28T18:24:06.824 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2023-11-28T18:24:06.824 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2023-11-28T18:24:06.824 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:24:06.836 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T18:24:06.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.971+0000 7f3d48f0f640 1 Processor -- start 2023-11-28T18:24:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.971+0000 7f3d48f0f640 1 -- start start 2023-11-28T18:24:06.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.971+0000 7f3d48f0f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3d440b3d30 0x7f3d441680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:06.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.971+0000 7f3d48f0f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d440b25b0 con 0x7f3d440d3c30 2023-11-28T18:24:06.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.971+0000 7f3d48f0f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d440d1c30 con 0x7f3d4415d4f0 2023-11-28T18:24:06.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.971+0000 7f3d48f0f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d440b2900 con 0x7f3d440b3d30 2023-11-28T18:24:06.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.971+0000 7f3d42d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3d440b3d30 0x7f3d441680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:06.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.971+0000 7f3d42d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3d440b3d30 0x7f3d441680a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36412/0 (socket says 172.21.15.105:36412) 2023-11-28T18:24:06.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.971+0000 7f3d42d9d640 1 -- 172.21.15.105:0/514661074 learned_addr learned my addr 172.21.15.105:0/514661074 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:06.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.971+0000 7f3d41d9b640 1 --1- 172.21.15.105:0/514661074 >> v1:172.21.15.105:6789/0 conn(0x7f3d440d3c30 0x7f3d440d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41346/0 (socket says 172.21.15.105:41346) 2023-11-28T18:24:06.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.971+0000 7f3d42d9d640 1 -- 172.21.15.105:0/514661074 >> v1:172.21.15.148:6789/0 conn(0x7f3d4415d4f0 legacy=0x7f3d440d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:06.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.971+0000 7f3d42d9d640 1 -- 172.21.15.105:0/514661074 >> v1:172.21.15.105:6789/0 conn(0x7f3d440d3c30 legacy=0x7f3d440d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:06.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.971+0000 7f3d42d9d640 1 -- 172.21.15.105:0/514661074 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d440cea30 con 0x7f3d440b3d30 2023-11-28T18:24:06.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.975+0000 7f3d42d9d640 1 --2- 172.21.15.105:0/514661074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d440b3d30 0x7f3d441680a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c009e50 tx=0x7f3d3c01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad332e06c15b603a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:06.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.975+0000 7f3d4159a640 1 -- 172.21.15.105:0/514661074 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c0a5780 con 0x7f3d440b3d30 2023-11-28T18:24:06.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.975+0000 7f3d4159a640 1 -- 172.21.15.105:0/514661074 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3c0a58e0 con 0x7f3d440b3d30 2023-11-28T18:24:06.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.975+0000 7f3d4159a640 1 -- 172.21.15.105:0/514661074 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c0a5c70 con 0x7f3d440b3d30 2023-11-28T18:24:06.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.975+0000 7f3d48f0f640 1 -- 172.21.15.105:0/514661074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d440b3d30 msgr2=0x7f3d441680a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:06.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.975+0000 7f3d48f0f640 1 --2- 172.21.15.105:0/514661074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d440b3d30 0x7f3d441680a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c009e50 tx=0x7f3d3c01c8c0 comp rx=0 tx=0).stop 2023-11-28T18:24:06.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.975+0000 7f3d48f0f640 1 -- 172.21.15.105:0/514661074 shutdown_connections 2023-11-28T18:24:06.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.975+0000 7f3d48f0f640 1 --2- 172.21.15.105:0/514661074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d440b3d30 0x7f3d441680a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:06.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.975+0000 7f3d48f0f640 1 -- 172.21.15.105:0/514661074 >> 172.21.15.105:0/514661074 conn(0x7f3d440d3030 msgr2=0x7f3d440d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:06.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.975+0000 7f3d48f0f640 1 -- 172.21.15.105:0/514661074 shutdown_connections 2023-11-28T18:24:06.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.975+0000 7f3d48f0f640 1 -- 172.21.15.105:0/514661074 wait complete. 2023-11-28T18:24:06.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d48f0f640 1 Processor -- start 2023-11-28T18:24:06.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d48f0f640 1 -- start start 2023-11-28T18:24:06.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d48f0f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d440b3d30 0x7f3d44154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:06.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d48f0f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d440d3c30 0x7f3d4415b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:06.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d48f0f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d4415d4f0 0x7f3d44158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:06.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d48f0f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d440d0a20 con 0x7f3d440b3d30 2023-11-28T18:24:06.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d48f0f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d44155570 con 0x7f3d440d3c30 2023-11-28T18:24:06.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d48f0f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d44155060 con 0x7f3d4415d4f0 2023-11-28T18:24:06.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d4259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d440b3d30 0x7f3d44154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:06.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d4259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d440b3d30 0x7f3d44154b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36414/0 (socket says 172.21.15.105:36414) 2023-11-28T18:24:06.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d4259c640 1 -- 172.21.15.105:0/707280814 learned_addr learned my addr 172.21.15.105:0/707280814 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:06.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d4259c640 1 -- 172.21.15.105:0/707280814 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d4415d4f0 msgr2=0x7f3d44158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:06.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d4259c640 1 --2- 172.21.15.105:0/707280814 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d4415d4f0 0x7f3d44158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:06.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d4259c640 1 -- 172.21.15.105:0/707280814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d440d3c30 msgr2=0x7f3d4415b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:06.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d4259c640 1 --2- 172.21.15.105:0/707280814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d440d3c30 0x7f3d4415b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:06.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d4259c640 1 -- 172.21.15.105:0/707280814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3c009b30 con 0x7f3d440b3d30 2023-11-28T18:24:06.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d4259c640 1 --2- 172.21.15.105:0/707280814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d440b3d30 0x7f3d44154b20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400c3c0 tx=0x7f3d3400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:06.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d32ffd640 1 -- 172.21.15.105:0/707280814 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d340052d0 con 0x7f3d440b3d30 2023-11-28T18:24:06.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d32ffd640 1 -- 172.21.15.105:0/707280814 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d34005430 con 0x7f3d440b3d30 2023-11-28T18:24:06.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d32ffd640 1 -- 172.21.15.105:0/707280814 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d34003840 con 0x7f3d440b3d30 2023-11-28T18:24:06.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d48f0f640 1 -- 172.21.15.105:0/707280814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d440cbd60 con 0x7f3d440b3d30 2023-11-28T18:24:06.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d48f0f640 1 -- 172.21.15.105:0/707280814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d4415ab30 con 0x7f3d440b3d30 2023-11-28T18:24:06.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.979+0000 7f3d48f0f640 1 -- 172.21.15.105:0/707280814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d4415a920 con 0x7f3d440b3d30 2023-11-28T18:24:06.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.983+0000 7f3d32ffd640 1 -- 172.21.15.105:0/707280814 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3d340039a0 con 0x7f3d440b3d30 2023-11-28T18:24:06.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.983+0000 7f3d32ffd640 1 --2- 172.21.15.105:0/707280814 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d240668d0 0x7f3d24068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:06.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.983+0000 7f3d32ffd640 1 -- 172.21.15.105:0/707280814 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3d34083d10 con 0x7f3d440b3d30 2023-11-28T18:24:06.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.983+0000 7f3d41d9b640 1 --2- 172.21.15.105:0/707280814 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d240668d0 0x7f3d24068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:06.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.983+0000 7f3d41d9b640 1 --2- 172.21.15.105:0/707280814 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d240668d0 0x7f3d24068d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3d44155bf0 tx=0x7f3d380079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:06.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:06.987+0000 7f3d32ffd640 1 -- 172.21.15.105:0/707280814 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3d34089020 con 0x7f3d440b3d30 2023-11-28T18:24:07.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:07.167+0000 7f3d48f0f640 1 -- 172.21.15.105:0/707280814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3d44067570 con 0x7f3d440b3d30 2023-11-28T18:24:07.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:07.167+0000 7f3d32ffd640 1 -- 172.21.15.105:0/707280814 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f3d34017070 con 0x7f3d440b3d30 2023-11-28T18:24:07.171 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T18:24:07.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:07.171+0000 7f3d48f0f640 1 -- 172.21.15.105:0/707280814 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d240668d0 msgr2=0x7f3d24068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:07.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:07.171+0000 7f3d48f0f640 1 --2- 172.21.15.105:0/707280814 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d240668d0 0x7f3d24068d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3d44155bf0 tx=0x7f3d380079d0 comp rx=0 tx=0).stop 2023-11-28T18:24:07.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:07.171+0000 7f3d48f0f640 1 -- 172.21.15.105:0/707280814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d440b3d30 msgr2=0x7f3d44154b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:07.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:07.171+0000 7f3d48f0f640 1 --2- 172.21.15.105:0/707280814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d440b3d30 0x7f3d44154b20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400c3c0 tx=0x7f3d3400c880 comp rx=0 tx=0).stop 2023-11-28T18:24:07.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:07.171+0000 7f3d48f0f640 1 -- 172.21.15.105:0/707280814 shutdown_connections 2023-11-28T18:24:07.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:07.171+0000 7f3d48f0f640 1 --2- 172.21.15.105:0/707280814 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d4415d4f0 0x7f3d44158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:07.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:07.171+0000 7f3d48f0f640 1 --2- 172.21.15.105:0/707280814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d440d3c30 0x7f3d4415b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:07.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:07.171+0000 7f3d48f0f640 1 --2- 172.21.15.105:0/707280814 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d240668d0 0x7f3d24068d90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:07.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:07.171+0000 7f3d48f0f640 1 --2- 172.21.15.105:0/707280814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d440b3d30 0x7f3d44154b20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:07.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:07.171+0000 7f3d48f0f640 1 -- 172.21.15.105:0/707280814 >> 172.21.15.105:0/707280814 conn(0x7f3d440d3030 msgr2=0x7f3d44157e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:07.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:07.171+0000 7f3d48f0f640 1 -- 172.21.15.105:0/707280814 shutdown_connections 2023-11-28T18:24:07.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:07.171+0000 7f3d48f0f640 1 -- 172.21.15.105:0/707280814 wait complete. 2023-11-28T18:24:07.192 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T18:24:07.192 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T18:24:07.192 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2023-11-28T18:24:07.193 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-28T18:24:07.193 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T18:24:07.193 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T18:24:07.193 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T18:24:07.193 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T18:24:07.194 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-28T18:24:07.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:07.194 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2023-11-28T18:24:07.205 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T18:24:07.205 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T18:24:07.205 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T18:24:07.205 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:07.206 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2023-11-28T18:24:07.206 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:07.206 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:24:07.206 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:24:07.206 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2023-11-28T18:24:07.206 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:24:07.207 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:24:07.207 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2023-11-28T18:24:07.207 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:24:07.207 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:24:07.207 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T18:24:07.207 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:24:07.207 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-28T18:24:07.208 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42372sec preferred_lft 42372sec 2023-11-28T18:24:07.208 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link 2023-11-28T18:24:07.208 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:07.208 INFO:teuthology.orchestra.run.smithi148.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-28T18:24:07.208 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 02:42:8a:56:dd:03 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:24:07.208 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-28T18:24:07.209 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:07.209 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T18:24:07.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:07.209 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-11-28T18:24:07.209 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-11-28T18:24:07.209 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add name ceph-brx type bridge 2023-11-28T18:24:07.209 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr flush dev ceph-brx 2023-11-28T18:24:07.209 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set ceph-brx up 2023-11-28T18:24:07.210 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T18:24:07.210 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-11-28T18:24:07.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:07.327 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T18:24:07.347 INFO:teuthology.orchestra.run.smithi148.stdout:1 2023-11-28T18:24:07.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:07.350 DEBUG:teuthology.orchestra.run.smithi148:> route 2023-11-28T18:24:07.425 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2023-11-28T18:24:07.425 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T18:24:07.425 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-28T18:24:07.425 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T18:24:07.426 INFO:teuthology.orchestra.run.smithi148.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-28T18:24:07.426 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T18:24:07.426 INFO:teuthology.orchestra.run.smithi148.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:24:07.426 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:24:07.426 INFO:teuthology.orchestra.run.smithi148.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:24:07.426 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T18:24:07.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:07.426 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-11-28T18:24:07.427 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-11-28T18:24:07.427 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T18:24:07.427 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T18:24:07.427 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T18:24:07.427 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-11-28T18:24:07.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:07.522 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2023-11-28T18:24:07.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:07.532 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list-id 2023-11-28T18:24:07.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:07.587 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-11-28T18:24:07.587 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-11-28T18:24:07.587 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T18:24:07.587 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-28T18:24:07.587 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-11-28T18:24:07.670 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-28T18:24:07.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:07.670 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-11-28T18:24:07.670 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-11-28T18:24:07.671 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-28T18:24:07.671 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T18:24:07.671 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-28T18:24:07.671 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-28T18:24:07.671 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-28T18:24:07.671 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-11-28T18:24:07.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:07.831 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-11-28T18:24:07.831 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-11-28T18:24:07.831 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2023-11-28T18:24:07.831 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T18:24:07.831 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-11-28T18:24:07.871 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-28T18:24:07.872 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:24:07.872 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-28T18:24:07.927 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-28T18:24:07.927 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:24:07.927 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-28T18:24:07.977 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2023-11-28T18:24:08.040 DEBUG:teuthology.orchestra.run.smithi148:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:24:08.096 INFO:teuthology.orchestra.run.smithi148.stdout:/sys 2023-11-28T18:24:08.096 INFO:teuthology.orchestra.run.smithi148.stdout:/proc 2023-11-28T18:24:08.096 INFO:teuthology.orchestra.run.smithi148.stdout:/dev 2023-11-28T18:24:08.096 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/pts 2023-11-28T18:24:08.096 INFO:teuthology.orchestra.run.smithi148.stdout:/run 2023-11-28T18:24:08.097 INFO:teuthology.orchestra.run.smithi148.stdout:/ 2023-11-28T18:24:08.097 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/security 2023-11-28T18:24:08.097 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/shm 2023-11-28T18:24:08.097 INFO:teuthology.orchestra.run.smithi148.stdout:/run/lock 2023-11-28T18:24:08.097 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/cgroup 2023-11-28T18:24:08.098 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/pstore 2023-11-28T18:24:08.098 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/bpf 2023-11-28T18:24:08.098 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:24:08.098 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/hugepages 2023-11-28T18:24:08.098 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/mqueue 2023-11-28T18:24:08.099 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/debug 2023-11-28T18:24:08.099 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/tracing 2023-11-28T18:24:08.099 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/fs/nfsd 2023-11-28T18:24:08.099 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/fuse/connections 2023-11-28T18:24:08.099 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/config 2023-11-28T18:24:08.099 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:24:08.099 INFO:teuthology.orchestra.run.smithi148.stdout:/run/rpc_pipefs 2023-11-28T18:24:08.100 INFO:teuthology.orchestra.run.smithi148.stdout:/run/user/1000 2023-11-28T18:24:08.100 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph 2023-11-28T18:24:08.100 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-4 2023-11-28T18:24:08.100 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-5 2023-11-28T18:24:08.100 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-6 2023-11-28T18:24:08.100 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-7 2023-11-28T18:24:08.100 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns 2023-11-28T18:24:08.100 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T18:24:08.100 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T18:24:08.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:08.101 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2023-11-28T18:24:08.150 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T18:24:08.150 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-28T18:24:08.196 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2023-11-28T18:24:08.218 DEBUG:teuthology.orchestra.run.smithi148:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:24:08.272 INFO:teuthology.orchestra.run.smithi148.stdout:/sys 2023-11-28T18:24:08.272 INFO:teuthology.orchestra.run.smithi148.stdout:/proc 2023-11-28T18:24:08.272 INFO:teuthology.orchestra.run.smithi148.stdout:/dev 2023-11-28T18:24:08.272 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/pts 2023-11-28T18:24:08.272 INFO:teuthology.orchestra.run.smithi148.stdout:/run 2023-11-28T18:24:08.272 INFO:teuthology.orchestra.run.smithi148.stdout:/ 2023-11-28T18:24:08.272 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/security 2023-11-28T18:24:08.272 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/shm 2023-11-28T18:24:08.272 INFO:teuthology.orchestra.run.smithi148.stdout:/run/lock 2023-11-28T18:24:08.272 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/cgroup 2023-11-28T18:24:08.273 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/pstore 2023-11-28T18:24:08.273 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/bpf 2023-11-28T18:24:08.273 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:24:08.273 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/hugepages 2023-11-28T18:24:08.273 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/mqueue 2023-11-28T18:24:08.273 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/debug 2023-11-28T18:24:08.273 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/tracing 2023-11-28T18:24:08.273 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/fs/nfsd 2023-11-28T18:24:08.273 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/fuse/connections 2023-11-28T18:24:08.273 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/config 2023-11-28T18:24:08.273 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:24:08.273 INFO:teuthology.orchestra.run.smithi148.stdout:/run/rpc_pipefs 2023-11-28T18:24:08.274 INFO:teuthology.orchestra.run.smithi148.stdout:/run/user/1000 2023-11-28T18:24:08.274 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph 2023-11-28T18:24:08.274 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-4 2023-11-28T18:24:08.274 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-5 2023-11-28T18:24:08.274 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-6 2023-11-28T18:24:08.274 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-7 2023-11-28T18:24:08.274 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns 2023-11-28T18:24:08.274 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T18:24:08.274 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T18:24:08.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:08.275 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2023-11-28T18:24:08.300 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi148.stderr:2023-11-28T18:24:08.295+0000 7fc32a7e8580 -1 init, newargv = 0x5610ce62cd70 newargc=15 2023-11-28T18:24:08.300 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi148.stderr:ceph-fuse[44203]: starting ceph client 2023-11-28T18:24:08.309 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi148.stderr:ceph-fuse[44203]: starting fuse 2023-11-28T18:24:09.410 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2023-11-28T18:24:09.434 DEBUG:teuthology.orchestra.run.smithi148:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:24:09.488 INFO:teuthology.orchestra.run.smithi148.stdout:/sys 2023-11-28T18:24:09.489 INFO:teuthology.orchestra.run.smithi148.stdout:/proc 2023-11-28T18:24:09.489 INFO:teuthology.orchestra.run.smithi148.stdout:/dev 2023-11-28T18:24:09.489 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/pts 2023-11-28T18:24:09.489 INFO:teuthology.orchestra.run.smithi148.stdout:/run 2023-11-28T18:24:09.489 INFO:teuthology.orchestra.run.smithi148.stdout:/ 2023-11-28T18:24:09.489 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/security 2023-11-28T18:24:09.489 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/shm 2023-11-28T18:24:09.489 INFO:teuthology.orchestra.run.smithi148.stdout:/run/lock 2023-11-28T18:24:09.490 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/cgroup 2023-11-28T18:24:09.490 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/pstore 2023-11-28T18:24:09.490 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/bpf 2023-11-28T18:24:09.490 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:24:09.490 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/hugepages 2023-11-28T18:24:09.490 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/mqueue 2023-11-28T18:24:09.490 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/debug 2023-11-28T18:24:09.491 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/tracing 2023-11-28T18:24:09.491 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/fs/nfsd 2023-11-28T18:24:09.491 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/fuse/connections 2023-11-28T18:24:09.491 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/config 2023-11-28T18:24:09.491 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:24:09.491 INFO:teuthology.orchestra.run.smithi148.stdout:/run/rpc_pipefs 2023-11-28T18:24:09.491 INFO:teuthology.orchestra.run.smithi148.stdout:/run/user/1000 2023-11-28T18:24:09.492 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph 2023-11-28T18:24:09.494 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-4 2023-11-28T18:24:09.495 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-5 2023-11-28T18:24:09.497 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-6 2023-11-28T18:24:09.498 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-7 2023-11-28T18:24:09.498 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns 2023-11-28T18:24:09.499 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T18:24:09.499 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T18:24:09.499 INFO:teuthology.orchestra.run.smithi148.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-28T18:24:09.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:09.501 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2023-11-28T18:24:09.555 INFO:teuthology.orchestra.run.smithi148.stdout:43 2023-11-28T18:24:09.555 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-28T18:24:09.555 DEBUG:teuthology.orchestra.run.smithi148:> sudo stdin-killer -- python3 -c ' 2023-11-28T18:24:09.556 DEBUG:teuthology.orchestra.run.smithi148:> import glob 2023-11-28T18:24:09.556 DEBUG:teuthology.orchestra.run.smithi148:> import re 2023-11-28T18:24:09.556 DEBUG:teuthology.orchestra.run.smithi148:> import os 2023-11-28T18:24:09.556 DEBUG:teuthology.orchestra.run.smithi148:> import subprocess 2023-11-28T18:24:09.556 DEBUG:teuthology.orchestra.run.smithi148:> 2023-11-28T18:24:09.556 DEBUG:teuthology.orchestra.run.smithi148:> def _find_admin_socket(client_name): 2023-11-28T18:24:09.556 DEBUG:teuthology.orchestra.run.smithi148:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-28T18:24:09.556 DEBUG:teuthology.orchestra.run.smithi148:> files = glob.glob(asok_path) 2023-11-28T18:24:09.556 DEBUG:teuthology.orchestra.run.smithi148:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-28T18:24:09.556 DEBUG:teuthology.orchestra.run.smithi148:> 2023-11-28T18:24:09.556 DEBUG:teuthology.orchestra.run.smithi148:> # Given a non-glob path, it better be there 2023-11-28T18:24:09.556 DEBUG:teuthology.orchestra.run.smithi148:> if "*" not in asok_path: 2023-11-28T18:24:09.556 DEBUG:teuthology.orchestra.run.smithi148:> assert(len(files) == 1) 2023-11-28T18:24:09.557 DEBUG:teuthology.orchestra.run.smithi148:> return files[0] 2023-11-28T18:24:09.557 DEBUG:teuthology.orchestra.run.smithi148:> 2023-11-28T18:24:09.557 DEBUG:teuthology.orchestra.run.smithi148:> for f in files: 2023-11-28T18:24:09.557 DEBUG:teuthology.orchestra.run.smithi148:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T18:24:09.557 DEBUG:teuthology.orchestra.run.smithi148:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T18:24:09.557 DEBUG:teuthology.orchestra.run.smithi148:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T18:24:09.557 DEBUG:teuthology.orchestra.run.smithi148:> contents = proc_f.read() 2023-11-28T18:24:09.557 DEBUG:teuthology.orchestra.run.smithi148:> if mountpoint in contents: 2023-11-28T18:24:09.557 DEBUG:teuthology.orchestra.run.smithi148:> return f 2023-11-28T18:24:09.557 DEBUG:teuthology.orchestra.run.smithi148:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T18:24:09.557 DEBUG:teuthology.orchestra.run.smithi148:> 2023-11-28T18:24:09.557 DEBUG:teuthology.orchestra.run.smithi148:> print(_find_admin_socket("client.1")) 2023-11-28T18:24:09.557 DEBUG:teuthology.orchestra.run.smithi148:> ' 2023-11-28T18:24:09.683 INFO:teuthology.orchestra.run.smithi148.stdout:/var/run/ceph/ceph-client.1.44203.asok 2023-11-28T18:24:09.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:24:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-28T18:24:09.696 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.44203.asok 2023-11-28T18:24:09.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:09.696 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.44203.asok status 2023-11-28T18:24:09.872 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-11-28T18:24:09.873 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": { 2023-11-28T18:24:09.873 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_sha1": "47c1c547372ed5bff7984647dc200bac0e019caf", 2023-11-28T18:24:09.873 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev)", 2023-11-28T18:24:09.873 INFO:teuthology.orchestra.run.smithi148.stdout: "entity_id": "1", 2023-11-28T18:24:09.873 INFO:teuthology.orchestra.run.smithi148.stdout: "hostname": "smithi148", 2023-11-28T18:24:09.873 INFO:teuthology.orchestra.run.smithi148.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-28T18:24:09.873 INFO:teuthology.orchestra.run.smithi148.stdout: "pid": "44203", 2023-11-28T18:24:09.873 INFO:teuthology.orchestra.run.smithi148.stdout: "root": "/" 2023-11-28T18:24:09.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-11-28T18:24:09.873 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_count": 0, 2023-11-28T18:24:09.873 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_pinned_count": 0, 2023-11-28T18:24:09.874 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 4846, 2023-11-28T18:24:09.874 INFO:teuthology.orchestra.run.smithi148.stdout: "inst": { 2023-11-28T18:24:09.874 INFO:teuthology.orchestra.run.smithi148.stdout: "name": { 2023-11-28T18:24:09.874 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client", 2023-11-28T18:24:09.874 INFO:teuthology.orchestra.run.smithi148.stdout: "num": 4846 2023-11-28T18:24:09.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-11-28T18:24:09.874 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2023-11-28T18:24:09.874 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-11-28T18:24:09.874 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.1:0", 2023-11-28T18:24:09.874 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 2817993299 2023-11-28T18:24:09.874 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-11-28T18:24:09.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-11-28T18:24:09.875 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2023-11-28T18:24:09.875 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-11-28T18:24:09.875 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.1:0", 2023-11-28T18:24:09.875 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 2817993299 2023-11-28T18:24:09.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-11-28T18:24:09.875 INFO:teuthology.orchestra.run.smithi148.stdout: "inst_str": "client.4846 192.168.0.1:0/2817993299", 2023-11-28T18:24:09.875 INFO:teuthology.orchestra.run.smithi148.stdout: "addr_str": "192.168.0.1:0/2817993299", 2023-11-28T18:24:09.875 INFO:teuthology.orchestra.run.smithi148.stdout: "inode_count": 1, 2023-11-28T18:24:09.875 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_epoch": 9, 2023-11-28T18:24:09.875 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch": 24, 2023-11-28T18:24:09.876 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch_barrier": 0, 2023-11-28T18:24:09.876 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklisted": false, 2023-11-28T18:24:09.876 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs" 2023-11-28T18:24:09.876 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-11-28T18:24:09.888 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T18:24:10.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.027+0000 7f76a730f640 1 Processor -- start 2023-11-28T18:24:10.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.027+0000 7f76a730f640 1 -- start start 2023-11-28T18:24:10.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.031+0000 7f76a730f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f76a0159a20 0x7f76a0070a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:10.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.031+0000 7f76a730f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76a0070f50 0x7f76a006a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:10.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.031+0000 7f76a730f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a015a020 con 0x7f76a0159640 2023-11-28T18:24:10.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.031+0000 7f76a730f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a015a9c0 con 0x7f76a0159a20 2023-11-28T18:24:10.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.031+0000 7f76a730f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a00713e0 con 0x7f76a0070f50 2023-11-28T18:24:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.031+0000 7f76a48aa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f76a0159a20 0x7f76a0070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.031+0000 7f76a50ab640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f76a0159640 0x7f76a006df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:35816/0 (socket says 172.21.15.105:35816) 2023-11-28T18:24:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.031+0000 7f76a50ab640 1 -- 172.21.15.105:0/2277594788 learned_addr learned my addr 172.21.15.105:0/2277594788 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.031+0000 7f76a58ac640 1 --2- 172.21.15.105:0/2277594788 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76a0070f50 0x7f76a006a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.031+0000 7f76a48aa640 1 -- 172.21.15.105:0/2277594788 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76a0070f50 msgr2=0x7f76a006a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.031+0000 7f76a48aa640 1 --2- 172.21.15.105:0/2277594788 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76a0070f50 0x7f76a006a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.031+0000 7f76a48aa640 1 -- 172.21.15.105:0/2277594788 >> v1:172.21.15.148:6789/0 conn(0x7f76a0159640 legacy=0x7f76a006df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.031+0000 7f7697fff640 1 -- 172.21.15.105:0/2277594788 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2352373779 0 0) 0x7f76a015a020 con 0x7f76a0159640 2023-11-28T18:24:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.031+0000 7f76a48aa640 1 -- 172.21.15.105:0/2277594788 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a015a300 con 0x7f76a0159a20 2023-11-28T18:24:10.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.035+0000 7f76a48aa640 1 --2- 172.21.15.105:0/2277594788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a0159a20 0x7f76a0070a10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f76900097c0 tx=0x7f769001c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19dd888bd6e978a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:10.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.035+0000 7f7697fff640 1 -- 172.21.15.105:0/2277594788 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76900ab600 con 0x7f76a0159a20 2023-11-28T18:24:10.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.035+0000 7f7697fff640 1 -- 172.21.15.105:0/2277594788 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76900a7050 con 0x7f76a0159a20 2023-11-28T18:24:10.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.035+0000 7f7697fff640 1 -- 172.21.15.105:0/2277594788 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76900ab980 con 0x7f76a0159a20 2023-11-28T18:24:10.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.035+0000 7f76a730f640 1 -- 172.21.15.105:0/2277594788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a0159a20 msgr2=0x7f76a0070a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:10.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.035+0000 7f76a730f640 1 --2- 172.21.15.105:0/2277594788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a0159a20 0x7f76a0070a10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f76900097c0 tx=0x7f769001c330 comp rx=0 tx=0).stop 2023-11-28T18:24:10.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.035+0000 7f76a730f640 1 -- 172.21.15.105:0/2277594788 shutdown_connections 2023-11-28T18:24:10.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.035+0000 7f76a730f640 1 --2- 172.21.15.105:0/2277594788 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76a0070f50 0x7f76a006a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:10.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.035+0000 7f76a730f640 1 --2- 172.21.15.105:0/2277594788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a0159a20 0x7f76a0070a10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:10.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.035+0000 7f76a730f640 1 -- 172.21.15.105:0/2277594788 >> 172.21.15.105:0/2277594788 conn(0x7f76a00cf540 msgr2=0x7f76a00cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:10.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.035+0000 7f76a730f640 1 -- 172.21.15.105:0/2277594788 shutdown_connections 2023-11-28T18:24:10.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.035+0000 7f76a730f640 1 -- 172.21.15.105:0/2277594788 wait complete. 2023-11-28T18:24:10.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.035+0000 7f76a730f640 1 Processor -- start 2023-11-28T18:24:10.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.035+0000 7f76a730f640 1 -- start start 2023-11-28T18:24:10.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f76a730f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a0070f50 0x7f76a014c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:10.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f76a730f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76a0159640 0x7f76a006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:10.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f76a730f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76a0159a20 0x7f76a006d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:10.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f76a730f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a014c5d0 con 0x7f76a0159a20 2023-11-28T18:24:10.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f76a730f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a014d6e0 con 0x7f76a0070f50 2023-11-28T18:24:10.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f76a730f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a00b4650 con 0x7f76a0159640 2023-11-28T18:24:10.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f76a58ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76a0159a20 0x7f76a006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:10.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f76a58ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76a0159a20 0x7f76a006d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36426/0 (socket says 172.21.15.105:36426) 2023-11-28T18:24:10.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f76a58ac640 1 -- 172.21.15.105:0/606424835 learned_addr learned my addr 172.21.15.105:0/606424835 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:10.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f76a50ab640 1 --2- 172.21.15.105:0/606424835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a0070f50 0x7f76a014c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f76a48aa640 1 --2- 172.21.15.105:0/606424835 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76a0159640 0x7f76a006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f76a58ac640 1 -- 172.21.15.105:0/606424835 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76a0159640 msgr2=0x7f76a006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f76a58ac640 1 --2- 172.21.15.105:0/606424835 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76a0159640 0x7f76a006ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f76a58ac640 1 -- 172.21.15.105:0/606424835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a0070f50 msgr2=0x7f76a014c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f76a58ac640 1 --2- 172.21.15.105:0/606424835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a0070f50 0x7f76a014c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f76a58ac640 1 -- 172.21.15.105:0/606424835 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7690009420 con 0x7f76a0159a20 2023-11-28T18:24:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f76a58ac640 1 --2- 172.21.15.105:0/606424835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76a0159a20 0x7f76a006d210 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f769800efc0 tx=0x7f769800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f7695ffb640 1 -- 172.21.15.105:0/606424835 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698014480 con 0x7f76a0159a20 2023-11-28T18:24:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f7695ffb640 1 -- 172.21.15.105:0/606424835 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769800f040 con 0x7f76a0159a20 2023-11-28T18:24:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f76a730f640 1 -- 172.21.15.105:0/606424835 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a00b3db0 con 0x7f76a0159a20 2023-11-28T18:24:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f7695ffb640 1 -- 172.21.15.105:0/606424835 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698014820 con 0x7f76a0159a20 2023-11-28T18:24:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f76a730f640 1 -- 172.21.15.105:0/606424835 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a00b4370 con 0x7f76a0159a20 2023-11-28T18:24:10.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.039+0000 7f76a730f640 1 -- 172.21.15.105:0/606424835 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a0067570 con 0x7f76a0159a20 2023-11-28T18:24:10.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.043+0000 7f7695ffb640 1 -- 172.21.15.105:0/606424835 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f7698014980 con 0x7f76a0159a20 2023-11-28T18:24:10.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.043+0000 7f7695ffb640 1 --2- 172.21.15.105:0/606424835 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f76840668d0 0x7f7684068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:10.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.043+0000 7f7695ffb640 1 -- 172.21.15.105:0/606424835 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7698085480 con 0x7f76a0159a20 2023-11-28T18:24:10.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.043+0000 7f76a50ab640 1 --2- 172.21.15.105:0/606424835 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f76840668d0 0x7f7684068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:10.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.043+0000 7f76a50ab640 1 --2- 172.21.15.105:0/606424835 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f76840668d0 0x7f7684068d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f769c007fd0 tx=0x7f769c007d60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:10.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.047+0000 7f7695ffb640 1 -- 172.21.15.105:0/606424835 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7698052470 con 0x7f76a0159a20 2023-11-28T18:24:10.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.231+0000 7f76a730f640 1 -- 172.21.15.105:0/606424835 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f76a0071b00 con 0x7f76a0159a20 2023-11-28T18:24:10.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.231+0000 7f7695ffb640 1 -- 172.21.15.105:0/606424835 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f7698055a90 con 0x7f76a0159a20 2023-11-28T18:24:10.235 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T18:24:10.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.231+0000 7f76a730f640 1 -- 172.21.15.105:0/606424835 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f76840668d0 msgr2=0x7f7684068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:10.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.231+0000 7f76a730f640 1 --2- 172.21.15.105:0/606424835 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f76840668d0 0x7f7684068d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f769c007fd0 tx=0x7f769c007d60 comp rx=0 tx=0).stop 2023-11-28T18:24:10.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.231+0000 7f76a730f640 1 -- 172.21.15.105:0/606424835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76a0159a20 msgr2=0x7f76a006d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:10.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.231+0000 7f76a730f640 1 --2- 172.21.15.105:0/606424835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76a0159a20 0x7f76a006d210 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f769800efc0 tx=0x7f769800c490 comp rx=0 tx=0).stop 2023-11-28T18:24:10.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.231+0000 7f76a730f640 1 -- 172.21.15.105:0/606424835 shutdown_connections 2023-11-28T18:24:10.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.231+0000 7f76a730f640 1 --2- 172.21.15.105:0/606424835 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f76840668d0 0x7f7684068d90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:10.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.231+0000 7f76a730f640 1 --2- 172.21.15.105:0/606424835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76a0159a20 0x7f76a006d210 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:10.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.231+0000 7f76a730f640 1 --2- 172.21.15.105:0/606424835 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76a0159640 0x7f76a006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:10.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.231+0000 7f76a730f640 1 --2- 172.21.15.105:0/606424835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a0070f50 0x7f76a014c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:10.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.231+0000 7f76a730f640 1 -- 172.21.15.105:0/606424835 >> 172.21.15.105:0/606424835 conn(0x7f76a00cf540 msgr2=0x7f76a01572c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:10.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.235+0000 7f76a730f640 1 -- 172.21.15.105:0/606424835 shutdown_connections 2023-11-28T18:24:10.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:10.235+0000 7f76a730f640 1 -- 172.21.15.105:0/606424835 wait complete. 2023-11-28T18:24:10.255 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T18:24:10.256 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T18:24:10.256 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2023-11-28T18:24:10.256 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-28T18:24:10.256 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T18:24:10.256 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T18:24:10.256 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T18:24:10.256 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T18:24:10.256 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-28T18:24:10.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:10.256 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2023-11-28T18:24:10.265 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T18:24:10.266 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T18:24:10.266 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T18:24:10.266 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:10.266 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2023-11-28T18:24:10.266 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:10.266 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:24:10.266 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:24:10.266 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2023-11-28T18:24:10.266 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:24:10.267 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:24:10.267 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2023-11-28T18:24:10.267 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:24:10.267 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:24:10.267 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T18:24:10.267 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:24:10.267 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-28T18:24:10.267 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42369sec preferred_lft 42369sec 2023-11-28T18:24:10.267 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link 2023-11-28T18:24:10.267 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:10.268 INFO:teuthology.orchestra.run.smithi148.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-28T18:24:10.268 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 02:42:8a:56:dd:03 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:24:10.268 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-28T18:24:10.268 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:10.268 INFO:teuthology.orchestra.run.smithi148.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T18:24:10.268 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-28T18:24:10.268 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T18:24:10.269 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:10.269 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-28T18:24:10.269 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:10.269 INFO:teuthology.orchestra.run.smithi148.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T18:24:10.269 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T18:24:10.269 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-28T18:24:10.269 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:10.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:10.270 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T18:24:10.333 INFO:teuthology.orchestra.run.smithi148.stdout:1 2023-11-28T18:24:10.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:10.335 DEBUG:teuthology.orchestra.run.smithi148:> route 2023-11-28T18:24:10.393 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2023-11-28T18:24:10.393 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T18:24:10.393 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-28T18:24:10.393 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T18:24:10.393 INFO:teuthology.orchestra.run.smithi148.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-28T18:24:10.393 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T18:24:10.393 INFO:teuthology.orchestra.run.smithi148.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:24:10.393 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:24:10.394 INFO:teuthology.orchestra.run.smithi148.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:24:10.394 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T18:24:10.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:10.394 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-11-28T18:24:10.394 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-11-28T18:24:10.394 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T18:24:10.394 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T18:24:10.395 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T18:24:10.395 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-11-28T18:24:10.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:10.485 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2023-11-28T18:24:10.495 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T18:24:10.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:10.495 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list-id 2023-11-28T18:24:10.547 INFO:teuthology.orchestra.run.smithi148.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-28T18:24:10.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:10.547 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-11-28T18:24:10.547 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-11-28T18:24:10.547 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T18:24:10.547 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-28T18:24:10.548 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-11-28T18:24:10.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:10.632 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T18:24:10.699 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T18:24:10.699 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T18:24:10.699 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T18:24:10.699 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:10.699 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2023-11-28T18:24:10.699 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:10.699 INFO:teuthology.orchestra.run.smithi148.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T18:24:10.699 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 9a:9e:1b:0b:64:34 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T18:24:10.699 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T18:24:10.699 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:10.700 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::989e:1bff:fe0b:6434/64 scope link 2023-11-28T18:24:10.700 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:10.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:10.701 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T18:24:10.766 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T18:24:10.766 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T18:24:10.767 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T18:24:10.767 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:10.767 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2023-11-28T18:24:10.767 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:10.767 INFO:teuthology.orchestra.run.smithi148.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T18:24:10.767 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 9a:9e:1b:0b:64:34 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T18:24:10.767 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T18:24:10.768 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:10.768 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::989e:1bff:fe0b:6434/64 scope link 2023-11-28T18:24:10.768 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:24:10.769 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-28T18:24:10.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:10.770 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-11-28T18:24:10.770 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-11-28T18:24:10.770 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-28T18:24:10.770 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T18:24:10.770 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-28T18:24:10.770 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-28T18:24:10.770 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-28T18:24:10.770 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-11-28T18:24:10.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:10.905 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-11-28T18:24:10.905 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-11-28T18:24:10.905 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.1 up 2023-11-28T18:24:10.905 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T18:24:10.905 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-11-28T18:24:10.945 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-11-28T18:24:10.945 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:24:10.945 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-28T18:24:11.000 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-28T18:24:11.000 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:24:11.000 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-28T18:24:11.049 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2023-11-28T18:24:11.111 DEBUG:teuthology.orchestra.run.smithi148:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:24:11.163 INFO:teuthology.orchestra.run.smithi148.stdout:/sys 2023-11-28T18:24:11.164 INFO:teuthology.orchestra.run.smithi148.stdout:/proc 2023-11-28T18:24:11.164 INFO:teuthology.orchestra.run.smithi148.stdout:/dev 2023-11-28T18:24:11.164 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/pts 2023-11-28T18:24:11.164 INFO:teuthology.orchestra.run.smithi148.stdout:/run 2023-11-28T18:24:11.164 INFO:teuthology.orchestra.run.smithi148.stdout:/ 2023-11-28T18:24:11.164 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/security 2023-11-28T18:24:11.164 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/shm 2023-11-28T18:24:11.164 INFO:teuthology.orchestra.run.smithi148.stdout:/run/lock 2023-11-28T18:24:11.164 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/cgroup 2023-11-28T18:24:11.164 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/pstore 2023-11-28T18:24:11.164 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/bpf 2023-11-28T18:24:11.165 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:24:11.165 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/hugepages 2023-11-28T18:24:11.165 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/mqueue 2023-11-28T18:24:11.165 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/debug 2023-11-28T18:24:11.165 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/tracing 2023-11-28T18:24:11.165 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/fs/nfsd 2023-11-28T18:24:11.165 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/fuse/connections 2023-11-28T18:24:11.165 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/config 2023-11-28T18:24:11.165 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:24:11.165 INFO:teuthology.orchestra.run.smithi148.stdout:/run/rpc_pipefs 2023-11-28T18:24:11.165 INFO:teuthology.orchestra.run.smithi148.stdout:/run/user/1000 2023-11-28T18:24:11.165 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph 2023-11-28T18:24:11.166 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-4 2023-11-28T18:24:11.166 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-5 2023-11-28T18:24:11.166 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-6 2023-11-28T18:24:11.166 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-7 2023-11-28T18:24:11.166 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns 2023-11-28T18:24:11.166 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T18:24:11.166 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T18:24:11.166 INFO:teuthology.orchestra.run.smithi148.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-28T18:24:11.166 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T18:24:11.166 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T18:24:11.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:11.167 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2023-11-28T18:24:11.218 INFO:teuthology.orchestra.run.smithi148.stdout:43 2023-11-28T18:24:11.218 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2023-11-28T18:24:11.218 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-11-28T18:24:11.265 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2023-11-28T18:24:11.287 DEBUG:teuthology.orchestra.run.smithi148:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:24:11.344 INFO:teuthology.orchestra.run.smithi148.stdout:/sys 2023-11-28T18:24:11.344 INFO:teuthology.orchestra.run.smithi148.stdout:/proc 2023-11-28T18:24:11.344 INFO:teuthology.orchestra.run.smithi148.stdout:/dev 2023-11-28T18:24:11.344 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/pts 2023-11-28T18:24:11.344 INFO:teuthology.orchestra.run.smithi148.stdout:/run 2023-11-28T18:24:11.344 INFO:teuthology.orchestra.run.smithi148.stdout:/ 2023-11-28T18:24:11.344 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/security 2023-11-28T18:24:11.344 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/shm 2023-11-28T18:24:11.344 INFO:teuthology.orchestra.run.smithi148.stdout:/run/lock 2023-11-28T18:24:11.344 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/cgroup 2023-11-28T18:24:11.344 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/pstore 2023-11-28T18:24:11.345 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/bpf 2023-11-28T18:24:11.345 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:24:11.345 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/hugepages 2023-11-28T18:24:11.345 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/mqueue 2023-11-28T18:24:11.345 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/debug 2023-11-28T18:24:11.345 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/tracing 2023-11-28T18:24:11.345 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/fs/nfsd 2023-11-28T18:24:11.345 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/fuse/connections 2023-11-28T18:24:11.345 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/config 2023-11-28T18:24:11.345 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:24:11.345 INFO:teuthology.orchestra.run.smithi148.stdout:/run/rpc_pipefs 2023-11-28T18:24:11.345 INFO:teuthology.orchestra.run.smithi148.stdout:/run/user/1000 2023-11-28T18:24:11.345 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph 2023-11-28T18:24:11.346 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-4 2023-11-28T18:24:11.346 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-5 2023-11-28T18:24:11.346 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-6 2023-11-28T18:24:11.346 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-7 2023-11-28T18:24:11.346 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns 2023-11-28T18:24:11.346 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T18:24:11.346 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T18:24:11.346 INFO:teuthology.orchestra.run.smithi148.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-28T18:24:11.346 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T18:24:11.346 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T18:24:11.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:11.347 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2023-11-28T18:24:11.364 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi148.stderr:2023-11-28T18:24:11.359+0000 7fd672811580 -1 init, newargv = 0x556310562c80 newargc=15 2023-11-28T18:24:11.364 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi148.stderr:ceph-fuse[44317]: starting ceph client 2023-11-28T18:24:11.372 INFO:teuthology.orchestra.run.smithi148.stdout:43 2023-11-28T18:24:11.372 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi148.stderr:ceph-fuse[44317]: starting fuse 2023-11-28T18:24:12.474 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2023-11-28T18:24:12.499 DEBUG:teuthology.orchestra.run.smithi148:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:24:12.553 INFO:teuthology.orchestra.run.smithi148.stdout:/sys 2023-11-28T18:24:12.553 INFO:teuthology.orchestra.run.smithi148.stdout:/proc 2023-11-28T18:24:12.553 INFO:teuthology.orchestra.run.smithi148.stdout:/dev 2023-11-28T18:24:12.553 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/pts 2023-11-28T18:24:12.553 INFO:teuthology.orchestra.run.smithi148.stdout:/run 2023-11-28T18:24:12.553 INFO:teuthology.orchestra.run.smithi148.stdout:/ 2023-11-28T18:24:12.553 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/security 2023-11-28T18:24:12.553 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/shm 2023-11-28T18:24:12.553 INFO:teuthology.orchestra.run.smithi148.stdout:/run/lock 2023-11-28T18:24:12.553 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/cgroup 2023-11-28T18:24:12.553 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/pstore 2023-11-28T18:24:12.554 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/bpf 2023-11-28T18:24:12.554 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:24:12.554 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/hugepages 2023-11-28T18:24:12.554 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/mqueue 2023-11-28T18:24:12.554 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/debug 2023-11-28T18:24:12.554 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/tracing 2023-11-28T18:24:12.554 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/fs/nfsd 2023-11-28T18:24:12.554 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/fuse/connections 2023-11-28T18:24:12.554 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/config 2023-11-28T18:24:12.554 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:24:12.554 INFO:teuthology.orchestra.run.smithi148.stdout:/run/rpc_pipefs 2023-11-28T18:24:12.554 INFO:teuthology.orchestra.run.smithi148.stdout:/run/user/1000 2023-11-28T18:24:12.554 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph 2023-11-28T18:24:12.555 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-4 2023-11-28T18:24:12.555 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-5 2023-11-28T18:24:12.555 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-6 2023-11-28T18:24:12.555 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-7 2023-11-28T18:24:12.555 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns 2023-11-28T18:24:12.555 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T18:24:12.555 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T18:24:12.555 INFO:teuthology.orchestra.run.smithi148.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-28T18:24:12.555 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T18:24:12.555 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T18:24:12.555 INFO:teuthology.orchestra.run.smithi148.stdout:/home/ubuntu/cephtest/mnt.3 2023-11-28T18:24:12.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:12.556 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2023-11-28T18:24:12.607 INFO:teuthology.orchestra.run.smithi148.stdout:43 2023-11-28T18:24:12.607 INFO:teuthology.orchestra.run.smithi148.stdout:44 2023-11-28T18:24:12.607 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2023-11-28T18:24:12.607 DEBUG:teuthology.orchestra.run.smithi148:> sudo stdin-killer -- python3 -c ' 2023-11-28T18:24:12.608 DEBUG:teuthology.orchestra.run.smithi148:> import glob 2023-11-28T18:24:12.608 DEBUG:teuthology.orchestra.run.smithi148:> import re 2023-11-28T18:24:12.608 DEBUG:teuthology.orchestra.run.smithi148:> import os 2023-11-28T18:24:12.608 DEBUG:teuthology.orchestra.run.smithi148:> import subprocess 2023-11-28T18:24:12.608 DEBUG:teuthology.orchestra.run.smithi148:> 2023-11-28T18:24:12.608 DEBUG:teuthology.orchestra.run.smithi148:> def _find_admin_socket(client_name): 2023-11-28T18:24:12.608 DEBUG:teuthology.orchestra.run.smithi148:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-11-28T18:24:12.608 DEBUG:teuthology.orchestra.run.smithi148:> files = glob.glob(asok_path) 2023-11-28T18:24:12.608 DEBUG:teuthology.orchestra.run.smithi148:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-11-28T18:24:12.608 DEBUG:teuthology.orchestra.run.smithi148:> 2023-11-28T18:24:12.608 DEBUG:teuthology.orchestra.run.smithi148:> # Given a non-glob path, it better be there 2023-11-28T18:24:12.608 DEBUG:teuthology.orchestra.run.smithi148:> if "*" not in asok_path: 2023-11-28T18:24:12.608 DEBUG:teuthology.orchestra.run.smithi148:> assert(len(files) == 1) 2023-11-28T18:24:12.609 DEBUG:teuthology.orchestra.run.smithi148:> return files[0] 2023-11-28T18:24:12.609 DEBUG:teuthology.orchestra.run.smithi148:> 2023-11-28T18:24:12.609 DEBUG:teuthology.orchestra.run.smithi148:> for f in files: 2023-11-28T18:24:12.609 DEBUG:teuthology.orchestra.run.smithi148:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T18:24:12.609 DEBUG:teuthology.orchestra.run.smithi148:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T18:24:12.609 DEBUG:teuthology.orchestra.run.smithi148:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T18:24:12.609 DEBUG:teuthology.orchestra.run.smithi148:> contents = proc_f.read() 2023-11-28T18:24:12.609 DEBUG:teuthology.orchestra.run.smithi148:> if mountpoint in contents: 2023-11-28T18:24:12.609 DEBUG:teuthology.orchestra.run.smithi148:> return f 2023-11-28T18:24:12.609 DEBUG:teuthology.orchestra.run.smithi148:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T18:24:12.609 DEBUG:teuthology.orchestra.run.smithi148:> 2023-11-28T18:24:12.609 DEBUG:teuthology.orchestra.run.smithi148:> print(_find_admin_socket("client.3")) 2023-11-28T18:24:12.609 DEBUG:teuthology.orchestra.run.smithi148:> ' 2023-11-28T18:24:12.739 INFO:teuthology.orchestra.run.smithi148.stdout:/var/run/ceph/ceph-client.3.44317.asok 2023-11-28T18:24:12.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-11-28T18:24:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-28T18:24:12.752 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.44317.asok 2023-11-28T18:24:12.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:12.753 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.44317.asok status 2023-11-28T18:24:12.924 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-11-28T18:24:12.924 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": { 2023-11-28T18:24:12.925 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_sha1": "47c1c547372ed5bff7984647dc200bac0e019caf", 2023-11-28T18:24:12.925 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev)", 2023-11-28T18:24:12.925 INFO:teuthology.orchestra.run.smithi148.stdout: "entity_id": "3", 2023-11-28T18:24:12.925 INFO:teuthology.orchestra.run.smithi148.stdout: "hostname": "smithi148", 2023-11-28T18:24:12.925 INFO:teuthology.orchestra.run.smithi148.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-11-28T18:24:12.925 INFO:teuthology.orchestra.run.smithi148.stdout: "pid": "44317", 2023-11-28T18:24:12.925 INFO:teuthology.orchestra.run.smithi148.stdout: "root": "/" 2023-11-28T18:24:12.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-11-28T18:24:12.926 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_count": 0, 2023-11-28T18:24:12.926 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_pinned_count": 0, 2023-11-28T18:24:12.926 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 4858, 2023-11-28T18:24:12.926 INFO:teuthology.orchestra.run.smithi148.stdout: "inst": { 2023-11-28T18:24:12.926 INFO:teuthology.orchestra.run.smithi148.stdout: "name": { 2023-11-28T18:24:12.926 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client", 2023-11-28T18:24:12.926 INFO:teuthology.orchestra.run.smithi148.stdout: "num": 4858 2023-11-28T18:24:12.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-11-28T18:24:12.927 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2023-11-28T18:24:12.927 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-11-28T18:24:12.927 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.2:0", 2023-11-28T18:24:12.927 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 530197223 2023-11-28T18:24:12.927 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-11-28T18:24:12.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-11-28T18:24:12.927 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2023-11-28T18:24:12.928 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-11-28T18:24:12.928 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.2:0", 2023-11-28T18:24:12.928 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 530197223 2023-11-28T18:24:12.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-11-28T18:24:12.928 INFO:teuthology.orchestra.run.smithi148.stdout: "inst_str": "client.4858 192.168.0.2:0/530197223", 2023-11-28T18:24:12.928 INFO:teuthology.orchestra.run.smithi148.stdout: "addr_str": "192.168.0.2:0/530197223", 2023-11-28T18:24:12.930 INFO:teuthology.orchestra.run.smithi148.stdout: "inode_count": 1, 2023-11-28T18:24:12.930 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_epoch": 9, 2023-11-28T18:24:12.930 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch": 24, 2023-11-28T18:24:12.930 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch_barrier": 0, 2023-11-28T18:24:12.930 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklisted": false, 2023-11-28T18:24:12.930 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs" 2023-11-28T18:24:12.930 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-11-28T18:24:12.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:12.941 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2023-11-28T18:24:12.941 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-28T18:24:12.952 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2023-11-28T18:24:12.952 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-28T18:24:12.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:12.953 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T18:24:13.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:13.014 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2023-11-28T18:24:13.015 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.2 2023-11-28T18:24:13.067 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2023-11-28T18:24:13.067 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-11-28T18:24:13.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:13.067 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-28T18:24:13.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:13.131 DEBUG:teuthology.orchestra.run.smithi148:> stat --file-system '--printf=%T 2023-11-28T18:24:13.131 DEBUG:teuthology.orchestra.run.smithi148:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-28T18:24:13.141 INFO:teuthology.orchestra.run.smithi148.stdout:fuseblk 2023-11-28T18:24:13.141 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-28T18:24:13.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:13.141 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-28T18:24:13.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:13.207 DEBUG:teuthology.orchestra.run.smithi148:> stat --file-system '--printf=%T 2023-11-28T18:24:13.207 DEBUG:teuthology.orchestra.run.smithi148:> ' -- /home/ubuntu/cephtest/mnt.3 2023-11-28T18:24:13.259 INFO:teuthology.orchestra.run.smithi148.stdout:fuseblk 2023-11-28T18:24:13.259 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-11-28T18:24:13.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:13.260 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-28T18:24:13.326 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-28T18:24:13.343 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:24:13.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.487+0000 7f56edadd640 1 Processor -- start 2023-11-28T18:24:13.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.487+0000 7f56edadd640 1 -- start start 2023-11-28T18:24:13.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.487+0000 7f56edadd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56e815d300 0x7f56e80d37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:13.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.487+0000 7f56edadd640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e80d0c60 con 0x7f56e80d1d60 2023-11-28T18:24:13.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.487+0000 7f56edadd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e80d0eb0 con 0x7f56e80d3ce0 2023-11-28T18:24:13.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.487+0000 7f56edadd640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e80d1470 con 0x7f56e815d300 2023-11-28T18:24:13.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.487+0000 7f56e7fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f56e80d1d60 0x7f56e80c7510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55818/0 (socket says 172.21.15.105:55818) 2023-11-28T18:24:13.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.487+0000 7f56e7fff640 1 -- 172.21.15.105:0/1999568921 learned_addr learned my addr 172.21.15.105:0/1999568921 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e77fe640 1 --2- 172.21.15.105:0/1999568921 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56e815d300 0x7f56e80d37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e67fc640 1 -- 172.21.15.105:0/1999568921 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4201029005 0 0) 0x7f56e80d0c60 con 0x7f56e80d1d60 2023-11-28T18:24:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e67fc640 1 -- 172.21.15.105:0/1999568921 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56d0003530 con 0x7f56e80d1d60 2023-11-28T18:24:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e67fc640 1 -- 172.21.15.105:0/1999568921 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2538068683 0 0) 0x7f56d0003530 con 0x7f56e80d1d60 2023-11-28T18:24:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e67fc640 1 -- 172.21.15.105:0/1999568921 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56e815d300 msgr2=0x7f56e80d37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e67fc640 1 --2- 172.21.15.105:0/1999568921 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56e815d300 0x7f56e80d37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e67fc640 1 -- 172.21.15.105:0/1999568921 >> v1:172.21.15.148:6789/0 conn(0x7f56e80d3ce0 legacy=0x7f56e80d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e67fc640 1 -- 172.21.15.105:0/1999568921 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e80d1190 con 0x7f56e80d1d60 2023-11-28T18:24:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e67fc640 1 -- 172.21.15.105:0/1999568921 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f56e0001a00 con 0x7f56e80d1d60 2023-11-28T18:24:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e67fc640 1 -- 172.21.15.105:0/1999568921 >> v1:172.21.15.105:6789/0 conn(0x7f56e80d1d60 legacy=0x7f56e80c7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e67fc640 1 --2- 172.21.15.105:0/1999568921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d00039b0 0x7f56d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e67fc640 1 --2- 172.21.15.105:0/1999568921 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56e815d300 0x7f56d00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e67fc640 1 --2- 172.21.15.105:0/1999568921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56d0004d10 0x7f56d0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e67fc640 1 -- 172.21.15.105:0/1999568921 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e80d1470 con 0x7f56d00039b0 2023-11-28T18:24:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e67fc640 1 -- 172.21.15.105:0/1999568921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d000ba40 con 0x7f56d0004d10 2023-11-28T18:24:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e67fc640 1 -- 172.21.15.105:0/1999568921 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d000bc00 con 0x7f56e815d300 2023-11-28T18:24:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e77fe640 1 --2- 172.21.15.105:0/1999568921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d00039b0 0x7f56d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e6ffd640 1 --2- 172.21.15.105:0/1999568921 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56e815d300 0x7f56d00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e7fff640 1 --2- 172.21.15.105:0/1999568921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56d0004d10 0x7f56d0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e77fe640 1 -- 172.21.15.105:0/1999568921 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56e815d300 msgr2=0x7f56d00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e77fe640 1 --2- 172.21.15.105:0/1999568921 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56e815d300 0x7f56d00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e77fe640 1 -- 172.21.15.105:0/1999568921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56d0004d10 msgr2=0x7f56d0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e77fe640 1 --2- 172.21.15.105:0/1999568921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56d0004d10 0x7f56d0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.491+0000 7f56e77fe640 1 -- 172.21.15.105:0/1999568921 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d0003530 con 0x7f56d00039b0 2023-11-28T18:24:13.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.495+0000 7f56e77fe640 1 --2- 172.21.15.105:0/1999568921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d00039b0 0x7f56d0003d90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f56d8009ae0 tx=0x7f56d801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db3b232ea22f20b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:13.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.495+0000 7f56e67fc640 1 -- 172.21.15.105:0/1999568921 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d80a6430 con 0x7f56d00039b0 2023-11-28T18:24:13.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.495+0000 7f56e67fc640 1 -- 172.21.15.105:0/1999568921 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d80a6590 con 0x7f56d00039b0 2023-11-28T18:24:13.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.495+0000 7f56e67fc640 1 -- 172.21.15.105:0/1999568921 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d80a6940 con 0x7f56d00039b0 2023-11-28T18:24:13.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.495+0000 7f56edadd640 1 -- 172.21.15.105:0/1999568921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d00039b0 msgr2=0x7f56d0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:13.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.495+0000 7f56edadd640 1 --2- 172.21.15.105:0/1999568921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d00039b0 0x7f56d0003d90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f56d8009ae0 tx=0x7f56d801ca50 comp rx=0 tx=0).stop 2023-11-28T18:24:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.495+0000 7f56e7fff640 1 -- 172.21.15.105:0/1999568921 reap_dead start 2023-11-28T18:24:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.495+0000 7f56edadd640 1 -- 172.21.15.105:0/1999568921 shutdown_connections 2023-11-28T18:24:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.495+0000 7f56edadd640 1 -- 172.21.15.105:0/1999568921 >> 172.21.15.105:0/1999568921 conn(0x7f56e809c360 msgr2=0x7f56e80d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.495+0000 7f56edadd640 1 -- 172.21.15.105:0/1999568921 shutdown_connections 2023-11-28T18:24:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.495+0000 7f56edadd640 1 -- 172.21.15.105:0/1999568921 wait complete. 2023-11-28T18:24:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.495+0000 7f56edadd640 1 Processor -- start 2023-11-28T18:24:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.495+0000 7f56edadd640 1 -- start start 2023-11-28T18:24:13.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56edadd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56d00039b0 0x7f56e81f9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:13.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56edadd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56e80b8910 0x7f56e80cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:13.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56edadd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56e80cc6b0 0x7f56e815ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:13.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56edadd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e80ccb40 con 0x7f56e80b8910 2023-11-28T18:24:13.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56edadd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e81f6a70 con 0x7f56d00039b0 2023-11-28T18:24:13.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56edadd640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e81f64b0 con 0x7f56e80cc6b0 2023-11-28T18:24:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56e6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56e80b8910 0x7f56e80cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56e6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56e80b8910 0x7f56e80cc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56042/0 (socket says 172.21.15.105:56042) 2023-11-28T18:24:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56e6ffd640 1 -- 172.21.15.105:0/2726763087 learned_addr learned my addr 172.21.15.105:0/2726763087 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56e77fe640 1 --2- 172.21.15.105:0/2726763087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56d00039b0 0x7f56e81f9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56e7fff640 1 --2- 172.21.15.105:0/2726763087 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56e80cc6b0 0x7f56e815ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56e6ffd640 1 -- 172.21.15.105:0/2726763087 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56e80cc6b0 msgr2=0x7f56e815ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56e6ffd640 1 --2- 172.21.15.105:0/2726763087 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56e80cc6b0 0x7f56e815ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56e6ffd640 1 -- 172.21.15.105:0/2726763087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56d00039b0 msgr2=0x7f56e81f9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56e6ffd640 1 --2- 172.21.15.105:0/2726763087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56d00039b0 0x7f56e81f9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:13.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56e6ffd640 1 -- 172.21.15.105:0/2726763087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e81f6790 con 0x7f56e80b8910 2023-11-28T18:24:13.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56e77fe640 1 --2- 172.21.15.105:0/2726763087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56d00039b0 0x7f56e81f9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:13.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56e6ffd640 1 --2- 172.21.15.105:0/2726763087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56e80b8910 0x7f56e80cc170 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f56dc00f770 tx=0x7f56dc00fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:13.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56cffff640 1 -- 172.21.15.105:0/2726763087 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc00fe80 con 0x7f56e80b8910 2023-11-28T18:24:13.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56cffff640 1 -- 172.21.15.105:0/2726763087 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56dc0052e0 con 0x7f56e80b8910 2023-11-28T18:24:13.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56cffff640 1 -- 172.21.15.105:0/2726763087 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc00d730 con 0x7f56e80b8910 2023-11-28T18:24:13.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56edadd640 1 -- 172.21.15.105:0/2726763087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e81f9240 con 0x7f56e80b8910 2023-11-28T18:24:13.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56edadd640 1 -- 172.21.15.105:0/2726763087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e81f9f80 con 0x7f56e80b8910 2023-11-28T18:24:13.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56edadd640 1 -- 172.21.15.105:0/2726763087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e8067570 con 0x7f56e80b8910 2023-11-28T18:24:13.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56cffff640 1 -- 172.21.15.105:0/2726763087 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f56dc00d890 con 0x7f56e80b8910 2023-11-28T18:24:13.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.499+0000 7f56cffff640 1 --2- 172.21.15.105:0/2726763087 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56d4066900 0x7f56d4068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:13.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.503+0000 7f56cffff640 1 -- 172.21.15.105:0/2726763087 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f56dc084700 con 0x7f56e80b8910 2023-11-28T18:24:13.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.503+0000 7f56e77fe640 1 --2- 172.21.15.105:0/2726763087 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56d4066900 0x7f56d4068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:13.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.503+0000 7f56e77fe640 1 --2- 172.21.15.105:0/2726763087 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56d4066900 0x7f56d4068dc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f56d80ad000 tx=0x7f56d80b2040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:13.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.507+0000 7f56cffff640 1 -- 172.21.15.105:0/2726763087 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f56dc0516e0 con 0x7f56e80b8910 2023-11-28T18:24:13.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.655+0000 7f56edadd640 1 -- 172.21.15.105:0/2726763087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56e81f96f0 con 0x7f56e80b8910 2023-11-28T18:24:13.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.655+0000 7f56cffff640 1 -- 172.21.15.105:0/2726763087 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18406 (secure 0 0 0) 0x7f56dc054d00 con 0x7f56e80b8910 2023-11-28T18:24:13.661 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:13.661 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":24,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:23:38.458954+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:24:13.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.659+0000 7f56edadd640 1 -- 172.21.15.105:0/2726763087 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56d4066900 msgr2=0x7f56d4068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.659+0000 7f56edadd640 1 --2- 172.21.15.105:0/2726763087 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56d4066900 0x7f56d4068dc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f56d80ad000 tx=0x7f56d80b2040 comp rx=0 tx=0).stop 2023-11-28T18:24:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.659+0000 7f56edadd640 1 -- 172.21.15.105:0/2726763087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56e80b8910 msgr2=0x7f56e80cc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.659+0000 7f56edadd640 1 --2- 172.21.15.105:0/2726763087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56e80b8910 0x7f56e80cc170 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f56dc00f770 tx=0x7f56dc00fc30 comp rx=0 tx=0).stop 2023-11-28T18:24:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.659+0000 7f56edadd640 1 -- 172.21.15.105:0/2726763087 shutdown_connections 2023-11-28T18:24:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.659+0000 7f56edadd640 1 --2- 172.21.15.105:0/2726763087 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56e80cc6b0 0x7f56e815ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.659+0000 7f56edadd640 1 --2- 172.21.15.105:0/2726763087 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56d4066900 0x7f56d4068dc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.659+0000 7f56edadd640 1 --2- 172.21.15.105:0/2726763087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56e80b8910 0x7f56e80cc170 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.659+0000 7f56edadd640 1 --2- 172.21.15.105:0/2726763087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56d00039b0 0x7f56e81f9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.659+0000 7f56edadd640 1 -- 172.21.15.105:0/2726763087 >> 172.21.15.105:0/2726763087 conn(0x7f56e809c360 msgr2=0x7f56e806b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.659+0000 7f56edadd640 1 -- 172.21.15.105:0/2726763087 shutdown_connections 2023-11-28T18:24:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.659+0000 7f56edadd640 1 -- 172.21.15.105:0/2726763087 wait complete. 2023-11-28T18:24:13.682 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:23:31.485422+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:23:32.274378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T18:23:36.100071+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T18:24:13.682 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:24:13.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.819+0000 7f3bac2c9640 1 Processor -- start 2023-11-28T18:24:13.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.819+0000 7f3bac2c9640 1 -- start start 2023-11-28T18:24:13.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.819+0000 7f3bac2c9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3ba4159f40 0x7f3ba40caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:13.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.819+0000 7f3bac2c9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3ba40ad850 0x7f3ba40701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:13.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.819+0000 7f3bac2c9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba406bf50 con 0x7f3ba40cc760 2023-11-28T18:24:13.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.819+0000 7f3bac2c9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba40ccbc0 con 0x7f3ba4159f40 2023-11-28T18:24:13.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.819+0000 7f3bac2c9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba415a320 con 0x7f3ba40ad850 2023-11-28T18:24:13.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.819+0000 7f3baa065640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3ba4159f40 0x7f3ba40caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:13.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.819+0000 7f3baa065640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3ba4159f40 0x7f3ba40caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56050/0 (socket says 172.21.15.105:56050) 2023-11-28T18:24:13.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.819+0000 7f3baa065640 1 -- 172.21.15.105:0/1045374711 learned_addr learned my addr 172.21.15.105:0/1045374711 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:13.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.819+0000 7f3baa866640 1 --2- 172.21.15.105:0/1045374711 >> v2:172.21.15.148:3300/0 conn(0x7f3ba40ad850 0x7f3ba40701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:13.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.819+0000 7f3baa065640 1 -- 172.21.15.105:0/1045374711 >> v2:172.21.15.148:3300/0 conn(0x7f3ba40ad850 msgr2=0x7f3ba40701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:13.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.819+0000 7f3baa065640 1 --2- 172.21.15.105:0/1045374711 >> v2:172.21.15.148:3300/0 conn(0x7f3ba40ad850 0x7f3ba40701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:13.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.819+0000 7f3baa065640 1 -- 172.21.15.105:0/1045374711 >> v1:172.21.15.148:6789/0 conn(0x7f3ba40cc760 legacy=0x7f3ba40ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:13.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.819+0000 7f3baa065640 1 -- 172.21.15.105:0/1045374711 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba4071380 con 0x7f3ba4159f40 2023-11-28T18:24:13.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.823+0000 7f3baa065640 1 --2- 172.21.15.105:0/1045374711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba4159f40 0x7f3ba40caf60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40b0850 tx=0x7f3ba001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce8f857a26cd67a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:13.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.823+0000 7f3ba9063640 1 -- 172.21.15.105:0/1045374711 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba00ab640 con 0x7f3ba4159f40 2023-11-28T18:24:13.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.823+0000 7f3ba9063640 1 -- 172.21.15.105:0/1045374711 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba00a7070 con 0x7f3ba4159f40 2023-11-28T18:24:13.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.823+0000 7f3ba9063640 1 -- 172.21.15.105:0/1045374711 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba00ab9c0 con 0x7f3ba4159f40 2023-11-28T18:24:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.823+0000 7f3bac2c9640 1 -- 172.21.15.105:0/1045374711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba4159f40 msgr2=0x7f3ba40caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.823+0000 7f3bac2c9640 1 --2- 172.21.15.105:0/1045374711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba4159f40 0x7f3ba40caf60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40b0850 tx=0x7f3ba001c440 comp rx=0 tx=0).stop 2023-11-28T18:24:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.823+0000 7f3bac2c9640 1 -- 172.21.15.105:0/1045374711 shutdown_connections 2023-11-28T18:24:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.823+0000 7f3bac2c9640 1 --2- 172.21.15.105:0/1045374711 >> v2:172.21.15.148:3300/0 conn(0x7f3ba40ad850 0x7f3ba40701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.823+0000 7f3bac2c9640 1 --2- 172.21.15.105:0/1045374711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba4159f40 0x7f3ba40caf60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.823+0000 7f3bac2c9640 1 -- 172.21.15.105:0/1045374711 >> 172.21.15.105:0/1045374711 conn(0x7f3ba40ceb90 msgr2=0x7f3ba40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.823+0000 7f3bac2c9640 1 -- 172.21.15.105:0/1045374711 shutdown_connections 2023-11-28T18:24:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.823+0000 7f3bac2c9640 1 -- 172.21.15.105:0/1045374711 wait complete. 2023-11-28T18:24:13.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3bac2c9640 1 Processor -- start 2023-11-28T18:24:13.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3bac2c9640 1 -- start start 2023-11-28T18:24:13.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3bac2c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ba40ad850 0x7f3ba4158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:13.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3bac2c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba40cc760 0x7f3ba413f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:13.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3bac2c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ba4159f40 0x7f3ba413fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:13.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3bac2c9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba41404e0 con 0x7f3ba40cc760 2023-11-28T18:24:13.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3bac2c9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba41406f0 con 0x7f3ba4159f40 2023-11-28T18:24:13.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3bac2c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba41409d0 con 0x7f3ba40ad850 2023-11-28T18:24:13.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3ba9864640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba40cc760 0x7f3ba413f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:13.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3baa065640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ba40ad850 0x7f3ba4158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:13.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3ba9864640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba40cc760 0x7f3ba413f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56060/0 (socket says 172.21.15.105:56060) 2023-11-28T18:24:13.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3ba9864640 1 -- 172.21.15.105:0/2164741264 learned_addr learned my addr 172.21.15.105:0/2164741264 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:13.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3baa866640 1 --2- 172.21.15.105:0/2164741264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ba4159f40 0x7f3ba413fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:13.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3ba9864640 1 -- 172.21.15.105:0/2164741264 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ba40ad850 msgr2=0x7f3ba4158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:13.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3ba9864640 1 --2- 172.21.15.105:0/2164741264 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ba40ad850 0x7f3ba4158fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:13.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3ba9864640 1 -- 172.21.15.105:0/2164741264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ba4159f40 msgr2=0x7f3ba413fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:13.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3ba9864640 1 --2- 172.21.15.105:0/2164741264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ba4159f40 0x7f3ba413fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:13.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3ba9864640 1 -- 172.21.15.105:0/2164741264 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba00094d0 con 0x7f3ba40cc760 2023-11-28T18:24:13.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3ba9864640 1 --2- 172.21.15.105:0/2164741264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba40cc760 0x7f3ba413f890 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400c3c0 tx=0x7f3b9400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:13.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3b9affd640 1 -- 172.21.15.105:0/2164741264 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b940052d0 con 0x7f3ba40cc760 2023-11-28T18:24:13.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3b9affd640 1 -- 172.21.15.105:0/2164741264 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b94005430 con 0x7f3ba40cc760 2023-11-28T18:24:13.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3b9affd640 1 -- 172.21.15.105:0/2164741264 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94003840 con 0x7f3ba40cc760 2023-11-28T18:24:13.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3bac2c9640 1 -- 172.21.15.105:0/2164741264 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba4140cb0 con 0x7f3ba40cc760 2023-11-28T18:24:13.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3bac2c9640 1 -- 172.21.15.105:0/2164741264 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba41594f0 con 0x7f3ba40cc760 2023-11-28T18:24:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.827+0000 7f3bac2c9640 1 -- 172.21.15.105:0/2164741264 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba4067570 con 0x7f3ba40cc760 2023-11-28T18:24:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.831+0000 7f3b9affd640 1 -- 172.21.15.105:0/2164741264 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3b940039a0 con 0x7f3ba40cc760 2023-11-28T18:24:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.831+0000 7f3b9affd640 1 --2- 172.21.15.105:0/2164741264 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3b880668d0 0x7f3b88068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.831+0000 7f3b9affd640 1 -- 172.21.15.105:0/2164741264 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3b94083be0 con 0x7f3ba40cc760 2023-11-28T18:24:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.831+0000 7f3baa065640 1 --2- 172.21.15.105:0/2164741264 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3b880668d0 0x7f3b88068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.831+0000 7f3baa065640 1 --2- 172.21.15.105:0/2164741264 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3b880668d0 0x7f3b88068d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00af000 tx=0x7f3ba00b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.835+0000 7f3b9affd640 1 -- 172.21.15.105:0/2164741264 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3b94050bd0 con 0x7f3ba40cc760 2023-11-28T18:24:13.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.991+0000 7f3bac2c9640 1 -- 172.21.15.105:0/2164741264 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3ba4141160 con 0x7f3ba40cc760 2023-11-28T18:24:13.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.991+0000 7f3b9affd640 1 -- 172.21.15.105:0/2164741264 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3b940170a0 con 0x7f3ba40cc760 2023-11-28T18:24:13.995 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:24:13.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.991+0000 7f3bac2c9640 1 -- 172.21.15.105:0/2164741264 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3b880668d0 msgr2=0x7f3b88068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.991+0000 7f3bac2c9640 1 --2- 172.21.15.105:0/2164741264 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3b880668d0 0x7f3b88068d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00af000 tx=0x7f3ba00b6040 comp rx=0 tx=0).stop 2023-11-28T18:24:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.991+0000 7f3bac2c9640 1 -- 172.21.15.105:0/2164741264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba40cc760 msgr2=0x7f3ba413f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.991+0000 7f3bac2c9640 1 --2- 172.21.15.105:0/2164741264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba40cc760 0x7f3ba413f890 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400c3c0 tx=0x7f3b9400c880 comp rx=0 tx=0).stop 2023-11-28T18:24:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.995+0000 7f3bac2c9640 1 -- 172.21.15.105:0/2164741264 shutdown_connections 2023-11-28T18:24:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.995+0000 7f3bac2c9640 1 --2- 172.21.15.105:0/2164741264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ba4159f40 0x7f3ba413fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.995+0000 7f3bac2c9640 1 --2- 172.21.15.105:0/2164741264 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3b880668d0 0x7f3b88068d90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.995+0000 7f3bac2c9640 1 --2- 172.21.15.105:0/2164741264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba40cc760 0x7f3ba413f890 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.995+0000 7f3bac2c9640 1 --2- 172.21.15.105:0/2164741264 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ba40ad850 0x7f3ba4158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.995+0000 7f3bac2c9640 1 -- 172.21.15.105:0/2164741264 >> 172.21.15.105:0/2164741264 conn(0x7f3ba40ceb90 msgr2=0x7f3ba40ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.995+0000 7f3bac2c9640 1 -- 172.21.15.105:0/2164741264 shutdown_connections 2023-11-28T18:24:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:13.995+0000 7f3bac2c9640 1 -- 172.21.15.105:0/2164741264 wait complete. 2023-11-28T18:24:14.016 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:24:14.163 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.159+0000 7faac1b61640 1 Processor -- start 2023-11-28T18:24:14.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.159+0000 7faac1b61640 1 -- start start 2023-11-28T18:24:14.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.159+0000 7faac1b61640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7faabc06c070 0x7faabc06de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:14.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.159+0000 7faac1b61640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faabc0cc8e0 0x7faabc0ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:14.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.159+0000 7faac1b61640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faabc06e3b0 con 0x7faabc15e9f0 2023-11-28T18:24:14.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.159+0000 7faac1b61640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faabc06c500 con 0x7faabc06c070 2023-11-28T18:24:14.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.159+0000 7faac1b61640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faabc06c7e0 con 0x7faabc0cc8e0 2023-11-28T18:24:14.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.159+0000 7faabbfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7faabc06c070 0x7faabc06de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:14.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.159+0000 7faabbfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7faabc06c070 0x7faabc06de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56062/0 (socket says 172.21.15.105:56062) 2023-11-28T18:24:14.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.159+0000 7faabbfff640 1 -- 172.21.15.105:0/196009685 learned_addr learned my addr 172.21.15.105:0/196009685 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:14.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.163+0000 7faac1360640 1 --2- 172.21.15.105:0/196009685 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faabc0cc8e0 0x7faabc0ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:14.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.163+0000 7faabbfff640 1 -- 172.21.15.105:0/196009685 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faabc0cc8e0 msgr2=0x7faabc0ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:14.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.163+0000 7faabbfff640 1 --2- 172.21.15.105:0/196009685 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faabc0cc8e0 0x7faabc0ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.163+0000 7faabbfff640 1 -- 172.21.15.105:0/196009685 >> v1:172.21.15.148:6789/0 conn(0x7faabc15e9f0 legacy=0x7faabc15edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:14.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.163+0000 7faabbfff640 1 -- 172.21.15.105:0/196009685 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faabc06cac0 con 0x7faabc06c070 2023-11-28T18:24:14.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.163+0000 7faabb7fe640 1 -- 172.21.15.105:0/196009685 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3476727914 0 0) 0x7faab0001120 con 0x7faabc15e9f0 2023-11-28T18:24:14.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.163+0000 7faabbfff640 1 --2- 172.21.15.105:0/196009685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faabc06c070 0x7faabc06de70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faaa8009870 tx=0x7faaa801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7512d75c7bdf376 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:14.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.163+0000 7faabb7fe640 1 -- 172.21.15.105:0/196009685 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa80a7070 con 0x7faabc06c070 2023-11-28T18:24:14.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.163+0000 7faabb7fe640 1 -- 172.21.15.105:0/196009685 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa80ab430 con 0x7faabc06c070 2023-11-28T18:24:14.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.163+0000 7faabb7fe640 1 -- 172.21.15.105:0/196009685 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa80ab7c0 con 0x7faabc06c070 2023-11-28T18:24:14.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.163+0000 7faac1b61640 1 -- 172.21.15.105:0/196009685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faabc06c070 msgr2=0x7faabc06de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:14.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.163+0000 7faac1b61640 1 --2- 172.21.15.105:0/196009685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faabc06c070 0x7faabc06de70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faaa8009870 tx=0x7faaa801c540 comp rx=0 tx=0).stop 2023-11-28T18:24:14.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faac1b61640 1 -- 172.21.15.105:0/196009685 shutdown_connections 2023-11-28T18:24:14.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faac1b61640 1 --2- 172.21.15.105:0/196009685 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faabc0cc8e0 0x7faabc0ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faac1b61640 1 --2- 172.21.15.105:0/196009685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faabc06c070 0x7faabc06de70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faac1b61640 1 -- 172.21.15.105:0/196009685 >> 172.21.15.105:0/196009685 conn(0x7faabc0ce9e0 msgr2=0x7faabc0cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:14.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faac1b61640 1 -- 172.21.15.105:0/196009685 shutdown_connections 2023-11-28T18:24:14.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faac1b61640 1 -- 172.21.15.105:0/196009685 wait complete. 2023-11-28T18:24:14.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faac1b61640 1 Processor -- start 2023-11-28T18:24:14.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faac1b61640 1 -- start start 2023-11-28T18:24:14.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faac1b61640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faabc0cc8e0 0x7faabc14d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:14.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faac1b61640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faabc15e9f0 0x7faabc0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:14.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faac1b61640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faabc0ca780 0x7faabc0c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:14.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faac1b61640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc0b2b30 con 0x7faabc0ca780 2023-11-28T18:24:14.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faac1b61640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc0b1a90 con 0x7faabc0cc8e0 2023-11-28T18:24:14.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faac1b61640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faabc0b1d70 con 0x7faabc15e9f0 2023-11-28T18:24:14.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faac1360640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faabc0ca780 0x7faabc0c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:14.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faac1360640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faabc0ca780 0x7faabc0c7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56076/0 (socket says 172.21.15.105:56076) 2023-11-28T18:24:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faac1360640 1 -- 172.21.15.105:0/3455939996 learned_addr learned my addr 172.21.15.105:0/3455939996 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faabbfff640 1 --2- 172.21.15.105:0/3455939996 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faabc15e9f0 0x7faabc0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faac0b5f640 1 --2- 172.21.15.105:0/3455939996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faabc0cc8e0 0x7faabc14d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faabbfff640 1 -- 172.21.15.105:0/3455939996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faabc0cc8e0 msgr2=0x7faabc14d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faabbfff640 1 --2- 172.21.15.105:0/3455939996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faabc0cc8e0 0x7faabc14d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faabbfff640 1 -- 172.21.15.105:0/3455939996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faabc0ca780 msgr2=0x7faabc0c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faabbfff640 1 --2- 172.21.15.105:0/3455939996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faabc0ca780 0x7faabc0c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faabbfff640 1 -- 172.21.15.105:0/3455939996 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa80094d0 con 0x7faabc15e9f0 2023-11-28T18:24:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.167+0000 7faac1360640 1 --2- 172.21.15.105:0/3455939996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faabc0ca780 0x7faabc0c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:24:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.171+0000 7faabbfff640 1 --2- 172.21.15.105:0/3455939996 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faabc15e9f0 0x7faabc0ca240 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faaa80ac000 tx=0x7faaa801ad80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.171+0000 7faab97fa640 1 -- 172.21.15.105:0/3455939996 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa8004220 con 0x7faabc15e9f0 2023-11-28T18:24:14.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.171+0000 7faab97fa640 1 -- 172.21.15.105:0/3455939996 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa8004380 con 0x7faabc15e9f0 2023-11-28T18:24:14.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.171+0000 7faab97fa640 1 -- 172.21.15.105:0/3455939996 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa8003840 con 0x7faabc15e9f0 2023-11-28T18:24:14.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.171+0000 7faac1b61640 1 -- 172.21.15.105:0/3455939996 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faabc06d720 con 0x7faabc15e9f0 2023-11-28T18:24:14.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.171+0000 7faac1b61640 1 -- 172.21.15.105:0/3455939996 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faabc0b2330 con 0x7faabc15e9f0 2023-11-28T18:24:14.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.171+0000 7faac1b61640 1 -- 172.21.15.105:0/3455939996 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faabc0710b0 con 0x7faabc15e9f0 2023-11-28T18:24:14.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.171+0000 7faab97fa640 1 -- 172.21.15.105:0/3455939996 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7faaa80039a0 con 0x7faabc15e9f0 2023-11-28T18:24:14.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.171+0000 7faab97fa640 1 --2- 172.21.15.105:0/3455939996 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faaa00668d0 0x7faaa0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:14.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.171+0000 7faab97fa640 1 -- 172.21.15.105:0/3455939996 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6721+0+0 (secure 0 0 0) 0x7faaa811e3b0 con 0x7faabc15e9f0 2023-11-28T18:24:14.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.171+0000 7faac0b5f640 1 --2- 172.21.15.105:0/3455939996 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faaa00668d0 0x7faaa0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:14.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.175+0000 7faac0b5f640 1 --2- 172.21.15.105:0/3455939996 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faaa00668d0 0x7faaa0068d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faab0007fd0 tx=0x7faab0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:14.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.175+0000 7faab97fa640 1 -- 172.21.15.105:0/3455939996 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faaa80eb290 con 0x7faabc15e9f0 2023-11-28T18:24:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.327+0000 7faac1b61640 1 -- 172.21.15.105:0/3455939996 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faabc067e30 con 0x7faabc15e9f0 2023-11-28T18:24:14.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.327+0000 7faab97fa640 1 -- 172.21.15.105:0/3455939996 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7faabc067e30 con 0x7faabc15e9f0 2023-11-28T18:24:14.330 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:14.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.327+0000 7faac1b61640 1 -- 172.21.15.105:0/3455939996 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faaa00668d0 msgr2=0x7faaa0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:14.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.327+0000 7faac1b61640 1 --2- 172.21.15.105:0/3455939996 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faaa00668d0 0x7faaa0068d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faab0007fd0 tx=0x7faab0009040 comp rx=0 tx=0).stop 2023-11-28T18:24:14.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.327+0000 7faac1b61640 1 -- 172.21.15.105:0/3455939996 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faabc15e9f0 msgr2=0x7faabc0ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:14.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.327+0000 7faac1b61640 1 --2- 172.21.15.105:0/3455939996 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faabc15e9f0 0x7faabc0ca240 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faaa80ac000 tx=0x7faaa801ad80 comp rx=0 tx=0).stop 2023-11-28T18:24:14.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.327+0000 7faac1b61640 1 -- 172.21.15.105:0/3455939996 shutdown_connections 2023-11-28T18:24:14.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.327+0000 7faac1b61640 1 --2- 172.21.15.105:0/3455939996 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faaa00668d0 0x7faaa0068d90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.327+0000 7faac1b61640 1 --2- 172.21.15.105:0/3455939996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faabc0ca780 0x7faabc0c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.327+0000 7faac1b61640 1 --2- 172.21.15.105:0/3455939996 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faabc15e9f0 0x7faabc0ca240 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.327+0000 7faac1b61640 1 --2- 172.21.15.105:0/3455939996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faabc0cc8e0 0x7faabc14d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.327+0000 7faac1b61640 1 -- 172.21.15.105:0/3455939996 >> 172.21.15.105:0/3455939996 conn(0x7faabc0ce9e0 msgr2=0x7faabc143a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:14.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.327+0000 7faac1b61640 1 -- 172.21.15.105:0/3455939996 shutdown_connections 2023-11-28T18:24:14.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.327+0000 7faac1b61640 1 -- 172.21.15.105:0/3455939996 wait complete. 2023-11-28T18:24:14.352 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:24:14.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.487+0000 7fd165c9a640 1 Processor -- start 2023-11-28T18:24:14.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.487+0000 7fd165c9a640 1 -- start start 2023-11-28T18:24:14.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.491+0000 7fd165c9a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd160159f40 0x7fd1600caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:14.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.491+0000 7fd165c9a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd16006bf50 con 0x7fd1600ad850 2023-11-28T18:24:14.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.491+0000 7fd165c9a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1600ccbc0 con 0x7fd1600cc760 2023-11-28T18:24:14.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.491+0000 7fd165c9a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16015a320 con 0x7fd160159f40 2023-11-28T18:24:14.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.491+0000 7fd15ffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd1600ad850 0x7fd1600701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55820/0 (socket says 172.21.15.105:55820) 2023-11-28T18:24:14.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.491+0000 7fd15ffff640 1 -- 172.21.15.105:0/2484952936 learned_addr learned my addr 172.21.15.105:0/2484952936 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:14.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.491+0000 7fd15f7fe640 1 --2- 172.21.15.105:0/2484952936 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd160159f40 0x7fd1600caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:14.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.491+0000 7fd15e7fc640 1 -- 172.21.15.105:0/2484952936 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2246606083 0 0) 0x7fd16006bf50 con 0x7fd1600ad850 2023-11-28T18:24:14.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.491+0000 7fd15e7fc640 1 -- 172.21.15.105:0/2484952936 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd148003530 con 0x7fd1600ad850 2023-11-28T18:24:14.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.491+0000 7fd15e7fc640 1 -- 172.21.15.105:0/2484952936 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2196457508 0 0) 0x7fd1600ccbc0 con 0x7fd1600cc760 2023-11-28T18:24:14.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.491+0000 7fd15e7fc640 1 -- 172.21.15.105:0/2484952936 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd16006bf50 con 0x7fd1600cc760 2023-11-28T18:24:14.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.491+0000 7fd15f7fe640 1 -- 172.21.15.105:0/2484952936 >> v1:172.21.15.148:6789/0 conn(0x7fd1600cc760 legacy=0x7fd1600ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:14.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.491+0000 7fd15f7fe640 1 -- 172.21.15.105:0/2484952936 >> v1:172.21.15.105:6789/0 conn(0x7fd1600ad850 legacy=0x7fd1600701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:14.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.491+0000 7fd15f7fe640 1 -- 172.21.15.105:0/2484952936 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd160071380 con 0x7fd160159f40 2023-11-28T18:24:14.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.495+0000 7fd15f7fe640 1 --2- 172.21.15.105:0/2484952936 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd160159f40 0x7fd1600caf60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd1500097f0 tx=0x7fd15001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fec58ff33dc90e5b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:14.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.495+0000 7fd15e7fc640 1 -- 172.21.15.105:0/2484952936 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1500ab600 con 0x7fd160159f40 2023-11-28T18:24:14.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.495+0000 7fd15e7fc640 1 -- 172.21.15.105:0/2484952936 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1500ab760 con 0x7fd160159f40 2023-11-28T18:24:14.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.495+0000 7fd15e7fc640 1 -- 172.21.15.105:0/2484952936 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1500abaf0 con 0x7fd160159f40 2023-11-28T18:24:14.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.495+0000 7fd165c9a640 1 -- 172.21.15.105:0/2484952936 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd160159f40 msgr2=0x7fd1600caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:14.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.495+0000 7fd165c9a640 1 --2- 172.21.15.105:0/2484952936 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd160159f40 0x7fd1600caf60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd1500097f0 tx=0x7fd15001c400 comp rx=0 tx=0).stop 2023-11-28T18:24:14.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.495+0000 7fd165c9a640 1 -- 172.21.15.105:0/2484952936 shutdown_connections 2023-11-28T18:24:14.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.495+0000 7fd165c9a640 1 --2- 172.21.15.105:0/2484952936 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd160159f40 0x7fd1600caf60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.495+0000 7fd165c9a640 1 -- 172.21.15.105:0/2484952936 >> 172.21.15.105:0/2484952936 conn(0x7fd1600ceb90 msgr2=0x7fd1600cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:14.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.495+0000 7fd165c9a640 1 -- 172.21.15.105:0/2484952936 shutdown_connections 2023-11-28T18:24:14.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.495+0000 7fd165c9a640 1 -- 172.21.15.105:0/2484952936 wait complete. 2023-11-28T18:24:14.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd165c9a640 1 Processor -- start 2023-11-28T18:24:14.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd165c9a640 1 -- start start 2023-11-28T18:24:14.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd165c9a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1600ad850 0x7fd160159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:14.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd165c9a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1600cc760 0x7fd16009aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:14.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd165c9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd160159f40 0x7fd16015c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:14.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd165c9a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16009b550 con 0x7fd160159f40 2023-11-28T18:24:14.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd165c9a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16009b830 con 0x7fd1600cc760 2023-11-28T18:24:14.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd165c9a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16009bb10 con 0x7fd1600ad850 2023-11-28T18:24:14.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd15ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd160159f40 0x7fd16015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:14.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd15ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd160159f40 0x7fd16015c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56086/0 (socket says 172.21.15.105:56086) 2023-11-28T18:24:14.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd15ffff640 1 -- 172.21.15.105:0/1147003742 learned_addr learned my addr 172.21.15.105:0/1147003742 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:14.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd15effd640 1 --2- 172.21.15.105:0/1147003742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1600cc760 0x7fd16009aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:14.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd15f7fe640 1 --2- 172.21.15.105:0/1147003742 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1600ad850 0x7fd160159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:14.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd15ffff640 1 -- 172.21.15.105:0/1147003742 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1600ad850 msgr2=0x7fd160159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:14.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd15ffff640 1 --2- 172.21.15.105:0/1147003742 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1600ad850 0x7fd160159360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd15ffff640 1 -- 172.21.15.105:0/1147003742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1600cc760 msgr2=0x7fd16009aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:14.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd15ffff640 1 --2- 172.21.15.105:0/1147003742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1600cc760 0x7fd16009aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd15ffff640 1 -- 172.21.15.105:0/1147003742 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd150009420 con 0x7fd160159f40 2023-11-28T18:24:14.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd15ffff640 1 --2- 172.21.15.105:0/1147003742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd160159f40 0x7fd16015c190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd15800cb40 tx=0x7fd15800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:14.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd147fff640 1 -- 172.21.15.105:0/1147003742 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15800aca0 con 0x7fd160159f40 2023-11-28T18:24:14.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd147fff640 1 -- 172.21.15.105:0/1147003742 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15800ae00 con 0x7fd160159f40 2023-11-28T18:24:14.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd165c9a640 1 -- 172.21.15.105:0/1147003742 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd16009bdf0 con 0x7fd160159f40 2023-11-28T18:24:14.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd147fff640 1 -- 172.21.15.105:0/1147003742 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd158012600 con 0x7fd160159f40 2023-11-28T18:24:14.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.499+0000 7fd165c9a640 1 -- 172.21.15.105:0/1147003742 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1600b19b0 con 0x7fd160159f40 2023-11-28T18:24:14.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.503+0000 7fd165c9a640 1 -- 172.21.15.105:0/1147003742 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd160067570 con 0x7fd160159f40 2023-11-28T18:24:14.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.503+0000 7fd147fff640 1 -- 172.21.15.105:0/1147003742 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd158007070 con 0x7fd160159f40 2023-11-28T18:24:14.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.503+0000 7fd147fff640 1 --2- 172.21.15.105:0/1147003742 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd14c066900 0x7fd14c068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:14.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.503+0000 7fd147fff640 1 -- 172.21.15.105:0/1147003742 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fd158084780 con 0x7fd160159f40 2023-11-28T18:24:14.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.503+0000 7fd15f7fe640 1 --2- 172.21.15.105:0/1147003742 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd14c066900 0x7fd14c068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:14.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.503+0000 7fd15f7fe640 1 --2- 172.21.15.105:0/1147003742 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd14c066900 0x7fd14c068dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd1500af000 tx=0x7fd1500a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:14.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.507+0000 7fd147fff640 1 -- 172.21.15.105:0/1147003742 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd1580515e0 con 0x7fd160159f40 2023-11-28T18:24:14.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.655+0000 7fd165c9a640 1 -- 172.21.15.105:0/1147003742 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd16015a6d0 con 0x7fd160159f40 2023-11-28T18:24:14.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.655+0000 7fd147fff640 1 -- 172.21.15.105:0/1147003742 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd158054c00 con 0x7fd160159f40 2023-11-28T18:24:14.661 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:24:14.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.659+0000 7fd165c9a640 1 -- 172.21.15.105:0/1147003742 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd14c066900 msgr2=0x7fd14c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:14.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.659+0000 7fd165c9a640 1 --2- 172.21.15.105:0/1147003742 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd14c066900 0x7fd14c068dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd1500af000 tx=0x7fd1500a7000 comp rx=0 tx=0).stop 2023-11-28T18:24:14.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.659+0000 7fd165c9a640 1 -- 172.21.15.105:0/1147003742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd160159f40 msgr2=0x7fd16015c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:14.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.659+0000 7fd165c9a640 1 --2- 172.21.15.105:0/1147003742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd160159f40 0x7fd16015c190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd15800cb40 tx=0x7fd15800a060 comp rx=0 tx=0).stop 2023-11-28T18:24:14.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.659+0000 7fd165c9a640 1 -- 172.21.15.105:0/1147003742 shutdown_connections 2023-11-28T18:24:14.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.659+0000 7fd165c9a640 1 --2- 172.21.15.105:0/1147003742 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd14c066900 0x7fd14c068dc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.659+0000 7fd165c9a640 1 --2- 172.21.15.105:0/1147003742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd160159f40 0x7fd16015c190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.659+0000 7fd165c9a640 1 --2- 172.21.15.105:0/1147003742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1600cc760 0x7fd16009aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.659+0000 7fd165c9a640 1 --2- 172.21.15.105:0/1147003742 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1600ad850 0x7fd160159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.659+0000 7fd165c9a640 1 -- 172.21.15.105:0/1147003742 >> 172.21.15.105:0/1147003742 conn(0x7fd1600ceb90 msgr2=0x7fd1600ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:14.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.659+0000 7fd165c9a640 1 -- 172.21.15.105:0/1147003742 shutdown_connections 2023-11-28T18:24:14.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.659+0000 7fd165c9a640 1 -- 172.21.15.105:0/1147003742 wait complete. 2023-11-28T18:24:14.682 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T18:24:14.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.819+0000 7f00e7372640 1 Processor -- start 2023-11-28T18:24:14.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.819+0000 7f00e7372640 1 -- start start 2023-11-28T18:24:14.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.819+0000 7f00e7372640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f00e00ad9f0 0x7f00e00ade80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:14.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.819+0000 7f00e7372640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00e0162820 con 0x7f00e00cc7d0 2023-11-28T18:24:14.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.819+0000 7f00e7372640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00e0161c10 con 0x7f00e006d610 2023-11-28T18:24:14.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.819+0000 7f00e7372640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00e00ae3c0 con 0x7f00e00ad9f0 2023-11-28T18:24:14.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.819+0000 7f00e590f640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f00e00cc7d0 0x7f00e00ccbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55830/0 (socket says 172.21.15.105:55830) 2023-11-28T18:24:14.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.819+0000 7f00e590f640 1 -- 172.21.15.105:0/382595961 learned_addr learned my addr 172.21.15.105:0/382595961 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:14.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.819+0000 7f00e490d640 1 --2- 172.21.15.105:0/382595961 >> v2:172.21.15.148:3300/0 conn(0x7f00e00ad9f0 0x7f00e00ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:14.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.819+0000 7f00d7fff640 1 -- 172.21.15.105:0/382595961 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549717751 0 0) 0x7f00e0162820 con 0x7f00e00cc7d0 2023-11-28T18:24:14.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.819+0000 7f00d7fff640 1 -- 172.21.15.105:0/382595961 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00cc003530 con 0x7f00e00cc7d0 2023-11-28T18:24:14.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.819+0000 7f00d7fff640 1 -- 172.21.15.105:0/382595961 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 693596065 0 0) 0x7f00e0161c10 con 0x7f00e006d610 2023-11-28T18:24:14.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.819+0000 7f00d7fff640 1 -- 172.21.15.105:0/382595961 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00e0162820 con 0x7f00e006d610 2023-11-28T18:24:14.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.819+0000 7f00e490d640 1 -- 172.21.15.105:0/382595961 >> v1:172.21.15.148:6789/0 conn(0x7f00e006d610 legacy=0x7f00e00cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:14.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.819+0000 7f00e490d640 1 -- 172.21.15.105:0/382595961 >> v1:172.21.15.105:6789/0 conn(0x7f00e00cc7d0 legacy=0x7f00e00ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:14.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.819+0000 7f00e490d640 1 -- 172.21.15.105:0/382595961 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e0161ef0 con 0x7f00e00ad9f0 2023-11-28T18:24:14.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.823+0000 7f00e490d640 1 --2- 172.21.15.105:0/382595961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00e00ad9f0 0x7f00e00ade80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f00d00097c0 tx=0x7f00d001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=251cf2678a69b1c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:14.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.823+0000 7f00d7fff640 1 -- 172.21.15.105:0/382595961 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d00ab650 con 0x7f00e00ad9f0 2023-11-28T18:24:14.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.823+0000 7f00d7fff640 1 -- 172.21.15.105:0/382595961 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d00ab7b0 con 0x7f00e00ad9f0 2023-11-28T18:24:14.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.823+0000 7f00d7fff640 1 -- 172.21.15.105:0/382595961 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d00abb20 con 0x7f00e00ad9f0 2023-11-28T18:24:14.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.823+0000 7f00e7372640 1 -- 172.21.15.105:0/382595961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00e00ad9f0 msgr2=0x7f00e00ade80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:14.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.823+0000 7f00e7372640 1 --2- 172.21.15.105:0/382595961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00e00ad9f0 0x7f00e00ade80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f00d00097c0 tx=0x7f00d001c450 comp rx=0 tx=0).stop 2023-11-28T18:24:14.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.823+0000 7f00e7372640 1 -- 172.21.15.105:0/382595961 shutdown_connections 2023-11-28T18:24:14.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.823+0000 7f00e7372640 1 --2- 172.21.15.105:0/382595961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00e00ad9f0 0x7f00e00ade80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.823+0000 7f00e7372640 1 -- 172.21.15.105:0/382595961 >> 172.21.15.105:0/382595961 conn(0x7f00e00ceb90 msgr2=0x7f00e00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:14.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.823+0000 7f00e7372640 1 -- 172.21.15.105:0/382595961 shutdown_connections 2023-11-28T18:24:14.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.823+0000 7f00e7372640 1 -- 172.21.15.105:0/382595961 wait complete. 2023-11-28T18:24:14.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00e7372640 1 Processor -- start 2023-11-28T18:24:14.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00e7372640 1 -- start start 2023-11-28T18:24:14.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00e7372640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00e006d610 0x7f00e009c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:14.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00e7372640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00e00ad9f0 0x7f00e0160b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:14.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00e7372640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00e00cc7d0 0x7f00e01610d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:14.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00e7372640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e009c900 con 0x7f00e006d610 2023-11-28T18:24:14.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00e7372640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e015d580 con 0x7f00e00cc7d0 2023-11-28T18:24:14.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00e7372640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e015c2e0 con 0x7f00e00ad9f0 2023-11-28T18:24:14.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00e510e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00e006d610 0x7f00e009c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:14.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00e510e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00e006d610 0x7f00e009c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56088/0 (socket says 172.21.15.105:56088) 2023-11-28T18:24:14.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00e510e640 1 -- 172.21.15.105:0/3969175633 learned_addr learned my addr 172.21.15.105:0/3969175633 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:14.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00e590f640 1 --2- 172.21.15.105:0/3969175633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00e00cc7d0 0x7f00e01610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:14.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00e490d640 1 --2- 172.21.15.105:0/3969175633 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00e00ad9f0 0x7f00e0160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:14.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00e510e640 1 -- 172.21.15.105:0/3969175633 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00e00ad9f0 msgr2=0x7f00e0160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:14.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00e510e640 1 --2- 172.21.15.105:0/3969175633 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00e00ad9f0 0x7f00e0160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00e510e640 1 -- 172.21.15.105:0/3969175633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00e00cc7d0 msgr2=0x7f00e01610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:14.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00e510e640 1 --2- 172.21.15.105:0/3969175633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00e00cc7d0 0x7f00e01610d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00e510e640 1 -- 172.21.15.105:0/3969175633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d0009420 con 0x7f00e006d610 2023-11-28T18:24:14.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00e510e640 1 --2- 172.21.15.105:0/3969175633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00e006d610 0x7f00e009c100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00c430 tx=0x7f00dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:14.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00d5ffb640 1 -- 172.21.15.105:0/3969175633 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc0052d0 con 0x7f00e006d610 2023-11-28T18:24:14.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00d5ffb640 1 -- 172.21.15.105:0/3969175633 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00dc005450 con 0x7f00e006d610 2023-11-28T18:24:14.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00d5ffb640 1 -- 172.21.15.105:0/3969175633 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc003840 con 0x7f00e006d610 2023-11-28T18:24:14.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00e7372640 1 -- 172.21.15.105:0/3969175633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e015cb40 con 0x7f00e006d610 2023-11-28T18:24:14.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.827+0000 7f00e7372640 1 -- 172.21.15.105:0/3969175633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e015c580 con 0x7f00e006d610 2023-11-28T18:24:14.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.831+0000 7f00e7372640 1 -- 172.21.15.105:0/3969175633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e01540c0 con 0x7f00e006d610 2023-11-28T18:24:14.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.831+0000 7f00d5ffb640 1 -- 172.21.15.105:0/3969175633 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f00dc0039a0 con 0x7f00e006d610 2023-11-28T18:24:14.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.831+0000 7f00d5ffb640 1 --2- 172.21.15.105:0/3969175633 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00c40668d0 0x7f00c4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:14.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.831+0000 7f00d5ffb640 1 -- 172.21.15.105:0/3969175633 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f00dc083e50 con 0x7f00e006d610 2023-11-28T18:24:14.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.831+0000 7f00e490d640 1 --2- 172.21.15.105:0/3969175633 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00c40668d0 0x7f00c4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:14.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.831+0000 7f00e490d640 1 --2- 172.21.15.105:0/3969175633 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00c40668d0 0x7f00c4068d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f00d00af000 tx=0x7f00d00a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:14.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.835+0000 7f00d5ffb640 1 -- 172.21.15.105:0/3969175633 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f00dc050ca0 con 0x7f00e006d610 2023-11-28T18:24:14.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.987+0000 7f00e7372640 1 -- 172.21.15.105:0/3969175633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f00e0067570 con 0x7f00e006d610 2023-11-28T18:24:14.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.987+0000 7f00d5ffb640 1 -- 172.21.15.105:0/3969175633 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f00dc0170a0 con 0x7f00e006d610 2023-11-28T18:24:14.990 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:14.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.987+0000 7f00e7372640 1 -- 172.21.15.105:0/3969175633 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00c40668d0 msgr2=0x7f00c4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:14.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.987+0000 7f00e7372640 1 --2- 172.21.15.105:0/3969175633 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00c40668d0 0x7f00c4068d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f00d00af000 tx=0x7f00d00a7000 comp rx=0 tx=0).stop 2023-11-28T18:24:14.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.987+0000 7f00e7372640 1 -- 172.21.15.105:0/3969175633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00e006d610 msgr2=0x7f00e009c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:14.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.987+0000 7f00e7372640 1 --2- 172.21.15.105:0/3969175633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00e006d610 0x7f00e009c100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00c430 tx=0x7f00dc00c8f0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.987+0000 7f00e7372640 1 -- 172.21.15.105:0/3969175633 shutdown_connections 2023-11-28T18:24:14.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.987+0000 7f00e7372640 1 --2- 172.21.15.105:0/3969175633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00e00cc7d0 0x7f00e01610d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.987+0000 7f00e7372640 1 --2- 172.21.15.105:0/3969175633 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00e00ad9f0 0x7f00e0160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.987+0000 7f00e7372640 1 --2- 172.21.15.105:0/3969175633 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00c40668d0 0x7f00c4068d90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.987+0000 7f00e7372640 1 --2- 172.21.15.105:0/3969175633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00e006d610 0x7f00e009c100 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:14.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.987+0000 7f00e7372640 1 -- 172.21.15.105:0/3969175633 >> 172.21.15.105:0/3969175633 conn(0x7f00e00ceb90 msgr2=0x7f00e00ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:14.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.987+0000 7f00e7372640 1 -- 172.21.15.105:0/3969175633 shutdown_connections 2023-11-28T18:24:14.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:14.987+0000 7f00e7372640 1 -- 172.21.15.105:0/3969175633 wait complete. 2023-11-28T18:24:15.011 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:24:15.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.151+0000 7f17119d1640 1 Processor -- start 2023-11-28T18:24:15.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.151+0000 7f17119d1640 1 -- start start 2023-11-28T18:24:15.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.151+0000 7f17119d1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f170c0d10e0 0x7f170c0d3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:15.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.151+0000 7f17119d1640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f170c0b4140 0x7f170c0d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:15.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.151+0000 7f17119d1640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f170c0d1d20 con 0x7f170c160530 2023-11-28T18:24:15.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.151+0000 7f17119d1640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170c0d01d0 con 0x7f170c0d10e0 2023-11-28T18:24:15.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.151+0000 7f17119d1640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170c0d04b0 con 0x7f170c0b4140 2023-11-28T18:24:15.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.151+0000 7f170affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f170c0d10e0 0x7f170c0d3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:15.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.151+0000 7f170b7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f170c160530 0x7f170c160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55836/0 (socket says 172.21.15.105:55836) 2023-11-28T18:24:15.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.151+0000 7f170b7fe640 1 -- 172.21.15.105:0/2551596781 learned_addr learned my addr 172.21.15.105:0/2551596781 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:15.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.151+0000 7f17027fc640 1 --2- 172.21.15.105:0/2551596781 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f170c0b4140 0x7f170c0d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:15.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.151+0000 7f170affd640 1 -- 172.21.15.105:0/2551596781 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f170c0b4140 msgr2=0x7f170c0d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:15.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.151+0000 7f170affd640 1 --2- 172.21.15.105:0/2551596781 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f170c0b4140 0x7f170c0d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.151+0000 7f170affd640 1 -- 172.21.15.105:0/2551596781 >> v1:172.21.15.105:6789/0 conn(0x7f170c160530 legacy=0x7f170c160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:15.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.151+0000 7f170affd640 1 -- 172.21.15.105:0/2551596781 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c0d0790 con 0x7f170c0d10e0 2023-11-28T18:24:15.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.151+0000 7f170a7fc640 1 -- 172.21.15.105:0/2551596781 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993045709 0 0) 0x7f170c0d1d20 con 0x7f170c160530 2023-11-28T18:24:15.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.155+0000 7f170affd640 1 --2- 172.21.15.105:0/2551596781 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f170c0d10e0 0x7f170c0d3bc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0097c0 tx=0x7f16fc01ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743c9fd39f512d1c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:15.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.155+0000 7f170a7fc640 1 -- 172.21.15.105:0/2551596781 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0ab610 con 0x7f170c0d10e0 2023-11-28T18:24:15.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.155+0000 7f170a7fc640 1 -- 172.21.15.105:0/2551596781 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16fc0ab770 con 0x7f170c0d10e0 2023-11-28T18:24:15.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.155+0000 7f170a7fc640 1 -- 172.21.15.105:0/2551596781 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0abb00 con 0x7f170c0d10e0 2023-11-28T18:24:15.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.155+0000 7f17119d1640 1 -- 172.21.15.105:0/2551596781 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f170c0d10e0 msgr2=0x7f170c0d3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:15.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.155+0000 7f17119d1640 1 --2- 172.21.15.105:0/2551596781 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f170c0d10e0 0x7f170c0d3bc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0097c0 tx=0x7f16fc01ce10 comp rx=0 tx=0).stop 2023-11-28T18:24:15.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.155+0000 7f17119d1640 1 -- 172.21.15.105:0/2551596781 shutdown_connections 2023-11-28T18:24:15.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.155+0000 7f17119d1640 1 --2- 172.21.15.105:0/2551596781 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f170c0b4140 0x7f170c0d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.155+0000 7f17119d1640 1 --2- 172.21.15.105:0/2551596781 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f170c0d10e0 0x7f170c0d3bc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.155+0000 7f17119d1640 1 -- 172.21.15.105:0/2551596781 >> 172.21.15.105:0/2551596781 conn(0x7f170c0d30a0 msgr2=0x7f170c0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:15.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.155+0000 7f17119d1640 1 -- 172.21.15.105:0/2551596781 shutdown_connections 2023-11-28T18:24:15.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.155+0000 7f17119d1640 1 -- 172.21.15.105:0/2551596781 wait complete. 2023-11-28T18:24:15.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f17119d1640 1 Processor -- start 2023-11-28T18:24:15.162 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f17119d1640 1 -- start start 2023-11-28T18:24:15.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f17119d1640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f170c0b4140 0x7f170c15c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:15.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f17119d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f170c0d10e0 0x7f170c153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:15.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f17119d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f170c160530 0x7f170c153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:15.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f17119d1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c155200 con 0x7f170c0d10e0 2023-11-28T18:24:15.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f17119d1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c0c5f30 con 0x7f170c160530 2023-11-28T18:24:15.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f17119d1640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170c1545f0 con 0x7f170c0b4140 2023-11-28T18:24:15.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f17027fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f170c0d10e0 0x7f170c153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:15.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f17027fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f170c0d10e0 0x7f170c153a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56108/0 (socket says 172.21.15.105:56108) 2023-11-28T18:24:15.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f17027fc640 1 -- 172.21.15.105:0/3819884701 learned_addr learned my addr 172.21.15.105:0/3819884701 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:15.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f170affd640 1 --2- 172.21.15.105:0/3819884701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f170c0b4140 0x7f170c15c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:15.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f170b7fe640 1 --2- 172.21.15.105:0/3819884701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f170c160530 0x7f170c153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:15.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f17027fc640 1 -- 172.21.15.105:0/3819884701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f170c0b4140 msgr2=0x7f170c15c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:15.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f17027fc640 1 --2- 172.21.15.105:0/3819884701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f170c0b4140 0x7f170c15c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f17027fc640 1 -- 172.21.15.105:0/3819884701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f170c160530 msgr2=0x7f170c153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:15.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f17027fc640 1 --2- 172.21.15.105:0/3819884701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f170c160530 0x7f170c153f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f17027fc640 1 -- 172.21.15.105:0/3819884701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc009420 con 0x7f170c0d10e0 2023-11-28T18:24:15.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f170b7fe640 1 --2- 172.21.15.105:0/3819884701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f170c160530 0x7f170c153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:15.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f17027fc640 1 --2- 172.21.15.105:0/3819884701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f170c0d10e0 0x7f170c153a10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f16f400d800 tx=0x7f16f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:15.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f1708ff9640 1 -- 172.21.15.105:0/3819884701 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f400b6c0 con 0x7f170c0d10e0 2023-11-28T18:24:15.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f1708ff9640 1 -- 172.21.15.105:0/3819884701 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f400b820 con 0x7f170c0d10e0 2023-11-28T18:24:15.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f1708ff9640 1 -- 172.21.15.105:0/3819884701 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f400bb90 con 0x7f170c0d10e0 2023-11-28T18:24:15.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f17119d1640 1 -- 172.21.15.105:0/3819884701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f170c1548d0 con 0x7f170c0d10e0 2023-11-28T18:24:15.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.159+0000 7f17119d1640 1 -- 172.21.15.105:0/3819884701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170c154e90 con 0x7f170c0d10e0 2023-11-28T18:24:15.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.163+0000 7f17119d1640 1 -- 172.21.15.105:0/3819884701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f170c155050 con 0x7f170c0d10e0 2023-11-28T18:24:15.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.163+0000 7f1708ff9640 1 -- 172.21.15.105:0/3819884701 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f16f400bcf0 con 0x7f170c0d10e0 2023-11-28T18:24:15.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.163+0000 7f1708ff9640 1 --2- 172.21.15.105:0/3819884701 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f16f0066900 0x7f16f0068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:15.170 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.163+0000 7f1708ff9640 1 -- 172.21.15.105:0/3819884701 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f16f4084e60 con 0x7f170c0d10e0 2023-11-28T18:24:15.170 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.163+0000 7f170affd640 1 --2- 172.21.15.105:0/3819884701 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f16f0066900 0x7f16f0068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:15.170 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.163+0000 7f170affd640 1 --2- 172.21.15.105:0/3819884701 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f16f0066900 0x7f16f0068dc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0af000 tx=0x7f16fc004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:15.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.167+0000 7f1708ff9640 1 -- 172.21.15.105:0/3819884701 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f16f4051b70 con 0x7f170c0d10e0 2023-11-28T18:24:15.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.323+0000 7f17119d1640 1 -- 172.21.15.105:0/3819884701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f170c068650 con 0x7f170c0d10e0 2023-11-28T18:24:15.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.323+0000 7f1708ff9640 1 -- 172.21.15.105:0/3819884701 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18887 (secure 0 0 0) 0x7f16f4055190 con 0x7f170c0d10e0 2023-11-28T18:24:15.328 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:15.328 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:24:15.048168+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:24:15.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.327+0000 7f17119d1640 1 -- 172.21.15.105:0/3819884701 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f16f0066900 msgr2=0x7f16f0068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:15.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.327+0000 7f17119d1640 1 --2- 172.21.15.105:0/3819884701 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f16f0066900 0x7f16f0068dc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0af000 tx=0x7f16fc004740 comp rx=0 tx=0).stop 2023-11-28T18:24:15.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.327+0000 7f17119d1640 1 -- 172.21.15.105:0/3819884701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f170c0d10e0 msgr2=0x7f170c153a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:15.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.327+0000 7f17119d1640 1 --2- 172.21.15.105:0/3819884701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f170c0d10e0 0x7f170c153a10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f16f400d800 tx=0x7f16f400dcc0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.327+0000 7f17119d1640 1 -- 172.21.15.105:0/3819884701 shutdown_connections 2023-11-28T18:24:15.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.327+0000 7f17119d1640 1 --2- 172.21.15.105:0/3819884701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f170c160530 0x7f170c153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.327+0000 7f17119d1640 1 --2- 172.21.15.105:0/3819884701 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f16f0066900 0x7f16f0068dc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.327+0000 7f17119d1640 1 --2- 172.21.15.105:0/3819884701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f170c0d10e0 0x7f170c153a10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.327+0000 7f17119d1640 1 --2- 172.21.15.105:0/3819884701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f170c0b4140 0x7f170c15c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.327+0000 7f17119d1640 1 -- 172.21.15.105:0/3819884701 >> 172.21.15.105:0/3819884701 conn(0x7f170c0d30a0 msgr2=0x7f170c0d3690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:15.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.327+0000 7f17119d1640 1 -- 172.21.15.105:0/3819884701 shutdown_connections 2023-11-28T18:24:15.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.327+0000 7f17119d1640 1 -- 172.21.15.105:0/3819884701 wait complete. 2023-11-28T18:24:15.349 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:23:31.485422+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:23:32.274378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T18:23:36.100071+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T18:24:15.350 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:24:15.490 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.487+0000 7f060a69a640 1 Processor -- start 2023-11-28T18:24:15.490 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.487+0000 7f060a69a640 1 -- start start 2023-11-28T18:24:15.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.487+0000 7f060a69a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0604159f40 0x7f06040caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:15.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.487+0000 7f060a69a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f06040cc760 0x7f06040ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:15.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.487+0000 7f060a69a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f060406bf50 con 0x7f06040ad680 2023-11-28T18:24:15.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.487+0000 7f060a69a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06040ccbc0 con 0x7f06040cc760 2023-11-28T18:24:15.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.487+0000 7f060a69a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060415a320 con 0x7f0604159f40 2023-11-28T18:24:15.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.487+0000 7f0603fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0604159f40 0x7f06040caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:15.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.487+0000 7f06037fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f06040cc760 0x7f06040ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:15.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.487+0000 7f0608c37640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f06040ad680 0x7f060415da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:50816/0 (socket says 172.21.15.105:50816) 2023-11-28T18:24:15.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.487+0000 7f0608c37640 1 -- 172.21.15.105:0/2300430178 learned_addr learned my addr 172.21.15.105:0/2300430178 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:15.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.487+0000 7f06037fe640 1 -- 172.21.15.105:0/2300430178 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0604159f40 msgr2=0x7f06040caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:15.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.487+0000 7f06037fe640 1 --2- 172.21.15.105:0/2300430178 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0604159f40 0x7f06040caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.487+0000 7f06037fe640 1 -- 172.21.15.105:0/2300430178 >> v1:172.21.15.148:6789/0 conn(0x7f06040ad680 legacy=0x7f060415da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:15.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.487+0000 7f06037fe640 1 -- 172.21.15.105:0/2300430178 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0604071380 con 0x7f06040cc760 2023-11-28T18:24:15.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.487+0000 7f0603fff640 1 --2- 172.21.15.105:0/2300430178 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0604159f40 0x7f06040caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:24:15.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.491+0000 7f06037fe640 1 --2- 172.21.15.105:0/2300430178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06040cc760 0x7f06040ad140 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f05f80097c0 tx=0x7f05f801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fa7fa91bbc89d55 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:15.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.491+0000 7f0602ffd640 1 -- 172.21.15.105:0/2300430178 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f80ac620 con 0x7f06040cc760 2023-11-28T18:24:15.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.491+0000 7f0602ffd640 1 -- 172.21.15.105:0/2300430178 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f80ac780 con 0x7f06040cc760 2023-11-28T18:24:15.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.491+0000 7f0602ffd640 1 -- 172.21.15.105:0/2300430178 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f80acb10 con 0x7f06040cc760 2023-11-28T18:24:15.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.491+0000 7f060a69a640 1 -- 172.21.15.105:0/2300430178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06040cc760 msgr2=0x7f06040ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:15.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.491+0000 7f060a69a640 1 --2- 172.21.15.105:0/2300430178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06040cc760 0x7f06040ad140 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f05f80097c0 tx=0x7f05f801c3c0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.491+0000 7f060a69a640 1 -- 172.21.15.105:0/2300430178 shutdown_connections 2023-11-28T18:24:15.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.491+0000 7f060a69a640 1 --2- 172.21.15.105:0/2300430178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06040cc760 0x7f06040ad140 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.491+0000 7f060a69a640 1 --2- 172.21.15.105:0/2300430178 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0604159f40 0x7f06040caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.491+0000 7f060a69a640 1 -- 172.21.15.105:0/2300430178 >> 172.21.15.105:0/2300430178 conn(0x7f06040ceb90 msgr2=0x7f06040cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:15.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.491+0000 7f060a69a640 1 -- 172.21.15.105:0/2300430178 shutdown_connections 2023-11-28T18:24:15.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.491+0000 7f060a69a640 1 -- 172.21.15.105:0/2300430178 wait complete. 2023-11-28T18:24:15.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f060a69a640 1 Processor -- start 2023-11-28T18:24:15.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f060a69a640 1 -- start start 2023-11-28T18:24:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f060a69a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06040ad680 0x7f060415cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f060a69a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06040cc760 0x7f060415a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f060a69a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0604159f40 0x7f0604159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f060a69a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0604153570 con 0x7f06040cc760 2023-11-28T18:24:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f060a69a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060415aaf0 con 0x7f06040ad680 2023-11-28T18:24:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f060a69a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060415bf50 con 0x7f0604159f40 2023-11-28T18:24:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f06037fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06040cc760 0x7f060415a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f06037fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06040cc760 0x7f060415a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56116/0 (socket says 172.21.15.105:56116) 2023-11-28T18:24:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f06037fe640 1 -- 172.21.15.105:0/2283363618 learned_addr learned my addr 172.21.15.105:0/2283363618 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f0603fff640 1 --2- 172.21.15.105:0/2283363618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06040ad680 0x7f060415cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:15.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f0608c37640 1 --2- 172.21.15.105:0/2283363618 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0604159f40 0x7f0604159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:15.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f06037fe640 1 -- 172.21.15.105:0/2283363618 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0604159f40 msgr2=0x7f0604159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:15.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f06037fe640 1 --2- 172.21.15.105:0/2283363618 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0604159f40 0x7f0604159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f06037fe640 1 -- 172.21.15.105:0/2283363618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06040ad680 msgr2=0x7f060415cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:15.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f06037fe640 1 --2- 172.21.15.105:0/2283363618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06040ad680 0x7f060415cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f06037fe640 1 -- 172.21.15.105:0/2283363618 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f4009420 con 0x7f06040cc760 2023-11-28T18:24:15.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f0608c37640 1 --2- 172.21.15.105:0/2283363618 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0604159f40 0x7f0604159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:15.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f06037fe640 1 --2- 172.21.15.105:0/2283363618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06040cc760 0x7f060415a460 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f05f80af040 tx=0x7f05f8019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:15.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f0600ff9640 1 -- 172.21.15.105:0/2283363618 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f80b5410 con 0x7f06040cc760 2023-11-28T18:24:15.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f0600ff9640 1 -- 172.21.15.105:0/2283363618 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f80b5590 con 0x7f06040cc760 2023-11-28T18:24:15.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f0600ff9640 1 -- 172.21.15.105:0/2283363618 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f80b5940 con 0x7f06040cc760 2023-11-28T18:24:15.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f060a69a640 1 -- 172.21.15.105:0/2283363618 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f8009420 con 0x7f06040cc760 2023-11-28T18:24:15.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.495+0000 7f060a69a640 1 -- 172.21.15.105:0/2283363618 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060415c4d0 con 0x7f06040cc760 2023-11-28T18:24:15.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.499+0000 7f060a69a640 1 -- 172.21.15.105:0/2283363618 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060415ccd0 con 0x7f06040cc760 2023-11-28T18:24:15.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.499+0000 7f0600ff9640 1 -- 172.21.15.105:0/2283363618 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f05f80b3040 con 0x7f06040cc760 2023-11-28T18:24:15.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.499+0000 7f0600ff9640 1 --2- 172.21.15.105:0/2283363618 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f05f00668d0 0x7f05f0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:15.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.499+0000 7f0600ff9640 1 -- 172.21.15.105:0/2283363618 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f05f811e810 con 0x7f06040cc760 2023-11-28T18:24:15.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.499+0000 7f0603fff640 1 --2- 172.21.15.105:0/2283363618 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f05f00668d0 0x7f05f0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:15.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.499+0000 7f0603fff640 1 --2- 172.21.15.105:0/2283363618 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f05f00668d0 0x7f05f0068d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009c50 tx=0x7f05f4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:15.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.503+0000 7f0600ff9640 1 -- 172.21.15.105:0/2283363618 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f060415ccd0 con 0x7f06040cc760 2023-11-28T18:24:15.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.651+0000 7f060a69a640 1 -- 172.21.15.105:0/2283363618 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f060415ccd0 con 0x7f06040cc760 2023-11-28T18:24:15.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.651+0000 7f0600ff9640 1 -- 172.21.15.105:0/2283363618 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f05f80c3020 con 0x7f06040cc760 2023-11-28T18:24:15.657 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:24:15.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.655+0000 7f060a69a640 1 -- 172.21.15.105:0/2283363618 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f05f00668d0 msgr2=0x7f05f0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:15.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.655+0000 7f060a69a640 1 --2- 172.21.15.105:0/2283363618 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f05f00668d0 0x7f05f0068d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009c50 tx=0x7f05f4009190 comp rx=0 tx=0).stop 2023-11-28T18:24:15.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.655+0000 7f060a69a640 1 -- 172.21.15.105:0/2283363618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06040cc760 msgr2=0x7f060415a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:15.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.655+0000 7f060a69a640 1 --2- 172.21.15.105:0/2283363618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06040cc760 0x7f060415a460 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f05f80af040 tx=0x7f05f8019160 comp rx=0 tx=0).stop 2023-11-28T18:24:15.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.655+0000 7f060a69a640 1 -- 172.21.15.105:0/2283363618 shutdown_connections 2023-11-28T18:24:15.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.655+0000 7f060a69a640 1 --2- 172.21.15.105:0/2283363618 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0604159f40 0x7f0604159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.655+0000 7f060a69a640 1 --2- 172.21.15.105:0/2283363618 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f05f00668d0 0x7f05f0068d90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.655+0000 7f060a69a640 1 --2- 172.21.15.105:0/2283363618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06040cc760 0x7f060415a460 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.655+0000 7f060a69a640 1 --2- 172.21.15.105:0/2283363618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06040ad680 0x7f060415cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.655+0000 7f060a69a640 1 -- 172.21.15.105:0/2283363618 >> 172.21.15.105:0/2283363618 conn(0x7f06040ceb90 msgr2=0x7f06040cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:15.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.655+0000 7f060a69a640 1 -- 172.21.15.105:0/2283363618 shutdown_connections 2023-11-28T18:24:15.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.655+0000 7f060a69a640 1 -- 172.21.15.105:0/2283363618 wait complete. 2023-11-28T18:24:15.678 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:24:15.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.815+0000 7efe4e8c9640 1 Processor -- start 2023-11-28T18:24:15.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.815+0000 7efe4e8c9640 1 -- start start 2023-11-28T18:24:15.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.815+0000 7efe4e8c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe48159f40 0x7efe480caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:15.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.815+0000 7efe4e8c9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efe480cc760 0x7efe480ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:15.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.815+0000 7efe4e8c9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4806bf50 con 0x7efe480ad680 2023-11-28T18:24:15.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.815+0000 7efe4e8c9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe480ccbc0 con 0x7efe480cc760 2023-11-28T18:24:15.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.815+0000 7efe4e8c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4815a320 con 0x7efe48159f40 2023-11-28T18:24:15.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.815+0000 7efe477fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efe480cc760 0x7efe480ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:15.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.815+0000 7efe4ce66640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efe480ad680 0x7efe4815da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:50828/0 (socket says 172.21.15.105:50828) 2023-11-28T18:24:15.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.815+0000 7efe4ce66640 1 -- 172.21.15.105:0/3728745791 learned_addr learned my addr 172.21.15.105:0/3728745791 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:15.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.815+0000 7efe47fff640 1 --2- 172.21.15.105:0/3728745791 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe48159f40 0x7efe480caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:15.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.815+0000 7efe477fe640 1 -- 172.21.15.105:0/3728745791 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe48159f40 msgr2=0x7efe480caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:15.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.815+0000 7efe477fe640 1 --2- 172.21.15.105:0/3728745791 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe48159f40 0x7efe480caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.815+0000 7efe477fe640 1 -- 172.21.15.105:0/3728745791 >> v1:172.21.15.148:6789/0 conn(0x7efe480ad680 legacy=0x7efe4815da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:15.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.815+0000 7efe477fe640 1 -- 172.21.15.105:0/3728745791 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe48071380 con 0x7efe480cc760 2023-11-28T18:24:15.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.819+0000 7efe477fe640 1 --2- 172.21.15.105:0/3728745791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe480cc760 0x7efe480ad140 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0097c0 tx=0x7efe3c01e5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f38879e93bf09a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:15.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.819+0000 7efe46ffd640 1 -- 172.21.15.105:0/3728745791 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c0b2600 con 0x7efe480cc760 2023-11-28T18:24:15.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.819+0000 7efe46ffd640 1 -- 172.21.15.105:0/3728745791 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c0b2760 con 0x7efe480cc760 2023-11-28T18:24:15.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.819+0000 7efe46ffd640 1 -- 172.21.15.105:0/3728745791 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c0b2af0 con 0x7efe480cc760 2023-11-28T18:24:15.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.819+0000 7efe4e8c9640 1 -- 172.21.15.105:0/3728745791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe480cc760 msgr2=0x7efe480ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:15.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.819+0000 7efe4e8c9640 1 --2- 172.21.15.105:0/3728745791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe480cc760 0x7efe480ad140 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0097c0 tx=0x7efe3c01e5d0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.819+0000 7efe4e8c9640 1 -- 172.21.15.105:0/3728745791 shutdown_connections 2023-11-28T18:24:15.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.819+0000 7efe4e8c9640 1 --2- 172.21.15.105:0/3728745791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe480cc760 0x7efe480ad140 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.819+0000 7efe4e8c9640 1 --2- 172.21.15.105:0/3728745791 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe48159f40 0x7efe480caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.819+0000 7efe4e8c9640 1 -- 172.21.15.105:0/3728745791 >> 172.21.15.105:0/3728745791 conn(0x7efe480ceb90 msgr2=0x7efe480cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:15.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.819+0000 7efe4e8c9640 1 -- 172.21.15.105:0/3728745791 shutdown_connections 2023-11-28T18:24:15.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.819+0000 7efe4e8c9640 1 -- 172.21.15.105:0/3728745791 wait complete. 2023-11-28T18:24:15.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe4e8c9640 1 Processor -- start 2023-11-28T18:24:15.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe4e8c9640 1 -- start start 2023-11-28T18:24:15.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe4e8c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe480ad680 0x7efe48148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:15.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe4e8c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe480cc760 0x7efe48099e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:15.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe4e8c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe48159f40 0x7efe4809a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:15.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe4e8c9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4815c040 con 0x7efe480cc760 2023-11-28T18:24:15.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe4e8c9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe48148d90 con 0x7efe48159f40 2023-11-28T18:24:15.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe4e8c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4815c460 con 0x7efe480ad680 2023-11-28T18:24:15.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe477fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe480cc760 0x7efe48099e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:15.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe477fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe480cc760 0x7efe48099e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56120/0 (socket says 172.21.15.105:56120) 2023-11-28T18:24:15.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe477fe640 1 -- 172.21.15.105:0/3178856659 learned_addr learned my addr 172.21.15.105:0/3178856659 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:15.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe47fff640 1 --2- 172.21.15.105:0/3178856659 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe480ad680 0x7efe48148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:15.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe4ce66640 1 --2- 172.21.15.105:0/3178856659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe48159f40 0x7efe4809a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:15.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe477fe640 1 -- 172.21.15.105:0/3178856659 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe480ad680 msgr2=0x7efe48148850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:15.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe477fe640 1 --2- 172.21.15.105:0/3178856659 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe480ad680 0x7efe48148850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe477fe640 1 -- 172.21.15.105:0/3178856659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe48159f40 msgr2=0x7efe4809a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:15.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe477fe640 1 --2- 172.21.15.105:0/3178856659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe48159f40 0x7efe4809a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe477fe640 1 -- 172.21.15.105:0/3178856659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c009420 con 0x7efe480cc760 2023-11-28T18:24:15.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe477fe640 1 --2- 172.21.15.105:0/3178856659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe480cc760 0x7efe48099e80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efe3c01cc80 tx=0x7efe3c01b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:15.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe44ff9640 1 -- 172.21.15.105:0/3178856659 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c0b2ce0 con 0x7efe480cc760 2023-11-28T18:24:15.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe44ff9640 1 -- 172.21.15.105:0/3178856659 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c0b2e60 con 0x7efe480cc760 2023-11-28T18:24:15.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe44ff9640 1 -- 172.21.15.105:0/3178856659 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c0ba680 con 0x7efe480cc760 2023-11-28T18:24:15.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe4e8c9640 1 -- 172.21.15.105:0/3178856659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe4815b3c0 con 0x7efe480cc760 2023-11-28T18:24:15.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.823+0000 7efe4e8c9640 1 -- 172.21.15.105:0/3178856659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe4815b980 con 0x7efe480cc760 2023-11-28T18:24:15.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.827+0000 7efe4e8c9640 1 -- 172.21.15.105:0/3178856659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe4815c180 con 0x7efe480cc760 2023-11-28T18:24:15.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.827+0000 7efe44ff9640 1 -- 172.21.15.105:0/3178856659 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7efe3c0aa400 con 0x7efe480cc760 2023-11-28T18:24:15.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.827+0000 7efe44ff9640 1 --2- 172.21.15.105:0/3178856659 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe340668d0 0x7efe34068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:15.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.827+0000 7efe44ff9640 1 -- 172.21.15.105:0/3178856659 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7efe3c11ebc0 con 0x7efe480cc760 2023-11-28T18:24:15.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.827+0000 7efe47fff640 1 --2- 172.21.15.105:0/3178856659 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe340668d0 0x7efe34068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:15.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.827+0000 7efe47fff640 1 --2- 172.21.15.105:0/3178856659 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe340668d0 0x7efe34068d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efe38009ac0 tx=0x7efe38009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:15.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.831+0000 7efe44ff9640 1 -- 172.21.15.105:0/3178856659 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efe4815c180 con 0x7efe480cc760 2023-11-28T18:24:15.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.979+0000 7efe4e8c9640 1 -- 172.21.15.105:0/3178856659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efe4815c180 con 0x7efe480cc760 2023-11-28T18:24:15.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.983+0000 7efe44ff9640 1 -- 172.21.15.105:0/3178856659 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7efe4815c180 con 0x7efe480cc760 2023-11-28T18:24:15.986 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:15.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.983+0000 7efe4e8c9640 1 -- 172.21.15.105:0/3178856659 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe340668d0 msgr2=0x7efe34068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:15.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.983+0000 7efe4e8c9640 1 --2- 172.21.15.105:0/3178856659 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe340668d0 0x7efe34068d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efe38009ac0 tx=0x7efe38009190 comp rx=0 tx=0).stop 2023-11-28T18:24:15.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.983+0000 7efe4e8c9640 1 -- 172.21.15.105:0/3178856659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe480cc760 msgr2=0x7efe48099e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:15.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.983+0000 7efe4e8c9640 1 --2- 172.21.15.105:0/3178856659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe480cc760 0x7efe48099e80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efe3c01cc80 tx=0x7efe3c01b370 comp rx=0 tx=0).stop 2023-11-28T18:24:15.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.983+0000 7efe4e8c9640 1 -- 172.21.15.105:0/3178856659 shutdown_connections 2023-11-28T18:24:15.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.983+0000 7efe4e8c9640 1 --2- 172.21.15.105:0/3178856659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe48159f40 0x7efe4809a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.983+0000 7efe4e8c9640 1 --2- 172.21.15.105:0/3178856659 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe340668d0 0x7efe34068d90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.983+0000 7efe4e8c9640 1 --2- 172.21.15.105:0/3178856659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe480cc760 0x7efe48099e80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.983+0000 7efe4e8c9640 1 --2- 172.21.15.105:0/3178856659 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe480ad680 0x7efe48148850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:15.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.983+0000 7efe4e8c9640 1 -- 172.21.15.105:0/3178856659 >> 172.21.15.105:0/3178856659 conn(0x7efe480ceb90 msgr2=0x7efe481534c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:15.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.983+0000 7efe4e8c9640 1 -- 172.21.15.105:0/3178856659 shutdown_connections 2023-11-28T18:24:15.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:15.983+0000 7efe4e8c9640 1 -- 172.21.15.105:0/3178856659 wait complete. 2023-11-28T18:24:16.006 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:24:16.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.143+0000 7f3a6d49a640 1 Processor -- start 2023-11-28T18:24:16.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.143+0000 7f3a6d49a640 1 -- start start 2023-11-28T18:24:16.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.147+0000 7f3a6d49a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3a6806c0a0 0x7f3a680ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:16.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.147+0000 7f3a6d49a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a680b3c40 con 0x7f3a680cdd80 2023-11-28T18:24:16.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.147+0000 7f3a6d49a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a6806ca80 con 0x7f3a6815d320 2023-11-28T18:24:16.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.147+0000 7f3a6d49a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a680ce770 con 0x7f3a6806c0a0 2023-11-28T18:24:16.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.147+0000 7f3a677fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3a6806c0a0 0x7f3a680ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:16.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.147+0000 7f3a5e7fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3a680cdd80 0x7f3a680cc0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55840/0 (socket says 172.21.15.105:55840) 2023-11-28T18:24:16.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.147+0000 7f3a5e7fc640 1 -- 172.21.15.105:0/2490955261 learned_addr learned my addr 172.21.15.105:0/2490955261 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:16.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.147+0000 7f3a677fe640 1 -- 172.21.15.105:0/2490955261 >> v1:172.21.15.148:6789/0 conn(0x7f3a6815d320 legacy=0x7f3a680cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:16.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.147+0000 7f3a677fe640 1 -- 172.21.15.105:0/2490955261 >> v1:172.21.15.105:6789/0 conn(0x7f3a680cdd80 legacy=0x7f3a680cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:16.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.147+0000 7f3a677fe640 1 -- 172.21.15.105:0/2490955261 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6815d700 con 0x7f3a6806c0a0 2023-11-28T18:24:16.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.147+0000 7f3a677fe640 1 --2- 172.21.15.105:0/2490955261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a6806c0a0 0x7f3a680ce230 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009d90 tx=0x7f3a6001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9fa9f88021887dd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:16.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.147+0000 7f3a667fc640 1 -- 172.21.15.105:0/2490955261 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a600a5950 con 0x7f3a6806c0a0 2023-11-28T18:24:16.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.151+0000 7f3a667fc640 1 -- 172.21.15.105:0/2490955261 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a600a5ab0 con 0x7f3a6806c0a0 2023-11-28T18:24:16.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.151+0000 7f3a667fc640 1 -- 172.21.15.105:0/2490955261 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a600a5e40 con 0x7f3a6806c0a0 2023-11-28T18:24:16.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.151+0000 7f3a6d49a640 1 -- 172.21.15.105:0/2490955261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a6806c0a0 msgr2=0x7f3a680ce230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:16.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.151+0000 7f3a6d49a640 1 --2- 172.21.15.105:0/2490955261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a6806c0a0 0x7f3a680ce230 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009d90 tx=0x7f3a6001c890 comp rx=0 tx=0).stop 2023-11-28T18:24:16.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.151+0000 7f3a6d49a640 1 -- 172.21.15.105:0/2490955261 shutdown_connections 2023-11-28T18:24:16.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.151+0000 7f3a6d49a640 1 --2- 172.21.15.105:0/2490955261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a6806c0a0 0x7f3a680ce230 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.151+0000 7f3a6d49a640 1 -- 172.21.15.105:0/2490955261 >> 172.21.15.105:0/2490955261 conn(0x7f3a680d0df0 msgr2=0x7f3a680d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:16.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.151+0000 7f3a6d49a640 1 -- 172.21.15.105:0/2490955261 shutdown_connections 2023-11-28T18:24:16.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.151+0000 7f3a6d49a640 1 -- 172.21.15.105:0/2490955261 wait complete. 2023-11-28T18:24:16.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.151+0000 7f3a6d49a640 1 Processor -- start 2023-11-28T18:24:16.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.151+0000 7f3a6d49a640 1 -- start start 2023-11-28T18:24:16.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.151+0000 7f3a6d49a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a6806c0a0 0x7f3a6815e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:16.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.151+0000 7f3a6d49a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a680cdd80 0x7f3a68158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:16.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.151+0000 7f3a6d49a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a6815d320 0x7f3a681589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:16.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.151+0000 7f3a6d49a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6815f1b0 con 0x7f3a680cdd80 2023-11-28T18:24:16.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.151+0000 7f3a6d49a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6815f490 con 0x7f3a6806c0a0 2023-11-28T18:24:16.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.151+0000 7f3a6d49a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6815f770 con 0x7f3a6815d320 2023-11-28T18:24:16.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.151+0000 7f3a5e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a680cdd80 0x7f3a68158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:16.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.151+0000 7f3a5e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a680cdd80 0x7f3a68158490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56130/0 (socket says 172.21.15.105:56130) 2023-11-28T18:24:16.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.151+0000 7f3a5e7fc640 1 -- 172.21.15.105:0/2635474081 learned_addr learned my addr 172.21.15.105:0/2635474081 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:16.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.151+0000 7f3a66ffd640 1 --2- 172.21.15.105:0/2635474081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a6806c0a0 0x7f3a6815e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:16.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.155+0000 7f3a677fe640 1 --2- 172.21.15.105:0/2635474081 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a6815d320 0x7f3a681589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:16.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.155+0000 7f3a5e7fc640 1 -- 172.21.15.105:0/2635474081 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a6815d320 msgr2=0x7f3a681589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:16.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.155+0000 7f3a5e7fc640 1 --2- 172.21.15.105:0/2635474081 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a6815d320 0x7f3a681589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.155+0000 7f3a5e7fc640 1 -- 172.21.15.105:0/2635474081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a6806c0a0 msgr2=0x7f3a6815e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:16.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.155+0000 7f3a5e7fc640 1 --2- 172.21.15.105:0/2635474081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a6806c0a0 0x7f3a6815e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.155+0000 7f3a5e7fc640 1 -- 172.21.15.105:0/2635474081 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a60009a70 con 0x7f3a680cdd80 2023-11-28T18:24:16.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.155+0000 7f3a5e7fc640 1 --2- 172.21.15.105:0/2635474081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a680cdd80 0x7f3a68158490 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800c450 tx=0x7f3a5800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:16.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.155+0000 7f3a5ffff640 1 -- 172.21.15.105:0/2635474081 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58005cf0 con 0x7f3a680cdd80 2023-11-28T18:24:16.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.155+0000 7f3a5ffff640 1 -- 172.21.15.105:0/2635474081 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a58005e50 con 0x7f3a680cdd80 2023-11-28T18:24:16.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.155+0000 7f3a6d49a640 1 -- 172.21.15.105:0/2635474081 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6815fa50 con 0x7f3a680cdd80 2023-11-28T18:24:16.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.155+0000 7f3a5ffff640 1 -- 172.21.15.105:0/2635474081 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58007270 con 0x7f3a680cdd80 2023-11-28T18:24:16.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.155+0000 7f3a6d49a640 1 -- 172.21.15.105:0/2635474081 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a680b4a00 con 0x7f3a680cdd80 2023-11-28T18:24:16.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.155+0000 7f3a6d49a640 1 -- 172.21.15.105:0/2635474081 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a68067570 con 0x7f3a680cdd80 2023-11-28T18:24:16.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.155+0000 7f3a5ffff640 1 -- 172.21.15.105:0/2635474081 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3a580073d0 con 0x7f3a680cdd80 2023-11-28T18:24:16.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.159+0000 7f3a5ffff640 1 --2- 172.21.15.105:0/2635474081 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3a480668d0 0x7f3a48068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:16.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.159+0000 7f3a5ffff640 1 -- 172.21.15.105:0/2635474081 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f3a58083c20 con 0x7f3a680cdd80 2023-11-28T18:24:16.162 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.159+0000 7f3a66ffd640 1 --2- 172.21.15.105:0/2635474081 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3a480668d0 0x7f3a48068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:16.162 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.159+0000 7f3a66ffd640 1 --2- 172.21.15.105:0/2635474081 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3a480668d0 0x7f3a48068d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3a50007fd0 tx=0x7f3a50009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:16.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.163+0000 7f3a5ffff640 1 -- 172.21.15.105:0/2635474081 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3a58050b10 con 0x7f3a680cdd80 2023-11-28T18:24:16.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.315+0000 7f3a6d49a640 1 -- 172.21.15.105:0/2635474081 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3a680d0910 con 0x7f3a680cdd80 2023-11-28T18:24:16.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.315+0000 7f3a5ffff640 1 -- 172.21.15.105:0/2635474081 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3a58017020 con 0x7f3a680cdd80 2023-11-28T18:24:16.318 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:24:16.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.315+0000 7f3a6d49a640 1 -- 172.21.15.105:0/2635474081 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3a480668d0 msgr2=0x7f3a48068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:16.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.315+0000 7f3a6d49a640 1 --2- 172.21.15.105:0/2635474081 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3a480668d0 0x7f3a48068d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3a50007fd0 tx=0x7f3a50009040 comp rx=0 tx=0).stop 2023-11-28T18:24:16.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.315+0000 7f3a6d49a640 1 -- 172.21.15.105:0/2635474081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a680cdd80 msgr2=0x7f3a68158490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:16.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.315+0000 7f3a6d49a640 1 --2- 172.21.15.105:0/2635474081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a680cdd80 0x7f3a68158490 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800c450 tx=0x7f3a5800c910 comp rx=0 tx=0).stop 2023-11-28T18:24:16.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.315+0000 7f3a6d49a640 1 -- 172.21.15.105:0/2635474081 shutdown_connections 2023-11-28T18:24:16.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.315+0000 7f3a6d49a640 1 --2- 172.21.15.105:0/2635474081 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a6815d320 0x7f3a681589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.315+0000 7f3a6d49a640 1 --2- 172.21.15.105:0/2635474081 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3a480668d0 0x7f3a48068d90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.315+0000 7f3a6d49a640 1 --2- 172.21.15.105:0/2635474081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a680cdd80 0x7f3a68158490 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.315+0000 7f3a6d49a640 1 --2- 172.21.15.105:0/2635474081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a6806c0a0 0x7f3a6815e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.315+0000 7f3a6d49a640 1 -- 172.21.15.105:0/2635474081 >> 172.21.15.105:0/2635474081 conn(0x7f3a680d0df0 msgr2=0x7f3a680cd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:16.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.315+0000 7f3a6d49a640 1 -- 172.21.15.105:0/2635474081 shutdown_connections 2023-11-28T18:24:16.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.315+0000 7f3a6d49a640 1 -- 172.21.15.105:0/2635474081 wait complete. 2023-11-28T18:24:16.339 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T18:24:16.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.475+0000 7f87d0df5640 1 Processor -- start 2023-11-28T18:24:16.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.475+0000 7f87d0df5640 1 -- start start 2023-11-28T18:24:16.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.479+0000 7f87d0df5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f87cc159f40 0x7f87cc0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:16.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.479+0000 7f87d0df5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f87cc0cc760 0x7f87cc0ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:16.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.479+0000 7f87d0df5640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87cc0ad680 0x7f87cc15da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:16.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.479+0000 7f87d0df5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87cc06bf50 con 0x7f87cc159f40 2023-11-28T18:24:16.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.479+0000 7f87d0df5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87cc0ccbc0 con 0x7f87cc0cc760 2023-11-28T18:24:16.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.479+0000 7f87d0df5640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc15a320 con 0x7f87cc0ad680 2023-11-28T18:24:16.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.479+0000 7f87ca59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f87cc159f40 0x7f87cc0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:16.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.479+0000 7f87ca59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f87cc159f40 0x7f87cc0caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56146/0 (socket says 172.21.15.105:56146) 2023-11-28T18:24:16.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.479+0000 7f87ca59c640 1 -- 172.21.15.105:0/996485730 learned_addr learned my addr 172.21.15.105:0/996485730 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:16.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.479+0000 7f87cad9d640 1 --2- 172.21.15.105:0/996485730 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87cc0ad680 0x7f87cc15da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:16.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.479+0000 7f87c3fff640 1 --2- 172.21.15.105:0/996485730 >> v2:172.21.15.148:3300/0 conn(0x7f87cc0cc760 0x7f87cc0ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:16.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.479+0000 7f87ca59c640 1 -- 172.21.15.105:0/996485730 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87cc0ad680 msgr2=0x7f87cc15da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:16.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.479+0000 7f87ca59c640 1 --2- 172.21.15.105:0/996485730 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87cc0ad680 0x7f87cc15da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.479+0000 7f87ca59c640 1 -- 172.21.15.105:0/996485730 >> v2:172.21.15.148:3300/0 conn(0x7f87cc0cc760 msgr2=0x7f87cc0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:16.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.479+0000 7f87ca59c640 1 --2- 172.21.15.105:0/996485730 >> v2:172.21.15.148:3300/0 conn(0x7f87cc0cc760 0x7f87cc0ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.479+0000 7f87ca59c640 1 -- 172.21.15.105:0/996485730 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc071380 con 0x7f87cc159f40 2023-11-28T18:24:16.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.479+0000 7f87c3fff640 1 --2- 172.21.15.105:0/996485730 >> v2:172.21.15.148:3300/0 conn(0x7f87cc0cc760 0x7f87cc0ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:16.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.483+0000 7f87ca59c640 1 --2- 172.21.15.105:0/996485730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87cc159f40 0x7f87cc0caf60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f87cc09d0d0 tx=0x7f87bc01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d50d0dd343fd361 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:16.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.483+0000 7f87c9d9b640 1 -- 172.21.15.105:0/996485730 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc0ab600 con 0x7f87cc159f40 2023-11-28T18:24:16.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.483+0000 7f87c9d9b640 1 -- 172.21.15.105:0/996485730 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87bc0ab760 con 0x7f87cc159f40 2023-11-28T18:24:16.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.483+0000 7f87c9d9b640 1 -- 172.21.15.105:0/996485730 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc0abaf0 con 0x7f87cc159f40 2023-11-28T18:24:16.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.483+0000 7f87d0df5640 1 -- 172.21.15.105:0/996485730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87cc159f40 msgr2=0x7f87cc0caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:16.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.483+0000 7f87d0df5640 1 --2- 172.21.15.105:0/996485730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87cc159f40 0x7f87cc0caf60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f87cc09d0d0 tx=0x7f87bc01c440 comp rx=0 tx=0).stop 2023-11-28T18:24:16.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.483+0000 7f87d0df5640 1 -- 172.21.15.105:0/996485730 shutdown_connections 2023-11-28T18:24:16.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.483+0000 7f87d0df5640 1 --2- 172.21.15.105:0/996485730 >> v2:172.21.15.148:3300/0 conn(0x7f87cc0cc760 0x7f87cc0ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.483+0000 7f87d0df5640 1 --2- 172.21.15.105:0/996485730 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87cc0ad680 0x7f87cc15da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.483+0000 7f87d0df5640 1 --2- 172.21.15.105:0/996485730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87cc159f40 0x7f87cc0caf60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.483+0000 7f87d0df5640 1 -- 172.21.15.105:0/996485730 >> 172.21.15.105:0/996485730 conn(0x7f87cc0ceb90 msgr2=0x7f87cc0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:16.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.483+0000 7f87d0df5640 1 -- 172.21.15.105:0/996485730 shutdown_connections 2023-11-28T18:24:16.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.483+0000 7f87d0df5640 1 -- 172.21.15.105:0/996485730 wait complete. 2023-11-28T18:24:16.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.483+0000 7f87d0df5640 1 Processor -- start 2023-11-28T18:24:16.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.483+0000 7f87d0df5640 1 -- start start 2023-11-28T18:24:16.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87d0df5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87cc0ad680 0x7f87cc1518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:16.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87d0df5640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87cc0cc760 0x7f87cc0b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:16.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87d0df5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87cc159f40 0x7f87cc1f6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:16.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87d0df5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc1527f0 con 0x7f87cc159f40 2023-11-28T18:24:16.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87d0df5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc09bfd0 con 0x7f87cc0ad680 2023-11-28T18:24:16.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87d0df5640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc099a50 con 0x7f87cc0cc760 2023-11-28T18:24:16.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87cad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87cc159f40 0x7f87cc1f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:16.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87cad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87cc159f40 0x7f87cc1f6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56148/0 (socket says 172.21.15.105:56148) 2023-11-28T18:24:16.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87cad9d640 1 -- 172.21.15.105:0/3319517862 learned_addr learned my addr 172.21.15.105:0/3319517862 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:16.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87ca59c640 1 --2- 172.21.15.105:0/3319517862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87cc0ad680 0x7f87cc1518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:16.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87c3fff640 1 --2- 172.21.15.105:0/3319517862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87cc0cc760 0x7f87cc0b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:16.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87cad9d640 1 -- 172.21.15.105:0/3319517862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87cc0cc760 msgr2=0x7f87cc0b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:16.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87cad9d640 1 --2- 172.21.15.105:0/3319517862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87cc0cc760 0x7f87cc0b4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87cad9d640 1 -- 172.21.15.105:0/3319517862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87cc0ad680 msgr2=0x7f87cc1518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:16.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87cad9d640 1 --2- 172.21.15.105:0/3319517862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87cc0ad680 0x7f87cc1518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87cad9d640 1 -- 172.21.15.105:0/3319517862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87bc0094d0 con 0x7f87cc159f40 2023-11-28T18:24:16.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87cad9d640 1 --2- 172.21.15.105:0/3319517862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87cc159f40 0x7f87cc1f6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f87c4007e90 tx=0x7f87c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:16.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87c2ffd640 1 -- 172.21.15.105:0/3319517862 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4014420 con 0x7f87cc159f40 2023-11-28T18:24:16.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87c2ffd640 1 -- 172.21.15.105:0/3319517862 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c40145a0 con 0x7f87cc159f40 2023-11-28T18:24:16.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87c2ffd640 1 -- 172.21.15.105:0/3319517862 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4014950 con 0x7f87cc159f40 2023-11-28T18:24:16.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87d0df5640 1 -- 172.21.15.105:0/3319517862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87cc09b450 con 0x7f87cc159f40 2023-11-28T18:24:16.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87d0df5640 1 -- 172.21.15.105:0/3319517862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87cc09ba10 con 0x7f87cc159f40 2023-11-28T18:24:16.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87d0df5640 1 -- 172.21.15.105:0/3319517862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87cc099bc0 con 0x7f87cc159f40 2023-11-28T18:24:16.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.487+0000 7f87c2ffd640 1 -- 172.21.15.105:0/3319517862 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f87c4014ab0 con 0x7f87cc159f40 2023-11-28T18:24:16.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.491+0000 7f87c2ffd640 1 --2- 172.21.15.105:0/3319517862 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f87ac0668d0 0x7f87ac068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:16.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.491+0000 7f87c2ffd640 1 -- 172.21.15.105:0/3319517862 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f87c4084fc0 con 0x7f87cc159f40 2023-11-28T18:24:16.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.491+0000 7f87ca59c640 1 --2- 172.21.15.105:0/3319517862 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f87ac0668d0 0x7f87ac068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:16.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.491+0000 7f87ca59c640 1 --2- 172.21.15.105:0/3319517862 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f87ac0668d0 0x7f87ac068d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0af000 tx=0x7f87bc0b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:16.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.491+0000 7f87c2ffd640 1 -- 172.21.15.105:0/3319517862 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f87c40b6280 con 0x7f87cc159f40 2023-11-28T18:24:16.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.643+0000 7f87d0df5640 1 -- 172.21.15.105:0/3319517862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f87cc067570 con 0x7f87cc159f40 2023-11-28T18:24:16.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.643+0000 7f87c2ffd640 1 -- 172.21.15.105:0/3319517862 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f87c4051f20 con 0x7f87cc159f40 2023-11-28T18:24:16.647 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:16.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.647+0000 7f87d0df5640 1 -- 172.21.15.105:0/3319517862 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f87ac0668d0 msgr2=0x7f87ac068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:16.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.647+0000 7f87d0df5640 1 --2- 172.21.15.105:0/3319517862 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f87ac0668d0 0x7f87ac068d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0af000 tx=0x7f87bc0b6040 comp rx=0 tx=0).stop 2023-11-28T18:24:16.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.647+0000 7f87d0df5640 1 -- 172.21.15.105:0/3319517862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87cc159f40 msgr2=0x7f87cc1f6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:16.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.647+0000 7f87d0df5640 1 --2- 172.21.15.105:0/3319517862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87cc159f40 0x7f87cc1f6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f87c4007e90 tx=0x7f87c400e4b0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.647+0000 7f87d0df5640 1 -- 172.21.15.105:0/3319517862 shutdown_connections 2023-11-28T18:24:16.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.647+0000 7f87d0df5640 1 --2- 172.21.15.105:0/3319517862 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f87ac0668d0 0x7f87ac068d90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.647+0000 7f87d0df5640 1 --2- 172.21.15.105:0/3319517862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87cc159f40 0x7f87cc1f6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.647+0000 7f87d0df5640 1 --2- 172.21.15.105:0/3319517862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87cc0cc760 0x7f87cc0b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.647+0000 7f87d0df5640 1 --2- 172.21.15.105:0/3319517862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87cc0ad680 0x7f87cc1518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.647+0000 7f87d0df5640 1 -- 172.21.15.105:0/3319517862 >> 172.21.15.105:0/3319517862 conn(0x7f87cc0ceb90 msgr2=0x7f87cc0ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:16.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.647+0000 7f87d0df5640 1 -- 172.21.15.105:0/3319517862 shutdown_connections 2023-11-28T18:24:16.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.647+0000 7f87d0df5640 1 -- 172.21.15.105:0/3319517862 wait complete. 2023-11-28T18:24:16.668 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:24:16.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.803+0000 7fcf8091e640 1 Processor -- start 2023-11-28T18:24:16.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.803+0000 7fcf8091e640 1 -- start start 2023-11-28T18:24:16.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.803+0000 7fcf8091e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcf7c0c99d0 0x7fcf7c0b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:16.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.803+0000 7fcf8091e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf7c0b1d00 0x7fcf7c071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:16.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.803+0000 7fcf8091e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf7c0cb230 con 0x7fcf7c15ae50 2023-11-28T18:24:16.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.803+0000 7fcf8091e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf7c0ca9d0 con 0x7fcf7c0c99d0 2023-11-28T18:24:16.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.803+0000 7fcf8091e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c0c9e50 con 0x7fcf7c0b1d00 2023-11-28T18:24:16.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.803+0000 7fcf7affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcf7c0c99d0 0x7fcf7c0b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:16.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.803+0000 7fcf7affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcf7c0c99d0 0x7fcf7c0b17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56160/0 (socket says 172.21.15.105:56160) 2023-11-28T18:24:16.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.803+0000 7fcf7affd640 1 -- 172.21.15.105:0/3782672477 learned_addr learned my addr 172.21.15.105:0/3782672477 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:16.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.803+0000 7fcf7bfff640 1 --2- 172.21.15.105:0/3782672477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf7c0b1d00 0x7fcf7c071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:16.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.803+0000 7fcf7affd640 1 -- 172.21.15.105:0/3782672477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf7c0b1d00 msgr2=0x7fcf7c071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:16.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.803+0000 7fcf7affd640 1 --2- 172.21.15.105:0/3782672477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf7c0b1d00 0x7fcf7c071660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.803+0000 7fcf7affd640 1 -- 172.21.15.105:0/3782672477 >> v1:172.21.15.148:6789/0 conn(0x7fcf7c15ae50 legacy=0x7fcf7c0cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:16.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.803+0000 7fcf7affd640 1 -- 172.21.15.105:0/3782672477 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7c0ca5c0 con 0x7fcf7c0c99d0 2023-11-28T18:24:16.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.807+0000 7fcf7affd640 1 --2- 172.21.15.105:0/3782672477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf7c0c99d0 0x7fcf7c0b17c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009870 tx=0x7fcf7001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e185bf8ea1bc3f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:16.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.807+0000 7fcf7a7fc640 1 -- 172.21.15.105:0/3782672477 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf700ab620 con 0x7fcf7c0c99d0 2023-11-28T18:24:16.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.807+0000 7fcf7a7fc640 1 -- 172.21.15.105:0/3782672477 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf700ab780 con 0x7fcf7c0c99d0 2023-11-28T18:24:16.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.807+0000 7fcf7a7fc640 1 -- 172.21.15.105:0/3782672477 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf700abb10 con 0x7fcf7c0c99d0 2023-11-28T18:24:16.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.807+0000 7fcf8091e640 1 -- 172.21.15.105:0/3782672477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf7c0c99d0 msgr2=0x7fcf7c0b17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:16.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.807+0000 7fcf8091e640 1 --2- 172.21.15.105:0/3782672477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf7c0c99d0 0x7fcf7c0b17c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009870 tx=0x7fcf7001c510 comp rx=0 tx=0).stop 2023-11-28T18:24:16.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.807+0000 7fcf8091e640 1 -- 172.21.15.105:0/3782672477 shutdown_connections 2023-11-28T18:24:16.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.807+0000 7fcf8091e640 1 --2- 172.21.15.105:0/3782672477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf7c0b1d00 0x7fcf7c071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.807+0000 7fcf8091e640 1 --2- 172.21.15.105:0/3782672477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf7c0c99d0 0x7fcf7c0b17c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.807+0000 7fcf8091e640 1 -- 172.21.15.105:0/3782672477 >> 172.21.15.105:0/3782672477 conn(0x7fcf7c0b0480 msgr2=0x7fcf7c0b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:16.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.807+0000 7fcf8091e640 1 -- 172.21.15.105:0/3782672477 shutdown_connections 2023-11-28T18:24:16.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.807+0000 7fcf8091e640 1 -- 172.21.15.105:0/3782672477 wait complete. 2023-11-28T18:24:16.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf8091e640 1 Processor -- start 2023-11-28T18:24:16.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf8091e640 1 -- start start 2023-11-28T18:24:16.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf8091e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf7c0b1d00 0x7fcf7c14cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:16.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf8091e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf7c0c99d0 0x7fcf7c0c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:16.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf8091e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf7c15ae50 0x7fcf7c0c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:16.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf8091e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c15bc30 con 0x7fcf7c0b1d00 2023-11-28T18:24:16.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf8091e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c0c88a0 con 0x7fcf7c0c99d0 2023-11-28T18:24:16.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf8091e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c156120 con 0x7fcf7c15ae50 2023-11-28T18:24:16.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf7b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf7c0b1d00 0x7fcf7c14cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:16.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf7b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf7c0b1d00 0x7fcf7c14cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56168/0 (socket says 172.21.15.105:56168) 2023-11-28T18:24:16.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf7b7fe640 1 -- 172.21.15.105:0/2164739762 learned_addr learned my addr 172.21.15.105:0/2164739762 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:16.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf7affd640 1 --2- 172.21.15.105:0/2164739762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf7c0c99d0 0x7fcf7c0c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:16.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf7bfff640 1 --2- 172.21.15.105:0/2164739762 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf7c15ae50 0x7fcf7c0c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:16.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf7b7fe640 1 -- 172.21.15.105:0/2164739762 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf7c15ae50 msgr2=0x7fcf7c0c5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:16.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf7b7fe640 1 --2- 172.21.15.105:0/2164739762 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf7c15ae50 0x7fcf7c0c5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf7b7fe640 1 -- 172.21.15.105:0/2164739762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf7c0c99d0 msgr2=0x7fcf7c0c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:16.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf7b7fe640 1 --2- 172.21.15.105:0/2164739762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf7c0c99d0 0x7fcf7c0c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf7b7fe640 1 -- 172.21.15.105:0/2164739762 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf700094d0 con 0x7fcf7c0b1d00 2023-11-28T18:24:16.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf7b7fe640 1 --2- 172.21.15.105:0/2164739762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf7c0b1d00 0x7fcf7c14cc20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00c500 tx=0x7fcf6c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:16.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf63fff640 1 -- 172.21.15.105:0/2164739762 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c005220 con 0x7fcf7c0b1d00 2023-11-28T18:24:16.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf63fff640 1 -- 172.21.15.105:0/2164739762 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6c005380 con 0x7fcf7c0b1d00 2023-11-28T18:24:16.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf63fff640 1 -- 172.21.15.105:0/2164739762 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c007270 con 0x7fcf7c0b1d00 2023-11-28T18:24:16.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf8091e640 1 -- 172.21.15.105:0/2164739762 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7c0c85c0 con 0x7fcf7c0b1d00 2023-11-28T18:24:16.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf8091e640 1 -- 172.21.15.105:0/2164739762 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7c15b8c0 con 0x7fcf7c0b1d00 2023-11-28T18:24:16.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.811+0000 7fcf8091e640 1 -- 172.21.15.105:0/2164739762 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf7c0ae160 con 0x7fcf7c0b1d00 2023-11-28T18:24:16.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.815+0000 7fcf63fff640 1 -- 172.21.15.105:0/2164739762 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fcf6c0073d0 con 0x7fcf7c0b1d00 2023-11-28T18:24:16.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.815+0000 7fcf63fff640 1 --2- 172.21.15.105:0/2164739762 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcf680668d0 0x7fcf68068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:16.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.815+0000 7fcf63fff640 1 -- 172.21.15.105:0/2164739762 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fcf6c0841c0 con 0x7fcf7c0b1d00 2023-11-28T18:24:16.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.815+0000 7fcf7affd640 1 --2- 172.21.15.105:0/2164739762 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcf680668d0 0x7fcf68068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:16.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.815+0000 7fcf7affd640 1 --2- 172.21.15.105:0/2164739762 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcf680668d0 0x7fcf68068d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcf700af000 tx=0x7fcf700a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:16.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.819+0000 7fcf63fff640 1 -- 172.21.15.105:0/2164739762 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcf6c050f70 con 0x7fcf7c0b1d00 2023-11-28T18:24:16.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.967+0000 7fcf8091e640 1 -- 172.21.15.105:0/2164739762 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf7c15b440 con 0x7fcf7c0b1d00 2023-11-28T18:24:16.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.971+0000 7fcf63fff640 1 -- 172.21.15.105:0/2164739762 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18887 (secure 0 0 0) 0x7fcf7c15b440 con 0x7fcf7c0b1d00 2023-11-28T18:24:16.974 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:16.974 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:24:15.048168+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:24:16.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.971+0000 7fcf8091e640 1 -- 172.21.15.105:0/2164739762 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcf680668d0 msgr2=0x7fcf68068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:16.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.971+0000 7fcf8091e640 1 --2- 172.21.15.105:0/2164739762 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcf680668d0 0x7fcf68068d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcf700af000 tx=0x7fcf700a7000 comp rx=0 tx=0).stop 2023-11-28T18:24:16.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.971+0000 7fcf8091e640 1 -- 172.21.15.105:0/2164739762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf7c0b1d00 msgr2=0x7fcf7c14cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:16.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.971+0000 7fcf8091e640 1 --2- 172.21.15.105:0/2164739762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf7c0b1d00 0x7fcf7c14cc20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00c500 tx=0x7fcf6c00c9c0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.971+0000 7fcf8091e640 1 -- 172.21.15.105:0/2164739762 shutdown_connections 2023-11-28T18:24:16.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.971+0000 7fcf8091e640 1 --2- 172.21.15.105:0/2164739762 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf7c15ae50 0x7fcf7c0c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.971+0000 7fcf8091e640 1 --2- 172.21.15.105:0/2164739762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf7c0c99d0 0x7fcf7c0c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.971+0000 7fcf8091e640 1 --2- 172.21.15.105:0/2164739762 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcf680668d0 0x7fcf68068d90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.971+0000 7fcf8091e640 1 --2- 172.21.15.105:0/2164739762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf7c0b1d00 0x7fcf7c14cc20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:16.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.971+0000 7fcf8091e640 1 -- 172.21.15.105:0/2164739762 >> 172.21.15.105:0/2164739762 conn(0x7fcf7c0b0480 msgr2=0x7fcf7c0b3670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:16.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.971+0000 7fcf8091e640 1 -- 172.21.15.105:0/2164739762 shutdown_connections 2023-11-28T18:24:16.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:16.971+0000 7fcf8091e640 1 -- 172.21.15.105:0/2164739762 wait complete. 2023-11-28T18:24:16.995 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:23:31.485422+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:23:32.274378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T18:23:36.100071+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T18:24:16.996 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:24:17.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.131+0000 7f11738fe640 1 Processor -- start 2023-11-28T18:24:17.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.131+0000 7f11738fe640 1 -- start start 2023-11-28T18:24:17.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.135+0000 7f11738fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f116c071a10 0x7f116c0ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.135+0000 7f11738fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f116c070a10 0x7f116c070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.135+0000 7f11738fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f116c0713a0 con 0x7f116c15d3c0 2023-11-28T18:24:17.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.135+0000 7f11738fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116c0cefa0 con 0x7f116c071a10 2023-11-28T18:24:17.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.135+0000 7f11738fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116c15d7a0 con 0x7f116c070a10 2023-11-28T18:24:17.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.135+0000 7f1170e99640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f116c071a10 0x7f116c0ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.135+0000 7f1170e99640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f116c071a10 0x7f116c0ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56176/0 (socket says 172.21.15.105:56176) 2023-11-28T18:24:17.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.135+0000 7f1170e99640 1 -- 172.21.15.105:0/2770337703 learned_addr learned my addr 172.21.15.105:0/2770337703 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:17.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.135+0000 7f1171e9b640 1 --2- 172.21.15.105:0/2770337703 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f116c070a10 0x7f116c070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.135+0000 7f1170e99640 1 -- 172.21.15.105:0/2770337703 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f116c070a10 msgr2=0x7f116c070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:17.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.135+0000 7f1170e99640 1 --2- 172.21.15.105:0/2770337703 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f116c070a10 0x7f116c070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.135+0000 7f1170e99640 1 -- 172.21.15.105:0/2770337703 >> v1:172.21.15.105:6789/0 conn(0x7f116c15d3c0 legacy=0x7f116c161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:17.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.135+0000 7f1170e99640 1 -- 172.21.15.105:0/2770337703 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116c0ce9e0 con 0x7f116c071a10 2023-11-28T18:24:17.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.135+0000 7f1171e9b640 1 --2- 172.21.15.105:0/2770337703 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f116c070a10 0x7f116c070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:17.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.139+0000 7f1170e99640 1 --2- 172.21.15.105:0/2770337703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f116c071a10 0x7f116c0ce190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f115c009870 tx=0x7f115c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceacfb15427fb78e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:17.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.139+0000 7f1163fff640 1 -- 172.21.15.105:0/2770337703 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c0ab680 con 0x7f116c071a10 2023-11-28T18:24:17.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.139+0000 7f1163fff640 1 -- 172.21.15.105:0/2770337703 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115c0ab7e0 con 0x7f116c071a10 2023-11-28T18:24:17.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.139+0000 7f1163fff640 1 -- 172.21.15.105:0/2770337703 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c0abb70 con 0x7f116c071a10 2023-11-28T18:24:17.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.139+0000 7f11738fe640 1 -- 172.21.15.105:0/2770337703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f116c071a10 msgr2=0x7f116c0ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:17.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.139+0000 7f11738fe640 1 --2- 172.21.15.105:0/2770337703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f116c071a10 0x7f116c0ce190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f115c009870 tx=0x7f115c01c480 comp rx=0 tx=0).stop 2023-11-28T18:24:17.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.139+0000 7f11738fe640 1 -- 172.21.15.105:0/2770337703 shutdown_connections 2023-11-28T18:24:17.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.139+0000 7f11738fe640 1 --2- 172.21.15.105:0/2770337703 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f116c070a10 0x7f116c070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.139+0000 7f11738fe640 1 --2- 172.21.15.105:0/2770337703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f116c071a10 0x7f116c0ce190 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.139+0000 7f11738fe640 1 -- 172.21.15.105:0/2770337703 >> 172.21.15.105:0/2770337703 conn(0x7f116c0cf540 msgr2=0x7f116c0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:17.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.139+0000 7f11738fe640 1 -- 172.21.15.105:0/2770337703 shutdown_connections 2023-11-28T18:24:17.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.139+0000 7f11738fe640 1 -- 172.21.15.105:0/2770337703 wait complete. 2023-11-28T18:24:17.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.139+0000 7f11738fe640 1 Processor -- start 2023-11-28T18:24:17.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.139+0000 7f11738fe640 1 -- start start 2023-11-28T18:24:17.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f11738fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f116c070a10 0x7f116c1545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f11738fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f116c071a10 0x7f116c154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f11738fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f116c15d3c0 0x7f116c15fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f11738fe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c154b30 con 0x7f116c071a10 2023-11-28T18:24:17.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f11738fe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c15d8e0 con 0x7f116c070a10 2023-11-28T18:24:17.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f11738fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116c1555e0 con 0x7f116c15d3c0 2023-11-28T18:24:17.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f1170e99640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f116c071a10 0x7f116c154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f1170e99640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f116c071a10 0x7f116c154ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56180/0 (socket says 172.21.15.105:56180) 2023-11-28T18:24:17.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f1170e99640 1 -- 172.21.15.105:0/757427633 learned_addr learned my addr 172.21.15.105:0/757427633 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:17.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f1171e9b640 1 --2- 172.21.15.105:0/757427633 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f116c15d3c0 0x7f116c15fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f117169a640 1 --2- 172.21.15.105:0/757427633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f116c070a10 0x7f116c1545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f1170e99640 1 -- 172.21.15.105:0/757427633 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f116c15d3c0 msgr2=0x7f116c15fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:17.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f1170e99640 1 --2- 172.21.15.105:0/757427633 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f116c15d3c0 0x7f116c15fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f1170e99640 1 -- 172.21.15.105:0/757427633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f116c070a10 msgr2=0x7f116c1545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:17.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f1170e99640 1 --2- 172.21.15.105:0/757427633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f116c070a10 0x7f116c1545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f1170e99640 1 -- 172.21.15.105:0/757427633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115c0094d0 con 0x7f116c071a10 2023-11-28T18:24:17.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f1170e99640 1 --2- 172.21.15.105:0/757427633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f116c071a10 0x7f116c154ce0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f115c01a0b0 tx=0x7f115c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:17.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f1161ffb640 1 -- 172.21.15.105:0/757427633 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c0b4800 con 0x7f116c071a10 2023-11-28T18:24:17.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f1161ffb640 1 -- 172.21.15.105:0/757427633 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115c0b4960 con 0x7f116c071a10 2023-11-28T18:24:17.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f1161ffb640 1 -- 172.21.15.105:0/757427633 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c0b4d10 con 0x7f116c071a10 2023-11-28T18:24:17.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f11738fe640 1 -- 172.21.15.105:0/757427633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f116c1558c0 con 0x7f116c071a10 2023-11-28T18:24:17.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f11738fe640 1 -- 172.21.15.105:0/757427633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116c155e80 con 0x7f116c071a10 2023-11-28T18:24:17.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.143+0000 7f11738fe640 1 -- 172.21.15.105:0/757427633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116c15da50 con 0x7f116c071a10 2023-11-28T18:24:17.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.147+0000 7f1161ffb640 1 -- 172.21.15.105:0/757427633 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f115c0c7440 con 0x7f116c071a10 2023-11-28T18:24:17.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.147+0000 7f1161ffb640 1 --2- 172.21.15.105:0/757427633 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1150066900 0x7f1150068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.147+0000 7f1161ffb640 1 -- 172.21.15.105:0/757427633 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f115c11f6b0 con 0x7f116c071a10 2023-11-28T18:24:17.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.147+0000 7f117169a640 1 --2- 172.21.15.105:0/757427633 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1150066900 0x7f1150068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.147+0000 7f117169a640 1 --2- 172.21.15.105:0/757427633 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1150066900 0x7f1150068dc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1168000f70 tx=0x7f1168009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:17.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.151+0000 7f1161ffb640 1 -- 172.21.15.105:0/757427633 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f115c0ec580 con 0x7f116c071a10 2023-11-28T18:24:17.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.303+0000 7f11738fe640 1 -- 172.21.15.105:0/757427633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f116c067570 con 0x7f116c071a10 2023-11-28T18:24:17.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.303+0000 7f1161ffb640 1 -- 172.21.15.105:0/757427633 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f115c0efba0 con 0x7f116c071a10 2023-11-28T18:24:17.307 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:24:17.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.303+0000 7f11738fe640 1 -- 172.21.15.105:0/757427633 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1150066900 msgr2=0x7f1150068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:17.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.303+0000 7f11738fe640 1 --2- 172.21.15.105:0/757427633 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1150066900 0x7f1150068dc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1168000f70 tx=0x7f1168009040 comp rx=0 tx=0).stop 2023-11-28T18:24:17.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.303+0000 7f11738fe640 1 -- 172.21.15.105:0/757427633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f116c071a10 msgr2=0x7f116c154ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:17.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.303+0000 7f11738fe640 1 --2- 172.21.15.105:0/757427633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f116c071a10 0x7f116c154ce0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f115c01a0b0 tx=0x7f115c01acc0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.303+0000 7f11738fe640 1 -- 172.21.15.105:0/757427633 shutdown_connections 2023-11-28T18:24:17.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.303+0000 7f11738fe640 1 --2- 172.21.15.105:0/757427633 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f116c15d3c0 0x7f116c15fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.303+0000 7f11738fe640 1 --2- 172.21.15.105:0/757427633 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1150066900 0x7f1150068dc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.303+0000 7f11738fe640 1 --2- 172.21.15.105:0/757427633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f116c071a10 0x7f116c154ce0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.303+0000 7f11738fe640 1 --2- 172.21.15.105:0/757427633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f116c070a10 0x7f116c1545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.303+0000 7f11738fe640 1 -- 172.21.15.105:0/757427633 >> 172.21.15.105:0/757427633 conn(0x7f116c0cf540 msgr2=0x7f116c06af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:17.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.303+0000 7f11738fe640 1 -- 172.21.15.105:0/757427633 shutdown_connections 2023-11-28T18:24:17.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.303+0000 7f11738fe640 1 -- 172.21.15.105:0/757427633 wait complete. 2023-11-28T18:24:17.328 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:24:17.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.463+0000 7f28ae780640 1 Processor -- start 2023-11-28T18:24:17.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.463+0000 7f28ae780640 1 -- start start 2023-11-28T18:24:17.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28ae780640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28a8071a10 0x7f28a80ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28ae780640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f28a8070a10 0x7f28a8070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28ae780640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28a80cefa0 con 0x7f28a815d3c0 2023-11-28T18:24:17.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28ae780640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28a80713c0 con 0x7f28a8070a10 2023-11-28T18:24:17.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28ae780640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a815d7a0 con 0x7f28a8071a10 2023-11-28T18:24:17.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a7fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f28a815d3c0 0x7f28a8161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55848/0 (socket says 172.21.15.105:55848) 2023-11-28T18:24:17.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a7fff640 1 -- 172.21.15.105:0/2044295196 learned_addr learned my addr 172.21.15.105:0/2044295196 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:17.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28acd1d640 1 --2- 172.21.15.105:0/2044295196 >> v2:172.21.15.148:3300/0 conn(0x7f28a8070a10 0x7f28a8070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a77fe640 1 --2- 172.21.15.105:0/2044295196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28a8071a10 0x7f28a80ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a6ffd640 1 -- 172.21.15.105:0/2044295196 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2737693048 0 0) 0x7f28a80cefa0 con 0x7f28a815d3c0 2023-11-28T18:24:17.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a6ffd640 1 -- 172.21.15.105:0/2044295196 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2890003530 con 0x7f28a815d3c0 2023-11-28T18:24:17.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a6ffd640 1 -- 172.21.15.105:0/2044295196 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 253326998 0 0) 0x7f2890003530 con 0x7f28a815d3c0 2023-11-28T18:24:17.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a6ffd640 1 -- 172.21.15.105:0/2044295196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28a8071a10 msgr2=0x7f28a80ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:17.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a6ffd640 1 --2- 172.21.15.105:0/2044295196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28a8071a10 0x7f28a80ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a6ffd640 1 -- 172.21.15.105:0/2044295196 >> v2:172.21.15.148:3300/0 conn(0x7f28a8070a10 msgr2=0x7f28a8070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:17.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a6ffd640 1 --2- 172.21.15.105:0/2044295196 >> v2:172.21.15.148:3300/0 conn(0x7f28a8070a10 0x7f28a8070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a6ffd640 1 -- 172.21.15.105:0/2044295196 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a80ce9e0 con 0x7f28a815d3c0 2023-11-28T18:24:17.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a6ffd640 1 -- 172.21.15.105:0/2044295196 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f2898001680 con 0x7f28a815d3c0 2023-11-28T18:24:17.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a6ffd640 1 -- 172.21.15.105:0/2044295196 >> v1:172.21.15.105:6789/0 conn(0x7f28a815d3c0 legacy=0x7f28a8161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:17.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a6ffd640 1 --2- 172.21.15.105:0/2044295196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2890003980 0x7f2890003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a6ffd640 1 --2- 172.21.15.105:0/2044295196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28900047d0 0x7f2890004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a6ffd640 1 --2- 172.21.15.105:0/2044295196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28a8071a10 0x7f2890005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a6ffd640 1 -- 172.21.15.105:0/2044295196 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a80713c0 con 0x7f2890003980 2023-11-28T18:24:17.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a6ffd640 1 -- 172.21.15.105:0/2044295196 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a815d7a0 con 0x7f28900047d0 2023-11-28T18:24:17.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a6ffd640 1 -- 172.21.15.105:0/2044295196 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289000b9c0 con 0x7f28a8071a10 2023-11-28T18:24:17.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a7fff640 1 --2- 172.21.15.105:0/2044295196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2890003980 0x7f2890003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a77fe640 1 --2- 172.21.15.105:0/2044295196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28900047d0 0x7f2890004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28acd1d640 1 --2- 172.21.15.105:0/2044295196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28a8071a10 0x7f2890005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a7fff640 1 -- 172.21.15.105:0/2044295196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28a8071a10 msgr2=0x7f2890005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:17.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a7fff640 1 --2- 172.21.15.105:0/2044295196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28a8071a10 0x7f2890005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a7fff640 1 -- 172.21.15.105:0/2044295196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28900047d0 msgr2=0x7f2890004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:17.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a7fff640 1 --2- 172.21.15.105:0/2044295196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28900047d0 0x7f2890004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28a7fff640 1 -- 172.21.15.105:0/2044295196 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2890003530 con 0x7f2890003980 2023-11-28T18:24:17.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.467+0000 7f28acd1d640 1 --2- 172.21.15.105:0/2044295196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28a8071a10 0x7f2890005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:17.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.471+0000 7f28a7fff640 1 --2- 172.21.15.105:0/2044295196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2890003980 0x7f2890003da0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f289800c240 tx=0x7f289801cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d90d298e7ac2fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:17.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.471+0000 7f28a6ffd640 1 -- 172.21.15.105:0/2044295196 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28980058e0 con 0x7f2890003980 2023-11-28T18:24:17.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.471+0000 7f28a6ffd640 1 -- 172.21.15.105:0/2044295196 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2898005a40 con 0x7f2890003980 2023-11-28T18:24:17.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.471+0000 7f28a6ffd640 1 -- 172.21.15.105:0/2044295196 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2898005dd0 con 0x7f2890003980 2023-11-28T18:24:17.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.471+0000 7f28ae780640 1 -- 172.21.15.105:0/2044295196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2890003980 msgr2=0x7f2890003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:17.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.471+0000 7f28ae780640 1 --2- 172.21.15.105:0/2044295196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2890003980 0x7f2890003da0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f289800c240 tx=0x7f289801cbd0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.471+0000 7f28acd1d640 1 -- 172.21.15.105:0/2044295196 reap_dead start 2023-11-28T18:24:17.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.471+0000 7f28ae780640 1 -- 172.21.15.105:0/2044295196 shutdown_connections 2023-11-28T18:24:17.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.471+0000 7f28ae780640 1 -- 172.21.15.105:0/2044295196 >> 172.21.15.105:0/2044295196 conn(0x7f28a80cf540 msgr2=0x7f28a80cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:17.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.471+0000 7f28ae780640 1 -- 172.21.15.105:0/2044295196 shutdown_connections 2023-11-28T18:24:17.479 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.471+0000 7f28ae780640 1 -- 172.21.15.105:0/2044295196 wait complete. 2023-11-28T18:24:17.479 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28ae780640 1 Processor -- start 2023-11-28T18:24:17.479 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28ae780640 1 -- start start 2023-11-28T18:24:17.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28ae780640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2890003980 0x7f28a815f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28ae780640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28a81f9780 0x7f28a814c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28ae780640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28a815a6d0 0x7f28a815ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28ae780640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a815b090 con 0x7f2890003980 2023-11-28T18:24:17.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28ae780640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a815fea0 con 0x7f28a815a6d0 2023-11-28T18:24:17.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28ae780640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a8159f70 con 0x7f28a81f9780 2023-11-28T18:24:17.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28a7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2890003980 0x7f28a815f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28a7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2890003980 0x7f28a815f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56194/0 (socket says 172.21.15.105:56194) 2023-11-28T18:24:17.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28a7fff640 1 -- 172.21.15.105:0/4179220820 learned_addr learned my addr 172.21.15.105:0/4179220820 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:17.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28a77fe640 1 --2- 172.21.15.105:0/4179220820 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28a81f9780 0x7f28a814c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28acd1d640 1 --2- 172.21.15.105:0/4179220820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28a815a6d0 0x7f28a815ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28a7fff640 1 -- 172.21.15.105:0/4179220820 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28a81f9780 msgr2=0x7f28a814c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:17.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28a7fff640 1 --2- 172.21.15.105:0/4179220820 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28a81f9780 0x7f28a814c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28a7fff640 1 -- 172.21.15.105:0/4179220820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28a815a6d0 msgr2=0x7f28a815ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:17.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28a7fff640 1 --2- 172.21.15.105:0/4179220820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28a815a6d0 0x7f28a815ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28a7fff640 1 -- 172.21.15.105:0/4179220820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a8159ae0 con 0x7f2890003980 2023-11-28T18:24:17.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28a7fff640 1 --2- 172.21.15.105:0/4179220820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2890003980 0x7f28a815f960 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f28980ab040 tx=0x7f2890005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:17.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28a4ff9640 1 -- 172.21.15.105:0/4179220820 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28980049f0 con 0x7f2890003980 2023-11-28T18:24:17.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28a4ff9640 1 -- 172.21.15.105:0/4179220820 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2898004b50 con 0x7f2890003980 2023-11-28T18:24:17.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28a4ff9640 1 -- 172.21.15.105:0/4179220820 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2898005e40 con 0x7f2890003980 2023-11-28T18:24:17.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28ae780640 1 -- 172.21.15.105:0/4179220820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a8158ed0 con 0x7f2890003980 2023-11-28T18:24:17.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28ae780640 1 -- 172.21.15.105:0/4179220820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a8159490 con 0x7f2890003980 2023-11-28T18:24:17.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.475+0000 7f28ae780640 1 -- 172.21.15.105:0/4179220820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a8067570 con 0x7f2890003980 2023-11-28T18:24:17.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.479+0000 7f28a4ff9640 1 -- 172.21.15.105:0/4179220820 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f28980a5740 con 0x7f2890003980 2023-11-28T18:24:17.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.479+0000 7f28a4ff9640 1 --2- 172.21.15.105:0/4179220820 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f28940668d0 0x7f2894068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.479+0000 7f28a4ff9640 1 -- 172.21.15.105:0/4179220820 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f289811e420 con 0x7f2890003980 2023-11-28T18:24:17.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.479+0000 7f28a77fe640 1 --2- 172.21.15.105:0/4179220820 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f28940668d0 0x7f2894068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.479+0000 7f28a77fe640 1 --2- 172.21.15.105:0/4179220820 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f28940668d0 0x7f2894068d90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f28a80b1270 tx=0x7f289c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:17.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.483+0000 7f28a4ff9640 1 -- 172.21.15.105:0/4179220820 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f28980eb1c0 con 0x7f2890003980 2023-11-28T18:24:17.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.631+0000 7f28ae780640 1 -- 172.21.15.105:0/4179220820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f28a8066960 con 0x7f2890003980 2023-11-28T18:24:17.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.631+0000 7f28a4ff9640 1 -- 172.21.15.105:0/4179220820 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f28980ee7e0 con 0x7f2890003980 2023-11-28T18:24:17.635 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:17.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.631+0000 7f28ae780640 1 -- 172.21.15.105:0/4179220820 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f28940668d0 msgr2=0x7f2894068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:17.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.631+0000 7f28ae780640 1 --2- 172.21.15.105:0/4179220820 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f28940668d0 0x7f2894068d90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f28a80b1270 tx=0x7f289c009190 comp rx=0 tx=0).stop 2023-11-28T18:24:17.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.631+0000 7f28ae780640 1 -- 172.21.15.105:0/4179220820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2890003980 msgr2=0x7f28a815f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:17.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.631+0000 7f28ae780640 1 --2- 172.21.15.105:0/4179220820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2890003980 0x7f28a815f960 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f28980ab040 tx=0x7f2890005800 comp rx=0 tx=0).stop 2023-11-28T18:24:17.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.631+0000 7f28ae780640 1 -- 172.21.15.105:0/4179220820 shutdown_connections 2023-11-28T18:24:17.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.631+0000 7f28ae780640 1 --2- 172.21.15.105:0/4179220820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28a815a6d0 0x7f28a815ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.631+0000 7f28ae780640 1 --2- 172.21.15.105:0/4179220820 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28a81f9780 0x7f28a814c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.631+0000 7f28ae780640 1 --2- 172.21.15.105:0/4179220820 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f28940668d0 0x7f2894068d90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.631+0000 7f28ae780640 1 --2- 172.21.15.105:0/4179220820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2890003980 0x7f28a815f960 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.631+0000 7f28ae780640 1 -- 172.21.15.105:0/4179220820 >> 172.21.15.105:0/4179220820 conn(0x7f28a80cf540 msgr2=0x7f28a80cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:17.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.631+0000 7f28ae780640 1 -- 172.21.15.105:0/4179220820 shutdown_connections 2023-11-28T18:24:17.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.631+0000 7f28ae780640 1 -- 172.21.15.105:0/4179220820 wait complete. 2023-11-28T18:24:17.656 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:24:17.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.791+0000 7f11d17f8640 1 Processor -- start 2023-11-28T18:24:17.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.791+0000 7f11d17f8640 1 -- start start 2023-11-28T18:24:17.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11d17f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11cc0b4290 0x7f11cc0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11d17f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f11cc15b030 0x7f11cc0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11d17f8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11cc0b4bb0 con 0x7f11cc071960 2023-11-28T18:24:17.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11d17f8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11cc0701d0 con 0x7f11cc15b030 2023-11-28T18:24:17.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11d17f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc06df80 con 0x7f11cc0b4290 2023-11-28T18:24:17.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11c27fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f11cc071960 0x7f11cc15f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55854/0 (socket says 172.21.15.105:55854) 2023-11-28T18:24:17.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11c27fc640 1 -- 172.21.15.105:0/6044849 learned_addr learned my addr 172.21.15.105:0/6044849 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:17.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11caffd640 1 --2- 172.21.15.105:0/6044849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11cc0b4290 0x7f11cc0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11cb7fe640 1 --2- 172.21.15.105:0/6044849 >> v2:172.21.15.148:3300/0 conn(0x7f11cc15b030 0x7f11cc0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11ca7fc640 1 -- 172.21.15.105:0/6044849 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3090539152 0 0) 0x7f11cc0b4bb0 con 0x7f11cc071960 2023-11-28T18:24:17.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11ca7fc640 1 -- 172.21.15.105:0/6044849 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11b8003530 con 0x7f11cc071960 2023-11-28T18:24:17.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11ca7fc640 1 -- 172.21.15.105:0/6044849 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4005354667 0 0) 0x7f11b8003530 con 0x7f11cc071960 2023-11-28T18:24:17.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11ca7fc640 1 -- 172.21.15.105:0/6044849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11cc0b4290 msgr2=0x7f11cc0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:17.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11ca7fc640 1 --2- 172.21.15.105:0/6044849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11cc0b4290 0x7f11cc0b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11ca7fc640 1 -- 172.21.15.105:0/6044849 >> v2:172.21.15.148:3300/0 conn(0x7f11cc15b030 msgr2=0x7f11cc0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:17.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11ca7fc640 1 --2- 172.21.15.105:0/6044849 >> v2:172.21.15.148:3300/0 conn(0x7f11cc15b030 0x7f11cc0709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11ca7fc640 1 -- 172.21.15.105:0/6044849 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11cc071df0 con 0x7f11cc071960 2023-11-28T18:24:17.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11ca7fc640 1 -- 172.21.15.105:0/6044849 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f11bc001680 con 0x7f11cc071960 2023-11-28T18:24:17.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11ca7fc640 1 -- 172.21.15.105:0/6044849 >> v1:172.21.15.105:6789/0 conn(0x7f11cc071960 legacy=0x7f11cc15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:17.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11ca7fc640 1 --2- 172.21.15.105:0/6044849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11b8003980 0x7f11b8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11ca7fc640 1 --2- 172.21.15.105:0/6044849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11cc0b4290 0x7f11b80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11ca7fc640 1 --2- 172.21.15.105:0/6044849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11b8004d10 0x7f11b8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11ca7fc640 1 -- 172.21.15.105:0/6044849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc0701d0 con 0x7f11b8003980 2023-11-28T18:24:17.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11ca7fc640 1 -- 172.21.15.105:0/6044849 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc06df80 con 0x7f11b8004d10 2023-11-28T18:24:17.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11ca7fc640 1 -- 172.21.15.105:0/6044849 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b800b990 con 0x7f11cc0b4290 2023-11-28T18:24:17.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11caffd640 1 --2- 172.21.15.105:0/6044849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11b8003980 0x7f11b8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11cb7fe640 1 --2- 172.21.15.105:0/6044849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11b8004d10 0x7f11b8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11c27fc640 1 --2- 172.21.15.105:0/6044849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11cc0b4290 0x7f11b80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11caffd640 1 -- 172.21.15.105:0/6044849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11cc0b4290 msgr2=0x7f11b80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:17.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11caffd640 1 --2- 172.21.15.105:0/6044849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11cc0b4290 0x7f11b80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11caffd640 1 -- 172.21.15.105:0/6044849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11b8004d10 msgr2=0x7f11b8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:17.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11caffd640 1 --2- 172.21.15.105:0/6044849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11b8004d10 0x7f11b8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.795+0000 7f11caffd640 1 -- 172.21.15.105:0/6044849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b8003530 con 0x7f11b8003980 2023-11-28T18:24:17.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.799+0000 7f11caffd640 1 --2- 172.21.15.105:0/6044849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11b8003980 0x7f11b8003da0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009ae0 tx=0x7f11b401ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b3598e40ae4641 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:17.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.799+0000 7f11ca7fc640 1 -- 172.21.15.105:0/6044849 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b40a6510 con 0x7f11b8003980 2023-11-28T18:24:17.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.799+0000 7f11ca7fc640 1 -- 172.21.15.105:0/6044849 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b40a6670 con 0x7f11b8003980 2023-11-28T18:24:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.799+0000 7f11ca7fc640 1 -- 172.21.15.105:0/6044849 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b40a6a20 con 0x7f11b8003980 2023-11-28T18:24:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.799+0000 7f11d17f8640 1 -- 172.21.15.105:0/6044849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11b8003980 msgr2=0x7f11b8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.799+0000 7f11d17f8640 1 --2- 172.21.15.105:0/6044849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11b8003980 0x7f11b8003da0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009ae0 tx=0x7f11b401ca20 comp rx=0 tx=0).stop 2023-11-28T18:24:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.799+0000 7f11cb7fe640 1 -- 172.21.15.105:0/6044849 reap_dead start 2023-11-28T18:24:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.799+0000 7f11d17f8640 1 -- 172.21.15.105:0/6044849 shutdown_connections 2023-11-28T18:24:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.799+0000 7f11d17f8640 1 -- 172.21.15.105:0/6044849 >> 172.21.15.105:0/6044849 conn(0x7f11cc0ceb90 msgr2=0x7f11cc0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.799+0000 7f11d17f8640 1 -- 172.21.15.105:0/6044849 shutdown_connections 2023-11-28T18:24:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.799+0000 7f11d17f8640 1 -- 172.21.15.105:0/6044849 wait complete. 2023-11-28T18:24:17.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.799+0000 7f11d17f8640 1 Processor -- start 2023-11-28T18:24:17.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.799+0000 7f11d17f8640 1 -- start start 2023-11-28T18:24:17.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11d17f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11b8003980 0x7f11cc14b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11d17f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11cc1583b0 0x7f11cc1640a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11d17f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11cc159b10 0x7f11cc0cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11d17f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc162db0 con 0x7f11b8003980 2023-11-28T18:24:17.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11d17f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc163000 con 0x7f11cc159b10 2023-11-28T18:24:17.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11d17f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc1632e0 con 0x7f11cc1583b0 2023-11-28T18:24:17.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11caffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11b8003980 0x7f11cc14b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11caffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11b8003980 0x7f11cc14b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56218/0 (socket says 172.21.15.105:56218) 2023-11-28T18:24:17.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11caffd640 1 -- 172.21.15.105:0/1068997642 learned_addr learned my addr 172.21.15.105:0/1068997642 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:17.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11cb7fe640 1 --2- 172.21.15.105:0/1068997642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11cc159b10 0x7f11cc0cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11c27fc640 1 --2- 172.21.15.105:0/1068997642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11cc1583b0 0x7f11cc1640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11caffd640 1 -- 172.21.15.105:0/1068997642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11cc1583b0 msgr2=0x7f11cc1640a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:17.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11caffd640 1 --2- 172.21.15.105:0/1068997642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11cc1583b0 0x7f11cc1640a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11caffd640 1 -- 172.21.15.105:0/1068997642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11cc159b10 msgr2=0x7f11cc0cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:17.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11caffd640 1 --2- 172.21.15.105:0/1068997642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11cc159b10 0x7f11cc0cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11caffd640 1 -- 172.21.15.105:0/1068997642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11cc1635c0 con 0x7f11b8003980 2023-11-28T18:24:17.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11caffd640 1 --2- 172.21.15.105:0/1068997642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11b8003980 0x7f11cc14b5f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f11b40b0040 tx=0x7f11b8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:17.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11c3fff640 1 -- 172.21.15.105:0/1068997642 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b40be070 con 0x7f11b8003980 2023-11-28T18:24:17.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11c3fff640 1 -- 172.21.15.105:0/1068997642 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b400f000 con 0x7f11b8003980 2023-11-28T18:24:17.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11c3fff640 1 -- 172.21.15.105:0/1068997642 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b40059a0 con 0x7f11b8003980 2023-11-28T18:24:17.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11d17f8640 1 -- 172.21.15.105:0/1068997642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11cc1638a0 con 0x7f11b8003980 2023-11-28T18:24:17.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11d17f8640 1 -- 172.21.15.105:0/1068997642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11cc14bb30 con 0x7f11b8003980 2023-11-28T18:24:17.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11d17f8640 1 -- 172.21.15.105:0/1068997642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11cc067570 con 0x7f11b8003980 2023-11-28T18:24:17.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.803+0000 7f11c3fff640 1 -- 172.21.15.105:0/1068997642 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f11b4005b00 con 0x7f11b8003980 2023-11-28T18:24:17.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.807+0000 7f11c3fff640 1 --2- 172.21.15.105:0/1068997642 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f11ac0668d0 0x7f11ac068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:17.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.807+0000 7f11c3fff640 1 -- 172.21.15.105:0/1068997642 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f11b40aa070 con 0x7f11b8003980 2023-11-28T18:24:17.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.807+0000 7f11c27fc640 1 --2- 172.21.15.105:0/1068997642 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f11ac0668d0 0x7f11ac068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:17.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.807+0000 7f11c27fc640 1 --2- 172.21.15.105:0/1068997642 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f11ac0668d0 0x7f11ac068d90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f11cc06e6c0 tx=0x7f11bc007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:17.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.811+0000 7f11c3fff640 1 -- 172.21.15.105:0/1068997642 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f11b40eb550 con 0x7f11b8003980 2023-11-28T18:24:17.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.959+0000 7f11d17f8640 1 -- 172.21.15.105:0/1068997642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f11cc0cf7f0 con 0x7f11b8003980 2023-11-28T18:24:17.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.959+0000 7f11c3fff640 1 -- 172.21.15.105:0/1068997642 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f11b40eeb70 con 0x7f11b8003980 2023-11-28T18:24:17.963 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:24:17.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.959+0000 7f11d17f8640 1 -- 172.21.15.105:0/1068997642 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f11ac0668d0 msgr2=0x7f11ac068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:17.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.959+0000 7f11d17f8640 1 --2- 172.21.15.105:0/1068997642 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f11ac0668d0 0x7f11ac068d90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f11cc06e6c0 tx=0x7f11bc007040 comp rx=0 tx=0).stop 2023-11-28T18:24:17.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.959+0000 7f11d17f8640 1 -- 172.21.15.105:0/1068997642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11b8003980 msgr2=0x7f11cc14b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:17.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.959+0000 7f11d17f8640 1 --2- 172.21.15.105:0/1068997642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11b8003980 0x7f11cc14b5f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f11b40b0040 tx=0x7f11b8005800 comp rx=0 tx=0).stop 2023-11-28T18:24:17.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.963+0000 7f11d17f8640 1 -- 172.21.15.105:0/1068997642 shutdown_connections 2023-11-28T18:24:17.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.963+0000 7f11d17f8640 1 --2- 172.21.15.105:0/1068997642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11cc159b10 0x7f11cc0cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.963+0000 7f11d17f8640 1 --2- 172.21.15.105:0/1068997642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11cc1583b0 0x7f11cc1640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.963+0000 7f11d17f8640 1 --2- 172.21.15.105:0/1068997642 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f11ac0668d0 0x7f11ac068d90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.963+0000 7f11d17f8640 1 --2- 172.21.15.105:0/1068997642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11b8003980 0x7f11cc14b5f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:17.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.963+0000 7f11d17f8640 1 -- 172.21.15.105:0/1068997642 >> 172.21.15.105:0/1068997642 conn(0x7f11cc0ceb90 msgr2=0x7f11cc06c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:17.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.963+0000 7f11d17f8640 1 -- 172.21.15.105:0/1068997642 shutdown_connections 2023-11-28T18:24:17.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:17.963+0000 7f11d17f8640 1 -- 172.21.15.105:0/1068997642 wait complete. 2023-11-28T18:24:17.988 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T18:24:18.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.127+0000 7f495d11d640 1 Processor -- start 2023-11-28T18:24:18.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.127+0000 7f495d11d640 1 -- start start 2023-11-28T18:24:18.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f495d11d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4958071960 0x7f495815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:18.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f495d11d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49580701d0 con 0x7f495815b030 2023-11-28T18:24:18.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f495d11d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f495806df80 con 0x7f49580b4290 2023-11-28T18:24:18.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f495d11d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4958071df0 con 0x7f4958071960 2023-11-28T18:24:18.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f495759e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f495815b030 0x7f49580709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55868/0 (socket says 172.21.15.105:55868) 2023-11-28T18:24:18.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f495759e640 1 -- 172.21.15.105:0/2776987840 learned_addr learned my addr 172.21.15.105:0/2776987840 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:18.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f4955d9b640 1 -- 172.21.15.105:0/2776987840 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2890158272 0 0) 0x7f49580701d0 con 0x7f495815b030 2023-11-28T18:24:18.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f4955d9b640 1 -- 172.21.15.105:0/2776987840 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4940003530 con 0x7f495815b030 2023-11-28T18:24:18.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f495659c640 1 --2- 172.21.15.105:0/2776987840 >> v2:172.21.15.148:3300/0 conn(0x7f4958071960 0x7f495815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:18.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f4955d9b640 1 -- 172.21.15.105:0/2776987840 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1610038115 0 0) 0x7f4940003530 con 0x7f495815b030 2023-11-28T18:24:18.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f4955d9b640 1 -- 172.21.15.105:0/2776987840 >> v2:172.21.15.148:3300/0 conn(0x7f4958071960 msgr2=0x7f495815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:18.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f4955d9b640 1 --2- 172.21.15.105:0/2776987840 >> v2:172.21.15.148:3300/0 conn(0x7f4958071960 0x7f495815f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f4955d9b640 1 -- 172.21.15.105:0/2776987840 >> v1:172.21.15.148:6789/0 conn(0x7f49580b4290 legacy=0x7f49580b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:18.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f4955d9b640 1 -- 172.21.15.105:0/2776987840 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49580720d0 con 0x7f495815b030 2023-11-28T18:24:18.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f4955d9b640 1 -- 172.21.15.105:0/2776987840 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f4950001a00 con 0x7f495815b030 2023-11-28T18:24:18.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f4955d9b640 1 -- 172.21.15.105:0/2776987840 >> v1:172.21.15.105:6789/0 conn(0x7f495815b030 legacy=0x7f49580709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:18.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f4955d9b640 1 --2- 172.21.15.105:0/2776987840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4940003980 0x7f4940003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:18.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f4955d9b640 1 --2- 172.21.15.105:0/2776987840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49400047d0 0x7f4940004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:18.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f4955d9b640 1 --2- 172.21.15.105:0/2776987840 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4940005190 0x7f494000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:18.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f4955d9b640 1 -- 172.21.15.105:0/2776987840 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4958071df0 con 0x7f4940003980 2023-11-28T18:24:18.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f4955d9b640 1 -- 172.21.15.105:0/2776987840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494000c0e0 con 0x7f49400047d0 2023-11-28T18:24:18.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f4955d9b640 1 -- 172.21.15.105:0/2776987840 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494000c2a0 con 0x7f4940005190 2023-11-28T18:24:18.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f4956d9d640 1 --2- 172.21.15.105:0/2776987840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4940003980 0x7f4940003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:18.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f495659c640 1 --2- 172.21.15.105:0/2776987840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49400047d0 0x7f4940004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:18.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f495759e640 1 --2- 172.21.15.105:0/2776987840 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4940005190 0x7f494000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:18.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f4956d9d640 1 -- 172.21.15.105:0/2776987840 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4940005190 msgr2=0x7f494000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:18.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f4956d9d640 1 --2- 172.21.15.105:0/2776987840 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4940005190 0x7f494000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f4956d9d640 1 -- 172.21.15.105:0/2776987840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49400047d0 msgr2=0x7f4940004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:18.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f4956d9d640 1 --2- 172.21.15.105:0/2776987840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49400047d0 0x7f4940004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f4956d9d640 1 -- 172.21.15.105:0/2776987840 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4940003530 con 0x7f4940003980 2023-11-28T18:24:18.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f495759e640 1 --2- 172.21.15.105:0/2776987840 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4940005190 0x7f494000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:18.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.131+0000 7f495759e640 1 -- 172.21.15.105:0/2776987840 reap_dead start 2023-11-28T18:24:18.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.135+0000 7f4956d9d640 1 --2- 172.21.15.105:0/2776987840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4940003980 0x7f4940003da0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4948001780 tx=0x7f494801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e690456679499f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:18.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.135+0000 7f4955d9b640 1 -- 172.21.15.105:0/2776987840 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49480052d0 con 0x7f4940003980 2023-11-28T18:24:18.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.135+0000 7f4955d9b640 1 -- 172.21.15.105:0/2776987840 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4948005450 con 0x7f4940003980 2023-11-28T18:24:18.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.135+0000 7f4955d9b640 1 -- 172.21.15.105:0/2776987840 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948003840 con 0x7f4940003980 2023-11-28T18:24:18.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.135+0000 7f495d11d640 1 -- 172.21.15.105:0/2776987840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4940003980 msgr2=0x7f4940003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:18.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.135+0000 7f495d11d640 1 --2- 172.21.15.105:0/2776987840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4940003980 0x7f4940003da0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4948001780 tx=0x7f494801c3b0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.135+0000 7f495d11d640 1 -- 172.21.15.105:0/2776987840 shutdown_connections 2023-11-28T18:24:18.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.135+0000 7f495d11d640 1 --2- 172.21.15.105:0/2776987840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4940003980 0x7f4940003da0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.135+0000 7f495d11d640 1 -- 172.21.15.105:0/2776987840 >> 172.21.15.105:0/2776987840 conn(0x7f49580ceb90 msgr2=0x7f49580cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:18.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.135+0000 7f495d11d640 1 -- 172.21.15.105:0/2776987840 shutdown_connections 2023-11-28T18:24:18.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.135+0000 7f495d11d640 1 -- 172.21.15.105:0/2776987840 wait complete. 2023-11-28T18:24:18.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f495d11d640 1 Processor -- start 2023-11-28T18:24:18.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f495d11d640 1 -- start start 2023-11-28T18:24:18.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f495d11d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4940003980 0x7f49581f9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:18.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f495d11d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f495814bdb0 0x7f49581f3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:18.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f495d11d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49581f5d70 0x7f49581597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:18.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f495d11d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49581f6ca0 con 0x7f495814bdb0 2023-11-28T18:24:18.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f495d11d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49581fa1d0 con 0x7f49581f5d70 2023-11-28T18:24:18.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f495d11d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49581f36d0 con 0x7f4940003980 2023-11-28T18:24:18.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f495659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f495814bdb0 0x7f49581f3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:18.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f495659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f495814bdb0 0x7f49581f3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56236/0 (socket says 172.21.15.105:56236) 2023-11-28T18:24:18.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f495659c640 1 -- 172.21.15.105:0/136749050 learned_addr learned my addr 172.21.15.105:0/136749050 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:18.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f4956d9d640 1 --2- 172.21.15.105:0/136749050 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4940003980 0x7f49581f9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:18.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f495759e640 1 --2- 172.21.15.105:0/136749050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49581f5d70 0x7f49581597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:18.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f495659c640 1 -- 172.21.15.105:0/136749050 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4940003980 msgr2=0x7f49581f9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:18.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f495659c640 1 --2- 172.21.15.105:0/136749050 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4940003980 0x7f49581f9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f495659c640 1 -- 172.21.15.105:0/136749050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49581f5d70 msgr2=0x7f49581597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:18.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f495659c640 1 --2- 172.21.15.105:0/136749050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49581f5d70 0x7f49581597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f495659c640 1 -- 172.21.15.105:0/136749050 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49581f66e0 con 0x7f495814bdb0 2023-11-28T18:24:18.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f495759e640 1 --2- 172.21.15.105:0/136749050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49581f5d70 0x7f49581597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:18.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f495659c640 1 --2- 172.21.15.105:0/136749050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f495814bdb0 0x7f49581f3190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f494c00b7f0 tx=0x7f494c00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:18.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f493f7fe640 1 -- 172.21.15.105:0/136749050 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c0041f0 con 0x7f495814bdb0 2023-11-28T18:24:18.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f493f7fe640 1 -- 172.21.15.105:0/136749050 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494c010a40 con 0x7f495814bdb0 2023-11-28T18:24:18.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f493f7fe640 1 -- 172.21.15.105:0/136749050 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c010dd0 con 0x7f495814bdb0 2023-11-28T18:24:18.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f495d11d640 1 -- 172.21.15.105:0/136749050 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49581f69c0 con 0x7f495814bdb0 2023-11-28T18:24:18.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.139+0000 7f495d11d640 1 -- 172.21.15.105:0/136749050 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49581f9770 con 0x7f495814bdb0 2023-11-28T18:24:18.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.143+0000 7f495d11d640 1 -- 172.21.15.105:0/136749050 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49581f9930 con 0x7f495814bdb0 2023-11-28T18:24:18.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.143+0000 7f493f7fe640 1 -- 172.21.15.105:0/136749050 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f494c014430 con 0x7f495814bdb0 2023-11-28T18:24:18.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.143+0000 7f493f7fe640 1 --2- 172.21.15.105:0/136749050 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f49440668d0 0x7f4944068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:18.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.143+0000 7f493f7fe640 1 -- 172.21.15.105:0/136749050 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f494c084c80 con 0x7f495814bdb0 2023-11-28T18:24:18.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.143+0000 7f4956d9d640 1 --2- 172.21.15.105:0/136749050 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f49440668d0 0x7f4944068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:18.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.143+0000 7f4956d9d640 1 --2- 172.21.15.105:0/136749050 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f49440668d0 0x7f4944068d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4948001750 tx=0x7f49480011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:18.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.147+0000 7f493f7fe640 1 -- 172.21.15.105:0/136749050 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f494c051c00 con 0x7f495814bdb0 2023-11-28T18:24:18.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.295+0000 7f495d11d640 1 -- 172.21.15.105:0/136749050 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4958067570 con 0x7f495814bdb0 2023-11-28T18:24:18.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.295+0000 7f493f7fe640 1 -- 172.21.15.105:0/136749050 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f494c0220a0 con 0x7f495814bdb0 2023-11-28T18:24:18.301 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:18.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.299+0000 7f495d11d640 1 -- 172.21.15.105:0/136749050 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f49440668d0 msgr2=0x7f4944068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:18.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.299+0000 7f495d11d640 1 --2- 172.21.15.105:0/136749050 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f49440668d0 0x7f4944068d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4948001750 tx=0x7f49480011a0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.299+0000 7f495d11d640 1 -- 172.21.15.105:0/136749050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f495814bdb0 msgr2=0x7f49581f3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:18.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.299+0000 7f495d11d640 1 --2- 172.21.15.105:0/136749050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f495814bdb0 0x7f49581f3190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f494c00b7f0 tx=0x7f494c00bcb0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.299+0000 7f495d11d640 1 -- 172.21.15.105:0/136749050 shutdown_connections 2023-11-28T18:24:18.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.299+0000 7f495d11d640 1 --2- 172.21.15.105:0/136749050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49581f5d70 0x7f49581597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.299+0000 7f495d11d640 1 --2- 172.21.15.105:0/136749050 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f49440668d0 0x7f4944068d90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.299+0000 7f495d11d640 1 --2- 172.21.15.105:0/136749050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f495814bdb0 0x7f49581f3190 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.299+0000 7f495d11d640 1 --2- 172.21.15.105:0/136749050 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4940003980 0x7f49581f9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.299+0000 7f495d11d640 1 -- 172.21.15.105:0/136749050 >> 172.21.15.105:0/136749050 conn(0x7f49580ceb90 msgr2=0x7f49580cadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:18.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.299+0000 7f495d11d640 1 -- 172.21.15.105:0/136749050 shutdown_connections 2023-11-28T18:24:18.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.299+0000 7f495d11d640 1 -- 172.21.15.105:0/136749050 wait complete. 2023-11-28T18:24:18.323 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:24:18.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.459+0000 7f21876d6640 1 Processor -- start 2023-11-28T18:24:18.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.459+0000 7f21876d6640 1 -- start start 2023-11-28T18:24:18.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.459+0000 7f21876d6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f218015d4f0 0x7f21800d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:18.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.459+0000 7f21876d6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f21800d3c30 0x7f21800d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:18.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.459+0000 7f21876d6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21800b25b0 con 0x7f21800d1890 2023-11-28T18:24:18.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.459+0000 7f21876d6640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21800b2900 con 0x7f21800d3c30 2023-11-28T18:24:18.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.459+0000 7f21876d6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21800d1cc0 con 0x7f218015d4f0 2023-11-28T18:24:18.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.459+0000 7f2184c71640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f21800d3c30 0x7f21800d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:18.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.459+0000 7f2184c71640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f21800d3c30 0x7f21800d1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56250/0 (socket says 172.21.15.105:56250) 2023-11-28T18:24:18.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.459+0000 7f2184c71640 1 -- 172.21.15.105:0/2572323084 learned_addr learned my addr 172.21.15.105:0/2572323084 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:18.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.459+0000 7f2185472640 1 --2- 172.21.15.105:0/2572323084 >> v2:172.21.15.148:3300/0 conn(0x7f218015d4f0 0x7f21800d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:18.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.459+0000 7f2184c71640 1 -- 172.21.15.105:0/2572323084 >> v2:172.21.15.148:3300/0 conn(0x7f218015d4f0 msgr2=0x7f21800d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:18.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.459+0000 7f2184c71640 1 --2- 172.21.15.105:0/2572323084 >> v2:172.21.15.148:3300/0 conn(0x7f218015d4f0 0x7f21800d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.459+0000 7f2184c71640 1 -- 172.21.15.105:0/2572323084 >> v1:172.21.15.148:6789/0 conn(0x7f21800d1890 legacy=0x7f21800cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:18.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.459+0000 7f2184c71640 1 -- 172.21.15.105:0/2572323084 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21800cea30 con 0x7f21800d3c30 2023-11-28T18:24:18.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.463+0000 7f2184c71640 1 --2- 172.21.15.105:0/2572323084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21800d3c30 0x7f21800d1350 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f218008bbf0 tx=0x7f217001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d0ca8653ffcee1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:18.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.463+0000 7f2177fff640 1 -- 172.21.15.105:0/2572323084 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21700ab640 con 0x7f21800d3c30 2023-11-28T18:24:18.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.463+0000 7f2177fff640 1 -- 172.21.15.105:0/2572323084 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21700a7070 con 0x7f21800d3c30 2023-11-28T18:24:18.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.463+0000 7f2177fff640 1 -- 172.21.15.105:0/2572323084 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21700ab9c0 con 0x7f21800d3c30 2023-11-28T18:24:18.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.463+0000 7f21876d6640 1 -- 172.21.15.105:0/2572323084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21800d3c30 msgr2=0x7f21800d1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:18.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.463+0000 7f21876d6640 1 --2- 172.21.15.105:0/2572323084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21800d3c30 0x7f21800d1350 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f218008bbf0 tx=0x7f217001c440 comp rx=0 tx=0).stop 2023-11-28T18:24:18.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.463+0000 7f21876d6640 1 -- 172.21.15.105:0/2572323084 shutdown_connections 2023-11-28T18:24:18.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.463+0000 7f21876d6640 1 --2- 172.21.15.105:0/2572323084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21800d3c30 0x7f21800d1350 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.463+0000 7f21876d6640 1 --2- 172.21.15.105:0/2572323084 >> v2:172.21.15.148:3300/0 conn(0x7f218015d4f0 0x7f21800d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.463+0000 7f21876d6640 1 -- 172.21.15.105:0/2572323084 >> 172.21.15.105:0/2572323084 conn(0x7f21800d3030 msgr2=0x7f21800d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:18.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.463+0000 7f21876d6640 1 -- 172.21.15.105:0/2572323084 shutdown_connections 2023-11-28T18:24:18.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.463+0000 7f21876d6640 1 -- 172.21.15.105:0/2572323084 wait complete. 2023-11-28T18:24:18.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f21876d6640 1 Processor -- start 2023-11-28T18:24:18.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f21876d6640 1 -- start start 2023-11-28T18:24:18.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f21876d6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21800d1890 0x7f21800c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:18.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f21876d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21800d3c30 0x7f21800b8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:18.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f21876d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f218015d4f0 0x7f21800b8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:18.479 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f21876d6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21800b7400 con 0x7f21800d1890 2023-11-28T18:24:18.479 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f21876d6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2180154810 con 0x7f218015d4f0 2023-11-28T18:24:18.479 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f21876d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21801f9a80 con 0x7f21800d3c30 2023-11-28T18:24:18.479 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f2185472640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21800d1890 0x7f21800c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:18.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f2185472640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21800d1890 0x7f21800c8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56256/0 (socket says 172.21.15.105:56256) 2023-11-28T18:24:18.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f2185472640 1 -- 172.21.15.105:0/3501899315 learned_addr learned my addr 172.21.15.105:0/3501899315 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:18.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f2184c71640 1 --2- 172.21.15.105:0/3501899315 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21800d3c30 0x7f21800b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:18.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f2185c73640 1 --2- 172.21.15.105:0/3501899315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f218015d4f0 0x7f21800b8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:18.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f2185472640 1 -- 172.21.15.105:0/3501899315 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21800d3c30 msgr2=0x7f21800b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:18.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f2185472640 1 --2- 172.21.15.105:0/3501899315 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21800d3c30 0x7f21800b8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f2185472640 1 -- 172.21.15.105:0/3501899315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f218015d4f0 msgr2=0x7f21800b8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:18.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f2185472640 1 --2- 172.21.15.105:0/3501899315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f218015d4f0 0x7f21800b8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f2185472640 1 -- 172.21.15.105:0/3501899315 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21700094d0 con 0x7f21800d1890 2023-11-28T18:24:18.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f2185c73640 1 --2- 172.21.15.105:0/3501899315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f218015d4f0 0x7f21800b8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:18.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f2185472640 1 --2- 172.21.15.105:0/3501899315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21800d1890 0x7f21800c8750 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f217c00e870 tx=0x7f217c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:18.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f2175ffb640 1 -- 172.21.15.105:0/3501899315 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c009510 con 0x7f21800d1890 2023-11-28T18:24:18.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f2175ffb640 1 -- 172.21.15.105:0/3501899315 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217c00eec0 con 0x7f21800d1890 2023-11-28T18:24:18.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f21876d6640 1 -- 172.21.15.105:0/3501899315 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2180155ac0 con 0x7f21800d1890 2023-11-28T18:24:18.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f2175ffb640 1 -- 172.21.15.105:0/3501899315 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c00cb40 con 0x7f21800d1890 2023-11-28T18:24:18.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.467+0000 7f21876d6640 1 -- 172.21.15.105:0/3501899315 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2180154d00 con 0x7f21800d1890 2023-11-28T18:24:18.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.471+0000 7f21876d6640 1 -- 172.21.15.105:0/3501899315 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2180067570 con 0x7f21800d1890 2023-11-28T18:24:18.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.471+0000 7f2175ffb640 1 -- 172.21.15.105:0/3501899315 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f217c00cca0 con 0x7f21800d1890 2023-11-28T18:24:18.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.471+0000 7f2175ffb640 1 --2- 172.21.15.105:0/3501899315 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f21640668d0 0x7f2164068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:18.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.471+0000 7f2175ffb640 1 -- 172.21.15.105:0/3501899315 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f217c0848c0 con 0x7f21800d1890 2023-11-28T18:24:18.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.471+0000 7f2184c71640 1 --2- 172.21.15.105:0/3501899315 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f21640668d0 0x7f2164068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:18.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.471+0000 7f2184c71640 1 --2- 172.21.15.105:0/3501899315 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f21640668d0 0x7f2164068d90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f21700af000 tx=0x7f21700b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:18.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.475+0000 7f2175ffb640 1 -- 172.21.15.105:0/3501899315 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f217c0517b0 con 0x7f21800d1890 2023-11-28T18:24:18.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.623+0000 7f21876d6640 1 -- 172.21.15.105:0/3501899315 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2180156220 con 0x7f21800d1890 2023-11-28T18:24:18.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.627+0000 7f2175ffb640 1 -- 172.21.15.105:0/3501899315 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18887 (secure 0 0 0) 0x7f217c054dd0 con 0x7f21800d1890 2023-11-28T18:24:18.630 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:18.630 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:24:15.048168+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:24:18.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.627+0000 7f21876d6640 1 -- 172.21.15.105:0/3501899315 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f21640668d0 msgr2=0x7f2164068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:18.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.627+0000 7f21876d6640 1 --2- 172.21.15.105:0/3501899315 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f21640668d0 0x7f2164068d90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f21700af000 tx=0x7f21700b6040 comp rx=0 tx=0).stop 2023-11-28T18:24:18.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.627+0000 7f21876d6640 1 -- 172.21.15.105:0/3501899315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21800d1890 msgr2=0x7f21800c8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:18.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.627+0000 7f21876d6640 1 --2- 172.21.15.105:0/3501899315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21800d1890 0x7f21800c8750 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f217c00e870 tx=0x7f217c00ec70 comp rx=0 tx=0).stop 2023-11-28T18:24:18.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.627+0000 7f21876d6640 1 -- 172.21.15.105:0/3501899315 shutdown_connections 2023-11-28T18:24:18.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.627+0000 7f21876d6640 1 --2- 172.21.15.105:0/3501899315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f218015d4f0 0x7f21800b8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.627+0000 7f21876d6640 1 --2- 172.21.15.105:0/3501899315 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21800d3c30 0x7f21800b8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.627+0000 7f21876d6640 1 --2- 172.21.15.105:0/3501899315 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f21640668d0 0x7f2164068d90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.627+0000 7f21876d6640 1 --2- 172.21.15.105:0/3501899315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21800d1890 0x7f21800c8750 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.627+0000 7f21876d6640 1 -- 172.21.15.105:0/3501899315 >> 172.21.15.105:0/3501899315 conn(0x7f21800d3030 msgr2=0x7f2180159080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:18.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.627+0000 7f21876d6640 1 -- 172.21.15.105:0/3501899315 shutdown_connections 2023-11-28T18:24:18.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.627+0000 7f21876d6640 1 -- 172.21.15.105:0/3501899315 wait complete. 2023-11-28T18:24:18.651 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:23:31.485422+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:23:32.274378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T18:23:36.100071+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T18:24:18.651 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:24:18.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.787+0000 7f796f479640 1 Processor -- start 2023-11-28T18:24:18.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.787+0000 7f796f479640 1 -- start start 2023-11-28T18:24:18.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.791+0000 7f796f479640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f79680caf60 0x7f79680cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:18.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.791+0000 7f796f479640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79680ad850 0x7f79680701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:18.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.791+0000 7f796f479640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f796806bf50 con 0x7f7968159f40 2023-11-28T18:24:18.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.791+0000 7f796f479640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79680cb3c0 con 0x7f79680caf60 2023-11-28T18:24:18.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.791+0000 7f796f479640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796815a320 con 0x7f79680ad850 2023-11-28T18:24:18.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.791+0000 7f796ca14640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f79680caf60 0x7f79680cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:18.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.791+0000 7f796ca14640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f79680caf60 0x7f79680cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56260/0 (socket says 172.21.15.105:56260) 2023-11-28T18:24:18.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.791+0000 7f796ca14640 1 -- 172.21.15.105:0/790454369 learned_addr learned my addr 172.21.15.105:0/790454369 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:18.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.791+0000 7f796da16640 1 --2- 172.21.15.105:0/790454369 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79680ad850 0x7f79680701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:18.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.791+0000 7f795ffff640 1 -- 172.21.15.105:0/790454369 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2161693528 0 0) 0x7f796806bf50 con 0x7f7968159f40 2023-11-28T18:24:18.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.791+0000 7f796ca14640 1 -- 172.21.15.105:0/790454369 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79680ad850 msgr2=0x7f79680701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:18.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.791+0000 7f796ca14640 1 --2- 172.21.15.105:0/790454369 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79680ad850 0x7f79680701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.791+0000 7f796ca14640 1 -- 172.21.15.105:0/790454369 >> v1:172.21.15.148:6789/0 conn(0x7f7968159f40 legacy=0x7f79680ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:18.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.791+0000 7f796ca14640 1 -- 172.21.15.105:0/790454369 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7968071380 con 0x7f79680caf60 2023-11-28T18:24:18.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.791+0000 7f796da16640 1 --2- 172.21.15.105:0/790454369 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79680ad850 0x7f79680701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:24:18.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.791+0000 7f796ca14640 1 --2- 172.21.15.105:0/790454369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79680caf60 0x7f79680cc760 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7958009870 tx=0x7f795801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691a41c3d13fc110 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:18.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f795ffff640 1 -- 172.21.15.105:0/790454369 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79580ab680 con 0x7f79680caf60 2023-11-28T18:24:18.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f795ffff640 1 -- 172.21.15.105:0/790454369 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79580ab7e0 con 0x7f79680caf60 2023-11-28T18:24:18.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f795ffff640 1 -- 172.21.15.105:0/790454369 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79580abb70 con 0x7f79680caf60 2023-11-28T18:24:18.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f796f479640 1 -- 172.21.15.105:0/790454369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79680caf60 msgr2=0x7f79680cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:18.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f796f479640 1 --2- 172.21.15.105:0/790454369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79680caf60 0x7f79680cc760 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7958009870 tx=0x7f795801c480 comp rx=0 tx=0).stop 2023-11-28T18:24:18.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f796f479640 1 -- 172.21.15.105:0/790454369 shutdown_connections 2023-11-28T18:24:18.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f796f479640 1 --2- 172.21.15.105:0/790454369 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79680ad850 0x7f79680701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f796f479640 1 --2- 172.21.15.105:0/790454369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79680caf60 0x7f79680cc760 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f796f479640 1 -- 172.21.15.105:0/790454369 >> 172.21.15.105:0/790454369 conn(0x7f79680ceb90 msgr2=0x7f79680cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:18.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f796f479640 1 -- 172.21.15.105:0/790454369 shutdown_connections 2023-11-28T18:24:18.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f796f479640 1 -- 172.21.15.105:0/790454369 wait complete. 2023-11-28T18:24:18.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f796f479640 1 Processor -- start 2023-11-28T18:24:18.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f796f479640 1 -- start start 2023-11-28T18:24:18.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f796f479640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79680ad850 0x7f796813e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:18.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f796f479640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79680caf60 0x7f79681f42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:18.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f796f479640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7968159f40 0x7f796815ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:18.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f796f479640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79681f5290 con 0x7f7968159f40 2023-11-28T18:24:18.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f796f479640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796813eb20 con 0x7f79680caf60 2023-11-28T18:24:18.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f796f479640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79681f4cd0 con 0x7f79680ad850 2023-11-28T18:24:18.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f796da16640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7968159f40 0x7f796815ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:18.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f796da16640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7968159f40 0x7f796815ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56264/0 (socket says 172.21.15.105:56264) 2023-11-28T18:24:18.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f796da16640 1 -- 172.21.15.105:0/1411983478 learned_addr learned my addr 172.21.15.105:0/1411983478 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:18.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f796d215640 1 --2- 172.21.15.105:0/1411983478 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79680ad850 0x7f796813e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:18.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.795+0000 7f796ca14640 1 --2- 172.21.15.105:0/1411983478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79680caf60 0x7f79681f42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:18.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.799+0000 7f796da16640 1 -- 172.21.15.105:0/1411983478 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79680ad850 msgr2=0x7f796813e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:18.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.799+0000 7f796da16640 1 --2- 172.21.15.105:0/1411983478 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79680ad850 0x7f796813e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.799+0000 7f796da16640 1 -- 172.21.15.105:0/1411983478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79680caf60 msgr2=0x7f79681f42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:18.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.799+0000 7f796da16640 1 --2- 172.21.15.105:0/1411983478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79680caf60 0x7f79681f42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.799+0000 7f796da16640 1 -- 172.21.15.105:0/1411983478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7960009a70 con 0x7f7968159f40 2023-11-28T18:24:18.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.799+0000 7f796da16640 1 --2- 172.21.15.105:0/1411983478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7968159f40 0x7f796815ada0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f796000efc0 tx=0x7f796000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:18.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.799+0000 7f795dffb640 1 -- 172.21.15.105:0/1411983478 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960009cd0 con 0x7f7968159f40 2023-11-28T18:24:18.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.799+0000 7f795dffb640 1 -- 172.21.15.105:0/1411983478 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796000f040 con 0x7f7968159f40 2023-11-28T18:24:18.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.799+0000 7f795dffb640 1 -- 172.21.15.105:0/1411983478 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79600039a0 con 0x7f7968159f40 2023-11-28T18:24:18.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.799+0000 7f796f479640 1 -- 172.21.15.105:0/1411983478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79580094d0 con 0x7f7968159f40 2023-11-28T18:24:18.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.799+0000 7f796f479640 1 -- 172.21.15.105:0/1411983478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79681540f0 con 0x7f7968159f40 2023-11-28T18:24:18.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.799+0000 7f796f479640 1 -- 172.21.15.105:0/1411983478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7968067570 con 0x7f7968159f40 2023-11-28T18:24:18.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.799+0000 7f795dffb640 1 -- 172.21.15.105:0/1411983478 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f7960014480 con 0x7f7968159f40 2023-11-28T18:24:18.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.799+0000 7f795dffb640 1 --2- 172.21.15.105:0/1411983478 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f794c0668d0 0x7f794c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:18.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.799+0000 7f795dffb640 1 -- 172.21.15.105:0/1411983478 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f79600852b0 con 0x7f7968159f40 2023-11-28T18:24:18.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.799+0000 7f796d215640 1 --2- 172.21.15.105:0/1411983478 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f794c0668d0 0x7f794c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:18.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.803+0000 7f796d215640 1 --2- 172.21.15.105:0/1411983478 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f794c0668d0 0x7f794c068d90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7964000f70 tx=0x7f7964009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:18.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.803+0000 7f795dffb640 1 -- 172.21.15.105:0/1411983478 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7960052230 con 0x7f7968159f40 2023-11-28T18:24:18.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.955+0000 7f796f479640 1 -- 172.21.15.105:0/1411983478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7968153d80 con 0x7f7968159f40 2023-11-28T18:24:18.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.955+0000 7f795dffb640 1 -- 172.21.15.105:0/1411983478 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7960055850 con 0x7f7968159f40 2023-11-28T18:24:18.959 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:24:18.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.955+0000 7f796f479640 1 -- 172.21.15.105:0/1411983478 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f794c0668d0 msgr2=0x7f794c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:18.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.955+0000 7f796f479640 1 --2- 172.21.15.105:0/1411983478 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f794c0668d0 0x7f794c068d90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7964000f70 tx=0x7f7964009040 comp rx=0 tx=0).stop 2023-11-28T18:24:18.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.955+0000 7f796f479640 1 -- 172.21.15.105:0/1411983478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7968159f40 msgr2=0x7f796815ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:18.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.955+0000 7f796f479640 1 --2- 172.21.15.105:0/1411983478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7968159f40 0x7f796815ada0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f796000efc0 tx=0x7f796000c490 comp rx=0 tx=0).stop 2023-11-28T18:24:18.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.955+0000 7f796f479640 1 -- 172.21.15.105:0/1411983478 shutdown_connections 2023-11-28T18:24:18.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.955+0000 7f796f479640 1 --2- 172.21.15.105:0/1411983478 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f794c0668d0 0x7f794c068d90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.955+0000 7f796f479640 1 --2- 172.21.15.105:0/1411983478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7968159f40 0x7f796815ada0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.955+0000 7f796f479640 1 --2- 172.21.15.105:0/1411983478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79680caf60 0x7f79681f42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.955+0000 7f796f479640 1 --2- 172.21.15.105:0/1411983478 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79680ad850 0x7f796813e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:18.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.955+0000 7f796f479640 1 -- 172.21.15.105:0/1411983478 >> 172.21.15.105:0/1411983478 conn(0x7f79680ceb90 msgr2=0x7f796814eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:18.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.955+0000 7f796f479640 1 -- 172.21.15.105:0/1411983478 shutdown_connections 2023-11-28T18:24:18.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:18.955+0000 7f796f479640 1 -- 172.21.15.105:0/1411983478 wait complete. 2023-11-28T18:24:18.979 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:24:19.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.115+0000 7ff1c7c32640 1 Processor -- start 2023-11-28T18:24:19.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.115+0000 7ff1c7c32640 1 -- start start 2023-11-28T18:24:19.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.119+0000 7ff1c7c32640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff1c00cc760 0x7ff1c0164d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:19.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.119+0000 7ff1c7c32640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c00ccc10 con 0x7ff1c00caf60 2023-11-28T18:24:19.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.119+0000 7ff1c7c32640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c006bf50 con 0x7ff1c0159f40 2023-11-28T18:24:19.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.119+0000 7ff1c7c32640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1c00cb3c0 con 0x7ff1c00cc760 2023-11-28T18:24:19.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.119+0000 7ff1c51cd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff1c00caf60 0x7ff1c00ad850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55878/0 (socket says 172.21.15.105:55878) 2023-11-28T18:24:19.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.119+0000 7ff1c51cd640 1 -- 172.21.15.105:0/1863494295 learned_addr learned my addr 172.21.15.105:0/1863494295 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:19.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.119+0000 7ff1c61cf640 1 --2- 172.21.15.105:0/1863494295 >> v2:172.21.15.105:3300/0 conn(0x7ff1c00cc760 0x7ff1c0164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:19.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.119+0000 7ff1c61cf640 1 -- 172.21.15.105:0/1863494295 >> v1:172.21.15.148:6789/0 conn(0x7ff1c0159f40 legacy=0x7ff1c00ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:19.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.119+0000 7ff1c61cf640 1 -- 172.21.15.105:0/1863494295 >> v1:172.21.15.105:6789/0 conn(0x7ff1c00caf60 legacy=0x7ff1c00ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:19.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.119+0000 7ff1c61cf640 1 -- 172.21.15.105:0/1863494295 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c015a320 con 0x7ff1c00cc760 2023-11-28T18:24:19.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.123+0000 7ff1c61cf640 1 --2- 172.21.15.105:0/1863494295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c00cc760 0x7ff1c0164d90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009d90 tx=0x7ff1b801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e8cddd56a4e730 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:19.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.123+0000 7ff1c49cc640 1 -- 172.21.15.105:0/1863494295 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b80a5910 con 0x7ff1c00cc760 2023-11-28T18:24:19.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.123+0000 7ff1c49cc640 1 -- 172.21.15.105:0/1863494295 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b80a5a70 con 0x7ff1c00cc760 2023-11-28T18:24:19.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.123+0000 7ff1c49cc640 1 -- 172.21.15.105:0/1863494295 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b80a5e00 con 0x7ff1c00cc760 2023-11-28T18:24:19.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.123+0000 7ff1c7c32640 1 -- 172.21.15.105:0/1863494295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c00cc760 msgr2=0x7ff1c0164d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:19.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.123+0000 7ff1c7c32640 1 --2- 172.21.15.105:0/1863494295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c00cc760 0x7ff1c0164d90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009d90 tx=0x7ff1b801c850 comp rx=0 tx=0).stop 2023-11-28T18:24:19.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.123+0000 7ff1c7c32640 1 -- 172.21.15.105:0/1863494295 shutdown_connections 2023-11-28T18:24:19.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.123+0000 7ff1c7c32640 1 --2- 172.21.15.105:0/1863494295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c00cc760 0x7ff1c0164d90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.123+0000 7ff1c7c32640 1 -- 172.21.15.105:0/1863494295 >> 172.21.15.105:0/1863494295 conn(0x7ff1c00ceb90 msgr2=0x7ff1c00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:19.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.123+0000 7ff1c7c32640 1 -- 172.21.15.105:0/1863494295 shutdown_connections 2023-11-28T18:24:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.123+0000 7ff1c7c32640 1 -- 172.21.15.105:0/1863494295 wait complete. 2023-11-28T18:24:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.123+0000 7ff1c7c32640 1 Processor -- start 2023-11-28T18:24:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.123+0000 7ff1c7c32640 1 -- start start 2023-11-28T18:24:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.127+0000 7ff1c7c32640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1c00caf60 0x7ff1c014afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.127+0000 7ff1c7c32640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c00cc760 0x7ff1c0143250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.127+0000 7ff1c7c32640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1c0159f40 0x7ff1c0143790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.127+0000 7ff1c7c32640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c0157ff0 con 0x7ff1c00cc760 2023-11-28T18:24:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.127+0000 7ff1c7c32640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c0158260 con 0x7ff1c00caf60 2023-11-28T18:24:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.127+0000 7ff1c7c32640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c0158540 con 0x7ff1c0159f40 2023-11-28T18:24:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.127+0000 7ff1c51cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c00cc760 0x7ff1c0143250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.127+0000 7ff1c51cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c00cc760 0x7ff1c0143250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56284/0 (socket says 172.21.15.105:56284) 2023-11-28T18:24:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.127+0000 7ff1c51cd640 1 -- 172.21.15.105:0/3588167412 learned_addr learned my addr 172.21.15.105:0/3588167412 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.127+0000 7ff1c51cd640 1 -- 172.21.15.105:0/3588167412 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1c0159f40 msgr2=0x7ff1c0143790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:19.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.127+0000 7ff1c51cd640 1 --2- 172.21.15.105:0/3588167412 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1c0159f40 0x7ff1c0143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.127+0000 7ff1c51cd640 1 -- 172.21.15.105:0/3588167412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1c00caf60 msgr2=0x7ff1c014afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:19.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.127+0000 7ff1c51cd640 1 --2- 172.21.15.105:0/3588167412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1c00caf60 0x7ff1c014afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.127+0000 7ff1c51cd640 1 -- 172.21.15.105:0/3588167412 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b8009a70 con 0x7ff1c00cc760 2023-11-28T18:24:19.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.127+0000 7ff1c51cd640 1 --2- 172.21.15.105:0/3588167412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c00cc760 0x7ff1c0143250 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000c400 tx=0x7ff1b000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:19.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.127+0000 7ff1b67fc640 1 -- 172.21.15.105:0/3588167412 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0005cf0 con 0x7ff1c00cc760 2023-11-28T18:24:19.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.127+0000 7ff1b67fc640 1 -- 172.21.15.105:0/3588167412 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b0005e50 con 0x7ff1c00cc760 2023-11-28T18:24:19.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.127+0000 7ff1b67fc640 1 -- 172.21.15.105:0/3588167412 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b00072d0 con 0x7ff1c00cc760 2023-11-28T18:24:19.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.127+0000 7ff1c7c32640 1 -- 172.21.15.105:0/3588167412 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c0158820 con 0x7ff1c00cc760 2023-11-28T18:24:19.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.127+0000 7ff1c7c32640 1 -- 172.21.15.105:0/3588167412 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c014b500 con 0x7ff1c00cc760 2023-11-28T18:24:19.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.127+0000 7ff1c7c32640 1 -- 172.21.15.105:0/3588167412 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c014b690 con 0x7ff1c00cc760 2023-11-28T18:24:19.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.127+0000 7ff1b67fc640 1 -- 172.21.15.105:0/3588167412 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff1b0007430 con 0x7ff1c00cc760 2023-11-28T18:24:19.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.131+0000 7ff1b67fc640 1 --2- 172.21.15.105:0/3588167412 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff1a40668d0 0x7ff1a4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:19.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.131+0000 7ff1b67fc640 1 -- 172.21.15.105:0/3588167412 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7ff1b0083da0 con 0x7ff1c00cc760 2023-11-28T18:24:19.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.131+0000 7ff1c59ce640 1 --2- 172.21.15.105:0/3588167412 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff1a40668d0 0x7ff1a4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:19.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.131+0000 7ff1c59ce640 1 --2- 172.21.15.105:0/3588167412 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff1a40668d0 0x7ff1a4068d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc003cf0 tx=0x7ff1bc0079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:19.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.135+0000 7ff1b67fc640 1 -- 172.21.15.105:0/3588167412 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff1b0050bf0 con 0x7ff1c00cc760 2023-11-28T18:24:19.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.283+0000 7ff1c7c32640 1 -- 172.21.15.105:0/3588167412 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1c0067570 con 0x7ff1c00cc760 2023-11-28T18:24:19.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.283+0000 7ff1b67fc640 1 -- 172.21.15.105:0/3588167412 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff1b0017020 con 0x7ff1c00cc760 2023-11-28T18:24:19.287 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:19.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.283+0000 7ff1c7c32640 1 -- 172.21.15.105:0/3588167412 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff1a40668d0 msgr2=0x7ff1a4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:19.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.283+0000 7ff1c7c32640 1 --2- 172.21.15.105:0/3588167412 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff1a40668d0 0x7ff1a4068d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc003cf0 tx=0x7ff1bc0079c0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.283+0000 7ff1c7c32640 1 -- 172.21.15.105:0/3588167412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c00cc760 msgr2=0x7ff1c0143250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:19.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.283+0000 7ff1c7c32640 1 --2- 172.21.15.105:0/3588167412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c00cc760 0x7ff1c0143250 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000c400 tx=0x7ff1b000c8c0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.283+0000 7ff1c7c32640 1 -- 172.21.15.105:0/3588167412 shutdown_connections 2023-11-28T18:24:19.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.283+0000 7ff1c7c32640 1 --2- 172.21.15.105:0/3588167412 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1c0159f40 0x7ff1c0143790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.283+0000 7ff1c7c32640 1 --2- 172.21.15.105:0/3588167412 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff1a40668d0 0x7ff1a4068d90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.283+0000 7ff1c7c32640 1 --2- 172.21.15.105:0/3588167412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c00cc760 0x7ff1c0143250 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.283+0000 7ff1c7c32640 1 --2- 172.21.15.105:0/3588167412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1c00caf60 0x7ff1c014afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.283+0000 7ff1c7c32640 1 -- 172.21.15.105:0/3588167412 >> 172.21.15.105:0/3588167412 conn(0x7ff1c00ceb90 msgr2=0x7ff1c00ad820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:19.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.283+0000 7ff1c7c32640 1 -- 172.21.15.105:0/3588167412 shutdown_connections 2023-11-28T18:24:19.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.283+0000 7ff1c7c32640 1 -- 172.21.15.105:0/3588167412 wait complete. 2023-11-28T18:24:19.308 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:24:19.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.442+0000 7f8ac48c9640 1 Processor -- start 2023-11-28T18:24:19.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.442+0000 7f8ac48c9640 1 -- start start 2023-11-28T18:24:19.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.446+0000 7f8ac48c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ac00b4290 0x7f8ac00b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:19.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.446+0000 7f8ac48c9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8ac0071960 0x7f8ac015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:19.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.446+0000 7f8ac48c9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8ac015b030 0x7f8ac00709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:19.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.446+0000 7f8ac48c9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac00b4bb0 con 0x7f8ac0071960 2023-11-28T18:24:19.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.446+0000 7f8ac48c9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac00701d0 con 0x7f8ac015b030 2023-11-28T18:24:19.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.446+0000 7f8ac48c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac006df80 con 0x7f8ac00b4290 2023-11-28T18:24:19.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.446+0000 7f8ab5d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8ac0071960 0x7f8ac015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:19.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.446+0000 7f8ab5d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8ac0071960 0x7f8ac015f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56298/0 (socket says 172.21.15.105:56298) 2023-11-28T18:24:19.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.446+0000 7f8ab5d9b640 1 -- 172.21.15.105:0/1917210902 learned_addr learned my addr 172.21.15.105:0/1917210902 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:19.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.446+0000 7f8ab5d9b640 1 -- 172.21.15.105:0/1917210902 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ac00b4290 msgr2=0x7f8ac00b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:19.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.446+0000 7f8abed9d640 1 --2- 172.21.15.105:0/1917210902 >> v2:172.21.15.148:3300/0 conn(0x7f8ac015b030 0x7f8ac00709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:19.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.446+0000 7f8ab5d9b640 1 --2- 172.21.15.105:0/1917210902 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ac00b4290 0x7f8ac00b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.446+0000 7f8ab5d9b640 1 -- 172.21.15.105:0/1917210902 >> v2:172.21.15.148:3300/0 conn(0x7f8ac015b030 msgr2=0x7f8ac00709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:19.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.446+0000 7f8ab5d9b640 1 --2- 172.21.15.105:0/1917210902 >> v2:172.21.15.148:3300/0 conn(0x7f8ac015b030 0x7f8ac00709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.446+0000 7f8ab5d9b640 1 -- 172.21.15.105:0/1917210902 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac0071df0 con 0x7f8ac0071960 2023-11-28T18:24:19.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.446+0000 7f8abed9d640 1 --2- 172.21.15.105:0/1917210902 >> v2:172.21.15.148:3300/0 conn(0x7f8ac015b030 0x7f8ac00709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:19.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.450+0000 7f8ab5d9b640 1 --2- 172.21.15.105:0/1917210902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ac0071960 0x7f8ac015f3b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8ac006bc50 tx=0x7f8aa801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc295d74cef5638 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:19.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.450+0000 7f8abdd9b640 1 -- 172.21.15.105:0/1917210902 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa80ac600 con 0x7f8ac0071960 2023-11-28T18:24:19.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.450+0000 7f8abdd9b640 1 -- 172.21.15.105:0/1917210902 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa80a8050 con 0x7f8ac0071960 2023-11-28T18:24:19.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.450+0000 7f8abdd9b640 1 -- 172.21.15.105:0/1917210902 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa80ac980 con 0x7f8ac0071960 2023-11-28T18:24:19.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.450+0000 7f8ac48c9640 1 -- 172.21.15.105:0/1917210902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ac0071960 msgr2=0x7f8ac015f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:19.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.450+0000 7f8ac48c9640 1 --2- 172.21.15.105:0/1917210902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ac0071960 0x7f8ac015f3b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8ac006bc50 tx=0x7f8aa801c4d0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.450+0000 7f8ac48c9640 1 -- 172.21.15.105:0/1917210902 shutdown_connections 2023-11-28T18:24:19.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.450+0000 7f8ac48c9640 1 --2- 172.21.15.105:0/1917210902 >> v2:172.21.15.148:3300/0 conn(0x7f8ac015b030 0x7f8ac00709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.450+0000 7f8ac48c9640 1 --2- 172.21.15.105:0/1917210902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ac0071960 0x7f8ac015f3b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.450+0000 7f8ac48c9640 1 --2- 172.21.15.105:0/1917210902 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ac00b4290 0x7f8ac00b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.450+0000 7f8ac48c9640 1 -- 172.21.15.105:0/1917210902 >> 172.21.15.105:0/1917210902 conn(0x7f8ac00ceb90 msgr2=0x7f8ac00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:19.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.450+0000 7f8ac48c9640 1 -- 172.21.15.105:0/1917210902 shutdown_connections 2023-11-28T18:24:19.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.450+0000 7f8ac48c9640 1 -- 172.21.15.105:0/1917210902 wait complete. 2023-11-28T18:24:19.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.450+0000 7f8ac48c9640 1 Processor -- start 2023-11-28T18:24:19.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.450+0000 7f8ac48c9640 1 -- start start 2023-11-28T18:24:19.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8ac48c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ac00b4290 0x7f8ac00b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:19.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8ac48c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ac015b030 0x7f8ac0150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:19.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8ac48c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ac014ddf0 0x7f8ac0099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:19.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8ac48c9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac014e510 con 0x7f8ac00b4290 2023-11-28T18:24:19.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8ac48c9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac014e2a0 con 0x7f8ac014ddf0 2023-11-28T18:24:19.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8ac48c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac009a280 con 0x7f8ac015b030 2023-11-28T18:24:19.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8abe59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ac00b4290 0x7f8ac00b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:19.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8abe59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ac00b4290 0x7f8ac00b3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56300/0 (socket says 172.21.15.105:56300) 2023-11-28T18:24:19.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8abe59c640 1 -- 172.21.15.105:0/2120555159 learned_addr learned my addr 172.21.15.105:0/2120555159 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:19.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8abed9d640 1 --2- 172.21.15.105:0/2120555159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ac014ddf0 0x7f8ac0099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:19.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8ab5d9b640 1 --2- 172.21.15.105:0/2120555159 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ac015b030 0x7f8ac0150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:19.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8abe59c640 1 -- 172.21.15.105:0/2120555159 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ac015b030 msgr2=0x7f8ac0150910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:19.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8abe59c640 1 --2- 172.21.15.105:0/2120555159 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ac015b030 0x7f8ac0150910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8abe59c640 1 -- 172.21.15.105:0/2120555159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ac014ddf0 msgr2=0x7f8ac0099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:19.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8abe59c640 1 --2- 172.21.15.105:0/2120555159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ac014ddf0 0x7f8ac0099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8abe59c640 1 -- 172.21.15.105:0/2120555159 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa80094d0 con 0x7f8ac00b4290 2023-11-28T18:24:19.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8abe59c640 1 --2- 172.21.15.105:0/2120555159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ac00b4290 0x7f8ac00b3460 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000ba50 tx=0x7f8ab000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:19.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8ab77fe640 1 -- 172.21.15.105:0/2120555159 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab000cbe0 con 0x7f8ac00b4290 2023-11-28T18:24:19.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8ab77fe640 1 -- 172.21.15.105:0/2120555159 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab000cd60 con 0x7f8ac00b4290 2023-11-28T18:24:19.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8ac48c9640 1 -- 172.21.15.105:0/2120555159 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac009a560 con 0x7f8ac00b4290 2023-11-28T18:24:19.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8ab77fe640 1 -- 172.21.15.105:0/2120555159 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0014680 con 0x7f8ac00b4290 2023-11-28T18:24:19.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8ac48c9640 1 -- 172.21.15.105:0/2120555159 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac009ab20 con 0x7f8ac00b4290 2023-11-28T18:24:19.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8ac48c9640 1 -- 172.21.15.105:0/2120555159 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac0067570 con 0x7f8ac00b4290 2023-11-28T18:24:19.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.454+0000 7f8ab77fe640 1 -- 172.21.15.105:0/2120555159 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8ab001d3b0 con 0x7f8ac00b4290 2023-11-28T18:24:19.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.458+0000 7f8ab77fe640 1 --2- 172.21.15.105:0/2120555159 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8aa00668d0 0x7f8aa0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:19.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.458+0000 7f8ab77fe640 1 -- 172.21.15.105:0/2120555159 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f8ab00847e0 con 0x7f8ac00b4290 2023-11-28T18:24:19.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.458+0000 7f8ab5d9b640 1 --2- 172.21.15.105:0/2120555159 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8aa00668d0 0x7f8aa0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:19.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.458+0000 7f8ab5d9b640 1 --2- 172.21.15.105:0/2120555159 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8aa00668d0 0x7f8aa0068d90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80ae000 tx=0x7f8aa80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:19.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.462+0000 7f8ab77fe640 1 -- 172.21.15.105:0/2120555159 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8ab0051630 con 0x7f8ac00b4290 2023-11-28T18:24:19.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.610+0000 7f8ac48c9640 1 -- 172.21.15.105:0/2120555159 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8ac00cf7f0 con 0x7f8ac00b4290 2023-11-28T18:24:19.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.610+0000 7f8ab77fe640 1 -- 172.21.15.105:0/2120555159 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8ab0019020 con 0x7f8ac00b4290 2023-11-28T18:24:19.614 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:24:19.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.610+0000 7f8ac48c9640 1 -- 172.21.15.105:0/2120555159 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8aa00668d0 msgr2=0x7f8aa0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:19.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.610+0000 7f8ac48c9640 1 --2- 172.21.15.105:0/2120555159 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8aa00668d0 0x7f8aa0068d90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80ae000 tx=0x7f8aa80a7000 comp rx=0 tx=0).stop 2023-11-28T18:24:19.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.610+0000 7f8ac48c9640 1 -- 172.21.15.105:0/2120555159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ac00b4290 msgr2=0x7f8ac00b3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:19.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.610+0000 7f8ac48c9640 1 --2- 172.21.15.105:0/2120555159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ac00b4290 0x7f8ac00b3460 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000ba50 tx=0x7f8ab000bf10 comp rx=0 tx=0).stop 2023-11-28T18:24:19.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.610+0000 7f8ac48c9640 1 -- 172.21.15.105:0/2120555159 shutdown_connections 2023-11-28T18:24:19.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.610+0000 7f8ac48c9640 1 --2- 172.21.15.105:0/2120555159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ac014ddf0 0x7f8ac0099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.610+0000 7f8ac48c9640 1 --2- 172.21.15.105:0/2120555159 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ac015b030 0x7f8ac0150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.610+0000 7f8ac48c9640 1 --2- 172.21.15.105:0/2120555159 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8aa00668d0 0x7f8aa0068d90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.610+0000 7f8ac48c9640 1 --2- 172.21.15.105:0/2120555159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ac00b4290 0x7f8ac00b3460 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.610+0000 7f8ac48c9640 1 -- 172.21.15.105:0/2120555159 >> 172.21.15.105:0/2120555159 conn(0x7f8ac00ceb90 msgr2=0x7f8ac00af600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:19.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.610+0000 7f8ac48c9640 1 -- 172.21.15.105:0/2120555159 shutdown_connections 2023-11-28T18:24:19.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.610+0000 7f8ac48c9640 1 -- 172.21.15.105:0/2120555159 wait complete. 2023-11-28T18:24:19.636 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T18:24:19.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.770+0000 7f8b94157640 1 Processor -- start 2023-11-28T18:24:19.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.770+0000 7f8b94157640 1 -- start start 2023-11-28T18:24:19.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.770+0000 7f8b94157640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8b8c159640 0x7f8b8c159a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:19.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.770+0000 7f8b94157640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b8c15a020 con 0x7f8b8c06df70 2023-11-28T18:24:19.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.770+0000 7f8b94157640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b8c15a300 con 0x7f8b8c071120 2023-11-28T18:24:19.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.770+0000 7f8b94157640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b8c15a9c0 con 0x7f8b8c159640 2023-11-28T18:24:19.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.770+0000 7f8b916f2640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8b8c06df70 0x7f8b8c070a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55886/0 (socket says 172.21.15.105:55886) 2023-11-28T18:24:19.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.770+0000 7f8b916f2640 1 -- 172.21.15.105:0/2158510903 learned_addr learned my addr 172.21.15.105:0/2158510903 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:19.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.770+0000 7f8b91ef3640 1 --2- 172.21.15.105:0/2158510903 >> v2:172.21.15.148:3300/0 conn(0x7f8b8c159640 0x7f8b8c159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:19.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.770+0000 7f8b90ef1640 1 -- 172.21.15.105:0/2158510903 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503990065 0 0) 0x7f8b8c15a020 con 0x7f8b8c06df70 2023-11-28T18:24:19.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.770+0000 7f8b90ef1640 1 -- 172.21.15.105:0/2158510903 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b78004640 con 0x7f8b8c06df70 2023-11-28T18:24:19.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.774+0000 7f8b90ef1640 1 -- 172.21.15.105:0/2158510903 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2937250122 0 0) 0x7f8b8c15a300 con 0x7f8b8c071120 2023-11-28T18:24:19.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.774+0000 7f8b90ef1640 1 -- 172.21.15.105:0/2158510903 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b8c15a020 con 0x7f8b8c071120 2023-11-28T18:24:19.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.774+0000 7f8b91ef3640 1 -- 172.21.15.105:0/2158510903 >> v1:172.21.15.148:6789/0 conn(0x7f8b8c071120 legacy=0x7f8b8c06a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:19.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.774+0000 7f8b91ef3640 1 -- 172.21.15.105:0/2158510903 >> v1:172.21.15.105:6789/0 conn(0x7f8b8c06df70 legacy=0x7f8b8c070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:19.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.774+0000 7f8b91ef3640 1 -- 172.21.15.105:0/2158510903 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c15a5e0 con 0x7f8b8c159640 2023-11-28T18:24:19.780 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.774+0000 7f8b91ef3640 1 --2- 172.21.15.105:0/2158510903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c159640 0x7f8b8c159a20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8b880097c0 tx=0x7f8b8801c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f1d0fcfbd7f183a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:19.781 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.774+0000 7f8b90ef1640 1 -- 172.21.15.105:0/2158510903 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b880ab600 con 0x7f8b8c159640 2023-11-28T18:24:19.781 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.774+0000 7f8b90ef1640 1 -- 172.21.15.105:0/2158510903 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b880a7050 con 0x7f8b8c159640 2023-11-28T18:24:19.781 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.774+0000 7f8b90ef1640 1 -- 172.21.15.105:0/2158510903 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b880ab980 con 0x7f8b8c159640 2023-11-28T18:24:19.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.778+0000 7f8b94157640 1 -- 172.21.15.105:0/2158510903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c159640 msgr2=0x7f8b8c159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:19.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.778+0000 7f8b94157640 1 --2- 172.21.15.105:0/2158510903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c159640 0x7f8b8c159a20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8b880097c0 tx=0x7f8b8801c360 comp rx=0 tx=0).stop 2023-11-28T18:24:19.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.778+0000 7f8b94157640 1 -- 172.21.15.105:0/2158510903 shutdown_connections 2023-11-28T18:24:19.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.778+0000 7f8b94157640 1 --2- 172.21.15.105:0/2158510903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c159640 0x7f8b8c159a20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.778+0000 7f8b94157640 1 -- 172.21.15.105:0/2158510903 >> 172.21.15.105:0/2158510903 conn(0x7f8b8c0cf540 msgr2=0x7f8b8c0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:19.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.778+0000 7f8b94157640 1 -- 172.21.15.105:0/2158510903 shutdown_connections 2023-11-28T18:24:19.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.778+0000 7f8b94157640 1 -- 172.21.15.105:0/2158510903 wait complete. 2023-11-28T18:24:19.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.778+0000 7f8b94157640 1 Processor -- start 2023-11-28T18:24:19.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.778+0000 7f8b94157640 1 -- start start 2023-11-28T18:24:19.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.778+0000 7f8b94157640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b8c06df70 0x7f8b8c13d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:19.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.778+0000 7f8b94157640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c071120 0x7f8b8c140430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:19.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.778+0000 7f8b94157640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b8c159640 0x7f8b8c151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:19.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.778+0000 7f8b94157640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c13d720 con 0x7f8b8c159640 2023-11-28T18:24:19.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.778+0000 7f8b94157640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c140970 con 0x7f8b8c071120 2023-11-28T18:24:19.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.778+0000 7f8b94157640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c13d990 con 0x7f8b8c06df70 2023-11-28T18:24:19.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.778+0000 7f8b926f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b8c159640 0x7f8b8c151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:19.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.778+0000 7f8b926f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b8c159640 0x7f8b8c151340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56316/0 (socket says 172.21.15.105:56316) 2023-11-28T18:24:19.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.778+0000 7f8b926f4640 1 -- 172.21.15.105:0/72090722 learned_addr learned my addr 172.21.15.105:0/72090722 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:19.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.782+0000 7f8b91ef3640 1 --2- 172.21.15.105:0/72090722 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b8c06df70 0x7f8b8c13d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:19.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.782+0000 7f8b916f2640 1 --2- 172.21.15.105:0/72090722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c071120 0x7f8b8c140430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:19.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.782+0000 7f8b926f4640 1 -- 172.21.15.105:0/72090722 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b8c06df70 msgr2=0x7f8b8c13d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:19.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.782+0000 7f8b926f4640 1 --2- 172.21.15.105:0/72090722 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b8c06df70 0x7f8b8c13d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.782+0000 7f8b926f4640 1 -- 172.21.15.105:0/72090722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c071120 msgr2=0x7f8b8c140430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:19.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.782+0000 7f8b926f4640 1 --2- 172.21.15.105:0/72090722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c071120 0x7f8b8c140430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.782+0000 7f8b926f4640 1 -- 172.21.15.105:0/72090722 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b88009420 con 0x7f8b8c159640 2023-11-28T18:24:19.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.782+0000 7f8b916f2640 1 --2- 172.21.15.105:0/72090722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c071120 0x7f8b8c140430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:19.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.782+0000 7f8b926f4640 1 --2- 172.21.15.105:0/72090722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b8c159640 0x7f8b8c151340 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400c7a0 tx=0x7f8b8400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:19.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.782+0000 7f8b827fc640 1 -- 172.21.15.105:0/72090722 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b84003b80 con 0x7f8b8c159640 2023-11-28T18:24:19.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.782+0000 7f8b827fc640 1 -- 172.21.15.105:0/72090722 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b84003ce0 con 0x7f8b8c159640 2023-11-28T18:24:19.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.782+0000 7f8b827fc640 1 -- 172.21.15.105:0/72090722 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8400aa90 con 0x7f8b8c159640 2023-11-28T18:24:19.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.782+0000 7f8b94157640 1 -- 172.21.15.105:0/72090722 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8c13dc30 con 0x7f8b8c159640 2023-11-28T18:24:19.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.782+0000 7f8b94157640 1 -- 172.21.15.105:0/72090722 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8c13e1f0 con 0x7f8b8c159640 2023-11-28T18:24:19.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.782+0000 7f8b94157640 1 -- 172.21.15.105:0/72090722 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8c067570 con 0x7f8b8c159640 2023-11-28T18:24:19.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.782+0000 7f8b827fc640 1 -- 172.21.15.105:0/72090722 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8b8400abf0 con 0x7f8b8c159640 2023-11-28T18:24:19.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.782+0000 7f8b827fc640 1 --2- 172.21.15.105:0/72090722 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b700668d0 0x7f8b70068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:19.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.782+0000 7f8b827fc640 1 -- 172.21.15.105:0/72090722 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f8b84084690 con 0x7f8b8c159640 2023-11-28T18:24:19.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.786+0000 7f8b91ef3640 1 --2- 172.21.15.105:0/72090722 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b700668d0 0x7f8b70068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:19.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.786+0000 7f8b91ef3640 1 --2- 172.21.15.105:0/72090722 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b700668d0 0x7f8b70068d90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8b880ad000 tx=0x7f8b880a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:19.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.786+0000 7f8b827fc640 1 -- 172.21.15.105:0/72090722 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8b840514c0 con 0x7f8b8c159640 2023-11-28T18:24:19.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.934+0000 7f8b94157640 1 -- 172.21.15.105:0/72090722 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8b8c071b00 con 0x7f8b8c159640 2023-11-28T18:24:19.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.938+0000 7f8b827fc640 1 -- 172.21.15.105:0/72090722 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8b84054ae0 con 0x7f8b8c159640 2023-11-28T18:24:19.942 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:19.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.938+0000 7f8b94157640 1 -- 172.21.15.105:0/72090722 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b700668d0 msgr2=0x7f8b70068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:19.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.938+0000 7f8b94157640 1 --2- 172.21.15.105:0/72090722 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b700668d0 0x7f8b70068d90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8b880ad000 tx=0x7f8b880a6000 comp rx=0 tx=0).stop 2023-11-28T18:24:19.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.938+0000 7f8b94157640 1 -- 172.21.15.105:0/72090722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b8c159640 msgr2=0x7f8b8c151340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:19.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.938+0000 7f8b94157640 1 --2- 172.21.15.105:0/72090722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b8c159640 0x7f8b8c151340 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400c7a0 tx=0x7f8b8400cc60 comp rx=0 tx=0).stop 2023-11-28T18:24:19.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.938+0000 7f8b94157640 1 -- 172.21.15.105:0/72090722 shutdown_connections 2023-11-28T18:24:19.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.938+0000 7f8b94157640 1 --2- 172.21.15.105:0/72090722 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b700668d0 0x7f8b70068d90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.938+0000 7f8b94157640 1 --2- 172.21.15.105:0/72090722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b8c159640 0x7f8b8c151340 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.938+0000 7f8b94157640 1 --2- 172.21.15.105:0/72090722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c071120 0x7f8b8c140430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.938+0000 7f8b94157640 1 --2- 172.21.15.105:0/72090722 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b8c06df70 0x7f8b8c13d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:19.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.938+0000 7f8b94157640 1 -- 172.21.15.105:0/72090722 >> 172.21.15.105:0/72090722 conn(0x7f8b8c0cf540 msgr2=0x7f8b8c0cc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:19.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.938+0000 7f8b94157640 1 -- 172.21.15.105:0/72090722 shutdown_connections 2023-11-28T18:24:19.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:19.938+0000 7f8b94157640 1 -- 172.21.15.105:0/72090722 wait complete. 2023-11-28T18:24:19.975 INFO:tasks.cephfs_test_runner:Starting test: test_ephemeral_pin_dist_conf_off (tasks.cephfs.test_exports.TestEphemeralPins) That turning off ephemeral distributed pin config prevents distribution. 2023-11-28T18:24:19.975 INFO:tasks.cephfs_test_runner:test_ephemeral_pin_dist_conf_off (tasks.cephfs.test_exports.TestEphemeralPins) 2023-11-28T18:24:19.975 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:24:20.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.110+0000 7f625ddd0640 1 Processor -- start 2023-11-28T18:24:20.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.110+0000 7f625ddd0640 1 -- start start 2023-11-28T18:24:20.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.110+0000 7f625ddd0640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6258142860 0x7f62580cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:20.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.110+0000 7f625ddd0640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f625806c470 0x7f625806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:20.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.110+0000 7f625ddd0640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6258143060 con 0x7f62580c8fc0 2023-11-28T18:24:20.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.110+0000 7f625ddd0640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6258159970 con 0x7f6258142860 2023-11-28T18:24:20.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.110+0000 7f625ddd0640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625806cf80 con 0x7f625806c470 2023-11-28T18:24:20.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.110+0000 7f625cdce640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6258142860 0x7f62580cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:20.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.110+0000 7f625d5cf640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f62580c8fc0 0x7f625815cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55888/0 (socket says 172.21.15.105:55888) 2023-11-28T18:24:20.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.110+0000 7f625d5cf640 1 -- 172.21.15.105:0/2019923642 learned_addr learned my addr 172.21.15.105:0/2019923642 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:20.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.110+0000 7f6257fff640 1 --2- 172.21.15.105:0/2019923642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f625806c470 0x7f625806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:20.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.114+0000 7f625cdce640 1 -- 172.21.15.105:0/2019923642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f625806c470 msgr2=0x7f625806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:20.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.114+0000 7f625cdce640 1 --2- 172.21.15.105:0/2019923642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f625806c470 0x7f625806c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.114+0000 7f625cdce640 1 -- 172.21.15.105:0/2019923642 >> v1:172.21.15.105:6789/0 conn(0x7f62580c8fc0 legacy=0x7f625815cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:20.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.114+0000 7f625cdce640 1 -- 172.21.15.105:0/2019923642 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625806d260 con 0x7f6258142860 2023-11-28T18:24:20.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.114+0000 7f62577fe640 1 -- 172.21.15.105:0/2019923642 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969059344 0 0) 0x7f6258143060 con 0x7f62580c8fc0 2023-11-28T18:24:20.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.114+0000 7f625cdce640 1 --2- 172.21.15.105:0/2019923642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6258142860 0x7f62580cb3f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f624c0097c0 tx=0x7f624c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef9ce6588d49f62 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:20.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.114+0000 7f62577fe640 1 -- 172.21.15.105:0/2019923642 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c0ac620 con 0x7f6258142860 2023-11-28T18:24:20.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.114+0000 7f62577fe640 1 -- 172.21.15.105:0/2019923642 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624c0ac780 con 0x7f6258142860 2023-11-28T18:24:20.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.114+0000 7f62577fe640 1 -- 172.21.15.105:0/2019923642 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c0acb10 con 0x7f6258142860 2023-11-28T18:24:20.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625ddd0640 1 -- 172.21.15.105:0/2019923642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6258142860 msgr2=0x7f62580cb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:20.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625ddd0640 1 --2- 172.21.15.105:0/2019923642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6258142860 0x7f62580cb3f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f624c0097c0 tx=0x7f624c01c3c0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625ddd0640 1 -- 172.21.15.105:0/2019923642 shutdown_connections 2023-11-28T18:24:20.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625ddd0640 1 --2- 172.21.15.105:0/2019923642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f625806c470 0x7f625806c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625ddd0640 1 --2- 172.21.15.105:0/2019923642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6258142860 0x7f62580cb3f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625ddd0640 1 -- 172.21.15.105:0/2019923642 >> 172.21.15.105:0/2019923642 conn(0x7f62580b0480 msgr2=0x7f62580b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:20.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625ddd0640 1 -- 172.21.15.105:0/2019923642 shutdown_connections 2023-11-28T18:24:20.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625ddd0640 1 -- 172.21.15.105:0/2019923642 wait complete. 2023-11-28T18:24:20.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625ddd0640 1 Processor -- start 2023-11-28T18:24:20.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625ddd0640 1 -- start start 2023-11-28T18:24:20.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625ddd0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f625806c470 0x7f6258159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:20.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625ddd0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62580c8fc0 0x7f62580c1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:20.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625ddd0640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6258142860 0x7f62581f7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:20.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625ddd0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62580afb10 con 0x7f625806c470 2023-11-28T18:24:20.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625ddd0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6258159ff0 con 0x7f62580c8fc0 2023-11-28T18:24:20.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625ddd0640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625815c340 con 0x7f6258142860 2023-11-28T18:24:20.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625cdce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f625806c470 0x7f6258159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:20.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625cdce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f625806c470 0x7f6258159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56332/0 (socket says 172.21.15.105:56332) 2023-11-28T18:24:20.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625cdce640 1 -- 172.21.15.105:0/410480246 learned_addr learned my addr 172.21.15.105:0/410480246 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:20.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f6257fff640 1 --2- 172.21.15.105:0/410480246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62580c8fc0 0x7f62580c1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:20.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625d5cf640 1 --2- 172.21.15.105:0/410480246 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6258142860 0x7f62581f7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:20.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625cdce640 1 -- 172.21.15.105:0/410480246 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6258142860 msgr2=0x7f62581f7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:20.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625cdce640 1 --2- 172.21.15.105:0/410480246 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6258142860 0x7f62581f7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625cdce640 1 -- 172.21.15.105:0/410480246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62580c8fc0 msgr2=0x7f62580c1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:20.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625cdce640 1 --2- 172.21.15.105:0/410480246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62580c8fc0 0x7f62580c1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625cdce640 1 -- 172.21.15.105:0/410480246 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f624c009420 con 0x7f625806c470 2023-11-28T18:24:20.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.118+0000 7f625cdce640 1 --2- 172.21.15.105:0/410480246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f625806c470 0x7f6258159ab0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f624c019ff0 tx=0x7f624c01ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:20.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.122+0000 7f62557fa640 1 -- 172.21.15.105:0/410480246 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c0b8450 con 0x7f625806c470 2023-11-28T18:24:20.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.122+0000 7f62557fa640 1 -- 172.21.15.105:0/410480246 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624c0b85b0 con 0x7f625806c470 2023-11-28T18:24:20.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.122+0000 7f62557fa640 1 -- 172.21.15.105:0/410480246 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c0b8960 con 0x7f625806c470 2023-11-28T18:24:20.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.122+0000 7f625ddd0640 1 -- 172.21.15.105:0/410480246 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f625815c620 con 0x7f625806c470 2023-11-28T18:24:20.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.122+0000 7f625ddd0640 1 -- 172.21.15.105:0/410480246 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62580c16c0 con 0x7f625806c470 2023-11-28T18:24:20.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.122+0000 7f625ddd0640 1 -- 172.21.15.105:0/410480246 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6258074580 con 0x7f625806c470 2023-11-28T18:24:20.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.122+0000 7f62557fa640 1 -- 172.21.15.105:0/410480246 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f624c0aa070 con 0x7f625806c470 2023-11-28T18:24:20.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.122+0000 7f62557fa640 1 --2- 172.21.15.105:0/410480246 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f623c066900 0x7f623c068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:20.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.122+0000 7f62557fa640 1 -- 172.21.15.105:0/410480246 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f624c11ee60 con 0x7f625806c470 2023-11-28T18:24:20.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.122+0000 7f6257fff640 1 --2- 172.21.15.105:0/410480246 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f623c066900 0x7f623c068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:20.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.122+0000 7f6257fff640 1 --2- 172.21.15.105:0/410480246 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f623c066900 0x7f623c068dc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f62580c8450 tx=0x7f6244008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:20.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.126+0000 7f62557fa640 1 -- 172.21.15.105:0/410480246 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f624c0ebca0 con 0x7f625806c470 2023-11-28T18:24:20.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.278+0000 7f625ddd0640 1 -- 172.21.15.105:0/410480246 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62580cc210 con 0x7f625806c470 2023-11-28T18:24:20.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.278+0000 7f62557fa640 1 -- 172.21.15.105:0/410480246 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18887 (secure 0 0 0) 0x7f624c0bc090 con 0x7f625806c470 2023-11-28T18:24:20.283 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:20.283 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:24:15.048168+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:24:20.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.278+0000 7f625ddd0640 1 -- 172.21.15.105:0/410480246 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f623c066900 msgr2=0x7f623c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:20.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.278+0000 7f625ddd0640 1 --2- 172.21.15.105:0/410480246 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f623c066900 0x7f623c068dc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f62580c8450 tx=0x7f6244008040 comp rx=0 tx=0).stop 2023-11-28T18:24:20.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.278+0000 7f625ddd0640 1 -- 172.21.15.105:0/410480246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f625806c470 msgr2=0x7f6258159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:20.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.278+0000 7f625ddd0640 1 --2- 172.21.15.105:0/410480246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f625806c470 0x7f6258159ab0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f624c019ff0 tx=0x7f624c01ac00 comp rx=0 tx=0).stop 2023-11-28T18:24:20.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.282+0000 7f625ddd0640 1 -- 172.21.15.105:0/410480246 shutdown_connections 2023-11-28T18:24:20.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.282+0000 7f625ddd0640 1 --2- 172.21.15.105:0/410480246 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6258142860 0x7f62581f7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.282+0000 7f625ddd0640 1 --2- 172.21.15.105:0/410480246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62580c8fc0 0x7f62580c1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.282+0000 7f625ddd0640 1 --2- 172.21.15.105:0/410480246 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f623c066900 0x7f623c068dc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.282+0000 7f625ddd0640 1 --2- 172.21.15.105:0/410480246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f625806c470 0x7f6258159ab0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.282+0000 7f625ddd0640 1 -- 172.21.15.105:0/410480246 >> 172.21.15.105:0/410480246 conn(0x7f62580b0480 msgr2=0x7f62580b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:20.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.282+0000 7f625ddd0640 1 -- 172.21.15.105:0/410480246 shutdown_connections 2023-11-28T18:24:20.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.282+0000 7f625ddd0640 1 -- 172.21.15.105:0/410480246 wait complete. 2023-11-28T18:24:20.308 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:23:31.485422+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:23:32.274378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T18:23:36.100071+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T18:24:20.309 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:24:20.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.446+0000 7f7fdb565640 1 Processor -- start 2023-11-28T18:24:20.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.446+0000 7f7fdb565640 1 -- start start 2023-11-28T18:24:20.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.446+0000 7f7fdb565640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fd40b4290 0x7f7fd40b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:20.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.446+0000 7f7fdb565640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7fd4071960 0x7f7fd415f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:20.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.446+0000 7f7fdb565640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fd40b4bb0 con 0x7f7fd415b030 2023-11-28T18:24:20.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.446+0000 7f7fdb565640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fd40701d0 con 0x7f7fd4071960 2023-11-28T18:24:20.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.446+0000 7f7fdb565640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd406df80 con 0x7f7fd40b4290 2023-11-28T18:24:20.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.446+0000 7f7fd8b00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7fd4071960 0x7f7fd415f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:20.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.446+0000 7f7fd8b00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7fd4071960 0x7f7fd415f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56344/0 (socket says 172.21.15.105:56344) 2023-11-28T18:24:20.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.446+0000 7f7fd9b02640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7fd415b030 0x7f7fd40709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55898/0 (socket says 172.21.15.105:55898) 2023-11-28T18:24:20.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.446+0000 7f7fd8b00640 1 -- 172.21.15.105:0/2541808156 learned_addr learned my addr 172.21.15.105:0/2541808156 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:20.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.446+0000 7f7fd9301640 1 --2- 172.21.15.105:0/2541808156 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fd40b4290 0x7f7fd40b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:20.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.446+0000 7f7fd8b00640 1 -- 172.21.15.105:0/2541808156 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fd40b4290 msgr2=0x7f7fd40b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:20.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.446+0000 7f7fd8b00640 1 --2- 172.21.15.105:0/2541808156 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fd40b4290 0x7f7fd40b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.446+0000 7f7fd8b00640 1 -- 172.21.15.105:0/2541808156 >> v1:172.21.15.105:6789/0 conn(0x7f7fd415b030 legacy=0x7f7fd40709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:20.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.446+0000 7f7fd8b00640 1 -- 172.21.15.105:0/2541808156 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd4071df0 con 0x7f7fd4071960 2023-11-28T18:24:20.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.446+0000 7f7fcbfff640 1 -- 172.21.15.105:0/2541808156 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2005469642 0 0) 0x7f7fd40b4bb0 con 0x7f7fd415b030 2023-11-28T18:24:20.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.450+0000 7f7fd8b00640 1 --2- 172.21.15.105:0/2541808156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fd4071960 0x7f7fd415f3b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009870 tx=0x7f7fc401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29cfd26deec1cbfc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:20.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.450+0000 7f7fcbfff640 1 -- 172.21.15.105:0/2541808156 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40aa670 con 0x7f7fd4071960 2023-11-28T18:24:20.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.450+0000 7f7fcbfff640 1 -- 172.21.15.105:0/2541808156 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc40aa7d0 con 0x7f7fd4071960 2023-11-28T18:24:20.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.450+0000 7f7fcbfff640 1 -- 172.21.15.105:0/2541808156 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40aab60 con 0x7f7fd4071960 2023-11-28T18:24:20.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.450+0000 7f7fdb565640 1 -- 172.21.15.105:0/2541808156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fd4071960 msgr2=0x7f7fd415f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:20.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.450+0000 7f7fdb565640 1 --2- 172.21.15.105:0/2541808156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fd4071960 0x7f7fd415f3b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009870 tx=0x7f7fc401c510 comp rx=0 tx=0).stop 2023-11-28T18:24:20.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.450+0000 7f7fdb565640 1 -- 172.21.15.105:0/2541808156 shutdown_connections 2023-11-28T18:24:20.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.450+0000 7f7fdb565640 1 --2- 172.21.15.105:0/2541808156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fd4071960 0x7f7fd415f3b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.450+0000 7f7fdb565640 1 --2- 172.21.15.105:0/2541808156 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fd40b4290 0x7f7fd40b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.450+0000 7f7fdb565640 1 -- 172.21.15.105:0/2541808156 >> 172.21.15.105:0/2541808156 conn(0x7f7fd40ceb90 msgr2=0x7f7fd40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:20.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.450+0000 7f7fdb565640 1 -- 172.21.15.105:0/2541808156 shutdown_connections 2023-11-28T18:24:20.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.450+0000 7f7fdb565640 1 -- 172.21.15.105:0/2541808156 wait complete. 2023-11-28T18:24:20.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.450+0000 7f7fdb565640 1 Processor -- start 2023-11-28T18:24:20.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.450+0000 7f7fdb565640 1 -- start start 2023-11-28T18:24:20.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fdb565640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fd40b4290 0x7f7fd4165010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:20.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fdb565640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fd415b030 0x7f7fd40b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:20.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fdb565640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fd41546f0 0x7f7fd40cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:20.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fdb565640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd4164b80 con 0x7f7fd41546f0 2023-11-28T18:24:20.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fdb565640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd4163f70 con 0x7f7fd40b4290 2023-11-28T18:24:20.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fdb565640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd4164250 con 0x7f7fd415b030 2023-11-28T18:24:20.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fd9b02640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fd41546f0 0x7f7fd40cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:20.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fd9b02640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fd41546f0 0x7f7fd40cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56346/0 (socket says 172.21.15.105:56346) 2023-11-28T18:24:20.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fd9b02640 1 -- 172.21.15.105:0/2970197576 learned_addr learned my addr 172.21.15.105:0/2970197576 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:20.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fd9301640 1 --2- 172.21.15.105:0/2970197576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fd40b4290 0x7f7fd4165010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:20.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fd8b00640 1 --2- 172.21.15.105:0/2970197576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fd415b030 0x7f7fd40b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:20.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fd9b02640 1 -- 172.21.15.105:0/2970197576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fd415b030 msgr2=0x7f7fd40b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:20.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fd9b02640 1 --2- 172.21.15.105:0/2970197576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fd415b030 0x7f7fd40b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fd9b02640 1 -- 172.21.15.105:0/2970197576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fd40b4290 msgr2=0x7f7fd4165010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:20.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fd9b02640 1 --2- 172.21.15.105:0/2970197576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fd40b4290 0x7f7fd4165010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fd9b02640 1 -- 172.21.15.105:0/2970197576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc40094d0 con 0x7f7fd41546f0 2023-11-28T18:24:20.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fd8b00640 1 --2- 172.21.15.105:0/2970197576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fd415b030 0x7f7fd40b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:20.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fd9b02640 1 --2- 172.21.15.105:0/2970197576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fd41546f0 0x7f7fd40cc760 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00c750 tx=0x7f7fcc00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:20.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fc9ffb640 1 -- 172.21.15.105:0/2970197576 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc00ce60 con 0x7f7fd41546f0 2023-11-28T18:24:20.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fc9ffb640 1 -- 172.21.15.105:0/2970197576 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fcc00a640 con 0x7f7fd41546f0 2023-11-28T18:24:20.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fdb565640 1 -- 172.21.15.105:0/2970197576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd4164530 con 0x7f7fd41546f0 2023-11-28T18:24:20.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fc9ffb640 1 -- 172.21.15.105:0/2970197576 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc00a9b0 con 0x7f7fd41546f0 2023-11-28T18:24:20.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fdb565640 1 -- 172.21.15.105:0/2970197576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd40b3e50 con 0x7f7fd41546f0 2023-11-28T18:24:20.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.454+0000 7f7fdb565640 1 -- 172.21.15.105:0/2970197576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd4164cf0 con 0x7f7fd41546f0 2023-11-28T18:24:20.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.458+0000 7f7fc9ffb640 1 -- 172.21.15.105:0/2970197576 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f7fcc00ab10 con 0x7f7fd41546f0 2023-11-28T18:24:20.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.458+0000 7f7fc9ffb640 1 --2- 172.21.15.105:0/2970197576 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7fb80668d0 0x7f7fb8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:20.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.458+0000 7f7fc9ffb640 1 -- 172.21.15.105:0/2970197576 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f7fcc084020 con 0x7f7fd41546f0 2023-11-28T18:24:20.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.458+0000 7f7fd9301640 1 --2- 172.21.15.105:0/2970197576 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7fb80668d0 0x7f7fb8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:20.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.458+0000 7f7fd9301640 1 --2- 172.21.15.105:0/2970197576 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7fb80668d0 0x7f7fb8068d90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0007a60 tx=0x7f7fd0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:20.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.462+0000 7f7fc9ffb640 1 -- 172.21.15.105:0/2970197576 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7fcc050f10 con 0x7f7fd41546f0 2023-11-28T18:24:20.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.610+0000 7f7fdb565640 1 -- 172.21.15.105:0/2970197576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7fd4067570 con 0x7f7fd41546f0 2023-11-28T18:24:20.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.610+0000 7f7fc9ffb640 1 -- 172.21.15.105:0/2970197576 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7fcc018020 con 0x7f7fd41546f0 2023-11-28T18:24:20.617 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:24:20.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.614+0000 7f7fdb565640 1 -- 172.21.15.105:0/2970197576 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7fb80668d0 msgr2=0x7f7fb8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:20.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.614+0000 7f7fdb565640 1 --2- 172.21.15.105:0/2970197576 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7fb80668d0 0x7f7fb8068d90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0007a60 tx=0x7f7fd0008040 comp rx=0 tx=0).stop 2023-11-28T18:24:20.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.614+0000 7f7fdb565640 1 -- 172.21.15.105:0/2970197576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fd41546f0 msgr2=0x7f7fd40cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:20.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.614+0000 7f7fdb565640 1 --2- 172.21.15.105:0/2970197576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fd41546f0 0x7f7fd40cc760 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00c750 tx=0x7f7fcc00cc10 comp rx=0 tx=0).stop 2023-11-28T18:24:20.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.614+0000 7f7fdb565640 1 -- 172.21.15.105:0/2970197576 shutdown_connections 2023-11-28T18:24:20.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.614+0000 7f7fdb565640 1 --2- 172.21.15.105:0/2970197576 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7fb80668d0 0x7f7fb8068d90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.614+0000 7f7fdb565640 1 --2- 172.21.15.105:0/2970197576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fd41546f0 0x7f7fd40cc760 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.614+0000 7f7fdb565640 1 --2- 172.21.15.105:0/2970197576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fd415b030 0x7f7fd40b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.614+0000 7f7fdb565640 1 --2- 172.21.15.105:0/2970197576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fd40b4290 0x7f7fd4165010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.614+0000 7f7fdb565640 1 -- 172.21.15.105:0/2970197576 >> 172.21.15.105:0/2970197576 conn(0x7f7fd40ceb90 msgr2=0x7f7fd40b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:20.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.614+0000 7f7fdb565640 1 -- 172.21.15.105:0/2970197576 shutdown_connections 2023-11-28T18:24:20.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.614+0000 7f7fdb565640 1 -- 172.21.15.105:0/2970197576 wait complete. 2023-11-28T18:24:20.638 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:24:20.780 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.774+0000 7f9d0f780640 1 Processor -- start 2023-11-28T18:24:20.780 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.774+0000 7f9d0f780640 1 -- start start 2023-11-28T18:24:20.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.774+0000 7f9d0f780640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9d080b4290 0x7f9d080b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:20.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.774+0000 7f9d0f780640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9d0815b030 0x7f9d080709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:20.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.774+0000 7f9d0f780640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d080b4bb0 con 0x7f9d08071960 2023-11-28T18:24:20.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.774+0000 7f9d0f780640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d080701d0 con 0x7f9d080b4290 2023-11-28T18:24:20.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.774+0000 7f9d0f780640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d0806df80 con 0x7f9d0815b030 2023-11-28T18:24:20.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.778+0000 7f9d0d51c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9d080b4290 0x7f9d080b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:20.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.778+0000 7f9d0d51c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9d080b4290 0x7f9d080b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56356/0 (socket says 172.21.15.105:56356) 2023-11-28T18:24:20.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.778+0000 7f9d0d51c640 1 -- 172.21.15.105:0/3194175369 learned_addr learned my addr 172.21.15.105:0/3194175369 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:20.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.778+0000 7f9d0d51c640 1 -- 172.21.15.105:0/3194175369 >> v2:172.21.15.148:3300/0 conn(0x7f9d0815b030 msgr2=0x7f9d080709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:20.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.778+0000 7f9d0dd1d640 1 --2- 172.21.15.105:0/3194175369 >> v2:172.21.15.148:3300/0 conn(0x7f9d0815b030 0x7f9d080709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:20.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.778+0000 7f9d0d51c640 1 --2- 172.21.15.105:0/3194175369 >> v2:172.21.15.148:3300/0 conn(0x7f9d0815b030 0x7f9d080709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.778+0000 7f9d0d51c640 1 -- 172.21.15.105:0/3194175369 >> v1:172.21.15.148:6789/0 conn(0x7f9d08071960 legacy=0x7f9d0815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:20.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.778+0000 7f9d0d51c640 1 -- 172.21.15.105:0/3194175369 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d08071df0 con 0x7f9d080b4290 2023-11-28T18:24:20.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.778+0000 7f9d0dd1d640 1 --2- 172.21.15.105:0/3194175369 >> v2:172.21.15.148:3300/0 conn(0x7f9d0815b030 0x7f9d080709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:20.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.778+0000 7f9d0d51c640 1 --2- 172.21.15.105:0/3194175369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d080b4290 0x7f9d080b4670 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9d0806e370 tx=0x7f9d0401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05b40b98c835a64 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:20.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.778+0000 7f9cfffff640 1 -- 172.21.15.105:0/3194175369 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d040ac600 con 0x7f9d080b4290 2023-11-28T18:24:20.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.778+0000 7f9cfffff640 1 -- 172.21.15.105:0/3194175369 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d040a8050 con 0x7f9d080b4290 2023-11-28T18:24:20.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.778+0000 7f9cfffff640 1 -- 172.21.15.105:0/3194175369 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d040ac980 con 0x7f9d080b4290 2023-11-28T18:24:20.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0f780640 1 -- 172.21.15.105:0/3194175369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d080b4290 msgr2=0x7f9d080b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:20.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0f780640 1 --2- 172.21.15.105:0/3194175369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d080b4290 0x7f9d080b4670 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9d0806e370 tx=0x7f9d0401c490 comp rx=0 tx=0).stop 2023-11-28T18:24:20.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0f780640 1 -- 172.21.15.105:0/3194175369 shutdown_connections 2023-11-28T18:24:20.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0f780640 1 --2- 172.21.15.105:0/3194175369 >> v2:172.21.15.148:3300/0 conn(0x7f9d0815b030 0x7f9d080709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0f780640 1 --2- 172.21.15.105:0/3194175369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d080b4290 0x7f9d080b4670 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0f780640 1 -- 172.21.15.105:0/3194175369 >> 172.21.15.105:0/3194175369 conn(0x7f9d080ceb90 msgr2=0x7f9d080cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:20.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0f780640 1 -- 172.21.15.105:0/3194175369 shutdown_connections 2023-11-28T18:24:20.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0f780640 1 -- 172.21.15.105:0/3194175369 wait complete. 2023-11-28T18:24:20.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0f780640 1 Processor -- start 2023-11-28T18:24:20.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0f780640 1 -- start start 2023-11-28T18:24:20.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0f780640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d080b4290 0x7f9d0814f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:20.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0f780640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d0815b030 0x7f9d080b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:20.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0f780640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d080b2970 0x7f9d08099f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:20.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0f780640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d080b2e80 con 0x7f9d080b4290 2023-11-28T18:24:20.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0f780640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d080b3160 con 0x7f9d0815b030 2023-11-28T18:24:20.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0f780640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d080b3440 con 0x7f9d080b2970 2023-11-28T18:24:20.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0d51c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d080b4290 0x7f9d0814f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:20.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0d51c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d080b4290 0x7f9d0814f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56364/0 (socket says 172.21.15.105:56364) 2023-11-28T18:24:20.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0d51c640 1 -- 172.21.15.105:0/3760241716 learned_addr learned my addr 172.21.15.105:0/3760241716 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:20.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0cd1b640 1 --2- 172.21.15.105:0/3760241716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d0815b030 0x7f9d080b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:20.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0dd1d640 1 --2- 172.21.15.105:0/3760241716 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d080b2970 0x7f9d08099f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:20.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0d51c640 1 -- 172.21.15.105:0/3760241716 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d080b2970 msgr2=0x7f9d08099f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:20.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0d51c640 1 --2- 172.21.15.105:0/3760241716 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d080b2970 0x7f9d08099f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0d51c640 1 -- 172.21.15.105:0/3760241716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d0815b030 msgr2=0x7f9d080b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:20.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0d51c640 1 --2- 172.21.15.105:0/3760241716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d0815b030 0x7f9d080b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:20.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0d51c640 1 -- 172.21.15.105:0/3760241716 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d040094d0 con 0x7f9d080b4290 2023-11-28T18:24:20.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.782+0000 7f9d0dd1d640 1 --2- 172.21.15.105:0/3760241716 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d080b2970 0x7f9d08099f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:20.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.786+0000 7f9d0d51c640 1 --2- 172.21.15.105:0/3760241716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d080b4290 0x7f9d0814f950 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9d0401a0c0 tx=0x7f9d0401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:20.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.786+0000 7f9cfdffb640 1 -- 172.21.15.105:0/3760241716 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d040b7650 con 0x7f9d080b4290 2023-11-28T18:24:20.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.786+0000 7f9cfdffb640 1 -- 172.21.15.105:0/3760241716 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d040b77b0 con 0x7f9d080b4290 2023-11-28T18:24:20.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.786+0000 7f9cfdffb640 1 -- 172.21.15.105:0/3760241716 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d040b7b20 con 0x7f9d080b4290 2023-11-28T18:24:20.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.786+0000 7f9d0f780640 1 -- 172.21.15.105:0/3760241716 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d080b3720 con 0x7f9d080b4290 2023-11-28T18:24:20.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.786+0000 7f9d0f780640 1 -- 172.21.15.105:0/3760241716 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0814fe90 con 0x7f9d080b4290 2023-11-28T18:24:20.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.786+0000 7f9d0f780640 1 -- 172.21.15.105:0/3760241716 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d08067570 con 0x7f9d080b4290 2023-11-28T18:24:20.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.786+0000 7f9cfdffb640 1 -- 172.21.15.105:0/3760241716 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f9d040b3070 con 0x7f9d080b4290 2023-11-28T18:24:20.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.786+0000 7f9cfdffb640 1 --2- 172.21.15.105:0/3760241716 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9cec0668d0 0x7f9cec068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:20.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.786+0000 7f9cfdffb640 1 -- 172.21.15.105:0/3760241716 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f9d0411eef0 con 0x7f9d080b4290 2023-11-28T18:24:20.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.786+0000 7f9d0cd1b640 1 --2- 172.21.15.105:0/3760241716 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9cec0668d0 0x7f9cec068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:20.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.790+0000 7f9d0cd1b640 1 --2- 172.21.15.105:0/3760241716 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9cec0668d0 0x7f9cec068d90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9d080b0060 tx=0x7f9cf8007e40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:20.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.790+0000 7f9cfdffb640 1 -- 172.21.15.105:0/3760241716 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9d040ebdc0 con 0x7f9d080b4290 2023-11-28T18:24:20.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.942+0000 7f9d0f780640 1 -- 172.21.15.105:0/3760241716 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9d08155d10 con 0x7f9d080b4290 2023-11-28T18:24:21.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.946+0000 7f9cfdffb640 1 -- 172.21.15.105:0/3760241716 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9d040ef3e0 con 0x7f9d080b4290 2023-11-28T18:24:21.092 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:21.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.946+0000 7f9d0f780640 1 -- 172.21.15.105:0/3760241716 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9cec0668d0 msgr2=0x7f9cec068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:21.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.946+0000 7f9d0f780640 1 --2- 172.21.15.105:0/3760241716 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9cec0668d0 0x7f9cec068d90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9d080b0060 tx=0x7f9cf8007e40 comp rx=0 tx=0).stop 2023-11-28T18:24:21.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.946+0000 7f9d0f780640 1 -- 172.21.15.105:0/3760241716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d080b4290 msgr2=0x7f9d0814f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:21.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.946+0000 7f9d0f780640 1 --2- 172.21.15.105:0/3760241716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d080b4290 0x7f9d0814f950 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9d0401a0c0 tx=0x7f9d0401acd0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.946+0000 7f9d0f780640 1 -- 172.21.15.105:0/3760241716 shutdown_connections 2023-11-28T18:24:21.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.946+0000 7f9d0f780640 1 --2- 172.21.15.105:0/3760241716 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d080b2970 0x7f9d08099f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.946+0000 7f9d0f780640 1 --2- 172.21.15.105:0/3760241716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d0815b030 0x7f9d080b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.946+0000 7f9d0f780640 1 --2- 172.21.15.105:0/3760241716 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9cec0668d0 0x7f9cec068d90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.946+0000 7f9d0f780640 1 --2- 172.21.15.105:0/3760241716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d080b4290 0x7f9d0814f950 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.946+0000 7f9d0f780640 1 -- 172.21.15.105:0/3760241716 >> 172.21.15.105:0/3760241716 conn(0x7f9d080ceb90 msgr2=0x7f9d08155f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:21.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.946+0000 7f9d0f780640 1 -- 172.21.15.105:0/3760241716 shutdown_connections 2023-11-28T18:24:21.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:20.946+0000 7f9d0f780640 1 -- 172.21.15.105:0/3760241716 wait complete. 2023-11-28T18:24:21.095 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:24:21.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.234+0000 7fa1ae5b8640 1 Processor -- start 2023-11-28T18:24:21.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.234+0000 7fa1ae5b8640 1 -- start start 2023-11-28T18:24:21.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.238+0000 7fa1ae5b8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa1a80caf60 0x7fa1a80cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:21.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.238+0000 7fa1ae5b8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a806bf50 con 0x7fa1a8159f40 2023-11-28T18:24:21.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.238+0000 7fa1ae5b8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a815a320 con 0x7fa1a80ad850 2023-11-28T18:24:21.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.238+0000 7fa1ae5b8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1a80cb3c0 con 0x7fa1a80caf60 2023-11-28T18:24:21.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.238+0000 7fa1a7fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa1a8159f40 0x7fa1a80ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49202/0 (socket says 172.21.15.105:49202) 2023-11-28T18:24:21.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.238+0000 7fa1a7fff640 1 -- 172.21.15.105:0/3316063015 learned_addr learned my addr 172.21.15.105:0/3316063015 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:21.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.238+0000 7fa1a77fe640 1 --2- 172.21.15.105:0/3316063015 >> v2:172.21.15.105:3300/0 conn(0x7fa1a80caf60 0x7fa1a80cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:21.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.238+0000 7fa1a6ffd640 1 -- 172.21.15.105:0/3316063015 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660300306 0 0) 0x7fa1a806bf50 con 0x7fa1a8159f40 2023-11-28T18:24:21.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.238+0000 7fa1a6ffd640 1 -- 172.21.15.105:0/3316063015 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa190003530 con 0x7fa1a8159f40 2023-11-28T18:24:21.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.238+0000 7fa1a77fe640 1 -- 172.21.15.105:0/3316063015 >> v1:172.21.15.148:6789/0 conn(0x7fa1a80ad850 legacy=0x7fa1a80701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:21.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.238+0000 7fa1a77fe640 1 -- 172.21.15.105:0/3316063015 >> v1:172.21.15.105:6789/0 conn(0x7fa1a8159f40 legacy=0x7fa1a80ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:21.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.238+0000 7fa1a77fe640 1 -- 172.21.15.105:0/3316063015 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a8071380 con 0x7fa1a80caf60 2023-11-28T18:24:21.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.242+0000 7fa1a77fe640 1 --2- 172.21.15.105:0/3316063015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1a80caf60 0x7fa1a80cc760 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0097c0 tx=0x7fa19c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f8a2902b0ffaf8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:21.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.242+0000 7fa1a6ffd640 1 -- 172.21.15.105:0/3316063015 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0ac620 con 0x7fa1a80caf60 2023-11-28T18:24:21.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.242+0000 7fa1a6ffd640 1 -- 172.21.15.105:0/3316063015 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19c0ac780 con 0x7fa1a80caf60 2023-11-28T18:24:21.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.242+0000 7fa1a6ffd640 1 -- 172.21.15.105:0/3316063015 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0acaf0 con 0x7fa1a80caf60 2023-11-28T18:24:21.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.242+0000 7fa1ae5b8640 1 -- 172.21.15.105:0/3316063015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1a80caf60 msgr2=0x7fa1a80cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:21.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.242+0000 7fa1ae5b8640 1 --2- 172.21.15.105:0/3316063015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1a80caf60 0x7fa1a80cc760 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0097c0 tx=0x7fa19c01c3c0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.242+0000 7fa1ae5b8640 1 -- 172.21.15.105:0/3316063015 shutdown_connections 2023-11-28T18:24:21.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.242+0000 7fa1ae5b8640 1 --2- 172.21.15.105:0/3316063015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1a80caf60 0x7fa1a80cc760 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.242+0000 7fa1ae5b8640 1 -- 172.21.15.105:0/3316063015 >> 172.21.15.105:0/3316063015 conn(0x7fa1a80ceb90 msgr2=0x7fa1a80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:21.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.242+0000 7fa1ae5b8640 1 -- 172.21.15.105:0/3316063015 shutdown_connections 2023-11-28T18:24:21.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.242+0000 7fa1ae5b8640 1 -- 172.21.15.105:0/3316063015 wait complete. 2023-11-28T18:24:21.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.242+0000 7fa1ae5b8640 1 Processor -- start 2023-11-28T18:24:21.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.242+0000 7fa1ae5b8640 1 -- start start 2023-11-28T18:24:21.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1ae5b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1a80ad850 0x7fa1a8149b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:21.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1ae5b8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1a80caf60 0x7fa1a815cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:21.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1ae5b8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1a8159f40 0x7fa1a8099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:21.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1ae5b8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a814c100 con 0x7fa1a8159f40 2023-11-28T18:24:21.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1ae5b8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a814c370 con 0x7fa1a80ad850 2023-11-28T18:24:21.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1ae5b8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a814c650 con 0x7fa1a80caf60 2023-11-28T18:24:21.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1acb55640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1a8159f40 0x7fa1a8099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:21.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1acb55640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1a8159f40 0x7fa1a8099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43454/0 (socket says 172.21.15.105:43454) 2023-11-28T18:24:21.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1acb55640 1 -- 172.21.15.105:0/1840067073 learned_addr learned my addr 172.21.15.105:0/1840067073 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:21.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1a7fff640 1 --2- 172.21.15.105:0/1840067073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1a80ad850 0x7fa1a8149b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:21.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1a77fe640 1 --2- 172.21.15.105:0/1840067073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1a80caf60 0x7fa1a815cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:21.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1acb55640 1 -- 172.21.15.105:0/1840067073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1a80caf60 msgr2=0x7fa1a815cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:21.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1acb55640 1 --2- 172.21.15.105:0/1840067073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1a80caf60 0x7fa1a815cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1acb55640 1 -- 172.21.15.105:0/1840067073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1a80ad850 msgr2=0x7fa1a8149b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:21.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1acb55640 1 --2- 172.21.15.105:0/1840067073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1a80ad850 0x7fa1a8149b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1acb55640 1 -- 172.21.15.105:0/1840067073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa19c009420 con 0x7fa1a8159f40 2023-11-28T18:24:21.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1acb55640 1 --2- 172.21.15.105:0/1840067073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1a8159f40 0x7fa1a8099a50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000c770 tx=0x7fa1a000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:21.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1a4ff9640 1 -- 172.21.15.105:0/1840067073 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0003b80 con 0x7fa1a8159f40 2023-11-28T18:24:21.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1a4ff9640 1 -- 172.21.15.105:0/1840067073 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a0003ce0 con 0x7fa1a8159f40 2023-11-28T18:24:21.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1ae5b8640 1 -- 172.21.15.105:0/1840067073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a814c930 con 0x7fa1a8159f40 2023-11-28T18:24:21.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1a4ff9640 1 -- 172.21.15.105:0/1840067073 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a000aa90 con 0x7fa1a8159f40 2023-11-28T18:24:21.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1ae5b8640 1 -- 172.21.15.105:0/1840067073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a814a060 con 0x7fa1a8159f40 2023-11-28T18:24:21.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1ae5b8640 1 -- 172.21.15.105:0/1840067073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a8067570 con 0x7fa1a8159f40 2023-11-28T18:24:21.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.246+0000 7fa1a4ff9640 1 -- 172.21.15.105:0/1840067073 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa1a000abf0 con 0x7fa1a8159f40 2023-11-28T18:24:21.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.250+0000 7fa1a4ff9640 1 --2- 172.21.15.105:0/1840067073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1940668d0 0x7fa194068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:21.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.250+0000 7fa1a4ff9640 1 -- 172.21.15.105:0/1840067073 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fa1a00845d0 con 0x7fa1a8159f40 2023-11-28T18:24:21.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.250+0000 7fa1a7fff640 1 --2- 172.21.15.105:0/1840067073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1940668d0 0x7fa194068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:21.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.250+0000 7fa1a7fff640 1 --2- 172.21.15.105:0/1840067073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1940668d0 0x7fa194068d90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa1980011f0 tx=0x7fa198007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:21.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.254+0000 7fa1a4ff9640 1 -- 172.21.15.105:0/1840067073 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa1a0051420 con 0x7fa1a8159f40 2023-11-28T18:24:21.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.402+0000 7fa1ae5b8640 1 -- 172.21.15.105:0/1840067073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa1a814cde0 con 0x7fa1a8159f40 2023-11-28T18:24:21.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.402+0000 7fa1a4ff9640 1 -- 172.21.15.105:0/1840067073 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa1a0054a40 con 0x7fa1a8159f40 2023-11-28T18:24:21.409 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:24:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.406+0000 7fa1ae5b8640 1 -- 172.21.15.105:0/1840067073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1940668d0 msgr2=0x7fa194068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.406+0000 7fa1ae5b8640 1 --2- 172.21.15.105:0/1840067073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1940668d0 0x7fa194068d90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa1980011f0 tx=0x7fa198007d40 comp rx=0 tx=0).stop 2023-11-28T18:24:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.406+0000 7fa1ae5b8640 1 -- 172.21.15.105:0/1840067073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1a8159f40 msgr2=0x7fa1a8099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.406+0000 7fa1ae5b8640 1 --2- 172.21.15.105:0/1840067073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1a8159f40 0x7fa1a8099a50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000c770 tx=0x7fa1a000cc30 comp rx=0 tx=0).stop 2023-11-28T18:24:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.406+0000 7fa1ae5b8640 1 -- 172.21.15.105:0/1840067073 shutdown_connections 2023-11-28T18:24:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.406+0000 7fa1ae5b8640 1 --2- 172.21.15.105:0/1840067073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1940668d0 0x7fa194068d90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.406+0000 7fa1ae5b8640 1 --2- 172.21.15.105:0/1840067073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1a8159f40 0x7fa1a8099a50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.406+0000 7fa1ae5b8640 1 --2- 172.21.15.105:0/1840067073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1a80caf60 0x7fa1a815cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.406+0000 7fa1ae5b8640 1 --2- 172.21.15.105:0/1840067073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1a80ad850 0x7fa1a8149b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.406+0000 7fa1ae5b8640 1 -- 172.21.15.105:0/1840067073 >> 172.21.15.105:0/1840067073 conn(0x7fa1a80ceb90 msgr2=0x7fa1a8156320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:21.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.406+0000 7fa1ae5b8640 1 -- 172.21.15.105:0/1840067073 shutdown_connections 2023-11-28T18:24:21.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.406+0000 7fa1ae5b8640 1 -- 172.21.15.105:0/1840067073 wait complete. 2023-11-28T18:24:21.431 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T18:24:21.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.566+0000 7f6d1e396640 1 Processor -- start 2023-11-28T18:24:21.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.566+0000 7f6d1e396640 1 -- start start 2023-11-28T18:24:21.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.566+0000 7f6d1e396640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6d1815b030 0x7f6d180709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:21.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.566+0000 7f6d1e396640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d180701d0 con 0x7f6d18071960 2023-11-28T18:24:21.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.566+0000 7f6d1e396640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d1806df80 con 0x7f6d180b4290 2023-11-28T18:24:21.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.566+0000 7f6d1e396640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d18071df0 con 0x7f6d1815b030 2023-11-28T18:24:21.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.566+0000 7f6d0ffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f6d18071960 0x7f6d1815f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49210/0 (socket says 172.21.15.105:49210) 2023-11-28T18:24:21.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.566+0000 7f6d0ffff640 1 -- 172.21.15.105:0/1017495880 learned_addr learned my addr 172.21.15.105:0/1017495880 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:21.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.566+0000 7f6d1c933640 1 --2- 172.21.15.105:0/1017495880 >> v2:172.21.15.148:3300/0 conn(0x7f6d1815b030 0x7f6d180709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:21.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.566+0000 7f6d177fe640 1 -- 172.21.15.105:0/1017495880 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837668904 0 0) 0x7f6d180701d0 con 0x7f6d18071960 2023-11-28T18:24:21.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d177fe640 1 -- 172.21.15.105:0/1017495880 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d04003530 con 0x7f6d18071960 2023-11-28T18:24:21.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d177fe640 1 -- 172.21.15.105:0/1017495880 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2641749122 0 0) 0x7f6d04003530 con 0x7f6d18071960 2023-11-28T18:24:21.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d177fe640 1 -- 172.21.15.105:0/1017495880 >> v2:172.21.15.148:3300/0 conn(0x7f6d1815b030 msgr2=0x7f6d180709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:21.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d177fe640 1 --2- 172.21.15.105:0/1017495880 >> v2:172.21.15.148:3300/0 conn(0x7f6d1815b030 0x7f6d180709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d177fe640 1 -- 172.21.15.105:0/1017495880 >> v1:172.21.15.148:6789/0 conn(0x7f6d180b4290 legacy=0x7f6d180b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:21.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d177fe640 1 -- 172.21.15.105:0/1017495880 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d180720d0 con 0x7f6d18071960 2023-11-28T18:24:21.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d177fe640 1 -- 172.21.15.105:0/1017495880 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f6d00001680 con 0x7f6d18071960 2023-11-28T18:24:21.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d177fe640 1 -- 172.21.15.105:0/1017495880 >> v1:172.21.15.105:6789/0 conn(0x7f6d18071960 legacy=0x7f6d1815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:21.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d177fe640 1 --2- 172.21.15.105:0/1017495880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d040039d0 0x7f6d04003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:21.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d177fe640 1 --2- 172.21.15.105:0/1017495880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d040047d0 0x7f6d04004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:21.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d177fe640 1 --2- 172.21.15.105:0/1017495880 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d04005190 0x7f6d0400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:21.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d177fe640 1 -- 172.21.15.105:0/1017495880 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d18071df0 con 0x7f6d040039d0 2023-11-28T18:24:21.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d177fe640 1 -- 172.21.15.105:0/1017495880 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0400c090 con 0x7f6d040047d0 2023-11-28T18:24:21.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d177fe640 1 -- 172.21.15.105:0/1017495880 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0400c250 con 0x7f6d04005190 2023-11-28T18:24:21.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d17fff640 1 --2- 172.21.15.105:0/1017495880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d040039d0 0x7f6d04003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:21.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d1c933640 1 --2- 172.21.15.105:0/1017495880 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d04005190 0x7f6d0400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:21.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d0ffff640 1 --2- 172.21.15.105:0/1017495880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d040047d0 0x7f6d04004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:21.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d17fff640 1 -- 172.21.15.105:0/1017495880 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d04005190 msgr2=0x7f6d0400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:21.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d17fff640 1 --2- 172.21.15.105:0/1017495880 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d04005190 0x7f6d0400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d17fff640 1 -- 172.21.15.105:0/1017495880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d040047d0 msgr2=0x7f6d04004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:21.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d17fff640 1 --2- 172.21.15.105:0/1017495880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d040047d0 0x7f6d04004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d17fff640 1 -- 172.21.15.105:0/1017495880 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d04003530 con 0x7f6d040039d0 2023-11-28T18:24:21.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d1c933640 1 -- 172.21.15.105:0/1017495880 reap_dead start 2023-11-28T18:24:21.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.570+0000 7f6d17fff640 1 --2- 172.21.15.105:0/1017495880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d040039d0 0x7f6d04003df0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6d08001780 tx=0x7f6d0801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9242ce97f709e182 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:21.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.574+0000 7f6d177fe640 1 -- 172.21.15.105:0/1017495880 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d080052d0 con 0x7f6d040039d0 2023-11-28T18:24:21.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.574+0000 7f6d177fe640 1 -- 172.21.15.105:0/1017495880 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d08005450 con 0x7f6d040039d0 2023-11-28T18:24:21.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.574+0000 7f6d177fe640 1 -- 172.21.15.105:0/1017495880 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d08003840 con 0x7f6d040039d0 2023-11-28T18:24:21.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.574+0000 7f6d1e396640 1 -- 172.21.15.105:0/1017495880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d040039d0 msgr2=0x7f6d04003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:21.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.574+0000 7f6d1e396640 1 --2- 172.21.15.105:0/1017495880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d040039d0 0x7f6d04003df0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6d08001780 tx=0x7f6d0801c3b0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.574+0000 7f6d1e396640 1 -- 172.21.15.105:0/1017495880 shutdown_connections 2023-11-28T18:24:21.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.574+0000 7f6d1e396640 1 --2- 172.21.15.105:0/1017495880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d040039d0 0x7f6d04003df0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.574+0000 7f6d1e396640 1 -- 172.21.15.105:0/1017495880 >> 172.21.15.105:0/1017495880 conn(0x7f6d180ceb90 msgr2=0x7f6d180cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:21.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.574+0000 7f6d1e396640 1 -- 172.21.15.105:0/1017495880 shutdown_connections 2023-11-28T18:24:21.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.574+0000 7f6d1e396640 1 -- 172.21.15.105:0/1017495880 wait complete. 2023-11-28T18:24:21.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.574+0000 7f6d1e396640 1 Processor -- start 2023-11-28T18:24:21.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.574+0000 7f6d1e396640 1 -- start start 2023-11-28T18:24:21.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.574+0000 7f6d1e396640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d040039d0 0x7f6d18151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:21.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.574+0000 7f6d1e396640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d18157390 0x7f6d1809a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:21.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.574+0000 7f6d1e396640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d1809a9a0 0x7f6d1809bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:21.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.574+0000 7f6d1e396640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d181520e0 con 0x7f6d18157390 2023-11-28T18:24:21.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.574+0000 7f6d1e396640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1809aeb0 con 0x7f6d1809a9a0 2023-11-28T18:24:21.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.574+0000 7f6d1e396640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1809b190 con 0x7f6d040039d0 2023-11-28T18:24:21.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.574+0000 7f6d0ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d18157390 0x7f6d1809a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:21.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.574+0000 7f6d0ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d18157390 0x7f6d1809a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43458/0 (socket says 172.21.15.105:43458) 2023-11-28T18:24:21.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.574+0000 7f6d0ffff640 1 -- 172.21.15.105:0/1564030001 learned_addr learned my addr 172.21.15.105:0/1564030001 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:21.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.574+0000 7f6d17fff640 1 --2- 172.21.15.105:0/1564030001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d040039d0 0x7f6d18151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:21.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.578+0000 7f6d1c933640 1 --2- 172.21.15.105:0/1564030001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d1809a9a0 0x7f6d1809bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:21.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.578+0000 7f6d0ffff640 1 -- 172.21.15.105:0/1564030001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d040039d0 msgr2=0x7f6d18151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:21.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.578+0000 7f6d0ffff640 1 --2- 172.21.15.105:0/1564030001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d040039d0 0x7f6d18151ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.578+0000 7f6d0ffff640 1 -- 172.21.15.105:0/1564030001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d1809a9a0 msgr2=0x7f6d1809bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:21.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.578+0000 7f6d0ffff640 1 --2- 172.21.15.105:0/1564030001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d1809a9a0 0x7f6d1809bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.578+0000 7f6d0ffff640 1 -- 172.21.15.105:0/1564030001 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1809b470 con 0x7f6d18157390 2023-11-28T18:24:21.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.578+0000 7f6d0ffff640 1 --2- 172.21.15.105:0/1564030001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d18157390 0x7f6d1809a460 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000e5a0 tx=0x7f6d0000ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:21.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.578+0000 7f6d157fa640 1 -- 172.21.15.105:0/1564030001 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d00012590 con 0x7f6d18157390 2023-11-28T18:24:21.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.578+0000 7f6d157fa640 1 -- 172.21.15.105:0/1564030001 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d000126f0 con 0x7f6d18157390 2023-11-28T18:24:21.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.578+0000 7f6d1e396640 1 -- 172.21.15.105:0/1564030001 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1809b750 con 0x7f6d18157390 2023-11-28T18:24:21.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.578+0000 7f6d157fa640 1 -- 172.21.15.105:0/1564030001 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d00012aa0 con 0x7f6d18157390 2023-11-28T18:24:21.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.578+0000 7f6d1e396640 1 -- 172.21.15.105:0/1564030001 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d180b00e0 con 0x7f6d18157390 2023-11-28T18:24:21.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.578+0000 7f6d1e396640 1 -- 172.21.15.105:0/1564030001 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d18067570 con 0x7f6d18157390 2023-11-28T18:24:21.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.578+0000 7f6d157fa640 1 -- 172.21.15.105:0/1564030001 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f6d00012c00 con 0x7f6d18157390 2023-11-28T18:24:21.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.578+0000 7f6d157fa640 1 --2- 172.21.15.105:0/1564030001 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6cf8066900 0x7f6cf8068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:21.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.578+0000 7f6d157fa640 1 -- 172.21.15.105:0/1564030001 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f6d00084480 con 0x7f6d18157390 2023-11-28T18:24:21.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.582+0000 7f6d17fff640 1 --2- 172.21.15.105:0/1564030001 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6cf8066900 0x7f6cf8068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:21.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.582+0000 7f6d17fff640 1 --2- 172.21.15.105:0/1564030001 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6cf8066900 0x7f6cf8068dc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6d08001750 tx=0x7f6d080011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:21.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.582+0000 7f6d157fa640 1 -- 172.21.15.105:0/1564030001 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6d00051400 con 0x7f6d18157390 2023-11-28T18:24:21.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.734+0000 7f6d1e396640 1 -- 172.21.15.105:0/1564030001 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6d180cf7f0 con 0x7f6d18157390 2023-11-28T18:24:21.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.734+0000 7f6d157fa640 1 -- 172.21.15.105:0/1564030001 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6d00054a20 con 0x7f6d18157390 2023-11-28T18:24:21.740 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:21.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.738+0000 7f6d1e396640 1 -- 172.21.15.105:0/1564030001 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6cf8066900 msgr2=0x7f6cf8068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:21.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.738+0000 7f6d1e396640 1 --2- 172.21.15.105:0/1564030001 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6cf8066900 0x7f6cf8068dc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6d08001750 tx=0x7f6d080011a0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.738+0000 7f6d1e396640 1 -- 172.21.15.105:0/1564030001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d18157390 msgr2=0x7f6d1809a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:21.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.738+0000 7f6d1e396640 1 --2- 172.21.15.105:0/1564030001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d18157390 0x7f6d1809a460 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000e5a0 tx=0x7f6d0000ea60 comp rx=0 tx=0).stop 2023-11-28T18:24:21.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.738+0000 7f6d1e396640 1 -- 172.21.15.105:0/1564030001 shutdown_connections 2023-11-28T18:24:21.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.738+0000 7f6d1e396640 1 --2- 172.21.15.105:0/1564030001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d1809a9a0 0x7f6d1809bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.738+0000 7f6d1e396640 1 --2- 172.21.15.105:0/1564030001 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6cf8066900 0x7f6cf8068dc0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.738+0000 7f6d1e396640 1 --2- 172.21.15.105:0/1564030001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d18157390 0x7f6d1809a460 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.738+0000 7f6d1e396640 1 --2- 172.21.15.105:0/1564030001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d040039d0 0x7f6d18151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.738+0000 7f6d1e396640 1 -- 172.21.15.105:0/1564030001 >> 172.21.15.105:0/1564030001 conn(0x7f6d180ceb90 msgr2=0x7f6d180adc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:21.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.738+0000 7f6d1e396640 1 -- 172.21.15.105:0/1564030001 shutdown_connections 2023-11-28T18:24:21.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.738+0000 7f6d1e396640 1 -- 172.21.15.105:0/1564030001 wait complete. 2023-11-28T18:24:21.761 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_conf_off' 2023-11-28T18:24:21.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.894+0000 7f6b0f45a640 1 Processor -- start 2023-11-28T18:24:21.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.894+0000 7f6b0f45a640 1 -- start start 2023-11-28T18:24:21.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.894+0000 7f6b0f45a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6b08154070 0x7f6b0815d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:21.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.894+0000 7f6b0f45a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b0806bf50 0x7f6b080ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:21.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.894+0000 7f6b0f45a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6b080adb80 0x7f6b080adff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:21.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.894+0000 7f6b0f45a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b08072540 con 0x7f6b080adb80 2023-11-28T18:24:21.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.894+0000 7f6b0f45a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b080ae530 con 0x7f6b08154070 2023-11-28T18:24:21.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.894+0000 7f6b0f45a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b080cc7f0 con 0x7f6b0806bf50 2023-11-28T18:24:21.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.894+0000 7f6b0d9f7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6b080adb80 0x7f6b080adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:21.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.898+0000 7f6b0d9f7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6b080adb80 0x7f6b080adff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43468/0 (socket says 172.21.15.105:43468) 2023-11-28T18:24:21.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.898+0000 7f6b0d9f7640 1 -- 172.21.15.105:0/2749182274 learned_addr learned my addr 172.21.15.105:0/2749182274 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:21.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.898+0000 7f6b0d1f6640 1 --2- 172.21.15.105:0/2749182274 >> v2:172.21.15.148:3300/0 conn(0x7f6b08154070 0x7f6b0815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:21.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.898+0000 7f6b0c9f5640 1 --2- 172.21.15.105:0/2749182274 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b0806bf50 0x7f6b080ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:21.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.898+0000 7f6b0d9f7640 1 -- 172.21.15.105:0/2749182274 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b0806bf50 msgr2=0x7f6b080ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:21.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.898+0000 7f6b0d9f7640 1 --2- 172.21.15.105:0/2749182274 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b0806bf50 0x7f6b080ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.898+0000 7f6b0d9f7640 1 -- 172.21.15.105:0/2749182274 >> v2:172.21.15.148:3300/0 conn(0x7f6b08154070 msgr2=0x7f6b0815d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:21.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.898+0000 7f6b0d9f7640 1 --2- 172.21.15.105:0/2749182274 >> v2:172.21.15.148:3300/0 conn(0x7f6b08154070 0x7f6b0815d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.898+0000 7f6b0d9f7640 1 -- 172.21.15.105:0/2749182274 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0815dd10 con 0x7f6b080adb80 2023-11-28T18:24:21.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.898+0000 7f6b0d1f6640 1 --2- 172.21.15.105:0/2749182274 >> v2:172.21.15.148:3300/0 conn(0x7f6b08154070 0x7f6b0815d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:21.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.898+0000 7f6b0d9f7640 1 --2- 172.21.15.105:0/2749182274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b080adb80 0x7f6b080adff0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6b0806fa90 tx=0x7f6b0001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9eca24fb58b4f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:21.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.898+0000 7f6afffff640 1 -- 172.21.15.105:0/2749182274 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b000a6460 con 0x7f6b080adb80 2023-11-28T18:24:21.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.898+0000 7f6afffff640 1 -- 172.21.15.105:0/2749182274 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b000a65c0 con 0x7f6b080adb80 2023-11-28T18:24:21.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.898+0000 7f6afffff640 1 -- 172.21.15.105:0/2749182274 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b000a6950 con 0x7f6b080adb80 2023-11-28T18:24:21.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0f45a640 1 -- 172.21.15.105:0/2749182274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b080adb80 msgr2=0x7f6b080adff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:21.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0f45a640 1 --2- 172.21.15.105:0/2749182274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b080adb80 0x7f6b080adff0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6b0806fa90 tx=0x7f6b0001c9a0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0f45a640 1 -- 172.21.15.105:0/2749182274 shutdown_connections 2023-11-28T18:24:21.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0f45a640 1 --2- 172.21.15.105:0/2749182274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b080adb80 0x7f6b080adff0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0f45a640 1 --2- 172.21.15.105:0/2749182274 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b0806bf50 0x7f6b080ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0f45a640 1 --2- 172.21.15.105:0/2749182274 >> v2:172.21.15.148:3300/0 conn(0x7f6b08154070 0x7f6b0815d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0f45a640 1 -- 172.21.15.105:0/2749182274 >> 172.21.15.105:0/2749182274 conn(0x7f6b080ceb90 msgr2=0x7f6b080cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:21.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0f45a640 1 -- 172.21.15.105:0/2749182274 shutdown_connections 2023-11-28T18:24:21.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0f45a640 1 -- 172.21.15.105:0/2749182274 wait complete. 2023-11-28T18:24:21.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0f45a640 1 Processor -- start 2023-11-28T18:24:21.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0f45a640 1 -- start start 2023-11-28T18:24:21.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0f45a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b0806bf50 0x7f6b0813a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:21.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0f45a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b080adb80 0x7f6b0814c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:21.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0f45a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b08154070 0x7f6b0814d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:21.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0f45a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b08153aa0 con 0x7f6b08154070 2023-11-28T18:24:21.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0f45a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0813aa00 con 0x7f6b080adb80 2023-11-28T18:24:21.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0f45a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b08152e90 con 0x7f6b0806bf50 2023-11-28T18:24:21.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0d9f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b08154070 0x7f6b0814d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:21.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0d9f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b08154070 0x7f6b0814d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43484/0 (socket says 172.21.15.105:43484) 2023-11-28T18:24:21.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0d9f7640 1 -- 172.21.15.105:0/3402825903 learned_addr learned my addr 172.21.15.105:0/3402825903 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:21.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0d9f7640 1 -- 172.21.15.105:0/3402825903 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b0806bf50 msgr2=0x7f6b0813a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:21.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0d9f7640 1 --2- 172.21.15.105:0/3402825903 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b0806bf50 0x7f6b0813a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0d9f7640 1 -- 172.21.15.105:0/3402825903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b080adb80 msgr2=0x7f6b0814c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:21.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0d9f7640 1 --2- 172.21.15.105:0/3402825903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b080adb80 0x7f6b0814c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:21.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0d9f7640 1 -- 172.21.15.105:0/3402825903 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b00009b30 con 0x7f6b08154070 2023-11-28T18:24:21.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.902+0000 7f6b0d9f7640 1 --2- 172.21.15.105:0/3402825903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b08154070 0x7f6b0814d2a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6b0001b050 tx=0x7f6b00019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:21.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.906+0000 7f6afdffb640 1 -- 172.21.15.105:0/3402825903 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b000a6460 con 0x7f6b08154070 2023-11-28T18:24:21.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.906+0000 7f6afdffb640 1 -- 172.21.15.105:0/3402825903 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b000a7640 con 0x7f6b08154070 2023-11-28T18:24:21.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.906+0000 7f6afdffb640 1 -- 172.21.15.105:0/3402825903 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b000a79d0 con 0x7f6b08154070 2023-11-28T18:24:21.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.906+0000 7f6b0f45a640 1 -- 172.21.15.105:0/3402825903 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b08153170 con 0x7f6b08154070 2023-11-28T18:24:21.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.906+0000 7f6b0f45a640 1 -- 172.21.15.105:0/3402825903 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b08153730 con 0x7f6b08154070 2023-11-28T18:24:21.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.906+0000 7f6b0f45a640 1 -- 172.21.15.105:0/3402825903 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b081538c0 con 0x7f6b08154070 2023-11-28T18:24:21.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.906+0000 7f6afdffb640 1 -- 172.21.15.105:0/3402825903 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f6b000a7b30 con 0x7f6b08154070 2023-11-28T18:24:21.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.906+0000 7f6afdffb640 1 --2- 172.21.15.105:0/3402825903 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6aec0668d0 0x7f6aec068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:21.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.910+0000 7f6afdffb640 1 -- 172.21.15.105:0/3402825903 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f6b0011e410 con 0x7f6b08154070 2023-11-28T18:24:21.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.910+0000 7f6b0d1f6640 1 --2- 172.21.15.105:0/3402825903 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6aec0668d0 0x7f6aec068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:21.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.910+0000 7f6b0d1f6640 1 --2- 172.21.15.105:0/3402825903 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6aec0668d0 0x7f6aec068d90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6b04006e90 tx=0x7f6b04008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:21.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:21.910+0000 7f6afdffb640 1 -- 172.21.15.105:0/3402825903 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6b000eb120 con 0x7f6b08154070 2023-11-28T18:24:22.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:22.118+0000 7f6b0f45a640 1 -- 172.21.15.105:0/3402825903 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_conf_off"]} v 0) v1 -- 0x7f6b08067570 con 0x7f6b08154070 2023-11-28T18:24:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.086+0000 7f6afdffb640 1 -- 172.21.15.105:0/3402825903 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_conf_off"]}]=0 v71) v1 ==== 158+0+0 (secure 0 0 0) 0x7f6b00009510 con 0x7f6b08154070 2023-11-28T18:24:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.090+0000 7f6b0f45a640 1 -- 172.21.15.105:0/3402825903 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6aec0668d0 msgr2=0x7f6aec068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.090+0000 7f6b0f45a640 1 --2- 172.21.15.105:0/3402825903 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6aec0668d0 0x7f6aec068d90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6b04006e90 tx=0x7f6b04008040 comp rx=0 tx=0).stop 2023-11-28T18:24:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.090+0000 7f6b0f45a640 1 -- 172.21.15.105:0/3402825903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b08154070 msgr2=0x7f6b0814d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.090+0000 7f6b0f45a640 1 --2- 172.21.15.105:0/3402825903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b08154070 0x7f6b0814d2a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6b0001b050 tx=0x7f6b00019740 comp rx=0 tx=0).stop 2023-11-28T18:24:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.090+0000 7f6b0f45a640 1 -- 172.21.15.105:0/3402825903 shutdown_connections 2023-11-28T18:24:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.090+0000 7f6b0f45a640 1 --2- 172.21.15.105:0/3402825903 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6aec0668d0 0x7f6aec068d90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.090+0000 7f6b0f45a640 1 --2- 172.21.15.105:0/3402825903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b08154070 0x7f6b0814d2a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.090+0000 7f6b0f45a640 1 --2- 172.21.15.105:0/3402825903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b080adb80 0x7f6b0814c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.090+0000 7f6b0f45a640 1 --2- 172.21.15.105:0/3402825903 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b0806bf50 0x7f6b0813a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:23.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.090+0000 7f6b0f45a640 1 -- 172.21.15.105:0/3402825903 >> 172.21.15.105:0/3402825903 conn(0x7f6b080ceb90 msgr2=0x7f6b081603e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:23.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.090+0000 7f6b0f45a640 1 -- 172.21.15.105:0/3402825903 shutdown_connections 2023-11-28T18:24:23.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.090+0000 7f6b0f45a640 1 -- 172.21.15.105:0/3402825903 wait complete. 2023-11-28T18:24:23.117 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T18:24:23.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.254+0000 7f1d1fec9640 1 Processor -- start 2023-11-28T18:24:23.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.258+0000 7f1d1fec9640 1 -- start start 2023-11-28T18:24:23.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.258+0000 7f1d1fec9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1d18071960 0x7f1d1815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:23.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.258+0000 7f1d1fec9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d180701d0 con 0x7f1d1815b030 2023-11-28T18:24:23.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.258+0000 7f1d1fec9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d1806df80 con 0x7f1d180b4290 2023-11-28T18:24:23.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.258+0000 7f1d1fec9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d18071df0 con 0x7f1d18071960 2023-11-28T18:24:23.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.258+0000 7f1d1e466640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f1d1815b030 0x7f1d180709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49216/0 (socket says 172.21.15.105:49216) 2023-11-28T18:24:23.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.258+0000 7f1d1e466640 1 -- 172.21.15.105:0/1745689170 learned_addr learned my addr 172.21.15.105:0/1745689170 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:23.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.258+0000 7f1d1d464640 1 --2- 172.21.15.105:0/1745689170 >> v2:172.21.15.148:3300/0 conn(0x7f1d18071960 0x7f1d1815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.258+0000 7f1d1cc63640 1 -- 172.21.15.105:0/1745689170 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1474869591 0 0) 0x7f1d180701d0 con 0x7f1d1815b030 2023-11-28T18:24:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.258+0000 7f1d1cc63640 1 -- 172.21.15.105:0/1745689170 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d04003530 con 0x7f1d1815b030 2023-11-28T18:24:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.258+0000 7f1d1cc63640 1 -- 172.21.15.105:0/1745689170 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4255430851 0 0) 0x7f1d1806df80 con 0x7f1d180b4290 2023-11-28T18:24:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.258+0000 7f1d1cc63640 1 -- 172.21.15.105:0/1745689170 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d180701d0 con 0x7f1d180b4290 2023-11-28T18:24:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.258+0000 7f1d1d464640 1 -- 172.21.15.105:0/1745689170 >> v1:172.21.15.148:6789/0 conn(0x7f1d180b4290 legacy=0x7f1d180b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.258+0000 7f1d1d464640 1 -- 172.21.15.105:0/1745689170 >> v1:172.21.15.105:6789/0 conn(0x7f1d1815b030 legacy=0x7f1d180709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.258+0000 7f1d1cc63640 1 -- 172.21.15.105:0/1745689170 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2090117257 0 0) 0x7f1d04003530 con 0x7f1d1815b030 2023-11-28T18:24:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.258+0000 7f1d1d464640 1 -- 172.21.15.105:0/1745689170 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d180720d0 con 0x7f1d18071960 2023-11-28T18:24:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.262+0000 7f1d1d464640 1 --2- 172.21.15.105:0/1745689170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d18071960 0x7f1d1815f3b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1d080097c0 tx=0x7f1d0801c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdca054a5d1e0a9d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.262+0000 7f1d1cc63640 1 -- 172.21.15.105:0/1745689170 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d10001a00 con 0x7f1d18071960 2023-11-28T18:24:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.262+0000 7f1d1cc63640 1 -- 172.21.15.105:0/1745689170 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d080a7050 con 0x7f1d18071960 2023-11-28T18:24:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.262+0000 7f1d1cc63640 1 -- 172.21.15.105:0/1745689170 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d080ab620 con 0x7f1d18071960 2023-11-28T18:24:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.262+0000 7f1d1fec9640 1 -- 172.21.15.105:0/1745689170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d18071960 msgr2=0x7f1d1815f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.262+0000 7f1d1fec9640 1 --2- 172.21.15.105:0/1745689170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d18071960 0x7f1d1815f3b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1d080097c0 tx=0x7f1d0801c370 comp rx=0 tx=0).stop 2023-11-28T18:24:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.262+0000 7f1d1fec9640 1 -- 172.21.15.105:0/1745689170 shutdown_connections 2023-11-28T18:24:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.262+0000 7f1d1fec9640 1 --2- 172.21.15.105:0/1745689170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d18071960 0x7f1d1815f3b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.262+0000 7f1d1fec9640 1 -- 172.21.15.105:0/1745689170 >> 172.21.15.105:0/1745689170 conn(0x7f1d180ceb90 msgr2=0x7f1d180cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.262+0000 7f1d1fec9640 1 -- 172.21.15.105:0/1745689170 shutdown_connections 2023-11-28T18:24:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.262+0000 7f1d1fec9640 1 -- 172.21.15.105:0/1745689170 wait complete. 2023-11-28T18:24:23.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d1fec9640 1 Processor -- start 2023-11-28T18:24:23.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d1fec9640 1 -- start start 2023-11-28T18:24:23.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d1fec9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d180b4290 0x7f1d18166380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:23.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d1fec9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d1815b030 0x7f1d180b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:23.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d1fec9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d1814c490 0x7f1d180cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:23.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d1fec9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d180cc900 con 0x7f1d1814c490 2023-11-28T18:24:23.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d1fec9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d180b4070 con 0x7f1d1815b030 2023-11-28T18:24:23.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d1fec9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d180b37d0 con 0x7f1d180b4290 2023-11-28T18:24:23.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d1e466640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d1814c490 0x7f1d180cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:23.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d1e466640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d1814c490 0x7f1d180cb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43486/0 (socket says 172.21.15.105:43486) 2023-11-28T18:24:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d1d464640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d1815b030 0x7f1d180b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d1e466640 1 -- 172.21.15.105:0/3185607670 learned_addr learned my addr 172.21.15.105:0/3185607670 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d1dc65640 1 --2- 172.21.15.105:0/3185607670 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d180b4290 0x7f1d18166380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d1e466640 1 -- 172.21.15.105:0/3185607670 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d180b4290 msgr2=0x7f1d18166380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d1e466640 1 --2- 172.21.15.105:0/3185607670 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d180b4290 0x7f1d18166380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d1e466640 1 -- 172.21.15.105:0/3185607670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d1815b030 msgr2=0x7f1d180b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d1e466640 1 --2- 172.21.15.105:0/3185607670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d1815b030 0x7f1d180b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d1e466640 1 -- 172.21.15.105:0/3185607670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d08009420 con 0x7f1d1814c490 2023-11-28T18:24:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d1e466640 1 --2- 172.21.15.105:0/3185607670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d1814c490 0x7f1d180cb7a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1d10005c10 tx=0x7f1d1000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d0e7fc640 1 -- 172.21.15.105:0/3185607670 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1000ce20 con 0x7f1d1814c490 2023-11-28T18:24:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d0e7fc640 1 -- 172.21.15.105:0/3185607670 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d10012400 con 0x7f1d1814c490 2023-11-28T18:24:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d1fec9640 1 -- 172.21.15.105:0/3185607670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d180b3ab0 con 0x7f1d1814c490 2023-11-28T18:24:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d0e7fc640 1 -- 172.21.15.105:0/3185607670 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d10012830 con 0x7f1d1814c490 2023-11-28T18:24:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d1fec9640 1 -- 172.21.15.105:0/3185607670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d181668c0 con 0x7f1d1814c490 2023-11-28T18:24:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.266+0000 7f1d1fec9640 1 -- 172.21.15.105:0/3185607670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d180cca40 con 0x7f1d1814c490 2023-11-28T18:24:23.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.270+0000 7f1d0e7fc640 1 -- 172.21.15.105:0/3185607670 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f1d10012990 con 0x7f1d1814c490 2023-11-28T18:24:23.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.270+0000 7f1d0e7fc640 1 --2- 172.21.15.105:0/3185607670 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1cfc0668d0 0x7f1cfc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:23.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.270+0000 7f1d0e7fc640 1 -- 172.21.15.105:0/3185607670 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f1d10084d70 con 0x7f1d1814c490 2023-11-28T18:24:23.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.270+0000 7f1d1dc65640 1 --2- 172.21.15.105:0/3185607670 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1cfc0668d0 0x7f1cfc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:23.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.270+0000 7f1d1dc65640 1 --2- 172.21.15.105:0/3185607670 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1cfc0668d0 0x7f1cfc068d90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1d140011f0 tx=0x7f1d14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:23.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.274+0000 7f1d0e7fc640 1 -- 172.21.15.105:0/3185607670 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1d180cca40 con 0x7f1d1814c490 2023-11-28T18:24:23.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.422+0000 7f1d1fec9640 1 -- 172.21.15.105:0/3185607670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1d180cca40 con 0x7f1d1814c490 2023-11-28T18:24:23.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.430+0000 7f1d0e7fc640 1 -- 172.21.15.105:0/3185607670 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1d180cca40 con 0x7f1d1814c490 2023-11-28T18:24:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.430+0000 7f1d1fec9640 1 -- 172.21.15.105:0/3185607670 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1cfc0668d0 msgr2=0x7f1cfc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.430+0000 7f1d1fec9640 1 --2- 172.21.15.105:0/3185607670 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1cfc0668d0 0x7f1cfc068d90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1d140011f0 tx=0x7f1d14009040 comp rx=0 tx=0).stop 2023-11-28T18:24:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.430+0000 7f1d1fec9640 1 -- 172.21.15.105:0/3185607670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d1814c490 msgr2=0x7f1d180cb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.430+0000 7f1d1fec9640 1 --2- 172.21.15.105:0/3185607670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d1814c490 0x7f1d180cb7a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1d10005c10 tx=0x7f1d1000a430 comp rx=0 tx=0).stop 2023-11-28T18:24:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.430+0000 7f1d1fec9640 1 -- 172.21.15.105:0/3185607670 shutdown_connections 2023-11-28T18:24:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.430+0000 7f1d1fec9640 1 --2- 172.21.15.105:0/3185607670 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1cfc0668d0 0x7f1cfc068d90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.430+0000 7f1d1fec9640 1 --2- 172.21.15.105:0/3185607670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d1814c490 0x7f1d180cb7a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.430+0000 7f1d1fec9640 1 --2- 172.21.15.105:0/3185607670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d1815b030 0x7f1d180b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:23.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.430+0000 7f1d1fec9640 1 --2- 172.21.15.105:0/3185607670 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d180b4290 0x7f1d18166380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:23.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.430+0000 7f1d1fec9640 1 -- 172.21.15.105:0/3185607670 >> 172.21.15.105:0/3185607670 conn(0x7f1d180ceb90 msgr2=0x7f1d180709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:23.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.430+0000 7f1d1fec9640 1 -- 172.21.15.105:0/3185607670 shutdown_connections 2023-11-28T18:24:23.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.430+0000 7f1d1fec9640 1 -- 172.21.15.105:0/3185607670 wait complete. 2023-11-28T18:24:23.456 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T18:24:23.456 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:24:23.520 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:24:23.609 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T18:24:23.609 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:24:23.609 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:24:23.618 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:24:23.618 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:24:23.674 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:24:23.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.850+0000 7f0d66f2a640 1 Processor -- start 2023-11-28T18:24:23.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.850+0000 7f0d66f2a640 1 -- start start 2023-11-28T18:24:23.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.850+0000 7f0d66f2a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0d6015b030 0x7f0d600709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:23.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.850+0000 7f0d66f2a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d600701d0 con 0x7f0d600b4290 2023-11-28T18:24:23.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.850+0000 7f0d66f2a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d6006df80 con 0x7f0d60071960 2023-11-28T18:24:23.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.850+0000 7f0d66f2a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d60071df0 con 0x7f0d6015b030 2023-11-28T18:24:23.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d64cc6640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0d600b4290 0x7f0d600b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49226/0 (socket says 172.21.15.105:49226) 2023-11-28T18:24:23.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d64cc6640 1 -- 172.21.15.105:0/496488425 learned_addr learned my addr 172.21.15.105:0/496488425 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:23.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d654c7640 1 --2- 172.21.15.105:0/496488425 >> v2:172.21.15.148:3300/0 conn(0x7f0d6015b030 0x7f0d600709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:23.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d5f7fe640 1 -- 172.21.15.105:0/496488425 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094201230 0 0) 0x7f0d600701d0 con 0x7f0d600b4290 2023-11-28T18:24:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d5f7fe640 1 -- 172.21.15.105:0/496488425 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d50003530 con 0x7f0d600b4290 2023-11-28T18:24:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d5f7fe640 1 -- 172.21.15.105:0/496488425 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1198636444 0 0) 0x7f0d50003530 con 0x7f0d600b4290 2023-11-28T18:24:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d5f7fe640 1 -- 172.21.15.105:0/496488425 >> v2:172.21.15.148:3300/0 conn(0x7f0d6015b030 msgr2=0x7f0d600709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d5f7fe640 1 --2- 172.21.15.105:0/496488425 >> v2:172.21.15.148:3300/0 conn(0x7f0d6015b030 0x7f0d600709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d5f7fe640 1 -- 172.21.15.105:0/496488425 >> v1:172.21.15.148:6789/0 conn(0x7f0d60071960 legacy=0x7f0d6015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d5f7fe640 1 -- 172.21.15.105:0/496488425 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d600720d0 con 0x7f0d600b4290 2023-11-28T18:24:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d5f7fe640 1 -- 172.21.15.105:0/496488425 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f0d54001680 con 0x7f0d600b4290 2023-11-28T18:24:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d5f7fe640 1 -- 172.21.15.105:0/496488425 >> v1:172.21.15.105:6789/0 conn(0x7f0d600b4290 legacy=0x7f0d600b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d5f7fe640 1 --2- 172.21.15.105:0/496488425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d50003980 0x7f0d50003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:23.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d5f7fe640 1 --2- 172.21.15.105:0/496488425 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d500047d0 0x7f0d50004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:23.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d5f7fe640 1 --2- 172.21.15.105:0/496488425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d50005190 0x7f0d5000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:23.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d5f7fe640 1 -- 172.21.15.105:0/496488425 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d60071df0 con 0x7f0d50005190 2023-11-28T18:24:23.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d5f7fe640 1 -- 172.21.15.105:0/496488425 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5000c030 con 0x7f0d50003980 2023-11-28T18:24:23.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d5f7fe640 1 -- 172.21.15.105:0/496488425 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5000c1f0 con 0x7f0d500047d0 2023-11-28T18:24:23.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d654c7640 1 --2- 172.21.15.105:0/496488425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d50005190 0x7f0d5000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:23.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d64cc6640 1 --2- 172.21.15.105:0/496488425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d50003980 0x7f0d50003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:23.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d5ffff640 1 --2- 172.21.15.105:0/496488425 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d500047d0 0x7f0d50004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:23.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d654c7640 1 -- 172.21.15.105:0/496488425 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d500047d0 msgr2=0x7f0d50004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:23.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d654c7640 1 --2- 172.21.15.105:0/496488425 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d500047d0 0x7f0d50004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:23.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d654c7640 1 -- 172.21.15.105:0/496488425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d50003980 msgr2=0x7f0d50003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:23.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d654c7640 1 --2- 172.21.15.105:0/496488425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d50003980 0x7f0d50003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:23.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d654c7640 1 -- 172.21.15.105:0/496488425 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d50003530 con 0x7f0d50005190 2023-11-28T18:24:23.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.854+0000 7f0d64cc6640 1 --2- 172.21.15.105:0/496488425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d50003980 0x7f0d50003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:23.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.858+0000 7f0d654c7640 1 -- 172.21.15.105:0/496488425 reap_dead start 2023-11-28T18:24:23.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.858+0000 7f0d654c7640 1 --2- 172.21.15.105:0/496488425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d50005190 0x7f0d5000b990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0d6015b440 tx=0x7f0d5801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd03c5769125c879 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:23.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.858+0000 7f0d5f7fe640 1 -- 172.21.15.105:0/496488425 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d580a5c50 con 0x7f0d50005190 2023-11-28T18:24:23.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.858+0000 7f0d5f7fe640 1 -- 172.21.15.105:0/496488425 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5800f040 con 0x7f0d50005190 2023-11-28T18:24:23.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.858+0000 7f0d5f7fe640 1 -- 172.21.15.105:0/496488425 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d580039a0 con 0x7f0d50005190 2023-11-28T18:24:23.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.858+0000 7f0d66f2a640 1 -- 172.21.15.105:0/496488425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d50005190 msgr2=0x7f0d5000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:23.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.858+0000 7f0d66f2a640 1 --2- 172.21.15.105:0/496488425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d50005190 0x7f0d5000b990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0d6015b440 tx=0x7f0d5801cc40 comp rx=0 tx=0).stop 2023-11-28T18:24:23.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.858+0000 7f0d66f2a640 1 -- 172.21.15.105:0/496488425 shutdown_connections 2023-11-28T18:24:23.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.858+0000 7f0d66f2a640 1 --2- 172.21.15.105:0/496488425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d50005190 0x7f0d5000b990 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:23.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.858+0000 7f0d66f2a640 1 -- 172.21.15.105:0/496488425 >> 172.21.15.105:0/496488425 conn(0x7f0d600ceb90 msgr2=0x7f0d600cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:23.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.858+0000 7f0d66f2a640 1 -- 172.21.15.105:0/496488425 shutdown_connections 2023-11-28T18:24:23.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.858+0000 7f0d66f2a640 1 -- 172.21.15.105:0/496488425 wait complete. 2023-11-28T18:24:23.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.858+0000 7f0d66f2a640 1 Processor -- start 2023-11-28T18:24:23.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.858+0000 7f0d66f2a640 1 -- start start 2023-11-28T18:24:23.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d66f2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d50005190 0x7f0d601555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:23.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d66f2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d60159d70 0x7f0d6015a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:23.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d66f2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d600ad5d0 0x7f0d600ada50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:23.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d66f2a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d60155b10 con 0x7f0d60159d70 2023-11-28T18:24:23.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d66f2a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6015a730 con 0x7f0d50005190 2023-11-28T18:24:23.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d66f2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d600cb9c0 con 0x7f0d600ad5d0 2023-11-28T18:24:23.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d5ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d60159d70 0x7f0d6015a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:23.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d5ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d60159d70 0x7f0d6015a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43504/0 (socket says 172.21.15.105:43504) 2023-11-28T18:24:23.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d5ffff640 1 -- 172.21.15.105:0/1631561623 learned_addr learned my addr 172.21.15.105:0/1631561623 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:23.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d64cc6640 1 --2- 172.21.15.105:0/1631561623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d50005190 0x7f0d601555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:23.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d654c7640 1 --2- 172.21.15.105:0/1631561623 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d600ad5d0 0x7f0d600ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:23.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d5ffff640 1 -- 172.21.15.105:0/1631561623 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d600ad5d0 msgr2=0x7f0d600ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:23.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d5ffff640 1 --2- 172.21.15.105:0/1631561623 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d600ad5d0 0x7f0d600ada50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:23.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d5ffff640 1 -- 172.21.15.105:0/1631561623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d50005190 msgr2=0x7f0d601555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:23.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d5ffff640 1 --2- 172.21.15.105:0/1631561623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d50005190 0x7f0d601555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:23.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d5ffff640 1 -- 172.21.15.105:0/1631561623 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d600ae330 con 0x7f0d60159d70 2023-11-28T18:24:23.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d5ffff640 1 --2- 172.21.15.105:0/1631561623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d60159d70 0x7f0d6015a1f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00e3f0 tx=0x7f0d4c00e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:23.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d5d7fa640 1 -- 172.21.15.105:0/1631561623 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c0052e0 con 0x7f0d60159d70 2023-11-28T18:24:23.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d5d7fa640 1 -- 172.21.15.105:0/1631561623 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4c005440 con 0x7f0d60159d70 2023-11-28T18:24:23.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d66f2a640 1 -- 172.21.15.105:0/1631561623 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d600ae610 con 0x7f0d60159d70 2023-11-28T18:24:23.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d5d7fa640 1 -- 172.21.15.105:0/1631561623 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c003820 con 0x7f0d60159d70 2023-11-28T18:24:23.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d66f2a640 1 -- 172.21.15.105:0/1631561623 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d600c82c0 con 0x7f0d60159d70 2023-11-28T18:24:23.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d66f2a640 1 -- 172.21.15.105:0/1631561623 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d60067570 con 0x7f0d60159d70 2023-11-28T18:24:23.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.862+0000 7f0d5d7fa640 1 -- 172.21.15.105:0/1631561623 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f0d4c003980 con 0x7f0d60159d70 2023-11-28T18:24:23.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.866+0000 7f0d5d7fa640 1 --2- 172.21.15.105:0/1631561623 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0d440668d0 0x7f0d44068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:23.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.866+0000 7f0d5d7fa640 1 -- 172.21.15.105:0/1631561623 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f0d4c083ff0 con 0x7f0d60159d70 2023-11-28T18:24:23.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.866+0000 7f0d64cc6640 1 --2- 172.21.15.105:0/1631561623 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0d440668d0 0x7f0d44068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:23.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.866+0000 7f0d64cc6640 1 --2- 172.21.15.105:0/1631561623 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0d440668d0 0x7f0d44068d90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0d54001000 tx=0x7f0d54007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:23.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:23.870+0000 7f0d5d7fa640 1 -- 172.21.15.105:0/1631561623 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0d4c0b5280 con 0x7f0d60159d70 2023-11-28T18:24:24.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.018+0000 7f0d66f2a640 1 -- 172.21.15.105:0/1631561623 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d600cf7f0 con 0x7f0d60159d70 2023-11-28T18:24:24.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.018+0000 7f0d5d7fa640 1 -- 172.21.15.105:0/1631561623 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18887 (secure 0 0 0) 0x7f0d4c019020 con 0x7f0d60159d70 2023-11-28T18:24:24.025 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:24.026 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:24:15.048168+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:24:24.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.022+0000 7f0d66f2a640 1 -- 172.21.15.105:0/1631561623 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0d440668d0 msgr2=0x7f0d44068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:24.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.022+0000 7f0d66f2a640 1 --2- 172.21.15.105:0/1631561623 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0d440668d0 0x7f0d44068d90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0d54001000 tx=0x7f0d54007040 comp rx=0 tx=0).stop 2023-11-28T18:24:24.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.022+0000 7f0d66f2a640 1 -- 172.21.15.105:0/1631561623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d60159d70 msgr2=0x7f0d6015a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:24.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.022+0000 7f0d66f2a640 1 --2- 172.21.15.105:0/1631561623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d60159d70 0x7f0d6015a1f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00e3f0 tx=0x7f0d4c00e8b0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.022+0000 7f0d66f2a640 1 -- 172.21.15.105:0/1631561623 shutdown_connections 2023-11-28T18:24:24.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.022+0000 7f0d66f2a640 1 --2- 172.21.15.105:0/1631561623 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d600ad5d0 0x7f0d600ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.022+0000 7f0d66f2a640 1 --2- 172.21.15.105:0/1631561623 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0d440668d0 0x7f0d44068d90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.022+0000 7f0d66f2a640 1 --2- 172.21.15.105:0/1631561623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d60159d70 0x7f0d6015a1f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.022+0000 7f0d66f2a640 1 --2- 172.21.15.105:0/1631561623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d50005190 0x7f0d601555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.022+0000 7f0d66f2a640 1 -- 172.21.15.105:0/1631561623 >> 172.21.15.105:0/1631561623 conn(0x7f0d600ceb90 msgr2=0x7f0d6009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:24.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.022+0000 7f0d66f2a640 1 -- 172.21.15.105:0/1631561623 shutdown_connections 2023-11-28T18:24:24.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.022+0000 7f0d66f2a640 1 -- 172.21.15.105:0/1631561623 wait complete. 2023-11-28T18:24:24.046 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:23:31.485422+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:23:32.274378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T18:23:36.100071+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T18:24:24.047 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:24:24.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.182+0000 7f7c7eeea640 1 Processor -- start 2023-11-28T18:24:24.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.182+0000 7f7c7eeea640 1 -- start start 2023-11-28T18:24:24.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.186+0000 7f7c7eeea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c78159610 0x7f7c780ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:24.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.186+0000 7f7c7eeea640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7c7806cef0 0x7f7c780ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:24.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.186+0000 7f7c7eeea640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c780cde30 con 0x7f7c780cc760 2023-11-28T18:24:24.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.186+0000 7f7c7eeea640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c781599f0 con 0x7f7c7806cef0 2023-11-28T18:24:24.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.186+0000 7f7c7eeea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7806de20 con 0x7f7c78159610 2023-11-28T18:24:24.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.186+0000 7f7c7d487640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7c7806cef0 0x7f7c780ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.186+0000 7f7c7d487640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7c7806cef0 0x7f7c780ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43520/0 (socket says 172.21.15.105:43520) 2023-11-28T18:24:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.186+0000 7f7c7d487640 1 -- 172.21.15.105:0/737208307 learned_addr learned my addr 172.21.15.105:0/737208307 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.186+0000 7f7c7cc86640 1 --2- 172.21.15.105:0/737208307 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c78159610 0x7f7c780ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.186+0000 7f7c7d487640 1 -- 172.21.15.105:0/737208307 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c78159610 msgr2=0x7f7c780ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.186+0000 7f7c7d487640 1 --2- 172.21.15.105:0/737208307 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c78159610 0x7f7c780ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.186+0000 7f7c7d487640 1 -- 172.21.15.105:0/737208307 >> v1:172.21.15.148:6789/0 conn(0x7f7c780cc760 legacy=0x7f7c7806c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.186+0000 7f7c7d487640 1 -- 172.21.15.105:0/737208307 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c780cd220 con 0x7f7c7806cef0 2023-11-28T18:24:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.190+0000 7f7c7d487640 1 --2- 172.21.15.105:0/737208307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c7806cef0 0x7f7c780ccbc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009e50 tx=0x7f7c7001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13bb5d869536f22 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.190+0000 7f7c777fe640 1 -- 172.21.15.105:0/737208307 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c700a5820 con 0x7f7c7806cef0 2023-11-28T18:24:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.190+0000 7f7c777fe640 1 -- 172.21.15.105:0/737208307 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c700a5980 con 0x7f7c7806cef0 2023-11-28T18:24:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.190+0000 7f7c777fe640 1 -- 172.21.15.105:0/737208307 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c700a5d10 con 0x7f7c7806cef0 2023-11-28T18:24:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.190+0000 7f7c7eeea640 1 -- 172.21.15.105:0/737208307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c7806cef0 msgr2=0x7f7c780ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.190+0000 7f7c7eeea640 1 --2- 172.21.15.105:0/737208307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c7806cef0 0x7f7c780ccbc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009e50 tx=0x7f7c7001c9e0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.190+0000 7f7c7eeea640 1 -- 172.21.15.105:0/737208307 shutdown_connections 2023-11-28T18:24:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.190+0000 7f7c7eeea640 1 --2- 172.21.15.105:0/737208307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c7806cef0 0x7f7c780ccbc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.190+0000 7f7c7eeea640 1 --2- 172.21.15.105:0/737208307 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c78159610 0x7f7c780ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.190+0000 7f7c7eeea640 1 -- 172.21.15.105:0/737208307 >> 172.21.15.105:0/737208307 conn(0x7f7c780ceb90 msgr2=0x7f7c780cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.190+0000 7f7c7eeea640 1 -- 172.21.15.105:0/737208307 shutdown_connections 2023-11-28T18:24:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.190+0000 7f7c7eeea640 1 -- 172.21.15.105:0/737208307 wait complete. 2023-11-28T18:24:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.190+0000 7f7c7eeea640 1 Processor -- start 2023-11-28T18:24:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.190+0000 7f7c7eeea640 1 -- start start 2023-11-28T18:24:24.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c7eeea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c7806cef0 0x7f7c7814b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:24.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c7eeea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c780cc760 0x7f7c7814afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:24.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c7eeea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c78159610 0x7f7c7814beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:24.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c7eeea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7814b520 con 0x7f7c78159610 2023-11-28T18:24:24.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c7eeea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c780ac820 con 0x7f7c7806cef0 2023-11-28T18:24:24.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c7eeea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c780acb00 con 0x7f7c780cc760 2023-11-28T18:24:24.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c7d487640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c78159610 0x7f7c7814beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:24.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c7cc86640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c7806cef0 0x7f7c7814b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:24.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c7d487640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c78159610 0x7f7c7814beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43534/0 (socket says 172.21.15.105:43534) 2023-11-28T18:24:24.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c7d487640 1 -- 172.21.15.105:0/3869675423 learned_addr learned my addr 172.21.15.105:0/3869675423 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:24.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c77fff640 1 --2- 172.21.15.105:0/3869675423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c780cc760 0x7f7c7814afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:24.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c7d487640 1 -- 172.21.15.105:0/3869675423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c780cc760 msgr2=0x7f7c7814afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:24.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c7d487640 1 --2- 172.21.15.105:0/3869675423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c780cc760 0x7f7c7814afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c7d487640 1 -- 172.21.15.105:0/3869675423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c7806cef0 msgr2=0x7f7c7814b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:24.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c7d487640 1 --2- 172.21.15.105:0/3869675423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c7806cef0 0x7f7c7814b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c7d487640 1 -- 172.21.15.105:0/3869675423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c70009b30 con 0x7f7c78159610 2023-11-28T18:24:24.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c7d487640 1 --2- 172.21.15.105:0/3869675423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c78159610 0x7f7c7814beb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7c700af040 tx=0x7f7c70019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:24.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c757fa640 1 -- 172.21.15.105:0/3869675423 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c700b8670 con 0x7f7c78159610 2023-11-28T18:24:24.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c757fa640 1 -- 172.21.15.105:0/3869675423 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c700b8810 con 0x7f7c78159610 2023-11-28T18:24:24.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c7eeea640 1 -- 172.21.15.105:0/3869675423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c780acde0 con 0x7f7c78159610 2023-11-28T18:24:24.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c757fa640 1 -- 172.21.15.105:0/3869675423 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c700b8bc0 con 0x7f7c78159610 2023-11-28T18:24:24.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c7eeea640 1 -- 172.21.15.105:0/3869675423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c780ad3a0 con 0x7f7c78159610 2023-11-28T18:24:24.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.194+0000 7f7c7eeea640 1 -- 172.21.15.105:0/3869675423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c780ad590 con 0x7f7c78159610 2023-11-28T18:24:24.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.198+0000 7f7c757fa640 1 -- 172.21.15.105:0/3869675423 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f7c700b8d20 con 0x7f7c78159610 2023-11-28T18:24:24.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.198+0000 7f7c757fa640 1 --2- 172.21.15.105:0/3869675423 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7c5c0668d0 0x7f7c5c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:24.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.198+0000 7f7c757fa640 1 -- 172.21.15.105:0/3869675423 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f7c7011fe00 con 0x7f7c78159610 2023-11-28T18:24:24.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.198+0000 7f7c7cc86640 1 --2- 172.21.15.105:0/3869675423 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7c5c0668d0 0x7f7c5c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:24.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.198+0000 7f7c7cc86640 1 --2- 172.21.15.105:0/3869675423 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7c5c0668d0 0x7f7c5c068d90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c0079c0 tx=0x7f7c6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:24.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.202+0000 7f7c757fa640 1 -- 172.21.15.105:0/3869675423 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7c700ecd70 con 0x7f7c78159610 2023-11-28T18:24:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.354+0000 7f7c7eeea640 1 -- 172.21.15.105:0/3869675423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7c780ad590 con 0x7f7c78159610 2023-11-28T18:24:24.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.354+0000 7f7c757fa640 1 -- 172.21.15.105:0/3869675423 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7c700f0390 con 0x7f7c78159610 2023-11-28T18:24:24.359 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:24:24.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.354+0000 7f7c7eeea640 1 -- 172.21.15.105:0/3869675423 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7c5c0668d0 msgr2=0x7f7c5c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:24.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.354+0000 7f7c7eeea640 1 --2- 172.21.15.105:0/3869675423 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7c5c0668d0 0x7f7c5c068d90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c0079c0 tx=0x7f7c6c008040 comp rx=0 tx=0).stop 2023-11-28T18:24:24.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.354+0000 7f7c7eeea640 1 -- 172.21.15.105:0/3869675423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c78159610 msgr2=0x7f7c7814beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:24.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.354+0000 7f7c7eeea640 1 --2- 172.21.15.105:0/3869675423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c78159610 0x7f7c7814beb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7c700af040 tx=0x7f7c70019780 comp rx=0 tx=0).stop 2023-11-28T18:24:24.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.354+0000 7f7c7eeea640 1 -- 172.21.15.105:0/3869675423 shutdown_connections 2023-11-28T18:24:24.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.354+0000 7f7c7eeea640 1 --2- 172.21.15.105:0/3869675423 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7c5c0668d0 0x7f7c5c068d90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.354+0000 7f7c7eeea640 1 --2- 172.21.15.105:0/3869675423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c78159610 0x7f7c7814beb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.354+0000 7f7c7eeea640 1 --2- 172.21.15.105:0/3869675423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c780cc760 0x7f7c7814afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.354+0000 7f7c7eeea640 1 --2- 172.21.15.105:0/3869675423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c7806cef0 0x7f7c7814b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.354+0000 7f7c7eeea640 1 -- 172.21.15.105:0/3869675423 >> 172.21.15.105:0/3869675423 conn(0x7f7c780ceb90 msgr2=0x7f7c78151e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:24.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.354+0000 7f7c7eeea640 1 -- 172.21.15.105:0/3869675423 shutdown_connections 2023-11-28T18:24:24.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.354+0000 7f7c7eeea640 1 -- 172.21.15.105:0/3869675423 wait complete. 2023-11-28T18:24:24.380 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:24:24.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.514+0000 7fefd2d10640 1 Processor -- start 2023-11-28T18:24:24.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.514+0000 7fefd2d10640 1 -- start start 2023-11-28T18:24:24.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd2d10640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc14adb0 0x7fefcc0cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:24.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd2d10640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefcc070010 con 0x7fefcc165450 2023-11-28T18:24:24.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd2d10640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefcc06c070 con 0x7fefcc164fc0 2023-11-28T18:24:24.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd2d10640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc0cb2f0 con 0x7fefcc14adb0 2023-11-28T18:24:24.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd250f640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fefcc165450 0x7fefcc0ca040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49232/0 (socket says 172.21.15.105:49232) 2023-11-28T18:24:24.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd250f640 1 -- 172.21.15.105:0/448051271 learned_addr learned my addr 172.21.15.105:0/448051271 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:24.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd1d0e640 1 --2- 172.21.15.105:0/448051271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc14adb0 0x7fefcc0cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:24.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd0d0c640 1 -- 172.21.15.105:0/448051271 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473441906 0 0) 0x7fefcc070010 con 0x7fefcc165450 2023-11-28T18:24:24.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd0d0c640 1 -- 172.21.15.105:0/448051271 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefb8003530 con 0x7fefcc165450 2023-11-28T18:24:24.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd0d0c640 1 -- 172.21.15.105:0/448051271 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3287430926 0 0) 0x7fefcc06c070 con 0x7fefcc164fc0 2023-11-28T18:24:24.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd0d0c640 1 -- 172.21.15.105:0/448051271 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefcc070010 con 0x7fefcc164fc0 2023-11-28T18:24:24.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd0d0c640 1 -- 172.21.15.105:0/448051271 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2927000084 0 0) 0x7fefb8003530 con 0x7fefcc165450 2023-11-28T18:24:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd0d0c640 1 -- 172.21.15.105:0/448051271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc14adb0 msgr2=0x7fefcc0cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd0d0c640 1 --2- 172.21.15.105:0/448051271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc14adb0 0x7fefcc0cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd0d0c640 1 -- 172.21.15.105:0/448051271 >> v1:172.21.15.148:6789/0 conn(0x7fefcc164fc0 legacy=0x7fefcc0cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd0d0c640 1 -- 172.21.15.105:0/448051271 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefcc14b190 con 0x7fefcc165450 2023-11-28T18:24:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd0d0c640 1 -- 172.21.15.105:0/448051271 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fefc4001a00 con 0x7fefcc165450 2023-11-28T18:24:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd0d0c640 1 -- 172.21.15.105:0/448051271 >> v1:172.21.15.105:6789/0 conn(0x7fefcc165450 legacy=0x7fefcc0ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd0d0c640 1 --2- 172.21.15.105:0/448051271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefb8003b40 0x7fefb8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd0d0c640 1 --2- 172.21.15.105:0/448051271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc14adb0 0x7fefb80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd0d0c640 1 --2- 172.21.15.105:0/448051271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefb8004d10 0x7fefb8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:24.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd0d0c640 1 -- 172.21.15.105:0/448051271 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc0cb2f0 con 0x7fefb8003b40 2023-11-28T18:24:24.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd0d0c640 1 -- 172.21.15.105:0/448051271 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb800ba40 con 0x7fefb8004d10 2023-11-28T18:24:24.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd0d0c640 1 -- 172.21.15.105:0/448051271 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb800bbb0 con 0x7fefcc14adb0 2023-11-28T18:24:24.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd1d0e640 1 --2- 172.21.15.105:0/448051271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefb8003b40 0x7fefb8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:24.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd250f640 1 --2- 172.21.15.105:0/448051271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefb8004d10 0x7fefb8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:24.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd150d640 1 --2- 172.21.15.105:0/448051271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc14adb0 0x7fefb80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:24.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd1d0e640 1 -- 172.21.15.105:0/448051271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc14adb0 msgr2=0x7fefb80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:24.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd1d0e640 1 --2- 172.21.15.105:0/448051271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc14adb0 0x7fefb80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd1d0e640 1 -- 172.21.15.105:0/448051271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefb8004d10 msgr2=0x7fefb8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:24.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd1d0e640 1 --2- 172.21.15.105:0/448051271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefb8004d10 0x7fefb8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd1d0e640 1 -- 172.21.15.105:0/448051271 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb8003530 con 0x7fefb8003b40 2023-11-28T18:24:24.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.518+0000 7fefd250f640 1 --2- 172.21.15.105:0/448051271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefb8004d10 0x7fefb8005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.522+0000 7fefd1d0e640 1 --2- 172.21.15.105:0/448051271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefb8003b40 0x7fefb8003f40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009ae0 tx=0x7fefc801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ee192617b368cf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.522+0000 7fefd0d0c640 1 -- 172.21.15.105:0/448051271 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc80a5530 con 0x7fefb8003b40 2023-11-28T18:24:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.522+0000 7fefd0d0c640 1 -- 172.21.15.105:0/448051271 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc80a5690 con 0x7fefb8003b40 2023-11-28T18:24:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.522+0000 7fefd0d0c640 1 -- 172.21.15.105:0/448051271 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc80a5a40 con 0x7fefb8003b40 2023-11-28T18:24:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.522+0000 7fefd2d10640 1 -- 172.21.15.105:0/448051271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefb8003b40 msgr2=0x7fefb8003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.522+0000 7fefd2d10640 1 --2- 172.21.15.105:0/448051271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefb8003b40 0x7fefb8003f40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009ae0 tx=0x7fefc801ca50 comp rx=0 tx=0).stop 2023-11-28T18:24:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.522+0000 7fefd250f640 1 -- 172.21.15.105:0/448051271 reap_dead start 2023-11-28T18:24:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.522+0000 7fefd2d10640 1 -- 172.21.15.105:0/448051271 shutdown_connections 2023-11-28T18:24:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.522+0000 7fefd2d10640 1 -- 172.21.15.105:0/448051271 >> 172.21.15.105:0/448051271 conn(0x7fefcc0ce9e0 msgr2=0x7fefcc0cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.522+0000 7fefd2d10640 1 -- 172.21.15.105:0/448051271 shutdown_connections 2023-11-28T18:24:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.522+0000 7fefd2d10640 1 -- 172.21.15.105:0/448051271 wait complete. 2023-11-28T18:24:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.522+0000 7fefd2d10640 1 Processor -- start 2023-11-28T18:24:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefd2d10640 1 -- start start 2023-11-28T18:24:24.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefd2d10640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefb8003b40 0x7fefcc160c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:24.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefd2d10640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc0b2e20 0x7fefcc144290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:24.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefd2d10640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefcc0c98b0 0x7fefcc0c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefd2d10640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc0ca270 con 0x7fefcc0c98b0 2023-11-28T18:24:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefd2d10640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc143e00 con 0x7fefb8003b40 2023-11-28T18:24:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefd2d10640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc1624d0 con 0x7fefcc0b2e20 2023-11-28T18:24:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefd250f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefcc0c98b0 0x7fefcc0c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefd250f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefcc0c98b0 0x7fefcc0c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43552/0 (socket says 172.21.15.105:43552) 2023-11-28T18:24:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefd150d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc0b2e20 0x7fefcc144290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefd250f640 1 -- 172.21.15.105:0/870535409 learned_addr learned my addr 172.21.15.105:0/870535409 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefd1d0e640 1 --2- 172.21.15.105:0/870535409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefb8003b40 0x7fefcc160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefd250f640 1 -- 172.21.15.105:0/870535409 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc0b2e20 msgr2=0x7fefcc144290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefd250f640 1 --2- 172.21.15.105:0/870535409 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc0b2e20 0x7fefcc144290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefd250f640 1 -- 172.21.15.105:0/870535409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefb8003b40 msgr2=0x7fefcc160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:24.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefd250f640 1 --2- 172.21.15.105:0/870535409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefb8003b40 0x7fefcc160c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefd250f640 1 -- 172.21.15.105:0/870535409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefcc1627b0 con 0x7fefcc0c98b0 2023-11-28T18:24:24.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefd250f640 1 --2- 172.21.15.105:0/870535409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefcc0c98b0 0x7fefcc0c9d30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fefcc165830 tx=0x7fefc400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:24.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefc27fc640 1 -- 172.21.15.105:0/870535409 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc4017070 con 0x7fefcc0c98b0 2023-11-28T18:24:24.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefc27fc640 1 -- 172.21.15.105:0/870535409 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc4007ae0 con 0x7fefcc0c98b0 2023-11-28T18:24:24.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefc27fc640 1 -- 172.21.15.105:0/870535409 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc400edc0 con 0x7fefcc0c98b0 2023-11-28T18:24:24.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefd2d10640 1 -- 172.21.15.105:0/870535409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefcc143a90 con 0x7fefcc0c98b0 2023-11-28T18:24:24.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefd2d10640 1 -- 172.21.15.105:0/870535409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefcc164b80 con 0x7fefcc0c98b0 2023-11-28T18:24:24.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.526+0000 7fefd2d10640 1 -- 172.21.15.105:0/870535409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefcc067e30 con 0x7fefcc0c98b0 2023-11-28T18:24:24.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.530+0000 7fefc27fc640 1 -- 172.21.15.105:0/870535409 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fefc4013420 con 0x7fefcc0c98b0 2023-11-28T18:24:24.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.530+0000 7fefc27fc640 1 --2- 172.21.15.105:0/870535409 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fefb0066900 0x7fefb0068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:24.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.530+0000 7fefc27fc640 1 -- 172.21.15.105:0/870535409 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fefc4084140 con 0x7fefcc0c98b0 2023-11-28T18:24:24.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.530+0000 7fefd1d0e640 1 --2- 172.21.15.105:0/870535409 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fefb0066900 0x7fefb0068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:24.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.530+0000 7fefd1d0e640 1 --2- 172.21.15.105:0/870535409 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fefb0066900 0x7fefb0068dc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fefc80af000 tx=0x7fefc80046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:24.539 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.534+0000 7fefc27fc640 1 -- 172.21.15.105:0/870535409 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fefcc067e30 con 0x7fefcc0c98b0 2023-11-28T18:24:24.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.682+0000 7fefd2d10640 1 -- 172.21.15.105:0/870535409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fefcc067e30 con 0x7fefcc0c98b0 2023-11-28T18:24:24.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.682+0000 7fefc27fc640 1 -- 172.21.15.105:0/870535409 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fefcc067e30 con 0x7fefcc0c98b0 2023-11-28T18:24:24.688 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:24.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.686+0000 7fefd2d10640 1 -- 172.21.15.105:0/870535409 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fefb0066900 msgr2=0x7fefb0068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:24.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.686+0000 7fefd2d10640 1 --2- 172.21.15.105:0/870535409 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fefb0066900 0x7fefb0068dc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fefc80af000 tx=0x7fefc80046d0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.686+0000 7fefd2d10640 1 -- 172.21.15.105:0/870535409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefcc0c98b0 msgr2=0x7fefcc0c9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:24.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.686+0000 7fefd2d10640 1 --2- 172.21.15.105:0/870535409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefcc0c98b0 0x7fefcc0c9d30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fefcc165830 tx=0x7fefc400e530 comp rx=0 tx=0).stop 2023-11-28T18:24:24.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.686+0000 7fefd2d10640 1 -- 172.21.15.105:0/870535409 shutdown_connections 2023-11-28T18:24:24.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.686+0000 7fefd2d10640 1 --2- 172.21.15.105:0/870535409 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fefb0066900 0x7fefb0068dc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.686+0000 7fefd2d10640 1 --2- 172.21.15.105:0/870535409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefcc0c98b0 0x7fefcc0c9d30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.686+0000 7fefd2d10640 1 --2- 172.21.15.105:0/870535409 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefcc0b2e20 0x7fefcc144290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.686+0000 7fefd2d10640 1 --2- 172.21.15.105:0/870535409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefb8003b40 0x7fefcc160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.686+0000 7fefd2d10640 1 -- 172.21.15.105:0/870535409 >> 172.21.15.105:0/870535409 conn(0x7fefcc0ce9e0 msgr2=0x7fefcc0aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:24.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.686+0000 7fefd2d10640 1 -- 172.21.15.105:0/870535409 shutdown_connections 2023-11-28T18:24:24.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.686+0000 7fefd2d10640 1 -- 172.21.15.105:0/870535409 wait complete. 2023-11-28T18:24:24.709 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:24:24.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.842+0000 7f52a8fd1640 1 Processor -- start 2023-11-28T18:24:24.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.842+0000 7f52a8fd1640 1 -- start start 2023-11-28T18:24:24.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.846+0000 7f52a8fd1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f52a4159610 0x7f52a40ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:24.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.846+0000 7f52a8fd1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f52a40cc760 0x7f52a40ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:24.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.846+0000 7f52a8fd1640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52a40cde30 con 0x7f52a406c7e0 2023-11-28T18:24:24.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.846+0000 7f52a8fd1640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a41599f0 con 0x7f52a4159610 2023-11-28T18:24:24.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.846+0000 7f52a8fd1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a406de20 con 0x7f52a40cc760 2023-11-28T18:24:24.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.846+0000 7f52a259c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f52a4159610 0x7f52a40ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:24.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.846+0000 7f52a2d9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f52a406c7e0 0x7f52a40abd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49242/0 (socket says 172.21.15.105:49242) 2023-11-28T18:24:24.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.846+0000 7f52a2d9d640 1 -- 172.21.15.105:0/2461080830 learned_addr learned my addr 172.21.15.105:0/2461080830 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:24.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.846+0000 7f5299d9b640 1 --2- 172.21.15.105:0/2461080830 >> v2:172.21.15.148:3300/0 conn(0x7f52a40cc760 0x7f52a40ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:24.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.846+0000 7f52a259c640 1 -- 172.21.15.105:0/2461080830 >> v2:172.21.15.148:3300/0 conn(0x7f52a40cc760 msgr2=0x7f52a40ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:24.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.846+0000 7f52a259c640 1 --2- 172.21.15.105:0/2461080830 >> v2:172.21.15.148:3300/0 conn(0x7f52a40cc760 0x7f52a40ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.846+0000 7f52a259c640 1 -- 172.21.15.105:0/2461080830 >> v1:172.21.15.105:6789/0 conn(0x7f52a406c7e0 legacy=0x7f52a40abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:24.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.846+0000 7f52a259c640 1 -- 172.21.15.105:0/2461080830 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a40cd220 con 0x7f52a4159610 2023-11-28T18:24:24.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.846+0000 7f52a1d9b640 1 -- 172.21.15.105:0/2461080830 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 563655961 0 0) 0x7f52a40cde30 con 0x7f52a406c7e0 2023-11-28T18:24:24.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.850+0000 7f52a259c640 1 --2- 172.21.15.105:0/2461080830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52a4159610 0x7f52a40ce2c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f52a40cfc90 tx=0x7f52a40acca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cace51f13395632e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:24.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.850+0000 7f52a1d9b640 1 -- 172.21.15.105:0/2461080830 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52940a59a0 con 0x7f52a4159610 2023-11-28T18:24:24.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.850+0000 7f52a1d9b640 1 -- 172.21.15.105:0/2461080830 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52940a5b00 con 0x7f52a4159610 2023-11-28T18:24:24.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.850+0000 7f52a1d9b640 1 -- 172.21.15.105:0/2461080830 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52940a5e90 con 0x7f52a4159610 2023-11-28T18:24:24.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.850+0000 7f52a8fd1640 1 -- 172.21.15.105:0/2461080830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52a4159610 msgr2=0x7f52a40ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:24.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.850+0000 7f52a8fd1640 1 --2- 172.21.15.105:0/2461080830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52a4159610 0x7f52a40ce2c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f52a40cfc90 tx=0x7f52a40acca0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.850+0000 7f52a8fd1640 1 -- 172.21.15.105:0/2461080830 shutdown_connections 2023-11-28T18:24:24.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.850+0000 7f52a8fd1640 1 --2- 172.21.15.105:0/2461080830 >> v2:172.21.15.148:3300/0 conn(0x7f52a40cc760 0x7f52a40ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.850+0000 7f52a8fd1640 1 --2- 172.21.15.105:0/2461080830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52a4159610 0x7f52a40ce2c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.850+0000 7f52a8fd1640 1 -- 172.21.15.105:0/2461080830 >> 172.21.15.105:0/2461080830 conn(0x7f52a40ceb90 msgr2=0x7f52a40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:24.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.850+0000 7f52a8fd1640 1 -- 172.21.15.105:0/2461080830 shutdown_connections 2023-11-28T18:24:24.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.850+0000 7f52a8fd1640 1 -- 172.21.15.105:0/2461080830 wait complete. 2023-11-28T18:24:24.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.850+0000 7f52a8fd1640 1 Processor -- start 2023-11-28T18:24:24.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.850+0000 7f52a8fd1640 1 -- start start 2023-11-28T18:24:24.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f52a8fd1640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52a406c7e0 0x7f52a409c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:24.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f52a8fd1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52a40cc760 0x7f52a4157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:24.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f52a8fd1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52a4159610 0x7f52a41583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:24.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f52a8fd1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a4159cd0 con 0x7f52a4159610 2023-11-28T18:24:24.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f52a8fd1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a4159fb0 con 0x7f52a40cc760 2023-11-28T18:24:24.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f52a8fd1640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a415a290 con 0x7f52a406c7e0 2023-11-28T18:24:24.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f52a2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52a4159610 0x7f52a41583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f52a2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52a4159610 0x7f52a41583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43580/0 (socket says 172.21.15.105:43580) 2023-11-28T18:24:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f52a2d9d640 1 -- 172.21.15.105:0/452974184 learned_addr learned my addr 172.21.15.105:0/452974184 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f52a259c640 1 --2- 172.21.15.105:0/452974184 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52a406c7e0 0x7f52a409c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f5299d9b640 1 --2- 172.21.15.105:0/452974184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52a40cc760 0x7f52a4157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f52a2d9d640 1 -- 172.21.15.105:0/452974184 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52a406c7e0 msgr2=0x7f52a409c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f52a2d9d640 1 --2- 172.21.15.105:0/452974184 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52a406c7e0 0x7f52a409c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f52a2d9d640 1 -- 172.21.15.105:0/452974184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52a40cc760 msgr2=0x7f52a4157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f52a2d9d640 1 --2- 172.21.15.105:0/452974184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52a40cc760 0x7f52a4157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f52a2d9d640 1 -- 172.21.15.105:0/452974184 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5294009a70 con 0x7f52a4159610 2023-11-28T18:24:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f5299d9b640 1 --2- 172.21.15.105:0/452974184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52a40cc760 0x7f52a4157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:24:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f52a2d9d640 1 --2- 172.21.15.105:0/452974184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52a4159610 0x7f52a41583b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f529c00c470 tx=0x7f529c00c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:24.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f529b7fe640 1 -- 172.21.15.105:0/452974184 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c0074e0 con 0x7f52a4159610 2023-11-28T18:24:24.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f529b7fe640 1 -- 172.21.15.105:0/452974184 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529c007640 con 0x7f52a4159610 2023-11-28T18:24:24.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f52a8fd1640 1 -- 172.21.15.105:0/452974184 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a415a570 con 0x7f52a4159610 2023-11-28T18:24:24.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f529b7fe640 1 -- 172.21.15.105:0/452974184 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c004a30 con 0x7f52a4159610 2023-11-28T18:24:24.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f52a8fd1640 1 -- 172.21.15.105:0/452974184 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a409c5d0 con 0x7f52a4159610 2023-11-28T18:24:24.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.854+0000 7f52a8fd1640 1 -- 172.21.15.105:0/452974184 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52a4067570 con 0x7f52a4159610 2023-11-28T18:24:24.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.858+0000 7f529b7fe640 1 -- 172.21.15.105:0/452974184 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f529c004b90 con 0x7f52a4159610 2023-11-28T18:24:24.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.858+0000 7f529b7fe640 1 --2- 172.21.15.105:0/452974184 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52840668d0 0x7f5284068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:24.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.858+0000 7f529b7fe640 1 -- 172.21.15.105:0/452974184 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f529c083ca0 con 0x7f52a4159610 2023-11-28T18:24:24.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.858+0000 7f52a259c640 1 --2- 172.21.15.105:0/452974184 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52840668d0 0x7f5284068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:24.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.858+0000 7f52a259c640 1 --2- 172.21.15.105:0/452974184 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52840668d0 0x7f5284068d90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f52940af000 tx=0x7f5294004ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:24.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:24.862+0000 7f529b7fe640 1 -- 172.21.15.105:0/452974184 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f529c050af0 con 0x7f52a4159610 2023-11-28T18:24:25.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.010+0000 7f52a8fd1640 1 -- 172.21.15.105:0/452974184 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f52a415aa20 con 0x7f52a4159610 2023-11-28T18:24:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.010+0000 7f529b7fe640 1 -- 172.21.15.105:0/452974184 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f529c018020 con 0x7f52a4159610 2023-11-28T18:24:25.015 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:24:25.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.010+0000 7f52a8fd1640 1 -- 172.21.15.105:0/452974184 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52840668d0 msgr2=0x7f5284068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:25.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.010+0000 7f52a8fd1640 1 --2- 172.21.15.105:0/452974184 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52840668d0 0x7f5284068d90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f52940af000 tx=0x7f5294004ac0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.010+0000 7f52a8fd1640 1 -- 172.21.15.105:0/452974184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52a4159610 msgr2=0x7f52a41583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:25.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.010+0000 7f52a8fd1640 1 --2- 172.21.15.105:0/452974184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52a4159610 0x7f52a41583b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f529c00c470 tx=0x7f529c00c930 comp rx=0 tx=0).stop 2023-11-28T18:24:25.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.010+0000 7f52a8fd1640 1 -- 172.21.15.105:0/452974184 shutdown_connections 2023-11-28T18:24:25.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.010+0000 7f52a8fd1640 1 --2- 172.21.15.105:0/452974184 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52840668d0 0x7f5284068d90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.010+0000 7f52a8fd1640 1 --2- 172.21.15.105:0/452974184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52a4159610 0x7f52a41583b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.010+0000 7f52a8fd1640 1 --2- 172.21.15.105:0/452974184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52a40cc760 0x7f52a4157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.010+0000 7f52a8fd1640 1 --2- 172.21.15.105:0/452974184 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52a406c7e0 0x7f52a409c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.010+0000 7f52a8fd1640 1 -- 172.21.15.105:0/452974184 >> 172.21.15.105:0/452974184 conn(0x7f52a40ceb90 msgr2=0x7f52a40abd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:25.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.010+0000 7f52a8fd1640 1 -- 172.21.15.105:0/452974184 shutdown_connections 2023-11-28T18:24:25.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.010+0000 7f52a8fd1640 1 -- 172.21.15.105:0/452974184 wait complete. 2023-11-28T18:24:25.037 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T18:24:25.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.174+0000 7f80e4cea640 1 Processor -- start 2023-11-28T18:24:25.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.174+0000 7f80e4cea640 1 -- start start 2023-11-28T18:24:25.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.174+0000 7f80e4cea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80e00caf60 0x7f80e00cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:25.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.174+0000 7f80e4cea640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f80e00ad850 0x7f80e00701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:25.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.174+0000 7f80e4cea640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80e006bf50 con 0x7f80e0159f40 2023-11-28T18:24:25.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.174+0000 7f80e4cea640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80e00cb3c0 con 0x7f80e00ad850 2023-11-28T18:24:25.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.174+0000 7f80e4cea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e015a320 con 0x7f80e00caf60 2023-11-28T18:24:25.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.174+0000 7f80ded9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f80e00ad850 0x7f80e00701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:25.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.174+0000 7f80de59c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f80e0159f40 0x7f80e00ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49252/0 (socket says 172.21.15.105:49252) 2023-11-28T18:24:25.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.174+0000 7f80de59c640 1 -- 172.21.15.105:0/4176274154 learned_addr learned my addr 172.21.15.105:0/4176274154 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:25.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.174+0000 7f80d7fff640 1 --2- 172.21.15.105:0/4176274154 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80e00caf60 0x7f80e00cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:25.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.174+0000 7f80ded9d640 1 -- 172.21.15.105:0/4176274154 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80e00caf60 msgr2=0x7f80e00cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:25.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.174+0000 7f80ded9d640 1 --2- 172.21.15.105:0/4176274154 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80e00caf60 0x7f80e00cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.174+0000 7f80ded9d640 1 -- 172.21.15.105:0/4176274154 >> v1:172.21.15.105:6789/0 conn(0x7f80e0159f40 legacy=0x7f80e00ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:25.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.174+0000 7f80ded9d640 1 -- 172.21.15.105:0/4176274154 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e0071380 con 0x7f80e00ad850 2023-11-28T18:24:25.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.174+0000 7f80ddd9b640 1 -- 172.21.15.105:0/4176274154 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1005104425 0 0) 0x7f80e006bf50 con 0x7f80e0159f40 2023-11-28T18:24:25.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.178+0000 7f80ded9d640 1 --2- 172.21.15.105:0/4176274154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e00ad850 0x7f80e00701d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f80d8009d90 tx=0x7f80d801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c6a4cec5c353f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:25.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.178+0000 7f80ddd9b640 1 -- 172.21.15.105:0/4176274154 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d80a5950 con 0x7f80e00ad850 2023-11-28T18:24:25.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.178+0000 7f80ddd9b640 1 -- 172.21.15.105:0/4176274154 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d80a5ab0 con 0x7f80e00ad850 2023-11-28T18:24:25.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.178+0000 7f80ddd9b640 1 -- 172.21.15.105:0/4176274154 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d80a5e40 con 0x7f80e00ad850 2023-11-28T18:24:25.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.178+0000 7f80e4cea640 1 -- 172.21.15.105:0/4176274154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e00ad850 msgr2=0x7f80e00701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:25.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.178+0000 7f80e4cea640 1 --2- 172.21.15.105:0/4176274154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e00ad850 0x7f80e00701d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f80d8009d90 tx=0x7f80d801c890 comp rx=0 tx=0).stop 2023-11-28T18:24:25.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.178+0000 7f80e4cea640 1 -- 172.21.15.105:0/4176274154 shutdown_connections 2023-11-28T18:24:25.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.178+0000 7f80e4cea640 1 --2- 172.21.15.105:0/4176274154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e00ad850 0x7f80e00701d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.178+0000 7f80e4cea640 1 --2- 172.21.15.105:0/4176274154 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80e00caf60 0x7f80e00cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.178+0000 7f80e4cea640 1 -- 172.21.15.105:0/4176274154 >> 172.21.15.105:0/4176274154 conn(0x7f80e00ceb90 msgr2=0x7f80e00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:25.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.178+0000 7f80e4cea640 1 -- 172.21.15.105:0/4176274154 shutdown_connections 2023-11-28T18:24:25.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.178+0000 7f80e4cea640 1 -- 172.21.15.105:0/4176274154 wait complete. 2023-11-28T18:24:25.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.178+0000 7f80e4cea640 1 Processor -- start 2023-11-28T18:24:25.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.178+0000 7f80e4cea640 1 -- start start 2023-11-28T18:24:25.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80e4cea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e00ad850 0x7f80e0146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:25.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80e4cea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80e00caf60 0x7f80e009c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:25.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80e4cea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80e0159f40 0x7f80e015b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:25.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80e4cea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e009bdb0 con 0x7f80e00ad850 2023-11-28T18:24:25.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80e4cea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e0146d70 con 0x7f80e00caf60 2023-11-28T18:24:25.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80e4cea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e014a810 con 0x7f80e0159f40 2023-11-28T18:24:25.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80de59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e00ad850 0x7f80e0146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:25.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80de59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e00ad850 0x7f80e0146830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43600/0 (socket says 172.21.15.105:43600) 2023-11-28T18:24:25.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80de59c640 1 -- 172.21.15.105:0/2998534154 learned_addr learned my addr 172.21.15.105:0/2998534154 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:25.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80d7fff640 1 --2- 172.21.15.105:0/2998534154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80e00caf60 0x7f80e009c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:25.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80ded9d640 1 --2- 172.21.15.105:0/2998534154 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80e0159f40 0x7f80e015b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:25.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80de59c640 1 -- 172.21.15.105:0/2998534154 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80e0159f40 msgr2=0x7f80e015b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:25.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80de59c640 1 --2- 172.21.15.105:0/2998534154 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80e0159f40 0x7f80e015b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80de59c640 1 -- 172.21.15.105:0/2998534154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80e00caf60 msgr2=0x7f80e009c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:25.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80de59c640 1 --2- 172.21.15.105:0/2998534154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80e00caf60 0x7f80e009c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80de59c640 1 -- 172.21.15.105:0/2998534154 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d8009a70 con 0x7f80e00ad850 2023-11-28T18:24:25.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80d7fff640 1 --2- 172.21.15.105:0/2998534154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80e00caf60 0x7f80e009c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:25.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80ded9d640 1 --2- 172.21.15.105:0/2998534154 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80e0159f40 0x7f80e015b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:25.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80de59c640 1 --2- 172.21.15.105:0/2998534154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e00ad850 0x7f80e0146830 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f80d000c480 tx=0x7f80d000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:25.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80d6ffd640 1 -- 172.21.15.105:0/2998534154 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d00074e0 con 0x7f80e00ad850 2023-11-28T18:24:25.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80d6ffd640 1 -- 172.21.15.105:0/2998534154 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d0007660 con 0x7f80e00ad850 2023-11-28T18:24:25.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80d6ffd640 1 -- 172.21.15.105:0/2998534154 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d0004a30 con 0x7f80e00ad850 2023-11-28T18:24:25.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80e4cea640 1 -- 172.21.15.105:0/2998534154 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e009ae50 con 0x7f80e00ad850 2023-11-28T18:24:25.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80e4cea640 1 -- 172.21.15.105:0/2998534154 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e009b410 con 0x7f80e00ad850 2023-11-28T18:24:25.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80e4cea640 1 -- 172.21.15.105:0/2998534154 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e0067570 con 0x7f80e00ad850 2023-11-28T18:24:25.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.182+0000 7f80d6ffd640 1 -- 172.21.15.105:0/2998534154 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f80d0004b90 con 0x7f80e00ad850 2023-11-28T18:24:25.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.186+0000 7f80d6ffd640 1 --2- 172.21.15.105:0/2998534154 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f80c00668d0 0x7f80c0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:25.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.186+0000 7f80d6ffd640 1 -- 172.21.15.105:0/2998534154 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f80d0083960 con 0x7f80e00ad850 2023-11-28T18:24:25.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.186+0000 7f80d7fff640 1 --2- 172.21.15.105:0/2998534154 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f80c00668d0 0x7f80c0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:25.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.186+0000 7f80d7fff640 1 --2- 172.21.15.105:0/2998534154 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f80c00668d0 0x7f80c0068d90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f80e0153ec0 tx=0x7f80c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:25.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.190+0000 7f80d6ffd640 1 -- 172.21.15.105:0/2998534154 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f80d00542c0 con 0x7f80e00ad850 2023-11-28T18:24:25.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.338+0000 7f80e4cea640 1 -- 172.21.15.105:0/2998534154 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f80e00cf950 con 0x7f80e00ad850 2023-11-28T18:24:25.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.338+0000 7f80d6ffd640 1 -- 172.21.15.105:0/2998534154 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f80d0018020 con 0x7f80e00ad850 2023-11-28T18:24:25.345 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.342+0000 7f80e4cea640 1 -- 172.21.15.105:0/2998534154 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f80c00668d0 msgr2=0x7f80c0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.342+0000 7f80e4cea640 1 --2- 172.21.15.105:0/2998534154 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f80c00668d0 0x7f80c0068d90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f80e0153ec0 tx=0x7f80c8008040 comp rx=0 tx=0).stop 2023-11-28T18:24:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.342+0000 7f80e4cea640 1 -- 172.21.15.105:0/2998534154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e00ad850 msgr2=0x7f80e0146830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.342+0000 7f80e4cea640 1 --2- 172.21.15.105:0/2998534154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e00ad850 0x7f80e0146830 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f80d000c480 tx=0x7f80d000c940 comp rx=0 tx=0).stop 2023-11-28T18:24:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.342+0000 7f80e4cea640 1 -- 172.21.15.105:0/2998534154 shutdown_connections 2023-11-28T18:24:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.342+0000 7f80e4cea640 1 --2- 172.21.15.105:0/2998534154 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80e0159f40 0x7f80e015b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.342+0000 7f80e4cea640 1 --2- 172.21.15.105:0/2998534154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80e00caf60 0x7f80e009c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.342+0000 7f80e4cea640 1 --2- 172.21.15.105:0/2998534154 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f80c00668d0 0x7f80c0068d90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.342+0000 7f80e4cea640 1 --2- 172.21.15.105:0/2998534154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e00ad850 0x7f80e0146830 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.342+0000 7f80e4cea640 1 -- 172.21.15.105:0/2998534154 >> 172.21.15.105:0/2998534154 conn(0x7f80e00ceb90 msgr2=0x7f80e00cda90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:25.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.342+0000 7f80e4cea640 1 -- 172.21.15.105:0/2998534154 shutdown_connections 2023-11-28T18:24:25.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.342+0000 7f80e4cea640 1 -- 172.21.15.105:0/2998534154 wait complete. 2023-11-28T18:24:25.366 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T18:24:25.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.498+0000 7f9177c5f640 1 Processor -- start 2023-11-28T18:24:25.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.498+0000 7f9177c5f640 1 -- start start 2023-11-28T18:24:25.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.502+0000 7f9177c5f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91700ade80 0x7f91700ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:25.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.502+0000 7f9177c5f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9170162820 con 0x7f917006d610 2023-11-28T18:24:25.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.502+0000 7f9177c5f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9170161c10 con 0x7f91700ad9f0 2023-11-28T18:24:25.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.502+0000 7f9177c5f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9170161ef0 con 0x7f91700ade80 2023-11-28T18:24:25.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.502+0000 7f91759fb640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f917006d610 0x7f91700cdff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49262/0 (socket says 172.21.15.105:49262) 2023-11-28T18:24:25.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.502+0000 7f91759fb640 1 -- 172.21.15.105:0/4215647633 learned_addr learned my addr 172.21.15.105:0/4215647633 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:25.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.502+0000 7f91761fc640 1 --2- 172.21.15.105:0/4215647633 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91700ade80 0x7f91700ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:25.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.502+0000 7f91749f9640 1 -- 172.21.15.105:0/4215647633 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3201036622 0 0) 0x7f9170162820 con 0x7f917006d610 2023-11-28T18:24:25.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.502+0000 7f91749f9640 1 -- 172.21.15.105:0/4215647633 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f915c003530 con 0x7f917006d610 2023-11-28T18:24:25.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.502+0000 7f91749f9640 1 -- 172.21.15.105:0/4215647633 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294667877 0 0) 0x7f9170161c10 con 0x7f91700ad9f0 2023-11-28T18:24:25.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.502+0000 7f91749f9640 1 -- 172.21.15.105:0/4215647633 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9170162820 con 0x7f91700ad9f0 2023-11-28T18:24:25.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.502+0000 7f91761fc640 1 -- 172.21.15.105:0/4215647633 >> v1:172.21.15.148:6789/0 conn(0x7f91700ad9f0 legacy=0x7f91700cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:25.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.502+0000 7f91749f9640 1 -- 172.21.15.105:0/4215647633 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2419904074 0 0) 0x7f915c003530 con 0x7f917006d610 2023-11-28T18:24:25.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.502+0000 7f91761fc640 1 -- 172.21.15.105:0/4215647633 >> v1:172.21.15.105:6789/0 conn(0x7f917006d610 legacy=0x7f91700cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:25.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.502+0000 7f91761fc640 1 -- 172.21.15.105:0/4215647633 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91701621d0 con 0x7f91700ade80 2023-11-28T18:24:25.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.506+0000 7f91761fc640 1 --2- 172.21.15.105:0/4215647633 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91700ade80 0x7f91700ccee0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9168009dc0 tx=0x7f916801c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8fc93fbb8c08c61 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:25.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.506+0000 7f91749f9640 1 -- 172.21.15.105:0/4215647633 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c001680 con 0x7f91700ade80 2023-11-28T18:24:25.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.506+0000 7f91749f9640 1 -- 172.21.15.105:0/4215647633 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91680a55b0 con 0x7f91700ade80 2023-11-28T18:24:25.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.506+0000 7f91749f9640 1 -- 172.21.15.105:0/4215647633 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91680a5920 con 0x7f91700ade80 2023-11-28T18:24:25.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.506+0000 7f9177c5f640 1 -- 172.21.15.105:0/4215647633 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91700ade80 msgr2=0x7f91700ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:25.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.506+0000 7f9177c5f640 1 --2- 172.21.15.105:0/4215647633 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91700ade80 0x7f91700ccee0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9168009dc0 tx=0x7f916801c860 comp rx=0 tx=0).stop 2023-11-28T18:24:25.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.506+0000 7f9177c5f640 1 -- 172.21.15.105:0/4215647633 shutdown_connections 2023-11-28T18:24:25.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.506+0000 7f9177c5f640 1 --2- 172.21.15.105:0/4215647633 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91700ade80 0x7f91700ccee0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.506+0000 7f9177c5f640 1 -- 172.21.15.105:0/4215647633 >> 172.21.15.105:0/4215647633 conn(0x7f91700ceb90 msgr2=0x7f91700cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:25.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.506+0000 7f9177c5f640 1 -- 172.21.15.105:0/4215647633 shutdown_connections 2023-11-28T18:24:25.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.506+0000 7f9177c5f640 1 -- 172.21.15.105:0/4215647633 wait complete. 2023-11-28T18:24:25.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.506+0000 7f9177c5f640 1 Processor -- start 2023-11-28T18:24:25.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.506+0000 7f9177c5f640 1 -- start start 2023-11-28T18:24:25.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.506+0000 7f9177c5f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f917006d610 0x7f9170160f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:25.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.506+0000 7f9177c5f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91700ad9f0 0x7f91701645b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.506+0000 7f9177c5f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91700ade80 0x7f917015e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.506+0000 7f9177c5f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9170160610 con 0x7f917006d610 2023-11-28T18:24:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.506+0000 7f9177c5f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9170163510 con 0x7f91700ade80 2023-11-28T18:24:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.506+0000 7f9177c5f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91701637f0 con 0x7f91700ad9f0 2023-11-28T18:24:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.510+0000 7f91759fb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f917006d610 0x7f9170160f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.510+0000 7f91759fb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f917006d610 0x7f9170160f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43610/0 (socket says 172.21.15.105:43610) 2023-11-28T18:24:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.510+0000 7f91759fb640 1 -- 172.21.15.105:0/1416511056 learned_addr learned my addr 172.21.15.105:0/1416511056 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.510+0000 7f91761fc640 1 --2- 172.21.15.105:0/1416511056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91700ade80 0x7f917015e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.510+0000 7f91751fa640 1 --2- 172.21.15.105:0/1416511056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91700ad9f0 0x7f91701645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.510+0000 7f91759fb640 1 -- 172.21.15.105:0/1416511056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91700ad9f0 msgr2=0x7f91701645b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.510+0000 7f91759fb640 1 --2- 172.21.15.105:0/1416511056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91700ad9f0 0x7f91701645b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.510+0000 7f91759fb640 1 -- 172.21.15.105:0/1416511056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91700ade80 msgr2=0x7f917015e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.510+0000 7f91759fb640 1 --2- 172.21.15.105:0/1416511056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91700ade80 0x7f917015e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.510+0000 7f91759fb640 1 -- 172.21.15.105:0/1416511056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9168009a70 con 0x7f917006d610 2023-11-28T18:24:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.510+0000 7f91751fa640 1 --2- 172.21.15.105:0/1416511056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91700ad9f0 0x7f91701645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.510+0000 7f91759fb640 1 --2- 172.21.15.105:0/1416511056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f917006d610 0x7f9170160f70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f916c00c700 tx=0x7f916c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.510+0000 7f91667fc640 1 -- 172.21.15.105:0/1416511056 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c005ae0 con 0x7f917006d610 2023-11-28T18:24:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.510+0000 7f91667fc640 1 -- 172.21.15.105:0/1416511056 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916c005c40 con 0x7f917006d610 2023-11-28T18:24:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.510+0000 7f91667fc640 1 -- 172.21.15.105:0/1416511056 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c007270 con 0x7f917006d610 2023-11-28T18:24:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.510+0000 7f9177c5f640 1 -- 172.21.15.105:0/1416511056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9170163ad0 con 0x7f917006d610 2023-11-28T18:24:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.510+0000 7f9177c5f640 1 -- 172.21.15.105:0/1416511056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9170164090 con 0x7f917006d610 2023-11-28T18:24:25.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.510+0000 7f9177c5f640 1 -- 172.21.15.105:0/1416511056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9170164280 con 0x7f917006d610 2023-11-28T18:24:25.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.510+0000 7f91667fc640 1 -- 172.21.15.105:0/1416511056 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f916c0073d0 con 0x7f917006d610 2023-11-28T18:24:25.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.514+0000 7f91667fc640 1 --2- 172.21.15.105:0/1416511056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f91540668d0 0x7f9154068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:25.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.514+0000 7f91667fc640 1 -- 172.21.15.105:0/1416511056 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f916c083dc0 con 0x7f917006d610 2023-11-28T18:24:25.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.514+0000 7f91751fa640 1 --2- 172.21.15.105:0/1416511056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f91540668d0 0x7f9154068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:25.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.514+0000 7f91751fa640 1 --2- 172.21.15.105:0/1416511056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f91540668d0 0x7f9154068d90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f917015db50 tx=0x7f9160009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:25.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.518+0000 7f91667fc640 1 -- 172.21.15.105:0/1416511056 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f916c050c90 con 0x7f917006d610 2023-11-28T18:24:25.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.682+0000 7f9177c5f640 1 -- 172.21.15.105:0/1416511056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9170067570 con 0x7f917006d610 2023-11-28T18:24:25.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.682+0000 7f91667fc640 1 -- 172.21.15.105:0/1416511056 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f916c017020 con 0x7f917006d610 2023-11-28T18:24:25.686 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2023-11-28T18:24:25.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.682+0000 7f9177c5f640 1 -- 172.21.15.105:0/1416511056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f91540668d0 msgr2=0x7f9154068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:25.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.682+0000 7f9177c5f640 1 --2- 172.21.15.105:0/1416511056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f91540668d0 0x7f9154068d90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f917015db50 tx=0x7f9160009040 comp rx=0 tx=0).stop 2023-11-28T18:24:25.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.682+0000 7f9177c5f640 1 -- 172.21.15.105:0/1416511056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f917006d610 msgr2=0x7f9170160f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:25.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.682+0000 7f9177c5f640 1 --2- 172.21.15.105:0/1416511056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f917006d610 0x7f9170160f70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f916c00c700 tx=0x7f916c00cbc0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.682+0000 7f9177c5f640 1 -- 172.21.15.105:0/1416511056 shutdown_connections 2023-11-28T18:24:25.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.682+0000 7f9177c5f640 1 --2- 172.21.15.105:0/1416511056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91700ade80 0x7f917015e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.682+0000 7f9177c5f640 1 --2- 172.21.15.105:0/1416511056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91700ad9f0 0x7f91701645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.682+0000 7f9177c5f640 1 --2- 172.21.15.105:0/1416511056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f91540668d0 0x7f9154068d90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.682+0000 7f9177c5f640 1 --2- 172.21.15.105:0/1416511056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f917006d610 0x7f9170160f70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.682+0000 7f9177c5f640 1 -- 172.21.15.105:0/1416511056 >> 172.21.15.105:0/1416511056 conn(0x7f91700ceb90 msgr2=0x7f91700aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:25.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.682+0000 7f9177c5f640 1 -- 172.21.15.105:0/1416511056 shutdown_connections 2023-11-28T18:24:25.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.682+0000 7f9177c5f640 1 -- 172.21.15.105:0/1416511056 wait complete. 2023-11-28T18:24:25.707 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:24:25.708 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:24:25.717 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:24:25.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.890+0000 7feb30d1d640 1 Processor -- start 2023-11-28T18:24:25.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.890+0000 7feb30d1d640 1 -- start start 2023-11-28T18:24:25.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.894+0000 7feb30d1d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7feb2c0caf60 0x7feb2c0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:25.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.894+0000 7feb30d1d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feb2c0ad850 0x7feb2c0701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:25.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.894+0000 7feb30d1d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2c06bf50 con 0x7feb2c159f40 2023-11-28T18:24:25.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.894+0000 7feb30d1d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb2c0cb3c0 con 0x7feb2c0caf60 2023-11-28T18:24:25.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.894+0000 7feb30d1d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb2c15a320 con 0x7feb2c0ad850 2023-11-28T18:24:25.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.894+0000 7feb23fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7feb2c0caf60 0x7feb2c0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:25.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.894+0000 7feb23fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7feb2c0caf60 0x7feb2c0cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43612/0 (socket says 172.21.15.105:43612) 2023-11-28T18:24:25.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.894+0000 7feb23fff640 1 -- 172.21.15.105:0/1580538554 learned_addr learned my addr 172.21.15.105:0/1580538554 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:25.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.894+0000 7feb2ad9d640 1 --2- 172.21.15.105:0/1580538554 >> v2:172.21.15.148:3300/0 conn(0x7feb2c0ad850 0x7feb2c0701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:25.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.894+0000 7feb23fff640 1 -- 172.21.15.105:0/1580538554 >> v2:172.21.15.148:3300/0 conn(0x7feb2c0ad850 msgr2=0x7feb2c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:25.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.894+0000 7feb23fff640 1 --2- 172.21.15.105:0/1580538554 >> v2:172.21.15.148:3300/0 conn(0x7feb2c0ad850 0x7feb2c0701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.894+0000 7feb23fff640 1 -- 172.21.15.105:0/1580538554 >> v1:172.21.15.148:6789/0 conn(0x7feb2c159f40 legacy=0x7feb2c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:25.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.894+0000 7feb23fff640 1 -- 172.21.15.105:0/1580538554 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c071380 con 0x7feb2c0caf60 2023-11-28T18:24:25.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.894+0000 7feb23fff640 1 --2- 172.21.15.105:0/1580538554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb2c0caf60 0x7feb2c0cc760 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0b0850 tx=0x7feb1401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e9535fe31c95fb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:25.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.898+0000 7feb29d9b640 1 -- 172.21.15.105:0/1580538554 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb140ab640 con 0x7feb2c0caf60 2023-11-28T18:24:25.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.898+0000 7feb29d9b640 1 -- 172.21.15.105:0/1580538554 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb140a7070 con 0x7feb2c0caf60 2023-11-28T18:24:25.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.898+0000 7feb29d9b640 1 -- 172.21.15.105:0/1580538554 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb140ab9c0 con 0x7feb2c0caf60 2023-11-28T18:24:25.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.898+0000 7feb30d1d640 1 -- 172.21.15.105:0/1580538554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb2c0caf60 msgr2=0x7feb2c0cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:25.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.898+0000 7feb30d1d640 1 --2- 172.21.15.105:0/1580538554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb2c0caf60 0x7feb2c0cc760 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0b0850 tx=0x7feb1401c440 comp rx=0 tx=0).stop 2023-11-28T18:24:25.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.898+0000 7feb30d1d640 1 -- 172.21.15.105:0/1580538554 shutdown_connections 2023-11-28T18:24:25.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.898+0000 7feb30d1d640 1 --2- 172.21.15.105:0/1580538554 >> v2:172.21.15.148:3300/0 conn(0x7feb2c0ad850 0x7feb2c0701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.898+0000 7feb30d1d640 1 --2- 172.21.15.105:0/1580538554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb2c0caf60 0x7feb2c0cc760 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.898+0000 7feb30d1d640 1 -- 172.21.15.105:0/1580538554 >> 172.21.15.105:0/1580538554 conn(0x7feb2c0ceb90 msgr2=0x7feb2c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:25.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.898+0000 7feb30d1d640 1 -- 172.21.15.105:0/1580538554 shutdown_connections 2023-11-28T18:24:25.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.898+0000 7feb30d1d640 1 -- 172.21.15.105:0/1580538554 wait complete. 2023-11-28T18:24:25.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.898+0000 7feb30d1d640 1 Processor -- start 2023-11-28T18:24:25.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.898+0000 7feb30d1d640 1 -- start start 2023-11-28T18:24:25.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.898+0000 7feb30d1d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb2c0ad850 0x7feb2c14d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:25.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.898+0000 7feb30d1d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb2c0caf60 0x7feb2c099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:25.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.898+0000 7feb30d1d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb2c159f40 0x7feb2c06ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:25.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.898+0000 7feb30d1d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c09b730 con 0x7feb2c0caf60 2023-11-28T18:24:25.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.898+0000 7feb30d1d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c09b9b0 con 0x7feb2c0ad850 2023-11-28T18:24:25.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.898+0000 7feb30d1d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c09bc90 con 0x7feb2c159f40 2023-11-28T18:24:25.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.902+0000 7feb23fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb2c0caf60 0x7feb2c099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:25.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.902+0000 7feb23fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb2c0caf60 0x7feb2c099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43616/0 (socket says 172.21.15.105:43616) 2023-11-28T18:24:25.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.902+0000 7feb23fff640 1 -- 172.21.15.105:0/2297863258 learned_addr learned my addr 172.21.15.105:0/2297863258 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:25.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.902+0000 7feb2a59c640 1 --2- 172.21.15.105:0/2297863258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb2c0ad850 0x7feb2c14d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:25.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.902+0000 7feb2ad9d640 1 --2- 172.21.15.105:0/2297863258 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb2c159f40 0x7feb2c06ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:25.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.902+0000 7feb23fff640 1 -- 172.21.15.105:0/2297863258 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb2c159f40 msgr2=0x7feb2c06ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:25.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.902+0000 7feb23fff640 1 --2- 172.21.15.105:0/2297863258 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb2c159f40 0x7feb2c06ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.902+0000 7feb23fff640 1 -- 172.21.15.105:0/2297863258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb2c0ad850 msgr2=0x7feb2c14d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:25.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.902+0000 7feb23fff640 1 --2- 172.21.15.105:0/2297863258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb2c0ad850 0x7feb2c14d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:25.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.902+0000 7feb23fff640 1 -- 172.21.15.105:0/2297863258 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb140094d0 con 0x7feb2c0caf60 2023-11-28T18:24:25.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.902+0000 7feb23fff640 1 --2- 172.21.15.105:0/2297863258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb2c0caf60 0x7feb2c099a50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7feb14019050 tx=0x7feb1401ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:25.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.902+0000 7feb22ffd640 1 -- 172.21.15.105:0/2297863258 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb140be3f0 con 0x7feb2c0caf60 2023-11-28T18:24:25.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.902+0000 7feb22ffd640 1 -- 172.21.15.105:0/2297863258 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb140be550 con 0x7feb2c0caf60 2023-11-28T18:24:25.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.902+0000 7feb30d1d640 1 -- 172.21.15.105:0/2297863258 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2c09bf70 con 0x7feb2c0caf60 2023-11-28T18:24:25.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.902+0000 7feb22ffd640 1 -- 172.21.15.105:0/2297863258 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb140be900 con 0x7feb2c0caf60 2023-11-28T18:24:25.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.902+0000 7feb30d1d640 1 -- 172.21.15.105:0/2297863258 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2c09c530 con 0x7feb2c0caf60 2023-11-28T18:24:25.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.902+0000 7feb30d1d640 1 -- 172.21.15.105:0/2297863258 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb2c067570 con 0x7feb2c0caf60 2023-11-28T18:24:25.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.902+0000 7feb22ffd640 1 -- 172.21.15.105:0/2297863258 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7feb140b43f0 con 0x7feb2c0caf60 2023-11-28T18:24:25.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.902+0000 7feb22ffd640 1 --2- 172.21.15.105:0/2297863258 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feb0c0668d0 0x7feb0c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:25.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.906+0000 7feb22ffd640 1 -- 172.21.15.105:0/2297863258 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7feb1411ec50 con 0x7feb2c0caf60 2023-11-28T18:24:25.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.906+0000 7feb2a59c640 1 --2- 172.21.15.105:0/2297863258 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feb0c0668d0 0x7feb0c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:25.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.906+0000 7feb2a59c640 1 --2- 172.21.15.105:0/2297863258 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feb0c0668d0 0x7feb0c068d90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7feb1c007fd0 tx=0x7feb1c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:25.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:25.906+0000 7feb22ffd640 1 -- 172.21.15.105:0/2297863258 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7feb140eb990 con 0x7feb2c0caf60 2023-11-28T18:24:26.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.058+0000 7feb30d1d640 1 -- 172.21.15.105:0/2297863258 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb2c0cf950 con 0x7feb2c0caf60 2023-11-28T18:24:26.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.058+0000 7feb22ffd640 1 -- 172.21.15.105:0/2297863258 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18887 (secure 0 0 0) 0x7feb140c4090 con 0x7feb2c0caf60 2023-11-28T18:24:26.064 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:26.064 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:24:15.048168+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:24:26.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.062+0000 7feb30d1d640 1 -- 172.21.15.105:0/2297863258 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feb0c0668d0 msgr2=0x7feb0c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:26.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.062+0000 7feb30d1d640 1 --2- 172.21.15.105:0/2297863258 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feb0c0668d0 0x7feb0c068d90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7feb1c007fd0 tx=0x7feb1c009040 comp rx=0 tx=0).stop 2023-11-28T18:24:26.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.062+0000 7feb30d1d640 1 -- 172.21.15.105:0/2297863258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb2c0caf60 msgr2=0x7feb2c099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:26.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.062+0000 7feb30d1d640 1 --2- 172.21.15.105:0/2297863258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb2c0caf60 0x7feb2c099a50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7feb14019050 tx=0x7feb1401ac80 comp rx=0 tx=0).stop 2023-11-28T18:24:26.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.062+0000 7feb30d1d640 1 -- 172.21.15.105:0/2297863258 shutdown_connections 2023-11-28T18:24:26.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.062+0000 7feb30d1d640 1 --2- 172.21.15.105:0/2297863258 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb2c159f40 0x7feb2c06ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.062+0000 7feb30d1d640 1 --2- 172.21.15.105:0/2297863258 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feb0c0668d0 0x7feb0c068d90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.062+0000 7feb30d1d640 1 --2- 172.21.15.105:0/2297863258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb2c0caf60 0x7feb2c099a50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.062+0000 7feb30d1d640 1 --2- 172.21.15.105:0/2297863258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb2c0ad850 0x7feb2c14d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.062+0000 7feb30d1d640 1 -- 172.21.15.105:0/2297863258 >> 172.21.15.105:0/2297863258 conn(0x7feb2c0ceb90 msgr2=0x7feb2c0cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:26.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.062+0000 7feb30d1d640 1 -- 172.21.15.105:0/2297863258 shutdown_connections 2023-11-28T18:24:26.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.062+0000 7feb30d1d640 1 -- 172.21.15.105:0/2297863258 wait complete. 2023-11-28T18:24:26.086 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:23:31.485422+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:23:32.274378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T18:23:36.100071+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T18:24:26.087 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:24:26.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.226+0000 7fea4dc54640 1 Processor -- start 2023-11-28T18:24:26.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.226+0000 7fea4dc54640 1 -- start start 2023-11-28T18:24:26.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.226+0000 7fea4dc54640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea48154070 0x7fea4815d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:26.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.226+0000 7fea4dc54640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fea480adb80 0x7fea480adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:26.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.226+0000 7fea4dc54640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea480ae510 con 0x7fea4806bf50 2023-11-28T18:24:26.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.226+0000 7fea4dc54640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea48072540 con 0x7fea480adb80 2023-11-28T18:24:26.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.226+0000 7fea4dc54640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea480cc7f0 con 0x7fea48154070 2023-11-28T18:24:26.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.226+0000 7fea47fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fea480adb80 0x7fea480adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:26.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.226+0000 7fea47fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fea480adb80 0x7fea480adfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43624/0 (socket says 172.21.15.105:43624) 2023-11-28T18:24:26.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.226+0000 7fea47fff640 1 -- 172.21.15.105:0/3583584112 learned_addr learned my addr 172.21.15.105:0/3583584112 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:26.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.226+0000 7fea477fe640 1 --2- 172.21.15.105:0/3583584112 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea48154070 0x7fea4815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:26.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.226+0000 7fea47fff640 1 -- 172.21.15.105:0/3583584112 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea48154070 msgr2=0x7fea4815d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:26.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.226+0000 7fea47fff640 1 --2- 172.21.15.105:0/3583584112 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea48154070 0x7fea4815d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.226+0000 7fea47fff640 1 -- 172.21.15.105:0/3583584112 >> v1:172.21.15.148:6789/0 conn(0x7fea4806bf50 legacy=0x7fea480ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:26.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.226+0000 7fea47fff640 1 -- 172.21.15.105:0/3583584112 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4815dd10 con 0x7fea480adb80 2023-11-28T18:24:26.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.230+0000 7fea47fff640 1 --2- 172.21.15.105:0/3583584112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea480adb80 0x7fea480adfd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fea40009e50 tx=0x7fea4001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2801258a26c61f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:26.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.230+0000 7fea46ffd640 1 -- 172.21.15.105:0/3583584112 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea400a64e0 con 0x7fea480adb80 2023-11-28T18:24:26.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.230+0000 7fea46ffd640 1 -- 172.21.15.105:0/3583584112 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea400a6640 con 0x7fea480adb80 2023-11-28T18:24:26.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.230+0000 7fea46ffd640 1 -- 172.21.15.105:0/3583584112 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea400a69d0 con 0x7fea480adb80 2023-11-28T18:24:26.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.230+0000 7fea4dc54640 1 -- 172.21.15.105:0/3583584112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea480adb80 msgr2=0x7fea480adfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:26.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.230+0000 7fea4dc54640 1 --2- 172.21.15.105:0/3583584112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea480adb80 0x7fea480adfd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fea40009e50 tx=0x7fea4001c9e0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.230+0000 7fea4dc54640 1 -- 172.21.15.105:0/3583584112 shutdown_connections 2023-11-28T18:24:26.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.230+0000 7fea4dc54640 1 --2- 172.21.15.105:0/3583584112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea480adb80 0x7fea480adfd0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.230+0000 7fea4dc54640 1 --2- 172.21.15.105:0/3583584112 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea48154070 0x7fea4815d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.230+0000 7fea4dc54640 1 -- 172.21.15.105:0/3583584112 >> 172.21.15.105:0/3583584112 conn(0x7fea480ceb90 msgr2=0x7fea480cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:26.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.230+0000 7fea4dc54640 1 -- 172.21.15.105:0/3583584112 shutdown_connections 2023-11-28T18:24:26.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.230+0000 7fea4dc54640 1 -- 172.21.15.105:0/3583584112 wait complete. 2023-11-28T18:24:26.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea4dc54640 1 Processor -- start 2023-11-28T18:24:26.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea4dc54640 1 -- start start 2023-11-28T18:24:26.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea4dc54640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea4806bf50 0x7fea4814fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:26.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea4dc54640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea480adb80 0x7fea48153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:26.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea4dc54640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea48154070 0x7fea4806c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:26.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea4dc54640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4814f5b0 con 0x7fea48154070 2023-11-28T18:24:26.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea4dc54640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4814efa0 con 0x7fea480adb80 2023-11-28T18:24:26.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea4dc54640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4814b010 con 0x7fea4806bf50 2023-11-28T18:24:26.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea47fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea48154070 0x7fea4806c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:26.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea47fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea48154070 0x7fea4806c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43636/0 (socket says 172.21.15.105:43636) 2023-11-28T18:24:26.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea47fff640 1 -- 172.21.15.105:0/2730578684 learned_addr learned my addr 172.21.15.105:0/2730578684 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:26.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea477fe640 1 --2- 172.21.15.105:0/2730578684 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea4806bf50 0x7fea4814fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:26.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea3ffff640 1 --2- 172.21.15.105:0/2730578684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea480adb80 0x7fea48153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:26.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea47fff640 1 -- 172.21.15.105:0/2730578684 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea4806bf50 msgr2=0x7fea4814fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:26.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea47fff640 1 --2- 172.21.15.105:0/2730578684 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea4806bf50 0x7fea4814fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea47fff640 1 -- 172.21.15.105:0/2730578684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea480adb80 msgr2=0x7fea48153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:26.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea47fff640 1 --2- 172.21.15.105:0/2730578684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea480adb80 0x7fea48153570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea47fff640 1 -- 172.21.15.105:0/2730578684 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea40009b30 con 0x7fea48154070 2023-11-28T18:24:26.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea47fff640 1 --2- 172.21.15.105:0/2730578684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea48154070 0x7fea4806c850 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fea4001b090 tx=0x7fea40019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:26.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea44ff9640 1 -- 172.21.15.105:0/2730578684 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea400b4070 con 0x7fea48154070 2023-11-28T18:24:26.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea44ff9640 1 -- 172.21.15.105:0/2730578684 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea40009150 con 0x7fea48154070 2023-11-28T18:24:26.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea4dc54640 1 -- 172.21.15.105:0/2730578684 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4814b2f0 con 0x7fea48154070 2023-11-28T18:24:26.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea44ff9640 1 -- 172.21.15.105:0/2730578684 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea400a6a10 con 0x7fea48154070 2023-11-28T18:24:26.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.234+0000 7fea4dc54640 1 -- 172.21.15.105:0/2730578684 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4814f240 con 0x7fea48154070 2023-11-28T18:24:26.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.238+0000 7fea4dc54640 1 -- 172.21.15.105:0/2730578684 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea48067570 con 0x7fea48154070 2023-11-28T18:24:26.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.238+0000 7fea44ff9640 1 -- 172.21.15.105:0/2730578684 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fea400b8070 con 0x7fea48154070 2023-11-28T18:24:26.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.238+0000 7fea44ff9640 1 --2- 172.21.15.105:0/2730578684 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fea280668d0 0x7fea28068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:26.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.238+0000 7fea44ff9640 1 -- 172.21.15.105:0/2730578684 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fea400aa070 con 0x7fea48154070 2023-11-28T18:24:26.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.238+0000 7fea477fe640 1 --2- 172.21.15.105:0/2730578684 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fea280668d0 0x7fea28068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:26.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.238+0000 7fea477fe640 1 --2- 172.21.15.105:0/2730578684 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fea280668d0 0x7fea28068d90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fea48161d60 tx=0x7fea38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:26.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.242+0000 7fea44ff9640 1 -- 172.21.15.105:0/2730578684 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fea400eb250 con 0x7fea48154070 2023-11-28T18:24:26.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.394+0000 7fea4dc54640 1 -- 172.21.15.105:0/2730578684 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fea480cf950 con 0x7fea48154070 2023-11-28T18:24:26.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.394+0000 7fea44ff9640 1 -- 172.21.15.105:0/2730578684 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fea400ee870 con 0x7fea48154070 2023-11-28T18:24:26.399 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:24:26.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.394+0000 7fea4dc54640 1 -- 172.21.15.105:0/2730578684 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fea280668d0 msgr2=0x7fea28068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:26.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.394+0000 7fea4dc54640 1 --2- 172.21.15.105:0/2730578684 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fea280668d0 0x7fea28068d90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fea48161d60 tx=0x7fea38008040 comp rx=0 tx=0).stop 2023-11-28T18:24:26.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.394+0000 7fea4dc54640 1 -- 172.21.15.105:0/2730578684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea48154070 msgr2=0x7fea4806c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:26.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.394+0000 7fea4dc54640 1 --2- 172.21.15.105:0/2730578684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea48154070 0x7fea4806c850 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fea4001b090 tx=0x7fea40019780 comp rx=0 tx=0).stop 2023-11-28T18:24:26.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.394+0000 7fea4dc54640 1 -- 172.21.15.105:0/2730578684 shutdown_connections 2023-11-28T18:24:26.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.394+0000 7fea4dc54640 1 --2- 172.21.15.105:0/2730578684 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fea280668d0 0x7fea28068d90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.394+0000 7fea4dc54640 1 --2- 172.21.15.105:0/2730578684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea48154070 0x7fea4806c850 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.394+0000 7fea4dc54640 1 --2- 172.21.15.105:0/2730578684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea480adb80 0x7fea48153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.394+0000 7fea4dc54640 1 --2- 172.21.15.105:0/2730578684 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea4806bf50 0x7fea4814fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.394+0000 7fea4dc54640 1 -- 172.21.15.105:0/2730578684 >> 172.21.15.105:0/2730578684 conn(0x7fea480ceb90 msgr2=0x7fea480cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:26.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.394+0000 7fea4dc54640 1 -- 172.21.15.105:0/2730578684 shutdown_connections 2023-11-28T18:24:26.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.394+0000 7fea4dc54640 1 -- 172.21.15.105:0/2730578684 wait complete. 2023-11-28T18:24:26.420 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:24:26.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.558+0000 7fc7ec9aa640 1 Processor -- start 2023-11-28T18:24:26.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.558+0000 7fc7ec9aa640 1 -- start start 2023-11-28T18:24:26.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.558+0000 7fc7ec9aa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc7e8154070 0x7fc7e815d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:26.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.558+0000 7fc7ec9aa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc7e80adb80 0x7fc7e80adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:26.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.558+0000 7fc7ec9aa640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7e80cc7f0 con 0x7fc7e806bf50 2023-11-28T18:24:26.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.558+0000 7fc7ec9aa640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7e8072540 con 0x7fc7e8154070 2023-11-28T18:24:26.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.558+0000 7fc7ec9aa640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7e80ae4f0 con 0x7fc7e80adb80 2023-11-28T18:24:26.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.558+0000 7fc7e5d9b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc7e806bf50 0x7fc7e80ce310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49266/0 (socket says 172.21.15.105:49266) 2023-11-28T18:24:26.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.558+0000 7fc7e5d9b640 1 -- 172.21.15.105:0/2025395973 learned_addr learned my addr 172.21.15.105:0/2025395973 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:26.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.558+0000 7fc7e659c640 1 --2- 172.21.15.105:0/2025395973 >> v2:172.21.15.105:3300/0 conn(0x7fc7e8154070 0x7fc7e815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:26.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.558+0000 7fc7e6d9d640 1 --2- 172.21.15.105:0/2025395973 >> v2:172.21.15.148:3300/0 conn(0x7fc7e80adb80 0x7fc7e80adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:26.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.558+0000 7fc7e659c640 1 -- 172.21.15.105:0/2025395973 >> v2:172.21.15.148:3300/0 conn(0x7fc7e80adb80 msgr2=0x7fc7e80adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:26.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.558+0000 7fc7e659c640 1 --2- 172.21.15.105:0/2025395973 >> v2:172.21.15.148:3300/0 conn(0x7fc7e80adb80 0x7fc7e80adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.558+0000 7fc7e659c640 1 -- 172.21.15.105:0/2025395973 >> v1:172.21.15.105:6789/0 conn(0x7fc7e806bf50 legacy=0x7fc7e80ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:26.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.558+0000 7fc7e559a640 1 -- 172.21.15.105:0/2025395973 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2650228876 0 0) 0x7fc7e80cc7f0 con 0x7fc7e806bf50 2023-11-28T18:24:26.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.558+0000 7fc7e659c640 1 -- 172.21.15.105:0/2025395973 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e815dd10 con 0x7fc7e8154070 2023-11-28T18:24:26.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.558+0000 7fc7e6d9d640 1 --2- 172.21.15.105:0/2025395973 >> v2:172.21.15.148:3300/0 conn(0x7fc7e80adb80 0x7fc7e80adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:26.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.562+0000 7fc7e659c640 1 --2- 172.21.15.105:0/2025395973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7e8154070 0x7fc7e815d700 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc7e806e600 tx=0x7fc7d801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53efdacf004e8988 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:26.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.562+0000 7fc7e559a640 1 -- 172.21.15.105:0/2025395973 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d80ab600 con 0x7fc7e8154070 2023-11-28T18:24:26.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.562+0000 7fc7e559a640 1 -- 172.21.15.105:0/2025395973 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d80a7050 con 0x7fc7e8154070 2023-11-28T18:24:26.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.562+0000 7fc7e559a640 1 -- 172.21.15.105:0/2025395973 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d80ab980 con 0x7fc7e8154070 2023-11-28T18:24:26.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.562+0000 7fc7ec9aa640 1 -- 172.21.15.105:0/2025395973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7e8154070 msgr2=0x7fc7e815d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:26.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.562+0000 7fc7ec9aa640 1 --2- 172.21.15.105:0/2025395973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7e8154070 0x7fc7e815d700 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc7e806e600 tx=0x7fc7d801c430 comp rx=0 tx=0).stop 2023-11-28T18:24:26.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.562+0000 7fc7ec9aa640 1 -- 172.21.15.105:0/2025395973 shutdown_connections 2023-11-28T18:24:26.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.562+0000 7fc7ec9aa640 1 --2- 172.21.15.105:0/2025395973 >> v2:172.21.15.148:3300/0 conn(0x7fc7e80adb80 0x7fc7e80adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.562+0000 7fc7ec9aa640 1 --2- 172.21.15.105:0/2025395973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7e8154070 0x7fc7e815d700 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.562+0000 7fc7ec9aa640 1 -- 172.21.15.105:0/2025395973 >> 172.21.15.105:0/2025395973 conn(0x7fc7e80ceb90 msgr2=0x7fc7e80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:26.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.562+0000 7fc7ec9aa640 1 -- 172.21.15.105:0/2025395973 shutdown_connections 2023-11-28T18:24:26.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.562+0000 7fc7ec9aa640 1 -- 172.21.15.105:0/2025395973 wait complete. 2023-11-28T18:24:26.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7ec9aa640 1 Processor -- start 2023-11-28T18:24:26.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7ec9aa640 1 -- start start 2023-11-28T18:24:26.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7ec9aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7e806bf50 0x7fc7e814aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:26.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7ec9aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7e80adb80 0x7fc7e814f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:26.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7ec9aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7e8154070 0x7fc7e815fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:26.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7ec9aa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e8160830 con 0x7fc7e80adb80 2023-11-28T18:24:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7ec9aa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e814af90 con 0x7fc7e8154070 2023-11-28T18:24:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7ec9aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e8160270 con 0x7fc7e806bf50 2023-11-28T18:24:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7e5d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7e80adb80 0x7fc7e814f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7e5d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7e80adb80 0x7fc7e814f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43640/0 (socket says 172.21.15.105:43640) 2023-11-28T18:24:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7e5d9b640 1 -- 172.21.15.105:0/3043216237 learned_addr learned my addr 172.21.15.105:0/3043216237 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7e659c640 1 --2- 172.21.15.105:0/3043216237 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7e806bf50 0x7fc7e814aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7e6d9d640 1 --2- 172.21.15.105:0/3043216237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7e8154070 0x7fc7e815fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7e5d9b640 1 -- 172.21.15.105:0/3043216237 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7e806bf50 msgr2=0x7fc7e814aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7e5d9b640 1 --2- 172.21.15.105:0/3043216237 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7e806bf50 0x7fc7e814aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7e5d9b640 1 -- 172.21.15.105:0/3043216237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7e8154070 msgr2=0x7fc7e815fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7e5d9b640 1 --2- 172.21.15.105:0/3043216237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7e8154070 0x7fc7e815fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7e5d9b640 1 -- 172.21.15.105:0/3043216237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d8009420 con 0x7fc7e80adb80 2023-11-28T18:24:26.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7e5d9b640 1 --2- 172.21.15.105:0/3043216237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7e80adb80 0x7fc7e814f060 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc0011f0 tx=0x7fc7dc00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:26.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7d6ffd640 1 -- 172.21.15.105:0/3043216237 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc0074e0 con 0x7fc7e80adb80 2023-11-28T18:24:26.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7d6ffd640 1 -- 172.21.15.105:0/3043216237 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7dc007640 con 0x7fc7e80adb80 2023-11-28T18:24:26.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7d6ffd640 1 -- 172.21.15.105:0/3043216237 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc004a10 con 0x7fc7e80adb80 2023-11-28T18:24:26.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7ec9aa640 1 -- 172.21.15.105:0/3043216237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e8160550 con 0x7fc7e80adb80 2023-11-28T18:24:26.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7ec9aa640 1 -- 172.21.15.105:0/3043216237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e814eb40 con 0x7fc7e80adb80 2023-11-28T18:24:26.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.566+0000 7fc7ec9aa640 1 -- 172.21.15.105:0/3043216237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7e80ccc60 con 0x7fc7e80adb80 2023-11-28T18:24:26.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.570+0000 7fc7d6ffd640 1 -- 172.21.15.105:0/3043216237 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc7dc004b70 con 0x7fc7e80adb80 2023-11-28T18:24:26.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.570+0000 7fc7d6ffd640 1 --2- 172.21.15.105:0/3043216237 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc7c8066900 0x7fc7c8068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:26.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.570+0000 7fc7d6ffd640 1 -- 172.21.15.105:0/3043216237 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fc7dc083ab0 con 0x7fc7e80adb80 2023-11-28T18:24:26.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.570+0000 7fc7e659c640 1 --2- 172.21.15.105:0/3043216237 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc7c8066900 0x7fc7c8068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:26.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.570+0000 7fc7e659c640 1 --2- 172.21.15.105:0/3043216237 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc7c8066900 0x7fc7c8068dc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80ad000 tx=0x7fc7d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:26.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.574+0000 7fc7d6ffd640 1 -- 172.21.15.105:0/3043216237 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc7e80ccc60 con 0x7fc7e80adb80 2023-11-28T18:24:26.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.726+0000 7fc7ec9aa640 1 -- 172.21.15.105:0/3043216237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc7e80ccc60 con 0x7fc7e80adb80 2023-11-28T18:24:26.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.726+0000 7fc7d6ffd640 1 -- 172.21.15.105:0/3043216237 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc7e80ccc60 con 0x7fc7e80adb80 2023-11-28T18:24:26.731 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:26.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.726+0000 7fc7ec9aa640 1 -- 172.21.15.105:0/3043216237 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc7c8066900 msgr2=0x7fc7c8068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:26.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.726+0000 7fc7ec9aa640 1 --2- 172.21.15.105:0/3043216237 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc7c8066900 0x7fc7c8068dc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80ad000 tx=0x7fc7d8008040 comp rx=0 tx=0).stop 2023-11-28T18:24:26.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.726+0000 7fc7ec9aa640 1 -- 172.21.15.105:0/3043216237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7e80adb80 msgr2=0x7fc7e814f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:26.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.726+0000 7fc7ec9aa640 1 --2- 172.21.15.105:0/3043216237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7e80adb80 0x7fc7e814f060 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc0011f0 tx=0x7fc7dc00c9b0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.726+0000 7fc7ec9aa640 1 -- 172.21.15.105:0/3043216237 shutdown_connections 2023-11-28T18:24:26.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.726+0000 7fc7ec9aa640 1 --2- 172.21.15.105:0/3043216237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7e8154070 0x7fc7e815fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.726+0000 7fc7ec9aa640 1 --2- 172.21.15.105:0/3043216237 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc7c8066900 0x7fc7c8068dc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.726+0000 7fc7ec9aa640 1 --2- 172.21.15.105:0/3043216237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7e80adb80 0x7fc7e814f060 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.726+0000 7fc7ec9aa640 1 --2- 172.21.15.105:0/3043216237 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7e806bf50 0x7fc7e814aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.726+0000 7fc7ec9aa640 1 -- 172.21.15.105:0/3043216237 >> 172.21.15.105:0/3043216237 conn(0x7fc7e80ceb90 msgr2=0x7fc7e80cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:26.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.730+0000 7fc7ec9aa640 1 -- 172.21.15.105:0/3043216237 shutdown_connections 2023-11-28T18:24:26.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.730+0000 7fc7ec9aa640 1 -- 172.21.15.105:0/3043216237 wait complete. 2023-11-28T18:24:26.753 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:24:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.886+0000 7f03b23e6640 1 Processor -- start 2023-11-28T18:24:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.886+0000 7f03b23e6640 1 -- start start 2023-11-28T18:24:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.890+0000 7f03b23e6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03ac0cc760 0x7f03ac164d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.890+0000 7f03b23e6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03ac0ccc10 con 0x7f03ac0caf60 2023-11-28T18:24:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.890+0000 7f03b23e6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03ac06bf50 con 0x7f03ac159f40 2023-11-28T18:24:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.890+0000 7f03b23e6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0cb3c0 con 0x7f03ac0cc760 2023-11-28T18:24:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.890+0000 7f03ab7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f03ac0caf60 0x7f03ac0ad850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49270/0 (socket says 172.21.15.105:49270) 2023-11-28T18:24:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.890+0000 7f03ab7fe640 1 -- 172.21.15.105:0/681771669 learned_addr learned my addr 172.21.15.105:0/681771669 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.890+0000 7f03b0983640 1 --2- 172.21.15.105:0/681771669 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03ac0cc760 0x7f03ac164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:26.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.890+0000 7f03aaffd640 1 -- 172.21.15.105:0/681771669 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129752294 0 0) 0x7f03ac0ccc10 con 0x7f03ac0caf60 2023-11-28T18:24:26.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.890+0000 7f03aaffd640 1 -- 172.21.15.105:0/681771669 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0394003530 con 0x7f03ac0caf60 2023-11-28T18:24:26.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.890+0000 7f03aaffd640 1 -- 172.21.15.105:0/681771669 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1112808419 0 0) 0x7f03ac06bf50 con 0x7f03ac159f40 2023-11-28T18:24:26.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.890+0000 7f03aaffd640 1 -- 172.21.15.105:0/681771669 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03ac0ccc10 con 0x7f03ac159f40 2023-11-28T18:24:26.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.890+0000 7f03b0983640 1 -- 172.21.15.105:0/681771669 >> v1:172.21.15.148:6789/0 conn(0x7f03ac159f40 legacy=0x7f03ac0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:26.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.890+0000 7f03b0983640 1 -- 172.21.15.105:0/681771669 >> v1:172.21.15.105:6789/0 conn(0x7f03ac0caf60 legacy=0x7f03ac0ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:26.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.890+0000 7f03aaffd640 1 -- 172.21.15.105:0/681771669 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1987962909 0 0) 0x7f0394003530 con 0x7f03ac0caf60 2023-11-28T18:24:26.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.890+0000 7f03b0983640 1 -- 172.21.15.105:0/681771669 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ac15a320 con 0x7f03ac0cc760 2023-11-28T18:24:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.894+0000 7f03b0983640 1 --2- 172.21.15.105:0/681771669 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03ac0cc760 0x7f03ac164d90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009dc0 tx=0x7f03a401c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=127ffc1156fc2630 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.894+0000 7f03aaffd640 1 -- 172.21.15.105:0/681771669 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a0001680 con 0x7f03ac0cc760 2023-11-28T18:24:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.894+0000 7f03aaffd640 1 -- 172.21.15.105:0/681771669 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a40a5610 con 0x7f03ac0cc760 2023-11-28T18:24:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.894+0000 7f03aaffd640 1 -- 172.21.15.105:0/681771669 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a40a5980 con 0x7f03ac0cc760 2023-11-28T18:24:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.894+0000 7f03b23e6640 1 -- 172.21.15.105:0/681771669 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03ac0cc760 msgr2=0x7f03ac164d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.894+0000 7f03b23e6640 1 --2- 172.21.15.105:0/681771669 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03ac0cc760 0x7f03ac164d90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009dc0 tx=0x7f03a401c7d0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.894+0000 7f03b23e6640 1 -- 172.21.15.105:0/681771669 shutdown_connections 2023-11-28T18:24:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.894+0000 7f03b23e6640 1 --2- 172.21.15.105:0/681771669 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03ac0cc760 0x7f03ac164d90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.894+0000 7f03b23e6640 1 -- 172.21.15.105:0/681771669 >> 172.21.15.105:0/681771669 conn(0x7f03ac0ceb90 msgr2=0x7f03ac0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.894+0000 7f03b23e6640 1 -- 172.21.15.105:0/681771669 shutdown_connections 2023-11-28T18:24:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.894+0000 7f03b23e6640 1 -- 172.21.15.105:0/681771669 wait complete. 2023-11-28T18:24:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.894+0000 7f03b23e6640 1 Processor -- start 2023-11-28T18:24:26.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.894+0000 7f03b23e6640 1 -- start start 2023-11-28T18:24:26.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03b23e6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03ac0caf60 0x7f03ac152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03b23e6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03ac0cc760 0x7f03ac142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03b23e6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03ac159f40 0x7f03ac142790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03b23e6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0af560 con 0x7f03ac159f40 2023-11-28T18:24:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03b23e6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0af800 con 0x7f03ac0caf60 2023-11-28T18:24:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03b23e6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0afae0 con 0x7f03ac0cc760 2023-11-28T18:24:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03b0983640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03ac159f40 0x7f03ac142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03b0983640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03ac159f40 0x7f03ac142790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43650/0 (socket says 172.21.15.105:43650) 2023-11-28T18:24:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03b0983640 1 -- 172.21.15.105:0/3379021238 learned_addr learned my addr 172.21.15.105:0/3379021238 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03ab7fe640 1 --2- 172.21.15.105:0/3379021238 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03ac0cc760 0x7f03ac142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03abfff640 1 --2- 172.21.15.105:0/3379021238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03ac0caf60 0x7f03ac152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03b0983640 1 -- 172.21.15.105:0/3379021238 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03ac0cc760 msgr2=0x7f03ac142250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03b0983640 1 --2- 172.21.15.105:0/3379021238 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03ac0cc760 0x7f03ac142250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03b0983640 1 -- 172.21.15.105:0/3379021238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03ac0caf60 msgr2=0x7f03ac152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03b0983640 1 --2- 172.21.15.105:0/3379021238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03ac0caf60 0x7f03ac152e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03b0983640 1 -- 172.21.15.105:0/3379021238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a4009a70 con 0x7f03ac159f40 2023-11-28T18:24:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03b0983640 1 --2- 172.21.15.105:0/3379021238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03ac159f40 0x7f03ac142790 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f03a401a400 tx=0x7f03a4019570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03a8ff9640 1 -- 172.21.15.105:0/3379021238 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a40a5b90 con 0x7f03ac159f40 2023-11-28T18:24:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03a8ff9640 1 -- 172.21.15.105:0/3379021238 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a40a5cf0 con 0x7f03ac159f40 2023-11-28T18:24:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03a8ff9640 1 -- 172.21.15.105:0/3379021238 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a4004240 con 0x7f03ac159f40 2023-11-28T18:24:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03b23e6640 1 -- 172.21.15.105:0/3379021238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ac0afdc0 con 0x7f03ac159f40 2023-11-28T18:24:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03b23e6640 1 -- 172.21.15.105:0/3379021238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ac153340 con 0x7f03ac159f40 2023-11-28T18:24:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.898+0000 7f03b23e6640 1 -- 172.21.15.105:0/3379021238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ac0b05c0 con 0x7f03ac159f40 2023-11-28T18:24:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.902+0000 7f03a8ff9640 1 -- 172.21.15.105:0/3379021238 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f03a40b4070 con 0x7f03ac159f40 2023-11-28T18:24:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.902+0000 7f03a8ff9640 1 --2- 172.21.15.105:0/3379021238 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f03980668d0 0x7f0398068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.902+0000 7f03a8ff9640 1 -- 172.21.15.105:0/3379021238 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f03a411f3d0 con 0x7f03ac159f40 2023-11-28T18:24:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.902+0000 7f03abfff640 1 --2- 172.21.15.105:0/3379021238 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f03980668d0 0x7f0398068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:26.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.902+0000 7f03abfff640 1 --2- 172.21.15.105:0/3379021238 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f03980668d0 0x7f0398068d90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f039c0011f0 tx=0x7f039c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:26.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:26.906+0000 7f03a8ff9640 1 -- 172.21.15.105:0/3379021238 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f03ac0b05c0 con 0x7f03ac159f40 2023-11-28T18:24:27.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.054+0000 7f03b23e6640 1 -- 172.21.15.105:0/3379021238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f03ac0b05c0 con 0x7f03ac159f40 2023-11-28T18:24:27.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.054+0000 7f03a8ff9640 1 -- 172.21.15.105:0/3379021238 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f03ac0b05c0 con 0x7f03ac159f40 2023-11-28T18:24:27.059 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:24:27.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.054+0000 7f03b23e6640 1 -- 172.21.15.105:0/3379021238 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f03980668d0 msgr2=0x7f0398068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:27.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.054+0000 7f03b23e6640 1 --2- 172.21.15.105:0/3379021238 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f03980668d0 0x7f0398068d90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f039c0011f0 tx=0x7f039c009040 comp rx=0 tx=0).stop 2023-11-28T18:24:27.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.054+0000 7f03b23e6640 1 -- 172.21.15.105:0/3379021238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03ac159f40 msgr2=0x7f03ac142790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:27.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.054+0000 7f03b23e6640 1 --2- 172.21.15.105:0/3379021238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03ac159f40 0x7f03ac142790 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f03a401a400 tx=0x7f03a4019570 comp rx=0 tx=0).stop 2023-11-28T18:24:27.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.054+0000 7f03b23e6640 1 -- 172.21.15.105:0/3379021238 shutdown_connections 2023-11-28T18:24:27.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.054+0000 7f03b23e6640 1 --2- 172.21.15.105:0/3379021238 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f03980668d0 0x7f0398068d90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:27.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.054+0000 7f03b23e6640 1 --2- 172.21.15.105:0/3379021238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03ac159f40 0x7f03ac142790 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:27.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.054+0000 7f03b23e6640 1 --2- 172.21.15.105:0/3379021238 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03ac0cc760 0x7f03ac142250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:27.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.054+0000 7f03b23e6640 1 --2- 172.21.15.105:0/3379021238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03ac0caf60 0x7f03ac152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:27.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.054+0000 7f03b23e6640 1 -- 172.21.15.105:0/3379021238 >> 172.21.15.105:0/3379021238 conn(0x7f03ac0ceb90 msgr2=0x7f03ac164b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:27.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.054+0000 7f03b23e6640 1 -- 172.21.15.105:0/3379021238 shutdown_connections 2023-11-28T18:24:27.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.054+0000 7f03b23e6640 1 -- 172.21.15.105:0/3379021238 wait complete. 2023-11-28T18:24:27.080 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T18:24:27.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.214+0000 7f595b25f640 1 Processor -- start 2023-11-28T18:24:27.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.214+0000 7f595b25f640 1 -- start start 2023-11-28T18:24:27.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.218+0000 7f595b25f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f595415d3c0 0x7f5954071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:27.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.218+0000 7f595b25f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5954070a10 0x7f5954070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:27.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.218+0000 7f595b25f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59540713a0 con 0x7f5954161870 2023-11-28T18:24:27.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.218+0000 7f595b25f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59540cefa0 con 0x7f595415d3c0 2023-11-28T18:24:27.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.218+0000 7f595b25f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595415d7a0 con 0x7f5954070a10 2023-11-28T18:24:27.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.218+0000 7f5958ffb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f595415d3c0 0x7f5954071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:27.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.218+0000 7f5958ffb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f595415d3c0 0x7f5954071a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43654/0 (socket says 172.21.15.105:43654) 2023-11-28T18:24:27.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.218+0000 7f5958ffb640 1 -- 172.21.15.105:0/2663783895 learned_addr learned my addr 172.21.15.105:0/2663783895 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:27.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.218+0000 7f59597fc640 1 --2- 172.21.15.105:0/2663783895 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5954070a10 0x7f5954070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:27.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.218+0000 7f5958ffb640 1 -- 172.21.15.105:0/2663783895 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5954070a10 msgr2=0x7f5954070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:27.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.218+0000 7f5958ffb640 1 --2- 172.21.15.105:0/2663783895 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5954070a10 0x7f5954070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:27.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.218+0000 7f5958ffb640 1 -- 172.21.15.105:0/2663783895 >> v1:172.21.15.148:6789/0 conn(0x7f5954161870 legacy=0x7f59540ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:27.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.218+0000 7f5958ffb640 1 -- 172.21.15.105:0/2663783895 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59540ce9e0 con 0x7f595415d3c0 2023-11-28T18:24:27.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.222+0000 7f5958ffb640 1 --2- 172.21.15.105:0/2663783895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f595415d3c0 0x7f5954071a10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5950009870 tx=0x7f595001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb9617d9e7bc4b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:27.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.222+0000 7f594b7fe640 1 -- 172.21.15.105:0/2663783895 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59500aa670 con 0x7f595415d3c0 2023-11-28T18:24:27.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.222+0000 7f594b7fe640 1 -- 172.21.15.105:0/2663783895 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59500aa7d0 con 0x7f595415d3c0 2023-11-28T18:24:27.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.222+0000 7f594b7fe640 1 -- 172.21.15.105:0/2663783895 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59500aab60 con 0x7f595415d3c0 2023-11-28T18:24:27.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.222+0000 7f595b25f640 1 -- 172.21.15.105:0/2663783895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f595415d3c0 msgr2=0x7f5954071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:27.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.222+0000 7f595b25f640 1 --2- 172.21.15.105:0/2663783895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f595415d3c0 0x7f5954071a10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5950009870 tx=0x7f595001c510 comp rx=0 tx=0).stop 2023-11-28T18:24:27.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.222+0000 7f595b25f640 1 -- 172.21.15.105:0/2663783895 shutdown_connections 2023-11-28T18:24:27.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.222+0000 7f595b25f640 1 --2- 172.21.15.105:0/2663783895 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5954070a10 0x7f5954070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:27.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.222+0000 7f595b25f640 1 --2- 172.21.15.105:0/2663783895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f595415d3c0 0x7f5954071a10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:27.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.222+0000 7f595b25f640 1 -- 172.21.15.105:0/2663783895 >> 172.21.15.105:0/2663783895 conn(0x7f59540cf540 msgr2=0x7f59540cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:27.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.222+0000 7f595b25f640 1 -- 172.21.15.105:0/2663783895 shutdown_connections 2023-11-28T18:24:27.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.222+0000 7f595b25f640 1 -- 172.21.15.105:0/2663783895 wait complete. 2023-11-28T18:24:27.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.222+0000 7f595b25f640 1 Processor -- start 2023-11-28T18:24:27.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.222+0000 7f595b25f640 1 -- start start 2023-11-28T18:24:27.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f595b25f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5954070a10 0x7f59541508b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:27.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f595b25f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f595415d3c0 0x7f595415dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:27.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f595b25f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5954161870 0x7f595415e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:27.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f595b25f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595415d8e0 con 0x7f5954070a10 2023-11-28T18:24:27.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f595b25f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59541524e0 con 0x7f5954161870 2023-11-28T18:24:27.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f595b25f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595414f860 con 0x7f595415d3c0 2023-11-28T18:24:27.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f5958ffb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5954070a10 0x7f59541508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:27.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f5958ffb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5954070a10 0x7f59541508b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43664/0 (socket says 172.21.15.105:43664) 2023-11-28T18:24:27.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f5958ffb640 1 -- 172.21.15.105:0/1515658805 learned_addr learned my addr 172.21.15.105:0/1515658805 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:27.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f594bfff640 1 --2- 172.21.15.105:0/1515658805 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f595415d3c0 0x7f595415dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:27.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f59597fc640 1 --2- 172.21.15.105:0/1515658805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5954161870 0x7f595415e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:27.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f5958ffb640 1 -- 172.21.15.105:0/1515658805 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f595415d3c0 msgr2=0x7f595415dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:27.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f5958ffb640 1 --2- 172.21.15.105:0/1515658805 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f595415d3c0 0x7f595415dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:27.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f5958ffb640 1 -- 172.21.15.105:0/1515658805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5954161870 msgr2=0x7f595415e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:27.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f5958ffb640 1 --2- 172.21.15.105:0/1515658805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5954161870 0x7f595415e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:27.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f5958ffb640 1 -- 172.21.15.105:0/1515658805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59500094d0 con 0x7f5954070a10 2023-11-28T18:24:27.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f59597fc640 1 --2- 172.21.15.105:0/1515658805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5954161870 0x7f595415e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:27.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f5958ffb640 1 --2- 172.21.15.105:0/1515658805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5954070a10 0x7f59541508b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f59500ac040 tx=0x7f595001ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:27.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f59497fa640 1 -- 172.21.15.105:0/1515658805 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59500b3800 con 0x7f5954070a10 2023-11-28T18:24:27.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f59497fa640 1 -- 172.21.15.105:0/1515658805 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59500b3980 con 0x7f5954070a10 2023-11-28T18:24:27.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f59497fa640 1 -- 172.21.15.105:0/1515658805 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59500b3d30 con 0x7f5954070a10 2023-11-28T18:24:27.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f595b25f640 1 -- 172.21.15.105:0/1515658805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5954152780 con 0x7f5954070a10 2023-11-28T18:24:27.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f595b25f640 1 -- 172.21.15.105:0/1515658805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5954152d40 con 0x7f5954070a10 2023-11-28T18:24:27.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f595b25f640 1 -- 172.21.15.105:0/1515658805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5954067570 con 0x7f5954070a10 2023-11-28T18:24:27.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.226+0000 7f59497fa640 1 -- 172.21.15.105:0/1515658805 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f59500b2380 con 0x7f5954070a10 2023-11-28T18:24:27.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.230+0000 7f59497fa640 1 --2- 172.21.15.105:0/1515658805 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59400668d0 0x7f5940068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:27.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.230+0000 7f59497fa640 1 -- 172.21.15.105:0/1515658805 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f595011f160 con 0x7f5954070a10 2023-11-28T18:24:27.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.230+0000 7f594bfff640 1 --2- 172.21.15.105:0/1515658805 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59400668d0 0x7f5940068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:27.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.230+0000 7f594bfff640 1 --2- 172.21.15.105:0/1515658805 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59400668d0 0x7f5940068d90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f59540afb20 tx=0x7f5944009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:27.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.234+0000 7f59497fa640 1 -- 172.21.15.105:0/1515658805 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f59500ec0e0 con 0x7f5954070a10 2023-11-28T18:24:27.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.386+0000 7f595b25f640 1 -- 172.21.15.105:0/1515658805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5954066960 con 0x7f5954070a10 2023-11-28T18:24:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.386+0000 7f59497fa640 1 -- 172.21.15.105:0/1515658805 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f59500ef700 con 0x7f5954070a10 2023-11-28T18:24:27.392 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.386+0000 7f595b25f640 1 -- 172.21.15.105:0/1515658805 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59400668d0 msgr2=0x7f5940068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.386+0000 7f595b25f640 1 --2- 172.21.15.105:0/1515658805 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59400668d0 0x7f5940068d90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f59540afb20 tx=0x7f5944009040 comp rx=0 tx=0).stop 2023-11-28T18:24:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.386+0000 7f595b25f640 1 -- 172.21.15.105:0/1515658805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5954070a10 msgr2=0x7f59541508b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.386+0000 7f595b25f640 1 --2- 172.21.15.105:0/1515658805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5954070a10 0x7f59541508b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f59500ac040 tx=0x7f595001ad50 comp rx=0 tx=0).stop 2023-11-28T18:24:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.386+0000 7f595b25f640 1 -- 172.21.15.105:0/1515658805 shutdown_connections 2023-11-28T18:24:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.386+0000 7f595b25f640 1 --2- 172.21.15.105:0/1515658805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5954161870 0x7f595415e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.386+0000 7f595b25f640 1 --2- 172.21.15.105:0/1515658805 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f595415d3c0 0x7f595415dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.386+0000 7f595b25f640 1 --2- 172.21.15.105:0/1515658805 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59400668d0 0x7f5940068d90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:27.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.386+0000 7f595b25f640 1 --2- 172.21.15.105:0/1515658805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5954070a10 0x7f59541508b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:27.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.386+0000 7f595b25f640 1 -- 172.21.15.105:0/1515658805 >> 172.21.15.105:0/1515658805 conn(0x7f59540cf540 msgr2=0x7f595406b140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:27.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.386+0000 7f595b25f640 1 -- 172.21.15.105:0/1515658805 shutdown_connections 2023-11-28T18:24:27.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.386+0000 7f595b25f640 1 -- 172.21.15.105:0/1515658805 wait complete. 2023-11-28T18:24:27.411 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T18:24:27.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.546+0000 7fa1f9009640 1 Processor -- start 2023-11-28T18:24:27.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.546+0000 7fa1f9009640 1 -- start start 2023-11-28T18:24:27.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.546+0000 7fa1f9009640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1f4071960 0x7fa1f415f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:27.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.546+0000 7fa1f9009640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f40701d0 con 0x7fa1f415b030 2023-11-28T18:24:27.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.546+0000 7fa1f9009640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f406df80 con 0x7fa1f40b4290 2023-11-28T18:24:27.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.546+0000 7fa1f9009640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f4071df0 con 0x7fa1f4071960 2023-11-28T18:24:27.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.546+0000 7fa1f359e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa1f415b030 0x7fa1f40709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49276/0 (socket says 172.21.15.105:49276) 2023-11-28T18:24:27.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.546+0000 7fa1f359e640 1 -- 172.21.15.105:0/335493252 learned_addr learned my addr 172.21.15.105:0/335493252 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:27.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.546+0000 7fa1ea59c640 1 --2- 172.21.15.105:0/335493252 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1f4071960 0x7fa1f415f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:27.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.546+0000 7fa1f259c640 1 -- 172.21.15.105:0/335493252 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365584006 0 0) 0x7fa1f40701d0 con 0x7fa1f415b030 2023-11-28T18:24:27.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.546+0000 7fa1f259c640 1 -- 172.21.15.105:0/335493252 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1e0003530 con 0x7fa1f415b030 2023-11-28T18:24:27.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.546+0000 7fa1f259c640 1 -- 172.21.15.105:0/335493252 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 466908239 0 0) 0x7fa1f406df80 con 0x7fa1f40b4290 2023-11-28T18:24:27.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.546+0000 7fa1f259c640 1 -- 172.21.15.105:0/335493252 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1f40701d0 con 0x7fa1f40b4290 2023-11-28T18:24:27.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.546+0000 7fa1ea59c640 1 -- 172.21.15.105:0/335493252 >> v1:172.21.15.148:6789/0 conn(0x7fa1f40b4290 legacy=0x7fa1f40b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:27.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.546+0000 7fa1ea59c640 1 -- 172.21.15.105:0/335493252 >> v1:172.21.15.105:6789/0 conn(0x7fa1f415b030 legacy=0x7fa1f40709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:27.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.546+0000 7fa1ea59c640 1 -- 172.21.15.105:0/335493252 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f40720d0 con 0x7fa1f4071960 2023-11-28T18:24:27.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.550+0000 7fa1ea59c640 1 --2- 172.21.15.105:0/335493252 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1f4071960 0x7fa1f415f3b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0097f0 tx=0x7fa1dc01c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b619e2d9e2a0382 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.550+0000 7fa1f259c640 1 -- 172.21.15.105:0/335493252 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc0aa5f0 con 0x7fa1f4071960 2023-11-28T18:24:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.550+0000 7fa1f259c640 1 -- 172.21.15.105:0/335493252 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1dc0aa750 con 0x7fa1f4071960 2023-11-28T18:24:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.550+0000 7fa1f259c640 1 -- 172.21.15.105:0/335493252 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc0aaae0 con 0x7fa1f4071960 2023-11-28T18:24:27.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.550+0000 7fa1f9009640 1 -- 172.21.15.105:0/335493252 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1f4071960 msgr2=0x7fa1f415f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:27.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.550+0000 7fa1f9009640 1 --2- 172.21.15.105:0/335493252 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1f4071960 0x7fa1f415f3b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0097f0 tx=0x7fa1dc01c490 comp rx=0 tx=0).stop 2023-11-28T18:24:27.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.550+0000 7fa1f9009640 1 -- 172.21.15.105:0/335493252 shutdown_connections 2023-11-28T18:24:27.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.550+0000 7fa1f9009640 1 --2- 172.21.15.105:0/335493252 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1f4071960 0x7fa1f415f3b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:27.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.550+0000 7fa1f9009640 1 -- 172.21.15.105:0/335493252 >> 172.21.15.105:0/335493252 conn(0x7fa1f40ceb90 msgr2=0x7fa1f40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:27.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.550+0000 7fa1f9009640 1 -- 172.21.15.105:0/335493252 shutdown_connections 2023-11-28T18:24:27.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.550+0000 7fa1f9009640 1 -- 172.21.15.105:0/335493252 wait complete. 2023-11-28T18:24:27.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1f9009640 1 Processor -- start 2023-11-28T18:24:27.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1f9009640 1 -- start start 2023-11-28T18:24:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1f9009640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1f40b4290 0x7fa1f4099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1f9009640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1f415b030 0x7fa1f4153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1f9009640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f40b3e50 0x7fa1f40ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1f9009640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f40b2160 con 0x7fa1f415b030 2023-11-28T18:24:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1f9009640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f409a940 con 0x7fa1f40b3e50 2023-11-28T18:24:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1f9009640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f409a380 con 0x7fa1f40b4290 2023-11-28T18:24:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1ea59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1f415b030 0x7fa1f4153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1ea59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1f415b030 0x7fa1f4153d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43668/0 (socket says 172.21.15.105:43668) 2023-11-28T18:24:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1ea59c640 1 -- 172.21.15.105:0/2610623411 learned_addr learned my addr 172.21.15.105:0/2610623411 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1f359e640 1 --2- 172.21.15.105:0/2610623411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f40b3e50 0x7fa1f40ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1f2d9d640 1 --2- 172.21.15.105:0/2610623411 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1f40b4290 0x7fa1f4099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1ea59c640 1 -- 172.21.15.105:0/2610623411 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1f40b4290 msgr2=0x7fa1f4099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1ea59c640 1 --2- 172.21.15.105:0/2610623411 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1f40b4290 0x7fa1f4099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1ea59c640 1 -- 172.21.15.105:0/2610623411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f40b3e50 msgr2=0x7fa1f40ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:27.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1ea59c640 1 --2- 172.21.15.105:0/2610623411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f40b3e50 0x7fa1f40ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:27.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1ea59c640 1 -- 172.21.15.105:0/2610623411 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc009420 con 0x7fa1f415b030 2023-11-28T18:24:27.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1f2d9d640 1 --2- 172.21.15.105:0/2610623411 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1f40b4290 0x7fa1f4099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:27.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1ea59c640 1 --2- 172.21.15.105:0/2610623411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1f415b030 0x7fa1f4153d40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0190a0 tx=0x7fa1dc01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:27.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1ebfff640 1 -- 172.21.15.105:0/2610623411 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc0be610 con 0x7fa1f415b030 2023-11-28T18:24:27.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1ebfff640 1 -- 172.21.15.105:0/2610623411 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1dc0be790 con 0x7fa1f415b030 2023-11-28T18:24:27.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1f9009640 1 -- 172.21.15.105:0/2610623411 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f409a660 con 0x7fa1f415b030 2023-11-28T18:24:27.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1ebfff640 1 -- 172.21.15.105:0/2610623411 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc0beb40 con 0x7fa1f415b030 2023-11-28T18:24:27.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1f9009640 1 -- 172.21.15.105:0/2610623411 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f4099f90 con 0x7fa1f415b030 2023-11-28T18:24:27.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.554+0000 7fa1f9009640 1 -- 172.21.15.105:0/2610623411 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f40b1be0 con 0x7fa1f415b030 2023-11-28T18:24:27.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.558+0000 7fa1ebfff640 1 -- 172.21.15.105:0/2610623411 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa1dc0beca0 con 0x7fa1f415b030 2023-11-28T18:24:27.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.558+0000 7fa1ebfff640 1 --2- 172.21.15.105:0/2610623411 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1d40668d0 0x7fa1d4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:27.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.558+0000 7fa1ebfff640 1 -- 172.21.15.105:0/2610623411 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fa1dc0a7070 con 0x7fa1f415b030 2023-11-28T18:24:27.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.558+0000 7fa1f2d9d640 1 --2- 172.21.15.105:0/2610623411 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1d40668d0 0x7fa1d4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:27.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.558+0000 7fa1f2d9d640 1 --2- 172.21.15.105:0/2610623411 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1d40668d0 0x7fa1d4068d90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4001570 tx=0x7fa1e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:27.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.562+0000 7fa1ebfff640 1 -- 172.21.15.105:0/2610623411 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa1f40b1be0 con 0x7fa1f415b030 2023-11-28T18:24:27.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.722+0000 7fa1f9009640 1 -- 172.21.15.105:0/2610623411 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa1f40b22a0 con 0x7fa1f415b030 2023-11-28T18:24:27.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.722+0000 7fa1ebfff640 1 -- 172.21.15.105:0/2610623411 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa1f40b22a0 con 0x7fa1f415b030 2023-11-28T18:24:27.728 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2023-11-28T18:24:27.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.726+0000 7fa1f9009640 1 -- 172.21.15.105:0/2610623411 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1d40668d0 msgr2=0x7fa1d4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:27.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.726+0000 7fa1f9009640 1 --2- 172.21.15.105:0/2610623411 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1d40668d0 0x7fa1d4068d90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4001570 tx=0x7fa1e4009040 comp rx=0 tx=0).stop 2023-11-28T18:24:27.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.726+0000 7fa1f9009640 1 -- 172.21.15.105:0/2610623411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1f415b030 msgr2=0x7fa1f4153d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:27.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.726+0000 7fa1f9009640 1 --2- 172.21.15.105:0/2610623411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1f415b030 0x7fa1f4153d40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0190a0 tx=0x7fa1dc01acd0 comp rx=0 tx=0).stop 2023-11-28T18:24:27.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.726+0000 7fa1f9009640 1 -- 172.21.15.105:0/2610623411 shutdown_connections 2023-11-28T18:24:27.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.726+0000 7fa1f9009640 1 --2- 172.21.15.105:0/2610623411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f40b3e50 0x7fa1f40ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:27.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.726+0000 7fa1f9009640 1 --2- 172.21.15.105:0/2610623411 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1d40668d0 0x7fa1d4068d90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:27.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.726+0000 7fa1f9009640 1 --2- 172.21.15.105:0/2610623411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1f415b030 0x7fa1f4153d40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:27.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.726+0000 7fa1f9009640 1 --2- 172.21.15.105:0/2610623411 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1f40b4290 0x7fa1f4099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:27.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.726+0000 7fa1f9009640 1 -- 172.21.15.105:0/2610623411 >> 172.21.15.105:0/2610623411 conn(0x7fa1f40ceb90 msgr2=0x7fa1f40cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:27.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.726+0000 7fa1f9009640 1 -- 172.21.15.105:0/2610623411 shutdown_connections 2023-11-28T18:24:27.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:27.726+0000 7fa1f9009640 1 -- 172.21.15.105:0/2610623411 wait complete. 2023-11-28T18:24:27.749 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2023-11-28T18:24:27.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:27.750 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-28T18:24:27.774 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:24:27.774 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:24:27.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:27.775 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T18:24:27.826 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:24:27.826 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:24:27.836 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:24:27.837 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:24:27.887 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:24:28.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.018+0000 7ff1b629a640 1 Processor -- start 2023-11-28T18:24:28.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.018+0000 7ff1b629a640 1 -- start start 2023-11-28T18:24:28.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.022+0000 7ff1b629a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff1b0159610 0x7ff1b00ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:28.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.022+0000 7ff1b629a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff1b006cef0 0x7ff1b00ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:28.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.022+0000 7ff1b629a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b00cde30 con 0x7ff1b00cc760 2023-11-28T18:24:28.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.022+0000 7ff1b629a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b01599f0 con 0x7ff1b0159610 2023-11-28T18:24:28.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.022+0000 7ff1b629a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b006de20 con 0x7ff1b006cef0 2023-11-28T18:24:28.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.022+0000 7ff1affff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff1b0159610 0x7ff1b00ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:28.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.022+0000 7ff1af7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff1b00cc760 0x7ff1b006c7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49278/0 (socket says 172.21.15.105:49278) 2023-11-28T18:24:28.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.022+0000 7ff1af7fe640 1 -- 172.21.15.105:0/2367582270 learned_addr learned my addr 172.21.15.105:0/2367582270 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:28.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.022+0000 7ff1b4837640 1 --2- 172.21.15.105:0/2367582270 >> v2:172.21.15.148:3300/0 conn(0x7ff1b006cef0 0x7ff1b00ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:28.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.022+0000 7ff1aeffd640 1 -- 172.21.15.105:0/2367582270 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4066991967 0 0) 0x7ff1b00cde30 con 0x7ff1b00cc760 2023-11-28T18:24:28.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.022+0000 7ff1affff640 1 -- 172.21.15.105:0/2367582270 >> v2:172.21.15.148:3300/0 conn(0x7ff1b006cef0 msgr2=0x7ff1b00ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:28.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.022+0000 7ff1affff640 1 --2- 172.21.15.105:0/2367582270 >> v2:172.21.15.148:3300/0 conn(0x7ff1b006cef0 0x7ff1b00ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.022+0000 7ff1affff640 1 -- 172.21.15.105:0/2367582270 >> v1:172.21.15.105:6789/0 conn(0x7ff1b00cc760 legacy=0x7ff1b006c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:28.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.022+0000 7ff1affff640 1 -- 172.21.15.105:0/2367582270 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b00cd220 con 0x7ff1b0159610 2023-11-28T18:24:28.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.026+0000 7ff1affff640 1 --2- 172.21.15.105:0/2367582270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1b0159610 0x7ff1b00ce2c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff1b006e600 tx=0x7ff1a001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83a0dcc15dabed7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:28.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.026+0000 7ff1aeffd640 1 -- 172.21.15.105:0/2367582270 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a00ac410 con 0x7ff1b0159610 2023-11-28T18:24:28.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.026+0000 7ff1aeffd640 1 -- 172.21.15.105:0/2367582270 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a00ac570 con 0x7ff1b0159610 2023-11-28T18:24:28.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.026+0000 7ff1aeffd640 1 -- 172.21.15.105:0/2367582270 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a00ac900 con 0x7ff1b0159610 2023-11-28T18:24:28.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.026+0000 7ff1b629a640 1 -- 172.21.15.105:0/2367582270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1b0159610 msgr2=0x7ff1b00ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:28.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.026+0000 7ff1b629a640 1 --2- 172.21.15.105:0/2367582270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1b0159610 0x7ff1b00ce2c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff1b006e600 tx=0x7ff1a001c430 comp rx=0 tx=0).stop 2023-11-28T18:24:28.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.026+0000 7ff1b629a640 1 -- 172.21.15.105:0/2367582270 shutdown_connections 2023-11-28T18:24:28.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.026+0000 7ff1b629a640 1 --2- 172.21.15.105:0/2367582270 >> v2:172.21.15.148:3300/0 conn(0x7ff1b006cef0 0x7ff1b00ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.026+0000 7ff1b629a640 1 --2- 172.21.15.105:0/2367582270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1b0159610 0x7ff1b00ce2c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.026+0000 7ff1b629a640 1 -- 172.21.15.105:0/2367582270 >> 172.21.15.105:0/2367582270 conn(0x7ff1b00ceb90 msgr2=0x7ff1b00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:28.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.026+0000 7ff1b629a640 1 -- 172.21.15.105:0/2367582270 shutdown_connections 2023-11-28T18:24:28.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.026+0000 7ff1b629a640 1 -- 172.21.15.105:0/2367582270 wait complete. 2023-11-28T18:24:28.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.026+0000 7ff1b629a640 1 Processor -- start 2023-11-28T18:24:28.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.026+0000 7ff1b629a640 1 -- start start 2023-11-28T18:24:28.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.026+0000 7ff1b629a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1b006cef0 0x7ff1b0140c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:28.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.026+0000 7ff1b629a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b00cc760 0x7ff1b00aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:28.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.026+0000 7ff1b629a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1b0159610 0x7ff1b00ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:28.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.026+0000 7ff1b629a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b01f3bf0 con 0x7ff1b006cef0 2023-11-28T18:24:28.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.026+0000 7ff1b629a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b0141170 con 0x7ff1b00cc760 2023-11-28T18:24:28.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.026+0000 7ff1b629a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b01f7650 con 0x7ff1b0159610 2023-11-28T18:24:28.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.026+0000 7ff1affff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1b006cef0 0x7ff1b0140c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:28.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.026+0000 7ff1affff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1b006cef0 0x7ff1b0140c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43684/0 (socket says 172.21.15.105:43684) 2023-11-28T18:24:28.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.030+0000 7ff1affff640 1 -- 172.21.15.105:0/4035839659 learned_addr learned my addr 172.21.15.105:0/4035839659 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:28.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.030+0000 7ff1af7fe640 1 --2- 172.21.15.105:0/4035839659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b00cc760 0x7ff1b00aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:28.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.030+0000 7ff1b4837640 1 --2- 172.21.15.105:0/4035839659 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1b0159610 0x7ff1b00ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:28.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.030+0000 7ff1affff640 1 -- 172.21.15.105:0/4035839659 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1b0159610 msgr2=0x7ff1b00ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:28.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.030+0000 7ff1affff640 1 --2- 172.21.15.105:0/4035839659 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1b0159610 0x7ff1b00ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.030+0000 7ff1affff640 1 -- 172.21.15.105:0/4035839659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b00cc760 msgr2=0x7ff1b00aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:28.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.030+0000 7ff1affff640 1 --2- 172.21.15.105:0/4035839659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b00cc760 0x7ff1b00aca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.030+0000 7ff1affff640 1 -- 172.21.15.105:0/4035839659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a0009420 con 0x7ff1b006cef0 2023-11-28T18:24:28.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.030+0000 7ff1b4837640 1 --2- 172.21.15.105:0/4035839659 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1b0159610 0x7ff1b00ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:28.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.030+0000 7ff1affff640 1 --2- 172.21.15.105:0/4035839659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1b006cef0 0x7ff1b0140c30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff1a00af040 tx=0x7ff1a00191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:28.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.030+0000 7ff1acff9640 1 -- 172.21.15.105:0/4035839659 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a00ac410 con 0x7ff1b006cef0 2023-11-28T18:24:28.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.030+0000 7ff1acff9640 1 -- 172.21.15.105:0/4035839659 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a0006e20 con 0x7ff1b006cef0 2023-11-28T18:24:28.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.030+0000 7ff1b629a640 1 -- 172.21.15.105:0/4035839659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b01f7930 con 0x7ff1b006cef0 2023-11-28T18:24:28.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.030+0000 7ff1acff9640 1 -- 172.21.15.105:0/4035839659 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a00b5860 con 0x7ff1b006cef0 2023-11-28T18:24:28.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.030+0000 7ff1b629a640 1 -- 172.21.15.105:0/4035839659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b01f7ef0 con 0x7ff1b006cef0 2023-11-28T18:24:28.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.030+0000 7ff1b629a640 1 -- 172.21.15.105:0/4035839659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b0067570 con 0x7ff1b006cef0 2023-11-28T18:24:28.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.030+0000 7ff1acff9640 1 -- 172.21.15.105:0/4035839659 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff1a00b3040 con 0x7ff1b006cef0 2023-11-28T18:24:28.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.030+0000 7ff1acff9640 1 --2- 172.21.15.105:0/4035839659 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff19c066900 0x7ff19c068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:28.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.030+0000 7ff1acff9640 1 -- 172.21.15.105:0/4035839659 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7ff1a011e850 con 0x7ff1b006cef0 2023-11-28T18:24:28.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.034+0000 7ff1af7fe640 1 --2- 172.21.15.105:0/4035839659 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff19c066900 0x7ff19c068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:28.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.034+0000 7ff1af7fe640 1 --2- 172.21.15.105:0/4035839659 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff19c066900 0x7ff19c068dc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0153250 tx=0x7ff1a4007b60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:28.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.034+0000 7ff1acff9640 1 -- 172.21.15.105:0/4035839659 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff1a00eb5f0 con 0x7ff1b006cef0 2023-11-28T18:24:28.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.190+0000 7ff1b629a640 1 -- 172.21.15.105:0/4035839659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1b00b3d20 con 0x7ff1b006cef0 2023-11-28T18:24:28.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.190+0000 7ff1acff9640 1 -- 172.21.15.105:0/4035839659 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18887 (secure 0 0 0) 0x7ff1a00eec10 con 0x7ff1b006cef0 2023-11-28T18:24:28.195 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:28.195 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:24:15.048168+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:24:28.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.190+0000 7ff1b629a640 1 -- 172.21.15.105:0/4035839659 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff19c066900 msgr2=0x7ff19c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:28.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.190+0000 7ff1b629a640 1 --2- 172.21.15.105:0/4035839659 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff19c066900 0x7ff19c068dc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0153250 tx=0x7ff1a4007b60 comp rx=0 tx=0).stop 2023-11-28T18:24:28.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.190+0000 7ff1b629a640 1 -- 172.21.15.105:0/4035839659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1b006cef0 msgr2=0x7ff1b0140c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:28.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.190+0000 7ff1b629a640 1 --2- 172.21.15.105:0/4035839659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1b006cef0 0x7ff1b0140c30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff1a00af040 tx=0x7ff1a00191d0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.190+0000 7ff1b629a640 1 -- 172.21.15.105:0/4035839659 shutdown_connections 2023-11-28T18:24:28.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.190+0000 7ff1b629a640 1 --2- 172.21.15.105:0/4035839659 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1b0159610 0x7ff1b00ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.190+0000 7ff1b629a640 1 --2- 172.21.15.105:0/4035839659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b00cc760 0x7ff1b00aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.190+0000 7ff1b629a640 1 --2- 172.21.15.105:0/4035839659 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff19c066900 0x7ff19c068dc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.190+0000 7ff1b629a640 1 --2- 172.21.15.105:0/4035839659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1b006cef0 0x7ff1b0140c30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.190+0000 7ff1b629a640 1 -- 172.21.15.105:0/4035839659 >> 172.21.15.105:0/4035839659 conn(0x7ff1b00ceb90 msgr2=0x7ff1b00b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:28.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.190+0000 7ff1b629a640 1 -- 172.21.15.105:0/4035839659 shutdown_connections 2023-11-28T18:24:28.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.190+0000 7ff1b629a640 1 -- 172.21.15.105:0/4035839659 wait complete. 2023-11-28T18:24:28.217 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:23:31.485422+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:23:32.274378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T18:23:36.100071+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T18:24:28.218 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:24:28.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.354+0000 7f5f46a79640 1 Processor -- start 2023-11-28T18:24:28.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.354+0000 7f5f46a79640 1 -- start start 2023-11-28T18:24:28.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.358+0000 7f5f46a79640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5f4006cef0 0x7f5f400abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:28.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.358+0000 7f5f46a79640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f400cde30 con 0x7f5f400ce2c0 2023-11-28T18:24:28.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.358+0000 7f5f46a79640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f401599f0 con 0x7f5f40159610 2023-11-28T18:24:28.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.358+0000 7f5f46a79640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4006de20 con 0x7f5f4006cef0 2023-11-28T18:24:28.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.358+0000 7f5f3ffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5f400ce2c0 0x7f5f4006c7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49280/0 (socket says 172.21.15.105:49280) 2023-11-28T18:24:28.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.358+0000 7f5f3ffff640 1 -- 172.21.15.105:0/3753974942 learned_addr learned my addr 172.21.15.105:0/3753974942 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:28.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.358+0000 7f5f45016640 1 --2- 172.21.15.105:0/3753974942 >> v2:172.21.15.105:3300/0 conn(0x7f5f4006cef0 0x7f5f400abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:28.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.358+0000 7f5f3f7fe640 1 -- 172.21.15.105:0/3753974942 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892676315 0 0) 0x7f5f400cde30 con 0x7f5f400ce2c0 2023-11-28T18:24:28.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.358+0000 7f5f3f7fe640 1 -- 172.21.15.105:0/3753974942 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f30003530 con 0x7f5f400ce2c0 2023-11-28T18:24:28.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.358+0000 7f5f45016640 1 -- 172.21.15.105:0/3753974942 >> v1:172.21.15.148:6789/0 conn(0x7f5f40159610 legacy=0x7f5f400cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:28.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.358+0000 7f5f45016640 1 -- 172.21.15.105:0/3753974942 >> v1:172.21.15.105:6789/0 conn(0x7f5f400ce2c0 legacy=0x7f5f4006c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:28.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.358+0000 7f5f45016640 1 -- 172.21.15.105:0/3753974942 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f400cd220 con 0x7f5f4006cef0 2023-11-28T18:24:28.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.362+0000 7f5f45016640 1 --2- 172.21.15.105:0/3753974942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f4006cef0 0x7f5f400abd80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009d90 tx=0x7f5f3801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48c8d563cf736f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:28.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.362+0000 7f5f3f7fe640 1 -- 172.21.15.105:0/3753974942 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f380a5870 con 0x7f5f4006cef0 2023-11-28T18:24:28.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.362+0000 7f5f3f7fe640 1 -- 172.21.15.105:0/3753974942 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f380a59d0 con 0x7f5f4006cef0 2023-11-28T18:24:28.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.362+0000 7f5f3f7fe640 1 -- 172.21.15.105:0/3753974942 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f380a5d60 con 0x7f5f4006cef0 2023-11-28T18:24:28.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.362+0000 7f5f46a79640 1 -- 172.21.15.105:0/3753974942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f4006cef0 msgr2=0x7f5f400abd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:28.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.362+0000 7f5f46a79640 1 --2- 172.21.15.105:0/3753974942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f4006cef0 0x7f5f400abd80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009d90 tx=0x7f5f3801c920 comp rx=0 tx=0).stop 2023-11-28T18:24:28.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.362+0000 7f5f46a79640 1 -- 172.21.15.105:0/3753974942 shutdown_connections 2023-11-28T18:24:28.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.362+0000 7f5f46a79640 1 --2- 172.21.15.105:0/3753974942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f4006cef0 0x7f5f400abd80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.362+0000 7f5f46a79640 1 -- 172.21.15.105:0/3753974942 >> 172.21.15.105:0/3753974942 conn(0x7f5f400ceb90 msgr2=0x7f5f400cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:28.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.362+0000 7f5f46a79640 1 -- 172.21.15.105:0/3753974942 shutdown_connections 2023-11-28T18:24:28.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.362+0000 7f5f46a79640 1 -- 172.21.15.105:0/3753974942 wait complete. 2023-11-28T18:24:28.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.362+0000 7f5f46a79640 1 Processor -- start 2023-11-28T18:24:28.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.362+0000 7f5f46a79640 1 -- start start 2023-11-28T18:24:28.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f46a79640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f4006cef0 0x7f5f40147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:28.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f46a79640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f400ce2c0 0x7f5f40159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:28.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f46a79640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f40159610 0x7f5f40157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:28.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f46a79640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f40147c80 con 0x7f5f4006cef0 2023-11-28T18:24:28.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f46a79640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f400aec00 con 0x7f5f40159610 2023-11-28T18:24:28.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f46a79640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f400aeee0 con 0x7f5f400ce2c0 2023-11-28T18:24:28.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f44815640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f4006cef0 0x7f5f40147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:28.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f44815640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f4006cef0 0x7f5f40147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43704/0 (socket says 172.21.15.105:43704) 2023-11-28T18:24:28.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f44815640 1 -- 172.21.15.105:0/1158717502 learned_addr learned my addr 172.21.15.105:0/1158717502 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:28.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f45016640 1 --2- 172.21.15.105:0/1158717502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f40159610 0x7f5f40157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:28.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f44815640 1 -- 172.21.15.105:0/1158717502 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f400ce2c0 msgr2=0x7f5f40159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:28.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f3ffff640 1 --2- 172.21.15.105:0/1158717502 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f400ce2c0 0x7f5f40159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:28.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f44815640 1 --2- 172.21.15.105:0/1158717502 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f400ce2c0 0x7f5f40159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f44815640 1 -- 172.21.15.105:0/1158717502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f40159610 msgr2=0x7f5f40157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:28.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f44815640 1 --2- 172.21.15.105:0/1158717502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f40159610 0x7f5f40157dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f44815640 1 -- 172.21.15.105:0/1158717502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f38009a70 con 0x7f5f4006cef0 2023-11-28T18:24:28.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f45016640 1 --2- 172.21.15.105:0/1158717502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f40159610 0x7f5f40157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:28.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f3ffff640 1 --2- 172.21.15.105:0/1158717502 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f400ce2c0 0x7f5f40159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:28.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f44815640 1 --2- 172.21.15.105:0/1158717502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f4006cef0 0x7f5f40147740 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400c3c0 tx=0x7f5f3400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:28.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f3d7fa640 1 -- 172.21.15.105:0/1158717502 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f340052d0 con 0x7f5f4006cef0 2023-11-28T18:24:28.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f3d7fa640 1 -- 172.21.15.105:0/1158717502 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f34005430 con 0x7f5f4006cef0 2023-11-28T18:24:28.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f46a79640 1 -- 172.21.15.105:0/1158717502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f400ccff0 con 0x7f5f4006cef0 2023-11-28T18:24:28.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f3d7fa640 1 -- 172.21.15.105:0/1158717502 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34003840 con 0x7f5f4006cef0 2023-11-28T18:24:28.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f46a79640 1 -- 172.21.15.105:0/1158717502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f400af4a0 con 0x7f5f4006cef0 2023-11-28T18:24:28.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f46a79640 1 -- 172.21.15.105:0/1158717502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4014d050 con 0x7f5f4006cef0 2023-11-28T18:24:28.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.366+0000 7f5f3d7fa640 1 -- 172.21.15.105:0/1158717502 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5f340039a0 con 0x7f5f4006cef0 2023-11-28T18:24:28.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.370+0000 7f5f3d7fa640 1 --2- 172.21.15.105:0/1158717502 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f240668d0 0x7f5f24068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:28.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.370+0000 7f5f3d7fa640 1 -- 172.21.15.105:0/1158717502 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f5f34083e60 con 0x7f5f4006cef0 2023-11-28T18:24:28.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.370+0000 7f5f3ffff640 1 --2- 172.21.15.105:0/1158717502 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f240668d0 0x7f5f24068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:28.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.370+0000 7f5f3ffff640 1 --2- 172.21.15.105:0/1158717502 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f240668d0 0x7f5f24068d90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5f4014cfe0 tx=0x7f5f2c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:28.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.374+0000 7f5f3d7fa640 1 -- 172.21.15.105:0/1158717502 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5f34050c00 con 0x7f5f4006cef0 2023-11-28T18:24:28.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.522+0000 7f5f46a79640 1 -- 172.21.15.105:0/1158717502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f40067570 con 0x7f5f4006cef0 2023-11-28T18:24:28.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.522+0000 7f5f3d7fa640 1 -- 172.21.15.105:0/1158717502 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f340170a0 con 0x7f5f4006cef0 2023-11-28T18:24:28.528 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:24:28.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.526+0000 7f5f46a79640 1 -- 172.21.15.105:0/1158717502 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f240668d0 msgr2=0x7f5f24068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:28.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.526+0000 7f5f46a79640 1 --2- 172.21.15.105:0/1158717502 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f240668d0 0x7f5f24068d90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5f4014cfe0 tx=0x7f5f2c009040 comp rx=0 tx=0).stop 2023-11-28T18:24:28.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.526+0000 7f5f46a79640 1 -- 172.21.15.105:0/1158717502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f4006cef0 msgr2=0x7f5f40147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:28.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.526+0000 7f5f46a79640 1 --2- 172.21.15.105:0/1158717502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f4006cef0 0x7f5f40147740 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400c3c0 tx=0x7f5f3400c880 comp rx=0 tx=0).stop 2023-11-28T18:24:28.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.526+0000 7f5f46a79640 1 -- 172.21.15.105:0/1158717502 shutdown_connections 2023-11-28T18:24:28.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.526+0000 7f5f46a79640 1 --2- 172.21.15.105:0/1158717502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f40159610 0x7f5f40157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.526+0000 7f5f46a79640 1 --2- 172.21.15.105:0/1158717502 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f400ce2c0 0x7f5f40159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.526+0000 7f5f46a79640 1 --2- 172.21.15.105:0/1158717502 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f240668d0 0x7f5f24068d90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.526+0000 7f5f46a79640 1 --2- 172.21.15.105:0/1158717502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f4006cef0 0x7f5f40147740 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.526+0000 7f5f46a79640 1 -- 172.21.15.105:0/1158717502 >> 172.21.15.105:0/1158717502 conn(0x7f5f400ceb90 msgr2=0x7f5f4014f310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:28.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.526+0000 7f5f46a79640 1 -- 172.21.15.105:0/1158717502 shutdown_connections 2023-11-28T18:24:28.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.526+0000 7f5f46a79640 1 -- 172.21.15.105:0/1158717502 wait complete. 2023-11-28T18:24:28.550 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:24:28.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.682+0000 7ff47bf80640 1 Processor -- start 2023-11-28T18:24:28.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff47bf80640 1 -- start start 2023-11-28T18:24:28.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff47bf80640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff47415b030 0x7ff4740709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:28.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff47bf80640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4740701d0 con 0x7ff4740b4290 2023-11-28T18:24:28.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff47bf80640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff47406df80 con 0x7ff474071960 2023-11-28T18:24:28.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff47bf80640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff474071df0 con 0x7ff47415b030 2023-11-28T18:24:28.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff479d1c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff4740b4290 0x7ff4740b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49292/0 (socket says 172.21.15.105:49292) 2023-11-28T18:24:28.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff479d1c640 1 -- 172.21.15.105:0/1241012980 learned_addr learned my addr 172.21.15.105:0/1241012980 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:28.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff47a51d640 1 --2- 172.21.15.105:0/1241012980 >> v2:172.21.15.148:3300/0 conn(0x7ff47415b030 0x7ff4740709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:28.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff478d1a640 1 -- 172.21.15.105:0/1241012980 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837065047 0 0) 0x7ff4740701d0 con 0x7ff4740b4290 2023-11-28T18:24:28.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff478d1a640 1 -- 172.21.15.105:0/1241012980 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff460003530 con 0x7ff4740b4290 2023-11-28T18:24:28.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff478d1a640 1 -- 172.21.15.105:0/1241012980 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 284424545 0 0) 0x7ff460003530 con 0x7ff4740b4290 2023-11-28T18:24:28.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff478d1a640 1 -- 172.21.15.105:0/1241012980 >> v2:172.21.15.148:3300/0 conn(0x7ff47415b030 msgr2=0x7ff4740709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:28.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff478d1a640 1 --2- 172.21.15.105:0/1241012980 >> v2:172.21.15.148:3300/0 conn(0x7ff47415b030 0x7ff4740709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff478d1a640 1 -- 172.21.15.105:0/1241012980 >> v1:172.21.15.148:6789/0 conn(0x7ff474071960 legacy=0x7ff47415f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:28.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff478d1a640 1 -- 172.21.15.105:0/1241012980 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4740720d0 con 0x7ff4740b4290 2023-11-28T18:24:28.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff478d1a640 1 -- 172.21.15.105:0/1241012980 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7ff470001680 con 0x7ff4740b4290 2023-11-28T18:24:28.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff478d1a640 1 -- 172.21.15.105:0/1241012980 >> v1:172.21.15.105:6789/0 conn(0x7ff4740b4290 legacy=0x7ff4740b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:28.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff478d1a640 1 --2- 172.21.15.105:0/1241012980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff460003980 0x7ff460003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:28.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff478d1a640 1 --2- 172.21.15.105:0/1241012980 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff460004380 0x7ff460004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:28.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff478d1a640 1 --2- 172.21.15.105:0/1241012980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff460005050 0x7ff46000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:28.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff478d1a640 1 -- 172.21.15.105:0/1241012980 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff474071df0 con 0x7ff460005050 2023-11-28T18:24:28.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff478d1a640 1 -- 172.21.15.105:0/1241012980 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46000bfb0 con 0x7ff460003980 2023-11-28T18:24:28.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff478d1a640 1 -- 172.21.15.105:0/1241012980 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff46000c170 con 0x7ff460004380 2023-11-28T18:24:28.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff47a51d640 1 --2- 172.21.15.105:0/1241012980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff460005050 0x7ff46000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:28.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff479d1c640 1 --2- 172.21.15.105:0/1241012980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff460003980 0x7ff460003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:28.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff47951b640 1 --2- 172.21.15.105:0/1241012980 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff460004380 0x7ff460004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:28.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff47a51d640 1 -- 172.21.15.105:0/1241012980 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff460004380 msgr2=0x7ff460004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:28.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff47a51d640 1 --2- 172.21.15.105:0/1241012980 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff460004380 0x7ff460004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff47a51d640 1 -- 172.21.15.105:0/1241012980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff460003980 msgr2=0x7ff460003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:28.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff47a51d640 1 --2- 172.21.15.105:0/1241012980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff460003980 0x7ff460003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff47a51d640 1 -- 172.21.15.105:0/1241012980 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff460003530 con 0x7ff460005050 2023-11-28T18:24:28.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff479d1c640 1 --2- 172.21.15.105:0/1241012980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff460003980 0x7ff460003d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:28.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.686+0000 7ff47951b640 1 --2- 172.21.15.105:0/1241012980 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff460004380 0x7ff460004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:28.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.690+0000 7ff47a51d640 1 -- 172.21.15.105:0/1241012980 reap_dead start 2023-11-28T18:24:28.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.690+0000 7ff47a51d640 1 --2- 172.21.15.105:0/1241012980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff460005050 0x7ff46000b990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff47415b440 tx=0x7ff46c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d73ae1a55899bad server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:28.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.690+0000 7ff478d1a640 1 -- 172.21.15.105:0/1241012980 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c0a5c50 con 0x7ff460005050 2023-11-28T18:24:28.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.690+0000 7ff478d1a640 1 -- 172.21.15.105:0/1241012980 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c00f040 con 0x7ff460005050 2023-11-28T18:24:28.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.690+0000 7ff478d1a640 1 -- 172.21.15.105:0/1241012980 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c0039a0 con 0x7ff460005050 2023-11-28T18:24:28.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.690+0000 7ff47bf80640 1 -- 172.21.15.105:0/1241012980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff460005050 msgr2=0x7ff46000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:28.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.690+0000 7ff47bf80640 1 --2- 172.21.15.105:0/1241012980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff460005050 0x7ff46000b990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff47415b440 tx=0x7ff46c01cc40 comp rx=0 tx=0).stop 2023-11-28T18:24:28.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.690+0000 7ff47bf80640 1 -- 172.21.15.105:0/1241012980 shutdown_connections 2023-11-28T18:24:28.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.690+0000 7ff47bf80640 1 --2- 172.21.15.105:0/1241012980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff460005050 0x7ff46000b990 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.690+0000 7ff47bf80640 1 -- 172.21.15.105:0/1241012980 >> 172.21.15.105:0/1241012980 conn(0x7ff4740ceb90 msgr2=0x7ff4740cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:28.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.690+0000 7ff47bf80640 1 -- 172.21.15.105:0/1241012980 shutdown_connections 2023-11-28T18:24:28.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.690+0000 7ff47bf80640 1 -- 172.21.15.105:0/1241012980 wait complete. 2023-11-28T18:24:28.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff47bf80640 1 Processor -- start 2023-11-28T18:24:28.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff47bf80640 1 -- start start 2023-11-28T18:24:28.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff47bf80640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff460005050 0x7ff47414a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:28.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff47bf80640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4741559b0 0x7ff47415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:28.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff47bf80640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4740cca80 0x7ff4740cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:28.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff47bf80640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4740cc660 con 0x7ff4740cca80 2023-11-28T18:24:28.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff47bf80640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4740ccf00 con 0x7ff460005050 2023-11-28T18:24:28.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff47bf80640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4740cba70 con 0x7ff4741559b0 2023-11-28T18:24:28.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff47a51d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4740cca80 0x7ff4740cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:28.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff47a51d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4740cca80 0x7ff4740cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43726/0 (socket says 172.21.15.105:43726) 2023-11-28T18:24:28.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff47a51d640 1 -- 172.21.15.105:0/608612814 learned_addr learned my addr 172.21.15.105:0/608612814 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:28.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff479d1c640 1 --2- 172.21.15.105:0/608612814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff460005050 0x7ff47414a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:28.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff47951b640 1 --2- 172.21.15.105:0/608612814 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4741559b0 0x7ff47415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:28.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff47a51d640 1 -- 172.21.15.105:0/608612814 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4741559b0 msgr2=0x7ff47415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:28.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff47a51d640 1 --2- 172.21.15.105:0/608612814 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4741559b0 0x7ff47415dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff47a51d640 1 -- 172.21.15.105:0/608612814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff460005050 msgr2=0x7ff47414a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:28.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff47a51d640 1 --2- 172.21.15.105:0/608612814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff460005050 0x7ff47414a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff47a51d640 1 -- 172.21.15.105:0/608612814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff474156190 con 0x7ff4740cca80 2023-11-28T18:24:28.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff479d1c640 1 --2- 172.21.15.105:0/608612814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff460005050 0x7ff47414a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:28.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff47951b640 1 --2- 172.21.15.105:0/608612814 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4741559b0 0x7ff47415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:28.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff47a51d640 1 --2- 172.21.15.105:0/608612814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4740cca80 0x7ff4740cd570 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff46c098770 tx=0x7ff46c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:28.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff46a7fc640 1 -- 172.21.15.105:0/608612814 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c003e70 con 0x7ff4740cca80 2023-11-28T18:24:28.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff46a7fc640 1 -- 172.21.15.105:0/608612814 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c0ae800 con 0x7ff4740cca80 2023-11-28T18:24:28.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff47bf80640 1 -- 172.21.15.105:0/608612814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4740cbcc0 con 0x7ff4740cca80 2023-11-28T18:24:28.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff46a7fc640 1 -- 172.21.15.105:0/608612814 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c0aeb70 con 0x7ff4740cca80 2023-11-28T18:24:28.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff47bf80640 1 -- 172.21.15.105:0/608612814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4740cc280 con 0x7ff4740cca80 2023-11-28T18:24:28.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.694+0000 7ff47bf80640 1 -- 172.21.15.105:0/608612814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4740cc7a0 con 0x7ff4740cca80 2023-11-28T18:24:28.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.698+0000 7ff46a7fc640 1 -- 172.21.15.105:0/608612814 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff46c0aecd0 con 0x7ff4740cca80 2023-11-28T18:24:28.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.698+0000 7ff46a7fc640 1 --2- 172.21.15.105:0/608612814 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff4580668d0 0x7ff458068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:28.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.698+0000 7ff46a7fc640 1 -- 172.21.15.105:0/608612814 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7ff46c1200b0 con 0x7ff4740cca80 2023-11-28T18:24:28.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.698+0000 7ff479d1c640 1 --2- 172.21.15.105:0/608612814 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff4580668d0 0x7ff458068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:28.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.698+0000 7ff479d1c640 1 --2- 172.21.15.105:0/608612814 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff4580668d0 0x7ff458068d90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff470000c10 tx=0x7ff470001160 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:28.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.702+0000 7ff46a7fc640 1 -- 172.21.15.105:0/608612814 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff4740cc7a0 con 0x7ff4740cca80 2023-11-28T18:24:28.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.850+0000 7ff47bf80640 1 -- 172.21.15.105:0/608612814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff4740cc7a0 con 0x7ff4740cca80 2023-11-28T18:24:28.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.850+0000 7ff46a7fc640 1 -- 172.21.15.105:0/608612814 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff4740cc7a0 con 0x7ff4740cca80 2023-11-28T18:24:28.855 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:28.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.850+0000 7ff47bf80640 1 -- 172.21.15.105:0/608612814 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff4580668d0 msgr2=0x7ff458068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:28.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.850+0000 7ff47bf80640 1 --2- 172.21.15.105:0/608612814 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff4580668d0 0x7ff458068d90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff470000c10 tx=0x7ff470001160 comp rx=0 tx=0).stop 2023-11-28T18:24:28.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.850+0000 7ff47bf80640 1 -- 172.21.15.105:0/608612814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4740cca80 msgr2=0x7ff4740cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:28.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.850+0000 7ff47bf80640 1 --2- 172.21.15.105:0/608612814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4740cca80 0x7ff4740cd570 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff46c098770 tx=0x7ff46c009750 comp rx=0 tx=0).stop 2023-11-28T18:24:28.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.850+0000 7ff47bf80640 1 -- 172.21.15.105:0/608612814 shutdown_connections 2023-11-28T18:24:28.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.850+0000 7ff47bf80640 1 --2- 172.21.15.105:0/608612814 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff4580668d0 0x7ff458068d90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.850+0000 7ff47bf80640 1 --2- 172.21.15.105:0/608612814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4740cca80 0x7ff4740cd570 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.850+0000 7ff47bf80640 1 --2- 172.21.15.105:0/608612814 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4741559b0 0x7ff47415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.850+0000 7ff47bf80640 1 --2- 172.21.15.105:0/608612814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff460005050 0x7ff47414a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:28.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.850+0000 7ff47bf80640 1 -- 172.21.15.105:0/608612814 >> 172.21.15.105:0/608612814 conn(0x7ff4740ceb90 msgr2=0x7ff47415a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:28.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.850+0000 7ff47bf80640 1 -- 172.21.15.105:0/608612814 shutdown_connections 2023-11-28T18:24:28.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:28.850+0000 7ff47bf80640 1 -- 172.21.15.105:0/608612814 wait complete. 2023-11-28T18:24:28.875 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:24:29.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f487986b640 1 Processor -- start 2023-11-28T18:24:29.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f487986b640 1 -- start start 2023-11-28T18:24:29.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f487986b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48740cc760 0x7f4874164d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:29.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f487986b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48740ccc10 con 0x7f48740caf60 2023-11-28T18:24:29.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f487986b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f487406bf50 con 0x7f4874159f40 2023-11-28T18:24:29.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f487986b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48740cb3c0 con 0x7f48740cc760 2023-11-28T18:24:29.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f486a7fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f48740caf60 0x7f48740ad850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49308/0 (socket says 172.21.15.105:49308) 2023-11-28T18:24:29.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f486a7fc640 1 -- 172.21.15.105:0/283696787 learned_addr learned my addr 172.21.15.105:0/283696787 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:29.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f48737fe640 1 --2- 172.21.15.105:0/283696787 >> v2:172.21.15.148:3300/0 conn(0x7f48740cc760 0x7f4874164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:29.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f48727fc640 1 -- 172.21.15.105:0/283696787 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425253408 0 0) 0x7f48740ccc10 con 0x7f48740caf60 2023-11-28T18:24:29.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f48727fc640 1 -- 172.21.15.105:0/283696787 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4860005740 con 0x7f48740caf60 2023-11-28T18:24:29.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f48727fc640 1 -- 172.21.15.105:0/283696787 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 152366520 0 0) 0x7f4860005740 con 0x7f48740caf60 2023-11-28T18:24:29.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f48727fc640 1 -- 172.21.15.105:0/283696787 >> v2:172.21.15.148:3300/0 conn(0x7f48740cc760 msgr2=0x7f4874164d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:29.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f48727fc640 1 --2- 172.21.15.105:0/283696787 >> v2:172.21.15.148:3300/0 conn(0x7f48740cc760 0x7f4874164d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f48727fc640 1 -- 172.21.15.105:0/283696787 >> v1:172.21.15.148:6789/0 conn(0x7f4874159f40 legacy=0x7f48740ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:29.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f48727fc640 1 -- 172.21.15.105:0/283696787 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487415a320 con 0x7f48740caf60 2023-11-28T18:24:29.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f48727fc640 1 -- 172.21.15.105:0/283696787 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f485c001a00 con 0x7f48740caf60 2023-11-28T18:24:29.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f48727fc640 1 -- 172.21.15.105:0/283696787 >> v1:172.21.15.105:6789/0 conn(0x7f48740caf60 legacy=0x7f48740ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:29.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f48727fc640 1 --2- 172.21.15.105:0/283696787 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4860005be0 0x7f4860006000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:29.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f48727fc640 1 --2- 172.21.15.105:0/283696787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48600069e0 0x7f4860006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:29.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f48727fc640 1 --2- 172.21.15.105:0/283696787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48600073f0 0x7f486000dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:29.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f48727fc640 1 -- 172.21.15.105:0/283696787 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48740cb3c0 con 0x7f48600073f0 2023-11-28T18:24:29.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f48727fc640 1 -- 172.21.15.105:0/283696787 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486000e340 con 0x7f48600069e0 2023-11-28T18:24:29.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f48727fc640 1 -- 172.21.15.105:0/283696787 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486000e500 con 0x7f4860005be0 2023-11-28T18:24:29.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f48737fe640 1 --2- 172.21.15.105:0/283696787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48600073f0 0x7f486000dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:29.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f486a7fc640 1 --2- 172.21.15.105:0/283696787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48600069e0 0x7f4860006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:29.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f4872ffd640 1 --2- 172.21.15.105:0/283696787 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4860005be0 0x7f4860006000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:29.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f48737fe640 1 -- 172.21.15.105:0/283696787 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4860005be0 msgr2=0x7f4860006000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:29.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f48737fe640 1 --2- 172.21.15.105:0/283696787 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4860005be0 0x7f4860006000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f48737fe640 1 -- 172.21.15.105:0/283696787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48600069e0 msgr2=0x7f4860006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:29.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f48737fe640 1 --2- 172.21.15.105:0/283696787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48600069e0 0x7f4860006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.014+0000 7f48737fe640 1 -- 172.21.15.105:0/283696787 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4860005740 con 0x7f48600073f0 2023-11-28T18:24:29.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.018+0000 7f48737fe640 1 -- 172.21.15.105:0/283696787 reap_dead start 2023-11-28T18:24:29.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.018+0000 7f48737fe640 1 --2- 172.21.15.105:0/283696787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48600073f0 0x7f486000dba0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f486c009ab0 tx=0x7f486c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57c465a2defbf13 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:29.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.018+0000 7f48727fc640 1 -- 172.21.15.105:0/283696787 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c004020 con 0x7f48600073f0 2023-11-28T18:24:29.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.018+0000 7f48727fc640 1 -- 172.21.15.105:0/283696787 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486c004180 con 0x7f48600073f0 2023-11-28T18:24:29.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.018+0000 7f48727fc640 1 -- 172.21.15.105:0/283696787 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c003620 con 0x7f48600073f0 2023-11-28T18:24:29.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.018+0000 7f487986b640 1 -- 172.21.15.105:0/283696787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48600073f0 msgr2=0x7f486000dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:29.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.018+0000 7f487986b640 1 --2- 172.21.15.105:0/283696787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48600073f0 0x7f486000dba0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f486c009ab0 tx=0x7f486c01ca70 comp rx=0 tx=0).stop 2023-11-28T18:24:29.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.018+0000 7f487986b640 1 -- 172.21.15.105:0/283696787 shutdown_connections 2023-11-28T18:24:29.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.018+0000 7f487986b640 1 --2- 172.21.15.105:0/283696787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48600073f0 0x7f486000dba0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.018+0000 7f487986b640 1 -- 172.21.15.105:0/283696787 >> 172.21.15.105:0/283696787 conn(0x7f48740ceb90 msgr2=0x7f48740cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:29.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.018+0000 7f487986b640 1 -- 172.21.15.105:0/283696787 shutdown_connections 2023-11-28T18:24:29.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.018+0000 7f487986b640 1 -- 172.21.15.105:0/283696787 wait complete. 2023-11-28T18:24:29.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f487986b640 1 Processor -- start 2023-11-28T18:24:29.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f487986b640 1 -- start start 2023-11-28T18:24:29.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f487986b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48600073f0 0x7f487409bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:29.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f487986b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48740b19b0 0x7f48740af840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:29.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f487986b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48740afd80 0x7f487415b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:29.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f487986b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4874099ca0 con 0x7f48600073f0 2023-11-28T18:24:29.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f487986b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48740b2e80 con 0x7f48740afd80 2023-11-28T18:24:29.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f487986b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48740b0200 con 0x7f48740b19b0 2023-11-28T18:24:29.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f4872ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48600073f0 0x7f487409bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:29.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f4872ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48600073f0 0x7f487409bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43750/0 (socket says 172.21.15.105:43750) 2023-11-28T18:24:29.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f4872ffd640 1 -- 172.21.15.105:0/3464318013 learned_addr learned my addr 172.21.15.105:0/3464318013 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:29.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f48737fe640 1 --2- 172.21.15.105:0/3464318013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48740afd80 0x7f487415b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:29.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f486a7fc640 1 --2- 172.21.15.105:0/3464318013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48740b19b0 0x7f48740af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:29.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f4872ffd640 1 -- 172.21.15.105:0/3464318013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48740b19b0 msgr2=0x7f48740af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:29.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f4872ffd640 1 --2- 172.21.15.105:0/3464318013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48740b19b0 0x7f48740af840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f4872ffd640 1 -- 172.21.15.105:0/3464318013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48740afd80 msgr2=0x7f487415b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:29.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f4872ffd640 1 --2- 172.21.15.105:0/3464318013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48740afd80 0x7f487415b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f4872ffd640 1 -- 172.21.15.105:0/3464318013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487409c320 con 0x7f48600073f0 2023-11-28T18:24:29.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f4872ffd640 1 --2- 172.21.15.105:0/3464318013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48600073f0 0x7f487409bde0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f486400f540 tx=0x7f486400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:29.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f4870ff9640 1 -- 172.21.15.105:0/3464318013 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4864005320 con 0x7f48600073f0 2023-11-28T18:24:29.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f4870ff9640 1 -- 172.21.15.105:0/3464318013 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4864005480 con 0x7f48600073f0 2023-11-28T18:24:29.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f4870ff9640 1 -- 172.21.15.105:0/3464318013 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4864007250 con 0x7f48600073f0 2023-11-28T18:24:29.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f487986b640 1 -- 172.21.15.105:0/3464318013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48740b4210 con 0x7f48600073f0 2023-11-28T18:24:29.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.022+0000 7f487986b640 1 -- 172.21.15.105:0/3464318013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48740b47d0 con 0x7f48600073f0 2023-11-28T18:24:29.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.026+0000 7f487986b640 1 -- 172.21.15.105:0/3464318013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4874099f10 con 0x7f48600073f0 2023-11-28T18:24:29.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.026+0000 7f4870ff9640 1 -- 172.21.15.105:0/3464318013 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f48640073b0 con 0x7f48600073f0 2023-11-28T18:24:29.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.026+0000 7f4870ff9640 1 --2- 172.21.15.105:0/3464318013 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4854066900 0x7f4854068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:29.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.026+0000 7f4870ff9640 1 -- 172.21.15.105:0/3464318013 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f4864083ea0 con 0x7f48600073f0 2023-11-28T18:24:29.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.026+0000 7f486a7fc640 1 --2- 172.21.15.105:0/3464318013 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4854066900 0x7f4854068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:29.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.026+0000 7f486a7fc640 1 --2- 172.21.15.105:0/3464318013 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4854066900 0x7f4854068dc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f48740722b0 tx=0x7f485c001ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:29.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.030+0000 7f4870ff9640 1 -- 172.21.15.105:0/3464318013 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4864050c40 con 0x7f48600073f0 2023-11-28T18:24:29.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.182+0000 7f487986b640 1 -- 172.21.15.105:0/3464318013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4874067570 con 0x7f48600073f0 2023-11-28T18:24:29.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.182+0000 7f4870ff9640 1 -- 172.21.15.105:0/3464318013 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f486401a0a0 con 0x7f48600073f0 2023-11-28T18:24:29.187 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:24:29.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.182+0000 7f487986b640 1 -- 172.21.15.105:0/3464318013 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4854066900 msgr2=0x7f4854068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:29.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.182+0000 7f487986b640 1 --2- 172.21.15.105:0/3464318013 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4854066900 0x7f4854068dc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f48740722b0 tx=0x7f485c001ef0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.182+0000 7f487986b640 1 -- 172.21.15.105:0/3464318013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48600073f0 msgr2=0x7f487409bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:29.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.182+0000 7f487986b640 1 --2- 172.21.15.105:0/3464318013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48600073f0 0x7f487409bde0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f486400f540 tx=0x7f486400fa00 comp rx=0 tx=0).stop 2023-11-28T18:24:29.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.182+0000 7f487986b640 1 -- 172.21.15.105:0/3464318013 shutdown_connections 2023-11-28T18:24:29.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.182+0000 7f487986b640 1 --2- 172.21.15.105:0/3464318013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48740afd80 0x7f487415b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.182+0000 7f487986b640 1 --2- 172.21.15.105:0/3464318013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48740b19b0 0x7f48740af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.182+0000 7f487986b640 1 --2- 172.21.15.105:0/3464318013 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4854066900 0x7f4854068dc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.182+0000 7f487986b640 1 --2- 172.21.15.105:0/3464318013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48600073f0 0x7f487409bde0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.182+0000 7f487986b640 1 -- 172.21.15.105:0/3464318013 >> 172.21.15.105:0/3464318013 conn(0x7f48740ceb90 msgr2=0x7f4874071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:29.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.182+0000 7f487986b640 1 -- 172.21.15.105:0/3464318013 shutdown_connections 2023-11-28T18:24:29.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.182+0000 7f487986b640 1 -- 172.21.15.105:0/3464318013 wait complete. 2023-11-28T18:24:29.210 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T18:24:29.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.346+0000 7f301f3c4640 1 Processor -- start 2023-11-28T18:24:29.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.346+0000 7f301f3c4640 1 -- start start 2023-11-28T18:24:29.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.346+0000 7f301f3c4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3018071960 0x7f301815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:29.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.346+0000 7f301f3c4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f301815b030 0x7f30180709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:29.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.346+0000 7f301f3c4640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30180701d0 con 0x7f30180b4290 2023-11-28T18:24:29.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.346+0000 7f301f3c4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f301806df80 con 0x7f3018071960 2023-11-28T18:24:29.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.346+0000 7f301f3c4640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3018071df0 con 0x7f301815b030 2023-11-28T18:24:29.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.350+0000 7f301c95f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3018071960 0x7f301815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:29.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.350+0000 7f301d160640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f30180b4290 0x7f30180b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49320/0 (socket says 172.21.15.105:49320) 2023-11-28T18:24:29.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.350+0000 7f301d160640 1 -- 172.21.15.105:0/3136544768 learned_addr learned my addr 172.21.15.105:0/3136544768 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:29.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.350+0000 7f301d961640 1 --2- 172.21.15.105:0/3136544768 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f301815b030 0x7f30180709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:29.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.350+0000 7f301c95f640 1 -- 172.21.15.105:0/3136544768 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f301815b030 msgr2=0x7f30180709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:29.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.350+0000 7f301c95f640 1 --2- 172.21.15.105:0/3136544768 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f301815b030 0x7f30180709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.350+0000 7f301c95f640 1 -- 172.21.15.105:0/3136544768 >> v1:172.21.15.105:6789/0 conn(0x7f30180b4290 legacy=0x7f30180b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:29.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.350+0000 7f300ffff640 1 -- 172.21.15.105:0/3136544768 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2327575370 0 0) 0x7f30180701d0 con 0x7f30180b4290 2023-11-28T18:24:29.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.350+0000 7f301c95f640 1 -- 172.21.15.105:0/3136544768 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30180720d0 con 0x7f3018071960 2023-11-28T18:24:29.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.350+0000 7f301c95f640 1 --2- 172.21.15.105:0/3136544768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3018071960 0x7f301815f3b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f30080097c0 tx=0x7f300801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1273b02e893161 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:29.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.350+0000 7f300ffff640 1 -- 172.21.15.105:0/3136544768 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30080ad660 con 0x7f3018071960 2023-11-28T18:24:29.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.350+0000 7f300ffff640 1 -- 172.21.15.105:0/3136544768 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30080ad7c0 con 0x7f3018071960 2023-11-28T18:24:29.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.350+0000 7f300ffff640 1 -- 172.21.15.105:0/3136544768 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30080adb50 con 0x7f3018071960 2023-11-28T18:24:29.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301f3c4640 1 -- 172.21.15.105:0/3136544768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3018071960 msgr2=0x7f301815f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:29.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301f3c4640 1 --2- 172.21.15.105:0/3136544768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3018071960 0x7f301815f3b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f30080097c0 tx=0x7f300801c450 comp rx=0 tx=0).stop 2023-11-28T18:24:29.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301f3c4640 1 -- 172.21.15.105:0/3136544768 shutdown_connections 2023-11-28T18:24:29.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301f3c4640 1 --2- 172.21.15.105:0/3136544768 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f301815b030 0x7f30180709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301f3c4640 1 --2- 172.21.15.105:0/3136544768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3018071960 0x7f301815f3b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301f3c4640 1 -- 172.21.15.105:0/3136544768 >> 172.21.15.105:0/3136544768 conn(0x7f30180ceb90 msgr2=0x7f30180cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:29.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301f3c4640 1 -- 172.21.15.105:0/3136544768 shutdown_connections 2023-11-28T18:24:29.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301f3c4640 1 -- 172.21.15.105:0/3136544768 wait complete. 2023-11-28T18:24:29.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301f3c4640 1 Processor -- start 2023-11-28T18:24:29.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301f3c4640 1 -- start start 2023-11-28T18:24:29.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301f3c4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30180b4290 0x7f301814df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:29.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301f3c4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f301815b030 0x7f3018159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:29.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301f3c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f301814e8f0 0x7f30180cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:29.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301f3c4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301814f1a0 con 0x7f301815b030 2023-11-28T18:24:29.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301f3c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3018159e70 con 0x7f301814e8f0 2023-11-28T18:24:29.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301f3c4640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f301815a150 con 0x7f30180b4290 2023-11-28T18:24:29.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301c95f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f301815b030 0x7f3018159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:29.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301c95f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f301815b030 0x7f3018159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43764/0 (socket says 172.21.15.105:43764) 2023-11-28T18:24:29.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301c95f640 1 -- 172.21.15.105:0/724953047 learned_addr learned my addr 172.21.15.105:0/724953047 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:29.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301d160640 1 --2- 172.21.15.105:0/724953047 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30180b4290 0x7f301814df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:29.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301d961640 1 --2- 172.21.15.105:0/724953047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f301814e8f0 0x7f30180cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:29.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301c95f640 1 -- 172.21.15.105:0/724953047 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30180b4290 msgr2=0x7f301814df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:29.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301c95f640 1 --2- 172.21.15.105:0/724953047 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30180b4290 0x7f301814df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301c95f640 1 -- 172.21.15.105:0/724953047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f301814e8f0 msgr2=0x7f30180cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:29.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301c95f640 1 --2- 172.21.15.105:0/724953047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f301814e8f0 0x7f30180cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301c95f640 1 -- 172.21.15.105:0/724953047 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3008009420 con 0x7f301815b030 2023-11-28T18:24:29.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.354+0000 7f301d160640 1 --2- 172.21.15.105:0/724953047 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30180b4290 0x7f301814df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:29.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.358+0000 7f301c95f640 1 --2- 172.21.15.105:0/724953047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f301815b030 0x7f3018159420 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f300801a080 tx=0x7f300801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:29.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.358+0000 7f300dffb640 1 -- 172.21.15.105:0/724953047 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3008004230 con 0x7f301815b030 2023-11-28T18:24:29.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.358+0000 7f300dffb640 1 -- 172.21.15.105:0/724953047 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3008004390 con 0x7f301815b030 2023-11-28T18:24:29.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.358+0000 7f300dffb640 1 -- 172.21.15.105:0/724953047 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3008003820 con 0x7f301815b030 2023-11-28T18:24:29.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.358+0000 7f301f3c4640 1 -- 172.21.15.105:0/724953047 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f301815a430 con 0x7f301815b030 2023-11-28T18:24:29.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.358+0000 7f301f3c4640 1 -- 172.21.15.105:0/724953047 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f301814e4a0 con 0x7f301815b030 2023-11-28T18:24:29.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.358+0000 7f301f3c4640 1 -- 172.21.15.105:0/724953047 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f301814f310 con 0x7f301815b030 2023-11-28T18:24:29.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.358+0000 7f300dffb640 1 -- 172.21.15.105:0/724953047 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3008003980 con 0x7f301815b030 2023-11-28T18:24:29.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.362+0000 7f300dffb640 1 --2- 172.21.15.105:0/724953047 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2ffc0668d0 0x7f2ffc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:29.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.362+0000 7f300dffb640 1 -- 172.21.15.105:0/724953047 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f300811f490 con 0x7f301815b030 2023-11-28T18:24:29.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.362+0000 7f301d160640 1 --2- 172.21.15.105:0/724953047 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2ffc0668d0 0x7f2ffc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:29.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.362+0000 7f301d160640 1 --2- 172.21.15.105:0/724953047 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2ffc0668d0 0x7f2ffc068d90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3014007fd0 tx=0x7f3014007c20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:29.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.366+0000 7f300dffb640 1 -- 172.21.15.105:0/724953047 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f30080ec360 con 0x7f301815b030 2023-11-28T18:24:29.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.514+0000 7f301f3c4640 1 -- 172.21.15.105:0/724953047 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3018067570 con 0x7f301815b030 2023-11-28T18:24:29.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.514+0000 7f300dffb640 1 -- 172.21.15.105:0/724953047 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f30080ef980 con 0x7f301815b030 2023-11-28T18:24:29.519 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:29.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.514+0000 7f301f3c4640 1 -- 172.21.15.105:0/724953047 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2ffc0668d0 msgr2=0x7f2ffc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:29.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.514+0000 7f301f3c4640 1 --2- 172.21.15.105:0/724953047 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2ffc0668d0 0x7f2ffc068d90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3014007fd0 tx=0x7f3014007c20 comp rx=0 tx=0).stop 2023-11-28T18:24:29.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.514+0000 7f301f3c4640 1 -- 172.21.15.105:0/724953047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f301815b030 msgr2=0x7f3018159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:29.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.514+0000 7f301f3c4640 1 --2- 172.21.15.105:0/724953047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f301815b030 0x7f3018159420 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f300801a080 tx=0x7f300801ac90 comp rx=0 tx=0).stop 2023-11-28T18:24:29.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.514+0000 7f301f3c4640 1 -- 172.21.15.105:0/724953047 shutdown_connections 2023-11-28T18:24:29.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.514+0000 7f301f3c4640 1 --2- 172.21.15.105:0/724953047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f301814e8f0 0x7f30180cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.514+0000 7f301f3c4640 1 --2- 172.21.15.105:0/724953047 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2ffc0668d0 0x7f2ffc068d90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.514+0000 7f301f3c4640 1 --2- 172.21.15.105:0/724953047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f301815b030 0x7f3018159420 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.514+0000 7f301f3c4640 1 --2- 172.21.15.105:0/724953047 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30180b4290 0x7f301814df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.514+0000 7f301f3c4640 1 -- 172.21.15.105:0/724953047 >> 172.21.15.105:0/724953047 conn(0x7f30180ceb90 msgr2=0x7f301815f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:29.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.514+0000 7f301f3c4640 1 -- 172.21.15.105:0/724953047 shutdown_connections 2023-11-28T18:24:29.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.514+0000 7f301f3c4640 1 -- 172.21.15.105:0/724953047 wait complete. 2023-11-28T18:24:29.539 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T18:24:29.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.674+0000 7fdf207f5640 1 Processor -- start 2023-11-28T18:24:29.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.674+0000 7fdf207f5640 1 -- start start 2023-11-28T18:24:29.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.674+0000 7fdf207f5640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf18159610 0x7fdf180ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:29.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.674+0000 7fdf207f5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdf1806cef0 0x7fdf180ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:29.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.674+0000 7fdf207f5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf180cde30 con 0x7fdf180cc760 2023-11-28T18:24:29.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.674+0000 7fdf207f5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf181599f0 con 0x7fdf1806cef0 2023-11-28T18:24:29.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.674+0000 7fdf207f5640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1806de20 con 0x7fdf18159610 2023-11-28T18:24:29.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.674+0000 7fdf1dd90640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fdf180cc760 0x7fdf1806c7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49328/0 (socket says 172.21.15.105:49328) 2023-11-28T18:24:29.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.674+0000 7fdf1dd90640 1 -- 172.21.15.105:0/3944081756 learned_addr learned my addr 172.21.15.105:0/3944081756 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:29.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.674+0000 7fdf1ed92640 1 --2- 172.21.15.105:0/3944081756 >> v2:172.21.15.105:3300/0 conn(0x7fdf1806cef0 0x7fdf180ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:29.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.674+0000 7fdf1e591640 1 --2- 172.21.15.105:0/3944081756 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf18159610 0x7fdf180ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:29.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.674+0000 7fdf1ed92640 1 -- 172.21.15.105:0/3944081756 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf18159610 msgr2=0x7fdf180ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:29.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.674+0000 7fdf1ed92640 1 --2- 172.21.15.105:0/3944081756 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf18159610 0x7fdf180ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.674+0000 7fdf1ed92640 1 -- 172.21.15.105:0/3944081756 >> v1:172.21.15.105:6789/0 conn(0x7fdf180cc760 legacy=0x7fdf1806c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:29.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.674+0000 7fdf1ed92640 1 -- 172.21.15.105:0/3944081756 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf180cd220 con 0x7fdf1806cef0 2023-11-28T18:24:29.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.678+0000 7fdf1ed92640 1 --2- 172.21.15.105:0/3944081756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf1806cef0 0x7fdf180ccbc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009d90 tx=0x7fdf1001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a61877b68d91 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:29.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.678+0000 7fdf1d58f640 1 -- 172.21.15.105:0/3944081756 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf100a5960 con 0x7fdf1806cef0 2023-11-28T18:24:29.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.678+0000 7fdf1d58f640 1 -- 172.21.15.105:0/3944081756 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf100a5ac0 con 0x7fdf1806cef0 2023-11-28T18:24:29.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.678+0000 7fdf1d58f640 1 -- 172.21.15.105:0/3944081756 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf100a5e50 con 0x7fdf1806cef0 2023-11-28T18:24:29.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.678+0000 7fdf207f5640 1 -- 172.21.15.105:0/3944081756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf1806cef0 msgr2=0x7fdf180ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:29.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.678+0000 7fdf207f5640 1 --2- 172.21.15.105:0/3944081756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf1806cef0 0x7fdf180ccbc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009d90 tx=0x7fdf1001c920 comp rx=0 tx=0).stop 2023-11-28T18:24:29.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.678+0000 7fdf207f5640 1 -- 172.21.15.105:0/3944081756 shutdown_connections 2023-11-28T18:24:29.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.678+0000 7fdf207f5640 1 --2- 172.21.15.105:0/3944081756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf1806cef0 0x7fdf180ccbc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.678+0000 7fdf207f5640 1 --2- 172.21.15.105:0/3944081756 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf18159610 0x7fdf180ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.678+0000 7fdf207f5640 1 -- 172.21.15.105:0/3944081756 >> 172.21.15.105:0/3944081756 conn(0x7fdf180ceb90 msgr2=0x7fdf180cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:29.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.678+0000 7fdf207f5640 1 -- 172.21.15.105:0/3944081756 shutdown_connections 2023-11-28T18:24:29.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.678+0000 7fdf207f5640 1 -- 172.21.15.105:0/3944081756 wait complete. 2023-11-28T18:24:29.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf207f5640 1 Processor -- start 2023-11-28T18:24:29.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf207f5640 1 -- start start 2023-11-28T18:24:29.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf207f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf1806cef0 0x7fdf1814c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:29.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf207f5640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf180cc760 0x7fdf1815bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:29.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf207f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf18159610 0x7fdf1815c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:29.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf207f5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1815c7e0 con 0x7fdf1806cef0 2023-11-28T18:24:29.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf207f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1815cac0 con 0x7fdf18159610 2023-11-28T18:24:29.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf207f5640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf18155aa0 con 0x7fdf180cc760 2023-11-28T18:24:29.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf1e591640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf1806cef0 0x7fdf1814c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:29.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf1e591640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf1806cef0 0x7fdf1814c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43774/0 (socket says 172.21.15.105:43774) 2023-11-28T18:24:29.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf1e591640 1 -- 172.21.15.105:0/2063924741 learned_addr learned my addr 172.21.15.105:0/2063924741 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:29.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf1e591640 1 -- 172.21.15.105:0/2063924741 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf180cc760 msgr2=0x7fdf1815bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:29.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf1e591640 1 --2- 172.21.15.105:0/2063924741 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf180cc760 0x7fdf1815bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf1e591640 1 -- 172.21.15.105:0/2063924741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf18159610 msgr2=0x7fdf1815c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:29.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf1e591640 1 --2- 172.21.15.105:0/2063924741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf18159610 0x7fdf1815c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf1e591640 1 -- 172.21.15.105:0/2063924741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf10009a70 con 0x7fdf1806cef0 2023-11-28T18:24:29.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf1e591640 1 --2- 172.21.15.105:0/2063924741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf1806cef0 0x7fdf1814c670 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400d6d0 tx=0x7fdf1400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:29.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf0effd640 1 -- 172.21.15.105:0/2063924741 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14007900 con 0x7fdf1806cef0 2023-11-28T18:24:29.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf0effd640 1 -- 172.21.15.105:0/2063924741 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf14007a60 con 0x7fdf1806cef0 2023-11-28T18:24:29.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf207f5640 1 -- 172.21.15.105:0/2063924741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf18155d80 con 0x7fdf1806cef0 2023-11-28T18:24:29.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf0effd640 1 -- 172.21.15.105:0/2063924741 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14007df0 con 0x7fdf1806cef0 2023-11-28T18:24:29.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf207f5640 1 -- 172.21.15.105:0/2063924741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1814cbb0 con 0x7fdf1806cef0 2023-11-28T18:24:29.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.682+0000 7fdf207f5640 1 -- 172.21.15.105:0/2063924741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf18067570 con 0x7fdf1806cef0 2023-11-28T18:24:29.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.686+0000 7fdf0effd640 1 -- 172.21.15.105:0/2063924741 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fdf14011460 con 0x7fdf1806cef0 2023-11-28T18:24:29.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.686+0000 7fdf0effd640 1 --2- 172.21.15.105:0/2063924741 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdefc0668d0 0x7fdefc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:29.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.686+0000 7fdf0effd640 1 -- 172.21.15.105:0/2063924741 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fdf14084b40 con 0x7fdf1806cef0 2023-11-28T18:24:29.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.686+0000 7fdf1dd90640 1 --2- 172.21.15.105:0/2063924741 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdefc0668d0 0x7fdefc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:29.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.686+0000 7fdf1dd90640 1 --2- 172.21.15.105:0/2063924741 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdefc0668d0 0x7fdefc068d90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdf08000f70 tx=0x7fdf08007a90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:29.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.690+0000 7fdf0effd640 1 -- 172.21.15.105:0/2063924741 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdf140518e0 con 0x7fdf1806cef0 2023-11-28T18:24:29.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.850+0000 7fdf207f5640 1 -- 172.21.15.105:0/2063924741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdf180cb170 con 0x7fdf1806cef0 2023-11-28T18:24:29.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.850+0000 7fdf0effd640 1 -- 172.21.15.105:0/2063924741 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdf1401f080 con 0x7fdf1806cef0 2023-11-28T18:24:29.856 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2023-11-28T18:24:29.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.854+0000 7fdf207f5640 1 -- 172.21.15.105:0/2063924741 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdefc0668d0 msgr2=0x7fdefc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:29.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.854+0000 7fdf207f5640 1 --2- 172.21.15.105:0/2063924741 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdefc0668d0 0x7fdefc068d90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdf08000f70 tx=0x7fdf08007a90 comp rx=0 tx=0).stop 2023-11-28T18:24:29.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.854+0000 7fdf207f5640 1 -- 172.21.15.105:0/2063924741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf1806cef0 msgr2=0x7fdf1814c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:29.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.854+0000 7fdf207f5640 1 --2- 172.21.15.105:0/2063924741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf1806cef0 0x7fdf1814c670 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400d6d0 tx=0x7fdf1400db90 comp rx=0 tx=0).stop 2023-11-28T18:24:29.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.854+0000 7fdf207f5640 1 -- 172.21.15.105:0/2063924741 shutdown_connections 2023-11-28T18:24:29.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.854+0000 7fdf207f5640 1 --2- 172.21.15.105:0/2063924741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf18159610 0x7fdf1815c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.854+0000 7fdf207f5640 1 --2- 172.21.15.105:0/2063924741 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf180cc760 0x7fdf1815bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.854+0000 7fdf207f5640 1 --2- 172.21.15.105:0/2063924741 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdefc0668d0 0x7fdefc068d90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.854+0000 7fdf207f5640 1 --2- 172.21.15.105:0/2063924741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf1806cef0 0x7fdf1814c670 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:29.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.854+0000 7fdf207f5640 1 -- 172.21.15.105:0/2063924741 >> 172.21.15.105:0/2063924741 conn(0x7fdf180ceb90 msgr2=0x7fdf18070c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:29.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.854+0000 7fdf207f5640 1 -- 172.21.15.105:0/2063924741 shutdown_connections 2023-11-28T18:24:29.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:29.854+0000 7fdf207f5640 1 -- 172.21.15.105:0/2063924741 wait complete. 2023-11-28T18:24:29.878 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:24:29.878 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:24:29.889 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:24:30.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.022+0000 7ff6ebdd1640 1 Processor -- start 2023-11-28T18:24:30.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.022+0000 7ff6ebdd1640 1 -- start start 2023-11-28T18:24:30.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.026+0000 7ff6ebdd1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff6e406d610 0x7ff6e40cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:30.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.026+0000 7ff6ebdd1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff6e40ce530 0x7ff6e40ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:30.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.026+0000 7ff6ebdd1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e4162820 con 0x7ff6e40adf30 2023-11-28T18:24:30.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.026+0000 7ff6ebdd1640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e40ae340 con 0x7ff6e406d610 2023-11-28T18:24:30.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.026+0000 7ff6ebdd1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e4161c10 con 0x7ff6e40ce530 2023-11-28T18:24:30.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.026+0000 7ff6e9b6d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff6e406d610 0x7ff6e40cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:30.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.026+0000 7ff6e9b6d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff6e406d610 0x7ff6e40cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43780/0 (socket says 172.21.15.105:43780) 2023-11-28T18:24:30.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.026+0000 7ff6e9b6d640 1 -- 172.21.15.105:0/1962000817 learned_addr learned my addr 172.21.15.105:0/1962000817 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:30.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.026+0000 7ff6e936c640 1 --2- 172.21.15.105:0/1962000817 >> v2:172.21.15.148:3300/0 conn(0x7ff6e40ce530 0x7ff6e40ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:30.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.026+0000 7ff6e9b6d640 1 -- 172.21.15.105:0/1962000817 >> v2:172.21.15.148:3300/0 conn(0x7ff6e40ce530 msgr2=0x7ff6e40ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:30.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.026+0000 7ff6e9b6d640 1 --2- 172.21.15.105:0/1962000817 >> v2:172.21.15.148:3300/0 conn(0x7ff6e40ce530 0x7ff6e40ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.026+0000 7ff6e9b6d640 1 -- 172.21.15.105:0/1962000817 >> v1:172.21.15.148:6789/0 conn(0x7ff6e40adf30 legacy=0x7ff6e41650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:30.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.026+0000 7ff6e9b6d640 1 -- 172.21.15.105:0/1962000817 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e4161ef0 con 0x7ff6e406d610 2023-11-28T18:24:30.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.026+0000 7ff6e9b6d640 1 --2- 172.21.15.105:0/1962000817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6e406d610 0x7ff6e40cdff0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4073a50 tx=0x7ff6e001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc3abdffb87252f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:30.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.026+0000 7ff6e8b6b640 1 -- 172.21.15.105:0/1962000817 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e00ab5f0 con 0x7ff6e406d610 2023-11-28T18:24:30.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.026+0000 7ff6e8b6b640 1 -- 172.21.15.105:0/1962000817 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e00ab750 con 0x7ff6e406d610 2023-11-28T18:24:30.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.030+0000 7ff6e8b6b640 1 -- 172.21.15.105:0/1962000817 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e00abae0 con 0x7ff6e406d610 2023-11-28T18:24:30.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.030+0000 7ff6ebdd1640 1 -- 172.21.15.105:0/1962000817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6e406d610 msgr2=0x7ff6e40cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:30.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.030+0000 7ff6ebdd1640 1 --2- 172.21.15.105:0/1962000817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6e406d610 0x7ff6e40cdff0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4073a50 tx=0x7ff6e001c4d0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.030+0000 7ff6ebdd1640 1 -- 172.21.15.105:0/1962000817 shutdown_connections 2023-11-28T18:24:30.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.030+0000 7ff6ebdd1640 1 --2- 172.21.15.105:0/1962000817 >> v2:172.21.15.148:3300/0 conn(0x7ff6e40ce530 0x7ff6e40ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.030+0000 7ff6ebdd1640 1 --2- 172.21.15.105:0/1962000817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6e406d610 0x7ff6e40cdff0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.030+0000 7ff6ebdd1640 1 -- 172.21.15.105:0/1962000817 >> 172.21.15.105:0/1962000817 conn(0x7ff6e40ceb90 msgr2=0x7ff6e40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:30.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.030+0000 7ff6ebdd1640 1 -- 172.21.15.105:0/1962000817 shutdown_connections 2023-11-28T18:24:30.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.030+0000 7ff6ebdd1640 1 -- 172.21.15.105:0/1962000817 wait complete. 2023-11-28T18:24:30.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.030+0000 7ff6ebdd1640 1 Processor -- start 2023-11-28T18:24:30.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.030+0000 7ff6ebdd1640 1 -- start start 2023-11-28T18:24:30.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.030+0000 7ff6ebdd1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6e406d610 0x7ff6e4160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:30.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.030+0000 7ff6ebdd1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6e40adf30 0x7ff6e415db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:30.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.030+0000 7ff6ebdd1640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6e40ce530 0x7ff6e415e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:30.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.030+0000 7ff6ebdd1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e40b1a50 con 0x7ff6e40adf30 2023-11-28T18:24:30.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.030+0000 7ff6ebdd1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e40b1d30 con 0x7ff6e406d610 2023-11-28T18:24:30.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.030+0000 7ff6ebdd1640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e40b2010 con 0x7ff6e40ce530 2023-11-28T18:24:30.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.030+0000 7ff6e936c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6e40adf30 0x7ff6e415db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:30.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.030+0000 7ff6e936c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6e40adf30 0x7ff6e415db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43794/0 (socket says 172.21.15.105:43794) 2023-11-28T18:24:30.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.030+0000 7ff6e936c640 1 -- 172.21.15.105:0/305208100 learned_addr learned my addr 172.21.15.105:0/305208100 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:30.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.030+0000 7ff6e9b6d640 1 --2- 172.21.15.105:0/305208100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6e406d610 0x7ff6e4160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:30.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.030+0000 7ff6ea36e640 1 --2- 172.21.15.105:0/305208100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6e40ce530 0x7ff6e415e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:30.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.034+0000 7ff6e936c640 1 -- 172.21.15.105:0/305208100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6e40ce530 msgr2=0x7ff6e415e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:30.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.034+0000 7ff6e936c640 1 --2- 172.21.15.105:0/305208100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6e40ce530 0x7ff6e415e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.034+0000 7ff6e936c640 1 -- 172.21.15.105:0/305208100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6e406d610 msgr2=0x7ff6e4160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:30.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.034+0000 7ff6e936c640 1 --2- 172.21.15.105:0/305208100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6e406d610 0x7ff6e4160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.034+0000 7ff6e936c640 1 -- 172.21.15.105:0/305208100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e00094d0 con 0x7ff6e40adf30 2023-11-28T18:24:30.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.034+0000 7ff6e936c640 1 --2- 172.21.15.105:0/305208100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6e40adf30 0x7ff6e415db80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400d5f0 tx=0x7ff6d400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:30.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.034+0000 7ff6da7fc640 1 -- 172.21.15.105:0/305208100 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40125f0 con 0x7ff6e40adf30 2023-11-28T18:24:30.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.034+0000 7ff6da7fc640 1 -- 172.21.15.105:0/305208100 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d4012750 con 0x7ff6e40adf30 2023-11-28T18:24:30.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.034+0000 7ff6da7fc640 1 -- 172.21.15.105:0/305208100 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4012ae0 con 0x7ff6e40adf30 2023-11-28T18:24:30.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.034+0000 7ff6ebdd1640 1 -- 172.21.15.105:0/305208100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e40b22f0 con 0x7ff6e40adf30 2023-11-28T18:24:30.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.034+0000 7ff6ebdd1640 1 -- 172.21.15.105:0/305208100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e41608c0 con 0x7ff6e40adf30 2023-11-28T18:24:30.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.034+0000 7ff6ebdd1640 1 -- 172.21.15.105:0/305208100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e4067570 con 0x7ff6e40adf30 2023-11-28T18:24:30.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.034+0000 7ff6da7fc640 1 -- 172.21.15.105:0/305208100 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff6d4012c40 con 0x7ff6e40adf30 2023-11-28T18:24:30.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.034+0000 7ff6da7fc640 1 --2- 172.21.15.105:0/305208100 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6c80668d0 0x7ff6c8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:30.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.034+0000 7ff6da7fc640 1 -- 172.21.15.105:0/305208100 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7ff6d4084d20 con 0x7ff6e40adf30 2023-11-28T18:24:30.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.034+0000 7ff6e9b6d640 1 --2- 172.21.15.105:0/305208100 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6c80668d0 0x7ff6c8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:30.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.038+0000 7ff6e9b6d640 1 --2- 172.21.15.105:0/305208100 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6c80668d0 0x7ff6c8068d90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00af000 tx=0x7ff6e00b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:30.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.038+0000 7ff6da7fc640 1 -- 172.21.15.105:0/305208100 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff6d4051ac0 con 0x7ff6e40adf30 2023-11-28T18:24:30.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.190+0000 7ff6ebdd1640 1 -- 172.21.15.105:0/305208100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6e4070940 con 0x7ff6e40adf30 2023-11-28T18:24:30.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.194+0000 7ff6da7fc640 1 -- 172.21.15.105:0/305208100 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18887 (secure 0 0 0) 0x7ff6d4007380 con 0x7ff6e40adf30 2023-11-28T18:24:30.198 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:30.198 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:24:15.048168+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:24:30.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.194+0000 7ff6ebdd1640 1 -- 172.21.15.105:0/305208100 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6c80668d0 msgr2=0x7ff6c8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:30.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.194+0000 7ff6ebdd1640 1 --2- 172.21.15.105:0/305208100 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6c80668d0 0x7ff6c8068d90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00af000 tx=0x7ff6e00b6040 comp rx=0 tx=0).stop 2023-11-28T18:24:30.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.194+0000 7ff6ebdd1640 1 -- 172.21.15.105:0/305208100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6e40adf30 msgr2=0x7ff6e415db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:30.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.194+0000 7ff6ebdd1640 1 --2- 172.21.15.105:0/305208100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6e40adf30 0x7ff6e415db80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400d5f0 tx=0x7ff6d400dab0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.194+0000 7ff6ebdd1640 1 -- 172.21.15.105:0/305208100 shutdown_connections 2023-11-28T18:24:30.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.194+0000 7ff6ebdd1640 1 --2- 172.21.15.105:0/305208100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6e40ce530 0x7ff6e415e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.194+0000 7ff6ebdd1640 1 --2- 172.21.15.105:0/305208100 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6c80668d0 0x7ff6c8068d90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.194+0000 7ff6ebdd1640 1 --2- 172.21.15.105:0/305208100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6e40adf30 0x7ff6e415db80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.194+0000 7ff6ebdd1640 1 --2- 172.21.15.105:0/305208100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6e406d610 0x7ff6e4160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.194+0000 7ff6ebdd1640 1 -- 172.21.15.105:0/305208100 >> 172.21.15.105:0/305208100 conn(0x7ff6e40ceb90 msgr2=0x7ff6e40ae910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:30.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.194+0000 7ff6ebdd1640 1 -- 172.21.15.105:0/305208100 shutdown_connections 2023-11-28T18:24:30.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.194+0000 7ff6ebdd1640 1 -- 172.21.15.105:0/305208100 wait complete. 2023-11-28T18:24:30.220 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:23:31.485422+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:23:32.274378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T18:23:36.100071+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T18:24:30.220 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:24:30.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.354+0000 7f14b46ea640 1 Processor -- start 2023-11-28T18:24:30.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.354+0000 7f14b46ea640 1 -- start start 2023-11-28T18:24:30.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.358+0000 7f14b46ea640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f14ac0b4290 0x7f14ac0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:30.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.358+0000 7f14b46ea640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f14ac15b030 0x7f14ac0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:30.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.358+0000 7f14b46ea640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14ac0b4bb0 con 0x7f14ac071960 2023-11-28T18:24:30.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.358+0000 7f14b46ea640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ac0701d0 con 0x7f14ac0b4290 2023-11-28T18:24:30.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.358+0000 7f14b46ea640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ac06df80 con 0x7f14ac15b030 2023-11-28T18:24:30.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.358+0000 7f14b2486640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f14ac0b4290 0x7f14ac0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:30.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.358+0000 7f14b2486640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f14ac0b4290 0x7f14ac0b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43798/0 (socket says 172.21.15.105:43798) 2023-11-28T18:24:30.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.358+0000 7f14b2486640 1 -- 172.21.15.105:0/678992792 learned_addr learned my addr 172.21.15.105:0/678992792 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:30.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.358+0000 7f14b2486640 1 -- 172.21.15.105:0/678992792 >> v2:172.21.15.148:3300/0 conn(0x7f14ac15b030 msgr2=0x7f14ac0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:30.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.358+0000 7f14b2486640 1 --2- 172.21.15.105:0/678992792 >> v2:172.21.15.148:3300/0 conn(0x7f14ac15b030 0x7f14ac0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.358+0000 7f14b2486640 1 -- 172.21.15.105:0/678992792 >> v1:172.21.15.148:6789/0 conn(0x7f14ac071960 legacy=0x7f14ac15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:30.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.358+0000 7f14b2486640 1 -- 172.21.15.105:0/678992792 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac071df0 con 0x7f14ac0b4290 2023-11-28T18:24:30.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.358+0000 7f14b2486640 1 --2- 172.21.15.105:0/678992792 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14ac0b4290 0x7f14ac0b4670 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f14ac06e370 tx=0x7f14a801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4fa33f59916dd2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:30.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.358+0000 7f14b1484640 1 -- 172.21.15.105:0/678992792 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a80ac620 con 0x7f14ac0b4290 2023-11-28T18:24:30.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.358+0000 7f14b1484640 1 -- 172.21.15.105:0/678992792 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a80ac780 con 0x7f14ac0b4290 2023-11-28T18:24:30.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.362+0000 7f14b1484640 1 -- 172.21.15.105:0/678992792 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a80acb10 con 0x7f14ac0b4290 2023-11-28T18:24:30.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.362+0000 7f14b46ea640 1 -- 172.21.15.105:0/678992792 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14ac0b4290 msgr2=0x7f14ac0b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:30.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.362+0000 7f14b46ea640 1 --2- 172.21.15.105:0/678992792 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14ac0b4290 0x7f14ac0b4670 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f14ac06e370 tx=0x7f14a801c4b0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.362+0000 7f14b46ea640 1 -- 172.21.15.105:0/678992792 shutdown_connections 2023-11-28T18:24:30.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.362+0000 7f14b46ea640 1 --2- 172.21.15.105:0/678992792 >> v2:172.21.15.148:3300/0 conn(0x7f14ac15b030 0x7f14ac0709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.362+0000 7f14b46ea640 1 --2- 172.21.15.105:0/678992792 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14ac0b4290 0x7f14ac0b4670 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.362+0000 7f14b46ea640 1 -- 172.21.15.105:0/678992792 >> 172.21.15.105:0/678992792 conn(0x7f14ac0ceb90 msgr2=0x7f14ac0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:30.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.362+0000 7f14b46ea640 1 -- 172.21.15.105:0/678992792 shutdown_connections 2023-11-28T18:24:30.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.362+0000 7f14b46ea640 1 -- 172.21.15.105:0/678992792 wait complete. 2023-11-28T18:24:30.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.362+0000 7f14b46ea640 1 Processor -- start 2023-11-28T18:24:30.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.362+0000 7f14b46ea640 1 -- start start 2023-11-28T18:24:30.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.362+0000 7f14b46ea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14ac0b4290 0x7f14ac0ae500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:30.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.362+0000 7f14b46ea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14ac15b030 0x7f14ac0aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:30.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.362+0000 7f14b46ea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14ac14a9e0 0x7f14ac0aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:30.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.362+0000 7f14b46ea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac153eb0 con 0x7f14ac0b4290 2023-11-28T18:24:30.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.362+0000 7f14b46ea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac15bae0 con 0x7f14ac14a9e0 2023-11-28T18:24:30.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.362+0000 7f14b46ea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac15bdc0 con 0x7f14ac15b030 2023-11-28T18:24:30.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.362+0000 7f14b2486640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14ac0b4290 0x7f14ac0ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:30.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.362+0000 7f14b2486640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14ac0b4290 0x7f14ac0ae500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43808/0 (socket says 172.21.15.105:43808) 2023-11-28T18:24:30.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.362+0000 7f14b2486640 1 -- 172.21.15.105:0/1313529552 learned_addr learned my addr 172.21.15.105:0/1313529552 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:30.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.362+0000 7f14b2c87640 1 --2- 172.21.15.105:0/1313529552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14ac14a9e0 0x7f14ac0aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:30.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.362+0000 7f14b1c85640 1 --2- 172.21.15.105:0/1313529552 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14ac15b030 0x7f14ac0aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:30.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.366+0000 7f14b2486640 1 -- 172.21.15.105:0/1313529552 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14ac15b030 msgr2=0x7f14ac0aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:30.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.366+0000 7f14b2486640 1 --2- 172.21.15.105:0/1313529552 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14ac15b030 0x7f14ac0aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.366+0000 7f14b2486640 1 -- 172.21.15.105:0/1313529552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14ac14a9e0 msgr2=0x7f14ac0aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:30.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.366+0000 7f14b2486640 1 --2- 172.21.15.105:0/1313529552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14ac14a9e0 0x7f14ac0aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.366+0000 7f14b2486640 1 -- 172.21.15.105:0/1313529552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a80094d0 con 0x7f14ac0b4290 2023-11-28T18:24:30.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.366+0000 7f14b1c85640 1 --2- 172.21.15.105:0/1313529552 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14ac15b030 0x7f14ac0aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:30.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.366+0000 7f14b2486640 1 --2- 172.21.15.105:0/1313529552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14ac0b4290 0x7f14ac0ae500 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f14a80af040 tx=0x7f14a801acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:30.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.366+0000 7f14a2ffd640 1 -- 172.21.15.105:0/1313529552 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a80b7620 con 0x7f14ac0b4290 2023-11-28T18:24:30.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.366+0000 7f14a2ffd640 1 -- 172.21.15.105:0/1313529552 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a80b77a0 con 0x7f14ac0b4290 2023-11-28T18:24:30.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.366+0000 7f14a2ffd640 1 -- 172.21.15.105:0/1313529552 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a80b7b50 con 0x7f14ac0b4290 2023-11-28T18:24:30.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.366+0000 7f14b46ea640 1 -- 172.21.15.105:0/1313529552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ac15c0a0 con 0x7f14ac0b4290 2023-11-28T18:24:30.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.366+0000 7f14b46ea640 1 -- 172.21.15.105:0/1313529552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ac1502c0 con 0x7f14ac0b4290 2023-11-28T18:24:30.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.366+0000 7f14b46ea640 1 -- 172.21.15.105:0/1313529552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ac150570 con 0x7f14ac0b4290 2023-11-28T18:24:30.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.366+0000 7f14a2ffd640 1 -- 172.21.15.105:0/1313529552 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f14a80b3040 con 0x7f14ac0b4290 2023-11-28T18:24:30.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.366+0000 7f14a2ffd640 1 --2- 172.21.15.105:0/1313529552 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f14900668d0 0x7f1490068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:30.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.366+0000 7f14a2ffd640 1 -- 172.21.15.105:0/1313529552 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f14a811ece0 con 0x7f14ac0b4290 2023-11-28T18:24:30.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.366+0000 7f14b1c85640 1 --2- 172.21.15.105:0/1313529552 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f14900668d0 0x7f1490068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:30.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.370+0000 7f14b1c85640 1 --2- 172.21.15.105:0/1313529552 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f14900668d0 0x7f1490068d90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f14ac09cd20 tx=0x7f149c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:30.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.370+0000 7f14a2ffd640 1 -- 172.21.15.105:0/1313529552 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f14a80ebb20 con 0x7f14ac0b4290 2023-11-28T18:24:30.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.522+0000 7f14b46ea640 1 -- 172.21.15.105:0/1313529552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f14ac150570 con 0x7f14ac0b4290 2023-11-28T18:24:30.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.522+0000 7f14a2ffd640 1 -- 172.21.15.105:0/1313529552 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14a80ef140 con 0x7f14ac0b4290 2023-11-28T18:24:30.527 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:24:30.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.522+0000 7f14b46ea640 1 -- 172.21.15.105:0/1313529552 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f14900668d0 msgr2=0x7f1490068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:30.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.522+0000 7f14b46ea640 1 --2- 172.21.15.105:0/1313529552 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f14900668d0 0x7f1490068d90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f14ac09cd20 tx=0x7f149c009040 comp rx=0 tx=0).stop 2023-11-28T18:24:30.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.522+0000 7f14b46ea640 1 -- 172.21.15.105:0/1313529552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14ac0b4290 msgr2=0x7f14ac0ae500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:30.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.522+0000 7f14b46ea640 1 --2- 172.21.15.105:0/1313529552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14ac0b4290 0x7f14ac0ae500 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f14a80af040 tx=0x7f14a801acf0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.522+0000 7f14b46ea640 1 -- 172.21.15.105:0/1313529552 shutdown_connections 2023-11-28T18:24:30.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.522+0000 7f14b46ea640 1 --2- 172.21.15.105:0/1313529552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14ac14a9e0 0x7f14ac0aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.522+0000 7f14b46ea640 1 --2- 172.21.15.105:0/1313529552 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14ac15b030 0x7f14ac0aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.522+0000 7f14b46ea640 1 --2- 172.21.15.105:0/1313529552 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f14900668d0 0x7f1490068d90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.522+0000 7f14b46ea640 1 --2- 172.21.15.105:0/1313529552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14ac0b4290 0x7f14ac0ae500 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.522+0000 7f14b46ea640 1 -- 172.21.15.105:0/1313529552 >> 172.21.15.105:0/1313529552 conn(0x7f14ac0ceb90 msgr2=0x7f14ac06c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:30.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.522+0000 7f14b46ea640 1 -- 172.21.15.105:0/1313529552 shutdown_connections 2023-11-28T18:24:30.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.522+0000 7f14b46ea640 1 -- 172.21.15.105:0/1313529552 wait complete. 2023-11-28T18:24:30.547 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:24:30.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.682+0000 7f7871654640 1 Processor -- start 2023-11-28T18:24:30.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.682+0000 7f7871654640 1 -- start start 2023-11-28T18:24:30.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.682+0000 7f7871654640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c15d4f0 0x7f786c0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:30.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.682+0000 7f7871654640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f786c0d3c30 0x7f786c0d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:30.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.682+0000 7f7871654640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786c0b25b0 con 0x7f786c0d1890 2023-11-28T18:24:30.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.682+0000 7f7871654640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f786c0b2900 con 0x7f786c0d3c30 2023-11-28T18:24:30.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.682+0000 7f7871654640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0cea30 con 0x7f786c15d4f0 2023-11-28T18:24:30.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.682+0000 7f786b7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f786c0d1890 0x7f786c0cfa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49338/0 (socket says 172.21.15.105:49338) 2023-11-28T18:24:30.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.682+0000 7f786b7fe640 1 -- 172.21.15.105:0/1285702472 learned_addr learned my addr 172.21.15.105:0/1285702472 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:30.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.682+0000 7f78627fc640 1 --2- 172.21.15.105:0/1285702472 >> v2:172.21.15.148:3300/0 conn(0x7f786c0d3c30 0x7f786c0d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:30.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.682+0000 7f786affd640 1 --2- 172.21.15.105:0/1285702472 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c15d4f0 0x7f786c0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:30.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.682+0000 7f786a7fc640 1 -- 172.21.15.105:0/1285702472 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925489456 0 0) 0x7f786c0b25b0 con 0x7f786c0d1890 2023-11-28T18:24:30.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.682+0000 7f786a7fc640 1 -- 172.21.15.105:0/1285702472 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7858003530 con 0x7f786c0d1890 2023-11-28T18:24:30.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.686+0000 7f786affd640 1 -- 172.21.15.105:0/1285702472 >> v2:172.21.15.148:3300/0 conn(0x7f786c0d3c30 msgr2=0x7f786c0d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:30.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.686+0000 7f786affd640 1 --2- 172.21.15.105:0/1285702472 >> v2:172.21.15.148:3300/0 conn(0x7f786c0d3c30 0x7f786c0d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.686+0000 7f786affd640 1 -- 172.21.15.105:0/1285702472 >> v1:172.21.15.105:6789/0 conn(0x7f786c0d1890 legacy=0x7f786c0cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:30.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.686+0000 7f786affd640 1 -- 172.21.15.105:0/1285702472 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c0ced10 con 0x7f786c15d4f0 2023-11-28T18:24:30.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.686+0000 7f78627fc640 1 --2- 172.21.15.105:0/1285702472 >> v2:172.21.15.148:3300/0 conn(0x7f786c0d3c30 0x7f786c0d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:24:30.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.686+0000 7f786affd640 1 --2- 172.21.15.105:0/1285702472 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c15d4f0 0x7f786c0d0e10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f785c0097c0 tx=0x7f785c01c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46cc26066e79a099 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:30.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.686+0000 7f786a7fc640 1 -- 172.21.15.105:0/1285702472 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c0ab600 con 0x7f786c15d4f0 2023-11-28T18:24:30.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.686+0000 7f786a7fc640 1 -- 172.21.15.105:0/1285702472 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785c0a7050 con 0x7f786c15d4f0 2023-11-28T18:24:30.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.686+0000 7f786a7fc640 1 -- 172.21.15.105:0/1285702472 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c0ab980 con 0x7f786c15d4f0 2023-11-28T18:24:30.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f7871654640 1 -- 172.21.15.105:0/1285702472 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c15d4f0 msgr2=0x7f786c0d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:30.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f7871654640 1 --2- 172.21.15.105:0/1285702472 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c15d4f0 0x7f786c0d0e10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f785c0097c0 tx=0x7f785c01c3c0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f7871654640 1 -- 172.21.15.105:0/1285702472 shutdown_connections 2023-11-28T18:24:30.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f7871654640 1 --2- 172.21.15.105:0/1285702472 >> v2:172.21.15.148:3300/0 conn(0x7f786c0d3c30 0x7f786c0d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f7871654640 1 --2- 172.21.15.105:0/1285702472 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c15d4f0 0x7f786c0d0e10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f7871654640 1 -- 172.21.15.105:0/1285702472 >> 172.21.15.105:0/1285702472 conn(0x7f786c0d3030 msgr2=0x7f786c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:30.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f7871654640 1 -- 172.21.15.105:0/1285702472 shutdown_connections 2023-11-28T18:24:30.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f7871654640 1 -- 172.21.15.105:0/1285702472 wait complete. 2023-11-28T18:24:30.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f7871654640 1 Processor -- start 2023-11-28T18:24:30.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f7871654640 1 -- start start 2023-11-28T18:24:30.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f7871654640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f786c0d1890 0x7f786c1548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:30.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f7871654640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c0d3c30 0x7f786c15c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:30.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f7871654640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f786c15d4f0 0x7f786c15ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:30.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f7871654640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0ca510 con 0x7f786c0d1890 2023-11-28T18:24:30.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f7871654640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0cc670 con 0x7f786c15d4f0 2023-11-28T18:24:30.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f7871654640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0cc950 con 0x7f786c0d3c30 2023-11-28T18:24:30.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f786affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f786c0d1890 0x7f786c1548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:30.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f786affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f786c0d1890 0x7f786c1548d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43814/0 (socket says 172.21.15.105:43814) 2023-11-28T18:24:30.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f786affd640 1 -- 172.21.15.105:0/624600233 learned_addr learned my addr 172.21.15.105:0/624600233 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:30.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f78627fc640 1 --2- 172.21.15.105:0/624600233 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c0d3c30 0x7f786c15c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:30.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f786b7fe640 1 --2- 172.21.15.105:0/624600233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f786c15d4f0 0x7f786c15ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:30.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f786affd640 1 -- 172.21.15.105:0/624600233 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c0d3c30 msgr2=0x7f786c15c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:30.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f786affd640 1 --2- 172.21.15.105:0/624600233 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c0d3c30 0x7f786c15c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f786affd640 1 -- 172.21.15.105:0/624600233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f786c15d4f0 msgr2=0x7f786c15ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:30.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f786affd640 1 --2- 172.21.15.105:0/624600233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f786c15d4f0 0x7f786c15ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.690+0000 7f786affd640 1 -- 172.21.15.105:0/624600233 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7854009420 con 0x7f786c0d1890 2023-11-28T18:24:30.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.694+0000 7f786affd640 1 --2- 172.21.15.105:0/624600233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f786c0d1890 0x7f786c1548d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f785c01aa70 tx=0x7f785c019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:30.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.694+0000 7f7863fff640 1 -- 172.21.15.105:0/624600233 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c006e20 con 0x7f786c0d1890 2023-11-28T18:24:30.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.694+0000 7f7863fff640 1 -- 172.21.15.105:0/624600233 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785c0b5470 con 0x7f786c0d1890 2023-11-28T18:24:30.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.694+0000 7f7871654640 1 -- 172.21.15.105:0/624600233 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785c009420 con 0x7f786c0d1890 2023-11-28T18:24:30.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.694+0000 7f7863fff640 1 -- 172.21.15.105:0/624600233 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c0b5840 con 0x7f786c0d1890 2023-11-28T18:24:30.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.694+0000 7f7871654640 1 -- 172.21.15.105:0/624600233 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786c154e10 con 0x7f786c0d1890 2023-11-28T18:24:30.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.694+0000 7f7871654640 1 -- 172.21.15.105:0/624600233 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786c067570 con 0x7f786c0d1890 2023-11-28T18:24:30.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.694+0000 7f7863fff640 1 -- 172.21.15.105:0/624600233 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f785c0b2040 con 0x7f786c0d1890 2023-11-28T18:24:30.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.694+0000 7f7863fff640 1 --2- 172.21.15.105:0/624600233 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f784c066900 0x7f784c068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:30.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.694+0000 7f7863fff640 1 -- 172.21.15.105:0/624600233 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f785c11eb60 con 0x7f786c0d1890 2023-11-28T18:24:30.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.694+0000 7f78627fc640 1 --2- 172.21.15.105:0/624600233 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f784c066900 0x7f784c068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:30.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.694+0000 7f78627fc640 1 --2- 172.21.15.105:0/624600233 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f784c066900 0x7f784c068dc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f786c0b5ad0 tx=0x7f7854008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:30.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.698+0000 7f7863fff640 1 -- 172.21.15.105:0/624600233 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f785c0eb9a0 con 0x7f786c0d1890 2023-11-28T18:24:30.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.846+0000 7f7871654640 1 -- 172.21.15.105:0/624600233 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f786c158ff0 con 0x7f786c0d1890 2023-11-28T18:24:30.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.850+0000 7f7863fff640 1 -- 172.21.15.105:0/624600233 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f785c0b00a0 con 0x7f786c0d1890 2023-11-28T18:24:30.853 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:30.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.850+0000 7f7871654640 1 -- 172.21.15.105:0/624600233 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f784c066900 msgr2=0x7f784c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:30.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.850+0000 7f7871654640 1 --2- 172.21.15.105:0/624600233 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f784c066900 0x7f784c068dc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f786c0b5ad0 tx=0x7f7854008040 comp rx=0 tx=0).stop 2023-11-28T18:24:30.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.850+0000 7f7871654640 1 -- 172.21.15.105:0/624600233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f786c0d1890 msgr2=0x7f786c1548d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:30.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.850+0000 7f7871654640 1 --2- 172.21.15.105:0/624600233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f786c0d1890 0x7f786c1548d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f785c01aa70 tx=0x7f785c019160 comp rx=0 tx=0).stop 2023-11-28T18:24:30.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.850+0000 7f7871654640 1 -- 172.21.15.105:0/624600233 shutdown_connections 2023-11-28T18:24:30.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.850+0000 7f7871654640 1 --2- 172.21.15.105:0/624600233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f786c15d4f0 0x7f786c15ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.850+0000 7f7871654640 1 --2- 172.21.15.105:0/624600233 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c0d3c30 0x7f786c15c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.850+0000 7f7871654640 1 --2- 172.21.15.105:0/624600233 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f784c066900 0x7f784c068dc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.850+0000 7f7871654640 1 --2- 172.21.15.105:0/624600233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f786c0d1890 0x7f786c1548d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:30.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.850+0000 7f7871654640 1 -- 172.21.15.105:0/624600233 >> 172.21.15.105:0/624600233 conn(0x7f786c0d3030 msgr2=0x7f786c1593a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:30.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.850+0000 7f7871654640 1 -- 172.21.15.105:0/624600233 shutdown_connections 2023-11-28T18:24:30.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:30.850+0000 7f7871654640 1 -- 172.21.15.105:0/624600233 wait complete. 2023-11-28T18:24:30.875 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:24:31.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6fd580640 1 Processor -- start 2023-11-28T18:24:31.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6fd580640 1 -- start start 2023-11-28T18:24:31.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6fd580640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6f80cdb50 0x7fa6f80ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6fd580640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f80d3e60 con 0x7fa6f80d00b0 2023-11-28T18:24:31.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6fd580640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f80cf080 con 0x7fa6f80b8a20 2023-11-28T18:24:31.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6fd580640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80cdfe0 con 0x7fa6f80cdb50 2023-11-28T18:24:31.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f67fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa6f80d00b0 0x7fa6f80b6da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49346/0 (socket says 172.21.15.105:49346) 2023-11-28T18:24:31.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f67fc640 1 -- 172.21.15.105:0/2608111157 learned_addr learned my addr 172.21.15.105:0/2608111157 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:31.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f77fe640 1 --2- 172.21.15.105:0/2608111157 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6f80cdb50 0x7fa6f80ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f5ffb640 1 -- 172.21.15.105:0/2608111157 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 606572277 0 0) 0x7fa6f80d3e60 con 0x7fa6f80d00b0 2023-11-28T18:24:31.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f5ffb640 1 -- 172.21.15.105:0/2608111157 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e0003530 con 0x7fa6f80d00b0 2023-11-28T18:24:31.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f5ffb640 1 -- 172.21.15.105:0/2608111157 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2860414694 0 0) 0x7fa6f80cf080 con 0x7fa6f80b8a20 2023-11-28T18:24:31.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f5ffb640 1 -- 172.21.15.105:0/2608111157 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f80d3e60 con 0x7fa6f80b8a20 2023-11-28T18:24:31.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f5ffb640 1 -- 172.21.15.105:0/2608111157 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1562333281 0 0) 0x7fa6e0003530 con 0x7fa6f80d00b0 2023-11-28T18:24:31.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f5ffb640 1 -- 172.21.15.105:0/2608111157 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6f80cdb50 msgr2=0x7fa6f80ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:31.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f5ffb640 1 --2- 172.21.15.105:0/2608111157 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6f80cdb50 0x7fa6f80ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f5ffb640 1 -- 172.21.15.105:0/2608111157 >> v1:172.21.15.148:6789/0 conn(0x7fa6f80b8a20 legacy=0x7fa6f80cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f5ffb640 1 -- 172.21.15.105:0/2608111157 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f80b5fe0 con 0x7fa6f80d00b0 2023-11-28T18:24:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f5ffb640 1 -- 172.21.15.105:0/2608111157 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fa6ec001680 con 0x7fa6f80d00b0 2023-11-28T18:24:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f5ffb640 1 -- 172.21.15.105:0/2608111157 >> v1:172.21.15.105:6789/0 conn(0x7fa6f80d00b0 legacy=0x7fa6f80b6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f5ffb640 1 --2- 172.21.15.105:0/2608111157 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6f80cdb50 0x7fa6e0003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f5ffb640 1 --2- 172.21.15.105:0/2608111157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6e00040f0 0x7fa6e00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f5ffb640 1 --2- 172.21.15.105:0/2608111157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6e0004d10 0x7fa6e0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f5ffb640 1 -- 172.21.15.105:0/2608111157 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e000baa0 con 0x7fa6e00040f0 2023-11-28T18:24:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f5ffb640 1 -- 172.21.15.105:0/2608111157 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e000bc10 con 0x7fa6e0004d10 2023-11-28T18:24:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f5ffb640 1 -- 172.21.15.105:0/2608111157 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e000bdd0 con 0x7fa6f80cdb50 2023-11-28T18:24:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f67fc640 1 --2- 172.21.15.105:0/2608111157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6e00040f0 0x7fa6e00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f6ffd640 1 --2- 172.21.15.105:0/2608111157 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6f80cdb50 0x7fa6e0003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.010+0000 7fa6f77fe640 1 --2- 172.21.15.105:0/2608111157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6e0004d10 0x7fa6e0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.014+0000 7fa6f67fc640 1 -- 172.21.15.105:0/2608111157 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6f80cdb50 msgr2=0x7fa6e0003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:31.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.014+0000 7fa6f67fc640 1 --2- 172.21.15.105:0/2608111157 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6f80cdb50 0x7fa6e0003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.014+0000 7fa6f67fc640 1 -- 172.21.15.105:0/2608111157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6e0004d10 msgr2=0x7fa6e0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:31.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.014+0000 7fa6f67fc640 1 --2- 172.21.15.105:0/2608111157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6e0004d10 0x7fa6e0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.014+0000 7fa6f67fc640 1 -- 172.21.15.105:0/2608111157 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e0003530 con 0x7fa6e00040f0 2023-11-28T18:24:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.014+0000 7fa6f67fc640 1 --2- 172.21.15.105:0/2608111157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6e00040f0 0x7fa6e00047d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec007fb0 tx=0x7fa6ec01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50314e13e93626a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.014+0000 7fa6f5ffb640 1 -- 172.21.15.105:0/2608111157 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec005ae0 con 0x7fa6e00040f0 2023-11-28T18:24:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.014+0000 7fa6f5ffb640 1 -- 172.21.15.105:0/2608111157 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ec005c40 con 0x7fa6e00040f0 2023-11-28T18:24:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.014+0000 7fa6f5ffb640 1 -- 172.21.15.105:0/2608111157 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0072b0 con 0x7fa6e00040f0 2023-11-28T18:24:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.014+0000 7fa6fd580640 1 -- 172.21.15.105:0/2608111157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6e00040f0 msgr2=0x7fa6e00047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.014+0000 7fa6fd580640 1 --2- 172.21.15.105:0/2608111157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6e00040f0 0x7fa6e00047d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec007fb0 tx=0x7fa6ec01ca40 comp rx=0 tx=0).stop 2023-11-28T18:24:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.014+0000 7fa6f77fe640 1 -- 172.21.15.105:0/2608111157 reap_dead start 2023-11-28T18:24:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.014+0000 7fa6fd580640 1 -- 172.21.15.105:0/2608111157 shutdown_connections 2023-11-28T18:24:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.014+0000 7fa6fd580640 1 -- 172.21.15.105:0/2608111157 >> 172.21.15.105:0/2608111157 conn(0x7fa6f809c340 msgr2=0x7fa6f80d2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.014+0000 7fa6fd580640 1 -- 172.21.15.105:0/2608111157 shutdown_connections 2023-11-28T18:24:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.014+0000 7fa6fd580640 1 -- 172.21.15.105:0/2608111157 wait complete. 2023-11-28T18:24:31.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6fd580640 1 Processor -- start 2023-11-28T18:24:31.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6fd580640 1 -- start start 2023-11-28T18:24:31.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6fd580640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6e00040f0 0x7fa6f81f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6fd580640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6f815eca0 0x7fa6f8158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6fd580640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6f81f1f80 0x7fa6f815cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6fd580640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f81f3720 con 0x7fa6e00040f0 2023-11-28T18:24:31.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6fd580640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f8157510 con 0x7fa6f81f1f80 2023-11-28T18:24:31.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6fd580640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f815f210 con 0x7fa6f815eca0 2023-11-28T18:24:31.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6f6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6e00040f0 0x7fa6f81f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6f6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6e00040f0 0x7fa6f81f31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33324/0 (socket says 172.21.15.105:33324) 2023-11-28T18:24:31.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6f6ffd640 1 -- 172.21.15.105:0/3545654373 learned_addr learned my addr 172.21.15.105:0/3545654373 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:31.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6f67fc640 1 --2- 172.21.15.105:0/3545654373 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6f815eca0 0x7fa6f8158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6f77fe640 1 --2- 172.21.15.105:0/3545654373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6f81f1f80 0x7fa6f815cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6f6ffd640 1 -- 172.21.15.105:0/3545654373 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6f815eca0 msgr2=0x7fa6f8158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:31.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6f6ffd640 1 --2- 172.21.15.105:0/3545654373 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6f815eca0 0x7fa6f8158e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6f6ffd640 1 -- 172.21.15.105:0/3545654373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6f81f1f80 msgr2=0x7fa6f815cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:31.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6f6ffd640 1 --2- 172.21.15.105:0/3545654373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6f81f1f80 0x7fa6f815cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6f6ffd640 1 -- 172.21.15.105:0/3545654373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f815f4f0 con 0x7fa6e00040f0 2023-11-28T18:24:31.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6f77fe640 1 --2- 172.21.15.105:0/3545654373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6f81f1f80 0x7fa6f815cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:31.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6f6ffd640 1 --2- 172.21.15.105:0/3545654373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6e00040f0 0x7fa6f81f31e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8012970 tx=0x7fa6e8012e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:31.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6df7fe640 1 -- 172.21.15.105:0/3545654373 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e80070a0 con 0x7fa6e00040f0 2023-11-28T18:24:31.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6df7fe640 1 -- 172.21.15.105:0/3545654373 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e8010c30 con 0x7fa6e00040f0 2023-11-28T18:24:31.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.018+0000 7fa6df7fe640 1 -- 172.21.15.105:0/3545654373 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8016480 con 0x7fa6e00040f0 2023-11-28T18:24:31.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.022+0000 7fa6fd580640 1 -- 172.21.15.105:0/3545654373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f815f7d0 con 0x7fa6e00040f0 2023-11-28T18:24:31.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.022+0000 7fa6fd580640 1 -- 172.21.15.105:0/3545654373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f8159360 con 0x7fa6e00040f0 2023-11-28T18:24:31.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.022+0000 7fa6fd580640 1 -- 172.21.15.105:0/3545654373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f8067570 con 0x7fa6e00040f0 2023-11-28T18:24:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.022+0000 7fa6df7fe640 1 -- 172.21.15.105:0/3545654373 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa6e80165e0 con 0x7fa6e00040f0 2023-11-28T18:24:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.022+0000 7fa6df7fe640 1 --2- 172.21.15.105:0/3545654373 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6e4066900 0x7fa6e4068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.022+0000 7fa6df7fe640 1 -- 172.21.15.105:0/3545654373 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fa6e80880e0 con 0x7fa6e00040f0 2023-11-28T18:24:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.022+0000 7fa6f67fc640 1 --2- 172.21.15.105:0/3545654373 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6e4066900 0x7fa6e4068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.022+0000 7fa6f67fc640 1 --2- 172.21.15.105:0/3545654373 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6e4066900 0x7fa6e4068dc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec007f80 tx=0x7fa6ec0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:31.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.026+0000 7fa6df7fe640 1 -- 172.21.15.105:0/3545654373 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa6e8054e80 con 0x7fa6e00040f0 2023-11-28T18:24:31.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.178+0000 7fa6fd580640 1 -- 172.21.15.105:0/3545654373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa6f806b490 con 0x7fa6e00040f0 2023-11-28T18:24:31.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.178+0000 7fa6df7fe640 1 -- 172.21.15.105:0/3545654373 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa6e80584a0 con 0x7fa6e00040f0 2023-11-28T18:24:31.186 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:24:31.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.178+0000 7fa6fd580640 1 -- 172.21.15.105:0/3545654373 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6e4066900 msgr2=0x7fa6e4068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:31.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.178+0000 7fa6fd580640 1 --2- 172.21.15.105:0/3545654373 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6e4066900 0x7fa6e4068dc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec007f80 tx=0x7fa6ec0a7070 comp rx=0 tx=0).stop 2023-11-28T18:24:31.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.178+0000 7fa6fd580640 1 -- 172.21.15.105:0/3545654373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6e00040f0 msgr2=0x7fa6f81f31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:31.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.178+0000 7fa6fd580640 1 --2- 172.21.15.105:0/3545654373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6e00040f0 0x7fa6f81f31e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8012970 tx=0x7fa6e8012e30 comp rx=0 tx=0).stop 2023-11-28T18:24:31.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.182+0000 7fa6fd580640 1 -- 172.21.15.105:0/3545654373 shutdown_connections 2023-11-28T18:24:31.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.182+0000 7fa6fd580640 1 --2- 172.21.15.105:0/3545654373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6f81f1f80 0x7fa6f815cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.182+0000 7fa6fd580640 1 --2- 172.21.15.105:0/3545654373 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6f815eca0 0x7fa6f8158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.182+0000 7fa6fd580640 1 --2- 172.21.15.105:0/3545654373 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6e4066900 0x7fa6e4068dc0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.182+0000 7fa6fd580640 1 --2- 172.21.15.105:0/3545654373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6e00040f0 0x7fa6f81f31e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.182+0000 7fa6fd580640 1 -- 172.21.15.105:0/3545654373 >> 172.21.15.105:0/3545654373 conn(0x7fa6f809c340 msgr2=0x7fa6f806bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:31.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.182+0000 7fa6fd580640 1 -- 172.21.15.105:0/3545654373 shutdown_connections 2023-11-28T18:24:31.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.182+0000 7fa6fd580640 1 -- 172.21.15.105:0/3545654373 wait complete. 2023-11-28T18:24:31.206 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T18:24:31.282 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:24:31.287 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:24:31.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.350+0000 7fd5279df640 1 Processor -- start 2023-11-28T18:24:31.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.350+0000 7fd5279df640 1 -- start start 2023-11-28T18:24:31.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.350+0000 7fd5279df640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd52006d610 0x7fd5200cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.350+0000 7fd5279df640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5200ce530 0x7fd5200ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd5279df640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5200ae380 con 0x7fd5200adf30 2023-11-28T18:24:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd5279df640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd520162820 con 0x7fd52006d610 2023-11-28T18:24:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd5279df640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd520161c10 con 0x7fd5200ce530 2023-11-28T18:24:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd525f7c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd5200adf30 0x7fd5201650f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35312/0 (socket says 172.21.15.105:35312) 2023-11-28T18:24:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd525f7c640 1 -- 172.21.15.105:0/82706474 learned_addr learned my addr 172.21.15.105:0/82706474 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd52577b640 1 --2- 172.21.15.105:0/82706474 >> v2:172.21.15.148:3300/0 conn(0x7fd52006d610 0x7fd5200cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd524f7a640 1 --2- 172.21.15.105:0/82706474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5200ce530 0x7fd5200ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd517fff640 1 -- 172.21.15.105:0/82706474 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2520735772 0 0) 0x7fd5180014a0 con 0x7fd5200adf30 2023-11-28T18:24:31.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd517fff640 1 -- 172.21.15.105:0/82706474 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd50c003530 con 0x7fd5200adf30 2023-11-28T18:24:31.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd517fff640 1 -- 172.21.15.105:0/82706474 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1926871260 0 0) 0x7fd50c003530 con 0x7fd5200adf30 2023-11-28T18:24:31.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd517fff640 1 -- 172.21.15.105:0/82706474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5200ce530 msgr2=0x7fd5200ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:31.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd517fff640 1 --2- 172.21.15.105:0/82706474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5200ce530 0x7fd5200ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd517fff640 1 -- 172.21.15.105:0/82706474 >> v2:172.21.15.148:3300/0 conn(0x7fd52006d610 msgr2=0x7fd5200cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:31.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd517fff640 1 --2- 172.21.15.105:0/82706474 >> v2:172.21.15.148:3300/0 conn(0x7fd52006d610 0x7fd5200cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd517fff640 1 -- 172.21.15.105:0/82706474 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd520161ef0 con 0x7fd5200adf30 2023-11-28T18:24:31.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd517fff640 1 -- 172.21.15.105:0/82706474 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fd518001b90 con 0x7fd5200adf30 2023-11-28T18:24:31.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd517fff640 1 -- 172.21.15.105:0/82706474 >> v1:172.21.15.105:6789/0 conn(0x7fd5200adf30 legacy=0x7fd5201650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:31.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd517fff640 1 --2- 172.21.15.105:0/82706474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd50c003980 0x7fd50c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd517fff640 1 --2- 172.21.15.105:0/82706474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd50c004370 0x7fd50c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd517fff640 1 --2- 172.21.15.105:0/82706474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd50c004ec0 0x7fd50c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd517fff640 1 -- 172.21.15.105:0/82706474 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd520162820 con 0x7fd50c004ec0 2023-11-28T18:24:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd517fff640 1 -- 172.21.15.105:0/82706474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd520161c10 con 0x7fd50c004370 2023-11-28T18:24:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd517fff640 1 -- 172.21.15.105:0/82706474 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c00bed0 con 0x7fd50c003980 2023-11-28T18:24:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd525f7c640 1 --2- 172.21.15.105:0/82706474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd50c004ec0 0x7fd50c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd524f7a640 1 --2- 172.21.15.105:0/82706474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd50c004370 0x7fd50c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd52577b640 1 --2- 172.21.15.105:0/82706474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd50c003980 0x7fd50c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd525f7c640 1 -- 172.21.15.105:0/82706474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd50c003980 msgr2=0x7fd50c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd525f7c640 1 --2- 172.21.15.105:0/82706474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd50c003980 0x7fd50c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd525f7c640 1 -- 172.21.15.105:0/82706474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd50c004370 msgr2=0x7fd50c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:31.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd525f7c640 1 --2- 172.21.15.105:0/82706474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd50c004370 0x7fd50c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.354+0000 7fd525f7c640 1 -- 172.21.15.105:0/82706474 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c003530 con 0x7fd50c004ec0 2023-11-28T18:24:31.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.358+0000 7fd525f7c640 1 --2- 172.21.15.105:0/82706474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd50c004ec0 0x7fd50c00b990 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd51800a840 tx=0x7fd51801d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41e83b30e396047 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:31.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.358+0000 7fd517fff640 1 -- 172.21.15.105:0/82706474 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5180ad630 con 0x7fd50c004ec0 2023-11-28T18:24:31.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.358+0000 7fd517fff640 1 -- 172.21.15.105:0/82706474 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5180a8070 con 0x7fd50c004ec0 2023-11-28T18:24:31.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.358+0000 7fd517fff640 1 -- 172.21.15.105:0/82706474 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5180ada30 con 0x7fd50c004ec0 2023-11-28T18:24:31.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.358+0000 7fd5279df640 1 -- 172.21.15.105:0/82706474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd50c004ec0 msgr2=0x7fd50c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:31.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.358+0000 7fd5279df640 1 --2- 172.21.15.105:0/82706474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd50c004ec0 0x7fd50c00b990 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd51800a840 tx=0x7fd51801d340 comp rx=0 tx=0).stop 2023-11-28T18:24:31.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.358+0000 7fd525f7c640 1 -- 172.21.15.105:0/82706474 reap_dead start 2023-11-28T18:24:31.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.358+0000 7fd5279df640 1 -- 172.21.15.105:0/82706474 shutdown_connections 2023-11-28T18:24:31.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.358+0000 7fd5279df640 1 -- 172.21.15.105:0/82706474 >> 172.21.15.105:0/82706474 conn(0x7fd5200ceb90 msgr2=0x7fd5200cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:31.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.358+0000 7fd5279df640 1 -- 172.21.15.105:0/82706474 shutdown_connections 2023-11-28T18:24:31.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.358+0000 7fd5279df640 1 -- 172.21.15.105:0/82706474 wait complete. 2023-11-28T18:24:31.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.358+0000 7fd5279df640 1 Processor -- start 2023-11-28T18:24:31.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.358+0000 7fd5279df640 1 -- start start 2023-11-28T18:24:31.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd5279df640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd50c004ec0 0x7fd520150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd5279df640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd52009ad60 0x7fd52009b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd5279df640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5201ea0c0 0x7fd5201ea570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd5279df640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52009b900 con 0x7fd5201ea0c0 2023-11-28T18:24:31.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd5279df640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5201eaab0 con 0x7fd50c004ec0 2023-11-28T18:24:31.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd5279df640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52009b720 con 0x7fd52009ad60 2023-11-28T18:24:31.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd525f7c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5201ea0c0 0x7fd5201ea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd525f7c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5201ea0c0 0x7fd5201ea570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33346/0 (socket says 172.21.15.105:33346) 2023-11-28T18:24:31.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd525f7c640 1 -- 172.21.15.105:0/2435111692 learned_addr learned my addr 172.21.15.105:0/2435111692 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:31.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd52577b640 1 --2- 172.21.15.105:0/2435111692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd50c004ec0 0x7fd520150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd524f7a640 1 --2- 172.21.15.105:0/2435111692 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd52009ad60 0x7fd52009b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd525f7c640 1 -- 172.21.15.105:0/2435111692 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd52009ad60 msgr2=0x7fd52009b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:31.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd525f7c640 1 --2- 172.21.15.105:0/2435111692 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd52009ad60 0x7fd52009b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd525f7c640 1 -- 172.21.15.105:0/2435111692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd50c004ec0 msgr2=0x7fd520150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:31.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd525f7c640 1 --2- 172.21.15.105:0/2435111692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd50c004ec0 0x7fd520150ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd525f7c640 1 -- 172.21.15.105:0/2435111692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52009a150 con 0x7fd5201ea0c0 2023-11-28T18:24:31.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd525f7c640 1 --2- 172.21.15.105:0/2435111692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5201ea0c0 0x7fd5201ea570 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd51801dbe0 tx=0x7fd50c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:31.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd515ffb640 1 -- 172.21.15.105:0/2435111692 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518001b50 con 0x7fd5201ea0c0 2023-11-28T18:24:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd515ffb640 1 -- 172.21.15.105:0/2435111692 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5180014a0 con 0x7fd5201ea0c0 2023-11-28T18:24:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd515ffb640 1 -- 172.21.15.105:0/2435111692 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5180ad630 con 0x7fd5201ea0c0 2023-11-28T18:24:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd5279df640 1 -- 172.21.15.105:0/2435111692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5201e7ab0 con 0x7fd5201ea0c0 2023-11-28T18:24:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd5279df640 1 -- 172.21.15.105:0/2435111692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5201e8070 con 0x7fd5201ea0c0 2023-11-28T18:24:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd5279df640 1 -- 172.21.15.105:0/2435111692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd520067570 con 0x7fd5201ea0c0 2023-11-28T18:24:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.362+0000 7fd515ffb640 1 -- 172.21.15.105:0/2435111692 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd5180aa030 con 0x7fd5201ea0c0 2023-11-28T18:24:31.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.366+0000 7fd515ffb640 1 --2- 172.21.15.105:0/2435111692 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5040668d0 0x7fd504068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.366+0000 7fd515ffb640 1 -- 172.21.15.105:0/2435111692 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fd51811e2d0 con 0x7fd5201ea0c0 2023-11-28T18:24:31.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.366+0000 7fd52577b640 1 --2- 172.21.15.105:0/2435111692 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5040668d0 0x7fd504068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.366+0000 7fd52577b640 1 --2- 172.21.15.105:0/2435111692 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5040668d0 0x7fd504068d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd5200b4730 tx=0x7fd51c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:31.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.370+0000 7fd515ffb640 1 -- 172.21.15.105:0/2435111692 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd5180eb1c0 con 0x7fd5201ea0c0 2023-11-28T18:24:31.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.522+0000 7fd5279df640 1 -- 172.21.15.105:0/2435111692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd5200ae380 con 0x7fd5201ea0c0 2023-11-28T18:24:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.522+0000 7fd515ffb640 1 -- 172.21.15.105:0/2435111692 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd5200ae380 con 0x7fd5201ea0c0 2023-11-28T18:24:31.528 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:31.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.526+0000 7fd5279df640 1 -- 172.21.15.105:0/2435111692 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5040668d0 msgr2=0x7fd504068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:31.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.526+0000 7fd5279df640 1 --2- 172.21.15.105:0/2435111692 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5040668d0 0x7fd504068d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd5200b4730 tx=0x7fd51c009190 comp rx=0 tx=0).stop 2023-11-28T18:24:31.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.526+0000 7fd5279df640 1 -- 172.21.15.105:0/2435111692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5201ea0c0 msgr2=0x7fd5201ea570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:31.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.526+0000 7fd5279df640 1 --2- 172.21.15.105:0/2435111692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5201ea0c0 0x7fd5201ea570 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd51801dbe0 tx=0x7fd50c005800 comp rx=0 tx=0).stop 2023-11-28T18:24:31.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.526+0000 7fd5279df640 1 -- 172.21.15.105:0/2435111692 shutdown_connections 2023-11-28T18:24:31.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.526+0000 7fd5279df640 1 --2- 172.21.15.105:0/2435111692 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5040668d0 0x7fd504068d90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.526+0000 7fd5279df640 1 --2- 172.21.15.105:0/2435111692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5201ea0c0 0x7fd5201ea570 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.526+0000 7fd5279df640 1 --2- 172.21.15.105:0/2435111692 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd52009ad60 0x7fd52009b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.526+0000 7fd5279df640 1 --2- 172.21.15.105:0/2435111692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd50c004ec0 0x7fd520150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.526+0000 7fd5279df640 1 -- 172.21.15.105:0/2435111692 >> 172.21.15.105:0/2435111692 conn(0x7fd5200ceb90 msgr2=0x7fd5200ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:31.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.526+0000 7fd5279df640 1 -- 172.21.15.105:0/2435111692 shutdown_connections 2023-11-28T18:24:31.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.526+0000 7fd5279df640 1 -- 172.21.15.105:0/2435111692 wait complete. 2023-11-28T18:24:31.549 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T18:24:31.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.730+0000 7f2eb40d6640 1 Processor -- start 2023-11-28T18:24:31.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.730+0000 7f2eb40d6640 1 -- start start 2023-11-28T18:24:31.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.730+0000 7f2eb40d6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2eac0ce2c0 0x7f2eac06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.730+0000 7f2eb40d6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2eac06cd20 0x7f2eac0abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.730+0000 7f2eb40d6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eac0cde30 con 0x7f2eac159610 2023-11-28T18:24:31.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.730+0000 7f2eb40d6640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eac06d190 con 0x7f2eac0ce2c0 2023-11-28T18:24:31.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.730+0000 7f2eb40d6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eac1599f0 con 0x7f2eac06cd20 2023-11-28T18:24:31.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.730+0000 7f2eb1671640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2eac0ce2c0 0x7f2eac06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.730+0000 7f2eb1671640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2eac0ce2c0 0x7f2eac06c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33352/0 (socket says 172.21.15.105:33352) 2023-11-28T18:24:31.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.730+0000 7f2eb1671640 1 -- 172.21.15.105:0/3543557918 learned_addr learned my addr 172.21.15.105:0/3543557918 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:31.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.730+0000 7f2eb1671640 1 -- 172.21.15.105:0/3543557918 >> v2:172.21.15.148:3300/0 conn(0x7f2eac06cd20 msgr2=0x7f2eac0abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:31.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.730+0000 7f2eb2673640 1 --2- 172.21.15.105:0/3543557918 >> v2:172.21.15.148:3300/0 conn(0x7f2eac06cd20 0x7f2eac0abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.730+0000 7f2eb1671640 1 --2- 172.21.15.105:0/3543557918 >> v2:172.21.15.148:3300/0 conn(0x7f2eac06cd20 0x7f2eac0abd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.730+0000 7f2eb1671640 1 -- 172.21.15.105:0/3543557918 >> v1:172.21.15.148:6789/0 conn(0x7f2eac159610 legacy=0x7f2eac0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:31.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.730+0000 7f2eb1671640 1 -- 172.21.15.105:0/3543557918 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eac06de20 con 0x7f2eac0ce2c0 2023-11-28T18:24:31.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.730+0000 7f2eb2673640 1 --2- 172.21.15.105:0/3543557918 >> v2:172.21.15.148:3300/0 conn(0x7f2eac06cd20 0x7f2eac0abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:31.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.734+0000 7f2eb1671640 1 --2- 172.21.15.105:0/3543557918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eac0ce2c0 0x7f2eac06c7e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2eac06e600 tx=0x7f2e9c01e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88d160f0b15de6c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:31.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.734+0000 7f2eb0e70640 1 -- 172.21.15.105:0/3543557918 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c004230 con 0x7f2eac0ce2c0 2023-11-28T18:24:31.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.734+0000 7f2eb0e70640 1 -- 172.21.15.105:0/3543557918 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9c0ab070 con 0x7f2eac0ce2c0 2023-11-28T18:24:31.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.734+0000 7f2eb0e70640 1 -- 172.21.15.105:0/3543557918 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c0a7790 con 0x7f2eac0ce2c0 2023-11-28T18:24:31.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.734+0000 7f2eb40d6640 1 -- 172.21.15.105:0/3543557918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eac0ce2c0 msgr2=0x7f2eac06c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:31.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.734+0000 7f2eb40d6640 1 --2- 172.21.15.105:0/3543557918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eac0ce2c0 0x7f2eac06c7e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2eac06e600 tx=0x7f2e9c01e940 comp rx=0 tx=0).stop 2023-11-28T18:24:31.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.734+0000 7f2eb40d6640 1 -- 172.21.15.105:0/3543557918 shutdown_connections 2023-11-28T18:24:31.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.734+0000 7f2eb40d6640 1 --2- 172.21.15.105:0/3543557918 >> v2:172.21.15.148:3300/0 conn(0x7f2eac06cd20 0x7f2eac0abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.734+0000 7f2eb40d6640 1 --2- 172.21.15.105:0/3543557918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eac0ce2c0 0x7f2eac06c7e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.734+0000 7f2eb40d6640 1 -- 172.21.15.105:0/3543557918 >> 172.21.15.105:0/3543557918 conn(0x7f2eac0ceb90 msgr2=0x7f2eac0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:31.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.734+0000 7f2eb40d6640 1 -- 172.21.15.105:0/3543557918 shutdown_connections 2023-11-28T18:24:31.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.734+0000 7f2eb40d6640 1 -- 172.21.15.105:0/3543557918 wait complete. 2023-11-28T18:24:31.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb40d6640 1 Processor -- start 2023-11-28T18:24:31.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb40d6640 1 -- start start 2023-11-28T18:24:31.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb40d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2eac06cd20 0x7f2eac1f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb40d6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eac0ce2c0 0x7f2eac162c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb40d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2eac159610 0x7f2eac1516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb40d6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac1485c0 con 0x7f2eac0ce2c0 2023-11-28T18:24:31.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb40d6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac155fb0 con 0x7f2eac06cd20 2023-11-28T18:24:31.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb40d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac147d60 con 0x7f2eac159610 2023-11-28T18:24:31.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb1671640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eac0ce2c0 0x7f2eac162c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb1671640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eac0ce2c0 0x7f2eac162c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33368/0 (socket says 172.21.15.105:33368) 2023-11-28T18:24:31.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb1671640 1 -- 172.21.15.105:0/28070416 learned_addr learned my addr 172.21.15.105:0/28070416 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:31.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb1e72640 1 --2- 172.21.15.105:0/28070416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2eac06cd20 0x7f2eac1f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb2673640 1 --2- 172.21.15.105:0/28070416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2eac159610 0x7f2eac1516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb1671640 1 -- 172.21.15.105:0/28070416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2eac159610 msgr2=0x7f2eac1516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:31.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb1671640 1 --2- 172.21.15.105:0/28070416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2eac159610 0x7f2eac1516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb1671640 1 -- 172.21.15.105:0/28070416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2eac06cd20 msgr2=0x7f2eac1f5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:31.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb1671640 1 --2- 172.21.15.105:0/28070416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2eac06cd20 0x7f2eac1f5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb1671640 1 -- 172.21.15.105:0/28070416 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c00b780 con 0x7f2eac0ce2c0 2023-11-28T18:24:31.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb1e72640 1 --2- 172.21.15.105:0/28070416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2eac06cd20 0x7f2eac1f5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:31.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb2673640 1 --2- 172.21.15.105:0/28070416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2eac159610 0x7f2eac1516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:31.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb1671640 1 --2- 172.21.15.105:0/28070416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eac0ce2c0 0x7f2eac162c70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c000c10 tx=0x7f2e9c01b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:31.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2ea27fc640 1 -- 172.21.15.105:0/28070416 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c0a7b20 con 0x7f2eac0ce2c0 2023-11-28T18:24:31.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2ea27fc640 1 -- 172.21.15.105:0/28070416 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9c0a7ca0 con 0x7f2eac0ce2c0 2023-11-28T18:24:31.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2ea27fc640 1 -- 172.21.15.105:0/28070416 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c0a7e50 con 0x7f2eac0ce2c0 2023-11-28T18:24:31.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb40d6640 1 -- 172.21.15.105:0/28070416 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eac0ccff0 con 0x7f2eac0ce2c0 2023-11-28T18:24:31.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb40d6640 1 -- 172.21.15.105:0/28070416 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eac1482e0 con 0x7f2eac0ce2c0 2023-11-28T18:24:31.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.738+0000 7f2eb40d6640 1 -- 172.21.15.105:0/28070416 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eac155d80 con 0x7f2eac0ce2c0 2023-11-28T18:24:31.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.742+0000 7f2ea27fc640 1 -- 172.21.15.105:0/28070416 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f2e9c003620 con 0x7f2eac0ce2c0 2023-11-28T18:24:31.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.742+0000 7f2ea27fc640 1 --2- 172.21.15.105:0/28070416 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2e900668d0 0x7f2e90068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:31.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.742+0000 7f2ea27fc640 1 -- 172.21.15.105:0/28070416 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f2e9c11e680 con 0x7f2eac0ce2c0 2023-11-28T18:24:31.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.742+0000 7f2eb1e72640 1 --2- 172.21.15.105:0/28070416 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2e900668d0 0x7f2e90068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:31.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.742+0000 7f2eb1e72640 1 --2- 172.21.15.105:0/28070416 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2e900668d0 0x7f2e90068d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8003e40 tx=0x7f2ea8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:31.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.746+0000 7f2ea27fc640 1 -- 172.21.15.105:0/28070416 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2eac155d80 con 0x7f2eac0ce2c0 2023-11-28T18:24:31.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.910+0000 7f2eb40d6640 1 -- 172.21.15.105:0/28070416 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2eac155d80 con 0x7f2eac0ce2c0 2023-11-28T18:24:31.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.910+0000 7f2ea27fc640 1 -- 172.21.15.105:0/28070416 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2eac155d80 con 0x7f2eac0ce2c0 2023-11-28T18:24:31.914 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2023-11-28T18:24:31.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.910+0000 7f2eb40d6640 1 -- 172.21.15.105:0/28070416 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2e900668d0 msgr2=0x7f2e90068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:31.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.910+0000 7f2eb40d6640 1 --2- 172.21.15.105:0/28070416 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2e900668d0 0x7f2e90068d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8003e40 tx=0x7f2ea8009040 comp rx=0 tx=0).stop 2023-11-28T18:24:31.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.910+0000 7f2eb40d6640 1 -- 172.21.15.105:0/28070416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eac0ce2c0 msgr2=0x7f2eac162c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:31.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.910+0000 7f2eb40d6640 1 --2- 172.21.15.105:0/28070416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eac0ce2c0 0x7f2eac162c70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c000c10 tx=0x7f2e9c01b6e0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.910+0000 7f2eb40d6640 1 -- 172.21.15.105:0/28070416 shutdown_connections 2023-11-28T18:24:31.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.910+0000 7f2eb40d6640 1 --2- 172.21.15.105:0/28070416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2eac159610 0x7f2eac1516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.910+0000 7f2eb40d6640 1 --2- 172.21.15.105:0/28070416 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2e900668d0 0x7f2e90068d90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.910+0000 7f2eb40d6640 1 --2- 172.21.15.105:0/28070416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eac0ce2c0 0x7f2eac162c70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.910+0000 7f2eb40d6640 1 --2- 172.21.15.105:0/28070416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2eac06cd20 0x7f2eac1f5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:31.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.910+0000 7f2eb40d6640 1 -- 172.21.15.105:0/28070416 >> 172.21.15.105:0/28070416 conn(0x7f2eac0ceb90 msgr2=0x7f2eac073180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:31.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.910+0000 7f2eb40d6640 1 -- 172.21.15.105:0/28070416 shutdown_connections 2023-11-28T18:24:31.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:31.910+0000 7f2eb40d6640 1 -- 172.21.15.105:0/28070416 wait complete. 2023-11-28T18:24:31.935 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi148.front.sepia.ceph.com... 2023-11-28T18:24:31.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:31.935 DEBUG:teuthology.orchestra.run.smithi148:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-28T18:24:31.965 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:24:31.966 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:24:31.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:31.966 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T18:24:32.035 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:24:32.035 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:24:32.044 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:24:32.044 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:24:32.093 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:24:32.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.270+0000 7f6094f96640 1 Processor -- start 2023-11-28T18:24:32.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.270+0000 7f6094f96640 1 -- start start 2023-11-28T18:24:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.270+0000 7f6094f96640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f60900ce2c0 0x7f609006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.270+0000 7f6094f96640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f609006cd20 0x7f60900abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.270+0000 7f6094f96640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60900cde30 con 0x7f6090159610 2023-11-28T18:24:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.270+0000 7f6094f96640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f609006d1b0 con 0x7f60900ce2c0 2023-11-28T18:24:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.270+0000 7f6094f96640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60901599f0 con 0x7f609006cd20 2023-11-28T18:24:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.270+0000 7f608e59c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f6090159610 0x7f60900cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35324/0 (socket says 172.21.15.105:35324) 2023-11-28T18:24:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.270+0000 7f608e59c640 1 -- 172.21.15.105:0/4043286034 learned_addr learned my addr 172.21.15.105:0/4043286034 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f6087fff640 1 --2- 172.21.15.105:0/4043286034 >> v2:172.21.15.148:3300/0 conn(0x7f60900ce2c0 0x7f609006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f608ed9d640 1 --2- 172.21.15.105:0/4043286034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f609006cd20 0x7f60900abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f608dd9b640 1 -- 172.21.15.105:0/4043286034 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1683961533 0 0) 0x7f60900cde30 con 0x7f6090159610 2023-11-28T18:24:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f608dd9b640 1 -- 172.21.15.105:0/4043286034 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f607c004640 con 0x7f6090159610 2023-11-28T18:24:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f608dd9b640 1 -- 172.21.15.105:0/4043286034 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2091716692 0 0) 0x7f607c004640 con 0x7f6090159610 2023-11-28T18:24:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f608dd9b640 1 -- 172.21.15.105:0/4043286034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f609006cd20 msgr2=0x7f60900abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:32.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f608dd9b640 1 --2- 172.21.15.105:0/4043286034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f609006cd20 0x7f60900abd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f608dd9b640 1 -- 172.21.15.105:0/4043286034 >> v2:172.21.15.148:3300/0 conn(0x7f60900ce2c0 msgr2=0x7f609006c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f608dd9b640 1 --2- 172.21.15.105:0/4043286034 >> v2:172.21.15.148:3300/0 conn(0x7f60900ce2c0 0x7f609006c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f608dd9b640 1 -- 172.21.15.105:0/4043286034 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609006de20 con 0x7f6090159610 2023-11-28T18:24:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f608dd9b640 1 -- 172.21.15.105:0/4043286034 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f6080001680 con 0x7f6090159610 2023-11-28T18:24:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f608dd9b640 1 -- 172.21.15.105:0/4043286034 >> v1:172.21.15.105:6789/0 conn(0x7f6090159610 legacy=0x7f60900cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f608dd9b640 1 --2- 172.21.15.105:0/4043286034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c004a90 0x7f607c004eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f608dd9b640 1 --2- 172.21.15.105:0/4043286034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f607c0058e0 0x7f607c005d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f608dd9b640 1 --2- 172.21.15.105:0/4043286034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f609006cd20 0x7f607c0062a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f608dd9b640 1 -- 172.21.15.105:0/4043286034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609006d1b0 con 0x7f607c0058e0 2023-11-28T18:24:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f608dd9b640 1 -- 172.21.15.105:0/4043286034 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60901599f0 con 0x7f607c004a90 2023-11-28T18:24:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f608dd9b640 1 -- 172.21.15.105:0/4043286034 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f607c00caa0 con 0x7f609006cd20 2023-11-28T18:24:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f6087fff640 1 --2- 172.21.15.105:0/4043286034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f607c0058e0 0x7f607c005d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f608e59c640 1 --2- 172.21.15.105:0/4043286034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c004a90 0x7f607c004eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f608ed9d640 1 --2- 172.21.15.105:0/4043286034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f609006cd20 0x7f607c0062a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f6087fff640 1 -- 172.21.15.105:0/4043286034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f609006cd20 msgr2=0x7f607c0062a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f6087fff640 1 --2- 172.21.15.105:0/4043286034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f609006cd20 0x7f607c0062a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f6087fff640 1 -- 172.21.15.105:0/4043286034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c004a90 msgr2=0x7f607c004eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f6087fff640 1 --2- 172.21.15.105:0/4043286034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c004a90 0x7f607c004eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.274+0000 7f6087fff640 1 -- 172.21.15.105:0/4043286034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c004640 con 0x7f607c0058e0 2023-11-28T18:24:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.278+0000 7f6087fff640 1 --2- 172.21.15.105:0/4043286034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f607c0058e0 0x7f607c005d60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6078009ae0 tx=0x7f607801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b8a8fb71da438d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.278+0000 7f608dd9b640 1 -- 172.21.15.105:0/4043286034 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60780a55a0 con 0x7f607c0058e0 2023-11-28T18:24:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.278+0000 7f608dd9b640 1 -- 172.21.15.105:0/4043286034 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60780a5700 con 0x7f607c0058e0 2023-11-28T18:24:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.278+0000 7f608dd9b640 1 -- 172.21.15.105:0/4043286034 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60780a5ab0 con 0x7f607c0058e0 2023-11-28T18:24:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.278+0000 7f6094f96640 1 -- 172.21.15.105:0/4043286034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f607c0058e0 msgr2=0x7f607c005d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.278+0000 7f6094f96640 1 --2- 172.21.15.105:0/4043286034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f607c0058e0 0x7f607c005d60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6078009ae0 tx=0x7f607801ca90 comp rx=0 tx=0).stop 2023-11-28T18:24:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.278+0000 7f608ed9d640 1 -- 172.21.15.105:0/4043286034 reap_dead start 2023-11-28T18:24:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.278+0000 7f6094f96640 1 -- 172.21.15.105:0/4043286034 shutdown_connections 2023-11-28T18:24:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.278+0000 7f6094f96640 1 -- 172.21.15.105:0/4043286034 >> 172.21.15.105:0/4043286034 conn(0x7f60900ceb90 msgr2=0x7f60900cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:32.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.278+0000 7f6094f96640 1 -- 172.21.15.105:0/4043286034 shutdown_connections 2023-11-28T18:24:32.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.278+0000 7f6094f96640 1 -- 172.21.15.105:0/4043286034 wait complete. 2023-11-28T18:24:32.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.278+0000 7f6094f96640 1 Processor -- start 2023-11-28T18:24:32.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.278+0000 7f6094f96640 1 -- start start 2023-11-28T18:24:32.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.282+0000 7f6094f96640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f607c0058e0 0x7f60900b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:32.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.282+0000 7f6094f96640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f609015c5f0 0x7f60901f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:32.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.282+0000 7f6094f96640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60901f59f0 0x7f60901f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:32.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.282+0000 7f6094f96640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609013ee50 con 0x7f607c0058e0 2023-11-28T18:24:32.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.282+0000 7f6094f96640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60900b2090 con 0x7f609015c5f0 2023-11-28T18:24:32.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.282+0000 7f6094f96640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60900ad1e0 con 0x7f60901f59f0 2023-11-28T18:24:32.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.282+0000 7f608e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f607c0058e0 0x7f60900b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:32.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.282+0000 7f608e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f607c0058e0 0x7f60900b19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33400/0 (socket says 172.21.15.105:33400) 2023-11-28T18:24:32.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.282+0000 7f608e59c640 1 -- 172.21.15.105:0/243026947 learned_addr learned my addr 172.21.15.105:0/243026947 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:32.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.282+0000 7f608e59c640 1 -- 172.21.15.105:0/243026947 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60901f59f0 msgr2=0x7f60901f5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:32.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.282+0000 7f608e59c640 1 --2- 172.21.15.105:0/243026947 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60901f59f0 0x7f60901f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.282+0000 7f608e59c640 1 -- 172.21.15.105:0/243026947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f609015c5f0 msgr2=0x7f60901f54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:32.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.282+0000 7f608e59c640 1 --2- 172.21.15.105:0/243026947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f609015c5f0 0x7f60901f54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.282+0000 7f608e59c640 1 -- 172.21.15.105:0/243026947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60900ad4c0 con 0x7f607c0058e0 2023-11-28T18:24:32.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.282+0000 7f608e59c640 1 --2- 172.21.15.105:0/243026947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f607c0058e0 0x7f60900b19b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6080012510 tx=0x7f60800129d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:32.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.282+0000 7f6086ffd640 1 -- 172.21.15.105:0/243026947 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6080014810 con 0x7f607c0058e0 2023-11-28T18:24:32.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.282+0000 7f6086ffd640 1 -- 172.21.15.105:0/243026947 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6080014970 con 0x7f607c0058e0 2023-11-28T18:24:32.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.282+0000 7f6086ffd640 1 -- 172.21.15.105:0/243026947 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6080014d20 con 0x7f607c0058e0 2023-11-28T18:24:32.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.282+0000 7f6094f96640 1 -- 172.21.15.105:0/243026947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60900ad7a0 con 0x7f607c0058e0 2023-11-28T18:24:32.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.282+0000 7f6094f96640 1 -- 172.21.15.105:0/243026947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609015ca70 con 0x7f607c0058e0 2023-11-28T18:24:32.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.282+0000 7f6094f96640 1 -- 172.21.15.105:0/243026947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6090067570 con 0x7f607c0058e0 2023-11-28T18:24:32.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.282+0000 7f6086ffd640 1 -- 172.21.15.105:0/243026947 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f60800153e0 con 0x7f607c0058e0 2023-11-28T18:24:32.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.286+0000 7f6086ffd640 1 --2- 172.21.15.105:0/243026947 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6070066900 0x7f6070068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:32.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.286+0000 7f6086ffd640 1 -- 172.21.15.105:0/243026947 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f60800870a0 con 0x7f607c0058e0 2023-11-28T18:24:32.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.286+0000 7f6087fff640 1 --2- 172.21.15.105:0/243026947 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6070066900 0x7f6070068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:32.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.286+0000 7f6087fff640 1 --2- 172.21.15.105:0/243026947 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6070066900 0x7f6070068dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f60780af000 tx=0x7f6078004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:32.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.286+0000 7f6086ffd640 1 -- 172.21.15.105:0/243026947 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6080053ee0 con 0x7f607c0058e0 2023-11-28T18:24:32.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.442+0000 7f6094f96640 1 -- 172.21.15.105:0/243026947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60900cf950 con 0x7f607c0058e0 2023-11-28T18:24:32.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.446+0000 7f6086ffd640 1 -- 172.21.15.105:0/243026947 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18887 (secure 0 0 0) 0x7f608001c090 con 0x7f607c0058e0 2023-11-28T18:24:32.450 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:32.451 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:24:15.048168+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:24:32.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.446+0000 7f6094f96640 1 -- 172.21.15.105:0/243026947 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6070066900 msgr2=0x7f6070068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:32.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.446+0000 7f6094f96640 1 --2- 172.21.15.105:0/243026947 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6070066900 0x7f6070068dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f60780af000 tx=0x7f6078004740 comp rx=0 tx=0).stop 2023-11-28T18:24:32.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.446+0000 7f6094f96640 1 -- 172.21.15.105:0/243026947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f607c0058e0 msgr2=0x7f60900b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:32.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.446+0000 7f6094f96640 1 --2- 172.21.15.105:0/243026947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f607c0058e0 0x7f60900b19b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6080012510 tx=0x7f60800129d0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.446+0000 7f6094f96640 1 -- 172.21.15.105:0/243026947 shutdown_connections 2023-11-28T18:24:32.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.446+0000 7f6094f96640 1 --2- 172.21.15.105:0/243026947 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60901f59f0 0x7f60901f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.446+0000 7f6094f96640 1 --2- 172.21.15.105:0/243026947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f609015c5f0 0x7f60901f54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.446+0000 7f6094f96640 1 --2- 172.21.15.105:0/243026947 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6070066900 0x7f6070068dc0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.446+0000 7f6094f96640 1 --2- 172.21.15.105:0/243026947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f607c0058e0 0x7f60900b19b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.446+0000 7f6094f96640 1 -- 172.21.15.105:0/243026947 >> 172.21.15.105:0/243026947 conn(0x7f60900ceb90 msgr2=0x7f6090158210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:32.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.446+0000 7f6094f96640 1 -- 172.21.15.105:0/243026947 shutdown_connections 2023-11-28T18:24:32.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.446+0000 7f6094f96640 1 -- 172.21.15.105:0/243026947 wait complete. 2023-11-28T18:24:32.471 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:23:31.485422+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:23:32.274378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T18:23:36.100071+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T18:24:32.472 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:24:32.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.606+0000 7f0354f96640 1 Processor -- start 2023-11-28T18:24:32.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.606+0000 7f0354f96640 1 -- start start 2023-11-28T18:24:32.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.610+0000 7f0354f96640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0350157910 0x7f03500c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:32.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.610+0000 7f0354f96640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0350070920 con 0x7f035006cc90 2023-11-28T18:24:32.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.610+0000 7f0354f96640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f035006d750 con 0x7f035015dfe0 2023-11-28T18:24:32.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.610+0000 7f0354f96640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0350157cf0 con 0x7f0350157910 2023-11-28T18:24:32.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.610+0000 7f034f7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0350157910 0x7f03500c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:32.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.610+0000 7f034effd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f035006cc90 0x7f035015d8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35340/0 (socket says 172.21.15.105:35340) 2023-11-28T18:24:32.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.610+0000 7f034f7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0350157910 0x7f03500c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33402/0 (socket says 172.21.15.105:33402) 2023-11-28T18:24:32.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.610+0000 7f034effd640 1 -- 172.21.15.105:0/4093825801 learned_addr learned my addr 172.21.15.105:0/4093825801 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:32.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.610+0000 7f034f7fe640 1 -- 172.21.15.105:0/4093825801 >> v1:172.21.15.148:6789/0 conn(0x7f035015dfe0 legacy=0x7f03500716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:32.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.610+0000 7f034f7fe640 1 -- 172.21.15.105:0/4093825801 >> v1:172.21.15.105:6789/0 conn(0x7f035006cc90 legacy=0x7f035015d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:32.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.610+0000 7f034f7fe640 1 -- 172.21.15.105:0/4093825801 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0350070be0 con 0x7f0350157910 2023-11-28T18:24:32.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.614+0000 7f034f7fe640 1 --2- 172.21.15.105:0/4093825801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0350157910 0x7f03500c9030 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f03400097c0 tx=0x7f034001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16236c0286b6a5a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:32.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.614+0000 7f034e7fc640 1 -- 172.21.15.105:0/4093825801 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03400ab5f0 con 0x7f0350157910 2023-11-28T18:24:32.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.614+0000 7f034e7fc640 1 -- 172.21.15.105:0/4093825801 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03400ab750 con 0x7f0350157910 2023-11-28T18:24:32.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.614+0000 7f034e7fc640 1 -- 172.21.15.105:0/4093825801 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03400abae0 con 0x7f0350157910 2023-11-28T18:24:32.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.614+0000 7f0354f96640 1 -- 172.21.15.105:0/4093825801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0350157910 msgr2=0x7f03500c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:32.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.614+0000 7f0354f96640 1 --2- 172.21.15.105:0/4093825801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0350157910 0x7f03500c9030 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f03400097c0 tx=0x7f034001c3f0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.614+0000 7f0354f96640 1 -- 172.21.15.105:0/4093825801 shutdown_connections 2023-11-28T18:24:32.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.614+0000 7f0354f96640 1 --2- 172.21.15.105:0/4093825801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0350157910 0x7f03500c9030 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.614+0000 7f0354f96640 1 -- 172.21.15.105:0/4093825801 >> 172.21.15.105:0/4093825801 conn(0x7f03500ce9e0 msgr2=0x7f03500cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:32.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.614+0000 7f0354f96640 1 -- 172.21.15.105:0/4093825801 shutdown_connections 2023-11-28T18:24:32.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.614+0000 7f0354f96640 1 -- 172.21.15.105:0/4093825801 wait complete. 2023-11-28T18:24:32.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.614+0000 7f0354f96640 1 Processor -- start 2023-11-28T18:24:32.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.614+0000 7f0354f96640 1 -- start start 2023-11-28T18:24:32.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f0354f96640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f035006cc90 0x7f035013f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f0354f96640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0350157910 0x7f035014f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f0354f96640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f035015dfe0 0x7f035015ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f0354f96640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035013f9d0 con 0x7f035006cc90 2023-11-28T18:24:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f0354f96640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035014ea70 con 0x7f035015dfe0 2023-11-28T18:24:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f0354f96640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03500ae620 con 0x7f0350157910 2023-11-28T18:24:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f034f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f035006cc90 0x7f035013f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f034f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f035006cc90 0x7f035013f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33416/0 (socket says 172.21.15.105:33416) 2023-11-28T18:24:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f034f7fe640 1 -- 172.21.15.105:0/612590821 learned_addr learned my addr 172.21.15.105:0/612590821 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f034effd640 1 --2- 172.21.15.105:0/612590821 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0350157910 0x7f035014f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f034ffff640 1 --2- 172.21.15.105:0/612590821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f035015dfe0 0x7f035015ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f034f7fe640 1 -- 172.21.15.105:0/612590821 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0350157910 msgr2=0x7f035014f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f034f7fe640 1 --2- 172.21.15.105:0/612590821 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0350157910 0x7f035014f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f034f7fe640 1 -- 172.21.15.105:0/612590821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f035015dfe0 msgr2=0x7f035015ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f034f7fe640 1 --2- 172.21.15.105:0/612590821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f035015dfe0 0x7f035015ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f034f7fe640 1 -- 172.21.15.105:0/612590821 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0340009420 con 0x7f035006cc90 2023-11-28T18:24:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f034effd640 1 --2- 172.21.15.105:0/612590821 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0350157910 0x7f035014f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f034f7fe640 1 --2- 172.21.15.105:0/612590821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f035006cc90 0x7f035013f490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f03400ad040 tx=0x7f034001ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f0337fff640 1 -- 172.21.15.105:0/612590821 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03400b5810 con 0x7f035006cc90 2023-11-28T18:24:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f0337fff640 1 -- 172.21.15.105:0/612590821 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03400b5990 con 0x7f035006cc90 2023-11-28T18:24:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f0354f96640 1 -- 172.21.15.105:0/612590821 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0350072360 con 0x7f035006cc90 2023-11-28T18:24:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f0337fff640 1 -- 172.21.15.105:0/612590821 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03400b5d40 con 0x7f035006cc90 2023-11-28T18:24:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f0354f96640 1 -- 172.21.15.105:0/612590821 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f035014efa0 con 0x7f035006cc90 2023-11-28T18:24:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f0354f96640 1 -- 172.21.15.105:0/612590821 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03500679d0 con 0x7f035006cc90 2023-11-28T18:24:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.618+0000 7f0337fff640 1 -- 172.21.15.105:0/612590821 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f03400bf410 con 0x7f035006cc90 2023-11-28T18:24:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.622+0000 7f0337fff640 1 --2- 172.21.15.105:0/612590821 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f033c0668d0 0x7f033c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.622+0000 7f0337fff640 1 -- 172.21.15.105:0/612590821 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f03400a8050 con 0x7f035006cc90 2023-11-28T18:24:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.622+0000 7f034effd640 1 --2- 172.21.15.105:0/612590821 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f033c0668d0 0x7f033c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.622+0000 7f034effd640 1 --2- 172.21.15.105:0/612590821 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f033c0668d0 0x7f033c068d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f035015cb70 tx=0x7f0344009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.622+0000 7f0337fff640 1 -- 172.21.15.105:0/612590821 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f03400ec2f0 con 0x7f035006cc90 2023-11-28T18:24:32.779 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.774+0000 7f0354f96640 1 -- 172.21.15.105:0/612590821 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0350074580 con 0x7f035006cc90 2023-11-28T18:24:32.780 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.774+0000 7f0337fff640 1 -- 172.21.15.105:0/612590821 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f03400ab5b0 con 0x7f035006cc90 2023-11-28T18:24:32.780 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:24:32.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.778+0000 7f0354f96640 1 -- 172.21.15.105:0/612590821 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f033c0668d0 msgr2=0x7f033c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:32.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.778+0000 7f0354f96640 1 --2- 172.21.15.105:0/612590821 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f033c0668d0 0x7f033c068d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f035015cb70 tx=0x7f0344009040 comp rx=0 tx=0).stop 2023-11-28T18:24:32.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.778+0000 7f0354f96640 1 -- 172.21.15.105:0/612590821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f035006cc90 msgr2=0x7f035013f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:32.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.778+0000 7f0354f96640 1 --2- 172.21.15.105:0/612590821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f035006cc90 0x7f035013f490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f03400ad040 tx=0x7f034001ac30 comp rx=0 tx=0).stop 2023-11-28T18:24:32.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.778+0000 7f0354f96640 1 -- 172.21.15.105:0/612590821 shutdown_connections 2023-11-28T18:24:32.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.778+0000 7f0354f96640 1 --2- 172.21.15.105:0/612590821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f035015dfe0 0x7f035015ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.778+0000 7f0354f96640 1 --2- 172.21.15.105:0/612590821 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0350157910 0x7f035014f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.778+0000 7f0354f96640 1 --2- 172.21.15.105:0/612590821 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f033c0668d0 0x7f033c068d90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.778+0000 7f0354f96640 1 --2- 172.21.15.105:0/612590821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f035006cc90 0x7f035013f490 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.778+0000 7f0354f96640 1 -- 172.21.15.105:0/612590821 >> 172.21.15.105:0/612590821 conn(0x7f03500ce9e0 msgr2=0x7f035006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:32.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.778+0000 7f0354f96640 1 -- 172.21.15.105:0/612590821 shutdown_connections 2023-11-28T18:24:32.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.778+0000 7f0354f96640 1 -- 172.21.15.105:0/612590821 wait complete. 2023-11-28T18:24:32.800 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:24:32.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.934+0000 7f0507e32640 1 Processor -- start 2023-11-28T18:24:32.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.934+0000 7f0507e32640 1 -- start start 2023-11-28T18:24:32.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.934+0000 7f0507e32640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05000b4290 0x7f05000b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:32.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.934+0000 7f0507e32640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0500071960 0x7f050015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:32.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.934+0000 7f0507e32640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05000b4bb0 con 0x7f050015b030 2023-11-28T18:24:32.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.934+0000 7f0507e32640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05000701d0 con 0x7f0500071960 2023-11-28T18:24:32.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.934+0000 7f0507e32640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050006df80 con 0x7f05000b4290 2023-11-28T18:24:32.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.938+0000 7f0505bce640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05000b4290 0x7f05000b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:32.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.938+0000 7f05053cd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0500071960 0x7f050015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:32.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.938+0000 7f0505bce640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05000b4290 0x7f05000b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:40598/0 (socket says 172.21.15.105:40598) 2023-11-28T18:24:32.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.938+0000 7f0505bce640 1 -- 172.21.15.105:0/2932003010 learned_addr learned my addr 172.21.15.105:0/2932003010 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:32.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.938+0000 7f05063cf640 1 --1- 172.21.15.105:0/2932003010 >> v1:172.21.15.148:6789/0 conn(0x7f050015b030 0x7f05000709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:40108/0 (socket says 172.21.15.105:40108) 2023-11-28T18:24:32.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.938+0000 7f0505bce640 1 -- 172.21.15.105:0/2932003010 >> v2:172.21.15.148:3300/0 conn(0x7f0500071960 msgr2=0x7f050015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:32.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.938+0000 7f0505bce640 1 --2- 172.21.15.105:0/2932003010 >> v2:172.21.15.148:3300/0 conn(0x7f0500071960 0x7f050015f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.938+0000 7f0505bce640 1 -- 172.21.15.105:0/2932003010 >> v1:172.21.15.148:6789/0 conn(0x7f050015b030 legacy=0x7f05000709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:32.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.938+0000 7f0505bce640 1 -- 172.21.15.105:0/2932003010 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0500071df0 con 0x7f05000b4290 2023-11-28T18:24:32.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.938+0000 7f0505bce640 1 --2- 172.21.15.105:0/2932003010 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05000b4290 0x7f05000b4670 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009870 tx=0x7f04fc01c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a03e06580f430c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:32.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f0504bcc640 1 -- 172.21.15.105:0/2932003010 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0aa670 con 0x7f05000b4290 2023-11-28T18:24:32.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f0504bcc640 1 -- 172.21.15.105:0/2932003010 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04fc0aa7d0 con 0x7f05000b4290 2023-11-28T18:24:32.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f0504bcc640 1 -- 172.21.15.105:0/2932003010 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0aab60 con 0x7f05000b4290 2023-11-28T18:24:32.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f0507e32640 1 -- 172.21.15.105:0/2932003010 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05000b4290 msgr2=0x7f05000b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:32.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f0507e32640 1 --2- 172.21.15.105:0/2932003010 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05000b4290 0x7f05000b4670 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009870 tx=0x7f04fc01c510 comp rx=0 tx=0).stop 2023-11-28T18:24:32.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f0507e32640 1 -- 172.21.15.105:0/2932003010 shutdown_connections 2023-11-28T18:24:32.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f0507e32640 1 --2- 172.21.15.105:0/2932003010 >> v2:172.21.15.148:3300/0 conn(0x7f0500071960 0x7f050015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f0507e32640 1 --2- 172.21.15.105:0/2932003010 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05000b4290 0x7f05000b4670 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f0507e32640 1 -- 172.21.15.105:0/2932003010 >> 172.21.15.105:0/2932003010 conn(0x7f05000ceb90 msgr2=0x7f05000cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:32.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f0507e32640 1 -- 172.21.15.105:0/2932003010 shutdown_connections 2023-11-28T18:24:32.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f0507e32640 1 -- 172.21.15.105:0/2932003010 wait complete. 2023-11-28T18:24:32.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f0507e32640 1 Processor -- start 2023-11-28T18:24:32.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f0507e32640 1 -- start start 2023-11-28T18:24:32.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f0507e32640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05000b4290 0x7f050015dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:32.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f0507e32640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f050015b030 0x7f05001540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:32.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f0507e32640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0500154900 0x7f05001f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:32.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f0507e32640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050015d870 con 0x7f05000b4290 2023-11-28T18:24:32.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f0507e32640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050015e2d0 con 0x7f050015b030 2023-11-28T18:24:32.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f0507e32640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05000adcd0 con 0x7f0500154900 2023-11-28T18:24:32.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f0505bce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05000b4290 0x7f050015dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:32.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f0505bce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05000b4290 0x7f050015dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33432/0 (socket says 172.21.15.105:33432) 2023-11-28T18:24:32.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f0505bce640 1 -- 172.21.15.105:0/265349646 learned_addr learned my addr 172.21.15.105:0/265349646 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:32.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.942+0000 7f05053cd640 1 --2- 172.21.15.105:0/265349646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f050015b030 0x7f05001540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:32.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.946+0000 7f05063cf640 1 --2- 172.21.15.105:0/265349646 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0500154900 0x7f05001f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:32.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.946+0000 7f0505bce640 1 -- 172.21.15.105:0/265349646 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0500154900 msgr2=0x7f05001f3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:32.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.946+0000 7f0505bce640 1 --2- 172.21.15.105:0/265349646 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0500154900 0x7f05001f3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.946+0000 7f0505bce640 1 -- 172.21.15.105:0/265349646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f050015b030 msgr2=0x7f05001540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:32.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.946+0000 7f0505bce640 1 --2- 172.21.15.105:0/265349646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f050015b030 0x7f05001540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:32.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.946+0000 7f0505bce640 1 -- 172.21.15.105:0/265349646 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04fc0094d0 con 0x7f05000b4290 2023-11-28T18:24:32.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.946+0000 7f0505bce640 1 --2- 172.21.15.105:0/265349646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05000b4290 0x7f050015dd90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0ac040 tx=0x7f04fc01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:32.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.946+0000 7f04f67fc640 1 -- 172.21.15.105:0/265349646 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0aa670 con 0x7f05000b4290 2023-11-28T18:24:32.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.946+0000 7f04f67fc640 1 -- 172.21.15.105:0/265349646 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04fc0b3820 con 0x7f05000b4290 2023-11-28T18:24:32.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.946+0000 7f04f67fc640 1 -- 172.21.15.105:0/265349646 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0b3bd0 con 0x7f05000b4290 2023-11-28T18:24:32.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.946+0000 7f0507e32640 1 -- 172.21.15.105:0/265349646 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05000adfb0 con 0x7f05000b4290 2023-11-28T18:24:32.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.946+0000 7f0507e32640 1 -- 172.21.15.105:0/265349646 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05000ae570 con 0x7f05000b4290 2023-11-28T18:24:32.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.946+0000 7f0507e32640 1 -- 172.21.15.105:0/265349646 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0500067570 con 0x7f05000b4290 2023-11-28T18:24:32.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.946+0000 7f04f67fc640 1 -- 172.21.15.105:0/265349646 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f04fc0b3d30 con 0x7f05000b4290 2023-11-28T18:24:32.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.946+0000 7f04f67fc640 1 --2- 172.21.15.105:0/265349646 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f04e40668d0 0x7f04e4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:32.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.946+0000 7f04f67fc640 1 -- 172.21.15.105:0/265349646 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f04fc11f070 con 0x7f05000b4290 2023-11-28T18:24:32.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.950+0000 7f05053cd640 1 --2- 172.21.15.105:0/265349646 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f04e40668d0 0x7f04e4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:32.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.950+0000 7f05053cd640 1 --2- 172.21.15.105:0/265349646 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f04e40668d0 0x7f04e4068d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0500154d70 tx=0x7f04f0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:32.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:32.954+0000 7f04f67fc640 1 -- 172.21.15.105:0/265349646 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f04fc0ebf60 con 0x7f05000b4290 2023-11-28T18:24:33.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.102+0000 7f0507e32640 1 -- 172.21.15.105:0/265349646 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0500165b60 con 0x7f05000b4290 2023-11-28T18:24:33.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.102+0000 7f04f67fc640 1 -- 172.21.15.105:0/265349646 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f04fc0c30a0 con 0x7f05000b4290 2023-11-28T18:24:33.109 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:33.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.106+0000 7f0507e32640 1 -- 172.21.15.105:0/265349646 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f04e40668d0 msgr2=0x7f04e4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:33.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.106+0000 7f0507e32640 1 --2- 172.21.15.105:0/265349646 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f04e40668d0 0x7f04e4068d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0500154d70 tx=0x7f04f0009190 comp rx=0 tx=0).stop 2023-11-28T18:24:33.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.106+0000 7f0507e32640 1 -- 172.21.15.105:0/265349646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05000b4290 msgr2=0x7f050015dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:33.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.106+0000 7f0507e32640 1 --2- 172.21.15.105:0/265349646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05000b4290 0x7f050015dd90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0ac040 tx=0x7f04fc01ad50 comp rx=0 tx=0).stop 2023-11-28T18:24:33.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.106+0000 7f0507e32640 1 -- 172.21.15.105:0/265349646 shutdown_connections 2023-11-28T18:24:33.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.106+0000 7f0507e32640 1 --2- 172.21.15.105:0/265349646 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0500154900 0x7f05001f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.106+0000 7f0507e32640 1 --2- 172.21.15.105:0/265349646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f050015b030 0x7f05001540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.106+0000 7f0507e32640 1 --2- 172.21.15.105:0/265349646 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f04e40668d0 0x7f04e4068d90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.106+0000 7f0507e32640 1 --2- 172.21.15.105:0/265349646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05000b4290 0x7f050015dd90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.106+0000 7f0507e32640 1 -- 172.21.15.105:0/265349646 >> 172.21.15.105:0/265349646 conn(0x7f05000ceb90 msgr2=0x7f05000cb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:33.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.106+0000 7f0507e32640 1 -- 172.21.15.105:0/265349646 shutdown_connections 2023-11-28T18:24:33.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.106+0000 7f0507e32640 1 -- 172.21.15.105:0/265349646 wait complete. 2023-11-28T18:24:33.130 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:24:33.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.266+0000 7ff94b32a640 1 Processor -- start 2023-11-28T18:24:33.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.266+0000 7ff94b32a640 1 -- start start 2023-11-28T18:24:33.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.266+0000 7ff94b32a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff944154070 0x7ff94415d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:33.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.266+0000 7ff94b32a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff9440adb80 0x7ff9440adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:33.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.266+0000 7ff94b32a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9440ae510 con 0x7ff94406bf50 2023-11-28T18:24:33.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.266+0000 7ff94b32a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff944072540 con 0x7ff9440adb80 2023-11-28T18:24:33.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.266+0000 7ff94b32a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9440cc7f0 con 0x7ff944154070 2023-11-28T18:24:33.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.266+0000 7ff9498c7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff9440adb80 0x7ff9440adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:33.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.266+0000 7ff9490c6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff944154070 0x7ff94415d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:33.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.266+0000 7ff9490c6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff944154070 0x7ff94415d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:40608/0 (socket says 172.21.15.105:40608) 2023-11-28T18:24:33.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.266+0000 7ff9490c6640 1 -- 172.21.15.105:0/4136566565 learned_addr learned my addr 172.21.15.105:0/4136566565 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:33.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.266+0000 7ff9498c7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff9440adb80 0x7ff9440adfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:59616/0 (socket says 172.21.15.105:59616) 2023-11-28T18:24:33.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.266+0000 7ff9490c6640 1 -- 172.21.15.105:0/4136566565 >> v2:172.21.15.148:3300/0 conn(0x7ff9440adb80 msgr2=0x7ff9440adfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:33.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.266+0000 7ff9490c6640 1 --2- 172.21.15.105:0/4136566565 >> v2:172.21.15.148:3300/0 conn(0x7ff9440adb80 0x7ff9440adfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.266+0000 7ff9490c6640 1 -- 172.21.15.105:0/4136566565 >> v1:172.21.15.148:6789/0 conn(0x7ff94406bf50 legacy=0x7ff9440ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:33.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.270+0000 7ff9490c6640 1 -- 172.21.15.105:0/4136566565 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94415dd10 con 0x7ff944154070 2023-11-28T18:24:33.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.270+0000 7ff9498c7640 1 --2- 172.21.15.105:0/4136566565 >> v2:172.21.15.148:3300/0 conn(0x7ff9440adb80 0x7ff9440adfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:24:33.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.270+0000 7ff9490c6640 1 --2- 172.21.15.105:0/4136566565 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff944154070 0x7ff94415d700 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff940009870 tx=0x7ff94001c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b546e997b02df55 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:33.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.270+0000 7ff93bfff640 1 -- 172.21.15.105:0/4136566565 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9400ab600 con 0x7ff944154070 2023-11-28T18:24:33.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.270+0000 7ff93bfff640 1 -- 172.21.15.105:0/4136566565 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9400a7050 con 0x7ff944154070 2023-11-28T18:24:33.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.270+0000 7ff93bfff640 1 -- 172.21.15.105:0/4136566565 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9400ab980 con 0x7ff944154070 2023-11-28T18:24:33.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.270+0000 7ff94b32a640 1 -- 172.21.15.105:0/4136566565 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff944154070 msgr2=0x7ff94415d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:33.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.270+0000 7ff94b32a640 1 --2- 172.21.15.105:0/4136566565 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff944154070 0x7ff94415d700 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff940009870 tx=0x7ff94001c510 comp rx=0 tx=0).stop 2023-11-28T18:24:33.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.270+0000 7ff94b32a640 1 -- 172.21.15.105:0/4136566565 shutdown_connections 2023-11-28T18:24:33.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.270+0000 7ff94b32a640 1 --2- 172.21.15.105:0/4136566565 >> v2:172.21.15.148:3300/0 conn(0x7ff9440adb80 0x7ff9440adfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.270+0000 7ff94b32a640 1 --2- 172.21.15.105:0/4136566565 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff944154070 0x7ff94415d700 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.270+0000 7ff94b32a640 1 -- 172.21.15.105:0/4136566565 >> 172.21.15.105:0/4136566565 conn(0x7ff9440ceb90 msgr2=0x7ff9440cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:33.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.270+0000 7ff94b32a640 1 -- 172.21.15.105:0/4136566565 shutdown_connections 2023-11-28T18:24:33.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.270+0000 7ff94b32a640 1 -- 172.21.15.105:0/4136566565 wait complete. 2023-11-28T18:24:33.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff94b32a640 1 Processor -- start 2023-11-28T18:24:33.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff94b32a640 1 -- start start 2023-11-28T18:24:33.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff94b32a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff94406bf50 0x7ff94414e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:33.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff94b32a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9440adb80 0x7ff94415f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:33.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff94b32a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff944154070 0x7ff944144d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:33.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff94b32a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94414fed0 con 0x7ff944154070 2023-11-28T18:24:33.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff94b32a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9441505c0 con 0x7ff9440adb80 2023-11-28T18:24:33.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff94b32a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff94414ee30 con 0x7ff94406bf50 2023-11-28T18:24:33.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff9498c7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff944154070 0x7ff944144d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:33.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff9490c6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff94406bf50 0x7ff94414e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:33.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff9498c7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff944154070 0x7ff944144d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33436/0 (socket says 172.21.15.105:33436) 2023-11-28T18:24:33.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff9498c7640 1 -- 172.21.15.105:0/3375787682 learned_addr learned my addr 172.21.15.105:0/3375787682 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:33.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff9488c5640 1 --2- 172.21.15.105:0/3375787682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9440adb80 0x7ff94415f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:33.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff9498c7640 1 -- 172.21.15.105:0/3375787682 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff94406bf50 msgr2=0x7ff94414e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:33.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff9498c7640 1 --2- 172.21.15.105:0/3375787682 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff94406bf50 0x7ff94414e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff9498c7640 1 -- 172.21.15.105:0/3375787682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9440adb80 msgr2=0x7ff94415f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:33.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff9498c7640 1 --2- 172.21.15.105:0/3375787682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9440adb80 0x7ff94415f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff9498c7640 1 -- 172.21.15.105:0/3375787682 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93c009a70 con 0x7ff944154070 2023-11-28T18:24:33.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff9490c6640 1 --2- 172.21.15.105:0/3375787682 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff94406bf50 0x7ff94414e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:24:33.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff9498c7640 1 --2- 172.21.15.105:0/3375787682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff944154070 0x7ff944144d20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00efc0 tx=0x7ff93c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:33.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff939ffb640 1 -- 172.21.15.105:0/3375787682 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c009d70 con 0x7ff944154070 2023-11-28T18:24:33.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff939ffb640 1 -- 172.21.15.105:0/3375787682 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93c00f040 con 0x7ff944154070 2023-11-28T18:24:33.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.274+0000 7ff94b32a640 1 -- 172.21.15.105:0/3375787682 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9400094d0 con 0x7ff944154070 2023-11-28T18:24:33.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.278+0000 7ff939ffb640 1 -- 172.21.15.105:0/3375787682 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c0039a0 con 0x7ff944154070 2023-11-28T18:24:33.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.278+0000 7ff94b32a640 1 -- 172.21.15.105:0/3375787682 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff94414f3f0 con 0x7ff944154070 2023-11-28T18:24:33.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.278+0000 7ff94b32a640 1 -- 172.21.15.105:0/3375787682 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9440ccc60 con 0x7ff944154070 2023-11-28T18:24:33.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.278+0000 7ff939ffb640 1 -- 172.21.15.105:0/3375787682 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff93c014480 con 0x7ff944154070 2023-11-28T18:24:33.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.278+0000 7ff939ffb640 1 --2- 172.21.15.105:0/3375787682 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff9280668d0 0x7ff928068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:33.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.278+0000 7ff939ffb640 1 -- 172.21.15.105:0/3375787682 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7ff93c085410 con 0x7ff944154070 2023-11-28T18:24:33.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.278+0000 7ff9490c6640 1 --2- 172.21.15.105:0/3375787682 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff9280668d0 0x7ff928068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:33.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.278+0000 7ff9490c6640 1 --2- 172.21.15.105:0/3375787682 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff9280668d0 0x7ff928068d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff9400ad000 tx=0x7ff9400a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:33.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.282+0000 7ff939ffb640 1 -- 172.21.15.105:0/3375787682 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff9440ccc60 con 0x7ff944154070 2023-11-28T18:24:33.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.434+0000 7ff94b32a640 1 -- 172.21.15.105:0/3375787682 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff9440ccc60 con 0x7ff944154070 2023-11-28T18:24:33.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.434+0000 7ff939ffb640 1 -- 172.21.15.105:0/3375787682 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff9440ccc60 con 0x7ff944154070 2023-11-28T18:24:33.438 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:24:33.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.434+0000 7ff94b32a640 1 -- 172.21.15.105:0/3375787682 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff9280668d0 msgr2=0x7ff928068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:33.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.434+0000 7ff94b32a640 1 --2- 172.21.15.105:0/3375787682 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff9280668d0 0x7ff928068d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff9400ad000 tx=0x7ff9400a6000 comp rx=0 tx=0).stop 2023-11-28T18:24:33.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.434+0000 7ff94b32a640 1 -- 172.21.15.105:0/3375787682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff944154070 msgr2=0x7ff944144d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:33.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.434+0000 7ff94b32a640 1 --2- 172.21.15.105:0/3375787682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff944154070 0x7ff944144d20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00efc0 tx=0x7ff93c009f90 comp rx=0 tx=0).stop 2023-11-28T18:24:33.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.434+0000 7ff94b32a640 1 -- 172.21.15.105:0/3375787682 shutdown_connections 2023-11-28T18:24:33.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.434+0000 7ff94b32a640 1 --2- 172.21.15.105:0/3375787682 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff9280668d0 0x7ff928068d90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.434+0000 7ff94b32a640 1 --2- 172.21.15.105:0/3375787682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff944154070 0x7ff944144d20 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.434+0000 7ff94b32a640 1 --2- 172.21.15.105:0/3375787682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9440adb80 0x7ff94415f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.434+0000 7ff94b32a640 1 --2- 172.21.15.105:0/3375787682 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff94406bf50 0x7ff94414e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.434+0000 7ff94b32a640 1 -- 172.21.15.105:0/3375787682 >> 172.21.15.105:0/3375787682 conn(0x7ff9440ceb90 msgr2=0x7ff9440cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:33.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.434+0000 7ff94b32a640 1 -- 172.21.15.105:0/3375787682 shutdown_connections 2023-11-28T18:24:33.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.434+0000 7ff94b32a640 1 -- 172.21.15.105:0/3375787682 wait complete. 2023-11-28T18:24:33.459 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T18:24:33.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.594+0000 7f5b9bc79640 1 Processor -- start 2023-11-28T18:24:33.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.594+0000 7f5b9bc79640 1 -- start start 2023-11-28T18:24:33.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.594+0000 7f5b9bc79640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b94159610 0x7f5b940ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.594+0000 7f5b9bc79640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5b940cc760 0x7f5b940ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.594+0000 7f5b9bc79640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5b9406c7e0 0x7f5b9406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.594+0000 7f5b9bc79640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b9406d1b0 con 0x7f5b940cc760 2023-11-28T18:24:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.594+0000 7f5b9bc79640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b940cde30 con 0x7f5b9406c7e0 2023-11-28T18:24:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.594+0000 7f5b9bc79640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b941599f0 con 0x7f5b94159610 2023-11-28T18:24:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.594+0000 7f5b99214640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5b940cc760 0x7f5b940ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.594+0000 7f5b99214640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5b940cc760 0x7f5b940ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33450/0 (socket says 172.21.15.105:33450) 2023-11-28T18:24:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.594+0000 7f5b99214640 1 -- 172.21.15.105:0/133257515 learned_addr learned my addr 172.21.15.105:0/133257515 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.594+0000 7f5b99a15640 1 --2- 172.21.15.105:0/133257515 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b94159610 0x7f5b940ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.594+0000 7f5b9a216640 1 --2- 172.21.15.105:0/133257515 >> v2:172.21.15.148:3300/0 conn(0x7f5b9406c7e0 0x7f5b9406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.594+0000 7f5b99214640 1 -- 172.21.15.105:0/133257515 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b94159610 msgr2=0x7f5b940ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.594+0000 7f5b99214640 1 --2- 172.21.15.105:0/133257515 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b94159610 0x7f5b940ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.594+0000 7f5b99214640 1 -- 172.21.15.105:0/133257515 >> v2:172.21.15.148:3300/0 conn(0x7f5b9406c7e0 msgr2=0x7f5b9406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.594+0000 7f5b99214640 1 --2- 172.21.15.105:0/133257515 >> v2:172.21.15.148:3300/0 conn(0x7f5b9406c7e0 0x7f5b9406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.594+0000 7f5b99214640 1 -- 172.21.15.105:0/133257515 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9406de20 con 0x7f5b940cc760 2023-11-28T18:24:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.594+0000 7f5b9a216640 1 --2- 172.21.15.105:0/133257515 >> v2:172.21.15.148:3300/0 conn(0x7f5b9406c7e0 0x7f5b9406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:33.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.598+0000 7f5b99214640 1 --2- 172.21.15.105:0/133257515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b940cc760 0x7f5b940ccbc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5b94072660 tx=0x7f5b8401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e82e47eecc4c350 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:33.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.598+0000 7f5b98a13640 1 -- 172.21.15.105:0/133257515 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b840aa5f0 con 0x7f5b940cc760 2023-11-28T18:24:33.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.598+0000 7f5b98a13640 1 -- 172.21.15.105:0/133257515 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b840aa750 con 0x7f5b940cc760 2023-11-28T18:24:33.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.598+0000 7f5b98a13640 1 -- 172.21.15.105:0/133257515 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b840aaae0 con 0x7f5b940cc760 2023-11-28T18:24:33.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.598+0000 7f5b9bc79640 1 -- 172.21.15.105:0/133257515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b940cc760 msgr2=0x7f5b940ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:33.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.598+0000 7f5b9bc79640 1 --2- 172.21.15.105:0/133257515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b940cc760 0x7f5b940ccbc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5b94072660 tx=0x7f5b8401c4d0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.598+0000 7f5b9bc79640 1 -- 172.21.15.105:0/133257515 shutdown_connections 2023-11-28T18:24:33.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.598+0000 7f5b9bc79640 1 --2- 172.21.15.105:0/133257515 >> v2:172.21.15.148:3300/0 conn(0x7f5b9406c7e0 0x7f5b9406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.598+0000 7f5b9bc79640 1 --2- 172.21.15.105:0/133257515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b940cc760 0x7f5b940ccbc0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.598+0000 7f5b9bc79640 1 --2- 172.21.15.105:0/133257515 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b94159610 0x7f5b940ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.598+0000 7f5b9bc79640 1 -- 172.21.15.105:0/133257515 >> 172.21.15.105:0/133257515 conn(0x7f5b940ceb90 msgr2=0x7f5b940cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:33.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b9bc79640 1 -- 172.21.15.105:0/133257515 shutdown_connections 2023-11-28T18:24:33.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b9bc79640 1 -- 172.21.15.105:0/133257515 wait complete. 2023-11-28T18:24:33.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b9bc79640 1 Processor -- start 2023-11-28T18:24:33.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b9bc79640 1 -- start start 2023-11-28T18:24:33.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b9bc79640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b9406c7e0 0x7f5b9409ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:33.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b9bc79640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b940cc760 0x7f5b94141650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:33.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b9bc79640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b94159610 0x7f5b94150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:33.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b9bc79640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9409c9b0 con 0x7f5b9406c7e0 2023-11-28T18:24:33.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b9bc79640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9409c110 con 0x7f5b940cc760 2023-11-28T18:24:33.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b9bc79640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9409c3f0 con 0x7f5b94159610 2023-11-28T18:24:33.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b99a15640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b9406c7e0 0x7f5b9409ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:33.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b99a15640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b9406c7e0 0x7f5b9409ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33456/0 (socket says 172.21.15.105:33456) 2023-11-28T18:24:33.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b99a15640 1 -- 172.21.15.105:0/4247744059 learned_addr learned my addr 172.21.15.105:0/4247744059 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:33.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b9a216640 1 --2- 172.21.15.105:0/4247744059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b94159610 0x7f5b94150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:33.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b99214640 1 --2- 172.21.15.105:0/4247744059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b940cc760 0x7f5b94141650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:33.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b99a15640 1 -- 172.21.15.105:0/4247744059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b94159610 msgr2=0x7f5b94150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:33.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b99a15640 1 --2- 172.21.15.105:0/4247744059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b94159610 0x7f5b94150190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b99a15640 1 -- 172.21.15.105:0/4247744059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b940cc760 msgr2=0x7f5b94141650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:33.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b99a15640 1 --2- 172.21.15.105:0/4247744059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b940cc760 0x7f5b94141650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b99a15640 1 -- 172.21.15.105:0/4247744059 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b840094d0 con 0x7f5b9406c7e0 2023-11-28T18:24:33.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b99214640 1 --2- 172.21.15.105:0/4247744059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b940cc760 0x7f5b94141650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:33.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b99a15640 1 --2- 172.21.15.105:0/4247744059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b9406c7e0 0x7f5b9409ba80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5b9000e870 tx=0x7f5b9000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:33.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b8a7fc640 1 -- 172.21.15.105:0/4247744059 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90009510 con 0x7f5b9406c7e0 2023-11-28T18:24:33.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b8a7fc640 1 -- 172.21.15.105:0/4247744059 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9000eec0 con 0x7f5b9406c7e0 2023-11-28T18:24:33.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b9bc79640 1 -- 172.21.15.105:0/4247744059 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b9409c6d0 con 0x7f5b9406c7e0 2023-11-28T18:24:33.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.602+0000 7f5b8a7fc640 1 -- 172.21.15.105:0/4247744059 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9000cb40 con 0x7f5b9406c7e0 2023-11-28T18:24:33.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.606+0000 7f5b9bc79640 1 -- 172.21.15.105:0/4247744059 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b94073600 con 0x7f5b9406c7e0 2023-11-28T18:24:33.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.606+0000 7f5b9bc79640 1 -- 172.21.15.105:0/4247744059 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b94151100 con 0x7f5b9406c7e0 2023-11-28T18:24:33.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.606+0000 7f5b8a7fc640 1 -- 172.21.15.105:0/4247744059 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5b9000cca0 con 0x7f5b9406c7e0 2023-11-28T18:24:33.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.606+0000 7f5b8a7fc640 1 --2- 172.21.15.105:0/4247744059 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5b78066900 0x7f5b78068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:33.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.606+0000 7f5b8a7fc640 1 -- 172.21.15.105:0/4247744059 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f5b90084980 con 0x7f5b9406c7e0 2023-11-28T18:24:33.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.606+0000 7f5b99214640 1 --2- 172.21.15.105:0/4247744059 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5b78066900 0x7f5b78068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:33.613 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.606+0000 7f5b99214640 1 --2- 172.21.15.105:0/4247744059 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5b78066900 0x7f5b78068dc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5b840ae000 tx=0x7f5b840b7040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:33.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.610+0000 7f5b8a7fc640 1 -- 172.21.15.105:0/4247744059 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5b94151100 con 0x7f5b9406c7e0 2023-11-28T18:24:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.758+0000 7f5b9bc79640 1 -- 172.21.15.105:0/4247744059 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5b94151100 con 0x7f5b9406c7e0 2023-11-28T18:24:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.762+0000 7f5b8a7fc640 1 -- 172.21.15.105:0/4247744059 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5b94151100 con 0x7f5b9406c7e0 2023-11-28T18:24:33.766 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.762+0000 7f5b9bc79640 1 -- 172.21.15.105:0/4247744059 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5b78066900 msgr2=0x7f5b78068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.762+0000 7f5b9bc79640 1 --2- 172.21.15.105:0/4247744059 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5b78066900 0x7f5b78068dc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5b840ae000 tx=0x7f5b840b7040 comp rx=0 tx=0).stop 2023-11-28T18:24:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.762+0000 7f5b9bc79640 1 -- 172.21.15.105:0/4247744059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b9406c7e0 msgr2=0x7f5b9409ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.762+0000 7f5b9bc79640 1 --2- 172.21.15.105:0/4247744059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b9406c7e0 0x7f5b9409ba80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5b9000e870 tx=0x7f5b9000ec70 comp rx=0 tx=0).stop 2023-11-28T18:24:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.762+0000 7f5b9bc79640 1 -- 172.21.15.105:0/4247744059 shutdown_connections 2023-11-28T18:24:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.762+0000 7f5b9bc79640 1 --2- 172.21.15.105:0/4247744059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b94159610 0x7f5b94150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.762+0000 7f5b9bc79640 1 --2- 172.21.15.105:0/4247744059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b940cc760 0x7f5b94141650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.762+0000 7f5b9bc79640 1 --2- 172.21.15.105:0/4247744059 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5b78066900 0x7f5b78068dc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.762+0000 7f5b9bc79640 1 --2- 172.21.15.105:0/4247744059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b9406c7e0 0x7f5b9409ba80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.762+0000 7f5b9bc79640 1 -- 172.21.15.105:0/4247744059 >> 172.21.15.105:0/4247744059 conn(0x7f5b940ceb90 msgr2=0x7f5b94153550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.762+0000 7f5b9bc79640 1 -- 172.21.15.105:0/4247744059 shutdown_connections 2023-11-28T18:24:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.762+0000 7f5b9bc79640 1 -- 172.21.15.105:0/4247744059 wait complete. 2023-11-28T18:24:33.786 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T18:24:33.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b826b640 1 Processor -- start 2023-11-28T18:24:33.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b826b640 1 -- start start 2023-11-28T18:24:33.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b826b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30b0071a10 0x7f30b00ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:33.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b826b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f30b0070a10 0x7f30b0070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:33.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b826b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30b00cefa0 con 0x7f30b015d3c0 2023-11-28T18:24:33.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b826b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b00713c0 con 0x7f30b0070a10 2023-11-28T18:24:33.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b826b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b015d7a0 con 0x7f30b0071a10 2023-11-28T18:24:33.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b6007640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f30b015d3c0 0x7f30b0161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35352/0 (socket says 172.21.15.105:35352) 2023-11-28T18:24:33.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b6007640 1 -- 172.21.15.105:0/764090235 learned_addr learned my addr 172.21.15.105:0/764090235 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:33.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b5005640 1 -- 172.21.15.105:0/764090235 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3126297191 0 0) 0x7f30b00cefa0 con 0x7f30b015d3c0 2023-11-28T18:24:33.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b5806640 1 --2- 172.21.15.105:0/764090235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30b0071a10 0x7f30b00ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:33.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b5005640 1 -- 172.21.15.105:0/764090235 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f309c003530 con 0x7f30b015d3c0 2023-11-28T18:24:33.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b6808640 1 --2- 172.21.15.105:0/764090235 >> v2:172.21.15.148:3300/0 conn(0x7f30b0070a10 0x7f30b0070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b5005640 1 -- 172.21.15.105:0/764090235 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 58674856 0 0) 0x7f309c003530 con 0x7f30b015d3c0 2023-11-28T18:24:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b5005640 1 -- 172.21.15.105:0/764090235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30b0071a10 msgr2=0x7f30b00ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b5005640 1 --2- 172.21.15.105:0/764090235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30b0071a10 0x7f30b00ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b5005640 1 -- 172.21.15.105:0/764090235 >> v2:172.21.15.148:3300/0 conn(0x7f30b0070a10 msgr2=0x7f30b0070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b5005640 1 --2- 172.21.15.105:0/764090235 >> v2:172.21.15.148:3300/0 conn(0x7f30b0070a10 0x7f30b0070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b5005640 1 -- 172.21.15.105:0/764090235 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b00ce9e0 con 0x7f30b015d3c0 2023-11-28T18:24:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b5005640 1 -- 172.21.15.105:0/764090235 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f30ac001680 con 0x7f30b015d3c0 2023-11-28T18:24:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b5005640 1 -- 172.21.15.105:0/764090235 >> v1:172.21.15.105:6789/0 conn(0x7f30b015d3c0 legacy=0x7f30b0161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b5005640 1 --2- 172.21.15.105:0/764090235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f309c003980 0x7f309c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b5005640 1 --2- 172.21.15.105:0/764090235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30b0071a10 0x7f309c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b5005640 1 --2- 172.21.15.105:0/764090235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f309c004d10 0x7f309c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b5005640 1 -- 172.21.15.105:0/764090235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b00713c0 con 0x7f309c003980 2023-11-28T18:24:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b5005640 1 -- 172.21.15.105:0/764090235 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b015d7a0 con 0x7f309c004d10 2023-11-28T18:24:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b5005640 1 -- 172.21.15.105:0/764090235 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309c00b9c0 con 0x7f30b0071a10 2023-11-28T18:24:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b6007640 1 --2- 172.21.15.105:0/764090235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f309c003980 0x7f309c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b5806640 1 --2- 172.21.15.105:0/764090235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30b0071a10 0x7f309c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b6808640 1 --2- 172.21.15.105:0/764090235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f309c004d10 0x7f309c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b6007640 1 -- 172.21.15.105:0/764090235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30b0071a10 msgr2=0x7f309c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b6007640 1 --2- 172.21.15.105:0/764090235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30b0071a10 0x7f309c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b6007640 1 -- 172.21.15.105:0/764090235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f309c004d10 msgr2=0x7f309c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b6007640 1 --2- 172.21.15.105:0/764090235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f309c004d10 0x7f309c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b6007640 1 -- 172.21.15.105:0/764090235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c003530 con 0x7f309c003980 2023-11-28T18:24:33.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.922+0000 7f30b6808640 1 --2- 172.21.15.105:0/764090235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f309c004d10 0x7f309c005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:33.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.926+0000 7f30b6007640 1 --2- 172.21.15.105:0/764090235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f309c003980 0x7f309c003da0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00c240 tx=0x7f30ac01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8205aa947bef9d57 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:33.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.926+0000 7f30b5005640 1 -- 172.21.15.105:0/764090235 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0058e0 con 0x7f309c003980 2023-11-28T18:24:33.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.926+0000 7f30b5005640 1 -- 172.21.15.105:0/764090235 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ac005a40 con 0x7f309c003980 2023-11-28T18:24:33.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.926+0000 7f30b5005640 1 -- 172.21.15.105:0/764090235 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac005e10 con 0x7f309c003980 2023-11-28T18:24:33.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.926+0000 7f30b826b640 1 -- 172.21.15.105:0/764090235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f309c003980 msgr2=0x7f309c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:33.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.926+0000 7f30b826b640 1 --2- 172.21.15.105:0/764090235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f309c003980 0x7f309c003da0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00c240 tx=0x7f30ac01cc30 comp rx=0 tx=0).stop 2023-11-28T18:24:33.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.926+0000 7f30b6808640 1 -- 172.21.15.105:0/764090235 reap_dead start 2023-11-28T18:24:33.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.926+0000 7f30b826b640 1 -- 172.21.15.105:0/764090235 shutdown_connections 2023-11-28T18:24:33.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.926+0000 7f30b826b640 1 -- 172.21.15.105:0/764090235 >> 172.21.15.105:0/764090235 conn(0x7f30b00cf540 msgr2=0x7f30b00cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:33.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.926+0000 7f30b826b640 1 -- 172.21.15.105:0/764090235 shutdown_connections 2023-11-28T18:24:33.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.926+0000 7f30b826b640 1 -- 172.21.15.105:0/764090235 wait complete. 2023-11-28T18:24:33.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30b826b640 1 Processor -- start 2023-11-28T18:24:33.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30b826b640 1 -- start start 2023-11-28T18:24:33.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30b826b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f309c003980 0x7f30b0148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:33.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30b826b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30b015ec90 0x7f30b00ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:33.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30b826b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30b00aeee0 0x7f30b01f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:33.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30b826b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b00af390 con 0x7f309c003980 2023-11-28T18:24:33.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30b826b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b0160120 con 0x7f30b00aeee0 2023-11-28T18:24:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30b826b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b015f910 con 0x7f30b015ec90 2023-11-28T18:24:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30b6007640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f309c003980 0x7f30b0148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30b6007640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f309c003980 0x7f30b0148f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33468/0 (socket says 172.21.15.105:33468) 2023-11-28T18:24:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30b6007640 1 -- 172.21.15.105:0/4158468284 learned_addr learned my addr 172.21.15.105:0/4158468284 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30b5806640 1 --2- 172.21.15.105:0/4158468284 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30b015ec90 0x7f30b00ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30b6808640 1 --2- 172.21.15.105:0/4158468284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30b00aeee0 0x7f30b01f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30b6007640 1 -- 172.21.15.105:0/4158468284 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30b015ec90 msgr2=0x7f30b00ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30b6007640 1 --2- 172.21.15.105:0/4158468284 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30b015ec90 0x7f30b00ae9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30b6007640 1 -- 172.21.15.105:0/4158468284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30b00aeee0 msgr2=0x7f30b01f6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30b6007640 1 --2- 172.21.15.105:0/4158468284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30b00aeee0 0x7f30b01f6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30b6007640 1 -- 172.21.15.105:0/4158468284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b01494c0 con 0x7f309c003980 2023-11-28T18:24:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30b6007640 1 --2- 172.21.15.105:0/4158468284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f309c003980 0x7f30b0148f80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00c240 tx=0x7f309c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:33.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30a6ffd640 1 -- 172.21.15.105:0/4158468284 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0a5820 con 0x7f309c003980 2023-11-28T18:24:33.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30a6ffd640 1 -- 172.21.15.105:0/4158468284 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ac0a5980 con 0x7f309c003980 2023-11-28T18:24:33.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30b826b640 1 -- 172.21.15.105:0/4158468284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b015fb60 con 0x7f309c003980 2023-11-28T18:24:33.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30a6ffd640 1 -- 172.21.15.105:0/4158468284 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0a5cf0 con 0x7f309c003980 2023-11-28T18:24:33.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.930+0000 7f30b826b640 1 -- 172.21.15.105:0/4158468284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b015e490 con 0x7f309c003980 2023-11-28T18:24:33.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.934+0000 7f30b826b640 1 -- 172.21.15.105:0/4158468284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b015e2f0 con 0x7f309c003980 2023-11-28T18:24:33.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.934+0000 7f30a6ffd640 1 -- 172.21.15.105:0/4158468284 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f30ac0047f0 con 0x7f309c003980 2023-11-28T18:24:33.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.934+0000 7f30a6ffd640 1 --2- 172.21.15.105:0/4158468284 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3094066900 0x7f3094068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:33.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.934+0000 7f30a6ffd640 1 -- 172.21.15.105:0/4158468284 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f30ac11e010 con 0x7f309c003980 2023-11-28T18:24:33.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.934+0000 7f30b5806640 1 --2- 172.21.15.105:0/4158468284 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3094066900 0x7f3094068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:33.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.934+0000 7f30b5806640 1 --2- 172.21.15.105:0/4158468284 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3094066900 0x7f3094068dc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f30b0161d30 tx=0x7f30a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:33.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:33.938+0000 7f30a6ffd640 1 -- 172.21.15.105:0/4158468284 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f30b015e2f0 con 0x7f309c003980 2023-11-28T18:24:34.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.102+0000 7f30b826b640 1 -- 172.21.15.105:0/4158468284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f30b015e2f0 con 0x7f309c003980 2023-11-28T18:24:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.106+0000 7f30a6ffd640 1 -- 172.21.15.105:0/4158468284 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f30b015e2f0 con 0x7f309c003980 2023-11-28T18:24:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2023-11-28T18:24:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.106+0000 7f30b826b640 1 -- 172.21.15.105:0/4158468284 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3094066900 msgr2=0x7f3094068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.106+0000 7f30b826b640 1 --2- 172.21.15.105:0/4158468284 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3094066900 0x7f3094068dc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f30b0161d30 tx=0x7f30a0008040 comp rx=0 tx=0).stop 2023-11-28T18:24:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.106+0000 7f30b826b640 1 -- 172.21.15.105:0/4158468284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f309c003980 msgr2=0x7f30b0148f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.106+0000 7f30b826b640 1 --2- 172.21.15.105:0/4158468284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f309c003980 0x7f30b0148f80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00c240 tx=0x7f309c005800 comp rx=0 tx=0).stop 2023-11-28T18:24:34.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.106+0000 7f30b826b640 1 -- 172.21.15.105:0/4158468284 shutdown_connections 2023-11-28T18:24:34.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.106+0000 7f30b826b640 1 --2- 172.21.15.105:0/4158468284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30b00aeee0 0x7f30b01f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.106+0000 7f30b826b640 1 --2- 172.21.15.105:0/4158468284 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30b015ec90 0x7f30b00ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.106+0000 7f30b826b640 1 --2- 172.21.15.105:0/4158468284 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3094066900 0x7f3094068dc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.106+0000 7f30b826b640 1 --2- 172.21.15.105:0/4158468284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f309c003980 0x7f30b0148f80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.106+0000 7f30b826b640 1 -- 172.21.15.105:0/4158468284 >> 172.21.15.105:0/4158468284 conn(0x7f30b00cf540 msgr2=0x7f30b00cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:34.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.106+0000 7f30b826b640 1 -- 172.21.15.105:0/4158468284 shutdown_connections 2023-11-28T18:24:34.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.106+0000 7f30b826b640 1 -- 172.21.15.105:0/4158468284 wait complete. 2023-11-28T18:24:34.131 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:24:34.131 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:24:34.141 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:24:34.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.318+0000 7fccafa79640 1 Processor -- start 2023-11-28T18:24:34.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.318+0000 7fccafa79640 1 -- start start 2023-11-28T18:24:34.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.318+0000 7fccafa79640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcca80caf60 0x7fcca80cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:34.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.318+0000 7fccafa79640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcca80ad850 0x7fcca80701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:34.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.318+0000 7fccafa79640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcca806bf50 con 0x7fcca8159f40 2023-11-28T18:24:34.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.318+0000 7fccafa79640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca80cb3c0 con 0x7fcca80ad850 2023-11-28T18:24:34.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.318+0000 7fccafa79640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca815a320 con 0x7fcca80caf60 2023-11-28T18:24:34.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.318+0000 7fccad815640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fcca8159f40 0x7fcca80ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35360/0 (socket says 172.21.15.105:35360) 2023-11-28T18:24:34.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.318+0000 7fccad815640 1 -- 172.21.15.105:0/2632799289 learned_addr learned my addr 172.21.15.105:0/2632799289 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:34.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.318+0000 7fccae016640 1 --2- 172.21.15.105:0/2632799289 >> v2:172.21.15.105:3300/0 conn(0x7fcca80ad850 0x7fcca80701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:34.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.318+0000 7fccad014640 1 --2- 172.21.15.105:0/2632799289 >> v2:172.21.15.148:3300/0 conn(0x7fcca80caf60 0x7fcca80cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:34.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.318+0000 7fccae016640 1 -- 172.21.15.105:0/2632799289 >> v2:172.21.15.148:3300/0 conn(0x7fcca80caf60 msgr2=0x7fcca80cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:34.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.318+0000 7fccae016640 1 --2- 172.21.15.105:0/2632799289 >> v2:172.21.15.148:3300/0 conn(0x7fcca80caf60 0x7fcca80cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.318+0000 7fccae016640 1 -- 172.21.15.105:0/2632799289 >> v1:172.21.15.105:6789/0 conn(0x7fcca8159f40 legacy=0x7fcca80ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:34.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.318+0000 7fccae016640 1 -- 172.21.15.105:0/2632799289 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca8071380 con 0x7fcca80ad850 2023-11-28T18:24:34.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.322+0000 7fccae016640 1 --2- 172.21.15.105:0/2632799289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca80ad850 0x7fcca80701d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcca809d0d0 tx=0x7fcca001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170c154b5864239e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:34.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.322+0000 7fccac813640 1 -- 172.21.15.105:0/2632799289 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca00a5910 con 0x7fcca80ad850 2023-11-28T18:24:34.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.322+0000 7fccac813640 1 -- 172.21.15.105:0/2632799289 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca00a5a70 con 0x7fcca80ad850 2023-11-28T18:24:34.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.322+0000 7fccac813640 1 -- 172.21.15.105:0/2632799289 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca00a5e00 con 0x7fcca80ad850 2023-11-28T18:24:34.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.322+0000 7fccafa79640 1 -- 172.21.15.105:0/2632799289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca80ad850 msgr2=0x7fcca80701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:34.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.322+0000 7fccafa79640 1 --2- 172.21.15.105:0/2632799289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca80ad850 0x7fcca80701d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcca809d0d0 tx=0x7fcca001c850 comp rx=0 tx=0).stop 2023-11-28T18:24:34.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.322+0000 7fccafa79640 1 -- 172.21.15.105:0/2632799289 shutdown_connections 2023-11-28T18:24:34.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.322+0000 7fccafa79640 1 --2- 172.21.15.105:0/2632799289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca80ad850 0x7fcca80701d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.322+0000 7fccafa79640 1 --2- 172.21.15.105:0/2632799289 >> v2:172.21.15.148:3300/0 conn(0x7fcca80caf60 0x7fcca80cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.322+0000 7fccafa79640 1 -- 172.21.15.105:0/2632799289 >> 172.21.15.105:0/2632799289 conn(0x7fcca80ceb90 msgr2=0x7fcca80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:34.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.322+0000 7fccafa79640 1 -- 172.21.15.105:0/2632799289 shutdown_connections 2023-11-28T18:24:34.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.322+0000 7fccafa79640 1 -- 172.21.15.105:0/2632799289 wait complete. 2023-11-28T18:24:34.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fccafa79640 1 Processor -- start 2023-11-28T18:24:34.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fccafa79640 1 -- start start 2023-11-28T18:24:34.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fccafa79640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca80ad850 0x7fcca815b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:34.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fccafa79640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca80caf60 0x7fcca8157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:34.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fccafa79640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcca8159f40 0x7fcca8153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:34.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fccafa79640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca815b5e0 con 0x7fcca80caf60 2023-11-28T18:24:34.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fccafa79640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca8158930 con 0x7fcca80ad850 2023-11-28T18:24:34.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fccafa79640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca8157a80 con 0x7fcca8159f40 2023-11-28T18:24:34.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fccad014640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca80caf60 0x7fcca8157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:34.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fccad014640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca80caf60 0x7fcca8157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33488/0 (socket says 172.21.15.105:33488) 2023-11-28T18:24:34.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fccad014640 1 -- 172.21.15.105:0/564684299 learned_addr learned my addr 172.21.15.105:0/564684299 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:34.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fccae016640 1 --2- 172.21.15.105:0/564684299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcca8159f40 0x7fcca8153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:34.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fccad815640 1 --2- 172.21.15.105:0/564684299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca80ad850 0x7fcca815b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:34.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fccad014640 1 -- 172.21.15.105:0/564684299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcca8159f40 msgr2=0x7fcca8153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:34.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fccad014640 1 --2- 172.21.15.105:0/564684299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcca8159f40 0x7fcca8153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fccad014640 1 -- 172.21.15.105:0/564684299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca80ad850 msgr2=0x7fcca815b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:34.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fccad014640 1 --2- 172.21.15.105:0/564684299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca80ad850 0x7fcca815b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fccad014640 1 -- 172.21.15.105:0/564684299 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca0009a70 con 0x7fcca80caf60 2023-11-28T18:24:34.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fccad014640 1 --2- 172.21.15.105:0/564684299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca80caf60 0x7fcca8157540 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800d5f0 tx=0x7fcc9800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:34.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fcc9e7fc640 1 -- 172.21.15.105:0/564684299 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc980125f0 con 0x7fcca80caf60 2023-11-28T18:24:34.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fcc9e7fc640 1 -- 172.21.15.105:0/564684299 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc98012750 con 0x7fcca80caf60 2023-11-28T18:24:34.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fcc9e7fc640 1 -- 172.21.15.105:0/564684299 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc98012ae0 con 0x7fcca80caf60 2023-11-28T18:24:34.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fccafa79640 1 -- 172.21.15.105:0/564684299 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca8157d20 con 0x7fcca80caf60 2023-11-28T18:24:34.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fccafa79640 1 -- 172.21.15.105:0/564684299 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca81582e0 con 0x7fcca80caf60 2023-11-28T18:24:34.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.326+0000 7fccafa79640 1 -- 172.21.15.105:0/564684299 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca8158ac0 con 0x7fcca80caf60 2023-11-28T18:24:34.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.330+0000 7fcc9e7fc640 1 -- 172.21.15.105:0/564684299 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fcc98012c40 con 0x7fcca80caf60 2023-11-28T18:24:34.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.330+0000 7fcc9e7fc640 1 --2- 172.21.15.105:0/564684299 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcc8c0668d0 0x7fcc8c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:34.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.330+0000 7fcc9e7fc640 1 -- 172.21.15.105:0/564684299 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fcc98084b80 con 0x7fcca80caf60 2023-11-28T18:24:34.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.330+0000 7fccad815640 1 --2- 172.21.15.105:0/564684299 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcc8c0668d0 0x7fcc8c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:34.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.330+0000 7fccad815640 1 --2- 172.21.15.105:0/564684299 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcc8c0668d0 0x7fcc8c068d90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcca4005890 tx=0x7fcca4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:34.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.334+0000 7fcc9e7fc640 1 -- 172.21.15.105:0/564684299 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcc98051a70 con 0x7fcca80caf60 2023-11-28T18:24:34.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.486+0000 7fccafa79640 1 -- 172.21.15.105:0/564684299 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcca8067570 con 0x7fcca80caf60 2023-11-28T18:24:34.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.486+0000 7fcc9e7fc640 1 -- 172.21.15.105:0/564684299 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18887 (secure 0 0 0) 0x7fcc98007380 con 0x7fcca80caf60 2023-11-28T18:24:34.493 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:34.493 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:24:15.048168+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:24:34.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.490+0000 7fccafa79640 1 -- 172.21.15.105:0/564684299 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcc8c0668d0 msgr2=0x7fcc8c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:34.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.490+0000 7fccafa79640 1 --2- 172.21.15.105:0/564684299 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcc8c0668d0 0x7fcc8c068d90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcca4005890 tx=0x7fcca4009040 comp rx=0 tx=0).stop 2023-11-28T18:24:34.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.490+0000 7fccafa79640 1 -- 172.21.15.105:0/564684299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca80caf60 msgr2=0x7fcca8157540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:34.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.490+0000 7fccafa79640 1 --2- 172.21.15.105:0/564684299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca80caf60 0x7fcca8157540 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800d5f0 tx=0x7fcc9800dab0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.490+0000 7fccafa79640 1 -- 172.21.15.105:0/564684299 shutdown_connections 2023-11-28T18:24:34.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.490+0000 7fccafa79640 1 --2- 172.21.15.105:0/564684299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcca8159f40 0x7fcca8153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.490+0000 7fccafa79640 1 --2- 172.21.15.105:0/564684299 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcc8c0668d0 0x7fcc8c068d90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.490+0000 7fccafa79640 1 --2- 172.21.15.105:0/564684299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca80caf60 0x7fcca8157540 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.490+0000 7fccafa79640 1 --2- 172.21.15.105:0/564684299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca80ad850 0x7fcca815b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.490+0000 7fccafa79640 1 -- 172.21.15.105:0/564684299 >> 172.21.15.105:0/564684299 conn(0x7fcca80ceb90 msgr2=0x7fcca806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:34.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.490+0000 7fccafa79640 1 -- 172.21.15.105:0/564684299 shutdown_connections 2023-11-28T18:24:34.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.490+0000 7fccafa79640 1 -- 172.21.15.105:0/564684299 wait complete. 2023-11-28T18:24:34.515 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:23:31.485422+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:23:32.274378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T18:23:36.100071+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T18:24:34.515 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:24:34.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.650+0000 7fe56cd9d640 1 Processor -- start 2023-11-28T18:24:34.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.650+0000 7fe56cd9d640 1 -- start start 2023-11-28T18:24:34.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.650+0000 7fe56cd9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe56815b030 0x7fe5680709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:34.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.650+0000 7fe56cd9d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5680701d0 con 0x7fe568071960 2023-11-28T18:24:34.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.650+0000 7fe56cd9d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe56806df80 con 0x7fe5680b4290 2023-11-28T18:24:34.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.650+0000 7fe56cd9d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe568071df0 con 0x7fe56815b030 2023-11-28T18:24:34.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.650+0000 7fe55dd9b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe568071960 0x7fe56815f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35364/0 (socket says 172.21.15.105:35364) 2023-11-28T18:24:34.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.650+0000 7fe55dd9b640 1 -- 172.21.15.105:0/1850022825 learned_addr learned my addr 172.21.15.105:0/1850022825 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:34.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.654+0000 7fe566d9d640 1 --2- 172.21.15.105:0/1850022825 >> v2:172.21.15.148:3300/0 conn(0x7fe56815b030 0x7fe5680709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:34.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.654+0000 7fe565d9b640 1 -- 172.21.15.105:0/1850022825 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3167263483 0 0) 0x7fe5680701d0 con 0x7fe568071960 2023-11-28T18:24:34.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.654+0000 7fe565d9b640 1 -- 172.21.15.105:0/1850022825 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe554003530 con 0x7fe568071960 2023-11-28T18:24:34.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.654+0000 7fe565d9b640 1 -- 172.21.15.105:0/1850022825 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 794185441 0 0) 0x7fe56806df80 con 0x7fe5680b4290 2023-11-28T18:24:34.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.654+0000 7fe565d9b640 1 -- 172.21.15.105:0/1850022825 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5680701d0 con 0x7fe5680b4290 2023-11-28T18:24:34.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.654+0000 7fe566d9d640 1 -- 172.21.15.105:0/1850022825 >> v1:172.21.15.148:6789/0 conn(0x7fe5680b4290 legacy=0x7fe5680b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:34.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.654+0000 7fe566d9d640 1 -- 172.21.15.105:0/1850022825 >> v1:172.21.15.105:6789/0 conn(0x7fe568071960 legacy=0x7fe56815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:34.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.654+0000 7fe566d9d640 1 -- 172.21.15.105:0/1850022825 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5680720d0 con 0x7fe56815b030 2023-11-28T18:24:34.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.654+0000 7fe566d9d640 1 --2- 172.21.15.105:0/1850022825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe56815b030 0x7fe5680709b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe5600097c0 tx=0x7fe56001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ef48ac815b8b00 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:34.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.654+0000 7fe565d9b640 1 -- 172.21.15.105:0/1850022825 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5600ad660 con 0x7fe56815b030 2023-11-28T18:24:34.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.654+0000 7fe565d9b640 1 -- 172.21.15.105:0/1850022825 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5600ad7c0 con 0x7fe56815b030 2023-11-28T18:24:34.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.654+0000 7fe565d9b640 1 -- 172.21.15.105:0/1850022825 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5600adb50 con 0x7fe56815b030 2023-11-28T18:24:34.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.658+0000 7fe56cd9d640 1 -- 172.21.15.105:0/1850022825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe56815b030 msgr2=0x7fe5680709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:34.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.658+0000 7fe56cd9d640 1 --2- 172.21.15.105:0/1850022825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe56815b030 0x7fe5680709b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe5600097c0 tx=0x7fe56001c450 comp rx=0 tx=0).stop 2023-11-28T18:24:34.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.658+0000 7fe56cd9d640 1 -- 172.21.15.105:0/1850022825 shutdown_connections 2023-11-28T18:24:34.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.658+0000 7fe56cd9d640 1 --2- 172.21.15.105:0/1850022825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe56815b030 0x7fe5680709b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.658+0000 7fe56cd9d640 1 -- 172.21.15.105:0/1850022825 >> 172.21.15.105:0/1850022825 conn(0x7fe5680ceb90 msgr2=0x7fe5680cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:34.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.658+0000 7fe56cd9d640 1 -- 172.21.15.105:0/1850022825 shutdown_connections 2023-11-28T18:24:34.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.658+0000 7fe56cd9d640 1 -- 172.21.15.105:0/1850022825 wait complete. 2023-11-28T18:24:34.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.658+0000 7fe56cd9d640 1 Processor -- start 2023-11-28T18:24:34.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.658+0000 7fe56cd9d640 1 -- start start 2023-11-28T18:24:34.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.658+0000 7fe56cd9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5680b4290 0x7fe5681547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:34.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.658+0000 7fe56cd9d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe56815b030 0x7fe56815b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:34.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.658+0000 7fe56cd9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5680b19b0 0x7fe568156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:34.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.658+0000 7fe56cd9d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe568154d30 con 0x7fe5680b4290 2023-11-28T18:24:34.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.658+0000 7fe56cd9d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5680b3010 con 0x7fe5680b19b0 2023-11-28T18:24:34.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.658+0000 7fe56cd9d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe56815b9f0 con 0x7fe56815b030 2023-11-28T18:24:34.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.658+0000 7fe56659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5680b4290 0x7fe5681547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:34.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.658+0000 7fe56659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5680b4290 0x7fe5681547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33492/0 (socket says 172.21.15.105:33492) 2023-11-28T18:24:34.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.658+0000 7fe56659c640 1 -- 172.21.15.105:0/2617049509 learned_addr learned my addr 172.21.15.105:0/2617049509 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:34.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.662+0000 7fe55dd9b640 1 --2- 172.21.15.105:0/2617049509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe56815b030 0x7fe56815b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:34.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.662+0000 7fe566d9d640 1 --2- 172.21.15.105:0/2617049509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5680b19b0 0x7fe568156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:34.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.662+0000 7fe56659c640 1 -- 172.21.15.105:0/2617049509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe56815b030 msgr2=0x7fe56815b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:34.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.662+0000 7fe56659c640 1 --2- 172.21.15.105:0/2617049509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe56815b030 0x7fe56815b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.662+0000 7fe56659c640 1 -- 172.21.15.105:0/2617049509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5680b19b0 msgr2=0x7fe568156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:34.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.662+0000 7fe56659c640 1 --2- 172.21.15.105:0/2617049509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5680b19b0 0x7fe568156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.662+0000 7fe56659c640 1 -- 172.21.15.105:0/2617049509 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe560009420 con 0x7fe5680b4290 2023-11-28T18:24:34.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.662+0000 7fe56659c640 1 --2- 172.21.15.105:0/2617049509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5680b4290 0x7fe5681547f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe55800c800 tx=0x7fe55800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:34.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.662+0000 7fe55f7fe640 1 -- 172.21.15.105:0/2617049509 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe558003b80 con 0x7fe5680b4290 2023-11-28T18:24:34.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.662+0000 7fe55f7fe640 1 -- 172.21.15.105:0/2617049509 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe558003d00 con 0x7fe5680b4290 2023-11-28T18:24:34.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.662+0000 7fe55f7fe640 1 -- 172.21.15.105:0/2617049509 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55800aa90 con 0x7fe5680b4290 2023-11-28T18:24:34.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.662+0000 7fe56cd9d640 1 -- 172.21.15.105:0/2617049509 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5680b2400 con 0x7fe5680b4290 2023-11-28T18:24:34.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.662+0000 7fe56cd9d640 1 -- 172.21.15.105:0/2617049509 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5680b29c0 con 0x7fe5680b4290 2023-11-28T18:24:34.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.662+0000 7fe56cd9d640 1 -- 172.21.15.105:0/2617049509 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe568067570 con 0x7fe5680b4290 2023-11-28T18:24:34.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.662+0000 7fe55f7fe640 1 -- 172.21.15.105:0/2617049509 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fe55800abf0 con 0x7fe5680b4290 2023-11-28T18:24:34.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.666+0000 7fe55f7fe640 1 --2- 172.21.15.105:0/2617049509 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe548066950 0x7fe548068e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:34.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.666+0000 7fe55f7fe640 1 -- 172.21.15.105:0/2617049509 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fe558084720 con 0x7fe5680b4290 2023-11-28T18:24:34.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.666+0000 7fe55dd9b640 1 --2- 172.21.15.105:0/2617049509 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe548066950 0x7fe548068e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:34.670 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.666+0000 7fe55dd9b640 1 --2- 172.21.15.105:0/2617049509 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe548066950 0x7fe548068e10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe56815f560 tx=0x7fe550009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:34.673 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.666+0000 7fe55f7fe640 1 -- 172.21.15.105:0/2617049509 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe558051560 con 0x7fe5680b4290 2023-11-28T18:24:34.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.818+0000 7fe56cd9d640 1 -- 172.21.15.105:0/2617049509 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5680cf7f0 con 0x7fe5680b4290 2023-11-28T18:24:34.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.818+0000 7fe55f7fe640 1 -- 172.21.15.105:0/2617049509 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe558054b80 con 0x7fe5680b4290 2023-11-28T18:24:34.825 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:24:34.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.822+0000 7fe56cd9d640 1 -- 172.21.15.105:0/2617049509 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe548066950 msgr2=0x7fe548068e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:34.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.822+0000 7fe56cd9d640 1 --2- 172.21.15.105:0/2617049509 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe548066950 0x7fe548068e10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe56815f560 tx=0x7fe550009040 comp rx=0 tx=0).stop 2023-11-28T18:24:34.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.822+0000 7fe56cd9d640 1 -- 172.21.15.105:0/2617049509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5680b4290 msgr2=0x7fe5681547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:34.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.822+0000 7fe56cd9d640 1 --2- 172.21.15.105:0/2617049509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5680b4290 0x7fe5681547f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe55800c800 tx=0x7fe55800ccc0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.822+0000 7fe56cd9d640 1 -- 172.21.15.105:0/2617049509 shutdown_connections 2023-11-28T18:24:34.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.822+0000 7fe56cd9d640 1 --2- 172.21.15.105:0/2617049509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5680b19b0 0x7fe568156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.822+0000 7fe56cd9d640 1 --2- 172.21.15.105:0/2617049509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe56815b030 0x7fe56815b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.822+0000 7fe56cd9d640 1 --2- 172.21.15.105:0/2617049509 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe548066950 0x7fe548068e10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.822+0000 7fe56cd9d640 1 --2- 172.21.15.105:0/2617049509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5680b4290 0x7fe5681547f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.822+0000 7fe56cd9d640 1 -- 172.21.15.105:0/2617049509 >> 172.21.15.105:0/2617049509 conn(0x7fe5680ceb90 msgr2=0x7fe5680cd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:34.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.822+0000 7fe56cd9d640 1 -- 172.21.15.105:0/2617049509 shutdown_connections 2023-11-28T18:24:34.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.822+0000 7fe56cd9d640 1 -- 172.21.15.105:0/2617049509 wait complete. 2023-11-28T18:24:34.845 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:24:34.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.982+0000 7f26ff9b8640 1 Processor -- start 2023-11-28T18:24:34.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.982+0000 7f26ff9b8640 1 -- start start 2023-11-28T18:24:34.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.982+0000 7f26ff9b8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f26f80b4290 0x7f26f80b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:34.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.982+0000 7f26ff9b8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f26f8071960 0x7f26f815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:34.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.982+0000 7f26ff9b8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f80b4bb0 con 0x7f26f815b030 2023-11-28T18:24:34.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.982+0000 7f26ff9b8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f80701d0 con 0x7f26f80b4290 2023-11-28T18:24:34.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.982+0000 7f26ff9b8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f806df80 con 0x7f26f8071960 2023-11-28T18:24:34.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.982+0000 7f26fd754640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f26f80b4290 0x7f26f80b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:34.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.982+0000 7f26fd754640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f26f80b4290 0x7f26f80b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33500/0 (socket says 172.21.15.105:33500) 2023-11-28T18:24:34.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.982+0000 7f26fd754640 1 -- 172.21.15.105:0/3520177616 learned_addr learned my addr 172.21.15.105:0/3520177616 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:34.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.982+0000 7f26fcf53640 1 --2- 172.21.15.105:0/3520177616 >> v2:172.21.15.148:3300/0 conn(0x7f26f8071960 0x7f26f815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:34.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.982+0000 7f26fd754640 1 -- 172.21.15.105:0/3520177616 >> v2:172.21.15.148:3300/0 conn(0x7f26f8071960 msgr2=0x7f26f815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:34.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.982+0000 7f26fd754640 1 --2- 172.21.15.105:0/3520177616 >> v2:172.21.15.148:3300/0 conn(0x7f26f8071960 0x7f26f815f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.982+0000 7f26fd754640 1 -- 172.21.15.105:0/3520177616 >> v1:172.21.15.105:6789/0 conn(0x7f26f815b030 legacy=0x7f26f80709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:34.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.982+0000 7f26fd754640 1 -- 172.21.15.105:0/3520177616 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f8071df0 con 0x7f26f80b4290 2023-11-28T18:24:34.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.986+0000 7f26fd754640 1 --2- 172.21.15.105:0/3520177616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f80b4290 0x7f26f80b4670 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f26f806e520 tx=0x7f26f401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6307d94b2477de81 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:34.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.986+0000 7f26effff640 1 -- 172.21.15.105:0/3520177616 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f40aa430 con 0x7f26f80b4290 2023-11-28T18:24:34.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.986+0000 7f26effff640 1 -- 172.21.15.105:0/3520177616 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f40aa590 con 0x7f26f80b4290 2023-11-28T18:24:34.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.986+0000 7f26effff640 1 -- 172.21.15.105:0/3520177616 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f40aa920 con 0x7f26f80b4290 2023-11-28T18:24:34.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.986+0000 7f26ff9b8640 1 -- 172.21.15.105:0/3520177616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f80b4290 msgr2=0x7f26f80b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:34.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.986+0000 7f26ff9b8640 1 --2- 172.21.15.105:0/3520177616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f80b4290 0x7f26f80b4670 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f26f806e520 tx=0x7f26f401c4d0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.986+0000 7f26ff9b8640 1 -- 172.21.15.105:0/3520177616 shutdown_connections 2023-11-28T18:24:34.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.986+0000 7f26ff9b8640 1 --2- 172.21.15.105:0/3520177616 >> v2:172.21.15.148:3300/0 conn(0x7f26f8071960 0x7f26f815f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.986+0000 7f26ff9b8640 1 --2- 172.21.15.105:0/3520177616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f80b4290 0x7f26f80b4670 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.986+0000 7f26ff9b8640 1 -- 172.21.15.105:0/3520177616 >> 172.21.15.105:0/3520177616 conn(0x7f26f80ceb90 msgr2=0x7f26f80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:34.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.986+0000 7f26ff9b8640 1 -- 172.21.15.105:0/3520177616 shutdown_connections 2023-11-28T18:24:34.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.986+0000 7f26ff9b8640 1 -- 172.21.15.105:0/3520177616 wait complete. 2023-11-28T18:24:34.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26ff9b8640 1 Processor -- start 2023-11-28T18:24:34.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26ff9b8640 1 -- start start 2023-11-28T18:24:34.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26ff9b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26f80b4290 0x7f26f815db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:34.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26ff9b8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26f815b030 0x7f26f80af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:34.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26ff9b8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f815be30 0x7f26f80afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:34.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26ff9b8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f8156a40 con 0x7f26f815be30 2023-11-28T18:24:34.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26ff9b8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f815e0a0 con 0x7f26f80b4290 2023-11-28T18:24:34.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26ff9b8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f815c3f0 con 0x7f26f815b030 2023-11-28T18:24:34.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26fdf55640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f815be30 0x7f26f80afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:34.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26fdf55640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f815be30 0x7f26f80afa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33512/0 (socket says 172.21.15.105:33512) 2023-11-28T18:24:34.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26fdf55640 1 -- 172.21.15.105:0/2454465239 learned_addr learned my addr 172.21.15.105:0/2454465239 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:34.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26fd754640 1 --2- 172.21.15.105:0/2454465239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26f80b4290 0x7f26f815db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:34.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26fcf53640 1 --2- 172.21.15.105:0/2454465239 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26f815b030 0x7f26f80af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:34.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26fdf55640 1 -- 172.21.15.105:0/2454465239 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26f815b030 msgr2=0x7f26f80af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:34.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26fdf55640 1 --2- 172.21.15.105:0/2454465239 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26f815b030 0x7f26f80af4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26fdf55640 1 -- 172.21.15.105:0/2454465239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26f80b4290 msgr2=0x7f26f815db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:34.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26fdf55640 1 --2- 172.21.15.105:0/2454465239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26f80b4290 0x7f26f815db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:34.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26fdf55640 1 -- 172.21.15.105:0/2454465239 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f40094d0 con 0x7f26f815be30 2023-11-28T18:24:34.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26fd754640 1 --2- 172.21.15.105:0/2454465239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26f80b4290 0x7f26f815db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:34.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26fdf55640 1 --2- 172.21.15.105:0/2454465239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f815be30 0x7f26f80afa00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f26f000c760 tx=0x7f26f000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:34.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26edffb640 1 -- 172.21.15.105:0/2454465239 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0003b80 con 0x7f26f815be30 2023-11-28T18:24:35.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26edffb640 1 -- 172.21.15.105:0/2454465239 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f0003ce0 con 0x7f26f815be30 2023-11-28T18:24:35.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26ff9b8640 1 -- 172.21.15.105:0/2454465239 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f815c6d0 con 0x7f26f815be30 2023-11-28T18:24:35.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26edffb640 1 -- 172.21.15.105:0/2454465239 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f000aa90 con 0x7f26f815be30 2023-11-28T18:24:35.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26ff9b8640 1 -- 172.21.15.105:0/2454465239 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f815cc90 con 0x7f26f815be30 2023-11-28T18:24:35.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.990+0000 7f26ff9b8640 1 -- 172.21.15.105:0/2454465239 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f80cb410 con 0x7f26f815be30 2023-11-28T18:24:35.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.994+0000 7f26edffb640 1 -- 172.21.15.105:0/2454465239 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f26f000abf0 con 0x7f26f815be30 2023-11-28T18:24:35.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.994+0000 7f26edffb640 1 --2- 172.21.15.105:0/2454465239 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f26dc0668d0 0x7f26dc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:35.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.994+0000 7f26edffb640 1 -- 172.21.15.105:0/2454465239 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f26f00845e0 con 0x7f26f815be30 2023-11-28T18:24:35.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.994+0000 7f26fd754640 1 --2- 172.21.15.105:0/2454465239 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f26dc0668d0 0x7f26dc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:35.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.994+0000 7f26fd754640 1 --2- 172.21.15.105:0/2454465239 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f26dc0668d0 0x7f26dc068d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f26f40ae000 tx=0x7f26f40a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:35.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:34.998+0000 7f26edffb640 1 -- 172.21.15.105:0/2454465239 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f26f0051380 con 0x7f26f815be30 2023-11-28T18:24:35.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.150+0000 7f26ff9b8640 1 -- 172.21.15.105:0/2454465239 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f26f80ad4b0 con 0x7f26f815be30 2023-11-28T18:24:35.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.150+0000 7f26edffb640 1 -- 172.21.15.105:0/2454465239 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f26f80ad4b0 con 0x7f26f815be30 2023-11-28T18:24:35.154 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:35.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.150+0000 7f26ff9b8640 1 -- 172.21.15.105:0/2454465239 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f26dc0668d0 msgr2=0x7f26dc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:35.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.150+0000 7f26ff9b8640 1 --2- 172.21.15.105:0/2454465239 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f26dc0668d0 0x7f26dc068d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f26f40ae000 tx=0x7f26f40a7000 comp rx=0 tx=0).stop 2023-11-28T18:24:35.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.150+0000 7f26ff9b8640 1 -- 172.21.15.105:0/2454465239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f815be30 msgr2=0x7f26f80afa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:35.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.150+0000 7f26ff9b8640 1 --2- 172.21.15.105:0/2454465239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f815be30 0x7f26f80afa00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f26f000c760 tx=0x7f26f000cc20 comp rx=0 tx=0).stop 2023-11-28T18:24:35.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.150+0000 7f26ff9b8640 1 -- 172.21.15.105:0/2454465239 shutdown_connections 2023-11-28T18:24:35.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.150+0000 7f26ff9b8640 1 --2- 172.21.15.105:0/2454465239 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f26dc0668d0 0x7f26dc068d90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.150+0000 7f26ff9b8640 1 --2- 172.21.15.105:0/2454465239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f815be30 0x7f26f80afa00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.150+0000 7f26ff9b8640 1 --2- 172.21.15.105:0/2454465239 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26f815b030 0x7f26f80af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.150+0000 7f26ff9b8640 1 --2- 172.21.15.105:0/2454465239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26f80b4290 0x7f26f815db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.150+0000 7f26ff9b8640 1 -- 172.21.15.105:0/2454465239 >> 172.21.15.105:0/2454465239 conn(0x7f26f80ceb90 msgr2=0x7f26f806c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:35.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.150+0000 7f26ff9b8640 1 -- 172.21.15.105:0/2454465239 shutdown_connections 2023-11-28T18:24:35.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.150+0000 7f26ff9b8640 1 -- 172.21.15.105:0/2454465239 wait complete. 2023-11-28T18:24:35.175 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:24:35.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.310+0000 7f207a556640 1 Processor -- start 2023-11-28T18:24:35.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.310+0000 7f207a556640 1 -- start start 2023-11-28T18:24:35.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.310+0000 7f207a556640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f207415e9f0 0x7f207406de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:35.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.310+0000 7f207a556640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f207406e3b0 con 0x7f207406c070 2023-11-28T18:24:35.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.310+0000 7f207a556640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f207406c500 con 0x7f20740cc8e0 2023-11-28T18:24:35.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.310+0000 7f207a556640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207406c7e0 con 0x7f207415e9f0 2023-11-28T18:24:35.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.310+0000 7f2078d53640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f207406c070 0x7f207415edd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35378/0 (socket says 172.21.15.105:35378) 2023-11-28T18:24:35.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.310+0000 7f2078d53640 1 -- 172.21.15.105:0/893590268 learned_addr learned my addr 172.21.15.105:0/893590268 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:35.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.310+0000 7f2079554640 1 --2- 172.21.15.105:0/893590268 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f207415e9f0 0x7f207406de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:35.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.310+0000 7f206bfff640 1 -- 172.21.15.105:0/893590268 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3334430353 0 0) 0x7f2064001120 con 0x7f207406c070 2023-11-28T18:24:35.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.310+0000 7f206bfff640 1 -- 172.21.15.105:0/893590268 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2060003530 con 0x7f207406c070 2023-11-28T18:24:35.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.314+0000 7f206bfff640 1 -- 172.21.15.105:0/893590268 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1108628125 0 0) 0x7f207406c500 con 0x7f20740cc8e0 2023-11-28T18:24:35.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.314+0000 7f206bfff640 1 -- 172.21.15.105:0/893590268 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2064001120 con 0x7f20740cc8e0 2023-11-28T18:24:35.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.314+0000 7f2079554640 1 -- 172.21.15.105:0/893590268 >> v1:172.21.15.148:6789/0 conn(0x7f20740cc8e0 legacy=0x7f20740ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:35.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.314+0000 7f2079554640 1 -- 172.21.15.105:0/893590268 >> v1:172.21.15.105:6789/0 conn(0x7f207406c070 legacy=0x7f207415edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:35.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.314+0000 7f206bfff640 1 -- 172.21.15.105:0/893590268 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2347346049 0 0) 0x7f2060003530 con 0x7f207406c070 2023-11-28T18:24:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.314+0000 7f2079554640 1 -- 172.21.15.105:0/893590268 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207406cac0 con 0x7f207415e9f0 2023-11-28T18:24:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.314+0000 7f2079554640 1 --2- 172.21.15.105:0/893590268 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f207415e9f0 0x7f207406de70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f20700097f0 tx=0x7f207001c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeb3d1e884686db1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.314+0000 7f206bfff640 1 -- 172.21.15.105:0/893590268 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2064001810 con 0x7f207415e9f0 2023-11-28T18:24:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.314+0000 7f206bfff640 1 -- 172.21.15.105:0/893590268 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20700a7050 con 0x7f207415e9f0 2023-11-28T18:24:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.314+0000 7f206bfff640 1 -- 172.21.15.105:0/893590268 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20700ab620 con 0x7f207415e9f0 2023-11-28T18:24:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.314+0000 7f207a556640 1 -- 172.21.15.105:0/893590268 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f207415e9f0 msgr2=0x7f207406de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.314+0000 7f207a556640 1 --2- 172.21.15.105:0/893590268 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f207415e9f0 0x7f207406de70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f20700097f0 tx=0x7f207001c3e0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f207a556640 1 -- 172.21.15.105:0/893590268 shutdown_connections 2023-11-28T18:24:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f207a556640 1 --2- 172.21.15.105:0/893590268 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f207415e9f0 0x7f207406de70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f207a556640 1 -- 172.21.15.105:0/893590268 >> 172.21.15.105:0/893590268 conn(0x7f20740ce9e0 msgr2=0x7f20740cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f207a556640 1 -- 172.21.15.105:0/893590268 shutdown_connections 2023-11-28T18:24:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f207a556640 1 -- 172.21.15.105:0/893590268 wait complete. 2023-11-28T18:24:35.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f207a556640 1 Processor -- start 2023-11-28T18:24:35.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f207a556640 1 -- start start 2023-11-28T18:24:35.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f207a556640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20740cc8e0 0x7f207415b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:35.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f207a556640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f207415e9f0 0x7f20741591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:35.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f207a556640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f207415a340 0x7f2074159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f207a556640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207413db90 con 0x7f207415a340 2023-11-28T18:24:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f207a556640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2074159c50 con 0x7f207415e9f0 2023-11-28T18:24:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f207a556640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207415a0d0 con 0x7f20740cc8e0 2023-11-28T18:24:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f2079d55640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f207415a340 0x7f2074159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f2079d55640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f207415a340 0x7f2074159710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33518/0 (socket says 172.21.15.105:33518) 2023-11-28T18:24:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f2079d55640 1 -- 172.21.15.105:0/3927434398 learned_addr learned my addr 172.21.15.105:0/3927434398 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f2079554640 1 --2- 172.21.15.105:0/3927434398 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20740cc8e0 0x7f207415b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f2078d53640 1 --2- 172.21.15.105:0/3927434398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f207415e9f0 0x7f20741591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f2079d55640 1 -- 172.21.15.105:0/3927434398 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20740cc8e0 msgr2=0x7f207415b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f2079d55640 1 --2- 172.21.15.105:0/3927434398 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20740cc8e0 0x7f207415b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f2079d55640 1 -- 172.21.15.105:0/3927434398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f207415e9f0 msgr2=0x7f20741591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f2079d55640 1 --2- 172.21.15.105:0/3927434398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f207415e9f0 0x7f20741591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f2079d55640 1 -- 172.21.15.105:0/3927434398 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2070009420 con 0x7f207415a340 2023-11-28T18:24:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f2079554640 1 --2- 172.21.15.105:0/3927434398 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20740cc8e0 0x7f207415b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.318+0000 7f2079d55640 1 --2- 172.21.15.105:0/3927434398 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f207415a340 0x7f2074159710 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f206c00c7c0 tx=0x7f206c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.322+0000 7f2069ffb640 1 -- 172.21.15.105:0/3927434398 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c003b80 con 0x7f207415a340 2023-11-28T18:24:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.322+0000 7f2069ffb640 1 -- 172.21.15.105:0/3927434398 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206c003ce0 con 0x7f207415a340 2023-11-28T18:24:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.322+0000 7f207a556640 1 -- 172.21.15.105:0/3927434398 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f207406d720 con 0x7f207415a340 2023-11-28T18:24:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.322+0000 7f2069ffb640 1 -- 172.21.15.105:0/3927434398 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c00aa50 con 0x7f207415a340 2023-11-28T18:24:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.322+0000 7f207a556640 1 -- 172.21.15.105:0/3927434398 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207415e080 con 0x7f207415a340 2023-11-28T18:24:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.322+0000 7f207a556640 1 -- 172.21.15.105:0/3927434398 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f207413dd00 con 0x7f207415a340 2023-11-28T18:24:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.322+0000 7f2069ffb640 1 -- 172.21.15.105:0/3927434398 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f206c00abb0 con 0x7f207415a340 2023-11-28T18:24:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.322+0000 7f2069ffb640 1 --2- 172.21.15.105:0/3927434398 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f20580668d0 0x7f2058068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.322+0000 7f2069ffb640 1 -- 172.21.15.105:0/3927434398 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f206c084710 con 0x7f207415a340 2023-11-28T18:24:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.322+0000 7f2079554640 1 --2- 172.21.15.105:0/3927434398 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f20580668d0 0x7f2058068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:35.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.326+0000 7f2079554640 1 --2- 172.21.15.105:0/3927434398 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f20580668d0 0x7f2058068d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f20700ad000 tx=0x7f20700a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:35.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.326+0000 7f2069ffb640 1 -- 172.21.15.105:0/3927434398 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f207413dd00 con 0x7f207415a340 2023-11-28T18:24:35.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.478+0000 7f207a556640 1 -- 172.21.15.105:0/3927434398 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f207413dcd0 con 0x7f207415a340 2023-11-28T18:24:35.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.478+0000 7f2069ffb640 1 -- 172.21.15.105:0/3927434398 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f206c051550 con 0x7f207415a340 2023-11-28T18:24:35.482 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:24:35.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.478+0000 7f207a556640 1 -- 172.21.15.105:0/3927434398 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f20580668d0 msgr2=0x7f2058068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:35.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.478+0000 7f207a556640 1 --2- 172.21.15.105:0/3927434398 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f20580668d0 0x7f2058068d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f20700ad000 tx=0x7f20700a6000 comp rx=0 tx=0).stop 2023-11-28T18:24:35.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.478+0000 7f207a556640 1 -- 172.21.15.105:0/3927434398 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f207415a340 msgr2=0x7f2074159710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:35.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.478+0000 7f207a556640 1 --2- 172.21.15.105:0/3927434398 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f207415a340 0x7f2074159710 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f206c00c7c0 tx=0x7f206c00cc80 comp rx=0 tx=0).stop 2023-11-28T18:24:35.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.478+0000 7f207a556640 1 -- 172.21.15.105:0/3927434398 shutdown_connections 2023-11-28T18:24:35.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.478+0000 7f207a556640 1 --2- 172.21.15.105:0/3927434398 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f20580668d0 0x7f2058068d90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.478+0000 7f207a556640 1 --2- 172.21.15.105:0/3927434398 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f207415a340 0x7f2074159710 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.478+0000 7f207a556640 1 --2- 172.21.15.105:0/3927434398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f207415e9f0 0x7f20741591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.478+0000 7f207a556640 1 --2- 172.21.15.105:0/3927434398 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20740cc8e0 0x7f207415b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.478+0000 7f207a556640 1 -- 172.21.15.105:0/3927434398 >> 172.21.15.105:0/3927434398 conn(0x7f20740ce9e0 msgr2=0x7f2074072eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:35.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.478+0000 7f207a556640 1 -- 172.21.15.105:0/3927434398 shutdown_connections 2023-11-28T18:24:35.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.478+0000 7f207a556640 1 -- 172.21.15.105:0/3927434398 wait complete. 2023-11-28T18:24:35.503 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T18:24:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.638+0000 7f99a713e640 1 Processor -- start 2023-11-28T18:24:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.638+0000 7f99a713e640 1 -- start start 2023-11-28T18:24:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.638+0000 7f99a713e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f99a00d10e0 0x7f99a00d3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.638+0000 7f99a713e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99a0160530 0x7f99a0160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.638+0000 7f99a713e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a00d0820 con 0x7f99a00b4140 2023-11-28T18:24:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.638+0000 7f99a713e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a0160ec0 con 0x7f99a00d10e0 2023-11-28T18:24:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.638+0000 7f99a713e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a00d1d20 con 0x7f99a0160530 2023-11-28T18:24:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.638+0000 7f99a56db640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99a0160530 0x7f99a0160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.638+0000 7f99a4eda640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f99a00d10e0 0x7f99a00d3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.638+0000 7f99a56db640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99a0160530 0x7f99a0160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:40716/0 (socket says 172.21.15.105:40716) 2023-11-28T18:24:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.638+0000 7f99a4eda640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f99a00d10e0 0x7f99a00d3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:59756/0 (socket says 172.21.15.105:59756) 2023-11-28T18:24:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.638+0000 7f99a56db640 1 -- 172.21.15.105:0/1613441208 learned_addr learned my addr 172.21.15.105:0/1613441208 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.642+0000 7f99a56db640 1 -- 172.21.15.105:0/1613441208 >> v2:172.21.15.148:3300/0 conn(0x7f99a00d10e0 msgr2=0x7f99a00d3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.642+0000 7f99a56db640 1 --2- 172.21.15.105:0/1613441208 >> v2:172.21.15.148:3300/0 conn(0x7f99a00d10e0 0x7f99a00d3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.642+0000 7f99a56db640 1 -- 172.21.15.105:0/1613441208 >> v1:172.21.15.148:6789/0 conn(0x7f99a00b4140 legacy=0x7f99a00d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.642+0000 7f99a56db640 1 -- 172.21.15.105:0/1613441208 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a00d01d0 con 0x7f99a0160530 2023-11-28T18:24:35.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.642+0000 7f99a56db640 1 --2- 172.21.15.105:0/1613441208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99a0160530 0x7f99a0160980 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9998009e50 tx=0x7f999801c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30afb4f87c662e92 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:35.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.642+0000 7f99977fe640 1 -- 172.21.15.105:0/1613441208 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99980a57f0 con 0x7f99a0160530 2023-11-28T18:24:35.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.642+0000 7f99977fe640 1 -- 172.21.15.105:0/1613441208 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99980a5950 con 0x7f99a0160530 2023-11-28T18:24:35.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.642+0000 7f99977fe640 1 -- 172.21.15.105:0/1613441208 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99980a5ce0 con 0x7f99a0160530 2023-11-28T18:24:35.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a713e640 1 -- 172.21.15.105:0/1613441208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99a0160530 msgr2=0x7f99a0160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:35.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a713e640 1 --2- 172.21.15.105:0/1613441208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99a0160530 0x7f99a0160980 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9998009e50 tx=0x7f999801c950 comp rx=0 tx=0).stop 2023-11-28T18:24:35.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a713e640 1 -- 172.21.15.105:0/1613441208 shutdown_connections 2023-11-28T18:24:35.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a713e640 1 --2- 172.21.15.105:0/1613441208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99a0160530 0x7f99a0160980 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a713e640 1 --2- 172.21.15.105:0/1613441208 >> v2:172.21.15.148:3300/0 conn(0x7f99a00d10e0 0x7f99a00d3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a713e640 1 -- 172.21.15.105:0/1613441208 >> 172.21.15.105:0/1613441208 conn(0x7f99a00d30a0 msgr2=0x7f99a00d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:35.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a713e640 1 -- 172.21.15.105:0/1613441208 shutdown_connections 2023-11-28T18:24:35.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a713e640 1 -- 172.21.15.105:0/1613441208 wait complete. 2023-11-28T18:24:35.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a713e640 1 Processor -- start 2023-11-28T18:24:35.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a713e640 1 -- start start 2023-11-28T18:24:35.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a713e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a00b4140 0x7f99a015f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:35.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a713e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99a00d10e0 0x7f99a0149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:35.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a713e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99a0160530 0x7f99a0149590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:35.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a713e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a00ccfb0 con 0x7f99a00b4140 2023-11-28T18:24:35.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a713e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a015c970 con 0x7f99a0160530 2023-11-28T18:24:35.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a713e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a015cc50 con 0x7f99a00d10e0 2023-11-28T18:24:35.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a4eda640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a00b4140 0x7f99a015f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:35.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a4eda640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a00b4140 0x7f99a015f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33524/0 (socket says 172.21.15.105:33524) 2023-11-28T18:24:35.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a4eda640 1 -- 172.21.15.105:0/3273508909 learned_addr learned my addr 172.21.15.105:0/3273508909 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:35.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a4eda640 1 -- 172.21.15.105:0/3273508909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99a00d10e0 msgr2=0x7f99a0149050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:35.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a56db640 1 --2- 172.21.15.105:0/3273508909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99a0160530 0x7f99a0149590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:35.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f9997fff640 1 --2- 172.21.15.105:0/3273508909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99a00d10e0 0x7f99a0149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:35.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a4eda640 1 --2- 172.21.15.105:0/3273508909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99a00d10e0 0x7f99a0149050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a4eda640 1 -- 172.21.15.105:0/3273508909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99a0160530 msgr2=0x7f99a0149590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:35.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a4eda640 1 --2- 172.21.15.105:0/3273508909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99a0160530 0x7f99a0149590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a4eda640 1 -- 172.21.15.105:0/3273508909 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9998009b30 con 0x7f99a00b4140 2023-11-28T18:24:35.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a56db640 1 --2- 172.21.15.105:0/3273508909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99a0160530 0x7f99a0149590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:35.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f9997fff640 1 --2- 172.21.15.105:0/3273508909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99a00d10e0 0x7f99a0149050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:35.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.646+0000 7f99a4eda640 1 --2- 172.21.15.105:0/3273508909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a00b4140 0x7f99a015f670 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f999c00e930 tx=0x7f999c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:35.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.650+0000 7f99957fa640 1 -- 172.21.15.105:0/3273508909 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c009690 con 0x7f99a00b4140 2023-11-28T18:24:35.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.650+0000 7f99957fa640 1 -- 172.21.15.105:0/3273508909 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999c00ca00 con 0x7f99a00b4140 2023-11-28T18:24:35.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.650+0000 7f99957fa640 1 -- 172.21.15.105:0/3273508909 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c00cd70 con 0x7f99a00b4140 2023-11-28T18:24:35.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.650+0000 7f99a713e640 1 -- 172.21.15.105:0/3273508909 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a015d0b0 con 0x7f99a00b4140 2023-11-28T18:24:35.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.650+0000 7f99a713e640 1 -- 172.21.15.105:0/3273508909 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a015d670 con 0x7f99a00b4140 2023-11-28T18:24:35.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.650+0000 7f99a713e640 1 -- 172.21.15.105:0/3273508909 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a00745f0 con 0x7f99a00b4140 2023-11-28T18:24:35.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.650+0000 7f99957fa640 1 -- 172.21.15.105:0/3273508909 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f999c013470 con 0x7f99a00b4140 2023-11-28T18:24:35.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.650+0000 7f99957fa640 1 --2- 172.21.15.105:0/3273508909 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f998c066900 0x7f998c068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:35.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.650+0000 7f99957fa640 1 -- 172.21.15.105:0/3273508909 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f999c0849e0 con 0x7f99a00b4140 2023-11-28T18:24:35.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.650+0000 7f9997fff640 1 --2- 172.21.15.105:0/3273508909 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f998c066900 0x7f998c068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:35.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.650+0000 7f9997fff640 1 --2- 172.21.15.105:0/3273508909 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f998c066900 0x7f998c068dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f99a01588e0 tx=0x7f9990009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:35.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.654+0000 7f99957fa640 1 -- 172.21.15.105:0/3273508909 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f999c051970 con 0x7f99a00b4140 2023-11-28T18:24:35.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.802+0000 7f99a713e640 1 -- 172.21.15.105:0/3273508909 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f99a00cf140 con 0x7f99a00b4140 2023-11-28T18:24:35.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.806+0000 7f99957fa640 1 -- 172.21.15.105:0/3273508909 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f999c054f90 con 0x7f99a00b4140 2023-11-28T18:24:35.810 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:35.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.806+0000 7f99a713e640 1 -- 172.21.15.105:0/3273508909 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f998c066900 msgr2=0x7f998c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:35.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.806+0000 7f99a713e640 1 --2- 172.21.15.105:0/3273508909 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f998c066900 0x7f998c068dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f99a01588e0 tx=0x7f9990009040 comp rx=0 tx=0).stop 2023-11-28T18:24:35.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.806+0000 7f99a713e640 1 -- 172.21.15.105:0/3273508909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a00b4140 msgr2=0x7f99a015f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:35.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.806+0000 7f99a713e640 1 --2- 172.21.15.105:0/3273508909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a00b4140 0x7f99a015f670 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f999c00e930 tx=0x7f999c00edf0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.806+0000 7f99a713e640 1 -- 172.21.15.105:0/3273508909 shutdown_connections 2023-11-28T18:24:35.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.806+0000 7f99a713e640 1 --2- 172.21.15.105:0/3273508909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99a0160530 0x7f99a0149590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.806+0000 7f99a713e640 1 --2- 172.21.15.105:0/3273508909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99a00d10e0 0x7f99a0149050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.806+0000 7f99a713e640 1 --2- 172.21.15.105:0/3273508909 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f998c066900 0x7f998c068dc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.806+0000 7f99a713e640 1 --2- 172.21.15.105:0/3273508909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a00b4140 0x7f99a015f670 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.806+0000 7f99a713e640 1 -- 172.21.15.105:0/3273508909 >> 172.21.15.105:0/3273508909 conn(0x7f99a00d30a0 msgr2=0x7f99a006c990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:35.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.806+0000 7f99a713e640 1 -- 172.21.15.105:0/3273508909 shutdown_connections 2023-11-28T18:24:35.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.806+0000 7f99a713e640 1 -- 172.21.15.105:0/3273508909 wait complete. 2023-11-28T18:24:35.830 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T18:24:35.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.962+0000 7f1d247c3640 1 Processor -- start 2023-11-28T18:24:35.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.962+0000 7f1d247c3640 1 -- start start 2023-11-28T18:24:35.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.966+0000 7f1d247c3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1d1c159f40 0x7f1d1c0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:35.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.966+0000 7f1d247c3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1d1c0cc760 0x7f1d1c0ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:35.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.966+0000 7f1d247c3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d1c06bf50 con 0x7f1d1c0ad680 2023-11-28T18:24:35.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.966+0000 7f1d247c3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d1c0ccbc0 con 0x7f1d1c159f40 2023-11-28T18:24:35.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.966+0000 7f1d247c3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d1c15a320 con 0x7f1d1c0cc760 2023-11-28T18:24:35.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.966+0000 7f1d2255f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1d1c159f40 0x7f1d1c0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:35.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.966+0000 7f1d2255f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1d1c159f40 0x7f1d1c0caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33538/0 (socket says 172.21.15.105:33538) 2023-11-28T18:24:35.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.966+0000 7f1d2255f640 1 -- 172.21.15.105:0/847367808 learned_addr learned my addr 172.21.15.105:0/847367808 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:35.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.966+0000 7f1d21d5e640 1 --2- 172.21.15.105:0/847367808 >> v2:172.21.15.148:3300/0 conn(0x7f1d1c0cc760 0x7f1d1c0ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:35.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.966+0000 7f1d2255f640 1 -- 172.21.15.105:0/847367808 >> v2:172.21.15.148:3300/0 conn(0x7f1d1c0cc760 msgr2=0x7f1d1c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:35.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.966+0000 7f1d2255f640 1 --2- 172.21.15.105:0/847367808 >> v2:172.21.15.148:3300/0 conn(0x7f1d1c0cc760 0x7f1d1c0ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.966+0000 7f1d2255f640 1 -- 172.21.15.105:0/847367808 >> v1:172.21.15.148:6789/0 conn(0x7f1d1c0ad680 legacy=0x7f1d1c15da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:35.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.966+0000 7f1d2255f640 1 -- 172.21.15.105:0/847367808 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1c071380 con 0x7f1d1c159f40 2023-11-28T18:24:35.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.966+0000 7f1d21d5e640 1 --2- 172.21.15.105:0/847367808 >> v2:172.21.15.148:3300/0 conn(0x7f1d1c0cc760 0x7f1d1c0ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:35.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.970+0000 7f1d2255f640 1 --2- 172.21.15.105:0/847367808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d1c159f40 0x7f1d1c0caf60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c09d0d0 tx=0x7f1d1801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9dff8bb372af2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:35.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.970+0000 7f1d2155d640 1 -- 172.21.15.105:0/847367808 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d180ab640 con 0x7f1d1c159f40 2023-11-28T18:24:35.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.970+0000 7f1d2155d640 1 -- 172.21.15.105:0/847367808 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d180a7070 con 0x7f1d1c159f40 2023-11-28T18:24:35.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.970+0000 7f1d2155d640 1 -- 172.21.15.105:0/847367808 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d180ab9c0 con 0x7f1d1c159f40 2023-11-28T18:24:35.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.970+0000 7f1d247c3640 1 -- 172.21.15.105:0/847367808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d1c159f40 msgr2=0x7f1d1c0caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:35.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.970+0000 7f1d247c3640 1 --2- 172.21.15.105:0/847367808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d1c159f40 0x7f1d1c0caf60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c09d0d0 tx=0x7f1d1801c440 comp rx=0 tx=0).stop 2023-11-28T18:24:35.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.970+0000 7f1d247c3640 1 -- 172.21.15.105:0/847367808 shutdown_connections 2023-11-28T18:24:35.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.970+0000 7f1d247c3640 1 --2- 172.21.15.105:0/847367808 >> v2:172.21.15.148:3300/0 conn(0x7f1d1c0cc760 0x7f1d1c0ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.970+0000 7f1d247c3640 1 --2- 172.21.15.105:0/847367808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d1c159f40 0x7f1d1c0caf60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.970+0000 7f1d247c3640 1 -- 172.21.15.105:0/847367808 >> 172.21.15.105:0/847367808 conn(0x7f1d1c0ceb90 msgr2=0x7f1d1c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:35.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.970+0000 7f1d247c3640 1 -- 172.21.15.105:0/847367808 shutdown_connections 2023-11-28T18:24:35.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.970+0000 7f1d247c3640 1 -- 172.21.15.105:0/847367808 wait complete. 2023-11-28T18:24:35.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.970+0000 7f1d247c3640 1 Processor -- start 2023-11-28T18:24:35.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.970+0000 7f1d247c3640 1 -- start start 2023-11-28T18:24:35.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.970+0000 7f1d247c3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d1c0ad680 0x7f1d1c165da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:35.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.970+0000 7f1d247c3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d1c0cc760 0x7f1d1c15a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:35.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.970+0000 7f1d247c3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d1c159f40 0x7f1d1c158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:35.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.970+0000 7f1d247c3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c15ccf0 con 0x7f1d1c0ad680 2023-11-28T18:24:35.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.970+0000 7f1d247c3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c15cfb0 con 0x7f1d1c0cc760 2023-11-28T18:24:35.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.970+0000 7f1d247c3640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c15d290 con 0x7f1d1c159f40 2023-11-28T18:24:35.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.974+0000 7f1d2255f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d1c0ad680 0x7f1d1c165da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:35.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.974+0000 7f1d2255f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d1c0ad680 0x7f1d1c165da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33548/0 (socket says 172.21.15.105:33548) 2023-11-28T18:24:35.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.974+0000 7f1d2255f640 1 -- 172.21.15.105:0/3873013491 learned_addr learned my addr 172.21.15.105:0/3873013491 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:35.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.974+0000 7f1d21d5e640 1 --2- 172.21.15.105:0/3873013491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d1c0cc760 0x7f1d1c15a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:35.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.974+0000 7f1d22d60640 1 --2- 172.21.15.105:0/3873013491 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d1c159f40 0x7f1d1c158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:35.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.974+0000 7f1d2255f640 1 -- 172.21.15.105:0/3873013491 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d1c159f40 msgr2=0x7f1d1c158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:35.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.974+0000 7f1d2255f640 1 --2- 172.21.15.105:0/3873013491 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d1c159f40 0x7f1d1c158b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.974+0000 7f1d2255f640 1 -- 172.21.15.105:0/3873013491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d1c0cc760 msgr2=0x7f1d1c15a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:35.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.974+0000 7f1d2255f640 1 --2- 172.21.15.105:0/3873013491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d1c0cc760 0x7f1d1c15a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:35.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.974+0000 7f1d2255f640 1 -- 172.21.15.105:0/3873013491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d180094d0 con 0x7f1d1c0ad680 2023-11-28T18:24:35.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.974+0000 7f1d21d5e640 1 --2- 172.21.15.105:0/3873013491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d1c0cc760 0x7f1d1c15a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:35.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.974+0000 7f1d2255f640 1 --2- 172.21.15.105:0/3873013491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d1c0ad680 0x7f1d1c165da0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1d18019050 tx=0x7f1d1801ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:35.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.974+0000 7f1d12ffd640 1 -- 172.21.15.105:0/3873013491 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d180be3f0 con 0x7f1d1c0ad680 2023-11-28T18:24:35.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.974+0000 7f1d12ffd640 1 -- 172.21.15.105:0/3873013491 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d180be550 con 0x7f1d1c0ad680 2023-11-28T18:24:35.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.974+0000 7f1d247c3640 1 -- 172.21.15.105:0/3873013491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1c152d70 con 0x7f1d1c0ad680 2023-11-28T18:24:35.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.974+0000 7f1d12ffd640 1 -- 172.21.15.105:0/3873013491 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d180be900 con 0x7f1d1c0ad680 2023-11-28T18:24:35.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.974+0000 7f1d247c3640 1 -- 172.21.15.105:0/3873013491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d1c0aeb60 con 0x7f1d1c0ad680 2023-11-28T18:24:35.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.974+0000 7f1d247c3640 1 -- 172.21.15.105:0/3873013491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d1c0723f0 con 0x7f1d1c0ad680 2023-11-28T18:24:35.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.974+0000 7f1d12ffd640 1 -- 172.21.15.105:0/3873013491 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f1d180c7440 con 0x7f1d1c0ad680 2023-11-28T18:24:35.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.974+0000 7f1d12ffd640 1 --2- 172.21.15.105:0/3873013491 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1d00066900 0x7f1d00068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:35.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.974+0000 7f1d12ffd640 1 -- 172.21.15.105:0/3873013491 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f1d1811f0c0 con 0x7f1d1c0ad680 2023-11-28T18:24:35.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.978+0000 7f1d21d5e640 1 --2- 172.21.15.105:0/3873013491 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1d00066900 0x7f1d00068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:35.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.978+0000 7f1d21d5e640 1 --2- 172.21.15.105:0/3873013491 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1d00066900 0x7f1d00068dc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c1f98f0 tx=0x7f1d0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:35.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:35.978+0000 7f1d12ffd640 1 -- 172.21.15.105:0/3873013491 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1d180ebf90 con 0x7f1d1c0ad680 2023-11-28T18:24:36.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.138+0000 7f1d247c3640 1 -- 172.21.15.105:0/3873013491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1d1c067570 con 0x7f1d1c0ad680 2023-11-28T18:24:36.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.142+0000 7f1d12ffd640 1 -- 172.21.15.105:0/3873013491 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1d180ef5b0 con 0x7f1d1c0ad680 2023-11-28T18:24:36.146 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2023-11-28T18:24:36.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.142+0000 7f1d247c3640 1 -- 172.21.15.105:0/3873013491 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1d00066900 msgr2=0x7f1d00068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:36.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.142+0000 7f1d247c3640 1 --2- 172.21.15.105:0/3873013491 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1d00066900 0x7f1d00068dc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c1f98f0 tx=0x7f1d0c008040 comp rx=0 tx=0).stop 2023-11-28T18:24:36.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.142+0000 7f1d247c3640 1 -- 172.21.15.105:0/3873013491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d1c0ad680 msgr2=0x7f1d1c165da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:36.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.142+0000 7f1d247c3640 1 --2- 172.21.15.105:0/3873013491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d1c0ad680 0x7f1d1c165da0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1d18019050 tx=0x7f1d1801ac80 comp rx=0 tx=0).stop 2023-11-28T18:24:36.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.142+0000 7f1d247c3640 1 -- 172.21.15.105:0/3873013491 shutdown_connections 2023-11-28T18:24:36.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.142+0000 7f1d247c3640 1 --2- 172.21.15.105:0/3873013491 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d1c159f40 0x7f1d1c158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.142+0000 7f1d247c3640 1 --2- 172.21.15.105:0/3873013491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d1c0cc760 0x7f1d1c15a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.142+0000 7f1d247c3640 1 --2- 172.21.15.105:0/3873013491 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1d00066900 0x7f1d00068dc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.142+0000 7f1d247c3640 1 --2- 172.21.15.105:0/3873013491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d1c0ad680 0x7f1d1c165da0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.142+0000 7f1d247c3640 1 -- 172.21.15.105:0/3873013491 >> 172.21.15.105:0/3873013491 conn(0x7f1d1c0ceb90 msgr2=0x7f1d1c0ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:36.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.142+0000 7f1d247c3640 1 -- 172.21.15.105:0/3873013491 shutdown_connections 2023-11-28T18:24:36.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.142+0000 7f1d247c3640 1 -- 172.21.15.105:0/3873013491 wait complete. 2023-11-28T18:24:36.167 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2023-11-28T18:24:36.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:36.168 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-11-28T18:24:36.189 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:24:36.189 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:24:36.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:36.190 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T18:24:36.246 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:24:36.247 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:24:36.257 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:24:36.258 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:24:36.315 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:24:36.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.450+0000 7f48e152b640 1 Processor -- start 2023-11-28T18:24:36.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.450+0000 7f48e152b640 1 -- start start 2023-11-28T18:24:36.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.450+0000 7f48e152b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48dc0c99d0 0x7f48dc0b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:36.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.450+0000 7f48e152b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f48dc0b1d00 0x7f48dc071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:36.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.450+0000 7f48e152b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48dc0cb230 con 0x7f48dc15ae50 2023-11-28T18:24:36.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.450+0000 7f48e152b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48dc0ca9d0 con 0x7f48dc0b1d00 2023-11-28T18:24:36.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.450+0000 7f48e152b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc0c9e50 con 0x7f48dc0c99d0 2023-11-28T18:24:36.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.450+0000 7f48e0d2a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f48dc0b1d00 0x7f48dc071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:36.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.450+0000 7f48e0d2a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f48dc0b1d00 0x7f48dc071660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33564/0 (socket says 172.21.15.105:33564) 2023-11-28T18:24:36.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.450+0000 7f48e0d2a640 1 -- 172.21.15.105:0/432370435 learned_addr learned my addr 172.21.15.105:0/432370435 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:36.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.450+0000 7f48db7fe640 1 --2- 172.21.15.105:0/432370435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48dc0c99d0 0x7f48dc0b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:36.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.450+0000 7f48e0d2a640 1 -- 172.21.15.105:0/432370435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48dc0c99d0 msgr2=0x7f48dc0b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:36.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.450+0000 7f48e0d2a640 1 --2- 172.21.15.105:0/432370435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48dc0c99d0 0x7f48dc0b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.450+0000 7f48e0d2a640 1 -- 172.21.15.105:0/432370435 >> v1:172.21.15.148:6789/0 conn(0x7f48dc15ae50 legacy=0x7f48dc0cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:36.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.450+0000 7f48e0d2a640 1 -- 172.21.15.105:0/432370435 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc0ca5c0 con 0x7f48dc0b1d00 2023-11-28T18:24:36.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.454+0000 7f48e0d2a640 1 --2- 172.21.15.105:0/432370435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48dc0b1d00 0x7f48dc071660 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009e50 tx=0x7f48d401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66df03f3db7e70a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:36.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.454+0000 7f48daffd640 1 -- 172.21.15.105:0/432370435 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d40a58a0 con 0x7f48dc0b1d00 2023-11-28T18:24:36.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.454+0000 7f48daffd640 1 -- 172.21.15.105:0/432370435 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d40a5a00 con 0x7f48dc0b1d00 2023-11-28T18:24:36.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.454+0000 7f48daffd640 1 -- 172.21.15.105:0/432370435 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d40a5d90 con 0x7f48dc0b1d00 2023-11-28T18:24:36.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.454+0000 7f48e152b640 1 -- 172.21.15.105:0/432370435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48dc0b1d00 msgr2=0x7f48dc071660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:36.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.454+0000 7f48e152b640 1 --2- 172.21.15.105:0/432370435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48dc0b1d00 0x7f48dc071660 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009e50 tx=0x7f48d401c9e0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.454+0000 7f48e152b640 1 -- 172.21.15.105:0/432370435 shutdown_connections 2023-11-28T18:24:36.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.454+0000 7f48e152b640 1 --2- 172.21.15.105:0/432370435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48dc0b1d00 0x7f48dc071660 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.454+0000 7f48e152b640 1 --2- 172.21.15.105:0/432370435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48dc0c99d0 0x7f48dc0b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.454+0000 7f48e152b640 1 -- 172.21.15.105:0/432370435 >> 172.21.15.105:0/432370435 conn(0x7f48dc0b0480 msgr2=0x7f48dc0b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.454+0000 7f48e152b640 1 -- 172.21.15.105:0/432370435 shutdown_connections 2023-11-28T18:24:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.454+0000 7f48e152b640 1 -- 172.21.15.105:0/432370435 wait complete. 2023-11-28T18:24:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.454+0000 7f48e152b640 1 Processor -- start 2023-11-28T18:24:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48e152b640 1 -- start start 2023-11-28T18:24:36.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48e152b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48dc0b1d00 0x7f48dc159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:36.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48e152b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48dc0c99d0 0x7f48dc0c24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:36.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48e152b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48dc15ae50 0x7f48dc145af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:36.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48e152b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc146780 con 0x7f48dc15ae50 2023-11-28T18:24:36.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48e152b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc0c2b80 con 0x7f48dc0c99d0 2023-11-28T18:24:36.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48e152b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc0c2e60 con 0x7f48dc0b1d00 2023-11-28T18:24:36.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48e0d2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48dc15ae50 0x7f48dc145af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:36.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48e0d2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48dc15ae50 0x7f48dc145af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33572/0 (socket says 172.21.15.105:33572) 2023-11-28T18:24:36.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48e0d2a640 1 -- 172.21.15.105:0/3003777175 learned_addr learned my addr 172.21.15.105:0/3003777175 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:36.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48dbfff640 1 --2- 172.21.15.105:0/3003777175 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48dc0b1d00 0x7f48dc159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:36.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48db7fe640 1 --2- 172.21.15.105:0/3003777175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48dc0c99d0 0x7f48dc0c24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:36.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48e0d2a640 1 -- 172.21.15.105:0/3003777175 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48dc0b1d00 msgr2=0x7f48dc159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:36.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48e0d2a640 1 --2- 172.21.15.105:0/3003777175 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48dc0b1d00 0x7f48dc159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48e0d2a640 1 -- 172.21.15.105:0/3003777175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48dc0c99d0 msgr2=0x7f48dc0c24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:36.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48e0d2a640 1 --2- 172.21.15.105:0/3003777175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48dc0c99d0 0x7f48dc0c24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48e0d2a640 1 -- 172.21.15.105:0/3003777175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d4009b30 con 0x7f48dc15ae50 2023-11-28T18:24:36.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48e0d2a640 1 --2- 172.21.15.105:0/3003777175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48dc15ae50 0x7f48dc145af0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f48d40af040 tx=0x7f48d4019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:36.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48d8ff9640 1 -- 172.21.15.105:0/3003777175 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d40b8670 con 0x7f48dc15ae50 2023-11-28T18:24:36.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48d8ff9640 1 -- 172.21.15.105:0/3003777175 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d40b8810 con 0x7f48dc15ae50 2023-11-28T18:24:36.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48e152b640 1 -- 172.21.15.105:0/3003777175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48dc0c3140 con 0x7f48dc15ae50 2023-11-28T18:24:36.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48d8ff9640 1 -- 172.21.15.105:0/3003777175 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d40b8bc0 con 0x7f48dc15ae50 2023-11-28T18:24:36.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48e152b640 1 -- 172.21.15.105:0/3003777175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48dc0cc800 con 0x7f48dc15ae50 2023-11-28T18:24:36.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.458+0000 7f48e152b640 1 -- 172.21.15.105:0/3003777175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48dc0cc9f0 con 0x7f48dc15ae50 2023-11-28T18:24:36.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.462+0000 7f48d8ff9640 1 -- 172.21.15.105:0/3003777175 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f48d40b8d20 con 0x7f48dc15ae50 2023-11-28T18:24:36.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.462+0000 7f48d8ff9640 1 --2- 172.21.15.105:0/3003777175 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48c8066900 0x7f48c8068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:36.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.462+0000 7f48d8ff9640 1 -- 172.21.15.105:0/3003777175 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f48d411fe10 con 0x7f48dc15ae50 2023-11-28T18:24:36.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.462+0000 7f48dbfff640 1 --2- 172.21.15.105:0/3003777175 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48c8066900 0x7f48c8068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:36.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.462+0000 7f48dbfff640 1 --2- 172.21.15.105:0/3003777175 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48c8066900 0x7f48c8068dc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f48cc007fd0 tx=0x7f48cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:36.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.466+0000 7f48d8ff9640 1 -- 172.21.15.105:0/3003777175 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f48d40ece20 con 0x7f48dc15ae50 2023-11-28T18:24:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.618+0000 7f48e152b640 1 -- 172.21.15.105:0/3003777175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48dc0c2fa0 con 0x7f48dc15ae50 2023-11-28T18:24:36.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.618+0000 7f48d8ff9640 1 -- 172.21.15.105:0/3003777175 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18887 (secure 0 0 0) 0x7f48dc0c2fa0 con 0x7f48dc15ae50 2023-11-28T18:24:36.623 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:36.624 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:24:15.048168+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:24:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.618+0000 7f48e152b640 1 -- 172.21.15.105:0/3003777175 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48c8066900 msgr2=0x7f48c8068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.618+0000 7f48e152b640 1 --2- 172.21.15.105:0/3003777175 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48c8066900 0x7f48c8068dc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f48cc007fd0 tx=0x7f48cc009040 comp rx=0 tx=0).stop 2023-11-28T18:24:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.618+0000 7f48e152b640 1 -- 172.21.15.105:0/3003777175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48dc15ae50 msgr2=0x7f48dc145af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.618+0000 7f48e152b640 1 --2- 172.21.15.105:0/3003777175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48dc15ae50 0x7f48dc145af0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f48d40af040 tx=0x7f48d4019780 comp rx=0 tx=0).stop 2023-11-28T18:24:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.618+0000 7f48e152b640 1 -- 172.21.15.105:0/3003777175 shutdown_connections 2023-11-28T18:24:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.618+0000 7f48e152b640 1 --2- 172.21.15.105:0/3003777175 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48c8066900 0x7f48c8068dc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.618+0000 7f48e152b640 1 --2- 172.21.15.105:0/3003777175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48dc15ae50 0x7f48dc145af0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.618+0000 7f48e152b640 1 --2- 172.21.15.105:0/3003777175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48dc0c99d0 0x7f48dc0c24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.618+0000 7f48e152b640 1 --2- 172.21.15.105:0/3003777175 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48dc0b1d00 0x7f48dc159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.618+0000 7f48e152b640 1 -- 172.21.15.105:0/3003777175 >> 172.21.15.105:0/3003777175 conn(0x7f48dc0b0480 msgr2=0x7f48dc06fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.618+0000 7f48e152b640 1 -- 172.21.15.105:0/3003777175 shutdown_connections 2023-11-28T18:24:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.622+0000 7f48e152b640 1 -- 172.21.15.105:0/3003777175 wait complete. 2023-11-28T18:24:36.645 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:23:31.485422+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:23:32.274378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T18:23:36.100071+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T18:24:36.646 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:24:36.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7fafa189a640 1 Processor -- start 2023-11-28T18:24:36.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7fafa189a640 1 -- start start 2023-11-28T18:24:36.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7fafa189a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7faf9c159f40 0x7faf9c0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:36.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7fafa189a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9c06bf50 con 0x7faf9c0ad850 2023-11-28T18:24:36.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7fafa189a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9c0ccbc0 con 0x7faf9c0cc760 2023-11-28T18:24:36.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7fafa189a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9c15a320 con 0x7faf9c159f40 2023-11-28T18:24:36.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf9b7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7faf9c0ad850 0x7faf9c0701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35392/0 (socket says 172.21.15.105:35392) 2023-11-28T18:24:36.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf9b7fe640 1 -- 172.21.15.105:0/1992481066 learned_addr learned my addr 172.21.15.105:0/1992481066 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:36.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf9affd640 1 --2- 172.21.15.105:0/1992481066 >> v2:172.21.15.148:3300/0 conn(0x7faf9c159f40 0x7faf9c0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:36.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf9a7fc640 1 -- 172.21.15.105:0/1992481066 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229679131 0 0) 0x7faf9c06bf50 con 0x7faf9c0ad850 2023-11-28T18:24:36.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf9a7fc640 1 -- 172.21.15.105:0/1992481066 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf88003530 con 0x7faf9c0ad850 2023-11-28T18:24:36.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf9a7fc640 1 -- 172.21.15.105:0/1992481066 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 240952936 0 0) 0x7faf88003530 con 0x7faf9c0ad850 2023-11-28T18:24:36.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf9a7fc640 1 -- 172.21.15.105:0/1992481066 >> v2:172.21.15.148:3300/0 conn(0x7faf9c159f40 msgr2=0x7faf9c0caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:36.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf9a7fc640 1 --2- 172.21.15.105:0/1992481066 >> v2:172.21.15.148:3300/0 conn(0x7faf9c159f40 0x7faf9c0caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf9a7fc640 1 -- 172.21.15.105:0/1992481066 >> v1:172.21.15.148:6789/0 conn(0x7faf9c0cc760 legacy=0x7faf9c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:36.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf9a7fc640 1 -- 172.21.15.105:0/1992481066 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9c071380 con 0x7faf9c0ad850 2023-11-28T18:24:36.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf9a7fc640 1 -- 172.21.15.105:0/1992481066 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7faf94001a00 con 0x7faf9c0ad850 2023-11-28T18:24:36.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf9a7fc640 1 -- 172.21.15.105:0/1992481066 >> v1:172.21.15.105:6789/0 conn(0x7faf9c0ad850 legacy=0x7faf9c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:36.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf9a7fc640 1 --2- 172.21.15.105:0/1992481066 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faf88003980 0x7faf88003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:36.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf9a7fc640 1 --2- 172.21.15.105:0/1992481066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf880047d0 0x7faf88004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:36.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf9a7fc640 1 --2- 172.21.15.105:0/1992481066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf88005190 0x7faf8800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:36.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf9a7fc640 1 -- 172.21.15.105:0/1992481066 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c15a320 con 0x7faf880047d0 2023-11-28T18:24:36.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf9a7fc640 1 -- 172.21.15.105:0/1992481066 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8800c130 con 0x7faf88005190 2023-11-28T18:24:36.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf9a7fc640 1 -- 172.21.15.105:0/1992481066 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8800c2f0 con 0x7faf88003980 2023-11-28T18:24:36.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf927fc640 1 --2- 172.21.15.105:0/1992481066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf880047d0 0x7faf88004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:36.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf9affd640 1 --2- 172.21.15.105:0/1992481066 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faf88003980 0x7faf88003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:36.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf9b7fe640 1 --2- 172.21.15.105:0/1992481066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf88005190 0x7faf8800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:36.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf927fc640 1 -- 172.21.15.105:0/1992481066 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faf88003980 msgr2=0x7faf88003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:36.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf927fc640 1 --2- 172.21.15.105:0/1992481066 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faf88003980 0x7faf88003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf927fc640 1 -- 172.21.15.105:0/1992481066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf88005190 msgr2=0x7faf8800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:36.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf927fc640 1 --2- 172.21.15.105:0/1992481066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf88005190 0x7faf8800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf9b7fe640 1 -- 172.21.15.105:0/1992481066 reap_dead start 2023-11-28T18:24:36.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.782+0000 7faf927fc640 1 -- 172.21.15.105:0/1992481066 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf88003530 con 0x7faf880047d0 2023-11-28T18:24:36.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.786+0000 7faf927fc640 1 --2- 172.21.15.105:0/1992481066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf880047d0 0x7faf88004c50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7faf84001780 tx=0x7faf8401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e98ea2f78161cb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:36.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.786+0000 7faf9a7fc640 1 -- 172.21.15.105:0/1992481066 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf840052d0 con 0x7faf880047d0 2023-11-28T18:24:36.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.786+0000 7faf9a7fc640 1 -- 172.21.15.105:0/1992481066 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf840a7050 con 0x7faf880047d0 2023-11-28T18:24:36.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.786+0000 7faf9a7fc640 1 -- 172.21.15.105:0/1992481066 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf84003600 con 0x7faf880047d0 2023-11-28T18:24:36.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.786+0000 7fafa189a640 1 -- 172.21.15.105:0/1992481066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf880047d0 msgr2=0x7faf88004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:36.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.786+0000 7fafa189a640 1 --2- 172.21.15.105:0/1992481066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf880047d0 0x7faf88004c50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7faf84001780 tx=0x7faf8401c3b0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.786+0000 7fafa189a640 1 -- 172.21.15.105:0/1992481066 shutdown_connections 2023-11-28T18:24:36.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.786+0000 7fafa189a640 1 --2- 172.21.15.105:0/1992481066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf880047d0 0x7faf88004c50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.786+0000 7fafa189a640 1 -- 172.21.15.105:0/1992481066 >> 172.21.15.105:0/1992481066 conn(0x7faf9c0ceb90 msgr2=0x7faf9c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:36.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.786+0000 7fafa189a640 1 -- 172.21.15.105:0/1992481066 shutdown_connections 2023-11-28T18:24:36.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.786+0000 7fafa189a640 1 -- 172.21.15.105:0/1992481066 wait complete. 2023-11-28T18:24:36.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7fafa189a640 1 Processor -- start 2023-11-28T18:24:36.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7fafa189a640 1 -- start start 2023-11-28T18:24:36.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7fafa189a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf880047d0 0x7faf9c1f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:36.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7fafa189a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf9c0ccd50 0x7faf9c0cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:36.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7fafa189a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faf9c157f10 0x7faf9c158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:36.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7fafa189a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0cd710 con 0x7faf9c0ccd50 2023-11-28T18:24:36.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7fafa189a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c158bd0 con 0x7faf880047d0 2023-11-28T18:24:36.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7fafa189a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c158eb0 con 0x7faf9c157f10 2023-11-28T18:24:36.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7faf927fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf9c0ccd50 0x7faf9c0cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:36.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7faf927fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf9c0ccd50 0x7faf9c0cd1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33596/0 (socket says 172.21.15.105:33596) 2023-11-28T18:24:36.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7faf927fc640 1 -- 172.21.15.105:0/2714330769 learned_addr learned my addr 172.21.15.105:0/2714330769 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:36.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7faf9affd640 1 --2- 172.21.15.105:0/2714330769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf880047d0 0x7faf9c1f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:36.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7faf9b7fe640 1 --2- 172.21.15.105:0/2714330769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faf9c157f10 0x7faf9c158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:36.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7faf927fc640 1 -- 172.21.15.105:0/2714330769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faf9c157f10 msgr2=0x7faf9c158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:36.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7faf927fc640 1 --2- 172.21.15.105:0/2714330769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faf9c157f10 0x7faf9c158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7faf927fc640 1 -- 172.21.15.105:0/2714330769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf880047d0 msgr2=0x7faf9c1f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:36.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7faf927fc640 1 --2- 172.21.15.105:0/2714330769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf880047d0 0x7faf9c1f61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7faf927fc640 1 -- 172.21.15.105:0/2714330769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9c159190 con 0x7faf9c0ccd50 2023-11-28T18:24:36.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7faf9b7fe640 1 --2- 172.21.15.105:0/2714330769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faf9c157f10 0x7faf9c158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:36.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7faf9affd640 1 --2- 172.21.15.105:0/2714330769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf880047d0 0x7faf9c1f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:36.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7faf927fc640 1 --2- 172.21.15.105:0/2714330769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf9c0ccd50 0x7faf9c0cd1d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7faf84005040 tx=0x7faf88005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:36.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7faf98ff9640 1 -- 172.21.15.105:0/2714330769 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf84007180 con 0x7faf9c0ccd50 2023-11-28T18:24:36.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7faf98ff9640 1 -- 172.21.15.105:0/2714330769 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf840072e0 con 0x7faf9c0ccd50 2023-11-28T18:24:36.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.790+0000 7faf98ff9640 1 -- 172.21.15.105:0/2714330769 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf84007650 con 0x7faf9c0ccd50 2023-11-28T18:24:36.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.794+0000 7fafa189a640 1 -- 172.21.15.105:0/2714330769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9c159470 con 0x7faf9c0ccd50 2023-11-28T18:24:36.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.794+0000 7fafa189a640 1 -- 172.21.15.105:0/2714330769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9c1588d0 con 0x7faf9c0ccd50 2023-11-28T18:24:36.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.794+0000 7fafa189a640 1 -- 172.21.15.105:0/2714330769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf9c067570 con 0x7faf9c0ccd50 2023-11-28T18:24:36.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.794+0000 7faf98ff9640 1 -- 172.21.15.105:0/2714330769 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7faf840067e0 con 0x7faf9c0ccd50 2023-11-28T18:24:36.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.794+0000 7faf98ff9640 1 --2- 172.21.15.105:0/2714330769 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faf7c066900 0x7faf7c068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:36.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.794+0000 7faf98ff9640 1 -- 172.21.15.105:0/2714330769 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7faf8411eb80 con 0x7faf9c0ccd50 2023-11-28T18:24:36.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.794+0000 7faf9affd640 1 --2- 172.21.15.105:0/2714330769 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faf7c066900 0x7faf7c068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:36.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.794+0000 7faf9affd640 1 --2- 172.21.15.105:0/2714330769 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faf7c066900 0x7faf7c068dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0501d0 tx=0x7faf8c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:36.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.798+0000 7faf98ff9640 1 -- 172.21.15.105:0/2714330769 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faf840eb9b0 con 0x7faf9c0ccd50 2023-11-28T18:24:36.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.954+0000 7fafa189a640 1 -- 172.21.15.105:0/2714330769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf9c0cf950 con 0x7faf9c0ccd50 2023-11-28T18:24:36.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.954+0000 7faf98ff9640 1 -- 172.21.15.105:0/2714330769 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf8411ee60 con 0x7faf9c0ccd50 2023-11-28T18:24:36.958 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:24:36.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.954+0000 7fafa189a640 1 -- 172.21.15.105:0/2714330769 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faf7c066900 msgr2=0x7faf7c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:36.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.954+0000 7fafa189a640 1 --2- 172.21.15.105:0/2714330769 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faf7c066900 0x7faf7c068dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0501d0 tx=0x7faf8c008040 comp rx=0 tx=0).stop 2023-11-28T18:24:36.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.954+0000 7fafa189a640 1 -- 172.21.15.105:0/2714330769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf9c0ccd50 msgr2=0x7faf9c0cd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:36.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.954+0000 7fafa189a640 1 --2- 172.21.15.105:0/2714330769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf9c0ccd50 0x7faf9c0cd1d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7faf84005040 tx=0x7faf88005800 comp rx=0 tx=0).stop 2023-11-28T18:24:36.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.954+0000 7fafa189a640 1 -- 172.21.15.105:0/2714330769 shutdown_connections 2023-11-28T18:24:36.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.954+0000 7fafa189a640 1 --2- 172.21.15.105:0/2714330769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faf9c157f10 0x7faf9c158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.954+0000 7fafa189a640 1 --2- 172.21.15.105:0/2714330769 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faf7c066900 0x7faf7c068dc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.954+0000 7fafa189a640 1 --2- 172.21.15.105:0/2714330769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf9c0ccd50 0x7faf9c0cd1d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.954+0000 7fafa189a640 1 --2- 172.21.15.105:0/2714330769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf880047d0 0x7faf9c1f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:36.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.954+0000 7fafa189a640 1 -- 172.21.15.105:0/2714330769 >> 172.21.15.105:0/2714330769 conn(0x7faf9c0ceb90 msgr2=0x7faf9c15d760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:36.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.954+0000 7fafa189a640 1 -- 172.21.15.105:0/2714330769 shutdown_connections 2023-11-28T18:24:36.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:36.954+0000 7fafa189a640 1 -- 172.21.15.105:0/2714330769 wait complete. 2023-11-28T18:24:36.982 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:24:37.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec7179d640 1 Processor -- start 2023-11-28T18:24:37.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec7179d640 1 -- start start 2023-11-28T18:24:37.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec7179d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fec6c071960 0x7fec6c15f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:37.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec7179d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec6c15b030 0x7fec6c0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:37.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec7179d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec6c0701d0 con 0x7fec6c0b4290 2023-11-28T18:24:37.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec7179d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec6c06df80 con 0x7fec6c071960 2023-11-28T18:24:37.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec7179d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c071df0 con 0x7fec6c15b030 2023-11-28T18:24:37.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6affd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fec6c0b4290 0x7fec6c0b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35396/0 (socket says 172.21.15.105:35396) 2023-11-28T18:24:37.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6affd640 1 -- 172.21.15.105:0/3148249174 learned_addr learned my addr 172.21.15.105:0/3148249174 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:37.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6b7fe640 1 --2- 172.21.15.105:0/3148249174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec6c15b030 0x7fec6c0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:37.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec63fff640 1 --2- 172.21.15.105:0/3148249174 >> v2:172.21.15.148:3300/0 conn(0x7fec6c071960 0x7fec6c15f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:37.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6a7fc640 1 -- 172.21.15.105:0/3148249174 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1159272012 0 0) 0x7fec6c0701d0 con 0x7fec6c0b4290 2023-11-28T18:24:37.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6a7fc640 1 -- 172.21.15.105:0/3148249174 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec58003530 con 0x7fec6c0b4290 2023-11-28T18:24:37.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6a7fc640 1 -- 172.21.15.105:0/3148249174 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4108120130 0 0) 0x7fec58003530 con 0x7fec6c0b4290 2023-11-28T18:24:37.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6a7fc640 1 -- 172.21.15.105:0/3148249174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec6c15b030 msgr2=0x7fec6c0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:37.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6a7fc640 1 --2- 172.21.15.105:0/3148249174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec6c15b030 0x7fec6c0709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6a7fc640 1 -- 172.21.15.105:0/3148249174 >> v2:172.21.15.148:3300/0 conn(0x7fec6c071960 msgr2=0x7fec6c15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:37.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6a7fc640 1 --2- 172.21.15.105:0/3148249174 >> v2:172.21.15.148:3300/0 conn(0x7fec6c071960 0x7fec6c15f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6a7fc640 1 -- 172.21.15.105:0/3148249174 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c0720d0 con 0x7fec6c0b4290 2023-11-28T18:24:37.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6a7fc640 1 -- 172.21.15.105:0/3148249174 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fec5c001680 con 0x7fec6c0b4290 2023-11-28T18:24:37.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6a7fc640 1 -- 172.21.15.105:0/3148249174 >> v1:172.21.15.105:6789/0 conn(0x7fec6c0b4290 legacy=0x7fec6c0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:37.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6a7fc640 1 --2- 172.21.15.105:0/3148249174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec58003980 0x7fec58003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:37.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6a7fc640 1 --2- 172.21.15.105:0/3148249174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec6c15b030 0x7fec580047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:37.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6a7fc640 1 --2- 172.21.15.105:0/3148249174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec58004d10 0x7fec58005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:37.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6a7fc640 1 -- 172.21.15.105:0/3148249174 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c06df80 con 0x7fec58003980 2023-11-28T18:24:37.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6a7fc640 1 -- 172.21.15.105:0/3148249174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c071df0 con 0x7fec58004d10 2023-11-28T18:24:37.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6a7fc640 1 -- 172.21.15.105:0/3148249174 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec5800b990 con 0x7fec6c15b030 2023-11-28T18:24:37.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6affd640 1 --2- 172.21.15.105:0/3148249174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec58003980 0x7fec58003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:37.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6b7fe640 1 --2- 172.21.15.105:0/3148249174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec58004d10 0x7fec58005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:37.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec63fff640 1 --2- 172.21.15.105:0/3148249174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec6c15b030 0x7fec580047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:37.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6affd640 1 -- 172.21.15.105:0/3148249174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec6c15b030 msgr2=0x7fec580047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:37.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6affd640 1 --2- 172.21.15.105:0/3148249174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec6c15b030 0x7fec580047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6affd640 1 -- 172.21.15.105:0/3148249174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec58004d10 msgr2=0x7fec58005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:37.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.118+0000 7fec6affd640 1 --2- 172.21.15.105:0/3148249174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec58004d10 0x7fec58005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.122+0000 7fec6affd640 1 -- 172.21.15.105:0/3148249174 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec58003530 con 0x7fec58003980 2023-11-28T18:24:37.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.122+0000 7fec63fff640 1 --2- 172.21.15.105:0/3148249174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec6c15b030 0x7fec580047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:37.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.122+0000 7fec6b7fe640 1 --2- 172.21.15.105:0/3148249174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec58004d10 0x7fec58005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:37.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.122+0000 7fec6affd640 1 --2- 172.21.15.105:0/3148249174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec58003980 0x7fec58003da0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00c190 tx=0x7fec5c01cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5f3e2d3c5c6014 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:37.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.122+0000 7fec6a7fc640 1 -- 172.21.15.105:0/3148249174 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0058e0 con 0x7fec58003980 2023-11-28T18:24:37.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.122+0000 7fec6a7fc640 1 -- 172.21.15.105:0/3148249174 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c005a60 con 0x7fec58003980 2023-11-28T18:24:37.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.122+0000 7fec6a7fc640 1 -- 172.21.15.105:0/3148249174 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c005e10 con 0x7fec58003980 2023-11-28T18:24:37.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.122+0000 7fec7179d640 1 -- 172.21.15.105:0/3148249174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec58003980 msgr2=0x7fec58003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:37.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.122+0000 7fec7179d640 1 --2- 172.21.15.105:0/3148249174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec58003980 0x7fec58003da0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00c190 tx=0x7fec5c01cbf0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.122+0000 7fec6b7fe640 1 -- 172.21.15.105:0/3148249174 reap_dead start 2023-11-28T18:24:37.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.122+0000 7fec7179d640 1 -- 172.21.15.105:0/3148249174 shutdown_connections 2023-11-28T18:24:37.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.122+0000 7fec7179d640 1 -- 172.21.15.105:0/3148249174 >> 172.21.15.105:0/3148249174 conn(0x7fec6c0ceb90 msgr2=0x7fec6c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:37.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.122+0000 7fec7179d640 1 -- 172.21.15.105:0/3148249174 shutdown_connections 2023-11-28T18:24:37.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.122+0000 7fec7179d640 1 -- 172.21.15.105:0/3148249174 wait complete. 2023-11-28T18:24:37.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec7179d640 1 Processor -- start 2023-11-28T18:24:37.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec7179d640 1 -- start start 2023-11-28T18:24:37.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec7179d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec58003980 0x7fec6c09c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:37.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec7179d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec6c0b0440 0x7fec6c0b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:37.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec7179d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec6c1412c0 0x7fec6c141750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:37.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec7179d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0b2370 con 0x7fec6c0b0440 2023-11-28T18:24:37.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec7179d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0add50 con 0x7fec6c1412c0 2023-11-28T18:24:37.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec7179d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c09c830 con 0x7fec58003980 2023-11-28T18:24:37.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec63fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec6c0b0440 0x7fec6c0b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:37.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec63fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec6c0b0440 0x7fec6c0b2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33612/0 (socket says 172.21.15.105:33612) 2023-11-28T18:24:37.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec63fff640 1 -- 172.21.15.105:0/1915647151 learned_addr learned my addr 172.21.15.105:0/1915647151 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:37.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec6affd640 1 --2- 172.21.15.105:0/1915647151 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec58003980 0x7fec6c09c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:37.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec6b7fe640 1 --2- 172.21.15.105:0/1915647151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec6c1412c0 0x7fec6c141750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:37.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec63fff640 1 -- 172.21.15.105:0/1915647151 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec58003980 msgr2=0x7fec6c09c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:37.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec63fff640 1 --2- 172.21.15.105:0/1915647151 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec58003980 0x7fec6c09c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec63fff640 1 -- 172.21.15.105:0/1915647151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec6c1412c0 msgr2=0x7fec6c141750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:37.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec63fff640 1 --2- 172.21.15.105:0/1915647151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec6c1412c0 0x7fec6c141750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec63fff640 1 -- 172.21.15.105:0/1915647151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c0ad4b0 con 0x7fec6c0b0440 2023-11-28T18:24:37.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec63fff640 1 --2- 172.21.15.105:0/1915647151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec6c0b0440 0x7fec6c0b2d60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fec54009fd0 tx=0x7fec54011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:37.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec637fe640 1 -- 172.21.15.105:0/1915647151 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec54004220 con 0x7fec6c0b0440 2023-11-28T18:24:37.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec637fe640 1 -- 172.21.15.105:0/1915647151 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec54004380 con 0x7fec6c0b0440 2023-11-28T18:24:37.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec7179d640 1 -- 172.21.15.105:0/1915647151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6c0ad790 con 0x7fec6c0b0440 2023-11-28T18:24:37.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.126+0000 7fec637fe640 1 -- 172.21.15.105:0/1915647151 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5400f9e0 con 0x7fec6c0b0440 2023-11-28T18:24:37.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.130+0000 7fec7179d640 1 -- 172.21.15.105:0/1915647151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6c0b2560 con 0x7fec6c0b0440 2023-11-28T18:24:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.130+0000 7fec7179d640 1 -- 172.21.15.105:0/1915647151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec6c067570 con 0x7fec6c0b0440 2023-11-28T18:24:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.130+0000 7fec637fe640 1 -- 172.21.15.105:0/1915647151 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fec5400fb40 con 0x7fec6c0b0440 2023-11-28T18:24:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.130+0000 7fec637fe640 1 --2- 172.21.15.105:0/1915647151 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec4c0668d0 0x7fec4c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.130+0000 7fec637fe640 1 -- 172.21.15.105:0/1915647151 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fec54084de0 con 0x7fec6c0b0440 2023-11-28T18:24:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.130+0000 7fec6affd640 1 --2- 172.21.15.105:0/1915647151 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec4c0668d0 0x7fec4c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:37.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.130+0000 7fec6affd640 1 --2- 172.21.15.105:0/1915647151 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec4c0668d0 0x7fec4c068d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0aa000 tx=0x7fec5c001940 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:37.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.134+0000 7fec637fe640 1 -- 172.21.15.105:0/1915647151 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fec54051b80 con 0x7fec6c0b0440 2023-11-28T18:24:37.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.286+0000 7fec7179d640 1 -- 172.21.15.105:0/1915647151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fec6c0ce350 con 0x7fec6c0b0440 2023-11-28T18:24:37.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.286+0000 7fec637fe640 1 -- 172.21.15.105:0/1915647151 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fec540220a0 con 0x7fec6c0b0440 2023-11-28T18:24:37.290 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:37.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.286+0000 7fec7179d640 1 -- 172.21.15.105:0/1915647151 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec4c0668d0 msgr2=0x7fec4c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:37.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.286+0000 7fec7179d640 1 --2- 172.21.15.105:0/1915647151 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec4c0668d0 0x7fec4c068d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0aa000 tx=0x7fec5c001940 comp rx=0 tx=0).stop 2023-11-28T18:24:37.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.286+0000 7fec7179d640 1 -- 172.21.15.105:0/1915647151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec6c0b0440 msgr2=0x7fec6c0b2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:37.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.286+0000 7fec7179d640 1 --2- 172.21.15.105:0/1915647151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec6c0b0440 0x7fec6c0b2d60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fec54009fd0 tx=0x7fec54011b30 comp rx=0 tx=0).stop 2023-11-28T18:24:37.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.286+0000 7fec7179d640 1 -- 172.21.15.105:0/1915647151 shutdown_connections 2023-11-28T18:24:37.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.286+0000 7fec7179d640 1 --2- 172.21.15.105:0/1915647151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec6c1412c0 0x7fec6c141750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.286+0000 7fec7179d640 1 --2- 172.21.15.105:0/1915647151 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec4c0668d0 0x7fec4c068d90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.286+0000 7fec7179d640 1 --2- 172.21.15.105:0/1915647151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec6c0b0440 0x7fec6c0b2d60 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.286+0000 7fec7179d640 1 --2- 172.21.15.105:0/1915647151 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec58003980 0x7fec6c09c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.286+0000 7fec7179d640 1 -- 172.21.15.105:0/1915647151 >> 172.21.15.105:0/1915647151 conn(0x7fec6c0ceb90 msgr2=0x7fec6c158630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:37.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.286+0000 7fec7179d640 1 -- 172.21.15.105:0/1915647151 shutdown_connections 2023-11-28T18:24:37.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.286+0000 7fec7179d640 1 -- 172.21.15.105:0/1915647151 wait complete. 2023-11-28T18:24:37.311 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:24:37.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.446+0000 7f1060956640 1 Processor -- start 2023-11-28T18:24:37.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.446+0000 7f1060956640 1 -- start start 2023-11-28T18:24:37.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.450+0000 7f1060956640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f105c159710 0x7f105c06c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:37.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.450+0000 7f1060956640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f105c06d070 con 0x7f105c0cd170 2023-11-28T18:24:37.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.450+0000 7f1060956640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f105c06cab0 con 0x7f105c0ce170 2023-11-28T18:24:37.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.450+0000 7f1060956640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105c15e050 con 0x7f105c159710 2023-11-28T18:24:37.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.450+0000 7f105bfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f105c0cd170 0x7f105c070010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35404/0 (socket says 172.21.15.105:35404) 2023-11-28T18:24:37.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.450+0000 7f105bfff640 1 -- 172.21.15.105:0/2828104251 learned_addr learned my addr 172.21.15.105:0/2828104251 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:37.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.450+0000 7f105b7fe640 1 --2- 172.21.15.105:0/2828104251 >> v2:172.21.15.148:3300/0 conn(0x7f105c159710 0x7f105c06c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:37.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.450+0000 7f105a7fc640 1 -- 172.21.15.105:0/2828104251 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406305622 0 0) 0x7f105c06d070 con 0x7f105c0cd170 2023-11-28T18:24:37.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.450+0000 7f105a7fc640 1 -- 172.21.15.105:0/2828104251 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1044003530 con 0x7f105c0cd170 2023-11-28T18:24:37.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.450+0000 7f105b7fe640 1 -- 172.21.15.105:0/2828104251 >> v1:172.21.15.148:6789/0 conn(0x7f105c0ce170 legacy=0x7f105c15d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:37.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.450+0000 7f105a7fc640 1 -- 172.21.15.105:0/2828104251 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4111989425 0 0) 0x7f105c06cab0 con 0x7f105c0ce170 2023-11-28T18:24:37.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.450+0000 7f105b7fe640 1 -- 172.21.15.105:0/2828104251 >> v1:172.21.15.105:6789/0 conn(0x7f105c0cd170 legacy=0x7f105c070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:37.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.450+0000 7f105b7fe640 1 -- 172.21.15.105:0/2828104251 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c06cd90 con 0x7f105c159710 2023-11-28T18:24:37.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.454+0000 7f105b7fe640 1 --2- 172.21.15.105:0/2828104251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f105c159710 0x7f105c06c070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f104c0097c0 tx=0x7f104c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26858893777d9ff3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:37.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.454+0000 7f105a7fc640 1 -- 172.21.15.105:0/2828104251 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c0ab650 con 0x7f105c159710 2023-11-28T18:24:37.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.454+0000 7f105a7fc640 1 -- 172.21.15.105:0/2828104251 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104c0ab7b0 con 0x7f105c159710 2023-11-28T18:24:37.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.454+0000 7f105a7fc640 1 -- 172.21.15.105:0/2828104251 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c0abb40 con 0x7f105c159710 2023-11-28T18:24:37.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.454+0000 7f1060956640 1 -- 172.21.15.105:0/2828104251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f105c159710 msgr2=0x7f105c06c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:37.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.454+0000 7f1060956640 1 --2- 172.21.15.105:0/2828104251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f105c159710 0x7f105c06c070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f104c0097c0 tx=0x7f104c01c450 comp rx=0 tx=0).stop 2023-11-28T18:24:37.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.454+0000 7f1060956640 1 -- 172.21.15.105:0/2828104251 shutdown_connections 2023-11-28T18:24:37.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.454+0000 7f1060956640 1 --2- 172.21.15.105:0/2828104251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f105c159710 0x7f105c06c070 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.454+0000 7f1060956640 1 -- 172.21.15.105:0/2828104251 >> 172.21.15.105:0/2828104251 conn(0x7f105c0ce9e0 msgr2=0x7f105c0cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:37.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.454+0000 7f1060956640 1 -- 172.21.15.105:0/2828104251 shutdown_connections 2023-11-28T18:24:37.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.454+0000 7f1060956640 1 -- 172.21.15.105:0/2828104251 wait complete. 2023-11-28T18:24:37.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.454+0000 7f1060956640 1 Processor -- start 2023-11-28T18:24:37.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.454+0000 7f1060956640 1 -- start start 2023-11-28T18:24:37.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f1060956640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c0cd170 0x7f105c147580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:37.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f1060956640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f105c0ce170 0x7f105c0c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:37.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f1060956640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f105c159710 0x7f105c070d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:37.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f1060956640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c14ea00 con 0x7f105c0cd170 2023-11-28T18:24:37.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f1060956640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c0c8820 con 0x7f105c0ce170 2023-11-28T18:24:37.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f1060956640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105c147ac0 con 0x7f105c159710 2023-11-28T18:24:37.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f105b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c0cd170 0x7f105c147580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:37.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f105b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c0cd170 0x7f105c147580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33616/0 (socket says 172.21.15.105:33616) 2023-11-28T18:24:37.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f105b7fe640 1 -- 172.21.15.105:0/2672256054 learned_addr learned my addr 172.21.15.105:0/2672256054 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:37.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f105bfff640 1 --2- 172.21.15.105:0/2672256054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f105c159710 0x7f105c070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:37.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f105affd640 1 --2- 172.21.15.105:0/2672256054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f105c0ce170 0x7f105c0c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:37.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f105b7fe640 1 -- 172.21.15.105:0/2672256054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f105c159710 msgr2=0x7f105c070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:37.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f105b7fe640 1 --2- 172.21.15.105:0/2672256054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f105c159710 0x7f105c070d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f105b7fe640 1 -- 172.21.15.105:0/2672256054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f105c0ce170 msgr2=0x7f105c0c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:37.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f105b7fe640 1 --2- 172.21.15.105:0/2672256054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f105c0ce170 0x7f105c0c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f105b7fe640 1 -- 172.21.15.105:0/2672256054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104c009420 con 0x7f105c0cd170 2023-11-28T18:24:37.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f105bfff640 1 --2- 172.21.15.105:0/2672256054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f105c159710 0x7f105c070d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:37.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f105affd640 1 --2- 172.21.15.105:0/2672256054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f105c0ce170 0x7f105c0c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:37.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f105b7fe640 1 --2- 172.21.15.105:0/2672256054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c0cd170 0x7f105c147580 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f104c0097c0 tx=0x7f104c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:37.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f1043fff640 1 -- 172.21.15.105:0/2672256054 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c0bf610 con 0x7f105c0cd170 2023-11-28T18:24:37.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f1043fff640 1 -- 172.21.15.105:0/2672256054 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104c0bf770 con 0x7f105c0cd170 2023-11-28T18:24:37.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f1060956640 1 -- 172.21.15.105:0/2672256054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105c0cdb10 con 0x7f105c0cd170 2023-11-28T18:24:37.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f1043fff640 1 -- 172.21.15.105:0/2672256054 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c0bfb00 con 0x7f105c0cd170 2023-11-28T18:24:37.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f1060956640 1 -- 172.21.15.105:0/2672256054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105c0c8540 con 0x7f105c0cd170 2023-11-28T18:24:37.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f1060956640 1 -- 172.21.15.105:0/2672256054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105c06c550 con 0x7f105c0cd170 2023-11-28T18:24:37.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.458+0000 7f1043fff640 1 -- 172.21.15.105:0/2672256054 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f104c0b5400 con 0x7f105c0cd170 2023-11-28T18:24:37.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.462+0000 7f1043fff640 1 --2- 172.21.15.105:0/2672256054 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10480668d0 0x7f1048068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:37.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.462+0000 7f1043fff640 1 -- 172.21.15.105:0/2672256054 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f104c11f430 con 0x7f105c0cd170 2023-11-28T18:24:37.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.462+0000 7f105affd640 1 --2- 172.21.15.105:0/2672256054 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10480668d0 0x7f1048068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:37.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.462+0000 7f105affd640 1 --2- 172.21.15.105:0/2672256054 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10480668d0 0x7f1048068d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f105c073850 tx=0x7f1050009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:37.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.466+0000 7f1043fff640 1 -- 172.21.15.105:0/2672256054 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f104c0ec260 con 0x7f105c0cd170 2023-11-28T18:24:37.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.614+0000 7f1060956640 1 -- 172.21.15.105:0/2672256054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f105c074580 con 0x7f105c0cd170 2023-11-28T18:24:37.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.614+0000 7f1043fff640 1 -- 172.21.15.105:0/2672256054 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f104c0ef880 con 0x7f105c0cd170 2023-11-28T18:24:37.621 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:24:37.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.618+0000 7f1060956640 1 -- 172.21.15.105:0/2672256054 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10480668d0 msgr2=0x7f1048068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:37.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.618+0000 7f1060956640 1 --2- 172.21.15.105:0/2672256054 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10480668d0 0x7f1048068d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f105c073850 tx=0x7f1050009040 comp rx=0 tx=0).stop 2023-11-28T18:24:37.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.618+0000 7f1060956640 1 -- 172.21.15.105:0/2672256054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c0cd170 msgr2=0x7f105c147580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:37.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.618+0000 7f1060956640 1 --2- 172.21.15.105:0/2672256054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c0cd170 0x7f105c147580 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f104c0097c0 tx=0x7f104c01ac90 comp rx=0 tx=0).stop 2023-11-28T18:24:37.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.618+0000 7f1060956640 1 -- 172.21.15.105:0/2672256054 shutdown_connections 2023-11-28T18:24:37.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.618+0000 7f1060956640 1 --2- 172.21.15.105:0/2672256054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f105c159710 0x7f105c070d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.618+0000 7f1060956640 1 --2- 172.21.15.105:0/2672256054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f105c0ce170 0x7f105c0c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.618+0000 7f1060956640 1 --2- 172.21.15.105:0/2672256054 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10480668d0 0x7f1048068d90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.618+0000 7f1060956640 1 --2- 172.21.15.105:0/2672256054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c0cd170 0x7f105c147580 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.618+0000 7f1060956640 1 -- 172.21.15.105:0/2672256054 >> 172.21.15.105:0/2672256054 conn(0x7f105c0ce9e0 msgr2=0x7f105c0ca4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:37.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.618+0000 7f1060956640 1 -- 172.21.15.105:0/2672256054 shutdown_connections 2023-11-28T18:24:37.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.618+0000 7f1060956640 1 -- 172.21.15.105:0/2672256054 wait complete. 2023-11-28T18:24:37.642 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T18:24:37.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.778+0000 7fc2bd09c640 1 Processor -- start 2023-11-28T18:24:37.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.778+0000 7fc2bd09c640 1 -- start start 2023-11-28T18:24:37.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.778+0000 7fc2bd09c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc2b815d4f0 0x7fc2b80d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:37.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.778+0000 7fc2bd09c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2b80d3c30 0x7fc2b80d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:37.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.778+0000 7fc2bd09c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b80b25b0 con 0x7fc2b80d1890 2023-11-28T18:24:37.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.778+0000 7fc2bd09c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b80b2900 con 0x7fc2b815d4f0 2023-11-28T18:24:37.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.778+0000 7fc2bd09c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b80cea30 con 0x7fc2b80d3c30 2023-11-28T18:24:37.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.778+0000 7fc2b759e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc2b80d1890 0x7fc2b80cfa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35412/0 (socket says 172.21.15.105:35412) 2023-11-28T18:24:37.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.778+0000 7fc2b759e640 1 -- 172.21.15.105:0/704309908 learned_addr learned my addr 172.21.15.105:0/704309908 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:37.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.778+0000 7fc2b659c640 1 --2- 172.21.15.105:0/704309908 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2b80d3c30 0x7fc2b80d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:37.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.778+0000 7fc2b6d9d640 1 --2- 172.21.15.105:0/704309908 >> v2:172.21.15.148:3300/0 conn(0x7fc2b815d4f0 0x7fc2b80d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:37.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.778+0000 7fc2b5d9b640 1 -- 172.21.15.105:0/704309908 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 36384507 0 0) 0x7fc2b80b25b0 con 0x7fc2b80d1890 2023-11-28T18:24:37.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.778+0000 7fc2b5d9b640 1 -- 172.21.15.105:0/704309908 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2a0003530 con 0x7fc2b80d1890 2023-11-28T18:24:37.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.778+0000 7fc2b659c640 1 -- 172.21.15.105:0/704309908 >> v2:172.21.15.148:3300/0 conn(0x7fc2b815d4f0 msgr2=0x7fc2b80d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:37.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.778+0000 7fc2b659c640 1 --2- 172.21.15.105:0/704309908 >> v2:172.21.15.148:3300/0 conn(0x7fc2b815d4f0 0x7fc2b80d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.778+0000 7fc2b659c640 1 -- 172.21.15.105:0/704309908 >> v1:172.21.15.105:6789/0 conn(0x7fc2b80d1890 legacy=0x7fc2b80cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:37.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.778+0000 7fc2b659c640 1 -- 172.21.15.105:0/704309908 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b80ced10 con 0x7fc2b80d3c30 2023-11-28T18:24:37.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.782+0000 7fc2b659c640 1 --2- 172.21.15.105:0/704309908 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2b80d3c30 0x7fc2b80d1350 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0097c0 tx=0x7fc2ac01c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1b915c8d5a98eb7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:37.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.782+0000 7fc2b5d9b640 1 -- 172.21.15.105:0/704309908 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac0ab600 con 0x7fc2b80d3c30 2023-11-28T18:24:37.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.782+0000 7fc2b5d9b640 1 -- 172.21.15.105:0/704309908 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ac0a7050 con 0x7fc2b80d3c30 2023-11-28T18:24:37.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.782+0000 7fc2b5d9b640 1 -- 172.21.15.105:0/704309908 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac0ab960 con 0x7fc2b80d3c30 2023-11-28T18:24:37.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.782+0000 7fc2bd09c640 1 -- 172.21.15.105:0/704309908 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2b80d3c30 msgr2=0x7fc2b80d1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:37.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.782+0000 7fc2bd09c640 1 --2- 172.21.15.105:0/704309908 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2b80d3c30 0x7fc2b80d1350 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0097c0 tx=0x7fc2ac01c3c0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.782+0000 7fc2bd09c640 1 -- 172.21.15.105:0/704309908 shutdown_connections 2023-11-28T18:24:37.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.782+0000 7fc2bd09c640 1 --2- 172.21.15.105:0/704309908 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2b80d3c30 0x7fc2b80d1350 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.782+0000 7fc2bd09c640 1 --2- 172.21.15.105:0/704309908 >> v2:172.21.15.148:3300/0 conn(0x7fc2b815d4f0 0x7fc2b80d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.782+0000 7fc2bd09c640 1 -- 172.21.15.105:0/704309908 >> 172.21.15.105:0/704309908 conn(0x7fc2b80d3030 msgr2=0x7fc2b80d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:37.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.782+0000 7fc2bd09c640 1 -- 172.21.15.105:0/704309908 shutdown_connections 2023-11-28T18:24:37.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.782+0000 7fc2bd09c640 1 -- 172.21.15.105:0/704309908 wait complete. 2023-11-28T18:24:37.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.786+0000 7fc2bd09c640 1 Processor -- start 2023-11-28T18:24:37.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.786+0000 7fc2bd09c640 1 -- start start 2023-11-28T18:24:37.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.786+0000 7fc2bd09c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2b80d1890 0x7fc2b815b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:37.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.786+0000 7fc2bd09c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b80d3c30 0x7fc2b80b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:37.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.786+0000 7fc2bd09c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2b815d4f0 0x7fc2b814f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:37.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.786+0000 7fc2bd09c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b815cb50 con 0x7fc2b80d3c30 2023-11-28T18:24:37.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.786+0000 7fc2bd09c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b815bfd0 con 0x7fc2b815d4f0 2023-11-28T18:24:37.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.786+0000 7fc2bd09c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b815c2b0 con 0x7fc2b80d1890 2023-11-28T18:24:37.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.786+0000 7fc2b659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b80d3c30 0x7fc2b80b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:37.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.786+0000 7fc2b659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b80d3c30 0x7fc2b80b19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33630/0 (socket says 172.21.15.105:33630) 2023-11-28T18:24:37.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.786+0000 7fc2b659c640 1 -- 172.21.15.105:0/3490550809 learned_addr learned my addr 172.21.15.105:0/3490550809 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:37.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.786+0000 7fc2b759e640 1 --2- 172.21.15.105:0/3490550809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2b815d4f0 0x7fc2b814f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:37.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.786+0000 7fc2b6d9d640 1 --2- 172.21.15.105:0/3490550809 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2b80d1890 0x7fc2b815b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:37.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.786+0000 7fc2b659c640 1 -- 172.21.15.105:0/3490550809 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2b80d1890 msgr2=0x7fc2b815b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:37.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.786+0000 7fc2b659c640 1 --2- 172.21.15.105:0/3490550809 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2b80d1890 0x7fc2b815b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.786+0000 7fc2b659c640 1 -- 172.21.15.105:0/3490550809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2b815d4f0 msgr2=0x7fc2b814f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:37.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.786+0000 7fc2b659c640 1 --2- 172.21.15.105:0/3490550809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2b815d4f0 0x7fc2b814f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.786+0000 7fc2b659c640 1 -- 172.21.15.105:0/3490550809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ac009420 con 0x7fc2b80d3c30 2023-11-28T18:24:37.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.786+0000 7fc2b659c640 1 --2- 172.21.15.105:0/3490550809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b80d3c30 0x7fc2b80b19a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac019ff0 tx=0x7fc2ac019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:37.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.790+0000 7fc29f7fe640 1 -- 172.21.15.105:0/3490550809 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac003a30 con 0x7fc2b80d3c30 2023-11-28T18:24:37.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.790+0000 7fc29f7fe640 1 -- 172.21.15.105:0/3490550809 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ac003b90 con 0x7fc2b80d3c30 2023-11-28T18:24:37.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.790+0000 7fc29f7fe640 1 -- 172.21.15.105:0/3490550809 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac0b5410 con 0x7fc2b80d3c30 2023-11-28T18:24:37.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.790+0000 7fc2bd09c640 1 -- 172.21.15.105:0/3490550809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b815c590 con 0x7fc2b80d3c30 2023-11-28T18:24:37.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.790+0000 7fc2bd09c640 1 -- 172.21.15.105:0/3490550809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b815bc50 con 0x7fc2b80d3c30 2023-11-28T18:24:37.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.790+0000 7fc2bd09c640 1 -- 172.21.15.105:0/3490550809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b8067570 con 0x7fc2b80d3c30 2023-11-28T18:24:37.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.790+0000 7fc29f7fe640 1 -- 172.21.15.105:0/3490550809 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc2ac0b2050 con 0x7fc2b80d3c30 2023-11-28T18:24:37.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.790+0000 7fc29f7fe640 1 --2- 172.21.15.105:0/3490550809 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc2a40668d0 0x7fc2a4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:37.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.790+0000 7fc29f7fe640 1 -- 172.21.15.105:0/3490550809 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fc2ac11eb20 con 0x7fc2b80d3c30 2023-11-28T18:24:37.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.794+0000 7fc2b6d9d640 1 --2- 172.21.15.105:0/3490550809 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc2a40668d0 0x7fc2a4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:37.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.794+0000 7fc2b6d9d640 1 --2- 172.21.15.105:0/3490550809 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc2a40668d0 0x7fc2a4068d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc2b808d7a0 tx=0x7fc2a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:37.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.794+0000 7fc29f7fe640 1 -- 172.21.15.105:0/3490550809 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc2ac0eb950 con 0x7fc2b80d3c30 2023-11-28T18:24:37.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.950+0000 7fc2bd09c640 1 -- 172.21.15.105:0/3490550809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc2b80b0f40 con 0x7fc2b80d3c30 2023-11-28T18:24:37.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.950+0000 7fc29f7fe640 1 -- 172.21.15.105:0/3490550809 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc2ac0eef70 con 0x7fc2b80d3c30 2023-11-28T18:24:37.955 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:37.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.950+0000 7fc2bd09c640 1 -- 172.21.15.105:0/3490550809 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc2a40668d0 msgr2=0x7fc2a4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:37.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.950+0000 7fc2bd09c640 1 --2- 172.21.15.105:0/3490550809 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc2a40668d0 0x7fc2a4068d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc2b808d7a0 tx=0x7fc2a8008040 comp rx=0 tx=0).stop 2023-11-28T18:24:37.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.950+0000 7fc2bd09c640 1 -- 172.21.15.105:0/3490550809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b80d3c30 msgr2=0x7fc2b80b19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:37.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.950+0000 7fc2bd09c640 1 --2- 172.21.15.105:0/3490550809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b80d3c30 0x7fc2b80b19a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac019ff0 tx=0x7fc2ac019160 comp rx=0 tx=0).stop 2023-11-28T18:24:37.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.950+0000 7fc2bd09c640 1 -- 172.21.15.105:0/3490550809 shutdown_connections 2023-11-28T18:24:37.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.950+0000 7fc2bd09c640 1 --2- 172.21.15.105:0/3490550809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2b815d4f0 0x7fc2b814f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.950+0000 7fc2bd09c640 1 --2- 172.21.15.105:0/3490550809 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc2a40668d0 0x7fc2a4068d90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.950+0000 7fc2bd09c640 1 --2- 172.21.15.105:0/3490550809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b80d3c30 0x7fc2b80b19a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.950+0000 7fc2bd09c640 1 --2- 172.21.15.105:0/3490550809 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2b80d1890 0x7fc2b815b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:37.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.950+0000 7fc2bd09c640 1 -- 172.21.15.105:0/3490550809 >> 172.21.15.105:0/3490550809 conn(0x7fc2b80d3030 msgr2=0x7fc2b81563e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:37.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.954+0000 7fc2bd09c640 1 -- 172.21.15.105:0/3490550809 shutdown_connections 2023-11-28T18:24:37.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:37.954+0000 7fc2bd09c640 1 -- 172.21.15.105:0/3490550809 wait complete. 2023-11-28T18:24:37.977 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T18:24:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.110+0000 7f1577c17640 1 Processor -- start 2023-11-28T18:24:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.110+0000 7f1577c17640 1 -- start start 2023-11-28T18:24:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.110+0000 7f1577c17640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f15700ad9f0 0x7f15700ade80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.110+0000 7f1577c17640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f15700cc7d0 0x7f15700ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.110+0000 7f1577c17640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1570162820 con 0x7f157006d610 2023-11-28T18:24:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.110+0000 7f1577c17640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15700ae3c0 con 0x7f15700cc7d0 2023-11-28T18:24:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.110+0000 7f1577c17640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1570161c10 con 0x7f15700ad9f0 2023-11-28T18:24:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.110+0000 7f15761b4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f15700cc7d0 0x7f15700ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.110+0000 7f15761b4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f15700cc7d0 0x7f15700ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33632/0 (socket says 172.21.15.105:33632) 2023-11-28T18:24:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.110+0000 7f15761b4640 1 -- 172.21.15.105:0/2947665966 learned_addr learned my addr 172.21.15.105:0/2947665966 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.110+0000 7f15751b2640 1 --2- 172.21.15.105:0/2947665966 >> v2:172.21.15.148:3300/0 conn(0x7f15700ad9f0 0x7f15700ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.114+0000 7f15761b4640 1 -- 172.21.15.105:0/2947665966 >> v2:172.21.15.148:3300/0 conn(0x7f15700ad9f0 msgr2=0x7f15700ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.114+0000 7f15761b4640 1 --2- 172.21.15.105:0/2947665966 >> v2:172.21.15.148:3300/0 conn(0x7f15700ad9f0 0x7f15700ade80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.114+0000 7f15761b4640 1 -- 172.21.15.105:0/2947665966 >> v1:172.21.15.148:6789/0 conn(0x7f157006d610 legacy=0x7f15700cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.114+0000 7f15761b4640 1 -- 172.21.15.105:0/2947665966 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1570161ef0 con 0x7f15700cc7d0 2023-11-28T18:24:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.114+0000 7f15761b4640 1 --2- 172.21.15.105:0/2947665966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15700cc7d0 0x7f15700ccbe0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1570073a50 tx=0x7f156801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cafb20eb328a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.114+0000 7f15749b1640 1 -- 172.21.15.105:0/2947665966 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15680a5760 con 0x7f15700cc7d0 2023-11-28T18:24:38.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.114+0000 7f15749b1640 1 -- 172.21.15.105:0/2947665966 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15680a58c0 con 0x7f15700cc7d0 2023-11-28T18:24:38.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.114+0000 7f15749b1640 1 -- 172.21.15.105:0/2947665966 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15680a5c50 con 0x7f15700cc7d0 2023-11-28T18:24:38.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.114+0000 7f1577c17640 1 -- 172.21.15.105:0/2947665966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15700cc7d0 msgr2=0x7f15700ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.114+0000 7f1577c17640 1 --2- 172.21.15.105:0/2947665966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15700cc7d0 0x7f15700ccbe0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1570073a50 tx=0x7f156801c9a0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f1577c17640 1 -- 172.21.15.105:0/2947665966 shutdown_connections 2023-11-28T18:24:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f1577c17640 1 --2- 172.21.15.105:0/2947665966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15700cc7d0 0x7f15700ccbe0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f1577c17640 1 --2- 172.21.15.105:0/2947665966 >> v2:172.21.15.148:3300/0 conn(0x7f15700ad9f0 0x7f15700ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f1577c17640 1 -- 172.21.15.105:0/2947665966 >> 172.21.15.105:0/2947665966 conn(0x7f15700ceb90 msgr2=0x7f15700cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f1577c17640 1 -- 172.21.15.105:0/2947665966 shutdown_connections 2023-11-28T18:24:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f1577c17640 1 -- 172.21.15.105:0/2947665966 wait complete. 2023-11-28T18:24:38.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f1577c17640 1 Processor -- start 2023-11-28T18:24:38.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f1577c17640 1 -- start start 2023-11-28T18:24:38.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f1577c17640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f157006d610 0x7f1570155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:38.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f1577c17640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15700ad9f0 0x7f1570160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:38.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f1577c17640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15700cc7d0 0x7f1570153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:38.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f1577c17640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1570160b90 con 0x7f15700ad9f0 2023-11-28T18:24:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f1577c17640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15701544d0 con 0x7f15700cc7d0 2023-11-28T18:24:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f1577c17640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15701547b0 con 0x7f157006d610 2023-11-28T18:24:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f15751b2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15700ad9f0 0x7f1570160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f15751b2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15700ad9f0 0x7f1570160590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33646/0 (socket says 172.21.15.105:33646) 2023-11-28T18:24:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f15751b2640 1 -- 172.21.15.105:0/3593871018 learned_addr learned my addr 172.21.15.105:0/3593871018 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f15759b3640 1 --2- 172.21.15.105:0/3593871018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f157006d610 0x7f1570155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f15761b4640 1 --2- 172.21.15.105:0/3593871018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15700cc7d0 0x7f1570153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f15751b2640 1 -- 172.21.15.105:0/3593871018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f157006d610 msgr2=0x7f1570155000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f15751b2640 1 --2- 172.21.15.105:0/3593871018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f157006d610 0x7f1570155000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f15751b2640 1 -- 172.21.15.105:0/3593871018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15700cc7d0 msgr2=0x7f1570153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f15751b2640 1 --2- 172.21.15.105:0/3593871018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15700cc7d0 0x7f1570153440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f15751b2640 1 -- 172.21.15.105:0/3593871018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1568009b30 con 0x7f15700ad9f0 2023-11-28T18:24:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f15759b3640 1 --2- 172.21.15.105:0/3593871018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f157006d610 0x7f1570155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.118+0000 7f15751b2640 1 --2- 172.21.15.105:0/3593871018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15700ad9f0 0x7f1570160590 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f156000d800 tx=0x7f156000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.122+0000 7f15667fc640 1 -- 172.21.15.105:0/3593871018 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156000b6c0 con 0x7f15700ad9f0 2023-11-28T18:24:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.122+0000 7f15667fc640 1 -- 172.21.15.105:0/3593871018 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156000b860 con 0x7f15700ad9f0 2023-11-28T18:24:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.122+0000 7f15667fc640 1 -- 172.21.15.105:0/3593871018 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156000bc10 con 0x7f15700ad9f0 2023-11-28T18:24:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.122+0000 7f1577c17640 1 -- 172.21.15.105:0/3593871018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1570154a90 con 0x7f15700ad9f0 2023-11-28T18:24:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.122+0000 7f1577c17640 1 -- 172.21.15.105:0/3593871018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15700cb260 con 0x7f15700ad9f0 2023-11-28T18:24:38.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.122+0000 7f1577c17640 1 -- 172.21.15.105:0/3593871018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1570067570 con 0x7f15700ad9f0 2023-11-28T18:24:38.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.122+0000 7f15667fc640 1 -- 172.21.15.105:0/3593871018 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f156001e460 con 0x7f15700ad9f0 2023-11-28T18:24:38.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.122+0000 7f15667fc640 1 --2- 172.21.15.105:0/3593871018 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15540668d0 0x7f1554068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:38.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.122+0000 7f15667fc640 1 -- 172.21.15.105:0/3593871018 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f1560084f40 con 0x7f15700ad9f0 2023-11-28T18:24:38.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.122+0000 7f15759b3640 1 --2- 172.21.15.105:0/3593871018 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15540668d0 0x7f1554068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:38.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.122+0000 7f15759b3640 1 --2- 172.21.15.105:0/3593871018 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15540668d0 0x7f1554068d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f15701fd8f0 tx=0x7f156c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:38.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.126+0000 7f15667fc640 1 -- 172.21.15.105:0/3593871018 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1560051ce0 con 0x7f15700ad9f0 2023-11-28T18:24:38.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.290+0000 7f1577c17640 1 -- 172.21.15.105:0/3593871018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f15700cf950 con 0x7f15700ad9f0 2023-11-28T18:24:38.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.290+0000 7f15667fc640 1 -- 172.21.15.105:0/3593871018 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1560055300 con 0x7f15700ad9f0 2023-11-28T18:24:38.295 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2023-11-28T18:24:38.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.290+0000 7f1577c17640 1 -- 172.21.15.105:0/3593871018 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15540668d0 msgr2=0x7f1554068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:38.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.290+0000 7f1577c17640 1 --2- 172.21.15.105:0/3593871018 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15540668d0 0x7f1554068d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f15701fd8f0 tx=0x7f156c009040 comp rx=0 tx=0).stop 2023-11-28T18:24:38.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.290+0000 7f1577c17640 1 -- 172.21.15.105:0/3593871018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15700ad9f0 msgr2=0x7f1570160590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:38.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.290+0000 7f1577c17640 1 --2- 172.21.15.105:0/3593871018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15700ad9f0 0x7f1570160590 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f156000d800 tx=0x7f156000dcc0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.290+0000 7f1577c17640 1 -- 172.21.15.105:0/3593871018 shutdown_connections 2023-11-28T18:24:38.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.290+0000 7f1577c17640 1 --2- 172.21.15.105:0/3593871018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15700cc7d0 0x7f1570153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.290+0000 7f1577c17640 1 --2- 172.21.15.105:0/3593871018 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15540668d0 0x7f1554068d90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.290+0000 7f1577c17640 1 --2- 172.21.15.105:0/3593871018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15700ad9f0 0x7f1570160590 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.290+0000 7f1577c17640 1 --2- 172.21.15.105:0/3593871018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f157006d610 0x7f1570155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.290+0000 7f1577c17640 1 -- 172.21.15.105:0/3593871018 >> 172.21.15.105:0/3593871018 conn(0x7f15700ceb90 msgr2=0x7f1570071af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:38.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.290+0000 7f1577c17640 1 -- 172.21.15.105:0/3593871018 shutdown_connections 2023-11-28T18:24:38.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.290+0000 7f1577c17640 1 -- 172.21.15.105:0/3593871018 wait complete. 2023-11-28T18:24:38.316 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:24:38.316 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:24:38.327 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:24:38.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.458+0000 7f20b811d640 1 Processor -- start 2023-11-28T18:24:38.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.458+0000 7f20b811d640 1 -- start start 2023-11-28T18:24:38.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.458+0000 7f20b811d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20b0159f40 0x7f20b00caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:38.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.458+0000 7f20b811d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f20b00cc760 0x7f20b00ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:38.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.462+0000 7f20b811d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b006bf50 con 0x7f20b00ad680 2023-11-28T18:24:38.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.462+0000 7f20b811d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b00ccbc0 con 0x7f20b00cc760 2023-11-28T18:24:38.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.462+0000 7f20b811d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b015a320 con 0x7f20b0159f40 2023-11-28T18:24:38.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.462+0000 7f20b56b8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f20b00cc760 0x7f20b00ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:38.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.462+0000 7f20b66ba640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f20b00ad680 0x7f20b015da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:40194/0 (socket says 172.21.15.105:40194) 2023-11-28T18:24:38.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.462+0000 7f20b66ba640 1 -- 172.21.15.105:0/3828066051 learned_addr learned my addr 172.21.15.105:0/3828066051 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:38.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.462+0000 7f20b5eb9640 1 --2- 172.21.15.105:0/3828066051 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20b0159f40 0x7f20b00caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:38.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.462+0000 7f20b56b8640 1 -- 172.21.15.105:0/3828066051 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20b0159f40 msgr2=0x7f20b00caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:38.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.462+0000 7f20b56b8640 1 --2- 172.21.15.105:0/3828066051 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20b0159f40 0x7f20b00caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.462+0000 7f20b56b8640 1 -- 172.21.15.105:0/3828066051 >> v1:172.21.15.148:6789/0 conn(0x7f20b00ad680 legacy=0x7f20b015da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:38.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.462+0000 7f20b56b8640 1 -- 172.21.15.105:0/3828066051 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b0071380 con 0x7f20b00cc760 2023-11-28T18:24:38.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.466+0000 7f20b56b8640 1 --2- 172.21.15.105:0/3828066051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20b00cc760 0x7f20b00ad140 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f20a00097c0 tx=0x7f20a001c280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0b04990a89ce749 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:38.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.466+0000 7f20b4eb7640 1 -- 172.21.15.105:0/3828066051 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a00ab600 con 0x7f20b00cc760 2023-11-28T18:24:38.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.466+0000 7f20b4eb7640 1 -- 172.21.15.105:0/3828066051 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a00a7050 con 0x7f20b00cc760 2023-11-28T18:24:38.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.466+0000 7f20b4eb7640 1 -- 172.21.15.105:0/3828066051 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a00ab980 con 0x7f20b00cc760 2023-11-28T18:24:38.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.466+0000 7f20b811d640 1 -- 172.21.15.105:0/3828066051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20b00cc760 msgr2=0x7f20b00ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:38.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.466+0000 7f20b811d640 1 --2- 172.21.15.105:0/3828066051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20b00cc760 0x7f20b00ad140 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f20a00097c0 tx=0x7f20a001c280 comp rx=0 tx=0).stop 2023-11-28T18:24:38.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.466+0000 7f20b811d640 1 -- 172.21.15.105:0/3828066051 shutdown_connections 2023-11-28T18:24:38.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.466+0000 7f20b811d640 1 --2- 172.21.15.105:0/3828066051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20b00cc760 0x7f20b00ad140 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.466+0000 7f20b811d640 1 --2- 172.21.15.105:0/3828066051 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20b0159f40 0x7f20b00caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.466+0000 7f20b811d640 1 -- 172.21.15.105:0/3828066051 >> 172.21.15.105:0/3828066051 conn(0x7f20b00ceb90 msgr2=0x7f20b00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:38.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.466+0000 7f20b811d640 1 -- 172.21.15.105:0/3828066051 shutdown_connections 2023-11-28T18:24:38.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.466+0000 7f20b811d640 1 -- 172.21.15.105:0/3828066051 wait complete. 2023-11-28T18:24:38.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.466+0000 7f20b811d640 1 Processor -- start 2023-11-28T18:24:38.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.466+0000 7f20b811d640 1 -- start start 2023-11-28T18:24:38.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.466+0000 7f20b811d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20b00ad680 0x7f20b0153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:38.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.466+0000 7f20b811d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20b00cc760 0x7f20b00b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:38.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.466+0000 7f20b811d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20b0159f40 0x7f20b00b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:38.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.466+0000 7f20b811d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b00b3210 con 0x7f20b00cc760 2023-11-28T18:24:38.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.466+0000 7f20b811d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b0150e10 con 0x7f20b0159f40 2023-11-28T18:24:38.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.466+0000 7f20b811d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b00b2590 con 0x7f20b00ad680 2023-11-28T18:24:38.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.470+0000 7f20b56b8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20b00cc760 0x7f20b00b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:38.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.470+0000 7f20b56b8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20b00cc760 0x7f20b00b19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33654/0 (socket says 172.21.15.105:33654) 2023-11-28T18:24:38.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.470+0000 7f20b56b8640 1 -- 172.21.15.105:0/3254819995 learned_addr learned my addr 172.21.15.105:0/3254819995 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:38.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.470+0000 7f20b5eb9640 1 --2- 172.21.15.105:0/3254819995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20b00ad680 0x7f20b0153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:38.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.470+0000 7f20b66ba640 1 --2- 172.21.15.105:0/3254819995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20b0159f40 0x7f20b00b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:38.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.470+0000 7f20b56b8640 1 -- 172.21.15.105:0/3254819995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20b00ad680 msgr2=0x7f20b0153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:38.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.470+0000 7f20b56b8640 1 --2- 172.21.15.105:0/3254819995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20b00ad680 0x7f20b0153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.470+0000 7f20b56b8640 1 -- 172.21.15.105:0/3254819995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20b0159f40 msgr2=0x7f20b00b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:38.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.470+0000 7f20b56b8640 1 --2- 172.21.15.105:0/3254819995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20b0159f40 0x7f20b00b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.470+0000 7f20b56b8640 1 -- 172.21.15.105:0/3254819995 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a0009420 con 0x7f20b00cc760 2023-11-28T18:24:38.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.470+0000 7f20b66ba640 1 --2- 172.21.15.105:0/3254819995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20b0159f40 0x7f20b00b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:38.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.470+0000 7f20b56b8640 1 --2- 172.21.15.105:0/3254819995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20b00cc760 0x7f20b00b19b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f20a00ae040 tx=0x7f20a0019020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:38.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.470+0000 7f20a67fc640 1 -- 172.21.15.105:0/3254819995 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a00b4820 con 0x7f20b00cc760 2023-11-28T18:24:38.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.470+0000 7f20a67fc640 1 -- 172.21.15.105:0/3254819995 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a00b4980 con 0x7f20b00cc760 2023-11-28T18:24:38.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.470+0000 7f20a67fc640 1 -- 172.21.15.105:0/3254819995 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a00b4d30 con 0x7f20b00cc760 2023-11-28T18:24:38.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.470+0000 7f20b811d640 1 -- 172.21.15.105:0/3254819995 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b00b2870 con 0x7f20b00cc760 2023-11-28T18:24:38.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.470+0000 7f20b811d640 1 -- 172.21.15.105:0/3254819995 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b00b2e30 con 0x7f20b00cc760 2023-11-28T18:24:38.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.470+0000 7f20b811d640 1 -- 172.21.15.105:0/3254819995 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b0067570 con 0x7f20b00cc760 2023-11-28T18:24:38.479 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.470+0000 7f20a67fc640 1 -- 172.21.15.105:0/3254819995 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f20a00b2040 con 0x7f20b00cc760 2023-11-28T18:24:38.479 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.474+0000 7f20a67fc640 1 --2- 172.21.15.105:0/3254819995 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f20940668d0 0x7f2094068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:38.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.474+0000 7f20a67fc640 1 -- 172.21.15.105:0/3254819995 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f20a011ee30 con 0x7f20b00cc760 2023-11-28T18:24:38.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.474+0000 7f20b5eb9640 1 --2- 172.21.15.105:0/3254819995 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f20940668d0 0x7f2094068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:38.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.474+0000 7f20b5eb9640 1 --2- 172.21.15.105:0/3254819995 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f20940668d0 0x7f2094068d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f20b00731a0 tx=0x7f20ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:38.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.478+0000 7f20a67fc640 1 -- 172.21.15.105:0/3254819995 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f20a00ebc60 con 0x7f20b00cc760 2023-11-28T18:24:38.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.626+0000 7f20b811d640 1 -- 172.21.15.105:0/3254819995 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20b00cf950 con 0x7f20b00cc760 2023-11-28T18:24:38.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.630+0000 7f20a67fc640 1 -- 172.21.15.105:0/3254819995 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18887 (secure 0 0 0) 0x7f20a00c2090 con 0x7f20b00cc760 2023-11-28T18:24:38.634 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:38.634 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:24:15.048168+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:24:38.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.630+0000 7f20b811d640 1 -- 172.21.15.105:0/3254819995 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f20940668d0 msgr2=0x7f2094068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:38.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.630+0000 7f20b811d640 1 --2- 172.21.15.105:0/3254819995 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f20940668d0 0x7f2094068d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f20b00731a0 tx=0x7f20ac008040 comp rx=0 tx=0).stop 2023-11-28T18:24:38.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.630+0000 7f20b811d640 1 -- 172.21.15.105:0/3254819995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20b00cc760 msgr2=0x7f20b00b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:38.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.630+0000 7f20b811d640 1 --2- 172.21.15.105:0/3254819995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20b00cc760 0x7f20b00b19b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f20a00ae040 tx=0x7f20a0019020 comp rx=0 tx=0).stop 2023-11-28T18:24:38.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.630+0000 7f20b811d640 1 -- 172.21.15.105:0/3254819995 shutdown_connections 2023-11-28T18:24:38.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.630+0000 7f20b811d640 1 --2- 172.21.15.105:0/3254819995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20b0159f40 0x7f20b00b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.630+0000 7f20b811d640 1 --2- 172.21.15.105:0/3254819995 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f20940668d0 0x7f2094068d90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.630+0000 7f20b811d640 1 --2- 172.21.15.105:0/3254819995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20b00cc760 0x7f20b00b19b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.630+0000 7f20b811d640 1 --2- 172.21.15.105:0/3254819995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20b00ad680 0x7f20b0153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.630+0000 7f20b811d640 1 -- 172.21.15.105:0/3254819995 >> 172.21.15.105:0/3254819995 conn(0x7f20b00ceb90 msgr2=0x7f20b00cd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:38.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.630+0000 7f20b811d640 1 -- 172.21.15.105:0/3254819995 shutdown_connections 2023-11-28T18:24:38.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.630+0000 7f20b811d640 1 -- 172.21.15.105:0/3254819995 wait complete. 2023-11-28T18:24:38.655 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:23:31.485422+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:23:32.274378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T18:23:36.100071+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T18:24:38.656 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:24:38.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.790+0000 7f741046b640 1 Processor -- start 2023-11-28T18:24:38.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.790+0000 7f741046b640 1 -- start start 2023-11-28T18:24:38.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.790+0000 7f741046b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f74080cc760 0x7f7408164d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:38.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.790+0000 7f741046b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74080ccc10 con 0x7f7408159f40 2023-11-28T18:24:38.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.790+0000 7f741046b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f740806bf50 con 0x7f74080caf60 2023-11-28T18:24:38.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.790+0000 7f741046b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74080cb3c0 con 0x7f74080cc760 2023-11-28T18:24:38.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.794+0000 7f740ea08640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f74080cc760 0x7f7408164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:38.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.794+0000 7f740ea08640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f74080cc760 0x7f7408164d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33662/0 (socket says 172.21.15.105:33662) 2023-11-28T18:24:38.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.794+0000 7f740ea08640 1 -- 172.21.15.105:0/3076046711 learned_addr learned my addr 172.21.15.105:0/3076046711 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:38.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.794+0000 7f740ea08640 1 -- 172.21.15.105:0/3076046711 >> v1:172.21.15.148:6789/0 conn(0x7f74080caf60 legacy=0x7f74080ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:38.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.794+0000 7f740ea08640 1 -- 172.21.15.105:0/3076046711 >> v1:172.21.15.105:6789/0 conn(0x7f7408159f40 legacy=0x7f74080ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:38.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.794+0000 7f740ea08640 1 -- 172.21.15.105:0/3076046711 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740815a320 con 0x7f74080cc760 2023-11-28T18:24:38.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.794+0000 7f740ea08640 1 --2- 172.21.15.105:0/3076046711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74080cc760 0x7f7408164d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7400009e50 tx=0x7f740001eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e6863e6155d531 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:38.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.794+0000 7f740d205640 1 -- 172.21.15.105:0/3076046711 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74000a7930 con 0x7f74080cc760 2023-11-28T18:24:38.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.794+0000 7f740d205640 1 -- 172.21.15.105:0/3076046711 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74000a7a90 con 0x7f74080cc760 2023-11-28T18:24:38.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.794+0000 7f740d205640 1 -- 172.21.15.105:0/3076046711 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74000a7e20 con 0x7f74080cc760 2023-11-28T18:24:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f741046b640 1 -- 172.21.15.105:0/3076046711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74080cc760 msgr2=0x7f7408164d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f741046b640 1 --2- 172.21.15.105:0/3076046711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74080cc760 0x7f7408164d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7400009e50 tx=0x7f740001eb60 comp rx=0 tx=0).stop 2023-11-28T18:24:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f741046b640 1 -- 172.21.15.105:0/3076046711 shutdown_connections 2023-11-28T18:24:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f741046b640 1 --2- 172.21.15.105:0/3076046711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74080cc760 0x7f7408164d90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f741046b640 1 -- 172.21.15.105:0/3076046711 >> 172.21.15.105:0/3076046711 conn(0x7f74080ceb90 msgr2=0x7f74080cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f741046b640 1 -- 172.21.15.105:0/3076046711 shutdown_connections 2023-11-28T18:24:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f741046b640 1 -- 172.21.15.105:0/3076046711 wait complete. 2023-11-28T18:24:38.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f741046b640 1 Processor -- start 2023-11-28T18:24:38.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f741046b640 1 -- start start 2023-11-28T18:24:38.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f741046b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74080caf60 0x7f7408142180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:38.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f741046b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74080cc760 0x7f7408144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:38.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f741046b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7408159f40 0x7f740809bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:38.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f741046b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7408143560 con 0x7f74080caf60 2023-11-28T18:24:38.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f741046b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7408143840 con 0x7f74080cc760 2023-11-28T18:24:38.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f741046b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7408143b20 con 0x7f7408159f40 2023-11-28T18:24:38.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f740e207640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74080caf60 0x7f7408142180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:38.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f740e207640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74080caf60 0x7f7408142180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33666/0 (socket says 172.21.15.105:33666) 2023-11-28T18:24:38.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f740e207640 1 -- 172.21.15.105:0/182242754 learned_addr learned my addr 172.21.15.105:0/182242754 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:38.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f740ea08640 1 --2- 172.21.15.105:0/182242754 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7408159f40 0x7f740809bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f740da06640 1 --2- 172.21.15.105:0/182242754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74080cc760 0x7f7408144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f740e207640 1 -- 172.21.15.105:0/182242754 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7408159f40 msgr2=0x7f740809bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f740e207640 1 --2- 172.21.15.105:0/182242754 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7408159f40 0x7f740809bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f740e207640 1 -- 172.21.15.105:0/182242754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74080cc760 msgr2=0x7f7408144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f740e207640 1 --2- 172.21.15.105:0/182242754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74080cc760 0x7f7408144600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f740e207640 1 -- 172.21.15.105:0/182242754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7400009b30 con 0x7f74080caf60 2023-11-28T18:24:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.798+0000 7f740ea08640 1 --2- 172.21.15.105:0/182242754 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7408159f40 0x7f740809bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.802+0000 7f740e207640 1 --2- 172.21.15.105:0/182242754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74080caf60 0x7f7408142180 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f740400c3c0 tx=0x7f740400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.802+0000 7f73feffd640 1 -- 172.21.15.105:0/182242754 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74040052d0 con 0x7f74080caf60 2023-11-28T18:24:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.802+0000 7f73feffd640 1 -- 172.21.15.105:0/182242754 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7404005430 con 0x7f74080caf60 2023-11-28T18:24:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.802+0000 7f741046b640 1 -- 172.21.15.105:0/182242754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7408143e00 con 0x7f74080caf60 2023-11-28T18:24:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.802+0000 7f73feffd640 1 -- 172.21.15.105:0/182242754 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7404003840 con 0x7f74080caf60 2023-11-28T18:24:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.802+0000 7f741046b640 1 -- 172.21.15.105:0/182242754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74081451a0 con 0x7f74080caf60 2023-11-28T18:24:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.802+0000 7f741046b640 1 -- 172.21.15.105:0/182242754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7408067570 con 0x7f74080caf60 2023-11-28T18:24:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.802+0000 7f73feffd640 1 -- 172.21.15.105:0/182242754 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f74040039a0 con 0x7f74080caf60 2023-11-28T18:24:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.802+0000 7f73feffd640 1 --2- 172.21.15.105:0/182242754 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f73ec0668d0 0x7f73ec068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.802+0000 7f73feffd640 1 -- 172.21.15.105:0/182242754 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f7404083da0 con 0x7f74080caf60 2023-11-28T18:24:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.802+0000 7f740da06640 1 --2- 172.21.15.105:0/182242754 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f73ec0668d0 0x7f73ec068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.806+0000 7f740da06640 1 --2- 172.21.15.105:0/182242754 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f73ec0668d0 0x7f73ec068d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f740814fd50 tx=0x7f73f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:38.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.806+0000 7f73feffd640 1 -- 172.21.15.105:0/182242754 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7404050bd0 con 0x7f74080caf60 2023-11-28T18:24:38.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.958+0000 7f741046b640 1 -- 172.21.15.105:0/182242754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74080cf950 con 0x7f74080caf60 2023-11-28T18:24:38.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.958+0000 7f73feffd640 1 -- 172.21.15.105:0/182242754 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f74040170a0 con 0x7f74080caf60 2023-11-28T18:24:38.963 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:24:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.958+0000 7f741046b640 1 -- 172.21.15.105:0/182242754 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f73ec0668d0 msgr2=0x7f73ec068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.958+0000 7f741046b640 1 --2- 172.21.15.105:0/182242754 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f73ec0668d0 0x7f73ec068d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f740814fd50 tx=0x7f73f8009040 comp rx=0 tx=0).stop 2023-11-28T18:24:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.958+0000 7f741046b640 1 -- 172.21.15.105:0/182242754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74080caf60 msgr2=0x7f7408142180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.958+0000 7f741046b640 1 --2- 172.21.15.105:0/182242754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74080caf60 0x7f7408142180 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f740400c3c0 tx=0x7f740400c880 comp rx=0 tx=0).stop 2023-11-28T18:24:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.958+0000 7f741046b640 1 -- 172.21.15.105:0/182242754 shutdown_connections 2023-11-28T18:24:38.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.958+0000 7f741046b640 1 --2- 172.21.15.105:0/182242754 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7408159f40 0x7f740809bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.958+0000 7f741046b640 1 --2- 172.21.15.105:0/182242754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74080cc760 0x7f7408144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.958+0000 7f741046b640 1 --2- 172.21.15.105:0/182242754 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f73ec0668d0 0x7f73ec068d90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.958+0000 7f741046b640 1 --2- 172.21.15.105:0/182242754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74080caf60 0x7f7408142180 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:38.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.958+0000 7f741046b640 1 -- 172.21.15.105:0/182242754 >> 172.21.15.105:0/182242754 conn(0x7f74080ceb90 msgr2=0x7f740814bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:38.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.958+0000 7f741046b640 1 -- 172.21.15.105:0/182242754 shutdown_connections 2023-11-28T18:24:38.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:38.958+0000 7f741046b640 1 -- 172.21.15.105:0/182242754 wait complete. 2023-11-28T18:24:38.984 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:24:39.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.122+0000 7f6d67980640 1 Processor -- start 2023-11-28T18:24:39.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.122+0000 7f6d67980640 1 -- start start 2023-11-28T18:24:39.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.122+0000 7f6d67980640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6d60159f40 0x7f6d600caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:39.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.122+0000 7f6d67980640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d600cc760 0x7f6d600ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:39.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.122+0000 7f6d67980640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6d600ad680 0x7f6d6015da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:39.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.122+0000 7f6d67980640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d6006bf50 con 0x7f6d600ad680 2023-11-28T18:24:39.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.122+0000 7f6d67980640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d600ccbc0 con 0x7f6d60159f40 2023-11-28T18:24:39.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.122+0000 7f6d67980640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6015a320 con 0x7f6d600cc760 2023-11-28T18:24:39.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.122+0000 7f6d65f1d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6d600ad680 0x7f6d6015da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:39.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.122+0000 7f6d65f1d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6d600ad680 0x7f6d6015da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33676/0 (socket says 172.21.15.105:33676) 2023-11-28T18:24:39.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.122+0000 7f6d65f1d640 1 -- 172.21.15.105:0/3522873545 learned_addr learned my addr 172.21.15.105:0/3522873545 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:39.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.122+0000 7f6d6571c640 1 --2- 172.21.15.105:0/3522873545 >> v2:172.21.15.148:3300/0 conn(0x7f6d60159f40 0x7f6d600caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:39.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.122+0000 7f6d64f1b640 1 --2- 172.21.15.105:0/3522873545 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d600cc760 0x7f6d600ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:39.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.122+0000 7f6d65f1d640 1 -- 172.21.15.105:0/3522873545 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d600cc760 msgr2=0x7f6d600ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:39.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.122+0000 7f6d65f1d640 1 --2- 172.21.15.105:0/3522873545 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d600cc760 0x7f6d600ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.122+0000 7f6d65f1d640 1 -- 172.21.15.105:0/3522873545 >> v2:172.21.15.148:3300/0 conn(0x7f6d60159f40 msgr2=0x7f6d600caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:39.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.122+0000 7f6d65f1d640 1 --2- 172.21.15.105:0/3522873545 >> v2:172.21.15.148:3300/0 conn(0x7f6d60159f40 0x7f6d600caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.122+0000 7f6d65f1d640 1 -- 172.21.15.105:0/3522873545 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d60071380 con 0x7f6d600ad680 2023-11-28T18:24:39.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.126+0000 7f6d65f1d640 1 --2- 172.21.15.105:0/3522873545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d600ad680 0x7f6d6015da20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6d6006e500 tx=0x7f6d5801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1b6f40bfde8460 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:39.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.126+0000 7f6d57fff640 1 -- 172.21.15.105:0/3522873545 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d580a5790 con 0x7f6d600ad680 2023-11-28T18:24:39.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.126+0000 7f6d57fff640 1 -- 172.21.15.105:0/3522873545 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d580a58f0 con 0x7f6d600ad680 2023-11-28T18:24:39.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.126+0000 7f6d57fff640 1 -- 172.21.15.105:0/3522873545 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d580a5c80 con 0x7f6d600ad680 2023-11-28T18:24:39.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.126+0000 7f6d67980640 1 -- 172.21.15.105:0/3522873545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d600ad680 msgr2=0x7f6d6015da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:39.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.126+0000 7f6d67980640 1 --2- 172.21.15.105:0/3522873545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d600ad680 0x7f6d6015da20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6d6006e500 tx=0x7f6d5801c910 comp rx=0 tx=0).stop 2023-11-28T18:24:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.126+0000 7f6d67980640 1 -- 172.21.15.105:0/3522873545 shutdown_connections 2023-11-28T18:24:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.126+0000 7f6d67980640 1 --2- 172.21.15.105:0/3522873545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d600ad680 0x7f6d6015da20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.126+0000 7f6d67980640 1 --2- 172.21.15.105:0/3522873545 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d600cc760 0x7f6d600ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.126+0000 7f6d67980640 1 --2- 172.21.15.105:0/3522873545 >> v2:172.21.15.148:3300/0 conn(0x7f6d60159f40 0x7f6d600caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.126+0000 7f6d67980640 1 -- 172.21.15.105:0/3522873545 >> 172.21.15.105:0/3522873545 conn(0x7f6d600ceb90 msgr2=0x7f6d600cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.126+0000 7f6d67980640 1 -- 172.21.15.105:0/3522873545 shutdown_connections 2023-11-28T18:24:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.126+0000 7f6d67980640 1 -- 172.21.15.105:0/3522873545 wait complete. 2023-11-28T18:24:39.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d67980640 1 Processor -- start 2023-11-28T18:24:39.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d67980640 1 -- start start 2023-11-28T18:24:39.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d67980640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d600ad680 0x7f6d60147620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:39.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d67980640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d600cc760 0x7f6d6014b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:39.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d67980640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d60159f40 0x7f6d6014b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:39.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d67980640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d60164a00 con 0x7f6d600ad680 2023-11-28T18:24:39.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d67980640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6014e970 con 0x7f6d60159f40 2023-11-28T18:24:39.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d67980640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6015a650 con 0x7f6d600cc760 2023-11-28T18:24:39.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d6571c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d600ad680 0x7f6d60147620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:39.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d6571c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d600ad680 0x7f6d60147620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33684/0 (socket says 172.21.15.105:33684) 2023-11-28T18:24:39.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d6571c640 1 -- 172.21.15.105:0/4294242732 learned_addr learned my addr 172.21.15.105:0/4294242732 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:39.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d64f1b640 1 --2- 172.21.15.105:0/4294242732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d600cc760 0x7f6d6014b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:39.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d65f1d640 1 --2- 172.21.15.105:0/4294242732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d60159f40 0x7f6d6014b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:39.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d6571c640 1 -- 172.21.15.105:0/4294242732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d600cc760 msgr2=0x7f6d6014b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:39.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d6571c640 1 --2- 172.21.15.105:0/4294242732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d600cc760 0x7f6d6014b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d6571c640 1 -- 172.21.15.105:0/4294242732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d60159f40 msgr2=0x7f6d6014b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:39.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d6571c640 1 --2- 172.21.15.105:0/4294242732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d60159f40 0x7f6d6014b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d6571c640 1 -- 172.21.15.105:0/4294242732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d58009b30 con 0x7f6d600ad680 2023-11-28T18:24:39.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d6571c640 1 --2- 172.21.15.105:0/4294242732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d600ad680 0x7f6d60147620 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00d5f0 tx=0x7f6d5c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:39.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d55ffb640 1 -- 172.21.15.105:0/4294242732 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c0125f0 con 0x7f6d600ad680 2023-11-28T18:24:39.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d55ffb640 1 -- 172.21.15.105:0/4294242732 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5c012750 con 0x7f6d600ad680 2023-11-28T18:24:39.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d55ffb640 1 -- 172.21.15.105:0/4294242732 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c012ae0 con 0x7f6d600ad680 2023-11-28T18:24:39.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.130+0000 7f6d67980640 1 -- 172.21.15.105:0/4294242732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d6015a930 con 0x7f6d600ad680 2023-11-28T18:24:39.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.134+0000 7f6d67980640 1 -- 172.21.15.105:0/4294242732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d6015aef0 con 0x7f6d600ad680 2023-11-28T18:24:39.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.134+0000 7f6d67980640 1 -- 172.21.15.105:0/4294242732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d60067570 con 0x7f6d600ad680 2023-11-28T18:24:39.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.134+0000 7f6d55ffb640 1 -- 172.21.15.105:0/4294242732 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f6d5c012c40 con 0x7f6d600ad680 2023-11-28T18:24:39.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.134+0000 7f6d55ffb640 1 --2- 172.21.15.105:0/4294242732 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6d440668d0 0x7f6d44068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:39.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.134+0000 7f6d55ffb640 1 -- 172.21.15.105:0/4294242732 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f6d5c084b30 con 0x7f6d600ad680 2023-11-28T18:24:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.134+0000 7f6d64f1b640 1 --2- 172.21.15.105:0/4294242732 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6d440668d0 0x7f6d44068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.134+0000 7f6d64f1b640 1 --2- 172.21.15.105:0/4294242732 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6d440668d0 0x7f6d44068d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6d6009cc30 tx=0x7f6d50008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:39.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.138+0000 7f6d55ffb640 1 -- 172.21.15.105:0/4294242732 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6d5c051ab0 con 0x7f6d600ad680 2023-11-28T18:24:39.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.290+0000 7f6d67980640 1 -- 172.21.15.105:0/4294242732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6d600cf950 con 0x7f6d600ad680 2023-11-28T18:24:39.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.290+0000 7f6d55ffb640 1 -- 172.21.15.105:0/4294242732 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6d5c0550d0 con 0x7f6d600ad680 2023-11-28T18:24:39.296 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:39.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.294+0000 7f6d67980640 1 -- 172.21.15.105:0/4294242732 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6d440668d0 msgr2=0x7f6d44068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:39.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.294+0000 7f6d67980640 1 --2- 172.21.15.105:0/4294242732 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6d440668d0 0x7f6d44068d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6d6009cc30 tx=0x7f6d50008040 comp rx=0 tx=0).stop 2023-11-28T18:24:39.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.294+0000 7f6d67980640 1 -- 172.21.15.105:0/4294242732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d600ad680 msgr2=0x7f6d60147620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:39.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.294+0000 7f6d67980640 1 --2- 172.21.15.105:0/4294242732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d600ad680 0x7f6d60147620 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00d5f0 tx=0x7f6d5c00dab0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.294+0000 7f6d67980640 1 -- 172.21.15.105:0/4294242732 shutdown_connections 2023-11-28T18:24:39.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.294+0000 7f6d67980640 1 --2- 172.21.15.105:0/4294242732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d60159f40 0x7f6d6014b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.294+0000 7f6d67980640 1 --2- 172.21.15.105:0/4294242732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d600cc760 0x7f6d6014b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.294+0000 7f6d67980640 1 --2- 172.21.15.105:0/4294242732 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6d440668d0 0x7f6d44068d90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.294+0000 7f6d67980640 1 --2- 172.21.15.105:0/4294242732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d600ad680 0x7f6d60147620 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.294+0000 7f6d67980640 1 -- 172.21.15.105:0/4294242732 >> 172.21.15.105:0/4294242732 conn(0x7f6d600ceb90 msgr2=0x7f6d600700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:39.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.294+0000 7f6d67980640 1 -- 172.21.15.105:0/4294242732 shutdown_connections 2023-11-28T18:24:39.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.294+0000 7f6d67980640 1 -- 172.21.15.105:0/4294242732 wait complete. 2023-11-28T18:24:39.317 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:24:39.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.450+0000 7ffb16ff5640 1 Processor -- start 2023-11-28T18:24:39.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.450+0000 7ffb16ff5640 1 -- start start 2023-11-28T18:24:39.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.454+0000 7ffb16ff5640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffb100b4290 0x7ffb100b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:39.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.454+0000 7ffb16ff5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffb1015b030 0x7ffb100709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:39.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.454+0000 7ffb16ff5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb100b4bb0 con 0x7ffb10071960 2023-11-28T18:24:39.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.454+0000 7ffb16ff5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb100701d0 con 0x7ffb1015b030 2023-11-28T18:24:39.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.454+0000 7ffb16ff5640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1006df80 con 0x7ffb100b4290 2023-11-28T18:24:39.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.454+0000 7ffb15592640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffb1015b030 0x7ffb100709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:39.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.454+0000 7ffb15592640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffb1015b030 0x7ffb100709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33698/0 (socket says 172.21.15.105:33698) 2023-11-28T18:24:39.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.454+0000 7ffb15592640 1 -- 172.21.15.105:0/1519135281 learned_addr learned my addr 172.21.15.105:0/1519135281 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:39.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.454+0000 7ffb15592640 1 -- 172.21.15.105:0/1519135281 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffb100b4290 msgr2=0x7ffb100b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:39.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.454+0000 7ffb15592640 1 --2- 172.21.15.105:0/1519135281 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffb100b4290 0x7ffb100b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.454+0000 7ffb15592640 1 -- 172.21.15.105:0/1519135281 >> v1:172.21.15.148:6789/0 conn(0x7ffb10071960 legacy=0x7ffb1015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:39.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.454+0000 7ffb15592640 1 -- 172.21.15.105:0/1519135281 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb10071df0 con 0x7ffb1015b030 2023-11-28T18:24:39.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.458+0000 7ffb15592640 1 --2- 172.21.15.105:0/1519135281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb1015b030 0x7ffb100709b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009e50 tx=0x7ffb0801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa87a3a3d42685 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:39.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.458+0000 7ffb077fe640 1 -- 172.21.15.105:0/1519135281 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb080a5730 con 0x7ffb1015b030 2023-11-28T18:24:39.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.458+0000 7ffb077fe640 1 -- 172.21.15.105:0/1519135281 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb080a5890 con 0x7ffb1015b030 2023-11-28T18:24:39.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.458+0000 7ffb077fe640 1 -- 172.21.15.105:0/1519135281 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb080a5c20 con 0x7ffb1015b030 2023-11-28T18:24:39.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.458+0000 7ffb16ff5640 1 -- 172.21.15.105:0/1519135281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb1015b030 msgr2=0x7ffb100709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:39.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.458+0000 7ffb16ff5640 1 --2- 172.21.15.105:0/1519135281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb1015b030 0x7ffb100709b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009e50 tx=0x7ffb0801c9a0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.458+0000 7ffb16ff5640 1 -- 172.21.15.105:0/1519135281 shutdown_connections 2023-11-28T18:24:39.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.458+0000 7ffb16ff5640 1 --2- 172.21.15.105:0/1519135281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb1015b030 0x7ffb100709b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.458+0000 7ffb16ff5640 1 --2- 172.21.15.105:0/1519135281 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffb100b4290 0x7ffb100b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.458+0000 7ffb16ff5640 1 -- 172.21.15.105:0/1519135281 >> 172.21.15.105:0/1519135281 conn(0x7ffb100ceb90 msgr2=0x7ffb100cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:39.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.458+0000 7ffb16ff5640 1 -- 172.21.15.105:0/1519135281 shutdown_connections 2023-11-28T18:24:39.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.458+0000 7ffb16ff5640 1 -- 172.21.15.105:0/1519135281 wait complete. 2023-11-28T18:24:39.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.458+0000 7ffb16ff5640 1 Processor -- start 2023-11-28T18:24:39.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.458+0000 7ffb16ff5640 1 -- start start 2023-11-28T18:24:39.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.458+0000 7ffb16ff5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb100b4290 0x7ffb1014e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:39.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb16ff5640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffb1015b030 0x7ffb10099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:39.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb16ff5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb1014eb40 0x7ffb10099f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:39.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb16ff5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1014f990 con 0x7ffb1014eb40 2023-11-28T18:24:39.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb16ff5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb10151420 con 0x7ffb100b4290 2023-11-28T18:24:39.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb16ff5640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1014f340 con 0x7ffb1015b030 2023-11-28T18:24:39.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb15592640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb1014eb40 0x7ffb10099f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:39.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb15592640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb1014eb40 0x7ffb10099f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33708/0 (socket says 172.21.15.105:33708) 2023-11-28T18:24:39.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb15592640 1 -- 172.21.15.105:0/114438959 learned_addr learned my addr 172.21.15.105:0/114438959 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:39.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb14d91640 1 --2- 172.21.15.105:0/114438959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb100b4290 0x7ffb1014e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:39.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb07fff640 1 --2- 172.21.15.105:0/114438959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffb1015b030 0x7ffb10099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:39.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb15592640 1 -- 172.21.15.105:0/114438959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffb1015b030 msgr2=0x7ffb10099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:39.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb15592640 1 --2- 172.21.15.105:0/114438959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffb1015b030 0x7ffb10099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb15592640 1 -- 172.21.15.105:0/114438959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb100b4290 msgr2=0x7ffb1014e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:39.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb15592640 1 --2- 172.21.15.105:0/114438959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb100b4290 0x7ffb1014e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb15592640 1 -- 172.21.15.105:0/114438959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb08009b30 con 0x7ffb1014eb40 2023-11-28T18:24:39.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb14d91640 1 --2- 172.21.15.105:0/114438959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb100b4290 0x7ffb1014e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:39.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb15592640 1 --2- 172.21.15.105:0/114438959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb1014eb40 0x7ffb10099f90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ffb080af040 tx=0x7ffb08019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:39.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb057fa640 1 -- 172.21.15.105:0/114438959 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb080b8670 con 0x7ffb1014eb40 2023-11-28T18:24:39.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb057fa640 1 -- 172.21.15.105:0/114438959 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb080b87d0 con 0x7ffb1014eb40 2023-11-28T18:24:39.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb057fa640 1 -- 172.21.15.105:0/114438959 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb080b8b80 con 0x7ffb1014eb40 2023-11-28T18:24:39.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb16ff5640 1 -- 172.21.15.105:0/114438959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb1009a6a0 con 0x7ffb1014eb40 2023-11-28T18:24:39.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb16ff5640 1 -- 172.21.15.105:0/114438959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb100ad4f0 con 0x7ffb1014eb40 2023-11-28T18:24:39.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb16ff5640 1 -- 172.21.15.105:0/114438959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb1014fad0 con 0x7ffb1014eb40 2023-11-28T18:24:39.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.462+0000 7ffb057fa640 1 -- 172.21.15.105:0/114438959 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ffb080b8ce0 con 0x7ffb1014eb40 2023-11-28T18:24:39.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.466+0000 7ffb057fa640 1 --2- 172.21.15.105:0/114438959 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffafc066900 0x7ffafc068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:39.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.466+0000 7ffb057fa640 1 -- 172.21.15.105:0/114438959 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7ffb0811f7b0 con 0x7ffb1014eb40 2023-11-28T18:24:39.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.466+0000 7ffb14d91640 1 --2- 172.21.15.105:0/114438959 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffafc066900 0x7ffafc068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:39.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.466+0000 7ffb14d91640 1 --2- 172.21.15.105:0/114438959 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffafc066900 0x7ffafc068dc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c006fd0 tx=0x7ffb0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:39.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.470+0000 7ffb057fa640 1 -- 172.21.15.105:0/114438959 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ffb1014fad0 con 0x7ffb1014eb40 2023-11-28T18:24:39.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.622+0000 7ffb16ff5640 1 -- 172.21.15.105:0/114438959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffb1014fad0 con 0x7ffb1014eb40 2023-11-28T18:24:39.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.622+0000 7ffb057fa640 1 -- 172.21.15.105:0/114438959 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffb1014fad0 con 0x7ffb1014eb40 2023-11-28T18:24:39.627 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:24:39.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.622+0000 7ffb16ff5640 1 -- 172.21.15.105:0/114438959 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffafc066900 msgr2=0x7ffafc068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:39.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.622+0000 7ffb16ff5640 1 --2- 172.21.15.105:0/114438959 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffafc066900 0x7ffafc068dc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c006fd0 tx=0x7ffb0c008040 comp rx=0 tx=0).stop 2023-11-28T18:24:39.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.622+0000 7ffb16ff5640 1 -- 172.21.15.105:0/114438959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb1014eb40 msgr2=0x7ffb10099f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:39.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.622+0000 7ffb16ff5640 1 --2- 172.21.15.105:0/114438959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb1014eb40 0x7ffb10099f90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ffb080af040 tx=0x7ffb08019740 comp rx=0 tx=0).stop 2023-11-28T18:24:39.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.622+0000 7ffb16ff5640 1 -- 172.21.15.105:0/114438959 shutdown_connections 2023-11-28T18:24:39.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.622+0000 7ffb16ff5640 1 --2- 172.21.15.105:0/114438959 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffafc066900 0x7ffafc068dc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.622+0000 7ffb16ff5640 1 --2- 172.21.15.105:0/114438959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb1014eb40 0x7ffb10099f90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.622+0000 7ffb16ff5640 1 --2- 172.21.15.105:0/114438959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffb1015b030 0x7ffb10099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.622+0000 7ffb16ff5640 1 --2- 172.21.15.105:0/114438959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb100b4290 0x7ffb1014e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.622+0000 7ffb16ff5640 1 -- 172.21.15.105:0/114438959 >> 172.21.15.105:0/114438959 conn(0x7ffb100ceb90 msgr2=0x7ffb101645a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:39.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.622+0000 7ffb16ff5640 1 -- 172.21.15.105:0/114438959 shutdown_connections 2023-11-28T18:24:39.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.622+0000 7ffb16ff5640 1 -- 172.21.15.105:0/114438959 wait complete. 2023-11-28T18:24:39.649 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T18:24:39.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.782+0000 7f95e2e5f640 1 Processor -- start 2023-11-28T18:24:39.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.782+0000 7f95e2e5f640 1 -- start start 2023-11-28T18:24:39.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.786+0000 7f95e2e5f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f95dc0caf60 0x7f95dc0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:39.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.786+0000 7f95e2e5f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95dc06bf50 con 0x7f95dc159f40 2023-11-28T18:24:39.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.786+0000 7f95e2e5f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95dc15a320 con 0x7f95dc0ad850 2023-11-28T18:24:39.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.786+0000 7f95e2e5f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95dc0cb3c0 con 0x7f95dc0caf60 2023-11-28T18:24:39.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.786+0000 7f95e0bfb640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f95dc159f40 0x7f95dc0ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35440/0 (socket says 172.21.15.105:35440) 2023-11-28T18:24:39.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.786+0000 7f95e0bfb640 1 -- 172.21.15.105:0/2430382161 learned_addr learned my addr 172.21.15.105:0/2430382161 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:39.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.786+0000 7f95d3fff640 1 --2- 172.21.15.105:0/2430382161 >> v2:172.21.15.148:3300/0 conn(0x7f95dc0caf60 0x7f95dc0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:39.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.786+0000 7f95d37fe640 1 -- 172.21.15.105:0/2430382161 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 160430861 0 0) 0x7f95dc06bf50 con 0x7f95dc159f40 2023-11-28T18:24:39.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.786+0000 7f95d37fe640 1 -- 172.21.15.105:0/2430382161 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95c4003530 con 0x7f95dc159f40 2023-11-28T18:24:39.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.786+0000 7f95d3fff640 1 -- 172.21.15.105:0/2430382161 >> v1:172.21.15.148:6789/0 conn(0x7f95dc0ad850 legacy=0x7f95dc0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:39.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.786+0000 7f95d37fe640 1 -- 172.21.15.105:0/2430382161 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892483329 0 0) 0x7f95dc15a320 con 0x7f95dc0ad850 2023-11-28T18:24:39.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.786+0000 7f95d3fff640 1 -- 172.21.15.105:0/2430382161 >> v1:172.21.15.105:6789/0 conn(0x7f95dc159f40 legacy=0x7f95dc0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:39.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.786+0000 7f95d3fff640 1 -- 172.21.15.105:0/2430382161 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95dc071380 con 0x7f95dc0caf60 2023-11-28T18:24:39.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.790+0000 7f95d3fff640 1 --2- 172.21.15.105:0/2430382161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95dc0caf60 0x7f95dc0cc760 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0097c0 tx=0x7f95cc01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=235da87da64b945a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:39.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.790+0000 7f95d37fe640 1 -- 172.21.15.105:0/2430382161 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc0ac620 con 0x7f95dc0caf60 2023-11-28T18:24:39.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.790+0000 7f95d37fe640 1 -- 172.21.15.105:0/2430382161 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95cc0ac780 con 0x7f95dc0caf60 2023-11-28T18:24:39.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.790+0000 7f95d37fe640 1 -- 172.21.15.105:0/2430382161 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc0acb10 con 0x7f95dc0caf60 2023-11-28T18:24:39.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.790+0000 7f95e2e5f640 1 -- 172.21.15.105:0/2430382161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95dc0caf60 msgr2=0x7f95dc0cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:39.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.790+0000 7f95e2e5f640 1 --2- 172.21.15.105:0/2430382161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95dc0caf60 0x7f95dc0cc760 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0097c0 tx=0x7f95cc01c3c0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.790+0000 7f95e2e5f640 1 -- 172.21.15.105:0/2430382161 shutdown_connections 2023-11-28T18:24:39.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.790+0000 7f95e2e5f640 1 --2- 172.21.15.105:0/2430382161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95dc0caf60 0x7f95dc0cc760 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.790+0000 7f95e2e5f640 1 -- 172.21.15.105:0/2430382161 >> 172.21.15.105:0/2430382161 conn(0x7f95dc0ceb90 msgr2=0x7f95dc0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:39.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.790+0000 7f95e2e5f640 1 -- 172.21.15.105:0/2430382161 shutdown_connections 2023-11-28T18:24:39.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.790+0000 7f95e2e5f640 1 -- 172.21.15.105:0/2430382161 wait complete. 2023-11-28T18:24:39.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.790+0000 7f95e2e5f640 1 Processor -- start 2023-11-28T18:24:39.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.790+0000 7f95e2e5f640 1 -- start start 2023-11-28T18:24:39.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95e2e5f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc0ad850 0x7f95dc0b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:39.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95e2e5f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95dc0caf60 0x7f95dc0b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:39.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95e2e5f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95dc159f40 0x7f95dc06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:39.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95e2e5f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc1469f0 con 0x7f95dc0ad850 2023-11-28T18:24:39.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95e2e5f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc0b2510 con 0x7f95dc0caf60 2023-11-28T18:24:39.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95e2e5f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc0b2790 con 0x7f95dc159f40 2023-11-28T18:24:39.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95e0bfb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc0ad850 0x7f95dc0b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:39.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95e0bfb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc0ad850 0x7f95dc0b3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33714/0 (socket says 172.21.15.105:33714) 2023-11-28T18:24:39.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95e0bfb640 1 -- 172.21.15.105:0/613824247 learned_addr learned my addr 172.21.15.105:0/613824247 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:39.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95d3fff640 1 --2- 172.21.15.105:0/613824247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95dc0caf60 0x7f95dc0b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:39.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95e13fc640 1 --2- 172.21.15.105:0/613824247 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95dc159f40 0x7f95dc06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:39.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95e0bfb640 1 -- 172.21.15.105:0/613824247 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95dc159f40 msgr2=0x7f95dc06c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:39.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95e0bfb640 1 --2- 172.21.15.105:0/613824247 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95dc159f40 0x7f95dc06c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95e0bfb640 1 -- 172.21.15.105:0/613824247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95dc0caf60 msgr2=0x7f95dc0b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:39.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95e0bfb640 1 --2- 172.21.15.105:0/613824247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95dc0caf60 0x7f95dc0b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95e0bfb640 1 -- 172.21.15.105:0/613824247 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc009420 con 0x7f95dc0ad850 2023-11-28T18:24:39.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95d3fff640 1 --2- 172.21.15.105:0/613824247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95dc0caf60 0x7f95dc0b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:39.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95e0bfb640 1 --2- 172.21.15.105:0/613824247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc0ad850 0x7f95dc0b3530 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f95d800c4d0 tx=0x7f95d800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:39.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95d17fa640 1 -- 172.21.15.105:0/613824247 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d8005cf0 con 0x7f95dc0ad850 2023-11-28T18:24:39.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95d17fa640 1 -- 172.21.15.105:0/613824247 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d8005e50 con 0x7f95dc0ad850 2023-11-28T18:24:39.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95d17fa640 1 -- 172.21.15.105:0/613824247 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d8007270 con 0x7f95dc0ad850 2023-11-28T18:24:39.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95e2e5f640 1 -- 172.21.15.105:0/613824247 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95dc0b2a50 con 0x7f95dc0ad850 2023-11-28T18:24:39.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95e2e5f640 1 -- 172.21.15.105:0/613824247 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95dc0b3010 con 0x7f95dc0ad850 2023-11-28T18:24:39.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95e2e5f640 1 -- 172.21.15.105:0/613824247 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95dc067570 con 0x7f95dc0ad850 2023-11-28T18:24:39.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.794+0000 7f95d17fa640 1 -- 172.21.15.105:0/613824247 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f95d80073d0 con 0x7f95dc0ad850 2023-11-28T18:24:39.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.798+0000 7f95d17fa640 1 --2- 172.21.15.105:0/613824247 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f95c80668d0 0x7f95c8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:39.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.798+0000 7f95d17fa640 1 -- 172.21.15.105:0/613824247 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f95d8083cf0 con 0x7f95dc0ad850 2023-11-28T18:24:39.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.798+0000 7f95d3fff640 1 --2- 172.21.15.105:0/613824247 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f95c80668d0 0x7f95c8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:39.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.798+0000 7f95d3fff640 1 --2- 172.21.15.105:0/613824247 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f95c80668d0 0x7f95c8068d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0ae000 tx=0x7f95cc0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:39.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.802+0000 7f95d17fa640 1 -- 172.21.15.105:0/613824247 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f95d8088070 con 0x7f95dc0ad850 2023-11-28T18:24:39.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.950+0000 7f95e2e5f640 1 -- 172.21.15.105:0/613824247 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f95dc0cf950 con 0x7f95dc0ad850 2023-11-28T18:24:39.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.950+0000 7f95d17fa640 1 -- 172.21.15.105:0/613824247 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f95d8017020 con 0x7f95dc0ad850 2023-11-28T18:24:39.955 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:39.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.950+0000 7f95e2e5f640 1 -- 172.21.15.105:0/613824247 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f95c80668d0 msgr2=0x7f95c8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:39.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.950+0000 7f95e2e5f640 1 --2- 172.21.15.105:0/613824247 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f95c80668d0 0x7f95c8068d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0ae000 tx=0x7f95cc0a7000 comp rx=0 tx=0).stop 2023-11-28T18:24:39.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.950+0000 7f95e2e5f640 1 -- 172.21.15.105:0/613824247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc0ad850 msgr2=0x7f95dc0b3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:39.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.950+0000 7f95e2e5f640 1 --2- 172.21.15.105:0/613824247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc0ad850 0x7f95dc0b3530 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f95d800c4d0 tx=0x7f95d800c990 comp rx=0 tx=0).stop 2023-11-28T18:24:39.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.950+0000 7f95e2e5f640 1 -- 172.21.15.105:0/613824247 shutdown_connections 2023-11-28T18:24:39.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.950+0000 7f95e2e5f640 1 --2- 172.21.15.105:0/613824247 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95dc159f40 0x7f95dc06c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.950+0000 7f95e2e5f640 1 --2- 172.21.15.105:0/613824247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95dc0caf60 0x7f95dc0b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.950+0000 7f95e2e5f640 1 --2- 172.21.15.105:0/613824247 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f95c80668d0 0x7f95c8068d90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.950+0000 7f95e2e5f640 1 --2- 172.21.15.105:0/613824247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc0ad850 0x7f95dc0b3530 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:39.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.950+0000 7f95e2e5f640 1 -- 172.21.15.105:0/613824247 >> 172.21.15.105:0/613824247 conn(0x7f95dc0ceb90 msgr2=0x7f95dc0cd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:39.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.950+0000 7f95e2e5f640 1 -- 172.21.15.105:0/613824247 shutdown_connections 2023-11-28T18:24:39.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:39.950+0000 7f95e2e5f640 1 -- 172.21.15.105:0/613824247 wait complete. 2023-11-28T18:24:39.975 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T18:24:40.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.110+0000 7f404f380640 1 Processor -- start 2023-11-28T18:24:40.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.110+0000 7f404f380640 1 -- start start 2023-11-28T18:24:40.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.114+0000 7f404f380640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f404806d610 0x7f40480cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:40.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.114+0000 7f404f380640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f40480ce530 0x7f40480ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:40.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.114+0000 7f404f380640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f40480adf30 0x7f40481650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:40.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.114+0000 7f404f380640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40480ae380 con 0x7f40480ce530 2023-11-28T18:24:40.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.114+0000 7f404f380640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4048162820 con 0x7f40480adf30 2023-11-28T18:24:40.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.114+0000 7f404f380640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4048161c10 con 0x7f404806d610 2023-11-28T18:24:40.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.114+0000 7f404c91b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f40480ce530 0x7f40480ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:40.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.114+0000 7f404c91b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f40480ce530 0x7f40480ad9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33724/0 (socket says 172.21.15.105:33724) 2023-11-28T18:24:40.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.114+0000 7f404c91b640 1 -- 172.21.15.105:0/2093948304 learned_addr learned my addr 172.21.15.105:0/2093948304 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:40.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.114+0000 7f404d11c640 1 --2- 172.21.15.105:0/2093948304 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f404806d610 0x7f40480cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:40.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.114+0000 7f404d91d640 1 --2- 172.21.15.105:0/2093948304 >> v2:172.21.15.148:3300/0 conn(0x7f40480adf30 0x7f40481650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:40.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.114+0000 7f404c91b640 1 -- 172.21.15.105:0/2093948304 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f404806d610 msgr2=0x7f40480cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:40.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.114+0000 7f404c91b640 1 --2- 172.21.15.105:0/2093948304 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f404806d610 0x7f40480cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.114+0000 7f404c91b640 1 -- 172.21.15.105:0/2093948304 >> v2:172.21.15.148:3300/0 conn(0x7f40480adf30 msgr2=0x7f40481650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:40.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.114+0000 7f404c91b640 1 --2- 172.21.15.105:0/2093948304 >> v2:172.21.15.148:3300/0 conn(0x7f40480adf30 0x7f40481650f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.114+0000 7f404c91b640 1 -- 172.21.15.105:0/2093948304 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4048161ef0 con 0x7f40480ce530 2023-11-28T18:24:40.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.114+0000 7f404d91d640 1 --2- 172.21.15.105:0/2093948304 >> v2:172.21.15.148:3300/0 conn(0x7f40480adf30 0x7f40481650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:40.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.114+0000 7f404d11c640 1 --2- 172.21.15.105:0/2093948304 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f404806d610 0x7f40480cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:40.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.114+0000 7f404c91b640 1 --2- 172.21.15.105:0/2093948304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40480ce530 0x7f40480ad9f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f40481661b0 tx=0x7f403801c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5592b2fcce009282 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:40.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.114+0000 7f403ffff640 1 -- 172.21.15.105:0/2093948304 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40380ab600 con 0x7f40480ce530 2023-11-28T18:24:40.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.114+0000 7f403ffff640 1 -- 172.21.15.105:0/2093948304 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40380a7050 con 0x7f40480ce530 2023-11-28T18:24:40.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.114+0000 7f403ffff640 1 -- 172.21.15.105:0/2093948304 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40380ab980 con 0x7f40480ce530 2023-11-28T18:24:40.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.118+0000 7f404f380640 1 -- 172.21.15.105:0/2093948304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40480ce530 msgr2=0x7f40480ad9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:40.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.118+0000 7f404f380640 1 --2- 172.21.15.105:0/2093948304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40480ce530 0x7f40480ad9f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f40481661b0 tx=0x7f403801c210 comp rx=0 tx=0).stop 2023-11-28T18:24:40.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.118+0000 7f404f380640 1 -- 172.21.15.105:0/2093948304 shutdown_connections 2023-11-28T18:24:40.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.118+0000 7f404f380640 1 --2- 172.21.15.105:0/2093948304 >> v2:172.21.15.148:3300/0 conn(0x7f40480adf30 0x7f40481650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.118+0000 7f404f380640 1 --2- 172.21.15.105:0/2093948304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40480ce530 0x7f40480ad9f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.118+0000 7f404f380640 1 --2- 172.21.15.105:0/2093948304 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f404806d610 0x7f40480cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.118+0000 7f404f380640 1 -- 172.21.15.105:0/2093948304 >> 172.21.15.105:0/2093948304 conn(0x7f40480ceb90 msgr2=0x7f40480cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:40.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.118+0000 7f404f380640 1 -- 172.21.15.105:0/2093948304 shutdown_connections 2023-11-28T18:24:40.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.118+0000 7f404f380640 1 -- 172.21.15.105:0/2093948304 wait complete. 2023-11-28T18:24:40.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.118+0000 7f404f380640 1 Processor -- start 2023-11-28T18:24:40.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.118+0000 7f404f380640 1 -- start start 2023-11-28T18:24:40.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.118+0000 7f404f380640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f404806d610 0x7f40481550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.118+0000 7f404f380640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40480adf30 0x7f404815c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.118+0000 7f404f380640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40480ce530 0x7f404815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.118+0000 7f404f380640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40481555e0 con 0x7f40480ce530 2023-11-28T18:24:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.118+0000 7f404f380640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404815edb0 con 0x7f40480adf30 2023-11-28T18:24:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.118+0000 7f404f380640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f404815b370 con 0x7f404806d610 2023-11-28T18:24:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.118+0000 7f404d91d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40480ce530 0x7f404815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.118+0000 7f404d91d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40480ce530 0x7f404815cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33740/0 (socket says 172.21.15.105:33740) 2023-11-28T18:24:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.118+0000 7f404d91d640 1 -- 172.21.15.105:0/2845821441 learned_addr learned my addr 172.21.15.105:0/2845821441 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.118+0000 7f404d11c640 1 --2- 172.21.15.105:0/2845821441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f404806d610 0x7f40481550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.122+0000 7f404c91b640 1 --2- 172.21.15.105:0/2845821441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40480adf30 0x7f404815c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.122+0000 7f404d91d640 1 -- 172.21.15.105:0/2845821441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f404806d610 msgr2=0x7f40481550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.122+0000 7f404d91d640 1 --2- 172.21.15.105:0/2845821441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f404806d610 0x7f40481550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.122+0000 7f404d91d640 1 -- 172.21.15.105:0/2845821441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40480adf30 msgr2=0x7f404815c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.122+0000 7f404d91d640 1 --2- 172.21.15.105:0/2845821441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40480adf30 0x7f404815c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.122+0000 7f404d91d640 1 -- 172.21.15.105:0/2845821441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40380094d0 con 0x7f40480ce530 2023-11-28T18:24:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.122+0000 7f404d91d640 1 --2- 172.21.15.105:0/2845821441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40480ce530 0x7f404815cdc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4040007ef0 tx=0x7f404000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.122+0000 7f403dffb640 1 -- 172.21.15.105:0/2845821441 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040003ba0 con 0x7f40480ce530 2023-11-28T18:24:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.122+0000 7f403dffb640 1 -- 172.21.15.105:0/2845821441 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404000f040 con 0x7f40480ce530 2023-11-28T18:24:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.122+0000 7f403dffb640 1 -- 172.21.15.105:0/2845821441 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404000de80 con 0x7f40480ce530 2023-11-28T18:24:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.122+0000 7f404f380640 1 -- 172.21.15.105:0/2845821441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f404815f000 con 0x7f40480ce530 2023-11-28T18:24:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.122+0000 7f404f380640 1 -- 172.21.15.105:0/2845821441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404815f5c0 con 0x7f40480ce530 2023-11-28T18:24:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.122+0000 7f404f380640 1 -- 172.21.15.105:0/2845821441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4048067570 con 0x7f40480ce530 2023-11-28T18:24:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.122+0000 7f403dffb640 1 -- 172.21.15.105:0/2845821441 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f4040004030 con 0x7f40480ce530 2023-11-28T18:24:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.122+0000 7f403dffb640 1 --2- 172.21.15.105:0/2845821441 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f402c0668d0 0x7f402c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.126+0000 7f403dffb640 1 -- 172.21.15.105:0/2845821441 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f4040084be0 con 0x7f40480ce530 2023-11-28T18:24:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.126+0000 7f404d11c640 1 --2- 172.21.15.105:0/2845821441 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f402c0668d0 0x7f402c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.126+0000 7f404d11c640 1 --2- 172.21.15.105:0/2845821441 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f402c0668d0 0x7f402c068d90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f404400c880 tx=0x7f4044008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.126+0000 7f403dffb640 1 -- 172.21.15.105:0/2845821441 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f40400519a0 con 0x7f40480ce530 2023-11-28T18:24:40.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.290+0000 7f404f380640 1 -- 172.21.15.105:0/2845821441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4048073690 con 0x7f40480ce530 2023-11-28T18:24:40.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.290+0000 7f403dffb640 1 -- 172.21.15.105:0/2845821441 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f404001a020 con 0x7f40480ce530 2023-11-28T18:24:40.295 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2023-11-28T18:24:40.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.290+0000 7f404f380640 1 -- 172.21.15.105:0/2845821441 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f402c0668d0 msgr2=0x7f402c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:40.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.290+0000 7f404f380640 1 --2- 172.21.15.105:0/2845821441 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f402c0668d0 0x7f402c068d90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f404400c880 tx=0x7f4044008040 comp rx=0 tx=0).stop 2023-11-28T18:24:40.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.290+0000 7f404f380640 1 -- 172.21.15.105:0/2845821441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40480ce530 msgr2=0x7f404815cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:40.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.290+0000 7f404f380640 1 --2- 172.21.15.105:0/2845821441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40480ce530 0x7f404815cdc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4040007ef0 tx=0x7f404000d4a0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.290+0000 7f404f380640 1 -- 172.21.15.105:0/2845821441 shutdown_connections 2023-11-28T18:24:40.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.290+0000 7f404f380640 1 --2- 172.21.15.105:0/2845821441 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f402c0668d0 0x7f402c068d90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.290+0000 7f404f380640 1 --2- 172.21.15.105:0/2845821441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40480ce530 0x7f404815cdc0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.290+0000 7f404f380640 1 --2- 172.21.15.105:0/2845821441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40480adf30 0x7f404815c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.290+0000 7f404f380640 1 --2- 172.21.15.105:0/2845821441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f404806d610 0x7f40481550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.290+0000 7f404f380640 1 -- 172.21.15.105:0/2845821441 >> 172.21.15.105:0/2845821441 conn(0x7f40480ceb90 msgr2=0x7f4048072f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:40.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.290+0000 7f404f380640 1 -- 172.21.15.105:0/2845821441 shutdown_connections 2023-11-28T18:24:40.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.290+0000 7f404f380640 1 -- 172.21.15.105:0/2845821441 wait complete. 2023-11-28T18:24:40.316 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi148.front.sepia.ceph.com... 2023-11-28T18:24:40.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:40.316 DEBUG:teuthology.orchestra.run.smithi148:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-11-28T18:24:40.342 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:24:40.342 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:24:40.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:24:40.342 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T18:24:40.396 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T18:24:40.396 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:24:40.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.530+0000 7f231179d640 1 Processor -- start 2023-11-28T18:24:40.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.530+0000 7f231179d640 1 -- start start 2023-11-28T18:24:40.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.530+0000 7f231179d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f230c15d4f0 0x7f230c0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:40.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.530+0000 7f231179d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f230c0b25b0 con 0x7f230c0b2d20 2023-11-28T18:24:40.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.530+0000 7f231179d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f230c0cea30 con 0x7f230c0d3c30 2023-11-28T18:24:40.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.530+0000 7f231179d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f230c0b2900 con 0x7f230c15d4f0 2023-11-28T18:24:40.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.530+0000 7f230affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f230c15d4f0 0x7f230c0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:40.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.530+0000 7f230b7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f230c0b2d20 0x7f230c0b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35448/0 (socket says 172.21.15.105:35448) 2023-11-28T18:24:40.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.530+0000 7f230b7fe640 1 -- 172.21.15.105:0/3358817125 learned_addr learned my addr 172.21.15.105:0/3358817125 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:40.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.530+0000 7f230affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f230c15d4f0 0x7f230c0d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33744/0 (socket says 172.21.15.105:33744) 2023-11-28T18:24:40.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.530+0000 7f230affd640 1 -- 172.21.15.105:0/3358817125 >> v1:172.21.15.148:6789/0 conn(0x7f230c0d3c30 legacy=0x7f230c0d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:40.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.530+0000 7f230affd640 1 -- 172.21.15.105:0/3358817125 >> v1:172.21.15.105:6789/0 conn(0x7f230c0b2d20 legacy=0x7f230c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:40.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.530+0000 7f230affd640 1 -- 172.21.15.105:0/3358817125 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230c0ced10 con 0x7f230c15d4f0 2023-11-28T18:24:40.538 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.534+0000 7f230affd640 1 --2- 172.21.15.105:0/3358817125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230c15d4f0 0x7f230c0d0e10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f22f40097c0 tx=0x7f22f401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16acea6409ed45d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:40.539 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.534+0000 7f230a7fc640 1 -- 172.21.15.105:0/3358817125 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f40ad660 con 0x7f230c15d4f0 2023-11-28T18:24:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.534+0000 7f230a7fc640 1 -- 172.21.15.105:0/3358817125 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f40ad7c0 con 0x7f230c15d4f0 2023-11-28T18:24:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.534+0000 7f230a7fc640 1 -- 172.21.15.105:0/3358817125 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f40adb50 con 0x7f230c15d4f0 2023-11-28T18:24:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.534+0000 7f231179d640 1 -- 172.21.15.105:0/3358817125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230c15d4f0 msgr2=0x7f230c0d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.534+0000 7f231179d640 1 --2- 172.21.15.105:0/3358817125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230c15d4f0 0x7f230c0d0e10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f22f40097c0 tx=0x7f22f401c3c0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.534+0000 7f231179d640 1 -- 172.21.15.105:0/3358817125 shutdown_connections 2023-11-28T18:24:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.534+0000 7f231179d640 1 --2- 172.21.15.105:0/3358817125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230c15d4f0 0x7f230c0d0e10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.534+0000 7f231179d640 1 -- 172.21.15.105:0/3358817125 >> 172.21.15.105:0/3358817125 conn(0x7f230c0d3030 msgr2=0x7f230c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.534+0000 7f231179d640 1 -- 172.21.15.105:0/3358817125 shutdown_connections 2023-11-28T18:24:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.534+0000 7f231179d640 1 -- 172.21.15.105:0/3358817125 wait complete. 2023-11-28T18:24:40.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f231179d640 1 Processor -- start 2023-11-28T18:24:40.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f231179d640 1 -- start start 2023-11-28T18:24:40.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f231179d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f230c0b2d20 0x7f230c13de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:40.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f231179d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f230c0d3c30 0x7f230c15c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:40.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f231179d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230c15d4f0 0x7f230c0b7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:40.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f231179d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c0d0a20 con 0x7f230c15d4f0 2023-11-28T18:24:40.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f231179d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c0b6d90 con 0x7f230c0d3c30 2023-11-28T18:24:40.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f231179d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230c159e60 con 0x7f230c0b2d20 2023-11-28T18:24:40.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f230b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230c15d4f0 0x7f230c0b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:40.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f230b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230c15d4f0 0x7f230c0b7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33752/0 (socket says 172.21.15.105:33752) 2023-11-28T18:24:40.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f230b7fe640 1 -- 172.21.15.105:0/1852325614 learned_addr learned my addr 172.21.15.105:0/1852325614 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:40.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f230affd640 1 --2- 172.21.15.105:0/1852325614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f230c0b2d20 0x7f230c13de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:40.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f23027fc640 1 --2- 172.21.15.105:0/1852325614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f230c0d3c30 0x7f230c15c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:40.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f230b7fe640 1 -- 172.21.15.105:0/1852325614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f230c0b2d20 msgr2=0x7f230c13de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:40.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f230b7fe640 1 --2- 172.21.15.105:0/1852325614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f230c0b2d20 0x7f230c13de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f230b7fe640 1 -- 172.21.15.105:0/1852325614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f230c0d3c30 msgr2=0x7f230c15c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:40.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f230b7fe640 1 --2- 172.21.15.105:0/1852325614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f230c0d3c30 0x7f230c15c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f230b7fe640 1 -- 172.21.15.105:0/1852325614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f4009420 con 0x7f230c15d4f0 2023-11-28T18:24:40.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f230affd640 1 --2- 172.21.15.105:0/1852325614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f230c0b2d20 0x7f230c13de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:40.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f23027fc640 1 --2- 172.21.15.105:0/1852325614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f230c0d3c30 0x7f230c15c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:40.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f230b7fe640 1 --2- 172.21.15.105:0/1852325614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230c15d4f0 0x7f230c0b7220 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f230400cab0 tx=0x7f230400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:40.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f2303fff640 1 -- 172.21.15.105:0/1852325614 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230400ac30 con 0x7f230c15d4f0 2023-11-28T18:24:40.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f2303fff640 1 -- 172.21.15.105:0/1852325614 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230400ad90 con 0x7f230c15d4f0 2023-11-28T18:24:40.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f231179d640 1 -- 172.21.15.105:0/1852325614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f230c159b80 con 0x7f230c15d4f0 2023-11-28T18:24:40.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f2303fff640 1 -- 172.21.15.105:0/1852325614 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2304012600 con 0x7f230c15d4f0 2023-11-28T18:24:40.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.538+0000 7f231179d640 1 -- 172.21.15.105:0/1852325614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f230c0b6a20 con 0x7f230c15d4f0 2023-11-28T18:24:40.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.542+0000 7f231179d640 1 -- 172.21.15.105:0/1852325614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f230c0b65a0 con 0x7f230c15d4f0 2023-11-28T18:24:40.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.542+0000 7f2303fff640 1 -- 172.21.15.105:0/1852325614 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f2304012760 con 0x7f230c15d4f0 2023-11-28T18:24:40.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.542+0000 7f2303fff640 1 --2- 172.21.15.105:0/1852325614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f22ec066900 0x7f22ec068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:40.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.542+0000 7f2303fff640 1 -- 172.21.15.105:0/1852325614 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f23040845a0 con 0x7f230c15d4f0 2023-11-28T18:24:40.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.542+0000 7f230affd640 1 --2- 172.21.15.105:0/1852325614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f22ec066900 0x7f22ec068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:40.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.542+0000 7f230affd640 1 --2- 172.21.15.105:0/1852325614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f22ec066900 0x7f22ec068dc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f22f40af000 tx=0x7f22f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:40.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.546+0000 7f2303fff640 1 -- 172.21.15.105:0/1852325614 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f230c0b65a0 con 0x7f230c15d4f0 2023-11-28T18:24:40.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.726+0000 7f231179d640 1 -- 172.21.15.105:0/1852325614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f230c0b65a0 con 0x7f230c15d4f0 2023-11-28T18:24:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.726+0000 7f2303fff640 1 -- 172.21.15.105:0/1852325614 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7f230c0b65a0 con 0x7f230c15d4f0 2023-11-28T18:24:40.731 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:40.731 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/390529893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":390529893},{"type":"v1","addr":"172.21.15.148:6835","nonce":390529893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3924571887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3924571887},{"type":"v1","addr":"172.21.15.148:6833","nonce":3924571887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3271971772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3271971772},{"type":"v1","addr":"172.21.15.105:6835","nonce":3271971772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:23:33.288743+0000","modified":"2023-11-28T18:23:50.425976+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2504263238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2504263238},{"type":"v1","addr":"172.21.15.105:6837","nonce":2504263238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T18:24:40.732 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 9 2023-11-28T18:24:40.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.726+0000 7f231179d640 1 -- 172.21.15.105:0/1852325614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f22ec066900 msgr2=0x7f22ec068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:40.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.726+0000 7f231179d640 1 --2- 172.21.15.105:0/1852325614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f22ec066900 0x7f22ec068dc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f22f40af000 tx=0x7f22f4008040 comp rx=0 tx=0).stop 2023-11-28T18:24:40.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.726+0000 7f231179d640 1 -- 172.21.15.105:0/1852325614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230c15d4f0 msgr2=0x7f230c0b7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:40.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.726+0000 7f231179d640 1 --2- 172.21.15.105:0/1852325614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230c15d4f0 0x7f230c0b7220 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f230400cab0 tx=0x7f230400cf70 comp rx=0 tx=0).stop 2023-11-28T18:24:40.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.726+0000 7f231179d640 1 -- 172.21.15.105:0/1852325614 shutdown_connections 2023-11-28T18:24:40.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.726+0000 7f231179d640 1 --2- 172.21.15.105:0/1852325614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f22ec066900 0x7f22ec068dc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.726+0000 7f231179d640 1 --2- 172.21.15.105:0/1852325614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230c15d4f0 0x7f230c0b7220 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.726+0000 7f231179d640 1 --2- 172.21.15.105:0/1852325614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f230c0d3c30 0x7f230c15c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.726+0000 7f231179d640 1 --2- 172.21.15.105:0/1852325614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f230c0b2d20 0x7f230c13de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.726+0000 7f231179d640 1 -- 172.21.15.105:0/1852325614 >> 172.21.15.105:0/1852325614 conn(0x7f230c0d3030 msgr2=0x7f230c0b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:40.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.730+0000 7f231179d640 1 -- 172.21.15.105:0/1852325614 shutdown_connections 2023-11-28T18:24:40.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.730+0000 7f231179d640 1 -- 172.21.15.105:0/1852325614 wait complete. 2023-11-28T18:24:40.753 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:24:40.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.886+0000 7f8a34a5e640 1 Processor -- start 2023-11-28T18:24:40.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.886+0000 7f8a34a5e640 1 -- start start 2023-11-28T18:24:40.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.886+0000 7f8a34a5e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a301575b0 0x7f8a3015ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:40.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.886+0000 7f8a34a5e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a30157990 0x7f8a300c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.886+0000 7f8a34a5e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a3015e0b0 con 0x7f8a3015d7b0 2023-11-28T18:24:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.886+0000 7f8a34a5e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a3015e390 con 0x7f8a30157990 2023-11-28T18:24:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.886+0000 7f8a34a5e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3015e670 con 0x7f8a301575b0 2023-11-28T18:24:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.886+0000 7f8a2659c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a30157990 0x7f8a300c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.886+0000 7f8a2659c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a30157990 0x7f8a300c9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33760/0 (socket says 172.21.15.105:33760) 2023-11-28T18:24:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.886+0000 7f8a2659c640 1 -- 172.21.15.105:0/4075653914 learned_addr learned my addr 172.21.15.105:0/4075653914 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.886+0000 7f8a2ed9d640 1 --2- 172.21.15.105:0/4075653914 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a301575b0 0x7f8a3015ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.886+0000 7f8a2659c640 1 -- 172.21.15.105:0/4075653914 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a301575b0 msgr2=0x7f8a3015ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.886+0000 7f8a2659c640 1 --2- 172.21.15.105:0/4075653914 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a301575b0 0x7f8a3015ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.886+0000 7f8a2659c640 1 -- 172.21.15.105:0/4075653914 >> v1:172.21.15.105:6789/0 conn(0x7f8a3015d7b0 legacy=0x7f8a3006cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:40.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.886+0000 7f8a2659c640 1 -- 172.21.15.105:0/4075653914 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a300c9750 con 0x7f8a30157990 2023-11-28T18:24:40.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.890+0000 7f8a2659c640 1 --2- 172.21.15.105:0/4075653914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a30157990 0x7f8a300c9f50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009e50 tx=0x7f8a1801c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e62d9525dc970c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:40.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.890+0000 7f8a2e59c640 1 -- 172.21.15.105:0/4075653914 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a180a57d0 con 0x7f8a30157990 2023-11-28T18:24:40.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.890+0000 7f8a2e59c640 1 -- 172.21.15.105:0/4075653914 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a180a5930 con 0x7f8a30157990 2023-11-28T18:24:40.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.890+0000 7f8a2e59c640 1 -- 172.21.15.105:0/4075653914 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a180a5cc0 con 0x7f8a30157990 2023-11-28T18:24:40.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.890+0000 7f8a34a5e640 1 -- 172.21.15.105:0/4075653914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a30157990 msgr2=0x7f8a300c9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:40.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.890+0000 7f8a34a5e640 1 --2- 172.21.15.105:0/4075653914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a30157990 0x7f8a300c9f50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009e50 tx=0x7f8a1801c9e0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.890+0000 7f8a34a5e640 1 -- 172.21.15.105:0/4075653914 shutdown_connections 2023-11-28T18:24:40.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.890+0000 7f8a34a5e640 1 --2- 172.21.15.105:0/4075653914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a30157990 0x7f8a300c9f50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.890+0000 7f8a34a5e640 1 --2- 172.21.15.105:0/4075653914 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a301575b0 0x7f8a3015ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.890+0000 7f8a34a5e640 1 -- 172.21.15.105:0/4075653914 >> 172.21.15.105:0/4075653914 conn(0x7f8a300b0480 msgr2=0x7f8a300b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:40.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.890+0000 7f8a34a5e640 1 -- 172.21.15.105:0/4075653914 shutdown_connections 2023-11-28T18:24:40.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.890+0000 7f8a34a5e640 1 -- 172.21.15.105:0/4075653914 wait complete. 2023-11-28T18:24:40.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a34a5e640 1 Processor -- start 2023-11-28T18:24:40.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a34a5e640 1 -- start start 2023-11-28T18:24:40.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a34a5e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a301575b0 0x7f8a301f50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:40.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a34a5e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a30157990 0x7f8a301591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:40.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a34a5e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a3015d7b0 0x7f8a301596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:40.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a34a5e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a300c1820 con 0x7f8a3015d7b0 2023-11-28T18:24:40.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a34a5e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a301f9a10 con 0x7f8a301575b0 2023-11-28T18:24:40.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a34a5e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a300c4640 con 0x7f8a30157990 2023-11-28T18:24:40.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a2f59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a3015d7b0 0x7f8a301596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:40.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a2ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a301575b0 0x7f8a301f50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:40.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a2f59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a3015d7b0 0x7f8a301596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33770/0 (socket says 172.21.15.105:33770) 2023-11-28T18:24:40.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a2f59e640 1 -- 172.21.15.105:0/35793976 learned_addr learned my addr 172.21.15.105:0/35793976 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:40.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a2659c640 1 --2- 172.21.15.105:0/35793976 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a30157990 0x7f8a301591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:40.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a2f59e640 1 -- 172.21.15.105:0/35793976 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a30157990 msgr2=0x7f8a301591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:40.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a2f59e640 1 --2- 172.21.15.105:0/35793976 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a30157990 0x7f8a301591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a2f59e640 1 -- 172.21.15.105:0/35793976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a301575b0 msgr2=0x7f8a301f50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:40.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a2f59e640 1 --2- 172.21.15.105:0/35793976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a301575b0 0x7f8a301f50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:40.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a2f59e640 1 -- 172.21.15.105:0/35793976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a18009b30 con 0x7f8a3015d7b0 2023-11-28T18:24:40.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a2ed9d640 1 --2- 172.21.15.105:0/35793976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a301575b0 0x7f8a301f50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:24:40.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a2f59e640 1 --2- 172.21.15.105:0/35793976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a3015d7b0 0x7f8a301596f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800c430 tx=0x7f8a2800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:40.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a27fff640 1 -- 172.21.15.105:0/35793976 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a28005220 con 0x7f8a3015d7b0 2023-11-28T18:24:40.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a27fff640 1 -- 172.21.15.105:0/35793976 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a28005380 con 0x7f8a3015d7b0 2023-11-28T18:24:40.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a27fff640 1 -- 172.21.15.105:0/35793976 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a28007270 con 0x7f8a3015d7b0 2023-11-28T18:24:40.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a34a5e640 1 -- 172.21.15.105:0/35793976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a300c4920 con 0x7f8a3015d7b0 2023-11-28T18:24:40.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.894+0000 7f8a34a5e640 1 -- 172.21.15.105:0/35793976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a300c42f0 con 0x7f8a3015d7b0 2023-11-28T18:24:40.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.898+0000 7f8a34a5e640 1 -- 172.21.15.105:0/35793976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a30074580 con 0x7f8a3015d7b0 2023-11-28T18:24:40.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.898+0000 7f8a27fff640 1 -- 172.21.15.105:0/35793976 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8a280073d0 con 0x7f8a3015d7b0 2023-11-28T18:24:40.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.898+0000 7f8a27fff640 1 --2- 172.21.15.105:0/35793976 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a100668d0 0x7f8a10068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:40.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.898+0000 7f8a27fff640 1 -- 172.21.15.105:0/35793976 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f8a28083ca0 con 0x7f8a3015d7b0 2023-11-28T18:24:40.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.898+0000 7f8a2ed9d640 1 --2- 172.21.15.105:0/35793976 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a100668d0 0x7f8a10068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:40.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.898+0000 7f8a2ed9d640 1 --2- 172.21.15.105:0/35793976 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a100668d0 0x7f8a10068d90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009eb0 tx=0x7f8a20009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:40.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:40.902+0000 7f8a27fff640 1 -- 172.21.15.105:0/35793976 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8a28050b80 con 0x7f8a3015d7b0 2023-11-28T18:24:41.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.050+0000 7f8a34a5e640 1 -- 172.21.15.105:0/35793976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a300c4dd0 con 0x7f8a3015d7b0 2023-11-28T18:24:41.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.054+0000 7f8a27fff640 1 -- 172.21.15.105:0/35793976 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18887 (secure 0 0 0) 0x7f8a280541a0 con 0x7f8a3015d7b0 2023-11-28T18:24:41.058 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:41.058 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:24:15.048168+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:24:41.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.054+0000 7f8a34a5e640 1 -- 172.21.15.105:0/35793976 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a100668d0 msgr2=0x7f8a10068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:41.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.054+0000 7f8a34a5e640 1 --2- 172.21.15.105:0/35793976 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a100668d0 0x7f8a10068d90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009eb0 tx=0x7f8a20009190 comp rx=0 tx=0).stop 2023-11-28T18:24:41.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.054+0000 7f8a34a5e640 1 -- 172.21.15.105:0/35793976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a3015d7b0 msgr2=0x7f8a301596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:41.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.054+0000 7f8a34a5e640 1 --2- 172.21.15.105:0/35793976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a3015d7b0 0x7f8a301596f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800c430 tx=0x7f8a2800c8f0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.054+0000 7f8a34a5e640 1 -- 172.21.15.105:0/35793976 shutdown_connections 2023-11-28T18:24:41.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.054+0000 7f8a34a5e640 1 --2- 172.21.15.105:0/35793976 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a100668d0 0x7f8a10068d90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.054+0000 7f8a34a5e640 1 --2- 172.21.15.105:0/35793976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a3015d7b0 0x7f8a301596f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.054+0000 7f8a34a5e640 1 --2- 172.21.15.105:0/35793976 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a30157990 0x7f8a301591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.054+0000 7f8a34a5e640 1 --2- 172.21.15.105:0/35793976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a301575b0 0x7f8a301f50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.054+0000 7f8a34a5e640 1 -- 172.21.15.105:0/35793976 >> 172.21.15.105:0/35793976 conn(0x7f8a300b0480 msgr2=0x7f8a300cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:41.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.054+0000 7f8a34a5e640 1 -- 172.21.15.105:0/35793976 shutdown_connections 2023-11-28T18:24:41.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.054+0000 7f8a34a5e640 1 -- 172.21.15.105:0/35793976 wait complete. 2023-11-28T18:24:41.079 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:23:31.485422+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:23:32.274378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T18:23:36.100071+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T18:24:41.080 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:24:41.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.214+0000 7ff4932fe640 1 Processor -- start 2023-11-28T18:24:41.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.218+0000 7ff4932fe640 1 -- start start 2023-11-28T18:24:41.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.218+0000 7ff4932fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff48c0b4290 0x7ff48c0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:41.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.218+0000 7ff4932fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48c0b4bb0 con 0x7ff48c071960 2023-11-28T18:24:41.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.218+0000 7ff4932fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48c0701d0 con 0x7ff48c15b030 2023-11-28T18:24:41.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.218+0000 7ff4932fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c06df80 con 0x7ff48c0b4290 2023-11-28T18:24:41.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.218+0000 7ff490899640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff48c071960 0x7ff48c15f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59802/0 (socket says 172.21.15.105:59802) 2023-11-28T18:24:41.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.218+0000 7ff490899640 1 -- 172.21.15.105:0/177150930 learned_addr learned my addr 172.21.15.105:0/177150930 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:41.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.218+0000 7ff49109a640 1 --2- 172.21.15.105:0/177150930 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff48c0b4290 0x7ff48c0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:41.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.218+0000 7ff483fff640 1 -- 172.21.15.105:0/177150930 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3257671460 0 0) 0x7ff48c0b4bb0 con 0x7ff48c071960 2023-11-28T18:24:41.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.218+0000 7ff483fff640 1 -- 172.21.15.105:0/177150930 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff478003530 con 0x7ff48c071960 2023-11-28T18:24:41.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.218+0000 7ff483fff640 1 -- 172.21.15.105:0/177150930 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2583875574 0 0) 0x7ff48c0701d0 con 0x7ff48c15b030 2023-11-28T18:24:41.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.218+0000 7ff483fff640 1 -- 172.21.15.105:0/177150930 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff48c0b4bb0 con 0x7ff48c15b030 2023-11-28T18:24:41.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.218+0000 7ff49109a640 1 -- 172.21.15.105:0/177150930 >> v1:172.21.15.148:6789/0 conn(0x7ff48c15b030 legacy=0x7ff48c0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:41.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.218+0000 7ff49109a640 1 -- 172.21.15.105:0/177150930 >> v1:172.21.15.105:6789/0 conn(0x7ff48c071960 legacy=0x7ff48c15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:41.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.218+0000 7ff49109a640 1 -- 172.21.15.105:0/177150930 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c071df0 con 0x7ff48c0b4290 2023-11-28T18:24:41.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.222+0000 7ff49109a640 1 --2- 172.21.15.105:0/177150930 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff48c0b4290 0x7ff48c0b4670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff4880097f0 tx=0x7ff48801c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d01e05908589f33c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:41.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.222+0000 7ff483fff640 1 -- 172.21.15.105:0/177150930 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4880aa5f0 con 0x7ff48c0b4290 2023-11-28T18:24:41.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.222+0000 7ff483fff640 1 -- 172.21.15.105:0/177150930 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4880aa750 con 0x7ff48c0b4290 2023-11-28T18:24:41.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.222+0000 7ff483fff640 1 -- 172.21.15.105:0/177150930 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4880aaae0 con 0x7ff48c0b4290 2023-11-28T18:24:41.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.222+0000 7ff4932fe640 1 -- 172.21.15.105:0/177150930 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff48c0b4290 msgr2=0x7ff48c0b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:41.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.222+0000 7ff4932fe640 1 --2- 172.21.15.105:0/177150930 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff48c0b4290 0x7ff48c0b4670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff4880097f0 tx=0x7ff48801c490 comp rx=0 tx=0).stop 2023-11-28T18:24:41.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.222+0000 7ff4932fe640 1 -- 172.21.15.105:0/177150930 shutdown_connections 2023-11-28T18:24:41.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.222+0000 7ff4932fe640 1 --2- 172.21.15.105:0/177150930 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff48c0b4290 0x7ff48c0b4670 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.222+0000 7ff4932fe640 1 -- 172.21.15.105:0/177150930 >> 172.21.15.105:0/177150930 conn(0x7ff48c0ceb90 msgr2=0x7ff48c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:41.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.222+0000 7ff4932fe640 1 -- 172.21.15.105:0/177150930 shutdown_connections 2023-11-28T18:24:41.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.222+0000 7ff4932fe640 1 -- 172.21.15.105:0/177150930 wait complete. 2023-11-28T18:24:41.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff4932fe640 1 Processor -- start 2023-11-28T18:24:41.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff4932fe640 1 -- start start 2023-11-28T18:24:41.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff4932fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff48c0b4290 0x7ff48c151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:41.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff4932fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff48c15b030 0x7ff48c0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:41.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff4932fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff48c1534a0 0x7ff48c158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:41.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff4932fe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c0c80e0 con 0x7ff48c0b4290 2023-11-28T18:24:41.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff4932fe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c0c83c0 con 0x7ff48c1534a0 2023-11-28T18:24:41.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff4932fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c0c86a0 con 0x7ff48c15b030 2023-11-28T18:24:41.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff49109a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff48c0b4290 0x7ff48c151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:41.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff49109a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff48c0b4290 0x7ff48c151910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58812/0 (socket says 172.21.15.105:58812) 2023-11-28T18:24:41.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff49109a640 1 -- 172.21.15.105:0/3783379080 learned_addr learned my addr 172.21.15.105:0/3783379080 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:41.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff490899640 1 --2- 172.21.15.105:0/3783379080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff48c15b030 0x7ff48c0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:41.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff49189b640 1 --2- 172.21.15.105:0/3783379080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff48c1534a0 0x7ff48c158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:41.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff49109a640 1 -- 172.21.15.105:0/3783379080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff48c15b030 msgr2=0x7ff48c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:41.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff49109a640 1 --2- 172.21.15.105:0/3783379080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff48c15b030 0x7ff48c0ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff49109a640 1 -- 172.21.15.105:0/3783379080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff48c1534a0 msgr2=0x7ff48c158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:41.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff49109a640 1 --2- 172.21.15.105:0/3783379080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff48c1534a0 0x7ff48c158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff49109a640 1 -- 172.21.15.105:0/3783379080 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff488009420 con 0x7ff48c0b4290 2023-11-28T18:24:41.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff49109a640 1 --2- 172.21.15.105:0/3783379080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff48c0b4290 0x7ff48c151910 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff4880190a0 tx=0x7ff48801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:41.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff481ffb640 1 -- 172.21.15.105:0/3783379080 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4880be610 con 0x7ff48c0b4290 2023-11-28T18:24:41.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff481ffb640 1 -- 172.21.15.105:0/3783379080 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4880be790 con 0x7ff48c0b4290 2023-11-28T18:24:41.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff481ffb640 1 -- 172.21.15.105:0/3783379080 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4880beb40 con 0x7ff48c0b4290 2023-11-28T18:24:41.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff4932fe640 1 -- 172.21.15.105:0/3783379080 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48c0c8980 con 0x7ff48c0b4290 2023-11-28T18:24:41.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff4932fe640 1 -- 172.21.15.105:0/3783379080 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48c0ce6a0 con 0x7ff48c0b4290 2023-11-28T18:24:41.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.226+0000 7ff4932fe640 1 -- 172.21.15.105:0/3783379080 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff48c067570 con 0x7ff48c0b4290 2023-11-28T18:24:41.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.230+0000 7ff481ffb640 1 -- 172.21.15.105:0/3783379080 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff4880b4400 con 0x7ff48c0b4290 2023-11-28T18:24:41.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.230+0000 7ff481ffb640 1 --2- 172.21.15.105:0/3783379080 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff4700668d0 0x7ff470068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:41.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.230+0000 7ff481ffb640 1 -- 172.21.15.105:0/3783379080 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7ff4880a7070 con 0x7ff48c0b4290 2023-11-28T18:24:41.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.230+0000 7ff490899640 1 --2- 172.21.15.105:0/3783379080 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff4700668d0 0x7ff470068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:41.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.230+0000 7ff490899640 1 --2- 172.21.15.105:0/3783379080 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff4700668d0 0x7ff470068d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff48c157290 tx=0x7ff47c007d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:41.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.234+0000 7ff481ffb640 1 -- 172.21.15.105:0/3783379080 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff4880ec3e0 con 0x7ff48c0b4290 2023-11-28T18:24:41.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.386+0000 7ff4932fe640 1 -- 172.21.15.105:0/3783379080 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff48c0c87e0 con 0x7ff48c0b4290 2023-11-28T18:24:41.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.386+0000 7ff481ffb640 1 -- 172.21.15.105:0/3783379080 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff48c0c87e0 con 0x7ff48c0b4290 2023-11-28T18:24:41.392 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:24:41.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.390+0000 7ff4932fe640 1 -- 172.21.15.105:0/3783379080 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff4700668d0 msgr2=0x7ff470068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:41.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.390+0000 7ff4932fe640 1 --2- 172.21.15.105:0/3783379080 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff4700668d0 0x7ff470068d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff48c157290 tx=0x7ff47c007d80 comp rx=0 tx=0).stop 2023-11-28T18:24:41.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.390+0000 7ff4932fe640 1 -- 172.21.15.105:0/3783379080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff48c0b4290 msgr2=0x7ff48c151910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:41.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.390+0000 7ff4932fe640 1 --2- 172.21.15.105:0/3783379080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff48c0b4290 0x7ff48c151910 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff4880190a0 tx=0x7ff48801acd0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.390+0000 7ff4932fe640 1 -- 172.21.15.105:0/3783379080 shutdown_connections 2023-11-28T18:24:41.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.390+0000 7ff4932fe640 1 --2- 172.21.15.105:0/3783379080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff48c1534a0 0x7ff48c158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.390+0000 7ff4932fe640 1 --2- 172.21.15.105:0/3783379080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff48c15b030 0x7ff48c0ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.390+0000 7ff4932fe640 1 --2- 172.21.15.105:0/3783379080 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff4700668d0 0x7ff470068d90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.390+0000 7ff4932fe640 1 --2- 172.21.15.105:0/3783379080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff48c0b4290 0x7ff48c151910 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.390+0000 7ff4932fe640 1 -- 172.21.15.105:0/3783379080 >> 172.21.15.105:0/3783379080 conn(0x7ff48c0ceb90 msgr2=0x7ff48c0af010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:41.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.390+0000 7ff4932fe640 1 -- 172.21.15.105:0/3783379080 shutdown_connections 2023-11-28T18:24:41.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.390+0000 7ff4932fe640 1 -- 172.21.15.105:0/3783379080 wait complete. 2023-11-28T18:24:41.413 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:24:41.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.546+0000 7f39380fe640 1 Processor -- start 2023-11-28T18:24:41.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.546+0000 7f39380fe640 1 -- start start 2023-11-28T18:24:41.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.550+0000 7f39380fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f39300b4290 0x7f39300b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:41.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.550+0000 7f39380fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3930071960 0x7f393015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:41.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.550+0000 7f39380fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39300b4bb0 con 0x7f393015b030 2023-11-28T18:24:41.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.550+0000 7f39380fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39300701d0 con 0x7f39300b4290 2023-11-28T18:24:41.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.550+0000 7f39380fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f393006df80 con 0x7f3930071960 2023-11-28T18:24:41.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.550+0000 7f3935e9a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f39300b4290 0x7f39300b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:41.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.550+0000 7f3935e9a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f39300b4290 0x7f39300b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58826/0 (socket says 172.21.15.105:58826) 2023-11-28T18:24:41.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.550+0000 7f3935e9a640 1 -- 172.21.15.105:0/1068281774 learned_addr learned my addr 172.21.15.105:0/1068281774 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:41.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.550+0000 7f3935e9a640 1 -- 172.21.15.105:0/1068281774 >> v2:172.21.15.148:3300/0 conn(0x7f3930071960 msgr2=0x7f393015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:41.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.550+0000 7f3935699640 1 --2- 172.21.15.105:0/1068281774 >> v2:172.21.15.148:3300/0 conn(0x7f3930071960 0x7f393015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:41.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.550+0000 7f3935e9a640 1 --2- 172.21.15.105:0/1068281774 >> v2:172.21.15.148:3300/0 conn(0x7f3930071960 0x7f393015f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.550+0000 7f3935e9a640 1 -- 172.21.15.105:0/1068281774 >> v1:172.21.15.148:6789/0 conn(0x7f393015b030 legacy=0x7f39300709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:41.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.550+0000 7f3935e9a640 1 -- 172.21.15.105:0/1068281774 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3930071df0 con 0x7f39300b4290 2023-11-28T18:24:41.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.550+0000 7f3935699640 1 --2- 172.21.15.105:0/1068281774 >> v2:172.21.15.148:3300/0 conn(0x7f3930071960 0x7f393015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:41.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.554+0000 7f3935e9a640 1 --2- 172.21.15.105:0/1068281774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39300b4290 0x7f39300b4670 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f393006e520 tx=0x7f392c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778698c0ed14ada3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:41.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.554+0000 7f3934e98640 1 -- 172.21.15.105:0/1068281774 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392c0aa5f0 con 0x7f39300b4290 2023-11-28T18:24:41.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.554+0000 7f3934e98640 1 -- 172.21.15.105:0/1068281774 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f392c0aa750 con 0x7f39300b4290 2023-11-28T18:24:41.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.554+0000 7f3934e98640 1 -- 172.21.15.105:0/1068281774 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392c0aaae0 con 0x7f39300b4290 2023-11-28T18:24:41.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.554+0000 7f39380fe640 1 -- 172.21.15.105:0/1068281774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39300b4290 msgr2=0x7f39300b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:41.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.554+0000 7f39380fe640 1 --2- 172.21.15.105:0/1068281774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39300b4290 0x7f39300b4670 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f393006e520 tx=0x7f392c01c490 comp rx=0 tx=0).stop 2023-11-28T18:24:41.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.554+0000 7f39380fe640 1 -- 172.21.15.105:0/1068281774 shutdown_connections 2023-11-28T18:24:41.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.554+0000 7f39380fe640 1 --2- 172.21.15.105:0/1068281774 >> v2:172.21.15.148:3300/0 conn(0x7f3930071960 0x7f393015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.554+0000 7f39380fe640 1 --2- 172.21.15.105:0/1068281774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39300b4290 0x7f39300b4670 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.554+0000 7f39380fe640 1 -- 172.21.15.105:0/1068281774 >> 172.21.15.105:0/1068281774 conn(0x7f39300ceb90 msgr2=0x7f39300cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:41.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.554+0000 7f39380fe640 1 -- 172.21.15.105:0/1068281774 shutdown_connections 2023-11-28T18:24:41.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.554+0000 7f39380fe640 1 -- 172.21.15.105:0/1068281774 wait complete. 2023-11-28T18:24:41.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.554+0000 7f39380fe640 1 Processor -- start 2023-11-28T18:24:41.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.554+0000 7f39380fe640 1 -- start start 2023-11-28T18:24:41.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.554+0000 7f39380fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39300b4290 0x7f393014f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:41.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f39380fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f393015b030 0x7f393015b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:41.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f39380fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f393014b3f0 0x7f393009b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:41.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f39380fe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393015c680 con 0x7f393014b3f0 2023-11-28T18:24:41.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f39380fe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393015bb00 con 0x7f39300b4290 2023-11-28T18:24:41.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f39380fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f393015bde0 con 0x7f393015b030 2023-11-28T18:24:41.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f393669b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f393014b3f0 0x7f393009b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:41.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f3935e9a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39300b4290 0x7f393014f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:41.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f393669b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f393014b3f0 0x7f393009b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58832/0 (socket says 172.21.15.105:58832) 2023-11-28T18:24:41.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f393669b640 1 -- 172.21.15.105:0/607193173 learned_addr learned my addr 172.21.15.105:0/607193173 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:41.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f3935699640 1 --2- 172.21.15.105:0/607193173 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f393015b030 0x7f393015b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:41.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f393669b640 1 -- 172.21.15.105:0/607193173 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f393015b030 msgr2=0x7f393015b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:41.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f393669b640 1 --2- 172.21.15.105:0/607193173 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f393015b030 0x7f393015b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f393669b640 1 -- 172.21.15.105:0/607193173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39300b4290 msgr2=0x7f393014f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:41.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f393669b640 1 --2- 172.21.15.105:0/607193173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39300b4290 0x7f393014f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f393669b640 1 -- 172.21.15.105:0/607193173 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392c0094d0 con 0x7f393014b3f0 2023-11-28T18:24:41.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f393669b640 1 --2- 172.21.15.105:0/607193173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f393014b3f0 0x7f393009b6e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f392800be30 tx=0x7f392800bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:41.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f39267fc640 1 -- 172.21.15.105:0/607193173 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392800caf0 con 0x7f393014b3f0 2023-11-28T18:24:41.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f39267fc640 1 -- 172.21.15.105:0/607193173 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f392800cc50 con 0x7f393014b3f0 2023-11-28T18:24:41.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f39380fe640 1 -- 172.21.15.105:0/607193173 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393015c0c0 con 0x7f393014b3f0 2023-11-28T18:24:41.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f39267fc640 1 -- 172.21.15.105:0/607193173 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3928014440 con 0x7f393014b3f0 2023-11-28T18:24:41.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f39380fe640 1 -- 172.21.15.105:0/607193173 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393014b880 con 0x7f393014b3f0 2023-11-28T18:24:41.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f39380fe640 1 -- 172.21.15.105:0/607193173 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39300af1e0 con 0x7f393014b3f0 2023-11-28T18:24:41.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.558+0000 7f39267fc640 1 -- 172.21.15.105:0/607193173 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f392801d3b0 con 0x7f393014b3f0 2023-11-28T18:24:41.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.562+0000 7f39267fc640 1 --2- 172.21.15.105:0/607193173 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3914066900 0x7f3914068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:41.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.562+0000 7f39267fc640 1 -- 172.21.15.105:0/607193173 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f3928084440 con 0x7f393014b3f0 2023-11-28T18:24:41.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.562+0000 7f3935e9a640 1 --2- 172.21.15.105:0/607193173 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3914066900 0x7f3914068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:41.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.562+0000 7f3935e9a640 1 --2- 172.21.15.105:0/607193173 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3914066900 0x7f3914068dc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f392c0ae000 tx=0x7f392c0b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:41.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.566+0000 7f39267fc640 1 -- 172.21.15.105:0/607193173 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f39300af1e0 con 0x7f393014b3f0 2023-11-28T18:24:41.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.714+0000 7f39380fe640 1 -- 172.21.15.105:0/607193173 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f39300af1e0 con 0x7f393014b3f0 2023-11-28T18:24:41.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.714+0000 7f39267fc640 1 -- 172.21.15.105:0/607193173 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3928051320 con 0x7f393014b3f0 2023-11-28T18:24:41.718 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:41.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.714+0000 7f39380fe640 1 -- 172.21.15.105:0/607193173 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3914066900 msgr2=0x7f3914068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:41.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.714+0000 7f39380fe640 1 --2- 172.21.15.105:0/607193173 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3914066900 0x7f3914068dc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f392c0ae000 tx=0x7f392c0b5040 comp rx=0 tx=0).stop 2023-11-28T18:24:41.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.714+0000 7f39380fe640 1 -- 172.21.15.105:0/607193173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f393014b3f0 msgr2=0x7f393009b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.714+0000 7f39380fe640 1 --2- 172.21.15.105:0/607193173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f393014b3f0 0x7f393009b6e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f392800be30 tx=0x7f392800bf20 comp rx=0 tx=0).stop 2023-11-28T18:24:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.714+0000 7f39380fe640 1 -- 172.21.15.105:0/607193173 shutdown_connections 2023-11-28T18:24:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.714+0000 7f39380fe640 1 --2- 172.21.15.105:0/607193173 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3914066900 0x7f3914068dc0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.714+0000 7f39380fe640 1 --2- 172.21.15.105:0/607193173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f393014b3f0 0x7f393009b6e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.714+0000 7f39380fe640 1 --2- 172.21.15.105:0/607193173 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f393015b030 0x7f393015b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.714+0000 7f39380fe640 1 --2- 172.21.15.105:0/607193173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39300b4290 0x7f393014f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.714+0000 7f39380fe640 1 -- 172.21.15.105:0/607193173 >> 172.21.15.105:0/607193173 conn(0x7f39300ceb90 msgr2=0x7f393015f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:41.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.714+0000 7f39380fe640 1 -- 172.21.15.105:0/607193173 shutdown_connections 2023-11-28T18:24:41.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.714+0000 7f39380fe640 1 -- 172.21.15.105:0/607193173 wait complete. 2023-11-28T18:24:41.739 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:24:41.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.874+0000 7f376f1d1640 1 Processor -- start 2023-11-28T18:24:41.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.874+0000 7f376f1d1640 1 -- start start 2023-11-28T18:24:41.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.874+0000 7f376f1d1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f376815d4f0 0x7f37680d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:41.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.874+0000 7f376f1d1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f37680b2d20 0x7f37680b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:41.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.874+0000 7f376f1d1640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37680b25b0 con 0x7f37680d3c30 2023-11-28T18:24:41.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.874+0000 7f376f1d1640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37680b2900 con 0x7f37680b2d20 2023-11-28T18:24:41.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.874+0000 7f376f1d1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37680d0a20 con 0x7f376815d4f0 2023-11-28T18:24:41.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.878+0000 7f376d76e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f37680b2d20 0x7f37680b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:41.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.878+0000 7f376d76e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f37680b2d20 0x7f37680b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58842/0 (socket says 172.21.15.105:58842) 2023-11-28T18:24:41.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.878+0000 7f376d76e640 1 -- 172.21.15.105:0/1881568464 learned_addr learned my addr 172.21.15.105:0/1881568464 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:41.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.878+0000 7f375ffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f37680d3c30 0x7f37680d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59816/0 (socket says 172.21.15.105:59816) 2023-11-28T18:24:41.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.878+0000 7f376cf6d640 1 --2- 172.21.15.105:0/1881568464 >> v2:172.21.15.148:3300/0 conn(0x7f376815d4f0 0x7f37680d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:41.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.878+0000 7f376d76e640 1 -- 172.21.15.105:0/1881568464 >> v2:172.21.15.148:3300/0 conn(0x7f376815d4f0 msgr2=0x7f37680d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:41.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.878+0000 7f376d76e640 1 --2- 172.21.15.105:0/1881568464 >> v2:172.21.15.148:3300/0 conn(0x7f376815d4f0 0x7f37680d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.878+0000 7f376d76e640 1 -- 172.21.15.105:0/1881568464 >> v1:172.21.15.105:6789/0 conn(0x7f37680d3c30 legacy=0x7f37680d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:41.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.878+0000 7f376d76e640 1 -- 172.21.15.105:0/1881568464 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37680cea30 con 0x7f37680b2d20 2023-11-28T18:24:41.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.878+0000 7f376d76e640 1 --2- 172.21.15.105:0/1881568464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37680b2d20 0x7f37680b3150 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f376808bbf0 tx=0x7f376001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aac5e420a10babe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:41.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.878+0000 7f375f7fe640 1 -- 172.21.15.105:0/1881568464 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37600a57d0 con 0x7f37680b2d20 2023-11-28T18:24:41.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.878+0000 7f375f7fe640 1 -- 172.21.15.105:0/1881568464 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37600a5930 con 0x7f37680b2d20 2023-11-28T18:24:41.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.878+0000 7f375f7fe640 1 -- 172.21.15.105:0/1881568464 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37600a5cc0 con 0x7f37680b2d20 2023-11-28T18:24:41.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376f1d1640 1 -- 172.21.15.105:0/1881568464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37680b2d20 msgr2=0x7f37680b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:41.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376f1d1640 1 --2- 172.21.15.105:0/1881568464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37680b2d20 0x7f37680b3150 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f376808bbf0 tx=0x7f376001c910 comp rx=0 tx=0).stop 2023-11-28T18:24:41.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376f1d1640 1 -- 172.21.15.105:0/1881568464 shutdown_connections 2023-11-28T18:24:41.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376f1d1640 1 --2- 172.21.15.105:0/1881568464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37680b2d20 0x7f37680b3150 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376f1d1640 1 --2- 172.21.15.105:0/1881568464 >> v2:172.21.15.148:3300/0 conn(0x7f376815d4f0 0x7f37680d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376f1d1640 1 -- 172.21.15.105:0/1881568464 >> 172.21.15.105:0/1881568464 conn(0x7f37680d3030 msgr2=0x7f37680d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:41.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376f1d1640 1 -- 172.21.15.105:0/1881568464 shutdown_connections 2023-11-28T18:24:41.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376f1d1640 1 -- 172.21.15.105:0/1881568464 wait complete. 2023-11-28T18:24:41.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376f1d1640 1 Processor -- start 2023-11-28T18:24:41.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376f1d1640 1 -- start start 2023-11-28T18:24:41.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376f1d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37680b2d20 0x7f376815b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:41.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376f1d1640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f37680d3c30 0x7f37680c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:41.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376f1d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f376815d4f0 0x7f37680ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:41.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376f1d1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37680c9ea0 con 0x7f376815d4f0 2023-11-28T18:24:41.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376f1d1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37680cab70 con 0x7f37680b2d20 2023-11-28T18:24:41.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376f1d1640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3768158a80 con 0x7f37680d3c30 2023-11-28T18:24:41.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376d76e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f376815d4f0 0x7f37680ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:41.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376d76e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f376815d4f0 0x7f37680ca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58856/0 (socket says 172.21.15.105:58856) 2023-11-28T18:24:41.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376d76e640 1 -- 172.21.15.105:0/1540021645 learned_addr learned my addr 172.21.15.105:0/1540021645 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:41.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376cf6d640 1 --2- 172.21.15.105:0/1540021645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37680b2d20 0x7f376815b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:41.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f375ffff640 1 --2- 172.21.15.105:0/1540021645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f37680d3c30 0x7f37680c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:41.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376d76e640 1 -- 172.21.15.105:0/1540021645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f37680d3c30 msgr2=0x7f37680c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:41.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376d76e640 1 --2- 172.21.15.105:0/1540021645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f37680d3c30 0x7f37680c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376d76e640 1 -- 172.21.15.105:0/1540021645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37680b2d20 msgr2=0x7f376815b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:41.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376d76e640 1 --2- 172.21.15.105:0/1540021645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37680b2d20 0x7f376815b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:41.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.882+0000 7f376d76e640 1 -- 172.21.15.105:0/1540021645 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3760009b30 con 0x7f376815d4f0 2023-11-28T18:24:41.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.886+0000 7f376d76e640 1 --2- 172.21.15.105:0/1540021645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f376815d4f0 0x7f37680ca100 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3760019520 tx=0x7f37600196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:41.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.886+0000 7f375d7fa640 1 -- 172.21.15.105:0/1540021645 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37600b8670 con 0x7f376815d4f0 2023-11-28T18:24:41.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.886+0000 7f375d7fa640 1 -- 172.21.15.105:0/1540021645 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37600b87d0 con 0x7f376815d4f0 2023-11-28T18:24:41.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.886+0000 7f375d7fa640 1 -- 172.21.15.105:0/1540021645 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37600b8b80 con 0x7f376815d4f0 2023-11-28T18:24:41.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.886+0000 7f376f1d1640 1 -- 172.21.15.105:0/1540021645 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3768158d60 con 0x7f376815d4f0 2023-11-28T18:24:41.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.886+0000 7f376f1d1640 1 -- 172.21.15.105:0/1540021645 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f376815b030 con 0x7f376815d4f0 2023-11-28T18:24:41.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.886+0000 7f376f1d1640 1 -- 172.21.15.105:0/1540021645 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3768067570 con 0x7f376815d4f0 2023-11-28T18:24:41.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.886+0000 7f375d7fa640 1 -- 172.21.15.105:0/1540021645 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f37600b8ce0 con 0x7f376815d4f0 2023-11-28T18:24:41.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.886+0000 7f375d7fa640 1 --2- 172.21.15.105:0/1540021645 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3754066900 0x7f3754068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:41.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.886+0000 7f375d7fa640 1 -- 172.21.15.105:0/1540021645 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f376011fd90 con 0x7f376815d4f0 2023-11-28T18:24:41.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.886+0000 7f376cf6d640 1 --2- 172.21.15.105:0/1540021645 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3754066900 0x7f3754068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:41.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.890+0000 7f376cf6d640 1 --2- 172.21.15.105:0/1540021645 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3754066900 0x7f3754068dc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3764009ac0 tx=0x7f3764009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:41.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:41.890+0000 7f375d7fa640 1 -- 172.21.15.105:0/1540021645 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f37600ecd10 con 0x7f376815d4f0 2023-11-28T18:24:42.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.042+0000 7f376f1d1640 1 -- 172.21.15.105:0/1540021645 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f37680b0070 con 0x7f376815d4f0 2023-11-28T18:24:42.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.042+0000 7f375d7fa640 1 -- 172.21.15.105:0/1540021645 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f37600bc020 con 0x7f376815d4f0 2023-11-28T18:24:42.048 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:24:42.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.046+0000 7f376f1d1640 1 -- 172.21.15.105:0/1540021645 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3754066900 msgr2=0x7f3754068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:42.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.046+0000 7f376f1d1640 1 --2- 172.21.15.105:0/1540021645 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3754066900 0x7f3754068dc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3764009ac0 tx=0x7f3764009190 comp rx=0 tx=0).stop 2023-11-28T18:24:42.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.046+0000 7f376f1d1640 1 -- 172.21.15.105:0/1540021645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f376815d4f0 msgr2=0x7f37680ca100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:42.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.046+0000 7f376f1d1640 1 --2- 172.21.15.105:0/1540021645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f376815d4f0 0x7f37680ca100 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3760019520 tx=0x7f37600196b0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.046+0000 7f376f1d1640 1 -- 172.21.15.105:0/1540021645 shutdown_connections 2023-11-28T18:24:42.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.046+0000 7f376f1d1640 1 --2- 172.21.15.105:0/1540021645 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3754066900 0x7f3754068dc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.046+0000 7f376f1d1640 1 --2- 172.21.15.105:0/1540021645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f376815d4f0 0x7f37680ca100 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.046+0000 7f376f1d1640 1 --2- 172.21.15.105:0/1540021645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f37680d3c30 0x7f37680c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.046+0000 7f376f1d1640 1 --2- 172.21.15.105:0/1540021645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37680b2d20 0x7f376815b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.046+0000 7f376f1d1640 1 -- 172.21.15.105:0/1540021645 >> 172.21.15.105:0/1540021645 conn(0x7f37680d3030 msgr2=0x7f376806c850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:42.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.046+0000 7f376f1d1640 1 -- 172.21.15.105:0/1540021645 shutdown_connections 2023-11-28T18:24:42.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.046+0000 7f376f1d1640 1 -- 172.21.15.105:0/1540021645 wait complete. 2023-11-28T18:24:42.069 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T18:24:42.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.206+0000 7fdeac8f3640 1 Processor -- start 2023-11-28T18:24:42.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.206+0000 7fdeac8f3640 1 -- start start 2023-11-28T18:24:42.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdeac8f3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdea80d1b00 0x7fdea8168090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:42.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdeac8f3640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea80cf440 con 0x7fdea815d1a0 2023-11-28T18:24:42.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdeac8f3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea80cffc0 con 0x7fdea80d0d40 2023-11-28T18:24:42.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdeac8f3640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea80ceb40 con 0x7fdea80d1b00 2023-11-28T18:24:42.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea77fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fdea815d1a0 0x7fdea80ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59826/0 (socket says 172.21.15.105:59826) 2023-11-28T18:24:42.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea77fe640 1 -- 172.21.15.105:0/745677464 learned_addr learned my addr 172.21.15.105:0/745677464 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:42.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea7fff640 1 --2- 172.21.15.105:0/745677464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdea80d1b00 0x7fdea8168090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:42.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea6ffd640 1 -- 172.21.15.105:0/745677464 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3004240926 0 0) 0x7fdea80cf440 con 0x7fdea815d1a0 2023-11-28T18:24:42.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea6ffd640 1 -- 172.21.15.105:0/745677464 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde94003530 con 0x7fdea815d1a0 2023-11-28T18:24:42.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea6ffd640 1 -- 172.21.15.105:0/745677464 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 296172971 0 0) 0x7fde94003530 con 0x7fdea815d1a0 2023-11-28T18:24:42.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea6ffd640 1 -- 172.21.15.105:0/745677464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdea80d1b00 msgr2=0x7fdea8168090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:42.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea6ffd640 1 --2- 172.21.15.105:0/745677464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdea80d1b00 0x7fdea8168090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea6ffd640 1 -- 172.21.15.105:0/745677464 >> v1:172.21.15.148:6789/0 conn(0x7fdea80d0d40 legacy=0x7fdea80d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea6ffd640 1 -- 172.21.15.105:0/745677464 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea80cf690 con 0x7fdea815d1a0 2023-11-28T18:24:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea6ffd640 1 -- 172.21.15.105:0/745677464 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fde98001680 con 0x7fdea815d1a0 2023-11-28T18:24:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea6ffd640 1 -- 172.21.15.105:0/745677464 >> v1:172.21.15.105:6789/0 conn(0x7fdea815d1a0 legacy=0x7fdea80ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea6ffd640 1 --2- 172.21.15.105:0/745677464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde940039b0 0x7fde94003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea6ffd640 1 --2- 172.21.15.105:0/745677464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdea80d1b00 0x7fde940047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea6ffd640 1 --2- 172.21.15.105:0/745677464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde94004d10 0x7fde94005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea6ffd640 1 -- 172.21.15.105:0/745677464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea80ceb40 con 0x7fde940039b0 2023-11-28T18:24:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea6ffd640 1 -- 172.21.15.105:0/745677464 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9400ba70 con 0x7fde94004d10 2023-11-28T18:24:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea6ffd640 1 -- 172.21.15.105:0/745677464 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9400bc30 con 0x7fdea80d1b00 2023-11-28T18:24:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea77fe640 1 --2- 172.21.15.105:0/745677464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde940039b0 0x7fde94003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea7fff640 1 --2- 172.21.15.105:0/745677464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde94004d10 0x7fde94005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fde9ffff640 1 --2- 172.21.15.105:0/745677464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdea80d1b00 0x7fde940047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea77fe640 1 -- 172.21.15.105:0/745677464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdea80d1b00 msgr2=0x7fde940047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:42.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea77fe640 1 --2- 172.21.15.105:0/745677464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdea80d1b00 0x7fde940047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea77fe640 1 -- 172.21.15.105:0/745677464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde94004d10 msgr2=0x7fde94005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:42.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea77fe640 1 --2- 172.21.15.105:0/745677464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde94004d10 0x7fde94005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fdea77fe640 1 -- 172.21.15.105:0/745677464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde94003530 con 0x7fde940039b0 2023-11-28T18:24:42.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.210+0000 7fde9ffff640 1 --2- 172.21.15.105:0/745677464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdea80d1b00 0x7fde940047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:42.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.214+0000 7fdea77fe640 1 --2- 172.21.15.105:0/745677464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde940039b0 0x7fde94003d90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fde9800c210 tx=0x7fde9801cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22673db6466c9f32 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:42.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.214+0000 7fdea6ffd640 1 -- 172.21.15.105:0/745677464 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde980a5b10 con 0x7fde940039b0 2023-11-28T18:24:42.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.214+0000 7fdea6ffd640 1 -- 172.21.15.105:0/745677464 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde980a5c90 con 0x7fde940039b0 2023-11-28T18:24:42.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.214+0000 7fdea6ffd640 1 -- 172.21.15.105:0/745677464 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde980058e0 con 0x7fde940039b0 2023-11-28T18:24:42.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.214+0000 7fdeac8f3640 1 -- 172.21.15.105:0/745677464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde940039b0 msgr2=0x7fde94003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:42.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.214+0000 7fdeac8f3640 1 --2- 172.21.15.105:0/745677464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde940039b0 0x7fde94003d90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fde9800c210 tx=0x7fde9801cc30 comp rx=0 tx=0).stop 2023-11-28T18:24:42.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.214+0000 7fdea7fff640 1 -- 172.21.15.105:0/745677464 reap_dead start 2023-11-28T18:24:42.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.214+0000 7fdeac8f3640 1 -- 172.21.15.105:0/745677464 shutdown_connections 2023-11-28T18:24:42.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.214+0000 7fdeac8f3640 1 -- 172.21.15.105:0/745677464 >> 172.21.15.105:0/745677464 conn(0x7fdea80d2e10 msgr2=0x7fdea80d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:42.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.214+0000 7fdeac8f3640 1 -- 172.21.15.105:0/745677464 shutdown_connections 2023-11-28T18:24:42.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.214+0000 7fdeac8f3640 1 -- 172.21.15.105:0/745677464 wait complete. 2023-11-28T18:24:42.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdeac8f3640 1 Processor -- start 2023-11-28T18:24:42.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdeac8f3640 1 -- start start 2023-11-28T18:24:42.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdeac8f3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde940039b0 0x7fdea815e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:42.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdeac8f3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdea8148050 0x7fdea815ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:42.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdeac8f3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdea815f3b0 0x7fdea81586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:42.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdeac8f3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea815df90 con 0x7fdea815f3b0 2023-11-28T18:24:42.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdeac8f3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea815d980 con 0x7fde940039b0 2023-11-28T18:24:42.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdeac8f3640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea815f8c0 con 0x7fdea8148050 2023-11-28T18:24:42.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdea7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdea815f3b0 0x7fdea81586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:42.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdea7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdea815f3b0 0x7fdea81586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58864/0 (socket says 172.21.15.105:58864) 2023-11-28T18:24:42.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdea7fff640 1 -- 172.21.15.105:0/1944897255 learned_addr learned my addr 172.21.15.105:0/1944897255 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:42.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdea7fff640 1 -- 172.21.15.105:0/1944897255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdea8148050 msgr2=0x7fdea815ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:42.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdea7fff640 1 --2- 172.21.15.105:0/1944897255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdea8148050 0x7fdea815ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdea7fff640 1 -- 172.21.15.105:0/1944897255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde940039b0 msgr2=0x7fdea815e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:42.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdea7fff640 1 --2- 172.21.15.105:0/1944897255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde940039b0 0x7fdea815e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdea7fff640 1 -- 172.21.15.105:0/1944897255 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea815dc20 con 0x7fdea815f3b0 2023-11-28T18:24:42.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdea7fff640 1 --2- 172.21.15.105:0/1944897255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdea815f3b0 0x7fdea81586d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdea8072b90 tx=0x7fdea000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:42.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdea4ff9640 1 -- 172.21.15.105:0/1944897255 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea000e040 con 0x7fdea815f3b0 2023-11-28T18:24:42.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdea4ff9640 1 -- 172.21.15.105:0/1944897255 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea000f040 con 0x7fdea815f3b0 2023-11-28T18:24:42.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdeac8f3640 1 -- 172.21.15.105:0/1944897255 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea815fba0 con 0x7fdea815f3b0 2023-11-28T18:24:42.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdea4ff9640 1 -- 172.21.15.105:0/1944897255 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea0015430 con 0x7fdea815f3b0 2023-11-28T18:24:42.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdeac8f3640 1 -- 172.21.15.105:0/1944897255 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea8156440 con 0x7fdea815f3b0 2023-11-28T18:24:42.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.218+0000 7fdeac8f3640 1 -- 172.21.15.105:0/1944897255 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea8074580 con 0x7fdea815f3b0 2023-11-28T18:24:42.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.222+0000 7fdea4ff9640 1 -- 172.21.15.105:0/1944897255 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fdea0015590 con 0x7fdea815f3b0 2023-11-28T18:24:42.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.222+0000 7fdea4ff9640 1 --2- 172.21.15.105:0/1944897255 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fde880668d0 0x7fde88068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:42.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.222+0000 7fdea4ff9640 1 -- 172.21.15.105:0/1944897255 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fdea0084f90 con 0x7fdea815f3b0 2023-11-28T18:24:42.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.222+0000 7fdea77fe640 1 --2- 172.21.15.105:0/1944897255 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fde880668d0 0x7fde88068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:42.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.222+0000 7fdea77fe640 1 --2- 172.21.15.105:0/1944897255 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fde880668d0 0x7fde88068d90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fde980ab000 tx=0x7fde98001a10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:42.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.226+0000 7fdea4ff9640 1 -- 172.21.15.105:0/1944897255 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdea0051d30 con 0x7fdea815f3b0 2023-11-28T18:24:42.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.378+0000 7fdeac8f3640 1 -- 172.21.15.105:0/1944897255 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdea815e140 con 0x7fdea815f3b0 2023-11-28T18:24:42.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.378+0000 7fdea4ff9640 1 -- 172.21.15.105:0/1944897255 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdea815e140 con 0x7fdea815f3b0 2023-11-28T18:24:42.383 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:24:42.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.378+0000 7fdeac8f3640 1 -- 172.21.15.105:0/1944897255 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fde880668d0 msgr2=0x7fde88068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:42.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.378+0000 7fdeac8f3640 1 --2- 172.21.15.105:0/1944897255 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fde880668d0 0x7fde88068d90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fde980ab000 tx=0x7fde98001a10 comp rx=0 tx=0).stop 2023-11-28T18:24:42.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.378+0000 7fdeac8f3640 1 -- 172.21.15.105:0/1944897255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdea815f3b0 msgr2=0x7fdea81586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:42.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.378+0000 7fdeac8f3640 1 --2- 172.21.15.105:0/1944897255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdea815f3b0 0x7fdea81586d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdea8072b90 tx=0x7fdea000f580 comp rx=0 tx=0).stop 2023-11-28T18:24:42.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.382+0000 7fdeac8f3640 1 -- 172.21.15.105:0/1944897255 shutdown_connections 2023-11-28T18:24:42.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.382+0000 7fdeac8f3640 1 --2- 172.21.15.105:0/1944897255 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fde880668d0 0x7fde88068d90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.382+0000 7fdeac8f3640 1 --2- 172.21.15.105:0/1944897255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdea815f3b0 0x7fdea81586d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.382+0000 7fdeac8f3640 1 --2- 172.21.15.105:0/1944897255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdea8148050 0x7fdea815ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.382+0000 7fdeac8f3640 1 --2- 172.21.15.105:0/1944897255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde940039b0 0x7fdea815e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.382+0000 7fdeac8f3640 1 -- 172.21.15.105:0/1944897255 >> 172.21.15.105:0/1944897255 conn(0x7fdea80d2e10 msgr2=0x7fdea80b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:42.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.382+0000 7fdeac8f3640 1 -- 172.21.15.105:0/1944897255 shutdown_connections 2023-11-28T18:24:42.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.382+0000 7fdeac8f3640 1 -- 172.21.15.105:0/1944897255 wait complete. 2023-11-28T18:24:42.405 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:24:42.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.538+0000 7fe51865a640 1 Processor -- start 2023-11-28T18:24:42.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.538+0000 7fe51865a640 1 -- start start 2023-11-28T18:24:42.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.542+0000 7fe51865a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe510159f40 0x7fe5100caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:42.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.542+0000 7fe51865a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe5100cc760 0x7fe5100ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:42.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.542+0000 7fe51865a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe51006bf50 con 0x7fe5100ad680 2023-11-28T18:24:42.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.542+0000 7fe51865a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5100ccbc0 con 0x7fe510159f40 2023-11-28T18:24:42.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.542+0000 7fe51865a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe51015a320 con 0x7fe5100cc760 2023-11-28T18:24:42.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.542+0000 7fe5163f6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe510159f40 0x7fe5100caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:42.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.542+0000 7fe5163f6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe510159f40 0x7fe5100caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58872/0 (socket says 172.21.15.105:58872) 2023-11-28T18:24:42.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.542+0000 7fe5163f6640 1 -- 172.21.15.105:0/876585565 learned_addr learned my addr 172.21.15.105:0/876585565 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:42.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.542+0000 7fe5163f6640 1 -- 172.21.15.105:0/876585565 >> v2:172.21.15.148:3300/0 conn(0x7fe5100cc760 msgr2=0x7fe5100ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:42.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.542+0000 7fe515bf5640 1 --2- 172.21.15.105:0/876585565 >> v2:172.21.15.148:3300/0 conn(0x7fe5100cc760 0x7fe5100ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:42.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.542+0000 7fe5163f6640 1 --2- 172.21.15.105:0/876585565 >> v2:172.21.15.148:3300/0 conn(0x7fe5100cc760 0x7fe5100ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.542+0000 7fe5163f6640 1 -- 172.21.15.105:0/876585565 >> v1:172.21.15.148:6789/0 conn(0x7fe5100ad680 legacy=0x7fe51015da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:42.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.542+0000 7fe5163f6640 1 -- 172.21.15.105:0/876585565 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe510071380 con 0x7fe510159f40 2023-11-28T18:24:42.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.542+0000 7fe515bf5640 1 --2- 172.21.15.105:0/876585565 >> v2:172.21.15.148:3300/0 conn(0x7fe5100cc760 0x7fe5100ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:42.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.546+0000 7fe5163f6640 1 --2- 172.21.15.105:0/876585565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe510159f40 0x7fe5100caf60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe51009d0d0 tx=0x7fe50c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faeaa397998de574 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:42.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.546+0000 7fe5153f4640 1 -- 172.21.15.105:0/876585565 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c0ab600 con 0x7fe510159f40 2023-11-28T18:24:42.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.546+0000 7fe5153f4640 1 -- 172.21.15.105:0/876585565 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50c0ab760 con 0x7fe510159f40 2023-11-28T18:24:42.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.546+0000 7fe5153f4640 1 -- 172.21.15.105:0/876585565 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c0abaf0 con 0x7fe510159f40 2023-11-28T18:24:42.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.546+0000 7fe51865a640 1 -- 172.21.15.105:0/876585565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe510159f40 msgr2=0x7fe5100caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:42.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.546+0000 7fe51865a640 1 --2- 172.21.15.105:0/876585565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe510159f40 0x7fe5100caf60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe51009d0d0 tx=0x7fe50c01c400 comp rx=0 tx=0).stop 2023-11-28T18:24:42.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.546+0000 7fe51865a640 1 -- 172.21.15.105:0/876585565 shutdown_connections 2023-11-28T18:24:42.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.546+0000 7fe51865a640 1 --2- 172.21.15.105:0/876585565 >> v2:172.21.15.148:3300/0 conn(0x7fe5100cc760 0x7fe5100ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.546+0000 7fe51865a640 1 --2- 172.21.15.105:0/876585565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe510159f40 0x7fe5100caf60 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.546+0000 7fe51865a640 1 -- 172.21.15.105:0/876585565 >> 172.21.15.105:0/876585565 conn(0x7fe5100ceb90 msgr2=0x7fe5100cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:42.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.546+0000 7fe51865a640 1 -- 172.21.15.105:0/876585565 shutdown_connections 2023-11-28T18:24:42.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.546+0000 7fe51865a640 1 -- 172.21.15.105:0/876585565 wait complete. 2023-11-28T18:24:42.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.546+0000 7fe51865a640 1 Processor -- start 2023-11-28T18:24:42.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.546+0000 7fe51865a640 1 -- start start 2023-11-28T18:24:42.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe51865a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5100ad680 0x7fe510150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:42.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe51865a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5100cc760 0x7fe510150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:42.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe51865a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe510159f40 0x7fe5100ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:42.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe51865a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe510099af0 con 0x7fe5100cc760 2023-11-28T18:24:42.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe51865a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe510150550 con 0x7fe5100ad680 2023-11-28T18:24:42.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe51865a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe51015d320 con 0x7fe510159f40 2023-11-28T18:24:42.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe515bf5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5100cc760 0x7fe510150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:42.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe515bf5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5100cc760 0x7fe510150010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58882/0 (socket says 172.21.15.105:58882) 2023-11-28T18:24:42.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe515bf5640 1 -- 172.21.15.105:0/3784144497 learned_addr learned my addr 172.21.15.105:0/3784144497 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:42.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe516bf7640 1 --2- 172.21.15.105:0/3784144497 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe510159f40 0x7fe5100ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:42.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe5163f6640 1 --2- 172.21.15.105:0/3784144497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5100ad680 0x7fe510150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:42.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe515bf5640 1 -- 172.21.15.105:0/3784144497 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe510159f40 msgr2=0x7fe5100ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:42.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe515bf5640 1 --2- 172.21.15.105:0/3784144497 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe510159f40 0x7fe5100ae700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe515bf5640 1 -- 172.21.15.105:0/3784144497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5100ad680 msgr2=0x7fe510150770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:42.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe515bf5640 1 --2- 172.21.15.105:0/3784144497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5100ad680 0x7fe510150770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe515bf5640 1 -- 172.21.15.105:0/3784144497 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50c0094d0 con 0x7fe5100cc760 2023-11-28T18:24:42.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe515bf5640 1 --2- 172.21.15.105:0/3784144497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5100cc760 0x7fe510150010 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe50000c7f0 tx=0x7fe50000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:42.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe506ffd640 1 -- 172.21.15.105:0/3784144497 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe500007820 con 0x7fe5100cc760 2023-11-28T18:24:42.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe506ffd640 1 -- 172.21.15.105:0/3784144497 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe500007980 con 0x7fe5100cc760 2023-11-28T18:24:42.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe51865a640 1 -- 172.21.15.105:0/3784144497 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51009a7f0 con 0x7fe5100cc760 2023-11-28T18:24:42.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe506ffd640 1 -- 172.21.15.105:0/3784144497 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe500007d30 con 0x7fe5100cc760 2023-11-28T18:24:42.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe51865a640 1 -- 172.21.15.105:0/3784144497 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe51015d040 con 0x7fe5100cc760 2023-11-28T18:24:42.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe51865a640 1 -- 172.21.15.105:0/3784144497 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe510067570 con 0x7fe5100cc760 2023-11-28T18:24:42.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.550+0000 7fe506ffd640 1 -- 172.21.15.105:0/3784144497 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fe5000113f0 con 0x7fe5100cc760 2023-11-28T18:24:42.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.554+0000 7fe506ffd640 1 --2- 172.21.15.105:0/3784144497 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe4f40668d0 0x7fe4f4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:42.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.554+0000 7fe506ffd640 1 -- 172.21.15.105:0/3784144497 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fe500084af0 con 0x7fe5100cc760 2023-11-28T18:24:42.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.554+0000 7fe5163f6640 1 --2- 172.21.15.105:0/3784144497 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe4f40668d0 0x7fe4f4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:42.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.554+0000 7fe5163f6640 1 --2- 172.21.15.105:0/3784144497 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe4f40668d0 0x7fe4f4068d90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0af000 tx=0x7fe50c0b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:42.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.558+0000 7fe506ffd640 1 -- 172.21.15.105:0/3784144497 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe5000519e0 con 0x7fe5100cc760 2023-11-28T18:24:42.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.734+0000 7fe51865a640 1 -- 172.21.15.105:0/3784144497 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe510155d10 con 0x7fe5100cc760 2023-11-28T18:24:42.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.738+0000 7fe506ffd640 1 -- 172.21.15.105:0/3784144497 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7fe50001f090 con 0x7fe5100cc760 2023-11-28T18:24:42.742 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:42.742 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/390529893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":390529893},{"type":"v1","addr":"172.21.15.148:6835","nonce":390529893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3924571887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3924571887},{"type":"v1","addr":"172.21.15.148:6833","nonce":3924571887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3271971772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3271971772},{"type":"v1","addr":"172.21.15.105:6835","nonce":3271971772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:23:33.288743+0000","modified":"2023-11-28T18:23:50.425976+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2504263238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2504263238},{"type":"v1","addr":"172.21.15.105:6837","nonce":2504263238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T18:24:42.742 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 9 2023-11-28T18:24:42.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.738+0000 7fe51865a640 1 -- 172.21.15.105:0/3784144497 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe4f40668d0 msgr2=0x7fe4f4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:42.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.738+0000 7fe51865a640 1 --2- 172.21.15.105:0/3784144497 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe4f40668d0 0x7fe4f4068d90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0af000 tx=0x7fe50c0b6040 comp rx=0 tx=0).stop 2023-11-28T18:24:42.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.738+0000 7fe51865a640 1 -- 172.21.15.105:0/3784144497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5100cc760 msgr2=0x7fe510150010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:42.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.738+0000 7fe51865a640 1 --2- 172.21.15.105:0/3784144497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5100cc760 0x7fe510150010 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe50000c7f0 tx=0x7fe50000ccb0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.738+0000 7fe51865a640 1 -- 172.21.15.105:0/3784144497 shutdown_connections 2023-11-28T18:24:42.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.738+0000 7fe51865a640 1 --2- 172.21.15.105:0/3784144497 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe510159f40 0x7fe5100ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.738+0000 7fe51865a640 1 --2- 172.21.15.105:0/3784144497 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe4f40668d0 0x7fe4f4068d90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.738+0000 7fe51865a640 1 --2- 172.21.15.105:0/3784144497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5100cc760 0x7fe510150010 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.738+0000 7fe51865a640 1 --2- 172.21.15.105:0/3784144497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5100ad680 0x7fe510150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.738+0000 7fe51865a640 1 -- 172.21.15.105:0/3784144497 >> 172.21.15.105:0/3784144497 conn(0x7fe5100ceb90 msgr2=0x7fe5100cdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:42.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.738+0000 7fe51865a640 1 -- 172.21.15.105:0/3784144497 shutdown_connections 2023-11-28T18:24:42.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.738+0000 7fe51865a640 1 -- 172.21.15.105:0/3784144497 wait complete. 2023-11-28T18:24:42.763 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:24:42.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.894+0000 7f7c3ec17640 1 Processor -- start 2023-11-28T18:24:42.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.894+0000 7f7c3ec17640 1 -- start start 2023-11-28T18:24:42.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.898+0000 7f7c3ec17640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c38159f40 0x7f7c380caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:42.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.898+0000 7f7c3ec17640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7c380cc760 0x7f7c380ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:42.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.898+0000 7f7c3ec17640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7c380ad680 0x7f7c3815da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:42.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.898+0000 7f7c3ec17640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c3806bf50 con 0x7f7c380cc760 2023-11-28T18:24:42.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.898+0000 7f7c3ec17640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c380ccbc0 con 0x7f7c380ad680 2023-11-28T18:24:42.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.898+0000 7f7c3ec17640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3815a320 con 0x7f7c38159f40 2023-11-28T18:24:42.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.898+0000 7f7c2ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7c380cc760 0x7f7c380ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:42.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.898+0000 7f7c2ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7c380cc760 0x7f7c380ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58898/0 (socket says 172.21.15.105:58898) 2023-11-28T18:24:42.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.898+0000 7f7c2ffff640 1 -- 172.21.15.105:0/961060223 learned_addr learned my addr 172.21.15.105:0/961060223 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:42.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.898+0000 7f7c3c9b3640 1 --2- 172.21.15.105:0/961060223 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c38159f40 0x7f7c380caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:42.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.898+0000 7f7c2ffff640 1 -- 172.21.15.105:0/961060223 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c38159f40 msgr2=0x7f7c380caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:42.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.898+0000 7f7c3d1b4640 1 --2- 172.21.15.105:0/961060223 >> v2:172.21.15.148:3300/0 conn(0x7f7c380ad680 0x7f7c3815da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:42.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.898+0000 7f7c2ffff640 1 --2- 172.21.15.105:0/961060223 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c38159f40 0x7f7c380caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.898+0000 7f7c2ffff640 1 -- 172.21.15.105:0/961060223 >> v2:172.21.15.148:3300/0 conn(0x7f7c380ad680 msgr2=0x7f7c3815da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:42.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.898+0000 7f7c2ffff640 1 --2- 172.21.15.105:0/961060223 >> v2:172.21.15.148:3300/0 conn(0x7f7c380ad680 0x7f7c3815da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.898+0000 7f7c2ffff640 1 -- 172.21.15.105:0/961060223 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c38071380 con 0x7f7c380cc760 2023-11-28T18:24:42.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.898+0000 7f7c3c9b3640 1 --2- 172.21.15.105:0/961060223 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c38159f40 0x7f7c380caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:42.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.898+0000 7f7c3d1b4640 1 --2- 172.21.15.105:0/961060223 >> v2:172.21.15.148:3300/0 conn(0x7f7c380ad680 0x7f7c3815da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:42.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.898+0000 7f7c2ffff640 1 --2- 172.21.15.105:0/961060223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c380cc760 0x7f7c380ad140 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7c380724d0 tx=0x7f7c2801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f29535ba7c8b6f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:42.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.902+0000 7f7c2f7fe640 1 -- 172.21.15.105:0/961060223 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c280ab600 con 0x7f7c380cc760 2023-11-28T18:24:42.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.902+0000 7f7c2f7fe640 1 -- 172.21.15.105:0/961060223 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c280ab760 con 0x7f7c380cc760 2023-11-28T18:24:42.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.902+0000 7f7c2f7fe640 1 -- 172.21.15.105:0/961060223 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c280abaf0 con 0x7f7c380cc760 2023-11-28T18:24:42.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.902+0000 7f7c3ec17640 1 -- 172.21.15.105:0/961060223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c380cc760 msgr2=0x7f7c380ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:42.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.902+0000 7f7c3ec17640 1 --2- 172.21.15.105:0/961060223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c380cc760 0x7f7c380ad140 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7c380724d0 tx=0x7f7c2801c440 comp rx=0 tx=0).stop 2023-11-28T18:24:42.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.902+0000 7f7c3ec17640 1 -- 172.21.15.105:0/961060223 shutdown_connections 2023-11-28T18:24:42.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.902+0000 7f7c3ec17640 1 --2- 172.21.15.105:0/961060223 >> v2:172.21.15.148:3300/0 conn(0x7f7c380ad680 0x7f7c3815da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.902+0000 7f7c3ec17640 1 --2- 172.21.15.105:0/961060223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c380cc760 0x7f7c380ad140 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.902+0000 7f7c3ec17640 1 --2- 172.21.15.105:0/961060223 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c38159f40 0x7f7c380caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.902+0000 7f7c3ec17640 1 -- 172.21.15.105:0/961060223 >> 172.21.15.105:0/961060223 conn(0x7f7c380ceb90 msgr2=0x7f7c380cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:42.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.902+0000 7f7c3ec17640 1 -- 172.21.15.105:0/961060223 shutdown_connections 2023-11-28T18:24:42.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.902+0000 7f7c3ec17640 1 -- 172.21.15.105:0/961060223 wait complete. 2023-11-28T18:24:42.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.902+0000 7f7c3ec17640 1 Processor -- start 2023-11-28T18:24:42.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.902+0000 7f7c3ec17640 1 -- start start 2023-11-28T18:24:42.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c3ec17640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c380ad680 0x7f7c3814fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:42.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c3ec17640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c380cc760 0x7f7c38152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:42.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c3ec17640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c38159f40 0x7f7c3814f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:42.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c3ec17640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c38154300 con 0x7f7c380ad680 2023-11-28T18:24:42.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c3ec17640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c381530e0 con 0x7f7c38159f40 2023-11-28T18:24:42.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c3ec17640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c38153380 con 0x7f7c380cc760 2023-11-28T18:24:42.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c3c9b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c380ad680 0x7f7c3814fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:42.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c3c9b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c380ad680 0x7f7c3814fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58902/0 (socket says 172.21.15.105:58902) 2023-11-28T18:24:42.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c3c9b3640 1 -- 172.21.15.105:0/4112675038 learned_addr learned my addr 172.21.15.105:0/4112675038 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:42.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c2ffff640 1 --2- 172.21.15.105:0/4112675038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c380cc760 0x7f7c38152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:42.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c3d1b4640 1 --2- 172.21.15.105:0/4112675038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c38159f40 0x7f7c3814f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:42.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c3c9b3640 1 -- 172.21.15.105:0/4112675038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c380cc760 msgr2=0x7f7c38152ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:42.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c3c9b3640 1 --2- 172.21.15.105:0/4112675038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c380cc760 0x7f7c38152ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c3c9b3640 1 -- 172.21.15.105:0/4112675038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c38159f40 msgr2=0x7f7c3814f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:42.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c3c9b3640 1 --2- 172.21.15.105:0/4112675038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c38159f40 0x7f7c3814f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:42.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c3c9b3640 1 -- 172.21.15.105:0/4112675038 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c280094d0 con 0x7f7c380ad680 2023-11-28T18:24:42.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c3c9b3640 1 --2- 172.21.15.105:0/4112675038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c380ad680 0x7f7c3814fdf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400c7f0 tx=0x7f7c3400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:42.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c2d7fa640 1 -- 172.21.15.105:0/4112675038 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c34007820 con 0x7f7c380ad680 2023-11-28T18:24:42.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c2d7fa640 1 -- 172.21.15.105:0/4112675038 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c34007980 con 0x7f7c380ad680 2023-11-28T18:24:42.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c3ec17640 1 -- 172.21.15.105:0/4112675038 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c38153660 con 0x7f7c380ad680 2023-11-28T18:24:42.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c2d7fa640 1 -- 172.21.15.105:0/4112675038 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c34007d30 con 0x7f7c380ad680 2023-11-28T18:24:42.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c3ec17640 1 -- 172.21.15.105:0/4112675038 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c38154510 con 0x7f7c380ad680 2023-11-28T18:24:42.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c3ec17640 1 -- 172.21.15.105:0/4112675038 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c38154700 con 0x7f7c380ad680 2023-11-28T18:24:42.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.906+0000 7f7c2d7fa640 1 -- 172.21.15.105:0/4112675038 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f7c340113f0 con 0x7f7c380ad680 2023-11-28T18:24:42.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.910+0000 7f7c2d7fa640 1 --2- 172.21.15.105:0/4112675038 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7c240668d0 0x7f7c24068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:42.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.910+0000 7f7c2d7fa640 1 -- 172.21.15.105:0/4112675038 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f7c34084a20 con 0x7f7c380ad680 2023-11-28T18:24:42.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.910+0000 7f7c2ffff640 1 --2- 172.21.15.105:0/4112675038 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7c240668d0 0x7f7c24068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:42.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.910+0000 7f7c2ffff640 1 --2- 172.21.15.105:0/4112675038 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7c240668d0 0x7f7c24068d90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7c280af000 tx=0x7f7c280b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:42.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:42.914+0000 7f7c2d7fa640 1 -- 172.21.15.105:0/4112675038 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7c340519a0 con 0x7f7c380ad680 2023-11-28T18:24:43.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.062+0000 7f7c3ec17640 1 -- 172.21.15.105:0/4112675038 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c38067570 con 0x7f7c380ad680 2023-11-28T18:24:43.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.066+0000 7f7c2d7fa640 1 -- 172.21.15.105:0/4112675038 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18887 (secure 0 0 0) 0x7f7c34054fc0 con 0x7f7c380ad680 2023-11-28T18:24:43.071 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:43.071 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:24:15.048168+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:24:43.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.066+0000 7f7c3ec17640 1 -- 172.21.15.105:0/4112675038 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7c240668d0 msgr2=0x7f7c24068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:43.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.066+0000 7f7c3ec17640 1 --2- 172.21.15.105:0/4112675038 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7c240668d0 0x7f7c24068d90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7c280af000 tx=0x7f7c280b6040 comp rx=0 tx=0).stop 2023-11-28T18:24:43.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.066+0000 7f7c3ec17640 1 -- 172.21.15.105:0/4112675038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c380ad680 msgr2=0x7f7c3814fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:43.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.066+0000 7f7c3ec17640 1 --2- 172.21.15.105:0/4112675038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c380ad680 0x7f7c3814fdf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400c7f0 tx=0x7f7c3400ccb0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.066+0000 7f7c3ec17640 1 -- 172.21.15.105:0/4112675038 shutdown_connections 2023-11-28T18:24:43.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.066+0000 7f7c3ec17640 1 --2- 172.21.15.105:0/4112675038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c38159f40 0x7f7c3814f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.066+0000 7f7c3ec17640 1 --2- 172.21.15.105:0/4112675038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c380cc760 0x7f7c38152ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.066+0000 7f7c3ec17640 1 --2- 172.21.15.105:0/4112675038 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7c240668d0 0x7f7c24068d90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.066+0000 7f7c3ec17640 1 --2- 172.21.15.105:0/4112675038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c380ad680 0x7f7c3814fdf0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.066+0000 7f7c3ec17640 1 -- 172.21.15.105:0/4112675038 >> 172.21.15.105:0/4112675038 conn(0x7f7c380ceb90 msgr2=0x7f7c38071aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:43.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.066+0000 7f7c3ec17640 1 -- 172.21.15.105:0/4112675038 shutdown_connections 2023-11-28T18:24:43.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.066+0000 7f7c3ec17640 1 -- 172.21.15.105:0/4112675038 wait complete. 2023-11-28T18:24:43.092 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T18:24:43.092 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:24:43.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bf59e640 1 Processor -- start 2023-11-28T18:24:43.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bf59e640 1 -- start start 2023-11-28T18:24:43.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bf59e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52c00cfb30 0x7f52c0167fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:43.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bf59e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c00ce840 con 0x7f52c015cef0 2023-11-28T18:24:43.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bf59e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c00cd990 con 0x7f52c015d2d0 2023-11-28T18:24:43.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bf59e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c00b4780 con 0x7f52c00cfb30 2023-11-28T18:24:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52be59c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f52c015cef0 0x7f52c00d1600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59828/0 (socket says 172.21.15.105:59828) 2023-11-28T18:24:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52be59c640 1 -- 172.21.15.105:0/1797331598 learned_addr learned my addr 172.21.15.105:0/1797331598 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bd59a640 1 -- 172.21.15.105:0/1797331598 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528869452 0 0) 0x7f52c00ce840 con 0x7f52c015cef0 2023-11-28T18:24:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bed9d640 1 --2- 172.21.15.105:0/1797331598 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52c00cfb30 0x7f52c0167fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bd59a640 1 -- 172.21.15.105:0/1797331598 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52a8003530 con 0x7f52c015cef0 2023-11-28T18:24:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bd59a640 1 -- 172.21.15.105:0/1797331598 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 741825268 0 0) 0x7f52a8003530 con 0x7f52c015cef0 2023-11-28T18:24:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bd59a640 1 -- 172.21.15.105:0/1797331598 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52c00cfb30 msgr2=0x7f52c0167fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bd59a640 1 --2- 172.21.15.105:0/1797331598 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52c00cfb30 0x7f52c0167fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bd59a640 1 -- 172.21.15.105:0/1797331598 >> v1:172.21.15.148:6789/0 conn(0x7f52c015d2d0 legacy=0x7f52c00cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bd59a640 1 -- 172.21.15.105:0/1797331598 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c00cecd0 con 0x7f52c015cef0 2023-11-28T18:24:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bd59a640 1 -- 172.21.15.105:0/1797331598 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f52b0001680 con 0x7f52c015cef0 2023-11-28T18:24:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bd59a640 1 -- 172.21.15.105:0/1797331598 >> v1:172.21.15.105:6789/0 conn(0x7f52c015cef0 legacy=0x7f52c00d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bd59a640 1 --2- 172.21.15.105:0/1797331598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52a80039b0 0x7f52a8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bd59a640 1 --2- 172.21.15.105:0/1797331598 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52c00cfb30 0x7f52a80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bd59a640 1 --2- 172.21.15.105:0/1797331598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52a8004d10 0x7f52a8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bd59a640 1 -- 172.21.15.105:0/1797331598 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c00b4780 con 0x7f52a8004d10 2023-11-28T18:24:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bd59a640 1 -- 172.21.15.105:0/1797331598 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a800ba70 con 0x7f52a80039b0 2023-11-28T18:24:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bd59a640 1 -- 172.21.15.105:0/1797331598 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a800bc30 con 0x7f52c00cfb30 2023-11-28T18:24:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bed9d640 1 --2- 172.21.15.105:0/1797331598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52a8004d10 0x7f52a8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52be59c640 1 --2- 172.21.15.105:0/1797331598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52a80039b0 0x7f52a8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bdd9b640 1 --2- 172.21.15.105:0/1797331598 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52c00cfb30 0x7f52a80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:43.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bed9d640 1 -- 172.21.15.105:0/1797331598 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52c00cfb30 msgr2=0x7f52a80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:43.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bed9d640 1 --2- 172.21.15.105:0/1797331598 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52c00cfb30 0x7f52a80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bed9d640 1 -- 172.21.15.105:0/1797331598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52a80039b0 msgr2=0x7f52a8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:43.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bed9d640 1 --2- 172.21.15.105:0/1797331598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52a80039b0 0x7f52a8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.226+0000 7f52bed9d640 1 -- 172.21.15.105:0/1797331598 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a8003530 con 0x7f52a8004d10 2023-11-28T18:24:43.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.230+0000 7f52bed9d640 1 --2- 172.21.15.105:0/1797331598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52a8004d10 0x7f52a8005190 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f52c008d0c0 tx=0x7f52b801cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d08f6573aa6bdb3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:43.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.230+0000 7f52bd59a640 1 -- 172.21.15.105:0/1797331598 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b80a5cb0 con 0x7f52a8004d10 2023-11-28T18:24:43.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.230+0000 7f52bd59a640 1 -- 172.21.15.105:0/1797331598 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b800f040 con 0x7f52a8004d10 2023-11-28T18:24:43.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.230+0000 7f52bd59a640 1 -- 172.21.15.105:0/1797331598 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b80ad660 con 0x7f52a8004d10 2023-11-28T18:24:43.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.230+0000 7f52bf59e640 1 -- 172.21.15.105:0/1797331598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52a8004d10 msgr2=0x7f52a8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:43.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.230+0000 7f52bf59e640 1 --2- 172.21.15.105:0/1797331598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52a8004d10 0x7f52a8005190 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f52c008d0c0 tx=0x7f52b801cbe0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.230+0000 7f52bed9d640 1 -- 172.21.15.105:0/1797331598 reap_dead start 2023-11-28T18:24:43.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.230+0000 7f52bf59e640 1 -- 172.21.15.105:0/1797331598 shutdown_connections 2023-11-28T18:24:43.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.230+0000 7f52bf59e640 1 -- 172.21.15.105:0/1797331598 >> 172.21.15.105:0/1797331598 conn(0x7f52c00d2e10 msgr2=0x7f52c00d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:43.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.230+0000 7f52bf59e640 1 -- 172.21.15.105:0/1797331598 shutdown_connections 2023-11-28T18:24:43.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.230+0000 7f52bf59e640 1 -- 172.21.15.105:0/1797331598 wait complete. 2023-11-28T18:24:43.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.234+0000 7f52bf59e640 1 Processor -- start 2023-11-28T18:24:43.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.234+0000 7f52bf59e640 1 -- start start 2023-11-28T18:24:43.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.234+0000 7f52bf59e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52a8004d10 0x7f52c00b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:43.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.234+0000 7f52bf59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c00b8380 0x7f52c01f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:43.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.234+0000 7f52bf59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52c01f33b0 0x7f52c0150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:43.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.234+0000 7f52bf59e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c015ec20 con 0x7f52c00b8380 2023-11-28T18:24:43.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.234+0000 7f52bf59e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c015ef00 con 0x7f52c01f33b0 2023-11-28T18:24:43.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.234+0000 7f52bf59e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c015f1e0 con 0x7f52a8004d10 2023-11-28T18:24:43.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.234+0000 7f52bdd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c00b8380 0x7f52c01f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:43.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.234+0000 7f52bdd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c00b8380 0x7f52c01f2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58918/0 (socket says 172.21.15.105:58918) 2023-11-28T18:24:43.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.234+0000 7f52bdd9b640 1 -- 172.21.15.105:0/1855477642 learned_addr learned my addr 172.21.15.105:0/1855477642 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:43.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.234+0000 7f52be59c640 1 --2- 172.21.15.105:0/1855477642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52a8004d10 0x7f52c00b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:43.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.234+0000 7f52bed9d640 1 --2- 172.21.15.105:0/1855477642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52c01f33b0 0x7f52c0150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:43.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.234+0000 7f52bdd9b640 1 -- 172.21.15.105:0/1855477642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52a8004d10 msgr2=0x7f52c00b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:43.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.234+0000 7f52bdd9b640 1 --2- 172.21.15.105:0/1855477642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52a8004d10 0x7f52c00b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.234+0000 7f52bdd9b640 1 -- 172.21.15.105:0/1855477642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52c01f33b0 msgr2=0x7f52c0150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:43.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.234+0000 7f52bdd9b640 1 --2- 172.21.15.105:0/1855477642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52c01f33b0 0x7f52c0150cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.234+0000 7f52bdd9b640 1 -- 172.21.15.105:0/1855477642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c015f4c0 con 0x7f52c00b8380 2023-11-28T18:24:43.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.234+0000 7f52bed9d640 1 --2- 172.21.15.105:0/1855477642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52c01f33b0 0x7f52c0150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:43.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.234+0000 7f52bdd9b640 1 --2- 172.21.15.105:0/1855477642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c00b8380 0x7f52c01f2e70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f52b4011bb0 tx=0x7f52b400f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:43.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.238+0000 7f52aeffd640 1 -- 172.21.15.105:0/1855477642 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4011ca0 con 0x7f52c00b8380 2023-11-28T18:24:43.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.238+0000 7f52aeffd640 1 -- 172.21.15.105:0/1855477642 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b4011e00 con 0x7f52c00b8380 2023-11-28T18:24:43.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.238+0000 7f52aeffd640 1 -- 172.21.15.105:0/1855477642 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4005310 con 0x7f52c00b8380 2023-11-28T18:24:43.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.238+0000 7f52bf59e640 1 -- 172.21.15.105:0/1855477642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c015f7a0 con 0x7f52c00b8380 2023-11-28T18:24:43.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.238+0000 7f52bf59e640 1 -- 172.21.15.105:0/1855477642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c00b38e0 con 0x7f52c00b8380 2023-11-28T18:24:43.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.238+0000 7f52bf59e640 1 -- 172.21.15.105:0/1855477642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c00d03c0 con 0x7f52c00b8380 2023-11-28T18:24:43.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.238+0000 7f52aeffd640 1 -- 172.21.15.105:0/1855477642 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f52b4003600 con 0x7f52c00b8380 2023-11-28T18:24:43.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.238+0000 7f52aeffd640 1 --2- 172.21.15.105:0/1855477642 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52a0066900 0x7f52a0068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:43.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.238+0000 7f52aeffd640 1 -- 172.21.15.105:0/1855477642 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f52b40880d0 con 0x7f52c00b8380 2023-11-28T18:24:43.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.238+0000 7f52be59c640 1 --2- 172.21.15.105:0/1855477642 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52a0066900 0x7f52a0068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:43.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.238+0000 7f52be59c640 1 --2- 172.21.15.105:0/1855477642 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52a0066900 0x7f52a0068dc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f52b00010f0 tx=0x7f52b0001c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:43.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.242+0000 7f52aeffd640 1 -- 172.21.15.105:0/1855477642 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f52b4054f00 con 0x7f52c00b8380 2023-11-28T18:24:43.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.426+0000 7f52bf59e640 1 -- 172.21.15.105:0/1855477642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52c00b2a50 con 0x7f52c00b8380 2023-11-28T18:24:43.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.426+0000 7f52aeffd640 1 -- 172.21.15.105:0/1855477642 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7f52b4019090 con 0x7f52c00b8380 2023-11-28T18:24:43.432 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:43.432 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/390529893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":390529893},{"type":"v1","addr":"172.21.15.148:6835","nonce":390529893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3924571887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3924571887},{"type":"v1","addr":"172.21.15.148:6833","nonce":3924571887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3271971772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3271971772},{"type":"v1","addr":"172.21.15.105:6835","nonce":3271971772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:23:33.288743+0000","modified":"2023-11-28T18:23:50.425976+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2504263238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2504263238},{"type":"v1","addr":"172.21.15.105:6837","nonce":2504263238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T18:24:43.432 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 9 2023-11-28T18:24:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.430+0000 7f52bf59e640 1 -- 172.21.15.105:0/1855477642 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52a0066900 msgr2=0x7f52a0068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.430+0000 7f52bf59e640 1 --2- 172.21.15.105:0/1855477642 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52a0066900 0x7f52a0068dc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f52b00010f0 tx=0x7f52b0001c00 comp rx=0 tx=0).stop 2023-11-28T18:24:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.430+0000 7f52bf59e640 1 -- 172.21.15.105:0/1855477642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c00b8380 msgr2=0x7f52c01f2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.430+0000 7f52bf59e640 1 --2- 172.21.15.105:0/1855477642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c00b8380 0x7f52c01f2e70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f52b4011bb0 tx=0x7f52b400f5a0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.430+0000 7f52bf59e640 1 -- 172.21.15.105:0/1855477642 shutdown_connections 2023-11-28T18:24:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.430+0000 7f52bf59e640 1 --2- 172.21.15.105:0/1855477642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52c01f33b0 0x7f52c0150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.430+0000 7f52bf59e640 1 --2- 172.21.15.105:0/1855477642 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52a0066900 0x7f52a0068dc0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.430+0000 7f52bf59e640 1 --2- 172.21.15.105:0/1855477642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c00b8380 0x7f52c01f2e70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.430+0000 7f52bf59e640 1 --2- 172.21.15.105:0/1855477642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52a8004d10 0x7f52c00b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.430+0000 7f52bf59e640 1 -- 172.21.15.105:0/1855477642 >> 172.21.15.105:0/1855477642 conn(0x7f52c00d2e10 msgr2=0x7f52c00b1840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.430+0000 7f52bf59e640 1 -- 172.21.15.105:0/1855477642 shutdown_connections 2023-11-28T18:24:43.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.430+0000 7f52bf59e640 1 -- 172.21.15.105:0/1855477642 wait complete. 2023-11-28T18:24:43.453 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:24:43.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.586+0000 7f1109580640 1 Processor -- start 2023-11-28T18:24:43.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.586+0000 7f1109580640 1 -- start start 2023-11-28T18:24:43.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.586+0000 7f1109580640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f11040b4290 0x7f11040b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:43.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.586+0000 7f1109580640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f110415b030 0x7f11040709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:43.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.586+0000 7f1109580640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11040b4bb0 con 0x7f1104071960 2023-11-28T18:24:43.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.586+0000 7f1109580640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11040701d0 con 0x7f11040b4290 2023-11-28T18:24:43.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.586+0000 7f1109580640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110406df80 con 0x7f110415b030 2023-11-28T18:24:43.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.586+0000 7f1102ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f11040b4290 0x7f11040b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:43.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.586+0000 7f1102ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f11040b4290 0x7f11040b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58930/0 (socket says 172.21.15.105:58930) 2023-11-28T18:24:43.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.586+0000 7f1102ffd640 1 -- 172.21.15.105:0/2333071916 learned_addr learned my addr 172.21.15.105:0/2333071916 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:43.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.586+0000 7f11037fe640 1 --2- 172.21.15.105:0/2333071916 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f110415b030 0x7f11040709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:43.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.586+0000 7f1102ffd640 1 -- 172.21.15.105:0/2333071916 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f110415b030 msgr2=0x7f11040709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:43.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.586+0000 7f1102ffd640 1 --2- 172.21.15.105:0/2333071916 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f110415b030 0x7f11040709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.586+0000 7f1102ffd640 1 -- 172.21.15.105:0/2333071916 >> v1:172.21.15.148:6789/0 conn(0x7f1104071960 legacy=0x7f110415f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:43.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.586+0000 7f11027fc640 1 -- 172.21.15.105:0/2333071916 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3421051087 0 0) 0x7f11040b4bb0 con 0x7f1104071960 2023-11-28T18:24:43.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.586+0000 7f1102ffd640 1 -- 172.21.15.105:0/2333071916 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1104071df0 con 0x7f11040b4290 2023-11-28T18:24:43.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.586+0000 7f11037fe640 1 --2- 172.21.15.105:0/2333071916 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f110415b030 0x7f11040709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:24:43.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.590+0000 7f1102ffd640 1 --2- 172.21.15.105:0/2333071916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11040b4290 0x7f11040b4670 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009870 tx=0x7f10f401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58bda6b0b352eb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:43.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.590+0000 7f11027fc640 1 -- 172.21.15.105:0/2333071916 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f40ab620 con 0x7f11040b4290 2023-11-28T18:24:43.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.590+0000 7f11027fc640 1 -- 172.21.15.105:0/2333071916 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f40ab780 con 0x7f11040b4290 2023-11-28T18:24:43.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.590+0000 7f11027fc640 1 -- 172.21.15.105:0/2333071916 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f40abb10 con 0x7f11040b4290 2023-11-28T18:24:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.590+0000 7f1109580640 1 -- 172.21.15.105:0/2333071916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11040b4290 msgr2=0x7f11040b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.590+0000 7f1109580640 1 --2- 172.21.15.105:0/2333071916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11040b4290 0x7f11040b4670 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009870 tx=0x7f10f401c510 comp rx=0 tx=0).stop 2023-11-28T18:24:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.590+0000 7f1109580640 1 -- 172.21.15.105:0/2333071916 shutdown_connections 2023-11-28T18:24:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.590+0000 7f1109580640 1 --2- 172.21.15.105:0/2333071916 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f110415b030 0x7f11040709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.590+0000 7f1109580640 1 --2- 172.21.15.105:0/2333071916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11040b4290 0x7f11040b4670 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.590+0000 7f1109580640 1 -- 172.21.15.105:0/2333071916 >> 172.21.15.105:0/2333071916 conn(0x7f11040ceb90 msgr2=0x7f11040cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.590+0000 7f1109580640 1 -- 172.21.15.105:0/2333071916 shutdown_connections 2023-11-28T18:24:43.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.590+0000 7f1109580640 1 -- 172.21.15.105:0/2333071916 wait complete. 2023-11-28T18:24:43.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f1109580640 1 Processor -- start 2023-11-28T18:24:43.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f1109580640 1 -- start start 2023-11-28T18:24:43.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f1109580640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11040b4290 0x7f11040cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:43.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f1109580640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f110415b030 0x7f110414d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:43.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f1109580640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1104151090 0x7f1104151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:43.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f1109580640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11040ce230 con 0x7f1104151090 2023-11-28T18:24:43.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f1109580640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1104150950 con 0x7f110415b030 2023-11-28T18:24:43.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f1109580640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110409b9e0 con 0x7f11040b4290 2023-11-28T18:24:43.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f11037fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1104151090 0x7f1104151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:43.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f11037fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1104151090 0x7f1104151540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58944/0 (socket says 172.21.15.105:58944) 2023-11-28T18:24:43.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f11037fe640 1 -- 172.21.15.105:0/557769291 learned_addr learned my addr 172.21.15.105:0/557769291 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:43.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f10fa7fc640 1 --2- 172.21.15.105:0/557769291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f110415b030 0x7f110414d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:43.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f1102ffd640 1 --2- 172.21.15.105:0/557769291 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11040b4290 0x7f11040cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:43.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f11037fe640 1 -- 172.21.15.105:0/557769291 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11040b4290 msgr2=0x7f11040cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:43.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f11037fe640 1 --2- 172.21.15.105:0/557769291 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11040b4290 0x7f11040cdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f11037fe640 1 -- 172.21.15.105:0/557769291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f110415b030 msgr2=0x7f110414d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:43.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f11037fe640 1 --2- 172.21.15.105:0/557769291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f110415b030 0x7f110414d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f11037fe640 1 -- 172.21.15.105:0/557769291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10fc009a70 con 0x7f1104151090 2023-11-28T18:24:43.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f11037fe640 1 --2- 172.21.15.105:0/557769291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1104151090 0x7f1104151540 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00efb0 tx=0x7f10fc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:43.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f10fbfff640 1 -- 172.21.15.105:0/557769291 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc014480 con 0x7f1104151090 2023-11-28T18:24:43.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f10fbfff640 1 -- 172.21.15.105:0/557769291 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10fc00f040 con 0x7f1104151090 2023-11-28T18:24:43.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f1109580640 1 -- 172.21.15.105:0/557769291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f40094d0 con 0x7f1104151090 2023-11-28T18:24:43.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f10fbfff640 1 -- 172.21.15.105:0/557769291 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc014800 con 0x7f1104151090 2023-11-28T18:24:43.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.594+0000 7f1109580640 1 -- 172.21.15.105:0/557769291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11040cd4f0 con 0x7f1104151090 2023-11-28T18:24:43.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.598+0000 7f1109580640 1 -- 172.21.15.105:0/557769291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1104067570 con 0x7f1104151090 2023-11-28T18:24:43.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.598+0000 7f10fbfff640 1 -- 172.21.15.105:0/557769291 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f10fc014960 con 0x7f1104151090 2023-11-28T18:24:43.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.598+0000 7f10fbfff640 1 --2- 172.21.15.105:0/557769291 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10e40668d0 0x7f10e4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:43.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.598+0000 7f10fbfff640 1 -- 172.21.15.105:0/557769291 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f10fc085870 con 0x7f1104151090 2023-11-28T18:24:43.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.598+0000 7f1102ffd640 1 --2- 172.21.15.105:0/557769291 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10e40668d0 0x7f10e4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:43.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.598+0000 7f1102ffd640 1 --2- 172.21.15.105:0/557769291 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10e40668d0 0x7f10e4068d90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f10f40af000 tx=0x7f10f40a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:43.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.602+0000 7f10fbfff640 1 -- 172.21.15.105:0/557769291 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f10fc0526b0 con 0x7f1104151090 2023-11-28T18:24:43.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.782+0000 7f1109580640 1 -- 172.21.15.105:0/557769291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11040cf7f0 con 0x7f1104151090 2023-11-28T18:24:43.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.782+0000 7f10fbfff640 1 -- 172.21.15.105:0/557769291 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7f10fc018090 con 0x7f1104151090 2023-11-28T18:24:43.787 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:43.788 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/390529893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":390529893},{"type":"v1","addr":"172.21.15.148:6835","nonce":390529893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3924571887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3924571887},{"type":"v1","addr":"172.21.15.148:6833","nonce":3924571887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3271971772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3271971772},{"type":"v1","addr":"172.21.15.105:6835","nonce":3271971772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:23:33.288743+0000","modified":"2023-11-28T18:23:50.425976+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2504263238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2504263238},{"type":"v1","addr":"172.21.15.105:6837","nonce":2504263238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T18:24:43.788 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 9 2023-11-28T18:24:43.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.782+0000 7f1109580640 1 -- 172.21.15.105:0/557769291 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10e40668d0 msgr2=0x7f10e4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:43.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.782+0000 7f1109580640 1 --2- 172.21.15.105:0/557769291 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10e40668d0 0x7f10e4068d90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f10f40af000 tx=0x7f10f40a7000 comp rx=0 tx=0).stop 2023-11-28T18:24:43.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.782+0000 7f1109580640 1 -- 172.21.15.105:0/557769291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1104151090 msgr2=0x7f1104151540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:43.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.782+0000 7f1109580640 1 --2- 172.21.15.105:0/557769291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1104151090 0x7f1104151540 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00efb0 tx=0x7f10fc00c490 comp rx=0 tx=0).stop 2023-11-28T18:24:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.782+0000 7f1109580640 1 -- 172.21.15.105:0/557769291 shutdown_connections 2023-11-28T18:24:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.782+0000 7f1109580640 1 --2- 172.21.15.105:0/557769291 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10e40668d0 0x7f10e4068d90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.782+0000 7f1109580640 1 --2- 172.21.15.105:0/557769291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1104151090 0x7f1104151540 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.782+0000 7f1109580640 1 --2- 172.21.15.105:0/557769291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f110415b030 0x7f110414d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.782+0000 7f1109580640 1 --2- 172.21.15.105:0/557769291 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11040b4290 0x7f11040cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.782+0000 7f1109580640 1 -- 172.21.15.105:0/557769291 >> 172.21.15.105:0/557769291 conn(0x7f11040ceb90 msgr2=0x7f1104165580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.786+0000 7f1109580640 1 -- 172.21.15.105:0/557769291 shutdown_connections 2023-11-28T18:24:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.786+0000 7f1109580640 1 -- 172.21.15.105:0/557769291 wait complete. 2023-11-28T18:24:43.808 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:24:43.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.942+0000 7efcdf03a640 1 Processor -- start 2023-11-28T18:24:43.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.942+0000 7efcdf03a640 1 -- start start 2023-11-28T18:24:43.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.942+0000 7efcdf03a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efcd815d1c0 0x7efcd80cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:43.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.942+0000 7efcdf03a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efcd806b8a0 0x7efcd806bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:43.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.942+0000 7efcdf03a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd80cb230 con 0x7efcd806c430 2023-11-28T18:24:43.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.942+0000 7efcdf03a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd806c8c0 con 0x7efcd815d1c0 2023-11-28T18:24:43.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.942+0000 7efcdf03a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd80b02a0 con 0x7efcd806b8a0 2023-11-28T18:24:43.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.942+0000 7efcde038640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efcd815d1c0 0x7efcd80cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:43.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.942+0000 7efcde038640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efcd815d1c0 0x7efcd80cf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58960/0 (socket says 172.21.15.105:58960) 2023-11-28T18:24:43.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.942+0000 7efcde038640 1 -- 172.21.15.105:0/3631171736 learned_addr learned my addr 172.21.15.105:0/3631171736 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:43.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.942+0000 7efcdd837640 1 --2- 172.21.15.105:0/3631171736 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efcd806b8a0 0x7efcd806bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:43.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.942+0000 7efcde038640 1 -- 172.21.15.105:0/3631171736 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efcd806b8a0 msgr2=0x7efcd806bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:43.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.942+0000 7efcde038640 1 --2- 172.21.15.105:0/3631171736 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efcd806b8a0 0x7efcd806bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.942+0000 7efcde038640 1 -- 172.21.15.105:0/3631171736 >> v1:172.21.15.148:6789/0 conn(0x7efcd806c430 legacy=0x7efcd80cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:43.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.942+0000 7efcde038640 1 -- 172.21.15.105:0/3631171736 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd80d19a0 con 0x7efcd815d1c0 2023-11-28T18:24:43.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.946+0000 7efcde038640 1 --2- 172.21.15.105:0/3631171736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcd815d1c0 0x7efcd80cf910 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009870 tx=0x7efcd401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4a169a36ff1b06 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:43.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.946+0000 7efcdd036640 1 -- 172.21.15.105:0/3631171736 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd4004230 con 0x7efcd815d1c0 2023-11-28T18:24:43.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.946+0000 7efcdd036640 1 -- 172.21.15.105:0/3631171736 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd40a7050 con 0x7efcd815d1c0 2023-11-28T18:24:43.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.946+0000 7efcdd036640 1 -- 172.21.15.105:0/3631171736 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd4003620 con 0x7efcd815d1c0 2023-11-28T18:24:43.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.946+0000 7efcdf03a640 1 -- 172.21.15.105:0/3631171736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcd815d1c0 msgr2=0x7efcd80cf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:43.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.946+0000 7efcdf03a640 1 --2- 172.21.15.105:0/3631171736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcd815d1c0 0x7efcd80cf910 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009870 tx=0x7efcd401c470 comp rx=0 tx=0).stop 2023-11-28T18:24:43.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.946+0000 7efcdf03a640 1 -- 172.21.15.105:0/3631171736 shutdown_connections 2023-11-28T18:24:43.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.946+0000 7efcdf03a640 1 --2- 172.21.15.105:0/3631171736 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efcd806b8a0 0x7efcd806bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.946+0000 7efcdf03a640 1 --2- 172.21.15.105:0/3631171736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcd815d1c0 0x7efcd80cf910 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.946+0000 7efcdf03a640 1 -- 172.21.15.105:0/3631171736 >> 172.21.15.105:0/3631171736 conn(0x7efcd80d0be0 msgr2=0x7efcd80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:43.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.946+0000 7efcdf03a640 1 -- 172.21.15.105:0/3631171736 shutdown_connections 2023-11-28T18:24:43.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.946+0000 7efcdf03a640 1 -- 172.21.15.105:0/3631171736 wait complete. 2023-11-28T18:24:43.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.946+0000 7efcdf03a640 1 Processor -- start 2023-11-28T18:24:43.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.946+0000 7efcdf03a640 1 -- start start 2023-11-28T18:24:43.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcdf03a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efcd806b8a0 0x7efcd815d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:43.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcdf03a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcd806c430 0x7efcd80c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:43.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcdf03a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcd815d1c0 0x7efcd815fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:43.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcdf03a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd815dc60 con 0x7efcd815d1c0 2023-11-28T18:24:43.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcdf03a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd80c7150 con 0x7efcd806c430 2023-11-28T18:24:43.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcdf03a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd815ea60 con 0x7efcd806b8a0 2023-11-28T18:24:43.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcde839640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcd815d1c0 0x7efcd815fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcde839640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcd815d1c0 0x7efcd815fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58974/0 (socket says 172.21.15.105:58974) 2023-11-28T18:24:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcde839640 1 -- 172.21.15.105:0/2071599753 learned_addr learned my addr 172.21.15.105:0/2071599753 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcde038640 1 --2- 172.21.15.105:0/2071599753 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efcd806b8a0 0x7efcd815d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcdd837640 1 --2- 172.21.15.105:0/2071599753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcd806c430 0x7efcd80c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcde839640 1 -- 172.21.15.105:0/2071599753 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efcd806b8a0 msgr2=0x7efcd815d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcde839640 1 --2- 172.21.15.105:0/2071599753 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efcd806b8a0 0x7efcd815d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcde839640 1 -- 172.21.15.105:0/2071599753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcd806c430 msgr2=0x7efcd80c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcde839640 1 --2- 172.21.15.105:0/2071599753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcd806c430 0x7efcd80c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcde839640 1 -- 172.21.15.105:0/2071599753 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd40094d0 con 0x7efcd815d1c0 2023-11-28T18:24:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcde839640 1 --2- 172.21.15.105:0/2071599753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcd815d1c0 0x7efcd815fe40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efcd000c7b0 tx=0x7efcd000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcceffd640 1 -- 172.21.15.105:0/2071599753 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0004b70 con 0x7efcd815d1c0 2023-11-28T18:24:43.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcceffd640 1 -- 172.21.15.105:0/2071599753 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd0004cd0 con 0x7efcd815d1c0 2023-11-28T18:24:43.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcdf03a640 1 -- 172.21.15.105:0/2071599753 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd815de50 con 0x7efcd815d1c0 2023-11-28T18:24:43.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcceffd640 1 -- 172.21.15.105:0/2071599753 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd000aa30 con 0x7efcd815d1c0 2023-11-28T18:24:43.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcdf03a640 1 -- 172.21.15.105:0/2071599753 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd815e410 con 0x7efcd815d1c0 2023-11-28T18:24:43.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.950+0000 7efcdf03a640 1 -- 172.21.15.105:0/2071599753 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd815ebf0 con 0x7efcd815d1c0 2023-11-28T18:24:43.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.954+0000 7efcceffd640 1 -- 172.21.15.105:0/2071599753 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7efcd000ab90 con 0x7efcd815d1c0 2023-11-28T18:24:43.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.954+0000 7efcceffd640 1 --2- 172.21.15.105:0/2071599753 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efcbc0668d0 0x7efcbc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:43.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.954+0000 7efcceffd640 1 -- 172.21.15.105:0/2071599753 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7efcd00848d0 con 0x7efcd815d1c0 2023-11-28T18:24:43.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.954+0000 7efcde038640 1 --2- 172.21.15.105:0/2071599753 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efcbc0668d0 0x7efcbc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:43.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.954+0000 7efcde038640 1 --2- 172.21.15.105:0/2071599753 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efcbc0668d0 0x7efcbc068d90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efcd40ad000 tx=0x7efcd40a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:43.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:43.958+0000 7efcceffd640 1 -- 172.21.15.105:0/2071599753 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efcd0051720 con 0x7efcd815d1c0 2023-11-28T18:24:44.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.110+0000 7efcdf03a640 1 -- 172.21.15.105:0/2071599753 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efcd8074580 con 0x7efcd815d1c0 2023-11-28T18:24:44.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.110+0000 7efcceffd640 1 -- 172.21.15.105:0/2071599753 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18887 (secure 0 0 0) 0x7efcd0054d40 con 0x7efcd815d1c0 2023-11-28T18:24:44.115 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:44.115 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:24:15.048168+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:23:31.485422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T18:23:32.274378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T18:23:36.100071+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:24:44.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.110+0000 7efcdf03a640 1 -- 172.21.15.105:0/2071599753 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efcbc0668d0 msgr2=0x7efcbc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:44.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.110+0000 7efcdf03a640 1 --2- 172.21.15.105:0/2071599753 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efcbc0668d0 0x7efcbc068d90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efcd40ad000 tx=0x7efcd40a6000 comp rx=0 tx=0).stop 2023-11-28T18:24:44.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.110+0000 7efcdf03a640 1 -- 172.21.15.105:0/2071599753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcd815d1c0 msgr2=0x7efcd815fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:44.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.110+0000 7efcdf03a640 1 --2- 172.21.15.105:0/2071599753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcd815d1c0 0x7efcd815fe40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efcd000c7b0 tx=0x7efcd000cc70 comp rx=0 tx=0).stop 2023-11-28T18:24:44.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.110+0000 7efcdf03a640 1 -- 172.21.15.105:0/2071599753 shutdown_connections 2023-11-28T18:24:44.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.110+0000 7efcdf03a640 1 --2- 172.21.15.105:0/2071599753 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efcbc0668d0 0x7efcbc068d90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.110+0000 7efcdf03a640 1 --2- 172.21.15.105:0/2071599753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcd815d1c0 0x7efcd815fe40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.110+0000 7efcdf03a640 1 --2- 172.21.15.105:0/2071599753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcd806c430 0x7efcd80c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.110+0000 7efcdf03a640 1 --2- 172.21.15.105:0/2071599753 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efcd806b8a0 0x7efcd815d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.110+0000 7efcdf03a640 1 -- 172.21.15.105:0/2071599753 >> 172.21.15.105:0/2071599753 conn(0x7efcd80d0be0 msgr2=0x7efcd815c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:44.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.110+0000 7efcdf03a640 1 -- 172.21.15.105:0/2071599753 shutdown_connections 2023-11-28T18:24:44.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.110+0000 7efcdf03a640 1 -- 172.21.15.105:0/2071599753 wait complete. 2023-11-28T18:24:44.135 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T18:24:44.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.274+0000 7f35643f8640 1 Processor -- start 2023-11-28T18:24:44.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.274+0000 7f35643f8640 1 -- start start 2023-11-28T18:24:44.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.274+0000 7f35643f8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f355c15d3c0 0x7f355c071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:44.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.274+0000 7f35643f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f355c161870 0x7f355c0ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:44.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.274+0000 7f35643f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f355c070a10 0x7f355c070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:44.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.274+0000 7f35643f8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f355c0713c0 con 0x7f355c15d3c0 2023-11-28T18:24:44.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.274+0000 7f35643f8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f355c0cefa0 con 0x7f355c070a10 2023-11-28T18:24:44.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.274+0000 7f35643f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355c15d7a0 con 0x7f355c161870 2023-11-28T18:24:44.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.274+0000 7f3562194640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f355c15d3c0 0x7f355c071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:44.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.274+0000 7f3562194640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f355c15d3c0 0x7f355c071a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58980/0 (socket says 172.21.15.105:58980) 2023-11-28T18:24:44.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.274+0000 7f3562194640 1 -- 172.21.15.105:0/1407380629 learned_addr learned my addr 172.21.15.105:0/1407380629 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:44.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.274+0000 7f3561993640 1 --2- 172.21.15.105:0/1407380629 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f355c161870 0x7f355c0ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:44.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.274+0000 7f3562995640 1 --2- 172.21.15.105:0/1407380629 >> v2:172.21.15.148:3300/0 conn(0x7f355c070a10 0x7f355c070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:44.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.274+0000 7f3562194640 1 -- 172.21.15.105:0/1407380629 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f355c161870 msgr2=0x7f355c0ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:44.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.274+0000 7f3562194640 1 --2- 172.21.15.105:0/1407380629 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f355c161870 0x7f355c0ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.274+0000 7f3562194640 1 -- 172.21.15.105:0/1407380629 >> v2:172.21.15.148:3300/0 conn(0x7f355c070a10 msgr2=0x7f355c070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:44.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.274+0000 7f3562194640 1 --2- 172.21.15.105:0/1407380629 >> v2:172.21.15.148:3300/0 conn(0x7f355c070a10 0x7f355c070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.274+0000 7f3562194640 1 -- 172.21.15.105:0/1407380629 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c0ce9e0 con 0x7f355c15d3c0 2023-11-28T18:24:44.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.278+0000 7f3562194640 1 --2- 172.21.15.105:0/1407380629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f355c15d3c0 0x7f355c071a10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f355c0cf4d0 tx=0x7f355801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675ae59c63c72700 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:44.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.278+0000 7f3561192640 1 -- 172.21.15.105:0/1407380629 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35580aa5f0 con 0x7f355c15d3c0 2023-11-28T18:24:44.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.278+0000 7f3561192640 1 -- 172.21.15.105:0/1407380629 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35580aa750 con 0x7f355c15d3c0 2023-11-28T18:24:44.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.278+0000 7f3561192640 1 -- 172.21.15.105:0/1407380629 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35580aaae0 con 0x7f355c15d3c0 2023-11-28T18:24:44.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.278+0000 7f35643f8640 1 -- 172.21.15.105:0/1407380629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f355c15d3c0 msgr2=0x7f355c071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:44.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.278+0000 7f35643f8640 1 --2- 172.21.15.105:0/1407380629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f355c15d3c0 0x7f355c071a10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f355c0cf4d0 tx=0x7f355801c4d0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.278+0000 7f35643f8640 1 -- 172.21.15.105:0/1407380629 shutdown_connections 2023-11-28T18:24:44.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.278+0000 7f35643f8640 1 --2- 172.21.15.105:0/1407380629 >> v2:172.21.15.148:3300/0 conn(0x7f355c070a10 0x7f355c070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.278+0000 7f35643f8640 1 --2- 172.21.15.105:0/1407380629 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f355c161870 0x7f355c0ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.278+0000 7f35643f8640 1 --2- 172.21.15.105:0/1407380629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f355c15d3c0 0x7f355c071a10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.278+0000 7f35643f8640 1 -- 172.21.15.105:0/1407380629 >> 172.21.15.105:0/1407380629 conn(0x7f355c0cf540 msgr2=0x7f355c0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:44.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.278+0000 7f35643f8640 1 -- 172.21.15.105:0/1407380629 shutdown_connections 2023-11-28T18:24:44.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.278+0000 7f35643f8640 1 -- 172.21.15.105:0/1407380629 wait complete. 2023-11-28T18:24:44.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f35643f8640 1 Processor -- start 2023-11-28T18:24:44.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f35643f8640 1 -- start start 2023-11-28T18:24:44.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f35643f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f355c070a10 0x7f355c14e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:44.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f35643f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f355c15d3c0 0x7f355c14ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:44.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f35643f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f355c161870 0x7f355c14f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:44.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f35643f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c15ef30 con 0x7f355c15d3c0 2023-11-28T18:24:44.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f35643f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c15e690 con 0x7f355c161870 2023-11-28T18:24:44.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f35643f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355c15e970 con 0x7f355c070a10 2023-11-28T18:24:44.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f3561993640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f355c15d3c0 0x7f355c14ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:44.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f3561993640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f355c15d3c0 0x7f355c14ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58994/0 (socket says 172.21.15.105:58994) 2023-11-28T18:24:44.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f3561993640 1 -- 172.21.15.105:0/247782857 learned_addr learned my addr 172.21.15.105:0/247782857 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:44.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f3562194640 1 --2- 172.21.15.105:0/247782857 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f355c070a10 0x7f355c14e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:44.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f3562995640 1 --2- 172.21.15.105:0/247782857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f355c161870 0x7f355c14f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:44.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f3561993640 1 -- 172.21.15.105:0/247782857 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f355c070a10 msgr2=0x7f355c14e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:44.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f3561993640 1 --2- 172.21.15.105:0/247782857 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f355c070a10 0x7f355c14e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f3561993640 1 -- 172.21.15.105:0/247782857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f355c161870 msgr2=0x7f355c14f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:44.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f3561993640 1 --2- 172.21.15.105:0/247782857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f355c161870 0x7f355c14f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f3561993640 1 -- 172.21.15.105:0/247782857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35580094d0 con 0x7f355c15d3c0 2023-11-28T18:24:44.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f3561993640 1 --2- 172.21.15.105:0/247782857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f355c15d3c0 0x7f355c14ec50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f354c00d800 tx=0x7f354c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:44.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f3552ffd640 1 -- 172.21.15.105:0/247782857 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c00b6c0 con 0x7f355c15d3c0 2023-11-28T18:24:44.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f3552ffd640 1 -- 172.21.15.105:0/247782857 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354c00b820 con 0x7f355c15d3c0 2023-11-28T18:24:44.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f3552ffd640 1 -- 172.21.15.105:0/247782857 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c00bb90 con 0x7f355c15d3c0 2023-11-28T18:24:44.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f35643f8640 1 -- 172.21.15.105:0/247782857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355c15ec50 con 0x7f355c15d3c0 2023-11-28T18:24:44.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f35643f8640 1 -- 172.21.15.105:0/247782857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f355c15e2d0 con 0x7f355c15d3c0 2023-11-28T18:24:44.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.282+0000 7f35643f8640 1 -- 172.21.15.105:0/247782857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f355c1516f0 con 0x7f355c15d3c0 2023-11-28T18:24:44.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.286+0000 7f3552ffd640 1 -- 172.21.15.105:0/247782857 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f354c00bcf0 con 0x7f355c15d3c0 2023-11-28T18:24:44.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.286+0000 7f3552ffd640 1 --2- 172.21.15.105:0/247782857 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f35400668d0 0x7f3540068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:44.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.286+0000 7f3552ffd640 1 -- 172.21.15.105:0/247782857 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f354c084cc0 con 0x7f355c15d3c0 2023-11-28T18:24:44.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.286+0000 7f3562194640 1 --2- 172.21.15.105:0/247782857 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f35400668d0 0x7f3540068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:44.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.286+0000 7f3562194640 1 --2- 172.21.15.105:0/247782857 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f35400668d0 0x7f3540068d90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f35580ae000 tx=0x7f35580b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:44.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.290+0000 7f3552ffd640 1 -- 172.21.15.105:0/247782857 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f355c1516f0 con 0x7f355c15d3c0 2023-11-28T18:24:44.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.470+0000 7f35643f8640 1 -- 172.21.15.105:0/247782857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f355c1516f0 con 0x7f355c15d3c0 2023-11-28T18:24:44.478 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-11-28T18:24:44.474+0000 7fd89b32a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T18:24:44.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.474+0000 7f3552ffd640 1 -- 172.21.15.105:0/247782857 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f355c1516f0 con 0x7f355c15d3c0 2023-11-28T18:24:44.480 INFO:teuthology.orchestra.run.smithi105.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T18:24:44.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.478+0000 7f35643f8640 1 -- 172.21.15.105:0/247782857 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f35400668d0 msgr2=0x7f3540068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:44.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.478+0000 7f35643f8640 1 --2- 172.21.15.105:0/247782857 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f35400668d0 0x7f3540068d90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f35580ae000 tx=0x7f35580b5040 comp rx=0 tx=0).stop 2023-11-28T18:24:44.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.478+0000 7f35643f8640 1 -- 172.21.15.105:0/247782857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f355c15d3c0 msgr2=0x7f355c14ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:44.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.478+0000 7f35643f8640 1 --2- 172.21.15.105:0/247782857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f355c15d3c0 0x7f355c14ec50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f354c00d800 tx=0x7f354c00dcc0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.478+0000 7f35643f8640 1 -- 172.21.15.105:0/247782857 shutdown_connections 2023-11-28T18:24:44.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.478+0000 7f35643f8640 1 --2- 172.21.15.105:0/247782857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f355c161870 0x7f355c14f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.478+0000 7f35643f8640 1 --2- 172.21.15.105:0/247782857 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f35400668d0 0x7f3540068d90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.478+0000 7f35643f8640 1 --2- 172.21.15.105:0/247782857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f355c15d3c0 0x7f355c14ec50 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.478+0000 7f35643f8640 1 --2- 172.21.15.105:0/247782857 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f355c070a10 0x7f355c14e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.478+0000 7f35643f8640 1 -- 172.21.15.105:0/247782857 >> 172.21.15.105:0/247782857 conn(0x7f355c0cf540 msgr2=0x7f355c0af9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:44.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.478+0000 7f35643f8640 1 -- 172.21.15.105:0/247782857 shutdown_connections 2023-11-28T18:24:44.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.478+0000 7f35643f8640 1 -- 172.21.15.105:0/247782857 wait complete. 2023-11-28T18:24:44.496 INFO:tasks.ceph.mds.a.smithi105.stderr: -4348> 2023-11-28T18:23:37.876+0000 7f30d6928640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T18:24:44.513 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T18:24:44.549 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2023-11-28T18:24:44.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.650+0000 7f2564b72640 1 Processor -- start 2023-11-28T18:24:44.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.650+0000 7f2564b72640 1 -- start start 2023-11-28T18:24:44.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.650+0000 7f2564b72640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25600cdd80 0x7f256006c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:44.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.650+0000 7f2564b72640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f25600ce230 0x7f25600cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:44.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.650+0000 7f2564b72640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25600b3c40 con 0x7f256015d320 2023-11-28T18:24:44.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.650+0000 7f2564b72640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f256006ca80 con 0x7f25600ce230 2023-11-28T18:24:44.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.650+0000 7f2564b72640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256015d700 con 0x7f25600cdd80 2023-11-28T18:24:44.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.650+0000 7f255e59c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f256015d320 0x7f25600cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59832/0 (socket says 172.21.15.105:59832) 2023-11-28T18:24:44.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.650+0000 7f255e59c640 1 -- 172.21.15.105:0/4000299552 learned_addr learned my addr 172.21.15.105:0/4000299552 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:44.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.650+0000 7f255ed9d640 1 --2- 172.21.15.105:0/4000299552 >> v2:172.21.15.105:3300/0 conn(0x7f25600ce230 0x7f25600cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:44.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.650+0000 7f2555d9b640 1 --2- 172.21.15.105:0/4000299552 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25600cdd80 0x7f256006c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:44.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.650+0000 7f255ed9d640 1 -- 172.21.15.105:0/4000299552 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25600cdd80 msgr2=0x7f256006c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:44.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.650+0000 7f255ed9d640 1 --2- 172.21.15.105:0/4000299552 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25600cdd80 0x7f256006c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.650+0000 7f255ed9d640 1 -- 172.21.15.105:0/4000299552 >> v1:172.21.15.105:6789/0 conn(0x7f256015d320 legacy=0x7f25600cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:44.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.650+0000 7f255ed9d640 1 -- 172.21.15.105:0/4000299552 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256006c7a0 con 0x7f25600ce230 2023-11-28T18:24:44.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.650+0000 7f2555d9b640 1 --2- 172.21.15.105:0/4000299552 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25600cdd80 0x7f256006c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:44.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.654+0000 7f255ed9d640 1 --2- 172.21.15.105:0/4000299552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25600ce230 0x7f25600cc0c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2558009d90 tx=0x7f255801c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41adac2d5b5015f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:44.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.654+0000 7f255dd9b640 1 -- 172.21.15.105:0/4000299552 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25580a5980 con 0x7f25600ce230 2023-11-28T18:24:44.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.654+0000 7f255dd9b640 1 -- 172.21.15.105:0/4000299552 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25580a5ae0 con 0x7f25600ce230 2023-11-28T18:24:44.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.654+0000 7f255dd9b640 1 -- 172.21.15.105:0/4000299552 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25580a5e70 con 0x7f25600ce230 2023-11-28T18:24:44.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.654+0000 7f2564b72640 1 -- 172.21.15.105:0/4000299552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25600ce230 msgr2=0x7f25600cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:44.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.654+0000 7f2564b72640 1 --2- 172.21.15.105:0/4000299552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25600ce230 0x7f25600cc0c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2558009d90 tx=0x7f255801c8c0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.654+0000 7f2564b72640 1 -- 172.21.15.105:0/4000299552 shutdown_connections 2023-11-28T18:24:44.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.654+0000 7f2564b72640 1 --2- 172.21.15.105:0/4000299552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25600ce230 0x7f25600cc0c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.654+0000 7f2564b72640 1 --2- 172.21.15.105:0/4000299552 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25600cdd80 0x7f256006c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.654+0000 7f2564b72640 1 -- 172.21.15.105:0/4000299552 >> 172.21.15.105:0/4000299552 conn(0x7f25600d0df0 msgr2=0x7f25600d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:44.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.654+0000 7f2564b72640 1 -- 172.21.15.105:0/4000299552 shutdown_connections 2023-11-28T18:24:44.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.654+0000 7f2564b72640 1 -- 172.21.15.105:0/4000299552 wait complete. 2023-11-28T18:24:44.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f2564b72640 1 Processor -- start 2023-11-28T18:24:44.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f2564b72640 1 -- start start 2023-11-28T18:24:44.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f2564b72640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25600cdd80 0x7f2560159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:44.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f2564b72640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25600ce230 0x7f2560154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:44.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f2564b72640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f256015d320 0x7f2560157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:44.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f2564b72640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25600b6d90 con 0x7f256015d320 2023-11-28T18:24:44.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f2564b72640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2560159b90 con 0x7f25600cdd80 2023-11-28T18:24:44.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f2564b72640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25601554d0 con 0x7f25600ce230 2023-11-28T18:24:44.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f255ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f256015d320 0x7f2560157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:44.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f255ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f256015d320 0x7f2560157bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59038/0 (socket says 172.21.15.105:59038) 2023-11-28T18:24:44.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f255ed9d640 1 -- 172.21.15.105:0/3958211339 learned_addr learned my addr 172.21.15.105:0/3958211339 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:44.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f2555d9b640 1 --2- 172.21.15.105:0/3958211339 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25600ce230 0x7f2560154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:44.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f255e59c640 1 --2- 172.21.15.105:0/3958211339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25600cdd80 0x7f2560159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:44.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f255ed9d640 1 -- 172.21.15.105:0/3958211339 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25600ce230 msgr2=0x7f2560154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:44.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f255ed9d640 1 --2- 172.21.15.105:0/3958211339 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25600ce230 0x7f2560154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f255ed9d640 1 -- 172.21.15.105:0/3958211339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25600cdd80 msgr2=0x7f2560159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:44.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f255ed9d640 1 --2- 172.21.15.105:0/3958211339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25600cdd80 0x7f2560159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:44.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f255ed9d640 1 -- 172.21.15.105:0/3958211339 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2558009a70 con 0x7f256015d320 2023-11-28T18:24:44.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f255ed9d640 1 --2- 172.21.15.105:0/3958211339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f256015d320 0x7f2560157bd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f255801af70 tx=0x7f2558019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:44.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f25577fe640 1 -- 172.21.15.105:0/3958211339 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25580b9860 con 0x7f256015d320 2023-11-28T18:24:44.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f25577fe640 1 -- 172.21.15.105:0/3958211339 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25580b99c0 con 0x7f256015d320 2023-11-28T18:24:44.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f25577fe640 1 -- 172.21.15.105:0/3958211339 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25580b9d30 con 0x7f256015d320 2023-11-28T18:24:44.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f2564b72640 1 -- 172.21.15.105:0/3958211339 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25600b6ab0 con 0x7f256015d320 2023-11-28T18:24:44.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.658+0000 7f2564b72640 1 -- 172.21.15.105:0/3958211339 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25601573d0 con 0x7f256015d320 2023-11-28T18:24:44.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.662+0000 7f2564b72640 1 -- 172.21.15.105:0/3958211339 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2560156f50 con 0x7f256015d320 2023-11-28T18:24:44.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.662+0000 7f25577fe640 1 -- 172.21.15.105:0/3958211339 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f25580ad440 con 0x7f256015d320 2023-11-28T18:24:44.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.662+0000 7f25577fe640 1 --2- 172.21.15.105:0/3958211339 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f25400668d0 0x7f2540068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:44.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.662+0000 7f25577fe640 1 -- 172.21.15.105:0/3958211339 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6807+0+0 (secure 0 0 0) 0x7f255811fc20 con 0x7f256015d320 2023-11-28T18:24:44.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.662+0000 7f255e59c640 1 --2- 172.21.15.105:0/3958211339 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f25400668d0 0x7f2540068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:44.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.662+0000 7f255e59c640 1 --2- 172.21.15.105:0/3958211339 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f25400668d0 0x7f2540068d90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2550005890 tx=0x7f2550009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:44.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.666+0000 7f25577fe640 1 -- 172.21.15.105:0/3958211339 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2560156f50 con 0x7f256015d320 2023-11-28T18:24:44.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:44.846+0000 7f2564b72640 1 -- 172.21.15.105:0/3958211339 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2560156f50 con 0x7f256015d320 2023-11-28T18:24:45.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.478+0000 7f25577fe640 1 -- 172.21.15.105:0/3958211339 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2560156f50 con 0x7f256015d320 2023-11-28T18:24:45.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.478+0000 7f2564b72640 1 -- 172.21.15.105:0/3958211339 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f25400668d0 msgr2=0x7f2540068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:45.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.478+0000 7f2564b72640 1 --2- 172.21.15.105:0/3958211339 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f25400668d0 0x7f2540068d90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2550005890 tx=0x7f2550009040 comp rx=0 tx=0).stop 2023-11-28T18:24:45.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.478+0000 7f2564b72640 1 -- 172.21.15.105:0/3958211339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f256015d320 msgr2=0x7f2560157bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:45.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.478+0000 7f2564b72640 1 --2- 172.21.15.105:0/3958211339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f256015d320 0x7f2560157bd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f255801af70 tx=0x7f2558019660 comp rx=0 tx=0).stop 2023-11-28T18:24:45.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.478+0000 7f2564b72640 1 -- 172.21.15.105:0/3958211339 shutdown_connections 2023-11-28T18:24:45.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.478+0000 7f2564b72640 1 --2- 172.21.15.105:0/3958211339 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f25400668d0 0x7f2540068d90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:45.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.478+0000 7f2564b72640 1 --2- 172.21.15.105:0/3958211339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f256015d320 0x7f2560157bd0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:45.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.478+0000 7f2564b72640 1 --2- 172.21.15.105:0/3958211339 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25600ce230 0x7f2560154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:45.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.478+0000 7f2564b72640 1 --2- 172.21.15.105:0/3958211339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25600cdd80 0x7f2560159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:45.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.478+0000 7f2564b72640 1 -- 172.21.15.105:0/3958211339 >> 172.21.15.105:0/3958211339 conn(0x7f25600d0df0 msgr2=0x7f256006ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:45.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.478+0000 7f2564b72640 1 -- 172.21.15.105:0/3958211339 shutdown_connections 2023-11-28T18:24:45.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.478+0000 7f2564b72640 1 -- 172.21.15.105:0/3958211339 wait complete. 2023-11-28T18:24:45.507 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T18:24:45.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.690+0000 7f49d4dc4640 1 Processor -- start 2023-11-28T18:24:45.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.690+0000 7f49d4dc4640 1 -- start start 2023-11-28T18:24:45.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.690+0000 7f49d4dc4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49d0159f40 0x7f49d00caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.690+0000 7f49d4dc4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f49d00cc760 0x7f49d00ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.690+0000 7f49d4dc4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f49d00ad680 0x7f49d015da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.690+0000 7f49d4dc4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d006bf50 con 0x7f49d00cc760 2023-11-28T18:24:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.690+0000 7f49d4dc4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d00ccbc0 con 0x7f49d00ad680 2023-11-28T18:24:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.690+0000 7f49d4dc4640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d015a320 con 0x7f49d0159f40 2023-11-28T18:24:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.690+0000 7f49c5d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f49d00cc760 0x7f49d00ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.690+0000 7f49c5d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f49d00cc760 0x7f49d00ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59054/0 (socket says 172.21.15.105:59054) 2023-11-28T18:24:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.690+0000 7f49c5d9b640 1 -- 172.21.15.105:0/2983540123 learned_addr learned my addr 172.21.15.105:0/2983540123 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.690+0000 7f49ce59c640 1 --2- 172.21.15.105:0/2983540123 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49d0159f40 0x7f49d00caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.690+0000 7f49ced9d640 1 --2- 172.21.15.105:0/2983540123 >> v2:172.21.15.148:3300/0 conn(0x7f49d00ad680 0x7f49d015da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:45.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.690+0000 7f49c5d9b640 1 -- 172.21.15.105:0/2983540123 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49d0159f40 msgr2=0x7f49d00caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:45.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.690+0000 7f49c5d9b640 1 --2- 172.21.15.105:0/2983540123 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49d0159f40 0x7f49d00caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:45.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.690+0000 7f49c5d9b640 1 -- 172.21.15.105:0/2983540123 >> v2:172.21.15.148:3300/0 conn(0x7f49d00ad680 msgr2=0x7f49d015da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:45.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.690+0000 7f49c5d9b640 1 --2- 172.21.15.105:0/2983540123 >> v2:172.21.15.148:3300/0 conn(0x7f49d00ad680 0x7f49d015da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:45.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.690+0000 7f49c5d9b640 1 -- 172.21.15.105:0/2983540123 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d0071380 con 0x7f49d00cc760 2023-11-28T18:24:45.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.694+0000 7f49c5d9b640 1 --2- 172.21.15.105:0/2983540123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49d00cc760 0x7f49d00ad140 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f49d00724d0 tx=0x7f49b801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94afa1385a46e7e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:45.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.694+0000 7f49cdd9b640 1 -- 172.21.15.105:0/2983540123 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b80a5790 con 0x7f49d00cc760 2023-11-28T18:24:45.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.694+0000 7f49cdd9b640 1 -- 172.21.15.105:0/2983540123 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b80a58f0 con 0x7f49d00cc760 2023-11-28T18:24:45.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.694+0000 7f49cdd9b640 1 -- 172.21.15.105:0/2983540123 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b80a5c80 con 0x7f49d00cc760 2023-11-28T18:24:45.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.694+0000 7f49d4dc4640 1 -- 172.21.15.105:0/2983540123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49d00cc760 msgr2=0x7f49d00ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:45.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.694+0000 7f49d4dc4640 1 --2- 172.21.15.105:0/2983540123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49d00cc760 0x7f49d00ad140 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f49d00724d0 tx=0x7f49b801c910 comp rx=0 tx=0).stop 2023-11-28T18:24:45.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.694+0000 7f49d4dc4640 1 -- 172.21.15.105:0/2983540123 shutdown_connections 2023-11-28T18:24:45.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.694+0000 7f49d4dc4640 1 --2- 172.21.15.105:0/2983540123 >> v2:172.21.15.148:3300/0 conn(0x7f49d00ad680 0x7f49d015da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:45.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.694+0000 7f49d4dc4640 1 --2- 172.21.15.105:0/2983540123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49d00cc760 0x7f49d00ad140 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:45.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.694+0000 7f49d4dc4640 1 --2- 172.21.15.105:0/2983540123 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49d0159f40 0x7f49d00caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:45.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.694+0000 7f49d4dc4640 1 -- 172.21.15.105:0/2983540123 >> 172.21.15.105:0/2983540123 conn(0x7f49d00ceb90 msgr2=0x7f49d00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:45.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.694+0000 7f49d4dc4640 1 -- 172.21.15.105:0/2983540123 shutdown_connections 2023-11-28T18:24:45.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.694+0000 7f49d4dc4640 1 -- 172.21.15.105:0/2983540123 wait complete. 2023-11-28T18:24:45.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49d4dc4640 1 Processor -- start 2023-11-28T18:24:45.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49d4dc4640 1 -- start start 2023-11-28T18:24:45.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49d4dc4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49d00ad680 0x7f49d01592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:45.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49d4dc4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49d00cc760 0x7f49d014b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:45.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49d4dc4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d0159f40 0x7f49d015b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:45.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49d4dc4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d015cde0 con 0x7f49d00ad680 2023-11-28T18:24:45.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49d4dc4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d014bf30 con 0x7f49d0159f40 2023-11-28T18:24:45.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49d4dc4640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d0158230 con 0x7f49d00cc760 2023-11-28T18:24:45.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49ce59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49d00ad680 0x7f49d01592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:45.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49ce59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49d00ad680 0x7f49d01592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59068/0 (socket says 172.21.15.105:59068) 2023-11-28T18:24:45.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49ce59c640 1 -- 172.21.15.105:0/736049075 learned_addr learned my addr 172.21.15.105:0/736049075 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:45.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49c5d9b640 1 --2- 172.21.15.105:0/736049075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49d00cc760 0x7f49d014b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:45.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49ced9d640 1 --2- 172.21.15.105:0/736049075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d0159f40 0x7f49d015b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:45.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49ce59c640 1 -- 172.21.15.105:0/736049075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49d00cc760 msgr2=0x7f49d014b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:45.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49ce59c640 1 --2- 172.21.15.105:0/736049075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49d00cc760 0x7f49d014b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:45.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49ce59c640 1 -- 172.21.15.105:0/736049075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d0159f40 msgr2=0x7f49d015b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:45.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49ce59c640 1 --2- 172.21.15.105:0/736049075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d0159f40 0x7f49d015b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:45.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49ce59c640 1 -- 172.21.15.105:0/736049075 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b8009b30 con 0x7f49d00ad680 2023-11-28T18:24:45.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49ced9d640 1 --2- 172.21.15.105:0/736049075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d0159f40 0x7f49d015b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:45.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49ce59c640 1 --2- 172.21.15.105:0/736049075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49d00ad680 0x7f49d01592d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f49c000e870 tx=0x7f49c000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:45.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49c77fe640 1 -- 172.21.15.105:0/736049075 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0009510 con 0x7f49d00ad680 2023-11-28T18:24:45.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49c77fe640 1 -- 172.21.15.105:0/736049075 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c000eec0 con 0x7f49d00ad680 2023-11-28T18:24:45.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49d4dc4640 1 -- 172.21.15.105:0/736049075 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d0158510 con 0x7f49d00ad680 2023-11-28T18:24:45.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49c77fe640 1 -- 172.21.15.105:0/736049075 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c000cb40 con 0x7f49d00ad680 2023-11-28T18:24:45.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.698+0000 7f49d4dc4640 1 -- 172.21.15.105:0/736049075 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d0158ad0 con 0x7f49d00ad680 2023-11-28T18:24:45.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.702+0000 7f49d4dc4640 1 -- 172.21.15.105:0/736049075 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d0067570 con 0x7f49d00ad680 2023-11-28T18:24:45.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.702+0000 7f49c77fe640 1 -- 172.21.15.105:0/736049075 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f49c000cca0 con 0x7f49d00ad680 2023-11-28T18:24:45.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.702+0000 7f49c77fe640 1 --2- 172.21.15.105:0/736049075 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f49b00668d0 0x7f49b0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:45.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.702+0000 7f49c77fe640 1 -- 172.21.15.105:0/736049075 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6807+0+0 (secure 0 0 0) 0x7f49c0084950 con 0x7f49d00ad680 2023-11-28T18:24:45.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.702+0000 7f49c5d9b640 1 --2- 172.21.15.105:0/736049075 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f49b00668d0 0x7f49b0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:45.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.702+0000 7f49c5d9b640 1 --2- 172.21.15.105:0/736049075 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f49b00668d0 0x7f49b0068d90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f49b80ae000 tx=0x7f49b80a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:45.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.706+0000 7f49c77fe640 1 -- 172.21.15.105:0/736049075 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f49c0051750 con 0x7f49d00ad680 2023-11-28T18:24:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:45.858+0000 7f49d4dc4640 1 -- 172.21.15.105:0/736049075 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f49d00cf950 con 0x7f49d00ad680 2023-11-28T18:24:46.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.482+0000 7f49c77fe640 1 -- 172.21.15.105:0/736049075 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7f49c0054d70 con 0x7f49d00ad680 2023-11-28T18:24:46.486 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' removed 2023-11-28T18:24:46.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.482+0000 7f49d4dc4640 1 -- 172.21.15.105:0/736049075 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f49b00668d0 msgr2=0x7f49b0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:46.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.482+0000 7f49d4dc4640 1 --2- 172.21.15.105:0/736049075 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f49b00668d0 0x7f49b0068d90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f49b80ae000 tx=0x7f49b80a7070 comp rx=0 tx=0).stop 2023-11-28T18:24:46.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.482+0000 7f49d4dc4640 1 -- 172.21.15.105:0/736049075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49d00ad680 msgr2=0x7f49d01592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:46.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.482+0000 7f49d4dc4640 1 --2- 172.21.15.105:0/736049075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49d00ad680 0x7f49d01592d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f49c000e870 tx=0x7f49c000ec70 comp rx=0 tx=0).stop 2023-11-28T18:24:46.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.482+0000 7f49d4dc4640 1 -- 172.21.15.105:0/736049075 shutdown_connections 2023-11-28T18:24:46.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.482+0000 7f49d4dc4640 1 --2- 172.21.15.105:0/736049075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d0159f40 0x7f49d015b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:46.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.482+0000 7f49d4dc4640 1 --2- 172.21.15.105:0/736049075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49d00cc760 0x7f49d014b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:46.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.482+0000 7f49d4dc4640 1 --2- 172.21.15.105:0/736049075 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f49b00668d0 0x7f49b0068d90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:46.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.482+0000 7f49d4dc4640 1 --2- 172.21.15.105:0/736049075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49d00ad680 0x7f49d01592d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:46.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.482+0000 7f49d4dc4640 1 -- 172.21.15.105:0/736049075 >> 172.21.15.105:0/736049075 conn(0x7f49d00ceb90 msgr2=0x7f49d00ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:46.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.482+0000 7f49d4dc4640 1 -- 172.21.15.105:0/736049075 shutdown_connections 2023-11-28T18:24:46.490 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.482+0000 7f49d4dc4640 1 -- 172.21.15.105:0/736049075 wait complete. 2023-11-28T18:24:46.517 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T18:24:46.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.654+0000 7fa7fe796640 1 Processor -- start 2023-11-28T18:24:46.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.658+0000 7fa7fe796640 1 -- start start 2023-11-28T18:24:46.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.658+0000 7fa7fe796640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa7f8157910 0x7fa7f80c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:46.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.658+0000 7fa7fe796640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f8070920 con 0x7fa7f806cc90 2023-11-28T18:24:46.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.658+0000 7fa7fe796640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f806d750 con 0x7fa7f815dfe0 2023-11-28T18:24:46.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.658+0000 7fa7fe796640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f8157cf0 con 0x7fa7f8157910 2023-11-28T18:24:46.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.658+0000 7fa7fd794640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa7f8157910 0x7fa7f80c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:46.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.658+0000 7fa7fcf93640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa7f806cc90 0x7fa7f815d8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59842/0 (socket says 172.21.15.105:59842) 2023-11-28T18:24:46.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.658+0000 7fa7fcf93640 1 -- 172.21.15.105:0/3818115883 learned_addr learned my addr 172.21.15.105:0/3818115883 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:46.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.658+0000 7fa7fd794640 1 -- 172.21.15.105:0/3818115883 >> v1:172.21.15.148:6789/0 conn(0x7fa7f815dfe0 legacy=0x7fa7f80716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:46.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.658+0000 7fa7fd794640 1 -- 172.21.15.105:0/3818115883 >> v1:172.21.15.105:6789/0 conn(0x7fa7f806cc90 legacy=0x7fa7f815d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:46.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.658+0000 7fa7effff640 1 -- 172.21.15.105:0/3818115883 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224847814 0 0) 0x7fa7f8070920 con 0x7fa7f806cc90 2023-11-28T18:24:46.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.658+0000 7fa7fd794640 1 -- 172.21.15.105:0/3818115883 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f8070be0 con 0x7fa7f8157910 2023-11-28T18:24:46.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.662+0000 7fa7fd794640 1 --2- 172.21.15.105:0/3818115883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f8157910 0x7fa7f80c9030 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40097c0 tx=0x7fa7f401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c430b8e3a67329 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:46.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.662+0000 7fa7effff640 1 -- 172.21.15.105:0/3818115883 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f40ac630 con 0x7fa7f8157910 2023-11-28T18:24:46.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.662+0000 7fa7effff640 1 -- 172.21.15.105:0/3818115883 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f40ac790 con 0x7fa7f8157910 2023-11-28T18:24:46.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.662+0000 7fa7effff640 1 -- 172.21.15.105:0/3818115883 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f40acb20 con 0x7fa7f8157910 2023-11-28T18:24:46.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.662+0000 7fa7fe796640 1 -- 172.21.15.105:0/3818115883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f8157910 msgr2=0x7fa7f80c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:46.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.662+0000 7fa7fe796640 1 --2- 172.21.15.105:0/3818115883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f8157910 0x7fa7f80c9030 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40097c0 tx=0x7fa7f401c3b0 comp rx=0 tx=0).stop 2023-11-28T18:24:46.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.662+0000 7fa7fe796640 1 -- 172.21.15.105:0/3818115883 shutdown_connections 2023-11-28T18:24:46.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.662+0000 7fa7fe796640 1 --2- 172.21.15.105:0/3818115883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f8157910 0x7fa7f80c9030 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:46.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.662+0000 7fa7fe796640 1 -- 172.21.15.105:0/3818115883 >> 172.21.15.105:0/3818115883 conn(0x7fa7f80ce9e0 msgr2=0x7fa7f80cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:46.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.662+0000 7fa7fe796640 1 -- 172.21.15.105:0/3818115883 shutdown_connections 2023-11-28T18:24:46.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.662+0000 7fa7fe796640 1 -- 172.21.15.105:0/3818115883 wait complete. 2023-11-28T18:24:46.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.662+0000 7fa7fe796640 1 Processor -- start 2023-11-28T18:24:46.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.662+0000 7fa7fe796640 1 -- start start 2023-11-28T18:24:46.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7fe796640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7f806cc90 0x7fa7f80b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:46.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7fe796640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f8157910 0x7fa7f814d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:46.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7fe796640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7f815dfe0 0x7fa7f80c3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7fe796640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f80b1a70 con 0x7fa7f8157910 2023-11-28T18:24:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7fe796640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f815c190 con 0x7fa7f815dfe0 2023-11-28T18:24:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7fe796640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f815bbd0 con 0x7fa7f806cc90 2023-11-28T18:24:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7fcf93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f8157910 0x7fa7f814d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7fcf93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f8157910 0x7fa7f814d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59090/0 (socket says 172.21.15.105:59090) 2023-11-28T18:24:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7fcf93640 1 -- 172.21.15.105:0/3523011883 learned_addr learned my addr 172.21.15.105:0/3523011883 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7fdf95640 1 --2- 172.21.15.105:0/3523011883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7f815dfe0 0x7fa7f80c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7fd794640 1 --2- 172.21.15.105:0/3523011883 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7f806cc90 0x7fa7f80b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7fcf93640 1 -- 172.21.15.105:0/3523011883 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7f806cc90 msgr2=0x7fa7f80b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7fcf93640 1 --2- 172.21.15.105:0/3523011883 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7f806cc90 0x7fa7f80b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7fcf93640 1 -- 172.21.15.105:0/3523011883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7f815dfe0 msgr2=0x7fa7f80c3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7fcf93640 1 --2- 172.21.15.105:0/3523011883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7f815dfe0 0x7fa7f80c3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7fcf93640 1 -- 172.21.15.105:0/3523011883 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f4009420 con 0x7fa7f8157910 2023-11-28T18:24:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7fdf95640 1 --2- 172.21.15.105:0/3523011883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7f815dfe0 0x7fa7f80c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7fcf93640 1 --2- 172.21.15.105:0/3523011883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f8157910 0x7fa7f814d250 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800c480 tx=0x7fa7e800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7edffb640 1 -- 172.21.15.105:0/3523011883 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e80074e0 con 0x7fa7f8157910 2023-11-28T18:24:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7edffb640 1 -- 172.21.15.105:0/3523011883 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e8007660 con 0x7fa7f8157910 2023-11-28T18:24:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7edffb640 1 -- 172.21.15.105:0/3523011883 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8004a30 con 0x7fa7f8157910 2023-11-28T18:24:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7fe796640 1 -- 172.21.15.105:0/3523011883 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f8072360 con 0x7fa7f8157910 2023-11-28T18:24:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7fe796640 1 -- 172.21.15.105:0/3523011883 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f80b1690 con 0x7fa7f8157910 2023-11-28T18:24:46.674 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7fe796640 1 -- 172.21.15.105:0/3523011883 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f80b1bb0 con 0x7fa7f8157910 2023-11-28T18:24:46.674 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.666+0000 7fa7edffb640 1 -- 172.21.15.105:0/3523011883 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa7e8004b90 con 0x7fa7f8157910 2023-11-28T18:24:46.674 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.670+0000 7fa7edffb640 1 --2- 172.21.15.105:0/3523011883 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa7dc066900 0x7fa7dc068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:46.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.670+0000 7fa7edffb640 1 -- 172.21.15.105:0/3523011883 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6369+0+0 (secure 0 0 0) 0x7fa7e80838c0 con 0x7fa7f8157910 2023-11-28T18:24:46.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.670+0000 7fa7fd794640 1 --2- 172.21.15.105:0/3523011883 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa7dc066900 0x7fa7dc068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:46.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.670+0000 7fa7fd794640 1 --2- 172.21.15.105:0/3523011883 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa7dc066900 0x7fa7dc068dc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40ae000 tx=0x7fa7f40a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:46.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.674+0000 7fa7edffb640 1 -- 172.21.15.105:0/3523011883 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa7f80b1bb0 con 0x7fa7f8157910 2023-11-28T18:24:46.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:46.822+0000 7fa7fe796640 1 -- 172.21.15.105:0/3523011883 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa7f80b1bb0 con 0x7fa7f8157910 2023-11-28T18:24:47.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.490+0000 7fa7edffb640 1 -- 172.21.15.105:0/3523011883 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa7f80b1bb0 con 0x7fa7f8157910 2023-11-28T18:24:47.495 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' removed 2023-11-28T18:24:47.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.494+0000 7fa7fe796640 1 -- 172.21.15.105:0/3523011883 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa7dc066900 msgr2=0x7fa7dc068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:47.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.494+0000 7fa7fe796640 1 --2- 172.21.15.105:0/3523011883 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa7dc066900 0x7fa7dc068dc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40ae000 tx=0x7fa7f40a7000 comp rx=0 tx=0).stop 2023-11-28T18:24:47.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.494+0000 7fa7fe796640 1 -- 172.21.15.105:0/3523011883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f8157910 msgr2=0x7fa7f814d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:47.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.494+0000 7fa7fe796640 1 --2- 172.21.15.105:0/3523011883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f8157910 0x7fa7f814d250 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800c480 tx=0x7fa7e800c940 comp rx=0 tx=0).stop 2023-11-28T18:24:47.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.494+0000 7fa7fe796640 1 -- 172.21.15.105:0/3523011883 shutdown_connections 2023-11-28T18:24:47.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.494+0000 7fa7fe796640 1 --2- 172.21.15.105:0/3523011883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7f815dfe0 0x7fa7f80c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:47.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.494+0000 7fa7fe796640 1 --2- 172.21.15.105:0/3523011883 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa7dc066900 0x7fa7dc068dc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:47.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.494+0000 7fa7fe796640 1 --2- 172.21.15.105:0/3523011883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f8157910 0x7fa7f814d250 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:47.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.494+0000 7fa7fe796640 1 --2- 172.21.15.105:0/3523011883 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7f806cc90 0x7fa7f80b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:47.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.494+0000 7fa7fe796640 1 -- 172.21.15.105:0/3523011883 >> 172.21.15.105:0/3523011883 conn(0x7fa7f80ce9e0 msgr2=0x7fa7f8156970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:47.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.494+0000 7fa7fe796640 1 -- 172.21.15.105:0/3523011883 shutdown_connections 2023-11-28T18:24:47.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.494+0000 7fa7fe796640 1 -- 172.21.15.105:0/3523011883 wait complete. 2023-11-28T18:24:47.532 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-11-28T18:24:47.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.670+0000 7fa27506b640 1 Processor -- start 2023-11-28T18:24:47.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.670+0000 7fa27506b640 1 -- start start 2023-11-28T18:24:47.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.674+0000 7fa27506b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa270159f40 0x7fa2700caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:47.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.674+0000 7fa27506b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2700cc760 0x7fa2700ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:47.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.674+0000 7fa27506b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa27006bf50 con 0x7fa2700ad680 2023-11-28T18:24:47.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.674+0000 7fa27506b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2700ccbc0 con 0x7fa270159f40 2023-11-28T18:24:47.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.674+0000 7fa27506b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa27015a320 con 0x7fa2700cc760 2023-11-28T18:24:47.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.674+0000 7fa26f59e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa2700ad680 0x7fa27015da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59858/0 (socket says 172.21.15.105:59858) 2023-11-28T18:24:47.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.674+0000 7fa26f59e640 1 -- 172.21.15.105:0/1732856562 learned_addr learned my addr 172.21.15.105:0/1732856562 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:47.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.674+0000 7fa26659c640 1 --2- 172.21.15.105:0/1732856562 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2700cc760 0x7fa2700ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:47.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.674+0000 7fa26ed9d640 1 --2- 172.21.15.105:0/1732856562 >> v2:172.21.15.148:3300/0 conn(0x7fa270159f40 0x7fa2700caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:47.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.674+0000 7fa26e59c640 1 -- 172.21.15.105:0/1732856562 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116126251 0 0) 0x7fa27006bf50 con 0x7fa2700ad680 2023-11-28T18:24:47.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.674+0000 7fa26e59c640 1 -- 172.21.15.105:0/1732856562 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa25c003530 con 0x7fa2700ad680 2023-11-28T18:24:47.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.674+0000 7fa26659c640 1 -- 172.21.15.105:0/1732856562 >> v2:172.21.15.148:3300/0 conn(0x7fa270159f40 msgr2=0x7fa2700caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:47.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.674+0000 7fa26659c640 1 --2- 172.21.15.105:0/1732856562 >> v2:172.21.15.148:3300/0 conn(0x7fa270159f40 0x7fa2700caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:47.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.674+0000 7fa26659c640 1 -- 172.21.15.105:0/1732856562 >> v1:172.21.15.105:6789/0 conn(0x7fa2700ad680 legacy=0x7fa27015da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:47.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.674+0000 7fa26659c640 1 -- 172.21.15.105:0/1732856562 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa270071380 con 0x7fa2700cc760 2023-11-28T18:24:47.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.678+0000 7fa26659c640 1 --2- 172.21.15.105:0/1732856562 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2700cc760 0x7fa2700ad140 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa2580097c0 tx=0x7fa25801e5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61525737c470489d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:47.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.678+0000 7fa26e59c640 1 -- 172.21.15.105:0/1732856562 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2580b2600 con 0x7fa2700cc760 2023-11-28T18:24:47.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.678+0000 7fa26e59c640 1 -- 172.21.15.105:0/1732856562 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2580b2760 con 0x7fa2700cc760 2023-11-28T18:24:47.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.678+0000 7fa26e59c640 1 -- 172.21.15.105:0/1732856562 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2580b2af0 con 0x7fa2700cc760 2023-11-28T18:24:47.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.678+0000 7fa27506b640 1 -- 172.21.15.105:0/1732856562 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2700cc760 msgr2=0x7fa2700ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:47.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.678+0000 7fa27506b640 1 --2- 172.21.15.105:0/1732856562 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2700cc760 0x7fa2700ad140 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa2580097c0 tx=0x7fa25801e5d0 comp rx=0 tx=0).stop 2023-11-28T18:24:47.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.678+0000 7fa27506b640 1 -- 172.21.15.105:0/1732856562 shutdown_connections 2023-11-28T18:24:47.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.678+0000 7fa27506b640 1 --2- 172.21.15.105:0/1732856562 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2700cc760 0x7fa2700ad140 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:47.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.678+0000 7fa27506b640 1 --2- 172.21.15.105:0/1732856562 >> v2:172.21.15.148:3300/0 conn(0x7fa270159f40 0x7fa2700caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:47.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.678+0000 7fa27506b640 1 -- 172.21.15.105:0/1732856562 >> 172.21.15.105:0/1732856562 conn(0x7fa2700ceb90 msgr2=0x7fa2700cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:47.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.678+0000 7fa27506b640 1 -- 172.21.15.105:0/1732856562 shutdown_connections 2023-11-28T18:24:47.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.678+0000 7fa27506b640 1 -- 172.21.15.105:0/1732856562 wait complete. 2023-11-28T18:24:47.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.678+0000 7fa27506b640 1 Processor -- start 2023-11-28T18:24:47.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.678+0000 7fa27506b640 1 -- start start 2023-11-28T18:24:47.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa27506b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2700ad680 0x7fa27015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:47.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa27506b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2700cc760 0x7fa27009c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:47.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa27506b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa270159f40 0x7fa27015bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:47.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa27506b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27015d410 con 0x7fa2700ad680 2023-11-28T18:24:47.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa27506b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa270153000 con 0x7fa2700cc760 2023-11-28T18:24:47.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa27506b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2701532e0 con 0x7fa270159f40 2023-11-28T18:24:47.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa26ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2700ad680 0x7fa27015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:47.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa26ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2700ad680 0x7fa27015ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59098/0 (socket says 172.21.15.105:59098) 2023-11-28T18:24:47.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa26ed9d640 1 -- 172.21.15.105:0/3097348409 learned_addr learned my addr 172.21.15.105:0/3097348409 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:47.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa26659c640 1 --2- 172.21.15.105:0/3097348409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2700cc760 0x7fa27009c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:47.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa26f59e640 1 --2- 172.21.15.105:0/3097348409 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa270159f40 0x7fa27015bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:47.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa26ed9d640 1 -- 172.21.15.105:0/3097348409 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa270159f40 msgr2=0x7fa27015bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:47.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa26ed9d640 1 --2- 172.21.15.105:0/3097348409 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa270159f40 0x7fa27015bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:47.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa26ed9d640 1 -- 172.21.15.105:0/3097348409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2700cc760 msgr2=0x7fa27009c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:47.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa26ed9d640 1 --2- 172.21.15.105:0/3097348409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2700cc760 0x7fa27009c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:47.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa26ed9d640 1 -- 172.21.15.105:0/3097348409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa258009420 con 0x7fa2700ad680 2023-11-28T18:24:47.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa26ed9d640 1 --2- 172.21.15.105:0/3097348409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2700ad680 0x7fa27015ced0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa26000efc0 tx=0x7fa26000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:47.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa267fff640 1 -- 172.21.15.105:0/3097348409 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa260009b80 con 0x7fa2700ad680 2023-11-28T18:24:47.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa267fff640 1 -- 172.21.15.105:0/3097348409 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26000f040 con 0x7fa2700ad680 2023-11-28T18:24:47.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa27506b640 1 -- 172.21.15.105:0/3097348409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2701535c0 con 0x7fa2700ad680 2023-11-28T18:24:47.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa267fff640 1 -- 172.21.15.105:0/3097348409 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2600146a0 con 0x7fa2700ad680 2023-11-28T18:24:47.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa27506b640 1 -- 172.21.15.105:0/3097348409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27015c9b0 con 0x7fa2700ad680 2023-11-28T18:24:47.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa27506b640 1 -- 172.21.15.105:0/3097348409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa270067570 con 0x7fa2700ad680 2023-11-28T18:24:47.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.682+0000 7fa267fff640 1 -- 172.21.15.105:0/3097348409 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa260014800 con 0x7fa2700ad680 2023-11-28T18:24:47.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.686+0000 7fa267fff640 1 --2- 172.21.15.105:0/3097348409 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa2500668d0 0x7fa250068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:47.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.686+0000 7fa267fff640 1 -- 172.21.15.105:0/3097348409 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5781+0+0 (secure 0 0 0) 0x7fa260084e50 con 0x7fa2700ad680 2023-11-28T18:24:47.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.686+0000 7fa26659c640 1 --2- 172.21.15.105:0/3097348409 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa2500668d0 0x7fa250068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:47.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.686+0000 7fa26659c640 1 --2- 172.21.15.105:0/3097348409 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa2500668d0 0x7fa250068d90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa2580b1000 tx=0x7fa258008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:47.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.690+0000 7fa267fff640 1 -- 172.21.15.105:0/3097348409 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa2600520f0 con 0x7fa2700ad680 2023-11-28T18:24:47.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:47.850+0000 7fa27506b640 1 -- 172.21.15.105:0/3097348409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa27015b8e0 con 0x7fa2700ad680 2023-11-28T18:24:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.490+0000 7fa267fff640 1 -- 172.21.15.105:0/3097348409 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) v1 ==== 178+0+0 (secure 0 0 0) 0x7fa260055710 con 0x7fa2700ad680 2023-11-28T18:24:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data_ec' removed 2023-11-28T18:24:48.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.494+0000 7fa27506b640 1 -- 172.21.15.105:0/3097348409 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa2500668d0 msgr2=0x7fa250068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:48.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.494+0000 7fa27506b640 1 --2- 172.21.15.105:0/3097348409 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa2500668d0 0x7fa250068d90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa2580b1000 tx=0x7fa258008040 comp rx=0 tx=0).stop 2023-11-28T18:24:48.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.494+0000 7fa27506b640 1 -- 172.21.15.105:0/3097348409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2700ad680 msgr2=0x7fa27015ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:48.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.494+0000 7fa27506b640 1 --2- 172.21.15.105:0/3097348409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2700ad680 0x7fa27015ced0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa26000efc0 tx=0x7fa26000c490 comp rx=0 tx=0).stop 2023-11-28T18:24:48.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.494+0000 7fa27506b640 1 -- 172.21.15.105:0/3097348409 shutdown_connections 2023-11-28T18:24:48.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.494+0000 7fa27506b640 1 --2- 172.21.15.105:0/3097348409 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa270159f40 0x7fa27015bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:48.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.494+0000 7fa27506b640 1 --2- 172.21.15.105:0/3097348409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2700cc760 0x7fa27009c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:48.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.494+0000 7fa27506b640 1 --2- 172.21.15.105:0/3097348409 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa2500668d0 0x7fa250068d90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:48.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.494+0000 7fa27506b640 1 --2- 172.21.15.105:0/3097348409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2700ad680 0x7fa27015ced0 secure :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa26000efc0 tx=0x7fa26000c490 comp rx=0 tx=0).stop 2023-11-28T18:24:48.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.494+0000 7fa27506b640 1 -- 172.21.15.105:0/3097348409 >> 172.21.15.105:0/3097348409 conn(0x7fa2700ceb90 msgr2=0x7fa2700ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:48.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.498+0000 7fa27506b640 1 -- 172.21.15.105:0/3097348409 shutdown_connections 2023-11-28T18:24:48.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.498+0000 7fa27506b640 1 -- 172.21.15.105:0/3097348409 wait complete. 2023-11-28T18:24:48.526 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T18:24:48.526 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T18:24:48.527 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T18:24:48.527 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:24:48.527 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T18:24:48.528 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T18:24:48.528 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T18:24:48.528 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:24:48.528 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T18:24:48.528 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T18:24:48.528 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T18:24:48.528 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:24:48.529 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T18:24:48.529 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T18:24:48.529 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T18:24:48.529 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:24:48.541 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T18:24:48.541 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T18:24:48.542 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T18:24:48.542 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T18:24:48.544 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T18:24:48.544 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T18:24:48.545 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T18:24:48.545 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T18:24:48.592 INFO:tasks.ceph.mds.a:Started 2023-11-28T18:24:48.592 DEBUG:teuthology.parallel:result is None 2023-11-28T18:24:48.593 INFO:tasks.ceph.mds.d:Started 2023-11-28T18:24:48.593 DEBUG:teuthology.parallel:result is None 2023-11-28T18:24:48.594 INFO:tasks.ceph.mds.c:Started 2023-11-28T18:24:48.594 DEBUG:teuthology.parallel:result is None 2023-11-28T18:24:48.595 INFO:tasks.ceph.mds.b:Started 2023-11-28T18:24:48.595 DEBUG:teuthology.parallel:result is None 2023-11-28T18:24:48.595 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T18:24:48.681 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2023-11-28T18:24:48.681 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2023-11-28T18:24:48.682 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2023-11-28T18:24:48.682 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2023-11-28T18:24:48.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.734+0000 7f45e726b640 1 Processor -- start 2023-11-28T18:24:48.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.734+0000 7f45e726b640 1 -- start start 2023-11-28T18:24:48.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.738+0000 7f45e726b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f45e00b4290 0x7f45e00b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:48.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.738+0000 7f45e726b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f45e0071960 0x7f45e015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:48.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.738+0000 7f45e726b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e00b4bb0 con 0x7f45e015b030 2023-11-28T18:24:48.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.738+0000 7f45e726b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e00701d0 con 0x7f45e0071960 2023-11-28T18:24:48.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.738+0000 7f45e726b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e006df80 con 0x7f45e00b4290 2023-11-28T18:24:48.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.738+0000 7f45e4806640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f45e0071960 0x7f45e015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:48.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.738+0000 7f45e4806640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f45e0071960 0x7f45e015f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59132/0 (socket says 172.21.15.105:59132) 2023-11-28T18:24:48.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.738+0000 7f45e4806640 1 -- 172.21.15.105:0/958761181 learned_addr learned my addr 172.21.15.105:0/958761181 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:48.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.738+0000 7f45e5808640 1 --1- 172.21.15.105:0/958761181 >> v1:172.21.15.105:6789/0 conn(0x7f45e015b030 0x7f45e00709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59880/0 (socket says 172.21.15.105:59880) 2023-11-28T18:24:48.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.738+0000 7f45e5007640 1 --2- 172.21.15.105:0/958761181 >> v2:172.21.15.148:3300/0 conn(0x7f45e00b4290 0x7f45e00b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:48.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.738+0000 7f45e4806640 1 -- 172.21.15.105:0/958761181 >> v2:172.21.15.148:3300/0 conn(0x7f45e00b4290 msgr2=0x7f45e00b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:48.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.738+0000 7f45e4806640 1 --2- 172.21.15.105:0/958761181 >> v2:172.21.15.148:3300/0 conn(0x7f45e00b4290 0x7f45e00b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:48.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.738+0000 7f45e4806640 1 -- 172.21.15.105:0/958761181 >> v1:172.21.15.105:6789/0 conn(0x7f45e015b030 legacy=0x7f45e00709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:48.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.738+0000 7f45e4806640 1 -- 172.21.15.105:0/958761181 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e0071df0 con 0x7f45e0071960 2023-11-28T18:24:48.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.738+0000 7f45d7fff640 1 -- 172.21.15.105:0/958761181 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1104132435 0 0) 0x7f45e00b4bb0 con 0x7f45e015b030 2023-11-28T18:24:48.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45e4806640 1 --2- 172.21.15.105:0/958761181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45e0071960 0x7f45e015f3b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f45e0073a70 tx=0x7f45d001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386615928a8cf0ff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:48.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45d7fff640 1 -- 172.21.15.105:0/958761181 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d00aa650 con 0x7f45e0071960 2023-11-28T18:24:48.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45d7fff640 1 -- 172.21.15.105:0/958761181 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d00aa7b0 con 0x7f45e0071960 2023-11-28T18:24:48.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45d7fff640 1 -- 172.21.15.105:0/958761181 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d00aab40 con 0x7f45e0071960 2023-11-28T18:24:48.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45e726b640 1 -- 172.21.15.105:0/958761181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45e0071960 msgr2=0x7f45e015f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:48.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45e726b640 1 --2- 172.21.15.105:0/958761181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45e0071960 0x7f45e015f3b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f45e0073a70 tx=0x7f45d001c4f0 comp rx=0 tx=0).stop 2023-11-28T18:24:48.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45e726b640 1 -- 172.21.15.105:0/958761181 shutdown_connections 2023-11-28T18:24:48.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45e726b640 1 --2- 172.21.15.105:0/958761181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45e0071960 0x7f45e015f3b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:48.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45e726b640 1 --2- 172.21.15.105:0/958761181 >> v2:172.21.15.148:3300/0 conn(0x7f45e00b4290 0x7f45e00b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:48.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45e726b640 1 -- 172.21.15.105:0/958761181 >> 172.21.15.105:0/958761181 conn(0x7f45e00ceb90 msgr2=0x7f45e00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:48.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45e726b640 1 -- 172.21.15.105:0/958761181 shutdown_connections 2023-11-28T18:24:48.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45e726b640 1 -- 172.21.15.105:0/958761181 wait complete. 2023-11-28T18:24:48.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45e726b640 1 Processor -- start 2023-11-28T18:24:48.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45e726b640 1 -- start start 2023-11-28T18:24:48.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45e726b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45e00b4290 0x7f45e0156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:48.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45e726b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e015b030 0x7f45e014e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:48.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45e726b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45e014eb90 0x7f45e014c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:48.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45e726b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e01572f0 con 0x7f45e00b4290 2023-11-28T18:24:48.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45e726b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e0158340 con 0x7f45e014eb90 2023-11-28T18:24:48.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45e726b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e0157520 con 0x7f45e015b030 2023-11-28T18:24:48.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45e5007640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45e00b4290 0x7f45e0156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:48.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45e5007640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45e00b4290 0x7f45e0156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59136/0 (socket says 172.21.15.105:59136) 2023-11-28T18:24:48.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.742+0000 7f45e5007640 1 -- 172.21.15.105:0/1887550374 learned_addr learned my addr 172.21.15.105:0/1887550374 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:48.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.746+0000 7f45e5808640 1 --2- 172.21.15.105:0/1887550374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45e014eb90 0x7f45e014c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:48.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.746+0000 7f45e4806640 1 --2- 172.21.15.105:0/1887550374 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e015b030 0x7f45e014e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:48.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.746+0000 7f45e5007640 1 -- 172.21.15.105:0/1887550374 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e015b030 msgr2=0x7f45e014e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:48.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.746+0000 7f45e5007640 1 --2- 172.21.15.105:0/1887550374 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e015b030 0x7f45e014e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:48.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.746+0000 7f45e5007640 1 -- 172.21.15.105:0/1887550374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45e014eb90 msgr2=0x7f45e014c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:48.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.746+0000 7f45e5007640 1 --2- 172.21.15.105:0/1887550374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45e014eb90 0x7f45e014c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:48.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.746+0000 7f45e5007640 1 -- 172.21.15.105:0/1887550374 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d00094d0 con 0x7f45e00b4290 2023-11-28T18:24:48.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.746+0000 7f45e5007640 1 --2- 172.21.15.105:0/1887550374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45e00b4290 0x7f45e0156db0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00e960 tx=0x7f45dc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:48.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.746+0000 7f45d5ffb640 1 -- 172.21.15.105:0/1887550374 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc009690 con 0x7f45e00b4290 2023-11-28T18:24:48.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.746+0000 7f45d5ffb640 1 -- 172.21.15.105:0/1887550374 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45dc00ca00 con 0x7f45e00b4290 2023-11-28T18:24:48.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.746+0000 7f45d5ffb640 1 -- 172.21.15.105:0/1887550374 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc00cd70 con 0x7f45e00b4290 2023-11-28T18:24:48.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.746+0000 7f45e726b640 1 -- 172.21.15.105:0/1887550374 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e0157800 con 0x7f45e00b4290 2023-11-28T18:24:48.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.746+0000 7f45e726b640 1 -- 172.21.15.105:0/1887550374 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e00ccbc0 con 0x7f45e00b4290 2023-11-28T18:24:48.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.746+0000 7f45e726b640 1 -- 172.21.15.105:0/1887550374 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e0157a40 con 0x7f45e00b4290 2023-11-28T18:24:48.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.746+0000 7f45d5ffb640 1 -- 172.21.15.105:0/1887550374 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f45dc013470 con 0x7f45e00b4290 2023-11-28T18:24:48.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.750+0000 7f45d5ffb640 1 --2- 172.21.15.105:0/1887550374 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f45c40668d0 0x7f45c4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:48.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.750+0000 7f45d5ffb640 1 -- 172.21.15.105:0/1887550374 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f45dc0844c0 con 0x7f45e00b4290 2023-11-28T18:24:48.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.750+0000 7f45e4806640 1 --2- 172.21.15.105:0/1887550374 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f45c40668d0 0x7f45c4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:48.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.750+0000 7f45e4806640 1 --2- 172.21.15.105:0/1887550374 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f45c40668d0 0x7f45c4068d90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f45d00ae000 tx=0x7f45d00a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:48.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.754+0000 7f45d5ffb640 1 -- 172.21.15.105:0/1887550374 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f45dc0518b0 con 0x7f45e00b4290 2023-11-28T18:24:48.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:48.922+0000 7f45e726b640 1 -- 172.21.15.105:0/1887550374 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f45e0067570 con 0x7f45e00b4290 2023-11-28T18:24:49.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.514+0000 7f45d5ffb640 1 -- 172.21.15.105:0/1887550374 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) v1 ==== 97+0+0 (secure 0 0 0) 0x7f45dc054ed0 con 0x7f45e00b4290 2023-11-28T18:24:49.519 INFO:teuthology.orchestra.run.smithi105.stderr: removed all blocklist entries 2023-11-28T18:24:49.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.514+0000 7f45e726b640 1 -- 172.21.15.105:0/1887550374 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f45c40668d0 msgr2=0x7f45c4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:49.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.514+0000 7f45e726b640 1 --2- 172.21.15.105:0/1887550374 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f45c40668d0 0x7f45c4068d90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f45d00ae000 tx=0x7f45d00a7000 comp rx=0 tx=0).stop 2023-11-28T18:24:49.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.514+0000 7f45e726b640 1 -- 172.21.15.105:0/1887550374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45e00b4290 msgr2=0x7f45e0156db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:49.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.514+0000 7f45e726b640 1 --2- 172.21.15.105:0/1887550374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45e00b4290 0x7f45e0156db0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00e960 tx=0x7f45dc00ee20 comp rx=0 tx=0).stop 2023-11-28T18:24:49.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.518+0000 7f45e726b640 1 -- 172.21.15.105:0/1887550374 shutdown_connections 2023-11-28T18:24:49.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.518+0000 7f45e726b640 1 --2- 172.21.15.105:0/1887550374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45e014eb90 0x7f45e014c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:49.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.518+0000 7f45e726b640 1 --2- 172.21.15.105:0/1887550374 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e015b030 0x7f45e014e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:49.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.518+0000 7f45e726b640 1 --2- 172.21.15.105:0/1887550374 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f45c40668d0 0x7f45c4068d90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:49.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.518+0000 7f45e726b640 1 --2- 172.21.15.105:0/1887550374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45e00b4290 0x7f45e0156db0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:49.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.518+0000 7f45e726b640 1 -- 172.21.15.105:0/1887550374 >> 172.21.15.105:0/1887550374 conn(0x7f45e00ceb90 msgr2=0x7f45e0159cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:49.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.518+0000 7f45e726b640 1 -- 172.21.15.105:0/1887550374 shutdown_connections 2023-11-28T18:24:49.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.518+0000 7f45e726b640 1 -- 172.21.15.105:0/1887550374 wait complete. 2023-11-28T18:24:49.546 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T18:24:49.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.682+0000 7f9111e5f640 1 Processor -- start 2023-11-28T18:24:49.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.682+0000 7f9111e5f640 1 -- start start 2023-11-28T18:24:49.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.682+0000 7f9111e5f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f910c15d4f0 0x7f910c0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:49.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.682+0000 7f9111e5f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f910c0b25b0 con 0x7f910c0b2d20 2023-11-28T18:24:49.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.682+0000 7f9111e5f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f910c0cea30 con 0x7f910c0d3c30 2023-11-28T18:24:49.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.682+0000 7f9111e5f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f910c0b2900 con 0x7f910c15d4f0 2023-11-28T18:24:49.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.682+0000 7f910b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f910c15d4f0 0x7f910c0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:49.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.682+0000 7f910bfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f910c0b2d20 0x7f910c0b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59892/0 (socket says 172.21.15.105:59892) 2023-11-28T18:24:49.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.682+0000 7f910bfff640 1 -- 172.21.15.105:0/2194604827 learned_addr learned my addr 172.21.15.105:0/2194604827 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:49.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.682+0000 7f910b7fe640 1 -- 172.21.15.105:0/2194604827 >> v1:172.21.15.148:6789/0 conn(0x7f910c0d3c30 legacy=0x7f910c0d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:49.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.682+0000 7f910b7fe640 1 -- 172.21.15.105:0/2194604827 >> v1:172.21.15.105:6789/0 conn(0x7f910c0b2d20 legacy=0x7f910c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:49.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.682+0000 7f910b7fe640 1 -- 172.21.15.105:0/2194604827 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910c0ced10 con 0x7f910c15d4f0 2023-11-28T18:24:49.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.686+0000 7f910b7fe640 1 --2- 172.21.15.105:0/2194604827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f910c15d4f0 0x7f910c0d0e10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0097c0 tx=0x7f90fc01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b567ecc69e5135f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:49.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.686+0000 7f910affd640 1 -- 172.21.15.105:0/2194604827 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc0ab600 con 0x7f910c15d4f0 2023-11-28T18:24:49.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.686+0000 7f910affd640 1 -- 172.21.15.105:0/2194604827 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90fc0a7050 con 0x7f910c15d4f0 2023-11-28T18:24:49.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.686+0000 7f910affd640 1 -- 172.21.15.105:0/2194604827 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc0ab980 con 0x7f910c15d4f0 2023-11-28T18:24:49.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.686+0000 7f9111e5f640 1 -- 172.21.15.105:0/2194604827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f910c15d4f0 msgr2=0x7f910c0d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:49.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.686+0000 7f9111e5f640 1 --2- 172.21.15.105:0/2194604827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f910c15d4f0 0x7f910c0d0e10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0097c0 tx=0x7f90fc01c380 comp rx=0 tx=0).stop 2023-11-28T18:24:49.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.686+0000 7f9111e5f640 1 -- 172.21.15.105:0/2194604827 shutdown_connections 2023-11-28T18:24:49.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.686+0000 7f9111e5f640 1 --2- 172.21.15.105:0/2194604827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f910c15d4f0 0x7f910c0d0e10 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:49.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.686+0000 7f9111e5f640 1 -- 172.21.15.105:0/2194604827 >> 172.21.15.105:0/2194604827 conn(0x7f910c0d3030 msgr2=0x7f910c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:49.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.686+0000 7f9111e5f640 1 -- 172.21.15.105:0/2194604827 shutdown_connections 2023-11-28T18:24:49.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.686+0000 7f9111e5f640 1 -- 172.21.15.105:0/2194604827 wait complete. 2023-11-28T18:24:49.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.686+0000 7f9111e5f640 1 Processor -- start 2023-11-28T18:24:49.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.686+0000 7f9111e5f640 1 -- start start 2023-11-28T18:24:49.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f9111e5f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c0b2d20 0x7f910c1602b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:49.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f9111e5f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f910c0d3c30 0x7f910c0b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:49.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f9111e5f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f910c15d4f0 0x7f910c0b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:49.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f9111e5f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c0d0a20 con 0x7f910c15d4f0 2023-11-28T18:24:49.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f9111e5f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c0b6f30 con 0x7f910c0b2d20 2023-11-28T18:24:49.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f9111e5f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910c0b8800 con 0x7f910c0d3c30 2023-11-28T18:24:49.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f910bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f910c15d4f0 0x7f910c0b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:49.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f910bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f910c15d4f0 0x7f910c0b82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59160/0 (socket says 172.21.15.105:59160) 2023-11-28T18:24:49.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f910bfff640 1 -- 172.21.15.105:0/4047790889 learned_addr learned my addr 172.21.15.105:0/4047790889 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:49.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f910b7fe640 1 --2- 172.21.15.105:0/4047790889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c0b2d20 0x7f910c1602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:49.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f9103fff640 1 --2- 172.21.15.105:0/4047790889 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f910c0d3c30 0x7f910c0b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:49.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f910bfff640 1 -- 172.21.15.105:0/4047790889 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f910c0d3c30 msgr2=0x7f910c0b69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:49.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f910bfff640 1 --2- 172.21.15.105:0/4047790889 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f910c0d3c30 0x7f910c0b69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:49.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f910bfff640 1 -- 172.21.15.105:0/4047790889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c0b2d20 msgr2=0x7f910c1602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:49.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f910bfff640 1 --2- 172.21.15.105:0/4047790889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c0b2d20 0x7f910c1602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:49.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f910bfff640 1 -- 172.21.15.105:0/4047790889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90fc009420 con 0x7f910c15d4f0 2023-11-28T18:24:49.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f910b7fe640 1 --2- 172.21.15.105:0/4047790889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c0b2d20 0x7f910c1602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:49.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f910bfff640 1 --2- 172.21.15.105:0/4047790889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f910c15d4f0 0x7f910c0b82c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f910400ca80 tx=0x7f910400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:49.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f9108ff9640 1 -- 172.21.15.105:0/4047790889 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910400ac30 con 0x7f910c15d4f0 2023-11-28T18:24:49.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f9108ff9640 1 -- 172.21.15.105:0/4047790889 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910400ad90 con 0x7f910c15d4f0 2023-11-28T18:24:49.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f9108ff9640 1 -- 172.21.15.105:0/4047790889 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9104012600 con 0x7f910c15d4f0 2023-11-28T18:24:49.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f9111e5f640 1 -- 172.21.15.105:0/4047790889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910c160940 con 0x7f910c15d4f0 2023-11-28T18:24:49.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f9111e5f640 1 -- 172.21.15.105:0/4047790889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910c0b5f10 con 0x7f910c15d4f0 2023-11-28T18:24:49.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f9111e5f640 1 -- 172.21.15.105:0/4047790889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f910c067570 con 0x7f910c15d4f0 2023-11-28T18:24:49.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.690+0000 7f9108ff9640 1 -- 172.21.15.105:0/4047790889 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f9104007070 con 0x7f910c15d4f0 2023-11-28T18:24:49.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.694+0000 7f9108ff9640 1 --2- 172.21.15.105:0/4047790889 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f90ec066900 0x7f90ec068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:49.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.694+0000 7f9108ff9640 1 -- 172.21.15.105:0/4047790889 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9104083ff0 con 0x7f910c15d4f0 2023-11-28T18:24:49.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.694+0000 7f910b7fe640 1 --2- 172.21.15.105:0/4047790889 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f90ec066900 0x7f90ec068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:49.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.694+0000 7f910b7fe640 1 --2- 172.21.15.105:0/4047790889 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f90ec066900 0x7f90ec068dc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0ad000 tx=0x7f90fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:49.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.698+0000 7f9108ff9640 1 -- 172.21.15.105:0/4047790889 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f91040b5280 con 0x7f910c15d4f0 2023-11-28T18:24:49.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.898+0000 7f9111e5f640 1 -- 172.21.15.105:0/4047790889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f910c159240 con 0x7f910c15d4f0 2023-11-28T18:24:49.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.898+0000 7f9108ff9640 1 -- 172.21.15.105:0/4047790889 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f9104017090 con 0x7f910c15d4f0 2023-11-28T18:24:49.903 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:49.903 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:24:49.903 INFO:teuthology.orchestra.run.smithi105.stdout: "auth_dump": [ 2023-11-28T18:24:49.904 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.904 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.a", 2023-11-28T18:24:49.904 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlOXSTGxAAU5kGNB3tznYsDNG51vCIZA==", 2023-11-28T18:24:49.904 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.904 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:24:49.904 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:24:49.904 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:24:49.905 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:24:49.905 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.905 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.905 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.905 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.b", 2023-11-28T18:24:49.905 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlrnbWJxAAKefl0YI2MhobnAOIj6WeaQ==", 2023-11-28T18:24:49.905 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.905 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:24:49.905 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:24:49.905 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:24:49.906 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:24:49.906 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.906 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.906 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.906 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.c", 2023-11-28T18:24:49.906 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2Zl5Y7GIhAA5OvgkNNFaI7LnzSsQVdFsQ==", 2023-11-28T18:24:49.906 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.906 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:24:49.906 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:24:49.906 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:24:49.906 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:24:49.907 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.907 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.907 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.907 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.d", 2023-11-28T18:24:49.907 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlmfYdLxAAreI7UE1YCjzkj6yl3s5Fwg==", 2023-11-28T18:24:49.907 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.907 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:24:49.907 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:24:49.907 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:24:49.907 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:24:49.907 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.907 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.908 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.908 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.0", 2023-11-28T18:24:49.908 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDtL2ZlphxQEBAAQ280OXh1zeaotIdVU8oZng==", 2023-11-28T18:24:49.908 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.908 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:24:49.908 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:24:49.908 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:24:49.908 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.908 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.908 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.908 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.1", 2023-11-28T18:24:49.908 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDwL2ZlezmOFxAARRBIOTkVJYfquIdcpW7sQQ==", 2023-11-28T18:24:49.909 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.909 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:24:49.910 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:24:49.910 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:24:49.910 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.910 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.910 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.910 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.2", 2023-11-28T18:24:49.910 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDzL2Zl4YcSHxAA2ww2F4TAD/0H/ZJ+aVMGEQ==", 2023-11-28T18:24:49.910 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.910 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:24:49.910 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:24:49.911 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:24:49.911 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.911 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.911 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.911 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.3", 2023-11-28T18:24:49.911 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD2L2ZlNxWxJRAAImp7TjTF18KwFTZ14DM6EA==", 2023-11-28T18:24:49.911 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.911 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:24:49.911 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:24:49.911 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:24:49.911 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.911 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.912 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.912 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.4", 2023-11-28T18:24:49.912 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD8L2ZlOAieLxAAz8g+anH8uJRfsttUS41GGQ==", 2023-11-28T18:24:49.912 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.912 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:24:49.912 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:24:49.912 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:24:49.912 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.912 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.912 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.913 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.5", 2023-11-28T18:24:49.913 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD/L2Zl5PKTNxAAPk+uIodLPjSd0EEMVT6iEw==", 2023-11-28T18:24:49.913 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.913 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:24:49.913 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:24:49.913 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:24:49.913 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.913 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.913 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.913 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.6", 2023-11-28T18:24:49.913 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQADMGZll+FUAxAAHlym5Z8rd/OF1911ZE2OUg==", 2023-11-28T18:24:49.913 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.914 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:24:49.914 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:24:49.914 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:24:49.914 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.914 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.914 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.914 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.7", 2023-11-28T18:24:49.914 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQAGMGZl8GO4ChAAVn3N6DnxUcs+ew09QvGo2w==", 2023-11-28T18:24:49.914 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.915 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:24:49.915 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:24:49.915 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:24:49.915 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.915 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.915 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.915 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.0", 2023-11-28T18:24:49.915 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlSt0VMxAAQ8+j1BgBPMJ2cuiZj8aG0w==", 2023-11-28T18:24:49.915 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.915 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:24:49.915 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:24:49.915 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:24:49.916 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:24:49.916 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.916 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.916 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.916 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.1", 2023-11-28T18:24:49.916 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDqL2ZlfZ1/ARAAhnJB6oj4Cc6p/oCjM6hTCw==", 2023-11-28T18:24:49.916 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.916 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:24:49.916 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:24:49.916 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:24:49.917 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:24:49.917 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.917 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.917 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.917 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.2", 2023-11-28T18:24:49.917 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlPLtyORAAU2GST6TJzlTfh6YqEb3U3w==", 2023-11-28T18:24:49.917 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.917 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:24:49.917 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:24:49.917 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:24:49.917 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:24:49.918 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.918 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.918 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.918 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.3", 2023-11-28T18:24:49.918 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDqL2ZlhiFuBxAAKQRxCdtqWpAHy1AVvdj9Sg==", 2023-11-28T18:24:49.918 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.920 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:24:49.920 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:24:49.920 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:24:49.920 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:24:49.920 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.920 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.920 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.920 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.admin", 2023-11-28T18:24:49.920 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDoL2ZlkuP2OhAAxsdZEF9122vkH53EwB8Qug==", 2023-11-28T18:24:49.920 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.921 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:24:49.921 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:24:49.921 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:24:49.921 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:24:49.921 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.921 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.921 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.921 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-mds", 2023-11-28T18:24:49.921 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQAMMGZlCD0CDxAA25bR1z+3RQDa1wWxcYr/Pg==", 2023-11-28T18:24:49.921 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.921 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-mds" 2023-11-28T18:24:49.921 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.922 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.922 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.922 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-mgr", 2023-11-28T18:24:49.922 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQAMMGZly0YCDxAA+vLU273pHe4VHlwCzf6gLQ==", 2023-11-28T18:24:49.922 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.922 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-28T18:24:49.922 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.922 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.922 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.922 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-osd", 2023-11-28T18:24:49.922 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQAMMGZlWU8CDxAA/5wwyhQXIETB4Rg794RWiQ==", 2023-11-28T18:24:49.923 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.923 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-osd" 2023-11-28T18:24:49.923 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.923 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.923 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.923 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-rbd", 2023-11-28T18:24:49.923 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQAMMGZlU1cCDxAAXtC8CAWNc2GJDjyoDQv5Xg==", 2023-11-28T18:24:49.923 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.923 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-28T18:24:49.923 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.923 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.925 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.925 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-28T18:24:49.925 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQAMMGZloV8CDxAARWrZZ5M39QPHc9nmYMyLUw==", 2023-11-28T18:24:49.925 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.925 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-28T18:24:49.925 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.925 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.925 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.926 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-rgw", 2023-11-28T18:24:49.926 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQAMMGZlLmgCDxAAaEfv8QmdbGWhtMlOEKWpKw==", 2023-11-28T18:24:49.926 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.926 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-28T18:24:49.926 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.926 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.926 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.926 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.x", 2023-11-28T18:24:49.927 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlEG49FxAAespsG+QK1WTpbmpV+DIwiw==", 2023-11-28T18:24:49.927 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.927 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:24:49.927 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2023-11-28T18:24:49.927 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:24:49.927 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.927 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:24:49.927 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:49.927 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.y", 2023-11-28T18:24:49.927 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlU89RExAAwMJp+8zZ6aFEt0KbiB6toA==", 2023-11-28T18:24:49.927 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:24:49.928 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:24:49.928 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2023-11-28T18:24:49.928 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:24:49.928 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.928 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:49.928 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2023-11-28T18:24:49.928 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:24:49.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.898+0000 7f9111e5f640 1 -- 172.21.15.105:0/4047790889 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f90ec066900 msgr2=0x7f90ec068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:49.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.898+0000 7f9111e5f640 1 --2- 172.21.15.105:0/4047790889 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f90ec066900 0x7f90ec068dc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0ad000 tx=0x7f90fc008040 comp rx=0 tx=0).stop 2023-11-28T18:24:49.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.898+0000 7f9111e5f640 1 -- 172.21.15.105:0/4047790889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f910c15d4f0 msgr2=0x7f910c0b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:49.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.898+0000 7f9111e5f640 1 --2- 172.21.15.105:0/4047790889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f910c15d4f0 0x7f910c0b82c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f910400ca80 tx=0x7f910400cf40 comp rx=0 tx=0).stop 2023-11-28T18:24:49.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.898+0000 7f9111e5f640 1 -- 172.21.15.105:0/4047790889 shutdown_connections 2023-11-28T18:24:49.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.898+0000 7f9111e5f640 1 --2- 172.21.15.105:0/4047790889 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f90ec066900 0x7f90ec068dc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:49.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.898+0000 7f9111e5f640 1 --2- 172.21.15.105:0/4047790889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f910c15d4f0 0x7f910c0b82c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:49.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.898+0000 7f9111e5f640 1 --2- 172.21.15.105:0/4047790889 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f910c0d3c30 0x7f910c0b69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:49.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.898+0000 7f9111e5f640 1 --2- 172.21.15.105:0/4047790889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c0b2d20 0x7f910c1602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:49.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.898+0000 7f9111e5f640 1 -- 172.21.15.105:0/4047790889 >> 172.21.15.105:0/4047790889 conn(0x7f910c0d3030 msgr2=0x7f910c072eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:49.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.898+0000 7f9111e5f640 1 -- 172.21.15.105:0/4047790889 shutdown_connections 2023-11-28T18:24:49.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:49.898+0000 7f9111e5f640 1 -- 172.21.15.105:0/4047790889 wait complete. 2023-11-28T18:24:49.932 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-28T18:24:50.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.066+0000 7f57756dd640 1 Processor -- start 2023-11-28T18:24:50.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.066+0000 7f57756dd640 1 -- start start 2023-11-28T18:24:50.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.066+0000 7f57756dd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5770159610 0x7f57700ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:50.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.066+0000 7f57756dd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f577006cef0 0x7f57700ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:50.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.066+0000 7f57756dd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57700cde30 con 0x7f57700cc760 2023-11-28T18:24:50.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.066+0000 7f57756dd640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57701599f0 con 0x7f577006cef0 2023-11-28T18:24:50.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.066+0000 7f57756dd640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577006de20 con 0x7f5770159610 2023-11-28T18:24:50.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.066+0000 7f576f7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f577006cef0 0x7f57700ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:50.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.066+0000 7f576f7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f577006cef0 0x7f57700ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59168/0 (socket says 172.21.15.105:59168) 2023-11-28T18:24:50.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.066+0000 7f576f7fe640 1 -- 172.21.15.105:0/535520741 learned_addr learned my addr 172.21.15.105:0/535520741 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:50.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.066+0000 7f576effd640 1 --2- 172.21.15.105:0/535520741 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5770159610 0x7f57700ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:50.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.070+0000 7f576f7fe640 1 -- 172.21.15.105:0/535520741 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5770159610 msgr2=0x7f57700ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:50.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.070+0000 7f576f7fe640 1 --2- 172.21.15.105:0/535520741 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5770159610 0x7f57700ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.070+0000 7f576f7fe640 1 -- 172.21.15.105:0/535520741 >> v1:172.21.15.148:6789/0 conn(0x7f57700cc760 legacy=0x7f577006c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:50.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.070+0000 7f576f7fe640 1 -- 172.21.15.105:0/535520741 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57700cd220 con 0x7f577006cef0 2023-11-28T18:24:50.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.070+0000 7f576f7fe640 1 --2- 172.21.15.105:0/535520741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577006cef0 0x7f57700ccbc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5768006fd0 tx=0x7f576801e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b86ac5b3f388b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:50.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.070+0000 7f576e7fc640 1 -- 172.21.15.105:0/535520741 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5768004230 con 0x7f577006cef0 2023-11-28T18:24:50.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.070+0000 7f576e7fc640 1 -- 172.21.15.105:0/535520741 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57680ab070 con 0x7f577006cef0 2023-11-28T18:24:50.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.070+0000 7f576e7fc640 1 -- 172.21.15.105:0/535520741 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57680a77f0 con 0x7f577006cef0 2023-11-28T18:24:50.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.070+0000 7f57756dd640 1 -- 172.21.15.105:0/535520741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577006cef0 msgr2=0x7f57700ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:50.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.070+0000 7f57756dd640 1 --2- 172.21.15.105:0/535520741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577006cef0 0x7f57700ccbc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5768006fd0 tx=0x7f576801e9a0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.070+0000 7f57756dd640 1 -- 172.21.15.105:0/535520741 shutdown_connections 2023-11-28T18:24:50.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.070+0000 7f57756dd640 1 --2- 172.21.15.105:0/535520741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577006cef0 0x7f57700ccbc0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.070+0000 7f57756dd640 1 --2- 172.21.15.105:0/535520741 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5770159610 0x7f57700ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.070+0000 7f57756dd640 1 -- 172.21.15.105:0/535520741 >> 172.21.15.105:0/535520741 conn(0x7f57700ceb90 msgr2=0x7f57700cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:50.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.074+0000 7f57756dd640 1 -- 172.21.15.105:0/535520741 shutdown_connections 2023-11-28T18:24:50.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.074+0000 7f57756dd640 1 -- 172.21.15.105:0/535520741 wait complete. 2023-11-28T18:24:50.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.074+0000 7f57756dd640 1 Processor -- start 2023-11-28T18:24:50.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.074+0000 7f57756dd640 1 -- start start 2023-11-28T18:24:50.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.074+0000 7f57756dd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f577006cef0 0x7f577015bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:50.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.074+0000 7f57756dd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57700cc760 0x7f577013cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:50.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.074+0000 7f57756dd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5770159610 0x7f57700701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:50.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.074+0000 7f57756dd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5770150070 con 0x7f57700cc760 2023-11-28T18:24:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.074+0000 7f57756dd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577015c520 con 0x7f5770159610 2023-11-28T18:24:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.074+0000 7f57756dd640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577014f180 con 0x7f577006cef0 2023-11-28T18:24:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.074+0000 7f57667fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57700cc760 0x7f577013cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.074+0000 7f57667fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57700cc760 0x7f577013cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59178/0 (socket says 172.21.15.105:59178) 2023-11-28T18:24:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.074+0000 7f57667fc640 1 -- 172.21.15.105:0/3150258511 learned_addr learned my addr 172.21.15.105:0/3150258511 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.074+0000 7f576effd640 1 --2- 172.21.15.105:0/3150258511 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f577006cef0 0x7f577015bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.074+0000 7f576f7fe640 1 --2- 172.21.15.105:0/3150258511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5770159610 0x7f57700701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.074+0000 7f57667fc640 1 -- 172.21.15.105:0/3150258511 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f577006cef0 msgr2=0x7f577015bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.074+0000 7f57667fc640 1 --2- 172.21.15.105:0/3150258511 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f577006cef0 0x7f577015bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.074+0000 7f57667fc640 1 -- 172.21.15.105:0/3150258511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5770159610 msgr2=0x7f57700701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.074+0000 7f57667fc640 1 --2- 172.21.15.105:0/3150258511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5770159610 0x7f57700701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.074+0000 7f57667fc640 1 -- 172.21.15.105:0/3150258511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576800b780 con 0x7f57700cc760 2023-11-28T18:24:50.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.074+0000 7f57667fc640 1 --2- 172.21.15.105:0/3150258511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57700cc760 0x7f577013cda0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f575800c7f0 tx=0x7f575800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:50.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.078+0000 7f5767fff640 1 -- 172.21.15.105:0/3150258511 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5758003b80 con 0x7f57700cc760 2023-11-28T18:24:50.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.078+0000 7f5767fff640 1 -- 172.21.15.105:0/3150258511 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5758003ce0 con 0x7f57700cc760 2023-11-28T18:24:50.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.078+0000 7f57756dd640 1 -- 172.21.15.105:0/3150258511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577014f460 con 0x7f57700cc760 2023-11-28T18:24:50.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.078+0000 7f5767fff640 1 -- 172.21.15.105:0/3150258511 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575800aa90 con 0x7f57700cc760 2023-11-28T18:24:50.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.078+0000 7f57756dd640 1 -- 172.21.15.105:0/3150258511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577014fa20 con 0x7f57700cc760 2023-11-28T18:24:50.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.078+0000 7f57756dd640 1 -- 172.21.15.105:0/3150258511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57701501e0 con 0x7f57700cc760 2023-11-28T18:24:50.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.078+0000 7f5767fff640 1 -- 172.21.15.105:0/3150258511 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f575800abf0 con 0x7f57700cc760 2023-11-28T18:24:50.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.078+0000 7f5767fff640 1 --2- 172.21.15.105:0/3150258511 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5750066900 0x7f5750068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:50.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.078+0000 7f5767fff640 1 -- 172.21.15.105:0/3150258511 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f5758083fe0 con 0x7f57700cc760 2023-11-28T18:24:50.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.078+0000 7f576effd640 1 --2- 172.21.15.105:0/3150258511 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5750066900 0x7f5750068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:50.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.082+0000 7f576effd640 1 --2- 172.21.15.105:0/3150258511 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5750066900 0x7f5750068dc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5760009ac0 tx=0x7f5760009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:50.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.082+0000 7f5767fff640 1 -- 172.21.15.105:0/3150258511 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5758051480 con 0x7f57700cc760 2023-11-28T18:24:50.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.290+0000 7f57756dd640 1 -- 172.21.15.105:0/3150258511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f5770067570 con 0x7f57700cc760 2023-11-28T18:24:50.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.290+0000 7f5767fff640 1 -- 172.21.15.105:0/3150258511 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5758054aa0 con 0x7f57700cc760 2023-11-28T18:24:50.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.294+0000 7f57756dd640 1 -- 172.21.15.105:0/3150258511 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5750066900 msgr2=0x7f5750068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:50.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.294+0000 7f57756dd640 1 --2- 172.21.15.105:0/3150258511 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5750066900 0x7f5750068dc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5760009ac0 tx=0x7f5760009190 comp rx=0 tx=0).stop 2023-11-28T18:24:50.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.294+0000 7f57756dd640 1 -- 172.21.15.105:0/3150258511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57700cc760 msgr2=0x7f577013cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:50.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.294+0000 7f57756dd640 1 --2- 172.21.15.105:0/3150258511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57700cc760 0x7f577013cda0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f575800c7f0 tx=0x7f575800ccb0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.294+0000 7f57756dd640 1 -- 172.21.15.105:0/3150258511 shutdown_connections 2023-11-28T18:24:50.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.294+0000 7f57756dd640 1 --2- 172.21.15.105:0/3150258511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5770159610 0x7f57700701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.294+0000 7f57756dd640 1 --2- 172.21.15.105:0/3150258511 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5750066900 0x7f5750068dc0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.294+0000 7f57756dd640 1 --2- 172.21.15.105:0/3150258511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57700cc760 0x7f577013cda0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.294+0000 7f57756dd640 1 --2- 172.21.15.105:0/3150258511 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f577006cef0 0x7f577015bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.294+0000 7f57756dd640 1 -- 172.21.15.105:0/3150258511 >> 172.21.15.105:0/3150258511 conn(0x7f57700ceb90 msgr2=0x7f57700ac440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:50.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.294+0000 7f57756dd640 1 -- 172.21.15.105:0/3150258511 shutdown_connections 2023-11-28T18:24:50.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.294+0000 7f57756dd640 1 -- 172.21.15.105:0/3150258511 wait complete. 2023-11-28T18:24:50.319 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-28T18:24:50.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.454+0000 7f459b017640 1 Processor -- start 2023-11-28T18:24:50.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.454+0000 7f459b017640 1 -- start start 2023-11-28T18:24:50.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.454+0000 7f459b017640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f459415cef0 0x7f45940d1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:50.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.454+0000 7f459b017640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f45940d1b40 0x7f45940cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:50.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.454+0000 7f459b017640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45940ce840 con 0x7f459415d2d0 2023-11-28T18:24:50.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.454+0000 7f459b017640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45940b4780 con 0x7f45940d1b40 2023-11-28T18:24:50.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.454+0000 7f459b017640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45940cd990 con 0x7f459415cef0 2023-11-28T18:24:50.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.454+0000 7f4599814640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f45940d1b40 0x7f45940cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:50.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.454+0000 7f4599814640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f45940d1b40 0x7f45940cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59180/0 (socket says 172.21.15.105:59180) 2023-11-28T18:24:50.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.454+0000 7f4599814640 1 -- 172.21.15.105:0/2926431733 learned_addr learned my addr 172.21.15.105:0/2926431733 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:50.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.454+0000 7f459a015640 1 --2- 172.21.15.105:0/2926431733 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f459415cef0 0x7f45940d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:50.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.454+0000 7f4599814640 1 -- 172.21.15.105:0/2926431733 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f459415cef0 msgr2=0x7f45940d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:50.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.454+0000 7f4599814640 1 --2- 172.21.15.105:0/2926431733 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f459415cef0 0x7f45940d1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.454+0000 7f4599814640 1 -- 172.21.15.105:0/2926431733 >> v1:172.21.15.105:6789/0 conn(0x7f459415d2d0 legacy=0x7f4594160c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:50.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.454+0000 7f4599013640 1 -- 172.21.15.105:0/2926431733 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1883112457 0 0) 0x7f45940ce840 con 0x7f459415d2d0 2023-11-28T18:24:50.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.454+0000 7f4599814640 1 -- 172.21.15.105:0/2926431733 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45940cecd0 con 0x7f45940d1b40 2023-11-28T18:24:50.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.454+0000 7f459a015640 1 --2- 172.21.15.105:0/2926431733 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f459415cef0 0x7f45940d1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:24:50.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.458+0000 7f4599814640 1 --2- 172.21.15.105:0/2926431733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45940d1b40 0x7f45940cf420 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4584009870 tx=0x7f458401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00962861024c4ad server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.458+0000 7f4599013640 1 -- 172.21.15.105:0/2926431733 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45840ad640 con 0x7f45940d1b40 2023-11-28T18:24:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.458+0000 7f4599013640 1 -- 172.21.15.105:0/2926431733 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45840ad7a0 con 0x7f45940d1b40 2023-11-28T18:24:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.458+0000 7f4599013640 1 -- 172.21.15.105:0/2926431733 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45840adb30 con 0x7f45940d1b40 2023-11-28T18:24:50.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.458+0000 7f459b017640 1 -- 172.21.15.105:0/2926431733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45940d1b40 msgr2=0x7f45940cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:50.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.458+0000 7f459b017640 1 --2- 172.21.15.105:0/2926431733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45940d1b40 0x7f45940cf420 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4584009870 tx=0x7f458401c480 comp rx=0 tx=0).stop 2023-11-28T18:24:50.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.458+0000 7f459b017640 1 -- 172.21.15.105:0/2926431733 shutdown_connections 2023-11-28T18:24:50.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.458+0000 7f459b017640 1 --2- 172.21.15.105:0/2926431733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45940d1b40 0x7f45940cf420 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.458+0000 7f459b017640 1 --2- 172.21.15.105:0/2926431733 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f459415cef0 0x7f45940d1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.458+0000 7f459b017640 1 -- 172.21.15.105:0/2926431733 >> 172.21.15.105:0/2926431733 conn(0x7f45940d2e10 msgr2=0x7f45940d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:50.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.458+0000 7f459b017640 1 -- 172.21.15.105:0/2926431733 shutdown_connections 2023-11-28T18:24:50.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.458+0000 7f459b017640 1 -- 172.21.15.105:0/2926431733 wait complete. 2023-11-28T18:24:50.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f459b017640 1 Processor -- start 2023-11-28T18:24:50.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f459b017640 1 -- start start 2023-11-28T18:24:50.479 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f459b017640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45940d1b40 0x7f4594157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:50.479 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f459b017640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f459415cef0 0x7f4594154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:50.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f459b017640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f459415d2d0 0x7f45941546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:50.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f459b017640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45940cca20 con 0x7f45940d1b40 2023-11-28T18:24:50.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f459b017640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45941566f0 con 0x7f459415d2d0 2023-11-28T18:24:50.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f459b017640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4594155ae0 con 0x7f459415cef0 2023-11-28T18:24:50.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f459a015640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45940d1b40 0x7f4594157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:50.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f459a015640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45940d1b40 0x7f4594157950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59190/0 (socket says 172.21.15.105:59190) 2023-11-28T18:24:50.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f4599814640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f459415cef0 0x7f4594154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:50.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f459a015640 1 -- 172.21.15.105:0/3725127915 learned_addr learned my addr 172.21.15.105:0/3725127915 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:50.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f459a816640 1 --2- 172.21.15.105:0/3725127915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f459415d2d0 0x7f45941546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:50.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f459a015640 1 -- 172.21.15.105:0/3725127915 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f459415cef0 msgr2=0x7f4594154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:50.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f459a015640 1 --2- 172.21.15.105:0/3725127915 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f459415cef0 0x7f4594154170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f459a015640 1 -- 172.21.15.105:0/3725127915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f459415d2d0 msgr2=0x7f45941546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:50.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f459a015640 1 --2- 172.21.15.105:0/3725127915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f459415d2d0 0x7f45941546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f459a015640 1 -- 172.21.15.105:0/3725127915 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4590009420 con 0x7f45940d1b40 2023-11-28T18:24:50.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f4599814640 1 --2- 172.21.15.105:0/3725127915 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f459415cef0 0x7f4594154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:24:50.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f459a015640 1 --2- 172.21.15.105:0/3725127915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45940d1b40 0x7f4594157950 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f459000ebe0 tx=0x7f459000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:50.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f458affd640 1 -- 172.21.15.105:0/3725127915 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459000eee0 con 0x7f45940d1b40 2023-11-28T18:24:50.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f458affd640 1 -- 172.21.15.105:0/3725127915 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459000cc30 con 0x7f45940d1b40 2023-11-28T18:24:50.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f458affd640 1 -- 172.21.15.105:0/3725127915 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4590013470 con 0x7f45940d1b40 2023-11-28T18:24:50.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f459b017640 1 -- 172.21.15.105:0/3725127915 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45840094d0 con 0x7f45940d1b40 2023-11-28T18:24:50.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.462+0000 7f459b017640 1 -- 172.21.15.105:0/3725127915 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45941560a0 con 0x7f45940d1b40 2023-11-28T18:24:50.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.466+0000 7f459b017640 1 -- 172.21.15.105:0/3725127915 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45940d03c0 con 0x7f45940d1b40 2023-11-28T18:24:50.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.466+0000 7f458affd640 1 -- 172.21.15.105:0/3725127915 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f45900135d0 con 0x7f45940d1b40 2023-11-28T18:24:50.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.466+0000 7f458affd640 1 --2- 172.21.15.105:0/3725127915 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4578066900 0x7f4578068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:50.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.466+0000 7f458affd640 1 -- 172.21.15.105:0/3725127915 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4590084410 con 0x7f45940d1b40 2023-11-28T18:24:50.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.466+0000 7f4599814640 1 --2- 172.21.15.105:0/3725127915 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4578066900 0x7f4578068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:50.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.466+0000 7f4599814640 1 --2- 172.21.15.105:0/3725127915 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4578066900 0x7f4578068dc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f45840af000 tx=0x7f4584008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:50.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.470+0000 7f458affd640 1 -- 172.21.15.105:0/3725127915 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4590051900 con 0x7f45940d1b40 2023-11-28T18:24:50.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.674+0000 7f459b017640 1 -- 172.21.15.105:0/3725127915 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f45940b41c0 con 0x7f45940d1b40 2023-11-28T18:24:50.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.678+0000 7f458affd640 1 -- 172.21.15.105:0/3725127915 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4590054f20 con 0x7f45940d1b40 2023-11-28T18:24:50.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.678+0000 7f459b017640 1 -- 172.21.15.105:0/3725127915 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4578066900 msgr2=0x7f4578068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:50.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.678+0000 7f459b017640 1 --2- 172.21.15.105:0/3725127915 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4578066900 0x7f4578068dc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f45840af000 tx=0x7f4584008040 comp rx=0 tx=0).stop 2023-11-28T18:24:50.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.678+0000 7f459b017640 1 -- 172.21.15.105:0/3725127915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45940d1b40 msgr2=0x7f4594157950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:50.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.678+0000 7f459b017640 1 --2- 172.21.15.105:0/3725127915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45940d1b40 0x7f4594157950 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f459000ebe0 tx=0x7f459000c520 comp rx=0 tx=0).stop 2023-11-28T18:24:50.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.682+0000 7f459b017640 1 -- 172.21.15.105:0/3725127915 shutdown_connections 2023-11-28T18:24:50.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.682+0000 7f459b017640 1 --2- 172.21.15.105:0/3725127915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f459415d2d0 0x7f45941546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.682+0000 7f459b017640 1 --2- 172.21.15.105:0/3725127915 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f459415cef0 0x7f4594154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.682+0000 7f459b017640 1 --2- 172.21.15.105:0/3725127915 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4578066900 0x7f4578068dc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.682+0000 7f459b017640 1 --2- 172.21.15.105:0/3725127915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45940d1b40 0x7f4594157950 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.682+0000 7f459b017640 1 -- 172.21.15.105:0/3725127915 >> 172.21.15.105:0/3725127915 conn(0x7f45940d2e10 msgr2=0x7f45940cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:50.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.682+0000 7f459b017640 1 -- 172.21.15.105:0/3725127915 shutdown_connections 2023-11-28T18:24:50.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.682+0000 7f459b017640 1 -- 172.21.15.105:0/3725127915 wait complete. 2023-11-28T18:24:50.704 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-28T18:24:50.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.838+0000 7fb51f90f640 1 Processor -- start 2023-11-28T18:24:50.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.838+0000 7fb51f90f640 1 -- start start 2023-11-28T18:24:50.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.838+0000 7fb51f90f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb51815d4f0 0x7fb5180d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:50.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.838+0000 7fb51f90f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb5180b2d20 0x7fb5180b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:50.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.838+0000 7fb51f90f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5180b25b0 con 0x7fb5180d3c30 2023-11-28T18:24:50.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.838+0000 7fb51f90f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5180b2900 con 0x7fb5180b2d20 2023-11-28T18:24:50.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.838+0000 7fb51f90f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5180d0a20 con 0x7fb51815d4f0 2023-11-28T18:24:50.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.838+0000 7fb51deac640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb5180b2d20 0x7fb5180b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:50.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.838+0000 7fb51deac640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb5180b2d20 0x7fb5180b31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:38888/0 (socket says 172.21.15.105:38888) 2023-11-28T18:24:50.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.838+0000 7fb51deac640 1 -- 172.21.15.105:0/760864817 learned_addr learned my addr 172.21.15.105:0/760864817 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:50.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.838+0000 7fb51d6ab640 1 --2- 172.21.15.105:0/760864817 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb51815d4f0 0x7fb5180d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:50.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.838+0000 7fb51deac640 1 -- 172.21.15.105:0/760864817 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb51815d4f0 msgr2=0x7fb5180d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:50.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.838+0000 7fb51deac640 1 --2- 172.21.15.105:0/760864817 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb51815d4f0 0x7fb5180d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.838+0000 7fb51deac640 1 -- 172.21.15.105:0/760864817 >> v1:172.21.15.148:6789/0 conn(0x7fb5180d3c30 legacy=0x7fb5180d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:50.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.838+0000 7fb51deac640 1 -- 172.21.15.105:0/760864817 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5180cea30 con 0x7fb5180b2d20 2023-11-28T18:24:50.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.838+0000 7fb51d6ab640 1 --2- 172.21.15.105:0/760864817 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb51815d4f0 0x7fb5180d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:24:50.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.842+0000 7fb51deac640 1 --2- 172.21.15.105:0/760864817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5180b2d20 0x7fb5180b31d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb510009e50 tx=0x7fb51001c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6dc25da63053df4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:50.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.842+0000 7fb50ffff640 1 -- 172.21.15.105:0/760864817 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5100a5810 con 0x7fb5180b2d20 2023-11-28T18:24:50.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.842+0000 7fb50ffff640 1 -- 172.21.15.105:0/760864817 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5100a5970 con 0x7fb5180b2d20 2023-11-28T18:24:50.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.842+0000 7fb50ffff640 1 -- 172.21.15.105:0/760864817 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5100a5d00 con 0x7fb5180b2d20 2023-11-28T18:24:50.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.842+0000 7fb51f90f640 1 -- 172.21.15.105:0/760864817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5180b2d20 msgr2=0x7fb5180b31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:50.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.842+0000 7fb51f90f640 1 --2- 172.21.15.105:0/760864817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5180b2d20 0x7fb5180b31d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb510009e50 tx=0x7fb51001c950 comp rx=0 tx=0).stop 2023-11-28T18:24:50.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.842+0000 7fb51f90f640 1 -- 172.21.15.105:0/760864817 shutdown_connections 2023-11-28T18:24:50.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.842+0000 7fb51f90f640 1 --2- 172.21.15.105:0/760864817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5180b2d20 0x7fb5180b31d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.842+0000 7fb51f90f640 1 --2- 172.21.15.105:0/760864817 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb51815d4f0 0x7fb5180d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.842+0000 7fb51f90f640 1 -- 172.21.15.105:0/760864817 >> 172.21.15.105:0/760864817 conn(0x7fb5180d3030 msgr2=0x7fb5180d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:50.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51f90f640 1 -- 172.21.15.105:0/760864817 shutdown_connections 2023-11-28T18:24:50.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51f90f640 1 -- 172.21.15.105:0/760864817 wait complete. 2023-11-28T18:24:50.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51f90f640 1 Processor -- start 2023-11-28T18:24:50.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51f90f640 1 -- start start 2023-11-28T18:24:50.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51f90f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5180b2d20 0x7fb518155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:50.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51f90f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5180d3c30 0x7fb518159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:50.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51f90f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb51815d4f0 0x7fb51814e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:50.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51f90f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51814eca0 con 0x7fb51815d4f0 2023-11-28T18:24:50.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51f90f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb518156440 con 0x7fb5180b2d20 2023-11-28T18:24:50.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51f90f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5181511f0 con 0x7fb5180d3c30 2023-11-28T18:24:50.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51deac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb51815d4f0 0x7fb51814e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:50.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51ceaa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5180d3c30 0x7fb518159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:50.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51d6ab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5180b2d20 0x7fb518155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:50.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51d6ab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5180b2d20 0x7fb518155f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:38894/0 (socket says 172.21.15.105:38894) 2023-11-28T18:24:50.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51deac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb51815d4f0 0x7fb51814e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59206/0 (socket says 172.21.15.105:59206) 2023-11-28T18:24:50.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51ceaa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5180d3c30 0x7fb518159050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:50248/0 (socket says 172.21.15.105:50248) 2023-11-28T18:24:50.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51d6ab640 1 -- 172.21.15.105:0/3473931656 learned_addr learned my addr 172.21.15.105:0/3473931656 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:50.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51d6ab640 1 -- 172.21.15.105:0/3473931656 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5180d3c30 msgr2=0x7fb518159050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:50.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51d6ab640 1 --2- 172.21.15.105:0/3473931656 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5180d3c30 0x7fb518159050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51d6ab640 1 -- 172.21.15.105:0/3473931656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb51815d4f0 msgr2=0x7fb51814e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:50.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51d6ab640 1 --2- 172.21.15.105:0/3473931656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb51815d4f0 0x7fb51814e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:50.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51d6ab640 1 -- 172.21.15.105:0/3473931656 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb514009420 con 0x7fb5180b2d20 2023-11-28T18:24:50.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51ceaa640 1 --2- 172.21.15.105:0/3473931656 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5180d3c30 0x7fb518159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:24:50.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.846+0000 7fb51deac640 1 --2- 172.21.15.105:0/3473931656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb51815d4f0 0x7fb51814e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:24:50.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.850+0000 7fb51d6ab640 1 --2- 172.21.15.105:0/3473931656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5180b2d20 0x7fb518155f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb51400e980 tx=0x7fb51400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:50.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.850+0000 7fb50dffb640 1 -- 172.21.15.105:0/3473931656 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51400ca00 con 0x7fb5180b2d20 2023-11-28T18:24:50.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.850+0000 7fb50dffb640 1 -- 172.21.15.105:0/3473931656 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51400cb60 con 0x7fb5180b2d20 2023-11-28T18:24:50.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.850+0000 7fb50dffb640 1 -- 172.21.15.105:0/3473931656 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb514013470 con 0x7fb5180b2d20 2023-11-28T18:24:50.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.850+0000 7fb51f90f640 1 -- 172.21.15.105:0/3473931656 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb510009b30 con 0x7fb5180b2d20 2023-11-28T18:24:50.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.850+0000 7fb51f90f640 1 -- 172.21.15.105:0/3473931656 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5181517b0 con 0x7fb5180b2d20 2023-11-28T18:24:50.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.850+0000 7fb51f90f640 1 -- 172.21.15.105:0/3473931656 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb518067570 con 0x7fb5180b2d20 2023-11-28T18:24:50.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.850+0000 7fb50dffb640 1 -- 172.21.15.105:0/3473931656 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fb5140135d0 con 0x7fb5180b2d20 2023-11-28T18:24:50.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.850+0000 7fb50dffb640 1 --2- 172.21.15.105:0/3473931656 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb4fc0668d0 0x7fb4fc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:50.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.850+0000 7fb50dffb640 1 -- 172.21.15.105:0/3473931656 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fb514084420 con 0x7fb5180b2d20 2023-11-28T18:24:50.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.850+0000 7fb51ceaa640 1 --2- 172.21.15.105:0/3473931656 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb4fc0668d0 0x7fb4fc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:50.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.854+0000 7fb51ceaa640 1 --2- 172.21.15.105:0/3473931656 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb4fc0668d0 0x7fb4fc068d90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb5181f9220 tx=0x7fb508009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:50.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:50.858+0000 7fb50dffb640 1 -- 172.21.15.105:0/3473931656 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb514051880 con 0x7fb5180b2d20 2023-11-28T18:24:51.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.058+0000 7fb51f90f640 1 -- 172.21.15.105:0/3473931656 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fb5180d0630 con 0x7fb5180b2d20 2023-11-28T18:24:51.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.066+0000 7fb50dffb640 1 -- 172.21.15.105:0/3473931656 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb514054ea0 con 0x7fb5180b2d20 2023-11-28T18:24:51.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.070+0000 7fb51f90f640 1 -- 172.21.15.105:0/3473931656 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb4fc0668d0 msgr2=0x7fb4fc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:51.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.070+0000 7fb51f90f640 1 --2- 172.21.15.105:0/3473931656 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb4fc0668d0 0x7fb4fc068d90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb5181f9220 tx=0x7fb508009040 comp rx=0 tx=0).stop 2023-11-28T18:24:51.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.070+0000 7fb51f90f640 1 -- 172.21.15.105:0/3473931656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5180b2d20 msgr2=0x7fb518155f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:51.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.070+0000 7fb51f90f640 1 --2- 172.21.15.105:0/3473931656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5180b2d20 0x7fb518155f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb51400e980 tx=0x7fb51400ee40 comp rx=0 tx=0).stop 2023-11-28T18:24:51.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.070+0000 7fb51f90f640 1 -- 172.21.15.105:0/3473931656 shutdown_connections 2023-11-28T18:24:51.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.070+0000 7fb51f90f640 1 --2- 172.21.15.105:0/3473931656 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb4fc0668d0 0x7fb4fc068d90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:51.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.070+0000 7fb51f90f640 1 --2- 172.21.15.105:0/3473931656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb51815d4f0 0x7fb51814e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:51.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.070+0000 7fb51f90f640 1 --2- 172.21.15.105:0/3473931656 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5180d3c30 0x7fb518159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:51.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.070+0000 7fb51f90f640 1 --2- 172.21.15.105:0/3473931656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5180b2d20 0x7fb518155f00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:51.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.070+0000 7fb51f90f640 1 -- 172.21.15.105:0/3473931656 >> 172.21.15.105:0/3473931656 conn(0x7fb5180d3030 msgr2=0x7fb5180b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:51.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.070+0000 7fb51f90f640 1 -- 172.21.15.105:0/3473931656 shutdown_connections 2023-11-28T18:24:51.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.070+0000 7fb51f90f640 1 -- 172.21.15.105:0/3473931656 wait complete. 2023-11-28T18:24:51.093 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-28T18:24:51.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.226+0000 7f2c6fc17640 1 Processor -- start 2023-11-28T18:24:51.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.226+0000 7f2c6fc17640 1 -- start start 2023-11-28T18:24:51.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.226+0000 7f2c6fc17640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2c6815d4f0 0x7f2c680d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:51.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.226+0000 7f2c6fc17640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c680b2d20 0x7f2c680b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:51.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.226+0000 7f2c6fc17640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c680b25b0 con 0x7f2c680d3c30 2023-11-28T18:24:51.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.226+0000 7f2c6fc17640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c680b2900 con 0x7f2c6815d4f0 2023-11-28T18:24:51.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.226+0000 7f2c6fc17640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c680d0a20 con 0x7f2c680b2d20 2023-11-28T18:24:51.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.226+0000 7f2c6e1b4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c680b2d20 0x7f2c680b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.226+0000 7f2c6d9b3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2c6815d4f0 0x7f2c680d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.226+0000 7f2c6e1b4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c680b2d20 0x7f2c680b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:60734/0 (socket says 172.21.15.105:60734) 2023-11-28T18:24:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.226+0000 7f2c6d9b3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2c6815d4f0 0x7f2c680d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:38980/0 (socket says 172.21.15.105:38980) 2023-11-28T18:24:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.226+0000 7f2c6e1b4640 1 -- 172.21.15.105:0/123370581 learned_addr learned my addr 172.21.15.105:0/123370581 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.226+0000 7f2c6e1b4640 1 -- 172.21.15.105:0/123370581 >> v2:172.21.15.148:3300/0 conn(0x7f2c6815d4f0 msgr2=0x7f2c680d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.226+0000 7f2c6e1b4640 1 --2- 172.21.15.105:0/123370581 >> v2:172.21.15.148:3300/0 conn(0x7f2c6815d4f0 0x7f2c680d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.226+0000 7f2c6e1b4640 1 -- 172.21.15.105:0/123370581 >> v1:172.21.15.148:6789/0 conn(0x7f2c680d3c30 legacy=0x7f2c680d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.226+0000 7f2c6e1b4640 1 -- 172.21.15.105:0/123370581 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c680cea30 con 0x7f2c680b2d20 2023-11-28T18:24:51.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.230+0000 7f2c6e1b4640 1 --2- 172.21.15.105:0/123370581 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c680b2d20 0x7f2c680b3190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009e50 tx=0x7f2c6001c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60c1b2a6166a417a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:51.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.230+0000 7f2c6c9b1640 1 -- 172.21.15.105:0/123370581 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c600a5810 con 0x7f2c680b2d20 2023-11-28T18:24:51.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.230+0000 7f2c6c9b1640 1 -- 172.21.15.105:0/123370581 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c600a5970 con 0x7f2c680b2d20 2023-11-28T18:24:51.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.230+0000 7f2c6c9b1640 1 -- 172.21.15.105:0/123370581 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c600a5d00 con 0x7f2c680b2d20 2023-11-28T18:24:51.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.230+0000 7f2c6fc17640 1 -- 172.21.15.105:0/123370581 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c680b2d20 msgr2=0x7f2c680b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:51.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.230+0000 7f2c6fc17640 1 --2- 172.21.15.105:0/123370581 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c680b2d20 0x7f2c680b3190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009e50 tx=0x7f2c6001c950 comp rx=0 tx=0).stop 2023-11-28T18:24:51.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.230+0000 7f2c6fc17640 1 -- 172.21.15.105:0/123370581 shutdown_connections 2023-11-28T18:24:51.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.230+0000 7f2c6fc17640 1 --2- 172.21.15.105:0/123370581 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c680b2d20 0x7f2c680b3190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:51.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.230+0000 7f2c6fc17640 1 --2- 172.21.15.105:0/123370581 >> v2:172.21.15.148:3300/0 conn(0x7f2c6815d4f0 0x7f2c680d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:51.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.230+0000 7f2c6fc17640 1 -- 172.21.15.105:0/123370581 >> 172.21.15.105:0/123370581 conn(0x7f2c680d3030 msgr2=0x7f2c680d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:51.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.234+0000 7f2c6fc17640 1 -- 172.21.15.105:0/123370581 shutdown_connections 2023-11-28T18:24:51.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.234+0000 7f2c6fc17640 1 -- 172.21.15.105:0/123370581 wait complete. 2023-11-28T18:24:51.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.234+0000 7f2c6fc17640 1 Processor -- start 2023-11-28T18:24:51.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.234+0000 7f2c6fc17640 1 -- start start 2023-11-28T18:24:51.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.234+0000 7f2c6fc17640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c680b2d20 0x7f2c681552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:51.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.234+0000 7f2c6fc17640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c680d3c30 0x7f2c68159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.234+0000 7f2c6fc17640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c6815d4f0 0x7f2c68153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.234+0000 7f2c6fc17640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c68154f60 con 0x7f2c680b2d20 2023-11-28T18:24:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.234+0000 7f2c6fc17640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c68153b90 con 0x7f2c6815d4f0 2023-11-28T18:24:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.234+0000 7f2c6fc17640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c680ccd20 con 0x7f2c680d3c30 2023-11-28T18:24:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.234+0000 7f2c6d9b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c680b2d20 0x7f2c681552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.234+0000 7f2c6d9b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c680b2d20 0x7f2c681552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36670/0 (socket says 172.21.15.105:36670) 2023-11-28T18:24:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.234+0000 7f2c6d9b3640 1 -- 172.21.15.105:0/58881714 learned_addr learned my addr 172.21.15.105:0/58881714 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.234+0000 7f2c6d1b2640 1 --2- 172.21.15.105:0/58881714 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c680d3c30 0x7f2c68159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.234+0000 7f2c6e1b4640 1 --2- 172.21.15.105:0/58881714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c6815d4f0 0x7f2c68153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.234+0000 7f2c6d9b3640 1 -- 172.21.15.105:0/58881714 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c680d3c30 msgr2=0x7f2c68159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.234+0000 7f2c6d9b3640 1 --2- 172.21.15.105:0/58881714 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c680d3c30 0x7f2c68159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.234+0000 7f2c6d9b3640 1 -- 172.21.15.105:0/58881714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c6815d4f0 msgr2=0x7f2c68153650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.234+0000 7f2c6d9b3640 1 --2- 172.21.15.105:0/58881714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c6815d4f0 0x7f2c68153650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.234+0000 7f2c6d9b3640 1 -- 172.21.15.105:0/58881714 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c60009b30 con 0x7f2c680b2d20 2023-11-28T18:24:51.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.234+0000 7f2c6d9b3640 1 --2- 172.21.15.105:0/58881714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c680b2d20 0x7f2c681552a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400e960 tx=0x7f2c6400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:51.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.238+0000 7f2c5e7fc640 1 -- 172.21.15.105:0/58881714 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64009690 con 0x7f2c680b2d20 2023-11-28T18:24:51.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.238+0000 7f2c5e7fc640 1 -- 172.21.15.105:0/58881714 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6400ca00 con 0x7f2c680b2d20 2023-11-28T18:24:51.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.238+0000 7f2c6fc17640 1 -- 172.21.15.105:0/58881714 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c68154910 con 0x7f2c680b2d20 2023-11-28T18:24:51.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.238+0000 7f2c5e7fc640 1 -- 172.21.15.105:0/58881714 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6400cd70 con 0x7f2c680b2d20 2023-11-28T18:24:51.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.238+0000 7f2c6fc17640 1 -- 172.21.15.105:0/58881714 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c680cbfa0 con 0x7f2c680b2d20 2023-11-28T18:24:51.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.238+0000 7f2c6fc17640 1 -- 172.21.15.105:0/58881714 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c68154770 con 0x7f2c680b2d20 2023-11-28T18:24:51.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.238+0000 7f2c5e7fc640 1 -- 172.21.15.105:0/58881714 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f2c64013470 con 0x7f2c680b2d20 2023-11-28T18:24:51.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.238+0000 7f2c5e7fc640 1 --2- 172.21.15.105:0/58881714 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2c4c0668d0 0x7f2c4c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:51.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.238+0000 7f2c5e7fc640 1 -- 172.21.15.105:0/58881714 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f2c64084400 con 0x7f2c680b2d20 2023-11-28T18:24:51.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.238+0000 7f2c6d1b2640 1 --2- 172.21.15.105:0/58881714 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2c4c0668d0 0x7f2c4c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:51.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.238+0000 7f2c6d1b2640 1 --2- 172.21.15.105:0/58881714 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2c4c0668d0 0x7f2c4c068d90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2c681f9300 tx=0x7f2c58009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:51.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.242+0000 7f2c5e7fc640 1 -- 172.21.15.105:0/58881714 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2c68154770 con 0x7f2c680b2d20 2023-11-28T18:24:51.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.446+0000 7f2c6fc17640 1 -- 172.21.15.105:0/58881714 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f2c68154770 con 0x7f2c680b2d20 2023-11-28T18:24:51.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.450+0000 7f2c5e7fc640 1 -- 172.21.15.105:0/58881714 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2c68154770 con 0x7f2c680b2d20 2023-11-28T18:24:51.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.450+0000 7f2c6fc17640 1 -- 172.21.15.105:0/58881714 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2c4c0668d0 msgr2=0x7f2c4c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:51.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.450+0000 7f2c6fc17640 1 --2- 172.21.15.105:0/58881714 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2c4c0668d0 0x7f2c4c068d90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2c681f9300 tx=0x7f2c58009040 comp rx=0 tx=0).stop 2023-11-28T18:24:51.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.450+0000 7f2c6fc17640 1 -- 172.21.15.105:0/58881714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c680b2d20 msgr2=0x7f2c681552a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:51.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.450+0000 7f2c6fc17640 1 --2- 172.21.15.105:0/58881714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c680b2d20 0x7f2c681552a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400e960 tx=0x7f2c6400ee20 comp rx=0 tx=0).stop 2023-11-28T18:24:51.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.450+0000 7f2c6fc17640 1 -- 172.21.15.105:0/58881714 shutdown_connections 2023-11-28T18:24:51.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.450+0000 7f2c6fc17640 1 --2- 172.21.15.105:0/58881714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c6815d4f0 0x7f2c68153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:51.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.450+0000 7f2c6fc17640 1 --2- 172.21.15.105:0/58881714 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c680d3c30 0x7f2c68159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:51.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.450+0000 7f2c6fc17640 1 --2- 172.21.15.105:0/58881714 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2c4c0668d0 0x7f2c4c068d90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:51.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.450+0000 7f2c6fc17640 1 --2- 172.21.15.105:0/58881714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c680b2d20 0x7f2c681552a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:51.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.450+0000 7f2c6fc17640 1 -- 172.21.15.105:0/58881714 >> 172.21.15.105:0/58881714 conn(0x7f2c680d3030 msgr2=0x7f2c680b0640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:51.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.454+0000 7f2c6fc17640 1 -- 172.21.15.105:0/58881714 shutdown_connections 2023-11-28T18:24:51.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.454+0000 7f2c6fc17640 1 -- 172.21.15.105:0/58881714 wait complete. 2023-11-28T18:24:51.476 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-28T18:24:51.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.610+0000 7f4da04aa640 1 Processor -- start 2023-11-28T18:24:51.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.610+0000 7f4da04aa640 1 -- start start 2023-11-28T18:24:51.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.614+0000 7f4da04aa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4d980d3c30 0x7f4d980d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:51.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.614+0000 7f4da04aa640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d980b25b0 con 0x7f4d9815d4f0 2023-11-28T18:24:51.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.614+0000 7f4da04aa640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d980cea30 con 0x7f4d980b2d20 2023-11-28T18:24:51.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.614+0000 7f4da04aa640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d980b2900 con 0x7f4d980d3c30 2023-11-28T18:24:51.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.614+0000 7f4d9da45640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4d980d3c30 0x7f4d980d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:51.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.614+0000 7f4d9e246640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4d9815d4f0 0x7f4d980d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54136/0 (socket says 172.21.15.105:54136) 2023-11-28T18:24:51.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.614+0000 7f4d9e246640 1 -- 172.21.15.105:0/3943637420 learned_addr learned my addr 172.21.15.105:0/3943637420 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:51.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.614+0000 7f4d9da45640 1 -- 172.21.15.105:0/3943637420 >> v1:172.21.15.148:6789/0 conn(0x7f4d980b2d20 legacy=0x7f4d980b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:51.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.614+0000 7f4d9da45640 1 -- 172.21.15.105:0/3943637420 >> v1:172.21.15.105:6789/0 conn(0x7f4d9815d4f0 legacy=0x7f4d980d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:51.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.614+0000 7f4d9da45640 1 -- 172.21.15.105:0/3943637420 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d980ced10 con 0x7f4d980d3c30 2023-11-28T18:24:51.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.614+0000 7f4d9d244640 1 -- 172.21.15.105:0/3943637420 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 819870963 0 0) 0x7f4d980b25b0 con 0x7f4d9815d4f0 2023-11-28T18:24:51.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.614+0000 7f4d9da45640 1 --2- 172.21.15.105:0/3943637420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d980d3c30 0x7f4d980d1520 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4d880097c0 tx=0x7f4d8801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f060db890c603f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:51.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4d9d244640 1 -- 172.21.15.105:0/3943637420 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d880ab600 con 0x7f4d980d3c30 2023-11-28T18:24:51.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4d9d244640 1 -- 172.21.15.105:0/3943637420 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d880a7050 con 0x7f4d980d3c30 2023-11-28T18:24:51.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4d9d244640 1 -- 172.21.15.105:0/3943637420 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d880ab980 con 0x7f4d980d3c30 2023-11-28T18:24:51.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4da04aa640 1 -- 172.21.15.105:0/3943637420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d980d3c30 msgr2=0x7f4d980d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:51.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4da04aa640 1 --2- 172.21.15.105:0/3943637420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d980d3c30 0x7f4d980d1520 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4d880097c0 tx=0x7f4d8801c380 comp rx=0 tx=0).stop 2023-11-28T18:24:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4da04aa640 1 -- 172.21.15.105:0/3943637420 shutdown_connections 2023-11-28T18:24:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4da04aa640 1 --2- 172.21.15.105:0/3943637420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d980d3c30 0x7f4d980d1520 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4da04aa640 1 -- 172.21.15.105:0/3943637420 >> 172.21.15.105:0/3943637420 conn(0x7f4d980d3030 msgr2=0x7f4d980d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4da04aa640 1 -- 172.21.15.105:0/3943637420 shutdown_connections 2023-11-28T18:24:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4da04aa640 1 -- 172.21.15.105:0/3943637420 wait complete. 2023-11-28T18:24:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4da04aa640 1 Processor -- start 2023-11-28T18:24:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4da04aa640 1 -- start start 2023-11-28T18:24:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4da04aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d980b2d20 0x7f4d98151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4da04aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d980d3c30 0x7f4d980c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:51.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4da04aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d9815d4f0 0x7f4d9806b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:51.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4da04aa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d980d0a20 con 0x7f4d980b2d20 2023-11-28T18:24:51.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4da04aa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9806ce10 con 0x7f4d980d3c30 2023-11-28T18:24:51.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4da04aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d981504c0 con 0x7f4d9815d4f0 2023-11-28T18:24:51.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4d9e246640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d980b2d20 0x7f4d98151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:51.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4d9e246640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d980b2d20 0x7f4d98151280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36684/0 (socket says 172.21.15.105:36684) 2023-11-28T18:24:51.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4d9e246640 1 -- 172.21.15.105:0/3173090813 learned_addr learned my addr 172.21.15.105:0/3173090813 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:51.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4d9ea47640 1 --2- 172.21.15.105:0/3173090813 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d9815d4f0 0x7f4d9806b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:51.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.618+0000 7f4d9da45640 1 --2- 172.21.15.105:0/3173090813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d980d3c30 0x7f4d980c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:51.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.622+0000 7f4d9e246640 1 -- 172.21.15.105:0/3173090813 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d9815d4f0 msgr2=0x7f4d9806b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:51.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.622+0000 7f4d9e246640 1 --2- 172.21.15.105:0/3173090813 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d9815d4f0 0x7f4d9806b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:51.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.622+0000 7f4d9e246640 1 -- 172.21.15.105:0/3173090813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d980d3c30 msgr2=0x7f4d980c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:51.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.622+0000 7f4d9e246640 1 --2- 172.21.15.105:0/3173090813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d980d3c30 0x7f4d980c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:51.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.622+0000 7f4d9e246640 1 -- 172.21.15.105:0/3173090813 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d88009420 con 0x7f4d980b2d20 2023-11-28T18:24:51.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.622+0000 7f4d9e246640 1 --2- 172.21.15.105:0/3173090813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d980b2d20 0x7f4d98151280 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400c480 tx=0x7f4d9400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:51.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.622+0000 7f4d8effd640 1 -- 172.21.15.105:0/3173090813 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d940074e0 con 0x7f4d980b2d20 2023-11-28T18:24:51.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.622+0000 7f4d8effd640 1 -- 172.21.15.105:0/3173090813 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d94007660 con 0x7f4d980b2d20 2023-11-28T18:24:51.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.622+0000 7f4d8effd640 1 -- 172.21.15.105:0/3173090813 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d94004a30 con 0x7f4d980b2d20 2023-11-28T18:24:51.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.622+0000 7f4da04aa640 1 -- 172.21.15.105:0/3173090813 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d981507a0 con 0x7f4d980b2d20 2023-11-28T18:24:51.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.622+0000 7f4da04aa640 1 -- 172.21.15.105:0/3173090813 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d98150d60 con 0x7f4d980b2d20 2023-11-28T18:24:51.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.622+0000 7f4da04aa640 1 -- 172.21.15.105:0/3173090813 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d98067570 con 0x7f4d980b2d20 2023-11-28T18:24:51.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.622+0000 7f4d8effd640 1 -- 172.21.15.105:0/3173090813 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f4d94004b90 con 0x7f4d980b2d20 2023-11-28T18:24:51.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.622+0000 7f4d8effd640 1 --2- 172.21.15.105:0/3173090813 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d7c0668d0 0x7f4d7c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:51.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.622+0000 7f4d8effd640 1 -- 172.21.15.105:0/3173090813 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4d940833a0 con 0x7f4d980b2d20 2023-11-28T18:24:51.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.622+0000 7f4d9da45640 1 --2- 172.21.15.105:0/3173090813 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d7c0668d0 0x7f4d7c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.626+0000 7f4d9da45640 1 --2- 172.21.15.105:0/3173090813 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d7c0668d0 0x7f4d7c068d90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4d880ad000 tx=0x7f4d88008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.626+0000 7f4d8effd640 1 -- 172.21.15.105:0/3173090813 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4d94050890 con 0x7f4d980b2d20 2023-11-28T18:24:51.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.830+0000 7f4da04aa640 1 -- 172.21.15.105:0/3173090813 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f4d9806b210 con 0x7f4d980b2d20 2023-11-28T18:24:51.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.834+0000 7f4d8effd640 1 -- 172.21.15.105:0/3173090813 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4d94081e60 con 0x7f4d980b2d20 2023-11-28T18:24:51.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.834+0000 7f4da04aa640 1 -- 172.21.15.105:0/3173090813 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d7c0668d0 msgr2=0x7f4d7c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:51.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.834+0000 7f4da04aa640 1 --2- 172.21.15.105:0/3173090813 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d7c0668d0 0x7f4d7c068d90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4d880ad000 tx=0x7f4d88008040 comp rx=0 tx=0).stop 2023-11-28T18:24:51.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.834+0000 7f4da04aa640 1 -- 172.21.15.105:0/3173090813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d980b2d20 msgr2=0x7f4d98151280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:51.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.834+0000 7f4da04aa640 1 --2- 172.21.15.105:0/3173090813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d980b2d20 0x7f4d98151280 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400c480 tx=0x7f4d9400c940 comp rx=0 tx=0).stop 2023-11-28T18:24:51.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.834+0000 7f4da04aa640 1 -- 172.21.15.105:0/3173090813 shutdown_connections 2023-11-28T18:24:51.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.834+0000 7f4da04aa640 1 --2- 172.21.15.105:0/3173090813 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d9815d4f0 0x7f4d9806b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:51.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.834+0000 7f4da04aa640 1 --2- 172.21.15.105:0/3173090813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d980d3c30 0x7f4d980c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:51.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.834+0000 7f4da04aa640 1 --2- 172.21.15.105:0/3173090813 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d7c0668d0 0x7f4d7c068d90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:51.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.834+0000 7f4da04aa640 1 --2- 172.21.15.105:0/3173090813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d980b2d20 0x7f4d98151280 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:51.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.834+0000 7f4da04aa640 1 -- 172.21.15.105:0/3173090813 >> 172.21.15.105:0/3173090813 conn(0x7f4d980d3030 msgr2=0x7f4d980d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:51.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.834+0000 7f4da04aa640 1 -- 172.21.15.105:0/3173090813 shutdown_connections 2023-11-28T18:24:51.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.834+0000 7f4da04aa640 1 -- 172.21.15.105:0/3173090813 wait complete. 2023-11-28T18:24:51.859 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-28T18:24:51.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.990+0000 7f787285f640 1 Processor -- start 2023-11-28T18:24:51.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.990+0000 7f787285f640 1 -- start start 2023-11-28T18:24:51.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f787285f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c0caf60 0x7f786c0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:51.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f787285f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786c06bf50 con 0x7f786c0ad850 2023-11-28T18:24:51.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f787285f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786c15a320 con 0x7f786c159f40 2023-11-28T18:24:51.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f787285f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0cb3c0 con 0x7f786c0caf60 2023-11-28T18:24:52.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f7870dfc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f786c0ad850 0x7f786c0701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54140/0 (socket says 172.21.15.105:54140) 2023-11-28T18:24:52.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f7870dfc640 1 -- 172.21.15.105:0/1507638136 learned_addr learned my addr 172.21.15.105:0/1507638136 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:52.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786b7fe640 1 --2- 172.21.15.105:0/1507638136 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c0caf60 0x7f786c0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:52.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786affd640 1 -- 172.21.15.105:0/1507638136 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1222302893 0 0) 0x7f786c06bf50 con 0x7f786c0ad850 2023-11-28T18:24:52.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786affd640 1 -- 172.21.15.105:0/1507638136 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7854003530 con 0x7f786c0ad850 2023-11-28T18:24:52.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786affd640 1 -- 172.21.15.105:0/1507638136 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626107009 0 0) 0x7f786c15a320 con 0x7f786c159f40 2023-11-28T18:24:52.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786affd640 1 -- 172.21.15.105:0/1507638136 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f786c06bf50 con 0x7f786c159f40 2023-11-28T18:24:52.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786affd640 1 -- 172.21.15.105:0/1507638136 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2788573223 0 0) 0x7f7854003530 con 0x7f786c0ad850 2023-11-28T18:24:52.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786affd640 1 -- 172.21.15.105:0/1507638136 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c0caf60 msgr2=0x7f786c0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:52.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786affd640 1 --2- 172.21.15.105:0/1507638136 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c0caf60 0x7f786c0cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786affd640 1 -- 172.21.15.105:0/1507638136 >> v1:172.21.15.148:6789/0 conn(0x7f786c159f40 legacy=0x7f786c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:52.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786affd640 1 -- 172.21.15.105:0/1507638136 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c071380 con 0x7f786c0ad850 2023-11-28T18:24:52.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786affd640 1 -- 172.21.15.105:0/1507638136 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f7864001a00 con 0x7f786c0ad850 2023-11-28T18:24:52.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786b7fe640 1 --2- 172.21.15.105:0/1507638136 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c0caf60 0x7f786c0cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:24:52.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786affd640 1 -- 172.21.15.105:0/1507638136 >> v1:172.21.15.105:6789/0 conn(0x7f786c0ad850 legacy=0x7f786c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:52.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786affd640 1 --2- 172.21.15.105:0/1507638136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7854003b40 0x7f7854003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:52.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786affd640 1 --2- 172.21.15.105:0/1507638136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78540047d0 0x7f7854004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:52.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786affd640 1 --2- 172.21.15.105:0/1507638136 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c0caf60 0x7f78540051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:52.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786affd640 1 -- 172.21.15.105:0/1507638136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0cb3c0 con 0x7f78540047d0 2023-11-28T18:24:52.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786affd640 1 -- 172.21.15.105:0/1507638136 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785400baa0 con 0x7f7854003b40 2023-11-28T18:24:52.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786affd640 1 -- 172.21.15.105:0/1507638136 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785400bc60 con 0x7f786c0caf60 2023-11-28T18:24:52.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786b7fe640 1 --2- 172.21.15.105:0/1507638136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78540047d0 0x7f7854004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:52.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786bfff640 1 --2- 172.21.15.105:0/1507638136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7854003b40 0x7f7854003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:52.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f7870dfc640 1 --2- 172.21.15.105:0/1507638136 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c0caf60 0x7f78540051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786b7fe640 1 -- 172.21.15.105:0/1507638136 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c0caf60 msgr2=0x7f78540051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786b7fe640 1 --2- 172.21.15.105:0/1507638136 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c0caf60 0x7f78540051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786b7fe640 1 -- 172.21.15.105:0/1507638136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7854003b40 msgr2=0x7f7854003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786b7fe640 1 --2- 172.21.15.105:0/1507638136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7854003b40 0x7f7854003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786b7fe640 1 -- 172.21.15.105:0/1507638136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7860009420 con 0x7f78540047d0 2023-11-28T18:24:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f7870dfc640 1 --2- 172.21.15.105:0/1507638136 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c0caf60 0x7f78540051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:52.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.994+0000 7f786bfff640 1 --2- 172.21.15.105:0/1507638136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7854003b40 0x7f7854003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.998+0000 7f786b7fe640 1 --2- 172.21.15.105:0/1507638136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78540047d0 0x7f7854004c50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7860009cb0 tx=0x7f786001cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d1ce63ff439176 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.998+0000 7f786affd640 1 -- 172.21.15.105:0/1507638136 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78600a5860 con 0x7f78540047d0 2023-11-28T18:24:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.998+0000 7f786affd640 1 -- 172.21.15.105:0/1507638136 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78600a59c0 con 0x7f78540047d0 2023-11-28T18:24:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.998+0000 7f786affd640 1 -- 172.21.15.105:0/1507638136 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78600a5d70 con 0x7f78540047d0 2023-11-28T18:24:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.998+0000 7f787285f640 1 -- 172.21.15.105:0/1507638136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78540047d0 msgr2=0x7f7854004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.998+0000 7f787285f640 1 --2- 172.21.15.105:0/1507638136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78540047d0 0x7f7854004c50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7860009cb0 tx=0x7f786001cbc0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.998+0000 7f7870dfc640 1 -- 172.21.15.105:0/1507638136 reap_dead start 2023-11-28T18:24:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.998+0000 7f787285f640 1 -- 172.21.15.105:0/1507638136 shutdown_connections 2023-11-28T18:24:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.998+0000 7f787285f640 1 -- 172.21.15.105:0/1507638136 >> 172.21.15.105:0/1507638136 conn(0x7f786c0ceb90 msgr2=0x7f786c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.998+0000 7f787285f640 1 -- 172.21.15.105:0/1507638136 shutdown_connections 2023-11-28T18:24:52.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.998+0000 7f787285f640 1 -- 172.21.15.105:0/1507638136 wait complete. 2023-11-28T18:24:52.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:51.998+0000 7f787285f640 1 Processor -- start 2023-11-28T18:24:52.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.002+0000 7f787285f640 1 -- start start 2023-11-28T18:24:52.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.002+0000 7f787285f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78540047d0 0x7f786c150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:52.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.002+0000 7f787285f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c09bfc0 0x7f786c09c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:52.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.002+0000 7f787285f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f786c0c9d60 0x7f786c0ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:52.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.002+0000 7f787285f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c15bd00 con 0x7f786c0c9d60 2023-11-28T18:24:52.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.002+0000 7f787285f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0b19b0 con 0x7f78540047d0 2023-11-28T18:24:52.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.002+0000 7f787285f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c15b110 con 0x7f786c09bfc0 2023-11-28T18:24:52.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.002+0000 7f7870dfc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f786c0c9d60 0x7f786c0ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:52.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.002+0000 7f7870dfc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f786c0c9d60 0x7f786c0ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36708/0 (socket says 172.21.15.105:36708) 2023-11-28T18:24:52.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.002+0000 7f7870dfc640 1 -- 172.21.15.105:0/513212852 learned_addr learned my addr 172.21.15.105:0/513212852 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:52.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.002+0000 7f7870dfc640 1 -- 172.21.15.105:0/513212852 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c09bfc0 msgr2=0x7f786c09c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:52.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.002+0000 7f7870dfc640 1 --2- 172.21.15.105:0/513212852 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c09bfc0 0x7f786c09c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.002+0000 7f7870dfc640 1 -- 172.21.15.105:0/513212852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78540047d0 msgr2=0x7f786c150460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:52.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.002+0000 7f7870dfc640 1 --2- 172.21.15.105:0/513212852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78540047d0 0x7f786c150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.002+0000 7f7870dfc640 1 -- 172.21.15.105:0/513212852 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c15b360 con 0x7f786c0c9d60 2023-11-28T18:24:52.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.002+0000 7f7870dfc640 1 --2- 172.21.15.105:0/513212852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f786c0c9d60 0x7f786c0ca1e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f786c0b4e80 tx=0x7f786400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:52.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.002+0000 7f7868ff9640 1 -- 172.21.15.105:0/513212852 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786400eb90 con 0x7f786c0c9d60 2023-11-28T18:24:52.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.002+0000 7f7868ff9640 1 -- 172.21.15.105:0/513212852 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786400ecf0 con 0x7f786c0c9d60 2023-11-28T18:24:52.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.002+0000 7f7868ff9640 1 -- 172.21.15.105:0/513212852 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864014670 con 0x7f786c0c9d60 2023-11-28T18:24:52.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.002+0000 7f787285f640 1 -- 172.21.15.105:0/513212852 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786c15b640 con 0x7f786c0c9d60 2023-11-28T18:24:52.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.002+0000 7f787285f640 1 -- 172.21.15.105:0/513212852 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786c09c980 con 0x7f786c0c9d60 2023-11-28T18:24:52.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.002+0000 7f787285f640 1 -- 172.21.15.105:0/513212852 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786c15be40 con 0x7f786c0c9d60 2023-11-28T18:24:52.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.006+0000 7f7868ff9640 1 -- 172.21.15.105:0/513212852 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f78640147d0 con 0x7f786c0c9d60 2023-11-28T18:24:52.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.006+0000 7f7868ff9640 1 --2- 172.21.15.105:0/513212852 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7858066900 0x7f7858068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:52.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.006+0000 7f7868ff9640 1 -- 172.21.15.105:0/513212852 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f7864083f30 con 0x7f786c0c9d60 2023-11-28T18:24:52.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.006+0000 7f786bfff640 1 --2- 172.21.15.105:0/513212852 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7858066900 0x7f7858068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:52.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.006+0000 7f786bfff640 1 --2- 172.21.15.105:0/513212852 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7858066900 0x7f7858068dc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f785c007af0 tx=0x7f785c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:52.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.010+0000 7f7868ff9640 1 -- 172.21.15.105:0/513212852 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f786c15be40 con 0x7f786c0c9d60 2023-11-28T18:24:52.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.210+0000 7f787285f640 1 -- 172.21.15.105:0/513212852 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f786c15be40 con 0x7f786c0c9d60 2023-11-28T18:24:52.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.214+0000 7f7868ff9640 1 -- 172.21.15.105:0/513212852 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f786c15be40 con 0x7f786c0c9d60 2023-11-28T18:24:52.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.218+0000 7f787285f640 1 -- 172.21.15.105:0/513212852 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7858066900 msgr2=0x7f7858068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:52.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.218+0000 7f787285f640 1 --2- 172.21.15.105:0/513212852 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7858066900 0x7f7858068dc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f785c007af0 tx=0x7f785c009040 comp rx=0 tx=0).stop 2023-11-28T18:24:52.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.218+0000 7f787285f640 1 -- 172.21.15.105:0/513212852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f786c0c9d60 msgr2=0x7f786c0ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:52.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.218+0000 7f787285f640 1 --2- 172.21.15.105:0/513212852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f786c0c9d60 0x7f786c0ca1e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f786c0b4e80 tx=0x7f786400e530 comp rx=0 tx=0).stop 2023-11-28T18:24:52.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.218+0000 7f787285f640 1 -- 172.21.15.105:0/513212852 shutdown_connections 2023-11-28T18:24:52.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.218+0000 7f787285f640 1 --2- 172.21.15.105:0/513212852 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7858066900 0x7f7858068dc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.218+0000 7f787285f640 1 --2- 172.21.15.105:0/513212852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f786c0c9d60 0x7f786c0ca1e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.218+0000 7f787285f640 1 --2- 172.21.15.105:0/513212852 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f786c09bfc0 0x7f786c09c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.218+0000 7f787285f640 1 --2- 172.21.15.105:0/513212852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78540047d0 0x7f786c150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.218+0000 7f787285f640 1 -- 172.21.15.105:0/513212852 >> 172.21.15.105:0/513212852 conn(0x7f786c0ceb90 msgr2=0x7f786c0afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:52.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.218+0000 7f787285f640 1 -- 172.21.15.105:0/513212852 shutdown_connections 2023-11-28T18:24:52.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.218+0000 7f787285f640 1 -- 172.21.15.105:0/513212852 wait complete. 2023-11-28T18:24:52.241 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:24:52.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.374+0000 7fae446d6640 1 Processor -- start 2023-11-28T18:24:52.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.374+0000 7fae446d6640 1 -- start start 2023-11-28T18:24:52.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.374+0000 7fae446d6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fae3c0cc760 0x7fae3c164d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.374+0000 7fae446d6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3c0ccc10 con 0x7fae3c159f40 2023-11-28T18:24:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.374+0000 7fae446d6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3c06bf50 con 0x7fae3c0caf60 2023-11-28T18:24:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.374+0000 7fae446d6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae3c0cb3c0 con 0x7fae3c0cc760 2023-11-28T18:24:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.374+0000 7fae42472640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fae3c159f40 0x7fae3c0ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54156/0 (socket says 172.21.15.105:54156) 2023-11-28T18:24:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.374+0000 7fae42472640 1 -- 172.21.15.105:0/1347827264 learned_addr learned my addr 172.21.15.105:0/1347827264 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:52.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae42c73640 1 --2- 172.21.15.105:0/1347827264 >> v2:172.21.15.148:3300/0 conn(0x7fae3c0cc760 0x7fae3c164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:52.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae41470640 1 -- 172.21.15.105:0/1347827264 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 647020406 0 0) 0x7fae3c0ccc10 con 0x7fae3c159f40 2023-11-28T18:24:52.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae41470640 1 -- 172.21.15.105:0/1347827264 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae28003530 con 0x7fae3c159f40 2023-11-28T18:24:52.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae41470640 1 -- 172.21.15.105:0/1347827264 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 431047588 0 0) 0x7fae28003530 con 0x7fae3c159f40 2023-11-28T18:24:52.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae41470640 1 -- 172.21.15.105:0/1347827264 >> v2:172.21.15.148:3300/0 conn(0x7fae3c0cc760 msgr2=0x7fae3c164d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:52.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae41470640 1 --2- 172.21.15.105:0/1347827264 >> v2:172.21.15.148:3300/0 conn(0x7fae3c0cc760 0x7fae3c164d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae41470640 1 -- 172.21.15.105:0/1347827264 >> v1:172.21.15.148:6789/0 conn(0x7fae3c0caf60 legacy=0x7fae3c0ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:52.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae41470640 1 -- 172.21.15.105:0/1347827264 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c15a320 con 0x7fae3c159f40 2023-11-28T18:24:52.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae41470640 1 -- 172.21.15.105:0/1347827264 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fae38001680 con 0x7fae3c159f40 2023-11-28T18:24:52.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae41470640 1 -- 172.21.15.105:0/1347827264 >> v1:172.21.15.105:6789/0 conn(0x7fae3c159f40 legacy=0x7fae3c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:52.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae41470640 1 --2- 172.21.15.105:0/1347827264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae28003980 0x7fae28003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:52.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae41470640 1 --2- 172.21.15.105:0/1347827264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae28004380 0x7fae28004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:52.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae41470640 1 --2- 172.21.15.105:0/1347827264 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae280050f0 0x7fae2800dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:52.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae41470640 1 -- 172.21.15.105:0/1347827264 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c0cb3c0 con 0x7fae28003980 2023-11-28T18:24:52.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae41470640 1 -- 172.21.15.105:0/1347827264 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2800e240 con 0x7fae28004380 2023-11-28T18:24:52.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae41470640 1 -- 172.21.15.105:0/1347827264 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2800e400 con 0x7fae280050f0 2023-11-28T18:24:52.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae42472640 1 --2- 172.21.15.105:0/1347827264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae28003980 0x7fae28003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:52.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae41c71640 1 --2- 172.21.15.105:0/1347827264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae28004380 0x7fae28004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:52.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae42c73640 1 --2- 172.21.15.105:0/1347827264 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae280050f0 0x7fae2800dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:52.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae42472640 1 -- 172.21.15.105:0/1347827264 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae280050f0 msgr2=0x7fae2800dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:52.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae42472640 1 --2- 172.21.15.105:0/1347827264 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae280050f0 0x7fae2800dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae42472640 1 -- 172.21.15.105:0/1347827264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae28004380 msgr2=0x7fae28004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:52.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae42472640 1 --2- 172.21.15.105:0/1347827264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae28004380 0x7fae28004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae42472640 1 -- 172.21.15.105:0/1347827264 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae28003530 con 0x7fae28003980 2023-11-28T18:24:52.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.378+0000 7fae42c73640 1 -- 172.21.15.105:0/1347827264 reap_dead start 2023-11-28T18:24:52.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.382+0000 7fae42472640 1 --2- 172.21.15.105:0/1347827264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae28003980 0x7fae28003d60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fae3800c210 tx=0x7fae3801cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47e07f6aa853dad server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:52.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.382+0000 7fae41470640 1 -- 172.21.15.105:0/1347827264 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38005ae0 con 0x7fae28003980 2023-11-28T18:24:52.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.382+0000 7fae41470640 1 -- 172.21.15.105:0/1347827264 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae38005c60 con 0x7fae28003980 2023-11-28T18:24:52.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.382+0000 7fae41470640 1 -- 172.21.15.105:0/1347827264 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae380072b0 con 0x7fae28003980 2023-11-28T18:24:52.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.382+0000 7fae446d6640 1 -- 172.21.15.105:0/1347827264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae28003980 msgr2=0x7fae28003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:52.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.382+0000 7fae446d6640 1 --2- 172.21.15.105:0/1347827264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae28003980 0x7fae28003d60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fae3800c210 tx=0x7fae3801cbb0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.382+0000 7fae446d6640 1 -- 172.21.15.105:0/1347827264 shutdown_connections 2023-11-28T18:24:52.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.382+0000 7fae446d6640 1 --2- 172.21.15.105:0/1347827264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae28003980 0x7fae28003d60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.382+0000 7fae446d6640 1 -- 172.21.15.105:0/1347827264 >> 172.21.15.105:0/1347827264 conn(0x7fae3c0ceb90 msgr2=0x7fae3c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:52.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.382+0000 7fae446d6640 1 -- 172.21.15.105:0/1347827264 shutdown_connections 2023-11-28T18:24:52.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.382+0000 7fae446d6640 1 -- 172.21.15.105:0/1347827264 wait complete. 2023-11-28T18:24:52.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.382+0000 7fae446d6640 1 Processor -- start 2023-11-28T18:24:52.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.382+0000 7fae446d6640 1 -- start start 2023-11-28T18:24:52.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.382+0000 7fae446d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae28003980 0x7fae3c1f6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:52.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.382+0000 7fae446d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae3c1526e0 0x7fae3c1567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:52.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.382+0000 7fae446d6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae3c152b60 0x7fae3c156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:52.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.382+0000 7fae446d6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c15b3a0 con 0x7fae3c152b60 2023-11-28T18:24:52.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.382+0000 7fae446d6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c15b1a0 con 0x7fae3c1526e0 2023-11-28T18:24:52.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.382+0000 7fae446d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c153130 con 0x7fae28003980 2023-11-28T18:24:52.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.386+0000 7fae42c73640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae3c152b60 0x7fae3c156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:52.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.386+0000 7fae42472640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae28003980 0x7fae3c1f6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:52.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.386+0000 7fae41c71640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae3c1526e0 0x7fae3c1567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:52.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.386+0000 7fae42c73640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae3c152b60 0x7fae3c156d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36712/0 (socket says 172.21.15.105:36712) 2023-11-28T18:24:52.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.386+0000 7fae42c73640 1 -- 172.21.15.105:0/2839756767 learned_addr learned my addr 172.21.15.105:0/2839756767 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:52.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.386+0000 7fae42c73640 1 -- 172.21.15.105:0/2839756767 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae28003980 msgr2=0x7fae3c1f6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:52.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.386+0000 7fae42c73640 1 --2- 172.21.15.105:0/2839756767 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae28003980 0x7fae3c1f6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.386+0000 7fae42c73640 1 -- 172.21.15.105:0/2839756767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae3c1526e0 msgr2=0x7fae3c1567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:52.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.386+0000 7fae42c73640 1 --2- 172.21.15.105:0/2839756767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae3c1526e0 0x7fae3c1567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.386+0000 7fae42c73640 1 -- 172.21.15.105:0/2839756767 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c153410 con 0x7fae3c152b60 2023-11-28T18:24:52.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.386+0000 7fae42472640 1 --2- 172.21.15.105:0/2839756767 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae28003980 0x7fae3c1f6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:24:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.386+0000 7fae42c73640 1 --2- 172.21.15.105:0/2839756767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae3c152b60 0x7fae3c156d30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0b4e80 tx=0x7fae3400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.386+0000 7fae32ffd640 1 -- 172.21.15.105:0/2839756767 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3400e040 con 0x7fae3c152b60 2023-11-28T18:24:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.386+0000 7fae32ffd640 1 -- 172.21.15.105:0/2839756767 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3400f040 con 0x7fae3c152b60 2023-11-28T18:24:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.386+0000 7fae446d6640 1 -- 172.21.15.105:0/2839756767 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3c1536f0 con 0x7fae3c152b60 2023-11-28T18:24:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.386+0000 7fae32ffd640 1 -- 172.21.15.105:0/2839756767 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34015430 con 0x7fae3c152b60 2023-11-28T18:24:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.386+0000 7fae446d6640 1 -- 172.21.15.105:0/2839756767 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3c1f6a60 con 0x7fae3c152b60 2023-11-28T18:24:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.386+0000 7fae446d6640 1 -- 172.21.15.105:0/2839756767 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3c15b4e0 con 0x7fae3c152b60 2023-11-28T18:24:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.386+0000 7fae32ffd640 1 -- 172.21.15.105:0/2839756767 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fae34015590 con 0x7fae3c152b60 2023-11-28T18:24:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.390+0000 7fae32ffd640 1 --2- 172.21.15.105:0/2839756767 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae20066900 0x7fae20068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.390+0000 7fae32ffd640 1 -- 172.21.15.105:0/2839756767 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fae340847d0 con 0x7fae3c152b60 2023-11-28T18:24:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.390+0000 7fae42472640 1 --2- 172.21.15.105:0/2839756767 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae20066900 0x7fae20068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:52.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.390+0000 7fae42472640 1 --2- 172.21.15.105:0/2839756767 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae20066900 0x7fae20068dc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fae380ab000 tx=0x7fae380b1040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:52.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.394+0000 7fae32ffd640 1 -- 172.21.15.105:0/2839756767 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fae3c15b4e0 con 0x7fae3c152b60 2023-11-28T18:24:52.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.546+0000 7fae446d6640 1 -- 172.21.15.105:0/2839756767 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae3c15b4e0 con 0x7fae3c152b60 2023-11-28T18:24:52.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.546+0000 7fae32ffd640 1 -- 172.21.15.105:0/2839756767 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+13122 (secure 0 0 0) 0x7fae3c15b4e0 con 0x7fae3c152b60 2023-11-28T18:24:52.552 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:52.552 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":34,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:24:49.527177+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:24:52.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.550+0000 7fae446d6640 1 -- 172.21.15.105:0/2839756767 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae20066900 msgr2=0x7fae20068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:52.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.550+0000 7fae446d6640 1 --2- 172.21.15.105:0/2839756767 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae20066900 0x7fae20068dc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fae380ab000 tx=0x7fae380b1040 comp rx=0 tx=0).stop 2023-11-28T18:24:52.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.550+0000 7fae446d6640 1 -- 172.21.15.105:0/2839756767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae3c152b60 msgr2=0x7fae3c156d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.550+0000 7fae446d6640 1 --2- 172.21.15.105:0/2839756767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae3c152b60 0x7fae3c156d30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0b4e80 tx=0x7fae3400f580 comp rx=0 tx=0).stop 2023-11-28T18:24:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.550+0000 7fae446d6640 1 -- 172.21.15.105:0/2839756767 shutdown_connections 2023-11-28T18:24:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.550+0000 7fae446d6640 1 --2- 172.21.15.105:0/2839756767 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae20066900 0x7fae20068dc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.550+0000 7fae446d6640 1 --2- 172.21.15.105:0/2839756767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae3c152b60 0x7fae3c156d30 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.550+0000 7fae446d6640 1 --2- 172.21.15.105:0/2839756767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae3c1526e0 0x7fae3c1567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.550+0000 7fae446d6640 1 --2- 172.21.15.105:0/2839756767 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae28003980 0x7fae3c1f6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.550+0000 7fae446d6640 1 -- 172.21.15.105:0/2839756767 >> 172.21.15.105:0/2839756767 conn(0x7fae3c0ceb90 msgr2=0x7fae3c15e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.550+0000 7fae446d6640 1 -- 172.21.15.105:0/2839756767 shutdown_connections 2023-11-28T18:24:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.550+0000 7fae446d6640 1 -- 172.21.15.105:0/2839756767 wait complete. 2023-11-28T18:24:52.573 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:24:52.573 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:24:52.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.710+0000 7f4df7d2a640 1 Processor -- start 2023-11-28T18:24:52.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.710+0000 7f4df7d2a640 1 -- start start 2023-11-28T18:24:52.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.710+0000 7f4df7d2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4df0159610 0x7f4df00ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:52.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.710+0000 7f4df7d2a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4df006cef0 0x7f4df00ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:52.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.710+0000 7f4df7d2a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df00cde30 con 0x7f4df00cc760 2023-11-28T18:24:52.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.710+0000 7f4df7d2a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df01599f0 con 0x7f4df006cef0 2023-11-28T18:24:52.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.710+0000 7f4df7d2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df006de20 con 0x7f4df0159610 2023-11-28T18:24:52.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.710+0000 7f4df62c7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4df006cef0 0x7f4df00ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:52.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.710+0000 7f4df62c7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4df006cef0 0x7f4df00ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36726/0 (socket says 172.21.15.105:36726) 2023-11-28T18:24:52.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.710+0000 7f4df62c7640 1 -- 172.21.15.105:0/4211748700 learned_addr learned my addr 172.21.15.105:0/4211748700 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:52.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.710+0000 7f4df5ac6640 1 --2- 172.21.15.105:0/4211748700 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4df0159610 0x7f4df00ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:52.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.710+0000 7f4df62c7640 1 -- 172.21.15.105:0/4211748700 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4df0159610 msgr2=0x7f4df00ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:52.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.710+0000 7f4df62c7640 1 --2- 172.21.15.105:0/4211748700 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4df0159610 0x7f4df00ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.710+0000 7f4df62c7640 1 -- 172.21.15.105:0/4211748700 >> v1:172.21.15.148:6789/0 conn(0x7f4df00cc760 legacy=0x7f4df006c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:52.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.710+0000 7f4df62c7640 1 -- 172.21.15.105:0/4211748700 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df00cd220 con 0x7f4df006cef0 2023-11-28T18:24:52.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.710+0000 7f4df5ac6640 1 --2- 172.21.15.105:0/4211748700 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4df0159610 0x7f4df00ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:52.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.714+0000 7f4df62c7640 1 --2- 172.21.15.105:0/4211748700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4df006cef0 0x7f4df00ccbc0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009e50 tx=0x7f4de801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7186a672b81a714a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:52.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.714+0000 7f4df4ac4640 1 -- 172.21.15.105:0/4211748700 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de80a57a0 con 0x7f4df006cef0 2023-11-28T18:24:52.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.714+0000 7f4df4ac4640 1 -- 172.21.15.105:0/4211748700 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de80a5900 con 0x7f4df006cef0 2023-11-28T18:24:52.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.714+0000 7f4df4ac4640 1 -- 172.21.15.105:0/4211748700 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de80a5c90 con 0x7f4df006cef0 2023-11-28T18:24:52.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.714+0000 7f4df7d2a640 1 -- 172.21.15.105:0/4211748700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4df006cef0 msgr2=0x7f4df00ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:52.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.714+0000 7f4df7d2a640 1 --2- 172.21.15.105:0/4211748700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4df006cef0 0x7f4df00ccbc0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009e50 tx=0x7f4de801c9a0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.714+0000 7f4df7d2a640 1 -- 172.21.15.105:0/4211748700 shutdown_connections 2023-11-28T18:24:52.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.714+0000 7f4df7d2a640 1 --2- 172.21.15.105:0/4211748700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4df006cef0 0x7f4df00ccbc0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.714+0000 7f4df7d2a640 1 --2- 172.21.15.105:0/4211748700 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4df0159610 0x7f4df00ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.714+0000 7f4df7d2a640 1 -- 172.21.15.105:0/4211748700 >> 172.21.15.105:0/4211748700 conn(0x7f4df00ceb90 msgr2=0x7f4df00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:52.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.714+0000 7f4df7d2a640 1 -- 172.21.15.105:0/4211748700 shutdown_connections 2023-11-28T18:24:52.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.714+0000 7f4df7d2a640 1 -- 172.21.15.105:0/4211748700 wait complete. 2023-11-28T18:24:52.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4df7d2a640 1 Processor -- start 2023-11-28T18:24:52.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4df7d2a640 1 -- start start 2023-11-28T18:24:52.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4df7d2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4df006cef0 0x7f4df009c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:52.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4df7d2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4df00cc760 0x7f4df0150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:52.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4df7d2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4df0159610 0x7f4df014ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:52.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4df7d2a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df014ed10 con 0x7f4df0159610 2023-11-28T18:24:52.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4df7d2a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df014e100 con 0x7f4df00cc760 2023-11-28T18:24:52.727 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4df7d2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df014e3e0 con 0x7f4df006cef0 2023-11-28T18:24:52.727 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4df62c7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4df0159610 0x7f4df014ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:52.727 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4df62c7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4df0159610 0x7f4df014ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36730/0 (socket says 172.21.15.105:36730) 2023-11-28T18:24:52.727 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4df62c7640 1 -- 172.21.15.105:0/2093782314 learned_addr learned my addr 172.21.15.105:0/2093782314 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:52.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4df5ac6640 1 --2- 172.21.15.105:0/2093782314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4df006cef0 0x7f4df009c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:52.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4df52c5640 1 --2- 172.21.15.105:0/2093782314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4df00cc760 0x7f4df0150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:52.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4df62c7640 1 -- 172.21.15.105:0/2093782314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4df006cef0 msgr2=0x7f4df009c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:52.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4df62c7640 1 --2- 172.21.15.105:0/2093782314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4df006cef0 0x7f4df009c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4df62c7640 1 -- 172.21.15.105:0/2093782314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4df00cc760 msgr2=0x7f4df0150b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:52.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4df62c7640 1 --2- 172.21.15.105:0/2093782314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4df00cc760 0x7f4df0150b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4df62c7640 1 -- 172.21.15.105:0/2093782314 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de8009b30 con 0x7f4df0159610 2023-11-28T18:24:52.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4df62c7640 1 --2- 172.21.15.105:0/2093782314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4df0159610 0x7f4df014ab40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4de80af040 tx=0x7f4de8019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:52.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4de67fc640 1 -- 172.21.15.105:0/2093782314 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de80b8670 con 0x7f4df0159610 2023-11-28T18:24:52.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4de67fc640 1 -- 172.21.15.105:0/2093782314 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de80b87d0 con 0x7f4df0159610 2023-11-28T18:24:52.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4df7d2a640 1 -- 172.21.15.105:0/2093782314 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df014e6c0 con 0x7f4df0159610 2023-11-28T18:24:52.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4de67fc640 1 -- 172.21.15.105:0/2093782314 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de80b8b40 con 0x7f4df0159610 2023-11-28T18:24:52.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.718+0000 7f4df7d2a640 1 -- 172.21.15.105:0/2093782314 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df009c800 con 0x7f4df0159610 2023-11-28T18:24:52.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.722+0000 7f4df7d2a640 1 -- 172.21.15.105:0/2093782314 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df0067570 con 0x7f4df0159610 2023-11-28T18:24:52.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.722+0000 7f4de67fc640 1 -- 172.21.15.105:0/2093782314 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f4de80b8ca0 con 0x7f4df0159610 2023-11-28T18:24:52.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.722+0000 7f4de67fc640 1 --2- 172.21.15.105:0/2093782314 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4dd4066900 0x7f4dd4068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:52.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.722+0000 7f4de67fc640 1 -- 172.21.15.105:0/2093782314 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4de811f9b0 con 0x7f4df0159610 2023-11-28T18:24:52.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.722+0000 7f4df5ac6640 1 --2- 172.21.15.105:0/2093782314 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4dd4066900 0x7f4dd4068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:52.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.722+0000 7f4df5ac6640 1 --2- 172.21.15.105:0/2093782314 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4dd4066900 0x7f4dd4068dc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4dec006fd0 tx=0x7f4dec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:52.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.726+0000 7f4de67fc640 1 -- 172.21.15.105:0/2093782314 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4de80ecfb0 con 0x7f4df0159610 2023-11-28T18:24:52.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.878+0000 7f4df7d2a640 1 -- 172.21.15.105:0/2093782314 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4df00cf950 con 0x7f4df0159610 2023-11-28T18:24:52.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.878+0000 7f4de67fc640 1 -- 172.21.15.105:0/2093782314 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4de80f05d0 con 0x7f4df0159610 2023-11-28T18:24:52.883 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:24:52.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.878+0000 7f4df7d2a640 1 -- 172.21.15.105:0/2093782314 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4dd4066900 msgr2=0x7f4dd4068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:52.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.878+0000 7f4df7d2a640 1 --2- 172.21.15.105:0/2093782314 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4dd4066900 0x7f4dd4068dc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4dec006fd0 tx=0x7f4dec008040 comp rx=0 tx=0).stop 2023-11-28T18:24:52.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.878+0000 7f4df7d2a640 1 -- 172.21.15.105:0/2093782314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4df0159610 msgr2=0x7f4df014ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:52.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.878+0000 7f4df7d2a640 1 --2- 172.21.15.105:0/2093782314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4df0159610 0x7f4df014ab40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4de80af040 tx=0x7f4de8019740 comp rx=0 tx=0).stop 2023-11-28T18:24:52.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.878+0000 7f4df7d2a640 1 -- 172.21.15.105:0/2093782314 shutdown_connections 2023-11-28T18:24:52.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.878+0000 7f4df7d2a640 1 --2- 172.21.15.105:0/2093782314 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4dd4066900 0x7f4dd4068dc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.878+0000 7f4df7d2a640 1 --2- 172.21.15.105:0/2093782314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4df0159610 0x7f4df014ab40 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.878+0000 7f4df7d2a640 1 --2- 172.21.15.105:0/2093782314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4df00cc760 0x7f4df0150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.878+0000 7f4df7d2a640 1 --2- 172.21.15.105:0/2093782314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4df006cef0 0x7f4df009c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:52.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.878+0000 7f4df7d2a640 1 -- 172.21.15.105:0/2093782314 >> 172.21.15.105:0/2093782314 conn(0x7f4df00ceb90 msgr2=0x7f4df0168680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:52.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.878+0000 7f4df7d2a640 1 -- 172.21.15.105:0/2093782314 shutdown_connections 2023-11-28T18:24:52.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:52.878+0000 7f4df7d2a640 1 -- 172.21.15.105:0/2093782314 wait complete. 2023-11-28T18:24:52.904 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T18:24:53.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.034+0000 7f64b17f5640 1 Processor -- start 2023-11-28T18:24:53.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.034+0000 7f64b17f5640 1 -- start start 2023-11-28T18:24:53.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.038+0000 7f64b17f5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f64ac0b4140 0x7f64ac0d3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:53.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.038+0000 7f64b17f5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ac0d1d20 con 0x7f64ac0d10e0 2023-11-28T18:24:53.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.038+0000 7f64b17f5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ac0d01d0 con 0x7f64ac160530 2023-11-28T18:24:53.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.038+0000 7f64b17f5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ac0d04b0 con 0x7f64ac0b4140 2023-11-28T18:24:53.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.038+0000 7f64a27fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f64ac0b4140 0x7f64ac0d3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:53.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.038+0000 7f64aaffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f64ac0d10e0 0x7f64ac0d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54172/0 (socket says 172.21.15.105:54172) 2023-11-28T18:24:53.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.038+0000 7f64aaffd640 1 -- 172.21.15.105:0/163955758 learned_addr learned my addr 172.21.15.105:0/163955758 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:53.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.038+0000 7f64a27fc640 1 -- 172.21.15.105:0/163955758 >> v1:172.21.15.148:6789/0 conn(0x7f64ac160530 legacy=0x7f64ac160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:53.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.038+0000 7f64a27fc640 1 -- 172.21.15.105:0/163955758 >> v1:172.21.15.105:6789/0 conn(0x7f64ac0d10e0 legacy=0x7f64ac0d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:53.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.038+0000 7f64a27fc640 1 -- 172.21.15.105:0/163955758 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ac0d0790 con 0x7f64ac0b4140 2023-11-28T18:24:53.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.038+0000 7f64aa7fc640 1 -- 172.21.15.105:0/163955758 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2245066977 0 0) 0x7f64ac0d1d20 con 0x7f64ac0d10e0 2023-11-28T18:24:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.042+0000 7f64a27fc640 1 --2- 172.21.15.105:0/163955758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64ac0b4140 0x7f64ac0d3bc0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6494009d90 tx=0x7f649401c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc4756c4ee23e10 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.042+0000 7f64aa7fc640 1 -- 172.21.15.105:0/163955758 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64940a5960 con 0x7f64ac0b4140 2023-11-28T18:24:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.042+0000 7f64aa7fc640 1 -- 172.21.15.105:0/163955758 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64940a5ac0 con 0x7f64ac0b4140 2023-11-28T18:24:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.042+0000 7f64aa7fc640 1 -- 172.21.15.105:0/163955758 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64940a5e50 con 0x7f64ac0b4140 2023-11-28T18:24:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.042+0000 7f64b17f5640 1 -- 172.21.15.105:0/163955758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64ac0b4140 msgr2=0x7f64ac0d3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.042+0000 7f64b17f5640 1 --2- 172.21.15.105:0/163955758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64ac0b4140 0x7f64ac0d3bc0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6494009d90 tx=0x7f649401c840 comp rx=0 tx=0).stop 2023-11-28T18:24:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.042+0000 7f64b17f5640 1 -- 172.21.15.105:0/163955758 shutdown_connections 2023-11-28T18:24:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.042+0000 7f64b17f5640 1 --2- 172.21.15.105:0/163955758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64ac0b4140 0x7f64ac0d3bc0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.042+0000 7f64b17f5640 1 -- 172.21.15.105:0/163955758 >> 172.21.15.105:0/163955758 conn(0x7f64ac0d30a0 msgr2=0x7f64ac0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.042+0000 7f64b17f5640 1 -- 172.21.15.105:0/163955758 shutdown_connections 2023-11-28T18:24:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.042+0000 7f64b17f5640 1 -- 172.21.15.105:0/163955758 wait complete. 2023-11-28T18:24:53.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.042+0000 7f64b17f5640 1 Processor -- start 2023-11-28T18:24:53.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.042+0000 7f64b17f5640 1 -- start start 2023-11-28T18:24:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64b17f5640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64ac0b4140 0x7f64ac1f95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64b17f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64ac0d10e0 0x7f64ac15f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64b17f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64ac160530 0x7f64ac156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64b17f5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac1f9ae0 con 0x7f64ac0d10e0 2023-11-28T18:24:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64b17f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac0ccf10 con 0x7f64ac160530 2023-11-28T18:24:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64b17f5640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac0cb640 con 0x7f64ac0b4140 2023-11-28T18:24:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64a27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64ac0d10e0 0x7f64ac15f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64a27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64ac0d10e0 0x7f64ac15f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36750/0 (socket says 172.21.15.105:36750) 2023-11-28T18:24:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64a27fc640 1 -- 172.21.15.105:0/1274994502 learned_addr learned my addr 172.21.15.105:0/1274994502 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64aaffd640 1 --2- 172.21.15.105:0/1274994502 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64ac0b4140 0x7f64ac1f95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64ab7fe640 1 --2- 172.21.15.105:0/1274994502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64ac160530 0x7f64ac156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:53.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64a27fc640 1 -- 172.21.15.105:0/1274994502 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64ac0b4140 msgr2=0x7f64ac1f95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:53.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64a27fc640 1 --2- 172.21.15.105:0/1274994502 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64ac0b4140 0x7f64ac1f95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:53.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64a27fc640 1 -- 172.21.15.105:0/1274994502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64ac160530 msgr2=0x7f64ac156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:53.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64a27fc640 1 --2- 172.21.15.105:0/1274994502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64ac160530 0x7f64ac156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:53.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64a27fc640 1 -- 172.21.15.105:0/1274994502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6494009a70 con 0x7f64ac0d10e0 2023-11-28T18:24:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64a27fc640 1 --2- 172.21.15.105:0/1274994502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64ac0d10e0 0x7f64ac15f000 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6494019450 tx=0x7f64940195e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64a3fff640 1 -- 172.21.15.105:0/1274994502 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64940ade30 con 0x7f64ac0d10e0 2023-11-28T18:24:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64a3fff640 1 -- 172.21.15.105:0/1274994502 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64940b8420 con 0x7f64ac0d10e0 2023-11-28T18:24:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64b17f5640 1 -- 172.21.15.105:0/1274994502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ac0cb920 con 0x7f64ac0d10e0 2023-11-28T18:24:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64a3fff640 1 -- 172.21.15.105:0/1274994502 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64940b87d0 con 0x7f64ac0d10e0 2023-11-28T18:24:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64b17f5640 1 -- 172.21.15.105:0/1274994502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ac0cbee0 con 0x7f64ac0d10e0 2023-11-28T18:24:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.046+0000 7f64b17f5640 1 -- 172.21.15.105:0/1274994502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ac068650 con 0x7f64ac0d10e0 2023-11-28T18:24:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.050+0000 7f64a3fff640 1 -- 172.21.15.105:0/1274994502 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f64940b8930 con 0x7f64ac0d10e0 2023-11-28T18:24:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.050+0000 7f64a3fff640 1 --2- 172.21.15.105:0/1274994502 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f648c066900 0x7f648c068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.050+0000 7f64a3fff640 1 -- 172.21.15.105:0/1274994502 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f649411f730 con 0x7f64ac0d10e0 2023-11-28T18:24:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.050+0000 7f64aaffd640 1 --2- 172.21.15.105:0/1274994502 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f648c066900 0x7f648c068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:53.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.050+0000 7f64aaffd640 1 --2- 172.21.15.105:0/1274994502 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f648c066900 0x7f648c068dc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f649c007fd0 tx=0x7f649c007d60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:53.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.054+0000 7f64a3fff640 1 -- 172.21.15.105:0/1274994502 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f64940ecc20 con 0x7f64ac0d10e0 2023-11-28T18:24:53.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.202+0000 7f64b17f5640 1 -- 172.21.15.105:0/1274994502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f64ac0ce670 con 0x7f64ac0d10e0 2023-11-28T18:24:53.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.202+0000 7f64a3fff640 1 -- 172.21.15.105:0/1274994502 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) v1 ==== 84+0+65 (secure 0 0 0) 0x7f64940b2060 con 0x7f64ac0d10e0 2023-11-28T18:24:53.210 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:53.210 INFO:teuthology.orchestra.run.smithi105.stdout:[ 2023-11-28T18:24:53.210 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:24:53.210 INFO:teuthology.orchestra.run.smithi105.stdout: "poolnum": 1, 2023-11-28T18:24:53.210 INFO:teuthology.orchestra.run.smithi105.stdout: "poolname": ".mgr" 2023-11-28T18:24:53.211 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:24:53.211 INFO:teuthology.orchestra.run.smithi105.stdout:] 2023-11-28T18:24:53.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.206+0000 7f64b17f5640 1 -- 172.21.15.105:0/1274994502 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f648c066900 msgr2=0x7f648c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:53.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.206+0000 7f64b17f5640 1 --2- 172.21.15.105:0/1274994502 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f648c066900 0x7f648c068dc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f649c007fd0 tx=0x7f649c007d60 comp rx=0 tx=0).stop 2023-11-28T18:24:53.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.206+0000 7f64b17f5640 1 -- 172.21.15.105:0/1274994502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64ac0d10e0 msgr2=0x7f64ac15f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:53.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.206+0000 7f64b17f5640 1 --2- 172.21.15.105:0/1274994502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64ac0d10e0 0x7f64ac15f000 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6494019450 tx=0x7f64940195e0 comp rx=0 tx=0).stop 2023-11-28T18:24:53.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.206+0000 7f64b17f5640 1 -- 172.21.15.105:0/1274994502 shutdown_connections 2023-11-28T18:24:53.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.206+0000 7f64b17f5640 1 --2- 172.21.15.105:0/1274994502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64ac160530 0x7f64ac156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:53.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.206+0000 7f64b17f5640 1 --2- 172.21.15.105:0/1274994502 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f648c066900 0x7f648c068dc0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:53.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.206+0000 7f64b17f5640 1 --2- 172.21.15.105:0/1274994502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64ac0d10e0 0x7f64ac15f000 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:53.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.206+0000 7f64b17f5640 1 --2- 172.21.15.105:0/1274994502 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64ac0b4140 0x7f64ac1f95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:53.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.206+0000 7f64b17f5640 1 -- 172.21.15.105:0/1274994502 >> 172.21.15.105:0/1274994502 conn(0x7f64ac0d30a0 msgr2=0x7f64ac0b4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:53.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.206+0000 7f64b17f5640 1 -- 172.21.15.105:0/1274994502 shutdown_connections 2023-11-28T18:24:53.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.206+0000 7f64b17f5640 1 -- 172.21.15.105:0/1274994502 wait complete. 2023-11-28T18:24:53.229 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T18:24:53.229 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T18:24:53.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.366+0000 7f2f7d89a640 1 Processor -- start 2023-11-28T18:24:53.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.366+0000 7f2f7d89a640 1 -- start start 2023-11-28T18:24:53.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.366+0000 7f2f7d89a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f7815d2a0 0x7f2f780ad960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:53.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.366+0000 7f2f7d89a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2f7815d680 0x7f2f780d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:53.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.366+0000 7f2f7d89a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f780b4290 con 0x7f2f7809a3f0 2023-11-28T18:24:53.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.366+0000 7f2f7d89a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f780b0070 con 0x7f2f7815d680 2023-11-28T18:24:53.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.366+0000 7f2f7d89a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f780b44e0 con 0x7f2f7815d2a0 2023-11-28T18:24:53.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.366+0000 7f2f77fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2f7815d680 0x7f2f780d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:53.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.366+0000 7f2f77fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2f7815d680 0x7f2f780d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36766/0 (socket says 172.21.15.105:36766) 2023-11-28T18:24:53.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.366+0000 7f2f77fff640 1 -- 172.21.15.105:0/1836109897 learned_addr learned my addr 172.21.15.105:0/1836109897 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:53.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.366+0000 7f2f77fff640 1 -- 172.21.15.105:0/1836109897 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f7815d2a0 msgr2=0x7f2f780ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:53.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.366+0000 7f2f76ffd640 1 --2- 172.21.15.105:0/1836109897 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f7815d2a0 0x7f2f780ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:53.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.366+0000 7f2f77fff640 1 --2- 172.21.15.105:0/1836109897 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f7815d2a0 0x7f2f780ad960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:53.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.366+0000 7f2f77fff640 1 -- 172.21.15.105:0/1836109897 >> v1:172.21.15.148:6789/0 conn(0x7f2f7809a3f0 legacy=0x7f2f780b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:53.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.366+0000 7f2f77fff640 1 -- 172.21.15.105:0/1836109897 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f780b47c0 con 0x7f2f7815d680 2023-11-28T18:24:53.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.366+0000 7f2f76ffd640 1 --2- 172.21.15.105:0/1836109897 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f7815d2a0 0x7f2f780ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:53.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.370+0000 7f2f77fff640 1 --2- 172.21.15.105:0/1836109897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7815d680 0x7f2f780d00b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009e50 tx=0x7f2f7001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d324ec7cedb3bc34 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:53.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.370+0000 7f2f75ffb640 1 -- 172.21.15.105:0/1836109897 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f700a57e0 con 0x7f2f7815d680 2023-11-28T18:24:53.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.370+0000 7f2f75ffb640 1 -- 172.21.15.105:0/1836109897 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f700a5940 con 0x7f2f7815d680 2023-11-28T18:24:53.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.370+0000 7f2f75ffb640 1 -- 172.21.15.105:0/1836109897 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f700a5cd0 con 0x7f2f7815d680 2023-11-28T18:24:53.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.370+0000 7f2f7d89a640 1 -- 172.21.15.105:0/1836109897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7815d680 msgr2=0x7f2f780d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:53.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.370+0000 7f2f7d89a640 1 --2- 172.21.15.105:0/1836109897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7815d680 0x7f2f780d00b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009e50 tx=0x7f2f7001c940 comp rx=0 tx=0).stop 2023-11-28T18:24:53.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.370+0000 7f2f7d89a640 1 -- 172.21.15.105:0/1836109897 shutdown_connections 2023-11-28T18:24:53.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.370+0000 7f2f7d89a640 1 --2- 172.21.15.105:0/1836109897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7815d680 0x7f2f780d00b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:53.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.370+0000 7f2f7d89a640 1 --2- 172.21.15.105:0/1836109897 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f7815d2a0 0x7f2f780ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:53.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.370+0000 7f2f7d89a640 1 -- 172.21.15.105:0/1836109897 >> 172.21.15.105:0/1836109897 conn(0x7f2f78099fe0 msgr2=0x7f2f780b3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:53.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.370+0000 7f2f7d89a640 1 -- 172.21.15.105:0/1836109897 shutdown_connections 2023-11-28T18:24:53.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.370+0000 7f2f7d89a640 1 -- 172.21.15.105:0/1836109897 wait complete. 2023-11-28T18:24:53.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.374+0000 7f2f7d89a640 1 Processor -- start 2023-11-28T18:24:53.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.374+0000 7f2f7d89a640 1 -- start start 2023-11-28T18:24:53.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.374+0000 7f2f7d89a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f7809a3f0 0x7f2f781599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:53.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.374+0000 7f2f7d89a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7815d2a0 0x7f2f78152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:53.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.374+0000 7f2f7d89a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f7815d680 0x7f2f780c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:53.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.374+0000 7f2f7d89a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f780c7620 con 0x7f2f7815d2a0 2023-11-28T18:24:53.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.374+0000 7f2f7d89a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f781545e0 con 0x7f2f7809a3f0 2023-11-28T18:24:53.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.374+0000 7f2f7d89a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f781548c0 con 0x7f2f7815d680 2023-11-28T18:24:53.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.374+0000 7f2f767fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7815d2a0 0x7f2f78152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:53.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.374+0000 7f2f767fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7815d2a0 0x7f2f78152c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36782/0 (socket says 172.21.15.105:36782) 2023-11-28T18:24:53.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.374+0000 7f2f767fc640 1 -- 172.21.15.105:0/3714501968 learned_addr learned my addr 172.21.15.105:0/3714501968 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:53.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.374+0000 7f2f767fc640 1 -- 172.21.15.105:0/3714501968 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f7815d680 msgr2=0x7f2f780c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:53.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.374+0000 7f2f767fc640 1 --2- 172.21.15.105:0/3714501968 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f7815d680 0x7f2f780c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:53.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.374+0000 7f2f767fc640 1 -- 172.21.15.105:0/3714501968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f7809a3f0 msgr2=0x7f2f781599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:53.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.374+0000 7f2f767fc640 1 --2- 172.21.15.105:0/3714501968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f7809a3f0 0x7f2f781599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:53.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.374+0000 7f2f767fc640 1 -- 172.21.15.105:0/3714501968 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f70009b30 con 0x7f2f7815d2a0 2023-11-28T18:24:53.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.374+0000 7f2f767fc640 1 --2- 172.21.15.105:0/3714501968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7815d2a0 0x7f2f78152c90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00c410 tx=0x7f2f6c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:53.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.374+0000 7f2f5ffff640 1 -- 172.21.15.105:0/3714501968 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c011790 con 0x7f2f7815d2a0 2023-11-28T18:24:53.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.374+0000 7f2f5ffff640 1 -- 172.21.15.105:0/3714501968 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6c0118f0 con 0x7f2f7815d2a0 2023-11-28T18:24:53.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.374+0000 7f2f7d89a640 1 -- 172.21.15.105:0/3714501968 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f78154ba0 con 0x7f2f7815d2a0 2023-11-28T18:24:53.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.374+0000 7f2f5ffff640 1 -- 172.21.15.105:0/3714501968 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c011ca0 con 0x7f2f7815d2a0 2023-11-28T18:24:53.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.374+0000 7f2f7d89a640 1 -- 172.21.15.105:0/3714501968 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f780c7210 con 0x7f2f7815d2a0 2023-11-28T18:24:53.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.378+0000 7f2f7d89a640 1 -- 172.21.15.105:0/3714501968 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f78154de0 con 0x7f2f7815d2a0 2023-11-28T18:24:53.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.378+0000 7f2f5ffff640 1 -- 172.21.15.105:0/3714501968 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f2f6c01c460 con 0x7f2f7815d2a0 2023-11-28T18:24:53.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.378+0000 7f2f5ffff640 1 --2- 172.21.15.105:0/3714501968 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2f640668d0 0x7f2f64068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:53.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.378+0000 7f2f5ffff640 1 -- 172.21.15.105:0/3714501968 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f2f6c084220 con 0x7f2f7815d2a0 2023-11-28T18:24:53.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.378+0000 7f2f76ffd640 1 --2- 172.21.15.105:0/3714501968 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2f640668d0 0x7f2f64068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:53.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.378+0000 7f2f76ffd640 1 --2- 172.21.15.105:0/3714501968 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2f640668d0 0x7f2f64068d90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800a8d0 tx=0x7f2f68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:53.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.382+0000 7f2f5ffff640 1 -- 172.21.15.105:0/3714501968 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2f78154de0 con 0x7f2f7815d2a0 2023-11-28T18:24:53.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:53.534+0000 7f2f7d89a640 1 -- 172.21.15.105:0/3714501968 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2f78154d90 con 0x7f2f7815d2a0 2023-11-28T18:24:54.539 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.534+0000 7f2f5ffff640 1 -- 172.21.15.105:0/3714501968 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2f6c051540 con 0x7f2f7815d2a0 2023-11-28T18:24:54.539 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' created 2023-11-28T18:24:54.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.534+0000 7f2f7d89a640 1 -- 172.21.15.105:0/3714501968 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2f640668d0 msgr2=0x7f2f64068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:54.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.534+0000 7f2f7d89a640 1 --2- 172.21.15.105:0/3714501968 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2f640668d0 0x7f2f64068d90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800a8d0 tx=0x7f2f68008040 comp rx=0 tx=0).stop 2023-11-28T18:24:54.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.534+0000 7f2f7d89a640 1 -- 172.21.15.105:0/3714501968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7815d2a0 msgr2=0x7f2f78152c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:54.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.534+0000 7f2f7d89a640 1 --2- 172.21.15.105:0/3714501968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7815d2a0 0x7f2f78152c90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00c410 tx=0x7f2f6c00c8d0 comp rx=0 tx=0).stop 2023-11-28T18:24:54.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.538+0000 7f2f7d89a640 1 -- 172.21.15.105:0/3714501968 shutdown_connections 2023-11-28T18:24:54.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.538+0000 7f2f7d89a640 1 --2- 172.21.15.105:0/3714501968 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f7815d680 0x7f2f780c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:54.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.538+0000 7f2f7d89a640 1 --2- 172.21.15.105:0/3714501968 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2f640668d0 0x7f2f64068d90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:54.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.538+0000 7f2f7d89a640 1 --2- 172.21.15.105:0/3714501968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7815d2a0 0x7f2f78152c90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:54.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.538+0000 7f2f7d89a640 1 --2- 172.21.15.105:0/3714501968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f7809a3f0 0x7f2f781599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:54.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.538+0000 7f2f7d89a640 1 -- 172.21.15.105:0/3714501968 >> 172.21.15.105:0/3714501968 conn(0x7f2f78099fe0 msgr2=0x7f2f780b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:54.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.538+0000 7f2f7d89a640 1 -- 172.21.15.105:0/3714501968 shutdown_connections 2023-11-28T18:24:54.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.538+0000 7f2f7d89a640 1 -- 172.21.15.105:0/3714501968 wait complete. 2023-11-28T18:24:54.568 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T18:24:54.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.706+0000 7f342c6c9640 1 Processor -- start 2023-11-28T18:24:54.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.706+0000 7f342c6c9640 1 -- start start 2023-11-28T18:24:54.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.710+0000 7f342c6c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f342415d4f0 0x7f34240d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:54.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.710+0000 7f342c6c9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34240b25b0 con 0x7f34240b2d20 2023-11-28T18:24:54.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.710+0000 7f342c6c9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34240cea30 con 0x7f34240d3c30 2023-11-28T18:24:54.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.710+0000 7f342c6c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34240b2900 con 0x7f342415d4f0 2023-11-28T18:24:54.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.710+0000 7f342ac66640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f34240b2d20 0x7f34240b31d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54180/0 (socket says 172.21.15.105:54180) 2023-11-28T18:24:54.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.710+0000 7f342ac66640 1 -- 172.21.15.105:0/4033480442 learned_addr learned my addr 172.21.15.105:0/4033480442 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:54.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.710+0000 7f342a465640 1 --2- 172.21.15.105:0/4033480442 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f342415d4f0 0x7f34240d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:54.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.710+0000 7f3429463640 1 -- 172.21.15.105:0/4033480442 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3691840051 0 0) 0x7f34240b25b0 con 0x7f34240b2d20 2023-11-28T18:24:54.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.710+0000 7f3429463640 1 -- 172.21.15.105:0/4033480442 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3410003530 con 0x7f34240b2d20 2023-11-28T18:24:54.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.710+0000 7f3429463640 1 -- 172.21.15.105:0/4033480442 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2282663898 0 0) 0x7f34240cea30 con 0x7f34240d3c30 2023-11-28T18:24:54.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.710+0000 7f3429463640 1 -- 172.21.15.105:0/4033480442 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34240b25b0 con 0x7f34240d3c30 2023-11-28T18:24:54.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.710+0000 7f342a465640 1 -- 172.21.15.105:0/4033480442 >> v1:172.21.15.148:6789/0 conn(0x7f34240d3c30 legacy=0x7f34240d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:54.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.710+0000 7f342a465640 1 -- 172.21.15.105:0/4033480442 >> v1:172.21.15.105:6789/0 conn(0x7f34240b2d20 legacy=0x7f34240b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:54.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.710+0000 7f342a465640 1 -- 172.21.15.105:0/4033480442 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34240ced10 con 0x7f342415d4f0 2023-11-28T18:24:54.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.714+0000 7f342a465640 1 --2- 172.21.15.105:0/4033480442 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f342415d4f0 0x7f34240d0e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f34200097f0 tx=0x7f342001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8c8dfba1490dc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:54.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.714+0000 7f3429463640 1 -- 172.21.15.105:0/4033480442 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34200ab600 con 0x7f342415d4f0 2023-11-28T18:24:54.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.714+0000 7f3429463640 1 -- 172.21.15.105:0/4033480442 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34200a7050 con 0x7f342415d4f0 2023-11-28T18:24:54.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.714+0000 7f3429463640 1 -- 172.21.15.105:0/4033480442 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34200ab980 con 0x7f342415d4f0 2023-11-28T18:24:54.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.714+0000 7f342c6c9640 1 -- 172.21.15.105:0/4033480442 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f342415d4f0 msgr2=0x7f34240d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:54.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.714+0000 7f342c6c9640 1 --2- 172.21.15.105:0/4033480442 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f342415d4f0 0x7f34240d0e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f34200097f0 tx=0x7f342001c400 comp rx=0 tx=0).stop 2023-11-28T18:24:54.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.714+0000 7f342c6c9640 1 -- 172.21.15.105:0/4033480442 shutdown_connections 2023-11-28T18:24:54.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.714+0000 7f342c6c9640 1 --2- 172.21.15.105:0/4033480442 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f342415d4f0 0x7f34240d0e10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:54.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.714+0000 7f342c6c9640 1 -- 172.21.15.105:0/4033480442 >> 172.21.15.105:0/4033480442 conn(0x7f34240d3030 msgr2=0x7f34240d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:54.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.714+0000 7f342c6c9640 1 -- 172.21.15.105:0/4033480442 shutdown_connections 2023-11-28T18:24:54.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.714+0000 7f342c6c9640 1 -- 172.21.15.105:0/4033480442 wait complete. 2023-11-28T18:24:54.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.714+0000 7f342c6c9640 1 Processor -- start 2023-11-28T18:24:54.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.714+0000 7f342c6c9640 1 -- start start 2023-11-28T18:24:54.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f342c6c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34240b2d20 0x7f3424153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:54.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f342c6c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34240d3c30 0x7f3424160010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:54.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f342c6c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f342415d4f0 0x7f34241542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:54.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f342c6c9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34240d0a20 con 0x7f34240d3c30 2023-11-28T18:24:54.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f342c6c9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34241565f0 con 0x7f34240b2d20 2023-11-28T18:24:54.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f342c6c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34240b6f40 con 0x7f342415d4f0 2023-11-28T18:24:54.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f3429c64640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34240d3c30 0x7f3424160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:54.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f3429c64640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34240d3c30 0x7f3424160010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36798/0 (socket says 172.21.15.105:36798) 2023-11-28T18:24:54.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f3429c64640 1 -- 172.21.15.105:0/786624384 learned_addr learned my addr 172.21.15.105:0/786624384 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:54.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f342a465640 1 --2- 172.21.15.105:0/786624384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34240b2d20 0x7f3424153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:54.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f342ac66640 1 --2- 172.21.15.105:0/786624384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f342415d4f0 0x7f34241542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:54.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f3429c64640 1 -- 172.21.15.105:0/786624384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f342415d4f0 msgr2=0x7f34241542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:54.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f3429c64640 1 --2- 172.21.15.105:0/786624384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f342415d4f0 0x7f34241542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:54.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f3429c64640 1 -- 172.21.15.105:0/786624384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34240b2d20 msgr2=0x7f3424153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:54.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f3429c64640 1 --2- 172.21.15.105:0/786624384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34240b2d20 0x7f3424153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:54.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f3429c64640 1 -- 172.21.15.105:0/786624384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3420009420 con 0x7f34240d3c30 2023-11-28T18:24:54.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f3429c64640 1 --2- 172.21.15.105:0/786624384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34240d3c30 0x7f3424160010 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f341400c460 tx=0x7f341400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:54.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f341affd640 1 -- 172.21.15.105:0/786624384 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34140052d0 con 0x7f34240d3c30 2023-11-28T18:24:54.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f341affd640 1 -- 172.21.15.105:0/786624384 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3414005450 con 0x7f34240d3c30 2023-11-28T18:24:54.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f341affd640 1 -- 172.21.15.105:0/786624384 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414012870 con 0x7f34240d3c30 2023-11-28T18:24:54.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f342c6c9640 1 -- 172.21.15.105:0/786624384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3424160550 con 0x7f34240d3c30 2023-11-28T18:24:54.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f342c6c9640 1 -- 172.21.15.105:0/786624384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342415f5c0 con 0x7f34240d3c30 2023-11-28T18:24:54.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.718+0000 7f342c6c9640 1 -- 172.21.15.105:0/786624384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3424067570 con 0x7f34240d3c30 2023-11-28T18:24:54.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.722+0000 7f341affd640 1 -- 172.21.15.105:0/786624384 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f34140129d0 con 0x7f34240d3c30 2023-11-28T18:24:54.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.722+0000 7f341affd640 1 --2- 172.21.15.105:0/786624384 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f34080668d0 0x7f3408068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:54.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.722+0000 7f341affd640 1 -- 172.21.15.105:0/786624384 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f3414084690 con 0x7f34240d3c30 2023-11-28T18:24:54.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.722+0000 7f342a465640 1 --2- 172.21.15.105:0/786624384 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f34080668d0 0x7f3408068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:54.727 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.722+0000 7f342a465640 1 --2- 172.21.15.105:0/786624384 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f34080668d0 0x7f3408068d90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f34200ad000 tx=0x7f34200a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:54.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.722+0000 7f341affd640 1 -- 172.21.15.105:0/786624384 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3414051a00 con 0x7f34240d3c30 2023-11-28T18:24:54.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:54.874+0000 7f342c6c9640 1 -- 172.21.15.105:0/786624384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3424160550 con 0x7f34240d3c30 2023-11-28T18:24:55.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.542+0000 7f341affd640 1 -- 172.21.15.105:0/786624384 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3424160550 con 0x7f34240d3c30 2023-11-28T18:24:55.548 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' created 2023-11-28T18:24:55.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.546+0000 7f342c6c9640 1 -- 172.21.15.105:0/786624384 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f34080668d0 msgr2=0x7f3408068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:55.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.546+0000 7f342c6c9640 1 --2- 172.21.15.105:0/786624384 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f34080668d0 0x7f3408068d90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f34200ad000 tx=0x7f34200a6000 comp rx=0 tx=0).stop 2023-11-28T18:24:55.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.546+0000 7f342c6c9640 1 -- 172.21.15.105:0/786624384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34240d3c30 msgr2=0x7f3424160010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:55.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.546+0000 7f342c6c9640 1 --2- 172.21.15.105:0/786624384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34240d3c30 0x7f3424160010 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f341400c460 tx=0x7f341400c920 comp rx=0 tx=0).stop 2023-11-28T18:24:55.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.546+0000 7f342c6c9640 1 -- 172.21.15.105:0/786624384 shutdown_connections 2023-11-28T18:24:55.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.546+0000 7f342c6c9640 1 --2- 172.21.15.105:0/786624384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f342415d4f0 0x7f34241542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:55.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.546+0000 7f342c6c9640 1 --2- 172.21.15.105:0/786624384 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f34080668d0 0x7f3408068d90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:55.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.546+0000 7f342c6c9640 1 --2- 172.21.15.105:0/786624384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34240d3c30 0x7f3424160010 secure :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f341400c460 tx=0x7f341400c920 comp rx=0 tx=0).stop 2023-11-28T18:24:55.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.546+0000 7f342c6c9640 1 --2- 172.21.15.105:0/786624384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34240b2d20 0x7f3424153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:55.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.546+0000 7f342c6c9640 1 -- 172.21.15.105:0/786624384 >> 172.21.15.105:0/786624384 conn(0x7f34240d3030 msgr2=0x7f34240d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:55.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.546+0000 7f342c6c9640 1 -- 172.21.15.105:0/786624384 shutdown_connections 2023-11-28T18:24:55.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.546+0000 7f342c6c9640 1 -- 172.21.15.105:0/786624384 wait complete. 2023-11-28T18:24:55.587 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T18:24:55.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.722+0000 7efc2a1b5640 1 Processor -- start 2023-11-28T18:24:55.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.722+0000 7efc2a1b5640 1 -- start start 2023-11-28T18:24:55.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc2a1b5640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc2415e9f0 0x7efc2406de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:55.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc2a1b5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efc240cc8e0 0x7efc240ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:55.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc2a1b5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc2406e3b0 con 0x7efc2406c070 2023-11-28T18:24:55.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc2a1b5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc2406c500 con 0x7efc240cc8e0 2023-11-28T18:24:55.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc2a1b5640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2406c7e0 con 0x7efc2415e9f0 2023-11-28T18:24:55.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc289b2640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7efc2406c070 0x7efc2415edd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54184/0 (socket says 172.21.15.105:54184) 2023-11-28T18:24:55.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc289b2640 1 -- 172.21.15.105:0/3520889849 learned_addr learned my addr 172.21.15.105:0/3520889849 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:55.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc299b4640 1 --2- 172.21.15.105:0/3520889849 >> v2:172.21.15.148:3300/0 conn(0x7efc240cc8e0 0x7efc240ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:55.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc291b3640 1 --2- 172.21.15.105:0/3520889849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc2415e9f0 0x7efc2406de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:55.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc1bfff640 1 -- 172.21.15.105:0/3520889849 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4235709975 0 0) 0x7efc14001120 con 0x7efc2406c070 2023-11-28T18:24:55.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc1bfff640 1 -- 172.21.15.105:0/3520889849 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc10003530 con 0x7efc2406c070 2023-11-28T18:24:55.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc1bfff640 1 -- 172.21.15.105:0/3520889849 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2501677874 0 0) 0x7efc10003530 con 0x7efc2406c070 2023-11-28T18:24:55.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc1bfff640 1 -- 172.21.15.105:0/3520889849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc2415e9f0 msgr2=0x7efc2406de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:55.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc1bfff640 1 --2- 172.21.15.105:0/3520889849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc2415e9f0 0x7efc2406de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:55.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc1bfff640 1 -- 172.21.15.105:0/3520889849 >> v2:172.21.15.148:3300/0 conn(0x7efc240cc8e0 msgr2=0x7efc240ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:55.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc1bfff640 1 --2- 172.21.15.105:0/3520889849 >> v2:172.21.15.148:3300/0 conn(0x7efc240cc8e0 0x7efc240ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:55.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc1bfff640 1 -- 172.21.15.105:0/3520889849 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2406cac0 con 0x7efc2406c070 2023-11-28T18:24:55.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc1bfff640 1 -- 172.21.15.105:0/3520889849 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7efc14001810 con 0x7efc2406c070 2023-11-28T18:24:55.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc1bfff640 1 -- 172.21.15.105:0/3520889849 >> v1:172.21.15.105:6789/0 conn(0x7efc2406c070 legacy=0x7efc2415edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:55.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc1bfff640 1 --2- 172.21.15.105:0/3520889849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc10003980 0x7efc10003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:55.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc1bfff640 1 --2- 172.21.15.105:0/3520889849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc100047d0 0x7efc10004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:55.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc1bfff640 1 --2- 172.21.15.105:0/3520889849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc2415e9f0 0x7efc10005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:55.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc1bfff640 1 -- 172.21.15.105:0/3520889849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2406c500 con 0x7efc100047d0 2023-11-28T18:24:55.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc1bfff640 1 -- 172.21.15.105:0/3520889849 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2406c7e0 con 0x7efc10003980 2023-11-28T18:24:55.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc1bfff640 1 -- 172.21.15.105:0/3520889849 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc1000b990 con 0x7efc2415e9f0 2023-11-28T18:24:55.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc289b2640 1 --2- 172.21.15.105:0/3520889849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc100047d0 0x7efc10004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:55.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc299b4640 1 --2- 172.21.15.105:0/3520889849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc2415e9f0 0x7efc10005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:55.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc291b3640 1 --2- 172.21.15.105:0/3520889849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc10003980 0x7efc10003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:55.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc289b2640 1 -- 172.21.15.105:0/3520889849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc2415e9f0 msgr2=0x7efc10005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:55.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc289b2640 1 --2- 172.21.15.105:0/3520889849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc2415e9f0 0x7efc10005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:55.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc289b2640 1 -- 172.21.15.105:0/3520889849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc10003980 msgr2=0x7efc10003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:55.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc289b2640 1 --2- 172.21.15.105:0/3520889849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc10003980 0x7efc10003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:55.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.726+0000 7efc289b2640 1 -- 172.21.15.105:0/3520889849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc10003530 con 0x7efc100047d0 2023-11-28T18:24:55.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.730+0000 7efc289b2640 1 --2- 172.21.15.105:0/3520889849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc100047d0 0x7efc10004c50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7efc1400c210 tx=0x7efc1401f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31a0595366434af server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:55.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.730+0000 7efc1bfff640 1 -- 172.21.15.105:0/3520889849 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc140a8ce0 con 0x7efc100047d0 2023-11-28T18:24:55.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.730+0000 7efc1bfff640 1 -- 172.21.15.105:0/3520889849 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc140a8e60 con 0x7efc100047d0 2023-11-28T18:24:55.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.730+0000 7efc1bfff640 1 -- 172.21.15.105:0/3520889849 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc14005b20 con 0x7efc100047d0 2023-11-28T18:24:55.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.730+0000 7efc2a1b5640 1 -- 172.21.15.105:0/3520889849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc100047d0 msgr2=0x7efc10004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:55.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.730+0000 7efc2a1b5640 1 --2- 172.21.15.105:0/3520889849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc100047d0 0x7efc10004c50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7efc1400c210 tx=0x7efc1401f3b0 comp rx=0 tx=0).stop 2023-11-28T18:24:55.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.730+0000 7efc299b4640 1 -- 172.21.15.105:0/3520889849 reap_dead start 2023-11-28T18:24:55.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.730+0000 7efc2a1b5640 1 -- 172.21.15.105:0/3520889849 shutdown_connections 2023-11-28T18:24:55.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.730+0000 7efc2a1b5640 1 -- 172.21.15.105:0/3520889849 >> 172.21.15.105:0/3520889849 conn(0x7efc240ce9e0 msgr2=0x7efc240cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:55.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.730+0000 7efc2a1b5640 1 -- 172.21.15.105:0/3520889849 shutdown_connections 2023-11-28T18:24:55.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.730+0000 7efc2a1b5640 1 -- 172.21.15.105:0/3520889849 wait complete. 2023-11-28T18:24:55.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc2a1b5640 1 Processor -- start 2023-11-28T18:24:55.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc2a1b5640 1 -- start start 2023-11-28T18:24:55.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc2a1b5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc100047d0 0x7efc2413eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:55.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc2a1b5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc24143640 0x7efc24142450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:55.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc2a1b5640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc24142990 0x7efc2413c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:55.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc2a1b5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2413f060 con 0x7efc24143640 2023-11-28T18:24:55.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc2a1b5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc240ca9c0 con 0x7efc100047d0 2023-11-28T18:24:55.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc2a1b5640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2413db50 con 0x7efc24142990 2023-11-28T18:24:55.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc289b2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc24143640 0x7efc24142450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:55.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc289b2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc24143640 0x7efc24142450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36824/0 (socket says 172.21.15.105:36824) 2023-11-28T18:24:55.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc289b2640 1 -- 172.21.15.105:0/1598684756 learned_addr learned my addr 172.21.15.105:0/1598684756 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:55.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc299b4640 1 --2- 172.21.15.105:0/1598684756 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc24142990 0x7efc2413c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:55.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc291b3640 1 --2- 172.21.15.105:0/1598684756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc100047d0 0x7efc2413eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:55.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc289b2640 1 -- 172.21.15.105:0/1598684756 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc24142990 msgr2=0x7efc2413c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:55.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc289b2640 1 --2- 172.21.15.105:0/1598684756 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc24142990 0x7efc2413c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:55.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc289b2640 1 -- 172.21.15.105:0/1598684756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc100047d0 msgr2=0x7efc2413eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:55.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc289b2640 1 --2- 172.21.15.105:0/1598684756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc100047d0 0x7efc2413eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:55.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc289b2640 1 -- 172.21.15.105:0/1598684756 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc240ca400 con 0x7efc24143640 2023-11-28T18:24:55.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc289b2640 1 --2- 172.21.15.105:0/1598684756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc24143640 0x7efc24142450 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7efc1400c210 tx=0x7efc10005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:55.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc19ffb640 1 -- 172.21.15.105:0/1598684756 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc140049f0 con 0x7efc24143640 2023-11-28T18:24:55.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc19ffb640 1 -- 172.21.15.105:0/1598684756 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc140b6070 con 0x7efc24143640 2023-11-28T18:24:55.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc2a1b5640 1 -- 172.21.15.105:0/1598684756 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc240ca6e0 con 0x7efc24143640 2023-11-28T18:24:55.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc19ffb640 1 -- 172.21.15.105:0/1598684756 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc14004d70 con 0x7efc24143640 2023-11-28T18:24:55.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc2a1b5640 1 -- 172.21.15.105:0/1598684756 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2413c380 con 0x7efc24143640 2023-11-28T18:24:55.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.734+0000 7efc2a1b5640 1 -- 172.21.15.105:0/1598684756 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc24074580 con 0x7efc24143640 2023-11-28T18:24:55.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.738+0000 7efc19ffb640 1 -- 172.21.15.105:0/1598684756 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7efc140070d0 con 0x7efc24143640 2023-11-28T18:24:55.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.738+0000 7efc19ffb640 1 --2- 172.21.15.105:0/1598684756 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc08066900 0x7efc08068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:55.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.738+0000 7efc19ffb640 1 -- 172.21.15.105:0/1598684756 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5929+0+0 (secure 0 0 0) 0x7efc140b8070 con 0x7efc24143640 2023-11-28T18:24:55.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.738+0000 7efc291b3640 1 --2- 172.21.15.105:0/1598684756 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc08066900 0x7efc08068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:55.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.738+0000 7efc291b3640 1 --2- 172.21.15.105:0/1598684756 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc08066900 0x7efc08068dc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efc20009a80 tx=0x7efc20009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:55.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.742+0000 7efc19ffb640 1 -- 172.21.15.105:0/1598684756 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efc140ee760 con 0x7efc24143640 2023-11-28T18:24:55.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:55.930+0000 7efc2a1b5640 1 -- 172.21.15.105:0/1598684756 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7efc2406e3b0 con 0x7efc24143640 2023-11-28T18:24:56.576 INFO:teuthology.orchestra.run.smithi105.stderr:new fs with metadata pool 5 and data pool 6 2023-11-28T18:24:56.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.566+0000 7efc19ffb640 1 -- 172.21.15.105:0/1598684756 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7efc2406e3b0 con 0x7efc24143640 2023-11-28T18:24:56.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.578+0000 7efc2a1b5640 1 -- 172.21.15.105:0/1598684756 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc08066900 msgr2=0x7efc08068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:56.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.578+0000 7efc2a1b5640 1 --2- 172.21.15.105:0/1598684756 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc08066900 0x7efc08068dc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efc20009a80 tx=0x7efc20009190 comp rx=0 tx=0).stop 2023-11-28T18:24:56.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.578+0000 7efc2a1b5640 1 -- 172.21.15.105:0/1598684756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc24143640 msgr2=0x7efc24142450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:56.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.578+0000 7efc2a1b5640 1 --2- 172.21.15.105:0/1598684756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc24143640 0x7efc24142450 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7efc1400c210 tx=0x7efc10005800 comp rx=0 tx=0).stop 2023-11-28T18:24:56.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.578+0000 7efc2a1b5640 1 -- 172.21.15.105:0/1598684756 shutdown_connections 2023-11-28T18:24:56.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.578+0000 7efc2a1b5640 1 --2- 172.21.15.105:0/1598684756 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc24142990 0x7efc2413c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:56.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.578+0000 7efc2a1b5640 1 --2- 172.21.15.105:0/1598684756 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc08066900 0x7efc08068dc0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:56.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.578+0000 7efc2a1b5640 1 --2- 172.21.15.105:0/1598684756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc24143640 0x7efc24142450 secure :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7efc1400c210 tx=0x7efc10005800 comp rx=0 tx=0).stop 2023-11-28T18:24:56.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.578+0000 7efc2a1b5640 1 --2- 172.21.15.105:0/1598684756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc100047d0 0x7efc2413eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:56.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.578+0000 7efc2a1b5640 1 -- 172.21.15.105:0/1598684756 >> 172.21.15.105:0/1598684756 conn(0x7efc240ce9e0 msgr2=0x7efc240cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:56.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.578+0000 7efc2a1b5640 1 -- 172.21.15.105:0/1598684756 shutdown_connections 2023-11-28T18:24:56.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.582+0000 7efc2a1b5640 1 -- 172.21.15.105:0/1598684756 wait complete. 2023-11-28T18:24:56.618 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:24:56.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.754+0000 7f15283df640 1 Processor -- start 2023-11-28T18:24:56.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.754+0000 7f15283df640 1 -- start start 2023-11-28T18:24:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.754+0000 7f15283df640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f152015b030 0x7f15200709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.754+0000 7f15283df640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15200701d0 con 0x7f15200b4290 2023-11-28T18:24:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.754+0000 7f15283df640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f152006df80 con 0x7f1520071960 2023-11-28T18:24:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.754+0000 7f15283df640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1520071df0 con 0x7f152015b030 2023-11-28T18:24:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.754+0000 7f152617b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f15200b4290 0x7f15200b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54194/0 (socket says 172.21.15.105:54194) 2023-11-28T18:24:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.754+0000 7f152617b640 1 -- 172.21.15.105:0/2142772924 learned_addr learned my addr 172.21.15.105:0/2142772924 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.754+0000 7f152697c640 1 --2- 172.21.15.105:0/2142772924 >> v2:172.21.15.148:3300/0 conn(0x7f152015b030 0x7f15200709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.754+0000 7f1525179640 1 -- 172.21.15.105:0/2142772924 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2332228168 0 0) 0x7f15200701d0 con 0x7f15200b4290 2023-11-28T18:24:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.754+0000 7f1525179640 1 -- 172.21.15.105:0/2142772924 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f150c003530 con 0x7f15200b4290 2023-11-28T18:24:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.754+0000 7f152697c640 1 -- 172.21.15.105:0/2142772924 >> v1:172.21.15.148:6789/0 conn(0x7f1520071960 legacy=0x7f152015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:56.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.754+0000 7f152697c640 1 -- 172.21.15.105:0/2142772924 >> v1:172.21.15.105:6789/0 conn(0x7f15200b4290 legacy=0x7f15200b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:56.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.754+0000 7f1525179640 1 -- 172.21.15.105:0/2142772924 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2175813342 0 0) 0x7f150c003530 con 0x7f15200b4290 2023-11-28T18:24:56.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.754+0000 7f152697c640 1 -- 172.21.15.105:0/2142772924 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15200720d0 con 0x7f152015b030 2023-11-28T18:24:56.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.758+0000 7f152697c640 1 --2- 172.21.15.105:0/2142772924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f152015b030 0x7f15200709b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1518009d90 tx=0x7f151801c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=920716fad6d176e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:56.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.758+0000 7f1525179640 1 -- 172.21.15.105:0/2142772924 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c001680 con 0x7f152015b030 2023-11-28T18:24:56.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.758+0000 7f1525179640 1 -- 172.21.15.105:0/2142772924 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15180a5590 con 0x7f152015b030 2023-11-28T18:24:56.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.758+0000 7f1525179640 1 -- 172.21.15.105:0/2142772924 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15180a5900 con 0x7f152015b030 2023-11-28T18:24:56.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.758+0000 7f15283df640 1 -- 172.21.15.105:0/2142772924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f152015b030 msgr2=0x7f15200709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:56.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.758+0000 7f15283df640 1 --2- 172.21.15.105:0/2142772924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f152015b030 0x7f15200709b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1518009d90 tx=0x7f151801c840 comp rx=0 tx=0).stop 2023-11-28T18:24:56.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.758+0000 7f15283df640 1 -- 172.21.15.105:0/2142772924 shutdown_connections 2023-11-28T18:24:56.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.758+0000 7f15283df640 1 --2- 172.21.15.105:0/2142772924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f152015b030 0x7f15200709b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:56.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.758+0000 7f15283df640 1 -- 172.21.15.105:0/2142772924 >> 172.21.15.105:0/2142772924 conn(0x7f15200ceb90 msgr2=0x7f15200cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:56.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.758+0000 7f15283df640 1 -- 172.21.15.105:0/2142772924 shutdown_connections 2023-11-28T18:24:56.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.758+0000 7f15283df640 1 -- 172.21.15.105:0/2142772924 wait complete. 2023-11-28T18:24:56.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f15283df640 1 Processor -- start 2023-11-28T18:24:56.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f15283df640 1 -- start start 2023-11-28T18:24:56.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f15283df640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15200b4290 0x7f1520153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:56.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f15283df640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f152015b030 0x7f152009b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:56.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f15283df640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1520156100 0x7f15201565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:56.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f15283df640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152009af30 con 0x7f1520156100 2023-11-28T18:24:56.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f15283df640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1520156af0 con 0x7f152015b030 2023-11-28T18:24:56.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f15283df640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1520154200 con 0x7f15200b4290 2023-11-28T18:24:56.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f152697c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1520156100 0x7f15201565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:56.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f152697c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1520156100 0x7f15201565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36832/0 (socket says 172.21.15.105:36832) 2023-11-28T18:24:56.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f152697c640 1 -- 172.21.15.105:0/635083109 learned_addr learned my addr 172.21.15.105:0/635083109 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:56.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f152597a640 1 --2- 172.21.15.105:0/635083109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f152015b030 0x7f152009b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:56.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f152617b640 1 --2- 172.21.15.105:0/635083109 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15200b4290 0x7f1520153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:56.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f152697c640 1 -- 172.21.15.105:0/635083109 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15200b4290 msgr2=0x7f1520153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:56.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f152697c640 1 --2- 172.21.15.105:0/635083109 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15200b4290 0x7f1520153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:56.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f152697c640 1 -- 172.21.15.105:0/635083109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f152015b030 msgr2=0x7f152009b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:56.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f152697c640 1 --2- 172.21.15.105:0/635083109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f152015b030 0x7f152009b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:56.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f152697c640 1 -- 172.21.15.105:0/635083109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1518009a70 con 0x7f1520156100 2023-11-28T18:24:56.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f152597a640 1 --2- 172.21.15.105:0/635083109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f152015b030 0x7f152009b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:56.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f152617b640 1 --2- 172.21.15.105:0/635083109 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15200b4290 0x7f1520153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:56.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f152697c640 1 --2- 172.21.15.105:0/635083109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1520156100 0x7f15201565b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f151801a470 tx=0x7f15180195e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:56.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f1516ffd640 1 -- 172.21.15.105:0/635083109 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15180bd070 con 0x7f1520156100 2023-11-28T18:24:56.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f1516ffd640 1 -- 172.21.15.105:0/635083109 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15180a5910 con 0x7f1520156100 2023-11-28T18:24:56.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f15283df640 1 -- 172.21.15.105:0/635083109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152009b890 con 0x7f1520156100 2023-11-28T18:24:56.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f1516ffd640 1 -- 172.21.15.105:0/635083109 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15180a5ca0 con 0x7f1520156100 2023-11-28T18:24:56.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.762+0000 7f15283df640 1 -- 172.21.15.105:0/635083109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152009a590 con 0x7f1520156100 2023-11-28T18:24:56.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.766+0000 7f15283df640 1 -- 172.21.15.105:0/635083109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15200cb410 con 0x7f1520156100 2023-11-28T18:24:56.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.766+0000 7f1516ffd640 1 -- 172.21.15.105:0/635083109 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f1518004040 con 0x7f1520156100 2023-11-28T18:24:56.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.766+0000 7f1516ffd640 1 --2- 172.21.15.105:0/635083109 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15040668d0 0x7f1504068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:56.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.766+0000 7f1516ffd640 1 -- 172.21.15.105:0/635083109 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f151811ef60 con 0x7f1520156100 2023-11-28T18:24:56.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.766+0000 7f152617b640 1 --2- 172.21.15.105:0/635083109 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15040668d0 0x7f1504068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:56.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.766+0000 7f152617b640 1 --2- 172.21.15.105:0/635083109 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15040668d0 0x7f1504068d90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f151c0011f0 tx=0x7f151c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:56.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.770+0000 7f1516ffd640 1 -- 172.21.15.105:0/635083109 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f15180ec0e0 con 0x7f1520156100 2023-11-28T18:24:56.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.922+0000 7f15283df640 1 -- 172.21.15.105:0/635083109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f152009b070 con 0x7f1520156100 2023-11-28T18:24:56.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.922+0000 7f1516ffd640 1 -- 172.21.15.105:0/635083109 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16847 (secure 0 0 0) 0x7f152009b070 con 0x7f1520156100 2023-11-28T18:24:56.927 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:56.927 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":37,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:24:55.936103+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:24:53.582892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T18:24:54.921978+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:24:56.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.922+0000 7f15283df640 1 -- 172.21.15.105:0/635083109 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15040668d0 msgr2=0x7f1504068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:56.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.922+0000 7f15283df640 1 --2- 172.21.15.105:0/635083109 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15040668d0 0x7f1504068d90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f151c0011f0 tx=0x7f151c009040 comp rx=0 tx=0).stop 2023-11-28T18:24:56.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.922+0000 7f15283df640 1 -- 172.21.15.105:0/635083109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1520156100 msgr2=0x7f15201565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:56.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.922+0000 7f15283df640 1 --2- 172.21.15.105:0/635083109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1520156100 0x7f15201565b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f151801a470 tx=0x7f15180195e0 comp rx=0 tx=0).stop 2023-11-28T18:24:56.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.922+0000 7f15283df640 1 -- 172.21.15.105:0/635083109 shutdown_connections 2023-11-28T18:24:56.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.922+0000 7f15283df640 1 --2- 172.21.15.105:0/635083109 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15040668d0 0x7f1504068d90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:56.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.922+0000 7f15283df640 1 --2- 172.21.15.105:0/635083109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1520156100 0x7f15201565b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:56.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.922+0000 7f15283df640 1 --2- 172.21.15.105:0/635083109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f152015b030 0x7f152009b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:56.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.922+0000 7f15283df640 1 --2- 172.21.15.105:0/635083109 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15200b4290 0x7f1520153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:56.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.922+0000 7f15283df640 1 -- 172.21.15.105:0/635083109 >> 172.21.15.105:0/635083109 conn(0x7f15200ceb90 msgr2=0x7f152006c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:56.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.922+0000 7f15283df640 1 -- 172.21.15.105:0/635083109 shutdown_connections 2023-11-28T18:24:56.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:56.922+0000 7f15283df640 1 -- 172.21.15.105:0/635083109 wait complete. 2023-11-28T18:24:56.948 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:24:57.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.082+0000 7ff79f59e640 1 Processor -- start 2023-11-28T18:24:57.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.082+0000 7ff79f59e640 1 -- start start 2023-11-28T18:24:57.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.082+0000 7ff79f59e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff7a00cfb30 0x7ff7a0167fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:57.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.082+0000 7ff79f59e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a00ce840 con 0x7ff7a015d2d0 2023-11-28T18:24:57.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.082+0000 7ff79f59e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a00cd990 con 0x7ff7a015cef0 2023-11-28T18:24:57.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.082+0000 7ff79f59e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7a00b4780 con 0x7ff7a00cfb30 2023-11-28T18:24:57.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.082+0000 7ff79ed9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff7a00cfb30 0x7ff7a0167fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:57.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.082+0000 7ff79dd9b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff7a015d2d0 0x7ff7a00cf420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54204/0 (socket says 172.21.15.105:54204) 2023-11-28T18:24:57.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.082+0000 7ff79ed9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff7a00cfb30 0x7ff7a0167fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36842/0 (socket says 172.21.15.105:36842) 2023-11-28T18:24:57.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.082+0000 7ff79dd9b640 1 -- 172.21.15.105:0/369522473 learned_addr learned my addr 172.21.15.105:0/369522473 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:57.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.082+0000 7ff79ed9d640 1 -- 172.21.15.105:0/369522473 >> v1:172.21.15.148:6789/0 conn(0x7ff7a015cef0 legacy=0x7ff7a00d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:57.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.082+0000 7ff79ed9d640 1 -- 172.21.15.105:0/369522473 >> v1:172.21.15.105:6789/0 conn(0x7ff7a015d2d0 legacy=0x7ff7a00cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:57.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.082+0000 7ff79ed9d640 1 -- 172.21.15.105:0/369522473 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a00cecd0 con 0x7ff7a00cfb30 2023-11-28T18:24:57.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.086+0000 7ff79ed9d640 1 --2- 172.21.15.105:0/369522473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7a00cfb30 0x7ff7a0167fe0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff798009d90 tx=0x7ff79801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b747b016a9d774d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:57.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.086+0000 7ff79d59a640 1 -- 172.21.15.105:0/369522473 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7980a5910 con 0x7ff7a00cfb30 2023-11-28T18:24:57.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.086+0000 7ff79d59a640 1 -- 172.21.15.105:0/369522473 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7980a5a70 con 0x7ff7a00cfb30 2023-11-28T18:24:57.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.086+0000 7ff79d59a640 1 -- 172.21.15.105:0/369522473 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7980a5e00 con 0x7ff7a00cfb30 2023-11-28T18:24:57.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.086+0000 7ff79f59e640 1 -- 172.21.15.105:0/369522473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7a00cfb30 msgr2=0x7ff7a0167fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:57.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.086+0000 7ff79f59e640 1 --2- 172.21.15.105:0/369522473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7a00cfb30 0x7ff7a0167fe0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff798009d90 tx=0x7ff79801c850 comp rx=0 tx=0).stop 2023-11-28T18:24:57.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.086+0000 7ff79f59e640 1 -- 172.21.15.105:0/369522473 shutdown_connections 2023-11-28T18:24:57.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.086+0000 7ff79f59e640 1 --2- 172.21.15.105:0/369522473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7a00cfb30 0x7ff7a0167fe0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:57.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.086+0000 7ff79f59e640 1 -- 172.21.15.105:0/369522473 >> 172.21.15.105:0/369522473 conn(0x7ff7a00d2e10 msgr2=0x7ff7a00d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:57.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.086+0000 7ff79f59e640 1 -- 172.21.15.105:0/369522473 shutdown_connections 2023-11-28T18:24:57.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.086+0000 7ff79f59e640 1 -- 172.21.15.105:0/369522473 wait complete. 2023-11-28T18:24:57.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79f59e640 1 Processor -- start 2023-11-28T18:24:57.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79f59e640 1 -- start start 2023-11-28T18:24:57.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79f59e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7a00cfb30 0x7ff7a0153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:57.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79f59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7a015cef0 0x7ff7a015d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:57.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79f59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a015d2d0 0x7ff7a015ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:57.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79f59e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a01535b0 con 0x7ff7a015cef0 2023-11-28T18:24:57.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79f59e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a0155550 con 0x7ff7a015d2d0 2023-11-28T18:24:57.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79f59e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a015f810 con 0x7ff7a00cfb30 2023-11-28T18:24:57.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7a015cef0 0x7ff7a015d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:57.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7a015cef0 0x7ff7a015d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36858/0 (socket says 172.21.15.105:36858) 2023-11-28T18:24:57.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79dd9b640 1 -- 172.21.15.105:0/1532051871 learned_addr learned my addr 172.21.15.105:0/1532051871 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:57.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79e59c640 1 --2- 172.21.15.105:0/1532051871 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7a00cfb30 0x7ff7a0153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:57.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79ed9d640 1 --2- 172.21.15.105:0/1532051871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a015d2d0 0x7ff7a015ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:57.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79dd9b640 1 -- 172.21.15.105:0/1532051871 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7a00cfb30 msgr2=0x7ff7a0153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:57.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79dd9b640 1 --2- 172.21.15.105:0/1532051871 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7a00cfb30 0x7ff7a0153070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:57.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79dd9b640 1 -- 172.21.15.105:0/1532051871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a015d2d0 msgr2=0x7ff7a015ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:57.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79dd9b640 1 --2- 172.21.15.105:0/1532051871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a015d2d0 0x7ff7a015ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:57.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79dd9b640 1 -- 172.21.15.105:0/1532051871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff798009a70 con 0x7ff7a015cef0 2023-11-28T18:24:57.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79e59c640 1 --2- 172.21.15.105:0/1532051871 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7a00cfb30 0x7ff7a0153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:24:57.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79dd9b640 1 --2- 172.21.15.105:0/1532051871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7a015cef0 0x7ff7a015d890 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff79400c370 tx=0x7ff79400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:57.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff78effd640 1 -- 172.21.15.105:0/1532051871 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff794005cf0 con 0x7ff7a015cef0 2023-11-28T18:24:57.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff78effd640 1 -- 172.21.15.105:0/1532051871 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff794005e50 con 0x7ff7a015cef0 2023-11-28T18:24:57.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff78effd640 1 -- 172.21.15.105:0/1532051871 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7940072f0 con 0x7ff7a015cef0 2023-11-28T18:24:57.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79f59e640 1 -- 172.21.15.105:0/1532051871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a015faf0 con 0x7ff7a015cef0 2023-11-28T18:24:57.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79f59e640 1 -- 172.21.15.105:0/1532051871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a01600b0 con 0x7ff7a015cef0 2023-11-28T18:24:57.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.090+0000 7ff79f59e640 1 -- 172.21.15.105:0/1532051871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a01602a0 con 0x7ff7a015cef0 2023-11-28T18:24:57.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.094+0000 7ff78effd640 1 -- 172.21.15.105:0/1532051871 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff794007450 con 0x7ff7a015cef0 2023-11-28T18:24:57.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.094+0000 7ff78effd640 1 --2- 172.21.15.105:0/1532051871 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff7800668d0 0x7ff780068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:57.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.094+0000 7ff78effd640 1 -- 172.21.15.105:0/1532051871 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff794083ba0 con 0x7ff7a015cef0 2023-11-28T18:24:57.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.094+0000 7ff79e59c640 1 --2- 172.21.15.105:0/1532051871 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff7800668d0 0x7ff780068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:57.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.094+0000 7ff79e59c640 1 --2- 172.21.15.105:0/1532051871 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff7800668d0 0x7ff780068d90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff7900016e0 tx=0x7ff790009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:57.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.098+0000 7ff78effd640 1 -- 172.21.15.105:0/1532051871 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff794050c00 con 0x7ff7a015cef0 2023-11-28T18:24:57.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.246+0000 7ff79f59e640 1 -- 172.21.15.105:0/1532051871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7a00d03c0 con 0x7ff7a015cef0 2023-11-28T18:24:57.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.250+0000 7ff78effd640 1 -- 172.21.15.105:0/1532051871 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16847 (secure 0 0 0) 0x7ff794017020 con 0x7ff7a015cef0 2023-11-28T18:24:57.254 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:57.254 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":37,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:24:55.936103+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:24:53.582892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T18:24:54.921978+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:24:57.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.250+0000 7ff79f59e640 1 -- 172.21.15.105:0/1532051871 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff7800668d0 msgr2=0x7ff780068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:57.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.250+0000 7ff79f59e640 1 --2- 172.21.15.105:0/1532051871 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff7800668d0 0x7ff780068d90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff7900016e0 tx=0x7ff790009040 comp rx=0 tx=0).stop 2023-11-28T18:24:57.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.250+0000 7ff79f59e640 1 -- 172.21.15.105:0/1532051871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7a015cef0 msgr2=0x7ff7a015d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:57.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.250+0000 7ff79f59e640 1 --2- 172.21.15.105:0/1532051871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7a015cef0 0x7ff7a015d890 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff79400c370 tx=0x7ff79400c830 comp rx=0 tx=0).stop 2023-11-28T18:24:57.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.250+0000 7ff79f59e640 1 -- 172.21.15.105:0/1532051871 shutdown_connections 2023-11-28T18:24:57.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.250+0000 7ff79f59e640 1 --2- 172.21.15.105:0/1532051871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a015d2d0 0x7ff7a015ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:57.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.250+0000 7ff79f59e640 1 --2- 172.21.15.105:0/1532051871 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff7800668d0 0x7ff780068d90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:57.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.250+0000 7ff79f59e640 1 --2- 172.21.15.105:0/1532051871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7a015cef0 0x7ff7a015d890 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:57.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.250+0000 7ff79f59e640 1 --2- 172.21.15.105:0/1532051871 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7a00cfb30 0x7ff7a0153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:57.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.250+0000 7ff79f59e640 1 -- 172.21.15.105:0/1532051871 >> 172.21.15.105:0/1532051871 conn(0x7ff7a00d2e10 msgr2=0x7ff7a006c600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:57.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.250+0000 7ff79f59e640 1 -- 172.21.15.105:0/1532051871 shutdown_connections 2023-11-28T18:24:57.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.250+0000 7ff79f59e640 1 -- 172.21.15.105:0/1532051871 wait complete. 2023-11-28T18:24:57.275 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T18:24:57.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.410+0000 7feca6b2a640 1 Processor -- start 2023-11-28T18:24:57.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.414+0000 7feca6b2a640 1 -- start start 2023-11-28T18:24:57.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.414+0000 7feca6b2a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feca0071960 0x7feca015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:57.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.414+0000 7feca6b2a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feca00701d0 con 0x7feca015b030 2023-11-28T18:24:57.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.414+0000 7feca6b2a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feca006df80 con 0x7feca00b4290 2023-11-28T18:24:57.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.414+0000 7feca6b2a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feca0071df0 con 0x7feca0071960 2023-11-28T18:24:57.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.414+0000 7feca50c7640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7feca015b030 0x7feca00709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54210/0 (socket says 172.21.15.105:54210) 2023-11-28T18:24:57.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.414+0000 7feca50c7640 1 -- 172.21.15.105:0/1640371832 learned_addr learned my addr 172.21.15.105:0/1640371832 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:57.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.414+0000 7fec97fff640 1 --2- 172.21.15.105:0/1640371832 >> v2:172.21.15.148:3300/0 conn(0x7feca0071960 0x7feca015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:57.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.414+0000 7fec977fe640 1 -- 172.21.15.105:0/1640371832 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3215992148 0 0) 0x7feca00701d0 con 0x7feca015b030 2023-11-28T18:24:57.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.414+0000 7fec977fe640 1 -- 172.21.15.105:0/1640371832 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec88003530 con 0x7feca015b030 2023-11-28T18:24:57.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.414+0000 7fec977fe640 1 -- 172.21.15.105:0/1640371832 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2830059974 0 0) 0x7feca006df80 con 0x7feca00b4290 2023-11-28T18:24:57.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.414+0000 7fec977fe640 1 -- 172.21.15.105:0/1640371832 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feca00701d0 con 0x7feca00b4290 2023-11-28T18:24:57.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.414+0000 7fec97fff640 1 -- 172.21.15.105:0/1640371832 >> v1:172.21.15.148:6789/0 conn(0x7feca00b4290 legacy=0x7feca00b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:57.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.414+0000 7fec97fff640 1 -- 172.21.15.105:0/1640371832 >> v1:172.21.15.105:6789/0 conn(0x7feca015b030 legacy=0x7feca00709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:57.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.414+0000 7fec97fff640 1 -- 172.21.15.105:0/1640371832 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca00720d0 con 0x7feca0071960 2023-11-28T18:24:57.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.418+0000 7fec97fff640 1 --2- 172.21.15.105:0/1640371832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feca0071960 0x7feca015f3b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fec900097c0 tx=0x7fec9001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d14d8c3baf3062e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:57.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.418+0000 7fec977fe640 1 -- 172.21.15.105:0/1640371832 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec900ab650 con 0x7feca0071960 2023-11-28T18:24:57.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.418+0000 7fec977fe640 1 -- 172.21.15.105:0/1640371832 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec900ab7b0 con 0x7feca0071960 2023-11-28T18:24:57.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.418+0000 7fec977fe640 1 -- 172.21.15.105:0/1640371832 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec900abb40 con 0x7feca0071960 2023-11-28T18:24:57.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.418+0000 7feca6b2a640 1 -- 172.21.15.105:0/1640371832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feca0071960 msgr2=0x7feca015f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:57.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.418+0000 7feca6b2a640 1 --2- 172.21.15.105:0/1640371832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feca0071960 0x7feca015f3b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fec900097c0 tx=0x7fec9001c450 comp rx=0 tx=0).stop 2023-11-28T18:24:57.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.418+0000 7feca6b2a640 1 -- 172.21.15.105:0/1640371832 shutdown_connections 2023-11-28T18:24:57.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.418+0000 7feca6b2a640 1 --2- 172.21.15.105:0/1640371832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feca0071960 0x7feca015f3b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:57.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.418+0000 7feca6b2a640 1 -- 172.21.15.105:0/1640371832 >> 172.21.15.105:0/1640371832 conn(0x7feca00ceb90 msgr2=0x7feca00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:57.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.418+0000 7feca6b2a640 1 -- 172.21.15.105:0/1640371832 shutdown_connections 2023-11-28T18:24:57.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.418+0000 7feca6b2a640 1 -- 172.21.15.105:0/1640371832 wait complete. 2023-11-28T18:24:57.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.418+0000 7feca6b2a640 1 Processor -- start 2023-11-28T18:24:57.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7feca6b2a640 1 -- start start 2023-11-28T18:24:57.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7feca6b2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feca00b4290 0x7feca014ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:57.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7feca6b2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feca015b030 0x7feca00b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:57.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7feca6b2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feca014d380 0x7feca00b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:57.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7feca6b2a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca00b2ae0 con 0x7feca015b030 2023-11-28T18:24:57.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7feca6b2a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca014d200 con 0x7feca00b4290 2023-11-28T18:24:57.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7feca6b2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca0152c50 con 0x7feca014d380 2023-11-28T18:24:57.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7fec97fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feca015b030 0x7feca00b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:57.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7fec97fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feca015b030 0x7feca00b1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36874/0 (socket says 172.21.15.105:36874) 2023-11-28T18:24:57.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7fec97fff640 1 -- 172.21.15.105:0/2800183579 learned_addr learned my addr 172.21.15.105:0/2800183579 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:57.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7feca48c6640 1 --2- 172.21.15.105:0/2800183579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feca00b4290 0x7feca014ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:57.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7feca50c7640 1 --2- 172.21.15.105:0/2800183579 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feca014d380 0x7feca00b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:57.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7fec97fff640 1 -- 172.21.15.105:0/2800183579 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feca014d380 msgr2=0x7feca00b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:57.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7fec97fff640 1 --2- 172.21.15.105:0/2800183579 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feca014d380 0x7feca00b22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:57.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7fec97fff640 1 -- 172.21.15.105:0/2800183579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feca00b4290 msgr2=0x7feca014ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:57.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7fec97fff640 1 --2- 172.21.15.105:0/2800183579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feca00b4290 0x7feca014ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:57.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7fec97fff640 1 -- 172.21.15.105:0/2800183579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec90009420 con 0x7feca015b030 2023-11-28T18:24:57.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7fec97fff640 1 --2- 172.21.15.105:0/2800183579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feca015b030 0x7feca00b1d70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fec900ad040 tx=0x7fec9001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:57.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7fec957fa640 1 -- 172.21.15.105:0/2800183579 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec900bf610 con 0x7feca015b030 2023-11-28T18:24:57.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7fec957fa640 1 -- 172.21.15.105:0/2800183579 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec900bf790 con 0x7feca015b030 2023-11-28T18:24:57.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7fec957fa640 1 -- 172.21.15.105:0/2800183579 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec900bfb40 con 0x7feca015b030 2023-11-28T18:24:57.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7feca6b2a640 1 -- 172.21.15.105:0/2800183579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feca0152ea0 con 0x7feca015b030 2023-11-28T18:24:57.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7feca6b2a640 1 -- 172.21.15.105:0/2800183579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feca0153460 con 0x7feca015b030 2023-11-28T18:24:57.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.422+0000 7feca6b2a640 1 -- 172.21.15.105:0/2800183579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feca0067570 con 0x7feca015b030 2023-11-28T18:24:57.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.426+0000 7fec957fa640 1 -- 172.21.15.105:0/2800183579 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fec900b5400 con 0x7feca015b030 2023-11-28T18:24:57.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.426+0000 7fec957fa640 1 --2- 172.21.15.105:0/2800183579 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec8c0668d0 0x7fec8c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:57.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.426+0000 7fec957fa640 1 -- 172.21.15.105:0/2800183579 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fec9011f0a0 con 0x7feca015b030 2023-11-28T18:24:57.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.426+0000 7feca48c6640 1 --2- 172.21.15.105:0/2800183579 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec8c0668d0 0x7fec8c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:57.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.426+0000 7feca48c6640 1 --2- 172.21.15.105:0/2800183579 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec8c0668d0 0x7fec8c068d90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0011f0 tx=0x7fec9c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:57.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.430+0000 7fec957fa640 1 -- 172.21.15.105:0/2800183579 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fec900ec260 con 0x7feca015b030 2023-11-28T18:24:57.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:57.614+0000 7feca6b2a640 1 -- 172.21.15.105:0/2800183579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7feca00cf7f0 con 0x7feca015b030 2023-11-28T18:24:58.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.578+0000 7fec957fa640 1 -- 172.21.15.105:0/2800183579 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7fec900ef880 con 0x7feca015b030 2023-11-28T18:24:58.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.582+0000 7feca6b2a640 1 -- 172.21.15.105:0/2800183579 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec8c0668d0 msgr2=0x7fec8c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:58.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.582+0000 7feca6b2a640 1 --2- 172.21.15.105:0/2800183579 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec8c0668d0 0x7fec8c068d90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0011f0 tx=0x7fec9c009040 comp rx=0 tx=0).stop 2023-11-28T18:24:58.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.582+0000 7feca6b2a640 1 -- 172.21.15.105:0/2800183579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feca015b030 msgr2=0x7feca00b1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:58.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.582+0000 7feca6b2a640 1 --2- 172.21.15.105:0/2800183579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feca015b030 0x7feca00b1d70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fec900ad040 tx=0x7fec9001ac90 comp rx=0 tx=0).stop 2023-11-28T18:24:58.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.582+0000 7feca6b2a640 1 -- 172.21.15.105:0/2800183579 shutdown_connections 2023-11-28T18:24:58.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.582+0000 7feca6b2a640 1 --2- 172.21.15.105:0/2800183579 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feca014d380 0x7feca00b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:58.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.582+0000 7feca6b2a640 1 --2- 172.21.15.105:0/2800183579 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec8c0668d0 0x7fec8c068d90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:58.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.582+0000 7feca6b2a640 1 --2- 172.21.15.105:0/2800183579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feca015b030 0x7feca00b1d70 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:58.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.582+0000 7feca6b2a640 1 --2- 172.21.15.105:0/2800183579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feca00b4290 0x7feca014ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:58.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.582+0000 7feca6b2a640 1 -- 172.21.15.105:0/2800183579 >> 172.21.15.105:0/2800183579 conn(0x7feca00ceb90 msgr2=0x7feca0157ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:58.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.582+0000 7feca6b2a640 1 -- 172.21.15.105:0/2800183579 shutdown_connections 2023-11-28T18:24:58.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.582+0000 7feca6b2a640 1 -- 172.21.15.105:0/2800183579 wait complete. 2023-11-28T18:24:58.611 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T18:24:58.611 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T18:24:58.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.750+0000 7f13215d1640 1 Processor -- start 2023-11-28T18:24:58.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.750+0000 7f13215d1640 1 -- start start 2023-11-28T18:24:58.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.750+0000 7f13215d1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f131c071960 0x7f131c15f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:58.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.750+0000 7f13215d1640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f131c15b030 0x7f131c0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:58.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.750+0000 7f13215d1640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f131c0701d0 con 0x7f131c0b4290 2023-11-28T18:24:58.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.750+0000 7f13215d1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f131c06df80 con 0x7f131c071960 2023-11-28T18:24:58.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.750+0000 7f13215d1640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131c071df0 con 0x7f131c15b030 2023-11-28T18:24:58.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131affd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f131c0b4290 0x7f131c0b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54222/0 (socket says 172.21.15.105:54222) 2023-11-28T18:24:58.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131affd640 1 -- 172.21.15.105:0/223940450 learned_addr learned my addr 172.21.15.105:0/223940450 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:24:58.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131a7fc640 1 -- 172.21.15.105:0/223940450 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 939747903 0 0) 0x7f131c0701d0 con 0x7f131c0b4290 2023-11-28T18:24:58.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131a7fc640 1 -- 172.21.15.105:0/223940450 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1308003530 con 0x7f131c0b4290 2023-11-28T18:24:58.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131b7fe640 1 --2- 172.21.15.105:0/223940450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f131c15b030 0x7f131c0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:58.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f13127fc640 1 --2- 172.21.15.105:0/223940450 >> v2:172.21.15.148:3300/0 conn(0x7f131c071960 0x7f131c15f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:58.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131a7fc640 1 -- 172.21.15.105:0/223940450 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3008657239 0 0) 0x7f1308003530 con 0x7f131c0b4290 2023-11-28T18:24:58.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131a7fc640 1 -- 172.21.15.105:0/223940450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f131c15b030 msgr2=0x7f131c0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:58.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131a7fc640 1 --2- 172.21.15.105:0/223940450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f131c15b030 0x7f131c0709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:58.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131a7fc640 1 -- 172.21.15.105:0/223940450 >> v2:172.21.15.148:3300/0 conn(0x7f131c071960 msgr2=0x7f131c15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:58.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131a7fc640 1 --2- 172.21.15.105:0/223940450 >> v2:172.21.15.148:3300/0 conn(0x7f131c071960 0x7f131c15f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:58.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131a7fc640 1 -- 172.21.15.105:0/223940450 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131c0720d0 con 0x7f131c0b4290 2023-11-28T18:24:58.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131a7fc640 1 -- 172.21.15.105:0/223940450 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f130c001a00 con 0x7f131c0b4290 2023-11-28T18:24:58.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131a7fc640 1 -- 172.21.15.105:0/223940450 >> v1:172.21.15.105:6789/0 conn(0x7f131c0b4290 legacy=0x7f131c0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:58.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131a7fc640 1 --2- 172.21.15.105:0/223940450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f131c15b030 0x7f1308003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:58.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131a7fc640 1 --2- 172.21.15.105:0/223940450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1308003f50 0x7f13080047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:58.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131a7fc640 1 --2- 172.21.15.105:0/223940450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1308004d10 0x7f1308005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:58.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131a7fc640 1 -- 172.21.15.105:0/223940450 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c06df80 con 0x7f1308004d10 2023-11-28T18:24:58.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131a7fc640 1 -- 172.21.15.105:0/223940450 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c071df0 con 0x7f1308003f50 2023-11-28T18:24:58.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131a7fc640 1 -- 172.21.15.105:0/223940450 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130800b990 con 0x7f131c15b030 2023-11-28T18:24:58.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131b7fe640 1 --2- 172.21.15.105:0/223940450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1308004d10 0x7f1308005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:58.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131affd640 1 --2- 172.21.15.105:0/223940450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f131c15b030 0x7f1308003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:58.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f13127fc640 1 --2- 172.21.15.105:0/223940450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1308003f50 0x7f13080047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:58.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131b7fe640 1 -- 172.21.15.105:0/223940450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f131c15b030 msgr2=0x7f1308003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:58.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131b7fe640 1 --2- 172.21.15.105:0/223940450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f131c15b030 0x7f1308003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:58.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131b7fe640 1 -- 172.21.15.105:0/223940450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1308003f50 msgr2=0x7f13080047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:58.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131b7fe640 1 --2- 172.21.15.105:0/223940450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1308003f50 0x7f13080047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:58.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131b7fe640 1 -- 172.21.15.105:0/223940450 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1308003530 con 0x7f1308004d10 2023-11-28T18:24:58.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.754+0000 7f131affd640 1 --2- 172.21.15.105:0/223940450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f131c15b030 0x7f1308003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:58.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.758+0000 7f131b7fe640 1 --2- 172.21.15.105:0/223940450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1308004d10 0x7f1308005190 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1314009ae0 tx=0x7f131401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814acbf225fe1a1e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:58.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.758+0000 7f131a7fc640 1 -- 172.21.15.105:0/223940450 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13140a5600 con 0x7f1308004d10 2023-11-28T18:24:58.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.758+0000 7f131a7fc640 1 -- 172.21.15.105:0/223940450 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13140a5760 con 0x7f1308004d10 2023-11-28T18:24:58.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.758+0000 7f131a7fc640 1 -- 172.21.15.105:0/223940450 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13140a5b10 con 0x7f1308004d10 2023-11-28T18:24:58.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.758+0000 7f13215d1640 1 -- 172.21.15.105:0/223940450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1308004d10 msgr2=0x7f1308005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:58.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.758+0000 7f13215d1640 1 --2- 172.21.15.105:0/223940450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1308004d10 0x7f1308005190 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1314009ae0 tx=0x7f131401ca10 comp rx=0 tx=0).stop 2023-11-28T18:24:58.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.758+0000 7f131b7fe640 1 -- 172.21.15.105:0/223940450 reap_dead start 2023-11-28T18:24:58.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.758+0000 7f13215d1640 1 -- 172.21.15.105:0/223940450 shutdown_connections 2023-11-28T18:24:58.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.758+0000 7f13215d1640 1 -- 172.21.15.105:0/223940450 >> 172.21.15.105:0/223940450 conn(0x7f131c0ceb90 msgr2=0x7f131c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:58.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.758+0000 7f13215d1640 1 -- 172.21.15.105:0/223940450 shutdown_connections 2023-11-28T18:24:58.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.758+0000 7f13215d1640 1 -- 172.21.15.105:0/223940450 wait complete. 2023-11-28T18:24:58.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.758+0000 7f13215d1640 1 Processor -- start 2023-11-28T18:24:58.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.758+0000 7f13215d1640 1 -- start start 2023-11-28T18:24:58.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f13215d1640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1308004d10 0x7f131c06c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:58.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f13215d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f131c150480 0x7f131c09b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:58.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f13215d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f131c1533c0 0x7f131c153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:58.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f13215d1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c0ccb60 con 0x7f131c1533c0 2023-11-28T18:24:58.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f13215d1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c150900 con 0x7f131c150480 2023-11-28T18:24:58.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f13215d1640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131c153d80 con 0x7f1308004d10 2023-11-28T18:24:58.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f131b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f131c1533c0 0x7f131c153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:58.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f131b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f131c1533c0 0x7f131c153840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36890/0 (socket says 172.21.15.105:36890) 2023-11-28T18:24:58.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f131b7fe640 1 -- 172.21.15.105:0/4220586898 learned_addr learned my addr 172.21.15.105:0/4220586898 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:58.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f13127fc640 1 --2- 172.21.15.105:0/4220586898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f131c150480 0x7f131c09b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:58.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f131affd640 1 --2- 172.21.15.105:0/4220586898 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1308004d10 0x7f131c06c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:58.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f131b7fe640 1 -- 172.21.15.105:0/4220586898 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1308004d10 msgr2=0x7f131c06c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:58.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f131b7fe640 1 --2- 172.21.15.105:0/4220586898 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1308004d10 0x7f131c06c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:58.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f131b7fe640 1 -- 172.21.15.105:0/4220586898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f131c150480 msgr2=0x7f131c09b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:58.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f131b7fe640 1 --2- 172.21.15.105:0/4220586898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f131c150480 0x7f131c09b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:58.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f131b7fe640 1 -- 172.21.15.105:0/4220586898 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131c09c1f0 con 0x7f131c1533c0 2023-11-28T18:24:58.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f131affd640 1 --2- 172.21.15.105:0/4220586898 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1308004d10 0x7f131c06c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:24:58.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f131b7fe640 1 --2- 172.21.15.105:0/4220586898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f131c1533c0 0x7f131c153840 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f13140b0040 tx=0x7f1308005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:58.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f1313fff640 1 -- 172.21.15.105:0/4220586898 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13140a5600 con 0x7f131c1533c0 2023-11-28T18:24:58.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f1313fff640 1 -- 172.21.15.105:0/4220586898 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1314004230 con 0x7f131c1533c0 2023-11-28T18:24:58.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f13215d1640 1 -- 172.21.15.105:0/4220586898 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131c09c4d0 con 0x7f131c1533c0 2023-11-28T18:24:58.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f1313fff640 1 -- 172.21.15.105:0/4220586898 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1314003620 con 0x7f131c1533c0 2023-11-28T18:24:58.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f13215d1640 1 -- 172.21.15.105:0/4220586898 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f131c0cc780 con 0x7f131c1533c0 2023-11-28T18:24:58.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f13215d1640 1 -- 172.21.15.105:0/4220586898 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f131c0ce830 con 0x7f131c1533c0 2023-11-28T18:24:58.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.762+0000 7f1313fff640 1 -- 172.21.15.105:0/4220586898 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f13140b4070 con 0x7f131c1533c0 2023-11-28T18:24:58.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.766+0000 7f1313fff640 1 --2- 172.21.15.105:0/4220586898 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f12fc0668c0 0x7f12fc068d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:58.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.766+0000 7f1313fff640 1 -- 172.21.15.105:0/4220586898 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f131411dd30 con 0x7f131c1533c0 2023-11-28T18:24:58.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.766+0000 7f131affd640 1 --2- 172.21.15.105:0/4220586898 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f12fc0668c0 0x7f12fc068d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:58.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.766+0000 7f131affd640 1 --2- 172.21.15.105:0/4220586898 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f12fc0668c0 0x7f12fc068d80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f130c001340 tx=0x7f130c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:58.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.770+0000 7f1313fff640 1 -- 172.21.15.105:0/4220586898 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f13140eadb0 con 0x7f131c1533c0 2023-11-28T18:24:58.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:58.946+0000 7f13215d1640 1 -- 172.21.15.105:0/4220586898 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f131c067570 con 0x7f131c1533c0 2023-11-28T18:24:59.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.586+0000 7f1313fff640 1 -- 172.21.15.105:0/4220586898 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f13140b20c0 con 0x7f131c1533c0 2023-11-28T18:24:59.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.590+0000 7f13215d1640 1 -- 172.21.15.105:0/4220586898 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f12fc0668c0 msgr2=0x7f12fc068d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:59.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.590+0000 7f13215d1640 1 --2- 172.21.15.105:0/4220586898 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f12fc0668c0 0x7f12fc068d80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f130c001340 tx=0x7f130c009040 comp rx=0 tx=0).stop 2023-11-28T18:24:59.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.590+0000 7f13215d1640 1 -- 172.21.15.105:0/4220586898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f131c1533c0 msgr2=0x7f131c153840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:59.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.590+0000 7f13215d1640 1 --2- 172.21.15.105:0/4220586898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f131c1533c0 0x7f131c153840 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f13140b0040 tx=0x7f1308005800 comp rx=0 tx=0).stop 2023-11-28T18:24:59.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.590+0000 7f13215d1640 1 -- 172.21.15.105:0/4220586898 shutdown_connections 2023-11-28T18:24:59.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.590+0000 7f13215d1640 1 --2- 172.21.15.105:0/4220586898 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f12fc0668c0 0x7f12fc068d80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:59.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.590+0000 7f13215d1640 1 --2- 172.21.15.105:0/4220586898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f131c1533c0 0x7f131c153840 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:59.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.590+0000 7f13215d1640 1 --2- 172.21.15.105:0/4220586898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f131c150480 0x7f131c09b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:59.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.590+0000 7f13215d1640 1 --2- 172.21.15.105:0/4220586898 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1308004d10 0x7f131c06c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.590+0000 7f13215d1640 1 -- 172.21.15.105:0/4220586898 >> 172.21.15.105:0/4220586898 conn(0x7f131c0ceb90 msgr2=0x7f131c0af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.590+0000 7f13215d1640 1 -- 172.21.15.105:0/4220586898 shutdown_connections 2023-11-28T18:24:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.590+0000 7f13215d1640 1 -- 172.21.15.105:0/4220586898 wait complete. 2023-11-28T18:24:59.614 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T18:24:59.614 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:24:59.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.746+0000 7f382b8d6640 1 Processor -- start 2023-11-28T18:24:59.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.746+0000 7f382b8d6640 1 -- start start 2023-11-28T18:24:59.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.750+0000 7f382b8d6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3824159f40 0x7f38240caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:59.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.750+0000 7f382b8d6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f382406bf50 con 0x7f38240ad850 2023-11-28T18:24:59.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.750+0000 7f382b8d6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38240ccbc0 con 0x7f38240cc760 2023-11-28T18:24:59.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.750+0000 7f382b8d6640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f382415a320 con 0x7f3824159f40 2023-11-28T18:24:59.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.750+0000 7f3829672640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3824159f40 0x7f38240caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:59.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.750+0000 7f3829672640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3824159f40 0x7f38240caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36902/0 (socket says 172.21.15.105:36902) 2023-11-28T18:24:59.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.750+0000 7f3829e73640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f38240ad850 0x7f38240701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54232/0 (socket says 172.21.15.105:54232) 2023-11-28T18:24:59.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.750+0000 7f3829672640 1 -- 172.21.15.105:0/2401315437 learned_addr learned my addr 172.21.15.105:0/2401315437 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:59.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.750+0000 7f3829672640 1 -- 172.21.15.105:0/2401315437 >> v1:172.21.15.148:6789/0 conn(0x7f38240cc760 legacy=0x7f38240ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:24:59.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.750+0000 7f3829672640 1 -- 172.21.15.105:0/2401315437 >> v1:172.21.15.105:6789/0 conn(0x7f38240ad850 legacy=0x7f38240701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:59.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.750+0000 7f381bfff640 1 -- 172.21.15.105:0/2401315437 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2985706345 0 0) 0x7f382406bf50 con 0x7f38240ad850 2023-11-28T18:24:59.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.750+0000 7f3829672640 1 -- 172.21.15.105:0/2401315437 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3824071380 con 0x7f3824159f40 2023-11-28T18:24:59.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.754+0000 7f3829672640 1 --2- 172.21.15.105:0/2401315437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3824159f40 0x7f38240caf60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3820009870 tx=0x7f382001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0ba36cc401892c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:59.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.754+0000 7f381bfff640 1 -- 172.21.15.105:0/2401315437 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38200ab640 con 0x7f3824159f40 2023-11-28T18:24:59.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.754+0000 7f381bfff640 1 -- 172.21.15.105:0/2401315437 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38200a7070 con 0x7f3824159f40 2023-11-28T18:24:59.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.754+0000 7f381bfff640 1 -- 172.21.15.105:0/2401315437 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38200ab9c0 con 0x7f3824159f40 2023-11-28T18:24:59.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.754+0000 7f382b8d6640 1 -- 172.21.15.105:0/2401315437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3824159f40 msgr2=0x7f38240caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:59.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.754+0000 7f382b8d6640 1 --2- 172.21.15.105:0/2401315437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3824159f40 0x7f38240caf60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3820009870 tx=0x7f382001c440 comp rx=0 tx=0).stop 2023-11-28T18:24:59.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.754+0000 7f382b8d6640 1 -- 172.21.15.105:0/2401315437 shutdown_connections 2023-11-28T18:24:59.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.754+0000 7f382b8d6640 1 --2- 172.21.15.105:0/2401315437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3824159f40 0x7f38240caf60 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:59.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.754+0000 7f382b8d6640 1 -- 172.21.15.105:0/2401315437 >> 172.21.15.105:0/2401315437 conn(0x7f38240ceb90 msgr2=0x7f38240cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:59.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.754+0000 7f382b8d6640 1 -- 172.21.15.105:0/2401315437 shutdown_connections 2023-11-28T18:24:59.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.754+0000 7f382b8d6640 1 -- 172.21.15.105:0/2401315437 wait complete. 2023-11-28T18:24:59.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.754+0000 7f382b8d6640 1 Processor -- start 2023-11-28T18:24:59.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.754+0000 7f382b8d6640 1 -- start start 2023-11-28T18:24:59.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.758+0000 7f382b8d6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38240ad850 0x7f382414c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:59.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.758+0000 7f382b8d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38240cc760 0x7f38240b4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:59.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.758+0000 7f382b8d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3824159f40 0x7f38240b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:59.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.758+0000 7f382b8d6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38240b20b0 con 0x7f38240ad850 2023-11-28T18:24:59.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.758+0000 7f382b8d6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382414c790 con 0x7f38240cc760 2023-11-28T18:24:59.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.758+0000 7f382b8d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38240b4160 con 0x7f3824159f40 2023-11-28T18:24:59.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.758+0000 7f3829672640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38240ad850 0x7f382414c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:59.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.758+0000 7f3829672640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38240ad850 0x7f382414c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36914/0 (socket says 172.21.15.105:36914) 2023-11-28T18:24:59.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.758+0000 7f3829672640 1 -- 172.21.15.105:0/4054057219 learned_addr learned my addr 172.21.15.105:0/4054057219 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:24:59.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.758+0000 7f3829672640 1 -- 172.21.15.105:0/4054057219 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3824159f40 msgr2=0x7f38240b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:59.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.758+0000 7f3829672640 1 --2- 172.21.15.105:0/4054057219 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3824159f40 0x7f38240b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:59.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.758+0000 7f3829672640 1 -- 172.21.15.105:0/4054057219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38240cc760 msgr2=0x7f38240b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:59.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.758+0000 7f3829672640 1 --2- 172.21.15.105:0/4054057219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38240cc760 0x7f38240b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:59.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.758+0000 7f3829672640 1 -- 172.21.15.105:0/4054057219 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38200094d0 con 0x7f38240ad850 2023-11-28T18:24:59.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.758+0000 7f3829672640 1 --2- 172.21.15.105:0/4054057219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38240ad850 0x7f382414c990 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f382001a070 tx=0x7f382001ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:59.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.758+0000 7f3819ffb640 1 -- 172.21.15.105:0/4054057219 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38200be5f0 con 0x7f38240ad850 2023-11-28T18:24:59.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.758+0000 7f3819ffb640 1 -- 172.21.15.105:0/4054057219 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38200be770 con 0x7f38240ad850 2023-11-28T18:24:59.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.758+0000 7f382b8d6640 1 -- 172.21.15.105:0/4054057219 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f382414fc90 con 0x7f38240ad850 2023-11-28T18:24:59.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.758+0000 7f3819ffb640 1 -- 172.21.15.105:0/4054057219 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38200beb20 con 0x7f38240ad850 2023-11-28T18:24:59.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.758+0000 7f382b8d6640 1 -- 172.21.15.105:0/4054057219 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3824150250 con 0x7f38240ad850 2023-11-28T18:24:59.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.758+0000 7f382b8d6640 1 -- 172.21.15.105:0/4054057219 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3824067570 con 0x7f38240ad850 2023-11-28T18:24:59.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.758+0000 7f3819ffb640 1 -- 172.21.15.105:0/4054057219 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f38200b43f0 con 0x7f38240ad850 2023-11-28T18:24:59.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.762+0000 7f3819ffb640 1 --2- 172.21.15.105:0/4054057219 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f38080668d0 0x7f3808068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:24:59.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.762+0000 7f3819ffb640 1 -- 172.21.15.105:0/4054057219 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f382011f340 con 0x7f38240ad850 2023-11-28T18:24:59.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.762+0000 7f3828e71640 1 --2- 172.21.15.105:0/4054057219 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f38080668d0 0x7f3808068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:24:59.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.762+0000 7f3828e71640 1 --2- 172.21.15.105:0/4054057219 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f38080668d0 0x7f3808068d90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f38240736d0 tx=0x7f3814006bd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:24:59.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.766+0000 7f3819ffb640 1 -- 172.21.15.105:0/4054057219 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f38200ec430 con 0x7f38240ad850 2023-11-28T18:24:59.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.946+0000 7f382b8d6640 1 -- 172.21.15.105:0/4054057219 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38240cf950 con 0x7f38240ad850 2023-11-28T18:24:59.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.946+0000 7f3819ffb640 1 -- 172.21.15.105:0/4054057219 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f38200efa50 con 0x7f38240ad850 2023-11-28T18:24:59.950 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:24:59.950 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/47307750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":47307750},{"type":"v1","addr":"172.21.15.148:6834","nonce":47307750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5392,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2878687703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2878687703},{"type":"v1","addr":"172.21.15.148:6835","nonce":2878687703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/681913204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":681913204},{"type":"v1","addr":"172.21.15.105:6836","nonce":681913204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:24:55.936428+0000","modified":"2023-11-28T18:24:58.953737+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5469":{"gid":5469,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/1876408690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":1876408690},{"type":"v1","addr":"172.21.15.105:6837","nonce":1876408690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T18:24:59.950 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 18 2023-11-28T18:24:59.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.946+0000 7f382b8d6640 1 -- 172.21.15.105:0/4054057219 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f38080668d0 msgr2=0x7f3808068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:59.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.946+0000 7f382b8d6640 1 --2- 172.21.15.105:0/4054057219 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f38080668d0 0x7f3808068d90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f38240736d0 tx=0x7f3814006bd0 comp rx=0 tx=0).stop 2023-11-28T18:24:59.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.946+0000 7f382b8d6640 1 -- 172.21.15.105:0/4054057219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38240ad850 msgr2=0x7f382414c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:24:59.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.946+0000 7f382b8d6640 1 --2- 172.21.15.105:0/4054057219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38240ad850 0x7f382414c990 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f382001a070 tx=0x7f382001ac80 comp rx=0 tx=0).stop 2023-11-28T18:24:59.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.946+0000 7f382b8d6640 1 -- 172.21.15.105:0/4054057219 shutdown_connections 2023-11-28T18:24:59.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.946+0000 7f382b8d6640 1 --2- 172.21.15.105:0/4054057219 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3824159f40 0x7f38240b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:59.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.946+0000 7f382b8d6640 1 --2- 172.21.15.105:0/4054057219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38240cc760 0x7f38240b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.946+0000 7f382b8d6640 1 --2- 172.21.15.105:0/4054057219 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f38080668d0 0x7f3808068d90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.946+0000 7f382b8d6640 1 --2- 172.21.15.105:0/4054057219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38240ad850 0x7f382414c990 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:24:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.946+0000 7f382b8d6640 1 -- 172.21.15.105:0/4054057219 >> 172.21.15.105:0/4054057219 conn(0x7f38240ceb90 msgr2=0x7f382406d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:24:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.946+0000 7f382b8d6640 1 -- 172.21.15.105:0/4054057219 shutdown_connections 2023-11-28T18:24:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:24:59.946+0000 7f382b8d6640 1 -- 172.21.15.105:0/4054057219 wait complete. 2023-11-28T18:24:59.971 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:25:00.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e469c640 1 Processor -- start 2023-11-28T18:25:00.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e469c640 1 -- start start 2023-11-28T18:25:00.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e469c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50dc15d4f0 0x7f50dc0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e469c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f50dc0d3c30 0x7f50dc0d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e469c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50dc0b25b0 con 0x7f50dc0d1890 2023-11-28T18:25:00.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e469c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50dc0b2900 con 0x7f50dc0d3c30 2023-11-28T18:25:00.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e469c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc0cea30 con 0x7f50dc15d4f0 2023-11-28T18:25:00.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e2c39640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f50dc0d1890 0x7f50dc0cfa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54246/0 (socket says 172.21.15.105:54246) 2023-11-28T18:25:00.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e2c39640 1 -- 172.21.15.105:0/1983329829 learned_addr learned my addr 172.21.15.105:0/1983329829 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:00.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e1c37640 1 --2- 172.21.15.105:0/1983329829 >> v2:172.21.15.148:3300/0 conn(0x7f50dc0d3c30 0x7f50dc0d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e2438640 1 --2- 172.21.15.105:0/1983329829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50dc15d4f0 0x7f50dc0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e1436640 1 -- 172.21.15.105:0/1983329829 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 141984962 0 0) 0x7f50dc0b25b0 con 0x7f50dc0d1890 2023-11-28T18:25:00.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e1436640 1 -- 172.21.15.105:0/1983329829 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50c8003530 con 0x7f50dc0d1890 2023-11-28T18:25:00.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e1436640 1 -- 172.21.15.105:0/1983329829 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1213955653 0 0) 0x7f50c8003530 con 0x7f50dc0d1890 2023-11-28T18:25:00.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e1436640 1 -- 172.21.15.105:0/1983329829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50dc15d4f0 msgr2=0x7f50dc0d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:00.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e1436640 1 --2- 172.21.15.105:0/1983329829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50dc15d4f0 0x7f50dc0d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e1436640 1 -- 172.21.15.105:0/1983329829 >> v2:172.21.15.148:3300/0 conn(0x7f50dc0d3c30 msgr2=0x7f50dc0d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:00.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e1436640 1 --2- 172.21.15.105:0/1983329829 >> v2:172.21.15.148:3300/0 conn(0x7f50dc0d3c30 0x7f50dc0d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e1436640 1 -- 172.21.15.105:0/1983329829 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50dc0ced10 con 0x7f50dc0d1890 2023-11-28T18:25:00.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e1436640 1 -- 172.21.15.105:0/1983329829 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f50d4001a00 con 0x7f50dc0d1890 2023-11-28T18:25:00.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e1436640 1 -- 172.21.15.105:0/1983329829 >> v1:172.21.15.105:6789/0 conn(0x7f50dc0d1890 legacy=0x7f50dc0cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:00.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e1436640 1 --2- 172.21.15.105:0/1983329829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c8003980 0x7f50c8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e1436640 1 --2- 172.21.15.105:0/1983329829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50dc15d4f0 0x7f50c80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e1436640 1 --2- 172.21.15.105:0/1983329829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c8004d10 0x7f50c8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e1436640 1 -- 172.21.15.105:0/1983329829 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc0b2900 con 0x7f50c8004d10 2023-11-28T18:25:00.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e1436640 1 -- 172.21.15.105:0/1983329829 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc0cea30 con 0x7f50c8003980 2023-11-28T18:25:00.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.106+0000 7f50e1436640 1 -- 172.21.15.105:0/1983329829 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c800b990 con 0x7f50dc15d4f0 2023-11-28T18:25:00.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.110+0000 7f50e2438640 1 --2- 172.21.15.105:0/1983329829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c8003980 0x7f50c8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.110+0000 7f50e2c39640 1 --2- 172.21.15.105:0/1983329829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c8004d10 0x7f50c8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.110+0000 7f50e1c37640 1 --2- 172.21.15.105:0/1983329829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50dc15d4f0 0x7f50c80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.110+0000 7f50e2438640 1 -- 172.21.15.105:0/1983329829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50dc15d4f0 msgr2=0x7f50c80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:00.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.110+0000 7f50e2438640 1 --2- 172.21.15.105:0/1983329829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50dc15d4f0 0x7f50c80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.110+0000 7f50e2438640 1 -- 172.21.15.105:0/1983329829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c8004d10 msgr2=0x7f50c8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:00.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.110+0000 7f50e2438640 1 --2- 172.21.15.105:0/1983329829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c8004d10 0x7f50c8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.110+0000 7f50e2438640 1 -- 172.21.15.105:0/1983329829 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c8003530 con 0x7f50c8003980 2023-11-28T18:25:00.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.110+0000 7f50e2c39640 1 --2- 172.21.15.105:0/1983329829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c8004d10 0x7f50c8005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:25:00.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.110+0000 7f50e1c37640 1 --2- 172.21.15.105:0/1983329829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50dc15d4f0 0x7f50c80047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:25:00.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.110+0000 7f50e2438640 1 --2- 172.21.15.105:0/1983329829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c8003980 0x7f50c8003da0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f50d8009ae0 tx=0x7f50d801ca50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55ae834611a022b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:00.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.114+0000 7f50e1436640 1 -- 172.21.15.105:0/1983329829 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d80a5640 con 0x7f50c8003980 2023-11-28T18:25:00.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.114+0000 7f50e1436640 1 -- 172.21.15.105:0/1983329829 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d80a57a0 con 0x7f50c8003980 2023-11-28T18:25:00.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.114+0000 7f50e1436640 1 -- 172.21.15.105:0/1983329829 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d80a5b50 con 0x7f50c8003980 2023-11-28T18:25:00.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.114+0000 7f50e469c640 1 -- 172.21.15.105:0/1983329829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c8003980 msgr2=0x7f50c8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:00.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.114+0000 7f50e469c640 1 --2- 172.21.15.105:0/1983329829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c8003980 0x7f50c8003da0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f50d8009ae0 tx=0x7f50d801ca50 comp rx=0 tx=0).stop 2023-11-28T18:25:00.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.114+0000 7f50e2c39640 1 -- 172.21.15.105:0/1983329829 reap_dead start 2023-11-28T18:25:00.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.114+0000 7f50e469c640 1 -- 172.21.15.105:0/1983329829 shutdown_connections 2023-11-28T18:25:00.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.114+0000 7f50e469c640 1 -- 172.21.15.105:0/1983329829 >> 172.21.15.105:0/1983329829 conn(0x7f50dc0d3030 msgr2=0x7f50dc0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:00.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.114+0000 7f50e469c640 1 -- 172.21.15.105:0/1983329829 shutdown_connections 2023-11-28T18:25:00.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.114+0000 7f50e469c640 1 -- 172.21.15.105:0/1983329829 wait complete. 2023-11-28T18:25:00.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.114+0000 7f50e469c640 1 Processor -- start 2023-11-28T18:25:00.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.114+0000 7f50e469c640 1 -- start start 2023-11-28T18:25:00.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.114+0000 7f50e469c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50c8003980 0x7f50dc15da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.114+0000 7f50e469c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50dc14b440 0x7f50dc0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.114+0000 7f50e469c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50dc0b42c0 0x7f50dc0b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.114+0000 7f50e469c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc0b4c80 con 0x7f50dc14b440 2023-11-28T18:25:00.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.114+0000 7f50e469c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc0cb7f0 con 0x7f50dc0b42c0 2023-11-28T18:25:00.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.114+0000 7f50e469c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc0cbad0 con 0x7f50c8003980 2023-11-28T18:25:00.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.114+0000 7f50e1c37640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50dc14b440 0x7f50dc0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.114+0000 7f50e1c37640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50dc14b440 0x7f50dc0cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36918/0 (socket says 172.21.15.105:36918) 2023-11-28T18:25:00.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.114+0000 7f50e1c37640 1 -- 172.21.15.105:0/2467258175 learned_addr learned my addr 172.21.15.105:0/2467258175 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:00.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.118+0000 7f50e2438640 1 --2- 172.21.15.105:0/2467258175 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50c8003980 0x7f50dc15da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.118+0000 7f50e2c39640 1 --2- 172.21.15.105:0/2467258175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50dc0b42c0 0x7f50dc0b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.118+0000 7f50e1c37640 1 -- 172.21.15.105:0/2467258175 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50c8003980 msgr2=0x7f50dc15da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:00.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.118+0000 7f50e1c37640 1 --2- 172.21.15.105:0/2467258175 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50c8003980 0x7f50dc15da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.118+0000 7f50e1c37640 1 -- 172.21.15.105:0/2467258175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50dc0b42c0 msgr2=0x7f50dc0b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:00.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.118+0000 7f50e1c37640 1 --2- 172.21.15.105:0/2467258175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50dc0b42c0 0x7f50dc0b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.118+0000 7f50e1c37640 1 -- 172.21.15.105:0/2467258175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50dc0cbdb0 con 0x7f50dc14b440 2023-11-28T18:25:00.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.118+0000 7f50e2c39640 1 --2- 172.21.15.105:0/2467258175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50dc0b42c0 0x7f50dc0b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:00.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.118+0000 7f50e1c37640 1 --2- 172.21.15.105:0/2467258175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50dc14b440 0x7f50dc0cc890 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f50cc011ca0 tx=0x7f50cc00f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:00.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.118+0000 7f50d2ffd640 1 -- 172.21.15.105:0/2467258175 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc011e50 con 0x7f50dc14b440 2023-11-28T18:25:00.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.118+0000 7f50d2ffd640 1 -- 172.21.15.105:0/2467258175 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50cc004020 con 0x7f50dc14b440 2023-11-28T18:25:00.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.118+0000 7f50e469c640 1 -- 172.21.15.105:0/2467258175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50dc0cc090 con 0x7f50dc14b440 2023-11-28T18:25:00.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.118+0000 7f50d2ffd640 1 -- 172.21.15.105:0/2467258175 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc0043d0 con 0x7f50dc14b440 2023-11-28T18:25:00.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.118+0000 7f50e469c640 1 -- 172.21.15.105:0/2467258175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50dc14b8c0 con 0x7f50dc14b440 2023-11-28T18:25:00.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.118+0000 7f50e469c640 1 -- 172.21.15.105:0/2467258175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50dc067570 con 0x7f50dc14b440 2023-11-28T18:25:00.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.118+0000 7f50d2ffd640 1 -- 172.21.15.105:0/2467258175 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f50cc003620 con 0x7f50dc14b440 2023-11-28T18:25:00.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.122+0000 7f50d2ffd640 1 --2- 172.21.15.105:0/2467258175 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f50c0066900 0x7f50c0068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.122+0000 7f50d2ffd640 1 -- 172.21.15.105:0/2467258175 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f50cc084c70 con 0x7f50dc14b440 2023-11-28T18:25:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.122+0000 7f50e2438640 1 --2- 172.21.15.105:0/2467258175 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f50c0066900 0x7f50c0068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.122+0000 7f50e2438640 1 --2- 172.21.15.105:0/2467258175 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f50c0066900 0x7f50c0068dc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f50d80af000 tx=0x7f50d80046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:00.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.122+0000 7f50d2ffd640 1 -- 172.21.15.105:0/2467258175 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f50cc051d70 con 0x7f50dc14b440 2023-11-28T18:25:00.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.278+0000 7f50e469c640 1 -- 172.21.15.105:0/2467258175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50dc072eb0 con 0x7f50dc14b440 2023-11-28T18:25:00.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.278+0000 7f50d2ffd640 1 -- 172.21.15.105:0/2467258175 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16834 (secure 0 0 0) 0x7f50cc022020 con 0x7f50dc14b440 2023-11-28T18:25:00.285 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:00.285 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":38,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:24:57.550814+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:24:53.582892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T18:24:54.921978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:25:00.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.282+0000 7f50e469c640 1 -- 172.21.15.105:0/2467258175 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f50c0066900 msgr2=0x7f50c0068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:00.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.282+0000 7f50e469c640 1 --2- 172.21.15.105:0/2467258175 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f50c0066900 0x7f50c0068dc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f50d80af000 tx=0x7f50d80046d0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.282+0000 7f50e469c640 1 -- 172.21.15.105:0/2467258175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50dc14b440 msgr2=0x7f50dc0cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:00.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.282+0000 7f50e469c640 1 --2- 172.21.15.105:0/2467258175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50dc14b440 0x7f50dc0cc890 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f50cc011ca0 tx=0x7f50cc00f610 comp rx=0 tx=0).stop 2023-11-28T18:25:00.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.282+0000 7f50e469c640 1 -- 172.21.15.105:0/2467258175 shutdown_connections 2023-11-28T18:25:00.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.282+0000 7f50e469c640 1 --2- 172.21.15.105:0/2467258175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50dc0b42c0 0x7f50dc0b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.282+0000 7f50e469c640 1 --2- 172.21.15.105:0/2467258175 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f50c0066900 0x7f50c0068dc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.282+0000 7f50e469c640 1 --2- 172.21.15.105:0/2467258175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50dc14b440 0x7f50dc0cc890 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.282+0000 7f50e469c640 1 --2- 172.21.15.105:0/2467258175 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50c8003980 0x7f50dc15da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.282+0000 7f50e469c640 1 -- 172.21.15.105:0/2467258175 >> 172.21.15.105:0/2467258175 conn(0x7f50dc0d3030 msgr2=0x7f50dc0b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:00.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.282+0000 7f50e469c640 1 -- 172.21.15.105:0/2467258175 shutdown_connections 2023-11-28T18:25:00.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.282+0000 7f50e469c640 1 -- 172.21.15.105:0/2467258175 wait complete. 2023-11-28T18:25:00.308 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T18:25:00.309 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:25:00.446 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.442+0000 7f0c96df8640 1 Processor -- start 2023-11-28T18:25:00.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.442+0000 7f0c96df8640 1 -- start start 2023-11-28T18:25:00.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.442+0000 7f0c96df8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0c9015d2d0 0x7f0c900cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.442+0000 7f0c96df8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c900cfdb0 con 0x7f0c9015cef0 2023-11-28T18:25:00.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.442+0000 7f0c96df8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c900ce840 con 0x7f0c900cf960 2023-11-28T18:25:00.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.442+0000 7f0c96df8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c900b4780 con 0x7f0c9015d2d0 2023-11-28T18:25:00.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.442+0000 7f0c955f5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0c9015d2d0 0x7f0c900cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.442+0000 7f0c955f5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0c9015d2d0 0x7f0c900cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36928/0 (socket says 172.21.15.105:36928) 2023-11-28T18:25:00.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.442+0000 7f0c955f5640 1 -- 172.21.15.105:0/2522757900 learned_addr learned my addr 172.21.15.105:0/2522757900 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:00.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.442+0000 7f0c95df6640 1 --1- 172.21.15.105:0/2522757900 >> v1:172.21.15.105:6789/0 conn(0x7f0c9015cef0 0x7f0c900d1600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54262/0 (socket says 172.21.15.105:54262) 2023-11-28T18:25:00.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.442+0000 7f0c955f5640 1 -- 172.21.15.105:0/2522757900 >> v1:172.21.15.148:6789/0 conn(0x7f0c900cf960 legacy=0x7f0c90166fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:00.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.442+0000 7f0c955f5640 1 -- 172.21.15.105:0/2522757900 >> v1:172.21.15.105:6789/0 conn(0x7f0c9015cef0 legacy=0x7f0c900d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:00.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.442+0000 7f0c955f5640 1 -- 172.21.15.105:0/2522757900 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c900cd990 con 0x7f0c9015d2d0 2023-11-28T18:25:00.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.446+0000 7f0c955f5640 1 --2- 172.21.15.105:0/2522757900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c9015d2d0 0x7f0c900cf420 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009870 tx=0x7f0c8001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7a0007734a45be server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:00.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.446+0000 7f0c94df4640 1 -- 172.21.15.105:0/2522757900 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c800ab650 con 0x7f0c9015d2d0 2023-11-28T18:25:00.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.446+0000 7f0c94df4640 1 -- 172.21.15.105:0/2522757900 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c800ab7b0 con 0x7f0c9015d2d0 2023-11-28T18:25:00.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.446+0000 7f0c94df4640 1 -- 172.21.15.105:0/2522757900 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c800abb40 con 0x7f0c9015d2d0 2023-11-28T18:25:00.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.446+0000 7f0c96df8640 1 -- 172.21.15.105:0/2522757900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c9015d2d0 msgr2=0x7f0c900cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:00.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.446+0000 7f0c96df8640 1 --2- 172.21.15.105:0/2522757900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c9015d2d0 0x7f0c900cf420 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009870 tx=0x7f0c8001c450 comp rx=0 tx=0).stop 2023-11-28T18:25:00.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.446+0000 7f0c96df8640 1 -- 172.21.15.105:0/2522757900 shutdown_connections 2023-11-28T18:25:00.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.446+0000 7f0c96df8640 1 --2- 172.21.15.105:0/2522757900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c9015d2d0 0x7f0c900cf420 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.446+0000 7f0c96df8640 1 -- 172.21.15.105:0/2522757900 >> 172.21.15.105:0/2522757900 conn(0x7f0c900d2e10 msgr2=0x7f0c900d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:00.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.446+0000 7f0c96df8640 1 -- 172.21.15.105:0/2522757900 shutdown_connections 2023-11-28T18:25:00.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.446+0000 7f0c96df8640 1 -- 172.21.15.105:0/2522757900 wait complete. 2023-11-28T18:25:00.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c96df8640 1 Processor -- start 2023-11-28T18:25:00.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c96df8640 1 -- start start 2023-11-28T18:25:00.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c96df8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c900cf960 0x7f0c900c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c96df8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c9015cef0 0x7f0c90160100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c96df8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c9015d2d0 0x7f0c900c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c96df8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c900b7500 con 0x7f0c9015cef0 2023-11-28T18:25:00.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c96df8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c900b77e0 con 0x7f0c9015d2d0 2023-11-28T18:25:00.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c96df8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c900b7ac0 con 0x7f0c900cf960 2023-11-28T18:25:00.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c955f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c9015cef0 0x7f0c90160100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c955f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c9015cef0 0x7f0c90160100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36934/0 (socket says 172.21.15.105:36934) 2023-11-28T18:25:00.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c955f5640 1 -- 172.21.15.105:0/2825945638 learned_addr learned my addr 172.21.15.105:0/2825945638 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:00.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c95df6640 1 --2- 172.21.15.105:0/2825945638 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c900cf960 0x7f0c900c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c955f5640 1 -- 172.21.15.105:0/2825945638 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c900cf960 msgr2=0x7f0c900c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:00.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c955f5640 1 --2- 172.21.15.105:0/2825945638 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c900cf960 0x7f0c900c6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c965f7640 1 --2- 172.21.15.105:0/2825945638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c9015d2d0 0x7f0c900c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c955f5640 1 -- 172.21.15.105:0/2825945638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c9015d2d0 msgr2=0x7f0c900c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:00.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c955f5640 1 --2- 172.21.15.105:0/2825945638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c9015d2d0 0x7f0c900c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c955f5640 1 -- 172.21.15.105:0/2825945638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c800094d0 con 0x7f0c9015cef0 2023-11-28T18:25:00.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c95df6640 1 --2- 172.21.15.105:0/2825945638 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c900cf960 0x7f0c900c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:00.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c965f7640 1 --2- 172.21.15.105:0/2825945638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c9015d2d0 0x7f0c900c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:00.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c955f5640 1 --2- 172.21.15.105:0/2825945638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c9015cef0 0x7f0c90160100 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0c800ad040 tx=0x7f0c800191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:00.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c867fc640 1 -- 172.21.15.105:0/2825945638 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c800b58d0 con 0x7f0c9015cef0 2023-11-28T18:25:00.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c867fc640 1 -- 172.21.15.105:0/2825945638 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c800b5a70 con 0x7f0c9015cef0 2023-11-28T18:25:00.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c867fc640 1 -- 172.21.15.105:0/2825945638 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c800b5e20 con 0x7f0c9015cef0 2023-11-28T18:25:00.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c96df8640 1 -- 172.21.15.105:0/2825945638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c900b7da0 con 0x7f0c9015cef0 2023-11-28T18:25:00.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.450+0000 7f0c96df8640 1 -- 172.21.15.105:0/2825945638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c900b82c0 con 0x7f0c9015cef0 2023-11-28T18:25:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.454+0000 7f0c96df8640 1 -- 172.21.15.105:0/2825945638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c900ce490 con 0x7f0c9015cef0 2023-11-28T18:25:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.454+0000 7f0c867fc640 1 -- 172.21.15.105:0/2825945638 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f0c800bf410 con 0x7f0c9015cef0 2023-11-28T18:25:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.454+0000 7f0c867fc640 1 --2- 172.21.15.105:0/2825945638 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0c740668d0 0x7f0c74068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.454+0000 7f0c867fc640 1 -- 172.21.15.105:0/2825945638 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0c8011f2b0 con 0x7f0c9015cef0 2023-11-28T18:25:00.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.454+0000 7f0c95df6640 1 --2- 172.21.15.105:0/2825945638 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0c740668d0 0x7f0c74068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.454+0000 7f0c95df6640 1 --2- 172.21.15.105:0/2825945638 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0c740668d0 0x7f0c74068d90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c000f70 tx=0x7f0c8c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:00.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.458+0000 7f0c867fc640 1 -- 172.21.15.105:0/2825945638 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0c800ec3b0 con 0x7f0c9015cef0 2023-11-28T18:25:00.533 INFO:tasks.ceph.mds.a.smithi105.stderr:2023-11-28T18:25:00.526+0000 7f86309ae640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T18:25:00.613 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.606+0000 7f0c96df8640 1 -- 172.21.15.105:0/2825945638 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c900b7950 con 0x7f0c740668d0 2023-11-28T18:25:00.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.618+0000 7f0c867fc640 1 -- 172.21.15.105:0/2825945638 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284236 (secure 0 0 0) 0x7f0c900b7950 con 0x7f0c740668d0 2023-11-28T18:25:00.625 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:00.626 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:25:00.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.618+0000 7f0c96df8640 1 -- 172.21.15.105:0/2825945638 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0c740668d0 msgr2=0x7f0c74068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:00.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.618+0000 7f0c96df8640 1 --2- 172.21.15.105:0/2825945638 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0c740668d0 0x7f0c74068d90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c000f70 tx=0x7f0c8c009040 comp rx=0 tx=0).stop 2023-11-28T18:25:00.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.618+0000 7f0c96df8640 1 -- 172.21.15.105:0/2825945638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c9015cef0 msgr2=0x7f0c90160100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:00.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.618+0000 7f0c96df8640 1 --2- 172.21.15.105:0/2825945638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c9015cef0 0x7f0c90160100 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0c800ad040 tx=0x7f0c800191f0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.622+0000 7f0c96df8640 1 -- 172.21.15.105:0/2825945638 shutdown_connections 2023-11-28T18:25:00.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.622+0000 7f0c96df8640 1 --2- 172.21.15.105:0/2825945638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c9015d2d0 0x7f0c900c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.622+0000 7f0c96df8640 1 --2- 172.21.15.105:0/2825945638 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0c740668d0 0x7f0c74068d90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.622+0000 7f0c96df8640 1 --2- 172.21.15.105:0/2825945638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c9015cef0 0x7f0c90160100 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.622+0000 7f0c96df8640 1 --2- 172.21.15.105:0/2825945638 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c900cf960 0x7f0c900c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.622+0000 7f0c96df8640 1 -- 172.21.15.105:0/2825945638 >> 172.21.15.105:0/2825945638 conn(0x7f0c900d2e10 msgr2=0x7f0c9006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:00.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.622+0000 7f0c96df8640 1 -- 172.21.15.105:0/2825945638 shutdown_connections 2023-11-28T18:25:00.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.622+0000 7f0c96df8640 1 -- 172.21.15.105:0/2825945638 wait complete. 2023-11-28T18:25:00.644 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2023-11-28T18:24:59.286911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220536,"kb_used_data":4368,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754184,"statfs":{"total":773094113280,"available":772868284416,"internally_reserved":0,"allocated":4472832,"data_stored":2568354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.747004"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.555352+0000","last_change":"2023-11-28T18:24:55.555352+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.555352+0000","last_undegraded":"2023-11-28T18:24:55.555352+0000","last_fullsized":"2023-11-28T18:24:55.555352+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554587+0000","last_change":"2023-11-28T18:24:55.554587+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554587+0000","last_undegraded":"2023-11-28T18:24:55.554587+0000","last_fullsized":"2023-11-28T18:24:55.554587+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558901+0000","last_change":"2023-11-28T18:24:55.558901+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558901+0000","last_undegraded":"2023-11-28T18:24:55.558901+0000","last_fullsized":"2023-11-28T18:24:55.558901+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.566872+0000","last_change":"2023-11-28T18:24:55.566872+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.566872+0000","last_undegraded":"2023-11-28T18:24:55.566872+0000","last_fullsized":"2023-11-28T18:24:55.566872+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.565232+0000","last_change":"2023-11-28T18:24:55.565232+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.565232+0000","last_undegraded":"2023-11-28T18:24:55.565232+0000","last_fullsized":"2023-11-28T18:24:55.565232+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.557666+0000","last_change":"2023-11-28T18:24:55.557666+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.557666+0000","last_undegraded":"2023-11-28T18:24:55.557666+0000","last_fullsized":"2023-11-28T18:24:55.557666+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.564284+0000","last_change":"2023-11-28T18:24:55.564284+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.564284+0000","last_undegraded":"2023-11-28T18:24:55.564284+0000","last_fullsized":"2023-11-28T18:24:55.564284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554637+0000","last_change":"2023-11-28T18:24:55.554637+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554637+0000","last_undegraded":"2023-11-28T18:24:55.554637+0000","last_fullsized":"2023-11-28T18:24:55.554637+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.556678+0000","last_change":"2023-11-28T18:24:55.556678+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.556678+0000","last_undegraded":"2023-11-28T18:24:55.556678+0000","last_fullsized":"2023-11-28T18:24:55.556678+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.566281+0000","last_change":"2023-11-28T18:24:55.566281+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.566281+0000","last_undegraded":"2023-11-28T18:24:55.566281+0000","last_fullsized":"2023-11-28T18:24:55.566281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.568480+0000","last_change":"2023-11-28T18:24:55.568480+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.568480+0000","last_undegraded":"2023-11-28T18:24:55.568480+0000","last_fullsized":"2023-11-28T18:24:55.568480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.556143+0000","last_change":"2023-11-28T18:24:55.556143+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.556143+0000","last_undegraded":"2023-11-28T18:24:55.556143+0000","last_fullsized":"2023-11-28T18:24:55.556143+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563047+0000","last_change":"2023-11-28T18:24:55.563047+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563047+0000","last_undegraded":"2023-11-28T18:24:55.563047+0000","last_fullsized":"2023-11-28T18:24:55.563047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.556775+0000","last_change":"2023-11-28T18:24:55.556775+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.556775+0000","last_undegraded":"2023-11-28T18:24:55.556775+0000","last_fullsized":"2023-11-28T18:24:55.556775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563722+0000","last_change":"2023-11-28T18:24:55.563722+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563722+0000","last_undegraded":"2023-11-28T18:24:55.563722+0000","last_fullsized":"2023-11-28T18:24:55.563722+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563809+0000","last_change":"2023-11-28T18:24:55.563809+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563809+0000","last_undegraded":"2023-11-28T18:24:55.563809+0000","last_fullsized":"2023-11-28T18:24:55.563809+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.561864+0000","last_change":"2023-11-28T18:24:55.561864+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.561864+0000","last_undegraded":"2023-11-28T18:24:55.561864+0000","last_fullsized":"2023-11-28T18:24:55.561864+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554411+0000","last_change":"2023-11-28T18:24:55.554411+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554411+0000","last_undegraded":"2023-11-28T18:24:55.554411+0000","last_fullsized":"2023-11-28T18:24:55.554411+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.568017+0000","last_change":"2023-11-28T18:24:55.568017+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.568017+0000","last_undegraded":"2023-11-28T18:24:55.568017+0000","last_fullsized":"2023-11-28T18:24:55.568017+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553039+0000","last_change":"2023-11-28T18:24:55.553039+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553039+0000","last_undegraded":"2023-11-28T18:24:55.553039+0000","last_fullsized":"2023-11-28T18:24:55.553039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.562468+0000","last_change":"2023-11-28T18:24:55.562468+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.562468+0000","last_undegraded":"2023-11-28T18:24:55.562468+0000","last_fullsized":"2023-11-28T18:24:55.562468+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.555237+0000","last_change":"2023-11-28T18:24:55.555237+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.555237+0000","last_undegraded":"2023-11-28T18:24:55.555237+0000","last_fullsized":"2023-11-28T18:24:55.555237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.559896+0000","last_change":"2023-11-28T18:24:55.559896+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.559896+0000","last_undegraded":"2023-11-28T18:24:55.559896+0000","last_fullsized":"2023-11-28T18:24:55.559896+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.562708+0000","last_change":"2023-11-28T18:24:55.562708+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.562708+0000","last_undegraded":"2023-11-28T18:24:55.562708+0000","last_fullsized":"2023-11-28T18:24:55.562708+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553693+0000","last_change":"2023-11-28T18:24:55.553693+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553693+0000","last_undegraded":"2023-11-28T18:24:55.553693+0000","last_fullsized":"2023-11-28T18:24:55.553693+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.561952+0000","last_change":"2023-11-28T18:24:55.561952+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.561952+0000","last_undegraded":"2023-11-28T18:24:55.561952+0000","last_fullsized":"2023-11-28T18:24:55.561952+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553056+0000","last_change":"2023-11-28T18:24:55.553056+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553056+0000","last_undegraded":"2023-11-28T18:24:55.553056+0000","last_fullsized":"2023-11-28T18:24:55.553056+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.567550+0000","last_change":"2023-11-28T18:24:55.567550+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.567550+0000","last_undegraded":"2023-11-28T18:24:55.567550+0000","last_fullsized":"2023-11-28T18:24:55.567550+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553657+0000","last_change":"2023-11-28T18:24:55.553657+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553657+0000","last_undegraded":"2023-11-28T18:24:55.553657+0000","last_fullsized":"2023-11-28T18:24:55.553657+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553875+0000","last_change":"2023-11-28T18:24:55.553875+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553875+0000","last_undegraded":"2023-11-28T18:24:55.553875+0000","last_fullsized":"2023-11-28T18:24:55.553875+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.559065+0000","last_change":"2023-11-28T18:24:55.559065+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.559065+0000","last_undegraded":"2023-11-28T18:24:55.559065+0000","last_fullsized":"2023-11-28T18:24:55.559065+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558556+0000","last_change":"2023-11-28T18:24:55.558556+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558556+0000","last_undegraded":"2023-11-28T18:24:55.558556+0000","last_fullsized":"2023-11-28T18:24:55.558556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.561337+0000","last_change":"2023-11-28T18:24:55.561337+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.561337+0000","last_undegraded":"2023-11-28T18:24:55.561337+0000","last_fullsized":"2023-11-28T18:24:55.561337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.578117+0000","last_change":"2023-11-28T18:24:55.573384+0000","last_active":"2023-11-28T18:24:55.578117+0000","last_peered":"2023-11-28T18:24:55.578117+0000","last_clean":"2023-11-28T18:24:55.578117+0000","last_became_active":"2023-11-28T18:24:55.571607+0000","last_became_peered":"2023-11-28T18:24:55.571607+0000","last_unstale":"2023-11-28T18:24:55.578117+0000","last_undegraded":"2023-11-28T18:24:55.578117+0000","last_fullsized":"2023-11-28T18:24:55.578117+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:17:39.496716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.562352+0000","last_change":"2023-11-28T18:24:55.562352+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.562352+0000","last_undegraded":"2023-11-28T18:24:55.562352+0000","last_fullsized":"2023-11-28T18:24:55.562352+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.579114+0000","last_change":"2023-11-28T18:24:55.573927+0000","last_active":"2023-11-28T18:24:55.579114+0000","last_peered":"2023-11-28T18:24:55.579114+0000","last_clean":"2023-11-28T18:24:55.579114+0000","last_became_active":"2023-11-28T18:24:55.573623+0000","last_became_peered":"2023-11-28T18:24:55.573623+0000","last_unstale":"2023-11-28T18:24:55.579114+0000","last_undegraded":"2023-11-28T18:24:55.579114+0000","last_fullsized":"2023-11-28T18:24:55.579114+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:29:45.071843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558261+0000","last_change":"2023-11-28T18:24:55.558261+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558261+0000","last_undegraded":"2023-11-28T18:24:55.558261+0000","last_fullsized":"2023-11-28T18:24:55.558261+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.573242+0000","last_change":"2023-11-28T18:24:55.570144+0000","last_active":"2023-11-28T18:24:55.573242+0000","last_peered":"2023-11-28T18:24:55.573242+0000","last_clean":"2023-11-28T18:24:55.573242+0000","last_became_active":"2023-11-28T18:24:55.569837+0000","last_became_peered":"2023-11-28T18:24:55.569837+0000","last_unstale":"2023-11-28T18:24:55.573242+0000","last_undegraded":"2023-11-28T18:24:55.573242+0000","last_fullsized":"2023-11-28T18:24:55.573242+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:17:08.512763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554982+0000","last_change":"2023-11-28T18:24:55.554982+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554982+0000","last_undegraded":"2023-11-28T18:24:55.554982+0000","last_fullsized":"2023-11-28T18:24:55.554982+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.572198+0000","last_change":"2023-11-28T18:24:55.570052+0000","last_active":"2023-11-28T18:24:55.572198+0000","last_peered":"2023-11-28T18:24:55.572198+0000","last_clean":"2023-11-28T18:24:55.572198+0000","last_became_active":"2023-11-28T18:24:55.569712+0000","last_became_peered":"2023-11-28T18:24:55.569712+0000","last_unstale":"2023-11-28T18:24:55.572198+0000","last_undegraded":"2023-11-28T18:24:55.572198+0000","last_fullsized":"2023-11-28T18:24:55.572198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:06.899740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558633+0000","last_change":"2023-11-28T18:24:55.558633+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558633+0000","last_undegraded":"2023-11-28T18:24:55.558633+0000","last_fullsized":"2023-11-28T18:24:55.558633+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.580374+0000","last_change":"2023-11-28T18:24:55.575819+0000","last_active":"2023-11-28T18:24:55.580374+0000","last_peered":"2023-11-28T18:24:55.580374+0000","last_clean":"2023-11-28T18:24:55.580374+0000","last_became_active":"2023-11-28T18:24:55.575541+0000","last_became_peered":"2023-11-28T18:24:55.575541+0000","last_unstale":"2023-11-28T18:24:55.580374+0000","last_undegraded":"2023-11-28T18:24:55.580374+0000","last_fullsized":"2023-11-28T18:24:55.580374+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:46.037101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563170+0000","last_change":"2023-11-28T18:24:55.563170+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563170+0000","last_undegraded":"2023-11-28T18:24:55.563170+0000","last_fullsized":"2023-11-28T18:24:55.563170+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.580797+0000","last_change":"2023-11-28T18:24:55.578529+0000","last_active":"2023-11-28T18:24:55.580797+0000","last_peered":"2023-11-28T18:24:55.580797+0000","last_clean":"2023-11-28T18:24:55.580797+0000","last_became_active":"2023-11-28T18:24:55.578252+0000","last_became_peered":"2023-11-28T18:24:55.578252+0000","last_unstale":"2023-11-28T18:24:55.580797+0000","last_undegraded":"2023-11-28T18:24:55.580797+0000","last_fullsized":"2023-11-28T18:24:55.580797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:50:55.081198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.555789+0000","last_change":"2023-11-28T18:24:55.555789+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.555789+0000","last_undegraded":"2023-11-28T18:24:55.555789+0000","last_fullsized":"2023-11-28T18:24:55.555789+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.578996+0000","last_change":"2023-11-28T18:24:55.574964+0000","last_active":"2023-11-28T18:24:55.578996+0000","last_peered":"2023-11-28T18:24:55.578996+0000","last_clean":"2023-11-28T18:24:55.578996+0000","last_became_active":"2023-11-28T18:24:55.574669+0000","last_became_peered":"2023-11-28T18:24:55.574669+0000","last_unstale":"2023-11-28T18:24:55.578996+0000","last_undegraded":"2023-11-28T18:24:55.578996+0000","last_fullsized":"2023-11-28T18:24:55.578996+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:40:45.446584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554497+0000","last_change":"2023-11-28T18:24:55.554497+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554497+0000","last_undegraded":"2023-11-28T18:24:55.554497+0000","last_fullsized":"2023-11-28T18:24:55.554497+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.567976+0000","last_change":"2023-11-28T18:24:55.563267+0000","last_active":"2023-11-28T18:24:55.567976+0000","last_peered":"2023-11-28T18:24:55.567976+0000","last_clean":"2023-11-28T18:24:55.567976+0000","last_became_active":"2023-11-28T18:24:55.562979+0000","last_became_peered":"2023-11-28T18:24:55.562979+0000","last_unstale":"2023-11-28T18:24:55.567976+0000","last_undegraded":"2023-11-28T18:24:55.567976+0000","last_fullsized":"2023-11-28T18:24:55.567976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:06.790640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.565983+0000","last_change":"2023-11-28T18:24:55.565983+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.565983+0000","last_undegraded":"2023-11-28T18:24:55.565983+0000","last_fullsized":"2023-11-28T18:24:55.565983+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.578882+0000","last_change":"2023-11-28T18:24:55.572743+0000","last_active":"2023-11-28T18:24:55.578882+0000","last_peered":"2023-11-28T18:24:55.578882+0000","last_clean":"2023-11-28T18:24:55.578882+0000","last_became_active":"2023-11-28T18:24:55.571872+0000","last_became_peered":"2023-11-28T18:24:55.571872+0000","last_unstale":"2023-11-28T18:24:55.578882+0000","last_undegraded":"2023-11-28T18:24:55.578882+0000","last_fullsized":"2023-11-28T18:24:55.578882+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:14.095135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.564500+0000","last_change":"2023-11-28T18:24:55.564500+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.564500+0000","last_undegraded":"2023-11-28T18:24:55.564500+0000","last_fullsized":"2023-11-28T18:24:55.564500+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581663+0000","last_change":"2023-11-28T18:24:55.579938+0000","last_active":"2023-11-28T18:24:55.581663+0000","last_peered":"2023-11-28T18:24:55.581663+0000","last_clean":"2023-11-28T18:24:55.581663+0000","last_became_active":"2023-11-28T18:24:55.579667+0000","last_became_peered":"2023-11-28T18:24:55.579667+0000","last_unstale":"2023-11-28T18:24:55.581663+0000","last_undegraded":"2023-11-28T18:24:55.581663+0000","last_fullsized":"2023-11-28T18:24:55.581663+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:14.095135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.556171+0000","last_change":"2023-11-28T18:24:55.556171+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.556171+0000","last_undegraded":"2023-11-28T18:24:55.556171+0000","last_fullsized":"2023-11-28T18:24:55.556171+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581601+0000","last_change":"2023-11-28T18:24:55.579772+0000","last_active":"2023-11-28T18:24:55.581601+0000","last_peered":"2023-11-28T18:24:55.581601+0000","last_clean":"2023-11-28T18:24:55.581601+0000","last_became_active":"2023-11-28T18:24:55.579492+0000","last_became_peered":"2023-11-28T18:24:55.579492+0000","last_unstale":"2023-11-28T18:24:55.581601+0000","last_undegraded":"2023-11-28T18:24:55.581601+0000","last_fullsized":"2023-11-28T18:24:55.581601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:29:37.604627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558779+0000","last_change":"2023-11-28T18:24:55.558779+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558779+0000","last_undegraded":"2023-11-28T18:24:55.558779+0000","last_fullsized":"2023-11-28T18:24:55.558779+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581425+0000","last_change":"2023-11-28T18:24:55.576648+0000","last_active":"2023-11-28T18:24:55.581425+0000","last_peered":"2023-11-28T18:24:55.581425+0000","last_clean":"2023-11-28T18:24:55.581425+0000","last_became_active":"2023-11-28T18:24:55.576382+0000","last_became_peered":"2023-11-28T18:24:55.576382+0000","last_unstale":"2023-11-28T18:24:55.581425+0000","last_undegraded":"2023-11-28T18:24:55.581425+0000","last_fullsized":"2023-11-28T18:24:55.581425+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:29:37.604627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558183+0000","last_change":"2023-11-28T18:24:55.558183+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558183+0000","last_undegraded":"2023-11-28T18:24:55.558183+0000","last_fullsized":"2023-11-28T18:24:55.558183+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.569235+0000","last_change":"2023-11-28T18:24:55.565583+0000","last_active":"2023-11-28T18:24:55.569235+0000","last_peered":"2023-11-28T18:24:55.569235+0000","last_clean":"2023-11-28T18:24:55.569235+0000","last_became_active":"2023-11-28T18:24:55.565335+0000","last_became_peered":"2023-11-28T18:24:55.565335+0000","last_unstale":"2023-11-28T18:24:55.569235+0000","last_undegraded":"2023-11-28T18:24:55.569235+0000","last_fullsized":"2023-11-28T18:24:55.569235+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:47:09.714446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553365+0000","last_change":"2023-11-28T18:24:55.553365+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553365+0000","last_undegraded":"2023-11-28T18:24:55.553365+0000","last_fullsized":"2023-11-28T18:24:55.553365+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.579362+0000","last_change":"2023-11-28T18:24:55.575066+0000","last_active":"2023-11-28T18:24:55.579362+0000","last_peered":"2023-11-28T18:24:55.579362+0000","last_clean":"2023-11-28T18:24:55.579362+0000","last_became_active":"2023-11-28T18:24:55.574770+0000","last_became_peered":"2023-11-28T18:24:55.574770+0000","last_unstale":"2023-11-28T18:24:55.579362+0000","last_undegraded":"2023-11-28T18:24:55.579362+0000","last_fullsized":"2023-11-28T18:24:55.579362+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:44.881911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.560801+0000","last_change":"2023-11-28T18:24:55.560801+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.560801+0000","last_undegraded":"2023-11-28T18:24:55.560801+0000","last_fullsized":"2023-11-28T18:24:55.560801+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.563818+0000","last_change":"2023-11-28T18:24:55.561985+0000","last_active":"2023-11-28T18:24:55.563818+0000","last_peered":"2023-11-28T18:24:55.563818+0000","last_clean":"2023-11-28T18:24:55.563818+0000","last_became_active":"2023-11-28T18:24:55.561673+0000","last_became_peered":"2023-11-28T18:24:55.561673+0000","last_unstale":"2023-11-28T18:24:55.563818+0000","last_undegraded":"2023-11-28T18:24:55.563818+0000","last_fullsized":"2023-11-28T18:24:55.563818+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:15:50.678362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554008+0000","last_change":"2023-11-28T18:24:55.554008+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554008+0000","last_undegraded":"2023-11-28T18:24:55.554008+0000","last_fullsized":"2023-11-28T18:24:55.554008+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.576918+0000","last_change":"2023-11-28T18:24:55.572674+0000","last_active":"2023-11-28T18:24:55.576918+0000","last_peered":"2023-11-28T18:24:55.576918+0000","last_clean":"2023-11-28T18:24:55.576918+0000","last_became_active":"2023-11-28T18:24:55.572209+0000","last_became_peered":"2023-11-28T18:24:55.572209+0000","last_unstale":"2023-11-28T18:24:55.576918+0000","last_undegraded":"2023-11-28T18:24:55.576918+0000","last_fullsized":"2023-11-28T18:24:55.576918+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:17:08.512763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553441+0000","last_change":"2023-11-28T18:24:55.553441+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553441+0000","last_undegraded":"2023-11-28T18:24:55.553441+0000","last_fullsized":"2023-11-28T18:24:55.553441+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.580400+0000","last_change":"2023-11-28T18:24:55.575893+0000","last_active":"2023-11-28T18:24:55.580400+0000","last_peered":"2023-11-28T18:24:55.580400+0000","last_clean":"2023-11-28T18:24:55.580400+0000","last_became_active":"2023-11-28T18:24:55.575425+0000","last_became_peered":"2023-11-28T18:24:55.575425+0000","last_unstale":"2023-11-28T18:24:55.580400+0000","last_undegraded":"2023-11-28T18:24:55.580400+0000","last_fullsized":"2023-11-28T18:24:55.580400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:42:09.903901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.568047+0000","last_change":"2023-11-28T18:24:55.568047+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.568047+0000","last_undegraded":"2023-11-28T18:24:55.568047+0000","last_fullsized":"2023-11-28T18:24:55.568047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.583021+0000","last_change":"2023-11-28T18:24:55.579138+0000","last_active":"2023-11-28T18:24:55.583021+0000","last_peered":"2023-11-28T18:24:55.583021+0000","last_clean":"2023-11-28T18:24:55.583021+0000","last_became_active":"2023-11-28T18:24:55.578827+0000","last_became_peered":"2023-11-28T18:24:55.578827+0000","last_unstale":"2023-11-28T18:24:55.583021+0000","last_undegraded":"2023-11-28T18:24:55.583021+0000","last_fullsized":"2023-11-28T18:24:55.583021+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:55:08.887224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558780+0000","last_change":"2023-11-28T18:24:55.558780+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558780+0000","last_undegraded":"2023-11-28T18:24:55.558780+0000","last_fullsized":"2023-11-28T18:24:55.558780+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581919+0000","last_change":"2023-11-28T18:24:55.578515+0000","last_active":"2023-11-28T18:24:55.581919+0000","last_peered":"2023-11-28T18:24:55.581919+0000","last_clean":"2023-11-28T18:24:55.581919+0000","last_became_active":"2023-11-28T18:24:55.578256+0000","last_became_peered":"2023-11-28T18:24:55.578256+0000","last_unstale":"2023-11-28T18:24:55.581919+0000","last_undegraded":"2023-11-28T18:24:55.581919+0000","last_fullsized":"2023-11-28T18:24:55.581919+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:34.823798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.569066+0000","last_change":"2023-11-28T18:24:55.569066+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.569066+0000","last_undegraded":"2023-11-28T18:24:55.569066+0000","last_fullsized":"2023-11-28T18:24:55.569066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.579484+0000","last_change":"2023-11-28T18:24:55.575784+0000","last_active":"2023-11-28T18:24:55.579484+0000","last_peered":"2023-11-28T18:24:55.579484+0000","last_clean":"2023-11-28T18:24:55.579484+0000","last_became_active":"2023-11-28T18:24:55.575518+0000","last_became_peered":"2023-11-28T18:24:55.575518+0000","last_unstale":"2023-11-28T18:24:55.579484+0000","last_undegraded":"2023-11-28T18:24:55.579484+0000","last_fullsized":"2023-11-28T18:24:55.579484+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:55:08.887224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563340+0000","last_change":"2023-11-28T18:24:55.563340+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563340+0000","last_undegraded":"2023-11-28T18:24:55.563340+0000","last_fullsized":"2023-11-28T18:24:55.563340+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581809+0000","last_change":"2023-11-28T18:24:55.578008+0000","last_active":"2023-11-28T18:24:55.581809+0000","last_peered":"2023-11-28T18:24:55.581809+0000","last_clean":"2023-11-28T18:24:55.581809+0000","last_became_active":"2023-11-28T18:24:55.577782+0000","last_became_peered":"2023-11-28T18:24:55.577782+0000","last_unstale":"2023-11-28T18:24:55.581809+0000","last_undegraded":"2023-11-28T18:24:55.581809+0000","last_fullsized":"2023-11-28T18:24:55.581809+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:20:31.030439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.557289+0000","last_change":"2023-11-28T18:24:55.557289+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.557289+0000","last_undegraded":"2023-11-28T18:24:55.557289+0000","last_fullsized":"2023-11-28T18:24:55.557289+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581727+0000","last_change":"2023-11-28T18:24:55.576784+0000","last_active":"2023-11-28T18:24:55.581727+0000","last_peered":"2023-11-28T18:24:55.581727+0000","last_clean":"2023-11-28T18:24:55.581727+0000","last_became_active":"2023-11-28T18:24:55.576508+0000","last_became_peered":"2023-11-28T18:24:55.576508+0000","last_unstale":"2023-11-28T18:24:55.581727+0000","last_undegraded":"2023-11-28T18:24:55.581727+0000","last_fullsized":"2023-11-28T18:24:55.581727+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:20:38.032106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563243+0000","last_change":"2023-11-28T18:24:55.563243+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563243+0000","last_undegraded":"2023-11-28T18:24:55.563243+0000","last_fullsized":"2023-11-28T18:24:55.563243+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.583609+0000","last_change":"2023-11-28T18:24:55.582487+0000","last_active":"2023-11-28T18:24:55.583609+0000","last_peered":"2023-11-28T18:24:55.583609+0000","last_clean":"2023-11-28T18:24:55.583609+0000","last_became_active":"2023-11-28T18:24:55.582038+0000","last_became_peered":"2023-11-28T18:24:55.582038+0000","last_unstale":"2023-11-28T18:24:55.583609+0000","last_undegraded":"2023-11-28T18:24:55.583609+0000","last_fullsized":"2023-11-28T18:24:55.583609+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:46.037101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553428+0000","last_change":"2023-11-28T18:24:55.553428+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553428+0000","last_undegraded":"2023-11-28T18:24:55.553428+0000","last_fullsized":"2023-11-28T18:24:55.553428+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581789+0000","last_change":"2023-11-28T18:24:55.577155+0000","last_active":"2023-11-28T18:24:55.581789+0000","last_peered":"2023-11-28T18:24:55.581789+0000","last_clean":"2023-11-28T18:24:55.581789+0000","last_became_active":"2023-11-28T18:24:55.576899+0000","last_became_peered":"2023-11-28T18:24:55.576899+0000","last_unstale":"2023-11-28T18:24:55.581789+0000","last_undegraded":"2023-11-28T18:24:55.581789+0000","last_fullsized":"2023-11-28T18:24:55.581789+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:18:46.680968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558068+0000","last_change":"2023-11-28T18:24:55.558068+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558068+0000","last_undegraded":"2023-11-28T18:24:55.558068+0000","last_fullsized":"2023-11-28T18:24:55.558068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581668+0000","last_change":"2023-11-28T18:24:55.579931+0000","last_active":"2023-11-28T18:24:55.581668+0000","last_peered":"2023-11-28T18:24:55.581668+0000","last_clean":"2023-11-28T18:24:55.581668+0000","last_became_active":"2023-11-28T18:24:55.579568+0000","last_became_peered":"2023-11-28T18:24:55.579568+0000","last_unstale":"2023-11-28T18:24:55.581668+0000","last_undegraded":"2023-11-28T18:24:55.581668+0000","last_fullsized":"2023-11-28T18:24:55.581668+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:47:09.714446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563929+0000","last_change":"2023-11-28T18:24:55.563929+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563929+0000","last_undegraded":"2023-11-28T18:24:55.563929+0000","last_fullsized":"2023-11-28T18:24:55.563929+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.584207+0000","last_change":"2023-11-28T18:24:55.580960+0000","last_active":"2023-11-28T18:24:55.584207+0000","last_peered":"2023-11-28T18:24:55.584207+0000","last_clean":"2023-11-28T18:24:55.584207+0000","last_became_active":"2023-11-28T18:24:55.580717+0000","last_became_peered":"2023-11-28T18:24:55.580717+0000","last_unstale":"2023-11-28T18:24:55.584207+0000","last_undegraded":"2023-11-28T18:24:55.584207+0000","last_fullsized":"2023-11-28T18:24:55.584207+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:15:07.052209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553436+0000","last_change":"2023-11-28T18:24:55.553436+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553436+0000","last_undegraded":"2023-11-28T18:24:55.553436+0000","last_fullsized":"2023-11-28T18:24:55.553436+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.573984+0000","last_change":"2023-11-28T18:24:55.569035+0000","last_active":"2023-11-28T18:24:55.573984+0000","last_peered":"2023-11-28T18:24:55.573984+0000","last_clean":"2023-11-28T18:24:55.573984+0000","last_became_active":"2023-11-28T18:24:55.568744+0000","last_became_peered":"2023-11-28T18:24:55.568744+0000","last_unstale":"2023-11-28T18:24:55.573984+0000","last_undegraded":"2023-11-28T18:24:55.573984+0000","last_fullsized":"2023-11-28T18:24:55.573984+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:47:09.714446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.557761+0000","last_change":"2023-11-28T18:24:55.557761+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.557761+0000","last_undegraded":"2023-11-28T18:24:55.557761+0000","last_fullsized":"2023-11-28T18:24:55.557761+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.568346+0000","last_change":"2023-11-28T18:24:55.565390+0000","last_active":"2023-11-28T18:24:55.568346+0000","last_peered":"2023-11-28T18:24:55.568346+0000","last_clean":"2023-11-28T18:24:55.568346+0000","last_became_active":"2023-11-28T18:24:55.565112+0000","last_became_peered":"2023-11-28T18:24:55.565112+0000","last_unstale":"2023-11-28T18:24:55.568346+0000","last_undegraded":"2023-11-28T18:24:55.568346+0000","last_fullsized":"2023-11-28T18:24:55.568346+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:46.037101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558220+0000","last_change":"2023-11-28T18:24:55.558220+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558220+0000","last_undegraded":"2023-11-28T18:24:55.558220+0000","last_fullsized":"2023-11-28T18:24:55.558220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.569353+0000","last_change":"2023-11-28T18:24:55.567103+0000","last_active":"2023-11-28T18:24:55.569353+0000","last_peered":"2023-11-28T18:24:55.569353+0000","last_clean":"2023-11-28T18:24:55.569353+0000","last_became_active":"2023-11-28T18:24:55.566687+0000","last_became_peered":"2023-11-28T18:24:55.566687+0000","last_unstale":"2023-11-28T18:24:55.569353+0000","last_undegraded":"2023-11-28T18:24:55.569353+0000","last_fullsized":"2023-11-28T18:24:55.569353+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:06.790640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.556256+0000","last_change":"2023-11-28T18:24:55.556256+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.556256+0000","last_undegraded":"2023-11-28T18:24:55.556256+0000","last_fullsized":"2023-11-28T18:24:55.556256+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.577756+0000","last_change":"2023-11-28T18:24:55.572730+0000","last_active":"2023-11-28T18:24:55.577756+0000","last_peered":"2023-11-28T18:24:55.577756+0000","last_clean":"2023-11-28T18:24:55.577756+0000","last_became_active":"2023-11-28T18:24:55.572440+0000","last_became_peered":"2023-11-28T18:24:55.572440+0000","last_unstale":"2023-11-28T18:24:55.577756+0000","last_undegraded":"2023-11-28T18:24:55.577756+0000","last_fullsized":"2023-11-28T18:24:55.577756+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:44.881911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.562201+0000","last_change":"2023-11-28T18:24:55.562201+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.562201+0000","last_undegraded":"2023-11-28T18:24:55.562201+0000","last_fullsized":"2023-11-28T18:24:55.562201+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.569553+0000","last_change":"2023-11-28T18:24:55.564212+0000","last_active":"2023-11-28T18:24:55.569553+0000","last_peered":"2023-11-28T18:24:55.569553+0000","last_clean":"2023-11-28T18:24:55.569553+0000","last_became_active":"2023-11-28T18:24:55.561715+0000","last_became_peered":"2023-11-28T18:24:55.561715+0000","last_unstale":"2023-11-28T18:24:55.569553+0000","last_undegraded":"2023-11-28T18:24:55.569553+0000","last_fullsized":"2023-11-28T18:24:55.569553+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:42:09.903901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.564919+0000","last_change":"2023-11-28T18:24:55.564919+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.564919+0000","last_undegraded":"2023-11-28T18:24:55.564919+0000","last_fullsized":"2023-11-28T18:24:55.564919+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.584030+0000","last_change":"2023-11-28T18:24:55.582776+0000","last_active":"2023-11-28T18:24:55.584030+0000","last_peered":"2023-11-28T18:24:55.584030+0000","last_clean":"2023-11-28T18:24:55.584030+0000","last_became_active":"2023-11-28T18:24:55.582457+0000","last_became_peered":"2023-11-28T18:24:55.582457+0000","last_unstale":"2023-11-28T18:24:55.584030+0000","last_undegraded":"2023-11-28T18:24:55.584030+0000","last_fullsized":"2023-11-28T18:24:55.584030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:29:45.071843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T18:24:54.547971+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:24:54.547971+0000","last_peered":"2023-11-28T18:24:54.547971+0000","last_clean":"2023-11-28T18:24:54.547971+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:24:54.547971+0000","last_undegraded":"2023-11-28T18:24:54.547971+0000","last_fullsized":"2023-11-28T18:24:54.547971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":765469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.63}]},{"osd":1,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.68999999999999995}]},{"osd":2,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.411,"5min":2.411,"15min":2.411},"last":0.59499999999999997}]},{"osd":3,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.3650000000000002,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":0.747}]},{"osd":4,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.46200000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.55400000000000005}]},{"osd":6,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.61299999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":175101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.55400000000000005}]},{"osd":1,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":4.2389999999999999,"5min":4.2389999999999999,"15min":4.2389999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":4.218,"5min":4.218,"15min":4.218},"last":0.67100000000000004}]},{"osd":2,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.69799999999999995}]},{"osd":3,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.64500000000000002}]},{"osd":4,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.623},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.71799999999999997}]},{"osd":5,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.73999999999999999}]},{"osd":7,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.752},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.84399999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":175101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.8929999999999998,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.9140000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":0.69399999999999995}]},{"osd":1,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.4809999999999999,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.495}]},{"osd":2,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":3.9510000000000001,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":4.0149999999999997,"5min":4.0149999999999997,"15min":4.0149999999999997},"last":0.58999999999999997}]},{"osd":3,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":4.0869999999999997,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.9900000000000002,"5min":3.9900000000000002,"15min":3.9900000000000002},"last":0.63800000000000001}]},{"osd":4,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.44},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.67500000000000004}]},{"osd":6,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.64700000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.66300000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":175101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":2.363,"5min":2.363,"15min":2.363},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.49399999999999999}]},{"osd":1,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.64200000000000002}]},{"osd":2,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.752},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":0.73499999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.60899999999999999}]},{"osd":5,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.71999999999999997}]},{"osd":6,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.56499999999999995}]},{"osd":7,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.69499999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":168506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.40400000000000003}]},{"osd":1,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46899999999999997}]},{"osd":2,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.49399999999999999}]},{"osd":4,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.64900000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.5019999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.60299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":168506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.56000000000000005}]},{"osd":1,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.63800000000000001}]},{"osd":3,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.66200000000000003}]},{"osd":4,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.71999999999999997}]},{"osd":5,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":7.4820000000000002,"5min":7.4820000000000002,"15min":7.4820000000000002},"last":0.83199999999999996}]},{"osd":6,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.749},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":3.9380000000000002,"5min":3.9380000000000002,"15min":3.9380000000000002},"last":0.77800000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":4.798,"5min":4.798,"15min":4.798},"last":0.82099999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":175101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.436}]},{"osd":2,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.53000000000000003}]},{"osd":3,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.502}]},{"osd":4,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.65100000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.59399999999999997}]},{"osd":6,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.72599999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":765469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.442},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.45800000000000002}]},{"osd":2,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.42599999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.48899999999999999}]},{"osd":4,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.624},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.64500000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.68700000000000006}]},{"osd":6,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.66100000000000003}]},{"osd":7,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.57099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:25:00.650 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:25:00.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d870f640 1 Processor -- start 2023-11-28T18:25:00.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d870f640 1 -- start start 2023-11-28T18:25:00.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d870f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51d00caf60 0x7f51d00cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d870f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f51d00ad850 0x7f51d00701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d870f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d006bf50 con 0x7f51d0159f40 2023-11-28T18:25:00.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d870f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d00cb3c0 con 0x7f51d00ad850 2023-11-28T18:25:00.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d870f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d015a320 con 0x7f51d00caf60 2023-11-28T18:25:00.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d64ab640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f51d0159f40 0x7f51d00ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54274/0 (socket says 172.21.15.105:54274) 2023-11-28T18:25:00.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d64ab640 1 -- 172.21.15.105:0/3982449587 learned_addr learned my addr 172.21.15.105:0/3982449587 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:00.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d54a9640 1 -- 172.21.15.105:0/3982449587 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1168901425 0 0) 0x7f51d006bf50 con 0x7f51d0159f40 2023-11-28T18:25:00.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d5caa640 1 --2- 172.21.15.105:0/3982449587 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51d00caf60 0x7f51d00cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d6cac640 1 --2- 172.21.15.105:0/3982449587 >> v2:172.21.15.148:3300/0 conn(0x7f51d00ad850 0x7f51d00701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d54a9640 1 -- 172.21.15.105:0/3982449587 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51bc003530 con 0x7f51d0159f40 2023-11-28T18:25:00.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d54a9640 1 -- 172.21.15.105:0/3982449587 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2584829092 0 0) 0x7f51bc003530 con 0x7f51d0159f40 2023-11-28T18:25:00.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d54a9640 1 -- 172.21.15.105:0/3982449587 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51d00caf60 msgr2=0x7f51d00cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:00.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d54a9640 1 --2- 172.21.15.105:0/3982449587 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51d00caf60 0x7f51d00cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d54a9640 1 -- 172.21.15.105:0/3982449587 >> v2:172.21.15.148:3300/0 conn(0x7f51d00ad850 msgr2=0x7f51d00701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:00.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d54a9640 1 --2- 172.21.15.105:0/3982449587 >> v2:172.21.15.148:3300/0 conn(0x7f51d00ad850 0x7f51d00701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d54a9640 1 -- 172.21.15.105:0/3982449587 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d0071380 con 0x7f51d0159f40 2023-11-28T18:25:00.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d54a9640 1 -- 172.21.15.105:0/3982449587 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f51cc001680 con 0x7f51d0159f40 2023-11-28T18:25:00.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d54a9640 1 -- 172.21.15.105:0/3982449587 >> v1:172.21.15.105:6789/0 conn(0x7f51d0159f40 legacy=0x7f51d00ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:00.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d54a9640 1 --2- 172.21.15.105:0/3982449587 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51d00caf60 0x7f51bc003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d54a9640 1 --2- 172.21.15.105:0/3982449587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51bc003f00 0x7f51bc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d54a9640 1 --2- 172.21.15.105:0/3982449587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51bc004d10 0x7f51bc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d54a9640 1 -- 172.21.15.105:0/3982449587 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d00cb3c0 con 0x7f51bc004d10 2023-11-28T18:25:00.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d54a9640 1 -- 172.21.15.105:0/3982449587 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d015a320 con 0x7f51bc003f00 2023-11-28T18:25:00.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d54a9640 1 -- 172.21.15.105:0/3982449587 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc00b9e0 con 0x7f51d00caf60 2023-11-28T18:25:00.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d64ab640 1 --2- 172.21.15.105:0/3982449587 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51d00caf60 0x7f51bc003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d6cac640 1 --2- 172.21.15.105:0/3982449587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51bc004d10 0x7f51bc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d5caa640 1 --2- 172.21.15.105:0/3982449587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51bc003f00 0x7f51bc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d6cac640 1 -- 172.21.15.105:0/3982449587 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51d00caf60 msgr2=0x7f51bc003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:00.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d6cac640 1 --2- 172.21.15.105:0/3982449587 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51d00caf60 0x7f51bc003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d6cac640 1 -- 172.21.15.105:0/3982449587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51bc003f00 msgr2=0x7f51bc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:00.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d6cac640 1 --2- 172.21.15.105:0/3982449587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51bc003f00 0x7f51bc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.818+0000 7f51d6cac640 1 -- 172.21.15.105:0/3982449587 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc003530 con 0x7f51bc004d10 2023-11-28T18:25:00.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.822+0000 7f51d6cac640 1 --2- 172.21.15.105:0/3982449587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51bc004d10 0x7f51bc005190 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009bd0 tx=0x7f51c801ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1953b55c28011ddf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:00.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.822+0000 7f51d54a9640 1 -- 172.21.15.105:0/3982449587 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c80a7ea0 con 0x7f51bc004d10 2023-11-28T18:25:00.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.822+0000 7f51d54a9640 1 -- 172.21.15.105:0/3982449587 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c800f040 con 0x7f51bc004d10 2023-11-28T18:25:00.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.822+0000 7f51d54a9640 1 -- 172.21.15.105:0/3982449587 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c80af680 con 0x7f51bc004d10 2023-11-28T18:25:00.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.822+0000 7f51d870f640 1 -- 172.21.15.105:0/3982449587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51bc004d10 msgr2=0x7f51bc005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:00.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.822+0000 7f51d870f640 1 --2- 172.21.15.105:0/3982449587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51bc004d10 0x7f51bc005190 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009bd0 tx=0x7f51c801ee90 comp rx=0 tx=0).stop 2023-11-28T18:25:00.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.822+0000 7f51d6cac640 1 -- 172.21.15.105:0/3982449587 reap_dead start 2023-11-28T18:25:00.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.822+0000 7f51d870f640 1 -- 172.21.15.105:0/3982449587 shutdown_connections 2023-11-28T18:25:00.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.822+0000 7f51d870f640 1 -- 172.21.15.105:0/3982449587 >> 172.21.15.105:0/3982449587 conn(0x7f51d00ceb90 msgr2=0x7f51d00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:00.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.822+0000 7f51d870f640 1 -- 172.21.15.105:0/3982449587 shutdown_connections 2023-11-28T18:25:00.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.822+0000 7f51d870f640 1 -- 172.21.15.105:0/3982449587 wait complete. 2023-11-28T18:25:00.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51d870f640 1 Processor -- start 2023-11-28T18:25:00.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51d870f640 1 -- start start 2023-11-28T18:25:00.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51d870f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51bc004d10 0x7f51d0141170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51d870f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51d015ccc0 0x7f51d0141830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51d870f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51d0141d70 0x7f51d01585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51d870f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d0158180 con 0x7f51d0141d70 2023-11-28T18:25:00.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51d870f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d009caf0 con 0x7f51bc004d10 2023-11-28T18:25:00.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51d870f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d0157db0 con 0x7f51d015ccc0 2023-11-28T18:25:00.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51d6cac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51d0141d70 0x7f51d01585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51d6cac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51d0141d70 0x7f51d01585a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36954/0 (socket says 172.21.15.105:36954) 2023-11-28T18:25:00.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51d6cac640 1 -- 172.21.15.105:0/282793710 learned_addr learned my addr 172.21.15.105:0/282793710 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:00.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51d5caa640 1 --2- 172.21.15.105:0/282793710 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51d015ccc0 0x7f51d0141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51d64ab640 1 --2- 172.21.15.105:0/282793710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51bc004d10 0x7f51d0141170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51d6cac640 1 -- 172.21.15.105:0/282793710 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51d015ccc0 msgr2=0x7f51d0141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:00.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51d6cac640 1 --2- 172.21.15.105:0/282793710 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51d015ccc0 0x7f51d0141830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51d6cac640 1 -- 172.21.15.105:0/282793710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51bc004d10 msgr2=0x7f51d0141170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:00.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51d6cac640 1 --2- 172.21.15.105:0/282793710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51bc004d10 0x7f51d0141170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51d6cac640 1 -- 172.21.15.105:0/282793710 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d015d140 con 0x7f51d0141d70 2023-11-28T18:25:00.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51d6cac640 1 --2- 172.21.15.105:0/282793710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51d0141d70 0x7f51d01585a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f51c80b2040 tx=0x7f51c8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:00.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51c6ffd640 1 -- 172.21.15.105:0/282793710 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c80b0c00 con 0x7f51d0141d70 2023-11-28T18:25:00.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51c6ffd640 1 -- 172.21.15.105:0/282793710 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c80b0d60 con 0x7f51d0141d70 2023-11-28T18:25:00.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51d870f640 1 -- 172.21.15.105:0/282793710 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d009c530 con 0x7f51d0141d70 2023-11-28T18:25:00.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51c6ffd640 1 -- 172.21.15.105:0/282793710 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8003be0 con 0x7f51d0141d70 2023-11-28T18:25:00.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.826+0000 7f51d870f640 1 -- 172.21.15.105:0/282793710 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d0158ae0 con 0x7f51d0141d70 2023-11-28T18:25:00.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.830+0000 7f51d870f640 1 -- 172.21.15.105:0/282793710 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d0067570 con 0x7f51d0141d70 2023-11-28T18:25:00.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.830+0000 7f51c6ffd640 1 -- 172.21.15.105:0/282793710 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f51c80b6040 con 0x7f51d0141d70 2023-11-28T18:25:00.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.830+0000 7f51c6ffd640 1 --2- 172.21.15.105:0/282793710 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f51b4066900 0x7f51b4068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:00.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.830+0000 7f51c6ffd640 1 -- 172.21.15.105:0/282793710 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f51c81206e0 con 0x7f51d0141d70 2023-11-28T18:25:00.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.830+0000 7f51d64ab640 1 --2- 172.21.15.105:0/282793710 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f51b4066900 0x7f51b4068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:00.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.830+0000 7f51d64ab640 1 --2- 172.21.15.105:0/282793710 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f51b4066900 0x7f51b4068dc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f51cc000c10 tx=0x7f51cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.834+0000 7f51c6ffd640 1 -- 172.21.15.105:0/282793710 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f51c80ed7f0 con 0x7f51d0141d70 2023-11-28T18:25:00.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.982+0000 7f51d870f640 1 -- 172.21.15.105:0/282793710 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f51d006c480 con 0x7f51b4066900 2023-11-28T18:25:00.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.986+0000 7f51c6ffd640 1 -- 172.21.15.105:0/282793710 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284236 (secure 0 0 0) 0x7f51cc0015e0 con 0x7f51b4066900 2023-11-28T18:25:00.996 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:00.997 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:25:00.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.990+0000 7f51d870f640 1 -- 172.21.15.105:0/282793710 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f51b4066900 msgr2=0x7f51b4068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:00.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.990+0000 7f51d870f640 1 --2- 172.21.15.105:0/282793710 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f51b4066900 0x7f51b4068dc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f51cc000c10 tx=0x7f51cc009040 comp rx=0 tx=0).stop 2023-11-28T18:25:00.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.990+0000 7f51d870f640 1 -- 172.21.15.105:0/282793710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51d0141d70 msgr2=0x7f51d01585a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:00.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.990+0000 7f51d870f640 1 --2- 172.21.15.105:0/282793710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51d0141d70 0x7f51d01585a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f51c80b2040 tx=0x7f51c8009750 comp rx=0 tx=0).stop 2023-11-28T18:25:00.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.990+0000 7f51d870f640 1 -- 172.21.15.105:0/282793710 shutdown_connections 2023-11-28T18:25:00.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.990+0000 7f51d870f640 1 --2- 172.21.15.105:0/282793710 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f51b4066900 0x7f51b4068dc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.990+0000 7f51d870f640 1 --2- 172.21.15.105:0/282793710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51d0141d70 0x7f51d01585a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.990+0000 7f51d870f640 1 --2- 172.21.15.105:0/282793710 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51d015ccc0 0x7f51d0141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.990+0000 7f51d870f640 1 --2- 172.21.15.105:0/282793710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51bc004d10 0x7f51d0141170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:00.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.990+0000 7f51d870f640 1 -- 172.21.15.105:0/282793710 >> 172.21.15.105:0/282793710 conn(0x7f51d00ceb90 msgr2=0x7f51d0154e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:00.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.990+0000 7f51d870f640 1 -- 172.21.15.105:0/282793710 shutdown_connections 2023-11-28T18:25:01.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:00.990+0000 7f51d870f640 1 -- 172.21.15.105:0/282793710 wait complete. 2023-11-28T18:25:01.016 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2023-11-28T18:24:59.286911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220536,"kb_used_data":4368,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754184,"statfs":{"total":773094113280,"available":772868284416,"internally_reserved":0,"allocated":4472832,"data_stored":2568354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.747004"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.555352+0000","last_change":"2023-11-28T18:24:55.555352+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.555352+0000","last_undegraded":"2023-11-28T18:24:55.555352+0000","last_fullsized":"2023-11-28T18:24:55.555352+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554587+0000","last_change":"2023-11-28T18:24:55.554587+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554587+0000","last_undegraded":"2023-11-28T18:24:55.554587+0000","last_fullsized":"2023-11-28T18:24:55.554587+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558901+0000","last_change":"2023-11-28T18:24:55.558901+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558901+0000","last_undegraded":"2023-11-28T18:24:55.558901+0000","last_fullsized":"2023-11-28T18:24:55.558901+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.566872+0000","last_change":"2023-11-28T18:24:55.566872+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.566872+0000","last_undegraded":"2023-11-28T18:24:55.566872+0000","last_fullsized":"2023-11-28T18:24:55.566872+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.565232+0000","last_change":"2023-11-28T18:24:55.565232+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.565232+0000","last_undegraded":"2023-11-28T18:24:55.565232+0000","last_fullsized":"2023-11-28T18:24:55.565232+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.557666+0000","last_change":"2023-11-28T18:24:55.557666+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.557666+0000","last_undegraded":"2023-11-28T18:24:55.557666+0000","last_fullsized":"2023-11-28T18:24:55.557666+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.564284+0000","last_change":"2023-11-28T18:24:55.564284+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.564284+0000","last_undegraded":"2023-11-28T18:24:55.564284+0000","last_fullsized":"2023-11-28T18:24:55.564284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554637+0000","last_change":"2023-11-28T18:24:55.554637+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554637+0000","last_undegraded":"2023-11-28T18:24:55.554637+0000","last_fullsized":"2023-11-28T18:24:55.554637+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.556678+0000","last_change":"2023-11-28T18:24:55.556678+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.556678+0000","last_undegraded":"2023-11-28T18:24:55.556678+0000","last_fullsized":"2023-11-28T18:24:55.556678+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.566281+0000","last_change":"2023-11-28T18:24:55.566281+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.566281+0000","last_undegraded":"2023-11-28T18:24:55.566281+0000","last_fullsized":"2023-11-28T18:24:55.566281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.568480+0000","last_change":"2023-11-28T18:24:55.568480+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.568480+0000","last_undegraded":"2023-11-28T18:24:55.568480+0000","last_fullsized":"2023-11-28T18:24:55.568480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.556143+0000","last_change":"2023-11-28T18:24:55.556143+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.556143+0000","last_undegraded":"2023-11-28T18:24:55.556143+0000","last_fullsized":"2023-11-28T18:24:55.556143+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563047+0000","last_change":"2023-11-28T18:24:55.563047+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563047+0000","last_undegraded":"2023-11-28T18:24:55.563047+0000","last_fullsized":"2023-11-28T18:24:55.563047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.556775+0000","last_change":"2023-11-28T18:24:55.556775+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.556775+0000","last_undegraded":"2023-11-28T18:24:55.556775+0000","last_fullsized":"2023-11-28T18:24:55.556775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563722+0000","last_change":"2023-11-28T18:24:55.563722+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563722+0000","last_undegraded":"2023-11-28T18:24:55.563722+0000","last_fullsized":"2023-11-28T18:24:55.563722+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563809+0000","last_change":"2023-11-28T18:24:55.563809+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563809+0000","last_undegraded":"2023-11-28T18:24:55.563809+0000","last_fullsized":"2023-11-28T18:24:55.563809+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.561864+0000","last_change":"2023-11-28T18:24:55.561864+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.561864+0000","last_undegraded":"2023-11-28T18:24:55.561864+0000","last_fullsized":"2023-11-28T18:24:55.561864+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554411+0000","last_change":"2023-11-28T18:24:55.554411+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554411+0000","last_undegraded":"2023-11-28T18:24:55.554411+0000","last_fullsized":"2023-11-28T18:24:55.554411+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.568017+0000","last_change":"2023-11-28T18:24:55.568017+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.568017+0000","last_undegraded":"2023-11-28T18:24:55.568017+0000","last_fullsized":"2023-11-28T18:24:55.568017+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553039+0000","last_change":"2023-11-28T18:24:55.553039+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553039+0000","last_undegraded":"2023-11-28T18:24:55.553039+0000","last_fullsized":"2023-11-28T18:24:55.553039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.562468+0000","last_change":"2023-11-28T18:24:55.562468+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.562468+0000","last_undegraded":"2023-11-28T18:24:55.562468+0000","last_fullsized":"2023-11-28T18:24:55.562468+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.555237+0000","last_change":"2023-11-28T18:24:55.555237+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.555237+0000","last_undegraded":"2023-11-28T18:24:55.555237+0000","last_fullsized":"2023-11-28T18:24:55.555237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.559896+0000","last_change":"2023-11-28T18:24:55.559896+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.559896+0000","last_undegraded":"2023-11-28T18:24:55.559896+0000","last_fullsized":"2023-11-28T18:24:55.559896+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.562708+0000","last_change":"2023-11-28T18:24:55.562708+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.562708+0000","last_undegraded":"2023-11-28T18:24:55.562708+0000","last_fullsized":"2023-11-28T18:24:55.562708+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553693+0000","last_change":"2023-11-28T18:24:55.553693+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553693+0000","last_undegraded":"2023-11-28T18:24:55.553693+0000","last_fullsized":"2023-11-28T18:24:55.553693+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.561952+0000","last_change":"2023-11-28T18:24:55.561952+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.561952+0000","last_undegraded":"2023-11-28T18:24:55.561952+0000","last_fullsized":"2023-11-28T18:24:55.561952+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553056+0000","last_change":"2023-11-28T18:24:55.553056+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553056+0000","last_undegraded":"2023-11-28T18:24:55.553056+0000","last_fullsized":"2023-11-28T18:24:55.553056+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.567550+0000","last_change":"2023-11-28T18:24:55.567550+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.567550+0000","last_undegraded":"2023-11-28T18:24:55.567550+0000","last_fullsized":"2023-11-28T18:24:55.567550+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553657+0000","last_change":"2023-11-28T18:24:55.553657+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553657+0000","last_undegraded":"2023-11-28T18:24:55.553657+0000","last_fullsized":"2023-11-28T18:24:55.553657+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553875+0000","last_change":"2023-11-28T18:24:55.553875+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553875+0000","last_undegraded":"2023-11-28T18:24:55.553875+0000","last_fullsized":"2023-11-28T18:24:55.553875+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.559065+0000","last_change":"2023-11-28T18:24:55.559065+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.559065+0000","last_undegraded":"2023-11-28T18:24:55.559065+0000","last_fullsized":"2023-11-28T18:24:55.559065+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558556+0000","last_change":"2023-11-28T18:24:55.558556+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558556+0000","last_undegraded":"2023-11-28T18:24:55.558556+0000","last_fullsized":"2023-11-28T18:24:55.558556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.561337+0000","last_change":"2023-11-28T18:24:55.561337+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.561337+0000","last_undegraded":"2023-11-28T18:24:55.561337+0000","last_fullsized":"2023-11-28T18:24:55.561337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.578117+0000","last_change":"2023-11-28T18:24:55.573384+0000","last_active":"2023-11-28T18:24:55.578117+0000","last_peered":"2023-11-28T18:24:55.578117+0000","last_clean":"2023-11-28T18:24:55.578117+0000","last_became_active":"2023-11-28T18:24:55.571607+0000","last_became_peered":"2023-11-28T18:24:55.571607+0000","last_unstale":"2023-11-28T18:24:55.578117+0000","last_undegraded":"2023-11-28T18:24:55.578117+0000","last_fullsized":"2023-11-28T18:24:55.578117+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:17:39.496716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.562352+0000","last_change":"2023-11-28T18:24:55.562352+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.562352+0000","last_undegraded":"2023-11-28T18:24:55.562352+0000","last_fullsized":"2023-11-28T18:24:55.562352+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.579114+0000","last_change":"2023-11-28T18:24:55.573927+0000","last_active":"2023-11-28T18:24:55.579114+0000","last_peered":"2023-11-28T18:24:55.579114+0000","last_clean":"2023-11-28T18:24:55.579114+0000","last_became_active":"2023-11-28T18:24:55.573623+0000","last_became_peered":"2023-11-28T18:24:55.573623+0000","last_unstale":"2023-11-28T18:24:55.579114+0000","last_undegraded":"2023-11-28T18:24:55.579114+0000","last_fullsized":"2023-11-28T18:24:55.579114+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:29:45.071843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558261+0000","last_change":"2023-11-28T18:24:55.558261+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558261+0000","last_undegraded":"2023-11-28T18:24:55.558261+0000","last_fullsized":"2023-11-28T18:24:55.558261+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.573242+0000","last_change":"2023-11-28T18:24:55.570144+0000","last_active":"2023-11-28T18:24:55.573242+0000","last_peered":"2023-11-28T18:24:55.573242+0000","last_clean":"2023-11-28T18:24:55.573242+0000","last_became_active":"2023-11-28T18:24:55.569837+0000","last_became_peered":"2023-11-28T18:24:55.569837+0000","last_unstale":"2023-11-28T18:24:55.573242+0000","last_undegraded":"2023-11-28T18:24:55.573242+0000","last_fullsized":"2023-11-28T18:24:55.573242+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:17:08.512763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554982+0000","last_change":"2023-11-28T18:24:55.554982+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554982+0000","last_undegraded":"2023-11-28T18:24:55.554982+0000","last_fullsized":"2023-11-28T18:24:55.554982+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.572198+0000","last_change":"2023-11-28T18:24:55.570052+0000","last_active":"2023-11-28T18:24:55.572198+0000","last_peered":"2023-11-28T18:24:55.572198+0000","last_clean":"2023-11-28T18:24:55.572198+0000","last_became_active":"2023-11-28T18:24:55.569712+0000","last_became_peered":"2023-11-28T18:24:55.569712+0000","last_unstale":"2023-11-28T18:24:55.572198+0000","last_undegraded":"2023-11-28T18:24:55.572198+0000","last_fullsized":"2023-11-28T18:24:55.572198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:06.899740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558633+0000","last_change":"2023-11-28T18:24:55.558633+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558633+0000","last_undegraded":"2023-11-28T18:24:55.558633+0000","last_fullsized":"2023-11-28T18:24:55.558633+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.580374+0000","last_change":"2023-11-28T18:24:55.575819+0000","last_active":"2023-11-28T18:24:55.580374+0000","last_peered":"2023-11-28T18:24:55.580374+0000","last_clean":"2023-11-28T18:24:55.580374+0000","last_became_active":"2023-11-28T18:24:55.575541+0000","last_became_peered":"2023-11-28T18:24:55.575541+0000","last_unstale":"2023-11-28T18:24:55.580374+0000","last_undegraded":"2023-11-28T18:24:55.580374+0000","last_fullsized":"2023-11-28T18:24:55.580374+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:46.037101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563170+0000","last_change":"2023-11-28T18:24:55.563170+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563170+0000","last_undegraded":"2023-11-28T18:24:55.563170+0000","last_fullsized":"2023-11-28T18:24:55.563170+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.580797+0000","last_change":"2023-11-28T18:24:55.578529+0000","last_active":"2023-11-28T18:24:55.580797+0000","last_peered":"2023-11-28T18:24:55.580797+0000","last_clean":"2023-11-28T18:24:55.580797+0000","last_became_active":"2023-11-28T18:24:55.578252+0000","last_became_peered":"2023-11-28T18:24:55.578252+0000","last_unstale":"2023-11-28T18:24:55.580797+0000","last_undegraded":"2023-11-28T18:24:55.580797+0000","last_fullsized":"2023-11-28T18:24:55.580797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:50:55.081198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.555789+0000","last_change":"2023-11-28T18:24:55.555789+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.555789+0000","last_undegraded":"2023-11-28T18:24:55.555789+0000","last_fullsized":"2023-11-28T18:24:55.555789+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.578996+0000","last_change":"2023-11-28T18:24:55.574964+0000","last_active":"2023-11-28T18:24:55.578996+0000","last_peered":"2023-11-28T18:24:55.578996+0000","last_clean":"2023-11-28T18:24:55.578996+0000","last_became_active":"2023-11-28T18:24:55.574669+0000","last_became_peered":"2023-11-28T18:24:55.574669+0000","last_unstale":"2023-11-28T18:24:55.578996+0000","last_undegraded":"2023-11-28T18:24:55.578996+0000","last_fullsized":"2023-11-28T18:24:55.578996+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:40:45.446584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554497+0000","last_change":"2023-11-28T18:24:55.554497+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554497+0000","last_undegraded":"2023-11-28T18:24:55.554497+0000","last_fullsized":"2023-11-28T18:24:55.554497+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.567976+0000","last_change":"2023-11-28T18:24:55.563267+0000","last_active":"2023-11-28T18:24:55.567976+0000","last_peered":"2023-11-28T18:24:55.567976+0000","last_clean":"2023-11-28T18:24:55.567976+0000","last_became_active":"2023-11-28T18:24:55.562979+0000","last_became_peered":"2023-11-28T18:24:55.562979+0000","last_unstale":"2023-11-28T18:24:55.567976+0000","last_undegraded":"2023-11-28T18:24:55.567976+0000","last_fullsized":"2023-11-28T18:24:55.567976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:06.790640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.565983+0000","last_change":"2023-11-28T18:24:55.565983+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.565983+0000","last_undegraded":"2023-11-28T18:24:55.565983+0000","last_fullsized":"2023-11-28T18:24:55.565983+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.578882+0000","last_change":"2023-11-28T18:24:55.572743+0000","last_active":"2023-11-28T18:24:55.578882+0000","last_peered":"2023-11-28T18:24:55.578882+0000","last_clean":"2023-11-28T18:24:55.578882+0000","last_became_active":"2023-11-28T18:24:55.571872+0000","last_became_peered":"2023-11-28T18:24:55.571872+0000","last_unstale":"2023-11-28T18:24:55.578882+0000","last_undegraded":"2023-11-28T18:24:55.578882+0000","last_fullsized":"2023-11-28T18:24:55.578882+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:14.095135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.564500+0000","last_change":"2023-11-28T18:24:55.564500+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.564500+0000","last_undegraded":"2023-11-28T18:24:55.564500+0000","last_fullsized":"2023-11-28T18:24:55.564500+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581663+0000","last_change":"2023-11-28T18:24:55.579938+0000","last_active":"2023-11-28T18:24:55.581663+0000","last_peered":"2023-11-28T18:24:55.581663+0000","last_clean":"2023-11-28T18:24:55.581663+0000","last_became_active":"2023-11-28T18:24:55.579667+0000","last_became_peered":"2023-11-28T18:24:55.579667+0000","last_unstale":"2023-11-28T18:24:55.581663+0000","last_undegraded":"2023-11-28T18:24:55.581663+0000","last_fullsized":"2023-11-28T18:24:55.581663+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:14.095135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.556171+0000","last_change":"2023-11-28T18:24:55.556171+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.556171+0000","last_undegraded":"2023-11-28T18:24:55.556171+0000","last_fullsized":"2023-11-28T18:24:55.556171+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581601+0000","last_change":"2023-11-28T18:24:55.579772+0000","last_active":"2023-11-28T18:24:55.581601+0000","last_peered":"2023-11-28T18:24:55.581601+0000","last_clean":"2023-11-28T18:24:55.581601+0000","last_became_active":"2023-11-28T18:24:55.579492+0000","last_became_peered":"2023-11-28T18:24:55.579492+0000","last_unstale":"2023-11-28T18:24:55.581601+0000","last_undegraded":"2023-11-28T18:24:55.581601+0000","last_fullsized":"2023-11-28T18:24:55.581601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:29:37.604627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558779+0000","last_change":"2023-11-28T18:24:55.558779+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558779+0000","last_undegraded":"2023-11-28T18:24:55.558779+0000","last_fullsized":"2023-11-28T18:24:55.558779+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581425+0000","last_change":"2023-11-28T18:24:55.576648+0000","last_active":"2023-11-28T18:24:55.581425+0000","last_peered":"2023-11-28T18:24:55.581425+0000","last_clean":"2023-11-28T18:24:55.581425+0000","last_became_active":"2023-11-28T18:24:55.576382+0000","last_became_peered":"2023-11-28T18:24:55.576382+0000","last_unstale":"2023-11-28T18:24:55.581425+0000","last_undegraded":"2023-11-28T18:24:55.581425+0000","last_fullsized":"2023-11-28T18:24:55.581425+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:29:37.604627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558183+0000","last_change":"2023-11-28T18:24:55.558183+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558183+0000","last_undegraded":"2023-11-28T18:24:55.558183+0000","last_fullsized":"2023-11-28T18:24:55.558183+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.569235+0000","last_change":"2023-11-28T18:24:55.565583+0000","last_active":"2023-11-28T18:24:55.569235+0000","last_peered":"2023-11-28T18:24:55.569235+0000","last_clean":"2023-11-28T18:24:55.569235+0000","last_became_active":"2023-11-28T18:24:55.565335+0000","last_became_peered":"2023-11-28T18:24:55.565335+0000","last_unstale":"2023-11-28T18:24:55.569235+0000","last_undegraded":"2023-11-28T18:24:55.569235+0000","last_fullsized":"2023-11-28T18:24:55.569235+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:47:09.714446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553365+0000","last_change":"2023-11-28T18:24:55.553365+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553365+0000","last_undegraded":"2023-11-28T18:24:55.553365+0000","last_fullsized":"2023-11-28T18:24:55.553365+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.579362+0000","last_change":"2023-11-28T18:24:55.575066+0000","last_active":"2023-11-28T18:24:55.579362+0000","last_peered":"2023-11-28T18:24:55.579362+0000","last_clean":"2023-11-28T18:24:55.579362+0000","last_became_active":"2023-11-28T18:24:55.574770+0000","last_became_peered":"2023-11-28T18:24:55.574770+0000","last_unstale":"2023-11-28T18:24:55.579362+0000","last_undegraded":"2023-11-28T18:24:55.579362+0000","last_fullsized":"2023-11-28T18:24:55.579362+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:44.881911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.560801+0000","last_change":"2023-11-28T18:24:55.560801+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.560801+0000","last_undegraded":"2023-11-28T18:24:55.560801+0000","last_fullsized":"2023-11-28T18:24:55.560801+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.563818+0000","last_change":"2023-11-28T18:24:55.561985+0000","last_active":"2023-11-28T18:24:55.563818+0000","last_peered":"2023-11-28T18:24:55.563818+0000","last_clean":"2023-11-28T18:24:55.563818+0000","last_became_active":"2023-11-28T18:24:55.561673+0000","last_became_peered":"2023-11-28T18:24:55.561673+0000","last_unstale":"2023-11-28T18:24:55.563818+0000","last_undegraded":"2023-11-28T18:24:55.563818+0000","last_fullsized":"2023-11-28T18:24:55.563818+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:15:50.678362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554008+0000","last_change":"2023-11-28T18:24:55.554008+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554008+0000","last_undegraded":"2023-11-28T18:24:55.554008+0000","last_fullsized":"2023-11-28T18:24:55.554008+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.576918+0000","last_change":"2023-11-28T18:24:55.572674+0000","last_active":"2023-11-28T18:24:55.576918+0000","last_peered":"2023-11-28T18:24:55.576918+0000","last_clean":"2023-11-28T18:24:55.576918+0000","last_became_active":"2023-11-28T18:24:55.572209+0000","last_became_peered":"2023-11-28T18:24:55.572209+0000","last_unstale":"2023-11-28T18:24:55.576918+0000","last_undegraded":"2023-11-28T18:24:55.576918+0000","last_fullsized":"2023-11-28T18:24:55.576918+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:17:08.512763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553441+0000","last_change":"2023-11-28T18:24:55.553441+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553441+0000","last_undegraded":"2023-11-28T18:24:55.553441+0000","last_fullsized":"2023-11-28T18:24:55.553441+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.580400+0000","last_change":"2023-11-28T18:24:55.575893+0000","last_active":"2023-11-28T18:24:55.580400+0000","last_peered":"2023-11-28T18:24:55.580400+0000","last_clean":"2023-11-28T18:24:55.580400+0000","last_became_active":"2023-11-28T18:24:55.575425+0000","last_became_peered":"2023-11-28T18:24:55.575425+0000","last_unstale":"2023-11-28T18:24:55.580400+0000","last_undegraded":"2023-11-28T18:24:55.580400+0000","last_fullsized":"2023-11-28T18:24:55.580400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:42:09.903901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.568047+0000","last_change":"2023-11-28T18:24:55.568047+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.568047+0000","last_undegraded":"2023-11-28T18:24:55.568047+0000","last_fullsized":"2023-11-28T18:24:55.568047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.583021+0000","last_change":"2023-11-28T18:24:55.579138+0000","last_active":"2023-11-28T18:24:55.583021+0000","last_peered":"2023-11-28T18:24:55.583021+0000","last_clean":"2023-11-28T18:24:55.583021+0000","last_became_active":"2023-11-28T18:24:55.578827+0000","last_became_peered":"2023-11-28T18:24:55.578827+0000","last_unstale":"2023-11-28T18:24:55.583021+0000","last_undegraded":"2023-11-28T18:24:55.583021+0000","last_fullsized":"2023-11-28T18:24:55.583021+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:55:08.887224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558780+0000","last_change":"2023-11-28T18:24:55.558780+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558780+0000","last_undegraded":"2023-11-28T18:24:55.558780+0000","last_fullsized":"2023-11-28T18:24:55.558780+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581919+0000","last_change":"2023-11-28T18:24:55.578515+0000","last_active":"2023-11-28T18:24:55.581919+0000","last_peered":"2023-11-28T18:24:55.581919+0000","last_clean":"2023-11-28T18:24:55.581919+0000","last_became_active":"2023-11-28T18:24:55.578256+0000","last_became_peered":"2023-11-28T18:24:55.578256+0000","last_unstale":"2023-11-28T18:24:55.581919+0000","last_undegraded":"2023-11-28T18:24:55.581919+0000","last_fullsized":"2023-11-28T18:24:55.581919+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:34.823798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.569066+0000","last_change":"2023-11-28T18:24:55.569066+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.569066+0000","last_undegraded":"2023-11-28T18:24:55.569066+0000","last_fullsized":"2023-11-28T18:24:55.569066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.579484+0000","last_change":"2023-11-28T18:24:55.575784+0000","last_active":"2023-11-28T18:24:55.579484+0000","last_peered":"2023-11-28T18:24:55.579484+0000","last_clean":"2023-11-28T18:24:55.579484+0000","last_became_active":"2023-11-28T18:24:55.575518+0000","last_became_peered":"2023-11-28T18:24:55.575518+0000","last_unstale":"2023-11-28T18:24:55.579484+0000","last_undegraded":"2023-11-28T18:24:55.579484+0000","last_fullsized":"2023-11-28T18:24:55.579484+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:55:08.887224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563340+0000","last_change":"2023-11-28T18:24:55.563340+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563340+0000","last_undegraded":"2023-11-28T18:24:55.563340+0000","last_fullsized":"2023-11-28T18:24:55.563340+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581809+0000","last_change":"2023-11-28T18:24:55.578008+0000","last_active":"2023-11-28T18:24:55.581809+0000","last_peered":"2023-11-28T18:24:55.581809+0000","last_clean":"2023-11-28T18:24:55.581809+0000","last_became_active":"2023-11-28T18:24:55.577782+0000","last_became_peered":"2023-11-28T18:24:55.577782+0000","last_unstale":"2023-11-28T18:24:55.581809+0000","last_undegraded":"2023-11-28T18:24:55.581809+0000","last_fullsized":"2023-11-28T18:24:55.581809+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:20:31.030439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.557289+0000","last_change":"2023-11-28T18:24:55.557289+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.557289+0000","last_undegraded":"2023-11-28T18:24:55.557289+0000","last_fullsized":"2023-11-28T18:24:55.557289+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581727+0000","last_change":"2023-11-28T18:24:55.576784+0000","last_active":"2023-11-28T18:24:55.581727+0000","last_peered":"2023-11-28T18:24:55.581727+0000","last_clean":"2023-11-28T18:24:55.581727+0000","last_became_active":"2023-11-28T18:24:55.576508+0000","last_became_peered":"2023-11-28T18:24:55.576508+0000","last_unstale":"2023-11-28T18:24:55.581727+0000","last_undegraded":"2023-11-28T18:24:55.581727+0000","last_fullsized":"2023-11-28T18:24:55.581727+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:20:38.032106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563243+0000","last_change":"2023-11-28T18:24:55.563243+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563243+0000","last_undegraded":"2023-11-28T18:24:55.563243+0000","last_fullsized":"2023-11-28T18:24:55.563243+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.583609+0000","last_change":"2023-11-28T18:24:55.582487+0000","last_active":"2023-11-28T18:24:55.583609+0000","last_peered":"2023-11-28T18:24:55.583609+0000","last_clean":"2023-11-28T18:24:55.583609+0000","last_became_active":"2023-11-28T18:24:55.582038+0000","last_became_peered":"2023-11-28T18:24:55.582038+0000","last_unstale":"2023-11-28T18:24:55.583609+0000","last_undegraded":"2023-11-28T18:24:55.583609+0000","last_fullsized":"2023-11-28T18:24:55.583609+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:46.037101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553428+0000","last_change":"2023-11-28T18:24:55.553428+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553428+0000","last_undegraded":"2023-11-28T18:24:55.553428+0000","last_fullsized":"2023-11-28T18:24:55.553428+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581789+0000","last_change":"2023-11-28T18:24:55.577155+0000","last_active":"2023-11-28T18:24:55.581789+0000","last_peered":"2023-11-28T18:24:55.581789+0000","last_clean":"2023-11-28T18:24:55.581789+0000","last_became_active":"2023-11-28T18:24:55.576899+0000","last_became_peered":"2023-11-28T18:24:55.576899+0000","last_unstale":"2023-11-28T18:24:55.581789+0000","last_undegraded":"2023-11-28T18:24:55.581789+0000","last_fullsized":"2023-11-28T18:24:55.581789+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:18:46.680968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558068+0000","last_change":"2023-11-28T18:24:55.558068+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558068+0000","last_undegraded":"2023-11-28T18:24:55.558068+0000","last_fullsized":"2023-11-28T18:24:55.558068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581668+0000","last_change":"2023-11-28T18:24:55.579931+0000","last_active":"2023-11-28T18:24:55.581668+0000","last_peered":"2023-11-28T18:24:55.581668+0000","last_clean":"2023-11-28T18:24:55.581668+0000","last_became_active":"2023-11-28T18:24:55.579568+0000","last_became_peered":"2023-11-28T18:24:55.579568+0000","last_unstale":"2023-11-28T18:24:55.581668+0000","last_undegraded":"2023-11-28T18:24:55.581668+0000","last_fullsized":"2023-11-28T18:24:55.581668+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:47:09.714446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563929+0000","last_change":"2023-11-28T18:24:55.563929+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563929+0000","last_undegraded":"2023-11-28T18:24:55.563929+0000","last_fullsized":"2023-11-28T18:24:55.563929+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.584207+0000","last_change":"2023-11-28T18:24:55.580960+0000","last_active":"2023-11-28T18:24:55.584207+0000","last_peered":"2023-11-28T18:24:55.584207+0000","last_clean":"2023-11-28T18:24:55.584207+0000","last_became_active":"2023-11-28T18:24:55.580717+0000","last_became_peered":"2023-11-28T18:24:55.580717+0000","last_unstale":"2023-11-28T18:24:55.584207+0000","last_undegraded":"2023-11-28T18:24:55.584207+0000","last_fullsized":"2023-11-28T18:24:55.584207+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:15:07.052209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553436+0000","last_change":"2023-11-28T18:24:55.553436+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553436+0000","last_undegraded":"2023-11-28T18:24:55.553436+0000","last_fullsized":"2023-11-28T18:24:55.553436+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.573984+0000","last_change":"2023-11-28T18:24:55.569035+0000","last_active":"2023-11-28T18:24:55.573984+0000","last_peered":"2023-11-28T18:24:55.573984+0000","last_clean":"2023-11-28T18:24:55.573984+0000","last_became_active":"2023-11-28T18:24:55.568744+0000","last_became_peered":"2023-11-28T18:24:55.568744+0000","last_unstale":"2023-11-28T18:24:55.573984+0000","last_undegraded":"2023-11-28T18:24:55.573984+0000","last_fullsized":"2023-11-28T18:24:55.573984+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:47:09.714446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.557761+0000","last_change":"2023-11-28T18:24:55.557761+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.557761+0000","last_undegraded":"2023-11-28T18:24:55.557761+0000","last_fullsized":"2023-11-28T18:24:55.557761+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.568346+0000","last_change":"2023-11-28T18:24:55.565390+0000","last_active":"2023-11-28T18:24:55.568346+0000","last_peered":"2023-11-28T18:24:55.568346+0000","last_clean":"2023-11-28T18:24:55.568346+0000","last_became_active":"2023-11-28T18:24:55.565112+0000","last_became_peered":"2023-11-28T18:24:55.565112+0000","last_unstale":"2023-11-28T18:24:55.568346+0000","last_undegraded":"2023-11-28T18:24:55.568346+0000","last_fullsized":"2023-11-28T18:24:55.568346+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:46.037101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558220+0000","last_change":"2023-11-28T18:24:55.558220+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558220+0000","last_undegraded":"2023-11-28T18:24:55.558220+0000","last_fullsized":"2023-11-28T18:24:55.558220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.569353+0000","last_change":"2023-11-28T18:24:55.567103+0000","last_active":"2023-11-28T18:24:55.569353+0000","last_peered":"2023-11-28T18:24:55.569353+0000","last_clean":"2023-11-28T18:24:55.569353+0000","last_became_active":"2023-11-28T18:24:55.566687+0000","last_became_peered":"2023-11-28T18:24:55.566687+0000","last_unstale":"2023-11-28T18:24:55.569353+0000","last_undegraded":"2023-11-28T18:24:55.569353+0000","last_fullsized":"2023-11-28T18:24:55.569353+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:06.790640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.556256+0000","last_change":"2023-11-28T18:24:55.556256+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.556256+0000","last_undegraded":"2023-11-28T18:24:55.556256+0000","last_fullsized":"2023-11-28T18:24:55.556256+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.577756+0000","last_change":"2023-11-28T18:24:55.572730+0000","last_active":"2023-11-28T18:24:55.577756+0000","last_peered":"2023-11-28T18:24:55.577756+0000","last_clean":"2023-11-28T18:24:55.577756+0000","last_became_active":"2023-11-28T18:24:55.572440+0000","last_became_peered":"2023-11-28T18:24:55.572440+0000","last_unstale":"2023-11-28T18:24:55.577756+0000","last_undegraded":"2023-11-28T18:24:55.577756+0000","last_fullsized":"2023-11-28T18:24:55.577756+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:44.881911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.562201+0000","last_change":"2023-11-28T18:24:55.562201+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.562201+0000","last_undegraded":"2023-11-28T18:24:55.562201+0000","last_fullsized":"2023-11-28T18:24:55.562201+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.569553+0000","last_change":"2023-11-28T18:24:55.564212+0000","last_active":"2023-11-28T18:24:55.569553+0000","last_peered":"2023-11-28T18:24:55.569553+0000","last_clean":"2023-11-28T18:24:55.569553+0000","last_became_active":"2023-11-28T18:24:55.561715+0000","last_became_peered":"2023-11-28T18:24:55.561715+0000","last_unstale":"2023-11-28T18:24:55.569553+0000","last_undegraded":"2023-11-28T18:24:55.569553+0000","last_fullsized":"2023-11-28T18:24:55.569553+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:42:09.903901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.564919+0000","last_change":"2023-11-28T18:24:55.564919+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.564919+0000","last_undegraded":"2023-11-28T18:24:55.564919+0000","last_fullsized":"2023-11-28T18:24:55.564919+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.584030+0000","last_change":"2023-11-28T18:24:55.582776+0000","last_active":"2023-11-28T18:24:55.584030+0000","last_peered":"2023-11-28T18:24:55.584030+0000","last_clean":"2023-11-28T18:24:55.584030+0000","last_became_active":"2023-11-28T18:24:55.582457+0000","last_became_peered":"2023-11-28T18:24:55.582457+0000","last_unstale":"2023-11-28T18:24:55.584030+0000","last_undegraded":"2023-11-28T18:24:55.584030+0000","last_fullsized":"2023-11-28T18:24:55.584030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:29:45.071843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T18:24:54.547971+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:24:54.547971+0000","last_peered":"2023-11-28T18:24:54.547971+0000","last_clean":"2023-11-28T18:24:54.547971+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:24:54.547971+0000","last_undegraded":"2023-11-28T18:24:54.547971+0000","last_fullsized":"2023-11-28T18:24:54.547971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":765469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.63}]},{"osd":1,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.68999999999999995}]},{"osd":2,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.411,"5min":2.411,"15min":2.411},"last":0.59499999999999997}]},{"osd":3,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.3650000000000002,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":0.747}]},{"osd":4,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.46200000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.55400000000000005}]},{"osd":6,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.61299999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":175101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.55400000000000005}]},{"osd":1,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":4.2389999999999999,"5min":4.2389999999999999,"15min":4.2389999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":4.218,"5min":4.218,"15min":4.218},"last":0.67100000000000004}]},{"osd":2,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.69799999999999995}]},{"osd":3,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.64500000000000002}]},{"osd":4,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.623},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.71799999999999997}]},{"osd":5,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.73999999999999999}]},{"osd":7,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.752},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.84399999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":175101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.8929999999999998,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.9140000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":0.69399999999999995}]},{"osd":1,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.4809999999999999,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.495}]},{"osd":2,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":3.9510000000000001,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":4.0149999999999997,"5min":4.0149999999999997,"15min":4.0149999999999997},"last":0.58999999999999997}]},{"osd":3,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":4.0869999999999997,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.9900000000000002,"5min":3.9900000000000002,"15min":3.9900000000000002},"last":0.63800000000000001}]},{"osd":4,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.44},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.67500000000000004}]},{"osd":6,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.64700000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.66300000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":175101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":2.363,"5min":2.363,"15min":2.363},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.49399999999999999}]},{"osd":1,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.64200000000000002}]},{"osd":2,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.752},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":0.73499999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.60899999999999999}]},{"osd":5,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.71999999999999997}]},{"osd":6,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.56499999999999995}]},{"osd":7,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.69499999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":168506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.40400000000000003}]},{"osd":1,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46899999999999997}]},{"osd":2,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.49399999999999999}]},{"osd":4,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.64900000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.5019999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.60299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":168506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.56000000000000005}]},{"osd":1,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.63800000000000001}]},{"osd":3,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.66200000000000003}]},{"osd":4,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.71999999999999997}]},{"osd":5,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":7.4820000000000002,"5min":7.4820000000000002,"15min":7.4820000000000002},"last":0.83199999999999996}]},{"osd":6,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.749},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":3.9380000000000002,"5min":3.9380000000000002,"15min":3.9380000000000002},"last":0.77800000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":4.798,"5min":4.798,"15min":4.798},"last":0.82099999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":175101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.436}]},{"osd":2,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.53000000000000003}]},{"osd":3,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.502}]},{"osd":4,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.65100000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.59399999999999997}]},{"osd":6,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.72599999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":765469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.442},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.45800000000000002}]},{"osd":2,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.42599999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.48899999999999999}]},{"osd":4,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.624},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.64500000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.68700000000000006}]},{"osd":6,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.66100000000000003}]},{"osd":7,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.57099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:25:01.021 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:25:01.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.186+0000 7ff8f7409640 1 Processor -- start 2023-11-28T18:25:01.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.186+0000 7ff8f7409640 1 -- start start 2023-11-28T18:25:01.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.190+0000 7ff8f7409640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff8f00ce2c0 0x7ff8f006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:01.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.190+0000 7ff8f7409640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff8f006cd20 0x7ff8f00abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:01.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.190+0000 7ff8f7409640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f00cde30 con 0x7ff8f0159610 2023-11-28T18:25:01.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.190+0000 7ff8f7409640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f006d190 con 0x7ff8f006cd20 2023-11-28T18:25:01.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.190+0000 7ff8f7409640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f01599f0 con 0x7ff8f00ce2c0 2023-11-28T18:25:01.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.190+0000 7ff8f59a6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff8f006cd20 0x7ff8f00abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:01.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.190+0000 7ff8f51a5640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff8f0159610 0x7ff8f00cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:40096/0 (socket says 172.21.15.105:40096) 2023-11-28T18:25:01.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.190+0000 7ff8f51a5640 1 -- 172.21.15.105:0/1241196529 learned_addr learned my addr 172.21.15.105:0/1241196529 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:01.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.190+0000 7ff8f49a4640 1 --2- 172.21.15.105:0/1241196529 >> v2:172.21.15.148:3300/0 conn(0x7ff8f00ce2c0 0x7ff8f006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:01.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.190+0000 7ff8f59a6640 1 -- 172.21.15.105:0/1241196529 >> v2:172.21.15.148:3300/0 conn(0x7ff8f00ce2c0 msgr2=0x7ff8f006c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:01.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.190+0000 7ff8f59a6640 1 --2- 172.21.15.105:0/1241196529 >> v2:172.21.15.148:3300/0 conn(0x7ff8f00ce2c0 0x7ff8f006c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.190+0000 7ff8f59a6640 1 -- 172.21.15.105:0/1241196529 >> v1:172.21.15.148:6789/0 conn(0x7ff8f0159610 legacy=0x7ff8f00cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:01.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.190+0000 7ff8e7fff640 1 -- 172.21.15.105:0/1241196529 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853199298 0 0) 0x7ff8f00cde30 con 0x7ff8f0159610 2023-11-28T18:25:01.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.190+0000 7ff8f59a6640 1 -- 172.21.15.105:0/1241196529 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f006de20 con 0x7ff8f006cd20 2023-11-28T18:25:01.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.194+0000 7ff8f59a6640 1 --2- 172.21.15.105:0/1241196529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f006cd20 0x7ff8f00abd80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00cfc90 tx=0x7ff8e801d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a96a6b026fe3dc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:01.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.194+0000 7ff8e7fff640 1 -- 172.21.15.105:0/1241196529 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e80a6960 con 0x7ff8f006cd20 2023-11-28T18:25:01.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.194+0000 7ff8e7fff640 1 -- 172.21.15.105:0/1241196529 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e80a6ac0 con 0x7ff8f006cd20 2023-11-28T18:25:01.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.194+0000 7ff8e7fff640 1 -- 172.21.15.105:0/1241196529 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e80a6e50 con 0x7ff8f006cd20 2023-11-28T18:25:01.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.194+0000 7ff8f7409640 1 -- 172.21.15.105:0/1241196529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f006cd20 msgr2=0x7ff8f00abd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:01.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.194+0000 7ff8f7409640 1 --2- 172.21.15.105:0/1241196529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f006cd20 0x7ff8f00abd80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00cfc90 tx=0x7ff8e801d9f0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.194+0000 7ff8f7409640 1 -- 172.21.15.105:0/1241196529 shutdown_connections 2023-11-28T18:25:01.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.194+0000 7ff8f7409640 1 --2- 172.21.15.105:0/1241196529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f006cd20 0x7ff8f00abd80 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.194+0000 7ff8f7409640 1 --2- 172.21.15.105:0/1241196529 >> v2:172.21.15.148:3300/0 conn(0x7ff8f00ce2c0 0x7ff8f006c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.194+0000 7ff8f7409640 1 -- 172.21.15.105:0/1241196529 >> 172.21.15.105:0/1241196529 conn(0x7ff8f00ceb90 msgr2=0x7ff8f00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:01.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.194+0000 7ff8f7409640 1 -- 172.21.15.105:0/1241196529 shutdown_connections 2023-11-28T18:25:01.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.194+0000 7ff8f7409640 1 -- 172.21.15.105:0/1241196529 wait complete. 2023-11-28T18:25:01.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.194+0000 7ff8f7409640 1 Processor -- start 2023-11-28T18:25:01.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.194+0000 7ff8f7409640 1 -- start start 2023-11-28T18:25:01.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8f7409640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8f006cd20 0x7ff8f013ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:01.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8f7409640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff8f00ce2c0 0x7ff8f00b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:01.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8f7409640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f0159610 0x7ff8f00b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:01.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8f7409640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f0141310 con 0x7ff8f0159610 2023-11-28T18:25:01.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8f7409640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f01415f0 con 0x7ff8f006cd20 2023-11-28T18:25:01.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8f7409640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f01418d0 con 0x7ff8f00ce2c0 2023-11-28T18:25:01.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8f59a6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f0159610 0x7ff8f00b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:01.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8f59a6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f0159610 0x7ff8f00b44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53808/0 (socket says 172.21.15.105:53808) 2023-11-28T18:25:01.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8f59a6640 1 -- 172.21.15.105:0/1339873885 learned_addr learned my addr 172.21.15.105:0/1339873885 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:01.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8f51a5640 1 --2- 172.21.15.105:0/1339873885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8f006cd20 0x7ff8f013ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:01.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8f49a4640 1 --2- 172.21.15.105:0/1339873885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff8f00ce2c0 0x7ff8f00b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:01.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8f59a6640 1 -- 172.21.15.105:0/1339873885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff8f00ce2c0 msgr2=0x7ff8f00b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:01.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8f59a6640 1 --2- 172.21.15.105:0/1339873885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff8f00ce2c0 0x7ff8f00b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8f59a6640 1 -- 172.21.15.105:0/1339873885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8f006cd20 msgr2=0x7ff8f013ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:01.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8f59a6640 1 --2- 172.21.15.105:0/1339873885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8f006cd20 0x7ff8f013ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8f59a6640 1 -- 172.21.15.105:0/1339873885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e8009a70 con 0x7ff8f0159610 2023-11-28T18:25:01.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8f59a6640 1 --2- 172.21.15.105:0/1339873885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f0159610 0x7ff8f00b44d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff8e801a600 tx=0x7ff8e801a790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:01.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8e5ffb640 1 -- 172.21.15.105:0/1339873885 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e800d070 con 0x7ff8f0159610 2023-11-28T18:25:01.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8e5ffb640 1 -- 172.21.15.105:0/1339873885 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e80afd60 con 0x7ff8f0159610 2023-11-28T18:25:01.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8f7409640 1 -- 172.21.15.105:0/1339873885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f00ccff0 con 0x7ff8f0159610 2023-11-28T18:25:01.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8e5ffb640 1 -- 172.21.15.105:0/1339873885 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e80b9660 con 0x7ff8f0159610 2023-11-28T18:25:01.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8f7409640 1 -- 172.21.15.105:0/1339873885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f0141e90 con 0x7ff8f0159610 2023-11-28T18:25:01.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.198+0000 7ff8f7409640 1 -- 172.21.15.105:0/1339873885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f0067570 con 0x7ff8f0159610 2023-11-28T18:25:01.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.202+0000 7ff8e5ffb640 1 -- 172.21.15.105:0/1339873885 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff8e80b5040 con 0x7ff8f0159610 2023-11-28T18:25:01.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.202+0000 7ff8e5ffb640 1 --2- 172.21.15.105:0/1339873885 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff8d40668d0 0x7ff8d4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:01.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.202+0000 7ff8e5ffb640 1 -- 172.21.15.105:0/1339873885 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff8e811f7f0 con 0x7ff8f0159610 2023-11-28T18:25:01.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.202+0000 7ff8f51a5640 1 --2- 172.21.15.105:0/1339873885 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff8d40668d0 0x7ff8d4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:01.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.202+0000 7ff8f51a5640 1 --2- 172.21.15.105:0/1339873885 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff8d40668d0 0x7ff8d4068d90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec007bc0 tx=0x7ff8ec007b10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:01.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.206+0000 7ff8e5ffb640 1 -- 172.21.15.105:0/1339873885 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff8e80ecab0 con 0x7ff8f0159610 2023-11-28T18:25:01.350 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:25:01.354 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:25:01.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.354+0000 7ff8f7409640 1 -- 172.21.15.105:0/1339873885 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff8f0141760 con 0x7ff8d40668d0 2023-11-28T18:25:01.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.362+0000 7ff8e5ffb640 1 -- 172.21.15.105:0/1339873885 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284236 (secure 0 0 0) 0x7ff8f0141760 con 0x7ff8d40668d0 2023-11-28T18:25:01.369 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:01.371 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:25:01.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.366+0000 7ff8f7409640 1 -- 172.21.15.105:0/1339873885 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff8d40668d0 msgr2=0x7ff8d4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:01.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.366+0000 7ff8f7409640 1 --2- 172.21.15.105:0/1339873885 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff8d40668d0 0x7ff8d4068d90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec007bc0 tx=0x7ff8ec007b10 comp rx=0 tx=0).stop 2023-11-28T18:25:01.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.366+0000 7ff8f7409640 1 -- 172.21.15.105:0/1339873885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f0159610 msgr2=0x7ff8f00b44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:01.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.366+0000 7ff8f7409640 1 --2- 172.21.15.105:0/1339873885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f0159610 0x7ff8f00b44d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff8e801a600 tx=0x7ff8e801a790 comp rx=0 tx=0).stop 2023-11-28T18:25:01.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.366+0000 7ff8f7409640 1 -- 172.21.15.105:0/1339873885 shutdown_connections 2023-11-28T18:25:01.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.366+0000 7ff8f7409640 1 --2- 172.21.15.105:0/1339873885 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff8d40668d0 0x7ff8d4068d90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.366+0000 7ff8f7409640 1 --2- 172.21.15.105:0/1339873885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f0159610 0x7ff8f00b44d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.366+0000 7ff8f7409640 1 --2- 172.21.15.105:0/1339873885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff8f00ce2c0 0x7ff8f00b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.366+0000 7ff8f7409640 1 --2- 172.21.15.105:0/1339873885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8f006cd20 0x7ff8f013ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.366+0000 7ff8f7409640 1 -- 172.21.15.105:0/1339873885 >> 172.21.15.105:0/1339873885 conn(0x7ff8f00ceb90 msgr2=0x7ff8f009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:01.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.366+0000 7ff8f7409640 1 -- 172.21.15.105:0/1339873885 shutdown_connections 2023-11-28T18:25:01.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.366+0000 7ff8f7409640 1 -- 172.21.15.105:0/1339873885 wait complete. 2023-11-28T18:25:01.390 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2023-11-28T18:25:01.287179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220536,"kb_used_data":4368,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754184,"statfs":{"total":773094113280,"available":772868284416,"internally_reserved":0,"allocated":4472832,"data_stored":2568354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001230"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.555352+0000","last_change":"2023-11-28T18:24:55.555352+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.555352+0000","last_undegraded":"2023-11-28T18:24:55.555352+0000","last_fullsized":"2023-11-28T18:24:55.555352+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554587+0000","last_change":"2023-11-28T18:24:55.554587+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554587+0000","last_undegraded":"2023-11-28T18:24:55.554587+0000","last_fullsized":"2023-11-28T18:24:55.554587+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558901+0000","last_change":"2023-11-28T18:24:55.558901+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558901+0000","last_undegraded":"2023-11-28T18:24:55.558901+0000","last_fullsized":"2023-11-28T18:24:55.558901+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.566872+0000","last_change":"2023-11-28T18:24:55.566872+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.566872+0000","last_undegraded":"2023-11-28T18:24:55.566872+0000","last_fullsized":"2023-11-28T18:24:55.566872+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.565232+0000","last_change":"2023-11-28T18:24:55.565232+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.565232+0000","last_undegraded":"2023-11-28T18:24:55.565232+0000","last_fullsized":"2023-11-28T18:24:55.565232+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.557666+0000","last_change":"2023-11-28T18:24:55.557666+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.557666+0000","last_undegraded":"2023-11-28T18:24:55.557666+0000","last_fullsized":"2023-11-28T18:24:55.557666+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.564284+0000","last_change":"2023-11-28T18:24:55.564284+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.564284+0000","last_undegraded":"2023-11-28T18:24:55.564284+0000","last_fullsized":"2023-11-28T18:24:55.564284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554637+0000","last_change":"2023-11-28T18:24:55.554637+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554637+0000","last_undegraded":"2023-11-28T18:24:55.554637+0000","last_fullsized":"2023-11-28T18:24:55.554637+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.556678+0000","last_change":"2023-11-28T18:24:55.556678+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.556678+0000","last_undegraded":"2023-11-28T18:24:55.556678+0000","last_fullsized":"2023-11-28T18:24:55.556678+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.566281+0000","last_change":"2023-11-28T18:24:55.566281+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.566281+0000","last_undegraded":"2023-11-28T18:24:55.566281+0000","last_fullsized":"2023-11-28T18:24:55.566281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.568480+0000","last_change":"2023-11-28T18:24:55.568480+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.568480+0000","last_undegraded":"2023-11-28T18:24:55.568480+0000","last_fullsized":"2023-11-28T18:24:55.568480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.556143+0000","last_change":"2023-11-28T18:24:55.556143+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.556143+0000","last_undegraded":"2023-11-28T18:24:55.556143+0000","last_fullsized":"2023-11-28T18:24:55.556143+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563047+0000","last_change":"2023-11-28T18:24:55.563047+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563047+0000","last_undegraded":"2023-11-28T18:24:55.563047+0000","last_fullsized":"2023-11-28T18:24:55.563047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.556775+0000","last_change":"2023-11-28T18:24:55.556775+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.556775+0000","last_undegraded":"2023-11-28T18:24:55.556775+0000","last_fullsized":"2023-11-28T18:24:55.556775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563722+0000","last_change":"2023-11-28T18:24:55.563722+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563722+0000","last_undegraded":"2023-11-28T18:24:55.563722+0000","last_fullsized":"2023-11-28T18:24:55.563722+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563809+0000","last_change":"2023-11-28T18:24:55.563809+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563809+0000","last_undegraded":"2023-11-28T18:24:55.563809+0000","last_fullsized":"2023-11-28T18:24:55.563809+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.561864+0000","last_change":"2023-11-28T18:24:55.561864+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.561864+0000","last_undegraded":"2023-11-28T18:24:55.561864+0000","last_fullsized":"2023-11-28T18:24:55.561864+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554411+0000","last_change":"2023-11-28T18:24:55.554411+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554411+0000","last_undegraded":"2023-11-28T18:24:55.554411+0000","last_fullsized":"2023-11-28T18:24:55.554411+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.568017+0000","last_change":"2023-11-28T18:24:55.568017+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.568017+0000","last_undegraded":"2023-11-28T18:24:55.568017+0000","last_fullsized":"2023-11-28T18:24:55.568017+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553039+0000","last_change":"2023-11-28T18:24:55.553039+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553039+0000","last_undegraded":"2023-11-28T18:24:55.553039+0000","last_fullsized":"2023-11-28T18:24:55.553039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.562468+0000","last_change":"2023-11-28T18:24:55.562468+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.562468+0000","last_undegraded":"2023-11-28T18:24:55.562468+0000","last_fullsized":"2023-11-28T18:24:55.562468+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.555237+0000","last_change":"2023-11-28T18:24:55.555237+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.555237+0000","last_undegraded":"2023-11-28T18:24:55.555237+0000","last_fullsized":"2023-11-28T18:24:55.555237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.559896+0000","last_change":"2023-11-28T18:24:55.559896+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.559896+0000","last_undegraded":"2023-11-28T18:24:55.559896+0000","last_fullsized":"2023-11-28T18:24:55.559896+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.562708+0000","last_change":"2023-11-28T18:24:55.562708+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.562708+0000","last_undegraded":"2023-11-28T18:24:55.562708+0000","last_fullsized":"2023-11-28T18:24:55.562708+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553693+0000","last_change":"2023-11-28T18:24:55.553693+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553693+0000","last_undegraded":"2023-11-28T18:24:55.553693+0000","last_fullsized":"2023-11-28T18:24:55.553693+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.561952+0000","last_change":"2023-11-28T18:24:55.561952+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.561952+0000","last_undegraded":"2023-11-28T18:24:55.561952+0000","last_fullsized":"2023-11-28T18:24:55.561952+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553056+0000","last_change":"2023-11-28T18:24:55.553056+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553056+0000","last_undegraded":"2023-11-28T18:24:55.553056+0000","last_fullsized":"2023-11-28T18:24:55.553056+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.567550+0000","last_change":"2023-11-28T18:24:55.567550+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.567550+0000","last_undegraded":"2023-11-28T18:24:55.567550+0000","last_fullsized":"2023-11-28T18:24:55.567550+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553657+0000","last_change":"2023-11-28T18:24:55.553657+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553657+0000","last_undegraded":"2023-11-28T18:24:55.553657+0000","last_fullsized":"2023-11-28T18:24:55.553657+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553875+0000","last_change":"2023-11-28T18:24:55.553875+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553875+0000","last_undegraded":"2023-11-28T18:24:55.553875+0000","last_fullsized":"2023-11-28T18:24:55.553875+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.559065+0000","last_change":"2023-11-28T18:24:55.559065+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.559065+0000","last_undegraded":"2023-11-28T18:24:55.559065+0000","last_fullsized":"2023-11-28T18:24:55.559065+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558556+0000","last_change":"2023-11-28T18:24:55.558556+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558556+0000","last_undegraded":"2023-11-28T18:24:55.558556+0000","last_fullsized":"2023-11-28T18:24:55.558556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.561337+0000","last_change":"2023-11-28T18:24:55.561337+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.561337+0000","last_undegraded":"2023-11-28T18:24:55.561337+0000","last_fullsized":"2023-11-28T18:24:55.561337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.578117+0000","last_change":"2023-11-28T18:24:55.573384+0000","last_active":"2023-11-28T18:24:55.578117+0000","last_peered":"2023-11-28T18:24:55.578117+0000","last_clean":"2023-11-28T18:24:55.578117+0000","last_became_active":"2023-11-28T18:24:55.571607+0000","last_became_peered":"2023-11-28T18:24:55.571607+0000","last_unstale":"2023-11-28T18:24:55.578117+0000","last_undegraded":"2023-11-28T18:24:55.578117+0000","last_fullsized":"2023-11-28T18:24:55.578117+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:17:39.496716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.562352+0000","last_change":"2023-11-28T18:24:55.562352+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.562352+0000","last_undegraded":"2023-11-28T18:24:55.562352+0000","last_fullsized":"2023-11-28T18:24:55.562352+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.579114+0000","last_change":"2023-11-28T18:24:55.573927+0000","last_active":"2023-11-28T18:24:55.579114+0000","last_peered":"2023-11-28T18:24:55.579114+0000","last_clean":"2023-11-28T18:24:55.579114+0000","last_became_active":"2023-11-28T18:24:55.573623+0000","last_became_peered":"2023-11-28T18:24:55.573623+0000","last_unstale":"2023-11-28T18:24:55.579114+0000","last_undegraded":"2023-11-28T18:24:55.579114+0000","last_fullsized":"2023-11-28T18:24:55.579114+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:29:45.071843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558261+0000","last_change":"2023-11-28T18:24:55.558261+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558261+0000","last_undegraded":"2023-11-28T18:24:55.558261+0000","last_fullsized":"2023-11-28T18:24:55.558261+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.573242+0000","last_change":"2023-11-28T18:24:55.570144+0000","last_active":"2023-11-28T18:24:55.573242+0000","last_peered":"2023-11-28T18:24:55.573242+0000","last_clean":"2023-11-28T18:24:55.573242+0000","last_became_active":"2023-11-28T18:24:55.569837+0000","last_became_peered":"2023-11-28T18:24:55.569837+0000","last_unstale":"2023-11-28T18:24:55.573242+0000","last_undegraded":"2023-11-28T18:24:55.573242+0000","last_fullsized":"2023-11-28T18:24:55.573242+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:17:08.512763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554982+0000","last_change":"2023-11-28T18:24:55.554982+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554982+0000","last_undegraded":"2023-11-28T18:24:55.554982+0000","last_fullsized":"2023-11-28T18:24:55.554982+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.572198+0000","last_change":"2023-11-28T18:24:55.570052+0000","last_active":"2023-11-28T18:24:55.572198+0000","last_peered":"2023-11-28T18:24:55.572198+0000","last_clean":"2023-11-28T18:24:55.572198+0000","last_became_active":"2023-11-28T18:24:55.569712+0000","last_became_peered":"2023-11-28T18:24:55.569712+0000","last_unstale":"2023-11-28T18:24:55.572198+0000","last_undegraded":"2023-11-28T18:24:55.572198+0000","last_fullsized":"2023-11-28T18:24:55.572198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:06.899740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558633+0000","last_change":"2023-11-28T18:24:55.558633+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558633+0000","last_undegraded":"2023-11-28T18:24:55.558633+0000","last_fullsized":"2023-11-28T18:24:55.558633+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.580374+0000","last_change":"2023-11-28T18:24:55.575819+0000","last_active":"2023-11-28T18:24:55.580374+0000","last_peered":"2023-11-28T18:24:55.580374+0000","last_clean":"2023-11-28T18:24:55.580374+0000","last_became_active":"2023-11-28T18:24:55.575541+0000","last_became_peered":"2023-11-28T18:24:55.575541+0000","last_unstale":"2023-11-28T18:24:55.580374+0000","last_undegraded":"2023-11-28T18:24:55.580374+0000","last_fullsized":"2023-11-28T18:24:55.580374+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:46.037101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563170+0000","last_change":"2023-11-28T18:24:55.563170+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563170+0000","last_undegraded":"2023-11-28T18:24:55.563170+0000","last_fullsized":"2023-11-28T18:24:55.563170+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.580797+0000","last_change":"2023-11-28T18:24:55.578529+0000","last_active":"2023-11-28T18:24:55.580797+0000","last_peered":"2023-11-28T18:24:55.580797+0000","last_clean":"2023-11-28T18:24:55.580797+0000","last_became_active":"2023-11-28T18:24:55.578252+0000","last_became_peered":"2023-11-28T18:24:55.578252+0000","last_unstale":"2023-11-28T18:24:55.580797+0000","last_undegraded":"2023-11-28T18:24:55.580797+0000","last_fullsized":"2023-11-28T18:24:55.580797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:50:55.081198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.555789+0000","last_change":"2023-11-28T18:24:55.555789+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.555789+0000","last_undegraded":"2023-11-28T18:24:55.555789+0000","last_fullsized":"2023-11-28T18:24:55.555789+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.578996+0000","last_change":"2023-11-28T18:24:55.574964+0000","last_active":"2023-11-28T18:24:55.578996+0000","last_peered":"2023-11-28T18:24:55.578996+0000","last_clean":"2023-11-28T18:24:55.578996+0000","last_became_active":"2023-11-28T18:24:55.574669+0000","last_became_peered":"2023-11-28T18:24:55.574669+0000","last_unstale":"2023-11-28T18:24:55.578996+0000","last_undegraded":"2023-11-28T18:24:55.578996+0000","last_fullsized":"2023-11-28T18:24:55.578996+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:40:45.446584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554497+0000","last_change":"2023-11-28T18:24:55.554497+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554497+0000","last_undegraded":"2023-11-28T18:24:55.554497+0000","last_fullsized":"2023-11-28T18:24:55.554497+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.567976+0000","last_change":"2023-11-28T18:24:55.563267+0000","last_active":"2023-11-28T18:24:55.567976+0000","last_peered":"2023-11-28T18:24:55.567976+0000","last_clean":"2023-11-28T18:24:55.567976+0000","last_became_active":"2023-11-28T18:24:55.562979+0000","last_became_peered":"2023-11-28T18:24:55.562979+0000","last_unstale":"2023-11-28T18:24:55.567976+0000","last_undegraded":"2023-11-28T18:24:55.567976+0000","last_fullsized":"2023-11-28T18:24:55.567976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:06.790640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.565983+0000","last_change":"2023-11-28T18:24:55.565983+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.565983+0000","last_undegraded":"2023-11-28T18:24:55.565983+0000","last_fullsized":"2023-11-28T18:24:55.565983+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.578882+0000","last_change":"2023-11-28T18:24:55.572743+0000","last_active":"2023-11-28T18:24:55.578882+0000","last_peered":"2023-11-28T18:24:55.578882+0000","last_clean":"2023-11-28T18:24:55.578882+0000","last_became_active":"2023-11-28T18:24:55.571872+0000","last_became_peered":"2023-11-28T18:24:55.571872+0000","last_unstale":"2023-11-28T18:24:55.578882+0000","last_undegraded":"2023-11-28T18:24:55.578882+0000","last_fullsized":"2023-11-28T18:24:55.578882+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:14.095135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.564500+0000","last_change":"2023-11-28T18:24:55.564500+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.564500+0000","last_undegraded":"2023-11-28T18:24:55.564500+0000","last_fullsized":"2023-11-28T18:24:55.564500+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581663+0000","last_change":"2023-11-28T18:24:55.579938+0000","last_active":"2023-11-28T18:24:55.581663+0000","last_peered":"2023-11-28T18:24:55.581663+0000","last_clean":"2023-11-28T18:24:55.581663+0000","last_became_active":"2023-11-28T18:24:55.579667+0000","last_became_peered":"2023-11-28T18:24:55.579667+0000","last_unstale":"2023-11-28T18:24:55.581663+0000","last_undegraded":"2023-11-28T18:24:55.581663+0000","last_fullsized":"2023-11-28T18:24:55.581663+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:14.095135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.556171+0000","last_change":"2023-11-28T18:24:55.556171+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.556171+0000","last_undegraded":"2023-11-28T18:24:55.556171+0000","last_fullsized":"2023-11-28T18:24:55.556171+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581601+0000","last_change":"2023-11-28T18:24:55.579772+0000","last_active":"2023-11-28T18:24:55.581601+0000","last_peered":"2023-11-28T18:24:55.581601+0000","last_clean":"2023-11-28T18:24:55.581601+0000","last_became_active":"2023-11-28T18:24:55.579492+0000","last_became_peered":"2023-11-28T18:24:55.579492+0000","last_unstale":"2023-11-28T18:24:55.581601+0000","last_undegraded":"2023-11-28T18:24:55.581601+0000","last_fullsized":"2023-11-28T18:24:55.581601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:29:37.604627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558779+0000","last_change":"2023-11-28T18:24:55.558779+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558779+0000","last_undegraded":"2023-11-28T18:24:55.558779+0000","last_fullsized":"2023-11-28T18:24:55.558779+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581425+0000","last_change":"2023-11-28T18:24:55.576648+0000","last_active":"2023-11-28T18:24:55.581425+0000","last_peered":"2023-11-28T18:24:55.581425+0000","last_clean":"2023-11-28T18:24:55.581425+0000","last_became_active":"2023-11-28T18:24:55.576382+0000","last_became_peered":"2023-11-28T18:24:55.576382+0000","last_unstale":"2023-11-28T18:24:55.581425+0000","last_undegraded":"2023-11-28T18:24:55.581425+0000","last_fullsized":"2023-11-28T18:24:55.581425+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:29:37.604627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558183+0000","last_change":"2023-11-28T18:24:55.558183+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558183+0000","last_undegraded":"2023-11-28T18:24:55.558183+0000","last_fullsized":"2023-11-28T18:24:55.558183+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.569235+0000","last_change":"2023-11-28T18:24:55.565583+0000","last_active":"2023-11-28T18:24:55.569235+0000","last_peered":"2023-11-28T18:24:55.569235+0000","last_clean":"2023-11-28T18:24:55.569235+0000","last_became_active":"2023-11-28T18:24:55.565335+0000","last_became_peered":"2023-11-28T18:24:55.565335+0000","last_unstale":"2023-11-28T18:24:55.569235+0000","last_undegraded":"2023-11-28T18:24:55.569235+0000","last_fullsized":"2023-11-28T18:24:55.569235+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:47:09.714446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553365+0000","last_change":"2023-11-28T18:24:55.553365+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553365+0000","last_undegraded":"2023-11-28T18:24:55.553365+0000","last_fullsized":"2023-11-28T18:24:55.553365+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.579362+0000","last_change":"2023-11-28T18:24:55.575066+0000","last_active":"2023-11-28T18:24:55.579362+0000","last_peered":"2023-11-28T18:24:55.579362+0000","last_clean":"2023-11-28T18:24:55.579362+0000","last_became_active":"2023-11-28T18:24:55.574770+0000","last_became_peered":"2023-11-28T18:24:55.574770+0000","last_unstale":"2023-11-28T18:24:55.579362+0000","last_undegraded":"2023-11-28T18:24:55.579362+0000","last_fullsized":"2023-11-28T18:24:55.579362+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:44.881911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.560801+0000","last_change":"2023-11-28T18:24:55.560801+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.560801+0000","last_undegraded":"2023-11-28T18:24:55.560801+0000","last_fullsized":"2023-11-28T18:24:55.560801+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.563818+0000","last_change":"2023-11-28T18:24:55.561985+0000","last_active":"2023-11-28T18:24:55.563818+0000","last_peered":"2023-11-28T18:24:55.563818+0000","last_clean":"2023-11-28T18:24:55.563818+0000","last_became_active":"2023-11-28T18:24:55.561673+0000","last_became_peered":"2023-11-28T18:24:55.561673+0000","last_unstale":"2023-11-28T18:24:55.563818+0000","last_undegraded":"2023-11-28T18:24:55.563818+0000","last_fullsized":"2023-11-28T18:24:55.563818+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:15:50.678362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554008+0000","last_change":"2023-11-28T18:24:55.554008+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554008+0000","last_undegraded":"2023-11-28T18:24:55.554008+0000","last_fullsized":"2023-11-28T18:24:55.554008+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.576918+0000","last_change":"2023-11-28T18:24:55.572674+0000","last_active":"2023-11-28T18:24:55.576918+0000","last_peered":"2023-11-28T18:24:55.576918+0000","last_clean":"2023-11-28T18:24:55.576918+0000","last_became_active":"2023-11-28T18:24:55.572209+0000","last_became_peered":"2023-11-28T18:24:55.572209+0000","last_unstale":"2023-11-28T18:24:55.576918+0000","last_undegraded":"2023-11-28T18:24:55.576918+0000","last_fullsized":"2023-11-28T18:24:55.576918+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:17:08.512763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553441+0000","last_change":"2023-11-28T18:24:55.553441+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553441+0000","last_undegraded":"2023-11-28T18:24:55.553441+0000","last_fullsized":"2023-11-28T18:24:55.553441+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.580400+0000","last_change":"2023-11-28T18:24:55.575893+0000","last_active":"2023-11-28T18:24:55.580400+0000","last_peered":"2023-11-28T18:24:55.580400+0000","last_clean":"2023-11-28T18:24:55.580400+0000","last_became_active":"2023-11-28T18:24:55.575425+0000","last_became_peered":"2023-11-28T18:24:55.575425+0000","last_unstale":"2023-11-28T18:24:55.580400+0000","last_undegraded":"2023-11-28T18:24:55.580400+0000","last_fullsized":"2023-11-28T18:24:55.580400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:42:09.903901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.568047+0000","last_change":"2023-11-28T18:24:55.568047+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.568047+0000","last_undegraded":"2023-11-28T18:24:55.568047+0000","last_fullsized":"2023-11-28T18:24:55.568047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.583021+0000","last_change":"2023-11-28T18:24:55.579138+0000","last_active":"2023-11-28T18:24:55.583021+0000","last_peered":"2023-11-28T18:24:55.583021+0000","last_clean":"2023-11-28T18:24:55.583021+0000","last_became_active":"2023-11-28T18:24:55.578827+0000","last_became_peered":"2023-11-28T18:24:55.578827+0000","last_unstale":"2023-11-28T18:24:55.583021+0000","last_undegraded":"2023-11-28T18:24:55.583021+0000","last_fullsized":"2023-11-28T18:24:55.583021+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:55:08.887224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558780+0000","last_change":"2023-11-28T18:24:55.558780+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558780+0000","last_undegraded":"2023-11-28T18:24:55.558780+0000","last_fullsized":"2023-11-28T18:24:55.558780+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581919+0000","last_change":"2023-11-28T18:24:55.578515+0000","last_active":"2023-11-28T18:24:55.581919+0000","last_peered":"2023-11-28T18:24:55.581919+0000","last_clean":"2023-11-28T18:24:55.581919+0000","last_became_active":"2023-11-28T18:24:55.578256+0000","last_became_peered":"2023-11-28T18:24:55.578256+0000","last_unstale":"2023-11-28T18:24:55.581919+0000","last_undegraded":"2023-11-28T18:24:55.581919+0000","last_fullsized":"2023-11-28T18:24:55.581919+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:34.823798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.569066+0000","last_change":"2023-11-28T18:24:55.569066+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.569066+0000","last_undegraded":"2023-11-28T18:24:55.569066+0000","last_fullsized":"2023-11-28T18:24:55.569066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.579484+0000","last_change":"2023-11-28T18:24:55.575784+0000","last_active":"2023-11-28T18:24:55.579484+0000","last_peered":"2023-11-28T18:24:55.579484+0000","last_clean":"2023-11-28T18:24:55.579484+0000","last_became_active":"2023-11-28T18:24:55.575518+0000","last_became_peered":"2023-11-28T18:24:55.575518+0000","last_unstale":"2023-11-28T18:24:55.579484+0000","last_undegraded":"2023-11-28T18:24:55.579484+0000","last_fullsized":"2023-11-28T18:24:55.579484+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:55:08.887224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563340+0000","last_change":"2023-11-28T18:24:55.563340+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563340+0000","last_undegraded":"2023-11-28T18:24:55.563340+0000","last_fullsized":"2023-11-28T18:24:55.563340+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581809+0000","last_change":"2023-11-28T18:24:55.578008+0000","last_active":"2023-11-28T18:24:55.581809+0000","last_peered":"2023-11-28T18:24:55.581809+0000","last_clean":"2023-11-28T18:24:55.581809+0000","last_became_active":"2023-11-28T18:24:55.577782+0000","last_became_peered":"2023-11-28T18:24:55.577782+0000","last_unstale":"2023-11-28T18:24:55.581809+0000","last_undegraded":"2023-11-28T18:24:55.581809+0000","last_fullsized":"2023-11-28T18:24:55.581809+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:20:31.030439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.557289+0000","last_change":"2023-11-28T18:24:55.557289+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.557289+0000","last_undegraded":"2023-11-28T18:24:55.557289+0000","last_fullsized":"2023-11-28T18:24:55.557289+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581727+0000","last_change":"2023-11-28T18:24:55.576784+0000","last_active":"2023-11-28T18:24:55.581727+0000","last_peered":"2023-11-28T18:24:55.581727+0000","last_clean":"2023-11-28T18:24:55.581727+0000","last_became_active":"2023-11-28T18:24:55.576508+0000","last_became_peered":"2023-11-28T18:24:55.576508+0000","last_unstale":"2023-11-28T18:24:55.581727+0000","last_undegraded":"2023-11-28T18:24:55.581727+0000","last_fullsized":"2023-11-28T18:24:55.581727+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:20:38.032106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563243+0000","last_change":"2023-11-28T18:24:55.563243+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563243+0000","last_undegraded":"2023-11-28T18:24:55.563243+0000","last_fullsized":"2023-11-28T18:24:55.563243+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.583609+0000","last_change":"2023-11-28T18:24:55.582487+0000","last_active":"2023-11-28T18:24:55.583609+0000","last_peered":"2023-11-28T18:24:55.583609+0000","last_clean":"2023-11-28T18:24:55.583609+0000","last_became_active":"2023-11-28T18:24:55.582038+0000","last_became_peered":"2023-11-28T18:24:55.582038+0000","last_unstale":"2023-11-28T18:24:55.583609+0000","last_undegraded":"2023-11-28T18:24:55.583609+0000","last_fullsized":"2023-11-28T18:24:55.583609+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:46.037101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553428+0000","last_change":"2023-11-28T18:24:55.553428+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553428+0000","last_undegraded":"2023-11-28T18:24:55.553428+0000","last_fullsized":"2023-11-28T18:24:55.553428+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581789+0000","last_change":"2023-11-28T18:24:55.577155+0000","last_active":"2023-11-28T18:24:55.581789+0000","last_peered":"2023-11-28T18:24:55.581789+0000","last_clean":"2023-11-28T18:24:55.581789+0000","last_became_active":"2023-11-28T18:24:55.576899+0000","last_became_peered":"2023-11-28T18:24:55.576899+0000","last_unstale":"2023-11-28T18:24:55.581789+0000","last_undegraded":"2023-11-28T18:24:55.581789+0000","last_fullsized":"2023-11-28T18:24:55.581789+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:18:46.680968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558068+0000","last_change":"2023-11-28T18:24:55.558068+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558068+0000","last_undegraded":"2023-11-28T18:24:55.558068+0000","last_fullsized":"2023-11-28T18:24:55.558068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581668+0000","last_change":"2023-11-28T18:24:55.579931+0000","last_active":"2023-11-28T18:24:55.581668+0000","last_peered":"2023-11-28T18:24:55.581668+0000","last_clean":"2023-11-28T18:24:55.581668+0000","last_became_active":"2023-11-28T18:24:55.579568+0000","last_became_peered":"2023-11-28T18:24:55.579568+0000","last_unstale":"2023-11-28T18:24:55.581668+0000","last_undegraded":"2023-11-28T18:24:55.581668+0000","last_fullsized":"2023-11-28T18:24:55.581668+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:47:09.714446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563929+0000","last_change":"2023-11-28T18:24:55.563929+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563929+0000","last_undegraded":"2023-11-28T18:24:55.563929+0000","last_fullsized":"2023-11-28T18:24:55.563929+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.584207+0000","last_change":"2023-11-28T18:24:55.580960+0000","last_active":"2023-11-28T18:24:55.584207+0000","last_peered":"2023-11-28T18:24:55.584207+0000","last_clean":"2023-11-28T18:24:55.584207+0000","last_became_active":"2023-11-28T18:24:55.580717+0000","last_became_peered":"2023-11-28T18:24:55.580717+0000","last_unstale":"2023-11-28T18:24:55.584207+0000","last_undegraded":"2023-11-28T18:24:55.584207+0000","last_fullsized":"2023-11-28T18:24:55.584207+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:15:07.052209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553436+0000","last_change":"2023-11-28T18:24:55.553436+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553436+0000","last_undegraded":"2023-11-28T18:24:55.553436+0000","last_fullsized":"2023-11-28T18:24:55.553436+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.573984+0000","last_change":"2023-11-28T18:24:55.569035+0000","last_active":"2023-11-28T18:24:55.573984+0000","last_peered":"2023-11-28T18:24:55.573984+0000","last_clean":"2023-11-28T18:24:55.573984+0000","last_became_active":"2023-11-28T18:24:55.568744+0000","last_became_peered":"2023-11-28T18:24:55.568744+0000","last_unstale":"2023-11-28T18:24:55.573984+0000","last_undegraded":"2023-11-28T18:24:55.573984+0000","last_fullsized":"2023-11-28T18:24:55.573984+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:47:09.714446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.557761+0000","last_change":"2023-11-28T18:24:55.557761+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.557761+0000","last_undegraded":"2023-11-28T18:24:55.557761+0000","last_fullsized":"2023-11-28T18:24:55.557761+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.568346+0000","last_change":"2023-11-28T18:24:55.565390+0000","last_active":"2023-11-28T18:24:55.568346+0000","last_peered":"2023-11-28T18:24:55.568346+0000","last_clean":"2023-11-28T18:24:55.568346+0000","last_became_active":"2023-11-28T18:24:55.565112+0000","last_became_peered":"2023-11-28T18:24:55.565112+0000","last_unstale":"2023-11-28T18:24:55.568346+0000","last_undegraded":"2023-11-28T18:24:55.568346+0000","last_fullsized":"2023-11-28T18:24:55.568346+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:46.037101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558220+0000","last_change":"2023-11-28T18:24:55.558220+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558220+0000","last_undegraded":"2023-11-28T18:24:55.558220+0000","last_fullsized":"2023-11-28T18:24:55.558220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.569353+0000","last_change":"2023-11-28T18:24:55.567103+0000","last_active":"2023-11-28T18:24:55.569353+0000","last_peered":"2023-11-28T18:24:55.569353+0000","last_clean":"2023-11-28T18:24:55.569353+0000","last_became_active":"2023-11-28T18:24:55.566687+0000","last_became_peered":"2023-11-28T18:24:55.566687+0000","last_unstale":"2023-11-28T18:24:55.569353+0000","last_undegraded":"2023-11-28T18:24:55.569353+0000","last_fullsized":"2023-11-28T18:24:55.569353+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:06.790640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.556256+0000","last_change":"2023-11-28T18:24:55.556256+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.556256+0000","last_undegraded":"2023-11-28T18:24:55.556256+0000","last_fullsized":"2023-11-28T18:24:55.556256+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.577756+0000","last_change":"2023-11-28T18:24:55.572730+0000","last_active":"2023-11-28T18:24:55.577756+0000","last_peered":"2023-11-28T18:24:55.577756+0000","last_clean":"2023-11-28T18:24:55.577756+0000","last_became_active":"2023-11-28T18:24:55.572440+0000","last_became_peered":"2023-11-28T18:24:55.572440+0000","last_unstale":"2023-11-28T18:24:55.577756+0000","last_undegraded":"2023-11-28T18:24:55.577756+0000","last_fullsized":"2023-11-28T18:24:55.577756+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:44.881911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.562201+0000","last_change":"2023-11-28T18:24:55.562201+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.562201+0000","last_undegraded":"2023-11-28T18:24:55.562201+0000","last_fullsized":"2023-11-28T18:24:55.562201+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.569553+0000","last_change":"2023-11-28T18:24:55.564212+0000","last_active":"2023-11-28T18:24:55.569553+0000","last_peered":"2023-11-28T18:24:55.569553+0000","last_clean":"2023-11-28T18:24:55.569553+0000","last_became_active":"2023-11-28T18:24:55.561715+0000","last_became_peered":"2023-11-28T18:24:55.561715+0000","last_unstale":"2023-11-28T18:24:55.569553+0000","last_undegraded":"2023-11-28T18:24:55.569553+0000","last_fullsized":"2023-11-28T18:24:55.569553+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:42:09.903901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.564919+0000","last_change":"2023-11-28T18:24:55.564919+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.564919+0000","last_undegraded":"2023-11-28T18:24:55.564919+0000","last_fullsized":"2023-11-28T18:24:55.564919+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.584030+0000","last_change":"2023-11-28T18:24:55.582776+0000","last_active":"2023-11-28T18:24:55.584030+0000","last_peered":"2023-11-28T18:24:55.584030+0000","last_clean":"2023-11-28T18:24:55.584030+0000","last_became_active":"2023-11-28T18:24:55.582457+0000","last_became_peered":"2023-11-28T18:24:55.582457+0000","last_unstale":"2023-11-28T18:24:55.584030+0000","last_undegraded":"2023-11-28T18:24:55.584030+0000","last_fullsized":"2023-11-28T18:24:55.584030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:29:45.071843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T18:24:54.547971+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:24:54.547971+0000","last_peered":"2023-11-28T18:24:54.547971+0000","last_clean":"2023-11-28T18:24:54.547971+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:24:54.547971+0000","last_undegraded":"2023-11-28T18:24:54.547971+0000","last_fullsized":"2023-11-28T18:24:54.547971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":765469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.63}]},{"osd":1,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.68999999999999995}]},{"osd":2,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.411,"5min":2.411,"15min":2.411},"last":0.59499999999999997}]},{"osd":3,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.3650000000000002,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":0.747}]},{"osd":4,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.46200000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.55400000000000005}]},{"osd":6,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.61299999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":175101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.55400000000000005}]},{"osd":1,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":4.2389999999999999,"5min":4.2389999999999999,"15min":4.2389999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":4.218,"5min":4.218,"15min":4.218},"last":0.67100000000000004}]},{"osd":2,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.69799999999999995}]},{"osd":3,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.64500000000000002}]},{"osd":4,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.623},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.71799999999999997}]},{"osd":5,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.73999999999999999}]},{"osd":7,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.752},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.84399999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":175101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.8929999999999998,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.9140000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":0.69399999999999995}]},{"osd":1,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.4809999999999999,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.495}]},{"osd":2,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":3.9510000000000001,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":4.0149999999999997,"5min":4.0149999999999997,"15min":4.0149999999999997},"last":0.58999999999999997}]},{"osd":3,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":4.0869999999999997,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.9900000000000002,"5min":3.9900000000000002,"15min":3.9900000000000002},"last":0.63800000000000001}]},{"osd":4,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.44},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.67500000000000004}]},{"osd":6,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.64700000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.66300000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":175101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":2.363,"5min":2.363,"15min":2.363},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.49399999999999999}]},{"osd":1,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.64200000000000002}]},{"osd":2,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.752},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":0.73499999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.60899999999999999}]},{"osd":5,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.71999999999999997}]},{"osd":6,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.56499999999999995}]},{"osd":7,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.69499999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":168506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.40400000000000003}]},{"osd":1,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46899999999999997}]},{"osd":2,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.49399999999999999}]},{"osd":4,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.64900000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.5019999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.60299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":168506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.56000000000000005}]},{"osd":1,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.63800000000000001}]},{"osd":3,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.66200000000000003}]},{"osd":4,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.71999999999999997}]},{"osd":5,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":7.4820000000000002,"5min":7.4820000000000002,"15min":7.4820000000000002},"last":0.83199999999999996}]},{"osd":6,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.749},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":3.9380000000000002,"5min":3.9380000000000002,"15min":3.9380000000000002},"last":0.77800000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":4.798,"5min":4.798,"15min":4.798},"last":0.82099999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":175101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.436}]},{"osd":2,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.53000000000000003}]},{"osd":3,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.502}]},{"osd":4,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.65100000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.59399999999999997}]},{"osd":6,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.72599999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":765469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.442},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.45800000000000002}]},{"osd":2,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.42599999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.48899999999999999}]},{"osd":4,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.624},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.64500000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.68700000000000006}]},{"osd":6,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.66100000000000003}]},{"osd":7,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.57099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:25:01.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2023-11-28T18:25:01.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.555352+0000', 'last_change': '2023-11-28T18:24:55.555352+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.555352+0000', 'last_undegraded': '2023-11-28T18:24:55.555352+0000', 'last_fullsized': '2023-11-28T18:24:55.555352+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T18:25:01.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2023-11-28T18:25:01.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.554587+0000', 'last_change': '2023-11-28T18:24:55.554587+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.554587+0000', 'last_undegraded': '2023-11-28T18:24:55.554587+0000', 'last_fullsized': '2023-11-28T18:24:55.554587+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T18:25:01.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2023-11-28T18:25:01.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.558901+0000', 'last_change': '2023-11-28T18:24:55.558901+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.558901+0000', 'last_undegraded': '2023-11-28T18:24:55.558901+0000', 'last_fullsized': '2023-11-28T18:24:55.558901+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T18:25:01.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2023-11-28T18:25:01.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.566872+0000', 'last_change': '2023-11-28T18:24:55.566872+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.566872+0000', 'last_undegraded': '2023-11-28T18:24:55.566872+0000', 'last_fullsized': '2023-11-28T18:24:55.566872+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T18:25:01.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2023-11-28T18:25:01.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.565232+0000', 'last_change': '2023-11-28T18:24:55.565232+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.565232+0000', 'last_undegraded': '2023-11-28T18:24:55.565232+0000', 'last_fullsized': '2023-11-28T18:24:55.565232+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T18:25:01.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2023-11-28T18:25:01.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.557666+0000', 'last_change': '2023-11-28T18:24:55.557666+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.557666+0000', 'last_undegraded': '2023-11-28T18:24:55.557666+0000', 'last_fullsized': '2023-11-28T18:24:55.557666+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T18:25:01.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2023-11-28T18:25:01.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.564284+0000', 'last_change': '2023-11-28T18:24:55.564284+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.564284+0000', 'last_undegraded': '2023-11-28T18:24:55.564284+0000', 'last_fullsized': '2023-11-28T18:24:55.564284+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T18:25:01.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2023-11-28T18:25:01.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.554637+0000', 'last_change': '2023-11-28T18:24:55.554637+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.554637+0000', 'last_undegraded': '2023-11-28T18:24:55.554637+0000', 'last_fullsized': '2023-11-28T18:24:55.554637+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T18:25:01.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2023-11-28T18:25:01.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.556678+0000', 'last_change': '2023-11-28T18:24:55.556678+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.556678+0000', 'last_undegraded': '2023-11-28T18:24:55.556678+0000', 'last_fullsized': '2023-11-28T18:24:55.556678+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T18:25:01.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2023-11-28T18:25:01.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.566281+0000', 'last_change': '2023-11-28T18:24:55.566281+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.566281+0000', 'last_undegraded': '2023-11-28T18:24:55.566281+0000', 'last_fullsized': '2023-11-28T18:24:55.566281+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T18:25:01.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2023-11-28T18:25:01.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.568480+0000', 'last_change': '2023-11-28T18:24:55.568480+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.568480+0000', 'last_undegraded': '2023-11-28T18:24:55.568480+0000', 'last_fullsized': '2023-11-28T18:24:55.568480+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T18:25:01.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2023-11-28T18:25:01.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.556143+0000', 'last_change': '2023-11-28T18:24:55.556143+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.556143+0000', 'last_undegraded': '2023-11-28T18:24:55.556143+0000', 'last_fullsized': '2023-11-28T18:24:55.556143+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T18:25:01.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2023-11-28T18:25:01.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.563047+0000', 'last_change': '2023-11-28T18:24:55.563047+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.563047+0000', 'last_undegraded': '2023-11-28T18:24:55.563047+0000', 'last_fullsized': '2023-11-28T18:24:55.563047+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T18:25:01.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2023-11-28T18:25:01.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.556775+0000', 'last_change': '2023-11-28T18:24:55.556775+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.556775+0000', 'last_undegraded': '2023-11-28T18:24:55.556775+0000', 'last_fullsized': '2023-11-28T18:24:55.556775+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T18:25:01.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2023-11-28T18:25:01.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.563722+0000', 'last_change': '2023-11-28T18:24:55.563722+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.563722+0000', 'last_undegraded': '2023-11-28T18:24:55.563722+0000', 'last_fullsized': '2023-11-28T18:24:55.563722+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T18:25:01.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2023-11-28T18:25:01.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.563809+0000', 'last_change': '2023-11-28T18:24:55.563809+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.563809+0000', 'last_undegraded': '2023-11-28T18:24:55.563809+0000', 'last_fullsized': '2023-11-28T18:24:55.563809+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T18:25:01.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2023-11-28T18:25:01.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.561864+0000', 'last_change': '2023-11-28T18:24:55.561864+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.561864+0000', 'last_undegraded': '2023-11-28T18:24:55.561864+0000', 'last_fullsized': '2023-11-28T18:24:55.561864+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T18:25:01.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2023-11-28T18:25:01.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.554411+0000', 'last_change': '2023-11-28T18:24:55.554411+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.554411+0000', 'last_undegraded': '2023-11-28T18:24:55.554411+0000', 'last_fullsized': '2023-11-28T18:24:55.554411+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T18:25:01.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2023-11-28T18:25:01.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.568017+0000', 'last_change': '2023-11-28T18:24:55.568017+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.568017+0000', 'last_undegraded': '2023-11-28T18:24:55.568017+0000', 'last_fullsized': '2023-11-28T18:24:55.568017+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T18:25:01.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2023-11-28T18:25:01.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.553039+0000', 'last_change': '2023-11-28T18:24:55.553039+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.553039+0000', 'last_undegraded': '2023-11-28T18:24:55.553039+0000', 'last_fullsized': '2023-11-28T18:24:55.553039+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T18:25:01.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2023-11-28T18:25:01.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.562468+0000', 'last_change': '2023-11-28T18:24:55.562468+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.562468+0000', 'last_undegraded': '2023-11-28T18:24:55.562468+0000', 'last_fullsized': '2023-11-28T18:24:55.562468+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T18:25:01.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2023-11-28T18:25:01.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.555237+0000', 'last_change': '2023-11-28T18:24:55.555237+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.555237+0000', 'last_undegraded': '2023-11-28T18:24:55.555237+0000', 'last_fullsized': '2023-11-28T18:24:55.555237+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T18:25:01.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2023-11-28T18:25:01.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.559896+0000', 'last_change': '2023-11-28T18:24:55.559896+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.559896+0000', 'last_undegraded': '2023-11-28T18:24:55.559896+0000', 'last_fullsized': '2023-11-28T18:24:55.559896+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T18:25:01.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2023-11-28T18:25:01.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.562708+0000', 'last_change': '2023-11-28T18:24:55.562708+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.562708+0000', 'last_undegraded': '2023-11-28T18:24:55.562708+0000', 'last_fullsized': '2023-11-28T18:24:55.562708+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T18:25:01.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2023-11-28T18:25:01.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.553693+0000', 'last_change': '2023-11-28T18:24:55.553693+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.553693+0000', 'last_undegraded': '2023-11-28T18:24:55.553693+0000', 'last_fullsized': '2023-11-28T18:24:55.553693+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T18:25:01.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2023-11-28T18:25:01.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.561952+0000', 'last_change': '2023-11-28T18:24:55.561952+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.561952+0000', 'last_undegraded': '2023-11-28T18:24:55.561952+0000', 'last_fullsized': '2023-11-28T18:24:55.561952+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T18:25:01.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2023-11-28T18:25:01.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.553056+0000', 'last_change': '2023-11-28T18:24:55.553056+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.553056+0000', 'last_undegraded': '2023-11-28T18:24:55.553056+0000', 'last_fullsized': '2023-11-28T18:24:55.553056+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T18:25:01.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2023-11-28T18:25:01.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.567550+0000', 'last_change': '2023-11-28T18:24:55.567550+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.567550+0000', 'last_undegraded': '2023-11-28T18:24:55.567550+0000', 'last_fullsized': '2023-11-28T18:24:55.567550+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T18:25:01.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2023-11-28T18:25:01.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.553657+0000', 'last_change': '2023-11-28T18:24:55.553657+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.553657+0000', 'last_undegraded': '2023-11-28T18:24:55.553657+0000', 'last_fullsized': '2023-11-28T18:24:55.553657+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T18:25:01.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2023-11-28T18:25:01.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.553875+0000', 'last_change': '2023-11-28T18:24:55.553875+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.553875+0000', 'last_undegraded': '2023-11-28T18:24:55.553875+0000', 'last_fullsized': '2023-11-28T18:24:55.553875+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T18:25:01.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2023-11-28T18:25:01.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.559065+0000', 'last_change': '2023-11-28T18:24:55.559065+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.559065+0000', 'last_undegraded': '2023-11-28T18:24:55.559065+0000', 'last_fullsized': '2023-11-28T18:24:55.559065+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T18:25:01.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2023-11-28T18:25:01.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.558556+0000', 'last_change': '2023-11-28T18:24:55.558556+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.558556+0000', 'last_undegraded': '2023-11-28T18:24:55.558556+0000', 'last_fullsized': '2023-11-28T18:24:55.558556+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T18:25:01.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2023-11-28T18:25:01.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.561337+0000', 'last_change': '2023-11-28T18:24:55.561337+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.561337+0000', 'last_undegraded': '2023-11-28T18:24:55.561337+0000', 'last_fullsized': '2023-11-28T18:24:55.561337+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T18:25:01.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2023-11-28T18:25:01.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.562352+0000', 'last_change': '2023-11-28T18:24:55.562352+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.562352+0000', 'last_undegraded': '2023-11-28T18:24:55.562352+0000', 'last_fullsized': '2023-11-28T18:24:55.562352+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T18:25:01.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2023-11-28T18:25:01.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.558261+0000', 'last_change': '2023-11-28T18:24:55.558261+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.558261+0000', 'last_undegraded': '2023-11-28T18:24:55.558261+0000', 'last_fullsized': '2023-11-28T18:24:55.558261+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T18:25:01.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2023-11-28T18:25:01.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.554982+0000', 'last_change': '2023-11-28T18:24:55.554982+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.554982+0000', 'last_undegraded': '2023-11-28T18:24:55.554982+0000', 'last_fullsized': '2023-11-28T18:24:55.554982+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T18:25:01.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2023-11-28T18:25:01.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.558633+0000', 'last_change': '2023-11-28T18:24:55.558633+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.558633+0000', 'last_undegraded': '2023-11-28T18:24:55.558633+0000', 'last_fullsized': '2023-11-28T18:24:55.558633+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T18:25:01.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2023-11-28T18:25:01.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.563170+0000', 'last_change': '2023-11-28T18:24:55.563170+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.563170+0000', 'last_undegraded': '2023-11-28T18:24:55.563170+0000', 'last_fullsized': '2023-11-28T18:24:55.563170+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T18:25:01.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2023-11-28T18:25:01.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.555789+0000', 'last_change': '2023-11-28T18:24:55.555789+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.555789+0000', 'last_undegraded': '2023-11-28T18:24:55.555789+0000', 'last_fullsized': '2023-11-28T18:24:55.555789+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T18:25:01.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2023-11-28T18:25:01.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.554497+0000', 'last_change': '2023-11-28T18:24:55.554497+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.554497+0000', 'last_undegraded': '2023-11-28T18:24:55.554497+0000', 'last_fullsized': '2023-11-28T18:24:55.554497+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T18:25:01.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2023-11-28T18:25:01.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.565983+0000', 'last_change': '2023-11-28T18:24:55.565983+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.565983+0000', 'last_undegraded': '2023-11-28T18:24:55.565983+0000', 'last_fullsized': '2023-11-28T18:24:55.565983+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T18:25:01.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2023-11-28T18:25:01.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.564500+0000', 'last_change': '2023-11-28T18:24:55.564500+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.564500+0000', 'last_undegraded': '2023-11-28T18:24:55.564500+0000', 'last_fullsized': '2023-11-28T18:24:55.564500+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T18:25:01.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2023-11-28T18:25:01.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.556171+0000', 'last_change': '2023-11-28T18:24:55.556171+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.556171+0000', 'last_undegraded': '2023-11-28T18:24:55.556171+0000', 'last_fullsized': '2023-11-28T18:24:55.556171+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T18:25:01.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2023-11-28T18:25:01.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.558779+0000', 'last_change': '2023-11-28T18:24:55.558779+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.558779+0000', 'last_undegraded': '2023-11-28T18:24:55.558779+0000', 'last_fullsized': '2023-11-28T18:24:55.558779+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T18:25:01.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2023-11-28T18:25:01.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.558183+0000', 'last_change': '2023-11-28T18:24:55.558183+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.558183+0000', 'last_undegraded': '2023-11-28T18:24:55.558183+0000', 'last_fullsized': '2023-11-28T18:24:55.558183+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T18:25:01.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2023-11-28T18:25:01.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.553365+0000', 'last_change': '2023-11-28T18:24:55.553365+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.553365+0000', 'last_undegraded': '2023-11-28T18:24:55.553365+0000', 'last_fullsized': '2023-11-28T18:24:55.553365+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T18:25:01.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2023-11-28T18:25:01.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.560801+0000', 'last_change': '2023-11-28T18:24:55.560801+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.560801+0000', 'last_undegraded': '2023-11-28T18:24:55.560801+0000', 'last_fullsized': '2023-11-28T18:24:55.560801+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T18:25:01.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2023-11-28T18:25:01.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.554008+0000', 'last_change': '2023-11-28T18:24:55.554008+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.554008+0000', 'last_undegraded': '2023-11-28T18:24:55.554008+0000', 'last_fullsized': '2023-11-28T18:24:55.554008+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T18:25:01.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2023-11-28T18:25:01.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.553441+0000', 'last_change': '2023-11-28T18:24:55.553441+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.553441+0000', 'last_undegraded': '2023-11-28T18:24:55.553441+0000', 'last_fullsized': '2023-11-28T18:24:55.553441+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T18:25:01.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2023-11-28T18:25:01.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.568047+0000', 'last_change': '2023-11-28T18:24:55.568047+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.568047+0000', 'last_undegraded': '2023-11-28T18:24:55.568047+0000', 'last_fullsized': '2023-11-28T18:24:55.568047+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T18:25:01.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2023-11-28T18:25:01.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.558780+0000', 'last_change': '2023-11-28T18:24:55.558780+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.558780+0000', 'last_undegraded': '2023-11-28T18:24:55.558780+0000', 'last_fullsized': '2023-11-28T18:24:55.558780+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T18:25:01.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2023-11-28T18:25:01.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.569066+0000', 'last_change': '2023-11-28T18:24:55.569066+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.569066+0000', 'last_undegraded': '2023-11-28T18:24:55.569066+0000', 'last_fullsized': '2023-11-28T18:24:55.569066+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T18:25:01.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2023-11-28T18:25:01.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.563340+0000', 'last_change': '2023-11-28T18:24:55.563340+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.563340+0000', 'last_undegraded': '2023-11-28T18:24:55.563340+0000', 'last_fullsized': '2023-11-28T18:24:55.563340+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T18:25:01.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2023-11-28T18:25:01.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.557289+0000', 'last_change': '2023-11-28T18:24:55.557289+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.557289+0000', 'last_undegraded': '2023-11-28T18:24:55.557289+0000', 'last_fullsized': '2023-11-28T18:24:55.557289+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T18:25:01.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2023-11-28T18:25:01.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.563243+0000', 'last_change': '2023-11-28T18:24:55.563243+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.563243+0000', 'last_undegraded': '2023-11-28T18:24:55.563243+0000', 'last_fullsized': '2023-11-28T18:24:55.563243+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T18:25:01.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2023-11-28T18:25:01.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.553428+0000', 'last_change': '2023-11-28T18:24:55.553428+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.553428+0000', 'last_undegraded': '2023-11-28T18:24:55.553428+0000', 'last_fullsized': '2023-11-28T18:24:55.553428+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T18:25:01.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2023-11-28T18:25:01.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.558068+0000', 'last_change': '2023-11-28T18:24:55.558068+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.558068+0000', 'last_undegraded': '2023-11-28T18:24:55.558068+0000', 'last_fullsized': '2023-11-28T18:24:55.558068+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T18:25:01.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2023-11-28T18:25:01.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.563929+0000', 'last_change': '2023-11-28T18:24:55.563929+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.563929+0000', 'last_undegraded': '2023-11-28T18:24:55.563929+0000', 'last_fullsized': '2023-11-28T18:24:55.563929+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T18:25:01.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2023-11-28T18:25:01.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.553436+0000', 'last_change': '2023-11-28T18:24:55.553436+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.553436+0000', 'last_undegraded': '2023-11-28T18:24:55.553436+0000', 'last_fullsized': '2023-11-28T18:24:55.553436+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T18:25:01.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2023-11-28T18:25:01.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.557761+0000', 'last_change': '2023-11-28T18:24:55.557761+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.557761+0000', 'last_undegraded': '2023-11-28T18:24:55.557761+0000', 'last_fullsized': '2023-11-28T18:24:55.557761+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T18:25:01.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2023-11-28T18:25:01.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.558220+0000', 'last_change': '2023-11-28T18:24:55.558220+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.558220+0000', 'last_undegraded': '2023-11-28T18:24:55.558220+0000', 'last_fullsized': '2023-11-28T18:24:55.558220+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T18:25:01.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2023-11-28T18:25:01.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.556256+0000', 'last_change': '2023-11-28T18:24:55.556256+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.556256+0000', 'last_undegraded': '2023-11-28T18:24:55.556256+0000', 'last_fullsized': '2023-11-28T18:24:55.556256+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T18:25:01.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2023-11-28T18:25:01.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.562201+0000', 'last_change': '2023-11-28T18:24:55.562201+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.562201+0000', 'last_undegraded': '2023-11-28T18:24:55.562201+0000', 'last_fullsized': '2023-11-28T18:24:55.562201+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T18:25:01.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2023-11-28T18:25:01.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T18:24:55.564919+0000', 'last_change': '2023-11-28T18:24:55.564919+0000', 'last_active': '2023-11-28T18:24:55.539940+0000', 'last_peered': '2023-11-28T18:24:55.539940+0000', 'last_clean': '2023-11-28T18:24:55.539940+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T18:24:55.564919+0000', 'last_undegraded': '2023-11-28T18:24:55.564919+0000', 'last_fullsized': '2023-11-28T18:24:55.564919+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'last_clean_scrub_stamp': '2023-11-28T18:24:55.539940+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T18:25:01.410 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-28T18:25:01.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.582+0000 7f79d1980640 1 Processor -- start 2023-11-28T18:25:01.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.582+0000 7f79d1980640 1 -- start start 2023-11-28T18:25:01.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.586+0000 7f79d1980640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f79cc0b3f50 0x7f79cc0cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:01.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.586+0000 7f79d1980640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79cc160eb0 con 0x7f79cc0b3b70 2023-11-28T18:25:01.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.586+0000 7f79d1980640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79cc161190 con 0x7f79cc146440 2023-11-28T18:25:01.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.586+0000 7f79d1980640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79cc161b30 con 0x7f79cc0b3f50 2023-11-28T18:25:01.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.586+0000 7f79c3fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f79cc0b3f50 0x7f79cc0cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:01.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.586+0000 7f79caffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f79cc0b3b70 0x7f79cc06dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39360/0 (socket says 172.21.15.105:39360) 2023-11-28T18:25:01.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.586+0000 7f79caffd640 1 -- 172.21.15.105:0/3590692928 learned_addr learned my addr 172.21.15.105:0/3590692928 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:01.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.586+0000 7f79c3fff640 1 -- 172.21.15.105:0/3590692928 >> v1:172.21.15.148:6789/0 conn(0x7f79cc146440 legacy=0x7f79cc0b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:01.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.586+0000 7f79c3fff640 1 -- 172.21.15.105:0/3590692928 >> v1:172.21.15.105:6789/0 conn(0x7f79cc0b3b70 legacy=0x7f79cc06dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:01.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.586+0000 7f79c3fff640 1 -- 172.21.15.105:0/3590692928 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79cc161470 con 0x7f79cc0b3f50 2023-11-28T18:25:01.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.586+0000 7f79ca7fc640 1 -- 172.21.15.105:0/3590692928 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1032323791 0 0) 0x7f79cc160eb0 con 0x7f79cc0b3b70 2023-11-28T18:25:01.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.586+0000 7f79c3fff640 1 --2- 172.21.15.105:0/3590692928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79cc0b3f50 0x7f79cc0cefa0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f79b40097c0 tx=0x7f79b401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b733edee8a987c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:01.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.586+0000 7f79ca7fc640 1 -- 172.21.15.105:0/3590692928 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b40ab600 con 0x7f79cc0b3f50 2023-11-28T18:25:01.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.590+0000 7f79ca7fc640 1 -- 172.21.15.105:0/3590692928 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b40a7050 con 0x7f79cc0b3f50 2023-11-28T18:25:01.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.590+0000 7f79ca7fc640 1 -- 172.21.15.105:0/3590692928 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b40ab980 con 0x7f79cc0b3f50 2023-11-28T18:25:01.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.590+0000 7f79d1980640 1 -- 172.21.15.105:0/3590692928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79cc0b3f50 msgr2=0x7f79cc0cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:01.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.590+0000 7f79d1980640 1 --2- 172.21.15.105:0/3590692928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79cc0b3f50 0x7f79cc0cefa0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f79b40097c0 tx=0x7f79b401c440 comp rx=0 tx=0).stop 2023-11-28T18:25:01.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.590+0000 7f79d1980640 1 -- 172.21.15.105:0/3590692928 shutdown_connections 2023-11-28T18:25:01.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.590+0000 7f79d1980640 1 --2- 172.21.15.105:0/3590692928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79cc0b3f50 0x7f79cc0cefa0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.590+0000 7f79d1980640 1 -- 172.21.15.105:0/3590692928 >> 172.21.15.105:0/3590692928 conn(0x7f79cc06a980 msgr2=0x7f79cc06ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:01.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.590+0000 7f79d1980640 1 -- 172.21.15.105:0/3590692928 shutdown_connections 2023-11-28T18:25:01.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.590+0000 7f79d1980640 1 -- 172.21.15.105:0/3590692928 wait complete. 2023-11-28T18:25:01.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.590+0000 7f79d1980640 1 Processor -- start 2023-11-28T18:25:01.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.590+0000 7f79d1980640 1 -- start start 2023-11-28T18:25:01.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79d1980640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79cc0b3b70 0x7f79cc145c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:01.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79d1980640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79cc0b3f50 0x7f79cc15c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:01.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79d1980640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79cc146440 0x7f79cc13e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:01.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79d1980640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc144bc0 con 0x7f79cc146440 2023-11-28T18:25:01.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79d1980640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc144ea0 con 0x7f79cc0b3f50 2023-11-28T18:25:01.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79d1980640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc145180 con 0x7f79cc0b3b70 2023-11-28T18:25:01.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79cb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79cc146440 0x7f79cc13e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:01.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79cb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79cc146440 0x7f79cc13e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53832/0 (socket says 172.21.15.105:53832) 2023-11-28T18:25:01.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79caffd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79cc0b3b70 0x7f79cc145c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:01.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79cb7fe640 1 -- 172.21.15.105:0/496574063 learned_addr learned my addr 172.21.15.105:0/496574063 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:01.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79c3fff640 1 --2- 172.21.15.105:0/496574063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79cc0b3f50 0x7f79cc15c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:01.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79cb7fe640 1 -- 172.21.15.105:0/496574063 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79cc0b3b70 msgr2=0x7f79cc145c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:01.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79cb7fe640 1 --2- 172.21.15.105:0/496574063 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79cc0b3b70 0x7f79cc145c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79cb7fe640 1 -- 172.21.15.105:0/496574063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79cc0b3f50 msgr2=0x7f79cc15c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:01.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79cb7fe640 1 --2- 172.21.15.105:0/496574063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79cc0b3f50 0x7f79cc15c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79cb7fe640 1 -- 172.21.15.105:0/496574063 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b4009420 con 0x7f79cc146440 2023-11-28T18:25:01.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79cb7fe640 1 --2- 172.21.15.105:0/496574063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79cc146440 0x7f79cc13e380 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f79c400c780 tx=0x7f79c400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:01.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79c8ff9640 1 -- 172.21.15.105:0/496574063 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c4003d90 con 0x7f79cc146440 2023-11-28T18:25:01.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79c8ff9640 1 -- 172.21.15.105:0/496574063 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c400ce90 con 0x7f79cc146440 2023-11-28T18:25:01.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79d1980640 1 -- 172.21.15.105:0/496574063 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79cc145460 con 0x7f79cc146440 2023-11-28T18:25:01.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79c8ff9640 1 -- 172.21.15.105:0/496574063 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c40052b0 con 0x7f79cc146440 2023-11-28T18:25:01.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79d1980640 1 -- 172.21.15.105:0/496574063 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79cc1461a0 con 0x7f79cc146440 2023-11-28T18:25:01.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.594+0000 7f79d1980640 1 -- 172.21.15.105:0/496574063 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79cc145980 con 0x7f79cc146440 2023-11-28T18:25:01.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.598+0000 7f79c8ff9640 1 -- 172.21.15.105:0/496574063 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f79c4005410 con 0x7f79cc146440 2023-11-28T18:25:01.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.598+0000 7f79c8ff9640 1 --2- 172.21.15.105:0/496574063 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f79ac0668d0 0x7f79ac068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:01.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.598+0000 7f79c8ff9640 1 -- 172.21.15.105:0/496574063 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f79c40839b0 con 0x7f79cc146440 2023-11-28T18:25:01.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.598+0000 7f79caffd640 1 --2- 172.21.15.105:0/496574063 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f79ac0668d0 0x7f79ac068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:01.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.598+0000 7f79caffd640 1 --2- 172.21.15.105:0/496574063 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f79ac0668d0 0x7f79ac068d90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f79bc000f70 tx=0x7f79bc007df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:01.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.602+0000 7f79c8ff9640 1 -- 172.21.15.105:0/496574063 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f79cc145980 con 0x7f79cc146440 2023-11-28T18:25:01.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.802+0000 7f79d1980640 1 -- 172.21.15.105:0/496574063 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f79cc145980 con 0x7f79cc146440 2023-11-28T18:25:01.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.802+0000 7f79c8ff9640 1 -- 172.21.15.105:0/496574063 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1033 (secure 0 0 0) 0x7f79cc145980 con 0x7f79cc146440 2023-11-28T18:25:01.808 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:01.808 INFO:teuthology.orchestra.run.smithi105.stdout:{"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":113,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":38,"num_osds":8,"num_up_osds":8,"osd_up_since":1701195800,"num_in_osds":8,"osd_in_since":1701195790,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225828864,"bytes_avail":772868284416,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":18,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5469}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-11-28T18:24:39.283409+0000","services":{}},"progress_events":{}} 2023-11-28T18:25:01.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.802+0000 7f79d1980640 1 -- 172.21.15.105:0/496574063 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f79ac0668d0 msgr2=0x7f79ac068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:01.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.802+0000 7f79d1980640 1 --2- 172.21.15.105:0/496574063 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f79ac0668d0 0x7f79ac068d90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f79bc000f70 tx=0x7f79bc007df0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.802+0000 7f79d1980640 1 -- 172.21.15.105:0/496574063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79cc146440 msgr2=0x7f79cc13e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:01.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.802+0000 7f79d1980640 1 --2- 172.21.15.105:0/496574063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79cc146440 0x7f79cc13e380 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f79c400c780 tx=0x7f79c400cc40 comp rx=0 tx=0).stop 2023-11-28T18:25:01.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.806+0000 7f79d1980640 1 -- 172.21.15.105:0/496574063 shutdown_connections 2023-11-28T18:25:01.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.806+0000 7f79d1980640 1 --2- 172.21.15.105:0/496574063 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f79ac0668d0 0x7f79ac068d90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.806+0000 7f79d1980640 1 --2- 172.21.15.105:0/496574063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79cc146440 0x7f79cc13e380 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.806+0000 7f79d1980640 1 --2- 172.21.15.105:0/496574063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79cc0b3f50 0x7f79cc15c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.806+0000 7f79d1980640 1 --2- 172.21.15.105:0/496574063 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79cc0b3b70 0x7f79cc145c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.806+0000 7f79d1980640 1 -- 172.21.15.105:0/496574063 >> 172.21.15.105:0/496574063 conn(0x7f79cc06a980 msgr2=0x7f79cc070990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:01.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.806+0000 7f79d1980640 1 -- 172.21.15.105:0/496574063 shutdown_connections 2023-11-28T18:25:01.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.806+0000 7f79d1980640 1 -- 172.21.15.105:0/496574063 wait complete. 2023-11-28T18:25:01.828 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-28T18:25:01.828 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:25:01.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.962+0000 7f6e279c4640 1 Processor -- start 2023-11-28T18:25:01.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.962+0000 7f6e279c4640 1 -- start start 2023-11-28T18:25:01.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.962+0000 7f6e279c4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6e2015bb90 0x7f6e200b3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:01.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.962+0000 7f6e279c4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e20165870 0x7f6e201639f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:01.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.962+0000 7f6e279c4640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e200b4300 con 0x7f6e200b3ea0 2023-11-28T18:25:01.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.962+0000 7f6e279c4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e20163560 con 0x7f6e2015bb90 2023-11-28T18:25:01.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.962+0000 7f6e279c4640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2015bf70 con 0x7f6e20165870 2023-11-28T18:25:01.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.962+0000 7f6e25760640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6e2015bb90 0x7f6e200b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:01.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.962+0000 7f6e25760640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6e2015bb90 0x7f6e200b3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53842/0 (socket says 172.21.15.105:53842) 2023-11-28T18:25:01.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.962+0000 7f6e25760640 1 -- 172.21.15.105:0/2588651894 learned_addr learned my addr 172.21.15.105:0/2588651894 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:01.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.962+0000 7f6e25760640 1 -- 172.21.15.105:0/2588651894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e20165870 msgr2=0x7f6e201639f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:01.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.962+0000 7f6e25760640 1 --2- 172.21.15.105:0/2588651894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e20165870 0x7f6e201639f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.962+0000 7f6e25760640 1 -- 172.21.15.105:0/2588651894 >> v1:172.21.15.105:6789/0 conn(0x7f6e200b3ea0 legacy=0x7f6e20165160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:01.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.962+0000 7f6e25760640 1 -- 172.21.15.105:0/2588651894 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e20162c50 con 0x7f6e2015bb90 2023-11-28T18:25:01.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.966+0000 7f6e25760640 1 --2- 172.21.15.105:0/2588651894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e2015bb90 0x7f6e200b3960 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c009870 tx=0x7f6e1c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18e579b09fc56b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:01.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.966+0000 7f6e17fff640 1 -- 172.21.15.105:0/2588651894 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c0ac600 con 0x7f6e2015bb90 2023-11-28T18:25:01.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.966+0000 7f6e17fff640 1 -- 172.21.15.105:0/2588651894 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e1c0a8050 con 0x7f6e2015bb90 2023-11-28T18:25:01.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.966+0000 7f6e17fff640 1 -- 172.21.15.105:0/2588651894 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c0ac980 con 0x7f6e2015bb90 2023-11-28T18:25:01.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.966+0000 7f6e279c4640 1 -- 172.21.15.105:0/2588651894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e2015bb90 msgr2=0x7f6e200b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:01.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.966+0000 7f6e279c4640 1 --2- 172.21.15.105:0/2588651894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e2015bb90 0x7f6e200b3960 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c009870 tx=0x7f6e1c01c4b0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.966+0000 7f6e279c4640 1 -- 172.21.15.105:0/2588651894 shutdown_connections 2023-11-28T18:25:01.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.966+0000 7f6e279c4640 1 --2- 172.21.15.105:0/2588651894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e20165870 0x7f6e201639f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.966+0000 7f6e279c4640 1 --2- 172.21.15.105:0/2588651894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e2015bb90 0x7f6e200b3960 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.966+0000 7f6e279c4640 1 -- 172.21.15.105:0/2588651894 >> 172.21.15.105:0/2588651894 conn(0x7f6e2006c550 msgr2=0x7f6e2006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:01.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.966+0000 7f6e279c4640 1 -- 172.21.15.105:0/2588651894 shutdown_connections 2023-11-28T18:25:01.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.966+0000 7f6e279c4640 1 -- 172.21.15.105:0/2588651894 wait complete. 2023-11-28T18:25:01.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e279c4640 1 Processor -- start 2023-11-28T18:25:01.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e279c4640 1 -- start start 2023-11-28T18:25:01.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e279c4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e200b3ea0 0x7f6e20158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:01.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e279c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e2015bb90 0x7f6e20149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:01.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e279c4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e20165870 0x7f6e200cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:01.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e279c4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2015aa50 con 0x7f6e20165870 2023-11-28T18:25:01.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e279c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e201599b0 con 0x7f6e2015bb90 2023-11-28T18:25:01.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e279c4640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e20159c90 con 0x7f6e200b3ea0 2023-11-28T18:25:01.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e25f61640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e20165870 0x7f6e200cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:01.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e25f61640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e20165870 0x7f6e200cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53846/0 (socket says 172.21.15.105:53846) 2023-11-28T18:25:01.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e25f61640 1 -- 172.21.15.105:0/1848123966 learned_addr learned my addr 172.21.15.105:0/1848123966 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:01.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e25760640 1 --2- 172.21.15.105:0/1848123966 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e200b3ea0 0x7f6e20158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:01.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e24f5f640 1 --2- 172.21.15.105:0/1848123966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e2015bb90 0x7f6e20149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:01.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e25f61640 1 -- 172.21.15.105:0/1848123966 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e200b3ea0 msgr2=0x7f6e20158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:01.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e25f61640 1 --2- 172.21.15.105:0/1848123966 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e200b3ea0 0x7f6e20158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e25f61640 1 -- 172.21.15.105:0/1848123966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e2015bb90 msgr2=0x7f6e20149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e25f61640 1 --2- 172.21.15.105:0/1848123966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e2015bb90 0x7f6e20149c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e25f61640 1 -- 172.21.15.105:0/1848123966 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1c0094d0 con 0x7f6e20165870 2023-11-28T18:25:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e25f61640 1 --2- 172.21.15.105:0/1848123966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e20165870 0x7f6e200cd870 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800b970 tx=0x7f6e1800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e15ffb640 1 -- 172.21.15.105:0/1848123966 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1800cb40 con 0x7f6e20165870 2023-11-28T18:25:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e15ffb640 1 -- 172.21.15.105:0/1848123966 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e1800ccc0 con 0x7f6e20165870 2023-11-28T18:25:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e279c4640 1 -- 172.21.15.105:0/1848123966 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e20159f70 con 0x7f6e20165870 2023-11-28T18:25:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e15ffb640 1 -- 172.21.15.105:0/1848123966 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18014680 con 0x7f6e20165870 2023-11-28T18:25:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.970+0000 7f6e279c4640 1 -- 172.21.15.105:0/1848123966 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e2015a530 con 0x7f6e20165870 2023-11-28T18:25:01.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.974+0000 7f6e279c4640 1 -- 172.21.15.105:0/1848123966 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e200745f0 con 0x7f6e20165870 2023-11-28T18:25:01.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.974+0000 7f6e15ffb640 1 -- 172.21.15.105:0/1848123966 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f6e1801e440 con 0x7f6e20165870 2023-11-28T18:25:01.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.974+0000 7f6e15ffb640 1 --2- 172.21.15.105:0/1848123966 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6e040668d0 0x7f6e04068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:01.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.974+0000 7f6e15ffb640 1 -- 172.21.15.105:0/1848123966 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6e18084490 con 0x7f6e20165870 2023-11-28T18:25:01.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.974+0000 7f6e25760640 1 --2- 172.21.15.105:0/1848123966 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6e040668d0 0x7f6e04068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:01.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.974+0000 7f6e25760640 1 --2- 172.21.15.105:0/1848123966 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6e040668d0 0x7f6e04068d90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0ae000 tx=0x7f6e1c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:01.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:01.978+0000 7f6e15ffb640 1 -- 172.21.15.105:0/1848123966 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6e18051580 con 0x7f6e20165870 2023-11-28T18:25:02.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:02.126+0000 7f6e279c4640 1 -- 172.21.15.105:0/1848123966 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6e20072790 con 0x7f6e040668d0 2023-11-28T18:25:02.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:02.130+0000 7f6e15ffb640 1 -- 172.21.15.105:0/1848123966 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284236 (secure 0 0 0) 0x7f6e20072790 con 0x7f6e040668d0 2023-11-28T18:25:02.139 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:02.141 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:25:02.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:02.134+0000 7f6e279c4640 1 -- 172.21.15.105:0/1848123966 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6e040668d0 msgr2=0x7f6e04068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:02.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:02.134+0000 7f6e279c4640 1 --2- 172.21.15.105:0/1848123966 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6e040668d0 0x7f6e04068d90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0ae000 tx=0x7f6e1c0a7000 comp rx=0 tx=0).stop 2023-11-28T18:25:02.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:02.134+0000 7f6e279c4640 1 -- 172.21.15.105:0/1848123966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e20165870 msgr2=0x7f6e200cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:02.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:02.134+0000 7f6e279c4640 1 --2- 172.21.15.105:0/1848123966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e20165870 0x7f6e200cd870 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800b970 tx=0x7f6e1800be30 comp rx=0 tx=0).stop 2023-11-28T18:25:02.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:02.134+0000 7f6e279c4640 1 -- 172.21.15.105:0/1848123966 shutdown_connections 2023-11-28T18:25:02.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:02.134+0000 7f6e279c4640 1 --2- 172.21.15.105:0/1848123966 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6e040668d0 0x7f6e04068d90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:02.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:02.134+0000 7f6e279c4640 1 --2- 172.21.15.105:0/1848123966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e20165870 0x7f6e200cd870 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:02.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:02.134+0000 7f6e279c4640 1 --2- 172.21.15.105:0/1848123966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e2015bb90 0x7f6e20149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:02.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:02.134+0000 7f6e279c4640 1 --2- 172.21.15.105:0/1848123966 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e200b3ea0 0x7f6e20158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:02.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:02.134+0000 7f6e279c4640 1 -- 172.21.15.105:0/1848123966 >> 172.21.15.105:0/1848123966 conn(0x7f6e2006c550 msgr2=0x7f6e2015fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:02.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:02.134+0000 7f6e279c4640 1 -- 172.21.15.105:0/1848123966 shutdown_connections 2023-11-28T18:25:02.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:02.134+0000 7f6e279c4640 1 -- 172.21.15.105:0/1848123966 wait complete. 2023-11-28T18:25:02.160 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2023-11-28T18:25:01.287179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220536,"kb_used_data":4368,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754184,"statfs":{"total":773094113280,"available":772868284416,"internally_reserved":0,"allocated":4472832,"data_stored":2568354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001230"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.555352+0000","last_change":"2023-11-28T18:24:55.555352+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.555352+0000","last_undegraded":"2023-11-28T18:24:55.555352+0000","last_fullsized":"2023-11-28T18:24:55.555352+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554587+0000","last_change":"2023-11-28T18:24:55.554587+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554587+0000","last_undegraded":"2023-11-28T18:24:55.554587+0000","last_fullsized":"2023-11-28T18:24:55.554587+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558901+0000","last_change":"2023-11-28T18:24:55.558901+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558901+0000","last_undegraded":"2023-11-28T18:24:55.558901+0000","last_fullsized":"2023-11-28T18:24:55.558901+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.566872+0000","last_change":"2023-11-28T18:24:55.566872+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.566872+0000","last_undegraded":"2023-11-28T18:24:55.566872+0000","last_fullsized":"2023-11-28T18:24:55.566872+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.565232+0000","last_change":"2023-11-28T18:24:55.565232+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.565232+0000","last_undegraded":"2023-11-28T18:24:55.565232+0000","last_fullsized":"2023-11-28T18:24:55.565232+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.557666+0000","last_change":"2023-11-28T18:24:55.557666+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.557666+0000","last_undegraded":"2023-11-28T18:24:55.557666+0000","last_fullsized":"2023-11-28T18:24:55.557666+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.564284+0000","last_change":"2023-11-28T18:24:55.564284+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.564284+0000","last_undegraded":"2023-11-28T18:24:55.564284+0000","last_fullsized":"2023-11-28T18:24:55.564284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554637+0000","last_change":"2023-11-28T18:24:55.554637+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554637+0000","last_undegraded":"2023-11-28T18:24:55.554637+0000","last_fullsized":"2023-11-28T18:24:55.554637+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.556678+0000","last_change":"2023-11-28T18:24:55.556678+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.556678+0000","last_undegraded":"2023-11-28T18:24:55.556678+0000","last_fullsized":"2023-11-28T18:24:55.556678+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.566281+0000","last_change":"2023-11-28T18:24:55.566281+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.566281+0000","last_undegraded":"2023-11-28T18:24:55.566281+0000","last_fullsized":"2023-11-28T18:24:55.566281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.568480+0000","last_change":"2023-11-28T18:24:55.568480+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.568480+0000","last_undegraded":"2023-11-28T18:24:55.568480+0000","last_fullsized":"2023-11-28T18:24:55.568480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.556143+0000","last_change":"2023-11-28T18:24:55.556143+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.556143+0000","last_undegraded":"2023-11-28T18:24:55.556143+0000","last_fullsized":"2023-11-28T18:24:55.556143+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563047+0000","last_change":"2023-11-28T18:24:55.563047+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563047+0000","last_undegraded":"2023-11-28T18:24:55.563047+0000","last_fullsized":"2023-11-28T18:24:55.563047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.556775+0000","last_change":"2023-11-28T18:24:55.556775+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.556775+0000","last_undegraded":"2023-11-28T18:24:55.556775+0000","last_fullsized":"2023-11-28T18:24:55.556775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563722+0000","last_change":"2023-11-28T18:24:55.563722+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563722+0000","last_undegraded":"2023-11-28T18:24:55.563722+0000","last_fullsized":"2023-11-28T18:24:55.563722+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563809+0000","last_change":"2023-11-28T18:24:55.563809+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563809+0000","last_undegraded":"2023-11-28T18:24:55.563809+0000","last_fullsized":"2023-11-28T18:24:55.563809+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.561864+0000","last_change":"2023-11-28T18:24:55.561864+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.561864+0000","last_undegraded":"2023-11-28T18:24:55.561864+0000","last_fullsized":"2023-11-28T18:24:55.561864+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554411+0000","last_change":"2023-11-28T18:24:55.554411+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554411+0000","last_undegraded":"2023-11-28T18:24:55.554411+0000","last_fullsized":"2023-11-28T18:24:55.554411+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.568017+0000","last_change":"2023-11-28T18:24:55.568017+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.568017+0000","last_undegraded":"2023-11-28T18:24:55.568017+0000","last_fullsized":"2023-11-28T18:24:55.568017+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553039+0000","last_change":"2023-11-28T18:24:55.553039+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553039+0000","last_undegraded":"2023-11-28T18:24:55.553039+0000","last_fullsized":"2023-11-28T18:24:55.553039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.562468+0000","last_change":"2023-11-28T18:24:55.562468+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.562468+0000","last_undegraded":"2023-11-28T18:24:55.562468+0000","last_fullsized":"2023-11-28T18:24:55.562468+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.555237+0000","last_change":"2023-11-28T18:24:55.555237+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.555237+0000","last_undegraded":"2023-11-28T18:24:55.555237+0000","last_fullsized":"2023-11-28T18:24:55.555237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.559896+0000","last_change":"2023-11-28T18:24:55.559896+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.559896+0000","last_undegraded":"2023-11-28T18:24:55.559896+0000","last_fullsized":"2023-11-28T18:24:55.559896+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.562708+0000","last_change":"2023-11-28T18:24:55.562708+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.562708+0000","last_undegraded":"2023-11-28T18:24:55.562708+0000","last_fullsized":"2023-11-28T18:24:55.562708+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553693+0000","last_change":"2023-11-28T18:24:55.553693+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553693+0000","last_undegraded":"2023-11-28T18:24:55.553693+0000","last_fullsized":"2023-11-28T18:24:55.553693+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.561952+0000","last_change":"2023-11-28T18:24:55.561952+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.561952+0000","last_undegraded":"2023-11-28T18:24:55.561952+0000","last_fullsized":"2023-11-28T18:24:55.561952+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553056+0000","last_change":"2023-11-28T18:24:55.553056+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553056+0000","last_undegraded":"2023-11-28T18:24:55.553056+0000","last_fullsized":"2023-11-28T18:24:55.553056+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.567550+0000","last_change":"2023-11-28T18:24:55.567550+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.567550+0000","last_undegraded":"2023-11-28T18:24:55.567550+0000","last_fullsized":"2023-11-28T18:24:55.567550+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553657+0000","last_change":"2023-11-28T18:24:55.553657+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553657+0000","last_undegraded":"2023-11-28T18:24:55.553657+0000","last_fullsized":"2023-11-28T18:24:55.553657+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553875+0000","last_change":"2023-11-28T18:24:55.553875+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553875+0000","last_undegraded":"2023-11-28T18:24:55.553875+0000","last_fullsized":"2023-11-28T18:24:55.553875+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.559065+0000","last_change":"2023-11-28T18:24:55.559065+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.559065+0000","last_undegraded":"2023-11-28T18:24:55.559065+0000","last_fullsized":"2023-11-28T18:24:55.559065+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558556+0000","last_change":"2023-11-28T18:24:55.558556+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558556+0000","last_undegraded":"2023-11-28T18:24:55.558556+0000","last_fullsized":"2023-11-28T18:24:55.558556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.561337+0000","last_change":"2023-11-28T18:24:55.561337+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.561337+0000","last_undegraded":"2023-11-28T18:24:55.561337+0000","last_fullsized":"2023-11-28T18:24:55.561337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.578117+0000","last_change":"2023-11-28T18:24:55.573384+0000","last_active":"2023-11-28T18:24:55.578117+0000","last_peered":"2023-11-28T18:24:55.578117+0000","last_clean":"2023-11-28T18:24:55.578117+0000","last_became_active":"2023-11-28T18:24:55.571607+0000","last_became_peered":"2023-11-28T18:24:55.571607+0000","last_unstale":"2023-11-28T18:24:55.578117+0000","last_undegraded":"2023-11-28T18:24:55.578117+0000","last_fullsized":"2023-11-28T18:24:55.578117+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:17:39.496716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.562352+0000","last_change":"2023-11-28T18:24:55.562352+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.562352+0000","last_undegraded":"2023-11-28T18:24:55.562352+0000","last_fullsized":"2023-11-28T18:24:55.562352+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.579114+0000","last_change":"2023-11-28T18:24:55.573927+0000","last_active":"2023-11-28T18:24:55.579114+0000","last_peered":"2023-11-28T18:24:55.579114+0000","last_clean":"2023-11-28T18:24:55.579114+0000","last_became_active":"2023-11-28T18:24:55.573623+0000","last_became_peered":"2023-11-28T18:24:55.573623+0000","last_unstale":"2023-11-28T18:24:55.579114+0000","last_undegraded":"2023-11-28T18:24:55.579114+0000","last_fullsized":"2023-11-28T18:24:55.579114+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:29:45.071843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558261+0000","last_change":"2023-11-28T18:24:55.558261+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558261+0000","last_undegraded":"2023-11-28T18:24:55.558261+0000","last_fullsized":"2023-11-28T18:24:55.558261+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.573242+0000","last_change":"2023-11-28T18:24:55.570144+0000","last_active":"2023-11-28T18:24:55.573242+0000","last_peered":"2023-11-28T18:24:55.573242+0000","last_clean":"2023-11-28T18:24:55.573242+0000","last_became_active":"2023-11-28T18:24:55.569837+0000","last_became_peered":"2023-11-28T18:24:55.569837+0000","last_unstale":"2023-11-28T18:24:55.573242+0000","last_undegraded":"2023-11-28T18:24:55.573242+0000","last_fullsized":"2023-11-28T18:24:55.573242+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:17:08.512763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554982+0000","last_change":"2023-11-28T18:24:55.554982+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554982+0000","last_undegraded":"2023-11-28T18:24:55.554982+0000","last_fullsized":"2023-11-28T18:24:55.554982+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.572198+0000","last_change":"2023-11-28T18:24:55.570052+0000","last_active":"2023-11-28T18:24:55.572198+0000","last_peered":"2023-11-28T18:24:55.572198+0000","last_clean":"2023-11-28T18:24:55.572198+0000","last_became_active":"2023-11-28T18:24:55.569712+0000","last_became_peered":"2023-11-28T18:24:55.569712+0000","last_unstale":"2023-11-28T18:24:55.572198+0000","last_undegraded":"2023-11-28T18:24:55.572198+0000","last_fullsized":"2023-11-28T18:24:55.572198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:06.899740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558633+0000","last_change":"2023-11-28T18:24:55.558633+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558633+0000","last_undegraded":"2023-11-28T18:24:55.558633+0000","last_fullsized":"2023-11-28T18:24:55.558633+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.580374+0000","last_change":"2023-11-28T18:24:55.575819+0000","last_active":"2023-11-28T18:24:55.580374+0000","last_peered":"2023-11-28T18:24:55.580374+0000","last_clean":"2023-11-28T18:24:55.580374+0000","last_became_active":"2023-11-28T18:24:55.575541+0000","last_became_peered":"2023-11-28T18:24:55.575541+0000","last_unstale":"2023-11-28T18:24:55.580374+0000","last_undegraded":"2023-11-28T18:24:55.580374+0000","last_fullsized":"2023-11-28T18:24:55.580374+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:46.037101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563170+0000","last_change":"2023-11-28T18:24:55.563170+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563170+0000","last_undegraded":"2023-11-28T18:24:55.563170+0000","last_fullsized":"2023-11-28T18:24:55.563170+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.580797+0000","last_change":"2023-11-28T18:24:55.578529+0000","last_active":"2023-11-28T18:24:55.580797+0000","last_peered":"2023-11-28T18:24:55.580797+0000","last_clean":"2023-11-28T18:24:55.580797+0000","last_became_active":"2023-11-28T18:24:55.578252+0000","last_became_peered":"2023-11-28T18:24:55.578252+0000","last_unstale":"2023-11-28T18:24:55.580797+0000","last_undegraded":"2023-11-28T18:24:55.580797+0000","last_fullsized":"2023-11-28T18:24:55.580797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:50:55.081198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.555789+0000","last_change":"2023-11-28T18:24:55.555789+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.555789+0000","last_undegraded":"2023-11-28T18:24:55.555789+0000","last_fullsized":"2023-11-28T18:24:55.555789+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.578996+0000","last_change":"2023-11-28T18:24:55.574964+0000","last_active":"2023-11-28T18:24:55.578996+0000","last_peered":"2023-11-28T18:24:55.578996+0000","last_clean":"2023-11-28T18:24:55.578996+0000","last_became_active":"2023-11-28T18:24:55.574669+0000","last_became_peered":"2023-11-28T18:24:55.574669+0000","last_unstale":"2023-11-28T18:24:55.578996+0000","last_undegraded":"2023-11-28T18:24:55.578996+0000","last_fullsized":"2023-11-28T18:24:55.578996+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:40:45.446584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554497+0000","last_change":"2023-11-28T18:24:55.554497+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554497+0000","last_undegraded":"2023-11-28T18:24:55.554497+0000","last_fullsized":"2023-11-28T18:24:55.554497+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.567976+0000","last_change":"2023-11-28T18:24:55.563267+0000","last_active":"2023-11-28T18:24:55.567976+0000","last_peered":"2023-11-28T18:24:55.567976+0000","last_clean":"2023-11-28T18:24:55.567976+0000","last_became_active":"2023-11-28T18:24:55.562979+0000","last_became_peered":"2023-11-28T18:24:55.562979+0000","last_unstale":"2023-11-28T18:24:55.567976+0000","last_undegraded":"2023-11-28T18:24:55.567976+0000","last_fullsized":"2023-11-28T18:24:55.567976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:06.790640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.565983+0000","last_change":"2023-11-28T18:24:55.565983+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.565983+0000","last_undegraded":"2023-11-28T18:24:55.565983+0000","last_fullsized":"2023-11-28T18:24:55.565983+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.578882+0000","last_change":"2023-11-28T18:24:55.572743+0000","last_active":"2023-11-28T18:24:55.578882+0000","last_peered":"2023-11-28T18:24:55.578882+0000","last_clean":"2023-11-28T18:24:55.578882+0000","last_became_active":"2023-11-28T18:24:55.571872+0000","last_became_peered":"2023-11-28T18:24:55.571872+0000","last_unstale":"2023-11-28T18:24:55.578882+0000","last_undegraded":"2023-11-28T18:24:55.578882+0000","last_fullsized":"2023-11-28T18:24:55.578882+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:14.095135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.564500+0000","last_change":"2023-11-28T18:24:55.564500+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.564500+0000","last_undegraded":"2023-11-28T18:24:55.564500+0000","last_fullsized":"2023-11-28T18:24:55.564500+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581663+0000","last_change":"2023-11-28T18:24:55.579938+0000","last_active":"2023-11-28T18:24:55.581663+0000","last_peered":"2023-11-28T18:24:55.581663+0000","last_clean":"2023-11-28T18:24:55.581663+0000","last_became_active":"2023-11-28T18:24:55.579667+0000","last_became_peered":"2023-11-28T18:24:55.579667+0000","last_unstale":"2023-11-28T18:24:55.581663+0000","last_undegraded":"2023-11-28T18:24:55.581663+0000","last_fullsized":"2023-11-28T18:24:55.581663+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:14.095135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.556171+0000","last_change":"2023-11-28T18:24:55.556171+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.556171+0000","last_undegraded":"2023-11-28T18:24:55.556171+0000","last_fullsized":"2023-11-28T18:24:55.556171+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581601+0000","last_change":"2023-11-28T18:24:55.579772+0000","last_active":"2023-11-28T18:24:55.581601+0000","last_peered":"2023-11-28T18:24:55.581601+0000","last_clean":"2023-11-28T18:24:55.581601+0000","last_became_active":"2023-11-28T18:24:55.579492+0000","last_became_peered":"2023-11-28T18:24:55.579492+0000","last_unstale":"2023-11-28T18:24:55.581601+0000","last_undegraded":"2023-11-28T18:24:55.581601+0000","last_fullsized":"2023-11-28T18:24:55.581601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:29:37.604627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558779+0000","last_change":"2023-11-28T18:24:55.558779+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558779+0000","last_undegraded":"2023-11-28T18:24:55.558779+0000","last_fullsized":"2023-11-28T18:24:55.558779+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581425+0000","last_change":"2023-11-28T18:24:55.576648+0000","last_active":"2023-11-28T18:24:55.581425+0000","last_peered":"2023-11-28T18:24:55.581425+0000","last_clean":"2023-11-28T18:24:55.581425+0000","last_became_active":"2023-11-28T18:24:55.576382+0000","last_became_peered":"2023-11-28T18:24:55.576382+0000","last_unstale":"2023-11-28T18:24:55.581425+0000","last_undegraded":"2023-11-28T18:24:55.581425+0000","last_fullsized":"2023-11-28T18:24:55.581425+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:29:37.604627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558183+0000","last_change":"2023-11-28T18:24:55.558183+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558183+0000","last_undegraded":"2023-11-28T18:24:55.558183+0000","last_fullsized":"2023-11-28T18:24:55.558183+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.569235+0000","last_change":"2023-11-28T18:24:55.565583+0000","last_active":"2023-11-28T18:24:55.569235+0000","last_peered":"2023-11-28T18:24:55.569235+0000","last_clean":"2023-11-28T18:24:55.569235+0000","last_became_active":"2023-11-28T18:24:55.565335+0000","last_became_peered":"2023-11-28T18:24:55.565335+0000","last_unstale":"2023-11-28T18:24:55.569235+0000","last_undegraded":"2023-11-28T18:24:55.569235+0000","last_fullsized":"2023-11-28T18:24:55.569235+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:47:09.714446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553365+0000","last_change":"2023-11-28T18:24:55.553365+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553365+0000","last_undegraded":"2023-11-28T18:24:55.553365+0000","last_fullsized":"2023-11-28T18:24:55.553365+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.579362+0000","last_change":"2023-11-28T18:24:55.575066+0000","last_active":"2023-11-28T18:24:55.579362+0000","last_peered":"2023-11-28T18:24:55.579362+0000","last_clean":"2023-11-28T18:24:55.579362+0000","last_became_active":"2023-11-28T18:24:55.574770+0000","last_became_peered":"2023-11-28T18:24:55.574770+0000","last_unstale":"2023-11-28T18:24:55.579362+0000","last_undegraded":"2023-11-28T18:24:55.579362+0000","last_fullsized":"2023-11-28T18:24:55.579362+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:44.881911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.560801+0000","last_change":"2023-11-28T18:24:55.560801+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.560801+0000","last_undegraded":"2023-11-28T18:24:55.560801+0000","last_fullsized":"2023-11-28T18:24:55.560801+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.563818+0000","last_change":"2023-11-28T18:24:55.561985+0000","last_active":"2023-11-28T18:24:55.563818+0000","last_peered":"2023-11-28T18:24:55.563818+0000","last_clean":"2023-11-28T18:24:55.563818+0000","last_became_active":"2023-11-28T18:24:55.561673+0000","last_became_peered":"2023-11-28T18:24:55.561673+0000","last_unstale":"2023-11-28T18:24:55.563818+0000","last_undegraded":"2023-11-28T18:24:55.563818+0000","last_fullsized":"2023-11-28T18:24:55.563818+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:15:50.678362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.554008+0000","last_change":"2023-11-28T18:24:55.554008+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.554008+0000","last_undegraded":"2023-11-28T18:24:55.554008+0000","last_fullsized":"2023-11-28T18:24:55.554008+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.576918+0000","last_change":"2023-11-28T18:24:55.572674+0000","last_active":"2023-11-28T18:24:55.576918+0000","last_peered":"2023-11-28T18:24:55.576918+0000","last_clean":"2023-11-28T18:24:55.576918+0000","last_became_active":"2023-11-28T18:24:55.572209+0000","last_became_peered":"2023-11-28T18:24:55.572209+0000","last_unstale":"2023-11-28T18:24:55.576918+0000","last_undegraded":"2023-11-28T18:24:55.576918+0000","last_fullsized":"2023-11-28T18:24:55.576918+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:17:08.512763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553441+0000","last_change":"2023-11-28T18:24:55.553441+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553441+0000","last_undegraded":"2023-11-28T18:24:55.553441+0000","last_fullsized":"2023-11-28T18:24:55.553441+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.580400+0000","last_change":"2023-11-28T18:24:55.575893+0000","last_active":"2023-11-28T18:24:55.580400+0000","last_peered":"2023-11-28T18:24:55.580400+0000","last_clean":"2023-11-28T18:24:55.580400+0000","last_became_active":"2023-11-28T18:24:55.575425+0000","last_became_peered":"2023-11-28T18:24:55.575425+0000","last_unstale":"2023-11-28T18:24:55.580400+0000","last_undegraded":"2023-11-28T18:24:55.580400+0000","last_fullsized":"2023-11-28T18:24:55.580400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:42:09.903901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.568047+0000","last_change":"2023-11-28T18:24:55.568047+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.568047+0000","last_undegraded":"2023-11-28T18:24:55.568047+0000","last_fullsized":"2023-11-28T18:24:55.568047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.583021+0000","last_change":"2023-11-28T18:24:55.579138+0000","last_active":"2023-11-28T18:24:55.583021+0000","last_peered":"2023-11-28T18:24:55.583021+0000","last_clean":"2023-11-28T18:24:55.583021+0000","last_became_active":"2023-11-28T18:24:55.578827+0000","last_became_peered":"2023-11-28T18:24:55.578827+0000","last_unstale":"2023-11-28T18:24:55.583021+0000","last_undegraded":"2023-11-28T18:24:55.583021+0000","last_fullsized":"2023-11-28T18:24:55.583021+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:55:08.887224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558780+0000","last_change":"2023-11-28T18:24:55.558780+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558780+0000","last_undegraded":"2023-11-28T18:24:55.558780+0000","last_fullsized":"2023-11-28T18:24:55.558780+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581919+0000","last_change":"2023-11-28T18:24:55.578515+0000","last_active":"2023-11-28T18:24:55.581919+0000","last_peered":"2023-11-28T18:24:55.581919+0000","last_clean":"2023-11-28T18:24:55.581919+0000","last_became_active":"2023-11-28T18:24:55.578256+0000","last_became_peered":"2023-11-28T18:24:55.578256+0000","last_unstale":"2023-11-28T18:24:55.581919+0000","last_undegraded":"2023-11-28T18:24:55.581919+0000","last_fullsized":"2023-11-28T18:24:55.581919+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:34.823798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.569066+0000","last_change":"2023-11-28T18:24:55.569066+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.569066+0000","last_undegraded":"2023-11-28T18:24:55.569066+0000","last_fullsized":"2023-11-28T18:24:55.569066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.579484+0000","last_change":"2023-11-28T18:24:55.575784+0000","last_active":"2023-11-28T18:24:55.579484+0000","last_peered":"2023-11-28T18:24:55.579484+0000","last_clean":"2023-11-28T18:24:55.579484+0000","last_became_active":"2023-11-28T18:24:55.575518+0000","last_became_peered":"2023-11-28T18:24:55.575518+0000","last_unstale":"2023-11-28T18:24:55.579484+0000","last_undegraded":"2023-11-28T18:24:55.579484+0000","last_fullsized":"2023-11-28T18:24:55.579484+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:55:08.887224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563340+0000","last_change":"2023-11-28T18:24:55.563340+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563340+0000","last_undegraded":"2023-11-28T18:24:55.563340+0000","last_fullsized":"2023-11-28T18:24:55.563340+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581809+0000","last_change":"2023-11-28T18:24:55.578008+0000","last_active":"2023-11-28T18:24:55.581809+0000","last_peered":"2023-11-28T18:24:55.581809+0000","last_clean":"2023-11-28T18:24:55.581809+0000","last_became_active":"2023-11-28T18:24:55.577782+0000","last_became_peered":"2023-11-28T18:24:55.577782+0000","last_unstale":"2023-11-28T18:24:55.581809+0000","last_undegraded":"2023-11-28T18:24:55.581809+0000","last_fullsized":"2023-11-28T18:24:55.581809+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:20:31.030439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.557289+0000","last_change":"2023-11-28T18:24:55.557289+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.557289+0000","last_undegraded":"2023-11-28T18:24:55.557289+0000","last_fullsized":"2023-11-28T18:24:55.557289+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581727+0000","last_change":"2023-11-28T18:24:55.576784+0000","last_active":"2023-11-28T18:24:55.581727+0000","last_peered":"2023-11-28T18:24:55.581727+0000","last_clean":"2023-11-28T18:24:55.581727+0000","last_became_active":"2023-11-28T18:24:55.576508+0000","last_became_peered":"2023-11-28T18:24:55.576508+0000","last_unstale":"2023-11-28T18:24:55.581727+0000","last_undegraded":"2023-11-28T18:24:55.581727+0000","last_fullsized":"2023-11-28T18:24:55.581727+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:20:38.032106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563243+0000","last_change":"2023-11-28T18:24:55.563243+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563243+0000","last_undegraded":"2023-11-28T18:24:55.563243+0000","last_fullsized":"2023-11-28T18:24:55.563243+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.583609+0000","last_change":"2023-11-28T18:24:55.582487+0000","last_active":"2023-11-28T18:24:55.583609+0000","last_peered":"2023-11-28T18:24:55.583609+0000","last_clean":"2023-11-28T18:24:55.583609+0000","last_became_active":"2023-11-28T18:24:55.582038+0000","last_became_peered":"2023-11-28T18:24:55.582038+0000","last_unstale":"2023-11-28T18:24:55.583609+0000","last_undegraded":"2023-11-28T18:24:55.583609+0000","last_fullsized":"2023-11-28T18:24:55.583609+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:46.037101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553428+0000","last_change":"2023-11-28T18:24:55.553428+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553428+0000","last_undegraded":"2023-11-28T18:24:55.553428+0000","last_fullsized":"2023-11-28T18:24:55.553428+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581789+0000","last_change":"2023-11-28T18:24:55.577155+0000","last_active":"2023-11-28T18:24:55.581789+0000","last_peered":"2023-11-28T18:24:55.581789+0000","last_clean":"2023-11-28T18:24:55.581789+0000","last_became_active":"2023-11-28T18:24:55.576899+0000","last_became_peered":"2023-11-28T18:24:55.576899+0000","last_unstale":"2023-11-28T18:24:55.581789+0000","last_undegraded":"2023-11-28T18:24:55.581789+0000","last_fullsized":"2023-11-28T18:24:55.581789+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:18:46.680968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558068+0000","last_change":"2023-11-28T18:24:55.558068+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558068+0000","last_undegraded":"2023-11-28T18:24:55.558068+0000","last_fullsized":"2023-11-28T18:24:55.558068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.581668+0000","last_change":"2023-11-28T18:24:55.579931+0000","last_active":"2023-11-28T18:24:55.581668+0000","last_peered":"2023-11-28T18:24:55.581668+0000","last_clean":"2023-11-28T18:24:55.581668+0000","last_became_active":"2023-11-28T18:24:55.579568+0000","last_became_peered":"2023-11-28T18:24:55.579568+0000","last_unstale":"2023-11-28T18:24:55.581668+0000","last_undegraded":"2023-11-28T18:24:55.581668+0000","last_fullsized":"2023-11-28T18:24:55.581668+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:47:09.714446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.563929+0000","last_change":"2023-11-28T18:24:55.563929+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.563929+0000","last_undegraded":"2023-11-28T18:24:55.563929+0000","last_fullsized":"2023-11-28T18:24:55.563929+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.584207+0000","last_change":"2023-11-28T18:24:55.580960+0000","last_active":"2023-11-28T18:24:55.584207+0000","last_peered":"2023-11-28T18:24:55.584207+0000","last_clean":"2023-11-28T18:24:55.584207+0000","last_became_active":"2023-11-28T18:24:55.580717+0000","last_became_peered":"2023-11-28T18:24:55.580717+0000","last_unstale":"2023-11-28T18:24:55.584207+0000","last_undegraded":"2023-11-28T18:24:55.584207+0000","last_fullsized":"2023-11-28T18:24:55.584207+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:15:07.052209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.553436+0000","last_change":"2023-11-28T18:24:55.553436+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.553436+0000","last_undegraded":"2023-11-28T18:24:55.553436+0000","last_fullsized":"2023-11-28T18:24:55.553436+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.573984+0000","last_change":"2023-11-28T18:24:55.569035+0000","last_active":"2023-11-28T18:24:55.573984+0000","last_peered":"2023-11-28T18:24:55.573984+0000","last_clean":"2023-11-28T18:24:55.573984+0000","last_became_active":"2023-11-28T18:24:55.568744+0000","last_became_peered":"2023-11-28T18:24:55.568744+0000","last_unstale":"2023-11-28T18:24:55.573984+0000","last_undegraded":"2023-11-28T18:24:55.573984+0000","last_fullsized":"2023-11-28T18:24:55.573984+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:47:09.714446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.557761+0000","last_change":"2023-11-28T18:24:55.557761+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.557761+0000","last_undegraded":"2023-11-28T18:24:55.557761+0000","last_fullsized":"2023-11-28T18:24:55.557761+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.568346+0000","last_change":"2023-11-28T18:24:55.565390+0000","last_active":"2023-11-28T18:24:55.568346+0000","last_peered":"2023-11-28T18:24:55.568346+0000","last_clean":"2023-11-28T18:24:55.568346+0000","last_became_active":"2023-11-28T18:24:55.565112+0000","last_became_peered":"2023-11-28T18:24:55.565112+0000","last_unstale":"2023-11-28T18:24:55.568346+0000","last_undegraded":"2023-11-28T18:24:55.568346+0000","last_fullsized":"2023-11-28T18:24:55.568346+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:46.037101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.558220+0000","last_change":"2023-11-28T18:24:55.558220+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.558220+0000","last_undegraded":"2023-11-28T18:24:55.558220+0000","last_fullsized":"2023-11-28T18:24:55.558220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.569353+0000","last_change":"2023-11-28T18:24:55.567103+0000","last_active":"2023-11-28T18:24:55.569353+0000","last_peered":"2023-11-28T18:24:55.569353+0000","last_clean":"2023-11-28T18:24:55.569353+0000","last_became_active":"2023-11-28T18:24:55.566687+0000","last_became_peered":"2023-11-28T18:24:55.566687+0000","last_unstale":"2023-11-28T18:24:55.569353+0000","last_undegraded":"2023-11-28T18:24:55.569353+0000","last_fullsized":"2023-11-28T18:24:55.569353+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:06.790640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.556256+0000","last_change":"2023-11-28T18:24:55.556256+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.556256+0000","last_undegraded":"2023-11-28T18:24:55.556256+0000","last_fullsized":"2023-11-28T18:24:55.556256+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.577756+0000","last_change":"2023-11-28T18:24:55.572730+0000","last_active":"2023-11-28T18:24:55.577756+0000","last_peered":"2023-11-28T18:24:55.577756+0000","last_clean":"2023-11-28T18:24:55.577756+0000","last_became_active":"2023-11-28T18:24:55.572440+0000","last_became_peered":"2023-11-28T18:24:55.572440+0000","last_unstale":"2023-11-28T18:24:55.577756+0000","last_undegraded":"2023-11-28T18:24:55.577756+0000","last_fullsized":"2023-11-28T18:24:55.577756+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:44.881911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.562201+0000","last_change":"2023-11-28T18:24:55.562201+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.562201+0000","last_undegraded":"2023-11-28T18:24:55.562201+0000","last_fullsized":"2023-11-28T18:24:55.562201+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.569553+0000","last_change":"2023-11-28T18:24:55.564212+0000","last_active":"2023-11-28T18:24:55.569553+0000","last_peered":"2023-11-28T18:24:55.569553+0000","last_clean":"2023-11-28T18:24:55.569553+0000","last_became_active":"2023-11-28T18:24:55.561715+0000","last_became_peered":"2023-11-28T18:24:55.561715+0000","last_unstale":"2023-11-28T18:24:55.569553+0000","last_undegraded":"2023-11-28T18:24:55.569553+0000","last_fullsized":"2023-11-28T18:24:55.569553+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:42:09.903901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T18:24:55.564919+0000","last_change":"2023-11-28T18:24:55.564919+0000","last_active":"2023-11-28T18:24:55.539940+0000","last_peered":"2023-11-28T18:24:55.539940+0000","last_clean":"2023-11-28T18:24:55.539940+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T18:24:55.564919+0000","last_undegraded":"2023-11-28T18:24:55.564919+0000","last_fullsized":"2023-11-28T18:24:55.564919+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-28T18:24:55.584030+0000","last_change":"2023-11-28T18:24:55.582776+0000","last_active":"2023-11-28T18:24:55.584030+0000","last_peered":"2023-11-28T18:24:55.584030+0000","last_clean":"2023-11-28T18:24:55.584030+0000","last_became_active":"2023-11-28T18:24:55.582457+0000","last_became_peered":"2023-11-28T18:24:55.582457+0000","last_unstale":"2023-11-28T18:24:55.584030+0000","last_undegraded":"2023-11-28T18:24:55.584030+0000","last_fullsized":"2023-11-28T18:24:55.584030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:29:45.071843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T18:24:54.547971+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:24:54.547971+0000","last_peered":"2023-11-28T18:24:54.547971+0000","last_clean":"2023-11-28T18:24:54.547971+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:24:54.547971+0000","last_undegraded":"2023-11-28T18:24:54.547971+0000","last_fullsized":"2023-11-28T18:24:54.547971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":765469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.63}]},{"osd":1,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.68999999999999995}]},{"osd":2,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.411,"5min":2.411,"15min":2.411},"last":0.59499999999999997}]},{"osd":3,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.3650000000000002,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":0.747}]},{"osd":4,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.46200000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.55400000000000005}]},{"osd":6,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.61299999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":175101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.55400000000000005}]},{"osd":1,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":4.2389999999999999,"5min":4.2389999999999999,"15min":4.2389999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":4.218,"5min":4.218,"15min":4.218},"last":0.67100000000000004}]},{"osd":2,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.69799999999999995}]},{"osd":3,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.64500000000000002}]},{"osd":4,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.623},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.71799999999999997}]},{"osd":5,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.73999999999999999}]},{"osd":7,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.752},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.84399999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":175101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.8929999999999998,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.9140000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":0.69399999999999995}]},{"osd":1,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.4809999999999999,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.495}]},{"osd":2,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":3.9510000000000001,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":4.0149999999999997,"5min":4.0149999999999997,"15min":4.0149999999999997},"last":0.58999999999999997}]},{"osd":3,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":4.0869999999999997,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.9900000000000002,"5min":3.9900000000000002,"15min":3.9900000000000002},"last":0.63800000000000001}]},{"osd":4,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.44},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.67500000000000004}]},{"osd":6,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.64700000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.66300000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":175101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":2.363,"5min":2.363,"15min":2.363},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.49399999999999999}]},{"osd":1,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.64200000000000002}]},{"osd":2,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.752},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":0.73499999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.60899999999999999}]},{"osd":5,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.71999999999999997}]},{"osd":6,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.56499999999999995}]},{"osd":7,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.69499999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":168506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.40400000000000003}]},{"osd":1,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46899999999999997}]},{"osd":2,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.49399999999999999}]},{"osd":4,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.64900000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.5019999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.60299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":168506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.56000000000000005}]},{"osd":1,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.63800000000000001}]},{"osd":3,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.66200000000000003}]},{"osd":4,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.71999999999999997}]},{"osd":5,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":7.4820000000000002,"5min":7.4820000000000002,"15min":7.4820000000000002},"last":0.83199999999999996}]},{"osd":6,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.749},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":3.9380000000000002,"5min":3.9380000000000002,"15min":3.9380000000000002},"last":0.77800000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":4.798,"5min":4.798,"15min":4.798},"last":0.82099999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":175101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.436}]},{"osd":2,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.53000000000000003}]},{"osd":3,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.502}]},{"osd":4,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.65100000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.59399999999999997}]},{"osd":6,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.72599999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":765469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.442},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.45800000000000002}]},{"osd":2,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.42599999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.48899999999999999}]},{"osd":4,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.624},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.64500000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.68700000000000006}]},{"osd":6,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.66100000000000003}]},{"osd":7,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.57099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:25:05.166 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:25:05.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.310+0000 7fdc23f2a640 1 Processor -- start 2023-11-28T18:25:05.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.310+0000 7fdc23f2a640 1 -- start start 2023-11-28T18:25:05.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.310+0000 7fdc23f2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc1c15d4f0 0x7fdc1c0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.310+0000 7fdc23f2a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdc1c0b2d20 0x7fdc1c0b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.310+0000 7fdc23f2a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1c0b25b0 con 0x7fdc1c0d3c30 2023-11-28T18:25:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.310+0000 7fdc23f2a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1c0b2900 con 0x7fdc1c0b2d20 2023-11-28T18:25:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.310+0000 7fdc23f2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0d0a20 con 0x7fdc1c15d4f0 2023-11-28T18:25:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.310+0000 7fdc224c7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdc1c0b2d20 0x7fdc1c0b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.310+0000 7fdc224c7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdc1c0b2d20 0x7fdc1c0b31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53858/0 (socket says 172.21.15.105:53858) 2023-11-28T18:25:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.310+0000 7fdc224c7640 1 -- 172.21.15.105:0/2658885965 learned_addr learned my addr 172.21.15.105:0/2658885965 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.310+0000 7fdc224c7640 1 -- 172.21.15.105:0/2658885965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc1c15d4f0 msgr2=0x7fdc1c0d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.310+0000 7fdc21cc6640 1 --2- 172.21.15.105:0/2658885965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc1c15d4f0 0x7fdc1c0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.310+0000 7fdc224c7640 1 --2- 172.21.15.105:0/2658885965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc1c15d4f0 0x7fdc1c0d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.310+0000 7fdc224c7640 1 -- 172.21.15.105:0/2658885965 >> v1:172.21.15.148:6789/0 conn(0x7fdc1c0d3c30 legacy=0x7fdc1c0d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.310+0000 7fdc224c7640 1 -- 172.21.15.105:0/2658885965 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c0cea30 con 0x7fdc1c0b2d20 2023-11-28T18:25:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.310+0000 7fdc21cc6640 1 --2- 172.21.15.105:0/2658885965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc1c15d4f0 0x7fdc1c0d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.314+0000 7fdc224c7640 1 --2- 172.21.15.105:0/2658885965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc1c0b2d20 0x7fdc1c0b31d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009e50 tx=0x7fdc1401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c878baadbba189 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.314+0000 7fdc20cc4640 1 -- 172.21.15.105:0/2658885965 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc140a5790 con 0x7fdc1c0b2d20 2023-11-28T18:25:05.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.314+0000 7fdc20cc4640 1 -- 172.21.15.105:0/2658885965 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc140a58f0 con 0x7fdc1c0b2d20 2023-11-28T18:25:05.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.314+0000 7fdc20cc4640 1 -- 172.21.15.105:0/2658885965 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc140a5c80 con 0x7fdc1c0b2d20 2023-11-28T18:25:05.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.314+0000 7fdc23f2a640 1 -- 172.21.15.105:0/2658885965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc1c0b2d20 msgr2=0x7fdc1c0b31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:05.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.314+0000 7fdc23f2a640 1 --2- 172.21.15.105:0/2658885965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc1c0b2d20 0x7fdc1c0b31d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009e50 tx=0x7fdc1401c910 comp rx=0 tx=0).stop 2023-11-28T18:25:05.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.314+0000 7fdc23f2a640 1 -- 172.21.15.105:0/2658885965 shutdown_connections 2023-11-28T18:25:05.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.314+0000 7fdc23f2a640 1 --2- 172.21.15.105:0/2658885965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc1c0b2d20 0x7fdc1c0b31d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:05.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.314+0000 7fdc23f2a640 1 --2- 172.21.15.105:0/2658885965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc1c15d4f0 0x7fdc1c0d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:05.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.314+0000 7fdc23f2a640 1 -- 172.21.15.105:0/2658885965 >> 172.21.15.105:0/2658885965 conn(0x7fdc1c0d3030 msgr2=0x7fdc1c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:05.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.314+0000 7fdc23f2a640 1 -- 172.21.15.105:0/2658885965 shutdown_connections 2023-11-28T18:25:05.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.314+0000 7fdc23f2a640 1 -- 172.21.15.105:0/2658885965 wait complete. 2023-11-28T18:25:05.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc23f2a640 1 Processor -- start 2023-11-28T18:25:05.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc23f2a640 1 -- start start 2023-11-28T18:25:05.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc23f2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc1c0b2d20 0x7fdc1c0c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:05.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc23f2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc1c0d3c30 0x7fdc1c0cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:05.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc23f2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc1c15d4f0 0x7fdc1c0b7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:05.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc23f2a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0b89f0 con 0x7fdc1c0d3c30 2023-11-28T18:25:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc23f2a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0c8560 con 0x7fdc1c0b2d20 2023-11-28T18:25:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc23f2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0b8710 con 0x7fdc1c15d4f0 2023-11-28T18:25:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc214c5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc1c0d3c30 0x7fdc1c0cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc214c5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc1c0d3c30 0x7fdc1c0cc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53866/0 (socket says 172.21.15.105:53866) 2023-11-28T18:25:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc214c5640 1 -- 172.21.15.105:0/4288226079 learned_addr learned my addr 172.21.15.105:0/4288226079 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc224c7640 1 --2- 172.21.15.105:0/4288226079 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc1c15d4f0 0x7fdc1c0b7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:05.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc21cc6640 1 --2- 172.21.15.105:0/4288226079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc1c0b2d20 0x7fdc1c0c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:05.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc214c5640 1 -- 172.21.15.105:0/4288226079 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc1c15d4f0 msgr2=0x7fdc1c0b7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:05.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc214c5640 1 --2- 172.21.15.105:0/4288226079 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc1c15d4f0 0x7fdc1c0b7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:05.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc214c5640 1 -- 172.21.15.105:0/4288226079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc1c0b2d20 msgr2=0x7fdc1c0c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:05.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc214c5640 1 --2- 172.21.15.105:0/4288226079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc1c0b2d20 0x7fdc1c0c9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:05.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc214c5640 1 -- 172.21.15.105:0/4288226079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc14009b30 con 0x7fdc1c0d3c30 2023-11-28T18:25:05.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc21cc6640 1 --2- 172.21.15.105:0/4288226079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc1c0b2d20 0x7fdc1c0c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:05.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc214c5640 1 --2- 172.21.15.105:0/4288226079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc1c0d3c30 0x7fdc1c0cc8e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00b410 tx=0x7fdc0c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:05.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc127fc640 1 -- 172.21.15.105:0/4288226079 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c004020 con 0x7fdc1c0d3c30 2023-11-28T18:25:05.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc127fc640 1 -- 172.21.15.105:0/4288226079 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0c004180 con 0x7fdc1c0d3c30 2023-11-28T18:25:05.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc127fc640 1 -- 172.21.15.105:0/4288226079 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c011630 con 0x7fdc1c0d3c30 2023-11-28T18:25:05.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc23f2a640 1 -- 172.21.15.105:0/4288226079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1c0c8800 con 0x7fdc1c0d3c30 2023-11-28T18:25:05.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.318+0000 7fdc23f2a640 1 -- 172.21.15.105:0/4288226079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c0b8270 con 0x7fdc1c0d3c30 2023-11-28T18:25:05.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.322+0000 7fdc23f2a640 1 -- 172.21.15.105:0/4288226079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc1c067570 con 0x7fdc1c0d3c30 2023-11-28T18:25:05.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.322+0000 7fdc127fc640 1 -- 172.21.15.105:0/4288226079 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fdc0c011790 con 0x7fdc1c0d3c30 2023-11-28T18:25:05.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.322+0000 7fdc127fc640 1 --2- 172.21.15.105:0/4288226079 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdc000668d0 0x7fdc00068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:05.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.322+0000 7fdc127fc640 1 -- 172.21.15.105:0/4288226079 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdc0c084210 con 0x7fdc1c0d3c30 2023-11-28T18:25:05.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.322+0000 7fdc21cc6640 1 --2- 172.21.15.105:0/4288226079 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdc000668d0 0x7fdc00068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:05.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.322+0000 7fdc21cc6640 1 --2- 172.21.15.105:0/4288226079 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdc000668d0 0x7fdc00068d90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdc18006fd0 tx=0x7fdc18008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:05.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.326+0000 7fdc127fc640 1 -- 172.21.15.105:0/4288226079 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdc0c051300 con 0x7fdc1c0d3c30 2023-11-28T18:25:05.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.474+0000 7fdc23f2a640 1 -- 172.21.15.105:0/4288226079 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdc1c0b84f0 con 0x7fdc000668d0 2023-11-28T18:25:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.482+0000 7fdc127fc640 1 -- 172.21.15.105:0/4288226079 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289574 (secure 0 0 0) 0x7fdc1800f860 con 0x7fdc000668d0 2023-11-28T18:25:05.490 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:25:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.486+0000 7fdc23f2a640 1 -- 172.21.15.105:0/4288226079 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdc000668d0 msgr2=0x7fdc00068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.486+0000 7fdc23f2a640 1 --2- 172.21.15.105:0/4288226079 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdc000668d0 0x7fdc00068d90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdc18006fd0 tx=0x7fdc18008040 comp rx=0 tx=0).stop 2023-11-28T18:25:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.486+0000 7fdc23f2a640 1 -- 172.21.15.105:0/4288226079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc1c0d3c30 msgr2=0x7fdc1c0cc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.486+0000 7fdc23f2a640 1 --2- 172.21.15.105:0/4288226079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc1c0d3c30 0x7fdc1c0cc8e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00b410 tx=0x7fdc0c00b8d0 comp rx=0 tx=0).stop 2023-11-28T18:25:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.486+0000 7fdc23f2a640 1 -- 172.21.15.105:0/4288226079 shutdown_connections 2023-11-28T18:25:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.486+0000 7fdc23f2a640 1 --2- 172.21.15.105:0/4288226079 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc1c15d4f0 0x7fdc1c0b7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.486+0000 7fdc23f2a640 1 --2- 172.21.15.105:0/4288226079 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdc000668d0 0x7fdc00068d90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:05.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.486+0000 7fdc23f2a640 1 --2- 172.21.15.105:0/4288226079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc1c0d3c30 0x7fdc1c0cc8e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:05.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.486+0000 7fdc23f2a640 1 --2- 172.21.15.105:0/4288226079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc1c0b2d20 0x7fdc1c0c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:05.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.486+0000 7fdc23f2a640 1 -- 172.21.15.105:0/4288226079 >> 172.21.15.105:0/4288226079 conn(0x7fdc1c0d3030 msgr2=0x7fdc1c06c820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:05.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.486+0000 7fdc23f2a640 1 -- 172.21.15.105:0/4288226079 shutdown_connections 2023-11-28T18:25:05.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.486+0000 7fdc23f2a640 1 -- 172.21.15.105:0/4288226079 wait complete. 2023-11-28T18:25:05.510 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2023-11-28T18:25:05.288207+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220864,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753856,"statfs":{"total":773094113280,"available":772867948544,"internally_reserved":0,"allocated":4759552,"data_stored":2696084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.734638"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.556140+0000","last_change":"2023-11-28T18:24:56.579294+0000","last_active":"2023-11-28T18:24:57.556140+0000","last_peered":"2023-11-28T18:24:57.556140+0000","last_clean":"2023-11-28T18:24:57.556140+0000","last_became_active":"2023-11-28T18:24:56.579028+0000","last_became_peered":"2023-11-28T18:24:56.579028+0000","last_unstale":"2023-11-28T18:24:57.556140+0000","last_undegraded":"2023-11-28T18:24:57.556140+0000","last_fullsized":"2023-11-28T18:24:57.556140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:25:11.882712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.555727+0000","last_change":"2023-11-28T18:24:56.567392+0000","last_active":"2023-11-28T18:24:57.555727+0000","last_peered":"2023-11-28T18:24:57.555727+0000","last_clean":"2023-11-28T18:24:57.555727+0000","last_became_active":"2023-11-28T18:24:56.567155+0000","last_became_peered":"2023-11-28T18:24:56.567155+0000","last_unstale":"2023-11-28T18:24:57.555727+0000","last_undegraded":"2023-11-28T18:24:57.555727+0000","last_fullsized":"2023-11-28T18:24:57.555727+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:07.551619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.733409+0000","last_change":"2023-11-28T18:24:56.580978+0000","last_active":"2023-11-28T18:24:57.733409+0000","last_peered":"2023-11-28T18:24:57.733409+0000","last_clean":"2023-11-28T18:24:57.733409+0000","last_became_active":"2023-11-28T18:24:56.580610+0000","last_became_peered":"2023-11-28T18:24:56.580610+0000","last_unstale":"2023-11-28T18:24:57.733409+0000","last_undegraded":"2023-11-28T18:24:57.733409+0000","last_fullsized":"2023-11-28T18:24:57.733409+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:46.112058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.733146+0000","last_change":"2023-11-28T18:24:56.561163+0000","last_active":"2023-11-28T18:24:57.733146+0000","last_peered":"2023-11-28T18:24:57.733146+0000","last_clean":"2023-11-28T18:24:57.733146+0000","last_became_active":"2023-11-28T18:24:56.560899+0000","last_became_peered":"2023-11-28T18:24:56.560899+0000","last_unstale":"2023-11-28T18:24:57.733146+0000","last_undegraded":"2023-11-28T18:24:57.733146+0000","last_fullsized":"2023-11-28T18:24:57.733146+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:50:56.089347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.733019+0000","last_change":"2023-11-28T18:24:56.560891+0000","last_active":"2023-11-28T18:24:57.733019+0000","last_peered":"2023-11-28T18:24:57.733019+0000","last_clean":"2023-11-28T18:24:57.733019+0000","last_became_active":"2023-11-28T18:24:56.560582+0000","last_became_peered":"2023-11-28T18:24:56.560582+0000","last_unstale":"2023-11-28T18:24:57.733019+0000","last_undegraded":"2023-11-28T18:24:57.733019+0000","last_fullsized":"2023-11-28T18:24:57.733019+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:17:09.520913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.732350+0000","last_change":"2023-11-28T18:24:56.581811+0000","last_active":"2023-11-28T18:24:57.732350+0000","last_peered":"2023-11-28T18:24:57.732350+0000","last_clean":"2023-11-28T18:24:57.732350+0000","last_became_active":"2023-11-28T18:24:56.581495+0000","last_became_peered":"2023-11-28T18:24:56.581495+0000","last_unstale":"2023-11-28T18:24:57.732350+0000","last_undegraded":"2023-11-28T18:24:57.732350+0000","last_fullsized":"2023-11-28T18:24:57.732350+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:30:07.542313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.732658+0000","last_change":"2023-11-28T18:24:56.585908+0000","last_active":"2023-11-28T18:24:57.732658+0000","last_peered":"2023-11-28T18:24:57.732658+0000","last_clean":"2023-11-28T18:24:57.732658+0000","last_became_active":"2023-11-28T18:24:56.584833+0000","last_became_peered":"2023-11-28T18:24:56.584833+0000","last_unstale":"2023-11-28T18:24:57.732658+0000","last_undegraded":"2023-11-28T18:24:57.732658+0000","last_fullsized":"2023-11-28T18:24:57.732658+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:07.551619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.555519+0000","last_change":"2023-11-28T18:24:56.578774+0000","last_active":"2023-11-28T18:24:57.555519+0000","last_peered":"2023-11-28T18:24:57.555519+0000","last_clean":"2023-11-28T18:24:57.555519+0000","last_became_active":"2023-11-28T18:24:56.577025+0000","last_became_peered":"2023-11-28T18:24:56.577025+0000","last_unstale":"2023-11-28T18:24:57.555519+0000","last_undegraded":"2023-11-28T18:24:57.555519+0000","last_fullsized":"2023-11-28T18:24:57.555519+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:22.586713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.734071+0000","last_change":"2023-11-28T18:24:56.576270+0000","last_active":"2023-11-28T18:24:57.734071+0000","last_peered":"2023-11-28T18:24:57.734071+0000","last_clean":"2023-11-28T18:24:57.734071+0000","last_became_active":"2023-11-28T18:24:56.575218+0000","last_became_peered":"2023-11-28T18:24:56.575218+0000","last_unstale":"2023-11-28T18:24:57.734071+0000","last_undegraded":"2023-11-28T18:24:57.734071+0000","last_fullsized":"2023-11-28T18:24:57.734071+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:51.562902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.733680+0000","last_change":"2023-11-28T18:24:56.586587+0000","last_active":"2023-11-28T18:24:57.733680+0000","last_peered":"2023-11-28T18:24:57.733680+0000","last_clean":"2023-11-28T18:24:57.733680+0000","last_became_active":"2023-11-28T18:24:56.586295+0000","last_became_peered":"2023-11-28T18:24:56.586295+0000","last_unstale":"2023-11-28T18:24:57.733680+0000","last_undegraded":"2023-11-28T18:24:57.733680+0000","last_fullsized":"2023-11-28T18:24:57.733680+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:35:33.909227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.733247+0000","last_change":"2023-11-28T18:24:56.575000+0000","last_active":"2023-11-28T18:24:57.733247+0000","last_peered":"2023-11-28T18:24:57.733247+0000","last_clean":"2023-11-28T18:24:57.733247+0000","last_became_active":"2023-11-28T18:24:56.574427+0000","last_became_peered":"2023-11-28T18:24:56.574427+0000","last_unstale":"2023-11-28T18:24:57.733247+0000","last_undegraded":"2023-11-28T18:24:57.733247+0000","last_fullsized":"2023-11-28T18:24:57.733247+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:25:11.882712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.733473+0000","last_change":"2023-11-28T18:24:56.574583+0000","last_active":"2023-11-28T18:24:57.733473+0000","last_peered":"2023-11-28T18:24:57.733473+0000","last_clean":"2023-11-28T18:24:57.733473+0000","last_became_active":"2023-11-28T18:24:56.574269+0000","last_became_peered":"2023-11-28T18:24:56.574269+0000","last_unstale":"2023-11-28T18:24:57.733473+0000","last_undegraded":"2023-11-28T18:24:57.733473+0000","last_fullsized":"2023-11-28T18:24:57.733473+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:22.586713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.732959+0000","last_change":"2023-11-28T18:24:56.578086+0000","last_active":"2023-11-28T18:24:57.732959+0000","last_peered":"2023-11-28T18:24:57.732959+0000","last_clean":"2023-11-28T18:24:57.732959+0000","last_became_active":"2023-11-28T18:24:56.577801+0000","last_became_peered":"2023-11-28T18:24:56.577801+0000","last_unstale":"2023-11-28T18:24:57.732959+0000","last_undegraded":"2023-11-28T18:24:57.732959+0000","last_fullsized":"2023-11-28T18:24:57.732959+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:32:57.901739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.556974+0000","last_change":"2023-11-28T18:24:56.590869+0000","last_active":"2023-11-28T18:24:57.556974+0000","last_peered":"2023-11-28T18:24:57.556974+0000","last_clean":"2023-11-28T18:24:57.556974+0000","last_became_active":"2023-11-28T18:24:56.584557+0000","last_became_peered":"2023-11-28T18:24:56.584557+0000","last_unstale":"2023-11-28T18:24:57.556974+0000","last_undegraded":"2023-11-28T18:24:57.556974+0000","last_fullsized":"2023-11-28T18:24:57.556974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:12:53.759750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.732785+0000","last_change":"2023-11-28T18:24:56.574990+0000","last_active":"2023-11-28T18:24:57.732785+0000","last_peered":"2023-11-28T18:24:57.732785+0000","last_clean":"2023-11-28T18:24:57.732785+0000","last_became_active":"2023-11-28T18:24:56.574637+0000","last_became_peered":"2023-11-28T18:24:56.574637+0000","last_unstale":"2023-11-28T18:24:57.732785+0000","last_undegraded":"2023-11-28T18:24:57.732785+0000","last_fullsized":"2023-11-28T18:24:57.732785+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:03.821752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.732606+0000","last_change":"2023-11-28T18:24:56.584717+0000","last_active":"2023-11-28T18:24:57.732606+0000","last_peered":"2023-11-28T18:24:57.732606+0000","last_clean":"2023-11-28T18:24:57.732606+0000","last_became_active":"2023-11-28T18:24:56.584382+0000","last_became_peered":"2023-11-28T18:24:56.584382+0000","last_unstale":"2023-11-28T18:24:57.732606+0000","last_undegraded":"2023-11-28T18:24:57.732606+0000","last_fullsized":"2023-11-28T18:24:57.732606+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:04:47.721518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.732945+0000","last_change":"2023-11-28T18:24:56.583390+0000","last_active":"2023-11-28T18:24:57.732945+0000","last_peered":"2023-11-28T18:24:57.732945+0000","last_clean":"2023-11-28T18:24:57.732945+0000","last_became_active":"2023-11-28T18:24:56.583117+0000","last_became_peered":"2023-11-28T18:24:56.583117+0000","last_unstale":"2023-11-28T18:24:57.732945+0000","last_undegraded":"2023-11-28T18:24:57.732945+0000","last_fullsized":"2023-11-28T18:24:57.732945+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:07.551619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.558477+0000","last_change":"2023-11-28T18:24:56.589489+0000","last_active":"2023-11-28T18:24:57.558477+0000","last_peered":"2023-11-28T18:24:57.558477+0000","last_clean":"2023-11-28T18:24:57.558477+0000","last_became_active":"2023-11-28T18:24:56.589221+0000","last_became_peered":"2023-11-28T18:24:56.589221+0000","last_unstale":"2023-11-28T18:24:57.558477+0000","last_undegraded":"2023-11-28T18:24:57.558477+0000","last_fullsized":"2023-11-28T18:24:57.558477+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:32:22.061384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.733267+0000","last_change":"2023-11-28T18:24:56.575965+0000","last_active":"2023-11-28T18:24:57.733267+0000","last_peered":"2023-11-28T18:24:57.733267+0000","last_clean":"2023-11-28T18:24:57.733267+0000","last_became_active":"2023-11-28T18:24:56.575678+0000","last_became_peered":"2023-11-28T18:24:56.575678+0000","last_unstale":"2023-11-28T18:24:57.733267+0000","last_undegraded":"2023-11-28T18:24:57.733267+0000","last_fullsized":"2023-11-28T18:24:57.733267+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:47.045250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.558466+0000","last_change":"2023-11-28T18:24:56.589407+0000","last_active":"2023-11-28T18:24:57.558466+0000","last_peered":"2023-11-28T18:24:57.558466+0000","last_clean":"2023-11-28T18:24:57.558466+0000","last_became_active":"2023-11-28T18:24:56.588852+0000","last_became_peered":"2023-11-28T18:24:56.588852+0000","last_unstale":"2023-11-28T18:24:57.558466+0000","last_undegraded":"2023-11-28T18:24:57.558466+0000","last_fullsized":"2023-11-28T18:24:57.558466+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:42:45.677994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.733176+0000","last_change":"2023-11-28T18:24:56.588773+0000","last_active":"2023-11-28T18:24:57.733176+0000","last_peered":"2023-11-28T18:24:57.733176+0000","last_clean":"2023-11-28T18:24:57.733176+0000","last_became_active":"2023-11-28T18:24:56.588497+0000","last_became_peered":"2023-11-28T18:24:56.588497+0000","last_unstale":"2023-11-28T18:24:57.733176+0000","last_undegraded":"2023-11-28T18:24:57.733176+0000","last_fullsized":"2023-11-28T18:24:57.733176+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:23:45.541371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.557763+0000","last_change":"2023-11-28T18:24:56.564595+0000","last_active":"2023-11-28T18:24:57.557763+0000","last_peered":"2023-11-28T18:24:57.557763+0000","last_clean":"2023-11-28T18:24:57.557763+0000","last_became_active":"2023-11-28T18:24:56.564304+0000","last_became_peered":"2023-11-28T18:24:56.564304+0000","last_unstale":"2023-11-28T18:24:57.557763+0000","last_undegraded":"2023-11-28T18:24:57.557763+0000","last_fullsized":"2023-11-28T18:24:57.557763+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:30:07.542313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.732742+0000","last_change":"2023-11-28T18:24:56.582844+0000","last_active":"2023-11-28T18:24:57.732742+0000","last_peered":"2023-11-28T18:24:57.732742+0000","last_clean":"2023-11-28T18:24:57.732742+0000","last_became_active":"2023-11-28T18:24:56.582539+0000","last_became_peered":"2023-11-28T18:24:56.582539+0000","last_unstale":"2023-11-28T18:24:57.732742+0000","last_undegraded":"2023-11-28T18:24:57.732742+0000","last_fullsized":"2023-11-28T18:24:57.732742+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:07:54.626598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.732655+0000","last_change":"2023-11-28T18:24:56.590580+0000","last_active":"2023-11-28T18:24:57.732655+0000","last_peered":"2023-11-28T18:24:57.732655+0000","last_clean":"2023-11-28T18:24:57.732655+0000","last_became_active":"2023-11-28T18:24:56.590254+0000","last_became_peered":"2023-11-28T18:24:56.590254+0000","last_unstale":"2023-11-28T18:24:57.732655+0000","last_undegraded":"2023-11-28T18:24:57.732655+0000","last_fullsized":"2023-11-28T18:24:57.732655+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:40:46.454734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.558549+0000","last_change":"2023-11-28T18:24:56.587254+0000","last_active":"2023-11-28T18:24:57.558549+0000","last_peered":"2023-11-28T18:24:57.558549+0000","last_clean":"2023-11-28T18:24:57.558549+0000","last_became_active":"2023-11-28T18:24:56.582252+0000","last_became_peered":"2023-11-28T18:24:56.582252+0000","last_unstale":"2023-11-28T18:24:57.558549+0000","last_undegraded":"2023-11-28T18:24:57.558549+0000","last_fullsized":"2023-11-28T18:24:57.558549+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:12:53.759750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.732451+0000","last_change":"2023-11-28T18:24:56.579951+0000","last_active":"2023-11-28T18:24:57.732451+0000","last_peered":"2023-11-28T18:24:57.732451+0000","last_clean":"2023-11-28T18:24:57.732451+0000","last_became_active":"2023-11-28T18:24:56.579646+0000","last_became_peered":"2023-11-28T18:24:56.579646+0000","last_unstale":"2023-11-28T18:24:57.732451+0000","last_undegraded":"2023-11-28T18:24:57.732451+0000","last_fullsized":"2023-11-28T18:24:57.732451+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:27:47.996939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.558579+0000","last_change":"2023-11-28T18:24:56.585574+0000","last_active":"2023-11-28T18:24:57.558579+0000","last_peered":"2023-11-28T18:24:57.558579+0000","last_clean":"2023-11-28T18:24:57.558579+0000","last_became_active":"2023-11-28T18:24:56.585275+0000","last_became_peered":"2023-11-28T18:24:56.585275+0000","last_unstale":"2023-11-28T18:24:57.558579+0000","last_undegraded":"2023-11-28T18:24:57.558579+0000","last_fullsized":"2023-11-28T18:24:57.558579+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:55:58.162074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.733302+0000","last_change":"2023-11-28T18:24:56.585549+0000","last_active":"2023-11-28T18:24:57.733302+0000","last_peered":"2023-11-28T18:24:57.733302+0000","last_clean":"2023-11-28T18:24:57.733302+0000","last_became_active":"2023-11-28T18:24:56.585260+0000","last_became_peered":"2023-11-28T18:24:56.585260+0000","last_unstale":"2023-11-28T18:24:57.733302+0000","last_undegraded":"2023-11-28T18:24:57.733302+0000","last_fullsized":"2023-11-28T18:24:57.733302+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:13:18.726572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.558298+0000","last_change":"2023-11-28T18:24:56.593583+0000","last_active":"2023-11-28T18:24:57.558298+0000","last_peered":"2023-11-28T18:24:57.558298+0000","last_clean":"2023-11-28T18:24:57.558298+0000","last_became_active":"2023-11-28T18:24:56.593237+0000","last_became_peered":"2023-11-28T18:24:56.593237+0000","last_unstale":"2023-11-28T18:24:57.558298+0000","last_undegraded":"2023-11-28T18:24:57.558298+0000","last_fullsized":"2023-11-28T18:24:57.558298+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:29:38.612776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.558182+0000","last_change":"2023-11-28T18:24:56.578810+0000","last_active":"2023-11-28T18:24:57.558182+0000","last_peered":"2023-11-28T18:24:57.558182+0000","last_clean":"2023-11-28T18:24:57.558182+0000","last_became_active":"2023-11-28T18:24:56.578467+0000","last_became_peered":"2023-11-28T18:24:56.578467+0000","last_unstale":"2023-11-28T18:24:57.558182+0000","last_undegraded":"2023-11-28T18:24:57.558182+0000","last_fullsized":"2023-11-28T18:24:57.558182+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:15.103285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.732808+0000","last_change":"2023-11-28T18:24:56.593406+0000","last_active":"2023-11-28T18:24:57.732808+0000","last_peered":"2023-11-28T18:24:57.732808+0000","last_clean":"2023-11-28T18:24:57.732808+0000","last_became_active":"2023-11-28T18:24:56.593109+0000","last_became_peered":"2023-11-28T18:24:56.593109+0000","last_unstale":"2023-11-28T18:24:57.732808+0000","last_undegraded":"2023-11-28T18:24:57.732808+0000","last_fullsized":"2023-11-28T18:24:57.732808+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:46.112058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.732235+0000","last_change":"2023-11-28T18:24:56.573250+0000","last_active":"2023-11-28T18:24:57.732235+0000","last_peered":"2023-11-28T18:24:57.732235+0000","last_clean":"2023-11-28T18:24:57.732235+0000","last_became_active":"2023-11-28T18:24:56.572916+0000","last_became_peered":"2023-11-28T18:24:56.572916+0000","last_unstale":"2023-11-28T18:24:57.732235+0000","last_undegraded":"2023-11-28T18:24:57.732235+0000","last_fullsized":"2023-11-28T18:24:57.732235+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:34:24.042019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.733046+0000","last_change":"2023-11-28T18:24:56.587356+0000","last_active":"2023-11-28T18:24:57.733046+0000","last_peered":"2023-11-28T18:24:57.733046+0000","last_clean":"2023-11-28T18:24:57.733046+0000","last_became_active":"2023-11-28T18:24:56.587089+0000","last_became_peered":"2023-11-28T18:24:56.587089+0000","last_unstale":"2023-11-28T18:24:57.733046+0000","last_undegraded":"2023-11-28T18:24:57.733046+0000","last_fullsized":"2023-11-28T18:24:57.733046+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:51.562902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.602647+0000","last_change":"2023-11-28T18:24:55.573384+0000","last_active":"2023-11-28T18:24:56.602647+0000","last_peered":"2023-11-28T18:24:56.602647+0000","last_clean":"2023-11-28T18:24:56.602647+0000","last_became_active":"2023-11-28T18:24:55.571607+0000","last_became_peered":"2023-11-28T18:24:55.571607+0000","last_unstale":"2023-11-28T18:24:56.602647+0000","last_undegraded":"2023-11-28T18:24:56.602647+0000","last_fullsized":"2023-11-28T18:24:56.602647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:47:09.714446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.732781+0000","last_change":"2023-11-28T18:24:56.583509+0000","last_active":"2023-11-28T18:24:57.732781+0000","last_peered":"2023-11-28T18:24:57.732781+0000","last_clean":"2023-11-28T18:24:57.732781+0000","last_became_active":"2023-11-28T18:24:56.579126+0000","last_became_peered":"2023-11-28T18:24:56.579126+0000","last_unstale":"2023-11-28T18:24:57.732781+0000","last_undegraded":"2023-11-28T18:24:57.732781+0000","last_fullsized":"2023-11-28T18:24:57.732781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:30.603278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.600548+0000","last_change":"2023-11-28T18:24:55.573927+0000","last_active":"2023-11-28T18:24:56.600548+0000","last_peered":"2023-11-28T18:24:56.600548+0000","last_clean":"2023-11-28T18:24:56.600548+0000","last_became_active":"2023-11-28T18:24:55.573623+0000","last_became_peered":"2023-11-28T18:24:55.573623+0000","last_unstale":"2023-11-28T18:24:56.600548+0000","last_undegraded":"2023-11-28T18:24:56.600548+0000","last_fullsized":"2023-11-28T18:24:56.600548+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:30:06.534164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.557707+0000","last_change":"2023-11-28T18:24:56.586628+0000","last_active":"2023-11-28T18:24:57.557707+0000","last_peered":"2023-11-28T18:24:57.557707+0000","last_clean":"2023-11-28T18:24:57.557707+0000","last_became_active":"2023-11-28T18:24:56.585518+0000","last_became_peered":"2023-11-28T18:24:56.585518+0000","last_unstale":"2023-11-28T18:24:57.557707+0000","last_undegraded":"2023-11-28T18:24:57.557707+0000","last_fullsized":"2023-11-28T18:24:57.557707+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:49:50.311251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.601589+0000","last_change":"2023-11-28T18:24:55.570144+0000","last_active":"2023-11-28T18:24:56.601589+0000","last_peered":"2023-11-28T18:24:56.601589+0000","last_clean":"2023-11-28T18:24:56.601589+0000","last_became_active":"2023-11-28T18:24:55.569837+0000","last_became_peered":"2023-11-28T18:24:55.569837+0000","last_unstale":"2023-11-28T18:24:56.601589+0000","last_undegraded":"2023-11-28T18:24:56.601589+0000","last_fullsized":"2023-11-28T18:24:56.601589+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:55:57.153924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.556111+0000","last_change":"2023-11-28T18:24:56.583847+0000","last_active":"2023-11-28T18:24:57.556111+0000","last_peered":"2023-11-28T18:24:57.556111+0000","last_clean":"2023-11-28T18:24:57.556111+0000","last_became_active":"2023-11-28T18:24:56.583620+0000","last_became_peered":"2023-11-28T18:24:56.583620+0000","last_unstale":"2023-11-28T18:24:57.556111+0000","last_undegraded":"2023-11-28T18:24:57.556111+0000","last_fullsized":"2023-11-28T18:24:57.556111+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:23:45.541371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.604585+0000","last_change":"2023-11-28T18:24:55.570052+0000","last_active":"2023-11-28T18:24:56.604585+0000","last_peered":"2023-11-28T18:24:56.604585+0000","last_clean":"2023-11-28T18:24:56.604585+0000","last_became_active":"2023-11-28T18:24:55.569712+0000","last_became_peered":"2023-11-28T18:24:55.569712+0000","last_unstale":"2023-11-28T18:24:56.604585+0000","last_undegraded":"2023-11-28T18:24:56.604585+0000","last_fullsized":"2023-11-28T18:24:56.604585+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:42:09.903901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.732589+0000","last_change":"2023-11-28T18:24:56.579626+0000","last_active":"2023-11-28T18:24:57.732589+0000","last_peered":"2023-11-28T18:24:57.732589+0000","last_clean":"2023-11-28T18:24:57.732589+0000","last_became_active":"2023-11-28T18:24:56.575702+0000","last_became_peered":"2023-11-28T18:24:56.575702+0000","last_unstale":"2023-11-28T18:24:57.732589+0000","last_undegraded":"2023-11-28T18:24:57.732589+0000","last_fullsized":"2023-11-28T18:24:57.732589+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:27:47.996939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.600793+0000","last_change":"2023-11-28T18:24:55.575819+0000","last_active":"2023-11-28T18:24:56.600793+0000","last_peered":"2023-11-28T18:24:56.600793+0000","last_clean":"2023-11-28T18:24:56.600793+0000","last_became_active":"2023-11-28T18:24:55.575541+0000","last_became_peered":"2023-11-28T18:24:55.575541+0000","last_unstale":"2023-11-28T18:24:56.600793+0000","last_undegraded":"2023-11-28T18:24:56.600793+0000","last_fullsized":"2023-11-28T18:24:56.600793+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:20:19.054582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.733099+0000","last_change":"2023-11-28T18:24:56.578336+0000","last_active":"2023-11-28T18:24:57.733099+0000","last_peered":"2023-11-28T18:24:57.733099+0000","last_clean":"2023-11-28T18:24:57.733099+0000","last_became_active":"2023-11-28T18:24:56.577291+0000","last_became_peered":"2023-11-28T18:24:56.577291+0000","last_unstale":"2023-11-28T18:24:57.733099+0000","last_undegraded":"2023-11-28T18:24:57.733099+0000","last_fullsized":"2023-11-28T18:24:57.733099+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:18.177165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.603267+0000","last_change":"2023-11-28T18:24:55.578529+0000","last_active":"2023-11-28T18:24:56.603267+0000","last_peered":"2023-11-28T18:24:56.603267+0000","last_clean":"2023-11-28T18:24:56.603267+0000","last_became_active":"2023-11-28T18:24:55.578252+0000","last_became_peered":"2023-11-28T18:24:55.578252+0000","last_unstale":"2023-11-28T18:24:56.603267+0000","last_undegraded":"2023-11-28T18:24:56.603267+0000","last_fullsized":"2023-11-28T18:24:56.603267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:47:09.714446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.557696+0000","last_change":"2023-11-28T18:24:56.569265+0000","last_active":"2023-11-28T18:24:57.557696+0000","last_peered":"2023-11-28T18:24:57.557696+0000","last_clean":"2023-11-28T18:24:57.557696+0000","last_became_active":"2023-11-28T18:24:56.568772+0000","last_became_peered":"2023-11-28T18:24:56.568772+0000","last_unstale":"2023-11-28T18:24:57.557696+0000","last_undegraded":"2023-11-28T18:24:57.557696+0000","last_fullsized":"2023-11-28T18:24:57.557696+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:32:11.498311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.566832+0000","last_change":"2023-11-28T18:24:55.574964+0000","last_active":"2023-11-28T18:24:56.566832+0000","last_peered":"2023-11-28T18:24:56.566832+0000","last_clean":"2023-11-28T18:24:56.566832+0000","last_became_active":"2023-11-28T18:24:55.574669+0000","last_became_peered":"2023-11-28T18:24:55.574669+0000","last_unstale":"2023-11-28T18:24:56.566832+0000","last_undegraded":"2023-11-28T18:24:56.566832+0000","last_fullsized":"2023-11-28T18:24:56.566832+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:42:44.669844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.555682+0000","last_change":"2023-11-28T18:24:56.578730+0000","last_active":"2023-11-28T18:24:57.555682+0000","last_peered":"2023-11-28T18:24:57.555682+0000","last_clean":"2023-11-28T18:24:57.555682+0000","last_became_active":"2023-11-28T18:24:56.577977+0000","last_became_peered":"2023-11-28T18:24:56.577977+0000","last_unstale":"2023-11-28T18:24:57.555682+0000","last_undegraded":"2023-11-28T18:24:57.555682+0000","last_fullsized":"2023-11-28T18:24:57.555682+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:24:42.258629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.557011+0000","last_change":"2023-11-28T18:24:55.563267+0000","last_active":"2023-11-28T18:24:56.557011+0000","last_peered":"2023-11-28T18:24:56.557011+0000","last_clean":"2023-11-28T18:24:56.557011+0000","last_became_active":"2023-11-28T18:24:55.562979+0000","last_became_peered":"2023-11-28T18:24:55.562979+0000","last_unstale":"2023-11-28T18:24:56.557011+0000","last_undegraded":"2023-11-28T18:24:56.557011+0000","last_fullsized":"2023-11-28T18:24:56.557011+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:29:37.604627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.733151+0000","last_change":"2023-11-28T18:24:56.560641+0000","last_active":"2023-11-28T18:24:57.733151+0000","last_peered":"2023-11-28T18:24:57.733151+0000","last_clean":"2023-11-28T18:24:57.733151+0000","last_became_active":"2023-11-28T18:24:56.560335+0000","last_became_peered":"2023-11-28T18:24:56.560335+0000","last_unstale":"2023-11-28T18:24:57.733151+0000","last_undegraded":"2023-11-28T18:24:57.733151+0000","last_fullsized":"2023-11-28T18:24:57.733151+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:29:46.079993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.558363+0000","last_change":"2023-11-28T18:24:55.572743+0000","last_active":"2023-11-28T18:24:56.558363+0000","last_peered":"2023-11-28T18:24:56.558363+0000","last_clean":"2023-11-28T18:24:56.558363+0000","last_became_active":"2023-11-28T18:24:55.571872+0000","last_became_peered":"2023-11-28T18:24:55.571872+0000","last_unstale":"2023-11-28T18:24:56.558363+0000","last_undegraded":"2023-11-28T18:24:56.558363+0000","last_fullsized":"2023-11-28T18:24:56.558363+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:45.103908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.733181+0000","last_change":"2023-11-28T18:24:56.592194+0000","last_active":"2023-11-28T18:24:57.733181+0000","last_peered":"2023-11-28T18:24:57.733181+0000","last_clean":"2023-11-28T18:24:57.733181+0000","last_became_active":"2023-11-28T18:24:56.591846+0000","last_became_peered":"2023-11-28T18:24:56.591846+0000","last_unstale":"2023-11-28T18:24:57.733181+0000","last_undegraded":"2023-11-28T18:24:57.733181+0000","last_fullsized":"2023-11-28T18:24:57.733181+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:03.821752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.566597+0000","last_change":"2023-11-28T18:24:55.579938+0000","last_active":"2023-11-28T18:24:56.566597+0000","last_peered":"2023-11-28T18:24:56.566597+0000","last_clean":"2023-11-28T18:24:56.566597+0000","last_became_active":"2023-11-28T18:24:55.579667+0000","last_became_peered":"2023-11-28T18:24:55.579667+0000","last_unstale":"2023-11-28T18:24:56.566597+0000","last_undegraded":"2023-11-28T18:24:56.566597+0000","last_fullsized":"2023-11-28T18:24:56.566597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:21.578564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.733307+0000","last_change":"2023-11-28T18:24:56.577947+0000","last_active":"2023-11-28T18:24:57.733307+0000","last_peered":"2023-11-28T18:24:57.733307+0000","last_clean":"2023-11-28T18:24:57.733307+0000","last_became_active":"2023-11-28T18:24:56.577624+0000","last_became_peered":"2023-11-28T18:24:56.577624+0000","last_unstale":"2023-11-28T18:24:57.733307+0000","last_undegraded":"2023-11-28T18:24:57.733307+0000","last_fullsized":"2023-11-28T18:24:57.733307+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:17:43.775976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.602715+0000","last_change":"2023-11-28T18:24:55.579772+0000","last_active":"2023-11-28T18:24:56.602715+0000","last_peered":"2023-11-28T18:24:56.602715+0000","last_clean":"2023-11-28T18:24:56.602715+0000","last_became_active":"2023-11-28T18:24:55.579492+0000","last_became_peered":"2023-11-28T18:24:55.579492+0000","last_unstale":"2023-11-28T18:24:56.602715+0000","last_undegraded":"2023-11-28T18:24:56.602715+0000","last_fullsized":"2023-11-28T18:24:56.602715+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:17:42.767826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.557875+0000","last_change":"2023-11-28T18:24:56.561515+0000","last_active":"2023-11-28T18:24:57.557875+0000","last_peered":"2023-11-28T18:24:57.557875+0000","last_clean":"2023-11-28T18:24:57.557875+0000","last_became_active":"2023-11-28T18:24:56.561160+0000","last_became_peered":"2023-11-28T18:24:56.561160+0000","last_unstale":"2023-11-28T18:24:57.557875+0000","last_undegraded":"2023-11-28T18:24:57.557875+0000","last_fullsized":"2023-11-28T18:24:57.557875+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:27:47.996939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.605683+0000","last_change":"2023-11-28T18:24:55.576648+0000","last_active":"2023-11-28T18:24:56.605683+0000","last_peered":"2023-11-28T18:24:56.605683+0000","last_clean":"2023-11-28T18:24:56.605683+0000","last_became_active":"2023-11-28T18:24:55.576382+0000","last_became_peered":"2023-11-28T18:24:55.576382+0000","last_unstale":"2023-11-28T18:24:56.605683+0000","last_undegraded":"2023-11-28T18:24:56.605683+0000","last_fullsized":"2023-11-28T18:24:56.605683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:30:06.534164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.732710+0000","last_change":"2023-11-28T18:24:56.575980+0000","last_active":"2023-11-28T18:24:57.732710+0000","last_peered":"2023-11-28T18:24:57.732710+0000","last_clean":"2023-11-28T18:24:57.732710+0000","last_became_active":"2023-11-28T18:24:56.575251+0000","last_became_peered":"2023-11-28T18:24:56.575251+0000","last_unstale":"2023-11-28T18:24:57.732710+0000","last_undegraded":"2023-11-28T18:24:57.732710+0000","last_fullsized":"2023-11-28T18:24:57.732710+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:30.603278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.600578+0000","last_change":"2023-11-28T18:24:55.565583+0000","last_active":"2023-11-28T18:24:56.600578+0000","last_peered":"2023-11-28T18:24:56.600578+0000","last_clean":"2023-11-28T18:24:56.600578+0000","last_became_active":"2023-11-28T18:24:55.565335+0000","last_became_peered":"2023-11-28T18:24:55.565335+0000","last_unstale":"2023-11-28T18:24:56.600578+0000","last_undegraded":"2023-11-28T18:24:56.600578+0000","last_fullsized":"2023-11-28T18:24:56.600578+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:27:46.988790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.555596+0000","last_change":"2023-11-28T18:24:56.578241+0000","last_active":"2023-11-28T18:24:57.555596+0000","last_peered":"2023-11-28T18:24:57.555596+0000","last_clean":"2023-11-28T18:24:57.555596+0000","last_became_active":"2023-11-28T18:24:56.577937+0000","last_became_peered":"2023-11-28T18:24:56.577937+0000","last_unstale":"2023-11-28T18:24:57.555596+0000","last_undegraded":"2023-11-28T18:24:57.555596+0000","last_fullsized":"2023-11-28T18:24:57.555596+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:51.562902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.568107+0000","last_change":"2023-11-28T18:24:55.575066+0000","last_active":"2023-11-28T18:24:56.568107+0000","last_peered":"2023-11-28T18:24:56.568107+0000","last_clean":"2023-11-28T18:24:56.568107+0000","last_became_active":"2023-11-28T18:24:55.574770+0000","last_became_peered":"2023-11-28T18:24:55.574770+0000","last_unstale":"2023-11-28T18:24:56.568107+0000","last_undegraded":"2023-11-28T18:24:56.568107+0000","last_fullsized":"2023-11-28T18:24:56.568107+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:12:52.751601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.732823+0000","last_change":"2023-11-28T18:24:56.582240+0000","last_active":"2023-11-28T18:24:57.732823+0000","last_peered":"2023-11-28T18:24:57.732823+0000","last_clean":"2023-11-28T18:24:57.732823+0000","last_became_active":"2023-11-28T18:24:56.581977+0000","last_became_peered":"2023-11-28T18:24:56.581977+0000","last_unstale":"2023-11-28T18:24:57.732823+0000","last_undegraded":"2023-11-28T18:24:57.732823+0000","last_fullsized":"2023-11-28T18:24:57.732823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:24:42.258629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.569683+0000","last_change":"2023-11-28T18:24:55.561985+0000","last_active":"2023-11-28T18:24:56.569683+0000","last_peered":"2023-11-28T18:24:56.569683+0000","last_clean":"2023-11-28T18:24:56.569683+0000","last_became_active":"2023-11-28T18:24:55.561673+0000","last_became_peered":"2023-11-28T18:24:55.561673+0000","last_unstale":"2023-11-28T18:24:56.569683+0000","last_undegraded":"2023-11-28T18:24:56.569683+0000","last_fullsized":"2023-11-28T18:24:56.569683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:17:08.512763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.555745+0000","last_change":"2023-11-28T18:24:56.578302+0000","last_active":"2023-11-28T18:24:57.555745+0000","last_peered":"2023-11-28T18:24:57.555745+0000","last_clean":"2023-11-28T18:24:57.555745+0000","last_became_active":"2023-11-28T18:24:56.576903+0000","last_became_peered":"2023-11-28T18:24:56.576903+0000","last_unstale":"2023-11-28T18:24:57.555745+0000","last_undegraded":"2023-11-28T18:24:57.555745+0000","last_fullsized":"2023-11-28T18:24:57.555745+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:03.821752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.604260+0000","last_change":"2023-11-28T18:24:55.572674+0000","last_active":"2023-11-28T18:24:56.604260+0000","last_peered":"2023-11-28T18:24:56.604260+0000","last_clean":"2023-11-28T18:24:56.604260+0000","last_became_active":"2023-11-28T18:24:55.572209+0000","last_became_peered":"2023-11-28T18:24:55.572209+0000","last_unstale":"2023-11-28T18:24:56.604260+0000","last_undegraded":"2023-11-28T18:24:56.604260+0000","last_fullsized":"2023-11-28T18:24:56.604260+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:29:37.604627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.556028+0000","last_change":"2023-11-28T18:24:56.583225+0000","last_active":"2023-11-28T18:24:57.556028+0000","last_peered":"2023-11-28T18:24:57.556028+0000","last_clean":"2023-11-28T18:24:57.556028+0000","last_became_active":"2023-11-28T18:24:56.582951+0000","last_became_peered":"2023-11-28T18:24:56.582951+0000","last_unstale":"2023-11-28T18:24:57.556028+0000","last_undegraded":"2023-11-28T18:24:57.556028+0000","last_fullsized":"2023-11-28T18:24:57.556028+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:18.177165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.602817+0000","last_change":"2023-11-28T18:24:55.575893+0000","last_active":"2023-11-28T18:24:56.602817+0000","last_peered":"2023-11-28T18:24:56.602817+0000","last_clean":"2023-11-28T18:24:56.602817+0000","last_became_active":"2023-11-28T18:24:55.575425+0000","last_became_peered":"2023-11-28T18:24:55.575425+0000","last_unstale":"2023-11-28T18:24:56.602817+0000","last_undegraded":"2023-11-28T18:24:56.602817+0000","last_fullsized":"2023-11-28T18:24:56.602817+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:45.103908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.733376+0000","last_change":"2023-11-28T18:24:56.577135+0000","last_active":"2023-11-28T18:24:57.733376+0000","last_peered":"2023-11-28T18:24:57.733376+0000","last_clean":"2023-11-28T18:24:57.733376+0000","last_became_active":"2023-11-28T18:24:56.576761+0000","last_became_peered":"2023-11-28T18:24:56.576761+0000","last_unstale":"2023-11-28T18:24:57.733376+0000","last_undegraded":"2023-11-28T18:24:57.733376+0000","last_fullsized":"2023-11-28T18:24:57.733376+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:28:36.081443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.601607+0000","last_change":"2023-11-28T18:24:55.579138+0000","last_active":"2023-11-28T18:24:56.601607+0000","last_peered":"2023-11-28T18:24:56.601607+0000","last_clean":"2023-11-28T18:24:56.601607+0000","last_became_active":"2023-11-28T18:24:55.578827+0000","last_became_peered":"2023-11-28T18:24:55.578827+0000","last_unstale":"2023-11-28T18:24:56.601607+0000","last_undegraded":"2023-11-28T18:24:56.601607+0000","last_fullsized":"2023-11-28T18:24:56.601607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:14.095135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.557540+0000","last_change":"2023-11-28T18:24:56.576103+0000","last_active":"2023-11-28T18:24:57.557540+0000","last_peered":"2023-11-28T18:24:57.557540+0000","last_clean":"2023-11-28T18:24:57.557540+0000","last_became_active":"2023-11-28T18:24:56.575701+0000","last_became_peered":"2023-11-28T18:24:56.575701+0000","last_unstale":"2023-11-28T18:24:57.557540+0000","last_undegraded":"2023-11-28T18:24:57.557540+0000","last_fullsized":"2023-11-28T18:24:57.557540+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:34:24.042019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.603593+0000","last_change":"2023-11-28T18:24:55.578515+0000","last_active":"2023-11-28T18:24:56.603593+0000","last_peered":"2023-11-28T18:24:56.603593+0000","last_clean":"2023-11-28T18:24:56.603593+0000","last_became_active":"2023-11-28T18:24:55.578256+0000","last_became_peered":"2023-11-28T18:24:55.578256+0000","last_unstale":"2023-11-28T18:24:56.603593+0000","last_undegraded":"2023-11-28T18:24:56.603593+0000","last_fullsized":"2023-11-28T18:24:56.603593+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:06.899740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.733546+0000","last_change":"2023-11-28T18:24:56.571164+0000","last_active":"2023-11-28T18:24:57.733546+0000","last_peered":"2023-11-28T18:24:57.733546+0000","last_clean":"2023-11-28T18:24:57.733546+0000","last_became_active":"2023-11-28T18:24:56.570881+0000","last_became_peered":"2023-11-28T18:24:56.570881+0000","last_unstale":"2023-11-28T18:24:57.733546+0000","last_undegraded":"2023-11-28T18:24:57.733546+0000","last_fullsized":"2023-11-28T18:24:57.733546+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:47:10.722595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.565689+0000","last_change":"2023-11-28T18:24:55.575784+0000","last_active":"2023-11-28T18:24:56.565689+0000","last_peered":"2023-11-28T18:24:56.565689+0000","last_clean":"2023-11-28T18:24:56.565689+0000","last_became_active":"2023-11-28T18:24:55.575518+0000","last_became_peered":"2023-11-28T18:24:55.575518+0000","last_unstale":"2023-11-28T18:24:56.565689+0000","last_undegraded":"2023-11-28T18:24:56.565689+0000","last_fullsized":"2023-11-28T18:24:56.565689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:14.095135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.732783+0000","last_change":"2023-11-28T18:24:56.590615+0000","last_active":"2023-11-28T18:24:57.732783+0000","last_peered":"2023-11-28T18:24:57.732783+0000","last_clean":"2023-11-28T18:24:57.732783+0000","last_became_active":"2023-11-28T18:24:56.590299+0000","last_became_peered":"2023-11-28T18:24:56.590299+0000","last_unstale":"2023-11-28T18:24:57.732783+0000","last_undegraded":"2023-11-28T18:24:57.732783+0000","last_fullsized":"2023-11-28T18:24:57.732783+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:32:57.901739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.600474+0000","last_change":"2023-11-28T18:24:55.578008+0000","last_active":"2023-11-28T18:24:56.600474+0000","last_peered":"2023-11-28T18:24:56.600474+0000","last_clean":"2023-11-28T18:24:56.600474+0000","last_became_active":"2023-11-28T18:24:55.577782+0000","last_became_peered":"2023-11-28T18:24:55.577782+0000","last_unstale":"2023-11-28T18:24:56.600474+0000","last_undegraded":"2023-11-28T18:24:56.600474+0000","last_fullsized":"2023-11-28T18:24:56.600474+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:55:08.887224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.732816+0000","last_change":"2023-11-28T18:24:56.592890+0000","last_active":"2023-11-28T18:24:57.732816+0000","last_peered":"2023-11-28T18:24:57.732816+0000","last_clean":"2023-11-28T18:24:57.732816+0000","last_became_active":"2023-11-28T18:24:56.591816+0000","last_became_peered":"2023-11-28T18:24:56.591816+0000","last_unstale":"2023-11-28T18:24:57.732816+0000","last_undegraded":"2023-11-28T18:24:57.732816+0000","last_fullsized":"2023-11-28T18:24:57.732816+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:20:20.062731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.555667+0000","last_change":"2023-11-28T18:24:55.576784+0000","last_active":"2023-11-28T18:24:56.555667+0000","last_peered":"2023-11-28T18:24:56.555667+0000","last_clean":"2023-11-28T18:24:56.555667+0000","last_became_active":"2023-11-28T18:24:55.576508+0000","last_became_peered":"2023-11-28T18:24:55.576508+0000","last_unstale":"2023-11-28T18:24:56.555667+0000","last_undegraded":"2023-11-28T18:24:56.555667+0000","last_fullsized":"2023-11-28T18:24:56.555667+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:38:20.581137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.732870+0000","last_change":"2023-11-28T18:24:56.581593+0000","last_active":"2023-11-28T18:24:57.732870+0000","last_peered":"2023-11-28T18:24:57.732870+0000","last_clean":"2023-11-28T18:24:57.732870+0000","last_became_active":"2023-11-28T18:24:56.581304+0000","last_became_peered":"2023-11-28T18:24:56.581304+0000","last_unstale":"2023-11-28T18:24:57.732870+0000","last_undegraded":"2023-11-28T18:24:57.732870+0000","last_fullsized":"2023-11-28T18:24:57.732870+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:07:54.626598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.570120+0000","last_change":"2023-11-28T18:24:55.582487+0000","last_active":"2023-11-28T18:24:56.570120+0000","last_peered":"2023-11-28T18:24:56.570120+0000","last_clean":"2023-11-28T18:24:56.570120+0000","last_became_active":"2023-11-28T18:24:55.582038+0000","last_became_peered":"2023-11-28T18:24:55.582038+0000","last_unstale":"2023-11-28T18:24:56.570120+0000","last_undegraded":"2023-11-28T18:24:56.570120+0000","last_fullsized":"2023-11-28T18:24:56.570120+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:55:57.153924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.555877+0000","last_change":"2023-11-28T18:24:56.578512+0000","last_active":"2023-11-28T18:24:57.555877+0000","last_peered":"2023-11-28T18:24:57.555877+0000","last_clean":"2023-11-28T18:24:57.555877+0000","last_became_active":"2023-11-28T18:24:56.577371+0000","last_became_peered":"2023-11-28T18:24:56.577371+0000","last_unstale":"2023-11-28T18:24:57.555877+0000","last_undegraded":"2023-11-28T18:24:57.555877+0000","last_fullsized":"2023-11-28T18:24:57.555877+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:32:57.901739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.556076+0000","last_change":"2023-11-28T18:24:55.577155+0000","last_active":"2023-11-28T18:24:56.556076+0000","last_peered":"2023-11-28T18:24:56.556076+0000","last_clean":"2023-11-28T18:24:56.556076+0000","last_became_active":"2023-11-28T18:24:55.576899+0000","last_became_peered":"2023-11-28T18:24:55.576899+0000","last_unstale":"2023-11-28T18:24:56.556076+0000","last_undegraded":"2023-11-28T18:24:56.556076+0000","last_fullsized":"2023-11-28T18:24:56.556076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:06.790640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.732325+0000","last_change":"2023-11-28T18:24:56.581263+0000","last_active":"2023-11-28T18:24:57.732325+0000","last_peered":"2023-11-28T18:24:57.732325+0000","last_clean":"2023-11-28T18:24:57.732325+0000","last_became_active":"2023-11-28T18:24:56.580967+0000","last_became_peered":"2023-11-28T18:24:56.580967+0000","last_unstale":"2023-11-28T18:24:57.732325+0000","last_undegraded":"2023-11-28T18:24:57.732325+0000","last_fullsized":"2023-11-28T18:24:57.732325+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:49:50.311251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.602849+0000","last_change":"2023-11-28T18:24:55.579931+0000","last_active":"2023-11-28T18:24:56.602849+0000","last_peered":"2023-11-28T18:24:56.602849+0000","last_clean":"2023-11-28T18:24:56.602849+0000","last_became_active":"2023-11-28T18:24:55.579568+0000","last_became_peered":"2023-11-28T18:24:55.579568+0000","last_unstale":"2023-11-28T18:24:56.602849+0000","last_undegraded":"2023-11-28T18:24:56.602849+0000","last_fullsized":"2023-11-28T18:24:56.602849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:50.554752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.733220+0000","last_change":"2023-11-28T18:24:56.599970+0000","last_active":"2023-11-28T18:24:57.733220+0000","last_peered":"2023-11-28T18:24:57.733220+0000","last_clean":"2023-11-28T18:24:57.733220+0000","last_became_active":"2023-11-28T18:24:56.599691+0000","last_became_peered":"2023-11-28T18:24:56.599691+0000","last_unstale":"2023-11-28T18:24:57.733220+0000","last_undegraded":"2023-11-28T18:24:57.733220+0000","last_fullsized":"2023-11-28T18:24:57.733220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:22.586713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.733295+0000","last_change":"2023-11-28T18:24:55.580960+0000","last_active":"2023-11-28T18:24:57.733295+0000","last_peered":"2023-11-28T18:24:57.733295+0000","last_clean":"2023-11-28T18:24:57.733295+0000","last_became_active":"2023-11-28T18:24:55.580717+0000","last_became_peered":"2023-11-28T18:24:55.580717+0000","last_unstale":"2023-11-28T18:24:57.733295+0000","last_undegraded":"2023-11-28T18:24:57.733295+0000","last_fullsized":"2023-11-28T18:24:57.733295+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:15:29.396986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.555811+0000","last_change":"2023-11-28T18:24:56.566334+0000","last_active":"2023-11-28T18:24:57.555811+0000","last_peered":"2023-11-28T18:24:57.555811+0000","last_clean":"2023-11-28T18:24:57.555811+0000","last_became_active":"2023-11-28T18:24:56.566016+0000","last_became_peered":"2023-11-28T18:24:56.566016+0000","last_unstale":"2023-11-28T18:24:57.555811+0000","last_undegraded":"2023-11-28T18:24:57.555811+0000","last_fullsized":"2023-11-28T18:24:57.555811+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:07:54.626598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.565675+0000","last_change":"2023-11-28T18:24:55.569035+0000","last_active":"2023-11-28T18:24:56.565675+0000","last_peered":"2023-11-28T18:24:56.565675+0000","last_clean":"2023-11-28T18:24:56.565675+0000","last_became_active":"2023-11-28T18:24:55.568744+0000","last_became_peered":"2023-11-28T18:24:55.568744+0000","last_unstale":"2023-11-28T18:24:56.565675+0000","last_undegraded":"2023-11-28T18:24:56.565675+0000","last_fullsized":"2023-11-28T18:24:56.565675+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:32:10.490162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.732838+0000","last_change":"2023-11-28T18:24:56.580375+0000","last_active":"2023-11-28T18:24:57.732838+0000","last_peered":"2023-11-28T18:24:57.732838+0000","last_clean":"2023-11-28T18:24:57.732838+0000","last_became_active":"2023-11-28T18:24:56.580151+0000","last_became_peered":"2023-11-28T18:24:56.580151+0000","last_unstale":"2023-11-28T18:24:57.732838+0000","last_undegraded":"2023-11-28T18:24:57.732838+0000","last_fullsized":"2023-11-28T18:24:57.732838+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:04:47.721518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.566188+0000","last_change":"2023-11-28T18:24:55.565390+0000","last_active":"2023-11-28T18:24:56.566188+0000","last_peered":"2023-11-28T18:24:56.566188+0000","last_clean":"2023-11-28T18:24:56.566188+0000","last_became_active":"2023-11-28T18:24:55.565112+0000","last_became_peered":"2023-11-28T18:24:55.565112+0000","last_unstale":"2023-11-28T18:24:56.566188+0000","last_undegraded":"2023-11-28T18:24:56.566188+0000","last_fullsized":"2023-11-28T18:24:56.566188+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:49:49.303101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.732251+0000","last_change":"2023-11-28T18:24:56.584103+0000","last_active":"2023-11-28T18:24:57.732251+0000","last_peered":"2023-11-28T18:24:57.732251+0000","last_clean":"2023-11-28T18:24:57.732251+0000","last_became_active":"2023-11-28T18:24:56.583828+0000","last_became_peered":"2023-11-28T18:24:56.583828+0000","last_unstale":"2023-11-28T18:24:57.732251+0000","last_undegraded":"2023-11-28T18:24:57.732251+0000","last_fullsized":"2023-11-28T18:24:57.732251+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:32:11.498311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.571341+0000","last_change":"2023-11-28T18:24:55.567103+0000","last_active":"2023-11-28T18:24:56.571341+0000","last_peered":"2023-11-28T18:24:56.571341+0000","last_clean":"2023-11-28T18:24:56.571341+0000","last_became_active":"2023-11-28T18:24:55.566687+0000","last_became_peered":"2023-11-28T18:24:55.566687+0000","last_unstale":"2023-11-28T18:24:56.571341+0000","last_undegraded":"2023-11-28T18:24:56.571341+0000","last_fullsized":"2023-11-28T18:24:56.571341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:50:55.081198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.557048+0000","last_change":"2023-11-28T18:24:56.570370+0000","last_active":"2023-11-28T18:24:57.557048+0000","last_peered":"2023-11-28T18:24:57.557048+0000","last_clean":"2023-11-28T18:24:57.557048+0000","last_became_active":"2023-11-28T18:24:56.569166+0000","last_became_peered":"2023-11-28T18:24:56.569166+0000","last_unstale":"2023-11-28T18:24:57.557048+0000","last_undegraded":"2023-11-28T18:24:57.557048+0000","last_fullsized":"2023-11-28T18:24:57.557048+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:55:58.162074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.556211+0000","last_change":"2023-11-28T18:24:55.572730+0000","last_active":"2023-11-28T18:24:56.556211+0000","last_peered":"2023-11-28T18:24:56.556211+0000","last_clean":"2023-11-28T18:24:56.556211+0000","last_became_active":"2023-11-28T18:24:55.572440+0000","last_became_peered":"2023-11-28T18:24:55.572440+0000","last_unstale":"2023-11-28T18:24:56.556211+0000","last_undegraded":"2023-11-28T18:24:56.556211+0000","last_fullsized":"2023-11-28T18:24:56.556211+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:29:45.071843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.733322+0000","last_change":"2023-11-28T18:24:56.584121+0000","last_active":"2023-11-28T18:24:57.733322+0000","last_peered":"2023-11-28T18:24:57.733322+0000","last_clean":"2023-11-28T18:24:57.733322+0000","last_became_active":"2023-11-28T18:24:56.583885+0000","last_became_peered":"2023-11-28T18:24:56.583885+0000","last_unstale":"2023-11-28T18:24:57.733322+0000","last_undegraded":"2023-11-28T18:24:57.733322+0000","last_fullsized":"2023-11-28T18:24:57.733322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:17:43.775976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.556664+0000","last_change":"2023-11-28T18:24:55.564212+0000","last_active":"2023-11-28T18:24:56.556664+0000","last_peered":"2023-11-28T18:24:56.556664+0000","last_clean":"2023-11-28T18:24:56.556664+0000","last_became_active":"2023-11-28T18:24:55.561715+0000","last_became_peered":"2023-11-28T18:24:55.561715+0000","last_unstale":"2023-11-28T18:24:56.556664+0000","last_undegraded":"2023-11-28T18:24:56.556664+0000","last_fullsized":"2023-11-28T18:24:56.556664+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:32:21.053234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T18:24:57.733526+0000","last_change":"2023-11-28T18:24:56.585609+0000","last_active":"2023-11-28T18:24:57.733526+0000","last_peered":"2023-11-28T18:24:57.733526+0000","last_clean":"2023-11-28T18:24:57.733526+0000","last_became_active":"2023-11-28T18:24:56.585329+0000","last_became_peered":"2023-11-28T18:24:56.585329+0000","last_unstale":"2023-11-28T18:24:57.733526+0000","last_undegraded":"2023-11-28T18:24:57.733526+0000","last_fullsized":"2023-11-28T18:24:57.733526+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:55.539940+0000","last_clean_scrub_stamp":"2023-11-28T18:24:55.539940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:03.388022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.571170+0000","last_change":"2023-11-28T18:24:55.582776+0000","last_active":"2023-11-28T18:24:56.571170+0000","last_peered":"2023-11-28T18:24:56.571170+0000","last_clean":"2023-11-28T18:24:56.571170+0000","last_became_active":"2023-11-28T18:24:55.582457+0000","last_became_peered":"2023-11-28T18:24:55.582457+0000","last_unstale":"2023-11-28T18:24:56.571170+0000","last_undegraded":"2023-11-28T18:24:56.571170+0000","last_fullsized":"2023-11-28T18:24:56.571170+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:24:54.531791+0000","last_clean_scrub_stamp":"2023-11-28T18:24:54.531791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:30:06.534164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":180,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T18:24:56.567009+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:24:56.567009+0000","last_peered":"2023-11-28T18:24:56.567009+0000","last_clean":"2023-11-28T18:24:56.567009+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:24:56.567009+0000","last_undegraded":"2023-11-28T18:24:56.567009+0000","last_fullsized":"2023-11-28T18:24:56.567009+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":780866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.64600000000000002}]},{"osd":1,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.58499999999999996}]},{"osd":2,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.411,"5min":2.411,"15min":2.411},"last":0.53300000000000003}]},{"osd":3,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.628},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.3650000000000002,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":0.78900000000000003}]},{"osd":4,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.70399999999999996}]},{"osd":5,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.60699999999999998}]},{"osd":6,"last update":"Tue Nov 28 18:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.77500000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":189398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.79800000000000004}]},{"osd":1,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":4.2389999999999999,"5min":4.2389999999999999,"15min":4.2389999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":4.218,"5min":4.218,"15min":4.218},"last":0.748}]},{"osd":2,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.76300000000000001}]},{"osd":3,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.69199999999999995}]},{"osd":4,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63500000000000001}]},{"osd":5,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.73299999999999998}]},{"osd":7,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.57099999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":446464,"data_stored":188928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.8929999999999998,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":0.623},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.9140000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":0.54700000000000004}]},{"osd":1,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.4809999999999999,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.56699999999999995}]},{"osd":2,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":3.9510000000000001,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":4.0149999999999997,"5min":4.0149999999999997,"15min":4.0149999999999997},"last":0.73699999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":4.0869999999999997,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.9900000000000002,"5min":3.9900000000000002,"15min":3.9900000000000002},"last":0.71099999999999997}]},{"osd":4,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.78800000000000003}]},{"osd":6,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.77200000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.81299999999999994}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":446464,"data_stored":189420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":2.363,"5min":2.363,"15min":2.363},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.67600000000000005}]},{"osd":1,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.85199999999999998}]},{"osd":2,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":0.81699999999999995}]},{"osd":3,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.83199999999999996}]},{"osd":5,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.46500000000000002}]},{"osd":6,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.57899999999999996}]},{"osd":7,"last update":"Tue Nov 28 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.70899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":190462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.44900000000000001}]},{"osd":1,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.42899999999999999}]},{"osd":2,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.46500000000000002}]},{"osd":4,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.88400000000000001}]},{"osd":5,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.5019999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.872},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.79300000000000004}]},{"osd":6,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.77700000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:24:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.746}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":188886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.48599999999999999}]},{"osd":1,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.52800000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.69599999999999995}]},{"osd":4,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.65900000000000003}]},{"osd":5,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":7.4820000000000002,"5min":7.4820000000000002,"15min":7.4820000000000002},"last":0.82799999999999996}]},{"osd":6,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":3.9380000000000002,"5min":3.9380000000000002,"15min":3.9380000000000002},"last":0.751}]},{"osd":7,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":4.798,"5min":4.798,"15min":4.798},"last":0.67600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":188906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.37}]},{"osd":2,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.62}]},{"osd":3,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.57899999999999996}]},{"osd":4,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.59499999999999997}]},{"osd":5,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.60999999999999999}]},{"osd":6,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.67100000000000004}]},{"osd":7,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.53500000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":779218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.45600000000000002}]},{"osd":2,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.503}]},{"osd":3,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.58199999999999996}]},{"osd":4,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.82599999999999996}]},{"osd":5,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.85899999999999999}]},{"osd":6,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.83899999999999997}]},{"osd":7,"last update":"Tue Nov 28 18:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.872},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.98099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:25:05.516 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T18:25:05.516 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-28T18:25:05.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.694+0000 7feaa8879640 1 Processor -- start 2023-11-28T18:25:05.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.698+0000 7feaa8879640 1 -- start start 2023-11-28T18:25:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.698+0000 7feaa8879640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feaa4071960 0x7feaa415f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.698+0000 7feaa8879640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7feaa415b030 0x7feaa40709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.698+0000 7feaa8879640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaa40701d0 con 0x7feaa40b4290 2023-11-28T18:25:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.698+0000 7feaa8879640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaa406df80 con 0x7feaa415b030 2023-11-28T18:25:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.698+0000 7feaa8879640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaa4071df0 con 0x7feaa4071960 2023-11-28T18:25:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.698+0000 7feaa2d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7feaa415b030 0x7feaa40709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.698+0000 7feaa2d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7feaa415b030 0x7feaa40709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53870/0 (socket says 172.21.15.105:53870) 2023-11-28T18:25:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.698+0000 7feaa2d9d640 1 -- 172.21.15.105:0/1294456870 learned_addr learned my addr 172.21.15.105:0/1294456870 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.698+0000 7feaa259c640 1 --1- 172.21.15.105:0/1294456870 >> v1:172.21.15.105:6789/0 conn(0x7feaa40b4290 0x7feaa40b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39378/0 (socket says 172.21.15.105:39378) 2023-11-28T18:25:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.698+0000 7feaa1d9b640 1 --2- 172.21.15.105:0/1294456870 >> v2:172.21.15.148:3300/0 conn(0x7feaa4071960 0x7feaa415f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:05.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.698+0000 7feaa2d9d640 1 -- 172.21.15.105:0/1294456870 >> v2:172.21.15.148:3300/0 conn(0x7feaa4071960 msgr2=0x7feaa415f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:05.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.698+0000 7feaa2d9d640 1 --2- 172.21.15.105:0/1294456870 >> v2:172.21.15.148:3300/0 conn(0x7feaa4071960 0x7feaa415f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:05.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.698+0000 7feaa2d9d640 1 -- 172.21.15.105:0/1294456870 >> v1:172.21.15.105:6789/0 conn(0x7feaa40b4290 legacy=0x7feaa40b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:05.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.698+0000 7feaa2d9d640 1 -- 172.21.15.105:0/1294456870 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa40720d0 con 0x7feaa415b030 2023-11-28T18:25:05.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.698+0000 7feaa159a640 1 -- 172.21.15.105:0/1294456870 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4047437663 0 0) 0x7feaa40701d0 con 0x7feaa40b4290 2023-11-28T18:25:05.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.698+0000 7feaa1d9b640 1 --2- 172.21.15.105:0/1294456870 >> v2:172.21.15.148:3300/0 conn(0x7feaa4071960 0x7feaa415f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:05.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.702+0000 7feaa2d9d640 1 --2- 172.21.15.105:0/1294456870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaa415b030 0x7feaa40709b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7feaa406e520 tx=0x7fea9c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9277bc7f9d7891 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:05.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.702+0000 7feaa159a640 1 -- 172.21.15.105:0/1294456870 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c0a5770 con 0x7feaa415b030 2023-11-28T18:25:05.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.702+0000 7feaa159a640 1 -- 172.21.15.105:0/1294456870 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9c0a58d0 con 0x7feaa415b030 2023-11-28T18:25:05.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.702+0000 7feaa159a640 1 -- 172.21.15.105:0/1294456870 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c0a5c60 con 0x7feaa415b030 2023-11-28T18:25:05.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.702+0000 7feaa8879640 1 -- 172.21.15.105:0/1294456870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaa415b030 msgr2=0x7feaa40709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:05.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.702+0000 7feaa8879640 1 --2- 172.21.15.105:0/1294456870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaa415b030 0x7feaa40709b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7feaa406e520 tx=0x7fea9c01c9a0 comp rx=0 tx=0).stop 2023-11-28T18:25:05.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.702+0000 7feaa8879640 1 -- 172.21.15.105:0/1294456870 shutdown_connections 2023-11-28T18:25:05.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.702+0000 7feaa8879640 1 --2- 172.21.15.105:0/1294456870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaa415b030 0x7feaa40709b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:05.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.702+0000 7feaa8879640 1 --2- 172.21.15.105:0/1294456870 >> v2:172.21.15.148:3300/0 conn(0x7feaa4071960 0x7feaa415f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:05.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.702+0000 7feaa8879640 1 -- 172.21.15.105:0/1294456870 >> 172.21.15.105:0/1294456870 conn(0x7feaa40ceb90 msgr2=0x7feaa40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:05.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.702+0000 7feaa8879640 1 -- 172.21.15.105:0/1294456870 shutdown_connections 2023-11-28T18:25:05.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.702+0000 7feaa8879640 1 -- 172.21.15.105:0/1294456870 wait complete. 2023-11-28T18:25:05.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.702+0000 7feaa8879640 1 Processor -- start 2023-11-28T18:25:05.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa8879640 1 -- start start 2023-11-28T18:25:05.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa8879640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaa40b4290 0x7feaa41f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:05.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa8879640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaa415b030 0x7feaa41f6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:05.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa8879640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feaa40b3dd0 0x7feaa41f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:05.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa8879640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa414ef50 con 0x7feaa415b030 2023-11-28T18:25:05.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa8879640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa41f6c80 con 0x7feaa40b4290 2023-11-28T18:25:05.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa8879640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa4099bd0 con 0x7feaa40b3dd0 2023-11-28T18:25:05.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaa415b030 0x7feaa41f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaa415b030 0x7feaa41f6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53874/0 (socket says 172.21.15.105:53874) 2023-11-28T18:25:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa1d9b640 1 -- 172.21.15.105:0/539944168 learned_addr learned my addr 172.21.15.105:0/539944168 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa259c640 1 --2- 172.21.15.105:0/539944168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaa40b4290 0x7feaa41f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa2d9d640 1 --2- 172.21.15.105:0/539944168 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feaa40b3dd0 0x7feaa41f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa1d9b640 1 -- 172.21.15.105:0/539944168 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feaa40b3dd0 msgr2=0x7feaa41f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa1d9b640 1 --2- 172.21.15.105:0/539944168 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feaa40b3dd0 0x7feaa41f56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa1d9b640 1 -- 172.21.15.105:0/539944168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaa40b4290 msgr2=0x7feaa41f9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa1d9b640 1 --2- 172.21.15.105:0/539944168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaa40b4290 0x7feaa41f9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa1d9b640 1 -- 172.21.15.105:0/539944168 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9c009b30 con 0x7feaa415b030 2023-11-28T18:25:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa259c640 1 --2- 172.21.15.105:0/539944168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaa40b4290 0x7feaa41f9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa2d9d640 1 --2- 172.21.15.105:0/539944168 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feaa40b3dd0 0x7feaa41f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa1d9b640 1 --2- 172.21.15.105:0/539944168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaa415b030 0x7feaa41f6740 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fea9800c7c0 tx=0x7fea9800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:05.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7fea92ffd640 1 -- 172.21.15.105:0/539944168 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea98007a30 con 0x7feaa415b030 2023-11-28T18:25:05.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7fea92ffd640 1 -- 172.21.15.105:0/539944168 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea98007b90 con 0x7feaa415b030 2023-11-28T18:25:05.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7fea92ffd640 1 -- 172.21.15.105:0/539944168 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9800ce70 con 0x7feaa415b030 2023-11-28T18:25:05.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa8879640 1 -- 172.21.15.105:0/539944168 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa4099eb0 con 0x7feaa415b030 2023-11-28T18:25:05.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa8879640 1 -- 172.21.15.105:0/539944168 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa41f5f40 con 0x7feaa415b030 2023-11-28T18:25:05.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.706+0000 7feaa8879640 1 -- 172.21.15.105:0/539944168 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa4067570 con 0x7feaa415b030 2023-11-28T18:25:05.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.710+0000 7fea92ffd640 1 -- 172.21.15.105:0/539944168 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fea980113f0 con 0x7feaa415b030 2023-11-28T18:25:05.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.710+0000 7fea92ffd640 1 --2- 172.21.15.105:0/539944168 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fea84066900 0x7fea84068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:05.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.710+0000 7fea92ffd640 1 -- 172.21.15.105:0/539944168 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fea98084ae0 con 0x7feaa415b030 2023-11-28T18:25:05.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.710+0000 7feaa259c640 1 --2- 172.21.15.105:0/539944168 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fea84066900 0x7fea84068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:05.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.710+0000 7feaa259c640 1 --2- 172.21.15.105:0/539944168 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fea84066900 0x7fea84068dc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fea94000f70 tx=0x7fea94007c40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:05.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.714+0000 7fea92ffd640 1 -- 172.21.15.105:0/539944168 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fea98051b40 con 0x7feaa415b030 2023-11-28T18:25:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.917+0000 7feaa8879640 1 -- 172.21.15.105:0/539944168 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7feaa40cf7f0 con 0x7feaa415b030 2023-11-28T18:25:05.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.917+0000 7fea92ffd640 1 -- 172.21.15.105:0/539944168 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7fea98055160 con 0x7feaa415b030 2023-11-28T18:25:05.925 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2023-11-28T18:25:05.925 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDpL2ZlSt0VMxAAQ8+j1BgBPMJ2cuiZj8aG0w== 2023-11-28T18:25:05.925 INFO:teuthology.orchestra.run.smithi105.stdout: caps mds = "allow" 2023-11-28T18:25:05.925 INFO:teuthology.orchestra.run.smithi105.stdout: caps mon = "allow r" 2023-11-28T18:25:05.925 INFO:teuthology.orchestra.run.smithi105.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-28T18:25:05.926 INFO:teuthology.orchestra.run.smithi105.stderr:updated caps for client.0 2023-11-28T18:25:05.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.921+0000 7feaa8879640 1 -- 172.21.15.105:0/539944168 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fea84066900 msgr2=0x7fea84068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:05.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.921+0000 7feaa8879640 1 --2- 172.21.15.105:0/539944168 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fea84066900 0x7fea84068dc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fea94000f70 tx=0x7fea94007c40 comp rx=0 tx=0).stop 2023-11-28T18:25:05.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.921+0000 7feaa8879640 1 -- 172.21.15.105:0/539944168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaa415b030 msgr2=0x7feaa41f6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:05.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.921+0000 7feaa8879640 1 --2- 172.21.15.105:0/539944168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaa415b030 0x7feaa41f6740 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fea9800c7c0 tx=0x7fea9800cc80 comp rx=0 tx=0).stop 2023-11-28T18:25:05.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.921+0000 7feaa8879640 1 -- 172.21.15.105:0/539944168 shutdown_connections 2023-11-28T18:25:05.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.921+0000 7feaa8879640 1 --2- 172.21.15.105:0/539944168 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feaa40b3dd0 0x7feaa41f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:05.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.921+0000 7feaa8879640 1 --2- 172.21.15.105:0/539944168 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fea84066900 0x7fea84068dc0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:05.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.921+0000 7feaa8879640 1 --2- 172.21.15.105:0/539944168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaa415b030 0x7feaa41f6740 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:05.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.921+0000 7feaa8879640 1 --2- 172.21.15.105:0/539944168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaa40b4290 0x7feaa41f9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:05.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.921+0000 7feaa8879640 1 -- 172.21.15.105:0/539944168 >> 172.21.15.105:0/539944168 conn(0x7feaa40ceb90 msgr2=0x7feaa415f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:05.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.921+0000 7feaa8879640 1 -- 172.21.15.105:0/539944168 shutdown_connections 2023-11-28T18:25:05.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:05.921+0000 7feaa8879640 1 -- 172.21.15.105:0/539944168 wait complete. 2023-11-28T18:25:05.947 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:25:06.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.077+0000 7fc363165640 1 Processor -- start 2023-11-28T18:25:06.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.077+0000 7fc363165640 1 -- start start 2023-11-28T18:25:06.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.081+0000 7fc363165640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc35c159f40 0x7fc35c0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.081+0000 7fc363165640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc35c0cc760 0x7fc35c0ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.081+0000 7fc363165640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc35c0ad680 0x7fc35c15da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.081+0000 7fc363165640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc35c06bf50 con 0x7fc35c0ad680 2023-11-28T18:25:06.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.081+0000 7fc363165640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc35c0ccbc0 con 0x7fc35c0cc760 2023-11-28T18:25:06.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.081+0000 7fc363165640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c15a320 con 0x7fc35c159f40 2023-11-28T18:25:06.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.081+0000 7fc361702640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc35c0ad680 0x7fc35c15da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.081+0000 7fc361702640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc35c0ad680 0x7fc35c15da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53880/0 (socket says 172.21.15.105:53880) 2023-11-28T18:25:06.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.081+0000 7fc361702640 1 -- 172.21.15.105:0/412392225 learned_addr learned my addr 172.21.15.105:0/412392225 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:06.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.081+0000 7fc353fff640 1 --2- 172.21.15.105:0/412392225 >> v2:172.21.15.148:3300/0 conn(0x7fc35c0cc760 0x7fc35c0ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.081+0000 7fc360f01640 1 --2- 172.21.15.105:0/412392225 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc35c159f40 0x7fc35c0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.081+0000 7fc361702640 1 -- 172.21.15.105:0/412392225 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc35c159f40 msgr2=0x7fc35c0caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:06.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.081+0000 7fc361702640 1 --2- 172.21.15.105:0/412392225 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc35c159f40 0x7fc35c0caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.081+0000 7fc361702640 1 -- 172.21.15.105:0/412392225 >> v2:172.21.15.148:3300/0 conn(0x7fc35c0cc760 msgr2=0x7fc35c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:06.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.081+0000 7fc361702640 1 --2- 172.21.15.105:0/412392225 >> v2:172.21.15.148:3300/0 conn(0x7fc35c0cc760 0x7fc35c0ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.081+0000 7fc361702640 1 -- 172.21.15.105:0/412392225 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c071380 con 0x7fc35c0ad680 2023-11-28T18:25:06.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.085+0000 7fc361702640 1 --2- 172.21.15.105:0/412392225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc35c0ad680 0x7fc35c15da20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0cfc90 tx=0x7fc35401eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544a506ce4a46391 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:06.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.085+0000 7fc3537fe640 1 -- 172.21.15.105:0/412392225 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35400b2d0 con 0x7fc35c0ad680 2023-11-28T18:25:06.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.085+0000 7fc3537fe640 1 -- 172.21.15.105:0/412392225 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35400b430 con 0x7fc35c0ad680 2023-11-28T18:25:06.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.085+0000 7fc3537fe640 1 -- 172.21.15.105:0/412392225 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3540a7bc0 con 0x7fc35c0ad680 2023-11-28T18:25:06.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.085+0000 7fc363165640 1 -- 172.21.15.105:0/412392225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc35c0ad680 msgr2=0x7fc35c15da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:06.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.085+0000 7fc363165640 1 --2- 172.21.15.105:0/412392225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc35c0ad680 0x7fc35c15da20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0cfc90 tx=0x7fc35401eb10 comp rx=0 tx=0).stop 2023-11-28T18:25:06.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.085+0000 7fc363165640 1 -- 172.21.15.105:0/412392225 shutdown_connections 2023-11-28T18:25:06.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.085+0000 7fc363165640 1 --2- 172.21.15.105:0/412392225 >> v2:172.21.15.148:3300/0 conn(0x7fc35c0cc760 0x7fc35c0ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.085+0000 7fc363165640 1 --2- 172.21.15.105:0/412392225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc35c0ad680 0x7fc35c15da20 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.085+0000 7fc363165640 1 --2- 172.21.15.105:0/412392225 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc35c159f40 0x7fc35c0caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.085+0000 7fc363165640 1 -- 172.21.15.105:0/412392225 >> 172.21.15.105:0/412392225 conn(0x7fc35c0ceb90 msgr2=0x7fc35c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:06.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.085+0000 7fc363165640 1 -- 172.21.15.105:0/412392225 shutdown_connections 2023-11-28T18:25:06.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.085+0000 7fc363165640 1 -- 172.21.15.105:0/412392225 wait complete. 2023-11-28T18:25:06.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.085+0000 7fc363165640 1 Processor -- start 2023-11-28T18:25:06.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.085+0000 7fc363165640 1 -- start start 2023-11-28T18:25:06.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc363165640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc35c0ad680 0x7fc35c150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc363165640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc35c0cc760 0x7fc35c0aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc363165640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc35c159f40 0x7fc35c0700d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc363165640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c15c460 con 0x7fc35c0ad680 2023-11-28T18:25:06.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc363165640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c15c690 con 0x7fc35c159f40 2023-11-28T18:25:06.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc363165640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c15c970 con 0x7fc35c0cc760 2023-11-28T18:25:06.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc360f01640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc35c0ad680 0x7fc35c150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc360f01640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc35c0ad680 0x7fc35c150ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53892/0 (socket says 172.21.15.105:53892) 2023-11-28T18:25:06.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc360f01640 1 -- 172.21.15.105:0/811691348 learned_addr learned my addr 172.21.15.105:0/811691348 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:06.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc353fff640 1 --2- 172.21.15.105:0/811691348 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc35c0cc760 0x7fc35c0aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc361702640 1 --2- 172.21.15.105:0/811691348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc35c159f40 0x7fc35c0700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc360f01640 1 -- 172.21.15.105:0/811691348 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc35c0cc760 msgr2=0x7fc35c0aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:06.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc360f01640 1 --2- 172.21.15.105:0/811691348 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc35c0cc760 0x7fc35c0aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc360f01640 1 -- 172.21.15.105:0/811691348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc35c159f40 msgr2=0x7fc35c0700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:06.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc360f01640 1 --2- 172.21.15.105:0/811691348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc35c159f40 0x7fc35c0700d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc360f01640 1 -- 172.21.15.105:0/811691348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35400bd40 con 0x7fc35c0ad680 2023-11-28T18:25:06.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc361702640 1 --2- 172.21.15.105:0/811691348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc35c159f40 0x7fc35c0700d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:06.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc360f01640 1 --2- 172.21.15.105:0/811691348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc35c0ad680 0x7fc35c150ab0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc35800d5f0 tx=0x7fc35800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:06.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc3517fa640 1 -- 172.21.15.105:0/811691348 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3580125f0 con 0x7fc35c0ad680 2023-11-28T18:25:06.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc3517fa640 1 -- 172.21.15.105:0/811691348 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc358012750 con 0x7fc35c0ad680 2023-11-28T18:25:06.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc3517fa640 1 -- 172.21.15.105:0/811691348 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc358012ae0 con 0x7fc35c0ad680 2023-11-28T18:25:06.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc363165640 1 -- 172.21.15.105:0/811691348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc35c15cc50 con 0x7fc35c0ad680 2023-11-28T18:25:06.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc363165640 1 -- 172.21.15.105:0/811691348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35c15d210 con 0x7fc35c0ad680 2023-11-28T18:25:06.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.089+0000 7fc363165640 1 -- 172.21.15.105:0/811691348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc35c067570 con 0x7fc35c0ad680 2023-11-28T18:25:06.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.093+0000 7fc3517fa640 1 -- 172.21.15.105:0/811691348 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc358012c40 con 0x7fc35c0ad680 2023-11-28T18:25:06.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.093+0000 7fc3517fa640 1 --2- 172.21.15.105:0/811691348 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc348066900 0x7fc348068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.093+0000 7fc3517fa640 1 -- 172.21.15.105:0/811691348 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc358084a10 con 0x7fc35c0ad680 2023-11-28T18:25:06.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.093+0000 7fc353fff640 1 --2- 172.21.15.105:0/811691348 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc348066900 0x7fc348068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.093+0000 7fc353fff640 1 --2- 172.21.15.105:0/811691348 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc348066900 0x7fc348068dc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc35c151c80 tx=0x7fc34c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:06.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.097+0000 7fc3517fa640 1 -- 172.21.15.105:0/811691348 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc358051a70 con 0x7fc35c0ad680 2023-11-28T18:25:06.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.277+0000 7fc363165640 1 -- 172.21.15.105:0/811691348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc35c0cf950 con 0x7fc35c0ad680 2023-11-28T18:25:06.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.277+0000 7fc3517fa640 1 -- 172.21.15.105:0/811691348 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fc358007380 con 0x7fc35c0ad680 2023-11-28T18:25:06.283 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:06.283 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/47307750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":47307750},{"type":"v1","addr":"172.21.15.148:6834","nonce":47307750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5392,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2878687703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2878687703},{"type":"v1","addr":"172.21.15.148:6835","nonce":2878687703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/681913204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":681913204},{"type":"v1","addr":"172.21.15.105:6836","nonce":681913204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:24:55.936428+0000","modified":"2023-11-28T18:24:58.953737+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5469":{"gid":5469,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/1876408690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":1876408690},{"type":"v1","addr":"172.21.15.105:6837","nonce":1876408690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T18:25:06.283 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 18 2023-11-28T18:25:06.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.277+0000 7fc363165640 1 -- 172.21.15.105:0/811691348 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc348066900 msgr2=0x7fc348068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:06.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.277+0000 7fc363165640 1 --2- 172.21.15.105:0/811691348 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc348066900 0x7fc348068dc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc35c151c80 tx=0x7fc34c008040 comp rx=0 tx=0).stop 2023-11-28T18:25:06.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.277+0000 7fc363165640 1 -- 172.21.15.105:0/811691348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc35c0ad680 msgr2=0x7fc35c150ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:06.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.277+0000 7fc363165640 1 --2- 172.21.15.105:0/811691348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc35c0ad680 0x7fc35c150ab0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc35800d5f0 tx=0x7fc35800dab0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.277+0000 7fc363165640 1 -- 172.21.15.105:0/811691348 shutdown_connections 2023-11-28T18:25:06.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.277+0000 7fc363165640 1 --2- 172.21.15.105:0/811691348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc35c159f40 0x7fc35c0700d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.277+0000 7fc363165640 1 --2- 172.21.15.105:0/811691348 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc35c0cc760 0x7fc35c0aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.277+0000 7fc363165640 1 --2- 172.21.15.105:0/811691348 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc348066900 0x7fc348068dc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.277+0000 7fc363165640 1 --2- 172.21.15.105:0/811691348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc35c0ad680 0x7fc35c150ab0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.277+0000 7fc363165640 1 -- 172.21.15.105:0/811691348 >> 172.21.15.105:0/811691348 conn(0x7fc35c0ceb90 msgr2=0x7fc35c0cd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:06.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.277+0000 7fc363165640 1 -- 172.21.15.105:0/811691348 shutdown_connections 2023-11-28T18:25:06.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.281+0000 7fc363165640 1 -- 172.21.15.105:0/811691348 wait complete. 2023-11-28T18:25:06.707 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:24:55.936428+0000', 'modified': '2023-11-28T18:24:58.953737+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5469': {'gid': 5469, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6837/1876408690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6835', 'nonce': 1876408690}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 1876408690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:25:06.707 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T18:25:06.708 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T18:25:06.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.841+0000 7f72b74fe640 1 Processor -- start 2023-11-28T18:25:06.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.841+0000 7f72b74fe640 1 -- start start 2023-11-28T18:25:06.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.845+0000 7f72b74fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f72b006d610 0x7f72b00cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.845+0000 7f72b74fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72b00ce530 0x7f72b00ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.845+0000 7f72b74fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f72b00adf30 0x7f72b01650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.845+0000 7f72b74fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b00ae380 con 0x7f72b00adf30 2023-11-28T18:25:06.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.845+0000 7f72b74fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b0162820 con 0x7f72b006d610 2023-11-28T18:25:06.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.845+0000 7f72b74fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b0161c10 con 0x7f72b00ce530 2023-11-28T18:25:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.845+0000 7f72b5a9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f72b00adf30 0x7f72b01650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.845+0000 7f72b5a9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f72b00adf30 0x7f72b01650f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53902/0 (socket says 172.21.15.105:53902) 2023-11-28T18:25:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.845+0000 7f72b5a9b640 1 -- 172.21.15.105:0/2906510344 learned_addr learned my addr 172.21.15.105:0/2906510344 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.845+0000 7f72b4a99640 1 --2- 172.21.15.105:0/2906510344 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72b00ce530 0x7f72b00ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.845+0000 7f72b529a640 1 --2- 172.21.15.105:0/2906510344 >> v2:172.21.15.148:3300/0 conn(0x7f72b006d610 0x7f72b00cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.845+0000 7f72b5a9b640 1 -- 172.21.15.105:0/2906510344 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72b00ce530 msgr2=0x7f72b00ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.845+0000 7f72b5a9b640 1 --2- 172.21.15.105:0/2906510344 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72b00ce530 0x7f72b00ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.845+0000 7f72b5a9b640 1 -- 172.21.15.105:0/2906510344 >> v2:172.21.15.148:3300/0 conn(0x7f72b006d610 msgr2=0x7f72b00cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.845+0000 7f72b5a9b640 1 --2- 172.21.15.105:0/2906510344 >> v2:172.21.15.148:3300/0 conn(0x7f72b006d610 0x7f72b00cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.845+0000 7f72b5a9b640 1 -- 172.21.15.105:0/2906510344 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b0161ef0 con 0x7f72b00adf30 2023-11-28T18:25:06.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.845+0000 7f72b5a9b640 1 --2- 172.21.15.105:0/2906510344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72b00adf30 0x7f72b01650f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f72b006fa90 tx=0x7f72a801c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe079a5d523e0ce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:06.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.849+0000 7f72a7fff640 1 -- 172.21.15.105:0/2906510344 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a80a5560 con 0x7f72b00adf30 2023-11-28T18:25:06.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.849+0000 7f72a7fff640 1 -- 172.21.15.105:0/2906510344 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a80a56c0 con 0x7f72b00adf30 2023-11-28T18:25:06.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.849+0000 7f72a7fff640 1 -- 172.21.15.105:0/2906510344 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a80a5a50 con 0x7f72b00adf30 2023-11-28T18:25:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.849+0000 7f72b74fe640 1 -- 172.21.15.105:0/2906510344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72b00adf30 msgr2=0x7f72b01650f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.849+0000 7f72b74fe640 1 --2- 172.21.15.105:0/2906510344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72b00adf30 0x7f72b01650f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f72b006fa90 tx=0x7f72a801c6e0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.849+0000 7f72b74fe640 1 -- 172.21.15.105:0/2906510344 shutdown_connections 2023-11-28T18:25:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.849+0000 7f72b74fe640 1 --2- 172.21.15.105:0/2906510344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72b00adf30 0x7f72b01650f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.849+0000 7f72b74fe640 1 --2- 172.21.15.105:0/2906510344 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72b00ce530 0x7f72b00ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.849+0000 7f72b74fe640 1 --2- 172.21.15.105:0/2906510344 >> v2:172.21.15.148:3300/0 conn(0x7f72b006d610 0x7f72b00cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.849+0000 7f72b74fe640 1 -- 172.21.15.105:0/2906510344 >> 172.21.15.105:0/2906510344 conn(0x7f72b00ceb90 msgr2=0x7f72b00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.849+0000 7f72b74fe640 1 -- 172.21.15.105:0/2906510344 shutdown_connections 2023-11-28T18:25:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.849+0000 7f72b74fe640 1 -- 172.21.15.105:0/2906510344 wait complete. 2023-11-28T18:25:06.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.849+0000 7f72b74fe640 1 Processor -- start 2023-11-28T18:25:06.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.849+0000 7f72b74fe640 1 -- start start 2023-11-28T18:25:06.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.849+0000 7f72b74fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72b006d610 0x7f72b0162b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.849+0000 7f72b74fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72b00adf30 0x7f72b01648e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.849+0000 7f72b74fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72b00ce530 0x7f72b0154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.849+0000 7f72b74fe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b015ec20 con 0x7f72b00adf30 2023-11-28T18:25:06.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.849+0000 7f72b74fe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b0163040 con 0x7f72b00ce530 2023-11-28T18:25:06.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.849+0000 7f72b74fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b0164e20 con 0x7f72b006d610 2023-11-28T18:25:06.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.849+0000 7f72b4a99640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72b00adf30 0x7f72b01648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.853+0000 7f72b4a99640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72b00adf30 0x7f72b01648e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53910/0 (socket says 172.21.15.105:53910) 2023-11-28T18:25:06.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.853+0000 7f72b4a99640 1 -- 172.21.15.105:0/4120796876 learned_addr learned my addr 172.21.15.105:0/4120796876 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:06.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.853+0000 7f72b529a640 1 --2- 172.21.15.105:0/4120796876 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72b006d610 0x7f72b0162b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.853+0000 7f72b5a9b640 1 --2- 172.21.15.105:0/4120796876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72b00ce530 0x7f72b0154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.853+0000 7f72b4a99640 1 -- 172.21.15.105:0/4120796876 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72b006d610 msgr2=0x7f72b0162b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:06.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.853+0000 7f72b4a99640 1 --2- 172.21.15.105:0/4120796876 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72b006d610 0x7f72b0162b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.853+0000 7f72b4a99640 1 -- 172.21.15.105:0/4120796876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72b00ce530 msgr2=0x7f72b0154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:06.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.853+0000 7f72b4a99640 1 --2- 172.21.15.105:0/4120796876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72b00ce530 0x7f72b0154170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.853+0000 7f72b4a99640 1 -- 172.21.15.105:0/4120796876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a8009b30 con 0x7f72b00adf30 2023-11-28T18:25:06.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.853+0000 7f72b4a99640 1 --2- 172.21.15.105:0/4120796876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72b00adf30 0x7f72b01648e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f72a000d5f0 tx=0x7f72a000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:06.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.853+0000 7f72a5ffb640 1 -- 172.21.15.105:0/4120796876 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a00125f0 con 0x7f72b00adf30 2023-11-28T18:25:06.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.853+0000 7f72a5ffb640 1 -- 172.21.15.105:0/4120796876 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a0012750 con 0x7f72b00adf30 2023-11-28T18:25:06.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.853+0000 7f72a5ffb640 1 -- 172.21.15.105:0/4120796876 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a0012ae0 con 0x7f72b00adf30 2023-11-28T18:25:06.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.853+0000 7f72b74fe640 1 -- 172.21.15.105:0/4120796876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b015ee70 con 0x7f72b00adf30 2023-11-28T18:25:06.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.853+0000 7f72b74fe640 1 -- 172.21.15.105:0/4120796876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b015f430 con 0x7f72b00adf30 2023-11-28T18:25:06.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.853+0000 7f72a5ffb640 1 -- 172.21.15.105:0/4120796876 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f72a0012c40 con 0x7f72b00adf30 2023-11-28T18:25:06.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72a5ffb640 1 --2- 172.21.15.105:0/4120796876 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7294066900 0x7f7294068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72a5ffb640 1 -- 172.21.15.105:0/4120796876 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f72a0084900 con 0x7f72b00adf30 2023-11-28T18:25:06.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72b529a640 1 --2- 172.21.15.105:0/4120796876 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7294066900 0x7f7294068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72ba85b000 1 Processor -- start 2023-11-28T18:25:06.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72ba85b000 1 -- start start 2023-11-28T18:25:06.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72b529a640 1 --2- 172.21.15.105:0/4120796876 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7294066900 0x7f7294068dc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f72b00b4c30 tx=0x7f72ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:06.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72ba85b000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe86c5d30 0x55ffe86c6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72ba85b000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffe86c6650 0x55ffe8678fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72ba85b000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffe8679500 0x55ffe86798e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72ba85b000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe8542110 con 0x55ffe86c5d30 2023-11-28T18:25:06.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72ba85b000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe84d29a0 con 0x55ffe86c6650 2023-11-28T18:25:06.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72ba85b000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffe84cfed0 con 0x55ffe8679500 2023-11-28T18:25:06.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72b529a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe86c5d30 0x55ffe86c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72b529a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe86c5d30 0x55ffe86c6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53922/0 (socket says 172.21.15.105:53922) 2023-11-28T18:25:06.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72b529a640 1 -- 172.21.15.105:0/4091210862 learned_addr learned my addr 172.21.15.105:0/4091210862 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:06.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72b4a99640 1 --2- 172.21.15.105:0/4091210862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffe86c6650 0x55ffe8678fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72b5a9b640 1 --2- 172.21.15.105:0/4091210862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffe8679500 0x55ffe86798e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72b529a640 1 -- 172.21.15.105:0/4091210862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffe8679500 msgr2=0x55ffe86798e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:06.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72b529a640 1 --2- 172.21.15.105:0/4091210862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffe8679500 0x55ffe86798e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72b529a640 1 -- 172.21.15.105:0/4091210862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffe86c6650 msgr2=0x55ffe8678fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:06.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72b529a640 1 --2- 172.21.15.105:0/4091210862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffe86c6650 0x55ffe8678fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72b529a640 1 -- 172.21.15.105:0/4091210862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffe81c3890 con 0x55ffe86c5d30 2023-11-28T18:25:06.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72b529a640 1 --2- 172.21.15.105:0/4091210862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe86c5d30 0x55ffe86c6110 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f72ac004010 tx=0x7f72ac013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daeee8f00a45966f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:06.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f7292ffd640 1 -- 172.21.15.105:0/4091210862 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac016c10 con 0x55ffe86c5d30 2023-11-28T18:25:06.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f7292ffd640 1 -- 172.21.15.105:0/4091210862 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ac01a070 con 0x55ffe86c5d30 2023-11-28T18:25:06.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f7292ffd640 1 -- 172.21.15.105:0/4091210862 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac021440 con 0x55ffe86c5d30 2023-11-28T18:25:06.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72ba85b000 1 -- 172.21.15.105:0/4091210862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe86c5d30 msgr2=0x55ffe86c6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:06.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72ba85b000 1 --2- 172.21.15.105:0/4091210862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe86c5d30 0x55ffe86c6110 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f72ac004010 tx=0x7f72ac013000 comp rx=0 tx=0).stop 2023-11-28T18:25:06.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72ba85b000 1 -- 172.21.15.105:0/4091210862 shutdown_connections 2023-11-28T18:25:06.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72ba85b000 1 --2- 172.21.15.105:0/4091210862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffe8679500 0x55ffe86798e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72ba85b000 1 --2- 172.21.15.105:0/4091210862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffe86c6650 0x55ffe8678fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72ba85b000 1 --2- 172.21.15.105:0/4091210862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe86c5d30 0x55ffe86c6110 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72ba85b000 1 -- 172.21.15.105:0/4091210862 >> 172.21.15.105:0/4091210862 conn(0x55ffe86a68c0 msgr2=0x55ffe86a8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:06.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72ba85b000 1 -- 172.21.15.105:0/4091210862 shutdown_connections 2023-11-28T18:25:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.857+0000 7f72ba85b000 1 -- 172.21.15.105:0/4091210862 wait complete. 2023-11-28T18:25:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72ba85b000 1 -- start start 2023-11-28T18:25:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72ba85b000 1 Processor -- start 2023-11-28T18:25:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72ba85b000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe8679500 0x55ffe867f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72ba85b000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffe86c5d30 0x55ffe867fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72ba85b000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffe86c6650 0x55ffe867ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72ba85b000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe84cb7c0 con 0x55ffe8679500 2023-11-28T18:25:06.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72ba85b000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe8483bf0 con 0x55ffe86c5d30 2023-11-28T18:25:06.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72ba85b000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffe8489a90 con 0x55ffe86c6650 2023-11-28T18:25:06.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72ba85b000 10 client.0 _ll_register_callbacks cb 0x55ffe8666e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:25:06.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72b529a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe8679500 0x55ffe867f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72b529a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe8679500 0x55ffe867f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53924/0 (socket says 172.21.15.105:53924) 2023-11-28T18:25:06.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72b529a640 1 -- 172.21.15.105:0/1083377692 learned_addr learned my addr 172.21.15.105:0/1083377692 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:06.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72b4a99640 1 --2- 172.21.15.105:0/1083377692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffe86c5d30 0x55ffe867fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72b5a9b640 1 --2- 172.21.15.105:0/1083377692 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffe86c6650 0x55ffe867ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72b529a640 1 -- 172.21.15.105:0/1083377692 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffe86c6650 msgr2=0x55ffe867ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72b529a640 1 --2- 172.21.15.105:0/1083377692 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffe86c6650 0x55ffe867ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72b529a640 1 -- 172.21.15.105:0/1083377692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffe86c5d30 msgr2=0x55ffe867fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72b529a640 1 --2- 172.21.15.105:0/1083377692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffe86c5d30 0x55ffe867fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72b529a640 1 -- 172.21.15.105:0/1083377692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ac00e040 con 0x55ffe8679500 2023-11-28T18:25:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72b4a99640 1 --2- 172.21.15.105:0/1083377692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffe86c5d30 0x55ffe867fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72b529a640 1 --2- 172.21.15.105:0/1083377692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe8679500 0x55ffe867f4e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f72ac023040 tx=0x7f72ac004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb49dc26cf1db86 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f728bfff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:25:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f728bfff640 1 -- 172.21.15.105:0/1083377692 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac021a00 con 0x55ffe8679500 2023-11-28T18:25:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f728bfff640 1 -- 172.21.15.105:0/1083377692 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f72ac021b60 con 0x55ffe8679500 2023-11-28T18:25:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72ba85b000 1 -- 172.21.15.105:0/1083377692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ffe848d470 con 0x55ffe8679500 2023-11-28T18:25:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f728bfff640 1 -- 172.21.15.105:0/1083377692 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72ac021d20 con 0x55ffe8679500 2023-11-28T18:25:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72ba85b000 10 client.5676 fetch_fsmap learned FSMap version 18 2023-11-28T18:25:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72ba85b000 1 -- 172.21.15.105:0/1083377692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55ffe848c190 con 0x55ffe8679500 2023-11-28T18:25:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f728bfff640 1 -- 172.21.15.105:0/1083377692 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ffe848d470 con 0x55ffe8679500 2023-11-28T18:25:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f728bfff640 1 -- 172.21.15.105:0/1083377692 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f72ac014430 con 0x55ffe8679500 2023-11-28T18:25:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72ba85b000 10 client.5676 fetch_fsmap finished waiting for FSMap version 18 2023-11-28T18:25:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72ba85b000 10 client.5676 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:25:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72ba85b000 20 client.5676 populate_metadata read hostname 'smithi105' 2023-11-28T18:25:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72ba85b000 1 --2- 172.21.15.105:0/1083377692 >> [v2:172.21.15.105:6835/1876408690,v1:172.21.15.105:6837/1876408690] conn(0x55ffe8680970 0x55ffe86d7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72ba85b000 4 client.5676 mds_command: new command op to 5469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:25:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.861+0000 7f72ba85b000 1 -- 172.21.15.105:0/1083377692 --> [v2:172.21.15.105:6835/1876408690,v1:172.21.15.105:6837/1876408690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffe8490b50 con 0x55ffe8680970 2023-11-28T18:25:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.865+0000 7f72b4a99640 1 --2- 172.21.15.105:0/1083377692 >> [v2:172.21.15.105:6835/1876408690,v1:172.21.15.105:6837/1876408690] conn(0x55ffe8680970 0x55ffe86d7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.865+0000 7f72b4a99640 1 --2- 172.21.15.105:0/1083377692 >> [v2:172.21.15.105:6835/1876408690,v1:172.21.15.105:6837/1876408690] conn(0x55ffe8680970 0x55ffe86d7ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d74f250dbcdde48 server_cookie=b9fd3a3d58031a3f in_seq=0 out_seq=0 2023-11-28T18:25:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.865+0000 7f728bfff640 10 client.5676 ms_handle_connect on v2:172.21.15.105:6835/1876408690 2023-11-28T18:25:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.865+0000 7f728bfff640 1 -- 172.21.15.105:0/1083377692 <== mds.0 v2:172.21.15.105:6835/1876408690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ffe8490b50 con 0x55ffe8680970 2023-11-28T18:25:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.865+0000 7f728bfff640 10 client.5676 handle_command_reply: tid=0 2023-11-28T18:25:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.865+0000 7f72ba85b000 1 client.5676 shutdown 2023-11-28T18:25:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.865+0000 7f72ba85b000 1 -- 172.21.15.105:0/1083377692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe8679500 msgr2=0x55ffe867f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.865+0000 7f72ba85b000 1 --2- 172.21.15.105:0/1083377692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe8679500 0x55ffe867f4e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f72ac023040 tx=0x7f72ac004740 comp rx=0 tx=0).stop 2023-11-28T18:25:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.865+0000 7f72ba85b000 1 -- 172.21.15.105:0/1083377692 shutdown_connections 2023-11-28T18:25:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.865+0000 7f72ba85b000 1 --2- 172.21.15.105:0/1083377692 >> [v2:172.21.15.105:6835/1876408690,v1:172.21.15.105:6837/1876408690] conn(0x55ffe8680970 0x55ffe86d7ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.865+0000 7f72ba85b000 1 --2- 172.21.15.105:0/1083377692 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffe86c6650 0x55ffe867ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.865+0000 7f72ba85b000 1 --2- 172.21.15.105:0/1083377692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffe86c5d30 0x55ffe867fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.865+0000 7f728bfff640 0 client.5676 ms_handle_reset on v2:172.21.15.105:6835/1876408690 2023-11-28T18:25:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.865+0000 7f72ba85b000 1 --2- 172.21.15.105:0/1083377692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe8679500 0x55ffe867f4e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.865+0000 7f72ba85b000 1 -- 172.21.15.105:0/1083377692 >> 172.21.15.105:0/1083377692 conn(0x55ffe86a68c0 msgr2=0x55ffe86786b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.865+0000 7f72ba85b000 1 -- 172.21.15.105:0/1083377692 shutdown_connections 2023-11-28T18:25:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.865+0000 7f72ba85b000 1 -- 172.21.15.105:0/1083377692 wait complete. 2023-11-28T18:25:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.865+0000 7f72ba85b000 20 client.5676 trim_cache size 0 max 16384 2023-11-28T18:25:06.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72ba85b000 1 Processor -- start 2023-11-28T18:25:06.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72ba85b000 1 -- start start 2023-11-28T18:25:06.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72ba85b000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffe8679500 0x55ffe86a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72ba85b000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe86c5d30 0x55ffe86a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72ba85b000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffe86c6650 0x55ffe86d7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72ba85b000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe8184940 con 0x55ffe86c5d30 2023-11-28T18:25:06.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72ba85b000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe848a8f0 con 0x55ffe86c6650 2023-11-28T18:25:06.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72ba85b000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffe81c41b0 con 0x55ffe8679500 2023-11-28T18:25:06.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72b4a99640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe86c5d30 0x55ffe86a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72b4a99640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe86c5d30 0x55ffe86a58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53926/0 (socket says 172.21.15.105:53926) 2023-11-28T18:25:06.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72b4a99640 1 -- 172.21.15.105:0/3626008193 learned_addr learned my addr 172.21.15.105:0/3626008193 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:06.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72b5a9b640 1 --2- 172.21.15.105:0/3626008193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffe86c6650 0x55ffe86d7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72b529a640 1 --2- 172.21.15.105:0/3626008193 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffe8679500 0x55ffe86a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72b4a99640 1 -- 172.21.15.105:0/3626008193 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffe8679500 msgr2=0x55ffe86a53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:06.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72b4a99640 1 --2- 172.21.15.105:0/3626008193 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffe8679500 0x55ffe86a53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72b4a99640 1 -- 172.21.15.105:0/3626008193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffe86c6650 msgr2=0x55ffe86d7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:06.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72b4a99640 1 --2- 172.21.15.105:0/3626008193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffe86c6650 0x55ffe86d7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72b4a99640 1 -- 172.21.15.105:0/3626008193 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffe8679370 con 0x55ffe86c5d30 2023-11-28T18:25:06.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72b4a99640 1 --2- 172.21.15.105:0/3626008193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe86c5d30 0x55ffe86a58e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f72a0082ce0 tx=0x7f72a0020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b812df7cddd45b23 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:06.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72937fe640 1 -- 172.21.15.105:0/3626008193 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a00538a0 con 0x55ffe86c5d30 2023-11-28T18:25:06.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72937fe640 1 -- 172.21.15.105:0/3626008193 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a0082d60 con 0x55ffe86c5d30 2023-11-28T18:25:06.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72937fe640 1 -- 172.21.15.105:0/3626008193 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a0051e70 con 0x55ffe86c5d30 2023-11-28T18:25:06.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72ba85b000 1 -- 172.21.15.105:0/3626008193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe86c5d30 msgr2=0x55ffe86a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:06.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72ba85b000 1 --2- 172.21.15.105:0/3626008193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe86c5d30 0x55ffe86a58e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f72a0082ce0 tx=0x7f72a0020040 comp rx=0 tx=0).stop 2023-11-28T18:25:06.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72ba85b000 1 -- 172.21.15.105:0/3626008193 shutdown_connections 2023-11-28T18:25:06.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72ba85b000 1 --2- 172.21.15.105:0/3626008193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffe86c6650 0x55ffe86d7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72ba85b000 1 --2- 172.21.15.105:0/3626008193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe86c5d30 0x55ffe86a58e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72ba85b000 1 --2- 172.21.15.105:0/3626008193 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffe8679500 0x55ffe86a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.881+0000 7f72ba85b000 1 -- 172.21.15.105:0/3626008193 >> 172.21.15.105:0/3626008193 conn(0x55ffe86a68c0 msgr2=0x55ffe8677720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:06.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72ba85b000 1 -- 172.21.15.105:0/3626008193 shutdown_connections 2023-11-28T18:25:06.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72ba85b000 1 -- 172.21.15.105:0/3626008193 wait complete. 2023-11-28T18:25:06.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72ba85b000 1 -- start start 2023-11-28T18:25:06.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72ba85b000 1 Processor -- start 2023-11-28T18:25:06.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72ba85b000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe8679500 0x55ffe8726810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72ba85b000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffe86c5d30 0x55ffe8726d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72ba85b000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffe86c6650 0x55ffe86e1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72ba85b000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe8568de0 con 0x55ffe8679500 2023-11-28T18:25:06.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72ba85b000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffe84c75c0 con 0x55ffe86c6650 2023-11-28T18:25:06.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72ba85b000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffe86dccd0 con 0x55ffe86c5d30 2023-11-28T18:25:06.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72ba85b000 10 client.0 _ll_register_callbacks cb 0x55ffe8542060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:25:06.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72b529a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe8679500 0x55ffe8726810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72b529a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe8679500 0x55ffe8726810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53942/0 (socket says 172.21.15.105:53942) 2023-11-28T18:25:06.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72b529a640 1 -- 172.21.15.105:0/4204906343 learned_addr learned my addr 172.21.15.105:0/4204906343 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:06.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72b4a99640 1 --2- 172.21.15.105:0/4204906343 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffe86c5d30 0x55ffe8726d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72b5a9b640 1 --2- 172.21.15.105:0/4204906343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffe86c6650 0x55ffe86e1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72b529a640 1 -- 172.21.15.105:0/4204906343 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffe86c5d30 msgr2=0x55ffe8726d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:06.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72b529a640 1 --2- 172.21.15.105:0/4204906343 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffe86c5d30 0x55ffe8726d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72b529a640 1 -- 172.21.15.105:0/4204906343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffe86c6650 msgr2=0x55ffe86e1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:06.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72b529a640 1 --2- 172.21.15.105:0/4204906343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffe86c6650 0x55ffe86e1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72b529a640 1 -- 172.21.15.105:0/4204906343 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b015f430 con 0x55ffe8679500 2023-11-28T18:25:06.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72b529a640 1 --2- 172.21.15.105:0/4204906343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe8679500 0x55ffe8726810 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0146e0 tx=0x7f72ac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6250741816ab8407 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:06.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f728bfff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:25:06.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f728bfff640 1 -- 172.21.15.105:0/4204906343 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac00a070 con 0x55ffe8679500 2023-11-28T18:25:06.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f728bfff640 1 -- 172.21.15.105:0/4204906343 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f72ac021e10 con 0x55ffe8679500 2023-11-28T18:25:06.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72ba85b000 1 -- 172.21.15.105:0/4204906343 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ffe84ca110 con 0x55ffe8679500 2023-11-28T18:25:06.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f728bfff640 1 -- 172.21.15.105:0/4204906343 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72ac004260 con 0x55ffe8679500 2023-11-28T18:25:06.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72ba85b000 10 client.5682 fetch_fsmap learned FSMap version 18 2023-11-28T18:25:06.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72ba85b000 1 -- 172.21.15.105:0/4204906343 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55ffe8472490 con 0x55ffe8679500 2023-11-28T18:25:06.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f728bfff640 1 -- 172.21.15.105:0/4204906343 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ffe84ca110 con 0x55ffe8679500 2023-11-28T18:25:06.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f728bfff640 1 -- 172.21.15.105:0/4204906343 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f72ac047060 con 0x55ffe8679500 2023-11-28T18:25:06.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72ba85b000 10 client.5682 fetch_fsmap finished waiting for FSMap version 18 2023-11-28T18:25:06.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72ba85b000 10 client.5682 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:25:06.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72ba85b000 20 client.5682 populate_metadata read hostname 'smithi105' 2023-11-28T18:25:06.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72ba85b000 1 --2- 172.21.15.105:0/4204906343 >> [v2:172.21.15.105:6835/1876408690,v1:172.21.15.105:6837/1876408690] conn(0x55ffe86e2430 0x55ffe86e2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:06.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72ba85b000 4 client.5682 mds_command: new command op to 5469 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:25:06.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72ba85b000 1 -- 172.21.15.105:0/4204906343 --> [v2:172.21.15.105:6835/1876408690,v1:172.21.15.105:6837/1876408690] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ffe84f8650 con 0x55ffe86e2430 2023-11-28T18:25:06.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.885+0000 7f72b4a99640 1 --2- 172.21.15.105:0/4204906343 >> [v2:172.21.15.105:6835/1876408690,v1:172.21.15.105:6837/1876408690] conn(0x55ffe86e2430 0x55ffe86e2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:06.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f728bfff640 10 client.5682 ms_handle_connect on v2:172.21.15.105:6835/1876408690 2023-11-28T18:25:06.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72b4a99640 1 --2- 172.21.15.105:0/4204906343 >> [v2:172.21.15.105:6835/1876408690,v1:172.21.15.105:6837/1876408690] conn(0x55ffe86e2430 0x55ffe86e2810 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66776b84ba8b9326 server_cookie=8c25b2644d311834 in_seq=0 out_seq=0 2023-11-28T18:25:06.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f728bfff640 1 -- 172.21.15.105:0/4204906343 <== mds.0 v2:172.21.15.105:6835/1876408690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55ffe84f8650 con 0x55ffe86e2430 2023-11-28T18:25:06.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f728bfff640 10 client.5682 handle_command_reply: tid=0 2023-11-28T18:25:06.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72ba85b000 1 client.5682 shutdown 2023-11-28T18:25:06.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72ba85b000 1 -- 172.21.15.105:0/4204906343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe8679500 msgr2=0x55ffe8726810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:06.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72ba85b000 1 --2- 172.21.15.105:0/4204906343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe8679500 0x55ffe8726810 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0146e0 tx=0x7f72ac016000 comp rx=0 tx=0).stop 2023-11-28T18:25:06.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72ba85b000 1 -- 172.21.15.105:0/4204906343 shutdown_connections 2023-11-28T18:25:06.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72ba85b000 1 --2- 172.21.15.105:0/4204906343 >> [v2:172.21.15.105:6835/1876408690,v1:172.21.15.105:6837/1876408690] conn(0x55ffe86e2430 0x55ffe86e2810 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72ba85b000 1 --2- 172.21.15.105:0/4204906343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffe86c6650 0x55ffe86e1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72ba85b000 1 --2- 172.21.15.105:0/4204906343 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffe86c5d30 0x55ffe8726d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f728bfff640 0 client.5682 ms_handle_reset on v2:172.21.15.105:6835/1876408690 2023-11-28T18:25:06.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72ba85b000 1 --2- 172.21.15.105:0/4204906343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe8679500 0x55ffe8726810 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72ba85b000 1 -- 172.21.15.105:0/4204906343 >> 172.21.15.105:0/4204906343 conn(0x55ffe86a68c0 msgr2=0x55ffe8677fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:06.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72ba85b000 1 -- 172.21.15.105:0/4204906343 shutdown_connections 2023-11-28T18:25:06.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72ba85b000 1 -- 172.21.15.105:0/4204906343 wait complete. 2023-11-28T18:25:06.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72ba85b000 20 client.5682 trim_cache size 0 max 16384 2023-11-28T18:25:06.905 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:25:06.905 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:25:06.906 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2023-11-28T18:25:06.906 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5469, 2023-11-28T18:25:06.906 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:25:06.906 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:25:06.906 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:25:06.906 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 10.321110794999999, 2023-11-28T18:25:06.907 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 18, 2023-11-28T18:25:06.907 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 38, 2023-11-28T18:25:06.907 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 38, 2023-11-28T18:25:06.907 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 18.206753470999999 2023-11-28T18:25:06.907 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:25:06.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72b74fe640 1 -- 172.21.15.105:0/4120796876 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7294066900 msgr2=0x7f7294068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:06.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72b74fe640 1 --2- 172.21.15.105:0/4120796876 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7294066900 0x7f7294068dc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f72b00b4c30 tx=0x7f72ac008040 comp rx=0 tx=0).stop 2023-11-28T18:25:06.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72b74fe640 1 -- 172.21.15.105:0/4120796876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72b00adf30 msgr2=0x7f72b01648e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:06.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72b74fe640 1 --2- 172.21.15.105:0/4120796876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72b00adf30 0x7f72b01648e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f72a000d5f0 tx=0x7f72a000dab0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72b74fe640 1 -- 172.21.15.105:0/4120796876 shutdown_connections 2023-11-28T18:25:06.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72b74fe640 1 --2- 172.21.15.105:0/4120796876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72b00ce530 0x7f72b0154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72b74fe640 1 --2- 172.21.15.105:0/4120796876 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7294066900 0x7f7294068dc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72b74fe640 1 --2- 172.21.15.105:0/4120796876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72b00adf30 0x7f72b01648e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72b74fe640 1 --2- 172.21.15.105:0/4120796876 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72b006d610 0x7f72b0162b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:06.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72b74fe640 1 -- 172.21.15.105:0/4120796876 >> 172.21.15.105:0/4120796876 conn(0x7f72b00ceb90 msgr2=0x7f72b00ac470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:06.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72b74fe640 1 -- 172.21.15.105:0/4120796876 shutdown_connections 2023-11-28T18:25:06.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:06.889+0000 7f72b74fe640 1 -- 172.21.15.105:0/4120796876 wait complete. 2023-11-28T18:25:06.913 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T18:25:07.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.045+0000 7f95e2d2a640 1 Processor -- start 2023-11-28T18:25:07.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.045+0000 7f95e2d2a640 1 -- start start 2023-11-28T18:25:07.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.045+0000 7f95e2d2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95dc0d3c30 0x7f95dc0d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:07.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.045+0000 7f95e2d2a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95dc0b25b0 con 0x7f95dc15d4f0 2023-11-28T18:25:07.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.045+0000 7f95e2d2a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95dc0cea30 con 0x7f95dc0b2d20 2023-11-28T18:25:07.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.045+0000 7f95e2d2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc0b2900 con 0x7f95dc0d3c30 2023-11-28T18:25:07.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.045+0000 7f95e0ac6640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f95dc15d4f0 0x7f95dc0d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39392/0 (socket says 172.21.15.105:39392) 2023-11-28T18:25:07.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.045+0000 7f95e0ac6640 1 -- 172.21.15.105:0/1071685786 learned_addr learned my addr 172.21.15.105:0/1071685786 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:07.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.045+0000 7f95db7fe640 1 -- 172.21.15.105:0/1071685786 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166908343 0 0) 0x7f95dc0b25b0 con 0x7f95dc15d4f0 2023-11-28T18:25:07.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.045+0000 7f95db7fe640 1 -- 172.21.15.105:0/1071685786 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95cc003530 con 0x7f95dc15d4f0 2023-11-28T18:25:07.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.045+0000 7f95dbfff640 1 --2- 172.21.15.105:0/1071685786 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95dc0d3c30 0x7f95dc0d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:07.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.045+0000 7f95db7fe640 1 -- 172.21.15.105:0/1071685786 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2724406540 0 0) 0x7f95cc003530 con 0x7f95dc15d4f0 2023-11-28T18:25:07.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.045+0000 7f95db7fe640 1 -- 172.21.15.105:0/1071685786 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95dc0d3c30 msgr2=0x7f95dc0d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:07.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.045+0000 7f95db7fe640 1 --2- 172.21.15.105:0/1071685786 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95dc0d3c30 0x7f95dc0d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:07.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.045+0000 7f95db7fe640 1 -- 172.21.15.105:0/1071685786 >> v1:172.21.15.148:6789/0 conn(0x7f95dc0b2d20 legacy=0x7f95dc0b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:07.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.045+0000 7f95db7fe640 1 -- 172.21.15.105:0/1071685786 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95dc0ced10 con 0x7f95dc15d4f0 2023-11-28T18:25:07.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.045+0000 7f95db7fe640 1 -- 172.21.15.105:0/1071685786 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f95d0001680 con 0x7f95dc15d4f0 2023-11-28T18:25:07.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.045+0000 7f95db7fe640 1 -- 172.21.15.105:0/1071685786 >> v1:172.21.15.105:6789/0 conn(0x7f95dc15d4f0 legacy=0x7f95dc0d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:07.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.045+0000 7f95db7fe640 1 --2- 172.21.15.105:0/1071685786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95cc0039b0 0x7f95cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:07.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.049+0000 7f95db7fe640 1 --2- 172.21.15.105:0/1071685786 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95dc0d3c30 0x7f95cc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:07.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.049+0000 7f95db7fe640 1 --2- 172.21.15.105:0/1071685786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95cc004d10 0x7f95cc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:07.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.049+0000 7f95db7fe640 1 -- 172.21.15.105:0/1071685786 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc0b2900 con 0x7f95cc004d10 2023-11-28T18:25:07.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.049+0000 7f95db7fe640 1 -- 172.21.15.105:0/1071685786 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc00bb50 con 0x7f95cc0039b0 2023-11-28T18:25:07.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.049+0000 7f95db7fe640 1 -- 172.21.15.105:0/1071685786 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc00bd10 con 0x7f95dc0d3c30 2023-11-28T18:25:07.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.049+0000 7f95e12c7640 1 --2- 172.21.15.105:0/1071685786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95cc004d10 0x7f95cc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:07.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.049+0000 7f95dbfff640 1 --2- 172.21.15.105:0/1071685786 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95dc0d3c30 0x7f95cc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:07.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.049+0000 7f95e0ac6640 1 --2- 172.21.15.105:0/1071685786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95cc0039b0 0x7f95cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:07.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.049+0000 7f95e12c7640 1 -- 172.21.15.105:0/1071685786 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95dc0d3c30 msgr2=0x7f95cc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:07.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.049+0000 7f95e12c7640 1 --2- 172.21.15.105:0/1071685786 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95dc0d3c30 0x7f95cc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:07.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.049+0000 7f95e12c7640 1 -- 172.21.15.105:0/1071685786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95cc0039b0 msgr2=0x7f95cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:07.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.049+0000 7f95e12c7640 1 --2- 172.21.15.105:0/1071685786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95cc0039b0 0x7f95cc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:07.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.049+0000 7f95e12c7640 1 -- 172.21.15.105:0/1071685786 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc003530 con 0x7f95cc004d10 2023-11-28T18:25:07.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.049+0000 7f95dbfff640 1 --2- 172.21.15.105:0/1071685786 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95dc0d3c30 0x7f95cc0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:25:07.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.049+0000 7f95e12c7640 1 --2- 172.21.15.105:0/1071685786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95cc004d10 0x7f95cc005190 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f95d4001e70 tx=0x7f95d401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116a81908916b8c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:07.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.049+0000 7f95db7fe640 1 -- 172.21.15.105:0/1071685786 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4003b80 con 0x7f95cc004d10 2023-11-28T18:25:07.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.049+0000 7f95db7fe640 1 -- 172.21.15.105:0/1071685786 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d4003ce0 con 0x7f95cc004d10 2023-11-28T18:25:07.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.049+0000 7f95db7fe640 1 -- 172.21.15.105:0/1071685786 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d40052c0 con 0x7f95cc004d10 2023-11-28T18:25:07.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.053+0000 7f95e2d2a640 1 -- 172.21.15.105:0/1071685786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95cc004d10 msgr2=0x7f95cc005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:07.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.053+0000 7f95e2d2a640 1 --2- 172.21.15.105:0/1071685786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95cc004d10 0x7f95cc005190 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f95d4001e70 tx=0x7f95d401c760 comp rx=0 tx=0).stop 2023-11-28T18:25:07.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.053+0000 7f95e12c7640 1 -- 172.21.15.105:0/1071685786 reap_dead start 2023-11-28T18:25:07.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.053+0000 7f95e2d2a640 1 -- 172.21.15.105:0/1071685786 shutdown_connections 2023-11-28T18:25:07.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.053+0000 7f95e2d2a640 1 -- 172.21.15.105:0/1071685786 >> 172.21.15.105:0/1071685786 conn(0x7f95dc0d3030 msgr2=0x7f95dc0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:07.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.053+0000 7f95e2d2a640 1 -- 172.21.15.105:0/1071685786 shutdown_connections 2023-11-28T18:25:07.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.053+0000 7f95e2d2a640 1 -- 172.21.15.105:0/1071685786 wait complete. 2023-11-28T18:25:07.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.053+0000 7f95e2d2a640 1 Processor -- start 2023-11-28T18:25:07.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.053+0000 7f95e2d2a640 1 -- start start 2023-11-28T18:25:07.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.053+0000 7f95e2d2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95cc004d10 0x7f95dc1506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:07.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.053+0000 7f95e2d2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc160a90 0x7f95dc0a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:07.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.053+0000 7f95e2d2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95dc1f2680 0x7f95dc1f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:07.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.053+0000 7f95e2d2a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc0d0a20 con 0x7f95dc160a90 2023-11-28T18:25:07.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.053+0000 7f95e2d2a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc0a0d00 con 0x7f95cc004d10 2023-11-28T18:25:07.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.053+0000 7f95e2d2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc15daf0 con 0x7f95dc1f2680 2023-11-28T18:25:07.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.053+0000 7f95dbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc160a90 0x7f95dc0a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:07.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.053+0000 7f95dbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc160a90 0x7f95dc0a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53960/0 (socket says 172.21.15.105:53960) 2023-11-28T18:25:07.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.053+0000 7f95dbfff640 1 -- 172.21.15.105:0/667242264 learned_addr learned my addr 172.21.15.105:0/667242264 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:07.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.053+0000 7f95e12c7640 1 --2- 172.21.15.105:0/667242264 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95dc1f2680 0x7f95dc1f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:07.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.057+0000 7f95e0ac6640 1 --2- 172.21.15.105:0/667242264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95cc004d10 0x7f95dc1506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:07.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.057+0000 7f95dbfff640 1 -- 172.21.15.105:0/667242264 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95dc1f2680 msgr2=0x7f95dc1f2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:07.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.057+0000 7f95dbfff640 1 --2- 172.21.15.105:0/667242264 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95dc1f2680 0x7f95dc1f2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:07.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.057+0000 7f95dbfff640 1 -- 172.21.15.105:0/667242264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95cc004d10 msgr2=0x7f95dc1506b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:07.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.057+0000 7f95dbfff640 1 --2- 172.21.15.105:0/667242264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95cc004d10 0x7f95dc1506b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:07.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.057+0000 7f95dbfff640 1 -- 172.21.15.105:0/667242264 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95dc15dd90 con 0x7f95dc160a90 2023-11-28T18:25:07.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.057+0000 7f95e0ac6640 1 --2- 172.21.15.105:0/667242264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95cc004d10 0x7f95dc1506b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:07.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.057+0000 7f95dbfff640 1 --2- 172.21.15.105:0/667242264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc160a90 0x7f95dc0a07c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f95c8011ca0 tx=0x7f95c800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:07.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.057+0000 7f95d97fa640 1 -- 172.21.15.105:0/667242264 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c800fd20 con 0x7f95dc160a90 2023-11-28T18:25:07.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.057+0000 7f95d97fa640 1 -- 172.21.15.105:0/667242264 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c800fe80 con 0x7f95dc160a90 2023-11-28T18:25:07.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.057+0000 7f95d97fa640 1 -- 172.21.15.105:0/667242264 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8004260 con 0x7f95dc160a90 2023-11-28T18:25:07.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.057+0000 7f95e2d2a640 1 -- 172.21.15.105:0/667242264 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95dc15e070 con 0x7f95dc160a90 2023-11-28T18:25:07.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.057+0000 7f95e2d2a640 1 -- 172.21.15.105:0/667242264 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95dc15e630 con 0x7f95dc160a90 2023-11-28T18:25:07.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.057+0000 7f95e2d2a640 1 -- 172.21.15.105:0/667242264 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95dc15e7f0 con 0x7f95dc160a90 2023-11-28T18:25:07.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.057+0000 7f95d97fa640 1 -- 172.21.15.105:0/667242264 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f95c80043c0 con 0x7f95dc160a90 2023-11-28T18:25:07.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.057+0000 7f95d97fa640 1 --2- 172.21.15.105:0/667242264 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f95c00668d0 0x7f95c0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:07.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.061+0000 7f95d97fa640 1 -- 172.21.15.105:0/667242264 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f95c8084d00 con 0x7f95dc160a90 2023-11-28T18:25:07.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.061+0000 7f95e0ac6640 1 --2- 172.21.15.105:0/667242264 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f95c00668d0 0x7f95c0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:07.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.061+0000 7f95e0ac6640 1 --2- 172.21.15.105:0/667242264 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f95c00668d0 0x7f95c0068d90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f95d0001bd0 tx=0x7f95d0001a60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:07.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.065+0000 7f95d97fa640 1 -- 172.21.15.105:0/667242264 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f95c8051e00 con 0x7f95dc160a90 2023-11-28T18:25:07.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.245+0000 7f95e2d2a640 1 -- 172.21.15.105:0/667242264 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f95dc067570 con 0x7f95dc160a90 2023-11-28T18:25:07.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.245+0000 7f95d97fa640 1 -- 172.21.15.105:0/667242264 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7f95c8022070 con 0x7f95dc160a90 2023-11-28T18:25:07.252 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T18:25:07.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.249+0000 7f95e2d2a640 1 -- 172.21.15.105:0/667242264 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f95c00668d0 msgr2=0x7f95c0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:07.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.249+0000 7f95e2d2a640 1 --2- 172.21.15.105:0/667242264 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f95c00668d0 0x7f95c0068d90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f95d0001bd0 tx=0x7f95d0001a60 comp rx=0 tx=0).stop 2023-11-28T18:25:07.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.249+0000 7f95e2d2a640 1 -- 172.21.15.105:0/667242264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc160a90 msgr2=0x7f95dc0a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:07.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.249+0000 7f95e2d2a640 1 --2- 172.21.15.105:0/667242264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc160a90 0x7f95dc0a07c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f95c8011ca0 tx=0x7f95c800f610 comp rx=0 tx=0).stop 2023-11-28T18:25:07.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.249+0000 7f95e2d2a640 1 -- 172.21.15.105:0/667242264 shutdown_connections 2023-11-28T18:25:07.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.249+0000 7f95e2d2a640 1 --2- 172.21.15.105:0/667242264 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95dc1f2680 0x7f95dc1f2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:07.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.249+0000 7f95e2d2a640 1 --2- 172.21.15.105:0/667242264 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f95c00668d0 0x7f95c0068d90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:07.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.249+0000 7f95e2d2a640 1 --2- 172.21.15.105:0/667242264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc160a90 0x7f95dc0a07c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:07.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.249+0000 7f95e2d2a640 1 --2- 172.21.15.105:0/667242264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95cc004d10 0x7f95dc1506b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:07.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.249+0000 7f95e2d2a640 1 -- 172.21.15.105:0/667242264 >> 172.21.15.105:0/667242264 conn(0x7f95dc0d3030 msgr2=0x7f95dc15b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:07.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.249+0000 7f95e2d2a640 1 -- 172.21.15.105:0/667242264 shutdown_connections 2023-11-28T18:25:07.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:07.249+0000 7f95e2d2a640 1 -- 172.21.15.105:0/667242264 wait complete. 2023-11-28T18:25:07.273 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T18:25:07.273 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T18:25:07.273 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2023-11-28T18:25:07.273 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T18:25:07.273 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T18:25:07.274 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T18:25:07.274 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T18:25:07.274 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T18:25:07.275 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:25:07.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:25:07.276 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2023-11-28T18:25:07.286 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T18:25:07.286 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T18:25:07.286 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T18:25:07.286 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:25:07.287 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-11-28T18:25:07.287 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:25:07.287 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:25:07.287 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:25:07.287 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2023-11-28T18:25:07.287 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:25:07.287 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:25:07.287 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2023-11-28T18:25:07.287 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:25:07.287 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:25:07.288 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T18:25:07.288 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:25:07.288 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-28T18:25:07.288 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42314sec preferred_lft 42314sec 2023-11-28T18:25:07.288 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2023-11-28T18:25:07.288 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:25:07.288 INFO:teuthology.orchestra.run.smithi105.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-28T18:25:07.288 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 02:42:cc:d4:87:d2 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:25:07.288 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-28T18:25:07.288 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:25:07.288 INFO:teuthology.orchestra.run.smithi105.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T18:25:07.289 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-28T18:25:07.289 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T18:25:07.289 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:25:07.289 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-28T18:25:07.289 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:25:07.289 INFO:teuthology.orchestra.run.smithi105.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T18:25:07.289 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:25:07.289 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-28T18:25:07.289 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:25:07.289 INFO:teuthology.orchestra.run.smithi105.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T18:25:07.289 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:25:07.289 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-28T18:25:07.289 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:25:07.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:25:07.290 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T18:25:07.349 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-11-28T18:25:07.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:25:07.350 DEBUG:teuthology.orchestra.run.smithi105:> route 2023-11-28T18:25:07.404 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2023-11-28T18:25:07.404 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T18:25:07.404 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-28T18:25:07.404 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T18:25:07.404 INFO:teuthology.orchestra.run.smithi105.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-28T18:25:07.404 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T18:25:07.404 INFO:teuthology.orchestra.run.smithi105.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:25:07.404 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:25:07.405 INFO:teuthology.orchestra.run.smithi105.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:25:07.405 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T18:25:07.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:25:07.405 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-11-28T18:25:07.405 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:25:07.405 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T18:25:07.405 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T18:25:07.406 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T18:25:07.406 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:25:07.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:25:07.497 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-11-28T18:25:07.506 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T18:25:07.507 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T18:25:07.507 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:25:07.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:25:07.507 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2023-11-28T18:25:07.574 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-28T18:25:07.574 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:25:07.574 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T18:25:07.629 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:25:07.630 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:25:07.630 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T18:25:07.685 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:25:07.749 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:25:07.802 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:25:07.802 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:25:07.803 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:25:07.803 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:25:07.803 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:25:07.803 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:25:07.803 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:25:07.803 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:25:07.803 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:25:07.803 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:25:07.803 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:25:07.803 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:25:07.803 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:25:07.804 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:25:07.804 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:25:07.804 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:25:07.804 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:25:07.804 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:25:07.804 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:25:07.804 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:25:07.804 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:25:07.804 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:25:07.804 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:25:07.804 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:25:07.804 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:25:07.805 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:25:07.805 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:25:07.805 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:25:07.805 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:25:07.805 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:25:07.805 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:25:07.805 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:25:07.805 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:25:07.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:25:07.806 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:25:07.857 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T18:25:07.857 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-28T18:25:07.904 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:25:07.926 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:25:07.979 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:25:07.979 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:25:07.979 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:25:07.979 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:25:07.979 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:25:07.979 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:25:07.979 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:25:07.980 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:25:07.980 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:25:07.980 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:25:07.980 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:25:07.980 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:25:07.980 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:25:07.980 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:25:07.980 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:25:07.980 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:25:07.980 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:25:07.980 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:25:07.980 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:25:07.981 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:25:07.981 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:25:07.981 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:25:07.981 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:25:07.981 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:25:07.981 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:25:07.981 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:25:07.981 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:25:07.981 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:25:07.981 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:25:07.981 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:25:07.981 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:25:07.982 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:25:07.982 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:25:07.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:25:07.983 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:25:07.998 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:2023-11-28T18:25:07.993+0000 7fab35b36580 -1 init, newargv = 0x55843b13aef0 newargc=15 2023-11-28T18:25:07.998 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[50275]: starting ceph client 2023-11-28T18:25:08.004 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[50275]: starting fuse 2023-11-28T18:25:08.007 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-11-28T18:25:08.007 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-28T18:25:08.008 DEBUG:teuthology.orchestra.run.smithi105:> sudo stdin-killer -- python3 -c ' 2023-11-28T18:25:08.008 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2023-11-28T18:25:08.008 DEBUG:teuthology.orchestra.run.smithi105:> import re 2023-11-28T18:25:08.008 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-11-28T18:25:08.008 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2023-11-28T18:25:08.008 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:25:08.008 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2023-11-28T18:25:08.008 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-28T18:25:08.008 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2023-11-28T18:25:08.009 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-28T18:25:08.009 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:25:08.009 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2023-11-28T18:25:08.009 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2023-11-28T18:25:08.009 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2023-11-28T18:25:08.009 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2023-11-28T18:25:08.009 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:25:08.009 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2023-11-28T18:25:08.009 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T18:25:08.009 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T18:25:08.009 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T18:25:08.010 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2023-11-28T18:25:08.010 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2023-11-28T18:25:08.010 DEBUG:teuthology.orchestra.run.smithi105:> return f 2023-11-28T18:25:08.010 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T18:25:08.010 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:25:08.010 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.0")) 2023-11-28T18:25:08.010 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-11-28T18:25:08.130 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.0.50275.asok 2023-11-28T18:25:08.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-28T18:25:08.141 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50275.asok 2023-11-28T18:25:08.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:25:08.141 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50275.asok status 2023-11-28T18:25:08.317 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:25:08.317 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2023-11-28T18:25:08.317 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "47c1c547372ed5bff7984647dc200bac0e019caf", 2023-11-28T18:25:08.317 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev)", 2023-11-28T18:25:08.317 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "0", 2023-11-28T18:25:08.317 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2023-11-28T18:25:08.318 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-28T18:25:08.318 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "50275", 2023-11-28T18:25:08.318 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2023-11-28T18:25:08.318 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:25:08.318 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2023-11-28T18:25:08.318 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2023-11-28T18:25:08.318 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5697, 2023-11-28T18:25:08.318 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2023-11-28T18:25:08.318 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2023-11-28T18:25:08.318 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2023-11-28T18:25:08.318 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 5697 2023-11-28T18:25:08.318 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:25:08.319 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-11-28T18:25:08.319 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-11-28T18:25:08.319 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-11-28T18:25:08.319 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 2492937982 2023-11-28T18:25:08.319 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:25:08.319 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:25:08.319 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-11-28T18:25:08.319 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-11-28T18:25:08.319 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-11-28T18:25:08.319 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 2492937982 2023-11-28T18:25:08.319 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:25:08.319 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.5697 192.168.0.1:0/2492937982", 2023-11-28T18:25:08.320 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/2492937982", 2023-11-28T18:25:08.320 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2023-11-28T18:25:08.320 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 18, 2023-11-28T18:25:08.320 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 38, 2023-11-28T18:25:08.320 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2023-11-28T18:25:08.320 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2023-11-28T18:25:08.320 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2023-11-28T18:25:08.320 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:25:08.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:25:08.332 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2023-11-28T18:25:08.332 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-28T18:25:08.341 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2023-11-28T18:25:08.342 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-28T18:25:08.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:25:08.342 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T18:25:08.405 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random True 2023-11-28T18:25:08.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.581+0000 7f7721c9c640 1 Processor -- start 2023-11-28T18:25:08.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.585+0000 7f7721c9c640 1 -- start start 2023-11-28T18:25:08.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.585+0000 7f7721c9c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f771c159f40 0x7f771c0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:08.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.585+0000 7f7721c9c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f771c0cc760 0x7f771c0ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:08.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.585+0000 7f7721c9c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f771c06bf50 con 0x7f771c0ad680 2023-11-28T18:25:08.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.585+0000 7f7721c9c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f771c0ccbc0 con 0x7f771c0cc760 2023-11-28T18:25:08.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.585+0000 7f7721c9c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771c15a320 con 0x7f771c159f40 2023-11-28T18:25:08.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.585+0000 7f771bfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f771c0ad680 0x7f771c15da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39402/0 (socket says 172.21.15.105:39402) 2023-11-28T18:25:08.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.585+0000 7f771bfff640 1 -- 172.21.15.105:0/2805907365 learned_addr learned my addr 172.21.15.105:0/2805907365 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:08.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.585+0000 7f771affd640 1 --2- 172.21.15.105:0/2805907365 >> v2:172.21.15.148:3300/0 conn(0x7f771c0cc760 0x7f771c0ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:08.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.585+0000 7f771b7fe640 1 --2- 172.21.15.105:0/2805907365 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f771c159f40 0x7f771c0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:08.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.585+0000 7f771a7fc640 1 -- 172.21.15.105:0/2805907365 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1231240173 0 0) 0x7f771c06bf50 con 0x7f771c0ad680 2023-11-28T18:25:08.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.585+0000 7f771a7fc640 1 -- 172.21.15.105:0/2805907365 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7704003530 con 0x7f771c0ad680 2023-11-28T18:25:08.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.585+0000 7f771affd640 1 -- 172.21.15.105:0/2805907365 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f771c159f40 msgr2=0x7f771c0caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:08.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.585+0000 7f771affd640 1 --2- 172.21.15.105:0/2805907365 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f771c159f40 0x7f771c0caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:08.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.585+0000 7f771affd640 1 -- 172.21.15.105:0/2805907365 >> v1:172.21.15.105:6789/0 conn(0x7f771c0ad680 legacy=0x7f771c15da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:08.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.585+0000 7f771a7fc640 1 -- 172.21.15.105:0/2805907365 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3590027129 0 0) 0x7f7704003530 con 0x7f771c0ad680 2023-11-28T18:25:08.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.585+0000 7f771affd640 1 -- 172.21.15.105:0/2805907365 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771c071380 con 0x7f771c0cc760 2023-11-28T18:25:08.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.589+0000 7f771affd640 1 --2- 172.21.15.105:0/2805907365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f771c0cc760 0x7f771c0ad140 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f77100097c0 tx=0x7f771001c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e30c83a488fef615 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:08.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.589+0000 7f771a7fc640 1 -- 172.21.15.105:0/2805907365 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7714001a00 con 0x7f771c0cc760 2023-11-28T18:25:08.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.589+0000 7f771a7fc640 1 -- 172.21.15.105:0/2805907365 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77100a7050 con 0x7f771c0cc760 2023-11-28T18:25:08.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.589+0000 7f771a7fc640 1 -- 172.21.15.105:0/2805907365 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77100ab620 con 0x7f771c0cc760 2023-11-28T18:25:08.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.589+0000 7f7721c9c640 1 -- 172.21.15.105:0/2805907365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f771c0cc760 msgr2=0x7f771c0ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:08.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.589+0000 7f7721c9c640 1 --2- 172.21.15.105:0/2805907365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f771c0cc760 0x7f771c0ad140 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f77100097c0 tx=0x7f771001c2e0 comp rx=0 tx=0).stop 2023-11-28T18:25:08.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.589+0000 7f7721c9c640 1 -- 172.21.15.105:0/2805907365 shutdown_connections 2023-11-28T18:25:08.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.589+0000 7f7721c9c640 1 --2- 172.21.15.105:0/2805907365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f771c0cc760 0x7f771c0ad140 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:08.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.589+0000 7f7721c9c640 1 --2- 172.21.15.105:0/2805907365 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f771c159f40 0x7f771c0caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:08.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.589+0000 7f7721c9c640 1 -- 172.21.15.105:0/2805907365 >> 172.21.15.105:0/2805907365 conn(0x7f771c0ceb90 msgr2=0x7f771c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:08.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.589+0000 7f7721c9c640 1 -- 172.21.15.105:0/2805907365 shutdown_connections 2023-11-28T18:25:08.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.589+0000 7f7721c9c640 1 -- 172.21.15.105:0/2805907365 wait complete. 2023-11-28T18:25:08.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.589+0000 7f7721c9c640 1 Processor -- start 2023-11-28T18:25:08.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f7721c9c640 1 -- start start 2023-11-28T18:25:08.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f7721c9c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f771c0ad680 0x7f771c15ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:08.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f7721c9c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f771c0cc760 0x7f771c15b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:08.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f7721c9c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f771c159f40 0x7f771c152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:08.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f7721c9c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771c15bb90 con 0x7f771c159f40 2023-11-28T18:25:08.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f7721c9c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771c13f950 con 0x7f771c0cc760 2023-11-28T18:25:08.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f7721c9c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771c15be10 con 0x7f771c0ad680 2023-11-28T18:25:08.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f771bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f771c159f40 0x7f771c152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:08.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f771bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f771c159f40 0x7f771c152790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53972/0 (socket says 172.21.15.105:53972) 2023-11-28T18:25:08.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f771bfff640 1 -- 172.21.15.105:0/2772410933 learned_addr learned my addr 172.21.15.105:0/2772410933 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:08.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f771b7fe640 1 --2- 172.21.15.105:0/2772410933 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f771c0ad680 0x7f771c15ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:08.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f771affd640 1 --2- 172.21.15.105:0/2772410933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f771c0cc760 0x7f771c15b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:08.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f771bfff640 1 -- 172.21.15.105:0/2772410933 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f771c0ad680 msgr2=0x7f771c15ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:08.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f771bfff640 1 --2- 172.21.15.105:0/2772410933 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f771c0ad680 0x7f771c15ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:08.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f771bfff640 1 -- 172.21.15.105:0/2772410933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f771c0cc760 msgr2=0x7f771c15b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:08.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f771bfff640 1 --2- 172.21.15.105:0/2772410933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f771c0cc760 0x7f771c15b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:08.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f771bfff640 1 -- 172.21.15.105:0/2772410933 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7710009420 con 0x7f771c159f40 2023-11-28T18:25:08.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f771bfff640 1 --2- 172.21.15.105:0/2772410933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f771c159f40 0x7f771c152790 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f771400cc70 tx=0x7f771400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:08.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f7703fff640 1 -- 172.21.15.105:0/2772410933 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771400ce80 con 0x7f771c159f40 2023-11-28T18:25:08.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f7703fff640 1 -- 172.21.15.105:0/2772410933 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7714012400 con 0x7f771c159f40 2023-11-28T18:25:08.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f7721c9c640 1 -- 172.21.15.105:0/2772410933 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f771c15c0f0 con 0x7f771c159f40 2023-11-28T18:25:08.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f7703fff640 1 -- 172.21.15.105:0/2772410933 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77140127b0 con 0x7f771c159f40 2023-11-28T18:25:08.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f7721c9c640 1 -- 172.21.15.105:0/2772410933 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f771c15c6b0 con 0x7f771c159f40 2023-11-28T18:25:08.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.593+0000 7f7721c9c640 1 -- 172.21.15.105:0/2772410933 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f771c067570 con 0x7f771c159f40 2023-11-28T18:25:08.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.597+0000 7f7703fff640 1 -- 172.21.15.105:0/2772410933 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f7714012910 con 0x7f771c159f40 2023-11-28T18:25:08.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.597+0000 7f7703fff640 1 --2- 172.21.15.105:0/2772410933 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f77080668d0 0x7f7708068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:08.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.597+0000 7f7703fff640 1 -- 172.21.15.105:0/2772410933 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7714084b50 con 0x7f771c159f40 2023-11-28T18:25:08.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.597+0000 7f771b7fe640 1 --2- 172.21.15.105:0/2772410933 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f77080668d0 0x7f7708068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:08.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.597+0000 7f771b7fe640 1 --2- 172.21.15.105:0/2772410933 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f77080668d0 0x7f7708068d90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f770c009ac0 tx=0x7f770c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:08.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.601+0000 7f7703fff640 1 -- 172.21.15.105:0/2772410933 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7714051cf0 con 0x7f771c159f40 2023-11-28T18:25:08.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.749+0000 7f7721c9c640 1 -- 172.21.15.105:0/2772410933 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random}] v 0) v1 -- 0x7f771c1572a0 con 0x7f771c159f40 2023-11-28T18:25:08.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.753+0000 7f7703fff640 1 -- 172.21.15.105:0/2772410933 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random}]=0 v11)=0 v11) v1 ==== 128+0+0 (secure 0 0 0) 0x7f7714055310 con 0x7f771c159f40 2023-11-28T18:25:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.757+0000 7f7721c9c640 1 -- 172.21.15.105:0/2772410933 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f77080668d0 msgr2=0x7f7708068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.757+0000 7f7721c9c640 1 --2- 172.21.15.105:0/2772410933 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f77080668d0 0x7f7708068d90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f770c009ac0 tx=0x7f770c009190 comp rx=0 tx=0).stop 2023-11-28T18:25:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.757+0000 7f7721c9c640 1 -- 172.21.15.105:0/2772410933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f771c159f40 msgr2=0x7f771c152790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.757+0000 7f7721c9c640 1 --2- 172.21.15.105:0/2772410933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f771c159f40 0x7f771c152790 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f771400cc70 tx=0x7f771400a430 comp rx=0 tx=0).stop 2023-11-28T18:25:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.757+0000 7f7721c9c640 1 -- 172.21.15.105:0/2772410933 shutdown_connections 2023-11-28T18:25:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.757+0000 7f7721c9c640 1 --2- 172.21.15.105:0/2772410933 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f77080668d0 0x7f7708068d90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.757+0000 7f7721c9c640 1 --2- 172.21.15.105:0/2772410933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f771c159f40 0x7f771c152790 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.757+0000 7f7721c9c640 1 --2- 172.21.15.105:0/2772410933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f771c0cc760 0x7f771c15b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.757+0000 7f7721c9c640 1 --2- 172.21.15.105:0/2772410933 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f771c0ad680 0x7f771c15ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.757+0000 7f7721c9c640 1 -- 172.21.15.105:0/2772410933 >> 172.21.15.105:0/2772410933 conn(0x7f771c0ceb90 msgr2=0x7f771c06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.757+0000 7f7721c9c640 1 -- 172.21.15.105:0/2772410933 shutdown_connections 2023-11-28T18:25:08.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.757+0000 7f7721c9c640 1 -- 172.21.15.105:0/2772410933 wait complete. 2023-11-28T18:25:08.782 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_distributed True 2023-11-28T18:25:08.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.917+0000 7f0d141aa640 1 Processor -- start 2023-11-28T18:25:08.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.917+0000 7f0d141aa640 1 -- start start 2023-11-28T18:25:08.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.917+0000 7f0d141aa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0d0c0b4290 0x7f0d0c0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:08.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.917+0000 7f0d141aa640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d0c0b4bb0 con 0x7f0d0c071960 2023-11-28T18:25:08.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.917+0000 7f0d141aa640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d0c0701d0 con 0x7f0d0c15b030 2023-11-28T18:25:08.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.917+0000 7f0d141aa640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d0c06df80 con 0x7f0d0c0b4290 2023-11-28T18:25:08.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.917+0000 7f0d11745640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0d0c071960 0x7f0d0c15f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39410/0 (socket says 172.21.15.105:39410) 2023-11-28T18:25:08.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.917+0000 7f0d11745640 1 -- 172.21.15.105:0/2869753779 learned_addr learned my addr 172.21.15.105:0/2869753779 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:08.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.917+0000 7f0d11f46640 1 --2- 172.21.15.105:0/2869753779 >> v2:172.21.15.105:3300/0 conn(0x7f0d0c0b4290 0x7f0d0c0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:08.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.917+0000 7f0d10f44640 1 -- 172.21.15.105:0/2869753779 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3375840545 0 0) 0x7f0d0c0b4bb0 con 0x7f0d0c071960 2023-11-28T18:25:08.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.917+0000 7f0d10f44640 1 -- 172.21.15.105:0/2869753779 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cf8003530 con 0x7f0d0c071960 2023-11-28T18:25:08.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.917+0000 7f0d11f46640 1 -- 172.21.15.105:0/2869753779 >> v1:172.21.15.148:6789/0 conn(0x7f0d0c15b030 legacy=0x7f0d0c0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:08.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.917+0000 7f0d11f46640 1 -- 172.21.15.105:0/2869753779 >> v1:172.21.15.105:6789/0 conn(0x7f0d0c071960 legacy=0x7f0d0c15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:08.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.917+0000 7f0d11f46640 1 -- 172.21.15.105:0/2869753779 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0c071df0 con 0x7f0d0c0b4290 2023-11-28T18:25:08.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.921+0000 7f0d11f46640 1 --2- 172.21.15.105:0/2869753779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d0c0b4290 0x7f0d0c0b4670 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0d080097c0 tx=0x7f0d0801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61ddeaded98304b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:08.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.921+0000 7f0d10f44640 1 -- 172.21.15.105:0/2869753779 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d080ab650 con 0x7f0d0c0b4290 2023-11-28T18:25:08.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.921+0000 7f0d10f44640 1 -- 172.21.15.105:0/2869753779 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d080ab7b0 con 0x7f0d0c0b4290 2023-11-28T18:25:08.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.921+0000 7f0d10f44640 1 -- 172.21.15.105:0/2869753779 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d080abb40 con 0x7f0d0c0b4290 2023-11-28T18:25:08.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.921+0000 7f0d141aa640 1 -- 172.21.15.105:0/2869753779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d0c0b4290 msgr2=0x7f0d0c0b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:08.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.921+0000 7f0d141aa640 1 --2- 172.21.15.105:0/2869753779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d0c0b4290 0x7f0d0c0b4670 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0d080097c0 tx=0x7f0d0801c450 comp rx=0 tx=0).stop 2023-11-28T18:25:08.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.921+0000 7f0d141aa640 1 -- 172.21.15.105:0/2869753779 shutdown_connections 2023-11-28T18:25:08.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.921+0000 7f0d141aa640 1 --2- 172.21.15.105:0/2869753779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d0c0b4290 0x7f0d0c0b4670 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:08.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.921+0000 7f0d141aa640 1 -- 172.21.15.105:0/2869753779 >> 172.21.15.105:0/2869753779 conn(0x7f0d0c0ceb90 msgr2=0x7f0d0c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:08.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.921+0000 7f0d141aa640 1 -- 172.21.15.105:0/2869753779 shutdown_connections 2023-11-28T18:25:08.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.921+0000 7f0d141aa640 1 -- 172.21.15.105:0/2869753779 wait complete. 2023-11-28T18:25:08.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d141aa640 1 Processor -- start 2023-11-28T18:25:08.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d141aa640 1 -- start start 2023-11-28T18:25:08.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d141aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d0c0b4290 0x7f0d0c099ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:08.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d141aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d0c15b030 0x7f0d0c159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:08.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d141aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d0c155240 0x7f0d0c156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:08.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d141aa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c159590 con 0x7f0d0c155240 2023-11-28T18:25:08.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d141aa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c165c00 con 0x7f0d0c15b030 2023-11-28T18:25:08.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d141aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c165640 con 0x7f0d0c0b4290 2023-11-28T18:25:08.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d12747640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d0c155240 0x7f0d0c156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:08.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d12747640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d0c155240 0x7f0d0c156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53990/0 (socket says 172.21.15.105:53990) 2023-11-28T18:25:08.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d12747640 1 -- 172.21.15.105:0/3296158084 learned_addr learned my addr 172.21.15.105:0/3296158084 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:08.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d11f46640 1 --2- 172.21.15.105:0/3296158084 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d0c0b4290 0x7f0d0c099ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:08.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d11745640 1 --2- 172.21.15.105:0/3296158084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d0c15b030 0x7f0d0c159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:08.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d12747640 1 -- 172.21.15.105:0/3296158084 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d0c0b4290 msgr2=0x7f0d0c099ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:08.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d12747640 1 --2- 172.21.15.105:0/3296158084 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d0c0b4290 0x7f0d0c099ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:08.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d12747640 1 -- 172.21.15.105:0/3296158084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d0c15b030 msgr2=0x7f0d0c159050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:08.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d12747640 1 --2- 172.21.15.105:0/3296158084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d0c15b030 0x7f0d0c159050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:08.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d12747640 1 -- 172.21.15.105:0/3296158084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d08009420 con 0x7f0d0c155240 2023-11-28T18:25:08.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d11745640 1 --2- 172.21.15.105:0/3296158084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d0c15b030 0x7f0d0c159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:08.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d12747640 1 --2- 172.21.15.105:0/3296158084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d0c155240 0x7f0d0c156160 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400c730 tx=0x7f0d0400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:08.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d027fc640 1 -- 172.21.15.105:0/3296158084 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d04003b80 con 0x7f0d0c155240 2023-11-28T18:25:08.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d027fc640 1 -- 172.21.15.105:0/3296158084 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d04003ce0 con 0x7f0d0c155240 2023-11-28T18:25:08.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d141aa640 1 -- 172.21.15.105:0/3296158084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d0c165920 con 0x7f0d0c155240 2023-11-28T18:25:08.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d027fc640 1 -- 172.21.15.105:0/3296158084 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d04005310 con 0x7f0d0c155240 2023-11-28T18:25:08.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d141aa640 1 -- 172.21.15.105:0/3296158084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0c158190 con 0x7f0d0c155240 2023-11-28T18:25:08.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.925+0000 7f0d141aa640 1 -- 172.21.15.105:0/3296158084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d0c157cd0 con 0x7f0d0c155240 2023-11-28T18:25:08.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.929+0000 7f0d027fc640 1 -- 172.21.15.105:0/3296158084 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f0d04005680 con 0x7f0d0c155240 2023-11-28T18:25:08.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.929+0000 7f0d027fc640 1 --2- 172.21.15.105:0/3296158084 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cf0066900 0x7f0cf0068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:08.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.929+0000 7f0d027fc640 1 -- 172.21.15.105:0/3296158084 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0d04084420 con 0x7f0d0c155240 2023-11-28T18:25:08.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.929+0000 7f0d11f46640 1 --2- 172.21.15.105:0/3296158084 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cf0066900 0x7f0cf0068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:08.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.929+0000 7f0d11f46640 1 --2- 172.21.15.105:0/3296158084 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cf0066900 0x7f0cf0068dc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0d080af000 tx=0x7f0d080a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:08.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:08.933+0000 7f0d027fc640 1 -- 172.21.15.105:0/3296158084 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0d04051510 con 0x7f0d0c155240 2023-11-28T18:25:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.085+0000 7f0d141aa640 1 -- 172.21.15.105:0/3296158084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_distributed}] v 0) v1 -- 0x7f0d0c067570 con 0x7f0d0c155240 2023-11-28T18:25:09.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.089+0000 7f0d027fc640 1 -- 172.21.15.105:0/3296158084 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_distributed}]=0 v12)=0 v12) v1 ==== 133+0+0 (secure 0 0 0) 0x7f0d04054b30 con 0x7f0d0c155240 2023-11-28T18:25:09.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.089+0000 7f0d141aa640 1 -- 172.21.15.105:0/3296158084 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cf0066900 msgr2=0x7f0cf0068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:09.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.089+0000 7f0d141aa640 1 --2- 172.21.15.105:0/3296158084 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cf0066900 0x7f0cf0068dc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0d080af000 tx=0x7f0d080a7000 comp rx=0 tx=0).stop 2023-11-28T18:25:09.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.089+0000 7f0d141aa640 1 -- 172.21.15.105:0/3296158084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d0c155240 msgr2=0x7f0d0c156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:09.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.089+0000 7f0d141aa640 1 --2- 172.21.15.105:0/3296158084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d0c155240 0x7f0d0c156160 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400c730 tx=0x7f0d0400cbf0 comp rx=0 tx=0).stop 2023-11-28T18:25:09.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.089+0000 7f0d141aa640 1 -- 172.21.15.105:0/3296158084 shutdown_connections 2023-11-28T18:25:09.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.089+0000 7f0d141aa640 1 --2- 172.21.15.105:0/3296158084 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cf0066900 0x7f0cf0068dc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:09.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.089+0000 7f0d141aa640 1 --2- 172.21.15.105:0/3296158084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d0c155240 0x7f0d0c156160 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:09.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.089+0000 7f0d141aa640 1 --2- 172.21.15.105:0/3296158084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d0c15b030 0x7f0d0c159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:09.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.089+0000 7f0d141aa640 1 --2- 172.21.15.105:0/3296158084 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d0c0b4290 0x7f0d0c099ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:09.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.089+0000 7f0d141aa640 1 -- 172.21.15.105:0/3296158084 >> 172.21.15.105:0/3296158084 conn(0x7f0d0c0ceb90 msgr2=0x7f0d0c0ad4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:09.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.089+0000 7f0d141aa640 1 -- 172.21.15.105:0/3296158084 shutdown_connections 2023-11-28T18:25:09.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.089+0000 7f0d141aa640 1 -- 172.21.15.105:0/3296158084 wait complete. 2023-11-28T18:25:09.116 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random_max 1.0 2023-11-28T18:25:09.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.249+0000 7fab0849a640 1 Processor -- start 2023-11-28T18:25:09.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.249+0000 7fab0849a640 1 -- start start 2023-11-28T18:25:09.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.249+0000 7fab0849a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fab0015d310 0x7fab0006c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:09.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.253+0000 7fab0849a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fab00070940 0x7fab00070db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:09.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.253+0000 7fab0849a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab000c9be0 con 0x7fab000c9720 2023-11-28T18:25:09.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.253+0000 7fab0849a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab000cbca0 con 0x7fab00070940 2023-11-28T18:25:09.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.253+0000 7fab0849a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab000cbf80 con 0x7fab0015d310 2023-11-28T18:25:09.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.253+0000 7fab05a35640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fab000c9720 0x7fab0006cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39416/0 (socket says 172.21.15.105:39416) 2023-11-28T18:25:09.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.253+0000 7fab06a37640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fab00070940 0x7fab00070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:09.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.253+0000 7fab05a35640 1 -- 172.21.15.105:0/4277102820 learned_addr learned my addr 172.21.15.105:0/4277102820 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:09.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.253+0000 7fab06236640 1 --2- 172.21.15.105:0/4277102820 >> v2:172.21.15.148:3300/0 conn(0x7fab0015d310 0x7fab0006c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:09.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.253+0000 7fab06a37640 1 -- 172.21.15.105:0/4277102820 >> v2:172.21.15.148:3300/0 conn(0x7fab0015d310 msgr2=0x7fab0006c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:09.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.253+0000 7fab06a37640 1 --2- 172.21.15.105:0/4277102820 >> v2:172.21.15.148:3300/0 conn(0x7fab0015d310 0x7fab0006c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:09.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.253+0000 7fab05234640 1 -- 172.21.15.105:0/4277102820 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 171952553 0 0) 0x7fab000c9be0 con 0x7fab000c9720 2023-11-28T18:25:09.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.253+0000 7fab06a37640 1 -- 172.21.15.105:0/4277102820 >> v1:172.21.15.105:6789/0 conn(0x7fab000c9720 legacy=0x7fab0006cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:09.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.253+0000 7fab06a37640 1 -- 172.21.15.105:0/4277102820 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab000cc260 con 0x7fab00070940 2023-11-28T18:25:09.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.253+0000 7fab06a37640 1 --2- 172.21.15.105:0/4277102820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab00070940 0x7fab00070db0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fab000b1ce0 tx=0x7faaf801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33da8e37d429505 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:09.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.253+0000 7fab05234640 1 -- 172.21.15.105:0/4277102820 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf80a58d0 con 0x7fab00070940 2023-11-28T18:25:09.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.257+0000 7fab05234640 1 -- 172.21.15.105:0/4277102820 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf80a5a30 con 0x7fab00070940 2023-11-28T18:25:09.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.257+0000 7fab05234640 1 -- 172.21.15.105:0/4277102820 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf80a5dc0 con 0x7fab00070940 2023-11-28T18:25:09.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.257+0000 7fab0849a640 1 -- 172.21.15.105:0/4277102820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab00070940 msgr2=0x7fab00070db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:09.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.257+0000 7fab0849a640 1 --2- 172.21.15.105:0/4277102820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab00070940 0x7fab00070db0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fab000b1ce0 tx=0x7faaf801c8e0 comp rx=0 tx=0).stop 2023-11-28T18:25:09.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.257+0000 7fab0849a640 1 -- 172.21.15.105:0/4277102820 shutdown_connections 2023-11-28T18:25:09.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.257+0000 7fab0849a640 1 --2- 172.21.15.105:0/4277102820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab00070940 0x7fab00070db0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:09.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.257+0000 7fab0849a640 1 --2- 172.21.15.105:0/4277102820 >> v2:172.21.15.148:3300/0 conn(0x7fab0015d310 0x7fab0006c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:09.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.257+0000 7fab0849a640 1 -- 172.21.15.105:0/4277102820 >> 172.21.15.105:0/4277102820 conn(0x7fab000b0c60 msgr2=0x7fab000b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:09.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.257+0000 7fab0849a640 1 -- 172.21.15.105:0/4277102820 shutdown_connections 2023-11-28T18:25:09.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.257+0000 7fab0849a640 1 -- 172.21.15.105:0/4277102820 wait complete. 2023-11-28T18:25:09.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.257+0000 7fab0849a640 1 Processor -- start 2023-11-28T18:25:09.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.257+0000 7fab0849a640 1 -- start start 2023-11-28T18:25:09.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.257+0000 7fab0849a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab00070940 0x7fab00154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:09.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.257+0000 7fab0849a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab000c9720 0x7fab00153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:09.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.257+0000 7fab0849a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0015d310 0x7fab00155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:09.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.257+0000 7fab0849a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab000cca40 con 0x7fab000c9720 2023-11-28T18:25:09.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.257+0000 7fab0849a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab000c7240 con 0x7fab0015d310 2023-11-28T18:25:09.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.257+0000 7fab0849a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab000c7520 con 0x7fab00070940 2023-11-28T18:25:09.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.257+0000 7fab05a35640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab000c9720 0x7fab00153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:09.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.257+0000 7fab05a35640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab000c9720 0x7fab00153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54000/0 (socket says 172.21.15.105:54000) 2023-11-28T18:25:09.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.257+0000 7fab05a35640 1 -- 172.21.15.105:0/119672105 learned_addr learned my addr 172.21.15.105:0/119672105 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:09.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.261+0000 7fab06236640 1 --2- 172.21.15.105:0/119672105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab00070940 0x7fab00154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:09.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.261+0000 7fab06a37640 1 --2- 172.21.15.105:0/119672105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0015d310 0x7fab00155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:09.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.261+0000 7fab05a35640 1 -- 172.21.15.105:0/119672105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab00070940 msgr2=0x7fab00154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:09.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.261+0000 7fab05a35640 1 --2- 172.21.15.105:0/119672105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab00070940 0x7fab00154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:09.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.261+0000 7fab05a35640 1 -- 172.21.15.105:0/119672105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0015d310 msgr2=0x7fab00155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:09.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.261+0000 7fab05a35640 1 --2- 172.21.15.105:0/119672105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0015d310 0x7fab00155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:09.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.261+0000 7fab05a35640 1 -- 172.21.15.105:0/119672105 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf8009a70 con 0x7fab000c9720 2023-11-28T18:25:09.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.261+0000 7fab05a35640 1 --2- 172.21.15.105:0/119672105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab000c9720 0x7fab00153150 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7faaf00011f0 tx=0x7faaf000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:09.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.261+0000 7faaf6ffd640 1 -- 172.21.15.105:0/119672105 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf00074e0 con 0x7fab000c9720 2023-11-28T18:25:09.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.261+0000 7faaf6ffd640 1 -- 172.21.15.105:0/119672105 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf0007640 con 0x7fab000c9720 2023-11-28T18:25:09.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.261+0000 7faaf6ffd640 1 -- 172.21.15.105:0/119672105 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf0004a10 con 0x7fab000c9720 2023-11-28T18:25:09.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.261+0000 7fab0849a640 1 -- 172.21.15.105:0/119672105 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab000c7800 con 0x7fab000c9720 2023-11-28T18:25:09.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.261+0000 7fab0849a640 1 -- 172.21.15.105:0/119672105 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab000b1440 con 0x7fab000c9720 2023-11-28T18:25:09.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.261+0000 7fab0849a640 1 -- 172.21.15.105:0/119672105 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab000c7cb0 con 0x7fab000c9720 2023-11-28T18:25:09.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.261+0000 7faaf6ffd640 1 -- 172.21.15.105:0/119672105 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7faaf0004b70 con 0x7fab000c9720 2023-11-28T18:25:09.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.261+0000 7faaf6ffd640 1 --2- 172.21.15.105:0/119672105 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faae40668d0 0x7faae4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:09.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.261+0000 7faaf6ffd640 1 -- 172.21.15.105:0/119672105 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7faaf0083890 con 0x7fab000c9720 2023-11-28T18:25:09.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.261+0000 7fab06236640 1 --2- 172.21.15.105:0/119672105 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faae40668d0 0x7faae4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:09.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.265+0000 7fab06236640 1 --2- 172.21.15.105:0/119672105 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faae40668d0 0x7faae4068d90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fab00156a80 tx=0x7faafc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:09.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.265+0000 7faaf6ffd640 1 -- 172.21.15.105:0/119672105 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faaf0050980 con 0x7fab000c9720 2023-11-28T18:25:09.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.429+0000 7fab0849a640 1 -- 172.21.15.105:0/119672105 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random_max}] v 0) v1 -- 0x7fab00067570 con 0x7fab000c9720 2023-11-28T18:25:09.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.437+0000 7faaf6ffd640 1 -- 172.21.15.105:0/119672105 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random_max}]=0 v13)=0 v13) v1 ==== 131+0+0 (secure 0 0 0) 0x7faaf0018020 con 0x7fab000c9720 2023-11-28T18:25:09.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.437+0000 7fab0849a640 1 -- 172.21.15.105:0/119672105 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faae40668d0 msgr2=0x7faae4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:09.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.437+0000 7fab0849a640 1 --2- 172.21.15.105:0/119672105 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faae40668d0 0x7faae4068d90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fab00156a80 tx=0x7faafc008040 comp rx=0 tx=0).stop 2023-11-28T18:25:09.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.437+0000 7fab0849a640 1 -- 172.21.15.105:0/119672105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab000c9720 msgr2=0x7fab00153150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:09.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.437+0000 7fab0849a640 1 --2- 172.21.15.105:0/119672105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab000c9720 0x7fab00153150 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7faaf00011f0 tx=0x7faaf000c9b0 comp rx=0 tx=0).stop 2023-11-28T18:25:09.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.437+0000 7fab0849a640 1 -- 172.21.15.105:0/119672105 shutdown_connections 2023-11-28T18:25:09.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.437+0000 7fab0849a640 1 --2- 172.21.15.105:0/119672105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0015d310 0x7fab00155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:09.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.437+0000 7fab0849a640 1 --2- 172.21.15.105:0/119672105 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faae40668d0 0x7faae4068d90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:09.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.437+0000 7fab0849a640 1 --2- 172.21.15.105:0/119672105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab000c9720 0x7fab00153150 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:09.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.437+0000 7fab0849a640 1 --2- 172.21.15.105:0/119672105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab00070940 0x7fab00154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:09.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.437+0000 7fab0849a640 1 -- 172.21.15.105:0/119672105 >> 172.21.15.105:0/119672105 conn(0x7fab000b0c60 msgr2=0x7fab000b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:09.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.437+0000 7fab0849a640 1 -- 172.21.15.105:0/119672105 shutdown_connections 2023-11-28T18:25:09.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:09.437+0000 7fab0849a640 1 -- 172.21.15.105:0/119672105 wait complete. 2023-11-28T18:25:09.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:25:09.463 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-28T18:25:09.463 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:25:09.463 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:25:09.463 DEBUG:teuthology.orchestra.run.smithi105:> # Use up a random number of inode numbers so the ephemeral pinning is not the same every test. 2023-11-28T18:25:09.463 DEBUG:teuthology.orchestra.run.smithi105:> mkdir .inode_number_thrash 2023-11-28T18:25:09.463 DEBUG:teuthology.orchestra.run.smithi105:> count=$((RANDOM % 1024)) 2023-11-28T18:25:09.464 DEBUG:teuthology.orchestra.run.smithi105:> for ((i = 0; i < count; i++)); do touch .inode_number_thrash/$i; done 2023-11-28T18:25:09.464 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf .inode_number_thrash 2023-11-28T18:25:09.464 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:25:16.206 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-11-28T18:25:16.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.341+0000 7fa7d365f640 1 Processor -- start 2023-11-28T18:25:16.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.341+0000 7fa7d365f640 1 -- start start 2023-11-28T18:25:16.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.345+0000 7fa7d365f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa7cc0ce2c0 0x7fa7cc06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:16.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.345+0000 7fa7d365f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7cc06cd20 0x7fa7cc0abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:16.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.345+0000 7fa7d365f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7cc0cde30 con 0x7fa7cc159610 2023-11-28T18:25:16.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.345+0000 7fa7d365f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7cc06d1b0 con 0x7fa7cc0ce2c0 2023-11-28T18:25:16.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.345+0000 7fa7d365f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc1599f0 con 0x7fa7cc06cd20 2023-11-28T18:25:16.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.345+0000 7fa7d0bfa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa7cc0ce2c0 0x7fa7cc06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:16.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.345+0000 7fa7d0bfa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa7cc0ce2c0 0x7fa7cc06c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40416/0 (socket says 172.21.15.105:40416) 2023-11-28T18:25:16.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.345+0000 7fa7d0bfa640 1 -- 172.21.15.105:0/1919742407 learned_addr learned my addr 172.21.15.105:0/1919742407 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:16.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.345+0000 7fa7d1bfc640 1 --2- 172.21.15.105:0/1919742407 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7cc06cd20 0x7fa7cc0abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:16.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.345+0000 7fa7d0bfa640 1 -- 172.21.15.105:0/1919742407 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7cc06cd20 msgr2=0x7fa7cc0abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:16.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.345+0000 7fa7d0bfa640 1 --2- 172.21.15.105:0/1919742407 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7cc06cd20 0x7fa7cc0abd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:16.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.345+0000 7fa7d0bfa640 1 -- 172.21.15.105:0/1919742407 >> v1:172.21.15.148:6789/0 conn(0x7fa7cc159610 legacy=0x7fa7cc0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:16.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.345+0000 7fa7d0bfa640 1 -- 172.21.15.105:0/1919742407 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7cc06de20 con 0x7fa7cc0ce2c0 2023-11-28T18:25:16.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.345+0000 7fa7d1bfc640 1 --2- 172.21.15.105:0/1919742407 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7cc06cd20 0x7fa7cc0abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:16.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.345+0000 7fa7d0bfa640 1 --2- 172.21.15.105:0/1919742407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7cc0ce2c0 0x7fa7cc06c7e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009870 tx=0x7fa7bc01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899d841a622417dd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:16.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.345+0000 7fa7c3fff640 1 -- 172.21.15.105:0/1919742407 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc0ad680 con 0x7fa7cc0ce2c0 2023-11-28T18:25:16.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.345+0000 7fa7c3fff640 1 -- 172.21.15.105:0/1919742407 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7bc0ad7e0 con 0x7fa7cc0ce2c0 2023-11-28T18:25:16.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.345+0000 7fa7c3fff640 1 -- 172.21.15.105:0/1919742407 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc0adb70 con 0x7fa7cc0ce2c0 2023-11-28T18:25:16.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.349+0000 7fa7d365f640 1 -- 172.21.15.105:0/1919742407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7cc0ce2c0 msgr2=0x7fa7cc06c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:16.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.349+0000 7fa7d365f640 1 --2- 172.21.15.105:0/1919742407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7cc0ce2c0 0x7fa7cc06c7e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009870 tx=0x7fa7bc01c510 comp rx=0 tx=0).stop 2023-11-28T18:25:16.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.349+0000 7fa7d365f640 1 -- 172.21.15.105:0/1919742407 shutdown_connections 2023-11-28T18:25:16.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.349+0000 7fa7d365f640 1 --2- 172.21.15.105:0/1919742407 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7cc06cd20 0x7fa7cc0abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:16.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.349+0000 7fa7d365f640 1 --2- 172.21.15.105:0/1919742407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7cc0ce2c0 0x7fa7cc06c7e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:16.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.349+0000 7fa7d365f640 1 -- 172.21.15.105:0/1919742407 >> 172.21.15.105:0/1919742407 conn(0x7fa7cc0ceb90 msgr2=0x7fa7cc0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:16.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.349+0000 7fa7d365f640 1 -- 172.21.15.105:0/1919742407 shutdown_connections 2023-11-28T18:25:16.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.349+0000 7fa7d365f640 1 -- 172.21.15.105:0/1919742407 wait complete. 2023-11-28T18:25:16.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.349+0000 7fa7d365f640 1 Processor -- start 2023-11-28T18:25:16.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.349+0000 7fa7d365f640 1 -- start start 2023-11-28T18:25:16.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.349+0000 7fa7d365f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7cc06cd20 0x7fa7cc14df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:16.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.349+0000 7fa7d365f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7cc0ce2c0 0x7fa7cc14fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:16.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.349+0000 7fa7d365f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7cc159610 0x7fa7cc151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:16.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.349+0000 7fa7d365f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc1514c0 con 0x7fa7cc06cd20 2023-11-28T18:25:16.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.349+0000 7fa7d365f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc1508b0 con 0x7fa7cc159610 2023-11-28T18:25:16.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.349+0000 7fa7d365f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc150b90 con 0x7fa7cc0ce2c0 2023-11-28T18:25:16.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.349+0000 7fa7d13fb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7cc06cd20 0x7fa7cc14df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:16.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.349+0000 7fa7d13fb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7cc06cd20 0x7fa7cc14df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40432/0 (socket says 172.21.15.105:40432) 2023-11-28T18:25:16.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.349+0000 7fa7d13fb640 1 -- 172.21.15.105:0/2409640902 learned_addr learned my addr 172.21.15.105:0/2409640902 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:16.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.349+0000 7fa7d0bfa640 1 --2- 172.21.15.105:0/2409640902 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7cc0ce2c0 0x7fa7cc14fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:16.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.349+0000 7fa7d1bfc640 1 --2- 172.21.15.105:0/2409640902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7cc159610 0x7fa7cc151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:16.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.353+0000 7fa7d13fb640 1 -- 172.21.15.105:0/2409640902 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7cc0ce2c0 msgr2=0x7fa7cc14fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:16.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.353+0000 7fa7d13fb640 1 --2- 172.21.15.105:0/2409640902 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7cc0ce2c0 0x7fa7cc14fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:16.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.353+0000 7fa7d13fb640 1 -- 172.21.15.105:0/2409640902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7cc159610 msgr2=0x7fa7cc151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:16.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.353+0000 7fa7d13fb640 1 --2- 172.21.15.105:0/2409640902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7cc159610 0x7fa7cc151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:16.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.353+0000 7fa7d13fb640 1 -- 172.21.15.105:0/2409640902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc0094d0 con 0x7fa7cc06cd20 2023-11-28T18:25:16.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.353+0000 7fa7d13fb640 1 --2- 172.21.15.105:0/2409640902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7cc06cd20 0x7fa7cc14df80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800c500 tx=0x7fa7c800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:16.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.353+0000 7fa7c1ffb640 1 -- 172.21.15.105:0/2409640902 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8005220 con 0x7fa7cc06cd20 2023-11-28T18:25:16.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.353+0000 7fa7c1ffb640 1 -- 172.21.15.105:0/2409640902 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c8005380 con 0x7fa7cc06cd20 2023-11-28T18:25:16.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.353+0000 7fa7d365f640 1 -- 172.21.15.105:0/2409640902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7cc0ccff0 con 0x7fa7cc06cd20 2023-11-28T18:25:16.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.353+0000 7fa7c1ffb640 1 -- 172.21.15.105:0/2409640902 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8007270 con 0x7fa7cc06cd20 2023-11-28T18:25:16.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.353+0000 7fa7d365f640 1 -- 172.21.15.105:0/2409640902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7cc151150 con 0x7fa7cc06cd20 2023-11-28T18:25:16.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.353+0000 7fa7d365f640 1 -- 172.21.15.105:0/2409640902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7cc067570 con 0x7fa7cc06cd20 2023-11-28T18:25:16.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.353+0000 7fa7c1ffb640 1 -- 172.21.15.105:0/2409640902 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa7c80073d0 con 0x7fa7cc06cd20 2023-11-28T18:25:16.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.357+0000 7fa7c1ffb640 1 --2- 172.21.15.105:0/2409640902 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa7b00668d0 0x7fa7b0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:16.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.357+0000 7fa7c1ffb640 1 -- 172.21.15.105:0/2409640902 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa7c8083ee0 con 0x7fa7cc06cd20 2023-11-28T18:25:16.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.357+0000 7fa7d0bfa640 1 --2- 172.21.15.105:0/2409640902 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa7b00668d0 0x7fa7b0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:16.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.357+0000 7fa7d0bfa640 1 --2- 172.21.15.105:0/2409640902 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa7b00668d0 0x7fa7b0068d90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0af000 tx=0x7fa7bc0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:16.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.361+0000 7fa7c1ffb640 1 -- 172.21.15.105:0/2409640902 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa7c8050f30 con 0x7fa7cc06cd20 2023-11-28T18:25:16.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:16.541+0000 7fa7d365f640 1 -- 172.21.15.105:0/2409640902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7fa7cc15c3c0 con 0x7fa7cc06cd20 2023-11-28T18:25:17.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.121+0000 7fa7c1ffb640 1 -- 172.21.15.105:0/2409640902 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7fa7c8054550 con 0x7fa7cc06cd20 2023-11-28T18:25:17.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.121+0000 7fa7d365f640 1 -- 172.21.15.105:0/2409640902 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa7b00668d0 msgr2=0x7fa7b0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:17.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.121+0000 7fa7d365f640 1 --2- 172.21.15.105:0/2409640902 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa7b00668d0 0x7fa7b0068d90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0af000 tx=0x7fa7bc0a7000 comp rx=0 tx=0).stop 2023-11-28T18:25:17.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.121+0000 7fa7d365f640 1 -- 172.21.15.105:0/2409640902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7cc06cd20 msgr2=0x7fa7cc14df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:17.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.121+0000 7fa7d365f640 1 --2- 172.21.15.105:0/2409640902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7cc06cd20 0x7fa7cc14df80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800c500 tx=0x7fa7c800c9c0 comp rx=0 tx=0).stop 2023-11-28T18:25:17.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.121+0000 7fa7d365f640 1 -- 172.21.15.105:0/2409640902 shutdown_connections 2023-11-28T18:25:17.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.121+0000 7fa7d365f640 1 --2- 172.21.15.105:0/2409640902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7cc159610 0x7fa7cc151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:17.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.121+0000 7fa7d365f640 1 --2- 172.21.15.105:0/2409640902 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7cc0ce2c0 0x7fa7cc14fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:17.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.121+0000 7fa7d365f640 1 --2- 172.21.15.105:0/2409640902 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa7b00668d0 0x7fa7b0068d90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:17.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.121+0000 7fa7d365f640 1 --2- 172.21.15.105:0/2409640902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7cc06cd20 0x7fa7cc14df80 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:17.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.121+0000 7fa7d365f640 1 -- 172.21.15.105:0/2409640902 >> 172.21.15.105:0/2409640902 conn(0x7fa7cc0ceb90 msgr2=0x7fa7cc0cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:17.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.121+0000 7fa7d365f640 1 -- 172.21.15.105:0/2409640902 shutdown_connections 2023-11-28T18:25:17.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.121+0000 7fa7d365f640 1 -- 172.21.15.105:0/2409640902 wait complete. 2023-11-28T18:25:17.158 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:25:17.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.289+0000 7fac9f6c9640 1 Processor -- start 2023-11-28T18:25:17.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.293+0000 7fac9f6c9640 1 -- start start 2023-11-28T18:25:17.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.293+0000 7fac9f6c9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fac980b3d30 0x7fac981680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:17.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.293+0000 7fac9f6c9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac980b25b0 con 0x7fac9815d4f0 2023-11-28T18:25:17.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.293+0000 7fac9f6c9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac980d1c30 con 0x7fac980d3c30 2023-11-28T18:25:17.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.293+0000 7fac9f6c9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac980b2900 con 0x7fac980b3d30 2023-11-28T18:25:17.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.293+0000 7fac9d465640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fac9815d4f0 0x7fac980d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54304/0 (socket says 172.21.15.105:54304) 2023-11-28T18:25:17.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.293+0000 7fac9d465640 1 -- 172.21.15.105:0/209184422 learned_addr learned my addr 172.21.15.105:0/209184422 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:17.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.293+0000 7fac9dc66640 1 --2- 172.21.15.105:0/209184422 >> v2:172.21.15.105:3300/0 conn(0x7fac980b3d30 0x7fac981680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:17.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.293+0000 7fac9dc66640 1 -- 172.21.15.105:0/209184422 >> v1:172.21.15.148:6789/0 conn(0x7fac980d3c30 legacy=0x7fac980d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:17.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.293+0000 7fac9dc66640 1 -- 172.21.15.105:0/209184422 >> v1:172.21.15.105:6789/0 conn(0x7fac9815d4f0 legacy=0x7fac980d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:17.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.293+0000 7fac9dc66640 1 -- 172.21.15.105:0/209184422 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac980cea30 con 0x7fac980b3d30 2023-11-28T18:25:17.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.297+0000 7fac9dc66640 1 --2- 172.21.15.105:0/209184422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac980b3d30 0x7fac981680a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fac90009d90 tx=0x7fac9001c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666caa4db6909435 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:17.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.297+0000 7fac8ffff640 1 -- 172.21.15.105:0/209184422 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac900a58c0 con 0x7fac980b3d30 2023-11-28T18:25:17.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.297+0000 7fac8ffff640 1 -- 172.21.15.105:0/209184422 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac900a5a20 con 0x7fac980b3d30 2023-11-28T18:25:17.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.297+0000 7fac8ffff640 1 -- 172.21.15.105:0/209184422 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac900a5db0 con 0x7fac980b3d30 2023-11-28T18:25:17.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.297+0000 7fac9f6c9640 1 -- 172.21.15.105:0/209184422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac980b3d30 msgr2=0x7fac981680a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:17.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.297+0000 7fac9f6c9640 1 --2- 172.21.15.105:0/209184422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac980b3d30 0x7fac981680a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fac90009d90 tx=0x7fac9001c800 comp rx=0 tx=0).stop 2023-11-28T18:25:17.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.297+0000 7fac9f6c9640 1 -- 172.21.15.105:0/209184422 shutdown_connections 2023-11-28T18:25:17.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.297+0000 7fac9f6c9640 1 --2- 172.21.15.105:0/209184422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac980b3d30 0x7fac981680a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:17.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.297+0000 7fac9f6c9640 1 -- 172.21.15.105:0/209184422 >> 172.21.15.105:0/209184422 conn(0x7fac980d3030 msgr2=0x7fac980d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:17.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.297+0000 7fac9f6c9640 1 -- 172.21.15.105:0/209184422 shutdown_connections 2023-11-28T18:25:17.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.297+0000 7fac9f6c9640 1 -- 172.21.15.105:0/209184422 wait complete. 2023-11-28T18:25:17.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.297+0000 7fac9f6c9640 1 Processor -- start 2023-11-28T18:25:17.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac9f6c9640 1 -- start start 2023-11-28T18:25:17.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac9f6c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac980b3d30 0x7fac981555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:17.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac9f6c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac980d3c30 0x7fac981568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:17.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac9f6c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fac9815d4f0 0x7fac98156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:17.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac9f6c9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac980d0a20 con 0x7fac980d3c30 2023-11-28T18:25:17.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac9f6c9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9814cb80 con 0x7fac980b3d30 2023-11-28T18:25:17.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac9f6c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac981563d0 con 0x7fac9815d4f0 2023-11-28T18:25:17.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac9cc64640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac980d3c30 0x7fac981568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:17.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac9cc64640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac980d3c30 0x7fac981568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40446/0 (socket says 172.21.15.105:40446) 2023-11-28T18:25:17.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac9cc64640 1 -- 172.21.15.105:0/1777872676 learned_addr learned my addr 172.21.15.105:0/1777872676 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:17.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac9dc66640 1 --2- 172.21.15.105:0/1777872676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fac9815d4f0 0x7fac98156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:17.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac9d465640 1 --2- 172.21.15.105:0/1777872676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac980b3d30 0x7fac981555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:17.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac9cc64640 1 -- 172.21.15.105:0/1777872676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fac9815d4f0 msgr2=0x7fac98156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:17.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac9cc64640 1 --2- 172.21.15.105:0/1777872676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fac9815d4f0 0x7fac98156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:17.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac9cc64640 1 -- 172.21.15.105:0/1777872676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac980b3d30 msgr2=0x7fac981555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:17.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac9cc64640 1 --2- 172.21.15.105:0/1777872676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac980b3d30 0x7fac981555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:17.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac9cc64640 1 -- 172.21.15.105:0/1777872676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac90009a70 con 0x7fac980d3c30 2023-11-28T18:25:17.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac9cc64640 1 --2- 172.21.15.105:0/1777872676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac980d3c30 0x7fac981568c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fac8800c3f0 tx=0x7fac8800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:17.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac8dffb640 1 -- 172.21.15.105:0/1777872676 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac880052d0 con 0x7fac980d3c30 2023-11-28T18:25:17.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac8dffb640 1 -- 172.21.15.105:0/1777872676 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac88005430 con 0x7fac980d3c30 2023-11-28T18:25:17.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac9f6c9640 1 -- 172.21.15.105:0/1777872676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac98150eb0 con 0x7fac980d3c30 2023-11-28T18:25:17.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac8dffb640 1 -- 172.21.15.105:0/1777872676 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac88003840 con 0x7fac980d3c30 2023-11-28T18:25:17.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac9f6c9640 1 -- 172.21.15.105:0/1777872676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac98151470 con 0x7fac980d3c30 2023-11-28T18:25:17.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.301+0000 7fac9f6c9640 1 -- 172.21.15.105:0/1777872676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac98067570 con 0x7fac980d3c30 2023-11-28T18:25:17.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.305+0000 7fac8dffb640 1 -- 172.21.15.105:0/1777872676 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fac880039a0 con 0x7fac980d3c30 2023-11-28T18:25:17.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.305+0000 7fac8dffb640 1 --2- 172.21.15.105:0/1777872676 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fac7c0668d0 0x7fac7c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:17.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.305+0000 7fac8dffb640 1 -- 172.21.15.105:0/1777872676 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fac88083a80 con 0x7fac980d3c30 2023-11-28T18:25:17.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.305+0000 7fac9d465640 1 --2- 172.21.15.105:0/1777872676 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fac7c0668d0 0x7fac7c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:17.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.305+0000 7fac9d465640 1 --2- 172.21.15.105:0/1777872676 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fac7c0668d0 0x7fac7c068d90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fac94005890 tx=0x7fac94009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:17.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.309+0000 7fac8dffb640 1 -- 172.21.15.105:0/1777872676 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fac88050bf0 con 0x7fac980d3c30 2023-11-28T18:25:17.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.489+0000 7fac9f6c9640 1 -- 172.21.15.105:0/1777872676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac98158f10 con 0x7fac980d3c30 2023-11-28T18:25:17.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.489+0000 7fac8dffb640 1 -- 172.21.15.105:0/1777872676 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4932 (secure 0 0 0) 0x7fac88017090 con 0x7fac980d3c30 2023-11-28T18:25:17.579 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:17.579 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/47307750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":47307750},{"type":"v1","addr":"172.21.15.148:6834","nonce":47307750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5392,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2878687703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2878687703},{"type":"v1","addr":"172.21.15.148:6835","nonce":2878687703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:24:55.936428+0000","modified":"2023-11-28T18:25:17.129291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1],"up":{"mds_0":5469,"mds_1":5463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5463":{"gid":5463,"name":"c","rank":1,"incarnation":20,"state":"up:creating","state_seq":1,"addr":"172.21.15.105:6836/681913204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":681913204},{"type":"v1","addr":"172.21.15.105:6836","nonce":681913204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5469":{"gid":5469,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/1876408690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":1876408690},{"type":"v1","addr":"172.21.15.105:6837","nonce":1876408690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T18:25:17.581 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 20 2023-11-28T18:25:17.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.493+0000 7fac9f6c9640 1 -- 172.21.15.105:0/1777872676 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fac7c0668d0 msgr2=0x7fac7c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:17.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.493+0000 7fac9f6c9640 1 --2- 172.21.15.105:0/1777872676 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fac7c0668d0 0x7fac7c068d90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fac94005890 tx=0x7fac94009040 comp rx=0 tx=0).stop 2023-11-28T18:25:17.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.493+0000 7fac9f6c9640 1 -- 172.21.15.105:0/1777872676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac980d3c30 msgr2=0x7fac981568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:17.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.493+0000 7fac9f6c9640 1 --2- 172.21.15.105:0/1777872676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac980d3c30 0x7fac981568c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fac8800c3f0 tx=0x7fac8800c8b0 comp rx=0 tx=0).stop 2023-11-28T18:25:17.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.493+0000 7fac9f6c9640 1 -- 172.21.15.105:0/1777872676 shutdown_connections 2023-11-28T18:25:17.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.493+0000 7fac9f6c9640 1 --2- 172.21.15.105:0/1777872676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fac9815d4f0 0x7fac98156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:17.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.493+0000 7fac9f6c9640 1 --2- 172.21.15.105:0/1777872676 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fac7c0668d0 0x7fac7c068d90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:17.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.493+0000 7fac9f6c9640 1 --2- 172.21.15.105:0/1777872676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac980d3c30 0x7fac981568c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:17.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.493+0000 7fac9f6c9640 1 --2- 172.21.15.105:0/1777872676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac980b3d30 0x7fac981555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:17.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.493+0000 7fac9f6c9640 1 -- 172.21.15.105:0/1777872676 >> 172.21.15.105:0/1777872676 conn(0x7fac980d3030 msgr2=0x7fac98167d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:17.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.493+0000 7fac9f6c9640 1 -- 172.21.15.105:0/1777872676 shutdown_connections 2023-11-28T18:25:17.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:17.493+0000 7fac9f6c9640 1 -- 172.21.15.105:0/1777872676 wait complete. 2023-11-28T18:25:17.985 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:24:55.936428+0000', 'modified': '2023-11-28T18:25:17.129291+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 5469, 'mds_1': 5463}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5463': {'gid': 5463, 'name': 'c', 'rank': 1, 'incarnation': 20, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.105:6836/681913204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 681913204}, {'type': 'v1', 'addr': '172.21.15.105:6836', 'nonce': 681913204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_5469': {'gid': 5469, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6837/1876408690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6835', 'nonce': 1876408690}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 1876408690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:25:17.986 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5463:up:creating 2023-11-28T18:25:18.987 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:25:19.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.129+0000 7f52de9aa640 1 Processor -- start 2023-11-28T18:25:19.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.129+0000 7f52de9aa640 1 -- start start 2023-11-28T18:25:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.129+0000 7f52de9aa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f52d8159f40 0x7f52d80caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.129+0000 7f52de9aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52d80ad850 0x7f52d80701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.129+0000 7f52de9aa640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d806bf50 con 0x7f52d80cc760 2023-11-28T18:25:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.129+0000 7f52de9aa640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d80ccbc0 con 0x7f52d8159f40 2023-11-28T18:25:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.129+0000 7f52de9aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d815a320 con 0x7f52d80ad850 2023-11-28T18:25:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.129+0000 7f52d77fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f52d80cc760 0x7f52d80ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54318/0 (socket says 172.21.15.105:54318) 2023-11-28T18:25:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.129+0000 7f52d77fe640 1 -- 172.21.15.105:0/2653632765 learned_addr learned my addr 172.21.15.105:0/2653632765 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.133+0000 7f52dcf47640 1 --2- 172.21.15.105:0/2653632765 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52d80ad850 0x7f52d80701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.133+0000 7f52d7fff640 1 --2- 172.21.15.105:0/2653632765 >> v2:172.21.15.148:3300/0 conn(0x7f52d8159f40 0x7f52d80caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.133+0000 7f52d6ffd640 1 -- 172.21.15.105:0/2653632765 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 311664690 0 0) 0x7f52d806bf50 con 0x7f52d80cc760 2023-11-28T18:25:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.133+0000 7f52d6ffd640 1 -- 172.21.15.105:0/2653632765 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52c0003530 con 0x7f52d80cc760 2023-11-28T18:25:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.133+0000 7f52dcf47640 1 -- 172.21.15.105:0/2653632765 >> v2:172.21.15.148:3300/0 conn(0x7f52d8159f40 msgr2=0x7f52d80caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:19.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.133+0000 7f52dcf47640 1 --2- 172.21.15.105:0/2653632765 >> v2:172.21.15.148:3300/0 conn(0x7f52d8159f40 0x7f52d80caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.133+0000 7f52dcf47640 1 -- 172.21.15.105:0/2653632765 >> v1:172.21.15.105:6789/0 conn(0x7f52d80cc760 legacy=0x7f52d80ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:19.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.133+0000 7f52dcf47640 1 -- 172.21.15.105:0/2653632765 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d8071380 con 0x7f52d80ad850 2023-11-28T18:25:19.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.133+0000 7f52d7fff640 1 --2- 172.21.15.105:0/2653632765 >> v2:172.21.15.148:3300/0 conn(0x7f52d8159f40 0x7f52d80caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:25:19.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.137+0000 7f52dcf47640 1 --2- 172.21.15.105:0/2653632765 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52d80ad850 0x7f52d80701d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f52d000be30 tx=0x7f52d001c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd5afdce6c316c53 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:19.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.141+0000 7f52d6ffd640 1 -- 172.21.15.105:0/2653632765 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d000b2d0 con 0x7f52d80ad850 2023-11-28T18:25:19.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.141+0000 7f52d6ffd640 1 -- 172.21.15.105:0/2653632765 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d00a57c0 con 0x7f52d80ad850 2023-11-28T18:25:19.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.141+0000 7f52d6ffd640 1 -- 172.21.15.105:0/2653632765 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d00a5b70 con 0x7f52d80ad850 2023-11-28T18:25:19.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.141+0000 7f52de9aa640 1 -- 172.21.15.105:0/2653632765 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52d80ad850 msgr2=0x7f52d80701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:19.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.141+0000 7f52de9aa640 1 --2- 172.21.15.105:0/2653632765 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52d80ad850 0x7f52d80701d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f52d000be30 tx=0x7f52d001c900 comp rx=0 tx=0).stop 2023-11-28T18:25:19.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.141+0000 7f52de9aa640 1 -- 172.21.15.105:0/2653632765 shutdown_connections 2023-11-28T18:25:19.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.141+0000 7f52de9aa640 1 --2- 172.21.15.105:0/2653632765 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52d80ad850 0x7f52d80701d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.141+0000 7f52de9aa640 1 --2- 172.21.15.105:0/2653632765 >> v2:172.21.15.148:3300/0 conn(0x7f52d8159f40 0x7f52d80caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.141+0000 7f52de9aa640 1 -- 172.21.15.105:0/2653632765 >> 172.21.15.105:0/2653632765 conn(0x7f52d80ceb90 msgr2=0x7f52d80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:19.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.141+0000 7f52de9aa640 1 -- 172.21.15.105:0/2653632765 shutdown_connections 2023-11-28T18:25:19.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.141+0000 7f52de9aa640 1 -- 172.21.15.105:0/2653632765 wait complete. 2023-11-28T18:25:19.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.141+0000 7f52de9aa640 1 Processor -- start 2023-11-28T18:25:19.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.141+0000 7f52de9aa640 1 -- start start 2023-11-28T18:25:19.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.141+0000 7f52de9aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52d80ad850 0x7f52d8145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.141+0000 7f52de9aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52d80cc760 0x7f52d814c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.145+0000 7f52de9aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52d8159f40 0x7f52d814e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.145+0000 7f52de9aa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d8145d40 con 0x7f52d8159f40 2023-11-28T18:25:19.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.145+0000 7f52de9aa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d814cc20 con 0x7f52d80cc760 2023-11-28T18:25:19.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.145+0000 7f52de9aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d814d060 con 0x7f52d80ad850 2023-11-28T18:25:19.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.145+0000 7f52dcf47640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52d8159f40 0x7f52d814e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.145+0000 7f52dcf47640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52d8159f40 0x7f52d814e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40454/0 (socket says 172.21.15.105:40454) 2023-11-28T18:25:19.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.145+0000 7f52dcf47640 1 -- 172.21.15.105:0/1540098697 learned_addr learned my addr 172.21.15.105:0/1540098697 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:19.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.145+0000 7f52d7fff640 1 --2- 172.21.15.105:0/1540098697 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52d80ad850 0x7f52d8145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.145+0000 7f52d77fe640 1 --2- 172.21.15.105:0/1540098697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52d80cc760 0x7f52d814c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.145+0000 7f52dcf47640 1 -- 172.21.15.105:0/1540098697 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52d80ad850 msgr2=0x7f52d8145800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:19.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.145+0000 7f52dcf47640 1 --2- 172.21.15.105:0/1540098697 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52d80ad850 0x7f52d8145800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.145+0000 7f52dcf47640 1 -- 172.21.15.105:0/1540098697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52d80cc760 msgr2=0x7f52d814c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:19.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.145+0000 7f52dcf47640 1 --2- 172.21.15.105:0/1540098697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52d80cc760 0x7f52d814c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.145+0000 7f52dcf47640 1 -- 172.21.15.105:0/1540098697 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c800d8c0 con 0x7f52d8159f40 2023-11-28T18:25:19.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.145+0000 7f52dcf47640 1 --2- 172.21.15.105:0/1540098697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52d8159f40 0x7f52d814e100 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f52d00af040 tx=0x7f52d00092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:19.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.145+0000 7f52d4ff9640 1 -- 172.21.15.105:0/1540098697 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d00a5b30 con 0x7f52d8159f40 2023-11-28T18:25:19.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.145+0000 7f52d4ff9640 1 -- 172.21.15.105:0/1540098697 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d00a5cb0 con 0x7f52d8159f40 2023-11-28T18:25:19.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.145+0000 7f52de9aa640 1 -- 172.21.15.105:0/1540098697 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d000ba90 con 0x7f52d8159f40 2023-11-28T18:25:19.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.145+0000 7f52d4ff9640 1 -- 172.21.15.105:0/1540098697 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d0004280 con 0x7f52d8159f40 2023-11-28T18:25:19.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.145+0000 7f52de9aa640 1 -- 172.21.15.105:0/1540098697 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d814d620 con 0x7f52d8159f40 2023-11-28T18:25:19.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.145+0000 7f52de9aa640 1 -- 172.21.15.105:0/1540098697 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d8067570 con 0x7f52d8159f40 2023-11-28T18:25:19.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.145+0000 7f52d4ff9640 1 -- 172.21.15.105:0/1540098697 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f52d00043e0 con 0x7f52d8159f40 2023-11-28T18:25:19.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.149+0000 7f52d4ff9640 1 --2- 172.21.15.105:0/1540098697 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52c40668d0 0x7f52c4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.149+0000 7f52d4ff9640 1 -- 172.21.15.105:0/1540098697 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f52d011f5a0 con 0x7f52d8159f40 2023-11-28T18:25:19.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.149+0000 7f52d7fff640 1 --2- 172.21.15.105:0/1540098697 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52c40668d0 0x7f52c4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.149+0000 7f52d7fff640 1 --2- 172.21.15.105:0/1540098697 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52c40668d0 0x7f52c4068d90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f52c8015340 tx=0x7f52c800f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:19.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.153+0000 7f52d4ff9640 1 -- 172.21.15.105:0/1540098697 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f52d00ec710 con 0x7f52d8159f40 2023-11-28T18:25:19.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.333+0000 7f52de9aa640 1 -- 172.21.15.105:0/1540098697 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52d80adce0 con 0x7f52d8159f40 2023-11-28T18:25:19.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.333+0000 7f52d4ff9640 1 -- 172.21.15.105:0/1540098697 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f52d00efd30 con 0x7f52d8159f40 2023-11-28T18:25:19.339 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:19.339 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/47307750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":47307750},{"type":"v1","addr":"172.21.15.148:6834","nonce":47307750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:24:55.936428+0000","modified":"2023-11-28T18:25:18.162937+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5469,"mds_1":5463,"mds_2":5392},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5392":{"gid":5392,"name":"d","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6835/2878687703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2878687703},{"type":"v1","addr":"172.21.15.148:6835","nonce":2878687703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5463":{"gid":5463,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.105:6836/681913204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":681913204},{"type":"v1","addr":"172.21.15.105:6836","nonce":681913204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5469":{"gid":5469,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/1876408690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":1876408690},{"type":"v1","addr":"172.21.15.105:6837","nonce":1876408690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T18:25:19.339 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 23 2023-11-28T18:25:19.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.333+0000 7f52de9aa640 1 -- 172.21.15.105:0/1540098697 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52c40668d0 msgr2=0x7f52c4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:19.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.333+0000 7f52de9aa640 1 --2- 172.21.15.105:0/1540098697 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52c40668d0 0x7f52c4068d90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f52c8015340 tx=0x7f52c800f040 comp rx=0 tx=0).stop 2023-11-28T18:25:19.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.333+0000 7f52de9aa640 1 -- 172.21.15.105:0/1540098697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52d8159f40 msgr2=0x7f52d814e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:19.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.333+0000 7f52de9aa640 1 --2- 172.21.15.105:0/1540098697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52d8159f40 0x7f52d814e100 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f52d00af040 tx=0x7f52d00092f0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.333+0000 7f52de9aa640 1 -- 172.21.15.105:0/1540098697 shutdown_connections 2023-11-28T18:25:19.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.333+0000 7f52de9aa640 1 --2- 172.21.15.105:0/1540098697 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52c40668d0 0x7f52c4068d90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.333+0000 7f52de9aa640 1 --2- 172.21.15.105:0/1540098697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52d8159f40 0x7f52d814e100 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.333+0000 7f52de9aa640 1 --2- 172.21.15.105:0/1540098697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52d80cc760 0x7f52d814c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.333+0000 7f52de9aa640 1 --2- 172.21.15.105:0/1540098697 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52d80ad850 0x7f52d8145800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.333+0000 7f52de9aa640 1 -- 172.21.15.105:0/1540098697 >> 172.21.15.105:0/1540098697 conn(0x7f52d80ceb90 msgr2=0x7f52d80cdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:19.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.333+0000 7f52de9aa640 1 -- 172.21.15.105:0/1540098697 shutdown_connections 2023-11-28T18:25:19.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.337+0000 7f52de9aa640 1 -- 172.21.15.105:0/1540098697 wait complete. 2023-11-28T18:25:19.763 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:24:55.936428+0000', 'modified': '2023-11-28T18:25:18.162937+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 5469, 'mds_1': 5463, 'mds_2': 5392}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5392': {'gid': 5392, 'name': 'd', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.148:6835/2878687703', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6833', 'nonce': 2878687703}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 2878687703}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_5463': {'gid': 5463, 'name': 'c', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.105:6836/681913204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 681913204}, {'type': 'v1', 'addr': '172.21.15.105:6836', 'nonce': 681913204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_5469': {'gid': 5469, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6837/1876408690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6835', 'nonce': 1876408690}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 1876408690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:25:19.763 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-11-28T18:25:19.763 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-28T18:25:19.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47becea640 1 Processor -- start 2023-11-28T18:25:19.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47becea640 1 -- start start 2023-11-28T18:25:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47becea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47b80caf60 0x7f47b80cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47becea640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47b806bf50 con 0x7f47b80ad850 2023-11-28T18:25:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47becea640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47b815a320 con 0x7f47b8159f40 2023-11-28T18:25:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47becea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b80cb3c0 con 0x7f47b80caf60 2023-11-28T18:25:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47bd287640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f47b80ad850 0x7f47b80701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54334/0 (socket says 172.21.15.105:54334) 2023-11-28T18:25:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47bd287640 1 -- 172.21.15.105:0/1175086239 learned_addr learned my addr 172.21.15.105:0/1175086239 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47b7fff640 1 --2- 172.21.15.105:0/1175086239 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47b80caf60 0x7f47b80cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47b77fe640 1 -- 172.21.15.105:0/1175086239 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705805422 0 0) 0x7f47b806bf50 con 0x7f47b80ad850 2023-11-28T18:25:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47b77fe640 1 -- 172.21.15.105:0/1175086239 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47a8003530 con 0x7f47b80ad850 2023-11-28T18:25:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47b77fe640 1 -- 172.21.15.105:0/1175086239 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87456122 0 0) 0x7f47b815a320 con 0x7f47b8159f40 2023-11-28T18:25:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47b77fe640 1 -- 172.21.15.105:0/1175086239 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47b806bf50 con 0x7f47b8159f40 2023-11-28T18:25:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47b77fe640 1 -- 172.21.15.105:0/1175086239 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 766382476 0 0) 0x7f47a8003530 con 0x7f47b80ad850 2023-11-28T18:25:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47b77fe640 1 -- 172.21.15.105:0/1175086239 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47b80caf60 msgr2=0x7f47b80cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47b77fe640 1 --2- 172.21.15.105:0/1175086239 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47b80caf60 0x7f47b80cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47b77fe640 1 -- 172.21.15.105:0/1175086239 >> v1:172.21.15.148:6789/0 conn(0x7f47b8159f40 legacy=0x7f47b80ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47b77fe640 1 -- 172.21.15.105:0/1175086239 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b8071380 con 0x7f47b80ad850 2023-11-28T18:25:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47b77fe640 1 -- 172.21.15.105:0/1175086239 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f47b0001a00 con 0x7f47b80ad850 2023-11-28T18:25:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47b77fe640 1 -- 172.21.15.105:0/1175086239 >> v1:172.21.15.105:6789/0 conn(0x7f47b80ad850 legacy=0x7f47b80701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47b77fe640 1 --2- 172.21.15.105:0/1175086239 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47b80caf60 0x7f47a8003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47b77fe640 1 --2- 172.21.15.105:0/1175086239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47a80040a0 0x7f47a80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47b77fe640 1 --2- 172.21.15.105:0/1175086239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47a8004d10 0x7f47a8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47b77fe640 1 -- 172.21.15.105:0/1175086239 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b80cb3c0 con 0x7f47a8004d10 2023-11-28T18:25:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47b77fe640 1 -- 172.21.15.105:0/1175086239 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a800bb20 con 0x7f47a80040a0 2023-11-28T18:25:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47b77fe640 1 -- 172.21.15.105:0/1175086239 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a800bcb0 con 0x7f47b80caf60 2023-11-28T18:25:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47bd287640 1 --2- 172.21.15.105:0/1175086239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47a8004d10 0x7f47a8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47bca86640 1 --2- 172.21.15.105:0/1175086239 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47b80caf60 0x7f47a8003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.897+0000 7f47b7fff640 1 --2- 172.21.15.105:0/1175086239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47a80040a0 0x7f47a80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.901+0000 7f47b7fff640 1 -- 172.21.15.105:0/1175086239 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47b80caf60 msgr2=0x7f47a8003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.901+0000 7f47b7fff640 1 --2- 172.21.15.105:0/1175086239 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47b80caf60 0x7f47a8003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.901+0000 7f47b7fff640 1 -- 172.21.15.105:0/1175086239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47a8004d10 msgr2=0x7f47a8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.901+0000 7f47b7fff640 1 --2- 172.21.15.105:0/1175086239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47a8004d10 0x7f47a8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.901+0000 7f47b7fff640 1 -- 172.21.15.105:0/1175086239 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a8003530 con 0x7f47a80040a0 2023-11-28T18:25:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.901+0000 7f47bd287640 1 --2- 172.21.15.105:0/1175086239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47a8004d10 0x7f47a8005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:25:19.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.901+0000 7f47b7fff640 1 --2- 172.21.15.105:0/1175086239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47a80040a0 0x7f47a80047d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009ae0 tx=0x7f47a401ca50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c028823446821a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:19.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.901+0000 7f47b77fe640 1 -- 172.21.15.105:0/1175086239 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a40a5640 con 0x7f47a80040a0 2023-11-28T18:25:19.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.901+0000 7f47b77fe640 1 -- 172.21.15.105:0/1175086239 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a40a57a0 con 0x7f47a80040a0 2023-11-28T18:25:19.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.901+0000 7f47b77fe640 1 -- 172.21.15.105:0/1175086239 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a40a5b50 con 0x7f47a80040a0 2023-11-28T18:25:19.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.901+0000 7f47becea640 1 -- 172.21.15.105:0/1175086239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47a80040a0 msgr2=0x7f47a80047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.901+0000 7f47becea640 1 --2- 172.21.15.105:0/1175086239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47a80040a0 0x7f47a80047d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009ae0 tx=0x7f47a401ca50 comp rx=0 tx=0).stop 2023-11-28T18:25:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.901+0000 7f47bd287640 1 -- 172.21.15.105:0/1175086239 reap_dead start 2023-11-28T18:25:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.901+0000 7f47becea640 1 -- 172.21.15.105:0/1175086239 shutdown_connections 2023-11-28T18:25:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.901+0000 7f47becea640 1 -- 172.21.15.105:0/1175086239 >> 172.21.15.105:0/1175086239 conn(0x7f47b80ceb90 msgr2=0x7f47b80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.901+0000 7f47becea640 1 -- 172.21.15.105:0/1175086239 shutdown_connections 2023-11-28T18:25:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.905+0000 7f47becea640 1 -- 172.21.15.105:0/1175086239 wait complete. 2023-11-28T18:25:19.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.905+0000 7f47becea640 1 Processor -- start 2023-11-28T18:25:19.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.905+0000 7f47becea640 1 -- start start 2023-11-28T18:25:19.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.905+0000 7f47becea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47a80040a0 0x7f47b8153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.905+0000 7f47becea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47b81f3840 0x7f47b815a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.905+0000 7f47becea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47b81551e0 0x7f47b8155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.905+0000 7f47becea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b8155bb0 con 0x7f47b81551e0 2023-11-28T18:25:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.905+0000 7f47becea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b8099b60 con 0x7f47b81f3840 2023-11-28T18:25:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.905+0000 7f47becea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b815ac90 con 0x7f47a80040a0 2023-11-28T18:25:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.905+0000 7f47bd287640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47b81551e0 0x7f47b8155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.905+0000 7f47bd287640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47b81551e0 0x7f47b8155670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40478/0 (socket says 172.21.15.105:40478) 2023-11-28T18:25:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.905+0000 7f47bd287640 1 -- 172.21.15.105:0/1536168453 learned_addr learned my addr 172.21.15.105:0/1536168453 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.905+0000 7f47b7fff640 1 --2- 172.21.15.105:0/1536168453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47b81f3840 0x7f47b815a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.905+0000 7f47bca86640 1 --2- 172.21.15.105:0/1536168453 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47a80040a0 0x7f47b8153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.905+0000 7f47bd287640 1 -- 172.21.15.105:0/1536168453 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47a80040a0 msgr2=0x7f47b8153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.905+0000 7f47bd287640 1 --2- 172.21.15.105:0/1536168453 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47a80040a0 0x7f47b8153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.905+0000 7f47bd287640 1 -- 172.21.15.105:0/1536168453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47b81f3840 msgr2=0x7f47b815a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.905+0000 7f47bd287640 1 --2- 172.21.15.105:0/1536168453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47b81f3840 0x7f47b815a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.905+0000 7f47bd287640 1 -- 172.21.15.105:0/1536168453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b815af70 con 0x7f47b81551e0 2023-11-28T18:25:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.909+0000 7f47bd287640 1 --2- 172.21.15.105:0/1536168453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47b81551e0 0x7f47b8155670 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f47b8073a50 tx=0x7f47b0007060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.909+0000 7f47b57fa640 1 -- 172.21.15.105:0/1536168453 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b0007930 con 0x7f47b81551e0 2023-11-28T18:25:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.909+0000 7f47b57fa640 1 -- 172.21.15.105:0/1536168453 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b0007a90 con 0x7f47b81551e0 2023-11-28T18:25:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.909+0000 7f47b57fa640 1 -- 172.21.15.105:0/1536168453 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b0013630 con 0x7f47b81551e0 2023-11-28T18:25:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.909+0000 7f47becea640 1 -- 172.21.15.105:0/1536168453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b815b250 con 0x7f47b81551e0 2023-11-28T18:25:19.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.909+0000 7f47becea640 1 -- 172.21.15.105:0/1536168453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b8153860 con 0x7f47b81551e0 2023-11-28T18:25:19.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.909+0000 7f47b57fa640 1 -- 172.21.15.105:0/1536168453 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f47b0013790 con 0x7f47b81551e0 2023-11-28T18:25:19.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.909+0000 7f47b57fa640 1 --2- 172.21.15.105:0/1536168453 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f479c0668d0 0x7f479c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.909+0000 7f47b57fa640 1 -- 172.21.15.105:0/1536168453 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f47b0084c40 con 0x7f47b81551e0 2023-11-28T18:25:19.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.909+0000 7f47bca86640 1 --2- 172.21.15.105:0/1536168453 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f479c0668d0 0x7f479c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.909+0000 7f47c2057000 1 Processor -- start 2023-11-28T18:25:19.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.909+0000 7f47c2057000 1 -- start start 2023-11-28T18:25:19.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47bca86640 1 --2- 172.21.15.105:0/1536168453 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f479c0668d0 0x7f479c068d90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f47ac005630 tx=0x7f47ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47c2057000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b385d30 0x555c5b386110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47c2057000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555c5b386650 0x555c5b338fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47c2057000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c5b339500 0x555c5b3398e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47c2057000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5b202110 con 0x555c5b385d30 2023-11-28T18:25:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47c2057000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5b1929a0 con 0x555c5b339500 2023-11-28T18:25:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47c2057000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c5b18fed0 con 0x555c5b386650 2023-11-28T18:25:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47bca86640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b385d30 0x555c5b386110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47bca86640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b385d30 0x555c5b386110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40492/0 (socket says 172.21.15.105:40492) 2023-11-28T18:25:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47bca86640 1 -- 172.21.15.105:0/2591250213 learned_addr learned my addr 172.21.15.105:0/2591250213 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47b7fff640 1 --2- 172.21.15.105:0/2591250213 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555c5b386650 0x555c5b338fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47bd287640 1 --2- 172.21.15.105:0/2591250213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c5b339500 0x555c5b3398e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47bca86640 1 -- 172.21.15.105:0/2591250213 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555c5b386650 msgr2=0x555c5b338fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47bca86640 1 --2- 172.21.15.105:0/2591250213 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555c5b386650 0x555c5b338fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47bca86640 1 -- 172.21.15.105:0/2591250213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c5b339500 msgr2=0x555c5b3398e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47bca86640 1 --2- 172.21.15.105:0/2591250213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c5b339500 0x555c5b3398e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47bca86640 1 -- 172.21.15.105:0/2591250213 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c5ae83890 con 0x555c5b385d30 2023-11-28T18:25:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47bd287640 1 --2- 172.21.15.105:0/2591250213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c5b339500 0x555c5b3398e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47bca86640 1 --2- 172.21.15.105:0/2591250213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b385d30 0x555c5b386110 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0079b0 tx=0x7f47ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184fdd61655f3d46 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47a27fc640 1 -- 172.21.15.105:0/2591250213 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac01c040 con 0x555c5b385d30 2023-11-28T18:25:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47a27fc640 1 -- 172.21.15.105:0/2591250213 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ac007610 con 0x555c5b385d30 2023-11-28T18:25:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47a27fc640 1 -- 172.21.15.105:0/2591250213 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac023600 con 0x555c5b385d30 2023-11-28T18:25:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47c2057000 1 -- 172.21.15.105:0/2591250213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b385d30 msgr2=0x555c5b386110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47c2057000 1 --2- 172.21.15.105:0/2591250213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b385d30 0x555c5b386110 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0079b0 tx=0x7f47ac017000 comp rx=0 tx=0).stop 2023-11-28T18:25:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47c2057000 1 -- 172.21.15.105:0/2591250213 shutdown_connections 2023-11-28T18:25:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47c2057000 1 --2- 172.21.15.105:0/2591250213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c5b339500 0x555c5b3398e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47c2057000 1 --2- 172.21.15.105:0/2591250213 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555c5b386650 0x555c5b338fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47c2057000 1 --2- 172.21.15.105:0/2591250213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b385d30 0x555c5b386110 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47c2057000 1 -- 172.21.15.105:0/2591250213 >> 172.21.15.105:0/2591250213 conn(0x555c5b3668c0 msgr2=0x555c5b368cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47c2057000 1 -- 172.21.15.105:0/2591250213 shutdown_connections 2023-11-28T18:25:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47c2057000 1 -- 172.21.15.105:0/2591250213 wait complete. 2023-11-28T18:25:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47c2057000 1 -- start start 2023-11-28T18:25:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.913+0000 7f47c2057000 1 Processor -- start 2023-11-28T18:25:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47c2057000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b339500 0x555c5b33f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47c2057000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c5b385d30 0x555c5b33fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47c2057000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555c5b386650 0x555c5b33ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47c2057000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5b18b7c0 con 0x555c5b339500 2023-11-28T18:25:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47c2057000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5b143bf0 con 0x555c5b385d30 2023-11-28T18:25:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47c2057000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c5b149a90 con 0x555c5b386650 2023-11-28T18:25:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47c2057000 10 client.0 _ll_register_callbacks cb 0x555c5b326e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:25:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47bca86640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b339500 0x555c5b33f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47bca86640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b339500 0x555c5b33f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40506/0 (socket says 172.21.15.105:40506) 2023-11-28T18:25:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47bca86640 1 -- 172.21.15.105:0/2226878053 learned_addr learned my addr 172.21.15.105:0/2226878053 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47bd287640 1 --2- 172.21.15.105:0/2226878053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555c5b386650 0x555c5b33ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47b7fff640 1 --2- 172.21.15.105:0/2226878053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c5b385d30 0x555c5b33fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47bca86640 1 -- 172.21.15.105:0/2226878053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555c5b386650 msgr2=0x555c5b33ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47bca86640 1 --2- 172.21.15.105:0/2226878053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555c5b386650 0x555c5b33ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47bca86640 1 -- 172.21.15.105:0/2226878053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c5b385d30 msgr2=0x555c5b33fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47bca86640 1 --2- 172.21.15.105:0/2226878053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c5b385d30 0x555c5b33fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47bca86640 1 -- 172.21.15.105:0/2226878053 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b806bf50 con 0x555c5b339500 2023-11-28T18:25:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47bca86640 1 --2- 172.21.15.105:0/2226878053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b339500 0x555c5b33f4e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f47ac015010 tx=0x7f47ac0012e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae892e44a6572d85 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47977fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:25:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47977fe640 1 -- 172.21.15.105:0/2226878053 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac0248b0 con 0x555c5b339500 2023-11-28T18:25:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47977fe640 1 -- 172.21.15.105:0/2226878053 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f47ac024a10 con 0x555c5b339500 2023-11-28T18:25:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47c2057000 1 -- 172.21.15.105:0/2226878053 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c5b14d470 con 0x555c5b339500 2023-11-28T18:25:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47977fe640 1 -- 172.21.15.105:0/2226878053 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47ac024bd0 con 0x555c5b339500 2023-11-28T18:25:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47c2057000 10 client.5754 fetch_fsmap learned FSMap version 23 2023-11-28T18:25:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47c2057000 1 -- 172.21.15.105:0/2226878053 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x555c5b14c190 con 0x555c5b339500 2023-11-28T18:25:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47977fe640 1 -- 172.21.15.105:0/2226878053 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555c5b14d470 con 0x555c5b339500 2023-11-28T18:25:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47977fe640 1 -- 172.21.15.105:0/2226878053 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 23) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f47ac01d070 con 0x555c5b339500 2023-11-28T18:25:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47c2057000 10 client.5754 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T18:25:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47c2057000 10 client.5754 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T18:25:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47c2057000 20 client.5754 populate_metadata read hostname 'smithi105' 2023-11-28T18:25:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47c2057000 1 --2- 172.21.15.105:0/2226878053 >> [v2:172.21.15.148:6833/2878687703,v1:172.21.15.148:6835/2878687703] conn(0x555c5b340970 0x555c5b397ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47c2057000 4 client.5754 mds_command: new command op to 5392 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:25:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47c2057000 1 -- 172.21.15.105:0/2226878053 --> [v2:172.21.15.148:6833/2878687703,v1:172.21.15.148:6835/2878687703] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c5b150b50 con 0x555c5b340970 2023-11-28T18:25:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47b7fff640 1 --2- 172.21.15.105:0/2226878053 >> [v2:172.21.15.148:6833/2878687703,v1:172.21.15.148:6835/2878687703] conn(0x555c5b340970 0x555c5b397ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47b7fff640 1 --2- 172.21.15.105:0/2226878053 >> [v2:172.21.15.148:6833/2878687703,v1:172.21.15.148:6835/2878687703] conn(0x555c5b340970 0x555c5b397ff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=61aeefabc137b684 server_cookie=9b7039db5c0e76ef in_seq=0 out_seq=0 2023-11-28T18:25:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.917+0000 7f47977fe640 10 client.5754 ms_handle_connect on v2:172.21.15.148:6833/2878687703 2023-11-28T18:25:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.921+0000 7f47977fe640 1 -- 172.21.15.105:0/2226878053 <== mds.2 v2:172.21.15.148:6833/2878687703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555c5b150b50 con 0x555c5b340970 2023-11-28T18:25:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.921+0000 7f47977fe640 10 client.5754 handle_command_reply: tid=0 2023-11-28T18:25:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.921+0000 7f47c2057000 1 client.5754 shutdown 2023-11-28T18:25:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.921+0000 7f47c2057000 1 -- 172.21.15.105:0/2226878053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b339500 msgr2=0x555c5b33f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.921+0000 7f47c2057000 1 --2- 172.21.15.105:0/2226878053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b339500 0x555c5b33f4e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f47ac015010 tx=0x7f47ac0012e0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.921+0000 7f47c2057000 1 -- 172.21.15.105:0/2226878053 shutdown_connections 2023-11-28T18:25:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.921+0000 7f47c2057000 1 --2- 172.21.15.105:0/2226878053 >> [v2:172.21.15.148:6833/2878687703,v1:172.21.15.148:6835/2878687703] conn(0x555c5b340970 0x555c5b397ff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.921+0000 7f47c2057000 1 --2- 172.21.15.105:0/2226878053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555c5b386650 0x555c5b33ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.921+0000 7f47c2057000 1 --2- 172.21.15.105:0/2226878053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c5b385d30 0x555c5b33fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.921+0000 7f47c2057000 1 --2- 172.21.15.105:0/2226878053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b339500 0x555c5b33f4e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.921+0000 7f47977fe640 0 client.5754 ms_handle_reset on v2:172.21.15.148:6833/2878687703 2023-11-28T18:25:19.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.921+0000 7f47c2057000 1 -- 172.21.15.105:0/2226878053 >> 172.21.15.105:0/2226878053 conn(0x555c5b3668c0 msgr2=0x555c5b3386b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:19.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.921+0000 7f47c2057000 1 -- 172.21.15.105:0/2226878053 shutdown_connections 2023-11-28T18:25:19.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.921+0000 7f47c2057000 1 -- 172.21.15.105:0/2226878053 wait complete. 2023-11-28T18:25:19.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.921+0000 7f47c2057000 20 client.5754 trim_cache size 0 max 16384 2023-11-28T18:25:19.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47c2057000 1 Processor -- start 2023-11-28T18:25:19.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47c2057000 1 -- start start 2023-11-28T18:25:19.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47c2057000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b339500 0x555c5b3653a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47c2057000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c5b385d30 0x555c5b3658e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47c2057000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555c5b386650 0x555c5b397ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47c2057000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5ae44940 con 0x555c5b339500 2023-11-28T18:25:19.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47c2057000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5b212570 con 0x555c5b385d30 2023-11-28T18:25:19.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47c2057000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c5b14a8f0 con 0x555c5b386650 2023-11-28T18:25:19.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47bca86640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b339500 0x555c5b3653a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47bca86640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b339500 0x555c5b3653a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40510/0 (socket says 172.21.15.105:40510) 2023-11-28T18:25:19.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47bca86640 1 -- 172.21.15.105:0/986333482 learned_addr learned my addr 172.21.15.105:0/986333482 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:19.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47bca86640 1 -- 172.21.15.105:0/986333482 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555c5b386650 msgr2=0x555c5b397ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:19.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47bca86640 1 --2- 172.21.15.105:0/986333482 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555c5b386650 0x555c5b397ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47bca86640 1 -- 172.21.15.105:0/986333482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c5b385d30 msgr2=0x555c5b3658e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:19.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47bca86640 1 --2- 172.21.15.105:0/986333482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c5b385d30 0x555c5b3658e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47bca86640 1 -- 172.21.15.105:0/986333482 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c5b132490 con 0x555c5b339500 2023-11-28T18:25:19.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47bca86640 1 --2- 172.21.15.105:0/986333482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b339500 0x555c5b3653a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0014f0 tx=0x7f47ac02f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d680407e22168aef server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:19.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47a2ffd640 1 -- 172.21.15.105:0/986333482 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac0235c0 con 0x555c5b339500 2023-11-28T18:25:19.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47a2ffd640 1 -- 172.21.15.105:0/986333482 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ac0246b0 con 0x555c5b339500 2023-11-28T18:25:19.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47a2ffd640 1 -- 172.21.15.105:0/986333482 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac024a20 con 0x555c5b339500 2023-11-28T18:25:19.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47c2057000 1 -- 172.21.15.105:0/986333482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b339500 msgr2=0x555c5b3653a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:19.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47c2057000 1 --2- 172.21.15.105:0/986333482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b339500 0x555c5b3653a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0014f0 tx=0x7f47ac02f000 comp rx=0 tx=0).stop 2023-11-28T18:25:19.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47c2057000 1 -- 172.21.15.105:0/986333482 shutdown_connections 2023-11-28T18:25:19.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47c2057000 1 --2- 172.21.15.105:0/986333482 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555c5b386650 0x555c5b397ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47c2057000 1 --2- 172.21.15.105:0/986333482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c5b385d30 0x555c5b3658e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47c2057000 1 --2- 172.21.15.105:0/986333482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b339500 0x555c5b3653a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47c2057000 1 -- 172.21.15.105:0/986333482 >> 172.21.15.105:0/986333482 conn(0x555c5b3668c0 msgr2=0x555c5b337720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:19.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47c2057000 1 -- 172.21.15.105:0/986333482 shutdown_connections 2023-11-28T18:25:19.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47c2057000 1 -- 172.21.15.105:0/986333482 wait complete. 2023-11-28T18:25:19.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.937+0000 7f47c2057000 1 -- start start 2023-11-28T18:25:19.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47c2057000 1 Processor -- start 2023-11-28T18:25:19.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47c2057000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c5b339500 0x555c5b3ef020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47c2057000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b385d30 0x555c5b3ef560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47c2057000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555c5b386650 0x555c5b3aa280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47c2057000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5b221440 con 0x555c5b385d30 2023-11-28T18:25:19.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47c2057000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c5b385be0 con 0x555c5b339500 2023-11-28T18:25:19.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47c2057000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c5b1552e0 con 0x555c5b386650 2023-11-28T18:25:19.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47c2057000 10 client.0 _ll_register_callbacks cb 0x555c5b202060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:25:19.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47b7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b385d30 0x555c5b3ef560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47bca86640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c5b339500 0x555c5b3ef020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47bd287640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555c5b386650 0x555c5b3aa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47bca86640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c5b339500 0x555c5b3ef020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:53102/0 (socket says 172.21.15.105:53102) 2023-11-28T18:25:19.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47bca86640 1 -- 172.21.15.105:0/1961778322 learned_addr learned my addr 172.21.15.105:0/1961778322 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:19.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47b7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b385d30 0x555c5b3ef560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40522/0 (socket says 172.21.15.105:40522) 2023-11-28T18:25:19.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47bca86640 1 -- 172.21.15.105:0/1961778322 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555c5b386650 msgr2=0x555c5b3aa280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:19.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47bca86640 1 --2- 172.21.15.105:0/1961778322 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555c5b386650 0x555c5b3aa280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47bca86640 1 -- 172.21.15.105:0/1961778322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b385d30 msgr2=0x555c5b3ef560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:19.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47bca86640 1 --2- 172.21.15.105:0/1961778322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b385d30 0x555c5b3ef560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47bca86640 1 -- 172.21.15.105:0/1961778322 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c5b14c190 con 0x555c5b339500 2023-11-28T18:25:19.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47b7fff640 1 --2- 172.21.15.105:0/1961778322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b385d30 0x555c5b3ef560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:25:19.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47bca86640 1 --2- 172.21.15.105:0/1961778322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c5b339500 0x555c5b3ef020 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f47ac017cc0 tx=0x7f47ac00c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc927214f2b98e48 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:19.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47977fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-11-28T18:25:19.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47977fe640 1 -- 172.21.15.105:0/1961778322 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac02fd20 con 0x555c5b339500 2023-11-28T18:25:19.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47977fe640 1 -- 172.21.15.105:0/1961778322 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f47ac04b480 con 0x555c5b339500 2023-11-28T18:25:19.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47c2057000 1 -- 172.21.15.105:0/1961778322 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c5b18a110 con 0x555c5b339500 2023-11-28T18:25:19.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47977fe640 1 -- 172.21.15.105:0/1961778322 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47ac02fe80 con 0x555c5b339500 2023-11-28T18:25:19.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47c2057000 10 client.5593 fetch_fsmap learned FSMap version 23 2023-11-28T18:25:19.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47c2057000 1 -- 172.21.15.105:0/1961778322 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x555c5ae83cc0 con 0x555c5b339500 2023-11-28T18:25:19.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47977fe640 1 -- 172.21.15.105:0/1961778322 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555c5b18a110 con 0x555c5b339500 2023-11-28T18:25:19.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47977fe640 1 -- 172.21.15.105:0/1961778322 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 23) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f47ac022040 con 0x555c5b339500 2023-11-28T18:25:19.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47c2057000 10 client.5593 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T18:25:19.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47c2057000 10 client.5593 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T18:25:19.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47c2057000 20 client.5593 populate_metadata read hostname 'smithi105' 2023-11-28T18:25:19.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47c2057000 1 --2- 172.21.15.105:0/1961778322 >> [v2:172.21.15.148:6833/2878687703,v1:172.21.15.148:6835/2878687703] conn(0x555c5b3aac40 0x555c5b3ab020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:19.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47c2057000 4 client.5593 mds_command: new command op to 5392 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:25:19.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47c2057000 1 -- 172.21.15.105:0/1961778322 --> [v2:172.21.15.148:6833/2878687703,v1:172.21.15.148:6835/2878687703] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555c5b1b8650 con 0x555c5b3aac40 2023-11-28T18:25:19.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.941+0000 7f47b7fff640 1 --2- 172.21.15.105:0/1961778322 >> [v2:172.21.15.148:6833/2878687703,v1:172.21.15.148:6835/2878687703] conn(0x555c5b3aac40 0x555c5b3ab020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:19.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47977fe640 10 client.5593 ms_handle_connect on v2:172.21.15.148:6833/2878687703 2023-11-28T18:25:19.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47b7fff640 1 --2- 172.21.15.105:0/1961778322 >> [v2:172.21.15.148:6833/2878687703,v1:172.21.15.148:6835/2878687703] conn(0x555c5b3aac40 0x555c5b3ab020 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e6229d8ebca534e3 server_cookie=dcdf19c182aee97b in_seq=0 out_seq=0 2023-11-28T18:25:19.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47977fe640 1 -- 172.21.15.105:0/1961778322 <== mds.2 v2:172.21.15.148:6833/2878687703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x555c5b1b8650 con 0x555c5b3aac40 2023-11-28T18:25:19.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47977fe640 10 client.5593 handle_command_reply: tid=0 2023-11-28T18:25:19.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47c2057000 1 client.5593 shutdown 2023-11-28T18:25:19.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47c2057000 1 -- 172.21.15.105:0/1961778322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c5b339500 msgr2=0x555c5b3ef020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:19.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47c2057000 1 --2- 172.21.15.105:0/1961778322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c5b339500 0x555c5b3ef020 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f47ac017cc0 tx=0x7f47ac00c000 comp rx=0 tx=0).stop 2023-11-28T18:25:19.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47c2057000 1 -- 172.21.15.105:0/1961778322 shutdown_connections 2023-11-28T18:25:19.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47c2057000 1 --2- 172.21.15.105:0/1961778322 >> [v2:172.21.15.148:6833/2878687703,v1:172.21.15.148:6835/2878687703] conn(0x555c5b3aac40 0x555c5b3ab020 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47c2057000 1 --2- 172.21.15.105:0/1961778322 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555c5b386650 0x555c5b3aa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47c2057000 1 --2- 172.21.15.105:0/1961778322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c5b385d30 0x555c5b3ef560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47c2057000 1 --2- 172.21.15.105:0/1961778322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555c5b339500 0x555c5b3ef020 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47977fe640 0 client.5593 ms_handle_reset on v2:172.21.15.148:6833/2878687703 2023-11-28T18:25:19.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47c2057000 1 -- 172.21.15.105:0/1961778322 >> 172.21.15.105:0/1961778322 conn(0x555c5b3668c0 msgr2=0x555c5b337fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:19.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47c2057000 1 -- 172.21.15.105:0/1961778322 shutdown_connections 2023-11-28T18:25:19.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47c2057000 1 -- 172.21.15.105:0/1961778322 wait complete. 2023-11-28T18:25:19.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47c2057000 20 client.5593 trim_cache size 0 max 16384 2023-11-28T18:25:19.956 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:25:19.956 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:25:19.956 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 2, 2023-11-28T18:25:19.956 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5392, 2023-11-28T18:25:19.957 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:25:19.957 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:25:19.957 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:25:19.957 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 1.8064373330000001, 2023-11-28T18:25:19.957 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 23, 2023-11-28T18:25:19.957 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 40, 2023-11-28T18:25:19.957 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 40, 2023-11-28T18:25:19.957 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 31.264614172000002 2023-11-28T18:25:19.957 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:25:19.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47becea640 1 -- 172.21.15.105:0/1536168453 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f479c0668d0 msgr2=0x7f479c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:19.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47becea640 1 --2- 172.21.15.105:0/1536168453 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f479c0668d0 0x7f479c068d90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f47ac005630 tx=0x7f47ac009040 comp rx=0 tx=0).stop 2023-11-28T18:25:19.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47becea640 1 -- 172.21.15.105:0/1536168453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47b81551e0 msgr2=0x7f47b8155670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:19.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47becea640 1 --2- 172.21.15.105:0/1536168453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47b81551e0 0x7f47b8155670 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f47b8073a50 tx=0x7f47b0007060 comp rx=0 tx=0).stop 2023-11-28T18:25:19.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47becea640 1 -- 172.21.15.105:0/1536168453 shutdown_connections 2023-11-28T18:25:19.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47becea640 1 --2- 172.21.15.105:0/1536168453 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f479c0668d0 0x7f479c068d90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47becea640 1 --2- 172.21.15.105:0/1536168453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47b81551e0 0x7f47b8155670 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47becea640 1 --2- 172.21.15.105:0/1536168453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47b81f3840 0x7f47b815a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47becea640 1 --2- 172.21.15.105:0/1536168453 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47a80040a0 0x7f47b8153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:19.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47becea640 1 -- 172.21.15.105:0/1536168453 >> 172.21.15.105:0/1536168453 conn(0x7f47b80ceb90 msgr2=0x7f47b806c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:19.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47becea640 1 -- 172.21.15.105:0/1536168453 shutdown_connections 2023-11-28T18:25:19.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:19.945+0000 7f47becea640 1 -- 172.21.15.105:0/1536168453 wait complete. 2023-11-28T18:25:19.969 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T18:25:20.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.101+0000 7f6ffba9a640 1 Processor -- start 2023-11-28T18:25:20.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.101+0000 7f6ffba9a640 1 -- start start 2023-11-28T18:25:20.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.101+0000 7f6ffba9a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6ff415d300 0x7f6ff40d37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.101+0000 7f6ffba9a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6ff40d1d60 0x7f6ff40c7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.101+0000 7f6ffba9a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff40d0c60 con 0x7f6ff40d3ce0 2023-11-28T18:25:20.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.101+0000 7f6ffba9a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff40d1470 con 0x7f6ff40d1d60 2023-11-28T18:25:20.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.101+0000 7f6ffba9a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff40cefa0 con 0x7f6ff415d300 2023-11-28T18:25:20.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.101+0000 7f6ffa037640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6ff40d1d60 0x7f6ff40c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.101+0000 7f6ffa037640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6ff40d1d60 0x7f6ff40c7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40536/0 (socket says 172.21.15.105:40536) 2023-11-28T18:25:20.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.101+0000 7f6ffa037640 1 -- 172.21.15.105:0/2901817623 learned_addr learned my addr 172.21.15.105:0/2901817623 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:20.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.101+0000 7f6ff9836640 1 --2- 172.21.15.105:0/2901817623 >> v2:172.21.15.148:3300/0 conn(0x7f6ff415d300 0x7f6ff40d37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.105+0000 7f6ffa037640 1 -- 172.21.15.105:0/2901817623 >> v2:172.21.15.148:3300/0 conn(0x7f6ff415d300 msgr2=0x7f6ff40d37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:20.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.105+0000 7f6ffa037640 1 --2- 172.21.15.105:0/2901817623 >> v2:172.21.15.148:3300/0 conn(0x7f6ff415d300 0x7f6ff40d37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.105+0000 7f6ffa037640 1 -- 172.21.15.105:0/2901817623 >> v1:172.21.15.148:6789/0 conn(0x7f6ff40d3ce0 legacy=0x7f6ff40d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:20.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.105+0000 7f6ffa037640 1 -- 172.21.15.105:0/2901817623 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff40d0eb0 con 0x7f6ff40d1d60 2023-11-28T18:25:20.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.105+0000 7f6ffa037640 1 --2- 172.21.15.105:0/2901817623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ff40d1d60 0x7f6ff40c7510 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6ff408c7e0 tx=0x7f6fec01c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359ca14ea5d2867b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:20.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.105+0000 7f6ff8834640 1 -- 172.21.15.105:0/2901817623 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec0a5790 con 0x7f6ff40d1d60 2023-11-28T18:25:20.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.105+0000 7f6ff8834640 1 -- 172.21.15.105:0/2901817623 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fec0a58f0 con 0x7f6ff40d1d60 2023-11-28T18:25:20.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.105+0000 7f6ff8834640 1 -- 172.21.15.105:0/2901817623 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec0a5c80 con 0x7f6ff40d1d60 2023-11-28T18:25:20.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.105+0000 7f6ffba9a640 1 -- 172.21.15.105:0/2901817623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ff40d1d60 msgr2=0x7f6ff40c7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:20.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.105+0000 7f6ffba9a640 1 --2- 172.21.15.105:0/2901817623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ff40d1d60 0x7f6ff40c7510 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6ff408c7e0 tx=0x7f6fec01c8d0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.105+0000 7f6ffba9a640 1 -- 172.21.15.105:0/2901817623 shutdown_connections 2023-11-28T18:25:20.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.105+0000 7f6ffba9a640 1 --2- 172.21.15.105:0/2901817623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ff40d1d60 0x7f6ff40c7510 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.105+0000 7f6ffba9a640 1 --2- 172.21.15.105:0/2901817623 >> v2:172.21.15.148:3300/0 conn(0x7f6ff415d300 0x7f6ff40d37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.105+0000 7f6ffba9a640 1 -- 172.21.15.105:0/2901817623 >> 172.21.15.105:0/2901817623 conn(0x7f6ff409c360 msgr2=0x7f6ff40d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:20.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ffba9a640 1 -- 172.21.15.105:0/2901817623 shutdown_connections 2023-11-28T18:25:20.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ffba9a640 1 -- 172.21.15.105:0/2901817623 wait complete. 2023-11-28T18:25:20.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ffba9a640 1 Processor -- start 2023-11-28T18:25:20.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ffba9a640 1 -- start start 2023-11-28T18:25:20.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ffba9a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ff40d1d60 0x7f6ff4151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ffba9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ff40d3ce0 0x7f6ff415fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ffba9a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ff415d300 0x7f6ff4152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ffba9a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff415f600 con 0x7f6ff40d3ce0 2023-11-28T18:25:20.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ffba9a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff415e9f0 con 0x7f6ff415d300 2023-11-28T18:25:20.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ffba9a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff415ecd0 con 0x7f6ff40d1d60 2023-11-28T18:25:20.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ff9035640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ff40d3ce0 0x7f6ff415fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ff9035640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ff40d3ce0 0x7f6ff415fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40544/0 (socket says 172.21.15.105:40544) 2023-11-28T18:25:20.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ff9035640 1 -- 172.21.15.105:0/3053909102 learned_addr learned my addr 172.21.15.105:0/3053909102 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:20.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ffa037640 1 --2- 172.21.15.105:0/3053909102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ff415d300 0x7f6ff4152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ff9836640 1 --2- 172.21.15.105:0/3053909102 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ff40d1d60 0x7f6ff4151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ff9035640 1 -- 172.21.15.105:0/3053909102 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ff40d1d60 msgr2=0x7f6ff4151be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:20.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ff9035640 1 --2- 172.21.15.105:0/3053909102 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ff40d1d60 0x7f6ff4151be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ff9035640 1 -- 172.21.15.105:0/3053909102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ff415d300 msgr2=0x7f6ff4152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:20.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ff9035640 1 --2- 172.21.15.105:0/3053909102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ff415d300 0x7f6ff4152410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ff9035640 1 -- 172.21.15.105:0/3053909102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec009b30 con 0x7f6ff40d3ce0 2023-11-28T18:25:20.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ff9836640 1 --2- 172.21.15.105:0/3053909102 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ff40d1d60 0x7f6ff4151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:20.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ff9035640 1 --2- 172.21.15.105:0/3053909102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ff40d3ce0 0x7f6ff415fa90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400c450 tx=0x7f6fe400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:20.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6fea7fc640 1 -- 172.21.15.105:0/3053909102 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4012830 con 0x7f6ff40d3ce0 2023-11-28T18:25:20.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6fea7fc640 1 -- 172.21.15.105:0/3053909102 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe40129b0 con 0x7f6ff40d3ce0 2023-11-28T18:25:20.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ffba9a640 1 -- 172.21.15.105:0/3053909102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff415efb0 con 0x7f6ff40d3ce0 2023-11-28T18:25:20.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6fea7fc640 1 -- 172.21.15.105:0/3053909102 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4012d60 con 0x7f6ff40d3ce0 2023-11-28T18:25:20.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.109+0000 7f6ffba9a640 1 -- 172.21.15.105:0/3053909102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff4152120 con 0x7f6ff40d3ce0 2023-11-28T18:25:20.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.113+0000 7f6fea7fc640 1 -- 172.21.15.105:0/3053909102 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f6fe401c460 con 0x7f6ff40d3ce0 2023-11-28T18:25:20.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.113+0000 7f6fea7fc640 1 --2- 172.21.15.105:0/3053909102 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6fd80668d0 0x7f6fd8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.113+0000 7f6fea7fc640 1 -- 172.21.15.105:0/3053909102 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6fe4084350 con 0x7f6ff40d3ce0 2023-11-28T18:25:20.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.113+0000 7f6ff9836640 1 --2- 172.21.15.105:0/3053909102 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6fd80668d0 0x7f6fd8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.113+0000 7f6ffee04000 1 Processor -- start 2023-11-28T18:25:20.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.113+0000 7f6ffee04000 1 -- start start 2023-11-28T18:25:20.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.113+0000 7f6ff9836640 1 --2- 172.21.15.105:0/3053909102 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6fd80668d0 0x7f6fd8068d90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000b9f0 tx=0x7f6ff0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:20.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffee04000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589be831d30 0x5589be832110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffee04000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5589be832650 0x5589be7e4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffee04000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be7e5500 0x5589be7e58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffee04000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589be6ae110 con 0x5589be7e5500 2023-11-28T18:25:20.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffee04000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589be63e9a0 con 0x5589be831d30 2023-11-28T18:25:20.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffee04000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589be63bed0 con 0x5589be832650 2023-11-28T18:25:20.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffa037640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be7e5500 0x5589be7e58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffa037640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be7e5500 0x5589be7e58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40550/0 (socket says 172.21.15.105:40550) 2023-11-28T18:25:20.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffa037640 1 -- 172.21.15.105:0/3215159941 learned_addr learned my addr 172.21.15.105:0/3215159941 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:20.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ff9836640 1 --2- 172.21.15.105:0/3215159941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589be831d30 0x5589be832110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ff9035640 1 --2- 172.21.15.105:0/3215159941 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5589be832650 0x5589be7e4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffa037640 1 -- 172.21.15.105:0/3215159941 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5589be832650 msgr2=0x5589be7e4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:20.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffa037640 1 --2- 172.21.15.105:0/3215159941 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5589be832650 0x5589be7e4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffa037640 1 -- 172.21.15.105:0/3215159941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589be831d30 msgr2=0x5589be832110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:20.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffa037640 1 --2- 172.21.15.105:0/3215159941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589be831d30 0x5589be832110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffa037640 1 -- 172.21.15.105:0/3215159941 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589be32f890 con 0x5589be7e5500 2023-11-28T18:25:20.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffa037640 1 --2- 172.21.15.105:0/3215159941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be7e5500 0x5589be7e58e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0af040 tx=0x7f6fec004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417d67d39a628769 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:20.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6fd77fe640 1 -- 172.21.15.105:0/3215159941 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec0a5c60 con 0x5589be7e5500 2023-11-28T18:25:20.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6fd77fe640 1 -- 172.21.15.105:0/3215159941 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fec0a5de0 con 0x5589be7e5500 2023-11-28T18:25:20.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6fd77fe640 1 -- 172.21.15.105:0/3215159941 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec0b6630 con 0x5589be7e5500 2023-11-28T18:25:20.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffee04000 1 -- 172.21.15.105:0/3215159941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be7e5500 msgr2=0x5589be7e58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:20.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffee04000 1 --2- 172.21.15.105:0/3215159941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be7e5500 0x5589be7e58e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0af040 tx=0x7f6fec004ac0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffee04000 1 -- 172.21.15.105:0/3215159941 shutdown_connections 2023-11-28T18:25:20.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffee04000 1 --2- 172.21.15.105:0/3215159941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be7e5500 0x5589be7e58e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffee04000 1 --2- 172.21.15.105:0/3215159941 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5589be832650 0x5589be7e4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffee04000 1 --2- 172.21.15.105:0/3215159941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589be831d30 0x5589be832110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffee04000 1 -- 172.21.15.105:0/3215159941 >> 172.21.15.105:0/3215159941 conn(0x5589be8128c0 msgr2=0x5589be814cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:20.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffee04000 1 -- 172.21.15.105:0/3215159941 shutdown_connections 2023-11-28T18:25:20.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffee04000 1 -- 172.21.15.105:0/3215159941 wait complete. 2023-11-28T18:25:20.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffee04000 1 -- start start 2023-11-28T18:25:20.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.117+0000 7f6ffee04000 1 Processor -- start 2023-11-28T18:25:20.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ffee04000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be7e5500 0x5589be7eb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ffee04000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589be831d30 0x5589be7eba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ffee04000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5589be832650 0x5589be7ebfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ffee04000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589be6377c0 con 0x5589be7e5500 2023-11-28T18:25:20.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ffee04000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589be5efbf0 con 0x5589be831d30 2023-11-28T18:25:20.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ffee04000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589be5f5a90 con 0x5589be832650 2023-11-28T18:25:20.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ffee04000 10 client.0 _ll_register_callbacks cb 0x5589be7d2e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:25:20.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ff9836640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be7e5500 0x5589be7eb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ff9836640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be7e5500 0x5589be7eb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40566/0 (socket says 172.21.15.105:40566) 2023-11-28T18:25:20.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ff9836640 1 -- 172.21.15.105:0/1849097365 learned_addr learned my addr 172.21.15.105:0/1849097365 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:20.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ffa037640 1 --2- 172.21.15.105:0/1849097365 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5589be832650 0x5589be7ebfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ff9035640 1 --2- 172.21.15.105:0/1849097365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589be831d30 0x5589be7eba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ff9836640 1 -- 172.21.15.105:0/1849097365 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5589be832650 msgr2=0x5589be7ebfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:20.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ff9836640 1 --2- 172.21.15.105:0/1849097365 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5589be832650 0x5589be7ebfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ff9836640 1 -- 172.21.15.105:0/1849097365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589be831d30 msgr2=0x5589be7eba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:20.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ff9836640 1 --2- 172.21.15.105:0/1849097365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589be831d30 0x5589be7eba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ff9836640 1 -- 172.21.15.105:0/1849097365 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff40d0eb0 con 0x5589be7e5500 2023-11-28T18:25:20.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ffa037640 1 --2- 172.21.15.105:0/1849097365 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5589be832650 0x5589be7ebfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:20.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ff9836640 1 --2- 172.21.15.105:0/1849097365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be7e5500 0x5589be7eb520 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00146e0 tx=0x7f6ff00046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748b4f9d7a045bd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:20.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6fd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:25:20.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6fd4ff9640 1 -- 172.21.15.105:0/1849097365 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff00158e0 con 0x5589be7e5500 2023-11-28T18:25:20.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6fd4ff9640 1 -- 172.21.15.105:0/1849097365 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6ff0015a40 con 0x5589be7e5500 2023-11-28T18:25:20.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ffee04000 1 -- 172.21.15.105:0/1849097365 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5589be5f9470 con 0x5589be7e5500 2023-11-28T18:25:20.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6fd4ff9640 1 -- 172.21.15.105:0/1849097365 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ff0015c00 con 0x5589be7e5500 2023-11-28T18:25:20.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ffee04000 10 client.5772 fetch_fsmap learned FSMap version 23 2023-11-28T18:25:20.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ffee04000 1 -- 172.21.15.105:0/1849097365 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x5589be5f8190 con 0x5589be7e5500 2023-11-28T18:25:20.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6fd4ff9640 1 -- 172.21.15.105:0/1849097365 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5589be5f9470 con 0x5589be7e5500 2023-11-28T18:25:20.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6fd4ff9640 1 -- 172.21.15.105:0/1849097365 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 23) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f6ff002a440 con 0x5589be7e5500 2023-11-28T18:25:20.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ffee04000 10 client.5772 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T18:25:20.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ffee04000 10 client.5772 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:25:20.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ffee04000 20 client.5772 populate_metadata read hostname 'smithi105' 2023-11-28T18:25:20.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ffee04000 1 --2- 172.21.15.105:0/1849097365 >> [v2:172.21.15.105:6834/681913204,v1:172.21.15.105:6836/681913204] conn(0x5589be7ec9b0 0x5589be843ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ffee04000 4 client.5772 mds_command: new command op to 5463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:25:20.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ffee04000 1 -- 172.21.15.105:0/1849097365 --> [v2:172.21.15.105:6834/681913204,v1:172.21.15.105:6836/681913204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589be5fcb50 con 0x5589be7ec9b0 2023-11-28T18:25:20.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ff9035640 1 --2- 172.21.15.105:0/1849097365 >> [v2:172.21.15.105:6834/681913204,v1:172.21.15.105:6836/681913204] conn(0x5589be7ec9b0 0x5589be843ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6fd4ff9640 10 client.5772 ms_handle_connect on v2:172.21.15.105:6834/681913204 2023-11-28T18:25:20.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.121+0000 7f6ff9035640 1 --2- 172.21.15.105:0/1849097365 >> [v2:172.21.15.105:6834/681913204,v1:172.21.15.105:6836/681913204] conn(0x5589be7ec9b0 0x5589be843ff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b30c26b67955591a server_cookie=81ab123a05c94351 in_seq=0 out_seq=0 2023-11-28T18:25:20.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.125+0000 7f6fd4ff9640 1 -- 172.21.15.105:0/1849097365 <== mds.1 v2:172.21.15.105:6834/681913204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5589be5fcb50 con 0x5589be7ec9b0 2023-11-28T18:25:20.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.125+0000 7f6fd4ff9640 10 client.5772 handle_command_reply: tid=0 2023-11-28T18:25:20.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.125+0000 7f6ffee04000 1 client.5772 shutdown 2023-11-28T18:25:20.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.125+0000 7f6ffee04000 1 -- 172.21.15.105:0/1849097365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be7e5500 msgr2=0x5589be7eb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:20.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.125+0000 7f6ffee04000 1 --2- 172.21.15.105:0/1849097365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be7e5500 0x5589be7eb520 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00146e0 tx=0x7f6ff00046d0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.125+0000 7f6ffee04000 1 -- 172.21.15.105:0/1849097365 shutdown_connections 2023-11-28T18:25:20.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.125+0000 7f6ffee04000 1 --2- 172.21.15.105:0/1849097365 >> [v2:172.21.15.105:6834/681913204,v1:172.21.15.105:6836/681913204] conn(0x5589be7ec9b0 0x5589be843ff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.125+0000 7f6ffee04000 1 --2- 172.21.15.105:0/1849097365 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5589be832650 0x5589be7ebfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.125+0000 7f6ffee04000 1 --2- 172.21.15.105:0/1849097365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589be831d30 0x5589be7eba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.125+0000 7f6fd4ff9640 0 client.5772 ms_handle_reset on v2:172.21.15.105:6834/681913204 2023-11-28T18:25:20.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.125+0000 7f6ffee04000 1 --2- 172.21.15.105:0/1849097365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be7e5500 0x5589be7eb520 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.125+0000 7f6ffee04000 1 -- 172.21.15.105:0/1849097365 >> 172.21.15.105:0/1849097365 conn(0x5589be8128c0 msgr2=0x5589be7e46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:20.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.125+0000 7f6ffee04000 1 -- 172.21.15.105:0/1849097365 shutdown_connections 2023-11-28T18:25:20.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.125+0000 7f6ffee04000 1 -- 172.21.15.105:0/1849097365 wait complete. 2023-11-28T18:25:20.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.125+0000 7f6ffee04000 20 client.5772 trim_cache size 0 max 16384 2023-11-28T18:25:20.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ffee04000 1 Processor -- start 2023-11-28T18:25:20.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ffee04000 1 -- start start 2023-11-28T18:25:20.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ffee04000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589be7e5500 0x5589be8113a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ffee04000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be831d30 0x5589be8118e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ffee04000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5589be832650 0x5589be843ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ffee04000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589be2f0940 con 0x5589be831d30 2023-11-28T18:25:20.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ffee04000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589be60cb70 con 0x5589be7e5500 2023-11-28T18:25:20.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ffee04000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589be634bc0 con 0x5589be832650 2023-11-28T18:25:20.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ff9035640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be831d30 0x5589be8118e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ff9035640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be831d30 0x5589be8118e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40574/0 (socket says 172.21.15.105:40574) 2023-11-28T18:25:20.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ff9035640 1 -- 172.21.15.105:0/2925198710 learned_addr learned my addr 172.21.15.105:0/2925198710 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:20.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ff9836640 1 --2- 172.21.15.105:0/2925198710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589be7e5500 0x5589be8113a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ffa037640 1 --2- 172.21.15.105:0/2925198710 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5589be832650 0x5589be843ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ff9035640 1 -- 172.21.15.105:0/2925198710 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5589be832650 msgr2=0x5589be843ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ff9035640 1 --2- 172.21.15.105:0/2925198710 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5589be832650 0x5589be843ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ff9035640 1 -- 172.21.15.105:0/2925198710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589be7e5500 msgr2=0x5589be8113a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ff9035640 1 --2- 172.21.15.105:0/2925198710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589be7e5500 0x5589be8113a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ff9035640 1 -- 172.21.15.105:0/2925198710 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589be5de490 con 0x5589be831d30 2023-11-28T18:25:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ff9035640 1 --2- 172.21.15.105:0/2925198710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be831d30 0x5589be8118e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4055070 tx=0x7f6fe40825e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00c51c03168a9dd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6fd7fff640 1 -- 172.21.15.105:0/2925198710 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe40840b0 con 0x5589be831d30 2023-11-28T18:25:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6fd7fff640 1 -- 172.21.15.105:0/2925198710 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe4082790 con 0x5589be831d30 2023-11-28T18:25:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6fd7fff640 1 -- 172.21.15.105:0/2925198710 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe40517f0 con 0x5589be831d30 2023-11-28T18:25:20.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ffee04000 1 -- 172.21.15.105:0/2925198710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be831d30 msgr2=0x5589be8118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:20.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ffee04000 1 --2- 172.21.15.105:0/2925198710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be831d30 0x5589be8118e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4055070 tx=0x7f6fe40825e0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ffee04000 1 -- 172.21.15.105:0/2925198710 shutdown_connections 2023-11-28T18:25:20.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ffee04000 1 --2- 172.21.15.105:0/2925198710 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5589be832650 0x5589be843ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ffee04000 1 --2- 172.21.15.105:0/2925198710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be831d30 0x5589be8118e0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ffee04000 1 --2- 172.21.15.105:0/2925198710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589be7e5500 0x5589be8113a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ffee04000 1 -- 172.21.15.105:0/2925198710 >> 172.21.15.105:0/2925198710 conn(0x5589be8128c0 msgr2=0x5589be7e3720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:20.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ffee04000 1 -- 172.21.15.105:0/2925198710 shutdown_connections 2023-11-28T18:25:20.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ffee04000 1 -- 172.21.15.105:0/2925198710 wait complete. 2023-11-28T18:25:20.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ffee04000 1 -- start start 2023-11-28T18:25:20.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.141+0000 7f6ffee04000 1 Processor -- start 2023-11-28T18:25:20.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.145+0000 7f6ffee04000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589be7e5500 0x5589be892810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.145+0000 7f6ffee04000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5589be831d30 0x5589be892d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.145+0000 7f6ffee04000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be832650 0x5589be84da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.145+0000 7f6ffee04000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589be7df430 con 0x5589be832650 2023-11-28T18:25:20.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.145+0000 7f6ffee04000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589be701720 con 0x5589be7e5500 2023-11-28T18:25:20.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.145+0000 7f6ffee04000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589be603670 con 0x5589be831d30 2023-11-28T18:25:20.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.145+0000 7f6ffee04000 10 client.0 _ll_register_callbacks cb 0x5589be6ae060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:25:20.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.145+0000 7f6ffa037640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be832650 0x5589be84da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.145+0000 7f6ffa037640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be832650 0x5589be84da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40590/0 (socket says 172.21.15.105:40590) 2023-11-28T18:25:20.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.145+0000 7f6ffa037640 1 -- 172.21.15.105:0/3622336117 learned_addr learned my addr 172.21.15.105:0/3622336117 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:20.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.145+0000 7f6ff9836640 1 --2- 172.21.15.105:0/3622336117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589be7e5500 0x5589be892810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.145+0000 7f6ff9035640 1 --2- 172.21.15.105:0/3622336117 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5589be831d30 0x5589be892d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.145+0000 7f6ffa037640 1 -- 172.21.15.105:0/3622336117 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5589be831d30 msgr2=0x5589be892d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:20.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.145+0000 7f6ffa037640 1 --2- 172.21.15.105:0/3622336117 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5589be831d30 0x5589be892d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.145+0000 7f6ffa037640 1 -- 172.21.15.105:0/3622336117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589be7e5500 msgr2=0x5589be892810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:20.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.145+0000 7f6ffa037640 1 --2- 172.21.15.105:0/3622336117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589be7e5500 0x5589be892810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.145+0000 7f6ffa037640 1 -- 172.21.15.105:0/3622336117 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff4152120 con 0x5589be832650 2023-11-28T18:25:20.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.145+0000 7f6ffa037640 1 --2- 172.21.15.105:0/3622336117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be832650 0x5589be84da70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6fec01cdb0 tx=0x7f6fec01b110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa84788c13fd3891 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:20.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.145+0000 7f6fd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:25:20.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6fd4ff9640 1 -- 172.21.15.105:0/3622336117 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec0cd070 con 0x5589be832650 2023-11-28T18:25:20.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ffee04000 1 -- 172.21.15.105:0/3622336117 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5589be636110 con 0x5589be832650 2023-11-28T18:25:20.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6fd4ff9640 1 -- 172.21.15.105:0/3622336117 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6fec0039a0 con 0x5589be832650 2023-11-28T18:25:20.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6fd4ff9640 1 -- 172.21.15.105:0/3622336117 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fec0a5470 con 0x5589be832650 2023-11-28T18:25:20.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ffee04000 10 client.5778 fetch_fsmap learned FSMap version 23 2023-11-28T18:25:20.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ffee04000 1 -- 172.21.15.105:0/3622336117 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x5589be32fcc0 con 0x5589be832650 2023-11-28T18:25:20.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6fd4ff9640 1 -- 172.21.15.105:0/3622336117 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5589be636110 con 0x5589be832650 2023-11-28T18:25:20.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6fd4ff9640 1 -- 172.21.15.105:0/3622336117 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 23) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f6fec0d0020 con 0x5589be832650 2023-11-28T18:25:20.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ffee04000 10 client.5778 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T18:25:20.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ffee04000 10 client.5778 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:25:20.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ffee04000 20 client.5778 populate_metadata read hostname 'smithi105' 2023-11-28T18:25:20.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ffee04000 1 --2- 172.21.15.105:0/3622336117 >> [v2:172.21.15.105:6834/681913204,v1:172.21.15.105:6836/681913204] conn(0x5589be84f1b0 0x5589be89b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ffee04000 4 client.5778 mds_command: new command op to 5463 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:25:20.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ffee04000 1 -- 172.21.15.105:0/3622336117 --> [v2:172.21.15.105:6834/681913204,v1:172.21.15.105:6836/681913204] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5589be664650 con 0x5589be84f1b0 2023-11-28T18:25:20.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ff9836640 1 --2- 172.21.15.105:0/3622336117 >> [v2:172.21.15.105:6834/681913204,v1:172.21.15.105:6836/681913204] conn(0x5589be84f1b0 0x5589be89b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ff9836640 1 --2- 172.21.15.105:0/3622336117 >> [v2:172.21.15.105:6834/681913204,v1:172.21.15.105:6836/681913204] conn(0x5589be84f1b0 0x5589be89b680 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=781f3bddb4ababa2 server_cookie=bbc3f8ca0ef8162e in_seq=0 out_seq=0 2023-11-28T18:25:20.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6fd4ff9640 10 client.5778 ms_handle_connect on v2:172.21.15.105:6834/681913204 2023-11-28T18:25:20.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6fd4ff9640 1 -- 172.21.15.105:0/3622336117 <== mds.1 v2:172.21.15.105:6834/681913204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5589be664650 con 0x5589be84f1b0 2023-11-28T18:25:20.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6fd4ff9640 10 client.5778 handle_command_reply: tid=0 2023-11-28T18:25:20.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ffee04000 1 client.5778 shutdown 2023-11-28T18:25:20.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ffee04000 1 -- 172.21.15.105:0/3622336117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be832650 msgr2=0x5589be84da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:20.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ffee04000 1 --2- 172.21.15.105:0/3622336117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be832650 0x5589be84da70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6fec01cdb0 tx=0x7f6fec01b110 comp rx=0 tx=0).stop 2023-11-28T18:25:20.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ffee04000 1 -- 172.21.15.105:0/3622336117 shutdown_connections 2023-11-28T18:25:20.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ffee04000 1 --2- 172.21.15.105:0/3622336117 >> [v2:172.21.15.105:6834/681913204,v1:172.21.15.105:6836/681913204] conn(0x5589be84f1b0 0x5589be89b680 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ffee04000 1 --2- 172.21.15.105:0/3622336117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589be832650 0x5589be84da70 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ffee04000 1 --2- 172.21.15.105:0/3622336117 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5589be831d30 0x5589be892d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ffee04000 1 --2- 172.21.15.105:0/3622336117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589be7e5500 0x5589be892810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ffee04000 1 -- 172.21.15.105:0/3622336117 >> 172.21.15.105:0/3622336117 conn(0x5589be8128c0 msgr2=0x5589be7e3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:20.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6fd4ff9640 0 client.5778 ms_handle_reset on v2:172.21.15.105:6834/681913204 2023-11-28T18:25:20.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ffee04000 1 -- 172.21.15.105:0/3622336117 shutdown_connections 2023-11-28T18:25:20.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ffee04000 1 -- 172.21.15.105:0/3622336117 wait complete. 2023-11-28T18:25:20.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.149+0000 7f6ffee04000 20 client.5778 trim_cache size 0 max 16384 2023-11-28T18:25:20.162 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:25:20.162 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:25:20.162 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 1, 2023-11-28T18:25:20.163 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5463, 2023-11-28T18:25:20.163 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:25:20.163 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:25:20.163 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:25:20.163 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 3.0244031859999998, 2023-11-28T18:25:20.164 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 23, 2023-11-28T18:25:20.164 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 40, 2023-11-28T18:25:20.164 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 40, 2023-11-28T18:25:20.164 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 31.470774587000001 2023-11-28T18:25:20.164 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:25:20.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.153+0000 7f6ffba9a640 1 -- 172.21.15.105:0/3053909102 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6fd80668d0 msgr2=0x7f6fd8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:20.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.153+0000 7f6ffba9a640 1 --2- 172.21.15.105:0/3053909102 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6fd80668d0 0x7f6fd8068d90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000b9f0 tx=0x7f6ff0008040 comp rx=0 tx=0).stop 2023-11-28T18:25:20.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.153+0000 7f6ffba9a640 1 -- 172.21.15.105:0/3053909102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ff40d3ce0 msgr2=0x7f6ff415fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:20.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.153+0000 7f6ffba9a640 1 --2- 172.21.15.105:0/3053909102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ff40d3ce0 0x7f6ff415fa90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400c450 tx=0x7f6fe400c910 comp rx=0 tx=0).stop 2023-11-28T18:25:20.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.153+0000 7f6ffba9a640 1 -- 172.21.15.105:0/3053909102 shutdown_connections 2023-11-28T18:25:20.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.153+0000 7f6ffba9a640 1 --2- 172.21.15.105:0/3053909102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ff415d300 0x7f6ff4152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.153+0000 7f6ffba9a640 1 --2- 172.21.15.105:0/3053909102 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6fd80668d0 0x7f6fd8068d90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.153+0000 7f6ffba9a640 1 --2- 172.21.15.105:0/3053909102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ff40d3ce0 0x7f6ff415fa90 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.153+0000 7f6ffba9a640 1 --2- 172.21.15.105:0/3053909102 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ff40d1d60 0x7f6ff4151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.153+0000 7f6ffba9a640 1 -- 172.21.15.105:0/3053909102 >> 172.21.15.105:0/3053909102 conn(0x7f6ff409c360 msgr2=0x7f6ff40b5480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:20.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.153+0000 7f6ffba9a640 1 -- 172.21.15.105:0/3053909102 shutdown_connections 2023-11-28T18:25:20.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.153+0000 7f6ffba9a640 1 -- 172.21.15.105:0/3053909102 wait complete. 2023-11-28T18:25:20.176 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T18:25:20.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddf12a640 1 Processor -- start 2023-11-28T18:25:20.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddf12a640 1 -- start start 2023-11-28T18:25:20.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddf12a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbdd815d300 0x7fbdd80d37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddf12a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbdd80d3ce0 0x7fbdd80d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddf12a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbdd80d1b90 0x7fbdd80cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddf12a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdd80d1470 con 0x7fbdd80d1b90 2023-11-28T18:25:20.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddf12a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdd80d2040 con 0x7fbdd80d3ce0 2023-11-28T18:25:20.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddf12a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd80cefa0 con 0x7fbdd815d300 2023-11-28T18:25:20.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddd6c7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbdd80d1b90 0x7fbdd80cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddd6c7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbdd80d1b90 0x7fbdd80cdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40598/0 (socket says 172.21.15.105:40598) 2023-11-28T18:25:20.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddd6c7640 1 -- 172.21.15.105:0/1964206330 learned_addr learned my addr 172.21.15.105:0/1964206330 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:20.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbdcffff640 1 --2- 172.21.15.105:0/1964206330 >> v2:172.21.15.148:3300/0 conn(0x7fbdd80d3ce0 0x7fbdd80d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddcec6640 1 --2- 172.21.15.105:0/1964206330 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbdd815d300 0x7fbdd80d37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddd6c7640 1 -- 172.21.15.105:0/1964206330 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbdd815d300 msgr2=0x7fbdd80d37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:20.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddd6c7640 1 --2- 172.21.15.105:0/1964206330 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbdd815d300 0x7fbdd80d37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddd6c7640 1 -- 172.21.15.105:0/1964206330 >> v2:172.21.15.148:3300/0 conn(0x7fbdd80d3ce0 msgr2=0x7fbdd80d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:20.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddd6c7640 1 --2- 172.21.15.105:0/1964206330 >> v2:172.21.15.148:3300/0 conn(0x7fbdd80d3ce0 0x7fbdd80d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddd6c7640 1 -- 172.21.15.105:0/1964206330 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd80d0c60 con 0x7fbdd80d1b90 2023-11-28T18:25:20.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbdcffff640 1 --2- 172.21.15.105:0/1964206330 >> v2:172.21.15.148:3300/0 conn(0x7fbdd80d3ce0 0x7fbdd80d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:20.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddd6c7640 1 --2- 172.21.15.105:0/1964206330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdd80d1b90 0x7fbdd80cdb60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbdd808c9d0 tx=0x7fbdd001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f85b9958a9fc959 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:20.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbdcf7fe640 1 -- 172.21.15.105:0/1964206330 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd00a5790 con 0x7fbdd80d1b90 2023-11-28T18:25:20.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbdcf7fe640 1 -- 172.21.15.105:0/1964206330 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd00a58f0 con 0x7fbdd80d1b90 2023-11-28T18:25:20.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbdcf7fe640 1 -- 172.21.15.105:0/1964206330 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd00a5c80 con 0x7fbdd80d1b90 2023-11-28T18:25:20.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddf12a640 1 -- 172.21.15.105:0/1964206330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdd80d1b90 msgr2=0x7fbdd80cdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:20.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddf12a640 1 --2- 172.21.15.105:0/1964206330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdd80d1b90 0x7fbdd80cdb60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbdd808c9d0 tx=0x7fbdd001c910 comp rx=0 tx=0).stop 2023-11-28T18:25:20.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddf12a640 1 -- 172.21.15.105:0/1964206330 shutdown_connections 2023-11-28T18:25:20.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddf12a640 1 --2- 172.21.15.105:0/1964206330 >> v2:172.21.15.148:3300/0 conn(0x7fbdd80d3ce0 0x7fbdd80d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddf12a640 1 --2- 172.21.15.105:0/1964206330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdd80d1b90 0x7fbdd80cdb60 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddf12a640 1 --2- 172.21.15.105:0/1964206330 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbdd815d300 0x7fbdd80d37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddf12a640 1 -- 172.21.15.105:0/1964206330 >> 172.21.15.105:0/1964206330 conn(0x7fbdd809c360 msgr2=0x7fbdd80d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:20.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddf12a640 1 -- 172.21.15.105:0/1964206330 shutdown_connections 2023-11-28T18:25:20.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.313+0000 7fbddf12a640 1 -- 172.21.15.105:0/1964206330 wait complete. 2023-11-28T18:25:20.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbddf12a640 1 Processor -- start 2023-11-28T18:25:20.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbddf12a640 1 -- start start 2023-11-28T18:25:20.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbddf12a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbdd80d1b90 0x7fbdd80cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbddf12a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdd80d3ce0 0x7fbdd81603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbddf12a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdd815d300 0x7fbdd8154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbddf12a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd8157740 con 0x7fbdd80d3ce0 2023-11-28T18:25:20.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbddf12a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd8158e20 con 0x7fbdd815d300 2023-11-28T18:25:20.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbddf12a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd8158b10 con 0x7fbdd80d1b90 2023-11-28T18:25:20.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbdcffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdd80d3ce0 0x7fbdd81603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbdcffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdd80d3ce0 0x7fbdd81603c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40606/0 (socket says 172.21.15.105:40606) 2023-11-28T18:25:20.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbdcffff640 1 -- 172.21.15.105:0/1224834034 learned_addr learned my addr 172.21.15.105:0/1224834034 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:20.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbddcec6640 1 --2- 172.21.15.105:0/1224834034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbdd80d1b90 0x7fbdd80cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbddd6c7640 1 --2- 172.21.15.105:0/1224834034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdd815d300 0x7fbdd8154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbdcffff640 1 -- 172.21.15.105:0/1224834034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbdd80d1b90 msgr2=0x7fbdd80cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:20.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbdcffff640 1 --2- 172.21.15.105:0/1224834034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbdd80d1b90 0x7fbdd80cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbdcffff640 1 -- 172.21.15.105:0/1224834034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdd815d300 msgr2=0x7fbdd8154f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:20.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbdcffff640 1 --2- 172.21.15.105:0/1224834034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdd815d300 0x7fbdd8154f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbdcffff640 1 -- 172.21.15.105:0/1224834034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd0009b30 con 0x7fbdd80d3ce0 2023-11-28T18:25:20.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbdcffff640 1 --2- 172.21.15.105:0/1224834034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdd80d3ce0 0x7fbdd81603c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800e810 tx=0x7fbdc800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:20.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbdcd7fa640 1 -- 172.21.15.105:0/1224834034 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8004030 con 0x7fbdd80d3ce0 2023-11-28T18:25:20.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbdcd7fa640 1 -- 172.21.15.105:0/1224834034 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc8004190 con 0x7fbdd80d3ce0 2023-11-28T18:25:20.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbdcd7fa640 1 -- 172.21.15.105:0/1224834034 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8004500 con 0x7fbdd80d3ce0 2023-11-28T18:25:20.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbddf12a640 1 -- 172.21.15.105:0/1224834034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd8155ad0 con 0x7fbdd80d3ce0 2023-11-28T18:25:20.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.317+0000 7fbddf12a640 1 -- 172.21.15.105:0/1224834034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd8156090 con 0x7fbdd80d3ce0 2023-11-28T18:25:20.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.321+0000 7fbdcd7fa640 1 -- 172.21.15.105:0/1224834034 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fbdc800c6d0 con 0x7fbdd80d3ce0 2023-11-28T18:25:20.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.321+0000 7fbdcd7fa640 1 --2- 172.21.15.105:0/1224834034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbdc40668d0 0x7fbdc4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.321+0000 7fbdcd7fa640 1 -- 172.21.15.105:0/1224834034 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbdc80840a0 con 0x7fbdd80d3ce0 2023-11-28T18:25:20.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.321+0000 7fbddcec6640 1 --2- 172.21.15.105:0/1224834034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbdc40668d0 0x7fbdc4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.321+0000 7fbde249e000 1 Processor -- start 2023-11-28T18:25:20.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.321+0000 7fbde249e000 1 -- start start 2023-11-28T18:25:20.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.321+0000 7fbddcec6640 1 --2- 172.21.15.105:0/1224834034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbdc40668d0 0x7fbdc4068d90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbdd40079c0 tx=0x7fbdd4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:20.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.321+0000 7fbde249e000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e46ec6cd30 0x55e46ec6d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.321+0000 7fbde249e000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec6d650 0x55e46ec1ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.321+0000 7fbde249e000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e46ec20500 0x55e46ec208e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbde249e000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46eae9110 con 0x55e46ec6d650 2023-11-28T18:25:20.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbde249e000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46ea799a0 con 0x55e46ec20500 2023-11-28T18:25:20.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbde249e000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e46ea76ed0 con 0x55e46ec6cd30 2023-11-28T18:25:20.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbdcffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec6d650 0x55e46ec1ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbdcffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec6d650 0x55e46ec1ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40622/0 (socket says 172.21.15.105:40622) 2023-11-28T18:25:20.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbdcffff640 1 -- 172.21.15.105:0/2722311608 learned_addr learned my addr 172.21.15.105:0/2722311608 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:20.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbddcec6640 1 --2- 172.21.15.105:0/2722311608 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e46ec6cd30 0x55e46ec6d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbddd6c7640 1 --2- 172.21.15.105:0/2722311608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e46ec20500 0x55e46ec208e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbdcffff640 1 -- 172.21.15.105:0/2722311608 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e46ec6cd30 msgr2=0x55e46ec6d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:20.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbdcffff640 1 --2- 172.21.15.105:0/2722311608 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e46ec6cd30 0x55e46ec6d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbdcffff640 1 -- 172.21.15.105:0/2722311608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e46ec20500 msgr2=0x55e46ec208e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:20.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbdcffff640 1 --2- 172.21.15.105:0/2722311608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e46ec20500 0x55e46ec208e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbdcffff640 1 -- 172.21.15.105:0/2722311608 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e46e76a890 con 0x55e46ec6d650 2023-11-28T18:25:20.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbdcffff640 1 --2- 172.21.15.105:0/2722311608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec6d650 0x55e46ec1ffc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8054de0 tx=0x7fbdc8054ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763d417f863aeee2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:20.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbdbe7fc640 1 -- 172.21.15.105:0/2722311608 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc80824e0 con 0x55e46ec6d650 2023-11-28T18:25:20.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbdbe7fc640 1 -- 172.21.15.105:0/2722311608 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc8051560 con 0x55e46ec6d650 2023-11-28T18:25:20.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbdbe7fc640 1 -- 172.21.15.105:0/2722311608 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc805ce10 con 0x55e46ec6d650 2023-11-28T18:25:20.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbde249e000 1 -- 172.21.15.105:0/2722311608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec6d650 msgr2=0x55e46ec1ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:20.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbde249e000 1 --2- 172.21.15.105:0/2722311608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec6d650 0x55e46ec1ffc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8054de0 tx=0x7fbdc8054ac0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbde249e000 1 -- 172.21.15.105:0/2722311608 shutdown_connections 2023-11-28T18:25:20.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbde249e000 1 --2- 172.21.15.105:0/2722311608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e46ec20500 0x55e46ec208e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbde249e000 1 --2- 172.21.15.105:0/2722311608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec6d650 0x55e46ec1ffc0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbde249e000 1 --2- 172.21.15.105:0/2722311608 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e46ec6cd30 0x55e46ec6d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbde249e000 1 -- 172.21.15.105:0/2722311608 >> 172.21.15.105:0/2722311608 conn(0x55e46ec4d8c0 msgr2=0x55e46ec4fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:20.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbde249e000 1 -- 172.21.15.105:0/2722311608 shutdown_connections 2023-11-28T18:25:20.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbde249e000 1 -- 172.21.15.105:0/2722311608 wait complete. 2023-11-28T18:25:20.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbde249e000 1 -- start start 2023-11-28T18:25:20.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbde249e000 1 Processor -- start 2023-11-28T18:25:20.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbde249e000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec20500 0x55e46ec264e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbde249e000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e46ec6cd30 0x55e46ec26a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbde249e000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e46ec6d650 0x55e46ec26f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbde249e000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46ea727c0 con 0x55e46ec20500 2023-11-28T18:25:20.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbde249e000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46ea2abf0 con 0x55e46ec6d650 2023-11-28T18:25:20.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbde249e000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e46ea30a90 con 0x55e46ec6cd30 2023-11-28T18:25:20.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.325+0000 7fbde249e000 10 client.0 _ll_register_callbacks cb 0x55e46ec0de20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:25:20.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbddcec6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec20500 0x55e46ec264e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbddcec6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec20500 0x55e46ec264e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40636/0 (socket says 172.21.15.105:40636) 2023-11-28T18:25:20.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbddcec6640 1 -- 172.21.15.105:0/2457767259 learned_addr learned my addr 172.21.15.105:0/2457767259 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:20.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbdcffff640 1 --2- 172.21.15.105:0/2457767259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e46ec6cd30 0x55e46ec26a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbddd6c7640 1 --2- 172.21.15.105:0/2457767259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e46ec6d650 0x55e46ec26f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbddcec6640 1 -- 172.21.15.105:0/2457767259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e46ec6cd30 msgr2=0x55e46ec26a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:20.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbddcec6640 1 --2- 172.21.15.105:0/2457767259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e46ec6cd30 0x55e46ec26a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbddcec6640 1 -- 172.21.15.105:0/2457767259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e46ec6d650 msgr2=0x55e46ec26f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:20.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbddcec6640 1 --2- 172.21.15.105:0/2457767259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e46ec6d650 0x55e46ec26f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbddcec6640 1 -- 172.21.15.105:0/2457767259 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd8156090 con 0x55e46ec20500 2023-11-28T18:25:20.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbddd6c7640 1 --2- 172.21.15.105:0/2457767259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e46ec6d650 0x55e46ec26f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:20.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbddcec6640 1 --2- 172.21.15.105:0/2457767259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec20500 0x55e46ec264e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4007400 tx=0x7fbdd4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5769deabf18dc1fb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:20.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbdb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:25:20.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbdb77fe640 1 -- 172.21.15.105:0/2457767259 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4014800 con 0x55e46ec20500 2023-11-28T18:25:20.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbdb77fe640 1 -- 172.21.15.105:0/2457767259 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbdd4014960 con 0x55e46ec20500 2023-11-28T18:25:20.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbde249e000 1 -- 172.21.15.105:0/2457767259 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e46ea34470 con 0x55e46ec20500 2023-11-28T18:25:20.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbdb77fe640 1 -- 172.21.15.105:0/2457767259 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdd4014b20 con 0x55e46ec20500 2023-11-28T18:25:20.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbde249e000 10 client.5790 fetch_fsmap learned FSMap version 23 2023-11-28T18:25:20.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbde249e000 1 -- 172.21.15.105:0/2457767259 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55e46ea33190 con 0x55e46ec20500 2023-11-28T18:25:20.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbdb77fe640 1 -- 172.21.15.105:0/2457767259 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e46ea34470 con 0x55e46ec20500 2023-11-28T18:25:20.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbdb77fe640 1 -- 172.21.15.105:0/2457767259 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 23) v1 ==== 3318+0+0 (secure 0 0 0) 0x7fbdd4014c80 con 0x55e46ec20500 2023-11-28T18:25:20.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbde249e000 10 client.5790 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T18:25:20.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbde249e000 10 client.5790 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:25:20.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbde249e000 20 client.5790 populate_metadata read hostname 'smithi105' 2023-11-28T18:25:20.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbde249e000 1 --2- 172.21.15.105:0/2457767259 >> [v2:172.21.15.105:6835/1876408690,v1:172.21.15.105:6837/1876408690] conn(0x55e46ec27970 0x55e46ec7eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbde249e000 4 client.5790 mds_command: new command op to 5469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:25:20.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbde249e000 1 -- 172.21.15.105:0/2457767259 --> [v2:172.21.15.105:6835/1876408690,v1:172.21.15.105:6837/1876408690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e46ea37b50 con 0x55e46ec27970 2023-11-28T18:25:20.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbdcffff640 1 --2- 172.21.15.105:0/2457767259 >> [v2:172.21.15.105:6835/1876408690,v1:172.21.15.105:6837/1876408690] conn(0x55e46ec27970 0x55e46ec7eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbdcffff640 1 --2- 172.21.15.105:0/2457767259 >> [v2:172.21.15.105:6835/1876408690,v1:172.21.15.105:6837/1876408690] conn(0x55e46ec27970 0x55e46ec7eff0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4939cd79bf052cc server_cookie=367c4430d6d1845a in_seq=0 out_seq=0 2023-11-28T18:25:20.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbdb77fe640 10 client.5790 ms_handle_connect on v2:172.21.15.105:6835/1876408690 2023-11-28T18:25:20.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbdb77fe640 1 -- 172.21.15.105:0/2457767259 <== mds.0 v2:172.21.15.105:6835/1876408690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e46ea37b50 con 0x55e46ec27970 2023-11-28T18:25:20.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbdb77fe640 10 client.5790 handle_command_reply: tid=0 2023-11-28T18:25:20.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbde249e000 1 client.5790 shutdown 2023-11-28T18:25:20.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbde249e000 1 -- 172.21.15.105:0/2457767259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec20500 msgr2=0x55e46ec264e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:20.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbde249e000 1 --2- 172.21.15.105:0/2457767259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec20500 0x55e46ec264e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4007400 tx=0x7fbdd4004740 comp rx=0 tx=0).stop 2023-11-28T18:25:20.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbde249e000 1 -- 172.21.15.105:0/2457767259 shutdown_connections 2023-11-28T18:25:20.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbde249e000 1 --2- 172.21.15.105:0/2457767259 >> [v2:172.21.15.105:6835/1876408690,v1:172.21.15.105:6837/1876408690] conn(0x55e46ec27970 0x55e46ec7eff0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbde249e000 1 --2- 172.21.15.105:0/2457767259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e46ec6d650 0x55e46ec26f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbde249e000 1 --2- 172.21.15.105:0/2457767259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e46ec6cd30 0x55e46ec26a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbde249e000 1 --2- 172.21.15.105:0/2457767259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec20500 0x55e46ec264e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbdb77fe640 0 client.5790 ms_handle_reset on v2:172.21.15.105:6835/1876408690 2023-11-28T18:25:20.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbde249e000 1 -- 172.21.15.105:0/2457767259 >> 172.21.15.105:0/2457767259 conn(0x55e46ec4d8c0 msgr2=0x55e46ec1f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:20.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbde249e000 1 -- 172.21.15.105:0/2457767259 shutdown_connections 2023-11-28T18:25:20.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbde249e000 1 -- 172.21.15.105:0/2457767259 wait complete. 2023-11-28T18:25:20.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.329+0000 7fbde249e000 20 client.5790 trim_cache size 0 max 16384 2023-11-28T18:25:20.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 Processor -- start 2023-11-28T18:25:20.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 -- start start 2023-11-28T18:25:20.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e46ec20500 0x55e46ec4c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec6cd30 0x55e46ec4c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e46ec6d650 0x55e46ec7eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46e72b940 con 0x55e46ec6cd30 2023-11-28T18:25:20.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46eab3440 con 0x55e46ec20500 2023-11-28T18:25:20.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e46ea3b910 con 0x55e46ec6d650 2023-11-28T18:25:20.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbdcffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec6cd30 0x55e46ec4c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbdcffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec6cd30 0x55e46ec4c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40646/0 (socket says 172.21.15.105:40646) 2023-11-28T18:25:20.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbdcffff640 1 -- 172.21.15.105:0/2803411343 learned_addr learned my addr 172.21.15.105:0/2803411343 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:20.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbdcffff640 1 -- 172.21.15.105:0/2803411343 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e46ec6d650 msgr2=0x55e46ec7eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:20.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbdcffff640 1 --2- 172.21.15.105:0/2803411343 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e46ec6d650 0x55e46ec7eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbdcffff640 1 -- 172.21.15.105:0/2803411343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e46ec20500 msgr2=0x55e46ec4c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:20.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbdcffff640 1 --2- 172.21.15.105:0/2803411343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e46ec20500 0x55e46ec4c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbdcffff640 1 -- 172.21.15.105:0/2803411343 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e46ea19490 con 0x55e46ec6cd30 2023-11-28T18:25:20.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbdcffff640 1 --2- 172.21.15.105:0/2803411343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec6cd30 0x55e46ec4c8e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8054b50 tx=0x7fbdc805b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1391038cdd11e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:20.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbdbeffd640 1 -- 172.21.15.105:0/2803411343 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8054c20 con 0x55e46ec6cd30 2023-11-28T18:25:20.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbdbeffd640 1 -- 172.21.15.105:0/2803411343 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc8053270 con 0x55e46ec6cd30 2023-11-28T18:25:20.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbdbeffd640 1 -- 172.21.15.105:0/2803411343 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8082420 con 0x55e46ec6cd30 2023-11-28T18:25:20.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 -- 172.21.15.105:0/2803411343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec6cd30 msgr2=0x55e46ec4c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:20.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 --2- 172.21.15.105:0/2803411343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec6cd30 0x55e46ec4c8e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8054b50 tx=0x7fbdc805b000 comp rx=0 tx=0).stop 2023-11-28T18:25:20.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 -- 172.21.15.105:0/2803411343 shutdown_connections 2023-11-28T18:25:20.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 --2- 172.21.15.105:0/2803411343 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e46ec6d650 0x55e46ec7eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 --2- 172.21.15.105:0/2803411343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec6cd30 0x55e46ec4c8e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 --2- 172.21.15.105:0/2803411343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e46ec20500 0x55e46ec4c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 -- 172.21.15.105:0/2803411343 >> 172.21.15.105:0/2803411343 conn(0x55e46ec4d8c0 msgr2=0x55e46ec1e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:20.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 -- 172.21.15.105:0/2803411343 shutdown_connections 2023-11-28T18:25:20.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 -- 172.21.15.105:0/2803411343 wait complete. 2023-11-28T18:25:20.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 -- start start 2023-11-28T18:25:20.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 Processor -- start 2023-11-28T18:25:20.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e46ec20500 0x55e46eccd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec6cd30 0x55e46eccdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e46ec6d650 0x55e46ec88a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46ea27760 con 0x55e46ec6cd30 2023-11-28T18:25:20.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46ea3e670 con 0x55e46ec20500 2023-11-28T18:25:20.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e46eb0fde0 con 0x55e46ec6d650 2023-11-28T18:25:20.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbde249e000 10 client.0 _ll_register_callbacks cb 0x55e46eae9060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:25:20.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbdcffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec6cd30 0x55e46eccdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbdcffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec6cd30 0x55e46eccdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40652/0 (socket says 172.21.15.105:40652) 2023-11-28T18:25:20.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.349+0000 7fbdcffff640 1 -- 172.21.15.105:0/1730678215 learned_addr learned my addr 172.21.15.105:0/1730678215 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:20.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbddd6c7640 1 --2- 172.21.15.105:0/1730678215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e46ec6d650 0x55e46ec88a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbdcffff640 1 -- 172.21.15.105:0/1730678215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e46ec6d650 msgr2=0x55e46ec88a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:20.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbddcec6640 1 --2- 172.21.15.105:0/1730678215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e46ec20500 0x55e46eccd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbdcffff640 1 --2- 172.21.15.105:0/1730678215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e46ec6d650 0x55e46ec88a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbdcffff640 1 -- 172.21.15.105:0/1730678215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e46ec20500 msgr2=0x55e46eccd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:20.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbdcffff640 1 --2- 172.21.15.105:0/1730678215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e46ec20500 0x55e46eccd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbdcffff640 1 -- 172.21.15.105:0/1730678215 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc807f040 con 0x55e46ec6cd30 2023-11-28T18:25:20.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbddd6c7640 1 --2- 172.21.15.105:0/1730678215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e46ec6d650 0x55e46ec88a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:20.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbddcec6640 1 --2- 172.21.15.105:0/1730678215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e46ec20500 0x55e46eccd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:20.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbdcffff640 1 --2- 172.21.15.105:0/1730678215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec6cd30 0x55e46eccdd50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800c4c0 tx=0x7fbdc801d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31cad4702f576ebf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:20.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbdb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:25:20.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbdb77fe640 1 -- 172.21.15.105:0/1730678215 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8082420 con 0x55e46ec6cd30 2023-11-28T18:25:20.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbdb77fe640 1 -- 172.21.15.105:0/1730678215 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbdc8082580 con 0x55e46ec6cd30 2023-11-28T18:25:20.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbde249e000 1 -- 172.21.15.105:0/1730678215 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e46ea71110 con 0x55e46ec6cd30 2023-11-28T18:25:20.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbdb77fe640 1 -- 172.21.15.105:0/1730678215 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdc805cc10 con 0x55e46ec6cd30 2023-11-28T18:25:20.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbde249e000 10 client.5796 fetch_fsmap learned FSMap version 23 2023-11-28T18:25:20.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbde249e000 1 -- 172.21.15.105:0/1730678215 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55e46e76acc0 con 0x55e46ec6cd30 2023-11-28T18:25:20.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbdb77fe640 1 -- 172.21.15.105:0/1730678215 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e46ea71110 con 0x55e46ec6cd30 2023-11-28T18:25:20.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbdb77fe640 1 -- 172.21.15.105:0/1730678215 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 23) v1 ==== 3318+0+0 (secure 0 0 0) 0x7fbdc8079070 con 0x55e46ec6cd30 2023-11-28T18:25:20.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbde249e000 10 client.5796 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T18:25:20.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbde249e000 10 client.5796 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:25:20.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbde249e000 20 client.5796 populate_metadata read hostname 'smithi105' 2023-11-28T18:25:20.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbde249e000 1 --2- 172.21.15.105:0/1730678215 >> [v2:172.21.15.105:6835/1876408690,v1:172.21.15.105:6837/1876408690] conn(0x55e46ec893d0 0x55e46ec897b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:20.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbde249e000 4 client.5796 mds_command: new command op to 5469 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:25:20.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbde249e000 1 -- 172.21.15.105:0/1730678215 --> [v2:172.21.15.105:6835/1876408690,v1:172.21.15.105:6837/1876408690] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e46ea9f650 con 0x55e46ec893d0 2023-11-28T18:25:20.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbddcec6640 1 --2- 172.21.15.105:0/1730678215 >> [v2:172.21.15.105:6835/1876408690,v1:172.21.15.105:6837/1876408690] conn(0x55e46ec893d0 0x55e46ec897b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:20.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbddcec6640 1 --2- 172.21.15.105:0/1730678215 >> [v2:172.21.15.105:6835/1876408690,v1:172.21.15.105:6837/1876408690] conn(0x55e46ec893d0 0x55e46ec897b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ac97fe47ab29b55 server_cookie=9610c8bff796acf9 in_seq=0 out_seq=0 2023-11-28T18:25:20.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbdb77fe640 10 client.5796 ms_handle_connect on v2:172.21.15.105:6835/1876408690 2023-11-28T18:25:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbdb77fe640 1 -- 172.21.15.105:0/1730678215 <== mds.0 v2:172.21.15.105:6835/1876408690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55e46ea9f650 con 0x55e46ec893d0 2023-11-28T18:25:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbdb77fe640 10 client.5796 handle_command_reply: tid=0 2023-11-28T18:25:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbde249e000 1 client.5796 shutdown 2023-11-28T18:25:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbde249e000 1 -- 172.21.15.105:0/1730678215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec6cd30 msgr2=0x55e46eccdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbde249e000 1 --2- 172.21.15.105:0/1730678215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec6cd30 0x55e46eccdd50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800c4c0 tx=0x7fbdc801d070 comp rx=0 tx=0).stop 2023-11-28T18:25:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbde249e000 1 -- 172.21.15.105:0/1730678215 shutdown_connections 2023-11-28T18:25:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbde249e000 1 --2- 172.21.15.105:0/1730678215 >> [v2:172.21.15.105:6835/1876408690,v1:172.21.15.105:6837/1876408690] conn(0x55e46ec893d0 0x55e46ec897b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbde249e000 1 --2- 172.21.15.105:0/1730678215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e46ec6d650 0x55e46ec88a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbde249e000 1 --2- 172.21.15.105:0/1730678215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e46ec6cd30 0x55e46eccdd50 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbdb77fe640 0 client.5796 ms_handle_reset on v2:172.21.15.105:6835/1876408690 2023-11-28T18:25:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbde249e000 1 --2- 172.21.15.105:0/1730678215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e46ec20500 0x55e46eccd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbde249e000 1 -- 172.21.15.105:0/1730678215 >> 172.21.15.105:0/1730678215 conn(0x55e46ec4d8c0 msgr2=0x55e46ec1efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:20.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbde249e000 1 -- 172.21.15.105:0/1730678215 shutdown_connections 2023-11-28T18:25:20.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbde249e000 1 -- 172.21.15.105:0/1730678215 wait complete. 2023-11-28T18:25:20.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbde249e000 20 client.5796 trim_cache size 0 max 16384 2023-11-28T18:25:20.374 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:25:20.374 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:25:20.374 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2023-11-28T18:25:20.374 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5469, 2023-11-28T18:25:20.375 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:25:20.375 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:25:20.375 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:25:20.375 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 23.786564772999998, 2023-11-28T18:25:20.375 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 23, 2023-11-28T18:25:20.375 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 40, 2023-11-28T18:25:20.375 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 40, 2023-11-28T18:25:20.375 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 31.672206680999999 2023-11-28T18:25:20.375 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:25:20.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbddf12a640 1 -- 172.21.15.105:0/1224834034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbdc40668d0 msgr2=0x7fbdc4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:20.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbddf12a640 1 --2- 172.21.15.105:0/1224834034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbdc40668d0 0x7fbdc4068d90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbdd40079c0 tx=0x7fbdd4008040 comp rx=0 tx=0).stop 2023-11-28T18:25:20.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbddf12a640 1 -- 172.21.15.105:0/1224834034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdd80d3ce0 msgr2=0x7fbdd81603c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:20.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbddf12a640 1 --2- 172.21.15.105:0/1224834034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdd80d3ce0 0x7fbdd81603c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800e810 tx=0x7fbdc800ecd0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbddf12a640 1 -- 172.21.15.105:0/1224834034 shutdown_connections 2023-11-28T18:25:20.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbddf12a640 1 --2- 172.21.15.105:0/1224834034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdd815d300 0x7fbdd8154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbddf12a640 1 --2- 172.21.15.105:0/1224834034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbdc40668d0 0x7fbdc4068d90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbddf12a640 1 --2- 172.21.15.105:0/1224834034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdd80d3ce0 0x7fbdd81603c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbddf12a640 1 --2- 172.21.15.105:0/1224834034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbdd80d1b90 0x7fbdd80cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:20.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbddf12a640 1 -- 172.21.15.105:0/1224834034 >> 172.21.15.105:0/1224834034 conn(0x7fbdd809c360 msgr2=0x7fbdd80c93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:20.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbddf12a640 1 -- 172.21.15.105:0/1224834034 shutdown_connections 2023-11-28T18:25:20.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:20.353+0000 7fbddf12a640 1 -- 172.21.15.105:0/1224834034 wait complete. 2023-11-28T18:25:20.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:25:20.379 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-28T18:25:20.379 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:25:20.379 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p tree 2023-11-28T18:25:20.379 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:25:20.379 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:25:20.380 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:25:20.380 DEBUG:teuthology.orchestra.run.smithi105:> for ((i = 0; i < 100; i++)); do 2023-11-28T18:25:20.380 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p "tree/$i" 2023-11-28T18:25:20.380 DEBUG:teuthology.orchestra.run.smithi105:> echo file > "tree/$i/file" 2023-11-28T18:25:20.380 DEBUG:teuthology.orchestra.run.smithi105:> done 2023-11-28T18:25:20.380 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:25:20.391 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree 2023-11-28T18:25:20.395 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i = 0 )) 2023-11-28T18:25:20.395 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.395 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/0 2023-11-28T18:25:20.399 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.405 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.405 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.406 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/1 2023-11-28T18:25:20.413 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.416 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.416 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.416 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/2 2023-11-28T18:25:20.422 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.424 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.424 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.425 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/3 2023-11-28T18:25:20.430 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.433 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.433 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.433 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/4 2023-11-28T18:25:20.440 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.442 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.442 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.442 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/5 2023-11-28T18:25:20.449 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.451 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.452 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.452 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/6 2023-11-28T18:25:20.458 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.462 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.462 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.462 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/7 2023-11-28T18:25:20.468 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.469 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.470 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.470 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/8 2023-11-28T18:25:20.476 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.479 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.479 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.480 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/9 2023-11-28T18:25:20.485 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.487 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.487 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.487 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/10 2023-11-28T18:25:20.493 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.496 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.496 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.496 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/11 2023-11-28T18:25:20.502 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.504 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.504 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.504 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/12 2023-11-28T18:25:20.510 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.513 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.513 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.513 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/13 2023-11-28T18:25:20.520 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.522 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.523 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.523 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/14 2023-11-28T18:25:20.529 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.531 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.532 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.532 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/15 2023-11-28T18:25:20.533 INFO:tasks.ceph.mds.a.smithi105.stderr:2023-11-28T18:25:20.529+0000 7f86309ae640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-28T18:25:20.534 INFO:tasks.ceph.mds.a.smithi105.stderr:2023-11-28T18:25:20.529+0000 7f86309ae640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-11-28T18:25:20.538 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.540 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.541 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.541 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/16 2023-11-28T18:25:20.547 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.549 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.549 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.549 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/17 2023-11-28T18:25:20.556 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.558 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.558 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.558 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/18 2023-11-28T18:25:20.567 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.567 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.567 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.568 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/19 2023-11-28T18:25:20.574 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.576 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.576 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.576 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/20 2023-11-28T18:25:20.585 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.587 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.587 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.587 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/21 2023-11-28T18:25:20.593 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.595 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.595 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.596 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/22 2023-11-28T18:25:20.602 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.603 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.604 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.604 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/23 2023-11-28T18:25:20.612 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.614 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.614 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.614 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/24 2023-11-28T18:25:20.621 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.623 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.623 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.623 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/25 2023-11-28T18:25:20.632 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.634 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.634 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.634 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/26 2023-11-28T18:25:20.640 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.643 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.643 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.643 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/27 2023-11-28T18:25:20.652 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.654 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.654 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.654 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/28 2023-11-28T18:25:20.665 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.666 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.666 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.666 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/29 2023-11-28T18:25:20.672 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.674 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.674 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.674 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/30 2023-11-28T18:25:20.684 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.686 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.686 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.686 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/31 2023-11-28T18:25:20.693 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.695 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.695 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.695 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/32 2023-11-28T18:25:20.701 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.703 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.704 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.704 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/33 2023-11-28T18:25:20.710 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.712 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.712 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.712 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/34 2023-11-28T18:25:20.718 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.720 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.721 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.721 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/35 2023-11-28T18:25:20.726 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.728 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.729 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.729 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/36 2023-11-28T18:25:20.735 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.737 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.737 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.737 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/37 2023-11-28T18:25:20.743 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.746 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.746 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.746 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/38 2023-11-28T18:25:20.752 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.754 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.754 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.754 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/39 2023-11-28T18:25:20.763 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.765 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.765 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.765 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/40 2023-11-28T18:25:20.771 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.773 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.773 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.773 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/41 2023-11-28T18:25:20.781 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.783 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.783 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.783 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/42 2023-11-28T18:25:20.792 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.794 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.794 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.794 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/43 2023-11-28T18:25:20.803 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.805 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.805 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.805 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/44 2023-11-28T18:25:20.811 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.813 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.813 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.813 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/45 2023-11-28T18:25:20.821 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.823 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.824 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.824 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/46 2023-11-28T18:25:20.830 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.833 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.833 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.833 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/47 2023-11-28T18:25:20.841 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.844 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.844 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.844 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/48 2023-11-28T18:25:20.852 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.855 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.855 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.855 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/49 2023-11-28T18:25:20.865 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.866 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.866 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.866 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/50 2023-11-28T18:25:20.874 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.876 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.876 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.876 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/51 2023-11-28T18:25:20.883 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.885 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.886 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.887 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/52 2023-11-28T18:25:20.892 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.895 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.895 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.895 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/53 2023-11-28T18:25:20.901 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.904 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.904 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.904 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/54 2023-11-28T18:25:20.910 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.913 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.913 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.913 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/55 2023-11-28T18:25:20.920 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.922 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.922 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.922 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/56 2023-11-28T18:25:20.928 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.931 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.931 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.931 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/57 2023-11-28T18:25:20.938 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.940 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.940 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.940 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/58 2023-11-28T18:25:20.947 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.949 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.949 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.949 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/59 2023-11-28T18:25:20.956 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.959 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.959 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.959 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/60 2023-11-28T18:25:20.968 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.970 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.970 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.971 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/61 2023-11-28T18:25:20.976 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.980 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.980 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.980 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/62 2023-11-28T18:25:20.986 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.988 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.988 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.988 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/63 2023-11-28T18:25:20.996 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:20.998 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:20.998 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:20.998 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/64 2023-11-28T18:25:21.005 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.007 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.008 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.008 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/65 2023-11-28T18:25:21.014 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.017 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.017 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.017 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/66 2023-11-28T18:25:21.024 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.026 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.026 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.026 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/67 2023-11-28T18:25:21.033 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.036 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.036 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.036 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/68 2023-11-28T18:25:21.042 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.046 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.046 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.046 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/69 2023-11-28T18:25:21.052 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.054 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.054 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.054 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/70 2023-11-28T18:25:21.061 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.063 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.063 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.063 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/71 2023-11-28T18:25:21.070 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.072 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.072 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.073 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/72 2023-11-28T18:25:21.079 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.082 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.082 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.082 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/73 2023-11-28T18:25:21.089 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.093 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.093 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.093 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/74 2023-11-28T18:25:21.098 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.101 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/75 2023-11-28T18:25:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.111 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/76 2023-11-28T18:25:21.117 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.120 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.120 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.120 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/77 2023-11-28T18:25:21.126 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.128 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.128 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.128 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/78 2023-11-28T18:25:21.134 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.136 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.137 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.137 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/79 2023-11-28T18:25:21.143 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.145 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.146 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.146 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/80 2023-11-28T18:25:21.152 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.155 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.155 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.155 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/81 2023-11-28T18:25:21.162 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.164 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.164 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.165 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/82 2023-11-28T18:25:21.171 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.173 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.174 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.174 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/83 2023-11-28T18:25:21.180 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.182 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.182 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.183 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/84 2023-11-28T18:25:21.189 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.192 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.192 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.192 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/85 2023-11-28T18:25:21.199 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.201 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.202 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.202 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/86 2023-11-28T18:25:21.208 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.211 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.211 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.211 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/87 2023-11-28T18:25:21.218 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.220 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.220 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.221 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/88 2023-11-28T18:25:21.227 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.230 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.230 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.230 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/89 2023-11-28T18:25:21.236 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.238 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.239 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.239 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/90 2023-11-28T18:25:21.244 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.247 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.247 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.247 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/91 2023-11-28T18:25:21.252 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.255 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.255 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.255 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/92 2023-11-28T18:25:21.264 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.267 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.267 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.271 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/93 2023-11-28T18:25:21.274 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.274 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.274 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.275 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/94 2023-11-28T18:25:21.281 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.282 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.282 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.283 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/95 2023-11-28T18:25:21.288 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.290 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.290 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.290 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/96 2023-11-28T18:25:21.301 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.302 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.302 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.302 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/97 2023-11-28T18:25:21.305 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.307 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.307 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.307 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/98 2023-11-28T18:25:21.314 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.316 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.316 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.316 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/99 2023-11-28T18:25:21.322 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:25:21.324 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:25:21.324 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:25:21.325 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_distributed False 2023-11-28T18:25:21.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.513+0000 7f7e1e796640 1 Processor -- start 2023-11-28T18:25:21.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.513+0000 7f7e1e796640 1 -- start start 2023-11-28T18:25:21.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.513+0000 7f7e1e796640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7e18159f40 0x7f7e180caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:21.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.513+0000 7f7e1e796640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7e180ad850 0x7f7e180701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:21.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.513+0000 7f7e1e796640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e1806bf50 con 0x7f7e180cc760 2023-11-28T18:25:21.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.513+0000 7f7e1e796640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e180ccbc0 con 0x7f7e18159f40 2023-11-28T18:25:21.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.513+0000 7f7e1e796640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1815a320 con 0x7f7e180ad850 2023-11-28T18:25:21.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.513+0000 7f7e17fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7e18159f40 0x7f7e180caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:21.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.513+0000 7f7e17fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7e18159f40 0x7f7e180caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36992/0 (socket says 172.21.15.105:36992) 2023-11-28T18:25:21.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.513+0000 7f7e17fff640 1 -- 172.21.15.105:0/3893418944 learned_addr learned my addr 172.21.15.105:0/3893418944 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:21.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.513+0000 7f7e1cd33640 1 --2- 172.21.15.105:0/3893418944 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7e180ad850 0x7f7e180701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:21.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.513+0000 7f7e17fff640 1 -- 172.21.15.105:0/3893418944 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7e180ad850 msgr2=0x7f7e180701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:21.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.513+0000 7f7e17fff640 1 --2- 172.21.15.105:0/3893418944 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7e180ad850 0x7f7e180701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:21.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.513+0000 7f7e17fff640 1 -- 172.21.15.105:0/3893418944 >> v1:172.21.15.148:6789/0 conn(0x7f7e180cc760 legacy=0x7f7e180ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:21.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.513+0000 7f7e17fff640 1 -- 172.21.15.105:0/3893418944 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e18071380 con 0x7f7e18159f40 2023-11-28T18:25:21.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.517+0000 7f7e17fff640 1 --2- 172.21.15.105:0/3893418944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e18159f40 0x7f7e180caf60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009870 tx=0x7f7e0801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc3e7f8ba589ae2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:21.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.517+0000 7f7e177fe640 1 -- 172.21.15.105:0/3893418944 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e080ab680 con 0x7f7e18159f40 2023-11-28T18:25:21.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.517+0000 7f7e177fe640 1 -- 172.21.15.105:0/3893418944 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e080ab7e0 con 0x7f7e18159f40 2023-11-28T18:25:21.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.517+0000 7f7e177fe640 1 -- 172.21.15.105:0/3893418944 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e080abb70 con 0x7f7e18159f40 2023-11-28T18:25:21.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.517+0000 7f7e1e796640 1 -- 172.21.15.105:0/3893418944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e18159f40 msgr2=0x7f7e180caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:21.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.517+0000 7f7e1e796640 1 --2- 172.21.15.105:0/3893418944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e18159f40 0x7f7e180caf60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009870 tx=0x7f7e0801c480 comp rx=0 tx=0).stop 2023-11-28T18:25:21.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.517+0000 7f7e1e796640 1 -- 172.21.15.105:0/3893418944 shutdown_connections 2023-11-28T18:25:21.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.517+0000 7f7e1e796640 1 --2- 172.21.15.105:0/3893418944 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7e180ad850 0x7f7e180701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:21.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.517+0000 7f7e1e796640 1 --2- 172.21.15.105:0/3893418944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e18159f40 0x7f7e180caf60 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:21.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.517+0000 7f7e1e796640 1 -- 172.21.15.105:0/3893418944 >> 172.21.15.105:0/3893418944 conn(0x7f7e180ceb90 msgr2=0x7f7e180cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:21.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.517+0000 7f7e1e796640 1 -- 172.21.15.105:0/3893418944 shutdown_connections 2023-11-28T18:25:21.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.517+0000 7f7e1e796640 1 -- 172.21.15.105:0/3893418944 wait complete. 2023-11-28T18:25:21.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e1e796640 1 Processor -- start 2023-11-28T18:25:21.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e1e796640 1 -- start start 2023-11-28T18:25:21.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e1e796640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e180ad850 0x7f7e180c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:21.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e1e796640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e180cc760 0x7f7e1814fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:21.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e1e796640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7e18159f40 0x7f7e18150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:21.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e1e796640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e180b3be0 con 0x7f7e180cc760 2023-11-28T18:25:21.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e1e796640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e180b3ec0 con 0x7f7e180ad850 2023-11-28T18:25:21.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e1e796640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e180b41a0 con 0x7f7e18159f40 2023-11-28T18:25:21.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e0ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e180cc760 0x7f7e1814fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:21.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e17fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e180ad850 0x7f7e180c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:21.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e0ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e180cc760 0x7f7e1814fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37008/0 (socket says 172.21.15.105:37008) 2023-11-28T18:25:21.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e0ffff640 1 -- 172.21.15.105:0/1141775992 learned_addr learned my addr 172.21.15.105:0/1141775992 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:21.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e1cd33640 1 --2- 172.21.15.105:0/1141775992 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7e18159f40 0x7f7e18150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:21.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e0ffff640 1 -- 172.21.15.105:0/1141775992 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7e18159f40 msgr2=0x7f7e18150210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:21.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e0ffff640 1 --2- 172.21.15.105:0/1141775992 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7e18159f40 0x7f7e18150210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:21.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e0ffff640 1 -- 172.21.15.105:0/1141775992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e180ad850 msgr2=0x7f7e180c48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:21.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e0ffff640 1 --2- 172.21.15.105:0/1141775992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e180ad850 0x7f7e180c48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:21.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e0ffff640 1 -- 172.21.15.105:0/1141775992 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e080094d0 con 0x7f7e180cc760 2023-11-28T18:25:21.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e0ffff640 1 --2- 172.21.15.105:0/1141775992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e180cc760 0x7f7e1814fcd0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000c3c0 tx=0x7f7e0000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:21.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e157fa640 1 -- 172.21.15.105:0/1141775992 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e000052d0 con 0x7f7e180cc760 2023-11-28T18:25:21.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e157fa640 1 -- 172.21.15.105:0/1141775992 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e00005430 con 0x7f7e180cc760 2023-11-28T18:25:21.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e1e796640 1 -- 172.21.15.105:0/1141775992 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e180b4480 con 0x7f7e180cc760 2023-11-28T18:25:21.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e157fa640 1 -- 172.21.15.105:0/1141775992 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e00003840 con 0x7f7e180cc760 2023-11-28T18:25:21.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.521+0000 7f7e1e796640 1 -- 172.21.15.105:0/1141775992 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e180c4e10 con 0x7f7e180cc760 2023-11-28T18:25:21.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.525+0000 7f7e1e796640 1 -- 172.21.15.105:0/1141775992 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e18067570 con 0x7f7e180cc760 2023-11-28T18:25:21.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.525+0000 7f7e157fa640 1 -- 172.21.15.105:0/1141775992 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f7e000039a0 con 0x7f7e180cc760 2023-11-28T18:25:21.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.525+0000 7f7e157fa640 1 --2- 172.21.15.105:0/1141775992 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7df80668d0 0x7f7df8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:21.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.525+0000 7f7e157fa640 1 -- 172.21.15.105:0/1141775992 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7e00083ab0 con 0x7f7e180cc760 2023-11-28T18:25:21.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.525+0000 7f7e17fff640 1 --2- 172.21.15.105:0/1141775992 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7df80668d0 0x7f7df8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:21.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.525+0000 7f7e17fff640 1 --2- 172.21.15.105:0/1141775992 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7df80668d0 0x7f7df8068d90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7e080af000 tx=0x7f7e080b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:21.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.529+0000 7f7e157fa640 1 -- 172.21.15.105:0/1141775992 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7e00050ba0 con 0x7f7e180cc760 2023-11-28T18:25:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.681+0000 7f7e1e796640 1 -- 172.21.15.105:0/1141775992 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_distributed}] v 0) v1 -- 0x7f7e180b4930 con 0x7f7e180cc760 2023-11-28T18:25:21.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.685+0000 7f7e157fa640 1 -- 172.21.15.105:0/1141775992 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_distributed}]=0 v14)=0 v14) v1 ==== 134+0+0 (secure 0 0 0) 0x7f7e000170c0 con 0x7f7e180cc760 2023-11-28T18:25:21.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.689+0000 7f7e1e796640 1 -- 172.21.15.105:0/1141775992 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7df80668d0 msgr2=0x7f7df8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:21.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.689+0000 7f7e1e796640 1 --2- 172.21.15.105:0/1141775992 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7df80668d0 0x7f7df8068d90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7e080af000 tx=0x7f7e080b6040 comp rx=0 tx=0).stop 2023-11-28T18:25:21.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.689+0000 7f7e1e796640 1 -- 172.21.15.105:0/1141775992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e180cc760 msgr2=0x7f7e1814fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:21.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.689+0000 7f7e1e796640 1 --2- 172.21.15.105:0/1141775992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e180cc760 0x7f7e1814fcd0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000c3c0 tx=0x7f7e0000c880 comp rx=0 tx=0).stop 2023-11-28T18:25:21.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.689+0000 7f7e1e796640 1 -- 172.21.15.105:0/1141775992 shutdown_connections 2023-11-28T18:25:21.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.689+0000 7f7e1e796640 1 --2- 172.21.15.105:0/1141775992 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7e18159f40 0x7f7e18150210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:21.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.689+0000 7f7e1e796640 1 --2- 172.21.15.105:0/1141775992 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7df80668d0 0x7f7df8068d90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:21.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.689+0000 7f7e1e796640 1 --2- 172.21.15.105:0/1141775992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e180cc760 0x7f7e1814fcd0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:21.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.689+0000 7f7e1e796640 1 --2- 172.21.15.105:0/1141775992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e180ad850 0x7f7e180c48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:21.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.689+0000 7f7e1e796640 1 -- 172.21.15.105:0/1141775992 >> 172.21.15.105:0/1141775992 conn(0x7f7e180ceb90 msgr2=0x7f7e180ae830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:21.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.689+0000 7f7e1e796640 1 -- 172.21.15.105:0/1141775992 shutdown_connections 2023-11-28T18:25:21.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:21.689+0000 7f7e1e796640 1 -- 172.21.15.105:0/1141775992 wait complete. 2023-11-28T18:25:21.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:25:21.708 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin.distributed -v 1 tree) 2023-11-28T18:25:31.417 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:25:31.421 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:25:36.724 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-11-28T18:25:36.895 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"3529","version":"3529","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1110","version":"1110","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/0","dirfrag":"0x1000000041e","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/1","dirfrag":"0x10000000420","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/2","dirfrag":"0x10000000422","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/3","dirfrag":"0x10000000424","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/4","dirfrag":"0x10000000426","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/5","dirfrag":"0x10000000428","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/6","dirfrag":"0x1000000042a","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/7","dirfrag":"0x1000000042c","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/8","dirfrag":"0x1000000042e","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/9","dirfrag":"0x10000000430","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/10","dirfrag":"0x10000000432","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/25","dirfrag":"0x10000000450","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/26","dirfrag":"0x10000000452","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/27","dirfrag":"0x10000000454","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/28","dirfrag":"0x10000000456","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/29","dirfrag":"0x10000000458","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/30","dirfrag":"0x1000000045a","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/31","dirfrag":"0x1000000045c","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/32","dirfrag":"0x1000000045e","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/33","dirfrag":"0x10000000460","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/34","dirfrag":"0x10000000462","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/35","dirfrag":"0x10000000464","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/36","dirfrag":"0x10000000466","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/37","dirfrag":"0x10000000468","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/38","dirfrag":"0x1000000046a","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/53","dirfrag":"0x10000000488","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/54","dirfrag":"0x1000000048a","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/55","dirfrag":"0x1000000048c","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/56","dirfrag":"0x1000000048e","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/57","dirfrag":"0x10000000490","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/58","dirfrag":"0x10000000492","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/59","dirfrag":"0x10000000494","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/60","dirfrag":"0x10000000496","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/61","dirfrag":"0x10000000498","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/62","dirfrag":"0x1000000049a","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/63","dirfrag":"0x1000000049c","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/64","dirfrag":"0x1000000049e","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/65","dirfrag":"0x100000004a0","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/66","dirfrag":"0x100000004a2","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/39","dirfrag":"0x1000000046c","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/40","dirfrag":"0x1000000046e","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/41","dirfrag":"0x10000000470","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/42","dirfrag":"0x10000000472","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/43","dirfrag":"0x10000000474","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/44","dirfrag":"0x10000000476","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/45","dirfrag":"0x10000000478","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/46","dirfrag":"0x1000000047a","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/47","dirfrag":"0x1000000047c","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/48","dirfrag":"0x1000000047e","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/49","dirfrag":"0x10000000480","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/50","dirfrag":"0x10000000482","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/51","dirfrag":"0x10000000484","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/52","dirfrag":"0x10000000486","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/11","dirfrag":"0x10000000434","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/12","dirfrag":"0x10000000436","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/13","dirfrag":"0x10000000438","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/14","dirfrag":"0x1000000043a","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/15","dirfrag":"0x1000000043c","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/16","dirfrag":"0x1000000043e","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/17","dirfrag":"0x10000000440","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/18","dirfrag":"0x10000000442","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/19","dirfrag":"0x10000000444","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/20","dirfrag":"0x10000000446","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/21","dirfrag":"0x10000000448","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/22","dirfrag":"0x1000000044a","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/23","dirfrag":"0x1000000044c","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/24","dirfrag":"0x1000000044e","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/67","dirfrag":"0x100000004a4","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/68","dirfrag":"0x100000004a6","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/69","dirfrag":"0x100000004a8","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/70","dirfrag":"0x100000004aa","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/71","dirfrag":"0x100000004ac","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/72","dirfrag":"0x100000004ae","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/73","dirfrag":"0x100000004b0","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/74","dirfrag":"0x100000004b2","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/75","dirfrag":"0x100000004b4","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/76","dirfrag":"0x100000004b6","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/77","dirfrag":"0x100000004b8","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/78","dirfrag":"0x100000004ba","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/79","dirfrag":"0x100000004bc","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/80","dirfrag":"0x100000004be","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/81","dirfrag":"0x100000004c0","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/82","dirfrag":"0x100000004c2","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/83","dirfrag":"0x100000004c4","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/84","dirfrag":"0x100000004c6","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/85","dirfrag":"0x100000004c8","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/86","dirfrag":"0x100000004ca","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/87","dirfrag":"0x100000004cc","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/88","dirfrag":"0x100000004ce","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/89","dirfrag":"0x100000004d0","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/90","dirfrag":"0x100000004d2","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/91","dirfrag":"0x100000004d4","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/92","dirfrag":"0x100000004d6","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/93","dirfrag":"0x100000004d8","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/94","dirfrag":"0x100000004da","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/95","dirfrag":"0x100000004dc","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/96","dirfrag":"0x100000004de","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/97","dirfrag":"0x100000004e0","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/98","dirfrag":"0x100000004e2","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree/99","dirfrag":"0x100000004e4","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2023-11-28T18:25:36.905 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1, "2": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "3529", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "3529" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "1110", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1110" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000041e", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/0", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000420", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/1", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000422", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/2", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000424", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/3", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000426", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/4", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000428", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/5", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000042a", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000042c", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/7", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000042e", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/8", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000430", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/9", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000432", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/10", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000450", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/25", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000452", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/26", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000454", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/27", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000456", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/28", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000458", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/29", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000045a", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/30", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000045c", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/31", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000045e", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/32", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000460", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/33", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000462", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/34", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000464", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/35", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000466", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/36", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000468", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/37", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000046a", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/38", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000488", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/53", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000048a", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/54", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000048c", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/55", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000048e", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/56", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000490", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/57", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000492", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/58", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000494", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/59", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000496", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/60", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000498", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/61", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000049a", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/62", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000049c", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/63", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000049e", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/64", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004a0", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/65", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004a2", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/66", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000046c", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/39", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000046e", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/40", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000470", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/41", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000472", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/42", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000474", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/43", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000476", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/44", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000478", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/45", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000047a", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/46", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000047c", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/47", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000047e", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/48", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000480", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/49", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000482", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/50", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000484", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/51", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000486", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/52", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000434", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/11", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000436", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/12", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000438", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/13", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000043a", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/14", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000043c", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/15", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000043e", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/16", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000440", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/17", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000442", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/18", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000444", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/19", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000446", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/20", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000448", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/21", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000044a", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/22", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000044c", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/23", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000044e", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/24", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004a4", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/67", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004a6", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/68", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004a8", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/69", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004aa", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/70", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004ac", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/71", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004ae", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/72", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004b0", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/73", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004b2", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/74", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004b4", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/75", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004b6", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/76", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004b8", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/77", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004ba", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/78", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004bc", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/79", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004be", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/80", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004c0", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/81", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004c2", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/82", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004c4", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/83", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004c6", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/84", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004c8", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/85", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004ca", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/86", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004cc", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/87", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004ce", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/88", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004d0", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/89", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004d2", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/90", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004d4", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/91", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004d6", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/92", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004d8", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/93", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004da", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/94", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004dc", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/95", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004de", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/96", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004e0", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/97", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004e2", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/98", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x100000004e4", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/99", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2023-11-28T18:25:36.905 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T18:25:36.906 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:25:36.972 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:25:37.057 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T18:25:37.057 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:25:37.057 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:25:37.068 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:25:37.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.241+0000 7fb3e9596640 1 Processor -- start 2023-11-28T18:25:37.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.241+0000 7fb3e9596640 1 -- start start 2023-11-28T18:25:37.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e9596640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb3e4159f40 0x7fb3e40caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e9596640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e406bf50 con 0x7fb3e40ad850 2023-11-28T18:25:37.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e9596640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e40ccbc0 con 0x7fb3e40cc760 2023-11-28T18:25:37.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e9596640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e415a320 con 0x7fb3e4159f40 2023-11-28T18:25:37.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e37fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb3e40ad850 0x7fb3e40701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50514/0 (socket says 172.21.15.105:50514) 2023-11-28T18:25:37.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e37fe640 1 -- 172.21.15.105:0/320429255 learned_addr learned my addr 172.21.15.105:0/320429255 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:37.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e2ffd640 1 --2- 172.21.15.105:0/320429255 >> v2:172.21.15.148:3300/0 conn(0x7fb3e4159f40 0x7fb3e40caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:37.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e27fc640 1 -- 172.21.15.105:0/320429255 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3173834762 0 0) 0x7fb3e406bf50 con 0x7fb3e40ad850 2023-11-28T18:25:37.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e27fc640 1 -- 172.21.15.105:0/320429255 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3d0003530 con 0x7fb3e40ad850 2023-11-28T18:25:37.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e27fc640 1 -- 172.21.15.105:0/320429255 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 478628614 0 0) 0x7fb3d0003530 con 0x7fb3e40ad850 2023-11-28T18:25:37.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e27fc640 1 -- 172.21.15.105:0/320429255 >> v2:172.21.15.148:3300/0 conn(0x7fb3e4159f40 msgr2=0x7fb3e40caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:37.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e27fc640 1 --2- 172.21.15.105:0/320429255 >> v2:172.21.15.148:3300/0 conn(0x7fb3e4159f40 0x7fb3e40caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e27fc640 1 -- 172.21.15.105:0/320429255 >> v1:172.21.15.148:6789/0 conn(0x7fb3e40cc760 legacy=0x7fb3e40ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:37.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e27fc640 1 -- 172.21.15.105:0/320429255 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e4071380 con 0x7fb3e40ad850 2023-11-28T18:25:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e27fc640 1 -- 172.21.15.105:0/320429255 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fb3dc001a00 con 0x7fb3e40ad850 2023-11-28T18:25:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e27fc640 1 -- 172.21.15.105:0/320429255 >> v1:172.21.15.105:6789/0 conn(0x7fb3e40ad850 legacy=0x7fb3e40701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e27fc640 1 --2- 172.21.15.105:0/320429255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3d0003980 0x7fb3d0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e27fc640 1 --2- 172.21.15.105:0/320429255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3d00047d0 0x7fb3d0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e27fc640 1 --2- 172.21.15.105:0/320429255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3d0005190 0x7fb3d000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e27fc640 1 -- 172.21.15.105:0/320429255 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e415a320 con 0x7fb3d0003980 2023-11-28T18:25:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e27fc640 1 -- 172.21.15.105:0/320429255 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d000c130 con 0x7fb3d00047d0 2023-11-28T18:25:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e27fc640 1 -- 172.21.15.105:0/320429255 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d000c2f0 con 0x7fb3d0005190 2023-11-28T18:25:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e2ffd640 1 --2- 172.21.15.105:0/320429255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3d0003980 0x7fb3d0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3da7fc640 1 --2- 172.21.15.105:0/320429255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3d00047d0 0x7fb3d0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e37fe640 1 --2- 172.21.15.105:0/320429255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3d0005190 0x7fb3d000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e2ffd640 1 -- 172.21.15.105:0/320429255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3d0005190 msgr2=0x7fb3d000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e2ffd640 1 --2- 172.21.15.105:0/320429255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3d0005190 0x7fb3d000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e2ffd640 1 -- 172.21.15.105:0/320429255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3d00047d0 msgr2=0x7fb3d0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e2ffd640 1 --2- 172.21.15.105:0/320429255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3d00047d0 0x7fb3d0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e2ffd640 1 -- 172.21.15.105:0/320429255 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d0003530 con 0x7fb3d0003980 2023-11-28T18:25:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3da7fc640 1 --2- 172.21.15.105:0/320429255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3d00047d0 0x7fb3d0004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.245+0000 7fb3e37fe640 1 -- 172.21.15.105:0/320429255 reap_dead start 2023-11-28T18:25:37.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.249+0000 7fb3e2ffd640 1 --2- 172.21.15.105:0/320429255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3d0003980 0x7fb3d0003da0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009ae0 tx=0x7fb3d401ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4147c7f69a4b5cc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.249+0000 7fb3e27fc640 1 -- 172.21.15.105:0/320429255 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d40a6430 con 0x7fb3d0003980 2023-11-28T18:25:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.249+0000 7fb3e27fc640 1 -- 172.21.15.105:0/320429255 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d40a6590 con 0x7fb3d0003980 2023-11-28T18:25:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.249+0000 7fb3e27fc640 1 -- 172.21.15.105:0/320429255 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d40a6940 con 0x7fb3d0003980 2023-11-28T18:25:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.249+0000 7fb3e9596640 1 -- 172.21.15.105:0/320429255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3d0003980 msgr2=0x7fb3d0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.249+0000 7fb3e9596640 1 --2- 172.21.15.105:0/320429255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3d0003980 0x7fb3d0003da0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009ae0 tx=0x7fb3d401ca40 comp rx=0 tx=0).stop 2023-11-28T18:25:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.249+0000 7fb3e9596640 1 -- 172.21.15.105:0/320429255 shutdown_connections 2023-11-28T18:25:37.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.249+0000 7fb3e9596640 1 --2- 172.21.15.105:0/320429255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3d0003980 0x7fb3d0003da0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.249+0000 7fb3e9596640 1 -- 172.21.15.105:0/320429255 >> 172.21.15.105:0/320429255 conn(0x7fb3e40ceb90 msgr2=0x7fb3e40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:37.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.249+0000 7fb3e9596640 1 -- 172.21.15.105:0/320429255 shutdown_connections 2023-11-28T18:25:37.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.249+0000 7fb3e9596640 1 -- 172.21.15.105:0/320429255 wait complete. 2023-11-28T18:25:37.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3e9596640 1 Processor -- start 2023-11-28T18:25:37.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3e9596640 1 -- start start 2023-11-28T18:25:37.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3e9596640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3d0003980 0x7fb3e40c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3e9596640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3e41584e0 0x7fb3e4156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3e9596640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3e4158940 0x7fb3e4156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3e9596640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e414add0 con 0x7fb3e41584e0 2023-11-28T18:25:37.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3e9596640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e4151db0 con 0x7fb3e4158940 2023-11-28T18:25:37.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3e9596640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e4152090 con 0x7fb3d0003980 2023-11-28T18:25:37.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3da7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3e41584e0 0x7fb3e4156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:37.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3da7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3e41584e0 0x7fb3e4156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48502/0 (socket says 172.21.15.105:48502) 2023-11-28T18:25:37.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3da7fc640 1 -- 172.21.15.105:0/1475241502 learned_addr learned my addr 172.21.15.105:0/1475241502 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:37.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3da7fc640 1 -- 172.21.15.105:0/1475241502 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3d0003980 msgr2=0x7fb3e40c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:37.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3da7fc640 1 --2- 172.21.15.105:0/1475241502 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3d0003980 0x7fb3e40c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3da7fc640 1 -- 172.21.15.105:0/1475241502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3e4158940 msgr2=0x7fb3e4156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:37.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3da7fc640 1 --2- 172.21.15.105:0/1475241502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3e4158940 0x7fb3e4156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3da7fc640 1 -- 172.21.15.105:0/1475241502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e4152370 con 0x7fb3e41584e0 2023-11-28T18:25:37.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3da7fc640 1 --2- 172.21.15.105:0/1475241502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3e41584e0 0x7fb3e4156150 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc00e480 tx=0x7fb3cc00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:37.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3dbfff640 1 -- 172.21.15.105:0/1475241502 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc017070 con 0x7fb3e41584e0 2023-11-28T18:25:37.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3dbfff640 1 -- 172.21.15.105:0/1475241502 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3cc0052e0 con 0x7fb3e41584e0 2023-11-28T18:25:37.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3dbfff640 1 -- 172.21.15.105:0/1475241502 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc005490 con 0x7fb3e41584e0 2023-11-28T18:25:37.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3e9596640 1 -- 172.21.15.105:0/1475241502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e4152650 con 0x7fb3e41584e0 2023-11-28T18:25:37.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3e9596640 1 -- 172.21.15.105:0/1475241502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e41f6220 con 0x7fb3e41584e0 2023-11-28T18:25:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.253+0000 7fb3e9596640 1 -- 172.21.15.105:0/1475241502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e4067570 con 0x7fb3e41584e0 2023-11-28T18:25:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.257+0000 7fb3dbfff640 1 -- 172.21.15.105:0/1475241502 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fb3cc007020 con 0x7fb3e41584e0 2023-11-28T18:25:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.257+0000 7fb3dbfff640 1 --2- 172.21.15.105:0/1475241502 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb3c4066900 0x7fb3c4068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.257+0000 7fb3dbfff640 1 -- 172.21.15.105:0/1475241502 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb3cc083d10 con 0x7fb3e41584e0 2023-11-28T18:25:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.257+0000 7fb3e2ffd640 1 --2- 172.21.15.105:0/1475241502 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb3c4066900 0x7fb3c4068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.257+0000 7fb3e2ffd640 1 --2- 172.21.15.105:0/1475241502 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb3c4066900 0x7fb3c4068dc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40ad000 tx=0x7fb3d40a9040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:37.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.261+0000 7fb3dbfff640 1 -- 172.21.15.105:0/1475241502 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb3cc089020 con 0x7fb3e41584e0 2023-11-28T18:25:37.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.413+0000 7fb3e9596640 1 -- 172.21.15.105:0/1475241502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3e406e3e0 con 0x7fb3e41584e0 2023-11-28T18:25:37.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.413+0000 7fb3dbfff640 1 -- 172.21.15.105:0/1475241502 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16827 (secure 0 0 0) 0x7fb3cc050d50 con 0x7fb3e41584e0 2023-11-28T18:25:37.416 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:37.416 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":40,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:25:15.107712+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:24:53.582892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T18:24:54.921978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.39","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:25:37.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.417+0000 7fb3e9596640 1 -- 172.21.15.105:0/1475241502 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb3c4066900 msgr2=0x7fb3c4068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:37.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.417+0000 7fb3e9596640 1 --2- 172.21.15.105:0/1475241502 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb3c4066900 0x7fb3c4068dc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40ad000 tx=0x7fb3d40a9040 comp rx=0 tx=0).stop 2023-11-28T18:25:37.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.417+0000 7fb3e9596640 1 -- 172.21.15.105:0/1475241502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3e41584e0 msgr2=0x7fb3e4156150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:37.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.417+0000 7fb3e9596640 1 --2- 172.21.15.105:0/1475241502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3e41584e0 0x7fb3e4156150 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc00e480 tx=0x7fb3cc00e940 comp rx=0 tx=0).stop 2023-11-28T18:25:37.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.417+0000 7fb3e9596640 1 -- 172.21.15.105:0/1475241502 shutdown_connections 2023-11-28T18:25:37.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.417+0000 7fb3e9596640 1 --2- 172.21.15.105:0/1475241502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3e4158940 0x7fb3e4156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.417+0000 7fb3e9596640 1 --2- 172.21.15.105:0/1475241502 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb3c4066900 0x7fb3c4068dc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.417+0000 7fb3e9596640 1 --2- 172.21.15.105:0/1475241502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3e41584e0 0x7fb3e4156150 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.417+0000 7fb3e9596640 1 --2- 172.21.15.105:0/1475241502 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3d0003980 0x7fb3e40c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.417+0000 7fb3e9596640 1 -- 172.21.15.105:0/1475241502 >> 172.21.15.105:0/1475241502 conn(0x7fb3e40ceb90 msgr2=0x7fb3e40cdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:37.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.417+0000 7fb3e9596640 1 -- 172.21.15.105:0/1475241502 shutdown_connections 2023-11-28T18:25:37.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.417+0000 7fb3e9596640 1 -- 172.21.15.105:0/1475241502 wait complete. 2023-11-28T18:25:37.437 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:24:53.582892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:24:54.921978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:25:37.438 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:25:37.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.577+0000 7fb004357640 1 Processor -- start 2023-11-28T18:25:37.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.577+0000 7fb004357640 1 -- start start 2023-11-28T18:25:37.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.577+0000 7fb004357640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faffc0caf60 0x7faffc0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.577+0000 7fb004357640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7faffc0ad850 0x7faffc0701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.577+0000 7fb004357640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faffc06bf50 con 0x7faffc159f40 2023-11-28T18:25:37.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.577+0000 7fb004357640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faffc0cb3c0 con 0x7faffc0ad850 2023-11-28T18:25:37.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.577+0000 7fb004357640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faffc15a320 con 0x7faffc0caf60 2023-11-28T18:25:37.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.577+0000 7fb0018f2640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faffc0caf60 0x7faffc0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:37.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.577+0000 7fb0028f4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7faffc0ad850 0x7faffc0701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:37.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.577+0000 7fb0018f2640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faffc0caf60 0x7faffc0cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:57998/0 (socket says 172.21.15.105:57998) 2023-11-28T18:25:37.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.577+0000 7fb0018f2640 1 -- 172.21.15.105:0/4223868484 learned_addr learned my addr 172.21.15.105:0/4223868484 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:37.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.577+0000 7fb0018f2640 1 -- 172.21.15.105:0/4223868484 >> v2:172.21.15.148:3300/0 conn(0x7faffc0ad850 msgr2=0x7faffc0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.577+0000 7fb0018f2640 1 --2- 172.21.15.105:0/4223868484 >> v2:172.21.15.148:3300/0 conn(0x7faffc0ad850 0x7faffc0701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.577+0000 7fb0018f2640 1 -- 172.21.15.105:0/4223868484 >> v1:172.21.15.148:6789/0 conn(0x7faffc159f40 legacy=0x7faffc0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.577+0000 7fb0018f2640 1 -- 172.21.15.105:0/4223868484 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc071380 con 0x7faffc0caf60 2023-11-28T18:25:37.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.581+0000 7fb0018f2640 1 --2- 172.21.15.105:0/4223868484 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faffc0caf60 0x7faffc0cc760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fafec009870 tx=0x7fafec01c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=360194366b4d75f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:37.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.581+0000 7fb0010f1640 1 -- 172.21.15.105:0/4223868484 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec0ab680 con 0x7faffc0caf60 2023-11-28T18:25:37.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.581+0000 7fb0010f1640 1 -- 172.21.15.105:0/4223868484 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafec0ab7e0 con 0x7faffc0caf60 2023-11-28T18:25:37.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.581+0000 7fb0010f1640 1 -- 172.21.15.105:0/4223868484 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec0abb70 con 0x7faffc0caf60 2023-11-28T18:25:37.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.581+0000 7fb004357640 1 -- 172.21.15.105:0/4223868484 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faffc0caf60 msgr2=0x7faffc0cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:37.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.581+0000 7fb004357640 1 --2- 172.21.15.105:0/4223868484 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faffc0caf60 0x7faffc0cc760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fafec009870 tx=0x7fafec01c480 comp rx=0 tx=0).stop 2023-11-28T18:25:37.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.581+0000 7fb004357640 1 -- 172.21.15.105:0/4223868484 shutdown_connections 2023-11-28T18:25:37.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.581+0000 7fb004357640 1 --2- 172.21.15.105:0/4223868484 >> v2:172.21.15.148:3300/0 conn(0x7faffc0ad850 0x7faffc0701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.581+0000 7fb004357640 1 --2- 172.21.15.105:0/4223868484 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faffc0caf60 0x7faffc0cc760 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.581+0000 7fb004357640 1 -- 172.21.15.105:0/4223868484 >> 172.21.15.105:0/4223868484 conn(0x7faffc0ceb90 msgr2=0x7faffc0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:37.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.581+0000 7fb004357640 1 -- 172.21.15.105:0/4223868484 shutdown_connections 2023-11-28T18:25:37.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.581+0000 7fb004357640 1 -- 172.21.15.105:0/4223868484 wait complete. 2023-11-28T18:25:37.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7fb004357640 1 Processor -- start 2023-11-28T18:25:37.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7fb004357640 1 -- start start 2023-11-28T18:25:37.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7fb004357640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faffc0ad850 0x7faffc146610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7fb004357640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faffc0caf60 0x7faffc0c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7fb004357640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faffc159f40 0x7faffc158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7fb004357640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc0c8840 con 0x7faffc159f40 2023-11-28T18:25:37.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7fb004357640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc0c8b20 con 0x7faffc0caf60 2023-11-28T18:25:37.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7fb004357640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faffc0c8e00 con 0x7faffc0ad850 2023-11-28T18:25:37.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7fb0028f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faffc159f40 0x7faffc158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:37.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7fb0020f3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faffc0ad850 0x7faffc146610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:37.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7fb0020f3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faffc0ad850 0x7faffc146610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:58006/0 (socket says 172.21.15.105:58006) 2023-11-28T18:25:37.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7fb0020f3640 1 -- 172.21.15.105:0/1588805609 learned_addr learned my addr 172.21.15.105:0/1588805609 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:37.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7fb0018f2640 1 --2- 172.21.15.105:0/1588805609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faffc0caf60 0x7faffc0c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:37.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7fb0020f3640 1 -- 172.21.15.105:0/1588805609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faffc0caf60 msgr2=0x7faffc0c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:37.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7fb0020f3640 1 --2- 172.21.15.105:0/1588805609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faffc0caf60 0x7faffc0c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7fb0020f3640 1 -- 172.21.15.105:0/1588805609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faffc159f40 msgr2=0x7faffc158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:37.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7fb0020f3640 1 --2- 172.21.15.105:0/1588805609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faffc159f40 0x7faffc158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7fb0020f3640 1 -- 172.21.15.105:0/1588805609 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec0094d0 con 0x7faffc0ad850 2023-11-28T18:25:37.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7fb0020f3640 1 --2- 172.21.15.105:0/1588805609 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faffc0ad850 0x7faffc146610 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faff800c3c0 tx=0x7faff800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:37.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7faff2ffd640 1 -- 172.21.15.105:0/1588805609 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff80052d0 con 0x7faffc0ad850 2023-11-28T18:25:37.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7faff2ffd640 1 -- 172.21.15.105:0/1588805609 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff8005430 con 0x7faffc0ad850 2023-11-28T18:25:37.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7faff2ffd640 1 -- 172.21.15.105:0/1588805609 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8003840 con 0x7faffc0ad850 2023-11-28T18:25:37.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7fb004357640 1 -- 172.21.15.105:0/1588805609 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faffc0c90e0 con 0x7faffc0ad850 2023-11-28T18:25:37.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.585+0000 7fb004357640 1 -- 172.21.15.105:0/1588805609 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faffc146b50 con 0x7faffc0ad850 2023-11-28T18:25:37.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.589+0000 7fb004357640 1 -- 172.21.15.105:0/1588805609 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faffc0c9600 con 0x7faffc0ad850 2023-11-28T18:25:37.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.589+0000 7faff2ffd640 1 -- 172.21.15.105:0/1588805609 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7faff80039a0 con 0x7faffc0ad850 2023-11-28T18:25:37.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.589+0000 7faff2ffd640 1 --2- 172.21.15.105:0/1588805609 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fafe00668d0 0x7fafe0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.589+0000 7faff2ffd640 1 -- 172.21.15.105:0/1588805609 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faff8083ae0 con 0x7faffc0ad850 2023-11-28T18:25:37.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.589+0000 7fb0018f2640 1 --2- 172.21.15.105:0/1588805609 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fafe00668d0 0x7fafe0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:37.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.589+0000 7fb0018f2640 1 --2- 172.21.15.105:0/1588805609 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fafe00668d0 0x7fafe0068d90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fafec0af000 tx=0x7fafec0b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:37.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.593+0000 7faff2ffd640 1 -- 172.21.15.105:0/1588805609 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faff8050bd0 con 0x7faffc0ad850 2023-11-28T18:25:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.745+0000 7fb004357640 1 -- 172.21.15.105:0/1588805609 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faffc0c8f40 con 0x7faffc0ad850 2023-11-28T18:25:37.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.745+0000 7faff2ffd640 1 -- 172.21.15.105:0/1588805609 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7faffc0c8f40 con 0x7faffc0ad850 2023-11-28T18:25:37.746 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:25:37.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.745+0000 7fb004357640 1 -- 172.21.15.105:0/1588805609 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fafe00668d0 msgr2=0x7fafe0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:37.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.745+0000 7fb004357640 1 --2- 172.21.15.105:0/1588805609 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fafe00668d0 0x7fafe0068d90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fafec0af000 tx=0x7fafec0b6040 comp rx=0 tx=0).stop 2023-11-28T18:25:37.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.745+0000 7fb004357640 1 -- 172.21.15.105:0/1588805609 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faffc0ad850 msgr2=0x7faffc146610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:37.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.745+0000 7fb004357640 1 --2- 172.21.15.105:0/1588805609 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faffc0ad850 0x7faffc146610 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faff800c3c0 tx=0x7faff800c880 comp rx=0 tx=0).stop 2023-11-28T18:25:37.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.745+0000 7fb004357640 1 -- 172.21.15.105:0/1588805609 shutdown_connections 2023-11-28T18:25:37.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.745+0000 7fb004357640 1 --2- 172.21.15.105:0/1588805609 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fafe00668d0 0x7fafe0068d90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.745+0000 7fb004357640 1 --2- 172.21.15.105:0/1588805609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faffc159f40 0x7faffc158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.745+0000 7fb004357640 1 --2- 172.21.15.105:0/1588805609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faffc0caf60 0x7faffc0c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.745+0000 7fb004357640 1 --2- 172.21.15.105:0/1588805609 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faffc0ad850 0x7faffc146610 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.745+0000 7fb004357640 1 -- 172.21.15.105:0/1588805609 >> 172.21.15.105:0/1588805609 conn(0x7faffc0ceb90 msgr2=0x7faffc165120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:37.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.745+0000 7fb004357640 1 -- 172.21.15.105:0/1588805609 shutdown_connections 2023-11-28T18:25:37.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.745+0000 7fb004357640 1 -- 172.21.15.105:0/1588805609 wait complete. 2023-11-28T18:25:37.767 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:25:37.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1689ce3640 1 Processor -- start 2023-11-28T18:25:37.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1689ce3640 1 -- start start 2023-11-28T18:25:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1689ce3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1684159f40 0x7f16840caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1689ce3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f16840cc760 0x7f16840ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1689ce3640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f168406bf50 con 0x7f16840ad680 2023-11-28T18:25:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1689ce3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16840ccbc0 con 0x7f16840cc760 2023-11-28T18:25:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1689ce3640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168415a320 con 0x7f1684159f40 2023-11-28T18:25:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1683fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f16840ad680 0x7f168415da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50518/0 (socket says 172.21.15.105:50518) 2023-11-28T18:25:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1683fff640 1 -- 172.21.15.105:0/2813330052 learned_addr learned my addr 172.21.15.105:0/2813330052 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:37.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f167bfff640 1 --2- 172.21.15.105:0/2813330052 >> v2:172.21.15.148:3300/0 conn(0x7f16840cc760 0x7f16840ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:37.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f16837fe640 1 --2- 172.21.15.105:0/2813330052 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1684159f40 0x7f16840caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:37.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1682ffd640 1 -- 172.21.15.105:0/2813330052 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 495845335 0 0) 0x7f168406bf50 con 0x7f16840ad680 2023-11-28T18:25:37.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1682ffd640 1 -- 172.21.15.105:0/2813330052 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1670003530 con 0x7f16840ad680 2023-11-28T18:25:37.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1682ffd640 1 -- 172.21.15.105:0/2813330052 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2916566637 0 0) 0x7f1670003530 con 0x7f16840ad680 2023-11-28T18:25:37.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1682ffd640 1 -- 172.21.15.105:0/2813330052 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1684159f40 msgr2=0x7f16840caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:37.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1682ffd640 1 --2- 172.21.15.105:0/2813330052 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1684159f40 0x7f16840caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1682ffd640 1 -- 172.21.15.105:0/2813330052 >> v2:172.21.15.148:3300/0 conn(0x7f16840cc760 msgr2=0x7f16840ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:37.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1682ffd640 1 --2- 172.21.15.105:0/2813330052 >> v2:172.21.15.148:3300/0 conn(0x7f16840cc760 0x7f16840ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1682ffd640 1 -- 172.21.15.105:0/2813330052 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1684071380 con 0x7f16840ad680 2023-11-28T18:25:37.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1682ffd640 1 -- 172.21.15.105:0/2813330052 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f167c001a00 con 0x7f16840ad680 2023-11-28T18:25:37.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1682ffd640 1 -- 172.21.15.105:0/2813330052 >> v1:172.21.15.105:6789/0 conn(0x7f16840ad680 legacy=0x7f168415da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:37.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1682ffd640 1 --2- 172.21.15.105:0/2813330052 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1670003980 0x7f1670003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1682ffd640 1 --2- 172.21.15.105:0/2813330052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1670004370 0x7f1670004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1682ffd640 1 --2- 172.21.15.105:0/2813330052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1670004f60 0x7f167000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1682ffd640 1 -- 172.21.15.105:0/2813330052 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16840ccbc0 con 0x7f1670004f60 2023-11-28T18:25:37.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1682ffd640 1 -- 172.21.15.105:0/2813330052 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168415a320 con 0x7f1670004370 2023-11-28T18:25:37.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1682ffd640 1 -- 172.21.15.105:0/2813330052 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167000bf00 con 0x7f1670003980 2023-11-28T18:25:37.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f16837fe640 1 --2- 172.21.15.105:0/2813330052 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1684159f40 0x7f16840caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:25:37.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f167bfff640 1 --2- 172.21.15.105:0/2813330052 >> v2:172.21.15.148:3300/0 conn(0x7f16840cc760 0x7f16840ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:25:37.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f1683fff640 1 --2- 172.21.15.105:0/2813330052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1670004f60 0x7f167000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:37.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.905+0000 7f167bfff640 1 --2- 172.21.15.105:0/2813330052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1670004370 0x7f1670004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:37.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.909+0000 7f16837fe640 1 --2- 172.21.15.105:0/2813330052 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1670003980 0x7f1670003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:37.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.909+0000 7f1683fff640 1 -- 172.21.15.105:0/2813330052 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1670003980 msgr2=0x7f1670003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:37.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.909+0000 7f1683fff640 1 --2- 172.21.15.105:0/2813330052 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1670003980 0x7f1670003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.909+0000 7f1683fff640 1 -- 172.21.15.105:0/2813330052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1670004370 msgr2=0x7f1670004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:37.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.909+0000 7f1683fff640 1 --2- 172.21.15.105:0/2813330052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1670004370 0x7f1670004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.909+0000 7f1683fff640 1 -- 172.21.15.105:0/2813330052 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1670003530 con 0x7f1670004f60 2023-11-28T18:25:37.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.909+0000 7f1683fff640 1 --2- 172.21.15.105:0/2813330052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1670004f60 0x7f167000b990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f167c00a860 tx=0x7f167c01fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b9756506b205cf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:37.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.909+0000 7f1682ffd640 1 -- 172.21.15.105:0/2813330052 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167c0aa6f0 con 0x7f1670004f60 2023-11-28T18:25:37.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.909+0000 7f1682ffd640 1 -- 172.21.15.105:0/2813330052 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f167c0aa850 con 0x7f1670004f60 2023-11-28T18:25:37.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.909+0000 7f1682ffd640 1 -- 172.21.15.105:0/2813330052 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167c0aac60 con 0x7f1670004f60 2023-11-28T18:25:37.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.909+0000 7f1689ce3640 1 -- 172.21.15.105:0/2813330052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1670004f60 msgr2=0x7f167000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:37.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.909+0000 7f1689ce3640 1 --2- 172.21.15.105:0/2813330052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1670004f60 0x7f167000b990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f167c00a860 tx=0x7f167c01fa60 comp rx=0 tx=0).stop 2023-11-28T18:25:37.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.909+0000 7f1683fff640 1 -- 172.21.15.105:0/2813330052 reap_dead start 2023-11-28T18:25:37.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f1689ce3640 1 -- 172.21.15.105:0/2813330052 shutdown_connections 2023-11-28T18:25:37.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f1689ce3640 1 -- 172.21.15.105:0/2813330052 >> 172.21.15.105:0/2813330052 conn(0x7f16840ceb90 msgr2=0x7f16840cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:37.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f1689ce3640 1 -- 172.21.15.105:0/2813330052 shutdown_connections 2023-11-28T18:25:37.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f1689ce3640 1 -- 172.21.15.105:0/2813330052 wait complete. 2023-11-28T18:25:37.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f1689ce3640 1 Processor -- start 2023-11-28T18:25:37.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f1689ce3640 1 -- start start 2023-11-28T18:25:37.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f1689ce3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1670004f60 0x7f168415c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f1689ce3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1684141f70 0x7f16841423f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f1689ce3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1684167260 0x7f16841676e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f1689ce3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168414d390 con 0x7f1684141f70 2023-11-28T18:25:37.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f1689ce3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168414cf00 con 0x7f1670004f60 2023-11-28T18:25:37.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f1689ce3640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168414c2f0 con 0x7f1684167260 2023-11-28T18:25:37.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f167bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1684141f70 0x7f16841423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:37.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f16837fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1670004f60 0x7f168415c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:37.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f167bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1684141f70 0x7f16841423f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48528/0 (socket says 172.21.15.105:48528) 2023-11-28T18:25:37.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f167bfff640 1 -- 172.21.15.105:0/4020593553 learned_addr learned my addr 172.21.15.105:0/4020593553 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f1683fff640 1 --2- 172.21.15.105:0/4020593553 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1684167260 0x7f16841676e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f167bfff640 1 -- 172.21.15.105:0/4020593553 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1684167260 msgr2=0x7f16841676e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f167bfff640 1 --2- 172.21.15.105:0/4020593553 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1684167260 0x7f16841676e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f167bfff640 1 -- 172.21.15.105:0/4020593553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1670004f60 msgr2=0x7f168415c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f167bfff640 1 --2- 172.21.15.105:0/4020593553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1670004f60 0x7f168415c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f167bfff640 1 -- 172.21.15.105:0/4020593553 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168414c5d0 con 0x7f1684141f70 2023-11-28T18:25:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.913+0000 7f16837fe640 1 --2- 172.21.15.105:0/4020593553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1670004f60 0x7f168415c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:25:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.917+0000 7f167bfff640 1 --2- 172.21.15.105:0/4020593553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1684141f70 0x7f16841423f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f166c011cd0 tx=0x7f166c00f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.917+0000 7f1680ff9640 1 -- 172.21.15.105:0/4020593553 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c00fd20 con 0x7f1684141f70 2023-11-28T18:25:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.917+0000 7f1680ff9640 1 -- 172.21.15.105:0/4020593553 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166c00fe80 con 0x7f1684141f70 2023-11-28T18:25:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.917+0000 7f1689ce3640 1 -- 172.21.15.105:0/4020593553 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f168414c8b0 con 0x7f1684141f70 2023-11-28T18:25:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.917+0000 7f1680ff9640 1 -- 172.21.15.105:0/4020593553 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c004250 con 0x7f1684141f70 2023-11-28T18:25:37.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.917+0000 7f1689ce3640 1 -- 172.21.15.105:0/4020593553 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1684142930 con 0x7f1684141f70 2023-11-28T18:25:37.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.917+0000 7f1689ce3640 1 -- 172.21.15.105:0/4020593553 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f168414d0b0 con 0x7f1684141f70 2023-11-28T18:25:37.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.917+0000 7f1680ff9640 1 -- 172.21.15.105:0/4020593553 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f166c0043b0 con 0x7f1684141f70 2023-11-28T18:25:37.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.917+0000 7f1680ff9640 1 --2- 172.21.15.105:0/4020593553 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1664066900 0x7f1664068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:37.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.917+0000 7f1680ff9640 1 -- 172.21.15.105:0/4020593553 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f166c084ca0 con 0x7f1684141f70 2023-11-28T18:25:37.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.917+0000 7f16837fe640 1 --2- 172.21.15.105:0/4020593553 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1664066900 0x7f1664068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:37.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.917+0000 7f16837fe640 1 --2- 172.21.15.105:0/4020593553 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1664066900 0x7f1664068dc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f168415d520 tx=0x7f1674009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:37.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:37.921+0000 7f1680ff9640 1 -- 172.21.15.105:0/4020593553 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f168414d0b0 con 0x7f1684141f70 2023-11-28T18:25:38.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.073+0000 7f1689ce3640 1 -- 172.21.15.105:0/4020593553 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f168414d070 con 0x7f1684141f70 2023-11-28T18:25:38.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.073+0000 7f1680ff9640 1 -- 172.21.15.105:0/4020593553 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f166c022020 con 0x7f1684141f70 2023-11-28T18:25:38.075 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:25:38.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.073+0000 7f1689ce3640 1 -- 172.21.15.105:0/4020593553 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1664066900 msgr2=0x7f1664068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:38.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.073+0000 7f1689ce3640 1 --2- 172.21.15.105:0/4020593553 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1664066900 0x7f1664068dc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f168415d520 tx=0x7f1674009190 comp rx=0 tx=0).stop 2023-11-28T18:25:38.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.073+0000 7f1689ce3640 1 -- 172.21.15.105:0/4020593553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1684141f70 msgr2=0x7f16841423f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:38.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.073+0000 7f1689ce3640 1 --2- 172.21.15.105:0/4020593553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1684141f70 0x7f16841423f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f166c011cd0 tx=0x7f166c00f610 comp rx=0 tx=0).stop 2023-11-28T18:25:38.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.073+0000 7f1689ce3640 1 -- 172.21.15.105:0/4020593553 shutdown_connections 2023-11-28T18:25:38.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.073+0000 7f1689ce3640 1 --2- 172.21.15.105:0/4020593553 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1684167260 0x7f16841676e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.073+0000 7f1689ce3640 1 --2- 172.21.15.105:0/4020593553 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1664066900 0x7f1664068dc0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.073+0000 7f1689ce3640 1 --2- 172.21.15.105:0/4020593553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1684141f70 0x7f16841423f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.073+0000 7f1689ce3640 1 --2- 172.21.15.105:0/4020593553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1670004f60 0x7f168415c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.073+0000 7f1689ce3640 1 -- 172.21.15.105:0/4020593553 >> 172.21.15.105:0/4020593553 conn(0x7f16840ceb90 msgr2=0x7f16840b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:38.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.073+0000 7f1689ce3640 1 -- 172.21.15.105:0/4020593553 shutdown_connections 2023-11-28T18:25:38.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.073+0000 7f1689ce3640 1 -- 172.21.15.105:0/4020593553 wait complete. 2023-11-28T18:25:38.096 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:25:38.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.233+0000 7fd607c9a640 1 Processor -- start 2023-11-28T18:25:38.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.233+0000 7fd607c9a640 1 -- start start 2023-11-28T18:25:38.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd607c9a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd600159640 0x7fd600159a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:38.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd607c9a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd60015a020 con 0x7fd60006df70 2023-11-28T18:25:38.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd607c9a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd60015a300 con 0x7fd600071120 2023-11-28T18:25:38.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd607c9a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd60015a9c0 con 0x7fd600159640 2023-11-28T18:25:38.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd605235640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd60006df70 0x7fd600070a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50520/0 (socket says 172.21.15.105:50520) 2023-11-28T18:25:38.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd605235640 1 -- 172.21.15.105:0/1693484801 learned_addr learned my addr 172.21.15.105:0/1693484801 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd605a36640 1 --2- 172.21.15.105:0/1693484801 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd600159640 0x7fd600159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd604a34640 1 -- 172.21.15.105:0/1693484801 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2363738944 0 0) 0x7fd60015a020 con 0x7fd60006df70 2023-11-28T18:25:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd604a34640 1 -- 172.21.15.105:0/1693484801 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5ec005740 con 0x7fd60006df70 2023-11-28T18:25:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd604a34640 1 -- 172.21.15.105:0/1693484801 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1348361772 0 0) 0x7fd5ec005740 con 0x7fd60006df70 2023-11-28T18:25:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd604a34640 1 -- 172.21.15.105:0/1693484801 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd600159640 msgr2=0x7fd600159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd604a34640 1 --2- 172.21.15.105:0/1693484801 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd600159640 0x7fd600159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd604a34640 1 -- 172.21.15.105:0/1693484801 >> v1:172.21.15.148:6789/0 conn(0x7fd600071120 legacy=0x7fd60006a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd604a34640 1 -- 172.21.15.105:0/1693484801 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60015a5e0 con 0x7fd60006df70 2023-11-28T18:25:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd604a34640 1 -- 172.21.15.105:0/1693484801 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fd5f0001680 con 0x7fd60006df70 2023-11-28T18:25:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd604a34640 1 -- 172.21.15.105:0/1693484801 >> v1:172.21.15.105:6789/0 conn(0x7fd60006df70 legacy=0x7fd600070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd604a34640 1 --2- 172.21.15.105:0/1693484801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5ec005bc0 0x7fd5ec005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd604a34640 1 --2- 172.21.15.105:0/1693484801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5ec0069e0 0x7fd5ec006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd604a34640 1 --2- 172.21.15.105:0/1693484801 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd600159640 0x7fd5ec007470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd604a34640 1 -- 172.21.15.105:0/1693484801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60015a9c0 con 0x7fd5ec0069e0 2023-11-28T18:25:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd604a34640 1 -- 172.21.15.105:0/1693484801 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec00dd60 con 0x7fd5ec005bc0 2023-11-28T18:25:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd604a34640 1 -- 172.21.15.105:0/1693484801 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec00df20 con 0x7fd600159640 2023-11-28T18:25:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd605235640 1 --2- 172.21.15.105:0/1693484801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5ec0069e0 0x7fd5ec006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd605a36640 1 --2- 172.21.15.105:0/1693484801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5ec005bc0 0x7fd5ec005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:38.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd606237640 1 --2- 172.21.15.105:0/1693484801 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd600159640 0x7fd5ec007470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:38.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd605235640 1 -- 172.21.15.105:0/1693484801 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd600159640 msgr2=0x7fd5ec007470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:38.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd605235640 1 --2- 172.21.15.105:0/1693484801 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd600159640 0x7fd5ec007470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd605235640 1 -- 172.21.15.105:0/1693484801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5ec005bc0 msgr2=0x7fd5ec005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:38.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd605235640 1 --2- 172.21.15.105:0/1693484801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5ec005bc0 0x7fd5ec005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd605235640 1 -- 172.21.15.105:0/1693484801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ec005740 con 0x7fd5ec0069e0 2023-11-28T18:25:38.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.237+0000 7fd606237640 1 --2- 172.21.15.105:0/1693484801 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd600159640 0x7fd5ec007470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:38.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.241+0000 7fd605235640 1 --2- 172.21.15.105:0/1693484801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5ec0069e0 0x7fd5ec006e60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000c240 tx=0x7fd5f001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5aadfcc13ed7037 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:38.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.241+0000 7fd604a34640 1 -- 172.21.15.105:0/1693484801 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0005ae0 con 0x7fd5ec0069e0 2023-11-28T18:25:38.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.241+0000 7fd604a34640 1 -- 172.21.15.105:0/1693484801 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f0005c40 con 0x7fd5ec0069e0 2023-11-28T18:25:38.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.241+0000 7fd604a34640 1 -- 172.21.15.105:0/1693484801 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0007070 con 0x7fd5ec0069e0 2023-11-28T18:25:38.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.241+0000 7fd607c9a640 1 -- 172.21.15.105:0/1693484801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5ec0069e0 msgr2=0x7fd5ec006e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:38.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.241+0000 7fd607c9a640 1 --2- 172.21.15.105:0/1693484801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5ec0069e0 0x7fd5ec006e60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000c240 tx=0x7fd5f001cc40 comp rx=0 tx=0).stop 2023-11-28T18:25:38.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.241+0000 7fd606237640 1 -- 172.21.15.105:0/1693484801 reap_dead start 2023-11-28T18:25:38.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.241+0000 7fd607c9a640 1 -- 172.21.15.105:0/1693484801 shutdown_connections 2023-11-28T18:25:38.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.241+0000 7fd607c9a640 1 -- 172.21.15.105:0/1693484801 >> 172.21.15.105:0/1693484801 conn(0x7fd6000cf540 msgr2=0x7fd6000cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:38.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.241+0000 7fd607c9a640 1 -- 172.21.15.105:0/1693484801 shutdown_connections 2023-11-28T18:25:38.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.241+0000 7fd607c9a640 1 -- 172.21.15.105:0/1693484801 wait complete. 2023-11-28T18:25:38.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.241+0000 7fd607c9a640 1 Processor -- start 2023-11-28T18:25:38.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.241+0000 7fd607c9a640 1 -- start start 2023-11-28T18:25:38.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd607c9a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5ec0069e0 0x7fd6000c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:38.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd607c9a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6001584a0 0x7fd6000c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:38.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd607c9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6000c6e30 0x7fd6000cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:38.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd607c9a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6000c82a0 con 0x7fd6000c6e30 2023-11-28T18:25:38.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd607c9a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd600149650 con 0x7fd5ec0069e0 2023-11-28T18:25:38.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd607c9a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6000c5850 con 0x7fd6001584a0 2023-11-28T18:25:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd606237640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6000c6e30 0x7fd6000cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd606237640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6000c6e30 0x7fd6000cc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48554/0 (socket says 172.21.15.105:48554) 2023-11-28T18:25:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd606237640 1 -- 172.21.15.105:0/2487569464 learned_addr learned my addr 172.21.15.105:0/2487569464 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd605235640 1 --2- 172.21.15.105:0/2487569464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6001584a0 0x7fd6000c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd605a36640 1 --2- 172.21.15.105:0/2487569464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5ec0069e0 0x7fd6000c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd606237640 1 -- 172.21.15.105:0/2487569464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6001584a0 msgr2=0x7fd6000c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd606237640 1 --2- 172.21.15.105:0/2487569464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6001584a0 0x7fd6000c68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd606237640 1 -- 172.21.15.105:0/2487569464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5ec0069e0 msgr2=0x7fd6000c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd606237640 1 --2- 172.21.15.105:0/2487569464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5ec0069e0 0x7fd6000c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd606237640 1 -- 172.21.15.105:0/2487569464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6000c5b30 con 0x7fd6000c6e30 2023-11-28T18:25:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd605a36640 1 --2- 172.21.15.105:0/2487569464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5ec0069e0 0x7fd6000c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd606237640 1 --2- 172.21.15.105:0/2487569464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6000c6e30 0x7fd6000cc000 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8000f90 tx=0x7fd5f800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:38.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd5f67fc640 1 -- 172.21.15.105:0/2487569464 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8017070 con 0x7fd6000c6e30 2023-11-28T18:25:38.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd5f67fc640 1 -- 172.21.15.105:0/2487569464 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f8006b60 con 0x7fd6000c6e30 2023-11-28T18:25:38.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd5f67fc640 1 -- 172.21.15.105:0/2487569464 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8006d10 con 0x7fd6000c6e30 2023-11-28T18:25:38.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd607c9a640 1 -- 172.21.15.105:0/2487569464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6000c5e10 con 0x7fd6000c6e30 2023-11-28T18:25:38.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd607c9a640 1 -- 172.21.15.105:0/2487569464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6000c63d0 con 0x7fd6000c6e30 2023-11-28T18:25:38.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd607c9a640 1 -- 172.21.15.105:0/2487569464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6000c65c0 con 0x7fd6000c6e30 2023-11-28T18:25:38.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.245+0000 7fd5f67fc640 1 -- 172.21.15.105:0/2487569464 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd5f8003980 con 0x7fd6000c6e30 2023-11-28T18:25:38.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.249+0000 7fd5f67fc640 1 --2- 172.21.15.105:0/2487569464 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5e40668d0 0x7fd5e4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:38.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.249+0000 7fd5f67fc640 1 -- 172.21.15.105:0/2487569464 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd5f8083e00 con 0x7fd6000c6e30 2023-11-28T18:25:38.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.249+0000 7fd605a36640 1 --2- 172.21.15.105:0/2487569464 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5e40668d0 0x7fd5e4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:38.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.249+0000 7fd605a36640 1 --2- 172.21.15.105:0/2487569464 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5e40668d0 0x7fd5e4068d90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009ad0 tx=0x7fd5fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:38.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.253+0000 7fd5f67fc640 1 -- 172.21.15.105:0/2487569464 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd5f8050e40 con 0x7fd6000c6e30 2023-11-28T18:25:38.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.401+0000 7fd607c9a640 1 -- 172.21.15.105:0/2487569464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd6000c65c0 con 0x7fd6000c6e30 2023-11-28T18:25:38.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.405+0000 7fd5f67fc640 1 -- 172.21.15.105:0/2487569464 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd5f8054460 con 0x7fd6000c6e30 2023-11-28T18:25:38.406 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:25:38.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.405+0000 7fd607c9a640 1 -- 172.21.15.105:0/2487569464 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5e40668d0 msgr2=0x7fd5e4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:38.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.405+0000 7fd607c9a640 1 --2- 172.21.15.105:0/2487569464 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5e40668d0 0x7fd5e4068d90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009ad0 tx=0x7fd5fc008040 comp rx=0 tx=0).stop 2023-11-28T18:25:38.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.405+0000 7fd607c9a640 1 -- 172.21.15.105:0/2487569464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6000c6e30 msgr2=0x7fd6000cc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:38.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.405+0000 7fd607c9a640 1 --2- 172.21.15.105:0/2487569464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6000c6e30 0x7fd6000cc000 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8000f90 tx=0x7fd5f800eb90 comp rx=0 tx=0).stop 2023-11-28T18:25:38.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.405+0000 7fd607c9a640 1 -- 172.21.15.105:0/2487569464 shutdown_connections 2023-11-28T18:25:38.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.405+0000 7fd607c9a640 1 --2- 172.21.15.105:0/2487569464 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5e40668d0 0x7fd5e4068d90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.405+0000 7fd607c9a640 1 --2- 172.21.15.105:0/2487569464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6000c6e30 0x7fd6000cc000 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.405+0000 7fd607c9a640 1 --2- 172.21.15.105:0/2487569464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6001584a0 0x7fd6000c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.405+0000 7fd607c9a640 1 --2- 172.21.15.105:0/2487569464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5ec0069e0 0x7fd6000c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.405+0000 7fd607c9a640 1 -- 172.21.15.105:0/2487569464 >> 172.21.15.105:0/2487569464 conn(0x7fd6000cf540 msgr2=0x7fd600157320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:38.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.405+0000 7fd607c9a640 1 -- 172.21.15.105:0/2487569464 shutdown_connections 2023-11-28T18:25:38.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.405+0000 7fd607c9a640 1 -- 172.21.15.105:0/2487569464 wait complete. 2023-11-28T18:25:38.426 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T18:25:38.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.565+0000 7f7903279640 1 Processor -- start 2023-11-28T18:25:38.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.565+0000 7f7903279640 1 -- start start 2023-11-28T18:25:38.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.565+0000 7f7903279640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78fc071960 0x7f78fc15f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:38.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.565+0000 7f7903279640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f78fc15b030 0x7f78fc0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:38.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.565+0000 7f7903279640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78fc0701d0 con 0x7f78fc0b4290 2023-11-28T18:25:38.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.565+0000 7f7903279640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78fc06df80 con 0x7f78fc15b030 2023-11-28T18:25:38.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.565+0000 7f7903279640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc071df0 con 0x7f78fc071960 2023-11-28T18:25:38.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.565+0000 7f7901816640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f78fc15b030 0x7f78fc0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:38.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.565+0000 7f7901816640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f78fc15b030 0x7f78fc0709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48556/0 (socket says 172.21.15.105:48556) 2023-11-28T18:25:38.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.565+0000 7f7901816640 1 -- 172.21.15.105:0/2974172691 learned_addr learned my addr 172.21.15.105:0/2974172691 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:38.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.565+0000 7f7900814640 1 --2- 172.21.15.105:0/2974172691 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78fc071960 0x7f78fc15f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:38.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.565+0000 7f7901816640 1 -- 172.21.15.105:0/2974172691 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78fc071960 msgr2=0x7f78fc15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:38.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.565+0000 7f7901816640 1 --2- 172.21.15.105:0/2974172691 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78fc071960 0x7f78fc15f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.565+0000 7f7901816640 1 -- 172.21.15.105:0/2974172691 >> v1:172.21.15.148:6789/0 conn(0x7f78fc0b4290 legacy=0x7f78fc0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:38.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.565+0000 7f7901816640 1 -- 172.21.15.105:0/2974172691 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc0720d0 con 0x7f78fc15b030 2023-11-28T18:25:38.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.569+0000 7f7901816640 1 --2- 172.21.15.105:0/2974172691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc15b030 0x7f78fc0709b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009e50 tx=0x7f78f401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0e622d8e6803d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:38.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.569+0000 7f78f3fff640 1 -- 172.21.15.105:0/2974172691 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f40a57c0 con 0x7f78fc15b030 2023-11-28T18:25:38.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.569+0000 7f78f3fff640 1 -- 172.21.15.105:0/2974172691 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f40a5920 con 0x7f78fc15b030 2023-11-28T18:25:38.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.569+0000 7f78f3fff640 1 -- 172.21.15.105:0/2974172691 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f40a5cb0 con 0x7f78fc15b030 2023-11-28T18:25:38.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.569+0000 7f7903279640 1 -- 172.21.15.105:0/2974172691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc15b030 msgr2=0x7f78fc0709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:38.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.569+0000 7f7903279640 1 --2- 172.21.15.105:0/2974172691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc15b030 0x7f78fc0709b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009e50 tx=0x7f78f401c9e0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.569+0000 7f7903279640 1 -- 172.21.15.105:0/2974172691 shutdown_connections 2023-11-28T18:25:38.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.569+0000 7f7903279640 1 --2- 172.21.15.105:0/2974172691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc15b030 0x7f78fc0709b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.569+0000 7f7903279640 1 --2- 172.21.15.105:0/2974172691 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78fc071960 0x7f78fc15f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.569+0000 7f7903279640 1 -- 172.21.15.105:0/2974172691 >> 172.21.15.105:0/2974172691 conn(0x7f78fc0ceb90 msgr2=0x7f78fc0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:38.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.569+0000 7f7903279640 1 -- 172.21.15.105:0/2974172691 shutdown_connections 2023-11-28T18:25:38.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.569+0000 7f7903279640 1 -- 172.21.15.105:0/2974172691 wait complete. 2023-11-28T18:25:38.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7903279640 1 Processor -- start 2023-11-28T18:25:38.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7903279640 1 -- start start 2023-11-28T18:25:38.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7903279640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78fc0b4290 0x7f78fc148bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:38.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7903279640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78fc15b030 0x7f78fc163f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:38.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7903279640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc154aa0 0x7f78fc164450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:38.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7903279640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0b3280 con 0x7f78fc154aa0 2023-11-28T18:25:38.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7903279640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0cd260 con 0x7f78fc15b030 2023-11-28T18:25:38.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7903279640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc149130 con 0x7f78fc0b4290 2023-11-28T18:25:38.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7901816640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc154aa0 0x7f78fc164450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:38.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7901816640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc154aa0 0x7f78fc164450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48566/0 (socket says 172.21.15.105:48566) 2023-11-28T18:25:38.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7901816640 1 -- 172.21.15.105:0/3958058974 learned_addr learned my addr 172.21.15.105:0/3958058974 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:38.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7901015640 1 --2- 172.21.15.105:0/3958058974 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78fc0b4290 0x7f78fc148bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:38.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7900814640 1 --2- 172.21.15.105:0/3958058974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78fc15b030 0x7f78fc163f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:38.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7901816640 1 -- 172.21.15.105:0/3958058974 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78fc0b4290 msgr2=0x7f78fc148bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:38.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7901816640 1 --2- 172.21.15.105:0/3958058974 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78fc0b4290 0x7f78fc148bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7901816640 1 -- 172.21.15.105:0/3958058974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78fc15b030 msgr2=0x7f78fc163f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:38.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7901816640 1 --2- 172.21.15.105:0/3958058974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78fc15b030 0x7f78fc163f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7901816640 1 -- 172.21.15.105:0/3958058974 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f4009b30 con 0x7f78fc154aa0 2023-11-28T18:25:38.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7901015640 1 --2- 172.21.15.105:0/3958058974 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78fc0b4290 0x7f78fc148bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:38.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7900814640 1 --2- 172.21.15.105:0/3958058974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78fc15b030 0x7f78fc163f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:38.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7901816640 1 --2- 172.21.15.105:0/3958058974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc154aa0 0x7f78fc164450 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f78f40af040 tx=0x7f78f4019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:38.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f78f1ffb640 1 -- 172.21.15.105:0/3958058974 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f40b8670 con 0x7f78fc154aa0 2023-11-28T18:25:38.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f78f1ffb640 1 -- 172.21.15.105:0/3958058974 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f40b87d0 con 0x7f78fc154aa0 2023-11-28T18:25:38.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f78f1ffb640 1 -- 172.21.15.105:0/3958058974 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f40b8b80 con 0x7f78fc154aa0 2023-11-28T18:25:38.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7903279640 1 -- 172.21.15.105:0/3958058974 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78fc0b2cc0 con 0x7f78fc154aa0 2023-11-28T18:25:38.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7903279640 1 -- 172.21.15.105:0/3958058974 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78fc0ccef0 con 0x7f78fc154aa0 2023-11-28T18:25:38.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.573+0000 7f7903279640 1 -- 172.21.15.105:0/3958058974 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78fc067570 con 0x7f78fc154aa0 2023-11-28T18:25:38.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.577+0000 7f78f1ffb640 1 -- 172.21.15.105:0/3958058974 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f78f40b8ce0 con 0x7f78fc154aa0 2023-11-28T18:25:38.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.577+0000 7f78f1ffb640 1 --2- 172.21.15.105:0/3958058974 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78e0066900 0x7f78e0068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:38.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.577+0000 7f78f1ffb640 1 -- 172.21.15.105:0/3958058974 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f78f411f6d0 con 0x7f78fc154aa0 2023-11-28T18:25:38.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.577+0000 7f7901015640 1 --2- 172.21.15.105:0/3958058974 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78e0066900 0x7f78e0068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:38.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.577+0000 7f7901015640 1 --2- 172.21.15.105:0/3958058974 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78e0066900 0x7f78e0068dc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f78fc165a50 tx=0x7f78f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:38.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.581+0000 7f78f1ffb640 1 -- 172.21.15.105:0/3958058974 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f78f40ec7a0 con 0x7f78fc154aa0 2023-11-28T18:25:38.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.741+0000 7f7903279640 1 -- 172.21.15.105:0/3958058974 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f78fc073690 con 0x7f78fc154aa0 2023-11-28T18:25:38.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.741+0000 7f78f1ffb640 1 -- 172.21.15.105:0/3958058974 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) v1 ==== 80+0+0 (secure 0 0 0) 0x7f78f411de60 con 0x7f78fc154aa0 2023-11-28T18:25:38.745 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2023-11-28T18:25:38.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.745+0000 7f7903279640 1 -- 172.21.15.105:0/3958058974 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78e0066900 msgr2=0x7f78e0068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:38.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.745+0000 7f7903279640 1 --2- 172.21.15.105:0/3958058974 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78e0066900 0x7f78e0068dc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f78fc165a50 tx=0x7f78f8009040 comp rx=0 tx=0).stop 2023-11-28T18:25:38.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.745+0000 7f7903279640 1 -- 172.21.15.105:0/3958058974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc154aa0 msgr2=0x7f78fc164450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:38.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.745+0000 7f7903279640 1 --2- 172.21.15.105:0/3958058974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc154aa0 0x7f78fc164450 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f78f40af040 tx=0x7f78f4019780 comp rx=0 tx=0).stop 2023-11-28T18:25:38.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.745+0000 7f7903279640 1 -- 172.21.15.105:0/3958058974 shutdown_connections 2023-11-28T18:25:38.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.745+0000 7f7903279640 1 --2- 172.21.15.105:0/3958058974 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78e0066900 0x7f78e0068dc0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.745+0000 7f7903279640 1 --2- 172.21.15.105:0/3958058974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc154aa0 0x7f78fc164450 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.745+0000 7f7903279640 1 --2- 172.21.15.105:0/3958058974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78fc15b030 0x7f78fc163f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.745+0000 7f7903279640 1 --2- 172.21.15.105:0/3958058974 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78fc0b4290 0x7f78fc148bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:38.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.745+0000 7f7903279640 1 -- 172.21.15.105:0/3958058974 >> 172.21.15.105:0/3958058974 conn(0x7f78fc0ceb90 msgr2=0x7f78fc1565c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:38.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.745+0000 7f7903279640 1 -- 172.21.15.105:0/3958058974 shutdown_connections 2023-11-28T18:25:38.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:38.745+0000 7f7903279640 1 -- 172.21.15.105:0/3958058974 wait complete. 2023-11-28T18:25:38.766 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2023-11-28T18:25:38.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:25:38.766 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-28T18:25:38.785 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[50275]: fuse finished with error 0 and tester_r 0 2023-11-28T18:25:38.789 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:25:38.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:25:38.789 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T18:25:38.962 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:25:38.962 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:25:38.972 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:25:38.972 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:25:38.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:25:38.972 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T18:25:39.022 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-28T18:25:39.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:25:39.023 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:25:39.023 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:25:39.032 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:25:39.032 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:25:39.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:25:39.032 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T18:25:39.081 INFO:teuthology.orchestra.run.smithi105.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-28T18:25:39.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:25:39.082 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:25:39.082 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:25:39.091 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:25:39.091 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:25:39.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:25:39.091 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T18:25:39.142 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-28T18:25:39.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:25:39.143 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:25:39.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.277+0000 7f9f61838640 1 Processor -- start 2023-11-28T18:25:39.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.277+0000 7f9f61838640 1 -- start start 2023-11-28T18:25:39.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f61838640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9f5c15cef0 0x7f9f5c0d1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:39.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f61838640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5c0ce840 con 0x7f9f5c0d1b40 2023-11-28T18:25:39.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f61838640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5c0cd990 con 0x7f9f5c0cfb30 2023-11-28T18:25:39.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f61838640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f5c0b4780 con 0x7f9f5c15cef0 2023-11-28T18:25:39.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f53fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9f5c0d1b40 0x7f9f5c0cf420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50522/0 (socket says 172.21.15.105:50522) 2023-11-28T18:25:39.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f53fff640 1 -- 172.21.15.105:0/2131036676 learned_addr learned my addr 172.21.15.105:0/2131036676 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:39.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f537fe640 1 -- 172.21.15.105:0/2131036676 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983658130 0 0) 0x7f9f5c0ce840 con 0x7f9f5c0d1b40 2023-11-28T18:25:39.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f537fe640 1 -- 172.21.15.105:0/2131036676 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f44003530 con 0x7f9f5c0d1b40 2023-11-28T18:25:39.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f60836640 1 --2- 172.21.15.105:0/2131036676 >> v2:172.21.15.148:3300/0 conn(0x7f9f5c15cef0 0x7f9f5c0d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:39.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f537fe640 1 -- 172.21.15.105:0/2131036676 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2021636868 0 0) 0x7f9f44003530 con 0x7f9f5c0d1b40 2023-11-28T18:25:39.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f537fe640 1 -- 172.21.15.105:0/2131036676 >> v2:172.21.15.148:3300/0 conn(0x7f9f5c15cef0 msgr2=0x7f9f5c0d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:39.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f537fe640 1 --2- 172.21.15.105:0/2131036676 >> v2:172.21.15.148:3300/0 conn(0x7f9f5c15cef0 0x7f9f5c0d1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f537fe640 1 -- 172.21.15.105:0/2131036676 >> v1:172.21.15.148:6789/0 conn(0x7f9f5c0cfb30 legacy=0x7f9f5c166fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:39.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f537fe640 1 -- 172.21.15.105:0/2131036676 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5c0cecd0 con 0x7f9f5c0d1b40 2023-11-28T18:25:39.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f537fe640 1 -- 172.21.15.105:0/2131036676 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f9f4c001680 con 0x7f9f5c0d1b40 2023-11-28T18:25:39.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f537fe640 1 -- 172.21.15.105:0/2131036676 >> v1:172.21.15.105:6789/0 conn(0x7f9f5c0d1b40 legacy=0x7f9f5c0cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:39.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f537fe640 1 --2- 172.21.15.105:0/2131036676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f44003980 0x7f9f44003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:39.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f537fe640 1 --2- 172.21.15.105:0/2131036676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f440047d0 0x7f9f44004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:39.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f537fe640 1 --2- 172.21.15.105:0/2131036676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f44005190 0x7f9f4400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:39.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f537fe640 1 -- 172.21.15.105:0/2131036676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c0b4780 con 0x7f9f44005190 2023-11-28T18:25:39.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f537fe640 1 -- 172.21.15.105:0/2131036676 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4400c130 con 0x7f9f440047d0 2023-11-28T18:25:39.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f537fe640 1 -- 172.21.15.105:0/2131036676 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4400c2f0 con 0x7f9f44003980 2023-11-28T18:25:39.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f61037640 1 --2- 172.21.15.105:0/2131036676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f44005190 0x7f9f4400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:39.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f60836640 1 --2- 172.21.15.105:0/2131036676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f44003980 0x7f9f44003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:39.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f53fff640 1 --2- 172.21.15.105:0/2131036676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f440047d0 0x7f9f44004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:39.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f61037640 1 -- 172.21.15.105:0/2131036676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f44003980 msgr2=0x7f9f44003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:39.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f61037640 1 --2- 172.21.15.105:0/2131036676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f44003980 0x7f9f44003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f61037640 1 -- 172.21.15.105:0/2131036676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f440047d0 msgr2=0x7f9f44004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:39.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f61037640 1 --2- 172.21.15.105:0/2131036676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f440047d0 0x7f9f44004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.281+0000 7f9f61037640 1 -- 172.21.15.105:0/2131036676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f44003530 con 0x7f9f44005190 2023-11-28T18:25:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.285+0000 7f9f61037640 1 -- 172.21.15.105:0/2131036676 reap_dead start 2023-11-28T18:25:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.285+0000 7f9f61037640 1 --2- 172.21.15.105:0/2131036676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f44005190 0x7f9f4400b990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9f54001e40 tx=0x7f9f5401c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15eaa712e9535ca0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.285+0000 7f9f537fe640 1 -- 172.21.15.105:0/2131036676 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54003b80 con 0x7f9f44005190 2023-11-28T18:25:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.285+0000 7f9f537fe640 1 -- 172.21.15.105:0/2131036676 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f54003ce0 con 0x7f9f44005190 2023-11-28T18:25:39.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.285+0000 7f9f537fe640 1 -- 172.21.15.105:0/2131036676 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54005310 con 0x7f9f44005190 2023-11-28T18:25:39.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.285+0000 7f9f61838640 1 -- 172.21.15.105:0/2131036676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f44005190 msgr2=0x7f9f4400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:39.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.285+0000 7f9f61838640 1 --2- 172.21.15.105:0/2131036676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f44005190 0x7f9f4400b990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9f54001e40 tx=0x7f9f5401c740 comp rx=0 tx=0).stop 2023-11-28T18:25:39.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.285+0000 7f9f61838640 1 -- 172.21.15.105:0/2131036676 shutdown_connections 2023-11-28T18:25:39.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.285+0000 7f9f61838640 1 --2- 172.21.15.105:0/2131036676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f44005190 0x7f9f4400b990 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.285+0000 7f9f61838640 1 -- 172.21.15.105:0/2131036676 >> 172.21.15.105:0/2131036676 conn(0x7f9f5c0d2e10 msgr2=0x7f9f5c0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:39.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.285+0000 7f9f61838640 1 -- 172.21.15.105:0/2131036676 shutdown_connections 2023-11-28T18:25:39.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.285+0000 7f9f61838640 1 -- 172.21.15.105:0/2131036676 wait complete. 2023-11-28T18:25:39.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.285+0000 7f9f61838640 1 Processor -- start 2023-11-28T18:25:39.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f61838640 1 -- start start 2023-11-28T18:25:39.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f61838640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f44005190 0x7f9f5c0c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:39.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f61838640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f5c0b5a30 0x7f9f5c0b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:39.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f61838640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f5c15af90 0x7f9f5c15b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:39.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f61838640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c15b950 con 0x7f9f5c15af90 2023-11-28T18:25:39.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f61838640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c0b8720 con 0x7f9f44005190 2023-11-28T18:25:39.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f61838640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c0c9040 con 0x7f9f5c0b5a30 2023-11-28T18:25:39.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f61037640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f5c15af90 0x7f9f5c15b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:39.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f61037640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f5c15af90 0x7f9f5c15b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48580/0 (socket says 172.21.15.105:48580) 2023-11-28T18:25:39.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f61037640 1 -- 172.21.15.105:0/1432805363 learned_addr learned my addr 172.21.15.105:0/1432805363 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:39.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f53fff640 1 --2- 172.21.15.105:0/1432805363 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f5c0b5a30 0x7f9f5c0b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:39.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f60836640 1 --2- 172.21.15.105:0/1432805363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f44005190 0x7f9f5c0c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:39.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f61037640 1 -- 172.21.15.105:0/1432805363 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f5c0b5a30 msgr2=0x7f9f5c0b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:39.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f61037640 1 --2- 172.21.15.105:0/1432805363 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f5c0b5a30 0x7f9f5c0b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f61037640 1 -- 172.21.15.105:0/1432805363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f44005190 msgr2=0x7f9f5c0c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:39.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f61037640 1 --2- 172.21.15.105:0/1432805363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f44005190 0x7f9f5c0c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f61037640 1 -- 172.21.15.105:0/1432805363 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5c0b7b10 con 0x7f9f5c15af90 2023-11-28T18:25:39.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f60836640 1 --2- 172.21.15.105:0/1432805363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f44005190 0x7f9f5c0c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:39.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f53fff640 1 --2- 172.21.15.105:0/1432805363 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f5c0b5a30 0x7f9f5c0b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:39.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f61037640 1 --2- 172.21.15.105:0/1432805363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f5c15af90 0x7f9f5c15b410 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9f5401cfe0 tx=0x7f9f44005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:39.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f517fa640 1 -- 172.21.15.105:0/1432805363 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54003b80 con 0x7f9f5c15af90 2023-11-28T18:25:39.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f61838640 1 -- 172.21.15.105:0/1432805363 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5c0b7df0 con 0x7f9f5c15af90 2023-11-28T18:25:39.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f517fa640 1 -- 172.21.15.105:0/1432805363 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f54007dc0 con 0x7f9f5c15af90 2023-11-28T18:25:39.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f517fa640 1 -- 172.21.15.105:0/1432805363 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54006da0 con 0x7f9f5c15af90 2023-11-28T18:25:39.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f61838640 1 -- 172.21.15.105:0/1432805363 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5c0b83b0 con 0x7f9f5c15af90 2023-11-28T18:25:39.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.289+0000 7f9f61838640 1 -- 172.21.15.105:0/1432805363 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f5c0b8860 con 0x7f9f5c15af90 2023-11-28T18:25:39.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.293+0000 7f9f517fa640 1 -- 172.21.15.105:0/1432805363 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f9f540a7070 con 0x7f9f5c15af90 2023-11-28T18:25:39.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.293+0000 7f9f517fa640 1 --2- 172.21.15.105:0/1432805363 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9f480668d0 0x7f9f48068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:39.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.293+0000 7f9f517fa640 1 -- 172.21.15.105:0/1432805363 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9f540b0070 con 0x7f9f5c15af90 2023-11-28T18:25:39.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.293+0000 7f9f60836640 1 --2- 172.21.15.105:0/1432805363 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9f480668d0 0x7f9f48068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:39.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.293+0000 7f9f60836640 1 --2- 172.21.15.105:0/1432805363 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9f480668d0 0x7f9f48068d90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9f580093f0 tx=0x7f9f58008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:39.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.297+0000 7f9f517fa640 1 -- 172.21.15.105:0/1432805363 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9f540eab80 con 0x7f9f5c15af90 2023-11-28T18:25:39.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.481+0000 7f9f61838640 1 -- 172.21.15.105:0/1432805363 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f5c0d03c0 con 0x7f9f5c15af90 2023-11-28T18:25:39.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.481+0000 7f9f517fa640 1 -- 172.21.15.105:0/1432805363 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f9f540ee1a0 con 0x7f9f5c15af90 2023-11-28T18:25:39.483 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:39.483 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/47307750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":47307750},{"type":"v1","addr":"172.21.15.148:6834","nonce":47307750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:24:55.936428+0000","modified":"2023-11-28T18:25:33.692231+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5469,"mds_1":5463,"mds_2":5392},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5392":{"gid":5392,"name":"d","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6835/2878687703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2878687703},{"type":"v1","addr":"172.21.15.148:6835","nonce":2878687703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5463":{"gid":5463,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.105:6836/681913204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":681913204},{"type":"v1","addr":"172.21.15.105:6836","nonce":681913204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5469":{"gid":5469,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/1876408690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":1876408690},{"type":"v1","addr":"172.21.15.105:6837","nonce":1876408690}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T18:25:39.484 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 25 2023-11-28T18:25:39.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.481+0000 7f9f61838640 1 -- 172.21.15.105:0/1432805363 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9f480668d0 msgr2=0x7f9f48068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:39.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.481+0000 7f9f61838640 1 --2- 172.21.15.105:0/1432805363 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9f480668d0 0x7f9f48068d90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9f580093f0 tx=0x7f9f58008040 comp rx=0 tx=0).stop 2023-11-28T18:25:39.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.481+0000 7f9f61838640 1 -- 172.21.15.105:0/1432805363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f5c15af90 msgr2=0x7f9f5c15b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:39.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.481+0000 7f9f61838640 1 --2- 172.21.15.105:0/1432805363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f5c15af90 0x7f9f5c15b410 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9f5401cfe0 tx=0x7f9f44005800 comp rx=0 tx=0).stop 2023-11-28T18:25:39.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.481+0000 7f9f61838640 1 -- 172.21.15.105:0/1432805363 shutdown_connections 2023-11-28T18:25:39.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.481+0000 7f9f61838640 1 --2- 172.21.15.105:0/1432805363 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9f480668d0 0x7f9f48068d90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.481+0000 7f9f61838640 1 --2- 172.21.15.105:0/1432805363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f5c15af90 0x7f9f5c15b410 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.481+0000 7f9f61838640 1 --2- 172.21.15.105:0/1432805363 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f5c0b5a30 0x7f9f5c0b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.481+0000 7f9f61838640 1 --2- 172.21.15.105:0/1432805363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f44005190 0x7f9f5c0c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.481+0000 7f9f61838640 1 -- 172.21.15.105:0/1432805363 >> 172.21.15.105:0/1432805363 conn(0x7f9f5c0d2e10 msgr2=0x7f9f5c06c400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:39.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.485+0000 7f9f61838640 1 -- 172.21.15.105:0/1432805363 shutdown_connections 2023-11-28T18:25:39.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.485+0000 7f9f61838640 1 -- 172.21.15.105:0/1432805363 wait complete. 2023-11-28T18:25:39.504 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:25:39.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.637+0000 7f7dfe032640 1 Processor -- start 2023-11-28T18:25:39.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.637+0000 7f7dfe032640 1 -- start start 2023-11-28T18:25:39.643 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.641+0000 7f7dfe032640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7df8071960 0x7f7df815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:39.643 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.641+0000 7f7dfe032640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7df80701d0 con 0x7f7df80b4290 2023-11-28T18:25:39.643 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.641+0000 7f7dfe032640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7df806df80 con 0x7f7df815b030 2023-11-28T18:25:39.643 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.641+0000 7f7dfe032640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7df8071df0 con 0x7f7df8071960 2023-11-28T18:25:39.643 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.641+0000 7f7df6ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7df8071960 0x7f7df815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:39.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.641+0000 7f7df6ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7df8071960 0x7f7df815f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48590/0 (socket says 172.21.15.105:48590) 2023-11-28T18:25:39.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.641+0000 7f7df6ffd640 1 -- 172.21.15.105:0/2034701461 learned_addr learned my addr 172.21.15.105:0/2034701461 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:39.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.641+0000 7f7df6ffd640 1 -- 172.21.15.105:0/2034701461 >> v1:172.21.15.148:6789/0 conn(0x7f7df815b030 legacy=0x7f7df80709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:39.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.641+0000 7f7df6ffd640 1 -- 172.21.15.105:0/2034701461 >> v1:172.21.15.105:6789/0 conn(0x7f7df80b4290 legacy=0x7f7df80b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:39.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.641+0000 7f7df6ffd640 1 -- 172.21.15.105:0/2034701461 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df80720d0 con 0x7f7df8071960 2023-11-28T18:25:39.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.645+0000 7f7df6ffd640 1 --2- 172.21.15.105:0/2034701461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7df8071960 0x7f7df815f3b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7dec009870 tx=0x7f7dec01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5a9e57f13a225b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:39.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.645+0000 7f7df67fc640 1 -- 172.21.15.105:0/2034701461 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec0aa430 con 0x7f7df8071960 2023-11-28T18:25:39.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.645+0000 7f7df67fc640 1 -- 172.21.15.105:0/2034701461 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dec0aa590 con 0x7f7df8071960 2023-11-28T18:25:39.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.645+0000 7f7df67fc640 1 -- 172.21.15.105:0/2034701461 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec0aa920 con 0x7f7df8071960 2023-11-28T18:25:39.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.645+0000 7f7dfe032640 1 -- 172.21.15.105:0/2034701461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7df8071960 msgr2=0x7f7df815f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:39.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.645+0000 7f7dfe032640 1 --2- 172.21.15.105:0/2034701461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7df8071960 0x7f7df815f3b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7dec009870 tx=0x7f7dec01c4d0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.645+0000 7f7dfe032640 1 -- 172.21.15.105:0/2034701461 shutdown_connections 2023-11-28T18:25:39.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.645+0000 7f7dfe032640 1 --2- 172.21.15.105:0/2034701461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7df8071960 0x7f7df815f3b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.645+0000 7f7dfe032640 1 -- 172.21.15.105:0/2034701461 >> 172.21.15.105:0/2034701461 conn(0x7f7df80ceb90 msgr2=0x7f7df80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:39.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.645+0000 7f7dfe032640 1 -- 172.21.15.105:0/2034701461 shutdown_connections 2023-11-28T18:25:39.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.645+0000 7f7dfe032640 1 -- 172.21.15.105:0/2034701461 wait complete. 2023-11-28T18:25:39.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.645+0000 7f7dfe032640 1 Processor -- start 2023-11-28T18:25:39.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.645+0000 7f7dfe032640 1 -- start start 2023-11-28T18:25:39.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7dfe032640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df80b4290 0x7f7df8152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:39.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7dfe032640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7df815b030 0x7f7df8099e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:39.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7dfe032640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df809a350 0x7f7df80cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:39.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7dfe032640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df809a8b0 con 0x7f7df815b030 2023-11-28T18:25:39.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7dfe032640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df80b1d80 con 0x7f7df80b4290 2023-11-28T18:25:39.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7dfe032640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df80b2060 con 0x7f7df809a350 2023-11-28T18:25:39.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7df6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7df815b030 0x7f7df8099e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:39.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7df6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7df815b030 0x7f7df8099e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48606/0 (socket says 172.21.15.105:48606) 2023-11-28T18:25:39.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7df6ffd640 1 -- 172.21.15.105:0/3646322637 learned_addr learned my addr 172.21.15.105:0/3646322637 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:39.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7df77fe640 1 --2- 172.21.15.105:0/3646322637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df80b4290 0x7f7df8152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:39.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7df7fff640 1 --2- 172.21.15.105:0/3646322637 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df809a350 0x7f7df80cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:39.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7df6ffd640 1 -- 172.21.15.105:0/3646322637 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df809a350 msgr2=0x7f7df80cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:39.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7df6ffd640 1 --2- 172.21.15.105:0/3646322637 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df809a350 0x7f7df80cd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7df6ffd640 1 -- 172.21.15.105:0/3646322637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df80b4290 msgr2=0x7f7df8152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:39.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7df6ffd640 1 --2- 172.21.15.105:0/3646322637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df80b4290 0x7f7df8152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7df6ffd640 1 -- 172.21.15.105:0/3646322637 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dec0094d0 con 0x7f7df815b030 2023-11-28T18:25:39.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7df6ffd640 1 --2- 172.21.15.105:0/3646322637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7df815b030 0x7f7df8099e10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0ac040 tx=0x7f7dec01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:39.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7ddffff640 1 -- 172.21.15.105:0/3646322637 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec0b38c0 con 0x7f7df815b030 2023-11-28T18:25:39.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7ddffff640 1 -- 172.21.15.105:0/3646322637 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dec0b3a40 con 0x7f7df815b030 2023-11-28T18:25:39.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7ddffff640 1 -- 172.21.15.105:0/3646322637 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec0b3df0 con 0x7f7df815b030 2023-11-28T18:25:39.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7dfe032640 1 -- 172.21.15.105:0/3646322637 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df80b2340 con 0x7f7df815b030 2023-11-28T18:25:39.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7dfe032640 1 -- 172.21.15.105:0/3646322637 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df8153570 con 0x7f7df815b030 2023-11-28T18:25:39.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7dfe032640 1 -- 172.21.15.105:0/3646322637 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df8067570 con 0x7f7df815b030 2023-11-28T18:25:39.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.649+0000 7f7ddffff640 1 -- 172.21.15.105:0/3646322637 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f7dec0b2380 con 0x7f7df815b030 2023-11-28T18:25:39.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.653+0000 7f7ddffff640 1 --2- 172.21.15.105:0/3646322637 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7de40668d0 0x7f7de4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:39.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.653+0000 7f7ddffff640 1 -- 172.21.15.105:0/3646322637 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7dec11ed90 con 0x7f7df815b030 2023-11-28T18:25:39.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.653+0000 7f7df77fe640 1 --2- 172.21.15.105:0/3646322637 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7de40668d0 0x7f7de4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:39.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.653+0000 7f7df77fe640 1 --2- 172.21.15.105:0/3646322637 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7de40668d0 0x7f7de4068d90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7de8000f70 tx=0x7f7de8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:39.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.657+0000 7f7ddffff640 1 -- 172.21.15.105:0/3646322637 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7dec0ebf10 con 0x7f7df815b030 2023-11-28T18:25:39.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.805+0000 7f7dfe032640 1 -- 172.21.15.105:0/3646322637 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7df80b27f0 con 0x7f7df815b030 2023-11-28T18:25:39.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.805+0000 7f7ddffff640 1 -- 172.21.15.105:0/3646322637 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16827 (secure 0 0 0) 0x7f7dec0c3090 con 0x7f7df815b030 2023-11-28T18:25:39.808 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:39.808 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":40,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:25:15.107712+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:24:53.582892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T18:24:54.921978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.39","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:25:39.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.809+0000 7f7dfe032640 1 -- 172.21.15.105:0/3646322637 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7de40668d0 msgr2=0x7f7de4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:39.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.809+0000 7f7dfe032640 1 --2- 172.21.15.105:0/3646322637 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7de40668d0 0x7f7de4068d90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7de8000f70 tx=0x7f7de8009040 comp rx=0 tx=0).stop 2023-11-28T18:25:39.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.809+0000 7f7dfe032640 1 -- 172.21.15.105:0/3646322637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7df815b030 msgr2=0x7f7df8099e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:39.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.809+0000 7f7dfe032640 1 --2- 172.21.15.105:0/3646322637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7df815b030 0x7f7df8099e10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0ac040 tx=0x7f7dec01ad10 comp rx=0 tx=0).stop 2023-11-28T18:25:39.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.809+0000 7f7dfe032640 1 -- 172.21.15.105:0/3646322637 shutdown_connections 2023-11-28T18:25:39.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.809+0000 7f7dfe032640 1 --2- 172.21.15.105:0/3646322637 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df809a350 0x7f7df80cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.809+0000 7f7dfe032640 1 --2- 172.21.15.105:0/3646322637 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7de40668d0 0x7f7de4068d90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.809+0000 7f7dfe032640 1 --2- 172.21.15.105:0/3646322637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7df815b030 0x7f7df8099e10 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.809+0000 7f7dfe032640 1 --2- 172.21.15.105:0/3646322637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df80b4290 0x7f7df8152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.809+0000 7f7dfe032640 1 -- 172.21.15.105:0/3646322637 >> 172.21.15.105:0/3646322637 conn(0x7f7df80ceb90 msgr2=0x7f7df806d250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:39.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.809+0000 7f7dfe032640 1 -- 172.21.15.105:0/3646322637 shutdown_connections 2023-11-28T18:25:39.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.809+0000 7f7dfe032640 1 -- 172.21.15.105:0/3646322637 wait complete. 2023-11-28T18:25:39.830 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:24:53.582892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:24:54.921978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:25:39.830 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:25:39.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.969+0000 7f842e1aa640 1 Processor -- start 2023-11-28T18:25:39.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.969+0000 7f842e1aa640 1 -- start start 2023-11-28T18:25:39.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.969+0000 7f842e1aa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f84280b8a20 0x7f84280cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:39.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.969+0000 7f842e1aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84280d00b0 0x7f84280b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:39.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.969+0000 7f842e1aa640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84280d3e60 con 0x7f84280ceb40 2023-11-28T18:25:39.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.969+0000 7f842e1aa640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84280b5fe0 con 0x7f84280b8a20 2023-11-28T18:25:39.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.969+0000 7f842e1aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84280b62c0 con 0x7f84280d00b0 2023-11-28T18:25:39.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.969+0000 7f84277fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f84280b8a20 0x7f84280cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:39.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.969+0000 7f8427fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84280d00b0 0x7f84280b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:39.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.969+0000 7f84277fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f84280b8a20 0x7f84280cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:46728/0 (socket says 172.21.15.105:46728) 2023-11-28T18:25:39.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.969+0000 7f84277fe640 1 -- 172.21.15.105:0/546493391 learned_addr learned my addr 172.21.15.105:0/546493391 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:39.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.969+0000 7f84277fe640 1 -- 172.21.15.105:0/546493391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84280d00b0 msgr2=0x7f84280b6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:39.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.969+0000 7f84277fe640 1 --2- 172.21.15.105:0/546493391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84280d00b0 0x7f84280b6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.969+0000 7f84277fe640 1 -- 172.21.15.105:0/546493391 >> v1:172.21.15.148:6789/0 conn(0x7f84280ceb40 legacy=0x7f84280cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:39.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.969+0000 7f84277fe640 1 -- 172.21.15.105:0/546493391 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84280b65a0 con 0x7f84280b8a20 2023-11-28T18:25:39.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.969+0000 7f8427fff640 1 --2- 172.21.15.105:0/546493391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84280d00b0 0x7f84280b6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:25:39.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.973+0000 7f84277fe640 1 --2- 172.21.15.105:0/546493391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84280b8a20 0x7f84280cdb50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8418009870 tx=0x7f841801c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27689690694822fe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:39.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.973+0000 7f84267fc640 1 -- 172.21.15.105:0/546493391 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84180ab670 con 0x7f84280b8a20 2023-11-28T18:25:39.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.973+0000 7f84267fc640 1 -- 172.21.15.105:0/546493391 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84180ab7d0 con 0x7f84280b8a20 2023-11-28T18:25:39.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.973+0000 7f84267fc640 1 -- 172.21.15.105:0/546493391 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84180abb60 con 0x7f84280b8a20 2023-11-28T18:25:39.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.973+0000 7f842e1aa640 1 -- 172.21.15.105:0/546493391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84280b8a20 msgr2=0x7f84280cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:39.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.973+0000 7f842e1aa640 1 --2- 172.21.15.105:0/546493391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84280b8a20 0x7f84280cdb50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8418009870 tx=0x7f841801c510 comp rx=0 tx=0).stop 2023-11-28T18:25:39.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.973+0000 7f842e1aa640 1 -- 172.21.15.105:0/546493391 shutdown_connections 2023-11-28T18:25:39.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.973+0000 7f842e1aa640 1 --2- 172.21.15.105:0/546493391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84280d00b0 0x7f84280b6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.973+0000 7f842e1aa640 1 --2- 172.21.15.105:0/546493391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84280b8a20 0x7f84280cdb50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.973+0000 7f842e1aa640 1 -- 172.21.15.105:0/546493391 >> 172.21.15.105:0/546493391 conn(0x7f842809c340 msgr2=0x7f84280d2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:39.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.973+0000 7f842e1aa640 1 -- 172.21.15.105:0/546493391 shutdown_connections 2023-11-28T18:25:39.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.973+0000 7f842e1aa640 1 -- 172.21.15.105:0/546493391 wait complete. 2023-11-28T18:25:39.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f842e1aa640 1 Processor -- start 2023-11-28T18:25:39.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f842e1aa640 1 -- start start 2023-11-28T18:25:39.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f842e1aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84280b8a20 0x7f8428155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:39.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f842e1aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84280ceb40 0x7f8428155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:39.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f842e1aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84280d00b0 0x7f8428157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:39.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f842e1aa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842815c900 con 0x7f84280b8a20 2023-11-28T18:25:39.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f842e1aa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8428158d00 con 0x7f84280ceb40 2023-11-28T18:25:39.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f842e1aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8428158740 con 0x7f84280d00b0 2023-11-28T18:25:39.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f84277fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84280b8a20 0x7f8428155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:39.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f84277fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84280b8a20 0x7f8428155960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48608/0 (socket says 172.21.15.105:48608) 2023-11-28T18:25:39.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f84277fe640 1 -- 172.21.15.105:0/3756504838 learned_addr learned my addr 172.21.15.105:0/3756504838 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:39.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f8426ffd640 1 --2- 172.21.15.105:0/3756504838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84280ceb40 0x7f8428155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:39.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f8427fff640 1 --2- 172.21.15.105:0/3756504838 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84280d00b0 0x7f8428157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:39.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f84277fe640 1 -- 172.21.15.105:0/3756504838 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84280d00b0 msgr2=0x7f8428157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:39.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f84277fe640 1 --2- 172.21.15.105:0/3756504838 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84280d00b0 0x7f8428157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f84277fe640 1 -- 172.21.15.105:0/3756504838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84280ceb40 msgr2=0x7f8428155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:39.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f84277fe640 1 --2- 172.21.15.105:0/3756504838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84280ceb40 0x7f8428155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:39.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f84277fe640 1 -- 172.21.15.105:0/3756504838 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8420009a70 con 0x7f84280b8a20 2023-11-28T18:25:39.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f8426ffd640 1 --2- 172.21.15.105:0/3756504838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84280ceb40 0x7f8428155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:39.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f84277fe640 1 --2- 172.21.15.105:0/3756504838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84280b8a20 0x7f8428155960 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f84180ad040 tx=0x7f841801ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:39.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f840ffff640 1 -- 172.21.15.105:0/3756504838 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84180b4800 con 0x7f84280b8a20 2023-11-28T18:25:39.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f840ffff640 1 -- 172.21.15.105:0/3756504838 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84180b4960 con 0x7f84280b8a20 2023-11-28T18:25:39.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f842e1aa640 1 -- 172.21.15.105:0/3756504838 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84180094d0 con 0x7f84280b8a20 2023-11-28T18:25:39.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f840ffff640 1 -- 172.21.15.105:0/3756504838 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84180b4d10 con 0x7f84280b8a20 2023-11-28T18:25:39.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.977+0000 7f842e1aa640 1 -- 172.21.15.105:0/3756504838 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842815c590 con 0x7f84280b8a20 2023-11-28T18:25:39.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.981+0000 7f842e1aa640 1 -- 172.21.15.105:0/3756504838 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8428067570 con 0x7f84280b8a20 2023-11-28T18:25:39.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.981+0000 7f840ffff640 1 -- 172.21.15.105:0/3756504838 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f84180c7440 con 0x7f84280b8a20 2023-11-28T18:25:39.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.981+0000 7f840ffff640 1 --2- 172.21.15.105:0/3756504838 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f84140668d0 0x7f8414068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:39.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.981+0000 7f840ffff640 1 -- 172.21.15.105:0/3756504838 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f841811f240 con 0x7f84280b8a20 2023-11-28T18:25:39.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.981+0000 7f8426ffd640 1 --2- 172.21.15.105:0/3756504838 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f84140668d0 0x7f8414068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:39.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.981+0000 7f8426ffd640 1 --2- 172.21.15.105:0/3756504838 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f84140668d0 0x7f8414068d90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f842815fde0 tx=0x7f841c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:39.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:39.985+0000 7f840ffff640 1 -- 172.21.15.105:0/3756504838 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f84180ec310 con 0x7f84280b8a20 2023-11-28T18:25:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.137+0000 7f842e1aa640 1 -- 172.21.15.105:0/3756504838 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f84280d1a90 con 0x7f84280b8a20 2023-11-28T18:25:40.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.137+0000 7f840ffff640 1 -- 172.21.15.105:0/3756504838 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f84180f4e60 con 0x7f84280b8a20 2023-11-28T18:25:40.139 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:25:40.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.137+0000 7f842e1aa640 1 -- 172.21.15.105:0/3756504838 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f84140668d0 msgr2=0x7f8414068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:40.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.137+0000 7f842e1aa640 1 --2- 172.21.15.105:0/3756504838 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f84140668d0 0x7f8414068d90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f842815fde0 tx=0x7f841c009040 comp rx=0 tx=0).stop 2023-11-28T18:25:40.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.137+0000 7f842e1aa640 1 -- 172.21.15.105:0/3756504838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84280b8a20 msgr2=0x7f8428155960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:40.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.137+0000 7f842e1aa640 1 --2- 172.21.15.105:0/3756504838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84280b8a20 0x7f8428155960 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f84180ad040 tx=0x7f841801ad50 comp rx=0 tx=0).stop 2023-11-28T18:25:40.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.137+0000 7f842e1aa640 1 -- 172.21.15.105:0/3756504838 shutdown_connections 2023-11-28T18:25:40.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.137+0000 7f842e1aa640 1 --2- 172.21.15.105:0/3756504838 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84280d00b0 0x7f8428157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.137+0000 7f842e1aa640 1 --2- 172.21.15.105:0/3756504838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84280ceb40 0x7f8428155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.137+0000 7f842e1aa640 1 --2- 172.21.15.105:0/3756504838 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f84140668d0 0x7f8414068d90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.137+0000 7f842e1aa640 1 --2- 172.21.15.105:0/3756504838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84280b8a20 0x7f8428155960 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.137+0000 7f842e1aa640 1 -- 172.21.15.105:0/3756504838 >> 172.21.15.105:0/3756504838 conn(0x7f842809c340 msgr2=0x7f842806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:40.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.137+0000 7f842e1aa640 1 -- 172.21.15.105:0/3756504838 shutdown_connections 2023-11-28T18:25:40.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.137+0000 7f842e1aa640 1 -- 172.21.15.105:0/3756504838 wait complete. 2023-11-28T18:25:40.160 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:25:40.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.297+0000 7fe0997f8640 1 Processor -- start 2023-11-28T18:25:40.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.297+0000 7fe0997f8640 1 -- start start 2023-11-28T18:25:40.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.301+0000 7fe0997f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe09415d4f0 0x7fe0940d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:40.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.301+0000 7fe0997f8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0940b25b0 con 0x7fe0940b2d20 2023-11-28T18:25:40.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.301+0000 7fe0997f8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0940cea30 con 0x7fe0940d3c30 2023-11-28T18:25:40.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.301+0000 7fe0997f8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0940b2900 con 0x7fe09415d4f0 2023-11-28T18:25:40.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.301+0000 7fe0937fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe0940b2d20 0x7fe0940b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50544/0 (socket says 172.21.15.105:50544) 2023-11-28T18:25:40.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.301+0000 7fe0937fe640 1 -- 172.21.15.105:0/1840910594 learned_addr learned my addr 172.21.15.105:0/1840910594 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:40.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.301+0000 7fe092ffd640 1 --2- 172.21.15.105:0/1840910594 >> v2:172.21.15.148:3300/0 conn(0x7fe09415d4f0 0x7fe0940d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:40.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.301+0000 7fe0927fc640 1 -- 172.21.15.105:0/1840910594 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4104939867 0 0) 0x7fe0940b25b0 con 0x7fe0940b2d20 2023-11-28T18:25:40.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.301+0000 7fe0927fc640 1 -- 172.21.15.105:0/1840910594 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe080003530 con 0x7fe0940b2d20 2023-11-28T18:25:40.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.301+0000 7fe092ffd640 1 -- 172.21.15.105:0/1840910594 >> v1:172.21.15.148:6789/0 conn(0x7fe0940d3c30 legacy=0x7fe0940d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:40.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.301+0000 7fe092ffd640 1 -- 172.21.15.105:0/1840910594 >> v1:172.21.15.105:6789/0 conn(0x7fe0940b2d20 legacy=0x7fe0940b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:40.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.301+0000 7fe0927fc640 1 -- 172.21.15.105:0/1840910594 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2612728401 0 0) 0x7fe080003530 con 0x7fe0940b2d20 2023-11-28T18:25:40.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.301+0000 7fe092ffd640 1 -- 172.21.15.105:0/1840910594 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0940ced10 con 0x7fe09415d4f0 2023-11-28T18:25:40.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.305+0000 7fe092ffd640 1 --2- 172.21.15.105:0/1840910594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe09415d4f0 0x7fe0940d0e10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe07c0097c0 tx=0x7fe07c01c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a4a52bac1d20816 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:40.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.305+0000 7fe0927fc640 1 -- 172.21.15.105:0/1840910594 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c001a00 con 0x7fe09415d4f0 2023-11-28T18:25:40.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.305+0000 7fe0927fc640 1 -- 172.21.15.105:0/1840910594 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c0a7050 con 0x7fe09415d4f0 2023-11-28T18:25:40.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.305+0000 7fe0927fc640 1 -- 172.21.15.105:0/1840910594 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c0ab600 con 0x7fe09415d4f0 2023-11-28T18:25:40.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.305+0000 7fe0997f8640 1 -- 172.21.15.105:0/1840910594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe09415d4f0 msgr2=0x7fe0940d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:40.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.305+0000 7fe0997f8640 1 --2- 172.21.15.105:0/1840910594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe09415d4f0 0x7fe0940d0e10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe07c0097c0 tx=0x7fe07c01c2e0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.305+0000 7fe0997f8640 1 -- 172.21.15.105:0/1840910594 shutdown_connections 2023-11-28T18:25:40.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.305+0000 7fe0997f8640 1 --2- 172.21.15.105:0/1840910594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe09415d4f0 0x7fe0940d0e10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.305+0000 7fe0997f8640 1 -- 172.21.15.105:0/1840910594 >> 172.21.15.105:0/1840910594 conn(0x7fe0940d3030 msgr2=0x7fe0940d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:40.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.305+0000 7fe0997f8640 1 -- 172.21.15.105:0/1840910594 shutdown_connections 2023-11-28T18:25:40.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.305+0000 7fe0997f8640 1 -- 172.21.15.105:0/1840910594 wait complete. 2023-11-28T18:25:40.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.305+0000 7fe0997f8640 1 Processor -- start 2023-11-28T18:25:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.305+0000 7fe0997f8640 1 -- start start 2023-11-28T18:25:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe0997f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0940b2d20 0x7fe0940c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe0997f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0940d3c30 0x7fe0940c6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe0997f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe09415d4f0 0x7fe0940c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe0997f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0940d0a20 con 0x7fe0940b2d20 2023-11-28T18:25:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe0997f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0940ccaa0 con 0x7fe0940d3c30 2023-11-28T18:25:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe0997f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0940c7090 con 0x7fe09415d4f0 2023-11-28T18:25:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe092ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0940b2d20 0x7fe0940c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe092ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0940b2d20 0x7fe0940c8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48612/0 (socket says 172.21.15.105:48612) 2023-11-28T18:25:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe092ffd640 1 -- 172.21.15.105:0/2210692994 learned_addr learned my addr 172.21.15.105:0/2210692994 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe0937fe640 1 --2- 172.21.15.105:0/2210692994 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe09415d4f0 0x7fe0940c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe092ffd640 1 -- 172.21.15.105:0/2210692994 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe09415d4f0 msgr2=0x7fe0940c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe08a7fc640 1 --2- 172.21.15.105:0/2210692994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0940d3c30 0x7fe0940c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe092ffd640 1 --2- 172.21.15.105:0/2210692994 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe09415d4f0 0x7fe0940c6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe092ffd640 1 -- 172.21.15.105:0/2210692994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0940d3c30 msgr2=0x7fe0940c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe092ffd640 1 --2- 172.21.15.105:0/2210692994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0940d3c30 0x7fe0940c6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe092ffd640 1 -- 172.21.15.105:0/2210692994 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07c009420 con 0x7fe0940b2d20 2023-11-28T18:25:40.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe0937fe640 1 --2- 172.21.15.105:0/2210692994 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe09415d4f0 0x7fe0940c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:40.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe08a7fc640 1 --2- 172.21.15.105:0/2210692994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0940d3c30 0x7fe0940c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:40.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe092ffd640 1 --2- 172.21.15.105:0/2210692994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0940b2d20 0x7fe0940c8f70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe07c0097c0 tx=0x7fe07c019080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:40.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe08bfff640 1 -- 172.21.15.105:0/2210692994 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c006e20 con 0x7fe0940b2d20 2023-11-28T18:25:40.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe08bfff640 1 -- 172.21.15.105:0/2210692994 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c0a4e90 con 0x7fe0940b2d20 2023-11-28T18:25:40.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe08bfff640 1 -- 172.21.15.105:0/2210692994 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c0b4600 con 0x7fe0940b2d20 2023-11-28T18:25:40.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe0997f8640 1 -- 172.21.15.105:0/2210692994 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0940ccd10 con 0x7fe0940b2d20 2023-11-28T18:25:40.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe0997f8640 1 -- 172.21.15.105:0/2210692994 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0940c7770 con 0x7fe0940b2d20 2023-11-28T18:25:40.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.309+0000 7fe0997f8640 1 -- 172.21.15.105:0/2210692994 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe09415db60 con 0x7fe0940b2d20 2023-11-28T18:25:40.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.313+0000 7fe08bfff640 1 -- 172.21.15.105:0/2210692994 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fe07c0b2040 con 0x7fe0940b2d20 2023-11-28T18:25:40.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.313+0000 7fe08bfff640 1 --2- 172.21.15.105:0/2210692994 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe0740668d0 0x7fe074068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:40.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.313+0000 7fe08bfff640 1 -- 172.21.15.105:0/2210692994 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe07c11e4f0 con 0x7fe0940b2d20 2023-11-28T18:25:40.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.313+0000 7fe08a7fc640 1 --2- 172.21.15.105:0/2210692994 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe0740668d0 0x7fe074068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:40.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.313+0000 7fe08a7fc640 1 --2- 172.21.15.105:0/2210692994 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe0740668d0 0x7fe074068d90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe09414bee0 tx=0x7fe084009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:40.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.317+0000 7fe08bfff640 1 -- 172.21.15.105:0/2210692994 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe07c0eb4b0 con 0x7fe0940b2d20 2023-11-28T18:25:40.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.465+0000 7fe0997f8640 1 -- 172.21.15.105:0/2210692994 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe094067570 con 0x7fe0940b2d20 2023-11-28T18:25:40.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.469+0000 7fe08bfff640 1 -- 172.21.15.105:0/2210692994 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe07c0eead0 con 0x7fe0940b2d20 2023-11-28T18:25:40.469 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:25:40.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.469+0000 7fe0997f8640 1 -- 172.21.15.105:0/2210692994 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe0740668d0 msgr2=0x7fe074068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:40.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.469+0000 7fe0997f8640 1 --2- 172.21.15.105:0/2210692994 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe0740668d0 0x7fe074068d90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe09414bee0 tx=0x7fe084009040 comp rx=0 tx=0).stop 2023-11-28T18:25:40.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.469+0000 7fe0997f8640 1 -- 172.21.15.105:0/2210692994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0940b2d20 msgr2=0x7fe0940c8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:40.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.469+0000 7fe0997f8640 1 --2- 172.21.15.105:0/2210692994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0940b2d20 0x7fe0940c8f70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe07c0097c0 tx=0x7fe07c019080 comp rx=0 tx=0).stop 2023-11-28T18:25:40.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.469+0000 7fe0997f8640 1 -- 172.21.15.105:0/2210692994 shutdown_connections 2023-11-28T18:25:40.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.469+0000 7fe0997f8640 1 --2- 172.21.15.105:0/2210692994 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe09415d4f0 0x7fe0940c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.469+0000 7fe0997f8640 1 --2- 172.21.15.105:0/2210692994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0940d3c30 0x7fe0940c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.469+0000 7fe0997f8640 1 --2- 172.21.15.105:0/2210692994 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe0740668d0 0x7fe074068d90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.469+0000 7fe0997f8640 1 --2- 172.21.15.105:0/2210692994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0940b2d20 0x7fe0940c8f70 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.469+0000 7fe0997f8640 1 -- 172.21.15.105:0/2210692994 >> 172.21.15.105:0/2210692994 conn(0x7fe0940d3030 msgr2=0x7fe0941551a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:40.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.469+0000 7fe0997f8640 1 -- 172.21.15.105:0/2210692994 shutdown_connections 2023-11-28T18:25:40.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.469+0000 7fe0997f8640 1 -- 172.21.15.105:0/2210692994 wait complete. 2023-11-28T18:25:40.490 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:25:40.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.629+0000 7fab194e3640 1 Processor -- start 2023-11-28T18:25:40.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.629+0000 7fab194e3640 1 -- start start 2023-11-28T18:25:40.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.629+0000 7fab194e3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab14159f40 0x7fab140caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:40.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.629+0000 7fab194e3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fab140cc760 0x7fab140ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:40.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.629+0000 7fab194e3640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab1406bf50 con 0x7fab140ad680 2023-11-28T18:25:40.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.629+0000 7fab194e3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab140ccbc0 con 0x7fab140cc760 2023-11-28T18:25:40.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.629+0000 7fab194e3640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1415a320 con 0x7fab14159f40 2023-11-28T18:25:40.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.629+0000 7fab0a7fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fab140cc760 0x7fab140ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:40.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.629+0000 7fab0a7fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fab140cc760 0x7fab140ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48618/0 (socket says 172.21.15.105:48618) 2023-11-28T18:25:40.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.629+0000 7fab0a7fc640 1 -- 172.21.15.105:0/2110123270 learned_addr learned my addr 172.21.15.105:0/2110123270 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:40.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.629+0000 7fab137fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fab140ad680 0x7fab1415da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50550/0 (socket says 172.21.15.105:50550) 2023-11-28T18:25:40.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.629+0000 7fab12ffd640 1 --2- 172.21.15.105:0/2110123270 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab14159f40 0x7fab140caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:40.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.629+0000 7fab0a7fc640 1 -- 172.21.15.105:0/2110123270 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab14159f40 msgr2=0x7fab140caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:40.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.629+0000 7fab0a7fc640 1 --2- 172.21.15.105:0/2110123270 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab14159f40 0x7fab140caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.629+0000 7fab0a7fc640 1 -- 172.21.15.105:0/2110123270 >> v1:172.21.15.105:6789/0 conn(0x7fab140ad680 legacy=0x7fab1415da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:40.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.629+0000 7fab0a7fc640 1 -- 172.21.15.105:0/2110123270 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab14071380 con 0x7fab140cc760 2023-11-28T18:25:40.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.629+0000 7fab12ffd640 1 --2- 172.21.15.105:0/2110123270 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab14159f40 0x7fab140caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:40.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.633+0000 7fab0a7fc640 1 --2- 172.21.15.105:0/2110123270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab140cc760 0x7fab140ad140 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fab04009870 tx=0x7fab0401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b6a9e24dbf698d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:40.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.633+0000 7fab127fc640 1 -- 172.21.15.105:0/2110123270 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab040ab680 con 0x7fab140cc760 2023-11-28T18:25:40.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.633+0000 7fab127fc640 1 -- 172.21.15.105:0/2110123270 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab040ab7e0 con 0x7fab140cc760 2023-11-28T18:25:40.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.633+0000 7fab127fc640 1 -- 172.21.15.105:0/2110123270 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab040abb70 con 0x7fab140cc760 2023-11-28T18:25:40.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.633+0000 7fab194e3640 1 -- 172.21.15.105:0/2110123270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab140cc760 msgr2=0x7fab140ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:40.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.633+0000 7fab194e3640 1 --2- 172.21.15.105:0/2110123270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab140cc760 0x7fab140ad140 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fab04009870 tx=0x7fab0401c480 comp rx=0 tx=0).stop 2023-11-28T18:25:40.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.633+0000 7fab194e3640 1 -- 172.21.15.105:0/2110123270 shutdown_connections 2023-11-28T18:25:40.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.633+0000 7fab194e3640 1 --2- 172.21.15.105:0/2110123270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab140cc760 0x7fab140ad140 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.633+0000 7fab194e3640 1 --2- 172.21.15.105:0/2110123270 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab14159f40 0x7fab140caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.633+0000 7fab194e3640 1 -- 172.21.15.105:0/2110123270 >> 172.21.15.105:0/2110123270 conn(0x7fab140ceb90 msgr2=0x7fab140cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:40.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.633+0000 7fab194e3640 1 -- 172.21.15.105:0/2110123270 shutdown_connections 2023-11-28T18:25:40.640 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.637+0000 7fab194e3640 1 -- 172.21.15.105:0/2110123270 wait complete. 2023-11-28T18:25:40.643 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.637+0000 7fab194e3640 1 Processor -- start 2023-11-28T18:25:40.643 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.637+0000 7fab194e3640 1 -- start start 2023-11-28T18:25:40.643 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.637+0000 7fab194e3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab140ad680 0x7fab141507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:40.643 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.637+0000 7fab194e3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab140cc760 0x7fab1415cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:40.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.637+0000 7fab194e3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab14159f40 0x7fab14158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:40.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.637+0000 7fab194e3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab141534f0 con 0x7fab140cc760 2023-11-28T18:25:40.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.637+0000 7fab194e3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1415a6c0 con 0x7fab140ad680 2023-11-28T18:25:40.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.637+0000 7fab194e3640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1415a9a0 con 0x7fab14159f40 2023-11-28T18:25:40.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.637+0000 7fab0a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab140cc760 0x7fab1415cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:40.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.637+0000 7fab0a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab140cc760 0x7fab1415cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48634/0 (socket says 172.21.15.105:48634) 2023-11-28T18:25:40.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.637+0000 7fab0a7fc640 1 -- 172.21.15.105:0/3792359826 learned_addr learned my addr 172.21.15.105:0/3792359826 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:40.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.637+0000 7fab12ffd640 1 --2- 172.21.15.105:0/3792359826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab140ad680 0x7fab141507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:40.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.637+0000 7fab137fe640 1 --2- 172.21.15.105:0/3792359826 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab14159f40 0x7fab14158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:40.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.637+0000 7fab0a7fc640 1 -- 172.21.15.105:0/3792359826 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab14159f40 msgr2=0x7fab14158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:40.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.637+0000 7fab0a7fc640 1 --2- 172.21.15.105:0/3792359826 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab14159f40 0x7fab14158a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.637+0000 7fab0a7fc640 1 -- 172.21.15.105:0/3792359826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab140ad680 msgr2=0x7fab141507c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:40.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.637+0000 7fab0a7fc640 1 --2- 172.21.15.105:0/3792359826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab140ad680 0x7fab141507c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.637+0000 7fab0a7fc640 1 -- 172.21.15.105:0/3792359826 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab040094d0 con 0x7fab140cc760 2023-11-28T18:25:40.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.637+0000 7fab0a7fc640 1 --2- 172.21.15.105:0/3792359826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab140cc760 0x7fab1415cb20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fab0401a0b0 tx=0x7fab0401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:40.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.641+0000 7fab0bfff640 1 -- 172.21.15.105:0/3792359826 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab040b4800 con 0x7fab140cc760 2023-11-28T18:25:40.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.641+0000 7fab0bfff640 1 -- 172.21.15.105:0/3792359826 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab040b4980 con 0x7fab140cc760 2023-11-28T18:25:40.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.641+0000 7fab0bfff640 1 -- 172.21.15.105:0/3792359826 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab040b4d30 con 0x7fab140cc760 2023-11-28T18:25:40.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.641+0000 7fab194e3640 1 -- 172.21.15.105:0/3792359826 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1415ac80 con 0x7fab140cc760 2023-11-28T18:25:40.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.641+0000 7fab194e3640 1 -- 172.21.15.105:0/3792359826 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1415b240 con 0x7fab140cc760 2023-11-28T18:25:40.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.641+0000 7fab194e3640 1 -- 172.21.15.105:0/3792359826 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1415b430 con 0x7fab140cc760 2023-11-28T18:25:40.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.641+0000 7fab0bfff640 1 -- 172.21.15.105:0/3792359826 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fab040b3380 con 0x7fab140cc760 2023-11-28T18:25:40.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.641+0000 7fab0bfff640 1 --2- 172.21.15.105:0/3792359826 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faaf40668d0 0x7faaf4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:40.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.641+0000 7fab0bfff640 1 -- 172.21.15.105:0/3792359826 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fab0411efe0 con 0x7fab140cc760 2023-11-28T18:25:40.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.641+0000 7fab12ffd640 1 --2- 172.21.15.105:0/3792359826 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faaf40668d0 0x7faaf4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:40.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.641+0000 7fab12ffd640 1 --2- 172.21.15.105:0/3792359826 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faaf40668d0 0x7faaf4068d90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7faafc00ba10 tx=0x7faafc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:40.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.645+0000 7fab0bfff640 1 -- 172.21.15.105:0/3792359826 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fab040ec1f0 con 0x7fab140cc760 2023-11-28T18:25:40.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.797+0000 7fab194e3640 1 -- 172.21.15.105:0/3792359826 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fab14067570 con 0x7fab140cc760 2023-11-28T18:25:40.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.797+0000 7fab0bfff640 1 -- 172.21.15.105:0/3792359826 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7fab040ab640 con 0x7fab140cc760 2023-11-28T18:25:40.798 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:25:40.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.797+0000 7fab194e3640 1 -- 172.21.15.105:0/3792359826 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faaf40668d0 msgr2=0x7faaf4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:40.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.797+0000 7fab194e3640 1 --2- 172.21.15.105:0/3792359826 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faaf40668d0 0x7faaf4068d90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7faafc00ba10 tx=0x7faafc008040 comp rx=0 tx=0).stop 2023-11-28T18:25:40.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.797+0000 7fab194e3640 1 -- 172.21.15.105:0/3792359826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab140cc760 msgr2=0x7fab1415cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:40.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.797+0000 7fab194e3640 1 --2- 172.21.15.105:0/3792359826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab140cc760 0x7fab1415cb20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fab0401a0b0 tx=0x7fab0401acc0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.797+0000 7fab194e3640 1 -- 172.21.15.105:0/3792359826 shutdown_connections 2023-11-28T18:25:40.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.797+0000 7fab194e3640 1 --2- 172.21.15.105:0/3792359826 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab14159f40 0x7fab14158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.797+0000 7fab194e3640 1 --2- 172.21.15.105:0/3792359826 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faaf40668d0 0x7faaf4068d90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.797+0000 7fab194e3640 1 --2- 172.21.15.105:0/3792359826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab140cc760 0x7fab1415cb20 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.797+0000 7fab194e3640 1 --2- 172.21.15.105:0/3792359826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab140ad680 0x7fab141507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.797+0000 7fab194e3640 1 -- 172.21.15.105:0/3792359826 >> 172.21.15.105:0/3792359826 conn(0x7fab140ceb90 msgr2=0x7fab14071aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:40.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.797+0000 7fab194e3640 1 -- 172.21.15.105:0/3792359826 shutdown_connections 2023-11-28T18:25:40.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.797+0000 7fab194e3640 1 -- 172.21.15.105:0/3792359826 wait complete. 2023-11-28T18:25:40.819 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:25:40.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.953+0000 7f9b11ec9640 1 Processor -- start 2023-11-28T18:25:40.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.953+0000 7f9b11ec9640 1 -- start start 2023-11-28T18:25:40.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.953+0000 7f9b11ec9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9b0c15cfc0 0x7f9b0c06dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:40.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.953+0000 7f9b11ec9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9b0c0c9720 0x7f9b0c070660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:40.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.953+0000 7f9b11ec9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b0c0cb010 con 0x7f9b0c06c620 2023-11-28T18:25:40.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.953+0000 7f9b11ec9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b0c15e620 con 0x7f9b0c0c9720 2023-11-28T18:25:40.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.953+0000 7f9b11ec9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c0c9c20 con 0x7f9b0c15cfc0 2023-11-28T18:25:40.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.953+0000 7f9b0bfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9b0c06c620 0x7f9b0c0cba90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50562/0 (socket says 172.21.15.105:50562) 2023-11-28T18:25:40.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.953+0000 7f9b0bfff640 1 -- 172.21.15.105:0/2913102429 learned_addr learned my addr 172.21.15.105:0/2913102429 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:40.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.957+0000 7f9b0b7fe640 1 --2- 172.21.15.105:0/2913102429 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9b0c15cfc0 0x7f9b0c06dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:40.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.957+0000 7f9b0affd640 1 --2- 172.21.15.105:0/2913102429 >> v2:172.21.15.148:3300/0 conn(0x7f9b0c0c9720 0x7f9b0c070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:40.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.957+0000 7f9b0a7fc640 1 -- 172.21.15.105:0/2913102429 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700223041 0 0) 0x7f9b0c0cb010 con 0x7f9b0c06c620 2023-11-28T18:25:40.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.957+0000 7f9b0a7fc640 1 -- 172.21.15.105:0/2913102429 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9af4003530 con 0x7f9b0c06c620 2023-11-28T18:25:40.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.957+0000 7f9b0b7fe640 1 -- 172.21.15.105:0/2913102429 >> v2:172.21.15.148:3300/0 conn(0x7f9b0c0c9720 msgr2=0x7f9b0c070660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:40.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.957+0000 7f9b0b7fe640 1 --2- 172.21.15.105:0/2913102429 >> v2:172.21.15.148:3300/0 conn(0x7f9b0c0c9720 0x7f9b0c070660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.957+0000 7f9b0b7fe640 1 -- 172.21.15.105:0/2913102429 >> v1:172.21.15.105:6789/0 conn(0x7f9b0c06c620 legacy=0x7f9b0c0cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:40.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.957+0000 7f9b0b7fe640 1 -- 172.21.15.105:0/2913102429 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0c15e340 con 0x7f9b0c15cfc0 2023-11-28T18:25:40.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.957+0000 7f9b0b7fe640 1 --2- 172.21.15.105:0/2913102429 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9b0c15cfc0 0x7f9b0c06dce0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0097c0 tx=0x7f9afc01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cdbb03a42b04642 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:40.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.957+0000 7f9b0a7fc640 1 -- 172.21.15.105:0/2913102429 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc0ab650 con 0x7f9b0c15cfc0 2023-11-28T18:25:40.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.961+0000 7f9b0a7fc640 1 -- 172.21.15.105:0/2913102429 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9afc0ab7b0 con 0x7f9b0c15cfc0 2023-11-28T18:25:40.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.961+0000 7f9b0a7fc640 1 -- 172.21.15.105:0/2913102429 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc0abb40 con 0x7f9b0c15cfc0 2023-11-28T18:25:40.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.961+0000 7f9b11ec9640 1 -- 172.21.15.105:0/2913102429 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9b0c15cfc0 msgr2=0x7f9b0c06dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:40.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.961+0000 7f9b11ec9640 1 --2- 172.21.15.105:0/2913102429 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9b0c15cfc0 0x7f9b0c06dce0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0097c0 tx=0x7f9afc01c450 comp rx=0 tx=0).stop 2023-11-28T18:25:40.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.961+0000 7f9b11ec9640 1 -- 172.21.15.105:0/2913102429 shutdown_connections 2023-11-28T18:25:40.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.961+0000 7f9b11ec9640 1 --2- 172.21.15.105:0/2913102429 >> v2:172.21.15.148:3300/0 conn(0x7f9b0c0c9720 0x7f9b0c070660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.961+0000 7f9b11ec9640 1 --2- 172.21.15.105:0/2913102429 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9b0c15cfc0 0x7f9b0c06dce0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.961+0000 7f9b11ec9640 1 -- 172.21.15.105:0/2913102429 >> 172.21.15.105:0/2913102429 conn(0x7f9b0c0b0c60 msgr2=0x7f9b0c0b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:40.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.961+0000 7f9b11ec9640 1 -- 172.21.15.105:0/2913102429 shutdown_connections 2023-11-28T18:25:40.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.961+0000 7f9b11ec9640 1 -- 172.21.15.105:0/2913102429 wait complete. 2023-11-28T18:25:40.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.961+0000 7f9b11ec9640 1 Processor -- start 2023-11-28T18:25:40.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.961+0000 7f9b11ec9640 1 -- start start 2023-11-28T18:25:40.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.961+0000 7f9b11ec9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b0c06c620 0x7f9b0c09baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:40.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.961+0000 7f9b11ec9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b0c0c9720 0x7f9b0c09bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:40.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.961+0000 7f9b11ec9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9b0c15cfc0 0x7f9b0c1f94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:40.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.961+0000 7f9b11ec9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c0b2d10 con 0x7f9b0c0c9720 2023-11-28T18:25:40.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.961+0000 7f9b11ec9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c09b320 con 0x7f9b0c06c620 2023-11-28T18:25:40.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.961+0000 7f9b11ec9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c0b26c0 con 0x7f9b0c15cfc0 2023-11-28T18:25:40.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.961+0000 7f9b0affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b0c0c9720 0x7f9b0c09bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:40.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.961+0000 7f9b0affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b0c0c9720 0x7f9b0c09bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48638/0 (socket says 172.21.15.105:48638) 2023-11-28T18:25:40.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.961+0000 7f9b0affd640 1 -- 172.21.15.105:0/300691439 learned_addr learned my addr 172.21.15.105:0/300691439 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:40.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.965+0000 7f9b0b7fe640 1 --2- 172.21.15.105:0/300691439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b0c06c620 0x7f9b0c09baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:40.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.965+0000 7f9b0bfff640 1 --2- 172.21.15.105:0/300691439 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9b0c15cfc0 0x7f9b0c1f94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:40.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.965+0000 7f9b0affd640 1 -- 172.21.15.105:0/300691439 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9b0c15cfc0 msgr2=0x7f9b0c1f94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:40.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.965+0000 7f9b0affd640 1 --2- 172.21.15.105:0/300691439 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9b0c15cfc0 0x7f9b0c1f94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.965+0000 7f9b0affd640 1 -- 172.21.15.105:0/300691439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b0c06c620 msgr2=0x7f9b0c09baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:40.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.965+0000 7f9b0affd640 1 --2- 172.21.15.105:0/300691439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b0c06c620 0x7f9b0c09baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:40.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.965+0000 7f9b0affd640 1 -- 172.21.15.105:0/300691439 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc009420 con 0x7f9b0c0c9720 2023-11-28T18:25:40.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.965+0000 7f9b0b7fe640 1 --2- 172.21.15.105:0/300691439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b0c06c620 0x7f9b0c09baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:40.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.965+0000 7f9b0affd640 1 --2- 172.21.15.105:0/300691439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b0c0c9720 0x7f9b0c09bfe0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000e930 tx=0x7f9b0000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:40.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.965+0000 7f9af3fff640 1 -- 172.21.15.105:0/300691439 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b00009690 con 0x7f9b0c0c9720 2023-11-28T18:25:40.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.965+0000 7f9af3fff640 1 -- 172.21.15.105:0/300691439 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0000ca00 con 0x7f9b0c0c9720 2023-11-28T18:25:40.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.965+0000 7f9af3fff640 1 -- 172.21.15.105:0/300691439 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0000cd70 con 0x7f9b0c0c9720 2023-11-28T18:25:40.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.965+0000 7f9b11ec9640 1 -- 172.21.15.105:0/300691439 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b0c06cfc0 con 0x7f9b0c0c9720 2023-11-28T18:25:40.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.965+0000 7f9b11ec9640 1 -- 172.21.15.105:0/300691439 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b0c0b22e0 con 0x7f9b0c0c9720 2023-11-28T18:25:40.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.965+0000 7f9b11ec9640 1 -- 172.21.15.105:0/300691439 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b0c067570 con 0x7f9b0c0c9720 2023-11-28T18:25:40.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.965+0000 7f9af3fff640 1 -- 172.21.15.105:0/300691439 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f9b00013470 con 0x7f9b0c0c9720 2023-11-28T18:25:40.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.969+0000 7f9af3fff640 1 --2- 172.21.15.105:0/300691439 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9af80668d0 0x7f9af8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:40.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.969+0000 7f9af3fff640 1 -- 172.21.15.105:0/300691439 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9b00084840 con 0x7f9b0c0c9720 2023-11-28T18:25:40.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.969+0000 7f9b0b7fe640 1 --2- 172.21.15.105:0/300691439 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9af80668d0 0x7f9af8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:40.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.969+0000 7f9b0b7fe640 1 --2- 172.21.15.105:0/300691439 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9af80668d0 0x7f9af8068d90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0af000 tx=0x7f9afc0b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:40.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:40.973+0000 7f9af3fff640 1 -- 172.21.15.105:0/300691439 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9b000519c0 con 0x7f9b0c0c9720 2023-11-28T18:25:41.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.153+0000 7f9b11ec9640 1 -- 172.21.15.105:0/300691439 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b0c06dac0 con 0x7f9b0c0c9720 2023-11-28T18:25:41.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.153+0000 7f9af3fff640 1 -- 172.21.15.105:0/300691439 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f9b00017090 con 0x7f9b0c0c9720 2023-11-28T18:25:41.154 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:41.154 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/47307750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":47307750},{"type":"v1","addr":"172.21.15.148:6834","nonce":47307750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:24:55.936428+0000","modified":"2023-11-28T18:25:33.692231+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5469,"mds_1":5463,"mds_2":5392},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5392":{"gid":5392,"name":"d","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6835/2878687703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2878687703},{"type":"v1","addr":"172.21.15.148:6835","nonce":2878687703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5463":{"gid":5463,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.105:6836/681913204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":681913204},{"type":"v1","addr":"172.21.15.105:6836","nonce":681913204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5469":{"gid":5469,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/1876408690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":1876408690},{"type":"v1","addr":"172.21.15.105:6837","nonce":1876408690}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T18:25:41.154 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 25 2023-11-28T18:25:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.153+0000 7f9b11ec9640 1 -- 172.21.15.105:0/300691439 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9af80668d0 msgr2=0x7f9af8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.153+0000 7f9b11ec9640 1 --2- 172.21.15.105:0/300691439 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9af80668d0 0x7f9af8068d90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0af000 tx=0x7f9afc0b6040 comp rx=0 tx=0).stop 2023-11-28T18:25:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.153+0000 7f9b11ec9640 1 -- 172.21.15.105:0/300691439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b0c0c9720 msgr2=0x7f9b0c09bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.153+0000 7f9b11ec9640 1 --2- 172.21.15.105:0/300691439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b0c0c9720 0x7f9b0c09bfe0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000e930 tx=0x7f9b0000edf0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.153+0000 7f9b11ec9640 1 -- 172.21.15.105:0/300691439 shutdown_connections 2023-11-28T18:25:41.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.153+0000 7f9b11ec9640 1 --2- 172.21.15.105:0/300691439 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9b0c15cfc0 0x7f9b0c1f94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.153+0000 7f9b11ec9640 1 --2- 172.21.15.105:0/300691439 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9af80668d0 0x7f9af8068d90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.153+0000 7f9b11ec9640 1 --2- 172.21.15.105:0/300691439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b0c0c9720 0x7f9b0c09bfe0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.153+0000 7f9b11ec9640 1 --2- 172.21.15.105:0/300691439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b0c06c620 0x7f9b0c09baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.153+0000 7f9b11ec9640 1 -- 172.21.15.105:0/300691439 >> 172.21.15.105:0/300691439 conn(0x7f9b0c0b0c60 msgr2=0x7f9b0c0ce1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.153+0000 7f9b11ec9640 1 -- 172.21.15.105:0/300691439 shutdown_connections 2023-11-28T18:25:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.153+0000 7f9b11ec9640 1 -- 172.21.15.105:0/300691439 wait complete. 2023-11-28T18:25:41.176 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:25:41.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.313+0000 7faa43baa640 1 Processor -- start 2023-11-28T18:25:41.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.313+0000 7faa43baa640 1 -- start start 2023-11-28T18:25:41.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.313+0000 7faa43baa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7faa3c15d4f0 0x7faa3c0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:41.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.313+0000 7faa43baa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa3c0d3c30 0x7faa3c0d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:41.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.313+0000 7faa43baa640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa3c0b25b0 con 0x7faa3c0d1890 2023-11-28T18:25:41.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.313+0000 7faa43baa640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa3c0b2900 con 0x7faa3c15d4f0 2023-11-28T18:25:41.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.313+0000 7faa43baa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c0cea30 con 0x7faa3c0d3c30 2023-11-28T18:25:41.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.313+0000 7faa41946640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7faa3c15d4f0 0x7faa3c0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:41.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.313+0000 7faa42147640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7faa3c0d1890 0x7faa3c0cfa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55810/0 (socket says 172.21.15.105:55810) 2023-11-28T18:25:41.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.313+0000 7faa42147640 1 -- 172.21.15.105:0/3550573627 learned_addr learned my addr 172.21.15.105:0/3550573627 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:41.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.313+0000 7faa41145640 1 --2- 172.21.15.105:0/3550573627 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa3c0d3c30 0x7faa3c0d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:41.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.313+0000 7faa41946640 1 -- 172.21.15.105:0/3550573627 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa3c0d3c30 msgr2=0x7faa3c0d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:41.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.313+0000 7faa41946640 1 --2- 172.21.15.105:0/3550573627 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa3c0d3c30 0x7faa3c0d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.313+0000 7faa41946640 1 -- 172.21.15.105:0/3550573627 >> v1:172.21.15.105:6789/0 conn(0x7faa3c0d1890 legacy=0x7faa3c0cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:41.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.313+0000 7faa41946640 1 -- 172.21.15.105:0/3550573627 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c0ced10 con 0x7faa3c15d4f0 2023-11-28T18:25:41.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.313+0000 7faa40944640 1 -- 172.21.15.105:0/3550573627 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848019590 0 0) 0x7faa3c0b25b0 con 0x7faa3c0d1890 2023-11-28T18:25:41.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.313+0000 7faa41145640 1 --2- 172.21.15.105:0/3550573627 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa3c0d3c30 0x7faa3c0d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:41.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.317+0000 7faa41946640 1 --2- 172.21.15.105:0/3550573627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa3c15d4f0 0x7faa3c0d0e10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7faa38009d90 tx=0x7faa3801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a15924fb41cae8a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:41.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.317+0000 7faa40944640 1 -- 172.21.15.105:0/3550573627 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa380a5970 con 0x7faa3c15d4f0 2023-11-28T18:25:41.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.317+0000 7faa40944640 1 -- 172.21.15.105:0/3550573627 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa380a5ad0 con 0x7faa3c15d4f0 2023-11-28T18:25:41.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.317+0000 7faa40944640 1 -- 172.21.15.105:0/3550573627 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa380a5e60 con 0x7faa3c15d4f0 2023-11-28T18:25:41.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.317+0000 7faa43baa640 1 -- 172.21.15.105:0/3550573627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa3c15d4f0 msgr2=0x7faa3c0d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:41.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.317+0000 7faa43baa640 1 --2- 172.21.15.105:0/3550573627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa3c15d4f0 0x7faa3c0d0e10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7faa38009d90 tx=0x7faa3801c890 comp rx=0 tx=0).stop 2023-11-28T18:25:41.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.317+0000 7faa43baa640 1 -- 172.21.15.105:0/3550573627 shutdown_connections 2023-11-28T18:25:41.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.317+0000 7faa43baa640 1 --2- 172.21.15.105:0/3550573627 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa3c0d3c30 0x7faa3c0d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.317+0000 7faa43baa640 1 --2- 172.21.15.105:0/3550573627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa3c15d4f0 0x7faa3c0d0e10 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.317+0000 7faa43baa640 1 -- 172.21.15.105:0/3550573627 >> 172.21.15.105:0/3550573627 conn(0x7faa3c0d3030 msgr2=0x7faa3c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:41.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.317+0000 7faa43baa640 1 -- 172.21.15.105:0/3550573627 shutdown_connections 2023-11-28T18:25:41.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.317+0000 7faa43baa640 1 -- 172.21.15.105:0/3550573627 wait complete. 2023-11-28T18:25:41.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa43baa640 1 Processor -- start 2023-11-28T18:25:41.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa43baa640 1 -- start start 2023-11-28T18:25:41.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa43baa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa3c0d1890 0x7faa3c0b8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:41.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa43baa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa3c0d3c30 0x7faa3c0cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:41.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa43baa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa3c15d4f0 0x7faa3c1f9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:41.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa43baa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c0b6580 con 0x7faa3c15d4f0 2023-11-28T18:25:41.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa43baa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c0b8970 con 0x7faa3c0d3c30 2023-11-28T18:25:41.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa43baa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c0b5c50 con 0x7faa3c0d1890 2023-11-28T18:25:41.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa42147640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa3c15d4f0 0x7faa3c1f9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:41.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa42147640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa3c15d4f0 0x7faa3c1f9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48224/0 (socket says 172.21.15.105:48224) 2023-11-28T18:25:41.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa42147640 1 -- 172.21.15.105:0/2303053583 learned_addr learned my addr 172.21.15.105:0/2303053583 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:41.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa41145640 1 --2- 172.21.15.105:0/2303053583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa3c0d3c30 0x7faa3c0cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:41.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa41946640 1 --2- 172.21.15.105:0/2303053583 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa3c0d1890 0x7faa3c0b8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:41.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa42147640 1 -- 172.21.15.105:0/2303053583 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa3c0d1890 msgr2=0x7faa3c0b8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:41.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa42147640 1 --2- 172.21.15.105:0/2303053583 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa3c0d1890 0x7faa3c0b8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa42147640 1 -- 172.21.15.105:0/2303053583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa3c0d3c30 msgr2=0x7faa3c0cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:41.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa42147640 1 --2- 172.21.15.105:0/2303053583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa3c0d3c30 0x7faa3c0cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa42147640 1 -- 172.21.15.105:0/2303053583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa38009a70 con 0x7faa3c15d4f0 2023-11-28T18:25:41.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa41946640 1 --2- 172.21.15.105:0/2303053583 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa3c0d1890 0x7faa3c0b8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:41.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa42147640 1 --2- 172.21.15.105:0/2303053583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa3c15d4f0 0x7faa3c1f9d60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7faa3400c3c0 tx=0x7faa3400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:41.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa327fc640 1 -- 172.21.15.105:0/2303053583 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa340074e0 con 0x7faa3c15d4f0 2023-11-28T18:25:41.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa327fc640 1 -- 172.21.15.105:0/2303053583 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa34007640 con 0x7faa3c15d4f0 2023-11-28T18:25:41.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa327fc640 1 -- 172.21.15.105:0/2303053583 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa34004a30 con 0x7faa3c15d4f0 2023-11-28T18:25:41.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa43baa640 1 -- 172.21.15.105:0/2303053583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa3c0b5f30 con 0x7faa3c15d4f0 2023-11-28T18:25:41.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.321+0000 7faa43baa640 1 -- 172.21.15.105:0/2303053583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa3c0b8680 con 0x7faa3c15d4f0 2023-11-28T18:25:41.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.325+0000 7faa43baa640 1 -- 172.21.15.105:0/2303053583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa3c0b66f0 con 0x7faa3c15d4f0 2023-11-28T18:25:41.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.325+0000 7faa327fc640 1 -- 172.21.15.105:0/2303053583 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7faa34004b90 con 0x7faa3c15d4f0 2023-11-28T18:25:41.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.325+0000 7faa327fc640 1 --2- 172.21.15.105:0/2303053583 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faa200668d0 0x7faa20068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:41.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.325+0000 7faa327fc640 1 -- 172.21.15.105:0/2303053583 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faa34084340 con 0x7faa3c15d4f0 2023-11-28T18:25:41.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.325+0000 7faa41946640 1 --2- 172.21.15.105:0/2303053583 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faa200668d0 0x7faa20068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:41.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.325+0000 7faa41946640 1 --2- 172.21.15.105:0/2303053583 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faa200668d0 0x7faa20068d90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faa380af000 tx=0x7faa380a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:41.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.329+0000 7faa327fc640 1 -- 172.21.15.105:0/2303053583 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faa34051380 con 0x7faa3c15d4f0 2023-11-28T18:25:41.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.481+0000 7faa43baa640 1 -- 172.21.15.105:0/2303053583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa3c067570 con 0x7faa3c15d4f0 2023-11-28T18:25:41.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.485+0000 7faa327fc640 1 -- 172.21.15.105:0/2303053583 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16827 (secure 0 0 0) 0x7faa340549a0 con 0x7faa3c15d4f0 2023-11-28T18:25:41.486 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:41.486 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":40,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:25:15.107712+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:24:53.582892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T18:24:54.921978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.39","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:25:41.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.485+0000 7faa43baa640 1 -- 172.21.15.105:0/2303053583 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faa200668d0 msgr2=0x7faa20068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:41.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.485+0000 7faa43baa640 1 --2- 172.21.15.105:0/2303053583 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faa200668d0 0x7faa20068d90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faa380af000 tx=0x7faa380a7070 comp rx=0 tx=0).stop 2023-11-28T18:25:41.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.485+0000 7faa43baa640 1 -- 172.21.15.105:0/2303053583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa3c15d4f0 msgr2=0x7faa3c1f9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:41.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.485+0000 7faa43baa640 1 --2- 172.21.15.105:0/2303053583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa3c15d4f0 0x7faa3c1f9d60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7faa3400c3c0 tx=0x7faa3400c880 comp rx=0 tx=0).stop 2023-11-28T18:25:41.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.485+0000 7faa43baa640 1 -- 172.21.15.105:0/2303053583 shutdown_connections 2023-11-28T18:25:41.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.485+0000 7faa43baa640 1 --2- 172.21.15.105:0/2303053583 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faa200668d0 0x7faa20068d90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.485+0000 7faa43baa640 1 --2- 172.21.15.105:0/2303053583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa3c15d4f0 0x7faa3c1f9d60 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.485+0000 7faa43baa640 1 --2- 172.21.15.105:0/2303053583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa3c0d3c30 0x7faa3c0cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.485+0000 7faa43baa640 1 --2- 172.21.15.105:0/2303053583 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa3c0d1890 0x7faa3c0b8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.485+0000 7faa43baa640 1 -- 172.21.15.105:0/2303053583 >> 172.21.15.105:0/2303053583 conn(0x7faa3c0d3030 msgr2=0x7faa3c156520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:41.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.485+0000 7faa43baa640 1 -- 172.21.15.105:0/2303053583 shutdown_connections 2023-11-28T18:25:41.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.485+0000 7faa43baa640 1 -- 172.21.15.105:0/2303053583 wait complete. 2023-11-28T18:25:41.508 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T18:25:41.508 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:25:41.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.645+0000 7f83e14ea640 1 Processor -- start 2023-11-28T18:25:41.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.645+0000 7f83e14ea640 1 -- start start 2023-11-28T18:25:41.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.645+0000 7f83e14ea640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f83dc15d4f0 0x7f83dc0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:41.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.645+0000 7f83e14ea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83dc0b2d20 0x7f83dc0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:41.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.645+0000 7f83e14ea640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83dc0b25b0 con 0x7f83dc0d3c30 2023-11-28T18:25:41.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.645+0000 7f83e14ea640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83dc0b2900 con 0x7f83dc15d4f0 2023-11-28T18:25:41.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.645+0000 7f83e14ea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc0d0a20 con 0x7f83dc0b2d20 2023-11-28T18:25:41.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.645+0000 7f83daffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f83dc15d4f0 0x7f83dc0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:41.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.645+0000 7f83daffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f83dc15d4f0 0x7f83dc0d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48238/0 (socket says 172.21.15.105:48238) 2023-11-28T18:25:41.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.645+0000 7f83daffd640 1 -- 172.21.15.105:0/3239900645 learned_addr learned my addr 172.21.15.105:0/3239900645 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:41.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.645+0000 7f83db7fe640 1 --2- 172.21.15.105:0/3239900645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83dc0b2d20 0x7f83dc0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:41.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.645+0000 7f83daffd640 1 -- 172.21.15.105:0/3239900645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83dc0b2d20 msgr2=0x7f83dc0b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:41.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.645+0000 7f83daffd640 1 --2- 172.21.15.105:0/3239900645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83dc0b2d20 0x7f83dc0b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.645+0000 7f83daffd640 1 -- 172.21.15.105:0/3239900645 >> v1:172.21.15.148:6789/0 conn(0x7f83dc0d3c30 legacy=0x7f83dc0d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:41.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.645+0000 7f83daffd640 1 -- 172.21.15.105:0/3239900645 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83dc0cea30 con 0x7f83dc15d4f0 2023-11-28T18:25:41.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.649+0000 7f83daffd640 1 --2- 172.21.15.105:0/3239900645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83dc15d4f0 0x7f83dc0d0e10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009870 tx=0x7f83c401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad12f03338664676 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:41.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.649+0000 7f83da7fc640 1 -- 172.21.15.105:0/3239900645 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c40ab680 con 0x7f83dc15d4f0 2023-11-28T18:25:41.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.649+0000 7f83da7fc640 1 -- 172.21.15.105:0/3239900645 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c40ab7e0 con 0x7f83dc15d4f0 2023-11-28T18:25:41.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.649+0000 7f83da7fc640 1 -- 172.21.15.105:0/3239900645 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c40abb70 con 0x7f83dc15d4f0 2023-11-28T18:25:41.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.649+0000 7f83e14ea640 1 -- 172.21.15.105:0/3239900645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83dc15d4f0 msgr2=0x7f83dc0d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:41.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.649+0000 7f83e14ea640 1 --2- 172.21.15.105:0/3239900645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83dc15d4f0 0x7f83dc0d0e10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009870 tx=0x7f83c401c480 comp rx=0 tx=0).stop 2023-11-28T18:25:41.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.649+0000 7f83e14ea640 1 -- 172.21.15.105:0/3239900645 shutdown_connections 2023-11-28T18:25:41.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.649+0000 7f83e14ea640 1 --2- 172.21.15.105:0/3239900645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83dc0b2d20 0x7f83dc0b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.649+0000 7f83e14ea640 1 --2- 172.21.15.105:0/3239900645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83dc15d4f0 0x7f83dc0d0e10 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.649+0000 7f83e14ea640 1 -- 172.21.15.105:0/3239900645 >> 172.21.15.105:0/3239900645 conn(0x7f83dc0d3030 msgr2=0x7f83dc0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:41.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.649+0000 7f83e14ea640 1 -- 172.21.15.105:0/3239900645 shutdown_connections 2023-11-28T18:25:41.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.649+0000 7f83e14ea640 1 -- 172.21.15.105:0/3239900645 wait complete. 2023-11-28T18:25:41.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83e14ea640 1 Processor -- start 2023-11-28T18:25:41.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83e14ea640 1 -- start start 2023-11-28T18:25:41.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83e14ea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83dc0b2d20 0x7f83dc14c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:41.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83e14ea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83dc0d3c30 0x7f83dc0ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:41.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83e14ea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83dc15d4f0 0x7f83dc159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:41.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83e14ea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc15b7b0 con 0x7f83dc0b2d20 2023-11-28T18:25:41.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83e14ea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc0b5bb0 con 0x7f83dc0d3c30 2023-11-28T18:25:41.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83e14ea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc0caae0 con 0x7f83dc15d4f0 2023-11-28T18:25:41.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83daffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83dc0b2d20 0x7f83dc14c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:41.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83daffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83dc0b2d20 0x7f83dc14c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48248/0 (socket says 172.21.15.105:48248) 2023-11-28T18:25:41.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83daffd640 1 -- 172.21.15.105:0/3078337122 learned_addr learned my addr 172.21.15.105:0/3078337122 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:41.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83d27fc640 1 --2- 172.21.15.105:0/3078337122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83dc0d3c30 0x7f83dc0ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:41.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83db7fe640 1 --2- 172.21.15.105:0/3078337122 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83dc15d4f0 0x7f83dc159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:41.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83daffd640 1 -- 172.21.15.105:0/3078337122 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83dc15d4f0 msgr2=0x7f83dc159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:41.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83daffd640 1 --2- 172.21.15.105:0/3078337122 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83dc15d4f0 0x7f83dc159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83daffd640 1 -- 172.21.15.105:0/3078337122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83dc0d3c30 msgr2=0x7f83dc0ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:41.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83daffd640 1 --2- 172.21.15.105:0/3078337122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83dc0d3c30 0x7f83dc0ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83daffd640 1 -- 172.21.15.105:0/3078337122 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c40094d0 con 0x7f83dc0b2d20 2023-11-28T18:25:41.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83d27fc640 1 --2- 172.21.15.105:0/3078337122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83dc0d3c30 0x7f83dc0ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:41.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83daffd640 1 --2- 172.21.15.105:0/3078337122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83dc0b2d20 0x7f83dc14c3a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f83c40ad040 tx=0x7f83c401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:41.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83d3fff640 1 -- 172.21.15.105:0/3078337122 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c40b4800 con 0x7f83dc0b2d20 2023-11-28T18:25:41.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83d3fff640 1 -- 172.21.15.105:0/3078337122 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c40b4980 con 0x7f83dc0b2d20 2023-11-28T18:25:41.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83d3fff640 1 -- 172.21.15.105:0/3078337122 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c40b4d30 con 0x7f83dc0b2d20 2023-11-28T18:25:41.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83e14ea640 1 -- 172.21.15.105:0/3078337122 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83dc15ae10 con 0x7f83dc0b2d20 2023-11-28T18:25:41.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.653+0000 7f83e14ea640 1 -- 172.21.15.105:0/3078337122 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83dc15b3d0 con 0x7f83dc0b2d20 2023-11-28T18:25:41.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.657+0000 7f83e14ea640 1 -- 172.21.15.105:0/3078337122 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83dc15b8f0 con 0x7f83dc0b2d20 2023-11-28T18:25:41.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.657+0000 7f83d3fff640 1 -- 172.21.15.105:0/3078337122 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f83c40c7440 con 0x7f83dc0b2d20 2023-11-28T18:25:41.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.657+0000 7f83d3fff640 1 --2- 172.21.15.105:0/3078337122 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f83bc0668d0 0x7f83bc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:41.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.657+0000 7f83d3fff640 1 -- 172.21.15.105:0/3078337122 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f83c411f420 con 0x7f83dc0b2d20 2023-11-28T18:25:41.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.657+0000 7f83d27fc640 1 --2- 172.21.15.105:0/3078337122 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f83bc0668d0 0x7f83bc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:41.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.657+0000 7f83d27fc640 1 --2- 172.21.15.105:0/3078337122 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f83bc0668d0 0x7f83bc068d90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f83dc15a4a0 tx=0x7f83cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:41.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.661+0000 7f83d3fff640 1 -- 172.21.15.105:0/3078337122 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f83dc15b8f0 con 0x7f83dc0b2d20 2023-11-28T18:25:41.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.841+0000 7f83e14ea640 1 -- 172.21.15.105:0/3078337122 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83dc15b8f0 con 0x7f83dc0b2d20 2023-11-28T18:25:41.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.841+0000 7f83d3fff640 1 -- 172.21.15.105:0/3078337122 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f83dc15b8f0 con 0x7f83dc0b2d20 2023-11-28T18:25:41.844 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:41.844 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/47307750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":47307750},{"type":"v1","addr":"172.21.15.148:6834","nonce":47307750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:24:55.936428+0000","modified":"2023-11-28T18:25:33.692231+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5469,"mds_1":5463,"mds_2":5392},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5392":{"gid":5392,"name":"d","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6835/2878687703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2878687703},{"type":"v1","addr":"172.21.15.148:6835","nonce":2878687703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5463":{"gid":5463,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.105:6836/681913204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":681913204},{"type":"v1","addr":"172.21.15.105:6836","nonce":681913204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5469":{"gid":5469,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/1876408690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":1876408690},{"type":"v1","addr":"172.21.15.105:6837","nonce":1876408690}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T18:25:41.844 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 25 2023-11-28T18:25:41.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.845+0000 7f83e14ea640 1 -- 172.21.15.105:0/3078337122 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f83bc0668d0 msgr2=0x7f83bc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:41.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.845+0000 7f83e14ea640 1 --2- 172.21.15.105:0/3078337122 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f83bc0668d0 0x7f83bc068d90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f83dc15a4a0 tx=0x7f83cc009040 comp rx=0 tx=0).stop 2023-11-28T18:25:41.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.845+0000 7f83e14ea640 1 -- 172.21.15.105:0/3078337122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83dc0b2d20 msgr2=0x7f83dc14c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:41.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.845+0000 7f83e14ea640 1 --2- 172.21.15.105:0/3078337122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83dc0b2d20 0x7f83dc14c3a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f83c40ad040 tx=0x7f83c401acc0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.845+0000 7f83e14ea640 1 -- 172.21.15.105:0/3078337122 shutdown_connections 2023-11-28T18:25:41.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.845+0000 7f83e14ea640 1 --2- 172.21.15.105:0/3078337122 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83dc15d4f0 0x7f83dc159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.845+0000 7f83e14ea640 1 --2- 172.21.15.105:0/3078337122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83dc0d3c30 0x7f83dc0ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.845+0000 7f83e14ea640 1 --2- 172.21.15.105:0/3078337122 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f83bc0668d0 0x7f83bc068d90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.845+0000 7f83e14ea640 1 --2- 172.21.15.105:0/3078337122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83dc0b2d20 0x7f83dc14c3a0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:41.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.845+0000 7f83e14ea640 1 -- 172.21.15.105:0/3078337122 >> 172.21.15.105:0/3078337122 conn(0x7f83dc0d3030 msgr2=0x7f83dc0d1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:41.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.845+0000 7f83e14ea640 1 -- 172.21.15.105:0/3078337122 shutdown_connections 2023-11-28T18:25:41.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:41.845+0000 7f83e14ea640 1 -- 172.21.15.105:0/3078337122 wait complete. 2023-11-28T18:25:41.865 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:25:42.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.001+0000 7efc97be6640 1 Processor -- start 2023-11-28T18:25:42.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.001+0000 7efc97be6640 1 -- start start 2023-11-28T18:25:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.001+0000 7efc97be6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc9015d4f0 0x7efc900d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.001+0000 7efc97be6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efc900b2d20 0x7efc900b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.001+0000 7efc97be6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc900b25b0 con 0x7efc900d3c30 2023-11-28T18:25:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.001+0000 7efc97be6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc900b2900 con 0x7efc900b2d20 2023-11-28T18:25:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.001+0000 7efc97be6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc900d0a20 con 0x7efc9015d4f0 2023-11-28T18:25:42.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.001+0000 7efc95982640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc9015d4f0 0x7efc900d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:42.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.001+0000 7efc96183640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efc900b2d20 0x7efc900b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:42.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.001+0000 7efc95982640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc9015d4f0 0x7efc900d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:51436/0 (socket says 172.21.15.105:51436) 2023-11-28T18:25:42.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.001+0000 7efc95982640 1 -- 172.21.15.105:0/857475544 learned_addr learned my addr 172.21.15.105:0/857475544 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:42.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.001+0000 7efc95982640 1 -- 172.21.15.105:0/857475544 >> v2:172.21.15.148:3300/0 conn(0x7efc900b2d20 msgr2=0x7efc900b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:42.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.001+0000 7efc95982640 1 --2- 172.21.15.105:0/857475544 >> v2:172.21.15.148:3300/0 conn(0x7efc900b2d20 0x7efc900b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.001+0000 7efc95982640 1 -- 172.21.15.105:0/857475544 >> v1:172.21.15.148:6789/0 conn(0x7efc900d3c30 legacy=0x7efc900d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:42.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.001+0000 7efc95982640 1 -- 172.21.15.105:0/857475544 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc900cea30 con 0x7efc9015d4f0 2023-11-28T18:25:42.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.005+0000 7efc95982640 1 --2- 172.21.15.105:0/857475544 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc9015d4f0 0x7efc900d0e10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efc8c009870 tx=0x7efc8c01c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c422432f3a4bd242 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:42.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.005+0000 7efc94980640 1 -- 172.21.15.105:0/857475544 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c0ab680 con 0x7efc9015d4f0 2023-11-28T18:25:42.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.005+0000 7efc94980640 1 -- 172.21.15.105:0/857475544 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8c0ab7e0 con 0x7efc9015d4f0 2023-11-28T18:25:42.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.005+0000 7efc94980640 1 -- 172.21.15.105:0/857475544 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c0abb70 con 0x7efc9015d4f0 2023-11-28T18:25:42.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.005+0000 7efc97be6640 1 -- 172.21.15.105:0/857475544 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc9015d4f0 msgr2=0x7efc900d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:42.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.005+0000 7efc97be6640 1 --2- 172.21.15.105:0/857475544 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc9015d4f0 0x7efc900d0e10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efc8c009870 tx=0x7efc8c01c480 comp rx=0 tx=0).stop 2023-11-28T18:25:42.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.005+0000 7efc97be6640 1 -- 172.21.15.105:0/857475544 shutdown_connections 2023-11-28T18:25:42.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.005+0000 7efc97be6640 1 --2- 172.21.15.105:0/857475544 >> v2:172.21.15.148:3300/0 conn(0x7efc900b2d20 0x7efc900b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.005+0000 7efc97be6640 1 --2- 172.21.15.105:0/857475544 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc9015d4f0 0x7efc900d0e10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.005+0000 7efc97be6640 1 -- 172.21.15.105:0/857475544 >> 172.21.15.105:0/857475544 conn(0x7efc900d3030 msgr2=0x7efc900d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:42.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.005+0000 7efc97be6640 1 -- 172.21.15.105:0/857475544 shutdown_connections 2023-11-28T18:25:42.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.005+0000 7efc97be6640 1 -- 172.21.15.105:0/857475544 wait complete. 2023-11-28T18:25:42.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc97be6640 1 Processor -- start 2023-11-28T18:25:42.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc97be6640 1 -- start start 2023-11-28T18:25:42.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc97be6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc900b2d20 0x7efc90160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:42.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc97be6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc900d3c30 0x7efc901526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:42.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc97be6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc9015d4f0 0x7efc9015dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:42.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc97be6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc90152c10 con 0x7efc9015d4f0 2023-11-28T18:25:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc97be6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9015e230 con 0x7efc900d3c30 2023-11-28T18:25:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc97be6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc900ca100 con 0x7efc900b2d20 2023-11-28T18:25:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc96183640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc9015d4f0 0x7efc9015dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc96183640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc9015d4f0 0x7efc9015dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48262/0 (socket says 172.21.15.105:48262) 2023-11-28T18:25:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc96183640 1 -- 172.21.15.105:0/2453345820 learned_addr learned my addr 172.21.15.105:0/2453345820 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc95982640 1 --2- 172.21.15.105:0/2453345820 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc900b2d20 0x7efc90160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc95181640 1 --2- 172.21.15.105:0/2453345820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc900d3c30 0x7efc901526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc96183640 1 -- 172.21.15.105:0/2453345820 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc900b2d20 msgr2=0x7efc90160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc96183640 1 --2- 172.21.15.105:0/2453345820 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc900b2d20 0x7efc90160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc96183640 1 -- 172.21.15.105:0/2453345820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc900d3c30 msgr2=0x7efc901526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc96183640 1 --2- 172.21.15.105:0/2453345820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc900d3c30 0x7efc901526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc96183640 1 -- 172.21.15.105:0/2453345820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8c0094d0 con 0x7efc9015d4f0 2023-11-28T18:25:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc96183640 1 --2- 172.21.15.105:0/2453345820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc9015d4f0 0x7efc9015dcf0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7efc8800efc0 tx=0x7efc8800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc867fc640 1 -- 172.21.15.105:0/2453345820 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc88014480 con 0x7efc9015d4f0 2023-11-28T18:25:42.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc867fc640 1 -- 172.21.15.105:0/2453345820 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8800f040 con 0x7efc9015d4f0 2023-11-28T18:25:42.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc867fc640 1 -- 172.21.15.105:0/2453345820 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc88014820 con 0x7efc9015d4f0 2023-11-28T18:25:42.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.009+0000 7efc97be6640 1 -- 172.21.15.105:0/2453345820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc900ca3c0 con 0x7efc9015d4f0 2023-11-28T18:25:42.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.013+0000 7efc97be6640 1 -- 172.21.15.105:0/2453345820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc900ca980 con 0x7efc9015d4f0 2023-11-28T18:25:42.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.013+0000 7efc97be6640 1 -- 172.21.15.105:0/2453345820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc90067570 con 0x7efc9015d4f0 2023-11-28T18:25:42.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.013+0000 7efc867fc640 1 -- 172.21.15.105:0/2453345820 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7efc88014980 con 0x7efc9015d4f0 2023-11-28T18:25:42.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.013+0000 7efc867fc640 1 --2- 172.21.15.105:0/2453345820 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc740668d0 0x7efc74068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:42.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.013+0000 7efc867fc640 1 -- 172.21.15.105:0/2453345820 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efc88085070 con 0x7efc9015d4f0 2023-11-28T18:25:42.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.013+0000 7efc95982640 1 --2- 172.21.15.105:0/2453345820 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc740668d0 0x7efc74068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:42.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.013+0000 7efc95982640 1 --2- 172.21.15.105:0/2453345820 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc740668d0 0x7efc74068d90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0af000 tx=0x7efc8c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:42.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.017+0000 7efc867fc640 1 -- 172.21.15.105:0/2453345820 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efc880b6280 con 0x7efc9015d4f0 2023-11-28T18:25:42.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.201+0000 7efc97be6640 1 -- 172.21.15.105:0/2453345820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc900b0b80 con 0x7efc9015d4f0 2023-11-28T18:25:42.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.201+0000 7efc867fc640 1 -- 172.21.15.105:0/2453345820 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4947 (secure 0 0 0) 0x7efc88018090 con 0x7efc9015d4f0 2023-11-28T18:25:42.202 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:42.202 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/47307750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":47307750},{"type":"v1","addr":"172.21.15.148:6834","nonce":47307750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:24:55.936428+0000","modified":"2023-11-28T18:25:33.692231+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5469,"mds_1":5463,"mds_2":5392},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5392":{"gid":5392,"name":"d","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6835/2878687703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2878687703},{"type":"v1","addr":"172.21.15.148:6835","nonce":2878687703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5463":{"gid":5463,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.105:6836/681913204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":681913204},{"type":"v1","addr":"172.21.15.105:6836","nonce":681913204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5469":{"gid":5469,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/1876408690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":1876408690},{"type":"v1","addr":"172.21.15.105:6837","nonce":1876408690}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T18:25:42.203 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 25 2023-11-28T18:25:42.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.201+0000 7efc97be6640 1 -- 172.21.15.105:0/2453345820 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc740668d0 msgr2=0x7efc74068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:42.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.201+0000 7efc97be6640 1 --2- 172.21.15.105:0/2453345820 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc740668d0 0x7efc74068d90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0af000 tx=0x7efc8c0a7000 comp rx=0 tx=0).stop 2023-11-28T18:25:42.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.201+0000 7efc97be6640 1 -- 172.21.15.105:0/2453345820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc9015d4f0 msgr2=0x7efc9015dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:42.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.201+0000 7efc97be6640 1 --2- 172.21.15.105:0/2453345820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc9015d4f0 0x7efc9015dcf0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7efc8800efc0 tx=0x7efc8800c490 comp rx=0 tx=0).stop 2023-11-28T18:25:42.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.201+0000 7efc97be6640 1 -- 172.21.15.105:0/2453345820 shutdown_connections 2023-11-28T18:25:42.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.201+0000 7efc97be6640 1 --2- 172.21.15.105:0/2453345820 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc740668d0 0x7efc74068d90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.201+0000 7efc97be6640 1 --2- 172.21.15.105:0/2453345820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc9015d4f0 0x7efc9015dcf0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.201+0000 7efc97be6640 1 --2- 172.21.15.105:0/2453345820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc900d3c30 0x7efc901526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.201+0000 7efc97be6640 1 --2- 172.21.15.105:0/2453345820 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc900b2d20 0x7efc90160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.201+0000 7efc97be6640 1 -- 172.21.15.105:0/2453345820 >> 172.21.15.105:0/2453345820 conn(0x7efc900d3030 msgr2=0x7efc900afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:42.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.201+0000 7efc97be6640 1 -- 172.21.15.105:0/2453345820 shutdown_connections 2023-11-28T18:25:42.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.201+0000 7efc97be6640 1 -- 172.21.15.105:0/2453345820 wait complete. 2023-11-28T18:25:42.223 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:25:42.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.357+0000 7fde15bf8640 1 Processor -- start 2023-11-28T18:25:42.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.357+0000 7fde15bf8640 1 -- start start 2023-11-28T18:25:42.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.361+0000 7fde15bf8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde10071960 0x7fde1015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:42.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.361+0000 7fde15bf8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fde1015b030 0x7fde100709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:42.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.361+0000 7fde15bf8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde100701d0 con 0x7fde100b4290 2023-11-28T18:25:42.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.361+0000 7fde15bf8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1006df80 con 0x7fde1015b030 2023-11-28T18:25:42.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.361+0000 7fde15bf8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde10071df0 con 0x7fde10071960 2023-11-28T18:25:42.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.361+0000 7fde0ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fde1015b030 0x7fde100709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:42.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.361+0000 7fde07fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde10071960 0x7fde1015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:42.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.361+0000 7fde0ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fde1015b030 0x7fde100709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:39496/0 (socket says 172.21.15.105:39496) 2023-11-28T18:25:42.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.361+0000 7fde0ffff640 1 -- 172.21.15.105:0/427117427 learned_addr learned my addr 172.21.15.105:0/427117427 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:42.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.361+0000 7fde07fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde10071960 0x7fde1015f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:51460/0 (socket says 172.21.15.105:51460) 2023-11-28T18:25:42.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.361+0000 7fde0ffff640 1 -- 172.21.15.105:0/427117427 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde10071960 msgr2=0x7fde1015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:42.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.361+0000 7fde0ffff640 1 --2- 172.21.15.105:0/427117427 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde10071960 0x7fde1015f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.361+0000 7fde0ffff640 1 -- 172.21.15.105:0/427117427 >> v1:172.21.15.148:6789/0 conn(0x7fde100b4290 legacy=0x7fde100b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:42.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.361+0000 7fde0ffff640 1 -- 172.21.15.105:0/427117427 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde100720d0 con 0x7fde1015b030 2023-11-28T18:25:42.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.361+0000 7fde0effd640 1 -- 172.21.15.105:0/427117427 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3215666449 0 0) 0x7fde100701d0 con 0x7fde100b4290 2023-11-28T18:25:42.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.365+0000 7fde0ffff640 1 --2- 172.21.15.105:0/427117427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde1015b030 0x7fde100709b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fde08009e50 tx=0x7fde0801c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c6112e8a0bd7396 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:42.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.365+0000 7fde0effd640 1 -- 172.21.15.105:0/427117427 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde080a57c0 con 0x7fde1015b030 2023-11-28T18:25:42.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.365+0000 7fde0effd640 1 -- 172.21.15.105:0/427117427 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde080a5920 con 0x7fde1015b030 2023-11-28T18:25:42.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.365+0000 7fde0effd640 1 -- 172.21.15.105:0/427117427 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde080a5cb0 con 0x7fde1015b030 2023-11-28T18:25:42.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.365+0000 7fde15bf8640 1 -- 172.21.15.105:0/427117427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde1015b030 msgr2=0x7fde100709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:42.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.365+0000 7fde15bf8640 1 --2- 172.21.15.105:0/427117427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde1015b030 0x7fde100709b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fde08009e50 tx=0x7fde0801c9e0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.365+0000 7fde15bf8640 1 -- 172.21.15.105:0/427117427 shutdown_connections 2023-11-28T18:25:42.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.365+0000 7fde15bf8640 1 --2- 172.21.15.105:0/427117427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde1015b030 0x7fde100709b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.365+0000 7fde15bf8640 1 --2- 172.21.15.105:0/427117427 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde10071960 0x7fde1015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.365+0000 7fde15bf8640 1 -- 172.21.15.105:0/427117427 >> 172.21.15.105:0/427117427 conn(0x7fde100ceb90 msgr2=0x7fde100cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:42.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.365+0000 7fde15bf8640 1 -- 172.21.15.105:0/427117427 shutdown_connections 2023-11-28T18:25:42.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.365+0000 7fde15bf8640 1 -- 172.21.15.105:0/427117427 wait complete. 2023-11-28T18:25:42.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.365+0000 7fde15bf8640 1 Processor -- start 2023-11-28T18:25:42.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.365+0000 7fde15bf8640 1 -- start start 2023-11-28T18:25:42.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.365+0000 7fde15bf8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde100b4290 0x7fde100cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:42.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde15bf8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde1015b030 0x7fde100cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:42.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde15bf8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde10154b60 0x7fde101413c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:42.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde15bf8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde10141130 con 0x7fde1015b030 2023-11-28T18:25:42.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde15bf8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde100cd450 con 0x7fde100b4290 2023-11-28T18:25:42.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde15bf8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde100ad550 con 0x7fde10154b60 2023-11-28T18:25:42.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde07fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde1015b030 0x7fde100cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:42.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde07fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde1015b030 0x7fde100cd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48272/0 (socket says 172.21.15.105:48272) 2023-11-28T18:25:42.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde07fff640 1 -- 172.21.15.105:0/2765622348 learned_addr learned my addr 172.21.15.105:0/2765622348 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:42.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde0f7fe640 1 --2- 172.21.15.105:0/2765622348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde100b4290 0x7fde100cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:42.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde0ffff640 1 --2- 172.21.15.105:0/2765622348 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde10154b60 0x7fde101413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:42.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde07fff640 1 -- 172.21.15.105:0/2765622348 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde10154b60 msgr2=0x7fde101413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:42.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde07fff640 1 --2- 172.21.15.105:0/2765622348 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde10154b60 0x7fde101413c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde07fff640 1 -- 172.21.15.105:0/2765622348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde100b4290 msgr2=0x7fde100cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:42.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde07fff640 1 --2- 172.21.15.105:0/2765622348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde100b4290 0x7fde100cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde07fff640 1 -- 172.21.15.105:0/2765622348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde08009b30 con 0x7fde1015b030 2023-11-28T18:25:42.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde07fff640 1 --2- 172.21.15.105:0/2765622348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde1015b030 0x7fde100cd8e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fddf800e930 tx=0x7fddf800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:42.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde0cff9640 1 -- 172.21.15.105:0/2765622348 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf8009690 con 0x7fde1015b030 2023-11-28T18:25:42.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde0cff9640 1 -- 172.21.15.105:0/2765622348 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf800ca00 con 0x7fde1015b030 2023-11-28T18:25:42.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde15bf8640 1 -- 172.21.15.105:0/2765622348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde100ad830 con 0x7fde1015b030 2023-11-28T18:25:42.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde0cff9640 1 -- 172.21.15.105:0/2765622348 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf800cd70 con 0x7fde1015b030 2023-11-28T18:25:42.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde15bf8640 1 -- 172.21.15.105:0/2765622348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1009c3f0 con 0x7fde1015b030 2023-11-28T18:25:42.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde15bf8640 1 -- 172.21.15.105:0/2765622348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde100ada70 con 0x7fde1015b030 2023-11-28T18:25:42.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.369+0000 7fde0cff9640 1 -- 172.21.15.105:0/2765622348 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fddf8013470 con 0x7fde1015b030 2023-11-28T18:25:42.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.373+0000 7fde0cff9640 1 --2- 172.21.15.105:0/2765622348 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fddf00668d0 0x7fddf0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:42.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.373+0000 7fde0cff9640 1 -- 172.21.15.105:0/2765622348 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fddf80848b0 con 0x7fde1015b030 2023-11-28T18:25:42.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.373+0000 7fde0f7fe640 1 --2- 172.21.15.105:0/2765622348 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fddf00668d0 0x7fddf0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:42.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.373+0000 7fde0f7fe640 1 --2- 172.21.15.105:0/2765622348 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fddf00668d0 0x7fddf0068d90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fde00000f70 tx=0x7fde00009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:42.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.377+0000 7fde0cff9640 1 -- 172.21.15.105:0/2765622348 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fddf80519c0 con 0x7fde1015b030 2023-11-28T18:25:42.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.525+0000 7fde15bf8640 1 -- 172.21.15.105:0/2765622348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde100cd5c0 con 0x7fde1015b030 2023-11-28T18:25:42.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.529+0000 7fde0cff9640 1 -- 172.21.15.105:0/2765622348 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16827 (secure 0 0 0) 0x7fddf8021090 con 0x7fde1015b030 2023-11-28T18:25:42.529 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:42.530 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":40,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:25:15.107712+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:24:53.582892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T18:24:54.921978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.39","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:25:42.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.529+0000 7fde15bf8640 1 -- 172.21.15.105:0/2765622348 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fddf00668d0 msgr2=0x7fddf0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:42.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.529+0000 7fde15bf8640 1 --2- 172.21.15.105:0/2765622348 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fddf00668d0 0x7fddf0068d90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fde00000f70 tx=0x7fde00009040 comp rx=0 tx=0).stop 2023-11-28T18:25:42.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.529+0000 7fde15bf8640 1 -- 172.21.15.105:0/2765622348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde1015b030 msgr2=0x7fde100cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:42.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.529+0000 7fde15bf8640 1 --2- 172.21.15.105:0/2765622348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde1015b030 0x7fde100cd8e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fddf800e930 tx=0x7fddf800edf0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.529+0000 7fde15bf8640 1 -- 172.21.15.105:0/2765622348 shutdown_connections 2023-11-28T18:25:42.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.529+0000 7fde15bf8640 1 --2- 172.21.15.105:0/2765622348 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde10154b60 0x7fde101413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.529+0000 7fde15bf8640 1 --2- 172.21.15.105:0/2765622348 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fddf00668d0 0x7fddf0068d90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.529+0000 7fde15bf8640 1 --2- 172.21.15.105:0/2765622348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde1015b030 0x7fde100cd8e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.529+0000 7fde15bf8640 1 --2- 172.21.15.105:0/2765622348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde100b4290 0x7fde100cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.529+0000 7fde15bf8640 1 -- 172.21.15.105:0/2765622348 >> 172.21.15.105:0/2765622348 conn(0x7fde100ceb90 msgr2=0x7fde10163f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:42.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.529+0000 7fde15bf8640 1 -- 172.21.15.105:0/2765622348 shutdown_connections 2023-11-28T18:25:42.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.529+0000 7fde15bf8640 1 -- 172.21.15.105:0/2765622348 wait complete. 2023-11-28T18:25:42.551 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T18:25:42.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.689+0000 7f2e0e565640 1 Processor -- start 2023-11-28T18:25:42.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.689+0000 7f2e0e565640 1 -- start start 2023-11-28T18:25:42.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.689+0000 7f2e0e565640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2e080b4290 0x7f2e080b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:42.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.689+0000 7f2e0e565640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2e0815b030 0x7f2e080709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:42.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.689+0000 7f2e0e565640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e080b4bb0 con 0x7f2e08071960 2023-11-28T18:25:42.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.689+0000 7f2e0e565640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e080701d0 con 0x7f2e080b4290 2023-11-28T18:25:42.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.689+0000 7f2e0e565640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0806df80 con 0x7f2e0815b030 2023-11-28T18:25:42.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.689+0000 7f2e07fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2e080b4290 0x7f2e080b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:42.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.689+0000 7f2e0cb02640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2e0815b030 0x7f2e080709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:42.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.689+0000 7f2e077fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2e08071960 0x7f2e0815f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:55026/0 (socket says 172.21.15.105:55026) 2023-11-28T18:25:42.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.689+0000 7f2e077fe640 1 -- 172.21.15.105:0/1444930348 learned_addr learned my addr 172.21.15.105:0/1444930348 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:42.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.689+0000 7f2e06ffd640 1 -- 172.21.15.105:0/1444930348 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 192671395 0 0) 0x7f2e080b4bb0 con 0x7f2e08071960 2023-11-28T18:25:42.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.689+0000 7f2e0cb02640 1 -- 172.21.15.105:0/1444930348 >> v2:172.21.15.148:3300/0 conn(0x7f2e080b4290 msgr2=0x7f2e080b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:42.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.689+0000 7f2e0cb02640 1 --2- 172.21.15.105:0/1444930348 >> v2:172.21.15.148:3300/0 conn(0x7f2e080b4290 0x7f2e080b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.689+0000 7f2e0cb02640 1 -- 172.21.15.105:0/1444930348 >> v1:172.21.15.148:6789/0 conn(0x7f2e08071960 legacy=0x7f2e0815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:42.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.689+0000 7f2e0cb02640 1 -- 172.21.15.105:0/1444930348 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e08071df0 con 0x7f2e0815b030 2023-11-28T18:25:42.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.689+0000 7f2e07fff640 1 --2- 172.21.15.105:0/1444930348 >> v2:172.21.15.148:3300/0 conn(0x7f2e080b4290 0x7f2e080b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:25:42.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.693+0000 7f2e0cb02640 1 --2- 172.21.15.105:0/1444930348 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2e0815b030 0x7f2e080709b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009d90 tx=0x7f2e0001c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af5031f636c21420 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:42.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.693+0000 7f2e06ffd640 1 -- 172.21.15.105:0/1444930348 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e000a58f0 con 0x7f2e0815b030 2023-11-28T18:25:42.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.693+0000 7f2e06ffd640 1 -- 172.21.15.105:0/1444930348 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e000a5a50 con 0x7f2e0815b030 2023-11-28T18:25:42.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.693+0000 7f2e06ffd640 1 -- 172.21.15.105:0/1444930348 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e000a5de0 con 0x7f2e0815b030 2023-11-28T18:25:42.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.693+0000 7f2e0e565640 1 -- 172.21.15.105:0/1444930348 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2e0815b030 msgr2=0x7f2e080709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:42.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.693+0000 7f2e0e565640 1 --2- 172.21.15.105:0/1444930348 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2e0815b030 0x7f2e080709b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009d90 tx=0x7f2e0001c920 comp rx=0 tx=0).stop 2023-11-28T18:25:42.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.693+0000 7f2e0e565640 1 -- 172.21.15.105:0/1444930348 shutdown_connections 2023-11-28T18:25:42.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.693+0000 7f2e0e565640 1 --2- 172.21.15.105:0/1444930348 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2e0815b030 0x7f2e080709b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.693+0000 7f2e0e565640 1 --2- 172.21.15.105:0/1444930348 >> v2:172.21.15.148:3300/0 conn(0x7f2e080b4290 0x7f2e080b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.693+0000 7f2e0e565640 1 -- 172.21.15.105:0/1444930348 >> 172.21.15.105:0/1444930348 conn(0x7f2e080ceb90 msgr2=0x7f2e080cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:42.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.693+0000 7f2e0e565640 1 -- 172.21.15.105:0/1444930348 shutdown_connections 2023-11-28T18:25:42.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.693+0000 7f2e0e565640 1 -- 172.21.15.105:0/1444930348 wait complete. 2023-11-28T18:25:42.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e0e565640 1 Processor -- start 2023-11-28T18:25:42.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e0e565640 1 -- start start 2023-11-28T18:25:42.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e0e565640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2e080b4290 0x7f2e081548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:42.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e0e565640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e0815b030 0x7f2e080b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:42.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e0e565640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e080b2c20 0x7f2e081412a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:42.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e0e565640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e080b2800 con 0x7f2e080b2c20 2023-11-28T18:25:42.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e0e565640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e080b2250 con 0x7f2e0815b030 2023-11-28T18:25:42.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e0e565640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e080b2460 con 0x7f2e080b4290 2023-11-28T18:25:42.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e0cb02640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e080b2c20 0x7f2e081412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:42.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e0cb02640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e080b2c20 0x7f2e081412a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48288/0 (socket says 172.21.15.105:48288) 2023-11-28T18:25:42.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e0cb02640 1 -- 172.21.15.105:0/1323471744 learned_addr learned my addr 172.21.15.105:0/1323471744 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:42.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e07fff640 1 --2- 172.21.15.105:0/1323471744 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2e080b4290 0x7f2e081548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:42.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e077fe640 1 --2- 172.21.15.105:0/1323471744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e0815b030 0x7f2e080b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:42.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e0cb02640 1 -- 172.21.15.105:0/1323471744 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2e080b4290 msgr2=0x7f2e081548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:42.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e0cb02640 1 --2- 172.21.15.105:0/1323471744 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2e080b4290 0x7f2e081548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e0cb02640 1 -- 172.21.15.105:0/1323471744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e0815b030 msgr2=0x7f2e080b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:42.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e0cb02640 1 --2- 172.21.15.105:0/1323471744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e0815b030 0x7f2e080b1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:42.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e0cb02640 1 -- 172.21.15.105:0/1323471744 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df8009420 con 0x7f2e080b2c20 2023-11-28T18:25:42.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e077fe640 1 --2- 172.21.15.105:0/1323471744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e0815b030 0x7f2e080b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:42.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e0cb02640 1 --2- 172.21.15.105:0/1323471744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e080b2c20 0x7f2e081412a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2e000b0040 tx=0x7f2e000196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:42.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e04ff9640 1 -- 172.21.15.105:0/1323471744 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e000b9860 con 0x7f2e080b2c20 2023-11-28T18:25:42.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e04ff9640 1 -- 172.21.15.105:0/1323471744 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e000b99c0 con 0x7f2e080b2c20 2023-11-28T18:25:42.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e04ff9640 1 -- 172.21.15.105:0/1323471744 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e000b9d50 con 0x7f2e080b2c20 2023-11-28T18:25:42.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e0e565640 1 -- 172.21.15.105:0/1323471744 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e00009a70 con 0x7f2e080b2c20 2023-11-28T18:25:42.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.697+0000 7f2e0e565640 1 -- 172.21.15.105:0/1323471744 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e08155530 con 0x7f2e080b2c20 2023-11-28T18:25:42.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.701+0000 7f2e0e565640 1 -- 172.21.15.105:0/1323471744 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e080b2940 con 0x7f2e080b2c20 2023-11-28T18:25:42.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.701+0000 7f2e04ff9640 1 -- 172.21.15.105:0/1323471744 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f2e000c4460 con 0x7f2e080b2c20 2023-11-28T18:25:42.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.701+0000 7f2e04ff9640 1 --2- 172.21.15.105:0/1323471744 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2df40668d0 0x7f2df4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:42.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.701+0000 7f2e04ff9640 1 -- 172.21.15.105:0/1323471744 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2e0011f9c0 con 0x7f2e080b2c20 2023-11-28T18:25:42.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.701+0000 7f2e07fff640 1 --2- 172.21.15.105:0/1323471744 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2df40668d0 0x7f2df4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:42.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.701+0000 7f2e07fff640 1 --2- 172.21.15.105:0/1323471744 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2df40668d0 0x7f2df4068d90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009c80 tx=0x7f2df8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:42.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.705+0000 7f2e04ff9640 1 -- 172.21.15.105:0/1323471744 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2e080b2940 con 0x7f2e080b2c20 2023-11-28T18:25:42.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.885+0000 7f2e0e565640 1 -- 172.21.15.105:0/1323471744 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2e080b2940 con 0x7f2e080b2c20 2023-11-28T18:25:47.748 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2023-11-28T18:25:47.749 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-11-28T18:25:42.885+0000 7fd89b32a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T18:25:47.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.889+0000 7f2e04ff9640 1 -- 172.21.15.105:0/1323471744 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2e080b2940 con 0x7f2e080b2c20 2023-11-28T18:25:47.751 INFO:teuthology.orchestra.run.smithi105.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T18:25:47.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.889+0000 7f2e0e565640 1 -- 172.21.15.105:0/1323471744 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2df40668d0 msgr2=0x7f2df4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:47.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.889+0000 7f2e0e565640 1 --2- 172.21.15.105:0/1323471744 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2df40668d0 0x7f2df4068d90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009c80 tx=0x7f2df8008040 comp rx=0 tx=0).stop 2023-11-28T18:25:47.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.893+0000 7f2e0e565640 1 -- 172.21.15.105:0/1323471744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e080b2c20 msgr2=0x7f2e081412a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:47.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.893+0000 7f2e0e565640 1 --2- 172.21.15.105:0/1323471744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e080b2c20 0x7f2e081412a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2e000b0040 tx=0x7f2e000196c0 comp rx=0 tx=0).stop 2023-11-28T18:25:47.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.893+0000 7f2e0e565640 1 -- 172.21.15.105:0/1323471744 shutdown_connections 2023-11-28T18:25:47.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.893+0000 7f2e0e565640 1 --2- 172.21.15.105:0/1323471744 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2df40668d0 0x7f2df4068d90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:47.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.893+0000 7f2e0e565640 1 --2- 172.21.15.105:0/1323471744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e080b2c20 0x7f2e081412a0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:47.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.893+0000 7f2e0e565640 1 --2- 172.21.15.105:0/1323471744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e0815b030 0x7f2e080b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:47.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.893+0000 7f2e0e565640 1 --2- 172.21.15.105:0/1323471744 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2e080b4290 0x7f2e081548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:47.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.893+0000 7f2e0e565640 1 -- 172.21.15.105:0/1323471744 >> 172.21.15.105:0/1323471744 conn(0x7f2e080ceb90 msgr2=0x7f2e0806c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:47.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.893+0000 7f2e0e565640 1 -- 172.21.15.105:0/1323471744 shutdown_connections 2023-11-28T18:25:47.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:42.893+0000 7f2e0e565640 1 -- 172.21.15.105:0/1323471744 wait complete. 2023-11-28T18:25:47.752 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T18:25:47.753 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2023-11-28T18:25:47.753 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2023-11-28T18:25:47.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.901+0000 7f3fdd867640 1 Processor -- start 2023-11-28T18:25:47.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.901+0000 7f3fdd867640 1 -- start start 2023-11-28T18:25:47.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.901+0000 7f3fdd867640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3fd815d350 0x7f3fd80d38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:47.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.901+0000 7f3fdd867640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fd80cfd90 0x7f3fd80b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:47.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.901+0000 7f3fdd867640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd80cdae0 con 0x7f3fd815d730 2023-11-28T18:25:47.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.901+0000 7f3fdd867640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd80d1f40 con 0x7f3fd815d350 2023-11-28T18:25:47.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.901+0000 7f3fdd867640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd80d16a0 con 0x7f3fd80cfd90 2023-11-28T18:25:47.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.901+0000 7f3fdc865640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3fd815d350 0x7f3fd80d38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:47.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.901+0000 7f3fdc865640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3fd815d350 0x7f3fd80d38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48338/0 (socket says 172.21.15.105:48338) 2023-11-28T18:25:47.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.901+0000 7f3fdc865640 1 -- 172.21.15.105:0/2554675710 learned_addr learned my addr 172.21.15.105:0/2554675710 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:47.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.901+0000 7f3fdc865640 1 -- 172.21.15.105:0/2554675710 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fd80cfd90 msgr2=0x7f3fd80b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:47.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.901+0000 7f3fdc865640 1 --2- 172.21.15.105:0/2554675710 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fd80cfd90 0x7f3fd80b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:47.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.901+0000 7f3fdc865640 1 -- 172.21.15.105:0/2554675710 >> v1:172.21.15.148:6789/0 conn(0x7f3fd815d730 legacy=0x7f3fd80cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:47.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.901+0000 7f3fdc865640 1 -- 172.21.15.105:0/2554675710 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd80d1980 con 0x7f3fd815d350 2023-11-28T18:25:47.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.905+0000 7f3fdc865640 1 --2- 172.21.15.105:0/2554675710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd815d350 0x7f3fd80d38b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009870 tx=0x7f3fd401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617eadb98185d05f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:47.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.905+0000 7f3fcf7fe640 1 -- 172.21.15.105:0/2554675710 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd40ab600 con 0x7f3fd815d350 2023-11-28T18:25:47.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.905+0000 7f3fcf7fe640 1 -- 172.21.15.105:0/2554675710 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd40a7050 con 0x7f3fd815d350 2023-11-28T18:25:47.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.905+0000 7f3fcf7fe640 1 -- 172.21.15.105:0/2554675710 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd40ab980 con 0x7f3fd815d350 2023-11-28T18:25:47.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.905+0000 7f3fdd867640 1 -- 172.21.15.105:0/2554675710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd815d350 msgr2=0x7f3fd80d38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:47.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.905+0000 7f3fdd867640 1 --2- 172.21.15.105:0/2554675710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd815d350 0x7f3fd80d38b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009870 tx=0x7f3fd401c490 comp rx=0 tx=0).stop 2023-11-28T18:25:47.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.905+0000 7f3fdd867640 1 -- 172.21.15.105:0/2554675710 shutdown_connections 2023-11-28T18:25:47.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.905+0000 7f3fdd867640 1 --2- 172.21.15.105:0/2554675710 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fd80cfd90 0x7f3fd80b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:47.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.905+0000 7f3fdd867640 1 --2- 172.21.15.105:0/2554675710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd815d350 0x7f3fd80d38b0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:47.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.905+0000 7f3fdd867640 1 -- 172.21.15.105:0/2554675710 >> 172.21.15.105:0/2554675710 conn(0x7f3fd80d2d90 msgr2=0x7f3fd80d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:47.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.905+0000 7f3fdd867640 1 -- 172.21.15.105:0/2554675710 shutdown_connections 2023-11-28T18:25:47.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.905+0000 7f3fdd867640 1 -- 172.21.15.105:0/2554675710 wait complete. 2023-11-28T18:25:47.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.905+0000 7f3fdd867640 1 Processor -- start 2023-11-28T18:25:47.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.905+0000 7f3fdd867640 1 -- start start 2023-11-28T18:25:47.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fdd867640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd80cfd90 0x7f3fd80b8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:47.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fdd867640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fd815d350 0x7f3fd815db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:47.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fdd867640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fd815d730 0x7f3fd815f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:47.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fdd867640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd815e0a0 con 0x7f3fd80cfd90 2023-11-28T18:25:47.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fdd867640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd81602b0 con 0x7f3fd815d730 2023-11-28T18:25:47.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fdd867640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd8160590 con 0x7f3fd815d350 2023-11-28T18:25:47.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fdc865640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd80cfd90 0x7f3fd80b8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:47.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fdc865640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd80cfd90 0x7f3fd80b8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48348/0 (socket says 172.21.15.105:48348) 2023-11-28T18:25:47.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fdc865640 1 -- 172.21.15.105:0/1865497814 learned_addr learned my addr 172.21.15.105:0/1865497814 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:47.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fcffff640 1 --2- 172.21.15.105:0/1865497814 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fd815d350 0x7f3fd815db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:47.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fdd066640 1 --2- 172.21.15.105:0/1865497814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fd815d730 0x7f3fd815f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:47.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fdc865640 1 -- 172.21.15.105:0/1865497814 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fd815d350 msgr2=0x7f3fd815db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:47.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fdc865640 1 --2- 172.21.15.105:0/1865497814 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fd815d350 0x7f3fd815db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:47.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fdc865640 1 -- 172.21.15.105:0/1865497814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fd815d730 msgr2=0x7f3fd815f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:47.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fdc865640 1 --2- 172.21.15.105:0/1865497814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fd815d730 0x7f3fd815f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:47.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fdc865640 1 -- 172.21.15.105:0/1865497814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd40094d0 con 0x7f3fd80cfd90 2023-11-28T18:25:47.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fdd066640 1 --2- 172.21.15.105:0/1865497814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fd815d730 0x7f3fd815f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:47.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fcffff640 1 --2- 172.21.15.105:0/1865497814 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fd815d350 0x7f3fd815db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:47.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fdc865640 1 --2- 172.21.15.105:0/1865497814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd80cfd90 0x7f3fd80b8520 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3fd401ab40 tx=0x7f3fd4019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:47.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fcd7fa640 1 -- 172.21.15.105:0/1865497814 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd40b6640 con 0x7f3fd80cfd90 2023-11-28T18:25:47.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fcd7fa640 1 -- 172.21.15.105:0/1865497814 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd40b67c0 con 0x7f3fd80cfd90 2023-11-28T18:25:47.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fdd867640 1 -- 172.21.15.105:0/1865497814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd8160870 con 0x7f3fd80cfd90 2023-11-28T18:25:47.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fcd7fa640 1 -- 172.21.15.105:0/1865497814 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd40b6b70 con 0x7f3fd80cfd90 2023-11-28T18:25:47.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fdd867640 1 -- 172.21.15.105:0/1865497814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd815fdc0 con 0x7f3fd80cfd90 2023-11-28T18:25:47.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fdd867640 1 -- 172.21.15.105:0/1865497814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd8160070 con 0x7f3fd80cfd90 2023-11-28T18:25:47.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.909+0000 7f3fcd7fa640 1 -- 172.21.15.105:0/1865497814 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3fd40b2040 con 0x7f3fd80cfd90 2023-11-28T18:25:47.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.913+0000 7f3fcd7fa640 1 --2- 172.21.15.105:0/1865497814 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3fc40668d0 0x7f3fc4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:47.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.913+0000 7f3fcd7fa640 1 -- 172.21.15.105:0/1865497814 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f3fd411e640 con 0x7f3fd80cfd90 2023-11-28T18:25:47.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.913+0000 7f3fcffff640 1 --2- 172.21.15.105:0/1865497814 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3fc40668d0 0x7f3fc4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:47.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.913+0000 7f3fcffff640 1 --2- 172.21.15.105:0/1865497814 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3fc40668d0 0x7f3fc4068d90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80b5ad0 tx=0x7f3fc8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:47.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:47.917+0000 7f3fcd7fa640 1 -- 172.21.15.105:0/1865497814 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3fd8160070 con 0x7f3fd80cfd90 2023-11-28T18:25:48.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:48.101+0000 7f3fdd867640 1 -- 172.21.15.105:0/1865497814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3fd81603f0 con 0x7f3fd80cfd90 2023-11-28T18:25:49.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.069+0000 7f3fcd7fa640 1 -- 172.21.15.105:0/1865497814 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3fd81603f0 con 0x7f3fd80cfd90 2023-11-28T18:25:49.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.069+0000 7f3fdd867640 1 -- 172.21.15.105:0/1865497814 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3fc40668d0 msgr2=0x7f3fc4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:49.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.069+0000 7f3fdd867640 1 --2- 172.21.15.105:0/1865497814 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3fc40668d0 0x7f3fc4068d90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80b5ad0 tx=0x7f3fc8008040 comp rx=0 tx=0).stop 2023-11-28T18:25:49.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.069+0000 7f3fdd867640 1 -- 172.21.15.105:0/1865497814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd80cfd90 msgr2=0x7f3fd80b8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:49.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.069+0000 7f3fdd867640 1 --2- 172.21.15.105:0/1865497814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd80cfd90 0x7f3fd80b8520 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3fd401ab40 tx=0x7f3fd4019230 comp rx=0 tx=0).stop 2023-11-28T18:25:49.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.073+0000 7f3fdd867640 1 -- 172.21.15.105:0/1865497814 shutdown_connections 2023-11-28T18:25:49.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.073+0000 7f3fdd867640 1 --2- 172.21.15.105:0/1865497814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fd815d730 0x7f3fd815f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:49.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.073+0000 7f3fdd867640 1 --2- 172.21.15.105:0/1865497814 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fd815d350 0x7f3fd815db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:49.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.073+0000 7f3fdd867640 1 --2- 172.21.15.105:0/1865497814 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3fc40668d0 0x7f3fc4068d90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:49.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.073+0000 7f3fdd867640 1 --2- 172.21.15.105:0/1865497814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd80cfd90 0x7f3fd80b8520 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:49.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.073+0000 7f3fdd867640 1 -- 172.21.15.105:0/1865497814 >> 172.21.15.105:0/1865497814 conn(0x7f3fd80d2d90 msgr2=0x7f3fd806ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:49.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.073+0000 7f3fdd867640 1 -- 172.21.15.105:0/1865497814 shutdown_connections 2023-11-28T18:25:49.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.073+0000 7f3fdd867640 1 -- 172.21.15.105:0/1865497814 wait complete. 2023-11-28T18:25:49.098 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T18:25:49.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.245+0000 7f7bd83aa640 1 Processor -- start 2023-11-28T18:25:49.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.245+0000 7f7bd83aa640 1 -- start start 2023-11-28T18:25:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.245+0000 7f7bd83aa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7bd00caf60 0x7f7bd00cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.245+0000 7f7bd83aa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7bd00ad850 0x7f7bd00701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.245+0000 7f7bd83aa640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd006bf50 con 0x7f7bd0159f40 2023-11-28T18:25:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.245+0000 7f7bd83aa640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd00cb3c0 con 0x7f7bd00ad850 2023-11-28T18:25:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.245+0000 7f7bd83aa640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd015a320 con 0x7f7bd00caf60 2023-11-28T18:25:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.245+0000 7f7bd6146640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7bd0159f40 0x7f7bd00ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55844/0 (socket says 172.21.15.105:55844) 2023-11-28T18:25:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.245+0000 7f7bd6146640 1 -- 172.21.15.105:0/2828601413 learned_addr learned my addr 172.21.15.105:0/2828601413 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.245+0000 7f7bd6947640 1 --2- 172.21.15.105:0/2828601413 >> v2:172.21.15.105:3300/0 conn(0x7f7bd00ad850 0x7f7bd00701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.245+0000 7f7bd5945640 1 --2- 172.21.15.105:0/2828601413 >> v2:172.21.15.148:3300/0 conn(0x7f7bd00caf60 0x7f7bd00cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.245+0000 7f7bd5144640 1 -- 172.21.15.105:0/2828601413 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1576445532 0 0) 0x7f7bd006bf50 con 0x7f7bd0159f40 2023-11-28T18:25:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.245+0000 7f7bd5144640 1 -- 172.21.15.105:0/2828601413 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bbc003530 con 0x7f7bd0159f40 2023-11-28T18:25:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.245+0000 7f7bd6947640 1 -- 172.21.15.105:0/2828601413 >> v2:172.21.15.148:3300/0 conn(0x7f7bd00caf60 msgr2=0x7f7bd00cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.245+0000 7f7bd6947640 1 --2- 172.21.15.105:0/2828601413 >> v2:172.21.15.148:3300/0 conn(0x7f7bd00caf60 0x7f7bd00cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.245+0000 7f7bd6947640 1 -- 172.21.15.105:0/2828601413 >> v1:172.21.15.105:6789/0 conn(0x7f7bd0159f40 legacy=0x7f7bd00ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.245+0000 7f7bd6947640 1 -- 172.21.15.105:0/2828601413 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd0071380 con 0x7f7bd00ad850 2023-11-28T18:25:49.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.249+0000 7f7bd6947640 1 --2- 172.21.15.105:0/2828601413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd00ad850 0x7f7bd00701d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7bd009d0d0 tx=0x7f7bc801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2dfb4ccae6e325 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:49.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.249+0000 7f7bd5144640 1 -- 172.21.15.105:0/2828601413 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc80a5910 con 0x7f7bd00ad850 2023-11-28T18:25:49.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.249+0000 7f7bd5144640 1 -- 172.21.15.105:0/2828601413 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc80a5a70 con 0x7f7bd00ad850 2023-11-28T18:25:49.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.249+0000 7f7bd5144640 1 -- 172.21.15.105:0/2828601413 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc80a5e00 con 0x7f7bd00ad850 2023-11-28T18:25:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.249+0000 7f7bd83aa640 1 -- 172.21.15.105:0/2828601413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd00ad850 msgr2=0x7f7bd00701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.249+0000 7f7bd83aa640 1 --2- 172.21.15.105:0/2828601413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd00ad850 0x7f7bd00701d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7bd009d0d0 tx=0x7f7bc801c850 comp rx=0 tx=0).stop 2023-11-28T18:25:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.249+0000 7f7bd83aa640 1 -- 172.21.15.105:0/2828601413 shutdown_connections 2023-11-28T18:25:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.249+0000 7f7bd83aa640 1 --2- 172.21.15.105:0/2828601413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd00ad850 0x7f7bd00701d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.249+0000 7f7bd83aa640 1 --2- 172.21.15.105:0/2828601413 >> v2:172.21.15.148:3300/0 conn(0x7f7bd00caf60 0x7f7bd00cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.249+0000 7f7bd83aa640 1 -- 172.21.15.105:0/2828601413 >> 172.21.15.105:0/2828601413 conn(0x7f7bd00ceb90 msgr2=0x7f7bd00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.249+0000 7f7bd83aa640 1 -- 172.21.15.105:0/2828601413 shutdown_connections 2023-11-28T18:25:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.249+0000 7f7bd83aa640 1 -- 172.21.15.105:0/2828601413 wait complete. 2023-11-28T18:25:49.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bd83aa640 1 Processor -- start 2023-11-28T18:25:49.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bd83aa640 1 -- start start 2023-11-28T18:25:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bd83aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7bd00ad850 0x7f7bd0145960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bd83aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd00caf60 0x7f7bd014df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bd83aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd0159f40 0x7f7bd014e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bd83aa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd015d3b0 con 0x7f7bd0159f40 2023-11-28T18:25:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bd83aa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd0141030 con 0x7f7bd00caf60 2023-11-28T18:25:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bd83aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd015cb10 con 0x7f7bd00ad850 2023-11-28T18:25:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bd6947640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd0159f40 0x7f7bd014e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bd6947640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd0159f40 0x7f7bd014e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48376/0 (socket says 172.21.15.105:48376) 2023-11-28T18:25:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bd6947640 1 -- 172.21.15.105:0/1593334071 learned_addr learned my addr 172.21.15.105:0/1593334071 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:49.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bd6146640 1 --2- 172.21.15.105:0/1593334071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7bd00ad850 0x7f7bd0145960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:49.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bd5945640 1 --2- 172.21.15.105:0/1593334071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd00caf60 0x7f7bd014df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:49.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bd6947640 1 -- 172.21.15.105:0/1593334071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7bd00ad850 msgr2=0x7f7bd0145960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:49.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bd6947640 1 --2- 172.21.15.105:0/1593334071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7bd00ad850 0x7f7bd0145960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:49.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bd6947640 1 -- 172.21.15.105:0/1593334071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd00caf60 msgr2=0x7f7bd014df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:49.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bd6947640 1 --2- 172.21.15.105:0/1593334071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd00caf60 0x7f7bd014df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:49.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bd6947640 1 -- 172.21.15.105:0/1593334071 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc8009a70 con 0x7f7bd0159f40 2023-11-28T18:25:49.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bd5945640 1 --2- 172.21.15.105:0/1593334071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd00caf60 0x7f7bd014df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:49.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bd6947640 1 --2- 172.21.15.105:0/1593334071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd0159f40 0x7f7bd014e440 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8019460 tx=0x7f7bc80195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:49.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bc6ffd640 1 -- 172.21.15.105:0/1593334071 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc80b9860 con 0x7f7bd0159f40 2023-11-28T18:25:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bc6ffd640 1 -- 172.21.15.105:0/1593334071 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc80b99c0 con 0x7f7bd0159f40 2023-11-28T18:25:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bc6ffd640 1 -- 172.21.15.105:0/1593334071 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc80b9d70 con 0x7f7bd0159f40 2023-11-28T18:25:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bd83aa640 1 -- 172.21.15.105:0/1593334071 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd015cdf0 con 0x7f7bd0159f40 2023-11-28T18:25:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.253+0000 7f7bd83aa640 1 -- 172.21.15.105:0/1593334071 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd0145ea0 con 0x7f7bd0159f40 2023-11-28T18:25:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.257+0000 7f7bd83aa640 1 -- 172.21.15.105:0/1593334071 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd0146090 con 0x7f7bd0159f40 2023-11-28T18:25:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.257+0000 7f7bc6ffd640 1 -- 172.21.15.105:0/1593334071 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f7bc80ad440 con 0x7f7bd0159f40 2023-11-28T18:25:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.257+0000 7f7bc6ffd640 1 --2- 172.21.15.105:0/1593334071 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7bb40668d0 0x7f7bb4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.257+0000 7f7bc6ffd640 1 -- 172.21.15.105:0/1593334071 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f7bc811fb00 con 0x7f7bd0159f40 2023-11-28T18:25:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.257+0000 7f7bd6146640 1 --2- 172.21.15.105:0/1593334071 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7bb40668d0 0x7f7bb4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.257+0000 7f7bd6146640 1 --2- 172.21.15.105:0/1593334071 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7bb40668d0 0x7f7bb4068d90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0011f0 tx=0x7f7bcc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:49.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.261+0000 7f7bc6ffd640 1 -- 172.21.15.105:0/1593334071 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7bd0146090 con 0x7f7bd0159f40 2023-11-28T18:25:49.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:49.413+0000 7f7bd83aa640 1 -- 172.21.15.105:0/1593334071 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7bd0146090 con 0x7f7bd0159f40 2023-11-28T18:25:50.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.073+0000 7f7bc6ffd640 1 -- 172.21.15.105:0/1593334071 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v42) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7bd0146090 con 0x7f7bd0159f40 2023-11-28T18:25:50.076 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' removed 2023-11-28T18:25:50.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.077+0000 7f7bd83aa640 1 -- 172.21.15.105:0/1593334071 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7bb40668d0 msgr2=0x7f7bb4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:50.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.077+0000 7f7bd83aa640 1 --2- 172.21.15.105:0/1593334071 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7bb40668d0 0x7f7bb4068d90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0011f0 tx=0x7f7bcc009040 comp rx=0 tx=0).stop 2023-11-28T18:25:50.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.077+0000 7f7bd83aa640 1 -- 172.21.15.105:0/1593334071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd0159f40 msgr2=0x7f7bd014e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:50.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.077+0000 7f7bd83aa640 1 --2- 172.21.15.105:0/1593334071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd0159f40 0x7f7bd014e440 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8019460 tx=0x7f7bc80195f0 comp rx=0 tx=0).stop 2023-11-28T18:25:50.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.081+0000 7f7bd83aa640 1 -- 172.21.15.105:0/1593334071 shutdown_connections 2023-11-28T18:25:50.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.081+0000 7f7bd83aa640 1 --2- 172.21.15.105:0/1593334071 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7bb40668d0 0x7f7bb4068d90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:50.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.081+0000 7f7bd83aa640 1 --2- 172.21.15.105:0/1593334071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd0159f40 0x7f7bd014e440 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:50.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.081+0000 7f7bd83aa640 1 --2- 172.21.15.105:0/1593334071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd00caf60 0x7f7bd014df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:50.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.081+0000 7f7bd83aa640 1 --2- 172.21.15.105:0/1593334071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7bd00ad850 0x7f7bd0145960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:50.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.081+0000 7f7bd83aa640 1 -- 172.21.15.105:0/1593334071 >> 172.21.15.105:0/1593334071 conn(0x7f7bd00ceb90 msgr2=0x7f7bd00ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:50.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.081+0000 7f7bd83aa640 1 -- 172.21.15.105:0/1593334071 shutdown_connections 2023-11-28T18:25:50.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.081+0000 7f7bd83aa640 1 -- 172.21.15.105:0/1593334071 wait complete. 2023-11-28T18:25:50.112 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T18:25:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.253+0000 7f37eaa17640 1 Processor -- start 2023-11-28T18:25:50.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.253+0000 7f37eaa17640 1 -- start start 2023-11-28T18:25:50.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.257+0000 7f37eaa17640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f37e40b4290 0x7f37e40b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:50.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.257+0000 7f37eaa17640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e40b4bb0 con 0x7f37e415b030 2023-11-28T18:25:50.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.257+0000 7f37eaa17640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e40701d0 con 0x7f37e4071960 2023-11-28T18:25:50.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.257+0000 7f37eaa17640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37e406df80 con 0x7f37e40b4290 2023-11-28T18:25:50.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.257+0000 7f37e3fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f37e40b4290 0x7f37e40b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:50.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.257+0000 7f37e8fb4640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f37e415b030 0x7f37e40709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55856/0 (socket says 172.21.15.105:55856) 2023-11-28T18:25:50.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.257+0000 7f37e8fb4640 1 -- 172.21.15.105:0/3632234817 learned_addr learned my addr 172.21.15.105:0/3632234817 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:50.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.257+0000 7f37e2ffd640 1 -- 172.21.15.105:0/3632234817 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355622946 0 0) 0x7f37e40b4bb0 con 0x7f37e415b030 2023-11-28T18:25:50.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.257+0000 7f37e2ffd640 1 -- 172.21.15.105:0/3632234817 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37cc003530 con 0x7f37e415b030 2023-11-28T18:25:50.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.257+0000 7f37e2ffd640 1 -- 172.21.15.105:0/3632234817 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 411908066 0 0) 0x7f37e40701d0 con 0x7f37e4071960 2023-11-28T18:25:50.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.257+0000 7f37e2ffd640 1 -- 172.21.15.105:0/3632234817 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37e40b4bb0 con 0x7f37e4071960 2023-11-28T18:25:50.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.257+0000 7f37e3fff640 1 -- 172.21.15.105:0/3632234817 >> v1:172.21.15.148:6789/0 conn(0x7f37e4071960 legacy=0x7f37e415f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:50.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.257+0000 7f37e3fff640 1 -- 172.21.15.105:0/3632234817 >> v1:172.21.15.105:6789/0 conn(0x7f37e415b030 legacy=0x7f37e40709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:50.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.257+0000 7f37e3fff640 1 -- 172.21.15.105:0/3632234817 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e4071df0 con 0x7f37e40b4290 2023-11-28T18:25:50.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.261+0000 7f37e3fff640 1 --2- 172.21.15.105:0/3632234817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37e40b4290 0x7f37e40b4670 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f37d40097c0 tx=0x7f37d401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734634a63f41c14 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:50.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.261+0000 7f37e2ffd640 1 -- 172.21.15.105:0/3632234817 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d40ab650 con 0x7f37e40b4290 2023-11-28T18:25:50.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.261+0000 7f37e2ffd640 1 -- 172.21.15.105:0/3632234817 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d40ab7b0 con 0x7f37e40b4290 2023-11-28T18:25:50.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.261+0000 7f37e2ffd640 1 -- 172.21.15.105:0/3632234817 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d40abb40 con 0x7f37e40b4290 2023-11-28T18:25:50.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.261+0000 7f37eaa17640 1 -- 172.21.15.105:0/3632234817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37e40b4290 msgr2=0x7f37e40b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:50.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.261+0000 7f37eaa17640 1 --2- 172.21.15.105:0/3632234817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37e40b4290 0x7f37e40b4670 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f37d40097c0 tx=0x7f37d401c450 comp rx=0 tx=0).stop 2023-11-28T18:25:50.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.261+0000 7f37eaa17640 1 -- 172.21.15.105:0/3632234817 shutdown_connections 2023-11-28T18:25:50.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.261+0000 7f37eaa17640 1 --2- 172.21.15.105:0/3632234817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37e40b4290 0x7f37e40b4670 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:50.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.261+0000 7f37eaa17640 1 -- 172.21.15.105:0/3632234817 >> 172.21.15.105:0/3632234817 conn(0x7f37e40ceb90 msgr2=0x7f37e40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:50.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.261+0000 7f37eaa17640 1 -- 172.21.15.105:0/3632234817 shutdown_connections 2023-11-28T18:25:50.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.261+0000 7f37eaa17640 1 -- 172.21.15.105:0/3632234817 wait complete. 2023-11-28T18:25:50.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.261+0000 7f37eaa17640 1 Processor -- start 2023-11-28T18:25:50.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.261+0000 7f37eaa17640 1 -- start start 2023-11-28T18:25:50.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37eaa17640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37e40b4290 0x7f37e4155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:50.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37eaa17640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37e415b030 0x7f37e4147d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:50.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37eaa17640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f37e40ade10 0x7f37e40ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:50.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37eaa17640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e40ae7e0 con 0x7f37e415b030 2023-11-28T18:25:50.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37eaa17640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e409a430 con 0x7f37e40b4290 2023-11-28T18:25:50.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37eaa17640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e41556e0 con 0x7f37e40ade10 2023-11-28T18:25:50.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37e37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37e415b030 0x7f37e4147d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:50.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37e3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37e40b4290 0x7f37e4155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:50.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37e37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37e415b030 0x7f37e4147d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48394/0 (socket says 172.21.15.105:48394) 2023-11-28T18:25:50.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37e37fe640 1 -- 172.21.15.105:0/1486443461 learned_addr learned my addr 172.21.15.105:0/1486443461 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:50.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37e8fb4640 1 --2- 172.21.15.105:0/1486443461 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f37e40ade10 0x7f37e40ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:50.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37e37fe640 1 -- 172.21.15.105:0/1486443461 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f37e40ade10 msgr2=0x7f37e40ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:50.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37e37fe640 1 --2- 172.21.15.105:0/1486443461 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f37e40ade10 0x7f37e40ae2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:50.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37e37fe640 1 -- 172.21.15.105:0/1486443461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37e40b4290 msgr2=0x7f37e4155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:50.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37e37fe640 1 --2- 172.21.15.105:0/1486443461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37e40b4290 0x7f37e4155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:50.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37e37fe640 1 -- 172.21.15.105:0/1486443461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d4009420 con 0x7f37e415b030 2023-11-28T18:25:50.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37e37fe640 1 --2- 172.21.15.105:0/1486443461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37e415b030 0x7f37e4147d50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f37d800c490 tx=0x7f37d800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:50.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37e0ff9640 1 -- 172.21.15.105:0/1486443461 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d80052d0 con 0x7f37e415b030 2023-11-28T18:25:50.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37e0ff9640 1 -- 172.21.15.105:0/1486443461 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d8005430 con 0x7f37e415b030 2023-11-28T18:25:50.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37e0ff9640 1 -- 172.21.15.105:0/1486443461 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d8012870 con 0x7f37e415b030 2023-11-28T18:25:50.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37eaa17640 1 -- 172.21.15.105:0/1486443461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e41560b0 con 0x7f37e415b030 2023-11-28T18:25:50.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37eaa17640 1 -- 172.21.15.105:0/1486443461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e409c4c0 con 0x7f37e415b030 2023-11-28T18:25:50.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.265+0000 7f37eaa17640 1 -- 172.21.15.105:0/1486443461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37e4155870 con 0x7f37e415b030 2023-11-28T18:25:50.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.269+0000 7f37e0ff9640 1 -- 172.21.15.105:0/1486443461 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f37d80129d0 con 0x7f37e415b030 2023-11-28T18:25:50.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.269+0000 7f37e0ff9640 1 --2- 172.21.15.105:0/1486443461 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f37d00668d0 0x7f37d0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:50.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.269+0000 7f37e0ff9640 1 -- 172.21.15.105:0/1486443461 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5878+0+0 (secure 0 0 0) 0x7f37d8083f80 con 0x7f37e415b030 2023-11-28T18:25:50.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.269+0000 7f37e3fff640 1 --2- 172.21.15.105:0/1486443461 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f37d00668d0 0x7f37d0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:50.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.269+0000 7f37e3fff640 1 --2- 172.21.15.105:0/1486443461 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f37d00668d0 0x7f37d0068d90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f37d40af000 tx=0x7f37d40a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:50.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.273+0000 7f37e0ff9640 1 -- 172.21.15.105:0/1486443461 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f37d80b5280 con 0x7f37e415b030 2023-11-28T18:25:50.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:50.425+0000 7f37eaa17640 1 -- 172.21.15.105:0/1486443461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f37e4067570 con 0x7f37e415b030 2023-11-28T18:25:51.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.077+0000 7f37e0ff9640 1 -- 172.21.15.105:0/1486443461 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v43) v1 ==== 169+0+0 (secure 0 0 0) 0x7f37d80511c0 con 0x7f37e415b030 2023-11-28T18:25:51.078 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' removed 2023-11-28T18:25:51.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.077+0000 7f37eaa17640 1 -- 172.21.15.105:0/1486443461 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f37d00668d0 msgr2=0x7f37d0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:51.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.077+0000 7f37eaa17640 1 --2- 172.21.15.105:0/1486443461 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f37d00668d0 0x7f37d0068d90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f37d40af000 tx=0x7f37d40a7000 comp rx=0 tx=0).stop 2023-11-28T18:25:51.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.077+0000 7f37eaa17640 1 -- 172.21.15.105:0/1486443461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37e415b030 msgr2=0x7f37e4147d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:51.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.077+0000 7f37eaa17640 1 --2- 172.21.15.105:0/1486443461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37e415b030 0x7f37e4147d50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f37d800c490 tx=0x7f37d800c950 comp rx=0 tx=0).stop 2023-11-28T18:25:51.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.077+0000 7f37eaa17640 1 -- 172.21.15.105:0/1486443461 shutdown_connections 2023-11-28T18:25:51.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.077+0000 7f37eaa17640 1 --2- 172.21.15.105:0/1486443461 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f37e40ade10 0x7f37e40ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.077+0000 7f37eaa17640 1 --2- 172.21.15.105:0/1486443461 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f37d00668d0 0x7f37d0068d90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.077+0000 7f37eaa17640 1 --2- 172.21.15.105:0/1486443461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37e415b030 0x7f37e4147d50 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.077+0000 7f37eaa17640 1 --2- 172.21.15.105:0/1486443461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37e40b4290 0x7f37e4155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.077+0000 7f37eaa17640 1 -- 172.21.15.105:0/1486443461 >> 172.21.15.105:0/1486443461 conn(0x7f37e40ceb90 msgr2=0x7f37e40afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:51.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.077+0000 7f37eaa17640 1 -- 172.21.15.105:0/1486443461 shutdown_connections 2023-11-28T18:25:51.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.077+0000 7f37eaa17640 1 -- 172.21.15.105:0/1486443461 wait complete. 2023-11-28T18:25:51.116 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random 2023-11-28T18:25:51.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.253+0000 7f2623b72640 1 Processor -- start 2023-11-28T18:25:51.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.253+0000 7f2623b72640 1 -- start start 2023-11-28T18:25:51.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.257+0000 7f2623b72640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f261c06cef0 0x7f261c0abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:51.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.257+0000 7f2623b72640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f261c0cde30 con 0x7f261c159610 2023-11-28T18:25:51.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.257+0000 7f2623b72640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f261c1599f0 con 0x7f261c0ce2c0 2023-11-28T18:25:51.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.257+0000 7f2623b72640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f261c06de20 con 0x7f261c06cef0 2023-11-28T18:25:51.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.257+0000 7f262190e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f261c159610 0x7f261c0cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39038/0 (socket says 172.21.15.105:39038) 2023-11-28T18:25:51.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.257+0000 7f262190e640 1 -- 172.21.15.105:0/3440411947 learned_addr learned my addr 172.21.15.105:0/3440411947 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:51.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.257+0000 7f262210f640 1 --2- 172.21.15.105:0/3440411947 >> v2:172.21.15.148:3300/0 conn(0x7f261c06cef0 0x7f261c0abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:51.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.257+0000 7f262090c640 1 -- 172.21.15.105:0/3440411947 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 125381586 0 0) 0x7f261c0cde30 con 0x7f261c159610 2023-11-28T18:25:51.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.257+0000 7f262090c640 1 -- 172.21.15.105:0/3440411947 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2608003450 con 0x7f261c159610 2023-11-28T18:25:51.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.257+0000 7f262210f640 1 -- 172.21.15.105:0/3440411947 >> v1:172.21.15.148:6789/0 conn(0x7f261c0ce2c0 legacy=0x7f261c06c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:51.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.257+0000 7f262090c640 1 -- 172.21.15.105:0/3440411947 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1003848301 0 0) 0x7f261c1599f0 con 0x7f261c0ce2c0 2023-11-28T18:25:51.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.257+0000 7f262210f640 1 -- 172.21.15.105:0/3440411947 >> v1:172.21.15.105:6789/0 conn(0x7f261c159610 legacy=0x7f261c0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:51.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.257+0000 7f262210f640 1 -- 172.21.15.105:0/3440411947 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c0cd220 con 0x7f261c06cef0 2023-11-28T18:25:51.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.261+0000 7f262210f640 1 --2- 172.21.15.105:0/3440411947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f261c06cef0 0x7f261c0abd80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2614009d90 tx=0x7f261401c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca8d4bb9da9b7206 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:51.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.261+0000 7f262090c640 1 -- 172.21.15.105:0/3440411947 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26140a5960 con 0x7f261c06cef0 2023-11-28T18:25:51.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.261+0000 7f262090c640 1 -- 172.21.15.105:0/3440411947 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26140a5ac0 con 0x7f261c06cef0 2023-11-28T18:25:51.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.261+0000 7f262090c640 1 -- 172.21.15.105:0/3440411947 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26140a5e50 con 0x7f261c06cef0 2023-11-28T18:25:51.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.261+0000 7f2623b72640 1 -- 172.21.15.105:0/3440411947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f261c06cef0 msgr2=0x7f261c0abd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:51.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.261+0000 7f2623b72640 1 --2- 172.21.15.105:0/3440411947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f261c06cef0 0x7f261c0abd80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2614009d90 tx=0x7f261401c920 comp rx=0 tx=0).stop 2023-11-28T18:25:51.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.261+0000 7f2623b72640 1 -- 172.21.15.105:0/3440411947 shutdown_connections 2023-11-28T18:25:51.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.261+0000 7f2623b72640 1 --2- 172.21.15.105:0/3440411947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f261c06cef0 0x7f261c0abd80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.261+0000 7f2623b72640 1 -- 172.21.15.105:0/3440411947 >> 172.21.15.105:0/3440411947 conn(0x7f261c0ceb90 msgr2=0x7f261c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:51.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.261+0000 7f2623b72640 1 -- 172.21.15.105:0/3440411947 shutdown_connections 2023-11-28T18:25:51.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.261+0000 7f2623b72640 1 -- 172.21.15.105:0/3440411947 wait complete. 2023-11-28T18:25:51.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.261+0000 7f2623b72640 1 Processor -- start 2023-11-28T18:25:51.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.261+0000 7f2623b72640 1 -- start start 2023-11-28T18:25:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f2623b72640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f261c06cef0 0x7f261c149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f2623b72640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f261c0ce2c0 0x7f261c14eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f2623b72640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f261c159610 0x7f261c14c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f2623b72640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261c14d990 con 0x7f261c06cef0 2023-11-28T18:25:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f2623b72640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261c14cd80 con 0x7f261c159610 2023-11-28T18:25:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f2623b72640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261c14d060 con 0x7f261c0ce2c0 2023-11-28T18:25:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f262190e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f261c06cef0 0x7f261c149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f262190e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f261c06cef0 0x7f261c149860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55224/0 (socket says 172.21.15.105:55224) 2023-11-28T18:25:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f262190e640 1 -- 172.21.15.105:0/3547071769 learned_addr learned my addr 172.21.15.105:0/3547071769 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f262210f640 1 --2- 172.21.15.105:0/3547071769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f261c159610 0x7f261c14c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f262110d640 1 --2- 172.21.15.105:0/3547071769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f261c0ce2c0 0x7f261c14eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f262190e640 1 -- 172.21.15.105:0/3547071769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f261c0ce2c0 msgr2=0x7f261c14eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f262190e640 1 --2- 172.21.15.105:0/3547071769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f261c0ce2c0 0x7f261c14eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f262190e640 1 -- 172.21.15.105:0/3547071769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f261c159610 msgr2=0x7f261c14c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f262190e640 1 --2- 172.21.15.105:0/3547071769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f261c159610 0x7f261c14c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f262190e640 1 -- 172.21.15.105:0/3547071769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2614009a70 con 0x7f261c06cef0 2023-11-28T18:25:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f262190e640 1 --2- 172.21.15.105:0/3547071769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f261c06cef0 0x7f261c149860 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f261800c3c0 tx=0x7f261800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f26127fc640 1 -- 172.21.15.105:0/3547071769 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618005cf0 con 0x7f261c06cef0 2023-11-28T18:25:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f26127fc640 1 -- 172.21.15.105:0/3547071769 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2618005e50 con 0x7f261c06cef0 2023-11-28T18:25:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f26127fc640 1 -- 172.21.15.105:0/3547071769 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26180072f0 con 0x7f261c06cef0 2023-11-28T18:25:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f2623b72640 1 -- 172.21.15.105:0/3547071769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f261c0ccff0 con 0x7f261c06cef0 2023-11-28T18:25:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f2623b72640 1 -- 172.21.15.105:0/3547071769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f261c14d620 con 0x7f261c06cef0 2023-11-28T18:25:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f2623b72640 1 -- 172.21.15.105:0/3547071769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f261c067570 con 0x7f261c06cef0 2023-11-28T18:25:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.265+0000 7f26127fc640 1 -- 172.21.15.105:0/3547071769 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f2618007450 con 0x7f261c06cef0 2023-11-28T18:25:51.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.269+0000 7f26127fc640 1 --2- 172.21.15.105:0/3547071769 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2600066900 0x7f2600068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:51.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.269+0000 7f26127fc640 1 -- 172.21.15.105:0/3547071769 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f2618083810 con 0x7f261c06cef0 2023-11-28T18:25:51.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.269+0000 7f262110d640 1 --2- 172.21.15.105:0/3547071769 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2600066900 0x7f2600068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:51.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.269+0000 7f262110d640 1 --2- 172.21.15.105:0/3547071769 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2600066900 0x7f2600068dc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f261c158f60 tx=0x7f260c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.273+0000 7f26127fc640 1 -- 172.21.15.105:0/3547071769 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2618050c10 con 0x7f261c06cef0 2023-11-28T18:25:51.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.425+0000 7f2623b72640 1 -- 172.21.15.105:0/3547071769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"} v 0) v1 -- 0x7f261c15ac80 con 0x7f261c06cef0 2023-11-28T18:25:51.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.429+0000 7f26127fc640 1 -- 172.21.15.105:0/3547071769 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"}]=0 v15) v1 ==== 110+0+0 (secure 0 0 0) 0x7f2618017070 con 0x7f261c06cef0 2023-11-28T18:25:51.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.429+0000 7f2623b72640 1 -- 172.21.15.105:0/3547071769 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2600066900 msgr2=0x7f2600068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:51.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.429+0000 7f2623b72640 1 --2- 172.21.15.105:0/3547071769 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2600066900 0x7f2600068dc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f261c158f60 tx=0x7f260c009040 comp rx=0 tx=0).stop 2023-11-28T18:25:51.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.429+0000 7f2623b72640 1 -- 172.21.15.105:0/3547071769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f261c06cef0 msgr2=0x7f261c149860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:51.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.429+0000 7f2623b72640 1 --2- 172.21.15.105:0/3547071769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f261c06cef0 0x7f261c149860 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f261800c3c0 tx=0x7f261800c880 comp rx=0 tx=0).stop 2023-11-28T18:25:51.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.429+0000 7f2623b72640 1 -- 172.21.15.105:0/3547071769 shutdown_connections 2023-11-28T18:25:51.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.429+0000 7f2623b72640 1 --2- 172.21.15.105:0/3547071769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f261c159610 0x7f261c14c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.429+0000 7f2623b72640 1 --2- 172.21.15.105:0/3547071769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f261c0ce2c0 0x7f261c14eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.429+0000 7f2623b72640 1 --2- 172.21.15.105:0/3547071769 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2600066900 0x7f2600068dc0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.429+0000 7f2623b72640 1 --2- 172.21.15.105:0/3547071769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f261c06cef0 0x7f261c149860 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.429+0000 7f2623b72640 1 -- 172.21.15.105:0/3547071769 >> 172.21.15.105:0/3547071769 conn(0x7f261c0ceb90 msgr2=0x7f261c153770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:51.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.429+0000 7f2623b72640 1 -- 172.21.15.105:0/3547071769 shutdown_connections 2023-11-28T18:25:51.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.429+0000 7f2623b72640 1 -- 172.21.15.105:0/3547071769 wait complete. 2023-11-28T18:25:51.450 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T18:25:51.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.589+0000 7fbefc69a640 1 Processor -- start 2023-11-28T18:25:51.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.589+0000 7fbefc69a640 1 -- start start 2023-11-28T18:25:51.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.589+0000 7fbefc69a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbef4159f40 0x7fbef40caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:51.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.589+0000 7fbefc69a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbef40ad850 0x7fbef40701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:51.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.589+0000 7fbefc69a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef406bf50 con 0x7fbef40cc760 2023-11-28T18:25:51.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.589+0000 7fbefc69a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef40ccbc0 con 0x7fbef4159f40 2023-11-28T18:25:51.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.589+0000 7fbefc69a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef415a320 con 0x7fbef40ad850 2023-11-28T18:25:51.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.589+0000 7fbefa436640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbef4159f40 0x7fbef40caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:51.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.589+0000 7fbefa436640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbef4159f40 0x7fbef40caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55230/0 (socket says 172.21.15.105:55230) 2023-11-28T18:25:51.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.589+0000 7fbefa436640 1 -- 172.21.15.105:0/1602663147 learned_addr learned my addr 172.21.15.105:0/1602663147 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:51.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.589+0000 7fbefac37640 1 --2- 172.21.15.105:0/1602663147 >> v2:172.21.15.148:3300/0 conn(0x7fbef40ad850 0x7fbef40701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:51.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.589+0000 7fbefa436640 1 -- 172.21.15.105:0/1602663147 >> v2:172.21.15.148:3300/0 conn(0x7fbef40ad850 msgr2=0x7fbef40701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:51.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.589+0000 7fbefa436640 1 --2- 172.21.15.105:0/1602663147 >> v2:172.21.15.148:3300/0 conn(0x7fbef40ad850 0x7fbef40701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.589+0000 7fbefa436640 1 -- 172.21.15.105:0/1602663147 >> v1:172.21.15.148:6789/0 conn(0x7fbef40cc760 legacy=0x7fbef40ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:51.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.589+0000 7fbefa436640 1 -- 172.21.15.105:0/1602663147 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef4071380 con 0x7fbef4159f40 2023-11-28T18:25:51.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.593+0000 7fbefa436640 1 --2- 172.21.15.105:0/1602663147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbef4159f40 0x7fbef40caf60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbef40b0850 tx=0x7fbef001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e350eed69fa088cf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:51.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.593+0000 7fbef9434640 1 -- 172.21.15.105:0/1602663147 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef00ab600 con 0x7fbef4159f40 2023-11-28T18:25:51.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.593+0000 7fbef9434640 1 -- 172.21.15.105:0/1602663147 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef00ab760 con 0x7fbef4159f40 2023-11-28T18:25:51.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.593+0000 7fbef9434640 1 -- 172.21.15.105:0/1602663147 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef00abaf0 con 0x7fbef4159f40 2023-11-28T18:25:51.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.593+0000 7fbefc69a640 1 -- 172.21.15.105:0/1602663147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbef4159f40 msgr2=0x7fbef40caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:51.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.593+0000 7fbefc69a640 1 --2- 172.21.15.105:0/1602663147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbef4159f40 0x7fbef40caf60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbef40b0850 tx=0x7fbef001c400 comp rx=0 tx=0).stop 2023-11-28T18:25:51.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.593+0000 7fbefc69a640 1 -- 172.21.15.105:0/1602663147 shutdown_connections 2023-11-28T18:25:51.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.593+0000 7fbefc69a640 1 --2- 172.21.15.105:0/1602663147 >> v2:172.21.15.148:3300/0 conn(0x7fbef40ad850 0x7fbef40701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.593+0000 7fbefc69a640 1 --2- 172.21.15.105:0/1602663147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbef4159f40 0x7fbef40caf60 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.593+0000 7fbefc69a640 1 -- 172.21.15.105:0/1602663147 >> 172.21.15.105:0/1602663147 conn(0x7fbef40ceb90 msgr2=0x7fbef40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:51.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.593+0000 7fbefc69a640 1 -- 172.21.15.105:0/1602663147 shutdown_connections 2023-11-28T18:25:51.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.593+0000 7fbefc69a640 1 -- 172.21.15.105:0/1602663147 wait complete. 2023-11-28T18:25:51.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbefc69a640 1 Processor -- start 2023-11-28T18:25:51.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbefc69a640 1 -- start start 2023-11-28T18:25:51.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbefc69a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbef40ad850 0x7fbef4155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:51.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbefc69a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbef40cc760 0x7fbef406c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:51.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbefc69a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbef4159f40 0x7fbef406cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:51.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbefc69a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef415a650 con 0x7fbef4159f40 2023-11-28T18:25:51.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbefc69a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef415a8f0 con 0x7fbef40cc760 2023-11-28T18:25:51.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbefc69a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef415abd0 con 0x7fbef40ad850 2023-11-28T18:25:51.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbefac37640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbef4159f40 0x7fbef406cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:51.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbefac37640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbef4159f40 0x7fbef406cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55234/0 (socket says 172.21.15.105:55234) 2023-11-28T18:25:51.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbefac37640 1 -- 172.21.15.105:0/2432840933 learned_addr learned my addr 172.21.15.105:0/2432840933 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:51.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbef9c35640 1 --2- 172.21.15.105:0/2432840933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbef40cc760 0x7fbef406c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:51.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbefa436640 1 --2- 172.21.15.105:0/2432840933 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbef40ad850 0x7fbef4155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:51.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbefac37640 1 -- 172.21.15.105:0/2432840933 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbef40ad850 msgr2=0x7fbef4155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:51.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbefac37640 1 --2- 172.21.15.105:0/2432840933 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbef40ad850 0x7fbef4155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbefac37640 1 -- 172.21.15.105:0/2432840933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbef40cc760 msgr2=0x7fbef406c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:51.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbefac37640 1 --2- 172.21.15.105:0/2432840933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbef40cc760 0x7fbef406c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbefac37640 1 -- 172.21.15.105:0/2432840933 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef00094d0 con 0x7fbef4159f40 2023-11-28T18:25:51.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbefac37640 1 --2- 172.21.15.105:0/2432840933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbef4159f40 0x7fbef406cf20 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbeec007e60 tx=0x7fbeec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbeeaffd640 1 -- 172.21.15.105:0/2432840933 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec014420 con 0x7fbef4159f40 2023-11-28T18:25:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbeeaffd640 1 -- 172.21.15.105:0/2432840933 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec0145a0 con 0x7fbef4159f40 2023-11-28T18:25:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbefc69a640 1 -- 172.21.15.105:0/2432840933 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef415aeb0 con 0x7fbef4159f40 2023-11-28T18:25:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbeeaffd640 1 -- 172.21.15.105:0/2432840933 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec014950 con 0x7fbef4159f40 2023-11-28T18:25:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.597+0000 7fbefc69a640 1 -- 172.21.15.105:0/2432840933 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef41fa250 con 0x7fbef4159f40 2023-11-28T18:25:51.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.601+0000 7fbefc69a640 1 -- 172.21.15.105:0/2432840933 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef415c550 con 0x7fbef4159f40 2023-11-28T18:25:51.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.601+0000 7fbeeaffd640 1 -- 172.21.15.105:0/2432840933 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fbeec014ab0 con 0x7fbef4159f40 2023-11-28T18:25:51.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.601+0000 7fbeeaffd640 1 --2- 172.21.15.105:0/2432840933 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbed80668d0 0x7fbed8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:51.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.601+0000 7fbeeaffd640 1 -- 172.21.15.105:0/2432840933 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fbeec084b30 con 0x7fbef4159f40 2023-11-28T18:25:51.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.601+0000 7fbefa436640 1 --2- 172.21.15.105:0/2432840933 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbed80668d0 0x7fbed8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:51.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.601+0000 7fbefa436640 1 --2- 172.21.15.105:0/2432840933 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbed80668d0 0x7fbed8068d90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbef00af000 tx=0x7fbef00b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:51.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.605+0000 7fbeeaffd640 1 -- 172.21.15.105:0/2432840933 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbeec051f20 con 0x7fbef4159f40 2023-11-28T18:25:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.757+0000 7fbefc69a640 1 -- 172.21.15.105:0/2432840933 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fbef415b0a0 con 0x7fbef4159f40 2023-11-28T18:25:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.761+0000 7fbeeaffd640 1 -- 172.21.15.105:0/2432840933 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v16) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbeec055540 con 0x7fbef4159f40 2023-11-28T18:25:51.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.761+0000 7fbefc69a640 1 -- 172.21.15.105:0/2432840933 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbed80668d0 msgr2=0x7fbed8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:51.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.761+0000 7fbefc69a640 1 --2- 172.21.15.105:0/2432840933 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbed80668d0 0x7fbed8068d90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbef00af000 tx=0x7fbef00b6040 comp rx=0 tx=0).stop 2023-11-28T18:25:51.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.761+0000 7fbefc69a640 1 -- 172.21.15.105:0/2432840933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbef4159f40 msgr2=0x7fbef406cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:51.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.761+0000 7fbefc69a640 1 --2- 172.21.15.105:0/2432840933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbef4159f40 0x7fbef406cf20 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbeec007e60 tx=0x7fbeec00e4b0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.765+0000 7fbefc69a640 1 -- 172.21.15.105:0/2432840933 shutdown_connections 2023-11-28T18:25:51.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.765+0000 7fbefc69a640 1 --2- 172.21.15.105:0/2432840933 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbed80668d0 0x7fbed8068d90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.765+0000 7fbefc69a640 1 --2- 172.21.15.105:0/2432840933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbef4159f40 0x7fbef406cf20 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.765+0000 7fbefc69a640 1 --2- 172.21.15.105:0/2432840933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbef40cc760 0x7fbef406c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.765+0000 7fbefc69a640 1 --2- 172.21.15.105:0/2432840933 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbef40ad850 0x7fbef4155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.765+0000 7fbefc69a640 1 -- 172.21.15.105:0/2432840933 >> 172.21.15.105:0/2432840933 conn(0x7fbef40ceb90 msgr2=0x7fbef40ccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:51.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.765+0000 7fbefc69a640 1 -- 172.21.15.105:0/2432840933 shutdown_connections 2023-11-28T18:25:51.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.765+0000 7fbefc69a640 1 -- 172.21.15.105:0/2432840933 wait complete. 2023-11-28T18:25:51.784 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_distributed 2023-11-28T18:25:51.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.921+0000 7f0164be6640 1 Processor -- start 2023-11-28T18:25:51.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.921+0000 7f0164be6640 1 -- start start 2023-11-28T18:25:51.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.921+0000 7f0164be6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01600ce2c0 0x7f016006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:51.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.921+0000 7f0164be6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f016006cd20 0x7f01600abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:51.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.921+0000 7f0164be6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01600cde30 con 0x7f0160159610 2023-11-28T18:25:51.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.921+0000 7f0164be6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f016006d1b0 con 0x7f016006cd20 2023-11-28T18:25:51.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.921+0000 7f0164be6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01601599f0 con 0x7f01600ce2c0 2023-11-28T18:25:51.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.925+0000 7f015ed9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f016006cd20 0x7f01600abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:51.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.925+0000 7f0157fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01600ce2c0 0x7f016006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:51.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.925+0000 7f015e59c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0160159610 0x7f01600cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:49884/0 (socket says 172.21.15.105:49884) 2023-11-28T18:25:51.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.925+0000 7f015e59c640 1 -- 172.21.15.105:0/398405314 learned_addr learned my addr 172.21.15.105:0/398405314 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:51.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.925+0000 7f015ed9d640 1 -- 172.21.15.105:0/398405314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01600ce2c0 msgr2=0x7f016006c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:51.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.925+0000 7f015ed9d640 1 --2- 172.21.15.105:0/398405314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01600ce2c0 0x7f016006c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.925+0000 7f015ed9d640 1 -- 172.21.15.105:0/398405314 >> v1:172.21.15.148:6789/0 conn(0x7f0160159610 legacy=0x7f01600cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:51.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.925+0000 7f015ed9d640 1 -- 172.21.15.105:0/398405314 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016006de20 con 0x7f016006cd20 2023-11-28T18:25:51.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.925+0000 7f0157fff640 1 --2- 172.21.15.105:0/398405314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01600ce2c0 0x7f016006c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:25:51.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.925+0000 7f015ed9d640 1 --2- 172.21.15.105:0/398405314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f016006cd20 0x7f01600abd80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0158009d90 tx=0x7f015801c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffd2a2f33c498d6f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:51.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f015dd9b640 1 -- 172.21.15.105:0/398405314 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01580a5870 con 0x7f016006cd20 2023-11-28T18:25:51.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f015dd9b640 1 -- 172.21.15.105:0/398405314 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01580a59d0 con 0x7f016006cd20 2023-11-28T18:25:51.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f015dd9b640 1 -- 172.21.15.105:0/398405314 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01580a5d60 con 0x7f016006cd20 2023-11-28T18:25:51.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f0164be6640 1 -- 172.21.15.105:0/398405314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f016006cd20 msgr2=0x7f01600abd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:51.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f0164be6640 1 --2- 172.21.15.105:0/398405314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f016006cd20 0x7f01600abd80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0158009d90 tx=0x7f015801c920 comp rx=0 tx=0).stop 2023-11-28T18:25:51.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f0164be6640 1 -- 172.21.15.105:0/398405314 shutdown_connections 2023-11-28T18:25:51.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f0164be6640 1 --2- 172.21.15.105:0/398405314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f016006cd20 0x7f01600abd80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f0164be6640 1 --2- 172.21.15.105:0/398405314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01600ce2c0 0x7f016006c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f0164be6640 1 -- 172.21.15.105:0/398405314 >> 172.21.15.105:0/398405314 conn(0x7f01600ceb90 msgr2=0x7f01600cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:51.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f0164be6640 1 -- 172.21.15.105:0/398405314 shutdown_connections 2023-11-28T18:25:51.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f0164be6640 1 -- 172.21.15.105:0/398405314 wait complete. 2023-11-28T18:25:51.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f0164be6640 1 Processor -- start 2023-11-28T18:25:51.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f0164be6640 1 -- start start 2023-11-28T18:25:51.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f0164be6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f016006cd20 0x7f016009a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f0164be6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01600ce2c0 0x7f01600b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f0164be6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0160159610 0x7f01600b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f0164be6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016009add0 con 0x7f016006cd20 2023-11-28T18:25:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f0164be6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01600b46c0 con 0x7f01600ce2c0 2023-11-28T18:25:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f0164be6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01600b1d50 con 0x7f0160159610 2023-11-28T18:25:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f015e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f016006cd20 0x7f016009a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f015e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f016006cd20 0x7f016009a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55246/0 (socket says 172.21.15.105:55246) 2023-11-28T18:25:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f015e59c640 1 -- 172.21.15.105:0/2872312720 learned_addr learned my addr 172.21.15.105:0/2872312720 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.929+0000 7f0157fff640 1 --2- 172.21.15.105:0/2872312720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01600ce2c0 0x7f01600b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.933+0000 7f015ed9d640 1 --2- 172.21.15.105:0/2872312720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0160159610 0x7f01600b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.933+0000 7f015e59c640 1 -- 172.21.15.105:0/2872312720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0160159610 msgr2=0x7f01600b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.933+0000 7f015e59c640 1 --2- 172.21.15.105:0/2872312720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0160159610 0x7f01600b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.933+0000 7f015e59c640 1 -- 172.21.15.105:0/2872312720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01600ce2c0 msgr2=0x7f01600b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.933+0000 7f015e59c640 1 --2- 172.21.15.105:0/2872312720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01600ce2c0 0x7f01600b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:51.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.933+0000 7f015e59c640 1 -- 172.21.15.105:0/2872312720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014800b6c0 con 0x7f016006cd20 2023-11-28T18:25:51.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.933+0000 7f0157fff640 1 --2- 172.21.15.105:0/2872312720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01600ce2c0 0x7f01600b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:25:51.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.933+0000 7f015e59c640 1 --2- 172.21.15.105:0/2872312720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f016006cd20 0x7f016009a3d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f015000c400 tx=0x7f015000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:51.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.933+0000 7f0156ffd640 1 -- 172.21.15.105:0/2872312720 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0150005cf0 con 0x7f016006cd20 2023-11-28T18:25:51.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.933+0000 7f0156ffd640 1 -- 172.21.15.105:0/2872312720 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0150005e50 con 0x7f016006cd20 2023-11-28T18:25:51.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.933+0000 7f0164be6640 1 -- 172.21.15.105:0/2872312720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0158009a70 con 0x7f016006cd20 2023-11-28T18:25:51.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.933+0000 7f0156ffd640 1 -- 172.21.15.105:0/2872312720 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01500072d0 con 0x7f016006cd20 2023-11-28T18:25:51.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.933+0000 7f0164be6640 1 -- 172.21.15.105:0/2872312720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01600ccff0 con 0x7f016006cd20 2023-11-28T18:25:51.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.933+0000 7f0164be6640 1 -- 172.21.15.105:0/2872312720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0160067570 con 0x7f016006cd20 2023-11-28T18:25:51.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.933+0000 7f0156ffd640 1 -- 172.21.15.105:0/2872312720 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f0150007430 con 0x7f016006cd20 2023-11-28T18:25:51.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.933+0000 7f0156ffd640 1 --2- 172.21.15.105:0/2872312720 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f01400668d0 0x7f0140068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:51.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.933+0000 7f0156ffd640 1 -- 172.21.15.105:0/2872312720 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f0150083730 con 0x7f016006cd20 2023-11-28T18:25:51.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.937+0000 7f0157fff640 1 --2- 172.21.15.105:0/2872312720 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f01400668d0 0x7f0140068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:51.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.937+0000 7f0157fff640 1 --2- 172.21.15.105:0/2872312720 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f01400668d0 0x7f0140068d90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f014800bfd0 tx=0x7f014800b3a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:51.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:51.937+0000 7f0156ffd640 1 -- 172.21.15.105:0/2872312720 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0150050bc0 con 0x7f016006cd20 2023-11-28T18:25:52.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.089+0000 7f0164be6640 1 -- 172.21.15.105:0/2872312720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"} v 0) v1 -- 0x7f0160151e70 con 0x7f016006cd20 2023-11-28T18:25:52.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.101+0000 7f0156ffd640 1 -- 172.21.15.105:0/2872312720 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"}]=0 v17) v1 ==== 115+0+0 (secure 0 0 0) 0x7f01500541e0 con 0x7f016006cd20 2023-11-28T18:25:52.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.105+0000 7f0164be6640 1 -- 172.21.15.105:0/2872312720 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f01400668d0 msgr2=0x7f0140068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:52.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.105+0000 7f0164be6640 1 --2- 172.21.15.105:0/2872312720 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f01400668d0 0x7f0140068d90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f014800bfd0 tx=0x7f014800b3a0 comp rx=0 tx=0).stop 2023-11-28T18:25:52.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.105+0000 7f0164be6640 1 -- 172.21.15.105:0/2872312720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f016006cd20 msgr2=0x7f016009a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:52.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.105+0000 7f0164be6640 1 --2- 172.21.15.105:0/2872312720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f016006cd20 0x7f016009a3d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f015000c400 tx=0x7f015000c8c0 comp rx=0 tx=0).stop 2023-11-28T18:25:52.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.105+0000 7f0164be6640 1 -- 172.21.15.105:0/2872312720 shutdown_connections 2023-11-28T18:25:52.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.105+0000 7f0164be6640 1 --2- 172.21.15.105:0/2872312720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0160159610 0x7f01600b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:52.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.105+0000 7f0164be6640 1 --2- 172.21.15.105:0/2872312720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01600ce2c0 0x7f01600b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:52.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.105+0000 7f0164be6640 1 --2- 172.21.15.105:0/2872312720 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f01400668d0 0x7f0140068d90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:52.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.105+0000 7f0164be6640 1 --2- 172.21.15.105:0/2872312720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f016006cd20 0x7f016009a3d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:52.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.105+0000 7f0164be6640 1 -- 172.21.15.105:0/2872312720 >> 172.21.15.105:0/2872312720 conn(0x7f01600ceb90 msgr2=0x7f01600cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:52.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.105+0000 7f0164be6640 1 -- 172.21.15.105:0/2872312720 shutdown_connections 2023-11-28T18:25:52.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.105+0000 7f0164be6640 1 -- 172.21.15.105:0/2872312720 wait complete. 2023-11-28T18:25:52.125 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random_max 2023-11-28T18:25:52.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.261+0000 7f48f011d640 1 Processor -- start 2023-11-28T18:25:52.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.261+0000 7f48f011d640 1 -- start start 2023-11-28T18:25:52.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.265+0000 7f48f011d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f48e8159f40 0x7f48e80caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:52.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.265+0000 7f48f011d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48e80ad850 0x7f48e80701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:52.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.265+0000 7f48f011d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e806bf50 con 0x7f48e80cc760 2023-11-28T18:25:52.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.265+0000 7f48f011d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e80ccbc0 con 0x7f48e8159f40 2023-11-28T18:25:52.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.265+0000 7f48f011d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e815a320 con 0x7f48e80ad850 2023-11-28T18:25:52.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.265+0000 7f48edeb9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f48e8159f40 0x7f48e80caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:52.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.265+0000 7f48edeb9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f48e8159f40 0x7f48e80caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55260/0 (socket says 172.21.15.105:55260) 2023-11-28T18:25:52.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.265+0000 7f48edeb9640 1 -- 172.21.15.105:0/3542929546 learned_addr learned my addr 172.21.15.105:0/3542929546 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:52.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.265+0000 7f48ee6ba640 1 --2- 172.21.15.105:0/3542929546 >> v2:172.21.15.148:3300/0 conn(0x7f48e80ad850 0x7f48e80701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:52.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.265+0000 7f48edeb9640 1 -- 172.21.15.105:0/3542929546 >> v2:172.21.15.148:3300/0 conn(0x7f48e80ad850 msgr2=0x7f48e80701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:52.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.265+0000 7f48edeb9640 1 --2- 172.21.15.105:0/3542929546 >> v2:172.21.15.148:3300/0 conn(0x7f48e80ad850 0x7f48e80701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:52.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.265+0000 7f48edeb9640 1 -- 172.21.15.105:0/3542929546 >> v1:172.21.15.148:6789/0 conn(0x7f48e80cc760 legacy=0x7f48e80ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:52.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.265+0000 7f48edeb9640 1 -- 172.21.15.105:0/3542929546 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e8071380 con 0x7f48e8159f40 2023-11-28T18:25:52.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.269+0000 7f48edeb9640 1 --2- 172.21.15.105:0/3542929546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48e8159f40 0x7f48e80caf60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f48e80b0850 tx=0x7f48e401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607c1c2fe64e1f71 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:52.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.269+0000 7f48eceb7640 1 -- 172.21.15.105:0/3542929546 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e40ab640 con 0x7f48e8159f40 2023-11-28T18:25:52.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.269+0000 7f48eceb7640 1 -- 172.21.15.105:0/3542929546 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e40a7070 con 0x7f48e8159f40 2023-11-28T18:25:52.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.269+0000 7f48eceb7640 1 -- 172.21.15.105:0/3542929546 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e40ab9c0 con 0x7f48e8159f40 2023-11-28T18:25:52.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.269+0000 7f48f011d640 1 -- 172.21.15.105:0/3542929546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48e8159f40 msgr2=0x7f48e80caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:52.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.269+0000 7f48f011d640 1 --2- 172.21.15.105:0/3542929546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48e8159f40 0x7f48e80caf60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f48e80b0850 tx=0x7f48e401c440 comp rx=0 tx=0).stop 2023-11-28T18:25:52.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.269+0000 7f48f011d640 1 -- 172.21.15.105:0/3542929546 shutdown_connections 2023-11-28T18:25:52.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.269+0000 7f48f011d640 1 --2- 172.21.15.105:0/3542929546 >> v2:172.21.15.148:3300/0 conn(0x7f48e80ad850 0x7f48e80701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:52.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.269+0000 7f48f011d640 1 --2- 172.21.15.105:0/3542929546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48e8159f40 0x7f48e80caf60 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:52.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.269+0000 7f48f011d640 1 -- 172.21.15.105:0/3542929546 >> 172.21.15.105:0/3542929546 conn(0x7f48e80ceb90 msgr2=0x7f48e80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:52.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.269+0000 7f48f011d640 1 -- 172.21.15.105:0/3542929546 shutdown_connections 2023-11-28T18:25:52.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.269+0000 7f48f011d640 1 -- 172.21.15.105:0/3542929546 wait complete. 2023-11-28T18:25:52.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.269+0000 7f48f011d640 1 Processor -- start 2023-11-28T18:25:52.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.269+0000 7f48f011d640 1 -- start start 2023-11-28T18:25:52.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48f011d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48e80ad850 0x7f48e8152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:52.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48f011d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48e80cc760 0x7f48e8151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:52.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48f011d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48e8159f40 0x7f48e81492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:52.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48f011d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e81562d0 con 0x7f48e80ad850 2023-11-28T18:25:52.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48f011d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e809a5e0 con 0x7f48e8159f40 2023-11-28T18:25:52.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48f011d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e8155750 con 0x7f48e80cc760 2023-11-28T18:25:52.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48edeb9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48e80ad850 0x7f48e8152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:52.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48edeb9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48e80ad850 0x7f48e8152f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55264/0 (socket says 172.21.15.105:55264) 2023-11-28T18:25:52.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48edeb9640 1 -- 172.21.15.105:0/1686962151 learned_addr learned my addr 172.21.15.105:0/1686962151 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:52.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48ee6ba640 1 --2- 172.21.15.105:0/1686962151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48e8159f40 0x7f48e81492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:52.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48ed6b8640 1 --2- 172.21.15.105:0/1686962151 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48e80cc760 0x7f48e8151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:52.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48edeb9640 1 -- 172.21.15.105:0/1686962151 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48e80cc760 msgr2=0x7f48e8151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:52.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48edeb9640 1 --2- 172.21.15.105:0/1686962151 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48e80cc760 0x7f48e8151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:52.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48edeb9640 1 -- 172.21.15.105:0/1686962151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48e8159f40 msgr2=0x7f48e81492b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:52.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48edeb9640 1 --2- 172.21.15.105:0/1686962151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48e8159f40 0x7f48e81492b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:52.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48edeb9640 1 -- 172.21.15.105:0/1686962151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e40094d0 con 0x7f48e80ad850 2023-11-28T18:25:52.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48ee6ba640 1 --2- 172.21.15.105:0/1686962151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48e8159f40 0x7f48e81492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:52.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48edeb9640 1 --2- 172.21.15.105:0/1686962151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48e80ad850 0x7f48e8152f80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f48e401c920 tx=0x7f48e401ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:52.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48de7fc640 1 -- 172.21.15.105:0/1686962151 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e40be3f0 con 0x7f48e80ad850 2023-11-28T18:25:52.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48de7fc640 1 -- 172.21.15.105:0/1686962151 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e40be570 con 0x7f48e80ad850 2023-11-28T18:25:52.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48f011d640 1 -- 172.21.15.105:0/1686962151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e8155a90 con 0x7f48e80ad850 2023-11-28T18:25:52.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48de7fc640 1 -- 172.21.15.105:0/1686962151 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e40be920 con 0x7f48e80ad850 2023-11-28T18:25:52.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48f011d640 1 -- 172.21.15.105:0/1686962151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e81553b0 con 0x7f48e80ad850 2023-11-28T18:25:52.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48f011d640 1 -- 172.21.15.105:0/1686962151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e8067570 con 0x7f48e80ad850 2023-11-28T18:25:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.273+0000 7f48de7fc640 1 -- 172.21.15.105:0/1686962151 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f48e40b43f0 con 0x7f48e80ad850 2023-11-28T18:25:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.277+0000 7f48de7fc640 1 --2- 172.21.15.105:0/1686962151 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48cc0668d0 0x7f48cc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.277+0000 7f48de7fc640 1 -- 172.21.15.105:0/1686962151 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f48e411e140 con 0x7f48e80ad850 2023-11-28T18:25:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.277+0000 7f48ed6b8640 1 --2- 172.21.15.105:0/1686962151 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48cc0668d0 0x7f48cc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.277+0000 7f48ed6b8640 1 --2- 172.21.15.105:0/1686962151 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48cc0668d0 0x7f48cc068d90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f48e809cc80 tx=0x7f48d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:52.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.281+0000 7f48de7fc640 1 -- 172.21.15.105:0/1686962151 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f48e40eb430 con 0x7f48e80ad850 2023-11-28T18:25:52.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.429+0000 7f48f011d640 1 -- 172.21.15.105:0/1686962151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"} v 0) v1 -- 0x7f48e80cf950 con 0x7f48e80ad850 2023-11-28T18:25:52.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.437+0000 7f48de7fc640 1 -- 172.21.15.105:0/1686962151 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"}]=0 v18) v1 ==== 114+0+0 (secure 0 0 0) 0x7f48e40eea50 con 0x7f48e80ad850 2023-11-28T18:25:52.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.437+0000 7f48f011d640 1 -- 172.21.15.105:0/1686962151 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48cc0668d0 msgr2=0x7f48cc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:52.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.437+0000 7f48f011d640 1 --2- 172.21.15.105:0/1686962151 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48cc0668d0 0x7f48cc068d90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f48e809cc80 tx=0x7f48d8009040 comp rx=0 tx=0).stop 2023-11-28T18:25:52.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.437+0000 7f48f011d640 1 -- 172.21.15.105:0/1686962151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48e80ad850 msgr2=0x7f48e8152f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:52.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.437+0000 7f48f011d640 1 --2- 172.21.15.105:0/1686962151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48e80ad850 0x7f48e8152f80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f48e401c920 tx=0x7f48e401ac80 comp rx=0 tx=0).stop 2023-11-28T18:25:52.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.437+0000 7f48f011d640 1 -- 172.21.15.105:0/1686962151 shutdown_connections 2023-11-28T18:25:52.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.437+0000 7f48f011d640 1 --2- 172.21.15.105:0/1686962151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48e8159f40 0x7f48e81492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:52.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.437+0000 7f48f011d640 1 --2- 172.21.15.105:0/1686962151 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48e80cc760 0x7f48e8151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:52.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.437+0000 7f48f011d640 1 --2- 172.21.15.105:0/1686962151 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48cc0668d0 0x7f48cc068d90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:52.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.437+0000 7f48f011d640 1 --2- 172.21.15.105:0/1686962151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48e80ad850 0x7f48e8152f80 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:52.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.437+0000 7f48f011d640 1 -- 172.21.15.105:0/1686962151 >> 172.21.15.105:0/1686962151 conn(0x7f48e80ceb90 msgr2=0x7f48e80cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:52.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.437+0000 7f48f011d640 1 -- 172.21.15.105:0/1686962151 shutdown_connections 2023-11-28T18:25:52.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.437+0000 7f48f011d640 1 -- 172.21.15.105:0/1686962151 wait complete. 2023-11-28T18:25:52.459 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_conf_off' 2023-11-28T18:25:52.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.597+0000 7fc86117d640 1 Processor -- start 2023-11-28T18:25:52.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.597+0000 7fc86117d640 1 -- start start 2023-11-28T18:25:52.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.601+0000 7fc86117d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc85c15d2d0 0x7fc85c0cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:52.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.601+0000 7fc86117d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc85c0cf960 0x7fc85c166fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:52.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.601+0000 7fc86117d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc85c0ce840 con 0x7fc85c15cef0 2023-11-28T18:25:52.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.601+0000 7fc86117d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85c0cfe10 con 0x7fc85c0cf960 2023-11-28T18:25:52.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.601+0000 7fc86117d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c0b4780 con 0x7fc85c15d2d0 2023-11-28T18:25:52.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.601+0000 7fc86097c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc85c0cf960 0x7fc85c166fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:52.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.601+0000 7fc86097c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc85c0cf960 0x7fc85c166fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55270/0 (socket says 172.21.15.105:55270) 2023-11-28T18:25:52.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.601+0000 7fc86097c640 1 -- 172.21.15.105:0/474729677 learned_addr learned my addr 172.21.15.105:0/474729677 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:52.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.601+0000 7fc85b7fe640 1 --2- 172.21.15.105:0/474729677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc85c15d2d0 0x7fc85c0cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:52.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.601+0000 7fc86097c640 1 -- 172.21.15.105:0/474729677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc85c15d2d0 msgr2=0x7fc85c0cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:52.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.601+0000 7fc86097c640 1 --2- 172.21.15.105:0/474729677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc85c15d2d0 0x7fc85c0cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:52.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.601+0000 7fc86097c640 1 -- 172.21.15.105:0/474729677 >> v1:172.21.15.148:6789/0 conn(0x7fc85c15cef0 legacy=0x7fc85c0d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:52.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.601+0000 7fc86097c640 1 -- 172.21.15.105:0/474729677 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c0cd990 con 0x7fc85c0cf960 2023-11-28T18:25:52.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.605+0000 7fc86097c640 1 --2- 172.21.15.105:0/474729677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc85c0cf960 0x7fc85c166fd0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc854009e50 tx=0x7fc85401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438c3e8c4a8e0b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:52.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.605+0000 7fc85affd640 1 -- 172.21.15.105:0/474729677 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8540a5820 con 0x7fc85c0cf960 2023-11-28T18:25:52.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.605+0000 7fc85affd640 1 -- 172.21.15.105:0/474729677 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8540a5980 con 0x7fc85c0cf960 2023-11-28T18:25:52.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.605+0000 7fc85affd640 1 -- 172.21.15.105:0/474729677 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8540a5d10 con 0x7fc85c0cf960 2023-11-28T18:25:52.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.605+0000 7fc86117d640 1 -- 172.21.15.105:0/474729677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc85c0cf960 msgr2=0x7fc85c166fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:52.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.605+0000 7fc86117d640 1 --2- 172.21.15.105:0/474729677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc85c0cf960 0x7fc85c166fd0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc854009e50 tx=0x7fc85401c960 comp rx=0 tx=0).stop 2023-11-28T18:25:52.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.605+0000 7fc86117d640 1 -- 172.21.15.105:0/474729677 shutdown_connections 2023-11-28T18:25:52.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.605+0000 7fc86117d640 1 --2- 172.21.15.105:0/474729677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc85c0cf960 0x7fc85c166fd0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:52.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.605+0000 7fc86117d640 1 --2- 172.21.15.105:0/474729677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc85c15d2d0 0x7fc85c0cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:52.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.605+0000 7fc86117d640 1 -- 172.21.15.105:0/474729677 >> 172.21.15.105:0/474729677 conn(0x7fc85c0d2e10 msgr2=0x7fc85c0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:52.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.605+0000 7fc86117d640 1 -- 172.21.15.105:0/474729677 shutdown_connections 2023-11-28T18:25:52.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.605+0000 7fc86117d640 1 -- 172.21.15.105:0/474729677 wait complete. 2023-11-28T18:25:52.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.605+0000 7fc86117d640 1 Processor -- start 2023-11-28T18:25:52.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.605+0000 7fc86117d640 1 -- start start 2023-11-28T18:25:52.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.605+0000 7fc86117d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc85c0cf960 0x7fc85c154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:52.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.605+0000 7fc86117d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc85c15cef0 0x7fc85c157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:52.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.609+0000 7fc86117d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c15d2d0 0x7fc85c15ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:52.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.609+0000 7fc86117d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c15e540 con 0x7fc85c15cef0 2023-11-28T18:25:52.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.609+0000 7fc86117d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c15e7e0 con 0x7fc85c15d2d0 2023-11-28T18:25:52.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.609+0000 7fc86117d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c152960 con 0x7fc85c0cf960 2023-11-28T18:25:52.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.609+0000 7fc85b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc85c15cef0 0x7fc85c157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:52.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.609+0000 7fc85bfff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc85c0cf960 0x7fc85c154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:52.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.609+0000 7fc85b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc85c15cef0 0x7fc85c157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55284/0 (socket says 172.21.15.105:55284) 2023-11-28T18:25:52.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.609+0000 7fc86097c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c15d2d0 0x7fc85c15ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:52.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.609+0000 7fc85b7fe640 1 -- 172.21.15.105:0/2172173534 learned_addr learned my addr 172.21.15.105:0/2172173534 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:52.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.609+0000 7fc85b7fe640 1 -- 172.21.15.105:0/2172173534 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc85c0cf960 msgr2=0x7fc85c154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:52.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.609+0000 7fc85b7fe640 1 --2- 172.21.15.105:0/2172173534 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc85c0cf960 0x7fc85c154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:52.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.609+0000 7fc85b7fe640 1 -- 172.21.15.105:0/2172173534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c15d2d0 msgr2=0x7fc85c15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:52.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.609+0000 7fc85b7fe640 1 --2- 172.21.15.105:0/2172173534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c15d2d0 0x7fc85c15ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:52.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.609+0000 7fc85b7fe640 1 -- 172.21.15.105:0/2172173534 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc854009b30 con 0x7fc85c15cef0 2023-11-28T18:25:52.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.609+0000 7fc85b7fe640 1 --2- 172.21.15.105:0/2172173534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc85c15cef0 0x7fc85c157730 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc85000d7d0 tx=0x7fc85000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:52.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.609+0000 7fc858ff9640 1 -- 172.21.15.105:0/2172173534 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc850018070 con 0x7fc85c15cef0 2023-11-28T18:25:52.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.609+0000 7fc858ff9640 1 -- 172.21.15.105:0/2172173534 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc850004020 con 0x7fc85c15cef0 2023-11-28T18:25:52.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.609+0000 7fc858ff9640 1 -- 172.21.15.105:0/2172173534 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8500041d0 con 0x7fc85c15cef0 2023-11-28T18:25:52.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.609+0000 7fc86117d640 1 -- 172.21.15.105:0/2172173534 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85c152c40 con 0x7fc85c15cef0 2023-11-28T18:25:52.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.609+0000 7fc86117d640 1 -- 172.21.15.105:0/2172173534 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85c1552c0 con 0x7fc85c15cef0 2023-11-28T18:25:52.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.609+0000 7fc86117d640 1 -- 172.21.15.105:0/2172173534 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85c1554b0 con 0x7fc85c15cef0 2023-11-28T18:25:52.613 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.609+0000 7fc858ff9640 1 -- 172.21.15.105:0/2172173534 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc850004330 con 0x7fc85c15cef0 2023-11-28T18:25:52.613 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.613+0000 7fc858ff9640 1 --2- 172.21.15.105:0/2172173534 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc8480668d0 0x7fc848068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:52.613 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.613+0000 7fc858ff9640 1 -- 172.21.15.105:0/2172173534 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fc850083ff0 con 0x7fc85c15cef0 2023-11-28T18:25:52.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.613+0000 7fc85bfff640 1 --2- 172.21.15.105:0/2172173534 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc8480668d0 0x7fc848068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:52.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.613+0000 7fc85bfff640 1 --2- 172.21.15.105:0/2172173534 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc8480668d0 0x7fc848068d90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc84c007fd0 tx=0x7fc84c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:52.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.617+0000 7fc858ff9640 1 -- 172.21.15.105:0/2172173534 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc850051480 con 0x7fc85c15cef0 2023-11-28T18:25:52.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:52.825+0000 7fc86117d640 1 -- 172.21.15.105:0/2172173534 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_conf_off"]} v 0) v1 -- 0x7fc85c0d03c0 con 0x7fc85c15cef0 2023-11-28T18:25:53.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.097+0000 7fc858ff9640 1 -- 172.21.15.105:0/2172173534 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_conf_off"]}]=0 v148) v1 ==== 155+0+0 (secure 0 0 0) 0x7fc850054aa0 con 0x7fc85c15cef0 2023-11-28T18:25:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.097+0000 7fc86117d640 1 -- 172.21.15.105:0/2172173534 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc8480668d0 msgr2=0x7fc848068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.097+0000 7fc86117d640 1 --2- 172.21.15.105:0/2172173534 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc8480668d0 0x7fc848068d90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc84c007fd0 tx=0x7fc84c009040 comp rx=0 tx=0).stop 2023-11-28T18:25:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.097+0000 7fc86117d640 1 -- 172.21.15.105:0/2172173534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc85c15cef0 msgr2=0x7fc85c157730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.097+0000 7fc86117d640 1 --2- 172.21.15.105:0/2172173534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc85c15cef0 0x7fc85c157730 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc85000d7d0 tx=0x7fc85000dc90 comp rx=0 tx=0).stop 2023-11-28T18:25:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.097+0000 7fc86117d640 1 -- 172.21.15.105:0/2172173534 shutdown_connections 2023-11-28T18:25:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.097+0000 7fc86117d640 1 --2- 172.21.15.105:0/2172173534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c15d2d0 0x7fc85c15ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.097+0000 7fc86117d640 1 --2- 172.21.15.105:0/2172173534 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc8480668d0 0x7fc848068d90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.097+0000 7fc86117d640 1 --2- 172.21.15.105:0/2172173534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc85c15cef0 0x7fc85c157730 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.097+0000 7fc86117d640 1 --2- 172.21.15.105:0/2172173534 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc85c0cf960 0x7fc85c154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.097+0000 7fc86117d640 1 -- 172.21.15.105:0/2172173534 >> 172.21.15.105:0/2172173534 conn(0x7fc85c0d2e10 msgr2=0x7fc85c0af980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:53.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.097+0000 7fc86117d640 1 -- 172.21.15.105:0/2172173534 shutdown_connections 2023-11-28T18:25:53.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.097+0000 7fc86117d640 1 -- 172.21.15.105:0/2172173534 wait complete. 2023-11-28T18:25:53.120 INFO:tasks.cephfs_test_runner:That turning off ephemeral distributed pin config prevents distribution. ... ok 2023-11-28T18:25:53.120 INFO:tasks.cephfs_test_runner:Starting test: test_ephemeral_pin_dist_failover (tasks.cephfs.test_exports.TestEphemeralPins) That MDS failover does not cause unnecessary migrations. 2023-11-28T18:25:53.121 INFO:tasks.cephfs_test_runner:test_ephemeral_pin_dist_failover (tasks.cephfs.test_exports.TestEphemeralPins) 2023-11-28T18:25:53.121 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:25:53.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.257+0000 7f7c8aa9a640 1 Processor -- start 2023-11-28T18:25:53.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.257+0000 7f7c8aa9a640 1 -- start start 2023-11-28T18:25:53.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.257+0000 7f7c8aa9a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c8415d4f0 0x7f7c840d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:53.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.257+0000 7f7c8aa9a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c840b25b0 con 0x7f7c840d3c30 2023-11-28T18:25:53.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.257+0000 7f7c8aa9a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c840cea30 con 0x7f7c840b2d20 2023-11-28T18:25:53.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.257+0000 7f7c8aa9a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c840b2900 con 0x7f7c8415d4f0 2023-11-28T18:25:53.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.257+0000 7f7c7bfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7c840d3c30 0x7f7c840d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39048/0 (socket says 172.21.15.105:39048) 2023-11-28T18:25:53.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.257+0000 7f7c7bfff640 1 -- 172.21.15.105:0/178011909 learned_addr learned my addr 172.21.15.105:0/178011909 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:53.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.257+0000 7f7c88836640 1 --2- 172.21.15.105:0/178011909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c8415d4f0 0x7f7c840d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:53.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.257+0000 7f7c7b7fe640 1 -- 172.21.15.105:0/178011909 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 542002845 0 0) 0x7f7c840b25b0 con 0x7f7c840d3c30 2023-11-28T18:25:53.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.257+0000 7f7c7b7fe640 1 -- 172.21.15.105:0/178011909 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c6c003530 con 0x7f7c840d3c30 2023-11-28T18:25:53.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.257+0000 7f7c7b7fe640 1 -- 172.21.15.105:0/178011909 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281622958 0 0) 0x7f7c840cea30 con 0x7f7c840b2d20 2023-11-28T18:25:53.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.257+0000 7f7c88836640 1 -- 172.21.15.105:0/178011909 >> v1:172.21.15.148:6789/0 conn(0x7f7c840b2d20 legacy=0x7f7c840b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:53.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.257+0000 7f7c88836640 1 -- 172.21.15.105:0/178011909 >> v1:172.21.15.105:6789/0 conn(0x7f7c840d3c30 legacy=0x7f7c840d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:53.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.257+0000 7f7c88836640 1 -- 172.21.15.105:0/178011909 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c840ced10 con 0x7f7c8415d4f0 2023-11-28T18:25:53.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.261+0000 7f7c88836640 1 --2- 172.21.15.105:0/178011909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c8415d4f0 0x7f7c840d0e10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7c800097f0 tx=0x7f7c8001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10dd1a7604745b0f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:53.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.261+0000 7f7c7b7fe640 1 -- 172.21.15.105:0/178011909 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c800ab600 con 0x7f7c8415d4f0 2023-11-28T18:25:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.261+0000 7f7c7b7fe640 1 -- 172.21.15.105:0/178011909 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c800a7050 con 0x7f7c8415d4f0 2023-11-28T18:25:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.261+0000 7f7c7b7fe640 1 -- 172.21.15.105:0/178011909 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c800ab980 con 0x7f7c8415d4f0 2023-11-28T18:25:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.261+0000 7f7c8aa9a640 1 -- 172.21.15.105:0/178011909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c8415d4f0 msgr2=0x7f7c840d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.261+0000 7f7c8aa9a640 1 --2- 172.21.15.105:0/178011909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c8415d4f0 0x7f7c840d0e10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7c800097f0 tx=0x7f7c8001c400 comp rx=0 tx=0).stop 2023-11-28T18:25:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.261+0000 7f7c8aa9a640 1 -- 172.21.15.105:0/178011909 shutdown_connections 2023-11-28T18:25:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.261+0000 7f7c8aa9a640 1 --2- 172.21.15.105:0/178011909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c8415d4f0 0x7f7c840d0e10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.261+0000 7f7c8aa9a640 1 -- 172.21.15.105:0/178011909 >> 172.21.15.105:0/178011909 conn(0x7f7c840d3030 msgr2=0x7f7c840d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.261+0000 7f7c8aa9a640 1 -- 172.21.15.105:0/178011909 shutdown_connections 2023-11-28T18:25:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.261+0000 7f7c8aa9a640 1 -- 172.21.15.105:0/178011909 wait complete. 2023-11-28T18:25:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c8aa9a640 1 Processor -- start 2023-11-28T18:25:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c8aa9a640 1 -- start start 2023-11-28T18:25:53.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c8aa9a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c840b2d20 0x7f7c840cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:53.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c8aa9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c840d3c30 0x7f7c84150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:53.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c8aa9a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c8415d4f0 0x7f7c8415a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:53.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c8aa9a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c840d0a20 con 0x7f7c840d3c30 2023-11-28T18:25:53.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c8aa9a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c840cb7c0 con 0x7f7c8415d4f0 2023-11-28T18:25:53.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c8aa9a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c840cbaa0 con 0x7f7c840b2d20 2023-11-28T18:25:53.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c7bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c840d3c30 0x7f7c84150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:53.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c7bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c840d3c30 0x7f7c84150f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55296/0 (socket says 172.21.15.105:55296) 2023-11-28T18:25:53.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c7bfff640 1 -- 172.21.15.105:0/640905143 learned_addr learned my addr 172.21.15.105:0/640905143 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:53.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c89037640 1 --2- 172.21.15.105:0/640905143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c8415d4f0 0x7f7c8415a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:53.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c88836640 1 --2- 172.21.15.105:0/640905143 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c840b2d20 0x7f7c840cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:53.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c7bfff640 1 -- 172.21.15.105:0/640905143 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c840b2d20 msgr2=0x7f7c840cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:53.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c7bfff640 1 --2- 172.21.15.105:0/640905143 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c840b2d20 0x7f7c840cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c7bfff640 1 -- 172.21.15.105:0/640905143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c8415d4f0 msgr2=0x7f7c8415a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:53.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c7bfff640 1 --2- 172.21.15.105:0/640905143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c8415d4f0 0x7f7c8415a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c7bfff640 1 -- 172.21.15.105:0/640905143 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c80009420 con 0x7f7c840d3c30 2023-11-28T18:25:53.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c89037640 1 --2- 172.21.15.105:0/640905143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c8415d4f0 0x7f7c8415a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:53.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c88836640 1 --2- 172.21.15.105:0/640905143 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c840b2d20 0x7f7c840cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:53.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c7bfff640 1 --2- 172.21.15.105:0/640905143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c840d3c30 0x7f7c84150f00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7c7400c3c0 tx=0x7f7c7400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:53.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c797fa640 1 -- 172.21.15.105:0/640905143 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c74005cf0 con 0x7f7c840d3c30 2023-11-28T18:25:53.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c797fa640 1 -- 172.21.15.105:0/640905143 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c74005e50 con 0x7f7c840d3c30 2023-11-28T18:25:53.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c8aa9a640 1 -- 172.21.15.105:0/640905143 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c840cbd80 con 0x7f7c840d3c30 2023-11-28T18:25:53.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c797fa640 1 -- 172.21.15.105:0/640905143 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c740072f0 con 0x7f7c840d3c30 2023-11-28T18:25:53.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.265+0000 7f7c8aa9a640 1 -- 172.21.15.105:0/640905143 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c840cc340 con 0x7f7c840d3c30 2023-11-28T18:25:53.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.269+0000 7f7c8aa9a640 1 -- 172.21.15.105:0/640905143 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c840cc530 con 0x7f7c840d3c30 2023-11-28T18:25:53.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.269+0000 7f7c797fa640 1 -- 172.21.15.105:0/640905143 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f7c74007450 con 0x7f7c840d3c30 2023-11-28T18:25:53.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.269+0000 7f7c797fa640 1 --2- 172.21.15.105:0/640905143 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7c700668d0 0x7f7c70068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:53.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.269+0000 7f7c797fa640 1 -- 172.21.15.105:0/640905143 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f7c740839c0 con 0x7f7c840d3c30 2023-11-28T18:25:53.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.269+0000 7f7c88836640 1 --2- 172.21.15.105:0/640905143 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7c700668d0 0x7f7c70068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:53.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.269+0000 7f7c88836640 1 --2- 172.21.15.105:0/640905143 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7c700668d0 0x7f7c70068d90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7c800ad000 tx=0x7f7c800a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:53.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.273+0000 7f7c797fa640 1 -- 172.21.15.105:0/640905143 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7c74050c70 con 0x7f7c840d3c30 2023-11-28T18:25:53.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.433+0000 7f7c8aa9a640 1 -- 172.21.15.105:0/640905143 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c84067570 con 0x7f7c840d3c30 2023-11-28T18:25:53.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.433+0000 7f7c797fa640 1 -- 172.21.15.105:0/640905143 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+13515 (secure 0 0 0) 0x7f7c74017070 con 0x7f7c840d3c30 2023-11-28T18:25:53.435 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:53.435 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":43,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:25:51.075552+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6835/2878687703":"2023-11-29T18:25:42.888414+0000","172.21.15.148:6833/2878687703":"2023-11-29T18:25:42.888414+0000","172.21.15.105:6836/681913204":"2023-11-29T18:25:42.888428+0000","172.21.15.105:6834/681913204":"2023-11-29T18:25:42.888428+0000","172.21.15.105:6837/1876408690":"2023-11-29T18:25:42.888432+0000","172.21.15.105:6835/1876408690":"2023-11-29T18:25:42.888432+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:25:53.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.433+0000 7f7c8aa9a640 1 -- 172.21.15.105:0/640905143 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7c700668d0 msgr2=0x7f7c70068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:53.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.433+0000 7f7c8aa9a640 1 --2- 172.21.15.105:0/640905143 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7c700668d0 0x7f7c70068d90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7c800ad000 tx=0x7f7c800a6000 comp rx=0 tx=0).stop 2023-11-28T18:25:53.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.433+0000 7f7c8aa9a640 1 -- 172.21.15.105:0/640905143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c840d3c30 msgr2=0x7f7c84150f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:53.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.433+0000 7f7c8aa9a640 1 --2- 172.21.15.105:0/640905143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c840d3c30 0x7f7c84150f00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7c7400c3c0 tx=0x7f7c7400c880 comp rx=0 tx=0).stop 2023-11-28T18:25:53.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.433+0000 7f7c8aa9a640 1 -- 172.21.15.105:0/640905143 shutdown_connections 2023-11-28T18:25:53.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.433+0000 7f7c8aa9a640 1 --2- 172.21.15.105:0/640905143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c8415d4f0 0x7f7c8415a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.433+0000 7f7c8aa9a640 1 --2- 172.21.15.105:0/640905143 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7c700668d0 0x7f7c70068d90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.433+0000 7f7c8aa9a640 1 --2- 172.21.15.105:0/640905143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c840d3c30 0x7f7c84150f00 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.433+0000 7f7c8aa9a640 1 --2- 172.21.15.105:0/640905143 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c840b2d20 0x7f7c840cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.433+0000 7f7c8aa9a640 1 -- 172.21.15.105:0/640905143 >> 172.21.15.105:0/640905143 conn(0x7f7c840d3030 msgr2=0x7f7c840afcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:53.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.433+0000 7f7c8aa9a640 1 -- 172.21.15.105:0/640905143 shutdown_connections 2023-11-28T18:25:53.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.433+0000 7f7c8aa9a640 1 -- 172.21.15.105:0/640905143 wait complete. 2023-11-28T18:25:53.455 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:25:53.456 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:25:53.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.593+0000 7fd5e109f640 1 Processor -- start 2023-11-28T18:25:53.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.597+0000 7fd5e109f640 1 -- start start 2023-11-28T18:25:53.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.597+0000 7fd5e109f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd5dc0ce170 0x7fd5dc15d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:53.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.597+0000 7fd5e109f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5dc0cd170 0x7fd5dc0cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:53.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.597+0000 7fd5e109f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5dc06d070 con 0x7fd5dc159710 2023-11-28T18:25:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.597+0000 7fd5e109f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5dc0cdb40 con 0x7fd5dc0ce170 2023-11-28T18:25:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.597+0000 7fd5e109f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc06cab0 con 0x7fd5dc0cd170 2023-11-28T18:25:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.597+0000 7fd5db7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd5dc0ce170 0x7fd5dc15d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.597+0000 7fd5dbfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd5dc159710 0x7fd5dc06c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39056/0 (socket says 172.21.15.105:39056) 2023-11-28T18:25:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.597+0000 7fd5dbfff640 1 -- 172.21.15.105:0/189425236 learned_addr learned my addr 172.21.15.105:0/189425236 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.597+0000 7fd5e089e640 1 --2- 172.21.15.105:0/189425236 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5dc0cd170 0x7fd5dc0cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.597+0000 7fd5db7fe640 1 -- 172.21.15.105:0/189425236 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5dc0cd170 msgr2=0x7fd5dc0cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.597+0000 7fd5db7fe640 1 --2- 172.21.15.105:0/189425236 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5dc0cd170 0x7fd5dc0cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.597+0000 7fd5db7fe640 1 -- 172.21.15.105:0/189425236 >> v1:172.21.15.105:6789/0 conn(0x7fd5dc159710 legacy=0x7fd5dc06c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:53.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.597+0000 7fd5db7fe640 1 -- 172.21.15.105:0/189425236 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5dc06cd90 con 0x7fd5dc0ce170 2023-11-28T18:25:53.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.601+0000 7fd5db7fe640 1 --2- 172.21.15.105:0/189425236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5dc0ce170 0x7fd5dc15d940 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00097c0 tx=0x7fd5d001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fa64f2540cad79 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:53.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.601+0000 7fd5daffd640 1 -- 172.21.15.105:0/189425236 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d00ab680 con 0x7fd5dc0ce170 2023-11-28T18:25:53.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.601+0000 7fd5daffd640 1 -- 172.21.15.105:0/189425236 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d00ab7e0 con 0x7fd5dc0ce170 2023-11-28T18:25:53.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.601+0000 7fd5daffd640 1 -- 172.21.15.105:0/189425236 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d00abb70 con 0x7fd5dc0ce170 2023-11-28T18:25:53.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.601+0000 7fd5e109f640 1 -- 172.21.15.105:0/189425236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5dc0ce170 msgr2=0x7fd5dc15d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:53.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.601+0000 7fd5e109f640 1 --2- 172.21.15.105:0/189425236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5dc0ce170 0x7fd5dc15d940 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00097c0 tx=0x7fd5d001c480 comp rx=0 tx=0).stop 2023-11-28T18:25:53.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.601+0000 7fd5e109f640 1 -- 172.21.15.105:0/189425236 shutdown_connections 2023-11-28T18:25:53.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.601+0000 7fd5e109f640 1 --2- 172.21.15.105:0/189425236 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5dc0cd170 0x7fd5dc0cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.601+0000 7fd5e109f640 1 --2- 172.21.15.105:0/189425236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5dc0ce170 0x7fd5dc15d940 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.601+0000 7fd5e109f640 1 -- 172.21.15.105:0/189425236 >> 172.21.15.105:0/189425236 conn(0x7fd5dc0ce9e0 msgr2=0x7fd5dc0cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.601+0000 7fd5e109f640 1 -- 172.21.15.105:0/189425236 shutdown_connections 2023-11-28T18:25:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.601+0000 7fd5e109f640 1 -- 172.21.15.105:0/189425236 wait complete. 2023-11-28T18:25:53.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.601+0000 7fd5e109f640 1 Processor -- start 2023-11-28T18:25:53.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.601+0000 7fd5e109f640 1 -- start start 2023-11-28T18:25:53.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5e109f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5dc0cd170 0x7fd5dc1f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:53.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5e109f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5dc0ce170 0x7fd5dc0b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:53.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5e109f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5dc159710 0x7fd5dc14c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:53.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5e109f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc0b3130 con 0x7fd5dc159710 2023-11-28T18:25:53.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5e109f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc0b2270 con 0x7fd5dc0ce170 2023-11-28T18:25:53.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5e109f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc0b24b0 con 0x7fd5dc0cd170 2023-11-28T18:25:53.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5e089e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5dc159710 0x7fd5dc14c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:53.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5e089e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5dc159710 0x7fd5dc14c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55308/0 (socket says 172.21.15.105:55308) 2023-11-28T18:25:53.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5e089e640 1 -- 172.21.15.105:0/4179042435 learned_addr learned my addr 172.21.15.105:0/4179042435 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:53.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5dbfff640 1 --2- 172.21.15.105:0/4179042435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5dc0cd170 0x7fd5dc1f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:53.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5db7fe640 1 --2- 172.21.15.105:0/4179042435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5dc0ce170 0x7fd5dc0b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:53.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5e089e640 1 -- 172.21.15.105:0/4179042435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5dc0cd170 msgr2=0x7fd5dc1f6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:53.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5e089e640 1 --2- 172.21.15.105:0/4179042435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5dc0cd170 0x7fd5dc1f6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5e089e640 1 -- 172.21.15.105:0/4179042435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5dc0ce170 msgr2=0x7fd5dc0b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:53.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5e089e640 1 --2- 172.21.15.105:0/4179042435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5dc0ce170 0x7fd5dc0b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5e089e640 1 -- 172.21.15.105:0/4179042435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d0009420 con 0x7fd5dc159710 2023-11-28T18:25:53.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5e089e640 1 --2- 172.21.15.105:0/4179042435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5dc159710 0x7fd5dc14c040 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4007e40 tx=0x7fd5d400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:53.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5d8ff9640 1 -- 172.21.15.105:0/4179042435 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4014620 con 0x7fd5dc159710 2023-11-28T18:25:53.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5d8ff9640 1 -- 172.21.15.105:0/4179042435 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d400ee90 con 0x7fd5dc159710 2023-11-28T18:25:53.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5d8ff9640 1 -- 172.21.15.105:0/4179042435 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40149e0 con 0x7fd5dc159710 2023-11-28T18:25:53.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5e109f640 1 -- 172.21.15.105:0/4179042435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5dc0b2790 con 0x7fd5dc159710 2023-11-28T18:25:53.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5e109f640 1 -- 172.21.15.105:0/4179042435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5dc0b2d50 con 0x7fd5dc159710 2023-11-28T18:25:53.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5e109f640 1 -- 172.21.15.105:0/4179042435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5dc1f6c50 con 0x7fd5dc159710 2023-11-28T18:25:53.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.605+0000 7fd5d8ff9640 1 -- 172.21.15.105:0/4179042435 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd5d4014b40 con 0x7fd5dc159710 2023-11-28T18:25:53.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.609+0000 7fd5d8ff9640 1 --2- 172.21.15.105:0/4179042435 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5c80668d0 0x7fd5c8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:53.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.609+0000 7fd5d8ff9640 1 -- 172.21.15.105:0/4179042435 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fd5d4084b80 con 0x7fd5dc159710 2023-11-28T18:25:53.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.609+0000 7fd5dbfff640 1 --2- 172.21.15.105:0/4179042435 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5c80668d0 0x7fd5c8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:53.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.609+0000 7fd5dbfff640 1 --2- 172.21.15.105:0/4179042435 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5c80668d0 0x7fd5c8068d90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc005890 tx=0x7fd5cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:53.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.613+0000 7fd5d8ff9640 1 -- 172.21.15.105:0/4179042435 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd5d4051f70 con 0x7fd5dc159710 2023-11-28T18:25:53.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.765+0000 7fd5e109f640 1 -- 172.21.15.105:0/4179042435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd5dc074580 con 0x7fd5dc159710 2023-11-28T18:25:53.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.765+0000 7fd5d8ff9640 1 -- 172.21.15.105:0/4179042435 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd5d4084e60 con 0x7fd5dc159710 2023-11-28T18:25:53.770 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:25:53.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.769+0000 7fd5e109f640 1 -- 172.21.15.105:0/4179042435 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5c80668d0 msgr2=0x7fd5c8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:53.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.769+0000 7fd5e109f640 1 --2- 172.21.15.105:0/4179042435 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5c80668d0 0x7fd5c8068d90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc005890 tx=0x7fd5cc009040 comp rx=0 tx=0).stop 2023-11-28T18:25:53.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.769+0000 7fd5e109f640 1 -- 172.21.15.105:0/4179042435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5dc159710 msgr2=0x7fd5dc14c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:53.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.769+0000 7fd5e109f640 1 --2- 172.21.15.105:0/4179042435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5dc159710 0x7fd5dc14c040 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4007e40 tx=0x7fd5d400e4b0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.769+0000 7fd5e109f640 1 -- 172.21.15.105:0/4179042435 shutdown_connections 2023-11-28T18:25:53.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.769+0000 7fd5e109f640 1 --2- 172.21.15.105:0/4179042435 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5c80668d0 0x7fd5c8068d90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.769+0000 7fd5e109f640 1 --2- 172.21.15.105:0/4179042435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5dc159710 0x7fd5dc14c040 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.769+0000 7fd5e109f640 1 --2- 172.21.15.105:0/4179042435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5dc0ce170 0x7fd5dc0b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.769+0000 7fd5e109f640 1 --2- 172.21.15.105:0/4179042435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5dc0cd170 0x7fd5dc1f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.769+0000 7fd5e109f640 1 -- 172.21.15.105:0/4179042435 >> 172.21.15.105:0/4179042435 conn(0x7fd5dc0ce9e0 msgr2=0x7fd5dc06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:53.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.769+0000 7fd5e109f640 1 -- 172.21.15.105:0/4179042435 shutdown_connections 2023-11-28T18:25:53.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.769+0000 7fd5e109f640 1 -- 172.21.15.105:0/4179042435 wait complete. 2023-11-28T18:25:53.791 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_failover' 2023-11-28T18:25:53.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.929+0000 7f2e72772640 1 Processor -- start 2023-11-28T18:25:53.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.929+0000 7f2e72772640 1 -- start start 2023-11-28T18:25:53.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.929+0000 7f2e72772640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2e6c15d4f0 0x7f2e6c0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:53.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.929+0000 7f2e72772640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2e6c0d3c30 0x7f2e6c0d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:53.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.929+0000 7f2e72772640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e6c0b25b0 con 0x7f2e6c0d1890 2023-11-28T18:25:53.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.929+0000 7f2e72772640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e6c0b2900 con 0x7f2e6c15d4f0 2023-11-28T18:25:53.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.929+0000 7f2e72772640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c0cea30 con 0x7f2e6c0d3c30 2023-11-28T18:25:53.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.929+0000 7f2e70d0f640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f2e6c0d1890 0x7f2e6c0cfa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39062/0 (socket says 172.21.15.105:39062) 2023-11-28T18:25:53.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.929+0000 7f2e70d0f640 1 -- 172.21.15.105:0/362631878 learned_addr learned my addr 172.21.15.105:0/362631878 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:53.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.929+0000 7f2e6bfff640 1 --2- 172.21.15.105:0/362631878 >> v2:172.21.15.148:3300/0 conn(0x7f2e6c15d4f0 0x7f2e6c0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:53.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.929+0000 7f2e6b7fe640 1 --2- 172.21.15.105:0/362631878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2e6c0d3c30 0x7f2e6c0d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:53.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.929+0000 7f2e6affd640 1 -- 172.21.15.105:0/362631878 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1324640955 0 0) 0x7f2e6c0b25b0 con 0x7f2e6c0d1890 2023-11-28T18:25:53.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.929+0000 7f2e6affd640 1 -- 172.21.15.105:0/362631878 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e54003530 con 0x7f2e6c0d1890 2023-11-28T18:25:53.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.929+0000 7f2e6bfff640 1 -- 172.21.15.105:0/362631878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2e6c0d3c30 msgr2=0x7f2e6c0d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:53.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.929+0000 7f2e6bfff640 1 --2- 172.21.15.105:0/362631878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2e6c0d3c30 0x7f2e6c0d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.929+0000 7f2e6bfff640 1 -- 172.21.15.105:0/362631878 >> v1:172.21.15.105:6789/0 conn(0x7f2e6c0d1890 legacy=0x7f2e6c0cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:53.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.929+0000 7f2e6bfff640 1 -- 172.21.15.105:0/362631878 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6c0ced10 con 0x7f2e6c15d4f0 2023-11-28T18:25:53.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.933+0000 7f2e6bfff640 1 --2- 172.21.15.105:0/362631878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e6c15d4f0 0x7f2e6c0d0e10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0097c0 tx=0x7f2e5c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1526acdb67a2a14c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:53.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.933+0000 7f2e6affd640 1 -- 172.21.15.105:0/362631878 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c0ab600 con 0x7f2e6c15d4f0 2023-11-28T18:25:53.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.933+0000 7f2e6affd640 1 -- 172.21.15.105:0/362631878 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e5c0a7050 con 0x7f2e6c15d4f0 2023-11-28T18:25:53.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.933+0000 7f2e6affd640 1 -- 172.21.15.105:0/362631878 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c0ab980 con 0x7f2e6c15d4f0 2023-11-28T18:25:53.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.933+0000 7f2e72772640 1 -- 172.21.15.105:0/362631878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e6c15d4f0 msgr2=0x7f2e6c0d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:53.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.933+0000 7f2e72772640 1 --2- 172.21.15.105:0/362631878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e6c15d4f0 0x7f2e6c0d0e10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0097c0 tx=0x7f2e5c01c3c0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.933+0000 7f2e72772640 1 -- 172.21.15.105:0/362631878 shutdown_connections 2023-11-28T18:25:53.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.933+0000 7f2e72772640 1 --2- 172.21.15.105:0/362631878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2e6c0d3c30 0x7f2e6c0d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.933+0000 7f2e72772640 1 --2- 172.21.15.105:0/362631878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e6c15d4f0 0x7f2e6c0d0e10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.933+0000 7f2e72772640 1 -- 172.21.15.105:0/362631878 >> 172.21.15.105:0/362631878 conn(0x7f2e6c0d3030 msgr2=0x7f2e6c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:53.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.933+0000 7f2e72772640 1 -- 172.21.15.105:0/362631878 shutdown_connections 2023-11-28T18:25:53.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.933+0000 7f2e72772640 1 -- 172.21.15.105:0/362631878 wait complete. 2023-11-28T18:25:53.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e72772640 1 Processor -- start 2023-11-28T18:25:53.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e72772640 1 -- start start 2023-11-28T18:25:53.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e72772640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e6c0d1890 0x7f2e6c0cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:53.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e72772640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e6c0d3c30 0x7f2e6c0ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:53.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e72772640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2e6c15d4f0 0x7f2e6c1514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:53.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e72772640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c152140 con 0x7f2e6c0d1890 2023-11-28T18:25:53.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e72772640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c0caa00 con 0x7f2e6c0d3c30 2023-11-28T18:25:53.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e72772640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c0cace0 con 0x7f2e6c15d4f0 2023-11-28T18:25:53.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e6bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e6c0d1890 0x7f2e6c0cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:53.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e6bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e6c0d1890 0x7f2e6c0cb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55324/0 (socket says 172.21.15.105:55324) 2023-11-28T18:25:53.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e6bfff640 1 -- 172.21.15.105:0/3131732880 learned_addr learned my addr 172.21.15.105:0/3131732880 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:53.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e6b7fe640 1 --2- 172.21.15.105:0/3131732880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e6c0d3c30 0x7f2e6c0ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:53.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e70d0f640 1 --2- 172.21.15.105:0/3131732880 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2e6c15d4f0 0x7f2e6c1514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:53.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e6bfff640 1 -- 172.21.15.105:0/3131732880 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2e6c15d4f0 msgr2=0x7f2e6c1514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:53.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e6bfff640 1 --2- 172.21.15.105:0/3131732880 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2e6c15d4f0 0x7f2e6c1514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e6bfff640 1 -- 172.21.15.105:0/3131732880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e6c0d3c30 msgr2=0x7f2e6c0ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:53.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e6bfff640 1 --2- 172.21.15.105:0/3131732880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e6c0d3c30 0x7f2e6c0ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:53.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e6bfff640 1 -- 172.21.15.105:0/3131732880 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5c009420 con 0x7f2e6c0d1890 2023-11-28T18:25:53.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e6b7fe640 1 --2- 172.21.15.105:0/3131732880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e6c0d3c30 0x7f2e6c0ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:53.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e6bfff640 1 --2- 172.21.15.105:0/3131732880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e6c0d1890 0x7f2e6c0cb770 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0ae040 tx=0x7f2e5c01ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:53.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e68ff9640 1 -- 172.21.15.105:0/3131732880 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c006e20 con 0x7f2e6c0d1890 2023-11-28T18:25:53.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e68ff9640 1 -- 172.21.15.105:0/3131732880 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e5c0b7450 con 0x7f2e6c0d1890 2023-11-28T18:25:53.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e68ff9640 1 -- 172.21.15.105:0/3131732880 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c0b7800 con 0x7f2e6c0d1890 2023-11-28T18:25:53.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e72772640 1 -- 172.21.15.105:0/3131732880 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e6c0cafc0 con 0x7f2e6c0d1890 2023-11-28T18:25:53.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.937+0000 7f2e72772640 1 -- 172.21.15.105:0/3131732880 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e6c0cbcb0 con 0x7f2e6c0d1890 2023-11-28T18:25:53.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.941+0000 7f2e72772640 1 -- 172.21.15.105:0/3131732880 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e6c067570 con 0x7f2e6c0d1890 2023-11-28T18:25:53.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.941+0000 7f2e68ff9640 1 -- 172.21.15.105:0/3131732880 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f2e5c0b7960 con 0x7f2e6c0d1890 2023-11-28T18:25:53.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.941+0000 7f2e68ff9640 1 --2- 172.21.15.105:0/3131732880 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2e580668d0 0x7f2e58068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:53.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.941+0000 7f2e68ff9640 1 -- 172.21.15.105:0/3131732880 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f2e5c11ea10 con 0x7f2e6c0d1890 2023-11-28T18:25:53.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.941+0000 7f2e6b7fe640 1 --2- 172.21.15.105:0/3131732880 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2e580668d0 0x7f2e58068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:53.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.941+0000 7f2e6b7fe640 1 --2- 172.21.15.105:0/3131732880 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2e580668d0 0x7f2e58068d90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0b8700 tx=0x7f2e60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:53.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:53.945+0000 7f2e68ff9640 1 -- 172.21.15.105:0/3131732880 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2e5c0ebdf0 con 0x7f2e6c0d1890 2023-11-28T18:25:54.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:54.149+0000 7f2e72772640 1 -- 172.21.15.105:0/3131732880 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_failover"]} v 0) v1 -- 0x7f2e6c0b1360 con 0x7f2e6c0d1890 2023-11-28T18:25:55.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.105+0000 7f2e68ff9640 1 -- 172.21.15.105:0/3131732880 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_failover"]}]=0 v150) v1 ==== 158+0+0 (secure 0 0 0) 0x7f2e5c0ef410 con 0x7f2e6c0d1890 2023-11-28T18:25:55.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.109+0000 7f2e72772640 1 -- 172.21.15.105:0/3131732880 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2e580668d0 msgr2=0x7f2e58068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:55.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.109+0000 7f2e72772640 1 --2- 172.21.15.105:0/3131732880 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2e580668d0 0x7f2e58068d90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0b8700 tx=0x7f2e60008040 comp rx=0 tx=0).stop 2023-11-28T18:25:55.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.109+0000 7f2e72772640 1 -- 172.21.15.105:0/3131732880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e6c0d1890 msgr2=0x7f2e6c0cb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:55.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.109+0000 7f2e72772640 1 --2- 172.21.15.105:0/3131732880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e6c0d1890 0x7f2e6c0cb770 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0ae040 tx=0x7f2e5c01ac00 comp rx=0 tx=0).stop 2023-11-28T18:25:55.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.109+0000 7f2e72772640 1 -- 172.21.15.105:0/3131732880 shutdown_connections 2023-11-28T18:25:55.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.109+0000 7f2e72772640 1 --2- 172.21.15.105:0/3131732880 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2e6c15d4f0 0x7f2e6c1514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:55.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.109+0000 7f2e72772640 1 --2- 172.21.15.105:0/3131732880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e6c0d3c30 0x7f2e6c0ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:55.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.109+0000 7f2e72772640 1 --2- 172.21.15.105:0/3131732880 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2e580668d0 0x7f2e58068d90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:55.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.109+0000 7f2e72772640 1 --2- 172.21.15.105:0/3131732880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e6c0d1890 0x7f2e6c0cb770 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:55.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.109+0000 7f2e72772640 1 -- 172.21.15.105:0/3131732880 >> 172.21.15.105:0/3131732880 conn(0x7f2e6c0d3030 msgr2=0x7f2e6c0cf6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:55.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.109+0000 7f2e72772640 1 -- 172.21.15.105:0/3131732880 shutdown_connections 2023-11-28T18:25:55.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.109+0000 7f2e72772640 1 -- 172.21.15.105:0/3131732880 wait complete. 2023-11-28T18:25:55.136 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T18:25:55.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.277+0000 7f7776e54640 1 Processor -- start 2023-11-28T18:25:55.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.277+0000 7f7776e54640 1 -- start start 2023-11-28T18:25:55.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.281+0000 7f7776e54640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7770159f40 0x7f77700caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:55.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.281+0000 7f7776e54640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f77700ad850 0x7f77700701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:55.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.281+0000 7f7776e54640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f777006bf50 con 0x7f77700cc760 2023-11-28T18:25:55.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.281+0000 7f7776e54640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77700ccbc0 con 0x7f77700ad850 2023-11-28T18:25:55.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.281+0000 7f7776e54640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f777015a320 con 0x7f7770159f40 2023-11-28T18:25:55.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.281+0000 7f77753f1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f77700ad850 0x7f77700701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:55.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.281+0000 7f77753f1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f77700ad850 0x7f77700701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55328/0 (socket says 172.21.15.105:55328) 2023-11-28T18:25:55.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.281+0000 7f77753f1640 1 -- 172.21.15.105:0/1093537178 learned_addr learned my addr 172.21.15.105:0/1093537178 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:55.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.281+0000 7f7774bf0640 1 --2- 172.21.15.105:0/1093537178 >> v2:172.21.15.148:3300/0 conn(0x7f7770159f40 0x7f77700caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:55.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.281+0000 7f77753f1640 1 -- 172.21.15.105:0/1093537178 >> v2:172.21.15.148:3300/0 conn(0x7f7770159f40 msgr2=0x7f77700caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:55.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.281+0000 7f77753f1640 1 --2- 172.21.15.105:0/1093537178 >> v2:172.21.15.148:3300/0 conn(0x7f7770159f40 0x7f77700caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:55.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.281+0000 7f77753f1640 1 -- 172.21.15.105:0/1093537178 >> v1:172.21.15.148:6789/0 conn(0x7f77700cc760 legacy=0x7f77700ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:55.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.281+0000 7f77753f1640 1 -- 172.21.15.105:0/1093537178 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7770071380 con 0x7f77700ad850 2023-11-28T18:25:55.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.281+0000 7f7774bf0640 1 --2- 172.21.15.105:0/1093537178 >> v2:172.21.15.148:3300/0 conn(0x7f7770159f40 0x7f77700caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:55.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.281+0000 7f77753f1640 1 --2- 172.21.15.105:0/1093537178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77700ad850 0x7f77700701d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f777006e500 tx=0x7f776801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae847d763e9d11c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:55.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.285+0000 7f77677fe640 1 -- 172.21.15.105:0/1093537178 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77680a57d0 con 0x7f77700ad850 2023-11-28T18:25:55.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.285+0000 7f77677fe640 1 -- 172.21.15.105:0/1093537178 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77680a5930 con 0x7f77700ad850 2023-11-28T18:25:55.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.285+0000 7f77677fe640 1 -- 172.21.15.105:0/1093537178 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77680a5cc0 con 0x7f77700ad850 2023-11-28T18:25:55.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.285+0000 7f7776e54640 1 -- 172.21.15.105:0/1093537178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77700ad850 msgr2=0x7f77700701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:55.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.285+0000 7f7776e54640 1 --2- 172.21.15.105:0/1093537178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77700ad850 0x7f77700701d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f777006e500 tx=0x7f776801c910 comp rx=0 tx=0).stop 2023-11-28T18:25:55.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.285+0000 7f7776e54640 1 -- 172.21.15.105:0/1093537178 shutdown_connections 2023-11-28T18:25:55.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.285+0000 7f7776e54640 1 --2- 172.21.15.105:0/1093537178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77700ad850 0x7f77700701d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:55.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.285+0000 7f7776e54640 1 --2- 172.21.15.105:0/1093537178 >> v2:172.21.15.148:3300/0 conn(0x7f7770159f40 0x7f77700caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:55.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.285+0000 7f7776e54640 1 -- 172.21.15.105:0/1093537178 >> 172.21.15.105:0/1093537178 conn(0x7f77700ceb90 msgr2=0x7f77700cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:55.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.285+0000 7f7776e54640 1 -- 172.21.15.105:0/1093537178 shutdown_connections 2023-11-28T18:25:55.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.285+0000 7f7776e54640 1 -- 172.21.15.105:0/1093537178 wait complete. 2023-11-28T18:25:55.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.285+0000 7f7776e54640 1 Processor -- start 2023-11-28T18:25:55.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.285+0000 7f7776e54640 1 -- start start 2023-11-28T18:25:55.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.285+0000 7f7776e54640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77700ad850 0x7f777015a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:55.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.285+0000 7f7776e54640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f77700cc760 0x7f777015c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:55.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.285+0000 7f7776e54640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7770159f40 0x7f7770149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:55.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.285+0000 7f7776e54640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777006e1c0 con 0x7f7770159f40 2023-11-28T18:25:55.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.285+0000 7f7776e54640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777015cf80 con 0x7f77700ad850 2023-11-28T18:25:55.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.285+0000 7f7776e54640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777015d260 con 0x7f77700cc760 2023-11-28T18:25:55.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.285+0000 7f77753f1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7770159f40 0x7f7770149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:55.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.285+0000 7f77753f1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7770159f40 0x7f7770149930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55332/0 (socket says 172.21.15.105:55332) 2023-11-28T18:25:55.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.285+0000 7f77753f1640 1 -- 172.21.15.105:0/817885414 learned_addr learned my addr 172.21.15.105:0/817885414 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:55.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.289+0000 7f7767fff640 1 --2- 172.21.15.105:0/817885414 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f77700cc760 0x7f777015c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:55.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.289+0000 7f7774bf0640 1 --2- 172.21.15.105:0/817885414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77700ad850 0x7f777015a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:55.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.289+0000 7f77753f1640 1 -- 172.21.15.105:0/817885414 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f77700cc760 msgr2=0x7f777015c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:55.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.289+0000 7f77753f1640 1 --2- 172.21.15.105:0/817885414 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f77700cc760 0x7f777015c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:55.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.289+0000 7f77753f1640 1 -- 172.21.15.105:0/817885414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77700ad850 msgr2=0x7f777015a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:55.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.289+0000 7f77753f1640 1 --2- 172.21.15.105:0/817885414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77700ad850 0x7f777015a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:55.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.289+0000 7f77753f1640 1 -- 172.21.15.105:0/817885414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7768009b30 con 0x7f7770159f40 2023-11-28T18:25:55.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.289+0000 7f7774bf0640 1 --2- 172.21.15.105:0/817885414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77700ad850 0x7f777015a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:55.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.289+0000 7f77753f1640 1 --2- 172.21.15.105:0/817885414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7770159f40 0x7f7770149930 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f77680af040 tx=0x7f77680196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:55.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.289+0000 7f77657fa640 1 -- 172.21.15.105:0/817885414 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77680b8670 con 0x7f7770159f40 2023-11-28T18:25:55.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.289+0000 7f77657fa640 1 -- 172.21.15.105:0/817885414 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77680b87d0 con 0x7f7770159f40 2023-11-28T18:25:55.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.289+0000 7f77657fa640 1 -- 172.21.15.105:0/817885414 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77680b8b60 con 0x7f7770159f40 2023-11-28T18:25:55.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.289+0000 7f7776e54640 1 -- 172.21.15.105:0/817885414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f777006de50 con 0x7f7770159f40 2023-11-28T18:25:55.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.289+0000 7f7776e54640 1 -- 172.21.15.105:0/817885414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f777015aaa0 con 0x7f7770159f40 2023-11-28T18:25:55.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.289+0000 7f7776e54640 1 -- 172.21.15.105:0/817885414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f777006e330 con 0x7f7770159f40 2023-11-28T18:25:55.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.289+0000 7f77657fa640 1 -- 172.21.15.105:0/817885414 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f77680b8cc0 con 0x7f7770159f40 2023-11-28T18:25:55.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.293+0000 7f77657fa640 1 --2- 172.21.15.105:0/817885414 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f775c0668d0 0x7f775c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:55.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.293+0000 7f77657fa640 1 -- 172.21.15.105:0/817885414 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f776811fa10 con 0x7f7770159f40 2023-11-28T18:25:55.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.293+0000 7f7774bf0640 1 --2- 172.21.15.105:0/817885414 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f775c0668d0 0x7f775c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:55.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.293+0000 7f7774bf0640 1 --2- 172.21.15.105:0/817885414 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f775c0668d0 0x7f775c068d90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f776c00b990 tx=0x7f776c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:55.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.297+0000 7f77657fa640 1 -- 172.21.15.105:0/817885414 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f77680ecd80 con 0x7f7770159f40 2023-11-28T18:25:55.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.445+0000 7f7776e54640 1 -- 172.21.15.105:0/817885414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7770067570 con 0x7f7770159f40 2023-11-28T18:25:55.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.449+0000 7f77657fa640 1 -- 172.21.15.105:0/817885414 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v19)=0 v19) v1 ==== 122+0+0 (secure 0 0 0) 0x7f77680f03a0 con 0x7f7770159f40 2023-11-28T18:25:55.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.453+0000 7f7776e54640 1 -- 172.21.15.105:0/817885414 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f775c0668d0 msgr2=0x7f775c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:55.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.453+0000 7f7776e54640 1 --2- 172.21.15.105:0/817885414 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f775c0668d0 0x7f775c068d90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f776c00b990 tx=0x7f776c008040 comp rx=0 tx=0).stop 2023-11-28T18:25:55.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.453+0000 7f7776e54640 1 -- 172.21.15.105:0/817885414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7770159f40 msgr2=0x7f7770149930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:55.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.453+0000 7f7776e54640 1 --2- 172.21.15.105:0/817885414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7770159f40 0x7f7770149930 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f77680af040 tx=0x7f77680196b0 comp rx=0 tx=0).stop 2023-11-28T18:25:55.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.453+0000 7f7776e54640 1 -- 172.21.15.105:0/817885414 shutdown_connections 2023-11-28T18:25:55.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.453+0000 7f7776e54640 1 --2- 172.21.15.105:0/817885414 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f775c0668d0 0x7f775c068d90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:55.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.453+0000 7f7776e54640 1 --2- 172.21.15.105:0/817885414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7770159f40 0x7f7770149930 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:55.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.453+0000 7f7776e54640 1 --2- 172.21.15.105:0/817885414 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f77700cc760 0x7f777015c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:55.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.453+0000 7f7776e54640 1 --2- 172.21.15.105:0/817885414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77700ad850 0x7f777015a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:55.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.453+0000 7f7776e54640 1 -- 172.21.15.105:0/817885414 >> 172.21.15.105:0/817885414 conn(0x7f77700ceb90 msgr2=0x7f777014b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:55.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.453+0000 7f7776e54640 1 -- 172.21.15.105:0/817885414 shutdown_connections 2023-11-28T18:25:55.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.453+0000 7f7776e54640 1 -- 172.21.15.105:0/817885414 wait complete. 2023-11-28T18:25:55.474 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T18:25:55.475 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:25:55.591 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:25:55.668 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T18:25:55.669 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:25:55.669 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:25:55.679 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:25:55.679 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:25:55.679 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:25:55.689 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:25:55.690 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:25:55.690 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:25:55.730 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:25:55.730 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:25:55.730 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:25:55.743 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:25:55.744 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T18:25:55.744 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:25:55.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.909+0000 7f687830f640 1 Processor -- start 2023-11-28T18:25:55.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.909+0000 7f687830f640 1 -- start start 2023-11-28T18:25:55.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.909+0000 7f687830f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f687015d4f0 0x7f68700d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:55.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.909+0000 7f687830f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68700b2d20 0x7f68700b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:55.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.909+0000 7f687830f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68700b25b0 con 0x7f68700d3c30 2023-11-28T18:25:55.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.909+0000 7f687830f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68700b2900 con 0x7f687015d4f0 2023-11-28T18:25:55.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.909+0000 7f687830f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68700d0a20 con 0x7f68700b2d20 2023-11-28T18:25:55.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.909+0000 7f68758aa640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f68700d3c30 0x7f68700d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:49920/0 (socket says 172.21.15.105:49920) 2023-11-28T18:25:55.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.909+0000 7f68760ab640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f687015d4f0 0x7f68700d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:55.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.909+0000 7f68758aa640 1 -- 172.21.15.105:0/2823523468 learned_addr learned my addr 172.21.15.105:0/2823523468 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:55.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.909+0000 7f68768ac640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68700b2d20 0x7f68700b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:55.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.909+0000 7f68750a9640 1 -- 172.21.15.105:0/2823523468 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3821711750 0 0) 0x7f68700b25b0 con 0x7f68700d3c30 2023-11-28T18:25:55.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.909+0000 7f68750a9640 1 -- 172.21.15.105:0/2823523468 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f685c003530 con 0x7f68700d3c30 2023-11-28T18:25:55.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.909+0000 7f68768ac640 1 -- 172.21.15.105:0/2823523468 >> v2:172.21.15.148:3300/0 conn(0x7f687015d4f0 msgr2=0x7f68700d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:55.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.909+0000 7f68768ac640 1 --2- 172.21.15.105:0/2823523468 >> v2:172.21.15.148:3300/0 conn(0x7f687015d4f0 0x7f68700d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:55.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.909+0000 7f68768ac640 1 -- 172.21.15.105:0/2823523468 >> v1:172.21.15.148:6789/0 conn(0x7f68700d3c30 legacy=0x7f68700d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:55.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.909+0000 7f68768ac640 1 -- 172.21.15.105:0/2823523468 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68700cea30 con 0x7f68700b2d20 2023-11-28T18:25:55.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.909+0000 7f68760ab640 1 --2- 172.21.15.105:0/2823523468 >> v2:172.21.15.148:3300/0 conn(0x7f687015d4f0 0x7f68700d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:25:55.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.913+0000 7f68768ac640 1 --2- 172.21.15.105:0/2823523468 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68700b2d20 0x7f68700b3190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6868009d90 tx=0x7f686801c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa301083ea5c24ec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:55.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.913+0000 7f68750a9640 1 -- 172.21.15.105:0/2823523468 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68680a5950 con 0x7f68700b2d20 2023-11-28T18:25:55.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.913+0000 7f68750a9640 1 -- 172.21.15.105:0/2823523468 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68680a5ab0 con 0x7f68700b2d20 2023-11-28T18:25:55.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.913+0000 7f68750a9640 1 -- 172.21.15.105:0/2823523468 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68680a5e20 con 0x7f68700b2d20 2023-11-28T18:25:55.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.913+0000 7f687830f640 1 -- 172.21.15.105:0/2823523468 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68700b2d20 msgr2=0x7f68700b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:55.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.913+0000 7f687830f640 1 --2- 172.21.15.105:0/2823523468 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68700b2d20 0x7f68700b3190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6868009d90 tx=0x7f686801c890 comp rx=0 tx=0).stop 2023-11-28T18:25:55.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.913+0000 7f687830f640 1 -- 172.21.15.105:0/2823523468 shutdown_connections 2023-11-28T18:25:55.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.913+0000 7f687830f640 1 --2- 172.21.15.105:0/2823523468 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68700b2d20 0x7f68700b3190 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:55.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.913+0000 7f687830f640 1 --2- 172.21.15.105:0/2823523468 >> v2:172.21.15.148:3300/0 conn(0x7f687015d4f0 0x7f68700d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:55.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.913+0000 7f687830f640 1 -- 172.21.15.105:0/2823523468 >> 172.21.15.105:0/2823523468 conn(0x7f68700d3030 msgr2=0x7f68700d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:55.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.913+0000 7f687830f640 1 -- 172.21.15.105:0/2823523468 shutdown_connections 2023-11-28T18:25:55.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.913+0000 7f687830f640 1 -- 172.21.15.105:0/2823523468 wait complete. 2023-11-28T18:25:55.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f687830f640 1 Processor -- start 2023-11-28T18:25:55.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f687830f640 1 -- start start 2023-11-28T18:25:55.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f687830f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68700b2d20 0x7f68700c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:55.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f687830f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68700d3c30 0x7f687014f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:55.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f687830f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f687015d4f0 0x7f68700b8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:55.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f687830f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68700ccfa0 con 0x7f68700b2d20 2023-11-28T18:25:55.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f687830f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68700cc420 con 0x7f68700d3c30 2023-11-28T18:25:55.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f687830f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68700cc700 con 0x7f687015d4f0 2023-11-28T18:25:55.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f68760ab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68700b2d20 0x7f68700c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:55.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f68760ab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68700b2d20 0x7f68700c99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55342/0 (socket says 172.21.15.105:55342) 2023-11-28T18:25:55.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f68760ab640 1 -- 172.21.15.105:0/541524402 learned_addr learned my addr 172.21.15.105:0/541524402 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:55.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f68758aa640 1 --2- 172.21.15.105:0/541524402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68700d3c30 0x7f687014f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:55.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f68768ac640 1 --2- 172.21.15.105:0/541524402 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f687015d4f0 0x7f68700b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:55.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f68760ab640 1 -- 172.21.15.105:0/541524402 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f687015d4f0 msgr2=0x7f68700b8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:55.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f68760ab640 1 --2- 172.21.15.105:0/541524402 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f687015d4f0 0x7f68700b8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:55.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f68760ab640 1 -- 172.21.15.105:0/541524402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68700d3c30 msgr2=0x7f687014f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:55.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f68760ab640 1 --2- 172.21.15.105:0/541524402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68700d3c30 0x7f687014f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:55.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f68760ab640 1 -- 172.21.15.105:0/541524402 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686c009420 con 0x7f68700b2d20 2023-11-28T18:25:55.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f68760ab640 1 --2- 172.21.15.105:0/541524402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68700b2d20 0x7f68700c99c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f686c00e980 tx=0x7f686c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:55.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f6866ffd640 1 -- 172.21.15.105:0/541524402 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c00ca00 con 0x7f68700b2d20 2023-11-28T18:25:55.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f6866ffd640 1 -- 172.21.15.105:0/541524402 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686c00cb60 con 0x7f68700b2d20 2023-11-28T18:25:55.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f6866ffd640 1 -- 172.21.15.105:0/541524402 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c013470 con 0x7f68700b2d20 2023-11-28T18:25:55.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f687830f640 1 -- 172.21.15.105:0/541524402 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6868009a70 con 0x7f68700b2d20 2023-11-28T18:25:55.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.917+0000 7f687830f640 1 -- 172.21.15.105:0/541524402 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68700cccc0 con 0x7f68700b2d20 2023-11-28T18:25:55.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.921+0000 7f687830f640 1 -- 172.21.15.105:0/541524402 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6870067570 con 0x7f68700b2d20 2023-11-28T18:25:55.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.921+0000 7f6866ffd640 1 -- 172.21.15.105:0/541524402 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f686c0135d0 con 0x7f68700b2d20 2023-11-28T18:25:55.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.921+0000 7f6866ffd640 1 --2- 172.21.15.105:0/541524402 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f68540668d0 0x7f6854068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:55.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.921+0000 7f6866ffd640 1 -- 172.21.15.105:0/541524402 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f686c084500 con 0x7f68700b2d20 2023-11-28T18:25:55.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.921+0000 7f68758aa640 1 --2- 172.21.15.105:0/541524402 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f68540668d0 0x7f6854068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:55.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.921+0000 7f68758aa640 1 --2- 172.21.15.105:0/541524402 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f68540668d0 0x7f6854068d90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6860007fd0 tx=0x7f6860009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:55.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:55.925+0000 7f6866ffd640 1 -- 172.21.15.105:0/541524402 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f686c0518f0 con 0x7f68700b2d20 2023-11-28T18:25:56.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.105+0000 7f687830f640 1 -- 172.21.15.105:0/541524402 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6870156b00 con 0x7f68700b2d20 2023-11-28T18:25:56.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.109+0000 7f6866ffd640 1 -- 172.21.15.105:0/541524402 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f686c021e90 con 0x7f68700b2d20 2023-11-28T18:25:56.109 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:56.109 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5339,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1754209821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":1754209821},{"type":"v1","addr":"172.21.15.148:6835","nonce":1754209821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5345,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2703548137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2703548137},{"type":"v1","addr":"172.21.15.105:6837","nonce":2703548137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/47307750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":47307750},{"type":"v1","addr":"172.21.15.148:6834","nonce":47307750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5737,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/1011802325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1011802325},{"type":"v1","addr":"172.21.15.105:6836","nonce":1011802325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[]} 2023-11-28T18:25:56.110 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 28 2023-11-28T18:25:56.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.109+0000 7f687830f640 1 -- 172.21.15.105:0/541524402 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f68540668d0 msgr2=0x7f6854068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:56.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.109+0000 7f687830f640 1 --2- 172.21.15.105:0/541524402 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f68540668d0 0x7f6854068d90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6860007fd0 tx=0x7f6860009040 comp rx=0 tx=0).stop 2023-11-28T18:25:56.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.109+0000 7f687830f640 1 -- 172.21.15.105:0/541524402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68700b2d20 msgr2=0x7f68700c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:56.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.109+0000 7f687830f640 1 --2- 172.21.15.105:0/541524402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68700b2d20 0x7f68700c99c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f686c00e980 tx=0x7f686c00ee40 comp rx=0 tx=0).stop 2023-11-28T18:25:56.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.109+0000 7f687830f640 1 -- 172.21.15.105:0/541524402 shutdown_connections 2023-11-28T18:25:56.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.109+0000 7f687830f640 1 --2- 172.21.15.105:0/541524402 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f687015d4f0 0x7f68700b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:56.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.109+0000 7f687830f640 1 --2- 172.21.15.105:0/541524402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68700d3c30 0x7f687014f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:56.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.109+0000 7f687830f640 1 --2- 172.21.15.105:0/541524402 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f68540668d0 0x7f6854068d90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:56.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.109+0000 7f687830f640 1 --2- 172.21.15.105:0/541524402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68700b2d20 0x7f68700c99c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:56.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.109+0000 7f687830f640 1 -- 172.21.15.105:0/541524402 >> 172.21.15.105:0/541524402 conn(0x7f68700d3030 msgr2=0x7f687015c750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:56.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.109+0000 7f687830f640 1 -- 172.21.15.105:0/541524402 shutdown_connections 2023-11-28T18:25:56.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.109+0000 7f687830f640 1 -- 172.21.15.105:0/541524402 wait complete. 2023-11-28T18:25:56.132 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T18:25:56.132 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T18:25:56.132 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T18:25:56.133 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:25:56.133 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T18:25:56.133 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T18:25:56.133 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T18:25:56.133 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:25:56.133 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T18:25:56.133 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T18:25:56.134 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T18:25:56.134 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:25:56.134 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T18:25:56.134 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T18:25:56.134 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T18:25:56.134 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:25:56.147 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T18:25:56.148 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T18:25:56.148 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T18:25:56.149 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T18:25:56.150 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T18:25:56.150 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T18:25:56.151 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T18:25:56.151 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T18:25:56.196 INFO:tasks.ceph.mds.a:Started 2023-11-28T18:25:56.197 DEBUG:teuthology.parallel:result is None 2023-11-28T18:25:56.199 INFO:tasks.ceph.mds.c:Started 2023-11-28T18:25:56.199 DEBUG:teuthology.parallel:result is None 2023-11-28T18:25:56.201 INFO:tasks.ceph.mds.b:Started 2023-11-28T18:25:56.201 DEBUG:teuthology.parallel:result is None 2023-11-28T18:25:56.203 INFO:tasks.ceph.mds.d:Started 2023-11-28T18:25:56.203 DEBUG:teuthology.parallel:result is None 2023-11-28T18:25:56.203 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T18:25:56.284 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2023-11-28T18:25:56.285 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2023-11-28T18:25:56.295 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2023-11-28T18:25:56.295 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2023-11-28T18:25:56.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.349+0000 7f2429b1d640 1 Processor -- start 2023-11-28T18:25:56.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.349+0000 7f2429b1d640 1 -- start start 2023-11-28T18:25:56.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.349+0000 7f2429b1d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f242415cfc0 0x7f242406dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:56.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.349+0000 7f2429b1d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f24240c9720 0x7f2424070660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:56.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.349+0000 7f2429b1d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24240cb010 con 0x7f242406c620 2023-11-28T18:25:56.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.349+0000 7f2429b1d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f242415e620 con 0x7f242415cfc0 2023-11-28T18:25:56.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.349+0000 7f2429b1d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24240c9c20 con 0x7f24240c9720 2023-11-28T18:25:56.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.349+0000 7f2423fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f242406c620 0x7f24240cba90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39074/0 (socket says 172.21.15.105:39074) 2023-11-28T18:25:56.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.349+0000 7f2423fff640 1 -- 172.21.15.105:0/653547959 learned_addr learned my addr 172.21.15.105:0/653547959 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:56.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.349+0000 7f24237fe640 1 --2- 172.21.15.105:0/653547959 >> v2:172.21.15.105:3300/0 conn(0x7f242415cfc0 0x7f242406dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:56.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.353+0000 7f2422ffd640 1 --2- 172.21.15.105:0/653547959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f24240c9720 0x7f2424070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:56.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.353+0000 7f24227fc640 1 -- 172.21.15.105:0/653547959 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473897163 0 0) 0x7f24240cb010 con 0x7f242406c620 2023-11-28T18:25:56.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.353+0000 7f24227fc640 1 -- 172.21.15.105:0/653547959 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f240c003530 con 0x7f242406c620 2023-11-28T18:25:56.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.353+0000 7f24237fe640 1 -- 172.21.15.105:0/653547959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f24240c9720 msgr2=0x7f2424070660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:56.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.353+0000 7f24237fe640 1 --2- 172.21.15.105:0/653547959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f24240c9720 0x7f2424070660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:56.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.353+0000 7f24237fe640 1 -- 172.21.15.105:0/653547959 >> v1:172.21.15.105:6789/0 conn(0x7f242406c620 legacy=0x7f24240cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:56.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.353+0000 7f24237fe640 1 -- 172.21.15.105:0/653547959 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242415e340 con 0x7f242415cfc0 2023-11-28T18:25:56.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.353+0000 7f24237fe640 1 --2- 172.21.15.105:0/653547959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242415cfc0 0x7f242406dce0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f24140097c0 tx=0x7f241401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbfd274162537de server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:56.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.353+0000 7f24227fc640 1 -- 172.21.15.105:0/653547959 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24140ab650 con 0x7f242415cfc0 2023-11-28T18:25:56.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.353+0000 7f24227fc640 1 -- 172.21.15.105:0/653547959 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24140ab7b0 con 0x7f242415cfc0 2023-11-28T18:25:56.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.353+0000 7f24227fc640 1 -- 172.21.15.105:0/653547959 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24140abb40 con 0x7f242415cfc0 2023-11-28T18:25:56.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f2429b1d640 1 -- 172.21.15.105:0/653547959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242415cfc0 msgr2=0x7f242406dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:56.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f2429b1d640 1 --2- 172.21.15.105:0/653547959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242415cfc0 0x7f242406dce0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f24140097c0 tx=0x7f241401c450 comp rx=0 tx=0).stop 2023-11-28T18:25:56.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f2429b1d640 1 -- 172.21.15.105:0/653547959 shutdown_connections 2023-11-28T18:25:56.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f2429b1d640 1 --2- 172.21.15.105:0/653547959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f24240c9720 0x7f2424070660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:56.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f2429b1d640 1 --2- 172.21.15.105:0/653547959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242415cfc0 0x7f242406dce0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:56.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f2429b1d640 1 -- 172.21.15.105:0/653547959 >> 172.21.15.105:0/653547959 conn(0x7f24240b0c60 msgr2=0x7f24240b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:56.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f2429b1d640 1 -- 172.21.15.105:0/653547959 shutdown_connections 2023-11-28T18:25:56.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f2429b1d640 1 -- 172.21.15.105:0/653547959 wait complete. 2023-11-28T18:25:56.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f2429b1d640 1 Processor -- start 2023-11-28T18:25:56.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f2429b1d640 1 -- start start 2023-11-28T18:25:56.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f2429b1d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242406c620 0x7f2424158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:56.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f2429b1d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24240c9720 0x7f24240b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:56.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f2429b1d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f242415cfc0 0x7f24240c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:56.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f2429b1d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24240b38c0 con 0x7f242406c620 2023-11-28T18:25:56.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f2429b1d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24240b2f20 con 0x7f24240c9720 2023-11-28T18:25:56.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f2429b1d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24240b3200 con 0x7f242415cfc0 2023-11-28T18:25:56.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f24237fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242406c620 0x7f2424158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:56.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f24237fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242406c620 0x7f2424158cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55382/0 (socket says 172.21.15.105:55382) 2023-11-28T18:25:56.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f24237fe640 1 -- 172.21.15.105:0/659776083 learned_addr learned my addr 172.21.15.105:0/659776083 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:56.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f24237fe640 1 -- 172.21.15.105:0/659776083 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f242415cfc0 msgr2=0x7f24240c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:56.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f24237fe640 1 --2- 172.21.15.105:0/659776083 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f242415cfc0 0x7f24240c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:56.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f24237fe640 1 -- 172.21.15.105:0/659776083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24240c9720 msgr2=0x7f24240b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:56.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f24237fe640 1 --2- 172.21.15.105:0/659776083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24240c9720 0x7f24240b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:56.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.357+0000 7f24237fe640 1 -- 172.21.15.105:0/659776083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2414009420 con 0x7f242406c620 2023-11-28T18:25:56.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.361+0000 7f24237fe640 1 --2- 172.21.15.105:0/659776083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242406c620 0x7f2424158cd0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f24140ad040 tx=0x7f241401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:56.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.361+0000 7f240bfff640 1 -- 172.21.15.105:0/659776083 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24140bf610 con 0x7f242406c620 2023-11-28T18:25:56.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.361+0000 7f240bfff640 1 -- 172.21.15.105:0/659776083 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24140bf790 con 0x7f242406c620 2023-11-28T18:25:56.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.361+0000 7f240bfff640 1 -- 172.21.15.105:0/659776083 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24140bfb40 con 0x7f242406c620 2023-11-28T18:25:56.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.361+0000 7f2429b1d640 1 -- 172.21.15.105:0/659776083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f242406cfc0 con 0x7f242406c620 2023-11-28T18:25:56.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.361+0000 7f2429b1d640 1 -- 172.21.15.105:0/659776083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2424159680 con 0x7f242406c620 2023-11-28T18:25:56.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.361+0000 7f2429b1d640 1 -- 172.21.15.105:0/659776083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24240b3a00 con 0x7f242406c620 2023-11-28T18:25:56.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.361+0000 7f240bfff640 1 -- 172.21.15.105:0/659776083 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f24140b5400 con 0x7f242406c620 2023-11-28T18:25:56.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.361+0000 7f240bfff640 1 --2- 172.21.15.105:0/659776083 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f24100668d0 0x7f2410068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:56.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.361+0000 7f240bfff640 1 -- 172.21.15.105:0/659776083 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f241411eed0 con 0x7f242406c620 2023-11-28T18:25:56.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.361+0000 7f2422ffd640 1 --2- 172.21.15.105:0/659776083 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f24100668d0 0x7f2410068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:56.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.365+0000 7f2422ffd640 1 --2- 172.21.15.105:0/659776083 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f24100668d0 0x7f2410068d90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f24240cc170 tx=0x7f2418008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:56.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.365+0000 7f240bfff640 1 -- 172.21.15.105:0/659776083 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f24240b3a00 con 0x7f242406c620 2023-11-28T18:25:56.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:56.525+0000 7f2429b1d640 1 -- 172.21.15.105:0/659776083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f24240b3a00 con 0x7f242406c620 2023-11-28T18:25:57.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.125+0000 7f240bfff640 1 -- 172.21.15.105:0/659776083 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v45) v1 ==== 97+0+0 (secure 0 0 0) 0x7f24140ec3d0 con 0x7f242406c620 2023-11-28T18:25:57.128 INFO:teuthology.orchestra.run.smithi105.stderr: removed all blocklist entries 2023-11-28T18:25:57.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.129+0000 7f2429b1d640 1 -- 172.21.15.105:0/659776083 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f24100668d0 msgr2=0x7f2410068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:57.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.129+0000 7f2429b1d640 1 --2- 172.21.15.105:0/659776083 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f24100668d0 0x7f2410068d90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f24240cc170 tx=0x7f2418008040 comp rx=0 tx=0).stop 2023-11-28T18:25:57.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.129+0000 7f2429b1d640 1 -- 172.21.15.105:0/659776083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242406c620 msgr2=0x7f2424158cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:57.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.129+0000 7f2429b1d640 1 --2- 172.21.15.105:0/659776083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242406c620 0x7f2424158cd0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f24140ad040 tx=0x7f241401ac90 comp rx=0 tx=0).stop 2023-11-28T18:25:57.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.129+0000 7f2429b1d640 1 -- 172.21.15.105:0/659776083 shutdown_connections 2023-11-28T18:25:57.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.129+0000 7f2429b1d640 1 --2- 172.21.15.105:0/659776083 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f242415cfc0 0x7f24240c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.129+0000 7f2429b1d640 1 --2- 172.21.15.105:0/659776083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24240c9720 0x7f24240b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.129+0000 7f2429b1d640 1 --2- 172.21.15.105:0/659776083 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f24100668d0 0x7f2410068d90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.129+0000 7f2429b1d640 1 --2- 172.21.15.105:0/659776083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242406c620 0x7f2424158cd0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.129+0000 7f2429b1d640 1 -- 172.21.15.105:0/659776083 >> 172.21.15.105:0/659776083 conn(0x7f24240b0c60 msgr2=0x7f24240b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:57.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.129+0000 7f2429b1d640 1 -- 172.21.15.105:0/659776083 shutdown_connections 2023-11-28T18:25:57.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.129+0000 7f2429b1d640 1 -- 172.21.15.105:0/659776083 wait complete. 2023-11-28T18:25:57.159 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T18:25:57.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.297+0000 7f4d8f25a640 1 Processor -- start 2023-11-28T18:25:57.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.297+0000 7f4d8f25a640 1 -- start start 2023-11-28T18:25:57.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.297+0000 7f4d8f25a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d880721d0 0x7f4d8806ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:57.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.297+0000 7f4d8f25a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4d8806a580 0x7f4d8806aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:57.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.297+0000 7f4d8f25a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d880b2ee0 con 0x7f4d880af340 2023-11-28T18:25:57.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.297+0000 7f4d8f25a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d880b34a0 con 0x7f4d8806a580 2023-11-28T18:25:57.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.297+0000 7f4d8f25a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8806f240 con 0x7f4d880721d0 2023-11-28T18:25:57.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.297+0000 7f4d8cff6640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4d880af340 0x7f4d880d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39078/0 (socket says 172.21.15.105:39078) 2023-11-28T18:25:57.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.297+0000 7f4d8cff6640 1 -- 172.21.15.105:0/1691309624 learned_addr learned my addr 172.21.15.105:0/1691309624 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:57.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.297+0000 7f4d8d7f7640 1 --2- 172.21.15.105:0/1691309624 >> v2:172.21.15.105:3300/0 conn(0x7f4d8806a580 0x7f4d8806aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:57.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.297+0000 7f4d7ffff640 1 --2- 172.21.15.105:0/1691309624 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d880721d0 0x7f4d8806ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:57.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.297+0000 7f4d7f7fe640 1 -- 172.21.15.105:0/1691309624 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3091736114 0 0) 0x7f4d880b2ee0 con 0x7f4d880af340 2023-11-28T18:25:57.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.297+0000 7f4d7f7fe640 1 -- 172.21.15.105:0/1691309624 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d70003530 con 0x7f4d880af340 2023-11-28T18:25:57.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.297+0000 7f4d8d7f7640 1 -- 172.21.15.105:0/1691309624 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d880721d0 msgr2=0x7f4d8806ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:57.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.297+0000 7f4d8d7f7640 1 --2- 172.21.15.105:0/1691309624 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d880721d0 0x7f4d8806ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.297+0000 7f4d8d7f7640 1 -- 172.21.15.105:0/1691309624 >> v1:172.21.15.105:6789/0 conn(0x7f4d880af340 legacy=0x7f4d880d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:57.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.297+0000 7f4d8d7f7640 1 -- 172.21.15.105:0/1691309624 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d880b31c0 con 0x7f4d8806a580 2023-11-28T18:25:57.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.297+0000 7f4d7ffff640 1 --2- 172.21.15.105:0/1691309624 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d880721d0 0x7f4d8806ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:57.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.301+0000 7f4d8d7f7640 1 --2- 172.21.15.105:0/1691309624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d8806a580 0x7f4d8806aa10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009d90 tx=0x7f4d8001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b0559f2ecac862 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:57.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.301+0000 7f4d7f7fe640 1 -- 172.21.15.105:0/1691309624 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d800a5810 con 0x7f4d8806a580 2023-11-28T18:25:57.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.301+0000 7f4d7f7fe640 1 -- 172.21.15.105:0/1691309624 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d800a5970 con 0x7f4d8806a580 2023-11-28T18:25:57.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.301+0000 7f4d7f7fe640 1 -- 172.21.15.105:0/1691309624 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d800a5d00 con 0x7f4d8806a580 2023-11-28T18:25:57.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.301+0000 7f4d8f25a640 1 -- 172.21.15.105:0/1691309624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d8806a580 msgr2=0x7f4d8806aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:57.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.301+0000 7f4d8f25a640 1 --2- 172.21.15.105:0/1691309624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d8806a580 0x7f4d8806aa10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009d90 tx=0x7f4d8001c950 comp rx=0 tx=0).stop 2023-11-28T18:25:57.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.301+0000 7f4d8f25a640 1 -- 172.21.15.105:0/1691309624 shutdown_connections 2023-11-28T18:25:57.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.301+0000 7f4d8f25a640 1 --2- 172.21.15.105:0/1691309624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d8806a580 0x7f4d8806aa10 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.301+0000 7f4d8f25a640 1 --2- 172.21.15.105:0/1691309624 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d880721d0 0x7f4d8806ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.301+0000 7f4d8f25a640 1 -- 172.21.15.105:0/1691309624 >> 172.21.15.105:0/1691309624 conn(0x7f4d8806b860 msgr2=0x7f4d8806bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:57.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.301+0000 7f4d8f25a640 1 -- 172.21.15.105:0/1691309624 shutdown_connections 2023-11-28T18:25:57.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.301+0000 7f4d8f25a640 1 -- 172.21.15.105:0/1691309624 wait complete. 2023-11-28T18:25:57.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d8f25a640 1 Processor -- start 2023-11-28T18:25:57.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d8f25a640 1 -- start start 2023-11-28T18:25:57.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d8f25a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d8806a580 0x7f4d88161d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:57.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d8f25a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d880721d0 0x7f4d8815ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:57.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d8f25a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d880af340 0x7f4d88140ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:57.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d8f25a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d88166d30 con 0x7f4d880af340 2023-11-28T18:25:57.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d8f25a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d88141680 con 0x7f4d880721d0 2023-11-28T18:25:57.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d8f25a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d88142c70 con 0x7f4d8806a580 2023-11-28T18:25:57.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d8d7f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d880af340 0x7f4d88140ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:57.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d8d7f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d880af340 0x7f4d88140ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55390/0 (socket says 172.21.15.105:55390) 2023-11-28T18:25:57.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d8d7f7640 1 -- 172.21.15.105:0/3904183866 learned_addr learned my addr 172.21.15.105:0/3904183866 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:57.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d8cff6640 1 --2- 172.21.15.105:0/3904183866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d8806a580 0x7f4d88161d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:57.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d7ffff640 1 --2- 172.21.15.105:0/3904183866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d880721d0 0x7f4d8815ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:57.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d8d7f7640 1 -- 172.21.15.105:0/3904183866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d8806a580 msgr2=0x7f4d88161d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:57.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d8d7f7640 1 --2- 172.21.15.105:0/3904183866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d8806a580 0x7f4d88161d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d8d7f7640 1 -- 172.21.15.105:0/3904183866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d880721d0 msgr2=0x7f4d8815ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:57.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d8d7f7640 1 --2- 172.21.15.105:0/3904183866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d880721d0 0x7f4d8815ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d8d7f7640 1 -- 172.21.15.105:0/3904183866 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d80009a70 con 0x7f4d880af340 2023-11-28T18:25:57.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d7ffff640 1 --2- 172.21.15.105:0/3904183866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d880721d0 0x7f4d8815ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:57.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d8cff6640 1 --2- 172.21.15.105:0/3904183866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d8806a580 0x7f4d88161d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:57.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d8d7f7640 1 --2- 172.21.15.105:0/3904183866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d880af340 0x7f4d88140ba0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4d800b0040 tx=0x7f4d800196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:57.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d7d7fa640 1 -- 172.21.15.105:0/3904183866 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d800b9860 con 0x7f4d880af340 2023-11-28T18:25:57.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d7d7fa640 1 -- 172.21.15.105:0/3904183866 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d800b99c0 con 0x7f4d880af340 2023-11-28T18:25:57.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d8f25a640 1 -- 172.21.15.105:0/3904183866 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d88166490 con 0x7f4d880af340 2023-11-28T18:25:57.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d7d7fa640 1 -- 172.21.15.105:0/3904183866 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d800b9d70 con 0x7f4d880af340 2023-11-28T18:25:57.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.305+0000 7f4d8f25a640 1 -- 172.21.15.105:0/3904183866 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d88166a50 con 0x7f4d880af340 2023-11-28T18:25:57.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.309+0000 7f4d8f25a640 1 -- 172.21.15.105:0/3904183866 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d881417f0 con 0x7f4d880af340 2023-11-28T18:25:57.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.309+0000 7f4d7d7fa640 1 -- 172.21.15.105:0/3904183866 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f4d800ad440 con 0x7f4d880af340 2023-11-28T18:25:57.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.309+0000 7f4d7d7fa640 1 --2- 172.21.15.105:0/3904183866 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d740668d0 0x7f4d74068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:57.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.309+0000 7f4d7d7fa640 1 -- 172.21.15.105:0/3904183866 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4d8011f520 con 0x7f4d880af340 2023-11-28T18:25:57.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.309+0000 7f4d8cff6640 1 --2- 172.21.15.105:0/3904183866 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d740668d0 0x7f4d74068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:57.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.309+0000 7f4d8cff6640 1 --2- 172.21.15.105:0/3904183866 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d740668d0 0x7f4d74068d90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4d840011f0 tx=0x7f4d84009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:57.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.313+0000 7f4d7d7fa640 1 -- 172.21.15.105:0/3904183866 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4d800ec960 con 0x7f4d880af340 2023-11-28T18:25:57.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.513+0000 7f4d8f25a640 1 -- 172.21.15.105:0/3904183866 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4d88067570 con 0x7f4d880af340 2023-11-28T18:25:57.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.513+0000 7f4d7d7fa640 1 -- 172.21.15.105:0/3904183866 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4d800eff80 con 0x7f4d880af340 2023-11-28T18:25:57.517 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:57.517 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:25:57.517 INFO:teuthology.orchestra.run.smithi105.stdout: "auth_dump": [ 2023-11-28T18:25:57.517 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:25:57.517 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.a", 2023-11-28T18:25:57.517 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlOXSTGxAAU5kGNB3tznYsDNG51vCIZA==", 2023-11-28T18:25:57.517 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:25:57.517 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:25:57.518 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:25:57.518 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:25:57.518 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:25:57.518 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:25:57.518 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:25:57.518 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:25:57.518 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.b", 2023-11-28T18:25:57.518 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlrnbWJxAAKefl0YI2MhobnAOIj6WeaQ==", 2023-11-28T18:25:57.518 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:25:57.518 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:25:57.519 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:25:57.519 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:25:57.519 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:25:57.519 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:25:57.519 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:25:57.519 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:25:57.519 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.c", 2023-11-28T18:25:57.519 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2Zl5Y7GIhAA5OvgkNNFaI7LnzSsQVdFsQ==", 2023-11-28T18:25:57.519 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:25:57.519 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:25:57.520 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:25:57.520 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:25:57.520 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:25:57.520 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:25:57.520 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:25:57.520 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:25:57.520 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.d", 2023-11-28T18:25:57.520 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlmfYdLxAAreI7UE1YCjzkj6yl3s5Fwg==", 2023-11-28T18:25:57.520 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:25:57.520 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:25:57.520 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:25:57.520 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:25:57.521 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:25:57.521 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:25:57.521 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:25:57.521 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:25:57.521 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.0", 2023-11-28T18:25:57.521 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDtL2ZlphxQEBAAQ280OXh1zeaotIdVU8oZng==", 2023-11-28T18:25:57.521 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:25:57.521 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:25:57.521 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:25:57.521 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:25:57.521 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:25:57.521 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:25:57.522 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:25:57.522 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.1", 2023-11-28T18:25:57.522 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDwL2ZlezmOFxAARRBIOTkVJYfquIdcpW7sQQ==", 2023-11-28T18:25:57.522 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:25:57.522 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:25:57.522 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:25:57.522 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:25:57.522 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:25:57.522 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:25:57.522 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:25:57.522 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.2", 2023-11-28T18:25:57.523 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDzL2Zl4YcSHxAA2ww2F4TAD/0H/ZJ+aVMGEQ==", 2023-11-28T18:25:57.523 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:25:57.523 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:25:57.523 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:25:57.523 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:25:57.523 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:25:57.523 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:25:57.523 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:25:57.523 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.3", 2023-11-28T18:25:57.523 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD2L2ZlNxWxJRAAImp7TjTF18KwFTZ14DM6EA==", 2023-11-28T18:25:57.523 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:25:57.523 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:25:57.524 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:25:57.524 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:25:57.524 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:25:57.524 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:25:57.524 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:25:57.524 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.4", 2023-11-28T18:25:57.524 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD8L2ZlOAieLxAAz8g+anH8uJRfsttUS41GGQ==", 2023-11-28T18:25:57.524 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:25:57.524 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:25:57.524 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:25:57.524 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:25:57.525 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:25:57.525 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:25:57.525 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:25:57.525 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.5", 2023-11-28T18:25:57.525 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD/L2Zl5PKTNxAAPk+uIodLPjSd0EEMVT6iEw==", 2023-11-28T18:25:57.525 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:25:57.526 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:25:57.526 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:25:57.526 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:25:57.526 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:25:57.526 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:25:57.526 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:25:57.526 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.6", 2023-11-28T18:25:57.526 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQADMGZll+FUAxAAHlym5Z8rd/OF1911ZE2OUg==", 2023-11-28T18:25:57.526 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:25:57.526 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:25:57.527 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:25:57.527 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:25:57.527 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:25:57.527 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:25:57.527 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:25:57.527 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.7", 2023-11-28T18:25:57.527 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQAGMGZl8GO4ChAAVn3N6DnxUcs+ew09QvGo2w==", 2023-11-28T18:25:57.527 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:25:57.527 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:25:57.527 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:25:57.527 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:25:57.527 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:25:57.528 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:25:57.528 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:25:57.528 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.0", 2023-11-28T18:25:57.528 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlSt0VMxAAQ8+j1BgBPMJ2cuiZj8aG0w==", 2023-11-28T18:25:57.528 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:25:57.528 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:25:57.528 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow r", 2023-11-28T18:25:57.528 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-28T18:25:57.528 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:25:57.528 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:25:57.528 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:25:57.528 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.1", 2023-11-28T18:25:57.529 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDqL2ZlfZ1/ARAAhnJB6oj4Cc6p/oCjM6hTCw==", 2023-11-28T18:25:57.529 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:25:57.529 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:25:57.529 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:25:57.529 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:25:57.529 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:25:57.529 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:25:57.529 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:25:57.529 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:25:57.529 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.2", 2023-11-28T18:25:57.529 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlPLtyORAAU2GST6TJzlTfh6YqEb3U3w==", 2023-11-28T18:25:57.529 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:25:57.530 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:25:57.530 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:25:57.530 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:25:57.530 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:25:57.530 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:25:57.530 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:25:57.530 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:25:57.530 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.3", 2023-11-28T18:25:57.530 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDqL2ZlhiFuBxAAKQRxCdtqWpAHy1AVvdj9Sg==", 2023-11-28T18:25:57.530 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:25:57.530 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:25:57.531 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:25:57.531 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:25:57.531 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:25:57.531 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:25:57.531 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:25:57.531 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:25:57.531 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.admin", 2023-11-28T18:25:57.531 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDoL2ZlkuP2OhAAxsdZEF9122vkH53EwB8Qug==", 2023-11-28T18:25:57.531 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:25:57.531 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:25:57.531 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:25:57.531 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:25:57.532 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:25:57.532 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:25:57.532 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:25:57.532 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:25:57.532 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.x", 2023-11-28T18:25:57.532 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlEG49FxAAespsG+QK1WTpbmpV+DIwiw==", 2023-11-28T18:25:57.532 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:25:57.532 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:25:57.532 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2023-11-28T18:25:57.532 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:25:57.532 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:25:57.532 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:25:57.533 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:25:57.533 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.y", 2023-11-28T18:25:57.533 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlU89RExAAwMJp+8zZ6aFEt0KbiB6toA==", 2023-11-28T18:25:57.533 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:25:57.533 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:25:57.533 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2023-11-28T18:25:57.533 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:25:57.533 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:25:57.533 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:25:57.533 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2023-11-28T18:25:57.533 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:25:57.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.517+0000 7f4d8f25a640 1 -- 172.21.15.105:0/3904183866 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d740668d0 msgr2=0x7f4d74068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:57.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.517+0000 7f4d8f25a640 1 --2- 172.21.15.105:0/3904183866 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d740668d0 0x7f4d74068d90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4d840011f0 tx=0x7f4d84009040 comp rx=0 tx=0).stop 2023-11-28T18:25:57.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.517+0000 7f4d8f25a640 1 -- 172.21.15.105:0/3904183866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d880af340 msgr2=0x7f4d88140ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:57.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.517+0000 7f4d8f25a640 1 --2- 172.21.15.105:0/3904183866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d880af340 0x7f4d88140ba0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4d800b0040 tx=0x7f4d800196f0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.517+0000 7f4d8f25a640 1 -- 172.21.15.105:0/3904183866 shutdown_connections 2023-11-28T18:25:57.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.517+0000 7f4d8f25a640 1 --2- 172.21.15.105:0/3904183866 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d740668d0 0x7f4d74068d90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.517+0000 7f4d8f25a640 1 --2- 172.21.15.105:0/3904183866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d880af340 0x7f4d88140ba0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.517+0000 7f4d8f25a640 1 --2- 172.21.15.105:0/3904183866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d880721d0 0x7f4d8815ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.517+0000 7f4d8f25a640 1 --2- 172.21.15.105:0/3904183866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d8806a580 0x7f4d88161d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.517+0000 7f4d8f25a640 1 -- 172.21.15.105:0/3904183866 >> 172.21.15.105:0/3904183866 conn(0x7f4d8806b860 msgr2=0x7f4d8806dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:57.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.517+0000 7f4d8f25a640 1 -- 172.21.15.105:0/3904183866 shutdown_connections 2023-11-28T18:25:57.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.517+0000 7f4d8f25a640 1 -- 172.21.15.105:0/3904183866 wait complete. 2023-11-28T18:25:57.539 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:25:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.673+0000 7fdf7185a640 1 Processor -- start 2023-11-28T18:25:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.673+0000 7fdf7185a640 1 -- start start 2023-11-28T18:25:57.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.673+0000 7fdf7185a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdf6c15d320 0x7fdf6c06c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.673+0000 7fdf7185a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf6c0ce230 0x7fdf6c0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.673+0000 7fdf7185a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdf6c0cdbb0 0x7fdf6c0cef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.673+0000 7fdf7185a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf6c0b3c40 con 0x7fdf6c0cdbb0 2023-11-28T18:25:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.673+0000 7fdf7185a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf6c06ca80 con 0x7fdf6c15d320 2023-11-28T18:25:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.673+0000 7fdf7185a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c15d700 con 0x7fdf6c0ce230 2023-11-28T18:25:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.673+0000 7fdf6b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdf6c0cdbb0 0x7fdf6c0cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.673+0000 7fdf6b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdf6c0cdbb0 0x7fdf6c0cef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55394/0 (socket says 172.21.15.105:55394) 2023-11-28T18:25:57.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.673+0000 7fdf6b7fe640 1 -- 172.21.15.105:0/679081775 learned_addr learned my addr 172.21.15.105:0/679081775 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:57.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.677+0000 7fdf6b7fe640 1 -- 172.21.15.105:0/679081775 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf6c0ce230 msgr2=0x7fdf6c0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.673+0000 7fdf627fc640 1 --2- 172.21.15.105:0/679081775 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf6c0ce230 0x7fdf6c0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.677+0000 7fdf6affd640 1 --2- 172.21.15.105:0/679081775 >> v2:172.21.15.148:3300/0 conn(0x7fdf6c15d320 0x7fdf6c06c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.677+0000 7fdf6b7fe640 1 --2- 172.21.15.105:0/679081775 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf6c0ce230 0x7fdf6c0cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.677+0000 7fdf6b7fe640 1 -- 172.21.15.105:0/679081775 >> v2:172.21.15.148:3300/0 conn(0x7fdf6c15d320 msgr2=0x7fdf6c06c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.677+0000 7fdf6b7fe640 1 --2- 172.21.15.105:0/679081775 >> v2:172.21.15.148:3300/0 conn(0x7fdf6c15d320 0x7fdf6c06c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.677+0000 7fdf6b7fe640 1 -- 172.21.15.105:0/679081775 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6c0cdfc0 con 0x7fdf6c0cdbb0 2023-11-28T18:25:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.677+0000 7fdf627fc640 1 --2- 172.21.15.105:0/679081775 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf6c0ce230 0x7fdf6c0cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:57.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.677+0000 7fdf6affd640 1 --2- 172.21.15.105:0/679081775 >> v2:172.21.15.148:3300/0 conn(0x7fdf6c15d320 0x7fdf6c06c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:57.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.677+0000 7fdf6b7fe640 1 --2- 172.21.15.105:0/679081775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf6c0cdbb0 0x7fdf6c0cef80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c071d40 tx=0x7fdf6401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f638d1a63fbaa99 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:57.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.677+0000 7fdf6a7fc640 1 -- 172.21.15.105:0/679081775 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf640a7070 con 0x7fdf6c0cdbb0 2023-11-28T18:25:57.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.677+0000 7fdf6a7fc640 1 -- 172.21.15.105:0/679081775 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf640ab430 con 0x7fdf6c0cdbb0 2023-11-28T18:25:57.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.677+0000 7fdf6a7fc640 1 -- 172.21.15.105:0/679081775 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf640ab7c0 con 0x7fdf6c0cdbb0 2023-11-28T18:25:57.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.677+0000 7fdf7185a640 1 -- 172.21.15.105:0/679081775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf6c0cdbb0 msgr2=0x7fdf6c0cef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:57.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.677+0000 7fdf7185a640 1 --2- 172.21.15.105:0/679081775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf6c0cdbb0 0x7fdf6c0cef80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c071d40 tx=0x7fdf6401c470 comp rx=0 tx=0).stop 2023-11-28T18:25:57.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.677+0000 7fdf7185a640 1 -- 172.21.15.105:0/679081775 shutdown_connections 2023-11-28T18:25:57.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.677+0000 7fdf7185a640 1 --2- 172.21.15.105:0/679081775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf6c0cdbb0 0x7fdf6c0cef80 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.677+0000 7fdf7185a640 1 --2- 172.21.15.105:0/679081775 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf6c0ce230 0x7fdf6c0cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.677+0000 7fdf7185a640 1 --2- 172.21.15.105:0/679081775 >> v2:172.21.15.148:3300/0 conn(0x7fdf6c15d320 0x7fdf6c06c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.677+0000 7fdf7185a640 1 -- 172.21.15.105:0/679081775 >> 172.21.15.105:0/679081775 conn(0x7fdf6c0d0df0 msgr2=0x7fdf6c0d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:57.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.681+0000 7fdf7185a640 1 -- 172.21.15.105:0/679081775 shutdown_connections 2023-11-28T18:25:57.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.681+0000 7fdf7185a640 1 -- 172.21.15.105:0/679081775 wait complete. 2023-11-28T18:25:57.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.681+0000 7fdf7185a640 1 Processor -- start 2023-11-28T18:25:57.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.681+0000 7fdf7185a640 1 -- start start 2023-11-28T18:25:57.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.681+0000 7fdf7185a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf6c0cdbb0 0x7fdf6c153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:57.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.681+0000 7fdf7185a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf6c0ce230 0x7fdf6c15e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:57.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.681+0000 7fdf7185a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf6c15d320 0x7fdf6c15e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:57.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.681+0000 7fdf7185a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c1567c0 con 0x7fdf6c0cdbb0 2023-11-28T18:25:57.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.681+0000 7fdf7185a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c0b41b0 con 0x7fdf6c15d320 2023-11-28T18:25:57.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.681+0000 7fdf7185a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c0b4490 con 0x7fdf6c0ce230 2023-11-28T18:25:57.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.681+0000 7fdf6affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf6c0cdbb0 0x7fdf6c153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:57.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.681+0000 7fdf6affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf6c0cdbb0 0x7fdf6c153860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55402/0 (socket says 172.21.15.105:55402) 2023-11-28T18:25:57.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.681+0000 7fdf6affd640 1 -- 172.21.15.105:0/3998653482 learned_addr learned my addr 172.21.15.105:0/3998653482 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:57.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.681+0000 7fdf627fc640 1 --2- 172.21.15.105:0/3998653482 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf6c0ce230 0x7fdf6c15e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:57.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.681+0000 7fdf6b7fe640 1 --2- 172.21.15.105:0/3998653482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf6c15d320 0x7fdf6c15e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:57.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.681+0000 7fdf6affd640 1 -- 172.21.15.105:0/3998653482 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf6c0ce230 msgr2=0x7fdf6c15e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:57.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.681+0000 7fdf6affd640 1 --2- 172.21.15.105:0/3998653482 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf6c0ce230 0x7fdf6c15e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.681+0000 7fdf6affd640 1 -- 172.21.15.105:0/3998653482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf6c15d320 msgr2=0x7fdf6c15e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:57.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.681+0000 7fdf6affd640 1 --2- 172.21.15.105:0/3998653482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf6c15d320 0x7fdf6c15e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.681+0000 7fdf6affd640 1 -- 172.21.15.105:0/3998653482 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf640094d0 con 0x7fdf6c0cdbb0 2023-11-28T18:25:57.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.681+0000 7fdf6affd640 1 --2- 172.21.15.105:0/3998653482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf6c0cdbb0 0x7fdf6c153860 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00ce00 tx=0x7fdf5c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:57.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.681+0000 7fdf68ff9640 1 -- 172.21.15.105:0/3998653482 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c00f890 con 0x7fdf6c0cdbb0 2023-11-28T18:25:57.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.685+0000 7fdf68ff9640 1 -- 172.21.15.105:0/3998653482 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5c00f9f0 con 0x7fdf6c0cdbb0 2023-11-28T18:25:57.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.685+0000 7fdf68ff9640 1 -- 172.21.15.105:0/3998653482 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c00fdc0 con 0x7fdf6c0cdbb0 2023-11-28T18:25:57.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.685+0000 7fdf7185a640 1 -- 172.21.15.105:0/3998653482 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf6c156ed0 con 0x7fdf6c0cdbb0 2023-11-28T18:25:57.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.685+0000 7fdf7185a640 1 -- 172.21.15.105:0/3998653482 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf6c157490 con 0x7fdf6c0cdbb0 2023-11-28T18:25:57.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.685+0000 7fdf7185a640 1 -- 172.21.15.105:0/3998653482 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf6c067570 con 0x7fdf6c0cdbb0 2023-11-28T18:25:57.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.685+0000 7fdf68ff9640 1 -- 172.21.15.105:0/3998653482 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fdf5c00b040 con 0x7fdf6c0cdbb0 2023-11-28T18:25:57.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.685+0000 7fdf68ff9640 1 --2- 172.21.15.105:0/3998653482 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdf4c0668d0 0x7fdf4c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:57.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.685+0000 7fdf68ff9640 1 -- 172.21.15.105:0/3998653482 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fdf5c084bc0 con 0x7fdf6c0cdbb0 2023-11-28T18:25:57.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.685+0000 7fdf627fc640 1 --2- 172.21.15.105:0/3998653482 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdf4c0668d0 0x7fdf4c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:57.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.685+0000 7fdf627fc640 1 --2- 172.21.15.105:0/3998653482 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdf4c0668d0 0x7fdf4c068d90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c09ef10 tx=0x7fdf54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:57.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.689+0000 7fdf68ff9640 1 -- 172.21.15.105:0/3998653482 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdf5c051ff0 con 0x7fdf6c0cdbb0 2023-11-28T18:25:57.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.841+0000 7fdf7185a640 1 -- 172.21.15.105:0/3998653482 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf6c0d0910 con 0x7fdf6c0cdbb0 2023-11-28T18:25:57.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.841+0000 7fdf68ff9640 1 -- 172.21.15.105:0/3998653482 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+13122 (secure 0 0 0) 0x7fdf5c01d020 con 0x7fdf6c0cdbb0 2023-11-28T18:25:57.842 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:57.842 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":47,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:25:57.139493+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:25:57.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.841+0000 7fdf7185a640 1 -- 172.21.15.105:0/3998653482 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdf4c0668d0 msgr2=0x7fdf4c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:57.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.841+0000 7fdf7185a640 1 --2- 172.21.15.105:0/3998653482 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdf4c0668d0 0x7fdf4c068d90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c09ef10 tx=0x7fdf54008040 comp rx=0 tx=0).stop 2023-11-28T18:25:57.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.841+0000 7fdf7185a640 1 -- 172.21.15.105:0/3998653482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf6c0cdbb0 msgr2=0x7fdf6c153860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:57.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.841+0000 7fdf7185a640 1 --2- 172.21.15.105:0/3998653482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf6c0cdbb0 0x7fdf6c153860 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00ce00 tx=0x7fdf5c00a650 comp rx=0 tx=0).stop 2023-11-28T18:25:57.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.841+0000 7fdf7185a640 1 -- 172.21.15.105:0/3998653482 shutdown_connections 2023-11-28T18:25:57.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.841+0000 7fdf7185a640 1 --2- 172.21.15.105:0/3998653482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf6c15d320 0x7fdf6c15e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.841+0000 7fdf7185a640 1 --2- 172.21.15.105:0/3998653482 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf6c0ce230 0x7fdf6c15e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.841+0000 7fdf7185a640 1 --2- 172.21.15.105:0/3998653482 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdf4c0668d0 0x7fdf4c068d90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.841+0000 7fdf7185a640 1 --2- 172.21.15.105:0/3998653482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf6c0cdbb0 0x7fdf6c153860 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:57.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.841+0000 7fdf7185a640 1 -- 172.21.15.105:0/3998653482 >> 172.21.15.105:0/3998653482 conn(0x7fdf6c0d0df0 msgr2=0x7fdf6c0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:57.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.841+0000 7fdf7185a640 1 -- 172.21.15.105:0/3998653482 shutdown_connections 2023-11-28T18:25:57.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:57.841+0000 7fdf7185a640 1 -- 172.21.15.105:0/3998653482 wait complete. 2023-11-28T18:25:57.863 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:25:57.864 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:25:58.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.001+0000 7f1429bf8640 1 Processor -- start 2023-11-28T18:25:58.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.001+0000 7f1429bf8640 1 -- start start 2023-11-28T18:25:58.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.005+0000 7f1429bf8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f142415d250 0x7f142415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:58.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.005+0000 7f1429bf8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f14240b5920 0x7f142415eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:58.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.005+0000 7f1429bf8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14240b45e0 con 0x7f142415f080 2023-11-28T18:25:58.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.005+0000 7f1429bf8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14240d1bb0 con 0x7f14240b5920 2023-11-28T18:25:58.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.005+0000 7f1429bf8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14240b4880 con 0x7f142415d250 2023-11-28T18:25:58.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.005+0000 7f141bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f14240b5920 0x7f142415eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:58.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.005+0000 7f14293f7640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f142415f080 0x7f142406eba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39092/0 (socket says 172.21.15.105:39092) 2023-11-28T18:25:58.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.005+0000 7f14293f7640 1 -- 172.21.15.105:0/1572300058 learned_addr learned my addr 172.21.15.105:0/1572300058 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:58.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.005+0000 7f1428bf6640 1 --2- 172.21.15.105:0/1572300058 >> v2:172.21.15.148:3300/0 conn(0x7f142415d250 0x7f142415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:58.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.005+0000 7f141bfff640 1 -- 172.21.15.105:0/1572300058 >> v2:172.21.15.148:3300/0 conn(0x7f142415d250 msgr2=0x7f142415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:58.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.005+0000 7f141bfff640 1 --2- 172.21.15.105:0/1572300058 >> v2:172.21.15.148:3300/0 conn(0x7f142415d250 0x7f142415d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.005+0000 7f141bfff640 1 -- 172.21.15.105:0/1572300058 >> v1:172.21.15.105:6789/0 conn(0x7f142415f080 legacy=0x7f142406eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:58.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.005+0000 7f141bfff640 1 -- 172.21.15.105:0/1572300058 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14240b4b60 con 0x7f14240b5920 2023-11-28T18:25:58.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.005+0000 7f141b7fe640 1 -- 172.21.15.105:0/1572300058 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 533410552 0 0) 0x7f14240b45e0 con 0x7f142415f080 2023-11-28T18:25:58.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.009+0000 7f141bfff640 1 --2- 172.21.15.105:0/1572300058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14240b5920 0x7f142415eb40 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f14240681b0 tx=0x7f141401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548d11f820fa7703 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:58.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.009+0000 7f141b7fe640 1 -- 172.21.15.105:0/1572300058 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14140ac600 con 0x7f14240b5920 2023-11-28T18:25:58.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.009+0000 7f141b7fe640 1 -- 172.21.15.105:0/1572300058 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14140a8050 con 0x7f14240b5920 2023-11-28T18:25:58.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.009+0000 7f141b7fe640 1 -- 172.21.15.105:0/1572300058 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14140ac980 con 0x7f14240b5920 2023-11-28T18:25:58.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.009+0000 7f1429bf8640 1 -- 172.21.15.105:0/1572300058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14240b5920 msgr2=0x7f142415eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:58.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.009+0000 7f1429bf8640 1 --2- 172.21.15.105:0/1572300058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14240b5920 0x7f142415eb40 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f14240681b0 tx=0x7f141401c3a0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.009+0000 7f1429bf8640 1 -- 172.21.15.105:0/1572300058 shutdown_connections 2023-11-28T18:25:58.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.009+0000 7f1429bf8640 1 --2- 172.21.15.105:0/1572300058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14240b5920 0x7f142415eb40 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.009+0000 7f1429bf8640 1 --2- 172.21.15.105:0/1572300058 >> v2:172.21.15.148:3300/0 conn(0x7f142415d250 0x7f142415d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.009+0000 7f1429bf8640 1 -- 172.21.15.105:0/1572300058 >> 172.21.15.105:0/1572300058 conn(0x7f1424099a50 msgr2=0x7f1424099e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:58.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.009+0000 7f1429bf8640 1 -- 172.21.15.105:0/1572300058 shutdown_connections 2023-11-28T18:25:58.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.009+0000 7f1429bf8640 1 -- 172.21.15.105:0/1572300058 wait complete. 2023-11-28T18:25:58.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.009+0000 7f1429bf8640 1 Processor -- start 2023-11-28T18:25:58.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.009+0000 7f1429bf8640 1 -- start start 2023-11-28T18:25:58.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f1429bf8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14240b5920 0x7f14241572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:58.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f1429bf8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f142415d250 0x7f142414e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:58.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f1429bf8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f142415f080 0x7f1424153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:58.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f1429bf8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14240cb760 con 0x7f142415d250 2023-11-28T18:25:58.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f1429bf8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1424157090 con 0x7f14240b5920 2023-11-28T18:25:58.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f1429bf8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f142414ee20 con 0x7f142415f080 2023-11-28T18:25:58.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f141bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f142415d250 0x7f142414e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:58.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f141bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f142415d250 0x7f142414e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55424/0 (socket says 172.21.15.105:55424) 2023-11-28T18:25:58.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f141bfff640 1 -- 172.21.15.105:0/186938477 learned_addr learned my addr 172.21.15.105:0/186938477 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:58.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f1428bf6640 1 --2- 172.21.15.105:0/186938477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14240b5920 0x7f14241572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:58.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f141bfff640 1 -- 172.21.15.105:0/186938477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f142415f080 msgr2=0x7f1424153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:58.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f141bfff640 1 --2- 172.21.15.105:0/186938477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f142415f080 0x7f1424153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f141bfff640 1 -- 172.21.15.105:0/186938477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14240b5920 msgr2=0x7f14241572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:58.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f141bfff640 1 --2- 172.21.15.105:0/186938477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14240b5920 0x7f14241572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f141bfff640 1 -- 172.21.15.105:0/186938477 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1414009420 con 0x7f142415d250 2023-11-28T18:25:58.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f1428bf6640 1 --2- 172.21.15.105:0/186938477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14240b5920 0x7f14241572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:58.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f141bfff640 1 --2- 172.21.15.105:0/186938477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f142415d250 0x7f142414e8e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1414018fb0 tx=0x7f141401abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:58.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f14197fa640 1 -- 172.21.15.105:0/186938477 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14140b7650 con 0x7f142415d250 2023-11-28T18:25:58.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f14197fa640 1 -- 172.21.15.105:0/186938477 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14140b77b0 con 0x7f142415d250 2023-11-28T18:25:58.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f14197fa640 1 -- 172.21.15.105:0/186938477 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14140b7b20 con 0x7f142415d250 2023-11-28T18:25:58.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f1429bf8640 1 -- 172.21.15.105:0/186938477 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1424158200 con 0x7f142415d250 2023-11-28T18:25:58.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f1429bf8640 1 -- 172.21.15.105:0/186938477 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1424157c40 con 0x7f142415d250 2023-11-28T18:25:58.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f1429bf8640 1 -- 172.21.15.105:0/186938477 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1424074580 con 0x7f142415d250 2023-11-28T18:25:58.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.013+0000 7f14197fa640 1 -- 172.21.15.105:0/186938477 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f14140b3070 con 0x7f142415d250 2023-11-28T18:25:58.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.017+0000 7f14197fa640 1 --2- 172.21.15.105:0/186938477 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1410066900 0x7f1410068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:58.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.017+0000 7f14197fa640 1 -- 172.21.15.105:0/186938477 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f141411e7a0 con 0x7f142415d250 2023-11-28T18:25:58.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.017+0000 7f1428bf6640 1 --2- 172.21.15.105:0/186938477 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1410066900 0x7f1410068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:58.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.017+0000 7f1428bf6640 1 --2- 172.21.15.105:0/186938477 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1410066900 0x7f1410068dc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1420009840 tx=0x7f1420008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:58.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.021+0000 7f14197fa640 1 -- 172.21.15.105:0/186938477 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f14140ebc80 con 0x7f142415d250 2023-11-28T18:25:58.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.172+0000 7f1429bf8640 1 -- 172.21.15.105:0/186938477 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f142406e770 con 0x7f142415d250 2023-11-28T18:25:58.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.172+0000 7f14197fa640 1 -- 172.21.15.105:0/186938477 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14140ef2a0 con 0x7f142415d250 2023-11-28T18:25:58.177 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:25:58.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.176+0000 7f1429bf8640 1 -- 172.21.15.105:0/186938477 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1410066900 msgr2=0x7f1410068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:58.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.176+0000 7f1429bf8640 1 --2- 172.21.15.105:0/186938477 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1410066900 0x7f1410068dc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1420009840 tx=0x7f1420008040 comp rx=0 tx=0).stop 2023-11-28T18:25:58.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.176+0000 7f1429bf8640 1 -- 172.21.15.105:0/186938477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f142415d250 msgr2=0x7f142414e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:58.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.176+0000 7f1429bf8640 1 --2- 172.21.15.105:0/186938477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f142415d250 0x7f142414e8e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1414018fb0 tx=0x7f141401abe0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.176+0000 7f1429bf8640 1 -- 172.21.15.105:0/186938477 shutdown_connections 2023-11-28T18:25:58.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.176+0000 7f1429bf8640 1 --2- 172.21.15.105:0/186938477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f142415f080 0x7f1424153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.176+0000 7f1429bf8640 1 --2- 172.21.15.105:0/186938477 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1410066900 0x7f1410068dc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.176+0000 7f1429bf8640 1 --2- 172.21.15.105:0/186938477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f142415d250 0x7f142414e8e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.176+0000 7f1429bf8640 1 --2- 172.21.15.105:0/186938477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14240b5920 0x7f14241572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.176+0000 7f1429bf8640 1 -- 172.21.15.105:0/186938477 >> 172.21.15.105:0/186938477 conn(0x7f1424099a50 msgr2=0x7f142415ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:58.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.176+0000 7f1429bf8640 1 -- 172.21.15.105:0/186938477 shutdown_connections 2023-11-28T18:25:58.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.176+0000 7f1429bf8640 1 -- 172.21.15.105:0/186938477 wait complete. 2023-11-28T18:25:58.198 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T18:25:58.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.332+0000 7fa8ad30f640 1 Processor -- start 2023-11-28T18:25:58.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.332+0000 7fa8ad30f640 1 -- start start 2023-11-28T18:25:58.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.336+0000 7fa8ad30f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8a80b3d30 0x7fa8a81680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:58.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.336+0000 7fa8ad30f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a80b25b0 con 0x7fa8a80d3c30 2023-11-28T18:25:58.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.336+0000 7fa8ad30f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a80d1c30 con 0x7fa8a815d4f0 2023-11-28T18:25:58.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.336+0000 7fa8ad30f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a80b2900 con 0x7fa8a80b3d30 2023-11-28T18:25:58.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.336+0000 7fa89e7fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa8a80d3c30 0x7fa8a80d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39098/0 (socket says 172.21.15.105:39098) 2023-11-28T18:25:58.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.336+0000 7fa89e7fc640 1 -- 172.21.15.105:0/1852570173 learned_addr learned my addr 172.21.15.105:0/1852570173 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:58.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.336+0000 7fa8a77fe640 1 --2- 172.21.15.105:0/1852570173 >> v2:172.21.15.148:3300/0 conn(0x7fa8a80b3d30 0x7fa8a81680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:58.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.336+0000 7fa8a67fc640 1 -- 172.21.15.105:0/1852570173 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2507210647 0 0) 0x7fa8a80b25b0 con 0x7fa8a80d3c30 2023-11-28T18:25:58.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.336+0000 7fa8a67fc640 1 -- 172.21.15.105:0/1852570173 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa894003530 con 0x7fa8a80d3c30 2023-11-28T18:25:58.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.336+0000 7fa8a77fe640 1 -- 172.21.15.105:0/1852570173 >> v1:172.21.15.148:6789/0 conn(0x7fa8a815d4f0 legacy=0x7fa8a80d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:58.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.336+0000 7fa8a77fe640 1 -- 172.21.15.105:0/1852570173 >> v1:172.21.15.105:6789/0 conn(0x7fa8a80d3c30 legacy=0x7fa8a80d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:58.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.336+0000 7fa8a77fe640 1 -- 172.21.15.105:0/1852570173 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a80cea30 con 0x7fa8a80b3d30 2023-11-28T18:25:58.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.336+0000 7fa8a77fe640 1 --2- 172.21.15.105:0/1852570173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8a80b3d30 0x7fa8a81680a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0009d90 tx=0x7fa8a001c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15aff903514b1d69 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:58.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.340+0000 7fa8a67fc640 1 -- 172.21.15.105:0/1852570173 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a00a58c0 con 0x7fa8a80b3d30 2023-11-28T18:25:58.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.340+0000 7fa8a67fc640 1 -- 172.21.15.105:0/1852570173 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a00a5a20 con 0x7fa8a80b3d30 2023-11-28T18:25:58.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.340+0000 7fa8a67fc640 1 -- 172.21.15.105:0/1852570173 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a00a5db0 con 0x7fa8a80b3d30 2023-11-28T18:25:58.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.340+0000 7fa8ad30f640 1 -- 172.21.15.105:0/1852570173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8a80b3d30 msgr2=0x7fa8a81680a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:58.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.340+0000 7fa8ad30f640 1 --2- 172.21.15.105:0/1852570173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8a80b3d30 0x7fa8a81680a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0009d90 tx=0x7fa8a001c800 comp rx=0 tx=0).stop 2023-11-28T18:25:58.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.340+0000 7fa8ad30f640 1 -- 172.21.15.105:0/1852570173 shutdown_connections 2023-11-28T18:25:58.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.340+0000 7fa8ad30f640 1 --2- 172.21.15.105:0/1852570173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8a80b3d30 0x7fa8a81680a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.340+0000 7fa8ad30f640 1 -- 172.21.15.105:0/1852570173 >> 172.21.15.105:0/1852570173 conn(0x7fa8a80d3030 msgr2=0x7fa8a80d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:58.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.340+0000 7fa8ad30f640 1 -- 172.21.15.105:0/1852570173 shutdown_connections 2023-11-28T18:25:58.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.340+0000 7fa8ad30f640 1 -- 172.21.15.105:0/1852570173 wait complete. 2023-11-28T18:25:58.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.340+0000 7fa8ad30f640 1 Processor -- start 2023-11-28T18:25:58.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.340+0000 7fa8ad30f640 1 -- start start 2023-11-28T18:25:58.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.340+0000 7fa8ad30f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8a80b3d30 0x7fa8a80b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:58.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.340+0000 7fa8ad30f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8a80d3c30 0x7fa8a80cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:58.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.340+0000 7fa8ad30f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8a815d4f0 0x7fa8a8153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:58.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.340+0000 7fa8ad30f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a80d0a20 con 0x7fa8a80d3c30 2023-11-28T18:25:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.340+0000 7fa8ad30f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a80b5cd0 con 0x7fa8a80b3d30 2023-11-28T18:25:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.340+0000 7fa8ad30f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a80b5f70 con 0x7fa8a815d4f0 2023-11-28T18:25:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.340+0000 7fa89e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8a80d3c30 0x7fa8a80cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.340+0000 7fa89e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8a80d3c30 0x7fa8a80cf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55428/0 (socket says 172.21.15.105:55428) 2023-11-28T18:25:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.344+0000 7fa89e7fc640 1 -- 172.21.15.105:0/3512648159 learned_addr learned my addr 172.21.15.105:0/3512648159 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.344+0000 7fa8a77fe640 1 --2- 172.21.15.105:0/3512648159 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8a815d4f0 0x7fa8a8153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.344+0000 7fa8a6ffd640 1 --2- 172.21.15.105:0/3512648159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8a80b3d30 0x7fa8a80b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.344+0000 7fa89e7fc640 1 -- 172.21.15.105:0/3512648159 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8a815d4f0 msgr2=0x7fa8a8153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.344+0000 7fa89e7fc640 1 --2- 172.21.15.105:0/3512648159 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8a815d4f0 0x7fa8a8153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.344+0000 7fa89e7fc640 1 -- 172.21.15.105:0/3512648159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8a80b3d30 msgr2=0x7fa8a80b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.344+0000 7fa89e7fc640 1 --2- 172.21.15.105:0/3512648159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8a80b3d30 0x7fa8a80b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.344+0000 7fa89e7fc640 1 -- 172.21.15.105:0/3512648159 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a0009a70 con 0x7fa8a80d3c30 2023-11-28T18:25:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.344+0000 7fa8a77fe640 1 --2- 172.21.15.105:0/3512648159 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8a815d4f0 0x7fa8a8153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:58.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.344+0000 7fa8a6ffd640 1 --2- 172.21.15.105:0/3512648159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8a80b3d30 0x7fa8a80b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:58.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.344+0000 7fa89e7fc640 1 --2- 172.21.15.105:0/3512648159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8a80d3c30 0x7fa8a80cf730 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa89000c420 tx=0x7fa89000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:58.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.344+0000 7fa89ffff640 1 -- 172.21.15.105:0/3512648159 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa890005cf0 con 0x7fa8a80d3c30 2023-11-28T18:25:58.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.344+0000 7fa89ffff640 1 -- 172.21.15.105:0/3512648159 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa890005e50 con 0x7fa8a80d3c30 2023-11-28T18:25:58.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.344+0000 7fa89ffff640 1 -- 172.21.15.105:0/3512648159 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8900072d0 con 0x7fa8a80d3c30 2023-11-28T18:25:58.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.344+0000 7fa8ad30f640 1 -- 172.21.15.105:0/3512648159 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a80b6250 con 0x7fa8a80d3c30 2023-11-28T18:25:58.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.344+0000 7fa8ad30f640 1 -- 172.21.15.105:0/3512648159 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a815c970 con 0x7fa8a80d3c30 2023-11-28T18:25:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.344+0000 7fa8ad30f640 1 -- 172.21.15.105:0/3512648159 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a8067570 con 0x7fa8a80d3c30 2023-11-28T18:25:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.344+0000 7fa89ffff640 1 -- 172.21.15.105:0/3512648159 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa890007430 con 0x7fa8a80d3c30 2023-11-28T18:25:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.348+0000 7fa89ffff640 1 --2- 172.21.15.105:0/3512648159 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa8880668d0 0x7fa888068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:58.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.348+0000 7fa89ffff640 1 -- 172.21.15.105:0/3512648159 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fa8900837e0 con 0x7fa8a80d3c30 2023-11-28T18:25:58.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.348+0000 7fa8a6ffd640 1 --2- 172.21.15.105:0/3512648159 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa8880668d0 0x7fa888068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:58.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.348+0000 7fa8a6ffd640 1 --2- 172.21.15.105:0/3512648159 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa8880668d0 0x7fa888068d90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa898000f70 tx=0x7fa898009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:58.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.352+0000 7fa89ffff640 1 -- 172.21.15.105:0/3512648159 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa890050c40 con 0x7fa8a80d3c30 2023-11-28T18:25:58.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.500+0000 7fa8ad30f640 1 -- 172.21.15.105:0/3512648159 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa8a8158720 con 0x7fa8a80d3c30 2023-11-28T18:25:58.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.504+0000 7fa89ffff640 1 -- 172.21.15.105:0/3512648159 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v47) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa890017090 con 0x7fa8a80d3c30 2023-11-28T18:25:58.506 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:25:58.507 INFO:teuthology.orchestra.run.smithi105.stdout:[ 2023-11-28T18:25:58.507 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:25:58.507 INFO:teuthology.orchestra.run.smithi105.stdout: "poolnum": 1, 2023-11-28T18:25:58.507 INFO:teuthology.orchestra.run.smithi105.stdout: "poolname": ".mgr" 2023-11-28T18:25:58.507 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:25:58.508 INFO:teuthology.orchestra.run.smithi105.stdout:] 2023-11-28T18:25:58.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.504+0000 7fa8ad30f640 1 -- 172.21.15.105:0/3512648159 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa8880668d0 msgr2=0x7fa888068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:58.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.504+0000 7fa8ad30f640 1 --2- 172.21.15.105:0/3512648159 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa8880668d0 0x7fa888068d90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa898000f70 tx=0x7fa898009040 comp rx=0 tx=0).stop 2023-11-28T18:25:58.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.504+0000 7fa8ad30f640 1 -- 172.21.15.105:0/3512648159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8a80d3c30 msgr2=0x7fa8a80cf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:58.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.504+0000 7fa8ad30f640 1 --2- 172.21.15.105:0/3512648159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8a80d3c30 0x7fa8a80cf730 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa89000c420 tx=0x7fa89000c8e0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.504+0000 7fa8ad30f640 1 -- 172.21.15.105:0/3512648159 shutdown_connections 2023-11-28T18:25:58.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.504+0000 7fa8ad30f640 1 --2- 172.21.15.105:0/3512648159 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8a815d4f0 0x7fa8a8153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.504+0000 7fa8ad30f640 1 --2- 172.21.15.105:0/3512648159 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa8880668d0 0x7fa888068d90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.504+0000 7fa8ad30f640 1 --2- 172.21.15.105:0/3512648159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8a80d3c30 0x7fa8a80cf730 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.504+0000 7fa8ad30f640 1 --2- 172.21.15.105:0/3512648159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8a80b3d30 0x7fa8a80b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.504+0000 7fa8ad30f640 1 -- 172.21.15.105:0/3512648159 >> 172.21.15.105:0/3512648159 conn(0x7fa8a80d3030 msgr2=0x7fa8a806c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:58.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.504+0000 7fa8ad30f640 1 -- 172.21.15.105:0/3512648159 shutdown_connections 2023-11-28T18:25:58.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.504+0000 7fa8ad30f640 1 -- 172.21.15.105:0/3512648159 wait complete. 2023-11-28T18:25:58.527 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T18:25:58.527 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T18:25:58.670 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27cee6b640 1 Processor -- start 2023-11-28T18:25:58.670 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27cee6b640 1 -- start start 2023-11-28T18:25:58.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27cee6b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27c806bf50 0x7f27c815d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:58.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27cee6b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c80cc7f0 con 0x7f27c80adb80 2023-11-28T18:25:58.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27cee6b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c815dd10 con 0x7f27c8154070 2023-11-28T18:25:58.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27cee6b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c8072540 con 0x7f27c806bf50 2023-11-28T18:25:58.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27cd408640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f27c80adb80 0x7f27c80acd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39112/0 (socket says 172.21.15.105:39112) 2023-11-28T18:25:58.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27cd408640 1 -- 172.21.15.105:0/3748633249 learned_addr learned my addr 172.21.15.105:0/3748633249 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:58.674 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27c7fff640 1 --2- 172.21.15.105:0/3748633249 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27c806bf50 0x7f27c815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:58.674 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27c77fe640 1 -- 172.21.15.105:0/3748633249 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941873177 0 0) 0x7f27c80cc7f0 con 0x7f27c80adb80 2023-11-28T18:25:58.674 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27c77fe640 1 -- 172.21.15.105:0/3748633249 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27b8003530 con 0x7f27c80adb80 2023-11-28T18:25:58.674 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27c77fe640 1 -- 172.21.15.105:0/3748633249 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2660120290 0 0) 0x7f27b8003530 con 0x7f27c80adb80 2023-11-28T18:25:58.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27c77fe640 1 -- 172.21.15.105:0/3748633249 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27c806bf50 msgr2=0x7f27c815d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:58.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27c77fe640 1 --2- 172.21.15.105:0/3748633249 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27c806bf50 0x7f27c815d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27c77fe640 1 -- 172.21.15.105:0/3748633249 >> v1:172.21.15.148:6789/0 conn(0x7f27c8154070 legacy=0x7f27c80ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:58.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27c77fe640 1 -- 172.21.15.105:0/3748633249 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c815dff0 con 0x7f27c80adb80 2023-11-28T18:25:58.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27c77fe640 1 -- 172.21.15.105:0/3748633249 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f27c0001a00 con 0x7f27c80adb80 2023-11-28T18:25:58.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27c77fe640 1 -- 172.21.15.105:0/3748633249 >> v1:172.21.15.105:6789/0 conn(0x7f27c80adb80 legacy=0x7f27c80acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:58.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27c77fe640 1 --2- 172.21.15.105:0/3748633249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27b80039b0 0x7f27b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:58.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27c77fe640 1 --2- 172.21.15.105:0/3748633249 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27c806bf50 0x7f27b80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:58.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27c77fe640 1 --2- 172.21.15.105:0/3748633249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27b8004d10 0x7f27b8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:58.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27c77fe640 1 -- 172.21.15.105:0/3748633249 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c8072540 con 0x7f27b8004d10 2023-11-28T18:25:58.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27c77fe640 1 -- 172.21.15.105:0/3748633249 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b800ba70 con 0x7f27b80039b0 2023-11-28T18:25:58.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27c77fe640 1 -- 172.21.15.105:0/3748633249 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b800bc30 con 0x7f27c806bf50 2023-11-28T18:25:58.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27cd408640 1 --2- 172.21.15.105:0/3748633249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27b8004d10 0x7f27b8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:58.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27ccc07640 1 --2- 172.21.15.105:0/3748633249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27b80039b0 0x7f27b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:58.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27c7fff640 1 --2- 172.21.15.105:0/3748633249 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27c806bf50 0x7f27b80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:58.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27cd408640 1 -- 172.21.15.105:0/3748633249 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27c806bf50 msgr2=0x7f27b80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:58.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27cd408640 1 --2- 172.21.15.105:0/3748633249 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27c806bf50 0x7f27b80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27cd408640 1 -- 172.21.15.105:0/3748633249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27b80039b0 msgr2=0x7f27b8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:58.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27cd408640 1 --2- 172.21.15.105:0/3748633249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27b80039b0 0x7f27b8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.668+0000 7f27cd408640 1 -- 172.21.15.105:0/3748633249 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b8003530 con 0x7f27b8004d10 2023-11-28T18:25:58.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.672+0000 7f27cd408640 1 --2- 172.21.15.105:0/3748633249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27b8004d10 0x7f27b8005190 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f27c0002240 tx=0x7f27c001d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea4b38a837e2365 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:58.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.672+0000 7f27c77fe640 1 -- 172.21.15.105:0/3748633249 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c00ad630 con 0x7f27b8004d10 2023-11-28T18:25:58.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.672+0000 7f27c77fe640 1 -- 172.21.15.105:0/3748633249 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c00a8070 con 0x7f27b8004d10 2023-11-28T18:25:58.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.672+0000 7f27c77fe640 1 -- 172.21.15.105:0/3748633249 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c00ada10 con 0x7f27b8004d10 2023-11-28T18:25:58.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.672+0000 7f27cee6b640 1 -- 172.21.15.105:0/3748633249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27b8004d10 msgr2=0x7f27b8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:58.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.672+0000 7f27cee6b640 1 --2- 172.21.15.105:0/3748633249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27b8004d10 0x7f27b8005190 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f27c0002240 tx=0x7f27c001d2e0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.672+0000 7f27cd408640 1 -- 172.21.15.105:0/3748633249 reap_dead start 2023-11-28T18:25:58.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.672+0000 7f27cee6b640 1 -- 172.21.15.105:0/3748633249 shutdown_connections 2023-11-28T18:25:58.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.672+0000 7f27cee6b640 1 -- 172.21.15.105:0/3748633249 >> 172.21.15.105:0/3748633249 conn(0x7f27c80ceb90 msgr2=0x7f27c80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:58.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.672+0000 7f27cee6b640 1 -- 172.21.15.105:0/3748633249 shutdown_connections 2023-11-28T18:25:58.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.672+0000 7f27cee6b640 1 -- 172.21.15.105:0/3748633249 wait complete. 2023-11-28T18:25:58.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27cee6b640 1 Processor -- start 2023-11-28T18:25:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27cee6b640 1 -- start start 2023-11-28T18:25:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27cee6b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27b8004d10 0x7f27c809aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27cee6b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27c8099a50 0x7f27c8145c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27cee6b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27c80b23e0 0x7f27c80b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:58.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27cee6b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c809ab60 con 0x7f27b8004d10 2023-11-28T18:25:58.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27cee6b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c8099f50 con 0x7f27c8099a50 2023-11-28T18:25:58.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27cee6b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c809a230 con 0x7f27c80b23e0 2023-11-28T18:25:58.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27ccc07640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27b8004d10 0x7f27c809aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:58.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27ccc07640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27b8004d10 0x7f27c809aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55446/0 (socket says 172.21.15.105:55446) 2023-11-28T18:25:58.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27ccc07640 1 -- 172.21.15.105:0/1315166164 learned_addr learned my addr 172.21.15.105:0/1315166164 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:58.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27c7fff640 1 --2- 172.21.15.105:0/1315166164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27c8099a50 0x7f27c8145c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:58.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27cd408640 1 --2- 172.21.15.105:0/1315166164 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27c80b23e0 0x7f27c80b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:58.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27ccc07640 1 -- 172.21.15.105:0/1315166164 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27c80b23e0 msgr2=0x7f27c80b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:58.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27ccc07640 1 --2- 172.21.15.105:0/1315166164 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27c80b23e0 0x7f27c80b3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27ccc07640 1 -- 172.21.15.105:0/1315166164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27c8099a50 msgr2=0x7f27c8145c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:58.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27ccc07640 1 --2- 172.21.15.105:0/1315166164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27c8099a50 0x7f27c8145c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:58.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27ccc07640 1 -- 172.21.15.105:0/1315166164 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c809a510 con 0x7f27b8004d10 2023-11-28T18:25:58.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27cd408640 1 --2- 172.21.15.105:0/1315166164 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27c80b23e0 0x7f27c80b3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:58.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27c7fff640 1 --2- 172.21.15.105:0/1315166164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27c8099a50 0x7f27c8145c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:25:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27ccc07640 1 --2- 172.21.15.105:0/1315166164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27b8004d10 0x7f27c809aff0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00f7f0 tx=0x7f27bc00fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27c57fa640 1 -- 172.21.15.105:0/1315166164 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc0050d0 con 0x7f27b8004d10 2023-11-28T18:25:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27c57fa640 1 -- 172.21.15.105:0/1315166164 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27bc005230 con 0x7f27b8004d10 2023-11-28T18:25:58.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27c57fa640 1 -- 172.21.15.105:0/1315166164 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc00d730 con 0x7f27b8004d10 2023-11-28T18:25:58.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27cee6b640 1 -- 172.21.15.105:0/1315166164 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c809a7f0 con 0x7f27b8004d10 2023-11-28T18:25:58.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.676+0000 7f27cee6b640 1 -- 172.21.15.105:0/1315166164 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c81461a0 con 0x7f27b8004d10 2023-11-28T18:25:58.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.680+0000 7f27cee6b640 1 -- 172.21.15.105:0/1315166164 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c8067570 con 0x7f27b8004d10 2023-11-28T18:25:58.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.680+0000 7f27c57fa640 1 -- 172.21.15.105:0/1315166164 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f27bc00d890 con 0x7f27b8004d10 2023-11-28T18:25:58.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.680+0000 7f27c57fa640 1 --2- 172.21.15.105:0/1315166164 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27ac066900 0x7f27ac068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:58.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.680+0000 7f27c57fa640 1 -- 172.21.15.105:0/1315166164 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f27bc0841f0 con 0x7f27b8004d10 2023-11-28T18:25:58.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.680+0000 7f27c7fff640 1 --2- 172.21.15.105:0/1315166164 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27ac066900 0x7f27ac068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:58.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.680+0000 7f27c7fff640 1 --2- 172.21.15.105:0/1315166164 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27ac066900 0x7f27ac068dc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f27c80b5090 tx=0x7f27b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:58.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.684+0000 7f27c57fa640 1 -- 172.21.15.105:0/1315166164 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f27bc051640 con 0x7f27b8004d10 2023-11-28T18:25:58.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:58.836+0000 7f27cee6b640 1 -- 172.21.15.105:0/1315166164 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f27c80cf950 con 0x7f27b8004d10 2023-11-28T18:25:59.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.144+0000 7f27c57fa640 1 -- 172.21.15.105:0/1315166164 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v48) v1 ==== 138+0+0 (secure 0 0 0) 0x7f27bc054c60 con 0x7f27b8004d10 2023-11-28T18:25:59.148 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' created 2023-11-28T18:25:59.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.148+0000 7f27cee6b640 1 -- 172.21.15.105:0/1315166164 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27ac066900 msgr2=0x7f27ac068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:59.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.148+0000 7f27cee6b640 1 --2- 172.21.15.105:0/1315166164 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27ac066900 0x7f27ac068dc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f27c80b5090 tx=0x7f27b4008040 comp rx=0 tx=0).stop 2023-11-28T18:25:59.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.148+0000 7f27cee6b640 1 -- 172.21.15.105:0/1315166164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27b8004d10 msgr2=0x7f27c809aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:59.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.148+0000 7f27cee6b640 1 --2- 172.21.15.105:0/1315166164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27b8004d10 0x7f27c809aff0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00f7f0 tx=0x7f27bc00fcb0 comp rx=0 tx=0).stop 2023-11-28T18:25:59.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.148+0000 7f27cee6b640 1 -- 172.21.15.105:0/1315166164 shutdown_connections 2023-11-28T18:25:59.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.148+0000 7f27cee6b640 1 --2- 172.21.15.105:0/1315166164 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27c80b23e0 0x7f27c80b3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:59.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.148+0000 7f27cee6b640 1 --2- 172.21.15.105:0/1315166164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27c8099a50 0x7f27c8145c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:59.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.148+0000 7f27cee6b640 1 --2- 172.21.15.105:0/1315166164 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27ac066900 0x7f27ac068dc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:59.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.148+0000 7f27cee6b640 1 --2- 172.21.15.105:0/1315166164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27b8004d10 0x7f27c809aff0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:59.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.148+0000 7f27cee6b640 1 -- 172.21.15.105:0/1315166164 >> 172.21.15.105:0/1315166164 conn(0x7f27c80ceb90 msgr2=0x7f27c80cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:59.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.148+0000 7f27cee6b640 1 -- 172.21.15.105:0/1315166164 shutdown_connections 2023-11-28T18:25:59.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.148+0000 7f27cee6b640 1 -- 172.21.15.105:0/1315166164 wait complete. 2023-11-28T18:25:59.176 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T18:25:59.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.320+0000 7f85562c9640 1 Processor -- start 2023-11-28T18:25:59.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.320+0000 7f85562c9640 1 -- start start 2023-11-28T18:25:59.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.320+0000 7f85562c9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f85500b4290 0x7f85500b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:59.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.320+0000 7f85562c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f855015b030 0x7f85500709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:59.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.320+0000 7f85562c9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85500b4bb0 con 0x7f8550071960 2023-11-28T18:25:59.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.320+0000 7f85562c9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85500701d0 con 0x7f85500b4290 2023-11-28T18:25:59.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.320+0000 7f85562c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855006df80 con 0x7f855015b030 2023-11-28T18:25:59.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.320+0000 7f854f7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8550071960 0x7f855015f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39126/0 (socket says 172.21.15.105:39126) 2023-11-28T18:25:59.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.320+0000 7f854f7fe640 1 -- 172.21.15.105:0/1736319920 learned_addr learned my addr 172.21.15.105:0/1736319920 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:25:59.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.320+0000 7f8554866640 1 --2- 172.21.15.105:0/1736319920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f855015b030 0x7f85500709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:59.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.320+0000 7f854ffff640 1 --2- 172.21.15.105:0/1736319920 >> v2:172.21.15.148:3300/0 conn(0x7f85500b4290 0x7f85500b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:59.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.320+0000 7f854effd640 1 -- 172.21.15.105:0/1736319920 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991907628 0 0) 0x7f85500b4bb0 con 0x7f8550071960 2023-11-28T18:25:59.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.320+0000 7f854effd640 1 -- 172.21.15.105:0/1736319920 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8538003530 con 0x7f8550071960 2023-11-28T18:25:59.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.320+0000 7f854ffff640 1 -- 172.21.15.105:0/1736319920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f855015b030 msgr2=0x7f85500709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:25:59.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.320+0000 7f854ffff640 1 --2- 172.21.15.105:0/1736319920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f855015b030 0x7f85500709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:59.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.320+0000 7f854ffff640 1 -- 172.21.15.105:0/1736319920 >> v1:172.21.15.105:6789/0 conn(0x7f8550071960 legacy=0x7f855015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:59.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.320+0000 7f854ffff640 1 -- 172.21.15.105:0/1736319920 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8550071df0 con 0x7f85500b4290 2023-11-28T18:25:59.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.324+0000 7f854ffff640 1 --2- 172.21.15.105:0/1736319920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85500b4290 0x7f85500b4670 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f85400097c0 tx=0x7f854001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8d481ab2bcc0f73 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:59.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.324+0000 7f854effd640 1 -- 172.21.15.105:0/1736319920 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85400ad660 con 0x7f85500b4290 2023-11-28T18:25:59.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.324+0000 7f854effd640 1 -- 172.21.15.105:0/1736319920 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85400ad7c0 con 0x7f85500b4290 2023-11-28T18:25:59.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.324+0000 7f854effd640 1 -- 172.21.15.105:0/1736319920 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85400adb50 con 0x7f85500b4290 2023-11-28T18:25:59.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.324+0000 7f85562c9640 1 -- 172.21.15.105:0/1736319920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85500b4290 msgr2=0x7f85500b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:59.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.324+0000 7f85562c9640 1 --2- 172.21.15.105:0/1736319920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85500b4290 0x7f85500b4670 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f85400097c0 tx=0x7f854001c450 comp rx=0 tx=0).stop 2023-11-28T18:25:59.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.324+0000 7f85562c9640 1 -- 172.21.15.105:0/1736319920 shutdown_connections 2023-11-28T18:25:59.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.324+0000 7f85562c9640 1 --2- 172.21.15.105:0/1736319920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f855015b030 0x7f85500709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:59.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.324+0000 7f85562c9640 1 --2- 172.21.15.105:0/1736319920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85500b4290 0x7f85500b4670 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:59.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.324+0000 7f85562c9640 1 -- 172.21.15.105:0/1736319920 >> 172.21.15.105:0/1736319920 conn(0x7f85500ceb90 msgr2=0x7f85500cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:25:59.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.324+0000 7f85562c9640 1 -- 172.21.15.105:0/1736319920 shutdown_connections 2023-11-28T18:25:59.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.324+0000 7f85562c9640 1 -- 172.21.15.105:0/1736319920 wait complete. 2023-11-28T18:25:59.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f85562c9640 1 Processor -- start 2023-11-28T18:25:59.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f85562c9640 1 -- start start 2023-11-28T18:25:59.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f85562c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85500b4290 0x7f8550099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:59.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f85562c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f855015b030 0x7f8550156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:59.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f85562c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f855009b3a0 0x7f85500cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:59.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f85562c9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855009a330 con 0x7f855015b030 2023-11-28T18:25:59.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f85562c9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855009a610 con 0x7f855009b3a0 2023-11-28T18:25:59.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f85562c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855009a8f0 con 0x7f85500b4290 2023-11-28T18:25:59.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f854f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f855015b030 0x7f8550156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:59.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f854f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f855015b030 0x7f8550156a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55454/0 (socket says 172.21.15.105:55454) 2023-11-28T18:25:59.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f854f7fe640 1 -- 172.21.15.105:0/2783175449 learned_addr learned my addr 172.21.15.105:0/2783175449 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:25:59.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f854ffff640 1 --2- 172.21.15.105:0/2783175449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85500b4290 0x7f8550099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:59.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f8554866640 1 --2- 172.21.15.105:0/2783175449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f855009b3a0 0x7f85500cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:59.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f854f7fe640 1 -- 172.21.15.105:0/2783175449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85500b4290 msgr2=0x7f8550099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:59.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f854f7fe640 1 --2- 172.21.15.105:0/2783175449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85500b4290 0x7f8550099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:59.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f854f7fe640 1 -- 172.21.15.105:0/2783175449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f855009b3a0 msgr2=0x7f85500cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:25:59.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f854f7fe640 1 --2- 172.21.15.105:0/2783175449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f855009b3a0 0x7f85500cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:25:59.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f854f7fe640 1 -- 172.21.15.105:0/2783175449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8540009420 con 0x7f855015b030 2023-11-28T18:25:59.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f8554866640 1 --2- 172.21.15.105:0/2783175449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f855009b3a0 0x7f85500cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:25:59.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f854f7fe640 1 --2- 172.21.15.105:0/2783175449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f855015b030 0x7f8550156a80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f854400c4d0 tx=0x7f854400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:59.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f854cff9640 1 -- 172.21.15.105:0/2783175449 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544005cf0 con 0x7f855015b030 2023-11-28T18:25:59.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f854cff9640 1 -- 172.21.15.105:0/2783175449 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8544005e50 con 0x7f855015b030 2023-11-28T18:25:59.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f854cff9640 1 -- 172.21.15.105:0/2783175449 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544007270 con 0x7f855015b030 2023-11-28T18:25:59.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f85562c9640 1 -- 172.21.15.105:0/2783175449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85501578a0 con 0x7f855015b030 2023-11-28T18:25:59.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.328+0000 7f85562c9640 1 -- 172.21.15.105:0/2783175449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8550156fc0 con 0x7f855015b030 2023-11-28T18:25:59.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.332+0000 7f85562c9640 1 -- 172.21.15.105:0/2783175449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8550157ac0 con 0x7f855015b030 2023-11-28T18:25:59.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.332+0000 7f854cff9640 1 -- 172.21.15.105:0/2783175449 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f85440073d0 con 0x7f855015b030 2023-11-28T18:25:59.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.332+0000 7f854cff9640 1 --2- 172.21.15.105:0/2783175449 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f853c0668d0 0x7f853c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:25:59.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.332+0000 7f854cff9640 1 -- 172.21.15.105:0/2783175449 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f85440838f0 con 0x7f855015b030 2023-11-28T18:25:59.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.332+0000 7f854ffff640 1 --2- 172.21.15.105:0/2783175449 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f853c0668d0 0x7f853c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:25:59.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.332+0000 7f854ffff640 1 --2- 172.21.15.105:0/2783175449 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f853c0668d0 0x7f853c068d90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f85400af000 tx=0x7f85400a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:25:59.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.336+0000 7f854cff9640 1 -- 172.21.15.105:0/2783175449 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8544050c80 con 0x7f855015b030 2023-11-28T18:25:59.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:25:59.484+0000 7f85562c9640 1 -- 172.21.15.105:0/2783175449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8550067570 con 0x7f855015b030 2023-11-28T18:26:00.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.156+0000 7f854cff9640 1 -- 172.21.15.105:0/2783175449 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v49) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8544017020 con 0x7f855015b030 2023-11-28T18:26:00.158 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' created 2023-11-28T18:26:00.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.156+0000 7f85562c9640 1 -- 172.21.15.105:0/2783175449 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f853c0668d0 msgr2=0x7f853c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:00.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.156+0000 7f85562c9640 1 --2- 172.21.15.105:0/2783175449 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f853c0668d0 0x7f853c068d90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f85400af000 tx=0x7f85400a7000 comp rx=0 tx=0).stop 2023-11-28T18:26:00.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.156+0000 7f85562c9640 1 -- 172.21.15.105:0/2783175449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f855015b030 msgr2=0x7f8550156a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:00.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.156+0000 7f85562c9640 1 --2- 172.21.15.105:0/2783175449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f855015b030 0x7f8550156a80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f854400c4d0 tx=0x7f854400c990 comp rx=0 tx=0).stop 2023-11-28T18:26:00.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.156+0000 7f85562c9640 1 -- 172.21.15.105:0/2783175449 shutdown_connections 2023-11-28T18:26:00.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.156+0000 7f85562c9640 1 --2- 172.21.15.105:0/2783175449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f855009b3a0 0x7f85500cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:00.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.156+0000 7f85562c9640 1 --2- 172.21.15.105:0/2783175449 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f853c0668d0 0x7f853c068d90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:00.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.156+0000 7f85562c9640 1 --2- 172.21.15.105:0/2783175449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f855015b030 0x7f8550156a80 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:00.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.156+0000 7f85562c9640 1 --2- 172.21.15.105:0/2783175449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85500b4290 0x7f8550099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:00.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.156+0000 7f85562c9640 1 -- 172.21.15.105:0/2783175449 >> 172.21.15.105:0/2783175449 conn(0x7f85500ceb90 msgr2=0x7f85500ad4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:00.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.156+0000 7f85562c9640 1 -- 172.21.15.105:0/2783175449 shutdown_connections 2023-11-28T18:26:00.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.156+0000 7f85562c9640 1 -- 172.21.15.105:0/2783175449 wait complete. 2023-11-28T18:26:00.205 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T18:26:00.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.344+0000 7f0d0fe9c640 1 Processor -- start 2023-11-28T18:26:00.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.344+0000 7f0d0fe9c640 1 -- start start 2023-11-28T18:26:00.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.348+0000 7f0d0fe9c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d0815d4f0 0x7f0d080d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:00.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.348+0000 7f0d0fe9c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0d080d3c30 0x7f0d080d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:00.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.348+0000 7f0d0fe9c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d080b25b0 con 0x7f0d080d1890 2023-11-28T18:26:00.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.348+0000 7f0d0fe9c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d080b2900 con 0x7f0d080d3c30 2023-11-28T18:26:00.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.348+0000 7f0d0fe9c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d080cea30 con 0x7f0d0815d4f0 2023-11-28T18:26:00.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.348+0000 7f0d0e439640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0d080d1890 0x7f0d080cfa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39136/0 (socket says 172.21.15.105:39136) 2023-11-28T18:26:00.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.348+0000 7f0d0e439640 1 -- 172.21.15.105:0/3088162134 learned_addr learned my addr 172.21.15.105:0/3088162134 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:00.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.348+0000 7f0d0d437640 1 --2- 172.21.15.105:0/3088162134 >> v2:172.21.15.148:3300/0 conn(0x7f0d080d3c30 0x7f0d080d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:00.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.348+0000 7f0d0dc38640 1 --2- 172.21.15.105:0/3088162134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d0815d4f0 0x7f0d080d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:00.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.348+0000 7f0d0cc36640 1 -- 172.21.15.105:0/3088162134 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942658170 0 0) 0x7f0d080b25b0 con 0x7f0d080d1890 2023-11-28T18:26:00.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.348+0000 7f0d0cc36640 1 -- 172.21.15.105:0/3088162134 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cf4003530 con 0x7f0d080d1890 2023-11-28T18:26:00.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.348+0000 7f0d0d437640 1 -- 172.21.15.105:0/3088162134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d0815d4f0 msgr2=0x7f0d080d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:00.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.348+0000 7f0d0d437640 1 --2- 172.21.15.105:0/3088162134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d0815d4f0 0x7f0d080d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:00.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.348+0000 7f0d0d437640 1 -- 172.21.15.105:0/3088162134 >> v1:172.21.15.105:6789/0 conn(0x7f0d080d1890 legacy=0x7f0d080cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:00.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.348+0000 7f0d0d437640 1 -- 172.21.15.105:0/3088162134 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d080ced10 con 0x7f0d080d3c30 2023-11-28T18:26:00.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.348+0000 7f0d0dc38640 1 --2- 172.21.15.105:0/3088162134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d0815d4f0 0x7f0d080d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:26:00.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.352+0000 7f0d0d437640 1 --2- 172.21.15.105:0/3088162134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d080d3c30 0x7f0d080d1350 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80097c0 tx=0x7f0cf801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fb59755f6f01938 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:00.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.352+0000 7f0d0cc36640 1 -- 172.21.15.105:0/3088162134 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf80ab600 con 0x7f0d080d3c30 2023-11-28T18:26:00.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.352+0000 7f0d0cc36640 1 -- 172.21.15.105:0/3088162134 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf80a7050 con 0x7f0d080d3c30 2023-11-28T18:26:00.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.352+0000 7f0d0cc36640 1 -- 172.21.15.105:0/3088162134 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf80ab980 con 0x7f0d080d3c30 2023-11-28T18:26:00.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.352+0000 7f0d0fe9c640 1 -- 172.21.15.105:0/3088162134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d080d3c30 msgr2=0x7f0d080d1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:00.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.352+0000 7f0d0fe9c640 1 --2- 172.21.15.105:0/3088162134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d080d3c30 0x7f0d080d1350 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80097c0 tx=0x7f0cf801c3c0 comp rx=0 tx=0).stop 2023-11-28T18:26:00.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.352+0000 7f0d0fe9c640 1 -- 172.21.15.105:0/3088162134 shutdown_connections 2023-11-28T18:26:00.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.352+0000 7f0d0fe9c640 1 --2- 172.21.15.105:0/3088162134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d080d3c30 0x7f0d080d1350 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:00.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.352+0000 7f0d0fe9c640 1 --2- 172.21.15.105:0/3088162134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d0815d4f0 0x7f0d080d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:00.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.352+0000 7f0d0fe9c640 1 -- 172.21.15.105:0/3088162134 >> 172.21.15.105:0/3088162134 conn(0x7f0d080d3030 msgr2=0x7f0d080d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:00.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.352+0000 7f0d0fe9c640 1 -- 172.21.15.105:0/3088162134 shutdown_connections 2023-11-28T18:26:00.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.352+0000 7f0d0fe9c640 1 -- 172.21.15.105:0/3088162134 wait complete. 2023-11-28T18:26:00.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.352+0000 7f0d0fe9c640 1 Processor -- start 2023-11-28T18:26:00.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.352+0000 7f0d0fe9c640 1 -- start start 2023-11-28T18:26:00.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0d0fe9c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d080d1890 0x7f0d080c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:00.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0d0fe9c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d080d3c30 0x7f0d081552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:00.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0d0fe9c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d0815d4f0 0x7f0d08153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:00.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0d0fe9c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0815e440 con 0x7f0d080d3c30 2023-11-28T18:26:00.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0d0fe9c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d081609e0 con 0x7f0d0815d4f0 2023-11-28T18:26:00.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0d0fe9c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d08160cc0 con 0x7f0d080d1890 2023-11-28T18:26:00.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0d0d437640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d080d3c30 0x7f0d081552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:00.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0d0d437640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d080d3c30 0x7f0d081552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55464/0 (socket says 172.21.15.105:55464) 2023-11-28T18:26:00.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0d0d437640 1 -- 172.21.15.105:0/1899766478 learned_addr learned my addr 172.21.15.105:0/1899766478 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:00.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0d0dc38640 1 --2- 172.21.15.105:0/1899766478 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d080d1890 0x7f0d080c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:00.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0d0e439640 1 --2- 172.21.15.105:0/1899766478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d0815d4f0 0x7f0d08153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:00.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0d0d437640 1 -- 172.21.15.105:0/1899766478 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d080d1890 msgr2=0x7f0d080c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:00.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0d0d437640 1 --2- 172.21.15.105:0/1899766478 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d080d1890 0x7f0d080c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:00.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0d0d437640 1 -- 172.21.15.105:0/1899766478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d0815d4f0 msgr2=0x7f0d08153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:00.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0d0d437640 1 --2- 172.21.15.105:0/1899766478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d0815d4f0 0x7f0d08153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:00.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0d0d437640 1 -- 172.21.15.105:0/1899766478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d04009420 con 0x7f0d080d3c30 2023-11-28T18:26:00.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0d0d437640 1 --2- 172.21.15.105:0/1899766478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d080d3c30 0x7f0d081552c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80ae040 tx=0x7f0cf801ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:00.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0cfe7fc640 1 -- 172.21.15.105:0/1899766478 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8006e20 con 0x7f0d080d3c30 2023-11-28T18:26:00.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0cfe7fc640 1 -- 172.21.15.105:0/1899766478 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf80b7450 con 0x7f0d080d3c30 2023-11-28T18:26:00.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0cfe7fc640 1 -- 172.21.15.105:0/1899766478 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf80b7800 con 0x7f0d080d3c30 2023-11-28T18:26:00.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0d0fe9c640 1 -- 172.21.15.105:0/1899766478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf8009420 con 0x7f0d080d3c30 2023-11-28T18:26:00.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0d0fe9c640 1 -- 172.21.15.105:0/1899766478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d080ca800 con 0x7f0d080d3c30 2023-11-28T18:26:00.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.356+0000 7f0d0fe9c640 1 -- 172.21.15.105:0/1899766478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d08160e00 con 0x7f0d080d3c30 2023-11-28T18:26:00.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.360+0000 7f0cfe7fc640 1 -- 172.21.15.105:0/1899766478 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f0cf80b7960 con 0x7f0d080d3c30 2023-11-28T18:26:00.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.360+0000 7f0cfe7fc640 1 --2- 172.21.15.105:0/1899766478 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cec0668d0 0x7f0cec068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:00.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.360+0000 7f0cfe7fc640 1 -- 172.21.15.105:0/1899766478 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f0cf811f400 con 0x7f0d080d3c30 2023-11-28T18:26:00.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.360+0000 7f0d0dc38640 1 --2- 172.21.15.105:0/1899766478 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cec0668d0 0x7f0cec068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:00.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.360+0000 7f0d0dc38640 1 --2- 172.21.15.105:0/1899766478 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cec0668d0 0x7f0cec068d90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009c50 tx=0x7f0d04009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:00.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.364+0000 7f0cfe7fc640 1 -- 172.21.15.105:0/1899766478 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0cf80ec5f0 con 0x7f0d080d3c30 2023-11-28T18:26:00.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:00.544+0000 7f0d0fe9c640 1 -- 172.21.15.105:0/1899766478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0d08160e00 con 0x7f0d080d3c30 2023-11-28T18:26:01.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.164+0000 7f0cfe7fc640 1 -- 172.21.15.105:0/1899766478 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v31) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0d08160e00 con 0x7f0d080d3c30 2023-11-28T18:26:01.172 INFO:teuthology.orchestra.run.smithi105.stderr:new fs with metadata pool 7 and data pool 8 2023-11-28T18:26:01.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.172+0000 7f0d0fe9c640 1 -- 172.21.15.105:0/1899766478 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cec0668d0 msgr2=0x7f0cec068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:01.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.172+0000 7f0d0fe9c640 1 --2- 172.21.15.105:0/1899766478 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cec0668d0 0x7f0cec068d90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009c50 tx=0x7f0d04009190 comp rx=0 tx=0).stop 2023-11-28T18:26:01.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.172+0000 7f0d0fe9c640 1 -- 172.21.15.105:0/1899766478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d080d3c30 msgr2=0x7f0d081552c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:01.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.172+0000 7f0d0fe9c640 1 --2- 172.21.15.105:0/1899766478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d080d3c30 0x7f0d081552c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80ae040 tx=0x7f0cf801ac00 comp rx=0 tx=0).stop 2023-11-28T18:26:01.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.176+0000 7f0d0fe9c640 1 -- 172.21.15.105:0/1899766478 shutdown_connections 2023-11-28T18:26:01.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.176+0000 7f0d0fe9c640 1 --2- 172.21.15.105:0/1899766478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d0815d4f0 0x7f0d08153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:01.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.176+0000 7f0d0fe9c640 1 --2- 172.21.15.105:0/1899766478 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cec0668d0 0x7f0cec068d90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:01.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.176+0000 7f0d0fe9c640 1 --2- 172.21.15.105:0/1899766478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d080d3c30 0x7f0d081552c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:01.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.176+0000 7f0d0fe9c640 1 --2- 172.21.15.105:0/1899766478 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d080d1890 0x7f0d080c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:01.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.176+0000 7f0d0fe9c640 1 -- 172.21.15.105:0/1899766478 >> 172.21.15.105:0/1899766478 conn(0x7f0d080d3030 msgr2=0x7f0d080cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:01.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.176+0000 7f0d0fe9c640 1 -- 172.21.15.105:0/1899766478 shutdown_connections 2023-11-28T18:26:01.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.176+0000 7f0d0fe9c640 1 -- 172.21.15.105:0/1899766478 wait complete. 2023-11-28T18:26:01.211 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:26:01.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.348+0000 7f028272a640 1 Processor -- start 2023-11-28T18:26:01.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.348+0000 7f028272a640 1 -- start start 2023-11-28T18:26:01.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.348+0000 7f028272a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f027c15d4f0 0x7f027c0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:01.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.348+0000 7f028272a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f027c0b25b0 con 0x7f027c0d3c30 2023-11-28T18:26:01.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.348+0000 7f028272a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f027c0cea30 con 0x7f027c0b2d20 2023-11-28T18:26:01.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.348+0000 7f028272a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f027c0b2900 con 0x7f027c15d4f0 2023-11-28T18:26:01.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.348+0000 7f027bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f027c15d4f0 0x7f027c0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:01.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.348+0000 7f027b7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f027c0d3c30 0x7f027c0d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57920/0 (socket says 172.21.15.105:57920) 2023-11-28T18:26:01.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.352+0000 7f027b7fe640 1 -- 172.21.15.105:0/2202446650 learned_addr learned my addr 172.21.15.105:0/2202446650 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:01.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.352+0000 7f027bfff640 1 -- 172.21.15.105:0/2202446650 >> v1:172.21.15.148:6789/0 conn(0x7f027c0b2d20 legacy=0x7f027c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:01.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.352+0000 7f027bfff640 1 -- 172.21.15.105:0/2202446650 >> v1:172.21.15.105:6789/0 conn(0x7f027c0d3c30 legacy=0x7f027c0d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:01.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.352+0000 7f027bfff640 1 -- 172.21.15.105:0/2202446650 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027c0ced10 con 0x7f027c15d4f0 2023-11-28T18:26:01.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.352+0000 7f027affd640 1 -- 172.21.15.105:0/2202446650 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3040157891 0 0) 0x7f027c0b25b0 con 0x7f027c0d3c30 2023-11-28T18:26:01.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.352+0000 7f027bfff640 1 --2- 172.21.15.105:0/2202446650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f027c15d4f0 0x7f027c0d0e10 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f026c0097c0 tx=0x7f026c01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f27328f70366a5d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:01.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.352+0000 7f027affd640 1 -- 172.21.15.105:0/2202446650 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c0ab600 con 0x7f027c15d4f0 2023-11-28T18:26:01.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.352+0000 7f027affd640 1 -- 172.21.15.105:0/2202446650 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026c0a7050 con 0x7f027c15d4f0 2023-11-28T18:26:01.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.352+0000 7f027affd640 1 -- 172.21.15.105:0/2202446650 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c0ab980 con 0x7f027c15d4f0 2023-11-28T18:26:01.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f028272a640 1 -- 172.21.15.105:0/2202446650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f027c15d4f0 msgr2=0x7f027c0d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:01.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f028272a640 1 --2- 172.21.15.105:0/2202446650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f027c15d4f0 0x7f027c0d0e10 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f026c0097c0 tx=0x7f026c01c380 comp rx=0 tx=0).stop 2023-11-28T18:26:01.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f028272a640 1 -- 172.21.15.105:0/2202446650 shutdown_connections 2023-11-28T18:26:01.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f028272a640 1 --2- 172.21.15.105:0/2202446650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f027c15d4f0 0x7f027c0d0e10 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:01.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f028272a640 1 -- 172.21.15.105:0/2202446650 >> 172.21.15.105:0/2202446650 conn(0x7f027c0d3030 msgr2=0x7f027c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:01.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f028272a640 1 -- 172.21.15.105:0/2202446650 shutdown_connections 2023-11-28T18:26:01.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f028272a640 1 -- 172.21.15.105:0/2202446650 wait complete. 2023-11-28T18:26:01.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f028272a640 1 Processor -- start 2023-11-28T18:26:01.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f028272a640 1 -- start start 2023-11-28T18:26:01.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f028272a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f027c0b2d20 0x7f027c160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:01.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f028272a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f027c0d3c30 0x7f027c06c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:01.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f028272a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f027c15d4f0 0x7f027c06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:01.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f028272a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027c0d0a20 con 0x7f027c0b2d20 2023-11-28T18:26:01.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f028272a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027c0cb4c0 con 0x7f027c0d3c30 2023-11-28T18:26:01.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f028272a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027c06ce70 con 0x7f027c15d4f0 2023-11-28T18:26:01.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f027bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f027c0b2d20 0x7f027c160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:01.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f027bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f027c0b2d20 0x7f027c160230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45836/0 (socket says 172.21.15.105:45836) 2023-11-28T18:26:01.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f027bfff640 1 -- 172.21.15.105:0/550919208 learned_addr learned my addr 172.21.15.105:0/550919208 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:01.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f027b7fe640 1 --2- 172.21.15.105:0/550919208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f027c0d3c30 0x7f027c06c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:01.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f027bfff640 1 -- 172.21.15.105:0/550919208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f027c15d4f0 msgr2=0x7f027c06c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:01.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f027bfff640 1 --2- 172.21.15.105:0/550919208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f027c15d4f0 0x7f027c06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:01.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f027bfff640 1 -- 172.21.15.105:0/550919208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f027c0d3c30 msgr2=0x7f027c06c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:01.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f027bfff640 1 --2- 172.21.15.105:0/550919208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f027c0d3c30 0x7f027c06c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:01.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f027bfff640 1 -- 172.21.15.105:0/550919208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026c009420 con 0x7f027c0b2d20 2023-11-28T18:26:01.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.356+0000 7f027b7fe640 1 --2- 172.21.15.105:0/550919208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f027c0d3c30 0x7f027c06c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:01.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.360+0000 7f027bfff640 1 --2- 172.21.15.105:0/550919208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f027c0b2d20 0x7f027c160230 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f026c000c10 tx=0x7f026c019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:01.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.360+0000 7f0278ff9640 1 -- 172.21.15.105:0/550919208 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c004230 con 0x7f027c0b2d20 2023-11-28T18:26:01.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.360+0000 7f0278ff9640 1 -- 172.21.15.105:0/550919208 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026c004390 con 0x7f027c0b2d20 2023-11-28T18:26:01.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.360+0000 7f0278ff9640 1 -- 172.21.15.105:0/550919208 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c003820 con 0x7f027c0b2d20 2023-11-28T18:26:01.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.360+0000 7f028272a640 1 -- 172.21.15.105:0/550919208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f027c0cab20 con 0x7f027c0b2d20 2023-11-28T18:26:01.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.360+0000 7f028272a640 1 -- 172.21.15.105:0/550919208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f027c0cb0e0 con 0x7f027c0b2d20 2023-11-28T18:26:01.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.360+0000 7f028272a640 1 -- 172.21.15.105:0/550919208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f027c067570 con 0x7f027c0b2d20 2023-11-28T18:26:01.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.360+0000 7f0278ff9640 1 -- 172.21.15.105:0/550919208 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f026c0b2040 con 0x7f027c0b2d20 2023-11-28T18:26:01.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.360+0000 7f0278ff9640 1 --2- 172.21.15.105:0/550919208 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0268066900 0x7f0268068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:01.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.360+0000 7f0278ff9640 1 -- 172.21.15.105:0/550919208 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f026c11e5d0 con 0x7f027c0b2d20 2023-11-28T18:26:01.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.360+0000 7f027b7fe640 1 --2- 172.21.15.105:0/550919208 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0268066900 0x7f0268068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:01.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.364+0000 7f027b7fe640 1 --2- 172.21.15.105:0/550919208 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0268066900 0x7f0268068dc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0270007fd0 tx=0x7f0270009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:01.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.364+0000 7f0278ff9640 1 -- 172.21.15.105:0/550919208 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f026c0eb630 con 0x7f027c0b2d20 2023-11-28T18:26:01.497 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:26:01.501 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:26:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.516+0000 7f028272a640 1 -- 172.21.15.105:0/550919208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f027c0730e0 con 0x7f027c0b2d20 2023-11-28T18:26:01.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.516+0000 7f0278ff9640 1 -- 172.21.15.105:0/550919208 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16847 (secure 0 0 0) 0x7f026c0b0090 con 0x7f027c0b2d20 2023-11-28T18:26:01.521 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:01.521 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":50,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:26:00.549985+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:25:58.881270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T18:25:59.532434+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:26:01.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.520+0000 7f028272a640 1 -- 172.21.15.105:0/550919208 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0268066900 msgr2=0x7f0268068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:01.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.520+0000 7f028272a640 1 --2- 172.21.15.105:0/550919208 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0268066900 0x7f0268068dc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0270007fd0 tx=0x7f0270009040 comp rx=0 tx=0).stop 2023-11-28T18:26:01.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.520+0000 7f028272a640 1 -- 172.21.15.105:0/550919208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f027c0b2d20 msgr2=0x7f027c160230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:01.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.520+0000 7f028272a640 1 --2- 172.21.15.105:0/550919208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f027c0b2d20 0x7f027c160230 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f026c000c10 tx=0x7f026c019120 comp rx=0 tx=0).stop 2023-11-28T18:26:01.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.520+0000 7f028272a640 1 -- 172.21.15.105:0/550919208 shutdown_connections 2023-11-28T18:26:01.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.520+0000 7f028272a640 1 --2- 172.21.15.105:0/550919208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f027c15d4f0 0x7f027c06c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:01.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.520+0000 7f028272a640 1 --2- 172.21.15.105:0/550919208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f027c0d3c30 0x7f027c06c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:01.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.520+0000 7f028272a640 1 --2- 172.21.15.105:0/550919208 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0268066900 0x7f0268068dc0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:01.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.520+0000 7f028272a640 1 --2- 172.21.15.105:0/550919208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f027c0b2d20 0x7f027c160230 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:01.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.520+0000 7f028272a640 1 -- 172.21.15.105:0/550919208 >> 172.21.15.105:0/550919208 conn(0x7f027c0d3030 msgr2=0x7f027c0cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:01.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.520+0000 7f028272a640 1 -- 172.21.15.105:0/550919208 shutdown_connections 2023-11-28T18:26:01.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.520+0000 7f028272a640 1 -- 172.21.15.105:0/550919208 wait complete. 2023-11-28T18:26:01.542 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:26:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.724+0000 7fefb6479640 1 Processor -- start 2023-11-28T18:26:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.724+0000 7fefb6479640 1 -- start start 2023-11-28T18:26:01.727 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.724+0000 7fefb6479640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fefb0159f40 0x7fefb00caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:01.727 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.724+0000 7fefb6479640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefb00ad850 0x7fefb00701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:01.727 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.724+0000 7fefb6479640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb006bf50 con 0x7fefb00cc760 2023-11-28T18:26:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.724+0000 7fefb6479640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb00ccbc0 con 0x7fefb0159f40 2023-11-28T18:26:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.724+0000 7fefb6479640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb015a320 con 0x7fefb00ad850 2023-11-28T18:26:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.724+0000 7fefaffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fefb0159f40 0x7fefb00caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.724+0000 7fefaffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fefb0159f40 0x7fefb00caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45846/0 (socket says 172.21.15.105:45846) 2023-11-28T18:26:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.724+0000 7fefaffff640 1 -- 172.21.15.105:0/1602162962 learned_addr learned my addr 172.21.15.105:0/1602162962 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.724+0000 7fefb4a16640 1 --2- 172.21.15.105:0/1602162962 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefb00ad850 0x7fefb00701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.724+0000 7fefaffff640 1 -- 172.21.15.105:0/1602162962 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefb00ad850 msgr2=0x7fefb00701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.724+0000 7fefaffff640 1 --2- 172.21.15.105:0/1602162962 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefb00ad850 0x7fefb00701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.724+0000 7fefaffff640 1 -- 172.21.15.105:0/1602162962 >> v1:172.21.15.148:6789/0 conn(0x7fefb00cc760 legacy=0x7fefb00ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.724+0000 7fefaffff640 1 -- 172.21.15.105:0/1602162962 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb0071380 con 0x7fefb0159f40 2023-11-28T18:26:01.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.728+0000 7fefaffff640 1 --2- 172.21.15.105:0/1602162962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefb0159f40 0x7fefb00caf60 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009870 tx=0x7fefa001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc398ebafcb05507 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:01.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.728+0000 7fefaeffd640 1 -- 172.21.15.105:0/1602162962 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa00ab680 con 0x7fefb0159f40 2023-11-28T18:26:01.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.728+0000 7fefaeffd640 1 -- 172.21.15.105:0/1602162962 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa00ab7e0 con 0x7fefb0159f40 2023-11-28T18:26:01.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.728+0000 7fefaeffd640 1 -- 172.21.15.105:0/1602162962 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa00abb70 con 0x7fefb0159f40 2023-11-28T18:26:01.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.728+0000 7fefb6479640 1 -- 172.21.15.105:0/1602162962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefb0159f40 msgr2=0x7fefb00caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.728+0000 7fefb6479640 1 --2- 172.21.15.105:0/1602162962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefb0159f40 0x7fefb00caf60 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009870 tx=0x7fefa001c480 comp rx=0 tx=0).stop 2023-11-28T18:26:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.728+0000 7fefb6479640 1 -- 172.21.15.105:0/1602162962 shutdown_connections 2023-11-28T18:26:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.728+0000 7fefb6479640 1 --2- 172.21.15.105:0/1602162962 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefb00ad850 0x7fefb00701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.728+0000 7fefb6479640 1 --2- 172.21.15.105:0/1602162962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefb0159f40 0x7fefb00caf60 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.728+0000 7fefb6479640 1 -- 172.21.15.105:0/1602162962 >> 172.21.15.105:0/1602162962 conn(0x7fefb00ceb90 msgr2=0x7fefb00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.728+0000 7fefb6479640 1 -- 172.21.15.105:0/1602162962 shutdown_connections 2023-11-28T18:26:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.728+0000 7fefb6479640 1 -- 172.21.15.105:0/1602162962 wait complete. 2023-11-28T18:26:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.728+0000 7fefb6479640 1 Processor -- start 2023-11-28T18:26:01.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefb6479640 1 -- start start 2023-11-28T18:26:01.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefb6479640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefb00ad850 0x7fefb01f3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:01.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefb6479640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefb00cc760 0x7fefb01572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:01.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefb6479640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefb0159f40 0x7fefb0099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:01.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefb6479640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb01f44e0 con 0x7fefb0159f40 2023-11-28T18:26:01.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefb6479640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb01f47c0 con 0x7fefb00cc760 2023-11-28T18:26:01.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefb6479640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb01f4aa0 con 0x7fefb00ad850 2023-11-28T18:26:01.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefb4a16640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefb0159f40 0x7fefb0099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:01.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefb4a16640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefb0159f40 0x7fefb0099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45856/0 (socket says 172.21.15.105:45856) 2023-11-28T18:26:01.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefb4a16640 1 -- 172.21.15.105:0/714966384 learned_addr learned my addr 172.21.15.105:0/714966384 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:01.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefaffff640 1 --2- 172.21.15.105:0/714966384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefb00ad850 0x7fefb01f3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:01.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefaf7fe640 1 --2- 172.21.15.105:0/714966384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefb00cc760 0x7fefb01572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:01.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefb4a16640 1 -- 172.21.15.105:0/714966384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefb00ad850 msgr2=0x7fefb01f3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:01.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefb4a16640 1 --2- 172.21.15.105:0/714966384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefb00ad850 0x7fefb01f3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:01.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefb4a16640 1 -- 172.21.15.105:0/714966384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefb00cc760 msgr2=0x7fefb01572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:01.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefb4a16640 1 --2- 172.21.15.105:0/714966384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefb00cc760 0x7fefb01572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:01.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefb4a16640 1 -- 172.21.15.105:0/714966384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa00094d0 con 0x7fefb0159f40 2023-11-28T18:26:01.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefb4a16640 1 --2- 172.21.15.105:0/714966384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefb0159f40 0x7fefb0099a50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fefa8007e60 tx=0x7fefa800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:01.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefacff9640 1 -- 172.21.15.105:0/714966384 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa8014420 con 0x7fefb0159f40 2023-11-28T18:26:01.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefacff9640 1 -- 172.21.15.105:0/714966384 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa80145a0 con 0x7fefb0159f40 2023-11-28T18:26:01.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefb6479640 1 -- 172.21.15.105:0/714966384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb01f4d80 con 0x7fefb0159f40 2023-11-28T18:26:01.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefacff9640 1 -- 172.21.15.105:0/714966384 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa8014950 con 0x7fefb0159f40 2023-11-28T18:26:01.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefb6479640 1 -- 172.21.15.105:0/714966384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb01f41f0 con 0x7fefb0159f40 2023-11-28T18:26:01.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.732+0000 7fefb6479640 1 -- 172.21.15.105:0/714966384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb0067570 con 0x7fefb0159f40 2023-11-28T18:26:01.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.736+0000 7fefacff9640 1 -- 172.21.15.105:0/714966384 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fefa8014ab0 con 0x7fefb0159f40 2023-11-28T18:26:01.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.736+0000 7fefacff9640 1 --2- 172.21.15.105:0/714966384 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fef9c0668d0 0x7fef9c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:01.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.736+0000 7fefacff9640 1 -- 172.21.15.105:0/714966384 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fefa8084df0 con 0x7fefb0159f40 2023-11-28T18:26:01.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.736+0000 7fefaffff640 1 --2- 172.21.15.105:0/714966384 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fef9c0668d0 0x7fef9c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:01.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.736+0000 7fefaffff640 1 --2- 172.21.15.105:0/714966384 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fef9c0668d0 0x7fef9c068d90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fefa00af000 tx=0x7fefa00b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:01.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.740+0000 7fefacff9640 1 -- 172.21.15.105:0/714966384 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fefa8051f00 con 0x7fefb0159f40 2023-11-28T18:26:01.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.888+0000 7fefb6479640 1 -- 172.21.15.105:0/714966384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefb00cf950 con 0x7fefb0159f40 2023-11-28T18:26:01.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.892+0000 7fefacff9640 1 -- 172.21.15.105:0/714966384 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16847 (secure 0 0 0) 0x7fefa8055520 con 0x7fefb0159f40 2023-11-28T18:26:01.894 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:01.894 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":50,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:26:00.549985+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:25:58.881270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T18:25:59.532434+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:26:01.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.892+0000 7fefb6479640 1 -- 172.21.15.105:0/714966384 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fef9c0668d0 msgr2=0x7fef9c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:01.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.892+0000 7fefb6479640 1 --2- 172.21.15.105:0/714966384 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fef9c0668d0 0x7fef9c068d90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fefa00af000 tx=0x7fefa00b6040 comp rx=0 tx=0).stop 2023-11-28T18:26:01.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.892+0000 7fefb6479640 1 -- 172.21.15.105:0/714966384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefb0159f40 msgr2=0x7fefb0099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:01.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.892+0000 7fefb6479640 1 --2- 172.21.15.105:0/714966384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefb0159f40 0x7fefb0099a50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fefa8007e60 tx=0x7fefa800e4b0 comp rx=0 tx=0).stop 2023-11-28T18:26:01.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.892+0000 7fefb6479640 1 -- 172.21.15.105:0/714966384 shutdown_connections 2023-11-28T18:26:01.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.892+0000 7fefb6479640 1 --2- 172.21.15.105:0/714966384 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fef9c0668d0 0x7fef9c068d90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:01.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.892+0000 7fefb6479640 1 --2- 172.21.15.105:0/714966384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefb0159f40 0x7fefb0099a50 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:01.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.892+0000 7fefb6479640 1 --2- 172.21.15.105:0/714966384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefb00cc760 0x7fefb01572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:01.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.892+0000 7fefb6479640 1 --2- 172.21.15.105:0/714966384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefb00ad850 0x7fefb01f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:01.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.892+0000 7fefb6479640 1 -- 172.21.15.105:0/714966384 >> 172.21.15.105:0/714966384 conn(0x7fefb00ceb90 msgr2=0x7fefb00b39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:01.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.892+0000 7fefb6479640 1 -- 172.21.15.105:0/714966384 shutdown_connections 2023-11-28T18:26:01.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:01.892+0000 7fefb6479640 1 -- 172.21.15.105:0/714966384 wait complete. 2023-11-28T18:26:01.918 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T18:26:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.092+0000 7ff03189c640 1 Processor -- start 2023-11-28T18:26:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.092+0000 7ff03189c640 1 -- start start 2023-11-28T18:26:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.096+0000 7ff03189c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff02c159f40 0x7ff02c0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.096+0000 7ff03189c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff02c0cc760 0x7ff02c0ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.096+0000 7ff03189c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff02c06bf50 con 0x7ff02c0ad680 2023-11-28T18:26:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.096+0000 7ff03189c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff02c0ccbc0 con 0x7ff02c0cc760 2023-11-28T18:26:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.096+0000 7ff03189c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c15a320 con 0x7ff02c159f40 2023-11-28T18:26:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.096+0000 7ff02b7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff02c0ad680 0x7ff02c15da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57926/0 (socket says 172.21.15.105:57926) 2023-11-28T18:26:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.096+0000 7ff02b7fe640 1 -- 172.21.15.105:0/2017425945 learned_addr learned my addr 172.21.15.105:0/2017425945 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.096+0000 7ff02affd640 1 --2- 172.21.15.105:0/2017425945 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff02c159f40 0x7ff02c0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.096+0000 7ff02a7fc640 1 --2- 172.21.15.105:0/2017425945 >> v2:172.21.15.148:3300/0 conn(0x7ff02c0cc760 0x7ff02c0ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.096+0000 7ff029ffb640 1 -- 172.21.15.105:0/2017425945 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 529865828 0 0) 0x7ff02c06bf50 con 0x7ff02c0ad680 2023-11-28T18:26:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.096+0000 7ff029ffb640 1 -- 172.21.15.105:0/2017425945 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff014005740 con 0x7ff02c0ad680 2023-11-28T18:26:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.096+0000 7ff029ffb640 1 -- 172.21.15.105:0/2017425945 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1469488262 0 0) 0x7ff014005740 con 0x7ff02c0ad680 2023-11-28T18:26:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.096+0000 7ff02affd640 1 -- 172.21.15.105:0/2017425945 >> v2:172.21.15.148:3300/0 conn(0x7ff02c0cc760 msgr2=0x7ff02c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.096+0000 7ff02affd640 1 --2- 172.21.15.105:0/2017425945 >> v2:172.21.15.148:3300/0 conn(0x7ff02c0cc760 0x7ff02c0ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.096+0000 7ff02affd640 1 -- 172.21.15.105:0/2017425945 >> v1:172.21.15.105:6789/0 conn(0x7ff02c0ad680 legacy=0x7ff02c15da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.096+0000 7ff02affd640 1 -- 172.21.15.105:0/2017425945 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02c071380 con 0x7ff02c159f40 2023-11-28T18:26:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.096+0000 7ff02a7fc640 1 --2- 172.21.15.105:0/2017425945 >> v2:172.21.15.148:3300/0 conn(0x7ff02c0cc760 0x7ff02c0ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:26:02.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.100+0000 7ff02affd640 1 --2- 172.21.15.105:0/2017425945 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff02c159f40 0x7ff02c0caf60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff01c0097c0 tx=0x7ff01c01c2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=229fde56d0fd2f51 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.100+0000 7ff029ffb640 1 -- 172.21.15.105:0/2017425945 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff024001a00 con 0x7ff02c159f40 2023-11-28T18:26:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.100+0000 7ff029ffb640 1 -- 172.21.15.105:0/2017425945 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01c0a7050 con 0x7ff02c159f40 2023-11-28T18:26:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.100+0000 7ff029ffb640 1 -- 172.21.15.105:0/2017425945 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c0ab600 con 0x7ff02c159f40 2023-11-28T18:26:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.100+0000 7ff03189c640 1 -- 172.21.15.105:0/2017425945 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff02c159f40 msgr2=0x7ff02c0caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.100+0000 7ff03189c640 1 --2- 172.21.15.105:0/2017425945 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff02c159f40 0x7ff02c0caf60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff01c0097c0 tx=0x7ff01c01c2e0 comp rx=0 tx=0).stop 2023-11-28T18:26:02.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.100+0000 7ff03189c640 1 -- 172.21.15.105:0/2017425945 shutdown_connections 2023-11-28T18:26:02.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.100+0000 7ff03189c640 1 --2- 172.21.15.105:0/2017425945 >> v2:172.21.15.148:3300/0 conn(0x7ff02c0cc760 0x7ff02c0ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:02.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.100+0000 7ff03189c640 1 --2- 172.21.15.105:0/2017425945 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff02c159f40 0x7ff02c0caf60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:02.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.100+0000 7ff03189c640 1 -- 172.21.15.105:0/2017425945 >> 172.21.15.105:0/2017425945 conn(0x7ff02c0ceb90 msgr2=0x7ff02c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:02.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.100+0000 7ff03189c640 1 -- 172.21.15.105:0/2017425945 shutdown_connections 2023-11-28T18:26:02.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.100+0000 7ff03189c640 1 -- 172.21.15.105:0/2017425945 wait complete. 2023-11-28T18:26:02.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.100+0000 7ff03189c640 1 Processor -- start 2023-11-28T18:26:02.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.100+0000 7ff03189c640 1 -- start start 2023-11-28T18:26:02.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff03189c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff02c0ad680 0x7ff02c14b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:02.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff03189c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff02c0cc760 0x7ff02c148970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:02.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff03189c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff02c159f40 0x7ff02c147bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:02.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff03189c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c147790 con 0x7ff02c0ad680 2023-11-28T18:26:02.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff03189c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c146ec0 con 0x7ff02c0cc760 2023-11-28T18:26:02.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff03189c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c14be10 con 0x7ff02c159f40 2023-11-28T18:26:02.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff02affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff02c0ad680 0x7ff02c14b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:02.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff02affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff02c0ad680 0x7ff02c14b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45858/0 (socket says 172.21.15.105:45858) 2023-11-28T18:26:02.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff02affd640 1 -- 172.21.15.105:0/1380214476 learned_addr learned my addr 172.21.15.105:0/1380214476 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:02.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff02b7fe640 1 --2- 172.21.15.105:0/1380214476 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff02c159f40 0x7ff02c147bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:02.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff02a7fc640 1 --2- 172.21.15.105:0/1380214476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff02c0cc760 0x7ff02c148970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:02.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff02affd640 1 -- 172.21.15.105:0/1380214476 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff02c159f40 msgr2=0x7ff02c147bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:02.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff02affd640 1 --2- 172.21.15.105:0/1380214476 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff02c159f40 0x7ff02c147bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:02.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff02affd640 1 -- 172.21.15.105:0/1380214476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff02c0cc760 msgr2=0x7ff02c148970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:02.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff02affd640 1 --2- 172.21.15.105:0/1380214476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff02c0cc760 0x7ff02c148970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:02.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff02affd640 1 -- 172.21.15.105:0/1380214476 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff020009420 con 0x7ff02c0ad680 2023-11-28T18:26:02.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff02affd640 1 --2- 172.21.15.105:0/1380214476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff02c0ad680 0x7ff02c14b830 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff01c01a990 tx=0x7ff01c019080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:02.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff013fff640 1 -- 172.21.15.105:0/1380214476 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c006e20 con 0x7ff02c0ad680 2023-11-28T18:26:02.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff013fff640 1 -- 172.21.15.105:0/1380214476 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01c0b4400 con 0x7ff02c0ad680 2023-11-28T18:26:02.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff013fff640 1 -- 172.21.15.105:0/1380214476 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c0b4770 con 0x7ff02c0ad680 2023-11-28T18:26:02.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff03189c640 1 -- 172.21.15.105:0/1380214476 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff01c009420 con 0x7ff02c0ad680 2023-11-28T18:26:02.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff03189c640 1 -- 172.21.15.105:0/1380214476 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff02c1473b0 con 0x7ff02c0ad680 2023-11-28T18:26:02.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff03189c640 1 -- 172.21.15.105:0/1380214476 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff02c067570 con 0x7ff02c0ad680 2023-11-28T18:26:02.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.104+0000 7ff013fff640 1 -- 172.21.15.105:0/1380214476 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff01c0b2040 con 0x7ff02c0ad680 2023-11-28T18:26:02.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.108+0000 7ff013fff640 1 --2- 172.21.15.105:0/1380214476 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff0180668d0 0x7ff018068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:02.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.108+0000 7ff013fff640 1 -- 172.21.15.105:0/1380214476 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff01c11e410 con 0x7ff02c0ad680 2023-11-28T18:26:02.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.108+0000 7ff02a7fc640 1 --2- 172.21.15.105:0/1380214476 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff0180668d0 0x7ff018068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:02.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.108+0000 7ff02a7fc640 1 --2- 172.21.15.105:0/1380214476 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff0180668d0 0x7ff018068d90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0b4c80 tx=0x7ff020008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:02.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.112+0000 7ff013fff640 1 -- 172.21.15.105:0/1380214476 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff01c0eb510 con 0x7ff02c0ad680 2023-11-28T18:26:02.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:02.300+0000 7ff03189c640 1 -- 172.21.15.105:0/1380214476 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff02c073380 con 0x7ff02c0ad680 2023-11-28T18:26:03.136 INFO:tasks.ceph.mds.b.smithi148.stderr:2023-11-28T18:26:03.132+0000 7f609c6a2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T18:26:03.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.176+0000 7ff013fff640 1 -- 172.21.15.105:0/1380214476 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff01c0eeb30 con 0x7ff02c0ad680 2023-11-28T18:26:03.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.180+0000 7ff03189c640 1 -- 172.21.15.105:0/1380214476 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff0180668d0 msgr2=0x7ff018068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:03.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.180+0000 7ff03189c640 1 --2- 172.21.15.105:0/1380214476 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff0180668d0 0x7ff018068d90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0b4c80 tx=0x7ff020008040 comp rx=0 tx=0).stop 2023-11-28T18:26:03.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.180+0000 7ff03189c640 1 -- 172.21.15.105:0/1380214476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff02c0ad680 msgr2=0x7ff02c14b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:03.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.180+0000 7ff03189c640 1 --2- 172.21.15.105:0/1380214476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff02c0ad680 0x7ff02c14b830 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff01c01a990 tx=0x7ff01c019080 comp rx=0 tx=0).stop 2023-11-28T18:26:03.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.180+0000 7ff03189c640 1 -- 172.21.15.105:0/1380214476 shutdown_connections 2023-11-28T18:26:03.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.180+0000 7ff03189c640 1 --2- 172.21.15.105:0/1380214476 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff02c159f40 0x7ff02c147bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:03.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.180+0000 7ff03189c640 1 --2- 172.21.15.105:0/1380214476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff02c0cc760 0x7ff02c148970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:03.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.180+0000 7ff03189c640 1 --2- 172.21.15.105:0/1380214476 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff0180668d0 0x7ff018068d90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:03.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.180+0000 7ff03189c640 1 --2- 172.21.15.105:0/1380214476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff02c0ad680 0x7ff02c14b830 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:03.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.180+0000 7ff03189c640 1 -- 172.21.15.105:0/1380214476 >> 172.21.15.105:0/1380214476 conn(0x7ff02c0ceb90 msgr2=0x7ff02c0ae7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:03.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.180+0000 7ff03189c640 1 -- 172.21.15.105:0/1380214476 shutdown_connections 2023-11-28T18:26:03.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.180+0000 7ff03189c640 1 -- 172.21.15.105:0/1380214476 wait complete. 2023-11-28T18:26:03.205 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T18:26:03.205 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T18:26:03.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.344+0000 7f253930f640 1 Processor -- start 2023-11-28T18:26:03.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.348+0000 7f253930f640 1 -- start start 2023-11-28T18:26:03.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.348+0000 7f253930f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2534143470 0x7f25340b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:03.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.348+0000 7f253930f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f253406c050 0x7f253406c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:03.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.348+0000 7f253930f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25340b1f60 con 0x7f25340cec40 2023-11-28T18:26:03.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.348+0000 7f253930f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2534070300 con 0x7f253406c050 2023-11-28T18:26:03.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.348+0000 7f253930f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25340705a0 con 0x7f2534143470 2023-11-28T18:26:03.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.348+0000 7f252a7fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f253406c050 0x7f253406c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:03.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.348+0000 7f252a7fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f253406c050 0x7f253406c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45864/0 (socket says 172.21.15.105:45864) 2023-11-28T18:26:03.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.348+0000 7f252a7fc640 1 -- 172.21.15.105:0/1860637752 learned_addr learned my addr 172.21.15.105:0/1860637752 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:03.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.348+0000 7f2532ffd640 1 --2- 172.21.15.105:0/1860637752 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2534143470 0x7f25340b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:03.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.348+0000 7f252a7fc640 1 -- 172.21.15.105:0/1860637752 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2534143470 msgr2=0x7f25340b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:03.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.348+0000 7f252a7fc640 1 --2- 172.21.15.105:0/1860637752 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2534143470 0x7f25340b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:03.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.348+0000 7f252a7fc640 1 -- 172.21.15.105:0/1860637752 >> v1:172.21.15.148:6789/0 conn(0x7f25340cec40 legacy=0x7f2534071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:03.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.348+0000 7f252a7fc640 1 -- 172.21.15.105:0/1860637752 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2534070880 con 0x7f253406c050 2023-11-28T18:26:03.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.352+0000 7f252a7fc640 1 --2- 172.21.15.105:0/1860637752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f253406c050 0x7f253406c4b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f251c00bb10 tx=0x7f251c01ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86c1f38833eb060 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:03.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.352+0000 7f25327fc640 1 -- 172.21.15.105:0/1860637752 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c0a7bd0 con 0x7f253406c050 2023-11-28T18:26:03.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.352+0000 7f25327fc640 1 -- 172.21.15.105:0/1860637752 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251c0a7d30 con 0x7f253406c050 2023-11-28T18:26:03.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.352+0000 7f25327fc640 1 -- 172.21.15.105:0/1860637752 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c0a7ea0 con 0x7f253406c050 2023-11-28T18:26:03.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.352+0000 7f253930f640 1 -- 172.21.15.105:0/1860637752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f253406c050 msgr2=0x7f253406c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:03.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.352+0000 7f253930f640 1 --2- 172.21.15.105:0/1860637752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f253406c050 0x7f253406c4b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f251c00bb10 tx=0x7f251c01ebe0 comp rx=0 tx=0).stop 2023-11-28T18:26:03.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.352+0000 7f253930f640 1 -- 172.21.15.105:0/1860637752 shutdown_connections 2023-11-28T18:26:03.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.352+0000 7f253930f640 1 --2- 172.21.15.105:0/1860637752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f253406c050 0x7f253406c4b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:03.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.352+0000 7f253930f640 1 --2- 172.21.15.105:0/1860637752 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2534143470 0x7f25340b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:03.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.352+0000 7f253930f640 1 -- 172.21.15.105:0/1860637752 >> 172.21.15.105:0/1860637752 conn(0x7f253406a750 msgr2=0x7f253406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:03.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.352+0000 7f253930f640 1 -- 172.21.15.105:0/1860637752 shutdown_connections 2023-11-28T18:26:03.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.352+0000 7f253930f640 1 -- 172.21.15.105:0/1860637752 wait complete. 2023-11-28T18:26:03.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.352+0000 7f253930f640 1 Processor -- start 2023-11-28T18:26:03.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.352+0000 7f253930f640 1 -- start start 2023-11-28T18:26:03.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f253930f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f253406c050 0x7f2534159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:03.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f253930f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25340cec40 0x7f253415cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:03.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f253930f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2534143470 0x7f253415d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:03.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f253930f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253415c270 con 0x7f253406c050 2023-11-28T18:26:03.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f253930f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253415b1d0 con 0x7f2534143470 2023-11-28T18:26:03.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f253930f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253415b4b0 con 0x7f25340cec40 2023-11-28T18:26:03.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f2532ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f253406c050 0x7f2534159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:03.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f2532ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f253406c050 0x7f2534159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45878/0 (socket says 172.21.15.105:45878) 2023-11-28T18:26:03.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f2532ffd640 1 -- 172.21.15.105:0/3290265156 learned_addr learned my addr 172.21.15.105:0/3290265156 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:03.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f252a7fc640 1 --2- 172.21.15.105:0/3290265156 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25340cec40 0x7f253415cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:03.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f25337fe640 1 --2- 172.21.15.105:0/3290265156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2534143470 0x7f253415d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:03.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f2532ffd640 1 -- 172.21.15.105:0/3290265156 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25340cec40 msgr2=0x7f253415cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:03.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f2532ffd640 1 --2- 172.21.15.105:0/3290265156 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25340cec40 0x7f253415cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:03.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f2532ffd640 1 -- 172.21.15.105:0/3290265156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2534143470 msgr2=0x7f253415d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:03.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f2532ffd640 1 --2- 172.21.15.105:0/3290265156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2534143470 0x7f253415d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:03.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f2532ffd640 1 -- 172.21.15.105:0/3290265156 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251c00b770 con 0x7f253406c050 2023-11-28T18:26:03.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f25337fe640 1 --2- 172.21.15.105:0/3290265156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2534143470 0x7f253415d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:03.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f2532ffd640 1 --2- 172.21.15.105:0/3290265156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f253406c050 0x7f2534159410 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f252400d5f0 tx=0x7f252400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:03.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f252bfff640 1 -- 172.21.15.105:0/3290265156 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25240125f0 con 0x7f253406c050 2023-11-28T18:26:03.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f252bfff640 1 -- 172.21.15.105:0/3290265156 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2524012750 con 0x7f253406c050 2023-11-28T18:26:03.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f252bfff640 1 -- 172.21.15.105:0/3290265156 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2524012ae0 con 0x7f253406c050 2023-11-28T18:26:03.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f253930f640 1 -- 172.21.15.105:0/3290265156 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25340cf5e0 con 0x7f253406c050 2023-11-28T18:26:03.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f253930f640 1 -- 172.21.15.105:0/3290265156 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253415ba70 con 0x7f253406c050 2023-11-28T18:26:03.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.356+0000 7f253930f640 1 -- 172.21.15.105:0/3290265156 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2534074580 con 0x7f253406c050 2023-11-28T18:26:03.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.360+0000 7f252bfff640 1 -- 172.21.15.105:0/3290265156 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f2524012c40 con 0x7f253406c050 2023-11-28T18:26:03.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.360+0000 7f252bfff640 1 --2- 172.21.15.105:0/3290265156 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f25140668d0 0x7f2514068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:03.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.360+0000 7f252bfff640 1 -- 172.21.15.105:0/3290265156 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2524084950 con 0x7f253406c050 2023-11-28T18:26:03.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.360+0000 7f252a7fc640 1 --2- 172.21.15.105:0/3290265156 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f25140668d0 0x7f2514068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:03.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.360+0000 7f252a7fc640 1 --2- 172.21.15.105:0/3290265156 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f25140668d0 0x7f2514068d90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f251c0b1000 tx=0x7f251c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:03.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.364+0000 7f252bfff640 1 -- 172.21.15.105:0/3290265156 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2524051ac0 con 0x7f253406c050 2023-11-28T18:26:03.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:03.548+0000 7f253930f640 1 -- 172.21.15.105:0/3290265156 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2534164b00 con 0x7f253406c050 2023-11-28T18:26:04.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.184+0000 7f252bfff640 1 -- 172.21.15.105:0/3290265156 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2524007380 con 0x7f253406c050 2023-11-28T18:26:04.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.184+0000 7f253930f640 1 -- 172.21.15.105:0/3290265156 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f25140668d0 msgr2=0x7f2514068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:04.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.184+0000 7f253930f640 1 --2- 172.21.15.105:0/3290265156 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f25140668d0 0x7f2514068d90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f251c0b1000 tx=0x7f251c008040 comp rx=0 tx=0).stop 2023-11-28T18:26:04.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.184+0000 7f253930f640 1 -- 172.21.15.105:0/3290265156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f253406c050 msgr2=0x7f2534159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:04.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.184+0000 7f253930f640 1 --2- 172.21.15.105:0/3290265156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f253406c050 0x7f2534159410 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f252400d5f0 tx=0x7f252400dab0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.184+0000 7f253930f640 1 -- 172.21.15.105:0/3290265156 shutdown_connections 2023-11-28T18:26:04.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.184+0000 7f253930f640 1 --2- 172.21.15.105:0/3290265156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2534143470 0x7f253415d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.184+0000 7f253930f640 1 --2- 172.21.15.105:0/3290265156 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25340cec40 0x7f253415cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.184+0000 7f253930f640 1 --2- 172.21.15.105:0/3290265156 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f25140668d0 0x7f2514068d90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.184+0000 7f253930f640 1 --2- 172.21.15.105:0/3290265156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f253406c050 0x7f2534159410 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.184+0000 7f253930f640 1 -- 172.21.15.105:0/3290265156 >> 172.21.15.105:0/3290265156 conn(0x7f253406a750 msgr2=0x7f25340b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:04.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.184+0000 7f253930f640 1 -- 172.21.15.105:0/3290265156 shutdown_connections 2023-11-28T18:26:04.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.184+0000 7f253930f640 1 -- 172.21.15.105:0/3290265156 wait complete. 2023-11-28T18:26:04.210 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T18:26:04.210 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:26:04.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.348+0000 7f6741617640 1 Processor -- start 2023-11-28T18:26:04.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.348+0000 7f6741617640 1 -- start start 2023-11-28T18:26:04.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.348+0000 7f6741617640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f673c160680 0x7f673c0cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:04.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.348+0000 7f6741617640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f673c0d1a80 0x7f673c0c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:04.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.348+0000 7f6741617640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f673c0ced00 con 0x7f673c160a60 2023-11-28T18:26:04.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.348+0000 7f6741617640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f673c0d38b0 con 0x7f673c160680 2023-11-28T18:26:04.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.348+0000 7f6741617640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f673c0d3ac0 con 0x7f673c0d1a80 2023-11-28T18:26:04.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.348+0000 7f673affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f673c160680 0x7f673c0cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:04.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.348+0000 7f673affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f673c160680 0x7f673c0cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45888/0 (socket says 172.21.15.105:45888) 2023-11-28T18:26:04.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.348+0000 7f673affd640 1 -- 172.21.15.105:0/2147180254 learned_addr learned my addr 172.21.15.105:0/2147180254 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:04.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.348+0000 7f673b7fe640 1 --2- 172.21.15.105:0/2147180254 >> v2:172.21.15.148:3300/0 conn(0x7f673c0d1a80 0x7f673c0c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:04.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.348+0000 7f673affd640 1 -- 172.21.15.105:0/2147180254 >> v2:172.21.15.148:3300/0 conn(0x7f673c0d1a80 msgr2=0x7f673c0c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:04.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.348+0000 7f673affd640 1 --2- 172.21.15.105:0/2147180254 >> v2:172.21.15.148:3300/0 conn(0x7f673c0d1a80 0x7f673c0c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.348+0000 7f673affd640 1 -- 172.21.15.105:0/2147180254 >> v1:172.21.15.148:6789/0 conn(0x7f673c160a60 legacy=0x7f673c0d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:04.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.348+0000 7f673affd640 1 -- 172.21.15.105:0/2147180254 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673c0d0cf0 con 0x7f673c160680 2023-11-28T18:26:04.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.352+0000 7f673affd640 1 --2- 172.21.15.105:0/2147180254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f673c160680 0x7f673c0cfda0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f673c0c98b0 tx=0x7f672c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad23a9834c104b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:04.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.352+0000 7f6739ffb640 1 -- 172.21.15.105:0/2147180254 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c0aa680 con 0x7f673c160680 2023-11-28T18:26:04.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.352+0000 7f6739ffb640 1 -- 172.21.15.105:0/2147180254 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672c0aa7e0 con 0x7f673c160680 2023-11-28T18:26:04.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.352+0000 7f6739ffb640 1 -- 172.21.15.105:0/2147180254 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c0aab70 con 0x7f673c160680 2023-11-28T18:26:04.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.352+0000 7f6741617640 1 -- 172.21.15.105:0/2147180254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f673c160680 msgr2=0x7f673c0cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:04.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.352+0000 7f6741617640 1 --2- 172.21.15.105:0/2147180254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f673c160680 0x7f673c0cfda0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f673c0c98b0 tx=0x7f672c01c500 comp rx=0 tx=0).stop 2023-11-28T18:26:04.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.352+0000 7f6741617640 1 -- 172.21.15.105:0/2147180254 shutdown_connections 2023-11-28T18:26:04.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.352+0000 7f6741617640 1 --2- 172.21.15.105:0/2147180254 >> v2:172.21.15.148:3300/0 conn(0x7f673c0d1a80 0x7f673c0c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.352+0000 7f6741617640 1 --2- 172.21.15.105:0/2147180254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f673c160680 0x7f673c0cfda0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.352+0000 7f6741617640 1 -- 172.21.15.105:0/2147180254 >> 172.21.15.105:0/2147180254 conn(0x7f673c0d2d90 msgr2=0x7f673c0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:04.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.352+0000 7f6741617640 1 -- 172.21.15.105:0/2147180254 shutdown_connections 2023-11-28T18:26:04.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.352+0000 7f6741617640 1 -- 172.21.15.105:0/2147180254 wait complete. 2023-11-28T18:26:04.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f6741617640 1 Processor -- start 2023-11-28T18:26:04.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f6741617640 1 -- start start 2023-11-28T18:26:04.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f6741617640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f673c0d1a80 0x7f673c1561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:04.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f6741617640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f673c160680 0x7f673c15f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:04.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f6741617640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f673c160a60 0x7f673c0b6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:04.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f6741617640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c14bd20 con 0x7f673c160a60 2023-11-28T18:26:04.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f6741617640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c0b8810 con 0x7f673c160680 2023-11-28T18:26:04.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f6741617640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673c15fed0 con 0x7f673c0d1a80 2023-11-28T18:26:04.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f673b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f673c160a60 0x7f673c0b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:04.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f673b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f673c160a60 0x7f673c0b6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45904/0 (socket says 172.21.15.105:45904) 2023-11-28T18:26:04.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f673a7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f673c160680 0x7f673c15f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:04.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f673b7fe640 1 -- 172.21.15.105:0/2937043210 learned_addr learned my addr 172.21.15.105:0/2937043210 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:04.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f673affd640 1 --2- 172.21.15.105:0/2937043210 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f673c0d1a80 0x7f673c1561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:04.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f673b7fe640 1 -- 172.21.15.105:0/2937043210 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f673c0d1a80 msgr2=0x7f673c1561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:04.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f673b7fe640 1 --2- 172.21.15.105:0/2937043210 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f673c0d1a80 0x7f673c1561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f673b7fe640 1 -- 172.21.15.105:0/2937043210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f673c160680 msgr2=0x7f673c15f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:04.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f673b7fe640 1 --2- 172.21.15.105:0/2937043210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f673c160680 0x7f673c15f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f673b7fe640 1 -- 172.21.15.105:0/2937043210 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672c0094d0 con 0x7f673c160a60 2023-11-28T18:26:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f673b7fe640 1 --2- 172.21.15.105:0/2937043210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f673c160a60 0x7f673c0b6640 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f673400efc0 tx=0x7f673400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f67237fe640 1 -- 172.21.15.105:0/2937043210 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6734014480 con 0x7f673c160a60 2023-11-28T18:26:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f67237fe640 1 -- 172.21.15.105:0/2937043210 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673400f040 con 0x7f673c160a60 2023-11-28T18:26:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f6741617640 1 -- 172.21.15.105:0/2937043210 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673c0c7b40 con 0x7f673c160a60 2023-11-28T18:26:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f67237fe640 1 -- 172.21.15.105:0/2937043210 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6734014820 con 0x7f673c160a60 2023-11-28T18:26:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f6741617640 1 -- 172.21.15.105:0/2937043210 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673c14c470 con 0x7f673c160a60 2023-11-28T18:26:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.356+0000 7f6741617640 1 -- 172.21.15.105:0/2937043210 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f673c074580 con 0x7f673c160a60 2023-11-28T18:26:04.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.360+0000 7f67237fe640 1 -- 172.21.15.105:0/2937043210 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f6734014980 con 0x7f673c160a60 2023-11-28T18:26:04.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.360+0000 7f67237fe640 1 --2- 172.21.15.105:0/2937043210 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f67280668d0 0x7f6728068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:04.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.360+0000 7f67237fe640 1 -- 172.21.15.105:0/2937043210 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6734085340 con 0x7f673c160a60 2023-11-28T18:26:04.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.360+0000 7f673affd640 1 --2- 172.21.15.105:0/2937043210 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f67280668d0 0x7f6728068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:04.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.360+0000 7f673affd640 1 --2- 172.21.15.105:0/2937043210 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f67280668d0 0x7f6728068d90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f672c0ae000 tx=0x7f672c0b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:04.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.364+0000 7f67237fe640 1 -- 172.21.15.105:0/2937043210 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6734052450 con 0x7f673c160a60 2023-11-28T18:26:04.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.540+0000 7f6741617640 1 -- 172.21.15.105:0/2937043210 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f673c157020 con 0x7f673c160a60 2023-11-28T18:26:04.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.544+0000 7f67237fe640 1 -- 172.21.15.105:0/2937043210 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f6734055a70 con 0x7f673c160a60 2023-11-28T18:26:04.546 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:04.546 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":35,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/735646078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":735646078},{"type":"v1","addr":"172.21.15.105:6836","nonce":735646078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5408,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/3765684108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":3765684108},{"type":"v1","addr":"172.21.15.105:6837","nonce":3765684108}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/897997046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":897997046},{"type":"v1","addr":"172.21.15.148:6835","nonce":897997046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:26:00.550298+0000","modified":"2023-11-28T18:26:03.550508+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5842},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5842":{"gid":5842,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6834/2696773413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2696773413},{"type":"v1","addr":"172.21.15.148:6834","nonce":2696773413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T18:26:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 35 2023-11-28T18:26:04.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.544+0000 7f6741617640 1 -- 172.21.15.105:0/2937043210 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f67280668d0 msgr2=0x7f6728068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:04.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.544+0000 7f6741617640 1 --2- 172.21.15.105:0/2937043210 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f67280668d0 0x7f6728068d90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f672c0ae000 tx=0x7f672c0b5040 comp rx=0 tx=0).stop 2023-11-28T18:26:04.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.544+0000 7f6741617640 1 -- 172.21.15.105:0/2937043210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f673c160a60 msgr2=0x7f673c0b6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:04.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.544+0000 7f6741617640 1 --2- 172.21.15.105:0/2937043210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f673c160a60 0x7f673c0b6640 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f673400efc0 tx=0x7f673400c490 comp rx=0 tx=0).stop 2023-11-28T18:26:04.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.544+0000 7f6741617640 1 -- 172.21.15.105:0/2937043210 shutdown_connections 2023-11-28T18:26:04.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.544+0000 7f6741617640 1 --2- 172.21.15.105:0/2937043210 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f67280668d0 0x7f6728068d90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.544+0000 7f6741617640 1 --2- 172.21.15.105:0/2937043210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f673c160a60 0x7f673c0b6640 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.544+0000 7f6741617640 1 --2- 172.21.15.105:0/2937043210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f673c160680 0x7f673c15f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.544+0000 7f6741617640 1 --2- 172.21.15.105:0/2937043210 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f673c0d1a80 0x7f673c1561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.544+0000 7f6741617640 1 -- 172.21.15.105:0/2937043210 >> 172.21.15.105:0/2937043210 conn(0x7f673c0d2d90 msgr2=0x7f673c0d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:04.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.544+0000 7f6741617640 1 -- 172.21.15.105:0/2937043210 shutdown_connections 2023-11-28T18:26:04.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.544+0000 7f6741617640 1 -- 172.21.15.105:0/2937043210 wait complete. 2023-11-28T18:26:04.567 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:26:04.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.700+0000 7f6bf7a17640 1 Processor -- start 2023-11-28T18:26:04.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.700+0000 7f6bf7a17640 1 -- start start 2023-11-28T18:26:04.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6bf7a17640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bf00ad9f0 0x7f6bf00ade50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:04.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6bf7a17640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bf00ae390 con 0x7f6bf006d610 2023-11-28T18:26:04.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6bf7a17640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bf0162820 con 0x7f6bf00cc7d0 2023-11-28T18:26:04.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6bf7a17640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf0161c10 con 0x7f6bf00ad9f0 2023-11-28T18:26:04.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6bf57b3640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f6bf006d610 0x7f6bf00cdff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57940/0 (socket says 172.21.15.105:57940) 2023-11-28T18:26:04.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6bf57b3640 1 -- 172.21.15.105:0/3404028632 learned_addr learned my addr 172.21.15.105:0/3404028632 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:04.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6bf4fb2640 1 --2- 172.21.15.105:0/3404028632 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bf00ad9f0 0x7f6bf00ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:04.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6be7fff640 1 -- 172.21.15.105:0/3404028632 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3797801961 0 0) 0x7f6bf00ae390 con 0x7f6bf006d610 2023-11-28T18:26:04.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6be7fff640 1 -- 172.21.15.105:0/3404028632 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bdc003530 con 0x7f6bf006d610 2023-11-28T18:26:04.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6be7fff640 1 -- 172.21.15.105:0/3404028632 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2549494419 0 0) 0x7f6bdc003530 con 0x7f6bf006d610 2023-11-28T18:26:04.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6be7fff640 1 -- 172.21.15.105:0/3404028632 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bf00ad9f0 msgr2=0x7f6bf00ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:04.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6be7fff640 1 --2- 172.21.15.105:0/3404028632 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bf00ad9f0 0x7f6bf00ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6be7fff640 1 -- 172.21.15.105:0/3404028632 >> v1:172.21.15.148:6789/0 conn(0x7f6bf00cc7d0 legacy=0x7f6bf00ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:04.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6be7fff640 1 -- 172.21.15.105:0/3404028632 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf0161ef0 con 0x7f6bf006d610 2023-11-28T18:26:04.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6be7fff640 1 -- 172.21.15.105:0/3404028632 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f6bec001680 con 0x7f6bf006d610 2023-11-28T18:26:04.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6be7fff640 1 -- 172.21.15.105:0/3404028632 >> v1:172.21.15.105:6789/0 conn(0x7f6bf006d610 legacy=0x7f6bf00cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:04.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6be7fff640 1 --2- 172.21.15.105:0/3404028632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bdc0039b0 0x7f6bdc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:04.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6be7fff640 1 --2- 172.21.15.105:0/3404028632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bdc0047d0 0x7f6bdc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:04.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6be7fff640 1 --2- 172.21.15.105:0/3404028632 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bf00ad9f0 0x7f6bdc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:04.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6be7fff640 1 -- 172.21.15.105:0/3404028632 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf0161c10 con 0x7f6bdc0039b0 2023-11-28T18:26:04.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6be7fff640 1 -- 172.21.15.105:0/3404028632 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc00ba40 con 0x7f6bdc0047d0 2023-11-28T18:26:04.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6be7fff640 1 -- 172.21.15.105:0/3404028632 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc00bbd0 con 0x7f6bf00ad9f0 2023-11-28T18:26:04.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6bf57b3640 1 --2- 172.21.15.105:0/3404028632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bdc0039b0 0x7f6bdc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:04.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6bf4fb2640 1 --2- 172.21.15.105:0/3404028632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bdc0047d0 0x7f6bdc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:04.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6bf5fb4640 1 --2- 172.21.15.105:0/3404028632 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bf00ad9f0 0x7f6bdc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:04.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6bf57b3640 1 -- 172.21.15.105:0/3404028632 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bf00ad9f0 msgr2=0x7f6bdc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:04.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6bf57b3640 1 --2- 172.21.15.105:0/3404028632 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bf00ad9f0 0x7f6bdc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6bf57b3640 1 -- 172.21.15.105:0/3404028632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bdc0047d0 msgr2=0x7f6bdc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:04.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6bf57b3640 1 --2- 172.21.15.105:0/3404028632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bdc0047d0 0x7f6bdc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.704+0000 7f6bf57b3640 1 -- 172.21.15.105:0/3404028632 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bdc003530 con 0x7f6bdc0039b0 2023-11-28T18:26:04.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.708+0000 7f6bf57b3640 1 --2- 172.21.15.105:0/3404028632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bdc0039b0 0x7f6bdc003d90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6bec00c0e0 tx=0x7f6bec01caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ff5da47ebdff4e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:04.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.708+0000 7f6be7fff640 1 -- 172.21.15.105:0/3404028632 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec005ae0 con 0x7f6bdc0039b0 2023-11-28T18:26:04.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.708+0000 7f6be7fff640 1 -- 172.21.15.105:0/3404028632 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bec005c40 con 0x7f6bdc0039b0 2023-11-28T18:26:04.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.708+0000 7f6be7fff640 1 -- 172.21.15.105:0/3404028632 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec007070 con 0x7f6bdc0039b0 2023-11-28T18:26:04.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.708+0000 7f6bf7a17640 1 -- 172.21.15.105:0/3404028632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bdc0039b0 msgr2=0x7f6bdc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:04.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.708+0000 7f6bf7a17640 1 --2- 172.21.15.105:0/3404028632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bdc0039b0 0x7f6bdc003d90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6bec00c0e0 tx=0x7f6bec01caf0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.708+0000 7f6bf5fb4640 1 -- 172.21.15.105:0/3404028632 reap_dead start 2023-11-28T18:26:04.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.708+0000 7f6bf7a17640 1 -- 172.21.15.105:0/3404028632 shutdown_connections 2023-11-28T18:26:04.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.708+0000 7f6bf7a17640 1 -- 172.21.15.105:0/3404028632 >> 172.21.15.105:0/3404028632 conn(0x7f6bf00ceb90 msgr2=0x7f6bf00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:04.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.708+0000 7f6bf7a17640 1 -- 172.21.15.105:0/3404028632 shutdown_connections 2023-11-28T18:26:04.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.708+0000 7f6bf7a17640 1 -- 172.21.15.105:0/3404028632 wait complete. 2023-11-28T18:26:04.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.708+0000 7f6bf7a17640 1 Processor -- start 2023-11-28T18:26:04.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.708+0000 7f6bf7a17640 1 -- start start 2023-11-28T18:26:04.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6bf7a17640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bdc0039b0 0x7f6bf015b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:04.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6bf7a17640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bf015b910 0x7f6bf00b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:04.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6bf7a17640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bf015be60 0x7f6bf00b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:04.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6bf7a17640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf016b2a0 con 0x7f6bf015be60 2023-11-28T18:26:04.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6bf7a17640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf015b720 con 0x7f6bdc0039b0 2023-11-28T18:26:04.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6bf7a17640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf016a970 con 0x7f6bf015b910 2023-11-28T18:26:04.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6bf5fb4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bf015be60 0x7f6bf00b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:04.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6bf57b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bdc0039b0 0x7f6bf015b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:04.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6bf5fb4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bf015be60 0x7f6bf00b25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45922/0 (socket says 172.21.15.105:45922) 2023-11-28T18:26:04.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6bf5fb4640 1 -- 172.21.15.105:0/1524383251 learned_addr learned my addr 172.21.15.105:0/1524383251 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:04.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6bf4fb2640 1 --2- 172.21.15.105:0/1524383251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bf015b910 0x7f6bf00b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:04.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6bf57b3640 1 -- 172.21.15.105:0/1524383251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bf015b910 msgr2=0x7f6bf00b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:04.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6bf57b3640 1 --2- 172.21.15.105:0/1524383251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bf015b910 0x7f6bf00b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6bf57b3640 1 -- 172.21.15.105:0/1524383251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bf015be60 msgr2=0x7f6bf00b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:04.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6bf57b3640 1 --2- 172.21.15.105:0/1524383251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bf015be60 0x7f6bf00b25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6bf57b3640 1 -- 172.21.15.105:0/1524383251 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf016ac50 con 0x7f6bdc0039b0 2023-11-28T18:26:04.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6bf5fb4640 1 --2- 172.21.15.105:0/1524383251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bf015be60 0x7f6bf00b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:26:04.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6bf57b3640 1 --2- 172.21.15.105:0/1524383251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bdc0039b0 0x7f6bf015b1e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6bec009000 tx=0x7f6bdc005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:04.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6be5ffb640 1 -- 172.21.15.105:0/1524383251 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec0b4070 con 0x7f6bdc0039b0 2023-11-28T18:26:04.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6be5ffb640 1 -- 172.21.15.105:0/1524383251 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bec0013c0 con 0x7f6bdc0039b0 2023-11-28T18:26:04.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6bf7a17640 1 -- 172.21.15.105:0/1524383251 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf016af30 con 0x7f6bdc0039b0 2023-11-28T18:26:04.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6be5ffb640 1 -- 172.21.15.105:0/1524383251 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec0070a0 con 0x7f6bdc0039b0 2023-11-28T18:26:04.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6bf7a17640 1 -- 172.21.15.105:0/1524383251 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf00b2b20 con 0x7f6bdc0039b0 2023-11-28T18:26:04.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.712+0000 7f6bf7a17640 1 -- 172.21.15.105:0/1524383251 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bf00b2d10 con 0x7f6bdc0039b0 2023-11-28T18:26:04.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.716+0000 7f6be5ffb640 1 -- 172.21.15.105:0/1524383251 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f6bec007200 con 0x7f6bdc0039b0 2023-11-28T18:26:04.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.716+0000 7f6be5ffb640 1 --2- 172.21.15.105:0/1524383251 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6bd4066900 0x7f6bd4068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:04.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.716+0000 7f6be5ffb640 1 -- 172.21.15.105:0/1524383251 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6bec11e070 con 0x7f6bdc0039b0 2023-11-28T18:26:04.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.716+0000 7f6bf4fb2640 1 --2- 172.21.15.105:0/1524383251 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6bd4066900 0x7f6bd4068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:04.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.716+0000 7f6bf4fb2640 1 --2- 172.21.15.105:0/1524383251 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6bd4066900 0x7f6bd4068dc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6bf01fe000 tx=0x7f6be0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:04.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.720+0000 7f6be5ffb640 1 -- 172.21.15.105:0/1524383251 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6bec0eb0d0 con 0x7f6bdc0039b0 2023-11-28T18:26:04.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.868+0000 7f6bf7a17640 1 -- 172.21.15.105:0/1524383251 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6bf0067570 con 0x7f6bdc0039b0 2023-11-28T18:26:04.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.872+0000 7f6be5ffb640 1 -- 172.21.15.105:0/1524383251 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16834 (secure 0 0 0) 0x7f6bec0074b0 con 0x7f6bdc0039b0 2023-11-28T18:26:04.874 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:04.874 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":51,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:26:02.163763+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:25:58.881270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T18:25:59.532434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:26:04.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.872+0000 7f6bf7a17640 1 -- 172.21.15.105:0/1524383251 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6bd4066900 msgr2=0x7f6bd4068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:04.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.872+0000 7f6bf7a17640 1 --2- 172.21.15.105:0/1524383251 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6bd4066900 0x7f6bd4068dc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6bf01fe000 tx=0x7f6be0009190 comp rx=0 tx=0).stop 2023-11-28T18:26:04.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.872+0000 7f6bf7a17640 1 -- 172.21.15.105:0/1524383251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bdc0039b0 msgr2=0x7f6bf015b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:04.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.872+0000 7f6bf7a17640 1 --2- 172.21.15.105:0/1524383251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bdc0039b0 0x7f6bf015b1e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6bec009000 tx=0x7f6bdc005800 comp rx=0 tx=0).stop 2023-11-28T18:26:04.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.872+0000 7f6bf7a17640 1 -- 172.21.15.105:0/1524383251 shutdown_connections 2023-11-28T18:26:04.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.872+0000 7f6bf7a17640 1 --2- 172.21.15.105:0/1524383251 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6bd4066900 0x7f6bd4068dc0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.872+0000 7f6bf7a17640 1 --2- 172.21.15.105:0/1524383251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bf015be60 0x7f6bf00b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.872+0000 7f6bf7a17640 1 --2- 172.21.15.105:0/1524383251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bf015b910 0x7f6bf00b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.872+0000 7f6bf7a17640 1 --2- 172.21.15.105:0/1524383251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bdc0039b0 0x7f6bf015b1e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:04.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.872+0000 7f6bf7a17640 1 -- 172.21.15.105:0/1524383251 >> 172.21.15.105:0/1524383251 conn(0x7f6bf00ceb90 msgr2=0x7f6bf0160c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:04.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.872+0000 7f6bf7a17640 1 -- 172.21.15.105:0/1524383251 shutdown_connections 2023-11-28T18:26:04.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:04.872+0000 7f6bf7a17640 1 -- 172.21.15.105:0/1524383251 wait complete. 2023-11-28T18:26:04.895 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T18:26:04.895 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:26:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.028+0000 7ff67149a640 1 Processor -- start 2023-11-28T18:26:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.032+0000 7ff67149a640 1 -- start start 2023-11-28T18:26:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.032+0000 7ff67149a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff66c159f40 0x7ff66c0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.032+0000 7ff67149a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff66c0cc760 0x7ff66c0ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.032+0000 7ff67149a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff66c06bf50 con 0x7ff66c0ad680 2023-11-28T18:26:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.032+0000 7ff67149a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff66c0ccbc0 con 0x7ff66c159f40 2023-11-28T18:26:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.032+0000 7ff67149a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c15a320 con 0x7ff66c0cc760 2023-11-28T18:26:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.032+0000 7ff66affd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff66c159f40 0x7ff66c0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.032+0000 7ff66a7fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff66c0cc760 0x7ff66c0ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.032+0000 7ff66a7fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff66c0cc760 0x7ff66c0ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:38616/0 (socket says 172.21.15.105:38616) 2023-11-28T18:26:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.032+0000 7ff66a7fc640 1 -- 172.21.15.105:0/1480310437 learned_addr learned my addr 172.21.15.105:0/1480310437 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.032+0000 7ff66affd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff66c159f40 0x7ff66c0caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:44250/0 (socket says 172.21.15.105:44250) 2023-11-28T18:26:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.032+0000 7ff66b7fe640 1 --1- 172.21.15.105:0/1480310437 >> v1:172.21.15.148:6789/0 conn(0x7ff66c0ad680 0x7ff66c15da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:33348/0 (socket says 172.21.15.105:33348) 2023-11-28T18:26:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.032+0000 7ff66a7fc640 1 -- 172.21.15.105:0/1480310437 >> v2:172.21.15.148:3300/0 conn(0x7ff66c159f40 msgr2=0x7ff66c0caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.032+0000 7ff66a7fc640 1 --2- 172.21.15.105:0/1480310437 >> v2:172.21.15.148:3300/0 conn(0x7ff66c159f40 0x7ff66c0caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.032+0000 7ff66a7fc640 1 -- 172.21.15.105:0/1480310437 >> v1:172.21.15.148:6789/0 conn(0x7ff66c0ad680 legacy=0x7ff66c15da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.032+0000 7ff66a7fc640 1 -- 172.21.15.105:0/1480310437 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66c071380 con 0x7ff66c0cc760 2023-11-28T18:26:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.036+0000 7ff66a7fc640 1 --2- 172.21.15.105:0/1480310437 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff66c0cc760 0x7ff66c0ad140 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff660009870 tx=0x7ff66001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de517bd475eefd75 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.036+0000 7ff669ffb640 1 -- 172.21.15.105:0/1480310437 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6600ab680 con 0x7ff66c0cc760 2023-11-28T18:26:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.036+0000 7ff669ffb640 1 -- 172.21.15.105:0/1480310437 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6600ab7e0 con 0x7ff66c0cc760 2023-11-28T18:26:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.036+0000 7ff669ffb640 1 -- 172.21.15.105:0/1480310437 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6600abb70 con 0x7ff66c0cc760 2023-11-28T18:26:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.036+0000 7ff67149a640 1 -- 172.21.15.105:0/1480310437 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff66c0cc760 msgr2=0x7ff66c0ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.036+0000 7ff67149a640 1 --2- 172.21.15.105:0/1480310437 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff66c0cc760 0x7ff66c0ad140 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff660009870 tx=0x7ff66001c480 comp rx=0 tx=0).stop 2023-11-28T18:26:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.036+0000 7ff67149a640 1 -- 172.21.15.105:0/1480310437 shutdown_connections 2023-11-28T18:26:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.036+0000 7ff67149a640 1 --2- 172.21.15.105:0/1480310437 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff66c0cc760 0x7ff66c0ad140 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.036+0000 7ff67149a640 1 --2- 172.21.15.105:0/1480310437 >> v2:172.21.15.148:3300/0 conn(0x7ff66c159f40 0x7ff66c0caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.036+0000 7ff67149a640 1 -- 172.21.15.105:0/1480310437 >> 172.21.15.105:0/1480310437 conn(0x7ff66c0ceb90 msgr2=0x7ff66c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:05.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.036+0000 7ff67149a640 1 -- 172.21.15.105:0/1480310437 shutdown_connections 2023-11-28T18:26:05.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.036+0000 7ff67149a640 1 -- 172.21.15.105:0/1480310437 wait complete. 2023-11-28T18:26:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff67149a640 1 Processor -- start 2023-11-28T18:26:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff67149a640 1 -- start start 2023-11-28T18:26:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff67149a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff66c0ad680 0x7ff66c1519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff67149a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff66c0cc760 0x7ff66c166940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff67149a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff66c159f40 0x7ff66c14f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff67149a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c1522f0 con 0x7ff66c0cc760 2023-11-28T18:26:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff67149a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c1525d0 con 0x7ff66c159f40 2023-11-28T18:26:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff67149a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c1528b0 con 0x7ff66c0ad680 2023-11-28T18:26:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff66a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff66c0cc760 0x7ff66c166940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff66affd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff66c0ad680 0x7ff66c1519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff66a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff66c0cc760 0x7ff66c166940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45938/0 (socket says 172.21.15.105:45938) 2023-11-28T18:26:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff66a7fc640 1 -- 172.21.15.105:0/781631657 learned_addr learned my addr 172.21.15.105:0/781631657 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff66b7fe640 1 --2- 172.21.15.105:0/781631657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff66c159f40 0x7ff66c14f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff66a7fc640 1 -- 172.21.15.105:0/781631657 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff66c0ad680 msgr2=0x7ff66c1519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff66a7fc640 1 --2- 172.21.15.105:0/781631657 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff66c0ad680 0x7ff66c1519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff66a7fc640 1 -- 172.21.15.105:0/781631657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff66c159f40 msgr2=0x7ff66c14f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff66a7fc640 1 --2- 172.21.15.105:0/781631657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff66c159f40 0x7ff66c14f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff66a7fc640 1 -- 172.21.15.105:0/781631657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6600094d0 con 0x7ff66c0cc760 2023-11-28T18:26:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff66affd640 1 --2- 172.21.15.105:0/781631657 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff66c0ad680 0x7ff66c1519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:26:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff66a7fc640 1 --2- 172.21.15.105:0/781631657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff66c0cc760 0x7ff66c166940 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff66001a0b0 tx=0x7ff66001acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff6537fe640 1 -- 172.21.15.105:0/781631657 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6600b4800 con 0x7ff66c0cc760 2023-11-28T18:26:05.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff6537fe640 1 -- 172.21.15.105:0/781631657 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6600b4980 con 0x7ff66c0cc760 2023-11-28T18:26:05.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff6537fe640 1 -- 172.21.15.105:0/781631657 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6600b4d30 con 0x7ff66c0cc760 2023-11-28T18:26:05.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff67149a640 1 -- 172.21.15.105:0/781631657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66c152b90 con 0x7ff66c0cc760 2023-11-28T18:26:05.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff67149a640 1 -- 172.21.15.105:0/781631657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff66c151f30 con 0x7ff66c0cc760 2023-11-28T18:26:05.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.040+0000 7ff67149a640 1 -- 172.21.15.105:0/781631657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff66c15a700 con 0x7ff66c0cc760 2023-11-28T18:26:05.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.044+0000 7ff6537fe640 1 -- 172.21.15.105:0/781631657 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff6600b3380 con 0x7ff66c0cc760 2023-11-28T18:26:05.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.044+0000 7ff6537fe640 1 --2- 172.21.15.105:0/781631657 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6580668d0 0x7ff658068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:05.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.044+0000 7ff6537fe640 1 -- 172.21.15.105:0/781631657 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff66011f030 con 0x7ff66c0cc760 2023-11-28T18:26:05.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.044+0000 7ff66affd640 1 --2- 172.21.15.105:0/781631657 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6580668d0 0x7ff658068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:05.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.044+0000 7ff66affd640 1 --2- 172.21.15.105:0/781631657 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6580668d0 0x7ff658068d90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009eb0 tx=0x7ff65c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:05.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.048+0000 7ff6537fe640 1 -- 172.21.15.105:0/781631657 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff66c15a700 con 0x7ff66c0cc760 2023-11-28T18:26:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.196+0000 7ff67149a640 1 -- 172.21.15.105:0/781631657 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff66c0cf730 con 0x7ff6580668d0 2023-11-28T18:26:05.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.204+0000 7ff6537fe640 1 -- 172.21.15.105:0/781631657 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289156 (secure 0 0 0) 0x7ff66c0cf730 con 0x7ff6580668d0 2023-11-28T18:26:05.210 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:05.212 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:26:05.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.208+0000 7ff67149a640 1 -- 172.21.15.105:0/781631657 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6580668d0 msgr2=0x7ff658068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:05.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.208+0000 7ff67149a640 1 --2- 172.21.15.105:0/781631657 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6580668d0 0x7ff658068d90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009eb0 tx=0x7ff65c009190 comp rx=0 tx=0).stop 2023-11-28T18:26:05.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.208+0000 7ff67149a640 1 -- 172.21.15.105:0/781631657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff66c0cc760 msgr2=0x7ff66c166940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:05.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.208+0000 7ff67149a640 1 --2- 172.21.15.105:0/781631657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff66c0cc760 0x7ff66c166940 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff66001a0b0 tx=0x7ff66001acc0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.208+0000 7ff67149a640 1 -- 172.21.15.105:0/781631657 shutdown_connections 2023-11-28T18:26:05.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.208+0000 7ff67149a640 1 --2- 172.21.15.105:0/781631657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff66c159f40 0x7ff66c14f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.208+0000 7ff67149a640 1 --2- 172.21.15.105:0/781631657 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6580668d0 0x7ff658068d90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.208+0000 7ff67149a640 1 --2- 172.21.15.105:0/781631657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff66c0cc760 0x7ff66c166940 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.208+0000 7ff67149a640 1 --2- 172.21.15.105:0/781631657 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff66c0ad680 0x7ff66c1519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.208+0000 7ff67149a640 1 -- 172.21.15.105:0/781631657 >> 172.21.15.105:0/781631657 conn(0x7ff66c0ceb90 msgr2=0x7ff66c072eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:05.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.208+0000 7ff67149a640 1 -- 172.21.15.105:0/781631657 shutdown_connections 2023-11-28T18:26:05.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.208+0000 7ff67149a640 1 -- 172.21.15.105:0/781631657 wait complete. 2023-11-28T18:26:05.230 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2023-11-28T18:26:03.300768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":68,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255192,"kb_used_data":5716,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754719528,"statfs":{"total":773094113280,"available":772832796672,"internally_reserved":0,"allocated":5853184,"data_stored":3231423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.151586"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.186966+0000","last_change":"2023-11-28T18:26:01.181834+0000","last_active":"2023-11-28T18:26:01.186966+0000","last_peered":"2023-11-28T18:26:01.186966+0000","last_clean":"2023-11-28T18:26:01.186966+0000","last_became_active":"2023-11-28T18:26:01.181524+0000","last_became_peered":"2023-11-28T18:26:01.181524+0000","last_unstale":"2023-11-28T18:26:01.186966+0000","last_undegraded":"2023-11-28T18:26:01.186966+0000","last_fullsized":"2023-11-28T18:26:01.186966+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:19:20.407923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.181983+0000","last_change":"2023-11-28T18:26:01.178324+0000","last_active":"2023-11-28T18:26:01.181983+0000","last_peered":"2023-11-28T18:26:01.181983+0000","last_clean":"2023-11-28T18:26:01.181983+0000","last_became_active":"2023-11-28T18:26:01.178100+0000","last_became_peered":"2023-11-28T18:26:01.178100+0000","last_unstale":"2023-11-28T18:26:01.181983+0000","last_undegraded":"2023-11-28T18:26:01.181983+0000","last_fullsized":"2023-11-28T18:26:01.181983+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:53:25.488049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.188388+0000","last_change":"2023-11-28T18:26:01.179213+0000","last_active":"2023-11-28T18:26:01.188388+0000","last_peered":"2023-11-28T18:26:01.188388+0000","last_clean":"2023-11-28T18:26:01.188388+0000","last_became_active":"2023-11-28T18:26:01.178919+0000","last_became_peered":"2023-11-28T18:26:01.178919+0000","last_unstale":"2023-11-28T18:26:01.188388+0000","last_undegraded":"2023-11-28T18:26:01.188388+0000","last_fullsized":"2023-11-28T18:26:01.188388+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:54:35.486618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.195289+0000","last_change":"2023-11-28T18:26:01.191139+0000","last_active":"2023-11-28T18:26:01.195289+0000","last_peered":"2023-11-28T18:26:01.195289+0000","last_clean":"2023-11-28T18:26:01.195289+0000","last_became_active":"2023-11-28T18:26:01.190752+0000","last_became_peered":"2023-11-28T18:26:01.190752+0000","last_unstale":"2023-11-28T18:26:01.195289+0000","last_undegraded":"2023-11-28T18:26:01.195289+0000","last_fullsized":"2023-11-28T18:26:01.195289+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:36.399960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.196685+0000","last_change":"2023-11-28T18:26:01.186179+0000","last_active":"2023-11-28T18:26:01.196685+0000","last_peered":"2023-11-28T18:26:01.196685+0000","last_clean":"2023-11-28T18:26:01.196685+0000","last_became_active":"2023-11-28T18:26:01.184960+0000","last_became_peered":"2023-11-28T18:26:01.184960+0000","last_unstale":"2023-11-28T18:26:01.196685+0000","last_undegraded":"2023-11-28T18:26:01.196685+0000","last_fullsized":"2023-11-28T18:26:01.196685+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:00:43.594153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.189253+0000","last_change":"2023-11-28T18:26:01.184813+0000","last_active":"2023-11-28T18:26:01.189253+0000","last_peered":"2023-11-28T18:26:01.189253+0000","last_clean":"2023-11-28T18:26:01.189253+0000","last_became_active":"2023-11-28T18:26:01.184541+0000","last_became_peered":"2023-11-28T18:26:01.184541+0000","last_unstale":"2023-11-28T18:26:01.189253+0000","last_undegraded":"2023-11-28T18:26:01.189253+0000","last_fullsized":"2023-11-28T18:26:01.189253+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:41:06.893526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.187938+0000","last_change":"2023-11-28T18:26:01.180760+0000","last_active":"2023-11-28T18:26:01.187938+0000","last_peered":"2023-11-28T18:26:01.187938+0000","last_clean":"2023-11-28T18:26:01.187938+0000","last_became_active":"2023-11-28T18:26:01.180414+0000","last_became_peered":"2023-11-28T18:26:01.180414+0000","last_unstale":"2023-11-28T18:26:01.187938+0000","last_undegraded":"2023-11-28T18:26:01.187938+0000","last_fullsized":"2023-11-28T18:26:01.187938+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:22:53.430433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.186211+0000","last_change":"2023-11-28T18:26:01.175611+0000","last_active":"2023-11-28T18:26:01.186211+0000","last_peered":"2023-11-28T18:26:01.186211+0000","last_clean":"2023-11-28T18:26:01.186211+0000","last_became_active":"2023-11-28T18:26:01.175354+0000","last_became_peered":"2023-11-28T18:26:01.175354+0000","last_unstale":"2023-11-28T18:26:01.186211+0000","last_undegraded":"2023-11-28T18:26:01.186211+0000","last_fullsized":"2023-11-28T18:26:01.186211+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:21:01.828174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.185705+0000","last_change":"2023-11-28T18:26:01.172741+0000","last_active":"2023-11-28T18:26:01.185705+0000","last_peered":"2023-11-28T18:26:01.185705+0000","last_clean":"2023-11-28T18:26:01.185705+0000","last_became_active":"2023-11-28T18:26:01.172467+0000","last_became_peered":"2023-11-28T18:26:01.172467+0000","last_unstale":"2023-11-28T18:26:01.185705+0000","last_undegraded":"2023-11-28T18:26:01.185705+0000","last_fullsized":"2023-11-28T18:26:01.185705+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:17:35.391422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.194596+0000","last_change":"2023-11-28T18:26:01.189717+0000","last_active":"2023-11-28T18:26:01.194596+0000","last_peered":"2023-11-28T18:26:01.194596+0000","last_clean":"2023-11-28T18:26:01.194596+0000","last_became_active":"2023-11-28T18:26:01.189446+0000","last_became_peered":"2023-11-28T18:26:01.189446+0000","last_unstale":"2023-11-28T18:26:01.194596+0000","last_undegraded":"2023-11-28T18:26:01.194596+0000","last_fullsized":"2023-11-28T18:26:01.194596+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:39:28.226366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.191823+0000","last_change":"2023-11-28T18:26:01.183299+0000","last_active":"2023-11-28T18:26:01.191823+0000","last_peered":"2023-11-28T18:26:01.191823+0000","last_clean":"2023-11-28T18:26:01.191823+0000","last_became_active":"2023-11-28T18:26:01.182848+0000","last_became_peered":"2023-11-28T18:26:01.182848+0000","last_unstale":"2023-11-28T18:26:01.191823+0000","last_undegraded":"2023-11-28T18:26:01.191823+0000","last_fullsized":"2023-11-28T18:26:01.191823+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:34:02.510862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.201988+0000","last_change":"2023-11-28T18:26:01.197025+0000","last_active":"2023-11-28T18:26:01.201988+0000","last_peered":"2023-11-28T18:26:01.201988+0000","last_clean":"2023-11-28T18:26:01.201988+0000","last_became_active":"2023-11-28T18:26:01.196745+0000","last_became_peered":"2023-11-28T18:26:01.196745+0000","last_unstale":"2023-11-28T18:26:01.201988+0000","last_undegraded":"2023-11-28T18:26:01.201988+0000","last_fullsized":"2023-11-28T18:26:01.201988+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:18:01.943125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.194680+0000","last_change":"2023-11-28T18:26:01.190290+0000","last_active":"2023-11-28T18:26:01.194680+0000","last_peered":"2023-11-28T18:26:01.194680+0000","last_clean":"2023-11-28T18:26:01.194680+0000","last_became_active":"2023-11-28T18:26:01.189701+0000","last_became_peered":"2023-11-28T18:26:01.189701+0000","last_unstale":"2023-11-28T18:26:01.194680+0000","last_undegraded":"2023-11-28T18:26:01.194680+0000","last_fullsized":"2023-11-28T18:26:01.194680+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:49.681252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.189783+0000","last_change":"2023-11-28T18:26:01.182292+0000","last_active":"2023-11-28T18:26:01.189783+0000","last_peered":"2023-11-28T18:26:01.189783+0000","last_clean":"2023-11-28T18:26:01.189783+0000","last_became_active":"2023-11-28T18:26:01.181852+0000","last_became_peered":"2023-11-28T18:26:01.181852+0000","last_unstale":"2023-11-28T18:26:01.189783+0000","last_undegraded":"2023-11-28T18:26:01.189783+0000","last_fullsized":"2023-11-28T18:26:01.189783+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:18:33.835504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.199823+0000","last_change":"2023-11-28T18:26:01.195150+0000","last_active":"2023-11-28T18:26:01.199823+0000","last_peered":"2023-11-28T18:26:01.199823+0000","last_clean":"2023-11-28T18:26:01.199823+0000","last_became_active":"2023-11-28T18:26:01.194716+0000","last_became_peered":"2023-11-28T18:26:01.194716+0000","last_unstale":"2023-11-28T18:26:01.199823+0000","last_undegraded":"2023-11-28T18:26:01.199823+0000","last_fullsized":"2023-11-28T18:26:01.199823+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:18:33.835504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.192221+0000","last_change":"2023-11-28T18:26:01.185312+0000","last_active":"2023-11-28T18:26:01.192221+0000","last_peered":"2023-11-28T18:26:01.192221+0000","last_clean":"2023-11-28T18:26:01.192221+0000","last_became_active":"2023-11-28T18:26:01.185070+0000","last_became_peered":"2023-11-28T18:26:01.185070+0000","last_unstale":"2023-11-28T18:26:01.192221+0000","last_undegraded":"2023-11-28T18:26:01.192221+0000","last_fullsized":"2023-11-28T18:26:01.192221+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:29:19.810995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.175619+0000","last_change":"2023-11-28T18:26:01.171993+0000","last_active":"2023-11-28T18:26:01.175619+0000","last_peered":"2023-11-28T18:26:01.175619+0000","last_clean":"2023-11-28T18:26:01.175619+0000","last_became_active":"2023-11-28T18:26:01.171661+0000","last_became_peered":"2023-11-28T18:26:01.171661+0000","last_unstale":"2023-11-28T18:26:01.175619+0000","last_undegraded":"2023-11-28T18:26:01.175619+0000","last_fullsized":"2023-11-28T18:26:01.175619+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:06.029532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.193503+0000","last_change":"2023-11-28T18:26:01.187351+0000","last_active":"2023-11-28T18:26:01.193503+0000","last_peered":"2023-11-28T18:26:01.193503+0000","last_clean":"2023-11-28T18:26:01.193503+0000","last_became_active":"2023-11-28T18:26:01.187033+0000","last_became_peered":"2023-11-28T18:26:01.187033+0000","last_unstale":"2023-11-28T18:26:01.193503+0000","last_undegraded":"2023-11-28T18:26:01.193503+0000","last_fullsized":"2023-11-28T18:26:01.193503+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:06.029532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.189812+0000","last_change":"2023-11-28T18:26:01.188488+0000","last_active":"2023-11-28T18:26:01.189812+0000","last_peered":"2023-11-28T18:26:01.189812+0000","last_clean":"2023-11-28T18:26:01.189812+0000","last_became_active":"2023-11-28T18:26:01.188162+0000","last_became_peered":"2023-11-28T18:26:01.188162+0000","last_unstale":"2023-11-28T18:26:01.189812+0000","last_undegraded":"2023-11-28T18:26:01.189812+0000","last_fullsized":"2023-11-28T18:26:01.189812+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:26:09.798191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.202468+0000","last_change":"2023-11-28T18:26:01.196782+0000","last_active":"2023-11-28T18:26:01.202468+0000","last_peered":"2023-11-28T18:26:01.202468+0000","last_clean":"2023-11-28T18:26:01.202468+0000","last_became_active":"2023-11-28T18:26:01.196278+0000","last_became_peered":"2023-11-28T18:26:01.196278+0000","last_unstale":"2023-11-28T18:26:01.202468+0000","last_undegraded":"2023-11-28T18:26:01.202468+0000","last_fullsized":"2023-11-28T18:26:01.202468+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:04:46.200690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.202502+0000","last_change":"2023-11-28T18:26:01.197765+0000","last_active":"2023-11-28T18:26:01.202502+0000","last_peered":"2023-11-28T18:26:01.202502+0000","last_clean":"2023-11-28T18:26:01.202502+0000","last_became_active":"2023-11-28T18:26:01.197470+0000","last_became_peered":"2023-11-28T18:26:01.197470+0000","last_unstale":"2023-11-28T18:26:01.202502+0000","last_undegraded":"2023-11-28T18:26:01.202502+0000","last_fullsized":"2023-11-28T18:26:01.202502+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:52:54.081912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.185551+0000","last_change":"2023-11-28T18:26:01.173587+0000","last_active":"2023-11-28T18:26:01.185551+0000","last_peered":"2023-11-28T18:26:01.185551+0000","last_clean":"2023-11-28T18:26:01.185551+0000","last_became_active":"2023-11-28T18:26:01.173280+0000","last_became_peered":"2023-11-28T18:26:01.173280+0000","last_unstale":"2023-11-28T18:26:01.185551+0000","last_undegraded":"2023-11-28T18:26:01.185551+0000","last_fullsized":"2023-11-28T18:26:01.185551+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:54:35.486618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.191081+0000","last_change":"2023-11-28T18:26:01.182286+0000","last_active":"2023-11-28T18:26:01.191081+0000","last_peered":"2023-11-28T18:26:01.191081+0000","last_clean":"2023-11-28T18:26:01.191081+0000","last_became_active":"2023-11-28T18:26:01.182037+0000","last_became_peered":"2023-11-28T18:26:01.182037+0000","last_unstale":"2023-11-28T18:26:01.191081+0000","last_undegraded":"2023-11-28T18:26:01.191081+0000","last_fullsized":"2023-11-28T18:26:01.191081+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:18:08.430874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.186377+0000","last_change":"2023-11-28T18:26:01.175268+0000","last_active":"2023-11-28T18:26:01.186377+0000","last_peered":"2023-11-28T18:26:01.186377+0000","last_clean":"2023-11-28T18:26:01.186377+0000","last_became_active":"2023-11-28T18:26:01.175054+0000","last_became_peered":"2023-11-28T18:26:01.175054+0000","last_unstale":"2023-11-28T18:26:01.186377+0000","last_undegraded":"2023-11-28T18:26:01.186377+0000","last_fullsized":"2023-11-28T18:26:01.186377+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:57.022136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.198824+0000","last_change":"2023-11-28T18:26:01.191960+0000","last_active":"2023-11-28T18:26:01.198824+0000","last_peered":"2023-11-28T18:26:01.198824+0000","last_clean":"2023-11-28T18:26:01.198824+0000","last_became_active":"2023-11-28T18:26:01.190212+0000","last_became_peered":"2023-11-28T18:26:01.190212+0000","last_unstale":"2023-11-28T18:26:01.198824+0000","last_undegraded":"2023-11-28T18:26:01.198824+0000","last_fullsized":"2023-11-28T18:26:01.198824+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:21:01.828174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.200696+0000","last_change":"2023-11-28T18:26:01.195584+0000","last_active":"2023-11-28T18:26:01.200696+0000","last_peered":"2023-11-28T18:26:01.200696+0000","last_clean":"2023-11-28T18:26:01.200696+0000","last_became_active":"2023-11-28T18:26:01.195320+0000","last_became_peered":"2023-11-28T18:26:01.195320+0000","last_unstale":"2023-11-28T18:26:01.200696+0000","last_undegraded":"2023-11-28T18:26:01.200696+0000","last_fullsized":"2023-11-28T18:26:01.200696+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:47.994732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.186639+0000","last_change":"2023-11-28T18:26:01.176382+0000","last_active":"2023-11-28T18:26:01.186639+0000","last_peered":"2023-11-28T18:26:01.186639+0000","last_clean":"2023-11-28T18:26:01.186639+0000","last_became_active":"2023-11-28T18:26:01.175756+0000","last_became_peered":"2023-11-28T18:26:01.175756+0000","last_unstale":"2023-11-28T18:26:01.186639+0000","last_undegraded":"2023-11-28T18:26:01.186639+0000","last_fullsized":"2023-11-28T18:26:01.186639+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:36:35.769346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.202450+0000","last_change":"2023-11-28T18:26:01.196597+0000","last_active":"2023-11-28T18:26:01.202450+0000","last_peered":"2023-11-28T18:26:01.202450+0000","last_clean":"2023-11-28T18:26:01.202450+0000","last_became_active":"2023-11-28T18:26:01.195960+0000","last_became_peered":"2023-11-28T18:26:01.195960+0000","last_unstale":"2023-11-28T18:26:01.202450+0000","last_undegraded":"2023-11-28T18:26:01.202450+0000","last_fullsized":"2023-11-28T18:26:01.202450+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:41:21.591185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.197800+0000","last_change":"2023-11-28T18:26:01.191018+0000","last_active":"2023-11-28T18:26:01.197800+0000","last_peered":"2023-11-28T18:26:01.197800+0000","last_clean":"2023-11-28T18:26:01.197800+0000","last_became_active":"2023-11-28T18:26:01.190727+0000","last_became_peered":"2023-11-28T18:26:01.190727+0000","last_unstale":"2023-11-28T18:26:01.197800+0000","last_undegraded":"2023-11-28T18:26:01.197800+0000","last_fullsized":"2023-11-28T18:26:01.197800+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:26.019007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.193369+0000","last_change":"2023-11-28T18:26:01.186839+0000","last_active":"2023-11-28T18:26:01.193369+0000","last_peered":"2023-11-28T18:26:01.193369+0000","last_clean":"2023-11-28T18:26:01.193369+0000","last_became_active":"2023-11-28T18:26:01.185197+0000","last_became_peered":"2023-11-28T18:26:01.185197+0000","last_unstale":"2023-11-28T18:26:01.193369+0000","last_undegraded":"2023-11-28T18:26:01.193369+0000","last_fullsized":"2023-11-28T18:26:01.193369+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:57.022136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.187921+0000","last_change":"2023-11-28T18:26:01.180580+0000","last_active":"2023-11-28T18:26:01.187921+0000","last_peered":"2023-11-28T18:26:01.187921+0000","last_clean":"2023-11-28T18:26:01.187921+0000","last_became_active":"2023-11-28T18:26:01.180122+0000","last_became_peered":"2023-11-28T18:26:01.180122+0000","last_unstale":"2023-11-28T18:26:01.187921+0000","last_undegraded":"2023-11-28T18:26:01.187921+0000","last_fullsized":"2023-11-28T18:26:01.187921+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:49:47.454676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.175987+0000","last_change":"2023-11-28T18:26:01.164582+0000","last_active":"2023-11-28T18:26:01.175987+0000","last_peered":"2023-11-28T18:26:01.175987+0000","last_clean":"2023-11-28T18:26:01.175987+0000","last_became_active":"2023-11-28T18:26:01.164261+0000","last_became_peered":"2023-11-28T18:26:01.164261+0000","last_unstale":"2023-11-28T18:26:01.175987+0000","last_undegraded":"2023-11-28T18:26:01.175987+0000","last_fullsized":"2023-11-28T18:26:01.175987+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:18:33.835504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.194386+0000","last_change":"2023-11-28T18:26:01.188785+0000","last_active":"2023-11-28T18:26:01.194386+0000","last_peered":"2023-11-28T18:26:01.194386+0000","last_clean":"2023-11-28T18:26:01.194386+0000","last_became_active":"2023-11-28T18:26:01.188417+0000","last_became_peered":"2023-11-28T18:26:01.188417+0000","last_unstale":"2023-11-28T18:26:01.194386+0000","last_undegraded":"2023-11-28T18:26:01.194386+0000","last_fullsized":"2023-11-28T18:26:01.194386+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:21:01.828174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.202498+0000","last_change":"2023-11-28T18:26:00.181204+0000","last_active":"2023-11-28T18:26:01.202498+0000","last_peered":"2023-11-28T18:26:01.202498+0000","last_clean":"2023-11-28T18:26:01.202498+0000","last_became_active":"2023-11-28T18:26:00.180952+0000","last_became_peered":"2023-11-28T18:26:00.180952+0000","last_unstale":"2023-11-28T18:26:01.202498+0000","last_undegraded":"2023-11-28T18:26:01.202498+0000","last_fullsized":"2023-11-28T18:26:01.202498+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:07:33.583566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.182486+0000","last_change":"2023-11-28T18:26:01.178676+0000","last_active":"2023-11-28T18:26:01.182486+0000","last_peered":"2023-11-28T18:26:01.182486+0000","last_clean":"2023-11-28T18:26:01.182486+0000","last_became_active":"2023-11-28T18:26:01.178355+0000","last_became_peered":"2023-11-28T18:26:01.178355+0000","last_unstale":"2023-11-28T18:26:01.182486+0000","last_undegraded":"2023-11-28T18:26:01.182486+0000","last_fullsized":"2023-11-28T18:26:01.182486+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:26.019007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.193126+0000","last_change":"2023-11-28T18:26:00.193130+0000","last_active":"2023-11-28T18:26:01.193126+0000","last_peered":"2023-11-28T18:26:01.193126+0000","last_clean":"2023-11-28T18:26:01.193126+0000","last_became_active":"2023-11-28T18:26:00.191726+0000","last_became_peered":"2023-11-28T18:26:00.191726+0000","last_unstale":"2023-11-28T18:26:01.193126+0000","last_undegraded":"2023-11-28T18:26:01.193126+0000","last_fullsized":"2023-11-28T18:26:01.193126+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:17.654673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.202345+0000","last_change":"2023-11-28T18:26:01.195855+0000","last_active":"2023-11-28T18:26:01.202345+0000","last_peered":"2023-11-28T18:26:01.202345+0000","last_clean":"2023-11-28T18:26:01.202345+0000","last_became_active":"2023-11-28T18:26:01.195573+0000","last_became_peered":"2023-11-28T18:26:01.195573+0000","last_unstale":"2023-11-28T18:26:01.202345+0000","last_undegraded":"2023-11-28T18:26:01.202345+0000","last_fullsized":"2023-11-28T18:26:01.202345+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:45:27.560489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.194152+0000","last_change":"2023-11-28T18:26:00.184884+0000","last_active":"2023-11-28T18:26:01.194152+0000","last_peered":"2023-11-28T18:26:01.194152+0000","last_clean":"2023-11-28T18:26:01.194152+0000","last_became_active":"2023-11-28T18:26:00.184589+0000","last_became_peered":"2023-11-28T18:26:00.184589+0000","last_unstale":"2023-11-28T18:26:01.194152+0000","last_undegraded":"2023-11-28T18:26:01.194152+0000","last_fullsized":"2023-11-28T18:26:01.194152+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:00:42.591355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.185686+0000","last_change":"2023-11-28T18:26:01.182012+0000","last_active":"2023-11-28T18:26:01.185686+0000","last_peered":"2023-11-28T18:26:01.185686+0000","last_clean":"2023-11-28T18:26:01.185686+0000","last_became_active":"2023-11-28T18:26:01.178102+0000","last_became_peered":"2023-11-28T18:26:01.178102+0000","last_unstale":"2023-11-28T18:26:01.185686+0000","last_undegraded":"2023-11-28T18:26:01.185686+0000","last_fullsized":"2023-11-28T18:26:01.185686+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:57.022136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.198971+0000","last_change":"2023-11-28T18:26:00.183924+0000","last_active":"2023-11-28T18:26:01.198971+0000","last_peered":"2023-11-28T18:26:01.198971+0000","last_clean":"2023-11-28T18:26:01.198971+0000","last_became_active":"2023-11-28T18:26:00.183597+0000","last_became_peered":"2023-11-28T18:26:00.183597+0000","last_unstale":"2023-11-28T18:26:01.198971+0000","last_undegraded":"2023-11-28T18:26:01.198971+0000","last_fullsized":"2023-11-28T18:26:01.198971+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:53:24.485251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.188842+0000","last_change":"2023-11-28T18:26:01.177692+0000","last_active":"2023-11-28T18:26:01.188842+0000","last_peered":"2023-11-28T18:26:01.188842+0000","last_clean":"2023-11-28T18:26:01.188842+0000","last_became_active":"2023-11-28T18:26:01.175183+0000","last_became_peered":"2023-11-28T18:26:01.175183+0000","last_unstale":"2023-11-28T18:26:01.188842+0000","last_undegraded":"2023-11-28T18:26:01.188842+0000","last_fullsized":"2023-11-28T18:26:01.188842+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:18.657471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.195283+0000","last_change":"2023-11-28T18:26:00.180361+0000","last_active":"2023-11-28T18:26:01.195283+0000","last_peered":"2023-11-28T18:26:01.195283+0000","last_clean":"2023-11-28T18:26:01.195283+0000","last_became_active":"2023-11-28T18:26:00.180083+0000","last_became_peered":"2023-11-28T18:26:00.180083+0000","last_unstale":"2023-11-28T18:26:01.195283+0000","last_undegraded":"2023-11-28T18:26:01.195283+0000","last_fullsized":"2023-11-28T18:26:01.195283+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:17:34.388624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.194334+0000","last_change":"2023-11-28T18:26:01.190569+0000","last_active":"2023-11-28T18:26:01.194334+0000","last_peered":"2023-11-28T18:26:01.194334+0000","last_clean":"2023-11-28T18:26:01.194334+0000","last_became_active":"2023-11-28T18:26:01.189932+0000","last_became_peered":"2023-11-28T18:26:01.189932+0000","last_unstale":"2023-11-28T18:26:01.194334+0000","last_undegraded":"2023-11-28T18:26:01.194334+0000","last_fullsized":"2023-11-28T18:26:01.194334+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:12:59.466374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.195621+0000","last_change":"2023-11-28T18:26:00.179098+0000","last_active":"2023-11-28T18:26:01.195621+0000","last_peered":"2023-11-28T18:26:01.195621+0000","last_clean":"2023-11-28T18:26:01.195621+0000","last_became_active":"2023-11-28T18:26:00.178746+0000","last_became_peered":"2023-11-28T18:26:00.178746+0000","last_unstale":"2023-11-28T18:26:01.195621+0000","last_undegraded":"2023-11-28T18:26:01.195621+0000","last_fullsized":"2023-11-28T18:26:01.195621+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:36:34.766548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.200717+0000","last_change":"2023-11-28T18:26:01.195661+0000","last_active":"2023-11-28T18:26:01.200717+0000","last_peered":"2023-11-28T18:26:01.200717+0000","last_clean":"2023-11-28T18:26:01.200717+0000","last_became_active":"2023-11-28T18:26:01.195412+0000","last_became_peered":"2023-11-28T18:26:01.195412+0000","last_unstale":"2023-11-28T18:26:01.200717+0000","last_undegraded":"2023-11-28T18:26:01.200717+0000","last_fullsized":"2023-11-28T18:26:01.200717+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:16.495437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.188479+0000","last_change":"2023-11-28T18:26:00.180447+0000","last_active":"2023-11-28T18:26:01.188479+0000","last_peered":"2023-11-28T18:26:01.188479+0000","last_clean":"2023-11-28T18:26:01.188479+0000","last_became_active":"2023-11-28T18:26:00.180078+0000","last_became_peered":"2023-11-28T18:26:00.180078+0000","last_unstale":"2023-11-28T18:26:01.188479+0000","last_undegraded":"2023-11-28T18:26:01.188479+0000","last_fullsized":"2023-11-28T18:26:01.188479+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:21.581173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.180047+0000","last_change":"2023-11-28T18:26:01.176283+0000","last_active":"2023-11-28T18:26:01.180047+0000","last_peered":"2023-11-28T18:26:01.180047+0000","last_clean":"2023-11-28T18:26:01.180047+0000","last_became_active":"2023-11-28T18:26:01.176005+0000","last_became_peered":"2023-11-28T18:26:01.176005+0000","last_unstale":"2023-11-28T18:26:01.180047+0000","last_undegraded":"2023-11-28T18:26:01.180047+0000","last_fullsized":"2023-11-28T18:26:01.180047+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:49:47.454676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.166373+0000","last_change":"2023-11-28T18:26:00.179112+0000","last_active":"2023-11-28T18:26:01.166373+0000","last_peered":"2023-11-28T18:26:01.166373+0000","last_clean":"2023-11-28T18:26:01.166373+0000","last_became_active":"2023-11-28T18:26:00.178840+0000","last_became_peered":"2023-11-28T18:26:00.178840+0000","last_unstale":"2023-11-28T18:26:01.166373+0000","last_undegraded":"2023-11-28T18:26:01.166373+0000","last_fullsized":"2023-11-28T18:26:01.166373+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:15.492639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.183005+0000","last_change":"2023-11-28T18:26:01.164201+0000","last_active":"2023-11-28T18:26:01.183005+0000","last_peered":"2023-11-28T18:26:01.183005+0000","last_clean":"2023-11-28T18:26:01.183005+0000","last_became_active":"2023-11-28T18:26:01.163872+0000","last_became_peered":"2023-11-28T18:26:01.163872+0000","last_unstale":"2023-11-28T18:26:01.183005+0000","last_undegraded":"2023-11-28T18:26:01.183005+0000","last_fullsized":"2023-11-28T18:26:01.183005+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:39:28.226366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.166765+0000","last_change":"2023-11-28T18:26:00.185129+0000","last_active":"2023-11-28T18:26:01.166765+0000","last_peered":"2023-11-28T18:26:01.166765+0000","last_clean":"2023-11-28T18:26:01.166765+0000","last_became_active":"2023-11-28T18:26:00.184847+0000","last_became_peered":"2023-11-28T18:26:00.184847+0000","last_unstale":"2023-11-28T18:26:01.166765+0000","last_undegraded":"2023-11-28T18:26:01.166765+0000","last_fullsized":"2023-11-28T18:26:01.166765+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:04:01.913690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.189101+0000","last_change":"2023-11-28T18:26:01.183045+0000","last_active":"2023-11-28T18:26:01.189101+0000","last_peered":"2023-11-28T18:26:01.189101+0000","last_clean":"2023-11-28T18:26:01.189101+0000","last_became_active":"2023-11-28T18:26:01.182734+0000","last_became_peered":"2023-11-28T18:26:01.182734+0000","last_unstale":"2023-11-28T18:26:01.189101+0000","last_undegraded":"2023-11-28T18:26:01.189101+0000","last_fullsized":"2023-11-28T18:26:01.189101+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:06.029532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.161778+0000","last_change":"2023-11-28T18:26:00.179088+0000","last_active":"2023-11-28T18:26:01.161778+0000","last_peered":"2023-11-28T18:26:01.161778+0000","last_clean":"2023-11-28T18:26:01.161778+0000","last_became_active":"2023-11-28T18:26:00.177867+0000","last_became_peered":"2023-11-28T18:26:00.177867+0000","last_unstale":"2023-11-28T18:26:01.161778+0000","last_undegraded":"2023-11-28T18:26:01.161778+0000","last_fullsized":"2023-11-28T18:26:01.161778+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:58.106337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.198161+0000","last_change":"2023-11-28T18:26:01.191290+0000","last_active":"2023-11-28T18:26:01.198161+0000","last_peered":"2023-11-28T18:26:01.198161+0000","last_clean":"2023-11-28T18:26:01.198161+0000","last_became_active":"2023-11-28T18:26:01.190636+0000","last_became_peered":"2023-11-28T18:26:01.190636+0000","last_unstale":"2023-11-28T18:26:01.198161+0000","last_undegraded":"2023-11-28T18:26:01.198161+0000","last_fullsized":"2023-11-28T18:26:01.198161+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:12:59.466374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.190433+0000","last_change":"2023-11-28T18:26:00.194212+0000","last_active":"2023-11-28T18:26:01.190433+0000","last_peered":"2023-11-28T18:26:01.190433+0000","last_clean":"2023-11-28T18:26:01.190433+0000","last_became_active":"2023-11-28T18:26:00.193908+0000","last_became_peered":"2023-11-28T18:26:00.193908+0000","last_unstale":"2023-11-28T18:26:01.190433+0000","last_undegraded":"2023-11-28T18:26:01.190433+0000","last_fullsized":"2023-11-28T18:26:01.190433+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:10.195481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.188888+0000","last_change":"2023-11-28T18:26:01.177990+0000","last_active":"2023-11-28T18:26:01.188888+0000","last_peered":"2023-11-28T18:26:01.188888+0000","last_clean":"2023-11-28T18:26:01.188888+0000","last_became_active":"2023-11-28T18:26:01.176061+0000","last_became_peered":"2023-11-28T18:26:01.176061+0000","last_unstale":"2023-11-28T18:26:01.188888+0000","last_undegraded":"2023-11-28T18:26:01.188888+0000","last_fullsized":"2023-11-28T18:26:01.188888+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:22.583971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.204635+0000","last_change":"2023-11-28T18:26:00.187840+0000","last_active":"2023-11-28T18:26:01.204635+0000","last_peered":"2023-11-28T18:26:01.204635+0000","last_clean":"2023-11-28T18:26:01.204635+0000","last_became_active":"2023-11-28T18:26:00.187581+0000","last_became_peered":"2023-11-28T18:26:00.187581+0000","last_unstale":"2023-11-28T18:26:01.204635+0000","last_undegraded":"2023-11-28T18:26:01.204635+0000","last_fullsized":"2023-11-28T18:26:01.204635+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:32:55.830726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.184418+0000","last_change":"2023-11-28T18:26:01.179326+0000","last_active":"2023-11-28T18:26:01.184418+0000","last_peered":"2023-11-28T18:26:01.184418+0000","last_clean":"2023-11-28T18:26:01.184418+0000","last_became_active":"2023-11-28T18:26:01.179054+0000","last_became_peered":"2023-11-28T18:26:01.179054+0000","last_unstale":"2023-11-28T18:26:01.184418+0000","last_undegraded":"2023-11-28T18:26:01.184418+0000","last_fullsized":"2023-11-28T18:26:01.184418+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:21:01.828174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.192113+0000","last_change":"2023-11-28T18:26:00.177241+0000","last_active":"2023-11-28T18:26:01.192113+0000","last_peered":"2023-11-28T18:26:01.192113+0000","last_clean":"2023-11-28T18:26:01.192113+0000","last_became_active":"2023-11-28T18:26:00.176494+0000","last_became_peered":"2023-11-28T18:26:00.176494+0000","last_unstale":"2023-11-28T18:26:01.192113+0000","last_undegraded":"2023-11-28T18:26:01.192113+0000","last_fullsized":"2023-11-28T18:26:01.192113+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:32:55.830726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.188371+0000","last_change":"2023-11-28T18:26:01.176737+0000","last_active":"2023-11-28T18:26:01.188371+0000","last_peered":"2023-11-28T18:26:01.188371+0000","last_clean":"2023-11-28T18:26:01.188371+0000","last_became_active":"2023-11-28T18:26:01.176443+0000","last_became_peered":"2023-11-28T18:26:01.176443+0000","last_unstale":"2023-11-28T18:26:01.188371+0000","last_undegraded":"2023-11-28T18:26:01.188371+0000","last_fullsized":"2023-11-28T18:26:01.188371+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:36.399960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.189660+0000","last_change":"2023-11-28T18:26:00.181777+0000","last_active":"2023-11-28T18:26:01.189660+0000","last_peered":"2023-11-28T18:26:01.189660+0000","last_clean":"2023-11-28T18:26:01.189660+0000","last_became_active":"2023-11-28T18:26:00.181502+0000","last_became_peered":"2023-11-28T18:26:00.181502+0000","last_unstale":"2023-11-28T18:26:01.189660+0000","last_undegraded":"2023-11-28T18:26:01.189660+0000","last_fullsized":"2023-11-28T18:26:01.189660+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:36:34.766548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.189156+0000","last_change":"2023-11-28T18:26:01.185400+0000","last_active":"2023-11-28T18:26:01.189156+0000","last_peered":"2023-11-28T18:26:01.189156+0000","last_clean":"2023-11-28T18:26:01.189156+0000","last_became_active":"2023-11-28T18:26:01.181587+0000","last_became_peered":"2023-11-28T18:26:01.181587+0000","last_unstale":"2023-11-28T18:26:01.189156+0000","last_undegraded":"2023-11-28T18:26:01.189156+0000","last_fullsized":"2023-11-28T18:26:01.189156+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:47.363263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.168685+0000","last_change":"2023-11-28T18:26:00.187600+0000","last_active":"2023-11-28T18:26:01.168685+0000","last_peered":"2023-11-28T18:26:01.168685+0000","last_clean":"2023-11-28T18:26:01.168685+0000","last_became_active":"2023-11-28T18:26:00.187309+0000","last_became_peered":"2023-11-28T18:26:00.187309+0000","last_unstale":"2023-11-28T18:26:01.168685+0000","last_undegraded":"2023-11-28T18:26:01.168685+0000","last_fullsized":"2023-11-28T18:26:01.168685+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:21:17.168147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.182241+0000","last_change":"2023-11-28T18:26:01.172719+0000","last_active":"2023-11-28T18:26:01.182241+0000","last_peered":"2023-11-28T18:26:01.182241+0000","last_clean":"2023-11-28T18:26:01.182241+0000","last_became_active":"2023-11-28T18:26:01.172390+0000","last_became_peered":"2023-11-28T18:26:01.172390+0000","last_unstale":"2023-11-28T18:26:01.182241+0000","last_undegraded":"2023-11-28T18:26:01.182241+0000","last_fullsized":"2023-11-28T18:26:01.182241+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:21:18.170945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.196458+0000","last_change":"2023-11-28T18:26:00.174487+0000","last_active":"2023-11-28T18:26:01.196458+0000","last_peered":"2023-11-28T18:26:01.196458+0000","last_clean":"2023-11-28T18:26:01.196458+0000","last_became_active":"2023-11-28T18:26:00.170686+0000","last_became_peered":"2023-11-28T18:26:00.170686+0000","last_unstale":"2023-11-28T18:26:01.196458+0000","last_undegraded":"2023-11-28T18:26:01.196458+0000","last_fullsized":"2023-11-28T18:26:01.196458+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:05.026734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.174202+0000","last_change":"2023-11-28T18:26:01.166308+0000","last_active":"2023-11-28T18:26:01.174202+0000","last_peered":"2023-11-28T18:26:01.174202+0000","last_clean":"2023-11-28T18:26:01.174202+0000","last_became_active":"2023-11-28T18:26:01.165886+0000","last_became_peered":"2023-11-28T18:26:01.165886+0000","last_unstale":"2023-11-28T18:26:01.174202+0000","last_undegraded":"2023-11-28T18:26:01.174202+0000","last_fullsized":"2023-11-28T18:26:01.174202+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:18:33.835504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.190612+0000","last_change":"2023-11-28T18:26:00.180415+0000","last_active":"2023-11-28T18:26:01.190612+0000","last_peered":"2023-11-28T18:26:01.190612+0000","last_clean":"2023-11-28T18:26:01.190612+0000","last_became_active":"2023-11-28T18:26:00.180092+0000","last_became_peered":"2023-11-28T18:26:00.180092+0000","last_unstale":"2023-11-28T18:26:01.190612+0000","last_undegraded":"2023-11-28T18:26:01.190612+0000","last_fullsized":"2023-11-28T18:26:01.190612+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:06:35.665714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.187895+0000","last_change":"2023-11-28T18:26:01.177655+0000","last_active":"2023-11-28T18:26:01.187895+0000","last_peered":"2023-11-28T18:26:01.187895+0000","last_clean":"2023-11-28T18:26:01.187895+0000","last_became_active":"2023-11-28T18:26:01.177382+0000","last_became_peered":"2023-11-28T18:26:01.177382+0000","last_unstale":"2023-11-28T18:26:01.187895+0000","last_undegraded":"2023-11-28T18:26:01.187895+0000","last_fullsized":"2023-11-28T18:26:01.187895+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:07:34.586364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.190771+0000","last_change":"2023-11-28T18:26:00.176652+0000","last_active":"2023-11-28T18:26:01.190771+0000","last_peered":"2023-11-28T18:26:01.190771+0000","last_clean":"2023-11-28T18:26:01.190771+0000","last_became_active":"2023-11-28T18:26:00.176321+0000","last_became_peered":"2023-11-28T18:26:00.176321+0000","last_unstale":"2023-11-28T18:26:01.190771+0000","last_undegraded":"2023-11-28T18:26:01.190771+0000","last_fullsized":"2023-11-28T18:26:01.190771+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:48:21.783489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.184378+0000","last_change":"2023-11-28T18:26:01.171631+0000","last_active":"2023-11-28T18:26:01.184378+0000","last_peered":"2023-11-28T18:26:01.184378+0000","last_clean":"2023-11-28T18:26:01.184378+0000","last_became_active":"2023-11-28T18:26:01.171306+0000","last_became_peered":"2023-11-28T18:26:01.171306+0000","last_unstale":"2023-11-28T18:26:01.184378+0000","last_undegraded":"2023-11-28T18:26:01.184378+0000","last_fullsized":"2023-11-28T18:26:01.184378+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:35:48.822843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.199344+0000","last_change":"2023-11-28T18:26:00.186268+0000","last_active":"2023-11-28T18:26:01.199344+0000","last_peered":"2023-11-28T18:26:01.199344+0000","last_clean":"2023-11-28T18:26:01.199344+0000","last_became_active":"2023-11-28T18:26:00.185914+0000","last_became_peered":"2023-11-28T18:26:00.185914+0000","last_unstale":"2023-11-28T18:26:01.199344+0000","last_undegraded":"2023-11-28T18:26:01.199344+0000","last_fullsized":"2023-11-28T18:26:01.199344+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:17.654673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.194552+0000","last_change":"2023-11-28T18:26:01.188915+0000","last_active":"2023-11-28T18:26:01.194552+0000","last_peered":"2023-11-28T18:26:01.194552+0000","last_clean":"2023-11-28T18:26:01.194552+0000","last_became_active":"2023-11-28T18:26:01.187374+0000","last_became_peered":"2023-11-28T18:26:01.187374+0000","last_unstale":"2023-11-28T18:26:01.194552+0000","last_undegraded":"2023-11-28T18:26:01.194552+0000","last_fullsized":"2023-11-28T18:26:01.194552+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:22.583971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.189750+0000","last_change":"2023-11-28T18:26:00.185051+0000","last_active":"2023-11-28T18:26:01.189750+0000","last_peered":"2023-11-28T18:26:01.189750+0000","last_clean":"2023-11-28T18:26:01.189750+0000","last_became_active":"2023-11-28T18:26:00.184343+0000","last_became_peered":"2023-11-28T18:26:00.184343+0000","last_unstale":"2023-11-28T18:26:01.189750+0000","last_undegraded":"2023-11-28T18:26:01.189750+0000","last_fullsized":"2023-11-28T18:26:01.189750+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:00:42.591355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.186480+0000","last_change":"2023-11-28T18:26:01.182289+0000","last_active":"2023-11-28T18:26:01.186480+0000","last_peered":"2023-11-28T18:26:01.186480+0000","last_clean":"2023-11-28T18:26:01.186480+0000","last_became_active":"2023-11-28T18:26:01.181963+0000","last_became_peered":"2023-11-28T18:26:01.181963+0000","last_unstale":"2023-11-28T18:26:01.186480+0000","last_undegraded":"2023-11-28T18:26:01.186480+0000","last_fullsized":"2023-11-28T18:26:01.186480+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:54:35.486618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.198026+0000","last_change":"2023-11-28T18:26:00.191584+0000","last_active":"2023-11-28T18:26:01.198026+0000","last_peered":"2023-11-28T18:26:01.198026+0000","last_clean":"2023-11-28T18:26:01.198026+0000","last_became_active":"2023-11-28T18:26:00.191070+0000","last_became_peered":"2023-11-28T18:26:00.191070+0000","last_unstale":"2023-11-28T18:26:01.198026+0000","last_undegraded":"2023-11-28T18:26:01.198026+0000","last_fullsized":"2023-11-28T18:26:01.198026+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:49:46.451878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.202325+0000","last_change":"2023-11-28T18:26:01.195765+0000","last_active":"2023-11-28T18:26:01.202325+0000","last_peered":"2023-11-28T18:26:01.202325+0000","last_clean":"2023-11-28T18:26:01.202325+0000","last_became_active":"2023-11-28T18:26:01.195459+0000","last_became_peered":"2023-11-28T18:26:01.195459+0000","last_unstale":"2023-11-28T18:26:01.202325+0000","last_undegraded":"2023-11-28T18:26:01.202325+0000","last_fullsized":"2023-11-28T18:26:01.202325+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:29:19.810995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.156000+0000","last_change":"2023-11-28T18:26:00.178641+0000","last_active":"2023-11-28T18:26:01.156000+0000","last_peered":"2023-11-28T18:26:01.156000+0000","last_clean":"2023-11-28T18:26:01.156000+0000","last_became_active":"2023-11-28T18:26:00.178393+0000","last_became_peered":"2023-11-28T18:26:00.178393+0000","last_unstale":"2023-11-28T18:26:01.156000+0000","last_undegraded":"2023-11-28T18:26:01.156000+0000","last_fullsized":"2023-11-28T18:26:01.156000+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:10.195481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.198733+0000","last_change":"2023-11-28T18:26:01.191534+0000","last_active":"2023-11-28T18:26:01.198733+0000","last_peered":"2023-11-28T18:26:01.198733+0000","last_clean":"2023-11-28T18:26:01.198733+0000","last_became_active":"2023-11-28T18:26:01.191235+0000","last_became_peered":"2023-11-28T18:26:01.191235+0000","last_unstale":"2023-11-28T18:26:01.198733+0000","last_undegraded":"2023-11-28T18:26:01.198733+0000","last_fullsized":"2023-11-28T18:26:01.198733+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:11.198280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.165796+0000","last_change":"2023-11-28T18:26:00.194687+0000","last_active":"2023-11-28T18:26:01.165796+0000","last_peered":"2023-11-28T18:26:01.165796+0000","last_clean":"2023-11-28T18:26:01.165796+0000","last_became_active":"2023-11-28T18:26:00.193360+0000","last_became_peered":"2023-11-28T18:26:00.193360+0000","last_unstale":"2023-11-28T18:26:01.165796+0000","last_undegraded":"2023-11-28T18:26:01.165796+0000","last_fullsized":"2023-11-28T18:26:01.165796+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:46.991934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.196949+0000","last_change":"2023-11-28T18:26:01.191891+0000","last_active":"2023-11-28T18:26:01.196949+0000","last_peered":"2023-11-28T18:26:01.196949+0000","last_clean":"2023-11-28T18:26:01.196949+0000","last_became_active":"2023-11-28T18:26:01.191637+0000","last_became_peered":"2023-11-28T18:26:01.191637+0000","last_unstale":"2023-11-28T18:26:01.196949+0000","last_undegraded":"2023-11-28T18:26:01.196949+0000","last_fullsized":"2023-11-28T18:26:01.196949+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:59.109135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.167632+0000","last_change":"2023-11-28T18:26:00.180240+0000","last_active":"2023-11-28T18:26:01.167632+0000","last_peered":"2023-11-28T18:26:01.167632+0000","last_clean":"2023-11-28T18:26:01.167632+0000","last_became_active":"2023-11-28T18:26:00.179986+0000","last_became_peered":"2023-11-28T18:26:00.179986+0000","last_unstale":"2023-11-28T18:26:01.167632+0000","last_undegraded":"2023-11-28T18:26:01.167632+0000","last_fullsized":"2023-11-28T18:26:01.167632+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:08:58.232922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.175966+0000","last_change":"2023-11-28T18:26:01.164428+0000","last_active":"2023-11-28T18:26:01.175966+0000","last_peered":"2023-11-28T18:26:01.175966+0000","last_clean":"2023-11-28T18:26:01.175966+0000","last_became_active":"2023-11-28T18:26:01.164064+0000","last_became_peered":"2023-11-28T18:26:01.164064+0000","last_unstale":"2023-11-28T18:26:01.175966+0000","last_undegraded":"2023-11-28T18:26:01.175966+0000","last_fullsized":"2023-11-28T18:26:01.175966+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:11.198280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.199731+0000","last_change":"2023-11-28T18:26:00.181694+0000","last_active":"2023-11-28T18:26:01.199731+0000","last_peered":"2023-11-28T18:26:01.199731+0000","last_clean":"2023-11-28T18:26:01.199731+0000","last_became_active":"2023-11-28T18:26:00.181397+0000","last_became_peered":"2023-11-28T18:26:00.181397+0000","last_unstale":"2023-11-28T18:26:01.199731+0000","last_undegraded":"2023-11-28T18:26:01.199731+0000","last_fullsized":"2023-11-28T18:26:01.199731+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:39:27.223567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.199567+0000","last_change":"2023-11-28T18:26:01.195945+0000","last_active":"2023-11-28T18:26:01.199567+0000","last_peered":"2023-11-28T18:26:01.199567+0000","last_clean":"2023-11-28T18:26:01.199567+0000","last_became_active":"2023-11-28T18:26:01.195698+0000","last_became_peered":"2023-11-28T18:26:01.195698+0000","last_unstale":"2023-11-28T18:26:01.199567+0000","last_undegraded":"2023-11-28T18:26:01.199567+0000","last_fullsized":"2023-11-28T18:26:01.199567+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:35:48.822843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.167861+0000","last_change":"2023-11-28T18:26:00.186551+0000","last_active":"2023-11-28T18:26:01.167861+0000","last_peered":"2023-11-28T18:26:01.167861+0000","last_clean":"2023-11-28T18:26:01.167861+0000","last_became_active":"2023-11-28T18:26:00.186199+0000","last_became_peered":"2023-11-28T18:26:00.186199+0000","last_unstale":"2023-11-28T18:26:01.167861+0000","last_undegraded":"2023-11-28T18:26:01.167861+0000","last_fullsized":"2023-11-28T18:26:01.167861+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:54:34.483820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.174163+0000","last_change":"2023-11-28T18:26:01.164790+0000","last_active":"2023-11-28T18:26:01.174163+0000","last_peered":"2023-11-28T18:26:01.174163+0000","last_clean":"2023-11-28T18:26:01.174163+0000","last_became_active":"2023-11-28T18:26:01.164511+0000","last_became_peered":"2023-11-28T18:26:01.164511+0000","last_unstale":"2023-11-28T18:26:01.174163+0000","last_undegraded":"2023-11-28T18:26:01.174163+0000","last_fullsized":"2023-11-28T18:26:01.174163+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:21:18.170945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.166100+0000","last_change":"2023-11-28T18:26:00.177595+0000","last_active":"2023-11-28T18:26:01.166100+0000","last_peered":"2023-11-28T18:26:01.166100+0000","last_clean":"2023-11-28T18:26:01.166100+0000","last_became_active":"2023-11-28T18:26:00.177325+0000","last_became_peered":"2023-11-28T18:26:00.177325+0000","last_unstale":"2023-11-28T18:26:01.166100+0000","last_undegraded":"2023-11-28T18:26:01.166100+0000","last_fullsized":"2023-11-28T18:26:01.166100+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:46.360465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.198682+0000","last_change":"2023-11-28T18:26:01.191424+0000","last_active":"2023-11-28T18:26:01.198682+0000","last_peered":"2023-11-28T18:26:01.198682+0000","last_clean":"2023-11-28T18:26:01.198682+0000","last_became_active":"2023-11-28T18:26:01.190341+0000","last_became_peered":"2023-11-28T18:26:01.190341+0000","last_unstale":"2023-11-28T18:26:01.198682+0000","last_undegraded":"2023-11-28T18:26:01.198682+0000","last_fullsized":"2023-11-28T18:26:01.198682+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:04:02.916488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.156608+0000","last_change":"2023-11-28T18:26:00.183062+0000","last_active":"2023-11-28T18:26:01.156608+0000","last_peered":"2023-11-28T18:26:01.156608+0000","last_clean":"2023-11-28T18:26:01.156608+0000","last_became_active":"2023-11-28T18:26:00.182811+0000","last_became_peered":"2023-11-28T18:26:00.182811+0000","last_unstale":"2023-11-28T18:26:01.156608+0000","last_undegraded":"2023-11-28T18:26:01.156608+0000","last_fullsized":"2023-11-28T18:26:01.156608+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:25.016209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.183476+0000","last_change":"2023-11-28T18:26:01.178630+0000","last_active":"2023-11-28T18:26:01.183476+0000","last_peered":"2023-11-28T18:26:01.183476+0000","last_clean":"2023-11-28T18:26:01.183476+0000","last_became_active":"2023-11-28T18:26:01.178336+0000","last_became_peered":"2023-11-28T18:26:01.178336+0000","last_unstale":"2023-11-28T18:26:01.183476+0000","last_undegraded":"2023-11-28T18:26:01.183476+0000","last_fullsized":"2023-11-28T18:26:01.183476+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:57.022136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.156130+0000","last_change":"2023-11-28T18:26:00.192594+0000","last_active":"2023-11-28T18:26:01.156130+0000","last_peered":"2023-11-28T18:26:01.156130+0000","last_clean":"2023-11-28T18:26:01.156130+0000","last_became_active":"2023-11-28T18:26:00.192322+0000","last_became_peered":"2023-11-28T18:26:00.192322+0000","last_unstale":"2023-11-28T18:26:01.156130+0000","last_undegraded":"2023-11-28T18:26:01.156130+0000","last_fullsized":"2023-11-28T18:26:01.156130+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:53:24.485251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.197905+0000","last_change":"2023-11-28T18:26:01.190742+0000","last_active":"2023-11-28T18:26:01.197905+0000","last_peered":"2023-11-28T18:26:01.197905+0000","last_clean":"2023-11-28T18:26:01.197905+0000","last_became_active":"2023-11-28T18:26:01.190454+0000","last_became_peered":"2023-11-28T18:26:01.190454+0000","last_unstale":"2023-11-28T18:26:01.197905+0000","last_undegraded":"2023-11-28T18:26:01.197905+0000","last_fullsized":"2023-11-28T18:26:01.197905+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:53:25.488049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.160504+0000","last_change":"2023-11-28T18:26:00.185271+0000","last_active":"2023-11-28T18:26:01.160504+0000","last_peered":"2023-11-28T18:26:01.160504+0000","last_clean":"2023-11-28T18:26:01.160504+0000","last_became_active":"2023-11-28T18:26:00.184993+0000","last_became_peered":"2023-11-28T18:26:00.184993+0000","last_unstale":"2023-11-28T18:26:01.160504+0000","last_undegraded":"2023-11-28T18:26:01.160504+0000","last_fullsized":"2023-11-28T18:26:01.160504+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:07:33.583566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.190958+0000","last_change":"2023-11-28T18:26:01.186582+0000","last_active":"2023-11-28T18:26:01.190958+0000","last_peered":"2023-11-28T18:26:01.190958+0000","last_clean":"2023-11-28T18:26:01.190958+0000","last_became_active":"2023-11-28T18:26:01.186329+0000","last_became_peered":"2023-11-28T18:26:01.186329+0000","last_unstale":"2023-11-28T18:26:01.190958+0000","last_undegraded":"2023-11-28T18:26:01.190958+0000","last_fullsized":"2023-11-28T18:26:01.190958+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:22:53.430433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.161028+0000","last_change":"2023-11-28T18:26:00.193648+0000","last_active":"2023-11-28T18:26:01.161028+0000","last_peered":"2023-11-28T18:26:01.161028+0000","last_clean":"2023-11-28T18:26:01.161028+0000","last_became_active":"2023-11-28T18:26:00.193395+0000","last_became_peered":"2023-11-28T18:26:00.193395+0000","last_unstale":"2023-11-28T18:26:01.161028+0000","last_undegraded":"2023-11-28T18:26:01.161028+0000","last_fullsized":"2023-11-28T18:26:01.161028+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:21:17.168147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.202439+0000","last_change":"2023-11-28T18:26:01.196447+0000","last_active":"2023-11-28T18:26:01.202439+0000","last_peered":"2023-11-28T18:26:01.202439+0000","last_clean":"2023-11-28T18:26:01.202439+0000","last_became_active":"2023-11-28T18:26:01.196190+0000","last_became_peered":"2023-11-28T18:26:01.196190+0000","last_unstale":"2023-11-28T18:26:01.202439+0000","last_undegraded":"2023-11-28T18:26:01.202439+0000","last_fullsized":"2023-11-28T18:26:01.202439+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:35:48.822843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.165481+0000","last_change":"2023-11-28T18:26:00.176639+0000","last_active":"2023-11-28T18:26:01.165481+0000","last_peered":"2023-11-28T18:26:01.165481+0000","last_clean":"2023-11-28T18:26:01.165481+0000","last_became_active":"2023-11-28T18:26:00.176259+0000","last_became_peered":"2023-11-28T18:26:00.176259+0000","last_unstale":"2023-11-28T18:26:01.165481+0000","last_undegraded":"2023-11-28T18:26:01.165481+0000","last_fullsized":"2023-11-28T18:26:01.165481+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:18:00.940327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T18:26:00.174525+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:26:00.174525+0000","last_peered":"2023-11-28T18:26:00.174525+0000","last_clean":"2023-11-28T18:26:00.174525+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:26:00.174525+0000","last_undegraded":"2023-11-28T18:26:00.174525+0000","last_fullsized":"2023-11-28T18:26:00.174525+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36492,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335348,"statfs":{"total":96636764160,"available":96599396352,"internally_reserved":0,"allocated":1175552,"data_stored":845817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.71699999999999997,"15min":0.73799999999999999},"min":{"1min":0.44500000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.77300000000000002,"5min":2.427,"15min":2.427},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.71599999999999997,"15min":0.73799999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77700000000000002,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.53400000000000003}]},{"osd":1,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.71199999999999997,"15min":0.73199999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.873,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.755,"15min":0.77400000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88600000000000001,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.621}]},{"osd":2,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.77500000000000002,"15min":0.79400000000000004},"min":{"1min":0.501,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.82699999999999996,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.73999999999999999,"15min":0.75900000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.83499999999999996,"5min":2.411,"15min":2.411},"last":0.5}]},{"osd":3,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.76300000000000001,"15min":0.78300000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.81599999999999995,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.80600000000000005,"15min":0.82299999999999995},"min":{"1min":0.54700000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.89100000000000001,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":0.70199999999999996}]},{"osd":4,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86099999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.64200000000000002,"15min":0.65400000000000003},"min":{"1min":0.39900000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.80600000000000005,"5min":0.877,"15min":0.877},"last":0.72299999999999998}]},{"osd":5,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68200000000000005,"15min":0.68799999999999994},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91700000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.70499999999999996,"15min":0.71499999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92300000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63500000000000001}]},{"osd":6,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.74399999999999999,"15min":0.755},"min":{"1min":0.48499999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.96499999999999997,"5min":1.042,"15min":1.042},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.73999999999999999,"15min":0.753},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85899999999999999,"5min":1.006,"15min":1.006},"last":0.55200000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31752,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340088,"statfs":{"total":96636764160,"available":96604250112,"internally_reserved":0,"allocated":581632,"data_stored":255449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63300000000000001,"15min":0.63400000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.72699999999999998,"15min":0.73499999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.88900000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.66000000000000003}]},{"osd":1,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.80800000000000005,"15min":0.83599999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.878,"5min":4.2389999999999999,"15min":4.2389999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.79400000000000004,"15min":0.81399999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92600000000000005,"5min":4.218,"15min":4.218},"last":0.69699999999999995}]},{"osd":2,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.71299999999999997,"15min":0.72499999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82899999999999996,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70899999999999996,"15min":0.71099999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.64600000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72699999999999998,"15min":0.73499999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93700000000000006,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.57999999999999996}]},{"osd":4,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.44600000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.93999999999999995,"5min":0.995,"15min":0.995},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.69199999999999995,"15min":0.70199999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89900000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.5}]},{"osd":5,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.68400000000000005,"15min":0.69299999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.86799999999999999,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.629},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.77000000000000002,"15min":0.77700000000000002},"min":{"1min":0.51800000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.91800000000000004,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.61599999999999999}]},{"osd":7,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.77600000000000002,"15min":0.78800000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.97399999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75800000000000001,"15min":0.76400000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.90300000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.68200000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31752,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340088,"statfs":{"total":96636764160,"available":96604250112,"internally_reserved":0,"allocated":581632,"data_stored":255449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.70799999999999996,"15min":0.71599999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.92400000000000004,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.746,"15min":0.76100000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.95599999999999996,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":0.65800000000000003}]},{"osd":1,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.74199999999999999,"15min":0.754},"min":{"1min":0.437,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91400000000000003,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.78500000000000003,"15min":0.81399999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86299999999999999,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.67600000000000005}]},{"osd":2,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.75800000000000001,"15min":0.77600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89900000000000002,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.621},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.78000000000000003,"15min":0.79600000000000004},"min":{"1min":0.51500000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94099999999999995,"5min":4.0149999999999997,"15min":4.0149999999999997},"last":0.52000000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.82299999999999995,"15min":0.84899999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.89000000000000001,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.81899999999999995,"15min":0.84999999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80000000000000004,"5min":3.9900000000000002,"15min":3.9900000000000002},"last":0.54300000000000004}]},{"osd":4,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.40699999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71299999999999997,"15min":0.71699999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.90400000000000003,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.72599999999999998}]},{"osd":6,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.372,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90700000000000003,"5min":1.071,"15min":1.071},"last":0.501},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71899999999999997,"15min":0.72199999999999998},"min":{"1min":0.47999999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.98099999999999998,"5min":1.756,"15min":1.756},"last":0.59099999999999997}]},{"osd":7,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.75,"15min":0.75700000000000001},"min":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97299999999999998,"5min":1.282,"15min":1.282},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.75600000000000001,"15min":0.75900000000000001},"min":{"1min":0.505,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93200000000000005,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.70799999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31752,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340088,"statfs":{"total":96636764160,"available":96604250112,"internally_reserved":0,"allocated":581632,"data_stored":255449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.67800000000000005,"15min":0.69699999999999995},"min":{"1min":0.33600000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.91900000000000004,"5min":2.363,"15min":2.363},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64100000000000001,"15min":0.64600000000000002},"min":{"1min":0.442,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88700000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.81799999999999995}]},{"osd":1,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.624,"15min":0.621},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.026,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.77800000000000002}]},{"osd":2,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.88},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.72699999999999998,"15min":0.73099999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98499999999999999,"5min":1.871,"15min":1.871},"last":0.627}]},{"osd":3,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64200000000000002,"15min":0.64500000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83799999999999997,"5min":0.878,"15min":0.878},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.83199999999999996,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.79700000000000004}]},{"osd":5,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.84399999999999997,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.63500000000000001,"15min":0.64500000000000002},"min":{"1min":0.35799999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82399999999999995,"5min":1.79,"15min":1.79},"last":0.75800000000000001}]},{"osd":6,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65000000000000002,"15min":0.65400000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85599999999999998,"5min":1.123,"15min":1.123},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70599999999999996,"15min":0.71199999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93000000000000005,"5min":1.925,"15min":1.925},"last":0.66600000000000004}]},{"osd":7,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.71399999999999997,"15min":0.72399999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90000000000000002,"5min":1.843,"15min":1.843},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.77800000000000002,"15min":0.78400000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.94199999999999995,"5min":1.593,"15min":1.593},"last":0.91600000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35928,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335912,"statfs":{"total":96636764160,"available":96599973888,"internally_reserved":0,"allocated":593920,"data_stored":262544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.26900000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.46000000000000002,"15min":0.46500000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.59299999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.52700000000000002}]},{"osd":1,"last update":"Tue Nov 28 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.46000000000000002,"15min":0.46400000000000002},"min":{"1min":0.30599999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.56299999999999994,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.495,"15min":0.499},"min":{"1min":0.33900000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.63200000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.55300000000000005}]},{"osd":2,"last update":"Tue Nov 28 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.51200000000000001,"15min":0.51800000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65000000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79700000000000004,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.50600000000000001}]},{"osd":4,"last update":"Tue Nov 28 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76500000000000001,"15min":0.77200000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95199999999999996,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76300000000000001,"15min":0.77100000000000002},"min":{"1min":0.56100000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0049999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.69199999999999995}]},{"osd":5,"last update":"Tue Nov 28 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.82799999999999996,"15min":0.84199999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.052,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.052,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.80600000000000005}]},{"osd":6,"last update":"Tue Nov 28 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.79200000000000004,"15min":0.80000000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0189999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.79400000000000004,"15min":0.80200000000000005},"min":{"1min":0.59899999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0760000000000001,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.82399999999999995}]},{"osd":7,"last update":"Tue Nov 28 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.80200000000000005,"15min":0.81499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93799999999999994,"5min":1.786,"15min":1.786},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.123,"5min":1.337,"15min":1.337},"last":0.753}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":581632,"data_stored":255449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.50700000000000001,"15min":0.51000000000000001},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.4470000000000001,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.5,"15min":0.51200000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.623,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.41999999999999998}]},{"osd":1,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.50700000000000001,"15min":0.51400000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.63800000000000001,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.49199999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.499,"15min":0.50600000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67700000000000005,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.53300000000000003}]},{"osd":4,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77800000000000002,"15min":0.77600000000000002},"min":{"1min":0.55600000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.012,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.73599999999999999}]},{"osd":5,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.76300000000000001,"15min":0.77100000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0069999999999999,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.997,"15min":1.0329999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.5449999999999999,"5min":7.4820000000000002,"15min":7.4820000000000002},"last":0.75}]},{"osd":6,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.79100000000000004,"15min":0.80300000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0449999999999999,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.85399999999999998,"15min":0.88100000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.90800000000000003,"5min":3.9380000000000002,"15min":3.9380000000000002},"last":0.76100000000000001}]},{"osd":7,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.83399999999999996,"15min":0.84699999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.94999999999999996,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.90600000000000003,"15min":0.93500000000000005},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0509999999999999,"5min":4.798,"15min":4.798},"last":0.72399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":581632,"data_stored":255449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.42299999999999999,"15min":0.42899999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.44800000000000001,"15min":0.45000000000000001},"min":{"1min":0.29899999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57799999999999996,"5min":0.62,"15min":0.62},"last":0.50800000000000001}]},{"osd":2,"last update":"Tue Nov 28 18:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.34100000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.63,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47899999999999998,"15min":0.47799999999999998},"min":{"1min":0.35499999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.38}]},{"osd":3,"last update":"Tue Nov 28 18:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.67300000000000004,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.501},{"interface":"front","average":{"1min":0.503,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.34699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.67600000000000005}]},{"osd":4,"last update":"Tue Nov 28 18:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.92200000000000004,"5min":1.512,"15min":1.512},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76600000000000001,"15min":0.77200000000000002},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.98099999999999998,"5min":1.155,"15min":1.155},"last":0.64000000000000001}]},{"osd":5,"last update":"Tue Nov 28 18:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.747,"15min":0.755},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.89900000000000002,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.745,"15min":0.755},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.92300000000000004,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.57299999999999995}]},{"osd":6,"last update":"Tue Nov 28 18:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.94499999999999995,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.96199999999999997,"5min":1.448,"15min":1.448},"last":0.65700000000000003}]},{"osd":7,"last update":"Tue Nov 28 18:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.753,"15min":0.75700000000000001},"min":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94899999999999995,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.61299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1175552,"data_stored":845817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 18:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.438,"15min":0.44600000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.51900000000000002,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.46600000000000003,"15min":0.47099999999999997},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56999999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.53800000000000003}]},{"osd":2,"last update":"Tue Nov 28 18:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.50600000000000001,"15min":0.51200000000000001},"min":{"1min":0.30599999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.627,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.50900000000000001,"15min":0.51800000000000002},"min":{"1min":0.33700000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.621,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.40100000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48899999999999999,"15min":0.49199999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.54500000000000004,"15min":0.55600000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.66800000000000004,"5min":1.087,"15min":1.087},"last":0.58799999999999997}]},{"osd":4,"last update":"Tue Nov 28 18:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71199999999999997,"15min":0.71299999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94199999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71699999999999997,"15min":0.71799999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.78800000000000003}]},{"osd":5,"last update":"Tue Nov 28 18:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.83699999999999997}]},{"osd":6,"last update":"Tue Nov 28 18:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.72799999999999998,"15min":0.73299999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.997,"5min":1.407,"15min":1.407},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.085,"5min":1.089,"15min":1.089},"last":0.77300000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.755,"15min":0.754},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.877},{"interface":"front","average":{"1min":0.746,"5min":0.748,"15min":0.748},"min":{"1min":0.57099999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.022,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.72199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:26:05.237 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:26:05.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.408+0000 7f7de5f1d640 1 Processor -- start 2023-11-28T18:26:05.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.408+0000 7f7de5f1d640 1 -- start start 2023-11-28T18:26:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.408+0000 7f7de5f1d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7de015d4f0 0x7f7de00d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.408+0000 7f7de5f1d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7de00b2d20 0x7f7de00b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.408+0000 7f7de5f1d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de00b25b0 con 0x7f7de00d3c30 2023-11-28T18:26:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.408+0000 7f7de5f1d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de00b2900 con 0x7f7de00b2d20 2023-11-28T18:26:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.408+0000 7f7de5f1d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de00d0a20 con 0x7f7de015d4f0 2023-11-28T18:26:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.408+0000 7f7ddffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7de00b2d20 0x7f7de00b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.408+0000 7f7ddffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7de00b2d20 0x7f7de00b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45946/0 (socket says 172.21.15.105:45946) 2023-11-28T18:26:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.408+0000 7f7ddffff640 1 -- 172.21.15.105:0/4099534255 learned_addr learned my addr 172.21.15.105:0/4099534255 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.408+0000 7f7ddf7fe640 1 --2- 172.21.15.105:0/4099534255 >> v2:172.21.15.148:3300/0 conn(0x7f7de015d4f0 0x7f7de00d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.408+0000 7f7ddffff640 1 -- 172.21.15.105:0/4099534255 >> v2:172.21.15.148:3300/0 conn(0x7f7de015d4f0 msgr2=0x7f7de00d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.408+0000 7f7ddffff640 1 --2- 172.21.15.105:0/4099534255 >> v2:172.21.15.148:3300/0 conn(0x7f7de015d4f0 0x7f7de00d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.408+0000 7f7ddffff640 1 -- 172.21.15.105:0/4099534255 >> v1:172.21.15.148:6789/0 conn(0x7f7de00d3c30 legacy=0x7f7de00d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.408+0000 7f7ddffff640 1 -- 172.21.15.105:0/4099534255 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de00cea30 con 0x7f7de00b2d20 2023-11-28T18:26:05.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.412+0000 7f7ddffff640 1 --2- 172.21.15.105:0/4099534255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7de00b2d20 0x7f7de00b3150 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7de008bbf0 tx=0x7f7dd801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd1c67843341b0e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:05.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.412+0000 7f7dde7fc640 1 -- 172.21.15.105:0/4099534255 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd80a57d0 con 0x7f7de00b2d20 2023-11-28T18:26:05.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.412+0000 7f7dde7fc640 1 -- 172.21.15.105:0/4099534255 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd80a5930 con 0x7f7de00b2d20 2023-11-28T18:26:05.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.412+0000 7f7dde7fc640 1 -- 172.21.15.105:0/4099534255 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd80a5cc0 con 0x7f7de00b2d20 2023-11-28T18:26:05.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.412+0000 7f7de5f1d640 1 -- 172.21.15.105:0/4099534255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7de00b2d20 msgr2=0x7f7de00b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:05.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.412+0000 7f7de5f1d640 1 --2- 172.21.15.105:0/4099534255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7de00b2d20 0x7f7de00b3150 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7de008bbf0 tx=0x7f7dd801c910 comp rx=0 tx=0).stop 2023-11-28T18:26:05.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.412+0000 7f7de5f1d640 1 -- 172.21.15.105:0/4099534255 shutdown_connections 2023-11-28T18:26:05.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.412+0000 7f7de5f1d640 1 --2- 172.21.15.105:0/4099534255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7de00b2d20 0x7f7de00b3150 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.412+0000 7f7de5f1d640 1 --2- 172.21.15.105:0/4099534255 >> v2:172.21.15.148:3300/0 conn(0x7f7de015d4f0 0x7f7de00d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.412+0000 7f7de5f1d640 1 -- 172.21.15.105:0/4099534255 >> 172.21.15.105:0/4099534255 conn(0x7f7de00d3030 msgr2=0x7f7de00d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:05.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.412+0000 7f7de5f1d640 1 -- 172.21.15.105:0/4099534255 shutdown_connections 2023-11-28T18:26:05.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.412+0000 7f7de5f1d640 1 -- 172.21.15.105:0/4099534255 wait complete. 2023-11-28T18:26:05.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7de5f1d640 1 Processor -- start 2023-11-28T18:26:05.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7de5f1d640 1 -- start start 2023-11-28T18:26:05.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7de5f1d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7de00b2d20 0x7f7de00b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:05.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7de5f1d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7de00d3c30 0x7f7de014a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:05.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7de5f1d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7de015d4f0 0x7f7de00cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:05.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7de5f1d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de0159cf0 con 0x7f7de00b2d20 2023-11-28T18:26:05.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7de5f1d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de00ca100 con 0x7f7de00d3c30 2023-11-28T18:26:05.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7de5f1d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de00cca40 con 0x7f7de015d4f0 2023-11-28T18:26:05.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7ddf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7de00b2d20 0x7f7de00b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7ddf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7de00b2d20 0x7f7de00b5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45956/0 (socket says 172.21.15.105:45956) 2023-11-28T18:26:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7ddf7fe640 1 -- 172.21.15.105:0/713038749 learned_addr learned my addr 172.21.15.105:0/713038749 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7ddeffd640 1 --2- 172.21.15.105:0/713038749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7de00d3c30 0x7f7de014a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7ddffff640 1 --2- 172.21.15.105:0/713038749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7de015d4f0 0x7f7de00cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7ddf7fe640 1 -- 172.21.15.105:0/713038749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7de015d4f0 msgr2=0x7f7de00cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7ddf7fe640 1 --2- 172.21.15.105:0/713038749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7de015d4f0 0x7f7de00cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7ddf7fe640 1 -- 172.21.15.105:0/713038749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7de00d3c30 msgr2=0x7f7de014a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7ddf7fe640 1 --2- 172.21.15.105:0/713038749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7de00d3c30 0x7f7de014a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7ddf7fe640 1 -- 172.21.15.105:0/713038749 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd8009b30 con 0x7f7de00b2d20 2023-11-28T18:26:05.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7ddffff640 1 --2- 172.21.15.105:0/713038749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7de015d4f0 0x7f7de00cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:05.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7ddf7fe640 1 --2- 172.21.15.105:0/713038749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7de00b2d20 0x7f7de00b5c30 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000e870 tx=0x7f7dd000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:05.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7dc7fff640 1 -- 172.21.15.105:0/713038749 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0009510 con 0x7f7de00b2d20 2023-11-28T18:26:05.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7dc7fff640 1 -- 172.21.15.105:0/713038749 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd000eec0 con 0x7f7de00b2d20 2023-11-28T18:26:05.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7de5f1d640 1 -- 172.21.15.105:0/713038749 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de00ccd20 con 0x7f7de00b2d20 2023-11-28T18:26:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7dc7fff640 1 -- 172.21.15.105:0/713038749 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd000cb40 con 0x7f7de00b2d20 2023-11-28T18:26:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.416+0000 7f7de5f1d640 1 -- 172.21.15.105:0/713038749 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de00ca630 con 0x7f7de00b2d20 2023-11-28T18:26:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.420+0000 7f7de5f1d640 1 -- 172.21.15.105:0/713038749 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7de00ccf60 con 0x7f7de00b2d20 2023-11-28T18:26:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.420+0000 7f7dc7fff640 1 -- 172.21.15.105:0/713038749 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f7dd000cca0 con 0x7f7de00b2d20 2023-11-28T18:26:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.420+0000 7f7dc7fff640 1 --2- 172.21.15.105:0/713038749 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7dcc0668d0 0x7f7dcc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.420+0000 7f7dc7fff640 1 -- 172.21.15.105:0/713038749 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7dd00846a0 con 0x7f7de00b2d20 2023-11-28T18:26:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.420+0000 7f7ddeffd640 1 --2- 172.21.15.105:0/713038749 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7dcc0668d0 0x7f7dcc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.420+0000 7f7ddeffd640 1 --2- 172.21.15.105:0/713038749 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7dcc0668d0 0x7f7dcc068d90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7de0155c80 tx=0x7f7dd4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.424+0000 7f7dc7fff640 1 -- 172.21.15.105:0/713038749 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7dd00517b0 con 0x7f7de00b2d20 2023-11-28T18:26:05.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.568+0000 7f7de5f1d640 1 -- 172.21.15.105:0/713038749 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7de00d1a60 con 0x7f7dcc0668d0 2023-11-28T18:26:05.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.576+0000 7f7dc7fff640 1 -- 172.21.15.105:0/713038749 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289156 (secure 0 0 0) 0x7f7dd40050d0 con 0x7f7dcc0668d0 2023-11-28T18:26:05.583 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:05.585 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:26:05.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.580+0000 7f7de5f1d640 1 -- 172.21.15.105:0/713038749 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7dcc0668d0 msgr2=0x7f7dcc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:05.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.580+0000 7f7de5f1d640 1 --2- 172.21.15.105:0/713038749 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7dcc0668d0 0x7f7dcc068d90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7de0155c80 tx=0x7f7dd4009040 comp rx=0 tx=0).stop 2023-11-28T18:26:05.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.580+0000 7f7de5f1d640 1 -- 172.21.15.105:0/713038749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7de00b2d20 msgr2=0x7f7de00b5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:05.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.580+0000 7f7de5f1d640 1 --2- 172.21.15.105:0/713038749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7de00b2d20 0x7f7de00b5c30 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000e870 tx=0x7f7dd000ec70 comp rx=0 tx=0).stop 2023-11-28T18:26:05.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.580+0000 7f7de5f1d640 1 -- 172.21.15.105:0/713038749 shutdown_connections 2023-11-28T18:26:05.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.580+0000 7f7de5f1d640 1 --2- 172.21.15.105:0/713038749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7de015d4f0 0x7f7de00cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.580+0000 7f7de5f1d640 1 --2- 172.21.15.105:0/713038749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7de00d3c30 0x7f7de014a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.580+0000 7f7de5f1d640 1 --2- 172.21.15.105:0/713038749 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7dcc0668d0 0x7f7dcc068d90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.580+0000 7f7de5f1d640 1 --2- 172.21.15.105:0/713038749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7de00b2d20 0x7f7de00b5c30 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.580+0000 7f7de5f1d640 1 -- 172.21.15.105:0/713038749 >> 172.21.15.105:0/713038749 conn(0x7f7de00d3030 msgr2=0x7f7de00afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:05.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.580+0000 7f7de5f1d640 1 -- 172.21.15.105:0/713038749 shutdown_connections 2023-11-28T18:26:05.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.580+0000 7f7de5f1d640 1 -- 172.21.15.105:0/713038749 wait complete. 2023-11-28T18:26:05.602 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2023-11-28T18:26:05.301077+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":68,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255192,"kb_used_data":5716,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754719528,"statfs":{"total":773094113280,"available":772832796672,"internally_reserved":0,"allocated":5853184,"data_stored":3231423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000943"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.186966+0000","last_change":"2023-11-28T18:26:01.181834+0000","last_active":"2023-11-28T18:26:01.186966+0000","last_peered":"2023-11-28T18:26:01.186966+0000","last_clean":"2023-11-28T18:26:01.186966+0000","last_became_active":"2023-11-28T18:26:01.181524+0000","last_became_peered":"2023-11-28T18:26:01.181524+0000","last_unstale":"2023-11-28T18:26:01.186966+0000","last_undegraded":"2023-11-28T18:26:01.186966+0000","last_fullsized":"2023-11-28T18:26:01.186966+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:19:20.407923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.181983+0000","last_change":"2023-11-28T18:26:01.178324+0000","last_active":"2023-11-28T18:26:01.181983+0000","last_peered":"2023-11-28T18:26:01.181983+0000","last_clean":"2023-11-28T18:26:01.181983+0000","last_became_active":"2023-11-28T18:26:01.178100+0000","last_became_peered":"2023-11-28T18:26:01.178100+0000","last_unstale":"2023-11-28T18:26:01.181983+0000","last_undegraded":"2023-11-28T18:26:01.181983+0000","last_fullsized":"2023-11-28T18:26:01.181983+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:53:25.488049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.188388+0000","last_change":"2023-11-28T18:26:01.179213+0000","last_active":"2023-11-28T18:26:01.188388+0000","last_peered":"2023-11-28T18:26:01.188388+0000","last_clean":"2023-11-28T18:26:01.188388+0000","last_became_active":"2023-11-28T18:26:01.178919+0000","last_became_peered":"2023-11-28T18:26:01.178919+0000","last_unstale":"2023-11-28T18:26:01.188388+0000","last_undegraded":"2023-11-28T18:26:01.188388+0000","last_fullsized":"2023-11-28T18:26:01.188388+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:54:35.486618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.195289+0000","last_change":"2023-11-28T18:26:01.191139+0000","last_active":"2023-11-28T18:26:01.195289+0000","last_peered":"2023-11-28T18:26:01.195289+0000","last_clean":"2023-11-28T18:26:01.195289+0000","last_became_active":"2023-11-28T18:26:01.190752+0000","last_became_peered":"2023-11-28T18:26:01.190752+0000","last_unstale":"2023-11-28T18:26:01.195289+0000","last_undegraded":"2023-11-28T18:26:01.195289+0000","last_fullsized":"2023-11-28T18:26:01.195289+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:36.399960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.196685+0000","last_change":"2023-11-28T18:26:01.186179+0000","last_active":"2023-11-28T18:26:01.196685+0000","last_peered":"2023-11-28T18:26:01.196685+0000","last_clean":"2023-11-28T18:26:01.196685+0000","last_became_active":"2023-11-28T18:26:01.184960+0000","last_became_peered":"2023-11-28T18:26:01.184960+0000","last_unstale":"2023-11-28T18:26:01.196685+0000","last_undegraded":"2023-11-28T18:26:01.196685+0000","last_fullsized":"2023-11-28T18:26:01.196685+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:00:43.594153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.189253+0000","last_change":"2023-11-28T18:26:01.184813+0000","last_active":"2023-11-28T18:26:01.189253+0000","last_peered":"2023-11-28T18:26:01.189253+0000","last_clean":"2023-11-28T18:26:01.189253+0000","last_became_active":"2023-11-28T18:26:01.184541+0000","last_became_peered":"2023-11-28T18:26:01.184541+0000","last_unstale":"2023-11-28T18:26:01.189253+0000","last_undegraded":"2023-11-28T18:26:01.189253+0000","last_fullsized":"2023-11-28T18:26:01.189253+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:41:06.893526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.187938+0000","last_change":"2023-11-28T18:26:01.180760+0000","last_active":"2023-11-28T18:26:01.187938+0000","last_peered":"2023-11-28T18:26:01.187938+0000","last_clean":"2023-11-28T18:26:01.187938+0000","last_became_active":"2023-11-28T18:26:01.180414+0000","last_became_peered":"2023-11-28T18:26:01.180414+0000","last_unstale":"2023-11-28T18:26:01.187938+0000","last_undegraded":"2023-11-28T18:26:01.187938+0000","last_fullsized":"2023-11-28T18:26:01.187938+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:22:53.430433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.186211+0000","last_change":"2023-11-28T18:26:01.175611+0000","last_active":"2023-11-28T18:26:01.186211+0000","last_peered":"2023-11-28T18:26:01.186211+0000","last_clean":"2023-11-28T18:26:01.186211+0000","last_became_active":"2023-11-28T18:26:01.175354+0000","last_became_peered":"2023-11-28T18:26:01.175354+0000","last_unstale":"2023-11-28T18:26:01.186211+0000","last_undegraded":"2023-11-28T18:26:01.186211+0000","last_fullsized":"2023-11-28T18:26:01.186211+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:21:01.828174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.185705+0000","last_change":"2023-11-28T18:26:01.172741+0000","last_active":"2023-11-28T18:26:01.185705+0000","last_peered":"2023-11-28T18:26:01.185705+0000","last_clean":"2023-11-28T18:26:01.185705+0000","last_became_active":"2023-11-28T18:26:01.172467+0000","last_became_peered":"2023-11-28T18:26:01.172467+0000","last_unstale":"2023-11-28T18:26:01.185705+0000","last_undegraded":"2023-11-28T18:26:01.185705+0000","last_fullsized":"2023-11-28T18:26:01.185705+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:17:35.391422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.194596+0000","last_change":"2023-11-28T18:26:01.189717+0000","last_active":"2023-11-28T18:26:01.194596+0000","last_peered":"2023-11-28T18:26:01.194596+0000","last_clean":"2023-11-28T18:26:01.194596+0000","last_became_active":"2023-11-28T18:26:01.189446+0000","last_became_peered":"2023-11-28T18:26:01.189446+0000","last_unstale":"2023-11-28T18:26:01.194596+0000","last_undegraded":"2023-11-28T18:26:01.194596+0000","last_fullsized":"2023-11-28T18:26:01.194596+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:39:28.226366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.191823+0000","last_change":"2023-11-28T18:26:01.183299+0000","last_active":"2023-11-28T18:26:01.191823+0000","last_peered":"2023-11-28T18:26:01.191823+0000","last_clean":"2023-11-28T18:26:01.191823+0000","last_became_active":"2023-11-28T18:26:01.182848+0000","last_became_peered":"2023-11-28T18:26:01.182848+0000","last_unstale":"2023-11-28T18:26:01.191823+0000","last_undegraded":"2023-11-28T18:26:01.191823+0000","last_fullsized":"2023-11-28T18:26:01.191823+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:34:02.510862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.201988+0000","last_change":"2023-11-28T18:26:01.197025+0000","last_active":"2023-11-28T18:26:01.201988+0000","last_peered":"2023-11-28T18:26:01.201988+0000","last_clean":"2023-11-28T18:26:01.201988+0000","last_became_active":"2023-11-28T18:26:01.196745+0000","last_became_peered":"2023-11-28T18:26:01.196745+0000","last_unstale":"2023-11-28T18:26:01.201988+0000","last_undegraded":"2023-11-28T18:26:01.201988+0000","last_fullsized":"2023-11-28T18:26:01.201988+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:18:01.943125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.194680+0000","last_change":"2023-11-28T18:26:01.190290+0000","last_active":"2023-11-28T18:26:01.194680+0000","last_peered":"2023-11-28T18:26:01.194680+0000","last_clean":"2023-11-28T18:26:01.194680+0000","last_became_active":"2023-11-28T18:26:01.189701+0000","last_became_peered":"2023-11-28T18:26:01.189701+0000","last_unstale":"2023-11-28T18:26:01.194680+0000","last_undegraded":"2023-11-28T18:26:01.194680+0000","last_fullsized":"2023-11-28T18:26:01.194680+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:49.681252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.189783+0000","last_change":"2023-11-28T18:26:01.182292+0000","last_active":"2023-11-28T18:26:01.189783+0000","last_peered":"2023-11-28T18:26:01.189783+0000","last_clean":"2023-11-28T18:26:01.189783+0000","last_became_active":"2023-11-28T18:26:01.181852+0000","last_became_peered":"2023-11-28T18:26:01.181852+0000","last_unstale":"2023-11-28T18:26:01.189783+0000","last_undegraded":"2023-11-28T18:26:01.189783+0000","last_fullsized":"2023-11-28T18:26:01.189783+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:18:33.835504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.199823+0000","last_change":"2023-11-28T18:26:01.195150+0000","last_active":"2023-11-28T18:26:01.199823+0000","last_peered":"2023-11-28T18:26:01.199823+0000","last_clean":"2023-11-28T18:26:01.199823+0000","last_became_active":"2023-11-28T18:26:01.194716+0000","last_became_peered":"2023-11-28T18:26:01.194716+0000","last_unstale":"2023-11-28T18:26:01.199823+0000","last_undegraded":"2023-11-28T18:26:01.199823+0000","last_fullsized":"2023-11-28T18:26:01.199823+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:18:33.835504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.192221+0000","last_change":"2023-11-28T18:26:01.185312+0000","last_active":"2023-11-28T18:26:01.192221+0000","last_peered":"2023-11-28T18:26:01.192221+0000","last_clean":"2023-11-28T18:26:01.192221+0000","last_became_active":"2023-11-28T18:26:01.185070+0000","last_became_peered":"2023-11-28T18:26:01.185070+0000","last_unstale":"2023-11-28T18:26:01.192221+0000","last_undegraded":"2023-11-28T18:26:01.192221+0000","last_fullsized":"2023-11-28T18:26:01.192221+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:29:19.810995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.175619+0000","last_change":"2023-11-28T18:26:01.171993+0000","last_active":"2023-11-28T18:26:01.175619+0000","last_peered":"2023-11-28T18:26:01.175619+0000","last_clean":"2023-11-28T18:26:01.175619+0000","last_became_active":"2023-11-28T18:26:01.171661+0000","last_became_peered":"2023-11-28T18:26:01.171661+0000","last_unstale":"2023-11-28T18:26:01.175619+0000","last_undegraded":"2023-11-28T18:26:01.175619+0000","last_fullsized":"2023-11-28T18:26:01.175619+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:06.029532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.193503+0000","last_change":"2023-11-28T18:26:01.187351+0000","last_active":"2023-11-28T18:26:01.193503+0000","last_peered":"2023-11-28T18:26:01.193503+0000","last_clean":"2023-11-28T18:26:01.193503+0000","last_became_active":"2023-11-28T18:26:01.187033+0000","last_became_peered":"2023-11-28T18:26:01.187033+0000","last_unstale":"2023-11-28T18:26:01.193503+0000","last_undegraded":"2023-11-28T18:26:01.193503+0000","last_fullsized":"2023-11-28T18:26:01.193503+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:06.029532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.189812+0000","last_change":"2023-11-28T18:26:01.188488+0000","last_active":"2023-11-28T18:26:01.189812+0000","last_peered":"2023-11-28T18:26:01.189812+0000","last_clean":"2023-11-28T18:26:01.189812+0000","last_became_active":"2023-11-28T18:26:01.188162+0000","last_became_peered":"2023-11-28T18:26:01.188162+0000","last_unstale":"2023-11-28T18:26:01.189812+0000","last_undegraded":"2023-11-28T18:26:01.189812+0000","last_fullsized":"2023-11-28T18:26:01.189812+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:26:09.798191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.202468+0000","last_change":"2023-11-28T18:26:01.196782+0000","last_active":"2023-11-28T18:26:01.202468+0000","last_peered":"2023-11-28T18:26:01.202468+0000","last_clean":"2023-11-28T18:26:01.202468+0000","last_became_active":"2023-11-28T18:26:01.196278+0000","last_became_peered":"2023-11-28T18:26:01.196278+0000","last_unstale":"2023-11-28T18:26:01.202468+0000","last_undegraded":"2023-11-28T18:26:01.202468+0000","last_fullsized":"2023-11-28T18:26:01.202468+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:04:46.200690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.202502+0000","last_change":"2023-11-28T18:26:01.197765+0000","last_active":"2023-11-28T18:26:01.202502+0000","last_peered":"2023-11-28T18:26:01.202502+0000","last_clean":"2023-11-28T18:26:01.202502+0000","last_became_active":"2023-11-28T18:26:01.197470+0000","last_became_peered":"2023-11-28T18:26:01.197470+0000","last_unstale":"2023-11-28T18:26:01.202502+0000","last_undegraded":"2023-11-28T18:26:01.202502+0000","last_fullsized":"2023-11-28T18:26:01.202502+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:52:54.081912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.185551+0000","last_change":"2023-11-28T18:26:01.173587+0000","last_active":"2023-11-28T18:26:01.185551+0000","last_peered":"2023-11-28T18:26:01.185551+0000","last_clean":"2023-11-28T18:26:01.185551+0000","last_became_active":"2023-11-28T18:26:01.173280+0000","last_became_peered":"2023-11-28T18:26:01.173280+0000","last_unstale":"2023-11-28T18:26:01.185551+0000","last_undegraded":"2023-11-28T18:26:01.185551+0000","last_fullsized":"2023-11-28T18:26:01.185551+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:54:35.486618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.191081+0000","last_change":"2023-11-28T18:26:01.182286+0000","last_active":"2023-11-28T18:26:01.191081+0000","last_peered":"2023-11-28T18:26:01.191081+0000","last_clean":"2023-11-28T18:26:01.191081+0000","last_became_active":"2023-11-28T18:26:01.182037+0000","last_became_peered":"2023-11-28T18:26:01.182037+0000","last_unstale":"2023-11-28T18:26:01.191081+0000","last_undegraded":"2023-11-28T18:26:01.191081+0000","last_fullsized":"2023-11-28T18:26:01.191081+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:18:08.430874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.186377+0000","last_change":"2023-11-28T18:26:01.175268+0000","last_active":"2023-11-28T18:26:01.186377+0000","last_peered":"2023-11-28T18:26:01.186377+0000","last_clean":"2023-11-28T18:26:01.186377+0000","last_became_active":"2023-11-28T18:26:01.175054+0000","last_became_peered":"2023-11-28T18:26:01.175054+0000","last_unstale":"2023-11-28T18:26:01.186377+0000","last_undegraded":"2023-11-28T18:26:01.186377+0000","last_fullsized":"2023-11-28T18:26:01.186377+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:57.022136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.198824+0000","last_change":"2023-11-28T18:26:01.191960+0000","last_active":"2023-11-28T18:26:01.198824+0000","last_peered":"2023-11-28T18:26:01.198824+0000","last_clean":"2023-11-28T18:26:01.198824+0000","last_became_active":"2023-11-28T18:26:01.190212+0000","last_became_peered":"2023-11-28T18:26:01.190212+0000","last_unstale":"2023-11-28T18:26:01.198824+0000","last_undegraded":"2023-11-28T18:26:01.198824+0000","last_fullsized":"2023-11-28T18:26:01.198824+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:21:01.828174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.200696+0000","last_change":"2023-11-28T18:26:01.195584+0000","last_active":"2023-11-28T18:26:01.200696+0000","last_peered":"2023-11-28T18:26:01.200696+0000","last_clean":"2023-11-28T18:26:01.200696+0000","last_became_active":"2023-11-28T18:26:01.195320+0000","last_became_peered":"2023-11-28T18:26:01.195320+0000","last_unstale":"2023-11-28T18:26:01.200696+0000","last_undegraded":"2023-11-28T18:26:01.200696+0000","last_fullsized":"2023-11-28T18:26:01.200696+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:47.994732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.186639+0000","last_change":"2023-11-28T18:26:01.176382+0000","last_active":"2023-11-28T18:26:01.186639+0000","last_peered":"2023-11-28T18:26:01.186639+0000","last_clean":"2023-11-28T18:26:01.186639+0000","last_became_active":"2023-11-28T18:26:01.175756+0000","last_became_peered":"2023-11-28T18:26:01.175756+0000","last_unstale":"2023-11-28T18:26:01.186639+0000","last_undegraded":"2023-11-28T18:26:01.186639+0000","last_fullsized":"2023-11-28T18:26:01.186639+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:36:35.769346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.202450+0000","last_change":"2023-11-28T18:26:01.196597+0000","last_active":"2023-11-28T18:26:01.202450+0000","last_peered":"2023-11-28T18:26:01.202450+0000","last_clean":"2023-11-28T18:26:01.202450+0000","last_became_active":"2023-11-28T18:26:01.195960+0000","last_became_peered":"2023-11-28T18:26:01.195960+0000","last_unstale":"2023-11-28T18:26:01.202450+0000","last_undegraded":"2023-11-28T18:26:01.202450+0000","last_fullsized":"2023-11-28T18:26:01.202450+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:41:21.591185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.197800+0000","last_change":"2023-11-28T18:26:01.191018+0000","last_active":"2023-11-28T18:26:01.197800+0000","last_peered":"2023-11-28T18:26:01.197800+0000","last_clean":"2023-11-28T18:26:01.197800+0000","last_became_active":"2023-11-28T18:26:01.190727+0000","last_became_peered":"2023-11-28T18:26:01.190727+0000","last_unstale":"2023-11-28T18:26:01.197800+0000","last_undegraded":"2023-11-28T18:26:01.197800+0000","last_fullsized":"2023-11-28T18:26:01.197800+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:26.019007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.193369+0000","last_change":"2023-11-28T18:26:01.186839+0000","last_active":"2023-11-28T18:26:01.193369+0000","last_peered":"2023-11-28T18:26:01.193369+0000","last_clean":"2023-11-28T18:26:01.193369+0000","last_became_active":"2023-11-28T18:26:01.185197+0000","last_became_peered":"2023-11-28T18:26:01.185197+0000","last_unstale":"2023-11-28T18:26:01.193369+0000","last_undegraded":"2023-11-28T18:26:01.193369+0000","last_fullsized":"2023-11-28T18:26:01.193369+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:57.022136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.187921+0000","last_change":"2023-11-28T18:26:01.180580+0000","last_active":"2023-11-28T18:26:01.187921+0000","last_peered":"2023-11-28T18:26:01.187921+0000","last_clean":"2023-11-28T18:26:01.187921+0000","last_became_active":"2023-11-28T18:26:01.180122+0000","last_became_peered":"2023-11-28T18:26:01.180122+0000","last_unstale":"2023-11-28T18:26:01.187921+0000","last_undegraded":"2023-11-28T18:26:01.187921+0000","last_fullsized":"2023-11-28T18:26:01.187921+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:49:47.454676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.175987+0000","last_change":"2023-11-28T18:26:01.164582+0000","last_active":"2023-11-28T18:26:01.175987+0000","last_peered":"2023-11-28T18:26:01.175987+0000","last_clean":"2023-11-28T18:26:01.175987+0000","last_became_active":"2023-11-28T18:26:01.164261+0000","last_became_peered":"2023-11-28T18:26:01.164261+0000","last_unstale":"2023-11-28T18:26:01.175987+0000","last_undegraded":"2023-11-28T18:26:01.175987+0000","last_fullsized":"2023-11-28T18:26:01.175987+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:18:33.835504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.194386+0000","last_change":"2023-11-28T18:26:01.188785+0000","last_active":"2023-11-28T18:26:01.194386+0000","last_peered":"2023-11-28T18:26:01.194386+0000","last_clean":"2023-11-28T18:26:01.194386+0000","last_became_active":"2023-11-28T18:26:01.188417+0000","last_became_peered":"2023-11-28T18:26:01.188417+0000","last_unstale":"2023-11-28T18:26:01.194386+0000","last_undegraded":"2023-11-28T18:26:01.194386+0000","last_fullsized":"2023-11-28T18:26:01.194386+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:21:01.828174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.202498+0000","last_change":"2023-11-28T18:26:00.181204+0000","last_active":"2023-11-28T18:26:01.202498+0000","last_peered":"2023-11-28T18:26:01.202498+0000","last_clean":"2023-11-28T18:26:01.202498+0000","last_became_active":"2023-11-28T18:26:00.180952+0000","last_became_peered":"2023-11-28T18:26:00.180952+0000","last_unstale":"2023-11-28T18:26:01.202498+0000","last_undegraded":"2023-11-28T18:26:01.202498+0000","last_fullsized":"2023-11-28T18:26:01.202498+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:07:33.583566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.182486+0000","last_change":"2023-11-28T18:26:01.178676+0000","last_active":"2023-11-28T18:26:01.182486+0000","last_peered":"2023-11-28T18:26:01.182486+0000","last_clean":"2023-11-28T18:26:01.182486+0000","last_became_active":"2023-11-28T18:26:01.178355+0000","last_became_peered":"2023-11-28T18:26:01.178355+0000","last_unstale":"2023-11-28T18:26:01.182486+0000","last_undegraded":"2023-11-28T18:26:01.182486+0000","last_fullsized":"2023-11-28T18:26:01.182486+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:26.019007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.193126+0000","last_change":"2023-11-28T18:26:00.193130+0000","last_active":"2023-11-28T18:26:01.193126+0000","last_peered":"2023-11-28T18:26:01.193126+0000","last_clean":"2023-11-28T18:26:01.193126+0000","last_became_active":"2023-11-28T18:26:00.191726+0000","last_became_peered":"2023-11-28T18:26:00.191726+0000","last_unstale":"2023-11-28T18:26:01.193126+0000","last_undegraded":"2023-11-28T18:26:01.193126+0000","last_fullsized":"2023-11-28T18:26:01.193126+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:17.654673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.202345+0000","last_change":"2023-11-28T18:26:01.195855+0000","last_active":"2023-11-28T18:26:01.202345+0000","last_peered":"2023-11-28T18:26:01.202345+0000","last_clean":"2023-11-28T18:26:01.202345+0000","last_became_active":"2023-11-28T18:26:01.195573+0000","last_became_peered":"2023-11-28T18:26:01.195573+0000","last_unstale":"2023-11-28T18:26:01.202345+0000","last_undegraded":"2023-11-28T18:26:01.202345+0000","last_fullsized":"2023-11-28T18:26:01.202345+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:45:27.560489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.194152+0000","last_change":"2023-11-28T18:26:00.184884+0000","last_active":"2023-11-28T18:26:01.194152+0000","last_peered":"2023-11-28T18:26:01.194152+0000","last_clean":"2023-11-28T18:26:01.194152+0000","last_became_active":"2023-11-28T18:26:00.184589+0000","last_became_peered":"2023-11-28T18:26:00.184589+0000","last_unstale":"2023-11-28T18:26:01.194152+0000","last_undegraded":"2023-11-28T18:26:01.194152+0000","last_fullsized":"2023-11-28T18:26:01.194152+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:00:42.591355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.185686+0000","last_change":"2023-11-28T18:26:01.182012+0000","last_active":"2023-11-28T18:26:01.185686+0000","last_peered":"2023-11-28T18:26:01.185686+0000","last_clean":"2023-11-28T18:26:01.185686+0000","last_became_active":"2023-11-28T18:26:01.178102+0000","last_became_peered":"2023-11-28T18:26:01.178102+0000","last_unstale":"2023-11-28T18:26:01.185686+0000","last_undegraded":"2023-11-28T18:26:01.185686+0000","last_fullsized":"2023-11-28T18:26:01.185686+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:57.022136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.198971+0000","last_change":"2023-11-28T18:26:00.183924+0000","last_active":"2023-11-28T18:26:01.198971+0000","last_peered":"2023-11-28T18:26:01.198971+0000","last_clean":"2023-11-28T18:26:01.198971+0000","last_became_active":"2023-11-28T18:26:00.183597+0000","last_became_peered":"2023-11-28T18:26:00.183597+0000","last_unstale":"2023-11-28T18:26:01.198971+0000","last_undegraded":"2023-11-28T18:26:01.198971+0000","last_fullsized":"2023-11-28T18:26:01.198971+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:53:24.485251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.188842+0000","last_change":"2023-11-28T18:26:01.177692+0000","last_active":"2023-11-28T18:26:01.188842+0000","last_peered":"2023-11-28T18:26:01.188842+0000","last_clean":"2023-11-28T18:26:01.188842+0000","last_became_active":"2023-11-28T18:26:01.175183+0000","last_became_peered":"2023-11-28T18:26:01.175183+0000","last_unstale":"2023-11-28T18:26:01.188842+0000","last_undegraded":"2023-11-28T18:26:01.188842+0000","last_fullsized":"2023-11-28T18:26:01.188842+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:18.657471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.195283+0000","last_change":"2023-11-28T18:26:00.180361+0000","last_active":"2023-11-28T18:26:01.195283+0000","last_peered":"2023-11-28T18:26:01.195283+0000","last_clean":"2023-11-28T18:26:01.195283+0000","last_became_active":"2023-11-28T18:26:00.180083+0000","last_became_peered":"2023-11-28T18:26:00.180083+0000","last_unstale":"2023-11-28T18:26:01.195283+0000","last_undegraded":"2023-11-28T18:26:01.195283+0000","last_fullsized":"2023-11-28T18:26:01.195283+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:17:34.388624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.194334+0000","last_change":"2023-11-28T18:26:01.190569+0000","last_active":"2023-11-28T18:26:01.194334+0000","last_peered":"2023-11-28T18:26:01.194334+0000","last_clean":"2023-11-28T18:26:01.194334+0000","last_became_active":"2023-11-28T18:26:01.189932+0000","last_became_peered":"2023-11-28T18:26:01.189932+0000","last_unstale":"2023-11-28T18:26:01.194334+0000","last_undegraded":"2023-11-28T18:26:01.194334+0000","last_fullsized":"2023-11-28T18:26:01.194334+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:12:59.466374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.195621+0000","last_change":"2023-11-28T18:26:00.179098+0000","last_active":"2023-11-28T18:26:01.195621+0000","last_peered":"2023-11-28T18:26:01.195621+0000","last_clean":"2023-11-28T18:26:01.195621+0000","last_became_active":"2023-11-28T18:26:00.178746+0000","last_became_peered":"2023-11-28T18:26:00.178746+0000","last_unstale":"2023-11-28T18:26:01.195621+0000","last_undegraded":"2023-11-28T18:26:01.195621+0000","last_fullsized":"2023-11-28T18:26:01.195621+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:36:34.766548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.200717+0000","last_change":"2023-11-28T18:26:01.195661+0000","last_active":"2023-11-28T18:26:01.200717+0000","last_peered":"2023-11-28T18:26:01.200717+0000","last_clean":"2023-11-28T18:26:01.200717+0000","last_became_active":"2023-11-28T18:26:01.195412+0000","last_became_peered":"2023-11-28T18:26:01.195412+0000","last_unstale":"2023-11-28T18:26:01.200717+0000","last_undegraded":"2023-11-28T18:26:01.200717+0000","last_fullsized":"2023-11-28T18:26:01.200717+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:16.495437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.188479+0000","last_change":"2023-11-28T18:26:00.180447+0000","last_active":"2023-11-28T18:26:01.188479+0000","last_peered":"2023-11-28T18:26:01.188479+0000","last_clean":"2023-11-28T18:26:01.188479+0000","last_became_active":"2023-11-28T18:26:00.180078+0000","last_became_peered":"2023-11-28T18:26:00.180078+0000","last_unstale":"2023-11-28T18:26:01.188479+0000","last_undegraded":"2023-11-28T18:26:01.188479+0000","last_fullsized":"2023-11-28T18:26:01.188479+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:21.581173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.180047+0000","last_change":"2023-11-28T18:26:01.176283+0000","last_active":"2023-11-28T18:26:01.180047+0000","last_peered":"2023-11-28T18:26:01.180047+0000","last_clean":"2023-11-28T18:26:01.180047+0000","last_became_active":"2023-11-28T18:26:01.176005+0000","last_became_peered":"2023-11-28T18:26:01.176005+0000","last_unstale":"2023-11-28T18:26:01.180047+0000","last_undegraded":"2023-11-28T18:26:01.180047+0000","last_fullsized":"2023-11-28T18:26:01.180047+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:49:47.454676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.166373+0000","last_change":"2023-11-28T18:26:00.179112+0000","last_active":"2023-11-28T18:26:01.166373+0000","last_peered":"2023-11-28T18:26:01.166373+0000","last_clean":"2023-11-28T18:26:01.166373+0000","last_became_active":"2023-11-28T18:26:00.178840+0000","last_became_peered":"2023-11-28T18:26:00.178840+0000","last_unstale":"2023-11-28T18:26:01.166373+0000","last_undegraded":"2023-11-28T18:26:01.166373+0000","last_fullsized":"2023-11-28T18:26:01.166373+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:15.492639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.183005+0000","last_change":"2023-11-28T18:26:01.164201+0000","last_active":"2023-11-28T18:26:01.183005+0000","last_peered":"2023-11-28T18:26:01.183005+0000","last_clean":"2023-11-28T18:26:01.183005+0000","last_became_active":"2023-11-28T18:26:01.163872+0000","last_became_peered":"2023-11-28T18:26:01.163872+0000","last_unstale":"2023-11-28T18:26:01.183005+0000","last_undegraded":"2023-11-28T18:26:01.183005+0000","last_fullsized":"2023-11-28T18:26:01.183005+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:39:28.226366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.166765+0000","last_change":"2023-11-28T18:26:00.185129+0000","last_active":"2023-11-28T18:26:01.166765+0000","last_peered":"2023-11-28T18:26:01.166765+0000","last_clean":"2023-11-28T18:26:01.166765+0000","last_became_active":"2023-11-28T18:26:00.184847+0000","last_became_peered":"2023-11-28T18:26:00.184847+0000","last_unstale":"2023-11-28T18:26:01.166765+0000","last_undegraded":"2023-11-28T18:26:01.166765+0000","last_fullsized":"2023-11-28T18:26:01.166765+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:04:01.913690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.189101+0000","last_change":"2023-11-28T18:26:01.183045+0000","last_active":"2023-11-28T18:26:01.189101+0000","last_peered":"2023-11-28T18:26:01.189101+0000","last_clean":"2023-11-28T18:26:01.189101+0000","last_became_active":"2023-11-28T18:26:01.182734+0000","last_became_peered":"2023-11-28T18:26:01.182734+0000","last_unstale":"2023-11-28T18:26:01.189101+0000","last_undegraded":"2023-11-28T18:26:01.189101+0000","last_fullsized":"2023-11-28T18:26:01.189101+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:06.029532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.161778+0000","last_change":"2023-11-28T18:26:00.179088+0000","last_active":"2023-11-28T18:26:01.161778+0000","last_peered":"2023-11-28T18:26:01.161778+0000","last_clean":"2023-11-28T18:26:01.161778+0000","last_became_active":"2023-11-28T18:26:00.177867+0000","last_became_peered":"2023-11-28T18:26:00.177867+0000","last_unstale":"2023-11-28T18:26:01.161778+0000","last_undegraded":"2023-11-28T18:26:01.161778+0000","last_fullsized":"2023-11-28T18:26:01.161778+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:58.106337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.198161+0000","last_change":"2023-11-28T18:26:01.191290+0000","last_active":"2023-11-28T18:26:01.198161+0000","last_peered":"2023-11-28T18:26:01.198161+0000","last_clean":"2023-11-28T18:26:01.198161+0000","last_became_active":"2023-11-28T18:26:01.190636+0000","last_became_peered":"2023-11-28T18:26:01.190636+0000","last_unstale":"2023-11-28T18:26:01.198161+0000","last_undegraded":"2023-11-28T18:26:01.198161+0000","last_fullsized":"2023-11-28T18:26:01.198161+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:12:59.466374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.190433+0000","last_change":"2023-11-28T18:26:00.194212+0000","last_active":"2023-11-28T18:26:01.190433+0000","last_peered":"2023-11-28T18:26:01.190433+0000","last_clean":"2023-11-28T18:26:01.190433+0000","last_became_active":"2023-11-28T18:26:00.193908+0000","last_became_peered":"2023-11-28T18:26:00.193908+0000","last_unstale":"2023-11-28T18:26:01.190433+0000","last_undegraded":"2023-11-28T18:26:01.190433+0000","last_fullsized":"2023-11-28T18:26:01.190433+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:10.195481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.188888+0000","last_change":"2023-11-28T18:26:01.177990+0000","last_active":"2023-11-28T18:26:01.188888+0000","last_peered":"2023-11-28T18:26:01.188888+0000","last_clean":"2023-11-28T18:26:01.188888+0000","last_became_active":"2023-11-28T18:26:01.176061+0000","last_became_peered":"2023-11-28T18:26:01.176061+0000","last_unstale":"2023-11-28T18:26:01.188888+0000","last_undegraded":"2023-11-28T18:26:01.188888+0000","last_fullsized":"2023-11-28T18:26:01.188888+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:22.583971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.204635+0000","last_change":"2023-11-28T18:26:00.187840+0000","last_active":"2023-11-28T18:26:01.204635+0000","last_peered":"2023-11-28T18:26:01.204635+0000","last_clean":"2023-11-28T18:26:01.204635+0000","last_became_active":"2023-11-28T18:26:00.187581+0000","last_became_peered":"2023-11-28T18:26:00.187581+0000","last_unstale":"2023-11-28T18:26:01.204635+0000","last_undegraded":"2023-11-28T18:26:01.204635+0000","last_fullsized":"2023-11-28T18:26:01.204635+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:32:55.830726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.184418+0000","last_change":"2023-11-28T18:26:01.179326+0000","last_active":"2023-11-28T18:26:01.184418+0000","last_peered":"2023-11-28T18:26:01.184418+0000","last_clean":"2023-11-28T18:26:01.184418+0000","last_became_active":"2023-11-28T18:26:01.179054+0000","last_became_peered":"2023-11-28T18:26:01.179054+0000","last_unstale":"2023-11-28T18:26:01.184418+0000","last_undegraded":"2023-11-28T18:26:01.184418+0000","last_fullsized":"2023-11-28T18:26:01.184418+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:21:01.828174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.192113+0000","last_change":"2023-11-28T18:26:00.177241+0000","last_active":"2023-11-28T18:26:01.192113+0000","last_peered":"2023-11-28T18:26:01.192113+0000","last_clean":"2023-11-28T18:26:01.192113+0000","last_became_active":"2023-11-28T18:26:00.176494+0000","last_became_peered":"2023-11-28T18:26:00.176494+0000","last_unstale":"2023-11-28T18:26:01.192113+0000","last_undegraded":"2023-11-28T18:26:01.192113+0000","last_fullsized":"2023-11-28T18:26:01.192113+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:32:55.830726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.188371+0000","last_change":"2023-11-28T18:26:01.176737+0000","last_active":"2023-11-28T18:26:01.188371+0000","last_peered":"2023-11-28T18:26:01.188371+0000","last_clean":"2023-11-28T18:26:01.188371+0000","last_became_active":"2023-11-28T18:26:01.176443+0000","last_became_peered":"2023-11-28T18:26:01.176443+0000","last_unstale":"2023-11-28T18:26:01.188371+0000","last_undegraded":"2023-11-28T18:26:01.188371+0000","last_fullsized":"2023-11-28T18:26:01.188371+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:36.399960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.189660+0000","last_change":"2023-11-28T18:26:00.181777+0000","last_active":"2023-11-28T18:26:01.189660+0000","last_peered":"2023-11-28T18:26:01.189660+0000","last_clean":"2023-11-28T18:26:01.189660+0000","last_became_active":"2023-11-28T18:26:00.181502+0000","last_became_peered":"2023-11-28T18:26:00.181502+0000","last_unstale":"2023-11-28T18:26:01.189660+0000","last_undegraded":"2023-11-28T18:26:01.189660+0000","last_fullsized":"2023-11-28T18:26:01.189660+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:36:34.766548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.189156+0000","last_change":"2023-11-28T18:26:01.185400+0000","last_active":"2023-11-28T18:26:01.189156+0000","last_peered":"2023-11-28T18:26:01.189156+0000","last_clean":"2023-11-28T18:26:01.189156+0000","last_became_active":"2023-11-28T18:26:01.181587+0000","last_became_peered":"2023-11-28T18:26:01.181587+0000","last_unstale":"2023-11-28T18:26:01.189156+0000","last_undegraded":"2023-11-28T18:26:01.189156+0000","last_fullsized":"2023-11-28T18:26:01.189156+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:47.363263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.168685+0000","last_change":"2023-11-28T18:26:00.187600+0000","last_active":"2023-11-28T18:26:01.168685+0000","last_peered":"2023-11-28T18:26:01.168685+0000","last_clean":"2023-11-28T18:26:01.168685+0000","last_became_active":"2023-11-28T18:26:00.187309+0000","last_became_peered":"2023-11-28T18:26:00.187309+0000","last_unstale":"2023-11-28T18:26:01.168685+0000","last_undegraded":"2023-11-28T18:26:01.168685+0000","last_fullsized":"2023-11-28T18:26:01.168685+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:21:17.168147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.182241+0000","last_change":"2023-11-28T18:26:01.172719+0000","last_active":"2023-11-28T18:26:01.182241+0000","last_peered":"2023-11-28T18:26:01.182241+0000","last_clean":"2023-11-28T18:26:01.182241+0000","last_became_active":"2023-11-28T18:26:01.172390+0000","last_became_peered":"2023-11-28T18:26:01.172390+0000","last_unstale":"2023-11-28T18:26:01.182241+0000","last_undegraded":"2023-11-28T18:26:01.182241+0000","last_fullsized":"2023-11-28T18:26:01.182241+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:21:18.170945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.196458+0000","last_change":"2023-11-28T18:26:00.174487+0000","last_active":"2023-11-28T18:26:01.196458+0000","last_peered":"2023-11-28T18:26:01.196458+0000","last_clean":"2023-11-28T18:26:01.196458+0000","last_became_active":"2023-11-28T18:26:00.170686+0000","last_became_peered":"2023-11-28T18:26:00.170686+0000","last_unstale":"2023-11-28T18:26:01.196458+0000","last_undegraded":"2023-11-28T18:26:01.196458+0000","last_fullsized":"2023-11-28T18:26:01.196458+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:05.026734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.174202+0000","last_change":"2023-11-28T18:26:01.166308+0000","last_active":"2023-11-28T18:26:01.174202+0000","last_peered":"2023-11-28T18:26:01.174202+0000","last_clean":"2023-11-28T18:26:01.174202+0000","last_became_active":"2023-11-28T18:26:01.165886+0000","last_became_peered":"2023-11-28T18:26:01.165886+0000","last_unstale":"2023-11-28T18:26:01.174202+0000","last_undegraded":"2023-11-28T18:26:01.174202+0000","last_fullsized":"2023-11-28T18:26:01.174202+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:18:33.835504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.190612+0000","last_change":"2023-11-28T18:26:00.180415+0000","last_active":"2023-11-28T18:26:01.190612+0000","last_peered":"2023-11-28T18:26:01.190612+0000","last_clean":"2023-11-28T18:26:01.190612+0000","last_became_active":"2023-11-28T18:26:00.180092+0000","last_became_peered":"2023-11-28T18:26:00.180092+0000","last_unstale":"2023-11-28T18:26:01.190612+0000","last_undegraded":"2023-11-28T18:26:01.190612+0000","last_fullsized":"2023-11-28T18:26:01.190612+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:06:35.665714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.187895+0000","last_change":"2023-11-28T18:26:01.177655+0000","last_active":"2023-11-28T18:26:01.187895+0000","last_peered":"2023-11-28T18:26:01.187895+0000","last_clean":"2023-11-28T18:26:01.187895+0000","last_became_active":"2023-11-28T18:26:01.177382+0000","last_became_peered":"2023-11-28T18:26:01.177382+0000","last_unstale":"2023-11-28T18:26:01.187895+0000","last_undegraded":"2023-11-28T18:26:01.187895+0000","last_fullsized":"2023-11-28T18:26:01.187895+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:07:34.586364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.190771+0000","last_change":"2023-11-28T18:26:00.176652+0000","last_active":"2023-11-28T18:26:01.190771+0000","last_peered":"2023-11-28T18:26:01.190771+0000","last_clean":"2023-11-28T18:26:01.190771+0000","last_became_active":"2023-11-28T18:26:00.176321+0000","last_became_peered":"2023-11-28T18:26:00.176321+0000","last_unstale":"2023-11-28T18:26:01.190771+0000","last_undegraded":"2023-11-28T18:26:01.190771+0000","last_fullsized":"2023-11-28T18:26:01.190771+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:48:21.783489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.184378+0000","last_change":"2023-11-28T18:26:01.171631+0000","last_active":"2023-11-28T18:26:01.184378+0000","last_peered":"2023-11-28T18:26:01.184378+0000","last_clean":"2023-11-28T18:26:01.184378+0000","last_became_active":"2023-11-28T18:26:01.171306+0000","last_became_peered":"2023-11-28T18:26:01.171306+0000","last_unstale":"2023-11-28T18:26:01.184378+0000","last_undegraded":"2023-11-28T18:26:01.184378+0000","last_fullsized":"2023-11-28T18:26:01.184378+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:35:48.822843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.199344+0000","last_change":"2023-11-28T18:26:00.186268+0000","last_active":"2023-11-28T18:26:01.199344+0000","last_peered":"2023-11-28T18:26:01.199344+0000","last_clean":"2023-11-28T18:26:01.199344+0000","last_became_active":"2023-11-28T18:26:00.185914+0000","last_became_peered":"2023-11-28T18:26:00.185914+0000","last_unstale":"2023-11-28T18:26:01.199344+0000","last_undegraded":"2023-11-28T18:26:01.199344+0000","last_fullsized":"2023-11-28T18:26:01.199344+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:17.654673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.194552+0000","last_change":"2023-11-28T18:26:01.188915+0000","last_active":"2023-11-28T18:26:01.194552+0000","last_peered":"2023-11-28T18:26:01.194552+0000","last_clean":"2023-11-28T18:26:01.194552+0000","last_became_active":"2023-11-28T18:26:01.187374+0000","last_became_peered":"2023-11-28T18:26:01.187374+0000","last_unstale":"2023-11-28T18:26:01.194552+0000","last_undegraded":"2023-11-28T18:26:01.194552+0000","last_fullsized":"2023-11-28T18:26:01.194552+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:22.583971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.189750+0000","last_change":"2023-11-28T18:26:00.185051+0000","last_active":"2023-11-28T18:26:01.189750+0000","last_peered":"2023-11-28T18:26:01.189750+0000","last_clean":"2023-11-28T18:26:01.189750+0000","last_became_active":"2023-11-28T18:26:00.184343+0000","last_became_peered":"2023-11-28T18:26:00.184343+0000","last_unstale":"2023-11-28T18:26:01.189750+0000","last_undegraded":"2023-11-28T18:26:01.189750+0000","last_fullsized":"2023-11-28T18:26:01.189750+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:00:42.591355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.186480+0000","last_change":"2023-11-28T18:26:01.182289+0000","last_active":"2023-11-28T18:26:01.186480+0000","last_peered":"2023-11-28T18:26:01.186480+0000","last_clean":"2023-11-28T18:26:01.186480+0000","last_became_active":"2023-11-28T18:26:01.181963+0000","last_became_peered":"2023-11-28T18:26:01.181963+0000","last_unstale":"2023-11-28T18:26:01.186480+0000","last_undegraded":"2023-11-28T18:26:01.186480+0000","last_fullsized":"2023-11-28T18:26:01.186480+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:54:35.486618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.198026+0000","last_change":"2023-11-28T18:26:00.191584+0000","last_active":"2023-11-28T18:26:01.198026+0000","last_peered":"2023-11-28T18:26:01.198026+0000","last_clean":"2023-11-28T18:26:01.198026+0000","last_became_active":"2023-11-28T18:26:00.191070+0000","last_became_peered":"2023-11-28T18:26:00.191070+0000","last_unstale":"2023-11-28T18:26:01.198026+0000","last_undegraded":"2023-11-28T18:26:01.198026+0000","last_fullsized":"2023-11-28T18:26:01.198026+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:49:46.451878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.202325+0000","last_change":"2023-11-28T18:26:01.195765+0000","last_active":"2023-11-28T18:26:01.202325+0000","last_peered":"2023-11-28T18:26:01.202325+0000","last_clean":"2023-11-28T18:26:01.202325+0000","last_became_active":"2023-11-28T18:26:01.195459+0000","last_became_peered":"2023-11-28T18:26:01.195459+0000","last_unstale":"2023-11-28T18:26:01.202325+0000","last_undegraded":"2023-11-28T18:26:01.202325+0000","last_fullsized":"2023-11-28T18:26:01.202325+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:29:19.810995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.156000+0000","last_change":"2023-11-28T18:26:00.178641+0000","last_active":"2023-11-28T18:26:01.156000+0000","last_peered":"2023-11-28T18:26:01.156000+0000","last_clean":"2023-11-28T18:26:01.156000+0000","last_became_active":"2023-11-28T18:26:00.178393+0000","last_became_peered":"2023-11-28T18:26:00.178393+0000","last_unstale":"2023-11-28T18:26:01.156000+0000","last_undegraded":"2023-11-28T18:26:01.156000+0000","last_fullsized":"2023-11-28T18:26:01.156000+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:10.195481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.198733+0000","last_change":"2023-11-28T18:26:01.191534+0000","last_active":"2023-11-28T18:26:01.198733+0000","last_peered":"2023-11-28T18:26:01.198733+0000","last_clean":"2023-11-28T18:26:01.198733+0000","last_became_active":"2023-11-28T18:26:01.191235+0000","last_became_peered":"2023-11-28T18:26:01.191235+0000","last_unstale":"2023-11-28T18:26:01.198733+0000","last_undegraded":"2023-11-28T18:26:01.198733+0000","last_fullsized":"2023-11-28T18:26:01.198733+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:11.198280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.165796+0000","last_change":"2023-11-28T18:26:00.194687+0000","last_active":"2023-11-28T18:26:01.165796+0000","last_peered":"2023-11-28T18:26:01.165796+0000","last_clean":"2023-11-28T18:26:01.165796+0000","last_became_active":"2023-11-28T18:26:00.193360+0000","last_became_peered":"2023-11-28T18:26:00.193360+0000","last_unstale":"2023-11-28T18:26:01.165796+0000","last_undegraded":"2023-11-28T18:26:01.165796+0000","last_fullsized":"2023-11-28T18:26:01.165796+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:46.991934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.196949+0000","last_change":"2023-11-28T18:26:01.191891+0000","last_active":"2023-11-28T18:26:01.196949+0000","last_peered":"2023-11-28T18:26:01.196949+0000","last_clean":"2023-11-28T18:26:01.196949+0000","last_became_active":"2023-11-28T18:26:01.191637+0000","last_became_peered":"2023-11-28T18:26:01.191637+0000","last_unstale":"2023-11-28T18:26:01.196949+0000","last_undegraded":"2023-11-28T18:26:01.196949+0000","last_fullsized":"2023-11-28T18:26:01.196949+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:59.109135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.167632+0000","last_change":"2023-11-28T18:26:00.180240+0000","last_active":"2023-11-28T18:26:01.167632+0000","last_peered":"2023-11-28T18:26:01.167632+0000","last_clean":"2023-11-28T18:26:01.167632+0000","last_became_active":"2023-11-28T18:26:00.179986+0000","last_became_peered":"2023-11-28T18:26:00.179986+0000","last_unstale":"2023-11-28T18:26:01.167632+0000","last_undegraded":"2023-11-28T18:26:01.167632+0000","last_fullsized":"2023-11-28T18:26:01.167632+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:08:58.232922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.175966+0000","last_change":"2023-11-28T18:26:01.164428+0000","last_active":"2023-11-28T18:26:01.175966+0000","last_peered":"2023-11-28T18:26:01.175966+0000","last_clean":"2023-11-28T18:26:01.175966+0000","last_became_active":"2023-11-28T18:26:01.164064+0000","last_became_peered":"2023-11-28T18:26:01.164064+0000","last_unstale":"2023-11-28T18:26:01.175966+0000","last_undegraded":"2023-11-28T18:26:01.175966+0000","last_fullsized":"2023-11-28T18:26:01.175966+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:11.198280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.199731+0000","last_change":"2023-11-28T18:26:00.181694+0000","last_active":"2023-11-28T18:26:01.199731+0000","last_peered":"2023-11-28T18:26:01.199731+0000","last_clean":"2023-11-28T18:26:01.199731+0000","last_became_active":"2023-11-28T18:26:00.181397+0000","last_became_peered":"2023-11-28T18:26:00.181397+0000","last_unstale":"2023-11-28T18:26:01.199731+0000","last_undegraded":"2023-11-28T18:26:01.199731+0000","last_fullsized":"2023-11-28T18:26:01.199731+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:39:27.223567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.199567+0000","last_change":"2023-11-28T18:26:01.195945+0000","last_active":"2023-11-28T18:26:01.199567+0000","last_peered":"2023-11-28T18:26:01.199567+0000","last_clean":"2023-11-28T18:26:01.199567+0000","last_became_active":"2023-11-28T18:26:01.195698+0000","last_became_peered":"2023-11-28T18:26:01.195698+0000","last_unstale":"2023-11-28T18:26:01.199567+0000","last_undegraded":"2023-11-28T18:26:01.199567+0000","last_fullsized":"2023-11-28T18:26:01.199567+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:35:48.822843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.167861+0000","last_change":"2023-11-28T18:26:00.186551+0000","last_active":"2023-11-28T18:26:01.167861+0000","last_peered":"2023-11-28T18:26:01.167861+0000","last_clean":"2023-11-28T18:26:01.167861+0000","last_became_active":"2023-11-28T18:26:00.186199+0000","last_became_peered":"2023-11-28T18:26:00.186199+0000","last_unstale":"2023-11-28T18:26:01.167861+0000","last_undegraded":"2023-11-28T18:26:01.167861+0000","last_fullsized":"2023-11-28T18:26:01.167861+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:54:34.483820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.174163+0000","last_change":"2023-11-28T18:26:01.164790+0000","last_active":"2023-11-28T18:26:01.174163+0000","last_peered":"2023-11-28T18:26:01.174163+0000","last_clean":"2023-11-28T18:26:01.174163+0000","last_became_active":"2023-11-28T18:26:01.164511+0000","last_became_peered":"2023-11-28T18:26:01.164511+0000","last_unstale":"2023-11-28T18:26:01.174163+0000","last_undegraded":"2023-11-28T18:26:01.174163+0000","last_fullsized":"2023-11-28T18:26:01.174163+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:21:18.170945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.166100+0000","last_change":"2023-11-28T18:26:00.177595+0000","last_active":"2023-11-28T18:26:01.166100+0000","last_peered":"2023-11-28T18:26:01.166100+0000","last_clean":"2023-11-28T18:26:01.166100+0000","last_became_active":"2023-11-28T18:26:00.177325+0000","last_became_peered":"2023-11-28T18:26:00.177325+0000","last_unstale":"2023-11-28T18:26:01.166100+0000","last_undegraded":"2023-11-28T18:26:01.166100+0000","last_fullsized":"2023-11-28T18:26:01.166100+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:46.360465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.198682+0000","last_change":"2023-11-28T18:26:01.191424+0000","last_active":"2023-11-28T18:26:01.198682+0000","last_peered":"2023-11-28T18:26:01.198682+0000","last_clean":"2023-11-28T18:26:01.198682+0000","last_became_active":"2023-11-28T18:26:01.190341+0000","last_became_peered":"2023-11-28T18:26:01.190341+0000","last_unstale":"2023-11-28T18:26:01.198682+0000","last_undegraded":"2023-11-28T18:26:01.198682+0000","last_fullsized":"2023-11-28T18:26:01.198682+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:04:02.916488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.156608+0000","last_change":"2023-11-28T18:26:00.183062+0000","last_active":"2023-11-28T18:26:01.156608+0000","last_peered":"2023-11-28T18:26:01.156608+0000","last_clean":"2023-11-28T18:26:01.156608+0000","last_became_active":"2023-11-28T18:26:00.182811+0000","last_became_peered":"2023-11-28T18:26:00.182811+0000","last_unstale":"2023-11-28T18:26:01.156608+0000","last_undegraded":"2023-11-28T18:26:01.156608+0000","last_fullsized":"2023-11-28T18:26:01.156608+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:25.016209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.183476+0000","last_change":"2023-11-28T18:26:01.178630+0000","last_active":"2023-11-28T18:26:01.183476+0000","last_peered":"2023-11-28T18:26:01.183476+0000","last_clean":"2023-11-28T18:26:01.183476+0000","last_became_active":"2023-11-28T18:26:01.178336+0000","last_became_peered":"2023-11-28T18:26:01.178336+0000","last_unstale":"2023-11-28T18:26:01.183476+0000","last_undegraded":"2023-11-28T18:26:01.183476+0000","last_fullsized":"2023-11-28T18:26:01.183476+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:57.022136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.156130+0000","last_change":"2023-11-28T18:26:00.192594+0000","last_active":"2023-11-28T18:26:01.156130+0000","last_peered":"2023-11-28T18:26:01.156130+0000","last_clean":"2023-11-28T18:26:01.156130+0000","last_became_active":"2023-11-28T18:26:00.192322+0000","last_became_peered":"2023-11-28T18:26:00.192322+0000","last_unstale":"2023-11-28T18:26:01.156130+0000","last_undegraded":"2023-11-28T18:26:01.156130+0000","last_fullsized":"2023-11-28T18:26:01.156130+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:53:24.485251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.197905+0000","last_change":"2023-11-28T18:26:01.190742+0000","last_active":"2023-11-28T18:26:01.197905+0000","last_peered":"2023-11-28T18:26:01.197905+0000","last_clean":"2023-11-28T18:26:01.197905+0000","last_became_active":"2023-11-28T18:26:01.190454+0000","last_became_peered":"2023-11-28T18:26:01.190454+0000","last_unstale":"2023-11-28T18:26:01.197905+0000","last_undegraded":"2023-11-28T18:26:01.197905+0000","last_fullsized":"2023-11-28T18:26:01.197905+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:53:25.488049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.160504+0000","last_change":"2023-11-28T18:26:00.185271+0000","last_active":"2023-11-28T18:26:01.160504+0000","last_peered":"2023-11-28T18:26:01.160504+0000","last_clean":"2023-11-28T18:26:01.160504+0000","last_became_active":"2023-11-28T18:26:00.184993+0000","last_became_peered":"2023-11-28T18:26:00.184993+0000","last_unstale":"2023-11-28T18:26:01.160504+0000","last_undegraded":"2023-11-28T18:26:01.160504+0000","last_fullsized":"2023-11-28T18:26:01.160504+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:07:33.583566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.190958+0000","last_change":"2023-11-28T18:26:01.186582+0000","last_active":"2023-11-28T18:26:01.190958+0000","last_peered":"2023-11-28T18:26:01.190958+0000","last_clean":"2023-11-28T18:26:01.190958+0000","last_became_active":"2023-11-28T18:26:01.186329+0000","last_became_peered":"2023-11-28T18:26:01.186329+0000","last_unstale":"2023-11-28T18:26:01.190958+0000","last_undegraded":"2023-11-28T18:26:01.190958+0000","last_fullsized":"2023-11-28T18:26:01.190958+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:22:53.430433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.161028+0000","last_change":"2023-11-28T18:26:00.193648+0000","last_active":"2023-11-28T18:26:01.161028+0000","last_peered":"2023-11-28T18:26:01.161028+0000","last_clean":"2023-11-28T18:26:01.161028+0000","last_became_active":"2023-11-28T18:26:00.193395+0000","last_became_peered":"2023-11-28T18:26:00.193395+0000","last_unstale":"2023-11-28T18:26:01.161028+0000","last_undegraded":"2023-11-28T18:26:01.161028+0000","last_fullsized":"2023-11-28T18:26:01.161028+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:21:17.168147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.202439+0000","last_change":"2023-11-28T18:26:01.196447+0000","last_active":"2023-11-28T18:26:01.202439+0000","last_peered":"2023-11-28T18:26:01.202439+0000","last_clean":"2023-11-28T18:26:01.202439+0000","last_became_active":"2023-11-28T18:26:01.196190+0000","last_became_peered":"2023-11-28T18:26:01.196190+0000","last_unstale":"2023-11-28T18:26:01.202439+0000","last_undegraded":"2023-11-28T18:26:01.202439+0000","last_fullsized":"2023-11-28T18:26:01.202439+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:26:00.149063+0000","last_clean_scrub_stamp":"2023-11-28T18:26:00.149063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:35:48.822843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T18:26:01.165481+0000","last_change":"2023-11-28T18:26:00.176639+0000","last_active":"2023-11-28T18:26:01.165481+0000","last_peered":"2023-11-28T18:26:01.165481+0000","last_clean":"2023-11-28T18:26:01.165481+0000","last_became_active":"2023-11-28T18:26:00.176259+0000","last_became_peered":"2023-11-28T18:26:00.176259+0000","last_unstale":"2023-11-28T18:26:01.165481+0000","last_undegraded":"2023-11-28T18:26:01.165481+0000","last_fullsized":"2023-11-28T18:26:01.165481+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:25:59.146265+0000","last_clean_scrub_stamp":"2023-11-28T18:25:59.146265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:18:00.940327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T18:26:00.174525+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:26:00.174525+0000","last_peered":"2023-11-28T18:26:00.174525+0000","last_clean":"2023-11-28T18:26:00.174525+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:26:00.174525+0000","last_undegraded":"2023-11-28T18:26:00.174525+0000","last_fullsized":"2023-11-28T18:26:00.174525+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36492,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335348,"statfs":{"total":96636764160,"available":96599396352,"internally_reserved":0,"allocated":1175552,"data_stored":845817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.71699999999999997,"15min":0.73799999999999999},"min":{"1min":0.44500000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.77300000000000002,"5min":2.427,"15min":2.427},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.71599999999999997,"15min":0.73799999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77700000000000002,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.53400000000000003}]},{"osd":1,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.71199999999999997,"15min":0.73199999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.873,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.755,"15min":0.77400000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88600000000000001,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.621}]},{"osd":2,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.77500000000000002,"15min":0.79400000000000004},"min":{"1min":0.501,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.82699999999999996,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.73999999999999999,"15min":0.75900000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.83499999999999996,"5min":2.411,"15min":2.411},"last":0.5}]},{"osd":3,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.76300000000000001,"15min":0.78300000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.81599999999999995,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.80600000000000005,"15min":0.82299999999999995},"min":{"1min":0.54700000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.89100000000000001,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":0.70199999999999996}]},{"osd":4,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86099999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.64200000000000002,"15min":0.65400000000000003},"min":{"1min":0.39900000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.80600000000000005,"5min":0.877,"15min":0.877},"last":0.72299999999999998}]},{"osd":5,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68200000000000005,"15min":0.68799999999999994},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91700000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.70499999999999996,"15min":0.71499999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92300000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63500000000000001}]},{"osd":6,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.74399999999999999,"15min":0.755},"min":{"1min":0.48499999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.96499999999999997,"5min":1.042,"15min":1.042},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.73999999999999999,"15min":0.753},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85899999999999999,"5min":1.006,"15min":1.006},"last":0.55200000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31752,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340088,"statfs":{"total":96636764160,"available":96604250112,"internally_reserved":0,"allocated":581632,"data_stored":255449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63300000000000001,"15min":0.63400000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.72699999999999998,"15min":0.73499999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.88900000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.66000000000000003}]},{"osd":1,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.80800000000000005,"15min":0.83599999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.878,"5min":4.2389999999999999,"15min":4.2389999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.79400000000000004,"15min":0.81399999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92600000000000005,"5min":4.218,"15min":4.218},"last":0.69699999999999995}]},{"osd":2,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.71299999999999997,"15min":0.72499999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82899999999999996,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70899999999999996,"15min":0.71099999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.64600000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72699999999999998,"15min":0.73499999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93700000000000006,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.57999999999999996}]},{"osd":4,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.44600000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.93999999999999995,"5min":0.995,"15min":0.995},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.69199999999999995,"15min":0.70199999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89900000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.5}]},{"osd":5,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.68400000000000005,"15min":0.69299999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.86799999999999999,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.629},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.77000000000000002,"15min":0.77700000000000002},"min":{"1min":0.51800000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.91800000000000004,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.61599999999999999}]},{"osd":7,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.77600000000000002,"15min":0.78800000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.97399999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75800000000000001,"15min":0.76400000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.90300000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.68200000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31752,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340088,"statfs":{"total":96636764160,"available":96604250112,"internally_reserved":0,"allocated":581632,"data_stored":255449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.70799999999999996,"15min":0.71599999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.92400000000000004,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.746,"15min":0.76100000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.95599999999999996,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":0.65800000000000003}]},{"osd":1,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.74199999999999999,"15min":0.754},"min":{"1min":0.437,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91400000000000003,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.78500000000000003,"15min":0.81399999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86299999999999999,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.67600000000000005}]},{"osd":2,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.75800000000000001,"15min":0.77600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89900000000000002,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.621},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.78000000000000003,"15min":0.79600000000000004},"min":{"1min":0.51500000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94099999999999995,"5min":4.0149999999999997,"15min":4.0149999999999997},"last":0.52000000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.82299999999999995,"15min":0.84899999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.89000000000000001,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.81899999999999995,"15min":0.84999999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80000000000000004,"5min":3.9900000000000002,"15min":3.9900000000000002},"last":0.54300000000000004}]},{"osd":4,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.40699999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71299999999999997,"15min":0.71699999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.90400000000000003,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.72599999999999998}]},{"osd":6,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.372,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90700000000000003,"5min":1.071,"15min":1.071},"last":0.501},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71899999999999997,"15min":0.72199999999999998},"min":{"1min":0.47999999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.98099999999999998,"5min":1.756,"15min":1.756},"last":0.59099999999999997}]},{"osd":7,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.75,"15min":0.75700000000000001},"min":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97299999999999998,"5min":1.282,"15min":1.282},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.75600000000000001,"15min":0.75900000000000001},"min":{"1min":0.505,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93200000000000005,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.70799999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31752,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340088,"statfs":{"total":96636764160,"available":96604250112,"internally_reserved":0,"allocated":581632,"data_stored":255449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.67800000000000005,"15min":0.69699999999999995},"min":{"1min":0.33600000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.91900000000000004,"5min":2.363,"15min":2.363},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64100000000000001,"15min":0.64600000000000002},"min":{"1min":0.442,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88700000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.81799999999999995}]},{"osd":1,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.624,"15min":0.621},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.026,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.77800000000000002}]},{"osd":2,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.88},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.72699999999999998,"15min":0.73099999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98499999999999999,"5min":1.871,"15min":1.871},"last":0.627}]},{"osd":3,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64200000000000002,"15min":0.64500000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83799999999999997,"5min":0.878,"15min":0.878},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.83199999999999996,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.79700000000000004}]},{"osd":5,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.84399999999999997,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.63500000000000001,"15min":0.64500000000000002},"min":{"1min":0.35799999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82399999999999995,"5min":1.79,"15min":1.79},"last":0.75800000000000001}]},{"osd":6,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65000000000000002,"15min":0.65400000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85599999999999998,"5min":1.123,"15min":1.123},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70599999999999996,"15min":0.71199999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93000000000000005,"5min":1.925,"15min":1.925},"last":0.66600000000000004}]},{"osd":7,"last update":"Tue Nov 28 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.71399999999999997,"15min":0.72399999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90000000000000002,"5min":1.843,"15min":1.843},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.77800000000000002,"15min":0.78400000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.94199999999999995,"5min":1.593,"15min":1.593},"last":0.91600000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35928,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335912,"statfs":{"total":96636764160,"available":96599973888,"internally_reserved":0,"allocated":593920,"data_stored":262544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.26900000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.46000000000000002,"15min":0.46500000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.59299999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.52700000000000002}]},{"osd":1,"last update":"Tue Nov 28 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.46000000000000002,"15min":0.46400000000000002},"min":{"1min":0.30599999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.56299999999999994,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.495,"15min":0.499},"min":{"1min":0.33900000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.63200000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.55300000000000005}]},{"osd":2,"last update":"Tue Nov 28 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.51200000000000001,"15min":0.51800000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65000000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79700000000000004,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.50600000000000001}]},{"osd":4,"last update":"Tue Nov 28 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76500000000000001,"15min":0.77200000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95199999999999996,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76300000000000001,"15min":0.77100000000000002},"min":{"1min":0.56100000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0049999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.69199999999999995}]},{"osd":5,"last update":"Tue Nov 28 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.82799999999999996,"15min":0.84199999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.052,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.052,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.80600000000000005}]},{"osd":6,"last update":"Tue Nov 28 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.79200000000000004,"15min":0.80000000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0189999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.79400000000000004,"15min":0.80200000000000005},"min":{"1min":0.59899999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0760000000000001,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.82399999999999995}]},{"osd":7,"last update":"Tue Nov 28 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.80200000000000005,"15min":0.81499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93799999999999994,"5min":1.786,"15min":1.786},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.123,"5min":1.337,"15min":1.337},"last":0.753}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":581632,"data_stored":255449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.50700000000000001,"15min":0.51000000000000001},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.4470000000000001,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.5,"15min":0.51200000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.623,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.41999999999999998}]},{"osd":1,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.50700000000000001,"15min":0.51400000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.63800000000000001,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.49199999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.499,"15min":0.50600000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67700000000000005,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.53300000000000003}]},{"osd":4,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77800000000000002,"15min":0.77600000000000002},"min":{"1min":0.55600000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.012,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.73599999999999999}]},{"osd":5,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.76300000000000001,"15min":0.77100000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0069999999999999,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.997,"15min":1.0329999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.5449999999999999,"5min":7.4820000000000002,"15min":7.4820000000000002},"last":0.75}]},{"osd":6,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.79100000000000004,"15min":0.80300000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0449999999999999,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.85399999999999998,"15min":0.88100000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.90800000000000003,"5min":3.9380000000000002,"15min":3.9380000000000002},"last":0.76100000000000001}]},{"osd":7,"last update":"Tue Nov 28 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.83399999999999996,"15min":0.84699999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.94999999999999996,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.90600000000000003,"15min":0.93500000000000005},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0509999999999999,"5min":4.798,"15min":4.798},"last":0.72399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":581632,"data_stored":255449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.42299999999999999,"15min":0.42899999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.44800000000000001,"15min":0.45000000000000001},"min":{"1min":0.29899999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57799999999999996,"5min":0.62,"15min":0.62},"last":0.50800000000000001}]},{"osd":2,"last update":"Tue Nov 28 18:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.34100000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.63,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47899999999999998,"15min":0.47799999999999998},"min":{"1min":0.35499999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.38}]},{"osd":3,"last update":"Tue Nov 28 18:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.67300000000000004,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.501},{"interface":"front","average":{"1min":0.503,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.34699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.67600000000000005}]},{"osd":4,"last update":"Tue Nov 28 18:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.92200000000000004,"5min":1.512,"15min":1.512},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76600000000000001,"15min":0.77200000000000002},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.98099999999999998,"5min":1.155,"15min":1.155},"last":0.64000000000000001}]},{"osd":5,"last update":"Tue Nov 28 18:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.747,"15min":0.755},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.89900000000000002,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.745,"15min":0.755},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.92300000000000004,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.57299999999999995}]},{"osd":6,"last update":"Tue Nov 28 18:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.94499999999999995,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.96199999999999997,"5min":1.448,"15min":1.448},"last":0.65700000000000003}]},{"osd":7,"last update":"Tue Nov 28 18:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.753,"15min":0.75700000000000001},"min":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94899999999999995,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.61299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1175552,"data_stored":845817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 18:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.438,"15min":0.44600000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.51900000000000002,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.46600000000000003,"15min":0.47099999999999997},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56999999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.53800000000000003}]},{"osd":2,"last update":"Tue Nov 28 18:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.50600000000000001,"15min":0.51200000000000001},"min":{"1min":0.30599999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.627,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.50900000000000001,"15min":0.51800000000000002},"min":{"1min":0.33700000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.621,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.40100000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48899999999999999,"15min":0.49199999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.54500000000000004,"15min":0.55600000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.66800000000000004,"5min":1.087,"15min":1.087},"last":0.58799999999999997}]},{"osd":4,"last update":"Tue Nov 28 18:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71199999999999997,"15min":0.71299999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94199999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71699999999999997,"15min":0.71799999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.78800000000000003}]},{"osd":5,"last update":"Tue Nov 28 18:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.83699999999999997}]},{"osd":6,"last update":"Tue Nov 28 18:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.72799999999999998,"15min":0.73299999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.997,"5min":1.407,"15min":1.407},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.085,"5min":1.089,"15min":1.089},"last":0.77300000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.755,"15min":0.754},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.877},{"interface":"front","average":{"1min":0.746,"5min":0.748,"15min":0.748},"min":{"1min":0.57099999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.022,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.72199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:26:05.609 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T18:26:05.609 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-28T18:26:05.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.784+0000 7fec4b53b640 1 Processor -- start 2023-11-28T18:26:05.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.784+0000 7fec4b53b640 1 -- start start 2023-11-28T18:26:05.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.788+0000 7fec4b53b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec4415d350 0x7fec440d38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:05.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.788+0000 7fec4b53b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fec4415d730 0x7fec440b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:05.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.788+0000 7fec4b53b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec440b28b0 con 0x7fec440b2400 2023-11-28T18:26:05.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.788+0000 7fec4b53b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec440cdae0 con 0x7fec4415d730 2023-11-28T18:26:05.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.788+0000 7fec4b53b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec440d1f40 con 0x7fec4415d350 2023-11-28T18:26:05.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.788+0000 7fec4ad3a640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fec440b2400 0x7fec440cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57952/0 (socket says 172.21.15.105:57952) 2023-11-28T18:26:05.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.788+0000 7fec4ad3a640 1 -- 172.21.15.105:0/2861040251 learned_addr learned my addr 172.21.15.105:0/2861040251 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:05.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.788+0000 7fec49d38640 1 --2- 172.21.15.105:0/2861040251 >> v2:172.21.15.148:3300/0 conn(0x7fec4415d730 0x7fec440b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:05.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.788+0000 7fec4a539640 1 --2- 172.21.15.105:0/2861040251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec4415d350 0x7fec440d38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:05.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.788+0000 7fec49537640 1 -- 172.21.15.105:0/2861040251 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777496628 0 0) 0x7fec440b28b0 con 0x7fec440b2400 2023-11-28T18:26:05.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.788+0000 7fec49537640 1 -- 172.21.15.105:0/2861040251 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec30003530 con 0x7fec440b2400 2023-11-28T18:26:05.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.788+0000 7fec49537640 1 -- 172.21.15.105:0/2861040251 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2856800028 0 0) 0x7fec30003530 con 0x7fec440b2400 2023-11-28T18:26:05.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.788+0000 7fec49d38640 1 -- 172.21.15.105:0/2861040251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec4415d350 msgr2=0x7fec440d38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:05.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.788+0000 7fec49d38640 1 --2- 172.21.15.105:0/2861040251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec4415d350 0x7fec440d38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.788+0000 7fec49d38640 1 -- 172.21.15.105:0/2861040251 >> v1:172.21.15.105:6789/0 conn(0x7fec440b2400 legacy=0x7fec440cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:05.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.788+0000 7fec49d38640 1 -- 172.21.15.105:0/2861040251 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec440d16a0 con 0x7fec4415d730 2023-11-28T18:26:05.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.792+0000 7fec49d38640 1 --2- 172.21.15.105:0/2861040251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec4415d730 0x7fec440b1e40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fec340097c0 tx=0x7fec3401c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f97ce99309a58f8a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:05.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.792+0000 7fec49537640 1 -- 172.21.15.105:0/2861040251 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c001a00 con 0x7fec4415d730 2023-11-28T18:26:05.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.792+0000 7fec49537640 1 -- 172.21.15.105:0/2861040251 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec340a7050 con 0x7fec4415d730 2023-11-28T18:26:05.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.792+0000 7fec49537640 1 -- 172.21.15.105:0/2861040251 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec340ab600 con 0x7fec4415d730 2023-11-28T18:26:05.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.792+0000 7fec4b53b640 1 -- 172.21.15.105:0/2861040251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec4415d730 msgr2=0x7fec440b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:05.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.792+0000 7fec4b53b640 1 --2- 172.21.15.105:0/2861040251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec4415d730 0x7fec440b1e40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fec340097c0 tx=0x7fec3401c310 comp rx=0 tx=0).stop 2023-11-28T18:26:05.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.792+0000 7fec4b53b640 1 -- 172.21.15.105:0/2861040251 shutdown_connections 2023-11-28T18:26:05.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.792+0000 7fec4b53b640 1 --2- 172.21.15.105:0/2861040251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec4415d730 0x7fec440b1e40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.792+0000 7fec4b53b640 1 --2- 172.21.15.105:0/2861040251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec4415d350 0x7fec440d38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.792+0000 7fec4b53b640 1 -- 172.21.15.105:0/2861040251 >> 172.21.15.105:0/2861040251 conn(0x7fec440d2d90 msgr2=0x7fec440d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:05.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.792+0000 7fec4b53b640 1 -- 172.21.15.105:0/2861040251 shutdown_connections 2023-11-28T18:26:05.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.792+0000 7fec4b53b640 1 -- 172.21.15.105:0/2861040251 wait complete. 2023-11-28T18:26:05.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.792+0000 7fec4b53b640 1 Processor -- start 2023-11-28T18:26:05.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.792+0000 7fec4b53b640 1 -- start start 2023-11-28T18:26:05.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec4b53b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec440b2400 0x7fec44149410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:05.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec4b53b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec4415d350 0x7fec440b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:05.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec4b53b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec4415d730 0x7fec44157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:05.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec4b53b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec44149950 con 0x7fec440b2400 2023-11-28T18:26:05.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec4b53b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4415eb60 con 0x7fec4415d350 2023-11-28T18:26:05.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec4b53b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4415f400 con 0x7fec4415d730 2023-11-28T18:26:05.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec4a539640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec440b2400 0x7fec44149410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:05.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec4a539640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec440b2400 0x7fec44149410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45970/0 (socket says 172.21.15.105:45970) 2023-11-28T18:26:05.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec4a539640 1 -- 172.21.15.105:0/2916616070 learned_addr learned my addr 172.21.15.105:0/2916616070 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:05.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec4ad3a640 1 --2- 172.21.15.105:0/2916616070 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec4415d730 0x7fec44157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:05.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec4a539640 1 -- 172.21.15.105:0/2916616070 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec4415d730 msgr2=0x7fec44157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:05.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec4a539640 1 --2- 172.21.15.105:0/2916616070 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec4415d730 0x7fec44157eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec4a539640 1 -- 172.21.15.105:0/2916616070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec4415d350 msgr2=0x7fec440b6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:05.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec49d38640 1 --2- 172.21.15.105:0/2916616070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec4415d350 0x7fec440b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:05.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec4a539640 1 --2- 172.21.15.105:0/2916616070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec4415d350 0x7fec440b6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:05.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec4a539640 1 -- 172.21.15.105:0/2916616070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec34009420 con 0x7fec440b2400 2023-11-28T18:26:05.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec49d38640 1 --2- 172.21.15.105:0/2916616070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec4415d350 0x7fec440b6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:05.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec4a539640 1 --2- 172.21.15.105:0/2916616070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec440b2400 0x7fec44149410 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fec4000e930 tx=0x7fec4000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:05.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec3affd640 1 -- 172.21.15.105:0/2916616070 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec40009690 con 0x7fec440b2400 2023-11-28T18:26:05.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec3affd640 1 -- 172.21.15.105:0/2916616070 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4000ca00 con 0x7fec440b2400 2023-11-28T18:26:05.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec4b53b640 1 -- 172.21.15.105:0/2916616070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec440c7690 con 0x7fec440b2400 2023-11-28T18:26:05.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec3affd640 1 -- 172.21.15.105:0/2916616070 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4000cd70 con 0x7fec440b2400 2023-11-28T18:26:05.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec4b53b640 1 -- 172.21.15.105:0/2916616070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec441583f0 con 0x7fec440b2400 2023-11-28T18:26:05.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec4b53b640 1 -- 172.21.15.105:0/2916616070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec44074580 con 0x7fec440b2400 2023-11-28T18:26:05.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.796+0000 7fec3affd640 1 -- 172.21.15.105:0/2916616070 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fec40013470 con 0x7fec440b2400 2023-11-28T18:26:05.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.800+0000 7fec3affd640 1 --2- 172.21.15.105:0/2916616070 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec28066900 0x7fec28068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:05.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.800+0000 7fec3affd640 1 -- 172.21.15.105:0/2916616070 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fec40084a80 con 0x7fec440b2400 2023-11-28T18:26:05.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.800+0000 7fec49d38640 1 --2- 172.21.15.105:0/2916616070 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec28066900 0x7fec28068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:05.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.800+0000 7fec49d38640 1 --2- 172.21.15.105:0/2916616070 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec28066900 0x7fec28068dc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fec340ad000 tx=0x7fec340a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:05.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:05.804+0000 7fec3affd640 1 -- 172.21.15.105:0/2916616070 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fec40051cd0 con 0x7fec440b2400 2023-11-28T18:26:06.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.004+0000 7fec4b53b640 1 -- 172.21.15.105:0/2916616070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fec4408cad0 con 0x7fec440b2400 2023-11-28T18:26:06.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.008+0000 7fec3affd640 1 -- 172.21.15.105:0/2916616070 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7fec400210b0 con 0x7fec440b2400 2023-11-28T18:26:06.011 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2023-11-28T18:26:06.011 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDpL2ZlSt0VMxAAQ8+j1BgBPMJ2cuiZj8aG0w== 2023-11-28T18:26:06.011 INFO:teuthology.orchestra.run.smithi105.stdout: caps mds = "allow" 2023-11-28T18:26:06.011 INFO:teuthology.orchestra.run.smithi105.stdout: caps mon = "allow r" 2023-11-28T18:26:06.011 INFO:teuthology.orchestra.run.smithi105.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-28T18:26:06.012 INFO:teuthology.orchestra.run.smithi105.stderr:updated caps for client.0 2023-11-28T18:26:06.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.008+0000 7fec4b53b640 1 -- 172.21.15.105:0/2916616070 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec28066900 msgr2=0x7fec28068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:06.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.008+0000 7fec4b53b640 1 --2- 172.21.15.105:0/2916616070 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec28066900 0x7fec28068dc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fec340ad000 tx=0x7fec340a6000 comp rx=0 tx=0).stop 2023-11-28T18:26:06.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.008+0000 7fec4b53b640 1 -- 172.21.15.105:0/2916616070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec440b2400 msgr2=0x7fec44149410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:06.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.008+0000 7fec4b53b640 1 --2- 172.21.15.105:0/2916616070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec440b2400 0x7fec44149410 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fec4000e930 tx=0x7fec4000edf0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.008+0000 7fec4b53b640 1 -- 172.21.15.105:0/2916616070 shutdown_connections 2023-11-28T18:26:06.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.008+0000 7fec4b53b640 1 --2- 172.21.15.105:0/2916616070 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec4415d730 0x7fec44157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.008+0000 7fec4b53b640 1 --2- 172.21.15.105:0/2916616070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec4415d350 0x7fec440b6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.008+0000 7fec4b53b640 1 --2- 172.21.15.105:0/2916616070 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec28066900 0x7fec28068dc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.008+0000 7fec4b53b640 1 --2- 172.21.15.105:0/2916616070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec440b2400 0x7fec44149410 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.008+0000 7fec4b53b640 1 -- 172.21.15.105:0/2916616070 >> 172.21.15.105:0/2916616070 conn(0x7fec440d2d90 msgr2=0x7fec4415c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:06.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.008+0000 7fec4b53b640 1 -- 172.21.15.105:0/2916616070 shutdown_connections 2023-11-28T18:26:06.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.008+0000 7fec4b53b640 1 -- 172.21.15.105:0/2916616070 wait complete. 2023-11-28T18:26:06.032 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:26:06.170 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.168+0000 7f918d2ad640 1 Processor -- start 2023-11-28T18:26:06.170 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.168+0000 7f918d2ad640 1 -- start start 2023-11-28T18:26:06.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.168+0000 7f918d2ad640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f918815d4f0 0x7f91880d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.168+0000 7f918d2ad640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91880d3c30 0x7f91880d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.168+0000 7f918d2ad640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91880b25b0 con 0x7f91880d1890 2023-11-28T18:26:06.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.168+0000 7f918d2ad640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91880b2900 con 0x7f918815d4f0 2023-11-28T18:26:06.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.168+0000 7f918d2ad640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91880cea30 con 0x7f91880d3c30 2023-11-28T18:26:06.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.168+0000 7f91877fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f91880d1890 0x7f91880cfa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57964/0 (socket says 172.21.15.105:57964) 2023-11-28T18:26:06.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.168+0000 7f91877fe640 1 -- 172.21.15.105:0/4023160196 learned_addr learned my addr 172.21.15.105:0/4023160196 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:06.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.168+0000 7f9186ffd640 1 --2- 172.21.15.105:0/4023160196 >> v2:172.21.15.148:3300/0 conn(0x7f918815d4f0 0x7f91880d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.168+0000 7f91867fc640 1 --2- 172.21.15.105:0/4023160196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91880d3c30 0x7f91880d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.168+0000 7f9185ffb640 1 -- 172.21.15.105:0/4023160196 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142674084 0 0) 0x7f91880b25b0 con 0x7f91880d1890 2023-11-28T18:26:06.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.168+0000 7f9185ffb640 1 -- 172.21.15.105:0/4023160196 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9170003530 con 0x7f91880d1890 2023-11-28T18:26:06.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.168+0000 7f9186ffd640 1 -- 172.21.15.105:0/4023160196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91880d3c30 msgr2=0x7f91880d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:06.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.168+0000 7f9185ffb640 1 -- 172.21.15.105:0/4023160196 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1963951775 0 0) 0x7f9170003530 con 0x7f91880d1890 2023-11-28T18:26:06.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.168+0000 7f9186ffd640 1 --2- 172.21.15.105:0/4023160196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91880d3c30 0x7f91880d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.168+0000 7f9186ffd640 1 -- 172.21.15.105:0/4023160196 >> v1:172.21.15.105:6789/0 conn(0x7f91880d1890 legacy=0x7f91880cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:06.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.168+0000 7f9186ffd640 1 -- 172.21.15.105:0/4023160196 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91880ced10 con 0x7f918815d4f0 2023-11-28T18:26:06.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.172+0000 7f9186ffd640 1 --2- 172.21.15.105:0/4023160196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f918815d4f0 0x7f91880d0e10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f91780097c0 tx=0x7f917801c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28d80614d98eec9e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:06.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.172+0000 7f9185ffb640 1 -- 172.21.15.105:0/4023160196 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180001a00 con 0x7f918815d4f0 2023-11-28T18:26:06.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.172+0000 7f9185ffb640 1 -- 172.21.15.105:0/4023160196 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91780a7050 con 0x7f918815d4f0 2023-11-28T18:26:06.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.172+0000 7f9185ffb640 1 -- 172.21.15.105:0/4023160196 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91780ab620 con 0x7f918815d4f0 2023-11-28T18:26:06.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.172+0000 7f918d2ad640 1 -- 172.21.15.105:0/4023160196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f918815d4f0 msgr2=0x7f91880d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:06.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.172+0000 7f918d2ad640 1 --2- 172.21.15.105:0/4023160196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f918815d4f0 0x7f91880d0e10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f91780097c0 tx=0x7f917801c2e0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.172+0000 7f918d2ad640 1 -- 172.21.15.105:0/4023160196 shutdown_connections 2023-11-28T18:26:06.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.172+0000 7f918d2ad640 1 --2- 172.21.15.105:0/4023160196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91880d3c30 0x7f91880d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.172+0000 7f918d2ad640 1 --2- 172.21.15.105:0/4023160196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f918815d4f0 0x7f91880d0e10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.172+0000 7f918d2ad640 1 -- 172.21.15.105:0/4023160196 >> 172.21.15.105:0/4023160196 conn(0x7f91880d3030 msgr2=0x7f91880d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:06.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.172+0000 7f918d2ad640 1 -- 172.21.15.105:0/4023160196 shutdown_connections 2023-11-28T18:26:06.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.172+0000 7f918d2ad640 1 -- 172.21.15.105:0/4023160196 wait complete. 2023-11-28T18:26:06.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.176+0000 7f918d2ad640 1 Processor -- start 2023-11-28T18:26:06.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.176+0000 7f918d2ad640 1 -- start start 2023-11-28T18:26:06.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.176+0000 7f918d2ad640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91880d1890 0x7f91880ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.176+0000 7f918d2ad640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91880d3c30 0x7f91880c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.176+0000 7f918d2ad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f918815d4f0 0x7f918806be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.176+0000 7f918d2ad640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91880c9150 con 0x7f91880d1890 2023-11-28T18:26:06.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.176+0000 7f918d2ad640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9188146540 con 0x7f918815d4f0 2023-11-28T18:26:06.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.176+0000 7f918d2ad640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9188146820 con 0x7f91880d3c30 2023-11-28T18:26:06.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.176+0000 7f9186ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91880d1890 0x7f91880ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.176+0000 7f9186ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91880d1890 0x7f91880ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45974/0 (socket says 172.21.15.105:45974) 2023-11-28T18:26:06.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.176+0000 7f9186ffd640 1 -- 172.21.15.105:0/1780595707 learned_addr learned my addr 172.21.15.105:0/1780595707 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:06.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.176+0000 7f91877fe640 1 --2- 172.21.15.105:0/1780595707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f918815d4f0 0x7f918806be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.176+0000 7f91867fc640 1 --2- 172.21.15.105:0/1780595707 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91880d3c30 0x7f91880c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.176+0000 7f9186ffd640 1 -- 172.21.15.105:0/1780595707 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91880d3c30 msgr2=0x7f91880c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:06.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.176+0000 7f9186ffd640 1 --2- 172.21.15.105:0/1780595707 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91880d3c30 0x7f91880c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.176+0000 7f9186ffd640 1 -- 172.21.15.105:0/1780595707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f918815d4f0 msgr2=0x7f918806be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:06.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.176+0000 7f9186ffd640 1 --2- 172.21.15.105:0/1780595707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f918815d4f0 0x7f918806be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.176+0000 7f9186ffd640 1 -- 172.21.15.105:0/1780595707 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9178009420 con 0x7f91880d1890 2023-11-28T18:26:06.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.176+0000 7f91877fe640 1 --2- 172.21.15.105:0/1780595707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f918815d4f0 0x7f918806be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:06.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.176+0000 7f9186ffd640 1 --2- 172.21.15.105:0/1780595707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91880d1890 0x7f91880ccac0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9178019f10 tx=0x7f9178019080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:06.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.180+0000 7f916f7fe640 1 -- 172.21.15.105:0/1780595707 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91780b4400 con 0x7f91880d1890 2023-11-28T18:26:06.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.180+0000 7f916f7fe640 1 -- 172.21.15.105:0/1780595707 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91780b4560 con 0x7f91880d1890 2023-11-28T18:26:06.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.180+0000 7f916f7fe640 1 -- 172.21.15.105:0/1780595707 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91780b4910 con 0x7f91880d1890 2023-11-28T18:26:06.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.180+0000 7f918d2ad640 1 -- 172.21.15.105:0/1780595707 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9188146b00 con 0x7f91880d1890 2023-11-28T18:26:06.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.180+0000 7f918d2ad640 1 -- 172.21.15.105:0/1780595707 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91881598e0 con 0x7f91880d1890 2023-11-28T18:26:06.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.180+0000 7f918d2ad640 1 -- 172.21.15.105:0/1780595707 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9188146fb0 con 0x7f91880d1890 2023-11-28T18:26:06.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.180+0000 7f916f7fe640 1 -- 172.21.15.105:0/1780595707 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f91780b2040 con 0x7f91880d1890 2023-11-28T18:26:06.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.180+0000 7f916f7fe640 1 --2- 172.21.15.105:0/1780595707 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f91740668d0 0x7f9174068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.180+0000 7f916f7fe640 1 -- 172.21.15.105:0/1780595707 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f917811e750 con 0x7f91880d1890 2023-11-28T18:26:06.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.180+0000 7f91867fc640 1 --2- 172.21.15.105:0/1780595707 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f91740668d0 0x7f9174068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.180+0000 7f91867fc640 1 --2- 172.21.15.105:0/1780595707 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f91740668d0 0x7f9174068d90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f91880b4a90 tx=0x7f917c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:06.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.184+0000 7f916f7fe640 1 -- 172.21.15.105:0/1780595707 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f91780eb840 con 0x7f91880d1890 2023-11-28T18:26:06.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.368+0000 7f918d2ad640 1 -- 172.21.15.105:0/1780595707 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9188067570 con 0x7f91880d1890 2023-11-28T18:26:06.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.368+0000 7f916f7fe640 1 -- 172.21.15.105:0/1780595707 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f91780b0090 con 0x7f91880d1890 2023-11-28T18:26:06.372 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:06.372 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":35,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/735646078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":735646078},{"type":"v1","addr":"172.21.15.105:6836","nonce":735646078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5408,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/3765684108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":3765684108},{"type":"v1","addr":"172.21.15.105:6837","nonce":3765684108}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/897997046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":897997046},{"type":"v1","addr":"172.21.15.148:6835","nonce":897997046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:26:00.550298+0000","modified":"2023-11-28T18:26:03.550508+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5842},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5842":{"gid":5842,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6834/2696773413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2696773413},{"type":"v1","addr":"172.21.15.148:6834","nonce":2696773413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T18:26:06.372 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 35 2023-11-28T18:26:06.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.372+0000 7f918d2ad640 1 -- 172.21.15.105:0/1780595707 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f91740668d0 msgr2=0x7f9174068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:06.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.372+0000 7f918d2ad640 1 --2- 172.21.15.105:0/1780595707 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f91740668d0 0x7f9174068d90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f91880b4a90 tx=0x7f917c008040 comp rx=0 tx=0).stop 2023-11-28T18:26:06.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.372+0000 7f918d2ad640 1 -- 172.21.15.105:0/1780595707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91880d1890 msgr2=0x7f91880ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:06.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.372+0000 7f918d2ad640 1 --2- 172.21.15.105:0/1780595707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91880d1890 0x7f91880ccac0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9178019f10 tx=0x7f9178019080 comp rx=0 tx=0).stop 2023-11-28T18:26:06.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.372+0000 7f918d2ad640 1 -- 172.21.15.105:0/1780595707 shutdown_connections 2023-11-28T18:26:06.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.372+0000 7f918d2ad640 1 --2- 172.21.15.105:0/1780595707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f918815d4f0 0x7f918806be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.372+0000 7f918d2ad640 1 --2- 172.21.15.105:0/1780595707 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91880d3c30 0x7f91880c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.372+0000 7f918d2ad640 1 --2- 172.21.15.105:0/1780595707 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f91740668d0 0x7f9174068d90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.372+0000 7f918d2ad640 1 --2- 172.21.15.105:0/1780595707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91880d1890 0x7f91880ccac0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.372+0000 7f918d2ad640 1 -- 172.21.15.105:0/1780595707 >> 172.21.15.105:0/1780595707 conn(0x7f91880d3030 msgr2=0x7f91880c94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:06.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.372+0000 7f918d2ad640 1 -- 172.21.15.105:0/1780595707 shutdown_connections 2023-11-28T18:26:06.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.372+0000 7f918d2ad640 1 -- 172.21.15.105:0/1780595707 wait complete. 2023-11-28T18:26:06.796 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:26:00.550298+0000', 'modified': '2023-11-28T18:26:03.550508+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5842}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5842': {'gid': 5842, 'name': 'b', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6834/2696773413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6832', 'nonce': 2696773413}, {'type': 'v1', 'addr': '172.21.15.148:6834', 'nonce': 2696773413}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:26:06.797 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T18:26:06.797 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-28T18:26:06.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.936+0000 7f3378f65640 1 Processor -- start 2023-11-28T18:26:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.936+0000 7f3378f65640 1 -- start start 2023-11-28T18:26:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.936+0000 7f3378f65640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f33740b4290 0x7f33740b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.936+0000 7f3378f65640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f337415b030 0x7f33740709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.936+0000 7f3378f65640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33740b4bb0 con 0x7f3374071960 2023-11-28T18:26:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.936+0000 7f3378f65640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33740701d0 con 0x7f33740b4290 2023-11-28T18:26:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.936+0000 7f3378f65640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f337406df80 con 0x7f337415b030 2023-11-28T18:26:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.936+0000 7f3371d9b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3374071960 0x7f337415f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57978/0 (socket says 172.21.15.105:57978) 2023-11-28T18:26:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.936+0000 7f3371d9b640 1 -- 172.21.15.105:0/2288754435 learned_addr learned my addr 172.21.15.105:0/2288754435 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.936+0000 7f337259c640 1 --2- 172.21.15.105:0/2288754435 >> v2:172.21.15.105:3300/0 conn(0x7f33740b4290 0x7f33740b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.936+0000 7f3372d9d640 1 --2- 172.21.15.105:0/2288754435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f337415b030 0x7f33740709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.936+0000 7f337259c640 1 -- 172.21.15.105:0/2288754435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f337415b030 msgr2=0x7f33740709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:06.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.936+0000 7f337259c640 1 --2- 172.21.15.105:0/2288754435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f337415b030 0x7f33740709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.936+0000 7f337259c640 1 -- 172.21.15.105:0/2288754435 >> v1:172.21.15.105:6789/0 conn(0x7f3374071960 legacy=0x7f337415f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:06.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.936+0000 7f337259c640 1 -- 172.21.15.105:0/2288754435 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3374071df0 con 0x7f33740b4290 2023-11-28T18:26:06.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.936+0000 7f337159a640 1 -- 172.21.15.105:0/2288754435 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3167313933 0 0) 0x7f33740b4bb0 con 0x7f3374071960 2023-11-28T18:26:06.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.940+0000 7f337259c640 1 --2- 172.21.15.105:0/2288754435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33740b4290 0x7f33740b4670 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f33640097c0 tx=0x7f336401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e81317aadd344d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:06.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.940+0000 7f337159a640 1 -- 172.21.15.105:0/2288754435 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33640ad660 con 0x7f33740b4290 2023-11-28T18:26:06.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.940+0000 7f337159a640 1 -- 172.21.15.105:0/2288754435 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33640ad7c0 con 0x7f33740b4290 2023-11-28T18:26:06.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.940+0000 7f337159a640 1 -- 172.21.15.105:0/2288754435 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33640adb50 con 0x7f33740b4290 2023-11-28T18:26:06.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.940+0000 7f3378f65640 1 -- 172.21.15.105:0/2288754435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33740b4290 msgr2=0x7f33740b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:06.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.940+0000 7f3378f65640 1 --2- 172.21.15.105:0/2288754435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33740b4290 0x7f33740b4670 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f33640097c0 tx=0x7f336401c450 comp rx=0 tx=0).stop 2023-11-28T18:26:06.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.940+0000 7f3378f65640 1 -- 172.21.15.105:0/2288754435 shutdown_connections 2023-11-28T18:26:06.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.940+0000 7f3378f65640 1 --2- 172.21.15.105:0/2288754435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f337415b030 0x7f33740709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.940+0000 7f3378f65640 1 --2- 172.21.15.105:0/2288754435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33740b4290 0x7f33740b4670 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.940+0000 7f3378f65640 1 -- 172.21.15.105:0/2288754435 >> 172.21.15.105:0/2288754435 conn(0x7f33740ceb90 msgr2=0x7f33740cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:06.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.940+0000 7f3378f65640 1 -- 172.21.15.105:0/2288754435 shutdown_connections 2023-11-28T18:26:06.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.940+0000 7f3378f65640 1 -- 172.21.15.105:0/2288754435 wait complete. 2023-11-28T18:26:06.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3378f65640 1 Processor -- start 2023-11-28T18:26:06.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3378f65640 1 -- start start 2023-11-28T18:26:06.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3378f65640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33740b4290 0x7f337414b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3378f65640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f337415b030 0x7f3374099f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3378f65640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3374154c30 0x7f33741550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3378f65640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3374155620 con 0x7f3374154c30 2023-11-28T18:26:06.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3378f65640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337414bda0 con 0x7f337415b030 2023-11-28T18:26:06.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3378f65640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3374153ac0 con 0x7f33740b4290 2023-11-28T18:26:06.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3372d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3374154c30 0x7f33741550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3372d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3374154c30 0x7f33741550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45996/0 (socket says 172.21.15.105:45996) 2023-11-28T18:26:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3372d9d640 1 -- 172.21.15.105:0/4186318382 learned_addr learned my addr 172.21.15.105:0/4186318382 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3371d9b640 1 --2- 172.21.15.105:0/4186318382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f337415b030 0x7f3374099f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f337259c640 1 --2- 172.21.15.105:0/4186318382 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33740b4290 0x7f337414b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3372d9d640 1 -- 172.21.15.105:0/4186318382 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33740b4290 msgr2=0x7f337414b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3372d9d640 1 --2- 172.21.15.105:0/4186318382 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33740b4290 0x7f337414b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3372d9d640 1 -- 172.21.15.105:0/4186318382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f337415b030 msgr2=0x7f3374099f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:06.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3372d9d640 1 --2- 172.21.15.105:0/4186318382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f337415b030 0x7f3374099f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3372d9d640 1 -- 172.21.15.105:0/4186318382 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3364009420 con 0x7f3374154c30 2023-11-28T18:26:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3371d9b640 1 --2- 172.21.15.105:0/4186318382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f337415b030 0x7f3374099f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f337259c640 1 --2- 172.21.15.105:0/4186318382 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33740b4290 0x7f337414b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3372d9d640 1 --2- 172.21.15.105:0/4186318382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3374154c30 0x7f33741550e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f336c009fd0 tx=0x7f336c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3362ffd640 1 -- 172.21.15.105:0/4186318382 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c014420 con 0x7f3374154c30 2023-11-28T18:26:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3362ffd640 1 -- 172.21.15.105:0/4186318382 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336c014580 con 0x7f3374154c30 2023-11-28T18:26:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3362ffd640 1 -- 172.21.15.105:0/4186318382 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c014930 con 0x7f3374154c30 2023-11-28T18:26:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3378f65640 1 -- 172.21.15.105:0/4186318382 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3374153da0 con 0x7f3374154c30 2023-11-28T18:26:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.944+0000 7f3378f65640 1 -- 172.21.15.105:0/4186318382 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33740ccba0 con 0x7f3374154c30 2023-11-28T18:26:06.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.948+0000 7f3362ffd640 1 -- 172.21.15.105:0/4186318382 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f336c014a90 con 0x7f3374154c30 2023-11-28T18:26:06.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.948+0000 7f3362ffd640 1 --2- 172.21.15.105:0/4186318382 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f33540668d0 0x7f3354068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.948+0000 7f3362ffd640 1 -- 172.21.15.105:0/4186318382 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f336c084eb0 con 0x7f3374154c30 2023-11-28T18:26:06.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.948+0000 7f337259c640 1 --2- 172.21.15.105:0/4186318382 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f33540668d0 0x7f3354068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.948+0000 7f337c2c0000 1 Processor -- start 2023-11-28T18:26:06.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.948+0000 7f337c2c0000 1 -- start start 2023-11-28T18:26:06.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.948+0000 7f337259c640 1 --2- 172.21.15.105:0/4186318382 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f33540668d0 0x7f3354068d90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f33640af000 tx=0x7f3364008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:06.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f337c2c0000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563828853d30 0x563828854110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f337c2c0000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828854650 0x563828806fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f337c2c0000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x563828807500 0x5638288078e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f337c2c0000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638286d0110 con 0x563828854650 2023-11-28T18:26:06.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f337c2c0000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638286609a0 con 0x563828853d30 2023-11-28T18:26:06.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f337c2c0000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x56382865ded0 con 0x563828807500 2023-11-28T18:26:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f3371d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828854650 0x563828806fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f3371d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828854650 0x563828806fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46000/0 (socket says 172.21.15.105:46000) 2023-11-28T18:26:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f3371d9b640 1 -- 172.21.15.105:0/2172122019 learned_addr learned my addr 172.21.15.105:0/2172122019 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f337259c640 1 --2- 172.21.15.105:0/2172122019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563828853d30 0x563828854110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f3372d9d640 1 --2- 172.21.15.105:0/2172122019 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x563828807500 0x5638288078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f3371d9b640 1 -- 172.21.15.105:0/2172122019 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x563828807500 msgr2=0x5638288078e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:06.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f3371d9b640 1 --2- 172.21.15.105:0/2172122019 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x563828807500 0x5638288078e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f3371d9b640 1 -- 172.21.15.105:0/2172122019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563828853d30 msgr2=0x563828854110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:06.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f3371d9b640 1 --2- 172.21.15.105:0/2172122019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563828853d30 0x563828854110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f3371d9b640 1 -- 172.21.15.105:0/2172122019 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563828351890 con 0x563828854650 2023-11-28T18:26:06.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f3371d9b640 1 --2- 172.21.15.105:0/2172122019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828854650 0x563828806fc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3368007500 tx=0x7f3368005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e259399d164c8ae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:06.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f3353fff640 1 -- 172.21.15.105:0/2172122019 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33680018a0 con 0x563828854650 2023-11-28T18:26:06.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f3353fff640 1 -- 172.21.15.105:0/2172122019 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3368001a00 con 0x563828854650 2023-11-28T18:26:06.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f3353fff640 1 -- 172.21.15.105:0/2172122019 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3368001d70 con 0x563828854650 2023-11-28T18:26:06.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f337c2c0000 1 -- 172.21.15.105:0/2172122019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828854650 msgr2=0x563828806fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:06.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f337c2c0000 1 --2- 172.21.15.105:0/2172122019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828854650 0x563828806fc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3368007500 tx=0x7f3368005bd0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f337c2c0000 1 -- 172.21.15.105:0/2172122019 shutdown_connections 2023-11-28T18:26:06.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f337c2c0000 1 --2- 172.21.15.105:0/2172122019 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x563828807500 0x5638288078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f337c2c0000 1 --2- 172.21.15.105:0/2172122019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828854650 0x563828806fc0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f337c2c0000 1 --2- 172.21.15.105:0/2172122019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563828853d30 0x563828854110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f337c2c0000 1 -- 172.21.15.105:0/2172122019 >> 172.21.15.105:0/2172122019 conn(0x5638288348c0 msgr2=0x563828836cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:06.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f337c2c0000 1 -- 172.21.15.105:0/2172122019 shutdown_connections 2023-11-28T18:26:06.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f337c2c0000 1 -- 172.21.15.105:0/2172122019 wait complete. 2023-11-28T18:26:06.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f337c2c0000 1 -- start start 2023-11-28T18:26:06.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.952+0000 7f337c2c0000 1 Processor -- start 2023-11-28T18:26:06.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f337c2c0000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x563828807500 0x56382880d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f337c2c0000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563828853d30 0x56382880da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f337c2c0000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828854650 0x56382880df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f337c2c0000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638286597c0 con 0x563828854650 2023-11-28T18:26:06.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f337c2c0000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563828611bf0 con 0x563828853d30 2023-11-28T18:26:06.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f337c2c0000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x563828617a90 con 0x563828807500 2023-11-28T18:26:06.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f337c2c0000 10 client.0 _ll_register_callbacks cb 0x5638287f4e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:06.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f3372d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828854650 0x56382880df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f3372d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828854650 0x56382880df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46016/0 (socket says 172.21.15.105:46016) 2023-11-28T18:26:06.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f3372d9d640 1 -- 172.21.15.105:0/3273481766 learned_addr learned my addr 172.21.15.105:0/3273481766 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:06.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f337259c640 1 --2- 172.21.15.105:0/3273481766 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x563828807500 0x56382880d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f3371d9b640 1 --2- 172.21.15.105:0/3273481766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563828853d30 0x56382880da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f3372d9d640 1 -- 172.21.15.105:0/3273481766 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x563828807500 msgr2=0x56382880d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:06.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f3372d9d640 1 --2- 172.21.15.105:0/3273481766 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x563828807500 0x56382880d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f3372d9d640 1 -- 172.21.15.105:0/3273481766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563828853d30 msgr2=0x56382880da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:06.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f3372d9d640 1 --2- 172.21.15.105:0/3273481766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563828853d30 0x56382880da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f3372d9d640 1 -- 172.21.15.105:0/3273481766 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f336800f040 con 0x563828854650 2023-11-28T18:26:06.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f3372d9d640 1 --2- 172.21.15.105:0/3273481766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828854650 0x56382880df60 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f336c00e9b0 tx=0x7f336c055c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f034fac08f40fb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:06.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f33517fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:06.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f33517fa640 1 -- 172.21.15.105:0/3273481766 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c05da40 con 0x563828854650 2023-11-28T18:26:06.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f33517fa640 1 -- 172.21.15.105:0/3273481766 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f336c05dba0 con 0x563828854650 2023-11-28T18:26:06.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f337c2c0000 1 -- 172.21.15.105:0/3273481766 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56382861b470 con 0x563828854650 2023-11-28T18:26:06.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f33517fa640 1 -- 172.21.15.105:0/3273481766 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f336c06d210 con 0x563828854650 2023-11-28T18:26:06.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f337c2c0000 10 client.6150 fetch_fsmap learned FSMap version 35 2023-11-28T18:26:06.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f337c2c0000 1 -- 172.21.15.105:0/3273481766 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x56382861a190 con 0x563828854650 2023-11-28T18:26:06.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f33517fa640 1 -- 172.21.15.105:0/3273481766 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56382861b470 con 0x563828854650 2023-11-28T18:26:06.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f33517fa640 1 -- 172.21.15.105:0/3273481766 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f336c005d20 con 0x563828854650 2023-11-28T18:26:06.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f337c2c0000 10 client.6150 fetch_fsmap finished waiting for FSMap version 35 2023-11-28T18:26:06.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f337c2c0000 10 client.6150 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T18:26:06.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f337c2c0000 20 client.6150 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:06.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f337c2c0000 1 --2- 172.21.15.105:0/3273481766 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x56382880e970 0x563828865ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f337c2c0000 4 client.6150 mds_command: new command op to 5842 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:26:06.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f337c2c0000 1 -- 172.21.15.105:0/3273481766 --> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56382861eb50 con 0x56382880e970 2023-11-28T18:26:06.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f337259c640 1 --2- 172.21.15.105:0/3273481766 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x56382880e970 0x563828865ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f337259c640 1 --2- 172.21.15.105:0/3273481766 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x56382880e970 0x563828865ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a48a142fb840da32 server_cookie=5cc6a13375e95926 in_seq=0 out_seq=0 2023-11-28T18:26:06.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.956+0000 7f33517fa640 10 client.6150 ms_handle_connect on v2:172.21.15.148:6832/2696773413 2023-11-28T18:26:06.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.960+0000 7f33517fa640 1 -- 172.21.15.105:0/3273481766 <== mds.0 v2:172.21.15.148:6832/2696773413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56382861eb50 con 0x56382880e970 2023-11-28T18:26:06.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.960+0000 7f33517fa640 10 client.6150 handle_command_reply: tid=0 2023-11-28T18:26:06.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.960+0000 7f337c2c0000 1 client.6150 shutdown 2023-11-28T18:26:06.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.960+0000 7f337c2c0000 1 -- 172.21.15.105:0/3273481766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828854650 msgr2=0x56382880df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:06.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.960+0000 7f337c2c0000 1 --2- 172.21.15.105:0/3273481766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828854650 0x56382880df60 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f336c00e9b0 tx=0x7f336c055c90 comp rx=0 tx=0).stop 2023-11-28T18:26:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.960+0000 7f337c2c0000 1 -- 172.21.15.105:0/3273481766 shutdown_connections 2023-11-28T18:26:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.960+0000 7f337c2c0000 1 --2- 172.21.15.105:0/3273481766 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x56382880e970 0x563828865ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.960+0000 7f337c2c0000 1 --2- 172.21.15.105:0/3273481766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828854650 0x56382880df60 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.960+0000 7f337c2c0000 1 --2- 172.21.15.105:0/3273481766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563828853d30 0x56382880da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.960+0000 7f33517fa640 0 client.6150 ms_handle_reset on v2:172.21.15.148:6832/2696773413 2023-11-28T18:26:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.960+0000 7f337c2c0000 1 --2- 172.21.15.105:0/3273481766 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x563828807500 0x56382880d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.960+0000 7f337c2c0000 1 -- 172.21.15.105:0/3273481766 >> 172.21.15.105:0/3273481766 conn(0x5638288348c0 msgr2=0x5638288066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.960+0000 7f337c2c0000 1 -- 172.21.15.105:0/3273481766 shutdown_connections 2023-11-28T18:26:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.960+0000 7f337c2c0000 1 -- 172.21.15.105:0/3273481766 wait complete. 2023-11-28T18:26:06.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.960+0000 7f337c2c0000 20 client.6150 trim_cache size 0 max 16384 2023-11-28T18:26:06.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.972+0000 7f337c2c0000 1 Processor -- start 2023-11-28T18:26:06.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.972+0000 7f337c2c0000 1 -- start start 2023-11-28T18:26:06.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f337c2c0000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563828807500 0x5638288333a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f337c2c0000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828853d30 0x5638288338e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f337c2c0000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x563828854650 0x563828865ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f337c2c0000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x563828312940 con 0x563828853d30 2023-11-28T18:26:06.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f337c2c0000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56382880ef80 con 0x563828807500 2023-11-28T18:26:06.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f337c2c0000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638286236c0 con 0x563828854650 2023-11-28T18:26:06.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f3371d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828853d30 0x5638288338e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f3371d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828853d30 0x5638288338e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46028/0 (socket says 172.21.15.105:46028) 2023-11-28T18:26:06.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f3371d9b640 1 -- 172.21.15.105:0/2070956849 learned_addr learned my addr 172.21.15.105:0/2070956849 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:06.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f3372d9d640 1 --2- 172.21.15.105:0/2070956849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x563828854650 0x563828865ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f337259c640 1 --2- 172.21.15.105:0/2070956849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563828807500 0x5638288333a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f3371d9b640 1 -- 172.21.15.105:0/2070956849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x563828854650 msgr2=0x563828865ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:06.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f3371d9b640 1 --2- 172.21.15.105:0/2070956849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x563828854650 0x563828865ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f3371d9b640 1 -- 172.21.15.105:0/2070956849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563828807500 msgr2=0x5638288333a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:06.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f3371d9b640 1 --2- 172.21.15.105:0/2070956849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563828807500 0x5638288333a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f3371d9b640 1 -- 172.21.15.105:0/2070956849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563828807370 con 0x563828853d30 2023-11-28T18:26:06.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f337259c640 1 --2- 172.21.15.105:0/2070956849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563828807500 0x5638288333a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:06.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f3371d9b640 1 --2- 172.21.15.105:0/2070956849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828853d30 0x5638288338e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3368007420 tx=0x7f336800c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66329ea771d343fe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:06.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f3360ff9640 1 -- 172.21.15.105:0/2070956849 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3368007cc0 con 0x563828853d30 2023-11-28T18:26:06.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f3360ff9640 1 -- 172.21.15.105:0/2070956849 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336800b070 con 0x563828853d30 2023-11-28T18:26:06.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f3360ff9640 1 -- 172.21.15.105:0/2070956849 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3368001ae0 con 0x563828853d30 2023-11-28T18:26:06.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f337c2c0000 1 -- 172.21.15.105:0/2070956849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828853d30 msgr2=0x5638288338e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:06.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f337c2c0000 1 --2- 172.21.15.105:0/2070956849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828853d30 0x5638288338e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3368007420 tx=0x7f336800c000 comp rx=0 tx=0).stop 2023-11-28T18:26:06.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f337c2c0000 1 -- 172.21.15.105:0/2070956849 shutdown_connections 2023-11-28T18:26:06.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f337c2c0000 1 --2- 172.21.15.105:0/2070956849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x563828854650 0x563828865ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f337c2c0000 1 --2- 172.21.15.105:0/2070956849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828853d30 0x5638288338e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f337c2c0000 1 --2- 172.21.15.105:0/2070956849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563828807500 0x5638288333a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f337c2c0000 1 -- 172.21.15.105:0/2070956849 >> 172.21.15.105:0/2070956849 conn(0x5638288348c0 msgr2=0x563828805720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:06.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f337c2c0000 1 -- 172.21.15.105:0/2070956849 shutdown_connections 2023-11-28T18:26:06.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f337c2c0000 1 -- 172.21.15.105:0/2070956849 wait complete. 2023-11-28T18:26:06.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f337c2c0000 1 -- start start 2023-11-28T18:26:06.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.976+0000 7f337c2c0000 1 Processor -- start 2023-11-28T18:26:06.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337c2c0000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828807500 0x5638288b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337c2c0000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563828853d30 0x5638288b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337c2c0000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x563828854650 0x56382886fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337c2c0000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x563828661b20 con 0x563828807500 2023-11-28T18:26:06.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337c2c0000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638285f0420 con 0x563828853d30 2023-11-28T18:26:06.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337c2c0000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638287227d0 con 0x563828854650 2023-11-28T18:26:06.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337c2c0000 10 client.0 _ll_register_callbacks cb 0x5638286d0060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:06.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828807500 0x5638288b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828807500 0x5638288b4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46042/0 (socket says 172.21.15.105:46042) 2023-11-28T18:26:06.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337259c640 1 -- 172.21.15.105:0/3499206775 learned_addr learned my addr 172.21.15.105:0/3499206775 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:06.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f3371d9b640 1 --2- 172.21.15.105:0/3499206775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563828853d30 0x5638288b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f3372d9d640 1 --2- 172.21.15.105:0/3499206775 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x563828854650 0x56382886fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337259c640 1 -- 172.21.15.105:0/3499206775 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x563828854650 msgr2=0x56382886fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:06.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337259c640 1 --2- 172.21.15.105:0/3499206775 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x563828854650 0x56382886fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337259c640 1 -- 172.21.15.105:0/3499206775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563828853d30 msgr2=0x5638288b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:06.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337259c640 1 --2- 172.21.15.105:0/3499206775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563828853d30 0x5638288b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337259c640 1 -- 172.21.15.105:0/3499206775 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563828351890 con 0x563828807500 2023-11-28T18:26:06.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337259c640 1 --2- 172.21.15.105:0/3499206775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828807500 0x5638288b4810 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f336401c9c0 tx=0x7f336401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04fa75799aad76c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:06.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f33517fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:06.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f33517fa640 1 -- 172.21.15.105:0/3499206775 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33640c9330 con 0x563828807500 2023-11-28T18:26:06.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f33517fa640 1 -- 172.21.15.105:0/3499206775 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f33640c9490 con 0x563828807500 2023-11-28T18:26:06.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337c2c0000 1 -- 172.21.15.105:0/3499206775 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563828658110 con 0x563828807500 2023-11-28T18:26:06.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f33517fa640 1 -- 172.21.15.105:0/3499206775 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33640c9650 con 0x563828807500 2023-11-28T18:26:06.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337c2c0000 10 client.6156 fetch_fsmap learned FSMap version 35 2023-11-28T18:26:06.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337c2c0000 1 -- 172.21.15.105:0/3499206775 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x563828600490 con 0x563828807500 2023-11-28T18:26:06.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f33517fa640 1 -- 172.21.15.105:0/3499206775 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563828658110 con 0x563828807500 2023-11-28T18:26:06.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f33517fa640 1 -- 172.21.15.105:0/3499206775 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f33640c3040 con 0x563828807500 2023-11-28T18:26:06.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337c2c0000 10 client.6156 fetch_fsmap finished waiting for FSMap version 35 2023-11-28T18:26:06.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337c2c0000 10 client.6156 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T18:26:06.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337c2c0000 20 client.6156 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:06.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337c2c0000 1 --2- 172.21.15.105:0/3499206775 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x5638288bb670 0x563828870580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:06.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337c2c0000 4 client.6156 mds_command: new command op to 5842 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:26:06.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337c2c0000 1 -- 172.21.15.105:0/3499206775 --> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563828686650 con 0x5638288bb670 2023-11-28T18:26:06.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f3371d9b640 1 --2- 172.21.15.105:0/3499206775 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x5638288bb670 0x563828870580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:06.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f3371d9b640 1 --2- 172.21.15.105:0/3499206775 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x5638288bb670 0x563828870580 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd4047c91954b2a8 server_cookie=1cc62a5ab8f52e79 in_seq=0 out_seq=0 2023-11-28T18:26:06.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f33517fa640 10 client.6156 ms_handle_connect on v2:172.21.15.148:6832/2696773413 2023-11-28T18:26:06.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f33517fa640 1 -- 172.21.15.105:0/3499206775 <== mds.0 v2:172.21.15.148:6832/2696773413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x563828686650 con 0x5638288bb670 2023-11-28T18:26:06.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f33517fa640 10 client.6156 handle_command_reply: tid=0 2023-11-28T18:26:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337c2c0000 1 client.6156 shutdown 2023-11-28T18:26:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337c2c0000 1 -- 172.21.15.105:0/3499206775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828807500 msgr2=0x5638288b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.980+0000 7f337c2c0000 1 --2- 172.21.15.105:0/3499206775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828807500 0x5638288b4810 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f336401c9c0 tx=0x7f336401ac90 comp rx=0 tx=0).stop 2023-11-28T18:26:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.984+0000 7f337c2c0000 1 -- 172.21.15.105:0/3499206775 shutdown_connections 2023-11-28T18:26:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.984+0000 7f337c2c0000 1 --2- 172.21.15.105:0/3499206775 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x5638288bb670 0x563828870580 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.984+0000 7f337c2c0000 1 --2- 172.21.15.105:0/3499206775 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x563828854650 0x56382886fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.984+0000 7f337c2c0000 1 --2- 172.21.15.105:0/3499206775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563828853d30 0x5638288b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.984+0000 7f33517fa640 0 client.6156 ms_handle_reset on v2:172.21.15.148:6832/2696773413 2023-11-28T18:26:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.984+0000 7f337c2c0000 1 --2- 172.21.15.105:0/3499206775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563828807500 0x5638288b4810 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.984+0000 7f337c2c0000 1 -- 172.21.15.105:0/3499206775 >> 172.21.15.105:0/3499206775 conn(0x5638288348c0 msgr2=0x563828805fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.984+0000 7f337c2c0000 1 -- 172.21.15.105:0/3499206775 shutdown_connections 2023-11-28T18:26:06.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.984+0000 7f337c2c0000 1 -- 172.21.15.105:0/3499206775 wait complete. 2023-11-28T18:26:06.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.984+0000 7f337c2c0000 20 client.6156 trim_cache size 0 max 16384 2023-11-28T18:26:06.992 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:26:06.992 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:26:06.992 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2023-11-28T18:26:06.993 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5842, 2023-11-28T18:26:06.993 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:26:06.993 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:26:06.993 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:26:06.993 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 5.8199430720000001, 2023-11-28T18:26:06.993 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 35, 2023-11-28T18:26:06.993 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 51, 2023-11-28T18:26:06.993 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 51, 2023-11-28T18:26:06.993 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 10.687188372 2023-11-28T18:26:06.993 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:26:06.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.984+0000 7f3378f65640 1 -- 172.21.15.105:0/4186318382 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f33540668d0 msgr2=0x7f3354068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:06.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.984+0000 7f3378f65640 1 --2- 172.21.15.105:0/4186318382 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f33540668d0 0x7f3354068d90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f33640af000 tx=0x7f3364008040 comp rx=0 tx=0).stop 2023-11-28T18:26:06.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.984+0000 7f3378f65640 1 -- 172.21.15.105:0/4186318382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3374154c30 msgr2=0x7f33741550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:06.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.984+0000 7f3378f65640 1 --2- 172.21.15.105:0/4186318382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3374154c30 0x7f33741550e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f336c009fd0 tx=0x7f336c00e4b0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.984+0000 7f3378f65640 1 -- 172.21.15.105:0/4186318382 shutdown_connections 2023-11-28T18:26:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.984+0000 7f3378f65640 1 --2- 172.21.15.105:0/4186318382 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f33540668d0 0x7f3354068d90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.984+0000 7f3378f65640 1 --2- 172.21.15.105:0/4186318382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3374154c30 0x7f33741550e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.984+0000 7f3378f65640 1 --2- 172.21.15.105:0/4186318382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f337415b030 0x7f3374099f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.984+0000 7f3378f65640 1 --2- 172.21.15.105:0/4186318382 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33740b4290 0x7f337414b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.984+0000 7f3378f65640 1 -- 172.21.15.105:0/4186318382 >> 172.21.15.105:0/4186318382 conn(0x7f33740ceb90 msgr2=0x7f33740b19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.984+0000 7f3378f65640 1 -- 172.21.15.105:0/4186318382 shutdown_connections 2023-11-28T18:26:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:06.984+0000 7f3378f65640 1 -- 172.21.15.105:0/4186318382 wait complete. 2023-11-28T18:26:07.005 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T18:26:07.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.140+0000 7f9da565a640 1 Processor -- start 2023-11-28T18:26:07.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.140+0000 7f9da565a640 1 -- start start 2023-11-28T18:26:07.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.140+0000 7f9da565a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9da015d4f0 0x7f9da00d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:07.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.140+0000 7f9da565a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da00b25b0 con 0x7f9da00b2d20 2023-11-28T18:26:07.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.140+0000 7f9da565a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da00cea30 con 0x7f9da00d3c30 2023-11-28T18:26:07.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.140+0000 7f9da565a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da00b2900 con 0x7f9da015d4f0 2023-11-28T18:26:07.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.140+0000 7f9d9f7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9da00b2d20 0x7f9da00b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57992/0 (socket says 172.21.15.105:57992) 2023-11-28T18:26:07.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.140+0000 7f9d9f7fe640 1 -- 172.21.15.105:0/1058033858 learned_addr learned my addr 172.21.15.105:0/1058033858 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:07.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.140+0000 7f9d9effd640 1 --2- 172.21.15.105:0/1058033858 >> v2:172.21.15.148:3300/0 conn(0x7f9da015d4f0 0x7f9da00d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:07.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.140+0000 7f9d9e7fc640 1 -- 172.21.15.105:0/1058033858 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112221811 0 0) 0x7f9da00b25b0 con 0x7f9da00b2d20 2023-11-28T18:26:07.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.140+0000 7f9d9e7fc640 1 -- 172.21.15.105:0/1058033858 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d8c003530 con 0x7f9da00b2d20 2023-11-28T18:26:07.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.140+0000 7f9d9effd640 1 -- 172.21.15.105:0/1058033858 >> v1:172.21.15.148:6789/0 conn(0x7f9da00d3c30 legacy=0x7f9da00d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:07.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.140+0000 7f9d9effd640 1 -- 172.21.15.105:0/1058033858 >> v1:172.21.15.105:6789/0 conn(0x7f9da00b2d20 legacy=0x7f9da00b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:07.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.140+0000 7f9d9effd640 1 -- 172.21.15.105:0/1058033858 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da00ced10 con 0x7f9da015d4f0 2023-11-28T18:26:07.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.144+0000 7f9d9effd640 1 --2- 172.21.15.105:0/1058033858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da015d4f0 0x7f9da00d0e10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9d900097c0 tx=0x7f9d9001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42016e6ccfecbb2a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:07.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.144+0000 7f9d9e7fc640 1 -- 172.21.15.105:0/1058033858 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d900ad660 con 0x7f9da015d4f0 2023-11-28T18:26:07.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.144+0000 7f9d9e7fc640 1 -- 172.21.15.105:0/1058033858 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d900ad7c0 con 0x7f9da015d4f0 2023-11-28T18:26:07.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.144+0000 7f9d9e7fc640 1 -- 172.21.15.105:0/1058033858 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d900adb30 con 0x7f9da015d4f0 2023-11-28T18:26:07.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.144+0000 7f9da565a640 1 -- 172.21.15.105:0/1058033858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da015d4f0 msgr2=0x7f9da00d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:07.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.144+0000 7f9da565a640 1 --2- 172.21.15.105:0/1058033858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da015d4f0 0x7f9da00d0e10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9d900097c0 tx=0x7f9d9001c3c0 comp rx=0 tx=0).stop 2023-11-28T18:26:07.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.144+0000 7f9da565a640 1 -- 172.21.15.105:0/1058033858 shutdown_connections 2023-11-28T18:26:07.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.144+0000 7f9da565a640 1 --2- 172.21.15.105:0/1058033858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da015d4f0 0x7f9da00d0e10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:07.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.144+0000 7f9da565a640 1 -- 172.21.15.105:0/1058033858 >> 172.21.15.105:0/1058033858 conn(0x7f9da00d3030 msgr2=0x7f9da00d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:07.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.144+0000 7f9da565a640 1 -- 172.21.15.105:0/1058033858 shutdown_connections 2023-11-28T18:26:07.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.144+0000 7f9da565a640 1 -- 172.21.15.105:0/1058033858 wait complete. 2023-11-28T18:26:07.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9da565a640 1 Processor -- start 2023-11-28T18:26:07.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9da565a640 1 -- start start 2023-11-28T18:26:07.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9da565a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da00b2d20 0x7f9da015fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:07.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9da565a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da00d3c30 0x7f9da015c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:07.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9da565a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9da015d4f0 0x7f9da0154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:07.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9da565a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da00d0a20 con 0x7f9da00d3c30 2023-11-28T18:26:07.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9da565a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da015f6f0 con 0x7f9da00b2d20 2023-11-28T18:26:07.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9da565a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da015f0c0 con 0x7f9da015d4f0 2023-11-28T18:26:07.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9d967fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da00d3c30 0x7f9da015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:07.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9d967fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da00d3c30 0x7f9da015c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46058/0 (socket says 172.21.15.105:46058) 2023-11-28T18:26:07.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9d967fc640 1 -- 172.21.15.105:0/1360886384 learned_addr learned my addr 172.21.15.105:0/1360886384 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:07.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9d9effd640 1 --2- 172.21.15.105:0/1360886384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da00b2d20 0x7f9da015fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:07.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9d9f7fe640 1 --2- 172.21.15.105:0/1360886384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9da015d4f0 0x7f9da0154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:07.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9d967fc640 1 -- 172.21.15.105:0/1360886384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9da015d4f0 msgr2=0x7f9da0154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:07.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9d967fc640 1 --2- 172.21.15.105:0/1360886384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9da015d4f0 0x7f9da0154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:07.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9d967fc640 1 -- 172.21.15.105:0/1360886384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da00b2d20 msgr2=0x7f9da015fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:07.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9d967fc640 1 --2- 172.21.15.105:0/1360886384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da00b2d20 0x7f9da015fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:07.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9d967fc640 1 -- 172.21.15.105:0/1360886384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d90009420 con 0x7f9da00d3c30 2023-11-28T18:26:07.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9d9f7fe640 1 --2- 172.21.15.105:0/1360886384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9da015d4f0 0x7f9da0154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:07.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9d967fc640 1 --2- 172.21.15.105:0/1360886384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da00d3c30 0x7f9da015c710 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800c730 tx=0x7f9d8800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:07.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9d97fff640 1 -- 172.21.15.105:0/1360886384 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d88006d60 con 0x7f9da00d3c30 2023-11-28T18:26:07.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9d97fff640 1 -- 172.21.15.105:0/1360886384 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8800ce40 con 0x7f9da00d3c30 2023-11-28T18:26:07.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9da565a640 1 -- 172.21.15.105:0/1360886384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da00cbfb0 con 0x7f9da00d3c30 2023-11-28T18:26:07.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9d97fff640 1 -- 172.21.15.105:0/1360886384 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d88003bc0 con 0x7f9da00d3c30 2023-11-28T18:26:07.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.148+0000 7f9da565a640 1 -- 172.21.15.105:0/1360886384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da00cc570 con 0x7f9da00d3c30 2023-11-28T18:26:07.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.152+0000 7f9da565a640 1 -- 172.21.15.105:0/1360886384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da015f830 con 0x7f9da00d3c30 2023-11-28T18:26:07.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.152+0000 7f9d97fff640 1 -- 172.21.15.105:0/1360886384 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f9d88003d20 con 0x7f9da00d3c30 2023-11-28T18:26:07.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.152+0000 7f9d97fff640 1 --2- 172.21.15.105:0/1360886384 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d800668d0 0x7f9d80068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:07.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.152+0000 7f9d97fff640 1 -- 172.21.15.105:0/1360886384 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9d88084150 con 0x7f9da00d3c30 2023-11-28T18:26:07.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.152+0000 7f9d9effd640 1 --2- 172.21.15.105:0/1360886384 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d800668d0 0x7f9d80068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:07.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.152+0000 7f9d9effd640 1 --2- 172.21.15.105:0/1360886384 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d800668d0 0x7f9d80068d90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9d900af000 tx=0x7f9d900a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:07.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.156+0000 7f9d97fff640 1 -- 172.21.15.105:0/1360886384 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9da015f830 con 0x7f9da00d3c30 2023-11-28T18:26:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.340+0000 7f9da565a640 1 -- 172.21.15.105:0/1360886384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9da015f830 con 0x7f9da00d3c30 2023-11-28T18:26:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.340+0000 7f9d97fff640 1 -- 172.21.15.105:0/1360886384 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9da015f830 con 0x7f9da00d3c30 2023-11-28T18:26:07.344 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T18:26:07.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.344+0000 7f9da565a640 1 -- 172.21.15.105:0/1360886384 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d800668d0 msgr2=0x7f9d80068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:07.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.344+0000 7f9da565a640 1 --2- 172.21.15.105:0/1360886384 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d800668d0 0x7f9d80068d90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9d900af000 tx=0x7f9d900a7000 comp rx=0 tx=0).stop 2023-11-28T18:26:07.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.344+0000 7f9da565a640 1 -- 172.21.15.105:0/1360886384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da00d3c30 msgr2=0x7f9da015c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:07.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.344+0000 7f9da565a640 1 --2- 172.21.15.105:0/1360886384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da00d3c30 0x7f9da015c710 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800c730 tx=0x7f9d8800cbf0 comp rx=0 tx=0).stop 2023-11-28T18:26:07.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.344+0000 7f9da565a640 1 -- 172.21.15.105:0/1360886384 shutdown_connections 2023-11-28T18:26:07.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.344+0000 7f9da565a640 1 --2- 172.21.15.105:0/1360886384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9da015d4f0 0x7f9da0154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:07.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.344+0000 7f9da565a640 1 --2- 172.21.15.105:0/1360886384 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d800668d0 0x7f9d80068d90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:07.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.344+0000 7f9da565a640 1 --2- 172.21.15.105:0/1360886384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da00d3c30 0x7f9da015c710 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:07.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.344+0000 7f9da565a640 1 --2- 172.21.15.105:0/1360886384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da00b2d20 0x7f9da015fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:07.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.344+0000 7f9da565a640 1 -- 172.21.15.105:0/1360886384 >> 172.21.15.105:0/1360886384 conn(0x7f9da00d3030 msgr2=0x7f9da00c98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:07.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.344+0000 7f9da565a640 1 -- 172.21.15.105:0/1360886384 shutdown_connections 2023-11-28T18:26:07.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:07.344+0000 7f9da565a640 1 -- 172.21.15.105:0/1360886384 wait complete. 2023-11-28T18:26:07.365 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T18:26:07.365 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T18:26:07.366 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2023-11-28T18:26:07.366 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T18:26:07.366 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T18:26:07.366 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T18:26:07.366 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T18:26:07.366 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T18:26:07.367 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:26:07.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:26:07.367 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2023-11-28T18:26:07.377 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T18:26:07.378 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T18:26:07.378 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T18:26:07.378 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:26:07.378 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-11-28T18:26:07.379 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:26:07.379 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:26:07.379 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:26:07.379 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2023-11-28T18:26:07.379 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:26:07.379 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:26:07.379 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2023-11-28T18:26:07.379 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:26:07.380 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:26:07.380 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T18:26:07.380 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:26:07.380 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-28T18:26:07.380 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42254sec preferred_lft 42254sec 2023-11-28T18:26:07.380 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2023-11-28T18:26:07.380 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:26:07.380 INFO:teuthology.orchestra.run.smithi105.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-28T18:26:07.380 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 02:42:cc:d4:87:d2 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:26:07.381 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-28T18:26:07.381 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:26:07.381 INFO:teuthology.orchestra.run.smithi105.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T18:26:07.381 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-28T18:26:07.381 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T18:26:07.381 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:26:07.381 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-28T18:26:07.381 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:26:07.381 INFO:teuthology.orchestra.run.smithi105.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T18:26:07.381 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:26:07.381 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-28T18:26:07.382 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:26:07.382 INFO:teuthology.orchestra.run.smithi105.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T18:26:07.382 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:26:07.382 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-28T18:26:07.382 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:26:07.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:26:07.383 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T18:26:07.443 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-11-28T18:26:07.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:26:07.446 DEBUG:teuthology.orchestra.run.smithi105:> route 2023-11-28T18:26:07.500 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2023-11-28T18:26:07.500 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T18:26:07.500 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-28T18:26:07.501 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T18:26:07.501 INFO:teuthology.orchestra.run.smithi105.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-28T18:26:07.501 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T18:26:07.501 INFO:teuthology.orchestra.run.smithi105.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:26:07.502 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:26:07.502 INFO:teuthology.orchestra.run.smithi105.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:26:07.502 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T18:26:07.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:26:07.502 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-11-28T18:26:07.502 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:26:07.503 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T18:26:07.503 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T18:26:07.503 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T18:26:07.503 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:26:07.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:26:07.592 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-11-28T18:26:07.602 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T18:26:07.602 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T18:26:07.603 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:26:07.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:26:07.603 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2023-11-28T18:26:07.665 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-28T18:26:07.665 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:26:07.665 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T18:26:07.718 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:26:07.718 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:26:07.718 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T18:26:07.769 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:26:07.833 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:26:07.886 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:26:07.887 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:26:07.887 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:26:07.887 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:26:07.887 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:26:07.887 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:26:07.887 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:26:07.887 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:26:07.887 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:26:07.887 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:26:07.887 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:26:07.887 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:26:07.888 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:26:07.888 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:26:07.888 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:26:07.888 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:26:07.888 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:26:07.888 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:26:07.888 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:26:07.888 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:26:07.888 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:26:07.888 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:26:07.888 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:26:07.888 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:26:07.889 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:26:07.889 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:26:07.889 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:26:07.889 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:26:07.889 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:26:07.889 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:26:07.889 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:26:07.889 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:26:07.889 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:26:07.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:26:07.890 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:26:07.941 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T18:26:07.941 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-28T18:26:07.992 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:26:08.013 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:26:08.070 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:26:08.070 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:26:08.070 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:26:08.070 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:26:08.070 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:26:08.070 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:26:08.071 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:26:08.071 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:26:08.071 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:26:08.071 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:26:08.071 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:26:08.071 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:26:08.071 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:26:08.071 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:26:08.071 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:26:08.071 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:26:08.071 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:26:08.071 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:26:08.072 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:26:08.072 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:26:08.072 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:26:08.072 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:26:08.072 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:26:08.072 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:26:08.072 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:26:08.072 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:26:08.072 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:26:08.072 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:26:08.072 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:26:08.073 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:26:08.073 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:26:08.073 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:26:08.073 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:26:08.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:26:08.073 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:26:08.090 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:2023-11-28T18:26:08.084+0000 7f53ba845580 -1 init, newargv = 0x5637cef9c170 newargc=15 2023-11-28T18:26:08.090 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[52772]: starting ceph client 2023-11-28T18:26:08.095 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[52772]: starting fuse 2023-11-28T18:26:08.100 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-11-28T18:26:08.100 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-28T18:26:08.101 DEBUG:teuthology.orchestra.run.smithi105:> sudo stdin-killer -- python3 -c ' 2023-11-28T18:26:08.101 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2023-11-28T18:26:08.101 DEBUG:teuthology.orchestra.run.smithi105:> import re 2023-11-28T18:26:08.101 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-11-28T18:26:08.101 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2023-11-28T18:26:08.101 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:26:08.101 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2023-11-28T18:26:08.101 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-28T18:26:08.101 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2023-11-28T18:26:08.101 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-28T18:26:08.102 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:26:08.102 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2023-11-28T18:26:08.102 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2023-11-28T18:26:08.102 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2023-11-28T18:26:08.102 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2023-11-28T18:26:08.102 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:26:08.102 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2023-11-28T18:26:08.102 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T18:26:08.102 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T18:26:08.102 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T18:26:08.102 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2023-11-28T18:26:08.102 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2023-11-28T18:26:08.103 DEBUG:teuthology.orchestra.run.smithi105:> return f 2023-11-28T18:26:08.103 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T18:26:08.103 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:26:08.103 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.0")) 2023-11-28T18:26:08.103 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-11-28T18:26:08.229 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.0.52772.asok 2023-11-28T18:26:08.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-28T18:26:08.240 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52772.asok 2023-11-28T18:26:08.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:26:08.241 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52772.asok status 2023-11-28T18:26:08.409 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:26:08.409 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2023-11-28T18:26:08.409 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "47c1c547372ed5bff7984647dc200bac0e019caf", 2023-11-28T18:26:08.409 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev)", 2023-11-28T18:26:08.409 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "0", 2023-11-28T18:26:08.409 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2023-11-28T18:26:08.409 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-28T18:26:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "52772", 2023-11-28T18:26:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2023-11-28T18:26:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2023-11-28T18:26:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2023-11-28T18:26:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 6168, 2023-11-28T18:26:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2023-11-28T18:26:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2023-11-28T18:26:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2023-11-28T18:26:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 6168 2023-11-28T18:26:08.410 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-11-28T18:26:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-11-28T18:26:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-11-28T18:26:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 3398543906 2023-11-28T18:26:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-11-28T18:26:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-11-28T18:26:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-11-28T18:26:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 3398543906 2023-11-28T18:26:08.411 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.6168 192.168.0.1:0/3398543906", 2023-11-28T18:26:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/3398543906", 2023-11-28T18:26:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2023-11-28T18:26:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 35, 2023-11-28T18:26:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 51, 2023-11-28T18:26:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2023-11-28T18:26:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2023-11-28T18:26:08.412 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2023-11-28T18:26:08.412 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:26:08.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:26:08.424 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2023-11-28T18:26:08.424 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-28T18:26:08.434 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2023-11-28T18:26:08.434 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-28T18:26:08.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:26:08.434 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T18:26:08.498 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random True 2023-11-28T18:26:08.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.676+0000 7f411ad0f640 1 Processor -- start 2023-11-28T18:26:08.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.676+0000 7f411ad0f640 1 -- start start 2023-11-28T18:26:08.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.676+0000 7f411ad0f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f411415d4f0 0x7f41140d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:08.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.676+0000 7f411ad0f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41140b2d20 0x7f41140b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:08.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.676+0000 7f411ad0f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41140b25b0 con 0x7f41140d3c30 2023-11-28T18:26:08.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.676+0000 7f411ad0f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41140b2900 con 0x7f411415d4f0 2023-11-28T18:26:08.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.676+0000 7f411ad0f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41140d0a20 con 0x7f41140b2d20 2023-11-28T18:26:08.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.676+0000 7f4118aab640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f411415d4f0 0x7f41140d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:08.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.676+0000 7f4118aab640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f411415d4f0 0x7f41140d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46062/0 (socket says 172.21.15.105:46062) 2023-11-28T18:26:08.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.676+0000 7f4118aab640 1 -- 172.21.15.105:0/4094528958 learned_addr learned my addr 172.21.15.105:0/4094528958 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:08.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.676+0000 7f41192ac640 1 --2- 172.21.15.105:0/4094528958 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41140b2d20 0x7f41140b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:08.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.676+0000 7f4118aab640 1 -- 172.21.15.105:0/4094528958 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41140b2d20 msgr2=0x7f41140b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:08.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.676+0000 7f4118aab640 1 --2- 172.21.15.105:0/4094528958 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41140b2d20 0x7f41140b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:08.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.676+0000 7f4118aab640 1 -- 172.21.15.105:0/4094528958 >> v1:172.21.15.148:6789/0 conn(0x7f41140d3c30 legacy=0x7f41140d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:08.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.676+0000 7f4118aab640 1 -- 172.21.15.105:0/4094528958 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41140cea30 con 0x7f411415d4f0 2023-11-28T18:26:08.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.680+0000 7f4118aab640 1 --2- 172.21.15.105:0/4094528958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f411415d4f0 0x7f41140d0e10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4110009870 tx=0x7f411001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e273a0eea0e2e92 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:08.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.680+0000 7f410b7fe640 1 -- 172.21.15.105:0/4094528958 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41100ab680 con 0x7f411415d4f0 2023-11-28T18:26:08.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.680+0000 7f410b7fe640 1 -- 172.21.15.105:0/4094528958 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41100ab7e0 con 0x7f411415d4f0 2023-11-28T18:26:08.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.680+0000 7f410b7fe640 1 -- 172.21.15.105:0/4094528958 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41100abb70 con 0x7f411415d4f0 2023-11-28T18:26:08.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.680+0000 7f411ad0f640 1 -- 172.21.15.105:0/4094528958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f411415d4f0 msgr2=0x7f41140d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:08.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.680+0000 7f411ad0f640 1 --2- 172.21.15.105:0/4094528958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f411415d4f0 0x7f41140d0e10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4110009870 tx=0x7f411001c480 comp rx=0 tx=0).stop 2023-11-28T18:26:08.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.680+0000 7f411ad0f640 1 -- 172.21.15.105:0/4094528958 shutdown_connections 2023-11-28T18:26:08.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.680+0000 7f411ad0f640 1 --2- 172.21.15.105:0/4094528958 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41140b2d20 0x7f41140b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:08.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.680+0000 7f411ad0f640 1 --2- 172.21.15.105:0/4094528958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f411415d4f0 0x7f41140d0e10 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:08.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.680+0000 7f411ad0f640 1 -- 172.21.15.105:0/4094528958 >> 172.21.15.105:0/4094528958 conn(0x7f41140d3030 msgr2=0x7f41140d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:08.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.680+0000 7f411ad0f640 1 -- 172.21.15.105:0/4094528958 shutdown_connections 2023-11-28T18:26:08.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.680+0000 7f411ad0f640 1 -- 172.21.15.105:0/4094528958 wait complete. 2023-11-28T18:26:08.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f411ad0f640 1 Processor -- start 2023-11-28T18:26:08.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f411ad0f640 1 -- start start 2023-11-28T18:26:08.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f411ad0f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41140b2d20 0x7f411414c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:08.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f411ad0f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41140d3c30 0x7f41140ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:08.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f411ad0f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f411415d4f0 0x7f4114159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:08.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f411ad0f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411415b7b0 con 0x7f411415d4f0 2023-11-28T18:26:08.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f411ad0f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41140b5bb0 con 0x7f41140b2d20 2023-11-28T18:26:08.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f411ad0f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41140caae0 con 0x7f41140d3c30 2023-11-28T18:26:08.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f41192ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f411415d4f0 0x7f4114159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:08.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f41192ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f411415d4f0 0x7f4114159490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46068/0 (socket says 172.21.15.105:46068) 2023-11-28T18:26:08.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f41192ac640 1 -- 172.21.15.105:0/3176600351 learned_addr learned my addr 172.21.15.105:0/3176600351 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:08.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f410bfff640 1 --2- 172.21.15.105:0/3176600351 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41140d3c30 0x7f41140ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:08.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f4118aab640 1 --2- 172.21.15.105:0/3176600351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41140b2d20 0x7f411414c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:08.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f41192ac640 1 -- 172.21.15.105:0/3176600351 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41140d3c30 msgr2=0x7f41140ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:08.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f41192ac640 1 --2- 172.21.15.105:0/3176600351 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41140d3c30 0x7f41140ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:08.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f41192ac640 1 -- 172.21.15.105:0/3176600351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41140b2d20 msgr2=0x7f411414c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:08.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f41192ac640 1 --2- 172.21.15.105:0/3176600351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41140b2d20 0x7f411414c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:08.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f41192ac640 1 -- 172.21.15.105:0/3176600351 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41100094d0 con 0x7f411415d4f0 2023-11-28T18:26:08.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f410bfff640 1 --2- 172.21.15.105:0/3176600351 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41140d3c30 0x7f41140ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:08.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f41192ac640 1 --2- 172.21.15.105:0/3176600351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f411415d4f0 0x7f4114159490 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f410c007e60 tx=0x7f410c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:08.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f41097fa640 1 -- 172.21.15.105:0/3176600351 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c014420 con 0x7f411415d4f0 2023-11-28T18:26:08.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f41097fa640 1 -- 172.21.15.105:0/3176600351 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410c0145a0 con 0x7f411415d4f0 2023-11-28T18:26:08.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f411ad0f640 1 -- 172.21.15.105:0/3176600351 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411415ae10 con 0x7f411415d4f0 2023-11-28T18:26:08.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f41097fa640 1 -- 172.21.15.105:0/3176600351 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c014950 con 0x7f411415d4f0 2023-11-28T18:26:08.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.684+0000 7f411ad0f640 1 -- 172.21.15.105:0/3176600351 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411415b3d0 con 0x7f411415d4f0 2023-11-28T18:26:08.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.688+0000 7f411ad0f640 1 -- 172.21.15.105:0/3176600351 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411415b8f0 con 0x7f411415d4f0 2023-11-28T18:26:08.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.688+0000 7f41097fa640 1 -- 172.21.15.105:0/3176600351 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f410c014ab0 con 0x7f411415d4f0 2023-11-28T18:26:08.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.688+0000 7f41097fa640 1 --2- 172.21.15.105:0/3176600351 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f41000668d0 0x7f4100068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:08.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.688+0000 7f41097fa640 1 -- 172.21.15.105:0/3176600351 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f410c084ea0 con 0x7f411415d4f0 2023-11-28T18:26:08.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.688+0000 7f4118aab640 1 --2- 172.21.15.105:0/3176600351 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f41000668d0 0x7f4100068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:08.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.688+0000 7f4118aab640 1 --2- 172.21.15.105:0/3176600351 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f41000668d0 0x7f4100068d90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f41100af000 tx=0x7f41100a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:08.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.692+0000 7f41097fa640 1 -- 172.21.15.105:0/3176600351 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f411415b8f0 con 0x7f411415d4f0 2023-11-28T18:26:08.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.840+0000 7f411ad0f640 1 -- 172.21.15.105:0/3176600351 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random}] v 0) v1 -- 0x7f411415b8f0 con 0x7f411415d4f0 2023-11-28T18:26:08.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.848+0000 7f41097fa640 1 -- 172.21.15.105:0/3176600351 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random}]=0 v20)=0 v20) v1 ==== 128+0+0 (secure 0 0 0) 0x7f411415b8f0 con 0x7f411415d4f0 2023-11-28T18:26:08.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.848+0000 7f411ad0f640 1 -- 172.21.15.105:0/3176600351 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f41000668d0 msgr2=0x7f4100068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:08.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.848+0000 7f411ad0f640 1 --2- 172.21.15.105:0/3176600351 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f41000668d0 0x7f4100068d90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f41100af000 tx=0x7f41100a7000 comp rx=0 tx=0).stop 2023-11-28T18:26:08.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.848+0000 7f411ad0f640 1 -- 172.21.15.105:0/3176600351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f411415d4f0 msgr2=0x7f4114159490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:08.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.848+0000 7f411ad0f640 1 --2- 172.21.15.105:0/3176600351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f411415d4f0 0x7f4114159490 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f410c007e60 tx=0x7f410c00e4b0 comp rx=0 tx=0).stop 2023-11-28T18:26:08.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.848+0000 7f411ad0f640 1 -- 172.21.15.105:0/3176600351 shutdown_connections 2023-11-28T18:26:08.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.848+0000 7f411ad0f640 1 --2- 172.21.15.105:0/3176600351 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f41000668d0 0x7f4100068d90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:08.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.848+0000 7f411ad0f640 1 --2- 172.21.15.105:0/3176600351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f411415d4f0 0x7f4114159490 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:08.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.848+0000 7f411ad0f640 1 --2- 172.21.15.105:0/3176600351 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41140d3c30 0x7f41140ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:08.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.848+0000 7f411ad0f640 1 --2- 172.21.15.105:0/3176600351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41140b2d20 0x7f411414c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:08.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.848+0000 7f411ad0f640 1 -- 172.21.15.105:0/3176600351 >> 172.21.15.105:0/3176600351 conn(0x7f41140d3030 msgr2=0x7f41140d1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:08.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.848+0000 7f411ad0f640 1 -- 172.21.15.105:0/3176600351 shutdown_connections 2023-11-28T18:26:08.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:08.848+0000 7f411ad0f640 1 -- 172.21.15.105:0/3176600351 wait complete. 2023-11-28T18:26:08.871 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_distributed True 2023-11-28T18:26:09.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.008+0000 7fae27ead640 1 Processor -- start 2023-11-28T18:26:09.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.008+0000 7fae27ead640 1 -- start start 2023-11-28T18:26:09.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.008+0000 7fae27ead640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fae20159f40 0x7fae200caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:09.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.008+0000 7fae27ead640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae200ad850 0x7fae200701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:09.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.008+0000 7fae27ead640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2006bf50 con 0x7fae200cc760 2023-11-28T18:26:09.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.008+0000 7fae27ead640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae200ccbc0 con 0x7fae20159f40 2023-11-28T18:26:09.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.008+0000 7fae27ead640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2015a320 con 0x7fae200ad850 2023-11-28T18:26:09.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.012+0000 7fae25448640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fae200cc760 0x7fae200ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:58006/0 (socket says 172.21.15.105:58006) 2023-11-28T18:26:09.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.012+0000 7fae25448640 1 -- 172.21.15.105:0/1008411491 learned_addr learned my addr 172.21.15.105:0/1008411491 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:09.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.012+0000 7fae25c49640 1 --2- 172.21.15.105:0/1008411491 >> v2:172.21.15.148:3300/0 conn(0x7fae20159f40 0x7fae200caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:09.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.012+0000 7fae2644a640 1 --2- 172.21.15.105:0/1008411491 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae200ad850 0x7fae200701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:09.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.012+0000 7fae24c47640 1 -- 172.21.15.105:0/1008411491 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2969301138 0 0) 0x7fae2006bf50 con 0x7fae200cc760 2023-11-28T18:26:09.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.012+0000 7fae24c47640 1 -- 172.21.15.105:0/1008411491 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae0c003530 con 0x7fae200cc760 2023-11-28T18:26:09.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.012+0000 7fae25c49640 1 -- 172.21.15.105:0/1008411491 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae200ad850 msgr2=0x7fae200701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:09.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.012+0000 7fae25c49640 1 --2- 172.21.15.105:0/1008411491 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae200ad850 0x7fae200701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:09.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.012+0000 7fae25c49640 1 -- 172.21.15.105:0/1008411491 >> v1:172.21.15.105:6789/0 conn(0x7fae200cc760 legacy=0x7fae200ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:09.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.012+0000 7fae25c49640 1 -- 172.21.15.105:0/1008411491 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae20071380 con 0x7fae20159f40 2023-11-28T18:26:09.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.012+0000 7fae2644a640 1 --2- 172.21.15.105:0/1008411491 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae200ad850 0x7fae200701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:26:09.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.012+0000 7fae25c49640 1 --2- 172.21.15.105:0/1008411491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae20159f40 0x7fae200caf60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0097c0 tx=0x7fae1c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c30d01080987b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:09.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.016+0000 7fae24c47640 1 -- 172.21.15.105:0/1008411491 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c0ac620 con 0x7fae20159f40 2023-11-28T18:26:09.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.016+0000 7fae24c47640 1 -- 172.21.15.105:0/1008411491 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1c0ac780 con 0x7fae20159f40 2023-11-28T18:26:09.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.016+0000 7fae24c47640 1 -- 172.21.15.105:0/1008411491 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c0acb10 con 0x7fae20159f40 2023-11-28T18:26:09.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.016+0000 7fae27ead640 1 -- 172.21.15.105:0/1008411491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae20159f40 msgr2=0x7fae200caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:09.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.016+0000 7fae27ead640 1 --2- 172.21.15.105:0/1008411491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae20159f40 0x7fae200caf60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0097c0 tx=0x7fae1c01c3c0 comp rx=0 tx=0).stop 2023-11-28T18:26:09.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.016+0000 7fae27ead640 1 -- 172.21.15.105:0/1008411491 shutdown_connections 2023-11-28T18:26:09.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.016+0000 7fae27ead640 1 --2- 172.21.15.105:0/1008411491 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae200ad850 0x7fae200701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:09.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.016+0000 7fae27ead640 1 --2- 172.21.15.105:0/1008411491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae20159f40 0x7fae200caf60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:09.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.016+0000 7fae27ead640 1 -- 172.21.15.105:0/1008411491 >> 172.21.15.105:0/1008411491 conn(0x7fae200ceb90 msgr2=0x7fae200cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:09.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.016+0000 7fae27ead640 1 -- 172.21.15.105:0/1008411491 shutdown_connections 2023-11-28T18:26:09.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.016+0000 7fae27ead640 1 -- 172.21.15.105:0/1008411491 wait complete. 2023-11-28T18:26:09.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.016+0000 7fae27ead640 1 Processor -- start 2023-11-28T18:26:09.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.016+0000 7fae27ead640 1 -- start start 2023-11-28T18:26:09.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.016+0000 7fae27ead640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae200ad850 0x7fae2009c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:09.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.016+0000 7fae27ead640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae200cc760 0x7fae200c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:09.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.016+0000 7fae27ead640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae20159f40 0x7fae2015a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:09.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.016+0000 7fae27ead640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae20099a50 con 0x7fae20159f40 2023-11-28T18:26:09.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.016+0000 7fae27ead640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae200c7f40 con 0x7fae200ad850 2023-11-28T18:26:09.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.016+0000 7fae27ead640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae20157b40 con 0x7fae200cc760 2023-11-28T18:26:09.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.020+0000 7fae2644a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae20159f40 0x7fae2015a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:09.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.020+0000 7fae2644a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae20159f40 0x7fae2015a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46084/0 (socket says 172.21.15.105:46084) 2023-11-28T18:26:09.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.020+0000 7fae2644a640 1 -- 172.21.15.105:0/4260080013 learned_addr learned my addr 172.21.15.105:0/4260080013 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:09.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.020+0000 7fae25c49640 1 --2- 172.21.15.105:0/4260080013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae200ad850 0x7fae2009c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:09.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.020+0000 7fae25448640 1 --2- 172.21.15.105:0/4260080013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae200cc760 0x7fae200c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:09.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.020+0000 7fae2644a640 1 -- 172.21.15.105:0/4260080013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae200cc760 msgr2=0x7fae200c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:09.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.020+0000 7fae2644a640 1 --2- 172.21.15.105:0/4260080013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae200cc760 0x7fae200c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:09.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.020+0000 7fae2644a640 1 -- 172.21.15.105:0/4260080013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae200ad850 msgr2=0x7fae2009c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:09.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.020+0000 7fae2644a640 1 --2- 172.21.15.105:0/4260080013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae200ad850 0x7fae2009c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:09.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.020+0000 7fae2644a640 1 -- 172.21.15.105:0/4260080013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae18009a70 con 0x7fae20159f40 2023-11-28T18:26:09.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.020+0000 7fae2644a640 1 --2- 172.21.15.105:0/4260080013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae20159f40 0x7fae2015a520 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fae1800efc0 tx=0x7fae18009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:09.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.020+0000 7fae167fc640 1 -- 172.21.15.105:0/4260080013 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae18009d70 con 0x7fae20159f40 2023-11-28T18:26:09.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.020+0000 7fae167fc640 1 -- 172.21.15.105:0/4260080013 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1800f040 con 0x7fae20159f40 2023-11-28T18:26:09.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.020+0000 7fae167fc640 1 -- 172.21.15.105:0/4260080013 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae180039a0 con 0x7fae20159f40 2023-11-28T18:26:09.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.020+0000 7fae27ead640 1 -- 172.21.15.105:0/4260080013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1c009420 con 0x7fae20159f40 2023-11-28T18:26:09.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.020+0000 7fae27ead640 1 -- 172.21.15.105:0/4260080013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae200c8430 con 0x7fae20159f40 2023-11-28T18:26:09.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.020+0000 7fae27ead640 1 -- 172.21.15.105:0/4260080013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae20067570 con 0x7fae20159f40 2023-11-28T18:26:09.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.020+0000 7fae167fc640 1 -- 172.21.15.105:0/4260080013 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fae180144a0 con 0x7fae20159f40 2023-11-28T18:26:09.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.024+0000 7fae167fc640 1 --2- 172.21.15.105:0/4260080013 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae040668d0 0x7fae04068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:09.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.024+0000 7fae167fc640 1 -- 172.21.15.105:0/4260080013 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fae180850d0 con 0x7fae20159f40 2023-11-28T18:26:09.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.024+0000 7fae25c49640 1 --2- 172.21.15.105:0/4260080013 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae040668d0 0x7fae04068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:09.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.024+0000 7fae25c49640 1 --2- 172.21.15.105:0/4260080013 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae040668d0 0x7fae04068d90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0ae000 tx=0x7fae1c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:09.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.028+0000 7fae167fc640 1 -- 172.21.15.105:0/4260080013 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fae18052280 con 0x7fae20159f40 2023-11-28T18:26:09.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.180+0000 7fae27ead640 1 -- 172.21.15.105:0/4260080013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_distributed}] v 0) v1 -- 0x7fae200c8650 con 0x7fae20159f40 2023-11-28T18:26:09.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.184+0000 7fae167fc640 1 -- 172.21.15.105:0/4260080013 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_distributed}]=0 v21)=0 v21) v1 ==== 133+0+0 (secure 0 0 0) 0x7fae180558a0 con 0x7fae20159f40 2023-11-28T18:26:09.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.184+0000 7fae27ead640 1 -- 172.21.15.105:0/4260080013 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae040668d0 msgr2=0x7fae04068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:09.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.184+0000 7fae27ead640 1 --2- 172.21.15.105:0/4260080013 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae040668d0 0x7fae04068d90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0ae000 tx=0x7fae1c0a7000 comp rx=0 tx=0).stop 2023-11-28T18:26:09.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.184+0000 7fae27ead640 1 -- 172.21.15.105:0/4260080013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae20159f40 msgr2=0x7fae2015a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:09.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.184+0000 7fae27ead640 1 --2- 172.21.15.105:0/4260080013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae20159f40 0x7fae2015a520 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fae1800efc0 tx=0x7fae18009f90 comp rx=0 tx=0).stop 2023-11-28T18:26:09.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.184+0000 7fae27ead640 1 -- 172.21.15.105:0/4260080013 shutdown_connections 2023-11-28T18:26:09.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.184+0000 7fae27ead640 1 --2- 172.21.15.105:0/4260080013 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae040668d0 0x7fae04068d90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:09.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.184+0000 7fae27ead640 1 --2- 172.21.15.105:0/4260080013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae20159f40 0x7fae2015a520 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:09.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.184+0000 7fae27ead640 1 --2- 172.21.15.105:0/4260080013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae200cc760 0x7fae200c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:09.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.184+0000 7fae27ead640 1 --2- 172.21.15.105:0/4260080013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae200ad850 0x7fae2009c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:09.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.184+0000 7fae27ead640 1 -- 172.21.15.105:0/4260080013 >> 172.21.15.105:0/4260080013 conn(0x7fae200ceb90 msgr2=0x7fae200ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:09.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.188+0000 7fae27ead640 1 -- 172.21.15.105:0/4260080013 shutdown_connections 2023-11-28T18:26:09.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.188+0000 7fae27ead640 1 -- 172.21.15.105:0/4260080013 wait complete. 2023-11-28T18:26:09.209 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random_max 1.0 2023-11-28T18:26:09.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.348+0000 7fdbbd89a640 1 Processor -- start 2023-11-28T18:26:09.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.348+0000 7fdbbd89a640 1 -- start start 2023-11-28T18:26:09.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.348+0000 7fdbbd89a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbb815d4f0 0x7fdbb80d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:09.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.348+0000 7fdbbd89a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdbb80d3c30 0x7fdbb80d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:09.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.348+0000 7fdbbd89a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdbb80d1890 0x7fdbb80cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:09.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.348+0000 7fdbbd89a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb80b2900 con 0x7fdbb80d1890 2023-11-28T18:26:09.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.348+0000 7fdbbd89a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb80b25b0 con 0x7fdbb80d3c30 2023-11-28T18:26:09.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.348+0000 7fdbbd89a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb80cea30 con 0x7fdbb815d4f0 2023-11-28T18:26:09.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.348+0000 7fdbb77fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdbb80d1890 0x7fdbb80cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:09.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.348+0000 7fdbb77fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdbb80d1890 0x7fdbb80cfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46100/0 (socket says 172.21.15.105:46100) 2023-11-28T18:26:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.348+0000 7fdbb77fe640 1 -- 172.21.15.105:0/2256276100 learned_addr learned my addr 172.21.15.105:0/2256276100 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.348+0000 7fdbb6ffd640 1 --2- 172.21.15.105:0/2256276100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbb815d4f0 0x7fdbb80d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.348+0000 7fdbae7fc640 1 --2- 172.21.15.105:0/2256276100 >> v2:172.21.15.148:3300/0 conn(0x7fdbb80d3c30 0x7fdbb80d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.348+0000 7fdbb77fe640 1 -- 172.21.15.105:0/2256276100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbb815d4f0 msgr2=0x7fdbb80d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.348+0000 7fdbb77fe640 1 --2- 172.21.15.105:0/2256276100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbb815d4f0 0x7fdbb80d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:09.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.348+0000 7fdbb77fe640 1 -- 172.21.15.105:0/2256276100 >> v2:172.21.15.148:3300/0 conn(0x7fdbb80d3c30 msgr2=0x7fdbb80d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:09.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.348+0000 7fdbb77fe640 1 --2- 172.21.15.105:0/2256276100 >> v2:172.21.15.148:3300/0 conn(0x7fdbb80d3c30 0x7fdbb80d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:09.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.348+0000 7fdbb77fe640 1 -- 172.21.15.105:0/2256276100 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb80ced10 con 0x7fdbb80d1890 2023-11-28T18:26:09.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.348+0000 7fdbae7fc640 1 --2- 172.21.15.105:0/2256276100 >> v2:172.21.15.148:3300/0 conn(0x7fdbb80d3c30 0x7fdbb80d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:09.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.352+0000 7fdbb77fe640 1 --2- 172.21.15.105:0/2256276100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbb80d1890 0x7fdbb80cfa10 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdbb808b2f0 tx=0x7fdbb001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b718abe08de438 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:09.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.352+0000 7fdbb67fc640 1 -- 172.21.15.105:0/2256276100 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb00a5730 con 0x7fdbb80d1890 2023-11-28T18:26:09.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.352+0000 7fdbb67fc640 1 -- 172.21.15.105:0/2256276100 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb00a5890 con 0x7fdbb80d1890 2023-11-28T18:26:09.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.352+0000 7fdbb67fc640 1 -- 172.21.15.105:0/2256276100 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb00a5c20 con 0x7fdbb80d1890 2023-11-28T18:26:09.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.352+0000 7fdbbd89a640 1 -- 172.21.15.105:0/2256276100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbb80d1890 msgr2=0x7fdbb80cfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:09.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.352+0000 7fdbbd89a640 1 --2- 172.21.15.105:0/2256276100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbb80d1890 0x7fdbb80cfa10 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdbb808b2f0 tx=0x7fdbb001c910 comp rx=0 tx=0).stop 2023-11-28T18:26:09.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.352+0000 7fdbbd89a640 1 -- 172.21.15.105:0/2256276100 shutdown_connections 2023-11-28T18:26:09.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.352+0000 7fdbbd89a640 1 --2- 172.21.15.105:0/2256276100 >> v2:172.21.15.148:3300/0 conn(0x7fdbb80d3c30 0x7fdbb80d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:09.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.352+0000 7fdbbd89a640 1 --2- 172.21.15.105:0/2256276100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbb80d1890 0x7fdbb80cfa10 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:09.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.352+0000 7fdbbd89a640 1 --2- 172.21.15.105:0/2256276100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbb815d4f0 0x7fdbb80d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:09.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.352+0000 7fdbbd89a640 1 -- 172.21.15.105:0/2256276100 >> 172.21.15.105:0/2256276100 conn(0x7fdbb80d3030 msgr2=0x7fdbb80d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:09.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.352+0000 7fdbbd89a640 1 -- 172.21.15.105:0/2256276100 shutdown_connections 2023-11-28T18:26:09.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.352+0000 7fdbbd89a640 1 -- 172.21.15.105:0/2256276100 wait complete. 2023-11-28T18:26:09.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.356+0000 7fdbbd89a640 1 Processor -- start 2023-11-28T18:26:09.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.356+0000 7fdbbd89a640 1 -- start start 2023-11-28T18:26:09.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.356+0000 7fdbbd89a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbb80d1890 0x7fdbb815c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:09.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.356+0000 7fdbbd89a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbb80d3c30 0x7fdbb80cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:09.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.356+0000 7fdbbd89a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbb815d4f0 0x7fdbb8154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:09.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.356+0000 7fdbbd89a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb80ca2a0 con 0x7fdbb80d1890 2023-11-28T18:26:09.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.356+0000 7fdbbd89a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb80ca580 con 0x7fdbb815d4f0 2023-11-28T18:26:09.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.356+0000 7fdbbd89a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb80ca860 con 0x7fdbb80d3c30 2023-11-28T18:26:09.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.356+0000 7fdbb6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbb80d1890 0x7fdbb815c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:09.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.356+0000 7fdbb6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbb80d1890 0x7fdbb815c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46104/0 (socket says 172.21.15.105:46104) 2023-11-28T18:26:09.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.356+0000 7fdbb6ffd640 1 -- 172.21.15.105:0/2651362874 learned_addr learned my addr 172.21.15.105:0/2651362874 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:09.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.356+0000 7fdbb6ffd640 1 -- 172.21.15.105:0/2651362874 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbb80d3c30 msgr2=0x7fdbb80cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:09.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.356+0000 7fdbb6ffd640 1 --2- 172.21.15.105:0/2651362874 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbb80d3c30 0x7fdbb80cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:09.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.356+0000 7fdbb6ffd640 1 -- 172.21.15.105:0/2651362874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbb815d4f0 msgr2=0x7fdbb8154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:09.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.356+0000 7fdbb6ffd640 1 --2- 172.21.15.105:0/2651362874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbb815d4f0 0x7fdbb8154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:09.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.356+0000 7fdbb6ffd640 1 -- 172.21.15.105:0/2651362874 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb0009b30 con 0x7fdbb80d1890 2023-11-28T18:26:09.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.356+0000 7fdbb6ffd640 1 --2- 172.21.15.105:0/2651362874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbb80d1890 0x7fdbb815c550 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdba800d5f0 tx=0x7fdba800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:09.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.356+0000 7fdbb4ff9640 1 -- 172.21.15.105:0/2651362874 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba80125f0 con 0x7fdbb80d1890 2023-11-28T18:26:09.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.356+0000 7fdbb4ff9640 1 -- 172.21.15.105:0/2651362874 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba8012750 con 0x7fdbb80d1890 2023-11-28T18:26:09.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.356+0000 7fdbb4ff9640 1 -- 172.21.15.105:0/2651362874 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8012ae0 con 0x7fdbb80d1890 2023-11-28T18:26:09.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.356+0000 7fdbbd89a640 1 -- 172.21.15.105:0/2651362874 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb80cab40 con 0x7fdbb80d1890 2023-11-28T18:26:09.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.356+0000 7fdbbd89a640 1 -- 172.21.15.105:0/2651362874 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb80cc7d0 con 0x7fdbb80d1890 2023-11-28T18:26:09.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.360+0000 7fdbbd89a640 1 -- 172.21.15.105:0/2651362874 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb8067570 con 0x7fdbb80d1890 2023-11-28T18:26:09.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.360+0000 7fdbb4ff9640 1 -- 172.21.15.105:0/2651362874 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fdba8012c40 con 0x7fdbb80d1890 2023-11-28T18:26:09.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.360+0000 7fdbb4ff9640 1 --2- 172.21.15.105:0/2651362874 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdb980668d0 0x7fdb98068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:09.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.360+0000 7fdbb4ff9640 1 -- 172.21.15.105:0/2651362874 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdba8084bb0 con 0x7fdbb80d1890 2023-11-28T18:26:09.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.360+0000 7fdbae7fc640 1 --2- 172.21.15.105:0/2651362874 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdb980668d0 0x7fdb98068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:09.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.360+0000 7fdbae7fc640 1 --2- 172.21.15.105:0/2651362874 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdb980668d0 0x7fdb98068d90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdbb81f9370 tx=0x7fdba0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:09.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.364+0000 7fdbb4ff9640 1 -- 172.21.15.105:0/2651362874 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdba8051b80 con 0x7fdbb80d1890 2023-11-28T18:26:09.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.512+0000 7fdbbd89a640 1 -- 172.21.15.105:0/2651362874 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random_max}] v 0) v1 -- 0x7fdbb80ca3e0 con 0x7fdbb80d1890 2023-11-28T18:26:09.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.520+0000 7fdbb4ff9640 1 -- 172.21.15.105:0/2651362874 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random_max}]=0 v22)=0 v22) v1 ==== 131+0+0 (secure 0 0 0) 0x7fdbb80ca3e0 con 0x7fdbb80d1890 2023-11-28T18:26:09.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.520+0000 7fdbbd89a640 1 -- 172.21.15.105:0/2651362874 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdb980668d0 msgr2=0x7fdb98068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:09.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.520+0000 7fdbbd89a640 1 --2- 172.21.15.105:0/2651362874 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdb980668d0 0x7fdb98068d90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdbb81f9370 tx=0x7fdba0008040 comp rx=0 tx=0).stop 2023-11-28T18:26:09.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.520+0000 7fdbbd89a640 1 -- 172.21.15.105:0/2651362874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbb80d1890 msgr2=0x7fdbb815c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:09.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.520+0000 7fdbbd89a640 1 --2- 172.21.15.105:0/2651362874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbb80d1890 0x7fdbb815c550 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdba800d5f0 tx=0x7fdba800dab0 comp rx=0 tx=0).stop 2023-11-28T18:26:09.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.520+0000 7fdbbd89a640 1 -- 172.21.15.105:0/2651362874 shutdown_connections 2023-11-28T18:26:09.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.520+0000 7fdbbd89a640 1 --2- 172.21.15.105:0/2651362874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbb815d4f0 0x7fdbb8154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:09.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.520+0000 7fdbbd89a640 1 --2- 172.21.15.105:0/2651362874 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbb80d3c30 0x7fdbb80cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:09.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.520+0000 7fdbbd89a640 1 --2- 172.21.15.105:0/2651362874 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdb980668d0 0x7fdb98068d90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:09.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.520+0000 7fdbbd89a640 1 --2- 172.21.15.105:0/2651362874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbb80d1890 0x7fdbb815c550 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:09.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.520+0000 7fdbbd89a640 1 -- 172.21.15.105:0/2651362874 >> 172.21.15.105:0/2651362874 conn(0x7fdbb80d3030 msgr2=0x7fdbb80d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:09.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.520+0000 7fdbbd89a640 1 -- 172.21.15.105:0/2651362874 shutdown_connections 2023-11-28T18:26:09.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:09.520+0000 7fdbbd89a640 1 -- 172.21.15.105:0/2651362874 wait complete. 2023-11-28T18:26:09.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:26:09.543 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-28T18:26:09.543 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:26:09.543 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:26:09.543 DEBUG:teuthology.orchestra.run.smithi105:> # Use up a random number of inode numbers so the ephemeral pinning is not the same every test. 2023-11-28T18:26:09.543 DEBUG:teuthology.orchestra.run.smithi105:> mkdir .inode_number_thrash 2023-11-28T18:26:09.543 DEBUG:teuthology.orchestra.run.smithi105:> count=$((RANDOM % 1024)) 2023-11-28T18:26:09.543 DEBUG:teuthology.orchestra.run.smithi105:> for ((i = 0; i < count; i++)); do touch .inode_number_thrash/$i; done 2023-11-28T18:26:09.543 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf .inode_number_thrash 2023-11-28T18:26:09.543 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:26:10.606 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-11-28T18:26:10.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f673438d640 1 Processor -- start 2023-11-28T18:26:10.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f673438d640 1 -- start start 2023-11-28T18:26:10.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f673438d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f672c15b030 0x7f672c0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:10.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f673438d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672c0701d0 con 0x7f672c071960 2023-11-28T18:26:10.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f673438d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672c06df80 con 0x7f672c0b4290 2023-11-28T18:26:10.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f673438d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f672c071df0 con 0x7f672c15b030 2023-11-28T18:26:10.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731928640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f672c071960 0x7f672c15f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:58014/0 (socket says 172.21.15.105:58014) 2023-11-28T18:26:10.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731928640 1 -- 172.21.15.105:0/598795881 learned_addr learned my addr 172.21.15.105:0/598795881 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:10.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f673292a640 1 --2- 172.21.15.105:0/598795881 >> v2:172.21.15.148:3300/0 conn(0x7f672c15b030 0x7f672c0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:10.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731127640 1 -- 172.21.15.105:0/598795881 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3655932623 0 0) 0x7f672c0701d0 con 0x7f672c071960 2023-11-28T18:26:10.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731127640 1 -- 172.21.15.105:0/598795881 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6718003530 con 0x7f672c071960 2023-11-28T18:26:10.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731127640 1 -- 172.21.15.105:0/598795881 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2958195736 0 0) 0x7f6718003530 con 0x7f672c071960 2023-11-28T18:26:10.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731127640 1 -- 172.21.15.105:0/598795881 >> v2:172.21.15.148:3300/0 conn(0x7f672c15b030 msgr2=0x7f672c0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:10.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731127640 1 --2- 172.21.15.105:0/598795881 >> v2:172.21.15.148:3300/0 conn(0x7f672c15b030 0x7f672c0709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:10.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731127640 1 -- 172.21.15.105:0/598795881 >> v1:172.21.15.148:6789/0 conn(0x7f672c0b4290 legacy=0x7f672c0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:10.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731127640 1 -- 172.21.15.105:0/598795881 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672c0720d0 con 0x7f672c071960 2023-11-28T18:26:10.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731127640 1 -- 172.21.15.105:0/598795881 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f671c001680 con 0x7f672c071960 2023-11-28T18:26:10.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731127640 1 -- 172.21.15.105:0/598795881 >> v1:172.21.15.105:6789/0 conn(0x7f672c071960 legacy=0x7f672c15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:10.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731127640 1 --2- 172.21.15.105:0/598795881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6718003980 0x7f6718003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:10.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731127640 1 --2- 172.21.15.105:0/598795881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6718004380 0x7f6718004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:10.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731127640 1 --2- 172.21.15.105:0/598795881 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6718005050 0x7f671800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:10.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731127640 1 -- 172.21.15.105:0/598795881 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c071df0 con 0x7f6718004380 2023-11-28T18:26:10.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731127640 1 -- 172.21.15.105:0/598795881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671800bfb0 con 0x7f6718003980 2023-11-28T18:26:10.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731127640 1 -- 172.21.15.105:0/598795881 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671800c170 con 0x7f6718005050 2023-11-28T18:26:10.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f673292a640 1 --2- 172.21.15.105:0/598795881 >> v2:172.21.15.148:3300/0 conn(0x7f672c15b030 0x7f672c0709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:26:10.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731928640 1 --2- 172.21.15.105:0/598795881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6718004380 0x7f6718004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:10.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6732129640 1 --2- 172.21.15.105:0/598795881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6718003980 0x7f6718003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:10.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f673292a640 1 --2- 172.21.15.105:0/598795881 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6718005050 0x7f671800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:10.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731928640 1 -- 172.21.15.105:0/598795881 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6718005050 msgr2=0x7f671800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:10.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731928640 1 --2- 172.21.15.105:0/598795881 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6718005050 0x7f671800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:10.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731928640 1 -- 172.21.15.105:0/598795881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6718003980 msgr2=0x7f6718003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:10.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731928640 1 --2- 172.21.15.105:0/598795881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6718003980 0x7f6718003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:10.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.744+0000 7f6731928640 1 -- 172.21.15.105:0/598795881 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6718003530 con 0x7f6718004380 2023-11-28T18:26:10.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.748+0000 7f673292a640 1 -- 172.21.15.105:0/598795881 reap_dead start 2023-11-28T18:26:10.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.748+0000 7f6731928640 1 --2- 172.21.15.105:0/598795881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6718004380 0x7f6718004850 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f671c00c210 tx=0x7f671c01cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e682ea829fdc4fce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:10.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.748+0000 7f6731127640 1 -- 172.21.15.105:0/598795881 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c005ae0 con 0x7f6718004380 2023-11-28T18:26:10.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.748+0000 7f6731127640 1 -- 172.21.15.105:0/598795881 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671c005c60 con 0x7f6718004380 2023-11-28T18:26:10.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.748+0000 7f6731127640 1 -- 172.21.15.105:0/598795881 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c0a5b00 con 0x7f6718004380 2023-11-28T18:26:10.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.748+0000 7f673438d640 1 -- 172.21.15.105:0/598795881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6718004380 msgr2=0x7f6718004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:10.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.748+0000 7f673438d640 1 --2- 172.21.15.105:0/598795881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6718004380 0x7f6718004850 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f671c00c210 tx=0x7f671c01cbc0 comp rx=0 tx=0).stop 2023-11-28T18:26:10.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.748+0000 7f673438d640 1 -- 172.21.15.105:0/598795881 shutdown_connections 2023-11-28T18:26:10.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.748+0000 7f673438d640 1 --2- 172.21.15.105:0/598795881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6718004380 0x7f6718004850 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:10.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.748+0000 7f673438d640 1 -- 172.21.15.105:0/598795881 >> 172.21.15.105:0/598795881 conn(0x7f672c0ceb90 msgr2=0x7f672c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:10.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.748+0000 7f673438d640 1 -- 172.21.15.105:0/598795881 shutdown_connections 2023-11-28T18:26:10.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.748+0000 7f673438d640 1 -- 172.21.15.105:0/598795881 wait complete. 2023-11-28T18:26:10.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.752+0000 7f673438d640 1 Processor -- start 2023-11-28T18:26:10.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.752+0000 7f673438d640 1 -- start start 2023-11-28T18:26:10.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.752+0000 7f673438d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6718004380 0x7f672c099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:10.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.752+0000 7f673438d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f672c1f3930 0x7f672c0cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:10.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.752+0000 7f673438d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f672c09b3f0 0x7f672c0ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:10.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.752+0000 7f673438d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c0c8050 con 0x7f6718004380 2023-11-28T18:26:10.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.752+0000 7f673438d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c09a350 con 0x7f672c09b3f0 2023-11-28T18:26:10.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.752+0000 7f673438d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672c09a630 con 0x7f672c1f3930 2023-11-28T18:26:10.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.752+0000 7f6732129640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6718004380 0x7f672c099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:10.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.752+0000 7f6732129640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6718004380 0x7f672c099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46126/0 (socket says 172.21.15.105:46126) 2023-11-28T18:26:10.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.752+0000 7f6732129640 1 -- 172.21.15.105:0/683696257 learned_addr learned my addr 172.21.15.105:0/683696257 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:10.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.752+0000 7f6731928640 1 --2- 172.21.15.105:0/683696257 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f672c1f3930 0x7f672c0cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:10.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.752+0000 7f673292a640 1 --2- 172.21.15.105:0/683696257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f672c09b3f0 0x7f672c0ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:10.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.752+0000 7f6732129640 1 -- 172.21.15.105:0/683696257 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f672c1f3930 msgr2=0x7f672c0cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:10.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.752+0000 7f6732129640 1 --2- 172.21.15.105:0/683696257 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f672c1f3930 0x7f672c0cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:10.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.752+0000 7f6732129640 1 -- 172.21.15.105:0/683696257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f672c09b3f0 msgr2=0x7f672c0ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:10.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.752+0000 7f6732129640 1 --2- 172.21.15.105:0/683696257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f672c09b3f0 0x7f672c0ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:10.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.752+0000 7f6732129640 1 -- 172.21.15.105:0/683696257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672c09a910 con 0x7f6718004380 2023-11-28T18:26:10.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.752+0000 7f6732129640 1 --2- 172.21.15.105:0/683696257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6718004380 0x7f672c099a50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f67280114e0 tx=0x7f67280119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:10.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.752+0000 7f6722ffd640 1 -- 172.21.15.105:0/683696257 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728015820 con 0x7f6718004380 2023-11-28T18:26:10.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.756+0000 7f6722ffd640 1 -- 172.21.15.105:0/683696257 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6728015980 con 0x7f6718004380 2023-11-28T18:26:10.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.756+0000 7f6722ffd640 1 -- 172.21.15.105:0/683696257 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728015d30 con 0x7f6718004380 2023-11-28T18:26:10.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.756+0000 7f673438d640 1 -- 172.21.15.105:0/683696257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f672c09abf0 con 0x7f6718004380 2023-11-28T18:26:10.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.756+0000 7f673438d640 1 -- 172.21.15.105:0/683696257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672c099f90 con 0x7f6718004380 2023-11-28T18:26:10.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.756+0000 7f673438d640 1 -- 172.21.15.105:0/683696257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f672c067570 con 0x7f6718004380 2023-11-28T18:26:10.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.756+0000 7f6722ffd640 1 -- 172.21.15.105:0/683696257 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f6728013390 con 0x7f6718004380 2023-11-28T18:26:10.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.756+0000 7f6722ffd640 1 --2- 172.21.15.105:0/683696257 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6710066900 0x7f6710068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:10.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.756+0000 7f6722ffd640 1 -- 172.21.15.105:0/683696257 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6728087330 con 0x7f6718004380 2023-11-28T18:26:10.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.756+0000 7f6731928640 1 --2- 172.21.15.105:0/683696257 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6710066900 0x7f6710068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:10.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.756+0000 7f6731928640 1 --2- 172.21.15.105:0/683696257 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6710066900 0x7f6710068dc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f671c0ab000 tx=0x7f671c0b1040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:10.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.760+0000 7f6722ffd640 1 -- 172.21.15.105:0/683696257 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f67280543b0 con 0x7f6718004380 2023-11-28T18:26:10.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:10.940+0000 7f673438d640 1 -- 172.21.15.105:0/683696257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f672c09b0a0 con 0x7f6718004380 2023-11-28T18:26:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.192+0000 7f6722ffd640 1 -- 172.21.15.105:0/683696257 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v36) v1 ==== 105+0+0 (secure 0 0 0) 0x7f67280579d0 con 0x7f6718004380 2023-11-28T18:26:11.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.192+0000 7f673438d640 1 -- 172.21.15.105:0/683696257 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6710066900 msgr2=0x7f6710068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:11.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.192+0000 7f673438d640 1 --2- 172.21.15.105:0/683696257 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6710066900 0x7f6710068dc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f671c0ab000 tx=0x7f671c0b1040 comp rx=0 tx=0).stop 2023-11-28T18:26:11.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.192+0000 7f673438d640 1 -- 172.21.15.105:0/683696257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6718004380 msgr2=0x7f672c099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:11.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.192+0000 7f673438d640 1 --2- 172.21.15.105:0/683696257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6718004380 0x7f672c099a50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f67280114e0 tx=0x7f67280119a0 comp rx=0 tx=0).stop 2023-11-28T18:26:11.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.192+0000 7f673438d640 1 -- 172.21.15.105:0/683696257 shutdown_connections 2023-11-28T18:26:11.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.192+0000 7f673438d640 1 --2- 172.21.15.105:0/683696257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f672c09b3f0 0x7f672c0ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:11.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.192+0000 7f673438d640 1 --2- 172.21.15.105:0/683696257 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f672c1f3930 0x7f672c0cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.192+0000 7f673438d640 1 --2- 172.21.15.105:0/683696257 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6710066900 0x7f6710068dc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.192+0000 7f673438d640 1 --2- 172.21.15.105:0/683696257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6718004380 0x7f672c099a50 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.192+0000 7f673438d640 1 -- 172.21.15.105:0/683696257 >> 172.21.15.105:0/683696257 conn(0x7f672c0ceb90 msgr2=0x7f672c06bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.192+0000 7f673438d640 1 -- 172.21.15.105:0/683696257 shutdown_connections 2023-11-28T18:26:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.192+0000 7f673438d640 1 -- 172.21.15.105:0/683696257 wait complete. 2023-11-28T18:26:11.221 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:26:11.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.360+0000 7fb1945b8640 1 Processor -- start 2023-11-28T18:26:11.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.360+0000 7fb1945b8640 1 -- start start 2023-11-28T18:26:11.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.360+0000 7fb1945b8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb18c0cdd80 0x7fb18c06c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:11.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.360+0000 7fb1945b8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb18c0ce230 0x7fb18c0cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:11.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.360+0000 7fb1945b8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18c0b3c40 con 0x7fb18c15d320 2023-11-28T18:26:11.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.360+0000 7fb1945b8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18c06ca80 con 0x7fb18c0ce230 2023-11-28T18:26:11.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.360+0000 7fb1945b8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18c15d700 con 0x7fb18c0cdd80 2023-11-28T18:26:11.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.360+0000 7fb192354640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb18c15d320 0x7fb18c0cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49188/0 (socket says 172.21.15.105:49188) 2023-11-28T18:26:11.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.360+0000 7fb192354640 1 -- 172.21.15.105:0/282464037 learned_addr learned my addr 172.21.15.105:0/282464037 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:11.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.360+0000 7fb192b55640 1 --2- 172.21.15.105:0/282464037 >> v2:172.21.15.105:3300/0 conn(0x7fb18c0ce230 0x7fb18c0cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:11.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.360+0000 7fb191b53640 1 --2- 172.21.15.105:0/282464037 >> v2:172.21.15.148:3300/0 conn(0x7fb18c0cdd80 0x7fb18c06c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:11.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.360+0000 7fb191352640 1 -- 172.21.15.105:0/282464037 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3447494346 0 0) 0x7fb18c0b3c40 con 0x7fb18c15d320 2023-11-28T18:26:11.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.360+0000 7fb191352640 1 -- 172.21.15.105:0/282464037 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb178003530 con 0x7fb18c15d320 2023-11-28T18:26:11.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.360+0000 7fb192b55640 1 -- 172.21.15.105:0/282464037 >> v2:172.21.15.148:3300/0 conn(0x7fb18c0cdd80 msgr2=0x7fb18c06c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:11.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.360+0000 7fb192b55640 1 --2- 172.21.15.105:0/282464037 >> v2:172.21.15.148:3300/0 conn(0x7fb18c0cdd80 0x7fb18c06c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:11.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.360+0000 7fb192b55640 1 -- 172.21.15.105:0/282464037 >> v1:172.21.15.105:6789/0 conn(0x7fb18c15d320 legacy=0x7fb18c0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:11.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.360+0000 7fb192b55640 1 -- 172.21.15.105:0/282464037 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c06c7a0 con 0x7fb18c0ce230 2023-11-28T18:26:11.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.364+0000 7fb192b55640 1 --2- 172.21.15.105:0/282464037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb18c0ce230 0x7fb18c0cc0c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb18c073a90 tx=0x7fb18401c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3bda223defd7dc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:11.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.364+0000 7fb191352640 1 -- 172.21.15.105:0/282464037 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1840a5940 con 0x7fb18c0ce230 2023-11-28T18:26:11.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.364+0000 7fb191352640 1 -- 172.21.15.105:0/282464037 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1840a5aa0 con 0x7fb18c0ce230 2023-11-28T18:26:11.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.364+0000 7fb191352640 1 -- 172.21.15.105:0/282464037 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1840a5e10 con 0x7fb18c0ce230 2023-11-28T18:26:11.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.364+0000 7fb1945b8640 1 -- 172.21.15.105:0/282464037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb18c0ce230 msgr2=0x7fb18c0cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:11.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.364+0000 7fb1945b8640 1 --2- 172.21.15.105:0/282464037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb18c0ce230 0x7fb18c0cc0c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb18c073a90 tx=0x7fb18401c880 comp rx=0 tx=0).stop 2023-11-28T18:26:11.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.364+0000 7fb1945b8640 1 -- 172.21.15.105:0/282464037 shutdown_connections 2023-11-28T18:26:11.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.364+0000 7fb1945b8640 1 --2- 172.21.15.105:0/282464037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb18c0ce230 0x7fb18c0cc0c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:11.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.364+0000 7fb1945b8640 1 --2- 172.21.15.105:0/282464037 >> v2:172.21.15.148:3300/0 conn(0x7fb18c0cdd80 0x7fb18c06c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:11.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.364+0000 7fb1945b8640 1 -- 172.21.15.105:0/282464037 >> 172.21.15.105:0/282464037 conn(0x7fb18c0d0df0 msgr2=0x7fb18c0d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:11.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.364+0000 7fb1945b8640 1 -- 172.21.15.105:0/282464037 shutdown_connections 2023-11-28T18:26:11.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.364+0000 7fb1945b8640 1 -- 172.21.15.105:0/282464037 wait complete. 2023-11-28T18:26:11.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb1945b8640 1 Processor -- start 2023-11-28T18:26:11.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb1945b8640 1 -- start start 2023-11-28T18:26:11.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb1945b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb18c0cdd80 0x7fb18c14ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:11.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb1945b8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb18c0ce230 0x7fb18c14ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:11.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb1945b8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb18c15d320 0x7fb18c14c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:11.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb1945b8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0b5240 con 0x7fb18c15d320 2023-11-28T18:26:11.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb1945b8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c14e830 con 0x7fb18c0cdd80 2023-11-28T18:26:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb1945b8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c14eb10 con 0x7fb18c0ce230 2023-11-28T18:26:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb192b55640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb18c15d320 0x7fb18c14c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb192b55640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb18c15d320 0x7fb18c14c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46794/0 (socket says 172.21.15.105:46794) 2023-11-28T18:26:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb192b55640 1 -- 172.21.15.105:0/1997781423 learned_addr learned my addr 172.21.15.105:0/1997781423 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb191b53640 1 --2- 172.21.15.105:0/1997781423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb18c0ce230 0x7fb18c14ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb192354640 1 --2- 172.21.15.105:0/1997781423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb18c0cdd80 0x7fb18c14ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb192b55640 1 -- 172.21.15.105:0/1997781423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb18c0ce230 msgr2=0x7fb18c14ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb192b55640 1 --2- 172.21.15.105:0/1997781423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb18c0ce230 0x7fb18c14ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb192b55640 1 -- 172.21.15.105:0/1997781423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb18c0cdd80 msgr2=0x7fb18c14ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb192b55640 1 --2- 172.21.15.105:0/1997781423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb18c0cdd80 0x7fb18c14ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb192b55640 1 -- 172.21.15.105:0/1997781423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb184009a70 con 0x7fb18c15d320 2023-11-28T18:26:11.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb192b55640 1 --2- 172.21.15.105:0/1997781423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb18c15d320 0x7fb18c14c280 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb184019490 tx=0x7fb184019620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:11.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb182ffd640 1 -- 172.21.15.105:0/1997781423 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1840aede0 con 0x7fb18c15d320 2023-11-28T18:26:11.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb182ffd640 1 -- 172.21.15.105:0/1997781423 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb184009450 con 0x7fb18c15d320 2023-11-28T18:26:11.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb182ffd640 1 -- 172.21.15.105:0/1997781423 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1840b7610 con 0x7fb18c15d320 2023-11-28T18:26:11.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb1945b8640 1 -- 172.21.15.105:0/1997781423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb18c14edf0 con 0x7fb18c15d320 2023-11-28T18:26:11.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb1945b8640 1 -- 172.21.15.105:0/1997781423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18c14f3b0 con 0x7fb18c15d320 2023-11-28T18:26:11.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.368+0000 7fb1945b8640 1 -- 172.21.15.105:0/1997781423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb18c15c890 con 0x7fb18c15d320 2023-11-28T18:26:11.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.372+0000 7fb182ffd640 1 -- 172.21.15.105:0/1997781423 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fb1840b4050 con 0x7fb18c15d320 2023-11-28T18:26:11.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.372+0000 7fb182ffd640 1 --2- 172.21.15.105:0/1997781423 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb1700668d0 0x7fb170068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:11.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.372+0000 7fb182ffd640 1 -- 172.21.15.105:0/1997781423 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb18411f870 con 0x7fb18c15d320 2023-11-28T18:26:11.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.372+0000 7fb192354640 1 --2- 172.21.15.105:0/1997781423 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb1700668d0 0x7fb170068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:11.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.372+0000 7fb192354640 1 --2- 172.21.15.105:0/1997781423 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb1700668d0 0x7fb170068d90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb188005890 tx=0x7fb188009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.376+0000 7fb182ffd640 1 -- 172.21.15.105:0/1997781423 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb1840ec8d0 con 0x7fb18c15d320 2023-11-28T18:26:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.560+0000 7fb1945b8640 1 -- 172.21.15.105:0/1997781423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb18c14f5a0 con 0x7fb18c15d320 2023-11-28T18:26:11.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.560+0000 7fb182ffd640 1 -- 172.21.15.105:0/1997781423 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4935 (secure 0 0 0) 0x7fb1840b2090 con 0x7fb18c15d320 2023-11-28T18:26:11.564 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:11.564 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":37,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/735646078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":735646078},{"type":"v1","addr":"172.21.15.105:6836","nonce":735646078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5408,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/3765684108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":3765684108},{"type":"v1","addr":"172.21.15.105:6837","nonce":3765684108}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:26:00.550298+0000","modified":"2023-11-28T18:26:11.196440+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1],"up":{"mds_0":5842,"mds_1":5414},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5414":{"gid":5414,"name":"d","rank":1,"incarnation":37,"state":"up:creating","state_seq":1,"addr":"172.21.15.148:6835/897997046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":897997046},{"type":"v1","addr":"172.21.15.148:6835","nonce":897997046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5842":{"gid":5842,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6834/2696773413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2696773413},{"type":"v1","addr":"172.21.15.148:6834","nonce":2696773413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T18:26:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 37 2023-11-28T18:26:11.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.564+0000 7fb1945b8640 1 -- 172.21.15.105:0/1997781423 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb1700668d0 msgr2=0x7fb170068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:11.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.564+0000 7fb1945b8640 1 --2- 172.21.15.105:0/1997781423 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb1700668d0 0x7fb170068d90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb188005890 tx=0x7fb188009040 comp rx=0 tx=0).stop 2023-11-28T18:26:11.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.564+0000 7fb1945b8640 1 -- 172.21.15.105:0/1997781423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb18c15d320 msgr2=0x7fb18c14c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:11.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.564+0000 7fb1945b8640 1 --2- 172.21.15.105:0/1997781423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb18c15d320 0x7fb18c14c280 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb184019490 tx=0x7fb184019620 comp rx=0 tx=0).stop 2023-11-28T18:26:11.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.564+0000 7fb1945b8640 1 -- 172.21.15.105:0/1997781423 shutdown_connections 2023-11-28T18:26:11.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.564+0000 7fb1945b8640 1 --2- 172.21.15.105:0/1997781423 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb1700668d0 0x7fb170068d90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:11.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.564+0000 7fb1945b8640 1 --2- 172.21.15.105:0/1997781423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb18c15d320 0x7fb18c14c280 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:11.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.564+0000 7fb1945b8640 1 --2- 172.21.15.105:0/1997781423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb18c0ce230 0x7fb18c14ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.564+0000 7fb1945b8640 1 --2- 172.21.15.105:0/1997781423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb18c0cdd80 0x7fb18c14ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.564+0000 7fb1945b8640 1 -- 172.21.15.105:0/1997781423 >> 172.21.15.105:0/1997781423 conn(0x7fb18c0d0df0 msgr2=0x7fb18c159b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.564+0000 7fb1945b8640 1 -- 172.21.15.105:0/1997781423 shutdown_connections 2023-11-28T18:26:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:11.564+0000 7fb1945b8640 1 -- 172.21.15.105:0/1997781423 wait complete. 2023-11-28T18:26:11.987 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:26:00.550298+0000', 'modified': '2023-11-28T18:26:11.196440+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 5842, 'mds_1': 5414}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5414': {'gid': 5414, 'name': 'd', 'rank': 1, 'incarnation': 37, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.148:6835/897997046', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6833', 'nonce': 897997046}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 897997046}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_5842': {'gid': 5842, 'name': 'b', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6834/2696773413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6832', 'nonce': 2696773413}, {'type': 'v1', 'addr': '172.21.15.148:6834', 'nonce': 2696773413}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:26:11.987 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5414:up:creating 2023-11-28T18:26:12.988 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:26:13.136 INFO:tasks.ceph.mds.b.smithi148.stderr:2023-11-28T18:26:13.132+0000 7f609c6a2640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-28T18:26:13.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.136+0000 7fa2f3e9c640 1 Processor -- start 2023-11-28T18:26:13.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.136+0000 7fa2f3e9c640 1 -- start start 2023-11-28T18:26:13.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.136+0000 7fa2f3e9c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2ec0b4290 0x7fa2ec0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.136+0000 7fa2f3e9c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa2ec071960 0x7fa2ec15f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.136+0000 7fa2f3e9c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ec0b4bb0 con 0x7fa2ec15b030 2023-11-28T18:26:13.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.136+0000 7fa2f3e9c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2ec0701d0 con 0x7fa2ec071960 2023-11-28T18:26:13.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.136+0000 7fa2f3e9c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec06df80 con 0x7fa2ec0b4290 2023-11-28T18:26:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.136+0000 7fa2f1c38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2ec0b4290 0x7fa2ec0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.136+0000 7fa2f1437640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa2ec071960 0x7fa2ec15f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.136+0000 7fa2f1c38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2ec0b4290 0x7fa2ec0b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:59546/0 (socket says 172.21.15.105:59546) 2023-11-28T18:26:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.136+0000 7fa2f1c38640 1 -- 172.21.15.105:0/341185984 learned_addr learned my addr 172.21.15.105:0/341185984 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.140+0000 7fa2f1c38640 1 -- 172.21.15.105:0/341185984 >> v2:172.21.15.148:3300/0 conn(0x7fa2ec071960 msgr2=0x7fa2ec15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.140+0000 7fa2f1c38640 1 --2- 172.21.15.105:0/341185984 >> v2:172.21.15.148:3300/0 conn(0x7fa2ec071960 0x7fa2ec15f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.140+0000 7fa2f1c38640 1 -- 172.21.15.105:0/341185984 >> v1:172.21.15.148:6789/0 conn(0x7fa2ec15b030 legacy=0x7fa2ec0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.140+0000 7fa2f1c38640 1 -- 172.21.15.105:0/341185984 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec071df0 con 0x7fa2ec0b4290 2023-11-28T18:26:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.140+0000 7fa2f1437640 1 --2- 172.21.15.105:0/341185984 >> v2:172.21.15.148:3300/0 conn(0x7fa2ec071960 0x7fa2ec15f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:26:13.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.140+0000 7fa2f1c38640 1 --2- 172.21.15.105:0/341185984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2ec0b4290 0x7fa2ec0b4670 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009870 tx=0x7fa2e801c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6a7258234861c4e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:13.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.140+0000 7fa2f0c36640 1 -- 172.21.15.105:0/341185984 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e80aa670 con 0x7fa2ec0b4290 2023-11-28T18:26:13.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.144+0000 7fa2f0c36640 1 -- 172.21.15.105:0/341185984 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e80aa7d0 con 0x7fa2ec0b4290 2023-11-28T18:26:13.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.144+0000 7fa2f0c36640 1 -- 172.21.15.105:0/341185984 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e80aab60 con 0x7fa2ec0b4290 2023-11-28T18:26:13.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.144+0000 7fa2f3e9c640 1 -- 172.21.15.105:0/341185984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2ec0b4290 msgr2=0x7fa2ec0b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:13.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.144+0000 7fa2f3e9c640 1 --2- 172.21.15.105:0/341185984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2ec0b4290 0x7fa2ec0b4670 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009870 tx=0x7fa2e801c510 comp rx=0 tx=0).stop 2023-11-28T18:26:13.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.144+0000 7fa2f3e9c640 1 -- 172.21.15.105:0/341185984 shutdown_connections 2023-11-28T18:26:13.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.144+0000 7fa2f3e9c640 1 --2- 172.21.15.105:0/341185984 >> v2:172.21.15.148:3300/0 conn(0x7fa2ec071960 0x7fa2ec15f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.144+0000 7fa2f3e9c640 1 --2- 172.21.15.105:0/341185984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2ec0b4290 0x7fa2ec0b4670 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.144+0000 7fa2f3e9c640 1 -- 172.21.15.105:0/341185984 >> 172.21.15.105:0/341185984 conn(0x7fa2ec0ceb90 msgr2=0x7fa2ec0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:13.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.144+0000 7fa2f3e9c640 1 -- 172.21.15.105:0/341185984 shutdown_connections 2023-11-28T18:26:13.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.144+0000 7fa2f3e9c640 1 -- 172.21.15.105:0/341185984 wait complete. 2023-11-28T18:26:13.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.144+0000 7fa2f3e9c640 1 Processor -- start 2023-11-28T18:26:13.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.144+0000 7fa2f3e9c640 1 -- start start 2023-11-28T18:26:13.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.144+0000 7fa2f3e9c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2ec0b4290 0x7fa2ec15dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.144+0000 7fa2f3e9c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ec15b030 0x7fa2ec1540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.144+0000 7fa2f3e9c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2ec154900 0x7fa2ec1f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.144+0000 7fa2f3e9c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec15d870 con 0x7fa2ec15b030 2023-11-28T18:26:13.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.144+0000 7fa2f3e9c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec15e2d0 con 0x7fa2ec0b4290 2023-11-28T18:26:13.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.144+0000 7fa2f3e9c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec0adcd0 con 0x7fa2ec154900 2023-11-28T18:26:13.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.144+0000 7fa2f1437640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ec15b030 0x7fa2ec1540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.144+0000 7fa2f1437640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ec15b030 0x7fa2ec1540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46796/0 (socket says 172.21.15.105:46796) 2023-11-28T18:26:13.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.144+0000 7fa2f1437640 1 -- 172.21.15.105:0/1793401228 learned_addr learned my addr 172.21.15.105:0/1793401228 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:13.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.148+0000 7fa2f1c38640 1 --2- 172.21.15.105:0/1793401228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2ec0b4290 0x7fa2ec15dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.148+0000 7fa2f2439640 1 --2- 172.21.15.105:0/1793401228 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2ec154900 0x7fa2ec1f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.148+0000 7fa2f1437640 1 -- 172.21.15.105:0/1793401228 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2ec154900 msgr2=0x7fa2ec1f3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:13.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.148+0000 7fa2f1437640 1 --2- 172.21.15.105:0/1793401228 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2ec154900 0x7fa2ec1f3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.148+0000 7fa2f1437640 1 -- 172.21.15.105:0/1793401228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2ec0b4290 msgr2=0x7fa2ec15dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:13.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.148+0000 7fa2f1437640 1 --2- 172.21.15.105:0/1793401228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2ec0b4290 0x7fa2ec15dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.148+0000 7fa2f1437640 1 -- 172.21.15.105:0/1793401228 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2dc009420 con 0x7fa2ec15b030 2023-11-28T18:26:13.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.148+0000 7fa2f1c38640 1 --2- 172.21.15.105:0/1793401228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2ec0b4290 0x7fa2ec15dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:13.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.148+0000 7fa2f1437640 1 --2- 172.21.15.105:0/1793401228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ec15b030 0x7fa2ec1540f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc00ebe0 tx=0x7fa2dc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:13.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.148+0000 7fa2e27fc640 1 -- 172.21.15.105:0/1793401228 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc00eee0 con 0x7fa2ec15b030 2023-11-28T18:26:13.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.148+0000 7fa2e27fc640 1 -- 172.21.15.105:0/1793401228 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2dc00cc30 con 0x7fa2ec15b030 2023-11-28T18:26:13.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.148+0000 7fa2f3e9c640 1 -- 172.21.15.105:0/1793401228 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e80094d0 con 0x7fa2ec15b030 2023-11-28T18:26:13.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.148+0000 7fa2e27fc640 1 -- 172.21.15.105:0/1793401228 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc013470 con 0x7fa2ec15b030 2023-11-28T18:26:13.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.148+0000 7fa2f3e9c640 1 -- 172.21.15.105:0/1793401228 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ec0ae570 con 0x7fa2ec15b030 2023-11-28T18:26:13.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.148+0000 7fa2f3e9c640 1 -- 172.21.15.105:0/1793401228 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ec067570 con 0x7fa2ec15b030 2023-11-28T18:26:13.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.148+0000 7fa2e27fc640 1 -- 172.21.15.105:0/1793401228 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa2dc0135d0 con 0x7fa2ec15b030 2023-11-28T18:26:13.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.148+0000 7fa2e27fc640 1 --2- 172.21.15.105:0/1793401228 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa2d00668d0 0x7fa2d0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.148+0000 7fa2e27fc640 1 -- 172.21.15.105:0/1793401228 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa2dc084960 con 0x7fa2ec15b030 2023-11-28T18:26:13.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.152+0000 7fa2f1c38640 1 --2- 172.21.15.105:0/1793401228 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa2d00668d0 0x7fa2d0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.152+0000 7fa2f1c38640 1 --2- 172.21.15.105:0/1793401228 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa2d00668d0 0x7fa2d0068d90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa2e80ae000 tx=0x7fa2e80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:13.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.156+0000 7fa2e27fc640 1 -- 172.21.15.105:0/1793401228 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa2dc0519c0 con 0x7fa2ec15b030 2023-11-28T18:26:13.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.344+0000 7fa2f3e9c640 1 -- 172.21.15.105:0/1793401228 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2ec165b60 con 0x7fa2ec15b030 2023-11-28T18:26:13.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.344+0000 7fa2e27fc640 1 -- 172.21.15.105:0/1793401228 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fa2dc017090 con 0x7fa2ec15b030 2023-11-28T18:26:13.348 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:13.348 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/735646078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":735646078},{"type":"v1","addr":"172.21.15.105:6836","nonce":735646078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:26:00.550298+0000","modified":"2023-11-28T18:26:12.238927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5842,"mds_1":5414,"mds_2":5408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5408":{"gid":5408,"name":"c","rank":2,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6837/3765684108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":3765684108},{"type":"v1","addr":"172.21.15.105:6837","nonce":3765684108}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5414":{"gid":5414,"name":"d","rank":1,"incarnation":37,"state":"up:active","state_seq":5,"addr":"172.21.15.148:6835/897997046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":897997046},{"type":"v1","addr":"172.21.15.148:6835","nonce":897997046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5842":{"gid":5842,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6834/2696773413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2696773413},{"type":"v1","addr":"172.21.15.148:6834","nonce":2696773413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T18:26:13.348 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 40 2023-11-28T18:26:13.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.348+0000 7fa2f3e9c640 1 -- 172.21.15.105:0/1793401228 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa2d00668d0 msgr2=0x7fa2d0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:13.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.348+0000 7fa2f3e9c640 1 --2- 172.21.15.105:0/1793401228 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa2d00668d0 0x7fa2d0068d90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa2e80ae000 tx=0x7fa2e80a7000 comp rx=0 tx=0).stop 2023-11-28T18:26:13.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.348+0000 7fa2f3e9c640 1 -- 172.21.15.105:0/1793401228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ec15b030 msgr2=0x7fa2ec1540f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:13.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.348+0000 7fa2f3e9c640 1 --2- 172.21.15.105:0/1793401228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ec15b030 0x7fa2ec1540f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc00ebe0 tx=0x7fa2dc00c520 comp rx=0 tx=0).stop 2023-11-28T18:26:13.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.348+0000 7fa2f3e9c640 1 -- 172.21.15.105:0/1793401228 shutdown_connections 2023-11-28T18:26:13.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.348+0000 7fa2f3e9c640 1 --2- 172.21.15.105:0/1793401228 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa2ec154900 0x7fa2ec1f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.348+0000 7fa2f3e9c640 1 --2- 172.21.15.105:0/1793401228 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa2d00668d0 0x7fa2d0068d90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.348+0000 7fa2f3e9c640 1 --2- 172.21.15.105:0/1793401228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2ec15b030 0x7fa2ec1540f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.348+0000 7fa2f3e9c640 1 --2- 172.21.15.105:0/1793401228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2ec0b4290 0x7fa2ec15dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.348+0000 7fa2f3e9c640 1 -- 172.21.15.105:0/1793401228 >> 172.21.15.105:0/1793401228 conn(0x7fa2ec0ceb90 msgr2=0x7fa2ec0cb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:13.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.348+0000 7fa2f3e9c640 1 -- 172.21.15.105:0/1793401228 shutdown_connections 2023-11-28T18:26:13.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.348+0000 7fa2f3e9c640 1 -- 172.21.15.105:0/1793401228 wait complete. 2023-11-28T18:26:13.772 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:26:00.550298+0000', 'modified': '2023-11-28T18:26:12.238927+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 5842, 'mds_1': 5414, 'mds_2': 5408}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5408': {'gid': 5408, 'name': 'c', 'rank': 2, 'incarnation': 39, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.105:6837/3765684108', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6835', 'nonce': 3765684108}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 3765684108}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_5414': {'gid': 5414, 'name': 'd', 'rank': 1, 'incarnation': 37, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.148:6835/897997046', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6833', 'nonce': 897997046}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 897997046}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_5842': {'gid': 5842, 'name': 'b', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6834/2696773413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6832', 'nonce': 2696773413}, {'type': 'v1', 'addr': '172.21.15.148:6834', 'nonce': 2696773413}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:26:13.772 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-11-28T18:26:13.772 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T18:26:13.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.908+0000 7f29241f8640 1 Processor -- start 2023-11-28T18:26:13.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.908+0000 7f29241f8640 1 -- start start 2023-11-28T18:26:13.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.908+0000 7f29241f8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f291c06d610 0x7f291c0cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.908+0000 7f29241f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f291c0ce530 0x7f291c0ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.908+0000 7f29241f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f291c0adf30 0x7f291c1650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.908+0000 7f29241f8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f291c162820 con 0x7f291c06d610 2023-11-28T18:26:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.908+0000 7f29241f8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f291c0ae360 con 0x7f291c0ce530 2023-11-28T18:26:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.908+0000 7f29241f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291c161c10 con 0x7f291c0adf30 2023-11-28T18:26:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.912+0000 7f2921f94640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f291c06d610 0x7f291c0cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.912+0000 7f2921f94640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f291c06d610 0x7f291c0cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46804/0 (socket says 172.21.15.105:46804) 2023-11-28T18:26:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.912+0000 7f2921f94640 1 -- 172.21.15.105:0/3598025998 learned_addr learned my addr 172.21.15.105:0/3598025998 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.912+0000 7f2921793640 1 --2- 172.21.15.105:0/3598025998 >> v2:172.21.15.148:3300/0 conn(0x7f291c0ce530 0x7f291c0ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.912+0000 7f2922795640 1 --2- 172.21.15.105:0/3598025998 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f291c0adf30 0x7f291c1650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.912+0000 7f2921f94640 1 -- 172.21.15.105:0/3598025998 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f291c0adf30 msgr2=0x7f291c1650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.912+0000 7f2921f94640 1 --2- 172.21.15.105:0/3598025998 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f291c0adf30 0x7f291c1650f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.912+0000 7f2921f94640 1 -- 172.21.15.105:0/3598025998 >> v2:172.21.15.148:3300/0 conn(0x7f291c0ce530 msgr2=0x7f291c0ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:13.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.912+0000 7f2921f94640 1 --2- 172.21.15.105:0/3598025998 >> v2:172.21.15.148:3300/0 conn(0x7f291c0ce530 0x7f291c0ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.912+0000 7f2921f94640 1 -- 172.21.15.105:0/3598025998 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291c161ef0 con 0x7f291c06d610 2023-11-28T18:26:13.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.912+0000 7f2922795640 1 --2- 172.21.15.105:0/3598025998 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f291c0adf30 0x7f291c1650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:13.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.912+0000 7f2921f94640 1 --2- 172.21.15.105:0/3598025998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f291c06d610 0x7f291c0cdff0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f291c073a50 tx=0x7f291801c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668a383fdebe7247 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.912+0000 7f2920f92640 1 -- 172.21.15.105:0/3598025998 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29180ab600 con 0x7f291c06d610 2023-11-28T18:26:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.912+0000 7f2920f92640 1 -- 172.21.15.105:0/3598025998 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29180a7050 con 0x7f291c06d610 2023-11-28T18:26:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.912+0000 7f2920f92640 1 -- 172.21.15.105:0/3598025998 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29180ab980 con 0x7f291c06d610 2023-11-28T18:26:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.916+0000 7f29241f8640 1 -- 172.21.15.105:0/3598025998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f291c06d610 msgr2=0x7f291c0cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.916+0000 7f29241f8640 1 --2- 172.21.15.105:0/3598025998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f291c06d610 0x7f291c0cdff0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f291c073a50 tx=0x7f291801c210 comp rx=0 tx=0).stop 2023-11-28T18:26:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.916+0000 7f29241f8640 1 -- 172.21.15.105:0/3598025998 shutdown_connections 2023-11-28T18:26:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.916+0000 7f29241f8640 1 --2- 172.21.15.105:0/3598025998 >> v2:172.21.15.148:3300/0 conn(0x7f291c0ce530 0x7f291c0ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.916+0000 7f29241f8640 1 --2- 172.21.15.105:0/3598025998 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f291c0adf30 0x7f291c1650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.916+0000 7f29241f8640 1 --2- 172.21.15.105:0/3598025998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f291c06d610 0x7f291c0cdff0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.916+0000 7f29241f8640 1 -- 172.21.15.105:0/3598025998 >> 172.21.15.105:0/3598025998 conn(0x7f291c0ceb90 msgr2=0x7f291c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.916+0000 7f29241f8640 1 -- 172.21.15.105:0/3598025998 shutdown_connections 2023-11-28T18:26:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.916+0000 7f29241f8640 1 -- 172.21.15.105:0/3598025998 wait complete. 2023-11-28T18:26:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.916+0000 7f29241f8640 1 Processor -- start 2023-11-28T18:26:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.916+0000 7f29241f8640 1 -- start start 2023-11-28T18:26:13.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.916+0000 7f29241f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f291c06d610 0x7f291c1608e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.916+0000 7f29241f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f291c0adf30 0x7f291c0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.916+0000 7f29241f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f291c0ce530 0x7f291c09a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.916+0000 7f29241f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291c160e20 con 0x7f291c0adf30 2023-11-28T18:26:13.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.916+0000 7f29241f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291c161300 con 0x7f291c06d610 2023-11-28T18:26:13.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.916+0000 7f29241f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291c1636a0 con 0x7f291c0ce530 2023-11-28T18:26:13.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.916+0000 7f2921793640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f291c0adf30 0x7f291c0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.916+0000 7f2921793640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f291c0adf30 0x7f291c0b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46810/0 (socket says 172.21.15.105:46810) 2023-11-28T18:26:13.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.916+0000 7f2921793640 1 -- 172.21.15.105:0/3044341789 learned_addr learned my addr 172.21.15.105:0/3044341789 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:13.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.916+0000 7f2922795640 1 --2- 172.21.15.105:0/3044341789 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f291c0ce530 0x7f291c09a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.916+0000 7f2921f94640 1 --2- 172.21.15.105:0/3044341789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f291c06d610 0x7f291c1608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.920+0000 7f2921793640 1 -- 172.21.15.105:0/3044341789 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f291c0ce530 msgr2=0x7f291c09a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:13.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.920+0000 7f2921793640 1 --2- 172.21.15.105:0/3044341789 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f291c0ce530 0x7f291c09a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.920+0000 7f2921793640 1 -- 172.21.15.105:0/3044341789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f291c06d610 msgr2=0x7f291c1608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:13.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.920+0000 7f2921793640 1 --2- 172.21.15.105:0/3044341789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f291c06d610 0x7f291c1608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.920+0000 7f2921793640 1 -- 172.21.15.105:0/3044341789 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29180094d0 con 0x7f291c0adf30 2023-11-28T18:26:13.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.920+0000 7f2921793640 1 --2- 172.21.15.105:0/3044341789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f291c0adf30 0x7f291c0b4460 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f290c00e8a0 tx=0x7f290c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:13.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.920+0000 7f29127fc640 1 -- 172.21.15.105:0/3044341789 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c009510 con 0x7f291c0adf30 2023-11-28T18:26:13.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.920+0000 7f29127fc640 1 -- 172.21.15.105:0/3044341789 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290c00c940 con 0x7f291c0adf30 2023-11-28T18:26:13.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.920+0000 7f29127fc640 1 -- 172.21.15.105:0/3044341789 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c00ccb0 con 0x7f291c0adf30 2023-11-28T18:26:13.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.920+0000 7f29241f8640 1 -- 172.21.15.105:0/3044341789 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f291c15a410 con 0x7f291c0adf30 2023-11-28T18:26:13.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.920+0000 7f29241f8640 1 -- 172.21.15.105:0/3044341789 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291c163050 con 0x7f291c0adf30 2023-11-28T18:26:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.920+0000 7f29127fc640 1 -- 172.21.15.105:0/3044341789 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f290c013470 con 0x7f291c0adf30 2023-11-28T18:26:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.920+0000 7f29127fc640 1 --2- 172.21.15.105:0/3044341789 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f29000668d0 0x7f2900068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.920+0000 7f29127fc640 1 -- 172.21.15.105:0/3044341789 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f290c084660 con 0x7f291c0adf30 2023-11-28T18:26:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.920+0000 7f2921f94640 1 --2- 172.21.15.105:0/3044341789 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f29000668d0 0x7f2900068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2927571000 1 Processor -- start 2023-11-28T18:26:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2927571000 1 -- start start 2023-11-28T18:26:13.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2921f94640 1 --2- 172.21.15.105:0/3044341789 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f29000668d0 0x7f2900068d90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f29180ad000 tx=0x7f2918008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:13.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2927571000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b3d30 0x55a8d91b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2927571000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8d91b4650 0x55a8d9166fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2927571000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a8d9167500 0x55a8d91678e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2927571000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d9030110 con 0x55a8d91b3d30 2023-11-28T18:26:13.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2927571000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d8fc09a0 con 0x55a8d91b4650 2023-11-28T18:26:13.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2927571000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8d8fbded0 con 0x55a8d9167500 2023-11-28T18:26:13.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2921f94640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b3d30 0x55a8d91b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2921f94640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b3d30 0x55a8d91b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46824/0 (socket says 172.21.15.105:46824) 2023-11-28T18:26:13.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2921f94640 1 -- 172.21.15.105:0/2533933181 learned_addr learned my addr 172.21.15.105:0/2533933181 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:13.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2921793640 1 --2- 172.21.15.105:0/2533933181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8d91b4650 0x55a8d9166fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2922795640 1 --2- 172.21.15.105:0/2533933181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a8d9167500 0x55a8d91678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2921f94640 1 -- 172.21.15.105:0/2533933181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a8d9167500 msgr2=0x55a8d91678e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:13.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2921f94640 1 --2- 172.21.15.105:0/2533933181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a8d9167500 0x55a8d91678e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2921f94640 1 -- 172.21.15.105:0/2533933181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8d91b4650 msgr2=0x55a8d9166fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:13.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2921f94640 1 --2- 172.21.15.105:0/2533933181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8d91b4650 0x55a8d9166fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2921f94640 1 -- 172.21.15.105:0/2533933181 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8d8cb1890 con 0x55a8d91b3d30 2023-11-28T18:26:13.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2921793640 1 --2- 172.21.15.105:0/2533933181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8d91b4650 0x55a8d9166fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:13.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2921f94640 1 --2- 172.21.15.105:0/2533933181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b3d30 0x55a8d91b4110 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f29180ae040 tx=0x7f2918004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe3e19774527de7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:13.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f28ff7fe640 1 -- 172.21.15.105:0/2533933181 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29180bb070 con 0x55a8d91b3d30 2023-11-28T18:26:13.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f28ff7fe640 1 -- 172.21.15.105:0/2533933181 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29180ab8f0 con 0x55a8d91b3d30 2023-11-28T18:26:13.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f28ff7fe640 1 -- 172.21.15.105:0/2533933181 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29180abc60 con 0x55a8d91b3d30 2023-11-28T18:26:13.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2927571000 1 -- 172.21.15.105:0/2533933181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b3d30 msgr2=0x55a8d91b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:13.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2927571000 1 --2- 172.21.15.105:0/2533933181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b3d30 0x55a8d91b4110 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f29180ae040 tx=0x7f2918004740 comp rx=0 tx=0).stop 2023-11-28T18:26:13.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2927571000 1 -- 172.21.15.105:0/2533933181 shutdown_connections 2023-11-28T18:26:13.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2927571000 1 --2- 172.21.15.105:0/2533933181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a8d9167500 0x55a8d91678e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2927571000 1 --2- 172.21.15.105:0/2533933181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8d91b4650 0x55a8d9166fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2927571000 1 --2- 172.21.15.105:0/2533933181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b3d30 0x55a8d91b4110 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2927571000 1 -- 172.21.15.105:0/2533933181 >> 172.21.15.105:0/2533933181 conn(0x55a8d91948c0 msgr2=0x55a8d9196cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:13.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2927571000 1 -- 172.21.15.105:0/2533933181 shutdown_connections 2023-11-28T18:26:13.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2927571000 1 -- 172.21.15.105:0/2533933181 wait complete. 2023-11-28T18:26:13.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.924+0000 7f2927571000 1 -- start start 2023-11-28T18:26:13.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2927571000 1 Processor -- start 2023-11-28T18:26:13.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2927571000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8d9167500 0x55a8d916d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2927571000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a8d91b3d30 0x55a8d916da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2927571000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b4650 0x55a8d916df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2927571000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d8fb97c0 con 0x55a8d91b4650 2023-11-28T18:26:13.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2927571000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d8f71bf0 con 0x55a8d9167500 2023-11-28T18:26:13.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2927571000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8d8f77a90 con 0x55a8d91b3d30 2023-11-28T18:26:13.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2927571000 10 client.0 _ll_register_callbacks cb 0x55a8d9154e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:13.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2922795640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b4650 0x55a8d916df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2921793640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a8d91b3d30 0x55a8d916da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2921793640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a8d91b3d30 0x55a8d916da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:59596/0 (socket says 172.21.15.105:59596) 2023-11-28T18:26:13.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2922795640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b4650 0x55a8d916df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46836/0 (socket says 172.21.15.105:46836) 2023-11-28T18:26:13.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2921793640 1 -- 172.21.15.105:0/1064735783 learned_addr learned my addr 172.21.15.105:0/1064735783 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:13.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2921f94640 1 --2- 172.21.15.105:0/1064735783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8d9167500 0x55a8d916d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2922795640 1 -- 172.21.15.105:0/1064735783 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a8d91b3d30 msgr2=0x55a8d916da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:13.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2922795640 1 --2- 172.21.15.105:0/1064735783 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a8d91b3d30 0x55a8d916da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2922795640 1 -- 172.21.15.105:0/1064735783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8d9167500 msgr2=0x55a8d916d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:13.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2922795640 1 --2- 172.21.15.105:0/1064735783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8d9167500 0x55a8d916d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2922795640 1 -- 172.21.15.105:0/1064735783 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291c161ef0 con 0x55a8d91b4650 2023-11-28T18:26:13.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2921793640 1 --2- 172.21.15.105:0/1064735783 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a8d91b3d30 0x55a8d916da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:26:13.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2922795640 1 --2- 172.21.15.105:0/1064735783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b4650 0x55a8d916df60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2914004340 tx=0x7f2914004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80c2f266faa2533 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:13.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f28fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:13.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f28fcff9640 1 -- 172.21.15.105:0/1064735783 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2914007b20 con 0x55a8d91b4650 2023-11-28T18:26:13.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f28fcff9640 1 -- 172.21.15.105:0/1064735783 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2914007c80 con 0x55a8d91b4650 2023-11-28T18:26:13.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2927571000 1 -- 172.21.15.105:0/1064735783 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8d8f7b470 con 0x55a8d91b4650 2023-11-28T18:26:13.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f28fcff9640 1 -- 172.21.15.105:0/1064735783 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2914007e40 con 0x55a8d91b4650 2023-11-28T18:26:13.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2927571000 10 client.6216 fetch_fsmap learned FSMap version 40 2023-11-28T18:26:13.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2927571000 1 -- 172.21.15.105:0/1064735783 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x55a8d8f7a190 con 0x55a8d91b4650 2023-11-28T18:26:13.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f28fcff9640 1 -- 172.21.15.105:0/1064735783 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a8d8f7b470 con 0x55a8d91b4650 2023-11-28T18:26:13.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f28fcff9640 1 -- 172.21.15.105:0/1064735783 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 40) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f2914014070 con 0x55a8d91b4650 2023-11-28T18:26:13.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2927571000 10 client.6216 fetch_fsmap finished waiting for FSMap version 40 2023-11-28T18:26:13.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2927571000 10 client.6216 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:26:13.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2927571000 20 client.6216 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:13.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2927571000 1 --2- 172.21.15.105:0/1064735783 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55a8d916e970 0x55a8d91c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2927571000 4 client.6216 mds_command: new command op to 5408 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:26:13.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2927571000 1 -- 172.21.15.105:0/1064735783 --> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8d8f7eb50 con 0x55a8d916e970 2023-11-28T18:26:13.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2921f94640 1 --2- 172.21.15.105:0/1064735783 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55a8d916e970 0x55a8d91c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f28fcff9640 10 client.6216 ms_handle_connect on v2:172.21.15.105:6835/3765684108 2023-11-28T18:26:13.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.928+0000 7f2921f94640 1 --2- 172.21.15.105:0/1064735783 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55a8d916e970 0x55a8d91c5ff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a4ef124a0391757 server_cookie=54f14c2707b95600 in_seq=0 out_seq=0 2023-11-28T18:26:13.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.932+0000 7f28fcff9640 1 -- 172.21.15.105:0/1064735783 <== mds.2 v2:172.21.15.105:6835/3765684108 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a8d8f7eb50 con 0x55a8d916e970 2023-11-28T18:26:13.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.932+0000 7f28fcff9640 10 client.6216 handle_command_reply: tid=0 2023-11-28T18:26:13.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.932+0000 7f2927571000 1 client.6216 shutdown 2023-11-28T18:26:13.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.932+0000 7f2927571000 1 -- 172.21.15.105:0/1064735783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b4650 msgr2=0x55a8d916df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:13.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.932+0000 7f2927571000 1 --2- 172.21.15.105:0/1064735783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b4650 0x55a8d916df60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2914004340 tx=0x7f2914004800 comp rx=0 tx=0).stop 2023-11-28T18:26:13.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.932+0000 7f2927571000 1 -- 172.21.15.105:0/1064735783 shutdown_connections 2023-11-28T18:26:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.932+0000 7f2927571000 1 --2- 172.21.15.105:0/1064735783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b4650 0x55a8d916df60 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.932+0000 7f2927571000 1 --2- 172.21.15.105:0/1064735783 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a8d91b3d30 0x55a8d916da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.932+0000 7f2927571000 1 --2- 172.21.15.105:0/1064735783 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55a8d916e970 0x55a8d91c5ff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.932+0000 7f2927571000 1 --2- 172.21.15.105:0/1064735783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8d9167500 0x55a8d916d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.932+0000 7f2927571000 1 -- 172.21.15.105:0/1064735783 >> 172.21.15.105:0/1064735783 conn(0x55a8d91948c0 msgr2=0x55a8d91666b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.932+0000 7f28fcff9640 0 client.6216 ms_handle_reset on v2:172.21.15.105:6835/3765684108 2023-11-28T18:26:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.932+0000 7f2927571000 1 -- 172.21.15.105:0/1064735783 shutdown_connections 2023-11-28T18:26:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.932+0000 7f2927571000 1 -- 172.21.15.105:0/1064735783 wait complete. 2023-11-28T18:26:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.932+0000 7f2927571000 20 client.6216 trim_cache size 0 max 16384 2023-11-28T18:26:13.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2927571000 1 Processor -- start 2023-11-28T18:26:13.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2927571000 1 -- start start 2023-11-28T18:26:13.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2927571000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8d9167500 0x55a8d91933a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2927571000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b3d30 0x55a8d91938e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2927571000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a8d91b4650 0x55a8d91c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2927571000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d8c72940 con 0x55a8d91b3d30 2023-11-28T18:26:13.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2927571000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d9197780 con 0x55a8d9167500 2023-11-28T18:26:13.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2927571000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8d8f548a0 con 0x55a8d91b4650 2023-11-28T18:26:13.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2921793640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b3d30 0x55a8d91938e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2921793640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b3d30 0x55a8d91938e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46852/0 (socket says 172.21.15.105:46852) 2023-11-28T18:26:13.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2921793640 1 -- 172.21.15.105:0/1691833740 learned_addr learned my addr 172.21.15.105:0/1691833740 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:13.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2921793640 1 -- 172.21.15.105:0/1691833740 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a8d91b4650 msgr2=0x55a8d91c5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:13.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2921793640 1 --2- 172.21.15.105:0/1691833740 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a8d91b4650 0x55a8d91c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2921793640 1 -- 172.21.15.105:0/1691833740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8d9167500 msgr2=0x55a8d91933a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:13.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2921793640 1 --2- 172.21.15.105:0/1691833740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8d9167500 0x55a8d91933a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2921793640 1 -- 172.21.15.105:0/1691833740 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8d9167370 con 0x55a8d91b3d30 2023-11-28T18:26:13.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2921793640 1 --2- 172.21.15.105:0/1691833740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b3d30 0x55a8d91938e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f290c0553a0 tx=0x7f290c055bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950dd12da80b2416 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:13.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f28fffff640 1 -- 172.21.15.105:0/1691833740 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c05d710 con 0x55a8d91b3d30 2023-11-28T18:26:13.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f28fffff640 1 -- 172.21.15.105:0/1691833740 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290c053550 con 0x55a8d91b3d30 2023-11-28T18:26:13.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f28fffff640 1 -- 172.21.15.105:0/1691833740 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c059160 con 0x55a8d91b3d30 2023-11-28T18:26:13.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2927571000 1 -- 172.21.15.105:0/1691833740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b3d30 msgr2=0x55a8d91938e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:13.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2927571000 1 --2- 172.21.15.105:0/1691833740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b3d30 0x55a8d91938e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f290c0553a0 tx=0x7f290c055bd0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2927571000 1 -- 172.21.15.105:0/1691833740 shutdown_connections 2023-11-28T18:26:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2927571000 1 --2- 172.21.15.105:0/1691833740 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a8d91b4650 0x55a8d91c5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2927571000 1 --2- 172.21.15.105:0/1691833740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b3d30 0x55a8d91938e0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2927571000 1 --2- 172.21.15.105:0/1691833740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8d9167500 0x55a8d91933a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2927571000 1 -- 172.21.15.105:0/1691833740 >> 172.21.15.105:0/1691833740 conn(0x55a8d91948c0 msgr2=0x55a8d9165720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2927571000 1 -- 172.21.15.105:0/1691833740 shutdown_connections 2023-11-28T18:26:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2927571000 1 -- 172.21.15.105:0/1691833740 wait complete. 2023-11-28T18:26:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.948+0000 7f2927571000 1 -- start start 2023-11-28T18:26:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2927571000 1 Processor -- start 2023-11-28T18:26:13.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2927571000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a8d9167500 0x55a8d9214810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2927571000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8d91b3d30 0x55a8d9214d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2927571000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b4650 0x55a8d91cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2927571000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d9161430 con 0x55a8d91b4650 2023-11-28T18:26:13.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2927571000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d91cace0 con 0x55a8d91b3d30 2023-11-28T18:26:13.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2927571000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8d916add0 con 0x55a8d9167500 2023-11-28T18:26:13.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2927571000 10 client.0 _ll_register_callbacks cb 0x55a8d9030060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:13.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2922795640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b4650 0x55a8d91cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2922795640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b4650 0x55a8d91cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46866/0 (socket says 172.21.15.105:46866) 2023-11-28T18:26:13.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2922795640 1 -- 172.21.15.105:0/3510391807 learned_addr learned my addr 172.21.15.105:0/3510391807 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:13.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2921f94640 1 --2- 172.21.15.105:0/3510391807 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a8d9167500 0x55a8d9214810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2921793640 1 --2- 172.21.15.105:0/3510391807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8d91b3d30 0x55a8d9214d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2922795640 1 -- 172.21.15.105:0/3510391807 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a8d9167500 msgr2=0x55a8d9214810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:13.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2922795640 1 --2- 172.21.15.105:0/3510391807 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a8d9167500 0x55a8d9214810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2922795640 1 -- 172.21.15.105:0/3510391807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8d91b3d30 msgr2=0x55a8d9214d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:13.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2922795640 1 --2- 172.21.15.105:0/3510391807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8d91b3d30 0x55a8d9214d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2922795640 1 -- 172.21.15.105:0/3510391807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291c163050 con 0x55a8d91b4650 2023-11-28T18:26:13.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2922795640 1 --2- 172.21.15.105:0/3510391807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b4650 0x55a8d91cfa70 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2914007dd0 tx=0x7f2914004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb084535ebd2045c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:13.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f28fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:13.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f28fcff9640 1 -- 172.21.15.105:0/3510391807 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2914003a80 con 0x55a8d91b4650 2023-11-28T18:26:13.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f28fcff9640 1 -- 172.21.15.105:0/3510391807 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2914007700 con 0x55a8d91b4650 2023-11-28T18:26:13.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2927571000 1 -- 172.21.15.105:0/3510391807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8d8fb8110 con 0x55a8d91b4650 2023-11-28T18:26:13.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f28fcff9640 1 -- 172.21.15.105:0/3510391807 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f291401e440 con 0x55a8d91b4650 2023-11-28T18:26:13.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2927571000 10 client.6222 fetch_fsmap learned FSMap version 40 2023-11-28T18:26:13.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2927571000 1 -- 172.21.15.105:0/3510391807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x55a8d8f60490 con 0x55a8d91b4650 2023-11-28T18:26:13.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f28fcff9640 1 -- 172.21.15.105:0/3510391807 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a8d8fb8110 con 0x55a8d91b4650 2023-11-28T18:26:13.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f28fcff9640 1 -- 172.21.15.105:0/3510391807 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 40) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f291401e600 con 0x55a8d91b4650 2023-11-28T18:26:13.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2927571000 10 client.6222 fetch_fsmap finished waiting for FSMap version 40 2023-11-28T18:26:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2927571000 10 client.6222 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:26:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2927571000 20 client.6222 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2927571000 1 --2- 172.21.15.105:0/3510391807 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55a8d921b670 0x55a8d921da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2927571000 4 client.6222 mds_command: new command op to 5408 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:26:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2927571000 1 -- 172.21.15.105:0/3510391807 --> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a8d8fe6650 con 0x55a8d921b670 2023-11-28T18:26:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2921f94640 1 --2- 172.21.15.105:0/3510391807 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55a8d921b670 0x55a8d921da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2921f94640 1 --2- 172.21.15.105:0/3510391807 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55a8d921b670 0x55a8d921da60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a255f566e979172 server_cookie=f132f787cd69eedc in_seq=0 out_seq=0 2023-11-28T18:26:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f28fcff9640 10 client.6222 ms_handle_connect on v2:172.21.15.105:6835/3765684108 2023-11-28T18:26:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f28fcff9640 1 -- 172.21.15.105:0/3510391807 <== mds.2 v2:172.21.15.105:6835/3765684108 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55a8d8fe6650 con 0x55a8d921b670 2023-11-28T18:26:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f28fcff9640 10 client.6222 handle_command_reply: tid=0 2023-11-28T18:26:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2927571000 1 client.6222 shutdown 2023-11-28T18:26:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2927571000 1 -- 172.21.15.105:0/3510391807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b4650 msgr2=0x55a8d91cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:13.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.952+0000 7f2927571000 1 --2- 172.21.15.105:0/3510391807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b4650 0x55a8d91cfa70 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2914007dd0 tx=0x7f2914004ac0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.956+0000 7f2927571000 1 -- 172.21.15.105:0/3510391807 shutdown_connections 2023-11-28T18:26:13.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.956+0000 7f2927571000 1 --2- 172.21.15.105:0/3510391807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8d91b4650 0x55a8d91cfa70 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.956+0000 7f2927571000 1 --2- 172.21.15.105:0/3510391807 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55a8d921b670 0x55a8d921da60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.956+0000 7f2927571000 1 --2- 172.21.15.105:0/3510391807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8d91b3d30 0x55a8d9214d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.956+0000 7f2927571000 1 --2- 172.21.15.105:0/3510391807 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a8d9167500 0x55a8d9214810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.956+0000 7f28fcff9640 0 client.6222 ms_handle_reset on v2:172.21.15.105:6835/3765684108 2023-11-28T18:26:13.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.956+0000 7f2927571000 1 -- 172.21.15.105:0/3510391807 >> 172.21.15.105:0/3510391807 conn(0x55a8d91948c0 msgr2=0x55a8d9165fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:13.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.956+0000 7f2927571000 1 -- 172.21.15.105:0/3510391807 shutdown_connections 2023-11-28T18:26:13.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.956+0000 7f2927571000 1 -- 172.21.15.105:0/3510391807 wait complete. 2023-11-28T18:26:13.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.956+0000 7f2927571000 20 client.6222 trim_cache size 0 max 16384 2023-11-28T18:26:13.965 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:26:13.965 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:26:13.965 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 2, 2023-11-28T18:26:13.965 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5408, 2023-11-28T18:26:13.965 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:26:13.965 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:26:13.965 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:26:13.965 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 1.7367636609999999, 2023-11-28T18:26:13.965 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 40, 2023-11-28T18:26:13.965 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 51, 2023-11-28T18:26:13.965 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 51, 2023-11-28T18:26:13.966 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 17.667717792000001 2023-11-28T18:26:13.966 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:26:13.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.956+0000 7f29241f8640 1 -- 172.21.15.105:0/3044341789 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f29000668d0 msgr2=0x7f2900068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:13.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.956+0000 7f29241f8640 1 --2- 172.21.15.105:0/3044341789 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f29000668d0 0x7f2900068d90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f29180ad000 tx=0x7f2918008040 comp rx=0 tx=0).stop 2023-11-28T18:26:13.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.956+0000 7f29241f8640 1 -- 172.21.15.105:0/3044341789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f291c0adf30 msgr2=0x7f291c0b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:13.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.956+0000 7f29241f8640 1 --2- 172.21.15.105:0/3044341789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f291c0adf30 0x7f291c0b4460 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f290c00e8a0 tx=0x7f290c00eca0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.956+0000 7f29241f8640 1 -- 172.21.15.105:0/3044341789 shutdown_connections 2023-11-28T18:26:13.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.956+0000 7f29241f8640 1 --2- 172.21.15.105:0/3044341789 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f291c0ce530 0x7f291c09a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.956+0000 7f29241f8640 1 --2- 172.21.15.105:0/3044341789 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f29000668d0 0x7f2900068d90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.956+0000 7f29241f8640 1 --2- 172.21.15.105:0/3044341789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f291c0adf30 0x7f291c0b4460 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.956+0000 7f29241f8640 1 --2- 172.21.15.105:0/3044341789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f291c06d610 0x7f291c1608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:13.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.956+0000 7f29241f8640 1 -- 172.21.15.105:0/3044341789 >> 172.21.15.105:0/3044341789 conn(0x7f291c0ceb90 msgr2=0x7f291c0acc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:13.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.956+0000 7f29241f8640 1 -- 172.21.15.105:0/3044341789 shutdown_connections 2023-11-28T18:26:13.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:13.956+0000 7f29241f8640 1 -- 172.21.15.105:0/3044341789 wait complete. 2023-11-28T18:26:13.976 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-28T18:26:14.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.112+0000 7f741afd1640 1 Processor -- start 2023-11-28T18:26:14.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.112+0000 7f741afd1640 1 -- start start 2023-11-28T18:26:14.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.112+0000 7f741afd1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f74140caf60 0x7f74140cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.112+0000 7f741afd1640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f741406bf50 con 0x7f7414159f40 2023-11-28T18:26:14.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.112+0000 7f741afd1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f741415a320 con 0x7f74140ad850 2023-11-28T18:26:14.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.112+0000 7f741afd1640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74140cb3c0 con 0x7f74140caf60 2023-11-28T18:26:14.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.112+0000 7f7418d6d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7414159f40 0x7f74140ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49202/0 (socket says 172.21.15.105:49202) 2023-11-28T18:26:14.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.112+0000 7f7418d6d640 1 -- 172.21.15.105:0/964002975 learned_addr learned my addr 172.21.15.105:0/964002975 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:14.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.116+0000 7f7413fff640 1 --2- 172.21.15.105:0/964002975 >> v2:172.21.15.105:3300/0 conn(0x7f74140caf60 0x7f74140cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.116+0000 7f74137fe640 1 -- 172.21.15.105:0/964002975 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190025054 0 0) 0x7f741406bf50 con 0x7f7414159f40 2023-11-28T18:26:14.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.116+0000 7f74137fe640 1 -- 172.21.15.105:0/964002975 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7404003530 con 0x7f7414159f40 2023-11-28T18:26:14.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.116+0000 7f7413fff640 1 -- 172.21.15.105:0/964002975 >> v1:172.21.15.148:6789/0 conn(0x7f74140ad850 legacy=0x7f74140701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:14.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.116+0000 7f7413fff640 1 -- 172.21.15.105:0/964002975 >> v1:172.21.15.105:6789/0 conn(0x7f7414159f40 legacy=0x7f74140ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:14.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.116+0000 7f7413fff640 1 -- 172.21.15.105:0/964002975 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7414071380 con 0x7f74140caf60 2023-11-28T18:26:14.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.116+0000 7f7413fff640 1 --2- 172.21.15.105:0/964002975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74140caf60 0x7f74140cc760 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f74000097c0 tx=0x7f740001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb38d8ee957483db server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:14.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.116+0000 7f74137fe640 1 -- 172.21.15.105:0/964002975 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74000ac620 con 0x7f74140caf60 2023-11-28T18:26:14.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.116+0000 7f74137fe640 1 -- 172.21.15.105:0/964002975 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74000ac780 con 0x7f74140caf60 2023-11-28T18:26:14.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.116+0000 7f74137fe640 1 -- 172.21.15.105:0/964002975 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74000acb10 con 0x7f74140caf60 2023-11-28T18:26:14.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f741afd1640 1 -- 172.21.15.105:0/964002975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74140caf60 msgr2=0x7f74140cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:14.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f741afd1640 1 --2- 172.21.15.105:0/964002975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74140caf60 0x7f74140cc760 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f74000097c0 tx=0x7f740001c3c0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f741afd1640 1 -- 172.21.15.105:0/964002975 shutdown_connections 2023-11-28T18:26:14.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f741afd1640 1 --2- 172.21.15.105:0/964002975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74140caf60 0x7f74140cc760 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f741afd1640 1 -- 172.21.15.105:0/964002975 >> 172.21.15.105:0/964002975 conn(0x7f74140ceb90 msgr2=0x7f74140cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:14.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f741afd1640 1 -- 172.21.15.105:0/964002975 shutdown_connections 2023-11-28T18:26:14.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f741afd1640 1 -- 172.21.15.105:0/964002975 wait complete. 2023-11-28T18:26:14.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f741afd1640 1 Processor -- start 2023-11-28T18:26:14.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f741afd1640 1 -- start start 2023-11-28T18:26:14.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f741afd1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74140ad850 0x7f7414153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f741afd1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74140caf60 0x7f74140b3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f741afd1640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7414159f40 0x7f74140b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f741afd1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741409c950 con 0x7f74140ad850 2023-11-28T18:26:14.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f741afd1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74141570b0 con 0x7f74140caf60 2023-11-28T18:26:14.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f741afd1640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7414154500 con 0x7f7414159f40 2023-11-28T18:26:14.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f7418d6d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74140ad850 0x7f7414153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f7418d6d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74140ad850 0x7f7414153fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46894/0 (socket says 172.21.15.105:46894) 2023-11-28T18:26:14.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f7418d6d640 1 -- 172.21.15.105:0/3827520148 learned_addr learned my addr 172.21.15.105:0/3827520148 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:14.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f7413fff640 1 --2- 172.21.15.105:0/3827520148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74140caf60 0x7f74140b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f741956e640 1 --2- 172.21.15.105:0/3827520148 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7414159f40 0x7f74140b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f7418d6d640 1 -- 172.21.15.105:0/3827520148 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7414159f40 msgr2=0x7f74140b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:14.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f7418d6d640 1 --2- 172.21.15.105:0/3827520148 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7414159f40 0x7f74140b1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f7418d6d640 1 -- 172.21.15.105:0/3827520148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74140caf60 msgr2=0x7f74140b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:14.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f7418d6d640 1 --2- 172.21.15.105:0/3827520148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74140caf60 0x7f74140b3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f7418d6d640 1 -- 172.21.15.105:0/3827520148 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7400009420 con 0x7f74140ad850 2023-11-28T18:26:14.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f741956e640 1 --2- 172.21.15.105:0/3827520148 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7414159f40 0x7f74140b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:14.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.120+0000 7f7418d6d640 1 --2- 172.21.15.105:0/3827520148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74140ad850 0x7f7414153fc0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f740800c420 tx=0x7f740800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:14.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.124+0000 7f74117fa640 1 -- 172.21.15.105:0/3827520148 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7408005cf0 con 0x7f74140ad850 2023-11-28T18:26:14.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.124+0000 7f74117fa640 1 -- 172.21.15.105:0/3827520148 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7408005e50 con 0x7f74140ad850 2023-11-28T18:26:14.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.124+0000 7f74117fa640 1 -- 172.21.15.105:0/3827520148 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74080072d0 con 0x7f74140ad850 2023-11-28T18:26:14.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.124+0000 7f741afd1640 1 -- 172.21.15.105:0/3827520148 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7414156af0 con 0x7f74140ad850 2023-11-28T18:26:14.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.124+0000 7f741afd1640 1 -- 172.21.15.105:0/3827520148 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7414099d30 con 0x7f74140ad850 2023-11-28T18:26:14.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.124+0000 7f74117fa640 1 -- 172.21.15.105:0/3827520148 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f7408007430 con 0x7f74140ad850 2023-11-28T18:26:14.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f74117fa640 1 --2- 172.21.15.105:0/3827520148 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f73f80668d0 0x7f73f8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f74117fa640 1 -- 172.21.15.105:0/3827520148 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7408083b00 con 0x7f74140ad850 2023-11-28T18:26:14.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f7413fff640 1 --2- 172.21.15.105:0/3827520148 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f73f80668d0 0x7f73f8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741e346000 1 Processor -- start 2023-11-28T18:26:14.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741e346000 1 -- start start 2023-11-28T18:26:14.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f7413fff640 1 --2- 172.21.15.105:0/3827520148 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f73f80668d0 0x7f73f8068d90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f74000ae000 tx=0x7f7400008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:14.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741e346000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b67fdd190 0x564b67fdd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741e346000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67fddab0 0x564b67f903d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741e346000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67f90910 0x564b67f90cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741e346000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b67e55110 con 0x564b67f90910 2023-11-28T18:26:14.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741e346000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b67de59a0 con 0x564b67fdd190 2023-11-28T18:26:14.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741e346000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b67de2ed0 con 0x564b67fddab0 2023-11-28T18:26:14.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741956e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67f90910 0x564b67f90cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741956e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67f90910 0x564b67f90cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46896/0 (socket says 172.21.15.105:46896) 2023-11-28T18:26:14.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741956e640 1 -- 172.21.15.105:0/3117173856 learned_addr learned my addr 172.21.15.105:0/3117173856 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:14.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f7418d6d640 1 --2- 172.21.15.105:0/3117173856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b67fdd190 0x564b67fdd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f7413fff640 1 --2- 172.21.15.105:0/3117173856 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67fddab0 0x564b67f903d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741956e640 1 -- 172.21.15.105:0/3117173856 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67fddab0 msgr2=0x564b67f903d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:14.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741956e640 1 --2- 172.21.15.105:0/3117173856 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67fddab0 0x564b67f903d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741956e640 1 -- 172.21.15.105:0/3117173856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b67fdd190 msgr2=0x564b67fdd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:14.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741956e640 1 --2- 172.21.15.105:0/3117173856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b67fdd190 0x564b67fdd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741956e640 1 -- 172.21.15.105:0/3117173856 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b67ad6890 con 0x564b67f90910 2023-11-28T18:26:14.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741956e640 1 --2- 172.21.15.105:0/3117173856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67f90910 0x564b67f90cf0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f740c001d30 tx=0x7f740c0021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3467d122495864 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:14.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f73fe7fc640 1 -- 172.21.15.105:0/3117173856 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c003b90 con 0x564b67f90910 2023-11-28T18:26:14.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f73fe7fc640 1 -- 172.21.15.105:0/3117173856 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740c003cf0 con 0x564b67f90910 2023-11-28T18:26:14.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f73fe7fc640 1 -- 172.21.15.105:0/3117173856 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c00eb20 con 0x564b67f90910 2023-11-28T18:26:14.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741e346000 1 -- 172.21.15.105:0/3117173856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67f90910 msgr2=0x564b67f90cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:14.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741e346000 1 --2- 172.21.15.105:0/3117173856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67f90910 0x564b67f90cf0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f740c001d30 tx=0x7f740c0021f0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741e346000 1 -- 172.21.15.105:0/3117173856 shutdown_connections 2023-11-28T18:26:14.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741e346000 1 --2- 172.21.15.105:0/3117173856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67f90910 0x564b67f90cf0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741e346000 1 --2- 172.21.15.105:0/3117173856 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67fddab0 0x564b67f903d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741e346000 1 --2- 172.21.15.105:0/3117173856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b67fdd190 0x564b67fdd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741e346000 1 -- 172.21.15.105:0/3117173856 >> 172.21.15.105:0/3117173856 conn(0x564b67fbdd20 msgr2=0x564b67fc0110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:14.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741e346000 1 -- 172.21.15.105:0/3117173856 shutdown_connections 2023-11-28T18:26:14.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.128+0000 7f741e346000 1 -- 172.21.15.105:0/3117173856 wait complete. 2023-11-28T18:26:14.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741e346000 1 -- start start 2023-11-28T18:26:14.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741e346000 1 Processor -- start 2023-11-28T18:26:14.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741e346000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67f90910 0x564b67fc2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741e346000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b67fdd190 0x564b67fc3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741e346000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67fddab0 0x564b67fc3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741e346000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b67dde7c0 con 0x564b67fddab0 2023-11-28T18:26:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741e346000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b67d96bf0 con 0x564b67fdd190 2023-11-28T18:26:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741e346000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b67d9ca90 con 0x564b67f90910 2023-11-28T18:26:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741e346000 10 client.0 _ll_register_callbacks cb 0x564b67f79e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741956e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67fddab0 0x564b67fc3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741956e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67fddab0 0x564b67fc3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46900/0 (socket says 172.21.15.105:46900) 2023-11-28T18:26:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741956e640 1 -- 172.21.15.105:0/1098602932 learned_addr learned my addr 172.21.15.105:0/1098602932 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f7418d6d640 1 --2- 172.21.15.105:0/1098602932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67f90910 0x564b67fc2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f7413fff640 1 --2- 172.21.15.105:0/1098602932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b67fdd190 0x564b67fc3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741956e640 1 -- 172.21.15.105:0/1098602932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67f90910 msgr2=0x564b67fc2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741956e640 1 --2- 172.21.15.105:0/1098602932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67f90910 0x564b67fc2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741956e640 1 -- 172.21.15.105:0/1098602932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b67fdd190 msgr2=0x564b67fc3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741956e640 1 --2- 172.21.15.105:0/1098602932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b67fdd190 0x564b67fc3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741956e640 1 -- 172.21.15.105:0/1098602932 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740c00f040 con 0x564b67fddab0 2023-11-28T18:26:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f7418d6d640 1 --2- 172.21.15.105:0/1098602932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67f90910 0x564b67fc2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741956e640 1 --2- 172.21.15.105:0/1098602932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67fddab0 0x564b67fc3770 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f740c000f90 tx=0x7f740c001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754fcf16400c2095 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f73f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f73f37fe640 1 -- 172.21.15.105:0/1098602932 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c006d70 con 0x564b67fddab0 2023-11-28T18:26:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f73f37fe640 1 -- 172.21.15.105:0/1098602932 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f740c00e920 con 0x564b67fddab0 2023-11-28T18:26:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741e346000 1 -- 172.21.15.105:0/1098602932 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b67da0470 con 0x564b67fddab0 2023-11-28T18:26:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f73f37fe640 1 -- 172.21.15.105:0/1098602932 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f740c00eb20 con 0x564b67fddab0 2023-11-28T18:26:14.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741e346000 10 client.6237 fetch_fsmap learned FSMap version 40 2023-11-28T18:26:14.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741e346000 1 -- 172.21.15.105:0/1098602932 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x564b67d9f190 con 0x564b67fddab0 2023-11-28T18:26:14.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f73f37fe640 1 -- 172.21.15.105:0/1098602932 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564b67da0470 con 0x564b67fddab0 2023-11-28T18:26:14.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f73f37fe640 1 -- 172.21.15.105:0/1098602932 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 40) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f740c00ec80 con 0x564b67fddab0 2023-11-28T18:26:14.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741e346000 10 client.6237 fetch_fsmap finished waiting for FSMap version 40 2023-11-28T18:26:14.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741e346000 10 client.6237 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T18:26:14.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741e346000 20 client.6237 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:14.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741e346000 1 --2- 172.21.15.105:0/1098602932 >> [v2:172.21.15.148:6833/897997046,v1:172.21.15.148:6835/897997046] conn(0x564b67fed440 0x564b67fef830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741e346000 4 client.6237 mds_command: new command op to 5414 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:26:14.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f741e346000 1 -- 172.21.15.105:0/1098602932 --> [v2:172.21.15.148:6833/897997046,v1:172.21.15.148:6835/897997046] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b67da3b50 con 0x564b67fed440 2023-11-28T18:26:14.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.132+0000 7f7418d6d640 1 --2- 172.21.15.105:0/1098602932 >> [v2:172.21.15.148:6833/897997046,v1:172.21.15.148:6835/897997046] conn(0x564b67fed440 0x564b67fef830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.136+0000 7f7418d6d640 1 --2- 172.21.15.105:0/1098602932 >> [v2:172.21.15.148:6833/897997046,v1:172.21.15.148:6835/897997046] conn(0x564b67fed440 0x564b67fef830 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=391061e860d7048a server_cookie=13ce08d9a8163156 in_seq=0 out_seq=0 2023-11-28T18:26:14.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.136+0000 7f73f37fe640 10 client.6237 ms_handle_connect on v2:172.21.15.148:6833/897997046 2023-11-28T18:26:14.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.136+0000 7f73f37fe640 1 -- 172.21.15.105:0/1098602932 <== mds.1 v2:172.21.15.148:6833/897997046 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564b67da3b50 con 0x564b67fed440 2023-11-28T18:26:14.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.136+0000 7f73f37fe640 10 client.6237 handle_command_reply: tid=0 2023-11-28T18:26:14.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.136+0000 7f741e346000 1 client.6237 shutdown 2023-11-28T18:26:14.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.136+0000 7f741e346000 1 -- 172.21.15.105:0/1098602932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67fddab0 msgr2=0x564b67fc3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:14.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.136+0000 7f741e346000 1 --2- 172.21.15.105:0/1098602932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67fddab0 0x564b67fc3770 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f740c000f90 tx=0x7f740c001260 comp rx=0 tx=0).stop 2023-11-28T18:26:14.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.136+0000 7f741e346000 1 -- 172.21.15.105:0/1098602932 shutdown_connections 2023-11-28T18:26:14.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.136+0000 7f741e346000 1 --2- 172.21.15.105:0/1098602932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67fddab0 0x564b67fc3770 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.136+0000 7f741e346000 1 --2- 172.21.15.105:0/1098602932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b67fdd190 0x564b67fc3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.136+0000 7f741e346000 1 --2- 172.21.15.105:0/1098602932 >> [v2:172.21.15.148:6833/897997046,v1:172.21.15.148:6835/897997046] conn(0x564b67fed440 0x564b67fef830 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.136+0000 7f741e346000 1 --2- 172.21.15.105:0/1098602932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67f90910 0x564b67fc2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.136+0000 7f741e346000 1 -- 172.21.15.105:0/1098602932 >> 172.21.15.105:0/1098602932 conn(0x564b67fbdd20 msgr2=0x564b67fc0110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:14.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.136+0000 7f73f37fe640 0 client.6237 ms_handle_reset on v2:172.21.15.148:6833/897997046 2023-11-28T18:26:14.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.136+0000 7f741e346000 1 -- 172.21.15.105:0/1098602932 shutdown_connections 2023-11-28T18:26:14.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.136+0000 7f741e346000 1 -- 172.21.15.105:0/1098602932 wait complete. 2023-11-28T18:26:14.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.136+0000 7f741e346000 20 client.6237 trim_cache size 0 max 16384 2023-11-28T18:26:14.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f741e346000 1 Processor -- start 2023-11-28T18:26:14.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f741e346000 1 -- start start 2023-11-28T18:26:14.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f741e346000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67f90910 0x564b67fbb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f741e346000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67fdd190 0x564b67fbb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f741e346000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b67fddab0 0x564b67fbbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f741e346000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b67a97940 con 0x564b67f90910 2023-11-28T18:26:14.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f741e346000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b67d798a0 con 0x564b67fddab0 2023-11-28T18:26:14.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f741e346000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b67de27c0 con 0x564b67fdd190 2023-11-28T18:26:14.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f7418d6d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67f90910 0x564b67fbb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f7418d6d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67f90910 0x564b67fbb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46902/0 (socket says 172.21.15.105:46902) 2023-11-28T18:26:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f7418d6d640 1 -- 172.21.15.105:0/1331958631 learned_addr learned my addr 172.21.15.105:0/1331958631 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f741956e640 1 --2- 172.21.15.105:0/1331958631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b67fddab0 0x564b67fbbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f7413fff640 1 --2- 172.21.15.105:0/1331958631 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67fdd190 0x564b67fbb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f7418d6d640 1 -- 172.21.15.105:0/1331958631 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67fdd190 msgr2=0x564b67fbb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f7418d6d640 1 --2- 172.21.15.105:0/1331958631 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67fdd190 0x564b67fbb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f7418d6d640 1 -- 172.21.15.105:0/1331958631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b67fddab0 msgr2=0x564b67fbbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f7418d6d640 1 --2- 172.21.15.105:0/1331958631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b67fddab0 0x564b67fbbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f7418d6d640 1 -- 172.21.15.105:0/1331958631 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b67d85490 con 0x564b67f90910 2023-11-28T18:26:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f741956e640 1 --2- 172.21.15.105:0/1331958631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b67fddab0 0x564b67fbbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f7413fff640 1 --2- 172.21.15.105:0/1331958631 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67fdd190 0x564b67fbb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f7418d6d640 1 --2- 172.21.15.105:0/1331958631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67f90910 0x564b67fbb3b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7408081ef0 tx=0x7f740800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a24f9219a2ddd4e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f73feffd640 1 -- 172.21.15.105:0/1331958631 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740806c1e0 con 0x564b67f90910 2023-11-28T18:26:14.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f73feffd640 1 -- 172.21.15.105:0/1331958631 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740806c070 con 0x564b67f90910 2023-11-28T18:26:14.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f73feffd640 1 -- 172.21.15.105:0/1331958631 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7408052d10 con 0x564b67f90910 2023-11-28T18:26:14.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f741e346000 1 -- 172.21.15.105:0/1331958631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67f90910 msgr2=0x564b67fbb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:14.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f741e346000 1 --2- 172.21.15.105:0/1331958631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67f90910 0x564b67fbb3b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7408081ef0 tx=0x7f740800cf90 comp rx=0 tx=0).stop 2023-11-28T18:26:14.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f741e346000 1 -- 172.21.15.105:0/1331958631 shutdown_connections 2023-11-28T18:26:14.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f741e346000 1 --2- 172.21.15.105:0/1331958631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b67fddab0 0x564b67fbbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f741e346000 1 --2- 172.21.15.105:0/1331958631 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67fdd190 0x564b67fbb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f741e346000 1 --2- 172.21.15.105:0/1331958631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67f90910 0x564b67fbb3b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f741e346000 1 -- 172.21.15.105:0/1331958631 >> 172.21.15.105:0/1331958631 conn(0x564b67fbdd20 msgr2=0x564b67fdc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:14.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f741e346000 1 -- 172.21.15.105:0/1331958631 shutdown_connections 2023-11-28T18:26:14.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.152+0000 7f741e346000 1 -- 172.21.15.105:0/1331958631 wait complete. 2023-11-28T18:26:14.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f741e346000 1 -- start start 2023-11-28T18:26:14.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f741e346000 1 Processor -- start 2023-11-28T18:26:14.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f741e346000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67f90910 0x564b67ff1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f741e346000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67fdd190 0x564b67fed820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f741e346000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b67fddab0 0x564b67fedd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f741e346000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b67d9de20 con 0x564b67fdd190 2023-11-28T18:26:14.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f741e346000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b67f94030 con 0x564b67fddab0 2023-11-28T18:26:14.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f741e346000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b67d9b9b0 con 0x564b67f90910 2023-11-28T18:26:14.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f741e346000 10 client.0 _ll_register_callbacks cb 0x564b67e55060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:14.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f7413fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67fdd190 0x564b67fed820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f7413fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67fdd190 0x564b67fed820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46916/0 (socket says 172.21.15.105:46916) 2023-11-28T18:26:14.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f7413fff640 1 -- 172.21.15.105:0/2133693465 learned_addr learned my addr 172.21.15.105:0/2133693465 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:14.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f741956e640 1 --2- 172.21.15.105:0/2133693465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b67fddab0 0x564b67fedd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f7413fff640 1 -- 172.21.15.105:0/2133693465 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67f90910 msgr2=0x564b67ff1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:14.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f7418d6d640 1 --2- 172.21.15.105:0/2133693465 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67f90910 0x564b67ff1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f7413fff640 1 --2- 172.21.15.105:0/2133693465 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67f90910 0x564b67ff1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f7413fff640 1 -- 172.21.15.105:0/2133693465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b67fddab0 msgr2=0x564b67fedd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:14.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f7413fff640 1 --2- 172.21.15.105:0/2133693465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b67fddab0 0x564b67fedd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f7413fff640 1 -- 172.21.15.105:0/2133693465 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7414099d30 con 0x564b67fdd190 2023-11-28T18:26:14.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f741956e640 1 --2- 172.21.15.105:0/2133693465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b67fddab0 0x564b67fedd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:14.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f7418d6d640 1 --2- 172.21.15.105:0/2133693465 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67f90910 0x564b67ff1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:14.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f7413fff640 1 --2- 172.21.15.105:0/2133693465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67fdd190 0x564b67fed820 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f740001c8a0 tx=0x7f74000046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a516d6cee88879b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:14.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f73f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:14.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f73f37fe640 1 -- 172.21.15.105:0/2133693465 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74000c5620 con 0x564b67fdd190 2023-11-28T18:26:14.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f73f37fe640 1 -- 172.21.15.105:0/2133693465 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f74000c5780 con 0x564b67fdd190 2023-11-28T18:26:14.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f741e346000 1 -- 172.21.15.105:0/2133693465 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b67ddd110 con 0x564b67fdd190 2023-11-28T18:26:14.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f73f37fe640 1 -- 172.21.15.105:0/2133693465 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74000c5980 con 0x564b67fdd190 2023-11-28T18:26:14.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f741e346000 10 client.6243 fetch_fsmap learned FSMap version 40 2023-11-28T18:26:14.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f741e346000 1 -- 172.21.15.105:0/2133693465 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x564b67ad6cc0 con 0x564b67fdd190 2023-11-28T18:26:14.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f73f37fe640 1 -- 172.21.15.105:0/2133693465 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564b67ddd110 con 0x564b67fdd190 2023-11-28T18:26:14.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f73f37fe640 1 -- 172.21.15.105:0/2133693465 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 40) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f74000c3040 con 0x564b67fdd190 2023-11-28T18:26:14.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f741e346000 10 client.6243 fetch_fsmap finished waiting for FSMap version 40 2023-11-28T18:26:14.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f741e346000 10 client.6243 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T18:26:14.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f741e346000 20 client.6243 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:14.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f741e346000 1 --2- 172.21.15.105:0/2133693465 >> [v2:172.21.15.148:6833/897997046,v1:172.21.15.148:6835/897997046] conn(0x564b68047ef0 0x564b6804a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f741e346000 4 client.6243 mds_command: new command op to 5414 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:26:14.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f741e346000 1 -- 172.21.15.105:0/2133693465 --> [v2:172.21.15.148:6833/897997046,v1:172.21.15.148:6835/897997046] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564b67e0b650 con 0x564b68047ef0 2023-11-28T18:26:14.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.156+0000 7f7418d6d640 1 --2- 172.21.15.105:0/2133693465 >> [v2:172.21.15.148:6833/897997046,v1:172.21.15.148:6835/897997046] conn(0x564b68047ef0 0x564b6804a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f7418d6d640 1 --2- 172.21.15.105:0/2133693465 >> [v2:172.21.15.148:6833/897997046,v1:172.21.15.148:6835/897997046] conn(0x564b68047ef0 0x564b6804a2e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b9ab6a8be3153c1c server_cookie=9a64eab7c54a6b1a in_seq=0 out_seq=0 2023-11-28T18:26:14.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f73f37fe640 10 client.6243 ms_handle_connect on v2:172.21.15.148:6833/897997046 2023-11-28T18:26:14.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f73f37fe640 1 -- 172.21.15.105:0/2133693465 <== mds.1 v2:172.21.15.148:6833/897997046 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x564b67e0b650 con 0x564b68047ef0 2023-11-28T18:26:14.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f73f37fe640 10 client.6243 handle_command_reply: tid=0 2023-11-28T18:26:14.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741e346000 1 client.6243 shutdown 2023-11-28T18:26:14.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741e346000 1 -- 172.21.15.105:0/2133693465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67fdd190 msgr2=0x564b67fed820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:14.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741e346000 1 --2- 172.21.15.105:0/2133693465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67fdd190 0x564b67fed820 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f740001c8a0 tx=0x7f74000046d0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741e346000 1 -- 172.21.15.105:0/2133693465 shutdown_connections 2023-11-28T18:26:14.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741e346000 1 --2- 172.21.15.105:0/2133693465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b67fddab0 0x564b67fedd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741e346000 1 --2- 172.21.15.105:0/2133693465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564b67fdd190 0x564b67fed820 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741e346000 1 --2- 172.21.15.105:0/2133693465 >> [v2:172.21.15.148:6833/897997046,v1:172.21.15.148:6835/897997046] conn(0x564b68047ef0 0x564b6804a2e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741e346000 1 --2- 172.21.15.105:0/2133693465 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b67f90910 0x564b67ff1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f73f37fe640 0 client.6243 ms_handle_reset on v2:172.21.15.148:6833/897997046 2023-11-28T18:26:14.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741e346000 1 -- 172.21.15.105:0/2133693465 >> 172.21.15.105:0/2133693465 conn(0x564b67fbdd20 msgr2=0x564b67fe9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:14.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741e346000 1 -- 172.21.15.105:0/2133693465 shutdown_connections 2023-11-28T18:26:14.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741e346000 1 -- 172.21.15.105:0/2133693465 wait complete. 2023-11-28T18:26:14.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741e346000 20 client.6243 trim_cache size 0 max 16384 2023-11-28T18:26:14.169 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:26:14.169 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:26:14.169 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 1, 2023-11-28T18:26:14.169 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5414, 2023-11-28T18:26:14.169 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:26:14.169 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:26:14.169 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:26:14.169 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 2.9629881600000001, 2023-11-28T18:26:14.169 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 40, 2023-11-28T18:26:14.170 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 51, 2023-11-28T18:26:14.170 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 51, 2023-11-28T18:26:14.170 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 17.863089863999999 2023-11-28T18:26:14.170 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:26:14.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741afd1640 1 -- 172.21.15.105:0/3827520148 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f73f80668d0 msgr2=0x7f73f8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:14.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741afd1640 1 --2- 172.21.15.105:0/3827520148 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f73f80668d0 0x7f73f8068d90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f74000ae000 tx=0x7f7400008040 comp rx=0 tx=0).stop 2023-11-28T18:26:14.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741afd1640 1 -- 172.21.15.105:0/3827520148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74140ad850 msgr2=0x7f7414153fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:14.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741afd1640 1 --2- 172.21.15.105:0/3827520148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74140ad850 0x7f7414153fc0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f740800c420 tx=0x7f740800c8e0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741afd1640 1 -- 172.21.15.105:0/3827520148 shutdown_connections 2023-11-28T18:26:14.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741afd1640 1 --2- 172.21.15.105:0/3827520148 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7414159f40 0x7f74140b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741afd1640 1 --2- 172.21.15.105:0/3827520148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74140caf60 0x7f74140b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741afd1640 1 --2- 172.21.15.105:0/3827520148 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f73f80668d0 0x7f73f8068d90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741afd1640 1 --2- 172.21.15.105:0/3827520148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74140ad850 0x7f7414153fc0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741afd1640 1 -- 172.21.15.105:0/3827520148 >> 172.21.15.105:0/3827520148 conn(0x7f74140ceb90 msgr2=0x7f74140ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:14.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741afd1640 1 -- 172.21.15.105:0/3827520148 shutdown_connections 2023-11-28T18:26:14.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.160+0000 7f741afd1640 1 -- 172.21.15.105:0/3827520148 wait complete. 2023-11-28T18:26:14.183 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-28T18:26:14.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.320+0000 7f0659fb8640 1 Processor -- start 2023-11-28T18:26:14.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.320+0000 7f0659fb8640 1 -- start start 2023-11-28T18:26:14.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.320+0000 7f0659fb8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f06540b3d30 0x7f06541680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.320+0000 7f0659fb8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06540b25b0 con 0x7f065415d4f0 2023-11-28T18:26:14.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.320+0000 7f0659fb8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06540d1c30 con 0x7f06540d3c30 2023-11-28T18:26:14.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.320+0000 7f0659fb8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06540b2900 con 0x7f06540b3d30 2023-11-28T18:26:14.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.320+0000 7f06537fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f065415d4f0 0x7f06540d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49218/0 (socket says 172.21.15.105:49218) 2023-11-28T18:26:14.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.320+0000 7f06537fe640 1 -- 172.21.15.105:0/2489823447 learned_addr learned my addr 172.21.15.105:0/2489823447 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:14.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.320+0000 7f0653fff640 1 --2- 172.21.15.105:0/2489823447 >> v2:172.21.15.105:3300/0 conn(0x7f06540b3d30 0x7f06541680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.324+0000 7f06527fc640 1 -- 172.21.15.105:0/2489823447 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1671442294 0 0) 0x7f06540b25b0 con 0x7f065415d4f0 2023-11-28T18:26:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.324+0000 7f06527fc640 1 -- 172.21.15.105:0/2489823447 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f063c003530 con 0x7f065415d4f0 2023-11-28T18:26:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.324+0000 7f0653fff640 1 -- 172.21.15.105:0/2489823447 >> v1:172.21.15.148:6789/0 conn(0x7f06540d3c30 legacy=0x7f06540d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.324+0000 7f0653fff640 1 -- 172.21.15.105:0/2489823447 >> v1:172.21.15.105:6789/0 conn(0x7f065415d4f0 legacy=0x7f06540d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.324+0000 7f0653fff640 1 -- 172.21.15.105:0/2489823447 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06540cea30 con 0x7f06540b3d30 2023-11-28T18:26:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.324+0000 7f0653fff640 1 --2- 172.21.15.105:0/2489823447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06540b3d30 0x7f06541680a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f064c009d90 tx=0x7f064c01c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9dba4d51ef3989 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.324+0000 7f06527fc640 1 -- 172.21.15.105:0/2489823447 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c0a5880 con 0x7f06540b3d30 2023-11-28T18:26:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.324+0000 7f06527fc640 1 -- 172.21.15.105:0/2489823447 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c0a59e0 con 0x7f06540b3d30 2023-11-28T18:26:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.324+0000 7f06527fc640 1 -- 172.21.15.105:0/2489823447 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c0a5d70 con 0x7f06540b3d30 2023-11-28T18:26:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f0659fb8640 1 -- 172.21.15.105:0/2489823447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06540b3d30 msgr2=0x7f06541680a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f0659fb8640 1 --2- 172.21.15.105:0/2489823447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06540b3d30 0x7f06541680a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f064c009d90 tx=0x7f064c01c7c0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f0659fb8640 1 -- 172.21.15.105:0/2489823447 shutdown_connections 2023-11-28T18:26:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f0659fb8640 1 --2- 172.21.15.105:0/2489823447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06540b3d30 0x7f06541680a0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f0659fb8640 1 -- 172.21.15.105:0/2489823447 >> 172.21.15.105:0/2489823447 conn(0x7f06540d3030 msgr2=0x7f06540d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:14.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f0659fb8640 1 -- 172.21.15.105:0/2489823447 shutdown_connections 2023-11-28T18:26:14.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f0659fb8640 1 -- 172.21.15.105:0/2489823447 wait complete. 2023-11-28T18:26:14.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f0659fb8640 1 Processor -- start 2023-11-28T18:26:14.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f0659fb8640 1 -- start start 2023-11-28T18:26:14.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f0659fb8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06540b3d30 0x7f06540b8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f0659fb8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06540d3c30 0x7f06540cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f0659fb8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f065415d4f0 0x7f065415a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f0659fb8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06540d0a20 con 0x7f06540b3d30 2023-11-28T18:26:14.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f0659fb8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0654159ca0 con 0x7f06540d3c30 2023-11-28T18:26:14.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f0659fb8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06540b71a0 con 0x7f065415d4f0 2023-11-28T18:26:14.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f06537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06540b3d30 0x7f06540b8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f06537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06540b3d30 0x7f06540b8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46930/0 (socket says 172.21.15.105:46930) 2023-11-28T18:26:14.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f06537fe640 1 -- 172.21.15.105:0/1315048708 learned_addr learned my addr 172.21.15.105:0/1315048708 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:14.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f0653fff640 1 --2- 172.21.15.105:0/1315048708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f065415d4f0 0x7f065415a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f0652ffd640 1 --2- 172.21.15.105:0/1315048708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06540d3c30 0x7f06540cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f06537fe640 1 -- 172.21.15.105:0/1315048708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f065415d4f0 msgr2=0x7f065415a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:14.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f06537fe640 1 --2- 172.21.15.105:0/1315048708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f065415d4f0 0x7f065415a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f06537fe640 1 -- 172.21.15.105:0/1315048708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06540d3c30 msgr2=0x7f06540cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:14.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f06537fe640 1 --2- 172.21.15.105:0/1315048708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06540d3c30 0x7f06540cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f06537fe640 1 -- 172.21.15.105:0/1315048708 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064c009a70 con 0x7f06540b3d30 2023-11-28T18:26:14.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.328+0000 7f0652ffd640 1 --2- 172.21.15.105:0/1315048708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06540d3c30 0x7f06540cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:14.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.332+0000 7f06537fe640 1 --2- 172.21.15.105:0/1315048708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06540b3d30 0x7f06540b8540 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f064400c420 tx=0x7f064400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:14.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.332+0000 7f063bfff640 1 -- 172.21.15.105:0/1315048708 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0644005cf0 con 0x7f06540b3d30 2023-11-28T18:26:14.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.332+0000 7f063bfff640 1 -- 172.21.15.105:0/1315048708 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0644005e50 con 0x7f06540b3d30 2023-11-28T18:26:14.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.332+0000 7f063bfff640 1 -- 172.21.15.105:0/1315048708 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06440072d0 con 0x7f06540b3d30 2023-11-28T18:26:14.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.332+0000 7f0659fb8640 1 -- 172.21.15.105:0/1315048708 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06540b7480 con 0x7f06540b3d30 2023-11-28T18:26:14.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.332+0000 7f0659fb8640 1 -- 172.21.15.105:0/1315048708 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0654159ef0 con 0x7f06540b3d30 2023-11-28T18:26:14.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.332+0000 7f063bfff640 1 -- 172.21.15.105:0/1315048708 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f0644007430 con 0x7f06540b3d30 2023-11-28T18:26:14.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.332+0000 7f063bfff640 1 --2- 172.21.15.105:0/1315048708 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f06400668d0 0x7f0640068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.332+0000 7f063bfff640 1 -- 172.21.15.105:0/1315048708 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f0644083a40 con 0x7f06540b3d30 2023-11-28T18:26:14.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.332+0000 7f0652ffd640 1 --2- 172.21.15.105:0/1315048708 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f06400668d0 0x7f0640068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f065d325000 1 Processor -- start 2023-11-28T18:26:14.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f065d325000 1 -- start start 2023-11-28T18:26:14.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f0652ffd640 1 --2- 172.21.15.105:0/1315048708 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f06400668d0 0x7f0640068d90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f06540cf540 tx=0x7f0648008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:14.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f065d325000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b3a16b7d30 0x55b3a16b8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f065d325000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a16b8650 0x55b3a166afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f065d325000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b3a166b500 0x55b3a166b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f065d325000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a1534110 con 0x55b3a16b8650 2023-11-28T18:26:14.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f065d325000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a14c49a0 con 0x55b3a166b500 2023-11-28T18:26:14.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f065d325000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3a14c1ed0 con 0x55b3a16b7d30 2023-11-28T18:26:14.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f0652ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a16b8650 0x55b3a166afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f0652ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a16b8650 0x55b3a166afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46946/0 (socket says 172.21.15.105:46946) 2023-11-28T18:26:14.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f0652ffd640 1 -- 172.21.15.105:0/4042866855 learned_addr learned my addr 172.21.15.105:0/4042866855 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:14.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f06537fe640 1 --2- 172.21.15.105:0/4042866855 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b3a16b7d30 0x55b3a16b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f0653fff640 1 --2- 172.21.15.105:0/4042866855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b3a166b500 0x55b3a166b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f0652ffd640 1 -- 172.21.15.105:0/4042866855 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b3a16b7d30 msgr2=0x55b3a16b8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:14.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f0652ffd640 1 --2- 172.21.15.105:0/4042866855 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b3a16b7d30 0x55b3a16b8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f0652ffd640 1 -- 172.21.15.105:0/4042866855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b3a166b500 msgr2=0x55b3a166b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:14.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f0652ffd640 1 --2- 172.21.15.105:0/4042866855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b3a166b500 0x55b3a166b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f0652ffd640 1 -- 172.21.15.105:0/4042866855 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3a11b5890 con 0x55b3a16b8650 2023-11-28T18:26:14.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f0653fff640 1 --2- 172.21.15.105:0/4042866855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b3a166b500 0x55b3a166b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:14.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f0652ffd640 1 --2- 172.21.15.105:0/4042866855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a16b8650 0x55b3a166afc0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f064800d830 tx=0x7f0648010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7864e303ed566ee5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:14.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f0639ffb640 1 -- 172.21.15.105:0/4042866855 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06480094a0 con 0x55b3a16b8650 2023-11-28T18:26:14.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f0639ffb640 1 -- 172.21.15.105:0/4042866855 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06480148b0 con 0x55b3a16b8650 2023-11-28T18:26:14.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f0639ffb640 1 -- 172.21.15.105:0/4042866855 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0648014c40 con 0x55b3a16b8650 2023-11-28T18:26:14.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f065d325000 1 -- 172.21.15.105:0/4042866855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a16b8650 msgr2=0x55b3a166afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:14.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f065d325000 1 --2- 172.21.15.105:0/4042866855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a16b8650 0x55b3a166afc0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f064800d830 tx=0x7f0648010000 comp rx=0 tx=0).stop 2023-11-28T18:26:14.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f065d325000 1 -- 172.21.15.105:0/4042866855 shutdown_connections 2023-11-28T18:26:14.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f065d325000 1 --2- 172.21.15.105:0/4042866855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b3a166b500 0x55b3a166b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f065d325000 1 --2- 172.21.15.105:0/4042866855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a16b8650 0x55b3a166afc0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f065d325000 1 --2- 172.21.15.105:0/4042866855 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b3a16b7d30 0x55b3a16b8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f065d325000 1 -- 172.21.15.105:0/4042866855 >> 172.21.15.105:0/4042866855 conn(0x55b3a16988c0 msgr2=0x55b3a169acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:14.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f065d325000 1 -- 172.21.15.105:0/4042866855 shutdown_connections 2023-11-28T18:26:14.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f065d325000 1 -- 172.21.15.105:0/4042866855 wait complete. 2023-11-28T18:26:14.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f065d325000 1 -- start start 2023-11-28T18:26:14.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.336+0000 7f065d325000 1 Processor -- start 2023-11-28T18:26:14.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f065d325000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a166b500 0x55b3a16714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f065d325000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b3a16b7d30 0x55b3a1671a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f065d325000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b3a16b8650 0x55b3a1671f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f065d325000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a14bd7c0 con 0x55b3a166b500 2023-11-28T18:26:14.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f065d325000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a1475bf0 con 0x55b3a16b8650 2023-11-28T18:26:14.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f065d325000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3a147ba90 con 0x55b3a16b7d30 2023-11-28T18:26:14.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f065d325000 10 client.0 _ll_register_callbacks cb 0x55b3a1658e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:14.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f06537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a166b500 0x55b3a16714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f06537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a166b500 0x55b3a16714e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46956/0 (socket says 172.21.15.105:46956) 2023-11-28T18:26:14.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f06537fe640 1 -- 172.21.15.105:0/1066508705 learned_addr learned my addr 172.21.15.105:0/1066508705 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:14.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f0652ffd640 1 --2- 172.21.15.105:0/1066508705 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b3a16b7d30 0x55b3a1671a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f0653fff640 1 --2- 172.21.15.105:0/1066508705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b3a16b8650 0x55b3a1671f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f06537fe640 1 -- 172.21.15.105:0/1066508705 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b3a16b7d30 msgr2=0x55b3a1671a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:14.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f06537fe640 1 --2- 172.21.15.105:0/1066508705 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b3a16b7d30 0x55b3a1671a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f06537fe640 1 -- 172.21.15.105:0/1066508705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b3a16b8650 msgr2=0x55b3a1671f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:14.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f06537fe640 1 --2- 172.21.15.105:0/1066508705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b3a16b8650 0x55b3a1671f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f06537fe640 1 -- 172.21.15.105:0/1066508705 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f064800e040 con 0x55b3a166b500 2023-11-28T18:26:14.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f0653fff640 1 --2- 172.21.15.105:0/1066508705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b3a16b8650 0x55b3a1671f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:14.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f06537fe640 1 --2- 172.21.15.105:0/1066508705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a166b500 0x55b3a16714e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0644081e80 tx=0x7f0644058ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33c9c9733abcdd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:14.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f06327fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:14.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f06327fc640 1 -- 172.21.15.105:0/1066508705 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0644052d60 con 0x55b3a166b500 2023-11-28T18:26:14.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f06327fc640 1 -- 172.21.15.105:0/1066508705 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f0644055b80 con 0x55b3a166b500 2023-11-28T18:26:14.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f065d325000 1 -- 172.21.15.105:0/1066508705 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3a147f470 con 0x55b3a166b500 2023-11-28T18:26:14.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f06327fc640 1 -- 172.21.15.105:0/1066508705 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0644055d40 con 0x55b3a166b500 2023-11-28T18:26:14.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f065d325000 10 client.6258 fetch_fsmap learned FSMap version 40 2023-11-28T18:26:14.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f065d325000 1 -- 172.21.15.105:0/1066508705 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x55b3a147e190 con 0x55b3a166b500 2023-11-28T18:26:14.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f06327fc640 1 -- 172.21.15.105:0/1066508705 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0644071080 con 0x55b3a166b500 2023-11-28T18:26:14.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f06327fc640 1 -- 172.21.15.105:0/1066508705 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 40) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f064407bb60 con 0x55b3a166b500 2023-11-28T18:26:14.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f065d325000 10 client.6258 fetch_fsmap finished waiting for FSMap version 40 2023-11-28T18:26:14.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f065d325000 10 client.6258 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T18:26:14.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f065d325000 20 client.6258 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:14.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f065d325000 1 --2- 172.21.15.105:0/1066508705 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55b3a1672970 0x55b3a16c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f065d325000 4 client.6258 mds_command: new command op to 5842 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:26:14.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f065d325000 1 -- 172.21.15.105:0/1066508705 --> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3a1482b50 con 0x55b3a1672970 2023-11-28T18:26:14.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f0652ffd640 1 --2- 172.21.15.105:0/1066508705 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55b3a1672970 0x55b3a16c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f0652ffd640 1 --2- 172.21.15.105:0/1066508705 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55b3a1672970 0x55b3a16c9ff0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daa86a39630d91e6 server_cookie=f87d5f7dfc1ee572 in_seq=0 out_seq=0 2023-11-28T18:26:14.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.340+0000 7f06327fc640 10 client.6258 ms_handle_connect on v2:172.21.15.148:6832/2696773413 2023-11-28T18:26:14.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.344+0000 7f06327fc640 1 -- 172.21.15.105:0/1066508705 <== mds.0 v2:172.21.15.148:6832/2696773413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b3a1482b50 con 0x55b3a1672970 2023-11-28T18:26:14.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.344+0000 7f06327fc640 10 client.6258 handle_command_reply: tid=0 2023-11-28T18:26:14.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.344+0000 7f065d325000 1 client.6258 shutdown 2023-11-28T18:26:14.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.344+0000 7f065d325000 1 -- 172.21.15.105:0/1066508705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a166b500 msgr2=0x55b3a16714e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:14.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.344+0000 7f065d325000 1 --2- 172.21.15.105:0/1066508705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a166b500 0x55b3a16714e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0644081e80 tx=0x7f0644058ae0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.344+0000 7f065d325000 1 -- 172.21.15.105:0/1066508705 shutdown_connections 2023-11-28T18:26:14.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.344+0000 7f065d325000 1 --2- 172.21.15.105:0/1066508705 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55b3a1672970 0x55b3a16c9ff0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.344+0000 7f065d325000 1 --2- 172.21.15.105:0/1066508705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b3a16b8650 0x55b3a1671f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.344+0000 7f065d325000 1 --2- 172.21.15.105:0/1066508705 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b3a16b7d30 0x55b3a1671a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.344+0000 7f065d325000 1 --2- 172.21.15.105:0/1066508705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a166b500 0x55b3a16714e0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.344+0000 7f065d325000 1 -- 172.21.15.105:0/1066508705 >> 172.21.15.105:0/1066508705 conn(0x55b3a16988c0 msgr2=0x55b3a166a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:14.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.344+0000 7f06327fc640 0 client.6258 ms_handle_reset on v2:172.21.15.148:6832/2696773413 2023-11-28T18:26:14.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.344+0000 7f065d325000 1 -- 172.21.15.105:0/1066508705 shutdown_connections 2023-11-28T18:26:14.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.344+0000 7f065d325000 1 -- 172.21.15.105:0/1066508705 wait complete. 2023-11-28T18:26:14.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.344+0000 7f065d325000 20 client.6258 trim_cache size 0 max 16384 2023-11-28T18:26:14.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.356+0000 7f065d325000 1 Processor -- start 2023-11-28T18:26:14.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f065d325000 1 -- start start 2023-11-28T18:26:14.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f065d325000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b3a166b500 0x55b3a16973a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f065d325000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b3a16b7d30 0x55b3a16978e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f065d325000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a16b8650 0x55b3a16c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f065d325000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a1176940 con 0x55b3a16b8650 2023-11-28T18:26:14.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f065d325000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a1665430 con 0x55b3a166b500 2023-11-28T18:26:14.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f065d325000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3a147c8f0 con 0x55b3a16b7d30 2023-11-28T18:26:14.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f0653fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a16b8650 0x55b3a16c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f0653fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a16b8650 0x55b3a16c9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46972/0 (socket says 172.21.15.105:46972) 2023-11-28T18:26:14.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f0653fff640 1 -- 172.21.15.105:0/3671143666 learned_addr learned my addr 172.21.15.105:0/3671143666 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:14.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f0652ffd640 1 --2- 172.21.15.105:0/3671143666 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b3a16b7d30 0x55b3a16978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f06537fe640 1 --2- 172.21.15.105:0/3671143666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b3a166b500 0x55b3a16973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f0653fff640 1 -- 172.21.15.105:0/3671143666 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b3a16b7d30 msgr2=0x55b3a16978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:14.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f0653fff640 1 --2- 172.21.15.105:0/3671143666 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b3a16b7d30 0x55b3a16978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f0653fff640 1 -- 172.21.15.105:0/3671143666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b3a166b500 msgr2=0x55b3a16973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:14.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f0653fff640 1 --2- 172.21.15.105:0/3671143666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b3a166b500 0x55b3a16973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f0653fff640 1 -- 172.21.15.105:0/3671143666 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3a1464490 con 0x55b3a16b8650 2023-11-28T18:26:14.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f0653fff640 1 --2- 172.21.15.105:0/3671143666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a16b8650 0x55b3a16c9ff0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f064c01a3f0 tx=0x7f064c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a286e145004620d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:14.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f063a7fc640 1 -- 172.21.15.105:0/3671143666 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c0bd070 con 0x55b3a16b8650 2023-11-28T18:26:14.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f063a7fc640 1 -- 172.21.15.105:0/3671143666 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c0a5860 con 0x55b3a16b8650 2023-11-28T18:26:14.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f063a7fc640 1 -- 172.21.15.105:0/3671143666 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c0a5d70 con 0x55b3a16b8650 2023-11-28T18:26:14.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f065d325000 1 -- 172.21.15.105:0/3671143666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a16b8650 msgr2=0x55b3a16c9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:14.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f065d325000 1 --2- 172.21.15.105:0/3671143666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a16b8650 0x55b3a16c9ff0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f064c01a3f0 tx=0x7f064c004ac0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f065d325000 1 -- 172.21.15.105:0/3671143666 shutdown_connections 2023-11-28T18:26:14.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f065d325000 1 --2- 172.21.15.105:0/3671143666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a16b8650 0x55b3a16c9ff0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f065d325000 1 --2- 172.21.15.105:0/3671143666 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b3a16b7d30 0x55b3a16978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f065d325000 1 --2- 172.21.15.105:0/3671143666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b3a166b500 0x55b3a16973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f065d325000 1 -- 172.21.15.105:0/3671143666 >> 172.21.15.105:0/3671143666 conn(0x55b3a16988c0 msgr2=0x55b3a1669720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:14.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f065d325000 1 -- 172.21.15.105:0/3671143666 shutdown_connections 2023-11-28T18:26:14.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f065d325000 1 -- 172.21.15.105:0/3671143666 wait complete. 2023-11-28T18:26:14.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f065d325000 1 -- start start 2023-11-28T18:26:14.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.360+0000 7f065d325000 1 Processor -- start 2023-11-28T18:26:14.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a166b500 0x55b3a1718810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b3a16b7d30 0x55b3a1718d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b3a16b8650 0x55b3a16d3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a14bc4e0 con 0x55b3a166b500 2023-11-28T18:26:14.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a14bcfd0 con 0x55b3a16b7d30 2023-11-28T18:26:14.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3a147a9b0 con 0x55b3a16b8650 2023-11-28T18:26:14.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 10 client.0 _ll_register_callbacks cb 0x55b3a1534060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:14.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f06537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a166b500 0x55b3a1718810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f06537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a166b500 0x55b3a1718810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46982/0 (socket says 172.21.15.105:46982) 2023-11-28T18:26:14.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f06537fe640 1 -- 172.21.15.105:0/655775984 learned_addr learned my addr 172.21.15.105:0/655775984 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:14.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f0652ffd640 1 --2- 172.21.15.105:0/655775984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b3a16b7d30 0x55b3a1718d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f0653fff640 1 --2- 172.21.15.105:0/655775984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b3a16b8650 0x55b3a16d3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f06537fe640 1 -- 172.21.15.105:0/655775984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b3a16b8650 msgr2=0x55b3a16d3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:14.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f06537fe640 1 --2- 172.21.15.105:0/655775984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b3a16b8650 0x55b3a16d3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f06537fe640 1 -- 172.21.15.105:0/655775984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b3a16b7d30 msgr2=0x55b3a1718d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:14.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f06537fe640 1 --2- 172.21.15.105:0/655775984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b3a16b7d30 0x55b3a1718d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f06537fe640 1 -- 172.21.15.105:0/655775984 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06540cea30 con 0x55b3a166b500 2023-11-28T18:26:14.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f0653fff640 1 --2- 172.21.15.105:0/655775984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b3a16b8650 0x55b3a16d3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:14.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f06537fe640 1 --2- 172.21.15.105:0/655775984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a166b500 0x55b3a1718810 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f064400cde0 tx=0x7f0644081d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=274389d0690c3358 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:14.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f06327fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:14.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f06327fc640 1 -- 172.21.15.105:0/655775984 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06440508f0 con 0x55b3a166b500 2023-11-28T18:26:14.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f06327fc640 1 -- 172.21.15.105:0/655775984 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f0644052d60 con 0x55b3a166b500 2023-11-28T18:26:14.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 1 -- 172.21.15.105:0/655775984 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3a14ea650 con 0x55b3a166b500 2023-11-28T18:26:14.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f06327fc640 1 -- 172.21.15.105:0/655775984 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f064400e040 con 0x55b3a166b500 2023-11-28T18:26:14.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 10 client.6264 fetch_fsmap learned FSMap version 40 2023-11-28T18:26:14.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 1 -- 172.21.15.105:0/655775984 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x55b3a11b5cc0 con 0x55b3a166b500 2023-11-28T18:26:14.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f06327fc640 1 -- 172.21.15.105:0/655775984 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064407a070 con 0x55b3a166b500 2023-11-28T18:26:14.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f06327fc640 1 -- 172.21.15.105:0/655775984 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 40) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f0644053860 con 0x55b3a166b500 2023-11-28T18:26:14.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 10 client.6264 fetch_fsmap finished waiting for FSMap version 40 2023-11-28T18:26:14.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 10 client.6264 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T18:26:14.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 20 client.6264 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:14.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 1 --2- 172.21.15.105:0/655775984 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55b3a16d45e0 0x55b3a16d49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:14.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 4 client.6264 mds_command: new command op to 5842 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:26:14.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 1 -- 172.21.15.105:0/655775984 --> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b3a14b67a0 con 0x55b3a16d45e0 2023-11-28T18:26:14.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f0652ffd640 1 --2- 172.21.15.105:0/655775984 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55b3a16d45e0 0x55b3a16d49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f06327fc640 10 client.6264 ms_handle_connect on v2:172.21.15.148:6832/2696773413 2023-11-28T18:26:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f0652ffd640 1 --2- 172.21.15.105:0/655775984 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55b3a16d45e0 0x55b3a16d49c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4163c6b175854bd5 server_cookie=31356ae68f258fe7 in_seq=0 out_seq=0 2023-11-28T18:26:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f06327fc640 1 -- 172.21.15.105:0/655775984 <== mds.0 v2:172.21.15.148:6832/2696773413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55b3a14b67a0 con 0x55b3a16d45e0 2023-11-28T18:26:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f06327fc640 10 client.6264 handle_command_reply: tid=0 2023-11-28T18:26:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 1 client.6264 shutdown 2023-11-28T18:26:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 1 -- 172.21.15.105:0/655775984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a166b500 msgr2=0x55b3a1718810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 1 --2- 172.21.15.105:0/655775984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a166b500 0x55b3a1718810 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f064400cde0 tx=0x7f0644081d50 comp rx=0 tx=0).stop 2023-11-28T18:26:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 1 -- 172.21.15.105:0/655775984 shutdown_connections 2023-11-28T18:26:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 1 --2- 172.21.15.105:0/655775984 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55b3a16d45e0 0x55b3a16d49c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 1 --2- 172.21.15.105:0/655775984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b3a16b8650 0x55b3a16d3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 1 --2- 172.21.15.105:0/655775984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b3a16b7d30 0x55b3a1718d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f06327fc640 0 client.6264 ms_handle_reset on v2:172.21.15.148:6832/2696773413 2023-11-28T18:26:14.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 1 --2- 172.21.15.105:0/655775984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3a166b500 0x55b3a1718810 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 1 -- 172.21.15.105:0/655775984 >> 172.21.15.105:0/655775984 conn(0x55b3a16988c0 msgr2=0x55b3a1669fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:14.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 1 -- 172.21.15.105:0/655775984 shutdown_connections 2023-11-28T18:26:14.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.364+0000 7f065d325000 1 -- 172.21.15.105:0/655775984 wait complete. 2023-11-28T18:26:14.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.368+0000 7f065d325000 20 client.6264 trim_cache size 0 max 16384 2023-11-28T18:26:14.374 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:26:14.374 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:26:14.374 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2023-11-28T18:26:14.375 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5842, 2023-11-28T18:26:14.375 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:26:14.375 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:26:14.375 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:26:14.375 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 13.203730974000001, 2023-11-28T18:26:14.375 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 40, 2023-11-28T18:26:14.375 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 52, 2023-11-28T18:26:14.375 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 51, 2023-11-28T18:26:14.375 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 18.070976234 2023-11-28T18:26:14.376 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:26:14.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.368+0000 7f0659fb8640 1 -- 172.21.15.105:0/1315048708 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f06400668d0 msgr2=0x7f0640068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:14.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.368+0000 7f0659fb8640 1 --2- 172.21.15.105:0/1315048708 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f06400668d0 0x7f0640068d90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f06540cf540 tx=0x7f0648008040 comp rx=0 tx=0).stop 2023-11-28T18:26:14.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.368+0000 7f0659fb8640 1 -- 172.21.15.105:0/1315048708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06540b3d30 msgr2=0x7f06540b8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:14.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.368+0000 7f0659fb8640 1 --2- 172.21.15.105:0/1315048708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06540b3d30 0x7f06540b8540 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f064400c420 tx=0x7f064400c8e0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.368+0000 7f0659fb8640 1 -- 172.21.15.105:0/1315048708 shutdown_connections 2023-11-28T18:26:14.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.368+0000 7f0659fb8640 1 --2- 172.21.15.105:0/1315048708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f065415d4f0 0x7f065415a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.368+0000 7f0659fb8640 1 --2- 172.21.15.105:0/1315048708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06540d3c30 0x7f06540cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.368+0000 7f0659fb8640 1 --2- 172.21.15.105:0/1315048708 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f06400668d0 0x7f0640068d90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.368+0000 7f0659fb8640 1 --2- 172.21.15.105:0/1315048708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06540b3d30 0x7f06540b8540 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:14.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.368+0000 7f0659fb8640 1 -- 172.21.15.105:0/1315048708 >> 172.21.15.105:0/1315048708 conn(0x7f06540d3030 msgr2=0x7f06540d1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:14.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.368+0000 7f0659fb8640 1 -- 172.21.15.105:0/1315048708 shutdown_connections 2023-11-28T18:26:14.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:14.368+0000 7f0659fb8640 1 -- 172.21.15.105:0/1315048708 wait complete. 2023-11-28T18:26:14.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:26:14.388 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-28T18:26:14.388 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:26:14.388 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p tree 2023-11-28T18:26:14.388 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:26:14.388 DEBUG:teuthology.orchestra.run.smithi105:> setfattr -n ceph.dir.pin.distributed -v 1 tree 2023-11-28T18:26:14.388 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:26:14.388 DEBUG:teuthology.orchestra.run.smithi105:> for ((i = 0; i < 100; i++)); do 2023-11-28T18:26:14.389 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p "tree/$i" 2023-11-28T18:26:14.389 DEBUG:teuthology.orchestra.run.smithi105:> echo file > "tree/$i/file" 2023-11-28T18:26:14.389 DEBUG:teuthology.orchestra.run.smithi105:> done 2023-11-28T18:26:14.389 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:26:14.401 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree 2023-11-28T18:26:14.404 INFO:teuthology.orchestra.run.smithi105.stderr:+ setfattr -n ceph.dir.pin.distributed -v 1 tree 2023-11-28T18:26:14.409 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i = 0 )) 2023-11-28T18:26:14.409 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.409 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/0 2023-11-28T18:26:14.415 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.422 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.422 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.422 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/1 2023-11-28T18:26:14.439 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.442 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.442 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.443 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/2 2023-11-28T18:26:14.449 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.451 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.451 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.452 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/3 2023-11-28T18:26:14.458 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.461 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.461 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.461 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/4 2023-11-28T18:26:14.467 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.470 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.470 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.470 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/5 2023-11-28T18:26:14.476 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.479 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.479 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.479 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/6 2023-11-28T18:26:14.485 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.488 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.488 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.488 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/7 2023-11-28T18:26:14.495 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.497 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.497 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.498 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/8 2023-11-28T18:26:14.504 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.507 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.507 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.507 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/9 2023-11-28T18:26:14.513 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.516 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.516 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.516 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/10 2023-11-28T18:26:14.522 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.525 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.525 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.525 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/11 2023-11-28T18:26:14.531 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.534 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.534 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.534 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/12 2023-11-28T18:26:14.540 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.543 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.543 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.543 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/13 2023-11-28T18:26:14.549 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.552 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.552 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.552 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/14 2023-11-28T18:26:14.558 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.561 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.561 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.561 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/15 2023-11-28T18:26:14.567 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.570 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.570 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.570 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/16 2023-11-28T18:26:14.576 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.579 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.579 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.579 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/17 2023-11-28T18:26:14.586 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.589 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.589 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.589 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/18 2023-11-28T18:26:14.594 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.597 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.597 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.597 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/19 2023-11-28T18:26:14.604 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.606 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.606 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.606 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/20 2023-11-28T18:26:14.612 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.615 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.615 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.615 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/21 2023-11-28T18:26:14.621 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.624 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.624 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.624 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/22 2023-11-28T18:26:14.631 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.634 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.634 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.635 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/23 2023-11-28T18:26:14.640 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.642 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.642 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.643 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/24 2023-11-28T18:26:14.648 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.651 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.652 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.652 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/25 2023-11-28T18:26:14.658 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.660 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.660 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.661 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/26 2023-11-28T18:26:14.667 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.670 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.670 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.670 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/27 2023-11-28T18:26:14.676 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.679 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.679 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.679 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/28 2023-11-28T18:26:14.686 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.689 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.689 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.690 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/29 2023-11-28T18:26:14.695 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.697 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.697 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.697 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/30 2023-11-28T18:26:14.705 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.707 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.708 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.708 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/31 2023-11-28T18:26:14.714 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.716 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.716 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.716 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/32 2023-11-28T18:26:14.722 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.725 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.725 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.725 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/33 2023-11-28T18:26:14.731 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.733 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.734 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.734 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/34 2023-11-28T18:26:14.740 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.742 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.743 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.743 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/35 2023-11-28T18:26:14.749 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.751 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.751 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.752 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/36 2023-11-28T18:26:14.757 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.760 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.760 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.760 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/37 2023-11-28T18:26:14.766 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.769 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.769 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.769 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/38 2023-11-28T18:26:14.776 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.778 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.778 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.778 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/39 2023-11-28T18:26:14.785 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.787 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.787 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.787 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/40 2023-11-28T18:26:14.793 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.796 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.796 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.796 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/41 2023-11-28T18:26:14.802 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.804 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.804 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.804 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/42 2023-11-28T18:26:14.811 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.813 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.813 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.813 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/43 2023-11-28T18:26:14.820 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.822 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.823 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.823 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/44 2023-11-28T18:26:14.829 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.831 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.831 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.831 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/45 2023-11-28T18:26:14.838 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.841 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.841 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.841 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/46 2023-11-28T18:26:14.847 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.849 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.849 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.849 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/47 2023-11-28T18:26:14.856 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.858 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.858 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.858 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/48 2023-11-28T18:26:14.864 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.867 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.868 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.868 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/49 2023-11-28T18:26:14.874 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.877 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/50 2023-11-28T18:26:14.883 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.885 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.885 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.885 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/51 2023-11-28T18:26:14.891 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.893 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.893 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.893 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/52 2023-11-28T18:26:14.899 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.902 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.902 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.902 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/53 2023-11-28T18:26:14.908 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.910 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.911 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.911 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/54 2023-11-28T18:26:14.917 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.919 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.919 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.920 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/55 2023-11-28T18:26:14.926 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.929 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.929 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.929 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/56 2023-11-28T18:26:14.935 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.937 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.937 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.937 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/57 2023-11-28T18:26:14.943 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.945 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.945 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.946 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/58 2023-11-28T18:26:14.952 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.954 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.954 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.955 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/59 2023-11-28T18:26:14.960 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.963 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.963 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.963 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/60 2023-11-28T18:26:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.972 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.972 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.973 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/61 2023-11-28T18:26:14.979 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.981 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.982 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.982 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/62 2023-11-28T18:26:14.988 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.990 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.990 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.991 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/63 2023-11-28T18:26:14.996 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:14.998 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:14.999 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:14.999 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/64 2023-11-28T18:26:15.004 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.007 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.007 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.007 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/65 2023-11-28T18:26:15.013 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.015 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.015 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.015 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/66 2023-11-28T18:26:15.021 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.023 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.024 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.024 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/67 2023-11-28T18:26:15.030 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.033 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.033 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.033 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/68 2023-11-28T18:26:15.038 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.041 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.041 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.041 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/69 2023-11-28T18:26:15.048 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.050 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.050 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.050 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/70 2023-11-28T18:26:15.056 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.059 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.059 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.059 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/71 2023-11-28T18:26:15.065 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.067 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.067 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.068 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/72 2023-11-28T18:26:15.075 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.077 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.077 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.077 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/73 2023-11-28T18:26:15.083 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.086 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.086 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.086 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/74 2023-11-28T18:26:15.094 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.096 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.096 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.097 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/75 2023-11-28T18:26:15.103 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.105 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.106 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.106 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/76 2023-11-28T18:26:15.113 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.115 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.115 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.116 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/77 2023-11-28T18:26:15.122 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.125 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.125 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.125 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/78 2023-11-28T18:26:15.132 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.134 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.135 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.135 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/79 2023-11-28T18:26:15.141 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.144 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.144 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.144 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/80 2023-11-28T18:26:15.151 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.154 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.154 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.154 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/81 2023-11-28T18:26:15.161 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.164 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.164 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.164 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/82 2023-11-28T18:26:15.171 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.174 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.174 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.174 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/83 2023-11-28T18:26:15.180 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.183 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.183 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.183 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/84 2023-11-28T18:26:15.190 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.193 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.193 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.193 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/85 2023-11-28T18:26:15.199 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.203 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.203 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.203 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/86 2023-11-28T18:26:15.212 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.215 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.215 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.215 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/87 2023-11-28T18:26:15.221 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.223 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.223 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.223 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/88 2023-11-28T18:26:15.230 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.233 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.233 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.233 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/89 2023-11-28T18:26:15.241 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.245 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.245 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.245 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/90 2023-11-28T18:26:15.254 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.256 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.256 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.256 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/91 2023-11-28T18:26:15.264 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.267 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.267 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.267 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/92 2023-11-28T18:26:15.275 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.277 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.277 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.277 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/93 2023-11-28T18:26:15.283 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.285 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.285 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.285 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/94 2023-11-28T18:26:15.291 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.294 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.294 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.294 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/95 2023-11-28T18:26:15.301 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.303 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.304 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.304 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/96 2023-11-28T18:26:15.310 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.312 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.312 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.312 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/97 2023-11-28T18:26:15.319 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.321 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.321 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.321 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/98 2023-11-28T18:26:15.327 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.330 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.330 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.330 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/99 2023-11-28T18:26:15.336 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:26:15.338 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:26:15.339 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:26:15.340 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-11-28T18:26:15.535 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-28T18:26:15.535 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:26:15.536 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2023-11-28T18:26:15.692 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-28T18:26:15.692 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:26:15.693 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-11-28T18:26:15.850 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"666","version":"664","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"155","version":"155","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2023-11-28T18:26:15.850 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "666", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "664" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "155", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "155" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:26:15.850 INFO:tasks.cephfs.cephfs_test_case:len=0 [] 2023-11-28T18:26:20.852 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-11-28T18:26:21.018 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-28T18:26:21.018 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:26:21.019 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2023-11-28T18:26:21.183 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-28T18:26:21.183 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:26:21.184 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-11-28T18:26:21.347 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"675","version":"675","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"157","version":"157","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000241.101*","snapid_first":2,"projected_version":"422","version":"422","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000241.110*","snapid_first":2,"projected_version":"442","version":"442","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000241.000*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2023-11-28T18:26:21.348 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "675", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "675" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "157", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "157" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000241.101*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "422", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "422" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000241.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "442", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "442" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000241.000*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "426", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:26:21.348 INFO:tasks.cephfs.cephfs_test_case:len=3 [{'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x10000000241.101*', 'snapid_first': 2, 'projected_version': '422', 'version': '422', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x10000000241.110*', 'snapid_first': 2, 'projected_version': '442', 'version': '442', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x10000000241.000*', 'snapid_first': 2, 'projected_version': '426', 'version': '426', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}] 2023-11-28T18:26:26.349 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-11-28T18:26:26.514 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"675","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":2},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000241.011*","snapid_first":2,"projected_version":"430","version":"430","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000241.010*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-11-28T18:26:26.515 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "snapid_first": 2, "states": [], "version": "675" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x10000000241.011*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "430", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "430" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x10000000241.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "426", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:26:26.515 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2023-11-28T18:26:26.681 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"675","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":3},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000241.111*","snapid_first":2,"projected_version":"416","version":"416","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000241.100*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000241.001*","snapid_first":2,"projected_version":"428","version":"428","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-11-28T18:26:26.682 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "snapid_first": 2, "states": [], "version": "675" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000241.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "416", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "416" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000241.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "426", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000241.001*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "428", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "428" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:26:26.682 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-11-28T18:26:26.838 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"675","version":"675","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":3,"2":2}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"157","version":"157","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000241.101*","snapid_first":2,"projected_version":"422","version":"422","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000241.100*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000241.011*","snapid_first":2,"projected_version":"430","version":"430","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000241.110*","snapid_first":2,"projected_version":"442","version":"442","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000241.000*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000241.001*","snapid_first":2,"projected_version":"428","version":"428","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000241.111*","snapid_first":2,"projected_version":"416","version":"416","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000241.010*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2023-11-28T18:26:26.840 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 3, "2": 2 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "675", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "675" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "157", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "157" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000241.101*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "422", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "422" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000241.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x10000000241.011*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "430", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "430" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000241.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "442", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "442" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000241.000*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "426", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000241.001*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "428", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "428" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000241.111*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "416", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "416" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x10000000241.010*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2023-11-28T18:26:26.840 INFO:tasks.cephfs.cephfs_test_case:len=8 [{'is_auth': True, 'auth_first': 2, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 2, 'dir': {'path': '/tree', 'dirfrag': '0x10000000241.011*', 'snapid_first': 2, 'projected_version': '430', 'version': '430', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '2', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {'0': 1}}, 'replica_state': {'authority': [2, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'ptrwaiter': 0, 'child': 1, 'frozen': 0, 'subtree': 1, 'importing': 0, 'replicated': 1, 'dirty': 1}, 'nref': 4}}, {'is_auth': True, 'auth_first': 2, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 2, 'dir': {'path': '/tree', 'dirfrag': '0x10000000241.010*', 'snapid_first': 2, 'projected_version': '426', 'version': '426', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '2', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {'0': 1}}, 'replica_state': {'authority': [2, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'ptrwaiter': 0, 'child': 1, 'frozen': 0, 'subtree': 1, 'importing': 0, 'replicated': 1, 'dirty': 1}, 'nref': 4}}, {'is_auth': True, 'auth_first': 1, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 1, 'dir': {'path': '/tree', 'dirfrag': '0x10000000241.111*', 'snapid_first': 2, 'projected_version': '416', 'version': '416', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '1', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {'0': 1}}, 'replica_state': {'authority': [1, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'ptrwaiter': 0, 'child': 1, 'frozen': 0, 'subtree': 1, 'importing': 0, 'replicated': 1, 'dirty': 1}, 'nref': 4}}, {'is_auth': True, 'auth_first': 1, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 1, 'dir': {'path': '/tree', 'dirfrag': '0x10000000241.100*', 'snapid_first': 2, 'projected_version': '426', 'version': '426', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '1', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {'0': 1}}, 'replica_state': {'authority': [1, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'ptrwaiter': 0, 'child': 1, 'frozen': 0, 'subtree': 1, 'importing': 0, 'replicated': 1, 'dirty': 1}, 'nref': 4}}, {'is_auth': True, 'auth_first': 1, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 1, 'dir': {'path': '/tree', 'dirfrag': '0x10000000241.001*', 'snapid_first': 2, 'projected_version': '428', 'version': '428', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '1', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {'0': 1}}, 'replica_state': {'authority': [1, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'ptrwaiter': 0, 'child': 1, 'frozen': 0, 'subtree': 1, 'importing': 0, 'replicated': 1, 'dirty': 1}, 'nref': 4}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x10000000241.101*', 'snapid_first': 2, 'projected_version': '422', 'version': '422', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x10000000241.110*', 'snapid_first': 2, 'projected_version': '442', 'version': '442', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x10000000241.000*', 'snapid_first': 2, 'projected_version': '426', 'version': '426', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}] 2023-11-28T18:26:26.840 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:26:26.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.976+0000 7f1a1a0c9640 1 Processor -- start 2023-11-28T18:26:26.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.976+0000 7f1a1a0c9640 1 -- start start 2023-11-28T18:26:26.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.976+0000 7f1a1a0c9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1a1415d4f0 0x7f1a140d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:26.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.976+0000 7f1a1a0c9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1a140b2d20 0x7f1a140b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:26.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.976+0000 7f1a1a0c9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a140b25b0 con 0x7f1a140d3c30 2023-11-28T18:26:26.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.976+0000 7f1a1a0c9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a140b2900 con 0x7f1a1415d4f0 2023-11-28T18:26:26.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.976+0000 7f1a1a0c9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a140d0a20 con 0x7f1a140b2d20 2023-11-28T18:26:26.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.976+0000 7f1a137fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1a1415d4f0 0x7f1a140d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:26.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.976+0000 7f1a12ffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f1a140d3c30 0x7f1a140d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35208/0 (socket says 172.21.15.105:35208) 2023-11-28T18:26:26.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.976+0000 7f1a12ffd640 1 -- 172.21.15.105:0/2124822179 learned_addr learned my addr 172.21.15.105:0/2124822179 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:26.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.976+0000 7f1a13fff640 1 --2- 172.21.15.105:0/2124822179 >> v2:172.21.15.148:3300/0 conn(0x7f1a140b2d20 0x7f1a140b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:26.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.976+0000 7f1a137fe640 1 -- 172.21.15.105:0/2124822179 >> v2:172.21.15.148:3300/0 conn(0x7f1a140b2d20 msgr2=0x7f1a140b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:26.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.976+0000 7f1a137fe640 1 --2- 172.21.15.105:0/2124822179 >> v2:172.21.15.148:3300/0 conn(0x7f1a140b2d20 0x7f1a140b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:26.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.976+0000 7f1a137fe640 1 -- 172.21.15.105:0/2124822179 >> v1:172.21.15.105:6789/0 conn(0x7f1a140d3c30 legacy=0x7f1a140d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:26.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.976+0000 7f1a137fe640 1 -- 172.21.15.105:0/2124822179 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a140cea30 con 0x7f1a1415d4f0 2023-11-28T18:26:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.980+0000 7f1a137fe640 1 --2- 172.21.15.105:0/2124822179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1415d4f0 0x7f1a140d0e10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1a14160fa0 tx=0x7f1a0401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80b4bac3b009376 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.980+0000 7f1a127fc640 1 -- 172.21.15.105:0/2124822179 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a040ad680 con 0x7f1a1415d4f0 2023-11-28T18:26:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.980+0000 7f1a127fc640 1 -- 172.21.15.105:0/2124822179 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a040ad7e0 con 0x7f1a1415d4f0 2023-11-28T18:26:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.980+0000 7f1a127fc640 1 -- 172.21.15.105:0/2124822179 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a040adb70 con 0x7f1a1415d4f0 2023-11-28T18:26:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.980+0000 7f1a1a0c9640 1 -- 172.21.15.105:0/2124822179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1415d4f0 msgr2=0x7f1a140d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.980+0000 7f1a1a0c9640 1 --2- 172.21.15.105:0/2124822179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1415d4f0 0x7f1a140d0e10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1a14160fa0 tx=0x7f1a0401c380 comp rx=0 tx=0).stop 2023-11-28T18:26:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.980+0000 7f1a1a0c9640 1 -- 172.21.15.105:0/2124822179 shutdown_connections 2023-11-28T18:26:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.980+0000 7f1a1a0c9640 1 --2- 172.21.15.105:0/2124822179 >> v2:172.21.15.148:3300/0 conn(0x7f1a140b2d20 0x7f1a140b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.980+0000 7f1a1a0c9640 1 --2- 172.21.15.105:0/2124822179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1415d4f0 0x7f1a140d0e10 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.980+0000 7f1a1a0c9640 1 -- 172.21.15.105:0/2124822179 >> 172.21.15.105:0/2124822179 conn(0x7f1a140d3030 msgr2=0x7f1a140d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:26.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.980+0000 7f1a1a0c9640 1 -- 172.21.15.105:0/2124822179 shutdown_connections 2023-11-28T18:26:26.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.980+0000 7f1a1a0c9640 1 -- 172.21.15.105:0/2124822179 wait complete. 2023-11-28T18:26:26.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f1a1a0c9640 1 Processor -- start 2023-11-28T18:26:26.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f1a1a0c9640 1 -- start start 2023-11-28T18:26:26.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f1a1a0c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a140b2d20 0x7f1a1414c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:26.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f1a1a0c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a140d3c30 0x7f1a1415c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:26.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f1a1a0c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1415d4f0 0x7f1a140ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:26.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f1a1a0c9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a140b8b90 con 0x7f1a1415d4f0 2023-11-28T18:26:26.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f1a1a0c9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a140ccd30 con 0x7f1a140b2d20 2023-11-28T18:26:26.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f1a1a0c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a141572d0 con 0x7f1a140d3c30 2023-11-28T18:26:26.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f1a13fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1415d4f0 0x7f1a140ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:26.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f1a13fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1415d4f0 0x7f1a140ca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35602/0 (socket says 172.21.15.105:35602) 2023-11-28T18:26:26.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f1a13fff640 1 -- 172.21.15.105:0/3418736107 learned_addr learned my addr 172.21.15.105:0/3418736107 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:26.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f1a137fe640 1 --2- 172.21.15.105:0/3418736107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a140b2d20 0x7f1a1414c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:26.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f1a12ffd640 1 --2- 172.21.15.105:0/3418736107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a140d3c30 0x7f1a1415c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:26.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f1a13fff640 1 -- 172.21.15.105:0/3418736107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a140d3c30 msgr2=0x7f1a1415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:26.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f1a13fff640 1 --2- 172.21.15.105:0/3418736107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a140d3c30 0x7f1a1415c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:26.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f1a13fff640 1 -- 172.21.15.105:0/3418736107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a140b2d20 msgr2=0x7f1a1414c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:26.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f1a13fff640 1 --2- 172.21.15.105:0/3418736107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a140b2d20 0x7f1a1414c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:26.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f1a13fff640 1 -- 172.21.15.105:0/3418736107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a04009420 con 0x7f1a1415d4f0 2023-11-28T18:26:26.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f1a13fff640 1 --2- 172.21.15.105:0/3418736107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1415d4f0 0x7f1a140ca100 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00efc0 tx=0x7f1a0c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:26.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f19fbfff640 1 -- 172.21.15.105:0/3418736107 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c009ba0 con 0x7f1a1415d4f0 2023-11-28T18:26:26.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f19fbfff640 1 -- 172.21.15.105:0/3418736107 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0c00f040 con 0x7f1a1415d4f0 2023-11-28T18:26:26.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f19fbfff640 1 -- 172.21.15.105:0/3418736107 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c0146c0 con 0x7f1a1415d4f0 2023-11-28T18:26:26.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f1a1a0c9640 1 -- 172.21.15.105:0/3418736107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a141575b0 con 0x7f1a1415d4f0 2023-11-28T18:26:26.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.984+0000 7f1a1a0c9640 1 -- 172.21.15.105:0/3418736107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a1415acf0 con 0x7f1a1415d4f0 2023-11-28T18:26:26.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.988+0000 7f1a1a0c9640 1 -- 172.21.15.105:0/3418736107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a1415af40 con 0x7f1a1415d4f0 2023-11-28T18:26:26.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.988+0000 7f19fbfff640 1 -- 172.21.15.105:0/3418736107 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f1a0c014820 con 0x7f1a1415d4f0 2023-11-28T18:26:26.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.988+0000 7f19fbfff640 1 --2- 172.21.15.105:0/3418736107 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1a000668d0 0x7f1a00068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:26.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.988+0000 7f19fbfff640 1 -- 172.21.15.105:0/3418736107 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f1a0c084fa0 con 0x7f1a1415d4f0 2023-11-28T18:26:26.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.988+0000 7f1a137fe640 1 --2- 172.21.15.105:0/3418736107 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1a000668d0 0x7f1a00068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:26.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.988+0000 7f1a137fe640 1 --2- 172.21.15.105:0/3418736107 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1a000668d0 0x7f1a00068d90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1a040af000 tx=0x7f1a040a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:26.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:26.992+0000 7f19fbfff640 1 -- 172.21.15.105:0/3418736107 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1a0c052070 con 0x7f1a1415d4f0 2023-11-28T18:26:27.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.172+0000 7f1a1a0c9640 1 -- 172.21.15.105:0/3418736107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a14067570 con 0x7f1a1415d4f0 2023-11-28T18:26:27.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.172+0000 7f19fbfff640 1 -- 172.21.15.105:0/3418736107 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f1a0c018090 con 0x7f1a1415d4f0 2023-11-28T18:26:27.175 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:27.175 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/735646078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":735646078},{"type":"v1","addr":"172.21.15.105:6836","nonce":735646078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:26:00.550298+0000","modified":"2023-11-28T18:26:21.303361+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5842,"mds_1":5414,"mds_2":5408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5408":{"gid":5408,"name":"c","rank":2,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6837/3765684108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":3765684108},{"type":"v1","addr":"172.21.15.105:6837","nonce":3765684108}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5414":{"gid":5414,"name":"d","rank":1,"incarnation":37,"state":"up:active","state_seq":5,"addr":"172.21.15.148:6835/897997046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":897997046},{"type":"v1","addr":"172.21.15.148:6835","nonce":897997046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5842":{"gid":5842,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6834/2696773413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2696773413},{"type":"v1","addr":"172.21.15.148:6834","nonce":2696773413}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T18:26:27.176 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 41 2023-11-28T18:26:27.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.172+0000 7f1a1a0c9640 1 -- 172.21.15.105:0/3418736107 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1a000668d0 msgr2=0x7f1a00068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.172+0000 7f1a1a0c9640 1 --2- 172.21.15.105:0/3418736107 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1a000668d0 0x7f1a00068d90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1a040af000 tx=0x7f1a040a7000 comp rx=0 tx=0).stop 2023-11-28T18:26:27.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.172+0000 7f1a1a0c9640 1 -- 172.21.15.105:0/3418736107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1415d4f0 msgr2=0x7f1a140ca100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.172+0000 7f1a1a0c9640 1 --2- 172.21.15.105:0/3418736107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1415d4f0 0x7f1a140ca100 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00efc0 tx=0x7f1a0c00c490 comp rx=0 tx=0).stop 2023-11-28T18:26:27.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.172+0000 7f1a1a0c9640 1 -- 172.21.15.105:0/3418736107 shutdown_connections 2023-11-28T18:26:27.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.172+0000 7f1a1a0c9640 1 --2- 172.21.15.105:0/3418736107 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1a000668d0 0x7f1a00068d90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.172+0000 7f1a1a0c9640 1 --2- 172.21.15.105:0/3418736107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1415d4f0 0x7f1a140ca100 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.172+0000 7f1a1a0c9640 1 --2- 172.21.15.105:0/3418736107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a140d3c30 0x7f1a1415c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.172+0000 7f1a1a0c9640 1 --2- 172.21.15.105:0/3418736107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a140b2d20 0x7f1a1414c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.172+0000 7f1a1a0c9640 1 -- 172.21.15.105:0/3418736107 >> 172.21.15.105:0/3418736107 conn(0x7f1a140d3030 msgr2=0x7f1a14072eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:27.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.176+0000 7f1a1a0c9640 1 -- 172.21.15.105:0/3418736107 shutdown_connections 2023-11-28T18:26:27.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.176+0000 7f1a1a0c9640 1 -- 172.21.15.105:0/3418736107 wait complete. 2023-11-28T18:26:27.196 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:2 perf dump 2023-11-28T18:26:27.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.332+0000 7f906272a640 1 Processor -- start 2023-11-28T18:26:27.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.336+0000 7f906272a640 1 -- start start 2023-11-28T18:26:27.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.336+0000 7f906272a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f905c159f40 0x7f905c0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.336+0000 7f906272a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f905c06bf50 con 0x7f905c0ad850 2023-11-28T18:26:27.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.336+0000 7f906272a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f905c0ccbc0 con 0x7f905c0cc760 2023-11-28T18:26:27.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.336+0000 7f906272a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f905c15a320 con 0x7f905c159f40 2023-11-28T18:26:27.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.336+0000 7f9060cc7640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f905c0ad850 0x7f905c0701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35224/0 (socket says 172.21.15.105:35224) 2023-11-28T18:26:27.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.336+0000 7f9060cc7640 1 -- 172.21.15.105:0/4237647749 learned_addr learned my addr 172.21.15.105:0/4237647749 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:27.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.336+0000 7f905bfff640 1 --2- 172.21.15.105:0/4237647749 >> v2:172.21.15.148:3300/0 conn(0x7f905c159f40 0x7f905c0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.336+0000 7f905affd640 1 -- 172.21.15.105:0/4237647749 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3658475525 0 0) 0x7f905c06bf50 con 0x7f905c0ad850 2023-11-28T18:26:27.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.336+0000 7f905affd640 1 -- 172.21.15.105:0/4237647749 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9044003530 con 0x7f905c0ad850 2023-11-28T18:26:27.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.336+0000 7f905bfff640 1 -- 172.21.15.105:0/4237647749 >> v1:172.21.15.148:6789/0 conn(0x7f905c0cc760 legacy=0x7f905c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.336+0000 7f905bfff640 1 -- 172.21.15.105:0/4237647749 >> v1:172.21.15.105:6789/0 conn(0x7f905c0ad850 legacy=0x7f905c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.336+0000 7f905affd640 1 -- 172.21.15.105:0/4237647749 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2797460088 0 0) 0x7f905c0ccbc0 con 0x7f905c0cc760 2023-11-28T18:26:27.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.336+0000 7f905bfff640 1 -- 172.21.15.105:0/4237647749 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905c071380 con 0x7f905c159f40 2023-11-28T18:26:27.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.340+0000 7f905bfff640 1 --2- 172.21.15.105:0/4237647749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f905c159f40 0x7f905c0caf60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f904c0097c0 tx=0x7f904c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8173db01b4baee5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:27.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.340+0000 7f905affd640 1 -- 172.21.15.105:0/4237647749 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c0ac620 con 0x7f905c159f40 2023-11-28T18:26:27.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.340+0000 7f905affd640 1 -- 172.21.15.105:0/4237647749 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c0ac780 con 0x7f905c159f40 2023-11-28T18:26:27.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.340+0000 7f905affd640 1 -- 172.21.15.105:0/4237647749 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c0acb10 con 0x7f905c159f40 2023-11-28T18:26:27.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.340+0000 7f906272a640 1 -- 172.21.15.105:0/4237647749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f905c159f40 msgr2=0x7f905c0caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.340+0000 7f906272a640 1 --2- 172.21.15.105:0/4237647749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f905c159f40 0x7f905c0caf60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f904c0097c0 tx=0x7f904c01c3c0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.340+0000 7f906272a640 1 -- 172.21.15.105:0/4237647749 shutdown_connections 2023-11-28T18:26:27.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.340+0000 7f906272a640 1 --2- 172.21.15.105:0/4237647749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f905c159f40 0x7f905c0caf60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.340+0000 7f906272a640 1 -- 172.21.15.105:0/4237647749 >> 172.21.15.105:0/4237647749 conn(0x7f905c0ceb90 msgr2=0x7f905c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:27.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.340+0000 7f906272a640 1 -- 172.21.15.105:0/4237647749 shutdown_connections 2023-11-28T18:26:27.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.340+0000 7f906272a640 1 -- 172.21.15.105:0/4237647749 wait complete. 2023-11-28T18:26:27.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f906272a640 1 Processor -- start 2023-11-28T18:26:27.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f906272a640 1 -- start start 2023-11-28T18:26:27.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f906272a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f905c0ad850 0x7f905c144c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f906272a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f905c0cc760 0x7f905c147ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f906272a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f905c159f40 0x7f905c0ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f906272a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c144a80 con 0x7f905c0cc760 2023-11-28T18:26:27.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f906272a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c15c160 con 0x7f905c159f40 2023-11-28T18:26:27.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f906272a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f905c15c400 con 0x7f905c0ad850 2023-11-28T18:26:27.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f905b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f905c0cc760 0x7f905c147ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f905b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f905c0cc760 0x7f905c147ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35616/0 (socket says 172.21.15.105:35616) 2023-11-28T18:26:27.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f905b7fe640 1 -- 172.21.15.105:0/1524243117 learned_addr learned my addr 172.21.15.105:0/1524243117 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:27.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f905bfff640 1 --2- 172.21.15.105:0/1524243117 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f905c0ad850 0x7f905c144c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f9060cc7640 1 --2- 172.21.15.105:0/1524243117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f905c159f40 0x7f905c0ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f905b7fe640 1 -- 172.21.15.105:0/1524243117 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f905c0ad850 msgr2=0x7f905c144c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f905b7fe640 1 --2- 172.21.15.105:0/1524243117 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f905c0ad850 0x7f905c144c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f905b7fe640 1 -- 172.21.15.105:0/1524243117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f905c159f40 msgr2=0x7f905c0ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f905b7fe640 1 --2- 172.21.15.105:0/1524243117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f905c159f40 0x7f905c0ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f905b7fe640 1 -- 172.21.15.105:0/1524243117 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904c009420 con 0x7f905c0cc760 2023-11-28T18:26:27.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f905b7fe640 1 --2- 172.21.15.105:0/1524243117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f905c0cc760 0x7f905c147ee0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f905000c480 tx=0x7f905000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:27.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f9058ff9640 1 -- 172.21.15.105:0/1524243117 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9050005480 con 0x7f905c0cc760 2023-11-28T18:26:27.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f9058ff9640 1 -- 172.21.15.105:0/1524243117 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9050003600 con 0x7f905c0cc760 2023-11-28T18:26:27.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f906272a640 1 -- 172.21.15.105:0/1524243117 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f905c15c6e0 con 0x7f905c0cc760 2023-11-28T18:26:27.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f9058ff9640 1 -- 172.21.15.105:0/1524243117 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90500039b0 con 0x7f905c0cc760 2023-11-28T18:26:27.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.344+0000 7f906272a640 1 -- 172.21.15.105:0/1524243117 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905c15aa20 con 0x7f905c0cc760 2023-11-28T18:26:27.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f9058ff9640 1 -- 172.21.15.105:0/1524243117 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f9050003b10 con 0x7f905c0cc760 2023-11-28T18:26:27.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f9058ff9640 1 --2- 172.21.15.105:0/1524243117 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f90480668d0 0x7f9048068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f9058ff9640 1 -- 172.21.15.105:0/1524243117 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f9050083ff0 con 0x7f905c0cc760 2023-11-28T18:26:27.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f905bfff640 1 --2- 172.21.15.105:0/1524243117 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f90480668d0 0x7f9048068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f9065aac000 1 Processor -- start 2023-11-28T18:26:27.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f9065aac000 1 -- start start 2023-11-28T18:26:27.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f905bfff640 1 --2- 172.21.15.105:0/1524243117 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f90480668d0 0x7f9048068d90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f904c0ae000 tx=0x7f904c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:27.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f9065aac000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56345370f7c0 0x56345370fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f9065aac000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634537100e0 0x5634536c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f9065aac000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634536c5520 0x5634536c5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f9065aac000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x56345351c9e0 con 0x5634537100e0 2023-11-28T18:26:27.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f9065aac000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563453519f10 con 0x5634536c5520 2023-11-28T18:26:27.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f9065aac000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x563453515800 con 0x56345370f7c0 2023-11-28T18:26:27.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f905b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634537100e0 0x5634536c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f905b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634537100e0 0x5634536c4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35626/0 (socket says 172.21.15.105:35626) 2023-11-28T18:26:27.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f905b7fe640 1 -- 172.21.15.105:0/2706140445 learned_addr learned my addr 172.21.15.105:0/2706140445 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:27.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f905bfff640 1 --2- 172.21.15.105:0/2706140445 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56345370f7c0 0x56345370fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f9060cc7640 1 --2- 172.21.15.105:0/2706140445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634536c5520 0x5634536c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f905b7fe640 1 -- 172.21.15.105:0/2706140445 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56345370f7c0 msgr2=0x56345370fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:27.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f905b7fe640 1 --2- 172.21.15.105:0/2706140445 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56345370f7c0 0x56345370fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f905b7fe640 1 -- 172.21.15.105:0/2706140445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634536c5520 msgr2=0x5634536c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:27.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f905b7fe640 1 --2- 172.21.15.105:0/2706140445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634536c5520 0x5634536c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.348+0000 7f905b7fe640 1 -- 172.21.15.105:0/2706140445 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56345320ddc0 con 0x5634537100e0 2023-11-28T18:26:27.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f905b7fe640 1 --2- 172.21.15.105:0/2706140445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634537100e0 0x5634536c4fe0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f905005b190 tx=0x7f9050054e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c192c720905687f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:27.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9041ffb640 1 -- 172.21.15.105:0/2706140445 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905007e070 con 0x5634537100e0 2023-11-28T18:26:27.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9041ffb640 1 -- 172.21.15.105:0/2706140445 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905005d0c0 con 0x5634537100e0 2023-11-28T18:26:27.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9041ffb640 1 -- 172.21.15.105:0/2706140445 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9050082440 con 0x5634537100e0 2023-11-28T18:26:27.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9065aac000 1 -- 172.21.15.105:0/2706140445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634537100e0 msgr2=0x5634536c4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9065aac000 1 --2- 172.21.15.105:0/2706140445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634537100e0 0x5634536c4fe0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f905005b190 tx=0x7f9050054e70 comp rx=0 tx=0).stop 2023-11-28T18:26:27.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9065aac000 1 -- 172.21.15.105:0/2706140445 shutdown_connections 2023-11-28T18:26:27.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9065aac000 1 --2- 172.21.15.105:0/2706140445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634536c5520 0x5634536c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9065aac000 1 --2- 172.21.15.105:0/2706140445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634537100e0 0x5634536c4fe0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9065aac000 1 --2- 172.21.15.105:0/2706140445 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56345370f7c0 0x56345370fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9065aac000 1 -- 172.21.15.105:0/2706140445 >> 172.21.15.105:0/2706140445 conn(0x5634536cdfb0 msgr2=0x5634536f2360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:27.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9065aac000 1 -- 172.21.15.105:0/2706140445 shutdown_connections 2023-11-28T18:26:27.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9065aac000 1 -- 172.21.15.105:0/2706140445 wait complete. 2023-11-28T18:26:27.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9065aac000 1 -- start start 2023-11-28T18:26:27.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9065aac000 1 Processor -- start 2023-11-28T18:26:27.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9065aac000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634536c5520 0x563453716910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9065aac000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56345370f7c0 0x563453716e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9065aac000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634537100e0 0x5634536f3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9065aac000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634534d3a70 con 0x5634537100e0 2023-11-28T18:26:27.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9065aac000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634534d3710 con 0x56345370f7c0 2023-11-28T18:26:27.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9065aac000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5634531cea40 con 0x5634536c5520 2023-11-28T18:26:27.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9065aac000 10 client.0 _ll_register_callbacks cb 0x5634536b0ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:27.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9060cc7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634537100e0 0x5634536f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9060cc7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634537100e0 0x5634536f3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35642/0 (socket says 172.21.15.105:35642) 2023-11-28T18:26:27.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9060cc7640 1 -- 172.21.15.105:0/1250736635 learned_addr learned my addr 172.21.15.105:0/1250736635 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:27.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f905b7fe640 1 --2- 172.21.15.105:0/1250736635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56345370f7c0 0x563453716e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f905bfff640 1 --2- 172.21.15.105:0/1250736635 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634536c5520 0x563453716910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9060cc7640 1 -- 172.21.15.105:0/1250736635 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634536c5520 msgr2=0x563453716910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:27.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9060cc7640 1 --2- 172.21.15.105:0/1250736635 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634536c5520 0x563453716910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9060cc7640 1 -- 172.21.15.105:0/1250736635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56345370f7c0 msgr2=0x563453716e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:27.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9060cc7640 1 --2- 172.21.15.105:0/1250736635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56345370f7c0 0x563453716e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9060cc7640 1 -- 172.21.15.105:0/1250736635 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905c15aa20 con 0x5634537100e0 2023-11-28T18:26:27.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f905bfff640 1 --2- 172.21.15.105:0/1250736635 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634536c5520 0x563453716910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:27.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f9060cc7640 1 --2- 172.21.15.105:0/1250736635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634537100e0 0x5634536f3590 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9054007ad0 tx=0x7f9054012690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a029ba87a68e699 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:27.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.352+0000 7f903affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:27.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f903affd640 1 -- 172.21.15.105:0/1250736635 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9054015440 con 0x5634537100e0 2023-11-28T18:26:27.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f9065aac000 1 -- 172.21.15.105:0/1250736635 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634534ceab0 con 0x5634537100e0 2023-11-28T18:26:27.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f903affd640 1 -- 172.21.15.105:0/1250736635 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f90540155a0 con 0x5634537100e0 2023-11-28T18:26:27.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f903affd640 1 -- 172.21.15.105:0/1250736635 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9054015760 con 0x5634537100e0 2023-11-28T18:26:27.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f9065aac000 10 client.6285 fetch_fsmap learned FSMap version 41 2023-11-28T18:26:27.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f9065aac000 1 -- 172.21.15.105:0/1250736635 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x5634534d6260 con 0x5634537100e0 2023-11-28T18:26:27.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f903affd640 1 -- 172.21.15.105:0/1250736635 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5634534ceab0 con 0x5634537100e0 2023-11-28T18:26:27.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f903affd640 1 -- 172.21.15.105:0/1250736635 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) v1 ==== 3326+0+0 (secure 0 0 0) 0x7f90540158c0 con 0x5634537100e0 2023-11-28T18:26:27.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f9065aac000 10 client.6285 fetch_fsmap finished waiting for FSMap version 41 2023-11-28T18:26:27.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f9065aac000 10 client.6285 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.c 2023-11-28T18:26:27.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f9065aac000 20 client.6285 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:27.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f9065aac000 1 --2- 172.21.15.105:0/1250736635 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x5634536f4300 0x5634536f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f9065aac000 4 client.6285 mds_command: new command op to 5408 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:26:27.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f9065aac000 1 -- 172.21.15.105:0/1250736635 --> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634534ca850 con 0x5634536f4300 2023-11-28T18:26:27.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f905bfff640 1 --2- 172.21.15.105:0/1250736635 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x5634536f4300 0x5634536f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f903affd640 10 client.6285 ms_handle_connect on v2:172.21.15.105:6835/3765684108 2023-11-28T18:26:27.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f905bfff640 1 --2- 172.21.15.105:0/1250736635 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x5634536f4300 0x5634536f46e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a280e7ef6ff50bf6 server_cookie=92c0f65629d798ef in_seq=0 out_seq=0 2023-11-28T18:26:27.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f903affd640 1 -- 172.21.15.105:0/1250736635 <== mds.2 v2:172.21.15.105:6835/3765684108 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5634534ca850 con 0x5634536f4300 2023-11-28T18:26:27.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f903affd640 10 client.6285 handle_command_reply: tid=0 2023-11-28T18:26:27.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f9065aac000 1 client.6285 shutdown 2023-11-28T18:26:27.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f9065aac000 1 -- 172.21.15.105:0/1250736635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634537100e0 msgr2=0x5634536f3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f9065aac000 1 --2- 172.21.15.105:0/1250736635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634537100e0 0x5634536f3590 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9054007ad0 tx=0x7f9054012690 comp rx=0 tx=0).stop 2023-11-28T18:26:27.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f9065aac000 1 -- 172.21.15.105:0/1250736635 shutdown_connections 2023-11-28T18:26:27.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f9065aac000 1 --2- 172.21.15.105:0/1250736635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634537100e0 0x5634536f3590 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f9065aac000 1 --2- 172.21.15.105:0/1250736635 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x5634536f4300 0x5634536f46e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f9065aac000 1 --2- 172.21.15.105:0/1250736635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56345370f7c0 0x563453716e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f9065aac000 1 --2- 172.21.15.105:0/1250736635 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634536c5520 0x563453716910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f903affd640 0 client.6285 ms_handle_reset on v2:172.21.15.105:6835/3765684108 2023-11-28T18:26:27.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f9065aac000 1 -- 172.21.15.105:0/1250736635 >> 172.21.15.105:0/1250736635 conn(0x5634536cdfb0 msgr2=0x5634536c46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:27.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f9065aac000 1 -- 172.21.15.105:0/1250736635 shutdown_connections 2023-11-28T18:26:27.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f9065aac000 1 -- 172.21.15.105:0/1250736635 wait complete. 2023-11-28T18:26:27.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.356+0000 7f9065aac000 20 client.6285 trim_cache size 0 max 16384 2023-11-28T18:26:27.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f9065aac000 1 Processor -- start 2023-11-28T18:26:27.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f9065aac000 1 -- start start 2023-11-28T18:26:27.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f9065aac000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634536c5520 0x5634536c4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f9065aac000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56345370f7c0 0x5634536c3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f9065aac000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634537100e0 0x5634536c3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f9065aac000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634536c9fb0 con 0x56345370f7c0 2023-11-28T18:26:27.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f9065aac000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563453519800 con 0x5634537100e0 2023-11-28T18:26:27.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f9065aac000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x563453513010 con 0x5634536c5520 2023-11-28T18:26:27.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f905b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56345370f7c0 0x5634536c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f905b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56345370f7c0 0x5634536c3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35648/0 (socket says 172.21.15.105:35648) 2023-11-28T18:26:27.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f905b7fe640 1 -- 172.21.15.105:0/579517315 learned_addr learned my addr 172.21.15.105:0/579517315 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:27.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f905b7fe640 1 -- 172.21.15.105:0/579517315 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634536c5520 msgr2=0x5634536c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:27.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f905b7fe640 1 --2- 172.21.15.105:0/579517315 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634536c5520 0x5634536c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f905b7fe640 1 -- 172.21.15.105:0/579517315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634537100e0 msgr2=0x5634536c3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:27.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f905b7fe640 1 --2- 172.21.15.105:0/579517315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634537100e0 0x5634536c3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f905b7fe640 1 -- 172.21.15.105:0/579517315 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634534bc760 con 0x56345370f7c0 2023-11-28T18:26:27.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f905b7fe640 1 --2- 172.21.15.105:0/579517315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56345370f7c0 0x5634536c3740 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9050054b70 tx=0x7f9050057000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebf5917edeacff8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:27.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f90427fc640 1 -- 172.21.15.105:0/579517315 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905007e040 con 0x56345370f7c0 2023-11-28T18:26:27.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f90427fc640 1 -- 172.21.15.105:0/579517315 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9050070070 con 0x56345370f7c0 2023-11-28T18:26:27.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f90427fc640 1 -- 172.21.15.105:0/579517315 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905005d0c0 con 0x56345370f7c0 2023-11-28T18:26:27.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f9065aac000 1 -- 172.21.15.105:0/579517315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56345370f7c0 msgr2=0x5634536c3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f9065aac000 1 --2- 172.21.15.105:0/579517315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56345370f7c0 0x5634536c3740 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9050054b70 tx=0x7f9050057000 comp rx=0 tx=0).stop 2023-11-28T18:26:27.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f9065aac000 1 -- 172.21.15.105:0/579517315 shutdown_connections 2023-11-28T18:26:27.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f9065aac000 1 --2- 172.21.15.105:0/579517315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634537100e0 0x5634536c3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f9065aac000 1 --2- 172.21.15.105:0/579517315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56345370f7c0 0x5634536c3740 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f9065aac000 1 --2- 172.21.15.105:0/579517315 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634536c5520 0x5634536c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f9065aac000 1 -- 172.21.15.105:0/579517315 >> 172.21.15.105:0/579517315 conn(0x5634536cdfb0 msgr2=0x56345370e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:27.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f9065aac000 1 -- 172.21.15.105:0/579517315 shutdown_connections 2023-11-28T18:26:27.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f9065aac000 1 -- 172.21.15.105:0/579517315 wait complete. 2023-11-28T18:26:27.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f9065aac000 1 -- start start 2023-11-28T18:26:27.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.372+0000 7f9065aac000 1 Processor -- start 2023-11-28T18:26:27.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9065aac000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634536c5520 0x563453714610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9065aac000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56345370f7c0 0x5634537124d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9065aac000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634537100e0 0x563453712a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9065aac000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634534eb7b0 con 0x5634537100e0 2023-11-28T18:26:27.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9065aac000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563453499c20 con 0x56345370f7c0 2023-11-28T18:26:27.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9065aac000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x563453514690 con 0x5634536c5520 2023-11-28T18:26:27.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9065aac000 10 client.0 _ll_register_callbacks cb 0x5634534dd840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:27.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9060cc7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634537100e0 0x563453712a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9060cc7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634537100e0 0x563453712a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35656/0 (socket says 172.21.15.105:35656) 2023-11-28T18:26:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9060cc7640 1 -- 172.21.15.105:0/4165476295 learned_addr learned my addr 172.21.15.105:0/4165476295 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f905b7fe640 1 --2- 172.21.15.105:0/4165476295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56345370f7c0 0x5634537124d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f905bfff640 1 --2- 172.21.15.105:0/4165476295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634536c5520 0x563453714610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9060cc7640 1 -- 172.21.15.105:0/4165476295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634536c5520 msgr2=0x563453714610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:27.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9060cc7640 1 --2- 172.21.15.105:0/4165476295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634536c5520 0x563453714610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9060cc7640 1 -- 172.21.15.105:0/4165476295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56345370f7c0 msgr2=0x5634537124d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:27.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9060cc7640 1 --2- 172.21.15.105:0/4165476295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56345370f7c0 0x5634537124d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9060cc7640 1 -- 172.21.15.105:0/4165476295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56345320ddc0 con 0x5634537100e0 2023-11-28T18:26:27.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9060cc7640 1 --2- 172.21.15.105:0/4165476295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634537100e0 0x563453712a10 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9054007aa0 tx=0x7f9054001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c3c562449a8707 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:27.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f903affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:27.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f903affd640 1 -- 172.21.15.105:0/4165476295 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9054001b50 con 0x5634537100e0 2023-11-28T18:26:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f903affd640 1 -- 172.21.15.105:0/4165476295 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f9054012ad0 con 0x5634537100e0 2023-11-28T18:26:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9065aac000 1 -- 172.21.15.105:0/4165476295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634534d83a0 con 0x5634537100e0 2023-11-28T18:26:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f903affd640 1 -- 172.21.15.105:0/4165476295 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9054007710 con 0x5634537100e0 2023-11-28T18:26:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9065aac000 10 client.6291 fetch_fsmap learned FSMap version 41 2023-11-28T18:26:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9065aac000 1 -- 172.21.15.105:0/4165476295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x5634534b0040 con 0x5634537100e0 2023-11-28T18:26:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f903affd640 1 -- 172.21.15.105:0/4165476295 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5634534d83a0 con 0x5634537100e0 2023-11-28T18:26:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f903affd640 1 -- 172.21.15.105:0/4165476295 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) v1 ==== 3326+0+0 (secure 0 0 0) 0x7f905401e640 con 0x5634537100e0 2023-11-28T18:26:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9065aac000 10 client.6291 fetch_fsmap finished waiting for FSMap version 41 2023-11-28T18:26:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9065aac000 10 client.6291 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.c 2023-11-28T18:26:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9065aac000 20 client.6291 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9065aac000 1 --2- 172.21.15.105:0/4165476295 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x56345376f810 0x56345376fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9065aac000 4 client.6291 mds_command: new command op to 5408 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2023-11-28T18:26:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9065aac000 1 -- 172.21.15.105:0/4165476295 --> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x56345350e7e0 con 0x56345376f810 2023-11-28T18:26:27.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f905bfff640 1 --2- 172.21.15.105:0/4165476295 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x56345376f810 0x56345376fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f905bfff640 1 --2- 172.21.15.105:0/4165476295 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x56345376f810 0x56345376fc20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=90be113c19c50f5b server_cookie=716817130be72110 in_seq=0 out_seq=0 2023-11-28T18:26:27.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f903affd640 10 client.6291 ms_handle_connect on v2:172.21.15.105:6835/3765684108 2023-11-28T18:26:27.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f903affd640 1 -- 172.21.15.105:0/4165476295 <== mds.2 v2:172.21.15.105:6835/3765684108 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17619 (crc 0 0 0) 0x56345350e7e0 con 0x56345376f810 2023-11-28T18:26:27.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f903affd640 10 client.6291 handle_command_reply: tid=0 2023-11-28T18:26:27.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.376+0000 7f9065aac000 1 client.6291 shutdown 2023-11-28T18:26:27.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f9065aac000 1 -- 172.21.15.105:0/4165476295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634537100e0 msgr2=0x563453712a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f9065aac000 1 --2- 172.21.15.105:0/4165476295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634537100e0 0x563453712a10 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9054007aa0 tx=0x7f9054001260 comp rx=0 tx=0).stop 2023-11-28T18:26:27.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f9065aac000 1 -- 172.21.15.105:0/4165476295 shutdown_connections 2023-11-28T18:26:27.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f9065aac000 1 --2- 172.21.15.105:0/4165476295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634537100e0 0x563453712a10 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f9065aac000 1 --2- 172.21.15.105:0/4165476295 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x56345376f810 0x56345376fc20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f9065aac000 1 --2- 172.21.15.105:0/4165476295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56345370f7c0 0x5634537124d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f9065aac000 1 --2- 172.21.15.105:0/4165476295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634536c5520 0x563453714610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f9065aac000 1 -- 172.21.15.105:0/4165476295 >> 172.21.15.105:0/4165476295 conn(0x5634536cdfb0 msgr2=0x56345370f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:27.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f903affd640 0 client.6291 ms_handle_reset on v2:172.21.15.105:6835/3765684108 2023-11-28T18:26:27.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f9065aac000 1 -- 172.21.15.105:0/4165476295 shutdown_connections 2023-11-28T18:26:27.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f9065aac000 1 -- 172.21.15.105:0/4165476295 wait complete. 2023-11-28T18:26:27.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f9065aac000 20 client.6291 trim_cache size 0 max 16384 2023-11-28T18:26:27.398 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:26:27.399 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2023-11-28T18:26:27.399 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 24, 2023-11-28T18:26:27.399 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 77, 2023-11-28T18:26:27.399 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 7117, 2023-11-28T18:26:27.399 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 38575, 2023-11-28T18:26:27.399 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 9, 2023-11-28T18:26:27.399 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 3, 2023-11-28T18:26:27.399 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.013957175, 2023-11-28T18:26:27.399 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.001840489, 2023-11-28T18:26:27.399 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.008361147, 2023-11-28T18:26:27.399 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000316139, 2023-11-28T18:26:27.400 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2023-11-28T18:26:27.400 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 76, 2023-11-28T18:26:27.400 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.044843558, 2023-11-28T18:26:27.400 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000590046 2023-11-28T18:26:27.400 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.400 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2023-11-28T18:26:27.400 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 11, 2023-11-28T18:26:27.401 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000003383, 2023-11-28T18:26:27.401 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000307 2023-11-28T18:26:27.401 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.401 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 1568, 2023-11-28T18:26:27.401 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 448 2023-11-28T18:26:27.401 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.401 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2023-11-28T18:26:27.402 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 65, 2023-11-28T18:26:27.402 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 59, 2023-11-28T18:26:27.402 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 116950, 2023-11-28T18:26:27.402 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 59330, 2023-11-28T18:26:27.402 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 10, 2023-11-28T18:26:27.403 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 7, 2023-11-28T18:26:27.403 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.010617970, 2023-11-28T18:26:27.403 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.001738482, 2023-11-28T18:26:27.403 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.010756305, 2023-11-28T18:26:27.403 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000507523, 2023-11-28T18:26:27.403 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2023-11-28T18:26:27.404 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 56, 2023-11-28T18:26:27.404 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.038186098, 2023-11-28T18:26:27.404 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000681894 2023-11-28T18:26:27.404 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.404 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2023-11-28T18:26:27.404 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 7, 2023-11-28T18:26:27.405 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000795, 2023-11-28T18:26:27.405 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000113 2023-11-28T18:26:27.405 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.405 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 110144, 2023-11-28T18:26:27.405 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 10480 2023-11-28T18:26:27.405 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.406 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2023-11-28T18:26:27.406 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 37, 2023-11-28T18:26:27.406 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 90, 2023-11-28T18:26:27.406 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 75819, 2023-11-28T18:26:27.406 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 158075, 2023-11-28T18:26:27.407 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 6, 2023-11-28T18:26:27.407 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 3, 2023-11-28T18:26:27.407 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.008912142, 2023-11-28T18:26:27.407 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.002811658, 2023-11-28T18:26:27.407 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.005963944, 2023-11-28T18:26:27.407 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000291334, 2023-11-28T18:26:27.408 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2023-11-28T18:26:27.408 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 91, 2023-11-28T18:26:27.408 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.017549285, 2023-11-28T18:26:27.408 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000192849 2023-11-28T18:26:27.408 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.408 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2023-11-28T18:26:27.408 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 81, 2023-11-28T18:26:27.408 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000044589, 2023-11-28T18:26:27.409 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000550 2023-11-28T18:26:27.409 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.409 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-28T18:26:27.409 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 93920 2023-11-28T18:26:27.409 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.409 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2023-11-28T18:26:27.409 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2023-11-28T18:26:27.410 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2023-11-28T18:26:27.410 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.410 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2023-11-28T18:26:27.410 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2023-11-28T18:26:27.410 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2023-11-28T18:26:27.410 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 37, 2023-11-28T18:26:27.410 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.002981218, 2023-11-28T18:26:27.410 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000080573 2023-11-28T18:26:27.410 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.411 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.411 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2023-11-28T18:26:27.411 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2023-11-28T18:26:27.411 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2023-11-28T18:26:27.411 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2, 2023-11-28T18:26:27.411 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000006684, 2023-11-28T18:26:27.411 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000003342 2023-11-28T18:26:27.412 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.412 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.412 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2023-11-28T18:26:27.412 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 0, 2023-11-28T18:26:27.412 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 0, 2023-11-28T18:26:27.412 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2023-11-28T18:26:27.412 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.412 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.412 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.413 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.413 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2023-11-28T18:26:27.413 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2023-11-28T18:26:27.413 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2023-11-28T18:26:27.413 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2023-11-28T18:26:27.413 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 11, 2023-11-28T18:26:27.413 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2023-11-28T18:26:27.413 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2023-11-28T18:26:27.413 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 63, 2023-11-28T18:26:27.414 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 0, 2023-11-28T18:26:27.414 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 63, 2023-11-28T18:26:27.414 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2023-11-28T18:26:27.414 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 63, 2023-11-28T18:26:27.414 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2023-11-28T18:26:27.414 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 52, 2023-11-28T18:26:27.414 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 52, 2023-11-28T18:26:27.415 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 4, 2023-11-28T18:26:27.415 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 2, 2023-11-28T18:26:27.415 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 0, 2023-11-28T18:26:27.415 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2023-11-28T18:26:27.415 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 2, 2023-11-28T18:26:27.415 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2023-11-28T18:26:27.415 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2023-11-28T18:26:27.415 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2023-11-28T18:26:27.416 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2023-11-28T18:26:27.416 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2023-11-28T18:26:27.416 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2023-11-28T18:26:27.416 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2023-11-28T18:26:27.416 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 2, 2023-11-28T18:26:27.416 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 52, 2023-11-28T18:26:27.416 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2023-11-28T18:26:27.416 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2023-11-28T18:26:27.417 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2023-11-28T18:26:27.417 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 99, 2023-11-28T18:26:27.417 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 50, 2023-11-28T18:26:27.417 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2023-11-28T18:26:27.417 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2023-11-28T18:26:27.417 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2023-11-28T18:26:27.418 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2023-11-28T18:26:27.418 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2023-11-28T18:26:27.418 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2023-11-28T18:26:27.418 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2023-11-28T18:26:27.418 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2023-11-28T18:26:27.418 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2023-11-28T18:26:27.418 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2023-11-28T18:26:27.418 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 0, 2023-11-28T18:26:27.419 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 0, 2023-11-28T18:26:27.419 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2023-11-28T18:26:27.419 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-28T18:26:27.419 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-28T18:26:27.419 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 0, 2023-11-28T18:26:27.419 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 0, 2023-11-28T18:26:27.420 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 0, 2023-11-28T18:26:27.420 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 0 2023-11-28T18:26:27.420 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.420 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2023-11-28T18:26:27.420 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2023-11-28T18:26:27.420 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2023-11-28T18:26:27.420 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2023-11-28T18:26:27.420 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 3, 2023-11-28T18:26:27.420 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 0, 2023-11-28T18:26:27.421 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2023-11-28T18:26:27.421 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2023-11-28T18:26:27.421 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2023-11-28T18:26:27.421 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 0, 2023-11-28T18:26:27.421 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 0, 2023-11-28T18:26:27.421 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2023-11-28T18:26:27.421 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2023-11-28T18:26:27.421 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2023-11-28T18:26:27.421 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2023-11-28T18:26:27.422 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2023-11-28T18:26:27.422 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2023-11-28T18:26:27.422 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2023-11-28T18:26:27.422 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2023-11-28T18:26:27.422 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2023-11-28T18:26:27.422 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2023-11-28T18:26:27.422 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2023-11-28T18:26:27.422 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2023-11-28T18:26:27.423 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2023-11-28T18:26:27.423 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.423 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2023-11-28T18:26:27.423 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2023-11-28T18:26:27.423 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 6, 2023-11-28T18:26:27.423 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2023-11-28T18:26:27.423 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2023-11-28T18:26:27.423 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 6, 2023-11-28T18:26:27.424 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2023-11-28T18:26:27.424 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2023-11-28T18:26:27.424 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 2, 2023-11-28T18:26:27.424 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2023-11-28T18:26:27.424 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2023-11-28T18:26:27.424 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 2, 2023-11-28T18:26:27.424 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2023-11-28T18:26:27.424 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2023-11-28T18:26:27.425 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2023-11-28T18:26:27.425 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2023-11-28T18:26:27.425 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 4233719, 2023-11-28T18:26:27.425 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 0, 2023-11-28T18:26:27.425 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2023-11-28T18:26:27.425 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4, 2023-11-28T18:26:27.425 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.015715047, 2023-11-28T18:26:27.426 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.003928761 2023-11-28T18:26:27.426 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.426 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 0 2023-11-28T18:26:27.426 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.426 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2023-11-28T18:26:27.426 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 66, 2023-11-28T18:26:27.426 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 66, 2023-11-28T18:26:27.426 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 0, 2023-11-28T18:26:27.427 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 40, 2023-11-28T18:26:27.427 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 41, 2023-11-28T18:26:27.427 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 1, 2023-11-28T18:26:27.427 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 63, 2023-11-28T18:26:27.427 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 63, 2023-11-28T18:26:27.427 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 0, 2023-11-28T18:26:27.427 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 52, 2023-11-28T18:26:27.427 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 52, 2023-11-28T18:26:27.427 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 0, 2023-11-28T18:26:27.427 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 46736, 2023-11-28T18:26:27.427 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 198952 2023-11-28T18:26:27.428 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.428 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2023-11-28T18:26:27.428 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 0, 2023-11-28T18:26:27.428 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2023-11-28T18:26:27.428 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 0, 2023-11-28T18:26:27.428 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 2, 2023-11-28T18:26:27.428 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2023-11-28T18:26:27.428 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2023-11-28T18:26:27.428 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.428 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.428 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.429 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.429 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2023-11-28T18:26:27.429 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.429 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.429 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.429 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.429 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2023-11-28T18:26:27.429 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.429 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.429 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.430 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.430 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2023-11-28T18:26:27.430 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.430 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.430 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.430 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.430 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2023-11-28T18:26:27.431 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.431 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.431 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.431 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.431 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2023-11-28T18:26:27.431 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.431 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.431 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.431 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.431 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2023-11-28T18:26:27.431 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.432 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.432 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.432 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.432 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2023-11-28T18:26:27.432 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.432 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.432 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.432 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.432 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2023-11-28T18:26:27.432 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.432 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.433 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.433 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.433 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2023-11-28T18:26:27.433 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.433 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.433 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.433 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.433 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2023-11-28T18:26:27.433 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.433 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.433 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.434 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.434 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2023-11-28T18:26:27.434 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.434 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.434 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.434 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.434 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2023-11-28T18:26:27.434 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.434 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.435 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.435 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.435 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2023-11-28T18:26:27.435 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.435 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.435 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.435 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.435 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2023-11-28T18:26:27.435 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.435 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.435 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.435 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.436 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2023-11-28T18:26:27.436 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.436 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.436 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.436 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.436 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2023-11-28T18:26:27.436 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.436 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.436 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.436 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.436 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2023-11-28T18:26:27.437 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.437 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.437 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.437 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.437 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2023-11-28T18:26:27.437 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.437 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.437 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.437 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.437 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2023-11-28T18:26:27.437 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.438 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.438 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.438 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.438 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2023-11-28T18:26:27.438 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.438 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.438 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.438 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.438 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2023-11-28T18:26:27.438 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.438 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.439 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.439 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.439 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2023-11-28T18:26:27.439 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.439 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.439 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.439 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.439 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2023-11-28T18:26:27.439 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.439 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.439 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.439 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.440 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2023-11-28T18:26:27.440 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.440 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.440 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.440 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.440 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2023-11-28T18:26:27.440 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.440 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.440 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.440 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.440 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2023-11-28T18:26:27.441 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.441 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.441 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.441 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.441 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2023-11-28T18:26:27.441 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.441 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.441 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.441 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.441 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2023-11-28T18:26:27.441 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.441 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.442 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.442 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.442 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2023-11-28T18:26:27.442 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2023-11-28T18:26:27.442 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2023-11-28T18:26:27.442 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.442 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.442 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.442 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.442 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.442 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2023-11-28T18:26:27.442 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 1, 2023-11-28T18:26:27.443 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 1, 2023-11-28T18:26:27.443 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 0, 2023-11-28T18:26:27.443 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 1, 2023-11-28T18:26:27.443 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2023-11-28T18:26:27.443 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 0, 2023-11-28T18:26:27.443 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 0, 2023-11-28T18:26:27.443 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 986, 2023-11-28T18:26:27.443 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2023-11-28T18:26:27.443 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.443 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2023-11-28T18:26:27.443 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2023-11-28T18:26:27.443 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2023-11-28T18:26:27.444 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2023-11-28T18:26:27.444 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2023-11-28T18:26:27.444 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2023-11-28T18:26:27.444 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2023-11-28T18:26:27.444 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-28T18:26:27.444 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2023-11-28T18:26:27.444 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-28T18:26:27.444 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2023-11-28T18:26:27.444 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2023-11-28T18:26:27.444 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2023-11-28T18:26:27.444 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-28T18:26:27.444 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2023-11-28T18:26:27.445 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2023-11-28T18:26:27.445 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2023-11-28T18:26:27.445 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2023-11-28T18:26:27.445 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2023-11-28T18:26:27.445 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-28T18:26:27.445 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2023-11-28T18:26:27.445 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-28T18:26:27.445 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2023-11-28T18:26:27.445 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2023-11-28T18:26:27.445 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2023-11-28T18:26:27.445 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2023-11-28T18:26:27.445 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2023-11-28T18:26:27.446 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-28T18:26:27.446 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2023-11-28T18:26:27.446 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2023-11-28T18:26:27.446 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2023-11-28T18:26:27.446 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2023-11-28T18:26:27.446 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2023-11-28T18:26:27.446 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2023-11-28T18:26:27.446 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2023-11-28T18:26:27.446 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2023-11-28T18:26:27.446 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2023-11-28T18:26:27.446 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 241007, 2023-11-28T18:26:27.446 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 68, 2023-11-28T18:26:27.447 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2023-11-28T18:26:27.447 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2023-11-28T18:26:27.447 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2023-11-28T18:26:27.447 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2023-11-28T18:26:27.447 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2023-11-28T18:26:27.447 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2023-11-28T18:26:27.447 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2023-11-28T18:26:27.447 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2023-11-28T18:26:27.447 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 5616, 2023-11-28T18:26:27.447 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 80, 2023-11-28T18:26:27.447 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2023-11-28T18:26:27.447 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2023-11-28T18:26:27.448 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2023-11-28T18:26:27.448 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2023-11-28T18:26:27.448 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 256680, 2023-11-28T18:26:27.448 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 1201, 2023-11-28T18:26:27.448 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2023-11-28T18:26:27.448 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2023-11-28T18:26:27.448 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2023-11-28T18:26:27.449 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2023-11-28T18:26:27.449 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.449 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2023-11-28T18:26:27.449 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2023-11-28T18:26:27.449 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2023-11-28T18:26:27.449 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 39, 2023-11-28T18:26:27.449 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 51250, 2023-11-28T18:26:27.449 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 21, 2023-11-28T18:26:27.449 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 39, 2023-11-28T18:26:27.449 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2023-11-28T18:26:27.450 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 39, 2023-11-28T18:26:27.450 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.327996151, 2023-11-28T18:26:27.450 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.008410157 2023-11-28T18:26:27.450 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.450 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2023-11-28T18:26:27.450 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2023-11-28T18:26:27.450 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 39, 2023-11-28T18:26:27.450 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 62 2023-11-28T18:26:27.450 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.450 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 39, 2023-11-28T18:26:27.450 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 0, 2023-11-28T18:26:27.450 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 39, 2023-11-28T18:26:27.450 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2023-11-28T18:26:27.451 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2023-11-28T18:26:27.451 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 0, 2023-11-28T18:26:27.451 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 10, 2023-11-28T18:26:27.451 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 0, 2023-11-28T18:26:27.451 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 4, 2023-11-28T18:26:27.451 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 5, 2023-11-28T18:26:27.451 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2023-11-28T18:26:27.451 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2023-11-28T18:26:27.451 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2023-11-28T18:26:27.451 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2023-11-28T18:26:27.451 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 6, 2023-11-28T18:26:27.451 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2023-11-28T18:26:27.452 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2023-11-28T18:26:27.452 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2023-11-28T18:26:27.452 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 0, 2023-11-28T18:26:27.452 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 20, 2023-11-28T18:26:27.452 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2023-11-28T18:26:27.452 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2023-11-28T18:26:27.452 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2023-11-28T18:26:27.452 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2023-11-28T18:26:27.452 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2023-11-28T18:26:27.452 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2023-11-28T18:26:27.452 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2023-11-28T18:26:27.453 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2023-11-28T18:26:27.453 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2023-11-28T18:26:27.453 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2023-11-28T18:26:27.453 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2023-11-28T18:26:27.453 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2023-11-28T18:26:27.453 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2023-11-28T18:26:27.453 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2023-11-28T18:26:27.453 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2023-11-28T18:26:27.453 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2023-11-28T18:26:27.453 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2023-11-28T18:26:27.453 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2023-11-28T18:26:27.453 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2023-11-28T18:26:27.453 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2023-11-28T18:26:27.453 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2023-11-28T18:26:27.454 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2023-11-28T18:26:27.454 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2023-11-28T18:26:27.454 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2023-11-28T18:26:27.454 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2023-11-28T18:26:27.454 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2023-11-28T18:26:27.454 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 53, 2023-11-28T18:26:27.454 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2023-11-28T18:26:27.454 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 2, 2023-11-28T18:26:27.454 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 7, 2023-11-28T18:26:27.454 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 7, 2023-11-28T18:26:27.454 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 0, 2023-11-28T18:26:27.454 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2023-11-28T18:26:27.455 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 16, 2023-11-28T18:26:27.455 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 0, 2023-11-28T18:26:27.455 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 1 2023-11-28T18:26:27.455 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.455 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2023-11-28T18:26:27.455 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 1, 2023-11-28T18:26:27.455 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2023-11-28T18:26:27.455 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2023-11-28T18:26:27.455 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2023-11-28T18:26:27.455 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.455 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2023-11-28T18:26:27.455 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2023-11-28T18:26:27.456 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.456 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2023-11-28T18:26:27.456 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2023-11-28T18:26:27.456 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 0, 2023-11-28T18:26:27.456 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2023-11-28T18:26:27.456 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 0, 2023-11-28T18:26:27.456 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 0, 2023-11-28T18:26:27.456 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2023-11-28T18:26:27.456 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.456 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-28T18:26:27.456 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:27.456 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2023-11-28T18:26:27.456 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:27.457 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 121, 2023-11-28T18:26:27.457 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 185115, 2023-11-28T18:26:27.457 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:27.457 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 121, 2023-11-28T18:26:27.457 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2023-11-28T18:26:27.457 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2023-11-28T18:26:27.457 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 121, 2023-11-28T18:26:27.457 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 185115, 2023-11-28T18:26:27.457 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:27.457 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.457 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.458 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.458 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.458 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.458 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2023-11-28T18:26:27.458 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:27.458 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2023-11-28T18:26:27.458 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:27.458 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2023-11-28T18:26:27.458 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2023-11-28T18:26:27.458 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:27.458 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2023-11-28T18:26:27.458 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 39, 2023-11-28T18:26:27.459 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 51250, 2023-11-28T18:26:27.459 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 39, 2023-11-28T18:26:27.459 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 51250, 2023-11-28T18:26:27.459 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:27.459 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.459 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.459 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.459 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.459 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.459 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2023-11-28T18:26:27.459 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:27.459 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2023-11-28T18:26:27.459 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:27.460 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2023-11-28T18:26:27.460 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2023-11-28T18:26:27.460 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:27.460 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2023-11-28T18:26:27.460 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 39, 2023-11-28T18:26:27.460 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 39, 2023-11-28T18:26:27.460 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 39, 2023-11-28T18:26:27.460 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 39, 2023-11-28T18:26:27.460 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:27.460 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.460 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.460 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.460 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.461 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.461 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2023-11-28T18:26:27.461 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:27.461 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2023-11-28T18:26:27.461 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:27.461 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2023-11-28T18:26:27.461 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2023-11-28T18:26:27.461 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:27.461 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2023-11-28T18:26:27.461 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2023-11-28T18:26:27.461 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2023-11-28T18:26:27.461 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2023-11-28T18:26:27.461 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2023-11-28T18:26:27.461 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:27.462 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.462 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.462 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.462 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.462 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.462 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x56496458ca00": { 2023-11-28T18:26:27.462 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:27.462 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2023-11-28T18:26:27.462 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:27.462 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 6, 2023-11-28T18:26:27.462 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 39415, 2023-11-28T18:26:27.462 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:27.463 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 6, 2023-11-28T18:26:27.463 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2023-11-28T18:26:27.463 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2023-11-28T18:26:27.463 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 4, 2023-11-28T18:26:27.463 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 39415, 2023-11-28T18:26:27.463 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:27.463 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.463 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.463 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.463 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.463 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.463 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:26:27.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f906272a640 1 -- 172.21.15.105:0/1524243117 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f90480668d0 msgr2=0x7f9048068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f906272a640 1 --2- 172.21.15.105:0/1524243117 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f90480668d0 0x7f9048068d90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f904c0ae000 tx=0x7f904c0a7000 comp rx=0 tx=0).stop 2023-11-28T18:26:27.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f906272a640 1 -- 172.21.15.105:0/1524243117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f905c0cc760 msgr2=0x7f905c147ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f906272a640 1 --2- 172.21.15.105:0/1524243117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f905c0cc760 0x7f905c147ee0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f905000c480 tx=0x7f905000c940 comp rx=0 tx=0).stop 2023-11-28T18:26:27.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f906272a640 1 -- 172.21.15.105:0/1524243117 shutdown_connections 2023-11-28T18:26:27.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f906272a640 1 --2- 172.21.15.105:0/1524243117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f905c159f40 0x7f905c0ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f906272a640 1 --2- 172.21.15.105:0/1524243117 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f90480668d0 0x7f9048068d90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f906272a640 1 --2- 172.21.15.105:0/1524243117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f905c0cc760 0x7f905c147ee0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f906272a640 1 --2- 172.21.15.105:0/1524243117 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f905c0ad850 0x7f905c144c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f906272a640 1 -- 172.21.15.105:0/1524243117 >> 172.21.15.105:0/1524243117 conn(0x7f905c0ceb90 msgr2=0x7f905c071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:27.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f906272a640 1 -- 172.21.15.105:0/1524243117 shutdown_connections 2023-11-28T18:26:27.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.380+0000 7f906272a640 1 -- 172.21.15.105:0/1524243117 wait complete. 2023-11-28T18:26:27.466 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:1 perf dump 2023-11-28T18:26:27.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.604+0000 7f754df2a640 1 Processor -- start 2023-11-28T18:26:27.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.604+0000 7f754df2a640 1 -- start start 2023-11-28T18:26:27.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.604+0000 7f754df2a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f75480ae290 0x7f754815d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.604+0000 7f754df2a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75480cc7f0 con 0x7f7548154070 2023-11-28T18:26:27.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.604+0000 7f754df2a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f754815dd10 con 0x7f754806bf50 2023-11-28T18:26:27.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.604+0000 7f754df2a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7548072540 con 0x7f75480ae290 2023-11-28T18:26:27.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.604+0000 7f75477fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7548154070 0x7f75480ce310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35226/0 (socket says 172.21.15.105:35226) 2023-11-28T18:26:27.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.604+0000 7f75477fe640 1 -- 172.21.15.105:0/1386074850 learned_addr learned my addr 172.21.15.105:0/1386074850 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:27.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.604+0000 7f7547fff640 1 --2- 172.21.15.105:0/1386074850 >> v2:172.21.15.105:3300/0 conn(0x7f75480ae290 0x7f754815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.604+0000 7f75467fc640 1 -- 172.21.15.105:0/1386074850 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2223193950 0 0) 0x7f75480cc7f0 con 0x7f7548154070 2023-11-28T18:26:27.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.604+0000 7f75467fc640 1 -- 172.21.15.105:0/1386074850 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7530003530 con 0x7f7548154070 2023-11-28T18:26:27.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.604+0000 7f7547fff640 1 -- 172.21.15.105:0/1386074850 >> v1:172.21.15.148:6789/0 conn(0x7f754806bf50 legacy=0x7f75480adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.604+0000 7f7547fff640 1 -- 172.21.15.105:0/1386074850 >> v1:172.21.15.105:6789/0 conn(0x7f7548154070 legacy=0x7f75480ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.604+0000 7f7547fff640 1 -- 172.21.15.105:0/1386074850 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754815dff0 con 0x7f75480ae290 2023-11-28T18:26:27.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.608+0000 7f7547fff640 1 --2- 172.21.15.105:0/1386074850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75480ae290 0x7f754815d700 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7540009d90 tx=0x7f754001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28757833cea08c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:27.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.608+0000 7f75467fc640 1 -- 172.21.15.105:0/1386074850 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75400a6620 con 0x7f75480ae290 2023-11-28T18:26:27.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.608+0000 7f75467fc640 1 -- 172.21.15.105:0/1386074850 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75400a6780 con 0x7f75480ae290 2023-11-28T18:26:27.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.608+0000 7f75467fc640 1 -- 172.21.15.105:0/1386074850 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75400a6b10 con 0x7f75480ae290 2023-11-28T18:26:27.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.608+0000 7f754df2a640 1 -- 172.21.15.105:0/1386074850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75480ae290 msgr2=0x7f754815d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.608+0000 7f754df2a640 1 --2- 172.21.15.105:0/1386074850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75480ae290 0x7f754815d700 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7540009d90 tx=0x7f754001c920 comp rx=0 tx=0).stop 2023-11-28T18:26:27.613 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.608+0000 7f754df2a640 1 -- 172.21.15.105:0/1386074850 shutdown_connections 2023-11-28T18:26:27.613 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.608+0000 7f754df2a640 1 --2- 172.21.15.105:0/1386074850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75480ae290 0x7f754815d700 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.613 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.608+0000 7f754df2a640 1 -- 172.21.15.105:0/1386074850 >> 172.21.15.105:0/1386074850 conn(0x7f75480ceb90 msgr2=0x7f75480cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:27.613 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.608+0000 7f754df2a640 1 -- 172.21.15.105:0/1386074850 shutdown_connections 2023-11-28T18:26:27.613 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.608+0000 7f754df2a640 1 -- 172.21.15.105:0/1386074850 wait complete. 2023-11-28T18:26:27.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f754df2a640 1 Processor -- start 2023-11-28T18:26:27.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f754df2a640 1 -- start start 2023-11-28T18:26:27.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f754df2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f754806bf50 0x7f754814b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f754df2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75480ae290 0x7f7548148400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f754df2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7548154070 0x7f7548160c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f754df2a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754809c640 con 0x7f75480ae290 2023-11-28T18:26:27.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f754df2a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7548148940 con 0x7f7548154070 2023-11-28T18:26:27.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f754df2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7548161a70 con 0x7f754806bf50 2023-11-28T18:26:27.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f7546ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75480ae290 0x7f7548148400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f7546ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75480ae290 0x7f7548148400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35674/0 (socket says 172.21.15.105:35674) 2023-11-28T18:26:27.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f7546ffd640 1 -- 172.21.15.105:0/2624752275 learned_addr learned my addr 172.21.15.105:0/2624752275 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:27.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f75477fe640 1 --2- 172.21.15.105:0/2624752275 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f754806bf50 0x7f754814b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f7547fff640 1 --2- 172.21.15.105:0/2624752275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7548154070 0x7f7548160c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f7546ffd640 1 -- 172.21.15.105:0/2624752275 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f754806bf50 msgr2=0x7f754814b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f7546ffd640 1 --2- 172.21.15.105:0/2624752275 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f754806bf50 0x7f754814b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f7546ffd640 1 -- 172.21.15.105:0/2624752275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7548154070 msgr2=0x7f7548160c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f7546ffd640 1 --2- 172.21.15.105:0/2624752275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7548154070 0x7f7548160c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f7546ffd640 1 -- 172.21.15.105:0/2624752275 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7540009a70 con 0x7f75480ae290 2023-11-28T18:26:27.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f7546ffd640 1 --2- 172.21.15.105:0/2624752275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75480ae290 0x7f7548148400 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f753c00c500 tx=0x7f753c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:27.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f752ffff640 1 -- 172.21.15.105:0/2624752275 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c005220 con 0x7f75480ae290 2023-11-28T18:26:27.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f752ffff640 1 -- 172.21.15.105:0/2624752275 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753c005380 con 0x7f75480ae290 2023-11-28T18:26:27.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f752ffff640 1 -- 172.21.15.105:0/2624752275 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c007270 con 0x7f75480ae290 2023-11-28T18:26:27.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f754df2a640 1 -- 172.21.15.105:0/2624752275 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75481614b0 con 0x7f75480ae290 2023-11-28T18:26:27.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.612+0000 7f754df2a640 1 -- 172.21.15.105:0/2624752275 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75481f93c0 con 0x7f75480ae290 2023-11-28T18:26:27.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.616+0000 7f752ffff640 1 -- 172.21.15.105:0/2624752275 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f753c0073d0 con 0x7f75480ae290 2023-11-28T18:26:27.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.616+0000 7f752ffff640 1 --2- 172.21.15.105:0/2624752275 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f75340668d0 0x7f7534068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.616+0000 7f752ffff640 1 -- 172.21.15.105:0/2624752275 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f753c083e10 con 0x7f75480ae290 2023-11-28T18:26:27.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.616+0000 7f75477fe640 1 --2- 172.21.15.105:0/2624752275 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f75340668d0 0x7f7534068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.616+0000 7f75512a3000 1 Processor -- start 2023-11-28T18:26:27.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.616+0000 7f75512a3000 1 -- start start 2023-11-28T18:26:27.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.616+0000 7f75477fe640 1 --2- 172.21.15.105:0/2624752275 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f75340668d0 0x7f7534068d90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f75380011f0 tx=0x7f7538007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:27.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.616+0000 7f75512a3000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55aa41c737c0 0x55aa41c73ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.616+0000 7f75512a3000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c740e0 0x55aa41c28fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.616+0000 7f75512a3000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa41c29520 0x55aa41c29900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.616+0000 7f75512a3000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa41a809e0 con 0x55aa41c740e0 2023-11-28T18:26:27.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.616+0000 7f75512a3000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa41a7df10 con 0x55aa41c29520 2023-11-28T18:26:27.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.616+0000 7f75512a3000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa41a79800 con 0x55aa41c737c0 2023-11-28T18:26:27.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.616+0000 7f7546ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c740e0 0x55aa41c28fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.616+0000 7f75477fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55aa41c737c0 0x55aa41c73ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.616+0000 7f7546ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c740e0 0x55aa41c28fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35684/0 (socket says 172.21.15.105:35684) 2023-11-28T18:26:27.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.616+0000 7f7546ffd640 1 -- 172.21.15.105:0/1155876277 learned_addr learned my addr 172.21.15.105:0/1155876277 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:27.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.616+0000 7f7547fff640 1 --2- 172.21.15.105:0/1155876277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa41c29520 0x55aa41c29900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f7546ffd640 1 -- 172.21.15.105:0/1155876277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55aa41c737c0 msgr2=0x55aa41c73ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:27.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f7546ffd640 1 --2- 172.21.15.105:0/1155876277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55aa41c737c0 0x55aa41c73ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f7546ffd640 1 -- 172.21.15.105:0/1155876277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa41c29520 msgr2=0x55aa41c29900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:27.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f7546ffd640 1 --2- 172.21.15.105:0/1155876277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa41c29520 0x55aa41c29900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f7546ffd640 1 -- 172.21.15.105:0/1155876277 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa41771dc0 con 0x55aa41c740e0 2023-11-28T18:26:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f7546ffd640 1 --2- 172.21.15.105:0/1155876277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c740e0 0x55aa41c28fe0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f753c05c8d0 tx=0x7f753c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4dbca348004bb81 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f752d7fa640 1 -- 172.21.15.105:0/1155876277 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c058c50 con 0x55aa41c740e0 2023-11-28T18:26:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f752d7fa640 1 -- 172.21.15.105:0/1155876277 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753c05ce00 con 0x55aa41c740e0 2023-11-28T18:26:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f752d7fa640 1 -- 172.21.15.105:0/1155876277 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c0512c0 con 0x55aa41c740e0 2023-11-28T18:26:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f75512a3000 1 -- 172.21.15.105:0/1155876277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c740e0 msgr2=0x55aa41c28fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f75512a3000 1 --2- 172.21.15.105:0/1155876277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c740e0 0x55aa41c28fe0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f753c05c8d0 tx=0x7f753c00f000 comp rx=0 tx=0).stop 2023-11-28T18:26:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f75512a3000 1 -- 172.21.15.105:0/1155876277 shutdown_connections 2023-11-28T18:26:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f75512a3000 1 --2- 172.21.15.105:0/1155876277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa41c29520 0x55aa41c29900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f75512a3000 1 --2- 172.21.15.105:0/1155876277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c740e0 0x55aa41c28fe0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f75512a3000 1 --2- 172.21.15.105:0/1155876277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55aa41c737c0 0x55aa41c73ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f75512a3000 1 -- 172.21.15.105:0/1155876277 >> 172.21.15.105:0/1155876277 conn(0x55aa41c31fb0 msgr2=0x55aa41c56360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f75512a3000 1 -- 172.21.15.105:0/1155876277 shutdown_connections 2023-11-28T18:26:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f75512a3000 1 -- 172.21.15.105:0/1155876277 wait complete. 2023-11-28T18:26:27.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f75512a3000 1 -- start start 2023-11-28T18:26:27.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f75512a3000 1 Processor -- start 2023-11-28T18:26:27.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f75512a3000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa41c29520 0x55aa41c30a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f75512a3000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c737c0 0x55aa41c57590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f75512a3000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55aa41c740e0 0x55aa41c57ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f75512a3000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa41a37a70 con 0x55aa41c737c0 2023-11-28T18:26:27.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f75512a3000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa41a37710 con 0x55aa41c29520 2023-11-28T18:26:27.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f75512a3000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa41732a40 con 0x55aa41c740e0 2023-11-28T18:26:27.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f75512a3000 10 client.0 _ll_register_callbacks cb 0x55aa41c14ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:27.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f7546ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c737c0 0x55aa41c57590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f7546ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c737c0 0x55aa41c57590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35694/0 (socket says 172.21.15.105:35694) 2023-11-28T18:26:27.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f7546ffd640 1 -- 172.21.15.105:0/4031495290 learned_addr learned my addr 172.21.15.105:0/4031495290 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:27.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f7547fff640 1 --2- 172.21.15.105:0/4031495290 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55aa41c740e0 0x55aa41c57ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f75477fe640 1 --2- 172.21.15.105:0/4031495290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa41c29520 0x55aa41c30a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.620+0000 7f7546ffd640 1 -- 172.21.15.105:0/4031495290 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55aa41c740e0 msgr2=0x55aa41c57ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:27.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f7546ffd640 1 --2- 172.21.15.105:0/4031495290 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55aa41c740e0 0x55aa41c57ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f7546ffd640 1 -- 172.21.15.105:0/4031495290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa41c29520 msgr2=0x55aa41c30a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:27.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f7546ffd640 1 --2- 172.21.15.105:0/4031495290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa41c29520 0x55aa41c30a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f7546ffd640 1 -- 172.21.15.105:0/4031495290 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75481f93c0 con 0x55aa41c737c0 2023-11-28T18:26:27.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f7546ffd640 1 --2- 172.21.15.105:0/4031495290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c737c0 0x55aa41c57590 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f753c05c8d0 tx=0x7f753c000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdf6e4404056925 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:27.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75267fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:27.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75267fc640 1 -- 172.21.15.105:0/4031495290 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c058c50 con 0x55aa41c737c0 2023-11-28T18:26:27.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75267fc640 1 -- 172.21.15.105:0/4031495290 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f753c071240 con 0x55aa41c737c0 2023-11-28T18:26:27.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75512a3000 1 -- 172.21.15.105:0/4031495290 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa41a32ab0 con 0x55aa41c737c0 2023-11-28T18:26:27.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75267fc640 1 -- 172.21.15.105:0/4031495290 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f753c0512c0 con 0x55aa41c737c0 2023-11-28T18:26:27.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75512a3000 10 client.6306 fetch_fsmap learned FSMap version 41 2023-11-28T18:26:27.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75512a3000 1 -- 172.21.15.105:0/4031495290 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x55aa41a3a260 con 0x55aa41c737c0 2023-11-28T18:26:27.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75267fc640 1 -- 172.21.15.105:0/4031495290 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753c07b020 con 0x55aa41c737c0 2023-11-28T18:26:27.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75267fc640 1 -- 172.21.15.105:0/4031495290 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) v1 ==== 3326+0+0 (secure 0 0 0) 0x7f753c019030 con 0x55aa41c737c0 2023-11-28T18:26:27.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75512a3000 10 client.6306 fetch_fsmap finished waiting for FSMap version 41 2023-11-28T18:26:27.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75512a3000 10 client.6306 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.d 2023-11-28T18:26:27.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75512a3000 20 client.6306 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:27.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75512a3000 1 --2- 172.21.15.105:0/4031495290 >> [v2:172.21.15.148:6833/897997046,v1:172.21.15.148:6835/897997046] conn(0x55aa41c79e00 0x55aa41c7a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75512a3000 4 client.6306 mds_command: new command op to 5414 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:26:27.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75512a3000 1 -- 172.21.15.105:0/4031495290 --> [v2:172.21.15.148:6833/897997046,v1:172.21.15.148:6835/897997046] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa41a2e850 con 0x55aa41c79e00 2023-11-28T18:26:27.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75477fe640 1 --2- 172.21.15.105:0/4031495290 >> [v2:172.21.15.148:6833/897997046,v1:172.21.15.148:6835/897997046] conn(0x55aa41c79e00 0x55aa41c7a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75267fc640 10 client.6306 ms_handle_connect on v2:172.21.15.148:6833/897997046 2023-11-28T18:26:27.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75477fe640 1 --2- 172.21.15.105:0/4031495290 >> [v2:172.21.15.148:6833/897997046,v1:172.21.15.148:6835/897997046] conn(0x55aa41c79e00 0x55aa41c7a210 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc4c6e130a6f5f6c server_cookie=a21d9692809d3154 in_seq=0 out_seq=0 2023-11-28T18:26:27.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75267fc640 1 -- 172.21.15.105:0/4031495290 <== mds.1 v2:172.21.15.148:6833/897997046 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aa41a2e850 con 0x55aa41c79e00 2023-11-28T18:26:27.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75267fc640 10 client.6306 handle_command_reply: tid=0 2023-11-28T18:26:27.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75512a3000 1 client.6306 shutdown 2023-11-28T18:26:27.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75512a3000 1 -- 172.21.15.105:0/4031495290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c737c0 msgr2=0x55aa41c57590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75512a3000 1 --2- 172.21.15.105:0/4031495290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c737c0 0x55aa41c57590 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f753c05c8d0 tx=0x7f753c000ee0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75512a3000 1 -- 172.21.15.105:0/4031495290 shutdown_connections 2023-11-28T18:26:27.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75512a3000 1 --2- 172.21.15.105:0/4031495290 >> [v2:172.21.15.148:6833/897997046,v1:172.21.15.148:6835/897997046] conn(0x55aa41c79e00 0x55aa41c7a210 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75512a3000 1 --2- 172.21.15.105:0/4031495290 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55aa41c740e0 0x55aa41c57ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75512a3000 1 --2- 172.21.15.105:0/4031495290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c737c0 0x55aa41c57590 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75512a3000 1 --2- 172.21.15.105:0/4031495290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa41c29520 0x55aa41c30a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75512a3000 1 -- 172.21.15.105:0/4031495290 >> 172.21.15.105:0/4031495290 conn(0x55aa41c31fb0 msgr2=0x55aa41c286d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:27.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75267fc640 0 client.6306 ms_handle_reset on v2:172.21.15.148:6833/897997046 2023-11-28T18:26:27.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75512a3000 1 -- 172.21.15.105:0/4031495290 shutdown_connections 2023-11-28T18:26:27.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75512a3000 1 -- 172.21.15.105:0/4031495290 wait complete. 2023-11-28T18:26:27.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.624+0000 7f75512a3000 20 client.6306 trim_cache size 0 max 16384 2023-11-28T18:26:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.640+0000 7f75512a3000 1 Processor -- start 2023-11-28T18:26:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.640+0000 7f75512a3000 1 -- start start 2023-11-28T18:26:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.640+0000 7f75512a3000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa41c30f80 0x55aa41cd0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.640+0000 7f75512a3000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55aa41c2dc90 0x55aa41cd0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.640+0000 7f75512a3000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c31fb0 0x55aa41cd10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.640+0000 7f75512a3000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa41a4f7b0 con 0x55aa41c31fb0 2023-11-28T18:26:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.640+0000 7f75512a3000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa41a75170 con 0x55aa41c30f80 2023-11-28T18:26:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.640+0000 7f75512a3000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa41a7d800 con 0x55aa41c2dc90 2023-11-28T18:26:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.640+0000 7f7547fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c31fb0 0x55aa41cd10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.640+0000 7f7547fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c31fb0 0x55aa41cd10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35700/0 (socket says 172.21.15.105:35700) 2023-11-28T18:26:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.640+0000 7f7547fff640 1 -- 172.21.15.105:0/561337092 learned_addr learned my addr 172.21.15.105:0/561337092 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.640+0000 7f75477fe640 1 --2- 172.21.15.105:0/561337092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa41c30f80 0x55aa41cd0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.640+0000 7f7546ffd640 1 --2- 172.21.15.105:0/561337092 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55aa41c2dc90 0x55aa41cd0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.640+0000 7f7547fff640 1 -- 172.21.15.105:0/561337092 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55aa41c2dc90 msgr2=0x55aa41cd0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.640+0000 7f7547fff640 1 --2- 172.21.15.105:0/561337092 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55aa41c2dc90 0x55aa41cd0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.640+0000 7f7547fff640 1 -- 172.21.15.105:0/561337092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa41c30f80 msgr2=0x55aa41cd0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.640+0000 7f7547fff640 1 --2- 172.21.15.105:0/561337092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa41c30f80 0x55aa41cd0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.640+0000 7f7547fff640 1 -- 172.21.15.105:0/561337092 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa41a20760 con 0x55aa41c31fb0 2023-11-28T18:26:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.640+0000 7f7547fff640 1 --2- 172.21.15.105:0/561337092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c31fb0 0x55aa41cd10c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f754001c960 tx=0x7f7540004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf12aa06e86a1feb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.640+0000 7f752dffb640 1 -- 172.21.15.105:0/561337092 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75400a6b70 con 0x55aa41c31fb0 2023-11-28T18:26:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.640+0000 7f752dffb640 1 -- 172.21.15.105:0/561337092 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75400c2070 con 0x55aa41c31fb0 2023-11-28T18:26:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.640+0000 7f752dffb640 1 -- 172.21.15.105:0/561337092 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75400c7430 con 0x55aa41c31fb0 2023-11-28T18:26:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75512a3000 1 -- 172.21.15.105:0/561337092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c31fb0 msgr2=0x55aa41cd10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75512a3000 1 --2- 172.21.15.105:0/561337092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c31fb0 0x55aa41cd10c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f754001c960 tx=0x7f7540004ac0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75512a3000 1 -- 172.21.15.105:0/561337092 shutdown_connections 2023-11-28T18:26:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75512a3000 1 --2- 172.21.15.105:0/561337092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c31fb0 0x55aa41cd10c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75512a3000 1 --2- 172.21.15.105:0/561337092 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55aa41c2dc90 0x55aa41cd0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75512a3000 1 --2- 172.21.15.105:0/561337092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa41c30f80 0x55aa41cd0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75512a3000 1 -- 172.21.15.105:0/561337092 >> 172.21.15.105:0/561337092 conn(0x55aa41c30ba0 msgr2=0x55aa41c72d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75512a3000 1 -- 172.21.15.105:0/561337092 shutdown_connections 2023-11-28T18:26:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75512a3000 1 -- 172.21.15.105:0/561337092 wait complete. 2023-11-28T18:26:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75512a3000 1 -- start start 2023-11-28T18:26:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75512a3000 1 Processor -- start 2023-11-28T18:26:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75512a3000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55aa41c2dc90 0x55aa41c78be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75512a3000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa41c30f80 0x55aa41c903e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75512a3000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c31fb0 0x55aa41c90920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75512a3000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa41a78690 con 0x55aa41c31fb0 2023-11-28T18:26:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75512a3000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa41a7c360 con 0x55aa41c30f80 2023-11-28T18:26:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75512a3000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa41a76c00 con 0x55aa41c2dc90 2023-11-28T18:26:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75512a3000 10 client.0 _ll_register_callbacks cb 0x55aa41a41840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f7547fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c31fb0 0x55aa41c90920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f7547fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c31fb0 0x55aa41c90920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35702/0 (socket says 172.21.15.105:35702) 2023-11-28T18:26:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f7547fff640 1 -- 172.21.15.105:0/1561330306 learned_addr learned my addr 172.21.15.105:0/1561330306 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f7547fff640 1 -- 172.21.15.105:0/1561330306 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55aa41c2dc90 msgr2=0x55aa41c78be0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T18:26:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f7547fff640 1 --2- 172.21.15.105:0/1561330306 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55aa41c2dc90 0x55aa41c78be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f7547fff640 1 -- 172.21.15.105:0/1561330306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa41c30f80 msgr2=0x55aa41c903e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T18:26:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f7547fff640 1 --2- 172.21.15.105:0/1561330306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa41c30f80 0x55aa41c903e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f7547fff640 1 -- 172.21.15.105:0/1561330306 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f754815dff0 con 0x55aa41c31fb0 2023-11-28T18:26:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f7547fff640 1 --2- 172.21.15.105:0/1561330306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c31fb0 0x55aa41c90920 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f754001a550 tx=0x7f7540004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5023d5b580f0a58 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75267fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75267fc640 1 -- 172.21.15.105:0/1561330306 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75400a6bf0 con 0x55aa41c31fb0 2023-11-28T18:26:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75267fc640 1 -- 172.21.15.105:0/1561330306 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f75400a6d50 con 0x55aa41c31fb0 2023-11-28T18:26:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75512a3000 1 -- 172.21.15.105:0/1561330306 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa41a3c3a0 con 0x55aa41c31fb0 2023-11-28T18:26:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75267fc640 1 -- 172.21.15.105:0/1561330306 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75400d1430 con 0x55aa41c31fb0 2023-11-28T18:26:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75512a3000 10 client.6312 fetch_fsmap learned FSMap version 41 2023-11-28T18:26:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75512a3000 1 -- 172.21.15.105:0/1561330306 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x55aa41a14040 con 0x55aa41c31fb0 2023-11-28T18:26:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.644+0000 7f75267fc640 1 -- 172.21.15.105:0/1561330306 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aa41a3c3a0 con 0x55aa41c31fb0 2023-11-28T18:26:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75267fc640 1 -- 172.21.15.105:0/1561330306 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) v1 ==== 3326+0+0 (secure 0 0 0) 0x7f75400bb040 con 0x55aa41c31fb0 2023-11-28T18:26:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75512a3000 10 client.6312 fetch_fsmap finished waiting for FSMap version 41 2023-11-28T18:26:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75512a3000 10 client.6312 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.d 2023-11-28T18:26:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75512a3000 20 client.6312 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75512a3000 1 --2- 172.21.15.105:0/1561330306 >> [v2:172.21.15.148:6833/897997046,v1:172.21.15.148:6835/897997046] conn(0x55aa41c29520 0x55aa41cd4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75512a3000 4 client.6312 mds_command: new command op to 5414 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2023-11-28T18:26:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75512a3000 1 -- 172.21.15.105:0/1561330306 --> [v2:172.21.15.148:6833/897997046,v1:172.21.15.148:6835/897997046] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55aa41a727e0 con 0x55aa41c29520 2023-11-28T18:26:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75477fe640 1 --2- 172.21.15.105:0/1561330306 >> [v2:172.21.15.148:6833/897997046,v1:172.21.15.148:6835/897997046] conn(0x55aa41c29520 0x55aa41cd4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75267fc640 10 client.6312 ms_handle_connect on v2:172.21.15.148:6833/897997046 2023-11-28T18:26:27.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75477fe640 1 --2- 172.21.15.105:0/1561330306 >> [v2:172.21.15.148:6833/897997046,v1:172.21.15.148:6835/897997046] conn(0x55aa41c29520 0x55aa41cd4680 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ab481b05e8ae9e9e server_cookie=dc5615b53e019977 in_seq=0 out_seq=0 2023-11-28T18:26:27.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75267fc640 1 -- 172.21.15.105:0/1561330306 <== mds.1 v2:172.21.15.148:6833/897997046 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17624 (crc 0 0 0) 0x55aa41a727e0 con 0x55aa41c29520 2023-11-28T18:26:27.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75267fc640 10 client.6312 handle_command_reply: tid=0 2023-11-28T18:26:27.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75512a3000 1 client.6312 shutdown 2023-11-28T18:26:27.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75512a3000 1 -- 172.21.15.105:0/1561330306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c31fb0 msgr2=0x55aa41c90920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75512a3000 1 --2- 172.21.15.105:0/1561330306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c31fb0 0x55aa41c90920 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f754001a550 tx=0x7f7540004a50 comp rx=0 tx=0).stop 2023-11-28T18:26:27.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75512a3000 1 -- 172.21.15.105:0/1561330306 shutdown_connections 2023-11-28T18:26:27.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75512a3000 1 --2- 172.21.15.105:0/1561330306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aa41c31fb0 0x55aa41c90920 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75512a3000 1 --2- 172.21.15.105:0/1561330306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa41c30f80 0x55aa41c903e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75512a3000 1 --2- 172.21.15.105:0/1561330306 >> [v2:172.21.15.148:6833/897997046,v1:172.21.15.148:6835/897997046] conn(0x55aa41c29520 0x55aa41cd4680 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75512a3000 1 --2- 172.21.15.105:0/1561330306 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55aa41c2dc90 0x55aa41c78be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75512a3000 1 -- 172.21.15.105:0/1561330306 >> 172.21.15.105:0/1561330306 conn(0x55aa41c30ba0 msgr2=0x55aa41c2f170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:27.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75267fc640 0 client.6312 ms_handle_reset on v2:172.21.15.148:6833/897997046 2023-11-28T18:26:27.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75512a3000 1 -- 172.21.15.105:0/1561330306 shutdown_connections 2023-11-28T18:26:27.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75512a3000 1 -- 172.21.15.105:0/1561330306 wait complete. 2023-11-28T18:26:27.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f75512a3000 20 client.6312 trim_cache size 0 max 16384 2023-11-28T18:26:27.657 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:26:27.657 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2023-11-28T18:26:27.657 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 48, 2023-11-28T18:26:27.658 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 105, 2023-11-28T18:26:27.658 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 96187, 2023-11-28T18:26:27.658 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 35054, 2023-11-28T18:26:27.658 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 8, 2023-11-28T18:26:27.658 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 2, 2023-11-28T18:26:27.658 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.017514115, 2023-11-28T18:26:27.658 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.004241630, 2023-11-28T18:26:27.658 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.010716714, 2023-11-28T18:26:27.658 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000322435, 2023-11-28T18:26:27.659 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2023-11-28T18:26:27.659 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 106, 2023-11-28T18:26:27.659 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.029673764, 2023-11-28T18:26:27.659 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000279941 2023-11-28T18:26:27.659 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.659 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2023-11-28T18:26:27.659 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 112, 2023-11-28T18:26:27.659 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000111286, 2023-11-28T18:26:27.659 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000993 2023-11-28T18:26:27.659 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.659 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 1568, 2023-11-28T18:26:27.660 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 448 2023-11-28T18:26:27.660 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.660 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2023-11-28T18:26:27.660 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 15, 2023-11-28T18:26:27.660 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 21, 2023-11-28T18:26:27.660 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 6189, 2023-11-28T18:26:27.660 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 78677, 2023-11-28T18:26:27.660 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 7, 2023-11-28T18:26:27.661 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 5, 2023-11-28T18:26:27.661 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.005446623, 2023-11-28T18:26:27.661 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.000968734, 2023-11-28T18:26:27.661 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.003920097, 2023-11-28T18:26:27.661 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000247939, 2023-11-28T18:26:27.661 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2023-11-28T18:26:27.661 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 21, 2023-11-28T18:26:27.661 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.019286927, 2023-11-28T18:26:27.661 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000918425 2023-11-28T18:26:27.661 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.662 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2023-11-28T18:26:27.662 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2, 2023-11-28T18:26:27.662 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000295, 2023-11-28T18:26:27.662 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000147 2023-11-28T18:26:27.662 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.662 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-28T18:26:27.662 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 63168 2023-11-28T18:26:27.662 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.662 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2023-11-28T18:26:27.662 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 81, 2023-11-28T18:26:27.663 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 140, 2023-11-28T18:26:27.663 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 121382, 2023-11-28T18:26:27.663 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 137285, 2023-11-28T18:26:27.663 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 11, 2023-11-28T18:26:27.663 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 8, 2023-11-28T18:26:27.663 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.014311112, 2023-11-28T18:26:27.663 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.003682793, 2023-11-28T18:26:27.663 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.011435345, 2023-11-28T18:26:27.663 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000630508, 2023-11-28T18:26:27.663 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2023-11-28T18:26:27.664 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 136, 2023-11-28T18:26:27.664 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.030178806, 2023-11-28T18:26:27.664 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000221902 2023-11-28T18:26:27.664 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.664 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2023-11-28T18:26:27.664 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 22, 2023-11-28T18:26:27.664 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000018213, 2023-11-28T18:26:27.664 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000827 2023-11-28T18:26:27.664 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.664 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 115120, 2023-11-28T18:26:27.664 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 12064 2023-11-28T18:26:27.665 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.665 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2023-11-28T18:26:27.665 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2023-11-28T18:26:27.665 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2023-11-28T18:26:27.665 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.665 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2023-11-28T18:26:27.665 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2023-11-28T18:26:27.665 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2023-11-28T18:26:27.665 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 56, 2023-11-28T18:26:27.665 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.002149916, 2023-11-28T18:26:27.665 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000038391 2023-11-28T18:26:27.666 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.666 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.666 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2023-11-28T18:26:27.666 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2023-11-28T18:26:27.666 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2023-11-28T18:26:27.666 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2, 2023-11-28T18:26:27.666 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000015027, 2023-11-28T18:26:27.666 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000007513 2023-11-28T18:26:27.666 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.667 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.667 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2023-11-28T18:26:27.667 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 0, 2023-11-28T18:26:27.667 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 0, 2023-11-28T18:26:27.667 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2023-11-28T18:26:27.667 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.667 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.667 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.667 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.667 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2023-11-28T18:26:27.668 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2023-11-28T18:26:27.668 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2023-11-28T18:26:27.668 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2023-11-28T18:26:27.668 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 11, 2023-11-28T18:26:27.668 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2023-11-28T18:26:27.668 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2023-11-28T18:26:27.668 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 75, 2023-11-28T18:26:27.668 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 0, 2023-11-28T18:26:27.668 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 75, 2023-11-28T18:26:27.668 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2023-11-28T18:26:27.668 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 75, 2023-11-28T18:26:27.668 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2023-11-28T18:26:27.669 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 64, 2023-11-28T18:26:27.669 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 64, 2023-11-28T18:26:27.669 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 5, 2023-11-28T18:26:27.669 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 3, 2023-11-28T18:26:27.669 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 0, 2023-11-28T18:26:27.669 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2023-11-28T18:26:27.669 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 3, 2023-11-28T18:26:27.669 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2023-11-28T18:26:27.669 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2023-11-28T18:26:27.669 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2023-11-28T18:26:27.670 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2023-11-28T18:26:27.670 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2023-11-28T18:26:27.670 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2023-11-28T18:26:27.670 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2023-11-28T18:26:27.670 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 3, 2023-11-28T18:26:27.670 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 64, 2023-11-28T18:26:27.670 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2023-11-28T18:26:27.670 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2023-11-28T18:26:27.670 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2023-11-28T18:26:27.670 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 99, 2023-11-28T18:26:27.671 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 50, 2023-11-28T18:26:27.671 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2023-11-28T18:26:27.671 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2023-11-28T18:26:27.671 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2023-11-28T18:26:27.671 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2023-11-28T18:26:27.671 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2023-11-28T18:26:27.671 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2023-11-28T18:26:27.671 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2023-11-28T18:26:27.671 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2023-11-28T18:26:27.671 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2023-11-28T18:26:27.672 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2023-11-28T18:26:27.672 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 0, 2023-11-28T18:26:27.672 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 0, 2023-11-28T18:26:27.672 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2023-11-28T18:26:27.672 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-28T18:26:27.672 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-28T18:26:27.672 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 0, 2023-11-28T18:26:27.672 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 0, 2023-11-28T18:26:27.673 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 0, 2023-11-28T18:26:27.673 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 0 2023-11-28T18:26:27.673 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.673 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2023-11-28T18:26:27.673 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2023-11-28T18:26:27.673 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2023-11-28T18:26:27.673 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2023-11-28T18:26:27.673 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 4, 2023-11-28T18:26:27.673 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 0, 2023-11-28T18:26:27.673 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2023-11-28T18:26:27.674 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2023-11-28T18:26:27.674 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2023-11-28T18:26:27.674 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 0, 2023-11-28T18:26:27.674 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 0, 2023-11-28T18:26:27.674 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2023-11-28T18:26:27.674 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2023-11-28T18:26:27.674 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2023-11-28T18:26:27.674 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2023-11-28T18:26:27.674 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2023-11-28T18:26:27.675 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2023-11-28T18:26:27.675 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2023-11-28T18:26:27.675 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2023-11-28T18:26:27.675 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2023-11-28T18:26:27.675 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2023-11-28T18:26:27.675 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2023-11-28T18:26:27.675 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2023-11-28T18:26:27.675 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2023-11-28T18:26:27.675 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.675 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2023-11-28T18:26:27.675 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2023-11-28T18:26:27.676 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 8, 2023-11-28T18:26:27.676 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2023-11-28T18:26:27.676 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2023-11-28T18:26:27.676 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 8, 2023-11-28T18:26:27.676 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2023-11-28T18:26:27.676 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2023-11-28T18:26:27.676 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 2, 2023-11-28T18:26:27.676 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2023-11-28T18:26:27.676 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2023-11-28T18:26:27.676 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 2, 2023-11-28T18:26:27.676 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2023-11-28T18:26:27.677 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2023-11-28T18:26:27.677 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2023-11-28T18:26:27.677 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2023-11-28T18:26:27.677 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 4243671, 2023-11-28T18:26:27.677 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 0, 2023-11-28T18:26:27.677 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2023-11-28T18:26:27.677 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 5, 2023-11-28T18:26:27.677 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.011311772, 2023-11-28T18:26:27.678 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.002262354 2023-11-28T18:26:27.678 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.678 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 0 2023-11-28T18:26:27.678 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.678 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2023-11-28T18:26:27.678 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 78, 2023-11-28T18:26:27.678 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 78, 2023-11-28T18:26:27.678 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 0, 2023-11-28T18:26:27.678 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 47, 2023-11-28T18:26:27.678 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 48, 2023-11-28T18:26:27.679 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 1, 2023-11-28T18:26:27.679 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 75, 2023-11-28T18:26:27.679 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 75, 2023-11-28T18:26:27.679 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 0, 2023-11-28T18:26:27.679 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 64, 2023-11-28T18:26:27.679 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 64, 2023-11-28T18:26:27.679 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 0, 2023-11-28T18:26:27.679 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 44612, 2023-11-28T18:26:27.680 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 198952 2023-11-28T18:26:27.680 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.680 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2023-11-28T18:26:27.680 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 0, 2023-11-28T18:26:27.680 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2023-11-28T18:26:27.680 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 0, 2023-11-28T18:26:27.680 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 2, 2023-11-28T18:26:27.680 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2023-11-28T18:26:27.680 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2023-11-28T18:26:27.681 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.681 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.681 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.681 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.681 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2023-11-28T18:26:27.681 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.681 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.681 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.681 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.681 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2023-11-28T18:26:27.682 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.682 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.682 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.682 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.682 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2023-11-28T18:26:27.682 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.682 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.682 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.682 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.683 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2023-11-28T18:26:27.683 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.683 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.683 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.683 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.683 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2023-11-28T18:26:27.683 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.683 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.683 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.683 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.684 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2023-11-28T18:26:27.684 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.684 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.684 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.684 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.684 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2023-11-28T18:26:27.684 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.684 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.684 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.685 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.685 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2023-11-28T18:26:27.685 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.685 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.685 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.685 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.685 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2023-11-28T18:26:27.685 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.685 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.686 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.686 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.686 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2023-11-28T18:26:27.686 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.686 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.686 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.686 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.686 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2023-11-28T18:26:27.686 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.687 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.687 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.687 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.687 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2023-11-28T18:26:27.687 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.687 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.687 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.687 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.687 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2023-11-28T18:26:27.687 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.688 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.688 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.688 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.688 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2023-11-28T18:26:27.688 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.688 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.688 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.688 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.688 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2023-11-28T18:26:27.688 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.688 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.689 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.689 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2023-11-28T18:26:27.689 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.689 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.689 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.689 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2023-11-28T18:26:27.689 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.689 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.689 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.689 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2023-11-28T18:26:27.690 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.690 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.690 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.690 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.690 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2023-11-28T18:26:27.690 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.690 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.690 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.690 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.690 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2023-11-28T18:26:27.690 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.690 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.690 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.691 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.691 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2023-11-28T18:26:27.691 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.691 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.691 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.691 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.691 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2023-11-28T18:26:27.691 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.691 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.691 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.691 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.691 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2023-11-28T18:26:27.691 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.692 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.692 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.692 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.692 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2023-11-28T18:26:27.692 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.692 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.692 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.692 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.692 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2023-11-28T18:26:27.692 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.692 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.692 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.692 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.693 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2023-11-28T18:26:27.693 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.693 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.693 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.693 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.693 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2023-11-28T18:26:27.693 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.693 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.693 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.693 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.693 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2023-11-28T18:26:27.693 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.694 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.694 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.694 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.694 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2023-11-28T18:26:27.694 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2023-11-28T18:26:27.694 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2023-11-28T18:26:27.694 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.694 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.694 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.694 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.694 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.695 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2023-11-28T18:26:27.695 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 1, 2023-11-28T18:26:27.695 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 1, 2023-11-28T18:26:27.695 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 0, 2023-11-28T18:26:27.695 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 1, 2023-11-28T18:26:27.695 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2023-11-28T18:26:27.695 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 0, 2023-11-28T18:26:27.695 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 0, 2023-11-28T18:26:27.695 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 984, 2023-11-28T18:26:27.695 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2023-11-28T18:26:27.695 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.695 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2023-11-28T18:26:27.696 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2023-11-28T18:26:27.696 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2023-11-28T18:26:27.696 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2023-11-28T18:26:27.696 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2023-11-28T18:26:27.696 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2023-11-28T18:26:27.696 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2023-11-28T18:26:27.696 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-28T18:26:27.696 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2023-11-28T18:26:27.696 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-28T18:26:27.696 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2023-11-28T18:26:27.696 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2023-11-28T18:26:27.696 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2023-11-28T18:26:27.696 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-28T18:26:27.697 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2023-11-28T18:26:27.697 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2023-11-28T18:26:27.697 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2023-11-28T18:26:27.697 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2023-11-28T18:26:27.697 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2023-11-28T18:26:27.697 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-28T18:26:27.697 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2023-11-28T18:26:27.697 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-28T18:26:27.697 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2023-11-28T18:26:27.697 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2023-11-28T18:26:27.697 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2023-11-28T18:26:27.697 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2023-11-28T18:26:27.698 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2023-11-28T18:26:27.698 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-28T18:26:27.698 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2023-11-28T18:26:27.698 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2023-11-28T18:26:27.698 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2023-11-28T18:26:27.698 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2023-11-28T18:26:27.698 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2023-11-28T18:26:27.698 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2023-11-28T18:26:27.698 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2023-11-28T18:26:27.699 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2023-11-28T18:26:27.699 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2023-11-28T18:26:27.699 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 245209, 2023-11-28T18:26:27.699 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 71, 2023-11-28T18:26:27.699 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2023-11-28T18:26:27.699 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2023-11-28T18:26:27.699 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2023-11-28T18:26:27.699 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2023-11-28T18:26:27.699 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2023-11-28T18:26:27.699 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2023-11-28T18:26:27.699 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2023-11-28T18:26:27.700 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2023-11-28T18:26:27.700 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 5616, 2023-11-28T18:26:27.700 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 80, 2023-11-28T18:26:27.700 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2023-11-28T18:26:27.700 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2023-11-28T18:26:27.700 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2023-11-28T18:26:27.700 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2023-11-28T18:26:27.700 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 304224, 2023-11-28T18:26:27.700 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 1436, 2023-11-28T18:26:27.700 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2023-11-28T18:26:27.700 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2023-11-28T18:26:27.700 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2023-11-28T18:26:27.701 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2023-11-28T18:26:27.701 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.701 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2023-11-28T18:26:27.701 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2023-11-28T18:26:27.701 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2023-11-28T18:26:27.701 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 41, 2023-11-28T18:26:27.701 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 61657, 2023-11-28T18:26:27.701 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 27, 2023-11-28T18:26:27.701 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 41, 2023-11-28T18:26:27.701 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2023-11-28T18:26:27.701 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 41, 2023-11-28T18:26:27.701 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.287995916, 2023-11-28T18:26:27.701 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.007024290 2023-11-28T18:26:27.702 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.702 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2023-11-28T18:26:27.702 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2023-11-28T18:26:27.702 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 41, 2023-11-28T18:26:27.702 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 64 2023-11-28T18:26:27.702 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.702 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 41, 2023-11-28T18:26:27.702 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 0, 2023-11-28T18:26:27.702 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 41, 2023-11-28T18:26:27.702 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2023-11-28T18:26:27.702 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2023-11-28T18:26:27.703 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 0, 2023-11-28T18:26:27.703 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 10, 2023-11-28T18:26:27.703 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 0, 2023-11-28T18:26:27.703 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 5, 2023-11-28T18:26:27.703 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 6, 2023-11-28T18:26:27.703 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2023-11-28T18:26:27.703 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2023-11-28T18:26:27.703 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2023-11-28T18:26:27.703 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2023-11-28T18:26:27.703 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 6, 2023-11-28T18:26:27.703 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2023-11-28T18:26:27.704 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2023-11-28T18:26:27.704 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2023-11-28T18:26:27.704 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 0, 2023-11-28T18:26:27.704 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 20, 2023-11-28T18:26:27.704 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2023-11-28T18:26:27.704 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2023-11-28T18:26:27.704 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2023-11-28T18:26:27.704 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2023-11-28T18:26:27.704 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2023-11-28T18:26:27.704 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2023-11-28T18:26:27.704 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2023-11-28T18:26:27.704 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2023-11-28T18:26:27.705 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2023-11-28T18:26:27.705 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2023-11-28T18:26:27.705 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2023-11-28T18:26:27.705 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2023-11-28T18:26:27.705 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2023-11-28T18:26:27.705 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2023-11-28T18:26:27.705 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2023-11-28T18:26:27.705 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2023-11-28T18:26:27.705 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2023-11-28T18:26:27.705 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2023-11-28T18:26:27.705 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2023-11-28T18:26:27.706 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2023-11-28T18:26:27.706 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2023-11-28T18:26:27.706 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2023-11-28T18:26:27.706 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2023-11-28T18:26:27.706 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2023-11-28T18:26:27.706 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2023-11-28T18:26:27.706 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2023-11-28T18:26:27.706 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 53, 2023-11-28T18:26:27.706 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2023-11-28T18:26:27.706 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 2, 2023-11-28T18:26:27.706 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 8, 2023-11-28T18:26:27.706 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 8, 2023-11-28T18:26:27.707 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 0, 2023-11-28T18:26:27.707 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2023-11-28T18:26:27.707 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 16, 2023-11-28T18:26:27.707 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 0, 2023-11-28T18:26:27.707 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 1 2023-11-28T18:26:27.707 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.707 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2023-11-28T18:26:27.707 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 1, 2023-11-28T18:26:27.707 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2023-11-28T18:26:27.707 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2023-11-28T18:26:27.707 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2023-11-28T18:26:27.708 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.708 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2023-11-28T18:26:27.708 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2023-11-28T18:26:27.708 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.708 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2023-11-28T18:26:27.708 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2023-11-28T18:26:27.708 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 0, 2023-11-28T18:26:27.708 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2023-11-28T18:26:27.708 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 0, 2023-11-28T18:26:27.708 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 0, 2023-11-28T18:26:27.708 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2023-11-28T18:26:27.708 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.709 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-28T18:26:27.709 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:27.709 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2023-11-28T18:26:27.709 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:27.709 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 139, 2023-11-28T18:26:27.709 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 206945, 2023-11-28T18:26:27.709 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:27.709 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 139, 2023-11-28T18:26:27.709 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2023-11-28T18:26:27.709 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2023-11-28T18:26:27.709 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 139, 2023-11-28T18:26:27.709 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 206945, 2023-11-28T18:26:27.710 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:27.710 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.710 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.710 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.710 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.710 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.710 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2023-11-28T18:26:27.710 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:27.710 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2023-11-28T18:26:27.710 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:27.710 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2023-11-28T18:26:27.710 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2023-11-28T18:26:27.711 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:27.711 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2023-11-28T18:26:27.711 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 41, 2023-11-28T18:26:27.711 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 61657, 2023-11-28T18:26:27.711 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 41, 2023-11-28T18:26:27.711 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 61657, 2023-11-28T18:26:27.711 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:27.711 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.711 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.711 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.711 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.711 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.712 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2023-11-28T18:26:27.712 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:27.712 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2023-11-28T18:26:27.712 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:27.712 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2023-11-28T18:26:27.712 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2023-11-28T18:26:27.712 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:27.712 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2023-11-28T18:26:27.712 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 41, 2023-11-28T18:26:27.712 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 41, 2023-11-28T18:26:27.713 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 41, 2023-11-28T18:26:27.713 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 41, 2023-11-28T18:26:27.713 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:27.713 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.713 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.713 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.713 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.713 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.713 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2023-11-28T18:26:27.713 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:27.713 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2023-11-28T18:26:27.713 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:27.714 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2023-11-28T18:26:27.714 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2023-11-28T18:26:27.714 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:27.714 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2023-11-28T18:26:27.714 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2023-11-28T18:26:27.714 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2023-11-28T18:26:27.714 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2023-11-28T18:26:27.714 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2023-11-28T18:26:27.714 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:27.714 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.714 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.714 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.715 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.715 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.715 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x5608dbe66a00": { 2023-11-28T18:26:27.715 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:27.715 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2023-11-28T18:26:27.715 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:27.715 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 8, 2023-11-28T18:26:27.715 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 49367, 2023-11-28T18:26:27.715 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:27.715 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 8, 2023-11-28T18:26:27.715 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2023-11-28T18:26:27.715 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2023-11-28T18:26:27.716 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 5, 2023-11-28T18:26:27.716 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 49367, 2023-11-28T18:26:27.716 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:27.716 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.716 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.716 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.716 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.716 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.716 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:26:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f754df2a640 1 -- 172.21.15.105:0/2624752275 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f75340668d0 msgr2=0x7f7534068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f754df2a640 1 --2- 172.21.15.105:0/2624752275 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f75340668d0 0x7f7534068d90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f75380011f0 tx=0x7f7538007d40 comp rx=0 tx=0).stop 2023-11-28T18:26:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f754df2a640 1 -- 172.21.15.105:0/2624752275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75480ae290 msgr2=0x7f7548148400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f754df2a640 1 --2- 172.21.15.105:0/2624752275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75480ae290 0x7f7548148400 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f753c00c500 tx=0x7f753c00c9c0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f754df2a640 1 -- 172.21.15.105:0/2624752275 shutdown_connections 2023-11-28T18:26:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f754df2a640 1 --2- 172.21.15.105:0/2624752275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7548154070 0x7f7548160c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f754df2a640 1 --2- 172.21.15.105:0/2624752275 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f75340668d0 0x7f7534068d90 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f754df2a640 1 --2- 172.21.15.105:0/2624752275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75480ae290 0x7f7548148400 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f754df2a640 1 --2- 172.21.15.105:0/2624752275 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f754806bf50 0x7f754814b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f754df2a640 1 -- 172.21.15.105:0/2624752275 >> 172.21.15.105:0/2624752275 conn(0x7f75480ceb90 msgr2=0x7f7548072f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:27.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f754df2a640 1 -- 172.21.15.105:0/2624752275 shutdown_connections 2023-11-28T18:26:27.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.648+0000 7f754df2a640 1 -- 172.21.15.105:0/2624752275 wait complete. 2023-11-28T18:26:27.719 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 perf dump 2023-11-28T18:26:27.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.856+0000 7fb34b479640 1 Processor -- start 2023-11-28T18:26:27.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.856+0000 7fb34b479640 1 -- start start 2023-11-28T18:26:27.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.856+0000 7fb34b479640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb344071960 0x7fb34415f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.856+0000 7fb34b479640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3440701d0 con 0x7fb3440b4290 2023-11-28T18:26:27.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.856+0000 7fb34b479640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb34406df80 con 0x7fb34415b030 2023-11-28T18:26:27.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.856+0000 7fb34b479640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb344071df0 con 0x7fb344071960 2023-11-28T18:26:27.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.856+0000 7fb349215640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb3440b4290 0x7fb3440b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35238/0 (socket says 172.21.15.105:35238) 2023-11-28T18:26:27.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.856+0000 7fb349215640 1 -- 172.21.15.105:0/1210911905 learned_addr learned my addr 172.21.15.105:0/1210911905 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:27.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.856+0000 7fb348a14640 1 --2- 172.21.15.105:0/1210911905 >> v2:172.21.15.105:3300/0 conn(0x7fb344071960 0x7fb34415f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.856+0000 7fb33bfff640 1 -- 172.21.15.105:0/1210911905 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1497044718 0 0) 0x7fb3440701d0 con 0x7fb3440b4290 2023-11-28T18:26:27.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.856+0000 7fb33bfff640 1 -- 172.21.15.105:0/1210911905 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb330003530 con 0x7fb3440b4290 2023-11-28T18:26:27.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.856+0000 7fb348a14640 1 -- 172.21.15.105:0/1210911905 >> v1:172.21.15.148:6789/0 conn(0x7fb34415b030 legacy=0x7fb3440709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.856+0000 7fb348a14640 1 -- 172.21.15.105:0/1210911905 >> v1:172.21.15.105:6789/0 conn(0x7fb3440b4290 legacy=0x7fb3440b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.856+0000 7fb348a14640 1 -- 172.21.15.105:0/1210911905 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3440720d0 con 0x7fb344071960 2023-11-28T18:26:27.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.860+0000 7fb348a14640 1 --2- 172.21.15.105:0/1210911905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb344071960 0x7fb34415f3b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb3340097c0 tx=0x7fb33401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ef3227a5cfb64a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:27.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.860+0000 7fb33bfff640 1 -- 172.21.15.105:0/1210911905 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3340ab650 con 0x7fb344071960 2023-11-28T18:26:27.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.860+0000 7fb33bfff640 1 -- 172.21.15.105:0/1210911905 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3340ab7b0 con 0x7fb344071960 2023-11-28T18:26:27.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.860+0000 7fb33bfff640 1 -- 172.21.15.105:0/1210911905 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3340abb40 con 0x7fb344071960 2023-11-28T18:26:27.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.860+0000 7fb34b479640 1 -- 172.21.15.105:0/1210911905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb344071960 msgr2=0x7fb34415f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.860+0000 7fb34b479640 1 --2- 172.21.15.105:0/1210911905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb344071960 0x7fb34415f3b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb3340097c0 tx=0x7fb33401c450 comp rx=0 tx=0).stop 2023-11-28T18:26:27.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.860+0000 7fb34b479640 1 -- 172.21.15.105:0/1210911905 shutdown_connections 2023-11-28T18:26:27.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.860+0000 7fb34b479640 1 --2- 172.21.15.105:0/1210911905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb344071960 0x7fb34415f3b0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.860+0000 7fb34b479640 1 -- 172.21.15.105:0/1210911905 >> 172.21.15.105:0/1210911905 conn(0x7fb3440ceb90 msgr2=0x7fb3440cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:27.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.860+0000 7fb34b479640 1 -- 172.21.15.105:0/1210911905 shutdown_connections 2023-11-28T18:26:27.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.860+0000 7fb34b479640 1 -- 172.21.15.105:0/1210911905 wait complete. 2023-11-28T18:26:27.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb34b479640 1 Processor -- start 2023-11-28T18:26:27.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb34b479640 1 -- start start 2023-11-28T18:26:27.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb34b479640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3440b4290 0x7fb344159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb34b479640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb34415b030 0x7fb3441656e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb34b479640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb344164140 0x7fb344153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb34b479640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb344165250 con 0x7fb344164140 2023-11-28T18:26:27.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb34b479640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb344164640 con 0x7fb3440b4290 2023-11-28T18:26:27.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb34b479640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb344164920 con 0x7fb34415b030 2023-11-28T18:26:27.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb349a16640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb344164140 0x7fb344153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb349a16640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb344164140 0x7fb344153900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35720/0 (socket says 172.21.15.105:35720) 2023-11-28T18:26:27.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb349a16640 1 -- 172.21.15.105:0/2801300688 learned_addr learned my addr 172.21.15.105:0/2801300688 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:27.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb348a14640 1 --2- 172.21.15.105:0/2801300688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb34415b030 0x7fb3441656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb349215640 1 --2- 172.21.15.105:0/2801300688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3440b4290 0x7fb344159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb349a16640 1 -- 172.21.15.105:0/2801300688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb34415b030 msgr2=0x7fb3441656e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb349a16640 1 --2- 172.21.15.105:0/2801300688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb34415b030 0x7fb3441656e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb349a16640 1 -- 172.21.15.105:0/2801300688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3440b4290 msgr2=0x7fb344159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb349a16640 1 --2- 172.21.15.105:0/2801300688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3440b4290 0x7fb344159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb349a16640 1 -- 172.21.15.105:0/2801300688 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb334009420 con 0x7fb344164140 2023-11-28T18:26:27.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb349a16640 1 --2- 172.21.15.105:0/2801300688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb344164140 0x7fb344153900 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00c730 tx=0x7fb33c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:27.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb339ffb640 1 -- 172.21.15.105:0/2801300688 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c003b80 con 0x7fb344164140 2023-11-28T18:26:27.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb339ffb640 1 -- 172.21.15.105:0/2801300688 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33c003ce0 con 0x7fb344164140 2023-11-28T18:26:27.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb339ffb640 1 -- 172.21.15.105:0/2801300688 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c005310 con 0x7fb344164140 2023-11-28T18:26:27.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb34b479640 1 -- 172.21.15.105:0/2801300688 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb344164c00 con 0x7fb344164140 2023-11-28T18:26:27.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.864+0000 7fb34b479640 1 -- 172.21.15.105:0/2801300688 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb344165c20 con 0x7fb344164140 2023-11-28T18:26:27.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb339ffb640 1 -- 172.21.15.105:0/2801300688 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fb33c005680 con 0x7fb344164140 2023-11-28T18:26:27.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb339ffb640 1 --2- 172.21.15.105:0/2801300688 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb328066900 0x7fb328068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb339ffb640 1 -- 172.21.15.105:0/2801300688 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fb33c084460 con 0x7fb344164140 2023-11-28T18:26:27.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb349215640 1 --2- 172.21.15.105:0/2801300688 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb328066900 0x7fb328068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb34e7f4000 1 Processor -- start 2023-11-28T18:26:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb349215640 1 --2- 172.21.15.105:0/2801300688 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb328066900 0x7fb328068dc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb3400011f0 tx=0x7fb340007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb34e7f4000 1 -- start start 2023-11-28T18:26:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb34e7f4000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c4680537c0 0x55c468053ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb34e7f4000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4680540e0 0x55c468008fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb34e7f4000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c468009520 0x55c468009900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb34e7f4000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c467e609e0 con 0x55c4680540e0 2023-11-28T18:26:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb34e7f4000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c467e5df10 con 0x55c468009520 2023-11-28T18:26:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb34e7f4000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c467e59800 con 0x55c4680537c0 2023-11-28T18:26:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb348a14640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4680540e0 0x55c468008fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb348a14640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4680540e0 0x55c468008fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35734/0 (socket says 172.21.15.105:35734) 2023-11-28T18:26:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb348a14640 1 -- 172.21.15.105:0/2439851103 learned_addr learned my addr 172.21.15.105:0/2439851103 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb349215640 1 --2- 172.21.15.105:0/2439851103 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c4680537c0 0x55c468053ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb349a16640 1 --2- 172.21.15.105:0/2439851103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c468009520 0x55c468009900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb348a14640 1 -- 172.21.15.105:0/2439851103 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c4680537c0 msgr2=0x55c468053ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb348a14640 1 --2- 172.21.15.105:0/2439851103 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c4680537c0 0x55c468053ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb348a14640 1 -- 172.21.15.105:0/2439851103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c468009520 msgr2=0x55c468009900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb348a14640 1 --2- 172.21.15.105:0/2439851103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c468009520 0x55c468009900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb348a14640 1 -- 172.21.15.105:0/2439851103 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c467b51dc0 con 0x55c4680540e0 2023-11-28T18:26:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb349215640 1 --2- 172.21.15.105:0/2439851103 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c4680537c0 0x55c468053ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb349a16640 1 --2- 172.21.15.105:0/2439851103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c468009520 0x55c468009900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.868+0000 7fb348a14640 1 --2- 172.21.15.105:0/2439851103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4680540e0 0x55c468008fe0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb334019060 tx=0x7fb334004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de156ee529f4f322 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb326ffd640 1 -- 172.21.15.105:0/2439851103 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3340ab920 con 0x55c4680540e0 2023-11-28T18:26:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb326ffd640 1 -- 172.21.15.105:0/2439851103 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3340aba80 con 0x55c4680540e0 2023-11-28T18:26:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb326ffd640 1 -- 172.21.15.105:0/2439851103 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3340abdf0 con 0x55c4680540e0 2023-11-28T18:26:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb34e7f4000 1 -- 172.21.15.105:0/2439851103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4680540e0 msgr2=0x55c468008fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb34e7f4000 1 --2- 172.21.15.105:0/2439851103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4680540e0 0x55c468008fe0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb334019060 tx=0x7fb334004740 comp rx=0 tx=0).stop 2023-11-28T18:26:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb34e7f4000 1 -- 172.21.15.105:0/2439851103 shutdown_connections 2023-11-28T18:26:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb34e7f4000 1 --2- 172.21.15.105:0/2439851103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c468009520 0x55c468009900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb34e7f4000 1 --2- 172.21.15.105:0/2439851103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4680540e0 0x55c468008fe0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb34e7f4000 1 --2- 172.21.15.105:0/2439851103 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c4680537c0 0x55c468053ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb34e7f4000 1 -- 172.21.15.105:0/2439851103 >> 172.21.15.105:0/2439851103 conn(0x55c468011fb0 msgr2=0x55c468036360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb34e7f4000 1 -- 172.21.15.105:0/2439851103 shutdown_connections 2023-11-28T18:26:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb34e7f4000 1 -- 172.21.15.105:0/2439851103 wait complete. 2023-11-28T18:26:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb34e7f4000 1 -- start start 2023-11-28T18:26:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb34e7f4000 1 Processor -- start 2023-11-28T18:26:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb34e7f4000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c468009520 0x55c468010a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb34e7f4000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c4680537c0 0x55c468037590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb34e7f4000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4680540e0 0x55c468037ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb34e7f4000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c467e17a70 con 0x55c4680540e0 2023-11-28T18:26:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb34e7f4000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c467e17710 con 0x55c468009520 2023-11-28T18:26:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb34e7f4000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c467b12a40 con 0x55c4680537c0 2023-11-28T18:26:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb34e7f4000 10 client.0 _ll_register_callbacks cb 0x55c467ff4ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb349a16640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4680540e0 0x55c468037ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb349a16640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4680540e0 0x55c468037ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35738/0 (socket says 172.21.15.105:35738) 2023-11-28T18:26:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb349a16640 1 -- 172.21.15.105:0/1124006342 learned_addr learned my addr 172.21.15.105:0/1124006342 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb349215640 1 --2- 172.21.15.105:0/1124006342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c468009520 0x55c468010a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb348a14640 1 --2- 172.21.15.105:0/1124006342 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c4680537c0 0x55c468037590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb349a16640 1 -- 172.21.15.105:0/1124006342 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c4680537c0 msgr2=0x55c468037590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:27.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb349a16640 1 --2- 172.21.15.105:0/1124006342 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c4680537c0 0x55c468037590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb349a16640 1 -- 172.21.15.105:0/1124006342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c468009520 msgr2=0x55c468010a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:27.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb349a16640 1 --2- 172.21.15.105:0/1124006342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c468009520 0x55c468010a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb349a16640 1 -- 172.21.15.105:0/1124006342 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3440720d0 con 0x55c4680540e0 2023-11-28T18:26:27.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb348a14640 1 --2- 172.21.15.105:0/1124006342 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c4680537c0 0x55c468037590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:27.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb349a16640 1 --2- 172.21.15.105:0/1124006342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4680540e0 0x55c468037ad0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00c700 tx=0x7fb33c0552b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8db46e404aceae5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:27.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb31ffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:27.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.872+0000 7fb31ffff640 1 -- 172.21.15.105:0/1124006342 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c06c070 con 0x55c4680540e0 2023-11-28T18:26:27.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb31ffff640 1 -- 172.21.15.105:0/1124006342 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fb33c059070 con 0x55c4680540e0 2023-11-28T18:26:27.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb34e7f4000 1 -- 172.21.15.105:0/1124006342 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c467e12ab0 con 0x55c4680540e0 2023-11-28T18:26:27.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb31ffff640 1 -- 172.21.15.105:0/1124006342 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb33c084070 con 0x55c4680540e0 2023-11-28T18:26:27.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb34e7f4000 10 client.6327 fetch_fsmap learned FSMap version 41 2023-11-28T18:26:27.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb34e7f4000 1 -- 172.21.15.105:0/1124006342 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x55c467e1a260 con 0x55c4680540e0 2023-11-28T18:26:27.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb31ffff640 1 -- 172.21.15.105:0/1124006342 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c467e12ab0 con 0x55c4680540e0 2023-11-28T18:26:27.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb31ffff640 1 -- 172.21.15.105:0/1124006342 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) v1 ==== 3326+0+0 (secure 0 0 0) 0x7fb33c019020 con 0x55c4680540e0 2023-11-28T18:26:27.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb34e7f4000 10 client.6327 fetch_fsmap finished waiting for FSMap version 41 2023-11-28T18:26:27.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb34e7f4000 10 client.6327 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2023-11-28T18:26:27.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb34e7f4000 20 client.6327 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:27.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb34e7f4000 1 --2- 172.21.15.105:0/1124006342 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55c468059e00 0x55c46805a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb34e7f4000 4 client.6327 mds_command: new command op to 5842 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:26:27.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb34e7f4000 1 -- 172.21.15.105:0/1124006342 --> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c467e0e850 con 0x55c468059e00 2023-11-28T18:26:27.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb349215640 1 --2- 172.21.15.105:0/1124006342 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55c468059e00 0x55c46805a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb349215640 1 --2- 172.21.15.105:0/1124006342 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55c468059e00 0x55c46805a210 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=607ac0a0340f3975 server_cookie=3ac65539faf2105a in_seq=0 out_seq=0 2023-11-28T18:26:27.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb31ffff640 10 client.6327 ms_handle_connect on v2:172.21.15.148:6832/2696773413 2023-11-28T18:26:27.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb31ffff640 1 -- 172.21.15.105:0/1124006342 <== mds.0 v2:172.21.15.148:6832/2696773413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c467e0e850 con 0x55c468059e00 2023-11-28T18:26:27.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb31ffff640 10 client.6327 handle_command_reply: tid=0 2023-11-28T18:26:27.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb34e7f4000 1 client.6327 shutdown 2023-11-28T18:26:27.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb34e7f4000 1 -- 172.21.15.105:0/1124006342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4680540e0 msgr2=0x55c468037ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb34e7f4000 1 --2- 172.21.15.105:0/1124006342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4680540e0 0x55c468037ad0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00c700 tx=0x7fb33c0552b0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb34e7f4000 1 -- 172.21.15.105:0/1124006342 shutdown_connections 2023-11-28T18:26:27.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb34e7f4000 1 --2- 172.21.15.105:0/1124006342 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55c468059e00 0x55c46805a210 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb34e7f4000 1 --2- 172.21.15.105:0/1124006342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4680540e0 0x55c468037ad0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb34e7f4000 1 --2- 172.21.15.105:0/1124006342 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c4680537c0 0x55c468037590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb34e7f4000 1 --2- 172.21.15.105:0/1124006342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c468009520 0x55c468010a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb34e7f4000 1 -- 172.21.15.105:0/1124006342 >> 172.21.15.105:0/1124006342 conn(0x55c468011fb0 msgr2=0x55c4680086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:27.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb31ffff640 0 client.6327 ms_handle_reset on v2:172.21.15.148:6832/2696773413 2023-11-28T18:26:27.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb34e7f4000 1 -- 172.21.15.105:0/1124006342 shutdown_connections 2023-11-28T18:26:27.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb34e7f4000 1 -- 172.21.15.105:0/1124006342 wait complete. 2023-11-28T18:26:27.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.876+0000 7fb34e7f4000 20 client.6327 trim_cache size 0 max 16384 2023-11-28T18:26:27.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb34e7f4000 1 Processor -- start 2023-11-28T18:26:27.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb34e7f4000 1 -- start start 2023-11-28T18:26:27.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb34e7f4000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c468009520 0x55c4680afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb34e7f4000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4680537c0 0x55c4680b0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb34e7f4000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c4680540e0 0x55c4680b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb34e7f4000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c467e57010 con 0x55c468009520 2023-11-28T18:26:27.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb34e7f4000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c467dddc20 con 0x55c4680537c0 2023-11-28T18:26:27.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb34e7f4000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c467e20710 con 0x55c4680540e0 2023-11-28T18:26:27.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb349215640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c468009520 0x55c4680afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb349215640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c468009520 0x55c4680afbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35740/0 (socket says 172.21.15.105:35740) 2023-11-28T18:26:27.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb349215640 1 -- 172.21.15.105:0/3473976145 learned_addr learned my addr 172.21.15.105:0/3473976145 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:27.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb349215640 1 -- 172.21.15.105:0/3473976145 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c4680540e0 msgr2=0x55c4680b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:27.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb349215640 1 --2- 172.21.15.105:0/3473976145 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c4680540e0 0x55c4680b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb349215640 1 -- 172.21.15.105:0/3473976145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4680537c0 msgr2=0x55c4680b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:27.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb349215640 1 --2- 172.21.15.105:0/3473976145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4680537c0 0x55c4680b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb349215640 1 -- 172.21.15.105:0/3473976145 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c467e00760 con 0x55c468009520 2023-11-28T18:26:27.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb349215640 1 --2- 172.21.15.105:0/3473976145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c468009520 0x55c4680afbf0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb340006ce0 tx=0x7fb340012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746c744a26dd3b0e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:27.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb3277fe640 1 -- 172.21.15.105:0/3473976145 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3400014f0 con 0x55c468009520 2023-11-28T18:26:27.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb3277fe640 1 -- 172.21.15.105:0/3473976145 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34002b070 con 0x55c468009520 2023-11-28T18:26:27.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb3277fe640 1 -- 172.21.15.105:0/3473976145 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb340001850 con 0x55c468009520 2023-11-28T18:26:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb34e7f4000 1 -- 172.21.15.105:0/3473976145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c468009520 msgr2=0x55c4680afbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb34e7f4000 1 --2- 172.21.15.105:0/3473976145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c468009520 0x55c4680afbf0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb340006ce0 tx=0x7fb340012000 comp rx=0 tx=0).stop 2023-11-28T18:26:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb34e7f4000 1 -- 172.21.15.105:0/3473976145 shutdown_connections 2023-11-28T18:26:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb34e7f4000 1 --2- 172.21.15.105:0/3473976145 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c4680540e0 0x55c4680b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb34e7f4000 1 --2- 172.21.15.105:0/3473976145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4680537c0 0x55c4680b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb34e7f4000 1 --2- 172.21.15.105:0/3473976145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c468009520 0x55c4680afbf0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb34e7f4000 1 -- 172.21.15.105:0/3473976145 >> 172.21.15.105:0/3473976145 conn(0x55c468011fb0 msgr2=0x55c468052440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb34e7f4000 1 -- 172.21.15.105:0/3473976145 shutdown_connections 2023-11-28T18:26:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.892+0000 7fb34e7f4000 1 -- 172.21.15.105:0/3473976145 wait complete. 2023-11-28T18:26:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb34e7f4000 1 -- start start 2023-11-28T18:26:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb34e7f4000 1 Processor -- start 2023-11-28T18:26:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb34e7f4000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c468009520 0x55c468058280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb34e7f4000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4680537c0 0x55c4680703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb34e7f4000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4680540e0 0x55c468070920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb34e7f4000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c467e53b00 con 0x55c4680537c0 2023-11-28T18:26:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb34e7f4000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c467e066e0 con 0x55c4680540e0 2023-11-28T18:26:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb34e7f4000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c46800ec10 con 0x55c468009520 2023-11-28T18:26:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb34e7f4000 10 client.0 _ll_register_callbacks cb 0x55c467e21840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb348a14640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4680537c0 0x55c4680703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb348a14640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4680537c0 0x55c4680703e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35750/0 (socket says 172.21.15.105:35750) 2023-11-28T18:26:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb348a14640 1 -- 172.21.15.105:0/13968608 learned_addr learned my addr 172.21.15.105:0/13968608 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb349215640 1 --2- 172.21.15.105:0/13968608 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c468009520 0x55c468058280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb349a16640 1 --2- 172.21.15.105:0/13968608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4680540e0 0x55c468070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb348a14640 1 -- 172.21.15.105:0/13968608 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c468009520 msgr2=0x55c468058280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:27.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb348a14640 1 --2- 172.21.15.105:0/13968608 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c468009520 0x55c468058280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb348a14640 1 -- 172.21.15.105:0/13968608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4680540e0 msgr2=0x55c468070920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:27.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb348a14640 1 --2- 172.21.15.105:0/13968608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4680540e0 0x55c468070920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb348a14640 1 -- 172.21.15.105:0/13968608 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb330003530 con 0x55c4680537c0 2023-11-28T18:26:27.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb349a16640 1 --2- 172.21.15.105:0/13968608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4680540e0 0x55c468070920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:27.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb348a14640 1 --2- 172.21.15.105:0/13968608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4680537c0 0x55c4680703e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb334019060 tx=0x7fb3340191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7151ed0d2dfde93d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:27.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb31ffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:27.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb31ffff640 1 -- 172.21.15.105:0/13968608 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb334006a30 con 0x55c4680537c0 2023-11-28T18:26:27.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb31ffff640 1 -- 172.21.15.105:0/13968608 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fb3340be400 con 0x55c4680537c0 2023-11-28T18:26:27.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb34e7f4000 1 -- 172.21.15.105:0/13968608 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c467e1c3a0 con 0x55c4680537c0 2023-11-28T18:26:27.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb31ffff640 1 -- 172.21.15.105:0/13968608 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3340059a0 con 0x55c4680537c0 2023-11-28T18:26:27.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb34e7f4000 10 client.6333 fetch_fsmap learned FSMap version 41 2023-11-28T18:26:27.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb34e7f4000 1 -- 172.21.15.105:0/13968608 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x55c467df4040 con 0x55c4680537c0 2023-11-28T18:26:27.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb31ffff640 1 -- 172.21.15.105:0/13968608 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c467e1c3a0 con 0x55c4680537c0 2023-11-28T18:26:27.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb31ffff640 1 -- 172.21.15.105:0/13968608 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) v1 ==== 3326+0+0 (secure 0 0 0) 0x7fb334005b00 con 0x55c4680537c0 2023-11-28T18:26:27.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb34e7f4000 10 client.6333 fetch_fsmap finished waiting for FSMap version 41 2023-11-28T18:26:27.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb34e7f4000 10 client.6333 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2023-11-28T18:26:27.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb34e7f4000 20 client.6333 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:27.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb34e7f4000 1 --2- 172.21.15.105:0/13968608 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55c4680b3c00 0x55c4680b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:27.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb34e7f4000 4 client.6333 mds_command: new command op to 5842 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2023-11-28T18:26:27.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb34e7f4000 1 -- 172.21.15.105:0/13968608 --> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55c467e527e0 con 0x55c4680b3c00 2023-11-28T18:26:27.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.896+0000 7fb349215640 1 --2- 172.21.15.105:0/13968608 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55c4680b3c00 0x55c4680b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:27.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb31ffff640 10 client.6333 ms_handle_connect on v2:172.21.15.148:6832/2696773413 2023-11-28T18:26:27.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb349215640 1 --2- 172.21.15.105:0/13968608 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55c4680b3c00 0x55c4680b3fe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33c42fff5b4656c1 server_cookie=fb3fa665b2901f57 in_seq=0 out_seq=0 2023-11-28T18:26:27.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb31ffff640 1 -- 172.21.15.105:0/13968608 <== mds.0 v2:172.21.15.148:6832/2696773413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17751 (crc 0 0 0) 0x55c467e527e0 con 0x55c4680b3c00 2023-11-28T18:26:27.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb31ffff640 10 client.6333 handle_command_reply: tid=0 2023-11-28T18:26:27.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34e7f4000 1 client.6333 shutdown 2023-11-28T18:26:27.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34e7f4000 1 -- 172.21.15.105:0/13968608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4680537c0 msgr2=0x55c4680703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34e7f4000 1 --2- 172.21.15.105:0/13968608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4680537c0 0x55c4680703e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb334019060 tx=0x7fb3340191f0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34e7f4000 1 -- 172.21.15.105:0/13968608 shutdown_connections 2023-11-28T18:26:27.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34e7f4000 1 --2- 172.21.15.105:0/13968608 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55c4680b3c00 0x55c4680b3fe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34e7f4000 1 --2- 172.21.15.105:0/13968608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4680540e0 0x55c468070920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34e7f4000 1 --2- 172.21.15.105:0/13968608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4680537c0 0x55c4680703e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb31ffff640 0 client.6333 ms_handle_reset on v2:172.21.15.148:6832/2696773413 2023-11-28T18:26:27.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34e7f4000 1 --2- 172.21.15.105:0/13968608 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c468009520 0x55c468058280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34e7f4000 1 -- 172.21.15.105:0/13968608 >> 172.21.15.105:0/13968608 conn(0x55c468011fb0 msgr2=0x55c468036540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:27.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34e7f4000 1 -- 172.21.15.105:0/13968608 shutdown_connections 2023-11-28T18:26:27.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34e7f4000 1 -- 172.21.15.105:0/13968608 wait complete. 2023-11-28T18:26:27.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34e7f4000 20 client.6333 trim_cache size 0 max 16384 2023-11-28T18:26:27.913 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:26:27.913 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2023-11-28T18:26:27.913 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 492, 2023-11-28T18:26:27.913 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 435, 2023-11-28T18:26:27.914 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 128514, 2023-11-28T18:26:27.914 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 1421500, 2023-11-28T18:26:27.914 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 8, 2023-11-28T18:26:27.914 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 3, 2023-11-28T18:26:27.914 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.048718460, 2023-11-28T18:26:27.914 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.015418971, 2023-11-28T18:26:27.914 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.016508777, 2023-11-28T18:26:27.914 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.011963744, 2023-11-28T18:26:27.914 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2023-11-28T18:26:27.915 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 433, 2023-11-28T18:26:27.915 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.088513154, 2023-11-28T18:26:27.915 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000204418 2023-11-28T18:26:27.915 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.915 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2023-11-28T18:26:27.915 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 119, 2023-11-28T18:26:27.915 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000014909, 2023-11-28T18:26:27.915 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000125 2023-11-28T18:26:27.915 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.915 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-28T18:26:27.915 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2023-11-28T18:26:27.916 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.916 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2023-11-28T18:26:27.916 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 206, 2023-11-28T18:26:27.916 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 143, 2023-11-28T18:26:27.916 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 170493, 2023-11-28T18:26:27.916 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 189792, 2023-11-28T18:26:27.916 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 7, 2023-11-28T18:26:27.916 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2023-11-28T18:26:27.916 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.025938910, 2023-11-28T18:26:27.916 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.006893210, 2023-11-28T18:26:27.916 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.019866649, 2023-11-28T18:26:27.917 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.002838019, 2023-11-28T18:26:27.917 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2023-11-28T18:26:27.917 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 143, 2023-11-28T18:26:27.917 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.089759321, 2023-11-28T18:26:27.917 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000627687 2023-11-28T18:26:27.917 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.917 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2023-11-28T18:26:27.917 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 97, 2023-11-28T18:26:27.917 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000011395, 2023-11-28T18:26:27.917 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000117 2023-11-28T18:26:27.918 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.918 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 124144, 2023-11-28T18:26:27.918 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 13024 2023-11-28T18:26:27.918 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.918 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2023-11-28T18:26:27.918 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 1044, 2023-11-28T18:26:27.918 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 1831, 2023-11-28T18:26:27.918 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 346991, 2023-11-28T18:26:27.918 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 881681, 2023-11-28T18:26:27.919 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 13, 2023-11-28T18:26:27.919 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 8, 2023-11-28T18:26:27.919 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.120999260, 2023-11-28T18:26:27.919 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.056556075, 2023-11-28T18:26:27.919 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.073494357, 2023-11-28T18:26:27.919 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.001670068, 2023-11-28T18:26:27.919 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2023-11-28T18:26:27.919 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1829, 2023-11-28T18:26:27.919 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.092061837, 2023-11-28T18:26:27.919 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000050334 2023-11-28T18:26:27.919 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.920 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2023-11-28T18:26:27.920 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2605, 2023-11-28T18:26:27.920 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000557047, 2023-11-28T18:26:27.920 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000213 2023-11-28T18:26:27.920 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.920 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-28T18:26:27.921 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 66416 2023-11-28T18:26:27.921 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.921 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2023-11-28T18:26:27.921 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2023-11-28T18:26:27.921 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2023-11-28T18:26:27.921 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.921 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2023-11-28T18:26:27.921 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2023-11-28T18:26:27.921 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2023-11-28T18:26:27.921 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 662, 2023-11-28T18:26:27.921 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.352933949, 2023-11-28T18:26:27.922 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000533132 2023-11-28T18:26:27.922 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.922 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.922 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2023-11-28T18:26:27.922 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2023-11-28T18:26:27.922 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2023-11-28T18:26:27.923 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 127, 2023-11-28T18:26:27.923 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.008517043, 2023-11-28T18:26:27.923 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000067063 2023-11-28T18:26:27.923 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.923 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.923 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2023-11-28T18:26:27.923 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 705, 2023-11-28T18:26:27.923 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 705, 2023-11-28T18:26:27.923 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2023-11-28T18:26:27.923 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 705, 2023-11-28T18:26:27.923 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 1.087538334, 2023-11-28T18:26:27.923 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001542607 2023-11-28T18:26:27.924 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.924 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2023-11-28T18:26:27.924 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2023-11-28T18:26:27.924 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2023-11-28T18:26:27.924 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2023-11-28T18:26:27.924 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 21, 2023-11-28T18:26:27.924 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 1, 2023-11-28T18:26:27.924 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2023-11-28T18:26:27.924 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 96, 2023-11-28T18:26:27.924 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 0, 2023-11-28T18:26:27.924 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 96, 2023-11-28T18:26:27.925 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2023-11-28T18:26:27.925 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 96, 2023-11-28T18:26:27.925 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 116, 2023-11-28T18:26:27.925 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 86, 2023-11-28T18:26:27.925 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 86, 2023-11-28T18:26:27.925 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 10, 2023-11-28T18:26:27.925 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 1404, 2023-11-28T18:26:27.925 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 553, 2023-11-28T18:26:27.925 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2023-11-28T18:26:27.925 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 0, 2023-11-28T18:26:27.925 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2023-11-28T18:26:27.926 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2023-11-28T18:26:27.926 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2023-11-28T18:26:27.926 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 24320, 2023-11-28T18:26:27.926 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 1, 2023-11-28T18:26:27.926 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 5, 2023-11-28T18:26:27.926 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 116, 2023-11-28T18:26:27.926 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 0, 2023-11-28T18:26:27.926 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 0, 2023-11-28T18:26:27.926 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2023-11-28T18:26:27.926 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2023-11-28T18:26:27.926 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2023-11-28T18:26:27.926 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 99, 2023-11-28T18:26:27.927 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 50, 2023-11-28T18:26:27.927 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2023-11-28T18:26:27.927 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2023-11-28T18:26:27.927 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2023-11-28T18:26:27.927 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2023-11-28T18:26:27.927 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2023-11-28T18:26:27.927 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2023-11-28T18:26:27.927 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2023-11-28T18:26:27.927 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2023-11-28T18:26:27.928 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2023-11-28T18:26:27.928 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2023-11-28T18:26:27.928 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 179, 2023-11-28T18:26:27.928 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 253, 2023-11-28T18:26:27.928 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2023-11-28T18:26:27.928 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-28T18:26:27.928 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-28T18:26:27.928 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 279, 2023-11-28T18:26:27.928 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 176, 2023-11-28T18:26:27.928 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 2, 2023-11-28T18:26:27.928 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 433 2023-11-28T18:26:27.928 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.929 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2023-11-28T18:26:27.929 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2023-11-28T18:26:27.929 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2023-11-28T18:26:27.929 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2023-11-28T18:26:27.929 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 0, 2023-11-28T18:26:27.929 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 7, 2023-11-28T18:26:27.929 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2023-11-28T18:26:27.929 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2023-11-28T18:26:27.929 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2023-11-28T18:26:27.929 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 77, 2023-11-28T18:26:27.929 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 77, 2023-11-28T18:26:27.929 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2023-11-28T18:26:27.930 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2023-11-28T18:26:27.930 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2023-11-28T18:26:27.930 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2023-11-28T18:26:27.930 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2023-11-28T18:26:27.930 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2023-11-28T18:26:27.930 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2023-11-28T18:26:27.930 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2023-11-28T18:26:27.931 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 5, 2023-11-28T18:26:27.931 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2023-11-28T18:26:27.931 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 1, 2023-11-28T18:26:27.931 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2023-11-28T18:26:27.931 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2023-11-28T18:26:27.931 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.931 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2023-11-28T18:26:27.931 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2023-11-28T18:26:27.931 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 625, 2023-11-28T18:26:27.931 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2023-11-28T18:26:27.931 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2023-11-28T18:26:27.932 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 625, 2023-11-28T18:26:27.932 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2023-11-28T18:26:27.932 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2023-11-28T18:26:27.932 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 2, 2023-11-28T18:26:27.932 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2023-11-28T18:26:27.932 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2023-11-28T18:26:27.933 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 2, 2023-11-28T18:26:27.933 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2023-11-28T18:26:27.933 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2023-11-28T18:26:27.933 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2023-11-28T18:26:27.933 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2023-11-28T18:26:27.933 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 5426981, 2023-11-28T18:26:27.933 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 0, 2023-11-28T18:26:27.934 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2023-11-28T18:26:27.934 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 344, 2023-11-28T18:26:27.934 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.815862606, 2023-11-28T18:26:27.934 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.002371693 2023-11-28T18:26:27.934 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.934 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 0 2023-11-28T18:26:27.934 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.934 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2023-11-28T18:26:27.934 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 98, 2023-11-28T18:26:27.934 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 291, 2023-11-28T18:26:27.935 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 193, 2023-11-28T18:26:27.935 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 62, 2023-11-28T18:26:27.935 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 122, 2023-11-28T18:26:27.935 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 60, 2023-11-28T18:26:27.935 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 96, 2023-11-28T18:26:27.935 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 365, 2023-11-28T18:26:27.935 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 269, 2023-11-28T18:26:27.935 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 86, 2023-11-28T18:26:27.935 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 279, 2023-11-28T18:26:27.935 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 193, 2023-11-28T18:26:27.935 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 53400, 2023-11-28T18:26:27.936 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 207144 2023-11-28T18:26:27.936 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.936 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2023-11-28T18:26:27.936 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 1059, 2023-11-28T18:26:27.936 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2023-11-28T18:26:27.936 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 705, 2023-11-28T18:26:27.936 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 8, 2023-11-28T18:26:27.936 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2023-11-28T18:26:27.936 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2023-11-28T18:26:27.937 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 176, 2023-11-28T18:26:27.937 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.331841107, 2023-11-28T18:26:27.937 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001885460 2023-11-28T18:26:27.937 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.937 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2023-11-28T18:26:27.937 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 16, 2023-11-28T18:26:27.937 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.003981788, 2023-11-28T18:26:27.937 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000248861 2023-11-28T18:26:27.937 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.937 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2023-11-28T18:26:27.938 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.938 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.938 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.938 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.938 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2023-11-28T18:26:27.938 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.938 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.938 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.938 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.938 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2023-11-28T18:26:27.939 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 331, 2023-11-28T18:26:27.939 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.057076197, 2023-11-28T18:26:27.939 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000172435 2023-11-28T18:26:27.939 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.939 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2023-11-28T18:26:27.939 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.939 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.939 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.939 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.939 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2023-11-28T18:26:27.939 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.940 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.940 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.940 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.940 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2023-11-28T18:26:27.940 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.940 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.940 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.940 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.940 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2023-11-28T18:26:27.940 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.940 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.941 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.941 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.941 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2023-11-28T18:26:27.941 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.941 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.941 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.941 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.941 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2023-11-28T18:26:27.941 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.941 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.941 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.941 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.942 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2023-11-28T18:26:27.942 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 102, 2023-11-28T18:26:27.942 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.289808368, 2023-11-28T18:26:27.942 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.002841258 2023-11-28T18:26:27.942 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.942 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2023-11-28T18:26:27.942 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.942 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.942 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.942 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.943 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2023-11-28T18:26:27.943 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.943 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.943 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.943 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.943 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2023-11-28T18:26:27.943 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.943 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.943 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.943 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.944 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2023-11-28T18:26:27.944 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2023-11-28T18:26:27.944 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.001026394, 2023-11-28T18:26:27.944 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001026394 2023-11-28T18:26:27.944 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.944 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2023-11-28T18:26:27.944 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.944 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.944 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.944 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.944 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2023-11-28T18:26:27.944 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.945 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.945 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.945 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.945 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2023-11-28T18:26:27.945 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.945 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.945 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.945 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.945 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2023-11-28T18:26:27.945 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2023-11-28T18:26:27.945 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.003064842, 2023-11-28T18:26:27.945 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.003064842 2023-11-28T18:26:27.945 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.946 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2023-11-28T18:26:27.946 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.946 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.946 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.946 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.946 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2023-11-28T18:26:27.946 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.946 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.946 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.946 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.946 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2023-11-28T18:26:27.946 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2023-11-28T18:26:27.947 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000418227, 2023-11-28T18:26:27.947 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000418227 2023-11-28T18:26:27.947 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.947 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2023-11-28T18:26:27.947 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.947 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.947 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.947 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.947 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2023-11-28T18:26:27.947 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.947 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.947 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.947 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.948 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2023-11-28T18:26:27.948 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.948 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.948 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.948 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.948 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2023-11-28T18:26:27.948 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2023-11-28T18:26:27.948 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000345721, 2023-11-28T18:26:27.948 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000345721 2023-11-28T18:26:27.948 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.948 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2023-11-28T18:26:27.948 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.948 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.949 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.949 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.949 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2023-11-28T18:26:27.949 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 76, 2023-11-28T18:26:27.949 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.399975690, 2023-11-28T18:26:27.949 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.005262838 2023-11-28T18:26:27.949 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.949 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2023-11-28T18:26:27.949 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2023-11-28T18:26:27.949 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2023-11-28T18:26:27.949 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.949 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.950 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.950 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.950 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.950 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2023-11-28T18:26:27.950 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 1, 2023-11-28T18:26:27.950 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 1, 2023-11-28T18:26:27.950 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 0, 2023-11-28T18:26:27.950 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 1, 2023-11-28T18:26:27.950 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2023-11-28T18:26:27.950 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 679, 2023-11-28T18:26:27.950 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 679, 2023-11-28T18:26:27.950 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 18, 2023-11-28T18:26:27.950 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2023-11-28T18:26:27.951 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.951 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2023-11-28T18:26:27.951 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2023-11-28T18:26:27.951 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2023-11-28T18:26:27.951 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2023-11-28T18:26:27.951 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2023-11-28T18:26:27.951 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2023-11-28T18:26:27.951 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2023-11-28T18:26:27.951 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-28T18:26:27.951 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2023-11-28T18:26:27.951 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-28T18:26:27.951 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2023-11-28T18:26:27.951 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2023-11-28T18:26:27.952 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2023-11-28T18:26:27.952 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-28T18:26:27.952 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2023-11-28T18:26:27.952 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2023-11-28T18:26:27.952 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2023-11-28T18:26:27.952 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2023-11-28T18:26:27.952 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2023-11-28T18:26:27.952 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-28T18:26:27.952 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2023-11-28T18:26:27.952 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-28T18:26:27.952 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2023-11-28T18:26:27.952 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2023-11-28T18:26:27.952 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2023-11-28T18:26:27.953 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2023-11-28T18:26:27.953 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2023-11-28T18:26:27.953 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-28T18:26:27.953 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2023-11-28T18:26:27.953 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2023-11-28T18:26:27.953 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2023-11-28T18:26:27.953 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2023-11-28T18:26:27.954 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2023-11-28T18:26:27.954 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2023-11-28T18:26:27.954 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2023-11-28T18:26:27.954 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2023-11-28T18:26:27.954 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2023-11-28T18:26:27.954 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 257070, 2023-11-28T18:26:27.954 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 93, 2023-11-28T18:26:27.954 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 88, 2023-11-28T18:26:27.954 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 1, 2023-11-28T18:26:27.954 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2023-11-28T18:26:27.954 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2023-11-28T18:26:27.954 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2023-11-28T18:26:27.954 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2023-11-28T18:26:27.955 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2023-11-28T18:26:27.955 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2023-11-28T18:26:27.955 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 5616, 2023-11-28T18:26:27.955 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 80, 2023-11-28T18:26:27.955 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2023-11-28T18:26:27.955 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2023-11-28T18:26:27.955 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2023-11-28T18:26:27.955 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2023-11-28T18:26:27.955 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 393445, 2023-11-28T18:26:27.955 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 2335, 2023-11-28T18:26:27.955 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2023-11-28T18:26:27.956 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2023-11-28T18:26:27.956 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2023-11-28T18:26:27.956 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2023-11-28T18:26:27.956 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.956 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2023-11-28T18:26:27.956 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2023-11-28T18:26:27.956 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2023-11-28T18:26:27.956 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 488, 2023-11-28T18:26:27.956 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 1371597, 2023-11-28T18:26:27.956 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 28, 2023-11-28T18:26:27.956 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 488, 2023-11-28T18:26:27.957 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2023-11-28T18:26:27.957 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 488, 2023-11-28T18:26:27.957 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 2.159972223, 2023-11-28T18:26:27.957 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.004426172 2023-11-28T18:26:27.957 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.957 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2023-11-28T18:26:27.957 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2023-11-28T18:26:27.957 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 488, 2023-11-28T18:26:27.957 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 529 2023-11-28T18:26:27.957 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.957 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 488, 2023-11-28T18:26:27.958 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 3, 2023-11-28T18:26:27.958 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 485, 2023-11-28T18:26:27.958 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2023-11-28T18:26:27.958 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2023-11-28T18:26:27.958 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 0, 2023-11-28T18:26:27.958 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 11, 2023-11-28T18:26:27.958 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 3, 2023-11-28T18:26:27.958 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 347, 2023-11-28T18:26:27.958 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 10, 2023-11-28T18:26:27.958 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2023-11-28T18:26:27.958 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2023-11-28T18:26:27.959 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2023-11-28T18:26:27.959 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 1, 2023-11-28T18:26:27.959 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 89, 2023-11-28T18:26:27.959 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2023-11-28T18:26:27.959 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2023-11-28T18:26:27.959 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2023-11-28T18:26:27.959 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 0, 2023-11-28T18:26:27.959 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 22, 2023-11-28T18:26:27.959 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2023-11-28T18:26:27.959 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2023-11-28T18:26:27.959 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2023-11-28T18:26:27.959 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2023-11-28T18:26:27.960 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2023-11-28T18:26:27.960 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2023-11-28T18:26:27.960 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2023-11-28T18:26:27.960 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2023-11-28T18:26:27.960 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2023-11-28T18:26:27.960 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2023-11-28T18:26:27.960 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2023-11-28T18:26:27.960 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2023-11-28T18:26:27.960 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2023-11-28T18:26:27.960 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2023-11-28T18:26:27.960 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2023-11-28T18:26:27.961 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2023-11-28T18:26:27.961 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2023-11-28T18:26:27.961 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2023-11-28T18:26:27.961 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2023-11-28T18:26:27.961 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2023-11-28T18:26:27.961 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2023-11-28T18:26:27.961 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2023-11-28T18:26:27.961 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2023-11-28T18:26:27.961 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2023-11-28T18:26:27.961 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2023-11-28T18:26:27.961 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2023-11-28T18:26:27.962 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 53, 2023-11-28T18:26:27.962 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2023-11-28T18:26:27.962 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 3, 2023-11-28T18:26:27.962 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 8, 2023-11-28T18:26:27.962 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 8, 2023-11-28T18:26:27.962 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 0, 2023-11-28T18:26:27.962 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2023-11-28T18:26:27.962 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 41, 2023-11-28T18:26:27.962 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 0, 2023-11-28T18:26:27.962 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 5 2023-11-28T18:26:27.962 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.962 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2023-11-28T18:26:27.963 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 1, 2023-11-28T18:26:27.963 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2023-11-28T18:26:27.963 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2023-11-28T18:26:27.963 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2023-11-28T18:26:27.963 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.963 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2023-11-28T18:26:27.963 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2023-11-28T18:26:27.963 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.964 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2023-11-28T18:26:27.964 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2023-11-28T18:26:27.964 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 10, 2023-11-28T18:26:27.964 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2023-11-28T18:26:27.964 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 10, 2023-11-28T18:26:27.964 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 77, 2023-11-28T18:26:27.964 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2023-11-28T18:26:27.964 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.964 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-28T18:26:27.964 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:27.965 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2023-11-28T18:26:27.965 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:27.965 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 1738, 2023-11-28T18:26:27.965 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 485059, 2023-11-28T18:26:27.965 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:27.965 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 1738, 2023-11-28T18:26:27.965 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2023-11-28T18:26:27.965 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2023-11-28T18:26:27.965 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 1738, 2023-11-28T18:26:27.965 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 485059, 2023-11-28T18:26:27.965 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:27.966 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.966 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.966 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.966 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.966 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.966 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2023-11-28T18:26:27.966 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:27.966 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2023-11-28T18:26:27.966 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:27.966 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2023-11-28T18:26:27.966 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2023-11-28T18:26:27.967 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:27.967 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2023-11-28T18:26:27.967 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 488, 2023-11-28T18:26:27.967 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 1379374, 2023-11-28T18:26:27.967 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 488, 2023-11-28T18:26:27.967 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 1379374, 2023-11-28T18:26:27.967 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:27.967 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.967 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.967 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.967 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.968 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.968 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2023-11-28T18:26:27.968 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:27.968 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2023-11-28T18:26:27.968 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:27.968 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2023-11-28T18:26:27.968 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2023-11-28T18:26:27.968 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:27.968 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2023-11-28T18:26:27.968 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 488, 2023-11-28T18:26:27.968 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 488, 2023-11-28T18:26:27.969 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 488, 2023-11-28T18:26:27.969 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 488, 2023-11-28T18:26:27.969 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:27.969 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.969 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.969 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.969 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.969 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.969 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2023-11-28T18:26:27.969 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:27.969 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2023-11-28T18:26:27.969 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:27.970 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 77, 2023-11-28T18:26:27.970 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 7777, 2023-11-28T18:26:27.970 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:27.970 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 77, 2023-11-28T18:26:27.970 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2023-11-28T18:26:27.970 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2023-11-28T18:26:27.970 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 3, 2023-11-28T18:26:27.970 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 7777, 2023-11-28T18:26:27.970 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:27.970 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.970 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.971 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.971 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.971 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:27.971 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x5607a9244a00": { 2023-11-28T18:26:27.971 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:27.971 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2023-11-28T18:26:27.971 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:27.971 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 625, 2023-11-28T18:26:27.971 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 1232677, 2023-11-28T18:26:27.971 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:27.971 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 625, 2023-11-28T18:26:27.972 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2023-11-28T18:26:27.972 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2023-11-28T18:26:27.972 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 344, 2023-11-28T18:26:27.972 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 1232677, 2023-11-28T18:26:27.972 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:27.972 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:27.972 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:27.972 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:27.972 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.972 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:27.972 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:26:27.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34b479640 1 -- 172.21.15.105:0/2801300688 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb328066900 msgr2=0x7fb328068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34b479640 1 --2- 172.21.15.105:0/2801300688 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb328066900 0x7fb328068dc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb3400011f0 tx=0x7fb340007d40 comp rx=0 tx=0).stop 2023-11-28T18:26:27.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34b479640 1 -- 172.21.15.105:0/2801300688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb344164140 msgr2=0x7fb344153900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:27.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34b479640 1 --2- 172.21.15.105:0/2801300688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb344164140 0x7fb344153900 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00c730 tx=0x7fb33c00cbf0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34b479640 1 -- 172.21.15.105:0/2801300688 shutdown_connections 2023-11-28T18:26:27.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34b479640 1 --2- 172.21.15.105:0/2801300688 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb328066900 0x7fb328068dc0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34b479640 1 --2- 172.21.15.105:0/2801300688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb344164140 0x7fb344153900 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34b479640 1 --2- 172.21.15.105:0/2801300688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb34415b030 0x7fb3441656e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34b479640 1 --2- 172.21.15.105:0/2801300688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3440b4290 0x7fb344159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:27.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34b479640 1 -- 172.21.15.105:0/2801300688 >> 172.21.15.105:0/2801300688 conn(0x7fb3440ceb90 msgr2=0x7fb3441577a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:27.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34b479640 1 -- 172.21.15.105:0/2801300688 shutdown_connections 2023-11-28T18:26:27.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:27.900+0000 7fb34b479640 1 -- 172.21.15.105:0/2801300688 wait complete. 2023-11-28T18:26:27.977 INFO:tasks.cephfs.test_exports:export stats: [(0, 5), (1, 0), (2, 0)] 2023-11-28T18:26:27.977 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 3:1 2023-11-28T18:26:28.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.112+0000 7fb459980640 1 Processor -- start 2023-11-28T18:26:28.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.112+0000 7fb459980640 1 -- start start 2023-11-28T18:26:28.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.116+0000 7fb459980640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb454159f40 0x7fb4540caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:28.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.116+0000 7fb459980640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45406bf50 con 0x7fb4540ad850 2023-11-28T18:26:28.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.116+0000 7fb459980640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4540ccbc0 con 0x7fb4540cc760 2023-11-28T18:26:28.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.116+0000 7fb459980640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45415a320 con 0x7fb454159f40 2023-11-28T18:26:28.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.116+0000 7fb4537fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb4540ad850 0x7fb4540701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35248/0 (socket says 172.21.15.105:35248) 2023-11-28T18:26:28.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.116+0000 7fb4537fe640 1 -- 172.21.15.105:0/1645717080 learned_addr learned my addr 172.21.15.105:0/1645717080 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:28.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.116+0000 7fb452ffd640 1 --2- 172.21.15.105:0/1645717080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb454159f40 0x7fb4540caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:28.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.116+0000 7fb451ffb640 1 -- 172.21.15.105:0/1645717080 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 642786161 0 0) 0x7fb45406bf50 con 0x7fb4540ad850 2023-11-28T18:26:28.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.116+0000 7fb451ffb640 1 -- 172.21.15.105:0/1645717080 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb43c003530 con 0x7fb4540ad850 2023-11-28T18:26:28.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.116+0000 7fb451ffb640 1 -- 172.21.15.105:0/1645717080 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4290681512 0 0) 0x7fb4540ccbc0 con 0x7fb4540cc760 2023-11-28T18:26:28.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.116+0000 7fb451ffb640 1 -- 172.21.15.105:0/1645717080 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb45406bf50 con 0x7fb4540cc760 2023-11-28T18:26:28.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.116+0000 7fb452ffd640 1 -- 172.21.15.105:0/1645717080 >> v1:172.21.15.148:6789/0 conn(0x7fb4540cc760 legacy=0x7fb4540ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:28.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.116+0000 7fb452ffd640 1 -- 172.21.15.105:0/1645717080 >> v1:172.21.15.105:6789/0 conn(0x7fb4540ad850 legacy=0x7fb4540701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:28.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.116+0000 7fb452ffd640 1 -- 172.21.15.105:0/1645717080 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb454071380 con 0x7fb454159f40 2023-11-28T18:26:28.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.120+0000 7fb452ffd640 1 --2- 172.21.15.105:0/1645717080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb454159f40 0x7fb4540caf60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb4440097f0 tx=0x7fb44401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41909540702c2e59 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:28.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.120+0000 7fb451ffb640 1 -- 172.21.15.105:0/1645717080 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4440ab600 con 0x7fb454159f40 2023-11-28T18:26:28.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.120+0000 7fb451ffb640 1 -- 172.21.15.105:0/1645717080 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4440ab760 con 0x7fb454159f40 2023-11-28T18:26:28.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.120+0000 7fb451ffb640 1 -- 172.21.15.105:0/1645717080 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4440abaf0 con 0x7fb454159f40 2023-11-28T18:26:28.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.120+0000 7fb459980640 1 -- 172.21.15.105:0/1645717080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb454159f40 msgr2=0x7fb4540caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:28.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.120+0000 7fb459980640 1 --2- 172.21.15.105:0/1645717080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb454159f40 0x7fb4540caf60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb4440097f0 tx=0x7fb44401c400 comp rx=0 tx=0).stop 2023-11-28T18:26:28.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.120+0000 7fb459980640 1 -- 172.21.15.105:0/1645717080 shutdown_connections 2023-11-28T18:26:28.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.120+0000 7fb459980640 1 --2- 172.21.15.105:0/1645717080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb454159f40 0x7fb4540caf60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:28.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.120+0000 7fb459980640 1 -- 172.21.15.105:0/1645717080 >> 172.21.15.105:0/1645717080 conn(0x7fb4540ceb90 msgr2=0x7fb4540cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:28.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.120+0000 7fb459980640 1 -- 172.21.15.105:0/1645717080 shutdown_connections 2023-11-28T18:26:28.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.120+0000 7fb459980640 1 -- 172.21.15.105:0/1645717080 wait complete. 2023-11-28T18:26:28.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.120+0000 7fb459980640 1 Processor -- start 2023-11-28T18:26:28.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.120+0000 7fb459980640 1 -- start start 2023-11-28T18:26:28.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb459980640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4540ad850 0x7fb45409c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:28.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb459980640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4540cc760 0x7fb45414c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:28.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb459980640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb454159f40 0x7fb45409a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:28.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb459980640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45409bf30 con 0x7fb4540ad850 2023-11-28T18:26:28.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb459980640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4540b2650 con 0x7fb4540cc760 2023-11-28T18:26:28.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb459980640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45409b320 con 0x7fb454159f40 2023-11-28T18:26:28.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb452ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4540ad850 0x7fb45409c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:28.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb452ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4540ad850 0x7fb45409c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35756/0 (socket says 172.21.15.105:35756) 2023-11-28T18:26:28.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb452ffd640 1 -- 172.21.15.105:0/1449167879 learned_addr learned my addr 172.21.15.105:0/1449167879 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:28.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb4537fe640 1 --2- 172.21.15.105:0/1449167879 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb454159f40 0x7fb45409a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:28.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb4527fc640 1 --2- 172.21.15.105:0/1449167879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4540cc760 0x7fb45414c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:28.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb452ffd640 1 -- 172.21.15.105:0/1449167879 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb454159f40 msgr2=0x7fb45409a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:28.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb452ffd640 1 --2- 172.21.15.105:0/1449167879 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb454159f40 0x7fb45409a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:28.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb452ffd640 1 -- 172.21.15.105:0/1449167879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4540cc760 msgr2=0x7fb45414c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:28.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb452ffd640 1 --2- 172.21.15.105:0/1449167879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4540cc760 0x7fb45414c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:28.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb452ffd640 1 -- 172.21.15.105:0/1449167879 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb444009420 con 0x7fb4540ad850 2023-11-28T18:26:28.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb4527fc640 1 --2- 172.21.15.105:0/1449167879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4540cc760 0x7fb45414c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:28.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb452ffd640 1 --2- 172.21.15.105:0/1449167879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4540ad850 0x7fb45409c3c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb444019010 tx=0x7fb44401ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:28.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb43bfff640 1 -- 172.21.15.105:0/1449167879 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4440bf610 con 0x7fb4540ad850 2023-11-28T18:26:28.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb43bfff640 1 -- 172.21.15.105:0/1449167879 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4440bf790 con 0x7fb4540ad850 2023-11-28T18:26:28.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb43bfff640 1 -- 172.21.15.105:0/1449167879 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4440bfb40 con 0x7fb4540ad850 2023-11-28T18:26:28.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb459980640 1 -- 172.21.15.105:0/1449167879 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45409b600 con 0x7fb4540ad850 2023-11-28T18:26:28.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb459980640 1 -- 172.21.15.105:0/1449167879 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb45409bbc0 con 0x7fb4540ad850 2023-11-28T18:26:28.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb459980640 1 -- 172.21.15.105:0/1449167879 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb454067570 con 0x7fb4540ad850 2023-11-28T18:26:28.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.124+0000 7fb43bfff640 1 -- 172.21.15.105:0/1449167879 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fb4440b5400 con 0x7fb4540ad850 2023-11-28T18:26:28.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.128+0000 7fb43bfff640 1 --2- 172.21.15.105:0/1449167879 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb4400668d0 0x7fb440068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:28.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.128+0000 7fb43bfff640 1 -- 172.21.15.105:0/1449167879 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fb44411f2e0 con 0x7fb4540ad850 2023-11-28T18:26:28.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.128+0000 7fb4527fc640 1 --2- 172.21.15.105:0/1449167879 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb4400668d0 0x7fb440068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:28.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.128+0000 7fb4527fc640 1 --2- 172.21.15.105:0/1449167879 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb4400668d0 0x7fb440068d90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb454071c70 tx=0x7fb448009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:28.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.132+0000 7fb43bfff640 1 -- 172.21.15.105:0/1449167879 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb4440ec560 con 0x7fb4540ad850 2023-11-28T18:26:28.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.336+0000 7fb459980640 1 -- 172.21.15.105:0/1449167879 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "3:1"} v 0) v1 -- 0x7fb4540cf950 con 0x7fb4540ad850 2023-11-28T18:26:28.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.340+0000 7fb43bfff640 1 -- 172.21.15.105:0/1449167879 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "3:1"}]=0 failed mds gid 5414 v42) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb4440efb80 con 0x7fb4540ad850 2023-11-28T18:26:28.343 INFO:teuthology.orchestra.run.smithi105.stderr:failed mds gid 5414 2023-11-28T18:26:28.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.340+0000 7fb459980640 1 -- 172.21.15.105:0/1449167879 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb4400668d0 msgr2=0x7fb440068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:28.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.340+0000 7fb459980640 1 --2- 172.21.15.105:0/1449167879 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb4400668d0 0x7fb440068d90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb454071c70 tx=0x7fb448009040 comp rx=0 tx=0).stop 2023-11-28T18:26:28.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.340+0000 7fb459980640 1 -- 172.21.15.105:0/1449167879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4540ad850 msgr2=0x7fb45409c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:28.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.340+0000 7fb459980640 1 --2- 172.21.15.105:0/1449167879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4540ad850 0x7fb45409c3c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb444019010 tx=0x7fb44401ac40 comp rx=0 tx=0).stop 2023-11-28T18:26:28.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.340+0000 7fb459980640 1 -- 172.21.15.105:0/1449167879 shutdown_connections 2023-11-28T18:26:28.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.340+0000 7fb459980640 1 --2- 172.21.15.105:0/1449167879 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb454159f40 0x7fb45409a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:28.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.340+0000 7fb459980640 1 --2- 172.21.15.105:0/1449167879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4540cc760 0x7fb45414c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:28.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.340+0000 7fb459980640 1 --2- 172.21.15.105:0/1449167879 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb4400668d0 0x7fb440068d90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:28.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.340+0000 7fb459980640 1 --2- 172.21.15.105:0/1449167879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4540ad850 0x7fb45409c3c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:28.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.340+0000 7fb459980640 1 -- 172.21.15.105:0/1449167879 >> 172.21.15.105:0/1449167879 conn(0x7fb4540ceb90 msgr2=0x7fb4540cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:28.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.340+0000 7fb459980640 1 -- 172.21.15.105:0/1449167879 shutdown_connections 2023-11-28T18:26:28.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.340+0000 7fb459980640 1 -- 172.21.15.105:0/1449167879 wait complete. 2023-11-28T18:26:28.369 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:26:28.406 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2023-11-28T18:26:28.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.504+0000 7f1075679640 1 Processor -- start 2023-11-28T18:26:28.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.504+0000 7f1075679640 1 -- start start 2023-11-28T18:26:28.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.508+0000 7f1075679640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f10700c7490 0x7f10700cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:28.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.508+0000 7f1075679640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f107006bda0 0x7f10700cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:28.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.508+0000 7f1075679640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10700c4a00 con 0x7f107006c790 2023-11-28T18:26:28.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.508+0000 7f1075679640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10700d0400 con 0x7f10700c7490 2023-11-28T18:26:28.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.508+0000 7f1075679640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10700c7870 con 0x7f107006bda0 2023-11-28T18:26:28.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.508+0000 7f106effd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f10700c7490 0x7f10700cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:28.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.508+0000 7f106effd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f10700c7490 0x7f10700cefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35766/0 (socket says 172.21.15.105:35766) 2023-11-28T18:26:28.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.508+0000 7f106effd640 1 -- 172.21.15.105:0/1850611233 learned_addr learned my addr 172.21.15.105:0/1850611233 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:28.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.508+0000 7f1067fff640 1 --1- 172.21.15.105:0/1850611233 >> v1:172.21.15.105:6789/0 conn(0x7f107006c790 0x7f107006b690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35264/0 (socket says 172.21.15.105:35264) 2023-11-28T18:26:28.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.508+0000 7f106f7fe640 1 --2- 172.21.15.105:0/1850611233 >> v2:172.21.15.148:3300/0 conn(0x7f107006bda0 0x7f10700cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:28.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.508+0000 7f106effd640 1 -- 172.21.15.105:0/1850611233 >> v2:172.21.15.148:3300/0 conn(0x7f107006bda0 msgr2=0x7f10700cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:28.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.508+0000 7f106effd640 1 --2- 172.21.15.105:0/1850611233 >> v2:172.21.15.148:3300/0 conn(0x7f107006bda0 0x7f10700cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:28.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.508+0000 7f106effd640 1 -- 172.21.15.105:0/1850611233 >> v1:172.21.15.105:6789/0 conn(0x7f107006c790 legacy=0x7f107006b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:28.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.508+0000 7f106effd640 1 -- 172.21.15.105:0/1850611233 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10700d0120 con 0x7f10700c7490 2023-11-28T18:26:28.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.508+0000 7f106e7fc640 1 -- 172.21.15.105:0/1850611233 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3357269434 0 0) 0x7f10700c4a00 con 0x7f107006c790 2023-11-28T18:26:28.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.512+0000 7f106effd640 1 --2- 172.21.15.105:0/1850611233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10700c7490 0x7f10700cefd0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1070071620 tx=0x7f106001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37066a393036dcdd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:28.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.512+0000 7f106e7fc640 1 -- 172.21.15.105:0/1850611233 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10600aa430 con 0x7f10700c7490 2023-11-28T18:26:28.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.512+0000 7f106e7fc640 1 -- 172.21.15.105:0/1850611233 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10600aa590 con 0x7f10700c7490 2023-11-28T18:26:28.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.512+0000 7f106e7fc640 1 -- 172.21.15.105:0/1850611233 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10600aa920 con 0x7f10700c7490 2023-11-28T18:26:28.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.512+0000 7f1075679640 1 -- 172.21.15.105:0/1850611233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10700c7490 msgr2=0x7f10700cefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:28.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.512+0000 7f1075679640 1 --2- 172.21.15.105:0/1850611233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10700c7490 0x7f10700cefd0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1070071620 tx=0x7f106001c4d0 comp rx=0 tx=0).stop 2023-11-28T18:26:28.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.512+0000 7f1075679640 1 -- 172.21.15.105:0/1850611233 shutdown_connections 2023-11-28T18:26:28.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.512+0000 7f1075679640 1 --2- 172.21.15.105:0/1850611233 >> v2:172.21.15.148:3300/0 conn(0x7f107006bda0 0x7f10700cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:28.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.512+0000 7f1075679640 1 --2- 172.21.15.105:0/1850611233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10700c7490 0x7f10700cefd0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:28.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.512+0000 7f1075679640 1 -- 172.21.15.105:0/1850611233 >> 172.21.15.105:0/1850611233 conn(0x7f10700d0eb0 msgr2=0x7f10700d12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:28.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.512+0000 7f1075679640 1 -- 172.21.15.105:0/1850611233 shutdown_connections 2023-11-28T18:26:28.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.512+0000 7f1075679640 1 -- 172.21.15.105:0/1850611233 wait complete. 2023-11-28T18:26:28.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.512+0000 7f1075679640 1 Processor -- start 2023-11-28T18:26:28.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.512+0000 7f1075679640 1 -- start start 2023-11-28T18:26:28.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f1075679640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f107006bda0 0x7f107015c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:28.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f1075679640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f107006c790 0x7f1070152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:28.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f1075679640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10700c7490 0x7f1070155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:28.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f1075679640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1070155a30 con 0x7f10700c7490 2023-11-28T18:26:28.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f1075679640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107015c4c0 con 0x7f107006c790 2023-11-28T18:26:28.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f1075679640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1070152ec0 con 0x7f107006bda0 2023-11-28T18:26:28.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f106f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10700c7490 0x7f1070155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:28.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f106f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10700c7490 0x7f1070155b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35770/0 (socket says 172.21.15.105:35770) 2023-11-28T18:26:28.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f106f7fe640 1 -- 172.21.15.105:0/1459162435 learned_addr learned my addr 172.21.15.105:0/1459162435 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:28.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f106effd640 1 --2- 172.21.15.105:0/1459162435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f107006bda0 0x7f107015c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:28.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f1067fff640 1 --2- 172.21.15.105:0/1459162435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f107006c790 0x7f1070152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:28.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f106f7fe640 1 -- 172.21.15.105:0/1459162435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f107006bda0 msgr2=0x7f107015c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:28.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f106f7fe640 1 --2- 172.21.15.105:0/1459162435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f107006bda0 0x7f107015c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:28.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f106f7fe640 1 -- 172.21.15.105:0/1459162435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f107006c790 msgr2=0x7f1070152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:28.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f106f7fe640 1 --2- 172.21.15.105:0/1459162435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f107006c790 0x7f1070152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:28.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f106f7fe640 1 -- 172.21.15.105:0/1459162435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10600094d0 con 0x7f10700c7490 2023-11-28T18:26:28.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f106f7fe640 1 --2- 172.21.15.105:0/1459162435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10700c7490 0x7f1070155b90 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f106800efc0 tx=0x7f106800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:28.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f10677fe640 1 -- 172.21.15.105:0/1459162435 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068009ba0 con 0x7f10700c7490 2023-11-28T18:26:28.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f10677fe640 1 -- 172.21.15.105:0/1459162435 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106800f040 con 0x7f10700c7490 2023-11-28T18:26:28.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f10677fe640 1 -- 172.21.15.105:0/1459162435 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10680146c0 con 0x7f10700c7490 2023-11-28T18:26:28.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f1075679640 1 -- 172.21.15.105:0/1459162435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1070155750 con 0x7f10700c7490 2023-11-28T18:26:28.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f1075679640 1 -- 172.21.15.105:0/1459162435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f107015c0e0 con 0x7f10700c7490 2023-11-28T18:26:28.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f1075679640 1 -- 172.21.15.105:0/1459162435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f107015bc60 con 0x7f10700c7490 2023-11-28T18:26:28.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.516+0000 7f10677fe640 1 -- 172.21.15.105:0/1459162435 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f1068014820 con 0x7f10700c7490 2023-11-28T18:26:28.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.520+0000 7f10677fe640 1 --2- 172.21.15.105:0/1459162435 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10500668d0 0x7f1050068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:28.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.520+0000 7f10677fe640 1 -- 172.21.15.105:0/1459162435 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f10680851d0 con 0x7f10700c7490 2023-11-28T18:26:28.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.520+0000 7f106effd640 1 --2- 172.21.15.105:0/1459162435 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10500668d0 0x7f1050068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:28.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.520+0000 7f106effd640 1 --2- 172.21.15.105:0/1459162435 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10500668d0 0x7f1050068d90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f10600ae000 tx=0x7f10600a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:28.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.524+0000 7f10677fe640 1 -- 172.21.15.105:0/1459162435 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f107015bc60 con 0x7f10700c7490 2023-11-28T18:26:28.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.700+0000 7f1075679640 1 -- 172.21.15.105:0/1459162435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f107015c600 con 0x7f10700c7490 2023-11-28T18:26:28.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.704+0000 7f10677fe640 1 -- 172.21.15.105:0/1459162435 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4161 (secure 0 0 0) 0x7f107015c600 con 0x7f10700c7490 2023-11-28T18:26:28.706 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:28.706 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:26:00.550298+0000","modified":"2023-11-28T18:26:28.348124+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5842,"mds_1":5405,"mds_2":5408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5405":{"gid":5405,"name":"a","rank":1,"incarnation":43,"state":"up:replay","state_seq":1,"addr":"172.21.15.105:6836/735646078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":735646078},{"type":"v1","addr":"172.21.15.105:6836","nonce":735646078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5408":{"gid":5408,"name":"c","rank":2,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6837/3765684108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":3765684108},{"type":"v1","addr":"172.21.15.105:6837","nonce":3765684108}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5842":{"gid":5842,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6834/2696773413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2696773413},{"type":"v1","addr":"172.21.15.148:6834","nonce":2696773413}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T18:26:28.706 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 43 2023-11-28T18:26:28.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.704+0000 7f1075679640 1 -- 172.21.15.105:0/1459162435 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10500668d0 msgr2=0x7f1050068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:28.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.704+0000 7f1075679640 1 --2- 172.21.15.105:0/1459162435 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10500668d0 0x7f1050068d90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f10600ae000 tx=0x7f10600a7000 comp rx=0 tx=0).stop 2023-11-28T18:26:28.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.704+0000 7f1075679640 1 -- 172.21.15.105:0/1459162435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10700c7490 msgr2=0x7f1070155b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:28.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.704+0000 7f1075679640 1 --2- 172.21.15.105:0/1459162435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10700c7490 0x7f1070155b90 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f106800efc0 tx=0x7f106800c490 comp rx=0 tx=0).stop 2023-11-28T18:26:28.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.704+0000 7f1075679640 1 -- 172.21.15.105:0/1459162435 shutdown_connections 2023-11-28T18:26:28.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.704+0000 7f1075679640 1 --2- 172.21.15.105:0/1459162435 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10500668d0 0x7f1050068d90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:28.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.704+0000 7f1075679640 1 --2- 172.21.15.105:0/1459162435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10700c7490 0x7f1070155b90 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:28.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.704+0000 7f1075679640 1 --2- 172.21.15.105:0/1459162435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f107006c790 0x7f1070152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:28.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.704+0000 7f1075679640 1 --2- 172.21.15.105:0/1459162435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f107006bda0 0x7f107015c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:28.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.704+0000 7f1075679640 1 -- 172.21.15.105:0/1459162435 >> 172.21.15.105:0/1459162435 conn(0x7f10700d0eb0 msgr2=0x7f10700c4910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:28.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.704+0000 7f1075679640 1 -- 172.21.15.105:0/1459162435 shutdown_connections 2023-11-28T18:26:28.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:28.704+0000 7f1075679640 1 -- 172.21.15.105:0/1459162435 wait complete. 2023-11-28T18:26:29.129 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:26:00.550298+0000', 'modified': '2023-11-28T18:26:28.348124+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 54, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 5842, 'mds_1': 5405, 'mds_2': 5408}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5405': {'gid': 5405, 'name': 'a', 'rank': 1, 'incarnation': 43, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.105:6836/735646078', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 735646078}, {'type': 'v1', 'addr': '172.21.15.105:6836', 'nonce': 735646078}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_5408': {'gid': 5408, 'name': 'c', 'rank': 2, 'incarnation': 39, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.105:6837/3765684108', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6835', 'nonce': 3765684108}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 3765684108}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_5842': {'gid': 5842, 'name': 'b', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6834/2696773413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6832', 'nonce': 2696773413}, {'type': 'v1', 'addr': '172.21.15.148:6834', 'nonce': 2696773413}]}, 'join_fscid': -1, 'export_targets': [1, 2], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:26:29.129 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5405:up:replay 2023-11-28T18:26:30.130 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:26:30.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.276+0000 7fec71d72640 1 Processor -- start 2023-11-28T18:26:30.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.276+0000 7fec71d72640 1 -- start start 2023-11-28T18:26:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.276+0000 7fec71d72640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fec6c15d4f0 0x7fec6c0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.276+0000 7fec71d72640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec6c0d3c30 0x7fec6c0d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.276+0000 7fec71d72640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec6c0b25b0 con 0x7fec6c0d1890 2023-11-28T18:26:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.276+0000 7fec71d72640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec6c0b2900 con 0x7fec6c15d4f0 2023-11-28T18:26:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.276+0000 7fec71d72640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0cea30 con 0x7fec6c0d3c30 2023-11-28T18:26:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.276+0000 7fec6bfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fec6c0d1890 0x7fec6c0cfa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35274/0 (socket says 172.21.15.105:35274) 2023-11-28T18:26:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.276+0000 7fec6bfff640 1 -- 172.21.15.105:0/2764781785 learned_addr learned my addr 172.21.15.105:0/2764781785 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.276+0000 7fec6b7fe640 1 --2- 172.21.15.105:0/2764781785 >> v2:172.21.15.148:3300/0 conn(0x7fec6c15d4f0 0x7fec6c0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.276+0000 7fec6affd640 1 --2- 172.21.15.105:0/2764781785 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec6c0d3c30 0x7fec6c0d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.276+0000 7fec6a7fc640 1 -- 172.21.15.105:0/2764781785 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148384744 0 0) 0x7fec6c0b25b0 con 0x7fec6c0d1890 2023-11-28T18:26:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.276+0000 7fec6a7fc640 1 -- 172.21.15.105:0/2764781785 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec54003530 con 0x7fec6c0d1890 2023-11-28T18:26:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.280+0000 7fec6b7fe640 1 -- 172.21.15.105:0/2764781785 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec6c0d3c30 msgr2=0x7fec6c0d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.280+0000 7fec6b7fe640 1 --2- 172.21.15.105:0/2764781785 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec6c0d3c30 0x7fec6c0d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.280+0000 7fec6b7fe640 1 -- 172.21.15.105:0/2764781785 >> v1:172.21.15.105:6789/0 conn(0x7fec6c0d1890 legacy=0x7fec6c0cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.280+0000 7fec6a7fc640 1 -- 172.21.15.105:0/2764781785 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3507234816 0 0) 0x7fec54003530 con 0x7fec6c0d1890 2023-11-28T18:26:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.280+0000 7fec6b7fe640 1 -- 172.21.15.105:0/2764781785 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c0ced10 con 0x7fec6c15d4f0 2023-11-28T18:26:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.280+0000 7fec6affd640 1 --2- 172.21.15.105:0/2764781785 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec6c0d3c30 0x7fec6c0d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:26:30.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.280+0000 7fec6b7fe640 1 --2- 172.21.15.105:0/2764781785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec6c15d4f0 0x7fec6c0d0e10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0097c0 tx=0x7fec5c01c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc117c92cfd11030 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:30.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.280+0000 7fec6a7fc640 1 -- 172.21.15.105:0/2764781785 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64001a00 con 0x7fec6c15d4f0 2023-11-28T18:26:30.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.280+0000 7fec6a7fc640 1 -- 172.21.15.105:0/2764781785 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c0a7050 con 0x7fec6c15d4f0 2023-11-28T18:26:30.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.280+0000 7fec6a7fc640 1 -- 172.21.15.105:0/2764781785 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0ab600 con 0x7fec6c15d4f0 2023-11-28T18:26:30.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec71d72640 1 -- 172.21.15.105:0/2764781785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec6c15d4f0 msgr2=0x7fec6c0d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:30.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec71d72640 1 --2- 172.21.15.105:0/2764781785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec6c15d4f0 0x7fec6c0d0e10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0097c0 tx=0x7fec5c01c2e0 comp rx=0 tx=0).stop 2023-11-28T18:26:30.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec71d72640 1 -- 172.21.15.105:0/2764781785 shutdown_connections 2023-11-28T18:26:30.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec71d72640 1 --2- 172.21.15.105:0/2764781785 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec6c0d3c30 0x7fec6c0d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:30.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec71d72640 1 --2- 172.21.15.105:0/2764781785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec6c15d4f0 0x7fec6c0d0e10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:30.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec71d72640 1 -- 172.21.15.105:0/2764781785 >> 172.21.15.105:0/2764781785 conn(0x7fec6c0d3030 msgr2=0x7fec6c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:30.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec71d72640 1 -- 172.21.15.105:0/2764781785 shutdown_connections 2023-11-28T18:26:30.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec71d72640 1 -- 172.21.15.105:0/2764781785 wait complete. 2023-11-28T18:26:30.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec71d72640 1 Processor -- start 2023-11-28T18:26:30.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec71d72640 1 -- start start 2023-11-28T18:26:30.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec71d72640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec6c0d1890 0x7fec6c14f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:30.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec71d72640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec6c0d3c30 0x7fec6c0b7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:30.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec71d72640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec6c15d4f0 0x7fec6c14d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:30.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec71d72640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0b6050 con 0x7fec6c0d1890 2023-11-28T18:26:30.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec71d72640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0b8a40 con 0x7fec6c0d3c30 2023-11-28T18:26:30.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec71d72640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0b8480 con 0x7fec6c15d4f0 2023-11-28T18:26:30.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec6b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec6c0d1890 0x7fec6c14f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:30.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec6b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec6c0d1890 0x7fec6c14f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35786/0 (socket says 172.21.15.105:35786) 2023-11-28T18:26:30.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec6b7fe640 1 -- 172.21.15.105:0/4075560745 learned_addr learned my addr 172.21.15.105:0/4075560745 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:30.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec6bfff640 1 --2- 172.21.15.105:0/4075560745 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec6c15d4f0 0x7fec6c14d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:30.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec6affd640 1 --2- 172.21.15.105:0/4075560745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec6c0d3c30 0x7fec6c0b7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:30.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec6b7fe640 1 -- 172.21.15.105:0/4075560745 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec6c15d4f0 msgr2=0x7fec6c14d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:30.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec6b7fe640 1 --2- 172.21.15.105:0/4075560745 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec6c15d4f0 0x7fec6c14d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:30.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec6b7fe640 1 -- 172.21.15.105:0/4075560745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec6c0d3c30 msgr2=0x7fec6c0b7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:30.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec6b7fe640 1 --2- 172.21.15.105:0/4075560745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec6c0d3c30 0x7fec6c0b7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:30.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec6b7fe640 1 -- 172.21.15.105:0/4075560745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec60009420 con 0x7fec6c0d1890 2023-11-28T18:26:30.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.284+0000 7fec6bfff640 1 --2- 172.21.15.105:0/4075560745 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec6c15d4f0 0x7fec6c14d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:30.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.288+0000 7fec6affd640 1 --2- 172.21.15.105:0/4075560745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec6c0d3c30 0x7fec6c0b7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:30.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.288+0000 7fec6b7fe640 1 --2- 172.21.15.105:0/4075560745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec6c0d1890 0x7fec6c14f2d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fec5c000c10 tx=0x7fec5c019080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:30.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.288+0000 7fec53fff640 1 -- 172.21.15.105:0/4075560745 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0b4400 con 0x7fec6c0d1890 2023-11-28T18:26:30.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.288+0000 7fec53fff640 1 -- 172.21.15.105:0/4075560745 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c0b4560 con 0x7fec6c0d1890 2023-11-28T18:26:30.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.288+0000 7fec53fff640 1 -- 172.21.15.105:0/4075560745 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0b48d0 con 0x7fec6c0d1890 2023-11-28T18:26:30.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.288+0000 7fec71d72640 1 -- 172.21.15.105:0/4075560745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec5c009420 con 0x7fec6c0d1890 2023-11-28T18:26:30.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.288+0000 7fec71d72640 1 -- 172.21.15.105:0/4075560745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6c0b5ce0 con 0x7fec6c0d1890 2023-11-28T18:26:30.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.288+0000 7fec71d72640 1 -- 172.21.15.105:0/4075560745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec6c067570 con 0x7fec6c0d1890 2023-11-28T18:26:30.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.288+0000 7fec53fff640 1 -- 172.21.15.105:0/4075560745 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fec5c0b2040 con 0x7fec6c0d1890 2023-11-28T18:26:30.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.288+0000 7fec53fff640 1 --2- 172.21.15.105:0/4075560745 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec580668d0 0x7fec58068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:30.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.288+0000 7fec53fff640 1 -- 172.21.15.105:0/4075560745 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7fec5c11e650 con 0x7fec6c0d1890 2023-11-28T18:26:30.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.288+0000 7fec6affd640 1 --2- 172.21.15.105:0/4075560745 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec580668d0 0x7fec58068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:30.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.292+0000 7fec6affd640 1 --2- 172.21.15.105:0/4075560745 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec580668d0 0x7fec58068d90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0c85e0 tx=0x7fec60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:30.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.292+0000 7fec53fff640 1 -- 172.21.15.105:0/4075560745 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fec5c0eb6b0 con 0x7fec6c0d1890 2023-11-28T18:26:30.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.480+0000 7fec71d72640 1 -- 172.21.15.105:0/4075560745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec6c0b0dd0 con 0x7fec6c0d1890 2023-11-28T18:26:30.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.480+0000 7fec53fff640 1 -- 172.21.15.105:0/4075560745 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4958 (secure 0 0 0) 0x7fec5c0eecd0 con 0x7fec6c0d1890 2023-11-28T18:26:30.482 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:30.482 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":45,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1686803161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":1686803161},{"type":"v1","addr":"172.21.15.148:6835","nonce":1686803161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:26:00.550298+0000","modified":"2023-11-28T18:26:29.359762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5842,"mds_1":5405,"mds_2":5408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5405":{"gid":5405,"name":"a","rank":1,"incarnation":43,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.105:6836/735646078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":735646078},{"type":"v1","addr":"172.21.15.105:6836","nonce":735646078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5408":{"gid":5408,"name":"c","rank":2,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6837/3765684108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":3765684108},{"type":"v1","addr":"172.21.15.105:6837","nonce":3765684108}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5842":{"gid":5842,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6834/2696773413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2696773413},{"type":"v1","addr":"172.21.15.148:6834","nonce":2696773413}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T18:26:30.482 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 45 2023-11-28T18:26:30.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.480+0000 7fec71d72640 1 -- 172.21.15.105:0/4075560745 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec580668d0 msgr2=0x7fec58068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:30.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.480+0000 7fec71d72640 1 --2- 172.21.15.105:0/4075560745 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec580668d0 0x7fec58068d90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0c85e0 tx=0x7fec60008040 comp rx=0 tx=0).stop 2023-11-28T18:26:30.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.480+0000 7fec71d72640 1 -- 172.21.15.105:0/4075560745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec6c0d1890 msgr2=0x7fec6c14f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:30.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.480+0000 7fec71d72640 1 --2- 172.21.15.105:0/4075560745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec6c0d1890 0x7fec6c14f2d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fec5c000c10 tx=0x7fec5c019080 comp rx=0 tx=0).stop 2023-11-28T18:26:30.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.480+0000 7fec71d72640 1 -- 172.21.15.105:0/4075560745 shutdown_connections 2023-11-28T18:26:30.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.480+0000 7fec71d72640 1 --2- 172.21.15.105:0/4075560745 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec6c15d4f0 0x7fec6c14d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:30.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.480+0000 7fec71d72640 1 --2- 172.21.15.105:0/4075560745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec6c0d3c30 0x7fec6c0b7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:30.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.480+0000 7fec71d72640 1 --2- 172.21.15.105:0/4075560745 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec580668d0 0x7fec58068d90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:30.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.480+0000 7fec71d72640 1 --2- 172.21.15.105:0/4075560745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec6c0d1890 0x7fec6c14f2d0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:30.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.480+0000 7fec71d72640 1 -- 172.21.15.105:0/4075560745 >> 172.21.15.105:0/4075560745 conn(0x7fec6c0d3030 msgr2=0x7fec6c0cf490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:30.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.480+0000 7fec71d72640 1 -- 172.21.15.105:0/4075560745 shutdown_connections 2023-11-28T18:26:30.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:30.480+0000 7fec71d72640 1 -- 172.21.15.105:0/4075560745 wait complete. 2023-11-28T18:26:30.905 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:26:00.550298+0000', 'modified': '2023-11-28T18:26:29.359762+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 54, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 5842, 'mds_1': 5405, 'mds_2': 5408}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5405': {'gid': 5405, 'name': 'a', 'rank': 1, 'incarnation': 43, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.105:6836/735646078', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 735646078}, {'type': 'v1', 'addr': '172.21.15.105:6836', 'nonce': 735646078}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_5408': {'gid': 5408, 'name': 'c', 'rank': 2, 'incarnation': 39, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.105:6837/3765684108', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6835', 'nonce': 3765684108}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 3765684108}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_5842': {'gid': 5842, 'name': 'b', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6834/2696773413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6832', 'nonce': 2696773413}, {'type': 'v1', 'addr': '172.21.15.148:6834', 'nonce': 2696773413}]}, 'join_fscid': -1, 'export_targets': [1, 2], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:26:30.905 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5405:up:reconnect 2023-11-28T18:26:31.723 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:26:31.727 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:26:31.906 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:26:32.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.044+0000 7f5499f80640 1 Processor -- start 2023-11-28T18:26:32.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.044+0000 7f5499f80640 1 -- start start 2023-11-28T18:26:32.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.044+0000 7f5499f80640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f549415d4f0 0x7f54940d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:32.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.044+0000 7f5499f80640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54940b2d20 0x7f54940b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:32.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.044+0000 7f5499f80640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54940b25b0 con 0x7f54940d3c30 2023-11-28T18:26:32.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.044+0000 7f5499f80640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54940b2900 con 0x7f54940b2d20 2023-11-28T18:26:32.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.044+0000 7f5499f80640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54940d0a20 con 0x7f549415d4f0 2023-11-28T18:26:32.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.044+0000 7f54937fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f549415d4f0 0x7f54940d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:32.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.044+0000 7f54937fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f549415d4f0 0x7f54940d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:60034/0 (socket says 172.21.15.105:60034) 2023-11-28T18:26:32.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.044+0000 7f54937fe640 1 -- 172.21.15.105:0/275389706 learned_addr learned my addr 172.21.15.105:0/275389706 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:32.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.044+0000 7f5493fff640 1 --2- 172.21.15.105:0/275389706 >> v2:172.21.15.148:3300/0 conn(0x7f54940b2d20 0x7f54940b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:32.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.044+0000 7f54937fe640 1 -- 172.21.15.105:0/275389706 >> v2:172.21.15.148:3300/0 conn(0x7f54940b2d20 msgr2=0x7f54940b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:32.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.044+0000 7f54937fe640 1 --2- 172.21.15.105:0/275389706 >> v2:172.21.15.148:3300/0 conn(0x7f54940b2d20 0x7f54940b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:32.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.044+0000 7f54937fe640 1 -- 172.21.15.105:0/275389706 >> v1:172.21.15.148:6789/0 conn(0x7f54940d3c30 legacy=0x7f54940d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:32.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.044+0000 7f54937fe640 1 -- 172.21.15.105:0/275389706 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54940cea30 con 0x7f549415d4f0 2023-11-28T18:26:32.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.048+0000 7f54937fe640 1 --2- 172.21.15.105:0/275389706 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f549415d4f0 0x7f54940d0e10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5484009870 tx=0x7f548401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13d8644d77e0fce9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:32.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.048+0000 7f5492ffd640 1 -- 172.21.15.105:0/275389706 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54840ab680 con 0x7f549415d4f0 2023-11-28T18:26:32.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.048+0000 7f5492ffd640 1 -- 172.21.15.105:0/275389706 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54840ab7e0 con 0x7f549415d4f0 2023-11-28T18:26:32.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.048+0000 7f5492ffd640 1 -- 172.21.15.105:0/275389706 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54840abb70 con 0x7f549415d4f0 2023-11-28T18:26:32.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.048+0000 7f5499f80640 1 -- 172.21.15.105:0/275389706 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f549415d4f0 msgr2=0x7f54940d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:32.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.048+0000 7f5499f80640 1 --2- 172.21.15.105:0/275389706 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f549415d4f0 0x7f54940d0e10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5484009870 tx=0x7f548401c480 comp rx=0 tx=0).stop 2023-11-28T18:26:32.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.048+0000 7f5499f80640 1 -- 172.21.15.105:0/275389706 shutdown_connections 2023-11-28T18:26:32.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.048+0000 7f5499f80640 1 --2- 172.21.15.105:0/275389706 >> v2:172.21.15.148:3300/0 conn(0x7f54940b2d20 0x7f54940b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:32.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.048+0000 7f5499f80640 1 --2- 172.21.15.105:0/275389706 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f549415d4f0 0x7f54940d0e10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:32.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.048+0000 7f5499f80640 1 -- 172.21.15.105:0/275389706 >> 172.21.15.105:0/275389706 conn(0x7f54940d3030 msgr2=0x7f54940d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:32.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.048+0000 7f5499f80640 1 -- 172.21.15.105:0/275389706 shutdown_connections 2023-11-28T18:26:32.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.048+0000 7f5499f80640 1 -- 172.21.15.105:0/275389706 wait complete. 2023-11-28T18:26:32.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f5499f80640 1 Processor -- start 2023-11-28T18:26:32.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f5499f80640 1 -- start start 2023-11-28T18:26:32.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f5499f80640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54940b2d20 0x7f5494160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:32.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f5499f80640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54940d3c30 0x7f54941526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:32.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f5499f80640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f549415d4f0 0x7f549415dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:32.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f5499f80640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5494152c10 con 0x7f54940d3c30 2023-11-28T18:26:32.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f5499f80640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549415e230 con 0x7f549415d4f0 2023-11-28T18:26:32.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f5499f80640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54940ca100 con 0x7f54940b2d20 2023-11-28T18:26:32.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f548bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54940d3c30 0x7f54941526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:32.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f548bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54940d3c30 0x7f54941526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44388/0 (socket says 172.21.15.105:44388) 2023-11-28T18:26:32.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f548bfff640 1 -- 172.21.15.105:0/3046480786 learned_addr learned my addr 172.21.15.105:0/3046480786 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:32.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f5493fff640 1 --2- 172.21.15.105:0/3046480786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f549415d4f0 0x7f549415dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:32.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f54937fe640 1 --2- 172.21.15.105:0/3046480786 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54940b2d20 0x7f5494160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:32.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f548bfff640 1 -- 172.21.15.105:0/3046480786 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54940b2d20 msgr2=0x7f5494160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:32.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f548bfff640 1 --2- 172.21.15.105:0/3046480786 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54940b2d20 0x7f5494160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:32.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f548bfff640 1 -- 172.21.15.105:0/3046480786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f549415d4f0 msgr2=0x7f549415dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:32.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f548bfff640 1 --2- 172.21.15.105:0/3046480786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f549415d4f0 0x7f549415dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:32.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f548bfff640 1 -- 172.21.15.105:0/3046480786 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54840094d0 con 0x7f54940d3c30 2023-11-28T18:26:32.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f548bfff640 1 --2- 172.21.15.105:0/3046480786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54940d3c30 0x7f54941526d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f547c00c500 tx=0x7f547c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:32.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f5490ff9640 1 -- 172.21.15.105:0/3046480786 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c005220 con 0x7f54940d3c30 2023-11-28T18:26:32.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f5490ff9640 1 -- 172.21.15.105:0/3046480786 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c005380 con 0x7f54940d3c30 2023-11-28T18:26:32.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f5499f80640 1 -- 172.21.15.105:0/3046480786 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54940ca3c0 con 0x7f54940d3c30 2023-11-28T18:26:32.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f5490ff9640 1 -- 172.21.15.105:0/3046480786 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c007270 con 0x7f54940d3c30 2023-11-28T18:26:32.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.052+0000 7f5499f80640 1 -- 172.21.15.105:0/3046480786 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54940ca980 con 0x7f54940d3c30 2023-11-28T18:26:32.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.056+0000 7f5499f80640 1 -- 172.21.15.105:0/3046480786 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5494067570 con 0x7f54940d3c30 2023-11-28T18:26:32.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.056+0000 7f5490ff9640 1 -- 172.21.15.105:0/3046480786 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f547c0073d0 con 0x7f54940d3c30 2023-11-28T18:26:32.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.056+0000 7f5490ff9640 1 --2- 172.21.15.105:0/3046480786 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f54740668d0 0x7f5474068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:32.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.056+0000 7f5490ff9640 1 -- 172.21.15.105:0/3046480786 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f547c083ea0 con 0x7f54940d3c30 2023-11-28T18:26:32.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.056+0000 7f54937fe640 1 --2- 172.21.15.105:0/3046480786 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f54740668d0 0x7f5474068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:32.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.056+0000 7f54937fe640 1 --2- 172.21.15.105:0/3046480786 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f54740668d0 0x7f5474068d90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f54840af000 tx=0x7f54840a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:32.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.060+0000 7f5490ff9640 1 -- 172.21.15.105:0/3046480786 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f547c050f30 con 0x7f54940d3c30 2023-11-28T18:26:32.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.244+0000 7f5499f80640 1 -- 172.21.15.105:0/3046480786 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54940b0b80 con 0x7f54940d3c30 2023-11-28T18:26:32.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.244+0000 7f5490ff9640 1 -- 172.21.15.105:0/3046480786 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4955 (secure 0 0 0) 0x7f547c054550 con 0x7f54940d3c30 2023-11-28T18:26:32.248 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:32.249 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":46,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1686803161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":1686803161},{"type":"v1","addr":"172.21.15.148:6835","nonce":1686803161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:26:00.550298+0000","modified":"2023-11-28T18:26:30.362963+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5842,"mds_1":5405,"mds_2":5408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5405":{"gid":5405,"name":"a","rank":1,"incarnation":43,"state":"up:rejoin","state_seq":12,"addr":"172.21.15.105:6836/735646078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":735646078},{"type":"v1","addr":"172.21.15.105:6836","nonce":735646078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5408":{"gid":5408,"name":"c","rank":2,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6837/3765684108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":3765684108},{"type":"v1","addr":"172.21.15.105:6837","nonce":3765684108}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5842":{"gid":5842,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6834/2696773413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2696773413},{"type":"v1","addr":"172.21.15.148:6834","nonce":2696773413}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T18:26:32.249 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 46 2023-11-28T18:26:32.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.248+0000 7f5499f80640 1 -- 172.21.15.105:0/3046480786 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f54740668d0 msgr2=0x7f5474068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:32.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.248+0000 7f5499f80640 1 --2- 172.21.15.105:0/3046480786 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f54740668d0 0x7f5474068d90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f54840af000 tx=0x7f54840a7000 comp rx=0 tx=0).stop 2023-11-28T18:26:32.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.248+0000 7f5499f80640 1 -- 172.21.15.105:0/3046480786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54940d3c30 msgr2=0x7f54941526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:32.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.248+0000 7f5499f80640 1 --2- 172.21.15.105:0/3046480786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54940d3c30 0x7f54941526d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f547c00c500 tx=0x7f547c00c9c0 comp rx=0 tx=0).stop 2023-11-28T18:26:32.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.248+0000 7f5499f80640 1 -- 172.21.15.105:0/3046480786 shutdown_connections 2023-11-28T18:26:32.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.248+0000 7f5499f80640 1 --2- 172.21.15.105:0/3046480786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f549415d4f0 0x7f549415dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:32.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.248+0000 7f5499f80640 1 --2- 172.21.15.105:0/3046480786 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f54740668d0 0x7f5474068d90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:32.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.248+0000 7f5499f80640 1 --2- 172.21.15.105:0/3046480786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54940d3c30 0x7f54941526d0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:32.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.248+0000 7f5499f80640 1 --2- 172.21.15.105:0/3046480786 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54940b2d20 0x7f5494160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:32.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.248+0000 7f5499f80640 1 -- 172.21.15.105:0/3046480786 >> 172.21.15.105:0/3046480786 conn(0x7f54940d3030 msgr2=0x7f54940afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:32.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.248+0000 7f5499f80640 1 -- 172.21.15.105:0/3046480786 shutdown_connections 2023-11-28T18:26:32.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:32.248+0000 7f5499f80640 1 -- 172.21.15.105:0/3046480786 wait complete. 2023-11-28T18:26:32.671 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:26:00.550298+0000', 'modified': '2023-11-28T18:26:30.362963+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 54, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 5842, 'mds_1': 5405, 'mds_2': 5408}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5405': {'gid': 5405, 'name': 'a', 'rank': 1, 'incarnation': 43, 'state': 'up:rejoin', 'state_seq': 12, 'addr': '172.21.15.105:6836/735646078', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 735646078}, {'type': 'v1', 'addr': '172.21.15.105:6836', 'nonce': 735646078}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_5408': {'gid': 5408, 'name': 'c', 'rank': 2, 'incarnation': 39, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.105:6837/3765684108', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6835', 'nonce': 3765684108}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 3765684108}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_5842': {'gid': 5842, 'name': 'b', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6834/2696773413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6832', 'nonce': 2696773413}, {'type': 'v1', 'addr': '172.21.15.148:6834', 'nonce': 2696773413}]}, 'join_fscid': -1, 'export_targets': [1, 2], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:26:32.671 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5405:up:rejoin 2023-11-28T18:26:33.137 INFO:tasks.ceph.mds.b.smithi148.stderr:2023-11-28T18:26:33.131+0000 7f609c6a2640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-28T18:26:33.672 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:26:33.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.816+0000 7f0da78dd640 1 Processor -- start 2023-11-28T18:26:33.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.816+0000 7f0da78dd640 1 -- start start 2023-11-28T18:26:33.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.820+0000 7f0da78dd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0da00ad9f0 0x7f0da00ade80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:33.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.820+0000 7f0da78dd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0da00cc7d0 0x7f0da00ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:33.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.820+0000 7f0da78dd640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0da0162820 con 0x7f0da006d610 2023-11-28T18:26:33.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.820+0000 7f0da78dd640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da00ae3c0 con 0x7f0da00ad9f0 2023-11-28T18:26:33.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.820+0000 7f0da78dd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da0161c10 con 0x7f0da00cc7d0 2023-11-28T18:26:33.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.820+0000 7f0da4e78640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0da00ad9f0 0x7f0da00ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:33.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.820+0000 7f0da5679640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0da006d610 0x7f0da00cdff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41492/0 (socket says 172.21.15.105:41492) 2023-11-28T18:26:33.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.820+0000 7f0da4e78640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0da00ad9f0 0x7f0da00ade80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44404/0 (socket says 172.21.15.105:44404) 2023-11-28T18:26:33.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.820+0000 7f0da5679640 1 -- 172.21.15.105:0/3961539061 learned_addr learned my addr 172.21.15.105:0/3961539061 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:33.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.820+0000 7f0da5e7a640 1 --2- 172.21.15.105:0/3961539061 >> v2:172.21.15.148:3300/0 conn(0x7f0da00cc7d0 0x7f0da00ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:33.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.820+0000 7f0da4e78640 1 -- 172.21.15.105:0/3961539061 >> v2:172.21.15.148:3300/0 conn(0x7f0da00cc7d0 msgr2=0x7f0da00ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:33.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.820+0000 7f0da4e78640 1 --2- 172.21.15.105:0/3961539061 >> v2:172.21.15.148:3300/0 conn(0x7f0da00cc7d0 0x7f0da00ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:33.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.820+0000 7f0da4e78640 1 -- 172.21.15.105:0/3961539061 >> v1:172.21.15.105:6789/0 conn(0x7f0da006d610 legacy=0x7f0da00cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:33.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.820+0000 7f0da4e78640 1 -- 172.21.15.105:0/3961539061 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da0161ef0 con 0x7f0da00ad9f0 2023-11-28T18:26:33.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.820+0000 7f0d97fff640 1 -- 172.21.15.105:0/3961539061 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 497908314 0 0) 0x7f0da0162820 con 0x7f0da006d610 2023-11-28T18:26:33.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.824+0000 7f0da4e78640 1 --2- 172.21.15.105:0/3961539061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0da00ad9f0 0x7f0da00ade80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f0da0067c50 tx=0x7f0d9001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e78158f81bf6d5b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:33.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.824+0000 7f0d97fff640 1 -- 172.21.15.105:0/3961539061 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d900ab600 con 0x7f0da00ad9f0 2023-11-28T18:26:33.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.824+0000 7f0d97fff640 1 -- 172.21.15.105:0/3961539061 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d900a7050 con 0x7f0da00ad9f0 2023-11-28T18:26:33.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.824+0000 7f0d97fff640 1 -- 172.21.15.105:0/3961539061 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d900ab980 con 0x7f0da00ad9f0 2023-11-28T18:26:33.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.824+0000 7f0da78dd640 1 -- 172.21.15.105:0/3961539061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0da00ad9f0 msgr2=0x7f0da00ade80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:33.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.824+0000 7f0da78dd640 1 --2- 172.21.15.105:0/3961539061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0da00ad9f0 0x7f0da00ade80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f0da0067c50 tx=0x7f0d9001c430 comp rx=0 tx=0).stop 2023-11-28T18:26:33.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.824+0000 7f0da78dd640 1 -- 172.21.15.105:0/3961539061 shutdown_connections 2023-11-28T18:26:33.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.824+0000 7f0da78dd640 1 --2- 172.21.15.105:0/3961539061 >> v2:172.21.15.148:3300/0 conn(0x7f0da00cc7d0 0x7f0da00ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:33.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.824+0000 7f0da78dd640 1 --2- 172.21.15.105:0/3961539061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0da00ad9f0 0x7f0da00ade80 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:33.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.824+0000 7f0da78dd640 1 -- 172.21.15.105:0/3961539061 >> 172.21.15.105:0/3961539061 conn(0x7f0da00ceb90 msgr2=0x7f0da00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:33.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.824+0000 7f0da78dd640 1 -- 172.21.15.105:0/3961539061 shutdown_connections 2023-11-28T18:26:33.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.824+0000 7f0da78dd640 1 -- 172.21.15.105:0/3961539061 wait complete. 2023-11-28T18:26:33.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.824+0000 7f0da78dd640 1 Processor -- start 2023-11-28T18:26:33.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.824+0000 7f0da78dd640 1 -- start start 2023-11-28T18:26:33.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0da78dd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0da006d610 0x7f0da0154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:33.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0da78dd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0da00ad9f0 0x7f0da0156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:33.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0da78dd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0da00cc7d0 0x7f0da00b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:33.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0da78dd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da0154a50 con 0x7f0da006d610 2023-11-28T18:26:33.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0da78dd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da015ece0 con 0x7f0da00cc7d0 2023-11-28T18:26:33.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0da78dd640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da015efc0 con 0x7f0da00ad9f0 2023-11-28T18:26:33.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0da5679640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0da006d610 0x7f0da0154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:33.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0da5679640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0da006d610 0x7f0da0154bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44420/0 (socket says 172.21.15.105:44420) 2023-11-28T18:26:33.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0da5679640 1 -- 172.21.15.105:0/3334383054 learned_addr learned my addr 172.21.15.105:0/3334383054 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:33.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0da5679640 1 -- 172.21.15.105:0/3334383054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0da00ad9f0 msgr2=0x7f0da0156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:33.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0da5679640 1 --2- 172.21.15.105:0/3334383054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0da00ad9f0 0x7f0da0156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:33.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0da5679640 1 -- 172.21.15.105:0/3334383054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0da00cc7d0 msgr2=0x7f0da00b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:33.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0da5e7a640 1 --2- 172.21.15.105:0/3334383054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0da00cc7d0 0x7f0da00b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:33.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0da5679640 1 --2- 172.21.15.105:0/3334383054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0da00cc7d0 0x7f0da00b44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:33.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0da5679640 1 -- 172.21.15.105:0/3334383054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d90009420 con 0x7f0da006d610 2023-11-28T18:26:33.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0da5e7a640 1 --2- 172.21.15.105:0/3334383054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0da00cc7d0 0x7f0da00b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:33.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0da5679640 1 --2- 172.21.15.105:0/3334383054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0da006d610 0x7f0da0154bd0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00c480 tx=0x7f0d9c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:33.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0d95ffb640 1 -- 172.21.15.105:0/3334383054 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c0074e0 con 0x7f0da006d610 2023-11-28T18:26:33.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0d95ffb640 1 -- 172.21.15.105:0/3334383054 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9c007660 con 0x7f0da006d610 2023-11-28T18:26:33.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0da78dd640 1 -- 172.21.15.105:0/3334383054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da01560f0 con 0x7f0da006d610 2023-11-28T18:26:33.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0d95ffb640 1 -- 172.21.15.105:0/3334383054 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c004a30 con 0x7f0da006d610 2023-11-28T18:26:33.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0da78dd640 1 -- 172.21.15.105:0/3334383054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da01566b0 con 0x7f0da006d610 2023-11-28T18:26:33.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0da78dd640 1 -- 172.21.15.105:0/3334383054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da01568a0 con 0x7f0da006d610 2023-11-28T18:26:33.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.828+0000 7f0d95ffb640 1 -- 172.21.15.105:0/3334383054 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f0d9c004b90 con 0x7f0da006d610 2023-11-28T18:26:33.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.832+0000 7f0d95ffb640 1 --2- 172.21.15.105:0/3334383054 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0d840668d0 0x7f0d84068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:33.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.832+0000 7f0d95ffb640 1 -- 172.21.15.105:0/3334383054 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f0d9c083b50 con 0x7f0da006d610 2023-11-28T18:26:33.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.832+0000 7f0da4e78640 1 --2- 172.21.15.105:0/3334383054 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0d840668d0 0x7f0d84068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:33.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.832+0000 7f0da4e78640 1 --2- 172.21.15.105:0/3334383054 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0d840668d0 0x7f0d84068d90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0d900ad000 tx=0x7f0d90004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:33.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:33.836+0000 7f0d95ffb640 1 -- 172.21.15.105:0/3334383054 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0d9c054640 con 0x7f0da006d610 2023-11-28T18:26:34.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.016+0000 7f0da78dd640 1 -- 172.21.15.105:0/3334383054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0da0067570 con 0x7f0da006d610 2023-11-28T18:26:34.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.016+0000 7f0d95ffb640 1 -- 172.21.15.105:0/3334383054 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4955 (secure 0 0 0) 0x7f0d9c018020 con 0x7f0da006d610 2023-11-28T18:26:34.018 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:34.018 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":47,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1686803161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":1686803161},{"type":"v1","addr":"172.21.15.148:6835","nonce":1686803161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:26:00.550298+0000","modified":"2023-11-28T18:26:31.373860+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5842,"mds_1":5405,"mds_2":5408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5405":{"gid":5405,"name":"a","rank":1,"incarnation":43,"state":"up:active","state_seq":13,"addr":"172.21.15.105:6836/735646078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":735646078},{"type":"v1","addr":"172.21.15.105:6836","nonce":735646078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5408":{"gid":5408,"name":"c","rank":2,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6837/3765684108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":3765684108},{"type":"v1","addr":"172.21.15.105:6837","nonce":3765684108}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5842":{"gid":5842,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6834/2696773413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2696773413},{"type":"v1","addr":"172.21.15.148:6834","nonce":2696773413}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T18:26:34.019 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 47 2023-11-28T18:26:34.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.016+0000 7f0da78dd640 1 -- 172.21.15.105:0/3334383054 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0d840668d0 msgr2=0x7f0d84068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.016+0000 7f0da78dd640 1 --2- 172.21.15.105:0/3334383054 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0d840668d0 0x7f0d84068d90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0d900ad000 tx=0x7f0d90004740 comp rx=0 tx=0).stop 2023-11-28T18:26:34.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.016+0000 7f0da78dd640 1 -- 172.21.15.105:0/3334383054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0da006d610 msgr2=0x7f0da0154bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.016+0000 7f0da78dd640 1 --2- 172.21.15.105:0/3334383054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0da006d610 0x7f0da0154bd0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00c480 tx=0x7f0d9c00c940 comp rx=0 tx=0).stop 2023-11-28T18:26:34.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.016+0000 7f0da78dd640 1 -- 172.21.15.105:0/3334383054 shutdown_connections 2023-11-28T18:26:34.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.016+0000 7f0da78dd640 1 --2- 172.21.15.105:0/3334383054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0da00cc7d0 0x7f0da00b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.016+0000 7f0da78dd640 1 --2- 172.21.15.105:0/3334383054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0da00ad9f0 0x7f0da0156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.016+0000 7f0da78dd640 1 --2- 172.21.15.105:0/3334383054 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0d840668d0 0x7f0d84068d90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.016+0000 7f0da78dd640 1 --2- 172.21.15.105:0/3334383054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0da006d610 0x7f0da0154bd0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.016+0000 7f0da78dd640 1 -- 172.21.15.105:0/3334383054 >> 172.21.15.105:0/3334383054 conn(0x7f0da00ceb90 msgr2=0x7f0da00cadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:34.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.016+0000 7f0da78dd640 1 -- 172.21.15.105:0/3334383054 shutdown_connections 2023-11-28T18:26:34.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.016+0000 7f0da78dd640 1 -- 172.21.15.105:0/3334383054 wait complete. 2023-11-28T18:26:34.443 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:26:00.550298+0000', 'modified': '2023-11-28T18:26:31.373860+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 54, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 5842, 'mds_1': 5405, 'mds_2': 5408}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5405': {'gid': 5405, 'name': 'a', 'rank': 1, 'incarnation': 43, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.105:6836/735646078', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 735646078}, {'type': 'v1', 'addr': '172.21.15.105:6836', 'nonce': 735646078}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_5408': {'gid': 5408, 'name': 'c', 'rank': 2, 'incarnation': 39, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.105:6837/3765684108', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6835', 'nonce': 3765684108}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 3765684108}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_5842': {'gid': 5842, 'name': 'b', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6834/2696773413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6832', 'nonce': 2696773413}, {'type': 'v1', 'addr': '172.21.15.148:6834', 'nonce': 2696773413}]}, 'join_fscid': -1, 'export_targets': [1, 2], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:26:34.443 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-11-28T18:26:34.444 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T18:26:34.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.580+0000 7f4902417640 1 Processor -- start 2023-11-28T18:26:34.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.580+0000 7f4902417640 1 -- start start 2023-11-28T18:26:34.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.580+0000 7f4902417640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48fc15d4f0 0x7f48fc0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.580+0000 7f4902417640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48fc0b2d20 0x7f48fc0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.580+0000 7f4902417640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48fc0b25b0 con 0x7f48fc0d3c30 2023-11-28T18:26:34.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.580+0000 7f4902417640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48fc0b2900 con 0x7f48fc15d4f0 2023-11-28T18:26:34.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.580+0000 7f4902417640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc0d0a20 con 0x7f48fc0b2d20 2023-11-28T18:26:34.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.580+0000 7f49009b4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48fc0b2d20 0x7f48fc0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.580+0000 7f48fbfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48fc15d4f0 0x7f48fc0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.580+0000 7f49009b4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48fc0b2d20 0x7f48fc0b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:60064/0 (socket says 172.21.15.105:60064) 2023-11-28T18:26:34.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.580+0000 7f49009b4640 1 -- 172.21.15.105:0/675326806 learned_addr learned my addr 172.21.15.105:0/675326806 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:34.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.584+0000 7f48faffd640 1 -- 172.21.15.105:0/675326806 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2158997959 0 0) 0x7f48fc0b25b0 con 0x7f48fc0d3c30 2023-11-28T18:26:34.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.584+0000 7f49009b4640 1 -- 172.21.15.105:0/675326806 >> v2:172.21.15.148:3300/0 conn(0x7f48fc15d4f0 msgr2=0x7f48fc0d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:34.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.584+0000 7f49009b4640 1 --2- 172.21.15.105:0/675326806 >> v2:172.21.15.148:3300/0 conn(0x7f48fc15d4f0 0x7f48fc0d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.584+0000 7f49009b4640 1 -- 172.21.15.105:0/675326806 >> v1:172.21.15.148:6789/0 conn(0x7f48fc0d3c30 legacy=0x7f48fc0d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.584+0000 7f49009b4640 1 -- 172.21.15.105:0/675326806 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48fc0cea30 con 0x7f48fc0b2d20 2023-11-28T18:26:34.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.584+0000 7f48fbfff640 1 --2- 172.21.15.105:0/675326806 >> v2:172.21.15.148:3300/0 conn(0x7f48fc15d4f0 0x7f48fc0d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:26:34.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.584+0000 7f49009b4640 1 --2- 172.21.15.105:0/675326806 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48fc0b2d20 0x7f48fc0b3190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009e50 tx=0x7f48f401c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36ee34bec5d9b7db server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:34.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.584+0000 7f48faffd640 1 -- 172.21.15.105:0/675326806 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f40a5810 con 0x7f48fc0b2d20 2023-11-28T18:26:34.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.584+0000 7f48faffd640 1 -- 172.21.15.105:0/675326806 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f40a5970 con 0x7f48fc0b2d20 2023-11-28T18:26:34.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.584+0000 7f48faffd640 1 -- 172.21.15.105:0/675326806 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f40a5d00 con 0x7f48fc0b2d20 2023-11-28T18:26:34.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.584+0000 7f4902417640 1 -- 172.21.15.105:0/675326806 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48fc0b2d20 msgr2=0x7f48fc0b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.584+0000 7f4902417640 1 --2- 172.21.15.105:0/675326806 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48fc0b2d20 0x7f48fc0b3190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009e50 tx=0x7f48f401c950 comp rx=0 tx=0).stop 2023-11-28T18:26:34.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.584+0000 7f4902417640 1 -- 172.21.15.105:0/675326806 shutdown_connections 2023-11-28T18:26:34.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.584+0000 7f4902417640 1 --2- 172.21.15.105:0/675326806 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48fc0b2d20 0x7f48fc0b3190 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.584+0000 7f4902417640 1 --2- 172.21.15.105:0/675326806 >> v2:172.21.15.148:3300/0 conn(0x7f48fc15d4f0 0x7f48fc0d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.584+0000 7f4902417640 1 -- 172.21.15.105:0/675326806 >> 172.21.15.105:0/675326806 conn(0x7f48fc0d3030 msgr2=0x7f48fc0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:34.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.584+0000 7f4902417640 1 -- 172.21.15.105:0/675326806 shutdown_connections 2023-11-28T18:26:34.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f4902417640 1 -- 172.21.15.105:0/675326806 wait complete. 2023-11-28T18:26:34.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f4902417640 1 Processor -- start 2023-11-28T18:26:34.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f4902417640 1 -- start start 2023-11-28T18:26:34.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f4902417640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48fc0b2d20 0x7f48fc0b86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f4902417640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48fc0d3c30 0x7f48fc0b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f4902417640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48fc15d4f0 0x7f48fc1577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f4902417640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc0ca100 con 0x7f48fc0d3c30 2023-11-28T18:26:34.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f4902417640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc0cac10 con 0x7f48fc0b2d20 2023-11-28T18:26:34.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f4902417640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc0ca2e0 con 0x7f48fc15d4f0 2023-11-28T18:26:34.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f48fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48fc0d3c30 0x7f48fc0b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f48fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48fc0d3c30 0x7f48fc0b66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44434/0 (socket says 172.21.15.105:44434) 2023-11-28T18:26:34.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f48fb7fe640 1 -- 172.21.15.105:0/2317064799 learned_addr learned my addr 172.21.15.105:0/2317064799 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:34.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f48fbfff640 1 --2- 172.21.15.105:0/2317064799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48fc0b2d20 0x7f48fc0b86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f49009b4640 1 --2- 172.21.15.105:0/2317064799 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48fc15d4f0 0x7f48fc1577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f48fb7fe640 1 -- 172.21.15.105:0/2317064799 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48fc15d4f0 msgr2=0x7f48fc1577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f48fb7fe640 1 --2- 172.21.15.105:0/2317064799 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48fc15d4f0 0x7f48fc1577e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f48fb7fe640 1 -- 172.21.15.105:0/2317064799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48fc0b2d20 msgr2=0x7f48fc0b86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f48fb7fe640 1 --2- 172.21.15.105:0/2317064799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48fc0b2d20 0x7f48fc0b86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f48fb7fe640 1 -- 172.21.15.105:0/2317064799 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec009420 con 0x7f48fc0d3c30 2023-11-28T18:26:34.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f49009b4640 1 --2- 172.21.15.105:0/2317064799 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48fc15d4f0 0x7f48fc1577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:34.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f48fb7fe640 1 --2- 172.21.15.105:0/2317064799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48fc0d3c30 0x7f48fc0b66f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f48f000c450 tx=0x7f48f000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:34.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f48f8ff9640 1 -- 172.21.15.105:0/2317064799 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0005480 con 0x7f48fc0d3c30 2023-11-28T18:26:34.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f48f8ff9640 1 -- 172.21.15.105:0/2317064799 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f0003600 con 0x7f48fc0d3c30 2023-11-28T18:26:34.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f4902417640 1 -- 172.21.15.105:0/2317064799 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f4009b30 con 0x7f48fc0d3c30 2023-11-28T18:26:34.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.588+0000 7f48f8ff9640 1 -- 172.21.15.105:0/2317064799 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f00039b0 con 0x7f48fc0d3c30 2023-11-28T18:26:34.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.592+0000 7f4902417640 1 -- 172.21.15.105:0/2317064799 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48fc0ca8a0 con 0x7f48fc0d3c30 2023-11-28T18:26:34.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.592+0000 7f48f8ff9640 1 -- 172.21.15.105:0/2317064799 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f48f0003b10 con 0x7f48fc0d3c30 2023-11-28T18:26:34.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.592+0000 7f48f8ff9640 1 --2- 172.21.15.105:0/2317064799 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48e80668d0 0x7f48e8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.592+0000 7f48f8ff9640 1 -- 172.21.15.105:0/2317064799 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f48f0083b50 con 0x7f48fc0d3c30 2023-11-28T18:26:34.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.592+0000 7f48fbfff640 1 --2- 172.21.15.105:0/2317064799 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48e80668d0 0x7f48e8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.592+0000 7f490577c000 1 Processor -- start 2023-11-28T18:26:34.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.592+0000 7f490577c000 1 -- start start 2023-11-28T18:26:34.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.592+0000 7f48fbfff640 1 --2- 172.21.15.105:0/2317064799 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48e80668d0 0x7f48e8068d90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0b2230 tx=0x7f48ec009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:34.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f490577c000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab67d30 0x55db0ab68110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f490577c000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55db0ab68650 0x55db0ab1afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f490577c000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db0ab1b500 0x55db0ab1b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f490577c000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db0a9e4110 con 0x55db0ab67d30 2023-11-28T18:26:34.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f490577c000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db0a9749a0 con 0x55db0ab1b500 2023-11-28T18:26:34.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f490577c000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db0a971ed0 con 0x55db0ab68650 2023-11-28T18:26:34.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f48fbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab67d30 0x55db0ab68110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f48fbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab67d30 0x55db0ab68110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44438/0 (socket says 172.21.15.105:44438) 2023-11-28T18:26:34.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f48fbfff640 1 -- 172.21.15.105:0/1209417566 learned_addr learned my addr 172.21.15.105:0/1209417566 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:34.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f49009b4640 1 --2- 172.21.15.105:0/1209417566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db0ab1b500 0x55db0ab1b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f48fb7fe640 1 --2- 172.21.15.105:0/1209417566 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55db0ab68650 0x55db0ab1afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f48fbfff640 1 -- 172.21.15.105:0/1209417566 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55db0ab68650 msgr2=0x55db0ab1afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:34.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f48fbfff640 1 --2- 172.21.15.105:0/1209417566 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55db0ab68650 0x55db0ab1afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f48fbfff640 1 -- 172.21.15.105:0/1209417566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db0ab1b500 msgr2=0x55db0ab1b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:34.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f48fbfff640 1 --2- 172.21.15.105:0/1209417566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db0ab1b500 0x55db0ab1b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f48fbfff640 1 -- 172.21.15.105:0/1209417566 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db0a665890 con 0x55db0ab67d30 2023-11-28T18:26:34.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f48fbfff640 1 --2- 172.21.15.105:0/1209417566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab67d30 0x55db0ab68110 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f48ec006a60 tx=0x7f48ec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e795be7291c16448 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:34.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f48e1ffb640 1 -- 172.21.15.105:0/1209417566 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec014d70 con 0x55db0ab67d30 2023-11-28T18:26:34.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f48e1ffb640 1 -- 172.21.15.105:0/1209417566 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ec0213f0 con 0x55db0ab67d30 2023-11-28T18:26:34.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f48e1ffb640 1 -- 172.21.15.105:0/1209417566 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec021780 con 0x55db0ab67d30 2023-11-28T18:26:34.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f490577c000 1 -- 172.21.15.105:0/1209417566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab67d30 msgr2=0x55db0ab68110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f490577c000 1 --2- 172.21.15.105:0/1209417566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab67d30 0x55db0ab68110 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f48ec006a60 tx=0x7f48ec014000 comp rx=0 tx=0).stop 2023-11-28T18:26:34.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f490577c000 1 -- 172.21.15.105:0/1209417566 shutdown_connections 2023-11-28T18:26:34.613 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f490577c000 1 --2- 172.21.15.105:0/1209417566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db0ab1b500 0x55db0ab1b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.613 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f490577c000 1 --2- 172.21.15.105:0/1209417566 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55db0ab68650 0x55db0ab1afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f490577c000 1 --2- 172.21.15.105:0/1209417566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab67d30 0x55db0ab68110 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f490577c000 1 -- 172.21.15.105:0/1209417566 >> 172.21.15.105:0/1209417566 conn(0x55db0ab488c0 msgr2=0x55db0ab4acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:34.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f490577c000 1 -- 172.21.15.105:0/1209417566 shutdown_connections 2023-11-28T18:26:34.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f490577c000 1 -- 172.21.15.105:0/1209417566 wait complete. 2023-11-28T18:26:34.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f490577c000 1 -- start start 2023-11-28T18:26:34.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.596+0000 7f490577c000 1 Processor -- start 2023-11-28T18:26:34.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55db0ab1b500 0x55db0ab214e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab67d30 0x55db0ab21a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db0ab68650 0x55db0ab21f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db0a96d7c0 con 0x55db0ab67d30 2023-11-28T18:26:34.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db0a925bf0 con 0x55db0ab68650 2023-11-28T18:26:34.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db0a92ba90 con 0x55db0ab1b500 2023-11-28T18:26:34.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 10 client.0 _ll_register_callbacks cb 0x55db0ab08e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:34.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f48fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab67d30 0x55db0ab21a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f48fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab67d30 0x55db0ab21a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44446/0 (socket says 172.21.15.105:44446) 2023-11-28T18:26:34.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f48fb7fe640 1 -- 172.21.15.105:0/197849670 learned_addr learned my addr 172.21.15.105:0/197849670 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:34.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f48fbfff640 1 --2- 172.21.15.105:0/197849670 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55db0ab1b500 0x55db0ab214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f49009b4640 1 --2- 172.21.15.105:0/197849670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db0ab68650 0x55db0ab21f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f48fb7fe640 1 -- 172.21.15.105:0/197849670 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55db0ab1b500 msgr2=0x55db0ab214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:34.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f48fb7fe640 1 --2- 172.21.15.105:0/197849670 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55db0ab1b500 0x55db0ab214e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f48fb7fe640 1 -- 172.21.15.105:0/197849670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db0ab68650 msgr2=0x55db0ab21f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:34.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f48fb7fe640 1 --2- 172.21.15.105:0/197849670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db0ab68650 0x55db0ab21f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f48fb7fe640 1 -- 172.21.15.105:0/197849670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ec009cf0 con 0x55db0ab67d30 2023-11-28T18:26:34.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f48fb7fe640 1 --2- 172.21.15.105:0/197849670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab67d30 0x55db0ab21a20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f48f0001ec0 tx=0x7f48f005cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529ecc17a45860a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:34.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f48daffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:34.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f48daffd640 1 -- 172.21.15.105:0/197849670 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0053f80 con 0x55db0ab67d30 2023-11-28T18:26:34.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f48daffd640 1 -- 172.21.15.105:0/197849670 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f48f005c650 con 0x55db0ab67d30 2023-11-28T18:26:34.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 1 -- 172.21.15.105:0/197849670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db0a92f470 con 0x55db0ab67d30 2023-11-28T18:26:34.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f48daffd640 1 -- 172.21.15.105:0/197849670 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48f005c810 con 0x55db0ab67d30 2023-11-28T18:26:34.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 10 client.6384 fetch_fsmap learned FSMap version 47 2023-11-28T18:26:34.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 1 -- 172.21.15.105:0/197849670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55db0a92e190 con 0x55db0ab67d30 2023-11-28T18:26:34.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f48daffd640 1 -- 172.21.15.105:0/197849670 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55db0a92f470 con 0x55db0ab67d30 2023-11-28T18:26:34.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f48daffd640 1 -- 172.21.15.105:0/197849670 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 47) v1 ==== 3326+0+0 (secure 0 0 0) 0x7f48f0055b10 con 0x55db0ab67d30 2023-11-28T18:26:34.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 10 client.6384 fetch_fsmap finished waiting for FSMap version 47 2023-11-28T18:26:34.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 10 client.6384 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:26:34.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 20 client.6384 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:34.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 1 --2- 172.21.15.105:0/197849670 >> [v2:172.21.15.105:6834/735646078,v1:172.21.15.105:6836/735646078] conn(0x55db0ab22970 0x55db0ab79ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 4 client.6384 mds_command: new command op to 5405 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:26:34.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 1 -- 172.21.15.105:0/197849670 --> [v2:172.21.15.105:6834/735646078,v1:172.21.15.105:6836/735646078] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db0a932b50 con 0x55db0ab22970 2023-11-28T18:26:34.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f48fbfff640 1 --2- 172.21.15.105:0/197849670 >> [v2:172.21.15.105:6834/735646078,v1:172.21.15.105:6836/735646078] conn(0x55db0ab22970 0x55db0ab79ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f48fbfff640 1 --2- 172.21.15.105:0/197849670 >> [v2:172.21.15.105:6834/735646078,v1:172.21.15.105:6836/735646078] conn(0x55db0ab22970 0x55db0ab79ff0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=18d43303c3722647 server_cookie=757f7b85b3110781 in_seq=0 out_seq=0 2023-11-28T18:26:34.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f48daffd640 10 client.6384 ms_handle_connect on v2:172.21.15.105:6834/735646078 2023-11-28T18:26:34.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f48daffd640 1 -- 172.21.15.105:0/197849670 <== mds.1 v2:172.21.15.105:6834/735646078 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55db0a932b50 con 0x55db0ab22970 2023-11-28T18:26:34.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f48daffd640 10 client.6384 handle_command_reply: tid=0 2023-11-28T18:26:34.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 1 client.6384 shutdown 2023-11-28T18:26:34.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 1 -- 172.21.15.105:0/197849670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab67d30 msgr2=0x55db0ab21a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 1 --2- 172.21.15.105:0/197849670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab67d30 0x55db0ab21a20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f48f0001ec0 tx=0x7f48f005cae0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 1 -- 172.21.15.105:0/197849670 shutdown_connections 2023-11-28T18:26:34.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 1 --2- 172.21.15.105:0/197849670 >> [v2:172.21.15.105:6834/735646078,v1:172.21.15.105:6836/735646078] conn(0x55db0ab22970 0x55db0ab79ff0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 1 --2- 172.21.15.105:0/197849670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db0ab68650 0x55db0ab21f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 1 --2- 172.21.15.105:0/197849670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab67d30 0x55db0ab21a20 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 1 --2- 172.21.15.105:0/197849670 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55db0ab1b500 0x55db0ab214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f48daffd640 0 client.6384 ms_handle_reset on v2:172.21.15.105:6834/735646078 2023-11-28T18:26:34.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.600+0000 7f490577c000 1 -- 172.21.15.105:0/197849670 >> 172.21.15.105:0/197849670 conn(0x55db0ab488c0 msgr2=0x55db0ab1a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:34.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.604+0000 7f490577c000 1 -- 172.21.15.105:0/197849670 shutdown_connections 2023-11-28T18:26:34.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.604+0000 7f490577c000 1 -- 172.21.15.105:0/197849670 wait complete. 2023-11-28T18:26:34.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.604+0000 7f490577c000 20 client.6384 trim_cache size 0 max 16384 2023-11-28T18:26:34.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.616+0000 7f490577c000 1 Processor -- start 2023-11-28T18:26:34.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.616+0000 7f490577c000 1 -- start start 2023-11-28T18:26:34.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab1b500 0x55db0ab473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55db0ab67d30 0x55db0ab478e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db0ab68650 0x55db0ab79ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db0a626940 con 0x55db0ab1b500 2023-11-28T18:26:34.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db0a92c8f0 con 0x55db0ab68650 2023-11-28T18:26:34.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db0a9088a0 con 0x55db0ab67d30 2023-11-28T18:26:34.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f48fbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab1b500 0x55db0ab473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f48fbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab1b500 0x55db0ab473a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44458/0 (socket says 172.21.15.105:44458) 2023-11-28T18:26:34.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f48fbfff640 1 -- 172.21.15.105:0/4012582630 learned_addr learned my addr 172.21.15.105:0/4012582630 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:34.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f48fbfff640 1 -- 172.21.15.105:0/4012582630 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55db0ab67d30 msgr2=0x55db0ab478e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:34.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f48fbfff640 1 --2- 172.21.15.105:0/4012582630 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55db0ab67d30 0x55db0ab478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f48fbfff640 1 -- 172.21.15.105:0/4012582630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db0ab68650 msgr2=0x55db0ab79ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:34.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f48fbfff640 1 --2- 172.21.15.105:0/4012582630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db0ab68650 0x55db0ab79ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f48fbfff640 1 -- 172.21.15.105:0/4012582630 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db0a914490 con 0x55db0ab1b500 2023-11-28T18:26:34.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f48fbfff640 1 --2- 172.21.15.105:0/4012582630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab1b500 0x55db0ab473a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f48ec014460 tx=0x7f48ec016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec4700cca206227 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:34.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f48e27fc640 1 -- 172.21.15.105:0/4012582630 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec030030 con 0x55db0ab1b500 2023-11-28T18:26:34.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f48e27fc640 1 -- 172.21.15.105:0/4012582630 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ec031040 con 0x55db0ab1b500 2023-11-28T18:26:34.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f48e27fc640 1 -- 172.21.15.105:0/4012582630 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec034a90 con 0x55db0ab1b500 2023-11-28T18:26:34.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 -- 172.21.15.105:0/4012582630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab1b500 msgr2=0x55db0ab473a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 --2- 172.21.15.105:0/4012582630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab1b500 0x55db0ab473a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f48ec014460 tx=0x7f48ec016000 comp rx=0 tx=0).stop 2023-11-28T18:26:34.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 -- 172.21.15.105:0/4012582630 shutdown_connections 2023-11-28T18:26:34.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 --2- 172.21.15.105:0/4012582630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db0ab68650 0x55db0ab79ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 --2- 172.21.15.105:0/4012582630 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55db0ab67d30 0x55db0ab478e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 --2- 172.21.15.105:0/4012582630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab1b500 0x55db0ab473a0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 -- 172.21.15.105:0/4012582630 >> 172.21.15.105:0/4012582630 conn(0x55db0ab488c0 msgr2=0x55db0ab19720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:34.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 -- 172.21.15.105:0/4012582630 shutdown_connections 2023-11-28T18:26:34.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 -- 172.21.15.105:0/4012582630 wait complete. 2023-11-28T18:26:34.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 -- start start 2023-11-28T18:26:34.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 Processor -- start 2023-11-28T18:26:34.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db0ab1b500 0x55db0abc8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab67d30 0x55db0abc8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55db0ab68650 0x55db0ab83a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db0a626940 con 0x55db0ab67d30 2023-11-28T18:26:34.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db0a9750c0 con 0x55db0ab1b500 2023-11-28T18:26:34.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db0a8f1a00 con 0x55db0ab68650 2023-11-28T18:26:34.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.620+0000 7f490577c000 10 client.0 _ll_register_callbacks cb 0x55db0a9e4060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:34.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab67d30 0x55db0abc8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab67d30 0x55db0abc8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44474/0 (socket says 172.21.15.105:44474) 2023-11-28T18:26:34.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48fb7fe640 1 -- 172.21.15.105:0/3572736896 learned_addr learned my addr 172.21.15.105:0/3572736896 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:34.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48fbfff640 1 --2- 172.21.15.105:0/3572736896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db0ab1b500 0x55db0abc8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f49009b4640 1 --2- 172.21.15.105:0/3572736896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55db0ab68650 0x55db0ab83a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48fb7fe640 1 -- 172.21.15.105:0/3572736896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55db0ab68650 msgr2=0x55db0ab83a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:34.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48fb7fe640 1 --2- 172.21.15.105:0/3572736896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55db0ab68650 0x55db0ab83a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48fb7fe640 1 -- 172.21.15.105:0/3572736896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db0ab1b500 msgr2=0x55db0abc8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:34.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48fb7fe640 1 --2- 172.21.15.105:0/3572736896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db0ab1b500 0x55db0abc8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48fb7fe640 1 -- 172.21.15.105:0/3572736896 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db0a665890 con 0x55db0ab67d30 2023-11-28T18:26:34.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f49009b4640 1 --2- 172.21.15.105:0/3572736896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55db0ab68650 0x55db0ab83a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:34.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48fbfff640 1 --2- 172.21.15.105:0/3572736896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db0ab1b500 0x55db0abc8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:34.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48fb7fe640 1 --2- 172.21.15.105:0/3572736896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab67d30 0x55db0abc8d50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f48f0050bf0 tx=0x7f48f0001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa16e756e32caae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:34.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48daffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:34.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48daffd640 1 -- 172.21.15.105:0/3572736896 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f000e040 con 0x55db0ab67d30 2023-11-28T18:26:34.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48daffd640 1 -- 172.21.15.105:0/3572736896 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f48f007baf0 con 0x55db0ab67d30 2023-11-28T18:26:34.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f490577c000 1 -- 172.21.15.105:0/3572736896 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db0a96c110 con 0x55db0ab67d30 2023-11-28T18:26:34.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48daffd640 1 -- 172.21.15.105:0/3572736896 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48f0054900 con 0x55db0ab67d30 2023-11-28T18:26:34.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f490577c000 10 client.6390 fetch_fsmap learned FSMap version 47 2023-11-28T18:26:34.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f490577c000 1 -- 172.21.15.105:0/3572736896 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55db0a665cc0 con 0x55db0ab67d30 2023-11-28T18:26:34.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48daffd640 1 -- 172.21.15.105:0/3572736896 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55db0a96c110 con 0x55db0ab67d30 2023-11-28T18:26:34.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48daffd640 1 -- 172.21.15.105:0/3572736896 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 47) v1 ==== 3326+0+0 (secure 0 0 0) 0x7f48f007bcb0 con 0x55db0ab67d30 2023-11-28T18:26:34.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f490577c000 10 client.6390 fetch_fsmap finished waiting for FSMap version 47 2023-11-28T18:26:34.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f490577c000 10 client.6390 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:26:34.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f490577c000 20 client.6390 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:34.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f490577c000 1 --2- 172.21.15.105:0/3572736896 >> [v2:172.21.15.105:6834/735646078,v1:172.21.15.105:6836/735646078] conn(0x55db0abcf670 0x55db0abd1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f490577c000 4 client.6390 mds_command: new command op to 5405 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:26:34.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f490577c000 1 -- 172.21.15.105:0/3572736896 --> [v2:172.21.15.105:6834/735646078,v1:172.21.15.105:6836/735646078] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55db0a99a650 con 0x55db0abcf670 2023-11-28T18:26:34.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48fbfff640 1 --2- 172.21.15.105:0/3572736896 >> [v2:172.21.15.105:6834/735646078,v1:172.21.15.105:6836/735646078] conn(0x55db0abcf670 0x55db0abd1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48fbfff640 1 --2- 172.21.15.105:0/3572736896 >> [v2:172.21.15.105:6834/735646078,v1:172.21.15.105:6836/735646078] conn(0x55db0abcf670 0x55db0abd1a60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=760a6bf4f4435525 server_cookie=976b24441889ae84 in_seq=0 out_seq=0 2023-11-28T18:26:34.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48daffd640 10 client.6390 ms_handle_connect on v2:172.21.15.105:6834/735646078 2023-11-28T18:26:34.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48daffd640 1 -- 172.21.15.105:0/3572736896 <== mds.1 v2:172.21.15.105:6834/735646078 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55db0a99a650 con 0x55db0abcf670 2023-11-28T18:26:34.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48daffd640 10 client.6390 handle_command_reply: tid=0 2023-11-28T18:26:34.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f490577c000 1 client.6390 shutdown 2023-11-28T18:26:34.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f490577c000 1 -- 172.21.15.105:0/3572736896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab67d30 msgr2=0x55db0abc8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f490577c000 1 --2- 172.21.15.105:0/3572736896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab67d30 0x55db0abc8d50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f48f0050bf0 tx=0x7f48f0001090 comp rx=0 tx=0).stop 2023-11-28T18:26:34.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f490577c000 1 -- 172.21.15.105:0/3572736896 shutdown_connections 2023-11-28T18:26:34.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f490577c000 1 --2- 172.21.15.105:0/3572736896 >> [v2:172.21.15.105:6834/735646078,v1:172.21.15.105:6836/735646078] conn(0x55db0abcf670 0x55db0abd1a60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f490577c000 1 --2- 172.21.15.105:0/3572736896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55db0ab68650 0x55db0ab83a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f490577c000 1 --2- 172.21.15.105:0/3572736896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db0ab67d30 0x55db0abc8d50 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f48daffd640 0 client.6390 ms_handle_reset on v2:172.21.15.105:6834/735646078 2023-11-28T18:26:34.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f490577c000 1 --2- 172.21.15.105:0/3572736896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db0ab1b500 0x55db0abc8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f490577c000 1 -- 172.21.15.105:0/3572736896 >> 172.21.15.105:0/3572736896 conn(0x55db0ab488c0 msgr2=0x55db0ab19fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:34.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f490577c000 1 -- 172.21.15.105:0/3572736896 shutdown_connections 2023-11-28T18:26:34.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f490577c000 1 -- 172.21.15.105:0/3572736896 wait complete. 2023-11-28T18:26:34.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f490577c000 20 client.6390 trim_cache size 0 max 16384 2023-11-28T18:26:34.635 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:26:34.635 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:26:34.635 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 1, 2023-11-28T18:26:34.635 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5405, 2023-11-28T18:26:34.635 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:26:34.635 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:26:34.635 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:26:34.635 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 6.2753400880000001, 2023-11-28T18:26:34.635 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 47, 2023-11-28T18:26:34.635 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 54, 2023-11-28T18:26:34.635 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 54, 2023-11-28T18:26:34.636 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 38.339406095000001 2023-11-28T18:26:34.636 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:26:34.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f4902417640 1 -- 172.21.15.105:0/2317064799 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48e80668d0 msgr2=0x7f48e8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f4902417640 1 --2- 172.21.15.105:0/2317064799 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48e80668d0 0x7f48e8068d90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0b2230 tx=0x7f48ec009190 comp rx=0 tx=0).stop 2023-11-28T18:26:34.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f4902417640 1 -- 172.21.15.105:0/2317064799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48fc0d3c30 msgr2=0x7f48fc0b66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f4902417640 1 --2- 172.21.15.105:0/2317064799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48fc0d3c30 0x7f48fc0b66f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f48f000c450 tx=0x7f48f000c910 comp rx=0 tx=0).stop 2023-11-28T18:26:34.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f4902417640 1 -- 172.21.15.105:0/2317064799 shutdown_connections 2023-11-28T18:26:34.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f4902417640 1 --2- 172.21.15.105:0/2317064799 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48fc15d4f0 0x7f48fc1577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f4902417640 1 --2- 172.21.15.105:0/2317064799 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f48e80668d0 0x7f48e8068d90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f4902417640 1 --2- 172.21.15.105:0/2317064799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48fc0d3c30 0x7f48fc0b66f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f4902417640 1 --2- 172.21.15.105:0/2317064799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48fc0b2d20 0x7f48fc0b86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.624+0000 7f4902417640 1 -- 172.21.15.105:0/2317064799 >> 172.21.15.105:0/2317064799 conn(0x7f48fc0d3030 msgr2=0x7f48fc072eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:34.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.628+0000 7f4902417640 1 -- 172.21.15.105:0/2317064799 shutdown_connections 2023-11-28T18:26:34.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.628+0000 7f4902417640 1 -- 172.21.15.105:0/2317064799 wait complete. 2023-11-28T18:26:34.648 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T18:26:34.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.784+0000 7fd85571d640 1 Processor -- start 2023-11-28T18:26:34.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.784+0000 7fd85571d640 1 -- start start 2023-11-28T18:26:34.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.784+0000 7fd85571d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd8500b4290 0x7fd8500b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.784+0000 7fd85571d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd85015b030 0x7fd8500709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.784+0000 7fd85571d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8500b4bb0 con 0x7fd850071960 2023-11-28T18:26:34.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.784+0000 7fd85571d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8500701d0 con 0x7fd85015b030 2023-11-28T18:26:34.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.784+0000 7fd85571d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85006df80 con 0x7fd8500b4290 2023-11-28T18:26:34.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.784+0000 7fd84effd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd8500b4290 0x7fd8500b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.784+0000 7fd84f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd85015b030 0x7fd8500709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.784+0000 7fd84effd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd8500b4290 0x7fd8500b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:60144/0 (socket says 172.21.15.105:60144) 2023-11-28T18:26:34.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.784+0000 7fd84effd640 1 -- 172.21.15.105:0/468342397 learned_addr learned my addr 172.21.15.105:0/468342397 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:34.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.788+0000 7fd84effd640 1 -- 172.21.15.105:0/468342397 >> v2:172.21.15.148:3300/0 conn(0x7fd85015b030 msgr2=0x7fd8500709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:34.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.788+0000 7fd84effd640 1 --2- 172.21.15.105:0/468342397 >> v2:172.21.15.148:3300/0 conn(0x7fd85015b030 0x7fd8500709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.788+0000 7fd84effd640 1 -- 172.21.15.105:0/468342397 >> v1:172.21.15.148:6789/0 conn(0x7fd850071960 legacy=0x7fd85015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.788+0000 7fd84effd640 1 -- 172.21.15.105:0/468342397 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd850071df0 con 0x7fd8500b4290 2023-11-28T18:26:34.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.788+0000 7fd84effd640 1 --2- 172.21.15.105:0/468342397 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd8500b4290 0x7fd8500b4670 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd840009870 tx=0x7fd84001c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=747e553fd226243d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:34.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.788+0000 7fd84e7fc640 1 -- 172.21.15.105:0/468342397 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8400ab620 con 0x7fd8500b4290 2023-11-28T18:26:34.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.788+0000 7fd84e7fc640 1 -- 172.21.15.105:0/468342397 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8400ab780 con 0x7fd8500b4290 2023-11-28T18:26:34.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.788+0000 7fd84e7fc640 1 -- 172.21.15.105:0/468342397 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8400abb10 con 0x7fd8500b4290 2023-11-28T18:26:34.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd85571d640 1 -- 172.21.15.105:0/468342397 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd8500b4290 msgr2=0x7fd8500b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd85571d640 1 --2- 172.21.15.105:0/468342397 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd8500b4290 0x7fd8500b4670 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd840009870 tx=0x7fd84001c510 comp rx=0 tx=0).stop 2023-11-28T18:26:34.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd85571d640 1 -- 172.21.15.105:0/468342397 shutdown_connections 2023-11-28T18:26:34.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd85571d640 1 --2- 172.21.15.105:0/468342397 >> v2:172.21.15.148:3300/0 conn(0x7fd85015b030 0x7fd8500709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd85571d640 1 --2- 172.21.15.105:0/468342397 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd8500b4290 0x7fd8500b4670 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd85571d640 1 -- 172.21.15.105:0/468342397 >> 172.21.15.105:0/468342397 conn(0x7fd8500ceb90 msgr2=0x7fd8500cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:34.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd85571d640 1 -- 172.21.15.105:0/468342397 shutdown_connections 2023-11-28T18:26:34.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd85571d640 1 -- 172.21.15.105:0/468342397 wait complete. 2023-11-28T18:26:34.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd85571d640 1 Processor -- start 2023-11-28T18:26:34.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd85571d640 1 -- start start 2023-11-28T18:26:34.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd85571d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd8500b4290 0x7fd85014b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd85571d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd85015b030 0x7fd8501553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd85571d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd85009a3f0 0x7fd85009a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd85571d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85009ade0 con 0x7fd85015b030 2023-11-28T18:26:34.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd85571d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85014bda0 con 0x7fd85009a3f0 2023-11-28T18:26:34.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd85571d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd850153e10 con 0x7fd8500b4290 2023-11-28T18:26:34.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd8467fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd85015b030 0x7fd8501553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd8467fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd85015b030 0x7fd8501553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44478/0 (socket says 172.21.15.105:44478) 2023-11-28T18:26:34.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd8467fc640 1 -- 172.21.15.105:0/3178095791 learned_addr learned my addr 172.21.15.105:0/3178095791 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:34.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd84effd640 1 --2- 172.21.15.105:0/3178095791 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd8500b4290 0x7fd85014b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd84f7fe640 1 --2- 172.21.15.105:0/3178095791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd85009a3f0 0x7fd85009a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd8467fc640 1 -- 172.21.15.105:0/3178095791 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd8500b4290 msgr2=0x7fd85014b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd8467fc640 1 --2- 172.21.15.105:0/3178095791 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd8500b4290 0x7fd85014b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd8467fc640 1 -- 172.21.15.105:0/3178095791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd85009a3f0 msgr2=0x7fd85009a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd8467fc640 1 --2- 172.21.15.105:0/3178095791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd85009a3f0 0x7fd85009a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.792+0000 7fd8467fc640 1 -- 172.21.15.105:0/3178095791 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8400094d0 con 0x7fd85015b030 2023-11-28T18:26:34.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.796+0000 7fd8467fc640 1 --2- 172.21.15.105:0/3178095791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd85015b030 0x7fd8501553f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd83800c500 tx=0x7fd83800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:34.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.796+0000 7fd847fff640 1 -- 172.21.15.105:0/3178095791 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd838005220 con 0x7fd85015b030 2023-11-28T18:26:34.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.796+0000 7fd847fff640 1 -- 172.21.15.105:0/3178095791 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd838005380 con 0x7fd85015b030 2023-11-28T18:26:34.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.796+0000 7fd85571d640 1 -- 172.21.15.105:0/3178095791 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd850153200 con 0x7fd85015b030 2023-11-28T18:26:34.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.796+0000 7fd847fff640 1 -- 172.21.15.105:0/3178095791 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd838007270 con 0x7fd85015b030 2023-11-28T18:26:34.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.796+0000 7fd85571d640 1 -- 172.21.15.105:0/3178095791 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8501537c0 con 0x7fd85015b030 2023-11-28T18:26:34.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.796+0000 7fd847fff640 1 -- 172.21.15.105:0/3178095791 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd8380073d0 con 0x7fd85015b030 2023-11-28T18:26:34.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.796+0000 7fd847fff640 1 --2- 172.21.15.105:0/3178095791 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd830066900 0x7fd830068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.796+0000 7fd858a83000 1 Processor -- start 2023-11-28T18:26:34.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.796+0000 7fd847fff640 1 -- 172.21.15.105:0/3178095791 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7fd838083e30 con 0x7fd85015b030 2023-11-28T18:26:34.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.796+0000 7fd84effd640 1 --2- 172.21.15.105:0/3178095791 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd830066900 0x7fd830068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd858a83000 1 -- start start 2023-11-28T18:26:34.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd84effd640 1 --2- 172.21.15.105:0/3178095791 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd830066900 0x7fd830068dc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd8400af000 tx=0x7fd8400b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:34.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd858a83000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55722d692330 0x55722d692710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd858a83000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692c50 0x55722d676d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd858a83000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55722d6772a0 0x55722d677680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd858a83000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55722d50a110 con 0x55722d692c50 2023-11-28T18:26:34.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd858a83000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55722d49a9a0 con 0x55722d692330 2023-11-28T18:26:34.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd858a83000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55722d497ed0 con 0x55722d6772a0 2023-11-28T18:26:34.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd8467fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692c50 0x55722d676d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd8467fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692c50 0x55722d676d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44488/0 (socket says 172.21.15.105:44488) 2023-11-28T18:26:34.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd8467fc640 1 -- 172.21.15.105:0/2585668654 learned_addr learned my addr 172.21.15.105:0/2585668654 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:34.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd84effd640 1 --2- 172.21.15.105:0/2585668654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55722d692330 0x55722d692710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd84f7fe640 1 --2- 172.21.15.105:0/2585668654 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55722d6772a0 0x55722d677680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd8467fc640 1 -- 172.21.15.105:0/2585668654 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55722d6772a0 msgr2=0x55722d677680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:34.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd8467fc640 1 --2- 172.21.15.105:0/2585668654 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55722d6772a0 0x55722d677680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd8467fc640 1 -- 172.21.15.105:0/2585668654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55722d692330 msgr2=0x55722d692710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:34.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd8467fc640 1 --2- 172.21.15.105:0/2585668654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55722d692330 0x55722d692710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd8467fc640 1 -- 172.21.15.105:0/2585668654 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55722d18b890 con 0x55722d692c50 2023-11-28T18:26:34.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd8467fc640 1 --2- 172.21.15.105:0/2585668654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692c50 0x55722d676d60 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd8380820f0 tx=0x7fd838007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86d90f740de0cf2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:34.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd844ff9640 1 -- 172.21.15.105:0/2585668654 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83805cdd0 con 0x55722d692c50 2023-11-28T18:26:34.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd844ff9640 1 -- 172.21.15.105:0/2585668654 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd838052cc0 con 0x55722d692c50 2023-11-28T18:26:34.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd844ff9640 1 -- 172.21.15.105:0/2585668654 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd838051290 con 0x55722d692c50 2023-11-28T18:26:34.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd858a83000 1 -- 172.21.15.105:0/2585668654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692c50 msgr2=0x55722d676d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd858a83000 1 --2- 172.21.15.105:0/2585668654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692c50 0x55722d676d60 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd8380820f0 tx=0x7fd838007680 comp rx=0 tx=0).stop 2023-11-28T18:26:34.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd858a83000 1 -- 172.21.15.105:0/2585668654 shutdown_connections 2023-11-28T18:26:34.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd858a83000 1 --2- 172.21.15.105:0/2585668654 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55722d6772a0 0x55722d677680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd858a83000 1 --2- 172.21.15.105:0/2585668654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692c50 0x55722d676d60 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd858a83000 1 --2- 172.21.15.105:0/2585668654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55722d692330 0x55722d692710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd858a83000 1 -- 172.21.15.105:0/2585668654 >> 172.21.15.105:0/2585668654 conn(0x55722d66e8c0 msgr2=0x55722d670cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:34.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd858a83000 1 -- 172.21.15.105:0/2585668654 shutdown_connections 2023-11-28T18:26:34.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd858a83000 1 -- 172.21.15.105:0/2585668654 wait complete. 2023-11-28T18:26:34.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.800+0000 7fd858a83000 1 -- start start 2023-11-28T18:26:34.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd858a83000 1 Processor -- start 2023-11-28T18:26:34.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd858a83000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55722d6772a0 0x55722d647ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd858a83000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692330 0x55722d648220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd858a83000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55722d692c50 0x55722d648760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd858a83000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55722d4937c0 con 0x55722d692330 2023-11-28T18:26:34.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd858a83000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55722d44bbf0 con 0x55722d692c50 2023-11-28T18:26:34.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd858a83000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55722d451a90 con 0x55722d6772a0 2023-11-28T18:26:34.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd858a83000 10 client.0 _ll_register_callbacks cb 0x55722d62ee20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:34.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd8467fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692330 0x55722d648220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd84effd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55722d6772a0 0x55722d647ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd8467fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692330 0x55722d648220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44496/0 (socket says 172.21.15.105:44496) 2023-11-28T18:26:34.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd8467fc640 1 -- 172.21.15.105:0/179549704 learned_addr learned my addr 172.21.15.105:0/179549704 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:34.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd84f7fe640 1 --2- 172.21.15.105:0/179549704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55722d692c50 0x55722d648760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd8467fc640 1 -- 172.21.15.105:0/179549704 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55722d6772a0 msgr2=0x55722d647ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:34.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd8467fc640 1 --2- 172.21.15.105:0/179549704 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55722d6772a0 0x55722d647ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd8467fc640 1 -- 172.21.15.105:0/179549704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55722d692c50 msgr2=0x55722d648760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:34.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd8467fc640 1 --2- 172.21.15.105:0/179549704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55722d692c50 0x55722d648760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd8467fc640 1 -- 172.21.15.105:0/179549704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8501537c0 con 0x55722d692330 2023-11-28T18:26:34.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd8467fc640 1 --2- 172.21.15.105:0/179549704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692330 0x55722d648220 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd83800cec0 tx=0x7fd838001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d976be951fdd0745 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:34.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd82dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:34.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd82dffb640 1 -- 172.21.15.105:0/179549704 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83805cdd0 con 0x55722d692330 2023-11-28T18:26:34.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd82dffb640 1 -- 172.21.15.105:0/179549704 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7fd838051290 con 0x55722d692330 2023-11-28T18:26:34.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd858a83000 1 -- 172.21.15.105:0/179549704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55722d455470 con 0x55722d692330 2023-11-28T18:26:34.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd82dffb640 1 -- 172.21.15.105:0/179549704 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd838051470 con 0x55722d692330 2023-11-28T18:26:34.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd858a83000 10 client.6399 fetch_fsmap learned FSMap version 47 2023-11-28T18:26:34.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd858a83000 1 -- 172.21.15.105:0/179549704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55722d454190 con 0x55722d692330 2023-11-28T18:26:34.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd82dffb640 1 -- 172.21.15.105:0/179549704 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55722d455470 con 0x55722d692330 2023-11-28T18:26:34.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd82dffb640 1 -- 172.21.15.105:0/179549704 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 47) v1 ==== 3326+0+0 (secure 0 0 0) 0x7fd838072d20 con 0x55722d692330 2023-11-28T18:26:34.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd858a83000 10 client.6399 fetch_fsmap finished waiting for FSMap version 47 2023-11-28T18:26:34.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd858a83000 10 client.6399 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:26:34.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd858a83000 20 client.6399 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:34.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd858a83000 1 --2- 172.21.15.105:0/179549704 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55722d649170 0x55722d6a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd858a83000 4 client.6399 mds_command: new command op to 5408 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:26:34.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd858a83000 1 -- 172.21.15.105:0/179549704 --> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55722d458b50 con 0x55722d649170 2023-11-28T18:26:34.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd84effd640 1 --2- 172.21.15.105:0/179549704 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55722d649170 0x55722d6a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd84effd640 1 --2- 172.21.15.105:0/179549704 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55722d649170 0x55722d6a35e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f7a61a4f157dd7ff server_cookie=b9ece219762f33c6 in_seq=0 out_seq=0 2023-11-28T18:26:34.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.804+0000 7fd82dffb640 10 client.6399 ms_handle_connect on v2:172.21.15.105:6835/3765684108 2023-11-28T18:26:34.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.808+0000 7fd82dffb640 1 -- 172.21.15.105:0/179549704 <== mds.2 v2:172.21.15.105:6835/3765684108 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55722d458b50 con 0x55722d649170 2023-11-28T18:26:34.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.808+0000 7fd82dffb640 10 client.6399 handle_command_reply: tid=0 2023-11-28T18:26:34.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.808+0000 7fd858a83000 1 client.6399 shutdown 2023-11-28T18:26:34.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.808+0000 7fd858a83000 1 -- 172.21.15.105:0/179549704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692330 msgr2=0x55722d648220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.808+0000 7fd858a83000 1 --2- 172.21.15.105:0/179549704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692330 0x55722d648220 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd83800cec0 tx=0x7fd838001130 comp rx=0 tx=0).stop 2023-11-28T18:26:34.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.808+0000 7fd858a83000 1 -- 172.21.15.105:0/179549704 shutdown_connections 2023-11-28T18:26:34.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.808+0000 7fd858a83000 1 --2- 172.21.15.105:0/179549704 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55722d649170 0x55722d6a35e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.808+0000 7fd858a83000 1 --2- 172.21.15.105:0/179549704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55722d692c50 0x55722d648760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.808+0000 7fd858a83000 1 --2- 172.21.15.105:0/179549704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692330 0x55722d648220 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.808+0000 7fd82dffb640 0 client.6399 ms_handle_reset on v2:172.21.15.105:6835/3765684108 2023-11-28T18:26:34.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.808+0000 7fd858a83000 1 --2- 172.21.15.105:0/179549704 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55722d6772a0 0x55722d647ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.808+0000 7fd858a83000 1 -- 172.21.15.105:0/179549704 >> 172.21.15.105:0/179549704 conn(0x55722d66e8c0 msgr2=0x55722d6406b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:34.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.808+0000 7fd858a83000 1 -- 172.21.15.105:0/179549704 shutdown_connections 2023-11-28T18:26:34.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.808+0000 7fd858a83000 1 -- 172.21.15.105:0/179549704 wait complete. 2023-11-28T18:26:34.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.808+0000 7fd858a83000 20 client.6399 trim_cache size 0 max 16384 2023-11-28T18:26:34.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd858a83000 1 Processor -- start 2023-11-28T18:26:34.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd858a83000 1 -- start start 2023-11-28T18:26:34.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd858a83000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55722d6772a0 0x55722d6ee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd858a83000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692330 0x55722d6eebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd858a83000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55722d692c50 0x55722d6ef100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd858a83000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55722d14c940 con 0x55722d692330 2023-11-28T18:26:34.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd858a83000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55722d4528f0 con 0x55722d6772a0 2023-11-28T18:26:34.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd858a83000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55722d4977c0 con 0x55722d692c50 2023-11-28T18:26:34.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd8467fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692330 0x55722d6eebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd8467fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692330 0x55722d6eebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44504/0 (socket says 172.21.15.105:44504) 2023-11-28T18:26:34.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd8467fc640 1 -- 172.21.15.105:0/2674659695 learned_addr learned my addr 172.21.15.105:0/2674659695 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd84effd640 1 --2- 172.21.15.105:0/2674659695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55722d6772a0 0x55722d6ee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd84f7fe640 1 --2- 172.21.15.105:0/2674659695 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55722d692c50 0x55722d6ef100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd8467fc640 1 -- 172.21.15.105:0/2674659695 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55722d692c50 msgr2=0x55722d6ef100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd8467fc640 1 --2- 172.21.15.105:0/2674659695 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55722d692c50 0x55722d6ef100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd8467fc640 1 -- 172.21.15.105:0/2674659695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55722d6772a0 msgr2=0x55722d6ee680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd8467fc640 1 --2- 172.21.15.105:0/2674659695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55722d6772a0 0x55722d6ee680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd8467fc640 1 -- 172.21.15.105:0/2674659695 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55722d43a490 con 0x55722d692330 2023-11-28T18:26:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd84f7fe640 1 --2- 172.21.15.105:0/2674659695 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55722d692c50 0x55722d6ef100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd8467fc640 1 --2- 172.21.15.105:0/2674659695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692330 0x55722d6eebc0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd838000fc0 tx=0x7fd83806e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6ecb1adff85441 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:34.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd8457fa640 1 -- 172.21.15.105:0/2674659695 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83800e040 con 0x55722d692330 2023-11-28T18:26:34.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd8457fa640 1 -- 172.21.15.105:0/2674659695 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83805cdd0 con 0x55722d692330 2023-11-28T18:26:34.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd8457fa640 1 -- 172.21.15.105:0/2674659695 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd838058c20 con 0x55722d692330 2023-11-28T18:26:34.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd858a83000 1 -- 172.21.15.105:0/2674659695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692330 msgr2=0x55722d6eebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd858a83000 1 --2- 172.21.15.105:0/2674659695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692330 0x55722d6eebc0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd838000fc0 tx=0x7fd83806e070 comp rx=0 tx=0).stop 2023-11-28T18:26:34.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd858a83000 1 -- 172.21.15.105:0/2674659695 shutdown_connections 2023-11-28T18:26:34.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd858a83000 1 --2- 172.21.15.105:0/2674659695 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55722d692c50 0x55722d6ef100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd858a83000 1 --2- 172.21.15.105:0/2674659695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692330 0x55722d6eebc0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd858a83000 1 --2- 172.21.15.105:0/2674659695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55722d6772a0 0x55722d6ee680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd858a83000 1 -- 172.21.15.105:0/2674659695 >> 172.21.15.105:0/2674659695 conn(0x55722d66e8c0 msgr2=0x55722d640630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:34.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd858a83000 1 -- 172.21.15.105:0/2674659695 shutdown_connections 2023-11-28T18:26:34.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd858a83000 1 -- 172.21.15.105:0/2674659695 wait complete. 2023-11-28T18:26:34.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd858a83000 1 -- start start 2023-11-28T18:26:34.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.824+0000 7fd858a83000 1 Processor -- start 2023-11-28T18:26:34.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55722d6772a0 0x55722d676680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692330 0x55722d6ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55722d692c50 0x55722d6ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55722d468b70 con 0x55722d692330 2023-11-28T18:26:34.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55722d492650 con 0x55722d692c50 2023-11-28T18:26:34.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55722d6a7830 con 0x55722d6772a0 2023-11-28T18:26:34.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 10 client.0 _ll_register_callbacks cb 0x55722d50a060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:34.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd8467fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692330 0x55722d6ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd8467fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692330 0x55722d6ae0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44518/0 (socket says 172.21.15.105:44518) 2023-11-28T18:26:34.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd8467fc640 1 -- 172.21.15.105:0/624672419 learned_addr learned my addr 172.21.15.105:0/624672419 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:34.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd84effd640 1 --2- 172.21.15.105:0/624672419 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55722d6772a0 0x55722d676680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd84f7fe640 1 --2- 172.21.15.105:0/624672419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55722d692c50 0x55722d6ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd8467fc640 1 -- 172.21.15.105:0/624672419 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55722d6772a0 msgr2=0x55722d676680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:34.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd8467fc640 1 --2- 172.21.15.105:0/624672419 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55722d6772a0 0x55722d676680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd8467fc640 1 -- 172.21.15.105:0/624672419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55722d692c50 msgr2=0x55722d6ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:34.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd8467fc640 1 --2- 172.21.15.105:0/624672419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55722d692c50 0x55722d6ae5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd8467fc640 1 -- 172.21.15.105:0/624672419 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55722d454190 con 0x55722d692330 2023-11-28T18:26:34.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd84effd640 1 --2- 172.21.15.105:0/624672419 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55722d6772a0 0x55722d676680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:34.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd8467fc640 1 --2- 172.21.15.105:0/624672419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692330 0x55722d6ae0b0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd83805c8a0 tx=0x7fd8380055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974b5c314bf8618b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:34.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd82dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:34.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd82dffb640 1 -- 172.21.15.105:0/624672419 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83800e040 con 0x55722d692330 2023-11-28T18:26:34.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd82dffb640 1 -- 172.21.15.105:0/624672419 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7fd838058c20 con 0x55722d692330 2023-11-28T18:26:34.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 1 -- 172.21.15.105:0/624672419 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55722d492110 con 0x55722d692330 2023-11-28T18:26:34.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd82dffb640 1 -- 172.21.15.105:0/624672419 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd83806c1d0 con 0x55722d692330 2023-11-28T18:26:34.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 10 client.6405 fetch_fsmap learned FSMap version 47 2023-11-28T18:26:34.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 1 -- 172.21.15.105:0/624672419 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55722d18bcc0 con 0x55722d692330 2023-11-28T18:26:34.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd82dffb640 1 -- 172.21.15.105:0/624672419 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55722d492110 con 0x55722d692330 2023-11-28T18:26:34.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd82dffb640 1 -- 172.21.15.105:0/624672419 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 47) v1 ==== 3326+0+0 (secure 0 0 0) 0x7fd838019030 con 0x55722d692330 2023-11-28T18:26:34.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 10 client.6405 fetch_fsmap finished waiting for FSMap version 47 2023-11-28T18:26:34.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 10 client.6405 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:26:34.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 20 client.6405 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:34.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 1 --2- 172.21.15.105:0/624672419 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55722d6f1d20 0x55722d6f2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 4 client.6405 mds_command: new command op to 5408 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:26:34.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 1 -- 172.21.15.105:0/624672419 --> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55722d4c0650 con 0x55722d6f1d20 2023-11-28T18:26:34.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd84effd640 1 --2- 172.21.15.105:0/624672419 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55722d6f1d20 0x55722d6f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd84effd640 1 --2- 172.21.15.105:0/624672419 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55722d6f1d20 0x55722d6f2100 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c304893fb18e8362 server_cookie=c14a724f7806ce97 in_seq=0 out_seq=0 2023-11-28T18:26:34.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd82dffb640 10 client.6405 ms_handle_connect on v2:172.21.15.105:6835/3765684108 2023-11-28T18:26:34.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd82dffb640 1 -- 172.21.15.105:0/624672419 <== mds.2 v2:172.21.15.105:6835/3765684108 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55722d4c0650 con 0x55722d6f1d20 2023-11-28T18:26:34.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd82dffb640 10 client.6405 handle_command_reply: tid=0 2023-11-28T18:26:34.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 1 client.6405 shutdown 2023-11-28T18:26:34.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 1 -- 172.21.15.105:0/624672419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692330 msgr2=0x55722d6ae0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 1 --2- 172.21.15.105:0/624672419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692330 0x55722d6ae0b0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd83805c8a0 tx=0x7fd8380055f0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 1 -- 172.21.15.105:0/624672419 shutdown_connections 2023-11-28T18:26:34.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 1 --2- 172.21.15.105:0/624672419 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55722d6f1d20 0x55722d6f2100 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 1 --2- 172.21.15.105:0/624672419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55722d692c50 0x55722d6ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 1 --2- 172.21.15.105:0/624672419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55722d692330 0x55722d6ae0b0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd82dffb640 0 client.6405 ms_handle_reset on v2:172.21.15.105:6835/3765684108 2023-11-28T18:26:34.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 1 --2- 172.21.15.105:0/624672419 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55722d6772a0 0x55722d676680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 1 -- 172.21.15.105:0/624672419 >> 172.21.15.105:0/624672419 conn(0x55722d66e8c0 msgr2=0x55722d66c390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:34.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 1 -- 172.21.15.105:0/624672419 shutdown_connections 2023-11-28T18:26:34.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 1 -- 172.21.15.105:0/624672419 wait complete. 2023-11-28T18:26:34.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.828+0000 7fd858a83000 20 client.6405 trim_cache size 0 max 16384 2023-11-28T18:26:34.841 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:26:34.841 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:26:34.841 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 2, 2023-11-28T18:26:34.841 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5408, 2023-11-28T18:26:34.841 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:26:34.842 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:26:34.842 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:26:34.842 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 22.612152887000001, 2023-11-28T18:26:34.842 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 47, 2023-11-28T18:26:34.842 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 54, 2023-11-28T18:26:34.842 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 54, 2023-11-28T18:26:34.842 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 38.543108228000001 2023-11-28T18:26:34.842 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:26:34.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.832+0000 7fd85571d640 1 -- 172.21.15.105:0/3178095791 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd830066900 msgr2=0x7fd830068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.832+0000 7fd85571d640 1 --2- 172.21.15.105:0/3178095791 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd830066900 0x7fd830068dc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd8400af000 tx=0x7fd8400b6040 comp rx=0 tx=0).stop 2023-11-28T18:26:34.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.832+0000 7fd85571d640 1 -- 172.21.15.105:0/3178095791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd85015b030 msgr2=0x7fd8501553f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.832+0000 7fd85571d640 1 --2- 172.21.15.105:0/3178095791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd85015b030 0x7fd8501553f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd83800c500 tx=0x7fd83800c9c0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.832+0000 7fd85571d640 1 -- 172.21.15.105:0/3178095791 shutdown_connections 2023-11-28T18:26:34.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.832+0000 7fd85571d640 1 --2- 172.21.15.105:0/3178095791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd85009a3f0 0x7fd85009a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.832+0000 7fd85571d640 1 --2- 172.21.15.105:0/3178095791 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd830066900 0x7fd830068dc0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.832+0000 7fd85571d640 1 --2- 172.21.15.105:0/3178095791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd85015b030 0x7fd8501553f0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.832+0000 7fd85571d640 1 --2- 172.21.15.105:0/3178095791 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd8500b4290 0x7fd85014b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.832+0000 7fd85571d640 1 -- 172.21.15.105:0/3178095791 >> 172.21.15.105:0/3178095791 conn(0x7fd8500ceb90 msgr2=0x7fd8500cadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:34.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.832+0000 7fd85571d640 1 -- 172.21.15.105:0/3178095791 shutdown_connections 2023-11-28T18:26:34.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.832+0000 7fd85571d640 1 -- 172.21.15.105:0/3178095791 wait complete. 2023-11-28T18:26:34.852 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-28T18:26:34.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.988+0000 7f422efdf640 1 Processor -- start 2023-11-28T18:26:34.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.988+0000 7f422efdf640 1 -- start start 2023-11-28T18:26:34.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.988+0000 7f422efdf640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4228159f40 0x7f42280caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.988+0000 7f422efdf640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f42280ad850 0x7f42280701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:34.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.988+0000 7f422efdf640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422806bf50 con 0x7f42280cc760 2023-11-28T18:26:34.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.988+0000 7f422efdf640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42280ccbc0 con 0x7f42280ad850 2023-11-28T18:26:34.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.988+0000 7f422efdf640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422815a320 con 0x7f4228159f40 2023-11-28T18:26:34.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.988+0000 7f422d57c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f42280ad850 0x7f42280701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.988+0000 7f422d57c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f42280ad850 0x7f42280701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44526/0 (socket says 172.21.15.105:44526) 2023-11-28T18:26:34.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.988+0000 7f422d57c640 1 -- 172.21.15.105:0/1195584775 learned_addr learned my addr 172.21.15.105:0/1195584775 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:34.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.988+0000 7f422cd7b640 1 --2- 172.21.15.105:0/1195584775 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4228159f40 0x7f42280caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:34.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.988+0000 7f422d57c640 1 -- 172.21.15.105:0/1195584775 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4228159f40 msgr2=0x7f42280caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:34.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.988+0000 7f422d57c640 1 --2- 172.21.15.105:0/1195584775 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4228159f40 0x7f42280caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.988+0000 7f422d57c640 1 -- 172.21.15.105:0/1195584775 >> v1:172.21.15.148:6789/0 conn(0x7f42280cc760 legacy=0x7f42280ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.988+0000 7f422d57c640 1 -- 172.21.15.105:0/1195584775 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4228071380 con 0x7f42280ad850 2023-11-28T18:26:34.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.992+0000 7f422d57c640 1 --2- 172.21.15.105:0/1195584775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42280ad850 0x7f42280701d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4220009e50 tx=0x7f422001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f927e870f17ed55f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:34.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.992+0000 7f42277fe640 1 -- 172.21.15.105:0/1195584775 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42200a5810 con 0x7f42280ad850 2023-11-28T18:26:34.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.992+0000 7f42277fe640 1 -- 172.21.15.105:0/1195584775 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42200a5970 con 0x7f42280ad850 2023-11-28T18:26:34.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.992+0000 7f42277fe640 1 -- 172.21.15.105:0/1195584775 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42200a5d00 con 0x7f42280ad850 2023-11-28T18:26:34.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.992+0000 7f422efdf640 1 -- 172.21.15.105:0/1195584775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42280ad850 msgr2=0x7f42280701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:34.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.992+0000 7f422efdf640 1 --2- 172.21.15.105:0/1195584775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42280ad850 0x7f42280701d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4220009e50 tx=0x7f422001c950 comp rx=0 tx=0).stop 2023-11-28T18:26:34.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.992+0000 7f422efdf640 1 -- 172.21.15.105:0/1195584775 shutdown_connections 2023-11-28T18:26:34.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.992+0000 7f422efdf640 1 --2- 172.21.15.105:0/1195584775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42280ad850 0x7f42280701d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.992+0000 7f422efdf640 1 --2- 172.21.15.105:0/1195584775 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4228159f40 0x7f42280caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:34.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.992+0000 7f422efdf640 1 -- 172.21.15.105:0/1195584775 >> 172.21.15.105:0/1195584775 conn(0x7f42280ceb90 msgr2=0x7f42280cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:34.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.992+0000 7f422efdf640 1 -- 172.21.15.105:0/1195584775 shutdown_connections 2023-11-28T18:26:34.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.992+0000 7f422efdf640 1 -- 172.21.15.105:0/1195584775 wait complete. 2023-11-28T18:26:35.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f422efdf640 1 Processor -- start 2023-11-28T18:26:35.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f422efdf640 1 -- start start 2023-11-28T18:26:35.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f422efdf640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42280ad850 0x7f422814d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:35.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f422efdf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42280cc760 0x7f4228152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:35.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f422efdf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4228159f40 0x7f422814a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:35.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f422efdf640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422814dd40 con 0x7f4228159f40 2023-11-28T18:26:35.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f422efdf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422815c430 con 0x7f42280cc760 2023-11-28T18:26:35.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f422efdf640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422814da70 con 0x7f42280ad850 2023-11-28T18:26:35.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f422d57c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4228159f40 0x7f422814a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:35.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f422d57c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4228159f40 0x7f422814a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44534/0 (socket says 172.21.15.105:44534) 2023-11-28T18:26:35.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f422cd7b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42280ad850 0x7f422814d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:35.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f422d57c640 1 -- 172.21.15.105:0/732565835 learned_addr learned my addr 172.21.15.105:0/732565835 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:35.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f4227fff640 1 --2- 172.21.15.105:0/732565835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42280cc760 0x7f4228152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:35.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f422d57c640 1 -- 172.21.15.105:0/732565835 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42280ad850 msgr2=0x7f422814d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:35.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f422d57c640 1 --2- 172.21.15.105:0/732565835 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42280ad850 0x7f422814d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f422d57c640 1 -- 172.21.15.105:0/732565835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42280cc760 msgr2=0x7f4228152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:35.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f422d57c640 1 --2- 172.21.15.105:0/732565835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42280cc760 0x7f4228152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f422d57c640 1 -- 172.21.15.105:0/732565835 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4220009b30 con 0x7f4228159f40 2023-11-28T18:26:35.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f422d57c640 1 --2- 172.21.15.105:0/732565835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4228159f40 0x7f422814a4f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f422001a580 tx=0x7f42200196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:35.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f42257fa640 1 -- 172.21.15.105:0/732565835 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42200b8670 con 0x7f4228159f40 2023-11-28T18:26:35.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f42257fa640 1 -- 172.21.15.105:0/732565835 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42200b87d0 con 0x7f4228159f40 2023-11-28T18:26:35.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f42257fa640 1 -- 172.21.15.105:0/732565835 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42200b8b80 con 0x7f4228159f40 2023-11-28T18:26:35.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f422efdf640 1 -- 172.21.15.105:0/732565835 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422815c6d0 con 0x7f4228159f40 2023-11-28T18:26:35.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:34.996+0000 7f422efdf640 1 -- 172.21.15.105:0/732565835 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422815cc90 con 0x7f4228159f40 2023-11-28T18:26:35.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.000+0000 7f42257fa640 1 -- 172.21.15.105:0/732565835 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f42200b8ce0 con 0x7f4228159f40 2023-11-28T18:26:35.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.000+0000 7f42257fa640 1 --2- 172.21.15.105:0/732565835 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f420c0668d0 0x7f420c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:35.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.000+0000 7f42257fa640 1 -- 172.21.15.105:0/732565835 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f422011fae0 con 0x7f4228159f40 2023-11-28T18:26:35.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.000+0000 7f422cd7b640 1 --2- 172.21.15.105:0/732565835 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f420c0668d0 0x7f420c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:35.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.000+0000 7f422cd7b640 1 --2- 172.21.15.105:0/732565835 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f420c0668d0 0x7f420c068d90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f421c0079a0 tx=0x7f421c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:35.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.000+0000 7f4232341000 1 Processor -- start 2023-11-28T18:26:35.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.000+0000 7f4232341000 1 -- start start 2023-11-28T18:26:35.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.000+0000 7f4232341000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acebe75d30 0x55acebe76110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:35.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.000+0000 7f4232341000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acebe76650 0x55acebe28fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:35.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.000+0000 7f4232341000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe29500 0x55acebe298e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:35.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.000+0000 7f4232341000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acebcf2110 con 0x55acebe29500 2023-11-28T18:26:35.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.000+0000 7f4232341000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acebc829a0 con 0x55acebe75d30 2023-11-28T18:26:35.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.000+0000 7f4232341000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acebc7fed0 con 0x55acebe76650 2023-11-28T18:26:35.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.000+0000 7f422d57c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe29500 0x55acebe298e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:35.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f422d57c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe29500 0x55acebe298e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44538/0 (socket says 172.21.15.105:44538) 2023-11-28T18:26:35.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f422d57c640 1 -- 172.21.15.105:0/3352181479 learned_addr learned my addr 172.21.15.105:0/3352181479 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:35.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f4227fff640 1 --2- 172.21.15.105:0/3352181479 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acebe76650 0x55acebe28fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:35.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f422cd7b640 1 --2- 172.21.15.105:0/3352181479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acebe75d30 0x55acebe76110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:35.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f422d57c640 1 -- 172.21.15.105:0/3352181479 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acebe76650 msgr2=0x55acebe28fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:35.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f422d57c640 1 --2- 172.21.15.105:0/3352181479 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acebe76650 0x55acebe28fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f422d57c640 1 -- 172.21.15.105:0/3352181479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acebe75d30 msgr2=0x55acebe76110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:35.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f422d57c640 1 --2- 172.21.15.105:0/3352181479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acebe75d30 0x55acebe76110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f422d57c640 1 -- 172.21.15.105:0/3352181479 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aceb973890 con 0x55acebe29500 2023-11-28T18:26:35.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f422d57c640 1 --2- 172.21.15.105:0/3352181479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe29500 0x55acebe298e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f42200ecfa0 tx=0x7f4220009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53412acd69bee3e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:35.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f42127fc640 1 -- 172.21.15.105:0/3352181479 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42200eee40 con 0x55acebe29500 2023-11-28T18:26:35.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f42127fc640 1 -- 172.21.15.105:0/3352181479 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42200ed1b0 con 0x55acebe29500 2023-11-28T18:26:35.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f42127fc640 1 -- 172.21.15.105:0/3352181479 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42200f0f20 con 0x55acebe29500 2023-11-28T18:26:35.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f4232341000 1 -- 172.21.15.105:0/3352181479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe29500 msgr2=0x55acebe298e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:35.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f4232341000 1 --2- 172.21.15.105:0/3352181479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe29500 0x55acebe298e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f42200ecfa0 tx=0x7f4220009810 comp rx=0 tx=0).stop 2023-11-28T18:26:35.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f4232341000 1 -- 172.21.15.105:0/3352181479 shutdown_connections 2023-11-28T18:26:35.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f4232341000 1 --2- 172.21.15.105:0/3352181479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe29500 0x55acebe298e0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f4232341000 1 --2- 172.21.15.105:0/3352181479 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acebe76650 0x55acebe28fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f4232341000 1 --2- 172.21.15.105:0/3352181479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acebe75d30 0x55acebe76110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f4232341000 1 -- 172.21.15.105:0/3352181479 >> 172.21.15.105:0/3352181479 conn(0x55acebe568c0 msgr2=0x55acebe58cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:35.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f4232341000 1 -- 172.21.15.105:0/3352181479 shutdown_connections 2023-11-28T18:26:35.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f4232341000 1 -- 172.21.15.105:0/3352181479 wait complete. 2023-11-28T18:26:35.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f4232341000 1 -- start start 2023-11-28T18:26:35.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f4232341000 1 Processor -- start 2023-11-28T18:26:35.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f4232341000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe29500 0x55acebe2f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:35.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f4232341000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acebe75d30 0x55acebe2fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:35.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f4232341000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acebe76650 0x55acebe2ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:35.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f4232341000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acebc7b7c0 con 0x55acebe29500 2023-11-28T18:26:35.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f4232341000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acebc33bf0 con 0x55acebe75d30 2023-11-28T18:26:35.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f4232341000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acebc39a90 con 0x55acebe76650 2023-11-28T18:26:35.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f4232341000 10 client.0 _ll_register_callbacks cb 0x55acebe16e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:35.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f422cd7b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe29500 0x55acebe2f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:35.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f422cd7b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe29500 0x55acebe2f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44544/0 (socket says 172.21.15.105:44544) 2023-11-28T18:26:35.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.004+0000 7f422cd7b640 1 -- 172.21.15.105:0/2591541951 learned_addr learned my addr 172.21.15.105:0/2591541951 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:35.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f422d57c640 1 --2- 172.21.15.105:0/2591541951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acebe76650 0x55acebe2ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:35.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f4227fff640 1 --2- 172.21.15.105:0/2591541951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acebe75d30 0x55acebe2fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:35.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f422cd7b640 1 -- 172.21.15.105:0/2591541951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acebe76650 msgr2=0x55acebe2ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:35.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f422cd7b640 1 --2- 172.21.15.105:0/2591541951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acebe76650 0x55acebe2ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f422cd7b640 1 -- 172.21.15.105:0/2591541951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acebe75d30 msgr2=0x55acebe2fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:35.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f422cd7b640 1 --2- 172.21.15.105:0/2591541951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acebe75d30 0x55acebe2fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f422cd7b640 1 -- 172.21.15.105:0/2591541951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422815cc90 con 0x55acebe29500 2023-11-28T18:26:35.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f422cd7b640 1 --2- 172.21.15.105:0/2591541951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe29500 0x55acebe2f4e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f421c016800 tx=0x7f421c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348e9431a70ce404 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:35.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f42077fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:35.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f42077fe640 1 -- 172.21.15.105:0/2591541951 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c0149b0 con 0x55acebe29500 2023-11-28T18:26:35.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f4232341000 1 -- 172.21.15.105:0/2591541951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55acebc3d470 con 0x55acebe29500 2023-11-28T18:26:35.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f42077fe640 1 -- 172.21.15.105:0/2591541951 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f421c014b10 con 0x55acebe29500 2023-11-28T18:26:35.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f42077fe640 1 -- 172.21.15.105:0/2591541951 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f421c014d10 con 0x55acebe29500 2023-11-28T18:26:35.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f4232341000 10 client.6417 fetch_fsmap learned FSMap version 47 2023-11-28T18:26:35.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f4232341000 1 -- 172.21.15.105:0/2591541951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55acebc3c190 con 0x55acebe29500 2023-11-28T18:26:35.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f42077fe640 1 -- 172.21.15.105:0/2591541951 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55acebc3d470 con 0x55acebe29500 2023-11-28T18:26:35.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f42077fe640 1 -- 172.21.15.105:0/2591541951 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 47) v1 ==== 3326+0+0 (secure 0 0 0) 0x7f421c02b440 con 0x55acebe29500 2023-11-28T18:26:35.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f4232341000 10 client.6417 fetch_fsmap finished waiting for FSMap version 47 2023-11-28T18:26:35.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f4232341000 10 client.6417 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T18:26:35.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f4232341000 20 client.6417 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:35.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f4232341000 1 --2- 172.21.15.105:0/2591541951 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55acebe30970 0x55acebe87ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:35.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f4232341000 4 client.6417 mds_command: new command op to 5842 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:26:35.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f4232341000 1 -- 172.21.15.105:0/2591541951 --> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acebc40b50 con 0x55acebe30970 2023-11-28T18:26:35.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f4227fff640 1 --2- 172.21.15.105:0/2591541951 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55acebe30970 0x55acebe87ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:35.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f4227fff640 1 --2- 172.21.15.105:0/2591541951 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55acebe30970 0x55acebe87ff0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5014e871d900ecf server_cookie=c1751032d323dd06 in_seq=0 out_seq=0 2023-11-28T18:26:35.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f42077fe640 10 client.6417 ms_handle_connect on v2:172.21.15.148:6832/2696773413 2023-11-28T18:26:35.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f42077fe640 1 -- 172.21.15.105:0/2591541951 <== mds.0 v2:172.21.15.148:6832/2696773413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55acebc40b50 con 0x55acebe30970 2023-11-28T18:26:35.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f42077fe640 10 client.6417 handle_command_reply: tid=0 2023-11-28T18:26:35.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.008+0000 7f4232341000 1 client.6417 shutdown 2023-11-28T18:26:35.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.012+0000 7f4232341000 1 -- 172.21.15.105:0/2591541951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe29500 msgr2=0x55acebe2f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:35.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.012+0000 7f4232341000 1 --2- 172.21.15.105:0/2591541951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe29500 0x55acebe2f4e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f421c016800 tx=0x7f421c0046d0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.012+0000 7f4232341000 1 -- 172.21.15.105:0/2591541951 shutdown_connections 2023-11-28T18:26:35.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.012+0000 7f4232341000 1 --2- 172.21.15.105:0/2591541951 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55acebe30970 0x55acebe87ff0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.012+0000 7f4232341000 1 --2- 172.21.15.105:0/2591541951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acebe76650 0x55acebe2ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.012+0000 7f4232341000 1 --2- 172.21.15.105:0/2591541951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acebe75d30 0x55acebe2fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.012+0000 7f4232341000 1 --2- 172.21.15.105:0/2591541951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe29500 0x55acebe2f4e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.012+0000 7f42077fe640 0 client.6417 ms_handle_reset on v2:172.21.15.148:6832/2696773413 2023-11-28T18:26:35.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.012+0000 7f4232341000 1 -- 172.21.15.105:0/2591541951 >> 172.21.15.105:0/2591541951 conn(0x55acebe568c0 msgr2=0x55acebe286b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:35.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.012+0000 7f4232341000 1 -- 172.21.15.105:0/2591541951 shutdown_connections 2023-11-28T18:26:35.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.012+0000 7f4232341000 1 -- 172.21.15.105:0/2591541951 wait complete. 2023-11-28T18:26:35.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.012+0000 7f4232341000 20 client.6417 trim_cache size 0 max 16384 2023-11-28T18:26:35.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.024+0000 7f4232341000 1 Processor -- start 2023-11-28T18:26:35.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.024+0000 7f4232341000 1 -- start start 2023-11-28T18:26:35.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acebe29500 0x55acebe553a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acebe75d30 0x55acebe558e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe76650 0x55acebe87ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aceb934940 con 0x55acebe76650 2023-11-28T18:26:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aceb9741b0 con 0x55acebe75d30 2023-11-28T18:26:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acebc3a8f0 con 0x55acebe29500 2023-11-28T18:26:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f422d57c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe76650 0x55acebe87ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f422d57c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe76650 0x55acebe87ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44548/0 (socket says 172.21.15.105:44548) 2023-11-28T18:26:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f422d57c640 1 -- 172.21.15.105:0/3292565277 learned_addr learned my addr 172.21.15.105:0/3292565277 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f422cd7b640 1 --2- 172.21.15.105:0/3292565277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acebe29500 0x55acebe553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4227fff640 1 --2- 172.21.15.105:0/3292565277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acebe75d30 0x55acebe558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f422d57c640 1 -- 172.21.15.105:0/3292565277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acebe29500 msgr2=0x55acebe553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f422d57c640 1 --2- 172.21.15.105:0/3292565277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acebe29500 0x55acebe553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f422d57c640 1 -- 172.21.15.105:0/3292565277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acebe75d30 msgr2=0x55acebe558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f422d57c640 1 --2- 172.21.15.105:0/3292565277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acebe75d30 0x55acebe558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f422d57c640 1 -- 172.21.15.105:0/3292565277 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acebe29370 con 0x55acebe76650 2023-11-28T18:26:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f422d57c640 1 --2- 172.21.15.105:0/3292565277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe76650 0x55acebe87ff0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f422001cec0 tx=0x7f42200ecf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e75dbd1fd8f57c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4212ffd640 1 -- 172.21.15.105:0/3292565277 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42200fc070 con 0x55acebe76650 2023-11-28T18:26:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4212ffd640 1 -- 172.21.15.105:0/3292565277 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42200eebe0 con 0x55acebe76650 2023-11-28T18:26:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4212ffd640 1 -- 172.21.15.105:0/3292565277 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42200ed1b0 con 0x55acebe76650 2023-11-28T18:26:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 -- 172.21.15.105:0/3292565277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe76650 msgr2=0x55acebe87ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 --2- 172.21.15.105:0/3292565277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe76650 0x55acebe87ff0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f422001cec0 tx=0x7f42200ecf10 comp rx=0 tx=0).stop 2023-11-28T18:26:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 -- 172.21.15.105:0/3292565277 shutdown_connections 2023-11-28T18:26:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 --2- 172.21.15.105:0/3292565277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe76650 0x55acebe87ff0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 --2- 172.21.15.105:0/3292565277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acebe75d30 0x55acebe558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 --2- 172.21.15.105:0/3292565277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acebe29500 0x55acebe553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 -- 172.21.15.105:0/3292565277 >> 172.21.15.105:0/3292565277 conn(0x55acebe568c0 msgr2=0x55acebe27720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 -- 172.21.15.105:0/3292565277 shutdown_connections 2023-11-28T18:26:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 -- 172.21.15.105:0/3292565277 wait complete. 2023-11-28T18:26:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 -- start start 2023-11-28T18:26:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 Processor -- start 2023-11-28T18:26:35.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acebe29500 0x55acebedee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:35.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acebe75d30 0x55acebedf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:35.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe76650 0x55acebe9a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:35.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acebc78bc0 con 0x55acebe76650 2023-11-28T18:26:35.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acebc7e320 con 0x55acebe29500 2023-11-28T18:26:35.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acebc868e0 con 0x55acebe75d30 2023-11-28T18:26:35.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.028+0000 7f4232341000 10 client.0 _ll_register_callbacks cb 0x55acebcf2060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:35.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f422d57c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe76650 0x55acebe9a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f422d57c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe76650 0x55acebe9a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44558/0 (socket says 172.21.15.105:44558) 2023-11-28T18:26:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f422d57c640 1 -- 172.21.15.105:0/3525489543 learned_addr learned my addr 172.21.15.105:0/3525489543 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f4227fff640 1 --2- 172.21.15.105:0/3525489543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acebe75d30 0x55acebedf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f422cd7b640 1 --2- 172.21.15.105:0/3525489543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acebe29500 0x55acebedee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f422d57c640 1 -- 172.21.15.105:0/3525489543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acebe75d30 msgr2=0x55acebedf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f422d57c640 1 --2- 172.21.15.105:0/3525489543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acebe75d30 0x55acebedf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f422d57c640 1 -- 172.21.15.105:0/3525489543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acebe29500 msgr2=0x55acebedee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f422d57c640 1 --2- 172.21.15.105:0/3525489543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acebe29500 0x55acebedee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f422d57c640 1 -- 172.21.15.105:0/3525489543 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aceb973890 con 0x55acebe76650 2023-11-28T18:26:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f422cd7b640 1 --2- 172.21.15.105:0/3525489543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acebe29500 0x55acebedee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:26:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f422d57c640 1 --2- 172.21.15.105:0/3525489543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe76650 0x55acebe9a280 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f422001b000 tx=0x7f42200a5900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916c41536ebafd59 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:35.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f42077fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:35.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f42077fe640 1 -- 172.21.15.105:0/3525489543 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42200fc040 con 0x55acebe76650 2023-11-28T18:26:35.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f42077fe640 1 -- 172.21.15.105:0/3525489543 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f42200f48f0 con 0x55acebe76650 2023-11-28T18:26:35.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f4232341000 1 -- 172.21.15.105:0/3525489543 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55acebc7a110 con 0x55acebe76650 2023-11-28T18:26:35.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f42077fe640 1 -- 172.21.15.105:0/3525489543 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42200ecde0 con 0x55acebe76650 2023-11-28T18:26:35.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f4232341000 10 client.6423 fetch_fsmap learned FSMap version 47 2023-11-28T18:26:35.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f4232341000 1 -- 172.21.15.105:0/3525489543 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55acebc22490 con 0x55acebe76650 2023-11-28T18:26:35.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f42077fe640 1 -- 172.21.15.105:0/3525489543 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55acebc7a110 con 0x55acebe76650 2023-11-28T18:26:35.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f42077fe640 1 -- 172.21.15.105:0/3525489543 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 47) v1 ==== 3326+0+0 (secure 0 0 0) 0x7f4220107050 con 0x55acebe76650 2023-11-28T18:26:35.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f4232341000 10 client.6423 fetch_fsmap finished waiting for FSMap version 47 2023-11-28T18:26:35.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f4232341000 10 client.6423 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T18:26:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f4232341000 20 client.6423 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f4232341000 1 --2- 172.21.15.105:0/3525489543 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55acebe9a7c0 0x55acebe9aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f4232341000 4 client.6423 mds_command: new command op to 5842 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:26:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f4232341000 1 -- 172.21.15.105:0/3525489543 --> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55acebca8650 con 0x55acebe9a7c0 2023-11-28T18:26:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f422cd7b640 1 --2- 172.21.15.105:0/3525489543 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55acebe9a7c0 0x55acebe9aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f42077fe640 10 client.6423 ms_handle_connect on v2:172.21.15.148:6832/2696773413 2023-11-28T18:26:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f422cd7b640 1 --2- 172.21.15.105:0/3525489543 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55acebe9a7c0 0x55acebe9aba0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7529345b73b5b55f server_cookie=1cf9efcfc5f72180 in_seq=0 out_seq=0 2023-11-28T18:26:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f42077fe640 1 -- 172.21.15.105:0/3525489543 <== mds.0 v2:172.21.15.148:6832/2696773413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55acebca8650 con 0x55acebe9a7c0 2023-11-28T18:26:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f42077fe640 10 client.6423 handle_command_reply: tid=0 2023-11-28T18:26:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f4232341000 1 client.6423 shutdown 2023-11-28T18:26:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f4232341000 1 -- 172.21.15.105:0/3525489543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe76650 msgr2=0x55acebe9a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f4232341000 1 --2- 172.21.15.105:0/3525489543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe76650 0x55acebe9a280 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f422001b000 tx=0x7f42200a5900 comp rx=0 tx=0).stop 2023-11-28T18:26:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f4232341000 1 -- 172.21.15.105:0/3525489543 shutdown_connections 2023-11-28T18:26:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f4232341000 1 --2- 172.21.15.105:0/3525489543 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55acebe9a7c0 0x55acebe9aba0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f4232341000 1 --2- 172.21.15.105:0/3525489543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acebe76650 0x55acebe9a280 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f4232341000 1 --2- 172.21.15.105:0/3525489543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acebe75d30 0x55acebedf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f4232341000 1 --2- 172.21.15.105:0/3525489543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acebe29500 0x55acebedee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f42077fe640 0 client.6423 ms_handle_reset on v2:172.21.15.148:6832/2696773413 2023-11-28T18:26:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f4232341000 1 -- 172.21.15.105:0/3525489543 >> 172.21.15.105:0/3525489543 conn(0x55acebe568c0 msgr2=0x55acebe27fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f4232341000 1 -- 172.21.15.105:0/3525489543 shutdown_connections 2023-11-28T18:26:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f4232341000 1 -- 172.21.15.105:0/3525489543 wait complete. 2023-11-28T18:26:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f4232341000 20 client.6423 trim_cache size 0 max 16384 2023-11-28T18:26:35.045 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:26:35.045 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:26:35.045 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2023-11-28T18:26:35.045 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5842, 2023-11-28T18:26:35.045 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:26:35.045 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:26:35.045 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:26:35.045 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 33.870999069, 2023-11-28T18:26:35.046 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 47, 2023-11-28T18:26:35.046 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 54, 2023-11-28T18:26:35.046 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 54, 2023-11-28T18:26:35.046 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 38.738245419000002 2023-11-28T18:26:35.046 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:26:35.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f422efdf640 1 -- 172.21.15.105:0/732565835 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f420c0668d0 msgr2=0x7f420c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:35.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f422efdf640 1 --2- 172.21.15.105:0/732565835 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f420c0668d0 0x7f420c068d90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f421c0079a0 tx=0x7f421c008040 comp rx=0 tx=0).stop 2023-11-28T18:26:35.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f422efdf640 1 -- 172.21.15.105:0/732565835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4228159f40 msgr2=0x7f422814a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:35.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.032+0000 7f422efdf640 1 --2- 172.21.15.105:0/732565835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4228159f40 0x7f422814a4f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f422001a580 tx=0x7f42200196f0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.036+0000 7f422efdf640 1 -- 172.21.15.105:0/732565835 shutdown_connections 2023-11-28T18:26:35.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.036+0000 7f422efdf640 1 --2- 172.21.15.105:0/732565835 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f420c0668d0 0x7f420c068d90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.036+0000 7f422efdf640 1 --2- 172.21.15.105:0/732565835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4228159f40 0x7f422814a4f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.036+0000 7f422efdf640 1 --2- 172.21.15.105:0/732565835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42280cc760 0x7f4228152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.036+0000 7f422efdf640 1 --2- 172.21.15.105:0/732565835 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42280ad850 0x7f422814d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:35.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.036+0000 7f422efdf640 1 -- 172.21.15.105:0/732565835 >> 172.21.15.105:0/732565835 conn(0x7f42280ceb90 msgr2=0x7f42280ccf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:35.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.036+0000 7f422efdf640 1 -- 172.21.15.105:0/732565835 shutdown_connections 2023-11-28T18:26:35.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:35.036+0000 7f422efdf640 1 -- 172.21.15.105:0/732565835 wait complete. 2023-11-28T18:26:45.056 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:26:45.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.200+0000 7fc654c9c640 1 Processor -- start 2023-11-28T18:26:45.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.200+0000 7fc654c9c640 1 -- start start 2023-11-28T18:26:45.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.204+0000 7fc654c9c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc650071960 0x7fc65015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.204+0000 7fc654c9c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc65015b030 0x7fc6500709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.204+0000 7fc654c9c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6500701d0 con 0x7fc6500b4290 2023-11-28T18:26:45.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.204+0000 7fc654c9c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc65006df80 con 0x7fc650071960 2023-11-28T18:26:45.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.204+0000 7fc654c9c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc650071df0 con 0x7fc65015b030 2023-11-28T18:26:45.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.204+0000 7fc645d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc650071960 0x7fc65015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.204+0000 7fc64e59c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc6500b4290 0x7fc6500b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46006/0 (socket says 172.21.15.105:46006) 2023-11-28T18:26:45.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.204+0000 7fc64e59c640 1 -- 172.21.15.105:0/3667327076 learned_addr learned my addr 172.21.15.105:0/3667327076 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:45.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.204+0000 7fc64ed9d640 1 --2- 172.21.15.105:0/3667327076 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc65015b030 0x7fc6500709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.204+0000 7fc645d9b640 1 -- 172.21.15.105:0/3667327076 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc65015b030 msgr2=0x7fc6500709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:45.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.204+0000 7fc645d9b640 1 --2- 172.21.15.105:0/3667327076 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc65015b030 0x7fc6500709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.204+0000 7fc645d9b640 1 -- 172.21.15.105:0/3667327076 >> v1:172.21.15.105:6789/0 conn(0x7fc6500b4290 legacy=0x7fc6500b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.204+0000 7fc645d9b640 1 -- 172.21.15.105:0/3667327076 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6500720d0 con 0x7fc650071960 2023-11-28T18:26:45.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.204+0000 7fc64dd9b640 1 -- 172.21.15.105:0/3667327076 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688436647 0 0) 0x7fc6500701d0 con 0x7fc6500b4290 2023-11-28T18:26:45.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.204+0000 7fc64ed9d640 1 --2- 172.21.15.105:0/3667327076 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc65015b030 0x7fc6500709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:45.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.208+0000 7fc645d9b640 1 --2- 172.21.15.105:0/3667327076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc650071960 0x7fc65015f3b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc6400097c0 tx=0x7fc64001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c648c2128c81226 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:45.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.208+0000 7fc64dd9b640 1 -- 172.21.15.105:0/3667327076 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6400ad660 con 0x7fc650071960 2023-11-28T18:26:45.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.208+0000 7fc64dd9b640 1 -- 172.21.15.105:0/3667327076 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6400ad7c0 con 0x7fc650071960 2023-11-28T18:26:45.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.208+0000 7fc64dd9b640 1 -- 172.21.15.105:0/3667327076 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6400adb50 con 0x7fc650071960 2023-11-28T18:26:45.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.208+0000 7fc654c9c640 1 -- 172.21.15.105:0/3667327076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc650071960 msgr2=0x7fc65015f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.208+0000 7fc654c9c640 1 --2- 172.21.15.105:0/3667327076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc650071960 0x7fc65015f3b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc6400097c0 tx=0x7fc64001c450 comp rx=0 tx=0).stop 2023-11-28T18:26:45.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.208+0000 7fc654c9c640 1 -- 172.21.15.105:0/3667327076 shutdown_connections 2023-11-28T18:26:45.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.208+0000 7fc654c9c640 1 --2- 172.21.15.105:0/3667327076 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc65015b030 0x7fc6500709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.208+0000 7fc654c9c640 1 --2- 172.21.15.105:0/3667327076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc650071960 0x7fc65015f3b0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.208+0000 7fc654c9c640 1 -- 172.21.15.105:0/3667327076 >> 172.21.15.105:0/3667327076 conn(0x7fc6500ceb90 msgr2=0x7fc6500cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:45.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.208+0000 7fc654c9c640 1 -- 172.21.15.105:0/3667327076 shutdown_connections 2023-11-28T18:26:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.208+0000 7fc654c9c640 1 -- 172.21.15.105:0/3667327076 wait complete. 2023-11-28T18:26:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.208+0000 7fc654c9c640 1 Processor -- start 2023-11-28T18:26:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.208+0000 7fc654c9c640 1 -- start start 2023-11-28T18:26:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc654c9c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6500b4290 0x7fc650154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc654c9c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc65015b030 0x7fc65015dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc654c9c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc650099a50 0x7fc650099f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc654c9c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65009a440 con 0x7fc6500b4290 2023-11-28T18:26:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc654c9c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65014e8b0 con 0x7fc65015b030 2023-11-28T18:26:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc654c9c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc650153f90 con 0x7fc650099a50 2023-11-28T18:26:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc64e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6500b4290 0x7fc650154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc64e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6500b4290 0x7fc650154420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42126/0 (socket says 172.21.15.105:42126) 2023-11-28T18:26:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc64e59c640 1 -- 172.21.15.105:0/691311880 learned_addr learned my addr 172.21.15.105:0/691311880 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc64ed9d640 1 --2- 172.21.15.105:0/691311880 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc650099a50 0x7fc650099f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc645d9b640 1 --2- 172.21.15.105:0/691311880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc65015b030 0x7fc65015dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc64e59c640 1 -- 172.21.15.105:0/691311880 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc650099a50 msgr2=0x7fc650099f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc64e59c640 1 --2- 172.21.15.105:0/691311880 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc650099a50 0x7fc650099f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc64e59c640 1 -- 172.21.15.105:0/691311880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc65015b030 msgr2=0x7fc65015dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc64e59c640 1 --2- 172.21.15.105:0/691311880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc65015b030 0x7fc65015dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc64e59c640 1 -- 172.21.15.105:0/691311880 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc640009420 con 0x7fc6500b4290 2023-11-28T18:26:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc645d9b640 1 --2- 172.21.15.105:0/691311880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc65015b030 0x7fc65015dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc64ed9d640 1 --2- 172.21.15.105:0/691311880 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc650099a50 0x7fc650099f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc64e59c640 1 --2- 172.21.15.105:0/691311880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6500b4290 0x7fc650154420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc63800c520 tx=0x7fc63800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc6477fe640 1 -- 172.21.15.105:0/691311880 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6380074e0 con 0x7fc6500b4290 2023-11-28T18:26:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc6477fe640 1 -- 172.21.15.105:0/691311880 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc638007640 con 0x7fc6500b4290 2023-11-28T18:26:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc654c9c640 1 -- 172.21.15.105:0/691311880 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65015e190 con 0x7fc6500b4290 2023-11-28T18:26:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc6477fe640 1 -- 172.21.15.105:0/691311880 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6380049f0 con 0x7fc6500b4290 2023-11-28T18:26:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc654c9c640 1 -- 172.21.15.105:0/691311880 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65013be10 con 0x7fc6500b4290 2023-11-28T18:26:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.212+0000 7fc654c9c640 1 -- 172.21.15.105:0/691311880 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc650067570 con 0x7fc6500b4290 2023-11-28T18:26:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.216+0000 7fc6477fe640 1 -- 172.21.15.105:0/691311880 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc638004b50 con 0x7fc6500b4290 2023-11-28T18:26:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.216+0000 7fc6477fe640 1 --2- 172.21.15.105:0/691311880 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc630066900 0x7fc630068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.216+0000 7fc6477fe640 1 -- 172.21.15.105:0/691311880 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7fc6380837d0 con 0x7fc6500b4290 2023-11-28T18:26:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.216+0000 7fc645d9b640 1 --2- 172.21.15.105:0/691311880 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc630066900 0x7fc630068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.216+0000 7fc645d9b640 1 --2- 172.21.15.105:0/691311880 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc630066900 0x7fc630068dc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc6400af000 tx=0x7fc6400a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:45.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.220+0000 7fc6477fe640 1 -- 172.21.15.105:0/691311880 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc638050850 con 0x7fc6500b4290 2023-11-28T18:26:45.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.404+0000 7fc654c9c640 1 -- 172.21.15.105:0/691311880 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc650165b60 con 0x7fc6500b4290 2023-11-28T18:26:45.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.404+0000 7fc6477fe640 1 -- 172.21.15.105:0/691311880 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4955 (secure 0 0 0) 0x7fc638018090 con 0x7fc6500b4290 2023-11-28T18:26:45.409 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:45.409 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":47,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1686803161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":1686803161},{"type":"v1","addr":"172.21.15.148:6835","nonce":1686803161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:26:00.550298+0000","modified":"2023-11-28T18:26:31.373860+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5842,"mds_1":5405,"mds_2":5408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5405":{"gid":5405,"name":"a","rank":1,"incarnation":43,"state":"up:active","state_seq":13,"addr":"172.21.15.105:6836/735646078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":735646078},{"type":"v1","addr":"172.21.15.105:6836","nonce":735646078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5408":{"gid":5408,"name":"c","rank":2,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6837/3765684108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":3765684108},{"type":"v1","addr":"172.21.15.105:6837","nonce":3765684108}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5842":{"gid":5842,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6834/2696773413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2696773413},{"type":"v1","addr":"172.21.15.148:6834","nonce":2696773413}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T18:26:45.409 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 47 2023-11-28T18:26:45.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.408+0000 7fc654c9c640 1 -- 172.21.15.105:0/691311880 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc630066900 msgr2=0x7fc630068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.408+0000 7fc654c9c640 1 --2- 172.21.15.105:0/691311880 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc630066900 0x7fc630068dc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc6400af000 tx=0x7fc6400a7000 comp rx=0 tx=0).stop 2023-11-28T18:26:45.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.408+0000 7fc654c9c640 1 -- 172.21.15.105:0/691311880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6500b4290 msgr2=0x7fc650154420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.408+0000 7fc654c9c640 1 --2- 172.21.15.105:0/691311880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6500b4290 0x7fc650154420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc63800c520 tx=0x7fc63800c9e0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.408+0000 7fc654c9c640 1 -- 172.21.15.105:0/691311880 shutdown_connections 2023-11-28T18:26:45.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.408+0000 7fc654c9c640 1 --2- 172.21.15.105:0/691311880 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc650099a50 0x7fc650099f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.408+0000 7fc654c9c640 1 --2- 172.21.15.105:0/691311880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc65015b030 0x7fc65015dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.408+0000 7fc654c9c640 1 --2- 172.21.15.105:0/691311880 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc630066900 0x7fc630068dc0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.408+0000 7fc654c9c640 1 --2- 172.21.15.105:0/691311880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6500b4290 0x7fc650154420 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.408+0000 7fc654c9c640 1 -- 172.21.15.105:0/691311880 >> 172.21.15.105:0/691311880 conn(0x7fc6500ceb90 msgr2=0x7fc65015f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:45.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.408+0000 7fc654c9c640 1 -- 172.21.15.105:0/691311880 shutdown_connections 2023-11-28T18:26:45.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.408+0000 7fc654c9c640 1 -- 172.21.15.105:0/691311880 wait complete. 2023-11-28T18:26:45.430 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:1 perf dump 2023-11-28T18:26:45.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.564+0000 7f99d74c8640 1 Processor -- start 2023-11-28T18:26:45.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.564+0000 7f99d74c8640 1 -- start start 2023-11-28T18:26:45.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.568+0000 7f99d74c8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f99d015e9f0 0x7f99d006de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.568+0000 7f99d74c8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f99d006c070 0x7f99d015edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.568+0000 7f99d74c8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99d006e3b0 con 0x7f99d015b420 2023-11-28T18:26:45.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.568+0000 7f99d74c8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d00cd8f0 con 0x7f99d006c070 2023-11-28T18:26:45.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.568+0000 7f99d74c8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d015f310 con 0x7f99d015e9f0 2023-11-28T18:26:45.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.568+0000 7f99d5cc5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f99d006c070 0x7f99d015edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.568+0000 7f99d5cc5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f99d006c070 0x7f99d015edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42138/0 (socket says 172.21.15.105:42138) 2023-11-28T18:26:45.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.568+0000 7f99d5cc5640 1 -- 172.21.15.105:0/1706828427 learned_addr learned my addr 172.21.15.105:0/1706828427 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:45.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.568+0000 7f99d64c6640 1 --2- 172.21.15.105:0/1706828427 >> v2:172.21.15.148:3300/0 conn(0x7f99d015e9f0 0x7f99d006de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.568+0000 7f99d5cc5640 1 -- 172.21.15.105:0/1706828427 >> v2:172.21.15.148:3300/0 conn(0x7f99d015e9f0 msgr2=0x7f99d006de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:45.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.568+0000 7f99d5cc5640 1 --2- 172.21.15.105:0/1706828427 >> v2:172.21.15.148:3300/0 conn(0x7f99d015e9f0 0x7f99d006de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.568+0000 7f99d5cc5640 1 -- 172.21.15.105:0/1706828427 >> v1:172.21.15.148:6789/0 conn(0x7f99d015b420 legacy=0x7f99d015b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:45.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.568+0000 7f99d5cc5640 1 -- 172.21.15.105:0/1706828427 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d006c500 con 0x7f99d006c070 2023-11-28T18:26:45.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.568+0000 7f99d64c6640 1 --2- 172.21.15.105:0/1706828427 >> v2:172.21.15.148:3300/0 conn(0x7f99d015e9f0 0x7f99d006de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:45.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.572+0000 7f99d5cc5640 1 --2- 172.21.15.105:0/1706828427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99d006c070 0x7f99d015edd0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f99d0073bd0 tx=0x7f99c001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893fd03c54033dab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:45.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.572+0000 7f99d54c4640 1 -- 172.21.15.105:0/1706828427 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c00ac410 con 0x7f99d006c070 2023-11-28T18:26:45.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.572+0000 7f99d54c4640 1 -- 172.21.15.105:0/1706828427 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c00ac570 con 0x7f99d006c070 2023-11-28T18:26:45.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.572+0000 7f99d54c4640 1 -- 172.21.15.105:0/1706828427 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c00ac900 con 0x7f99d006c070 2023-11-28T18:26:45.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.572+0000 7f99d74c8640 1 -- 172.21.15.105:0/1706828427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99d006c070 msgr2=0x7f99d015edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.572+0000 7f99d74c8640 1 --2- 172.21.15.105:0/1706828427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99d006c070 0x7f99d015edd0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f99d0073bd0 tx=0x7f99c001c4c0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.572+0000 7f99d74c8640 1 -- 172.21.15.105:0/1706828427 shutdown_connections 2023-11-28T18:26:45.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.572+0000 7f99d74c8640 1 --2- 172.21.15.105:0/1706828427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99d006c070 0x7f99d015edd0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.572+0000 7f99d74c8640 1 --2- 172.21.15.105:0/1706828427 >> v2:172.21.15.148:3300/0 conn(0x7f99d015e9f0 0x7f99d006de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.572+0000 7f99d74c8640 1 -- 172.21.15.105:0/1706828427 >> 172.21.15.105:0/1706828427 conn(0x7f99d00ce9e0 msgr2=0x7f99d00cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:45.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.572+0000 7f99d74c8640 1 -- 172.21.15.105:0/1706828427 shutdown_connections 2023-11-28T18:26:45.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.572+0000 7f99d74c8640 1 -- 172.21.15.105:0/1706828427 wait complete. 2023-11-28T18:26:45.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.572+0000 7f99d74c8640 1 Processor -- start 2023-11-28T18:26:45.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.572+0000 7f99d74c8640 1 -- start start 2023-11-28T18:26:45.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.576+0000 7f99d74c8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99d015b420 0x7f99d013b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.576+0000 7f99d74c8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99d015e9f0 0x7f99d013a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.576+0000 7f99d74c8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99d0155b20 0x7f99d0155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.576+0000 7f99d74c8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d013b690 con 0x7f99d015b420 2023-11-28T18:26:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.576+0000 7f99d74c8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d0155690 con 0x7f99d015e9f0 2023-11-28T18:26:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.576+0000 7f99d74c8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d013aba0 con 0x7f99d0155b20 2023-11-28T18:26:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.576+0000 7f99d64c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99d015b420 0x7f99d013b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.576+0000 7f99d64c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99d015b420 0x7f99d013b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42148/0 (socket says 172.21.15.105:42148) 2023-11-28T18:26:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.576+0000 7f99d64c6640 1 -- 172.21.15.105:0/8087652 learned_addr learned my addr 172.21.15.105:0/8087652 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.576+0000 7f99d5cc5640 1 --2- 172.21.15.105:0/8087652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99d015e9f0 0x7f99d013a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.576+0000 7f99d6cc7640 1 --2- 172.21.15.105:0/8087652 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99d0155b20 0x7f99d0155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.576+0000 7f99d64c6640 1 -- 172.21.15.105:0/8087652 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99d0155b20 msgr2=0x7f99d0155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.576+0000 7f99d64c6640 1 --2- 172.21.15.105:0/8087652 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99d0155b20 0x7f99d0155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.576+0000 7f99d64c6640 1 -- 172.21.15.105:0/8087652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99d015e9f0 msgr2=0x7f99d013a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.576+0000 7f99d64c6640 1 --2- 172.21.15.105:0/8087652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99d015e9f0 0x7f99d013a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.576+0000 7f99d64c6640 1 -- 172.21.15.105:0/8087652 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c00094d0 con 0x7f99d015b420 2023-11-28T18:26:45.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.576+0000 7f99d64c6640 1 --2- 172.21.15.105:0/8087652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99d015b420 0x7f99d013b150 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00c7f0 tx=0x7f99cc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:45.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.576+0000 7f99c6ffd640 1 -- 172.21.15.105:0/8087652 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc00cea0 con 0x7f99d015b420 2023-11-28T18:26:45.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.576+0000 7f99c6ffd640 1 -- 172.21.15.105:0/8087652 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99cc007a30 con 0x7f99d015b420 2023-11-28T18:26:45.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.576+0000 7f99c6ffd640 1 -- 172.21.15.105:0/8087652 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc007de0 con 0x7f99d015b420 2023-11-28T18:26:45.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.576+0000 7f99d74c8640 1 -- 172.21.15.105:0/8087652 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99d0154d60 con 0x7f99d015b420 2023-11-28T18:26:45.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.576+0000 7f99d74c8640 1 -- 172.21.15.105:0/8087652 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d0155320 con 0x7f99d015b420 2023-11-28T18:26:45.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.580+0000 7f99c6ffd640 1 -- 172.21.15.105:0/8087652 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f99cc0113f0 con 0x7f99d015b420 2023-11-28T18:26:45.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.580+0000 7f99c6ffd640 1 --2- 172.21.15.105:0/8087652 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99b40668d0 0x7f99b4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.580+0000 7f99c6ffd640 1 -- 172.21.15.105:0/8087652 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f99cc084590 con 0x7f99d015b420 2023-11-28T18:26:45.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.580+0000 7f99d5cc5640 1 --2- 172.21.15.105:0/8087652 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99b40668d0 0x7f99b4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.580+0000 7f99dbaac000 1 Processor -- start 2023-11-28T18:26:45.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.580+0000 7f99dbaac000 1 -- start start 2023-11-28T18:26:45.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.580+0000 7f99d5cc5640 1 --2- 172.21.15.105:0/8087652 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99b40668d0 0x7f99b4068d90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f99c00ae000 tx=0x7f99c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:45.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.580+0000 7f99dbaac000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bc614f67c0 0x55bc614f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.580+0000 7f99dbaac000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bc614f70e0 0x55bc614abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.580+0000 7f99dbaac000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614ac520 0x55bc614ac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.580+0000 7f99dbaac000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc613039e0 con 0x55bc614ac520 2023-11-28T18:26:45.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.580+0000 7f99dbaac000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc61300f10 con 0x55bc614f70e0 2023-11-28T18:26:45.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.580+0000 7f99dbaac000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc612fc800 con 0x55bc614f67c0 2023-11-28T18:26:45.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.580+0000 7f99d6cc7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614ac520 0x55bc614ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.580+0000 7f99d6cc7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614ac520 0x55bc614ac900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42160/0 (socket says 172.21.15.105:42160) 2023-11-28T18:26:45.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.580+0000 7f99d6cc7640 1 -- 172.21.15.105:0/772022456 learned_addr learned my addr 172.21.15.105:0/772022456 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:45.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.580+0000 7f99d64c6640 1 --2- 172.21.15.105:0/772022456 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bc614f67c0 0x55bc614f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.580+0000 7f99d5cc5640 1 --2- 172.21.15.105:0/772022456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bc614f70e0 0x55bc614abfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99d6cc7640 1 -- 172.21.15.105:0/772022456 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bc614f67c0 msgr2=0x55bc614f6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:45.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99d6cc7640 1 --2- 172.21.15.105:0/772022456 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bc614f67c0 0x55bc614f6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99d6cc7640 1 -- 172.21.15.105:0/772022456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bc614f70e0 msgr2=0x55bc614abfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:45.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99d6cc7640 1 --2- 172.21.15.105:0/772022456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bc614f70e0 0x55bc614abfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99d6cc7640 1 -- 172.21.15.105:0/772022456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc60ff4dc0 con 0x55bc614ac520 2023-11-28T18:26:45.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99d64c6640 1 --2- 172.21.15.105:0/772022456 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bc614f67c0 0x55bc614f6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:26:45.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99d6cc7640 1 --2- 172.21.15.105:0/772022456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614ac520 0x55bc614ac900 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f99c80056d0 tx=0x7f99c800f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d5075d1e5a040f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:45.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99b3fff640 1 -- 172.21.15.105:0/772022456 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c80051a0 con 0x55bc614ac520 2023-11-28T18:26:45.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99b3fff640 1 -- 172.21.15.105:0/772022456 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c800fec0 con 0x55bc614ac520 2023-11-28T18:26:45.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99b3fff640 1 -- 172.21.15.105:0/772022456 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c8014670 con 0x55bc614ac520 2023-11-28T18:26:45.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99dbaac000 1 -- 172.21.15.105:0/772022456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614ac520 msgr2=0x55bc614ac900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99dbaac000 1 --2- 172.21.15.105:0/772022456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614ac520 0x55bc614ac900 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f99c80056d0 tx=0x7f99c800f5a0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99dbaac000 1 -- 172.21.15.105:0/772022456 shutdown_connections 2023-11-28T18:26:45.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99dbaac000 1 --2- 172.21.15.105:0/772022456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614ac520 0x55bc614ac900 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99dbaac000 1 --2- 172.21.15.105:0/772022456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bc614f70e0 0x55bc614abfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99dbaac000 1 --2- 172.21.15.105:0/772022456 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bc614f67c0 0x55bc614f6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99dbaac000 1 -- 172.21.15.105:0/772022456 >> 172.21.15.105:0/772022456 conn(0x55bc614b4fb0 msgr2=0x55bc614d9360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:45.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99dbaac000 1 -- 172.21.15.105:0/772022456 shutdown_connections 2023-11-28T18:26:45.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99dbaac000 1 -- 172.21.15.105:0/772022456 wait complete. 2023-11-28T18:26:45.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99dbaac000 1 -- start start 2023-11-28T18:26:45.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99dbaac000 1 Processor -- start 2023-11-28T18:26:45.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99dbaac000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bc614ac520 0x55bc614fd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99dbaac000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bc614f67c0 0x55bc614fde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99dbaac000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614f70e0 0x55bc614da590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99dbaac000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc612baa70 con 0x55bc614f70e0 2023-11-28T18:26:45.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99dbaac000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc612ba710 con 0x55bc614ac520 2023-11-28T18:26:45.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99dbaac000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc60fb5a40 con 0x55bc614f67c0 2023-11-28T18:26:45.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99dbaac000 10 client.0 _ll_register_callbacks cb 0x55bc61497ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:45.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99d6cc7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614f70e0 0x55bc614da590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99d6cc7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614f70e0 0x55bc614da590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42170/0 (socket says 172.21.15.105:42170) 2023-11-28T18:26:45.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99d6cc7640 1 -- 172.21.15.105:0/2783186748 learned_addr learned my addr 172.21.15.105:0/2783186748 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:45.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99d5cc5640 1 --2- 172.21.15.105:0/2783186748 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bc614f67c0 0x55bc614fde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.584+0000 7f99d64c6640 1 --2- 172.21.15.105:0/2783186748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bc614ac520 0x55bc614fd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99d6cc7640 1 -- 172.21.15.105:0/2783186748 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bc614f67c0 msgr2=0x55bc614fde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:45.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99d6cc7640 1 --2- 172.21.15.105:0/2783186748 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bc614f67c0 0x55bc614fde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99d6cc7640 1 -- 172.21.15.105:0/2783186748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bc614ac520 msgr2=0x55bc614fd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:45.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99d6cc7640 1 --2- 172.21.15.105:0/2783186748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bc614ac520 0x55bc614fd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99d6cc7640 1 -- 172.21.15.105:0/2783186748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d0155320 con 0x55bc614f70e0 2023-11-28T18:26:45.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99d6cc7640 1 --2- 172.21.15.105:0/2783186748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614f70e0 0x55bc614da590 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f99c800faa0 tx=0x7f99c8003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8e3c1789e2205 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:45.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:45.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99b17fa640 1 -- 172.21.15.105:0/2783186748 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c8015990 con 0x55bc614f70e0 2023-11-28T18:26:45.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99b17fa640 1 -- 172.21.15.105:0/2783186748 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f99c8023070 con 0x55bc614f70e0 2023-11-28T18:26:45.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99dbaac000 1 -- 172.21.15.105:0/2783186748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc612b5ab0 con 0x55bc614f70e0 2023-11-28T18:26:45.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99b17fa640 1 -- 172.21.15.105:0/2783186748 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99c8015af0 con 0x55bc614f70e0 2023-11-28T18:26:45.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99dbaac000 10 client.6444 fetch_fsmap learned FSMap version 47 2023-11-28T18:26:45.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99dbaac000 1 -- 172.21.15.105:0/2783186748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x7f99c8005300 con 0x55bc614f70e0 2023-11-28T18:26:45.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99b17fa640 1 -- 172.21.15.105:0/2783186748 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bc612b5ab0 con 0x55bc614f70e0 2023-11-28T18:26:45.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99b17fa640 1 -- 172.21.15.105:0/2783186748 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 47) v1 ==== 3326+0+0 (secure 0 0 0) 0x7f99c8012070 con 0x55bc614f70e0 2023-11-28T18:26:45.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99dbaac000 10 client.6444 fetch_fsmap finished waiting for FSMap version 47 2023-11-28T18:26:45.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99dbaac000 10 client.6444 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.a 2023-11-28T18:26:45.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99dbaac000 20 client.6444 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:45.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99dbaac000 1 --2- 172.21.15.105:0/2783186748 >> [v2:172.21.15.105:6834/735646078,v1:172.21.15.105:6836/735646078] conn(0x55bc614db270 0x55bc614db650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99dbaac000 4 client.6444 mds_command: new command op to 5405 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:26:45.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99dbaac000 1 -- 172.21.15.105:0/2783186748 --> [v2:172.21.15.105:6834/735646078,v1:172.21.15.105:6836/735646078] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc612b1850 con 0x55bc614db270 2023-11-28T18:26:45.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99d64c6640 1 --2- 172.21.15.105:0/2783186748 >> [v2:172.21.15.105:6834/735646078,v1:172.21.15.105:6836/735646078] conn(0x55bc614db270 0x55bc614db650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99d64c6640 1 --2- 172.21.15.105:0/2783186748 >> [v2:172.21.15.105:6834/735646078,v1:172.21.15.105:6836/735646078] conn(0x55bc614db270 0x55bc614db650 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=509cd2db8d3c9f3c server_cookie=c7b7ed7259af564d in_seq=0 out_seq=0 2023-11-28T18:26:45.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99b17fa640 10 client.6444 ms_handle_connect on v2:172.21.15.105:6834/735646078 2023-11-28T18:26:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99b17fa640 1 -- 172.21.15.105:0/2783186748 <== mds.1 v2:172.21.15.105:6834/735646078 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bc612b1850 con 0x55bc614db270 2023-11-28T18:26:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99b17fa640 10 client.6444 handle_command_reply: tid=0 2023-11-28T18:26:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99dbaac000 1 client.6444 shutdown 2023-11-28T18:26:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99dbaac000 1 -- 172.21.15.105:0/2783186748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614f70e0 msgr2=0x55bc614da590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99dbaac000 1 --2- 172.21.15.105:0/2783186748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614f70e0 0x55bc614da590 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f99c800faa0 tx=0x7f99c8003a80 comp rx=0 tx=0).stop 2023-11-28T18:26:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99dbaac000 1 -- 172.21.15.105:0/2783186748 shutdown_connections 2023-11-28T18:26:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99dbaac000 1 --2- 172.21.15.105:0/2783186748 >> [v2:172.21.15.105:6834/735646078,v1:172.21.15.105:6836/735646078] conn(0x55bc614db270 0x55bc614db650 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99dbaac000 1 --2- 172.21.15.105:0/2783186748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614f70e0 0x55bc614da590 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99dbaac000 1 --2- 172.21.15.105:0/2783186748 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bc614f67c0 0x55bc614fde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99b17fa640 0 client.6444 ms_handle_reset on v2:172.21.15.105:6834/735646078 2023-11-28T18:26:45.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99dbaac000 1 --2- 172.21.15.105:0/2783186748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bc614ac520 0x55bc614fd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99dbaac000 1 -- 172.21.15.105:0/2783186748 >> 172.21.15.105:0/2783186748 conn(0x55bc614b4fb0 msgr2=0x55bc614ab6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:45.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99dbaac000 1 -- 172.21.15.105:0/2783186748 shutdown_connections 2023-11-28T18:26:45.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99dbaac000 1 -- 172.21.15.105:0/2783186748 wait complete. 2023-11-28T18:26:45.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.588+0000 7f99dbaac000 20 client.6444 trim_cache size 0 max 16384 2023-11-28T18:26:45.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.604+0000 7f99dbaac000 1 Processor -- start 2023-11-28T18:26:45.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.604+0000 7f99dbaac000 1 -- start start 2023-11-28T18:26:45.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.604+0000 7f99dbaac000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bc614b4fb0 0x55bc614f5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.604+0000 7f99dbaac000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614ac520 0x55bc614f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.604+0000 7f99dbaac000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bc614f67c0 0x55bc614aa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.604+0000 7f99dbaac000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc614b10f0 con 0x55bc614ac520 2023-11-28T18:26:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.604+0000 7f99dbaac000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc61280c20 con 0x55bc614b4fb0 2023-11-28T18:26:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.604+0000 7f99dbaac000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc612d27b0 con 0x55bc614f67c0 2023-11-28T18:26:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.604+0000 7f99d5cc5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614ac520 0x55bc614f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.604+0000 7f99d5cc5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614ac520 0x55bc614f5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42182/0 (socket says 172.21.15.105:42182) 2023-11-28T18:26:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.604+0000 7f99d5cc5640 1 -- 172.21.15.105:0/825112365 learned_addr learned my addr 172.21.15.105:0/825112365 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.604+0000 7f99d64c6640 1 --2- 172.21.15.105:0/825112365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bc614b4fb0 0x55bc614f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.604+0000 7f99d6cc7640 1 --2- 172.21.15.105:0/825112365 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bc614f67c0 0x55bc614aa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.604+0000 7f99d5cc5640 1 -- 172.21.15.105:0/825112365 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bc614f67c0 msgr2=0x55bc614aa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.604+0000 7f99d5cc5640 1 --2- 172.21.15.105:0/825112365 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bc614f67c0 0x55bc614aa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.604+0000 7f99d5cc5640 1 -- 172.21.15.105:0/825112365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bc614b4fb0 msgr2=0x55bc614f5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:45.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.604+0000 7f99d5cc5640 1 --2- 172.21.15.105:0/825112365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bc614b4fb0 0x55bc614f5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.604+0000 7f99d5cc5640 1 -- 172.21.15.105:0/825112365 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc612bd260 con 0x55bc614ac520 2023-11-28T18:26:45.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99d5cc5640 1 --2- 172.21.15.105:0/825112365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614ac520 0x55bc614f5c60 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f99c001ab70 tx=0x7f99c00a6040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9be1d657a023b8e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:45.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99c4ff9640 1 -- 172.21.15.105:0/825112365 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c00bd050 con 0x55bc614ac520 2023-11-28T18:26:45.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99c4ff9640 1 -- 172.21.15.105:0/825112365 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c00be040 con 0x55bc614ac520 2023-11-28T18:26:45.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99c4ff9640 1 -- 172.21.15.105:0/825112365 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c00b77a0 con 0x55bc614ac520 2023-11-28T18:26:45.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99dbaac000 1 -- 172.21.15.105:0/825112365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614ac520 msgr2=0x55bc614f5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99dbaac000 1 --2- 172.21.15.105:0/825112365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614ac520 0x55bc614f5c60 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f99c001ab70 tx=0x7f99c00a6040 comp rx=0 tx=0).stop 2023-11-28T18:26:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99dbaac000 1 -- 172.21.15.105:0/825112365 shutdown_connections 2023-11-28T18:26:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99dbaac000 1 --2- 172.21.15.105:0/825112365 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bc614f67c0 0x55bc614aa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99dbaac000 1 --2- 172.21.15.105:0/825112365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614ac520 0x55bc614f5c60 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99dbaac000 1 --2- 172.21.15.105:0/825112365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bc614b4fb0 0x55bc614f5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99dbaac000 1 -- 172.21.15.105:0/825112365 >> 172.21.15.105:0/825112365 conn(0x55bc614fcdf0 msgr2=0x55bc61503ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99dbaac000 1 -- 172.21.15.105:0/825112365 shutdown_connections 2023-11-28T18:26:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99dbaac000 1 -- 172.21.15.105:0/825112365 wait complete. 2023-11-28T18:26:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99dbaac000 1 -- start start 2023-11-28T18:26:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99dbaac000 1 Processor -- start 2023-11-28T18:26:45.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99dbaac000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bc614ac520 0x55bc614fa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99dbaac000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bc614b4fb0 0x55bc6151c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99dbaac000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614f67c0 0x55bc6151cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99dbaac000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc612ff360 con 0x55bc614f67c0 2023-11-28T18:26:45.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99dbaac000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc612fb690 con 0x55bc614ac520 2023-11-28T18:26:45.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99dbaac000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc612b65d0 con 0x55bc614b4fb0 2023-11-28T18:26:45.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99dbaac000 10 client.0 _ll_register_callbacks cb 0x55bc612c4840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:45.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99d6cc7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614f67c0 0x55bc6151cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99d6cc7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614f67c0 0x55bc6151cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42188/0 (socket says 172.21.15.105:42188) 2023-11-28T18:26:45.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99d6cc7640 1 -- 172.21.15.105:0/2391638813 learned_addr learned my addr 172.21.15.105:0/2391638813 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:45.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99d64c6640 1 --2- 172.21.15.105:0/2391638813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bc614ac520 0x55bc614fa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99d5cc5640 1 --2- 172.21.15.105:0/2391638813 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bc614b4fb0 0x55bc6151c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99d6cc7640 1 -- 172.21.15.105:0/2391638813 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bc614b4fb0 msgr2=0x55bc6151c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:45.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99d6cc7640 1 --2- 172.21.15.105:0/2391638813 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bc614b4fb0 0x55bc6151c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99d6cc7640 1 -- 172.21.15.105:0/2391638813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bc614ac520 msgr2=0x55bc614fa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:45.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99d6cc7640 1 --2- 172.21.15.105:0/2391638813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bc614ac520 0x55bc614fa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.608+0000 7f99d6cc7640 1 -- 172.21.15.105:0/2391638813 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d006c500 con 0x55bc614f67c0 2023-11-28T18:26:45.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99d6cc7640 1 --2- 172.21.15.105:0/2391638813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614f67c0 0x55bc6151cd20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f99c800faa0 tx=0x7f99c8003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a028c37ba7d365ac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:45.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:45.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99b17fa640 1 -- 172.21.15.105:0/2391638813 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c800fb50 con 0x55bc614f67c0 2023-11-28T18:26:45.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99b17fa640 1 -- 172.21.15.105:0/2391638813 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f99c8023030 con 0x55bc614f67c0 2023-11-28T18:26:45.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99dbaac000 1 -- 172.21.15.105:0/2391638813 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc614b1230 con 0x55bc614f67c0 2023-11-28T18:26:45.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99b17fa640 1 -- 172.21.15.105:0/2391638813 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99c8005520 con 0x55bc614f67c0 2023-11-28T18:26:45.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99dbaac000 10 client.6450 fetch_fsmap learned FSMap version 47 2023-11-28T18:26:45.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99dbaac000 1 -- 172.21.15.105:0/2391638813 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55bc612a3760 con 0x55bc614f67c0 2023-11-28T18:26:45.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99b17fa640 1 -- 172.21.15.105:0/2391638813 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bc614b1230 con 0x55bc614f67c0 2023-11-28T18:26:45.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99b17fa640 1 -- 172.21.15.105:0/2391638813 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 47) v1 ==== 3326+0+0 (secure 0 0 0) 0x7f99c8012040 con 0x55bc614f67c0 2023-11-28T18:26:45.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99dbaac000 10 client.6450 fetch_fsmap finished waiting for FSMap version 47 2023-11-28T18:26:45.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99dbaac000 10 client.6450 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.a 2023-11-28T18:26:45.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99dbaac000 20 client.6450 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:45.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99dbaac000 1 --2- 172.21.15.105:0/2391638813 >> [v2:172.21.15.105:6834/735646078,v1:172.21.15.105:6836/735646078] conn(0x55bc614f70e0 0x55bc61567c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99dbaac000 4 client.6450 mds_command: new command op to 5405 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2023-11-28T18:26:45.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99dbaac000 1 -- 172.21.15.105:0/2391638813 --> [v2:172.21.15.105:6834/735646078,v1:172.21.15.105:6836/735646078] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55bc612bf3a0 con 0x55bc614f70e0 2023-11-28T18:26:45.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99d64c6640 1 --2- 172.21.15.105:0/2391638813 >> [v2:172.21.15.105:6834/735646078,v1:172.21.15.105:6836/735646078] conn(0x55bc614f70e0 0x55bc61567c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99b17fa640 10 client.6450 ms_handle_connect on v2:172.21.15.105:6834/735646078 2023-11-28T18:26:45.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99d64c6640 1 --2- 172.21.15.105:0/2391638813 >> [v2:172.21.15.105:6834/735646078,v1:172.21.15.105:6836/735646078] conn(0x55bc614f70e0 0x55bc61567c50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1dc3fb4a643e2338 server_cookie=a054f04dcf87d08d in_seq=0 out_seq=0 2023-11-28T18:26:45.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99b17fa640 1 -- 172.21.15.105:0/2391638813 <== mds.1 v2:172.21.15.105:6834/735646078 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17613 (crc 0 0 0) 0x55bc612bf3a0 con 0x55bc614f70e0 2023-11-28T18:26:45.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99b17fa640 10 client.6450 handle_command_reply: tid=0 2023-11-28T18:26:45.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99dbaac000 1 client.6450 shutdown 2023-11-28T18:26:45.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99dbaac000 1 -- 172.21.15.105:0/2391638813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614f67c0 msgr2=0x55bc6151cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99dbaac000 1 --2- 172.21.15.105:0/2391638813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614f67c0 0x55bc6151cd20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f99c800faa0 tx=0x7f99c8003d50 comp rx=0 tx=0).stop 2023-11-28T18:26:45.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99dbaac000 1 -- 172.21.15.105:0/2391638813 shutdown_connections 2023-11-28T18:26:45.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99dbaac000 1 --2- 172.21.15.105:0/2391638813 >> [v2:172.21.15.105:6834/735646078,v1:172.21.15.105:6836/735646078] conn(0x55bc614f70e0 0x55bc61567c50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99dbaac000 1 --2- 172.21.15.105:0/2391638813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc614f67c0 0x55bc6151cd20 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99dbaac000 1 --2- 172.21.15.105:0/2391638813 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bc614b4fb0 0x55bc6151c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99dbaac000 1 --2- 172.21.15.105:0/2391638813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bc614ac520 0x55bc614fa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99dbaac000 1 -- 172.21.15.105:0/2391638813 >> 172.21.15.105:0/2391638813 conn(0x55bc614fcdf0 msgr2=0x55bc61503760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:45.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99b17fa640 0 client.6450 ms_handle_reset on v2:172.21.15.105:6834/735646078 2023-11-28T18:26:45.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99dbaac000 1 -- 172.21.15.105:0/2391638813 shutdown_connections 2023-11-28T18:26:45.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99dbaac000 1 -- 172.21.15.105:0/2391638813 wait complete. 2023-11-28T18:26:45.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99dbaac000 20 client.6450 trim_cache size 0 max 16384 2023-11-28T18:26:45.620 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:26:45.621 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2023-11-28T18:26:45.621 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 23, 2023-11-28T18:26:45.621 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 27, 2023-11-28T18:26:45.621 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 61898, 2023-11-28T18:26:45.621 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 7955, 2023-11-28T18:26:45.621 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 10, 2023-11-28T18:26:45.621 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 3, 2023-11-28T18:26:45.621 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.010582645, 2023-11-28T18:26:45.621 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.000718447, 2023-11-28T18:26:45.621 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.005040131, 2023-11-28T18:26:45.621 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000465368, 2023-11-28T18:26:45.622 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2023-11-28T18:26:45.622 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 23, 2023-11-28T18:26:45.622 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.004414124, 2023-11-28T18:26:45.622 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000191918 2023-11-28T18:26:45.622 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.622 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2023-11-28T18:26:45.622 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4, 2023-11-28T18:26:45.622 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000941, 2023-11-28T18:26:45.622 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000235 2023-11-28T18:26:45.622 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.622 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-28T18:26:45.622 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2023-11-28T18:26:45.623 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.623 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2023-11-28T18:26:45.623 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 59, 2023-11-28T18:26:45.623 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 49, 2023-11-28T18:26:45.623 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 137267, 2023-11-28T18:26:45.623 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 26679, 2023-11-28T18:26:45.623 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 6, 2023-11-28T18:26:45.623 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 5, 2023-11-28T18:26:45.623 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.013355555, 2023-11-28T18:26:45.623 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.002380848, 2023-11-28T18:26:45.623 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.011790942, 2023-11-28T18:26:45.623 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000784285, 2023-11-28T18:26:45.624 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2023-11-28T18:26:45.624 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 49, 2023-11-28T18:26:45.624 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.017101470, 2023-11-28T18:26:45.624 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000349009 2023-11-28T18:26:45.624 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.624 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2023-11-28T18:26:45.624 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 11, 2023-11-28T18:26:45.624 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000002141, 2023-11-28T18:26:45.624 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000194 2023-11-28T18:26:45.625 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.625 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 114608, 2023-11-28T18:26:45.625 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 12736 2023-11-28T18:26:45.625 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.625 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2023-11-28T18:26:45.625 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 25, 2023-11-28T18:26:45.625 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 36, 2023-11-28T18:26:45.625 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 14725, 2023-11-28T18:26:45.625 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 147983, 2023-11-28T18:26:45.625 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 10, 2023-11-28T18:26:45.625 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 6, 2023-11-28T18:26:45.625 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.009103303, 2023-11-28T18:26:45.626 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.002386627, 2023-11-28T18:26:45.626 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.005983937, 2023-11-28T18:26:45.626 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000260024, 2023-11-28T18:26:45.626 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2023-11-28T18:26:45.626 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 36, 2023-11-28T18:26:45.626 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.006719995, 2023-11-28T18:26:45.626 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000186666 2023-11-28T18:26:45.626 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.626 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2023-11-28T18:26:45.626 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 28, 2023-11-28T18:26:45.626 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000005371, 2023-11-28T18:26:45.627 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000191 2023-11-28T18:26:45.627 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.627 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-28T18:26:45.627 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 94624 2023-11-28T18:26:45.627 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.627 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2023-11-28T18:26:45.627 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2023-11-28T18:26:45.627 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2023-11-28T18:26:45.627 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.627 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2023-11-28T18:26:45.627 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2023-11-28T18:26:45.628 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2023-11-28T18:26:45.628 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 43, 2023-11-28T18:26:45.628 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.005123209, 2023-11-28T18:26:45.628 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000119144 2023-11-28T18:26:45.628 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:45.628 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.628 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2023-11-28T18:26:45.628 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2023-11-28T18:26:45.628 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2023-11-28T18:26:45.628 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4, 2023-11-28T18:26:45.628 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000255081, 2023-11-28T18:26:45.628 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000063770 2023-11-28T18:26:45.629 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:45.629 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.629 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2023-11-28T18:26:45.629 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 0, 2023-11-28T18:26:45.629 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 0, 2023-11-28T18:26:45.629 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2023-11-28T18:26:45.629 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.629 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.629 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.629 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.629 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2023-11-28T18:26:45.629 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2023-11-28T18:26:45.630 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 11, 2023-11-28T18:26:45.630 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 10, 2023-11-28T18:26:45.630 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 0, 2023-11-28T18:26:45.630 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2023-11-28T18:26:45.630 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2023-11-28T18:26:45.630 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 75, 2023-11-28T18:26:45.630 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 7, 2023-11-28T18:26:45.630 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 68, 2023-11-28T18:26:45.630 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2023-11-28T18:26:45.630 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 75, 2023-11-28T18:26:45.630 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2023-11-28T18:26:45.630 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 0, 2023-11-28T18:26:45.631 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 0, 2023-11-28T18:26:45.631 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 5, 2023-11-28T18:26:45.631 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 0, 2023-11-28T18:26:45.631 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 0, 2023-11-28T18:26:45.631 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2023-11-28T18:26:45.631 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 0, 2023-11-28T18:26:45.631 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2023-11-28T18:26:45.631 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2023-11-28T18:26:45.631 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2023-11-28T18:26:45.631 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2023-11-28T18:26:45.631 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2023-11-28T18:26:45.631 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2023-11-28T18:26:45.632 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2023-11-28T18:26:45.632 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 0, 2023-11-28T18:26:45.632 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 0, 2023-11-28T18:26:45.632 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2023-11-28T18:26:45.632 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2023-11-28T18:26:45.632 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2023-11-28T18:26:45.632 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 100, 2023-11-28T18:26:45.632 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 500, 2023-11-28T18:26:45.632 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2023-11-28T18:26:45.632 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2023-11-28T18:26:45.632 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2023-11-28T18:26:45.632 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2023-11-28T18:26:45.633 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2023-11-28T18:26:45.633 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2023-11-28T18:26:45.633 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2023-11-28T18:26:45.633 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2023-11-28T18:26:45.633 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2023-11-28T18:26:45.633 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2023-11-28T18:26:45.633 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 0, 2023-11-28T18:26:45.633 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 0, 2023-11-28T18:26:45.633 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2023-11-28T18:26:45.633 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-28T18:26:45.633 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-28T18:26:45.633 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 0, 2023-11-28T18:26:45.634 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 0, 2023-11-28T18:26:45.634 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 1, 2023-11-28T18:26:45.634 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 0 2023-11-28T18:26:45.634 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.634 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2023-11-28T18:26:45.634 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2023-11-28T18:26:45.634 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2023-11-28T18:26:45.634 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2023-11-28T18:26:45.634 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 0, 2023-11-28T18:26:45.634 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 0, 2023-11-28T18:26:45.634 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2023-11-28T18:26:45.635 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2023-11-28T18:26:45.635 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2023-11-28T18:26:45.635 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 0, 2023-11-28T18:26:45.635 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 0, 2023-11-28T18:26:45.635 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2023-11-28T18:26:45.635 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2023-11-28T18:26:45.635 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2023-11-28T18:26:45.635 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2023-11-28T18:26:45.635 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2023-11-28T18:26:45.635 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2023-11-28T18:26:45.635 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2023-11-28T18:26:45.635 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2023-11-28T18:26:45.636 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2023-11-28T18:26:45.636 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2023-11-28T18:26:45.636 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2023-11-28T18:26:45.636 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2023-11-28T18:26:45.636 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2023-11-28T18:26:45.636 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.636 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2023-11-28T18:26:45.636 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2023-11-28T18:26:45.636 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 1, 2023-11-28T18:26:45.636 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2023-11-28T18:26:45.636 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2023-11-28T18:26:45.636 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 9, 2023-11-28T18:26:45.637 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2023-11-28T18:26:45.637 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2023-11-28T18:26:45.637 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 0, 2023-11-28T18:26:45.637 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2023-11-28T18:26:45.637 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2023-11-28T18:26:45.637 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 2, 2023-11-28T18:26:45.637 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2023-11-28T18:26:45.637 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2023-11-28T18:26:45.637 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2023-11-28T18:26:45.637 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2023-11-28T18:26:45.637 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 4243787, 2023-11-28T18:26:45.637 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 4243610, 2023-11-28T18:26:45.638 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2023-11-28T18:26:45.638 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2023-11-28T18:26:45.638 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.002334953, 2023-11-28T18:26:45.638 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.002334953 2023-11-28T18:26:45.638 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.638 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 8 2023-11-28T18:26:45.638 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.638 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2023-11-28T18:26:45.638 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 78, 2023-11-28T18:26:45.638 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 78, 2023-11-28T18:26:45.638 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 0, 2023-11-28T18:26:45.639 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 47, 2023-11-28T18:26:45.639 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 47, 2023-11-28T18:26:45.639 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 0, 2023-11-28T18:26:45.639 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 75, 2023-11-28T18:26:45.639 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 75, 2023-11-28T18:26:45.639 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 0, 2023-11-28T18:26:45.639 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 0, 2023-11-28T18:26:45.639 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 32, 2023-11-28T18:26:45.639 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 32, 2023-11-28T18:26:45.639 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 46260, 2023-11-28T18:26:45.639 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 215336 2023-11-28T18:26:45.639 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.640 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2023-11-28T18:26:45.640 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 0, 2023-11-28T18:26:45.640 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2023-11-28T18:26:45.640 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 0, 2023-11-28T18:26:45.640 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 0, 2023-11-28T18:26:45.640 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2023-11-28T18:26:45.640 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2023-11-28T18:26:45.640 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.640 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.640 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.640 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.640 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2023-11-28T18:26:45.641 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.641 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.641 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.641 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.641 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2023-11-28T18:26:45.641 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.641 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.641 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.641 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.641 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2023-11-28T18:26:45.641 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.641 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.642 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.642 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.642 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2023-11-28T18:26:45.642 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.642 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.642 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.642 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.642 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2023-11-28T18:26:45.642 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.642 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.642 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.642 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.643 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2023-11-28T18:26:45.643 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.643 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.643 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.643 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.643 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2023-11-28T18:26:45.643 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.643 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.643 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.643 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.643 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2023-11-28T18:26:45.643 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.644 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.644 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.644 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.644 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2023-11-28T18:26:45.644 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.644 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.644 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.644 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.644 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2023-11-28T18:26:45.644 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.644 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.644 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.645 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.645 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2023-11-28T18:26:45.645 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.645 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.645 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.645 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.645 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2023-11-28T18:26:45.645 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.646 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.646 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.646 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.646 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2023-11-28T18:26:45.646 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.646 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.646 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.646 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.646 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2023-11-28T18:26:45.646 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.646 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.646 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.647 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.647 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2023-11-28T18:26:45.647 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.647 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.647 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.647 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.647 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2023-11-28T18:26:45.647 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.647 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.647 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.647 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.648 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2023-11-28T18:26:45.648 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.648 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.648 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.648 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.648 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2023-11-28T18:26:45.648 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.648 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.648 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.648 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.648 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2023-11-28T18:26:45.648 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.649 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.649 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.649 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.649 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2023-11-28T18:26:45.649 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.649 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.649 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.649 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.649 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2023-11-28T18:26:45.649 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.649 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.650 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.650 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.650 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2023-11-28T18:26:45.650 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.650 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.650 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.650 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.650 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2023-11-28T18:26:45.650 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.650 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.650 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.650 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.651 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2023-11-28T18:26:45.651 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.651 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.651 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.651 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.651 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2023-11-28T18:26:45.651 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.651 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.651 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.651 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.651 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2023-11-28T18:26:45.652 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.652 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.652 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.652 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.652 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2023-11-28T18:26:45.652 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.652 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.652 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.652 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.652 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2023-11-28T18:26:45.652 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.652 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.653 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.653 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.653 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2023-11-28T18:26:45.653 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2023-11-28T18:26:45.653 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2023-11-28T18:26:45.653 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.653 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.653 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.653 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:45.653 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.653 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2023-11-28T18:26:45.653 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 1, 2023-11-28T18:26:45.654 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 1, 2023-11-28T18:26:45.654 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 0, 2023-11-28T18:26:45.654 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 1, 2023-11-28T18:26:45.654 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2023-11-28T18:26:45.654 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 0, 2023-11-28T18:26:45.654 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 0, 2023-11-28T18:26:45.654 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 1001, 2023-11-28T18:26:45.654 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2023-11-28T18:26:45.654 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.654 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2023-11-28T18:26:45.654 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2023-11-28T18:26:45.654 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2023-11-28T18:26:45.655 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2023-11-28T18:26:45.655 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2023-11-28T18:26:45.655 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2023-11-28T18:26:45.655 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2023-11-28T18:26:45.655 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-28T18:26:45.655 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2023-11-28T18:26:45.655 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-28T18:26:45.655 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2023-11-28T18:26:45.655 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2023-11-28T18:26:45.655 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2023-11-28T18:26:45.655 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-28T18:26:45.655 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2023-11-28T18:26:45.656 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2023-11-28T18:26:45.656 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2023-11-28T18:26:45.656 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2023-11-28T18:26:45.656 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2023-11-28T18:26:45.656 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-28T18:26:45.656 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2023-11-28T18:26:45.656 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-28T18:26:45.656 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2023-11-28T18:26:45.656 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2023-11-28T18:26:45.656 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2023-11-28T18:26:45.656 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2023-11-28T18:26:45.656 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2023-11-28T18:26:45.657 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-28T18:26:45.657 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2023-11-28T18:26:45.657 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2023-11-28T18:26:45.657 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2023-11-28T18:26:45.657 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2023-11-28T18:26:45.657 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2023-11-28T18:26:45.657 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2023-11-28T18:26:45.657 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2023-11-28T18:26:45.657 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2023-11-28T18:26:45.657 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2023-11-28T18:26:45.657 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 245079, 2023-11-28T18:26:45.657 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 71, 2023-11-28T18:26:45.658 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2023-11-28T18:26:45.658 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2023-11-28T18:26:45.658 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2023-11-28T18:26:45.658 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2023-11-28T18:26:45.658 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2023-11-28T18:26:45.658 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2023-11-28T18:26:45.658 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2023-11-28T18:26:45.658 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2023-11-28T18:26:45.658 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 5848, 2023-11-28T18:26:45.658 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 95, 2023-11-28T18:26:45.658 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2023-11-28T18:26:45.658 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2023-11-28T18:26:45.659 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2023-11-28T18:26:45.659 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2023-11-28T18:26:45.659 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 277808, 2023-11-28T18:26:45.659 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 950, 2023-11-28T18:26:45.659 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2023-11-28T18:26:45.659 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2023-11-28T18:26:45.659 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2023-11-28T18:26:45.659 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2023-11-28T18:26:45.659 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.659 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2023-11-28T18:26:45.659 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2023-11-28T18:26:45.659 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2023-11-28T18:26:45.660 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 42, 2023-11-28T18:26:45.660 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 1131, 2023-11-28T18:26:45.660 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 0, 2023-11-28T18:26:45.660 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 42, 2023-11-28T18:26:45.660 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2023-11-28T18:26:45.660 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 42, 2023-11-28T18:26:45.660 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.075998304, 2023-11-28T18:26:45.660 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001809483 2023-11-28T18:26:45.660 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.660 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2023-11-28T18:26:45.660 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2023-11-28T18:26:45.660 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 42, 2023-11-28T18:26:45.661 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 87 2023-11-28T18:26:45.661 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.661 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 42, 2023-11-28T18:26:45.661 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 33, 2023-11-28T18:26:45.661 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 9, 2023-11-28T18:26:45.661 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2023-11-28T18:26:45.661 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2023-11-28T18:26:45.661 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 3, 2023-11-28T18:26:45.661 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 0, 2023-11-28T18:26:45.661 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 6, 2023-11-28T18:26:45.661 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 1, 2023-11-28T18:26:45.662 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 1, 2023-11-28T18:26:45.662 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2023-11-28T18:26:45.662 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2023-11-28T18:26:45.662 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 1, 2023-11-28T18:26:45.662 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2023-11-28T18:26:45.662 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 5, 2023-11-28T18:26:45.662 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2023-11-28T18:26:45.662 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2023-11-28T18:26:45.662 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2023-11-28T18:26:45.662 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 22, 2023-11-28T18:26:45.662 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 0, 2023-11-28T18:26:45.662 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2023-11-28T18:26:45.662 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2023-11-28T18:26:45.663 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2023-11-28T18:26:45.663 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2023-11-28T18:26:45.663 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2023-11-28T18:26:45.663 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2023-11-28T18:26:45.663 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2023-11-28T18:26:45.663 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2023-11-28T18:26:45.663 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2023-11-28T18:26:45.663 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2023-11-28T18:26:45.663 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2023-11-28T18:26:45.663 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2023-11-28T18:26:45.663 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2023-11-28T18:26:45.663 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2023-11-28T18:26:45.664 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2023-11-28T18:26:45.664 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2023-11-28T18:26:45.664 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2023-11-28T18:26:45.664 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2023-11-28T18:26:45.664 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2023-11-28T18:26:45.664 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2023-11-28T18:26:45.664 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2023-11-28T18:26:45.664 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2023-11-28T18:26:45.664 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2023-11-28T18:26:45.664 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2023-11-28T18:26:45.664 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2023-11-28T18:26:45.664 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2023-11-28T18:26:45.664 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 0, 2023-11-28T18:26:45.665 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2023-11-28T18:26:45.665 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 0, 2023-11-28T18:26:45.665 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 8, 2023-11-28T18:26:45.665 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 8, 2023-11-28T18:26:45.665 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 0, 2023-11-28T18:26:45.665 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2023-11-28T18:26:45.665 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 1, 2023-11-28T18:26:45.665 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 46, 2023-11-28T18:26:45.665 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 1 2023-11-28T18:26:45.665 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.665 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2023-11-28T18:26:45.666 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 0, 2023-11-28T18:26:45.666 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2023-11-28T18:26:45.666 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2023-11-28T18:26:45.666 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2023-11-28T18:26:45.666 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.666 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2023-11-28T18:26:45.666 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2023-11-28T18:26:45.666 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.666 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2023-11-28T18:26:45.666 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2023-11-28T18:26:45.666 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 0, 2023-11-28T18:26:45.666 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2023-11-28T18:26:45.666 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 0, 2023-11-28T18:26:45.667 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 0, 2023-11-28T18:26:45.667 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2023-11-28T18:26:45.667 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.667 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-28T18:26:45.667 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:45.667 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2023-11-28T18:26:45.667 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:45.667 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 103, 2023-11-28T18:26:45.667 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 201387, 2023-11-28T18:26:45.667 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:45.667 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 103, 2023-11-28T18:26:45.668 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2023-11-28T18:26:45.668 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2023-11-28T18:26:45.668 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 103, 2023-11-28T18:26:45.668 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 201387, 2023-11-28T18:26:45.668 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:45.668 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.668 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.668 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.668 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:45.668 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.668 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2023-11-28T18:26:45.668 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:45.668 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2023-11-28T18:26:45.669 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:45.669 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2023-11-28T18:26:45.669 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2023-11-28T18:26:45.669 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:45.669 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2023-11-28T18:26:45.669 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 42, 2023-11-28T18:26:45.669 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 50250, 2023-11-28T18:26:45.669 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 42, 2023-11-28T18:26:45.669 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 50250, 2023-11-28T18:26:45.669 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:45.669 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.669 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.669 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.670 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:45.670 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.670 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2023-11-28T18:26:45.670 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:45.670 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2023-11-28T18:26:45.670 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:45.670 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2023-11-28T18:26:45.670 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2023-11-28T18:26:45.670 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:45.670 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2023-11-28T18:26:45.670 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 42, 2023-11-28T18:26:45.670 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 42, 2023-11-28T18:26:45.671 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 42, 2023-11-28T18:26:45.671 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 42, 2023-11-28T18:26:45.671 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:45.671 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.671 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.671 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.671 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:45.671 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.671 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2023-11-28T18:26:45.671 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:45.671 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2023-11-28T18:26:45.671 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:45.671 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2023-11-28T18:26:45.672 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2023-11-28T18:26:45.672 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:45.672 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2023-11-28T18:26:45.672 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2023-11-28T18:26:45.672 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2023-11-28T18:26:45.672 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2023-11-28T18:26:45.672 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2023-11-28T18:26:45.672 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:45.672 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.672 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.672 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.672 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:45.672 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.672 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x5585002f85a0": { 2023-11-28T18:26:45.673 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:45.673 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2023-11-28T18:26:45.673 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:45.673 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 1, 2023-11-28T18:26:45.673 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 116, 2023-11-28T18:26:45.673 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:45.673 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 1, 2023-11-28T18:26:45.673 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2023-11-28T18:26:45.673 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2023-11-28T18:26:45.673 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 1, 2023-11-28T18:26:45.673 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 116, 2023-11-28T18:26:45.673 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:45.673 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.674 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.674 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.674 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:45.674 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:45.674 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:26:45.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99d74c8640 1 -- 172.21.15.105:0/8087652 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99b40668d0 msgr2=0x7f99b4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99d74c8640 1 --2- 172.21.15.105:0/8087652 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99b40668d0 0x7f99b4068d90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f99c00ae000 tx=0x7f99c0009040 comp rx=0 tx=0).stop 2023-11-28T18:26:45.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99d74c8640 1 -- 172.21.15.105:0/8087652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99d015b420 msgr2=0x7f99d013b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99d74c8640 1 --2- 172.21.15.105:0/8087652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99d015b420 0x7f99d013b150 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00c7f0 tx=0x7f99cc00ccb0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99d74c8640 1 -- 172.21.15.105:0/8087652 shutdown_connections 2023-11-28T18:26:45.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99d74c8640 1 --2- 172.21.15.105:0/8087652 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99d0155b20 0x7f99d0155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99d74c8640 1 --2- 172.21.15.105:0/8087652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99d015e9f0 0x7f99d013a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99d74c8640 1 --2- 172.21.15.105:0/8087652 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99b40668d0 0x7f99b4068d90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99d74c8640 1 --2- 172.21.15.105:0/8087652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99d015b420 0x7f99d013b150 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99d74c8640 1 -- 172.21.15.105:0/8087652 >> 172.21.15.105:0/8087652 conn(0x7f99d00ce9e0 msgr2=0x7f99d015de40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:45.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99d74c8640 1 -- 172.21.15.105:0/8087652 shutdown_connections 2023-11-28T18:26:45.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.612+0000 7f99d74c8640 1 -- 172.21.15.105:0/8087652 wait complete. 2023-11-28T18:26:45.677 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:2 perf dump 2023-11-28T18:26:45.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.812+0000 7f82aed1d640 1 Processor -- start 2023-11-28T18:26:45.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.812+0000 7f82aed1d640 1 -- start start 2023-11-28T18:26:45.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.812+0000 7f82aed1d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f82a8159f40 0x7f82a80caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.816+0000 7f82aed1d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a806bf50 con 0x7f82a80ad850 2023-11-28T18:26:45.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.816+0000 7f82aed1d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a80ccbc0 con 0x7f82a80cc760 2023-11-28T18:26:45.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.816+0000 7f82aed1d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a815a320 con 0x7f82a8159f40 2023-11-28T18:26:45.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.816+0000 7f82ad2ba640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f82a80ad850 0x7f82a80701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46020/0 (socket says 172.21.15.105:46020) 2023-11-28T18:26:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.816+0000 7f82ad2ba640 1 -- 172.21.15.105:0/1112589865 learned_addr learned my addr 172.21.15.105:0/1112589865 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.816+0000 7f82acab9640 1 --2- 172.21.15.105:0/1112589865 >> v2:172.21.15.148:3300/0 conn(0x7f82a8159f40 0x7f82a80caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.816+0000 7f829f7fe640 1 -- 172.21.15.105:0/1112589865 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1703709898 0 0) 0x7f82a806bf50 con 0x7f82a80ad850 2023-11-28T18:26:45.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.816+0000 7f829f7fe640 1 -- 172.21.15.105:0/1112589865 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8290003530 con 0x7f82a80ad850 2023-11-28T18:26:45.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.816+0000 7f82acab9640 1 -- 172.21.15.105:0/1112589865 >> v1:172.21.15.148:6789/0 conn(0x7f82a80cc760 legacy=0x7f82a80ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.816+0000 7f82acab9640 1 -- 172.21.15.105:0/1112589865 >> v1:172.21.15.105:6789/0 conn(0x7f82a80ad850 legacy=0x7f82a80701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.816+0000 7f82acab9640 1 -- 172.21.15.105:0/1112589865 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a8071380 con 0x7f82a8159f40 2023-11-28T18:26:45.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.816+0000 7f82acab9640 1 --2- 172.21.15.105:0/1112589865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82a8159f40 0x7f82a80caf60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f82a40097c0 tx=0x7f82a401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc3c00b11db5d615 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.820+0000 7f829f7fe640 1 -- 172.21.15.105:0/1112589865 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a40ac620 con 0x7f82a8159f40 2023-11-28T18:26:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.820+0000 7f829f7fe640 1 -- 172.21.15.105:0/1112589865 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a40ac780 con 0x7f82a8159f40 2023-11-28T18:26:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.820+0000 7f829f7fe640 1 -- 172.21.15.105:0/1112589865 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a40acaf0 con 0x7f82a8159f40 2023-11-28T18:26:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.820+0000 7f82aed1d640 1 -- 172.21.15.105:0/1112589865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82a8159f40 msgr2=0x7f82a80caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.820+0000 7f82aed1d640 1 --2- 172.21.15.105:0/1112589865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82a8159f40 0x7f82a80caf60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f82a40097c0 tx=0x7f82a401c3c0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.820+0000 7f82aed1d640 1 -- 172.21.15.105:0/1112589865 shutdown_connections 2023-11-28T18:26:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.820+0000 7f82aed1d640 1 --2- 172.21.15.105:0/1112589865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82a8159f40 0x7f82a80caf60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.820+0000 7f82aed1d640 1 -- 172.21.15.105:0/1112589865 >> 172.21.15.105:0/1112589865 conn(0x7f82a80ceb90 msgr2=0x7f82a80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.820+0000 7f82aed1d640 1 -- 172.21.15.105:0/1112589865 shutdown_connections 2023-11-28T18:26:45.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.820+0000 7f82aed1d640 1 -- 172.21.15.105:0/1112589865 wait complete. 2023-11-28T18:26:45.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.820+0000 7f82aed1d640 1 Processor -- start 2023-11-28T18:26:45.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.820+0000 7f82aed1d640 1 -- start start 2023-11-28T18:26:45.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.820+0000 7f82aed1d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82a80ad850 0x7f82a809c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.820+0000 7f82aed1d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82a80cc760 0x7f82a80af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.820+0000 7f82aed1d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82a8159f40 0x7f82a80af610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.820+0000 7f82aed1d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a814ba60 con 0x7f82a80ad850 2023-11-28T18:26:45.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.820+0000 7f82aed1d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a80b1d20 con 0x7f82a8159f40 2023-11-28T18:26:45.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.820+0000 7f82aed1d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a80b2000 con 0x7f82a80cc760 2023-11-28T18:26:45.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.820+0000 7f82acab9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82a80ad850 0x7f82a809c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.820+0000 7f82acab9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82a80ad850 0x7f82a809c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42194/0 (socket says 172.21.15.105:42194) 2023-11-28T18:26:45.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.820+0000 7f82acab9640 1 -- 172.21.15.105:0/2381479265 learned_addr learned my addr 172.21.15.105:0/2381479265 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:45.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.820+0000 7f829ffff640 1 --2- 172.21.15.105:0/2381479265 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82a80cc760 0x7f82a80af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.824+0000 7f82ad2ba640 1 --2- 172.21.15.105:0/2381479265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82a8159f40 0x7f82a80af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.824+0000 7f82acab9640 1 -- 172.21.15.105:0/2381479265 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82a80cc760 msgr2=0x7f82a80af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.824+0000 7f82acab9640 1 --2- 172.21.15.105:0/2381479265 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82a80cc760 0x7f82a80af0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.824+0000 7f82acab9640 1 -- 172.21.15.105:0/2381479265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82a8159f40 msgr2=0x7f82a80af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.824+0000 7f82acab9640 1 --2- 172.21.15.105:0/2381479265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82a8159f40 0x7f82a80af610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.824+0000 7f82acab9640 1 -- 172.21.15.105:0/2381479265 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a4009420 con 0x7f82a80ad850 2023-11-28T18:26:45.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.824+0000 7f82acab9640 1 --2- 172.21.15.105:0/2381479265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82a80ad850 0x7f82a809c330 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f82a4019ff0 tx=0x7f82a4019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:45.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.824+0000 7f829d7fa640 1 -- 172.21.15.105:0/2381479265 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a40ac620 con 0x7f82a80ad850 2023-11-28T18:26:45.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.824+0000 7f829d7fa640 1 -- 172.21.15.105:0/2381479265 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a4004030 con 0x7f82a80ad850 2023-11-28T18:26:45.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.824+0000 7f829d7fa640 1 -- 172.21.15.105:0/2381479265 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a40043a0 con 0x7f82a80ad850 2023-11-28T18:26:45.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.824+0000 7f82aed1d640 1 -- 172.21.15.105:0/2381479265 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a80b22e0 con 0x7f82a80ad850 2023-11-28T18:26:45.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.824+0000 7f82aed1d640 1 -- 172.21.15.105:0/2381479265 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a80b28a0 con 0x7f82a80ad850 2023-11-28T18:26:45.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.824+0000 7f829d7fa640 1 -- 172.21.15.105:0/2381479265 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f82a40b3040 con 0x7f82a80ad850 2023-11-28T18:26:45.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.824+0000 7f829d7fa640 1 --2- 172.21.15.105:0/2381479265 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82940668d0 0x7f8294068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.824+0000 7f829d7fa640 1 -- 172.21.15.105:0/2381479265 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f82a411e500 con 0x7f82a80ad850 2023-11-28T18:26:45.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.824+0000 7f829ffff640 1 --2- 172.21.15.105:0/2381479265 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82940668d0 0x7f8294068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82b2080000 1 Processor -- start 2023-11-28T18:26:45.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f829ffff640 1 --2- 172.21.15.105:0/2381479265 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82940668d0 0x7f8294068d90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f82a80b4c60 tx=0x7f8298009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:45.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82b2080000 1 -- start start 2023-11-28T18:26:45.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82b2080000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c336503dc0 0x55c3365041a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82b2080000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3365046e0 0x55c3364b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82b2080000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c3364b9930 0x55c3364b9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82b2080000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c33630c9e0 con 0x55c336503dc0 2023-11-28T18:26:45.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82b2080000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c336309f10 con 0x55c3365046e0 2023-11-28T18:26:45.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82b2080000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c336305800 con 0x55c3364b9930 2023-11-28T18:26:45.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82acab9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c336503dc0 0x55c3365041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82acab9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c336503dc0 0x55c3365041a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42210/0 (socket says 172.21.15.105:42210) 2023-11-28T18:26:45.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82acab9640 1 -- 172.21.15.105:0/489081796 learned_addr learned my addr 172.21.15.105:0/489081796 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:45.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f829ffff640 1 --2- 172.21.15.105:0/489081796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3365046e0 0x55c3364b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82ad2ba640 1 --2- 172.21.15.105:0/489081796 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c3364b9930 0x55c3364b9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82acab9640 1 -- 172.21.15.105:0/489081796 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c3364b9930 msgr2=0x55c3364b9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:45.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82acab9640 1 --2- 172.21.15.105:0/489081796 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c3364b9930 0x55c3364b9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82acab9640 1 -- 172.21.15.105:0/489081796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3365046e0 msgr2=0x55c3364b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:45.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82acab9640 1 --2- 172.21.15.105:0/489081796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3365046e0 0x55c3364b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82acab9640 1 -- 172.21.15.105:0/489081796 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c335ffddc0 con 0x55c336503dc0 2023-11-28T18:26:45.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f829ffff640 1 --2- 172.21.15.105:0/489081796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3365046e0 0x55c3364b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:45.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82acab9640 1 --2- 172.21.15.105:0/489081796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c336503dc0 0x55c3365041a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f82a411c760 tx=0x7f82a40eeaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795562bd443747fb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:45.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f828e7fc640 1 -- 172.21.15.105:0/489081796 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a40f30d0 con 0x55c336503dc0 2023-11-28T18:26:45.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f828e7fc640 1 -- 172.21.15.105:0/489081796 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a40f32e0 con 0x55c336503dc0 2023-11-28T18:26:45.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f828e7fc640 1 -- 172.21.15.105:0/489081796 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a40f7490 con 0x55c336503dc0 2023-11-28T18:26:45.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82b2080000 1 -- 172.21.15.105:0/489081796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c336503dc0 msgr2=0x55c3365041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82b2080000 1 --2- 172.21.15.105:0/489081796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c336503dc0 0x55c3365041a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f82a411c760 tx=0x7f82a40eeaf0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82b2080000 1 -- 172.21.15.105:0/489081796 shutdown_connections 2023-11-28T18:26:45.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82b2080000 1 --2- 172.21.15.105:0/489081796 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c3364b9930 0x55c3364b9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82b2080000 1 --2- 172.21.15.105:0/489081796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3365046e0 0x55c3364b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82b2080000 1 --2- 172.21.15.105:0/489081796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c336503dc0 0x55c3365041a0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82b2080000 1 -- 172.21.15.105:0/489081796 >> 172.21.15.105:0/489081796 conn(0x55c3364bdfb0 msgr2=0x55c3364e2360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:45.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82b2080000 1 -- 172.21.15.105:0/489081796 shutdown_connections 2023-11-28T18:26:45.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82b2080000 1 -- 172.21.15.105:0/489081796 wait complete. 2023-11-28T18:26:45.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.828+0000 7f82b2080000 1 -- start start 2023-11-28T18:26:45.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82b2080000 1 Processor -- start 2023-11-28T18:26:45.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82b2080000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3364b9930 0x55c3364e6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82b2080000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c336503dc0 0x55c3364e6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82b2080000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3365046e0 0x55c3364e74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82b2080000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3362c3a70 con 0x55c3365046e0 2023-11-28T18:26:45.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82b2080000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3362c3710 con 0x55c3364b9930 2023-11-28T18:26:45.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82b2080000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c335fbea40 con 0x55c336503dc0 2023-11-28T18:26:45.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82b2080000 10 client.0 _ll_register_callbacks cb 0x55c3364a0ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:45.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82ad2ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3365046e0 0x55c3364e74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82ad2ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3365046e0 0x55c3364e74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42222/0 (socket says 172.21.15.105:42222) 2023-11-28T18:26:45.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82ad2ba640 1 -- 172.21.15.105:0/272512876 learned_addr learned my addr 172.21.15.105:0/272512876 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:45.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82acab9640 1 --2- 172.21.15.105:0/272512876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3364b9930 0x55c3364e6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f829ffff640 1 --2- 172.21.15.105:0/272512876 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c336503dc0 0x55c3364e6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82ad2ba640 1 -- 172.21.15.105:0/272512876 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c336503dc0 msgr2=0x55c3364e6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:45.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82ad2ba640 1 --2- 172.21.15.105:0/272512876 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c336503dc0 0x55c3364e6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82ad2ba640 1 -- 172.21.15.105:0/272512876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3364b9930 msgr2=0x55c3364e6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:45.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82ad2ba640 1 --2- 172.21.15.105:0/272512876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3364b9930 0x55c3364e6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82ad2ba640 1 -- 172.21.15.105:0/272512876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a80b28a0 con 0x55c3365046e0 2023-11-28T18:26:45.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82ad2ba640 1 --2- 172.21.15.105:0/272512876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3365046e0 0x55c3364e74d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f82a0007390 tx=0x7f82a0007850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69dca3168167ee8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:45.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82877fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:45.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82877fe640 1 -- 172.21.15.105:0/272512876 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a00107c0 con 0x55c3365046e0 2023-11-28T18:26:45.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82877fe640 1 -- 172.21.15.105:0/272512876 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f82a0010920 con 0x55c3365046e0 2023-11-28T18:26:45.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82b2080000 1 -- 172.21.15.105:0/272512876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3362beab0 con 0x55c3365046e0 2023-11-28T18:26:45.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82877fe640 1 -- 172.21.15.105:0/272512876 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82a0010ae0 con 0x55c3365046e0 2023-11-28T18:26:45.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82b2080000 10 client.6462 fetch_fsmap learned FSMap version 47 2023-11-28T18:26:45.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82b2080000 1 -- 172.21.15.105:0/272512876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55c3362c6260 con 0x55c3365046e0 2023-11-28T18:26:45.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82877fe640 1 -- 172.21.15.105:0/272512876 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c3362beab0 con 0x55c3365046e0 2023-11-28T18:26:45.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82877fe640 1 -- 172.21.15.105:0/272512876 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 47) v1 ==== 3326+0+0 (secure 0 0 0) 0x7f82a0010c40 con 0x55c3365046e0 2023-11-28T18:26:45.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82b2080000 10 client.6462 fetch_fsmap finished waiting for FSMap version 47 2023-11-28T18:26:45.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82b2080000 10 client.6462 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.c 2023-11-28T18:26:45.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82b2080000 20 client.6462 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:45.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82b2080000 1 --2- 172.21.15.105:0/272512876 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55c3364e7ee0 0x55c336514e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82b2080000 4 client.6462 mds_command: new command op to 5408 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:26:45.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82b2080000 1 -- 172.21.15.105:0/272512876 --> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3362ba850 con 0x55c3364e7ee0 2023-11-28T18:26:45.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82acab9640 1 --2- 172.21.15.105:0/272512876 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55c3364e7ee0 0x55c336514e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82acab9640 1 --2- 172.21.15.105:0/272512876 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55c3364e7ee0 0x55c336514e60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6bddd2fab116e2cc server_cookie=29048c662f2974db in_seq=0 out_seq=0 2023-11-28T18:26:45.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.832+0000 7f82877fe640 10 client.6462 ms_handle_connect on v2:172.21.15.105:6835/3765684108 2023-11-28T18:26:45.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.836+0000 7f82877fe640 1 -- 172.21.15.105:0/272512876 <== mds.2 v2:172.21.15.105:6835/3765684108 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c3362ba850 con 0x55c3364e7ee0 2023-11-28T18:26:45.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.836+0000 7f82877fe640 10 client.6462 handle_command_reply: tid=0 2023-11-28T18:26:45.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.836+0000 7f82b2080000 1 client.6462 shutdown 2023-11-28T18:26:45.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.836+0000 7f82b2080000 1 -- 172.21.15.105:0/272512876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3365046e0 msgr2=0x55c3364e74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.836+0000 7f82b2080000 1 --2- 172.21.15.105:0/272512876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3365046e0 0x55c3364e74d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f82a0007390 tx=0x7f82a0007850 comp rx=0 tx=0).stop 2023-11-28T18:26:45.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.836+0000 7f82b2080000 1 -- 172.21.15.105:0/272512876 shutdown_connections 2023-11-28T18:26:45.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.836+0000 7f82b2080000 1 --2- 172.21.15.105:0/272512876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3365046e0 0x55c3364e74d0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.836+0000 7f82b2080000 1 --2- 172.21.15.105:0/272512876 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c336503dc0 0x55c3364e6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.836+0000 7f82b2080000 1 --2- 172.21.15.105:0/272512876 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55c3364e7ee0 0x55c336514e60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.836+0000 7f82b2080000 1 --2- 172.21.15.105:0/272512876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3364b9930 0x55c3364e6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.836+0000 7f82877fe640 0 client.6462 ms_handle_reset on v2:172.21.15.105:6835/3765684108 2023-11-28T18:26:45.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.836+0000 7f82b2080000 1 -- 172.21.15.105:0/272512876 >> 172.21.15.105:0/272512876 conn(0x55c3364bdfb0 msgr2=0x55c3364b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:45.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.836+0000 7f82b2080000 1 -- 172.21.15.105:0/272512876 shutdown_connections 2023-11-28T18:26:45.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.836+0000 7f82b2080000 1 -- 172.21.15.105:0/272512876 wait complete. 2023-11-28T18:26:45.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.836+0000 7f82b2080000 20 client.6462 trim_cache size 0 max 16384 2023-11-28T18:26:45.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.848+0000 7f82b2080000 1 Processor -- start 2023-11-28T18:26:45.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.848+0000 7f82b2080000 1 -- start start 2023-11-28T18:26:45.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.848+0000 7f82b2080000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3364b9930 0x55c3364b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.848+0000 7f82b2080000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c336503dc0 0x55c3364b6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.848+0000 7f82b2080000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3365046e0 0x55c3364b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.848+0000 7f82b2080000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3362bf5d0 con 0x55c3364b9930 2023-11-28T18:26:45.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.848+0000 7f82b2080000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3362db7b0 con 0x55c3365046e0 2023-11-28T18:26:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.848+0000 7f82b2080000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c336301170 con 0x55c336503dc0 2023-11-28T18:26:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.848+0000 7f82acab9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3364b9930 0x55c3364b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.848+0000 7f82acab9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3364b9930 0x55c3364b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42224/0 (socket says 172.21.15.105:42224) 2023-11-28T18:26:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.848+0000 7f82acab9640 1 -- 172.21.15.105:0/4066905884 learned_addr learned my addr 172.21.15.105:0/4066905884 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.848+0000 7f82acab9640 1 -- 172.21.15.105:0/4066905884 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c336503dc0 msgr2=0x55c3364b6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.848+0000 7f82acab9640 1 --2- 172.21.15.105:0/4066905884 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c336503dc0 0x55c3364b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.848+0000 7f82acab9640 1 -- 172.21.15.105:0/4066905884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3365046e0 msgr2=0x55c3364b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.848+0000 7f82acab9640 1 --2- 172.21.15.105:0/4066905884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3365046e0 0x55c3364b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.848+0000 7f82acab9640 1 -- 172.21.15.105:0/4066905884 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3362ac760 con 0x55c3364b9930 2023-11-28T18:26:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82acab9640 1 --2- 172.21.15.105:0/4066905884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3364b9930 0x55c3364b5ff0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f82a401c8a0 tx=0x7f82a410d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e084948cbe5a8e8c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f828effd640 1 -- 172.21.15.105:0/4066905884 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a410a070 con 0x55c3364b9930 2023-11-28T18:26:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f828effd640 1 -- 172.21.15.105:0/4066905884 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a40eb4d0 con 0x55c3364b9930 2023-11-28T18:26:45.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f828effd640 1 -- 172.21.15.105:0/4066905884 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a40f30d0 con 0x55c3364b9930 2023-11-28T18:26:45.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82b2080000 1 -- 172.21.15.105:0/4066905884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3364b9930 msgr2=0x55c3364b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82b2080000 1 --2- 172.21.15.105:0/4066905884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3364b9930 0x55c3364b5ff0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f82a401c8a0 tx=0x7f82a410d040 comp rx=0 tx=0).stop 2023-11-28T18:26:45.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82b2080000 1 -- 172.21.15.105:0/4066905884 shutdown_connections 2023-11-28T18:26:45.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82b2080000 1 --2- 172.21.15.105:0/4066905884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3365046e0 0x55c3364b6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82b2080000 1 --2- 172.21.15.105:0/4066905884 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c336503dc0 0x55c3364b6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82b2080000 1 --2- 172.21.15.105:0/4066905884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3364b9930 0x55c3364b5ff0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82b2080000 1 -- 172.21.15.105:0/4066905884 >> 172.21.15.105:0/4066905884 conn(0x55c3364bdfb0 msgr2=0x55c336502c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:45.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82b2080000 1 -- 172.21.15.105:0/4066905884 shutdown_connections 2023-11-28T18:26:45.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82b2080000 1 -- 172.21.15.105:0/4066905884 wait complete. 2023-11-28T18:26:45.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82b2080000 1 -- start start 2023-11-28T18:26:45.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82b2080000 1 Processor -- start 2023-11-28T18:26:45.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82b2080000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3364b9930 0x55c336513ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82b2080000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c336503dc0 0x55c3365143e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82b2080000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3365046e0 0x55c336514920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82b2080000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3362ffb00 con 0x55c3365046e0 2023-11-28T18:26:45.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82b2080000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c336309070 con 0x55c3364b9930 2023-11-28T18:26:45.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82b2080000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c3364e4700 con 0x55c336503dc0 2023-11-28T18:26:45.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82b2080000 10 client.0 _ll_register_callbacks cb 0x55c3362cd840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:45.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82ad2ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3365046e0 0x55c336514920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82ad2ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3365046e0 0x55c336514920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42240/0 (socket says 172.21.15.105:42240) 2023-11-28T18:26:45.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82ad2ba640 1 -- 172.21.15.105:0/3280193597 learned_addr learned my addr 172.21.15.105:0/3280193597 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:45.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f829ffff640 1 --2- 172.21.15.105:0/3280193597 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c336503dc0 0x55c3365143e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82acab9640 1 --2- 172.21.15.105:0/3280193597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3364b9930 0x55c336513ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82ad2ba640 1 -- 172.21.15.105:0/3280193597 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c336503dc0 msgr2=0x55c3365143e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:45.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82ad2ba640 1 --2- 172.21.15.105:0/3280193597 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c336503dc0 0x55c3365143e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82ad2ba640 1 -- 172.21.15.105:0/3280193597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3364b9930 msgr2=0x55c336513ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:45.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82ad2ba640 1 --2- 172.21.15.105:0/3280193597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3364b9930 0x55c336513ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82ad2ba640 1 -- 172.21.15.105:0/3280193597 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c335ffddc0 con 0x55c3365046e0 2023-11-28T18:26:45.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82ad2ba640 1 --2- 172.21.15.105:0/3280193597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3365046e0 0x55c336514920 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f82a0007890 tx=0x7f82a0001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c533f930198cf2a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:45.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.852+0000 7f82877fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:45.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82877fe640 1 -- 172.21.15.105:0/3280193597 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a0001f20 con 0x55c3365046e0 2023-11-28T18:26:45.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82877fe640 1 -- 172.21.15.105:0/3280193597 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f82a00105b0 con 0x55c3365046e0 2023-11-28T18:26:45.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82b2080000 1 -- 172.21.15.105:0/3280193597 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3362c83a0 con 0x55c3365046e0 2023-11-28T18:26:45.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82877fe640 1 -- 172.21.15.105:0/3280193597 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82a0005be0 con 0x55c3365046e0 2023-11-28T18:26:45.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82b2080000 10 client.6468 fetch_fsmap learned FSMap version 47 2023-11-28T18:26:45.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82b2080000 1 -- 172.21.15.105:0/3280193597 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55c3362a0040 con 0x55c3365046e0 2023-11-28T18:26:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82877fe640 1 -- 172.21.15.105:0/3280193597 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c3362c83a0 con 0x55c3365046e0 2023-11-28T18:26:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82877fe640 1 -- 172.21.15.105:0/3280193597 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 47) v1 ==== 3326+0+0 (secure 0 0 0) 0x7f82a00146b0 con 0x55c3365046e0 2023-11-28T18:26:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82b2080000 10 client.6468 fetch_fsmap finished waiting for FSMap version 47 2023-11-28T18:26:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82b2080000 10 client.6468 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.c 2023-11-28T18:26:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82b2080000 20 client.6468 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82b2080000 1 --2- 172.21.15.105:0/3280193597 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55c336514e60 0x55c336515270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82b2080000 4 client.6468 mds_command: new command op to 5408 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2023-11-28T18:26:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82b2080000 1 -- 172.21.15.105:0/3280193597 --> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55c3362fe7e0 con 0x55c336514e60 2023-11-28T18:26:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82acab9640 1 --2- 172.21.15.105:0/3280193597 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55c336514e60 0x55c336515270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82acab9640 1 --2- 172.21.15.105:0/3280193597 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55c336514e60 0x55c336515270 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4237a3e99cf72884 server_cookie=a63e466b6c0c154a in_seq=0 out_seq=0 2023-11-28T18:26:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82877fe640 10 client.6468 ms_handle_connect on v2:172.21.15.105:6835/3765684108 2023-11-28T18:26:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82877fe640 1 -- 172.21.15.105:0/3280193597 <== mds.2 v2:172.21.15.105:6835/3765684108 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17631 (crc 0 0 0) 0x55c3362fe7e0 con 0x55c336514e60 2023-11-28T18:26:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82877fe640 10 client.6468 handle_command_reply: tid=0 2023-11-28T18:26:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82b2080000 1 client.6468 shutdown 2023-11-28T18:26:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82b2080000 1 -- 172.21.15.105:0/3280193597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3365046e0 msgr2=0x55c336514920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82b2080000 1 --2- 172.21.15.105:0/3280193597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3365046e0 0x55c336514920 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f82a0007890 tx=0x7f82a0001260 comp rx=0 tx=0).stop 2023-11-28T18:26:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82b2080000 1 -- 172.21.15.105:0/3280193597 shutdown_connections 2023-11-28T18:26:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82b2080000 1 --2- 172.21.15.105:0/3280193597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3365046e0 0x55c336514920 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82b2080000 1 --2- 172.21.15.105:0/3280193597 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c336503dc0 0x55c3365143e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82b2080000 1 --2- 172.21.15.105:0/3280193597 >> [v2:172.21.15.105:6835/3765684108,v1:172.21.15.105:6837/3765684108] conn(0x55c336514e60 0x55c336515270 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82b2080000 1 --2- 172.21.15.105:0/3280193597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3364b9930 0x55c336513ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82877fe640 0 client.6468 ms_handle_reset on v2:172.21.15.105:6835/3765684108 2023-11-28T18:26:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82b2080000 1 -- 172.21.15.105:0/3280193597 >> 172.21.15.105:0/3280193597 conn(0x55c3364bdfb0 msgr2=0x55c336503320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82b2080000 1 -- 172.21.15.105:0/3280193597 shutdown_connections 2023-11-28T18:26:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82b2080000 1 -- 172.21.15.105:0/3280193597 wait complete. 2023-11-28T18:26:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82b2080000 20 client.6468 trim_cache size 0 max 16384 2023-11-28T18:26:45.864 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:26:45.865 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2023-11-28T18:26:45.865 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 25, 2023-11-28T18:26:45.865 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 78, 2023-11-28T18:26:45.865 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 7235, 2023-11-28T18:26:45.865 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 38693, 2023-11-28T18:26:45.865 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 9, 2023-11-28T18:26:45.865 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 2, 2023-11-28T18:26:45.865 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.014570648, 2023-11-28T18:26:45.865 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.001922286, 2023-11-28T18:26:45.865 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.008436750, 2023-11-28T18:26:45.866 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000316139, 2023-11-28T18:26:45.866 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2023-11-28T18:26:45.866 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 77, 2023-11-28T18:26:45.866 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.044870514, 2023-11-28T18:26:45.866 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000582733 2023-11-28T18:26:45.866 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.866 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2023-11-28T18:26:45.866 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 13, 2023-11-28T18:26:45.866 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000003862, 2023-11-28T18:26:45.866 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000297 2023-11-28T18:26:45.866 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.867 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 1568, 2023-11-28T18:26:45.867 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 448 2023-11-28T18:26:45.867 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.867 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2023-11-28T18:26:45.867 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 88, 2023-11-28T18:26:45.867 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 76, 2023-11-28T18:26:45.867 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 141572, 2023-11-28T18:26:45.867 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 127145, 2023-11-28T18:26:45.867 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 16, 2023-11-28T18:26:45.867 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 8, 2023-11-28T18:26:45.867 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.015590863, 2023-11-28T18:26:45.868 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.002706370, 2023-11-28T18:26:45.868 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.014287909, 2023-11-28T18:26:45.868 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000634260, 2023-11-28T18:26:45.868 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2023-11-28T18:26:45.868 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 73, 2023-11-28T18:26:45.868 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.041358224, 2023-11-28T18:26:45.868 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000566551 2023-11-28T18:26:45.868 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.868 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2023-11-28T18:26:45.868 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 22, 2023-11-28T18:26:45.868 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000002885, 2023-11-28T18:26:45.868 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000131 2023-11-28T18:26:45.869 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.869 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 126144, 2023-11-28T18:26:45.869 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 13776 2023-11-28T18:26:45.869 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.869 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2023-11-28T18:26:45.869 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 48, 2023-11-28T18:26:45.869 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 109, 2023-11-28T18:26:45.869 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 84443, 2023-11-28T18:26:45.869 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 176983, 2023-11-28T18:26:45.869 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 6, 2023-11-28T18:26:45.869 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 3, 2023-11-28T18:26:45.869 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.010898176, 2023-11-28T18:26:45.870 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.003978595, 2023-11-28T18:26:45.870 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.006691117, 2023-11-28T18:26:45.870 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000404989, 2023-11-28T18:26:45.870 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2023-11-28T18:26:45.870 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 110, 2023-11-28T18:26:45.870 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.018769477, 2023-11-28T18:26:45.870 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000170631 2023-11-28T18:26:45.870 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.870 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2023-11-28T18:26:45.870 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 104, 2023-11-28T18:26:45.870 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000050159, 2023-11-28T18:26:45.870 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000482 2023-11-28T18:26:45.871 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.871 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-28T18:26:45.871 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 100960 2023-11-28T18:26:45.871 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.871 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2023-11-28T18:26:45.871 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2023-11-28T18:26:45.871 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2023-11-28T18:26:45.871 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.871 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2023-11-28T18:26:45.871 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2023-11-28T18:26:45.871 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2023-11-28T18:26:45.871 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 37, 2023-11-28T18:26:45.871 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.002981218, 2023-11-28T18:26:45.872 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000080573 2023-11-28T18:26:45.872 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:45.872 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.872 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2023-11-28T18:26:45.872 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2023-11-28T18:26:45.872 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2023-11-28T18:26:45.872 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2, 2023-11-28T18:26:45.872 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000006684, 2023-11-28T18:26:45.872 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000003342 2023-11-28T18:26:45.872 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:45.872 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.872 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2023-11-28T18:26:45.873 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 0, 2023-11-28T18:26:45.873 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 0, 2023-11-28T18:26:45.873 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2023-11-28T18:26:45.873 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.873 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.873 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.873 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.873 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2023-11-28T18:26:45.873 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2023-11-28T18:26:45.873 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2023-11-28T18:26:45.873 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2023-11-28T18:26:45.873 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 11, 2023-11-28T18:26:45.874 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2023-11-28T18:26:45.874 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2023-11-28T18:26:45.874 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 63, 2023-11-28T18:26:45.874 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 0, 2023-11-28T18:26:45.874 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 63, 2023-11-28T18:26:45.874 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2023-11-28T18:26:45.874 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 63, 2023-11-28T18:26:45.874 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2023-11-28T18:26:45.874 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 52, 2023-11-28T18:26:45.874 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 52, 2023-11-28T18:26:45.874 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 4, 2023-11-28T18:26:45.874 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 2, 2023-11-28T18:26:45.875 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 0, 2023-11-28T18:26:45.875 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2023-11-28T18:26:45.875 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 2, 2023-11-28T18:26:45.875 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2023-11-28T18:26:45.875 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2023-11-28T18:26:45.875 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2023-11-28T18:26:45.875 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 1222, 2023-11-28T18:26:45.875 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2023-11-28T18:26:45.875 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2023-11-28T18:26:45.875 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2023-11-28T18:26:45.875 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 2, 2023-11-28T18:26:45.875 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 52, 2023-11-28T18:26:45.876 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2023-11-28T18:26:45.876 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2023-11-28T18:26:45.876 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2023-11-28T18:26:45.876 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 100, 2023-11-28T18:26:45.876 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 500, 2023-11-28T18:26:45.876 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2023-11-28T18:26:45.876 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2023-11-28T18:26:45.876 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2023-11-28T18:26:45.876 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2023-11-28T18:26:45.876 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2023-11-28T18:26:45.876 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2023-11-28T18:26:45.876 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2023-11-28T18:26:45.876 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2023-11-28T18:26:45.877 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2023-11-28T18:26:45.877 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2023-11-28T18:26:45.877 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 0, 2023-11-28T18:26:45.877 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 0, 2023-11-28T18:26:45.877 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2023-11-28T18:26:45.877 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-28T18:26:45.877 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-28T18:26:45.877 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 0, 2023-11-28T18:26:45.877 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 0, 2023-11-28T18:26:45.877 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 0, 2023-11-28T18:26:45.877 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 0 2023-11-28T18:26:45.877 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.878 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2023-11-28T18:26:45.878 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2023-11-28T18:26:45.878 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2023-11-28T18:26:45.878 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2023-11-28T18:26:45.878 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 3, 2023-11-28T18:26:45.878 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 0, 2023-11-28T18:26:45.878 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2023-11-28T18:26:45.878 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2023-11-28T18:26:45.878 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2023-11-28T18:26:45.878 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 0, 2023-11-28T18:26:45.878 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 0, 2023-11-28T18:26:45.879 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2023-11-28T18:26:45.879 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2023-11-28T18:26:45.879 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2023-11-28T18:26:45.879 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2023-11-28T18:26:45.879 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2023-11-28T18:26:45.879 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2023-11-28T18:26:45.879 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2023-11-28T18:26:45.879 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2023-11-28T18:26:45.879 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2023-11-28T18:26:45.879 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2023-11-28T18:26:45.879 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2023-11-28T18:26:45.879 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2023-11-28T18:26:45.880 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2023-11-28T18:26:45.880 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.880 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2023-11-28T18:26:45.880 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2023-11-28T18:26:45.880 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 6, 2023-11-28T18:26:45.880 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2023-11-28T18:26:45.880 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2023-11-28T18:26:45.880 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 6, 2023-11-28T18:26:45.880 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2023-11-28T18:26:45.880 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2023-11-28T18:26:45.880 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 2, 2023-11-28T18:26:45.880 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2023-11-28T18:26:45.880 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2023-11-28T18:26:45.881 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 2, 2023-11-28T18:26:45.881 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2023-11-28T18:26:45.881 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2023-11-28T18:26:45.881 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2023-11-28T18:26:45.881 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2023-11-28T18:26:45.881 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 4233719, 2023-11-28T18:26:45.881 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 0, 2023-11-28T18:26:45.881 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2023-11-28T18:26:45.881 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4, 2023-11-28T18:26:45.881 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.015715047, 2023-11-28T18:26:45.881 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.003928761 2023-11-28T18:26:45.881 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.881 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 0 2023-11-28T18:26:45.882 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.882 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2023-11-28T18:26:45.882 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 66, 2023-11-28T18:26:45.882 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 66, 2023-11-28T18:26:45.882 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 0, 2023-11-28T18:26:45.882 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 40, 2023-11-28T18:26:45.882 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 41, 2023-11-28T18:26:45.882 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 1, 2023-11-28T18:26:45.882 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 63, 2023-11-28T18:26:45.882 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 63, 2023-11-28T18:26:45.882 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 0, 2023-11-28T18:26:45.882 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 52, 2023-11-28T18:26:45.882 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 52, 2023-11-28T18:26:45.883 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 0, 2023-11-28T18:26:45.883 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 47376, 2023-11-28T18:26:45.883 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 198952 2023-11-28T18:26:45.883 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.883 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2023-11-28T18:26:45.883 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 0, 2023-11-28T18:26:45.883 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2023-11-28T18:26:45.883 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 0, 2023-11-28T18:26:45.883 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 3, 2023-11-28T18:26:45.883 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2023-11-28T18:26:45.883 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2023-11-28T18:26:45.884 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.884 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.884 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.884 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.884 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2023-11-28T18:26:45.884 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.884 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.884 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.884 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.884 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2023-11-28T18:26:45.884 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.884 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.885 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.885 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.885 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2023-11-28T18:26:45.885 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.885 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.885 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.885 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.885 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2023-11-28T18:26:45.885 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.885 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.885 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.886 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.886 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2023-11-28T18:26:45.886 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.886 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.886 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.886 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.886 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2023-11-28T18:26:45.886 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.886 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.886 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.886 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.886 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2023-11-28T18:26:45.887 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.887 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.887 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.887 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.887 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2023-11-28T18:26:45.887 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.887 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.887 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.887 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.887 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2023-11-28T18:26:45.887 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.887 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.887 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.888 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.888 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2023-11-28T18:26:45.888 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.888 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.888 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.888 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.888 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2023-11-28T18:26:45.888 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.888 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.888 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.888 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.888 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2023-11-28T18:26:45.889 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.889 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.889 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.889 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.889 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2023-11-28T18:26:45.889 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.889 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.889 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.889 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.889 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2023-11-28T18:26:45.889 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.889 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.890 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.890 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.890 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2023-11-28T18:26:45.890 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.890 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.890 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.890 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.890 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2023-11-28T18:26:45.890 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.890 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.890 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.891 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.891 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2023-11-28T18:26:45.891 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.891 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.891 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.891 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.891 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2023-11-28T18:26:45.891 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.891 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.891 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.891 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.891 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2023-11-28T18:26:45.892 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.892 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.892 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.892 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.892 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2023-11-28T18:26:45.892 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.892 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.892 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.892 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.892 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2023-11-28T18:26:45.892 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.892 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.893 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.893 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.893 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2023-11-28T18:26:45.893 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.893 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.893 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.893 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.893 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2023-11-28T18:26:45.893 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.893 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.893 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.893 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.893 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2023-11-28T18:26:45.894 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.894 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.894 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.894 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.894 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2023-11-28T18:26:45.894 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.894 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.894 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.894 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.894 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2023-11-28T18:26:45.894 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.894 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.894 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.895 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.895 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2023-11-28T18:26:45.895 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.895 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.895 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.895 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.895 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2023-11-28T18:26:45.895 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.895 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.895 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.895 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.895 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2023-11-28T18:26:45.895 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2023-11-28T18:26:45.896 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2023-11-28T18:26:45.896 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.896 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.896 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.896 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:45.896 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.896 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2023-11-28T18:26:45.896 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 1, 2023-11-28T18:26:45.896 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 1, 2023-11-28T18:26:45.896 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 0, 2023-11-28T18:26:45.896 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 1, 2023-11-28T18:26:45.896 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2023-11-28T18:26:45.897 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 0, 2023-11-28T18:26:45.897 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 0, 2023-11-28T18:26:45.897 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 1001, 2023-11-28T18:26:45.897 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2023-11-28T18:26:45.897 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.897 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2023-11-28T18:26:45.897 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2023-11-28T18:26:45.897 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2023-11-28T18:26:45.897 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2023-11-28T18:26:45.898 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2023-11-28T18:26:45.898 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2023-11-28T18:26:45.898 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2023-11-28T18:26:45.898 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-28T18:26:45.898 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2023-11-28T18:26:45.898 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-28T18:26:45.898 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2023-11-28T18:26:45.898 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2023-11-28T18:26:45.898 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2023-11-28T18:26:45.898 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-28T18:26:45.898 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2023-11-28T18:26:45.898 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2023-11-28T18:26:45.899 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2023-11-28T18:26:45.899 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2023-11-28T18:26:45.899 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2023-11-28T18:26:45.899 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-28T18:26:45.899 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2023-11-28T18:26:45.899 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-28T18:26:45.899 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2023-11-28T18:26:45.899 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2023-11-28T18:26:45.899 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2023-11-28T18:26:45.899 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2023-11-28T18:26:45.899 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2023-11-28T18:26:45.899 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-28T18:26:45.899 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2023-11-28T18:26:45.900 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2023-11-28T18:26:45.900 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2023-11-28T18:26:45.900 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2023-11-28T18:26:45.900 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2023-11-28T18:26:45.900 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2023-11-28T18:26:45.900 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2023-11-28T18:26:45.900 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2023-11-28T18:26:45.900 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2023-11-28T18:26:45.900 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 220951, 2023-11-28T18:26:45.900 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 63, 2023-11-28T18:26:45.900 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2023-11-28T18:26:45.901 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2023-11-28T18:26:45.901 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2023-11-28T18:26:45.901 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2023-11-28T18:26:45.901 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2023-11-28T18:26:45.901 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2023-11-28T18:26:45.901 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2023-11-28T18:26:45.901 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2023-11-28T18:26:45.901 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 5848, 2023-11-28T18:26:45.901 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 95, 2023-11-28T18:26:45.901 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2023-11-28T18:26:45.901 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2023-11-28T18:26:45.901 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2023-11-28T18:26:45.901 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2023-11-28T18:26:45.902 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 256680, 2023-11-28T18:26:45.902 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 1201, 2023-11-28T18:26:45.902 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2023-11-28T18:26:45.902 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2023-11-28T18:26:45.902 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2023-11-28T18:26:45.902 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2023-11-28T18:26:45.902 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.902 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2023-11-28T18:26:45.902 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2023-11-28T18:26:45.902 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2023-11-28T18:26:45.902 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 39, 2023-11-28T18:26:45.902 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 51250, 2023-11-28T18:26:45.902 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 21, 2023-11-28T18:26:45.903 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 39, 2023-11-28T18:26:45.903 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2023-11-28T18:26:45.903 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 39, 2023-11-28T18:26:45.903 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.327996151, 2023-11-28T18:26:45.903 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.008410157 2023-11-28T18:26:45.903 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.903 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2023-11-28T18:26:45.903 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2023-11-28T18:26:45.903 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 39, 2023-11-28T18:26:45.903 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 62 2023-11-28T18:26:45.903 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.903 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 39, 2023-11-28T18:26:45.904 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 0, 2023-11-28T18:26:45.904 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 39, 2023-11-28T18:26:45.904 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2023-11-28T18:26:45.904 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2023-11-28T18:26:45.904 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 0, 2023-11-28T18:26:45.904 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 10, 2023-11-28T18:26:45.904 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 0, 2023-11-28T18:26:45.904 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 4, 2023-11-28T18:26:45.904 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 5, 2023-11-28T18:26:45.904 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2023-11-28T18:26:45.904 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2023-11-28T18:26:45.904 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2023-11-28T18:26:45.904 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2023-11-28T18:26:45.905 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 6, 2023-11-28T18:26:45.905 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2023-11-28T18:26:45.905 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2023-11-28T18:26:45.905 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2023-11-28T18:26:45.905 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 0, 2023-11-28T18:26:45.905 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 20, 2023-11-28T18:26:45.905 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2023-11-28T18:26:45.905 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2023-11-28T18:26:45.905 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2023-11-28T18:26:45.905 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2023-11-28T18:26:45.905 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2023-11-28T18:26:45.905 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2023-11-28T18:26:45.905 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2023-11-28T18:26:45.906 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2023-11-28T18:26:45.906 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2023-11-28T18:26:45.906 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2023-11-28T18:26:45.906 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2023-11-28T18:26:45.906 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2023-11-28T18:26:45.906 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2023-11-28T18:26:45.906 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2023-11-28T18:26:45.906 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2023-11-28T18:26:45.906 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2023-11-28T18:26:45.906 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2023-11-28T18:26:45.906 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2023-11-28T18:26:45.907 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2023-11-28T18:26:45.907 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2023-11-28T18:26:45.907 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2023-11-28T18:26:45.907 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2023-11-28T18:26:45.907 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2023-11-28T18:26:45.907 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2023-11-28T18:26:45.907 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2023-11-28T18:26:45.907 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2023-11-28T18:26:45.907 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 54, 2023-11-28T18:26:45.907 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2023-11-28T18:26:45.907 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 3, 2023-11-28T18:26:45.907 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 7, 2023-11-28T18:26:45.908 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 7, 2023-11-28T18:26:45.908 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 0, 2023-11-28T18:26:45.908 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2023-11-28T18:26:45.908 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 16, 2023-11-28T18:26:45.908 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 0, 2023-11-28T18:26:45.908 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 1 2023-11-28T18:26:45.908 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.908 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2023-11-28T18:26:45.908 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 1, 2023-11-28T18:26:45.908 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2023-11-28T18:26:45.909 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2023-11-28T18:26:45.909 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2023-11-28T18:26:45.909 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.909 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2023-11-28T18:26:45.909 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2023-11-28T18:26:45.909 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.909 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2023-11-28T18:26:45.909 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2023-11-28T18:26:45.909 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 0, 2023-11-28T18:26:45.909 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2023-11-28T18:26:45.909 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 0, 2023-11-28T18:26:45.910 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 0, 2023-11-28T18:26:45.910 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2023-11-28T18:26:45.910 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.910 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-28T18:26:45.910 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:45.910 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2023-11-28T18:26:45.910 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:45.910 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 156, 2023-11-28T18:26:45.911 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 214782, 2023-11-28T18:26:45.911 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:45.911 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 156, 2023-11-28T18:26:45.911 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2023-11-28T18:26:45.911 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2023-11-28T18:26:45.911 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 156, 2023-11-28T18:26:45.911 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 214782, 2023-11-28T18:26:45.911 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:45.911 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.911 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.911 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.912 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:45.912 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.912 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2023-11-28T18:26:45.912 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:45.912 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2023-11-28T18:26:45.912 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:45.912 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2023-11-28T18:26:45.912 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2023-11-28T18:26:45.912 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:45.912 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2023-11-28T18:26:45.913 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 39, 2023-11-28T18:26:45.913 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 51250, 2023-11-28T18:26:45.913 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 39, 2023-11-28T18:26:45.913 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 51250, 2023-11-28T18:26:45.913 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:45.913 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.913 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.913 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.913 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:45.913 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.913 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2023-11-28T18:26:45.913 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:45.914 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2023-11-28T18:26:45.914 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:45.914 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2023-11-28T18:26:45.914 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2023-11-28T18:26:45.914 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:45.914 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2023-11-28T18:26:45.914 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 39, 2023-11-28T18:26:45.914 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 39, 2023-11-28T18:26:45.914 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 39, 2023-11-28T18:26:45.914 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 39, 2023-11-28T18:26:45.915 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:45.915 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.915 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.915 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.915 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:45.915 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.915 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2023-11-28T18:26:45.915 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:45.915 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2023-11-28T18:26:45.915 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:45.916 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2023-11-28T18:26:45.916 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2023-11-28T18:26:45.916 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:45.916 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2023-11-28T18:26:45.916 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2023-11-28T18:26:45.916 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2023-11-28T18:26:45.916 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2023-11-28T18:26:45.916 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2023-11-28T18:26:45.916 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:45.916 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.916 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.917 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.917 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:45.917 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:45.917 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x56496458ca00": { 2023-11-28T18:26:45.917 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:45.917 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2023-11-28T18:26:45.917 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:45.917 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 6, 2023-11-28T18:26:45.917 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 39415, 2023-11-28T18:26:45.917 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:45.917 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 6, 2023-11-28T18:26:45.918 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2023-11-28T18:26:45.918 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2023-11-28T18:26:45.918 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 4, 2023-11-28T18:26:45.918 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 39415, 2023-11-28T18:26:45.918 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:45.918 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:45.918 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:45.918 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:45.918 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:45.918 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:45.919 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:26:45.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82aed1d640 1 -- 172.21.15.105:0/2381479265 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82940668d0 msgr2=0x7f8294068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82aed1d640 1 --2- 172.21.15.105:0/2381479265 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82940668d0 0x7f8294068d90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f82a80b4c60 tx=0x7f8298009040 comp rx=0 tx=0).stop 2023-11-28T18:26:45.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82aed1d640 1 -- 172.21.15.105:0/2381479265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82a80ad850 msgr2=0x7f82a809c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:45.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82aed1d640 1 --2- 172.21.15.105:0/2381479265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82a80ad850 0x7f82a809c330 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f82a4019ff0 tx=0x7f82a4019160 comp rx=0 tx=0).stop 2023-11-28T18:26:45.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82aed1d640 1 -- 172.21.15.105:0/2381479265 shutdown_connections 2023-11-28T18:26:45.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82aed1d640 1 --2- 172.21.15.105:0/2381479265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82a8159f40 0x7f82a80af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82aed1d640 1 --2- 172.21.15.105:0/2381479265 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82a80cc760 0x7f82a80af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82aed1d640 1 --2- 172.21.15.105:0/2381479265 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82940668d0 0x7f8294068d90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82aed1d640 1 --2- 172.21.15.105:0/2381479265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82a80ad850 0x7f82a809c330 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:45.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82aed1d640 1 -- 172.21.15.105:0/2381479265 >> 172.21.15.105:0/2381479265 conn(0x7f82a80ceb90 msgr2=0x7f82a8153330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:45.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82aed1d640 1 -- 172.21.15.105:0/2381479265 shutdown_connections 2023-11-28T18:26:45.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:45.856+0000 7f82aed1d640 1 -- 172.21.15.105:0/2381479265 wait complete. 2023-11-28T18:26:45.922 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 perf dump 2023-11-28T18:26:46.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.056+0000 7f0f35576640 1 Processor -- start 2023-11-28T18:26:46.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.056+0000 7f0f35576640 1 -- start start 2023-11-28T18:26:46.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.060+0000 7f0f35576640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0f300d0d40 0x7f0f300d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.060+0000 7f0f35576640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0f300d0450 0x7f0f300d1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.060+0000 7f0f35576640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f300cf440 con 0x7f0f3015d1a0 2023-11-28T18:26:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.060+0000 7f0f35576640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f300cffc0 con 0x7f0f300d0450 2023-11-28T18:26:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.060+0000 7f0f35576640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f300ceb40 con 0x7f0f300d0d40 2023-11-28T18:26:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.060+0000 7f0f2ffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0f3015d1a0 0x7f0f300ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46026/0 (socket says 172.21.15.105:46026) 2023-11-28T18:26:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.060+0000 7f0f2ffff640 1 -- 172.21.15.105:0/1457539784 learned_addr learned my addr 172.21.15.105:0/1457539784 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.060+0000 7f0f2f7fe640 1 --2- 172.21.15.105:0/1457539784 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0f300d0d40 0x7f0f300d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.060+0000 7f0f34d75640 1 --2- 172.21.15.105:0/1457539784 >> v2:172.21.15.148:3300/0 conn(0x7f0f300d0450 0x7f0f300d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.060+0000 7f0f2effd640 1 -- 172.21.15.105:0/1457539784 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 520233726 0 0) 0x7f0f300cf440 con 0x7f0f3015d1a0 2023-11-28T18:26:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.060+0000 7f0f2effd640 1 -- 172.21.15.105:0/1457539784 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f18003530 con 0x7f0f3015d1a0 2023-11-28T18:26:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.060+0000 7f0f2f7fe640 1 -- 172.21.15.105:0/1457539784 >> v2:172.21.15.148:3300/0 conn(0x7f0f300d0450 msgr2=0x7f0f300d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.060+0000 7f0f2f7fe640 1 --2- 172.21.15.105:0/1457539784 >> v2:172.21.15.148:3300/0 conn(0x7f0f300d0450 0x7f0f300d1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.060+0000 7f0f2f7fe640 1 -- 172.21.15.105:0/1457539784 >> v1:172.21.15.105:6789/0 conn(0x7f0f3015d1a0 legacy=0x7f0f300ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.060+0000 7f0f2f7fe640 1 -- 172.21.15.105:0/1457539784 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f300cf690 con 0x7f0f300d0d40 2023-11-28T18:26:46.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.064+0000 7f0f2f7fe640 1 --2- 172.21.15.105:0/1457539784 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0f300d0d40 0x7f0f300d13f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0f240097c0 tx=0x7f0f2401c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b9528c66cfc0ce2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:46.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.064+0000 7f0f2effd640 1 -- 172.21.15.105:0/1457539784 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f240ac620 con 0x7f0f300d0d40 2023-11-28T18:26:46.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.064+0000 7f0f2effd640 1 -- 172.21.15.105:0/1457539784 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f240ac780 con 0x7f0f300d0d40 2023-11-28T18:26:46.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.064+0000 7f0f2effd640 1 -- 172.21.15.105:0/1457539784 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f240acb10 con 0x7f0f300d0d40 2023-11-28T18:26:46.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.064+0000 7f0f35576640 1 -- 172.21.15.105:0/1457539784 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0f300d0d40 msgr2=0x7f0f300d13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:46.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.064+0000 7f0f35576640 1 --2- 172.21.15.105:0/1457539784 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0f300d0d40 0x7f0f300d13f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0f240097c0 tx=0x7f0f2401c3c0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.064+0000 7f0f35576640 1 -- 172.21.15.105:0/1457539784 shutdown_connections 2023-11-28T18:26:46.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.064+0000 7f0f35576640 1 --2- 172.21.15.105:0/1457539784 >> v2:172.21.15.148:3300/0 conn(0x7f0f300d0450 0x7f0f300d1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.064+0000 7f0f35576640 1 --2- 172.21.15.105:0/1457539784 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0f300d0d40 0x7f0f300d13f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.064+0000 7f0f35576640 1 -- 172.21.15.105:0/1457539784 >> 172.21.15.105:0/1457539784 conn(0x7f0f300d2e10 msgr2=0x7f0f300d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:46.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.064+0000 7f0f35576640 1 -- 172.21.15.105:0/1457539784 shutdown_connections 2023-11-28T18:26:46.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.064+0000 7f0f35576640 1 -- 172.21.15.105:0/1457539784 wait complete. 2023-11-28T18:26:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.064+0000 7f0f35576640 1 Processor -- start 2023-11-28T18:26:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.064+0000 7f0f35576640 1 -- start start 2023-11-28T18:26:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f35576640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0f300d0450 0x7f0f30159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f35576640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f300d0d40 0x7f0f300ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f35576640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f3015d1a0 0x7f0f300cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f35576640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f30159ba0 con 0x7f0f300d0d40 2023-11-28T18:26:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f35576640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f301591d0 con 0x7f0f3015d1a0 2023-11-28T18:26:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f35576640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f300cb5a0 con 0x7f0f300d0450 2023-11-28T18:26:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f2f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f300d0d40 0x7f0f300ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f2f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f300d0d40 0x7f0f300ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42244/0 (socket says 172.21.15.105:42244) 2023-11-28T18:26:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f2f7fe640 1 -- 172.21.15.105:0/1744104367 learned_addr learned my addr 172.21.15.105:0/1744104367 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f2ffff640 1 --2- 172.21.15.105:0/1744104367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0f300d0450 0x7f0f30159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f34d75640 1 --2- 172.21.15.105:0/1744104367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f3015d1a0 0x7f0f300cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f2f7fe640 1 -- 172.21.15.105:0/1744104367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0f300d0450 msgr2=0x7f0f30159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f2f7fe640 1 --2- 172.21.15.105:0/1744104367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0f300d0450 0x7f0f30159660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f2f7fe640 1 -- 172.21.15.105:0/1744104367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f3015d1a0 msgr2=0x7f0f300cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f2f7fe640 1 --2- 172.21.15.105:0/1744104367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f3015d1a0 0x7f0f300cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f2f7fe640 1 -- 172.21.15.105:0/1744104367 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f24009420 con 0x7f0f300d0d40 2023-11-28T18:26:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f2f7fe640 1 --2- 172.21.15.105:0/1744104367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f300d0d40 0x7f0f300ca7a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0f240af040 tx=0x7f0f24019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f2cff9640 1 -- 172.21.15.105:0/1744104367 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f240ac620 con 0x7f0f300d0d40 2023-11-28T18:26:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f2cff9640 1 -- 172.21.15.105:0/1744104367 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f240b5410 con 0x7f0f300d0d40 2023-11-28T18:26:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f35576640 1 -- 172.21.15.105:0/1744104367 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f300cb2c0 con 0x7f0f300d0d40 2023-11-28T18:26:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f2cff9640 1 -- 172.21.15.105:0/1744104367 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f240b57e0 con 0x7f0f300d0d40 2023-11-28T18:26:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f35576640 1 -- 172.21.15.105:0/1744104367 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f30157fc0 con 0x7f0f300d0d40 2023-11-28T18:26:46.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.068+0000 7f0f2cff9640 1 -- 172.21.15.105:0/1744104367 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f0f240b3040 con 0x7f0f300d0d40 2023-11-28T18:26:46.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f2cff9640 1 --2- 172.21.15.105:0/1744104367 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0f1c0668d0 0x7f0f1c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f2cff9640 1 -- 172.21.15.105:0/1744104367 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f0f2411e660 con 0x7f0f300d0d40 2023-11-28T18:26:46.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f2ffff640 1 --2- 172.21.15.105:0/1744104367 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0f1c0668d0 0x7f0f1c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f39b38000 1 Processor -- start 2023-11-28T18:26:46.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f39b38000 1 -- start start 2023-11-28T18:26:46.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f2ffff640 1 --2- 172.21.15.105:0/1744104367 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0f1c0668d0 0x7f0f1c068d90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0f200011f0 tx=0x7f0f20007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f39b38000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb1fcf37c0 0x55fb1fcf3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f39b38000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb1fcf40e0 0x55fb1fca8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f39b38000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fca9520 0x55fb1fca9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f39b38000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1fb009e0 con 0x55fb1fca9520 2023-11-28T18:26:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f39b38000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1fafdf10 con 0x55fb1fcf37c0 2023-11-28T18:26:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f39b38000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb1faf9800 con 0x55fb1fcf40e0 2023-11-28T18:26:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f34d75640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fca9520 0x55fb1fca9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f34d75640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fca9520 0x55fb1fca9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42252/0 (socket says 172.21.15.105:42252) 2023-11-28T18:26:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f34d75640 1 -- 172.21.15.105:0/4091989583 learned_addr learned my addr 172.21.15.105:0/4091989583 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f2f7fe640 1 --2- 172.21.15.105:0/4091989583 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb1fcf40e0 0x55fb1fca8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f2ffff640 1 --2- 172.21.15.105:0/4091989583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb1fcf37c0 0x55fb1fcf3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f34d75640 1 -- 172.21.15.105:0/4091989583 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb1fcf40e0 msgr2=0x55fb1fca8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:46.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f34d75640 1 --2- 172.21.15.105:0/4091989583 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb1fcf40e0 0x55fb1fca8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f34d75640 1 -- 172.21.15.105:0/4091989583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb1fcf37c0 msgr2=0x55fb1fcf3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:46.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f34d75640 1 --2- 172.21.15.105:0/4091989583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb1fcf37c0 0x55fb1fcf3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f34d75640 1 -- 172.21.15.105:0/4091989583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb1f7f1dc0 con 0x55fb1fca9520 2023-11-28T18:26:46.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f2ffff640 1 --2- 172.21.15.105:0/4091989583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb1fcf37c0 0x55fb1fcf3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:46.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f34d75640 1 --2- 172.21.15.105:0/4091989583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fca9520 0x55fb1fca9900 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0f280114a0 tx=0x7f0f28009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb53907be91b0bc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:46.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f15ffb640 1 -- 172.21.15.105:0/4091989583 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2800e040 con 0x55fb1fca9520 2023-11-28T18:26:46.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f15ffb640 1 -- 172.21.15.105:0/4091989583 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2800f040 con 0x55fb1fca9520 2023-11-28T18:26:46.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f15ffb640 1 -- 172.21.15.105:0/4091989583 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f28015a40 con 0x55fb1fca9520 2023-11-28T18:26:46.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f39b38000 1 -- 172.21.15.105:0/4091989583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fca9520 msgr2=0x55fb1fca9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:46.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.072+0000 7f0f39b38000 1 --2- 172.21.15.105:0/4091989583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fca9520 0x55fb1fca9900 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0f280114a0 tx=0x7f0f28009750 comp rx=0 tx=0).stop 2023-11-28T18:26:46.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f39b38000 1 -- 172.21.15.105:0/4091989583 shutdown_connections 2023-11-28T18:26:46.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f39b38000 1 --2- 172.21.15.105:0/4091989583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fca9520 0x55fb1fca9900 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f39b38000 1 --2- 172.21.15.105:0/4091989583 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb1fcf40e0 0x55fb1fca8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f39b38000 1 --2- 172.21.15.105:0/4091989583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb1fcf37c0 0x55fb1fcf3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f39b38000 1 -- 172.21.15.105:0/4091989583 >> 172.21.15.105:0/4091989583 conn(0x55fb1fcb1fb0 msgr2=0x55fb1fcd6360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:46.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f39b38000 1 -- 172.21.15.105:0/4091989583 shutdown_connections 2023-11-28T18:26:46.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f39b38000 1 -- 172.21.15.105:0/4091989583 wait complete. 2023-11-28T18:26:46.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f39b38000 1 -- start start 2023-11-28T18:26:46.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f39b38000 1 Processor -- start 2023-11-28T18:26:46.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f39b38000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fca9520 0x55fb1fcfa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f39b38000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb1fcf37c0 0x55fb1fcfae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f39b38000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb1fcf40e0 0x55fb1fcd7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f39b38000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1fab7a70 con 0x55fb1fca9520 2023-11-28T18:26:46.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f39b38000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1fab7710 con 0x55fb1fcf40e0 2023-11-28T18:26:46.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f39b38000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb1f7b2a40 con 0x55fb1fcf37c0 2023-11-28T18:26:46.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f39b38000 10 client.0 _ll_register_callbacks cb 0x55fb1fc94ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:46.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f2ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fca9520 0x55fb1fcfa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f2ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fca9520 0x55fb1fcfa910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42254/0 (socket says 172.21.15.105:42254) 2023-11-28T18:26:46.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f2ffff640 1 -- 172.21.15.105:0/2607411935 learned_addr learned my addr 172.21.15.105:0/2607411935 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:46.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f34d75640 1 --2- 172.21.15.105:0/2607411935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb1fcf40e0 0x55fb1fcd7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f2f7fe640 1 --2- 172.21.15.105:0/2607411935 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb1fcf37c0 0x55fb1fcfae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f2ffff640 1 -- 172.21.15.105:0/2607411935 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb1fcf37c0 msgr2=0x55fb1fcfae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:46.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f2ffff640 1 --2- 172.21.15.105:0/2607411935 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb1fcf37c0 0x55fb1fcfae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f2ffff640 1 -- 172.21.15.105:0/2607411935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb1fcf40e0 msgr2=0x55fb1fcd7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:46.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f2ffff640 1 --2- 172.21.15.105:0/2607411935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb1fcf40e0 0x55fb1fcd7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f2ffff640 1 -- 172.21.15.105:0/2607411935 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f28011230 con 0x55fb1fca9520 2023-11-28T18:26:46.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f34d75640 1 --2- 172.21.15.105:0/2607411935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb1fcf40e0 0x55fb1fcd7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:46.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f2ffff640 1 --2- 172.21.15.105:0/2607411935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fca9520 0x55fb1fcfa910 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0f20006c70 tx=0x7f0f20001160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6133db1ba46ade server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:46.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f0effd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:46.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f0effd640 1 -- 172.21.15.105:0/2607411935 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f20026070 con 0x55fb1fca9520 2023-11-28T18:26:46.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f0effd640 1 -- 172.21.15.105:0/2607411935 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f0f2001c040 con 0x55fb1fca9520 2023-11-28T18:26:46.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f39b38000 1 -- 172.21.15.105:0/2607411935 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb1fab2ab0 con 0x55fb1fca9520 2023-11-28T18:26:46.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f0effd640 1 -- 172.21.15.105:0/2607411935 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f20005a70 con 0x55fb1fca9520 2023-11-28T18:26:46.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f39b38000 10 client.6480 fetch_fsmap learned FSMap version 47 2023-11-28T18:26:46.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f39b38000 1 -- 172.21.15.105:0/2607411935 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55fb1faba260 con 0x55fb1fca9520 2023-11-28T18:26:46.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f0effd640 1 -- 172.21.15.105:0/2607411935 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f20030020 con 0x55fb1fca9520 2023-11-28T18:26:46.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f0effd640 1 -- 172.21.15.105:0/2607411935 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 47) v1 ==== 3326+0+0 (secure 0 0 0) 0x7f0f20029020 con 0x55fb1fca9520 2023-11-28T18:26:46.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.076+0000 7f0f39b38000 10 client.6480 fetch_fsmap finished waiting for FSMap version 47 2023-11-28T18:26:46.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f39b38000 10 client.6480 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2023-11-28T18:26:46.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f39b38000 20 client.6480 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:46.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f39b38000 1 --2- 172.21.15.105:0/2607411935 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55fb1fcd8300 0x55fb1fcd8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f39b38000 4 client.6480 mds_command: new command op to 5842 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:26:46.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f39b38000 1 -- 172.21.15.105:0/2607411935 --> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb1faae850 con 0x55fb1fcd8300 2023-11-28T18:26:46.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f2f7fe640 1 --2- 172.21.15.105:0/2607411935 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55fb1fcd8300 0x55fb1fcd8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f2f7fe640 1 --2- 172.21.15.105:0/2607411935 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55fb1fcd8300 0x55fb1fcd8710 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f45a824772d80bf3 server_cookie=f0ece6075e341def in_seq=0 out_seq=0 2023-11-28T18:26:46.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f0effd640 10 client.6480 ms_handle_connect on v2:172.21.15.148:6832/2696773413 2023-11-28T18:26:46.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f0effd640 1 -- 172.21.15.105:0/2607411935 <== mds.0 v2:172.21.15.148:6832/2696773413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fb1faae850 con 0x55fb1fcd8300 2023-11-28T18:26:46.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f0effd640 10 client.6480 handle_command_reply: tid=0 2023-11-28T18:26:46.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f39b38000 1 client.6480 shutdown 2023-11-28T18:26:46.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f39b38000 1 -- 172.21.15.105:0/2607411935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fca9520 msgr2=0x55fb1fcfa910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:46.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f39b38000 1 --2- 172.21.15.105:0/2607411935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fca9520 0x55fb1fcfa910 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0f20006c70 tx=0x7f0f20001160 comp rx=0 tx=0).stop 2023-11-28T18:26:46.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f39b38000 1 -- 172.21.15.105:0/2607411935 shutdown_connections 2023-11-28T18:26:46.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f39b38000 1 --2- 172.21.15.105:0/2607411935 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55fb1fcd8300 0x55fb1fcd8710 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f39b38000 1 --2- 172.21.15.105:0/2607411935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb1fcf40e0 0x55fb1fcd7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f39b38000 1 --2- 172.21.15.105:0/2607411935 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb1fcf37c0 0x55fb1fcfae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f0effd640 0 client.6480 ms_handle_reset on v2:172.21.15.148:6832/2696773413 2023-11-28T18:26:46.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f39b38000 1 --2- 172.21.15.105:0/2607411935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fca9520 0x55fb1fcfa910 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f39b38000 1 -- 172.21.15.105:0/2607411935 >> 172.21.15.105:0/2607411935 conn(0x55fb1fcb1fb0 msgr2=0x55fb1fca86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:46.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f39b38000 1 -- 172.21.15.105:0/2607411935 shutdown_connections 2023-11-28T18:26:46.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f39b38000 1 -- 172.21.15.105:0/2607411935 wait complete. 2023-11-28T18:26:46.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.080+0000 7f0f39b38000 20 client.6480 trim_cache size 0 max 16384 2023-11-28T18:26:46.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.092+0000 7f0f39b38000 1 Processor -- start 2023-11-28T18:26:46.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.092+0000 7f0f39b38000 1 -- start start 2023-11-28T18:26:46.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f39b38000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fcb1fb0 0x55fb1fd50020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f39b38000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb1fca9520 0x55fb1fd50560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f39b38000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb1fcf37c0 0x55fb1fd50aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f39b38000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1faf8690 con 0x55fb1fcb1fb0 2023-11-28T18:26:46.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f39b38000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1fa7dc20 con 0x55fb1fca9520 2023-11-28T18:26:46.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f39b38000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb1facf7b0 con 0x55fb1fcf37c0 2023-11-28T18:26:46.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f2ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fcb1fb0 0x55fb1fd50020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f2ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fcb1fb0 0x55fb1fd50020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42264/0 (socket says 172.21.15.105:42264) 2023-11-28T18:26:46.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f2ffff640 1 -- 172.21.15.105:0/4142907193 learned_addr learned my addr 172.21.15.105:0/4142907193 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:46.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f2f7fe640 1 --2- 172.21.15.105:0/4142907193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb1fca9520 0x55fb1fd50560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f34d75640 1 --2- 172.21.15.105:0/4142907193 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb1fcf37c0 0x55fb1fd50aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f2ffff640 1 -- 172.21.15.105:0/4142907193 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb1fcf37c0 msgr2=0x55fb1fd50aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:46.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f2ffff640 1 --2- 172.21.15.105:0/4142907193 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb1fcf37c0 0x55fb1fd50aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f2ffff640 1 -- 172.21.15.105:0/4142907193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb1fca9520 msgr2=0x55fb1fd50560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:46.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f2ffff640 1 --2- 172.21.15.105:0/4142907193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb1fca9520 0x55fb1fd50560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f2ffff640 1 -- 172.21.15.105:0/4142907193 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb1faa0760 con 0x55fb1fcb1fb0 2023-11-28T18:26:46.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f2ffff640 1 --2- 172.21.15.105:0/4142907193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fcb1fb0 0x55fb1fd50020 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0f20001ec0 tx=0x7f0f2001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa3b665abacc3d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:46.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f167fc640 1 -- 172.21.15.105:0/4142907193 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f20026020 con 0x55fb1fcb1fb0 2023-11-28T18:26:46.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f167fc640 1 -- 172.21.15.105:0/4142907193 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f20030070 con 0x55fb1fcb1fb0 2023-11-28T18:26:46.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f167fc640 1 -- 172.21.15.105:0/4142907193 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f20014cb0 con 0x55fb1fcb1fb0 2023-11-28T18:26:46.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f39b38000 1 -- 172.21.15.105:0/4142907193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fcb1fb0 msgr2=0x55fb1fd50020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:46.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f39b38000 1 --2- 172.21.15.105:0/4142907193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fcb1fb0 0x55fb1fd50020 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0f20001ec0 tx=0x7f0f2001d000 comp rx=0 tx=0).stop 2023-11-28T18:26:46.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f39b38000 1 -- 172.21.15.105:0/4142907193 shutdown_connections 2023-11-28T18:26:46.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f39b38000 1 --2- 172.21.15.105:0/4142907193 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb1fcf37c0 0x55fb1fd50aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f39b38000 1 --2- 172.21.15.105:0/4142907193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb1fca9520 0x55fb1fd50560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f39b38000 1 --2- 172.21.15.105:0/4142907193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fcb1fb0 0x55fb1fd50020 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f39b38000 1 -- 172.21.15.105:0/4142907193 >> 172.21.15.105:0/4142907193 conn(0x55fb1fcadc90 msgr2=0x55fb1fcf3040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:46.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f39b38000 1 -- 172.21.15.105:0/4142907193 shutdown_connections 2023-11-28T18:26:46.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f39b38000 1 -- 172.21.15.105:0/4142907193 wait complete. 2023-11-28T18:26:46.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f39b38000 1 -- start start 2023-11-28T18:26:46.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.096+0000 7f0f39b38000 1 Processor -- start 2023-11-28T18:26:46.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f39b38000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb1fca9520 0x55fb1fd54380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f39b38000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb1fcb1fb0 0x55fb1fd548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f39b38000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fcf37c0 0x55fb1fd0efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f39b38000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1fd0a130 con 0x55fb1fcf37c0 2023-11-28T18:26:46.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f39b38000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1fcf9a60 con 0x55fb1fcb1fb0 2023-11-28T18:26:46.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f39b38000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb1faf7010 con 0x55fb1fca9520 2023-11-28T18:26:46.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f39b38000 10 client.0 _ll_register_callbacks cb 0x55fb1fac1840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:26:46.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f34d75640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fcf37c0 0x55fb1fd0efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f2ffff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb1fca9520 0x55fb1fd54380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f2f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb1fcb1fb0 0x55fb1fd548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f34d75640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fcf37c0 0x55fb1fd0efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42266/0 (socket says 172.21.15.105:42266) 2023-11-28T18:26:46.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f34d75640 1 -- 172.21.15.105:0/646823271 learned_addr learned my addr 172.21.15.105:0/646823271 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:46.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f34d75640 1 -- 172.21.15.105:0/646823271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb1fca9520 msgr2=0x55fb1fd54380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:46.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f34d75640 1 --2- 172.21.15.105:0/646823271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb1fca9520 0x55fb1fd54380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f34d75640 1 -- 172.21.15.105:0/646823271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb1fcb1fb0 msgr2=0x55fb1fd548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:46.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f34d75640 1 --2- 172.21.15.105:0/646823271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb1fcb1fb0 0x55fb1fd548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f34d75640 1 -- 172.21.15.105:0/646823271 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb1faba260 con 0x55fb1fcf37c0 2023-11-28T18:26:46.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f34d75640 1 --2- 172.21.15.105:0/646823271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fcf37c0 0x55fb1fd0efc0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0f28019000 tx=0x7f0f28004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6e0c2da80c851e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:46.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f0effd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:26:46.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f0effd640 1 -- 172.21.15.105:0/646823271 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2800e040 con 0x55fb1fcf37c0 2023-11-28T18:26:46.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f0effd640 1 -- 172.21.15.105:0/646823271 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f0f28009d60 con 0x55fb1fcf37c0 2023-11-28T18:26:46.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f39b38000 1 -- 172.21.15.105:0/646823271 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb1fabc3a0 con 0x55fb1fcf37c0 2023-11-28T18:26:46.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f0effd640 1 -- 172.21.15.105:0/646823271 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f28015d00 con 0x55fb1fcf37c0 2023-11-28T18:26:46.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f39b38000 10 client.6486 fetch_fsmap learned FSMap version 47 2023-11-28T18:26:46.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f39b38000 1 -- 172.21.15.105:0/646823271 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55fb1fa94040 con 0x55fb1fcf37c0 2023-11-28T18:26:46.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f0effd640 1 -- 172.21.15.105:0/646823271 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb1fabc3a0 con 0x55fb1fcf37c0 2023-11-28T18:26:46.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f0effd640 1 -- 172.21.15.105:0/646823271 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 47) v1 ==== 3326+0+0 (secure 0 0 0) 0x7f0f2801b020 con 0x55fb1fcf37c0 2023-11-28T18:26:46.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f39b38000 10 client.6486 fetch_fsmap finished waiting for FSMap version 47 2023-11-28T18:26:46.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f39b38000 10 client.6486 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2023-11-28T18:26:46.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f39b38000 20 client.6486 populate_metadata read hostname 'smithi105' 2023-11-28T18:26:46.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f39b38000 1 --2- 172.21.15.105:0/646823271 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55fb1fcf40e0 0x55fb1fd0f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f39b38000 4 client.6486 mds_command: new command op to 5842 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2023-11-28T18:26:46.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f39b38000 1 -- 172.21.15.105:0/646823271 --> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55fb1faf27e0 con 0x55fb1fcf40e0 2023-11-28T18:26:46.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f2ffff640 1 --2- 172.21.15.105:0/646823271 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55fb1fcf40e0 0x55fb1fd0f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f0effd640 10 client.6486 ms_handle_connect on v2:172.21.15.148:6832/2696773413 2023-11-28T18:26:46.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f2ffff640 1 --2- 172.21.15.105:0/646823271 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55fb1fcf40e0 0x55fb1fd0f920 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61bc8f7fbdf165da server_cookie=ed988be23db0db2d in_seq=0 out_seq=0 2023-11-28T18:26:46.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f0effd640 1 -- 172.21.15.105:0/646823271 <== mds.0 v2:172.21.15.148:6832/2696773413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17753 (crc 0 0 0) 0x55fb1faf27e0 con 0x55fb1fcf40e0 2023-11-28T18:26:46.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f0effd640 10 client.6486 handle_command_reply: tid=0 2023-11-28T18:26:46.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.100+0000 7f0f39b38000 1 client.6486 shutdown 2023-11-28T18:26:46.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f39b38000 1 -- 172.21.15.105:0/646823271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fcf37c0 msgr2=0x55fb1fd0efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:46.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f39b38000 1 --2- 172.21.15.105:0/646823271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fcf37c0 0x55fb1fd0efc0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0f28019000 tx=0x7f0f28004ac0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f39b38000 1 -- 172.21.15.105:0/646823271 shutdown_connections 2023-11-28T18:26:46.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f39b38000 1 --2- 172.21.15.105:0/646823271 >> [v2:172.21.15.148:6832/2696773413,v1:172.21.15.148:6834/2696773413] conn(0x55fb1fcf40e0 0x55fb1fd0f920 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f39b38000 1 --2- 172.21.15.105:0/646823271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb1fcf37c0 0x55fb1fd0efc0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f39b38000 1 --2- 172.21.15.105:0/646823271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb1fcb1fb0 0x55fb1fd548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f39b38000 1 --2- 172.21.15.105:0/646823271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb1fca9520 0x55fb1fd54380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f0effd640 0 client.6486 ms_handle_reset on v2:172.21.15.148:6832/2696773413 2023-11-28T18:26:46.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f39b38000 1 -- 172.21.15.105:0/646823271 >> 172.21.15.105:0/646823271 conn(0x55fb1fcadc90 msgr2=0x55fb1fd00760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:46.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f39b38000 1 -- 172.21.15.105:0/646823271 shutdown_connections 2023-11-28T18:26:46.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f39b38000 1 -- 172.21.15.105:0/646823271 wait complete. 2023-11-28T18:26:46.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f39b38000 20 client.6486 trim_cache size 0 max 16384 2023-11-28T18:26:46.111 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:26:46.111 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2023-11-28T18:26:46.111 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 510, 2023-11-28T18:26:46.111 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 450, 2023-11-28T18:26:46.111 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 141580, 2023-11-28T18:26:46.111 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 1434588, 2023-11-28T18:26:46.111 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 9, 2023-11-28T18:26:46.111 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 3, 2023-11-28T18:26:46.111 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.051692423, 2023-11-28T18:26:46.112 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.016148025, 2023-11-28T18:26:46.112 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.018120519, 2023-11-28T18:26:46.112 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.012314174, 2023-11-28T18:26:46.112 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2023-11-28T18:26:46.112 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 448, 2023-11-28T18:26:46.112 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.093782093, 2023-11-28T18:26:46.112 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000209335 2023-11-28T18:26:46.112 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.112 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2023-11-28T18:26:46.112 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 143, 2023-11-28T18:26:46.112 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000036174, 2023-11-28T18:26:46.113 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000252 2023-11-28T18:26:46.113 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.113 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-28T18:26:46.113 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2023-11-28T18:26:46.113 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.113 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2023-11-28T18:26:46.113 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 233, 2023-11-28T18:26:46.113 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 160, 2023-11-28T18:26:46.113 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 198361, 2023-11-28T18:26:46.113 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 201712, 2023-11-28T18:26:46.113 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 7, 2023-11-28T18:26:46.113 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2023-11-28T18:26:46.114 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.029189733, 2023-11-28T18:26:46.114 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.008194275, 2023-11-28T18:26:46.114 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.021520897, 2023-11-28T18:26:46.114 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.003385023, 2023-11-28T18:26:46.114 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2023-11-28T18:26:46.114 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 160, 2023-11-28T18:26:46.114 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.091120078, 2023-11-28T18:26:46.114 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000569500 2023-11-28T18:26:46.114 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.114 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2023-11-28T18:26:46.114 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 119, 2023-11-28T18:26:46.114 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000016232, 2023-11-28T18:26:46.114 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000136 2023-11-28T18:26:46.115 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.115 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 140144, 2023-11-28T18:26:46.115 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 16320 2023-11-28T18:26:46.115 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.115 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2023-11-28T18:26:46.115 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 1067, 2023-11-28T18:26:46.115 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 1840, 2023-11-28T18:26:46.115 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 355208, 2023-11-28T18:26:46.115 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 945517, 2023-11-28T18:26:46.115 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 18, 2023-11-28T18:26:46.115 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 7, 2023-11-28T18:26:46.115 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.126118250, 2023-11-28T18:26:46.115 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.057590616, 2023-11-28T18:26:46.116 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.076215436, 2023-11-28T18:26:46.116 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.002730073, 2023-11-28T18:26:46.116 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2023-11-28T18:26:46.116 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1838, 2023-11-28T18:26:46.116 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.092368014, 2023-11-28T18:26:46.116 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000050254 2023-11-28T18:26:46.116 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.116 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2023-11-28T18:26:46.116 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2629, 2023-11-28T18:26:46.116 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000561187, 2023-11-28T18:26:46.116 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000213 2023-11-28T18:26:46.116 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.116 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-28T18:26:46.116 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 73520 2023-11-28T18:26:46.117 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.117 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2023-11-28T18:26:46.117 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2023-11-28T18:26:46.117 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2023-11-28T18:26:46.117 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.117 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2023-11-28T18:26:46.117 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2023-11-28T18:26:46.117 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2023-11-28T18:26:46.117 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 665, 2023-11-28T18:26:46.117 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.353316381, 2023-11-28T18:26:46.117 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000531302 2023-11-28T18:26:46.117 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:46.117 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.117 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2023-11-28T18:26:46.118 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2023-11-28T18:26:46.118 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2023-11-28T18:26:46.118 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 127, 2023-11-28T18:26:46.118 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.008517043, 2023-11-28T18:26:46.118 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000067063 2023-11-28T18:26:46.118 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:46.118 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.118 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2023-11-28T18:26:46.118 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 705, 2023-11-28T18:26:46.118 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 705, 2023-11-28T18:26:46.119 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2023-11-28T18:26:46.119 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 705, 2023-11-28T18:26:46.119 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 1.087538334, 2023-11-28T18:26:46.119 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001542607 2023-11-28T18:26:46.119 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.119 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2023-11-28T18:26:46.119 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2023-11-28T18:26:46.119 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2023-11-28T18:26:46.119 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2023-11-28T18:26:46.119 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 21, 2023-11-28T18:26:46.119 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 1, 2023-11-28T18:26:46.119 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2023-11-28T18:26:46.119 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 96, 2023-11-28T18:26:46.120 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 0, 2023-11-28T18:26:46.120 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 96, 2023-11-28T18:26:46.120 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2023-11-28T18:26:46.120 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 96, 2023-11-28T18:26:46.120 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 116, 2023-11-28T18:26:46.120 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 86, 2023-11-28T18:26:46.120 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 86, 2023-11-28T18:26:46.120 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 10, 2023-11-28T18:26:46.120 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 1404, 2023-11-28T18:26:46.120 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 553, 2023-11-28T18:26:46.120 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2023-11-28T18:26:46.120 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 0, 2023-11-28T18:26:46.120 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2023-11-28T18:26:46.120 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2023-11-28T18:26:46.120 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2023-11-28T18:26:46.121 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 1951, 2023-11-28T18:26:46.121 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2023-11-28T18:26:46.121 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 5, 2023-11-28T18:26:46.121 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 116, 2023-11-28T18:26:46.121 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 0, 2023-11-28T18:26:46.121 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 0, 2023-11-28T18:26:46.121 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2023-11-28T18:26:46.121 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2023-11-28T18:26:46.121 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2023-11-28T18:26:46.121 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 100, 2023-11-28T18:26:46.121 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 500, 2023-11-28T18:26:46.121 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2023-11-28T18:26:46.121 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2023-11-28T18:26:46.121 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2023-11-28T18:26:46.122 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2023-11-28T18:26:46.122 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2023-11-28T18:26:46.122 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2023-11-28T18:26:46.122 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2023-11-28T18:26:46.122 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2023-11-28T18:26:46.122 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2023-11-28T18:26:46.122 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2023-11-28T18:26:46.122 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 179, 2023-11-28T18:26:46.122 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 253, 2023-11-28T18:26:46.122 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2023-11-28T18:26:46.122 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-28T18:26:46.122 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-28T18:26:46.122 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 279, 2023-11-28T18:26:46.123 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 176, 2023-11-28T18:26:46.123 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 2, 2023-11-28T18:26:46.123 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 433 2023-11-28T18:26:46.123 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.123 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2023-11-28T18:26:46.123 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2023-11-28T18:26:46.123 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2023-11-28T18:26:46.123 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2023-11-28T18:26:46.123 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 0, 2023-11-28T18:26:46.123 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 7, 2023-11-28T18:26:46.123 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2023-11-28T18:26:46.124 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2023-11-28T18:26:46.124 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2023-11-28T18:26:46.124 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 77, 2023-11-28T18:26:46.124 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 77, 2023-11-28T18:26:46.124 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2023-11-28T18:26:46.124 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2023-11-28T18:26:46.124 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2023-11-28T18:26:46.124 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2023-11-28T18:26:46.124 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2023-11-28T18:26:46.124 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2023-11-28T18:26:46.124 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2023-11-28T18:26:46.124 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2023-11-28T18:26:46.124 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 5, 2023-11-28T18:26:46.125 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2023-11-28T18:26:46.125 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 1, 2023-11-28T18:26:46.125 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2023-11-28T18:26:46.125 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2023-11-28T18:26:46.125 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.125 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2023-11-28T18:26:46.125 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2023-11-28T18:26:46.125 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 626, 2023-11-28T18:26:46.125 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2023-11-28T18:26:46.125 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2023-11-28T18:26:46.125 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 626, 2023-11-28T18:26:46.125 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2023-11-28T18:26:46.125 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2023-11-28T18:26:46.125 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 2, 2023-11-28T18:26:46.125 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2023-11-28T18:26:46.126 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2023-11-28T18:26:46.126 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 2, 2023-11-28T18:26:46.126 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2023-11-28T18:26:46.126 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2023-11-28T18:26:46.126 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2023-11-28T18:26:46.126 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2023-11-28T18:26:46.126 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 5428638, 2023-11-28T18:26:46.126 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 0, 2023-11-28T18:26:46.126 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2023-11-28T18:26:46.126 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 345, 2023-11-28T18:26:46.126 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.818773215, 2023-11-28T18:26:46.126 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.002373255 2023-11-28T18:26:46.126 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.127 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 0 2023-11-28T18:26:46.127 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.127 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2023-11-28T18:26:46.127 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 98, 2023-11-28T18:26:46.127 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 291, 2023-11-28T18:26:46.127 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 193, 2023-11-28T18:26:46.127 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 62, 2023-11-28T18:26:46.127 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 122, 2023-11-28T18:26:46.127 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 60, 2023-11-28T18:26:46.127 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 96, 2023-11-28T18:26:46.127 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 365, 2023-11-28T18:26:46.127 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 269, 2023-11-28T18:26:46.127 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 86, 2023-11-28T18:26:46.128 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 279, 2023-11-28T18:26:46.128 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 193, 2023-11-28T18:26:46.128 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 53528, 2023-11-28T18:26:46.128 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 207144 2023-11-28T18:26:46.128 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.128 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2023-11-28T18:26:46.128 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 1059, 2023-11-28T18:26:46.128 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2023-11-28T18:26:46.128 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 705, 2023-11-28T18:26:46.128 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 9, 2023-11-28T18:26:46.128 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2023-11-28T18:26:46.128 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2023-11-28T18:26:46.128 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 176, 2023-11-28T18:26:46.129 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.331841107, 2023-11-28T18:26:46.129 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001885460 2023-11-28T18:26:46.129 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.129 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2023-11-28T18:26:46.129 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 16, 2023-11-28T18:26:46.129 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.003981788, 2023-11-28T18:26:46.129 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000248861 2023-11-28T18:26:46.129 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.129 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2023-11-28T18:26:46.129 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.129 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.129 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.129 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.130 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2023-11-28T18:26:46.130 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.130 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.130 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.130 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.130 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2023-11-28T18:26:46.130 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 331, 2023-11-28T18:26:46.130 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.057076197, 2023-11-28T18:26:46.130 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000172435 2023-11-28T18:26:46.130 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.130 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2023-11-28T18:26:46.130 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.130 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.131 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.131 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.131 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2023-11-28T18:26:46.131 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.131 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.131 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.131 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.131 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2023-11-28T18:26:46.131 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.131 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.131 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.131 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.131 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2023-11-28T18:26:46.132 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.132 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.132 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.132 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.132 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2023-11-28T18:26:46.132 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.132 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.132 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.132 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.132 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2023-11-28T18:26:46.132 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.132 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.132 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.132 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.133 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2023-11-28T18:26:46.133 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 102, 2023-11-28T18:26:46.133 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.289808368, 2023-11-28T18:26:46.133 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.002841258 2023-11-28T18:26:46.133 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.133 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2023-11-28T18:26:46.133 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.133 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.133 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.133 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.133 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2023-11-28T18:26:46.133 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.133 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.134 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.134 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.134 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2023-11-28T18:26:46.134 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.134 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.134 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.134 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.134 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2023-11-28T18:26:46.134 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2023-11-28T18:26:46.134 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.001026394, 2023-11-28T18:26:46.134 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001026394 2023-11-28T18:26:46.134 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.134 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2023-11-28T18:26:46.135 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.135 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.135 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.135 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.135 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2023-11-28T18:26:46.135 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.135 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.135 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.135 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.135 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2023-11-28T18:26:46.135 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.135 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.135 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.135 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.136 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2023-11-28T18:26:46.136 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2023-11-28T18:26:46.136 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.003064842, 2023-11-28T18:26:46.136 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.003064842 2023-11-28T18:26:46.136 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.136 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2023-11-28T18:26:46.136 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.136 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.136 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.136 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.136 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2023-11-28T18:26:46.136 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.136 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.137 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.137 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.137 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2023-11-28T18:26:46.137 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2023-11-28T18:26:46.137 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000418227, 2023-11-28T18:26:46.137 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000418227 2023-11-28T18:26:46.137 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.137 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2023-11-28T18:26:46.137 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.137 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.137 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.137 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.137 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2023-11-28T18:26:46.138 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.138 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.138 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.138 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.138 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2023-11-28T18:26:46.138 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.138 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.138 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.138 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.138 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2023-11-28T18:26:46.138 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2023-11-28T18:26:46.138 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000345721, 2023-11-28T18:26:46.138 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000345721 2023-11-28T18:26:46.139 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.139 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2023-11-28T18:26:46.139 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.139 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.139 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.139 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.139 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2023-11-28T18:26:46.139 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 76, 2023-11-28T18:26:46.139 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.399975690, 2023-11-28T18:26:46.139 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.005262838 2023-11-28T18:26:46.139 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.139 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2023-11-28T18:26:46.139 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2023-11-28T18:26:46.140 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2023-11-28T18:26:46.140 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.140 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.140 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.140 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:46.140 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.140 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2023-11-28T18:26:46.140 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 1, 2023-11-28T18:26:46.140 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 1, 2023-11-28T18:26:46.140 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 0, 2023-11-28T18:26:46.140 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 1, 2023-11-28T18:26:46.140 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2023-11-28T18:26:46.140 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 679, 2023-11-28T18:26:46.141 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 679, 2023-11-28T18:26:46.141 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 33, 2023-11-28T18:26:46.141 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2023-11-28T18:26:46.141 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.141 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2023-11-28T18:26:46.141 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2023-11-28T18:26:46.141 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2023-11-28T18:26:46.141 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2023-11-28T18:26:46.141 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2023-11-28T18:26:46.141 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2023-11-28T18:26:46.141 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2023-11-28T18:26:46.141 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-28T18:26:46.142 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2023-11-28T18:26:46.142 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-28T18:26:46.142 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2023-11-28T18:26:46.142 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2023-11-28T18:26:46.142 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2023-11-28T18:26:46.142 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-28T18:26:46.142 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2023-11-28T18:26:46.142 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2023-11-28T18:26:46.142 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2023-11-28T18:26:46.142 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2023-11-28T18:26:46.142 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2023-11-28T18:26:46.142 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-28T18:26:46.142 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2023-11-28T18:26:46.143 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-28T18:26:46.143 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2023-11-28T18:26:46.143 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2023-11-28T18:26:46.143 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2023-11-28T18:26:46.143 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2023-11-28T18:26:46.143 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2023-11-28T18:26:46.143 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-28T18:26:46.143 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2023-11-28T18:26:46.143 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2023-11-28T18:26:46.143 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2023-11-28T18:26:46.143 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2023-11-28T18:26:46.144 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2023-11-28T18:26:46.144 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2023-11-28T18:26:46.144 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2023-11-28T18:26:46.144 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2023-11-28T18:26:46.144 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2023-11-28T18:26:46.144 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 257070, 2023-11-28T18:26:46.144 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 93, 2023-11-28T18:26:46.144 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 88, 2023-11-28T18:26:46.144 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 1, 2023-11-28T18:26:46.144 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2023-11-28T18:26:46.144 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2023-11-28T18:26:46.144 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2023-11-28T18:26:46.144 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2023-11-28T18:26:46.145 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2023-11-28T18:26:46.145 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2023-11-28T18:26:46.145 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 5848, 2023-11-28T18:26:46.145 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 95, 2023-11-28T18:26:46.145 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2023-11-28T18:26:46.145 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2023-11-28T18:26:46.145 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2023-11-28T18:26:46.145 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2023-11-28T18:26:46.145 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 393445, 2023-11-28T18:26:46.146 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 2335, 2023-11-28T18:26:46.146 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2023-11-28T18:26:46.146 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2023-11-28T18:26:46.146 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2023-11-28T18:26:46.146 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2023-11-28T18:26:46.146 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.146 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2023-11-28T18:26:46.146 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2023-11-28T18:26:46.146 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2023-11-28T18:26:46.146 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 490, 2023-11-28T18:26:46.146 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 1373344, 2023-11-28T18:26:46.147 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 28, 2023-11-28T18:26:46.147 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 490, 2023-11-28T18:26:46.147 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2023-11-28T18:26:46.147 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 490, 2023-11-28T18:26:46.147 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 2.171971850, 2023-11-28T18:26:46.147 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.004432595 2023-11-28T18:26:46.147 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.147 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2023-11-28T18:26:46.147 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2023-11-28T18:26:46.147 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 490, 2023-11-28T18:26:46.147 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 531 2023-11-28T18:26:46.147 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.148 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 490, 2023-11-28T18:26:46.148 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 3, 2023-11-28T18:26:46.148 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 487, 2023-11-28T18:26:46.148 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2023-11-28T18:26:46.148 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2023-11-28T18:26:46.148 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 0, 2023-11-28T18:26:46.148 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 11, 2023-11-28T18:26:46.148 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 3, 2023-11-28T18:26:46.148 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 348, 2023-11-28T18:26:46.148 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 11, 2023-11-28T18:26:46.148 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2023-11-28T18:26:46.148 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2023-11-28T18:26:46.148 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2023-11-28T18:26:46.149 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 1, 2023-11-28T18:26:46.149 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 89, 2023-11-28T18:26:46.149 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2023-11-28T18:26:46.149 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2023-11-28T18:26:46.149 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2023-11-28T18:26:46.149 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 0, 2023-11-28T18:26:46.149 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 22, 2023-11-28T18:26:46.149 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2023-11-28T18:26:46.149 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2023-11-28T18:26:46.149 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2023-11-28T18:26:46.149 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2023-11-28T18:26:46.149 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2023-11-28T18:26:46.150 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2023-11-28T18:26:46.150 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2023-11-28T18:26:46.150 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2023-11-28T18:26:46.150 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2023-11-28T18:26:46.150 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2023-11-28T18:26:46.150 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2023-11-28T18:26:46.150 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2023-11-28T18:26:46.150 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2023-11-28T18:26:46.150 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2023-11-28T18:26:46.150 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2023-11-28T18:26:46.150 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2023-11-28T18:26:46.151 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2023-11-28T18:26:46.151 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2023-11-28T18:26:46.151 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2023-11-28T18:26:46.151 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2023-11-28T18:26:46.151 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2023-11-28T18:26:46.151 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2023-11-28T18:26:46.151 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2023-11-28T18:26:46.151 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2023-11-28T18:26:46.151 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2023-11-28T18:26:46.151 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2023-11-28T18:26:46.151 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 54, 2023-11-28T18:26:46.151 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2023-11-28T18:26:46.152 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 4, 2023-11-28T18:26:46.152 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 8, 2023-11-28T18:26:46.152 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 8, 2023-11-28T18:26:46.152 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 0, 2023-11-28T18:26:46.152 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2023-11-28T18:26:46.152 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 41, 2023-11-28T18:26:46.152 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 0, 2023-11-28T18:26:46.152 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 5 2023-11-28T18:26:46.152 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.152 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2023-11-28T18:26:46.152 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 1, 2023-11-28T18:26:46.152 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2023-11-28T18:26:46.152 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2023-11-28T18:26:46.153 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2023-11-28T18:26:46.153 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.153 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2023-11-28T18:26:46.153 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2023-11-28T18:26:46.153 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.153 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2023-11-28T18:26:46.153 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2023-11-28T18:26:46.153 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 10, 2023-11-28T18:26:46.153 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2023-11-28T18:26:46.153 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 10, 2023-11-28T18:26:46.153 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 77, 2023-11-28T18:26:46.153 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2023-11-28T18:26:46.154 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.154 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-28T18:26:46.154 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:46.154 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2023-11-28T18:26:46.154 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:46.154 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 1806, 2023-11-28T18:26:46.154 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 527543, 2023-11-28T18:26:46.154 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:46.154 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 1806, 2023-11-28T18:26:46.154 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2023-11-28T18:26:46.154 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2023-11-28T18:26:46.154 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 1806, 2023-11-28T18:26:46.154 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 527543, 2023-11-28T18:26:46.155 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:46.155 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.155 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.155 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.155 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:46.155 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.155 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2023-11-28T18:26:46.155 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:46.155 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2023-11-28T18:26:46.155 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:46.155 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2023-11-28T18:26:46.155 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2023-11-28T18:26:46.156 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:46.156 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2023-11-28T18:26:46.156 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 490, 2023-11-28T18:26:46.156 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 1381121, 2023-11-28T18:26:46.156 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 490, 2023-11-28T18:26:46.156 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 1381121, 2023-11-28T18:26:46.156 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:46.156 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.156 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.156 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.156 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:46.156 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.157 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2023-11-28T18:26:46.157 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:46.157 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2023-11-28T18:26:46.157 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:46.157 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2023-11-28T18:26:46.157 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2023-11-28T18:26:46.157 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:46.157 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2023-11-28T18:26:46.157 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 490, 2023-11-28T18:26:46.157 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 490, 2023-11-28T18:26:46.157 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 490, 2023-11-28T18:26:46.157 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 490, 2023-11-28T18:26:46.157 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:46.157 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.158 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.158 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.158 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:46.158 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.158 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2023-11-28T18:26:46.158 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:46.158 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2023-11-28T18:26:46.158 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:46.158 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 77, 2023-11-28T18:26:46.158 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 7777, 2023-11-28T18:26:46.158 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:46.158 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 77, 2023-11-28T18:26:46.159 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2023-11-28T18:26:46.159 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2023-11-28T18:26:46.159 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 3, 2023-11-28T18:26:46.159 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 7777, 2023-11-28T18:26:46.159 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:46.159 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.159 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.159 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.159 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:46.159 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:26:46.159 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x5607a9244a00": { 2023-11-28T18:26:46.159 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2023-11-28T18:26:46.159 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2023-11-28T18:26:46.160 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2023-11-28T18:26:46.160 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 626, 2023-11-28T18:26:46.160 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 1234334, 2023-11-28T18:26:46.160 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2023-11-28T18:26:46.160 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 626, 2023-11-28T18:26:46.160 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2023-11-28T18:26:46.160 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2023-11-28T18:26:46.160 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 345, 2023-11-28T18:26:46.160 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 1234334, 2023-11-28T18:26:46.160 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2023-11-28T18:26:46.160 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2023-11-28T18:26:46.160 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2023-11-28T18:26:46.160 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2023-11-28T18:26:46.161 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:46.161 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:26:46.161 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:26:46.162 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f35576640 1 -- 172.21.15.105:0/1744104367 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0f1c0668d0 msgr2=0x7f0f1c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:46.162 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f35576640 1 --2- 172.21.15.105:0/1744104367 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0f1c0668d0 0x7f0f1c068d90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0f200011f0 tx=0x7f0f20007d40 comp rx=0 tx=0).stop 2023-11-28T18:26:46.162 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f35576640 1 -- 172.21.15.105:0/1744104367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f300d0d40 msgr2=0x7f0f300ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:46.162 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f35576640 1 --2- 172.21.15.105:0/1744104367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f300d0d40 0x7f0f300ca7a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0f240af040 tx=0x7f0f24019160 comp rx=0 tx=0).stop 2023-11-28T18:26:46.162 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f35576640 1 -- 172.21.15.105:0/1744104367 shutdown_connections 2023-11-28T18:26:46.163 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f35576640 1 --2- 172.21.15.105:0/1744104367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f3015d1a0 0x7f0f300cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.163 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f35576640 1 --2- 172.21.15.105:0/1744104367 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0f1c0668d0 0x7f0f1c068d90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.163 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f35576640 1 --2- 172.21.15.105:0/1744104367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f300d0d40 0x7f0f300ca7a0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.163 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f35576640 1 --2- 172.21.15.105:0/1744104367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0f300d0450 0x7f0f30159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.163 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f35576640 1 -- 172.21.15.105:0/1744104367 >> 172.21.15.105:0/1744104367 conn(0x7f0f300d2e10 msgr2=0x7f0f300b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:46.163 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f35576640 1 -- 172.21.15.105:0/1744104367 shutdown_connections 2023-11-28T18:26:46.163 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.104+0000 7f0f35576640 1 -- 172.21.15.105:0/1744104367 wait complete. 2023-11-28T18:26:46.163 INFO:tasks.cephfs.test_exports:export stats: [(0, 5), (1, 0), (2, 0)] 2023-11-28T18:26:46.164 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T18:26:46.164 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:26:46.263 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:26:46.357 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T18:26:46.357 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:26:46.357 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:26:46.366 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:26:46.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.548+0000 7f3dfbe9a640 1 Processor -- start 2023-11-28T18:26:46.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.548+0000 7f3dfbe9a640 1 -- start start 2023-11-28T18:26:46.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.548+0000 7f3dfbe9a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3df40cdd80 0x7f3df406c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.548+0000 7f3dfbe9a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3df40ce230 0x7f3df40cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.548+0000 7f3dfbe9a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df40b3c40 con 0x7f3df415d320 2023-11-28T18:26:46.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.548+0000 7f3dfbe9a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3df406ca80 con 0x7f3df40ce230 2023-11-28T18:26:46.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.548+0000 7f3dfbe9a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df415d700 con 0x7f3df40cdd80 2023-11-28T18:26:46.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.548+0000 7f3dfa437640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3df40ce230 0x7f3df40cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.548+0000 7f3df9c36640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3df415d320 0x7f3df40cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46032/0 (socket says 172.21.15.105:46032) 2023-11-28T18:26:46.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.548+0000 7f3df9c36640 1 -- 172.21.15.105:0/2630336750 learned_addr learned my addr 172.21.15.105:0/2630336750 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:46.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.548+0000 7f3df9435640 1 --2- 172.21.15.105:0/2630336750 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3df40cdd80 0x7f3df406c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.548+0000 7f3dfa437640 1 -- 172.21.15.105:0/2630336750 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3df40cdd80 msgr2=0x7f3df406c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:46.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.548+0000 7f3dfa437640 1 --2- 172.21.15.105:0/2630336750 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3df40cdd80 0x7f3df406c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.548+0000 7f3dfa437640 1 -- 172.21.15.105:0/2630336750 >> v1:172.21.15.105:6789/0 conn(0x7f3df415d320 legacy=0x7f3df40cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:46.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.548+0000 7f3dfa437640 1 -- 172.21.15.105:0/2630336750 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df406c7a0 con 0x7f3df40ce230 2023-11-28T18:26:46.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.548+0000 7f3df8c34640 1 -- 172.21.15.105:0/2630336750 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3869633338 0 0) 0x7f3df40b3c40 con 0x7f3df415d320 2023-11-28T18:26:46.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.552+0000 7f3dfa437640 1 --2- 172.21.15.105:0/2630336750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df40ce230 0x7f3df40cc0c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3dec009d90 tx=0x7f3dec01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ff7b6d7604180a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:46.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.552+0000 7f3df8c34640 1 -- 172.21.15.105:0/2630336750 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec0a5980 con 0x7f3df40ce230 2023-11-28T18:26:46.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.552+0000 7f3df8c34640 1 -- 172.21.15.105:0/2630336750 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dec0a5ae0 con 0x7f3df40ce230 2023-11-28T18:26:46.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.552+0000 7f3df8c34640 1 -- 172.21.15.105:0/2630336750 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec0a5e70 con 0x7f3df40ce230 2023-11-28T18:26:46.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.552+0000 7f3dfbe9a640 1 -- 172.21.15.105:0/2630336750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df40ce230 msgr2=0x7f3df40cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:46.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.552+0000 7f3dfbe9a640 1 --2- 172.21.15.105:0/2630336750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df40ce230 0x7f3df40cc0c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3dec009d90 tx=0x7f3dec01c8c0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.552+0000 7f3dfbe9a640 1 -- 172.21.15.105:0/2630336750 shutdown_connections 2023-11-28T18:26:46.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.552+0000 7f3dfbe9a640 1 --2- 172.21.15.105:0/2630336750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df40ce230 0x7f3df40cc0c0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.552+0000 7f3dfbe9a640 1 --2- 172.21.15.105:0/2630336750 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3df40cdd80 0x7f3df406c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.552+0000 7f3dfbe9a640 1 -- 172.21.15.105:0/2630336750 >> 172.21.15.105:0/2630336750 conn(0x7f3df40d0df0 msgr2=0x7f3df40d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:46.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.552+0000 7f3dfbe9a640 1 -- 172.21.15.105:0/2630336750 shutdown_connections 2023-11-28T18:26:46.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.552+0000 7f3dfbe9a640 1 -- 172.21.15.105:0/2630336750 wait complete. 2023-11-28T18:26:46.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3dfbe9a640 1 Processor -- start 2023-11-28T18:26:46.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3dfbe9a640 1 -- start start 2023-11-28T18:26:46.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3dfbe9a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3df40cdd80 0x7f3df40b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3dfbe9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df40ce230 0x7f3df41592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3dfbe9a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3df415d320 0x7f3df4154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3dfbe9a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df40b5720 con 0x7f3df40ce230 2023-11-28T18:26:46.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3dfbe9a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41554f0 con 0x7f3df40cdd80 2023-11-28T18:26:46.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3dfbe9a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df40b4140 con 0x7f3df415d320 2023-11-28T18:26:46.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3df9435640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df40ce230 0x7f3df41592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3df9435640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df40ce230 0x7f3df41592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42294/0 (socket says 172.21.15.105:42294) 2023-11-28T18:26:46.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3df9435640 1 -- 172.21.15.105:0/2573481277 learned_addr learned my addr 172.21.15.105:0/2573481277 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:46.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3df9c36640 1 --2- 172.21.15.105:0/2573481277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3df40cdd80 0x7f3df40b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3dfa437640 1 --2- 172.21.15.105:0/2573481277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3df415d320 0x7f3df4154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3df9435640 1 -- 172.21.15.105:0/2573481277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3df415d320 msgr2=0x7f3df4154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:46.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3df9435640 1 --2- 172.21.15.105:0/2573481277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3df415d320 0x7f3df4154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3df9435640 1 -- 172.21.15.105:0/2573481277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3df40cdd80 msgr2=0x7f3df40b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:46.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3df9435640 1 --2- 172.21.15.105:0/2573481277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3df40cdd80 0x7f3df40b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3df9435640 1 -- 172.21.15.105:0/2573481277 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec009a70 con 0x7f3df40ce230 2023-11-28T18:26:46.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3df9435640 1 --2- 172.21.15.105:0/2573481277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df40ce230 0x7f3df41592d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3de400d830 tx=0x7f3de400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:46.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3dea7fc640 1 -- 172.21.15.105:0/2573481277 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de400b6c0 con 0x7f3df40ce230 2023-11-28T18:26:46.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3dea7fc640 1 -- 172.21.15.105:0/2573481277 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de400b820 con 0x7f3df40ce230 2023-11-28T18:26:46.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3dfbe9a640 1 -- 172.21.15.105:0/2573481277 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df40b4420 con 0x7f3df40ce230 2023-11-28T18:26:46.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3dea7fc640 1 -- 172.21.15.105:0/2573481277 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de400bb90 con 0x7f3df40ce230 2023-11-28T18:26:46.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3dfbe9a640 1 -- 172.21.15.105:0/2573481277 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df40b49e0 con 0x7f3df40ce230 2023-11-28T18:26:46.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.556+0000 7f3dfbe9a640 1 -- 172.21.15.105:0/2573481277 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df4067570 con 0x7f3df40ce230 2023-11-28T18:26:46.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.560+0000 7f3dea7fc640 1 -- 172.21.15.105:0/2573481277 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3de401e460 con 0x7f3df40ce230 2023-11-28T18:26:46.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.560+0000 7f3dea7fc640 1 --2- 172.21.15.105:0/2573481277 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3dd80668d0 0x7f3dd8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.560+0000 7f3dea7fc640 1 -- 172.21.15.105:0/2573481277 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f3de4084e50 con 0x7f3df40ce230 2023-11-28T18:26:46.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.560+0000 7f3df9c36640 1 --2- 172.21.15.105:0/2573481277 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3dd80668d0 0x7f3dd8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.560+0000 7f3df9c36640 1 --2- 172.21.15.105:0/2573481277 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3dd80668d0 0x7f3dd8068d90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3df0000f70 tx=0x7f3df0007c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:46.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.564+0000 7f3dea7fc640 1 -- 172.21.15.105:0/2573481277 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3de4051e20 con 0x7f3df40ce230 2023-11-28T18:26:46.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.712+0000 7f3dfbe9a640 1 -- 172.21.15.105:0/2573481277 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3df40d0910 con 0x7f3df40ce230 2023-11-28T18:26:46.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.716+0000 7f3dea7fc640 1 -- 172.21.15.105:0/2573481277 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+17059 (secure 0 0 0) 0x7f3de4055440 con 0x7f3df40ce230 2023-11-28T18:26:46.718 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:46.718 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":54,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:26:28.339225+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:25:58.881270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T18:25:59.532434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.39","mappings":[{"from":0,"to":6}]},{"pgid":"8.3d","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6835/897997046":"2023-11-29T18:26:28.339203+0000","172.21.15.148:6833/897997046":"2023-11-29T18:26:28.339203+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:26:46.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.716+0000 7f3dfbe9a640 1 -- 172.21.15.105:0/2573481277 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3dd80668d0 msgr2=0x7f3dd8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:46.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.716+0000 7f3dfbe9a640 1 --2- 172.21.15.105:0/2573481277 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3dd80668d0 0x7f3dd8068d90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3df0000f70 tx=0x7f3df0007c10 comp rx=0 tx=0).stop 2023-11-28T18:26:46.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.716+0000 7f3dfbe9a640 1 -- 172.21.15.105:0/2573481277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df40ce230 msgr2=0x7f3df41592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:46.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.716+0000 7f3dfbe9a640 1 --2- 172.21.15.105:0/2573481277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df40ce230 0x7f3df41592d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3de400d830 tx=0x7f3de400dcf0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.716+0000 7f3dfbe9a640 1 -- 172.21.15.105:0/2573481277 shutdown_connections 2023-11-28T18:26:46.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.716+0000 7f3dfbe9a640 1 --2- 172.21.15.105:0/2573481277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3df415d320 0x7f3df4154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.716+0000 7f3dfbe9a640 1 --2- 172.21.15.105:0/2573481277 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3dd80668d0 0x7f3dd8068d90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.716+0000 7f3dfbe9a640 1 --2- 172.21.15.105:0/2573481277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df40ce230 0x7f3df41592d0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.716+0000 7f3dfbe9a640 1 --2- 172.21.15.105:0/2573481277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3df40cdd80 0x7f3df40b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.716+0000 7f3dfbe9a640 1 -- 172.21.15.105:0/2573481277 >> 172.21.15.105:0/2573481277 conn(0x7f3df40d0df0 msgr2=0x7f3df40cf7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:46.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.716+0000 7f3dfbe9a640 1 -- 172.21.15.105:0/2573481277 shutdown_connections 2023-11-28T18:26:46.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.716+0000 7f3dfbe9a640 1 -- 172.21.15.105:0/2573481277 wait complete. 2023-11-28T18:26:46.739 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:25:58.881270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:25:59.532434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:26:46.739 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:26:46.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.876+0000 7fe1bf5f8640 1 Processor -- start 2023-11-28T18:26:46.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.876+0000 7fe1bf5f8640 1 -- start start 2023-11-28T18:26:46.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.880+0000 7fe1bf5f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe1b80b4290 0x7fe1b80b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.880+0000 7fe1bf5f8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe1b8071960 0x7fe1b815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.880+0000 7fe1bf5f8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b80b4bb0 con 0x7fe1b815b030 2023-11-28T18:26:46.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.880+0000 7fe1bf5f8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b80701d0 con 0x7fe1b8071960 2023-11-28T18:26:46.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.880+0000 7fe1bf5f8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b806df80 con 0x7fe1b80b4290 2023-11-28T18:26:46.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.880+0000 7fe1bcb93640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe1b8071960 0x7fe1b815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.880+0000 7fe1bcb93640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe1b8071960 0x7fe1b815f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42300/0 (socket says 172.21.15.105:42300) 2023-11-28T18:26:46.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.880+0000 7fe1bdb95640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe1b815b030 0x7fe1b80709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46042/0 (socket says 172.21.15.105:46042) 2023-11-28T18:26:46.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.880+0000 7fe1bcb93640 1 -- 172.21.15.105:0/4288485454 learned_addr learned my addr 172.21.15.105:0/4288485454 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:46.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.880+0000 7fe1bd394640 1 --2- 172.21.15.105:0/4288485454 >> v2:172.21.15.148:3300/0 conn(0x7fe1b80b4290 0x7fe1b80b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.880+0000 7fe1bcb93640 1 -- 172.21.15.105:0/4288485454 >> v2:172.21.15.148:3300/0 conn(0x7fe1b80b4290 msgr2=0x7fe1b80b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:46.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.880+0000 7fe1bcb93640 1 --2- 172.21.15.105:0/4288485454 >> v2:172.21.15.148:3300/0 conn(0x7fe1b80b4290 0x7fe1b80b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.880+0000 7fe1bcb93640 1 -- 172.21.15.105:0/4288485454 >> v1:172.21.15.105:6789/0 conn(0x7fe1b815b030 legacy=0x7fe1b80709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:46.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.880+0000 7fe1bcb93640 1 -- 172.21.15.105:0/4288485454 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b8071df0 con 0x7fe1b8071960 2023-11-28T18:26:46.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.880+0000 7fe1affff640 1 -- 172.21.15.105:0/4288485454 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482237001 0 0) 0x7fe1b80b4bb0 con 0x7fe1b815b030 2023-11-28T18:26:46.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.880+0000 7fe1bd394640 1 --2- 172.21.15.105:0/4288485454 >> v2:172.21.15.148:3300/0 conn(0x7fe1b80b4290 0x7fe1b80b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:46.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.884+0000 7fe1bcb93640 1 --2- 172.21.15.105:0/4288485454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b8071960 0x7fe1b815f3b0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8073a70 tx=0x7fe1a801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cdff97d756abca4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:46.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.884+0000 7fe1affff640 1 -- 172.21.15.105:0/4288485454 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a80aa650 con 0x7fe1b8071960 2023-11-28T18:26:46.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.884+0000 7fe1affff640 1 -- 172.21.15.105:0/4288485454 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a80aa7b0 con 0x7fe1b8071960 2023-11-28T18:26:46.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.884+0000 7fe1affff640 1 -- 172.21.15.105:0/4288485454 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a80aab40 con 0x7fe1b8071960 2023-11-28T18:26:46.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.884+0000 7fe1bf5f8640 1 -- 172.21.15.105:0/4288485454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b8071960 msgr2=0x7fe1b815f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:46.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.884+0000 7fe1bf5f8640 1 --2- 172.21.15.105:0/4288485454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b8071960 0x7fe1b815f3b0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8073a70 tx=0x7fe1a801c4f0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.884+0000 7fe1bf5f8640 1 -- 172.21.15.105:0/4288485454 shutdown_connections 2023-11-28T18:26:46.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.884+0000 7fe1bf5f8640 1 --2- 172.21.15.105:0/4288485454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b8071960 0x7fe1b815f3b0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.884+0000 7fe1bf5f8640 1 --2- 172.21.15.105:0/4288485454 >> v2:172.21.15.148:3300/0 conn(0x7fe1b80b4290 0x7fe1b80b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.884+0000 7fe1bf5f8640 1 -- 172.21.15.105:0/4288485454 >> 172.21.15.105:0/4288485454 conn(0x7fe1b80ceb90 msgr2=0x7fe1b80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:46.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.884+0000 7fe1bf5f8640 1 -- 172.21.15.105:0/4288485454 shutdown_connections 2023-11-28T18:26:46.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.884+0000 7fe1bf5f8640 1 -- 172.21.15.105:0/4288485454 wait complete. 2023-11-28T18:26:46.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.884+0000 7fe1bf5f8640 1 Processor -- start 2023-11-28T18:26:46.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.884+0000 7fe1bf5f8640 1 -- start start 2023-11-28T18:26:46.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1bf5f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1b80b4290 0x7fe1b80b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1bf5f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1b815b030 0x7fe1b8159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1bf5f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b81596d0 0x7fe1b814bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1bf5f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b8159b80 con 0x7fe1b81596d0 2023-11-28T18:26:46.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1bf5f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b80b3a60 con 0x7fe1b80b4290 2023-11-28T18:26:46.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1bf5f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b80b2f40 con 0x7fe1b815b030 2023-11-28T18:26:46.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1bdb95640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b81596d0 0x7fe1b814bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1bdb95640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b81596d0 0x7fe1b814bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42304/0 (socket says 172.21.15.105:42304) 2023-11-28T18:26:46.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1bdb95640 1 -- 172.21.15.105:0/2477209951 learned_addr learned my addr 172.21.15.105:0/2477209951 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:46.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1bd394640 1 --2- 172.21.15.105:0/2477209951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1b80b4290 0x7fe1b80b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1bcb93640 1 --2- 172.21.15.105:0/2477209951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1b815b030 0x7fe1b8159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1bdb95640 1 -- 172.21.15.105:0/2477209951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1b815b030 msgr2=0x7fe1b8159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:46.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1bdb95640 1 --2- 172.21.15.105:0/2477209951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1b815b030 0x7fe1b8159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1bdb95640 1 -- 172.21.15.105:0/2477209951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1b80b4290 msgr2=0x7fe1b80b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:46.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1bdb95640 1 --2- 172.21.15.105:0/2477209951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1b80b4290 0x7fe1b80b2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:46.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1bdb95640 1 -- 172.21.15.105:0/2477209951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a80094d0 con 0x7fe1b81596d0 2023-11-28T18:26:46.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1bcb93640 1 --2- 172.21.15.105:0/2477209951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1b815b030 0x7fe1b8159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:46.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1bdb95640 1 --2- 172.21.15.105:0/2477209951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b81596d0 0x7fe1b814bd90 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000c7b0 tx=0x7fe1b000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:46.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1adffb640 1 -- 172.21.15.105:0/2477209951 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b000ce60 con 0x7fe1b81596d0 2023-11-28T18:26:46.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1adffb640 1 -- 172.21.15.105:0/2477209951 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b000a640 con 0x7fe1b81596d0 2023-11-28T18:26:46.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1bf5f8640 1 -- 172.21.15.105:0/2477209951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b80b31c0 con 0x7fe1b81596d0 2023-11-28T18:26:46.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1adffb640 1 -- 172.21.15.105:0/2477209951 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b000a9f0 con 0x7fe1b81596d0 2023-11-28T18:26:46.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1bf5f8640 1 -- 172.21.15.105:0/2477209951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b80b3780 con 0x7fe1b81596d0 2023-11-28T18:26:46.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1bf5f8640 1 -- 172.21.15.105:0/2477209951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b8067570 con 0x7fe1b81596d0 2023-11-28T18:26:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.888+0000 7fe1adffb640 1 -- 172.21.15.105:0/2477209951 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fe1b000ab50 con 0x7fe1b81596d0 2023-11-28T18:26:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.892+0000 7fe1adffb640 1 --2- 172.21.15.105:0/2477209951 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe19c066900 0x7fe19c068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.892+0000 7fe1adffb640 1 -- 172.21.15.105:0/2477209951 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7fe1b0083cb0 con 0x7fe1b81596d0 2023-11-28T18:26:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.892+0000 7fe1bd394640 1 --2- 172.21.15.105:0/2477209951 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe19c066900 0x7fe19c068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:46.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.892+0000 7fe1bd394640 1 --2- 172.21.15.105:0/2477209951 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe19c066900 0x7fe19c068dc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe1b809bde0 tx=0x7fe1b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:46.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:46.896+0000 7fe1adffb640 1 -- 172.21.15.105:0/2477209951 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe1b0050dc0 con 0x7fe1b81596d0 2023-11-28T18:26:47.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.044+0000 7fe1bf5f8640 1 -- 172.21.15.105:0/2477209951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe1b80cf7f0 con 0x7fe1b81596d0 2023-11-28T18:26:47.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.044+0000 7fe1adffb640 1 -- 172.21.15.105:0/2477209951 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe1b0018020 con 0x7fe1b81596d0 2023-11-28T18:26:47.047 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:26:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.044+0000 7fe1bf5f8640 1 -- 172.21.15.105:0/2477209951 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe19c066900 msgr2=0x7fe19c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.044+0000 7fe1bf5f8640 1 --2- 172.21.15.105:0/2477209951 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe19c066900 0x7fe19c068dc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe1b809bde0 tx=0x7fe1b4008040 comp rx=0 tx=0).stop 2023-11-28T18:26:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.044+0000 7fe1bf5f8640 1 -- 172.21.15.105:0/2477209951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b81596d0 msgr2=0x7fe1b814bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.044+0000 7fe1bf5f8640 1 --2- 172.21.15.105:0/2477209951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b81596d0 0x7fe1b814bd90 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000c7b0 tx=0x7fe1b000cc70 comp rx=0 tx=0).stop 2023-11-28T18:26:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.044+0000 7fe1bf5f8640 1 -- 172.21.15.105:0/2477209951 shutdown_connections 2023-11-28T18:26:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.044+0000 7fe1bf5f8640 1 --2- 172.21.15.105:0/2477209951 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe19c066900 0x7fe19c068dc0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.044+0000 7fe1bf5f8640 1 --2- 172.21.15.105:0/2477209951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b81596d0 0x7fe1b814bd90 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.044+0000 7fe1bf5f8640 1 --2- 172.21.15.105:0/2477209951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1b815b030 0x7fe1b8159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.044+0000 7fe1bf5f8640 1 --2- 172.21.15.105:0/2477209951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1b80b4290 0x7fe1b80b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.044+0000 7fe1bf5f8640 1 -- 172.21.15.105:0/2477209951 >> 172.21.15.105:0/2477209951 conn(0x7fe1b80ceb90 msgr2=0x7fe1b80cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:47.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.044+0000 7fe1bf5f8640 1 -- 172.21.15.105:0/2477209951 shutdown_connections 2023-11-28T18:26:47.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.044+0000 7fe1bf5f8640 1 -- 172.21.15.105:0/2477209951 wait complete. 2023-11-28T18:26:47.068 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:26:47.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.204+0000 7f0821609640 1 Processor -- start 2023-11-28T18:26:47.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.204+0000 7f0821609640 1 -- start start 2023-11-28T18:26:47.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.208+0000 7f0821609640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f081c15d4f0 0x7f081c0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:47.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.208+0000 7f0821609640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f081c0b2d20 0x7f081c0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:47.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.208+0000 7f0821609640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081c0b25b0 con 0x7f081c0d3c30 2023-11-28T18:26:47.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.208+0000 7f0821609640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f081c0b2900 con 0x7f081c15d4f0 2023-11-28T18:26:47.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.208+0000 7f0821609640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081c0d0a20 con 0x7f081c0b2d20 2023-11-28T18:26:47.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.208+0000 7f081affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f081c15d4f0 0x7f081c0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:47.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.208+0000 7f081a7fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f081c0d3c30 0x7f081c0d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46048/0 (socket says 172.21.15.105:46048) 2023-11-28T18:26:47.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.208+0000 7f081a7fc640 1 -- 172.21.15.105:0/2086852793 learned_addr learned my addr 172.21.15.105:0/2086852793 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:47.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.208+0000 7f081b7fe640 1 --2- 172.21.15.105:0/2086852793 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f081c0b2d20 0x7f081c0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:47.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.208+0000 7f081affd640 1 -- 172.21.15.105:0/2086852793 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f081c0b2d20 msgr2=0x7f081c0b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:47.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.208+0000 7f081affd640 1 --2- 172.21.15.105:0/2086852793 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f081c0b2d20 0x7f081c0b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.208+0000 7f081affd640 1 -- 172.21.15.105:0/2086852793 >> v1:172.21.15.105:6789/0 conn(0x7f081c0d3c30 legacy=0x7f081c0d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:47.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.208+0000 7f081affd640 1 -- 172.21.15.105:0/2086852793 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081c0cea30 con 0x7f081c15d4f0 2023-11-28T18:26:47.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.208+0000 7f0819ffb640 1 -- 172.21.15.105:0/2086852793 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468414732 0 0) 0x7f081c0b25b0 con 0x7f081c0d3c30 2023-11-28T18:26:47.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.208+0000 7f081b7fe640 1 --2- 172.21.15.105:0/2086852793 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f081c0b2d20 0x7f081c0b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:47.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.212+0000 7f081affd640 1 --2- 172.21.15.105:0/2086852793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f081c15d4f0 0x7f081c0d0e10 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f080c0097c0 tx=0x7f080c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3634b65ca4900f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:47.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.212+0000 7f0819ffb640 1 -- 172.21.15.105:0/2086852793 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c0ac620 con 0x7f081c15d4f0 2023-11-28T18:26:47.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.212+0000 7f0819ffb640 1 -- 172.21.15.105:0/2086852793 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080c0ac780 con 0x7f081c15d4f0 2023-11-28T18:26:47.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.212+0000 7f0819ffb640 1 -- 172.21.15.105:0/2086852793 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c0acb10 con 0x7f081c15d4f0 2023-11-28T18:26:47.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.212+0000 7f0821609640 1 -- 172.21.15.105:0/2086852793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f081c15d4f0 msgr2=0x7f081c0d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:47.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.212+0000 7f0821609640 1 --2- 172.21.15.105:0/2086852793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f081c15d4f0 0x7f081c0d0e10 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f080c0097c0 tx=0x7f080c01c3c0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.212+0000 7f0821609640 1 -- 172.21.15.105:0/2086852793 shutdown_connections 2023-11-28T18:26:47.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.212+0000 7f0821609640 1 --2- 172.21.15.105:0/2086852793 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f081c0b2d20 0x7f081c0b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.212+0000 7f0821609640 1 --2- 172.21.15.105:0/2086852793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f081c15d4f0 0x7f081c0d0e10 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.212+0000 7f0821609640 1 -- 172.21.15.105:0/2086852793 >> 172.21.15.105:0/2086852793 conn(0x7f081c0d3030 msgr2=0x7f081c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:47.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.212+0000 7f0821609640 1 -- 172.21.15.105:0/2086852793 shutdown_connections 2023-11-28T18:26:47.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.212+0000 7f0821609640 1 -- 172.21.15.105:0/2086852793 wait complete. 2023-11-28T18:26:47.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.212+0000 7f0821609640 1 Processor -- start 2023-11-28T18:26:47.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.212+0000 7f0821609640 1 -- start start 2023-11-28T18:26:47.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f0821609640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f081c0b2d20 0x7f081c15bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:47.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f0821609640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f081c0d3c30 0x7f081c151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:47.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f0821609640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f081c15d4f0 0x7f081c14cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:47.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f0821609640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c154470 con 0x7f081c15d4f0 2023-11-28T18:26:47.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f0821609640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c153eb0 con 0x7f081c0d3c30 2023-11-28T18:26:47.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f0821609640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081c154190 con 0x7f081c0b2d20 2023-11-28T18:26:47.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f081b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f081c15d4f0 0x7f081c14cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:47.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f081b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f081c15d4f0 0x7f081c14cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42310/0 (socket says 172.21.15.105:42310) 2023-11-28T18:26:47.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f081b7fe640 1 -- 172.21.15.105:0/3686609747 learned_addr learned my addr 172.21.15.105:0/3686609747 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:47.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f081a7fc640 1 --2- 172.21.15.105:0/3686609747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f081c0d3c30 0x7f081c151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:47.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f081affd640 1 --2- 172.21.15.105:0/3686609747 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f081c0b2d20 0x7f081c15bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:47.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f081b7fe640 1 -- 172.21.15.105:0/3686609747 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f081c0b2d20 msgr2=0x7f081c15bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f081b7fe640 1 --2- 172.21.15.105:0/3686609747 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f081c0b2d20 0x7f081c15bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f081b7fe640 1 -- 172.21.15.105:0/3686609747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f081c0d3c30 msgr2=0x7f081c151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f081b7fe640 1 --2- 172.21.15.105:0/3686609747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f081c0d3c30 0x7f081c151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f081b7fe640 1 -- 172.21.15.105:0/3686609747 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f080c009420 con 0x7f081c15d4f0 2023-11-28T18:26:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f081b7fe640 1 --2- 172.21.15.105:0/3686609747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f081c15d4f0 0x7f081c14cf80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f081400cd50 tx=0x7f081400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f08037fe640 1 -- 172.21.15.105:0/3686609747 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081400f680 con 0x7f081c15d4f0 2023-11-28T18:26:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f08037fe640 1 -- 172.21.15.105:0/3686609747 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081400f800 con 0x7f081c15d4f0 2023-11-28T18:26:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f0821609640 1 -- 172.21.15.105:0/3686609747 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081c06bd40 con 0x7f081c15d4f0 2023-11-28T18:26:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f08037fe640 1 -- 172.21.15.105:0/3686609747 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081400fbb0 con 0x7f081c15d4f0 2023-11-28T18:26:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f0821609640 1 -- 172.21.15.105:0/3686609747 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f081c15c100 con 0x7f081c15d4f0 2023-11-28T18:26:47.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f0821609640 1 -- 172.21.15.105:0/3686609747 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f081c06bf60 con 0x7f081c15d4f0 2023-11-28T18:26:47.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.216+0000 7f08037fe640 1 -- 172.21.15.105:0/3686609747 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f081400b040 con 0x7f081c15d4f0 2023-11-28T18:26:47.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.220+0000 7f08037fe640 1 --2- 172.21.15.105:0/3686609747 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f08080668d0 0x7f0808068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:47.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.220+0000 7f08037fe640 1 -- 172.21.15.105:0/3686609747 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f0814084f50 con 0x7f081c15d4f0 2023-11-28T18:26:47.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.220+0000 7f081affd640 1 --2- 172.21.15.105:0/3686609747 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f08080668d0 0x7f0808068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:47.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.220+0000 7f081affd640 1 --2- 172.21.15.105:0/3686609747 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f08080668d0 0x7f0808068d90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f080c0ae000 tx=0x7f080c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.224+0000 7f08037fe640 1 -- 172.21.15.105:0/3686609747 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0814052070 con 0x7f081c15d4f0 2023-11-28T18:26:47.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.376+0000 7f0821609640 1 -- 172.21.15.105:0/3686609747 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f081c067570 con 0x7f081c15d4f0 2023-11-28T18:26:47.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.376+0000 7f08037fe640 1 -- 172.21.15.105:0/3686609747 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7f081401d0a0 con 0x7f081c15d4f0 2023-11-28T18:26:47.381 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:26:47.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.380+0000 7f0821609640 1 -- 172.21.15.105:0/3686609747 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f08080668d0 msgr2=0x7f0808068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:47.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.380+0000 7f0821609640 1 --2- 172.21.15.105:0/3686609747 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f08080668d0 0x7f0808068d90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f080c0ae000 tx=0x7f080c008040 comp rx=0 tx=0).stop 2023-11-28T18:26:47.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.380+0000 7f0821609640 1 -- 172.21.15.105:0/3686609747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f081c15d4f0 msgr2=0x7f081c14cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:47.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.380+0000 7f0821609640 1 --2- 172.21.15.105:0/3686609747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f081c15d4f0 0x7f081c14cf80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f081400cd50 tx=0x7f081400a650 comp rx=0 tx=0).stop 2023-11-28T18:26:47.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.380+0000 7f0821609640 1 -- 172.21.15.105:0/3686609747 shutdown_connections 2023-11-28T18:26:47.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.380+0000 7f0821609640 1 --2- 172.21.15.105:0/3686609747 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f08080668d0 0x7f0808068d90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.380+0000 7f0821609640 1 --2- 172.21.15.105:0/3686609747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f081c15d4f0 0x7f081c14cf80 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.380+0000 7f0821609640 1 --2- 172.21.15.105:0/3686609747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f081c0d3c30 0x7f081c151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.380+0000 7f0821609640 1 --2- 172.21.15.105:0/3686609747 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f081c0b2d20 0x7f081c15bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.380+0000 7f0821609640 1 -- 172.21.15.105:0/3686609747 >> 172.21.15.105:0/3686609747 conn(0x7f081c0d3030 msgr2=0x7f081c0afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:47.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.380+0000 7f0821609640 1 -- 172.21.15.105:0/3686609747 shutdown_connections 2023-11-28T18:26:47.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.380+0000 7f0821609640 1 -- 172.21.15.105:0/3686609747 wait complete. 2023-11-28T18:26:47.402 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:26:47.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.540+0000 7ff127de6640 1 Processor -- start 2023-11-28T18:26:47.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.540+0000 7ff127de6640 1 -- start start 2023-11-28T18:26:47.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.540+0000 7ff127de6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff120154070 0x7ff12015d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:47.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.540+0000 7ff127de6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1200cc7f0 con 0x7ff1200adb80 2023-11-28T18:26:47.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.540+0000 7ff127de6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12015dd10 con 0x7ff12006bf50 2023-11-28T18:26:47.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.540+0000 7ff127de6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff120072540 con 0x7ff120154070 2023-11-28T18:26:47.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.540+0000 7ff126383640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff1200adb80 0x7ff1200acd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46056/0 (socket says 172.21.15.105:46056) 2023-11-28T18:26:47.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.540+0000 7ff126383640 1 -- 172.21.15.105:0/1604376241 learned_addr learned my addr 172.21.15.105:0/1604376241 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:47.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.540+0000 7ff125b82640 1 --2- 172.21.15.105:0/1604376241 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff120154070 0x7ff12015d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:47.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.540+0000 7ff124b80640 1 -- 172.21.15.105:0/1604376241 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 869750624 0 0) 0x7ff1200cc7f0 con 0x7ff1200adb80 2023-11-28T18:26:47.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.540+0000 7ff124b80640 1 -- 172.21.15.105:0/1604376241 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff10c003530 con 0x7ff1200adb80 2023-11-28T18:26:47.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.540+0000 7ff124b80640 1 -- 172.21.15.105:0/1604376241 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1980830335 0 0) 0x7ff12015dd10 con 0x7ff12006bf50 2023-11-28T18:26:47.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.540+0000 7ff124b80640 1 -- 172.21.15.105:0/1604376241 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1200cc7f0 con 0x7ff12006bf50 2023-11-28T18:26:47.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.540+0000 7ff125b82640 1 -- 172.21.15.105:0/1604376241 >> v1:172.21.15.148:6789/0 conn(0x7ff12006bf50 legacy=0x7ff1200ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:47.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.540+0000 7ff125b82640 1 -- 172.21.15.105:0/1604376241 >> v1:172.21.15.105:6789/0 conn(0x7ff1200adb80 legacy=0x7ff1200acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:47.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.540+0000 7ff125b82640 1 -- 172.21.15.105:0/1604376241 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12015dff0 con 0x7ff120154070 2023-11-28T18:26:47.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.544+0000 7ff125b82640 1 --2- 172.21.15.105:0/1604376241 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff120154070 0x7ff12015d700 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff11c0097f0 tx=0x7ff11c01c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a9975a597920c0d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:47.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.544+0000 7ff124b80640 1 -- 172.21.15.105:0/1604376241 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c0aa5f0 con 0x7ff120154070 2023-11-28T18:26:47.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.544+0000 7ff124b80640 1 -- 172.21.15.105:0/1604376241 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11c0aa750 con 0x7ff120154070 2023-11-28T18:26:47.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.544+0000 7ff124b80640 1 -- 172.21.15.105:0/1604376241 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c0aaae0 con 0x7ff120154070 2023-11-28T18:26:47.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.544+0000 7ff127de6640 1 -- 172.21.15.105:0/1604376241 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff120154070 msgr2=0x7ff12015d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:47.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.544+0000 7ff127de6640 1 --2- 172.21.15.105:0/1604376241 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff120154070 0x7ff12015d700 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff11c0097f0 tx=0x7ff11c01c490 comp rx=0 tx=0).stop 2023-11-28T18:26:47.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.544+0000 7ff127de6640 1 -- 172.21.15.105:0/1604376241 shutdown_connections 2023-11-28T18:26:47.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.544+0000 7ff127de6640 1 --2- 172.21.15.105:0/1604376241 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff120154070 0x7ff12015d700 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.544+0000 7ff127de6640 1 -- 172.21.15.105:0/1604376241 >> 172.21.15.105:0/1604376241 conn(0x7ff1200ceb90 msgr2=0x7ff1200cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:47.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.544+0000 7ff127de6640 1 -- 172.21.15.105:0/1604376241 shutdown_connections 2023-11-28T18:26:47.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.544+0000 7ff127de6640 1 -- 172.21.15.105:0/1604376241 wait complete. 2023-11-28T18:26:47.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff127de6640 1 Processor -- start 2023-11-28T18:26:47.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff127de6640 1 -- start start 2023-11-28T18:26:47.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff127de6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff12006bf50 0x7ff1200ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:47.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff127de6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1200adb80 0x7ff12009b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:47.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff127de6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff120154070 0x7ff12015fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:47.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff127de6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12009ac70 con 0x7ff120154070 2023-11-28T18:26:47.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff127de6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1200c8b90 con 0x7ff1200adb80 2023-11-28T18:26:47.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff127de6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12009aec0 con 0x7ff12006bf50 2023-11-28T18:26:47.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff126383640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff120154070 0x7ff12015fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:47.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff125b82640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff12006bf50 0x7ff1200ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:47.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff125381640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1200adb80 0x7ff12009b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:47.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff126383640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff120154070 0x7ff12015fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42324/0 (socket says 172.21.15.105:42324) 2023-11-28T18:26:47.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff126383640 1 -- 172.21.15.105:0/2129553530 learned_addr learned my addr 172.21.15.105:0/2129553530 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:47.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff126383640 1 -- 172.21.15.105:0/2129553530 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff12006bf50 msgr2=0x7ff1200ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:47.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff126383640 1 --2- 172.21.15.105:0/2129553530 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff12006bf50 0x7ff1200ca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff126383640 1 -- 172.21.15.105:0/2129553530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1200adb80 msgr2=0x7ff12009b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:47.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff126383640 1 --2- 172.21.15.105:0/2129553530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1200adb80 0x7ff12009b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff126383640 1 -- 172.21.15.105:0/2129553530 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11c009420 con 0x7ff120154070 2023-11-28T18:26:47.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff126383640 1 --2- 172.21.15.105:0/2129553530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff120154070 0x7ff12015fd20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7ff11800cb40 tx=0x7ff11800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:47.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff1167fc640 1 -- 172.21.15.105:0/2129553530 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11800aca0 con 0x7ff120154070 2023-11-28T18:26:47.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff1167fc640 1 -- 172.21.15.105:0/2129553530 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11800ae20 con 0x7ff120154070 2023-11-28T18:26:47.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff127de6640 1 -- 172.21.15.105:0/2129553530 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12009b1a0 con 0x7ff120154070 2023-11-28T18:26:47.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff1167fc640 1 -- 172.21.15.105:0/2129553530 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118012640 con 0x7ff120154070 2023-11-28T18:26:47.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.548+0000 7ff127de6640 1 -- 172.21.15.105:0/2129553530 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1200caaa0 con 0x7ff120154070 2023-11-28T18:26:47.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.552+0000 7ff127de6640 1 -- 172.21.15.105:0/2129553530 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff120067570 con 0x7ff120154070 2023-11-28T18:26:47.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.552+0000 7ff1167fc640 1 -- 172.21.15.105:0/2129553530 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff118007070 con 0x7ff120154070 2023-11-28T18:26:47.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.552+0000 7ff1167fc640 1 --2- 172.21.15.105:0/2129553530 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff1040668d0 0x7ff104068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:47.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.552+0000 7ff1167fc640 1 -- 172.21.15.105:0/2129553530 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7ff118084570 con 0x7ff120154070 2023-11-28T18:26:47.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.552+0000 7ff125b82640 1 --2- 172.21.15.105:0/2129553530 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff1040668d0 0x7ff104068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:47.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.552+0000 7ff125b82640 1 --2- 172.21.15.105:0/2129553530 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff1040668d0 0x7ff104068d90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff11c0ae000 tx=0x7ff11c0b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:47.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.556+0000 7ff1167fc640 1 -- 172.21.15.105:0/2129553530 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff118051680 con 0x7ff120154070 2023-11-28T18:26:47.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.704+0000 7ff127de6640 1 -- 172.21.15.105:0/2129553530 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff1200cf950 con 0x7ff120154070 2023-11-28T18:26:47.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.704+0000 7ff1167fc640 1 -- 172.21.15.105:0/2129553530 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff118054ca0 con 0x7ff120154070 2023-11-28T18:26:47.708 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:26:47.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.708+0000 7ff127de6640 1 -- 172.21.15.105:0/2129553530 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff1040668d0 msgr2=0x7ff104068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:47.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.708+0000 7ff127de6640 1 --2- 172.21.15.105:0/2129553530 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff1040668d0 0x7ff104068d90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff11c0ae000 tx=0x7ff11c0b5040 comp rx=0 tx=0).stop 2023-11-28T18:26:47.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.708+0000 7ff127de6640 1 -- 172.21.15.105:0/2129553530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff120154070 msgr2=0x7ff12015fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:47.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.708+0000 7ff127de6640 1 --2- 172.21.15.105:0/2129553530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff120154070 0x7ff12015fd20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7ff11800cb40 tx=0x7ff11800a060 comp rx=0 tx=0).stop 2023-11-28T18:26:47.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.708+0000 7ff127de6640 1 -- 172.21.15.105:0/2129553530 shutdown_connections 2023-11-28T18:26:47.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.708+0000 7ff127de6640 1 --2- 172.21.15.105:0/2129553530 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff1040668d0 0x7ff104068d90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.708+0000 7ff127de6640 1 --2- 172.21.15.105:0/2129553530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff120154070 0x7ff12015fd20 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.708+0000 7ff127de6640 1 --2- 172.21.15.105:0/2129553530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1200adb80 0x7ff12009b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.708+0000 7ff127de6640 1 --2- 172.21.15.105:0/2129553530 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff12006bf50 0x7ff1200ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.708+0000 7ff127de6640 1 -- 172.21.15.105:0/2129553530 >> 172.21.15.105:0/2129553530 conn(0x7ff1200ceb90 msgr2=0x7ff1200730c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:47.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.708+0000 7ff127de6640 1 -- 172.21.15.105:0/2129553530 shutdown_connections 2023-11-28T18:26:47.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.708+0000 7ff127de6640 1 -- 172.21.15.105:0/2129553530 wait complete. 2023-11-28T18:26:47.729 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T18:26:47.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.864+0000 7f36680fe640 1 Processor -- start 2023-11-28T18:26:47.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.864+0000 7f36680fe640 1 -- start start 2023-11-28T18:26:47.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.868+0000 7f36680fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f36600b4290 0x7f36600b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:47.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.868+0000 7f36680fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3660071960 0x7f366015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:47.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.868+0000 7f36680fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36600b4bb0 con 0x7f366015b030 2023-11-28T18:26:47.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.868+0000 7f36680fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36600701d0 con 0x7f3660071960 2023-11-28T18:26:47.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.868+0000 7f36680fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f366006df80 con 0x7f36600b4290 2023-11-28T18:26:47.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.868+0000 7f3665699640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3660071960 0x7f366015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:47.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.868+0000 7f3665699640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3660071960 0x7f366015f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42332/0 (socket says 172.21.15.105:42332) 2023-11-28T18:26:47.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.868+0000 7f3665699640 1 -- 172.21.15.105:0/913523187 learned_addr learned my addr 172.21.15.105:0/913523187 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:47.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.868+0000 7f3665e9a640 1 --2- 172.21.15.105:0/913523187 >> v2:172.21.15.148:3300/0 conn(0x7f36600b4290 0x7f36600b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:47.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.868+0000 7f3665699640 1 -- 172.21.15.105:0/913523187 >> v2:172.21.15.148:3300/0 conn(0x7f36600b4290 msgr2=0x7f36600b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:47.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.868+0000 7f3665699640 1 --2- 172.21.15.105:0/913523187 >> v2:172.21.15.148:3300/0 conn(0x7f36600b4290 0x7f36600b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.868+0000 7f3665699640 1 -- 172.21.15.105:0/913523187 >> v1:172.21.15.148:6789/0 conn(0x7f366015b030 legacy=0x7f36600709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:47.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.868+0000 7f3665699640 1 -- 172.21.15.105:0/913523187 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3660071df0 con 0x7f3660071960 2023-11-28T18:26:47.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.868+0000 7f3665699640 1 --2- 172.21.15.105:0/913523187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3660071960 0x7f366015f3b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f3660073a70 tx=0x7f365001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a13e4af77c7aeb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:47.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f3664e98640 1 -- 172.21.15.105:0/913523187 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36500aa5f0 con 0x7f3660071960 2023-11-28T18:26:47.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f3664e98640 1 -- 172.21.15.105:0/913523187 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36500aa750 con 0x7f3660071960 2023-11-28T18:26:47.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f3664e98640 1 -- 172.21.15.105:0/913523187 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36500aaae0 con 0x7f3660071960 2023-11-28T18:26:47.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f36680fe640 1 -- 172.21.15.105:0/913523187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3660071960 msgr2=0x7f366015f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:47.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f36680fe640 1 --2- 172.21.15.105:0/913523187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3660071960 0x7f366015f3b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f3660073a70 tx=0x7f365001c490 comp rx=0 tx=0).stop 2023-11-28T18:26:47.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f36680fe640 1 -- 172.21.15.105:0/913523187 shutdown_connections 2023-11-28T18:26:47.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f36680fe640 1 --2- 172.21.15.105:0/913523187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3660071960 0x7f366015f3b0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f36680fe640 1 --2- 172.21.15.105:0/913523187 >> v2:172.21.15.148:3300/0 conn(0x7f36600b4290 0x7f36600b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f36680fe640 1 -- 172.21.15.105:0/913523187 >> 172.21.15.105:0/913523187 conn(0x7f36600ceb90 msgr2=0x7f36600cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:47.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f36680fe640 1 -- 172.21.15.105:0/913523187 shutdown_connections 2023-11-28T18:26:47.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f36680fe640 1 -- 172.21.15.105:0/913523187 wait complete. 2023-11-28T18:26:47.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f36680fe640 1 Processor -- start 2023-11-28T18:26:47.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f36680fe640 1 -- start start 2023-11-28T18:26:47.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f36680fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36600b4290 0x7f3660152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:47.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f36680fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f366015b030 0x7f3660147a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:47.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f36680fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f366014b820 0x7f366014fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:47.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f36680fe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3660152a70 con 0x7f366015b030 2023-11-28T18:26:47.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f36680fe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366014bdc0 con 0x7f366014b820 2023-11-28T18:26:47.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f36680fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f366014c0a0 con 0x7f36600b4290 2023-11-28T18:26:47.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f3665699640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f366015b030 0x7f3660147a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:47.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f3665699640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f366015b030 0x7f3660147a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42348/0 (socket says 172.21.15.105:42348) 2023-11-28T18:26:47.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f3665699640 1 -- 172.21.15.105:0/3723972979 learned_addr learned my addr 172.21.15.105:0/3723972979 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:47.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f3665e9a640 1 --2- 172.21.15.105:0/3723972979 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36600b4290 0x7f3660152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:47.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.872+0000 7f366669b640 1 --2- 172.21.15.105:0/3723972979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f366014b820 0x7f366014fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:47.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.876+0000 7f3665699640 1 -- 172.21.15.105:0/3723972979 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36600b4290 msgr2=0x7f3660152530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:47.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.876+0000 7f3665699640 1 --2- 172.21.15.105:0/3723972979 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36600b4290 0x7f3660152530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.876+0000 7f3665699640 1 -- 172.21.15.105:0/3723972979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f366014b820 msgr2=0x7f366014fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.876+0000 7f3665699640 1 --2- 172.21.15.105:0/3723972979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f366014b820 0x7f366014fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.876+0000 7f3665699640 1 -- 172.21.15.105:0/3723972979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36500094d0 con 0x7f366015b030 2023-11-28T18:26:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.876+0000 7f3665699640 1 --2- 172.21.15.105:0/3723972979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f366015b030 0x7f3660147a20 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f365001a0c0 tx=0x7f365001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.876+0000 7f36567fc640 1 -- 172.21.15.105:0/3723972979 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36500aa5f0 con 0x7f366015b030 2023-11-28T18:26:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.876+0000 7f36567fc640 1 -- 172.21.15.105:0/3723972979 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36500b3800 con 0x7f366015b030 2023-11-28T18:26:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.876+0000 7f36567fc640 1 -- 172.21.15.105:0/3723972979 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36500b3bb0 con 0x7f366015b030 2023-11-28T18:26:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.876+0000 7f36680fe640 1 -- 172.21.15.105:0/3723972979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f366014c380 con 0x7f366015b030 2023-11-28T18:26:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.876+0000 7f36680fe640 1 -- 172.21.15.105:0/3723972979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f366014c940 con 0x7f366015b030 2023-11-28T18:26:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.876+0000 7f36680fe640 1 -- 172.21.15.105:0/3723972979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366014cb30 con 0x7f366015b030 2023-11-28T18:26:47.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.876+0000 7f36567fc640 1 -- 172.21.15.105:0/3723972979 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f36500b3d10 con 0x7f366015b030 2023-11-28T18:26:47.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.876+0000 7f36567fc640 1 --2- 172.21.15.105:0/3723972979 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f36440668d0 0x7f3644068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:47.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.876+0000 7f36567fc640 1 -- 172.21.15.105:0/3723972979 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f36500a7070 con 0x7f366015b030 2023-11-28T18:26:47.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.876+0000 7f3665e9a640 1 --2- 172.21.15.105:0/3723972979 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f36440668d0 0x7f3644068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:47.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.880+0000 7f3665e9a640 1 --2- 172.21.15.105:0/3723972979 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f36440668d0 0x7f3644068d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f365c007a60 tx=0x7f365c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:47.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:47.880+0000 7f36567fc640 1 -- 172.21.15.105:0/3723972979 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f36500ebc00 con 0x7f366015b030 2023-11-28T18:26:48.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.044+0000 7f36680fe640 1 -- 172.21.15.105:0/3723972979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3660067570 con 0x7f366015b030 2023-11-28T18:26:48.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.044+0000 7f36567fc640 1 -- 172.21.15.105:0/3723972979 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v54) v1 ==== 80+0+122 (secure 0 0 0) 0x7f36500c3020 con 0x7f366015b030 2023-11-28T18:26:48.046 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.148:6835/897997046 2023-11-29T18:26:28.339203+0000 2023-11-28T18:26:48.046 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.148:6833/897997046 2023-11-29T18:26:28.339203+0000 2023-11-28T18:26:48.046 INFO:teuthology.orchestra.run.smithi105.stderr:listed 2 entries 2023-11-28T18:26:48.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.044+0000 7f36680fe640 1 -- 172.21.15.105:0/3723972979 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f36440668d0 msgr2=0x7f3644068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:48.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.044+0000 7f36680fe640 1 --2- 172.21.15.105:0/3723972979 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f36440668d0 0x7f3644068d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f365c007a60 tx=0x7f365c008040 comp rx=0 tx=0).stop 2023-11-28T18:26:48.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.044+0000 7f36680fe640 1 -- 172.21.15.105:0/3723972979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f366015b030 msgr2=0x7f3660147a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:48.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.044+0000 7f36680fe640 1 --2- 172.21.15.105:0/3723972979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f366015b030 0x7f3660147a20 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f365001a0c0 tx=0x7f365001acd0 comp rx=0 tx=0).stop 2023-11-28T18:26:48.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.044+0000 7f36680fe640 1 -- 172.21.15.105:0/3723972979 shutdown_connections 2023-11-28T18:26:48.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.044+0000 7f36680fe640 1 --2- 172.21.15.105:0/3723972979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f366014b820 0x7f366014fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:48.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.044+0000 7f36680fe640 1 --2- 172.21.15.105:0/3723972979 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f36440668d0 0x7f3644068d90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:48.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.044+0000 7f36680fe640 1 --2- 172.21.15.105:0/3723972979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f366015b030 0x7f3660147a20 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:48.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.044+0000 7f36680fe640 1 --2- 172.21.15.105:0/3723972979 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36600b4290 0x7f3660152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:48.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.044+0000 7f36680fe640 1 -- 172.21.15.105:0/3723972979 >> 172.21.15.105:0/3723972979 conn(0x7f36600ceb90 msgr2=0x7f3660159ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:48.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.044+0000 7f36680fe640 1 -- 172.21.15.105:0/3723972979 shutdown_connections 2023-11-28T18:26:48.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.044+0000 7f36680fe640 1 -- 172.21.15.105:0/3723972979 wait complete. 2023-11-28T18:26:48.066 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2023-11-28T18:26:48.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:26:48.067 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-28T18:26:48.086 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[52772]: fuse finished with error 0 and tester_r 0 2023-11-28T18:26:48.088 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:26:48.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:26:48.089 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T18:26:48.264 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:26:48.264 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:26:48.274 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:26:48.274 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:26:48.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:26:48.275 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T18:26:48.326 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-28T18:26:48.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:26:48.327 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:26:48.328 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:26:48.336 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:26:48.337 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:26:48.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:26:48.337 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T18:26:48.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:26:48.394 INFO:teuthology.orchestra.run.smithi105.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-28T18:26:48.394 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:26:48.394 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:26:48.404 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:26:48.404 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:26:48.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:26:48.404 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T18:26:48.454 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-28T18:26:48.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:26:48.455 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:26:48.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.588+0000 7f0ce8832640 1 Processor -- start 2023-11-28T18:26:48.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.588+0000 7f0ce8832640 1 -- start start 2023-11-28T18:26:48.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.588+0000 7f0ce8832640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0ce4159610 0x7f0ce40ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:48.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.588+0000 7f0ce8832640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0ce406cef0 0x7f0ce40ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:48.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.588+0000 7f0ce8832640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce40cde30 con 0x7f0ce40cc760 2023-11-28T18:26:48.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.588+0000 7f0ce8832640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce41599f0 con 0x7f0ce406cef0 2023-11-28T18:26:48.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.588+0000 7f0ce8832640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce406de20 con 0x7f0ce4159610 2023-11-28T18:26:48.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.588+0000 7f0ce2d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0ce406cef0 0x7f0ce40ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:48.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.588+0000 7f0cd9d9b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0ce40cc760 0x7f0ce406c7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46064/0 (socket says 172.21.15.105:46064) 2023-11-28T18:26:48.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.588+0000 7f0cd9d9b640 1 -- 172.21.15.105:0/3928304554 learned_addr learned my addr 172.21.15.105:0/3928304554 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:48.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.588+0000 7f0ce259c640 1 --2- 172.21.15.105:0/3928304554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0ce4159610 0x7f0ce40ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:48.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.592+0000 7f0ce2d9d640 1 -- 172.21.15.105:0/3928304554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0ce4159610 msgr2=0x7f0ce40ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:48.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.592+0000 7f0ce2d9d640 1 --2- 172.21.15.105:0/3928304554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0ce4159610 0x7f0ce40ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:48.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.592+0000 7f0ce2d9d640 1 -- 172.21.15.105:0/3928304554 >> v1:172.21.15.105:6789/0 conn(0x7f0ce40cc760 legacy=0x7f0ce406c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:48.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.592+0000 7f0ce2d9d640 1 -- 172.21.15.105:0/3928304554 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce40cd220 con 0x7f0ce406cef0 2023-11-28T18:26:48.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.592+0000 7f0ce1d9b640 1 -- 172.21.15.105:0/3928304554 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871276632 0 0) 0x7f0ce40cde30 con 0x7f0ce40cc760 2023-11-28T18:26:48.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.592+0000 7f0ce2d9d640 1 --2- 172.21.15.105:0/3928304554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ce406cef0 0x7f0ce40ccbc0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00aea0 tx=0x7f0cdc01da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a695ab60bf1278 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:48.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.592+0000 7f0ce1d9b640 1 -- 172.21.15.105:0/3928304554 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc00a2d0 con 0x7f0ce406cef0 2023-11-28T18:26:48.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.592+0000 7f0ce1d9b640 1 -- 172.21.15.105:0/3928304554 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc0a6860 con 0x7f0ce406cef0 2023-11-28T18:26:48.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.592+0000 7f0ce1d9b640 1 -- 172.21.15.105:0/3928304554 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc0a6bf0 con 0x7f0ce406cef0 2023-11-28T18:26:48.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.592+0000 7f0ce8832640 1 -- 172.21.15.105:0/3928304554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ce406cef0 msgr2=0x7f0ce40ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:48.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.592+0000 7f0ce8832640 1 --2- 172.21.15.105:0/3928304554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ce406cef0 0x7f0ce40ccbc0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00aea0 tx=0x7f0cdc01da20 comp rx=0 tx=0).stop 2023-11-28T18:26:48.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.592+0000 7f0ce8832640 1 -- 172.21.15.105:0/3928304554 shutdown_connections 2023-11-28T18:26:48.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce8832640 1 --2- 172.21.15.105:0/3928304554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ce406cef0 0x7f0ce40ccbc0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:48.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce8832640 1 --2- 172.21.15.105:0/3928304554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0ce4159610 0x7f0ce40ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:48.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce8832640 1 -- 172.21.15.105:0/3928304554 >> 172.21.15.105:0/3928304554 conn(0x7f0ce40ceb90 msgr2=0x7f0ce40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:48.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce8832640 1 -- 172.21.15.105:0/3928304554 shutdown_connections 2023-11-28T18:26:48.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce8832640 1 -- 172.21.15.105:0/3928304554 wait complete. 2023-11-28T18:26:48.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce8832640 1 Processor -- start 2023-11-28T18:26:48.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce8832640 1 -- start start 2023-11-28T18:26:48.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce8832640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ce406cef0 0x7f0ce40ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:48.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce8832640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0ce40cc760 0x7f0ce413dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:48.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce8832640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ce4159610 0x7f0ce413e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:48.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce8832640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce40ad4f0 con 0x7f0ce4159610 2023-11-28T18:26:48.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce8832640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce413e730 con 0x7f0ce406cef0 2023-11-28T18:26:48.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce8832640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce4144890 con 0x7f0ce40cc760 2023-11-28T18:26:48.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ce4159610 0x7f0ce413e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:48.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ce4159610 0x7f0ce413e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42358/0 (socket says 172.21.15.105:42358) 2023-11-28T18:26:48.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce2d9d640 1 -- 172.21.15.105:0/1490730876 learned_addr learned my addr 172.21.15.105:0/1490730876 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:48.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce259c640 1 --2- 172.21.15.105:0/1490730876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ce406cef0 0x7f0ce40ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:48.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0cd9d9b640 1 --2- 172.21.15.105:0/1490730876 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0ce40cc760 0x7f0ce413dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:48.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce2d9d640 1 -- 172.21.15.105:0/1490730876 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0ce40cc760 msgr2=0x7f0ce413dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:48.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce2d9d640 1 --2- 172.21.15.105:0/1490730876 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0ce40cc760 0x7f0ce413dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:48.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce2d9d640 1 -- 172.21.15.105:0/1490730876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ce406cef0 msgr2=0x7f0ce40ad910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:48.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce2d9d640 1 --2- 172.21.15.105:0/1490730876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ce406cef0 0x7f0ce40ad910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:48.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce2d9d640 1 -- 172.21.15.105:0/1490730876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cdc00ab80 con 0x7f0ce4159610 2023-11-28T18:26:48.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.596+0000 7f0ce259c640 1 --2- 172.21.15.105:0/1490730876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ce406cef0 0x7f0ce40ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:48.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.600+0000 7f0ce2d9d640 1 --2- 172.21.15.105:0/1490730876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ce4159610 0x7f0ce413e1f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc0b0040 tx=0x7f0cdc01a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:48.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.600+0000 7f0cdb7fe640 1 -- 172.21.15.105:0/1490730876 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc003ba0 con 0x7f0ce4159610 2023-11-28T18:26:48.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.600+0000 7f0cdb7fe640 1 -- 172.21.15.105:0/1490730876 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc003d00 con 0x7f0ce4159610 2023-11-28T18:26:48.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.600+0000 7f0cdb7fe640 1 -- 172.21.15.105:0/1490730876 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc004220 con 0x7f0ce4159610 2023-11-28T18:26:48.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.600+0000 7f0ce8832640 1 -- 172.21.15.105:0/1490730876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce4099dc0 con 0x7f0ce4159610 2023-11-28T18:26:48.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.600+0000 7f0ce8832640 1 -- 172.21.15.105:0/1490730876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce40ace30 con 0x7f0ce4159610 2023-11-28T18:26:48.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.600+0000 7f0ce8832640 1 -- 172.21.15.105:0/1490730876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce40ad630 con 0x7f0ce4159610 2023-11-28T18:26:48.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.600+0000 7f0cdb7fe640 1 -- 172.21.15.105:0/1490730876 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f0cdc004380 con 0x7f0ce4159610 2023-11-28T18:26:48.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.600+0000 7f0cdb7fe640 1 --2- 172.21.15.105:0/1490730876 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cc40668d0 0x7f0cc4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:48.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.600+0000 7f0cdb7fe640 1 -- 172.21.15.105:0/1490730876 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f0cdc11e4f0 con 0x7f0ce4159610 2023-11-28T18:26:48.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.600+0000 7f0ce259c640 1 --2- 172.21.15.105:0/1490730876 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cc40668d0 0x7f0cc4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:48.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.600+0000 7f0ce259c640 1 --2- 172.21.15.105:0/1490730876 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cc40668d0 0x7f0cc4068d90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009b10 tx=0x7f0ccc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:48.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.604+0000 7f0cdb7fe640 1 -- 172.21.15.105:0/1490730876 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0ce40ad630 con 0x7f0ce4159610 2023-11-28T18:26:48.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.784+0000 7f0ce8832640 1 -- 172.21.15.105:0/1490730876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ce40ad630 con 0x7f0ce4159610 2023-11-28T18:26:48.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.784+0000 7f0cdb7fe640 1 -- 172.21.15.105:0/1490730876 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4955 (secure 0 0 0) 0x7f0ce40ad630 con 0x7f0ce4159610 2023-11-28T18:26:48.789 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:48.789 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":47,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1686803161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":1686803161},{"type":"v1","addr":"172.21.15.148:6835","nonce":1686803161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:26:00.550298+0000","modified":"2023-11-28T18:26:31.373860+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5842,"mds_1":5405,"mds_2":5408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5405":{"gid":5405,"name":"a","rank":1,"incarnation":43,"state":"up:active","state_seq":13,"addr":"172.21.15.105:6836/735646078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":735646078},{"type":"v1","addr":"172.21.15.105:6836","nonce":735646078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5408":{"gid":5408,"name":"c","rank":2,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6837/3765684108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":3765684108},{"type":"v1","addr":"172.21.15.105:6837","nonce":3765684108}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5842":{"gid":5842,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6834/2696773413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2696773413},{"type":"v1","addr":"172.21.15.148:6834","nonce":2696773413}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T18:26:48.789 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 47 2023-11-28T18:26:48.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.788+0000 7f0ce8832640 1 -- 172.21.15.105:0/1490730876 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cc40668d0 msgr2=0x7f0cc4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:48.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.788+0000 7f0ce8832640 1 --2- 172.21.15.105:0/1490730876 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cc40668d0 0x7f0cc4068d90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009b10 tx=0x7f0ccc008040 comp rx=0 tx=0).stop 2023-11-28T18:26:48.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.788+0000 7f0ce8832640 1 -- 172.21.15.105:0/1490730876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ce4159610 msgr2=0x7f0ce413e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:48.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.788+0000 7f0ce8832640 1 --2- 172.21.15.105:0/1490730876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ce4159610 0x7f0ce413e1f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc0b0040 tx=0x7f0cdc01a7c0 comp rx=0 tx=0).stop 2023-11-28T18:26:48.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.788+0000 7f0ce8832640 1 -- 172.21.15.105:0/1490730876 shutdown_connections 2023-11-28T18:26:48.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.788+0000 7f0ce8832640 1 --2- 172.21.15.105:0/1490730876 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cc40668d0 0x7f0cc4068d90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:48.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.788+0000 7f0ce8832640 1 --2- 172.21.15.105:0/1490730876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ce4159610 0x7f0ce413e1f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:48.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.788+0000 7f0ce8832640 1 --2- 172.21.15.105:0/1490730876 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0ce40cc760 0x7f0ce413dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:48.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.788+0000 7f0ce8832640 1 --2- 172.21.15.105:0/1490730876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ce406cef0 0x7f0ce40ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:48.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.788+0000 7f0ce8832640 1 -- 172.21.15.105:0/1490730876 >> 172.21.15.105:0/1490730876 conn(0x7f0ce40ceb90 msgr2=0x7f0ce40af770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:48.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.788+0000 7f0ce8832640 1 -- 172.21.15.105:0/1490730876 shutdown_connections 2023-11-28T18:26:48.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.788+0000 7f0ce8832640 1 -- 172.21.15.105:0/1490730876 wait complete. 2023-11-28T18:26:48.810 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:26:48.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.944+0000 7fb1116dd640 1 Processor -- start 2023-11-28T18:26:48.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.944+0000 7fb1116dd640 1 -- start start 2023-11-28T18:26:48.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.944+0000 7fb1116dd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb10c0c7490 0x7fb10c0cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:48.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.944+0000 7fb1116dd640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10c0c4a00 con 0x7fb10c06bda0 2023-11-28T18:26:48.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.944+0000 7fb1116dd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10c0d0400 con 0x7fb10c06c790 2023-11-28T18:26:48.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.944+0000 7fb1116dd640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb10c0c7870 con 0x7fb10c0c7490 2023-11-28T18:26:48.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.948+0000 7fb10b7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb10c06bda0 0x7fb10c0cc180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46068/0 (socket says 172.21.15.105:46068) 2023-11-28T18:26:48.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.948+0000 7fb10affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb10c0c7490 0x7fb10c0cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:48.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.948+0000 7fb10b7fe640 1 -- 172.21.15.105:0/381298049 learned_addr learned my addr 172.21.15.105:0/381298049 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:48.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.948+0000 7fb10affd640 1 -- 172.21.15.105:0/381298049 >> v1:172.21.15.148:6789/0 conn(0x7fb10c06c790 legacy=0x7fb10c06b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:48.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.948+0000 7fb10affd640 1 -- 172.21.15.105:0/381298049 >> v1:172.21.15.105:6789/0 conn(0x7fb10c06bda0 legacy=0x7fb10c0cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:48.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.948+0000 7fb10affd640 1 -- 172.21.15.105:0/381298049 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c0d0120 con 0x7fb10c0c7490 2023-11-28T18:26:48.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.948+0000 7fb10a7fc640 1 -- 172.21.15.105:0/381298049 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537954482 0 0) 0x7fb10c0c4a00 con 0x7fb10c06bda0 2023-11-28T18:26:48.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.948+0000 7fb10affd640 1 --2- 172.21.15.105:0/381298049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb10c0c7490 0x7fb10c0cefd0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009d90 tx=0x7fb0fc01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ad9975e1b97ff1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:48.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.948+0000 7fb10a7fc640 1 -- 172.21.15.105:0/381298049 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc0a59e0 con 0x7fb10c0c7490 2023-11-28T18:26:48.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.948+0000 7fb10a7fc640 1 -- 172.21.15.105:0/381298049 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc0a5b40 con 0x7fb10c0c7490 2023-11-28T18:26:48.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.948+0000 7fb10a7fc640 1 -- 172.21.15.105:0/381298049 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc0ad440 con 0x7fb10c0c7490 2023-11-28T18:26:48.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb1116dd640 1 -- 172.21.15.105:0/381298049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb10c0c7490 msgr2=0x7fb10c0cefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:48.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb1116dd640 1 --2- 172.21.15.105:0/381298049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb10c0c7490 0x7fb10c0cefd0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009d90 tx=0x7fb0fc01c920 comp rx=0 tx=0).stop 2023-11-28T18:26:48.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb1116dd640 1 -- 172.21.15.105:0/381298049 shutdown_connections 2023-11-28T18:26:48.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb1116dd640 1 --2- 172.21.15.105:0/381298049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb10c0c7490 0x7fb10c0cefd0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:48.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb1116dd640 1 -- 172.21.15.105:0/381298049 >> 172.21.15.105:0/381298049 conn(0x7fb10c0d0eb0 msgr2=0x7fb10c0d12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:48.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb1116dd640 1 -- 172.21.15.105:0/381298049 shutdown_connections 2023-11-28T18:26:48.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb1116dd640 1 -- 172.21.15.105:0/381298049 wait complete. 2023-11-28T18:26:48.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb1116dd640 1 Processor -- start 2023-11-28T18:26:48.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb1116dd640 1 -- start start 2023-11-28T18:26:48.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb1116dd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb10c06bda0 0x7fb10c14bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:48.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb1116dd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb10c06c790 0x7fb10c0ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:48.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb1116dd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb10c0c7490 0x7fb10c0ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:48.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb1116dd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c1580b0 con 0x7fb10c06bda0 2023-11-28T18:26:48.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb1116dd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c14c020 con 0x7fb10c0c7490 2023-11-28T18:26:48.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb1116dd640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c159b80 con 0x7fb10c06c790 2023-11-28T18:26:48.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb10affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb10c06bda0 0x7fb10c14bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:48.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb10affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb10c06bda0 0x7fb10c14bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42368/0 (socket says 172.21.15.105:42368) 2023-11-28T18:26:48.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb10affd640 1 -- 172.21.15.105:0/787127215 learned_addr learned my addr 172.21.15.105:0/787127215 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:48.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb10b7fe640 1 --2- 172.21.15.105:0/787127215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb10c0c7490 0x7fb10c0ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:48.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb1027fc640 1 --2- 172.21.15.105:0/787127215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb10c06c790 0x7fb10c0ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:48.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb10affd640 1 -- 172.21.15.105:0/787127215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb10c06c790 msgr2=0x7fb10c0ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:48.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb10affd640 1 --2- 172.21.15.105:0/787127215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb10c06c790 0x7fb10c0ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:48.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb10affd640 1 -- 172.21.15.105:0/787127215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb10c0c7490 msgr2=0x7fb10c0ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:48.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb10affd640 1 --2- 172.21.15.105:0/787127215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb10c0c7490 0x7fb10c0ca610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:48.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.952+0000 7fb10affd640 1 -- 172.21.15.105:0/787127215 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0fc009a70 con 0x7fb10c06bda0 2023-11-28T18:26:48.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.956+0000 7fb10affd640 1 --2- 172.21.15.105:0/787127215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb10c06bda0 0x7fb10c14bae0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009d90 tx=0x7fb0fc0196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:48.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.956+0000 7fb103fff640 1 -- 172.21.15.105:0/787127215 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc0a59e0 con 0x7fb10c06bda0 2023-11-28T18:26:48.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.956+0000 7fb103fff640 1 -- 172.21.15.105:0/787127215 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc0b7410 con 0x7fb10c06bda0 2023-11-28T18:26:48.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.956+0000 7fb1116dd640 1 -- 172.21.15.105:0/787127215 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10c157780 con 0x7fb10c06bda0 2023-11-28T18:26:48.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.956+0000 7fb103fff640 1 -- 172.21.15.105:0/787127215 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc0b77c0 con 0x7fb10c06bda0 2023-11-28T18:26:48.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.956+0000 7fb1116dd640 1 -- 172.21.15.105:0/787127215 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10c157d40 con 0x7fb10c06bda0 2023-11-28T18:26:48.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.956+0000 7fb1116dd640 1 -- 172.21.15.105:0/787127215 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10c067570 con 0x7fb10c06bda0 2023-11-28T18:26:48.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.956+0000 7fb103fff640 1 -- 172.21.15.105:0/787127215 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fb0fc0b4050 con 0x7fb10c06bda0 2023-11-28T18:26:48.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.956+0000 7fb103fff640 1 --2- 172.21.15.105:0/787127215 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb0ec0668d0 0x7fb0ec068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:48.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.956+0000 7fb103fff640 1 -- 172.21.15.105:0/787127215 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7fb0fc11fef0 con 0x7fb10c06bda0 2023-11-28T18:26:48.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.956+0000 7fb1027fc640 1 --2- 172.21.15.105:0/787127215 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb0ec0668d0 0x7fb0ec068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:48.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.960+0000 7fb1027fc640 1 --2- 172.21.15.105:0/787127215 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb0ec0668d0 0x7fb0ec068d90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb10c152650 tx=0x7fb0f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:48.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:48.960+0000 7fb103fff640 1 -- 172.21.15.105:0/787127215 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb0fc0ed090 con 0x7fb10c06bda0 2023-11-28T18:26:49.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.112+0000 7fb1116dd640 1 -- 172.21.15.105:0/787127215 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb10c0d0780 con 0x7fb10c06bda0 2023-11-28T18:26:49.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.116+0000 7fb103fff640 1 -- 172.21.15.105:0/787127215 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+17059 (secure 0 0 0) 0x7fb10c0d0780 con 0x7fb10c06bda0 2023-11-28T18:26:49.118 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:49.118 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":54,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:26:28.339225+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:25:58.881270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T18:25:59.532434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.39","mappings":[{"from":0,"to":6}]},{"pgid":"8.3d","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6835/897997046":"2023-11-29T18:26:28.339203+0000","172.21.15.148:6833/897997046":"2023-11-29T18:26:28.339203+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:26:49.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.116+0000 7fb1116dd640 1 -- 172.21.15.105:0/787127215 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb0ec0668d0 msgr2=0x7fb0ec068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:49.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.116+0000 7fb1116dd640 1 --2- 172.21.15.105:0/787127215 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb0ec0668d0 0x7fb0ec068d90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb10c152650 tx=0x7fb0f4009040 comp rx=0 tx=0).stop 2023-11-28T18:26:49.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.116+0000 7fb1116dd640 1 -- 172.21.15.105:0/787127215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb10c06bda0 msgr2=0x7fb10c14bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:49.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.116+0000 7fb1116dd640 1 --2- 172.21.15.105:0/787127215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb10c06bda0 0x7fb10c14bae0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009d90 tx=0x7fb0fc0196c0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.116+0000 7fb1116dd640 1 -- 172.21.15.105:0/787127215 shutdown_connections 2023-11-28T18:26:49.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.116+0000 7fb1116dd640 1 --2- 172.21.15.105:0/787127215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb10c0c7490 0x7fb10c0ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.116+0000 7fb1116dd640 1 --2- 172.21.15.105:0/787127215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb10c06c790 0x7fb10c0ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.116+0000 7fb1116dd640 1 --2- 172.21.15.105:0/787127215 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb0ec0668d0 0x7fb0ec068d90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.116+0000 7fb1116dd640 1 --2- 172.21.15.105:0/787127215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb10c06bda0 0x7fb10c14bae0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.116+0000 7fb1116dd640 1 -- 172.21.15.105:0/787127215 >> 172.21.15.105:0/787127215 conn(0x7fb10c0d0eb0 msgr2=0x7fb10c0c4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:49.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.116+0000 7fb1116dd640 1 -- 172.21.15.105:0/787127215 shutdown_connections 2023-11-28T18:26:49.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.116+0000 7fb1116dd640 1 -- 172.21.15.105:0/787127215 wait complete. 2023-11-28T18:26:49.139 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:25:58.881270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:25:59.532434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:26:49.139 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:26:49.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.280+0000 7fbd43180640 1 Processor -- start 2023-11-28T18:26:49.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.280+0000 7fbd43180640 1 -- start start 2023-11-28T18:26:49.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.284+0000 7fbd43180640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd3c159610 0x7fbd3c0ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:49.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.284+0000 7fbd43180640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbd3c0cc760 0x7fbd3c0ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:49.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.284+0000 7fbd43180640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbd3c06c7e0 0x7fbd3c06cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:49.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.284+0000 7fbd43180640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd3c06d1b0 con 0x7fbd3c0cc760 2023-11-28T18:26:49.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.284+0000 7fbd43180640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd3c0cde30 con 0x7fbd3c06c7e0 2023-11-28T18:26:49.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.284+0000 7fbd43180640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c1599f0 con 0x7fbd3c159610 2023-11-28T18:26:49.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.284+0000 7fbd33fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbd3c0cc760 0x7fbd3c0ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:49.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.284+0000 7fbd33fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbd3c0cc760 0x7fbd3c0ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42378/0 (socket says 172.21.15.105:42378) 2023-11-28T18:26:49.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.284+0000 7fbd33fff640 1 -- 172.21.15.105:0/4167098182 learned_addr learned my addr 172.21.15.105:0/4167098182 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:49.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.284+0000 7fbd4171d640 1 --2- 172.21.15.105:0/4167098182 >> v2:172.21.15.148:3300/0 conn(0x7fbd3c06c7e0 0x7fbd3c06cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:49.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.284+0000 7fbd40f1c640 1 --2- 172.21.15.105:0/4167098182 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd3c159610 0x7fbd3c0ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:49.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.284+0000 7fbd33fff640 1 -- 172.21.15.105:0/4167098182 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd3c159610 msgr2=0x7fbd3c0ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:49.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.284+0000 7fbd33fff640 1 --2- 172.21.15.105:0/4167098182 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd3c159610 0x7fbd3c0ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.284+0000 7fbd33fff640 1 -- 172.21.15.105:0/4167098182 >> v2:172.21.15.148:3300/0 conn(0x7fbd3c06c7e0 msgr2=0x7fbd3c06cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:49.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.284+0000 7fbd33fff640 1 --2- 172.21.15.105:0/4167098182 >> v2:172.21.15.148:3300/0 conn(0x7fbd3c06c7e0 0x7fbd3c06cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.284+0000 7fbd33fff640 1 -- 172.21.15.105:0/4167098182 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3c06de20 con 0x7fbd3c0cc760 2023-11-28T18:26:49.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.284+0000 7fbd4171d640 1 --2- 172.21.15.105:0/4167098182 >> v2:172.21.15.148:3300/0 conn(0x7fbd3c06c7e0 0x7fbd3c06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:49.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.284+0000 7fbd40f1c640 1 --2- 172.21.15.105:0/4167098182 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd3c159610 0x7fbd3c0ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:49.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.288+0000 7fbd33fff640 1 --2- 172.21.15.105:0/4167098182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd3c0cc760 0x7fbd3c0ccbc0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c072660 tx=0x7fbd2c01d8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ad1b584109f623 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:49.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.288+0000 7fbd337fe640 1 -- 172.21.15.105:0/4167098182 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c0a6890 con 0x7fbd3c0cc760 2023-11-28T18:26:49.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.288+0000 7fbd337fe640 1 -- 172.21.15.105:0/4167098182 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd2c0a69f0 con 0x7fbd3c0cc760 2023-11-28T18:26:49.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.288+0000 7fbd337fe640 1 -- 172.21.15.105:0/4167098182 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c0a6d80 con 0x7fbd3c0cc760 2023-11-28T18:26:49.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.288+0000 7fbd43180640 1 -- 172.21.15.105:0/4167098182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd3c0cc760 msgr2=0x7fbd3c0ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:49.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.288+0000 7fbd43180640 1 --2- 172.21.15.105:0/4167098182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd3c0cc760 0x7fbd3c0ccbc0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c072660 tx=0x7fbd2c01d8a0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.288+0000 7fbd43180640 1 -- 172.21.15.105:0/4167098182 shutdown_connections 2023-11-28T18:26:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.288+0000 7fbd43180640 1 --2- 172.21.15.105:0/4167098182 >> v2:172.21.15.148:3300/0 conn(0x7fbd3c06c7e0 0x7fbd3c06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.288+0000 7fbd43180640 1 --2- 172.21.15.105:0/4167098182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd3c0cc760 0x7fbd3c0ccbc0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.288+0000 7fbd43180640 1 --2- 172.21.15.105:0/4167098182 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd3c159610 0x7fbd3c0ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.288+0000 7fbd43180640 1 -- 172.21.15.105:0/4167098182 >> 172.21.15.105:0/4167098182 conn(0x7fbd3c0ceb90 msgr2=0x7fbd3c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:49.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.288+0000 7fbd43180640 1 -- 172.21.15.105:0/4167098182 shutdown_connections 2023-11-28T18:26:49.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.288+0000 7fbd43180640 1 -- 172.21.15.105:0/4167098182 wait complete. 2023-11-28T18:26:49.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.288+0000 7fbd43180640 1 Processor -- start 2023-11-28T18:26:49.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.288+0000 7fbd43180640 1 -- start start 2023-11-28T18:26:49.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd43180640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd3c06c7e0 0x7fbd3c14d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:49.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd43180640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd3c0cc760 0x7fbd3c1503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:49.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd43180640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd3c159610 0x7fbd3c1508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:49.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd43180640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c14dde0 con 0x7fbd3c159610 2023-11-28T18:26:49.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd43180640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c0ac950 con 0x7fbd3c0cc760 2023-11-28T18:26:49.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd43180640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c14f050 con 0x7fbd3c06c7e0 2023-11-28T18:26:49.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd4171d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd3c159610 0x7fbd3c1508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:49.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd4171d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd3c159610 0x7fbd3c1508f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42392/0 (socket says 172.21.15.105:42392) 2023-11-28T18:26:49.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd4171d640 1 -- 172.21.15.105:0/2923118987 learned_addr learned my addr 172.21.15.105:0/2923118987 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:49.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd40f1c640 1 --2- 172.21.15.105:0/2923118987 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd3c06c7e0 0x7fbd3c14d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:49.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd33fff640 1 --2- 172.21.15.105:0/2923118987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd3c0cc760 0x7fbd3c1503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:49.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd4171d640 1 -- 172.21.15.105:0/2923118987 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd3c06c7e0 msgr2=0x7fbd3c14d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:49.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd4171d640 1 --2- 172.21.15.105:0/2923118987 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd3c06c7e0 0x7fbd3c14d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd4171d640 1 -- 172.21.15.105:0/2923118987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd3c0cc760 msgr2=0x7fbd3c1503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:49.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd4171d640 1 --2- 172.21.15.105:0/2923118987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd3c0cc760 0x7fbd3c1503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd4171d640 1 -- 172.21.15.105:0/2923118987 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd2c00a670 con 0x7fbd3c159610 2023-11-28T18:26:49.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd33fff640 1 --2- 172.21.15.105:0/2923118987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd3c0cc760 0x7fbd3c1503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:49.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd4171d640 1 --2- 172.21.15.105:0/2923118987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd3c159610 0x7fbd3c1508f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fbd3400cf20 tx=0x7fbd3400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:49.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd317fa640 1 -- 172.21.15.105:0/2923118987 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3400d840 con 0x7fbd3c159610 2023-11-28T18:26:49.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd317fa640 1 -- 172.21.15.105:0/2923118987 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd3400d9a0 con 0x7fbd3c159610 2023-11-28T18:26:49.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd43180640 1 -- 172.21.15.105:0/2923118987 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd3c14f330 con 0x7fbd3c159610 2023-11-28T18:26:49.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd317fa640 1 -- 172.21.15.105:0/2923118987 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3400dd50 con 0x7fbd3c159610 2023-11-28T18:26:49.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd43180640 1 -- 172.21.15.105:0/2923118987 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3c14f8f0 con 0x7fbd3c159610 2023-11-28T18:26:49.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd43180640 1 -- 172.21.15.105:0/2923118987 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd3c0acb30 con 0x7fbd3c159610 2023-11-28T18:26:49.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.292+0000 7fbd317fa640 1 -- 172.21.15.105:0/2923118987 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fbd3401e3b0 con 0x7fbd3c159610 2023-11-28T18:26:49.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.296+0000 7fbd317fa640 1 --2- 172.21.15.105:0/2923118987 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbd28066900 0x7fbd28068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:49.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.296+0000 7fbd317fa640 1 -- 172.21.15.105:0/2923118987 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7fbd34084fe0 con 0x7fbd3c159610 2023-11-28T18:26:49.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.296+0000 7fbd40f1c640 1 --2- 172.21.15.105:0/2923118987 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbd28066900 0x7fbd28068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:49.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.296+0000 7fbd40f1c640 1 --2- 172.21.15.105:0/2923118987 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbd28066900 0x7fbd28068dc0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbd38006fd0 tx=0x7fbd38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:49.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.300+0000 7fbd317fa640 1 -- 172.21.15.105:0/2923118987 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbd3c0acb30 con 0x7fbd3c159610 2023-11-28T18:26:49.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.456+0000 7fbd43180640 1 -- 172.21.15.105:0/2923118987 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbd3c067570 con 0x7fbd3c159610 2023-11-28T18:26:49.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.456+0000 7fbd317fa640 1 -- 172.21.15.105:0/2923118987 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbd3401a0a0 con 0x7fbd3c159610 2023-11-28T18:26:49.458 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:26:49.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.456+0000 7fbd43180640 1 -- 172.21.15.105:0/2923118987 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbd28066900 msgr2=0x7fbd28068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:49.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.456+0000 7fbd43180640 1 --2- 172.21.15.105:0/2923118987 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbd28066900 0x7fbd28068dc0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbd38006fd0 tx=0x7fbd38008040 comp rx=0 tx=0).stop 2023-11-28T18:26:49.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.456+0000 7fbd43180640 1 -- 172.21.15.105:0/2923118987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd3c159610 msgr2=0x7fbd3c1508f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:49.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.456+0000 7fbd43180640 1 --2- 172.21.15.105:0/2923118987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd3c159610 0x7fbd3c1508f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fbd3400cf20 tx=0x7fbd3400a710 comp rx=0 tx=0).stop 2023-11-28T18:26:49.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.456+0000 7fbd43180640 1 -- 172.21.15.105:0/2923118987 shutdown_connections 2023-11-28T18:26:49.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.456+0000 7fbd43180640 1 --2- 172.21.15.105:0/2923118987 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbd28066900 0x7fbd28068dc0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.456+0000 7fbd43180640 1 --2- 172.21.15.105:0/2923118987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd3c159610 0x7fbd3c1508f0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.456+0000 7fbd43180640 1 --2- 172.21.15.105:0/2923118987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd3c0cc760 0x7fbd3c1503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.456+0000 7fbd43180640 1 --2- 172.21.15.105:0/2923118987 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd3c06c7e0 0x7fbd3c14d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.456+0000 7fbd43180640 1 -- 172.21.15.105:0/2923118987 >> 172.21.15.105:0/2923118987 conn(0x7fbd3c0ceb90 msgr2=0x7fbd3c153ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:49.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.456+0000 7fbd43180640 1 -- 172.21.15.105:0/2923118987 shutdown_connections 2023-11-28T18:26:49.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.456+0000 7fbd43180640 1 -- 172.21.15.105:0/2923118987 wait complete. 2023-11-28T18:26:49.479 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:26:49.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.616+0000 7ffb3fefe640 1 Processor -- start 2023-11-28T18:26:49.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.616+0000 7ffb3fefe640 1 -- start start 2023-11-28T18:26:49.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.616+0000 7ffb3fefe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffb380caf60 0x7ffb380cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:49.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.616+0000 7ffb3fefe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb3806bf50 con 0x7ffb380ad850 2023-11-28T18:26:49.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.616+0000 7ffb3fefe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb3815a320 con 0x7ffb38159f40 2023-11-28T18:26:49.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.616+0000 7ffb3fefe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb380cb3c0 con 0x7ffb380caf60 2023-11-28T18:26:49.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.616+0000 7ffb3e49b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ffb380ad850 0x7ffb380701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46080/0 (socket says 172.21.15.105:46080) 2023-11-28T18:26:49.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.616+0000 7ffb3e49b640 1 -- 172.21.15.105:0/1481531991 learned_addr learned my addr 172.21.15.105:0/1481531991 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:49.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.616+0000 7ffb3d499640 1 --2- 172.21.15.105:0/1481531991 >> v2:172.21.15.148:3300/0 conn(0x7ffb380caf60 0x7ffb380cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:49.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.616+0000 7ffb3cc98640 1 -- 172.21.15.105:0/1481531991 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3628263657 0 0) 0x7ffb3806bf50 con 0x7ffb380ad850 2023-11-28T18:26:49.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.620+0000 7ffb3cc98640 1 -- 172.21.15.105:0/1481531991 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb24003530 con 0x7ffb380ad850 2023-11-28T18:26:49.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.620+0000 7ffb3cc98640 1 -- 172.21.15.105:0/1481531991 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3234541571 0 0) 0x7ffb24003530 con 0x7ffb380ad850 2023-11-28T18:26:49.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.620+0000 7ffb3d499640 1 -- 172.21.15.105:0/1481531991 >> v1:172.21.15.148:6789/0 conn(0x7ffb38159f40 legacy=0x7ffb380ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:49.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.620+0000 7ffb3d499640 1 -- 172.21.15.105:0/1481531991 >> v1:172.21.15.105:6789/0 conn(0x7ffb380ad850 legacy=0x7ffb380701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:49.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.620+0000 7ffb3d499640 1 -- 172.21.15.105:0/1481531991 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb38071380 con 0x7ffb380caf60 2023-11-28T18:26:49.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.620+0000 7ffb3d499640 1 --2- 172.21.15.105:0/1481531991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb380caf60 0x7ffb380cc760 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffb280097c0 tx=0x7ffb2801c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9a19eb81cf3ba8f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:49.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.620+0000 7ffb3cc98640 1 -- 172.21.15.105:0/1481531991 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30001a00 con 0x7ffb380caf60 2023-11-28T18:26:49.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.620+0000 7ffb3cc98640 1 -- 172.21.15.105:0/1481531991 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb280a7050 con 0x7ffb380caf60 2023-11-28T18:26:49.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.620+0000 7ffb3cc98640 1 -- 172.21.15.105:0/1481531991 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb280ab620 con 0x7ffb380caf60 2023-11-28T18:26:49.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3fefe640 1 -- 172.21.15.105:0/1481531991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb380caf60 msgr2=0x7ffb380cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:49.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3fefe640 1 --2- 172.21.15.105:0/1481531991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb380caf60 0x7ffb380cc760 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffb280097c0 tx=0x7ffb2801c2e0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3fefe640 1 -- 172.21.15.105:0/1481531991 shutdown_connections 2023-11-28T18:26:49.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3fefe640 1 --2- 172.21.15.105:0/1481531991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb380caf60 0x7ffb380cc760 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3fefe640 1 -- 172.21.15.105:0/1481531991 >> 172.21.15.105:0/1481531991 conn(0x7ffb380ceb90 msgr2=0x7ffb380cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:49.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3fefe640 1 -- 172.21.15.105:0/1481531991 shutdown_connections 2023-11-28T18:26:49.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3fefe640 1 -- 172.21.15.105:0/1481531991 wait complete. 2023-11-28T18:26:49.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3fefe640 1 Processor -- start 2023-11-28T18:26:49.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3fefe640 1 -- start start 2023-11-28T18:26:49.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3fefe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb380ad850 0x7ffb3814c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:49.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3fefe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffb380caf60 0x7ffb3814fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3fefe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb38159f40 0x7ffb3814b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3fefe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3814ca60 con 0x7ffb38159f40 2023-11-28T18:26:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3fefe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3814dc60 con 0x7ffb380ad850 2023-11-28T18:26:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3fefe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb38151c40 con 0x7ffb380caf60 2023-11-28T18:26:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3e49b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb38159f40 0x7ffb3814b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:49.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3e49b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb38159f40 0x7ffb3814b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42404/0 (socket says 172.21.15.105:42404) 2023-11-28T18:26:49.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3e49b640 1 -- 172.21.15.105:0/1768086761 learned_addr learned my addr 172.21.15.105:0/1768086761 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:49.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3d499640 1 --2- 172.21.15.105:0/1768086761 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffb380caf60 0x7ffb3814fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:49.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3dc9a640 1 --2- 172.21.15.105:0/1768086761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb380ad850 0x7ffb3814c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:49.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3e49b640 1 -- 172.21.15.105:0/1768086761 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffb380caf60 msgr2=0x7ffb3814fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:49.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3e49b640 1 --2- 172.21.15.105:0/1768086761 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffb380caf60 0x7ffb3814fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3e49b640 1 -- 172.21.15.105:0/1768086761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb380ad850 msgr2=0x7ffb3814c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:49.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3e49b640 1 --2- 172.21.15.105:0/1768086761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb380ad850 0x7ffb3814c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3e49b640 1 -- 172.21.15.105:0/1768086761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb28009420 con 0x7ffb38159f40 2023-11-28T18:26:49.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3dc9a640 1 --2- 172.21.15.105:0/1768086761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb380ad850 0x7ffb3814c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:49.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.624+0000 7ffb3e49b640 1 --2- 172.21.15.105:0/1768086761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb38159f40 0x7ffb3814b8d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000cc70 tx=0x7ffb3000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:49.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.628+0000 7ffb2e7fc640 1 -- 172.21.15.105:0/1768086761 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3000ce80 con 0x7ffb38159f40 2023-11-28T18:26:49.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.628+0000 7ffb2e7fc640 1 -- 172.21.15.105:0/1768086761 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb30012400 con 0x7ffb38159f40 2023-11-28T18:26:49.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.628+0000 7ffb2e7fc640 1 -- 172.21.15.105:0/1768086761 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb300127d0 con 0x7ffb38159f40 2023-11-28T18:26:49.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.628+0000 7ffb3fefe640 1 -- 172.21.15.105:0/1768086761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3814d980 con 0x7ffb38159f40 2023-11-28T18:26:49.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.628+0000 7ffb3fefe640 1 -- 172.21.15.105:0/1768086761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb381518d0 con 0x7ffb38159f40 2023-11-28T18:26:49.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.628+0000 7ffb3fefe640 1 -- 172.21.15.105:0/1768086761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb38151db0 con 0x7ffb38159f40 2023-11-28T18:26:49.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.628+0000 7ffb2e7fc640 1 -- 172.21.15.105:0/1768086761 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ffb30007070 con 0x7ffb38159f40 2023-11-28T18:26:49.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.628+0000 7ffb2e7fc640 1 --2- 172.21.15.105:0/1768086761 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffb1c066900 0x7ffb1c068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:49.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.628+0000 7ffb2e7fc640 1 -- 172.21.15.105:0/1768086761 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7ffb30084770 con 0x7ffb38159f40 2023-11-28T18:26:49.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.628+0000 7ffb3dc9a640 1 --2- 172.21.15.105:0/1768086761 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffb1c066900 0x7ffb1c068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:49.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.628+0000 7ffb3dc9a640 1 --2- 172.21.15.105:0/1768086761 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffb1c066900 0x7ffb1c068dc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ffb381f9530 tx=0x7ffb34009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:49.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.632+0000 7ffb2e7fc640 1 -- 172.21.15.105:0/1768086761 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ffb300517d0 con 0x7ffb38159f40 2023-11-28T18:26:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.780+0000 7ffb3fefe640 1 -- 172.21.15.105:0/1768086761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffb38067570 con 0x7ffb38159f40 2023-11-28T18:26:49.785 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.780+0000 7ffb2e7fc640 1 -- 172.21.15.105:0/1768086761 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffb30054df0 con 0x7ffb38159f40 2023-11-28T18:26:49.785 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:26:49.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.784+0000 7ffb3fefe640 1 -- 172.21.15.105:0/1768086761 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffb1c066900 msgr2=0x7ffb1c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:49.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.784+0000 7ffb3fefe640 1 --2- 172.21.15.105:0/1768086761 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffb1c066900 0x7ffb1c068dc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ffb381f9530 tx=0x7ffb34009040 comp rx=0 tx=0).stop 2023-11-28T18:26:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.784+0000 7ffb3fefe640 1 -- 172.21.15.105:0/1768086761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb38159f40 msgr2=0x7ffb3814b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.784+0000 7ffb3fefe640 1 --2- 172.21.15.105:0/1768086761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb38159f40 0x7ffb3814b8d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000cc70 tx=0x7ffb3000a430 comp rx=0 tx=0).stop 2023-11-28T18:26:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.784+0000 7ffb3fefe640 1 -- 172.21.15.105:0/1768086761 shutdown_connections 2023-11-28T18:26:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.784+0000 7ffb3fefe640 1 --2- 172.21.15.105:0/1768086761 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffb1c066900 0x7ffb1c068dc0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.784+0000 7ffb3fefe640 1 --2- 172.21.15.105:0/1768086761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb38159f40 0x7ffb3814b8d0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.784+0000 7ffb3fefe640 1 --2- 172.21.15.105:0/1768086761 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffb380caf60 0x7ffb3814fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.784+0000 7ffb3fefe640 1 --2- 172.21.15.105:0/1768086761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb380ad850 0x7ffb3814c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.784+0000 7ffb3fefe640 1 -- 172.21.15.105:0/1768086761 >> 172.21.15.105:0/1768086761 conn(0x7ffb380ceb90 msgr2=0x7ffb380afcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.784+0000 7ffb3fefe640 1 -- 172.21.15.105:0/1768086761 shutdown_connections 2023-11-28T18:26:49.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.784+0000 7ffb3fefe640 1 -- 172.21.15.105:0/1768086761 wait complete. 2023-11-28T18:26:49.806 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:26:49.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.944+0000 7f2366e17640 1 Processor -- start 2023-11-28T18:26:49.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.944+0000 7f2366e17640 1 -- start start 2023-11-28T18:26:49.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.944+0000 7f2366e17640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2360071960 0x7f236015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:49.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.944+0000 7f2366e17640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f236015b030 0x7f23600709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:49.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.944+0000 7f2366e17640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23600701d0 con 0x7f23600b4290 2023-11-28T18:26:49.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.944+0000 7f2366e17640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236006df80 con 0x7f236015b030 2023-11-28T18:26:49.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.944+0000 7f2366e17640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2360071df0 con 0x7f2360071960 2023-11-28T18:26:49.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.944+0000 7f23653b4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f236015b030 0x7f23600709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:49.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.944+0000 7f235ffff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2360071960 0x7f236015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:49.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.944+0000 7f235ffff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2360071960 0x7f236015f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:45540/0 (socket says 172.21.15.105:45540) 2023-11-28T18:26:49.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.944+0000 7f23653b4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f236015b030 0x7f23600709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:52818/0 (socket says 172.21.15.105:52818) 2023-11-28T18:26:49.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.944+0000 7f235ffff640 1 -- 172.21.15.105:0/3460181059 learned_addr learned my addr 172.21.15.105:0/3460181059 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:49.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.944+0000 7f23653b4640 1 -- 172.21.15.105:0/3460181059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2360071960 msgr2=0x7f236015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:49.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.944+0000 7f23653b4640 1 --2- 172.21.15.105:0/3460181059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2360071960 0x7f236015f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.944+0000 7f23653b4640 1 -- 172.21.15.105:0/3460181059 >> v1:172.21.15.148:6789/0 conn(0x7f23600b4290 legacy=0x7f23600b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:49.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.944+0000 7f23653b4640 1 -- 172.21.15.105:0/3460181059 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23600720d0 con 0x7f236015b030 2023-11-28T18:26:49.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.948+0000 7f23653b4640 1 --2- 172.21.15.105:0/3460181059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f236015b030 0x7f23600709b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2358009d90 tx=0x7f235801c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=252696f47c676995 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:49.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.948+0000 7f235f7fe640 1 -- 172.21.15.105:0/3460181059 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23580a58f0 con 0x7f236015b030 2023-11-28T18:26:49.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.948+0000 7f235f7fe640 1 -- 172.21.15.105:0/3460181059 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23580a5a50 con 0x7f236015b030 2023-11-28T18:26:49.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.948+0000 7f235f7fe640 1 -- 172.21.15.105:0/3460181059 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23580a5de0 con 0x7f236015b030 2023-11-28T18:26:49.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.948+0000 7f2366e17640 1 -- 172.21.15.105:0/3460181059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f236015b030 msgr2=0x7f23600709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:49.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.948+0000 7f2366e17640 1 --2- 172.21.15.105:0/3460181059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f236015b030 0x7f23600709b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2358009d90 tx=0x7f235801c920 comp rx=0 tx=0).stop 2023-11-28T18:26:49.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.948+0000 7f2366e17640 1 -- 172.21.15.105:0/3460181059 shutdown_connections 2023-11-28T18:26:49.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.948+0000 7f2366e17640 1 --2- 172.21.15.105:0/3460181059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f236015b030 0x7f23600709b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.948+0000 7f2366e17640 1 --2- 172.21.15.105:0/3460181059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2360071960 0x7f236015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.948+0000 7f2366e17640 1 -- 172.21.15.105:0/3460181059 >> 172.21.15.105:0/3460181059 conn(0x7f23600ceb90 msgr2=0x7f23600cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:49.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.948+0000 7f2366e17640 1 -- 172.21.15.105:0/3460181059 shutdown_connections 2023-11-28T18:26:49.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.948+0000 7f2366e17640 1 -- 172.21.15.105:0/3460181059 wait complete. 2023-11-28T18:26:49.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f2366e17640 1 Processor -- start 2023-11-28T18:26:49.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f2366e17640 1 -- start start 2023-11-28T18:26:49.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f2366e17640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23600b4290 0x7f2360165230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:49.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f2366e17640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f236015b030 0x7f236014f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:49.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f2366e17640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2360157570 0x7f236013c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:49.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f2366e17640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23600cd070 con 0x7f236015b030 2023-11-28T18:26:49.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f2366e17640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2360165770 con 0x7f2360157570 2023-11-28T18:26:49.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f2366e17640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236014f6e0 con 0x7f23600b4290 2023-11-28T18:26:49.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f235ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f236015b030 0x7f236014f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:49.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f235ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f236015b030 0x7f236014f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42416/0 (socket says 172.21.15.105:42416) 2023-11-28T18:26:49.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f235ffff640 1 -- 172.21.15.105:0/764798954 learned_addr learned my addr 172.21.15.105:0/764798954 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:49.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f2364bb3640 1 --2- 172.21.15.105:0/764798954 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23600b4290 0x7f2360165230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:49.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f23653b4640 1 --2- 172.21.15.105:0/764798954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2360157570 0x7f236013c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:49.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f235ffff640 1 -- 172.21.15.105:0/764798954 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23600b4290 msgr2=0x7f2360165230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:49.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f235ffff640 1 --2- 172.21.15.105:0/764798954 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23600b4290 0x7f2360165230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f235ffff640 1 -- 172.21.15.105:0/764798954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2360157570 msgr2=0x7f236013c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:49.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f235ffff640 1 --2- 172.21.15.105:0/764798954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2360157570 0x7f236013c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:49.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f235ffff640 1 -- 172.21.15.105:0/764798954 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2358009a70 con 0x7f236015b030 2023-11-28T18:26:49.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f235ffff640 1 --2- 172.21.15.105:0/764798954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f236015b030 0x7f236014f1a0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f234c00e8c0 tx=0x7f234c00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:49.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f235d7fa640 1 -- 172.21.15.105:0/764798954 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c009620 con 0x7f236015b030 2023-11-28T18:26:49.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f235d7fa640 1 -- 172.21.15.105:0/764798954 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234c00c940 con 0x7f236015b030 2023-11-28T18:26:49.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f2366e17640 1 -- 172.21.15.105:0/764798954 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23600cc760 con 0x7f236015b030 2023-11-28T18:26:49.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f235d7fa640 1 -- 172.21.15.105:0/764798954 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c00ccf0 con 0x7f236015b030 2023-11-28T18:26:49.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f2366e17640 1 -- 172.21.15.105:0/764798954 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23600cc9b0 con 0x7f236015b030 2023-11-28T18:26:49.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.952+0000 7f2366e17640 1 -- 172.21.15.105:0/764798954 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2360067570 con 0x7f236015b030 2023-11-28T18:26:49.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.956+0000 7f235d7fa640 1 -- 172.21.15.105:0/764798954 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f234c013470 con 0x7f236015b030 2023-11-28T18:26:49.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.956+0000 7f235d7fa640 1 --2- 172.21.15.105:0/764798954 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2344066900 0x7f2344068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:49.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.956+0000 7f235d7fa640 1 -- 172.21.15.105:0/764798954 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f234c084660 con 0x7f236015b030 2023-11-28T18:26:49.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.956+0000 7f2364bb3640 1 --2- 172.21.15.105:0/764798954 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2344066900 0x7f2344068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:49.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.956+0000 7f2364bb3640 1 --2- 172.21.15.105:0/764798954 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2344066900 0x7f2344068dc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2354007fd0 tx=0x7f2354009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:49.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:49.960+0000 7f235d7fa640 1 -- 172.21.15.105:0/764798954 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f234c051770 con 0x7f236015b030 2023-11-28T18:26:50.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.107+0000 7f2366e17640 1 -- 172.21.15.105:0/764798954 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f23601550b0 con 0x7f236015b030 2023-11-28T18:26:50.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.111+0000 7f235d7fa640 1 -- 172.21.15.105:0/764798954 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7f234c054d90 con 0x7f236015b030 2023-11-28T18:26:50.114 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:26:50.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.111+0000 7f2366e17640 1 -- 172.21.15.105:0/764798954 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2344066900 msgr2=0x7f2344068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:50.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.111+0000 7f2366e17640 1 --2- 172.21.15.105:0/764798954 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2344066900 0x7f2344068dc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2354007fd0 tx=0x7f2354009040 comp rx=0 tx=0).stop 2023-11-28T18:26:50.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.111+0000 7f2366e17640 1 -- 172.21.15.105:0/764798954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f236015b030 msgr2=0x7f236014f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:50.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.111+0000 7f2366e17640 1 --2- 172.21.15.105:0/764798954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f236015b030 0x7f236014f1a0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f234c00e8c0 tx=0x7f234c00ed80 comp rx=0 tx=0).stop 2023-11-28T18:26:50.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.111+0000 7f2366e17640 1 -- 172.21.15.105:0/764798954 shutdown_connections 2023-11-28T18:26:50.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.111+0000 7f2366e17640 1 --2- 172.21.15.105:0/764798954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2360157570 0x7f236013c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.111+0000 7f2366e17640 1 --2- 172.21.15.105:0/764798954 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2344066900 0x7f2344068dc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.111+0000 7f2366e17640 1 --2- 172.21.15.105:0/764798954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f236015b030 0x7f236014f1a0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.111+0000 7f2366e17640 1 --2- 172.21.15.105:0/764798954 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23600b4290 0x7f2360165230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.111+0000 7f2366e17640 1 -- 172.21.15.105:0/764798954 >> 172.21.15.105:0/764798954 conn(0x7f23600ceb90 msgr2=0x7f2360158af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:50.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.111+0000 7f2366e17640 1 -- 172.21.15.105:0/764798954 shutdown_connections 2023-11-28T18:26:50.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.111+0000 7f2366e17640 1 -- 172.21.15.105:0/764798954 wait complete. 2023-11-28T18:26:50.135 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:26:50.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.267+0000 7fb68d31d640 1 Processor -- start 2023-11-28T18:26:50.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.267+0000 7fb68d31d640 1 -- start start 2023-11-28T18:26:50.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.267+0000 7fb68d31d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb688159610 0x7fb6880ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:50.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.267+0000 7fb68d31d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6880cde30 con 0x7fb6880cc760 2023-11-28T18:26:50.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.267+0000 7fb68d31d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6881599f0 con 0x7fb68806cef0 2023-11-28T18:26:50.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.267+0000 7fb68d31d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb68806de20 con 0x7fb688159610 2023-11-28T18:26:50.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.271+0000 7fb67e7fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb6880cc760 0x7fb68806c7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46092/0 (socket says 172.21.15.105:46092) 2023-11-28T18:26:50.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.271+0000 7fb67e7fc640 1 -- 172.21.15.105:0/1991209564 learned_addr learned my addr 172.21.15.105:0/1991209564 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:50.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.271+0000 7fb686ffd640 1 --2- 172.21.15.105:0/1991209564 >> v2:172.21.15.105:3300/0 conn(0x7fb688159610 0x7fb6880ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:50.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.271+0000 7fb686ffd640 1 -- 172.21.15.105:0/1991209564 >> v1:172.21.15.148:6789/0 conn(0x7fb68806cef0 legacy=0x7fb6880abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:50.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.271+0000 7fb686ffd640 1 -- 172.21.15.105:0/1991209564 >> v1:172.21.15.105:6789/0 conn(0x7fb6880cc760 legacy=0x7fb68806c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:50.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.271+0000 7fb686ffd640 1 -- 172.21.15.105:0/1991209564 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6880cd220 con 0x7fb688159610 2023-11-28T18:26:50.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.271+0000 7fb6867fc640 1 -- 172.21.15.105:0/1991209564 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2753952775 0 0) 0x7fb6880cde30 con 0x7fb6880cc760 2023-11-28T18:26:50.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.271+0000 7fb686ffd640 1 --2- 172.21.15.105:0/1991209564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb688159610 0x7fb6880ce2c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb6780097c0 tx=0x7fb6880acca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa178e7dd8bd7d17 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:50.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.271+0000 7fb6867fc640 1 -- 172.21.15.105:0/1991209564 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6780ad660 con 0x7fb688159610 2023-11-28T18:26:50.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.271+0000 7fb6867fc640 1 -- 172.21.15.105:0/1991209564 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6780ad7c0 con 0x7fb688159610 2023-11-28T18:26:50.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.271+0000 7fb6867fc640 1 -- 172.21.15.105:0/1991209564 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6780adb50 con 0x7fb688159610 2023-11-28T18:26:50.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb68d31d640 1 -- 172.21.15.105:0/1991209564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb688159610 msgr2=0x7fb6880ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:50.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb68d31d640 1 --2- 172.21.15.105:0/1991209564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb688159610 0x7fb6880ce2c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb6780097c0 tx=0x7fb6880acca0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb68d31d640 1 -- 172.21.15.105:0/1991209564 shutdown_connections 2023-11-28T18:26:50.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb68d31d640 1 --2- 172.21.15.105:0/1991209564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb688159610 0x7fb6880ce2c0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb68d31d640 1 -- 172.21.15.105:0/1991209564 >> 172.21.15.105:0/1991209564 conn(0x7fb6880ceb90 msgr2=0x7fb6880cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:50.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb68d31d640 1 -- 172.21.15.105:0/1991209564 shutdown_connections 2023-11-28T18:26:50.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb68d31d640 1 -- 172.21.15.105:0/1991209564 wait complete. 2023-11-28T18:26:50.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb68d31d640 1 Processor -- start 2023-11-28T18:26:50.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb68d31d640 1 -- start start 2023-11-28T18:26:50.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb68d31d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68806cef0 0x7fb688152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:50.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb68d31d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6880cc760 0x7fb68815d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:50.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb68d31d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb688159610 0x7fb68815d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:50.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb68d31d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6880b19b0 con 0x7fb6880cc760 2023-11-28T18:26:50.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb68d31d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6880b2530 con 0x7fb68806cef0 2023-11-28T18:26:50.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb68d31d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6880b1c00 con 0x7fb688159610 2023-11-28T18:26:50.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb67e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6880cc760 0x7fb68815d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:50.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb67e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6880cc760 0x7fb68815d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42434/0 (socket says 172.21.15.105:42434) 2023-11-28T18:26:50.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb67e7fc640 1 -- 172.21.15.105:0/1878649578 learned_addr learned my addr 172.21.15.105:0/1878649578 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:50.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb686ffd640 1 --2- 172.21.15.105:0/1878649578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68806cef0 0x7fb688152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:50.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb6877fe640 1 --2- 172.21.15.105:0/1878649578 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb688159610 0x7fb68815d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:50.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb67e7fc640 1 -- 172.21.15.105:0/1878649578 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb688159610 msgr2=0x7fb68815d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:50.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb67e7fc640 1 --2- 172.21.15.105:0/1878649578 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb688159610 0x7fb68815d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb67e7fc640 1 -- 172.21.15.105:0/1878649578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68806cef0 msgr2=0x7fb688152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:50.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb67e7fc640 1 --2- 172.21.15.105:0/1878649578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68806cef0 0x7fb688152280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.275+0000 7fb67e7fc640 1 -- 172.21.15.105:0/1878649578 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb678009420 con 0x7fb6880cc760 2023-11-28T18:26:50.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.279+0000 7fb67e7fc640 1 --2- 172.21.15.105:0/1878649578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6880cc760 0x7fb68815d010 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb67000c8f0 tx=0x7fb67000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:50.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.279+0000 7fb67ffff640 1 -- 172.21.15.105:0/1878649578 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67000a640 con 0x7fb6880cc760 2023-11-28T18:26:50.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.279+0000 7fb67ffff640 1 -- 172.21.15.105:0/1878649578 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67000a7a0 con 0x7fb6880cc760 2023-11-28T18:26:50.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.279+0000 7fb68d31d640 1 -- 172.21.15.105:0/1878649578 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6880b1ee0 con 0x7fb6880cc760 2023-11-28T18:26:50.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.279+0000 7fb67ffff640 1 -- 172.21.15.105:0/1878649578 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67000ab50 con 0x7fb6880cc760 2023-11-28T18:26:50.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.279+0000 7fb68d31d640 1 -- 172.21.15.105:0/1878649578 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68809a700 con 0x7fb6880cc760 2023-11-28T18:26:50.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.279+0000 7fb68d31d640 1 -- 172.21.15.105:0/1878649578 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb688067570 con 0x7fb6880cc760 2023-11-28T18:26:50.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.279+0000 7fb67ffff640 1 -- 172.21.15.105:0/1878649578 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fb67000acb0 con 0x7fb6880cc760 2023-11-28T18:26:50.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.279+0000 7fb67ffff640 1 --2- 172.21.15.105:0/1878649578 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb668066900 0x7fb668068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:50.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.279+0000 7fb67ffff640 1 -- 172.21.15.105:0/1878649578 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7fb6700839f0 con 0x7fb6880cc760 2023-11-28T18:26:50.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.279+0000 7fb686ffd640 1 --2- 172.21.15.105:0/1878649578 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb668066900 0x7fb668068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:50.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.283+0000 7fb686ffd640 1 --2- 172.21.15.105:0/1878649578 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb668066900 0x7fb668068dc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb6780af000 tx=0x7fb678008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:50.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.283+0000 7fb67ffff640 1 -- 172.21.15.105:0/1878649578 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb670050ba0 con 0x7fb6880cc760 2023-11-28T18:26:50.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.467+0000 7fb68d31d640 1 -- 172.21.15.105:0/1878649578 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6880cf950 con 0x7fb6880cc760 2023-11-28T18:26:50.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.467+0000 7fb67ffff640 1 -- 172.21.15.105:0/1878649578 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4955 (secure 0 0 0) 0x7fb670018020 con 0x7fb6880cc760 2023-11-28T18:26:50.471 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:50.471 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":47,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1686803161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":1686803161},{"type":"v1","addr":"172.21.15.148:6835","nonce":1686803161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:26:00.550298+0000","modified":"2023-11-28T18:26:31.373860+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5842,"mds_1":5405,"mds_2":5408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5405":{"gid":5405,"name":"a","rank":1,"incarnation":43,"state":"up:active","state_seq":13,"addr":"172.21.15.105:6836/735646078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":735646078},{"type":"v1","addr":"172.21.15.105:6836","nonce":735646078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5408":{"gid":5408,"name":"c","rank":2,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6837/3765684108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":3765684108},{"type":"v1","addr":"172.21.15.105:6837","nonce":3765684108}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5842":{"gid":5842,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6834/2696773413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2696773413},{"type":"v1","addr":"172.21.15.148:6834","nonce":2696773413}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T18:26:50.472 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 47 2023-11-28T18:26:50.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.467+0000 7fb68d31d640 1 -- 172.21.15.105:0/1878649578 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb668066900 msgr2=0x7fb668068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:50.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.467+0000 7fb68d31d640 1 --2- 172.21.15.105:0/1878649578 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb668066900 0x7fb668068dc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb6780af000 tx=0x7fb678008040 comp rx=0 tx=0).stop 2023-11-28T18:26:50.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.467+0000 7fb68d31d640 1 -- 172.21.15.105:0/1878649578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6880cc760 msgr2=0x7fb68815d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.467+0000 7fb68d31d640 1 --2- 172.21.15.105:0/1878649578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6880cc760 0x7fb68815d010 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb67000c8f0 tx=0x7fb67000cdb0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.467+0000 7fb68d31d640 1 -- 172.21.15.105:0/1878649578 shutdown_connections 2023-11-28T18:26:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.467+0000 7fb68d31d640 1 --2- 172.21.15.105:0/1878649578 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb688159610 0x7fb68815d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.467+0000 7fb68d31d640 1 --2- 172.21.15.105:0/1878649578 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb668066900 0x7fb668068dc0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.467+0000 7fb68d31d640 1 --2- 172.21.15.105:0/1878649578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6880cc760 0x7fb68815d010 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.467+0000 7fb68d31d640 1 --2- 172.21.15.105:0/1878649578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68806cef0 0x7fb688152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.467+0000 7fb68d31d640 1 -- 172.21.15.105:0/1878649578 >> 172.21.15.105:0/1878649578 conn(0x7fb6880ceb90 msgr2=0x7fb68815c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.471+0000 7fb68d31d640 1 -- 172.21.15.105:0/1878649578 shutdown_connections 2023-11-28T18:26:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.471+0000 7fb68d31d640 1 -- 172.21.15.105:0/1878649578 wait complete. 2023-11-28T18:26:50.493 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:26:50.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.627+0000 7fae47efe640 1 Processor -- start 2023-11-28T18:26:50.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.627+0000 7fae47efe640 1 -- start start 2023-11-28T18:26:50.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.627+0000 7fae47efe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fae40159610 0x7fae400ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:50.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.627+0000 7fae47efe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fae400cc760 0x7fae400ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:50.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.627+0000 7fae47efe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae400cde30 con 0x7fae4006c7e0 2023-11-28T18:26:50.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.627+0000 7fae47efe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae401599f0 con 0x7fae400cc760 2023-11-28T18:26:50.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.627+0000 7fae47efe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae4006de20 con 0x7fae40159610 2023-11-28T18:26:50.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.631+0000 7fae45499640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fae400cc760 0x7fae400ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:50.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.631+0000 7fae45499640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fae400cc760 0x7fae400ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42442/0 (socket says 172.21.15.105:42442) 2023-11-28T18:26:50.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.631+0000 7fae45499640 1 -- 172.21.15.105:0/1733242489 learned_addr learned my addr 172.21.15.105:0/1733242489 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:50.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.631+0000 7fae45c9a640 1 --2- 172.21.15.105:0/1733242489 >> v2:172.21.15.148:3300/0 conn(0x7fae40159610 0x7fae400ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:50.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.631+0000 7fae45499640 1 -- 172.21.15.105:0/1733242489 >> v2:172.21.15.148:3300/0 conn(0x7fae40159610 msgr2=0x7fae400ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:50.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.631+0000 7fae45499640 1 --2- 172.21.15.105:0/1733242489 >> v2:172.21.15.148:3300/0 conn(0x7fae40159610 0x7fae400ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.631+0000 7fae45499640 1 -- 172.21.15.105:0/1733242489 >> v1:172.21.15.148:6789/0 conn(0x7fae4006c7e0 legacy=0x7fae400abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:50.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.631+0000 7fae45499640 1 -- 172.21.15.105:0/1733242489 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae400cd220 con 0x7fae400cc760 2023-11-28T18:26:50.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.635+0000 7fae45499640 1 --2- 172.21.15.105:0/1733242489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae400cc760 0x7fae400ccbf0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fae400cc6b0 tx=0x7fae400acca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ae6ac4b3101c4f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:50.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.635+0000 7fae44c98640 1 -- 172.21.15.105:0/1733242489 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae300b1600 con 0x7fae400cc760 2023-11-28T18:26:50.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.635+0000 7fae44c98640 1 -- 172.21.15.105:0/1733242489 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae300b1760 con 0x7fae400cc760 2023-11-28T18:26:50.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.635+0000 7fae44c98640 1 -- 172.21.15.105:0/1733242489 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae300b1af0 con 0x7fae400cc760 2023-11-28T18:26:50.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.635+0000 7fae47efe640 1 -- 172.21.15.105:0/1733242489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae400cc760 msgr2=0x7fae400ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:50.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.635+0000 7fae47efe640 1 --2- 172.21.15.105:0/1733242489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae400cc760 0x7fae400ccbf0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fae400cc6b0 tx=0x7fae400acca0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.635+0000 7fae47efe640 1 -- 172.21.15.105:0/1733242489 shutdown_connections 2023-11-28T18:26:50.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.635+0000 7fae47efe640 1 --2- 172.21.15.105:0/1733242489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae400cc760 0x7fae400ccbf0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.635+0000 7fae47efe640 1 --2- 172.21.15.105:0/1733242489 >> v2:172.21.15.148:3300/0 conn(0x7fae40159610 0x7fae400ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.635+0000 7fae47efe640 1 -- 172.21.15.105:0/1733242489 >> 172.21.15.105:0/1733242489 conn(0x7fae400ceb90 msgr2=0x7fae400cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:50.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.635+0000 7fae47efe640 1 -- 172.21.15.105:0/1733242489 shutdown_connections 2023-11-28T18:26:50.640 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.635+0000 7fae47efe640 1 -- 172.21.15.105:0/1733242489 wait complete. 2023-11-28T18:26:50.640 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.635+0000 7fae47efe640 1 Processor -- start 2023-11-28T18:26:50.640 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.635+0000 7fae47efe640 1 -- start start 2023-11-28T18:26:50.643 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae47efe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae4006c7e0 0x7fae401520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:50.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae47efe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae400cc760 0x7fae40099fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:50.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae47efe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae40159610 0x7fae4009c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:50.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae47efe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4009ac00 con 0x7fae40159610 2023-11-28T18:26:50.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae47efe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae40150d70 con 0x7fae400cc760 2023-11-28T18:26:50.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae47efe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae40151010 con 0x7fae4006c7e0 2023-11-28T18:26:50.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae4649b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae40159610 0x7fae4009c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:50.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae45499640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae400cc760 0x7fae40099fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:50.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae4649b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae40159610 0x7fae4009c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42456/0 (socket says 172.21.15.105:42456) 2023-11-28T18:26:50.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae4649b640 1 -- 172.21.15.105:0/2311015320 learned_addr learned my addr 172.21.15.105:0/2311015320 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:50.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae45c9a640 1 --2- 172.21.15.105:0/2311015320 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae4006c7e0 0x7fae401520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:50.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae4649b640 1 -- 172.21.15.105:0/2311015320 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae4006c7e0 msgr2=0x7fae401520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:50.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae4649b640 1 --2- 172.21.15.105:0/2311015320 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae4006c7e0 0x7fae401520b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae4649b640 1 -- 172.21.15.105:0/2311015320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae400cc760 msgr2=0x7fae40099fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:50.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae4649b640 1 --2- 172.21.15.105:0/2311015320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae400cc760 0x7fae40099fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae4649b640 1 -- 172.21.15.105:0/2311015320 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae300094d0 con 0x7fae40159610 2023-11-28T18:26:50.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae4649b640 1 --2- 172.21.15.105:0/2311015320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae40159610 0x7fae4009c140 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fae3800c770 tx=0x7fae3800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:50.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae367fc640 1 -- 172.21.15.105:0/2311015320 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38003b80 con 0x7fae40159610 2023-11-28T18:26:50.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae367fc640 1 -- 172.21.15.105:0/2311015320 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae38003ce0 con 0x7fae40159610 2023-11-28T18:26:50.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae47efe640 1 -- 172.21.15.105:0/2311015320 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae4006d180 con 0x7fae40159610 2023-11-28T18:26:50.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae367fc640 1 -- 172.21.15.105:0/2311015320 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3800aa90 con 0x7fae40159610 2023-11-28T18:26:50.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae47efe640 1 -- 172.21.15.105:0/2311015320 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae401515d0 con 0x7fae40159610 2023-11-28T18:26:50.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae47efe640 1 -- 172.21.15.105:0/2311015320 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae40067570 con 0x7fae40159610 2023-11-28T18:26:50.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.639+0000 7fae367fc640 1 -- 172.21.15.105:0/2311015320 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fae3800abf0 con 0x7fae40159610 2023-11-28T18:26:50.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.643+0000 7fae367fc640 1 --2- 172.21.15.105:0/2311015320 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae240668d0 0x7fae24068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:50.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.643+0000 7fae367fc640 1 -- 172.21.15.105:0/2311015320 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7fae380843a0 con 0x7fae40159610 2023-11-28T18:26:50.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.643+0000 7fae45c9a640 1 --2- 172.21.15.105:0/2311015320 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae240668d0 0x7fae24068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:50.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.643+0000 7fae45c9a640 1 --2- 172.21.15.105:0/2311015320 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae240668d0 0x7fae24068d90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009ac0 tx=0x7fae3c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:50.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.647+0000 7fae367fc640 1 -- 172.21.15.105:0/2311015320 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fae38051420 con 0x7fae40159610 2023-11-28T18:26:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.795+0000 7fae47efe640 1 -- 172.21.15.105:0/2311015320 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae400cf950 con 0x7fae40159610 2023-11-28T18:26:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.795+0000 7fae367fc640 1 -- 172.21.15.105:0/2311015320 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+17059 (secure 0 0 0) 0x7fae38054a40 con 0x7fae40159610 2023-11-28T18:26:50.801 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:50.801 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":54,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:26:28.339225+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:25:58.881270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T18:25:59.532434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.39","mappings":[{"from":0,"to":6}]},{"pgid":"8.3d","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6835/897997046":"2023-11-29T18:26:28.339203+0000","172.21.15.148:6833/897997046":"2023-11-29T18:26:28.339203+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:26:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.799+0000 7fae47efe640 1 -- 172.21.15.105:0/2311015320 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae240668d0 msgr2=0x7fae24068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.799+0000 7fae47efe640 1 --2- 172.21.15.105:0/2311015320 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae240668d0 0x7fae24068d90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009ac0 tx=0x7fae3c009190 comp rx=0 tx=0).stop 2023-11-28T18:26:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.799+0000 7fae47efe640 1 -- 172.21.15.105:0/2311015320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae40159610 msgr2=0x7fae4009c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.799+0000 7fae47efe640 1 --2- 172.21.15.105:0/2311015320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae40159610 0x7fae4009c140 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fae3800c770 tx=0x7fae3800cc30 comp rx=0 tx=0).stop 2023-11-28T18:26:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.799+0000 7fae47efe640 1 -- 172.21.15.105:0/2311015320 shutdown_connections 2023-11-28T18:26:50.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.799+0000 7fae47efe640 1 --2- 172.21.15.105:0/2311015320 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae240668d0 0x7fae24068d90 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.799+0000 7fae47efe640 1 --2- 172.21.15.105:0/2311015320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae40159610 0x7fae4009c140 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.799+0000 7fae47efe640 1 --2- 172.21.15.105:0/2311015320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae400cc760 0x7fae40099fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.799+0000 7fae47efe640 1 --2- 172.21.15.105:0/2311015320 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae4006c7e0 0x7fae401520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.799+0000 7fae47efe640 1 -- 172.21.15.105:0/2311015320 >> 172.21.15.105:0/2311015320 conn(0x7fae400ceb90 msgr2=0x7fae40071170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.799+0000 7fae47efe640 1 -- 172.21.15.105:0/2311015320 shutdown_connections 2023-11-28T18:26:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.799+0000 7fae47efe640 1 -- 172.21.15.105:0/2311015320 wait complete. 2023-11-28T18:26:50.822 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T18:26:50.822 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:26:50.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.955+0000 7f1d39ae3640 1 Processor -- start 2023-11-28T18:26:50.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.955+0000 7f1d39ae3640 1 -- start start 2023-11-28T18:26:50.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.959+0000 7f1d39ae3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1d34159f40 0x7f1d340caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:50.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.959+0000 7f1d39ae3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d340ad850 0x7f1d340701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:50.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.959+0000 7f1d39ae3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d3406bf50 con 0x7f1d340cc760 2023-11-28T18:26:50.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.959+0000 7f1d39ae3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d340ccbc0 con 0x7f1d34159f40 2023-11-28T18:26:50.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.959+0000 7f1d39ae3640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3415a320 con 0x7f1d340ad850 2023-11-28T18:26:50.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.959+0000 7f1d337fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1d34159f40 0x7f1d340caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:50.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.959+0000 7f1d33fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d340ad850 0x7f1d340701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:50.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.959+0000 7f1d2bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f1d340cc760 0x7f1d340ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:55748/0 (socket says 172.21.15.105:55748) 2023-11-28T18:26:50.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.959+0000 7f1d2bfff640 1 -- 172.21.15.105:0/2200109166 learned_addr learned my addr 172.21.15.105:0/2200109166 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:50.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.959+0000 7f1d337fe640 1 -- 172.21.15.105:0/2200109166 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d340ad850 msgr2=0x7f1d340701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:50.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.959+0000 7f1d337fe640 1 --2- 172.21.15.105:0/2200109166 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d340ad850 0x7f1d340701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.959+0000 7f1d337fe640 1 -- 172.21.15.105:0/2200109166 >> v1:172.21.15.148:6789/0 conn(0x7f1d340cc760 legacy=0x7f1d340ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:50.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.959+0000 7f1d337fe640 1 -- 172.21.15.105:0/2200109166 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d34071380 con 0x7f1d34159f40 2023-11-28T18:26:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.959+0000 7f1d337fe640 1 --2- 172.21.15.105:0/2200109166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d34159f40 0x7f1d340caf60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1d240097c0 tx=0x7f1d2401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa671c2585380f52 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.963+0000 7f1d32ffd640 1 -- 172.21.15.105:0/2200109166 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d240ac620 con 0x7f1d34159f40 2023-11-28T18:26:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.963+0000 7f1d32ffd640 1 -- 172.21.15.105:0/2200109166 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d240ac780 con 0x7f1d34159f40 2023-11-28T18:26:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.963+0000 7f1d32ffd640 1 -- 172.21.15.105:0/2200109166 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d240acb10 con 0x7f1d34159f40 2023-11-28T18:26:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.963+0000 7f1d39ae3640 1 -- 172.21.15.105:0/2200109166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d34159f40 msgr2=0x7f1d340caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.963+0000 7f1d39ae3640 1 --2- 172.21.15.105:0/2200109166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d34159f40 0x7f1d340caf60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1d240097c0 tx=0x7f1d2401c3c0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.963+0000 7f1d39ae3640 1 -- 172.21.15.105:0/2200109166 shutdown_connections 2023-11-28T18:26:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.963+0000 7f1d39ae3640 1 --2- 172.21.15.105:0/2200109166 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d340ad850 0x7f1d340701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.963+0000 7f1d39ae3640 1 --2- 172.21.15.105:0/2200109166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d34159f40 0x7f1d340caf60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.963+0000 7f1d39ae3640 1 -- 172.21.15.105:0/2200109166 >> 172.21.15.105:0/2200109166 conn(0x7f1d340ceb90 msgr2=0x7f1d340cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.963+0000 7f1d39ae3640 1 -- 172.21.15.105:0/2200109166 shutdown_connections 2023-11-28T18:26:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.963+0000 7f1d39ae3640 1 -- 172.21.15.105:0/2200109166 wait complete. 2023-11-28T18:26:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.963+0000 7f1d39ae3640 1 Processor -- start 2023-11-28T18:26:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.963+0000 7f1d39ae3640 1 -- start start 2023-11-28T18:26:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.963+0000 7f1d39ae3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d340ad850 0x7f1d34148270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.963+0000 7f1d39ae3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d340cc760 0x7f1d34149970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.963+0000 7f1d39ae3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d34159f40 0x7f1d3414e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.963+0000 7f1d39ae3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3414a3e0 con 0x7f1d340cc760 2023-11-28T18:26:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.963+0000 7f1d39ae3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3414a6c0 con 0x7f1d340ad850 2023-11-28T18:26:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.963+0000 7f1d39ae3640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3414a9a0 con 0x7f1d34159f40 2023-11-28T18:26:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.967+0000 7f1d2bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d340cc760 0x7f1d34149970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.967+0000 7f1d337fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d340ad850 0x7f1d34148270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.967+0000 7f1d2bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d340cc760 0x7f1d34149970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42470/0 (socket says 172.21.15.105:42470) 2023-11-28T18:26:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.967+0000 7f1d2bfff640 1 -- 172.21.15.105:0/1564367115 learned_addr learned my addr 172.21.15.105:0/1564367115 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.967+0000 7f1d33fff640 1 --2- 172.21.15.105:0/1564367115 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d34159f40 0x7f1d3414e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.967+0000 7f1d2bfff640 1 -- 172.21.15.105:0/1564367115 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d34159f40 msgr2=0x7f1d3414e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.967+0000 7f1d2bfff640 1 --2- 172.21.15.105:0/1564367115 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d34159f40 0x7f1d3414e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.967+0000 7f1d2bfff640 1 -- 172.21.15.105:0/1564367115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d340ad850 msgr2=0x7f1d34148270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.967+0000 7f1d2bfff640 1 --2- 172.21.15.105:0/1564367115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d340ad850 0x7f1d34148270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.967+0000 7f1d2bfff640 1 -- 172.21.15.105:0/1564367115 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d24009420 con 0x7f1d340cc760 2023-11-28T18:26:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.967+0000 7f1d337fe640 1 --2- 172.21.15.105:0/1564367115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d340ad850 0x7f1d34148270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:26:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.967+0000 7f1d2bfff640 1 --2- 172.21.15.105:0/1564367115 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d340cc760 0x7f1d34149970 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00c370 tx=0x7f1d1c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.967+0000 7f1d30ff9640 1 -- 172.21.15.105:0/1564367115 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c005cf0 con 0x7f1d340cc760 2023-11-28T18:26:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.967+0000 7f1d30ff9640 1 -- 172.21.15.105:0/1564367115 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1c005e50 con 0x7f1d340cc760 2023-11-28T18:26:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.967+0000 7f1d39ae3640 1 -- 172.21.15.105:0/1564367115 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d3414ac80 con 0x7f1d340cc760 2023-11-28T18:26:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.967+0000 7f1d30ff9640 1 -- 172.21.15.105:0/1564367115 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c0072f0 con 0x7f1d340cc760 2023-11-28T18:26:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.967+0000 7f1d39ae3640 1 -- 172.21.15.105:0/1564367115 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d341487b0 con 0x7f1d340cc760 2023-11-28T18:26:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.967+0000 7f1d39ae3640 1 -- 172.21.15.105:0/1564367115 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d34067570 con 0x7f1d340cc760 2023-11-28T18:26:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.967+0000 7f1d30ff9640 1 -- 172.21.15.105:0/1564367115 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f1d1c007450 con 0x7f1d340cc760 2023-11-28T18:26:50.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.971+0000 7f1d30ff9640 1 --2- 172.21.15.105:0/1564367115 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1d140668d0 0x7f1d14068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:50.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.971+0000 7f1d30ff9640 1 -- 172.21.15.105:0/1564367115 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f1d1c083ad0 con 0x7f1d340cc760 2023-11-28T18:26:50.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.971+0000 7f1d337fe640 1 --2- 172.21.15.105:0/1564367115 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1d140668d0 0x7f1d14068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:50.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.971+0000 7f1d337fe640 1 --2- 172.21.15.105:0/1564367115 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1d140668d0 0x7f1d14068d90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1d240ae000 tx=0x7f1d240a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:50.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:50.975+0000 7f1d30ff9640 1 -- 172.21.15.105:0/1564367115 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1d1c050be0 con 0x7f1d340cc760 2023-11-28T18:26:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.151+0000 7f1d39ae3640 1 -- 172.21.15.105:0/1564367115 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d340ce120 con 0x7f1d340cc760 2023-11-28T18:26:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.155+0000 7f1d30ff9640 1 -- 172.21.15.105:0/1564367115 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4955 (secure 0 0 0) 0x7f1d1c017020 con 0x7f1d340cc760 2023-11-28T18:26:51.158 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:51.158 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":47,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1686803161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":1686803161},{"type":"v1","addr":"172.21.15.148:6835","nonce":1686803161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:26:00.550298+0000","modified":"2023-11-28T18:26:31.373860+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5842,"mds_1":5405,"mds_2":5408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5405":{"gid":5405,"name":"a","rank":1,"incarnation":43,"state":"up:active","state_seq":13,"addr":"172.21.15.105:6836/735646078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":735646078},{"type":"v1","addr":"172.21.15.105:6836","nonce":735646078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5408":{"gid":5408,"name":"c","rank":2,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6837/3765684108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":3765684108},{"type":"v1","addr":"172.21.15.105:6837","nonce":3765684108}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5842":{"gid":5842,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6834/2696773413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2696773413},{"type":"v1","addr":"172.21.15.148:6834","nonce":2696773413}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T18:26:51.158 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 47 2023-11-28T18:26:51.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.155+0000 7f1d39ae3640 1 -- 172.21.15.105:0/1564367115 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1d140668d0 msgr2=0x7f1d14068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:51.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.155+0000 7f1d39ae3640 1 --2- 172.21.15.105:0/1564367115 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1d140668d0 0x7f1d14068d90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1d240ae000 tx=0x7f1d240a7000 comp rx=0 tx=0).stop 2023-11-28T18:26:51.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.155+0000 7f1d39ae3640 1 -- 172.21.15.105:0/1564367115 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d340cc760 msgr2=0x7f1d34149970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.155+0000 7f1d39ae3640 1 --2- 172.21.15.105:0/1564367115 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d340cc760 0x7f1d34149970 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00c370 tx=0x7f1d1c00c830 comp rx=0 tx=0).stop 2023-11-28T18:26:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.155+0000 7f1d39ae3640 1 -- 172.21.15.105:0/1564367115 shutdown_connections 2023-11-28T18:26:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.155+0000 7f1d39ae3640 1 --2- 172.21.15.105:0/1564367115 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d34159f40 0x7f1d3414e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.155+0000 7f1d39ae3640 1 --2- 172.21.15.105:0/1564367115 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1d140668d0 0x7f1d14068d90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.155+0000 7f1d39ae3640 1 --2- 172.21.15.105:0/1564367115 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d340cc760 0x7f1d34149970 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.155+0000 7f1d39ae3640 1 --2- 172.21.15.105:0/1564367115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d340ad850 0x7f1d34148270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.155+0000 7f1d39ae3640 1 -- 172.21.15.105:0/1564367115 >> 172.21.15.105:0/1564367115 conn(0x7f1d340ceb90 msgr2=0x7f1d3415a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.155+0000 7f1d39ae3640 1 -- 172.21.15.105:0/1564367115 shutdown_connections 2023-11-28T18:26:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.155+0000 7f1d39ae3640 1 -- 172.21.15.105:0/1564367115 wait complete. 2023-11-28T18:26:51.179 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:26:51.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.315+0000 7fd112217640 1 Processor -- start 2023-11-28T18:26:51.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.315+0000 7fd112217640 1 -- start start 2023-11-28T18:26:51.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.315+0000 7fd112217640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd10c15cef0 0x7fd10c0d1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:51.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.315+0000 7fd112217640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd10c0ce840 con 0x7fd10c0cfb30 2023-11-28T18:26:51.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.315+0000 7fd112217640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd10c0cd990 con 0x7fd10c0d1b40 2023-11-28T18:26:51.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.315+0000 7fd112217640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c0b4780 con 0x7fd10c15cef0 2023-11-28T18:26:51.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.315+0000 7fd111a16640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd10c0cfb30 0x7fd10c166fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:53494/0 (socket says 172.21.15.105:53494) 2023-11-28T18:26:51.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.315+0000 7fd111a16640 1 -- 172.21.15.105:0/4014689749 learned_addr learned my addr 172.21.15.105:0/4014689749 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:51.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.315+0000 7fd111215640 1 --2- 172.21.15.105:0/4014689749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd10c15cef0 0x7fd10c0d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:51.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.315+0000 7fd103fff640 1 -- 172.21.15.105:0/4014689749 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1264418358 0 0) 0x7fd10c0cd990 con 0x7fd10c0d1b40 2023-11-28T18:26:51.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.315+0000 7fd103fff640 1 -- 172.21.15.105:0/4014689749 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0f8003530 con 0x7fd10c0d1b40 2023-11-28T18:26:51.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.315+0000 7fd103fff640 1 -- 172.21.15.105:0/4014689749 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1498051737 0 0) 0x7fd10c0ce840 con 0x7fd10c0cfb30 2023-11-28T18:26:51.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.315+0000 7fd103fff640 1 -- 172.21.15.105:0/4014689749 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd10c0cd990 con 0x7fd10c0cfb30 2023-11-28T18:26:51.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.315+0000 7fd111215640 1 -- 172.21.15.105:0/4014689749 >> v1:172.21.15.148:6789/0 conn(0x7fd10c0d1b40 legacy=0x7fd10c0cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:51.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.315+0000 7fd111215640 1 -- 172.21.15.105:0/4014689749 >> v1:172.21.15.105:6789/0 conn(0x7fd10c0cfb30 legacy=0x7fd10c166fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:51.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.315+0000 7fd111215640 1 -- 172.21.15.105:0/4014689749 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10c0cecd0 con 0x7fd10c15cef0 2023-11-28T18:26:51.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.319+0000 7fd111215640 1 --2- 172.21.15.105:0/4014689749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd10c15cef0 0x7fd10c0d1600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd1080097f0 tx=0x7fd10801c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce849073083ffe91 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:51.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.319+0000 7fd103fff640 1 -- 172.21.15.105:0/4014689749 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1080ab600 con 0x7fd10c15cef0 2023-11-28T18:26:51.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.319+0000 7fd103fff640 1 -- 172.21.15.105:0/4014689749 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1080ab760 con 0x7fd10c15cef0 2023-11-28T18:26:51.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.319+0000 7fd103fff640 1 -- 172.21.15.105:0/4014689749 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1080abaf0 con 0x7fd10c15cef0 2023-11-28T18:26:51.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.319+0000 7fd112217640 1 -- 172.21.15.105:0/4014689749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd10c15cef0 msgr2=0x7fd10c0d1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:51.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.319+0000 7fd112217640 1 --2- 172.21.15.105:0/4014689749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd10c15cef0 0x7fd10c0d1600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd1080097f0 tx=0x7fd10801c400 comp rx=0 tx=0).stop 2023-11-28T18:26:51.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd112217640 1 -- 172.21.15.105:0/4014689749 shutdown_connections 2023-11-28T18:26:51.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd112217640 1 --2- 172.21.15.105:0/4014689749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd10c15cef0 0x7fd10c0d1600 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:51.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd112217640 1 -- 172.21.15.105:0/4014689749 >> 172.21.15.105:0/4014689749 conn(0x7fd10c0d2e10 msgr2=0x7fd10c0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:51.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd112217640 1 -- 172.21.15.105:0/4014689749 shutdown_connections 2023-11-28T18:26:51.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd112217640 1 -- 172.21.15.105:0/4014689749 wait complete. 2023-11-28T18:26:51.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd112217640 1 Processor -- start 2023-11-28T18:26:51.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd112217640 1 -- start start 2023-11-28T18:26:51.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd112217640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd10c0cfb30 0x7fd10c154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:51.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd112217640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd10c0d1b40 0x7fd10c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd112217640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd10c15cef0 0x7fd10c0b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd112217640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c15b180 con 0x7fd10c0d1b40 2023-11-28T18:26:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd112217640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c15a570 con 0x7fd10c15cef0 2023-11-28T18:26:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd112217640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c15a850 con 0x7fd10c0cfb30 2023-11-28T18:26:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd110a14640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd10c0d1b40 0x7fd10c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd111215640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd10c0cfb30 0x7fd10c154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd110a14640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd10c0d1b40 0x7fd10c0b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50660/0 (socket says 172.21.15.105:50660) 2023-11-28T18:26:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd110a14640 1 -- 172.21.15.105:0/3543727056 learned_addr learned my addr 172.21.15.105:0/3543727056 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd111a16640 1 --2- 172.21.15.105:0/3543727056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd10c15cef0 0x7fd10c0b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd110a14640 1 -- 172.21.15.105:0/3543727056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd10c0cfb30 msgr2=0x7fd10c154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd110a14640 1 --2- 172.21.15.105:0/3543727056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd10c0cfb30 0x7fd10c154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:51.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd110a14640 1 -- 172.21.15.105:0/3543727056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd10c15cef0 msgr2=0x7fd10c0b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:51.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd110a14640 1 --2- 172.21.15.105:0/3543727056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd10c15cef0 0x7fd10c0b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:51.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd110a14640 1 -- 172.21.15.105:0/3543727056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd108009420 con 0x7fd10c0d1b40 2023-11-28T18:26:51.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.323+0000 7fd111215640 1 --2- 172.21.15.105:0/3543727056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd10c0cfb30 0x7fd10c154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:26:51.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.327+0000 7fd110a14640 1 --2- 172.21.15.105:0/3543727056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd10c0d1b40 0x7fd10c0b3260 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00c460 tx=0x7fd0fc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:51.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.327+0000 7fd101ffb640 1 -- 172.21.15.105:0/3543727056 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc0052d0 con 0x7fd10c0d1b40 2023-11-28T18:26:51.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.327+0000 7fd101ffb640 1 -- 172.21.15.105:0/3543727056 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0fc005430 con 0x7fd10c0d1b40 2023-11-28T18:26:51.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.327+0000 7fd101ffb640 1 -- 172.21.15.105:0/3543727056 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc012870 con 0x7fd10c0d1b40 2023-11-28T18:26:51.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.327+0000 7fd112217640 1 -- 172.21.15.105:0/3543727056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd10c15ab30 con 0x7fd10c0d1b40 2023-11-28T18:26:51.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.327+0000 7fd112217640 1 -- 172.21.15.105:0/3543727056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd10c155390 con 0x7fd10c0d1b40 2023-11-28T18:26:51.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.327+0000 7fd112217640 1 -- 172.21.15.105:0/3543727056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd10c15b330 con 0x7fd10c0d1b40 2023-11-28T18:26:51.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.327+0000 7fd101ffb640 1 -- 172.21.15.105:0/3543727056 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd0fc0129d0 con 0x7fd10c0d1b40 2023-11-28T18:26:51.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.327+0000 7fd101ffb640 1 --2- 172.21.15.105:0/3543727056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd0f00668d0 0x7fd0f0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:51.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.327+0000 7fd101ffb640 1 -- 172.21.15.105:0/3543727056 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7fd0fc084060 con 0x7fd10c0d1b40 2023-11-28T18:26:51.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.327+0000 7fd111215640 1 --2- 172.21.15.105:0/3543727056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd0f00668d0 0x7fd0f0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:51.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.327+0000 7fd111215640 1 --2- 172.21.15.105:0/3543727056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd0f00668d0 0x7fd0f0068d90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd1080af000 tx=0x7fd1080b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:51.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.331+0000 7fd101ffb640 1 -- 172.21.15.105:0/3543727056 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd10c15b330 con 0x7fd10c0d1b40 2023-11-28T18:26:51.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.515+0000 7fd112217640 1 -- 172.21.15.105:0/3543727056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd10c0679b0 con 0x7fd10c0d1b40 2023-11-28T18:26:51.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.515+0000 7fd101ffb640 1 -- 172.21.15.105:0/3543727056 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4955 (secure 0 0 0) 0x7fd0fc054c10 con 0x7fd10c0d1b40 2023-11-28T18:26:51.519 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:51.519 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":47,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1686803161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":1686803161},{"type":"v1","addr":"172.21.15.148:6835","nonce":1686803161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:26:00.550298+0000","modified":"2023-11-28T18:26:31.373860+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5842,"mds_1":5405,"mds_2":5408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5405":{"gid":5405,"name":"a","rank":1,"incarnation":43,"state":"up:active","state_seq":13,"addr":"172.21.15.105:6836/735646078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":735646078},{"type":"v1","addr":"172.21.15.105:6836","nonce":735646078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5408":{"gid":5408,"name":"c","rank":2,"incarnation":39,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6837/3765684108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":3765684108},{"type":"v1","addr":"172.21.15.105:6837","nonce":3765684108}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5842":{"gid":5842,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6834/2696773413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2696773413},{"type":"v1","addr":"172.21.15.148:6834","nonce":2696773413}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T18:26:51.520 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 47 2023-11-28T18:26:51.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.515+0000 7fd112217640 1 -- 172.21.15.105:0/3543727056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd0f00668d0 msgr2=0x7fd0f0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:51.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.515+0000 7fd112217640 1 --2- 172.21.15.105:0/3543727056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd0f00668d0 0x7fd0f0068d90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd1080af000 tx=0x7fd1080b6040 comp rx=0 tx=0).stop 2023-11-28T18:26:51.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.515+0000 7fd112217640 1 -- 172.21.15.105:0/3543727056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd10c0d1b40 msgr2=0x7fd10c0b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:51.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.515+0000 7fd112217640 1 --2- 172.21.15.105:0/3543727056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd10c0d1b40 0x7fd10c0b3260 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00c460 tx=0x7fd0fc00c920 comp rx=0 tx=0).stop 2023-11-28T18:26:51.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.515+0000 7fd112217640 1 -- 172.21.15.105:0/3543727056 shutdown_connections 2023-11-28T18:26:51.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.515+0000 7fd112217640 1 --2- 172.21.15.105:0/3543727056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd10c15cef0 0x7fd10c0b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:51.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.515+0000 7fd112217640 1 --2- 172.21.15.105:0/3543727056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd0f00668d0 0x7fd0f0068d90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:51.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.515+0000 7fd112217640 1 --2- 172.21.15.105:0/3543727056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd10c0d1b40 0x7fd10c0b3260 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:51.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.515+0000 7fd112217640 1 --2- 172.21.15.105:0/3543727056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd10c0cfb30 0x7fd10c154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:51.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.515+0000 7fd112217640 1 -- 172.21.15.105:0/3543727056 >> 172.21.15.105:0/3543727056 conn(0x7fd10c0d2e10 msgr2=0x7fd10c0af980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:51.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.519+0000 7fd112217640 1 -- 172.21.15.105:0/3543727056 shutdown_connections 2023-11-28T18:26:51.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.519+0000 7fd112217640 1 -- 172.21.15.105:0/3543727056 wait complete. 2023-11-28T18:26:51.540 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:26:51.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.671+0000 7f678a632640 1 Processor -- start 2023-11-28T18:26:51.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.671+0000 7f678a632640 1 -- start start 2023-11-28T18:26:51.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.675+0000 7f678a632640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6784159610 0x7f67840ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:51.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.675+0000 7f678a632640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f678406cef0 0x7f67840ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:51.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.675+0000 7f678a632640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67840cde30 con 0x7f67840cc760 2023-11-28T18:26:51.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.675+0000 7f678a632640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67841599f0 con 0x7f6784159610 2023-11-28T18:26:51.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.675+0000 7f678a632640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678406de20 con 0x7f678406cef0 2023-11-28T18:26:51.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.675+0000 7f6783fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6784159610 0x7f67840ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:51.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.675+0000 7f6783fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6784159610 0x7f67840ce2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50668/0 (socket says 172.21.15.105:50668) 2023-11-28T18:26:51.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.675+0000 7f6783fff640 1 -- 172.21.15.105:0/870995567 learned_addr learned my addr 172.21.15.105:0/870995567 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:51.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.675+0000 7f6788bcf640 1 --2- 172.21.15.105:0/870995567 >> v2:172.21.15.148:3300/0 conn(0x7f678406cef0 0x7f67840ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:51.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.675+0000 7f6783fff640 1 -- 172.21.15.105:0/870995567 >> v2:172.21.15.148:3300/0 conn(0x7f678406cef0 msgr2=0x7f67840ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:51.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.675+0000 7f6783fff640 1 --2- 172.21.15.105:0/870995567 >> v2:172.21.15.148:3300/0 conn(0x7f678406cef0 0x7f67840ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:51.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.675+0000 7f6783fff640 1 -- 172.21.15.105:0/870995567 >> v1:172.21.15.148:6789/0 conn(0x7f67840cc760 legacy=0x7f678406c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:51.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.675+0000 7f6783fff640 1 -- 172.21.15.105:0/870995567 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67840cd220 con 0x7f6784159610 2023-11-28T18:26:51.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.679+0000 7f6783fff640 1 --2- 172.21.15.105:0/870995567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6784159610 0x7f67840ce2c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f678406e600 tx=0x7f677401d8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510d3cb933e79236 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:51.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.679+0000 7f6782ffd640 1 -- 172.21.15.105:0/870995567 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67740a68d0 con 0x7f6784159610 2023-11-28T18:26:51.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.679+0000 7f6782ffd640 1 -- 172.21.15.105:0/870995567 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67740a6a30 con 0x7f6784159610 2023-11-28T18:26:51.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.679+0000 7f6782ffd640 1 -- 172.21.15.105:0/870995567 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67740a6dc0 con 0x7f6784159610 2023-11-28T18:26:51.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.679+0000 7f678a632640 1 -- 172.21.15.105:0/870995567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6784159610 msgr2=0x7f67840ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:51.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.679+0000 7f678a632640 1 --2- 172.21.15.105:0/870995567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6784159610 0x7f67840ce2c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f678406e600 tx=0x7f677401d8a0 comp rx=0 tx=0).stop 2023-11-28T18:26:51.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.679+0000 7f678a632640 1 -- 172.21.15.105:0/870995567 shutdown_connections 2023-11-28T18:26:51.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.679+0000 7f678a632640 1 --2- 172.21.15.105:0/870995567 >> v2:172.21.15.148:3300/0 conn(0x7f678406cef0 0x7f67840ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:51.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.679+0000 7f678a632640 1 --2- 172.21.15.105:0/870995567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6784159610 0x7f67840ce2c0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:51.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.679+0000 7f678a632640 1 -- 172.21.15.105:0/870995567 >> 172.21.15.105:0/870995567 conn(0x7f67840ceb90 msgr2=0x7f67840cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:51.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.679+0000 7f678a632640 1 -- 172.21.15.105:0/870995567 shutdown_connections 2023-11-28T18:26:51.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.679+0000 7f678a632640 1 -- 172.21.15.105:0/870995567 wait complete. 2023-11-28T18:26:51.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.679+0000 7f678a632640 1 Processor -- start 2023-11-28T18:26:51.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.679+0000 7f678a632640 1 -- start start 2023-11-28T18:26:51.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f678a632640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f678406cef0 0x7f6784143670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:51.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f678a632640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67840cc760 0x7f678409a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:51.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f678a632640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6784159610 0x7f678409a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:51.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f678a632640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6784143bb0 con 0x7f678406cef0 2023-11-28T18:26:51.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f678a632640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678409b570 con 0x7f67840cc760 2023-11-28T18:26:51.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f678a632640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678409afb0 con 0x7f6784159610 2023-11-28T18:26:51.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f6783fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f678406cef0 0x7f6784143670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:51.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f6783fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f678406cef0 0x7f6784143670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50678/0 (socket says 172.21.15.105:50678) 2023-11-28T18:26:51.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f6783fff640 1 -- 172.21.15.105:0/1505737305 learned_addr learned my addr 172.21.15.105:0/1505737305 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:51.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f6788bcf640 1 --2- 172.21.15.105:0/1505737305 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6784159610 0x7f678409a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:51.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f67837fe640 1 --2- 172.21.15.105:0/1505737305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67840cc760 0x7f678409a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:51.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f6783fff640 1 -- 172.21.15.105:0/1505737305 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6784159610 msgr2=0x7f678409a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:51.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f6783fff640 1 --2- 172.21.15.105:0/1505737305 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6784159610 0x7f678409a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:51.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f6783fff640 1 -- 172.21.15.105:0/1505737305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67840cc760 msgr2=0x7f678409a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:51.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f6783fff640 1 --2- 172.21.15.105:0/1505737305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67840cc760 0x7f678409a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:51.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f6783fff640 1 -- 172.21.15.105:0/1505737305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677400a670 con 0x7f678406cef0 2023-11-28T18:26:51.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f67837fe640 1 --2- 172.21.15.105:0/1505737305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67840cc760 0x7f678409a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:51.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f6783fff640 1 --2- 172.21.15.105:0/1505737305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f678406cef0 0x7f6784143670 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f677400a1c0 tx=0x7f677401c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:51.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f6780ff9640 1 -- 172.21.15.105:0/1505737305 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67740ae990 con 0x7f678406cef0 2023-11-28T18:26:51.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f6780ff9640 1 -- 172.21.15.105:0/1505737305 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67740aeaf0 con 0x7f678406cef0 2023-11-28T18:26:51.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f6780ff9640 1 -- 172.21.15.105:0/1505737305 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67740aeea0 con 0x7f678406cef0 2023-11-28T18:26:51.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f678a632640 1 -- 172.21.15.105:0/1505737305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678409b290 con 0x7f678406cef0 2023-11-28T18:26:51.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f678a632640 1 -- 172.21.15.105:0/1505737305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67841444c0 con 0x7f678406cef0 2023-11-28T18:26:51.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f678a632640 1 -- 172.21.15.105:0/1505737305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6784144040 con 0x7f678406cef0 2023-11-28T18:26:51.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.683+0000 7f6780ff9640 1 -- 172.21.15.105:0/1505737305 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f67740b5070 con 0x7f678406cef0 2023-11-28T18:26:51.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.687+0000 7f6780ff9640 1 --2- 172.21.15.105:0/1505737305 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f67700668d0 0x7f6770068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:51.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.687+0000 7f6780ff9640 1 -- 172.21.15.105:0/1505737305 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f677411f320 con 0x7f678406cef0 2023-11-28T18:26:51.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.687+0000 7f67837fe640 1 --2- 172.21.15.105:0/1505737305 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f67700668d0 0x7f6770068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:51.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.687+0000 7f67837fe640 1 --2- 172.21.15.105:0/1505737305 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f67700668d0 0x7f6770068d90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6778000f70 tx=0x7f6778009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:51.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.691+0000 7f6780ff9640 1 -- 172.21.15.105:0/1505737305 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6784144040 con 0x7f678406cef0 2023-11-28T18:26:51.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.835+0000 7f678a632640 1 -- 172.21.15.105:0/1505737305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6784144040 con 0x7f678406cef0 2023-11-28T18:26:51.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.839+0000 7f6780ff9640 1 -- 172.21.15.105:0/1505737305 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+17059 (secure 0 0 0) 0x7f6784144040 con 0x7f678406cef0 2023-11-28T18:26:51.842 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:51.843 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":54,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:26:28.339225+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:25:58.881270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T18:25:59.532434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.39","mappings":[{"from":0,"to":6}]},{"pgid":"8.3d","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6835/897997046":"2023-11-29T18:26:28.339203+0000","172.21.15.148:6833/897997046":"2023-11-29T18:26:28.339203+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:26:51.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.839+0000 7f678a632640 1 -- 172.21.15.105:0/1505737305 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f67700668d0 msgr2=0x7f6770068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:51.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.839+0000 7f678a632640 1 --2- 172.21.15.105:0/1505737305 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f67700668d0 0x7f6770068d90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6778000f70 tx=0x7f6778009040 comp rx=0 tx=0).stop 2023-11-28T18:26:51.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.839+0000 7f678a632640 1 -- 172.21.15.105:0/1505737305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f678406cef0 msgr2=0x7f6784143670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:51.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.839+0000 7f678a632640 1 --2- 172.21.15.105:0/1505737305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f678406cef0 0x7f6784143670 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f677400a1c0 tx=0x7f677401c0e0 comp rx=0 tx=0).stop 2023-11-28T18:26:51.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.839+0000 7f678a632640 1 -- 172.21.15.105:0/1505737305 shutdown_connections 2023-11-28T18:26:51.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.839+0000 7f678a632640 1 --2- 172.21.15.105:0/1505737305 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6784159610 0x7f678409a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:51.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.839+0000 7f678a632640 1 --2- 172.21.15.105:0/1505737305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67840cc760 0x7f678409a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:51.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.839+0000 7f678a632640 1 --2- 172.21.15.105:0/1505737305 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f67700668d0 0x7f6770068d90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:51.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.839+0000 7f678a632640 1 --2- 172.21.15.105:0/1505737305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f678406cef0 0x7f6784143670 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:51.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.839+0000 7f678a632640 1 -- 172.21.15.105:0/1505737305 >> 172.21.15.105:0/1505737305 conn(0x7f67840ceb90 msgr2=0x7f67841586a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:51.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.839+0000 7f678a632640 1 -- 172.21.15.105:0/1505737305 shutdown_connections 2023-11-28T18:26:51.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.839+0000 7f678a632640 1 -- 172.21.15.105:0/1505737305 wait complete. 2023-11-28T18:26:51.865 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T18:26:52.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.999+0000 7f0c25dd1640 1 Processor -- start 2023-11-28T18:26:52.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.999+0000 7f0c25dd1640 1 -- start start 2023-11-28T18:26:52.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.999+0000 7f0c25dd1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0c20154070 0x7f0c2015d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:52.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.999+0000 7f0c25dd1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0c200adb80 0x7f0c200adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.999+0000 7f0c25dd1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c200cc7f0 con 0x7f0c2006bf50 2023-11-28T18:26:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.999+0000 7f0c25dd1640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c20072540 con 0x7f0c20154070 2023-11-28T18:26:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:51.999+0000 7f0c25dd1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c200ae4f0 con 0x7f0c200adb80 2023-11-28T18:26:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.003+0000 7f0c1f7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0c20154070 0x7f0c2015d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.003+0000 7f0c1f7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0c20154070 0x7f0c2015d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50684/0 (socket says 172.21.15.105:50684) 2023-11-28T18:26:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.003+0000 7f0c1f7fe640 1 -- 172.21.15.105:0/3374056798 learned_addr learned my addr 172.21.15.105:0/3374056798 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.003+0000 7f0c1ffff640 1 --2- 172.21.15.105:0/3374056798 >> v2:172.21.15.148:3300/0 conn(0x7f0c200adb80 0x7f0c200adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.003+0000 7f0c1f7fe640 1 -- 172.21.15.105:0/3374056798 >> v2:172.21.15.148:3300/0 conn(0x7f0c200adb80 msgr2=0x7f0c200adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.003+0000 7f0c1f7fe640 1 --2- 172.21.15.105:0/3374056798 >> v2:172.21.15.148:3300/0 conn(0x7f0c200adb80 0x7f0c200adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:52.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.003+0000 7f0c1f7fe640 1 -- 172.21.15.105:0/3374056798 >> v1:172.21.15.148:6789/0 conn(0x7f0c2006bf50 legacy=0x7f0c200ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:52.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.003+0000 7f0c1f7fe640 1 -- 172.21.15.105:0/3374056798 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2015dd10 con 0x7f0c20154070 2023-11-28T18:26:52.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.003+0000 7f0c1f7fe640 1 --2- 172.21.15.105:0/3374056798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c20154070 0x7f0c2015d700 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0c2006e600 tx=0x7f0c1001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be5acfef7efabf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:52.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.003+0000 7f0c1e7fc640 1 -- 172.21.15.105:0/3374056798 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100ab600 con 0x7f0c20154070 2023-11-28T18:26:52.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.003+0000 7f0c1e7fc640 1 -- 172.21.15.105:0/3374056798 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c100a7050 con 0x7f0c20154070 2023-11-28T18:26:52.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.003+0000 7f0c1e7fc640 1 -- 172.21.15.105:0/3374056798 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100ab980 con 0x7f0c20154070 2023-11-28T18:26:52.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.007+0000 7f0c25dd1640 1 -- 172.21.15.105:0/3374056798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c20154070 msgr2=0x7f0c2015d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:52.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.007+0000 7f0c25dd1640 1 --2- 172.21.15.105:0/3374056798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c20154070 0x7f0c2015d700 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0c2006e600 tx=0x7f0c1001c4d0 comp rx=0 tx=0).stop 2023-11-28T18:26:52.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.007+0000 7f0c25dd1640 1 -- 172.21.15.105:0/3374056798 shutdown_connections 2023-11-28T18:26:52.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.007+0000 7f0c25dd1640 1 --2- 172.21.15.105:0/3374056798 >> v2:172.21.15.148:3300/0 conn(0x7f0c200adb80 0x7f0c200adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.007+0000 7f0c25dd1640 1 --2- 172.21.15.105:0/3374056798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c20154070 0x7f0c2015d700 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.007+0000 7f0c25dd1640 1 -- 172.21.15.105:0/3374056798 >> 172.21.15.105:0/3374056798 conn(0x7f0c200ceb90 msgr2=0x7f0c200cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.007+0000 7f0c25dd1640 1 -- 172.21.15.105:0/3374056798 shutdown_connections 2023-11-28T18:26:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.007+0000 7f0c25dd1640 1 -- 172.21.15.105:0/3374056798 wait complete. 2023-11-28T18:26:52.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.007+0000 7f0c25dd1640 1 Processor -- start 2023-11-28T18:26:52.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.007+0000 7f0c25dd1640 1 -- start start 2023-11-28T18:26:52.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.007+0000 7f0c25dd1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c2006bf50 0x7f0c20147fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:52.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.007+0000 7f0c25dd1640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c200adb80 0x7f0c201536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:52.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.007+0000 7f0c25dd1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c20154070 0x7f0c20148660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:52.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.007+0000 7f0c25dd1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c201484f0 con 0x7f0c2006bf50 2023-11-28T18:26:52.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.007+0000 7f0c25dd1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2014af80 con 0x7f0c20154070 2023-11-28T18:26:52.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.007+0000 7f0c25dd1640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2014aa40 con 0x7f0c200adb80 2023-11-28T18:26:52.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.007+0000 7f0c1f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c2006bf50 0x7f0c20147fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:52.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.007+0000 7f0c1f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c2006bf50 0x7f0c20147fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50694/0 (socket says 172.21.15.105:50694) 2023-11-28T18:26:52.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.007+0000 7f0c1f7fe640 1 -- 172.21.15.105:0/3533627655 learned_addr learned my addr 172.21.15.105:0/3533627655 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:52.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.011+0000 7f0c1effd640 1 --2- 172.21.15.105:0/3533627655 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c200adb80 0x7f0c201536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:52.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.011+0000 7f0c1ffff640 1 --2- 172.21.15.105:0/3533627655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c20154070 0x7f0c20148660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:52.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.011+0000 7f0c1f7fe640 1 -- 172.21.15.105:0/3533627655 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c200adb80 msgr2=0x7f0c201536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:52.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.011+0000 7f0c1f7fe640 1 --2- 172.21.15.105:0/3533627655 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c200adb80 0x7f0c201536b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:52.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.011+0000 7f0c1f7fe640 1 -- 172.21.15.105:0/3533627655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c20154070 msgr2=0x7f0c20148660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:52.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.011+0000 7f0c1f7fe640 1 --2- 172.21.15.105:0/3533627655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c20154070 0x7f0c20148660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:52.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.011+0000 7f0c1f7fe640 1 -- 172.21.15.105:0/3533627655 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c100094d0 con 0x7f0c2006bf50 2023-11-28T18:26:52.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.011+0000 7f0c1ffff640 1 --2- 172.21.15.105:0/3533627655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c20154070 0x7f0c20148660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:52.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.011+0000 7f0c1f7fe640 1 --2- 172.21.15.105:0/3533627655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c2006bf50 0x7f0c20147fb0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f0c100ae040 tx=0x7f0c10019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:52.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.011+0000 7f0c07fff640 1 -- 172.21.15.105:0/3533627655 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100b4620 con 0x7f0c2006bf50 2023-11-28T18:26:52.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.011+0000 7f0c07fff640 1 -- 172.21.15.105:0/3533627655 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c100b4780 con 0x7f0c2006bf50 2023-11-28T18:26:52.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.011+0000 7f0c25dd1640 1 -- 172.21.15.105:0/3533627655 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2014e880 con 0x7f0c2006bf50 2023-11-28T18:26:52.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.011+0000 7f0c07fff640 1 -- 172.21.15.105:0/3533627655 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100b4b30 con 0x7f0c2006bf50 2023-11-28T18:26:52.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.011+0000 7f0c25dd1640 1 -- 172.21.15.105:0/3533627655 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c20150410 con 0x7f0c2006bf50 2023-11-28T18:26:52.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.011+0000 7f0c25dd1640 1 -- 172.21.15.105:0/3533627655 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c20067570 con 0x7f0c2006bf50 2023-11-28T18:26:52.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.011+0000 7f0c07fff640 1 -- 172.21.15.105:0/3533627655 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f0c100b2070 con 0x7f0c2006bf50 2023-11-28T18:26:52.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.011+0000 7f0c07fff640 1 --2- 172.21.15.105:0/3533627655 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0c0c0668d0 0x7f0c0c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:52.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.011+0000 7f0c07fff640 1 -- 172.21.15.105:0/3533627655 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f0c1011d870 con 0x7f0c2006bf50 2023-11-28T18:26:52.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.015+0000 7f0c1effd640 1 --2- 172.21.15.105:0/3533627655 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0c0c0668d0 0x7f0c0c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:52.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.015+0000 7f0c1effd640 1 --2- 172.21.15.105:0/3533627655 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0c0c0668d0 0x7f0c0c068d90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0c200b4d50 tx=0x7f0c14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:52.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.015+0000 7f0c07fff640 1 -- 172.21.15.105:0/3533627655 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0c100eaa20 con 0x7f0c2006bf50 2023-11-28T18:26:52.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.195+0000 7f0c25dd1640 1 -- 172.21.15.105:0/3533627655 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0c200cf950 con 0x7f0c2006bf50 2023-11-28T18:26:52.200 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-11-28T18:26:52.195+0000 7fd89b32a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T18:26:52.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.199+0000 7f0c07fff640 1 -- 172.21.15.105:0/3533627655 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0c100ee040 con 0x7f0c2006bf50 2023-11-28T18:26:52.203 INFO:teuthology.orchestra.run.smithi105.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T18:26:52.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.203+0000 7f0c25dd1640 1 -- 172.21.15.105:0/3533627655 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0c0c0668d0 msgr2=0x7f0c0c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:52.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.203+0000 7f0c25dd1640 1 --2- 172.21.15.105:0/3533627655 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0c0c0668d0 0x7f0c0c068d90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0c200b4d50 tx=0x7f0c14009040 comp rx=0 tx=0).stop 2023-11-28T18:26:52.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.203+0000 7f0c25dd1640 1 -- 172.21.15.105:0/3533627655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c2006bf50 msgr2=0x7f0c20147fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:52.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.203+0000 7f0c25dd1640 1 --2- 172.21.15.105:0/3533627655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c2006bf50 0x7f0c20147fb0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f0c100ae040 tx=0x7f0c10019270 comp rx=0 tx=0).stop 2023-11-28T18:26:52.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.203+0000 7f0c25dd1640 1 -- 172.21.15.105:0/3533627655 shutdown_connections 2023-11-28T18:26:52.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.203+0000 7f0c25dd1640 1 --2- 172.21.15.105:0/3533627655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c20154070 0x7f0c20148660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:52.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.203+0000 7f0c25dd1640 1 --2- 172.21.15.105:0/3533627655 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c200adb80 0x7f0c201536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:52.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.203+0000 7f0c25dd1640 1 --2- 172.21.15.105:0/3533627655 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0c0c0668d0 0x7f0c0c068d90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:52.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.203+0000 7f0c25dd1640 1 --2- 172.21.15.105:0/3533627655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c2006bf50 0x7f0c20147fb0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:52.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.203+0000 7f0c25dd1640 1 -- 172.21.15.105:0/3533627655 >> 172.21.15.105:0/3533627655 conn(0x7f0c200ceb90 msgr2=0x7f0c20161090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:52.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.203+0000 7f0c25dd1640 1 -- 172.21.15.105:0/3533627655 shutdown_connections 2023-11-28T18:26:52.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.203+0000 7f0c25dd1640 1 -- 172.21.15.105:0/3533627655 wait complete. 2023-11-28T18:26:52.228 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T18:26:52.228 INFO:tasks.ceph.mds.b.smithi148.stderr: -2219> 2023-11-28T18:26:33.131+0000 7f609c6a2640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-28T18:26:52.274 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2023-11-28T18:26:52.276 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2023-11-28T18:26:52.286 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2023-11-28T18:26:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.375+0000 7ff3e19a7640 1 Processor -- start 2023-11-28T18:26:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.375+0000 7ff3e19a7640 1 -- start start 2023-11-28T18:26:52.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.375+0000 7ff3e19a7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff3dc15d350 0x7ff3dc0d38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:52.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.375+0000 7ff3e19a7640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3dc15d730 0x7ff3dc0b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:52.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.375+0000 7ff3e19a7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff3dc0b2400 0x7ff3dc0cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:52.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.375+0000 7ff3e19a7640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc0b28b0 con 0x7ff3dc0b2400 2023-11-28T18:26:52.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.375+0000 7ff3e19a7640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc0cdae0 con 0x7ff3dc15d350 2023-11-28T18:26:52.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.375+0000 7ff3e19a7640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0d1f40 con 0x7ff3dc15d730 2023-11-28T18:26:52.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.375+0000 7ff3e11a6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff3dc0b2400 0x7ff3dc0cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:52.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.375+0000 7ff3e11a6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff3dc0b2400 0x7ff3dc0cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50730/0 (socket says 172.21.15.105:50730) 2023-11-28T18:26:52.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.375+0000 7ff3e11a6640 1 -- 172.21.15.105:0/2834521208 learned_addr learned my addr 172.21.15.105:0/2834521208 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:52.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.375+0000 7ff3d3fff640 1 --2- 172.21.15.105:0/2834521208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3dc15d730 0x7ff3dc0b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:52.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.375+0000 7ff3e09a5640 1 --2- 172.21.15.105:0/2834521208 >> v2:172.21.15.148:3300/0 conn(0x7ff3dc15d350 0x7ff3dc0d38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:52.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.375+0000 7ff3e11a6640 1 -- 172.21.15.105:0/2834521208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3dc15d730 msgr2=0x7ff3dc0b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:52.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.375+0000 7ff3e11a6640 1 --2- 172.21.15.105:0/2834521208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3dc15d730 0x7ff3dc0b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:52.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.375+0000 7ff3e11a6640 1 -- 172.21.15.105:0/2834521208 >> v2:172.21.15.148:3300/0 conn(0x7ff3dc15d350 msgr2=0x7ff3dc0d38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.375+0000 7ff3e11a6640 1 --2- 172.21.15.105:0/2834521208 >> v2:172.21.15.148:3300/0 conn(0x7ff3dc15d350 0x7ff3dc0d38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.375+0000 7ff3e11a6640 1 -- 172.21.15.105:0/2834521208 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3dc0d16a0 con 0x7ff3dc0b2400 2023-11-28T18:26:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.375+0000 7ff3d3fff640 1 --2- 172.21.15.105:0/2834521208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3dc15d730 0x7ff3dc0b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.375+0000 7ff3e09a5640 1 --2- 172.21.15.105:0/2834521208 >> v2:172.21.15.148:3300/0 conn(0x7ff3dc15d350 0x7ff3dc0d38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.379+0000 7ff3e11a6640 1 --2- 172.21.15.105:0/2834521208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3dc0b2400 0x7ff3dc0cf680 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc08b520 tx=0x7ff3d401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb16807f98457219 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:52.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.379+0000 7ff3d37fe640 1 -- 172.21.15.105:0/2834521208 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d40a55c0 con 0x7ff3dc0b2400 2023-11-28T18:26:52.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.379+0000 7ff3d37fe640 1 -- 172.21.15.105:0/2834521208 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d40a5720 con 0x7ff3dc0b2400 2023-11-28T18:26:52.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.379+0000 7ff3d37fe640 1 -- 172.21.15.105:0/2834521208 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d40a5ab0 con 0x7ff3dc0b2400 2023-11-28T18:26:52.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.379+0000 7ff3e19a7640 1 -- 172.21.15.105:0/2834521208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3dc0b2400 msgr2=0x7ff3dc0cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:52.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.379+0000 7ff3e19a7640 1 --2- 172.21.15.105:0/2834521208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3dc0b2400 0x7ff3dc0cf680 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc08b520 tx=0x7ff3d401c940 comp rx=0 tx=0).stop 2023-11-28T18:26:52.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.379+0000 7ff3e19a7640 1 -- 172.21.15.105:0/2834521208 shutdown_connections 2023-11-28T18:26:52.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.379+0000 7ff3e19a7640 1 --2- 172.21.15.105:0/2834521208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3dc0b2400 0x7ff3dc0cf680 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:52.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.379+0000 7ff3e19a7640 1 --2- 172.21.15.105:0/2834521208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3dc15d730 0x7ff3dc0b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:52.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.379+0000 7ff3e19a7640 1 --2- 172.21.15.105:0/2834521208 >> v2:172.21.15.148:3300/0 conn(0x7ff3dc15d350 0x7ff3dc0d38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:52.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.379+0000 7ff3e19a7640 1 -- 172.21.15.105:0/2834521208 >> 172.21.15.105:0/2834521208 conn(0x7ff3dc0d2d90 msgr2=0x7ff3dc0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:52.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.379+0000 7ff3e19a7640 1 -- 172.21.15.105:0/2834521208 shutdown_connections 2023-11-28T18:26:52.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.379+0000 7ff3e19a7640 1 -- 172.21.15.105:0/2834521208 wait complete. 2023-11-28T18:26:52.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3e19a7640 1 Processor -- start 2023-11-28T18:26:52.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3e19a7640 1 -- start start 2023-11-28T18:26:52.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3e19a7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3dc0b2400 0x7ff3dc155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:52.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3e19a7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3dc15d350 0x7ff3dc0b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:52.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3e19a7640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3dc15d730 0x7ff3dc0b0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:52.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3e19a7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0b60a0 con 0x7ff3dc15d350 2023-11-28T18:26:52.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3e19a7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0b77e0 con 0x7ff3dc0b2400 2023-11-28T18:26:52.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3e19a7640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc1542b0 con 0x7ff3dc15d730 2023-11-28T18:26:52.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3d3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3dc15d350 0x7ff3dc0b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:52.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3d3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3dc15d350 0x7ff3dc0b0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50746/0 (socket says 172.21.15.105:50746) 2023-11-28T18:26:52.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3d3fff640 1 -- 172.21.15.105:0/2374822417 learned_addr learned my addr 172.21.15.105:0/2374822417 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:52.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3e11a6640 1 --2- 172.21.15.105:0/2374822417 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3dc15d730 0x7ff3dc0b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:52.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3e09a5640 1 --2- 172.21.15.105:0/2374822417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3dc0b2400 0x7ff3dc155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:52.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3d3fff640 1 -- 172.21.15.105:0/2374822417 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3dc15d730 msgr2=0x7ff3dc0b0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:52.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3d3fff640 1 --2- 172.21.15.105:0/2374822417 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3dc15d730 0x7ff3dc0b0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:52.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3d3fff640 1 -- 172.21.15.105:0/2374822417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3dc0b2400 msgr2=0x7ff3dc155e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:52.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3d3fff640 1 --2- 172.21.15.105:0/2374822417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3dc0b2400 0x7ff3dc155e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:52.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3d3fff640 1 -- 172.21.15.105:0/2374822417 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d4009b30 con 0x7ff3dc15d350 2023-11-28T18:26:52.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3e11a6640 1 --2- 172.21.15.105:0/2374822417 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3dc15d730 0x7ff3dc0b0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:52.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3d3fff640 1 --2- 172.21.15.105:0/2374822417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3dc15d350 0x7ff3dc0b0350 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00c7f0 tx=0x7ff3cc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:52.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3d17fa640 1 -- 172.21.15.105:0/2374822417 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc004220 con 0x7ff3dc15d350 2023-11-28T18:26:52.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3d17fa640 1 -- 172.21.15.105:0/2374822417 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3cc004380 con 0x7ff3dc15d350 2023-11-28T18:26:52.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3e19a7640 1 -- 172.21.15.105:0/2374822417 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3dc154590 con 0x7ff3dc15d350 2023-11-28T18:26:52.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3d17fa640 1 -- 172.21.15.105:0/2374822417 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc003840 con 0x7ff3dc15d350 2023-11-28T18:26:52.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.383+0000 7ff3e19a7640 1 -- 172.21.15.105:0/2374822417 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3dc0b5d30 con 0x7ff3dc15d350 2023-11-28T18:26:52.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.387+0000 7ff3e19a7640 1 -- 172.21.15.105:0/2374822417 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3dc0b6210 con 0x7ff3dc15d350 2023-11-28T18:26:52.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.387+0000 7ff3d17fa640 1 -- 172.21.15.105:0/2374822417 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff3cc003bb0 con 0x7ff3dc15d350 2023-11-28T18:26:52.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.391+0000 7ff3d17fa640 1 --2- 172.21.15.105:0/2374822417 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff3c8066900 0x7ff3c8068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:52.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.391+0000 7ff3d17fa640 1 -- 172.21.15.105:0/2374822417 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff3cc084600 con 0x7ff3dc15d350 2023-11-28T18:26:52.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.391+0000 7ff3e09a5640 1 --2- 172.21.15.105:0/2374822417 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff3c8066900 0x7ff3c8068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:52.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.391+0000 7ff3e09a5640 1 --2- 172.21.15.105:0/2374822417 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff3c8066900 0x7ff3c8068dc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8006b20 tx=0x7ff3d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:52.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.395+0000 7ff3d17fa640 1 -- 172.21.15.105:0/2374822417 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff3cc051560 con 0x7ff3dc15d350 2023-11-28T18:26:52.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:52.575+0000 7ff3e19a7640 1 -- 172.21.15.105:0/2374822417 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff3dc074580 con 0x7ff3dc15d350 2023-11-28T18:26:53.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.203+0000 7ff3d17fa640 1 -- 172.21.15.105:0/2374822417 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff3cc003e60 con 0x7ff3dc15d350 2023-11-28T18:26:53.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.203+0000 7ff3e19a7640 1 -- 172.21.15.105:0/2374822417 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff3c8066900 msgr2=0x7ff3c8068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:53.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.203+0000 7ff3e19a7640 1 --2- 172.21.15.105:0/2374822417 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff3c8066900 0x7ff3c8068dc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8006b20 tx=0x7ff3d8008040 comp rx=0 tx=0).stop 2023-11-28T18:26:53.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.203+0000 7ff3e19a7640 1 -- 172.21.15.105:0/2374822417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3dc15d350 msgr2=0x7ff3dc0b0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:53.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.203+0000 7ff3e19a7640 1 --2- 172.21.15.105:0/2374822417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3dc15d350 0x7ff3dc0b0350 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00c7f0 tx=0x7ff3cc00ccb0 comp rx=0 tx=0).stop 2023-11-28T18:26:53.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.203+0000 7ff3e19a7640 1 -- 172.21.15.105:0/2374822417 shutdown_connections 2023-11-28T18:26:53.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.203+0000 7ff3e19a7640 1 --2- 172.21.15.105:0/2374822417 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3dc15d730 0x7ff3dc0b0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:53.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.203+0000 7ff3e19a7640 1 --2- 172.21.15.105:0/2374822417 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff3c8066900 0x7ff3c8068dc0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:53.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.203+0000 7ff3e19a7640 1 --2- 172.21.15.105:0/2374822417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3dc15d350 0x7ff3dc0b0350 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:53.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.203+0000 7ff3e19a7640 1 --2- 172.21.15.105:0/2374822417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3dc0b2400 0x7ff3dc155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:53.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.203+0000 7ff3e19a7640 1 -- 172.21.15.105:0/2374822417 >> 172.21.15.105:0/2374822417 conn(0x7ff3dc0d2d90 msgr2=0x7ff3dc0b1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:53.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.203+0000 7ff3e19a7640 1 -- 172.21.15.105:0/2374822417 shutdown_connections 2023-11-28T18:26:53.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.203+0000 7ff3e19a7640 1 -- 172.21.15.105:0/2374822417 wait complete. 2023-11-28T18:26:53.230 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T18:26:53.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.411+0000 7fa3812e3640 1 Processor -- start 2023-11-28T18:26:53.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.411+0000 7fa3812e3640 1 -- start start 2023-11-28T18:26:53.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.411+0000 7fa3812e3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa37c0d3c30 0x7fa37c0d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:53.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.411+0000 7fa3812e3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa37c0b2d20 0x7fa37c0b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:53.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.411+0000 7fa3812e3640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa37c0b25b0 con 0x7fa37c15d4f0 2023-11-28T18:26:53.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.411+0000 7fa3812e3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa37c0b2900 con 0x7fa37c0b2d20 2023-11-28T18:26:53.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.411+0000 7fa3812e3640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c0d0a20 con 0x7fa37c0d3c30 2023-11-28T18:26:53.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.411+0000 7fa37affd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa37c15d4f0 0x7fa37c0d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:53520/0 (socket says 172.21.15.105:53520) 2023-11-28T18:26:53.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.411+0000 7fa37affd640 1 -- 172.21.15.105:0/2398928703 learned_addr learned my addr 172.21.15.105:0/2398928703 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:53.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.411+0000 7fa37a7fc640 1 --2- 172.21.15.105:0/2398928703 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa37c0d3c30 0x7fa37c0d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:53.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.411+0000 7fa37b7fe640 1 --2- 172.21.15.105:0/2398928703 >> v2:172.21.15.148:3300/0 conn(0x7fa37c0b2d20 0x7fa37c0b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:53.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.411+0000 7fa379ffb640 1 -- 172.21.15.105:0/2398928703 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321275150 0 0) 0x7fa37c0b25b0 con 0x7fa37c15d4f0 2023-11-28T18:26:53.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.415+0000 7fa379ffb640 1 -- 172.21.15.105:0/2398928703 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa364003530 con 0x7fa37c15d4f0 2023-11-28T18:26:53.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.415+0000 7fa37b7fe640 1 -- 172.21.15.105:0/2398928703 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa37c0d3c30 msgr2=0x7fa37c0d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:53.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.415+0000 7fa37b7fe640 1 --2- 172.21.15.105:0/2398928703 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa37c0d3c30 0x7fa37c0d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:53.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.415+0000 7fa37b7fe640 1 -- 172.21.15.105:0/2398928703 >> v1:172.21.15.105:6789/0 conn(0x7fa37c15d4f0 legacy=0x7fa37c0d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:53.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.415+0000 7fa379ffb640 1 -- 172.21.15.105:0/2398928703 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3298578404 0 0) 0x7fa364003530 con 0x7fa37c15d4f0 2023-11-28T18:26:53.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.415+0000 7fa37b7fe640 1 -- 172.21.15.105:0/2398928703 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c0cea30 con 0x7fa37c0b2d20 2023-11-28T18:26:53.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.415+0000 7fa37b7fe640 1 --2- 172.21.15.105:0/2398928703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c0b2d20 0x7fa37c0b31d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa374009d90 tx=0x7fa37401c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dc49b65765a9e7f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:53.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.415+0000 7fa379ffb640 1 -- 172.21.15.105:0/2398928703 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c001680 con 0x7fa37c0b2d20 2023-11-28T18:26:53.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.415+0000 7fa379ffb640 1 -- 172.21.15.105:0/2398928703 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3740a55b0 con 0x7fa37c0b2d20 2023-11-28T18:26:53.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.415+0000 7fa379ffb640 1 -- 172.21.15.105:0/2398928703 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3740a5920 con 0x7fa37c0b2d20 2023-11-28T18:26:53.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.419+0000 7fa3812e3640 1 -- 172.21.15.105:0/2398928703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c0b2d20 msgr2=0x7fa37c0b31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:53.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.419+0000 7fa3812e3640 1 --2- 172.21.15.105:0/2398928703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c0b2d20 0x7fa37c0b31d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa374009d90 tx=0x7fa37401c7b0 comp rx=0 tx=0).stop 2023-11-28T18:26:53.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.419+0000 7fa3812e3640 1 -- 172.21.15.105:0/2398928703 shutdown_connections 2023-11-28T18:26:53.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.419+0000 7fa3812e3640 1 --2- 172.21.15.105:0/2398928703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c0b2d20 0x7fa37c0b31d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:53.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.419+0000 7fa3812e3640 1 --2- 172.21.15.105:0/2398928703 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa37c0d3c30 0x7fa37c0d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:53.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.419+0000 7fa3812e3640 1 -- 172.21.15.105:0/2398928703 >> 172.21.15.105:0/2398928703 conn(0x7fa37c0d3030 msgr2=0x7fa37c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:53.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.419+0000 7fa3812e3640 1 -- 172.21.15.105:0/2398928703 shutdown_connections 2023-11-28T18:26:53.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.419+0000 7fa3812e3640 1 -- 172.21.15.105:0/2398928703 wait complete. 2023-11-28T18:26:53.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.419+0000 7fa3812e3640 1 Processor -- start 2023-11-28T18:26:53.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.419+0000 7fa3812e3640 1 -- start start 2023-11-28T18:26:53.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.419+0000 7fa3812e3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa37c0b2d20 0x7fa37c0c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:53.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.419+0000 7fa3812e3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c0d3c30 0x7fa37c0ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:53.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.419+0000 7fa3812e3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa37c15d4f0 0x7fa37c0d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:53.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.419+0000 7fa3812e3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c0b7c40 con 0x7fa37c15d4f0 2023-11-28T18:26:53.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.419+0000 7fa3812e3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c0b7eb0 con 0x7fa37c0d3c30 2023-11-28T18:26:53.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.419+0000 7fa3812e3640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c0b8190 con 0x7fa37c0b2d20 2023-11-28T18:26:53.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.419+0000 7fa37b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa37c15d4f0 0x7fa37c0d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:53.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.419+0000 7fa37b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa37c15d4f0 0x7fa37c0d1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50756/0 (socket says 172.21.15.105:50756) 2023-11-28T18:26:53.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.419+0000 7fa37b7fe640 1 -- 172.21.15.105:0/2116621257 learned_addr learned my addr 172.21.15.105:0/2116621257 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:53.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.419+0000 7fa37affd640 1 --2- 172.21.15.105:0/2116621257 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa37c0b2d20 0x7fa37c0c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:53.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.419+0000 7fa37a7fc640 1 --2- 172.21.15.105:0/2116621257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c0d3c30 0x7fa37c0ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:53.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.423+0000 7fa37b7fe640 1 -- 172.21.15.105:0/2116621257 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa37c0b2d20 msgr2=0x7fa37c0c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:53.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.423+0000 7fa37b7fe640 1 --2- 172.21.15.105:0/2116621257 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa37c0b2d20 0x7fa37c0c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:53.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.423+0000 7fa37b7fe640 1 -- 172.21.15.105:0/2116621257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c0d3c30 msgr2=0x7fa37c0ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:53.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.423+0000 7fa37b7fe640 1 --2- 172.21.15.105:0/2116621257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c0d3c30 0x7fa37c0ca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:53.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.423+0000 7fa37b7fe640 1 -- 172.21.15.105:0/2116621257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa374009a70 con 0x7fa37c15d4f0 2023-11-28T18:26:53.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.423+0000 7fa37b7fe640 1 --2- 172.21.15.105:0/2116621257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa37c15d4f0 0x7fa37c0d1e20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa37401d050 tx=0x7fa374019550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:53.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.423+0000 7fa3637fe640 1 -- 172.21.15.105:0/2116621257 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3740a5930 con 0x7fa37c15d4f0 2023-11-28T18:26:53.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.423+0000 7fa3637fe640 1 -- 172.21.15.105:0/2116621257 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3740a5a90 con 0x7fa37c15d4f0 2023-11-28T18:26:53.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.423+0000 7fa3637fe640 1 -- 172.21.15.105:0/2116621257 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3740a5e60 con 0x7fa37c15d4f0 2023-11-28T18:26:53.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.423+0000 7fa3812e3640 1 -- 172.21.15.105:0/2116621257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa37c0b8470 con 0x7fa37c15d4f0 2023-11-28T18:26:53.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.423+0000 7fa3812e3640 1 -- 172.21.15.105:0/2116621257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa37c0c8db0 con 0x7fa37c15d4f0 2023-11-28T18:26:53.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.423+0000 7fa3812e3640 1 -- 172.21.15.105:0/2116621257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37c0b8990 con 0x7fa37c15d4f0 2023-11-28T18:26:53.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.423+0000 7fa3637fe640 1 -- 172.21.15.105:0/2116621257 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa374004040 con 0x7fa37c15d4f0 2023-11-28T18:26:53.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.427+0000 7fa3637fe640 1 --2- 172.21.15.105:0/2116621257 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa3680668d0 0x7fa368068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:53.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.427+0000 7fa3637fe640 1 -- 172.21.15.105:0/2116621257 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa37411f770 con 0x7fa37c15d4f0 2023-11-28T18:26:53.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.427+0000 7fa37affd640 1 --2- 172.21.15.105:0/2116621257 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa3680668d0 0x7fa368068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:53.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.427+0000 7fa37affd640 1 --2- 172.21.15.105:0/2116621257 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa3680668d0 0x7fa368068d90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0011f0 tx=0x7fa36c0016e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:53.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.427+0000 7fa3637fe640 1 -- 172.21.15.105:0/2116621257 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa3740ec890 con 0x7fa37c15d4f0 2023-11-28T18:26:53.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:53.579+0000 7fa3812e3640 1 -- 172.21.15.105:0/2116621257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa37c0b8990 con 0x7fa37c15d4f0 2023-11-28T18:26:54.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.403+0000 7fa3637fe640 1 -- 172.21.15.105:0/2116621257 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v56) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa37c0b8990 con 0x7fa37c15d4f0 2023-11-28T18:26:54.405 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' removed 2023-11-28T18:26:54.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.407+0000 7fa3812e3640 1 -- 172.21.15.105:0/2116621257 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa3680668d0 msgr2=0x7fa368068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:54.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.407+0000 7fa3812e3640 1 --2- 172.21.15.105:0/2116621257 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa3680668d0 0x7fa368068d90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0011f0 tx=0x7fa36c0016e0 comp rx=0 tx=0).stop 2023-11-28T18:26:54.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.407+0000 7fa3812e3640 1 -- 172.21.15.105:0/2116621257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa37c15d4f0 msgr2=0x7fa37c0d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:54.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.407+0000 7fa3812e3640 1 --2- 172.21.15.105:0/2116621257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa37c15d4f0 0x7fa37c0d1e20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa37401d050 tx=0x7fa374019550 comp rx=0 tx=0).stop 2023-11-28T18:26:54.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.407+0000 7fa3812e3640 1 -- 172.21.15.105:0/2116621257 shutdown_connections 2023-11-28T18:26:54.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.407+0000 7fa3812e3640 1 --2- 172.21.15.105:0/2116621257 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa3680668d0 0x7fa368068d90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:54.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.407+0000 7fa3812e3640 1 --2- 172.21.15.105:0/2116621257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa37c15d4f0 0x7fa37c0d1e20 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:54.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.407+0000 7fa3812e3640 1 --2- 172.21.15.105:0/2116621257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c0d3c30 0x7fa37c0ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:54.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.407+0000 7fa3812e3640 1 --2- 172.21.15.105:0/2116621257 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa37c0b2d20 0x7fa37c0c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:54.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.407+0000 7fa3812e3640 1 -- 172.21.15.105:0/2116621257 >> 172.21.15.105:0/2116621257 conn(0x7fa37c0d3030 msgr2=0x7fa37c06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:54.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.411+0000 7fa3812e3640 1 -- 172.21.15.105:0/2116621257 shutdown_connections 2023-11-28T18:26:54.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.411+0000 7fa3812e3640 1 -- 172.21.15.105:0/2116621257 wait complete. 2023-11-28T18:26:54.443 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T18:26:54.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.583+0000 7f75bc1c3640 1 Processor -- start 2023-11-28T18:26:54.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.583+0000 7f75bc1c3640 1 -- start start 2023-11-28T18:26:54.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.583+0000 7f75bc1c3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f75b40b3d30 0x7f75b41680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:54.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.583+0000 7f75bc1c3640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b40b25b0 con 0x7f75b40d3c30 2023-11-28T18:26:54.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.583+0000 7f75bc1c3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b40d1c30 con 0x7f75b415d4f0 2023-11-28T18:26:54.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.583+0000 7f75bc1c3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b40b2900 con 0x7f75b40b3d30 2023-11-28T18:26:54.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.583+0000 7f75ba760640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f75b40b3d30 0x7f75b41680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:54.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.583+0000 7f75ba760640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f75b40b3d30 0x7f75b41680a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50764/0 (socket says 172.21.15.105:50764) 2023-11-28T18:26:54.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.583+0000 7f75ba760640 1 -- 172.21.15.105:0/721823423 learned_addr learned my addr 172.21.15.105:0/721823423 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:54.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.583+0000 7f75ba760640 1 -- 172.21.15.105:0/721823423 >> v1:172.21.15.148:6789/0 conn(0x7f75b415d4f0 legacy=0x7f75b40d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:54.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.583+0000 7f75ba760640 1 -- 172.21.15.105:0/721823423 >> v1:172.21.15.105:6789/0 conn(0x7f75b40d3c30 legacy=0x7f75b40d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:54.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.583+0000 7f75ba760640 1 -- 172.21.15.105:0/721823423 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b40cea30 con 0x7f75b40b3d30 2023-11-28T18:26:54.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.583+0000 7f75b8f5d640 1 -- 172.21.15.105:0/721823423 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877397981 0 0) 0x7f75b40b25b0 con 0x7f75b40d3c30 2023-11-28T18:26:54.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.587+0000 7f75ba760640 1 --2- 172.21.15.105:0/721823423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75b40b3d30 0x7f75b41680a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f75ac009e50 tx=0x7f75ac01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b57da47bc22801 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:54.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.587+0000 7f75b8f5d640 1 -- 172.21.15.105:0/721823423 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac0a5780 con 0x7f75b40b3d30 2023-11-28T18:26:54.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.587+0000 7f75b8f5d640 1 -- 172.21.15.105:0/721823423 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ac0a58e0 con 0x7f75b40b3d30 2023-11-28T18:26:54.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.587+0000 7f75b8f5d640 1 -- 172.21.15.105:0/721823423 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac0a5c70 con 0x7f75b40b3d30 2023-11-28T18:26:54.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.587+0000 7f75bc1c3640 1 -- 172.21.15.105:0/721823423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75b40b3d30 msgr2=0x7f75b41680a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:54.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.587+0000 7f75bc1c3640 1 --2- 172.21.15.105:0/721823423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75b40b3d30 0x7f75b41680a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f75ac009e50 tx=0x7f75ac01c8c0 comp rx=0 tx=0).stop 2023-11-28T18:26:54.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.587+0000 7f75bc1c3640 1 -- 172.21.15.105:0/721823423 shutdown_connections 2023-11-28T18:26:54.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.587+0000 7f75bc1c3640 1 --2- 172.21.15.105:0/721823423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75b40b3d30 0x7f75b41680a0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:54.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.587+0000 7f75bc1c3640 1 -- 172.21.15.105:0/721823423 >> 172.21.15.105:0/721823423 conn(0x7f75b40d3030 msgr2=0x7f75b40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:54.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.587+0000 7f75bc1c3640 1 -- 172.21.15.105:0/721823423 shutdown_connections 2023-11-28T18:26:54.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.587+0000 7f75bc1c3640 1 -- 172.21.15.105:0/721823423 wait complete. 2023-11-28T18:26:54.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75bc1c3640 1 Processor -- start 2023-11-28T18:26:54.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75bc1c3640 1 -- start start 2023-11-28T18:26:54.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75bc1c3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75b40b3d30 0x7f75b40b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:54.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75bc1c3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75b40d3c30 0x7f75b40b84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:54.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75bc1c3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75b415d4f0 0x7f75b4155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:54.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75bc1c3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b40d0a20 con 0x7f75b40d3c30 2023-11-28T18:26:54.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75bc1c3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b40b8a20 con 0x7f75b415d4f0 2023-11-28T18:26:54.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75bc1c3640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b40b7370 con 0x7f75b40b3d30 2023-11-28T18:26:54.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75b975e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75b40d3c30 0x7f75b40b84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:54.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75b975e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75b40d3c30 0x7f75b40b84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50768/0 (socket says 172.21.15.105:50768) 2023-11-28T18:26:54.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75b975e640 1 -- 172.21.15.105:0/3371989197 learned_addr learned my addr 172.21.15.105:0/3371989197 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:54.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75b975e640 1 -- 172.21.15.105:0/3371989197 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75b40b3d30 msgr2=0x7f75b40b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:54.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75b975e640 1 --2- 172.21.15.105:0/3371989197 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75b40b3d30 0x7f75b40b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:54.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75ba760640 1 --2- 172.21.15.105:0/3371989197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75b415d4f0 0x7f75b4155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:54.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75b975e640 1 -- 172.21.15.105:0/3371989197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75b415d4f0 msgr2=0x7f75b4155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:54.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75b975e640 1 --2- 172.21.15.105:0/3371989197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75b415d4f0 0x7f75b4155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:54.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75b975e640 1 -- 172.21.15.105:0/3371989197 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ac009b30 con 0x7f75b40d3c30 2023-11-28T18:26:54.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75ba760640 1 --2- 172.21.15.105:0/3371989197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75b415d4f0 0x7f75b4155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:54.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75b975e640 1 --2- 172.21.15.105:0/3371989197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75b40d3c30 0x7f75b40b84e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f75a400c440 tx=0x7f75a400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:54.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75aa7fc640 1 -- 172.21.15.105:0/3371989197 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a4005480 con 0x7f75b40d3c30 2023-11-28T18:26:54.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75aa7fc640 1 -- 172.21.15.105:0/3371989197 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a4003600 con 0x7f75b40d3c30 2023-11-28T18:26:54.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75aa7fc640 1 -- 172.21.15.105:0/3371989197 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a40039b0 con 0x7f75b40d3c30 2023-11-28T18:26:54.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75bc1c3640 1 -- 172.21.15.105:0/3371989197 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b40b6760 con 0x7f75b40d3c30 2023-11-28T18:26:54.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.591+0000 7f75bc1c3640 1 -- 172.21.15.105:0/3371989197 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b40b6d20 con 0x7f75b40d3c30 2023-11-28T18:26:54.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.595+0000 7f75bc1c3640 1 -- 172.21.15.105:0/3371989197 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b4067570 con 0x7f75b40d3c30 2023-11-28T18:26:54.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.595+0000 7f75aa7fc640 1 -- 172.21.15.105:0/3371989197 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f75a4003b10 con 0x7f75b40d3c30 2023-11-28T18:26:54.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.595+0000 7f75aa7fc640 1 --2- 172.21.15.105:0/3371989197 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f75980668d0 0x7f7598068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:54.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.595+0000 7f75aa7fc640 1 -- 172.21.15.105:0/3371989197 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5993+0+0 (secure 0 0 0) 0x7f75a40840b0 con 0x7f75b40d3c30 2023-11-28T18:26:54.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.595+0000 7f75b9f5f640 1 --2- 172.21.15.105:0/3371989197 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f75980668d0 0x7f7598068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:54.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.595+0000 7f75b9f5f640 1 --2- 172.21.15.105:0/3371989197 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f75980668d0 0x7f7598068d90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f75b0000f70 tx=0x7f75b00079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:54.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.599+0000 7f75aa7fc640 1 -- 172.21.15.105:0/3371989197 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f75a4054bc0 con 0x7f75b40d3c30 2023-11-28T18:26:54.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:54.747+0000 7f75bc1c3640 1 -- 172.21.15.105:0/3371989197 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f75b40b74e0 con 0x7f75b40d3c30 2023-11-28T18:26:55.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.407+0000 7f75aa7fc640 1 -- 172.21.15.105:0/3371989197 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v57) v1 ==== 169+0+0 (secure 0 0 0) 0x7f75a4054750 con 0x7f75b40d3c30 2023-11-28T18:26:55.412 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' removed 2023-11-28T18:26:55.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.411+0000 7f75bc1c3640 1 -- 172.21.15.105:0/3371989197 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f75980668d0 msgr2=0x7f7598068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:55.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.411+0000 7f75bc1c3640 1 --2- 172.21.15.105:0/3371989197 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f75980668d0 0x7f7598068d90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f75b0000f70 tx=0x7f75b00079c0 comp rx=0 tx=0).stop 2023-11-28T18:26:55.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.411+0000 7f75bc1c3640 1 -- 172.21.15.105:0/3371989197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75b40d3c30 msgr2=0x7f75b40b84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:55.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.411+0000 7f75bc1c3640 1 --2- 172.21.15.105:0/3371989197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75b40d3c30 0x7f75b40b84e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f75a400c440 tx=0x7f75a400c900 comp rx=0 tx=0).stop 2023-11-28T18:26:55.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.411+0000 7f75bc1c3640 1 -- 172.21.15.105:0/3371989197 shutdown_connections 2023-11-28T18:26:55.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.411+0000 7f75bc1c3640 1 --2- 172.21.15.105:0/3371989197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75b415d4f0 0x7f75b4155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:55.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.411+0000 7f75bc1c3640 1 --2- 172.21.15.105:0/3371989197 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f75980668d0 0x7f7598068d90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:55.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.411+0000 7f75bc1c3640 1 --2- 172.21.15.105:0/3371989197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75b40d3c30 0x7f75b40b84e0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:55.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.411+0000 7f75bc1c3640 1 --2- 172.21.15.105:0/3371989197 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75b40b3d30 0x7f75b40b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:55.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.411+0000 7f75bc1c3640 1 -- 172.21.15.105:0/3371989197 >> 172.21.15.105:0/3371989197 conn(0x7f75b40d3030 msgr2=0x7f75b40d1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:55.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.411+0000 7f75bc1c3640 1 -- 172.21.15.105:0/3371989197 shutdown_connections 2023-11-28T18:26:55.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.411+0000 7f75bc1c3640 1 -- 172.21.15.105:0/3371989197 wait complete. 2023-11-28T18:26:55.447 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random 2023-11-28T18:26:55.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.583+0000 7f8a2ac79640 1 Processor -- start 2023-11-28T18:26:55.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.583+0000 7f8a2ac79640 1 -- start start 2023-11-28T18:26:55.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.583+0000 7f8a2ac79640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a2406d610 0x7f8a240cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:55.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.583+0000 7f8a2ac79640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a240ae100 0x7f8a240cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:55.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.583+0000 7f8a2ac79640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a24162820 con 0x7f8a240ce530 2023-11-28T18:26:55.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.583+0000 7f8a2ac79640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a24161c10 con 0x7f8a2406d610 2023-11-28T18:26:55.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.583+0000 7f8a2ac79640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a24161ef0 con 0x7f8a240ae100 2023-11-28T18:26:55.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.583+0000 7f8a28a15640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a2406d610 0x7f8a240cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:55.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.583+0000 7f8a28a15640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a2406d610 0x7f8a240cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50784/0 (socket says 172.21.15.105:50784) 2023-11-28T18:26:55.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.583+0000 7f8a28a15640 1 -- 172.21.15.105:0/115109408 learned_addr learned my addr 172.21.15.105:0/115109408 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:55.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.583+0000 7f8a29216640 1 --2- 172.21.15.105:0/115109408 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a240ae100 0x7f8a240cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:55.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.583+0000 7f8a28a15640 1 -- 172.21.15.105:0/115109408 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a240ae100 msgr2=0x7f8a240cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:55.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.583+0000 7f8a28a15640 1 --2- 172.21.15.105:0/115109408 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a240ae100 0x7f8a240cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:55.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.583+0000 7f8a28a15640 1 -- 172.21.15.105:0/115109408 >> v1:172.21.15.148:6789/0 conn(0x7f8a240ce530 legacy=0x7f8a240ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:55.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.583+0000 7f8a28a15640 1 -- 172.21.15.105:0/115109408 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a241621d0 con 0x7f8a2406d610 2023-11-28T18:26:55.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.587+0000 7f8a28a15640 1 --2- 172.21.15.105:0/115109408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a2406d610 0x7f8a240cdff0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009870 tx=0x7f8a2001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52c4ca28a091c18 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:55.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.587+0000 7f8a1b7fe640 1 -- 172.21.15.105:0/115109408 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a200a95f0 con 0x7f8a2406d610 2023-11-28T18:26:55.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.587+0000 7f8a1b7fe640 1 -- 172.21.15.105:0/115109408 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a200a9750 con 0x7f8a2406d610 2023-11-28T18:26:55.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.587+0000 7f8a1b7fe640 1 -- 172.21.15.105:0/115109408 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a200a9ae0 con 0x7f8a2406d610 2023-11-28T18:26:55.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.587+0000 7f8a2ac79640 1 -- 172.21.15.105:0/115109408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a2406d610 msgr2=0x7f8a240cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:55.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.587+0000 7f8a2ac79640 1 --2- 172.21.15.105:0/115109408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a2406d610 0x7f8a240cdff0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009870 tx=0x7f8a2001c510 comp rx=0 tx=0).stop 2023-11-28T18:26:55.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.587+0000 7f8a2ac79640 1 -- 172.21.15.105:0/115109408 shutdown_connections 2023-11-28T18:26:55.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.587+0000 7f8a2ac79640 1 --2- 172.21.15.105:0/115109408 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a240ae100 0x7f8a240cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:55.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.587+0000 7f8a2ac79640 1 --2- 172.21.15.105:0/115109408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a2406d610 0x7f8a240cdff0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:55.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.587+0000 7f8a2ac79640 1 -- 172.21.15.105:0/115109408 >> 172.21.15.105:0/115109408 conn(0x7f8a240ceb90 msgr2=0x7f8a240cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:55.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.587+0000 7f8a2ac79640 1 -- 172.21.15.105:0/115109408 shutdown_connections 2023-11-28T18:26:55.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.587+0000 7f8a2ac79640 1 -- 172.21.15.105:0/115109408 wait complete. 2023-11-28T18:26:55.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a2ac79640 1 Processor -- start 2023-11-28T18:26:55.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a2ac79640 1 -- start start 2023-11-28T18:26:55.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a2ac79640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a2406d610 0x7f8a24160350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:55.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a2ac79640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a240ae100 0x7f8a240b3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:55.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a2ac79640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a240ce530 0x7f8a240b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:55.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a2ac79640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a240b4b00 con 0x7f8a240ae100 2023-11-28T18:26:55.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a2ac79640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a24163b90 con 0x7f8a240ce530 2023-11-28T18:26:55.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a2ac79640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a240b3ef0 con 0x7f8a2406d610 2023-11-28T18:26:55.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a1bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a240ae100 0x7f8a240b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:55.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a1bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a240ae100 0x7f8a240b3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50796/0 (socket says 172.21.15.105:50796) 2023-11-28T18:26:55.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a1bfff640 1 -- 172.21.15.105:0/1735830139 learned_addr learned my addr 172.21.15.105:0/1735830139 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:55.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a29216640 1 --2- 172.21.15.105:0/1735830139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a240ce530 0x7f8a240b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:55.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a28a15640 1 --2- 172.21.15.105:0/1735830139 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a2406d610 0x7f8a24160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:55.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a1bfff640 1 -- 172.21.15.105:0/1735830139 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a2406d610 msgr2=0x7f8a24160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:55.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a1bfff640 1 --2- 172.21.15.105:0/1735830139 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a2406d610 0x7f8a24160350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:55.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a1bfff640 1 -- 172.21.15.105:0/1735830139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a240ce530 msgr2=0x7f8a240b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:55.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a1bfff640 1 --2- 172.21.15.105:0/1735830139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a240ce530 0x7f8a240b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:55.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a1bfff640 1 -- 172.21.15.105:0/1735830139 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a200094d0 con 0x7f8a240ae100 2023-11-28T18:26:55.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a28a15640 1 --2- 172.21.15.105:0/1735830139 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a2406d610 0x7f8a24160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:55.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a1bfff640 1 --2- 172.21.15.105:0/1735830139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a240ae100 0x7f8a240b3310 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400c3c0 tx=0x7f8a1400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:55.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a197fa640 1 -- 172.21.15.105:0/1735830139 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a140052d0 con 0x7f8a240ae100 2023-11-28T18:26:55.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a197fa640 1 -- 172.21.15.105:0/1735830139 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a14005430 con 0x7f8a240ae100 2023-11-28T18:26:55.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a197fa640 1 -- 172.21.15.105:0/1735830139 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14003840 con 0x7f8a240ae100 2023-11-28T18:26:55.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.591+0000 7f8a2ac79640 1 -- 172.21.15.105:0/1735830139 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a240b41d0 con 0x7f8a240ae100 2023-11-28T18:26:55.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.595+0000 7f8a2ac79640 1 -- 172.21.15.105:0/1735830139 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a240b4790 con 0x7f8a240ae100 2023-11-28T18:26:55.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.595+0000 7f8a2ac79640 1 -- 172.21.15.105:0/1735830139 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a24067570 con 0x7f8a240ae100 2023-11-28T18:26:55.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.595+0000 7f8a197fa640 1 -- 172.21.15.105:0/1735830139 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8a140039a0 con 0x7f8a240ae100 2023-11-28T18:26:55.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.595+0000 7f8a197fa640 1 --2- 172.21.15.105:0/1735830139 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a100668d0 0x7f8a10068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:55.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.595+0000 7f8a197fa640 1 -- 172.21.15.105:0/1735830139 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f8a140837a0 con 0x7f8a240ae100 2023-11-28T18:26:55.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.595+0000 7f8a28a15640 1 --2- 172.21.15.105:0/1735830139 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a100668d0 0x7f8a10068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:55.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.595+0000 7f8a28a15640 1 --2- 172.21.15.105:0/1735830139 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a100668d0 0x7f8a10068d90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8a200ae000 tx=0x7f8a200b3040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:55.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.599+0000 7f8a197fa640 1 -- 172.21.15.105:0/1735830139 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8a14050bd0 con 0x7f8a240ae100 2023-11-28T18:26:55.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.763+0000 7f8a2ac79640 1 -- 172.21.15.105:0/1735830139 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"} v 0) v1 -- 0x7f8a24099ae0 con 0x7f8a240ae100 2023-11-28T18:26:55.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.767+0000 7f8a197fa640 1 -- 172.21.15.105:0/1735830139 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"}]=0 v23) v1 ==== 110+0+0 (secure 0 0 0) 0x7f8a140541f0 con 0x7f8a240ae100 2023-11-28T18:26:55.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.771+0000 7f8a2ac79640 1 -- 172.21.15.105:0/1735830139 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a100668d0 msgr2=0x7f8a10068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:55.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.771+0000 7f8a2ac79640 1 --2- 172.21.15.105:0/1735830139 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a100668d0 0x7f8a10068d90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8a200ae000 tx=0x7f8a200b3040 comp rx=0 tx=0).stop 2023-11-28T18:26:55.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.771+0000 7f8a2ac79640 1 -- 172.21.15.105:0/1735830139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a240ae100 msgr2=0x7f8a240b3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:55.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.771+0000 7f8a2ac79640 1 --2- 172.21.15.105:0/1735830139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a240ae100 0x7f8a240b3310 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400c3c0 tx=0x7f8a1400c880 comp rx=0 tx=0).stop 2023-11-28T18:26:55.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.771+0000 7f8a2ac79640 1 -- 172.21.15.105:0/1735830139 shutdown_connections 2023-11-28T18:26:55.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.771+0000 7f8a2ac79640 1 --2- 172.21.15.105:0/1735830139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a240ce530 0x7f8a240b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:55.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.771+0000 7f8a2ac79640 1 --2- 172.21.15.105:0/1735830139 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a100668d0 0x7f8a10068d90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:55.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.771+0000 7f8a2ac79640 1 --2- 172.21.15.105:0/1735830139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a240ae100 0x7f8a240b3310 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:55.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.771+0000 7f8a2ac79640 1 --2- 172.21.15.105:0/1735830139 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a2406d610 0x7f8a24160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:55.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.771+0000 7f8a2ac79640 1 -- 172.21.15.105:0/1735830139 >> 172.21.15.105:0/1735830139 conn(0x7f8a240ceb90 msgr2=0x7f8a240730d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:55.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.771+0000 7f8a2ac79640 1 -- 172.21.15.105:0/1735830139 shutdown_connections 2023-11-28T18:26:55.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.771+0000 7f8a2ac79640 1 -- 172.21.15.105:0/1735830139 wait complete. 2023-11-28T18:26:55.794 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T18:26:55.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c4765640 1 Processor -- start 2023-11-28T18:26:55.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c4765640 1 -- start start 2023-11-28T18:26:55.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c4765640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe7bc0cc760 0x7fe7bc164d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:55.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c4765640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7bc0ccc10 con 0x7fe7bc159f40 2023-11-28T18:26:55.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c4765640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7bc06bf50 con 0x7fe7bc0caf60 2023-11-28T18:26:55.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c4765640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7bc0cb3c0 con 0x7fe7bc0cc760 2023-11-28T18:26:55.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c2501640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe7bc159f40 0x7fe7bc0ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:53536/0 (socket says 172.21.15.105:53536) 2023-11-28T18:26:55.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c2501640 1 -- 172.21.15.105:0/1345769859 learned_addr learned my addr 172.21.15.105:0/1345769859 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:55.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c2d02640 1 --2- 172.21.15.105:0/1345769859 >> v2:172.21.15.148:3300/0 conn(0x7fe7bc0cc760 0x7fe7bc164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:55.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c14ff640 1 -- 172.21.15.105:0/1345769859 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864367171 0 0) 0x7fe7bc0ccc10 con 0x7fe7bc159f40 2023-11-28T18:26:55.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c14ff640 1 -- 172.21.15.105:0/1345769859 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7a8005740 con 0x7fe7bc159f40 2023-11-28T18:26:55.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c14ff640 1 -- 172.21.15.105:0/1345769859 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1390002026 0 0) 0x7fe7bc06bf50 con 0x7fe7bc0caf60 2023-11-28T18:26:55.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c14ff640 1 -- 172.21.15.105:0/1345769859 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7bc0ccc10 con 0x7fe7bc0caf60 2023-11-28T18:26:55.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c14ff640 1 -- 172.21.15.105:0/1345769859 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 380453840 0 0) 0x7fe7a8005740 con 0x7fe7bc159f40 2023-11-28T18:26:55.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c14ff640 1 -- 172.21.15.105:0/1345769859 >> v2:172.21.15.148:3300/0 conn(0x7fe7bc0cc760 msgr2=0x7fe7bc164d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:55.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c14ff640 1 --2- 172.21.15.105:0/1345769859 >> v2:172.21.15.148:3300/0 conn(0x7fe7bc0cc760 0x7fe7bc164d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:55.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c14ff640 1 -- 172.21.15.105:0/1345769859 >> v1:172.21.15.148:6789/0 conn(0x7fe7bc0caf60 legacy=0x7fe7bc0ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:55.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c14ff640 1 -- 172.21.15.105:0/1345769859 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc15a320 con 0x7fe7bc159f40 2023-11-28T18:26:55.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c14ff640 1 -- 172.21.15.105:0/1345769859 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fe7b8001680 con 0x7fe7bc159f40 2023-11-28T18:26:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c14ff640 1 -- 172.21.15.105:0/1345769859 >> v1:172.21.15.105:6789/0 conn(0x7fe7bc159f40 legacy=0x7fe7bc0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c14ff640 1 --2- 172.21.15.105:0/1345769859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7a8005d20 0x7fe7a8006120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c14ff640 1 --2- 172.21.15.105:0/1345769859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7a80069e0 0x7fe7a8006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c14ff640 1 --2- 172.21.15.105:0/1345769859 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7a80073a0 0x7fe7a800dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c14ff640 1 -- 172.21.15.105:0/1345769859 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc0cb3c0 con 0x7fe7a80069e0 2023-11-28T18:26:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c14ff640 1 -- 172.21.15.105:0/1345769859 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a800e2d0 con 0x7fe7a8005d20 2023-11-28T18:26:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c14ff640 1 -- 172.21.15.105:0/1345769859 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a800e490 con 0x7fe7a80073a0 2023-11-28T18:26:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c2d02640 1 --2- 172.21.15.105:0/1345769859 >> v2:172.21.15.148:3300/0 conn(0x7fe7bc0cc760 0x7fe7bc164d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:26:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c1d00640 1 --2- 172.21.15.105:0/1345769859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7a80069e0 0x7fe7a8006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c2501640 1 --2- 172.21.15.105:0/1345769859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7a8005d20 0x7fe7a8006120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c2d02640 1 --2- 172.21.15.105:0/1345769859 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7a80073a0 0x7fe7a800dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:55.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c1d00640 1 -- 172.21.15.105:0/1345769859 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7a80073a0 msgr2=0x7fe7a800dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:55.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c1d00640 1 --2- 172.21.15.105:0/1345769859 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7a80073a0 0x7fe7a800dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:55.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c1d00640 1 -- 172.21.15.105:0/1345769859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7a8005d20 msgr2=0x7fe7a8006120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:55.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c1d00640 1 --2- 172.21.15.105:0/1345769859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7a8005d20 0x7fe7a8006120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:55.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c1d00640 1 -- 172.21.15.105:0/1345769859 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a8005740 con 0x7fe7a80069e0 2023-11-28T18:26:55.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.931+0000 7fe7c2d02640 1 -- 172.21.15.105:0/1345769859 reap_dead start 2023-11-28T18:26:55.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.935+0000 7fe7c1d00640 1 --2- 172.21.15.105:0/1345769859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7a80069e0 0x7fe7a8006e60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac0018d0 tx=0x7fe7ac01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b605a68201d96f7a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:55.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.935+0000 7fe7c14ff640 1 -- 172.21.15.105:0/1345769859 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac0052d0 con 0x7fe7a80069e0 2023-11-28T18:26:55.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.935+0000 7fe7c14ff640 1 -- 172.21.15.105:0/1345769859 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ac0a7050 con 0x7fe7a80069e0 2023-11-28T18:26:55.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.935+0000 7fe7c14ff640 1 -- 172.21.15.105:0/1345769859 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac003600 con 0x7fe7a80069e0 2023-11-28T18:26:55.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.935+0000 7fe7c4765640 1 -- 172.21.15.105:0/1345769859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7a80069e0 msgr2=0x7fe7a8006e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:55.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.935+0000 7fe7c4765640 1 --2- 172.21.15.105:0/1345769859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7a80069e0 0x7fe7a8006e60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac0018d0 tx=0x7fe7ac01c4d0 comp rx=0 tx=0).stop 2023-11-28T18:26:55.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.935+0000 7fe7c4765640 1 -- 172.21.15.105:0/1345769859 shutdown_connections 2023-11-28T18:26:55.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.935+0000 7fe7c4765640 1 --2- 172.21.15.105:0/1345769859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7a80069e0 0x7fe7a8006e60 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:55.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.935+0000 7fe7c4765640 1 -- 172.21.15.105:0/1345769859 >> 172.21.15.105:0/1345769859 conn(0x7fe7bc0ceb90 msgr2=0x7fe7bc0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:55.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.935+0000 7fe7c4765640 1 -- 172.21.15.105:0/1345769859 shutdown_connections 2023-11-28T18:26:55.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.935+0000 7fe7c4765640 1 -- 172.21.15.105:0/1345769859 wait complete. 2023-11-28T18:26:55.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7c4765640 1 Processor -- start 2023-11-28T18:26:55.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7c4765640 1 -- start start 2023-11-28T18:26:55.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7c4765640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7a80069e0 0x7fe7bc09a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:55.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7c4765640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7bc099ac0 0x7fe7bc0b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:55.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7c4765640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7bc0b19b0 0x7fe7bc14f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:55.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7c4765640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc15c0f0 con 0x7fe7bc0b19b0 2023-11-28T18:26:55.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7c4765640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc15ba70 con 0x7fe7bc099ac0 2023-11-28T18:26:55.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7c4765640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc09b0b0 con 0x7fe7a80069e0 2023-11-28T18:26:55.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7c2d02640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7bc0b19b0 0x7fe7bc14f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:55.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7c2d02640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7bc0b19b0 0x7fe7bc14f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50818/0 (socket says 172.21.15.105:50818) 2023-11-28T18:26:55.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7c2d02640 1 -- 172.21.15.105:0/856521269 learned_addr learned my addr 172.21.15.105:0/856521269 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:55.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7c2501640 1 --2- 172.21.15.105:0/856521269 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7a80069e0 0x7fe7bc09a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:55.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7c1d00640 1 --2- 172.21.15.105:0/856521269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7bc099ac0 0x7fe7bc0b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:55.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7c2d02640 1 -- 172.21.15.105:0/856521269 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7a80069e0 msgr2=0x7fe7bc09a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:55.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7c2d02640 1 --2- 172.21.15.105:0/856521269 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7a80069e0 0x7fe7bc09a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:55.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7c2d02640 1 -- 172.21.15.105:0/856521269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7bc099ac0 msgr2=0x7fe7bc0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:55.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7c2d02640 1 --2- 172.21.15.105:0/856521269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7bc099ac0 0x7fe7bc0b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:55.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7c2d02640 1 -- 172.21.15.105:0/856521269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc09b390 con 0x7fe7bc0b19b0 2023-11-28T18:26:55.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7c2d02640 1 --2- 172.21.15.105:0/856521269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7bc0b19b0 0x7fe7bc14f660 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0b4e80 tx=0x7fe7b400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:55.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7b2ffd640 1 -- 172.21.15.105:0/856521269 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b400e040 con 0x7fe7bc0b19b0 2023-11-28T18:26:55.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7b2ffd640 1 -- 172.21.15.105:0/856521269 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b400f040 con 0x7fe7bc0b19b0 2023-11-28T18:26:55.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7b2ffd640 1 -- 172.21.15.105:0/856521269 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4003be0 con 0x7fe7bc0b19b0 2023-11-28T18:26:55.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7c4765640 1 -- 172.21.15.105:0/856521269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7bc15bd10 con 0x7fe7bc0b19b0 2023-11-28T18:26:55.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.939+0000 7fe7c4765640 1 -- 172.21.15.105:0/856521269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7bc0b46c0 con 0x7fe7bc0b19b0 2023-11-28T18:26:55.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.943+0000 7fe7c4765640 1 -- 172.21.15.105:0/856521269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7bc15c230 con 0x7fe7bc0b19b0 2023-11-28T18:26:55.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.943+0000 7fe7b2ffd640 1 -- 172.21.15.105:0/856521269 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fe7b4015430 con 0x7fe7bc0b19b0 2023-11-28T18:26:55.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.943+0000 7fe7b2ffd640 1 --2- 172.21.15.105:0/856521269 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe7a00668d0 0x7fe7a0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:55.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.943+0000 7fe7b2ffd640 1 -- 172.21.15.105:0/856521269 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5521+0+0 (secure 0 0 0) 0x7fe7b4084d60 con 0x7fe7bc0b19b0 2023-11-28T18:26:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.943+0000 7fe7c2501640 1 --2- 172.21.15.105:0/856521269 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe7a00668d0 0x7fe7a0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.943+0000 7fe7c2501640 1 --2- 172.21.15.105:0/856521269 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe7a00668d0 0x7fe7a0068d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8000c10 tx=0x7fe7b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:55.947+0000 7fe7b2ffd640 1 -- 172.21.15.105:0/856521269 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe7bc15c230 con 0x7fe7bc0b19b0 2023-11-28T18:26:56.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.099+0000 7fe7c4765640 1 -- 172.21.15.105:0/856521269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe7bc15c230 con 0x7fe7bc0b19b0 2023-11-28T18:26:56.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.103+0000 7fe7b2ffd640 1 -- 172.21.15.105:0/856521269 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v24) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe7bc15c230 con 0x7fe7bc0b19b0 2023-11-28T18:26:56.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.107+0000 7fe7c4765640 1 -- 172.21.15.105:0/856521269 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe7a00668d0 msgr2=0x7fe7a0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:56.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.107+0000 7fe7c4765640 1 --2- 172.21.15.105:0/856521269 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe7a00668d0 0x7fe7a0068d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8000c10 tx=0x7fe7b8009040 comp rx=0 tx=0).stop 2023-11-28T18:26:56.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.107+0000 7fe7c4765640 1 -- 172.21.15.105:0/856521269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7bc0b19b0 msgr2=0x7fe7bc14f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:56.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.107+0000 7fe7c4765640 1 --2- 172.21.15.105:0/856521269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7bc0b19b0 0x7fe7bc14f660 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0b4e80 tx=0x7fe7b400f580 comp rx=0 tx=0).stop 2023-11-28T18:26:56.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.107+0000 7fe7c4765640 1 -- 172.21.15.105:0/856521269 shutdown_connections 2023-11-28T18:26:56.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.107+0000 7fe7c4765640 1 --2- 172.21.15.105:0/856521269 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe7a00668d0 0x7fe7a0068d90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.107+0000 7fe7c4765640 1 --2- 172.21.15.105:0/856521269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7bc0b19b0 0x7fe7bc14f660 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.107+0000 7fe7c4765640 1 --2- 172.21.15.105:0/856521269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7bc099ac0 0x7fe7bc0b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.107+0000 7fe7c4765640 1 --2- 172.21.15.105:0/856521269 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7a80069e0 0x7fe7bc09a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.107+0000 7fe7c4765640 1 -- 172.21.15.105:0/856521269 >> 172.21.15.105:0/856521269 conn(0x7fe7bc0ceb90 msgr2=0x7fe7bc0cd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:56.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.107+0000 7fe7c4765640 1 -- 172.21.15.105:0/856521269 shutdown_connections 2023-11-28T18:26:56.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.107+0000 7fe7c4765640 1 -- 172.21.15.105:0/856521269 wait complete. 2023-11-28T18:26:56.130 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_distributed 2023-11-28T18:26:56.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f10625f5640 1 Processor -- start 2023-11-28T18:26:56.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f10625f5640 1 -- start start 2023-11-28T18:26:56.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f10625f5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f105c0caf60 0x7f105c0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:56.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f10625f5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f105c06bf50 con 0x7f105c0ad850 2023-11-28T18:26:56.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f10625f5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f105c15a320 con 0x7f105c159f40 2023-11-28T18:26:56.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f10625f5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105c0cb3c0 con 0x7f105c0caf60 2023-11-28T18:26:56.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f1060b92640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f105c0ad850 0x7f105c0701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:53540/0 (socket says 172.21.15.105:53540) 2023-11-28T18:26:56.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f1060b92640 1 -- 172.21.15.105:0/3569257909 learned_addr learned my addr 172.21.15.105:0/3569257909 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:56.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f105b7fe640 1 --2- 172.21.15.105:0/3569257909 >> v2:172.21.15.148:3300/0 conn(0x7f105c0caf60 0x7f105c0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:56.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f105affd640 1 -- 172.21.15.105:0/3569257909 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1442406942 0 0) 0x7f105c06bf50 con 0x7f105c0ad850 2023-11-28T18:26:56.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f105affd640 1 -- 172.21.15.105:0/3569257909 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1044003530 con 0x7f105c0ad850 2023-11-28T18:26:56.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f105affd640 1 -- 172.21.15.105:0/3569257909 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3406809534 0 0) 0x7f105c15a320 con 0x7f105c159f40 2023-11-28T18:26:56.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f105affd640 1 -- 172.21.15.105:0/3569257909 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f105c06bf50 con 0x7f105c159f40 2023-11-28T18:26:56.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f105affd640 1 -- 172.21.15.105:0/3569257909 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3905974930 0 0) 0x7f1044003530 con 0x7f105c0ad850 2023-11-28T18:26:56.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f105affd640 1 -- 172.21.15.105:0/3569257909 >> v2:172.21.15.148:3300/0 conn(0x7f105c0caf60 msgr2=0x7f105c0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:56.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f105affd640 1 --2- 172.21.15.105:0/3569257909 >> v2:172.21.15.148:3300/0 conn(0x7f105c0caf60 0x7f105c0cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f105affd640 1 -- 172.21.15.105:0/3569257909 >> v1:172.21.15.148:6789/0 conn(0x7f105c159f40 legacy=0x7f105c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:56.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f105affd640 1 -- 172.21.15.105:0/3569257909 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c071380 con 0x7f105c0ad850 2023-11-28T18:26:56.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f105affd640 1 -- 172.21.15.105:0/3569257909 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f1054001a00 con 0x7f105c0ad850 2023-11-28T18:26:56.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f105affd640 1 -- 172.21.15.105:0/3569257909 >> v1:172.21.15.105:6789/0 conn(0x7f105c0ad850 legacy=0x7f105c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:56.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f105affd640 1 --2- 172.21.15.105:0/3569257909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1044003b10 0x7f1044003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:56.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f105affd640 1 --2- 172.21.15.105:0/3569257909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10440047d0 0x7f1044004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:56.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f105affd640 1 --2- 172.21.15.105:0/3569257909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1044005190 0x7f104400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:56.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f105affd640 1 -- 172.21.15.105:0/3569257909 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c0cb3c0 con 0x7f1044005190 2023-11-28T18:26:56.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f105affd640 1 -- 172.21.15.105:0/3569257909 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104400c0c0 con 0x7f1044003b10 2023-11-28T18:26:56.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f105affd640 1 -- 172.21.15.105:0/3569257909 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f104400c280 con 0x7f10440047d0 2023-11-28T18:26:56.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.267+0000 7f105b7fe640 1 --2- 172.21.15.105:0/3569257909 >> v2:172.21.15.148:3300/0 conn(0x7f105c0caf60 0x7f105c0cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:26:56.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.271+0000 7f1060b92640 1 --2- 172.21.15.105:0/3569257909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1044005190 0x7f104400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:56.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.271+0000 7f105bfff640 1 --2- 172.21.15.105:0/3569257909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1044003b10 0x7f1044003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:56.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.271+0000 7f105b7fe640 1 --2- 172.21.15.105:0/3569257909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10440047d0 0x7f1044004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:56.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.271+0000 7f1060b92640 1 -- 172.21.15.105:0/3569257909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10440047d0 msgr2=0x7f1044004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:56.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.271+0000 7f1060b92640 1 --2- 172.21.15.105:0/3569257909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10440047d0 0x7f1044004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.271+0000 7f1060b92640 1 -- 172.21.15.105:0/3569257909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1044003b10 msgr2=0x7f1044003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:56.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.271+0000 7f1060b92640 1 --2- 172.21.15.105:0/3569257909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1044003b10 0x7f1044003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.271+0000 7f1060b92640 1 -- 172.21.15.105:0/3569257909 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1044003530 con 0x7f1044005190 2023-11-28T18:26:56.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.271+0000 7f1060b92640 1 -- 172.21.15.105:0/3569257909 reap_dead start 2023-11-28T18:26:56.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.271+0000 7f1060b92640 1 --2- 172.21.15.105:0/3569257909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1044005190 0x7f104400b990 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f105400a7a0 tx=0x7f105401d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42dfa504c6184bfe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:56.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.271+0000 7f105affd640 1 -- 172.21.15.105:0/3569257909 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10540ac620 con 0x7f1044005190 2023-11-28T18:26:56.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.271+0000 7f105affd640 1 -- 172.21.15.105:0/3569257909 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10540a8070 con 0x7f1044005190 2023-11-28T18:26:56.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.271+0000 7f105affd640 1 -- 172.21.15.105:0/3569257909 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10540ac9e0 con 0x7f1044005190 2023-11-28T18:26:56.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f10625f5640 1 -- 172.21.15.105:0/3569257909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1044005190 msgr2=0x7f104400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:56.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f10625f5640 1 --2- 172.21.15.105:0/3569257909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1044005190 0x7f104400b990 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f105400a7a0 tx=0x7f105401d2a0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f10625f5640 1 -- 172.21.15.105:0/3569257909 shutdown_connections 2023-11-28T18:26:56.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f10625f5640 1 --2- 172.21.15.105:0/3569257909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1044005190 0x7f104400b990 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f10625f5640 1 -- 172.21.15.105:0/3569257909 >> 172.21.15.105:0/3569257909 conn(0x7f105c0ceb90 msgr2=0x7f105c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:56.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f10625f5640 1 -- 172.21.15.105:0/3569257909 shutdown_connections 2023-11-28T18:26:56.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f10625f5640 1 -- 172.21.15.105:0/3569257909 wait complete. 2023-11-28T18:26:56.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f10625f5640 1 Processor -- start 2023-11-28T18:26:56.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f10625f5640 1 -- start start 2023-11-28T18:26:56.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f10625f5640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1044005190 0x7f105c0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:56.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f10625f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f105c0b4300 0x7f105c0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:56.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f10625f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c15c4f0 0x7f105c15c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:56.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f10625f5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c15ceb0 con 0x7f105c15c4f0 2023-11-28T18:26:56.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f10625f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c09a940 con 0x7f105c0b4300 2023-11-28T18:26:56.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f10625f5640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105c0b1ec0 con 0x7f1044005190 2023-11-28T18:26:56.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f1060b92640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c15c4f0 0x7f105c15c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:56.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f105bfff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1044005190 0x7f105c0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:56.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f105bfff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1044005190 0x7f105c0b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:41670/0 (socket says 172.21.15.105:41670) 2023-11-28T18:26:56.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f105bfff640 1 -- 172.21.15.105:0/606176676 learned_addr learned my addr 172.21.15.105:0/606176676 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:56.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f105b7fe640 1 --2- 172.21.15.105:0/606176676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f105c0b4300 0x7f105c0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:56.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f1060b92640 1 -- 172.21.15.105:0/606176676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1044005190 msgr2=0x7f105c0b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:56.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f1060b92640 1 --2- 172.21.15.105:0/606176676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1044005190 0x7f105c0b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f1060b92640 1 -- 172.21.15.105:0/606176676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f105c0b4300 msgr2=0x7f105c0b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:56.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f1060b92640 1 --2- 172.21.15.105:0/606176676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f105c0b4300 0x7f105c0b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.275+0000 7f1060b92640 1 -- 172.21.15.105:0/606176676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c099a50 con 0x7f105c15c4f0 2023-11-28T18:26:56.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.279+0000 7f1060b92640 1 --2- 172.21.15.105:0/606176676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c15c4f0 0x7f105c15c970 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f105c0b4e80 tx=0x7f1044005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:56.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.279+0000 7f1058ff9640 1 -- 172.21.15.105:0/606176676 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1054004d70 con 0x7f105c15c4f0 2023-11-28T18:26:56.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.279+0000 7f1058ff9640 1 -- 172.21.15.105:0/606176676 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1054005030 con 0x7f105c15c4f0 2023-11-28T18:26:56.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.279+0000 7f10625f5640 1 -- 172.21.15.105:0/606176676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105c099d30 con 0x7f105c15c4f0 2023-11-28T18:26:56.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.279+0000 7f1058ff9640 1 -- 172.21.15.105:0/606176676 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10540053e0 con 0x7f105c15c4f0 2023-11-28T18:26:56.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.279+0000 7f10625f5640 1 -- 172.21.15.105:0/606176676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105c09a2f0 con 0x7f105c15c4f0 2023-11-28T18:26:56.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.279+0000 7f10625f5640 1 -- 172.21.15.105:0/606176676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105c09abf0 con 0x7f105c15c4f0 2023-11-28T18:26:56.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.279+0000 7f1058ff9640 1 -- 172.21.15.105:0/606176676 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f10540aa070 con 0x7f105c15c4f0 2023-11-28T18:26:56.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.279+0000 7f1058ff9640 1 --2- 172.21.15.105:0/606176676 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10480668f0 0x7f1048068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:56.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.279+0000 7f1058ff9640 1 -- 172.21.15.105:0/606176676 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f10540b3070 con 0x7f105c15c4f0 2023-11-28T18:26:56.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.279+0000 7f105bfff640 1 --2- 172.21.15.105:0/606176676 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10480668f0 0x7f1048068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:56.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.279+0000 7f105bfff640 1 --2- 172.21.15.105:0/606176676 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10480668f0 0x7f1048068db0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f104c005630 tx=0x7f104c001490 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:56.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.283+0000 7f1058ff9640 1 -- 172.21.15.105:0/606176676 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f10540eb170 con 0x7f105c15c4f0 2023-11-28T18:26:56.443 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.439+0000 7f10625f5640 1 -- 172.21.15.105:0/606176676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"} v 0) v1 -- 0x7f105c067570 con 0x7f105c15c4f0 2023-11-28T18:26:56.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.443+0000 7f1058ff9640 1 -- 172.21.15.105:0/606176676 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"}]=0 v25) v1 ==== 115+0+0 (secure 0 0 0) 0x7f10540ee790 con 0x7f105c15c4f0 2023-11-28T18:26:56.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.447+0000 7f10625f5640 1 -- 172.21.15.105:0/606176676 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10480668f0 msgr2=0x7f1048068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:56.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.447+0000 7f10625f5640 1 --2- 172.21.15.105:0/606176676 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10480668f0 0x7f1048068db0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f104c005630 tx=0x7f104c001490 comp rx=0 tx=0).stop 2023-11-28T18:26:56.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.447+0000 7f10625f5640 1 -- 172.21.15.105:0/606176676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c15c4f0 msgr2=0x7f105c15c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:56.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.447+0000 7f10625f5640 1 --2- 172.21.15.105:0/606176676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c15c4f0 0x7f105c15c970 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f105c0b4e80 tx=0x7f1044005800 comp rx=0 tx=0).stop 2023-11-28T18:26:56.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.447+0000 7f10625f5640 1 -- 172.21.15.105:0/606176676 shutdown_connections 2023-11-28T18:26:56.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.447+0000 7f10625f5640 1 --2- 172.21.15.105:0/606176676 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10480668f0 0x7f1048068db0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.447+0000 7f10625f5640 1 --2- 172.21.15.105:0/606176676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c15c4f0 0x7f105c15c970 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.447+0000 7f10625f5640 1 --2- 172.21.15.105:0/606176676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f105c0b4300 0x7f105c0b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.447+0000 7f10625f5640 1 --2- 172.21.15.105:0/606176676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1044005190 0x7f105c0b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.447+0000 7f10625f5640 1 -- 172.21.15.105:0/606176676 >> 172.21.15.105:0/606176676 conn(0x7f105c0ceb90 msgr2=0x7f105c0ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:56.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.447+0000 7f10625f5640 1 -- 172.21.15.105:0/606176676 shutdown_connections 2023-11-28T18:26:56.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.447+0000 7f10625f5640 1 -- 172.21.15.105:0/606176676 wait complete. 2023-11-28T18:26:56.469 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random_max 2023-11-28T18:26:56.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.603+0000 7fc6746d6640 1 Processor -- start 2023-11-28T18:26:56.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.603+0000 7fc6746d6640 1 -- start start 2023-11-28T18:26:56.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.607+0000 7fc6746d6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc66c0b4290 0x7fc66c0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:56.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.607+0000 7fc6746d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc66c071960 0x7fc66c15f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:56.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.607+0000 7fc6746d6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc66c0b4bb0 con 0x7fc66c15b030 2023-11-28T18:26:56.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.607+0000 7fc6746d6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc66c0701d0 con 0x7fc66c0b4290 2023-11-28T18:26:56.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.607+0000 7fc6746d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c06df80 con 0x7fc66c071960 2023-11-28T18:26:56.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.607+0000 7fc672c73640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc66c15b030 0x7fc66c0709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:53544/0 (socket says 172.21.15.105:53544) 2023-11-28T18:26:56.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.607+0000 7fc672c73640 1 -- 172.21.15.105:0/2057857677 learned_addr learned my addr 172.21.15.105:0/2057857677 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:56.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.607+0000 7fc671c71640 1 --2- 172.21.15.105:0/2057857677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc66c071960 0x7fc66c15f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:56.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.607+0000 7fc672472640 1 --2- 172.21.15.105:0/2057857677 >> v2:172.21.15.148:3300/0 conn(0x7fc66c0b4290 0x7fc66c0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:56.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.607+0000 7fc671470640 1 -- 172.21.15.105:0/2057857677 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1108068296 0 0) 0x7fc66c0b4bb0 con 0x7fc66c15b030 2023-11-28T18:26:56.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.607+0000 7fc671470640 1 -- 172.21.15.105:0/2057857677 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc658003530 con 0x7fc66c15b030 2023-11-28T18:26:56.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.607+0000 7fc671c71640 1 -- 172.21.15.105:0/2057857677 >> v2:172.21.15.148:3300/0 conn(0x7fc66c0b4290 msgr2=0x7fc66c0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:56.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.607+0000 7fc671c71640 1 --2- 172.21.15.105:0/2057857677 >> v2:172.21.15.148:3300/0 conn(0x7fc66c0b4290 0x7fc66c0b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.613 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.607+0000 7fc671c71640 1 -- 172.21.15.105:0/2057857677 >> v1:172.21.15.105:6789/0 conn(0x7fc66c15b030 legacy=0x7fc66c0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:56.613 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.607+0000 7fc671c71640 1 -- 172.21.15.105:0/2057857677 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66c071df0 con 0x7fc66c071960 2023-11-28T18:26:56.613 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.611+0000 7fc671c71640 1 --2- 172.21.15.105:0/2057857677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc66c071960 0x7fc66c15f3b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc65c0097c0 tx=0x7fc65c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbe670a83c469ebb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:56.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.611+0000 7fc671470640 1 -- 172.21.15.105:0/2057857677 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c0ab650 con 0x7fc66c071960 2023-11-28T18:26:56.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.611+0000 7fc671470640 1 -- 172.21.15.105:0/2057857677 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65c0ab7b0 con 0x7fc66c071960 2023-11-28T18:26:56.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.611+0000 7fc671470640 1 -- 172.21.15.105:0/2057857677 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c0abb40 con 0x7fc66c071960 2023-11-28T18:26:56.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.611+0000 7fc6746d6640 1 -- 172.21.15.105:0/2057857677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc66c071960 msgr2=0x7fc66c15f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:56.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.611+0000 7fc6746d6640 1 --2- 172.21.15.105:0/2057857677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc66c071960 0x7fc66c15f3b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc65c0097c0 tx=0x7fc65c01c450 comp rx=0 tx=0).stop 2023-11-28T18:26:56.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.611+0000 7fc6746d6640 1 -- 172.21.15.105:0/2057857677 shutdown_connections 2023-11-28T18:26:56.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.611+0000 7fc6746d6640 1 --2- 172.21.15.105:0/2057857677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc66c071960 0x7fc66c15f3b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.611+0000 7fc6746d6640 1 --2- 172.21.15.105:0/2057857677 >> v2:172.21.15.148:3300/0 conn(0x7fc66c0b4290 0x7fc66c0b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.611+0000 7fc6746d6640 1 -- 172.21.15.105:0/2057857677 >> 172.21.15.105:0/2057857677 conn(0x7fc66c0ceb90 msgr2=0x7fc66c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:56.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.611+0000 7fc6746d6640 1 -- 172.21.15.105:0/2057857677 shutdown_connections 2023-11-28T18:26:56.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.611+0000 7fc6746d6640 1 -- 172.21.15.105:0/2057857677 wait complete. 2023-11-28T18:26:56.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.611+0000 7fc6746d6640 1 Processor -- start 2023-11-28T18:26:56.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.611+0000 7fc6746d6640 1 -- start start 2023-11-28T18:26:56.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc6746d6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc66c0b4290 0x7fc66c1f5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:56.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc6746d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc66c15b030 0x7fc66c1f6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:56.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc6746d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc66c099a50 0x7fc66c09ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:56.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc6746d6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c09ba10 con 0x7fc66c0b4290 2023-11-28T18:26:56.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc6746d6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c1f66b0 con 0x7fc66c15b030 2023-11-28T18:26:56.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc6746d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c09ab90 con 0x7fc66c099a50 2023-11-28T18:26:56.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc672472640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc66c0b4290 0x7fc66c1f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:56.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc672472640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc66c0b4290 0x7fc66c1f5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50838/0 (socket says 172.21.15.105:50838) 2023-11-28T18:26:56.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc672472640 1 -- 172.21.15.105:0/3215168656 learned_addr learned my addr 172.21.15.105:0/3215168656 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:56.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc671c71640 1 --2- 172.21.15.105:0/3215168656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc66c15b030 0x7fc66c1f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:56.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc672472640 1 -- 172.21.15.105:0/3215168656 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc66c099a50 msgr2=0x7fc66c09ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:56.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc672472640 1 --2- 172.21.15.105:0/3215168656 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc66c099a50 0x7fc66c09ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc672472640 1 -- 172.21.15.105:0/3215168656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc66c15b030 msgr2=0x7fc66c1f6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:56.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc672472640 1 --2- 172.21.15.105:0/3215168656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc66c15b030 0x7fc66c1f6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc672472640 1 -- 172.21.15.105:0/3215168656 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65c009420 con 0x7fc66c0b4290 2023-11-28T18:26:56.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc671c71640 1 --2- 172.21.15.105:0/3215168656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc66c15b030 0x7fc66c1f6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:56.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc672472640 1 --2- 172.21.15.105:0/3215168656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc66c0b4290 0x7fc66c1f5140 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc66800e930 tx=0x7fc66800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:56.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc662ffd640 1 -- 172.21.15.105:0/3215168656 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc668009690 con 0x7fc66c0b4290 2023-11-28T18:26:56.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc662ffd640 1 -- 172.21.15.105:0/3215168656 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66800ca00 con 0x7fc66c0b4290 2023-11-28T18:26:56.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc6746d6640 1 -- 172.21.15.105:0/3215168656 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc66c09a010 con 0x7fc66c0b4290 2023-11-28T18:26:56.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc662ffd640 1 -- 172.21.15.105:0/3215168656 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66800cd70 con 0x7fc66c0b4290 2023-11-28T18:26:56.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc6746d6640 1 -- 172.21.15.105:0/3215168656 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc66c09a5d0 con 0x7fc66c0b4290 2023-11-28T18:26:56.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc6746d6640 1 -- 172.21.15.105:0/3215168656 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66c067570 con 0x7fc66c0b4290 2023-11-28T18:26:56.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.615+0000 7fc662ffd640 1 -- 172.21.15.105:0/3215168656 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc668013470 con 0x7fc66c0b4290 2023-11-28T18:26:56.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.619+0000 7fc662ffd640 1 --2- 172.21.15.105:0/3215168656 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6500668d0 0x7fc650068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:56.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.619+0000 7fc662ffd640 1 -- 172.21.15.105:0/3215168656 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5521+0+0 (secure 0 0 0) 0x7fc668084660 con 0x7fc66c0b4290 2023-11-28T18:26:56.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.619+0000 7fc671c71640 1 --2- 172.21.15.105:0/3215168656 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6500668d0 0x7fc650068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:56.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.619+0000 7fc671c71640 1 --2- 172.21.15.105:0/3215168656 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6500668d0 0x7fc650068d90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc65c0af000 tx=0x7fc65c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:56.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.623+0000 7fc662ffd640 1 -- 172.21.15.105:0/3215168656 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc668051a00 con 0x7fc66c0b4290 2023-11-28T18:26:56.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.771+0000 7fc6746d6640 1 -- 172.21.15.105:0/3215168656 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"} v 0) v1 -- 0x7fc66c0cf7f0 con 0x7fc66c0b4290 2023-11-28T18:26:56.781 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.779+0000 7fc662ffd640 1 -- 172.21.15.105:0/3215168656 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"}]=0 v26) v1 ==== 114+0+0 (secure 0 0 0) 0x7fc668055020 con 0x7fc66c0b4290 2023-11-28T18:26:56.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.779+0000 7fc6746d6640 1 -- 172.21.15.105:0/3215168656 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6500668d0 msgr2=0x7fc650068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:56.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.779+0000 7fc6746d6640 1 --2- 172.21.15.105:0/3215168656 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6500668d0 0x7fc650068d90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc65c0af000 tx=0x7fc65c008040 comp rx=0 tx=0).stop 2023-11-28T18:26:56.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.779+0000 7fc6746d6640 1 -- 172.21.15.105:0/3215168656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc66c0b4290 msgr2=0x7fc66c1f5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:56.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.779+0000 7fc6746d6640 1 --2- 172.21.15.105:0/3215168656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc66c0b4290 0x7fc66c1f5140 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc66800e930 tx=0x7fc66800edf0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.779+0000 7fc6746d6640 1 -- 172.21.15.105:0/3215168656 shutdown_connections 2023-11-28T18:26:56.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.779+0000 7fc6746d6640 1 --2- 172.21.15.105:0/3215168656 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc66c099a50 0x7fc66c09ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.779+0000 7fc6746d6640 1 --2- 172.21.15.105:0/3215168656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc66c15b030 0x7fc66c1f6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.779+0000 7fc6746d6640 1 --2- 172.21.15.105:0/3215168656 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6500668d0 0x7fc650068d90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.779+0000 7fc6746d6640 1 --2- 172.21.15.105:0/3215168656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc66c0b4290 0x7fc66c1f5140 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.779+0000 7fc6746d6640 1 -- 172.21.15.105:0/3215168656 >> 172.21.15.105:0/3215168656 conn(0x7fc66c0ceb90 msgr2=0x7fc66c0cb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:56.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.779+0000 7fc6746d6640 1 -- 172.21.15.105:0/3215168656 shutdown_connections 2023-11-28T18:26:56.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.779+0000 7fc6746d6640 1 -- 172.21.15.105:0/3215168656 wait complete. 2023-11-28T18:26:56.802 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_failover' 2023-11-28T18:26:56.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.935+0000 7f18df3a7640 1 Processor -- start 2023-11-28T18:26:56.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.935+0000 7f18df3a7640 1 -- start start 2023-11-28T18:26:56.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.935+0000 7f18df3a7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f18d8157990 0x7f18d815ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:56.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.935+0000 7f18df3a7640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d815e0b0 con 0x7f18d81575b0 2023-11-28T18:26:56.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.935+0000 7f18df3a7640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d815e390 con 0x7f18d815d7b0 2023-11-28T18:26:56.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.935+0000 7f18df3a7640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d80ca660 con 0x7f18d8157990 2023-11-28T18:26:56.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18de3a5640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f18d81575b0 0x7f18d80c9f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:53550/0 (socket says 172.21.15.105:53550) 2023-11-28T18:26:56.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18de3a5640 1 -- 172.21.15.105:0/3306180374 learned_addr learned my addr 172.21.15.105:0/3306180374 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:56.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18ddba4640 1 --2- 172.21.15.105:0/3306180374 >> v2:172.21.15.148:3300/0 conn(0x7f18d8157990 0x7f18d815ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:56.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18dd3a3640 1 -- 172.21.15.105:0/3306180374 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3437668148 0 0) 0x7f18d815e0b0 con 0x7f18d81575b0 2023-11-28T18:26:56.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18dd3a3640 1 -- 172.21.15.105:0/3306180374 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18c4005740 con 0x7f18d81575b0 2023-11-28T18:26:56.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18dd3a3640 1 -- 172.21.15.105:0/3306180374 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1602981296 0 0) 0x7f18c4005740 con 0x7f18d81575b0 2023-11-28T18:26:56.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18dd3a3640 1 -- 172.21.15.105:0/3306180374 >> v2:172.21.15.148:3300/0 conn(0x7f18d8157990 msgr2=0x7f18d815ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:56.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18dd3a3640 1 --2- 172.21.15.105:0/3306180374 >> v2:172.21.15.148:3300/0 conn(0x7f18d8157990 0x7f18d815ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18dd3a3640 1 -- 172.21.15.105:0/3306180374 >> v1:172.21.15.148:6789/0 conn(0x7f18d815d7b0 legacy=0x7f18d806cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:56.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18dd3a3640 1 -- 172.21.15.105:0/3306180374 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d815e670 con 0x7f18d81575b0 2023-11-28T18:26:56.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18dd3a3640 1 -- 172.21.15.105:0/3306180374 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f18d4001680 con 0x7f18d81575b0 2023-11-28T18:26:56.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18dd3a3640 1 -- 172.21.15.105:0/3306180374 >> v1:172.21.15.105:6789/0 conn(0x7f18d81575b0 legacy=0x7f18d80c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:56.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18dd3a3640 1 --2- 172.21.15.105:0/3306180374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18c4005b90 0x7f18c4005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:56.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18dd3a3640 1 --2- 172.21.15.105:0/3306180374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18c40069e0 0x7f18c4006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:56.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18dd3a3640 1 --2- 172.21.15.105:0/3306180374 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18c40073a0 0x7f18c400dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:56.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18dd3a3640 1 -- 172.21.15.105:0/3306180374 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d80ca660 con 0x7f18c40069e0 2023-11-28T18:26:56.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18dd3a3640 1 -- 172.21.15.105:0/3306180374 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c400e240 con 0x7f18c4005b90 2023-11-28T18:26:56.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18dd3a3640 1 -- 172.21.15.105:0/3306180374 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c400e400 con 0x7f18c40073a0 2023-11-28T18:26:56.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18ddba4640 1 --2- 172.21.15.105:0/3306180374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18c40069e0 0x7f18c4006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:56.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18de3a5640 1 --2- 172.21.15.105:0/3306180374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18c4005b90 0x7f18c4005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:56.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18deba6640 1 --2- 172.21.15.105:0/3306180374 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18c40073a0 0x7f18c400dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:56.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18ddba4640 1 -- 172.21.15.105:0/3306180374 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18c40073a0 msgr2=0x7f18c400dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:56.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18ddba4640 1 --2- 172.21.15.105:0/3306180374 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18c40073a0 0x7f18c400dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18ddba4640 1 -- 172.21.15.105:0/3306180374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18c4005b90 msgr2=0x7f18c4005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:56.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18ddba4640 1 --2- 172.21.15.105:0/3306180374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18c4005b90 0x7f18c4005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18ddba4640 1 -- 172.21.15.105:0/3306180374 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c4005740 con 0x7f18c40069e0 2023-11-28T18:26:56.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18deba6640 1 --2- 172.21.15.105:0/3306180374 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18c40073a0 0x7f18c400dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:56.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.939+0000 7f18deba6640 1 -- 172.21.15.105:0/3306180374 reap_dead start 2023-11-28T18:26:56.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.943+0000 7f18ddba4640 1 --2- 172.21.15.105:0/3306180374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18c40069e0 0x7f18c4006e60 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f18c8009ae0 tx=0x7f18c801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5947427745ee04 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:56.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.943+0000 7f18dd3a3640 1 -- 172.21.15.105:0/3306180374 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c80a6440 con 0x7f18c40069e0 2023-11-28T18:26:56.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.943+0000 7f18dd3a3640 1 -- 172.21.15.105:0/3306180374 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c80a65a0 con 0x7f18c40069e0 2023-11-28T18:26:56.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.943+0000 7f18dd3a3640 1 -- 172.21.15.105:0/3306180374 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c80a6950 con 0x7f18c40069e0 2023-11-28T18:26:56.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.943+0000 7f18df3a7640 1 -- 172.21.15.105:0/3306180374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18c40069e0 msgr2=0x7f18c4006e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:56.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.943+0000 7f18df3a7640 1 --2- 172.21.15.105:0/3306180374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18c40069e0 0x7f18c4006e60 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f18c8009ae0 tx=0x7f18c801ca50 comp rx=0 tx=0).stop 2023-11-28T18:26:56.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.943+0000 7f18df3a7640 1 -- 172.21.15.105:0/3306180374 shutdown_connections 2023-11-28T18:26:56.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.943+0000 7f18df3a7640 1 --2- 172.21.15.105:0/3306180374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18c40069e0 0x7f18c4006e60 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.943+0000 7f18df3a7640 1 -- 172.21.15.105:0/3306180374 >> 172.21.15.105:0/3306180374 conn(0x7f18d80b0480 msgr2=0x7f18d80b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:56.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.943+0000 7f18df3a7640 1 -- 172.21.15.105:0/3306180374 shutdown_connections 2023-11-28T18:26:56.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.943+0000 7f18df3a7640 1 -- 172.21.15.105:0/3306180374 wait complete. 2023-11-28T18:26:56.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.943+0000 7f18df3a7640 1 Processor -- start 2023-11-28T18:26:56.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.943+0000 7f18df3a7640 1 -- start start 2023-11-28T18:26:56.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18df3a7640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18c40069e0 0x7f18d80ad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:56.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18df3a7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18d8156040 0x7f18d81504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:56.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18df3a7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d80cae40 0x7f18d80cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:56.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18df3a7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d81f3ca0 con 0x7f18d8156040 2023-11-28T18:26:56.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18df3a7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d80cb810 con 0x7f18d80cae40 2023-11-28T18:26:56.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18df3a7640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d81f2dd0 con 0x7f18c40069e0 2023-11-28T18:26:56.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18ddba4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18d8156040 0x7f18d81504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:56.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18ddba4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18d8156040 0x7f18d81504b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50850/0 (socket says 172.21.15.105:50850) 2023-11-28T18:26:56.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18ddba4640 1 -- 172.21.15.105:0/424200021 learned_addr learned my addr 172.21.15.105:0/424200021 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:56.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18deba6640 1 --2- 172.21.15.105:0/424200021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d80cae40 0x7f18d80cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:56.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18de3a5640 1 --2- 172.21.15.105:0/424200021 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18c40069e0 0x7f18d80ad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:56.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18ddba4640 1 -- 172.21.15.105:0/424200021 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18c40069e0 msgr2=0x7f18d80ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:56.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18ddba4640 1 --2- 172.21.15.105:0/424200021 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18c40069e0 0x7f18d80ad9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18ddba4640 1 -- 172.21.15.105:0/424200021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d80cae40 msgr2=0x7f18d80cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:56.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18ddba4640 1 --2- 172.21.15.105:0/424200021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d80cae40 0x7f18d80cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:56.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18ddba4640 1 -- 172.21.15.105:0/424200021 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d81f3020 con 0x7f18d8156040 2023-11-28T18:26:56.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18deba6640 1 --2- 172.21.15.105:0/424200021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d80cae40 0x7f18d80cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:56.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18ddba4640 1 --2- 172.21.15.105:0/424200021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18d8156040 0x7f18d81504b0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f18c8009ae0 tx=0x7f18c80197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:56.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18ceffd640 1 -- 172.21.15.105:0/424200021 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c80a6c20 con 0x7f18d8156040 2023-11-28T18:26:56.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18ceffd640 1 -- 172.21.15.105:0/424200021 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c80a6d80 con 0x7f18d8156040 2023-11-28T18:26:56.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18ceffd640 1 -- 172.21.15.105:0/424200021 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c80b9660 con 0x7f18d8156040 2023-11-28T18:26:56.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18df3a7640 1 -- 172.21.15.105:0/424200021 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d81f3300 con 0x7f18d8156040 2023-11-28T18:26:56.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18df3a7640 1 -- 172.21.15.105:0/424200021 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d81f38c0 con 0x7f18d8156040 2023-11-28T18:26:56.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18df3a7640 1 -- 172.21.15.105:0/424200021 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d81f3ca0 con 0x7f18d8156040 2023-11-28T18:26:56.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.947+0000 7f18ceffd640 1 -- 172.21.15.105:0/424200021 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f18c80a7420 con 0x7f18d8156040 2023-11-28T18:26:56.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.951+0000 7f18ceffd640 1 --2- 172.21.15.105:0/424200021 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f18bc0668d0 0x7f18bc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:56.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.951+0000 7f18ceffd640 1 -- 172.21.15.105:0/424200021 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f18c811ddf0 con 0x7f18d8156040 2023-11-28T18:26:56.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.951+0000 7f18de3a5640 1 --2- 172.21.15.105:0/424200021 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f18bc0668d0 0x7f18bc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:56.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.951+0000 7f18de3a5640 1 --2- 172.21.15.105:0/424200021 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f18bc0668d0 0x7f18bc068d90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f18d4001060 tx=0x7f18d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:56.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:56.955+0000 7f18ceffd640 1 -- 172.21.15.105:0/424200021 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f18c80eb0d0 con 0x7f18d8156040 2023-11-28T18:26:57.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.155+0000 7f18df3a7640 1 -- 172.21.15.105:0/424200021 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_failover"]} v 0) v1 -- 0x7f18d8074580 con 0x7f18d8156040 2023-11-28T18:26:57.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.415+0000 7f18ceffd640 1 -- 172.21.15.105:0/424200021 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_failover"]}]=0 v201) v1 ==== 155+0+0 (secure 0 0 0) 0x7f18c80bf020 con 0x7f18d8156040 2023-11-28T18:26:57.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.415+0000 7f18df3a7640 1 -- 172.21.15.105:0/424200021 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f18bc0668d0 msgr2=0x7f18bc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:57.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.415+0000 7f18df3a7640 1 --2- 172.21.15.105:0/424200021 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f18bc0668d0 0x7f18bc068d90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f18d4001060 tx=0x7f18d4009040 comp rx=0 tx=0).stop 2023-11-28T18:26:57.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.415+0000 7f18df3a7640 1 -- 172.21.15.105:0/424200021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18d8156040 msgr2=0x7f18d81504b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:57.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.415+0000 7f18df3a7640 1 --2- 172.21.15.105:0/424200021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18d8156040 0x7f18d81504b0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f18c8009ae0 tx=0x7f18c80197f0 comp rx=0 tx=0).stop 2023-11-28T18:26:57.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.415+0000 7f18df3a7640 1 -- 172.21.15.105:0/424200021 shutdown_connections 2023-11-28T18:26:57.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.415+0000 7f18df3a7640 1 --2- 172.21.15.105:0/424200021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d80cae40 0x7f18d80cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:57.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.415+0000 7f18df3a7640 1 --2- 172.21.15.105:0/424200021 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f18bc0668d0 0x7f18bc068d90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:57.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.415+0000 7f18df3a7640 1 --2- 172.21.15.105:0/424200021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18d8156040 0x7f18d81504b0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:57.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.415+0000 7f18df3a7640 1 --2- 172.21.15.105:0/424200021 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18c40069e0 0x7f18d80ad9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:57.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.415+0000 7f18df3a7640 1 -- 172.21.15.105:0/424200021 >> 172.21.15.105:0/424200021 conn(0x7f18d80b0480 msgr2=0x7f18d8071950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:57.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.415+0000 7f18df3a7640 1 -- 172.21.15.105:0/424200021 shutdown_connections 2023-11-28T18:26:57.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.415+0000 7f18df3a7640 1 -- 172.21.15.105:0/424200021 wait complete. 2023-11-28T18:26:57.439 INFO:tasks.cephfs_test_runner:That MDS failover does not cause unnecessary migrations. ... ok 2023-11-28T18:26:57.439 INFO:tasks.cephfs_test_runner:Starting test: test_ephemeral_pin_dist_off (tasks.cephfs.test_exports.TestEphemeralPins) That turning off ephemeral distributed pin merges subtrees. 2023-11-28T18:26:57.439 INFO:tasks.cephfs_test_runner:test_ephemeral_pin_dist_off (tasks.cephfs.test_exports.TestEphemeralPins) 2023-11-28T18:26:57.439 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:26:57.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f663bb0f640 1 Processor -- start 2023-11-28T18:26:57.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f663bb0f640 1 -- start start 2023-11-28T18:26:57.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f663bb0f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f663415d4f0 0x7f66340d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:57.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f663bb0f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66340b25b0 con 0x7f66340b2d20 2023-11-28T18:26:57.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f663bb0f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66340cea30 con 0x7f66340d3c30 2023-11-28T18:26:57.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f663bb0f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66340b2900 con 0x7f663415d4f0 2023-11-28T18:26:57.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f663a0ac640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f66340b2d20 0x7f66340b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:53556/0 (socket says 172.21.15.105:53556) 2023-11-28T18:26:57.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f663a0ac640 1 -- 172.21.15.105:0/2139860680 learned_addr learned my addr 172.21.15.105:0/2139860680 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:57.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66398ab640 1 --2- 172.21.15.105:0/2139860680 >> v2:172.21.15.148:3300/0 conn(0x7f663415d4f0 0x7f66340d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:57.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66388a9640 1 -- 172.21.15.105:0/2139860680 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3782828938 0 0) 0x7f66340b25b0 con 0x7f66340b2d20 2023-11-28T18:26:57.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66388a9640 1 -- 172.21.15.105:0/2139860680 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6620003530 con 0x7f66340b2d20 2023-11-28T18:26:57.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66388a9640 1 -- 172.21.15.105:0/2139860680 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3854266050 0 0) 0x7f6620003530 con 0x7f66340b2d20 2023-11-28T18:26:57.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66388a9640 1 -- 172.21.15.105:0/2139860680 >> v2:172.21.15.148:3300/0 conn(0x7f663415d4f0 msgr2=0x7f66340d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:57.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66388a9640 1 --2- 172.21.15.105:0/2139860680 >> v2:172.21.15.148:3300/0 conn(0x7f663415d4f0 0x7f66340d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:57.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66388a9640 1 -- 172.21.15.105:0/2139860680 >> v1:172.21.15.148:6789/0 conn(0x7f66340d3c30 legacy=0x7f66340d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:57.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66388a9640 1 -- 172.21.15.105:0/2139860680 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66340ced10 con 0x7f66340b2d20 2023-11-28T18:26:57.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66388a9640 1 -- 172.21.15.105:0/2139860680 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f662c001a00 con 0x7f66340b2d20 2023-11-28T18:26:57.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66388a9640 1 -- 172.21.15.105:0/2139860680 >> v1:172.21.15.105:6789/0 conn(0x7f66340b2d20 legacy=0x7f66340b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:57.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66388a9640 1 --2- 172.21.15.105:0/2139860680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6620003980 0x7f6620003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:57.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66388a9640 1 --2- 172.21.15.105:0/2139860680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66200047d0 0x7f6620004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:57.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66388a9640 1 --2- 172.21.15.105:0/2139860680 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6620005190 0x7f662000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:57.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66388a9640 1 -- 172.21.15.105:0/2139860680 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66340b2900 con 0x7f6620003980 2023-11-28T18:26:57.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66388a9640 1 -- 172.21.15.105:0/2139860680 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662000c090 con 0x7f66200047d0 2023-11-28T18:26:57.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66388a9640 1 -- 172.21.15.105:0/2139860680 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662000c250 con 0x7f6620005190 2023-11-28T18:26:57.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66398ab640 1 --2- 172.21.15.105:0/2139860680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6620003980 0x7f6620003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:57.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f663a0ac640 1 --2- 172.21.15.105:0/2139860680 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6620005190 0x7f662000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:57.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66390aa640 1 --2- 172.21.15.105:0/2139860680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66200047d0 0x7f6620004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:57.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66398ab640 1 -- 172.21.15.105:0/2139860680 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6620005190 msgr2=0x7f662000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:57.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66398ab640 1 --2- 172.21.15.105:0/2139860680 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6620005190 0x7f662000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:57.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66398ab640 1 -- 172.21.15.105:0/2139860680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66200047d0 msgr2=0x7f6620004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:57.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66398ab640 1 --2- 172.21.15.105:0/2139860680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66200047d0 0x7f6620004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:57.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f66398ab640 1 -- 172.21.15.105:0/2139860680 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6620003530 con 0x7f6620003980 2023-11-28T18:26:57.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.575+0000 7f663a0ac640 1 -- 172.21.15.105:0/2139860680 reap_dead start 2023-11-28T18:26:57.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.579+0000 7f66398ab640 1 --2- 172.21.15.105:0/2139860680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6620003980 0x7f6620003d60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6630009ae0 tx=0x7f663001ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754df4894cacccd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:57.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.579+0000 7f66388a9640 1 -- 172.21.15.105:0/2139860680 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66300a5520 con 0x7f6620003980 2023-11-28T18:26:57.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.579+0000 7f66388a9640 1 -- 172.21.15.105:0/2139860680 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66300a5680 con 0x7f6620003980 2023-11-28T18:26:57.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.579+0000 7f66388a9640 1 -- 172.21.15.105:0/2139860680 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66300a5a30 con 0x7f6620003980 2023-11-28T18:26:57.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.579+0000 7f663bb0f640 1 -- 172.21.15.105:0/2139860680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6620003980 msgr2=0x7f6620003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:57.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.579+0000 7f663bb0f640 1 --2- 172.21.15.105:0/2139860680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6620003980 0x7f6620003d60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6630009ae0 tx=0x7f663001ca40 comp rx=0 tx=0).stop 2023-11-28T18:26:57.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.579+0000 7f663bb0f640 1 -- 172.21.15.105:0/2139860680 shutdown_connections 2023-11-28T18:26:57.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.579+0000 7f663bb0f640 1 --2- 172.21.15.105:0/2139860680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6620003980 0x7f6620003d60 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:57.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.579+0000 7f663bb0f640 1 -- 172.21.15.105:0/2139860680 >> 172.21.15.105:0/2139860680 conn(0x7f66340d3030 msgr2=0x7f66340d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:57.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.579+0000 7f663bb0f640 1 -- 172.21.15.105:0/2139860680 shutdown_connections 2023-11-28T18:26:57.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.579+0000 7f663bb0f640 1 -- 172.21.15.105:0/2139860680 wait complete. 2023-11-28T18:26:57.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f663bb0f640 1 Processor -- start 2023-11-28T18:26:57.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f663bb0f640 1 -- start start 2023-11-28T18:26:57.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f663bb0f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6620003980 0x7f66341555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:57.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f663bb0f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66340ccca0 0x7f66340b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:57.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f663bb0f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f663415e110 0x7f663415e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:57.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f663bb0f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66340d0a20 con 0x7f6620003980 2023-11-28T18:26:57.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f663bb0f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66340ca100 con 0x7f66340ccca0 2023-11-28T18:26:57.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f663bb0f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66340b6510 con 0x7f663415e110 2023-11-28T18:26:57.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f66398ab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6620003980 0x7f66341555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:57.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f66398ab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6620003980 0x7f66341555e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50868/0 (socket says 172.21.15.105:50868) 2023-11-28T18:26:57.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f66398ab640 1 -- 172.21.15.105:0/541647676 learned_addr learned my addr 172.21.15.105:0/541647676 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:57.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f66390aa640 1 --2- 172.21.15.105:0/541647676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66340ccca0 0x7f66340b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:57.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f663a0ac640 1 --2- 172.21.15.105:0/541647676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f663415e110 0x7f663415e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:57.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f66398ab640 1 -- 172.21.15.105:0/541647676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f663415e110 msgr2=0x7f663415e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:57.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f66398ab640 1 --2- 172.21.15.105:0/541647676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f663415e110 0x7f663415e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:57.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f66398ab640 1 -- 172.21.15.105:0/541647676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66340ccca0 msgr2=0x7f66340b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:57.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f66398ab640 1 --2- 172.21.15.105:0/541647676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66340ccca0 0x7f66340b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:57.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f66398ab640 1 -- 172.21.15.105:0/541647676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66340b67f0 con 0x7f6620003980 2023-11-28T18:26:57.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f66390aa640 1 --2- 172.21.15.105:0/541647676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66340ccca0 0x7f66340b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:26:57.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f66398ab640 1 --2- 172.21.15.105:0/541647676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6620003980 0x7f66341555e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f66300b0040 tx=0x7f66300197e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:57.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f662a7fc640 1 -- 172.21.15.105:0/541647676 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66300a5520 con 0x7f6620003980 2023-11-28T18:26:57.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f662a7fc640 1 -- 172.21.15.105:0/541647676 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663000f000 con 0x7f6620003980 2023-11-28T18:26:57.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f663bb0f640 1 -- 172.21.15.105:0/541647676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66340b6ad0 con 0x7f6620003980 2023-11-28T18:26:57.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f662a7fc640 1 -- 172.21.15.105:0/541647676 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6630004240 con 0x7f6620003980 2023-11-28T18:26:57.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.583+0000 7f663bb0f640 1 -- 172.21.15.105:0/541647676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66340b6120 con 0x7f6620003980 2023-11-28T18:26:57.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.587+0000 7f663bb0f640 1 -- 172.21.15.105:0/541647676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66340b6c60 con 0x7f6620003980 2023-11-28T18:26:57.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.587+0000 7f662a7fc640 1 -- 172.21.15.105:0/541647676 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f66300b4070 con 0x7f6620003980 2023-11-28T18:26:57.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.587+0000 7f662a7fc640 1 --2- 172.21.15.105:0/541647676 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f66180668d0 0x7f6618068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:57.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.587+0000 7f662a7fc640 1 -- 172.21.15.105:0/541647676 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f663011e510 con 0x7f6620003980 2023-11-28T18:26:57.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.587+0000 7f66390aa640 1 --2- 172.21.15.105:0/541647676 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f66180668d0 0x7f6618068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:57.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.587+0000 7f66390aa640 1 --2- 172.21.15.105:0/541647676 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f66180668d0 0x7f6618068d90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f66340a0ec0 tx=0x7f6624009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:57.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.591+0000 7f662a7fc640 1 -- 172.21.15.105:0/541647676 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f66300eb7f0 con 0x7f6620003980 2023-11-28T18:26:57.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.743+0000 7f663bb0f640 1 -- 172.21.15.105:0/541647676 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6634067570 con 0x7f6620003980 2023-11-28T18:26:57.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.743+0000 7f662a7fc640 1 -- 172.21.15.105:0/541647676 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+13645 (secure 0 0 0) 0x7f66300eee10 con 0x7f6620003980 2023-11-28T18:26:57.747 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:26:57.747 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":57,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:26:55.405978+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:6836/735646078":"2023-11-29T18:26:52.200953+0000","172.21.15.105:6835/3765684108":"2023-11-29T18:26:52.200966+0000","172.21.15.105:6834/735646078":"2023-11-29T18:26:52.200953+0000","172.21.15.148:6834/2696773413":"2023-11-29T18:26:52.200972+0000","172.21.15.148:6832/2696773413":"2023-11-29T18:26:52.200972+0000","172.21.15.105:6837/3765684108":"2023-11-29T18:26:52.200966+0000","172.21.15.148:6835/897997046":"2023-11-29T18:26:28.339203+0000","172.21.15.148:6833/897997046":"2023-11-29T18:26:28.339203+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:26:57.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.743+0000 7f663bb0f640 1 -- 172.21.15.105:0/541647676 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f66180668d0 msgr2=0x7f6618068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:57.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.743+0000 7f663bb0f640 1 --2- 172.21.15.105:0/541647676 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f66180668d0 0x7f6618068d90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f66340a0ec0 tx=0x7f6624009040 comp rx=0 tx=0).stop 2023-11-28T18:26:57.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.743+0000 7f663bb0f640 1 -- 172.21.15.105:0/541647676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6620003980 msgr2=0x7f66341555e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:57.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.743+0000 7f663bb0f640 1 --2- 172.21.15.105:0/541647676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6620003980 0x7f66341555e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f66300b0040 tx=0x7f66300197e0 comp rx=0 tx=0).stop 2023-11-28T18:26:57.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.743+0000 7f663bb0f640 1 -- 172.21.15.105:0/541647676 shutdown_connections 2023-11-28T18:26:57.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.743+0000 7f663bb0f640 1 --2- 172.21.15.105:0/541647676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f663415e110 0x7f663415e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:57.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.743+0000 7f663bb0f640 1 --2- 172.21.15.105:0/541647676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66340ccca0 0x7f66340b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:57.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.743+0000 7f663bb0f640 1 --2- 172.21.15.105:0/541647676 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f66180668d0 0x7f6618068d90 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:57.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.743+0000 7f663bb0f640 1 --2- 172.21.15.105:0/541647676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6620003980 0x7f66341555e0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:57.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.743+0000 7f663bb0f640 1 -- 172.21.15.105:0/541647676 >> 172.21.15.105:0/541647676 conn(0x7f66340d3030 msgr2=0x7f663415ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:57.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.747+0000 7f663bb0f640 1 -- 172.21.15.105:0/541647676 shutdown_connections 2023-11-28T18:26:57.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.747+0000 7f663bb0f640 1 -- 172.21.15.105:0/541647676 wait complete. 2023-11-28T18:26:57.768 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:26:57.769 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:26:57.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.907+0000 7f5c90772640 1 Processor -- start 2023-11-28T18:26:57.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.907+0000 7f5c90772640 1 -- start start 2023-11-28T18:26:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.907+0000 7f5c90772640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5c880caf60 0x7f5c880cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:57.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.907+0000 7f5c90772640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c880ad850 0x7f5c880701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:57.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.907+0000 7f5c90772640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c8806bf50 con 0x7f5c88159f40 2023-11-28T18:26:57.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.907+0000 7f5c90772640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c880cb3c0 con 0x7f5c880caf60 2023-11-28T18:26:57.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.907+0000 7f5c90772640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8815a320 con 0x7f5c880ad850 2023-11-28T18:26:57.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.907+0000 7f5c8e50e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5c88159f40 0x7f5c880ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:53572/0 (socket says 172.21.15.105:53572) 2023-11-28T18:26:57.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.907+0000 7f5c8e50e640 1 -- 172.21.15.105:0/2173607729 learned_addr learned my addr 172.21.15.105:0/2173607729 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:57.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.907+0000 7f5c8dd0d640 1 --2- 172.21.15.105:0/2173607729 >> v2:172.21.15.105:3300/0 conn(0x7f5c880caf60 0x7f5c880cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:57.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.907+0000 7f5c8d50c640 1 -- 172.21.15.105:0/2173607729 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3193403053 0 0) 0x7f5c8806bf50 con 0x7f5c88159f40 2023-11-28T18:26:57.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.907+0000 7f5c8dd0d640 1 -- 172.21.15.105:0/2173607729 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c880ad850 msgr2=0x7f5c880701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:57.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.907+0000 7f5c8ed0f640 1 --2- 172.21.15.105:0/2173607729 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c880ad850 0x7f5c880701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:57.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.907+0000 7f5c8dd0d640 1 --2- 172.21.15.105:0/2173607729 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c880ad850 0x7f5c880701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:57.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.907+0000 7f5c8dd0d640 1 -- 172.21.15.105:0/2173607729 >> v1:172.21.15.105:6789/0 conn(0x7f5c88159f40 legacy=0x7f5c880ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:57.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.907+0000 7f5c8dd0d640 1 -- 172.21.15.105:0/2173607729 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c88071380 con 0x7f5c880caf60 2023-11-28T18:26:57.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.907+0000 7f5c8ed0f640 1 --2- 172.21.15.105:0/2173607729 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c880ad850 0x7f5c880701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:57.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.911+0000 7f5c8dd0d640 1 --2- 172.21.15.105:0/2173607729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c880caf60 0x7f5c880cc760 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5c780097c0 tx=0x7f5c7801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ecea32b053c8975 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:57.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.911+0000 7f5c8d50c640 1 -- 172.21.15.105:0/2173607729 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c780ac620 con 0x7f5c880caf60 2023-11-28T18:26:57.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.911+0000 7f5c8d50c640 1 -- 172.21.15.105:0/2173607729 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c780ac780 con 0x7f5c880caf60 2023-11-28T18:26:57.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.911+0000 7f5c8d50c640 1 -- 172.21.15.105:0/2173607729 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c780acb10 con 0x7f5c880caf60 2023-11-28T18:26:57.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.911+0000 7f5c90772640 1 -- 172.21.15.105:0/2173607729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c880caf60 msgr2=0x7f5c880cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:57.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.911+0000 7f5c90772640 1 --2- 172.21.15.105:0/2173607729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c880caf60 0x7f5c880cc760 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5c780097c0 tx=0x7f5c7801c3c0 comp rx=0 tx=0).stop 2023-11-28T18:26:57.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.911+0000 7f5c90772640 1 -- 172.21.15.105:0/2173607729 shutdown_connections 2023-11-28T18:26:57.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.911+0000 7f5c90772640 1 --2- 172.21.15.105:0/2173607729 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c880ad850 0x7f5c880701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:57.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.911+0000 7f5c90772640 1 --2- 172.21.15.105:0/2173607729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c880caf60 0x7f5c880cc760 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:57.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.911+0000 7f5c90772640 1 -- 172.21.15.105:0/2173607729 >> 172.21.15.105:0/2173607729 conn(0x7f5c880ceb90 msgr2=0x7f5c880cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:57.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.911+0000 7f5c90772640 1 -- 172.21.15.105:0/2173607729 shutdown_connections 2023-11-28T18:26:57.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.911+0000 7f5c90772640 1 -- 172.21.15.105:0/2173607729 wait complete. 2023-11-28T18:26:57.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.915+0000 7f5c90772640 1 Processor -- start 2023-11-28T18:26:57.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.915+0000 7f5c90772640 1 -- start start 2023-11-28T18:26:57.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.915+0000 7f5c90772640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c880ad850 0x7f5c88162bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:57.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.915+0000 7f5c90772640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c880caf60 0x7f5c880b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:57.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.915+0000 7f5c90772640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c88159f40 0x7f5c88141230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:57.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.915+0000 7f5c90772640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c88099bf0 con 0x7f5c88159f40 2023-11-28T18:26:57.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.915+0000 7f5c90772640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8814dbf0 con 0x7f5c880caf60 2023-11-28T18:26:57.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.915+0000 7f5c90772640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c881632c0 con 0x7f5c880ad850 2023-11-28T18:26:57.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.915+0000 7f5c8ed0f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c88159f40 0x7f5c88141230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:57.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.915+0000 7f5c8ed0f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c88159f40 0x7f5c88141230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50884/0 (socket says 172.21.15.105:50884) 2023-11-28T18:26:57.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.915+0000 7f5c8ed0f640 1 -- 172.21.15.105:0/2426320457 learned_addr learned my addr 172.21.15.105:0/2426320457 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:57.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.915+0000 7f5c8ed0f640 1 -- 172.21.15.105:0/2426320457 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c880ad850 msgr2=0x7f5c88162bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:57.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.915+0000 7f5c8ed0f640 1 --2- 172.21.15.105:0/2426320457 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c880ad850 0x7f5c88162bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:57.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.915+0000 7f5c8ed0f640 1 -- 172.21.15.105:0/2426320457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c880caf60 msgr2=0x7f5c880b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:57.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.915+0000 7f5c8ed0f640 1 --2- 172.21.15.105:0/2426320457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c880caf60 0x7f5c880b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:57.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.915+0000 7f5c8ed0f640 1 -- 172.21.15.105:0/2426320457 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c78009420 con 0x7f5c88159f40 2023-11-28T18:26:57.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.915+0000 7f5c8ed0f640 1 --2- 172.21.15.105:0/2426320457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c88159f40 0x7f5c88141230 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000cd50 tx=0x7f5c8000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:57.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.915+0000 7f5c7effd640 1 -- 172.21.15.105:0/2426320457 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8000f680 con 0x7f5c88159f40 2023-11-28T18:26:57.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.915+0000 7f5c7effd640 1 -- 172.21.15.105:0/2426320457 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8000f800 con 0x7f5c88159f40 2023-11-28T18:26:57.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.915+0000 7f5c90772640 1 -- 172.21.15.105:0/2426320457 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c881635a0 con 0x7f5c88159f40 2023-11-28T18:26:57.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.915+0000 7f5c7effd640 1 -- 172.21.15.105:0/2426320457 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8000fbb0 con 0x7f5c88159f40 2023-11-28T18:26:57.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.915+0000 7f5c90772640 1 -- 172.21.15.105:0/2426320457 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c88163b60 con 0x7f5c88159f40 2023-11-28T18:26:57.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.919+0000 7f5c90772640 1 -- 172.21.15.105:0/2426320457 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c88099d30 con 0x7f5c88159f40 2023-11-28T18:26:57.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.919+0000 7f5c7effd640 1 -- 172.21.15.105:0/2426320457 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5c8000b040 con 0x7f5c88159f40 2023-11-28T18:26:57.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.919+0000 7f5c7effd640 1 --2- 172.21.15.105:0/2426320457 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5c6c0668d0 0x7f5c6c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:57.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.919+0000 7f5c7effd640 1 -- 172.21.15.105:0/2426320457 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f5c80084d90 con 0x7f5c88159f40 2023-11-28T18:26:57.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.919+0000 7f5c8e50e640 1 --2- 172.21.15.105:0/2426320457 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5c6c0668d0 0x7f5c6c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:57.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.919+0000 7f5c8e50e640 1 --2- 172.21.15.105:0/2426320457 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5c6c0668d0 0x7f5c6c068d90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5c840011f0 tx=0x7f5c84007930 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:57.923+0000 7f5c7effd640 1 -- 172.21.15.105:0/2426320457 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5c88099d30 con 0x7f5c88159f40 2023-11-28T18:26:58.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.071+0000 7f5c90772640 1 -- 172.21.15.105:0/2426320457 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5c88099d30 con 0x7f5c88159f40 2023-11-28T18:26:58.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.071+0000 7f5c7effd640 1 -- 172.21.15.105:0/2426320457 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5c88099d30 con 0x7f5c88159f40 2023-11-28T18:26:58.077 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:26:58.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.075+0000 7f5c90772640 1 -- 172.21.15.105:0/2426320457 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5c6c0668d0 msgr2=0x7f5c6c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:58.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.075+0000 7f5c90772640 1 --2- 172.21.15.105:0/2426320457 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5c6c0668d0 0x7f5c6c068d90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5c840011f0 tx=0x7f5c84007930 comp rx=0 tx=0).stop 2023-11-28T18:26:58.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.075+0000 7f5c90772640 1 -- 172.21.15.105:0/2426320457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c88159f40 msgr2=0x7f5c88141230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:58.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.075+0000 7f5c90772640 1 --2- 172.21.15.105:0/2426320457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c88159f40 0x7f5c88141230 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000cd50 tx=0x7f5c8000a650 comp rx=0 tx=0).stop 2023-11-28T18:26:58.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.075+0000 7f5c90772640 1 -- 172.21.15.105:0/2426320457 shutdown_connections 2023-11-28T18:26:58.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.075+0000 7f5c90772640 1 --2- 172.21.15.105:0/2426320457 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5c6c0668d0 0x7f5c6c068d90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:58.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.075+0000 7f5c90772640 1 --2- 172.21.15.105:0/2426320457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c88159f40 0x7f5c88141230 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:58.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.075+0000 7f5c90772640 1 --2- 172.21.15.105:0/2426320457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c880caf60 0x7f5c880b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:58.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.075+0000 7f5c90772640 1 --2- 172.21.15.105:0/2426320457 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c880ad850 0x7f5c88162bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:58.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.075+0000 7f5c90772640 1 -- 172.21.15.105:0/2426320457 >> 172.21.15.105:0/2426320457 conn(0x7f5c880ceb90 msgr2=0x7f5c880cd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:58.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.075+0000 7f5c90772640 1 -- 172.21.15.105:0/2426320457 shutdown_connections 2023-11-28T18:26:58.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.075+0000 7f5c90772640 1 -- 172.21.15.105:0/2426320457 wait complete. 2023-11-28T18:26:58.098 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_off' 2023-11-28T18:26:58.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.231+0000 7f0e9bbe6640 1 Processor -- start 2023-11-28T18:26:58.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.231+0000 7f0e9bbe6640 1 -- start start 2023-11-28T18:26:58.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.235+0000 7f0e9bbe6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e94071a10 0x7f0e940ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:58.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.235+0000 7f0e9bbe6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0e94070a10 0x7f0e94070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:58.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.235+0000 7f0e9bbe6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e940cefa0 con 0x7f0e9415d3c0 2023-11-28T18:26:58.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.235+0000 7f0e9bbe6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e940713c0 con 0x7f0e94070a10 2023-11-28T18:26:58.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.235+0000 7f0e9bbe6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9415d7a0 con 0x7f0e94071a10 2023-11-28T18:26:58.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.235+0000 7f0e99982640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0e9415d3c0 0x7f0e94161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:53574/0 (socket says 172.21.15.105:53574) 2023-11-28T18:26:58.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.235+0000 7f0e99982640 1 -- 172.21.15.105:0/856354331 learned_addr learned my addr 172.21.15.105:0/856354331 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:58.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.235+0000 7f0e9a183640 1 --2- 172.21.15.105:0/856354331 >> v2:172.21.15.148:3300/0 conn(0x7f0e94070a10 0x7f0e94070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:58.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.235+0000 7f0e99181640 1 --2- 172.21.15.105:0/856354331 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e94071a10 0x7f0e940ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:58.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.235+0000 7f0e98980640 1 -- 172.21.15.105:0/856354331 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802276803 0 0) 0x7f0e940cefa0 con 0x7f0e9415d3c0 2023-11-28T18:26:58.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.235+0000 7f0e98980640 1 -- 172.21.15.105:0/856354331 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e80003530 con 0x7f0e9415d3c0 2023-11-28T18:26:58.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.235+0000 7f0e98980640 1 -- 172.21.15.105:0/856354331 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3665436654 0 0) 0x7f0e80003530 con 0x7f0e9415d3c0 2023-11-28T18:26:58.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.235+0000 7f0e9a183640 1 -- 172.21.15.105:0/856354331 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e94071a10 msgr2=0x7f0e940ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:58.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.235+0000 7f0e9a183640 1 --2- 172.21.15.105:0/856354331 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e94071a10 0x7f0e940ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:58.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.235+0000 7f0e9a183640 1 -- 172.21.15.105:0/856354331 >> v1:172.21.15.105:6789/0 conn(0x7f0e9415d3c0 legacy=0x7f0e94161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:58.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.235+0000 7f0e9a183640 1 -- 172.21.15.105:0/856354331 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e940ce9e0 con 0x7f0e94070a10 2023-11-28T18:26:58.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.239+0000 7f0e9a183640 1 --2- 172.21.15.105:0/856354331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e94070a10 0x7f0e94070e80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009d90 tx=0x7f0e8c01c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a5a3cf58b490536 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:58.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.239+0000 7f0e98980640 1 -- 172.21.15.105:0/856354331 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e90001680 con 0x7f0e94070a10 2023-11-28T18:26:58.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.239+0000 7f0e98980640 1 -- 172.21.15.105:0/856354331 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8c0a64d0 con 0x7f0e94070a10 2023-11-28T18:26:58.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.239+0000 7f0e98980640 1 -- 172.21.15.105:0/856354331 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c0a6840 con 0x7f0e94070a10 2023-11-28T18:26:58.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.239+0000 7f0e9bbe6640 1 -- 172.21.15.105:0/856354331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e94070a10 msgr2=0x7f0e94070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:58.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.239+0000 7f0e9bbe6640 1 --2- 172.21.15.105:0/856354331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e94070a10 0x7f0e94070e80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009d90 tx=0x7f0e8c01c7b0 comp rx=0 tx=0).stop 2023-11-28T18:26:58.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.239+0000 7f0e9bbe6640 1 -- 172.21.15.105:0/856354331 shutdown_connections 2023-11-28T18:26:58.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.239+0000 7f0e9bbe6640 1 --2- 172.21.15.105:0/856354331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e94070a10 0x7f0e94070e80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:58.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.239+0000 7f0e9bbe6640 1 --2- 172.21.15.105:0/856354331 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e94071a10 0x7f0e940ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:58.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.239+0000 7f0e9bbe6640 1 -- 172.21.15.105:0/856354331 >> 172.21.15.105:0/856354331 conn(0x7f0e940cf540 msgr2=0x7f0e940cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:58.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.239+0000 7f0e9bbe6640 1 -- 172.21.15.105:0/856354331 shutdown_connections 2023-11-28T18:26:58.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.239+0000 7f0e9bbe6640 1 -- 172.21.15.105:0/856354331 wait complete. 2023-11-28T18:26:58.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.239+0000 7f0e9bbe6640 1 Processor -- start 2023-11-28T18:26:58.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.239+0000 7f0e9bbe6640 1 -- start start 2023-11-28T18:26:58.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.239+0000 7f0e9bbe6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e94070a10 0x7f0e9415fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:58.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e9bbe6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e94071a10 0x7f0e94167850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:58.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e9bbe6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e9415d3c0 0x7f0e941582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:58.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e9bbe6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e94167d90 con 0x7f0e94071a10 2023-11-28T18:26:58.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e9bbe6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e94158830 con 0x7f0e9415d3c0 2023-11-28T18:26:58.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e9bbe6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e94157e60 con 0x7f0e94070a10 2023-11-28T18:26:58.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e99181640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e94071a10 0x7f0e94167850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:58.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e99181640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e94071a10 0x7f0e94167850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50894/0 (socket says 172.21.15.105:50894) 2023-11-28T18:26:58.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e99181640 1 -- 172.21.15.105:0/1357883781 learned_addr learned my addr 172.21.15.105:0/1357883781 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:58.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e9a183640 1 --2- 172.21.15.105:0/1357883781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e9415d3c0 0x7f0e941582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:58.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e99982640 1 --2- 172.21.15.105:0/1357883781 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e94070a10 0x7f0e9415fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:58.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e99181640 1 -- 172.21.15.105:0/1357883781 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e94070a10 msgr2=0x7f0e9415fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:58.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e99181640 1 --2- 172.21.15.105:0/1357883781 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e94070a10 0x7f0e9415fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:58.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e99181640 1 -- 172.21.15.105:0/1357883781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e9415d3c0 msgr2=0x7f0e941582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:58.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e99181640 1 --2- 172.21.15.105:0/1357883781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e9415d3c0 0x7f0e941582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:58.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e99181640 1 -- 172.21.15.105:0/1357883781 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c009a70 con 0x7f0e94071a10 2023-11-28T18:26:58.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e99181640 1 --2- 172.21.15.105:0/1357883781 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e94071a10 0x7f0e94167850 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400e960 tx=0x7f0e8400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:58.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e8a7fc640 1 -- 172.21.15.105:0/1357883781 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e84009690 con 0x7f0e94071a10 2023-11-28T18:26:58.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e8a7fc640 1 -- 172.21.15.105:0/1357883781 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8400ca00 con 0x7f0e94071a10 2023-11-28T18:26:58.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e8a7fc640 1 -- 172.21.15.105:0/1357883781 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8400cd70 con 0x7f0e94071a10 2023-11-28T18:26:58.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e9bbe6640 1 -- 172.21.15.105:0/1357883781 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e94157530 con 0x7f0e94071a10 2023-11-28T18:26:58.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e9bbe6640 1 -- 172.21.15.105:0/1357883781 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e94157af0 con 0x7f0e94071a10 2023-11-28T18:26:58.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e9bbe6640 1 -- 172.21.15.105:0/1357883781 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e94158010 con 0x7f0e94071a10 2023-11-28T18:26:58.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.243+0000 7f0e8a7fc640 1 -- 172.21.15.105:0/1357883781 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f0e84013470 con 0x7f0e94071a10 2023-11-28T18:26:58.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.247+0000 7f0e8a7fc640 1 --2- 172.21.15.105:0/1357883781 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0e78066900 0x7f0e78068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:58.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.247+0000 7f0e8a7fc640 1 -- 172.21.15.105:0/1357883781 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f0e84084610 con 0x7f0e94071a10 2023-11-28T18:26:58.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.247+0000 7f0e99982640 1 --2- 172.21.15.105:0/1357883781 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0e78066900 0x7f0e78068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:58.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.247+0000 7f0e99982640 1 --2- 172.21.15.105:0/1357883781 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0e78066900 0x7f0e78068dc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0e900011f0 tx=0x7f0e90009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:58.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.251+0000 7f0e8a7fc640 1 -- 172.21.15.105:0/1357883781 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0e94158010 con 0x7f0e94071a10 2023-11-28T18:26:58.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:58.455+0000 7f0e9bbe6640 1 -- 172.21.15.105:0/1357883781 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_off"]} v 0) v1 -- 0x7f0e94157fd0 con 0x7f0e94071a10 2023-11-28T18:26:59.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.423+0000 7f0e8a7fc640 1 -- 172.21.15.105:0/1357883781 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_off"]}]=0 v203) v1 ==== 153+0+0 (secure 0 0 0) 0x7f0e84051860 con 0x7f0e94071a10 2023-11-28T18:26:59.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.423+0000 7f0e9bbe6640 1 -- 172.21.15.105:0/1357883781 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0e78066900 msgr2=0x7f0e78068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:59.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.423+0000 7f0e9bbe6640 1 --2- 172.21.15.105:0/1357883781 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0e78066900 0x7f0e78068dc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0e900011f0 tx=0x7f0e90009040 comp rx=0 tx=0).stop 2023-11-28T18:26:59.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.423+0000 7f0e9bbe6640 1 -- 172.21.15.105:0/1357883781 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e94071a10 msgr2=0x7f0e94167850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:59.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.423+0000 7f0e9bbe6640 1 --2- 172.21.15.105:0/1357883781 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e94071a10 0x7f0e94167850 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400e960 tx=0x7f0e8400ee20 comp rx=0 tx=0).stop 2023-11-28T18:26:59.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.423+0000 7f0e9bbe6640 1 -- 172.21.15.105:0/1357883781 shutdown_connections 2023-11-28T18:26:59.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.423+0000 7f0e9bbe6640 1 --2- 172.21.15.105:0/1357883781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e9415d3c0 0x7f0e941582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:59.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.423+0000 7f0e9bbe6640 1 --2- 172.21.15.105:0/1357883781 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0e78066900 0x7f0e78068dc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:59.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.423+0000 7f0e9bbe6640 1 --2- 172.21.15.105:0/1357883781 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e94071a10 0x7f0e94167850 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:59.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.423+0000 7f0e9bbe6640 1 --2- 172.21.15.105:0/1357883781 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e94070a10 0x7f0e9415fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:59.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.423+0000 7f0e9bbe6640 1 -- 172.21.15.105:0/1357883781 >> 172.21.15.105:0/1357883781 conn(0x7f0e940cf540 msgr2=0x7f0e940af9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:59.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.423+0000 7f0e9bbe6640 1 -- 172.21.15.105:0/1357883781 shutdown_connections 2023-11-28T18:26:59.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.423+0000 7f0e9bbe6640 1 -- 172.21.15.105:0/1357883781 wait complete. 2023-11-28T18:26:59.449 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T18:26:59.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.587+0000 7faff459d640 1 Processor -- start 2023-11-28T18:26:59.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.587+0000 7faff459d640 1 -- start start 2023-11-28T18:26:59.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.587+0000 7faff459d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafec159f40 0x7fafec0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:59.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.587+0000 7faff459d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafec06bf50 con 0x7fafec0ad850 2023-11-28T18:26:59.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.587+0000 7faff459d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafec0ccbc0 con 0x7fafec0cc760 2023-11-28T18:26:59.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.587+0000 7faff459d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafec15a320 con 0x7fafec159f40 2023-11-28T18:26:59.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.587+0000 7faff2b3a640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fafec0ad850 0x7fafec0701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:53582/0 (socket says 172.21.15.105:53582) 2023-11-28T18:26:59.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.587+0000 7faff2b3a640 1 -- 172.21.15.105:0/2497557396 learned_addr learned my addr 172.21.15.105:0/2497557396 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:26:59.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.587+0000 7faff2339640 1 --2- 172.21.15.105:0/2497557396 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafec159f40 0x7fafec0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:59.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.587+0000 7faff1337640 1 -- 172.21.15.105:0/2497557396 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113377961 0 0) 0x7fafec06bf50 con 0x7fafec0ad850 2023-11-28T18:26:59.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.587+0000 7faff1337640 1 -- 172.21.15.105:0/2497557396 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafd8003530 con 0x7fafec0ad850 2023-11-28T18:26:59.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.587+0000 7faff1337640 1 -- 172.21.15.105:0/2497557396 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176128549 0 0) 0x7fafec0ccbc0 con 0x7fafec0cc760 2023-11-28T18:26:59.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.587+0000 7faff1337640 1 -- 172.21.15.105:0/2497557396 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafec06bf50 con 0x7fafec0cc760 2023-11-28T18:26:59.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.587+0000 7faff1337640 1 -- 172.21.15.105:0/2497557396 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3671043497 0 0) 0x7fafd8003530 con 0x7fafec0ad850 2023-11-28T18:26:59.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.587+0000 7faff1337640 1 -- 172.21.15.105:0/2497557396 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafec159f40 msgr2=0x7fafec0caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.587+0000 7faff1337640 1 --2- 172.21.15.105:0/2497557396 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafec159f40 0x7fafec0caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.587+0000 7faff1337640 1 -- 172.21.15.105:0/2497557396 >> v1:172.21.15.148:6789/0 conn(0x7fafec0cc760 legacy=0x7fafec0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.587+0000 7faff1337640 1 -- 172.21.15.105:0/2497557396 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec071380 con 0x7fafec0ad850 2023-11-28T18:26:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.587+0000 7faff1337640 1 -- 172.21.15.105:0/2497557396 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fafe4001a00 con 0x7fafec0ad850 2023-11-28T18:26:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.587+0000 7faff2339640 1 --2- 172.21.15.105:0/2497557396 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafec159f40 0x7fafec0caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:26:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.587+0000 7faff1337640 1 -- 172.21.15.105:0/2497557396 >> v1:172.21.15.105:6789/0 conn(0x7fafec0ad850 legacy=0x7fafec0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.587+0000 7faff1337640 1 --2- 172.21.15.105:0/2497557396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafd8003b40 0x7fafd8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.591+0000 7faff1337640 1 --2- 172.21.15.105:0/2497557396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafd80047d0 0x7fafd8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.591+0000 7faff1337640 1 --2- 172.21.15.105:0/2497557396 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafec159f40 0x7fafd8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.591+0000 7faff1337640 1 -- 172.21.15.105:0/2497557396 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec15a320 con 0x7fafd80047d0 2023-11-28T18:26:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.591+0000 7faff1337640 1 -- 172.21.15.105:0/2497557396 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd800dc80 con 0x7fafd8003b40 2023-11-28T18:26:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.591+0000 7faff1337640 1 -- 172.21.15.105:0/2497557396 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd800de40 con 0x7fafec159f40 2023-11-28T18:26:59.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.591+0000 7faff1b38640 1 --2- 172.21.15.105:0/2497557396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafd80047d0 0x7fafd8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:59.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.591+0000 7faff2339640 1 --2- 172.21.15.105:0/2497557396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafd8003b40 0x7fafd8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:59.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.591+0000 7faff2b3a640 1 --2- 172.21.15.105:0/2497557396 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafec159f40 0x7fafd8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:59.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.591+0000 7faff1b38640 1 -- 172.21.15.105:0/2497557396 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafec159f40 msgr2=0x7fafd8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:59.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.591+0000 7faff1b38640 1 --2- 172.21.15.105:0/2497557396 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafec159f40 0x7fafd8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:59.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.591+0000 7faff1b38640 1 -- 172.21.15.105:0/2497557396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafd8003b40 msgr2=0x7fafd8003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:26:59.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.591+0000 7faff1b38640 1 --2- 172.21.15.105:0/2497557396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafd8003b40 0x7fafd8003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:59.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.591+0000 7faff1b38640 1 -- 172.21.15.105:0/2497557396 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe8009420 con 0x7fafd80047d0 2023-11-28T18:26:59.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.591+0000 7faff1b38640 1 --2- 172.21.15.105:0/2497557396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafd80047d0 0x7fafd8004c50 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fafdc007fd0 tx=0x7fafdc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43adacaeef9281f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:59.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.591+0000 7faff1337640 1 -- 172.21.15.105:0/2497557396 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc0052d0 con 0x7fafd80047d0 2023-11-28T18:26:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.591+0000 7faff1337640 1 -- 172.21.15.105:0/2497557396 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafdc0a7050 con 0x7fafd80047d0 2023-11-28T18:26:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.591+0000 7faff1337640 1 -- 172.21.15.105:0/2497557396 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc003600 con 0x7fafd80047d0 2023-11-28T18:26:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff459d640 1 -- 172.21.15.105:0/2497557396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafd80047d0 msgr2=0x7fafd8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff459d640 1 --2- 172.21.15.105:0/2497557396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafd80047d0 0x7fafd8004c50 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fafdc007fd0 tx=0x7fafdc01c4f0 comp rx=0 tx=0).stop 2023-11-28T18:26:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff2b3a640 1 -- 172.21.15.105:0/2497557396 reap_dead start 2023-11-28T18:26:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff459d640 1 -- 172.21.15.105:0/2497557396 shutdown_connections 2023-11-28T18:26:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff459d640 1 -- 172.21.15.105:0/2497557396 >> 172.21.15.105:0/2497557396 conn(0x7fafec0ceb90 msgr2=0x7fafec0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff459d640 1 -- 172.21.15.105:0/2497557396 shutdown_connections 2023-11-28T18:26:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff459d640 1 -- 172.21.15.105:0/2497557396 wait complete. 2023-11-28T18:26:59.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff459d640 1 Processor -- start 2023-11-28T18:26:59.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff459d640 1 -- start start 2023-11-28T18:26:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff459d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafd80047d0 0x7fafec0b4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff459d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafec143450 0x7fafec159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff459d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafec0b1ad0 0x7fafec0b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:59.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff459d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec0b2490 con 0x7fafd80047d0 2023-11-28T18:26:59.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff459d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec1443d0 con 0x7fafec143450 2023-11-28T18:26:59.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff459d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafec1438d0 con 0x7fafec0b1ad0 2023-11-28T18:26:59.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff2339640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafd80047d0 0x7fafec0b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:59.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff2339640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafd80047d0 0x7fafec0b4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50904/0 (socket says 172.21.15.105:50904) 2023-11-28T18:26:59.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff2339640 1 -- 172.21.15.105:0/2849052985 learned_addr learned my addr 172.21.15.105:0/2849052985 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:26:59.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff1b38640 1 --2- 172.21.15.105:0/2849052985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafec143450 0x7fafec159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:59.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff2339640 1 -- 172.21.15.105:0/2849052985 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafec0b1ad0 msgr2=0x7fafec0b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:59.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff2339640 1 --2- 172.21.15.105:0/2849052985 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafec0b1ad0 0x7fafec0b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:59.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff2339640 1 -- 172.21.15.105:0/2849052985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafec143450 msgr2=0x7fafec159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:59.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff2339640 1 --2- 172.21.15.105:0/2849052985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafec143450 0x7fafec159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:59.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff2339640 1 -- 172.21.15.105:0/2849052985 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec143b30 con 0x7fafd80047d0 2023-11-28T18:26:59.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.595+0000 7faff1b38640 1 --2- 172.21.15.105:0/2849052985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafec143450 0x7fafec159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:26:59.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.599+0000 7faff2339640 1 --2- 172.21.15.105:0/2849052985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafd80047d0 0x7fafec0b4640 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fafe8011c50 tx=0x7fafe800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:59.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.599+0000 7fafe2ffd640 1 -- 172.21.15.105:0/2849052985 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe800fcc0 con 0x7fafd80047d0 2023-11-28T18:26:59.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.599+0000 7fafe2ffd640 1 -- 172.21.15.105:0/2849052985 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe800fe20 con 0x7fafd80047d0 2023-11-28T18:26:59.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.599+0000 7fafe2ffd640 1 -- 172.21.15.105:0/2849052985 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe8004240 con 0x7fafd80047d0 2023-11-28T18:26:59.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.599+0000 7faff459d640 1 -- 172.21.15.105:0/2849052985 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafec143e10 con 0x7fafd80047d0 2023-11-28T18:26:59.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.599+0000 7faff459d640 1 -- 172.21.15.105:0/2849052985 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafec15b160 con 0x7fafd80047d0 2023-11-28T18:26:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.599+0000 7faff459d640 1 -- 172.21.15.105:0/2849052985 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafec067570 con 0x7fafd80047d0 2023-11-28T18:26:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.599+0000 7fafe2ffd640 1 -- 172.21.15.105:0/2849052985 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fafe80043a0 con 0x7fafd80047d0 2023-11-28T18:26:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.599+0000 7fafe2ffd640 1 --2- 172.21.15.105:0/2849052985 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fafd00668d0 0x7fafd0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:26:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.599+0000 7fafe2ffd640 1 -- 172.21.15.105:0/2849052985 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5521+0+0 (secure 0 0 0) 0x7fafe8084950 con 0x7fafd80047d0 2023-11-28T18:26:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.599+0000 7faff1b38640 1 --2- 172.21.15.105:0/2849052985 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fafd00668d0 0x7fafd0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:26:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.603+0000 7faff1b38640 1 --2- 172.21.15.105:0/2849052985 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fafd00668d0 0x7fafd0068d90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fafdc0ac000 tx=0x7fafdc0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:26:59.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.603+0000 7fafe2ffd640 1 -- 172.21.15.105:0/2849052985 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fafe8051c30 con 0x7fafd80047d0 2023-11-28T18:26:59.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.755+0000 7faff459d640 1 -- 172.21.15.105:0/2849052985 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fafec0cf950 con 0x7fafd80047d0 2023-11-28T18:26:59.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.759+0000 7fafe2ffd640 1 -- 172.21.15.105:0/2849052985 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v27)=0 v27) v1 ==== 122+0+0 (secure 0 0 0) 0x7fafe8022070 con 0x7fafd80047d0 2023-11-28T18:26:59.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.763+0000 7faff459d640 1 -- 172.21.15.105:0/2849052985 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fafd00668d0 msgr2=0x7fafd0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:59.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.763+0000 7faff459d640 1 --2- 172.21.15.105:0/2849052985 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fafd00668d0 0x7fafd0068d90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fafdc0ac000 tx=0x7fafdc0a6000 comp rx=0 tx=0).stop 2023-11-28T18:26:59.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.763+0000 7faff459d640 1 -- 172.21.15.105:0/2849052985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafd80047d0 msgr2=0x7fafec0b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:26:59.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.763+0000 7faff459d640 1 --2- 172.21.15.105:0/2849052985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafd80047d0 0x7fafec0b4640 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fafe8011c50 tx=0x7fafe800f610 comp rx=0 tx=0).stop 2023-11-28T18:26:59.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.763+0000 7faff459d640 1 -- 172.21.15.105:0/2849052985 shutdown_connections 2023-11-28T18:26:59.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.763+0000 7faff459d640 1 --2- 172.21.15.105:0/2849052985 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafec0b1ad0 0x7fafec0b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:59.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.763+0000 7faff459d640 1 --2- 172.21.15.105:0/2849052985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafec143450 0x7fafec159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:59.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.763+0000 7faff459d640 1 --2- 172.21.15.105:0/2849052985 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fafd00668d0 0x7fafd0068d90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:59.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.763+0000 7faff459d640 1 --2- 172.21.15.105:0/2849052985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafd80047d0 0x7fafec0b4640 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:26:59.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.763+0000 7faff459d640 1 -- 172.21.15.105:0/2849052985 >> 172.21.15.105:0/2849052985 conn(0x7fafec0ceb90 msgr2=0x7fafec0cdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:26:59.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.763+0000 7faff459d640 1 -- 172.21.15.105:0/2849052985 shutdown_connections 2023-11-28T18:26:59.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:26:59.763+0000 7faff459d640 1 -- 172.21.15.105:0/2849052985 wait complete. 2023-11-28T18:26:59.786 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T18:26:59.786 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:26:59.879 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:26:59.961 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T18:26:59.961 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:26:59.961 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:26:59.970 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:26:59.971 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:26:59.971 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:26:59.980 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:26:59.980 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:26:59.981 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:27:00.026 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:27:00.027 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:27:00.027 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:27:00.037 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:27:00.037 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T18:27:00.037 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:27:00.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.203+0000 7f82eff72640 1 Processor -- start 2023-11-28T18:27:00.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.203+0000 7f82eff72640 1 -- start start 2023-11-28T18:27:00.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.203+0000 7f82eff72640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e815d300 0x7f82e80d37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.203+0000 7f82eff72640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f82e80d3ce0 0x7f82e80d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.203+0000 7f82eff72640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e80d0c60 con 0x7f82e80d1b90 2023-11-28T18:27:00.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.203+0000 7f82eff72640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e80d1470 con 0x7f82e80d3ce0 2023-11-28T18:27:00.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.203+0000 7f82eff72640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e80d2040 con 0x7f82e815d300 2023-11-28T18:27:00.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.203+0000 7f82ed50d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f82e80d3ce0 0x7f82e80d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:00.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.203+0000 7f82ed50d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f82e80d3ce0 0x7f82e80d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50914/0 (socket says 172.21.15.105:50914) 2023-11-28T18:27:00.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.203+0000 7f82ed50d640 1 -- 172.21.15.105:0/1201948128 learned_addr learned my addr 172.21.15.105:0/1201948128 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:00.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.203+0000 7f82edd0e640 1 --2- 172.21.15.105:0/1201948128 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e815d300 0x7f82e80d37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:00.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.207+0000 7f82ed50d640 1 -- 172.21.15.105:0/1201948128 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e815d300 msgr2=0x7f82e80d37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:00.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.207+0000 7f82ed50d640 1 --2- 172.21.15.105:0/1201948128 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e815d300 0x7f82e80d37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:00.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.207+0000 7f82ed50d640 1 -- 172.21.15.105:0/1201948128 >> v1:172.21.15.148:6789/0 conn(0x7f82e80d1b90 legacy=0x7f82e80cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:00.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.207+0000 7f82ed50d640 1 -- 172.21.15.105:0/1201948128 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e80d0eb0 con 0x7f82e80d3ce0 2023-11-28T18:27:00.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.207+0000 7f82edd0e640 1 --2- 172.21.15.105:0/1201948128 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e815d300 0x7f82e80d37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.207+0000 7f82ed50d640 1 --2- 172.21.15.105:0/1201948128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82e80d3ce0 0x7f82e80d1650 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f82d8009870 tx=0x7f82d801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd63502190b95f49 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.207+0000 7f82ecd0c640 1 -- 172.21.15.105:0/1201948128 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d80ac600 con 0x7f82e80d3ce0 2023-11-28T18:27:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.207+0000 7f82ecd0c640 1 -- 172.21.15.105:0/1201948128 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d80a8050 con 0x7f82e80d3ce0 2023-11-28T18:27:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.207+0000 7f82ecd0c640 1 -- 172.21.15.105:0/1201948128 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d80ac980 con 0x7f82e80d3ce0 2023-11-28T18:27:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.207+0000 7f82eff72640 1 -- 172.21.15.105:0/1201948128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82e80d3ce0 msgr2=0x7f82e80d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.207+0000 7f82eff72640 1 --2- 172.21.15.105:0/1201948128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82e80d3ce0 0x7f82e80d1650 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f82d8009870 tx=0x7f82d801c4b0 comp rx=0 tx=0).stop 2023-11-28T18:27:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.207+0000 7f82eff72640 1 -- 172.21.15.105:0/1201948128 shutdown_connections 2023-11-28T18:27:00.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.207+0000 7f82eff72640 1 --2- 172.21.15.105:0/1201948128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82e80d3ce0 0x7f82e80d1650 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:00.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.207+0000 7f82eff72640 1 --2- 172.21.15.105:0/1201948128 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e815d300 0x7f82e80d37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:00.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.207+0000 7f82eff72640 1 -- 172.21.15.105:0/1201948128 >> 172.21.15.105:0/1201948128 conn(0x7f82e809c360 msgr2=0x7f82e80d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:00.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82eff72640 1 -- 172.21.15.105:0/1201948128 shutdown_connections 2023-11-28T18:27:00.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82eff72640 1 -- 172.21.15.105:0/1201948128 wait complete. 2023-11-28T18:27:00.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82eff72640 1 Processor -- start 2023-11-28T18:27:00.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82eff72640 1 -- start start 2023-11-28T18:27:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82eff72640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82e80d1b90 0x7f82e815a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82eff72640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e80d3ce0 0x7f82e815a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82eff72640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82e815d300 0x7f82e8152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82eff72640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e815aee0 con 0x7f82e80d1b90 2023-11-28T18:27:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82eff72640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e815b110 con 0x7f82e815d300 2023-11-28T18:27:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82eff72640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e815b3f0 con 0x7f82e80d3ce0 2023-11-28T18:27:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82edd0e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82e80d1b90 0x7f82e815a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82edd0e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82e80d1b90 0x7f82e815a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50928/0 (socket says 172.21.15.105:50928) 2023-11-28T18:27:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82edd0e640 1 -- 172.21.15.105:0/4167584003 learned_addr learned my addr 172.21.15.105:0/4167584003 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82ee50f640 1 --2- 172.21.15.105:0/4167584003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82e815d300 0x7f82e8152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82ed50d640 1 --2- 172.21.15.105:0/4167584003 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e80d3ce0 0x7f82e815a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82edd0e640 1 -- 172.21.15.105:0/4167584003 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e80d3ce0 msgr2=0x7f82e815a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82edd0e640 1 --2- 172.21.15.105:0/4167584003 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e80d3ce0 0x7f82e815a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82edd0e640 1 -- 172.21.15.105:0/4167584003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82e815d300 msgr2=0x7f82e8152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:00.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82edd0e640 1 --2- 172.21.15.105:0/4167584003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82e815d300 0x7f82e8152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:00.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82edd0e640 1 -- 172.21.15.105:0/4167584003 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d80094d0 con 0x7f82e80d1b90 2023-11-28T18:27:00.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82ee50f640 1 --2- 172.21.15.105:0/4167584003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82e815d300 0x7f82e8152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:00.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82ed50d640 1 --2- 172.21.15.105:0/4167584003 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e80d3ce0 0x7f82e815a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:00.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.211+0000 7f82edd0e640 1 --2- 172.21.15.105:0/4167584003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82e80d1b90 0x7f82e815a000 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f82e400e810 tx=0x7f82e400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:00.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.215+0000 7f82de7fc640 1 -- 172.21.15.105:0/4167584003 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e400c8d0 con 0x7f82e80d1b90 2023-11-28T18:27:00.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.215+0000 7f82de7fc640 1 -- 172.21.15.105:0/4167584003 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e400ca30 con 0x7f82e80d1b90 2023-11-28T18:27:00.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.215+0000 7f82de7fc640 1 -- 172.21.15.105:0/4167584003 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e400cdc0 con 0x7f82e80d1b90 2023-11-28T18:27:00.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.215+0000 7f82eff72640 1 -- 172.21.15.105:0/4167584003 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e815b6d0 con 0x7f82e80d1b90 2023-11-28T18:27:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.215+0000 7f82eff72640 1 -- 172.21.15.105:0/4167584003 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e815bc90 con 0x7f82e80d1b90 2023-11-28T18:27:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.215+0000 7f82eff72640 1 -- 172.21.15.105:0/4167584003 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e8067570 con 0x7f82e80d1b90 2023-11-28T18:27:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.215+0000 7f82de7fc640 1 -- 172.21.15.105:0/4167584003 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f82e4013410 con 0x7f82e80d1b90 2023-11-28T18:27:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.215+0000 7f82de7fc640 1 --2- 172.21.15.105:0/4167584003 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82cc0668d0 0x7f82cc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.215+0000 7f82de7fc640 1 -- 172.21.15.105:0/4167584003 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f82e4084230 con 0x7f82e80d1b90 2023-11-28T18:27:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.215+0000 7f82ed50d640 1 --2- 172.21.15.105:0/4167584003 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82cc0668d0 0x7f82cc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.219+0000 7f82ed50d640 1 --2- 172.21.15.105:0/4167584003 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82cc0668d0 0x7f82cc068d90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f82d80ae000 tx=0x7f82d80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.219+0000 7f82de7fc640 1 -- 172.21.15.105:0/4167584003 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f82e4051480 con 0x7f82e80d1b90 2023-11-28T18:27:00.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.403+0000 7f82eff72640 1 -- 172.21.15.105:0/4167584003 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82e80c6510 con 0x7f82e80d1b90 2023-11-28T18:27:00.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.403+0000 7f82de7fc640 1 -- 172.21.15.105:0/4167584003 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f82e4021e90 con 0x7f82e80d1b90 2023-11-28T18:27:00.407 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:27:00.407 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1686803161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":1686803161},{"type":"v1","addr":"172.21.15.148:6835","nonce":1686803161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":5804,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/208471852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":208471852},{"type":"v1","addr":"172.21.15.148:6834","nonce":208471852}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":6612,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/254096763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":254096763},{"type":"v1","addr":"172.21.15.105:6835","nonce":254096763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":6615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/3578801380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3578801380},{"type":"v1","addr":"172.21.15.105:6837","nonce":3578801380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[]} 2023-11-28T18:27:00.408 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 49 2023-11-28T18:27:00.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.403+0000 7f82eff72640 1 -- 172.21.15.105:0/4167584003 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82cc0668d0 msgr2=0x7f82cc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:00.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.403+0000 7f82eff72640 1 --2- 172.21.15.105:0/4167584003 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82cc0668d0 0x7f82cc068d90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f82d80ae000 tx=0x7f82d80a7000 comp rx=0 tx=0).stop 2023-11-28T18:27:00.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.407+0000 7f82eff72640 1 -- 172.21.15.105:0/4167584003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82e80d1b90 msgr2=0x7f82e815a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:00.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.407+0000 7f82eff72640 1 --2- 172.21.15.105:0/4167584003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82e80d1b90 0x7f82e815a000 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f82e400e810 tx=0x7f82e400ecd0 comp rx=0 tx=0).stop 2023-11-28T18:27:00.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.407+0000 7f82eff72640 1 -- 172.21.15.105:0/4167584003 shutdown_connections 2023-11-28T18:27:00.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.407+0000 7f82eff72640 1 --2- 172.21.15.105:0/4167584003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82e815d300 0x7f82e8152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:00.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.407+0000 7f82eff72640 1 --2- 172.21.15.105:0/4167584003 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82e80d3ce0 0x7f82e815a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:00.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.407+0000 7f82eff72640 1 --2- 172.21.15.105:0/4167584003 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82cc0668d0 0x7f82cc068d90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:00.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.407+0000 7f82eff72640 1 --2- 172.21.15.105:0/4167584003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82e80d1b90 0x7f82e815a000 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:00.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.407+0000 7f82eff72640 1 -- 172.21.15.105:0/4167584003 >> 172.21.15.105:0/4167584003 conn(0x7f82e809c360 msgr2=0x7f82e81560a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:00.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.407+0000 7f82eff72640 1 -- 172.21.15.105:0/4167584003 shutdown_connections 2023-11-28T18:27:00.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.407+0000 7f82eff72640 1 -- 172.21.15.105:0/4167584003 wait complete. 2023-11-28T18:27:00.429 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T18:27:00.430 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T18:27:00.430 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T18:27:00.430 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:27:00.430 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T18:27:00.430 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T18:27:00.431 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T18:27:00.431 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:27:00.431 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T18:27:00.431 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T18:27:00.432 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T18:27:00.432 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:27:00.432 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T18:27:00.432 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T18:27:00.432 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T18:27:00.432 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:27:00.445 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T18:27:00.446 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T18:27:00.447 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T18:27:00.447 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T18:27:00.449 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T18:27:00.449 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T18:27:00.450 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T18:27:00.451 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T18:27:00.491 INFO:tasks.ceph.mds.a:Started 2023-11-28T18:27:00.492 DEBUG:teuthology.parallel:result is None 2023-11-28T18:27:00.494 INFO:tasks.ceph.mds.c:Started 2023-11-28T18:27:00.495 DEBUG:teuthology.parallel:result is None 2023-11-28T18:27:00.496 INFO:tasks.ceph.mds.b:Started 2023-11-28T18:27:00.497 DEBUG:teuthology.parallel:result is None 2023-11-28T18:27:00.498 INFO:tasks.ceph.mds.d:Started 2023-11-28T18:27:00.499 DEBUG:teuthology.parallel:result is None 2023-11-28T18:27:00.499 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T18:27:00.580 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2023-11-28T18:27:00.584 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2023-11-28T18:27:00.590 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2023-11-28T18:27:00.590 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2023-11-28T18:27:00.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.643+0000 7fa52390f640 1 Processor -- start 2023-11-28T18:27:00.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.643+0000 7fa52390f640 1 -- start start 2023-11-28T18:27:00.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.647+0000 7fa52390f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa51c154070 0x7fa51c15d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:00.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.647+0000 7fa52390f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa51c06bf50 0x7fa51c0ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:00.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.647+0000 7fa52390f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa51c0cc7f0 con 0x7fa51c0adb80 2023-11-28T18:27:00.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.647+0000 7fa52390f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa51c072540 con 0x7fa51c06bf50 2023-11-28T18:27:00.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.647+0000 7fa52390f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c15dd10 con 0x7fa51c154070 2023-11-28T18:27:00.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.647+0000 7fa5216ab640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa51c154070 0x7fa51c15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:00.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.647+0000 7fa520eaa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa51c06bf50 0x7fa51c0ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:00.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.647+0000 7fa521eac640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa51c0adb80 0x7fa51c0acd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:52460/0 (socket says 172.21.15.105:52460) 2023-11-28T18:27:00.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.647+0000 7fa521eac640 1 -- 172.21.15.105:0/2307246367 learned_addr learned my addr 172.21.15.105:0/2307246367 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:00.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.647+0000 7fa513fff640 1 -- 172.21.15.105:0/2307246367 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592085565 0 0) 0x7fa51c0cc7f0 con 0x7fa51c0adb80 2023-11-28T18:27:00.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.647+0000 7fa513fff640 1 -- 172.21.15.105:0/2307246367 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa508003530 con 0x7fa51c0adb80 2023-11-28T18:27:00.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.647+0000 7fa5216ab640 1 -- 172.21.15.105:0/2307246367 >> v2:172.21.15.148:3300/0 conn(0x7fa51c06bf50 msgr2=0x7fa51c0ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:00.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.647+0000 7fa5216ab640 1 --2- 172.21.15.105:0/2307246367 >> v2:172.21.15.148:3300/0 conn(0x7fa51c06bf50 0x7fa51c0ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:00.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.647+0000 7fa5216ab640 1 -- 172.21.15.105:0/2307246367 >> v1:172.21.15.148:6789/0 conn(0x7fa51c0adb80 legacy=0x7fa51c0acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:00.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.647+0000 7fa5216ab640 1 -- 172.21.15.105:0/2307246367 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c15dff0 con 0x7fa51c154070 2023-11-28T18:27:00.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.647+0000 7fa520eaa640 1 --2- 172.21.15.105:0/2307246367 >> v2:172.21.15.148:3300/0 conn(0x7fa51c06bf50 0x7fa51c0ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:27:00.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.647+0000 7fa5216ab640 1 --2- 172.21.15.105:0/2307246367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa51c154070 0x7fa51c15d700 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa5180097c0 tx=0x7fa51801c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83bbd15f30707521 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:00.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa513fff640 1 -- 172.21.15.105:0/2307246367 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5180ad640 con 0x7fa51c154070 2023-11-28T18:27:00.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa513fff640 1 -- 172.21.15.105:0/2307246367 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5180ad7a0 con 0x7fa51c154070 2023-11-28T18:27:00.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa513fff640 1 -- 172.21.15.105:0/2307246367 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5180adb30 con 0x7fa51c154070 2023-11-28T18:27:00.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa52390f640 1 -- 172.21.15.105:0/2307246367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa51c154070 msgr2=0x7fa51c15d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:00.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa52390f640 1 --2- 172.21.15.105:0/2307246367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa51c154070 0x7fa51c15d700 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa5180097c0 tx=0x7fa51801c390 comp rx=0 tx=0).stop 2023-11-28T18:27:00.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa52390f640 1 -- 172.21.15.105:0/2307246367 shutdown_connections 2023-11-28T18:27:00.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa52390f640 1 --2- 172.21.15.105:0/2307246367 >> v2:172.21.15.148:3300/0 conn(0x7fa51c06bf50 0x7fa51c0ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:00.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa52390f640 1 --2- 172.21.15.105:0/2307246367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa51c154070 0x7fa51c15d700 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:00.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa52390f640 1 -- 172.21.15.105:0/2307246367 >> 172.21.15.105:0/2307246367 conn(0x7fa51c0ceb90 msgr2=0x7fa51c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:00.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa52390f640 1 -- 172.21.15.105:0/2307246367 shutdown_connections 2023-11-28T18:27:00.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa52390f640 1 -- 172.21.15.105:0/2307246367 wait complete. 2023-11-28T18:27:00.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa52390f640 1 Processor -- start 2023-11-28T18:27:00.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa52390f640 1 -- start start 2023-11-28T18:27:00.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa52390f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa51c06bf50 0x7fa51c147a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:00.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa52390f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa51c0adb80 0x7fa51c1501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:00.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa52390f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa51c154070 0x7fa51c09c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:00.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa52390f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c14fdd0 con 0x7fa51c154070 2023-11-28T18:27:00.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa52390f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c14f150 con 0x7fa51c06bf50 2023-11-28T18:27:00.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa52390f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c14f430 con 0x7fa51c0adb80 2023-11-28T18:27:00.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa5216ab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa51c06bf50 0x7fa51c147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:00.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa521eac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa51c154070 0x7fa51c09c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:00.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa5216ab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa51c06bf50 0x7fa51c147a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:57346/0 (socket says 172.21.15.105:57346) 2023-11-28T18:27:00.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa5216ab640 1 -- 172.21.15.105:0/4261349215 learned_addr learned my addr 172.21.15.105:0/4261349215 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:00.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.651+0000 7fa520eaa640 1 --2- 172.21.15.105:0/4261349215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa51c0adb80 0x7fa51c1501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:00.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.655+0000 7fa520eaa640 1 -- 172.21.15.105:0/4261349215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa51c06bf50 msgr2=0x7fa51c147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:00.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.655+0000 7fa520eaa640 1 --2- 172.21.15.105:0/4261349215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa51c06bf50 0x7fa51c147a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:00.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.655+0000 7fa520eaa640 1 -- 172.21.15.105:0/4261349215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa51c154070 msgr2=0x7fa51c09c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:00.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.655+0000 7fa520eaa640 1 --2- 172.21.15.105:0/4261349215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa51c154070 0x7fa51c09c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:00.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.655+0000 7fa520eaa640 1 -- 172.21.15.105:0/4261349215 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50c009420 con 0x7fa51c0adb80 2023-11-28T18:27:00.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.655+0000 7fa5216ab640 1 --2- 172.21.15.105:0/4261349215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa51c06bf50 0x7fa51c147a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:27:00.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.655+0000 7fa521eac640 1 --2- 172.21.15.105:0/4261349215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa51c154070 0x7fa51c09c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:27:00.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.655+0000 7fa520eaa640 1 --2- 172.21.15.105:0/4261349215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa51c0adb80 0x7fa51c1501f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00e980 tx=0x7fa50c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:00.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.655+0000 7fa511ffb640 1 -- 172.21.15.105:0/4261349215 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c00ca00 con 0x7fa51c0adb80 2023-11-28T18:27:00.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.655+0000 7fa511ffb640 1 -- 172.21.15.105:0/4261349215 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50c00cb80 con 0x7fa51c0adb80 2023-11-28T18:27:00.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.655+0000 7fa511ffb640 1 -- 172.21.15.105:0/4261349215 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c013470 con 0x7fa51c0adb80 2023-11-28T18:27:00.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.655+0000 7fa52390f640 1 -- 172.21.15.105:0/4261349215 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa518009420 con 0x7fa51c0adb80 2023-11-28T18:27:00.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.655+0000 7fa52390f640 1 -- 172.21.15.105:0/4261349215 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51c0ae520 con 0x7fa51c0adb80 2023-11-28T18:27:00.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.655+0000 7fa52390f640 1 -- 172.21.15.105:0/4261349215 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa51c160af0 con 0x7fa51c0adb80 2023-11-28T18:27:00.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.655+0000 7fa511ffb640 1 -- 172.21.15.105:0/4261349215 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa50c004020 con 0x7fa51c0adb80 2023-11-28T18:27:00.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.659+0000 7fa511ffb640 1 --2- 172.21.15.105:0/4261349215 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa500066900 0x7fa500068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:00.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.659+0000 7fa511ffb640 1 -- 172.21.15.105:0/4261349215 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5521+0+0 (secure 0 0 0) 0x7fa50c084570 con 0x7fa51c0adb80 2023-11-28T18:27:00.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.659+0000 7fa5216ab640 1 --2- 172.21.15.105:0/4261349215 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa500066900 0x7fa500068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:00.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.659+0000 7fa5216ab640 1 --2- 172.21.15.105:0/4261349215 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa500066900 0x7fa500068dc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa51801c3d0 tx=0x7fa5180a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:00.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.663+0000 7fa511ffb640 1 -- 172.21.15.105:0/4261349215 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa50c051980 con 0x7fa51c0adb80 2023-11-28T18:27:00.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:00.819+0000 7fa52390f640 1 -- 172.21.15.105:0/4261349215 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa51c14ff10 con 0x7fa51c0adb80 2023-11-28T18:27:01.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.439+0000 7fa511ffb640 1 -- 172.21.15.105:0/4261349215 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v59) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa51c14ff10 con 0x7fa51c0adb80 2023-11-28T18:27:01.442 INFO:teuthology.orchestra.run.smithi105.stderr: removed all blocklist entries 2023-11-28T18:27:01.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.439+0000 7fa52390f640 1 -- 172.21.15.105:0/4261349215 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa500066900 msgr2=0x7fa500068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:01.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.439+0000 7fa52390f640 1 --2- 172.21.15.105:0/4261349215 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa500066900 0x7fa500068dc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa51801c3d0 tx=0x7fa5180a7070 comp rx=0 tx=0).stop 2023-11-28T18:27:01.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.439+0000 7fa52390f640 1 -- 172.21.15.105:0/4261349215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa51c0adb80 msgr2=0x7fa51c1501f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:01.446 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.439+0000 7fa52390f640 1 --2- 172.21.15.105:0/4261349215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa51c0adb80 0x7fa51c1501f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00e980 tx=0x7fa50c00ee40 comp rx=0 tx=0).stop 2023-11-28T18:27:01.446 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.439+0000 7fa52390f640 1 -- 172.21.15.105:0/4261349215 shutdown_connections 2023-11-28T18:27:01.446 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.439+0000 7fa52390f640 1 --2- 172.21.15.105:0/4261349215 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa500066900 0x7fa500068dc0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:01.446 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.443+0000 7fa52390f640 1 --2- 172.21.15.105:0/4261349215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa51c154070 0x7fa51c09c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:01.446 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.443+0000 7fa52390f640 1 --2- 172.21.15.105:0/4261349215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa51c0adb80 0x7fa51c1501f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:01.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.443+0000 7fa52390f640 1 --2- 172.21.15.105:0/4261349215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa51c06bf50 0x7fa51c147a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:01.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.443+0000 7fa52390f640 1 -- 172.21.15.105:0/4261349215 >> 172.21.15.105:0/4261349215 conn(0x7fa51c0ceb90 msgr2=0x7fa51c161510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:01.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.443+0000 7fa52390f640 1 -- 172.21.15.105:0/4261349215 shutdown_connections 2023-11-28T18:27:01.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.443+0000 7fa52390f640 1 -- 172.21.15.105:0/4261349215 wait complete. 2023-11-28T18:27:01.467 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T18:27:01.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.599+0000 7fa5318fe640 1 Processor -- start 2023-11-28T18:27:01.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.599+0000 7fa5318fe640 1 -- start start 2023-11-28T18:27:01.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.603+0000 7fa5318fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa52c0d00b0 0x7fa52c0cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:01.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.603+0000 7fa5318fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa52c0d3e60 con 0x7fa52c0ceb40 2023-11-28T18:27:01.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.603+0000 7fa5318fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa52c0b5fe0 con 0x7fa52c0b8a20 2023-11-28T18:27:01.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.603+0000 7fa5318fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c0b62c0 con 0x7fa52c0d00b0 2023-11-28T18:27:01.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.603+0000 7fa52b7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa52c0ceb40 0x7fa52c0b6da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51228/0 (socket says 172.21.15.105:51228) 2023-11-28T18:27:01.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.603+0000 7fa52b7fe640 1 -- 172.21.15.105:0/3651903009 learned_addr learned my addr 172.21.15.105:0/3651903009 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:01.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.603+0000 7fa5227fc640 1 --2- 172.21.15.105:0/3651903009 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa52c0d00b0 0x7fa52c0cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:01.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.603+0000 7fa52a7fc640 1 -- 172.21.15.105:0/3651903009 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3444536335 0 0) 0x7fa52c0d3e60 con 0x7fa52c0ceb40 2023-11-28T18:27:01.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.603+0000 7fa52a7fc640 1 -- 172.21.15.105:0/3651903009 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa518003530 con 0x7fa52c0ceb40 2023-11-28T18:27:01.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.603+0000 7fa5227fc640 1 -- 172.21.15.105:0/3651903009 >> v1:172.21.15.148:6789/0 conn(0x7fa52c0b8a20 legacy=0x7fa52c0cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:01.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.603+0000 7fa52a7fc640 1 -- 172.21.15.105:0/3651903009 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792446055 0 0) 0x7fa52c0b5fe0 con 0x7fa52c0b8a20 2023-11-28T18:27:01.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.603+0000 7fa5227fc640 1 -- 172.21.15.105:0/3651903009 >> v1:172.21.15.105:6789/0 conn(0x7fa52c0ceb40 legacy=0x7fa52c0b6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:01.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.603+0000 7fa5227fc640 1 -- 172.21.15.105:0/3651903009 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c0b65a0 con 0x7fa52c0d00b0 2023-11-28T18:27:01.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.607+0000 7fa5227fc640 1 --2- 172.21.15.105:0/3651903009 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa52c0d00b0 0x7fa52c0cdb50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0097f0 tx=0x7fa51c01c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7d72cae601351c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:01.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.607+0000 7fa52a7fc640 1 -- 172.21.15.105:0/3651903009 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0ab5f0 con 0x7fa52c0d00b0 2023-11-28T18:27:01.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.607+0000 7fa52a7fc640 1 -- 172.21.15.105:0/3651903009 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c0ab750 con 0x7fa52c0d00b0 2023-11-28T18:27:01.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.607+0000 7fa52a7fc640 1 -- 172.21.15.105:0/3651903009 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0abae0 con 0x7fa52c0d00b0 2023-11-28T18:27:01.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.607+0000 7fa5318fe640 1 -- 172.21.15.105:0/3651903009 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa52c0d00b0 msgr2=0x7fa52c0cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:01.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.607+0000 7fa5318fe640 1 --2- 172.21.15.105:0/3651903009 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa52c0d00b0 0x7fa52c0cdb50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0097f0 tx=0x7fa51c01c490 comp rx=0 tx=0).stop 2023-11-28T18:27:01.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.607+0000 7fa5318fe640 1 -- 172.21.15.105:0/3651903009 shutdown_connections 2023-11-28T18:27:01.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.607+0000 7fa5318fe640 1 --2- 172.21.15.105:0/3651903009 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa52c0d00b0 0x7fa52c0cdb50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:01.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.607+0000 7fa5318fe640 1 -- 172.21.15.105:0/3651903009 >> 172.21.15.105:0/3651903009 conn(0x7fa52c09c340 msgr2=0x7fa52c0d2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:01.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.607+0000 7fa5318fe640 1 -- 172.21.15.105:0/3651903009 shutdown_connections 2023-11-28T18:27:01.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.607+0000 7fa5318fe640 1 -- 172.21.15.105:0/3651903009 wait complete. 2023-11-28T18:27:01.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.607+0000 7fa5318fe640 1 Processor -- start 2023-11-28T18:27:01.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.607+0000 7fa5318fe640 1 -- start start 2023-11-28T18:27:01.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa5318fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa52c0b8a20 0x7fa52c0ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:01.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa5318fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa52c0ceb40 0x7fa52c0aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:01.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa5318fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa52c0d00b0 0x7fa52c1f92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:01.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa5318fe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c0b2cc0 con 0x7fa52c0ceb40 2023-11-28T18:27:01.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa5318fe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c0ccff0 con 0x7fa52c0d00b0 2023-11-28T18:27:01.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa5318fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c0b2f60 con 0x7fa52c0b8a20 2023-11-28T18:27:01.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa5227fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa52c0ceb40 0x7fa52c0aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:01.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa5227fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa52c0ceb40 0x7fa52c0aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44048/0 (socket says 172.21.15.105:44048) 2023-11-28T18:27:01.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa5227fc640 1 -- 172.21.15.105:0/722807951 learned_addr learned my addr 172.21.15.105:0/722807951 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:01.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa52b7fe640 1 --2- 172.21.15.105:0/722807951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa52c0d00b0 0x7fa52c1f92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:01.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa52affd640 1 --2- 172.21.15.105:0/722807951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa52c0b8a20 0x7fa52c0ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:01.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa5227fc640 1 -- 172.21.15.105:0/722807951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa52c0b8a20 msgr2=0x7fa52c0ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:01.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa5227fc640 1 --2- 172.21.15.105:0/722807951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa52c0b8a20 0x7fa52c0ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:01.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa5227fc640 1 -- 172.21.15.105:0/722807951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa52c0d00b0 msgr2=0x7fa52c1f92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:01.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa5227fc640 1 --2- 172.21.15.105:0/722807951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa52c0d00b0 0x7fa52c1f92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:01.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa5227fc640 1 -- 172.21.15.105:0/722807951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c009420 con 0x7fa52c0ceb40 2023-11-28T18:27:01.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa5227fc640 1 --2- 172.21.15.105:0/722807951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa52c0ceb40 0x7fa52c0aff00 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0097f0 tx=0x7fa51c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:01.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa528ff9640 1 -- 172.21.15.105:0/722807951 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0b5600 con 0x7fa52c0ceb40 2023-11-28T18:27:01.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa528ff9640 1 -- 172.21.15.105:0/722807951 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c0b5780 con 0x7fa52c0ceb40 2023-11-28T18:27:01.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa5318fe640 1 -- 172.21.15.105:0/722807951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52c0b3240 con 0x7fa52c0ceb40 2023-11-28T18:27:01.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa528ff9640 1 -- 172.21.15.105:0/722807951 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0b5b30 con 0x7fa52c0ceb40 2023-11-28T18:27:01.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa5318fe640 1 -- 172.21.15.105:0/722807951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52c0b3800 con 0x7fa52c0ceb40 2023-11-28T18:27:01.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa5318fe640 1 -- 172.21.15.105:0/722807951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa52c067570 con 0x7fa52c0ceb40 2023-11-28T18:27:01.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.611+0000 7fa528ff9640 1 -- 172.21.15.105:0/722807951 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa51c0b5c90 con 0x7fa52c0ceb40 2023-11-28T18:27:01.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.615+0000 7fa528ff9640 1 --2- 172.21.15.105:0/722807951 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa50c0668d0 0x7fa50c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:01.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.615+0000 7fa528ff9640 1 -- 172.21.15.105:0/722807951 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fa51c0a8050 con 0x7fa52c0ceb40 2023-11-28T18:27:01.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.615+0000 7fa52affd640 1 --2- 172.21.15.105:0/722807951 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa50c0668d0 0x7fa50c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:01.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.615+0000 7fa52affd640 1 --2- 172.21.15.105:0/722807951 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa50c0668d0 0x7fa50c068d90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa514007fd0 tx=0x7fa514009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:01.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.619+0000 7fa528ff9640 1 -- 172.21.15.105:0/722807951 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa51c0ebb60 con 0x7fa52c0ceb40 2023-11-28T18:27:01.771 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:27:01.775 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:27:01.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.815+0000 7fa5318fe640 1 -- 172.21.15.105:0/722807951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa52c0cdef0 con 0x7fa52c0ceb40 2023-11-28T18:27:01.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.815+0000 7fa528ff9640 1 -- 172.21.15.105:0/722807951 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa51c0c4020 con 0x7fa52c0ceb40 2023-11-28T18:27:01.822 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:27:01.822 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:27:01.822 INFO:teuthology.orchestra.run.smithi105.stdout: "auth_dump": [ 2023-11-28T18:27:01.822 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:27:01.822 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.a", 2023-11-28T18:27:01.822 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlOXSTGxAAU5kGNB3tznYsDNG51vCIZA==", 2023-11-28T18:27:01.822 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:27:01.822 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:27:01.822 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:27:01.822 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:27:01.823 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:27:01.823 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:27:01.823 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:27:01.823 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:27:01.823 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.b", 2023-11-28T18:27:01.823 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlrnbWJxAAKefl0YI2MhobnAOIj6WeaQ==", 2023-11-28T18:27:01.823 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:27:01.823 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:27:01.823 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:27:01.823 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:27:01.823 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:27:01.823 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:27:01.824 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:27:01.824 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:27:01.824 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.c", 2023-11-28T18:27:01.824 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2Zl5Y7GIhAA5OvgkNNFaI7LnzSsQVdFsQ==", 2023-11-28T18:27:01.824 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:27:01.824 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:27:01.824 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:27:01.824 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:27:01.824 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:27:01.824 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:27:01.824 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:27:01.825 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:27:01.825 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.d", 2023-11-28T18:27:01.825 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlmfYdLxAAreI7UE1YCjzkj6yl3s5Fwg==", 2023-11-28T18:27:01.825 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:27:01.825 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:27:01.825 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:27:01.825 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:27:01.825 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:27:01.825 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:27:01.825 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:27:01.825 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:27:01.825 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.0", 2023-11-28T18:27:01.826 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDtL2ZlphxQEBAAQ280OXh1zeaotIdVU8oZng==", 2023-11-28T18:27:01.826 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:27:01.826 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:27:01.826 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:27:01.826 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:27:01.826 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:27:01.826 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:27:01.826 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:27:01.826 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.1", 2023-11-28T18:27:01.826 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDwL2ZlezmOFxAARRBIOTkVJYfquIdcpW7sQQ==", 2023-11-28T18:27:01.826 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:27:01.826 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:27:01.827 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:27:01.827 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:27:01.827 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:27:01.827 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:27:01.827 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:27:01.827 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.2", 2023-11-28T18:27:01.827 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDzL2Zl4YcSHxAA2ww2F4TAD/0H/ZJ+aVMGEQ==", 2023-11-28T18:27:01.827 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:27:01.827 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:27:01.827 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:27:01.827 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:27:01.827 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:27:01.828 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:27:01.828 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:27:01.828 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.3", 2023-11-28T18:27:01.828 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD2L2ZlNxWxJRAAImp7TjTF18KwFTZ14DM6EA==", 2023-11-28T18:27:01.828 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:27:01.828 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:27:01.828 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:27:01.828 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:27:01.828 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:27:01.828 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:27:01.828 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:27:01.829 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.4", 2023-11-28T18:27:01.829 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD8L2ZlOAieLxAAz8g+anH8uJRfsttUS41GGQ==", 2023-11-28T18:27:01.829 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:27:01.829 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:27:01.829 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:27:01.829 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:27:01.829 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:27:01.829 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:27:01.829 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:27:01.829 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.5", 2023-11-28T18:27:01.829 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD/L2Zl5PKTNxAAPk+uIodLPjSd0EEMVT6iEw==", 2023-11-28T18:27:01.829 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:27:01.830 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:27:01.830 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:27:01.830 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:27:01.830 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:27:01.830 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:27:01.830 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:27:01.830 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.6", 2023-11-28T18:27:01.830 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQADMGZll+FUAxAAHlym5Z8rd/OF1911ZE2OUg==", 2023-11-28T18:27:01.830 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:27:01.830 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:27:01.831 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:27:01.831 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:27:01.831 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:27:01.831 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:27:01.831 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:27:01.831 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.7", 2023-11-28T18:27:01.831 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQAGMGZl8GO4ChAAVn3N6DnxUcs+ew09QvGo2w==", 2023-11-28T18:27:01.831 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:27:01.831 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:27:01.831 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:27:01.831 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:27:01.831 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:27:01.832 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:27:01.832 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:27:01.832 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.0", 2023-11-28T18:27:01.832 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlSt0VMxAAQ8+j1BgBPMJ2cuiZj8aG0w==", 2023-11-28T18:27:01.832 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:27:01.832 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:27:01.832 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow r", 2023-11-28T18:27:01.832 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-28T18:27:01.832 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:27:01.832 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:27:01.832 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:27:01.833 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.1", 2023-11-28T18:27:01.833 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDqL2ZlfZ1/ARAAhnJB6oj4Cc6p/oCjM6hTCw==", 2023-11-28T18:27:01.833 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:27:01.833 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:27:01.833 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:27:01.833 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:27:01.833 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:27:01.833 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:27:01.833 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:27:01.833 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:27:01.833 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.2", 2023-11-28T18:27:01.833 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlPLtyORAAU2GST6TJzlTfh6YqEb3U3w==", 2023-11-28T18:27:01.834 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:27:01.834 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:27:01.834 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:27:01.834 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:27:01.834 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:27:01.834 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:27:01.834 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:27:01.834 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:27:01.834 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.3", 2023-11-28T18:27:01.834 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDqL2ZlhiFuBxAAKQRxCdtqWpAHy1AVvdj9Sg==", 2023-11-28T18:27:01.834 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:27:01.835 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:27:01.835 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:27:01.835 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:27:01.835 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:27:01.835 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:27:01.835 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:27:01.835 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:27:01.835 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.admin", 2023-11-28T18:27:01.835 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDoL2ZlkuP2OhAAxsdZEF9122vkH53EwB8Qug==", 2023-11-28T18:27:01.835 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:27:01.835 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:27:01.835 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:27:01.836 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:27:01.836 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:27:01.836 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:27:01.836 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:27:01.836 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:27:01.836 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.x", 2023-11-28T18:27:01.836 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlEG49FxAAespsG+QK1WTpbmpV+DIwiw==", 2023-11-28T18:27:01.836 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:27:01.836 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:27:01.836 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2023-11-28T18:27:01.836 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:27:01.836 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:27:01.837 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:27:01.837 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:27:01.837 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.y", 2023-11-28T18:27:01.837 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlU89RExAAwMJp+8zZ6aFEt0KbiB6toA==", 2023-11-28T18:27:01.837 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:27:01.837 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:27:01.837 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2023-11-28T18:27:01.837 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:27:01.837 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:27:01.837 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:27:01.837 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2023-11-28T18:27:01.837 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:27:01.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.819+0000 7fa5318fe640 1 -- 172.21.15.105:0/722807951 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa50c0668d0 msgr2=0x7fa50c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:01.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.819+0000 7fa5318fe640 1 --2- 172.21.15.105:0/722807951 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa50c0668d0 0x7fa50c068d90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa514007fd0 tx=0x7fa514009040 comp rx=0 tx=0).stop 2023-11-28T18:27:01.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.819+0000 7fa5318fe640 1 -- 172.21.15.105:0/722807951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa52c0ceb40 msgr2=0x7fa52c0aff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:01.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.819+0000 7fa5318fe640 1 --2- 172.21.15.105:0/722807951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa52c0ceb40 0x7fa52c0aff00 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0097f0 tx=0x7fa51c01acd0 comp rx=0 tx=0).stop 2023-11-28T18:27:01.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.819+0000 7fa5318fe640 1 -- 172.21.15.105:0/722807951 shutdown_connections 2023-11-28T18:27:01.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.819+0000 7fa5318fe640 1 --2- 172.21.15.105:0/722807951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa52c0d00b0 0x7fa52c1f92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:01.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.819+0000 7fa5318fe640 1 --2- 172.21.15.105:0/722807951 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa50c0668d0 0x7fa50c068d90 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:01.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.819+0000 7fa5318fe640 1 --2- 172.21.15.105:0/722807951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa52c0ceb40 0x7fa52c0aff00 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:01.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.819+0000 7fa5318fe640 1 --2- 172.21.15.105:0/722807951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa52c0b8a20 0x7fa52c0ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:01.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.819+0000 7fa5318fe640 1 -- 172.21.15.105:0/722807951 >> 172.21.15.105:0/722807951 conn(0x7fa52c09c340 msgr2=0x7fa52c1606b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:01.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.819+0000 7fa5318fe640 1 -- 172.21.15.105:0/722807951 shutdown_connections 2023-11-28T18:27:01.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:01.819+0000 7fa5318fe640 1 -- 172.21.15.105:0/722807951 wait complete. 2023-11-28T18:27:01.843 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:27:02.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.015+0000 7f1f3dff5640 1 Processor -- start 2023-11-28T18:27:02.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.015+0000 7f1f3dff5640 1 -- start start 2023-11-28T18:27:02.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.019+0000 7f1f3dff5640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f380b4290 0x7f1f380b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:02.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.019+0000 7f1f3dff5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1f38071960 0x7f1f3815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:02.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.019+0000 7f1f3dff5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1f3815b030 0x7f1f380709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:02.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.019+0000 7f1f3dff5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f380b4bb0 con 0x7f1f3815b030 2023-11-28T18:27:02.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.019+0000 7f1f3dff5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f380701d0 con 0x7f1f38071960 2023-11-28T18:27:02.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.019+0000 7f1f3dff5640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3806df80 con 0x7f1f380b4290 2023-11-28T18:27:02.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.019+0000 7f1f37fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1f3815b030 0x7f1f380709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:02.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.019+0000 7f1f37fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1f3815b030 0x7f1f380709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44058/0 (socket says 172.21.15.105:44058) 2023-11-28T18:27:02.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.019+0000 7f1f37fff640 1 -- 172.21.15.105:0/2955299277 learned_addr learned my addr 172.21.15.105:0/2955299277 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:02.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.019+0000 7f1f377fe640 1 --2- 172.21.15.105:0/2955299277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f380b4290 0x7f1f380b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:02.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.019+0000 7f1f36ffd640 1 --2- 172.21.15.105:0/2955299277 >> v2:172.21.15.148:3300/0 conn(0x7f1f38071960 0x7f1f3815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:02.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.019+0000 7f1f37fff640 1 -- 172.21.15.105:0/2955299277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f380b4290 msgr2=0x7f1f380b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:02.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.019+0000 7f1f37fff640 1 --2- 172.21.15.105:0/2955299277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f380b4290 0x7f1f380b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.019+0000 7f1f37fff640 1 -- 172.21.15.105:0/2955299277 >> v2:172.21.15.148:3300/0 conn(0x7f1f38071960 msgr2=0x7f1f3815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:02.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.019+0000 7f1f37fff640 1 --2- 172.21.15.105:0/2955299277 >> v2:172.21.15.148:3300/0 conn(0x7f1f38071960 0x7f1f3815f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.019+0000 7f1f37fff640 1 -- 172.21.15.105:0/2955299277 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f38071df0 con 0x7f1f3815b030 2023-11-28T18:27:02.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.023+0000 7f1f37fff640 1 --2- 172.21.15.105:0/2955299277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f3815b030 0x7f1f380709b0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f1f380cc710 tx=0x7f1f3001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e684aa2e8572bb88 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:02.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.023+0000 7f1f367fc640 1 -- 172.21.15.105:0/2955299277 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f300a5730 con 0x7f1f3815b030 2023-11-28T18:27:02.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.023+0000 7f1f367fc640 1 -- 172.21.15.105:0/2955299277 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f300a5890 con 0x7f1f3815b030 2023-11-28T18:27:02.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.023+0000 7f1f367fc640 1 -- 172.21.15.105:0/2955299277 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f300a5c20 con 0x7f1f3815b030 2023-11-28T18:27:02.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.023+0000 7f1f3dff5640 1 -- 172.21.15.105:0/2955299277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f3815b030 msgr2=0x7f1f380709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:02.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.023+0000 7f1f3dff5640 1 --2- 172.21.15.105:0/2955299277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f3815b030 0x7f1f380709b0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f1f380cc710 tx=0x7f1f3001c9a0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.023+0000 7f1f3dff5640 1 -- 172.21.15.105:0/2955299277 shutdown_connections 2023-11-28T18:27:02.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.023+0000 7f1f3dff5640 1 --2- 172.21.15.105:0/2955299277 >> v2:172.21.15.148:3300/0 conn(0x7f1f38071960 0x7f1f3815f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.023+0000 7f1f3dff5640 1 --2- 172.21.15.105:0/2955299277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f3815b030 0x7f1f380709b0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.023+0000 7f1f3dff5640 1 --2- 172.21.15.105:0/2955299277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f380b4290 0x7f1f380b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.023+0000 7f1f3dff5640 1 -- 172.21.15.105:0/2955299277 >> 172.21.15.105:0/2955299277 conn(0x7f1f380ceb90 msgr2=0x7f1f380cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:02.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.023+0000 7f1f3dff5640 1 -- 172.21.15.105:0/2955299277 shutdown_connections 2023-11-28T18:27:02.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.023+0000 7f1f3dff5640 1 -- 172.21.15.105:0/2955299277 wait complete. 2023-11-28T18:27:02.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.023+0000 7f1f3dff5640 1 Processor -- start 2023-11-28T18:27:02.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.023+0000 7f1f3dff5640 1 -- start start 2023-11-28T18:27:02.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f3dff5640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f380b4290 0x7f1f38151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:02.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f3dff5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f3815b030 0x7f1f38151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:02.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f3dff5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f38152370 0x7f1f3809b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:02.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f3dff5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f38165a50 con 0x7f1f3815b030 2023-11-28T18:27:02.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f3dff5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f38159e70 con 0x7f1f38152370 2023-11-28T18:27:02.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f3dff5640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3815a150 con 0x7f1f380b4290 2023-11-28T18:27:02.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f36ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f3815b030 0x7f1f38151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:02.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f36ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f3815b030 0x7f1f38151590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44074/0 (socket says 172.21.15.105:44074) 2023-11-28T18:27:02.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f36ffd640 1 -- 172.21.15.105:0/2246813191 learned_addr learned my addr 172.21.15.105:0/2246813191 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:02.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f37fff640 1 --2- 172.21.15.105:0/2246813191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f38152370 0x7f1f3809b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:02.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f377fe640 1 --2- 172.21.15.105:0/2246813191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f380b4290 0x7f1f38151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:02.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f36ffd640 1 -- 172.21.15.105:0/2246813191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f380b4290 msgr2=0x7f1f38151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:02.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f36ffd640 1 --2- 172.21.15.105:0/2246813191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f380b4290 0x7f1f38151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f36ffd640 1 -- 172.21.15.105:0/2246813191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f38152370 msgr2=0x7f1f3809b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:02.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f36ffd640 1 --2- 172.21.15.105:0/2246813191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f38152370 0x7f1f3809b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f36ffd640 1 -- 172.21.15.105:0/2246813191 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f30009b30 con 0x7f1f3815b030 2023-11-28T18:27:02.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f377fe640 1 --2- 172.21.15.105:0/2246813191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f380b4290 0x7f1f38151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:02.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f36ffd640 1 --2- 172.21.15.105:0/2246813191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f3815b030 0x7f1f38151590 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00e930 tx=0x7f1f2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:02.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f1ffff640 1 -- 172.21.15.105:0/2246813191 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c009690 con 0x7f1f3815b030 2023-11-28T18:27:02.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f1ffff640 1 -- 172.21.15.105:0/2246813191 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2c00ca00 con 0x7f1f3815b030 2023-11-28T18:27:02.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f3dff5640 1 -- 172.21.15.105:0/2246813191 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3815a430 con 0x7f1f3815b030 2023-11-28T18:27:02.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f1ffff640 1 -- 172.21.15.105:0/2246813191 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c00cd70 con 0x7f1f3815b030 2023-11-28T18:27:02.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f3dff5640 1 -- 172.21.15.105:0/2246813191 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f38159a90 con 0x7f1f3815b030 2023-11-28T18:27:02.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.027+0000 7f1f3dff5640 1 -- 172.21.15.105:0/2246813191 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f38067570 con 0x7f1f3815b030 2023-11-28T18:27:02.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.031+0000 7f1f1ffff640 1 -- 172.21.15.105:0/2246813191 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f1f2c013470 con 0x7f1f3815b030 2023-11-28T18:27:02.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.031+0000 7f1f1ffff640 1 --2- 172.21.15.105:0/2246813191 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1f240668d0 0x7f1f24068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:02.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.031+0000 7f1f1ffff640 1 -- 172.21.15.105:0/2246813191 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f1f2c0843c0 con 0x7f1f3815b030 2023-11-28T18:27:02.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.031+0000 7f1f377fe640 1 --2- 172.21.15.105:0/2246813191 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1f240668d0 0x7f1f24068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:02.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.031+0000 7f1f377fe640 1 --2- 172.21.15.105:0/2246813191 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1f240668d0 0x7f1f24068d90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1f38155ee0 tx=0x7f1f28008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:02.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.035+0000 7f1f1ffff640 1 -- 172.21.15.105:0/2246813191 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1f2c051970 con 0x7f1f3815b030 2023-11-28T18:27:02.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.183+0000 7f1f3dff5640 1 -- 172.21.15.105:0/2246813191 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f380cf7f0 con 0x7f1f3815b030 2023-11-28T18:27:02.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.183+0000 7f1f1ffff640 1 -- 172.21.15.105:0/2246813191 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+13122 (secure 0 0 0) 0x7f1f2c054f90 con 0x7f1f3815b030 2023-11-28T18:27:02.188 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:27:02.188 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":61,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:27:01.446627+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:27:02.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.187+0000 7f1f3dff5640 1 -- 172.21.15.105:0/2246813191 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1f240668d0 msgr2=0x7f1f24068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:02.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.187+0000 7f1f3dff5640 1 --2- 172.21.15.105:0/2246813191 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1f240668d0 0x7f1f24068d90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1f38155ee0 tx=0x7f1f28008040 comp rx=0 tx=0).stop 2023-11-28T18:27:02.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.187+0000 7f1f3dff5640 1 -- 172.21.15.105:0/2246813191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f3815b030 msgr2=0x7f1f38151590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:02.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.187+0000 7f1f3dff5640 1 --2- 172.21.15.105:0/2246813191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f3815b030 0x7f1f38151590 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00e930 tx=0x7f1f2c00edf0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.187+0000 7f1f3dff5640 1 -- 172.21.15.105:0/2246813191 shutdown_connections 2023-11-28T18:27:02.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.187+0000 7f1f3dff5640 1 --2- 172.21.15.105:0/2246813191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f38152370 0x7f1f3809b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.187+0000 7f1f3dff5640 1 --2- 172.21.15.105:0/2246813191 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1f240668d0 0x7f1f24068d90 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.187+0000 7f1f3dff5640 1 --2- 172.21.15.105:0/2246813191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f3815b030 0x7f1f38151590 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.187+0000 7f1f3dff5640 1 --2- 172.21.15.105:0/2246813191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f380b4290 0x7f1f38151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.187+0000 7f1f3dff5640 1 -- 172.21.15.105:0/2246813191 >> 172.21.15.105:0/2246813191 conn(0x7f1f380ceb90 msgr2=0x7f1f380b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:02.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.187+0000 7f1f3dff5640 1 -- 172.21.15.105:0/2246813191 shutdown_connections 2023-11-28T18:27:02.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.187+0000 7f1f3dff5640 1 -- 172.21.15.105:0/2246813191 wait complete. 2023-11-28T18:27:02.210 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:27:02.211 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:27:02.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.347+0000 7fb56f59e640 1 Processor -- start 2023-11-28T18:27:02.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.347+0000 7fb56f59e640 1 -- start start 2023-11-28T18:27:02.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.351+0000 7fb56f59e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb5700c9030 0x7fb57015d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:02.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.351+0000 7fb56f59e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb57015de10 0x7fb5700716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:02.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.351+0000 7fb56f59e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb570070920 con 0x7fb570157910 2023-11-28T18:27:02.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.351+0000 7fb56f59e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb570157cf0 con 0x7fb5700c9030 2023-11-28T18:27:02.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.351+0000 7fb56f59e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57015e280 con 0x7fb57015de10 2023-11-28T18:27:02.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.351+0000 7fb56dd9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb5700c9030 0x7fb57015d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:02.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.351+0000 7fb56dd9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb5700c9030 0x7fb57015d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44080/0 (socket says 172.21.15.105:44080) 2023-11-28T18:27:02.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.351+0000 7fb56dd9b640 1 -- 172.21.15.105:0/59439254 learned_addr learned my addr 172.21.15.105:0/59439254 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:02.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.351+0000 7fb56ed9d640 1 --2- 172.21.15.105:0/59439254 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb57015de10 0x7fb5700716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:02.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.351+0000 7fb56dd9b640 1 -- 172.21.15.105:0/59439254 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb57015de10 msgr2=0x7fb5700716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:02.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.351+0000 7fb56dd9b640 1 --2- 172.21.15.105:0/59439254 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb57015de10 0x7fb5700716c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.351+0000 7fb56dd9b640 1 -- 172.21.15.105:0/59439254 >> v1:172.21.15.148:6789/0 conn(0x7fb570157910 legacy=0x7fb57006cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:02.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.351+0000 7fb56dd9b640 1 -- 172.21.15.105:0/59439254 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5700c9470 con 0x7fb5700c9030 2023-11-28T18:27:02.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.351+0000 7fb56dd9b640 1 --2- 172.21.15.105:0/59439254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5700c9030 0x7fb57015d8d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb564009870 tx=0x7fb56401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874e10e90f3a74af server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:02.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.355+0000 7fb56d59a640 1 -- 172.21.15.105:0/59439254 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5640ab680 con 0x7fb5700c9030 2023-11-28T18:27:02.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.355+0000 7fb56d59a640 1 -- 172.21.15.105:0/59439254 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5640ab7e0 con 0x7fb5700c9030 2023-11-28T18:27:02.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.355+0000 7fb56d59a640 1 -- 172.21.15.105:0/59439254 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5640abb70 con 0x7fb5700c9030 2023-11-28T18:27:02.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.355+0000 7fb56f59e640 1 -- 172.21.15.105:0/59439254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5700c9030 msgr2=0x7fb57015d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:02.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.355+0000 7fb56f59e640 1 --2- 172.21.15.105:0/59439254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5700c9030 0x7fb57015d8d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb564009870 tx=0x7fb56401c480 comp rx=0 tx=0).stop 2023-11-28T18:27:02.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.355+0000 7fb56f59e640 1 -- 172.21.15.105:0/59439254 shutdown_connections 2023-11-28T18:27:02.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.355+0000 7fb56f59e640 1 --2- 172.21.15.105:0/59439254 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb57015de10 0x7fb5700716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.355+0000 7fb56f59e640 1 --2- 172.21.15.105:0/59439254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5700c9030 0x7fb57015d8d0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.355+0000 7fb56f59e640 1 -- 172.21.15.105:0/59439254 >> 172.21.15.105:0/59439254 conn(0x7fb5700ce9e0 msgr2=0x7fb5700cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:02.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.355+0000 7fb56f59e640 1 -- 172.21.15.105:0/59439254 shutdown_connections 2023-11-28T18:27:02.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.355+0000 7fb56f59e640 1 -- 172.21.15.105:0/59439254 wait complete. 2023-11-28T18:27:02.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.355+0000 7fb56f59e640 1 Processor -- start 2023-11-28T18:27:02.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.355+0000 7fb56f59e640 1 -- start start 2023-11-28T18:27:02.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.355+0000 7fb56f59e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5700c9030 0x7fb57015c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:02.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.355+0000 7fb56f59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb570157910 0x7fb57014d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:02.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.359+0000 7fb56f59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb57015de10 0x7fb57006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:02.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.359+0000 7fb56f59e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb570153970 con 0x7fb570157910 2023-11-28T18:27:02.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.359+0000 7fb56f59e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57015a7c0 con 0x7fb57015de10 2023-11-28T18:27:02.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.359+0000 7fb56f59e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb570153320 con 0x7fb5700c9030 2023-11-28T18:27:02.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.359+0000 7fb56dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb570157910 0x7fb57014d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:02.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.359+0000 7fb56dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb570157910 0x7fb57014d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44092/0 (socket says 172.21.15.105:44092) 2023-11-28T18:27:02.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.359+0000 7fb56dd9b640 1 -- 172.21.15.105:0/2487842144 learned_addr learned my addr 172.21.15.105:0/2487842144 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:02.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.359+0000 7fb56e59c640 1 --2- 172.21.15.105:0/2487842144 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5700c9030 0x7fb57015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:02.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.359+0000 7fb56ed9d640 1 --2- 172.21.15.105:0/2487842144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb57015de10 0x7fb57006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:02.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.359+0000 7fb56dd9b640 1 -- 172.21.15.105:0/2487842144 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5700c9030 msgr2=0x7fb57015c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:02.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.359+0000 7fb56dd9b640 1 --2- 172.21.15.105:0/2487842144 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5700c9030 0x7fb57015c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.359+0000 7fb56dd9b640 1 -- 172.21.15.105:0/2487842144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb57015de10 msgr2=0x7fb57006c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:02.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.359+0000 7fb56dd9b640 1 --2- 172.21.15.105:0/2487842144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb57015de10 0x7fb57006c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.359+0000 7fb56dd9b640 1 -- 172.21.15.105:0/2487842144 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5640094d0 con 0x7fb570157910 2023-11-28T18:27:02.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.359+0000 7fb56dd9b640 1 --2- 172.21.15.105:0/2487842144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb570157910 0x7fb57014d5e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb56401a0b0 tx=0x7fb56401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:02.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.359+0000 7fb55effd640 1 -- 172.21.15.105:0/2487842144 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5640ab680 con 0x7fb570157910 2023-11-28T18:27:02.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.359+0000 7fb55effd640 1 -- 172.21.15.105:0/2487842144 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5640b4800 con 0x7fb570157910 2023-11-28T18:27:02.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.359+0000 7fb55effd640 1 -- 172.21.15.105:0/2487842144 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5640b4bb0 con 0x7fb570157910 2023-11-28T18:27:02.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.359+0000 7fb56f59e640 1 -- 172.21.15.105:0/2487842144 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb570153600 con 0x7fb570157910 2023-11-28T18:27:02.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.359+0000 7fb56f59e640 1 -- 172.21.15.105:0/2487842144 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb57015aaa0 con 0x7fb570157910 2023-11-28T18:27:02.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.359+0000 7fb56f59e640 1 -- 172.21.15.105:0/2487842144 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb570074580 con 0x7fb570157910 2023-11-28T18:27:02.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.359+0000 7fb55effd640 1 -- 172.21.15.105:0/2487842144 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fb5640b4d10 con 0x7fb570157910 2023-11-28T18:27:02.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.363+0000 7fb55effd640 1 --2- 172.21.15.105:0/2487842144 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb5500668d0 0x7fb550068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:02.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.363+0000 7fb55effd640 1 -- 172.21.15.105:0/2487842144 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fb56411ec30 con 0x7fb570157910 2023-11-28T18:27:02.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.363+0000 7fb56e59c640 1 --2- 172.21.15.105:0/2487842144 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb5500668d0 0x7fb550068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:02.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.363+0000 7fb56e59c640 1 --2- 172.21.15.105:0/2487842144 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb5500668d0 0x7fb550068d90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb560007fd0 tx=0x7fb560009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:02.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.363+0000 7fb55effd640 1 -- 172.21.15.105:0/2487842144 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb5640ec090 con 0x7fb570157910 2023-11-28T18:27:02.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.519+0000 7fb56f59e640 1 -- 172.21.15.105:0/2487842144 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb570067e30 con 0x7fb570157910 2023-11-28T18:27:02.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.519+0000 7fb55effd640 1 -- 172.21.15.105:0/2487842144 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb570067e30 con 0x7fb570157910 2023-11-28T18:27:02.524 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:27:02.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.523+0000 7fb56f59e640 1 -- 172.21.15.105:0/2487842144 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb5500668d0 msgr2=0x7fb550068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:02.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.523+0000 7fb56f59e640 1 --2- 172.21.15.105:0/2487842144 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb5500668d0 0x7fb550068d90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb560007fd0 tx=0x7fb560009040 comp rx=0 tx=0).stop 2023-11-28T18:27:02.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.523+0000 7fb56f59e640 1 -- 172.21.15.105:0/2487842144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb570157910 msgr2=0x7fb57014d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:02.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.523+0000 7fb56f59e640 1 --2- 172.21.15.105:0/2487842144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb570157910 0x7fb57014d5e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb56401a0b0 tx=0x7fb56401acc0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.523+0000 7fb56f59e640 1 -- 172.21.15.105:0/2487842144 shutdown_connections 2023-11-28T18:27:02.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.523+0000 7fb56f59e640 1 --2- 172.21.15.105:0/2487842144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb57015de10 0x7fb57006c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.523+0000 7fb56f59e640 1 --2- 172.21.15.105:0/2487842144 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb5500668d0 0x7fb550068d90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.523+0000 7fb56f59e640 1 --2- 172.21.15.105:0/2487842144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb570157910 0x7fb57014d5e0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.523+0000 7fb56f59e640 1 --2- 172.21.15.105:0/2487842144 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5700c9030 0x7fb57015c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.523+0000 7fb56f59e640 1 -- 172.21.15.105:0/2487842144 >> 172.21.15.105:0/2487842144 conn(0x7fb5700ce9e0 msgr2=0x7fb57006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:02.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.523+0000 7fb56f59e640 1 -- 172.21.15.105:0/2487842144 shutdown_connections 2023-11-28T18:27:02.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.523+0000 7fb56f59e640 1 -- 172.21.15.105:0/2487842144 wait complete. 2023-11-28T18:27:02.545 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T18:27:02.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.679+0000 7efe369c4640 1 Processor -- start 2023-11-28T18:27:02.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.679+0000 7efe369c4640 1 -- start start 2023-11-28T18:27:02.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.679+0000 7efe369c4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efe3015d4f0 0x7efe300d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:02.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.679+0000 7efe369c4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe300b2d20 0x7efe300b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:02.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.679+0000 7efe369c4640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe300b25b0 con 0x7efe300d3c30 2023-11-28T18:27:02.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.679+0000 7efe369c4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe300b2900 con 0x7efe3015d4f0 2023-11-28T18:27:02.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.679+0000 7efe369c4640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe300d0a20 con 0x7efe300b2d20 2023-11-28T18:27:02.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.679+0000 7efe2f7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7efe300d3c30 0x7efe300d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51234/0 (socket says 172.21.15.105:51234) 2023-11-28T18:27:02.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.679+0000 7efe2f7fe640 1 -- 172.21.15.105:0/3938420352 learned_addr learned my addr 172.21.15.105:0/3938420352 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:02.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.679+0000 7efe34f61640 1 --2- 172.21.15.105:0/3938420352 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe300b2d20 0x7efe300b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:02.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.679+0000 7efe2ffff640 1 --2- 172.21.15.105:0/3938420352 >> v2:172.21.15.148:3300/0 conn(0x7efe3015d4f0 0x7efe300d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:02.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.679+0000 7efe2effd640 1 -- 172.21.15.105:0/3938420352 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964470383 0 0) 0x7efe300b25b0 con 0x7efe300d3c30 2023-11-28T18:27:02.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.679+0000 7efe2effd640 1 -- 172.21.15.105:0/3938420352 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe18003530 con 0x7efe300d3c30 2023-11-28T18:27:02.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.679+0000 7efe34f61640 1 -- 172.21.15.105:0/3938420352 >> v2:172.21.15.148:3300/0 conn(0x7efe3015d4f0 msgr2=0x7efe300d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:02.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.679+0000 7efe34f61640 1 --2- 172.21.15.105:0/3938420352 >> v2:172.21.15.148:3300/0 conn(0x7efe3015d4f0 0x7efe300d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.679+0000 7efe34f61640 1 -- 172.21.15.105:0/3938420352 >> v1:172.21.15.105:6789/0 conn(0x7efe300d3c30 legacy=0x7efe300d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:02.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.679+0000 7efe34f61640 1 -- 172.21.15.105:0/3938420352 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe300cea30 con 0x7efe300b2d20 2023-11-28T18:27:02.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.683+0000 7efe34f61640 1 --2- 172.21.15.105:0/3938420352 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe300b2d20 0x7efe300b3190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efe28009d90 tx=0x7efe2801c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd004f5a8ab5b2d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:02.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.683+0000 7efe2effd640 1 -- 172.21.15.105:0/3938420352 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe280a5950 con 0x7efe300b2d20 2023-11-28T18:27:02.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.683+0000 7efe2effd640 1 -- 172.21.15.105:0/3938420352 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe280a5ab0 con 0x7efe300b2d20 2023-11-28T18:27:02.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.683+0000 7efe2effd640 1 -- 172.21.15.105:0/3938420352 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe280a5e40 con 0x7efe300b2d20 2023-11-28T18:27:02.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.683+0000 7efe369c4640 1 -- 172.21.15.105:0/3938420352 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe300b2d20 msgr2=0x7efe300b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:02.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.683+0000 7efe369c4640 1 --2- 172.21.15.105:0/3938420352 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe300b2d20 0x7efe300b3190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efe28009d90 tx=0x7efe2801c890 comp rx=0 tx=0).stop 2023-11-28T18:27:02.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.683+0000 7efe369c4640 1 -- 172.21.15.105:0/3938420352 shutdown_connections 2023-11-28T18:27:02.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.683+0000 7efe369c4640 1 --2- 172.21.15.105:0/3938420352 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe300b2d20 0x7efe300b3190 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.683+0000 7efe369c4640 1 --2- 172.21.15.105:0/3938420352 >> v2:172.21.15.148:3300/0 conn(0x7efe3015d4f0 0x7efe300d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.683+0000 7efe369c4640 1 -- 172.21.15.105:0/3938420352 >> 172.21.15.105:0/3938420352 conn(0x7efe300d3030 msgr2=0x7efe300d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:02.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.683+0000 7efe369c4640 1 -- 172.21.15.105:0/3938420352 shutdown_connections 2023-11-28T18:27:02.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.683+0000 7efe369c4640 1 -- 172.21.15.105:0/3938420352 wait complete. 2023-11-28T18:27:02.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe369c4640 1 Processor -- start 2023-11-28T18:27:02.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe369c4640 1 -- start start 2023-11-28T18:27:02.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe369c4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe300b2d20 0x7efe300b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:02.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe369c4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe300d3c30 0x7efe300cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:02.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe369c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe3015d4f0 0x7efe30154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:02.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe369c4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3015de30 con 0x7efe300d3c30 2023-11-28T18:27:02.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe369c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3015e110 con 0x7efe3015d4f0 2023-11-28T18:27:02.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe369c4640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3015e3f0 con 0x7efe300b2d20 2023-11-28T18:27:02.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe2f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe300d3c30 0x7efe300cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:02.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe2f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe300d3c30 0x7efe300cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44098/0 (socket says 172.21.15.105:44098) 2023-11-28T18:27:02.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe2f7fe640 1 -- 172.21.15.105:0/1504908506 learned_addr learned my addr 172.21.15.105:0/1504908506 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:02.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe2ffff640 1 --2- 172.21.15.105:0/1504908506 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe300b2d20 0x7efe300b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:02.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe34f61640 1 --2- 172.21.15.105:0/1504908506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe3015d4f0 0x7efe30154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:02.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe2f7fe640 1 -- 172.21.15.105:0/1504908506 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe300b2d20 msgr2=0x7efe300b84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:02.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe2f7fe640 1 --2- 172.21.15.105:0/1504908506 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe300b2d20 0x7efe300b84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe2f7fe640 1 -- 172.21.15.105:0/1504908506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe3015d4f0 msgr2=0x7efe30154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:02.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe2f7fe640 1 --2- 172.21.15.105:0/1504908506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe3015d4f0 0x7efe30154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe2f7fe640 1 -- 172.21.15.105:0/1504908506 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe28009a70 con 0x7efe300d3c30 2023-11-28T18:27:02.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe2f7fe640 1 --2- 172.21.15.105:0/1504908506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe300d3c30 0x7efe300cb950 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7efe2400c420 tx=0x7efe2400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:02.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe2cff9640 1 -- 172.21.15.105:0/1504908506 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe24005cf0 con 0x7efe300d3c30 2023-11-28T18:27:02.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe2cff9640 1 -- 172.21.15.105:0/1504908506 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe24005e50 con 0x7efe300d3c30 2023-11-28T18:27:02.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe369c4640 1 -- 172.21.15.105:0/1504908506 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3015e6d0 con 0x7efe300d3c30 2023-11-28T18:27:02.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe2cff9640 1 -- 172.21.15.105:0/1504908506 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe240072d0 con 0x7efe300d3c30 2023-11-28T18:27:02.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.687+0000 7efe369c4640 1 -- 172.21.15.105:0/1504908506 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe300b8a10 con 0x7efe300d3c30 2023-11-28T18:27:02.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.691+0000 7efe369c4640 1 -- 172.21.15.105:0/1504908506 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe30067570 con 0x7efe300d3c30 2023-11-28T18:27:02.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.691+0000 7efe2cff9640 1 -- 172.21.15.105:0/1504908506 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7efe24007430 con 0x7efe300d3c30 2023-11-28T18:27:02.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.691+0000 7efe2cff9640 1 --2- 172.21.15.105:0/1504908506 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe1c066900 0x7efe1c068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:02.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.691+0000 7efe2cff9640 1 -- 172.21.15.105:0/1504908506 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5177+0+0 (secure 0 0 0) 0x7efe24083800 con 0x7efe300d3c30 2023-11-28T18:27:02.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.691+0000 7efe2ffff640 1 --2- 172.21.15.105:0/1504908506 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe1c066900 0x7efe1c068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:02.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.691+0000 7efe2ffff640 1 --2- 172.21.15.105:0/1504908506 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe1c066900 0x7efe1c068dc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efe20009b60 tx=0x7efe20009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:02.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.695+0000 7efe2cff9640 1 -- 172.21.15.105:0/1504908506 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efe24050bd0 con 0x7efe300d3c30 2023-11-28T18:27:02.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.847+0000 7efe369c4640 1 -- 172.21.15.105:0/1504908506 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7efe30073140 con 0x7efe300d3c30 2023-11-28T18:27:02.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.847+0000 7efe2cff9640 1 -- 172.21.15.105:0/1504908506 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v61) v1 ==== 84+0+65 (secure 0 0 0) 0x7efe24017020 con 0x7efe300d3c30 2023-11-28T18:27:02.852 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:27:02.853 INFO:teuthology.orchestra.run.smithi105.stdout:[ 2023-11-28T18:27:02.853 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:27:02.853 INFO:teuthology.orchestra.run.smithi105.stdout: "poolnum": 1, 2023-11-28T18:27:02.853 INFO:teuthology.orchestra.run.smithi105.stdout: "poolname": ".mgr" 2023-11-28T18:27:02.853 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:27:02.853 INFO:teuthology.orchestra.run.smithi105.stdout:] 2023-11-28T18:27:02.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.847+0000 7efe369c4640 1 -- 172.21.15.105:0/1504908506 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe1c066900 msgr2=0x7efe1c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:02.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.847+0000 7efe369c4640 1 --2- 172.21.15.105:0/1504908506 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe1c066900 0x7efe1c068dc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efe20009b60 tx=0x7efe20009190 comp rx=0 tx=0).stop 2023-11-28T18:27:02.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.847+0000 7efe369c4640 1 -- 172.21.15.105:0/1504908506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe300d3c30 msgr2=0x7efe300cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:02.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.847+0000 7efe369c4640 1 --2- 172.21.15.105:0/1504908506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe300d3c30 0x7efe300cb950 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7efe2400c420 tx=0x7efe2400c8e0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.847+0000 7efe369c4640 1 -- 172.21.15.105:0/1504908506 shutdown_connections 2023-11-28T18:27:02.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.847+0000 7efe369c4640 1 --2- 172.21.15.105:0/1504908506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe3015d4f0 0x7efe30154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.847+0000 7efe369c4640 1 --2- 172.21.15.105:0/1504908506 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe1c066900 0x7efe1c068dc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.847+0000 7efe369c4640 1 --2- 172.21.15.105:0/1504908506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe300d3c30 0x7efe300cb950 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.847+0000 7efe369c4640 1 --2- 172.21.15.105:0/1504908506 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe300b2d20 0x7efe300b84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:02.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.847+0000 7efe369c4640 1 -- 172.21.15.105:0/1504908506 >> 172.21.15.105:0/1504908506 conn(0x7efe300d3030 msgr2=0x7efe300d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:02.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.847+0000 7efe369c4640 1 -- 172.21.15.105:0/1504908506 shutdown_connections 2023-11-28T18:27:02.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:02.847+0000 7efe369c4640 1 -- 172.21.15.105:0/1504908506 wait complete. 2023-11-28T18:27:02.871 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T18:27:02.872 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T18:27:03.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.007+0000 7f76925f8640 1 Processor -- start 2023-11-28T18:27:03.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.007+0000 7f76925f8640 1 -- start start 2023-11-28T18:27:03.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.011+0000 7f76925f8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f768c071a10 0x7f768c0ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:03.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.011+0000 7f76925f8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f768c0cefa0 con 0x7f768c070a10 2023-11-28T18:27:03.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.011+0000 7f76925f8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f768c15d7a0 con 0x7f768c15d3c0 2023-11-28T18:27:03.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.011+0000 7f76925f8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f768c0ce9e0 con 0x7f768c071a10 2023-11-28T18:27:03.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.011+0000 7f768b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f768c071a10 0x7f768c0ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:03.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.011+0000 7f768b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f768c071a10 0x7f768c0ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44106/0 (socket says 172.21.15.105:44106) 2023-11-28T18:27:03.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.011+0000 7f7690b95640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f768c070a10 0x7f768c070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51244/0 (socket says 172.21.15.105:51244) 2023-11-28T18:27:03.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.011+0000 7f768b7fe640 1 -- 172.21.15.105:0/877721229 learned_addr learned my addr 172.21.15.105:0/877721229 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:03.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.011+0000 7f768b7fe640 1 -- 172.21.15.105:0/877721229 >> v1:172.21.15.148:6789/0 conn(0x7f768c15d3c0 legacy=0x7f768c161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:03.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.011+0000 7f768b7fe640 1 -- 172.21.15.105:0/877721229 >> v1:172.21.15.105:6789/0 conn(0x7f768c070a10 legacy=0x7f768c070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:03.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.011+0000 7f768b7fe640 1 -- 172.21.15.105:0/877721229 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768c0cecc0 con 0x7f768c071a10 2023-11-28T18:27:03.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.011+0000 7f768affd640 1 -- 172.21.15.105:0/877721229 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4240623319 0 0) 0x7f768c0cefa0 con 0x7f768c070a10 2023-11-28T18:27:03.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.015+0000 7f768b7fe640 1 --2- 172.21.15.105:0/877721229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f768c071a10 0x7f768c0ce190 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f7680009870 tx=0x7f768001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8051138a9499d5e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:03.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.015+0000 7f768affd640 1 -- 172.21.15.105:0/877721229 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76800ab680 con 0x7f768c071a10 2023-11-28T18:27:03.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.015+0000 7f768affd640 1 -- 172.21.15.105:0/877721229 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76800ab7e0 con 0x7f768c071a10 2023-11-28T18:27:03.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.015+0000 7f768affd640 1 -- 172.21.15.105:0/877721229 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76800abb70 con 0x7f768c071a10 2023-11-28T18:27:03.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.015+0000 7f76925f8640 1 -- 172.21.15.105:0/877721229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f768c071a10 msgr2=0x7f768c0ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:03.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.015+0000 7f76925f8640 1 --2- 172.21.15.105:0/877721229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f768c071a10 0x7f768c0ce190 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f7680009870 tx=0x7f768001c480 comp rx=0 tx=0).stop 2023-11-28T18:27:03.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.015+0000 7f76925f8640 1 -- 172.21.15.105:0/877721229 shutdown_connections 2023-11-28T18:27:03.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.015+0000 7f76925f8640 1 --2- 172.21.15.105:0/877721229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f768c071a10 0x7f768c0ce190 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:03.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.015+0000 7f76925f8640 1 -- 172.21.15.105:0/877721229 >> 172.21.15.105:0/877721229 conn(0x7f768c0cf540 msgr2=0x7f768c0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:03.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.015+0000 7f76925f8640 1 -- 172.21.15.105:0/877721229 shutdown_connections 2023-11-28T18:27:03.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.015+0000 7f76925f8640 1 -- 172.21.15.105:0/877721229 wait complete. 2023-11-28T18:27:03.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.015+0000 7f76925f8640 1 Processor -- start 2023-11-28T18:27:03.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.015+0000 7f76925f8640 1 -- start start 2023-11-28T18:27:03.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.015+0000 7f76925f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f768c070a10 0x7f768c0b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:03.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.015+0000 7f76925f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f768c071a10 0x7f768c1600a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:03.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.015+0000 7f76925f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f768c15d3c0 0x7f768c15d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:03.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.015+0000 7f76925f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c158cf0 con 0x7f768c070a10 2023-11-28T18:27:03.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.015+0000 7f76925f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c159af0 con 0x7f768c071a10 2023-11-28T18:27:03.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.015+0000 7f76925f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f768c158f20 con 0x7f768c15d3c0 2023-11-28T18:27:03.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.019+0000 7f768bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f768c070a10 0x7f768c0b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:03.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.019+0000 7f768bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f768c070a10 0x7f768c0b39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44120/0 (socket says 172.21.15.105:44120) 2023-11-28T18:27:03.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.019+0000 7f768bfff640 1 -- 172.21.15.105:0/3149005304 learned_addr learned my addr 172.21.15.105:0/3149005304 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:03.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.019+0000 7f768b7fe640 1 --2- 172.21.15.105:0/3149005304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f768c071a10 0x7f768c1600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:03.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.019+0000 7f7690b95640 1 --2- 172.21.15.105:0/3149005304 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f768c15d3c0 0x7f768c15d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:03.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.019+0000 7f768bfff640 1 -- 172.21.15.105:0/3149005304 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f768c15d3c0 msgr2=0x7f768c15d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:03.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.019+0000 7f768bfff640 1 --2- 172.21.15.105:0/3149005304 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f768c15d3c0 0x7f768c15d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:03.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.019+0000 7f768bfff640 1 -- 172.21.15.105:0/3149005304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f768c071a10 msgr2=0x7f768c1600a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:03.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.019+0000 7f768bfff640 1 --2- 172.21.15.105:0/3149005304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f768c071a10 0x7f768c1600a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:03.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.019+0000 7f768bfff640 1 -- 172.21.15.105:0/3149005304 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76800094d0 con 0x7f768c070a10 2023-11-28T18:27:03.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.019+0000 7f7690b95640 1 --2- 172.21.15.105:0/3149005304 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f768c15d3c0 0x7f768c15d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:03.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.019+0000 7f768bfff640 1 --2- 172.21.15.105:0/3149005304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f768c070a10 0x7f768c0b39a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f767c00c500 tx=0x7f767c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:03.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.019+0000 7f7688ff9640 1 -- 172.21.15.105:0/3149005304 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c005220 con 0x7f768c070a10 2023-11-28T18:27:03.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.019+0000 7f7688ff9640 1 -- 172.21.15.105:0/3149005304 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767c005380 con 0x7f768c070a10 2023-11-28T18:27:03.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.019+0000 7f76925f8640 1 -- 172.21.15.105:0/3149005304 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f768c1591c0 con 0x7f768c070a10 2023-11-28T18:27:03.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.019+0000 7f7688ff9640 1 -- 172.21.15.105:0/3149005304 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c007270 con 0x7f768c070a10 2023-11-28T18:27:03.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.019+0000 7f76925f8640 1 -- 172.21.15.105:0/3149005304 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f768c154c90 con 0x7f768c070a10 2023-11-28T18:27:03.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.019+0000 7f76925f8640 1 -- 172.21.15.105:0/3149005304 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f768c1595b0 con 0x7f768c070a10 2023-11-28T18:27:03.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.019+0000 7f7688ff9640 1 -- 172.21.15.105:0/3149005304 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f767c0073d0 con 0x7f768c070a10 2023-11-28T18:27:03.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.019+0000 7f7688ff9640 1 --2- 172.21.15.105:0/3149005304 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f76780668d0 0x7f7678068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:03.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.019+0000 7f7688ff9640 1 -- 172.21.15.105:0/3149005304 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f767c083b70 con 0x7f768c070a10 2023-11-28T18:27:03.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.023+0000 7f768b7fe640 1 --2- 172.21.15.105:0/3149005304 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f76780668d0 0x7f7678068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:03.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.023+0000 7f768b7fe640 1 --2- 172.21.15.105:0/3149005304 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f76780668d0 0x7f7678068d90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f76800af000 tx=0x7f76800b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:03.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.023+0000 7f7688ff9640 1 -- 172.21.15.105:0/3149005304 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f767c050f30 con 0x7f768c070a10 2023-11-28T18:27:03.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:03.175+0000 7f76925f8640 1 -- 172.21.15.105:0/3149005304 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f768c067570 con 0x7f768c070a10 2023-11-28T18:27:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.083+0000 7f7688ff9640 1 -- 172.21.15.105:0/3149005304 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v62) v1 ==== 138+0+0 (secure 0 0 0) 0x7f767c054550 con 0x7f768c070a10 2023-11-28T18:27:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' created 2023-11-28T18:27:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.087+0000 7f76925f8640 1 -- 172.21.15.105:0/3149005304 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f76780668d0 msgr2=0x7f7678068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.087+0000 7f76925f8640 1 --2- 172.21.15.105:0/3149005304 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f76780668d0 0x7f7678068d90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f76800af000 tx=0x7f76800b6040 comp rx=0 tx=0).stop 2023-11-28T18:27:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.087+0000 7f76925f8640 1 -- 172.21.15.105:0/3149005304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f768c070a10 msgr2=0x7f768c0b39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.087+0000 7f76925f8640 1 --2- 172.21.15.105:0/3149005304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f768c070a10 0x7f768c0b39a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f767c00c500 tx=0x7f767c00c9c0 comp rx=0 tx=0).stop 2023-11-28T18:27:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.087+0000 7f76925f8640 1 -- 172.21.15.105:0/3149005304 shutdown_connections 2023-11-28T18:27:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.087+0000 7f76925f8640 1 --2- 172.21.15.105:0/3149005304 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f768c15d3c0 0x7f768c15d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.087+0000 7f76925f8640 1 --2- 172.21.15.105:0/3149005304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f768c071a10 0x7f768c1600a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.087+0000 7f76925f8640 1 --2- 172.21.15.105:0/3149005304 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f76780668d0 0x7f7678068d90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.087+0000 7f76925f8640 1 --2- 172.21.15.105:0/3149005304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f768c070a10 0x7f768c0b39a0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.087+0000 7f76925f8640 1 -- 172.21.15.105:0/3149005304 >> 172.21.15.105:0/3149005304 conn(0x7f768c0cf540 msgr2=0x7f768c06af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.087+0000 7f76925f8640 1 -- 172.21.15.105:0/3149005304 shutdown_connections 2023-11-28T18:27:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.087+0000 7f76925f8640 1 -- 172.21.15.105:0/3149005304 wait complete. 2023-11-28T18:27:04.120 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T18:27:04.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.263+0000 7fbc4159d640 1 Processor -- start 2023-11-28T18:27:04.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.263+0000 7fbc4159d640 1 -- start start 2023-11-28T18:27:04.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.263+0000 7fbc4159d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc3c159f40 0x7fbc3c0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:04.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.263+0000 7fbc4159d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbc3c0cc760 0x7fbc3c0ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:04.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.263+0000 7fbc4159d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3c06bf50 con 0x7fbc3c0ad680 2023-11-28T18:27:04.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.263+0000 7fbc4159d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc3c0ccbc0 con 0x7fbc3c0cc760 2023-11-28T18:27:04.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.263+0000 7fbc4159d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c15a320 con 0x7fbc3c159f40 2023-11-28T18:27:04.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.263+0000 7fbc327fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbc3c0cc760 0x7fbc3c0ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:04.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.263+0000 7fbc327fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbc3c0cc760 0x7fbc3c0ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44130/0 (socket says 172.21.15.105:44130) 2023-11-28T18:27:04.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.263+0000 7fbc327fc640 1 -- 172.21.15.105:0/3514645838 learned_addr learned my addr 172.21.15.105:0/3514645838 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:04.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.263+0000 7fbc3affd640 1 --2- 172.21.15.105:0/3514645838 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc3c159f40 0x7fbc3c0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:04.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.263+0000 7fbc327fc640 1 -- 172.21.15.105:0/3514645838 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc3c159f40 msgr2=0x7fbc3c0caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:04.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.263+0000 7fbc327fc640 1 --2- 172.21.15.105:0/3514645838 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc3c159f40 0x7fbc3c0caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:04.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.263+0000 7fbc327fc640 1 -- 172.21.15.105:0/3514645838 >> v1:172.21.15.148:6789/0 conn(0x7fbc3c0ad680 legacy=0x7fbc3c15da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:04.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.263+0000 7fbc327fc640 1 -- 172.21.15.105:0/3514645838 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3c071380 con 0x7fbc3c0cc760 2023-11-28T18:27:04.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.267+0000 7fbc327fc640 1 --2- 172.21.15.105:0/3514645838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc3c0cc760 0x7fbc3c0ad140 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009e50 tx=0x7fbc2401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77bd199260c11eea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:04.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.267+0000 7fbc3a7fc640 1 -- 172.21.15.105:0/3514645838 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc240a5810 con 0x7fbc3c0cc760 2023-11-28T18:27:04.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.267+0000 7fbc3a7fc640 1 -- 172.21.15.105:0/3514645838 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc240a5970 con 0x7fbc3c0cc760 2023-11-28T18:27:04.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.267+0000 7fbc3a7fc640 1 -- 172.21.15.105:0/3514645838 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc240a5d00 con 0x7fbc3c0cc760 2023-11-28T18:27:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.267+0000 7fbc4159d640 1 -- 172.21.15.105:0/3514645838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc3c0cc760 msgr2=0x7fbc3c0ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.267+0000 7fbc4159d640 1 --2- 172.21.15.105:0/3514645838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc3c0cc760 0x7fbc3c0ad140 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009e50 tx=0x7fbc2401c950 comp rx=0 tx=0).stop 2023-11-28T18:27:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.267+0000 7fbc4159d640 1 -- 172.21.15.105:0/3514645838 shutdown_connections 2023-11-28T18:27:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.267+0000 7fbc4159d640 1 --2- 172.21.15.105:0/3514645838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc3c0cc760 0x7fbc3c0ad140 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.267+0000 7fbc4159d640 1 --2- 172.21.15.105:0/3514645838 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc3c159f40 0x7fbc3c0caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.267+0000 7fbc4159d640 1 -- 172.21.15.105:0/3514645838 >> 172.21.15.105:0/3514645838 conn(0x7fbc3c0ceb90 msgr2=0x7fbc3c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:04.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.267+0000 7fbc4159d640 1 -- 172.21.15.105:0/3514645838 shutdown_connections 2023-11-28T18:27:04.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.267+0000 7fbc4159d640 1 -- 172.21.15.105:0/3514645838 wait complete. 2023-11-28T18:27:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc4159d640 1 Processor -- start 2023-11-28T18:27:04.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc4159d640 1 -- start start 2023-11-28T18:27:04.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc4159d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc3c0ad680 0x7fbc3c14c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:04.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc4159d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc3c0cc760 0x7fbc3c15cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:04.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc4159d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc3c159f40 0x7fbc3c1528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:04.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc4159d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c14d480 con 0x7fbc3c0ad680 2023-11-28T18:27:04.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc4159d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c15a6c0 con 0x7fbc3c0cc760 2023-11-28T18:27:04.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc4159d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c15a9a0 con 0x7fbc3c159f40 2023-11-28T18:27:04.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc3affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc3c0ad680 0x7fbc3c14c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:04.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc3affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc3c0ad680 0x7fbc3c14c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44134/0 (socket says 172.21.15.105:44134) 2023-11-28T18:27:04.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc3affd640 1 -- 172.21.15.105:0/1050855501 learned_addr learned my addr 172.21.15.105:0/1050855501 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:04.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc327fc640 1 --2- 172.21.15.105:0/1050855501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc3c0cc760 0x7fbc3c15cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:04.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc3b7fe640 1 --2- 172.21.15.105:0/1050855501 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc3c159f40 0x7fbc3c1528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:04.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc3affd640 1 -- 172.21.15.105:0/1050855501 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc3c159f40 msgr2=0x7fbc3c1528d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:04.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc3affd640 1 --2- 172.21.15.105:0/1050855501 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc3c159f40 0x7fbc3c1528d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:04.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc3affd640 1 -- 172.21.15.105:0/1050855501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc3c0cc760 msgr2=0x7fbc3c15cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:04.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc3affd640 1 --2- 172.21.15.105:0/1050855501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc3c0cc760 0x7fbc3c15cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:04.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc3affd640 1 -- 172.21.15.105:0/1050855501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc24009b30 con 0x7fbc3c0ad680 2023-11-28T18:27:04.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc3b7fe640 1 --2- 172.21.15.105:0/1050855501 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc3c159f40 0x7fbc3c1528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:04.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc3affd640 1 --2- 172.21.15.105:0/1050855501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc3c0ad680 0x7fbc3c14c890 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00d5f0 tx=0x7fbc2c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:04.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc33fff640 1 -- 172.21.15.105:0/1050855501 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c0125f0 con 0x7fbc3c0ad680 2023-11-28T18:27:04.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc33fff640 1 -- 172.21.15.105:0/1050855501 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2c012750 con 0x7fbc3c0ad680 2023-11-28T18:27:04.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc4159d640 1 -- 172.21.15.105:0/1050855501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc3c15ac80 con 0x7fbc3c0ad680 2023-11-28T18:27:04.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc33fff640 1 -- 172.21.15.105:0/1050855501 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c012ae0 con 0x7fbc3c0ad680 2023-11-28T18:27:04.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.271+0000 7fbc4159d640 1 -- 172.21.15.105:0/1050855501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc3c15b240 con 0x7fbc3c0ad680 2023-11-28T18:27:04.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.275+0000 7fbc4159d640 1 -- 172.21.15.105:0/1050855501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc3c15b400 con 0x7fbc3c0ad680 2023-11-28T18:27:04.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.275+0000 7fbc33fff640 1 -- 172.21.15.105:0/1050855501 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fbc2c012c40 con 0x7fbc3c0ad680 2023-11-28T18:27:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.275+0000 7fbc33fff640 1 --2- 172.21.15.105:0/1050855501 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbc1c0668d0 0x7fbc1c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.275+0000 7fbc33fff640 1 -- 172.21.15.105:0/1050855501 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5547+0+0 (secure 0 0 0) 0x7fbc2c0847e0 con 0x7fbc3c0ad680 2023-11-28T18:27:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.275+0000 7fbc327fc640 1 --2- 172.21.15.105:0/1050855501 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbc1c0668d0 0x7fbc1c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.275+0000 7fbc327fc640 1 --2- 172.21.15.105:0/1050855501 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbc1c0668d0 0x7fbc1c068d90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbc240ae000 tx=0x7fbc240a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.279+0000 7fbc33fff640 1 -- 172.21.15.105:0/1050855501 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbc2c051ab0 con 0x7fbc3c0ad680 2023-11-28T18:27:04.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:04.431+0000 7fbc4159d640 1 -- 172.21.15.105:0/1050855501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbc3c067570 con 0x7fbc3c0ad680 2023-11-28T18:27:05.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.087+0000 7fbc33fff640 1 -- 172.21.15.105:0/1050855501 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v63) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbc2c0550d0 con 0x7fbc3c0ad680 2023-11-28T18:27:05.092 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' created 2023-11-28T18:27:05.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.091+0000 7fbc4159d640 1 -- 172.21.15.105:0/1050855501 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbc1c0668d0 msgr2=0x7fbc1c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:05.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.091+0000 7fbc4159d640 1 --2- 172.21.15.105:0/1050855501 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbc1c0668d0 0x7fbc1c068d90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbc240ae000 tx=0x7fbc240a7070 comp rx=0 tx=0).stop 2023-11-28T18:27:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.091+0000 7fbc4159d640 1 -- 172.21.15.105:0/1050855501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc3c0ad680 msgr2=0x7fbc3c14c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.091+0000 7fbc4159d640 1 --2- 172.21.15.105:0/1050855501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc3c0ad680 0x7fbc3c14c890 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00d5f0 tx=0x7fbc2c00dab0 comp rx=0 tx=0).stop 2023-11-28T18:27:05.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.091+0000 7fbc4159d640 1 -- 172.21.15.105:0/1050855501 shutdown_connections 2023-11-28T18:27:05.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.091+0000 7fbc4159d640 1 --2- 172.21.15.105:0/1050855501 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc3c159f40 0x7fbc3c1528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:05.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.091+0000 7fbc4159d640 1 --2- 172.21.15.105:0/1050855501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc3c0cc760 0x7fbc3c15cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:05.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.091+0000 7fbc4159d640 1 --2- 172.21.15.105:0/1050855501 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbc1c0668d0 0x7fbc1c068d90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:05.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.091+0000 7fbc4159d640 1 --2- 172.21.15.105:0/1050855501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc3c0ad680 0x7fbc3c14c890 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:05.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.091+0000 7fbc4159d640 1 -- 172.21.15.105:0/1050855501 >> 172.21.15.105:0/1050855501 conn(0x7fbc3c0ceb90 msgr2=0x7fbc3c15d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:05.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.091+0000 7fbc4159d640 1 -- 172.21.15.105:0/1050855501 shutdown_connections 2023-11-28T18:27:05.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.091+0000 7fbc4159d640 1 -- 172.21.15.105:0/1050855501 wait complete. 2023-11-28T18:27:05.135 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T18:27:05.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.271+0000 7f6eb8ed6640 1 Processor -- start 2023-11-28T18:27:05.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.271+0000 7f6eb8ed6640 1 -- start start 2023-11-28T18:27:05.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.275+0000 7f6eb8ed6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6eb40b4290 0x7f6eb40b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:05.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.275+0000 7f6eb8ed6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb40b4bb0 con 0x7f6eb4071960 2023-11-28T18:27:05.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.275+0000 7f6eb8ed6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb40701d0 con 0x7f6eb415b030 2023-11-28T18:27:05.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.275+0000 7f6eb8ed6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb406df80 con 0x7f6eb40b4290 2023-11-28T18:27:05.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.275+0000 7f6ea9d9b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f6eb4071960 0x7f6eb415f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51254/0 (socket says 172.21.15.105:51254) 2023-11-28T18:27:05.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.275+0000 7f6ea9d9b640 1 -- 172.21.15.105:0/3825588832 learned_addr learned my addr 172.21.15.105:0/3825588832 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:05.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.275+0000 7f6eb259c640 1 --2- 172.21.15.105:0/3825588832 >> v2:172.21.15.148:3300/0 conn(0x7f6eb40b4290 0x7f6eb40b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:05.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.275+0000 7f6eb1d9b640 1 -- 172.21.15.105:0/3825588832 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247943529 0 0) 0x7f6eb40b4bb0 con 0x7f6eb4071960 2023-11-28T18:27:05.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.275+0000 7f6eb1d9b640 1 -- 172.21.15.105:0/3825588832 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ea0003530 con 0x7f6eb4071960 2023-11-28T18:27:05.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.275+0000 7f6eb1d9b640 1 -- 172.21.15.105:0/3825588832 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 780640486 0 0) 0x7f6eb40701d0 con 0x7f6eb415b030 2023-11-28T18:27:05.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.275+0000 7f6eb1d9b640 1 -- 172.21.15.105:0/3825588832 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6eb40b4bb0 con 0x7f6eb415b030 2023-11-28T18:27:05.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.275+0000 7f6eb1d9b640 1 -- 172.21.15.105:0/3825588832 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2336591583 0 0) 0x7f6ea0003530 con 0x7f6eb4071960 2023-11-28T18:27:05.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.275+0000 7f6eb259c640 1 -- 172.21.15.105:0/3825588832 >> v1:172.21.15.148:6789/0 conn(0x7f6eb415b030 legacy=0x7f6eb40709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:05.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.275+0000 7f6eb259c640 1 -- 172.21.15.105:0/3825588832 >> v1:172.21.15.105:6789/0 conn(0x7f6eb4071960 legacy=0x7f6eb415f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:05.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.275+0000 7f6eb259c640 1 -- 172.21.15.105:0/3825588832 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb4071df0 con 0x7f6eb40b4290 2023-11-28T18:27:05.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.279+0000 7f6eb259c640 1 --2- 172.21.15.105:0/3825588832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6eb40b4290 0x7f6eb40b4670 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009d90 tx=0x7f6ea401c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bca1133ef37f9c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:05.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.279+0000 7f6eb1d9b640 1 -- 172.21.15.105:0/3825588832 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c001600 con 0x7f6eb40b4290 2023-11-28T18:27:05.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.279+0000 7f6eb1d9b640 1 -- 172.21.15.105:0/3825588832 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea40a5680 con 0x7f6eb40b4290 2023-11-28T18:27:05.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.279+0000 7f6eb1d9b640 1 -- 172.21.15.105:0/3825588832 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea40a59f0 con 0x7f6eb40b4290 2023-11-28T18:27:05.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.279+0000 7f6eb8ed6640 1 -- 172.21.15.105:0/3825588832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6eb40b4290 msgr2=0x7f6eb40b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:05.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.279+0000 7f6eb8ed6640 1 --2- 172.21.15.105:0/3825588832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6eb40b4290 0x7f6eb40b4670 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009d90 tx=0x7f6ea401c840 comp rx=0 tx=0).stop 2023-11-28T18:27:05.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.279+0000 7f6eb8ed6640 1 -- 172.21.15.105:0/3825588832 shutdown_connections 2023-11-28T18:27:05.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.279+0000 7f6eb8ed6640 1 --2- 172.21.15.105:0/3825588832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6eb40b4290 0x7f6eb40b4670 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:05.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.279+0000 7f6eb8ed6640 1 -- 172.21.15.105:0/3825588832 >> 172.21.15.105:0/3825588832 conn(0x7f6eb40ceb90 msgr2=0x7f6eb40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:05.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.279+0000 7f6eb8ed6640 1 -- 172.21.15.105:0/3825588832 shutdown_connections 2023-11-28T18:27:05.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.279+0000 7f6eb8ed6640 1 -- 172.21.15.105:0/3825588832 wait complete. 2023-11-28T18:27:05.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eb8ed6640 1 Processor -- start 2023-11-28T18:27:05.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eb8ed6640 1 -- start start 2023-11-28T18:27:05.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eb8ed6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6eb40b4290 0x7f6eb41477b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:05.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eb8ed6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6eb415b030 0x7f6eb409a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:05.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eb8ed6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6eb41f60e0 0x7f6eb409a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:05.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eb8ed6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb409afd0 con 0x7f6eb40b4290 2023-11-28T18:27:05.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eb8ed6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb40b2930 con 0x7f6eb41f60e0 2023-11-28T18:27:05.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eb8ed6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb40aeb40 con 0x7f6eb415b030 2023-11-28T18:27:05.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eb259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6eb40b4290 0x7f6eb41477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:05.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eb259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6eb40b4290 0x7f6eb41477b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44150/0 (socket says 172.21.15.105:44150) 2023-11-28T18:27:05.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6ea9d9b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6eb415b030 0x7f6eb409a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:05.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eb259c640 1 -- 172.21.15.105:0/4223369229 learned_addr learned my addr 172.21.15.105:0/4223369229 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:05.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eb2d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6eb41f60e0 0x7f6eb409a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:05.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eb259c640 1 -- 172.21.15.105:0/4223369229 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6eb415b030 msgr2=0x7f6eb409a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:05.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eb259c640 1 --2- 172.21.15.105:0/4223369229 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6eb415b030 0x7f6eb409a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:05.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eb259c640 1 -- 172.21.15.105:0/4223369229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6eb41f60e0 msgr2=0x7f6eb409a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:05.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eb259c640 1 --2- 172.21.15.105:0/4223369229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6eb41f60e0 0x7f6eb409a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:05.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eb259c640 1 -- 172.21.15.105:0/4223369229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea4009a70 con 0x7f6eb40b4290 2023-11-28T18:27:05.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6ea9d9b640 1 --2- 172.21.15.105:0/4223369229 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6eb415b030 0x7f6eb409a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:27:05.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eb259c640 1 --2- 172.21.15.105:0/4223369229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6eb40b4290 0x7f6eb41477b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4019450 tx=0x7f6ea40195e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:05.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eab7fe640 1 -- 172.21.15.105:0/4223369229 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4009450 con 0x7f6eb40b4290 2023-11-28T18:27:05.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eab7fe640 1 -- 172.21.15.105:0/4223369229 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea40ad650 con 0x7f6eb40b4290 2023-11-28T18:27:05.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eb8ed6640 1 -- 172.21.15.105:0/4223369229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb40b22e0 con 0x7f6eb40b4290 2023-11-28T18:27:05.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eab7fe640 1 -- 172.21.15.105:0/4223369229 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea40ad9c0 con 0x7f6eb40b4290 2023-11-28T18:27:05.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eb8ed6640 1 -- 172.21.15.105:0/4223369229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb40aee20 con 0x7f6eb40b4290 2023-11-28T18:27:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.283+0000 7f6eb8ed6640 1 -- 172.21.15.105:0/4223369229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb4067570 con 0x7f6eb40b4290 2023-11-28T18:27:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.287+0000 7f6eab7fe640 1 -- 172.21.15.105:0/4223369229 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f6ea40adb20 con 0x7f6eb40b4290 2023-11-28T18:27:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.287+0000 7f6eab7fe640 1 --2- 172.21.15.105:0/4223369229 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6e940668d0 0x7f6e94068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.287+0000 7f6eab7fe640 1 -- 172.21.15.105:0/4223369229 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f6ea411f8f0 con 0x7f6eb40b4290 2023-11-28T18:27:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.287+0000 7f6ea9d9b640 1 --2- 172.21.15.105:0/4223369229 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6e940668d0 0x7f6e94068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.287+0000 7f6ea9d9b640 1 --2- 172.21.15.105:0/4223369229 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6e940668d0 0x7f6e94068d90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c000fc0 tx=0x7f6e9c001380 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.291+0000 7f6eab7fe640 1 -- 172.21.15.105:0/4223369229 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6ea40ecb70 con 0x7f6eb40b4290 2023-11-28T18:27:05.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:05.475+0000 7f6eb8ed6640 1 -- 172.21.15.105:0/4223369229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6eb40cb280 con 0x7f6eb40b4290 2023-11-28T18:27:06.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.103+0000 7f6eab7fe640 1 -- 172.21.15.105:0/4223369229 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v52) v1 ==== 173+0+0 (secure 0 0 0) 0x7f6ea40f0190 con 0x7f6eb40b4290 2023-11-28T18:27:06.109 INFO:teuthology.orchestra.run.smithi105.stderr:new fs with metadata pool 9 and data pool 10 2023-11-28T18:27:06.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.107+0000 7f6eb8ed6640 1 -- 172.21.15.105:0/4223369229 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6e940668d0 msgr2=0x7f6e94068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:06.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.107+0000 7f6eb8ed6640 1 --2- 172.21.15.105:0/4223369229 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6e940668d0 0x7f6e94068d90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c000fc0 tx=0x7f6e9c001380 comp rx=0 tx=0).stop 2023-11-28T18:27:06.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.107+0000 7f6eb8ed6640 1 -- 172.21.15.105:0/4223369229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6eb40b4290 msgr2=0x7f6eb41477b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:06.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.107+0000 7f6eb8ed6640 1 --2- 172.21.15.105:0/4223369229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6eb40b4290 0x7f6eb41477b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4019450 tx=0x7f6ea40195e0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.107+0000 7f6eb8ed6640 1 -- 172.21.15.105:0/4223369229 shutdown_connections 2023-11-28T18:27:06.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.107+0000 7f6eb8ed6640 1 --2- 172.21.15.105:0/4223369229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6eb41f60e0 0x7f6eb409a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.107+0000 7f6eb8ed6640 1 --2- 172.21.15.105:0/4223369229 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6eb415b030 0x7f6eb409a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.107+0000 7f6eb8ed6640 1 --2- 172.21.15.105:0/4223369229 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6e940668d0 0x7f6e94068d90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.107+0000 7f6eb8ed6640 1 --2- 172.21.15.105:0/4223369229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6eb40b4290 0x7f6eb41477b0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.107+0000 7f6eb8ed6640 1 -- 172.21.15.105:0/4223369229 >> 172.21.15.105:0/4223369229 conn(0x7f6eb40ceb90 msgr2=0x7f6eb4147ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:06.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.111+0000 7f6eb8ed6640 1 -- 172.21.15.105:0/4223369229 shutdown_connections 2023-11-28T18:27:06.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.111+0000 7f6eb8ed6640 1 -- 172.21.15.105:0/4223369229 wait complete. 2023-11-28T18:27:06.151 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:27:06.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.283+0000 7fab30c5a640 1 Processor -- start 2023-11-28T18:27:06.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.287+0000 7fab30c5a640 1 -- start start 2023-11-28T18:27:06.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.287+0000 7fab30c5a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fab2c0cc760 0x7fab2c164d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:06.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.287+0000 7fab30c5a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab2c0ccc10 con 0x7fab2c0caf60 2023-11-28T18:27:06.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.287+0000 7fab30c5a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab2c06bf50 con 0x7fab2c159f40 2023-11-28T18:27:06.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.287+0000 7fab30c5a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab2c0cb3c0 con 0x7fab2c0cc760 2023-11-28T18:27:06.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.287+0000 7fab23fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fab2c0caf60 0x7fab2c0ad850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51270/0 (socket says 172.21.15.105:51270) 2023-11-28T18:27:06.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.287+0000 7fab23fff640 1 -- 172.21.15.105:0/2866774701 learned_addr learned my addr 172.21.15.105:0/2866774701 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:06.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.287+0000 7fab2ad9d640 1 --2- 172.21.15.105:0/2866774701 >> v2:172.21.15.105:3300/0 conn(0x7fab2c0cc760 0x7fab2c164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:06.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.287+0000 7fab29d9b640 1 -- 172.21.15.105:0/2866774701 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2212931080 0 0) 0x7fab2c0ccc10 con 0x7fab2c0caf60 2023-11-28T18:27:06.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.287+0000 7fab2ad9d640 1 -- 172.21.15.105:0/2866774701 >> v1:172.21.15.148:6789/0 conn(0x7fab2c159f40 legacy=0x7fab2c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:06.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.287+0000 7fab2ad9d640 1 -- 172.21.15.105:0/2866774701 >> v1:172.21.15.105:6789/0 conn(0x7fab2c0caf60 legacy=0x7fab2c0ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:06.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.287+0000 7fab2ad9d640 1 -- 172.21.15.105:0/2866774701 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c15a320 con 0x7fab2c0cc760 2023-11-28T18:27:06.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.291+0000 7fab2ad9d640 1 --2- 172.21.15.105:0/2866774701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab2c0cc760 0x7fab2c164d90 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fab24009d90 tx=0x7fab2401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3e4f51189b15a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:06.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.291+0000 7fab29d9b640 1 -- 172.21.15.105:0/2866774701 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab240a5950 con 0x7fab2c0cc760 2023-11-28T18:27:06.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.291+0000 7fab29d9b640 1 -- 172.21.15.105:0/2866774701 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab240a5ab0 con 0x7fab2c0cc760 2023-11-28T18:27:06.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.291+0000 7fab29d9b640 1 -- 172.21.15.105:0/2866774701 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab240a5e40 con 0x7fab2c0cc760 2023-11-28T18:27:06.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.291+0000 7fab30c5a640 1 -- 172.21.15.105:0/2866774701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab2c0cc760 msgr2=0x7fab2c164d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:06.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.291+0000 7fab30c5a640 1 --2- 172.21.15.105:0/2866774701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab2c0cc760 0x7fab2c164d90 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fab24009d90 tx=0x7fab2401c890 comp rx=0 tx=0).stop 2023-11-28T18:27:06.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.291+0000 7fab30c5a640 1 -- 172.21.15.105:0/2866774701 shutdown_connections 2023-11-28T18:27:06.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.291+0000 7fab30c5a640 1 --2- 172.21.15.105:0/2866774701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab2c0cc760 0x7fab2c164d90 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.291+0000 7fab30c5a640 1 -- 172.21.15.105:0/2866774701 >> 172.21.15.105:0/2866774701 conn(0x7fab2c0ceb90 msgr2=0x7fab2c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:06.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.291+0000 7fab30c5a640 1 -- 172.21.15.105:0/2866774701 shutdown_connections 2023-11-28T18:27:06.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.291+0000 7fab30c5a640 1 -- 172.21.15.105:0/2866774701 wait complete. 2023-11-28T18:27:06.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.291+0000 7fab30c5a640 1 Processor -- start 2023-11-28T18:27:06.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab30c5a640 1 -- start start 2023-11-28T18:27:06.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab30c5a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab2c0caf60 0x7fab2c15bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:06.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab30c5a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab2c0cc760 0x7fab2c147d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:06.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab30c5a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab2c159f40 0x7fab2c158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:06.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab30c5a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c15c0b0 con 0x7fab2c159f40 2023-11-28T18:27:06.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab30c5a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c148290 con 0x7fab2c0caf60 2023-11-28T18:27:06.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab30c5a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c154160 con 0x7fab2c0cc760 2023-11-28T18:27:06.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab2ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab2c159f40 0x7fab2c158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:06.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab2ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab2c159f40 0x7fab2c158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44172/0 (socket says 172.21.15.105:44172) 2023-11-28T18:27:06.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab2ad9d640 1 -- 172.21.15.105:0/639985565 learned_addr learned my addr 172.21.15.105:0/639985565 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:06.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab2a59c640 1 --2- 172.21.15.105:0/639985565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab2c0caf60 0x7fab2c15bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:06.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab23fff640 1 --2- 172.21.15.105:0/639985565 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab2c0cc760 0x7fab2c147d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:06.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab2ad9d640 1 -- 172.21.15.105:0/639985565 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab2c0cc760 msgr2=0x7fab2c147d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:06.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab2ad9d640 1 --2- 172.21.15.105:0/639985565 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab2c0cc760 0x7fab2c147d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab2ad9d640 1 -- 172.21.15.105:0/639985565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab2c0caf60 msgr2=0x7fab2c15bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:06.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab2ad9d640 1 --2- 172.21.15.105:0/639985565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab2c0caf60 0x7fab2c15bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab2ad9d640 1 -- 172.21.15.105:0/639985565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab24009a70 con 0x7fab2c159f40 2023-11-28T18:27:06.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab23fff640 1 --2- 172.21.15.105:0/639985565 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab2c0cc760 0x7fab2c147d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:27:06.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab2ad9d640 1 --2- 172.21.15.105:0/639985565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab2c159f40 0x7fab2c158590 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fab240b0040 tx=0x7fab24019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:06.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab22ffd640 1 -- 172.21.15.105:0/639985565 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab240ade40 con 0x7fab2c159f40 2023-11-28T18:27:06.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab22ffd640 1 -- 172.21.15.105:0/639985565 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab240b9450 con 0x7fab2c159f40 2023-11-28T18:27:06.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab30c5a640 1 -- 172.21.15.105:0/639985565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2c154440 con 0x7fab2c159f40 2023-11-28T18:27:06.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab22ffd640 1 -- 172.21.15.105:0/639985565 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab240b9800 con 0x7fab2c159f40 2023-11-28T18:27:06.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab30c5a640 1 -- 172.21.15.105:0/639985565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2c154a00 con 0x7fab2c159f40 2023-11-28T18:27:06.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.295+0000 7fab30c5a640 1 -- 172.21.15.105:0/639985565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2c067570 con 0x7fab2c159f40 2023-11-28T18:27:06.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.299+0000 7fab22ffd640 1 -- 172.21.15.105:0/639985565 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fab240b9960 con 0x7fab2c159f40 2023-11-28T18:27:06.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.299+0000 7fab22ffd640 1 --2- 172.21.15.105:0/639985565 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fab0c066900 0x7fab0c068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:06.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.299+0000 7fab22ffd640 1 -- 172.21.15.105:0/639985565 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fab2411fe20 con 0x7fab2c159f40 2023-11-28T18:27:06.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.299+0000 7fab2a59c640 1 --2- 172.21.15.105:0/639985565 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fab0c066900 0x7fab0c068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:06.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.299+0000 7fab2a59c640 1 --2- 172.21.15.105:0/639985565 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fab0c066900 0x7fab0c068dc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fab1c000f70 tx=0x7fab1c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:06.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.303+0000 7fab22ffd640 1 -- 172.21.15.105:0/639985565 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fab240ed070 con 0x7fab2c159f40 2023-11-28T18:27:06.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.455+0000 7fab30c5a640 1 -- 172.21.15.105:0/639985565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab2c0cf950 con 0x7fab2c159f40 2023-11-28T18:27:06.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.455+0000 7fab22ffd640 1 -- 172.21.15.105:0/639985565 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16789 (secure 0 0 0) 0x7fab240f0690 con 0x7fab2c159f40 2023-11-28T18:27:06.459 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:27:06.459 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":64,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:27:05.479498+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:27:03.225100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T18:27:04.478619+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:27:06.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.455+0000 7fab30c5a640 1 -- 172.21.15.105:0/639985565 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fab0c066900 msgr2=0x7fab0c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:06.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.455+0000 7fab30c5a640 1 --2- 172.21.15.105:0/639985565 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fab0c066900 0x7fab0c068dc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fab1c000f70 tx=0x7fab1c009040 comp rx=0 tx=0).stop 2023-11-28T18:27:06.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.455+0000 7fab30c5a640 1 -- 172.21.15.105:0/639985565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab2c159f40 msgr2=0x7fab2c158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:06.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.455+0000 7fab30c5a640 1 --2- 172.21.15.105:0/639985565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab2c159f40 0x7fab2c158590 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fab240b0040 tx=0x7fab24019630 comp rx=0 tx=0).stop 2023-11-28T18:27:06.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.455+0000 7fab30c5a640 1 -- 172.21.15.105:0/639985565 shutdown_connections 2023-11-28T18:27:06.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.455+0000 7fab30c5a640 1 --2- 172.21.15.105:0/639985565 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fab0c066900 0x7fab0c068dc0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.455+0000 7fab30c5a640 1 --2- 172.21.15.105:0/639985565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab2c159f40 0x7fab2c158590 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.455+0000 7fab30c5a640 1 --2- 172.21.15.105:0/639985565 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab2c0cc760 0x7fab2c147d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.455+0000 7fab30c5a640 1 --2- 172.21.15.105:0/639985565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab2c0caf60 0x7fab2c15bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.455+0000 7fab30c5a640 1 -- 172.21.15.105:0/639985565 >> 172.21.15.105:0/639985565 conn(0x7fab2c0ceb90 msgr2=0x7fab2c0ad140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:06.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.455+0000 7fab30c5a640 1 -- 172.21.15.105:0/639985565 shutdown_connections 2023-11-28T18:27:06.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.455+0000 7fab30c5a640 1 -- 172.21.15.105:0/639985565 wait complete. 2023-11-28T18:27:06.480 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:27:06.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.611+0000 7fcb97017640 1 Processor -- start 2023-11-28T18:27:06.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.611+0000 7fcb97017640 1 -- start start 2023-11-28T18:27:06.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.615+0000 7fcb97017640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb90159f40 0x7fcb900caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:06.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.615+0000 7fcb97017640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcb900ad850 0x7fcb900701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:06.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.615+0000 7fcb97017640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb9006bf50 con 0x7fcb900cc760 2023-11-28T18:27:06.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.615+0000 7fcb97017640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb900ccbc0 con 0x7fcb900ad850 2023-11-28T18:27:06.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.615+0000 7fcb97017640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9015a320 con 0x7fcb90159f40 2023-11-28T18:27:06.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.615+0000 7fcb87fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcb900cc760 0x7fcb900ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:35900/0 (socket says 172.21.15.105:35900) 2023-11-28T18:27:06.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.615+0000 7fcb87fff640 1 -- 172.21.15.105:0/2638305279 learned_addr learned my addr 172.21.15.105:0/2638305279 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:06.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.615+0000 7fcb955b4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcb900ad850 0x7fcb900701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:06.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.615+0000 7fcb94db3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb90159f40 0x7fcb900caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:06.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.615+0000 7fcb877fe640 1 -- 172.21.15.105:0/2638305279 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3705251223 0 0) 0x7fcb9006bf50 con 0x7fcb900cc760 2023-11-28T18:27:06.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.615+0000 7fcb877fe640 1 -- 172.21.15.105:0/2638305279 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb78003530 con 0x7fcb900cc760 2023-11-28T18:27:06.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.615+0000 7fcb955b4640 1 -- 172.21.15.105:0/2638305279 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb90159f40 msgr2=0x7fcb900caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:06.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.615+0000 7fcb955b4640 1 --2- 172.21.15.105:0/2638305279 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb90159f40 0x7fcb900caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.615+0000 7fcb955b4640 1 -- 172.21.15.105:0/2638305279 >> v1:172.21.15.148:6789/0 conn(0x7fcb900cc760 legacy=0x7fcb900ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:06.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.615+0000 7fcb955b4640 1 -- 172.21.15.105:0/2638305279 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb90071380 con 0x7fcb900ad850 2023-11-28T18:27:06.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.615+0000 7fcb94db3640 1 --2- 172.21.15.105:0/2638305279 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb90159f40 0x7fcb900caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:27:06.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.619+0000 7fcb955b4640 1 --2- 172.21.15.105:0/2638305279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb900ad850 0x7fcb900701d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009d90 tx=0x7fcb8801c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7a4a0f0d3f4a23a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:06.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.619+0000 7fcb877fe640 1 -- 172.21.15.105:0/2638305279 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb880a5950 con 0x7fcb900ad850 2023-11-28T18:27:06.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.619+0000 7fcb877fe640 1 -- 172.21.15.105:0/2638305279 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb880a5ab0 con 0x7fcb900ad850 2023-11-28T18:27:06.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.619+0000 7fcb877fe640 1 -- 172.21.15.105:0/2638305279 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb880a5e40 con 0x7fcb900ad850 2023-11-28T18:27:06.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.619+0000 7fcb97017640 1 -- 172.21.15.105:0/2638305279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb900ad850 msgr2=0x7fcb900701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:06.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.619+0000 7fcb97017640 1 --2- 172.21.15.105:0/2638305279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb900ad850 0x7fcb900701d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009d90 tx=0x7fcb8801c890 comp rx=0 tx=0).stop 2023-11-28T18:27:06.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.619+0000 7fcb97017640 1 -- 172.21.15.105:0/2638305279 shutdown_connections 2023-11-28T18:27:06.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.619+0000 7fcb97017640 1 --2- 172.21.15.105:0/2638305279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb900ad850 0x7fcb900701d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.619+0000 7fcb97017640 1 --2- 172.21.15.105:0/2638305279 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb90159f40 0x7fcb900caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.619+0000 7fcb97017640 1 -- 172.21.15.105:0/2638305279 >> 172.21.15.105:0/2638305279 conn(0x7fcb900ceb90 msgr2=0x7fcb900cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:06.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.619+0000 7fcb97017640 1 -- 172.21.15.105:0/2638305279 shutdown_connections 2023-11-28T18:27:06.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.619+0000 7fcb97017640 1 -- 172.21.15.105:0/2638305279 wait complete. 2023-11-28T18:27:06.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.619+0000 7fcb97017640 1 Processor -- start 2023-11-28T18:27:06.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb97017640 1 -- start start 2023-11-28T18:27:06.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb97017640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb900ad850 0x7fcb9015cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:06.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb97017640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb900cc760 0x7fcb9009c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:06.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb97017640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb90159f40 0x7fcb9014e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:06.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb97017640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9015d3c0 con 0x7fcb90159f40 2023-11-28T18:27:06.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb97017640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9014b900 con 0x7fcb900ad850 2023-11-28T18:27:06.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb97017640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9015bb50 con 0x7fcb900cc760 2023-11-28T18:27:06.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb955b4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb90159f40 0x7fcb9014e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:06.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb955b4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb90159f40 0x7fcb9014e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44178/0 (socket says 172.21.15.105:44178) 2023-11-28T18:27:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb955b4640 1 -- 172.21.15.105:0/2406977134 learned_addr learned my addr 172.21.15.105:0/2406977134 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb87fff640 1 --2- 172.21.15.105:0/2406977134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb900cc760 0x7fcb9009c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb94db3640 1 --2- 172.21.15.105:0/2406977134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb900ad850 0x7fcb9015cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb955b4640 1 -- 172.21.15.105:0/2406977134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb900cc760 msgr2=0x7fcb9009c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb955b4640 1 --2- 172.21.15.105:0/2406977134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb900cc760 0x7fcb9009c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb955b4640 1 -- 172.21.15.105:0/2406977134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb900ad850 msgr2=0x7fcb9015cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb955b4640 1 --2- 172.21.15.105:0/2406977134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb900ad850 0x7fcb9015cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb955b4640 1 -- 172.21.15.105:0/2406977134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c009420 con 0x7fcb90159f40 2023-11-28T18:27:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb94db3640 1 --2- 172.21.15.105:0/2406977134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb900ad850 0x7fcb9015cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb955b4640 1 --2- 172.21.15.105:0/2406977134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb90159f40 0x7fcb9014e8a0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fcb880b0040 tx=0x7fcb88019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:06.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb857fa640 1 -- 172.21.15.105:0/2406977134 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb880adce0 con 0x7fcb90159f40 2023-11-28T18:27:06.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb857fa640 1 -- 172.21.15.105:0/2406977134 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb880ade40 con 0x7fcb90159f40 2023-11-28T18:27:06.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb857fa640 1 -- 172.21.15.105:0/2406977134 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb880b9690 con 0x7fcb90159f40 2023-11-28T18:27:06.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb97017640 1 -- 172.21.15.105:0/2406977134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb88009a70 con 0x7fcb90159f40 2023-11-28T18:27:06.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb97017640 1 -- 172.21.15.105:0/2406977134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb900b1a50 con 0x7fcb90159f40 2023-11-28T18:27:06.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.623+0000 7fcb97017640 1 -- 172.21.15.105:0/2406977134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb900b1c40 con 0x7fcb90159f40 2023-11-28T18:27:06.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.627+0000 7fcb857fa640 1 -- 172.21.15.105:0/2406977134 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fcb880b97f0 con 0x7fcb90159f40 2023-11-28T18:27:06.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.627+0000 7fcb857fa640 1 --2- 172.21.15.105:0/2406977134 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcb7c0668d0 0x7fcb7c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:06.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.627+0000 7fcb857fa640 1 -- 172.21.15.105:0/2406977134 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcb8811fc30 con 0x7fcb90159f40 2023-11-28T18:27:06.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.627+0000 7fcb94db3640 1 --2- 172.21.15.105:0/2406977134 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcb7c0668d0 0x7fcb7c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:06.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.627+0000 7fcb94db3640 1 --2- 172.21.15.105:0/2406977134 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcb7c0668d0 0x7fcb7c068d90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c009ca0 tx=0x7fcb8c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:06.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.631+0000 7fcb857fa640 1 -- 172.21.15.105:0/2406977134 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcb880ece40 con 0x7fcb90159f40 2023-11-28T18:27:06.785 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.779+0000 7fcb97017640 1 -- 172.21.15.105:0/2406977134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb90067570 con 0x7fcb90159f40 2023-11-28T18:27:06.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.783+0000 7fcb857fa640 1 -- 172.21.15.105:0/2406977134 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16789 (secure 0 0 0) 0x7fcb88009450 con 0x7fcb90159f40 2023-11-28T18:27:06.786 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:27:06.787 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":64,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:27:05.479498+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:27:03.225100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T18:27:04.478619+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:27:06.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.783+0000 7fcb97017640 1 -- 172.21.15.105:0/2406977134 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcb7c0668d0 msgr2=0x7fcb7c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:06.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.783+0000 7fcb97017640 1 --2- 172.21.15.105:0/2406977134 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcb7c0668d0 0x7fcb7c068d90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c009ca0 tx=0x7fcb8c008040 comp rx=0 tx=0).stop 2023-11-28T18:27:06.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.783+0000 7fcb97017640 1 -- 172.21.15.105:0/2406977134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb90159f40 msgr2=0x7fcb9014e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:06.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.783+0000 7fcb97017640 1 --2- 172.21.15.105:0/2406977134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb90159f40 0x7fcb9014e8a0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fcb880b0040 tx=0x7fcb88019630 comp rx=0 tx=0).stop 2023-11-28T18:27:06.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.783+0000 7fcb97017640 1 -- 172.21.15.105:0/2406977134 shutdown_connections 2023-11-28T18:27:06.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.783+0000 7fcb97017640 1 --2- 172.21.15.105:0/2406977134 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcb7c0668d0 0x7fcb7c068d90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.783+0000 7fcb97017640 1 --2- 172.21.15.105:0/2406977134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb90159f40 0x7fcb9014e8a0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.783+0000 7fcb97017640 1 --2- 172.21.15.105:0/2406977134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb900cc760 0x7fcb9009c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.783+0000 7fcb97017640 1 --2- 172.21.15.105:0/2406977134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb900ad850 0x7fcb9015cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.783+0000 7fcb97017640 1 -- 172.21.15.105:0/2406977134 >> 172.21.15.105:0/2406977134 conn(0x7fcb900ceb90 msgr2=0x7fcb900af340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:06.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.783+0000 7fcb97017640 1 -- 172.21.15.105:0/2406977134 shutdown_connections 2023-11-28T18:27:06.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.783+0000 7fcb97017640 1 -- 172.21.15.105:0/2406977134 wait complete. 2023-11-28T18:27:06.808 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T18:27:06.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.943+0000 7f5653232640 1 Processor -- start 2023-11-28T18:27:06.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.943+0000 7f5653232640 1 -- start start 2023-11-28T18:27:06.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.943+0000 7f5653232640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f564c0caf60 0x7f564c0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:06.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.943+0000 7f5653232640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f564c0ad850 0x7f564c0701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:06.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.947+0000 7f5653232640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f564c06bf50 con 0x7f564c159f40 2023-11-28T18:27:06.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.947+0000 7f5653232640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f564c0cb3c0 con 0x7f564c0caf60 2023-11-28T18:27:06.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.947+0000 7f5653232640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f564c15a320 con 0x7f564c0ad850 2023-11-28T18:27:06.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.947+0000 7f5650fce640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f564c159f40 0x7f564c0ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51272/0 (socket says 172.21.15.105:51272) 2023-11-28T18:27:06.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.947+0000 7f5650fce640 1 -- 172.21.15.105:0/2592307429 learned_addr learned my addr 172.21.15.105:0/2592307429 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.947+0000 7f5643fff640 1 --2- 172.21.15.105:0/2592307429 >> v2:172.21.15.105:3300/0 conn(0x7f564c0caf60 0x7f564c0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.947+0000 7f56517cf640 1 --2- 172.21.15.105:0/2592307429 >> v2:172.21.15.148:3300/0 conn(0x7f564c0ad850 0x7f564c0701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.947+0000 7f5643fff640 1 -- 172.21.15.105:0/2592307429 >> v2:172.21.15.148:3300/0 conn(0x7f564c0ad850 msgr2=0x7f564c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.947+0000 7f5643fff640 1 --2- 172.21.15.105:0/2592307429 >> v2:172.21.15.148:3300/0 conn(0x7f564c0ad850 0x7f564c0701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.947+0000 7f5643fff640 1 -- 172.21.15.105:0/2592307429 >> v1:172.21.15.105:6789/0 conn(0x7f564c159f40 legacy=0x7f564c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:06.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.947+0000 7f5643fff640 1 -- 172.21.15.105:0/2592307429 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564c071380 con 0x7f564c0caf60 2023-11-28T18:27:06.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.947+0000 7f5643fff640 1 --2- 172.21.15.105:0/2592307429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f564c0caf60 0x7f564c0cc760 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f564c0b0850 tx=0x7f563c01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd53cbb1dbaa2ad server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.947+0000 7f56437fe640 1 -- 172.21.15.105:0/2592307429 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c0ac600 con 0x7f564c0caf60 2023-11-28T18:27:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.947+0000 7f56437fe640 1 -- 172.21.15.105:0/2592307429 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563c0a8050 con 0x7f564c0caf60 2023-11-28T18:27:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.947+0000 7f56437fe640 1 -- 172.21.15.105:0/2592307429 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c0ac980 con 0x7f564c0caf60 2023-11-28T18:27:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.951+0000 7f5653232640 1 -- 172.21.15.105:0/2592307429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f564c0caf60 msgr2=0x7f564c0cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.951+0000 7f5653232640 1 --2- 172.21.15.105:0/2592307429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f564c0caf60 0x7f564c0cc760 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f564c0b0850 tx=0x7f563c01c3a0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.951+0000 7f5653232640 1 -- 172.21.15.105:0/2592307429 shutdown_connections 2023-11-28T18:27:06.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.951+0000 7f5653232640 1 --2- 172.21.15.105:0/2592307429 >> v2:172.21.15.148:3300/0 conn(0x7f564c0ad850 0x7f564c0701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.951+0000 7f5653232640 1 --2- 172.21.15.105:0/2592307429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f564c0caf60 0x7f564c0cc760 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.951+0000 7f5653232640 1 -- 172.21.15.105:0/2592307429 >> 172.21.15.105:0/2592307429 conn(0x7f564c0ceb90 msgr2=0x7f564c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:06.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.951+0000 7f5653232640 1 -- 172.21.15.105:0/2592307429 shutdown_connections 2023-11-28T18:27:06.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.951+0000 7f5653232640 1 -- 172.21.15.105:0/2592307429 wait complete. 2023-11-28T18:27:06.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.951+0000 7f5653232640 1 Processor -- start 2023-11-28T18:27:06.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.951+0000 7f5653232640 1 -- start start 2023-11-28T18:27:06.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.951+0000 7f5653232640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f564c0ad850 0x7f564c14b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:06.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.951+0000 7f5653232640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f564c0caf60 0x7f564c0c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:06.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.951+0000 7f5653232640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f564c159f40 0x7f564c0c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:06.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.951+0000 7f5653232640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c151280 con 0x7f564c0ad850 2023-11-28T18:27:06.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.951+0000 7f5653232640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c1505e0 con 0x7f564c0caf60 2023-11-28T18:27:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.951+0000 7f5653232640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564c150950 con 0x7f564c159f40 2023-11-28T18:27:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.951+0000 7f5650fce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f564c0ad850 0x7f564c14b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.951+0000 7f5650fce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f564c0ad850 0x7f564c14b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44184/0 (socket says 172.21.15.105:44184) 2023-11-28T18:27:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.951+0000 7f5650fce640 1 -- 172.21.15.105:0/3450818490 learned_addr learned my addr 172.21.15.105:0/3450818490 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.951+0000 7f5643fff640 1 --2- 172.21.15.105:0/3450818490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f564c0caf60 0x7f564c0c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.951+0000 7f56517cf640 1 --2- 172.21.15.105:0/3450818490 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f564c159f40 0x7f564c0c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.955+0000 7f5650fce640 1 -- 172.21.15.105:0/3450818490 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f564c159f40 msgr2=0x7f564c0c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.955+0000 7f5650fce640 1 --2- 172.21.15.105:0/3450818490 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f564c159f40 0x7f564c0c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.955+0000 7f5650fce640 1 -- 172.21.15.105:0/3450818490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f564c0caf60 msgr2=0x7f564c0c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.955+0000 7f5650fce640 1 --2- 172.21.15.105:0/3450818490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f564c0caf60 0x7f564c0c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.955+0000 7f5650fce640 1 -- 172.21.15.105:0/3450818490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c009420 con 0x7f564c0ad850 2023-11-28T18:27:06.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.955+0000 7f5650fce640 1 --2- 172.21.15.105:0/3450818490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f564c0ad850 0x7f564c14b330 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f564800c400 tx=0x7f564800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:06.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.955+0000 7f56417fa640 1 -- 172.21.15.105:0/3450818490 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5648005cf0 con 0x7f564c0ad850 2023-11-28T18:27:06.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.955+0000 7f56417fa640 1 -- 172.21.15.105:0/3450818490 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5648005e50 con 0x7f564c0ad850 2023-11-28T18:27:06.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.955+0000 7f56417fa640 1 -- 172.21.15.105:0/3450818490 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56480072d0 con 0x7f564c0ad850 2023-11-28T18:27:06.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.955+0000 7f5653232640 1 -- 172.21.15.105:0/3450818490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f564c150c30 con 0x7f564c0ad850 2023-11-28T18:27:06.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.955+0000 7f5653232640 1 -- 172.21.15.105:0/3450818490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f564c14b870 con 0x7f564c0ad850 2023-11-28T18:27:06.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.955+0000 7f5653232640 1 -- 172.21.15.105:0/3450818490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f564c067570 con 0x7f564c0ad850 2023-11-28T18:27:06.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.955+0000 7f56417fa640 1 -- 172.21.15.105:0/3450818490 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5648007430 con 0x7f564c0ad850 2023-11-28T18:27:06.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.955+0000 7f56417fa640 1 --2- 172.21.15.105:0/3450818490 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56380668d0 0x7f5638068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:06.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.955+0000 7f56417fa640 1 -- 172.21.15.105:0/3450818490 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5648083a50 con 0x7f564c0ad850 2023-11-28T18:27:06.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.955+0000 7f5643fff640 1 --2- 172.21.15.105:0/3450818490 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56380668d0 0x7f5638068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:06.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.959+0000 7f5643fff640 1 --2- 172.21.15.105:0/3450818490 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56380668d0 0x7f5638068d90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f563c0ae000 tx=0x7f563c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:06.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:06.959+0000 7f56417fa640 1 -- 172.21.15.105:0/3450818490 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5648050bf0 con 0x7f564c0ad850 2023-11-28T18:27:07.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:07.143+0000 7f5653232640 1 -- 172.21.15.105:0/3450818490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f564c0cf950 con 0x7f564c0ad850 2023-11-28T18:27:07.451 INFO:tasks.ceph.mds.c.smithi105.stderr:2023-11-28T18:27:07.451+0000 7f5589075640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T18:27:08.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.135+0000 7f56417fa640 1 -- 172.21.15.105:0/3450818490 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v55) v1 ==== 118+0+0 (secure 0 0 0) 0x7f56480170b0 con 0x7f564c0ad850 2023-11-28T18:27:08.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.139+0000 7f5653232640 1 -- 172.21.15.105:0/3450818490 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56380668d0 msgr2=0x7f5638068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:08.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.139+0000 7f5653232640 1 --2- 172.21.15.105:0/3450818490 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56380668d0 0x7f5638068d90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f563c0ae000 tx=0x7f563c008040 comp rx=0 tx=0).stop 2023-11-28T18:27:08.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.139+0000 7f5653232640 1 -- 172.21.15.105:0/3450818490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f564c0ad850 msgr2=0x7f564c14b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:08.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.139+0000 7f5653232640 1 --2- 172.21.15.105:0/3450818490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f564c0ad850 0x7f564c14b330 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f564800c400 tx=0x7f564800c8c0 comp rx=0 tx=0).stop 2023-11-28T18:27:08.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.139+0000 7f5653232640 1 -- 172.21.15.105:0/3450818490 shutdown_connections 2023-11-28T18:27:08.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.139+0000 7f5653232640 1 --2- 172.21.15.105:0/3450818490 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f564c159f40 0x7f564c0c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:08.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.139+0000 7f5653232640 1 --2- 172.21.15.105:0/3450818490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f564c0caf60 0x7f564c0c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:08.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.139+0000 7f5653232640 1 --2- 172.21.15.105:0/3450818490 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56380668d0 0x7f5638068d90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:08.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.139+0000 7f5653232640 1 --2- 172.21.15.105:0/3450818490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f564c0ad850 0x7f564c14b330 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:08.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.139+0000 7f5653232640 1 -- 172.21.15.105:0/3450818490 >> 172.21.15.105:0/3450818490 conn(0x7f564c0ceb90 msgr2=0x7f564c0af030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:08.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.139+0000 7f5653232640 1 -- 172.21.15.105:0/3450818490 shutdown_connections 2023-11-28T18:27:08.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.139+0000 7f5653232640 1 -- 172.21.15.105:0/3450818490 wait complete. 2023-11-28T18:27:08.166 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T18:27:08.167 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T18:27:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.307+0000 7fd457238640 1 Processor -- start 2023-11-28T18:27:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.307+0000 7fd457238640 1 -- start start 2023-11-28T18:27:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.311+0000 7fd457238640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd45015d350 0x7fd4500d38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.311+0000 7fd457238640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd4500cfd90 0x7fd4500b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.311+0000 7fd457238640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4500cdae0 con 0x7fd45015d730 2023-11-28T18:27:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.311+0000 7fd457238640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4500d1f40 con 0x7fd4500cfd90 2023-11-28T18:27:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.311+0000 7fd457238640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4500d16a0 con 0x7fd45015d350 2023-11-28T18:27:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.311+0000 7fd456a37640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd4500cfd90 0x7fd4500b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.311+0000 7fd456a37640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd4500cfd90 0x7fd4500b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44200/0 (socket says 172.21.15.105:44200) 2023-11-28T18:27:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.311+0000 7fd456a37640 1 -- 172.21.15.105:0/4038323751 learned_addr learned my addr 172.21.15.105:0/4038323751 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.311+0000 7fd456236640 1 --2- 172.21.15.105:0/4038323751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd45015d350 0x7fd4500d38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.311+0000 7fd456a37640 1 -- 172.21.15.105:0/4038323751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd45015d350 msgr2=0x7fd4500d38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.311+0000 7fd456a37640 1 --2- 172.21.15.105:0/4038323751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd45015d350 0x7fd4500d38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.311+0000 7fd456a37640 1 -- 172.21.15.105:0/4038323751 >> v1:172.21.15.148:6789/0 conn(0x7fd45015d730 legacy=0x7fd4500cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.311+0000 7fd456a37640 1 -- 172.21.15.105:0/4038323751 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4500d1980 con 0x7fd4500cfd90 2023-11-28T18:27:08.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.311+0000 7fd456a37640 1 --2- 172.21.15.105:0/4038323751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4500cfd90 0x7fd4500b1e40 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd448009e50 tx=0x7fd44801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3541efbe266121 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:08.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd455234640 1 -- 172.21.15.105:0/4038323751 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4480a5730 con 0x7fd4500cfd90 2023-11-28T18:27:08.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd455234640 1 -- 172.21.15.105:0/4038323751 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4480a5890 con 0x7fd4500cfd90 2023-11-28T18:27:08.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd455234640 1 -- 172.21.15.105:0/4038323751 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4480a5c20 con 0x7fd4500cfd90 2023-11-28T18:27:08.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd457238640 1 -- 172.21.15.105:0/4038323751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4500cfd90 msgr2=0x7fd4500b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:08.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd457238640 1 --2- 172.21.15.105:0/4038323751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4500cfd90 0x7fd4500b1e40 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd448009e50 tx=0x7fd44801c980 comp rx=0 tx=0).stop 2023-11-28T18:27:08.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd457238640 1 -- 172.21.15.105:0/4038323751 shutdown_connections 2023-11-28T18:27:08.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd457238640 1 --2- 172.21.15.105:0/4038323751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4500cfd90 0x7fd4500b1e40 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:08.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd457238640 1 --2- 172.21.15.105:0/4038323751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd45015d350 0x7fd4500d38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:08.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd457238640 1 -- 172.21.15.105:0/4038323751 >> 172.21.15.105:0/4038323751 conn(0x7fd4500d2d90 msgr2=0x7fd4500d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:08.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd457238640 1 -- 172.21.15.105:0/4038323751 shutdown_connections 2023-11-28T18:27:08.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd457238640 1 -- 172.21.15.105:0/4038323751 wait complete. 2023-11-28T18:27:08.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd457238640 1 Processor -- start 2023-11-28T18:27:08.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd457238640 1 -- start start 2023-11-28T18:27:08.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd457238640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4500cfd90 0x7fd4500b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:08.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd457238640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd45015d350 0x7fd450154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:08.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd457238640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd45015d730 0x7fd450155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:08.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd457238640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd450157240 con 0x7fd45015d730 2023-11-28T18:27:08.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd457238640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4500ca5e0 con 0x7fd4500cfd90 2023-11-28T18:27:08.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd457238640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4500ca8c0 con 0x7fd45015d350 2023-11-28T18:27:08.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd456a37640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd45015d730 0x7fd450155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:08.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd456a37640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd45015d730 0x7fd450155130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44216/0 (socket says 172.21.15.105:44216) 2023-11-28T18:27:08.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd456a37640 1 -- 172.21.15.105:0/1298605229 learned_addr learned my addr 172.21.15.105:0/1298605229 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:08.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.315+0000 7fd455a35640 1 --2- 172.21.15.105:0/1298605229 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd45015d350 0x7fd450154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:08.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.319+0000 7fd456236640 1 --2- 172.21.15.105:0/1298605229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4500cfd90 0x7fd4500b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:08.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.319+0000 7fd456a37640 1 -- 172.21.15.105:0/1298605229 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd45015d350 msgr2=0x7fd450154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:08.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.319+0000 7fd456a37640 1 --2- 172.21.15.105:0/1298605229 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd45015d350 0x7fd450154bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:08.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.319+0000 7fd456a37640 1 -- 172.21.15.105:0/1298605229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4500cfd90 msgr2=0x7fd4500b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:08.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.319+0000 7fd456a37640 1 --2- 172.21.15.105:0/1298605229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4500cfd90 0x7fd4500b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:08.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.319+0000 7fd456a37640 1 -- 172.21.15.105:0/1298605229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd448009b30 con 0x7fd45015d730 2023-11-28T18:27:08.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.319+0000 7fd455a35640 1 --2- 172.21.15.105:0/1298605229 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd45015d350 0x7fd450154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:08.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.319+0000 7fd456236640 1 --2- 172.21.15.105:0/1298605229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4500cfd90 0x7fd4500b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:08.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.319+0000 7fd456a37640 1 --2- 172.21.15.105:0/1298605229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd45015d730 0x7fd450155130 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fd44801a5b0 tx=0x7fd448019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:08.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.319+0000 7fd446ffd640 1 -- 172.21.15.105:0/1298605229 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4480b8670 con 0x7fd45015d730 2023-11-28T18:27:08.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.319+0000 7fd446ffd640 1 -- 172.21.15.105:0/1298605229 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4480b87d0 con 0x7fd45015d730 2023-11-28T18:27:08.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.319+0000 7fd446ffd640 1 -- 172.21.15.105:0/1298605229 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4480b8b60 con 0x7fd45015d730 2023-11-28T18:27:08.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.319+0000 7fd457238640 1 -- 172.21.15.105:0/1298605229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4500caba0 con 0x7fd45015d730 2023-11-28T18:27:08.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.319+0000 7fd457238640 1 -- 172.21.15.105:0/1298605229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd450159200 con 0x7fd45015d730 2023-11-28T18:27:08.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.319+0000 7fd457238640 1 -- 172.21.15.105:0/1298605229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4501573b0 con 0x7fd45015d730 2023-11-28T18:27:08.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.319+0000 7fd446ffd640 1 -- 172.21.15.105:0/1298605229 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd4480b8cc0 con 0x7fd45015d730 2023-11-28T18:27:08.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.319+0000 7fd446ffd640 1 --2- 172.21.15.105:0/1298605229 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd4340668d0 0x7fd434068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:08.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.319+0000 7fd446ffd640 1 -- 172.21.15.105:0/1298605229 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd44811fc70 con 0x7fd45015d730 2023-11-28T18:27:08.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.319+0000 7fd456236640 1 --2- 172.21.15.105:0/1298605229 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd4340668d0 0x7fd434068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:08.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.323+0000 7fd456236640 1 --2- 172.21.15.105:0/1298605229 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd4340668d0 0x7fd434068d90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd450155e20 tx=0x7fd44c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:08.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.327+0000 7fd446ffd640 1 -- 172.21.15.105:0/1298605229 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd4480ecff0 con 0x7fd45015d730 2023-11-28T18:27:08.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:08.507+0000 7fd457238640 1 -- 172.21.15.105:0/1298605229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd450074580 con 0x7fd45015d730 2023-11-28T18:27:09.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.139+0000 7fd446ffd640 1 -- 172.21.15.105:0/1298605229 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v56) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd4480bc0c0 con 0x7fd45015d730 2023-11-28T18:27:09.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.139+0000 7fd457238640 1 -- 172.21.15.105:0/1298605229 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd4340668d0 msgr2=0x7fd434068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:09.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.139+0000 7fd457238640 1 --2- 172.21.15.105:0/1298605229 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd4340668d0 0x7fd434068d90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd450155e20 tx=0x7fd44c008040 comp rx=0 tx=0).stop 2023-11-28T18:27:09.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.139+0000 7fd457238640 1 -- 172.21.15.105:0/1298605229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd45015d730 msgr2=0x7fd450155130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:09.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.139+0000 7fd457238640 1 --2- 172.21.15.105:0/1298605229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd45015d730 0x7fd450155130 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fd44801a5b0 tx=0x7fd448019720 comp rx=0 tx=0).stop 2023-11-28T18:27:09.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.139+0000 7fd457238640 1 -- 172.21.15.105:0/1298605229 shutdown_connections 2023-11-28T18:27:09.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.139+0000 7fd457238640 1 --2- 172.21.15.105:0/1298605229 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd4340668d0 0x7fd434068d90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.139+0000 7fd457238640 1 --2- 172.21.15.105:0/1298605229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd45015d730 0x7fd450155130 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.139+0000 7fd457238640 1 --2- 172.21.15.105:0/1298605229 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd45015d350 0x7fd450154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.139+0000 7fd457238640 1 --2- 172.21.15.105:0/1298605229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4500cfd90 0x7fd4500b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.139+0000 7fd457238640 1 -- 172.21.15.105:0/1298605229 >> 172.21.15.105:0/1298605229 conn(0x7fd4500d2d90 msgr2=0x7fd450159da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:09.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.139+0000 7fd457238640 1 -- 172.21.15.105:0/1298605229 shutdown_connections 2023-11-28T18:27:09.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.139+0000 7fd457238640 1 -- 172.21.15.105:0/1298605229 wait complete. 2023-11-28T18:27:09.164 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T18:27:09.165 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:27:09.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.299+0000 7faa348dc640 1 Processor -- start 2023-11-28T18:27:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.299+0000 7faa348dc640 1 -- start start 2023-11-28T18:27:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.299+0000 7faa348dc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa3015cef0 0x7faa300d1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.299+0000 7faa348dc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7faa300d1b40 0x7faa300cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.299+0000 7faa348dc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa300ce840 con 0x7faa3015d2d0 2023-11-28T18:27:09.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.299+0000 7faa348dc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa300b4780 con 0x7faa300d1b40 2023-11-28T18:27:09.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.299+0000 7faa348dc640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa300cd990 con 0x7faa3015cef0 2023-11-28T18:27:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.303+0000 7faa2effd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7faa300d1b40 0x7faa300cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.303+0000 7faa2f7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa3015cef0 0x7faa300d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.303+0000 7faa2ffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7faa3015d2d0 0x7faa30160c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:35912/0 (socket says 172.21.15.105:35912) 2023-11-28T18:27:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.303+0000 7faa2ffff640 1 -- 172.21.15.105:0/589914167 learned_addr learned my addr 172.21.15.105:0/589914167 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.303+0000 7faa2e7fc640 1 -- 172.21.15.105:0/589914167 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996657413 0 0) 0x7faa300ce840 con 0x7faa3015d2d0 2023-11-28T18:27:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.303+0000 7faa2f7fe640 1 -- 172.21.15.105:0/589914167 >> v2:172.21.15.148:3300/0 conn(0x7faa300d1b40 msgr2=0x7faa300cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.303+0000 7faa2f7fe640 1 --2- 172.21.15.105:0/589914167 >> v2:172.21.15.148:3300/0 conn(0x7faa300d1b40 0x7faa300cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.303+0000 7faa2f7fe640 1 -- 172.21.15.105:0/589914167 >> v1:172.21.15.148:6789/0 conn(0x7faa3015d2d0 legacy=0x7faa30160c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.303+0000 7faa2f7fe640 1 -- 172.21.15.105:0/589914167 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa300cecd0 con 0x7faa3015cef0 2023-11-28T18:27:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.303+0000 7faa2effd640 1 --2- 172.21.15.105:0/589914167 >> v2:172.21.15.148:3300/0 conn(0x7faa300d1b40 0x7faa300cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:27:09.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.303+0000 7faa2f7fe640 1 --2- 172.21.15.105:0/589914167 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa3015cef0 0x7faa300d1600 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faa200097c0 tx=0x7faa2001c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4ef2dc61192439b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:09.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.307+0000 7faa2e7fc640 1 -- 172.21.15.105:0/589914167 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa200ab600 con 0x7faa3015cef0 2023-11-28T18:27:09.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.307+0000 7faa2e7fc640 1 -- 172.21.15.105:0/589914167 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa200a7050 con 0x7faa3015cef0 2023-11-28T18:27:09.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.307+0000 7faa2e7fc640 1 -- 172.21.15.105:0/589914167 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa200ab980 con 0x7faa3015cef0 2023-11-28T18:27:09.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.307+0000 7faa348dc640 1 -- 172.21.15.105:0/589914167 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa3015cef0 msgr2=0x7faa300d1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:09.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.307+0000 7faa348dc640 1 --2- 172.21.15.105:0/589914167 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa3015cef0 0x7faa300d1600 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faa200097c0 tx=0x7faa2001c3c0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.307+0000 7faa348dc640 1 -- 172.21.15.105:0/589914167 shutdown_connections 2023-11-28T18:27:09.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.307+0000 7faa348dc640 1 --2- 172.21.15.105:0/589914167 >> v2:172.21.15.148:3300/0 conn(0x7faa300d1b40 0x7faa300cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.307+0000 7faa348dc640 1 --2- 172.21.15.105:0/589914167 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa3015cef0 0x7faa300d1600 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.307+0000 7faa348dc640 1 -- 172.21.15.105:0/589914167 >> 172.21.15.105:0/589914167 conn(0x7faa300d2e10 msgr2=0x7faa300d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:09.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.307+0000 7faa348dc640 1 -- 172.21.15.105:0/589914167 shutdown_connections 2023-11-28T18:27:09.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.307+0000 7faa348dc640 1 -- 172.21.15.105:0/589914167 wait complete. 2023-11-28T18:27:09.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.307+0000 7faa348dc640 1 Processor -- start 2023-11-28T18:27:09.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.307+0000 7faa348dc640 1 -- start start 2023-11-28T18:27:09.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.307+0000 7faa348dc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa300d1b40 0x7faa30159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:09.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.307+0000 7faa348dc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa3015cef0 0x7faa30158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:09.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.307+0000 7faa348dc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa3015d2d0 0x7faa300b81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:09.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.307+0000 7faa348dc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa30158b00 con 0x7faa3015d2d0 2023-11-28T18:27:09.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.307+0000 7faa348dc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa300cac10 con 0x7faa3015cef0 2023-11-28T18:27:09.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.307+0000 7faa348dc640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa30158d50 con 0x7faa300d1b40 2023-11-28T18:27:09.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.311+0000 7faa2ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa3015d2d0 0x7faa300b81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:09.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.311+0000 7faa2ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa3015d2d0 0x7faa300b81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44226/0 (socket says 172.21.15.105:44226) 2023-11-28T18:27:09.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.311+0000 7faa2ffff640 1 -- 172.21.15.105:0/3628855894 learned_addr learned my addr 172.21.15.105:0/3628855894 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:09.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.311+0000 7faa2f7fe640 1 --2- 172.21.15.105:0/3628855894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa300d1b40 0x7faa30159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:09.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.311+0000 7faa2effd640 1 --2- 172.21.15.105:0/3628855894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa3015cef0 0x7faa30158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:09.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.311+0000 7faa2ffff640 1 -- 172.21.15.105:0/3628855894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa300d1b40 msgr2=0x7faa30159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:09.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.311+0000 7faa2ffff640 1 --2- 172.21.15.105:0/3628855894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa300d1b40 0x7faa30159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.311+0000 7faa2ffff640 1 -- 172.21.15.105:0/3628855894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa3015cef0 msgr2=0x7faa30158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:09.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.311+0000 7faa2ffff640 1 --2- 172.21.15.105:0/3628855894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa3015cef0 0x7faa30158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.311+0000 7faa2ffff640 1 -- 172.21.15.105:0/3628855894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa24009420 con 0x7faa3015d2d0 2023-11-28T18:27:09.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.311+0000 7faa2ffff640 1 --2- 172.21.15.105:0/3628855894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa3015d2d0 0x7faa300b81a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7faa2800c780 tx=0x7faa2800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:09.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.311+0000 7faa17fff640 1 -- 172.21.15.105:0/3628855894 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2800ce90 con 0x7faa3015d2d0 2023-11-28T18:27:09.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.311+0000 7faa17fff640 1 -- 172.21.15.105:0/3628855894 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2800a640 con 0x7faa3015d2d0 2023-11-28T18:27:09.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.311+0000 7faa348dc640 1 -- 172.21.15.105:0/3628855894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa20009420 con 0x7faa3015d2d0 2023-11-28T18:27:09.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.311+0000 7faa17fff640 1 -- 172.21.15.105:0/3628855894 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2800a9d0 con 0x7faa3015d2d0 2023-11-28T18:27:09.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.311+0000 7faa348dc640 1 -- 172.21.15.105:0/3628855894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa30159310 con 0x7faa3015d2d0 2023-11-28T18:27:09.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.311+0000 7faa348dc640 1 -- 172.21.15.105:0/3628855894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa30159530 con 0x7faa3015d2d0 2023-11-28T18:27:09.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.311+0000 7faa17fff640 1 -- 172.21.15.105:0/3628855894 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7faa2800ab30 con 0x7faa3015d2d0 2023-11-28T18:27:09.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.315+0000 7faa17fff640 1 --2- 172.21.15.105:0/3628855894 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faa1c0668d0 0x7faa1c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:09.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.315+0000 7faa17fff640 1 -- 172.21.15.105:0/3628855894 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7faa28083d50 con 0x7faa3015d2d0 2023-11-28T18:27:09.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.315+0000 7faa2f7fe640 1 --2- 172.21.15.105:0/3628855894 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faa1c0668d0 0x7faa1c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:09.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.315+0000 7faa2f7fe640 1 --2- 172.21.15.105:0/3628855894 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faa1c0668d0 0x7faa1c068d90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7faa200ad000 tx=0x7faa200a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:09.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.315+0000 7faa17fff640 1 -- 172.21.15.105:0/3628855894 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faa28087030 con 0x7faa3015d2d0 2023-11-28T18:27:09.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.495+0000 7faa348dc640 1 -- 172.21.15.105:0/3628855894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa300d3690 con 0x7faa3015d2d0 2023-11-28T18:27:09.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.495+0000 7faa17fff640 1 -- 172.21.15.105:0/3628855894 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4920 (secure 0 0 0) 0x7faa28018090 con 0x7faa3015d2d0 2023-11-28T18:27:09.501 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:27:09.501 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":56,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/234000461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":234000461},{"type":"v1","addr":"172.21.15.148:6834","nonce":234000461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6436,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2396073410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2396073410},{"type":"v1","addr":"172.21.15.148:6835","nonce":2396073410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6717,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/380088531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":380088531},{"type":"v1","addr":"172.21.15.105:6835","nonce":380088531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:27:05.479811+0000","modified":"2023-11-28T18:27:08.510202+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6720},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6720":{"gid":6720,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/3442512486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3442512486},{"type":"v1","addr":"172.21.15.105:6837","nonce":3442512486}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T18:27:09.501 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 56 2023-11-28T18:27:09.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.499+0000 7faa348dc640 1 -- 172.21.15.105:0/3628855894 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faa1c0668d0 msgr2=0x7faa1c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:09.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.499+0000 7faa348dc640 1 --2- 172.21.15.105:0/3628855894 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faa1c0668d0 0x7faa1c068d90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7faa200ad000 tx=0x7faa200a6000 comp rx=0 tx=0).stop 2023-11-28T18:27:09.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.499+0000 7faa348dc640 1 -- 172.21.15.105:0/3628855894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa3015d2d0 msgr2=0x7faa300b81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:09.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.499+0000 7faa348dc640 1 --2- 172.21.15.105:0/3628855894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa3015d2d0 0x7faa300b81a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7faa2800c780 tx=0x7faa2800cc40 comp rx=0 tx=0).stop 2023-11-28T18:27:09.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.499+0000 7faa348dc640 1 -- 172.21.15.105:0/3628855894 shutdown_connections 2023-11-28T18:27:09.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.499+0000 7faa348dc640 1 --2- 172.21.15.105:0/3628855894 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7faa1c0668d0 0x7faa1c068d90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.499+0000 7faa348dc640 1 --2- 172.21.15.105:0/3628855894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa3015d2d0 0x7faa300b81a0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.499+0000 7faa348dc640 1 --2- 172.21.15.105:0/3628855894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa3015cef0 0x7faa30158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.499+0000 7faa348dc640 1 --2- 172.21.15.105:0/3628855894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa300d1b40 0x7faa30159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.499+0000 7faa348dc640 1 -- 172.21.15.105:0/3628855894 >> 172.21.15.105:0/3628855894 conn(0x7faa300d2e10 msgr2=0x7faa300aff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:09.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.499+0000 7faa348dc640 1 -- 172.21.15.105:0/3628855894 shutdown_connections 2023-11-28T18:27:09.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.499+0000 7faa348dc640 1 -- 172.21.15.105:0/3628855894 wait complete. 2023-11-28T18:27:09.521 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:27:09.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.655+0000 7f79bb05f640 1 Processor -- start 2023-11-28T18:27:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.655+0000 7f79bb05f640 1 -- start start 2023-11-28T18:27:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.655+0000 7f79bb05f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b415d4f0 0x7f79b40d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.655+0000 7f79bb05f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f79b40d3c30 0x7f79b40d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:09.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.655+0000 7f79bb05f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b40b25b0 con 0x7f79b40d1890 2023-11-28T18:27:09.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.655+0000 7f79bb05f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b40b2900 con 0x7f79b40d3c30 2023-11-28T18:27:09.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.655+0000 7f79bb05f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b40cea30 con 0x7f79b415d4f0 2023-11-28T18:27:09.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.655+0000 7f79b8dfb640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b415d4f0 0x7f79b40d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:09.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.655+0000 7f79b3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f79b40d3c30 0x7f79b40d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:09.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.655+0000 7f79b8dfb640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b415d4f0 0x7f79b40d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:55006/0 (socket says 172.21.15.105:55006) 2023-11-28T18:27:09.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.655+0000 7f79b8dfb640 1 -- 172.21.15.105:0/2591822072 learned_addr learned my addr 172.21.15.105:0/2591822072 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:09.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.655+0000 7f79b95fc640 1 --1- 172.21.15.105:0/2591822072 >> v1:172.21.15.148:6789/0 conn(0x7f79b40d1890 0x7f79b40cfa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:35918/0 (socket says 172.21.15.105:35918) 2023-11-28T18:27:09.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.655+0000 7f79b3fff640 1 -- 172.21.15.105:0/2591822072 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b415d4f0 msgr2=0x7f79b40d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:09.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.655+0000 7f79b3fff640 1 --2- 172.21.15.105:0/2591822072 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b415d4f0 0x7f79b40d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.655+0000 7f79b3fff640 1 -- 172.21.15.105:0/2591822072 >> v1:172.21.15.148:6789/0 conn(0x7f79b40d1890 legacy=0x7f79b40cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:09.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.655+0000 7f79b3fff640 1 -- 172.21.15.105:0/2591822072 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b40ced10 con 0x7f79b40d3c30 2023-11-28T18:27:09.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.659+0000 7f79b3fff640 1 --2- 172.21.15.105:0/2591822072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79b40d3c30 0x7f79b40d1350 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f79a00097c0 tx=0x7f79a001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21521b22a59f24f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:09.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.659+0000 7f79b37fe640 1 -- 172.21.15.105:0/2591822072 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a00ab600 con 0x7f79b40d3c30 2023-11-28T18:27:09.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.659+0000 7f79b37fe640 1 -- 172.21.15.105:0/2591822072 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a00a7050 con 0x7f79b40d3c30 2023-11-28T18:27:09.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.659+0000 7f79b37fe640 1 -- 172.21.15.105:0/2591822072 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a00ab980 con 0x7f79b40d3c30 2023-11-28T18:27:09.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.659+0000 7f79bb05f640 1 -- 172.21.15.105:0/2591822072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79b40d3c30 msgr2=0x7f79b40d1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:09.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.659+0000 7f79bb05f640 1 --2- 172.21.15.105:0/2591822072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79b40d3c30 0x7f79b40d1350 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f79a00097c0 tx=0x7f79a001c3c0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.659+0000 7f79bb05f640 1 -- 172.21.15.105:0/2591822072 shutdown_connections 2023-11-28T18:27:09.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.659+0000 7f79bb05f640 1 --2- 172.21.15.105:0/2591822072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79b40d3c30 0x7f79b40d1350 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.659+0000 7f79bb05f640 1 --2- 172.21.15.105:0/2591822072 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b415d4f0 0x7f79b40d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.659+0000 7f79bb05f640 1 -- 172.21.15.105:0/2591822072 >> 172.21.15.105:0/2591822072 conn(0x7f79b40d3030 msgr2=0x7f79b40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:09.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.659+0000 7f79bb05f640 1 -- 172.21.15.105:0/2591822072 shutdown_connections 2023-11-28T18:27:09.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.659+0000 7f79bb05f640 1 -- 172.21.15.105:0/2591822072 wait complete. 2023-11-28T18:27:09.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79bb05f640 1 Processor -- start 2023-11-28T18:27:09.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79bb05f640 1 -- start start 2023-11-28T18:27:09.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79bb05f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79b40d1890 0x7f79b415db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:09.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79bb05f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b40d3c30 0x7f79b414f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:09.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79bb05f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79b415d4f0 0x7f79b4154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:09.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79bb05f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b41547c0 con 0x7f79b40d1890 2023-11-28T18:27:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79bb05f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b41541b0 con 0x7f79b415d4f0 2023-11-28T18:27:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79bb05f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b40cba90 con 0x7f79b40d3c30 2023-11-28T18:27:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79b8dfb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79b40d1890 0x7f79b415db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79b8dfb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79b40d1890 0x7f79b415db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44230/0 (socket says 172.21.15.105:44230) 2023-11-28T18:27:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79b8dfb640 1 -- 172.21.15.105:0/3213755134 learned_addr learned my addr 172.21.15.105:0/3213755134 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79b95fc640 1 --2- 172.21.15.105:0/3213755134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79b415d4f0 0x7f79b4154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79b3fff640 1 --2- 172.21.15.105:0/3213755134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b40d3c30 0x7f79b414f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79b8dfb640 1 -- 172.21.15.105:0/3213755134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b40d3c30 msgr2=0x7f79b414f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79b8dfb640 1 --2- 172.21.15.105:0/3213755134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b40d3c30 0x7f79b414f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79b8dfb640 1 -- 172.21.15.105:0/3213755134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79b415d4f0 msgr2=0x7f79b4154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79b8dfb640 1 --2- 172.21.15.105:0/3213755134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79b415d4f0 0x7f79b4154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79b8dfb640 1 -- 172.21.15.105:0/3213755134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a0009420 con 0x7f79b40d1890 2023-11-28T18:27:09.670 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79b3fff640 1 --2- 172.21.15.105:0/3213755134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b40d3c30 0x7f79b414f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:09.670 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79b8dfb640 1 --2- 172.21.15.105:0/3213755134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79b40d1890 0x7f79b415db60 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f79a800e7f0 tx=0x7f79a800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:09.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79b17fa640 1 -- 172.21.15.105:0/3213755134 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a8009580 con 0x7f79b40d1890 2023-11-28T18:27:09.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79b17fa640 1 -- 172.21.15.105:0/3213755134 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a800c940 con 0x7f79b40d1890 2023-11-28T18:27:09.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79bb05f640 1 -- 172.21.15.105:0/3213755134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b414fd50 con 0x7f79b40d1890 2023-11-28T18:27:09.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.663+0000 7f79b17fa640 1 -- 172.21.15.105:0/3213755134 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a800ccf0 con 0x7f79b40d1890 2023-11-28T18:27:09.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.667+0000 7f79bb05f640 1 -- 172.21.15.105:0/3213755134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b4154450 con 0x7f79b40d1890 2023-11-28T18:27:09.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.667+0000 7f79bb05f640 1 -- 172.21.15.105:0/3213755134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b4154010 con 0x7f79b40d1890 2023-11-28T18:27:09.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.667+0000 7f79b17fa640 1 -- 172.21.15.105:0/3213755134 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f79a8013470 con 0x7f79b40d1890 2023-11-28T18:27:09.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.667+0000 7f79b17fa640 1 --2- 172.21.15.105:0/3213755134 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f79980668d0 0x7f7998068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:09.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.667+0000 7f79b17fa640 1 -- 172.21.15.105:0/3213755134 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f79a8084560 con 0x7f79b40d1890 2023-11-28T18:27:09.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.667+0000 7f79b3fff640 1 --2- 172.21.15.105:0/3213755134 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f79980668d0 0x7f7998068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:09.673 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.671+0000 7f79b3fff640 1 --2- 172.21.15.105:0/3213755134 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f79980668d0 0x7f7998068d90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f79a00ad000 tx=0x7f79a00a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:09.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.671+0000 7f79b17fa640 1 -- 172.21.15.105:0/3213755134 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f79b4154010 con 0x7f79b40d1890 2023-11-28T18:27:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.823+0000 7f79bb05f640 1 -- 172.21.15.105:0/3213755134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79b4154010 con 0x7f79b40d1890 2023-11-28T18:27:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.823+0000 7f79b17fa640 1 -- 172.21.15.105:0/3213755134 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16776 (secure 0 0 0) 0x7f79b4154010 con 0x7f79b40d1890 2023-11-28T18:27:09.827 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:27:09.827 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":65,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:27:07.121705+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:27:03.225100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T18:27:04.478619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:27:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.823+0000 7f79bb05f640 1 -- 172.21.15.105:0/3213755134 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f79980668d0 msgr2=0x7f7998068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.823+0000 7f79bb05f640 1 --2- 172.21.15.105:0/3213755134 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f79980668d0 0x7f7998068d90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f79a00ad000 tx=0x7f79a00a6000 comp rx=0 tx=0).stop 2023-11-28T18:27:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.823+0000 7f79bb05f640 1 -- 172.21.15.105:0/3213755134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79b40d1890 msgr2=0x7f79b415db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.823+0000 7f79bb05f640 1 --2- 172.21.15.105:0/3213755134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79b40d1890 0x7f79b415db60 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f79a800e7f0 tx=0x7f79a800ecb0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.823+0000 7f79bb05f640 1 -- 172.21.15.105:0/3213755134 shutdown_connections 2023-11-28T18:27:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.823+0000 7f79bb05f640 1 --2- 172.21.15.105:0/3213755134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79b415d4f0 0x7f79b4154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.823+0000 7f79bb05f640 1 --2- 172.21.15.105:0/3213755134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b40d3c30 0x7f79b414f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.823+0000 7f79bb05f640 1 --2- 172.21.15.105:0/3213755134 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f79980668d0 0x7f7998068d90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.823+0000 7f79bb05f640 1 --2- 172.21.15.105:0/3213755134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79b40d1890 0x7f79b415db60 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.823+0000 7f79bb05f640 1 -- 172.21.15.105:0/3213755134 >> 172.21.15.105:0/3213755134 conn(0x7f79b40d3030 msgr2=0x7f79b40cde80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.823+0000 7f79bb05f640 1 -- 172.21.15.105:0/3213755134 shutdown_connections 2023-11-28T18:27:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.823+0000 7f79bb05f640 1 -- 172.21.15.105:0/3213755134 wait complete. 2023-11-28T18:27:09.849 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T18:27:09.849 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:27:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.983+0000 7f879229c640 1 Processor -- start 2023-11-28T18:27:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.987+0000 7f879229c640 1 -- start start 2023-11-28T18:27:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.987+0000 7f879229c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f878c15ae50 0x7f878c0c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.987+0000 7f879229c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f878c0cbb40 0x7f878c071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.987+0000 7f879229c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f878c0cb230 con 0x7f878c0cb6c0 2023-11-28T18:27:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.987+0000 7f879229c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f878c0c9f10 con 0x7f878c0cbb40 2023-11-28T18:27:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.987+0000 7f879229c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f878c0cbfb0 con 0x7f878c15ae50 2023-11-28T18:27:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.987+0000 7f8791a9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f878c0cbb40 0x7f878c071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.987+0000 7f8791a9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f878c0cbb40 0x7f878c071660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44246/0 (socket says 172.21.15.105:44246) 2023-11-28T18:27:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.987+0000 7f8791a9b640 1 -- 172.21.15.105:0/3372585562 learned_addr learned my addr 172.21.15.105:0/3372585562 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.987+0000 7f879129a640 1 --2- 172.21.15.105:0/3372585562 >> v2:172.21.15.148:3300/0 conn(0x7f878c15ae50 0x7f878c0c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.987+0000 7f8791a9b640 1 -- 172.21.15.105:0/3372585562 >> v2:172.21.15.148:3300/0 conn(0x7f878c15ae50 msgr2=0x7f878c0c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.987+0000 7f8791a9b640 1 --2- 172.21.15.105:0/3372585562 >> v2:172.21.15.148:3300/0 conn(0x7f878c15ae50 0x7f878c0c99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.987+0000 7f8791a9b640 1 -- 172.21.15.105:0/3372585562 >> v1:172.21.15.105:6789/0 conn(0x7f878c0cb6c0 legacy=0x7f878c0b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.987+0000 7f8791a9b640 1 -- 172.21.15.105:0/3372585562 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878c0ca5c0 con 0x7f878c0cbb40 2023-11-28T18:27:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.987+0000 7f8783fff640 1 -- 172.21.15.105:0/3372585562 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3702267780 0 0) 0x7f878c0cb230 con 0x7f878c0cb6c0 2023-11-28T18:27:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.987+0000 7f879129a640 1 --2- 172.21.15.105:0/3372585562 >> v2:172.21.15.148:3300/0 conn(0x7f878c15ae50 0x7f878c0c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:09.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.991+0000 7f8791a9b640 1 --2- 172.21.15.105:0/3372585562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f878c0cbb40 0x7f878c071660 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f878c072330 tx=0x7f878401c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8819d0e48eceed server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:09.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.991+0000 7f8783fff640 1 -- 172.21.15.105:0/3372585562 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87840a5780 con 0x7f878c0cbb40 2023-11-28T18:27:09.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.991+0000 7f8783fff640 1 -- 172.21.15.105:0/3372585562 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87840a58e0 con 0x7f878c0cbb40 2023-11-28T18:27:09.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.991+0000 7f8783fff640 1 -- 172.21.15.105:0/3372585562 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87840a5c70 con 0x7f878c0cbb40 2023-11-28T18:27:09.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.991+0000 7f879229c640 1 -- 172.21.15.105:0/3372585562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f878c0cbb40 msgr2=0x7f878c071660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:09.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.991+0000 7f879229c640 1 --2- 172.21.15.105:0/3372585562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f878c0cbb40 0x7f878c071660 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f878c072330 tx=0x7f878401c9d0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.991+0000 7f879229c640 1 -- 172.21.15.105:0/3372585562 shutdown_connections 2023-11-28T18:27:09.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.991+0000 7f879229c640 1 --2- 172.21.15.105:0/3372585562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f878c0cbb40 0x7f878c071660 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.991+0000 7f879229c640 1 --2- 172.21.15.105:0/3372585562 >> v2:172.21.15.148:3300/0 conn(0x7f878c15ae50 0x7f878c0c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:09.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.991+0000 7f879229c640 1 -- 172.21.15.105:0/3372585562 >> 172.21.15.105:0/3372585562 conn(0x7f878c0b0480 msgr2=0x7f878c0b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:09.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.991+0000 7f879229c640 1 -- 172.21.15.105:0/3372585562 shutdown_connections 2023-11-28T18:27:09.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.991+0000 7f879229c640 1 -- 172.21.15.105:0/3372585562 wait complete. 2023-11-28T18:27:09.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.991+0000 7f879229c640 1 Processor -- start 2023-11-28T18:27:09.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.991+0000 7f879229c640 1 -- start start 2023-11-28T18:27:10.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f879229c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f878c0cb6c0 0x7f878c153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:10.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f879229c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f878c0cbb40 0x7f878c155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:10.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f879229c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f878c15ae50 0x7f878c1595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:10.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f879229c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c0c33e0 con 0x7f878c0cbb40 2023-11-28T18:27:10.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f879229c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c0c27d0 con 0x7f878c15ae50 2023-11-28T18:27:10.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f879229c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878c0c2ab0 con 0x7f878c0cb6c0 2023-11-28T18:27:10.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f8790a99640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f878c0cbb40 0x7f878c155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:10.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f8790a99640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f878c0cbb40 0x7f878c155610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44256/0 (socket says 172.21.15.105:44256) 2023-11-28T18:27:10.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f8790a99640 1 -- 172.21.15.105:0/392787477 learned_addr learned my addr 172.21.15.105:0/392787477 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:10.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f879129a640 1 --2- 172.21.15.105:0/392787477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f878c0cb6c0 0x7f878c153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:10.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f8791a9b640 1 --2- 172.21.15.105:0/392787477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f878c15ae50 0x7f878c1595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:10.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f8790a99640 1 -- 172.21.15.105:0/392787477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f878c0cb6c0 msgr2=0x7f878c153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:10.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f8790a99640 1 --2- 172.21.15.105:0/392787477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f878c0cb6c0 0x7f878c153950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f8790a99640 1 -- 172.21.15.105:0/392787477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f878c15ae50 msgr2=0x7f878c1595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:10.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f8790a99640 1 --2- 172.21.15.105:0/392787477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f878c15ae50 0x7f878c1595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f8790a99640 1 -- 172.21.15.105:0/392787477 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8784009b30 con 0x7f878c0cbb40 2023-11-28T18:27:10.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f8790a99640 1 --2- 172.21.15.105:0/392787477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f878c0cbb40 0x7f878c155610 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f877c001ec0 tx=0x7f877c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:10.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f8781ffb640 1 -- 172.21.15.105:0/392787477 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c0054e0 con 0x7f878c0cbb40 2023-11-28T18:27:10.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f8781ffb640 1 -- 172.21.15.105:0/392787477 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877c003600 con 0x7f878c0cbb40 2023-11-28T18:27:10.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f879229c640 1 -- 172.21.15.105:0/392787477 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f878c0c2d90 con 0x7f878c0cbb40 2023-11-28T18:27:10.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f8781ffb640 1 -- 172.21.15.105:0/392787477 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c0039b0 con 0x7f878c0cbb40 2023-11-28T18:27:10.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f879229c640 1 -- 172.21.15.105:0/392787477 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f878c155b50 con 0x7f878c0cbb40 2023-11-28T18:27:10.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f879229c640 1 -- 172.21.15.105:0/392787477 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f878c0b1e80 con 0x7f878c0cbb40 2023-11-28T18:27:10.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.995+0000 7f8781ffb640 1 -- 172.21.15.105:0/392787477 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f877c003b10 con 0x7f878c0cbb40 2023-11-28T18:27:10.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.999+0000 7f8781ffb640 1 --2- 172.21.15.105:0/392787477 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f87700668d0 0x7f8770068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:10.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.999+0000 7f8781ffb640 1 -- 172.21.15.105:0/392787477 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f877c083b90 con 0x7f878c0cbb40 2023-11-28T18:27:10.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.999+0000 7f879129a640 1 --2- 172.21.15.105:0/392787477 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f87700668d0 0x7f8770068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:10.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:09.999+0000 7f879129a640 1 --2- 172.21.15.105:0/392787477 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f87700668d0 0x7f8770068d90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f878800c900 tx=0x7f8788008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:10.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.003+0000 7f8781ffb640 1 -- 172.21.15.105:0/392787477 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f877c050c80 con 0x7f878c0cbb40 2023-11-28T18:27:10.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.151+0000 7f879229c640 1 -- 172.21.15.105:0/392787477 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f878c0c2c20 con 0x7f87700668d0 2023-11-28T18:27:10.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.159+0000 7f8781ffb640 1 -- 172.21.15.105:0/392787477 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289322 (secure 0 0 0) 0x7f878c0c2c20 con 0x7f87700668d0 2023-11-28T18:27:10.167 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:27:10.169 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:27:10.170 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.163+0000 7f879229c640 1 -- 172.21.15.105:0/392787477 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f87700668d0 msgr2=0x7f8770068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:10.170 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.163+0000 7f879229c640 1 --2- 172.21.15.105:0/392787477 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f87700668d0 0x7f8770068d90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f878800c900 tx=0x7f8788008040 comp rx=0 tx=0).stop 2023-11-28T18:27:10.170 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.163+0000 7f879229c640 1 -- 172.21.15.105:0/392787477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f878c0cbb40 msgr2=0x7f878c155610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:10.170 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.163+0000 7f879229c640 1 --2- 172.21.15.105:0/392787477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f878c0cbb40 0x7f878c155610 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f877c001ec0 tx=0x7f877c00c8e0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.170 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.163+0000 7f879229c640 1 -- 172.21.15.105:0/392787477 shutdown_connections 2023-11-28T18:27:10.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.163+0000 7f879229c640 1 --2- 172.21.15.105:0/392787477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f878c15ae50 0x7f878c1595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.163+0000 7f879229c640 1 --2- 172.21.15.105:0/392787477 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f87700668d0 0x7f8770068d90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.163+0000 7f879229c640 1 --2- 172.21.15.105:0/392787477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f878c0cbb40 0x7f878c155610 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.163+0000 7f879229c640 1 --2- 172.21.15.105:0/392787477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f878c0cb6c0 0x7f878c153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.163+0000 7f879229c640 1 -- 172.21.15.105:0/392787477 >> 172.21.15.105:0/392787477 conn(0x7f878c0b0480 msgr2=0x7f878c0b3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:10.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.163+0000 7f879229c640 1 -- 172.21.15.105:0/392787477 shutdown_connections 2023-11-28T18:27:10.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.163+0000 7f879229c640 1 -- 172.21.15.105:0/392787477 wait complete. 2023-11-28T18:27:10.187 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2023-11-28T18:27:09.315404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":269132,"kb_used_data":7204,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754705588,"statfs":{"total":773094113280,"available":772818522112,"internally_reserved":0,"allocated":7376896,"data_stored":3954860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":268095026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.226373"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.147970+0000","last_change":"2023-11-28T18:27:06.142439+0000","last_active":"2023-11-28T18:27:06.147970+0000","last_peered":"2023-11-28T18:27:06.147970+0000","last_clean":"2023-11-28T18:27:06.147970+0000","last_became_active":"2023-11-28T18:27:06.142185+0000","last_became_peered":"2023-11-28T18:27:06.142185+0000","last_unstale":"2023-11-28T18:27:06.147970+0000","last_undegraded":"2023-11-28T18:27:06.147970+0000","last_fullsized":"2023-11-28T18:27:06.147970+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:30:10.866128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.145033+0000","last_change":"2023-11-28T18:27:06.138796+0000","last_active":"2023-11-28T18:27:06.145033+0000","last_peered":"2023-11-28T18:27:06.145033+0000","last_clean":"2023-11-28T18:27:06.145033+0000","last_became_active":"2023-11-28T18:27:06.138513+0000","last_became_peered":"2023-11-28T18:27:06.138513+0000","last_unstale":"2023-11-28T18:27:06.145033+0000","last_undegraded":"2023-11-28T18:27:06.145033+0000","last_fullsized":"2023-11-28T18:27:06.145033+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:53:09.924003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.154469+0000","last_change":"2023-11-28T18:27:06.148765+0000","last_active":"2023-11-28T18:27:06.154469+0000","last_peered":"2023-11-28T18:27:06.154469+0000","last_clean":"2023-11-28T18:27:06.154469+0000","last_became_active":"2023-11-28T18:27:06.148381+0000","last_became_peered":"2023-11-28T18:27:06.148381+0000","last_unstale":"2023-11-28T18:27:06.154469+0000","last_undegraded":"2023-11-28T18:27:06.154469+0000","last_fullsized":"2023-11-28T18:27:06.154469+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:49:16.270543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.155939+0000","last_change":"2023-11-28T18:27:06.141251+0000","last_active":"2023-11-28T18:27:06.155939+0000","last_peered":"2023-11-28T18:27:06.155939+0000","last_clean":"2023-11-28T18:27:06.155939+0000","last_became_active":"2023-11-28T18:27:06.140804+0000","last_became_peered":"2023-11-28T18:27:06.140804+0000","last_unstale":"2023-11-28T18:27:06.155939+0000","last_undegraded":"2023-11-28T18:27:06.155939+0000","last_fullsized":"2023-11-28T18:27:06.155939+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:04:43.561084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.157013+0000","last_change":"2023-11-28T18:27:06.153434+0000","last_active":"2023-11-28T18:27:06.157013+0000","last_peered":"2023-11-28T18:27:06.157013+0000","last_clean":"2023-11-28T18:27:06.157013+0000","last_became_active":"2023-11-28T18:27:06.153158+0000","last_became_peered":"2023-11-28T18:27:06.153158+0000","last_unstale":"2023-11-28T18:27:06.157013+0000","last_undegraded":"2023-11-28T18:27:06.157013+0000","last_fullsized":"2023-11-28T18:27:06.157013+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:04:43.561084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.145084+0000","last_change":"2023-11-28T18:27:06.137310+0000","last_active":"2023-11-28T18:27:06.145084+0000","last_peered":"2023-11-28T18:27:06.145084+0000","last_clean":"2023-11-28T18:27:06.145084+0000","last_became_active":"2023-11-28T18:27:06.137051+0000","last_became_peered":"2023-11-28T18:27:06.137051+0000","last_unstale":"2023-11-28T18:27:06.145084+0000","last_undegraded":"2023-11-28T18:27:06.145084+0000","last_fullsized":"2023-11-28T18:27:06.145084+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:05:25.033948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.142918+0000","last_change":"2023-11-28T18:27:06.135215+0000","last_active":"2023-11-28T18:27:06.142918+0000","last_peered":"2023-11-28T18:27:06.142918+0000","last_clean":"2023-11-28T18:27:06.142918+0000","last_became_active":"2023-11-28T18:27:06.133785+0000","last_became_peered":"2023-11-28T18:27:06.133785+0000","last_unstale":"2023-11-28T18:27:06.142918+0000","last_undegraded":"2023-11-28T18:27:06.142918+0000","last_fullsized":"2023-11-28T18:27:06.142918+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:41.339034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.138246+0000","last_change":"2023-11-28T18:27:06.131192+0000","last_active":"2023-11-28T18:27:06.138246+0000","last_peered":"2023-11-28T18:27:06.138246+0000","last_clean":"2023-11-28T18:27:06.138246+0000","last_became_active":"2023-11-28T18:27:06.130898+0000","last_became_peered":"2023-11-28T18:27:06.130898+0000","last_unstale":"2023-11-28T18:27:06.138246+0000","last_undegraded":"2023-11-28T18:27:06.138246+0000","last_fullsized":"2023-11-28T18:27:06.138246+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:10:22.954858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.156766+0000","last_change":"2023-11-28T18:27:06.150904+0000","last_active":"2023-11-28T18:27:06.156766+0000","last_peered":"2023-11-28T18:27:06.156766+0000","last_clean":"2023-11-28T18:27:06.156766+0000","last_became_active":"2023-11-28T18:27:06.150620+0000","last_became_peered":"2023-11-28T18:27:06.150620+0000","last_unstale":"2023-11-28T18:27:06.156766+0000","last_undegraded":"2023-11-28T18:27:06.156766+0000","last_fullsized":"2023-11-28T18:27:06.156766+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:30.288695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.147128+0000","last_change":"2023-11-28T18:27:06.143292+0000","last_active":"2023-11-28T18:27:06.147128+0000","last_peered":"2023-11-28T18:27:06.147128+0000","last_clean":"2023-11-28T18:27:06.147128+0000","last_became_active":"2023-11-28T18:27:06.143012+0000","last_became_peered":"2023-11-28T18:27:06.143012+0000","last_unstale":"2023-11-28T18:27:06.147128+0000","last_undegraded":"2023-11-28T18:27:06.147128+0000","last_fullsized":"2023-11-28T18:27:06.147128+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:03:06.598375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.155149+0000","last_change":"2023-11-28T18:27:06.140378+0000","last_active":"2023-11-28T18:27:06.155149+0000","last_peered":"2023-11-28T18:27:06.155149+0000","last_clean":"2023-11-28T18:27:06.155149+0000","last_became_active":"2023-11-28T18:27:06.139892+0000","last_became_peered":"2023-11-28T18:27:06.139892+0000","last_unstale":"2023-11-28T18:27:06.155149+0000","last_undegraded":"2023-11-28T18:27:06.155149+0000","last_fullsized":"2023-11-28T18:27:06.155149+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:32.696569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.137398+0000","last_change":"2023-11-28T18:27:06.130584+0000","last_active":"2023-11-28T18:27:06.137398+0000","last_peered":"2023-11-28T18:27:06.137398+0000","last_clean":"2023-11-28T18:27:06.137398+0000","last_became_active":"2023-11-28T18:27:06.130256+0000","last_became_peered":"2023-11-28T18:27:06.130256+0000","last_unstale":"2023-11-28T18:27:06.137398+0000","last_undegraded":"2023-11-28T18:27:06.137398+0000","last_fullsized":"2023-11-28T18:27:06.137398+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:31:14.921516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.140875+0000","last_change":"2023-11-28T18:27:06.135731+0000","last_active":"2023-11-28T18:27:06.140875+0000","last_peered":"2023-11-28T18:27:06.140875+0000","last_clean":"2023-11-28T18:27:06.140875+0000","last_became_active":"2023-11-28T18:27:06.134763+0000","last_became_peered":"2023-11-28T18:27:06.134763+0000","last_unstale":"2023-11-28T18:27:06.140875+0000","last_undegraded":"2023-11-28T18:27:06.140875+0000","last_fullsized":"2023-11-28T18:27:06.140875+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:46.633835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.147460+0000","last_change":"2023-11-28T18:27:06.143876+0000","last_active":"2023-11-28T18:27:06.147460+0000","last_peered":"2023-11-28T18:27:06.147460+0000","last_clean":"2023-11-28T18:27:06.147460+0000","last_became_active":"2023-11-28T18:27:06.142991+0000","last_became_peered":"2023-11-28T18:27:06.142991+0000","last_unstale":"2023-11-28T18:27:06.147460+0000","last_undegraded":"2023-11-28T18:27:06.147460+0000","last_fullsized":"2023-11-28T18:27:06.147460+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:31:14.921516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.152161+0000","last_change":"2023-11-28T18:27:06.145466+0000","last_active":"2023-11-28T18:27:06.152161+0000","last_peered":"2023-11-28T18:27:06.152161+0000","last_clean":"2023-11-28T18:27:06.152161+0000","last_became_active":"2023-11-28T18:27:06.145205+0000","last_became_peered":"2023-11-28T18:27:06.145205+0000","last_unstale":"2023-11-28T18:27:06.152161+0000","last_undegraded":"2023-11-28T18:27:06.152161+0000","last_fullsized":"2023-11-28T18:27:06.152161+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:10:22.954858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.143960+0000","last_change":"2023-11-28T18:27:06.134316+0000","last_active":"2023-11-28T18:27:06.143960+0000","last_peered":"2023-11-28T18:27:06.143960+0000","last_clean":"2023-11-28T18:27:06.143960+0000","last_became_active":"2023-11-28T18:27:06.133990+0000","last_became_peered":"2023-11-28T18:27:06.133990+0000","last_unstale":"2023-11-28T18:27:06.143960+0000","last_undegraded":"2023-11-28T18:27:06.143960+0000","last_fullsized":"2023-11-28T18:27:06.143960+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:04.405447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.153140+0000","last_change":"2023-11-28T18:27:06.152179+0000","last_active":"2023-11-28T18:27:06.153140+0000","last_peered":"2023-11-28T18:27:06.153140+0000","last_clean":"2023-11-28T18:27:06.153140+0000","last_became_active":"2023-11-28T18:27:06.151911+0000","last_became_peered":"2023-11-28T18:27:06.151911+0000","last_unstale":"2023-11-28T18:27:06.153140+0000","last_undegraded":"2023-11-28T18:27:06.153140+0000","last_fullsized":"2023-11-28T18:27:06.153140+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:36:22.268715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.155168+0000","last_change":"2023-11-28T18:27:06.140909+0000","last_active":"2023-11-28T18:27:06.155168+0000","last_peered":"2023-11-28T18:27:06.155168+0000","last_clean":"2023-11-28T18:27:06.155168+0000","last_became_active":"2023-11-28T18:27:06.140577+0000","last_became_peered":"2023-11-28T18:27:06.140577+0000","last_unstale":"2023-11-28T18:27:06.155168+0000","last_undegraded":"2023-11-28T18:27:06.155168+0000","last_fullsized":"2023-11-28T18:27:06.155168+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:53.072933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.137411+0000","last_change":"2023-11-28T18:27:06.132399+0000","last_active":"2023-11-28T18:27:06.137411+0000","last_peered":"2023-11-28T18:27:06.137411+0000","last_clean":"2023-11-28T18:27:06.137411+0000","last_became_active":"2023-11-28T18:27:06.132106+0000","last_became_peered":"2023-11-28T18:27:06.132106+0000","last_unstale":"2023-11-28T18:27:06.137411+0000","last_undegraded":"2023-11-28T18:27:06.137411+0000","last_fullsized":"2023-11-28T18:27:06.137411+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:08:49.174545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.139431+0000","last_change":"2023-11-28T18:27:06.132575+0000","last_active":"2023-11-28T18:27:06.139431+0000","last_peered":"2023-11-28T18:27:06.139431+0000","last_clean":"2023-11-28T18:27:06.139431+0000","last_became_active":"2023-11-28T18:27:06.131896+0000","last_became_peered":"2023-11-28T18:27:06.131896+0000","last_unstale":"2023-11-28T18:27:06.139431+0000","last_undegraded":"2023-11-28T18:27:06.139431+0000","last_fullsized":"2023-11-28T18:27:06.139431+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:09.802415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.140857+0000","last_change":"2023-11-28T18:27:06.135561+0000","last_active":"2023-11-28T18:27:06.140857+0000","last_peered":"2023-11-28T18:27:06.140857+0000","last_clean":"2023-11-28T18:27:06.140857+0000","last_became_active":"2023-11-28T18:27:06.134311+0000","last_became_peered":"2023-11-28T18:27:06.134311+0000","last_unstale":"2023-11-28T18:27:06.140857+0000","last_undegraded":"2023-11-28T18:27:06.140857+0000","last_fullsized":"2023-11-28T18:27:06.140857+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:53:58.080348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.135869+0000","last_change":"2023-11-28T18:27:06.131654+0000","last_active":"2023-11-28T18:27:06.135869+0000","last_peered":"2023-11-28T18:27:06.135869+0000","last_clean":"2023-11-28T18:27:06.135869+0000","last_became_active":"2023-11-28T18:27:06.131372+0000","last_became_peered":"2023-11-28T18:27:06.131372+0000","last_unstale":"2023-11-28T18:27:06.135869+0000","last_undegraded":"2023-11-28T18:27:06.135869+0000","last_fullsized":"2023-11-28T18:27:06.135869+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:05:17.307507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.133325+0000","last_change":"2023-11-28T18:27:06.131021+0000","last_active":"2023-11-28T18:27:06.133325+0000","last_peered":"2023-11-28T18:27:06.133325+0000","last_clean":"2023-11-28T18:27:06.133325+0000","last_became_active":"2023-11-28T18:27:06.130770+0000","last_became_peered":"2023-11-28T18:27:06.130770+0000","last_unstale":"2023-11-28T18:27:06.133325+0000","last_undegraded":"2023-11-28T18:27:06.133325+0000","last_fullsized":"2023-11-28T18:27:06.133325+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:30.288695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.147760+0000","last_change":"2023-11-28T18:27:06.141539+0000","last_active":"2023-11-28T18:27:06.147760+0000","last_peered":"2023-11-28T18:27:06.147760+0000","last_clean":"2023-11-28T18:27:06.147760+0000","last_became_active":"2023-11-28T18:27:06.141264+0000","last_became_peered":"2023-11-28T18:27:06.141264+0000","last_unstale":"2023-11-28T18:27:06.147760+0000","last_undegraded":"2023-11-28T18:27:06.147760+0000","last_fullsized":"2023-11-28T18:27:06.147760+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:46.633835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.141208+0000","last_change":"2023-11-28T18:27:06.137690+0000","last_active":"2023-11-28T18:27:06.141208+0000","last_peered":"2023-11-28T18:27:06.141208+0000","last_clean":"2023-11-28T18:27:06.141208+0000","last_became_active":"2023-11-28T18:27:06.136614+0000","last_became_peered":"2023-11-28T18:27:06.136614+0000","last_unstale":"2023-11-28T18:27:06.141208+0000","last_undegraded":"2023-11-28T18:27:06.141208+0000","last_fullsized":"2023-11-28T18:27:06.141208+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:05:34.965476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.141385+0000","last_change":"2023-11-28T18:27:06.136893+0000","last_active":"2023-11-28T18:27:06.141385+0000","last_peered":"2023-11-28T18:27:06.141385+0000","last_clean":"2023-11-28T18:27:06.141385+0000","last_became_active":"2023-11-28T18:27:06.136463+0000","last_became_peered":"2023-11-28T18:27:06.136463+0000","last_unstale":"2023-11-28T18:27:06.141385+0000","last_undegraded":"2023-11-28T18:27:06.141385+0000","last_fullsized":"2023-11-28T18:27:06.141385+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:03:06.598375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.155101+0000","last_change":"2023-11-28T18:27:06.140146+0000","last_active":"2023-11-28T18:27:06.155101+0000","last_peered":"2023-11-28T18:27:06.155101+0000","last_clean":"2023-11-28T18:27:06.155101+0000","last_became_active":"2023-11-28T18:27:06.139556+0000","last_became_peered":"2023-11-28T18:27:06.139556+0000","last_unstale":"2023-11-28T18:27:06.155101+0000","last_undegraded":"2023-11-28T18:27:06.155101+0000","last_fullsized":"2023-11-28T18:27:06.155101+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:19:50.305528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.147197+0000","last_change":"2023-11-28T18:27:06.140992+0000","last_active":"2023-11-28T18:27:06.147197+0000","last_peered":"2023-11-28T18:27:06.147197+0000","last_clean":"2023-11-28T18:27:06.147197+0000","last_became_active":"2023-11-28T18:27:06.140738+0000","last_became_peered":"2023-11-28T18:27:06.140738+0000","last_unstale":"2023-11-28T18:27:06.147197+0000","last_undegraded":"2023-11-28T18:27:06.147197+0000","last_fullsized":"2023-11-28T18:27:06.147197+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:01:34.681664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.154516+0000","last_change":"2023-11-28T18:27:06.150689+0000","last_active":"2023-11-28T18:27:06.154516+0000","last_peered":"2023-11-28T18:27:06.154516+0000","last_clean":"2023-11-28T18:27:06.154516+0000","last_became_active":"2023-11-28T18:27:06.150371+0000","last_became_peered":"2023-11-28T18:27:06.150371+0000","last_unstale":"2023-11-28T18:27:06.154516+0000","last_undegraded":"2023-11-28T18:27:06.154516+0000","last_fullsized":"2023-11-28T18:27:06.154516+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:05:34.965476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.140810+0000","last_change":"2023-11-28T18:27:06.133697+0000","last_active":"2023-11-28T18:27:06.140810+0000","last_peered":"2023-11-28T18:27:06.140810+0000","last_clean":"2023-11-28T18:27:06.140810+0000","last_became_active":"2023-11-28T18:27:06.133378+0000","last_became_peered":"2023-11-28T18:27:06.133378+0000","last_unstale":"2023-11-28T18:27:06.140810+0000","last_undegraded":"2023-11-28T18:27:06.140810+0000","last_fullsized":"2023-11-28T18:27:06.140810+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:54:07.729456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.154404+0000","last_change":"2023-11-28T18:27:06.134156+0000","last_active":"2023-11-28T18:27:06.154404+0000","last_peered":"2023-11-28T18:27:06.154404+0000","last_clean":"2023-11-28T18:27:06.154404+0000","last_became_active":"2023-11-28T18:27:06.133843+0000","last_became_peered":"2023-11-28T18:27:06.133843+0000","last_unstale":"2023-11-28T18:27:06.154404+0000","last_undegraded":"2023-11-28T18:27:06.154404+0000","last_fullsized":"2023-11-28T18:27:06.154404+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:08:49.174545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.144946+0000","last_change":"2023-11-28T18:27:06.135151+0000","last_active":"2023-11-28T18:27:06.144946+0000","last_peered":"2023-11-28T18:27:06.144946+0000","last_clean":"2023-11-28T18:27:06.144946+0000","last_became_active":"2023-11-28T18:27:06.134776+0000","last_became_peered":"2023-11-28T18:27:06.134776+0000","last_unstale":"2023-11-28T18:27:06.144946+0000","last_undegraded":"2023-11-28T18:27:06.144946+0000","last_fullsized":"2023-11-28T18:27:06.144946+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:58:50.027962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.153329+0000","last_change":"2023-11-28T18:27:06.147051+0000","last_active":"2023-11-28T18:27:06.153329+0000","last_peered":"2023-11-28T18:27:06.153329+0000","last_clean":"2023-11-28T18:27:06.153329+0000","last_became_active":"2023-11-28T18:27:06.146763+0000","last_became_peered":"2023-11-28T18:27:06.146763+0000","last_unstale":"2023-11-28T18:27:06.153329+0000","last_undegraded":"2023-11-28T18:27:06.153329+0000","last_fullsized":"2023-11-28T18:27:06.153329+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:19:50.305528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"64'4","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.149132+0000","last_change":"2023-11-28T18:27:05.122804+0000","last_active":"2023-11-28T18:27:06.149132+0000","last_peered":"2023-11-28T18:27:06.149132+0000","last_clean":"2023-11-28T18:27:06.149132+0000","last_became_active":"2023-11-28T18:27:05.122553+0000","last_became_peered":"2023-11-28T18:27:05.122553+0000","last_unstale":"2023-11-28T18:27:06.149132+0000","last_undegraded":"2023-11-28T18:27:06.149132+0000","last_fullsized":"2023-11-28T18:27:06.149132+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:53:57.074647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.155230+0000","last_change":"2023-11-28T18:27:06.148896+0000","last_active":"2023-11-28T18:27:06.155230+0000","last_peered":"2023-11-28T18:27:06.155230+0000","last_clean":"2023-11-28T18:27:06.155230+0000","last_became_active":"2023-11-28T18:27:06.148549+0000","last_became_peered":"2023-11-28T18:27:06.148549+0000","last_unstale":"2023-11-28T18:27:06.155230+0000","last_undegraded":"2023-11-28T18:27:06.155230+0000","last_fullsized":"2023-11-28T18:27:06.155230+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:45.390936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.137000+0000","last_change":"2023-11-28T18:27:05.126258+0000","last_active":"2023-11-28T18:27:06.137000+0000","last_peered":"2023-11-28T18:27:06.137000+0000","last_clean":"2023-11-28T18:27:06.137000+0000","last_became_active":"2023-11-28T18:27:05.125922+0000","last_became_peered":"2023-11-28T18:27:05.125922+0000","last_unstale":"2023-11-28T18:27:06.137000+0000","last_undegraded":"2023-11-28T18:27:06.137000+0000","last_fullsized":"2023-11-28T18:27:06.137000+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:11:24.324549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.145065+0000","last_change":"2023-11-28T18:27:06.137214+0000","last_active":"2023-11-28T18:27:06.145065+0000","last_peered":"2023-11-28T18:27:06.145065+0000","last_clean":"2023-11-28T18:27:06.145065+0000","last_became_active":"2023-11-28T18:27:06.136805+0000","last_became_peered":"2023-11-28T18:27:06.136805+0000","last_unstale":"2023-11-28T18:27:06.145065+0000","last_undegraded":"2023-11-28T18:27:06.145065+0000","last_fullsized":"2023-11-28T18:27:06.145065+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:09.943693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.148053+0000","last_change":"2023-11-28T18:27:05.121661+0000","last_active":"2023-11-28T18:27:06.148053+0000","last_peered":"2023-11-28T18:27:06.148053+0000","last_clean":"2023-11-28T18:27:06.148053+0000","last_became_active":"2023-11-28T18:27:05.121367+0000","last_became_peered":"2023-11-28T18:27:05.121367+0000","last_unstale":"2023-11-28T18:27:06.148053+0000","last_undegraded":"2023-11-28T18:27:06.148053+0000","last_fullsized":"2023-11-28T18:27:06.148053+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:20:41.505129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.154334+0000","last_change":"2023-11-28T18:27:06.147653+0000","last_active":"2023-11-28T18:27:06.154334+0000","last_peered":"2023-11-28T18:27:06.154334+0000","last_clean":"2023-11-28T18:27:06.154334+0000","last_became_active":"2023-11-28T18:27:06.147461+0000","last_became_peered":"2023-11-28T18:27:06.147461+0000","last_unstale":"2023-11-28T18:27:06.154334+0000","last_undegraded":"2023-11-28T18:27:06.154334+0000","last_fullsized":"2023-11-28T18:27:06.154334+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:04:18.273513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.142152+0000","last_change":"2023-11-28T18:27:05.122474+0000","last_active":"2023-11-28T18:27:06.142152+0000","last_peered":"2023-11-28T18:27:06.142152+0000","last_clean":"2023-11-28T18:27:06.142152+0000","last_became_active":"2023-11-28T18:27:05.122194+0000","last_became_peered":"2023-11-28T18:27:05.122194+0000","last_unstale":"2023-11-28T18:27:06.142152+0000","last_undegraded":"2023-11-28T18:27:06.142152+0000","last_fullsized":"2023-11-28T18:27:06.142152+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:11:36.524909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.146181+0000","last_change":"2023-11-28T18:27:06.142803+0000","last_active":"2023-11-28T18:27:06.146181+0000","last_peered":"2023-11-28T18:27:06.146181+0000","last_clean":"2023-11-28T18:27:06.146181+0000","last_became_active":"2023-11-28T18:27:06.142524+0000","last_became_peered":"2023-11-28T18:27:06.142524+0000","last_unstale":"2023-11-28T18:27:06.146181+0000","last_undegraded":"2023-11-28T18:27:06.146181+0000","last_fullsized":"2023-11-28T18:27:06.146181+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:05:07.855561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.138262+0000","last_change":"2023-11-28T18:27:05.119528+0000","last_active":"2023-11-28T18:27:06.138262+0000","last_peered":"2023-11-28T18:27:06.138262+0000","last_clean":"2023-11-28T18:27:06.138262+0000","last_became_active":"2023-11-28T18:27:05.119169+0000","last_became_peered":"2023-11-28T18:27:05.119169+0000","last_unstale":"2023-11-28T18:27:06.138262+0000","last_undegraded":"2023-11-28T18:27:06.138262+0000","last_fullsized":"2023-11-28T18:27:06.138262+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:04:17.267811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.152676+0000","last_change":"2023-11-28T18:27:06.151822+0000","last_active":"2023-11-28T18:27:06.152676+0000","last_peered":"2023-11-28T18:27:06.152676+0000","last_clean":"2023-11-28T18:27:06.152676+0000","last_became_active":"2023-11-28T18:27:06.151587+0000","last_became_peered":"2023-11-28T18:27:06.151587+0000","last_unstale":"2023-11-28T18:27:06.152676+0000","last_undegraded":"2023-11-28T18:27:06.152676+0000","last_fullsized":"2023-11-28T18:27:06.152676+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:13:14.594224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.133627+0000","last_change":"2023-11-28T18:27:05.112741+0000","last_active":"2023-11-28T18:27:06.133627+0000","last_peered":"2023-11-28T18:27:06.133627+0000","last_clean":"2023-11-28T18:27:06.133627+0000","last_became_active":"2023-11-28T18:27:05.112443+0000","last_became_peered":"2023-11-28T18:27:05.112443+0000","last_unstale":"2023-11-28T18:27:06.133627+0000","last_undegraded":"2023-11-28T18:27:06.133627+0000","last_fullsized":"2023-11-28T18:27:06.133627+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:33:14.262575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.153261+0000","last_change":"2023-11-28T18:27:06.146804+0000","last_active":"2023-11-28T18:27:06.153261+0000","last_peered":"2023-11-28T18:27:06.153261+0000","last_clean":"2023-11-28T18:27:06.153261+0000","last_became_active":"2023-11-28T18:27:06.146510+0000","last_became_peered":"2023-11-28T18:27:06.146510+0000","last_unstale":"2023-11-28T18:27:06.153261+0000","last_undegraded":"2023-11-28T18:27:06.153261+0000","last_fullsized":"2023-11-28T18:27:06.153261+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:53.072933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.112577+0000","last_change":"2023-11-28T18:27:05.113425+0000","last_active":"2023-11-28T18:27:06.112577+0000","last_peered":"2023-11-28T18:27:06.112577+0000","last_clean":"2023-11-28T18:27:06.112577+0000","last_became_active":"2023-11-28T18:27:05.111871+0000","last_became_peered":"2023-11-28T18:27:05.111871+0000","last_unstale":"2023-11-28T18:27:06.112577+0000","last_undegraded":"2023-11-28T18:27:06.112577+0000","last_fullsized":"2023-11-28T18:27:06.112577+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:24:41.870988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.132741+0000","last_change":"2023-11-28T18:27:06.129616+0000","last_active":"2023-11-28T18:27:06.132741+0000","last_peered":"2023-11-28T18:27:06.132741+0000","last_clean":"2023-11-28T18:27:06.132741+0000","last_became_active":"2023-11-28T18:27:06.129366+0000","last_became_peered":"2023-11-28T18:27:06.129366+0000","last_unstale":"2023-11-28T18:27:06.132741+0000","last_undegraded":"2023-11-28T18:27:06.132741+0000","last_fullsized":"2023-11-28T18:27:06.132741+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:36:38.407270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.130327+0000","last_change":"2023-11-28T18:27:05.123567+0000","last_active":"2023-11-28T18:27:06.130327+0000","last_peered":"2023-11-28T18:27:06.130327+0000","last_clean":"2023-11-28T18:27:06.130327+0000","last_became_active":"2023-11-28T18:27:05.123267+0000","last_became_peered":"2023-11-28T18:27:05.123267+0000","last_unstale":"2023-11-28T18:27:06.130327+0000","last_undegraded":"2023-11-28T18:27:06.130327+0000","last_fullsized":"2023-11-28T18:27:06.130327+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:28:03.116448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.147178+0000","last_change":"2023-11-28T18:27:06.140874+0000","last_active":"2023-11-28T18:27:06.147178+0000","last_peered":"2023-11-28T18:27:06.147178+0000","last_clean":"2023-11-28T18:27:06.147178+0000","last_became_active":"2023-11-28T18:27:06.140379+0000","last_became_peered":"2023-11-28T18:27:06.140379+0000","last_unstale":"2023-11-28T18:27:06.147178+0000","last_undegraded":"2023-11-28T18:27:06.147178+0000","last_fullsized":"2023-11-28T18:27:06.147178+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:37.808591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.123289+0000","last_change":"2023-11-28T18:27:05.129282+0000","last_active":"2023-11-28T18:27:06.123289+0000","last_peered":"2023-11-28T18:27:06.123289+0000","last_clean":"2023-11-28T18:27:06.123289+0000","last_became_active":"2023-11-28T18:27:05.128963+0000","last_became_peered":"2023-11-28T18:27:05.128963+0000","last_unstale":"2023-11-28T18:27:06.123289+0000","last_undegraded":"2023-11-28T18:27:06.123289+0000","last_fullsized":"2023-11-28T18:27:06.123289+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:08.796713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.132254+0000","last_change":"2023-11-28T18:27:06.125687+0000","last_active":"2023-11-28T18:27:06.132254+0000","last_peered":"2023-11-28T18:27:06.132254+0000","last_clean":"2023-11-28T18:27:06.132254+0000","last_became_active":"2023-11-28T18:27:06.125413+0000","last_became_peered":"2023-11-28T18:27:06.125413+0000","last_unstale":"2023-11-28T18:27:06.132254+0000","last_undegraded":"2023-11-28T18:27:06.132254+0000","last_fullsized":"2023-11-28T18:27:06.132254+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:58:50.027962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.120173+0000","last_change":"2023-11-28T18:27:05.109290+0000","last_active":"2023-11-28T18:27:06.120173+0000","last_peered":"2023-11-28T18:27:06.120173+0000","last_clean":"2023-11-28T18:27:06.120173+0000","last_became_active":"2023-11-28T18:27:05.108985+0000","last_became_peered":"2023-11-28T18:27:05.108985+0000","last_unstale":"2023-11-28T18:27:06.120173+0000","last_undegraded":"2023-11-28T18:27:06.120173+0000","last_fullsized":"2023-11-28T18:27:06.120173+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:05:31.008179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.145999+0000","last_change":"2023-11-28T18:27:06.139897+0000","last_active":"2023-11-28T18:27:06.145999+0000","last_peered":"2023-11-28T18:27:06.145999+0000","last_clean":"2023-11-28T18:27:06.145999+0000","last_became_active":"2023-11-28T18:27:06.139625+0000","last_became_peered":"2023-11-28T18:27:06.139625+0000","last_unstale":"2023-11-28T18:27:06.145999+0000","last_undegraded":"2023-11-28T18:27:06.145999+0000","last_fullsized":"2023-11-28T18:27:06.145999+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:45:48.134616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"64'3","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.147448+0000","last_change":"2023-11-28T18:27:05.129508+0000","last_active":"2023-11-28T18:27:06.147448+0000","last_peered":"2023-11-28T18:27:06.147448+0000","last_clean":"2023-11-28T18:27:06.147448+0000","last_became_active":"2023-11-28T18:27:05.129112+0000","last_became_peered":"2023-11-28T18:27:05.129112+0000","last_unstale":"2023-11-28T18:27:06.147448+0000","last_undegraded":"2023-11-28T18:27:06.147448+0000","last_fullsized":"2023-11-28T18:27:06.147448+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:40:44.065622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.151942+0000","last_change":"2023-11-28T18:27:06.129334+0000","last_active":"2023-11-28T18:27:06.151942+0000","last_peered":"2023-11-28T18:27:06.151942+0000","last_clean":"2023-11-28T18:27:06.151942+0000","last_became_active":"2023-11-28T18:27:06.129013+0000","last_became_peered":"2023-11-28T18:27:06.129013+0000","last_unstale":"2023-11-28T18:27:06.151942+0000","last_undegraded":"2023-11-28T18:27:06.151942+0000","last_fullsized":"2023-11-28T18:27:06.151942+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:45:48.134616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"64'6","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.157756+0000","last_change":"2023-11-28T18:27:05.116939+0000","last_active":"2023-11-28T18:27:06.157756+0000","last_peered":"2023-11-28T18:27:06.157756+0000","last_clean":"2023-11-28T18:27:06.157756+0000","last_became_active":"2023-11-28T18:27:05.115980+0000","last_became_peered":"2023-11-28T18:27:05.115980+0000","last_unstale":"2023-11-28T18:27:06.157756+0000","last_undegraded":"2023-11-28T18:27:06.157756+0000","last_fullsized":"2023-11-28T18:27:06.157756+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:00:05.602763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.154265+0000","last_change":"2023-11-28T18:27:06.134129+0000","last_active":"2023-11-28T18:27:06.154265+0000","last_peered":"2023-11-28T18:27:06.154265+0000","last_clean":"2023-11-28T18:27:06.154265+0000","last_became_active":"2023-11-28T18:27:06.133843+0000","last_became_peered":"2023-11-28T18:27:06.133843+0000","last_unstale":"2023-11-28T18:27:06.154265+0000","last_undegraded":"2023-11-28T18:27:06.154265+0000","last_fullsized":"2023-11-28T18:27:06.154265+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:42:25.052676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.142074+0000","last_change":"2023-11-28T18:27:05.125191+0000","last_active":"2023-11-28T18:27:06.142074+0000","last_peered":"2023-11-28T18:27:06.142074+0000","last_clean":"2023-11-28T18:27:06.142074+0000","last_became_active":"2023-11-28T18:27:05.124894+0000","last_became_peered":"2023-11-28T18:27:05.124894+0000","last_unstale":"2023-11-28T18:27:06.142074+0000","last_undegraded":"2023-11-28T18:27:06.142074+0000","last_fullsized":"2023-11-28T18:27:06.142074+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:54:06.723754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.142571+0000","last_change":"2023-11-28T18:27:06.139795+0000","last_active":"2023-11-28T18:27:06.142571+0000","last_peered":"2023-11-28T18:27:06.142571+0000","last_clean":"2023-11-28T18:27:06.142571+0000","last_became_active":"2023-11-28T18:27:06.138920+0000","last_became_peered":"2023-11-28T18:27:06.138920+0000","last_unstale":"2023-11-28T18:27:06.142571+0000","last_undegraded":"2023-11-28T18:27:06.142571+0000","last_fullsized":"2023-11-28T18:27:06.142571+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:10.114405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.117399+0000","last_change":"2023-11-28T18:27:05.119912+0000","last_active":"2023-11-28T18:27:06.117399+0000","last_peered":"2023-11-28T18:27:06.117399+0000","last_clean":"2023-11-28T18:27:06.117399+0000","last_became_active":"2023-11-28T18:27:05.119578+0000","last_became_peered":"2023-11-28T18:27:05.119578+0000","last_unstale":"2023-11-28T18:27:06.117399+0000","last_undegraded":"2023-11-28T18:27:06.117399+0000","last_fullsized":"2023-11-28T18:27:06.117399+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:40:44.065622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.146152+0000","last_change":"2023-11-28T18:27:06.138277+0000","last_active":"2023-11-28T18:27:06.146152+0000","last_peered":"2023-11-28T18:27:06.146152+0000","last_clean":"2023-11-28T18:27:06.146152+0000","last_became_active":"2023-11-28T18:27:06.138012+0000","last_became_peered":"2023-11-28T18:27:06.138012+0000","last_unstale":"2023-11-28T18:27:06.146152+0000","last_undegraded":"2023-11-28T18:27:06.146152+0000","last_fullsized":"2023-11-28T18:27:06.146152+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:32.696569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.122822+0000","last_change":"2023-11-28T18:27:05.124365+0000","last_active":"2023-11-28T18:27:06.122822+0000","last_peered":"2023-11-28T18:27:06.122822+0000","last_clean":"2023-11-28T18:27:06.122822+0000","last_became_active":"2023-11-28T18:27:05.124121+0000","last_became_peered":"2023-11-28T18:27:05.124121+0000","last_unstale":"2023-11-28T18:27:06.122822+0000","last_undegraded":"2023-11-28T18:27:06.122822+0000","last_fullsized":"2023-11-28T18:27:06.122822+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:24:41.870988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.141717+0000","last_change":"2023-11-28T18:27:06.134616+0000","last_active":"2023-11-28T18:27:06.141717+0000","last_peered":"2023-11-28T18:27:06.141717+0000","last_clean":"2023-11-28T18:27:06.141717+0000","last_became_active":"2023-11-28T18:27:06.134340+0000","last_became_peered":"2023-11-28T18:27:06.134340+0000","last_unstale":"2023-11-28T18:27:06.141717+0000","last_undegraded":"2023-11-28T18:27:06.141717+0000","last_fullsized":"2023-11-28T18:27:06.141717+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:44:20.474673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"64'8","reported_seq":19,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.157192+0000","last_change":"2023-11-28T18:27:05.124103+0000","last_active":"2023-11-28T18:27:06.157192+0000","last_peered":"2023-11-28T18:27:06.157192+0000","last_clean":"2023-11-28T18:27:06.157192+0000","last_became_active":"2023-11-28T18:27:05.123733+0000","last_became_peered":"2023-11-28T18:27:05.123733+0000","last_unstale":"2023-11-28T18:27:06.157192+0000","last_undegraded":"2023-11-28T18:27:06.157192+0000","last_fullsized":"2023-11-28T18:27:06.157192+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:17:12.911982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.156835+0000","last_change":"2023-11-28T18:27:06.152999+0000","last_active":"2023-11-28T18:27:06.156835+0000","last_peered":"2023-11-28T18:27:06.156835+0000","last_clean":"2023-11-28T18:27:06.156835+0000","last_became_active":"2023-11-28T18:27:06.152690+0000","last_became_peered":"2023-11-28T18:27:06.152690+0000","last_unstale":"2023-11-28T18:27:06.156835+0000","last_undegraded":"2023-11-28T18:27:06.156835+0000","last_fullsized":"2023-11-28T18:27:06.156835+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:59:07.440975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.144291+0000","last_change":"2023-11-28T18:27:05.119906+0000","last_active":"2023-11-28T18:27:06.144291+0000","last_peered":"2023-11-28T18:27:06.144291+0000","last_clean":"2023-11-28T18:27:06.144291+0000","last_became_active":"2023-11-28T18:27:05.119559+0000","last_became_peered":"2023-11-28T18:27:05.119559+0000","last_unstale":"2023-11-28T18:27:06.144291+0000","last_undegraded":"2023-11-28T18:27:06.144291+0000","last_fullsized":"2023-11-28T18:27:06.144291+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:29.282994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.154525+0000","last_change":"2023-11-28T18:27:06.150196+0000","last_active":"2023-11-28T18:27:06.154525+0000","last_peered":"2023-11-28T18:27:06.154525+0000","last_clean":"2023-11-28T18:27:06.154525+0000","last_became_active":"2023-11-28T18:27:06.149890+0000","last_became_peered":"2023-11-28T18:27:06.149890+0000","last_unstale":"2023-11-28T18:27:06.154525+0000","last_undegraded":"2023-11-28T18:27:06.154525+0000","last_fullsized":"2023-11-28T18:27:06.154525+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:45.390936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.154686+0000","last_change":"2023-11-28T18:27:05.126550+0000","last_active":"2023-11-28T18:27:06.154686+0000","last_peered":"2023-11-28T18:27:06.154686+0000","last_clean":"2023-11-28T18:27:06.154686+0000","last_became_active":"2023-11-28T18:27:05.126178+0000","last_became_peered":"2023-11-28T18:27:05.126178+0000","last_unstale":"2023-11-28T18:27:06.154686+0000","last_undegraded":"2023-11-28T18:27:06.154686+0000","last_fullsized":"2023-11-28T18:27:06.154686+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:54:33.856624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.155269+0000","last_change":"2023-11-28T18:27:06.149602+0000","last_active":"2023-11-28T18:27:06.155269+0000","last_peered":"2023-11-28T18:27:06.155269+0000","last_clean":"2023-11-28T18:27:06.155269+0000","last_became_active":"2023-11-28T18:27:06.149282+0000","last_became_peered":"2023-11-28T18:27:06.149282+0000","last_unstale":"2023-11-28T18:27:06.155269+0000","last_undegraded":"2023-11-28T18:27:06.155269+0000","last_fullsized":"2023-11-28T18:27:06.155269+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:31:37.666806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.145938+0000","last_change":"2023-11-28T18:27:05.129430+0000","last_active":"2023-11-28T18:27:06.145938+0000","last_peered":"2023-11-28T18:27:06.145938+0000","last_clean":"2023-11-28T18:27:06.145938+0000","last_became_active":"2023-11-28T18:27:05.129105+0000","last_became_peered":"2023-11-28T18:27:05.129105+0000","last_unstale":"2023-11-28T18:27:06.145938+0000","last_undegraded":"2023-11-28T18:27:06.145938+0000","last_fullsized":"2023-11-28T18:27:06.145938+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:33:14.262575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.146110+0000","last_change":"2023-11-28T18:27:06.137664+0000","last_active":"2023-11-28T18:27:06.146110+0000","last_peered":"2023-11-28T18:27:06.146110+0000","last_clean":"2023-11-28T18:27:06.146110+0000","last_became_active":"2023-11-28T18:27:06.137361+0000","last_became_peered":"2023-11-28T18:27:06.137361+0000","last_unstale":"2023-11-28T18:27:06.146110+0000","last_undegraded":"2023-11-28T18:27:06.146110+0000","last_fullsized":"2023-11-28T18:27:06.146110+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:05:32.013881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.112933+0000","last_change":"2023-11-28T18:27:05.107466+0000","last_active":"2023-11-28T18:27:06.112933+0000","last_peered":"2023-11-28T18:27:06.112933+0000","last_clean":"2023-11-28T18:27:06.112933+0000","last_became_active":"2023-11-28T18:27:05.107194+0000","last_became_peered":"2023-11-28T18:27:05.107194+0000","last_unstale":"2023-11-28T18:27:06.112933+0000","last_undegraded":"2023-11-28T18:27:06.112933+0000","last_fullsized":"2023-11-28T18:27:06.112933+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:07:59.431923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.144927+0000","last_change":"2023-11-28T18:27:06.134727+0000","last_active":"2023-11-28T18:27:06.144927+0000","last_peered":"2023-11-28T18:27:06.144927+0000","last_clean":"2023-11-28T18:27:06.144927+0000","last_became_active":"2023-11-28T18:27:06.134342+0000","last_became_peered":"2023-11-28T18:27:06.134342+0000","last_unstale":"2023-11-28T18:27:06.144927+0000","last_undegraded":"2023-11-28T18:27:06.144927+0000","last_fullsized":"2023-11-28T18:27:06.144927+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:08:00.437625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.146775+0000","last_change":"2023-11-28T18:27:05.124788+0000","last_active":"2023-11-28T18:27:06.146775+0000","last_peered":"2023-11-28T18:27:06.146775+0000","last_clean":"2023-11-28T18:27:06.146775+0000","last_became_active":"2023-11-28T18:27:05.124319+0000","last_became_peered":"2023-11-28T18:27:05.124319+0000","last_unstale":"2023-11-28T18:27:06.146775+0000","last_undegraded":"2023-11-28T18:27:06.146775+0000","last_fullsized":"2023-11-28T18:27:06.146775+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:23.451635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.139900+0000","last_change":"2023-11-28T18:27:06.135266+0000","last_active":"2023-11-28T18:27:06.139900+0000","last_peered":"2023-11-28T18:27:06.139900+0000","last_clean":"2023-11-28T18:27:06.139900+0000","last_became_active":"2023-11-28T18:27:06.134624+0000","last_became_peered":"2023-11-28T18:27:06.134624+0000","last_unstale":"2023-11-28T18:27:06.139900+0000","last_undegraded":"2023-11-28T18:27:06.139900+0000","last_fullsized":"2023-11-28T18:27:06.139900+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:00:06.608464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.117528+0000","last_change":"2023-11-28T18:27:05.115691+0000","last_active":"2023-11-28T18:27:06.117528+0000","last_peered":"2023-11-28T18:27:06.117528+0000","last_clean":"2023-11-28T18:27:06.117528+0000","last_became_active":"2023-11-28T18:27:05.115379+0000","last_became_peered":"2023-11-28T18:27:05.115379+0000","last_unstale":"2023-11-28T18:27:06.117528+0000","last_undegraded":"2023-11-28T18:27:06.117528+0000","last_fullsized":"2023-11-28T18:27:06.117528+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:54:33.856624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.150447+0000","last_change":"2023-11-28T18:27:06.146058+0000","last_active":"2023-11-28T18:27:06.150447+0000","last_peered":"2023-11-28T18:27:06.150447+0000","last_clean":"2023-11-28T18:27:06.150447+0000","last_became_active":"2023-11-28T18:27:06.145771+0000","last_became_peered":"2023-11-28T18:27:06.145771+0000","last_unstale":"2023-11-28T18:27:06.150447+0000","last_undegraded":"2023-11-28T18:27:06.150447+0000","last_fullsized":"2023-11-28T18:27:06.150447+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:31:37.666806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.124291+0000","last_change":"2023-11-28T18:27:05.126023+0000","last_active":"2023-11-28T18:27:06.124291+0000","last_peered":"2023-11-28T18:27:06.124291+0000","last_clean":"2023-11-28T18:27:06.124291+0000","last_became_active":"2023-11-28T18:27:05.125654+0000","last_became_peered":"2023-11-28T18:27:05.125654+0000","last_unstale":"2023-11-28T18:27:06.124291+0000","last_undegraded":"2023-11-28T18:27:06.124291+0000","last_fullsized":"2023-11-28T18:27:06.124291+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:18.615768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.144983+0000","last_change":"2023-11-28T18:27:06.135584+0000","last_active":"2023-11-28T18:27:06.144983+0000","last_peered":"2023-11-28T18:27:06.144983+0000","last_clean":"2023-11-28T18:27:06.144983+0000","last_became_active":"2023-11-28T18:27:06.135320+0000","last_became_peered":"2023-11-28T18:27:06.135320+0000","last_unstale":"2023-11-28T18:27:06.144983+0000","last_undegraded":"2023-11-28T18:27:06.144983+0000","last_fullsized":"2023-11-28T18:27:06.144983+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:36:38.407270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.136123+0000","last_change":"2023-11-28T18:27:05.123131+0000","last_active":"2023-11-28T18:27:06.136123+0000","last_peered":"2023-11-28T18:27:06.136123+0000","last_clean":"2023-11-28T18:27:06.136123+0000","last_became_active":"2023-11-28T18:27:05.122836+0000","last_became_peered":"2023-11-28T18:27:05.122836+0000","last_unstale":"2023-11-28T18:27:06.136123+0000","last_undegraded":"2023-11-28T18:27:06.136123+0000","last_fullsized":"2023-11-28T18:27:06.136123+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:59:06.435274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.148041+0000","last_change":"2023-11-28T18:27:06.143560+0000","last_active":"2023-11-28T18:27:06.148041+0000","last_peered":"2023-11-28T18:27:06.148041+0000","last_clean":"2023-11-28T18:27:06.148041+0000","last_became_active":"2023-11-28T18:27:06.143297+0000","last_became_peered":"2023-11-28T18:27:06.143297+0000","last_unstale":"2023-11-28T18:27:06.148041+0000","last_undegraded":"2023-11-28T18:27:06.148041+0000","last_fullsized":"2023-11-28T18:27:06.148041+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:52.393749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.118879+0000","last_change":"2023-11-28T18:27:05.119212+0000","last_active":"2023-11-28T18:27:06.118879+0000","last_peered":"2023-11-28T18:27:06.118879+0000","last_clean":"2023-11-28T18:27:06.118879+0000","last_became_active":"2023-11-28T18:27:05.118919+0000","last_became_peered":"2023-11-28T18:27:05.118919+0000","last_unstale":"2023-11-28T18:27:06.118879+0000","last_undegraded":"2023-11-28T18:27:06.118879+0000","last_fullsized":"2023-11-28T18:27:06.118879+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:33:14.262575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.142271+0000","last_change":"2023-11-28T18:27:06.139708+0000","last_active":"2023-11-28T18:27:06.142271+0000","last_peered":"2023-11-28T18:27:06.142271+0000","last_clean":"2023-11-28T18:27:06.142271+0000","last_became_active":"2023-11-28T18:27:06.139444+0000","last_became_peered":"2023-11-28T18:27:06.139444+0000","last_unstale":"2023-11-28T18:27:06.142271+0000","last_undegraded":"2023-11-28T18:27:06.142271+0000","last_fullsized":"2023-11-28T18:27:06.142271+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:35:51.815845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.128602+0000","last_change":"2023-11-28T18:27:05.121664+0000","last_active":"2023-11-28T18:27:06.128602+0000","last_peered":"2023-11-28T18:27:06.128602+0000","last_clean":"2023-11-28T18:27:06.128602+0000","last_became_active":"2023-11-28T18:27:05.121386+0000","last_became_peered":"2023-11-28T18:27:05.121386+0000","last_unstale":"2023-11-28T18:27:06.128602+0000","last_undegraded":"2023-11-28T18:27:06.128602+0000","last_fullsized":"2023-11-28T18:27:06.128602+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:42.302881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.146130+0000","last_change":"2023-11-28T18:27:06.137452+0000","last_active":"2023-11-28T18:27:06.146130+0000","last_peered":"2023-11-28T18:27:06.146130+0000","last_clean":"2023-11-28T18:27:06.146130+0000","last_became_active":"2023-11-28T18:27:06.136976+0000","last_became_peered":"2023-11-28T18:27:06.136976+0000","last_unstale":"2023-11-28T18:27:06.146130+0000","last_undegraded":"2023-11-28T18:27:06.146130+0000","last_fullsized":"2023-11-28T18:27:06.146130+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:17:13.917683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.123463+0000","last_change":"2023-11-28T18:27:05.116737+0000","last_active":"2023-11-28T18:27:06.123463+0000","last_peered":"2023-11-28T18:27:06.123463+0000","last_clean":"2023-11-28T18:27:06.123463+0000","last_became_active":"2023-11-28T18:27:05.115610+0000","last_became_peered":"2023-11-28T18:27:05.115610+0000","last_unstale":"2023-11-28T18:27:06.123463+0000","last_undegraded":"2023-11-28T18:27:06.123463+0000","last_fullsized":"2023-11-28T18:27:06.123463+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:09.108703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.148046+0000","last_change":"2023-11-28T18:27:06.144755+0000","last_active":"2023-11-28T18:27:06.148046+0000","last_peered":"2023-11-28T18:27:06.148046+0000","last_clean":"2023-11-28T18:27:06.148046+0000","last_became_active":"2023-11-28T18:27:06.144442+0000","last_became_peered":"2023-11-28T18:27:06.144442+0000","last_unstale":"2023-11-28T18:27:06.148046+0000","last_undegraded":"2023-11-28T18:27:06.148046+0000","last_fullsized":"2023-11-28T18:27:06.148046+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:25.132096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.122366+0000","last_change":"2023-11-28T18:27:05.125243+0000","last_active":"2023-11-28T18:27:06.122366+0000","last_peered":"2023-11-28T18:27:06.122366+0000","last_clean":"2023-11-28T18:27:06.122366+0000","last_became_active":"2023-11-28T18:27:05.124974+0000","last_became_peered":"2023-11-28T18:27:05.124974+0000","last_unstale":"2023-11-28T18:27:06.122366+0000","last_undegraded":"2023-11-28T18:27:06.122366+0000","last_fullsized":"2023-11-28T18:27:06.122366+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:35:50.810143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.154422+0000","last_change":"2023-11-28T18:27:06.134299+0000","last_active":"2023-11-28T18:27:06.154422+0000","last_peered":"2023-11-28T18:27:06.154422+0000","last_clean":"2023-11-28T18:27:06.154422+0000","last_became_active":"2023-11-28T18:27:06.134015+0000","last_became_peered":"2023-11-28T18:27:06.134015+0000","last_unstale":"2023-11-28T18:27:06.154422+0000","last_undegraded":"2023-11-28T18:27:06.154422+0000","last_fullsized":"2023-11-28T18:27:06.154422+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:25.132096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.112375+0000","last_change":"2023-11-28T18:27:05.106005+0000","last_active":"2023-11-28T18:27:06.112375+0000","last_peered":"2023-11-28T18:27:06.112375+0000","last_clean":"2023-11-28T18:27:06.112375+0000","last_became_active":"2023-11-28T18:27:05.105732+0000","last_became_peered":"2023-11-28T18:27:05.105732+0000","last_unstale":"2023-11-28T18:27:06.112375+0000","last_undegraded":"2023-11-28T18:27:06.112375+0000","last_fullsized":"2023-11-28T18:27:06.112375+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:59:06.435274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.152171+0000","last_change":"2023-11-28T18:27:06.149503+0000","last_active":"2023-11-28T18:27:06.152171+0000","last_peered":"2023-11-28T18:27:06.152171+0000","last_clean":"2023-11-28T18:27:06.152171+0000","last_became_active":"2023-11-28T18:27:06.149225+0000","last_became_peered":"2023-11-28T18:27:06.149225+0000","last_unstale":"2023-11-28T18:27:06.152171+0000","last_undegraded":"2023-11-28T18:27:06.152171+0000","last_fullsized":"2023-11-28T18:27:06.152171+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:51.714834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.112662+0000","last_change":"2023-11-28T18:27:05.111957+0000","last_active":"2023-11-28T18:27:06.112662+0000","last_peered":"2023-11-28T18:27:06.112662+0000","last_clean":"2023-11-28T18:27:06.112662+0000","last_became_active":"2023-11-28T18:27:05.111661+0000","last_became_peered":"2023-11-28T18:27:05.111661+0000","last_unstale":"2023-11-28T18:27:06.112662+0000","last_undegraded":"2023-11-28T18:27:06.112662+0000","last_fullsized":"2023-11-28T18:27:06.112662+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:49:15.264841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.146160+0000","last_change":"2023-11-28T18:27:06.142958+0000","last_active":"2023-11-28T18:27:06.146160+0000","last_peered":"2023-11-28T18:27:06.146160+0000","last_clean":"2023-11-28T18:27:06.146160+0000","last_became_active":"2023-11-28T18:27:06.142596+0000","last_became_peered":"2023-11-28T18:27:06.142596+0000","last_unstale":"2023-11-28T18:27:06.146160+0000","last_undegraded":"2023-11-28T18:27:06.146160+0000","last_fullsized":"2023-11-28T18:27:06.146160+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:57:16.137353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.119813+0000","last_change":"2023-11-28T18:27:05.110708+0000","last_active":"2023-11-28T18:27:06.119813+0000","last_peered":"2023-11-28T18:27:06.119813+0000","last_clean":"2023-11-28T18:27:06.119813+0000","last_became_active":"2023-11-28T18:27:05.110436+0000","last_became_peered":"2023-11-28T18:27:05.110436+0000","last_unstale":"2023-11-28T18:27:06.119813+0000","last_undegraded":"2023-11-28T18:27:06.119813+0000","last_fullsized":"2023-11-28T18:27:06.119813+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:33:16.579076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.145980+0000","last_change":"2023-11-28T18:27:06.143100+0000","last_active":"2023-11-28T18:27:06.145980+0000","last_peered":"2023-11-28T18:27:06.145980+0000","last_clean":"2023-11-28T18:27:06.145980+0000","last_became_active":"2023-11-28T18:27:06.142615+0000","last_became_peered":"2023-11-28T18:27:06.142615+0000","last_unstale":"2023-11-28T18:27:06.145980+0000","last_undegraded":"2023-11-28T18:27:06.145980+0000","last_fullsized":"2023-11-28T18:27:06.145980+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:42:25.052676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.111862+0000","last_change":"2023-11-28T18:27:05.116233+0000","last_active":"2023-11-28T18:27:06.111862+0000","last_peered":"2023-11-28T18:27:06.111862+0000","last_clean":"2023-11-28T18:27:06.111862+0000","last_became_active":"2023-11-28T18:27:05.115934+0000","last_became_peered":"2023-11-28T18:27:05.115934+0000","last_unstale":"2023-11-28T18:27:06.111862+0000","last_undegraded":"2023-11-28T18:27:06.111862+0000","last_fullsized":"2023-11-28T18:27:06.111862+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:54:33.856624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":202,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T18:27:05.117105+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:27:05.117105+0000","last_peered":"2023-11-28T18:27:05.117105+0000","last_clean":"2023-11-28T18:27:05.117105+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:27:05.117105+0000","last_undegraded":"2023-11-28T18:27:05.117105+0000","last_fullsized":"2023-11-28T18:27:05.117105+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36692,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":1384448,"data_stored":941063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.68899999999999995,"15min":0.72899999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.98099999999999998,"5min":2.427,"15min":2.427},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.69699999999999995,"15min":0.73099999999999998},"min":{"1min":0.504,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91700000000000004,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.60599999999999998}]},{"osd":1,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.69699999999999995,"15min":0.72699999999999998},"min":{"1min":0.44500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.96099999999999997,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.73299999999999998,"15min":0.76700000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86399999999999999,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.623}]},{"osd":2,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.752,"15min":0.78600000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94599999999999995,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.439},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.71599999999999997,"15min":0.751},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86399999999999999,"5min":2.411,"15min":2.411},"last":0.68500000000000005}]},{"osd":3,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.73699999999999999,"15min":0.77500000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.87,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.77800000000000002,"15min":0.81399999999999995},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.92900000000000005,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":0.495}]},{"osd":4,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.36299999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77000000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.623,"15min":0.64800000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75700000000000001,"5min":0.877,"15min":0.877},"last":0.46600000000000003}]},{"osd":5,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.66100000000000003,"15min":0.68100000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89700000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.68300000000000005,"15min":0.70699999999999996},"min":{"1min":0.441,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82499999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.71099999999999997}]},{"osd":6,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.72599999999999998,"15min":0.749},"min":{"1min":0.48799999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.85099999999999998,"5min":1.042,"15min":1.042},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.70799999999999996,"15min":0.74199999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83999999999999997,"5min":1.006,"15min":1.006},"last":0.67500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":761856,"data_stored":342824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.90700000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72099999999999997,"15min":0.73299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.91400000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.70899999999999996}]},{"osd":1,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.77300000000000002,"15min":0.82499999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.879,"5min":4.2389999999999999,"15min":4.2389999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.76600000000000001,"15min":0.80500000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.88400000000000001,"5min":4.218,"15min":4.218},"last":0.61899999999999999}]},{"osd":2,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70099999999999996,"15min":0.72099999999999997},"min":{"1min":0.45100000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85599999999999998,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69999999999999996,"15min":0.70799999999999996},"min":{"1min":0.47599999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90000000000000002,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.65200000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.71499999999999997,"15min":0.73099999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88200000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68899999999999995,"15min":0.68999999999999995},"min":{"1min":0.53000000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.82899999999999996,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.52500000000000002}]},{"osd":4,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.66500000000000004,"15min":0.67600000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84599999999999997,"5min":0.995,"15min":0.995},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.66500000000000004,"15min":0.69299999999999995},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.93600000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.67200000000000004}]},{"osd":5,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.66900000000000004,"15min":0.68799999999999994},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.436},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.749,"15min":0.77000000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.89100000000000001,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.51400000000000001}]},{"osd":7,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.755,"15min":0.78100000000000003},"min":{"1min":0.45100000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90900000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.495},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.73899999999999999,"15min":0.75700000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.86699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.63800000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":761856,"data_stored":342824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71599999999999997,"15min":0.71899999999999997},"min":{"1min":0.54300000000000004,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.999,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74099999999999999,"15min":0.75900000000000001},"min":{"1min":0.55600000000000005,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.97799999999999998,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":0.66300000000000003}]},{"osd":1,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73199999999999998,"15min":0.75},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93000000000000005,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.77900000000000003,"15min":0.81200000000000006},"min":{"1min":0.57699999999999996,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.022,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.53300000000000003}]},{"osd":2,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.74199999999999999,"15min":0.77000000000000002},"min":{"1min":0.44400000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93200000000000005,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.753,"15min":0.78700000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86199999999999999,"5min":4.0149999999999997,"15min":4.0149999999999997},"last":0.56399999999999995}]},{"osd":3,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.80700000000000005,"15min":0.84299999999999997},"min":{"1min":0.56200000000000006,"5min":0.495,"15min":0.495},"max":{"1min":0.95599999999999996,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.78800000000000003,"15min":0.83999999999999997},"min":{"1min":0.51800000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96999999999999997,"5min":3.9900000000000002,"15min":3.9900000000000002},"last":0.59899999999999998}]},{"osd":4,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80600000000000005,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70299999999999996,"15min":0.71399999999999997},"min":{"1min":0.437,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.98799999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.58199999999999996}]},{"osd":6,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.41899999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91200000000000003,"5min":1.071,"15min":1.071},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71399999999999997,"15min":0.71999999999999997},"min":{"1min":0.52500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86199999999999999,"5min":1.756,"15min":1.756},"last":0.52100000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.752,"15min":0.75800000000000001},"min":{"1min":0.54200000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.99299999999999999,"5min":1.282,"15min":1.282},"last":0.628},{"interface":"front","average":{"1min":0.745,"5min":0.753,"15min":0.75800000000000001},"min":{"1min":0.51100000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.96899999999999997,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.72099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":761856,"data_stored":342824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66900000000000004,"15min":0.69399999999999995},"min":{"1min":0.41999999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.82999999999999996,"5min":2.363,"15min":2.363},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.67900000000000005}]},{"osd":1,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63,"15min":0.623},"min":{"1min":0.49299999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.86099999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.65400000000000003,"15min":0.65000000000000002},"min":{"1min":0.42799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85799999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.76400000000000001}]},{"osd":2,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.49199999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72199999999999998,"15min":0.72999999999999998},"min":{"1min":0.504,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91900000000000004,"5min":1.871,"15min":1.871},"last":0.72799999999999998}]},{"osd":3,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.495,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.872,"5min":0.878,"15min":0.878},"last":0.745},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.51600000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.89600000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.69999999999999996}]},{"osd":5,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.41499999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81200000000000006,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.629,"15min":0.64300000000000002},"min":{"1min":0.442,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90500000000000003,"5min":1.79,"15min":1.79},"last":0.56699999999999995}]},{"osd":6,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.63700000000000001,"15min":0.64900000000000002},"min":{"1min":0.41599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88600000000000001,"5min":1.123,"15min":1.123},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.69899999999999995,"15min":0.70999999999999996},"min":{"1min":0.55900000000000005,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89200000000000002,"5min":1.925,"15min":1.925},"last":0.85399999999999998}]},{"osd":7,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70499999999999996,"15min":0.72099999999999997},"min":{"1min":0.48499999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88100000000000001,"5min":1.843,"15min":1.843},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.76500000000000001,"15min":0.78000000000000003},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.91600000000000004,"5min":1.593,"15min":1.593},"last":0.87}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40248,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331592,"statfs":{"total":96636764160,"available":96595550208,"internally_reserved":0,"allocated":761856,"data_stored":342824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.41699999999999998,"15min":0.41099999999999998},"min":{"1min":0.30499999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":2.0169999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.46100000000000002,"15min":0.46500000000000002},"min":{"1min":0.33000000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73899999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":2.7080000000000002}]},{"osd":1,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.36099999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.64600000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.5,"15min":0.501},"min":{"1min":0.376,"5min":0.247,"15min":0.247},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":4.5599999999999996}]},{"osd":2,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51300000000000001,"15min":0.51900000000000002},"min":{"1min":0.35499999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79800000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":2.48},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81200000000000006,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":2.2919999999999998}]},{"osd":4,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.76000000000000001,"15min":0.77000000000000002},"min":{"1min":0.53800000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1060000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":12.695},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76200000000000001,"15min":0.77100000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0840000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":6.3250000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.81599999999999995,"15min":0.83799999999999997},"min":{"1min":0.56399999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2809999999999999,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.9430000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.57299999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2390000000000001,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":2.5760000000000001}]},{"osd":6,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.79400000000000004,"15min":0.80100000000000005},"min":{"1min":0.58299999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.173,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":6.1790000000000003},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.79700000000000004,"15min":0.80300000000000005},"min":{"1min":0.55200000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.246,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.984}]},{"osd":7,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.80200000000000005,"15min":0.81499999999999995},"min":{"1min":0.56999999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1080000000000001,"5min":1.786,"15min":1.786},"last":3.1709999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1419999999999999,"5min":1.337,"15min":1.337},"last":6.2249999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31940,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":774144,"data_stored":349919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.48899999999999999,"15min":0.504},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67600000000000005,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.48599999999999999,"15min":0.50800000000000001},"min":{"1min":0.33700000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.64100000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.46100000000000002}]},{"osd":1,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.51400000000000001,"15min":0.52800000000000002},"min":{"1min":0.315,"5min":0.308,"15min":0.308},"max":{"1min":0.59699999999999998,"5min":1.48,"15min":1.48},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51000000000000001,"15min":0.51500000000000001},"min":{"1min":0.39700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84399999999999997,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.51300000000000001}]},{"osd":3,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.48899999999999999,"15min":0.503},"min":{"1min":0.35699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.62,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71299999999999997,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.58699999999999997}]},{"osd":4,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.77500000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1459999999999999,"5min":1.512,"15min":1.512},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76100000000000001,"15min":0.77900000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.996,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.75800000000000001}]},{"osd":5,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.75,"15min":0.76700000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.85999999999999999,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.93899999999999995,"15min":1.014},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0389999999999999,"5min":7.4820000000000002,"15min":7.4820000000000002},"last":0.79100000000000004}]},{"osd":6,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78200000000000003,"15min":0.80000000000000004},"min":{"1min":0.55000000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.97299999999999998,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.82299999999999995,"15min":0.871},"min":{"1min":0.53000000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.94699999999999995,"5min":3.9380000000000002,"15min":3.9380000000000002},"last":0.73899999999999999}]},{"osd":7,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.81899999999999995,"15min":0.84199999999999997},"min":{"1min":0.56299999999999994,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.032,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.86499999999999999,"15min":0.92100000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.002,"5min":4.798,"15min":4.798},"last":0.77500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":802816,"data_stored":352295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:26:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.41699999999999998,"15min":0.42699999999999999},"min":{"1min":0.25900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.373},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44600000000000001,"15min":0.45000000000000001},"min":{"1min":0.28100000000000003,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.60599999999999998,"5min":0.62,"15min":0.62},"last":0.48299999999999998}]},{"osd":2,"last update":"Tue Nov 28 18:26:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.441,"15min":0.44700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.55700000000000005,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47899999999999998,"15min":0.47799999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.67000000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.45900000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:26:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75600000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.38600000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.42799999999999999}]},{"osd":4,"last update":"Tue Nov 28 18:26:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72699999999999998,"15min":0.747},"min":{"1min":0.41099999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98599999999999999,"5min":1.512,"15min":1.512},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.75600000000000001,"15min":0.76900000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92400000000000004,"5min":1.155,"15min":1.155},"last":1.0780000000000001}]},{"osd":5,"last update":"Tue Nov 28 18:26:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.72999999999999998,"15min":0.75},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91000000000000003,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.72299999999999998,"15min":0.748},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.83999999999999997,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.90200000000000002}]},{"osd":6,"last update":"Tue Nov 28 18:26:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.753,"15min":0.77700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.85399999999999998,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.78300000000000003,"15min":0.80800000000000005},"min":{"1min":0.51400000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.88900000000000001,"5min":1.448,"15min":1.448},"last":1.0429999999999999}]},{"osd":7,"last update":"Tue Nov 28 18:26:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.73799999999999999,"15min":0.752},"min":{"1min":0.54600000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.82199999999999995,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.75700000000000001,"15min":0.77800000000000002},"min":{"1min":0.57199999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90900000000000003,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.0109999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1368064,"data_stored":940287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.438,"15min":0.44600000000000001},"min":{"1min":0.32000000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.46800000000000003,"15min":0.47199999999999998},"min":{"1min":0.314,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64600000000000002,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.46999999999999997}]},{"osd":2,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.503,"15min":0.51100000000000001},"min":{"1min":0.371,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.60899999999999999,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.505,"15min":0.51700000000000002},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77800000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.35799999999999998}]},{"osd":3,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.33300000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.67500000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.54400000000000004,"15min":0.55600000000000005},"min":{"1min":0.40500000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80000000000000004,"5min":1.087,"15min":1.087},"last":0.45200000000000001}]},{"osd":4,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.49299999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.56899999999999995,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.996,"5min":1.244,"15min":1.244},"last":0.64500000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.52300000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0489999999999999,"5min":1.121,"15min":1.121},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.748,"15min":0.74399999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0329999999999999,"5min":1.075,"15min":1.075},"last":0.57199999999999995}]},{"osd":6,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.73699999999999999,"15min":0.73599999999999999},"min":{"1min":0.52500000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1140000000000001,"5min":1.407,"15min":1.407},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.749,"15min":0.73899999999999999},"min":{"1min":0.56899999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.087,"5min":1.089,"15min":1.089},"last":0.66300000000000003}]},{"osd":7,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0089999999999999,"5min":1.212,"15min":1.212},"last":0.626},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.75600000000000001,"15min":0.751},"min":{"1min":0.56000000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.67800000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:27:10.193 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:27:10.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.363+0000 7fc4dc765640 1 Processor -- start 2023-11-28T18:27:10.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.363+0000 7fc4dc765640 1 -- start start 2023-11-28T18:27:10.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.363+0000 7fc4dc765640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc4d40b3d30 0x7fc4d41680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:10.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.363+0000 7fc4dc765640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d40b25b0 con 0x7fc4d415d4f0 2023-11-28T18:27:10.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.363+0000 7fc4dc765640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d40d1c30 con 0x7fc4d40d3c30 2023-11-28T18:27:10.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.363+0000 7fc4dc765640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d40b2900 con 0x7fc4d40b3d30 2023-11-28T18:27:10.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.363+0000 7fc4dad02640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc4d40b3d30 0x7fc4d41680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:10.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.363+0000 7fc4dad02640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc4d40b3d30 0x7fc4d41680a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44266/0 (socket says 172.21.15.105:44266) 2023-11-28T18:27:10.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.363+0000 7fc4dad02640 1 -- 172.21.15.105:0/4036018658 learned_addr learned my addr 172.21.15.105:0/4036018658 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:10.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.363+0000 7fc4dad02640 1 -- 172.21.15.105:0/4036018658 >> v1:172.21.15.148:6789/0 conn(0x7fc4d40d3c30 legacy=0x7fc4d40d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:10.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.363+0000 7fc4dad02640 1 -- 172.21.15.105:0/4036018658 >> v1:172.21.15.105:6789/0 conn(0x7fc4d415d4f0 legacy=0x7fc4d40d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:10.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.363+0000 7fc4d94ff640 1 -- 172.21.15.105:0/4036018658 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2272290856 0 0) 0x7fc4d40b25b0 con 0x7fc4d415d4f0 2023-11-28T18:27:10.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.363+0000 7fc4dad02640 1 -- 172.21.15.105:0/4036018658 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d40cea30 con 0x7fc4d40b3d30 2023-11-28T18:27:10.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.367+0000 7fc4dad02640 1 --2- 172.21.15.105:0/4036018658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4d40b3d30 0x7fc4d41680a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009e50 tx=0x7fc4cc01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b6da9570b87332 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:10.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.367+0000 7fc4d94ff640 1 -- 172.21.15.105:0/4036018658 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc0a5780 con 0x7fc4d40b3d30 2023-11-28T18:27:10.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.367+0000 7fc4d94ff640 1 -- 172.21.15.105:0/4036018658 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4cc0a58e0 con 0x7fc4d40b3d30 2023-11-28T18:27:10.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.367+0000 7fc4d94ff640 1 -- 172.21.15.105:0/4036018658 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc0a5c70 con 0x7fc4d40b3d30 2023-11-28T18:27:10.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.367+0000 7fc4dc765640 1 -- 172.21.15.105:0/4036018658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4d40b3d30 msgr2=0x7fc4d41680a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:10.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.367+0000 7fc4dc765640 1 --2- 172.21.15.105:0/4036018658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4d40b3d30 0x7fc4d41680a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009e50 tx=0x7fc4cc01c8c0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.367+0000 7fc4dc765640 1 -- 172.21.15.105:0/4036018658 shutdown_connections 2023-11-28T18:27:10.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.367+0000 7fc4dc765640 1 --2- 172.21.15.105:0/4036018658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4d40b3d30 0x7fc4d41680a0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.367+0000 7fc4dc765640 1 -- 172.21.15.105:0/4036018658 >> 172.21.15.105:0/4036018658 conn(0x7fc4d40d3030 msgr2=0x7fc4d40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:10.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.367+0000 7fc4dc765640 1 -- 172.21.15.105:0/4036018658 shutdown_connections 2023-11-28T18:27:10.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.367+0000 7fc4dc765640 1 -- 172.21.15.105:0/4036018658 wait complete. 2023-11-28T18:27:10.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.367+0000 7fc4dc765640 1 Processor -- start 2023-11-28T18:27:10.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4dc765640 1 -- start start 2023-11-28T18:27:10.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4dc765640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4d40b3d30 0x7fc4d40cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:10.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4dc765640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4d40d3c30 0x7fc4d41547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:10.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4dc765640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc4d415d4f0 0x7fc4d4159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:10.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4dc765640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d40d0a20 con 0x7fc4d40d3c30 2023-11-28T18:27:10.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4dc765640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d40cc160 con 0x7fc4d40b3d30 2023-11-28T18:27:10.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4dc765640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d40cb900 con 0x7fc4d415d4f0 2023-11-28T18:27:10.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4d9d00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4d40d3c30 0x7fc4d41547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4d9d00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4d40d3c30 0x7fc4d41547d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44278/0 (socket says 172.21.15.105:44278) 2023-11-28T18:27:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4d9d00640 1 -- 172.21.15.105:0/372424694 learned_addr learned my addr 172.21.15.105:0/372424694 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4da501640 1 --2- 172.21.15.105:0/372424694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4d40b3d30 0x7fc4d40cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4dad02640 1 --2- 172.21.15.105:0/372424694 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc4d415d4f0 0x7fc4d4159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4d9d00640 1 -- 172.21.15.105:0/372424694 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc4d415d4f0 msgr2=0x7fc4d4159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4d9d00640 1 --2- 172.21.15.105:0/372424694 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc4d415d4f0 0x7fc4d4159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4d9d00640 1 -- 172.21.15.105:0/372424694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4d40b3d30 msgr2=0x7fc4d40cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4d9d00640 1 --2- 172.21.15.105:0/372424694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4d40b3d30 0x7fc4d40cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4d9d00640 1 -- 172.21.15.105:0/372424694 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc009b30 con 0x7fc4d40d3c30 2023-11-28T18:27:10.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4da501640 1 --2- 172.21.15.105:0/372424694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4d40b3d30 0x7fc4d40cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:10.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4dad02640 1 --2- 172.21.15.105:0/372424694 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc4d415d4f0 0x7fc4d4159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:10.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4d9d00640 1 --2- 172.21.15.105:0/372424694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4d40d3c30 0x7fc4d41547d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400c3c0 tx=0x7fc4c400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:10.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4caffd640 1 -- 172.21.15.105:0/372424694 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c40052d0 con 0x7fc4d40d3c30 2023-11-28T18:27:10.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4caffd640 1 -- 172.21.15.105:0/372424694 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c4005430 con 0x7fc4d40d3c30 2023-11-28T18:27:10.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4dc765640 1 -- 172.21.15.105:0/372424694 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d40cbba0 con 0x7fc4d40d3c30 2023-11-28T18:27:10.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4caffd640 1 -- 172.21.15.105:0/372424694 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4003840 con 0x7fc4d40d3c30 2023-11-28T18:27:10.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4dc765640 1 -- 172.21.15.105:0/372424694 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d40b6760 con 0x7fc4d40d3c30 2023-11-28T18:27:10.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.371+0000 7fc4dc765640 1 -- 172.21.15.105:0/372424694 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d40d0850 con 0x7fc4d40d3c30 2023-11-28T18:27:10.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.375+0000 7fc4caffd640 1 -- 172.21.15.105:0/372424694 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc4c40039a0 con 0x7fc4d40d3c30 2023-11-28T18:27:10.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.375+0000 7fc4caffd640 1 --2- 172.21.15.105:0/372424694 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc4b80668d0 0x7fc4b8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:10.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.375+0000 7fc4caffd640 1 -- 172.21.15.105:0/372424694 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc4c4083b70 con 0x7fc4d40d3c30 2023-11-28T18:27:10.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.375+0000 7fc4da501640 1 --2- 172.21.15.105:0/372424694 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc4b80668d0 0x7fc4b8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:10.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.375+0000 7fc4da501640 1 --2- 172.21.15.105:0/372424694 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc4b80668d0 0x7fc4b8068d90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc4d415c450 tx=0x7fc4d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:10.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.379+0000 7fc4caffd640 1 -- 172.21.15.105:0/372424694 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc4c4050bd0 con 0x7fc4d40d3c30 2023-11-28T18:27:10.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.527+0000 7fc4dc765640 1 -- 172.21.15.105:0/372424694 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc4d416ae00 con 0x7fc4b80668d0 2023-11-28T18:27:10.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.535+0000 7fc4caffd640 1 -- 172.21.15.105:0/372424694 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289322 (secure 0 0 0) 0x7fc4d0001330 con 0x7fc4b80668d0 2023-11-28T18:27:10.538 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:27:10.541 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:27:10.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.535+0000 7fc4dc765640 1 -- 172.21.15.105:0/372424694 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc4b80668d0 msgr2=0x7fc4b8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:10.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.535+0000 7fc4dc765640 1 --2- 172.21.15.105:0/372424694 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc4b80668d0 0x7fc4b8068d90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc4d415c450 tx=0x7fc4d0009040 comp rx=0 tx=0).stop 2023-11-28T18:27:10.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.535+0000 7fc4dc765640 1 -- 172.21.15.105:0/372424694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4d40d3c30 msgr2=0x7fc4d41547d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:10.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.535+0000 7fc4dc765640 1 --2- 172.21.15.105:0/372424694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4d40d3c30 0x7fc4d41547d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400c3c0 tx=0x7fc4c400c880 comp rx=0 tx=0).stop 2023-11-28T18:27:10.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.535+0000 7fc4dc765640 1 -- 172.21.15.105:0/372424694 shutdown_connections 2023-11-28T18:27:10.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.535+0000 7fc4dc765640 1 --2- 172.21.15.105:0/372424694 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc4d415d4f0 0x7fc4d4159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.535+0000 7fc4dc765640 1 --2- 172.21.15.105:0/372424694 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc4b80668d0 0x7fc4b8068d90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.535+0000 7fc4dc765640 1 --2- 172.21.15.105:0/372424694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4d40d3c30 0x7fc4d41547d0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.535+0000 7fc4dc765640 1 --2- 172.21.15.105:0/372424694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4d40b3d30 0x7fc4d40cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.539+0000 7fc4dc765640 1 -- 172.21.15.105:0/372424694 >> 172.21.15.105:0/372424694 conn(0x7fc4d40d3030 msgr2=0x7fc4d406c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:10.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.539+0000 7fc4dc765640 1 -- 172.21.15.105:0/372424694 shutdown_connections 2023-11-28T18:27:10.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.539+0000 7fc4dc765640 1 -- 172.21.15.105:0/372424694 wait complete. 2023-11-28T18:27:10.566 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2023-11-28T18:27:09.315404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":269132,"kb_used_data":7204,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754705588,"statfs":{"total":773094113280,"available":772818522112,"internally_reserved":0,"allocated":7376896,"data_stored":3954860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":268095026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.226373"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.147970+0000","last_change":"2023-11-28T18:27:06.142439+0000","last_active":"2023-11-28T18:27:06.147970+0000","last_peered":"2023-11-28T18:27:06.147970+0000","last_clean":"2023-11-28T18:27:06.147970+0000","last_became_active":"2023-11-28T18:27:06.142185+0000","last_became_peered":"2023-11-28T18:27:06.142185+0000","last_unstale":"2023-11-28T18:27:06.147970+0000","last_undegraded":"2023-11-28T18:27:06.147970+0000","last_fullsized":"2023-11-28T18:27:06.147970+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:30:10.866128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.145033+0000","last_change":"2023-11-28T18:27:06.138796+0000","last_active":"2023-11-28T18:27:06.145033+0000","last_peered":"2023-11-28T18:27:06.145033+0000","last_clean":"2023-11-28T18:27:06.145033+0000","last_became_active":"2023-11-28T18:27:06.138513+0000","last_became_peered":"2023-11-28T18:27:06.138513+0000","last_unstale":"2023-11-28T18:27:06.145033+0000","last_undegraded":"2023-11-28T18:27:06.145033+0000","last_fullsized":"2023-11-28T18:27:06.145033+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:53:09.924003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.154469+0000","last_change":"2023-11-28T18:27:06.148765+0000","last_active":"2023-11-28T18:27:06.154469+0000","last_peered":"2023-11-28T18:27:06.154469+0000","last_clean":"2023-11-28T18:27:06.154469+0000","last_became_active":"2023-11-28T18:27:06.148381+0000","last_became_peered":"2023-11-28T18:27:06.148381+0000","last_unstale":"2023-11-28T18:27:06.154469+0000","last_undegraded":"2023-11-28T18:27:06.154469+0000","last_fullsized":"2023-11-28T18:27:06.154469+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:49:16.270543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.155939+0000","last_change":"2023-11-28T18:27:06.141251+0000","last_active":"2023-11-28T18:27:06.155939+0000","last_peered":"2023-11-28T18:27:06.155939+0000","last_clean":"2023-11-28T18:27:06.155939+0000","last_became_active":"2023-11-28T18:27:06.140804+0000","last_became_peered":"2023-11-28T18:27:06.140804+0000","last_unstale":"2023-11-28T18:27:06.155939+0000","last_undegraded":"2023-11-28T18:27:06.155939+0000","last_fullsized":"2023-11-28T18:27:06.155939+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:04:43.561084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.157013+0000","last_change":"2023-11-28T18:27:06.153434+0000","last_active":"2023-11-28T18:27:06.157013+0000","last_peered":"2023-11-28T18:27:06.157013+0000","last_clean":"2023-11-28T18:27:06.157013+0000","last_became_active":"2023-11-28T18:27:06.153158+0000","last_became_peered":"2023-11-28T18:27:06.153158+0000","last_unstale":"2023-11-28T18:27:06.157013+0000","last_undegraded":"2023-11-28T18:27:06.157013+0000","last_fullsized":"2023-11-28T18:27:06.157013+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:04:43.561084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.145084+0000","last_change":"2023-11-28T18:27:06.137310+0000","last_active":"2023-11-28T18:27:06.145084+0000","last_peered":"2023-11-28T18:27:06.145084+0000","last_clean":"2023-11-28T18:27:06.145084+0000","last_became_active":"2023-11-28T18:27:06.137051+0000","last_became_peered":"2023-11-28T18:27:06.137051+0000","last_unstale":"2023-11-28T18:27:06.145084+0000","last_undegraded":"2023-11-28T18:27:06.145084+0000","last_fullsized":"2023-11-28T18:27:06.145084+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:05:25.033948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.142918+0000","last_change":"2023-11-28T18:27:06.135215+0000","last_active":"2023-11-28T18:27:06.142918+0000","last_peered":"2023-11-28T18:27:06.142918+0000","last_clean":"2023-11-28T18:27:06.142918+0000","last_became_active":"2023-11-28T18:27:06.133785+0000","last_became_peered":"2023-11-28T18:27:06.133785+0000","last_unstale":"2023-11-28T18:27:06.142918+0000","last_undegraded":"2023-11-28T18:27:06.142918+0000","last_fullsized":"2023-11-28T18:27:06.142918+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:41.339034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.138246+0000","last_change":"2023-11-28T18:27:06.131192+0000","last_active":"2023-11-28T18:27:06.138246+0000","last_peered":"2023-11-28T18:27:06.138246+0000","last_clean":"2023-11-28T18:27:06.138246+0000","last_became_active":"2023-11-28T18:27:06.130898+0000","last_became_peered":"2023-11-28T18:27:06.130898+0000","last_unstale":"2023-11-28T18:27:06.138246+0000","last_undegraded":"2023-11-28T18:27:06.138246+0000","last_fullsized":"2023-11-28T18:27:06.138246+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:10:22.954858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.156766+0000","last_change":"2023-11-28T18:27:06.150904+0000","last_active":"2023-11-28T18:27:06.156766+0000","last_peered":"2023-11-28T18:27:06.156766+0000","last_clean":"2023-11-28T18:27:06.156766+0000","last_became_active":"2023-11-28T18:27:06.150620+0000","last_became_peered":"2023-11-28T18:27:06.150620+0000","last_unstale":"2023-11-28T18:27:06.156766+0000","last_undegraded":"2023-11-28T18:27:06.156766+0000","last_fullsized":"2023-11-28T18:27:06.156766+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:30.288695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.147128+0000","last_change":"2023-11-28T18:27:06.143292+0000","last_active":"2023-11-28T18:27:06.147128+0000","last_peered":"2023-11-28T18:27:06.147128+0000","last_clean":"2023-11-28T18:27:06.147128+0000","last_became_active":"2023-11-28T18:27:06.143012+0000","last_became_peered":"2023-11-28T18:27:06.143012+0000","last_unstale":"2023-11-28T18:27:06.147128+0000","last_undegraded":"2023-11-28T18:27:06.147128+0000","last_fullsized":"2023-11-28T18:27:06.147128+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:03:06.598375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.155149+0000","last_change":"2023-11-28T18:27:06.140378+0000","last_active":"2023-11-28T18:27:06.155149+0000","last_peered":"2023-11-28T18:27:06.155149+0000","last_clean":"2023-11-28T18:27:06.155149+0000","last_became_active":"2023-11-28T18:27:06.139892+0000","last_became_peered":"2023-11-28T18:27:06.139892+0000","last_unstale":"2023-11-28T18:27:06.155149+0000","last_undegraded":"2023-11-28T18:27:06.155149+0000","last_fullsized":"2023-11-28T18:27:06.155149+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:32.696569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.137398+0000","last_change":"2023-11-28T18:27:06.130584+0000","last_active":"2023-11-28T18:27:06.137398+0000","last_peered":"2023-11-28T18:27:06.137398+0000","last_clean":"2023-11-28T18:27:06.137398+0000","last_became_active":"2023-11-28T18:27:06.130256+0000","last_became_peered":"2023-11-28T18:27:06.130256+0000","last_unstale":"2023-11-28T18:27:06.137398+0000","last_undegraded":"2023-11-28T18:27:06.137398+0000","last_fullsized":"2023-11-28T18:27:06.137398+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:31:14.921516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.140875+0000","last_change":"2023-11-28T18:27:06.135731+0000","last_active":"2023-11-28T18:27:06.140875+0000","last_peered":"2023-11-28T18:27:06.140875+0000","last_clean":"2023-11-28T18:27:06.140875+0000","last_became_active":"2023-11-28T18:27:06.134763+0000","last_became_peered":"2023-11-28T18:27:06.134763+0000","last_unstale":"2023-11-28T18:27:06.140875+0000","last_undegraded":"2023-11-28T18:27:06.140875+0000","last_fullsized":"2023-11-28T18:27:06.140875+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:46.633835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.147460+0000","last_change":"2023-11-28T18:27:06.143876+0000","last_active":"2023-11-28T18:27:06.147460+0000","last_peered":"2023-11-28T18:27:06.147460+0000","last_clean":"2023-11-28T18:27:06.147460+0000","last_became_active":"2023-11-28T18:27:06.142991+0000","last_became_peered":"2023-11-28T18:27:06.142991+0000","last_unstale":"2023-11-28T18:27:06.147460+0000","last_undegraded":"2023-11-28T18:27:06.147460+0000","last_fullsized":"2023-11-28T18:27:06.147460+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:31:14.921516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.152161+0000","last_change":"2023-11-28T18:27:06.145466+0000","last_active":"2023-11-28T18:27:06.152161+0000","last_peered":"2023-11-28T18:27:06.152161+0000","last_clean":"2023-11-28T18:27:06.152161+0000","last_became_active":"2023-11-28T18:27:06.145205+0000","last_became_peered":"2023-11-28T18:27:06.145205+0000","last_unstale":"2023-11-28T18:27:06.152161+0000","last_undegraded":"2023-11-28T18:27:06.152161+0000","last_fullsized":"2023-11-28T18:27:06.152161+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:10:22.954858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.143960+0000","last_change":"2023-11-28T18:27:06.134316+0000","last_active":"2023-11-28T18:27:06.143960+0000","last_peered":"2023-11-28T18:27:06.143960+0000","last_clean":"2023-11-28T18:27:06.143960+0000","last_became_active":"2023-11-28T18:27:06.133990+0000","last_became_peered":"2023-11-28T18:27:06.133990+0000","last_unstale":"2023-11-28T18:27:06.143960+0000","last_undegraded":"2023-11-28T18:27:06.143960+0000","last_fullsized":"2023-11-28T18:27:06.143960+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:04.405447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.153140+0000","last_change":"2023-11-28T18:27:06.152179+0000","last_active":"2023-11-28T18:27:06.153140+0000","last_peered":"2023-11-28T18:27:06.153140+0000","last_clean":"2023-11-28T18:27:06.153140+0000","last_became_active":"2023-11-28T18:27:06.151911+0000","last_became_peered":"2023-11-28T18:27:06.151911+0000","last_unstale":"2023-11-28T18:27:06.153140+0000","last_undegraded":"2023-11-28T18:27:06.153140+0000","last_fullsized":"2023-11-28T18:27:06.153140+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:36:22.268715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.155168+0000","last_change":"2023-11-28T18:27:06.140909+0000","last_active":"2023-11-28T18:27:06.155168+0000","last_peered":"2023-11-28T18:27:06.155168+0000","last_clean":"2023-11-28T18:27:06.155168+0000","last_became_active":"2023-11-28T18:27:06.140577+0000","last_became_peered":"2023-11-28T18:27:06.140577+0000","last_unstale":"2023-11-28T18:27:06.155168+0000","last_undegraded":"2023-11-28T18:27:06.155168+0000","last_fullsized":"2023-11-28T18:27:06.155168+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:53.072933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.137411+0000","last_change":"2023-11-28T18:27:06.132399+0000","last_active":"2023-11-28T18:27:06.137411+0000","last_peered":"2023-11-28T18:27:06.137411+0000","last_clean":"2023-11-28T18:27:06.137411+0000","last_became_active":"2023-11-28T18:27:06.132106+0000","last_became_peered":"2023-11-28T18:27:06.132106+0000","last_unstale":"2023-11-28T18:27:06.137411+0000","last_undegraded":"2023-11-28T18:27:06.137411+0000","last_fullsized":"2023-11-28T18:27:06.137411+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:08:49.174545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.139431+0000","last_change":"2023-11-28T18:27:06.132575+0000","last_active":"2023-11-28T18:27:06.139431+0000","last_peered":"2023-11-28T18:27:06.139431+0000","last_clean":"2023-11-28T18:27:06.139431+0000","last_became_active":"2023-11-28T18:27:06.131896+0000","last_became_peered":"2023-11-28T18:27:06.131896+0000","last_unstale":"2023-11-28T18:27:06.139431+0000","last_undegraded":"2023-11-28T18:27:06.139431+0000","last_fullsized":"2023-11-28T18:27:06.139431+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:09.802415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.140857+0000","last_change":"2023-11-28T18:27:06.135561+0000","last_active":"2023-11-28T18:27:06.140857+0000","last_peered":"2023-11-28T18:27:06.140857+0000","last_clean":"2023-11-28T18:27:06.140857+0000","last_became_active":"2023-11-28T18:27:06.134311+0000","last_became_peered":"2023-11-28T18:27:06.134311+0000","last_unstale":"2023-11-28T18:27:06.140857+0000","last_undegraded":"2023-11-28T18:27:06.140857+0000","last_fullsized":"2023-11-28T18:27:06.140857+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:53:58.080348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.135869+0000","last_change":"2023-11-28T18:27:06.131654+0000","last_active":"2023-11-28T18:27:06.135869+0000","last_peered":"2023-11-28T18:27:06.135869+0000","last_clean":"2023-11-28T18:27:06.135869+0000","last_became_active":"2023-11-28T18:27:06.131372+0000","last_became_peered":"2023-11-28T18:27:06.131372+0000","last_unstale":"2023-11-28T18:27:06.135869+0000","last_undegraded":"2023-11-28T18:27:06.135869+0000","last_fullsized":"2023-11-28T18:27:06.135869+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:05:17.307507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.133325+0000","last_change":"2023-11-28T18:27:06.131021+0000","last_active":"2023-11-28T18:27:06.133325+0000","last_peered":"2023-11-28T18:27:06.133325+0000","last_clean":"2023-11-28T18:27:06.133325+0000","last_became_active":"2023-11-28T18:27:06.130770+0000","last_became_peered":"2023-11-28T18:27:06.130770+0000","last_unstale":"2023-11-28T18:27:06.133325+0000","last_undegraded":"2023-11-28T18:27:06.133325+0000","last_fullsized":"2023-11-28T18:27:06.133325+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:30.288695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.147760+0000","last_change":"2023-11-28T18:27:06.141539+0000","last_active":"2023-11-28T18:27:06.147760+0000","last_peered":"2023-11-28T18:27:06.147760+0000","last_clean":"2023-11-28T18:27:06.147760+0000","last_became_active":"2023-11-28T18:27:06.141264+0000","last_became_peered":"2023-11-28T18:27:06.141264+0000","last_unstale":"2023-11-28T18:27:06.147760+0000","last_undegraded":"2023-11-28T18:27:06.147760+0000","last_fullsized":"2023-11-28T18:27:06.147760+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:46.633835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.141208+0000","last_change":"2023-11-28T18:27:06.137690+0000","last_active":"2023-11-28T18:27:06.141208+0000","last_peered":"2023-11-28T18:27:06.141208+0000","last_clean":"2023-11-28T18:27:06.141208+0000","last_became_active":"2023-11-28T18:27:06.136614+0000","last_became_peered":"2023-11-28T18:27:06.136614+0000","last_unstale":"2023-11-28T18:27:06.141208+0000","last_undegraded":"2023-11-28T18:27:06.141208+0000","last_fullsized":"2023-11-28T18:27:06.141208+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:05:34.965476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.141385+0000","last_change":"2023-11-28T18:27:06.136893+0000","last_active":"2023-11-28T18:27:06.141385+0000","last_peered":"2023-11-28T18:27:06.141385+0000","last_clean":"2023-11-28T18:27:06.141385+0000","last_became_active":"2023-11-28T18:27:06.136463+0000","last_became_peered":"2023-11-28T18:27:06.136463+0000","last_unstale":"2023-11-28T18:27:06.141385+0000","last_undegraded":"2023-11-28T18:27:06.141385+0000","last_fullsized":"2023-11-28T18:27:06.141385+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:03:06.598375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.155101+0000","last_change":"2023-11-28T18:27:06.140146+0000","last_active":"2023-11-28T18:27:06.155101+0000","last_peered":"2023-11-28T18:27:06.155101+0000","last_clean":"2023-11-28T18:27:06.155101+0000","last_became_active":"2023-11-28T18:27:06.139556+0000","last_became_peered":"2023-11-28T18:27:06.139556+0000","last_unstale":"2023-11-28T18:27:06.155101+0000","last_undegraded":"2023-11-28T18:27:06.155101+0000","last_fullsized":"2023-11-28T18:27:06.155101+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:19:50.305528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.147197+0000","last_change":"2023-11-28T18:27:06.140992+0000","last_active":"2023-11-28T18:27:06.147197+0000","last_peered":"2023-11-28T18:27:06.147197+0000","last_clean":"2023-11-28T18:27:06.147197+0000","last_became_active":"2023-11-28T18:27:06.140738+0000","last_became_peered":"2023-11-28T18:27:06.140738+0000","last_unstale":"2023-11-28T18:27:06.147197+0000","last_undegraded":"2023-11-28T18:27:06.147197+0000","last_fullsized":"2023-11-28T18:27:06.147197+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:01:34.681664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.154516+0000","last_change":"2023-11-28T18:27:06.150689+0000","last_active":"2023-11-28T18:27:06.154516+0000","last_peered":"2023-11-28T18:27:06.154516+0000","last_clean":"2023-11-28T18:27:06.154516+0000","last_became_active":"2023-11-28T18:27:06.150371+0000","last_became_peered":"2023-11-28T18:27:06.150371+0000","last_unstale":"2023-11-28T18:27:06.154516+0000","last_undegraded":"2023-11-28T18:27:06.154516+0000","last_fullsized":"2023-11-28T18:27:06.154516+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:05:34.965476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.140810+0000","last_change":"2023-11-28T18:27:06.133697+0000","last_active":"2023-11-28T18:27:06.140810+0000","last_peered":"2023-11-28T18:27:06.140810+0000","last_clean":"2023-11-28T18:27:06.140810+0000","last_became_active":"2023-11-28T18:27:06.133378+0000","last_became_peered":"2023-11-28T18:27:06.133378+0000","last_unstale":"2023-11-28T18:27:06.140810+0000","last_undegraded":"2023-11-28T18:27:06.140810+0000","last_fullsized":"2023-11-28T18:27:06.140810+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:54:07.729456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.154404+0000","last_change":"2023-11-28T18:27:06.134156+0000","last_active":"2023-11-28T18:27:06.154404+0000","last_peered":"2023-11-28T18:27:06.154404+0000","last_clean":"2023-11-28T18:27:06.154404+0000","last_became_active":"2023-11-28T18:27:06.133843+0000","last_became_peered":"2023-11-28T18:27:06.133843+0000","last_unstale":"2023-11-28T18:27:06.154404+0000","last_undegraded":"2023-11-28T18:27:06.154404+0000","last_fullsized":"2023-11-28T18:27:06.154404+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:08:49.174545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.144946+0000","last_change":"2023-11-28T18:27:06.135151+0000","last_active":"2023-11-28T18:27:06.144946+0000","last_peered":"2023-11-28T18:27:06.144946+0000","last_clean":"2023-11-28T18:27:06.144946+0000","last_became_active":"2023-11-28T18:27:06.134776+0000","last_became_peered":"2023-11-28T18:27:06.134776+0000","last_unstale":"2023-11-28T18:27:06.144946+0000","last_undegraded":"2023-11-28T18:27:06.144946+0000","last_fullsized":"2023-11-28T18:27:06.144946+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:58:50.027962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.153329+0000","last_change":"2023-11-28T18:27:06.147051+0000","last_active":"2023-11-28T18:27:06.153329+0000","last_peered":"2023-11-28T18:27:06.153329+0000","last_clean":"2023-11-28T18:27:06.153329+0000","last_became_active":"2023-11-28T18:27:06.146763+0000","last_became_peered":"2023-11-28T18:27:06.146763+0000","last_unstale":"2023-11-28T18:27:06.153329+0000","last_undegraded":"2023-11-28T18:27:06.153329+0000","last_fullsized":"2023-11-28T18:27:06.153329+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:19:50.305528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"64'4","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.149132+0000","last_change":"2023-11-28T18:27:05.122804+0000","last_active":"2023-11-28T18:27:06.149132+0000","last_peered":"2023-11-28T18:27:06.149132+0000","last_clean":"2023-11-28T18:27:06.149132+0000","last_became_active":"2023-11-28T18:27:05.122553+0000","last_became_peered":"2023-11-28T18:27:05.122553+0000","last_unstale":"2023-11-28T18:27:06.149132+0000","last_undegraded":"2023-11-28T18:27:06.149132+0000","last_fullsized":"2023-11-28T18:27:06.149132+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:53:57.074647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.155230+0000","last_change":"2023-11-28T18:27:06.148896+0000","last_active":"2023-11-28T18:27:06.155230+0000","last_peered":"2023-11-28T18:27:06.155230+0000","last_clean":"2023-11-28T18:27:06.155230+0000","last_became_active":"2023-11-28T18:27:06.148549+0000","last_became_peered":"2023-11-28T18:27:06.148549+0000","last_unstale":"2023-11-28T18:27:06.155230+0000","last_undegraded":"2023-11-28T18:27:06.155230+0000","last_fullsized":"2023-11-28T18:27:06.155230+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:45.390936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.137000+0000","last_change":"2023-11-28T18:27:05.126258+0000","last_active":"2023-11-28T18:27:06.137000+0000","last_peered":"2023-11-28T18:27:06.137000+0000","last_clean":"2023-11-28T18:27:06.137000+0000","last_became_active":"2023-11-28T18:27:05.125922+0000","last_became_peered":"2023-11-28T18:27:05.125922+0000","last_unstale":"2023-11-28T18:27:06.137000+0000","last_undegraded":"2023-11-28T18:27:06.137000+0000","last_fullsized":"2023-11-28T18:27:06.137000+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:11:24.324549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.145065+0000","last_change":"2023-11-28T18:27:06.137214+0000","last_active":"2023-11-28T18:27:06.145065+0000","last_peered":"2023-11-28T18:27:06.145065+0000","last_clean":"2023-11-28T18:27:06.145065+0000","last_became_active":"2023-11-28T18:27:06.136805+0000","last_became_peered":"2023-11-28T18:27:06.136805+0000","last_unstale":"2023-11-28T18:27:06.145065+0000","last_undegraded":"2023-11-28T18:27:06.145065+0000","last_fullsized":"2023-11-28T18:27:06.145065+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:09.943693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.148053+0000","last_change":"2023-11-28T18:27:05.121661+0000","last_active":"2023-11-28T18:27:06.148053+0000","last_peered":"2023-11-28T18:27:06.148053+0000","last_clean":"2023-11-28T18:27:06.148053+0000","last_became_active":"2023-11-28T18:27:05.121367+0000","last_became_peered":"2023-11-28T18:27:05.121367+0000","last_unstale":"2023-11-28T18:27:06.148053+0000","last_undegraded":"2023-11-28T18:27:06.148053+0000","last_fullsized":"2023-11-28T18:27:06.148053+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:20:41.505129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.154334+0000","last_change":"2023-11-28T18:27:06.147653+0000","last_active":"2023-11-28T18:27:06.154334+0000","last_peered":"2023-11-28T18:27:06.154334+0000","last_clean":"2023-11-28T18:27:06.154334+0000","last_became_active":"2023-11-28T18:27:06.147461+0000","last_became_peered":"2023-11-28T18:27:06.147461+0000","last_unstale":"2023-11-28T18:27:06.154334+0000","last_undegraded":"2023-11-28T18:27:06.154334+0000","last_fullsized":"2023-11-28T18:27:06.154334+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:04:18.273513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.142152+0000","last_change":"2023-11-28T18:27:05.122474+0000","last_active":"2023-11-28T18:27:06.142152+0000","last_peered":"2023-11-28T18:27:06.142152+0000","last_clean":"2023-11-28T18:27:06.142152+0000","last_became_active":"2023-11-28T18:27:05.122194+0000","last_became_peered":"2023-11-28T18:27:05.122194+0000","last_unstale":"2023-11-28T18:27:06.142152+0000","last_undegraded":"2023-11-28T18:27:06.142152+0000","last_fullsized":"2023-11-28T18:27:06.142152+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:11:36.524909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.146181+0000","last_change":"2023-11-28T18:27:06.142803+0000","last_active":"2023-11-28T18:27:06.146181+0000","last_peered":"2023-11-28T18:27:06.146181+0000","last_clean":"2023-11-28T18:27:06.146181+0000","last_became_active":"2023-11-28T18:27:06.142524+0000","last_became_peered":"2023-11-28T18:27:06.142524+0000","last_unstale":"2023-11-28T18:27:06.146181+0000","last_undegraded":"2023-11-28T18:27:06.146181+0000","last_fullsized":"2023-11-28T18:27:06.146181+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:05:07.855561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.138262+0000","last_change":"2023-11-28T18:27:05.119528+0000","last_active":"2023-11-28T18:27:06.138262+0000","last_peered":"2023-11-28T18:27:06.138262+0000","last_clean":"2023-11-28T18:27:06.138262+0000","last_became_active":"2023-11-28T18:27:05.119169+0000","last_became_peered":"2023-11-28T18:27:05.119169+0000","last_unstale":"2023-11-28T18:27:06.138262+0000","last_undegraded":"2023-11-28T18:27:06.138262+0000","last_fullsized":"2023-11-28T18:27:06.138262+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:04:17.267811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.152676+0000","last_change":"2023-11-28T18:27:06.151822+0000","last_active":"2023-11-28T18:27:06.152676+0000","last_peered":"2023-11-28T18:27:06.152676+0000","last_clean":"2023-11-28T18:27:06.152676+0000","last_became_active":"2023-11-28T18:27:06.151587+0000","last_became_peered":"2023-11-28T18:27:06.151587+0000","last_unstale":"2023-11-28T18:27:06.152676+0000","last_undegraded":"2023-11-28T18:27:06.152676+0000","last_fullsized":"2023-11-28T18:27:06.152676+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:13:14.594224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.133627+0000","last_change":"2023-11-28T18:27:05.112741+0000","last_active":"2023-11-28T18:27:06.133627+0000","last_peered":"2023-11-28T18:27:06.133627+0000","last_clean":"2023-11-28T18:27:06.133627+0000","last_became_active":"2023-11-28T18:27:05.112443+0000","last_became_peered":"2023-11-28T18:27:05.112443+0000","last_unstale":"2023-11-28T18:27:06.133627+0000","last_undegraded":"2023-11-28T18:27:06.133627+0000","last_fullsized":"2023-11-28T18:27:06.133627+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:33:14.262575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.153261+0000","last_change":"2023-11-28T18:27:06.146804+0000","last_active":"2023-11-28T18:27:06.153261+0000","last_peered":"2023-11-28T18:27:06.153261+0000","last_clean":"2023-11-28T18:27:06.153261+0000","last_became_active":"2023-11-28T18:27:06.146510+0000","last_became_peered":"2023-11-28T18:27:06.146510+0000","last_unstale":"2023-11-28T18:27:06.153261+0000","last_undegraded":"2023-11-28T18:27:06.153261+0000","last_fullsized":"2023-11-28T18:27:06.153261+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:53.072933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.112577+0000","last_change":"2023-11-28T18:27:05.113425+0000","last_active":"2023-11-28T18:27:06.112577+0000","last_peered":"2023-11-28T18:27:06.112577+0000","last_clean":"2023-11-28T18:27:06.112577+0000","last_became_active":"2023-11-28T18:27:05.111871+0000","last_became_peered":"2023-11-28T18:27:05.111871+0000","last_unstale":"2023-11-28T18:27:06.112577+0000","last_undegraded":"2023-11-28T18:27:06.112577+0000","last_fullsized":"2023-11-28T18:27:06.112577+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:24:41.870988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.132741+0000","last_change":"2023-11-28T18:27:06.129616+0000","last_active":"2023-11-28T18:27:06.132741+0000","last_peered":"2023-11-28T18:27:06.132741+0000","last_clean":"2023-11-28T18:27:06.132741+0000","last_became_active":"2023-11-28T18:27:06.129366+0000","last_became_peered":"2023-11-28T18:27:06.129366+0000","last_unstale":"2023-11-28T18:27:06.132741+0000","last_undegraded":"2023-11-28T18:27:06.132741+0000","last_fullsized":"2023-11-28T18:27:06.132741+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:36:38.407270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.130327+0000","last_change":"2023-11-28T18:27:05.123567+0000","last_active":"2023-11-28T18:27:06.130327+0000","last_peered":"2023-11-28T18:27:06.130327+0000","last_clean":"2023-11-28T18:27:06.130327+0000","last_became_active":"2023-11-28T18:27:05.123267+0000","last_became_peered":"2023-11-28T18:27:05.123267+0000","last_unstale":"2023-11-28T18:27:06.130327+0000","last_undegraded":"2023-11-28T18:27:06.130327+0000","last_fullsized":"2023-11-28T18:27:06.130327+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:28:03.116448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.147178+0000","last_change":"2023-11-28T18:27:06.140874+0000","last_active":"2023-11-28T18:27:06.147178+0000","last_peered":"2023-11-28T18:27:06.147178+0000","last_clean":"2023-11-28T18:27:06.147178+0000","last_became_active":"2023-11-28T18:27:06.140379+0000","last_became_peered":"2023-11-28T18:27:06.140379+0000","last_unstale":"2023-11-28T18:27:06.147178+0000","last_undegraded":"2023-11-28T18:27:06.147178+0000","last_fullsized":"2023-11-28T18:27:06.147178+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:37.808591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.123289+0000","last_change":"2023-11-28T18:27:05.129282+0000","last_active":"2023-11-28T18:27:06.123289+0000","last_peered":"2023-11-28T18:27:06.123289+0000","last_clean":"2023-11-28T18:27:06.123289+0000","last_became_active":"2023-11-28T18:27:05.128963+0000","last_became_peered":"2023-11-28T18:27:05.128963+0000","last_unstale":"2023-11-28T18:27:06.123289+0000","last_undegraded":"2023-11-28T18:27:06.123289+0000","last_fullsized":"2023-11-28T18:27:06.123289+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:08.796713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.132254+0000","last_change":"2023-11-28T18:27:06.125687+0000","last_active":"2023-11-28T18:27:06.132254+0000","last_peered":"2023-11-28T18:27:06.132254+0000","last_clean":"2023-11-28T18:27:06.132254+0000","last_became_active":"2023-11-28T18:27:06.125413+0000","last_became_peered":"2023-11-28T18:27:06.125413+0000","last_unstale":"2023-11-28T18:27:06.132254+0000","last_undegraded":"2023-11-28T18:27:06.132254+0000","last_fullsized":"2023-11-28T18:27:06.132254+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:58:50.027962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.120173+0000","last_change":"2023-11-28T18:27:05.109290+0000","last_active":"2023-11-28T18:27:06.120173+0000","last_peered":"2023-11-28T18:27:06.120173+0000","last_clean":"2023-11-28T18:27:06.120173+0000","last_became_active":"2023-11-28T18:27:05.108985+0000","last_became_peered":"2023-11-28T18:27:05.108985+0000","last_unstale":"2023-11-28T18:27:06.120173+0000","last_undegraded":"2023-11-28T18:27:06.120173+0000","last_fullsized":"2023-11-28T18:27:06.120173+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:05:31.008179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.145999+0000","last_change":"2023-11-28T18:27:06.139897+0000","last_active":"2023-11-28T18:27:06.145999+0000","last_peered":"2023-11-28T18:27:06.145999+0000","last_clean":"2023-11-28T18:27:06.145999+0000","last_became_active":"2023-11-28T18:27:06.139625+0000","last_became_peered":"2023-11-28T18:27:06.139625+0000","last_unstale":"2023-11-28T18:27:06.145999+0000","last_undegraded":"2023-11-28T18:27:06.145999+0000","last_fullsized":"2023-11-28T18:27:06.145999+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:45:48.134616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"64'3","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.147448+0000","last_change":"2023-11-28T18:27:05.129508+0000","last_active":"2023-11-28T18:27:06.147448+0000","last_peered":"2023-11-28T18:27:06.147448+0000","last_clean":"2023-11-28T18:27:06.147448+0000","last_became_active":"2023-11-28T18:27:05.129112+0000","last_became_peered":"2023-11-28T18:27:05.129112+0000","last_unstale":"2023-11-28T18:27:06.147448+0000","last_undegraded":"2023-11-28T18:27:06.147448+0000","last_fullsized":"2023-11-28T18:27:06.147448+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:40:44.065622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.151942+0000","last_change":"2023-11-28T18:27:06.129334+0000","last_active":"2023-11-28T18:27:06.151942+0000","last_peered":"2023-11-28T18:27:06.151942+0000","last_clean":"2023-11-28T18:27:06.151942+0000","last_became_active":"2023-11-28T18:27:06.129013+0000","last_became_peered":"2023-11-28T18:27:06.129013+0000","last_unstale":"2023-11-28T18:27:06.151942+0000","last_undegraded":"2023-11-28T18:27:06.151942+0000","last_fullsized":"2023-11-28T18:27:06.151942+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:45:48.134616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"64'6","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.157756+0000","last_change":"2023-11-28T18:27:05.116939+0000","last_active":"2023-11-28T18:27:06.157756+0000","last_peered":"2023-11-28T18:27:06.157756+0000","last_clean":"2023-11-28T18:27:06.157756+0000","last_became_active":"2023-11-28T18:27:05.115980+0000","last_became_peered":"2023-11-28T18:27:05.115980+0000","last_unstale":"2023-11-28T18:27:06.157756+0000","last_undegraded":"2023-11-28T18:27:06.157756+0000","last_fullsized":"2023-11-28T18:27:06.157756+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:00:05.602763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.154265+0000","last_change":"2023-11-28T18:27:06.134129+0000","last_active":"2023-11-28T18:27:06.154265+0000","last_peered":"2023-11-28T18:27:06.154265+0000","last_clean":"2023-11-28T18:27:06.154265+0000","last_became_active":"2023-11-28T18:27:06.133843+0000","last_became_peered":"2023-11-28T18:27:06.133843+0000","last_unstale":"2023-11-28T18:27:06.154265+0000","last_undegraded":"2023-11-28T18:27:06.154265+0000","last_fullsized":"2023-11-28T18:27:06.154265+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:42:25.052676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.142074+0000","last_change":"2023-11-28T18:27:05.125191+0000","last_active":"2023-11-28T18:27:06.142074+0000","last_peered":"2023-11-28T18:27:06.142074+0000","last_clean":"2023-11-28T18:27:06.142074+0000","last_became_active":"2023-11-28T18:27:05.124894+0000","last_became_peered":"2023-11-28T18:27:05.124894+0000","last_unstale":"2023-11-28T18:27:06.142074+0000","last_undegraded":"2023-11-28T18:27:06.142074+0000","last_fullsized":"2023-11-28T18:27:06.142074+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:54:06.723754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.142571+0000","last_change":"2023-11-28T18:27:06.139795+0000","last_active":"2023-11-28T18:27:06.142571+0000","last_peered":"2023-11-28T18:27:06.142571+0000","last_clean":"2023-11-28T18:27:06.142571+0000","last_became_active":"2023-11-28T18:27:06.138920+0000","last_became_peered":"2023-11-28T18:27:06.138920+0000","last_unstale":"2023-11-28T18:27:06.142571+0000","last_undegraded":"2023-11-28T18:27:06.142571+0000","last_fullsized":"2023-11-28T18:27:06.142571+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:10.114405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.117399+0000","last_change":"2023-11-28T18:27:05.119912+0000","last_active":"2023-11-28T18:27:06.117399+0000","last_peered":"2023-11-28T18:27:06.117399+0000","last_clean":"2023-11-28T18:27:06.117399+0000","last_became_active":"2023-11-28T18:27:05.119578+0000","last_became_peered":"2023-11-28T18:27:05.119578+0000","last_unstale":"2023-11-28T18:27:06.117399+0000","last_undegraded":"2023-11-28T18:27:06.117399+0000","last_fullsized":"2023-11-28T18:27:06.117399+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:40:44.065622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.146152+0000","last_change":"2023-11-28T18:27:06.138277+0000","last_active":"2023-11-28T18:27:06.146152+0000","last_peered":"2023-11-28T18:27:06.146152+0000","last_clean":"2023-11-28T18:27:06.146152+0000","last_became_active":"2023-11-28T18:27:06.138012+0000","last_became_peered":"2023-11-28T18:27:06.138012+0000","last_unstale":"2023-11-28T18:27:06.146152+0000","last_undegraded":"2023-11-28T18:27:06.146152+0000","last_fullsized":"2023-11-28T18:27:06.146152+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:32.696569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.122822+0000","last_change":"2023-11-28T18:27:05.124365+0000","last_active":"2023-11-28T18:27:06.122822+0000","last_peered":"2023-11-28T18:27:06.122822+0000","last_clean":"2023-11-28T18:27:06.122822+0000","last_became_active":"2023-11-28T18:27:05.124121+0000","last_became_peered":"2023-11-28T18:27:05.124121+0000","last_unstale":"2023-11-28T18:27:06.122822+0000","last_undegraded":"2023-11-28T18:27:06.122822+0000","last_fullsized":"2023-11-28T18:27:06.122822+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:24:41.870988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.141717+0000","last_change":"2023-11-28T18:27:06.134616+0000","last_active":"2023-11-28T18:27:06.141717+0000","last_peered":"2023-11-28T18:27:06.141717+0000","last_clean":"2023-11-28T18:27:06.141717+0000","last_became_active":"2023-11-28T18:27:06.134340+0000","last_became_peered":"2023-11-28T18:27:06.134340+0000","last_unstale":"2023-11-28T18:27:06.141717+0000","last_undegraded":"2023-11-28T18:27:06.141717+0000","last_fullsized":"2023-11-28T18:27:06.141717+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:44:20.474673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"64'8","reported_seq":19,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.157192+0000","last_change":"2023-11-28T18:27:05.124103+0000","last_active":"2023-11-28T18:27:06.157192+0000","last_peered":"2023-11-28T18:27:06.157192+0000","last_clean":"2023-11-28T18:27:06.157192+0000","last_became_active":"2023-11-28T18:27:05.123733+0000","last_became_peered":"2023-11-28T18:27:05.123733+0000","last_unstale":"2023-11-28T18:27:06.157192+0000","last_undegraded":"2023-11-28T18:27:06.157192+0000","last_fullsized":"2023-11-28T18:27:06.157192+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:17:12.911982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.156835+0000","last_change":"2023-11-28T18:27:06.152999+0000","last_active":"2023-11-28T18:27:06.156835+0000","last_peered":"2023-11-28T18:27:06.156835+0000","last_clean":"2023-11-28T18:27:06.156835+0000","last_became_active":"2023-11-28T18:27:06.152690+0000","last_became_peered":"2023-11-28T18:27:06.152690+0000","last_unstale":"2023-11-28T18:27:06.156835+0000","last_undegraded":"2023-11-28T18:27:06.156835+0000","last_fullsized":"2023-11-28T18:27:06.156835+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:59:07.440975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.144291+0000","last_change":"2023-11-28T18:27:05.119906+0000","last_active":"2023-11-28T18:27:06.144291+0000","last_peered":"2023-11-28T18:27:06.144291+0000","last_clean":"2023-11-28T18:27:06.144291+0000","last_became_active":"2023-11-28T18:27:05.119559+0000","last_became_peered":"2023-11-28T18:27:05.119559+0000","last_unstale":"2023-11-28T18:27:06.144291+0000","last_undegraded":"2023-11-28T18:27:06.144291+0000","last_fullsized":"2023-11-28T18:27:06.144291+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:29.282994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.154525+0000","last_change":"2023-11-28T18:27:06.150196+0000","last_active":"2023-11-28T18:27:06.154525+0000","last_peered":"2023-11-28T18:27:06.154525+0000","last_clean":"2023-11-28T18:27:06.154525+0000","last_became_active":"2023-11-28T18:27:06.149890+0000","last_became_peered":"2023-11-28T18:27:06.149890+0000","last_unstale":"2023-11-28T18:27:06.154525+0000","last_undegraded":"2023-11-28T18:27:06.154525+0000","last_fullsized":"2023-11-28T18:27:06.154525+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:45.390936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.154686+0000","last_change":"2023-11-28T18:27:05.126550+0000","last_active":"2023-11-28T18:27:06.154686+0000","last_peered":"2023-11-28T18:27:06.154686+0000","last_clean":"2023-11-28T18:27:06.154686+0000","last_became_active":"2023-11-28T18:27:05.126178+0000","last_became_peered":"2023-11-28T18:27:05.126178+0000","last_unstale":"2023-11-28T18:27:06.154686+0000","last_undegraded":"2023-11-28T18:27:06.154686+0000","last_fullsized":"2023-11-28T18:27:06.154686+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:54:33.856624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.155269+0000","last_change":"2023-11-28T18:27:06.149602+0000","last_active":"2023-11-28T18:27:06.155269+0000","last_peered":"2023-11-28T18:27:06.155269+0000","last_clean":"2023-11-28T18:27:06.155269+0000","last_became_active":"2023-11-28T18:27:06.149282+0000","last_became_peered":"2023-11-28T18:27:06.149282+0000","last_unstale":"2023-11-28T18:27:06.155269+0000","last_undegraded":"2023-11-28T18:27:06.155269+0000","last_fullsized":"2023-11-28T18:27:06.155269+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:31:37.666806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.145938+0000","last_change":"2023-11-28T18:27:05.129430+0000","last_active":"2023-11-28T18:27:06.145938+0000","last_peered":"2023-11-28T18:27:06.145938+0000","last_clean":"2023-11-28T18:27:06.145938+0000","last_became_active":"2023-11-28T18:27:05.129105+0000","last_became_peered":"2023-11-28T18:27:05.129105+0000","last_unstale":"2023-11-28T18:27:06.145938+0000","last_undegraded":"2023-11-28T18:27:06.145938+0000","last_fullsized":"2023-11-28T18:27:06.145938+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:33:14.262575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.146110+0000","last_change":"2023-11-28T18:27:06.137664+0000","last_active":"2023-11-28T18:27:06.146110+0000","last_peered":"2023-11-28T18:27:06.146110+0000","last_clean":"2023-11-28T18:27:06.146110+0000","last_became_active":"2023-11-28T18:27:06.137361+0000","last_became_peered":"2023-11-28T18:27:06.137361+0000","last_unstale":"2023-11-28T18:27:06.146110+0000","last_undegraded":"2023-11-28T18:27:06.146110+0000","last_fullsized":"2023-11-28T18:27:06.146110+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:05:32.013881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.112933+0000","last_change":"2023-11-28T18:27:05.107466+0000","last_active":"2023-11-28T18:27:06.112933+0000","last_peered":"2023-11-28T18:27:06.112933+0000","last_clean":"2023-11-28T18:27:06.112933+0000","last_became_active":"2023-11-28T18:27:05.107194+0000","last_became_peered":"2023-11-28T18:27:05.107194+0000","last_unstale":"2023-11-28T18:27:06.112933+0000","last_undegraded":"2023-11-28T18:27:06.112933+0000","last_fullsized":"2023-11-28T18:27:06.112933+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:07:59.431923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.144927+0000","last_change":"2023-11-28T18:27:06.134727+0000","last_active":"2023-11-28T18:27:06.144927+0000","last_peered":"2023-11-28T18:27:06.144927+0000","last_clean":"2023-11-28T18:27:06.144927+0000","last_became_active":"2023-11-28T18:27:06.134342+0000","last_became_peered":"2023-11-28T18:27:06.134342+0000","last_unstale":"2023-11-28T18:27:06.144927+0000","last_undegraded":"2023-11-28T18:27:06.144927+0000","last_fullsized":"2023-11-28T18:27:06.144927+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:08:00.437625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.146775+0000","last_change":"2023-11-28T18:27:05.124788+0000","last_active":"2023-11-28T18:27:06.146775+0000","last_peered":"2023-11-28T18:27:06.146775+0000","last_clean":"2023-11-28T18:27:06.146775+0000","last_became_active":"2023-11-28T18:27:05.124319+0000","last_became_peered":"2023-11-28T18:27:05.124319+0000","last_unstale":"2023-11-28T18:27:06.146775+0000","last_undegraded":"2023-11-28T18:27:06.146775+0000","last_fullsized":"2023-11-28T18:27:06.146775+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:23.451635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.139900+0000","last_change":"2023-11-28T18:27:06.135266+0000","last_active":"2023-11-28T18:27:06.139900+0000","last_peered":"2023-11-28T18:27:06.139900+0000","last_clean":"2023-11-28T18:27:06.139900+0000","last_became_active":"2023-11-28T18:27:06.134624+0000","last_became_peered":"2023-11-28T18:27:06.134624+0000","last_unstale":"2023-11-28T18:27:06.139900+0000","last_undegraded":"2023-11-28T18:27:06.139900+0000","last_fullsized":"2023-11-28T18:27:06.139900+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:00:06.608464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.117528+0000","last_change":"2023-11-28T18:27:05.115691+0000","last_active":"2023-11-28T18:27:06.117528+0000","last_peered":"2023-11-28T18:27:06.117528+0000","last_clean":"2023-11-28T18:27:06.117528+0000","last_became_active":"2023-11-28T18:27:05.115379+0000","last_became_peered":"2023-11-28T18:27:05.115379+0000","last_unstale":"2023-11-28T18:27:06.117528+0000","last_undegraded":"2023-11-28T18:27:06.117528+0000","last_fullsized":"2023-11-28T18:27:06.117528+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:54:33.856624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.150447+0000","last_change":"2023-11-28T18:27:06.146058+0000","last_active":"2023-11-28T18:27:06.150447+0000","last_peered":"2023-11-28T18:27:06.150447+0000","last_clean":"2023-11-28T18:27:06.150447+0000","last_became_active":"2023-11-28T18:27:06.145771+0000","last_became_peered":"2023-11-28T18:27:06.145771+0000","last_unstale":"2023-11-28T18:27:06.150447+0000","last_undegraded":"2023-11-28T18:27:06.150447+0000","last_fullsized":"2023-11-28T18:27:06.150447+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:31:37.666806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.124291+0000","last_change":"2023-11-28T18:27:05.126023+0000","last_active":"2023-11-28T18:27:06.124291+0000","last_peered":"2023-11-28T18:27:06.124291+0000","last_clean":"2023-11-28T18:27:06.124291+0000","last_became_active":"2023-11-28T18:27:05.125654+0000","last_became_peered":"2023-11-28T18:27:05.125654+0000","last_unstale":"2023-11-28T18:27:06.124291+0000","last_undegraded":"2023-11-28T18:27:06.124291+0000","last_fullsized":"2023-11-28T18:27:06.124291+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:18.615768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.144983+0000","last_change":"2023-11-28T18:27:06.135584+0000","last_active":"2023-11-28T18:27:06.144983+0000","last_peered":"2023-11-28T18:27:06.144983+0000","last_clean":"2023-11-28T18:27:06.144983+0000","last_became_active":"2023-11-28T18:27:06.135320+0000","last_became_peered":"2023-11-28T18:27:06.135320+0000","last_unstale":"2023-11-28T18:27:06.144983+0000","last_undegraded":"2023-11-28T18:27:06.144983+0000","last_fullsized":"2023-11-28T18:27:06.144983+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:36:38.407270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.136123+0000","last_change":"2023-11-28T18:27:05.123131+0000","last_active":"2023-11-28T18:27:06.136123+0000","last_peered":"2023-11-28T18:27:06.136123+0000","last_clean":"2023-11-28T18:27:06.136123+0000","last_became_active":"2023-11-28T18:27:05.122836+0000","last_became_peered":"2023-11-28T18:27:05.122836+0000","last_unstale":"2023-11-28T18:27:06.136123+0000","last_undegraded":"2023-11-28T18:27:06.136123+0000","last_fullsized":"2023-11-28T18:27:06.136123+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:59:06.435274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.148041+0000","last_change":"2023-11-28T18:27:06.143560+0000","last_active":"2023-11-28T18:27:06.148041+0000","last_peered":"2023-11-28T18:27:06.148041+0000","last_clean":"2023-11-28T18:27:06.148041+0000","last_became_active":"2023-11-28T18:27:06.143297+0000","last_became_peered":"2023-11-28T18:27:06.143297+0000","last_unstale":"2023-11-28T18:27:06.148041+0000","last_undegraded":"2023-11-28T18:27:06.148041+0000","last_fullsized":"2023-11-28T18:27:06.148041+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:52.393749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.118879+0000","last_change":"2023-11-28T18:27:05.119212+0000","last_active":"2023-11-28T18:27:06.118879+0000","last_peered":"2023-11-28T18:27:06.118879+0000","last_clean":"2023-11-28T18:27:06.118879+0000","last_became_active":"2023-11-28T18:27:05.118919+0000","last_became_peered":"2023-11-28T18:27:05.118919+0000","last_unstale":"2023-11-28T18:27:06.118879+0000","last_undegraded":"2023-11-28T18:27:06.118879+0000","last_fullsized":"2023-11-28T18:27:06.118879+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:33:14.262575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.142271+0000","last_change":"2023-11-28T18:27:06.139708+0000","last_active":"2023-11-28T18:27:06.142271+0000","last_peered":"2023-11-28T18:27:06.142271+0000","last_clean":"2023-11-28T18:27:06.142271+0000","last_became_active":"2023-11-28T18:27:06.139444+0000","last_became_peered":"2023-11-28T18:27:06.139444+0000","last_unstale":"2023-11-28T18:27:06.142271+0000","last_undegraded":"2023-11-28T18:27:06.142271+0000","last_fullsized":"2023-11-28T18:27:06.142271+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:35:51.815845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.128602+0000","last_change":"2023-11-28T18:27:05.121664+0000","last_active":"2023-11-28T18:27:06.128602+0000","last_peered":"2023-11-28T18:27:06.128602+0000","last_clean":"2023-11-28T18:27:06.128602+0000","last_became_active":"2023-11-28T18:27:05.121386+0000","last_became_peered":"2023-11-28T18:27:05.121386+0000","last_unstale":"2023-11-28T18:27:06.128602+0000","last_undegraded":"2023-11-28T18:27:06.128602+0000","last_fullsized":"2023-11-28T18:27:06.128602+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:42.302881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.146130+0000","last_change":"2023-11-28T18:27:06.137452+0000","last_active":"2023-11-28T18:27:06.146130+0000","last_peered":"2023-11-28T18:27:06.146130+0000","last_clean":"2023-11-28T18:27:06.146130+0000","last_became_active":"2023-11-28T18:27:06.136976+0000","last_became_peered":"2023-11-28T18:27:06.136976+0000","last_unstale":"2023-11-28T18:27:06.146130+0000","last_undegraded":"2023-11-28T18:27:06.146130+0000","last_fullsized":"2023-11-28T18:27:06.146130+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:17:13.917683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.123463+0000","last_change":"2023-11-28T18:27:05.116737+0000","last_active":"2023-11-28T18:27:06.123463+0000","last_peered":"2023-11-28T18:27:06.123463+0000","last_clean":"2023-11-28T18:27:06.123463+0000","last_became_active":"2023-11-28T18:27:05.115610+0000","last_became_peered":"2023-11-28T18:27:05.115610+0000","last_unstale":"2023-11-28T18:27:06.123463+0000","last_undegraded":"2023-11-28T18:27:06.123463+0000","last_fullsized":"2023-11-28T18:27:06.123463+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:09.108703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.148046+0000","last_change":"2023-11-28T18:27:06.144755+0000","last_active":"2023-11-28T18:27:06.148046+0000","last_peered":"2023-11-28T18:27:06.148046+0000","last_clean":"2023-11-28T18:27:06.148046+0000","last_became_active":"2023-11-28T18:27:06.144442+0000","last_became_peered":"2023-11-28T18:27:06.144442+0000","last_unstale":"2023-11-28T18:27:06.148046+0000","last_undegraded":"2023-11-28T18:27:06.148046+0000","last_fullsized":"2023-11-28T18:27:06.148046+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:25.132096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.122366+0000","last_change":"2023-11-28T18:27:05.125243+0000","last_active":"2023-11-28T18:27:06.122366+0000","last_peered":"2023-11-28T18:27:06.122366+0000","last_clean":"2023-11-28T18:27:06.122366+0000","last_became_active":"2023-11-28T18:27:05.124974+0000","last_became_peered":"2023-11-28T18:27:05.124974+0000","last_unstale":"2023-11-28T18:27:06.122366+0000","last_undegraded":"2023-11-28T18:27:06.122366+0000","last_fullsized":"2023-11-28T18:27:06.122366+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:35:50.810143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.154422+0000","last_change":"2023-11-28T18:27:06.134299+0000","last_active":"2023-11-28T18:27:06.154422+0000","last_peered":"2023-11-28T18:27:06.154422+0000","last_clean":"2023-11-28T18:27:06.154422+0000","last_became_active":"2023-11-28T18:27:06.134015+0000","last_became_peered":"2023-11-28T18:27:06.134015+0000","last_unstale":"2023-11-28T18:27:06.154422+0000","last_undegraded":"2023-11-28T18:27:06.154422+0000","last_fullsized":"2023-11-28T18:27:06.154422+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:25.132096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.112375+0000","last_change":"2023-11-28T18:27:05.106005+0000","last_active":"2023-11-28T18:27:06.112375+0000","last_peered":"2023-11-28T18:27:06.112375+0000","last_clean":"2023-11-28T18:27:06.112375+0000","last_became_active":"2023-11-28T18:27:05.105732+0000","last_became_peered":"2023-11-28T18:27:05.105732+0000","last_unstale":"2023-11-28T18:27:06.112375+0000","last_undegraded":"2023-11-28T18:27:06.112375+0000","last_fullsized":"2023-11-28T18:27:06.112375+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:59:06.435274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.152171+0000","last_change":"2023-11-28T18:27:06.149503+0000","last_active":"2023-11-28T18:27:06.152171+0000","last_peered":"2023-11-28T18:27:06.152171+0000","last_clean":"2023-11-28T18:27:06.152171+0000","last_became_active":"2023-11-28T18:27:06.149225+0000","last_became_peered":"2023-11-28T18:27:06.149225+0000","last_unstale":"2023-11-28T18:27:06.152171+0000","last_undegraded":"2023-11-28T18:27:06.152171+0000","last_fullsized":"2023-11-28T18:27:06.152171+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:51.714834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.112662+0000","last_change":"2023-11-28T18:27:05.111957+0000","last_active":"2023-11-28T18:27:06.112662+0000","last_peered":"2023-11-28T18:27:06.112662+0000","last_clean":"2023-11-28T18:27:06.112662+0000","last_became_active":"2023-11-28T18:27:05.111661+0000","last_became_peered":"2023-11-28T18:27:05.111661+0000","last_unstale":"2023-11-28T18:27:06.112662+0000","last_undegraded":"2023-11-28T18:27:06.112662+0000","last_fullsized":"2023-11-28T18:27:06.112662+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:49:15.264841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.146160+0000","last_change":"2023-11-28T18:27:06.142958+0000","last_active":"2023-11-28T18:27:06.146160+0000","last_peered":"2023-11-28T18:27:06.146160+0000","last_clean":"2023-11-28T18:27:06.146160+0000","last_became_active":"2023-11-28T18:27:06.142596+0000","last_became_peered":"2023-11-28T18:27:06.142596+0000","last_unstale":"2023-11-28T18:27:06.146160+0000","last_undegraded":"2023-11-28T18:27:06.146160+0000","last_fullsized":"2023-11-28T18:27:06.146160+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:57:16.137353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.119813+0000","last_change":"2023-11-28T18:27:05.110708+0000","last_active":"2023-11-28T18:27:06.119813+0000","last_peered":"2023-11-28T18:27:06.119813+0000","last_clean":"2023-11-28T18:27:06.119813+0000","last_became_active":"2023-11-28T18:27:05.110436+0000","last_became_peered":"2023-11-28T18:27:05.110436+0000","last_unstale":"2023-11-28T18:27:06.119813+0000","last_undegraded":"2023-11-28T18:27:06.119813+0000","last_fullsized":"2023-11-28T18:27:06.119813+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:33:16.579076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.145980+0000","last_change":"2023-11-28T18:27:06.143100+0000","last_active":"2023-11-28T18:27:06.145980+0000","last_peered":"2023-11-28T18:27:06.145980+0000","last_clean":"2023-11-28T18:27:06.145980+0000","last_became_active":"2023-11-28T18:27:06.142615+0000","last_became_peered":"2023-11-28T18:27:06.142615+0000","last_unstale":"2023-11-28T18:27:06.145980+0000","last_undegraded":"2023-11-28T18:27:06.145980+0000","last_fullsized":"2023-11-28T18:27:06.145980+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:05.088136+0000","last_clean_scrub_stamp":"2023-11-28T18:27:05.088136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:42:25.052676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T18:27:06.111862+0000","last_change":"2023-11-28T18:27:05.116233+0000","last_active":"2023-11-28T18:27:06.111862+0000","last_peered":"2023-11-28T18:27:06.111862+0000","last_clean":"2023-11-28T18:27:06.111862+0000","last_became_active":"2023-11-28T18:27:05.115934+0000","last_became_peered":"2023-11-28T18:27:05.115934+0000","last_unstale":"2023-11-28T18:27:06.111862+0000","last_undegraded":"2023-11-28T18:27:06.111862+0000","last_fullsized":"2023-11-28T18:27:06.111862+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:27:04.082434+0000","last_clean_scrub_stamp":"2023-11-28T18:27:04.082434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:54:33.856624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":202,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T18:27:05.117105+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:27:05.117105+0000","last_peered":"2023-11-28T18:27:05.117105+0000","last_clean":"2023-11-28T18:27:05.117105+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:27:05.117105+0000","last_undegraded":"2023-11-28T18:27:05.117105+0000","last_fullsized":"2023-11-28T18:27:05.117105+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36692,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":1384448,"data_stored":941063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.68899999999999995,"15min":0.72899999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.98099999999999998,"5min":2.427,"15min":2.427},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.69699999999999995,"15min":0.73099999999999998},"min":{"1min":0.504,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91700000000000004,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.60599999999999998}]},{"osd":1,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.69699999999999995,"15min":0.72699999999999998},"min":{"1min":0.44500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.96099999999999997,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.73299999999999998,"15min":0.76700000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86399999999999999,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.623}]},{"osd":2,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.752,"15min":0.78600000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94599999999999995,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.439},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.71599999999999997,"15min":0.751},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86399999999999999,"5min":2.411,"15min":2.411},"last":0.68500000000000005}]},{"osd":3,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.73699999999999999,"15min":0.77500000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.87,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.77800000000000002,"15min":0.81399999999999995},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.92900000000000005,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":0.495}]},{"osd":4,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.36299999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77000000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.623,"15min":0.64800000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75700000000000001,"5min":0.877,"15min":0.877},"last":0.46600000000000003}]},{"osd":5,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.66100000000000003,"15min":0.68100000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89700000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.68300000000000005,"15min":0.70699999999999996},"min":{"1min":0.441,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82499999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.71099999999999997}]},{"osd":6,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.72599999999999998,"15min":0.749},"min":{"1min":0.48799999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.85099999999999998,"5min":1.042,"15min":1.042},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.70799999999999996,"15min":0.74199999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83999999999999997,"5min":1.006,"15min":1.006},"last":0.67500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":761856,"data_stored":342824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.90700000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72099999999999997,"15min":0.73299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.91400000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.70899999999999996}]},{"osd":1,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.77300000000000002,"15min":0.82499999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.879,"5min":4.2389999999999999,"15min":4.2389999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.76600000000000001,"15min":0.80500000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.88400000000000001,"5min":4.218,"15min":4.218},"last":0.61899999999999999}]},{"osd":2,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70099999999999996,"15min":0.72099999999999997},"min":{"1min":0.45100000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85599999999999998,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69999999999999996,"15min":0.70799999999999996},"min":{"1min":0.47599999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90000000000000002,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.65200000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.71499999999999997,"15min":0.73099999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88200000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68899999999999995,"15min":0.68999999999999995},"min":{"1min":0.53000000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.82899999999999996,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.52500000000000002}]},{"osd":4,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.66500000000000004,"15min":0.67600000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84599999999999997,"5min":0.995,"15min":0.995},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.66500000000000004,"15min":0.69299999999999995},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.93600000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.67200000000000004}]},{"osd":5,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.66900000000000004,"15min":0.68799999999999994},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.436},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.749,"15min":0.77000000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.89100000000000001,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.51400000000000001}]},{"osd":7,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.755,"15min":0.78100000000000003},"min":{"1min":0.45100000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90900000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.495},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.73899999999999999,"15min":0.75700000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.86699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.63800000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":761856,"data_stored":342824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71599999999999997,"15min":0.71899999999999997},"min":{"1min":0.54300000000000004,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.999,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74099999999999999,"15min":0.75900000000000001},"min":{"1min":0.55600000000000005,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.97799999999999998,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":0.66300000000000003}]},{"osd":1,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73199999999999998,"15min":0.75},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93000000000000005,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.77900000000000003,"15min":0.81200000000000006},"min":{"1min":0.57699999999999996,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.022,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.53300000000000003}]},{"osd":2,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.74199999999999999,"15min":0.77000000000000002},"min":{"1min":0.44400000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93200000000000005,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.753,"15min":0.78700000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86199999999999999,"5min":4.0149999999999997,"15min":4.0149999999999997},"last":0.56399999999999995}]},{"osd":3,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.80700000000000005,"15min":0.84299999999999997},"min":{"1min":0.56200000000000006,"5min":0.495,"15min":0.495},"max":{"1min":0.95599999999999996,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.78800000000000003,"15min":0.83999999999999997},"min":{"1min":0.51800000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96999999999999997,"5min":3.9900000000000002,"15min":3.9900000000000002},"last":0.59899999999999998}]},{"osd":4,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80600000000000005,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70299999999999996,"15min":0.71399999999999997},"min":{"1min":0.437,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.98799999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.58199999999999996}]},{"osd":6,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.41899999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91200000000000003,"5min":1.071,"15min":1.071},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71399999999999997,"15min":0.71999999999999997},"min":{"1min":0.52500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86199999999999999,"5min":1.756,"15min":1.756},"last":0.52100000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.752,"15min":0.75800000000000001},"min":{"1min":0.54200000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.99299999999999999,"5min":1.282,"15min":1.282},"last":0.628},{"interface":"front","average":{"1min":0.745,"5min":0.753,"15min":0.75800000000000001},"min":{"1min":0.51100000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.96899999999999997,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.72099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":761856,"data_stored":342824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66900000000000004,"15min":0.69399999999999995},"min":{"1min":0.41999999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.82999999999999996,"5min":2.363,"15min":2.363},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.67900000000000005}]},{"osd":1,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63,"15min":0.623},"min":{"1min":0.49299999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.86099999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.65400000000000003,"15min":0.65000000000000002},"min":{"1min":0.42799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85799999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.76400000000000001}]},{"osd":2,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.49199999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72199999999999998,"15min":0.72999999999999998},"min":{"1min":0.504,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91900000000000004,"5min":1.871,"15min":1.871},"last":0.72799999999999998}]},{"osd":3,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.495,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.872,"5min":0.878,"15min":0.878},"last":0.745},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.51600000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.89600000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.69999999999999996}]},{"osd":5,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.41499999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81200000000000006,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.629,"15min":0.64300000000000002},"min":{"1min":0.442,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90500000000000003,"5min":1.79,"15min":1.79},"last":0.56699999999999995}]},{"osd":6,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.63700000000000001,"15min":0.64900000000000002},"min":{"1min":0.41599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88600000000000001,"5min":1.123,"15min":1.123},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.69899999999999995,"15min":0.70999999999999996},"min":{"1min":0.55900000000000005,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89200000000000002,"5min":1.925,"15min":1.925},"last":0.85399999999999998}]},{"osd":7,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70499999999999996,"15min":0.72099999999999997},"min":{"1min":0.48499999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88100000000000001,"5min":1.843,"15min":1.843},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.76500000000000001,"15min":0.78000000000000003},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.91600000000000004,"5min":1.593,"15min":1.593},"last":0.87}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40248,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331592,"statfs":{"total":96636764160,"available":96595550208,"internally_reserved":0,"allocated":761856,"data_stored":342824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.41699999999999998,"15min":0.41099999999999998},"min":{"1min":0.30499999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":2.0169999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.46100000000000002,"15min":0.46500000000000002},"min":{"1min":0.33000000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73899999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":2.7080000000000002}]},{"osd":1,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.36099999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.64600000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.5,"15min":0.501},"min":{"1min":0.376,"5min":0.247,"15min":0.247},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":4.5599999999999996}]},{"osd":2,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51300000000000001,"15min":0.51900000000000002},"min":{"1min":0.35499999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79800000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":2.48},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81200000000000006,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":2.2919999999999998}]},{"osd":4,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.76000000000000001,"15min":0.77000000000000002},"min":{"1min":0.53800000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1060000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":12.695},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76200000000000001,"15min":0.77100000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0840000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":6.3250000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.81599999999999995,"15min":0.83799999999999997},"min":{"1min":0.56399999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2809999999999999,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.9430000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.57299999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2390000000000001,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":2.5760000000000001}]},{"osd":6,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.79400000000000004,"15min":0.80100000000000005},"min":{"1min":0.58299999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.173,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":6.1790000000000003},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.79700000000000004,"15min":0.80300000000000005},"min":{"1min":0.55200000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.246,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.984}]},{"osd":7,"last update":"Tue Nov 28 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.80200000000000005,"15min":0.81499999999999995},"min":{"1min":0.56999999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1080000000000001,"5min":1.786,"15min":1.786},"last":3.1709999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1419999999999999,"5min":1.337,"15min":1.337},"last":6.2249999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31940,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":774144,"data_stored":349919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.48899999999999999,"15min":0.504},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67600000000000005,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.48599999999999999,"15min":0.50800000000000001},"min":{"1min":0.33700000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.64100000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.46100000000000002}]},{"osd":1,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.51400000000000001,"15min":0.52800000000000002},"min":{"1min":0.315,"5min":0.308,"15min":0.308},"max":{"1min":0.59699999999999998,"5min":1.48,"15min":1.48},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51000000000000001,"15min":0.51500000000000001},"min":{"1min":0.39700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84399999999999997,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.51300000000000001}]},{"osd":3,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.48899999999999999,"15min":0.503},"min":{"1min":0.35699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.62,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71299999999999997,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.58699999999999997}]},{"osd":4,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.77500000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1459999999999999,"5min":1.512,"15min":1.512},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76100000000000001,"15min":0.77900000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.996,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.75800000000000001}]},{"osd":5,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.75,"15min":0.76700000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.85999999999999999,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.93899999999999995,"15min":1.014},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0389999999999999,"5min":7.4820000000000002,"15min":7.4820000000000002},"last":0.79100000000000004}]},{"osd":6,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78200000000000003,"15min":0.80000000000000004},"min":{"1min":0.55000000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.97299999999999998,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.82299999999999995,"15min":0.871},"min":{"1min":0.53000000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.94699999999999995,"5min":3.9380000000000002,"15min":3.9380000000000002},"last":0.73899999999999999}]},{"osd":7,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.81899999999999995,"15min":0.84199999999999997},"min":{"1min":0.56299999999999994,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.032,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.86499999999999999,"15min":0.92100000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.002,"5min":4.798,"15min":4.798},"last":0.77500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":802816,"data_stored":352295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:26:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.41699999999999998,"15min":0.42699999999999999},"min":{"1min":0.25900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.373},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44600000000000001,"15min":0.45000000000000001},"min":{"1min":0.28100000000000003,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.60599999999999998,"5min":0.62,"15min":0.62},"last":0.48299999999999998}]},{"osd":2,"last update":"Tue Nov 28 18:26:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.441,"15min":0.44700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.55700000000000005,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47899999999999998,"15min":0.47799999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.67000000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.45900000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:26:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75600000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.38600000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.42799999999999999}]},{"osd":4,"last update":"Tue Nov 28 18:26:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72699999999999998,"15min":0.747},"min":{"1min":0.41099999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98599999999999999,"5min":1.512,"15min":1.512},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.75600000000000001,"15min":0.76900000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92400000000000004,"5min":1.155,"15min":1.155},"last":1.0780000000000001}]},{"osd":5,"last update":"Tue Nov 28 18:26:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.72999999999999998,"15min":0.75},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91000000000000003,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.72299999999999998,"15min":0.748},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.83999999999999997,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.90200000000000002}]},{"osd":6,"last update":"Tue Nov 28 18:26:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.753,"15min":0.77700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.85399999999999998,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.78300000000000003,"15min":0.80800000000000005},"min":{"1min":0.51400000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.88900000000000001,"5min":1.448,"15min":1.448},"last":1.0429999999999999}]},{"osd":7,"last update":"Tue Nov 28 18:26:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.73799999999999999,"15min":0.752},"min":{"1min":0.54600000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.82199999999999995,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.75700000000000001,"15min":0.77800000000000002},"min":{"1min":0.57199999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90900000000000003,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.0109999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1368064,"data_stored":940287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.438,"15min":0.44600000000000001},"min":{"1min":0.32000000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.46800000000000003,"15min":0.47199999999999998},"min":{"1min":0.314,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64600000000000002,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.46999999999999997}]},{"osd":2,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.503,"15min":0.51100000000000001},"min":{"1min":0.371,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.60899999999999999,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.505,"15min":0.51700000000000002},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77800000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.35799999999999998}]},{"osd":3,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.33300000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.67500000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.54400000000000004,"15min":0.55600000000000005},"min":{"1min":0.40500000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80000000000000004,"5min":1.087,"15min":1.087},"last":0.45200000000000001}]},{"osd":4,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.49299999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.56899999999999995,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.996,"5min":1.244,"15min":1.244},"last":0.64500000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.52300000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0489999999999999,"5min":1.121,"15min":1.121},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.748,"15min":0.74399999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0329999999999999,"5min":1.075,"15min":1.075},"last":0.57199999999999995}]},{"osd":6,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.73699999999999999,"15min":0.73599999999999999},"min":{"1min":0.52500000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1140000000000001,"5min":1.407,"15min":1.407},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.749,"15min":0.73899999999999999},"min":{"1min":0.56899999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.087,"5min":1.089,"15min":1.089},"last":0.66300000000000003}]},{"osd":7,"last update":"Tue Nov 28 18:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0089999999999999,"5min":1.212,"15min":1.212},"last":0.626},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.75600000000000001,"15min":0.751},"min":{"1min":0.56000000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.67800000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:27:10.576 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T18:27:10.577 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-28T18:27:10.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.751+0000 7fbe60901640 1 Processor -- start 2023-11-28T18:27:10.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.751+0000 7fbe60901640 1 -- start start 2023-11-28T18:27:10.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.751+0000 7fbe60901640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbe5c159610 0x7fbe5c0ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:10.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.751+0000 7fbe60901640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe5c0cde30 con 0x7fbe5c0cc760 2023-11-28T18:27:10.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.751+0000 7fbe60901640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe5c1599f0 con 0x7fbe5c06cef0 2023-11-28T18:27:10.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.751+0000 7fbe60901640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe5c06de20 con 0x7fbe5c159610 2023-11-28T18:27:10.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.751+0000 7fbe5a59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbe5c159610 0x7fbe5c0ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:10.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.751+0000 7fbe5a59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbe5c159610 0x7fbe5c0ce2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44286/0 (socket says 172.21.15.105:44286) 2023-11-28T18:27:10.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.751+0000 7fbe5a59c640 1 -- 172.21.15.105:0/1560732973 learned_addr learned my addr 172.21.15.105:0/1560732973 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:10.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.751+0000 7fbe59d9b640 1 --1- 172.21.15.105:0/1560732973 >> v1:172.21.15.105:6789/0 conn(0x7fbe5c0cc760 0x7fbe5c06c7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51284/0 (socket says 172.21.15.105:51284) 2023-11-28T18:27:10.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.751+0000 7fbe5a59c640 1 -- 172.21.15.105:0/1560732973 >> v1:172.21.15.148:6789/0 conn(0x7fbe5c06cef0 legacy=0x7fbe5c0abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:10.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.751+0000 7fbe5a59c640 1 -- 172.21.15.105:0/1560732973 >> v1:172.21.15.105:6789/0 conn(0x7fbe5c0cc760 legacy=0x7fbe5c06c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:10.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.755+0000 7fbe5a59c640 1 -- 172.21.15.105:0/1560732973 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe5c0cd220 con 0x7fbe5c159610 2023-11-28T18:27:10.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.755+0000 7fbe5a59c640 1 --2- 172.21.15.105:0/1560732973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe5c159610 0x7fbe5c0ce2c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c006fd0 tx=0x7fbe5c0acca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb8a0a1fb92080d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:10.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.755+0000 7fbe5959a640 1 -- 172.21.15.105:0/1560732973 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c004230 con 0x7fbe5c159610 2023-11-28T18:27:10.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.755+0000 7fbe5959a640 1 -- 172.21.15.105:0/1560732973 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4c0ab070 con 0x7fbe5c159610 2023-11-28T18:27:10.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.755+0000 7fbe5959a640 1 -- 172.21.15.105:0/1560732973 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c0a7740 con 0x7fbe5c159610 2023-11-28T18:27:10.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.755+0000 7fbe60901640 1 -- 172.21.15.105:0/1560732973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe5c159610 msgr2=0x7fbe5c0ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:10.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.755+0000 7fbe60901640 1 --2- 172.21.15.105:0/1560732973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe5c159610 0x7fbe5c0ce2c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c006fd0 tx=0x7fbe5c0acca0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.755+0000 7fbe60901640 1 -- 172.21.15.105:0/1560732973 shutdown_connections 2023-11-28T18:27:10.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.755+0000 7fbe60901640 1 --2- 172.21.15.105:0/1560732973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe5c159610 0x7fbe5c0ce2c0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.755+0000 7fbe60901640 1 -- 172.21.15.105:0/1560732973 >> 172.21.15.105:0/1560732973 conn(0x7fbe5c0ceb90 msgr2=0x7fbe5c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:10.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.755+0000 7fbe60901640 1 -- 172.21.15.105:0/1560732973 shutdown_connections 2023-11-28T18:27:10.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.755+0000 7fbe60901640 1 -- 172.21.15.105:0/1560732973 wait complete. 2023-11-28T18:27:10.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.759+0000 7fbe60901640 1 Processor -- start 2023-11-28T18:27:10.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.759+0000 7fbe60901640 1 -- start start 2023-11-28T18:27:10.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.759+0000 7fbe60901640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe5c06cef0 0x7fbe5c152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:10.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.759+0000 7fbe60901640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe5c0cc760 0x7fbe5c0b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:10.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.759+0000 7fbe60901640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe5c159610 0x7fbe5c0b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:10.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.759+0000 7fbe60901640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c14df60 con 0x7fbe5c159610 2023-11-28T18:27:10.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.759+0000 7fbe60901640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c1548e0 con 0x7fbe5c0cc760 2023-11-28T18:27:10.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.759+0000 7fbe60901640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c153840 con 0x7fbe5c06cef0 2023-11-28T18:27:10.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.759+0000 7fbe5ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe5c159610 0x7fbe5c0b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:10.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.759+0000 7fbe5ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe5c159610 0x7fbe5c0b2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44294/0 (socket says 172.21.15.105:44294) 2023-11-28T18:27:10.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.759+0000 7fbe5ad9d640 1 -- 172.21.15.105:0/3558941175 learned_addr learned my addr 172.21.15.105:0/3558941175 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:10.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.759+0000 7fbe59d9b640 1 --2- 172.21.15.105:0/3558941175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe5c0cc760 0x7fbe5c0b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:10.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.759+0000 7fbe5ad9d640 1 -- 172.21.15.105:0/3558941175 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe5c06cef0 msgr2=0x7fbe5c152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:10.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.759+0000 7fbe5a59c640 1 --2- 172.21.15.105:0/3558941175 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe5c06cef0 0x7fbe5c152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:10.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.759+0000 7fbe5ad9d640 1 --2- 172.21.15.105:0/3558941175 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe5c06cef0 0x7fbe5c152e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.759+0000 7fbe5ad9d640 1 -- 172.21.15.105:0/3558941175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe5c0cc760 msgr2=0x7fbe5c0b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:10.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.759+0000 7fbe5ad9d640 1 --2- 172.21.15.105:0/3558941175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe5c0cc760 0x7fbe5c0b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.759+0000 7fbe5ad9d640 1 -- 172.21.15.105:0/3558941175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4c00b780 con 0x7fbe5c159610 2023-11-28T18:27:10.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.759+0000 7fbe59d9b640 1 --2- 172.21.15.105:0/3558941175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe5c0cc760 0x7fbe5c0b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:27:10.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.759+0000 7fbe5a59c640 1 --2- 172.21.15.105:0/3558941175 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe5c06cef0 0x7fbe5c152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:27:10.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.759+0000 7fbe5ad9d640 1 --2- 172.21.15.105:0/3558941175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe5c159610 0x7fbe5c0b2420 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400c780 tx=0x7fbe5400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:10.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.763+0000 7fbe4affd640 1 -- 172.21.15.105:0/3558941175 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5400aa50 con 0x7fbe5c159610 2023-11-28T18:27:10.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.763+0000 7fbe4affd640 1 -- 172.21.15.105:0/3558941175 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5400abf0 con 0x7fbe5c159610 2023-11-28T18:27:10.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.763+0000 7fbe60901640 1 -- 172.21.15.105:0/3558941175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe5c153b20 con 0x7fbe5c159610 2023-11-28T18:27:10.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.763+0000 7fbe4affd640 1 -- 172.21.15.105:0/3558941175 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5400ce30 con 0x7fbe5c159610 2023-11-28T18:27:10.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.763+0000 7fbe60901640 1 -- 172.21.15.105:0/3558941175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe5c1540e0 con 0x7fbe5c159610 2023-11-28T18:27:10.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.763+0000 7fbe60901640 1 -- 172.21.15.105:0/3558941175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe5c067570 con 0x7fbe5c159610 2023-11-28T18:27:10.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.763+0000 7fbe4affd640 1 -- 172.21.15.105:0/3558941175 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fbe54012420 con 0x7fbe5c159610 2023-11-28T18:27:10.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.763+0000 7fbe4affd640 1 --2- 172.21.15.105:0/3558941175 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbe3c0668d0 0x7fbe3c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:10.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.763+0000 7fbe4affd640 1 -- 172.21.15.105:0/3558941175 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbe54084ac0 con 0x7fbe5c159610 2023-11-28T18:27:10.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.763+0000 7fbe5a59c640 1 --2- 172.21.15.105:0/3558941175 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbe3c0668d0 0x7fbe3c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:10.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.763+0000 7fbe5a59c640 1 --2- 172.21.15.105:0/3558941175 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbe3c0668d0 0x7fbe3c068d90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c0b1000 tx=0x7fbe4c004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:10.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.767+0000 7fbe4affd640 1 -- 172.21.15.105:0/3558941175 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbe54051b20 con 0x7fbe5c159610 2023-11-28T18:27:10.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.967+0000 7fbe60901640 1 -- 172.21.15.105:0/3558941175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fbe5c0cf950 con 0x7fbe5c159610 2023-11-28T18:27:10.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.971+0000 7fbe4affd640 1 -- 172.21.15.105:0/3558941175 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7fbe54003eb0 con 0x7fbe5c159610 2023-11-28T18:27:10.974 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2023-11-28T18:27:10.974 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDpL2ZlSt0VMxAAQ8+j1BgBPMJ2cuiZj8aG0w== 2023-11-28T18:27:10.974 INFO:teuthology.orchestra.run.smithi105.stdout: caps mds = "allow" 2023-11-28T18:27:10.975 INFO:teuthology.orchestra.run.smithi105.stdout: caps mon = "allow r" 2023-11-28T18:27:10.975 INFO:teuthology.orchestra.run.smithi105.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-28T18:27:10.975 INFO:teuthology.orchestra.run.smithi105.stderr:updated caps for client.0 2023-11-28T18:27:10.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.971+0000 7fbe60901640 1 -- 172.21.15.105:0/3558941175 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbe3c0668d0 msgr2=0x7fbe3c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:10.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.971+0000 7fbe60901640 1 --2- 172.21.15.105:0/3558941175 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbe3c0668d0 0x7fbe3c068d90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c0b1000 tx=0x7fbe4c004740 comp rx=0 tx=0).stop 2023-11-28T18:27:10.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.971+0000 7fbe60901640 1 -- 172.21.15.105:0/3558941175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe5c159610 msgr2=0x7fbe5c0b2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:10.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.971+0000 7fbe60901640 1 --2- 172.21.15.105:0/3558941175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe5c159610 0x7fbe5c0b2420 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400c780 tx=0x7fbe5400cc40 comp rx=0 tx=0).stop 2023-11-28T18:27:10.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.971+0000 7fbe60901640 1 -- 172.21.15.105:0/3558941175 shutdown_connections 2023-11-28T18:27:10.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.971+0000 7fbe60901640 1 --2- 172.21.15.105:0/3558941175 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbe3c0668d0 0x7fbe3c068d90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.971+0000 7fbe60901640 1 --2- 172.21.15.105:0/3558941175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe5c159610 0x7fbe5c0b2420 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.971+0000 7fbe60901640 1 --2- 172.21.15.105:0/3558941175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe5c0cc760 0x7fbe5c0b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.971+0000 7fbe60901640 1 --2- 172.21.15.105:0/3558941175 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe5c06cef0 0x7fbe5c152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:10.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.971+0000 7fbe60901640 1 -- 172.21.15.105:0/3558941175 >> 172.21.15.105:0/3558941175 conn(0x7fbe5c0ceb90 msgr2=0x7fbe5c15bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:10.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.971+0000 7fbe60901640 1 -- 172.21.15.105:0/3558941175 shutdown_connections 2023-11-28T18:27:10.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:10.971+0000 7fbe60901640 1 -- 172.21.15.105:0/3558941175 wait complete. 2023-11-28T18:27:10.995 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:27:11.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.131+0000 7f5f2f2f5640 1 Processor -- start 2023-11-28T18:27:11.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.131+0000 7f5f2f2f5640 1 -- start start 2023-11-28T18:27:11.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.131+0000 7f5f2f2f5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5f2815d2d0 0x7f5f280cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.131+0000 7f5f2f2f5640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f280cf960 0x7f5f28166fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.131+0000 7f5f2f2f5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f280ce840 con 0x7f5f2815cef0 2023-11-28T18:27:11.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.131+0000 7f5f2f2f5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f280cfdf0 con 0x7f5f2815d2d0 2023-11-28T18:27:11.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.131+0000 7f5f2f2f5640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f280b4780 con 0x7f5f280cf960 2023-11-28T18:27:11.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.131+0000 7f5f2daf2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5f2815d2d0 0x7f5f280cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.131+0000 7f5f2daf2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5f2815d2d0 0x7f5f280cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:52236/0 (socket says 172.21.15.105:52236) 2023-11-28T18:27:11.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.131+0000 7f5f2daf2640 1 -- 172.21.15.105:0/3988567043 learned_addr learned my addr 172.21.15.105:0/3988567043 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:11.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.131+0000 7f5f2eaf4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f280cf960 0x7f5f28166fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.131+0000 7f5f2eaf4640 1 -- 172.21.15.105:0/3988567043 >> v2:172.21.15.148:3300/0 conn(0x7f5f2815d2d0 msgr2=0x7f5f280cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:11.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.131+0000 7f5f2eaf4640 1 --2- 172.21.15.105:0/3988567043 >> v2:172.21.15.148:3300/0 conn(0x7f5f2815d2d0 0x7f5f280cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.131+0000 7f5f2eaf4640 1 -- 172.21.15.105:0/3988567043 >> v1:172.21.15.148:6789/0 conn(0x7f5f2815cef0 legacy=0x7f5f280d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:11.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.131+0000 7f5f2eaf4640 1 -- 172.21.15.105:0/3988567043 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f280cd990 con 0x7f5f280cf960 2023-11-28T18:27:11.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.135+0000 7f5f2eaf4640 1 --2- 172.21.15.105:0/3988567043 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f280cf960 0x7f5f28166fd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009d90 tx=0x7f5f2001c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90dacdeb56a168b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:11.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.135+0000 7f5f2d2f1640 1 -- 172.21.15.105:0/3988567043 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f200a5960 con 0x7f5f280cf960 2023-11-28T18:27:11.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.135+0000 7f5f2d2f1640 1 -- 172.21.15.105:0/3988567043 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f200a5ac0 con 0x7f5f280cf960 2023-11-28T18:27:11.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.135+0000 7f5f2d2f1640 1 -- 172.21.15.105:0/3988567043 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f200a5e50 con 0x7f5f280cf960 2023-11-28T18:27:11.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.135+0000 7f5f2f2f5640 1 -- 172.21.15.105:0/3988567043 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f280cf960 msgr2=0x7f5f28166fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:11.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.135+0000 7f5f2f2f5640 1 --2- 172.21.15.105:0/3988567043 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f280cf960 0x7f5f28166fd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009d90 tx=0x7f5f2001c8a0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.135+0000 7f5f2f2f5640 1 -- 172.21.15.105:0/3988567043 shutdown_connections 2023-11-28T18:27:11.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.135+0000 7f5f2f2f5640 1 --2- 172.21.15.105:0/3988567043 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f280cf960 0x7f5f28166fd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.135+0000 7f5f2f2f5640 1 --2- 172.21.15.105:0/3988567043 >> v2:172.21.15.148:3300/0 conn(0x7f5f2815d2d0 0x7f5f280cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.135+0000 7f5f2f2f5640 1 -- 172.21.15.105:0/3988567043 >> 172.21.15.105:0/3988567043 conn(0x7f5f280d2e10 msgr2=0x7f5f280d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:11.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.135+0000 7f5f2f2f5640 1 -- 172.21.15.105:0/3988567043 shutdown_connections 2023-11-28T18:27:11.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.135+0000 7f5f2f2f5640 1 -- 172.21.15.105:0/3988567043 wait complete. 2023-11-28T18:27:11.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f2f2f5640 1 Processor -- start 2023-11-28T18:27:11.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f2f2f5640 1 -- start start 2023-11-28T18:27:11.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f2f2f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f280cf960 0x7f5f28153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f2f2f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f2815cef0 0x7f5f28154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f2f2f5640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f2815d2d0 0x7f5f28152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f2f2f5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f280ccdd0 con 0x7f5f2815cef0 2023-11-28T18:27:11.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f2f2f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f28153960 con 0x7f5f280cf960 2023-11-28T18:27:11.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f2f2f5640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f28152cb0 con 0x7f5f2815d2d0 2023-11-28T18:27:11.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f2daf2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f2815cef0 0x7f5f28154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f2daf2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f2815cef0 0x7f5f28154db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55854/0 (socket says 172.21.15.105:55854) 2023-11-28T18:27:11.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f2daf2640 1 -- 172.21.15.105:0/1590912154 learned_addr learned my addr 172.21.15.105:0/1590912154 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:11.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f2e2f3640 1 --2- 172.21.15.105:0/1590912154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f280cf960 0x7f5f28153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f2eaf4640 1 --2- 172.21.15.105:0/1590912154 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f2815d2d0 0x7f5f28152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f2daf2640 1 -- 172.21.15.105:0/1590912154 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f2815d2d0 msgr2=0x7f5f28152770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:11.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f2daf2640 1 --2- 172.21.15.105:0/1590912154 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f2815d2d0 0x7f5f28152770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f2daf2640 1 -- 172.21.15.105:0/1590912154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f280cf960 msgr2=0x7f5f28153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:11.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f2daf2640 1 --2- 172.21.15.105:0/1590912154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f280cf960 0x7f5f28153df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f2daf2640 1 -- 172.21.15.105:0/1590912154 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f20009a70 con 0x7f5f2815cef0 2023-11-28T18:27:11.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f2daf2640 1 --2- 172.21.15.105:0/1590912154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f2815cef0 0x7f5f28154db0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800e8a0 tx=0x7f5f1800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:11.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f1effd640 1 -- 172.21.15.105:0/1590912154 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f180094e0 con 0x7f5f2815cef0 2023-11-28T18:27:11.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f1effd640 1 -- 172.21.15.105:0/1590912154 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1800c940 con 0x7f5f2815cef0 2023-11-28T18:27:11.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f1effd640 1 -- 172.21.15.105:0/1590912154 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1800cd10 con 0x7f5f2815cef0 2023-11-28T18:27:11.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f2f2f5640 1 -- 172.21.15.105:0/1590912154 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f28153030 con 0x7f5f2815cef0 2023-11-28T18:27:11.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.139+0000 7f5f2f2f5640 1 -- 172.21.15.105:0/1590912154 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f281535f0 con 0x7f5f2815cef0 2023-11-28T18:27:11.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.143+0000 7f5f2f2f5640 1 -- 172.21.15.105:0/1590912154 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f280d03c0 con 0x7f5f2815cef0 2023-11-28T18:27:11.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.143+0000 7f5f1effd640 1 -- 172.21.15.105:0/1590912154 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5f18013470 con 0x7f5f2815cef0 2023-11-28T18:27:11.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.143+0000 7f5f1effd640 1 --2- 172.21.15.105:0/1590912154 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f0c0668d0 0x7f5f0c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.143+0000 7f5f1effd640 1 -- 172.21.15.105:0/1590912154 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5f180846b0 con 0x7f5f2815cef0 2023-11-28T18:27:11.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.143+0000 7f5f2e2f3640 1 --2- 172.21.15.105:0/1590912154 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f0c0668d0 0x7f5f0c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.143+0000 7f5f2e2f3640 1 --2- 172.21.15.105:0/1590912154 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f0c0668d0 0x7f5f0c068d90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5f24001810 tx=0x7f5f24009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:11.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.147+0000 7f5f1effd640 1 -- 172.21.15.105:0/1590912154 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5f180517c0 con 0x7f5f2815cef0 2023-11-28T18:27:11.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.327+0000 7f5f2f2f5640 1 -- 172.21.15.105:0/1590912154 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f280afc60 con 0x7f5f2815cef0 2023-11-28T18:27:11.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.327+0000 7f5f1effd640 1 -- 172.21.15.105:0/1590912154 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f5f18054de0 con 0x7f5f2815cef0 2023-11-28T18:27:11.331 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:27:11.332 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":56,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/234000461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":234000461},{"type":"v1","addr":"172.21.15.148:6834","nonce":234000461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6436,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2396073410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2396073410},{"type":"v1","addr":"172.21.15.148:6835","nonce":2396073410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6717,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/380088531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":380088531},{"type":"v1","addr":"172.21.15.105:6835","nonce":380088531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:27:05.479811+0000","modified":"2023-11-28T18:27:08.510202+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6720},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6720":{"gid":6720,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/3442512486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3442512486},{"type":"v1","addr":"172.21.15.105:6837","nonce":3442512486}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T18:27:11.332 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 56 2023-11-28T18:27:11.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.331+0000 7f5f2f2f5640 1 -- 172.21.15.105:0/1590912154 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f0c0668d0 msgr2=0x7f5f0c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:11.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.331+0000 7f5f2f2f5640 1 --2- 172.21.15.105:0/1590912154 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f0c0668d0 0x7f5f0c068d90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5f24001810 tx=0x7f5f24009040 comp rx=0 tx=0).stop 2023-11-28T18:27:11.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.331+0000 7f5f2f2f5640 1 -- 172.21.15.105:0/1590912154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f2815cef0 msgr2=0x7f5f28154db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:11.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.331+0000 7f5f2f2f5640 1 --2- 172.21.15.105:0/1590912154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f2815cef0 0x7f5f28154db0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800e8a0 tx=0x7f5f1800ed60 comp rx=0 tx=0).stop 2023-11-28T18:27:11.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.331+0000 7f5f2f2f5640 1 -- 172.21.15.105:0/1590912154 shutdown_connections 2023-11-28T18:27:11.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.331+0000 7f5f2f2f5640 1 --2- 172.21.15.105:0/1590912154 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f2815d2d0 0x7f5f28152770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.331+0000 7f5f2f2f5640 1 --2- 172.21.15.105:0/1590912154 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f0c0668d0 0x7f5f0c068d90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.331+0000 7f5f2f2f5640 1 --2- 172.21.15.105:0/1590912154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f2815cef0 0x7f5f28154db0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.331+0000 7f5f2f2f5640 1 --2- 172.21.15.105:0/1590912154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f280cf960 0x7f5f28153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.331+0000 7f5f2f2f5640 1 -- 172.21.15.105:0/1590912154 >> 172.21.15.105:0/1590912154 conn(0x7f5f280d2e10 msgr2=0x7f5f280d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:11.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.331+0000 7f5f2f2f5640 1 -- 172.21.15.105:0/1590912154 shutdown_connections 2023-11-28T18:27:11.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.331+0000 7f5f2f2f5640 1 -- 172.21.15.105:0/1590912154 wait complete. 2023-11-28T18:27:11.755 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:27:05.479811+0000', 'modified': '2023-11-28T18:27:08.510202+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6720}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6720': {'gid': 6720, 'name': 'c', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6837/3442512486', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 3442512486}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 3442512486}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:27:11.755 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T18:27:11.755 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T18:27:11.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.891+0000 7f6bb111d640 1 Processor -- start 2023-11-28T18:27:11.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.891+0000 7f6bb111d640 1 -- start start 2023-11-28T18:27:11.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.891+0000 7f6bb111d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bac0ce2c0 0x7f6bac06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.891+0000 7f6bb111d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6bac06cd20 0x7f6bac0abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.891+0000 7f6bb111d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bac0cde30 con 0x7f6bac159610 2023-11-28T18:27:11.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.891+0000 7f6bb111d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bac06d1b0 con 0x7f6bac06cd20 2023-11-28T18:27:11.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.891+0000 7f6bb111d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac1599f0 con 0x7f6bac0ce2c0 2023-11-28T18:27:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.891+0000 7f6bab59e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6bac06cd20 0x7f6bac0abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.891+0000 7f6baad9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6bac159610 0x7f6bac0cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:59128/0 (socket says 172.21.15.105:59128) 2023-11-28T18:27:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.891+0000 7f6baad9d640 1 -- 172.21.15.105:0/3699211525 learned_addr learned my addr 172.21.15.105:0/3699211525 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.891+0000 7f6baa59c640 1 --2- 172.21.15.105:0/3699211525 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bac0ce2c0 0x7f6bac06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.891+0000 7f6ba9d9b640 1 -- 172.21.15.105:0/3699211525 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3677721825 0 0) 0x7f6bac0cde30 con 0x7f6bac159610 2023-11-28T18:27:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.891+0000 7f6ba9d9b640 1 -- 172.21.15.105:0/3699211525 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b94003530 con 0x7f6bac159610 2023-11-28T18:27:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.891+0000 7f6bab59e640 1 -- 172.21.15.105:0/3699211525 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bac0ce2c0 msgr2=0x7f6bac06c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.891+0000 7f6bab59e640 1 --2- 172.21.15.105:0/3699211525 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bac0ce2c0 0x7f6bac06c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.891+0000 7f6bab59e640 1 -- 172.21.15.105:0/3699211525 >> v1:172.21.15.148:6789/0 conn(0x7f6bac159610 legacy=0x7f6bac0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:11.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.891+0000 7f6bab59e640 1 -- 172.21.15.105:0/3699211525 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac06de20 con 0x7f6bac06cd20 2023-11-28T18:27:11.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.891+0000 7f6baa59c640 1 --2- 172.21.15.105:0/3699211525 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bac0ce2c0 0x7f6bac06c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:27:11.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.895+0000 7f6bab59e640 1 --2- 172.21.15.105:0/3699211525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bac06cd20 0x7f6bac0abd80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009d90 tx=0x7f6ba401c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=141fa82a043742a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:11.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.895+0000 7f6ba9d9b640 1 -- 172.21.15.105:0/3699211525 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba40a5870 con 0x7f6bac06cd20 2023-11-28T18:27:11.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.895+0000 7f6ba9d9b640 1 -- 172.21.15.105:0/3699211525 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba40a59d0 con 0x7f6bac06cd20 2023-11-28T18:27:11.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.895+0000 7f6ba9d9b640 1 -- 172.21.15.105:0/3699211525 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba40a5d60 con 0x7f6bac06cd20 2023-11-28T18:27:11.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.895+0000 7f6bb111d640 1 -- 172.21.15.105:0/3699211525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bac06cd20 msgr2=0x7f6bac0abd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:11.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.895+0000 7f6bb111d640 1 --2- 172.21.15.105:0/3699211525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bac06cd20 0x7f6bac0abd80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009d90 tx=0x7f6ba401c920 comp rx=0 tx=0).stop 2023-11-28T18:27:11.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.895+0000 7f6bb111d640 1 -- 172.21.15.105:0/3699211525 shutdown_connections 2023-11-28T18:27:11.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.895+0000 7f6bb111d640 1 --2- 172.21.15.105:0/3699211525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bac06cd20 0x7f6bac0abd80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.895+0000 7f6bb111d640 1 --2- 172.21.15.105:0/3699211525 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bac0ce2c0 0x7f6bac06c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.895+0000 7f6bb111d640 1 -- 172.21.15.105:0/3699211525 >> 172.21.15.105:0/3699211525 conn(0x7f6bac0ceb90 msgr2=0x7f6bac0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:11.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.895+0000 7f6bb111d640 1 -- 172.21.15.105:0/3699211525 shutdown_connections 2023-11-28T18:27:11.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.895+0000 7f6bb111d640 1 -- 172.21.15.105:0/3699211525 wait complete. 2023-11-28T18:27:11.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6bb111d640 1 Processor -- start 2023-11-28T18:27:11.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6bb111d640 1 -- start start 2023-11-28T18:27:11.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6bb111d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bac06cd20 0x7f6bac1415c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6bb111d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bac0ce2c0 0x7f6bac156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6bb111d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bac159610 0x7f6bac150080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6bb111d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac0b40c0 con 0x7f6bac06cd20 2023-11-28T18:27:11.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6bb111d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac0b1f70 con 0x7f6bac0ce2c0 2023-11-28T18:27:11.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6bb111d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac0b4360 con 0x7f6bac159610 2023-11-28T18:27:11.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6baad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bac06cd20 0x7f6bac1415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6baad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bac06cd20 0x7f6bac1415c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55864/0 (socket says 172.21.15.105:55864) 2023-11-28T18:27:11.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6baad9d640 1 -- 172.21.15.105:0/2972763142 learned_addr learned my addr 172.21.15.105:0/2972763142 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:11.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6baa59c640 1 --2- 172.21.15.105:0/2972763142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bac0ce2c0 0x7f6bac156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6bab59e640 1 --2- 172.21.15.105:0/2972763142 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bac159610 0x7f6bac150080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6baad9d640 1 -- 172.21.15.105:0/2972763142 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bac159610 msgr2=0x7f6bac150080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:11.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6baad9d640 1 --2- 172.21.15.105:0/2972763142 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bac159610 0x7f6bac150080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6baad9d640 1 -- 172.21.15.105:0/2972763142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bac0ce2c0 msgr2=0x7f6bac156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:11.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6baad9d640 1 --2- 172.21.15.105:0/2972763142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bac0ce2c0 0x7f6bac156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6baad9d640 1 -- 172.21.15.105:0/2972763142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba000a5c0 con 0x7f6bac06cd20 2023-11-28T18:27:11.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6baad9d640 1 --2- 172.21.15.105:0/2972763142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bac06cd20 0x7f6bac1415c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00c420 tx=0x7f6b9c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:11.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6b937fe640 1 -- 172.21.15.105:0/2972763142 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c005cf0 con 0x7f6bac06cd20 2023-11-28T18:27:11.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6b937fe640 1 -- 172.21.15.105:0/2972763142 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9c005e50 con 0x7f6bac06cd20 2023-11-28T18:27:11.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6bb111d640 1 -- 172.21.15.105:0/2972763142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba4009a70 con 0x7f6bac06cd20 2023-11-28T18:27:11.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6b937fe640 1 -- 172.21.15.105:0/2972763142 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0072d0 con 0x7f6bac06cd20 2023-11-28T18:27:11.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.899+0000 7f6bb111d640 1 -- 172.21.15.105:0/2972763142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bac0ccff0 con 0x7f6bac06cd20 2023-11-28T18:27:11.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.903+0000 7f6b937fe640 1 -- 172.21.15.105:0/2972763142 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f6b9c007430 con 0x7f6bac06cd20 2023-11-28T18:27:11.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.903+0000 7f6b937fe640 1 --2- 172.21.15.105:0/2972763142 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b980668d0 0x7f6b98068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.903+0000 7f6b937fe640 1 -- 172.21.15.105:0/2972763142 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6b9c083b30 con 0x7f6bac06cd20 2023-11-28T18:27:11.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.903+0000 7f6baa59c640 1 --2- 172.21.15.105:0/2972763142 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b980668d0 0x7f6b98068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.903+0000 7f6bb4484000 1 Processor -- start 2023-11-28T18:27:11.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.903+0000 7f6baa59c640 1 --2- 172.21.15.105:0/2972763142 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b980668d0 0x7f6b98068d90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6bac15ccd0 tx=0x7f6ba000a2a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:11.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.903+0000 7f6bb4484000 1 -- start start 2023-11-28T18:27:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6bb4484000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556bed382d30 0x556bed383110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6bb4484000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed383650 0x556bed335fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6bb4484000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556bed336500 0x556bed3368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6bb4484000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bed1ff110 con 0x556bed383650 2023-11-28T18:27:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6bb4484000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bed18f9a0 con 0x556bed382d30 2023-11-28T18:27:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6bb4484000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x556bed18ced0 con 0x556bed336500 2023-11-28T18:27:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6baa59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed383650 0x556bed335fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6baa59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed383650 0x556bed335fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55868/0 (socket says 172.21.15.105:55868) 2023-11-28T18:27:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6baa59c640 1 -- 172.21.15.105:0/2373363475 learned_addr learned my addr 172.21.15.105:0/2373363475 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6bab59e640 1 --2- 172.21.15.105:0/2373363475 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556bed336500 0x556bed3368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6baad9d640 1 --2- 172.21.15.105:0/2373363475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556bed382d30 0x556bed383110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6baa59c640 1 -- 172.21.15.105:0/2373363475 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556bed336500 msgr2=0x556bed3368e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6baa59c640 1 --2- 172.21.15.105:0/2373363475 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556bed336500 0x556bed3368e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6baa59c640 1 -- 172.21.15.105:0/2373363475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556bed382d30 msgr2=0x556bed383110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6baa59c640 1 --2- 172.21.15.105:0/2373363475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556bed382d30 0x556bed383110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6baa59c640 1 -- 172.21.15.105:0/2373363475 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bece80890 con 0x556bed383650 2023-11-28T18:27:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6baa59c640 1 --2- 172.21.15.105:0/2373363475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed383650 0x556bed335fc0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000abf0 tx=0x7f6ba0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9b5e8a064ab35b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6b90ff9640 1 -- 172.21.15.105:0/2373363475 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0020650 con 0x556bed383650 2023-11-28T18:27:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6b90ff9640 1 -- 172.21.15.105:0/2373363475 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba00207b0 con 0x556bed383650 2023-11-28T18:27:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6b90ff9640 1 -- 172.21.15.105:0/2373363475 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0020b60 con 0x556bed383650 2023-11-28T18:27:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6bb4484000 1 -- 172.21.15.105:0/2373363475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed383650 msgr2=0x556bed335fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6bb4484000 1 --2- 172.21.15.105:0/2373363475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed383650 0x556bed335fc0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000abf0 tx=0x7f6ba0014000 comp rx=0 tx=0).stop 2023-11-28T18:27:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6bb4484000 1 -- 172.21.15.105:0/2373363475 shutdown_connections 2023-11-28T18:27:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6bb4484000 1 --2- 172.21.15.105:0/2373363475 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556bed336500 0x556bed3368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6bb4484000 1 --2- 172.21.15.105:0/2373363475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed383650 0x556bed335fc0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6bb4484000 1 --2- 172.21.15.105:0/2373363475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556bed382d30 0x556bed383110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6bb4484000 1 -- 172.21.15.105:0/2373363475 >> 172.21.15.105:0/2373363475 conn(0x556bed3638c0 msgr2=0x556bed365cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:11.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6bb4484000 1 -- 172.21.15.105:0/2373363475 shutdown_connections 2023-11-28T18:27:11.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6bb4484000 1 -- 172.21.15.105:0/2373363475 wait complete. 2023-11-28T18:27:11.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6bb4484000 1 -- start start 2023-11-28T18:27:11.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.907+0000 7f6bb4484000 1 Processor -- start 2023-11-28T18:27:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556bed336500 0x556bed33c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed382d30 0x556bed33ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556bed383650 0x556bed33cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bed1887c0 con 0x556bed382d30 2023-11-28T18:27:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bed140bf0 con 0x556bed336500 2023-11-28T18:27:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x556bed146a90 con 0x556bed383650 2023-11-28T18:27:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 10 client.0 _ll_register_callbacks cb 0x556bed323e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:27:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6baa59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed382d30 0x556bed33ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6baa59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed382d30 0x556bed33ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55882/0 (socket says 172.21.15.105:55882) 2023-11-28T18:27:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6baa59c640 1 -- 172.21.15.105:0/1984406825 learned_addr learned my addr 172.21.15.105:0/1984406825 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6baad9d640 1 --2- 172.21.15.105:0/1984406825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556bed336500 0x556bed33c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bab59e640 1 --2- 172.21.15.105:0/1984406825 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556bed383650 0x556bed33cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6baa59c640 1 -- 172.21.15.105:0/1984406825 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556bed383650 msgr2=0x556bed33cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6baa59c640 1 --2- 172.21.15.105:0/1984406825 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556bed383650 0x556bed33cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6baa59c640 1 -- 172.21.15.105:0/1984406825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556bed336500 msgr2=0x556bed33c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6baa59c640 1 --2- 172.21.15.105:0/1984406825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556bed336500 0x556bed33c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6baa59c640 1 -- 172.21.15.105:0/1984406825 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba000ae20 con 0x556bed382d30 2023-11-28T18:27:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bab59e640 1 --2- 172.21.15.105:0/1984406825 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556bed383650 0x556bed33cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6baa59c640 1 --2- 172.21.15.105:0/1984406825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed382d30 0x556bed33ca20 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0029680 tx=0x7f6ba0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f6871be0697bff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6b89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:27:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6b89ffb640 1 -- 172.21.15.105:0/1984406825 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0029c70 con 0x556bed382d30 2023-11-28T18:27:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6b89ffb640 1 -- 172.21.15.105:0/1984406825 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6ba0029dd0 con 0x556bed382d30 2023-11-28T18:27:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 1 -- 172.21.15.105:0/1984406825 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556bed14a470 con 0x556bed382d30 2023-11-28T18:27:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6b89ffb640 1 -- 172.21.15.105:0/1984406825 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ba0020450 con 0x556bed382d30 2023-11-28T18:27:11.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 10 client.6846 fetch_fsmap learned FSMap version 56 2023-11-28T18:27:11.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 1 -- 172.21.15.105:0/1984406825 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x556bed149190 con 0x556bed382d30 2023-11-28T18:27:11.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6b89ffb640 1 -- 172.21.15.105:0/1984406825 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556bed14a470 con 0x556bed382d30 2023-11-28T18:27:11.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6b89ffb640 1 -- 172.21.15.105:0/1984406825 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6ba00205b0 con 0x556bed382d30 2023-11-28T18:27:11.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 10 client.6846 fetch_fsmap finished waiting for FSMap version 56 2023-11-28T18:27:11.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 10 client.6846 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:27:11.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 20 client.6846 populate_metadata read hostname 'smithi105' 2023-11-28T18:27:11.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 1 --2- 172.21.15.105:0/1984406825 >> [v2:172.21.15.105:6836/3442512486,v1:172.21.15.105:6837/3442512486] conn(0x556bed33d970 0x556bed394ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 4 client.6846 mds_command: new command op to 6720 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:27:11.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 1 -- 172.21.15.105:0/1984406825 --> [v2:172.21.15.105:6836/3442512486,v1:172.21.15.105:6837/3442512486] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556bed14db50 con 0x556bed33d970 2023-11-28T18:27:11.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6baad9d640 1 --2- 172.21.15.105:0/1984406825 >> [v2:172.21.15.105:6836/3442512486,v1:172.21.15.105:6837/3442512486] conn(0x556bed33d970 0x556bed394ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6baad9d640 1 --2- 172.21.15.105:0/1984406825 >> [v2:172.21.15.105:6836/3442512486,v1:172.21.15.105:6837/3442512486] conn(0x556bed33d970 0x556bed394ff0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd09e003d2802ff7 server_cookie=1f428aae9634054e in_seq=0 out_seq=0 2023-11-28T18:27:11.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6b89ffb640 10 client.6846 ms_handle_connect on v2:172.21.15.105:6836/3442512486 2023-11-28T18:27:11.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6b89ffb640 1 -- 172.21.15.105:0/1984406825 <== mds.0 v2:172.21.15.105:6836/3442512486 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556bed14db50 con 0x556bed33d970 2023-11-28T18:27:11.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6b89ffb640 10 client.6846 handle_command_reply: tid=0 2023-11-28T18:27:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 1 client.6846 shutdown 2023-11-28T18:27:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 1 -- 172.21.15.105:0/1984406825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed382d30 msgr2=0x556bed33ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 1 --2- 172.21.15.105:0/1984406825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed382d30 0x556bed33ca20 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0029680 tx=0x7f6ba0004740 comp rx=0 tx=0).stop 2023-11-28T18:27:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 1 -- 172.21.15.105:0/1984406825 shutdown_connections 2023-11-28T18:27:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 1 --2- 172.21.15.105:0/1984406825 >> [v2:172.21.15.105:6836/3442512486,v1:172.21.15.105:6837/3442512486] conn(0x556bed33d970 0x556bed394ff0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 1 --2- 172.21.15.105:0/1984406825 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556bed383650 0x556bed33cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 1 --2- 172.21.15.105:0/1984406825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed382d30 0x556bed33ca20 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6b89ffb640 0 client.6846 ms_handle_reset on v2:172.21.15.105:6836/3442512486 2023-11-28T18:27:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 1 --2- 172.21.15.105:0/1984406825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556bed336500 0x556bed33c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 1 -- 172.21.15.105:0/1984406825 >> 172.21.15.105:0/1984406825 conn(0x556bed3638c0 msgr2=0x556bed3356b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 1 -- 172.21.15.105:0/1984406825 shutdown_connections 2023-11-28T18:27:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.911+0000 7f6bb4484000 1 -- 172.21.15.105:0/1984406825 wait complete. 2023-11-28T18:27:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.915+0000 7f6bb4484000 20 client.6846 trim_cache size 0 max 16384 2023-11-28T18:27:11.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.927+0000 7f6bb4484000 1 Processor -- start 2023-11-28T18:27:11.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.927+0000 7f6bb4484000 1 -- start start 2023-11-28T18:27:11.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.927+0000 7f6bb4484000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed336500 0x556bed3623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.927+0000 7f6bb4484000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556bed382d30 0x556bed3628e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.927+0000 7f6bb4484000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556bed383650 0x556bed394ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.927+0000 7f6bb4484000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bece41940 con 0x556bed336500 2023-11-28T18:27:11.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.927+0000 7f6bb4484000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bed184130 con 0x556bed382d30 2023-11-28T18:27:11.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.927+0000 7f6bb4484000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x556bed187fd0 con 0x556bed383650 2023-11-28T18:27:11.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6baad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed336500 0x556bed3623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6baad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed336500 0x556bed3623a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55886/0 (socket says 172.21.15.105:55886) 2023-11-28T18:27:11.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6baad9d640 1 -- 172.21.15.105:0/3895738700 learned_addr learned my addr 172.21.15.105:0/3895738700 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:11.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6baad9d640 1 -- 172.21.15.105:0/3895738700 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556bed383650 msgr2=0x556bed394ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:11.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6baad9d640 1 --2- 172.21.15.105:0/3895738700 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556bed383650 0x556bed394ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6baad9d640 1 -- 172.21.15.105:0/3895738700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556bed382d30 msgr2=0x556bed3628e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6baad9d640 1 --2- 172.21.15.105:0/3895738700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556bed382d30 0x556bed3628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6baad9d640 1 -- 172.21.15.105:0/3895738700 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac0ade00 con 0x556bed336500 2023-11-28T18:27:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6baad9d640 1 --2- 172.21.15.105:0/3895738700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed336500 0x556bed3623a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c050e00 tx=0x7f6b9c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81eb201d5f687030 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6b917fa640 1 -- 172.21.15.105:0/3895738700 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c081f90 con 0x556bed336500 2023-11-28T18:27:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6b917fa640 1 -- 172.21.15.105:0/3895738700 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9c052ca0 con 0x556bed336500 2023-11-28T18:27:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6b917fa640 1 -- 172.21.15.105:0/3895738700 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c05c6c0 con 0x556bed336500 2023-11-28T18:27:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6bb4484000 1 -- 172.21.15.105:0/3895738700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed336500 msgr2=0x556bed3623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6bb4484000 1 --2- 172.21.15.105:0/3895738700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed336500 0x556bed3623a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c050e00 tx=0x7f6b9c00cf90 comp rx=0 tx=0).stop 2023-11-28T18:27:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6bb4484000 1 -- 172.21.15.105:0/3895738700 shutdown_connections 2023-11-28T18:27:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6bb4484000 1 --2- 172.21.15.105:0/3895738700 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556bed383650 0x556bed394ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6bb4484000 1 --2- 172.21.15.105:0/3895738700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556bed382d30 0x556bed3628e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6bb4484000 1 --2- 172.21.15.105:0/3895738700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed336500 0x556bed3623a0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6bb4484000 1 -- 172.21.15.105:0/3895738700 >> 172.21.15.105:0/3895738700 conn(0x556bed3638c0 msgr2=0x556bed334720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:11.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6bb4484000 1 -- 172.21.15.105:0/3895738700 shutdown_connections 2023-11-28T18:27:11.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6bb4484000 1 -- 172.21.15.105:0/3895738700 wait complete. 2023-11-28T18:27:11.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6bb4484000 1 -- start start 2023-11-28T18:27:11.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6bb4484000 1 Processor -- start 2023-11-28T18:27:11.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6bb4484000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556bed336500 0x556bed3e3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6bb4484000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed382d30 0x556bed3e3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6bb4484000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556bed383650 0x556bed39ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6bb4484000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bed20f570 con 0x556bed382d30 2023-11-28T18:27:11.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6bb4484000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bed182ac0 con 0x556bed383650 2023-11-28T18:27:11.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6bb4484000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x556bed18c7c0 con 0x556bed336500 2023-11-28T18:27:11.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6bb4484000 10 client.0 _ll_register_callbacks cb 0x556bed1ff060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:27:11.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6baa59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed382d30 0x556bed3e3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6baa59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed382d30 0x556bed3e3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55898/0 (socket says 172.21.15.105:55898) 2023-11-28T18:27:11.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6baa59c640 1 -- 172.21.15.105:0/3342165984 learned_addr learned my addr 172.21.15.105:0/3342165984 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:11.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6bab59e640 1 --2- 172.21.15.105:0/3342165984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556bed383650 0x556bed39ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6baad9d640 1 --2- 172.21.15.105:0/3342165984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556bed336500 0x556bed3e3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6baa59c640 1 -- 172.21.15.105:0/3342165984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556bed336500 msgr2=0x556bed3e3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:11.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6baa59c640 1 --2- 172.21.15.105:0/3342165984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556bed336500 0x556bed3e3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6baa59c640 1 -- 172.21.15.105:0/3342165984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556bed383650 msgr2=0x556bed39ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:11.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6baa59c640 1 --2- 172.21.15.105:0/3342165984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556bed383650 0x556bed39ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6baa59c640 1 -- 172.21.15.105:0/3342165984 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bac0ccff0 con 0x556bed382d30 2023-11-28T18:27:11.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6bab59e640 1 --2- 172.21.15.105:0/3342165984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556bed383650 0x556bed39ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:11.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.931+0000 7f6baad9d640 1 --2- 172.21.15.105:0/3342165984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556bed336500 0x556bed3e3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:11.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6baa59c640 1 --2- 172.21.15.105:0/3342165984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed382d30 0x556bed3e3d50 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f6ba003a040 tx=0x7f6ba00046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc6c5d77eb64b30 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:11.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6b89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:27:11.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6b89ffb640 1 -- 172.21.15.105:0/3342165984 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba000aa70 con 0x556bed382d30 2023-11-28T18:27:11.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6b89ffb640 1 -- 172.21.15.105:0/3342165984 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6ba0011c10 con 0x556bed382d30 2023-11-28T18:27:11.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb4484000 1 -- 172.21.15.105:0/3342165984 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556bed187110 con 0x556bed382d30 2023-11-28T18:27:11.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6b89ffb640 1 -- 172.21.15.105:0/3342165984 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ba0029a70 con 0x556bed382d30 2023-11-28T18:27:11.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb4484000 10 client.6852 fetch_fsmap learned FSMap version 56 2023-11-28T18:27:11.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb4484000 1 -- 172.21.15.105:0/3342165984 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x556bed12f490 con 0x556bed382d30 2023-11-28T18:27:11.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6b89ffb640 1 -- 172.21.15.105:0/3342165984 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556bed187110 con 0x556bed382d30 2023-11-28T18:27:11.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6b89ffb640 1 -- 172.21.15.105:0/3342165984 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6ba0029bd0 con 0x556bed382d30 2023-11-28T18:27:11.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb4484000 10 client.6852 fetch_fsmap finished waiting for FSMap version 56 2023-11-28T18:27:11.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb4484000 10 client.6852 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:27:11.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb4484000 20 client.6852 populate_metadata read hostname 'smithi105' 2023-11-28T18:27:11.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb4484000 1 --2- 172.21.15.105:0/3342165984 >> [v2:172.21.15.105:6836/3442512486,v1:172.21.15.105:6837/3442512486] conn(0x556bed39f400 0x556bed39f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:11.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb4484000 4 client.6852 mds_command: new command op to 6720 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:27:11.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb4484000 1 -- 172.21.15.105:0/3342165984 --> [v2:172.21.15.105:6836/3442512486,v1:172.21.15.105:6837/3442512486] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556bed1b5650 con 0x556bed39f400 2023-11-28T18:27:11.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6baad9d640 1 --2- 172.21.15.105:0/3342165984 >> [v2:172.21.15.105:6836/3442512486,v1:172.21.15.105:6837/3442512486] conn(0x556bed39f400 0x556bed39f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:11.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6baad9d640 1 --2- 172.21.15.105:0/3342165984 >> [v2:172.21.15.105:6836/3442512486,v1:172.21.15.105:6837/3442512486] conn(0x556bed39f400 0x556bed39f7e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=234a573cbe21d47c server_cookie=ffa8bc183f0ac0f6 in_seq=0 out_seq=0 2023-11-28T18:27:11.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6b89ffb640 10 client.6852 ms_handle_connect on v2:172.21.15.105:6836/3442512486 2023-11-28T18:27:11.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6b89ffb640 1 -- 172.21.15.105:0/3342165984 <== mds.0 v2:172.21.15.105:6836/3442512486 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x556bed1b5650 con 0x556bed39f400 2023-11-28T18:27:11.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6b89ffb640 10 client.6852 handle_command_reply: tid=0 2023-11-28T18:27:11.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb4484000 1 client.6852 shutdown 2023-11-28T18:27:11.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb4484000 1 -- 172.21.15.105:0/3342165984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed382d30 msgr2=0x556bed3e3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:11.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb4484000 1 --2- 172.21.15.105:0/3342165984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed382d30 0x556bed3e3d50 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f6ba003a040 tx=0x7f6ba00046d0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb4484000 1 -- 172.21.15.105:0/3342165984 shutdown_connections 2023-11-28T18:27:11.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb4484000 1 --2- 172.21.15.105:0/3342165984 >> [v2:172.21.15.105:6836/3442512486,v1:172.21.15.105:6837/3442512486] conn(0x556bed39f400 0x556bed39f7e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb4484000 1 --2- 172.21.15.105:0/3342165984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556bed383650 0x556bed39ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb4484000 1 --2- 172.21.15.105:0/3342165984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556bed382d30 0x556bed3e3d50 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6b89ffb640 0 client.6852 ms_handle_reset on v2:172.21.15.105:6836/3442512486 2023-11-28T18:27:11.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb4484000 1 --2- 172.21.15.105:0/3342165984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556bed336500 0x556bed3e3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb4484000 1 -- 172.21.15.105:0/3342165984 >> 172.21.15.105:0/3342165984 conn(0x556bed3638c0 msgr2=0x556bed334fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:11.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb4484000 1 -- 172.21.15.105:0/3342165984 shutdown_connections 2023-11-28T18:27:11.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb4484000 1 -- 172.21.15.105:0/3342165984 wait complete. 2023-11-28T18:27:11.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb4484000 20 client.6852 trim_cache size 0 max 16384 2023-11-28T18:27:11.945 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:27:11.945 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:27:11.946 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2023-11-28T18:27:11.946 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 6720, 2023-11-28T18:27:11.946 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:27:11.946 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:27:11.946 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:27:11.946 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 5.8295709410000001, 2023-11-28T18:27:11.946 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 56, 2023-11-28T18:27:11.946 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 65, 2023-11-28T18:27:11.946 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 65, 2023-11-28T18:27:11.946 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 11.349344586999999 2023-11-28T18:27:11.946 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:27:11.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb111d640 1 -- 172.21.15.105:0/2972763142 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b980668d0 msgr2=0x7f6b98068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:11.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb111d640 1 --2- 172.21.15.105:0/2972763142 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b980668d0 0x7f6b98068d90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6bac15ccd0 tx=0x7f6ba000a2a0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb111d640 1 -- 172.21.15.105:0/2972763142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bac06cd20 msgr2=0x7f6bac1415c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:11.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb111d640 1 --2- 172.21.15.105:0/2972763142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bac06cd20 0x7f6bac1415c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00c420 tx=0x7f6b9c00c8e0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb111d640 1 -- 172.21.15.105:0/2972763142 shutdown_connections 2023-11-28T18:27:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb111d640 1 --2- 172.21.15.105:0/2972763142 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bac159610 0x7f6bac150080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb111d640 1 --2- 172.21.15.105:0/2972763142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bac0ce2c0 0x7f6bac156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb111d640 1 --2- 172.21.15.105:0/2972763142 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b980668d0 0x7f6b98068d90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb111d640 1 --2- 172.21.15.105:0/2972763142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bac06cd20 0x7f6bac1415c0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb111d640 1 -- 172.21.15.105:0/2972763142 >> 172.21.15.105:0/2972763142 conn(0x7f6bac0ceb90 msgr2=0x7f6bac159b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb111d640 1 -- 172.21.15.105:0/2972763142 shutdown_connections 2023-11-28T18:27:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:11.935+0000 7f6bb111d640 1 -- 172.21.15.105:0/2972763142 wait complete. 2023-11-28T18:27:11.959 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T18:27:12.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.091+0000 7f5cdacea640 1 Processor -- start 2023-11-28T18:27:12.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.091+0000 7f5cdacea640 1 -- start start 2023-11-28T18:27:12.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.095+0000 7f5cdacea640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5cd40caf60 0x7f5cd40cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:12.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.095+0000 7f5cdacea640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5cd40ad850 0x7f5cd40701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:12.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.095+0000 7f5cdacea640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd406bf50 con 0x7f5cd4159f40 2023-11-28T18:27:12.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.095+0000 7f5cdacea640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd40cb3c0 con 0x7f5cd40ad850 2023-11-28T18:27:12.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.095+0000 7f5cdacea640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd415a320 con 0x7f5cd40caf60 2023-11-28T18:27:12.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.095+0000 7f5cd9287640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5cd40ad850 0x7f5cd40701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:12.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.095+0000 7f5cd9287640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5cd40ad850 0x7f5cd40701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55914/0 (socket says 172.21.15.105:55914) 2023-11-28T18:27:12.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.095+0000 7f5cd9287640 1 -- 172.21.15.105:0/2966167078 learned_addr learned my addr 172.21.15.105:0/2966167078 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:12.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.095+0000 7f5cd3fff640 1 --2- 172.21.15.105:0/2966167078 >> v2:172.21.15.148:3300/0 conn(0x7f5cd40caf60 0x7f5cd40cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:12.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.095+0000 7f5cd9287640 1 -- 172.21.15.105:0/2966167078 >> v2:172.21.15.148:3300/0 conn(0x7f5cd40caf60 msgr2=0x7f5cd40cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:12.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.095+0000 7f5cd9287640 1 --2- 172.21.15.105:0/2966167078 >> v2:172.21.15.148:3300/0 conn(0x7f5cd40caf60 0x7f5cd40cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:12.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.095+0000 7f5cd9287640 1 -- 172.21.15.105:0/2966167078 >> v1:172.21.15.148:6789/0 conn(0x7f5cd4159f40 legacy=0x7f5cd40ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:12.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.095+0000 7f5cd9287640 1 -- 172.21.15.105:0/2966167078 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd4071380 con 0x7f5cd40ad850 2023-11-28T18:27:12.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.095+0000 7f5cd3fff640 1 --2- 172.21.15.105:0/2966167078 >> v2:172.21.15.148:3300/0 conn(0x7f5cd40caf60 0x7f5cd40cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:12.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.095+0000 7f5cd9287640 1 --2- 172.21.15.105:0/2966167078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cd40ad850 0x7f5cd40701d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f5cd409d0d0 tx=0x7f5ccc01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867ddce4486cdb67 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:12.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.095+0000 7f5cd37fe640 1 -- 172.21.15.105:0/2966167078 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc0a57d0 con 0x7f5cd40ad850 2023-11-28T18:27:12.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cd37fe640 1 -- 172.21.15.105:0/2966167078 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ccc0a5930 con 0x7f5cd40ad850 2023-11-28T18:27:12.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cd37fe640 1 -- 172.21.15.105:0/2966167078 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc0a5cc0 con 0x7f5cd40ad850 2023-11-28T18:27:12.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cdacea640 1 -- 172.21.15.105:0/2966167078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cd40ad850 msgr2=0x7f5cd40701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:12.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cdacea640 1 --2- 172.21.15.105:0/2966167078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cd40ad850 0x7f5cd40701d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f5cd409d0d0 tx=0x7f5ccc01c910 comp rx=0 tx=0).stop 2023-11-28T18:27:12.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cdacea640 1 -- 172.21.15.105:0/2966167078 shutdown_connections 2023-11-28T18:27:12.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cdacea640 1 --2- 172.21.15.105:0/2966167078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cd40ad850 0x7f5cd40701d0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:12.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cdacea640 1 --2- 172.21.15.105:0/2966167078 >> v2:172.21.15.148:3300/0 conn(0x7f5cd40caf60 0x7f5cd40cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:12.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cdacea640 1 -- 172.21.15.105:0/2966167078 >> 172.21.15.105:0/2966167078 conn(0x7f5cd40ceb90 msgr2=0x7f5cd40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:12.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cdacea640 1 -- 172.21.15.105:0/2966167078 shutdown_connections 2023-11-28T18:27:12.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cdacea640 1 -- 172.21.15.105:0/2966167078 wait complete. 2023-11-28T18:27:12.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cdacea640 1 Processor -- start 2023-11-28T18:27:12.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cdacea640 1 -- start start 2023-11-28T18:27:12.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cdacea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5cd40ad850 0x7f5cd4149500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:12.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cdacea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cd40caf60 0x7f5cd41514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:12.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cdacea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cd4159f40 0x7f5cd415bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:12.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cdacea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd41472b0 con 0x7f5cd4159f40 2023-11-28T18:27:12.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cdacea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd4147590 con 0x7f5cd40caf60 2023-11-28T18:27:12.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cdacea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd4147870 con 0x7f5cd40ad850 2023-11-28T18:27:12.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cd9287640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cd4159f40 0x7f5cd415bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:12.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cd9287640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cd4159f40 0x7f5cd415bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55926/0 (socket says 172.21.15.105:55926) 2023-11-28T18:27:12.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cd9287640 1 -- 172.21.15.105:0/311985594 learned_addr learned my addr 172.21.15.105:0/311985594 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:12.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cd3fff640 1 --2- 172.21.15.105:0/311985594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cd40caf60 0x7f5cd41514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:12.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.099+0000 7f5cd8a86640 1 --2- 172.21.15.105:0/311985594 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5cd40ad850 0x7f5cd4149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:12.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.103+0000 7f5cd9287640 1 -- 172.21.15.105:0/311985594 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5cd40ad850 msgr2=0x7f5cd4149500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:12.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.103+0000 7f5cd9287640 1 --2- 172.21.15.105:0/311985594 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5cd40ad850 0x7f5cd4149500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:12.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.103+0000 7f5cd9287640 1 -- 172.21.15.105:0/311985594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cd40caf60 msgr2=0x7f5cd41514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:12.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.103+0000 7f5cd9287640 1 --2- 172.21.15.105:0/311985594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cd40caf60 0x7f5cd41514e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:12.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.103+0000 7f5cd9287640 1 -- 172.21.15.105:0/311985594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ccc009b30 con 0x7f5cd4159f40 2023-11-28T18:27:12.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.103+0000 7f5cd8a86640 1 --2- 172.21.15.105:0/311985594 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5cd40ad850 0x7f5cd4149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:12.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.103+0000 7f5cd9287640 1 --2- 172.21.15.105:0/311985594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cd4159f40 0x7f5cd415bbb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc0af040 tx=0x7f5ccc0196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:12.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.103+0000 7f5cd17fa640 1 -- 172.21.15.105:0/311985594 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc0b8670 con 0x7f5cd4159f40 2023-11-28T18:27:12.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.103+0000 7f5cd17fa640 1 -- 172.21.15.105:0/311985594 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ccc0b87f0 con 0x7f5cd4159f40 2023-11-28T18:27:12.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.103+0000 7f5cd17fa640 1 -- 172.21.15.105:0/311985594 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc0b8ba0 con 0x7f5cd4159f40 2023-11-28T18:27:12.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.103+0000 7f5cdacea640 1 -- 172.21.15.105:0/311985594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd4147b50 con 0x7f5cd4159f40 2023-11-28T18:27:12.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.103+0000 7f5cdacea640 1 -- 172.21.15.105:0/311985594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd4146ef0 con 0x7f5cd4159f40 2023-11-28T18:27:12.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.103+0000 7f5cdacea640 1 -- 172.21.15.105:0/311985594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd4147080 con 0x7f5cd4159f40 2023-11-28T18:27:12.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.103+0000 7f5cd17fa640 1 -- 172.21.15.105:0/311985594 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5ccc0b8d00 con 0x7f5cd4159f40 2023-11-28T18:27:12.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.107+0000 7f5cd17fa640 1 --2- 172.21.15.105:0/311985594 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5cb80668d0 0x7f5cb8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:12.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.107+0000 7f5cd17fa640 1 -- 172.21.15.105:0/311985594 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5ccc11fad0 con 0x7f5cd4159f40 2023-11-28T18:27:12.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.107+0000 7f5cd8a86640 1 --2- 172.21.15.105:0/311985594 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5cb80668d0 0x7f5cb8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:12.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.107+0000 7f5cd8a86640 1 --2- 172.21.15.105:0/311985594 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5cb80668d0 0x7f5cb8068d90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8000f70 tx=0x7f5cc8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:12.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.111+0000 7f5cd17fa640 1 -- 172.21.15.105:0/311985594 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5ccc0ecbb0 con 0x7f5cd4159f40 2023-11-28T18:27:12.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.291+0000 7f5cdacea640 1 -- 172.21.15.105:0/311985594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5cd41479b0 con 0x7f5cd4159f40 2023-11-28T18:27:12.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.291+0000 7f5cd17fa640 1 -- 172.21.15.105:0/311985594 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5ccc0bc060 con 0x7f5cd4159f40 2023-11-28T18:27:12.297 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T18:27:12.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.295+0000 7f5cdacea640 1 -- 172.21.15.105:0/311985594 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5cb80668d0 msgr2=0x7f5cb8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:12.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.295+0000 7f5cdacea640 1 --2- 172.21.15.105:0/311985594 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5cb80668d0 0x7f5cb8068d90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8000f70 tx=0x7f5cc8009040 comp rx=0 tx=0).stop 2023-11-28T18:27:12.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.295+0000 7f5cdacea640 1 -- 172.21.15.105:0/311985594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cd4159f40 msgr2=0x7f5cd415bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.295+0000 7f5cdacea640 1 --2- 172.21.15.105:0/311985594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cd4159f40 0x7f5cd415bbb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc0af040 tx=0x7f5ccc0196b0 comp rx=0 tx=0).stop 2023-11-28T18:27:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.295+0000 7f5cdacea640 1 -- 172.21.15.105:0/311985594 shutdown_connections 2023-11-28T18:27:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.295+0000 7f5cdacea640 1 --2- 172.21.15.105:0/311985594 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5cb80668d0 0x7f5cb8068d90 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.295+0000 7f5cdacea640 1 --2- 172.21.15.105:0/311985594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cd4159f40 0x7f5cd415bbb0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.295+0000 7f5cdacea640 1 --2- 172.21.15.105:0/311985594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cd40caf60 0x7f5cd41514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.295+0000 7f5cdacea640 1 --2- 172.21.15.105:0/311985594 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5cd40ad850 0x7f5cd4149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.295+0000 7f5cdacea640 1 -- 172.21.15.105:0/311985594 >> 172.21.15.105:0/311985594 conn(0x7f5cd40ceb90 msgr2=0x7f5cd40cd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.295+0000 7f5cdacea640 1 -- 172.21.15.105:0/311985594 shutdown_connections 2023-11-28T18:27:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:12.295+0000 7f5cdacea640 1 -- 172.21.15.105:0/311985594 wait complete. 2023-11-28T18:27:12.318 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T18:27:12.318 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T18:27:12.318 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2023-11-28T18:27:12.319 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T18:27:12.319 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T18:27:12.319 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T18:27:12.319 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T18:27:12.319 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T18:27:12.319 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:27:12.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:27:12.319 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2023-11-28T18:27:12.329 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T18:27:12.329 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T18:27:12.330 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T18:27:12.330 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:27:12.330 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-11-28T18:27:12.330 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:27:12.331 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:27:12.331 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:27:12.331 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2023-11-28T18:27:12.331 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:27:12.332 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:27:12.332 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2023-11-28T18:27:12.332 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:27:12.333 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:27:12.333 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T18:27:12.333 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:27:12.333 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-28T18:27:12.333 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42189sec preferred_lft 42189sec 2023-11-28T18:27:12.333 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2023-11-28T18:27:12.333 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:27:12.333 INFO:teuthology.orchestra.run.smithi105.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-28T18:27:12.334 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 02:42:cc:d4:87:d2 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:27:12.334 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-28T18:27:12.334 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:27:12.334 INFO:teuthology.orchestra.run.smithi105.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T18:27:12.334 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-28T18:27:12.334 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T18:27:12.334 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:27:12.334 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-28T18:27:12.334 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:27:12.334 INFO:teuthology.orchestra.run.smithi105.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T18:27:12.334 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:27:12.335 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-28T18:27:12.335 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:27:12.335 INFO:teuthology.orchestra.run.smithi105.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T18:27:12.335 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:27:12.335 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-28T18:27:12.335 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:27:12.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:27:12.335 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T18:27:12.391 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-11-28T18:27:12.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:27:12.393 DEBUG:teuthology.orchestra.run.smithi105:> route 2023-11-28T18:27:12.447 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2023-11-28T18:27:12.447 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T18:27:12.447 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-28T18:27:12.448 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T18:27:12.448 INFO:teuthology.orchestra.run.smithi105.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-28T18:27:12.448 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T18:27:12.448 INFO:teuthology.orchestra.run.smithi105.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:27:12.448 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:27:12.448 INFO:teuthology.orchestra.run.smithi105.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:27:12.448 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T18:27:12.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:27:12.449 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-11-28T18:27:12.449 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:27:12.449 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T18:27:12.449 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T18:27:12.449 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T18:27:12.449 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:27:12.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:27:12.543 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-11-28T18:27:12.552 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T18:27:12.553 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T18:27:12.553 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:27:12.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:27:12.553 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2023-11-28T18:27:12.617 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-28T18:27:12.617 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:27:12.617 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T18:27:12.669 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:27:12.670 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:27:12.670 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T18:27:12.724 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:27:12.786 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:27:12.842 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:27:12.842 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:27:12.842 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:27:12.842 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:27:12.843 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:27:12.843 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:27:12.843 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:27:12.843 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:27:12.843 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:27:12.843 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:27:12.843 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:27:12.843 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:27:12.843 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:27:12.843 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:27:12.843 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:27:12.843 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:27:12.844 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:27:12.844 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:27:12.844 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:27:12.844 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:27:12.844 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:27:12.844 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:27:12.844 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:27:12.844 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:27:12.844 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:27:12.844 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:27:12.844 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:27:12.844 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:27:12.845 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:27:12.845 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:27:12.845 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:27:12.845 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:27:12.845 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:27:12.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:27:12.846 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:27:12.897 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T18:27:12.897 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-28T18:27:12.944 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:27:12.966 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:27:13.018 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:27:13.019 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:27:13.019 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:27:13.019 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:27:13.019 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:27:13.019 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:27:13.019 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:27:13.020 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:27:13.020 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:27:13.020 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:27:13.020 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:27:13.020 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:27:13.020 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:27:13.020 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:27:13.021 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:27:13.021 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:27:13.021 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:27:13.021 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:27:13.021 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:27:13.021 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:27:13.021 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:27:13.021 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:27:13.022 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:27:13.022 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:27:13.022 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:27:13.022 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:27:13.022 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:27:13.022 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:27:13.022 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:27:13.023 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:27:13.023 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:27:13.023 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:27:13.023 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:27:13.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:27:13.024 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:27:13.039 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:2023-11-28T18:27:13.035+0000 7f3425d13580 -1 init, newargv = 0x564a47295050 newargc=15 2023-11-28T18:27:13.039 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[55430]: starting ceph client 2023-11-28T18:27:13.046 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[55430]: starting fuse 2023-11-28T18:27:13.048 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-11-28T18:27:13.049 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-28T18:27:13.049 DEBUG:teuthology.orchestra.run.smithi105:> sudo stdin-killer -- python3 -c ' 2023-11-28T18:27:13.049 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2023-11-28T18:27:13.049 DEBUG:teuthology.orchestra.run.smithi105:> import re 2023-11-28T18:27:13.049 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-11-28T18:27:13.049 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2023-11-28T18:27:13.050 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:27:13.050 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2023-11-28T18:27:13.050 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-28T18:27:13.050 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2023-11-28T18:27:13.050 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-28T18:27:13.050 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:27:13.050 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2023-11-28T18:27:13.050 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2023-11-28T18:27:13.051 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2023-11-28T18:27:13.051 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2023-11-28T18:27:13.051 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:27:13.051 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2023-11-28T18:27:13.051 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T18:27:13.051 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T18:27:13.051 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T18:27:13.051 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2023-11-28T18:27:13.051 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2023-11-28T18:27:13.052 DEBUG:teuthology.orchestra.run.smithi105:> return f 2023-11-28T18:27:13.052 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T18:27:13.052 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:27:13.052 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.0")) 2023-11-28T18:27:13.052 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-11-28T18:27:13.183 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.0.55430.asok 2023-11-28T18:27:13.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-28T18:27:13.194 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.55430.asok 2023-11-28T18:27:13.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:27:13.194 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.55430.asok status 2023-11-28T18:27:13.366 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:27:13.366 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2023-11-28T18:27:13.366 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "47c1c547372ed5bff7984647dc200bac0e019caf", 2023-11-28T18:27:13.366 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev)", 2023-11-28T18:27:13.366 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "0", 2023-11-28T18:27:13.367 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2023-11-28T18:27:13.367 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-28T18:27:13.367 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "55430", 2023-11-28T18:27:13.367 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2023-11-28T18:27:13.367 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:27:13.367 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2023-11-28T18:27:13.367 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2023-11-28T18:27:13.367 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 6864, 2023-11-28T18:27:13.367 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2023-11-28T18:27:13.367 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2023-11-28T18:27:13.367 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2023-11-28T18:27:13.367 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 6864 2023-11-28T18:27:13.368 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:27:13.368 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-11-28T18:27:13.368 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-11-28T18:27:13.368 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-11-28T18:27:13.368 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 2794316481 2023-11-28T18:27:13.368 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:27:13.368 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:27:13.368 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-11-28T18:27:13.368 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-11-28T18:27:13.368 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-11-28T18:27:13.368 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 2794316481 2023-11-28T18:27:13.368 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:27:13.369 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.6864 192.168.0.1:0/2794316481", 2023-11-28T18:27:13.369 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/2794316481", 2023-11-28T18:27:13.369 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2023-11-28T18:27:13.369 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 56, 2023-11-28T18:27:13.369 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 65, 2023-11-28T18:27:13.369 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2023-11-28T18:27:13.369 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2023-11-28T18:27:13.369 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2023-11-28T18:27:13.369 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:27:13.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:27:13.381 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2023-11-28T18:27:13.381 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-28T18:27:13.391 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2023-11-28T18:27:13.391 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-28T18:27:13.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:27:13.392 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T18:27:13.453 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random True 2023-11-28T18:27:13.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.635+0000 7facaba46640 1 Processor -- start 2023-11-28T18:27:13.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.635+0000 7facaba46640 1 -- start start 2023-11-28T18:27:13.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.635+0000 7facaba46640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7faca406d610 0x7faca40cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:13.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.635+0000 7facaba46640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faca40ae100 0x7faca40cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:13.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.635+0000 7facaba46640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca4162820 con 0x7faca40ce530 2023-11-28T18:27:13.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.639+0000 7facaba46640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca4161c10 con 0x7faca406d610 2023-11-28T18:27:13.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.639+0000 7facaba46640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca4161ef0 con 0x7faca40ae100 2023-11-28T18:27:13.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.639+0000 7faca8fe1640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7faca40ce530 0x7faca40ad9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47172/0 (socket says 172.21.15.105:47172) 2023-11-28T18:27:13.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.639+0000 7faca8fe1640 1 -- 172.21.15.105:0/2473416038 learned_addr learned my addr 172.21.15.105:0/2473416038 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:13.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.639+0000 7faca97e2640 1 --2- 172.21.15.105:0/2473416038 >> v2:172.21.15.105:3300/0 conn(0x7faca406d610 0x7faca40cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:13.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.639+0000 7faca9fe3640 1 --2- 172.21.15.105:0/2473416038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faca40ae100 0x7faca40cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:13.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.639+0000 7fac9bfff640 1 -- 172.21.15.105:0/2473416038 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296554906 0 0) 0x7faca4162820 con 0x7faca40ce530 2023-11-28T18:27:13.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.639+0000 7fac9bfff640 1 -- 172.21.15.105:0/2473416038 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac90003530 con 0x7faca40ce530 2023-11-28T18:27:13.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.639+0000 7faca97e2640 1 -- 172.21.15.105:0/2473416038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faca40ae100 msgr2=0x7faca40cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:13.643 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.639+0000 7faca97e2640 1 --2- 172.21.15.105:0/2473416038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faca40ae100 0x7faca40cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:13.643 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.639+0000 7faca97e2640 1 -- 172.21.15.105:0/2473416038 >> v1:172.21.15.105:6789/0 conn(0x7faca40ce530 legacy=0x7faca40ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:13.643 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.639+0000 7faca97e2640 1 -- 172.21.15.105:0/2473416038 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca41621d0 con 0x7faca406d610 2023-11-28T18:27:13.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.639+0000 7faca97e2640 1 --2- 172.21.15.105:0/2473416038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faca406d610 0x7faca40cdff0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7faca00097c0 tx=0x7faca001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27877596be54acd9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:13.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.639+0000 7fac9bfff640 1 -- 172.21.15.105:0/2473416038 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca00ab670 con 0x7faca406d610 2023-11-28T18:27:13.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.643+0000 7fac9bfff640 1 -- 172.21.15.105:0/2473416038 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca00ab7d0 con 0x7faca406d610 2023-11-28T18:27:13.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.643+0000 7fac9bfff640 1 -- 172.21.15.105:0/2473416038 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca00abb40 con 0x7faca406d610 2023-11-28T18:27:13.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.643+0000 7facaba46640 1 -- 172.21.15.105:0/2473416038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faca406d610 msgr2=0x7faca40cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:13.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.643+0000 7facaba46640 1 --2- 172.21.15.105:0/2473416038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faca406d610 0x7faca40cdff0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7faca00097c0 tx=0x7faca001c450 comp rx=0 tx=0).stop 2023-11-28T18:27:13.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.643+0000 7facaba46640 1 -- 172.21.15.105:0/2473416038 shutdown_connections 2023-11-28T18:27:13.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.643+0000 7facaba46640 1 --2- 172.21.15.105:0/2473416038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faca40ae100 0x7faca40cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:13.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.643+0000 7facaba46640 1 --2- 172.21.15.105:0/2473416038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faca406d610 0x7faca40cdff0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:13.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.643+0000 7facaba46640 1 -- 172.21.15.105:0/2473416038 >> 172.21.15.105:0/2473416038 conn(0x7faca40ceb90 msgr2=0x7faca40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:13.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.643+0000 7facaba46640 1 -- 172.21.15.105:0/2473416038 shutdown_connections 2023-11-28T18:27:13.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.643+0000 7facaba46640 1 -- 172.21.15.105:0/2473416038 wait complete. 2023-11-28T18:27:13.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.643+0000 7facaba46640 1 Processor -- start 2023-11-28T18:27:13.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.643+0000 7facaba46640 1 -- start start 2023-11-28T18:27:13.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.643+0000 7facaba46640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faca406d610 0x7faca406c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:13.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.643+0000 7facaba46640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faca40ae100 0x7faca415bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:13.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.643+0000 7facaba46640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faca40ce530 0x7faca415c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:13.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.643+0000 7facaba46640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca41649b0 con 0x7faca406d610 2023-11-28T18:27:13.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.643+0000 7facaba46640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca4164c90 con 0x7faca40ce530 2023-11-28T18:27:13.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.643+0000 7facaba46640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca4164f70 con 0x7faca40ae100 2023-11-28T18:27:13.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.643+0000 7faca97e2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faca406d610 0x7faca406c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:13.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.643+0000 7faca97e2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faca406d610 0x7faca406c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55944/0 (socket says 172.21.15.105:55944) 2023-11-28T18:27:13.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.643+0000 7faca97e2640 1 -- 172.21.15.105:0/104231516 learned_addr learned my addr 172.21.15.105:0/104231516 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:13.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.647+0000 7faca9fe3640 1 --2- 172.21.15.105:0/104231516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faca40ce530 0x7faca415c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:13.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.647+0000 7faca8fe1640 1 --2- 172.21.15.105:0/104231516 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faca40ae100 0x7faca415bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:13.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.647+0000 7faca97e2640 1 -- 172.21.15.105:0/104231516 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faca40ae100 msgr2=0x7faca415bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:13.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.647+0000 7faca97e2640 1 --2- 172.21.15.105:0/104231516 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faca40ae100 0x7faca415bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:13.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.647+0000 7faca97e2640 1 -- 172.21.15.105:0/104231516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faca40ce530 msgr2=0x7faca415c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:13.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.647+0000 7faca97e2640 1 --2- 172.21.15.105:0/104231516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faca40ce530 0x7faca415c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:13.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.647+0000 7faca97e2640 1 -- 172.21.15.105:0/104231516 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca0009420 con 0x7faca406d610 2023-11-28T18:27:13.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.647+0000 7faca9fe3640 1 --2- 172.21.15.105:0/104231516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faca40ce530 0x7faca415c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:27:13.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.647+0000 7faca8fe1640 1 --2- 172.21.15.105:0/104231516 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faca40ae100 0x7faca415bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:27:13.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.647+0000 7faca97e2640 1 --2- 172.21.15.105:0/104231516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faca406d610 0x7faca406c1b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7faca00ad040 tx=0x7faca001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:13.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.647+0000 7fac99ffb640 1 -- 172.21.15.105:0/104231516 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca00bf610 con 0x7faca406d610 2023-11-28T18:27:13.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.647+0000 7fac99ffb640 1 -- 172.21.15.105:0/104231516 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca00bf770 con 0x7faca406d610 2023-11-28T18:27:13.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.647+0000 7facaba46640 1 -- 172.21.15.105:0/104231516 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca4165250 con 0x7faca406d610 2023-11-28T18:27:13.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.647+0000 7fac99ffb640 1 -- 172.21.15.105:0/104231516 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca00bfae0 con 0x7faca406d610 2023-11-28T18:27:13.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.647+0000 7facaba46640 1 -- 172.21.15.105:0/104231516 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca4164660 con 0x7faca406d610 2023-11-28T18:27:13.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.647+0000 7facaba46640 1 -- 172.21.15.105:0/104231516 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca4067570 con 0x7faca406d610 2023-11-28T18:27:13.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.647+0000 7fac99ffb640 1 -- 172.21.15.105:0/104231516 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7faca00bfc40 con 0x7faca406d610 2023-11-28T18:27:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.647+0000 7fac99ffb640 1 --2- 172.21.15.105:0/104231516 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fac88066900 0x7fac88068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.651+0000 7fac99ffb640 1 -- 172.21.15.105:0/104231516 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7faca011f230 con 0x7faca406d610 2023-11-28T18:27:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.651+0000 7faca8fe1640 1 --2- 172.21.15.105:0/104231516 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fac88066900 0x7fac88068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.651+0000 7faca8fe1640 1 --2- 172.21.15.105:0/104231516 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fac88066900 0x7fac88068dc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7faca4165f10 tx=0x7fac94009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:13.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.655+0000 7fac99ffb640 1 -- 172.21.15.105:0/104231516 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faca00ec290 con 0x7faca406d610 2023-11-28T18:27:13.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.803+0000 7facaba46640 1 -- 172.21.15.105:0/104231516 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random}] v 0) v1 -- 0x7faca4165700 con 0x7faca406d610 2023-11-28T18:27:13.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.807+0000 7fac99ffb640 1 -- 172.21.15.105:0/104231516 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random}]=0 v28)=0 v28) v1 ==== 128+0+0 (secure 0 0 0) 0x7faca00ef8b0 con 0x7faca406d610 2023-11-28T18:27:13.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.811+0000 7facaba46640 1 -- 172.21.15.105:0/104231516 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fac88066900 msgr2=0x7fac88068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:13.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.811+0000 7facaba46640 1 --2- 172.21.15.105:0/104231516 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fac88066900 0x7fac88068dc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7faca4165f10 tx=0x7fac94009040 comp rx=0 tx=0).stop 2023-11-28T18:27:13.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.811+0000 7facaba46640 1 -- 172.21.15.105:0/104231516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faca406d610 msgr2=0x7faca406c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:13.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.811+0000 7facaba46640 1 --2- 172.21.15.105:0/104231516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faca406d610 0x7faca406c1b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7faca00ad040 tx=0x7faca001ac90 comp rx=0 tx=0).stop 2023-11-28T18:27:13.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.811+0000 7facaba46640 1 -- 172.21.15.105:0/104231516 shutdown_connections 2023-11-28T18:27:13.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.811+0000 7facaba46640 1 --2- 172.21.15.105:0/104231516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faca40ce530 0x7faca415c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:13.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.811+0000 7facaba46640 1 --2- 172.21.15.105:0/104231516 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faca40ae100 0x7faca415bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:13.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.811+0000 7facaba46640 1 --2- 172.21.15.105:0/104231516 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fac88066900 0x7fac88068dc0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:13.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.811+0000 7facaba46640 1 --2- 172.21.15.105:0/104231516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faca406d610 0x7faca406c1b0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:13.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.811+0000 7facaba46640 1 -- 172.21.15.105:0/104231516 >> 172.21.15.105:0/104231516 conn(0x7faca40ceb90 msgr2=0x7faca40ac040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:13.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.811+0000 7facaba46640 1 -- 172.21.15.105:0/104231516 shutdown_connections 2023-11-28T18:27:13.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.811+0000 7facaba46640 1 -- 172.21.15.105:0/104231516 wait complete. 2023-11-28T18:27:13.834 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_distributed True 2023-11-28T18:27:13.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.971+0000 7fee896d6640 1 Processor -- start 2023-11-28T18:27:13.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.971+0000 7fee896d6640 1 -- start start 2023-11-28T18:27:13.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.971+0000 7fee896d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee840cf0c0 0x7fee840cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:13.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.971+0000 7fee896d6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee8415d560 con 0x7fee84157520 2023-11-28T18:27:13.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.971+0000 7fee896d6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee840cf550 con 0x7fee840cec40 2023-11-28T18:27:13.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.971+0000 7fee896d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee8415d820 con 0x7fee840cf0c0 2023-11-28T18:27:13.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.971+0000 7fee82ffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fee84157520 0x7fee840ad8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47184/0 (socket says 172.21.15.105:47184) 2023-11-28T18:27:13.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.971+0000 7fee82ffd640 1 -- 172.21.15.105:0/3439628000 learned_addr learned my addr 172.21.15.105:0/3439628000 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:13.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.971+0000 7fee837fe640 1 --2- 172.21.15.105:0/3439628000 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee840cf0c0 0x7fee840cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:13.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.971+0000 7fee827fc640 1 -- 172.21.15.105:0/3439628000 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2091021607 0 0) 0x7fee8415d560 con 0x7fee84157520 2023-11-28T18:27:13.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.971+0000 7fee827fc640 1 -- 172.21.15.105:0/3439628000 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee70003530 con 0x7fee84157520 2023-11-28T18:27:13.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.971+0000 7fee827fc640 1 -- 172.21.15.105:0/3439628000 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165413540 0 0) 0x7fee840cf550 con 0x7fee840cec40 2023-11-28T18:27:13.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.971+0000 7fee827fc640 1 -- 172.21.15.105:0/3439628000 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee8415d560 con 0x7fee840cec40 2023-11-28T18:27:13.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.971+0000 7fee837fe640 1 -- 172.21.15.105:0/3439628000 >> v1:172.21.15.148:6789/0 conn(0x7fee840cec40 legacy=0x7fee840ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:13.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.971+0000 7fee837fe640 1 -- 172.21.15.105:0/3439628000 >> v1:172.21.15.105:6789/0 conn(0x7fee84157520 legacy=0x7fee840ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:13.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.971+0000 7fee837fe640 1 -- 172.21.15.105:0/3439628000 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8415db00 con 0x7fee840cf0c0 2023-11-28T18:27:13.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.975+0000 7fee837fe640 1 --2- 172.21.15.105:0/3439628000 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee840cf0c0 0x7fee840cd1d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009dc0 tx=0x7fee7c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c5f7e89d15cf9b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:13.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.975+0000 7fee827fc640 1 -- 172.21.15.105:0/3439628000 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c0a5a50 con 0x7fee840cf0c0 2023-11-28T18:27:13.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.975+0000 7fee827fc640 1 -- 172.21.15.105:0/3439628000 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7c0a5bb0 con 0x7fee840cf0c0 2023-11-28T18:27:13.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.975+0000 7fee827fc640 1 -- 172.21.15.105:0/3439628000 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c0ad440 con 0x7fee840cf0c0 2023-11-28T18:27:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.975+0000 7fee896d6640 1 -- 172.21.15.105:0/3439628000 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee840cf0c0 msgr2=0x7fee840cd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.975+0000 7fee896d6640 1 --2- 172.21.15.105:0/3439628000 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee840cf0c0 0x7fee840cd1d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009dc0 tx=0x7fee7c01c970 comp rx=0 tx=0).stop 2023-11-28T18:27:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.975+0000 7fee896d6640 1 -- 172.21.15.105:0/3439628000 shutdown_connections 2023-11-28T18:27:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.975+0000 7fee896d6640 1 --2- 172.21.15.105:0/3439628000 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee840cf0c0 0x7fee840cd1d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.975+0000 7fee896d6640 1 -- 172.21.15.105:0/3439628000 >> 172.21.15.105:0/3439628000 conn(0x7fee8406a750 msgr2=0x7fee8406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.975+0000 7fee896d6640 1 -- 172.21.15.105:0/3439628000 shutdown_connections 2023-11-28T18:27:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.975+0000 7fee896d6640 1 -- 172.21.15.105:0/3439628000 wait complete. 2023-11-28T18:27:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee896d6640 1 Processor -- start 2023-11-28T18:27:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee896d6640 1 -- start start 2023-11-28T18:27:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee896d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee840cec40 0x7fee84145160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee896d6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee840cf0c0 0x7fee8415cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee896d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee84157520 0x7fee8406d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee896d6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee84159d20 con 0x7fee840cf0c0 2023-11-28T18:27:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee896d6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee841456a0 con 0x7fee84157520 2023-11-28T18:27:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee896d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee841590a0 con 0x7fee840cec40 2023-11-28T18:27:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee7a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee840cf0c0 0x7fee8415cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee82ffd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee840cec40 0x7fee84145160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee7a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee840cf0c0 0x7fee8415cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55950/0 (socket says 172.21.15.105:55950) 2023-11-28T18:27:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee7a7fc640 1 -- 172.21.15.105:0/4089328100 learned_addr learned my addr 172.21.15.105:0/4089328100 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:13.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee837fe640 1 --2- 172.21.15.105:0/4089328100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee84157520 0x7fee8406d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:13.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee7a7fc640 1 -- 172.21.15.105:0/4089328100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee840cec40 msgr2=0x7fee84145160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:13.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee7a7fc640 1 --2- 172.21.15.105:0/4089328100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee840cec40 0x7fee84145160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:13.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee7a7fc640 1 -- 172.21.15.105:0/4089328100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee84157520 msgr2=0x7fee8406d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:13.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee7a7fc640 1 --2- 172.21.15.105:0/4089328100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee84157520 0x7fee8406d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:13.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee7a7fc640 1 -- 172.21.15.105:0/4089328100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c009a70 con 0x7fee840cf0c0 2023-11-28T18:27:13.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee7a7fc640 1 --2- 172.21.15.105:0/4089328100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee840cf0c0 0x7fee8415cca0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00c430 tx=0x7fee6c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:13.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee7bfff640 1 -- 172.21.15.105:0/4089328100 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c0052d0 con 0x7fee840cf0c0 2023-11-28T18:27:13.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee7bfff640 1 -- 172.21.15.105:0/4089328100 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6c005450 con 0x7fee840cf0c0 2023-11-28T18:27:13.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee7bfff640 1 -- 172.21.15.105:0/4089328100 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c003840 con 0x7fee840cf0c0 2023-11-28T18:27:13.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee896d6640 1 -- 172.21.15.105:0/4089328100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee84159380 con 0x7fee840cf0c0 2023-11-28T18:27:13.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.979+0000 7fee896d6640 1 -- 172.21.15.105:0/4089328100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee84159940 con 0x7fee840cf0c0 2023-11-28T18:27:13.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.983+0000 7fee896d6640 1 -- 172.21.15.105:0/4089328100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee84074580 con 0x7fee840cf0c0 2023-11-28T18:27:13.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.983+0000 7fee7bfff640 1 -- 172.21.15.105:0/4089328100 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fee6c0039a0 con 0x7fee840cf0c0 2023-11-28T18:27:13.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.983+0000 7fee7bfff640 1 --2- 172.21.15.105:0/4089328100 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fee640668d0 0x7fee64068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:13.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.983+0000 7fee7bfff640 1 -- 172.21.15.105:0/4089328100 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fee6c083bc0 con 0x7fee840cf0c0 2023-11-28T18:27:13.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.983+0000 7fee82ffd640 1 --2- 172.21.15.105:0/4089328100 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fee640668d0 0x7fee64068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:13.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.983+0000 7fee82ffd640 1 --2- 172.21.15.105:0/4089328100 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fee640668d0 0x7fee64068d90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fee74007fd0 tx=0x7fee74007d60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:13.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:13.987+0000 7fee7bfff640 1 -- 172.21.15.105:0/4089328100 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fee6c050cd0 con 0x7fee840cf0c0 2023-11-28T18:27:14.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.139+0000 7fee896d6640 1 -- 172.21.15.105:0/4089328100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_distributed}] v 0) v1 -- 0x7fee840ada60 con 0x7fee840cf0c0 2023-11-28T18:27:14.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.143+0000 7fee7bfff640 1 -- 172.21.15.105:0/4089328100 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_distributed}]=0 v29)=0 v29) v1 ==== 133+0+0 (secure 0 0 0) 0x7fee6c0170c0 con 0x7fee840cf0c0 2023-11-28T18:27:14.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.147+0000 7fee896d6640 1 -- 172.21.15.105:0/4089328100 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fee640668d0 msgr2=0x7fee64068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:14.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.147+0000 7fee896d6640 1 --2- 172.21.15.105:0/4089328100 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fee640668d0 0x7fee64068d90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fee74007fd0 tx=0x7fee74007d60 comp rx=0 tx=0).stop 2023-11-28T18:27:14.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.147+0000 7fee896d6640 1 -- 172.21.15.105:0/4089328100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee840cf0c0 msgr2=0x7fee8415cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:14.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.147+0000 7fee896d6640 1 --2- 172.21.15.105:0/4089328100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee840cf0c0 0x7fee8415cca0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00c430 tx=0x7fee6c00c8f0 comp rx=0 tx=0).stop 2023-11-28T18:27:14.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.147+0000 7fee896d6640 1 -- 172.21.15.105:0/4089328100 shutdown_connections 2023-11-28T18:27:14.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.147+0000 7fee896d6640 1 --2- 172.21.15.105:0/4089328100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee84157520 0x7fee8406d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:14.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.147+0000 7fee896d6640 1 --2- 172.21.15.105:0/4089328100 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fee640668d0 0x7fee64068d90 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:14.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.147+0000 7fee896d6640 1 --2- 172.21.15.105:0/4089328100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee840cf0c0 0x7fee8415cca0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:14.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.147+0000 7fee896d6640 1 --2- 172.21.15.105:0/4089328100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee840cec40 0x7fee84145160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:14.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.147+0000 7fee896d6640 1 -- 172.21.15.105:0/4089328100 >> 172.21.15.105:0/4089328100 conn(0x7fee8406a750 msgr2=0x7fee8414f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:14.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.147+0000 7fee896d6640 1 -- 172.21.15.105:0/4089328100 shutdown_connections 2023-11-28T18:27:14.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.147+0000 7fee896d6640 1 -- 172.21.15.105:0/4089328100 wait complete. 2023-11-28T18:27:14.171 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random_max 1.0 2023-11-28T18:27:14.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f323d965640 1 Processor -- start 2023-11-28T18:27:14.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f323d965640 1 -- start start 2023-11-28T18:27:14.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f323d965640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32380cdd80 0x7f323806c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:14.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f323d965640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f32380ce230 0x7f32380cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:14.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f323d965640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32380b3c40 con 0x7f323815d320 2023-11-28T18:27:14.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f323d965640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323806ca80 con 0x7f32380ce230 2023-11-28T18:27:14.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f323d965640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323815d700 con 0x7f32380cdd80 2023-11-28T18:27:14.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3236ffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f323815d320 0x7f32380cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47198/0 (socket says 172.21.15.105:47198) 2023-11-28T18:27:14.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3236ffd640 1 -- 172.21.15.105:0/1441957918 learned_addr learned my addr 172.21.15.105:0/1441957918 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:14.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f32367fc640 1 --2- 172.21.15.105:0/1441957918 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32380cdd80 0x7f323806c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:14.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f32377fe640 1 --2- 172.21.15.105:0/1441957918 >> v2:172.21.15.148:3300/0 conn(0x7f32380ce230 0x7f32380cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:14.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3235ffb640 1 -- 172.21.15.105:0/1441957918 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441973814 0 0) 0x7f32380b3c40 con 0x7f323815d320 2023-11-28T18:27:14.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3235ffb640 1 -- 172.21.15.105:0/1441957918 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3220003530 con 0x7f323815d320 2023-11-28T18:27:14.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3235ffb640 1 -- 172.21.15.105:0/1441957918 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1983483395 0 0) 0x7f3220003530 con 0x7f323815d320 2023-11-28T18:27:14.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3235ffb640 1 -- 172.21.15.105:0/1441957918 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32380cdd80 msgr2=0x7f323806c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:14.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3235ffb640 1 --2- 172.21.15.105:0/1441957918 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32380cdd80 0x7f323806c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:14.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3235ffb640 1 -- 172.21.15.105:0/1441957918 >> v2:172.21.15.148:3300/0 conn(0x7f32380ce230 msgr2=0x7f32380cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:14.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3235ffb640 1 --2- 172.21.15.105:0/1441957918 >> v2:172.21.15.148:3300/0 conn(0x7f32380ce230 0x7f32380cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:14.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3235ffb640 1 -- 172.21.15.105:0/1441957918 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323806c7a0 con 0x7f323815d320 2023-11-28T18:27:14.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3235ffb640 1 -- 172.21.15.105:0/1441957918 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f3228001680 con 0x7f323815d320 2023-11-28T18:27:14.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3235ffb640 1 -- 172.21.15.105:0/1441957918 >> v1:172.21.15.105:6789/0 conn(0x7f323815d320 legacy=0x7f32380cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:14.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3235ffb640 1 --2- 172.21.15.105:0/1441957918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3220003980 0x7f3220003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:14.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3235ffb640 1 --2- 172.21.15.105:0/1441957918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32200047d0 0x7f3220004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:14.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3235ffb640 1 --2- 172.21.15.105:0/1441957918 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32380cdd80 0x7f3220005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:14.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3235ffb640 1 -- 172.21.15.105:0/1441957918 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323806ca80 con 0x7f3220003980 2023-11-28T18:27:14.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3235ffb640 1 -- 172.21.15.105:0/1441957918 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323815d700 con 0x7f32200047d0 2023-11-28T18:27:14.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3235ffb640 1 -- 172.21.15.105:0/1441957918 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322000b9e0 con 0x7f32380cdd80 2023-11-28T18:27:14.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3236ffd640 1 --2- 172.21.15.105:0/1441957918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3220003980 0x7f3220003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:14.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f32367fc640 1 --2- 172.21.15.105:0/1441957918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32200047d0 0x7f3220004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:14.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f32377fe640 1 --2- 172.21.15.105:0/1441957918 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32380cdd80 0x7f3220005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:14.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3236ffd640 1 -- 172.21.15.105:0/1441957918 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32380cdd80 msgr2=0x7f3220005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:14.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3236ffd640 1 --2- 172.21.15.105:0/1441957918 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32380cdd80 0x7f3220005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:14.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3236ffd640 1 -- 172.21.15.105:0/1441957918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32200047d0 msgr2=0x7f3220004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:14.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3236ffd640 1 --2- 172.21.15.105:0/1441957918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32200047d0 0x7f3220004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:14.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.311+0000 7f3236ffd640 1 -- 172.21.15.105:0/1441957918 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3220003530 con 0x7f3220003980 2023-11-28T18:27:14.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.315+0000 7f3236ffd640 1 --2- 172.21.15.105:0/1441957918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3220003980 0x7f3220003da0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f322800c140 tx=0x7f322801cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ffe0e8f411ae74 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:14.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.315+0000 7f3235ffb640 1 -- 172.21.15.105:0/1441957918 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32280058e0 con 0x7f3220003980 2023-11-28T18:27:14.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.315+0000 7f3235ffb640 1 -- 172.21.15.105:0/1441957918 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3228005a60 con 0x7f3220003980 2023-11-28T18:27:14.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.315+0000 7f3235ffb640 1 -- 172.21.15.105:0/1441957918 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3228005e10 con 0x7f3220003980 2023-11-28T18:27:14.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.315+0000 7f323d965640 1 -- 172.21.15.105:0/1441957918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3220003980 msgr2=0x7f3220003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:14.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.315+0000 7f323d965640 1 --2- 172.21.15.105:0/1441957918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3220003980 0x7f3220003da0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f322800c140 tx=0x7f322801cb50 comp rx=0 tx=0).stop 2023-11-28T18:27:14.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.315+0000 7f32377fe640 1 -- 172.21.15.105:0/1441957918 reap_dead start 2023-11-28T18:27:14.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.315+0000 7f323d965640 1 -- 172.21.15.105:0/1441957918 shutdown_connections 2023-11-28T18:27:14.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.315+0000 7f323d965640 1 -- 172.21.15.105:0/1441957918 >> 172.21.15.105:0/1441957918 conn(0x7f32380d0df0 msgr2=0x7f32380d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:14.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.315+0000 7f323d965640 1 -- 172.21.15.105:0/1441957918 shutdown_connections 2023-11-28T18:27:14.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.315+0000 7f323d965640 1 -- 172.21.15.105:0/1441957918 wait complete. 2023-11-28T18:27:14.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f323d965640 1 Processor -- start 2023-11-28T18:27:14.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f323d965640 1 -- start start 2023-11-28T18:27:14.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f323d965640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3220003980 0x7f32381603f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:14.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f323d965640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32380c7ec0 0x7f32381f5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:14.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f323d965640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32381f9000 0x7f32381f9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:14.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f323d965640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32381f99a0 con 0x7f32381f9000 2023-11-28T18:27:14.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f323d965640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323815f0a0 con 0x7f32380c7ec0 2023-11-28T18:27:14.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f323d965640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323815a8d0 con 0x7f3220003980 2023-11-28T18:27:14.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f32377fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32381f9000 0x7f32381f9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:14.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f32377fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32381f9000 0x7f32381f9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55976/0 (socket says 172.21.15.105:55976) 2023-11-28T18:27:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f32377fe640 1 -- 172.21.15.105:0/1478407997 learned_addr learned my addr 172.21.15.105:0/1478407997 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f3236ffd640 1 --2- 172.21.15.105:0/1478407997 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3220003980 0x7f32381603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f32367fc640 1 --2- 172.21.15.105:0/1478407997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32380c7ec0 0x7f32381f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f32377fe640 1 -- 172.21.15.105:0/1478407997 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3220003980 msgr2=0x7f32381603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f32377fe640 1 --2- 172.21.15.105:0/1478407997 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3220003980 0x7f32381603f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f32377fe640 1 -- 172.21.15.105:0/1478407997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32380c7ec0 msgr2=0x7f32381f5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f32377fe640 1 --2- 172.21.15.105:0/1478407997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32380c7ec0 0x7f32381f5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f32377fe640 1 -- 172.21.15.105:0/1478407997 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323815abb0 con 0x7f32381f9000 2023-11-28T18:27:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f32377fe640 1 --2- 172.21.15.105:0/1478407997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32381f9000 0x7f32381f9460 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f32380716b0 tx=0x7f323000f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f321ffff640 1 -- 172.21.15.105:0/1478407997 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323000e040 con 0x7f32381f9000 2023-11-28T18:27:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f321ffff640 1 -- 172.21.15.105:0/1478407997 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323000f040 con 0x7f32381f9000 2023-11-28T18:27:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f321ffff640 1 -- 172.21.15.105:0/1478407997 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3230015430 con 0x7f32381f9000 2023-11-28T18:27:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f323d965640 1 -- 172.21.15.105:0/1478407997 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323815ae90 con 0x7f32381f9000 2023-11-28T18:27:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.319+0000 7f323d965640 1 -- 172.21.15.105:0/1478407997 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32380c8340 con 0x7f32381f9000 2023-11-28T18:27:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.323+0000 7f323d965640 1 -- 172.21.15.105:0/1478407997 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f323815aa10 con 0x7f32381f9000 2023-11-28T18:27:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.323+0000 7f321ffff640 1 -- 172.21.15.105:0/1478407997 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3230015590 con 0x7f32381f9000 2023-11-28T18:27:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.323+0000 7f321ffff640 1 --2- 172.21.15.105:0/1478407997 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f32240668d0 0x7f3224068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.323+0000 7f321ffff640 1 -- 172.21.15.105:0/1478407997 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3230084b80 con 0x7f32381f9000 2023-11-28T18:27:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.323+0000 7f3236ffd640 1 --2- 172.21.15.105:0/1478407997 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f32240668d0 0x7f3224068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.323+0000 7f3236ffd640 1 --2- 172.21.15.105:0/1478407997 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f32240668d0 0x7f3224068d90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f32280aa000 tx=0x7f32280019b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:14.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.327+0000 7f321ffff640 1 -- 172.21.15.105:0/1478407997 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f323815aa10 con 0x7f32381f9000 2023-11-28T18:27:14.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.479+0000 7f323d965640 1 -- 172.21.15.105:0/1478407997 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random_max}] v 0) v1 -- 0x7f323815aa10 con 0x7f32381f9000 2023-11-28T18:27:14.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.483+0000 7f321ffff640 1 -- 172.21.15.105:0/1478407997 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random_max}]=0 v30)=0 v30) v1 ==== 131+0+0 (secure 0 0 0) 0x7f323815aa10 con 0x7f32381f9000 2023-11-28T18:27:14.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.487+0000 7f323d965640 1 -- 172.21.15.105:0/1478407997 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f32240668d0 msgr2=0x7f3224068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:14.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.487+0000 7f323d965640 1 --2- 172.21.15.105:0/1478407997 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f32240668d0 0x7f3224068d90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f32280aa000 tx=0x7f32280019b0 comp rx=0 tx=0).stop 2023-11-28T18:27:14.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.487+0000 7f323d965640 1 -- 172.21.15.105:0/1478407997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32381f9000 msgr2=0x7f32381f9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:14.490 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.487+0000 7f323d965640 1 --2- 172.21.15.105:0/1478407997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32381f9000 0x7f32381f9460 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f32380716b0 tx=0x7f323000f1b0 comp rx=0 tx=0).stop 2023-11-28T18:27:14.490 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.487+0000 7f323d965640 1 -- 172.21.15.105:0/1478407997 shutdown_connections 2023-11-28T18:27:14.490 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.487+0000 7f323d965640 1 --2- 172.21.15.105:0/1478407997 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f32240668d0 0x7f3224068d90 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:14.490 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.487+0000 7f323d965640 1 --2- 172.21.15.105:0/1478407997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32381f9000 0x7f32381f9460 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:14.490 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.487+0000 7f323d965640 1 --2- 172.21.15.105:0/1478407997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32380c7ec0 0x7f32381f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:14.490 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.487+0000 7f323d965640 1 --2- 172.21.15.105:0/1478407997 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3220003980 0x7f32381603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:14.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.487+0000 7f323d965640 1 -- 172.21.15.105:0/1478407997 >> 172.21.15.105:0/1478407997 conn(0x7f32380d0df0 msgr2=0x7f32380cd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:14.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.487+0000 7f323d965640 1 -- 172.21.15.105:0/1478407997 shutdown_connections 2023-11-28T18:27:14.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:14.487+0000 7f323d965640 1 -- 172.21.15.105:0/1478407997 wait complete. 2023-11-28T18:27:14.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:27:14.508 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-28T18:27:14.508 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:27:14.509 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:27:14.509 DEBUG:teuthology.orchestra.run.smithi105:> # Use up a random number of inode numbers so the ephemeral pinning is not the same every test. 2023-11-28T18:27:14.509 DEBUG:teuthology.orchestra.run.smithi105:> mkdir .inode_number_thrash 2023-11-28T18:27:14.509 DEBUG:teuthology.orchestra.run.smithi105:> count=$((RANDOM % 1024)) 2023-11-28T18:27:14.509 DEBUG:teuthology.orchestra.run.smithi105:> for ((i = 0; i < count; i++)); do touch .inode_number_thrash/$i; done 2023-11-28T18:27:14.509 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf .inode_number_thrash 2023-11-28T18:27:14.509 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:27:22.776 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-11-28T18:27:22.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.911+0000 7efdfbf57640 1 Processor -- start 2023-11-28T18:27:22.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.911+0000 7efdfbf57640 1 -- start start 2023-11-28T18:27:22.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.915+0000 7efdfbf57640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efdf4159f40 0x7efdf40caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:22.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.915+0000 7efdfbf57640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efdf40ad850 0x7efdf40701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:22.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.915+0000 7efdfbf57640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf406bf50 con 0x7efdf40cc760 2023-11-28T18:27:22.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.915+0000 7efdfbf57640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf40ccbc0 con 0x7efdf40ad850 2023-11-28T18:27:22.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.915+0000 7efdfbf57640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf415a320 con 0x7efdf4159f40 2023-11-28T18:27:22.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.915+0000 7efdfa4f4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efdf40ad850 0x7efdf40701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:22.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.915+0000 7efdf94f2640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efdf40cc760 0x7efdf40ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:49530/0 (socket says 172.21.15.105:49530) 2023-11-28T18:27:22.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.915+0000 7efdf9cf3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efdf4159f40 0x7efdf40caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:22.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.915+0000 7efdfa4f4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efdf40ad850 0x7efdf40701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:48490/0 (socket says 172.21.15.105:48490) 2023-11-28T18:27:22.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.915+0000 7efdf94f2640 1 -- 172.21.15.105:0/3093594967 learned_addr learned my addr 172.21.15.105:0/3093594967 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:22.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.915+0000 7efdfa4f4640 1 -- 172.21.15.105:0/3093594967 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efdf4159f40 msgr2=0x7efdf40caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:22.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.915+0000 7efdfa4f4640 1 --2- 172.21.15.105:0/3093594967 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efdf4159f40 0x7efdf40caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:22.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.915+0000 7efdfa4f4640 1 -- 172.21.15.105:0/3093594967 >> v1:172.21.15.148:6789/0 conn(0x7efdf40cc760 legacy=0x7efdf40ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:22.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.915+0000 7efdfa4f4640 1 -- 172.21.15.105:0/3093594967 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf4071380 con 0x7efdf40ad850 2023-11-28T18:27:22.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.919+0000 7efdfa4f4640 1 --2- 172.21.15.105:0/3093594967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdf40ad850 0x7efdf40701d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efdec009d90 tx=0x7efdec01c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7815d498e21b82a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:22.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.919+0000 7efdf8cf1640 1 -- 172.21.15.105:0/3093594967 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec0a5950 con 0x7efdf40ad850 2023-11-28T18:27:22.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.919+0000 7efdf8cf1640 1 -- 172.21.15.105:0/3093594967 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdec0a5ab0 con 0x7efdf40ad850 2023-11-28T18:27:22.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.919+0000 7efdf8cf1640 1 -- 172.21.15.105:0/3093594967 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec0a5e40 con 0x7efdf40ad850 2023-11-28T18:27:22.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.919+0000 7efdfbf57640 1 -- 172.21.15.105:0/3093594967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdf40ad850 msgr2=0x7efdf40701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:22.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.919+0000 7efdfbf57640 1 --2- 172.21.15.105:0/3093594967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdf40ad850 0x7efdf40701d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efdec009d90 tx=0x7efdec01c890 comp rx=0 tx=0).stop 2023-11-28T18:27:22.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.919+0000 7efdfbf57640 1 -- 172.21.15.105:0/3093594967 shutdown_connections 2023-11-28T18:27:22.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.919+0000 7efdfbf57640 1 --2- 172.21.15.105:0/3093594967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdf40ad850 0x7efdf40701d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:22.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.919+0000 7efdfbf57640 1 --2- 172.21.15.105:0/3093594967 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efdf4159f40 0x7efdf40caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:22.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.919+0000 7efdfbf57640 1 -- 172.21.15.105:0/3093594967 >> 172.21.15.105:0/3093594967 conn(0x7efdf40ceb90 msgr2=0x7efdf40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:22.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.919+0000 7efdfbf57640 1 -- 172.21.15.105:0/3093594967 shutdown_connections 2023-11-28T18:27:22.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.919+0000 7efdfbf57640 1 -- 172.21.15.105:0/3093594967 wait complete. 2023-11-28T18:27:22.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.919+0000 7efdfbf57640 1 Processor -- start 2023-11-28T18:27:22.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.919+0000 7efdfbf57640 1 -- start start 2023-11-28T18:27:22.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdfbf57640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efdf40ad850 0x7efdf414d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:22.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdfbf57640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdf40cc760 0x7efdf4158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:22.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdfbf57640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdf4159f40 0x7efdf40af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:22.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdfbf57640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf4158630 con 0x7efdf4159f40 2023-11-28T18:27:22.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdfbf57640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf414dbc0 con 0x7efdf40cc760 2023-11-28T18:27:22.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdfbf57640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf4157ab0 con 0x7efdf40ad850 2023-11-28T18:27:22.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdfa4f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdf4159f40 0x7efdf40af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:22.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdfa4f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdf4159f40 0x7efdf40af1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37698/0 (socket says 172.21.15.105:37698) 2023-11-28T18:27:22.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdfa4f4640 1 -- 172.21.15.105:0/3251617133 learned_addr learned my addr 172.21.15.105:0/3251617133 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:22.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdf9cf3640 1 --2- 172.21.15.105:0/3251617133 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efdf40ad850 0x7efdf414d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:22.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdf94f2640 1 --2- 172.21.15.105:0/3251617133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdf40cc760 0x7efdf4158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:22.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdfa4f4640 1 -- 172.21.15.105:0/3251617133 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efdf40ad850 msgr2=0x7efdf414d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:22.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdfa4f4640 1 --2- 172.21.15.105:0/3251617133 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efdf40ad850 0x7efdf414d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:22.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdfa4f4640 1 -- 172.21.15.105:0/3251617133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdf40cc760 msgr2=0x7efdf4158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:22.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdfa4f4640 1 --2- 172.21.15.105:0/3251617133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdf40cc760 0x7efdf4158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:22.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdfa4f4640 1 -- 172.21.15.105:0/3251617133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec009a70 con 0x7efdf4159f40 2023-11-28T18:27:22.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdfa4f4640 1 --2- 172.21.15.105:0/3251617133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdf4159f40 0x7efdf40af1d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7efdec009d90 tx=0x7efdec019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:22.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdea7fc640 1 -- 172.21.15.105:0/3251617133 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec0add90 con 0x7efdf4159f40 2023-11-28T18:27:22.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdea7fc640 1 -- 172.21.15.105:0/3251617133 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdec009450 con 0x7efdf4159f40 2023-11-28T18:27:22.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdfbf57640 1 -- 172.21.15.105:0/3251617133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf4157d90 con 0x7efdf4159f40 2023-11-28T18:27:22.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdea7fc640 1 -- 172.21.15.105:0/3251617133 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec0b9690 con 0x7efdf4159f40 2023-11-28T18:27:22.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdfbf57640 1 -- 172.21.15.105:0/3251617133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf4158350 con 0x7efdf4159f40 2023-11-28T18:27:22.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.923+0000 7efdfbf57640 1 -- 172.21.15.105:0/3251617133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf4067570 con 0x7efdf4159f40 2023-11-28T18:27:22.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.927+0000 7efdea7fc640 1 -- 172.21.15.105:0/3251617133 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7efdec0b97f0 con 0x7efdf4159f40 2023-11-28T18:27:22.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.927+0000 7efdea7fc640 1 --2- 172.21.15.105:0/3251617133 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efdd80668d0 0x7efdd8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:22.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.927+0000 7efdea7fc640 1 -- 172.21.15.105:0/3251617133 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7efdec11fc40 con 0x7efdf4159f40 2023-11-28T18:27:22.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.927+0000 7efdf9cf3640 1 --2- 172.21.15.105:0/3251617133 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efdd80668d0 0x7efdd8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:22.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.927+0000 7efdf9cf3640 1 --2- 172.21.15.105:0/3251617133 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efdd80668d0 0x7efdd8068d90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efdf4070410 tx=0x7efdf0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:22.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:22.931+0000 7efdea7fc640 1 -- 172.21.15.105:0/3251617133 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efdec0ece60 con 0x7efdf4159f40 2023-11-28T18:27:23.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:23.111+0000 7efdfbf57640 1 -- 172.21.15.105:0/3251617133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7efdf40cb280 con 0x7efdf4159f40 2023-11-28T18:27:24.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.091+0000 7efdea7fc640 1 -- 172.21.15.105:0/3251617133 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v57) v1 ==== 105+0+0 (secure 0 0 0) 0x7efdec0f0480 con 0x7efdf4159f40 2023-11-28T18:27:24.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.091+0000 7efdfbf57640 1 -- 172.21.15.105:0/3251617133 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efdd80668d0 msgr2=0x7efdd8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:24.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.091+0000 7efdfbf57640 1 --2- 172.21.15.105:0/3251617133 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efdd80668d0 0x7efdd8068d90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efdf4070410 tx=0x7efdf0009190 comp rx=0 tx=0).stop 2023-11-28T18:27:24.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.091+0000 7efdfbf57640 1 -- 172.21.15.105:0/3251617133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdf4159f40 msgr2=0x7efdf40af1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:24.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.091+0000 7efdfbf57640 1 --2- 172.21.15.105:0/3251617133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdf4159f40 0x7efdf40af1d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7efdec009d90 tx=0x7efdec019630 comp rx=0 tx=0).stop 2023-11-28T18:27:24.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.091+0000 7efdfbf57640 1 -- 172.21.15.105:0/3251617133 shutdown_connections 2023-11-28T18:27:24.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.091+0000 7efdfbf57640 1 --2- 172.21.15.105:0/3251617133 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efdd80668d0 0x7efdd8068d90 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:24.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.091+0000 7efdfbf57640 1 --2- 172.21.15.105:0/3251617133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdf4159f40 0x7efdf40af1d0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:24.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.091+0000 7efdfbf57640 1 --2- 172.21.15.105:0/3251617133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdf40cc760 0x7efdf4158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:24.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.091+0000 7efdfbf57640 1 --2- 172.21.15.105:0/3251617133 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efdf40ad850 0x7efdf414d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:24.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.091+0000 7efdfbf57640 1 -- 172.21.15.105:0/3251617133 >> 172.21.15.105:0/3251617133 conn(0x7efdf40ceb90 msgr2=0x7efdf4072eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:24.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.091+0000 7efdfbf57640 1 -- 172.21.15.105:0/3251617133 shutdown_connections 2023-11-28T18:27:24.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.091+0000 7efdfbf57640 1 -- 172.21.15.105:0/3251617133 wait complete. 2023-11-28T18:27:24.125 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:27:24.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.283+0000 7f1858b56640 1 Processor -- start 2023-11-28T18:27:24.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.283+0000 7f1858b56640 1 -- start start 2023-11-28T18:27:24.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.283+0000 7f1858b56640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f185415d2d0 0x7f18540cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:24.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.283+0000 7f1858b56640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18540cf960 0x7f1854166fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:24.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.283+0000 7f1858b56640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18540ce840 con 0x7f185415cef0 2023-11-28T18:27:24.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.283+0000 7f1858b56640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18540cfdf0 con 0x7f185415d2d0 2023-11-28T18:27:24.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.283+0000 7f1858b56640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18540b4780 con 0x7f18540cf960 2023-11-28T18:27:24.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.283+0000 7f184bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f185415d2d0 0x7f18540cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:24.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.283+0000 7f18537fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f185415cef0 0x7f18540d1600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33560/0 (socket says 172.21.15.105:33560) 2023-11-28T18:27:24.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.283+0000 7f18537fe640 1 -- 172.21.15.105:0/2678642887 learned_addr learned my addr 172.21.15.105:0/2678642887 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:24.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.283+0000 7f1853fff640 1 --2- 172.21.15.105:0/2678642887 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18540cf960 0x7f1854166fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:24.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.283+0000 7f184bfff640 1 -- 172.21.15.105:0/2678642887 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18540cf960 msgr2=0x7f1854166fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:24.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.283+0000 7f184bfff640 1 --2- 172.21.15.105:0/2678642887 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18540cf960 0x7f1854166fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:24.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.283+0000 7f184bfff640 1 -- 172.21.15.105:0/2678642887 >> v1:172.21.15.105:6789/0 conn(0x7f185415cef0 legacy=0x7f18540d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:24.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.283+0000 7f184bfff640 1 -- 172.21.15.105:0/2678642887 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18540cd990 con 0x7f185415d2d0 2023-11-28T18:27:24.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.287+0000 7f184bfff640 1 --2- 172.21.15.105:0/2678642887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f185415d2d0 0x7f18540cf420 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f183c0097f0 tx=0x7f183c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774ccc6ec8d1a7b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:24.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.287+0000 7f1852ffd640 1 -- 172.21.15.105:0/2678642887 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c0ac630 con 0x7f185415d2d0 2023-11-28T18:27:24.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.287+0000 7f1852ffd640 1 -- 172.21.15.105:0/2678642887 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183c0ac790 con 0x7f185415d2d0 2023-11-28T18:27:24.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.287+0000 7f1852ffd640 1 -- 172.21.15.105:0/2678642887 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c0acb20 con 0x7f185415d2d0 2023-11-28T18:27:24.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.287+0000 7f1858b56640 1 -- 172.21.15.105:0/2678642887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f185415d2d0 msgr2=0x7f18540cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:24.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.287+0000 7f1858b56640 1 --2- 172.21.15.105:0/2678642887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f185415d2d0 0x7f18540cf420 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f183c0097f0 tx=0x7f183c01c3d0 comp rx=0 tx=0).stop 2023-11-28T18:27:24.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.287+0000 7f1858b56640 1 -- 172.21.15.105:0/2678642887 shutdown_connections 2023-11-28T18:27:24.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.287+0000 7f1858b56640 1 --2- 172.21.15.105:0/2678642887 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18540cf960 0x7f1854166fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:24.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.287+0000 7f1858b56640 1 --2- 172.21.15.105:0/2678642887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f185415d2d0 0x7f18540cf420 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:24.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.287+0000 7f1858b56640 1 -- 172.21.15.105:0/2678642887 >> 172.21.15.105:0/2678642887 conn(0x7f18540d2e10 msgr2=0x7f18540d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:24.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.287+0000 7f1858b56640 1 -- 172.21.15.105:0/2678642887 shutdown_connections 2023-11-28T18:27:24.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.287+0000 7f1858b56640 1 -- 172.21.15.105:0/2678642887 wait complete. 2023-11-28T18:27:24.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f1858b56640 1 Processor -- start 2023-11-28T18:27:24.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f1858b56640 1 -- start start 2023-11-28T18:27:24.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f1858b56640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18540cf960 0x7f1854154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:24.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f1858b56640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f185415cef0 0x7f1854157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:24.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f1858b56640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f185415d2d0 0x7f1854157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:24.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f1858b56640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1854155bb0 con 0x7f18540cf960 2023-11-28T18:27:24.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f1858b56640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1854155e90 con 0x7f185415d2d0 2023-11-28T18:27:24.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f1858b56640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1854156170 con 0x7f185415cef0 2023-11-28T18:27:24.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f18537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18540cf960 0x7f1854154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:24.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f18537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18540cf960 0x7f1854154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37716/0 (socket says 172.21.15.105:37716) 2023-11-28T18:27:24.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f18537fe640 1 -- 172.21.15.105:0/2503661715 learned_addr learned my addr 172.21.15.105:0/2503661715 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:24.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f1853fff640 1 --2- 172.21.15.105:0/2503661715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f185415d2d0 0x7f1854157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:24.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f18537fe640 1 -- 172.21.15.105:0/2503661715 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f185415cef0 msgr2=0x7f1854157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:24.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f184bfff640 1 --2- 172.21.15.105:0/2503661715 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f185415cef0 0x7f1854157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:24.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f18537fe640 1 --2- 172.21.15.105:0/2503661715 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f185415cef0 0x7f1854157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:24.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f18537fe640 1 -- 172.21.15.105:0/2503661715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f185415d2d0 msgr2=0x7f1854157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:24.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f18537fe640 1 --2- 172.21.15.105:0/2503661715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f185415d2d0 0x7f1854157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:24.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f18537fe640 1 -- 172.21.15.105:0/2503661715 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183c009420 con 0x7f18540cf960 2023-11-28T18:27:24.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f1853fff640 1 --2- 172.21.15.105:0/2503661715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f185415d2d0 0x7f1854157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:27:24.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f184bfff640 1 --2- 172.21.15.105:0/2503661715 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f185415cef0 0x7f1854157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:27:24.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f18537fe640 1 --2- 172.21.15.105:0/2503661715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18540cf960 0x7f1854154770 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f184400c500 tx=0x7f184400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:24.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f1850ff9640 1 -- 172.21.15.105:0/2503661715 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1844005cf0 con 0x7f18540cf960 2023-11-28T18:27:24.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f1850ff9640 1 -- 172.21.15.105:0/2503661715 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1844005e50 con 0x7f18540cf960 2023-11-28T18:27:24.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f1850ff9640 1 -- 172.21.15.105:0/2503661715 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18440072f0 con 0x7f18540cf960 2023-11-28T18:27:24.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f1858b56640 1 -- 172.21.15.105:0/2503661715 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1854156450 con 0x7f18540cf960 2023-11-28T18:27:24.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.291+0000 7f1858b56640 1 -- 172.21.15.105:0/2503661715 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1854156c50 con 0x7f18540cf960 2023-11-28T18:27:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.295+0000 7f1858b56640 1 -- 172.21.15.105:0/2503661715 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1854156970 con 0x7f18540cf960 2023-11-28T18:27:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.295+0000 7f1850ff9640 1 -- 172.21.15.105:0/2503661715 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f1844007450 con 0x7f18540cf960 2023-11-28T18:27:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.295+0000 7f1850ff9640 1 --2- 172.21.15.105:0/2503661715 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f18340668d0 0x7f1834068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.295+0000 7f1850ff9640 1 -- 172.21.15.105:0/2503661715 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f1844083ac0 con 0x7f18540cf960 2023-11-28T18:27:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.295+0000 7f184bfff640 1 --2- 172.21.15.105:0/2503661715 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f18340668d0 0x7f1834068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.295+0000 7f184bfff640 1 --2- 172.21.15.105:0/2503661715 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f18340668d0 0x7f1834068d90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f183c0ae000 tx=0x7f183c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:24.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.299+0000 7f1850ff9640 1 -- 172.21.15.105:0/2503661715 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1844050bd0 con 0x7f18540cf960 2023-11-28T18:27:24.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.479+0000 7f1858b56640 1 -- 172.21.15.105:0/2503661715 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18540cf4b0 con 0x7f18540cf960 2023-11-28T18:27:24.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.479+0000 7f1850ff9640 1 -- 172.21.15.105:0/2503661715 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4936 (secure 0 0 0) 0x7f1844017020 con 0x7f18540cf960 2023-11-28T18:27:24.484 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:27:24.484 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":58,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/234000461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":234000461},{"type":"v1","addr":"172.21.15.148:6834","nonce":234000461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6436,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2396073410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2396073410},{"type":"v1","addr":"172.21.15.148:6835","nonce":2396073410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:27:05.479811+0000","modified":"2023-11-28T18:27:24.097194+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1],"up":{"mds_0":6720,"mds_1":6717},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6717":{"gid":6717,"name":"a","rank":1,"incarnation":58,"state":"up:creating","state_seq":1,"addr":"172.21.15.105:6835/380088531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":380088531},{"type":"v1","addr":"172.21.15.105:6835","nonce":380088531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6720":{"gid":6720,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/3442512486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3442512486},{"type":"v1","addr":"172.21.15.105:6837","nonce":3442512486}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T18:27:24.485 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 58 2023-11-28T18:27:24.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.483+0000 7f1858b56640 1 -- 172.21.15.105:0/2503661715 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f18340668d0 msgr2=0x7f1834068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:24.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.483+0000 7f1858b56640 1 --2- 172.21.15.105:0/2503661715 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f18340668d0 0x7f1834068d90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f183c0ae000 tx=0x7f183c0a7000 comp rx=0 tx=0).stop 2023-11-28T18:27:24.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.483+0000 7f1858b56640 1 -- 172.21.15.105:0/2503661715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18540cf960 msgr2=0x7f1854154770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:24.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.483+0000 7f1858b56640 1 --2- 172.21.15.105:0/2503661715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18540cf960 0x7f1854154770 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f184400c500 tx=0x7f184400c9c0 comp rx=0 tx=0).stop 2023-11-28T18:27:24.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.483+0000 7f1858b56640 1 -- 172.21.15.105:0/2503661715 shutdown_connections 2023-11-28T18:27:24.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.483+0000 7f1858b56640 1 --2- 172.21.15.105:0/2503661715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f185415d2d0 0x7f1854157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:24.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.483+0000 7f1858b56640 1 --2- 172.21.15.105:0/2503661715 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f185415cef0 0x7f1854157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:24.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.483+0000 7f1858b56640 1 --2- 172.21.15.105:0/2503661715 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f18340668d0 0x7f1834068d90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:24.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.483+0000 7f1858b56640 1 --2- 172.21.15.105:0/2503661715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18540cf960 0x7f1854154770 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:24.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.483+0000 7f1858b56640 1 -- 172.21.15.105:0/2503661715 >> 172.21.15.105:0/2503661715 conn(0x7f18540d2e10 msgr2=0x7f185406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:24.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.483+0000 7f1858b56640 1 -- 172.21.15.105:0/2503661715 shutdown_connections 2023-11-28T18:27:24.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:24.483+0000 7f1858b56640 1 -- 172.21.15.105:0/2503661715 wait complete. 2023-11-28T18:27:24.907 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:27:05.479811+0000', 'modified': '2023-11-28T18:27:24.097194+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 6720, 'mds_1': 6717}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6717': {'gid': 6717, 'name': 'a', 'rank': 1, 'incarnation': 58, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.105:6835/380088531', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 380088531}, {'type': 'v1', 'addr': '172.21.15.105:6835', 'nonce': 380088531}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_6720': {'gid': 6720, 'name': 'c', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6837/3442512486', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 3442512486}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 3442512486}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:27:24.908 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6717:up:creating 2023-11-28T18:27:25.909 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:27:26.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.055+0000 7fb86a1aa640 1 Processor -- start 2023-11-28T18:27:26.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.055+0000 7fb86a1aa640 1 -- start start 2023-11-28T18:27:26.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.055+0000 7fb86a1aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb86415d4f0 0x7fb8640d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.055+0000 7fb86a1aa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb8640b2d20 0x7fb8640b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.055+0000 7fb86a1aa640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8640b25b0 con 0x7fb8640d3c30 2023-11-28T18:27:26.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.055+0000 7fb86a1aa640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8640b2900 con 0x7fb8640b2d20 2023-11-28T18:27:26.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.055+0000 7fb86a1aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8640d0a20 con 0x7fb86415d4f0 2023-11-28T18:27:26.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.055+0000 7fb863fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb8640b2d20 0x7fb8640b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.055+0000 7fb85bfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb8640d3c30 0x7fb8640d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33572/0 (socket says 172.21.15.105:33572) 2023-11-28T18:27:26.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.055+0000 7fb85bfff640 1 -- 172.21.15.105:0/2966759060 learned_addr learned my addr 172.21.15.105:0/2966759060 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:26.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.055+0000 7fb8637fe640 1 --2- 172.21.15.105:0/2966759060 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb86415d4f0 0x7fb8640d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.055+0000 7fb863fff640 1 -- 172.21.15.105:0/2966759060 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb86415d4f0 msgr2=0x7fb8640d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:26.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.055+0000 7fb863fff640 1 --2- 172.21.15.105:0/2966759060 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb86415d4f0 0x7fb8640d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.055+0000 7fb863fff640 1 -- 172.21.15.105:0/2966759060 >> v1:172.21.15.105:6789/0 conn(0x7fb8640d3c30 legacy=0x7fb8640d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:26.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.055+0000 7fb863fff640 1 -- 172.21.15.105:0/2966759060 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8640cea30 con 0x7fb8640b2d20 2023-11-28T18:27:26.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.055+0000 7fb8637fe640 1 --2- 172.21.15.105:0/2966759060 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb86415d4f0 0x7fb8640d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:26.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.055+0000 7fb862ffd640 1 -- 172.21.15.105:0/2966759060 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460337823 0 0) 0x7fb8640b25b0 con 0x7fb8640d3c30 2023-11-28T18:27:26.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.059+0000 7fb863fff640 1 --2- 172.21.15.105:0/2966759060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8640b2d20 0x7fb8640b31d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009d90 tx=0x7fb85c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4754130e240efbcf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:26.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.059+0000 7fb862ffd640 1 -- 172.21.15.105:0/2966759060 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c0a5950 con 0x7fb8640b2d20 2023-11-28T18:27:26.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.059+0000 7fb862ffd640 1 -- 172.21.15.105:0/2966759060 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85c0a5ab0 con 0x7fb8640b2d20 2023-11-28T18:27:26.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.059+0000 7fb862ffd640 1 -- 172.21.15.105:0/2966759060 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c0a5e40 con 0x7fb8640b2d20 2023-11-28T18:27:26.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.059+0000 7fb86a1aa640 1 -- 172.21.15.105:0/2966759060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8640b2d20 msgr2=0x7fb8640b31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:26.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.059+0000 7fb86a1aa640 1 --2- 172.21.15.105:0/2966759060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8640b2d20 0x7fb8640b31d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009d90 tx=0x7fb85c01c890 comp rx=0 tx=0).stop 2023-11-28T18:27:26.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.059+0000 7fb86a1aa640 1 -- 172.21.15.105:0/2966759060 shutdown_connections 2023-11-28T18:27:26.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.059+0000 7fb86a1aa640 1 --2- 172.21.15.105:0/2966759060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8640b2d20 0x7fb8640b31d0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.059+0000 7fb86a1aa640 1 --2- 172.21.15.105:0/2966759060 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb86415d4f0 0x7fb8640d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.059+0000 7fb86a1aa640 1 -- 172.21.15.105:0/2966759060 >> 172.21.15.105:0/2966759060 conn(0x7fb8640d3030 msgr2=0x7fb8640d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:26.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.059+0000 7fb86a1aa640 1 -- 172.21.15.105:0/2966759060 shutdown_connections 2023-11-28T18:27:26.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.059+0000 7fb86a1aa640 1 -- 172.21.15.105:0/2966759060 wait complete. 2023-11-28T18:27:26.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb86a1aa640 1 Processor -- start 2023-11-28T18:27:26.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb86a1aa640 1 -- start start 2023-11-28T18:27:26.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb86a1aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8640b2d20 0x7fb86415bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb86a1aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8640d3c30 0x7fb8640cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb86a1aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb86415d4f0 0x7fb86415dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb86a1aa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb864157080 con 0x7fb8640d3c30 2023-11-28T18:27:26.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb86a1aa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb864157920 con 0x7fb8640b2d20 2023-11-28T18:27:26.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb86a1aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb864159250 con 0x7fb86415d4f0 2023-11-28T18:27:26.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb85bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8640d3c30 0x7fb8640cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb85bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8640d3c30 0x7fb8640cc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37726/0 (socket says 172.21.15.105:37726) 2023-11-28T18:27:26.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb85bfff640 1 -- 172.21.15.105:0/1825963891 learned_addr learned my addr 172.21.15.105:0/1825963891 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:26.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb863fff640 1 --2- 172.21.15.105:0/1825963891 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb86415d4f0 0x7fb86415dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb8637fe640 1 --2- 172.21.15.105:0/1825963891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8640b2d20 0x7fb86415bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb85bfff640 1 -- 172.21.15.105:0/1825963891 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb86415d4f0 msgr2=0x7fb86415dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:26.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb85bfff640 1 --2- 172.21.15.105:0/1825963891 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb86415d4f0 0x7fb86415dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb85bfff640 1 -- 172.21.15.105:0/1825963891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8640b2d20 msgr2=0x7fb86415bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:26.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb85bfff640 1 --2- 172.21.15.105:0/1825963891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8640b2d20 0x7fb86415bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb85bfff640 1 -- 172.21.15.105:0/1825963891 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85c009a70 con 0x7fb8640d3c30 2023-11-28T18:27:26.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb8637fe640 1 --2- 172.21.15.105:0/1825963891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8640b2d20 0x7fb86415bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:26.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb85bfff640 1 --2- 172.21.15.105:0/1825963891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8640d3c30 0x7fb8640cc6f0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00c4e0 tx=0x7fb84c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:26.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb860ff9640 1 -- 172.21.15.105:0/1825963891 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c0074e0 con 0x7fb8640d3c30 2023-11-28T18:27:26.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb860ff9640 1 -- 172.21.15.105:0/1825963891 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84c007660 con 0x7fb8640d3c30 2023-11-28T18:27:26.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb860ff9640 1 -- 172.21.15.105:0/1825963891 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c004a30 con 0x7fb8640d3c30 2023-11-28T18:27:26.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb86a1aa640 1 -- 172.21.15.105:0/1825963891 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8640ccdd0 con 0x7fb8640d3c30 2023-11-28T18:27:26.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.063+0000 7fb86a1aa640 1 -- 172.21.15.105:0/1825963891 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8641575b0 con 0x7fb8640d3c30 2023-11-28T18:27:26.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.067+0000 7fb86a1aa640 1 -- 172.21.15.105:0/1825963891 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb864067570 con 0x7fb8640d3c30 2023-11-28T18:27:26.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.067+0000 7fb860ff9640 1 -- 172.21.15.105:0/1825963891 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fb84c004b90 con 0x7fb8640d3c30 2023-11-28T18:27:26.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.067+0000 7fb860ff9640 1 --2- 172.21.15.105:0/1825963891 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb8440668d0 0x7fb844068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.067+0000 7fb860ff9640 1 -- 172.21.15.105:0/1825963891 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fb84c083750 con 0x7fb8640d3c30 2023-11-28T18:27:26.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.067+0000 7fb8637fe640 1 --2- 172.21.15.105:0/1825963891 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb8440668d0 0x7fb844068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.067+0000 7fb8637fe640 1 --2- 172.21.15.105:0/1825963891 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb8440668d0 0x7fb844068d90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb854009970 tx=0x7fb854008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:26.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.071+0000 7fb860ff9640 1 -- 172.21.15.105:0/1825963891 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb84c0542f0 con 0x7fb8640d3c30 2023-11-28T18:27:26.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.255+0000 7fb86a1aa640 1 -- 172.21.15.105:0/1825963891 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8640b11d0 con 0x7fb8640d3c30 2023-11-28T18:27:26.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.255+0000 7fb860ff9640 1 -- 172.21.15.105:0/1825963891 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fb84c018090 con 0x7fb8640d3c30 2023-11-28T18:27:26.260 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:27:26.260 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":61,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/234000461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":234000461},{"type":"v1","addr":"172.21.15.148:6834","nonce":234000461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:27:05.479811+0000","modified":"2023-11-28T18:27:25.123907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6720,"mds_1":6717,"mds_2":6436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6436":{"gid":6436,"name":"b","rank":2,"incarnation":60,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6835/2396073410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2396073410},{"type":"v1","addr":"172.21.15.148:6835","nonce":2396073410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6717":{"gid":6717,"name":"a","rank":1,"incarnation":58,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6835/380088531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":380088531},{"type":"v1","addr":"172.21.15.105:6835","nonce":380088531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6720":{"gid":6720,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/3442512486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3442512486},{"type":"v1","addr":"172.21.15.105:6837","nonce":3442512486}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T18:27:26.260 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 61 2023-11-28T18:27:26.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.259+0000 7fb86a1aa640 1 -- 172.21.15.105:0/1825963891 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb8440668d0 msgr2=0x7fb844068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:26.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.259+0000 7fb86a1aa640 1 --2- 172.21.15.105:0/1825963891 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb8440668d0 0x7fb844068d90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb854009970 tx=0x7fb854008040 comp rx=0 tx=0).stop 2023-11-28T18:27:26.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.259+0000 7fb86a1aa640 1 -- 172.21.15.105:0/1825963891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8640d3c30 msgr2=0x7fb8640cc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:26.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.259+0000 7fb86a1aa640 1 --2- 172.21.15.105:0/1825963891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8640d3c30 0x7fb8640cc6f0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00c4e0 tx=0x7fb84c00c9a0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.259+0000 7fb86a1aa640 1 -- 172.21.15.105:0/1825963891 shutdown_connections 2023-11-28T18:27:26.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.259+0000 7fb86a1aa640 1 --2- 172.21.15.105:0/1825963891 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb86415d4f0 0x7fb86415dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.259+0000 7fb86a1aa640 1 --2- 172.21.15.105:0/1825963891 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb8440668d0 0x7fb844068d90 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.259+0000 7fb86a1aa640 1 --2- 172.21.15.105:0/1825963891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8640d3c30 0x7fb8640cc6f0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.259+0000 7fb86a1aa640 1 --2- 172.21.15.105:0/1825963891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8640b2d20 0x7fb86415bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.259+0000 7fb86a1aa640 1 -- 172.21.15.105:0/1825963891 >> 172.21.15.105:0/1825963891 conn(0x7fb8640d3030 msgr2=0x7fb864155c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:26.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.259+0000 7fb86a1aa640 1 -- 172.21.15.105:0/1825963891 shutdown_connections 2023-11-28T18:27:26.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.259+0000 7fb86a1aa640 1 -- 172.21.15.105:0/1825963891 wait complete. 2023-11-28T18:27:26.684 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:27:05.479811+0000', 'modified': '2023-11-28T18:27:25.123907+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 6720, 'mds_1': 6717, 'mds_2': 6436}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6436': {'gid': 6436, 'name': 'b', 'rank': 2, 'incarnation': 60, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.148:6835/2396073410', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6833', 'nonce': 2396073410}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 2396073410}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_6717': {'gid': 6717, 'name': 'a', 'rank': 1, 'incarnation': 58, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.105:6835/380088531', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 380088531}, {'type': 'v1', 'addr': '172.21.15.105:6835', 'nonce': 380088531}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_6720': {'gid': 6720, 'name': 'c', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6837/3442512486', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 3442512486}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 3442512486}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:27:26.684 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-11-28T18:27:26.684 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-28T18:27:26.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.819+0000 7f83155d1640 1 Processor -- start 2023-11-28T18:27:26.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.819+0000 7f83155d1640 1 -- start start 2023-11-28T18:27:26.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f83155d1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8310159610 0x7f83100ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f83155d1640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83100cde30 con 0x7f831006cef0 2023-11-28T18:27:26.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f83155d1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83101599f0 con 0x7f83100cc760 2023-11-28T18:27:26.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f83155d1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f831006de20 con 0x7f8310159610 2023-11-28T18:27:26.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f830f7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f831006cef0 0x7f83100abd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33576/0 (socket says 172.21.15.105:33576) 2023-11-28T18:27:26.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f830f7fe640 1 -- 172.21.15.105:0/2705322075 learned_addr learned my addr 172.21.15.105:0/2705322075 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:26.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f830effd640 1 --2- 172.21.15.105:0/2705322075 >> v2:172.21.15.148:3300/0 conn(0x7f8310159610 0x7f83100ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f830e7fc640 1 -- 172.21.15.105:0/2705322075 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4123763948 0 0) 0x7f83100cde30 con 0x7f831006cef0 2023-11-28T18:27:26.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f830e7fc640 1 -- 172.21.15.105:0/2705322075 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82fc003530 con 0x7f831006cef0 2023-11-28T18:27:26.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f830e7fc640 1 -- 172.21.15.105:0/2705322075 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3447872546 0 0) 0x7f82fc003530 con 0x7f831006cef0 2023-11-28T18:27:26.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f830e7fc640 1 -- 172.21.15.105:0/2705322075 >> v2:172.21.15.148:3300/0 conn(0x7f8310159610 msgr2=0x7f83100ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:26.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f830e7fc640 1 --2- 172.21.15.105:0/2705322075 >> v2:172.21.15.148:3300/0 conn(0x7f8310159610 0x7f83100ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f830e7fc640 1 -- 172.21.15.105:0/2705322075 >> v1:172.21.15.148:6789/0 conn(0x7f83100cc760 legacy=0x7f831006c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:26.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f830e7fc640 1 -- 172.21.15.105:0/2705322075 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83100cd220 con 0x7f831006cef0 2023-11-28T18:27:26.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f830e7fc640 1 -- 172.21.15.105:0/2705322075 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f8308001a00 con 0x7f831006cef0 2023-11-28T18:27:26.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f830e7fc640 1 -- 172.21.15.105:0/2705322075 >> v1:172.21.15.105:6789/0 conn(0x7f831006cef0 legacy=0x7f83100abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:26.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f830e7fc640 1 --2- 172.21.15.105:0/2705322075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82fc003980 0x7f82fc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f830e7fc640 1 --2- 172.21.15.105:0/2705322075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82fc0047d0 0x7f82fc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f830e7fc640 1 --2- 172.21.15.105:0/2705322075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82fc005190 0x7f82fc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f830e7fc640 1 -- 172.21.15.105:0/2705322075 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831006de20 con 0x7f82fc0047d0 2023-11-28T18:27:26.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f830e7fc640 1 -- 172.21.15.105:0/2705322075 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc00c030 con 0x7f82fc005190 2023-11-28T18:27:26.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f830e7fc640 1 -- 172.21.15.105:0/2705322075 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc00e400 con 0x7f82fc003980 2023-11-28T18:27:26.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f83067fc640 1 --2- 172.21.15.105:0/2705322075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82fc0047d0 0x7f82fc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f830effd640 1 --2- 172.21.15.105:0/2705322075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82fc003980 0x7f82fc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f830f7fe640 1 --2- 172.21.15.105:0/2705322075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82fc005190 0x7f82fc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f83067fc640 1 -- 172.21.15.105:0/2705322075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82fc003980 msgr2=0x7f82fc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:26.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f83067fc640 1 --2- 172.21.15.105:0/2705322075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82fc003980 0x7f82fc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f83067fc640 1 -- 172.21.15.105:0/2705322075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82fc005190 msgr2=0x7f82fc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:26.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f83067fc640 1 --2- 172.21.15.105:0/2705322075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82fc005190 0x7f82fc00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f83067fc640 1 -- 172.21.15.105:0/2705322075 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc003530 con 0x7f82fc0047d0 2023-11-28T18:27:26.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f830f7fe640 1 --2- 172.21.15.105:0/2705322075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82fc005190 0x7f82fc00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:26.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.823+0000 7f830f7fe640 1 -- 172.21.15.105:0/2705322075 reap_dead start 2023-11-28T18:27:26.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.827+0000 7f83067fc640 1 --2- 172.21.15.105:0/2705322075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82fc0047d0 0x7f82fc004c50 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f8300001780 tx=0x7f830001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530682a5d4be11bf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:26.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.827+0000 7f830e7fc640 1 -- 172.21.15.105:0/2705322075 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83000052d0 con 0x7f82fc0047d0 2023-11-28T18:27:26.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.827+0000 7f830e7fc640 1 -- 172.21.15.105:0/2705322075 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83000a7050 con 0x7f82fc0047d0 2023-11-28T18:27:26.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.827+0000 7f830e7fc640 1 -- 172.21.15.105:0/2705322075 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8300003600 con 0x7f82fc0047d0 2023-11-28T18:27:26.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.827+0000 7f83155d1640 1 -- 172.21.15.105:0/2705322075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82fc0047d0 msgr2=0x7f82fc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:26.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.827+0000 7f83155d1640 1 --2- 172.21.15.105:0/2705322075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82fc0047d0 0x7f82fc004c50 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f8300001780 tx=0x7f830001c3e0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.827+0000 7f83155d1640 1 -- 172.21.15.105:0/2705322075 shutdown_connections 2023-11-28T18:27:26.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.827+0000 7f83155d1640 1 --2- 172.21.15.105:0/2705322075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82fc0047d0 0x7f82fc004c50 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.827+0000 7f83155d1640 1 -- 172.21.15.105:0/2705322075 >> 172.21.15.105:0/2705322075 conn(0x7f83100ceb90 msgr2=0x7f83100cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:26.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.827+0000 7f83155d1640 1 -- 172.21.15.105:0/2705322075 shutdown_connections 2023-11-28T18:27:26.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.827+0000 7f83155d1640 1 -- 172.21.15.105:0/2705322075 wait complete. 2023-11-28T18:27:26.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f83155d1640 1 Processor -- start 2023-11-28T18:27:26.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f83155d1640 1 -- start start 2023-11-28T18:27:26.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f83155d1640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82fc0047d0 0x7f831014e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f83155d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f831015a720 0x7f83100b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f83155d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8310159b80 0x7f831015a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f83155d1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831015a550 con 0x7f831015a720 2023-11-28T18:27:26.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f83155d1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831014ecd0 con 0x7f8310159b80 2023-11-28T18:27:26.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f83155d1640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83100b1ef0 con 0x7f82fc0047d0 2023-11-28T18:27:26.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f83067fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f831015a720 0x7f83100b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f83067fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f831015a720 0x7f83100b19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37748/0 (socket says 172.21.15.105:37748) 2023-11-28T18:27:26.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f83067fc640 1 -- 172.21.15.105:0/3983356434 learned_addr learned my addr 172.21.15.105:0/3983356434 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:26.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f830effd640 1 --2- 172.21.15.105:0/3983356434 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82fc0047d0 0x7f831014e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f830f7fe640 1 --2- 172.21.15.105:0/3983356434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8310159b80 0x7f831015a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f83067fc640 1 -- 172.21.15.105:0/3983356434 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82fc0047d0 msgr2=0x7f831014e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:26.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f83067fc640 1 --2- 172.21.15.105:0/3983356434 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82fc0047d0 0x7f831014e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f83067fc640 1 -- 172.21.15.105:0/3983356434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8310159b80 msgr2=0x7f831015a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:26.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f83067fc640 1 --2- 172.21.15.105:0/3983356434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8310159b80 0x7f831015a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f83067fc640 1 -- 172.21.15.105:0/3983356434 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83100c8840 con 0x7f831015a720 2023-11-28T18:27:26.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f830f7fe640 1 --2- 172.21.15.105:0/3983356434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8310159b80 0x7f831015a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:26.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f83067fc640 1 --2- 172.21.15.105:0/3983356434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f831015a720 0x7f83100b19b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f830001c210 tx=0x7f82fc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:26.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f8307fff640 1 -- 172.21.15.105:0/3983356434 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8300003e40 con 0x7f831015a720 2023-11-28T18:27:26.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f8307fff640 1 -- 172.21.15.105:0/3983356434 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8300007a40 con 0x7f831015a720 2023-11-28T18:27:26.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f83155d1640 1 -- 172.21.15.105:0/3983356434 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83100c8b20 con 0x7f831015a720 2023-11-28T18:27:26.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f8307fff640 1 -- 172.21.15.105:0/3983356434 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8300007270 con 0x7f831015a720 2023-11-28T18:27:26.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.831+0000 7f83155d1640 1 -- 172.21.15.105:0/3983356434 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83100c90e0 con 0x7f831015a720 2023-11-28T18:27:26.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.835+0000 7f8307fff640 1 -- 172.21.15.105:0/3983356434 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f83000073d0 con 0x7f831015a720 2023-11-28T18:27:26.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.835+0000 7f8307fff640 1 --2- 172.21.15.105:0/3983356434 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82f00668d0 0x7f82f0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.835+0000 7f8307fff640 1 -- 172.21.15.105:0/3983356434 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f830011e910 con 0x7f831015a720 2023-11-28T18:27:26.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.835+0000 7f830effd640 1 --2- 172.21.15.105:0/3983356434 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82f00668d0 0x7f82f0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.835+0000 7f8318949000 1 Processor -- start 2023-11-28T18:27:26.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.835+0000 7f830effd640 1 --2- 172.21.15.105:0/3983356434 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82f00668d0 0x7f82f0068d90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f83100c7ff0 tx=0x7f82f8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:26.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.835+0000 7f8318949000 1 -- start start 2023-11-28T18:27:26.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.835+0000 7f8318949000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55858410ed30 0x55858410f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.835+0000 7f8318949000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55858410f650 0x5585840c1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.835+0000 7f8318949000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5585840c2500 0x5585840c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.835+0000 7f8318949000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x558583f8b110 con 0x55858410f650 2023-11-28T18:27:26.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.835+0000 7f8318949000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558583f1b9a0 con 0x55858410ed30 2023-11-28T18:27:26.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.835+0000 7f8318949000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x558583f18ed0 con 0x5585840c2500 2023-11-28T18:27:26.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.835+0000 7f83067fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55858410f650 0x5585840c1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.835+0000 7f83067fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55858410f650 0x5585840c1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37754/0 (socket says 172.21.15.105:37754) 2023-11-28T18:27:26.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.835+0000 7f83067fc640 1 -- 172.21.15.105:0/2642862809 learned_addr learned my addr 172.21.15.105:0/2642862809 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:26.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f830effd640 1 --2- 172.21.15.105:0/2642862809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55858410ed30 0x55858410f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f830f7fe640 1 --2- 172.21.15.105:0/2642862809 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5585840c2500 0x5585840c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f83067fc640 1 -- 172.21.15.105:0/2642862809 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5585840c2500 msgr2=0x5585840c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f83067fc640 1 --2- 172.21.15.105:0/2642862809 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5585840c2500 0x5585840c28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f83067fc640 1 -- 172.21.15.105:0/2642862809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55858410ed30 msgr2=0x55858410f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f83067fc640 1 --2- 172.21.15.105:0/2642862809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55858410ed30 0x55858410f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f83067fc640 1 -- 172.21.15.105:0/2642862809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558583c0c890 con 0x55858410f650 2023-11-28T18:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f830effd640 1 --2- 172.21.15.105:0/2642862809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55858410ed30 0x55858410f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f83067fc640 1 --2- 172.21.15.105:0/2642862809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55858410f650 0x5585840c1fc0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f83000ef5c0 tx=0x7f83000ef050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8f26c06e381f3a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f8304ff9640 1 -- 172.21.15.105:0/2642862809 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83000ef630 con 0x55858410f650 2023-11-28T18:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f8304ff9640 1 -- 172.21.15.105:0/2642862809 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83000f3780 con 0x55858410f650 2023-11-28T18:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f8304ff9640 1 -- 172.21.15.105:0/2642862809 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83000f7930 con 0x55858410f650 2023-11-28T18:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f8318949000 1 -- 172.21.15.105:0/2642862809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55858410f650 msgr2=0x5585840c1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f8318949000 1 --2- 172.21.15.105:0/2642862809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55858410f650 0x5585840c1fc0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f83000ef5c0 tx=0x7f83000ef050 comp rx=0 tx=0).stop 2023-11-28T18:27:26.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f8318949000 1 -- 172.21.15.105:0/2642862809 shutdown_connections 2023-11-28T18:27:26.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f8318949000 1 --2- 172.21.15.105:0/2642862809 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5585840c2500 0x5585840c28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f8318949000 1 --2- 172.21.15.105:0/2642862809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55858410f650 0x5585840c1fc0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f8318949000 1 --2- 172.21.15.105:0/2642862809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55858410ed30 0x55858410f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f8318949000 1 -- 172.21.15.105:0/2642862809 >> 172.21.15.105:0/2642862809 conn(0x5585840ef8c0 msgr2=0x5585840f1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:26.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f8318949000 1 -- 172.21.15.105:0/2642862809 shutdown_connections 2023-11-28T18:27:26.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f8318949000 1 -- 172.21.15.105:0/2642862809 wait complete. 2023-11-28T18:27:26.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f8318949000 1 -- start start 2023-11-28T18:27:26.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f8318949000 1 Processor -- start 2023-11-28T18:27:26.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f8318949000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5585840c2500 0x5585841164b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f8318949000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55858410ed30 0x5585841169f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f8318949000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55858410f650 0x5585840c8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f8318949000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x558583f147c0 con 0x55858410ed30 2023-11-28T18:27:26.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f8318949000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558583eccbf0 con 0x5585840c2500 2023-11-28T18:27:26.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f8318949000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x558583ed2a90 con 0x55858410f650 2023-11-28T18:27:26.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f8318949000 10 client.0 _ll_register_callbacks cb 0x5585840afe20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:27:26.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f83067fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55858410ed30 0x5585841169f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f83067fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55858410ed30 0x5585841169f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37756/0 (socket says 172.21.15.105:37756) 2023-11-28T18:27:26.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f83067fc640 1 -- 172.21.15.105:0/1670921877 learned_addr learned my addr 172.21.15.105:0/1670921877 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:26.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f830f7fe640 1 --2- 172.21.15.105:0/1670921877 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55858410f650 0x5585840c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f830effd640 1 --2- 172.21.15.105:0/1670921877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5585840c2500 0x5585841164b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.839+0000 7f83067fc640 1 -- 172.21.15.105:0/1670921877 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55858410f650 msgr2=0x5585840c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:26.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f83067fc640 1 --2- 172.21.15.105:0/1670921877 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55858410f650 0x5585840c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f83067fc640 1 -- 172.21.15.105:0/1670921877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5585840c2500 msgr2=0x5585841164b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:26.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f83067fc640 1 --2- 172.21.15.105:0/1670921877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5585840c2500 0x5585841164b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f83067fc640 1 -- 172.21.15.105:0/1670921877 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83100c90e0 con 0x55858410ed30 2023-11-28T18:27:26.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f830effd640 1 --2- 172.21.15.105:0/1670921877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5585840c2500 0x5585841164b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:26.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f83067fc640 1 --2- 172.21.15.105:0/1670921877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55858410ed30 0x5585841169f0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f8300007bb0 tx=0x7f8300007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a148c8795da2909a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:26.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f82edffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:27:26.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f82edffb640 1 -- 172.21.15.105:0/1670921877 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83000ef630 con 0x55858410ed30 2023-11-28T18:27:26.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f82edffb640 1 -- 172.21.15.105:0/1670921877 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f83000f7930 con 0x55858410ed30 2023-11-28T18:27:26.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f8318949000 1 -- 172.21.15.105:0/1670921877 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558583ed6470 con 0x55858410ed30 2023-11-28T18:27:26.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f82edffb640 1 -- 172.21.15.105:0/1670921877 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f830010d420 con 0x55858410ed30 2023-11-28T18:27:26.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f8318949000 10 client.6918 fetch_fsmap learned FSMap version 61 2023-11-28T18:27:26.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f8318949000 1 -- 172.21.15.105:0/1670921877 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x558583ed5190 con 0x55858410ed30 2023-11-28T18:27:26.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f82edffb640 1 -- 172.21.15.105:0/1670921877 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558583ed6470 con 0x55858410ed30 2023-11-28T18:27:26.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f82edffb640 1 -- 172.21.15.105:0/1670921877 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 61) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f83000c4020 con 0x55858410ed30 2023-11-28T18:27:26.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f8318949000 10 client.6918 fetch_fsmap finished waiting for FSMap version 61 2023-11-28T18:27:26.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f8318949000 10 client.6918 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T18:27:26.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f8318949000 20 client.6918 populate_metadata read hostname 'smithi105' 2023-11-28T18:27:26.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f8318949000 1 --2- 172.21.15.105:0/1670921877 >> [v2:172.21.15.148:6833/2396073410,v1:172.21.15.148:6835/2396073410] conn(0x5585840c9080 0x5585840c9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f8318949000 4 client.6918 mds_command: new command op to 6436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:27:26.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f8318949000 1 -- 172.21.15.105:0/1670921877 --> [v2:172.21.15.148:6833/2396073410,v1:172.21.15.148:6835/2396073410] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558583ed9b50 con 0x5585840c9080 2023-11-28T18:27:26.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f830effd640 1 --2- 172.21.15.105:0/1670921877 >> [v2:172.21.15.148:6833/2396073410,v1:172.21.15.148:6835/2396073410] conn(0x5585840c9080 0x5585840c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f830effd640 1 --2- 172.21.15.105:0/1670921877 >> [v2:172.21.15.148:6833/2396073410,v1:172.21.15.148:6835/2396073410] conn(0x5585840c9080 0x5585840c9460 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf73104c389cad2f server_cookie=156874ef240871b2 in_seq=0 out_seq=0 2023-11-28T18:27:26.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f82edffb640 10 client.6918 ms_handle_connect on v2:172.21.15.148:6833/2396073410 2023-11-28T18:27:26.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f82edffb640 1 -- 172.21.15.105:0/1670921877 <== mds.2 v2:172.21.15.148:6833/2396073410 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558583ed9b50 con 0x5585840c9080 2023-11-28T18:27:26.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f82edffb640 10 client.6918 handle_command_reply: tid=0 2023-11-28T18:27:26.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f8318949000 1 client.6918 shutdown 2023-11-28T18:27:26.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f8318949000 1 -- 172.21.15.105:0/1670921877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55858410ed30 msgr2=0x5585841169f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:26.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f8318949000 1 --2- 172.21.15.105:0/1670921877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55858410ed30 0x5585841169f0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f8300007bb0 tx=0x7f8300007c90 comp rx=0 tx=0).stop 2023-11-28T18:27:26.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f8318949000 1 -- 172.21.15.105:0/1670921877 shutdown_connections 2023-11-28T18:27:26.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f8318949000 1 --2- 172.21.15.105:0/1670921877 >> [v2:172.21.15.148:6833/2396073410,v1:172.21.15.148:6835/2396073410] conn(0x5585840c9080 0x5585840c9460 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f8318949000 1 --2- 172.21.15.105:0/1670921877 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55858410f650 0x5585840c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f8318949000 1 --2- 172.21.15.105:0/1670921877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55858410ed30 0x5585841169f0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f82edffb640 0 client.6918 ms_handle_reset on v2:172.21.15.148:6833/2396073410 2023-11-28T18:27:26.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f8318949000 1 --2- 172.21.15.105:0/1670921877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5585840c2500 0x5585841164b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f8318949000 1 -- 172.21.15.105:0/1670921877 >> 172.21.15.105:0/1670921877 conn(0x5585840ef8c0 msgr2=0x5585840c16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:26.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f8318949000 1 -- 172.21.15.105:0/1670921877 shutdown_connections 2023-11-28T18:27:26.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f8318949000 1 -- 172.21.15.105:0/1670921877 wait complete. 2023-11-28T18:27:26.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.843+0000 7f8318949000 20 client.6918 trim_cache size 0 max 16384 2023-11-28T18:27:26.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.859+0000 7f8318949000 1 Processor -- start 2023-11-28T18:27:26.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.859+0000 7f8318949000 1 -- start start 2023-11-28T18:27:26.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.859+0000 7f8318949000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585840c2500 0x5585840ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.859+0000 7f8318949000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55858410ed30 0x5585840c9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.859+0000 7f8318949000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55858410f650 0x5585840c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.859+0000 7f8318949000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x558583bcd940 con 0x5585840c2500 2023-11-28T18:27:26.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.859+0000 7f8318949000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585840bc430 con 0x55858410ed30 2023-11-28T18:27:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.859+0000 7f8318949000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585840cc510 con 0x55858410f650 2023-11-28T18:27:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.859+0000 7f830effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585840c2500 0x5585840ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f830effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585840c2500 0x5585840ca5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37762/0 (socket says 172.21.15.105:37762) 2023-11-28T18:27:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f830effd640 1 -- 172.21.15.105:0/4126725138 learned_addr learned my addr 172.21.15.105:0/4126725138 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f830f7fe640 1 --2- 172.21.15.105:0/4126725138 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55858410f650 0x5585840c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f83067fc640 1 --2- 172.21.15.105:0/4126725138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55858410ed30 0x5585840c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f830effd640 1 -- 172.21.15.105:0/4126725138 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55858410f650 msgr2=0x5585840c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f830effd640 1 --2- 172.21.15.105:0/4126725138 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55858410f650 0x5585840c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f830effd640 1 -- 172.21.15.105:0/4126725138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55858410ed30 msgr2=0x5585840c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f830effd640 1 --2- 172.21.15.105:0/4126725138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55858410ed30 0x5585840c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f830effd640 1 -- 172.21.15.105:0/4126725138 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558583ebb490 con 0x5585840c2500 2023-11-28T18:27:26.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f830effd640 1 --2- 172.21.15.105:0/4126725138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585840c2500 0x5585840ca5c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f82f8004310 tx=0x7f82f801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a797b810950e5f5e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:26.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f83057fa640 1 -- 172.21.15.105:0/4126725138 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f802a040 con 0x5585840c2500 2023-11-28T18:27:26.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f83057fa640 1 -- 172.21.15.105:0/4126725138 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f8019070 con 0x5585840c2500 2023-11-28T18:27:26.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f83057fa640 1 -- 172.21.15.105:0/4126725138 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f802bd20 con 0x5585840c2500 2023-11-28T18:27:26.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f8318949000 1 -- 172.21.15.105:0/4126725138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585840c2500 msgr2=0x5585840ca5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:26.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f8318949000 1 --2- 172.21.15.105:0/4126725138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585840c2500 0x5585840ca5c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f82f8004310 tx=0x7f82f801b000 comp rx=0 tx=0).stop 2023-11-28T18:27:26.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f8318949000 1 -- 172.21.15.105:0/4126725138 shutdown_connections 2023-11-28T18:27:26.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f8318949000 1 --2- 172.21.15.105:0/4126725138 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55858410f650 0x5585840c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f8318949000 1 --2- 172.21.15.105:0/4126725138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55858410ed30 0x5585840c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f8318949000 1 --2- 172.21.15.105:0/4126725138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585840c2500 0x5585840ca5c0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f8318949000 1 -- 172.21.15.105:0/4126725138 >> 172.21.15.105:0/4126725138 conn(0x5585840ef8c0 msgr2=0x55858411cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:26.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f8318949000 1 -- 172.21.15.105:0/4126725138 shutdown_connections 2023-11-28T18:27:26.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f8318949000 1 -- 172.21.15.105:0/4126725138 wait complete. 2023-11-28T18:27:26.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f8318949000 1 -- start start 2023-11-28T18:27:26.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f8318949000 1 Processor -- start 2023-11-28T18:27:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f8318949000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5585840c2500 0x55858416f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f8318949000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55858410ed30 0x55858416fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f8318949000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55858410f650 0x558584113d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f8318949000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585840c5e10 con 0x55858410ed30 2023-11-28T18:27:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f8318949000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558583eccbf0 con 0x55858410f650 2023-11-28T18:27:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f8318949000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x558583f0eac0 con 0x5585840c2500 2023-11-28T18:27:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f8318949000 10 client.0 _ll_register_callbacks cb 0x558583f8b060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:27:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f83067fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55858410ed30 0x55858416fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f83067fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55858410ed30 0x55858416fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37776/0 (socket says 172.21.15.105:37776) 2023-11-28T18:27:26.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f83067fc640 1 -- 172.21.15.105:0/423428830 learned_addr learned my addr 172.21.15.105:0/423428830 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:26.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f830effd640 1 --2- 172.21.15.105:0/423428830 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5585840c2500 0x55858416f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.863+0000 7f830f7fe640 1 --2- 172.21.15.105:0/423428830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55858410f650 0x558584113d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f83067fc640 1 -- 172.21.15.105:0/423428830 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5585840c2500 msgr2=0x55858416f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:26.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f83067fc640 1 --2- 172.21.15.105:0/423428830 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5585840c2500 0x55858416f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f83067fc640 1 -- 172.21.15.105:0/423428830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55858410f650 msgr2=0x558584113d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:26.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f83067fc640 1 --2- 172.21.15.105:0/423428830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55858410f650 0x558584113d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f83067fc640 1 -- 172.21.15.105:0/423428830 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558583c0ccc0 con 0x55858410ed30 2023-11-28T18:27:26.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f83067fc640 1 --2- 172.21.15.105:0/423428830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55858410ed30 0x55858416fef0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f83000ee5e0 tx=0x7f83000eb970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd5a29531f54909 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:26.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f82edffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:27:26.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f82edffb640 1 -- 172.21.15.105:0/423428830 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83000ef170 con 0x55858410ed30 2023-11-28T18:27:26.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f82edffb640 1 -- 172.21.15.105:0/423428830 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f83000efd30 con 0x55858410ed30 2023-11-28T18:27:26.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f8318949000 1 -- 172.21.15.105:0/423428830 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585840c8710 con 0x55858410ed30 2023-11-28T18:27:26.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f82edffb640 1 -- 172.21.15.105:0/423428830 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83000f7930 con 0x55858410ed30 2023-11-28T18:27:26.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f8318949000 10 client.6924 fetch_fsmap learned FSMap version 61 2023-11-28T18:27:26.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f8318949000 1 -- 172.21.15.105:0/423428830 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x558583e3da50 con 0x55858410ed30 2023-11-28T18:27:26.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f82edffb640 1 -- 172.21.15.105:0/423428830 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5585840c8710 con 0x55858410ed30 2023-11-28T18:27:26.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f82edffb640 1 -- 172.21.15.105:0/423428830 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 61) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f83000c4040 con 0x55858410ed30 2023-11-28T18:27:26.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f8318949000 10 client.6924 fetch_fsmap finished waiting for FSMap version 61 2023-11-28T18:27:26.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f8318949000 10 client.6924 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T18:27:26.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f8318949000 20 client.6924 populate_metadata read hostname 'smithi105' 2023-11-28T18:27:26.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f8318949000 1 --2- 172.21.15.105:0/423428830 >> [v2:172.21.15.148:6833/2396073410,v1:172.21.15.148:6835/2396073410] conn(0x558584114690 0x558584114a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:26.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f8318949000 4 client.6924 mds_command: new command op to 6436 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:27:26.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f8318949000 1 -- 172.21.15.105:0/423428830 --> [v2:172.21.15.148:6833/2396073410,v1:172.21.15.148:6835/2396073410] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558583f13110 con 0x558584114690 2023-11-28T18:27:26.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f830effd640 1 --2- 172.21.15.105:0/423428830 >> [v2:172.21.15.148:6833/2396073410,v1:172.21.15.148:6835/2396073410] conn(0x558584114690 0x558584114a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:26.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f830effd640 1 --2- 172.21.15.105:0/423428830 >> [v2:172.21.15.148:6833/2396073410,v1:172.21.15.148:6835/2396073410] conn(0x558584114690 0x558584114a70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=54766ed704f56c62 server_cookie=4b7d98f49794a835 in_seq=0 out_seq=0 2023-11-28T18:27:26.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f82edffb640 10 client.6924 ms_handle_connect on v2:172.21.15.148:6833/2396073410 2023-11-28T18:27:26.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f82edffb640 1 -- 172.21.15.105:0/423428830 <== mds.2 v2:172.21.15.148:6833/2396073410 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x7f82f8027150 con 0x558584114690 2023-11-28T18:27:26.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f82edffb640 10 client.6924 handle_command_reply: tid=0 2023-11-28T18:27:26.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f8318949000 1 client.6924 shutdown 2023-11-28T18:27:26.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f8318949000 1 -- 172.21.15.105:0/423428830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55858410ed30 msgr2=0x55858416fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:26.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f8318949000 1 --2- 172.21.15.105:0/423428830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55858410ed30 0x55858416fef0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f83000ee5e0 tx=0x7f83000eb970 comp rx=0 tx=0).stop 2023-11-28T18:27:26.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f8318949000 1 -- 172.21.15.105:0/423428830 shutdown_connections 2023-11-28T18:27:26.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f8318949000 1 --2- 172.21.15.105:0/423428830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55858410f650 0x558584113d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f8318949000 1 --2- 172.21.15.105:0/423428830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55858410ed30 0x55858416fef0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f8318949000 1 --2- 172.21.15.105:0/423428830 >> [v2:172.21.15.148:6833/2396073410,v1:172.21.15.148:6835/2396073410] conn(0x558584114690 0x558584114a70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f8318949000 1 --2- 172.21.15.105:0/423428830 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5585840c2500 0x55858416f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f82edffb640 0 client.6924 ms_handle_reset on v2:172.21.15.148:6833/2396073410 2023-11-28T18:27:26.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f8318949000 1 -- 172.21.15.105:0/423428830 >> 172.21.15.105:0/423428830 conn(0x5585840ef8c0 msgr2=0x5585840c9460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:26.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f8318949000 1 -- 172.21.15.105:0/423428830 shutdown_connections 2023-11-28T18:27:26.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f8318949000 1 -- 172.21.15.105:0/423428830 wait complete. 2023-11-28T18:27:26.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f8318949000 20 client.6924 trim_cache size 0 max 16384 2023-11-28T18:27:26.879 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:27:26.879 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:27:26.879 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 2, 2023-11-28T18:27:26.879 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 6436, 2023-11-28T18:27:26.879 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:27:26.879 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:27:26.879 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:27:26.879 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 1.7633557639999999, 2023-11-28T18:27:26.880 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 61, 2023-11-28T18:27:26.880 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 67, 2023-11-28T18:27:26.880 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 67, 2023-11-28T18:27:26.880 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 26.278015174 2023-11-28T18:27:26.880 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:27:26.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f83155d1640 1 -- 172.21.15.105:0/3983356434 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82f00668d0 msgr2=0x7f82f0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:26.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f83155d1640 1 --2- 172.21.15.105:0/3983356434 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82f00668d0 0x7f82f0068d90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f83100c7ff0 tx=0x7f82f8009190 comp rx=0 tx=0).stop 2023-11-28T18:27:26.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f83155d1640 1 -- 172.21.15.105:0/3983356434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f831015a720 msgr2=0x7f83100b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:26.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f83155d1640 1 --2- 172.21.15.105:0/3983356434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f831015a720 0x7f83100b19b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f830001c210 tx=0x7f82fc005800 comp rx=0 tx=0).stop 2023-11-28T18:27:26.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f83155d1640 1 -- 172.21.15.105:0/3983356434 shutdown_connections 2023-11-28T18:27:26.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f83155d1640 1 --2- 172.21.15.105:0/3983356434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8310159b80 0x7f831015a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f83155d1640 1 --2- 172.21.15.105:0/3983356434 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f82f00668d0 0x7f82f0068d90 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f83155d1640 1 --2- 172.21.15.105:0/3983356434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f831015a720 0x7f83100b19b0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f83155d1640 1 --2- 172.21.15.105:0/3983356434 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82fc0047d0 0x7f831014e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:26.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f83155d1640 1 -- 172.21.15.105:0/3983356434 >> 172.21.15.105:0/3983356434 conn(0x7f83100ceb90 msgr2=0x7f8310070c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:26.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f83155d1640 1 -- 172.21.15.105:0/3983356434 shutdown_connections 2023-11-28T18:27:26.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:26.867+0000 7f83155d1640 1 -- 172.21.15.105:0/3983356434 wait complete. 2023-11-28T18:27:26.891 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T18:27:27.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.023+0000 7f15e52c9640 1 Processor -- start 2023-11-28T18:27:27.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.027+0000 7f15e52c9640 1 -- start start 2023-11-28T18:27:27.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.027+0000 7f15e52c9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f15e00b4290 0x7f15e00b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.027+0000 7f15e52c9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f15e015b030 0x7f15e00709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.027+0000 7f15e52c9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15e00b4bb0 con 0x7f15e0071960 2023-11-28T18:27:27.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.027+0000 7f15e52c9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15e00701d0 con 0x7f15e015b030 2023-11-28T18:27:27.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.027+0000 7f15e52c9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15e006df80 con 0x7f15e00b4290 2023-11-28T18:27:27.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.027+0000 7f15d67fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f15e0071960 0x7f15e015f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33584/0 (socket says 172.21.15.105:33584) 2023-11-28T18:27:27.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.027+0000 7f15d67fc640 1 -- 172.21.15.105:0/1750088056 learned_addr learned my addr 172.21.15.105:0/1750088056 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:27.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.027+0000 7f15df7fe640 1 --2- 172.21.15.105:0/1750088056 >> v2:172.21.15.105:3300/0 conn(0x7f15e015b030 0x7f15e00709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.027+0000 7f15deffd640 1 --2- 172.21.15.105:0/1750088056 >> v2:172.21.15.148:3300/0 conn(0x7f15e00b4290 0x7f15e00b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.027+0000 7f15df7fe640 1 -- 172.21.15.105:0/1750088056 >> v2:172.21.15.148:3300/0 conn(0x7f15e00b4290 msgr2=0x7f15e00b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:27.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.027+0000 7f15df7fe640 1 --2- 172.21.15.105:0/1750088056 >> v2:172.21.15.148:3300/0 conn(0x7f15e00b4290 0x7f15e00b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.027+0000 7f15df7fe640 1 -- 172.21.15.105:0/1750088056 >> v1:172.21.15.105:6789/0 conn(0x7f15e0071960 legacy=0x7f15e015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:27.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.027+0000 7f15df7fe640 1 -- 172.21.15.105:0/1750088056 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e0071df0 con 0x7f15e015b030 2023-11-28T18:27:27.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.027+0000 7f15de7fc640 1 -- 172.21.15.105:0/1750088056 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1789588523 0 0) 0x7f15e00b4bb0 con 0x7f15e0071960 2023-11-28T18:27:27.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.031+0000 7f15df7fe640 1 --2- 172.21.15.105:0/1750088056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15e015b030 0x7f15e00709b0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f15e0073a70 tx=0x7f15d801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1363be92a55a6363 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:27.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.031+0000 7f15de7fc640 1 -- 172.21.15.105:0/1750088056 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d80ab600 con 0x7f15e015b030 2023-11-28T18:27:27.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.031+0000 7f15de7fc640 1 -- 172.21.15.105:0/1750088056 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d80a7050 con 0x7f15e015b030 2023-11-28T18:27:27.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.031+0000 7f15de7fc640 1 -- 172.21.15.105:0/1750088056 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d80ab980 con 0x7f15e015b030 2023-11-28T18:27:27.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.031+0000 7f15e52c9640 1 -- 172.21.15.105:0/1750088056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15e015b030 msgr2=0x7f15e00709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:27.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.031+0000 7f15e52c9640 1 --2- 172.21.15.105:0/1750088056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15e015b030 0x7f15e00709b0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f15e0073a70 tx=0x7f15d801c430 comp rx=0 tx=0).stop 2023-11-28T18:27:27.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.031+0000 7f15e52c9640 1 -- 172.21.15.105:0/1750088056 shutdown_connections 2023-11-28T18:27:27.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.031+0000 7f15e52c9640 1 --2- 172.21.15.105:0/1750088056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15e015b030 0x7f15e00709b0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.031+0000 7f15e52c9640 1 --2- 172.21.15.105:0/1750088056 >> v2:172.21.15.148:3300/0 conn(0x7f15e00b4290 0x7f15e00b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.031+0000 7f15e52c9640 1 -- 172.21.15.105:0/1750088056 >> 172.21.15.105:0/1750088056 conn(0x7f15e00ceb90 msgr2=0x7f15e00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:27.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.031+0000 7f15e52c9640 1 -- 172.21.15.105:0/1750088056 shutdown_connections 2023-11-28T18:27:27.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.031+0000 7f15e52c9640 1 -- 172.21.15.105:0/1750088056 wait complete. 2023-11-28T18:27:27.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.031+0000 7f15e52c9640 1 Processor -- start 2023-11-28T18:27:27.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.031+0000 7f15e52c9640 1 -- start start 2023-11-28T18:27:27.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.035+0000 7f15e52c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15e00b4290 0x7f15e014bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.035+0000 7f15e52c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15e015b030 0x7f15e014c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.035+0000 7f15e52c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15e014c9d0 0x7f15e009c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.035+0000 7f15e52c9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e009b080 con 0x7f15e014c9d0 2023-11-28T18:27:27.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.035+0000 7f15e52c9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e009b320 con 0x7f15e015b030 2023-11-28T18:27:27.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.035+0000 7f15e52c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e009b600 con 0x7f15e00b4290 2023-11-28T18:27:27.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.035+0000 7f15df7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15e014c9d0 0x7f15e009c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.035+0000 7f15df7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15e014c9d0 0x7f15e009c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37788/0 (socket says 172.21.15.105:37788) 2023-11-28T18:27:27.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.035+0000 7f15df7fe640 1 -- 172.21.15.105:0/1862955246 learned_addr learned my addr 172.21.15.105:0/1862955246 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:27.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.035+0000 7f15deffd640 1 --2- 172.21.15.105:0/1862955246 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15e00b4290 0x7f15e014bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.035+0000 7f15d67fc640 1 --2- 172.21.15.105:0/1862955246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15e015b030 0x7f15e014c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.035+0000 7f15df7fe640 1 -- 172.21.15.105:0/1862955246 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15e00b4290 msgr2=0x7f15e014bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:27.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.035+0000 7f15df7fe640 1 --2- 172.21.15.105:0/1862955246 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15e00b4290 0x7f15e014bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.035+0000 7f15df7fe640 1 -- 172.21.15.105:0/1862955246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15e015b030 msgr2=0x7f15e014c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:27.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.035+0000 7f15df7fe640 1 --2- 172.21.15.105:0/1862955246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15e015b030 0x7f15e014c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.035+0000 7f15df7fe640 1 -- 172.21.15.105:0/1862955246 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d8009420 con 0x7f15e014c9d0 2023-11-28T18:27:27.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.035+0000 7f15df7fe640 1 --2- 172.21.15.105:0/1862955246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15e014c9d0 0x7f15e009c0e0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f15d8019040 tx=0x7f15d801ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:27.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.035+0000 7f15d7fff640 1 -- 172.21.15.105:0/1862955246 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d80b4420 con 0x7f15e014c9d0 2023-11-28T18:27:27.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.035+0000 7f15d7fff640 1 -- 172.21.15.105:0/1862955246 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d80b4580 con 0x7f15e014c9d0 2023-11-28T18:27:27.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.035+0000 7f15d7fff640 1 -- 172.21.15.105:0/1862955246 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d80b4910 con 0x7f15e014c9d0 2023-11-28T18:27:27.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.035+0000 7f15e52c9640 1 -- 172.21.15.105:0/1862955246 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e009b8e0 con 0x7f15e014c9d0 2023-11-28T18:27:27.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.035+0000 7f15e52c9640 1 -- 172.21.15.105:0/1862955246 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e014c1d0 con 0x7f15e014c9d0 2023-11-28T18:27:27.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15d7fff640 1 -- 172.21.15.105:0/1862955246 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f15d80b2070 con 0x7f15e014c9d0 2023-11-28T18:27:27.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15d7fff640 1 --2- 172.21.15.105:0/1862955246 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15c00668d0 0x7f15c0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15d7fff640 1 -- 172.21.15.105:0/1862955246 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f15d811d7e0 con 0x7f15e014c9d0 2023-11-28T18:27:27.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15deffd640 1 --2- 172.21.15.105:0/1862955246 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15c00668d0 0x7f15c0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15e862b000 1 Processor -- start 2023-11-28T18:27:27.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15e862b000 1 -- start start 2023-11-28T18:27:27.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15deffd640 1 --2- 172.21.15.105:0/1862955246 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15c00668d0 0x7f15c0068d90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f15d0003b90 tx=0x7f15d00098d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:27.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15e862b000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f020e79330 0x55f020e79710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15e862b000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f020e79c50 0x55f020e5dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15e862b000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e5e2a0 0x55f020e5e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15e862b000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f020cf1110 con 0x55f020e5e2a0 2023-11-28T18:27:27.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15e862b000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f020c819a0 con 0x55f020e79330 2023-11-28T18:27:27.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15e862b000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f020c7eed0 con 0x55f020e79c50 2023-11-28T18:27:27.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15df7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e5e2a0 0x55f020e5e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15d67fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f020e79c50 0x55f020e5dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15deffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f020e79330 0x55f020e79710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15df7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e5e2a0 0x55f020e5e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37798/0 (socket says 172.21.15.105:37798) 2023-11-28T18:27:27.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15df7fe640 1 -- 172.21.15.105:0/955940155 learned_addr learned my addr 172.21.15.105:0/955940155 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:27.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15df7fe640 1 -- 172.21.15.105:0/955940155 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f020e79c50 msgr2=0x55f020e5dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:27.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15df7fe640 1 --2- 172.21.15.105:0/955940155 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f020e79c50 0x55f020e5dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15df7fe640 1 -- 172.21.15.105:0/955940155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f020e79330 msgr2=0x55f020e79710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:27.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15df7fe640 1 --2- 172.21.15.105:0/955940155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f020e79330 0x55f020e79710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15df7fe640 1 -- 172.21.15.105:0/955940155 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f020972890 con 0x55f020e5e2a0 2023-11-28T18:27:27.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15df7fe640 1 --2- 172.21.15.105:0/955940155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e5e2a0 0x55f020e5e680 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f15d80ee4b0 tx=0x7f15d80191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87720a7a0255cbbe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:27.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15d4ff9640 1 -- 172.21.15.105:0/955940155 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d80f6770 con 0x55f020e5e2a0 2023-11-28T18:27:27.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15d4ff9640 1 -- 172.21.15.105:0/955940155 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d80ec8c0 con 0x55f020e5e2a0 2023-11-28T18:27:27.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.039+0000 7f15d4ff9640 1 -- 172.21.15.105:0/955940155 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d80eda10 con 0x55f020e5e2a0 2023-11-28T18:27:27.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15e862b000 1 -- 172.21.15.105:0/955940155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e5e2a0 msgr2=0x55f020e5e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:27.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15e862b000 1 --2- 172.21.15.105:0/955940155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e5e2a0 0x55f020e5e680 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f15d80ee4b0 tx=0x7f15d80191d0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15e862b000 1 -- 172.21.15.105:0/955940155 shutdown_connections 2023-11-28T18:27:27.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15e862b000 1 --2- 172.21.15.105:0/955940155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e5e2a0 0x55f020e5e680 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15e862b000 1 --2- 172.21.15.105:0/955940155 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f020e79c50 0x55f020e5dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15e862b000 1 --2- 172.21.15.105:0/955940155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f020e79330 0x55f020e79710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15e862b000 1 -- 172.21.15.105:0/955940155 >> 172.21.15.105:0/955940155 conn(0x55f020e558c0 msgr2=0x55f020e57cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:27.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15e862b000 1 -- 172.21.15.105:0/955940155 shutdown_connections 2023-11-28T18:27:27.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15e862b000 1 -- 172.21.15.105:0/955940155 wait complete. 2023-11-28T18:27:27.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15e862b000 1 -- start start 2023-11-28T18:27:27.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15e862b000 1 Processor -- start 2023-11-28T18:27:27.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15e862b000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f020e5e2a0 0x55f020e2ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15e862b000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e79330 0x55f020e2f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15e862b000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f020e79c50 0x55f020e2f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15e862b000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f020c7a7c0 con 0x55f020e79330 2023-11-28T18:27:27.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15e862b000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f020c32bf0 con 0x55f020e79c50 2023-11-28T18:27:27.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15e862b000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f020c38a90 con 0x55f020e5e2a0 2023-11-28T18:27:27.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15e862b000 10 client.0 _ll_register_callbacks cb 0x55f020e15e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:27:27.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15d67fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e79330 0x55f020e2f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15d67fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e79330 0x55f020e2f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37800/0 (socket says 172.21.15.105:37800) 2023-11-28T18:27:27.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15d67fc640 1 -- 172.21.15.105:0/85724096 learned_addr learned my addr 172.21.15.105:0/85724096 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:27.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15deffd640 1 --2- 172.21.15.105:0/85724096 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f020e5e2a0 0x55f020e2ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15df7fe640 1 --2- 172.21.15.105:0/85724096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f020e79c50 0x55f020e2f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15d67fc640 1 -- 172.21.15.105:0/85724096 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f020e5e2a0 msgr2=0x55f020e2ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:27.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15d67fc640 1 --2- 172.21.15.105:0/85724096 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f020e5e2a0 0x55f020e2ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15d67fc640 1 -- 172.21.15.105:0/85724096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f020e79c50 msgr2=0x55f020e2f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:27.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15d67fc640 1 --2- 172.21.15.105:0/85724096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f020e79c50 0x55f020e2f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15d67fc640 1 -- 172.21.15.105:0/85724096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e014c1d0 con 0x55f020e79330 2023-11-28T18:27:27.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15d67fc640 1 --2- 172.21.15.105:0/85724096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e79330 0x55f020e2f2b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f15c8004f90 tx=0x7f15c8001790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bed5650e602f7b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:27.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:27:27.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15bdffb640 1 -- 172.21.15.105:0/85724096 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c80145b0 con 0x55f020e79330 2023-11-28T18:27:27.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15bdffb640 1 -- 172.21.15.105:0/85724096 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f15c800c950 con 0x55f020e79330 2023-11-28T18:27:27.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15e862b000 1 -- 172.21.15.105:0/85724096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f020c3c470 con 0x55f020e79330 2023-11-28T18:27:27.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.043+0000 7f15bdffb640 1 -- 172.21.15.105:0/85724096 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15c800cb10 con 0x55f020e79330 2023-11-28T18:27:27.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15e862b000 10 client.6939 fetch_fsmap learned FSMap version 61 2023-11-28T18:27:27.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15e862b000 1 -- 172.21.15.105:0/85724096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x55f020c3b190 con 0x55f020e79330 2023-11-28T18:27:27.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15bdffb640 1 -- 172.21.15.105:0/85724096 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c8024020 con 0x55f020e79330 2023-11-28T18:27:27.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15bdffb640 1 -- 172.21.15.105:0/85724096 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 61) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f15c8012050 con 0x55f020e79330 2023-11-28T18:27:27.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15e862b000 10 client.6939 fetch_fsmap finished waiting for FSMap version 61 2023-11-28T18:27:27.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15e862b000 10 client.6939 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:27:27.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15e862b000 20 client.6939 populate_metadata read hostname 'smithi105' 2023-11-28T18:27:27.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15e862b000 1 --2- 172.21.15.105:0/85724096 >> [v2:172.21.15.105:6834/380088531,v1:172.21.15.105:6835/380088531] conn(0x55f020e30200 0x55f020e8a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15e862b000 4 client.6939 mds_command: new command op to 6717 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:27:27.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15e862b000 1 -- 172.21.15.105:0/85724096 --> [v2:172.21.15.105:6834/380088531,v1:172.21.15.105:6835/380088531] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f020c3fb50 con 0x55f020e30200 2023-11-28T18:27:27.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15deffd640 1 --2- 172.21.15.105:0/85724096 >> [v2:172.21.15.105:6834/380088531,v1:172.21.15.105:6835/380088531] conn(0x55f020e30200 0x55f020e8a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15deffd640 1 --2- 172.21.15.105:0/85724096 >> [v2:172.21.15.105:6834/380088531,v1:172.21.15.105:6835/380088531] conn(0x55f020e30200 0x55f020e8a5e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a6ac568a5e2dca3 server_cookie=93c5a598dae18c59 in_seq=0 out_seq=0 2023-11-28T18:27:27.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15bdffb640 10 client.6939 ms_handle_connect on v2:172.21.15.105:6834/380088531 2023-11-28T18:27:27.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15bdffb640 1 -- 172.21.15.105:0/85724096 <== mds.1 v2:172.21.15.105:6834/380088531 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f020c3fb50 con 0x55f020e30200 2023-11-28T18:27:27.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15bdffb640 10 client.6939 handle_command_reply: tid=0 2023-11-28T18:27:27.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15e862b000 1 client.6939 shutdown 2023-11-28T18:27:27.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15e862b000 1 -- 172.21.15.105:0/85724096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e79330 msgr2=0x55f020e2f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:27.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15e862b000 1 --2- 172.21.15.105:0/85724096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e79330 0x55f020e2f2b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f15c8004f90 tx=0x7f15c8001790 comp rx=0 tx=0).stop 2023-11-28T18:27:27.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15e862b000 1 -- 172.21.15.105:0/85724096 shutdown_connections 2023-11-28T18:27:27.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15e862b000 1 --2- 172.21.15.105:0/85724096 >> [v2:172.21.15.105:6834/380088531,v1:172.21.15.105:6835/380088531] conn(0x55f020e30200 0x55f020e8a5e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15e862b000 1 --2- 172.21.15.105:0/85724096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f020e79c50 0x55f020e2f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15bdffb640 0 client.6939 ms_handle_reset on v2:172.21.15.105:6834/380088531 2023-11-28T18:27:27.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15e862b000 1 --2- 172.21.15.105:0/85724096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e79330 0x55f020e2f2b0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15e862b000 1 --2- 172.21.15.105:0/85724096 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f020e5e2a0 0x55f020e2ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15e862b000 1 -- 172.21.15.105:0/85724096 >> 172.21.15.105:0/85724096 conn(0x55f020e558c0 msgr2=0x55f020e276b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:27.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15e862b000 1 -- 172.21.15.105:0/85724096 shutdown_connections 2023-11-28T18:27:27.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15e862b000 1 -- 172.21.15.105:0/85724096 wait complete. 2023-11-28T18:27:27.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.047+0000 7f15e862b000 20 client.6939 trim_cache size 0 max 16384 2023-11-28T18:27:27.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.063+0000 7f15e862b000 1 Processor -- start 2023-11-28T18:27:27.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.063+0000 7f15e862b000 1 -- start start 2023-11-28T18:27:27.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.063+0000 7f15e862b000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e5e2a0 0x55f020ed5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.063+0000 7f15e862b000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f020e79330 0x55f020ed5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.063+0000 7f15e862b000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f020e79c50 0x55f020ed6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.063+0000 7f15e862b000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f020933940 con 0x55f020e5e2a0 2023-11-28T18:27:27.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.063+0000 7f15e862b000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f020cf1110 con 0x55f020e79330 2023-11-28T18:27:27.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.063+0000 7f15e862b000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f020c158a0 con 0x55f020e79c50 2023-11-28T18:27:27.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.063+0000 7f15deffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e5e2a0 0x55f020ed5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.063+0000 7f15deffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e5e2a0 0x55f020ed5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37808/0 (socket says 172.21.15.105:37808) 2023-11-28T18:27:27.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.063+0000 7f15deffd640 1 -- 172.21.15.105:0/906995392 learned_addr learned my addr 172.21.15.105:0/906995392 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:27.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.063+0000 7f15deffd640 1 -- 172.21.15.105:0/906995392 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f020e79c50 msgr2=0x55f020ed6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:27.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.063+0000 7f15deffd640 1 --2- 172.21.15.105:0/906995392 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f020e79c50 0x55f020ed6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.063+0000 7f15deffd640 1 -- 172.21.15.105:0/906995392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f020e79330 msgr2=0x55f020ed5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:27.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.063+0000 7f15deffd640 1 --2- 172.21.15.105:0/906995392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f020e79330 0x55f020ed5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.063+0000 7f15deffd640 1 -- 172.21.15.105:0/906995392 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f020c21490 con 0x55f020e5e2a0 2023-11-28T18:27:27.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.063+0000 7f15deffd640 1 --2- 172.21.15.105:0/906995392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e5e2a0 0x55f020ed5680 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f15d00044d0 tx=0x7f15d000b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8aa039e41019fde server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:27.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.063+0000 7f15d57fa640 1 -- 172.21.15.105:0/906995392 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d002a070 con 0x55f020e5e2a0 2023-11-28T18:27:27.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.063+0000 7f15d57fa640 1 -- 172.21.15.105:0/906995392 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d002b310 con 0x55f020e5e2a0 2023-11-28T18:27:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.063+0000 7f15d57fa640 1 -- 172.21.15.105:0/906995392 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d002b4a0 con 0x55f020e5e2a0 2023-11-28T18:27:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 1 -- 172.21.15.105:0/906995392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e5e2a0 msgr2=0x55f020ed5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 1 --2- 172.21.15.105:0/906995392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e5e2a0 0x55f020ed5680 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f15d00044d0 tx=0x7f15d000b050 comp rx=0 tx=0).stop 2023-11-28T18:27:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 1 -- 172.21.15.105:0/906995392 shutdown_connections 2023-11-28T18:27:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 1 --2- 172.21.15.105:0/906995392 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f020e79c50 0x55f020ed6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 1 --2- 172.21.15.105:0/906995392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f020e79330 0x55f020ed5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 1 --2- 172.21.15.105:0/906995392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e5e2a0 0x55f020ed5680 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 1 -- 172.21.15.105:0/906995392 >> 172.21.15.105:0/906995392 conn(0x55f020e558c0 msgr2=0x55f020e27630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 1 -- 172.21.15.105:0/906995392 shutdown_connections 2023-11-28T18:27:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 1 -- 172.21.15.105:0/906995392 wait complete. 2023-11-28T18:27:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 1 -- start start 2023-11-28T18:27:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 1 Processor -- start 2023-11-28T18:27:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f020e5e2a0 0x55f020e5d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f020e79330 0x55f020e950b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e79c50 0x55f020e955f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f020c74ac0 con 0x55f020e79c50 2023-11-28T18:27:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f020c2f760 con 0x55f020e5e2a0 2023-11-28T18:27:27.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f020c765c0 con 0x55f020e79330 2023-11-28T18:27:27.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 10 client.0 _ll_register_callbacks cb 0x55f020cf1060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:27:27.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15df7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e79c50 0x55f020e955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15df7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e79c50 0x55f020e955f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37816/0 (socket says 172.21.15.105:37816) 2023-11-28T18:27:27.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15df7fe640 1 -- 172.21.15.105:0/193084388 learned_addr learned my addr 172.21.15.105:0/193084388 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:27.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15deffd640 1 --2- 172.21.15.105:0/193084388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f020e5e2a0 0x55f020e5d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15d67fc640 1 --2- 172.21.15.105:0/193084388 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f020e79330 0x55f020e950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15df7fe640 1 -- 172.21.15.105:0/193084388 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f020e79330 msgr2=0x55f020e950b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:27.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15df7fe640 1 --2- 172.21.15.105:0/193084388 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f020e79330 0x55f020e950b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15df7fe640 1 -- 172.21.15.105:0/193084388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f020e5e2a0 msgr2=0x55f020e5d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:27.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15df7fe640 1 --2- 172.21.15.105:0/193084388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f020e5e2a0 0x55f020e5d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15df7fe640 1 -- 172.21.15.105:0/193084388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d0011040 con 0x55f020e79c50 2023-11-28T18:27:27.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15d67fc640 1 --2- 172.21.15.105:0/193084388 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f020e79330 0x55f020e950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:27.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15df7fe640 1 --2- 172.21.15.105:0/193084388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e79c50 0x55f020e955f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f15d80abfd0 tx=0x7f15d811d050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce83ce7324c70b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:27.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:27:27.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15bdffb640 1 -- 172.21.15.105:0/193084388 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d810a070 con 0x55f020e79c50 2023-11-28T18:27:27.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15bdffb640 1 -- 172.21.15.105:0/193084388 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f15d80f6770 con 0x55f020e79c50 2023-11-28T18:27:27.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 1 -- 172.21.15.105:0/193084388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f020c79110 con 0x55f020e79c50 2023-11-28T18:27:27.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15bdffb640 1 -- 172.21.15.105:0/193084388 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15d80ea860 con 0x55f020e79c50 2023-11-28T18:27:27.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 10 client.6945 fetch_fsmap learned FSMap version 61 2023-11-28T18:27:27.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 1 -- 172.21.15.105:0/193084388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x55f020972cc0 con 0x55f020e79c50 2023-11-28T18:27:27.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15bdffb640 1 -- 172.21.15.105:0/193084388 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f020c79110 con 0x55f020e79c50 2023-11-28T18:27:27.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15bdffb640 1 -- 172.21.15.105:0/193084388 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 61) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f15d8105040 con 0x55f020e79c50 2023-11-28T18:27:27.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 10 client.6945 fetch_fsmap finished waiting for FSMap version 61 2023-11-28T18:27:27.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 10 client.6945 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:27:27.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 20 client.6945 populate_metadata read hostname 'smithi105' 2023-11-28T18:27:27.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 1 --2- 172.21.15.105:0/193084388 >> [v2:172.21.15.105:6834/380088531,v1:172.21.15.105:6835/380088531] conn(0x55f020ed8d20 0x55f020ed9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 4 client.6945 mds_command: new command op to 6717 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:27:27.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.067+0000 7f15e862b000 1 -- 172.21.15.105:0/193084388 --> [v2:172.21.15.105:6834/380088531,v1:172.21.15.105:6835/380088531] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f020ca7650 con 0x55f020ed8d20 2023-11-28T18:27:27.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15deffd640 1 --2- 172.21.15.105:0/193084388 >> [v2:172.21.15.105:6834/380088531,v1:172.21.15.105:6835/380088531] conn(0x55f020ed8d20 0x55f020ed9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15deffd640 1 --2- 172.21.15.105:0/193084388 >> [v2:172.21.15.105:6834/380088531,v1:172.21.15.105:6835/380088531] conn(0x55f020ed8d20 0x55f020ed9100 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ea5039b0c8c6102 server_cookie=d0564ff6e3e1e897 in_seq=0 out_seq=0 2023-11-28T18:27:27.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15bdffb640 10 client.6945 ms_handle_connect on v2:172.21.15.105:6834/380088531 2023-11-28T18:27:27.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15bdffb640 1 -- 172.21.15.105:0/193084388 <== mds.1 v2:172.21.15.105:6834/380088531 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55f020ca7650 con 0x55f020ed8d20 2023-11-28T18:27:27.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15bdffb640 10 client.6945 handle_command_reply: tid=0 2023-11-28T18:27:27.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e862b000 1 client.6945 shutdown 2023-11-28T18:27:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e862b000 1 -- 172.21.15.105:0/193084388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e79c50 msgr2=0x55f020e955f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e862b000 1 --2- 172.21.15.105:0/193084388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e79c50 0x55f020e955f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f15d80abfd0 tx=0x7f15d811d050 comp rx=0 tx=0).stop 2023-11-28T18:27:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e862b000 1 -- 172.21.15.105:0/193084388 shutdown_connections 2023-11-28T18:27:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e862b000 1 --2- 172.21.15.105:0/193084388 >> [v2:172.21.15.105:6834/380088531,v1:172.21.15.105:6835/380088531] conn(0x55f020ed8d20 0x55f020ed9100 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e862b000 1 --2- 172.21.15.105:0/193084388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f020e79c50 0x55f020e955f0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e862b000 1 --2- 172.21.15.105:0/193084388 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f020e79330 0x55f020e950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15bdffb640 0 client.6945 ms_handle_reset on v2:172.21.15.105:6834/380088531 2023-11-28T18:27:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e862b000 1 --2- 172.21.15.105:0/193084388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f020e5e2a0 0x55f020e5d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e862b000 1 -- 172.21.15.105:0/193084388 >> 172.21.15.105:0/193084388 conn(0x55f020e558c0 msgr2=0x55f020e53390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e862b000 1 -- 172.21.15.105:0/193084388 shutdown_connections 2023-11-28T18:27:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e862b000 1 -- 172.21.15.105:0/193084388 wait complete. 2023-11-28T18:27:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e862b000 20 client.6945 trim_cache size 0 max 16384 2023-11-28T18:27:27.090 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:27:27.090 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:27:27.090 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 1, 2023-11-28T18:27:27.090 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 6717, 2023-11-28T18:27:27.090 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:27:27.090 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:27:27.090 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:27:27.090 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 2.9733801780000002, 2023-11-28T18:27:27.090 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 61, 2023-11-28T18:27:27.091 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 67, 2023-11-28T18:27:27.091 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 67, 2023-11-28T18:27:27.091 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 26.489711132 2023-11-28T18:27:27.091 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:27:27.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e52c9640 1 -- 172.21.15.105:0/1862955246 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15c00668d0 msgr2=0x7f15c0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e52c9640 1 --2- 172.21.15.105:0/1862955246 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15c00668d0 0x7f15c0068d90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f15d0003b90 tx=0x7f15d00098d0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e52c9640 1 -- 172.21.15.105:0/1862955246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15e014c9d0 msgr2=0x7f15e009c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e52c9640 1 --2- 172.21.15.105:0/1862955246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15e014c9d0 0x7f15e009c0e0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f15d8019040 tx=0x7f15d801ac70 comp rx=0 tx=0).stop 2023-11-28T18:27:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e52c9640 1 -- 172.21.15.105:0/1862955246 shutdown_connections 2023-11-28T18:27:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e52c9640 1 --2- 172.21.15.105:0/1862955246 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15c00668d0 0x7f15c0068d90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e52c9640 1 --2- 172.21.15.105:0/1862955246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15e014c9d0 0x7f15e009c0e0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e52c9640 1 --2- 172.21.15.105:0/1862955246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15e015b030 0x7f15e014c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e52c9640 1 --2- 172.21.15.105:0/1862955246 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15e00b4290 0x7f15e014bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e52c9640 1 -- 172.21.15.105:0/1862955246 >> 172.21.15.105:0/1862955246 conn(0x7f15e00ceb90 msgr2=0x7f15e006c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e52c9640 1 -- 172.21.15.105:0/1862955246 shutdown_connections 2023-11-28T18:27:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.071+0000 7f15e52c9640 1 -- 172.21.15.105:0/1862955246 wait complete. 2023-11-28T18:27:27.093 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T18:27:27.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.231+0000 7fbfcd501640 1 Processor -- start 2023-11-28T18:27:27.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.231+0000 7fbfcd501640 1 -- start start 2023-11-28T18:27:27.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.231+0000 7fbfcd501640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfc8159610 0x7fbfc80ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.231+0000 7fbfcd501640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbfc80cc760 0x7fbfc80ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.231+0000 7fbfcd501640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc80cde30 con 0x7fbfc806c7e0 2023-11-28T18:27:27.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.231+0000 7fbfcd501640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc81599f0 con 0x7fbfc80cc760 2023-11-28T18:27:27.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.231+0000 7fbfcd501640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc806de20 con 0x7fbfc8159610 2023-11-28T18:27:27.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.231+0000 7fbfbe7fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbfc80cc760 0x7fbfc80ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.231+0000 7fbfbe7fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbfc80cc760 0x7fbfc80ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37824/0 (socket says 172.21.15.105:37824) 2023-11-28T18:27:27.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.231+0000 7fbfbe7fc640 1 -- 172.21.15.105:0/1976844154 learned_addr learned my addr 172.21.15.105:0/1976844154 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:27.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.231+0000 7fbfbe7fc640 1 -- 172.21.15.105:0/1976844154 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfc8159610 msgr2=0x7fbfc80ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:27.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.231+0000 7fbfbe7fc640 1 --2- 172.21.15.105:0/1976844154 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfc8159610 0x7fbfc80ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.231+0000 7fbfbe7fc640 1 -- 172.21.15.105:0/1976844154 >> v1:172.21.15.148:6789/0 conn(0x7fbfc806c7e0 legacy=0x7fbfc80abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:27.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.231+0000 7fbfbe7fc640 1 -- 172.21.15.105:0/1976844154 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc80cd220 con 0x7fbfc80cc760 2023-11-28T18:27:27.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.235+0000 7fbfbe7fc640 1 --2- 172.21.15.105:0/1976844154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc80cc760 0x7fbfc80ccbc0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009870 tx=0x7fbfc80acca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53cfaa1ffe16e9d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:27.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.235+0000 7fbfc67fc640 1 -- 172.21.15.105:0/1976844154 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb80aa5f0 con 0x7fbfc80cc760 2023-11-28T18:27:27.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.235+0000 7fbfc67fc640 1 -- 172.21.15.105:0/1976844154 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb80aa750 con 0x7fbfc80cc760 2023-11-28T18:27:27.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.235+0000 7fbfc67fc640 1 -- 172.21.15.105:0/1976844154 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb80aaae0 con 0x7fbfc80cc760 2023-11-28T18:27:27.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.235+0000 7fbfcd501640 1 -- 172.21.15.105:0/1976844154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc80cc760 msgr2=0x7fbfc80ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:27.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.235+0000 7fbfcd501640 1 --2- 172.21.15.105:0/1976844154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc80cc760 0x7fbfc80ccbc0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009870 tx=0x7fbfc80acca0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.235+0000 7fbfcd501640 1 -- 172.21.15.105:0/1976844154 shutdown_connections 2023-11-28T18:27:27.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.235+0000 7fbfcd501640 1 --2- 172.21.15.105:0/1976844154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc80cc760 0x7fbfc80ccbc0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.235+0000 7fbfcd501640 1 --2- 172.21.15.105:0/1976844154 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfc8159610 0x7fbfc80ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.235+0000 7fbfcd501640 1 -- 172.21.15.105:0/1976844154 >> 172.21.15.105:0/1976844154 conn(0x7fbfc80ceb90 msgr2=0x7fbfc80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:27.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.235+0000 7fbfcd501640 1 -- 172.21.15.105:0/1976844154 shutdown_connections 2023-11-28T18:27:27.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.235+0000 7fbfcd501640 1 -- 172.21.15.105:0/1976844154 wait complete. 2023-11-28T18:27:27.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfcd501640 1 Processor -- start 2023-11-28T18:27:27.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfcd501640 1 -- start start 2023-11-28T18:27:27.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfcd501640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfc806c7e0 0x7fbfc80ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfcd501640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfc80cc760 0x7fbfc81511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfcd501640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc8159610 0x7fbfc81516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfcd501640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8151d60 con 0x7fbfc8159610 2023-11-28T18:27:27.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfcd501640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8152040 con 0x7fbfc806c7e0 2023-11-28T18:27:27.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfcd501640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8152320 con 0x7fbfc80cc760 2023-11-28T18:27:27.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfc77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc8159610 0x7fbfc81516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfc77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc8159610 0x7fbfc81516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37828/0 (socket says 172.21.15.105:37828) 2023-11-28T18:27:27.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfc77fe640 1 -- 172.21.15.105:0/3009327339 learned_addr learned my addr 172.21.15.105:0/3009327339 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:27.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfbe7fc640 1 --2- 172.21.15.105:0/3009327339 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfc80cc760 0x7fbfc81511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfc6ffd640 1 --2- 172.21.15.105:0/3009327339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfc806c7e0 0x7fbfc80ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfc77fe640 1 -- 172.21.15.105:0/3009327339 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfc80cc760 msgr2=0x7fbfc81511a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:27.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfc77fe640 1 --2- 172.21.15.105:0/3009327339 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfc80cc760 0x7fbfc81511a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfc77fe640 1 -- 172.21.15.105:0/3009327339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfc806c7e0 msgr2=0x7fbfc80ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:27.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfc77fe640 1 --2- 172.21.15.105:0/3009327339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfc806c7e0 0x7fbfc80ca610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfc77fe640 1 -- 172.21.15.105:0/3009327339 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb80094d0 con 0x7fbfc8159610 2023-11-28T18:27:27.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfc77fe640 1 --2- 172.21.15.105:0/3009327339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc8159610 0x7fbfc81516e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000c780 tx=0x7fbfc000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:27.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfbffff640 1 -- 172.21.15.105:0/3009327339 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0003d90 con 0x7fbfc8159610 2023-11-28T18:27:27.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfbffff640 1 -- 172.21.15.105:0/3009327339 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc000ce90 con 0x7fbfc8159610 2023-11-28T18:27:27.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfbffff640 1 -- 172.21.15.105:0/3009327339 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc00052b0 con 0x7fbfc8159610 2023-11-28T18:27:27.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfcd501640 1 -- 172.21.15.105:0/3009327339 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc8152600 con 0x7fbfc8159610 2023-11-28T18:27:27.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.239+0000 7fbfcd501640 1 -- 172.21.15.105:0/3009327339 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc80cab50 con 0x7fbfc8159610 2023-11-28T18:27:27.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.243+0000 7fbfbffff640 1 -- 172.21.15.105:0/3009327339 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fbfc0005410 con 0x7fbfc8159610 2023-11-28T18:27:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.243+0000 7fbfbffff640 1 --2- 172.21.15.105:0/3009327339 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbfa80668d0 0x7fbfa8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.243+0000 7fbfbffff640 1 -- 172.21.15.105:0/3009327339 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fbfc0083a30 con 0x7fbfc8159610 2023-11-28T18:27:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.243+0000 7fbfc6ffd640 1 --2- 172.21.15.105:0/3009327339 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbfa80668d0 0x7fbfa8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.243+0000 7fbfd0871000 1 Processor -- start 2023-11-28T18:27:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.243+0000 7fbfd0871000 1 -- start start 2023-11-28T18:27:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.243+0000 7fbfc6ffd640 1 --2- 172.21.15.105:0/3009327339 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbfa80668d0 0x7fbfa8068d90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00098f0 tx=0x7fbfb0006f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.243+0000 7fbfd0871000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634c1fd0d30 0x5634c1fd1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.243+0000 7fbfd0871000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd1650 0x5634c1f83fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.243+0000 7fbfd0871000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634c1f84500 0x5634c1f848e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.243+0000 7fbfd0871000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c1e4d110 con 0x5634c1fd1650 2023-11-28T18:27:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.243+0000 7fbfd0871000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c1ddd9a0 con 0x5634c1fd0d30 2023-11-28T18:27:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.243+0000 7fbfd0871000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5634c1ddaed0 con 0x5634c1f84500 2023-11-28T18:27:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.243+0000 7fbfbe7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd1650 0x5634c1f83fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.243+0000 7fbfc6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634c1fd0d30 0x5634c1fd1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.243+0000 7fbfbe7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd1650 0x5634c1f83fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37838/0 (socket says 172.21.15.105:37838) 2023-11-28T18:27:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.243+0000 7fbfbe7fc640 1 -- 172.21.15.105:0/2714556732 learned_addr learned my addr 172.21.15.105:0/2714556732 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.243+0000 7fbfc77fe640 1 --2- 172.21.15.105:0/2714556732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634c1f84500 0x5634c1f848e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfbe7fc640 1 -- 172.21.15.105:0/2714556732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634c1f84500 msgr2=0x5634c1f848e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfbe7fc640 1 --2- 172.21.15.105:0/2714556732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634c1f84500 0x5634c1f848e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfbe7fc640 1 -- 172.21.15.105:0/2714556732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634c1fd0d30 msgr2=0x5634c1fd1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfbe7fc640 1 --2- 172.21.15.105:0/2714556732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634c1fd0d30 0x5634c1fd1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfbe7fc640 1 -- 172.21.15.105:0/2714556732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634c1ace890 con 0x5634c1fd1650 2023-11-28T18:27:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfc6ffd640 1 --2- 172.21.15.105:0/2714556732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634c1fd0d30 0x5634c1fd1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:27:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfbe7fc640 1 --2- 172.21.15.105:0/2714556732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd1650 0x5634c1f83fc0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80ac040 tx=0x7fbfb8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ca8662d1503904 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfbcff9640 1 -- 172.21.15.105:0/2714556732 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb80aab00 con 0x5634c1fd1650 2023-11-28T18:27:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfbcff9640 1 -- 172.21.15.105:0/2714556732 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb80aaca0 con 0x5634c1fd1650 2023-11-28T18:27:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfbcff9640 1 -- 172.21.15.105:0/2714556732 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb80be660 con 0x5634c1fd1650 2023-11-28T18:27:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfd0871000 1 -- 172.21.15.105:0/2714556732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd1650 msgr2=0x5634c1f83fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:27.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfd0871000 1 --2- 172.21.15.105:0/2714556732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd1650 0x5634c1f83fc0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80ac040 tx=0x7fbfb8004740 comp rx=0 tx=0).stop 2023-11-28T18:27:27.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfd0871000 1 -- 172.21.15.105:0/2714556732 shutdown_connections 2023-11-28T18:27:27.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfd0871000 1 --2- 172.21.15.105:0/2714556732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634c1f84500 0x5634c1f848e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfd0871000 1 --2- 172.21.15.105:0/2714556732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd1650 0x5634c1f83fc0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfd0871000 1 --2- 172.21.15.105:0/2714556732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634c1fd0d30 0x5634c1fd1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfd0871000 1 -- 172.21.15.105:0/2714556732 >> 172.21.15.105:0/2714556732 conn(0x5634c1fb18c0 msgr2=0x5634c1fb3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:27.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfd0871000 1 -- 172.21.15.105:0/2714556732 shutdown_connections 2023-11-28T18:27:27.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfd0871000 1 -- 172.21.15.105:0/2714556732 wait complete. 2023-11-28T18:27:27.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfd0871000 1 -- start start 2023-11-28T18:27:27.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfd0871000 1 Processor -- start 2023-11-28T18:27:27.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfd0871000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634c1f84500 0x5634c1f8a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfd0871000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd0d30 0x5634c1f8aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfd0871000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634c1fd1650 0x5634c1f8afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfd0871000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c1dd67c0 con 0x5634c1fd0d30 2023-11-28T18:27:27.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfd0871000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c1d8ebf0 con 0x5634c1fd1650 2023-11-28T18:27:27.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfd0871000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5634c1d94a90 con 0x5634c1f84500 2023-11-28T18:27:27.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfd0871000 10 client.0 _ll_register_callbacks cb 0x5634c1f71e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:27:27.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.247+0000 7fbfbe7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd0d30 0x5634c1f8aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfbe7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd0d30 0x5634c1f8aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37850/0 (socket says 172.21.15.105:37850) 2023-11-28T18:27:27.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfbe7fc640 1 -- 172.21.15.105:0/3410946688 learned_addr learned my addr 172.21.15.105:0/3410946688 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:27.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfc77fe640 1 --2- 172.21.15.105:0/3410946688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634c1fd1650 0x5634c1f8afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfc6ffd640 1 --2- 172.21.15.105:0/3410946688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634c1f84500 0x5634c1f8a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfbe7fc640 1 -- 172.21.15.105:0/3410946688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634c1f84500 msgr2=0x5634c1f8a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:27.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfbe7fc640 1 --2- 172.21.15.105:0/3410946688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634c1f84500 0x5634c1f8a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfbe7fc640 1 -- 172.21.15.105:0/3410946688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634c1fd1650 msgr2=0x5634c1f8afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:27.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfbe7fc640 1 --2- 172.21.15.105:0/3410946688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634c1fd1650 0x5634c1f8afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfbe7fc640 1 -- 172.21.15.105:0/3410946688 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb000e850 con 0x5634c1fd0d30 2023-11-28T18:27:27.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfbe7fc640 1 --2- 172.21.15.105:0/3410946688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd0d30 0x5634c1f8aa60 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8008010 tx=0x7fbfb801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f599cfc733ccf43b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:27.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:27:27.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfa5ffb640 1 -- 172.21.15.105:0/3410946688 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb80aab00 con 0x5634c1fd0d30 2023-11-28T18:27:27.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfa5ffb640 1 -- 172.21.15.105:0/3410946688 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fbfb80a7070 con 0x5634c1fd0d30 2023-11-28T18:27:27.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfd0871000 1 -- 172.21.15.105:0/3410946688 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634c1d98470 con 0x5634c1fd0d30 2023-11-28T18:27:27.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfa5ffb640 1 -- 172.21.15.105:0/3410946688 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfb8003620 con 0x5634c1fd0d30 2023-11-28T18:27:27.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfd0871000 10 client.6957 fetch_fsmap learned FSMap version 61 2023-11-28T18:27:27.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfd0871000 1 -- 172.21.15.105:0/3410946688 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x7fbfc80cd220 con 0x5634c1fd0d30 2023-11-28T18:27:27.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfa5ffb640 1 -- 172.21.15.105:0/3410946688 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5634c1d98470 con 0x5634c1fd0d30 2023-11-28T18:27:27.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfa5ffb640 1 -- 172.21.15.105:0/3410946688 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 61) v1 ==== 3318+0+0 (secure 0 0 0) 0x7fbfb80c6020 con 0x5634c1fd0d30 2023-11-28T18:27:27.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfd0871000 10 client.6957 fetch_fsmap finished waiting for FSMap version 61 2023-11-28T18:27:27.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfd0871000 10 client.6957 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:27:27.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfd0871000 20 client.6957 populate_metadata read hostname 'smithi105' 2023-11-28T18:27:27.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfd0871000 1 --2- 172.21.15.105:0/3410946688 >> [v2:172.21.15.105:6836/3442512486,v1:172.21.15.105:6837/3442512486] conn(0x5634c1f8b9b0 0x5634c1fe2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfd0871000 4 client.6957 mds_command: new command op to 6720 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:27:27.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfd0871000 1 -- 172.21.15.105:0/3410946688 --> [v2:172.21.15.105:6836/3442512486,v1:172.21.15.105:6837/3442512486] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634c1d9bb50 con 0x5634c1f8b9b0 2023-11-28T18:27:27.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfc6ffd640 1 --2- 172.21.15.105:0/3410946688 >> [v2:172.21.15.105:6836/3442512486,v1:172.21.15.105:6837/3442512486] conn(0x5634c1f8b9b0 0x5634c1fe2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfc6ffd640 1 --2- 172.21.15.105:0/3410946688 >> [v2:172.21.15.105:6836/3442512486,v1:172.21.15.105:6837/3442512486] conn(0x5634c1f8b9b0 0x5634c1fe2ff0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b73dc3a192fa42a server_cookie=aca8f7615f7124fb in_seq=0 out_seq=0 2023-11-28T18:27:27.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfa5ffb640 10 client.6957 ms_handle_connect on v2:172.21.15.105:6836/3442512486 2023-11-28T18:27:27.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfa5ffb640 1 -- 172.21.15.105:0/3410946688 <== mds.0 v2:172.21.15.105:6836/3442512486 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5634c1d9bb50 con 0x5634c1f8b9b0 2023-11-28T18:27:27.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfa5ffb640 10 client.6957 handle_command_reply: tid=0 2023-11-28T18:27:27.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfd0871000 1 client.6957 shutdown 2023-11-28T18:27:27.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfd0871000 1 -- 172.21.15.105:0/3410946688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd0d30 msgr2=0x5634c1f8aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:27.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfd0871000 1 --2- 172.21.15.105:0/3410946688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd0d30 0x5634c1f8aa60 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8008010 tx=0x7fbfb801ad10 comp rx=0 tx=0).stop 2023-11-28T18:27:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfd0871000 1 -- 172.21.15.105:0/3410946688 shutdown_connections 2023-11-28T18:27:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfd0871000 1 --2- 172.21.15.105:0/3410946688 >> [v2:172.21.15.105:6836/3442512486,v1:172.21.15.105:6837/3442512486] conn(0x5634c1f8b9b0 0x5634c1fe2ff0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfd0871000 1 --2- 172.21.15.105:0/3410946688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634c1fd1650 0x5634c1f8afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfd0871000 1 --2- 172.21.15.105:0/3410946688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd0d30 0x5634c1f8aa60 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfd0871000 1 --2- 172.21.15.105:0/3410946688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634c1f84500 0x5634c1f8a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfa5ffb640 0 client.6957 ms_handle_reset on v2:172.21.15.105:6836/3442512486 2023-11-28T18:27:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfd0871000 1 -- 172.21.15.105:0/3410946688 >> 172.21.15.105:0/3410946688 conn(0x5634c1fb18c0 msgr2=0x5634c1f836b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfd0871000 1 -- 172.21.15.105:0/3410946688 shutdown_connections 2023-11-28T18:27:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfd0871000 1 -- 172.21.15.105:0/3410946688 wait complete. 2023-11-28T18:27:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.251+0000 7fbfd0871000 20 client.6957 trim_cache size 0 max 16384 2023-11-28T18:27:27.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.267+0000 7fbfd0871000 1 Processor -- start 2023-11-28T18:27:27.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.267+0000 7fbfd0871000 1 -- start start 2023-11-28T18:27:27.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.267+0000 7fbfd0871000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634c1f84500 0x5634c1fb03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.267+0000 7fbfd0871000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634c1fd0d30 0x5634c1fb08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.267+0000 7fbfd0871000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd1650 0x5634c1fe2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.267+0000 7fbfd0871000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c1a8f940 con 0x5634c1fd1650 2023-11-28T18:27:27.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.267+0000 7fbfd0871000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c1f8bfc0 con 0x5634c1fd0d30 2023-11-28T18:27:27.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.267+0000 7fbfd0871000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5634c1d5da00 con 0x5634c1f84500 2023-11-28T18:27:27.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.267+0000 7fbfc77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd1650 0x5634c1fe2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.267+0000 7fbfc77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd1650 0x5634c1fe2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37854/0 (socket says 172.21.15.105:37854) 2023-11-28T18:27:27.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.267+0000 7fbfc77fe640 1 -- 172.21.15.105:0/1808700158 learned_addr learned my addr 172.21.15.105:0/1808700158 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:27.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.267+0000 7fbfbe7fc640 1 --2- 172.21.15.105:0/1808700158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634c1fd0d30 0x5634c1fb08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.267+0000 7fbfc6ffd640 1 --2- 172.21.15.105:0/1808700158 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634c1f84500 0x5634c1fb03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.267+0000 7fbfc77fe640 1 -- 172.21.15.105:0/1808700158 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634c1f84500 msgr2=0x5634c1fb03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:27.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.267+0000 7fbfc77fe640 1 --2- 172.21.15.105:0/1808700158 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634c1f84500 0x5634c1fb03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.267+0000 7fbfc77fe640 1 -- 172.21.15.105:0/1808700158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634c1fd0d30 msgr2=0x5634c1fb08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:27.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.267+0000 7fbfc77fe640 1 --2- 172.21.15.105:0/1808700158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634c1fd0d30 0x5634c1fb08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfc77fe640 1 -- 172.21.15.105:0/1808700158 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634c1d97190 con 0x5634c1fd1650 2023-11-28T18:27:27.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfbe7fc640 1 --2- 172.21.15.105:0/1808700158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634c1fd0d30 0x5634c1fb08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:27.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfc6ffd640 1 --2- 172.21.15.105:0/1808700158 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634c1f84500 0x5634c1fb03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:27.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfc77fe640 1 --2- 172.21.15.105:0/1808700158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd1650 0x5634c1fe2ff0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00547a0 tx=0x7fbfc001d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703043c720a427d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:27.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfbd7fa640 1 -- 172.21.15.105:0/1808700158 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0052bb0 con 0x5634c1fd1650 2023-11-28T18:27:27.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfbd7fa640 1 -- 172.21.15.105:0/1808700158 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc00056c0 con 0x5634c1fd1650 2023-11-28T18:27:27.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfbd7fa640 1 -- 172.21.15.105:0/1808700158 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0005830 con 0x5634c1fd1650 2023-11-28T18:27:27.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfd0871000 1 -- 172.21.15.105:0/1808700158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd1650 msgr2=0x5634c1fe2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:27.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfd0871000 1 --2- 172.21.15.105:0/1808700158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd1650 0x5634c1fe2ff0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00547a0 tx=0x7fbfc001d040 comp rx=0 tx=0).stop 2023-11-28T18:27:27.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfd0871000 1 -- 172.21.15.105:0/1808700158 shutdown_connections 2023-11-28T18:27:27.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfd0871000 1 --2- 172.21.15.105:0/1808700158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd1650 0x5634c1fe2ff0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfd0871000 1 --2- 172.21.15.105:0/1808700158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634c1fd0d30 0x5634c1fb08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfd0871000 1 --2- 172.21.15.105:0/1808700158 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634c1f84500 0x5634c1fb03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfd0871000 1 -- 172.21.15.105:0/1808700158 >> 172.21.15.105:0/1808700158 conn(0x5634c1fb18c0 msgr2=0x5634c1f82720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:27.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfd0871000 1 -- 172.21.15.105:0/1808700158 shutdown_connections 2023-11-28T18:27:27.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfd0871000 1 -- 172.21.15.105:0/1808700158 wait complete. 2023-11-28T18:27:27.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfd0871000 1 -- start start 2023-11-28T18:27:27.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfd0871000 1 Processor -- start 2023-11-28T18:27:27.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfd0871000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634c1f84500 0x5634c203a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfd0871000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd0d30 0x5634c203a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfd0871000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634c1fd1650 0x5634c1ff5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfd0871000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c1ddcf30 con 0x5634c1fd0d30 2023-11-28T18:27:27.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfd0871000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634c1dd25c0 con 0x5634c1f84500 2023-11-28T18:27:27.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfd0871000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5634c1e6c440 con 0x5634c1fd1650 2023-11-28T18:27:27.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfd0871000 10 client.0 _ll_register_callbacks cb 0x5634c1e4d060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:27:27.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfbe7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd0d30 0x5634c203a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfbe7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd0d30 0x5634c203a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37864/0 (socket says 172.21.15.105:37864) 2023-11-28T18:27:27.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfbe7fc640 1 -- 172.21.15.105:0/3047450042 learned_addr learned my addr 172.21.15.105:0/3047450042 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:27.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfc6ffd640 1 --2- 172.21.15.105:0/3047450042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634c1f84500 0x5634c203a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfc77fe640 1 --2- 172.21.15.105:0/3047450042 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634c1fd1650 0x5634c1ff5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfbe7fc640 1 -- 172.21.15.105:0/3047450042 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634c1fd1650 msgr2=0x5634c1ff5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:27.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfbe7fc640 1 --2- 172.21.15.105:0/3047450042 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634c1fd1650 0x5634c1ff5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfbe7fc640 1 -- 172.21.15.105:0/3047450042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634c1f84500 msgr2=0x5634c203a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:27.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfbe7fc640 1 --2- 172.21.15.105:0/3047450042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634c1f84500 0x5634c203a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfbe7fc640 1 -- 172.21.15.105:0/3047450042 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc80cab50 con 0x5634c1fd0d30 2023-11-28T18:27:27.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfc77fe640 1 --2- 172.21.15.105:0/3047450042 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634c1fd1650 0x5634c1ff5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:27.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfbe7fc640 1 --2- 172.21.15.105:0/3047450042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd0d30 0x5634c203a560 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800b0b0 tx=0x7fbfb8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1666692d616d0848 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:27.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.271+0000 7fbfa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:27:27.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfa5ffb640 1 -- 172.21.15.105:0/3047450042 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8003a30 con 0x5634c1fd0d30 2023-11-28T18:27:27.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfa5ffb640 1 -- 172.21.15.105:0/3047450042 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fbfb8003b90 con 0x5634c1fd0d30 2023-11-28T18:27:27.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfd0871000 1 -- 172.21.15.105:0/3047450042 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634c1dd5110 con 0x5634c1fd0d30 2023-11-28T18:27:27.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfa5ffb640 1 -- 172.21.15.105:0/3047450042 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfb8003d50 con 0x5634c1fd0d30 2023-11-28T18:27:27.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfd0871000 10 client.6963 fetch_fsmap learned FSMap version 61 2023-11-28T18:27:27.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfd0871000 1 -- 172.21.15.105:0/3047450042 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x5634c1d7d490 con 0x5634c1fd0d30 2023-11-28T18:27:27.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfa5ffb640 1 -- 172.21.15.105:0/3047450042 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5634c1dd5110 con 0x5634c1fd0d30 2023-11-28T18:27:27.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfa5ffb640 1 -- 172.21.15.105:0/3047450042 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 61) v1 ==== 3318+0+0 (secure 0 0 0) 0x7fbfb80c6070 con 0x5634c1fd0d30 2023-11-28T18:27:27.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfd0871000 10 client.6963 fetch_fsmap finished waiting for FSMap version 61 2023-11-28T18:27:27.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfd0871000 10 client.6963 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:27:27.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfd0871000 20 client.6963 populate_metadata read hostname 'smithi105' 2023-11-28T18:27:27.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfd0871000 1 --2- 172.21.15.105:0/3047450042 >> [v2:172.21.15.105:6836/3442512486,v1:172.21.15.105:6837/3442512486] conn(0x5634c1ff5c40 0x5634c1ff6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:27.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfd0871000 4 client.6963 mds_command: new command op to 6720 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:27:27.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfd0871000 1 -- 172.21.15.105:0/3047450042 --> [v2:172.21.15.105:6836/3442512486,v1:172.21.15.105:6837/3442512486] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5634c1e03650 con 0x5634c1ff5c40 2023-11-28T18:27:27.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfc6ffd640 1 --2- 172.21.15.105:0/3047450042 >> [v2:172.21.15.105:6836/3442512486,v1:172.21.15.105:6837/3442512486] conn(0x5634c1ff5c40 0x5634c1ff6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:27.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfa5ffb640 10 client.6963 ms_handle_connect on v2:172.21.15.105:6836/3442512486 2023-11-28T18:27:27.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfc6ffd640 1 --2- 172.21.15.105:0/3047450042 >> [v2:172.21.15.105:6836/3442512486,v1:172.21.15.105:6837/3442512486] conn(0x5634c1ff5c40 0x5634c1ff6020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adeed3dcfd76147c server_cookie=4d76e34dbaafb2f5 in_seq=0 out_seq=0 2023-11-28T18:27:27.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfa5ffb640 1 -- 172.21.15.105:0/3047450042 <== mds.0 v2:172.21.15.105:6836/3442512486 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5634c1e03650 con 0x5634c1ff5c40 2023-11-28T18:27:27.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfa5ffb640 10 client.6963 handle_command_reply: tid=0 2023-11-28T18:27:27.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfd0871000 1 client.6963 shutdown 2023-11-28T18:27:27.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfd0871000 1 -- 172.21.15.105:0/3047450042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd0d30 msgr2=0x5634c203a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:27.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfd0871000 1 --2- 172.21.15.105:0/3047450042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd0d30 0x5634c203a560 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800b0b0 tx=0x7fbfb8008040 comp rx=0 tx=0).stop 2023-11-28T18:27:27.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfd0871000 1 -- 172.21.15.105:0/3047450042 shutdown_connections 2023-11-28T18:27:27.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfd0871000 1 --2- 172.21.15.105:0/3047450042 >> [v2:172.21.15.105:6836/3442512486,v1:172.21.15.105:6837/3442512486] conn(0x5634c1ff5c40 0x5634c1ff6020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfd0871000 1 --2- 172.21.15.105:0/3047450042 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5634c1fd1650 0x5634c1ff5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfd0871000 1 --2- 172.21.15.105:0/3047450042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634c1fd0d30 0x5634c203a560 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfd0871000 1 --2- 172.21.15.105:0/3047450042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634c1f84500 0x5634c203a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfa5ffb640 0 client.6963 ms_handle_reset on v2:172.21.15.105:6836/3442512486 2023-11-28T18:27:27.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfd0871000 1 -- 172.21.15.105:0/3047450042 >> 172.21.15.105:0/3047450042 conn(0x5634c1fb18c0 msgr2=0x5634c1f82fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:27.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfd0871000 1 -- 172.21.15.105:0/3047450042 shutdown_connections 2023-11-28T18:27:27.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfd0871000 1 -- 172.21.15.105:0/3047450042 wait complete. 2023-11-28T18:27:27.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfd0871000 20 client.6963 trim_cache size 0 max 16384 2023-11-28T18:27:27.287 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:27:27.287 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:27:27.287 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2023-11-28T18:27:27.287 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 6720, 2023-11-28T18:27:27.287 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:27:27.287 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:27:27.287 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:27:27.287 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 21.169386967000001, 2023-11-28T18:27:27.287 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 61, 2023-11-28T18:27:27.288 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 67, 2023-11-28T18:27:27.288 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 67, 2023-11-28T18:27:27.288 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 26.689161114000001 2023-11-28T18:27:27.288 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:27:27.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfcd501640 1 -- 172.21.15.105:0/3009327339 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbfa80668d0 msgr2=0x7fbfa8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:27.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfcd501640 1 --2- 172.21.15.105:0/3009327339 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbfa80668d0 0x7fbfa8068d90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00098f0 tx=0x7fbfb0006f90 comp rx=0 tx=0).stop 2023-11-28T18:27:27.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfcd501640 1 -- 172.21.15.105:0/3009327339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc8159610 msgr2=0x7fbfc81516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:27.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfcd501640 1 --2- 172.21.15.105:0/3009327339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc8159610 0x7fbfc81516e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000c780 tx=0x7fbfc000cc40 comp rx=0 tx=0).stop 2023-11-28T18:27:27.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfcd501640 1 -- 172.21.15.105:0/3009327339 shutdown_connections 2023-11-28T18:27:27.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfcd501640 1 --2- 172.21.15.105:0/3009327339 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbfa80668d0 0x7fbfa8068d90 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfcd501640 1 --2- 172.21.15.105:0/3009327339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc8159610 0x7fbfc81516e0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfcd501640 1 --2- 172.21.15.105:0/3009327339 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfc80cc760 0x7fbfc81511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfcd501640 1 --2- 172.21.15.105:0/3009327339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfc806c7e0 0x7fbfc80ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:27.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfcd501640 1 -- 172.21.15.105:0/3009327339 >> 172.21.15.105:0/3009327339 conn(0x7fbfc80ceb90 msgr2=0x7fbfc8156a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:27.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfcd501640 1 -- 172.21.15.105:0/3009327339 shutdown_connections 2023-11-28T18:27:27.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:27.275+0000 7fbfcd501640 1 -- 172.21.15.105:0/3009327339 wait complete. 2023-11-28T18:27:27.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:27:27.299 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-28T18:27:27.299 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:27:27.299 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p tree 2023-11-28T18:27:27.299 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:27:27.299 DEBUG:teuthology.orchestra.run.smithi105:> setfattr -n ceph.dir.pin.distributed -v 1 tree 2023-11-28T18:27:27.300 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:27:27.300 DEBUG:teuthology.orchestra.run.smithi105:> for ((i = 0; i < 100; i++)); do 2023-11-28T18:27:27.300 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p "tree/$i" 2023-11-28T18:27:27.300 DEBUG:teuthology.orchestra.run.smithi105:> echo file > "tree/$i/file" 2023-11-28T18:27:27.300 DEBUG:teuthology.orchestra.run.smithi105:> done 2023-11-28T18:27:27.300 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:27:27.312 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree 2023-11-28T18:27:27.315 INFO:teuthology.orchestra.run.smithi105.stderr:+ setfattr -n ceph.dir.pin.distributed -v 1 tree 2023-11-28T18:27:27.319 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i = 0 )) 2023-11-28T18:27:27.319 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.319 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/0 2023-11-28T18:27:27.323 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.329 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.329 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.329 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/1 2023-11-28T18:27:27.337 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.340 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.340 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.340 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/2 2023-11-28T18:27:27.347 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.349 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.349 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.349 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/3 2023-11-28T18:27:27.356 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.358 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.358 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.359 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/4 2023-11-28T18:27:27.364 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.367 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.367 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.367 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/5 2023-11-28T18:27:27.373 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.375 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.375 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.375 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/6 2023-11-28T18:27:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.383 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.383 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.384 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/7 2023-11-28T18:27:27.390 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/8 2023-11-28T18:27:27.398 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.401 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.401 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.401 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/9 2023-11-28T18:27:27.407 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.410 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.410 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.410 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/10 2023-11-28T18:27:27.416 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.418 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.418 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.419 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/11 2023-11-28T18:27:27.425 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/12 2023-11-28T18:27:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/13 2023-11-28T18:27:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.444 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.444 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/14 2023-11-28T18:27:27.449 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.451 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.451 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.452 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/15 2023-11-28T18:27:27.457 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/16 2023-11-28T18:27:27.466 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/17 2023-11-28T18:27:27.474 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.476 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.476 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.476 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/18 2023-11-28T18:27:27.482 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.484 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.485 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.485 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/19 2023-11-28T18:27:27.490 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.492 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.493 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.493 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/20 2023-11-28T18:27:27.498 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.501 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.501 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.501 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/21 2023-11-28T18:27:27.506 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.509 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.509 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.509 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/22 2023-11-28T18:27:27.514 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.517 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.517 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.517 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/23 2023-11-28T18:27:27.523 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.525 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.525 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.525 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/24 2023-11-28T18:27:27.531 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.533 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.534 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.534 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/25 2023-11-28T18:27:27.539 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.541 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.542 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.542 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/26 2023-11-28T18:27:27.547 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.550 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.550 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.550 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/27 2023-11-28T18:27:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.558 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.558 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.558 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/28 2023-11-28T18:27:27.564 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.566 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.566 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.566 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/29 2023-11-28T18:27:27.573 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/30 2023-11-28T18:27:27.582 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.584 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.584 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.584 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/31 2023-11-28T18:27:27.590 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.592 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.592 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.592 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/32 2023-11-28T18:27:27.598 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.601 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.601 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.601 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/33 2023-11-28T18:27:27.607 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.609 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.609 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.609 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/34 2023-11-28T18:27:27.615 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.617 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.618 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.618 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/35 2023-11-28T18:27:27.623 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/36 2023-11-28T18:27:27.631 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.634 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.634 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.634 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/37 2023-11-28T18:27:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/38 2023-11-28T18:27:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/39 2023-11-28T18:27:27.656 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.658 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.658 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.659 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/40 2023-11-28T18:27:27.665 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.667 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.667 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.667 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/41 2023-11-28T18:27:27.673 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.675 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.675 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.675 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/42 2023-11-28T18:27:27.680 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.682 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.683 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.683 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/43 2023-11-28T18:27:27.688 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.690 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.691 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.691 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/44 2023-11-28T18:27:27.697 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.699 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.699 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.699 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/45 2023-11-28T18:27:27.704 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.706 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.707 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.707 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/46 2023-11-28T18:27:27.712 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.714 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.715 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.715 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/47 2023-11-28T18:27:27.720 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.722 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.722 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.723 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/48 2023-11-28T18:27:27.728 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.731 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.731 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.731 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/49 2023-11-28T18:27:27.737 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.739 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.739 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.740 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/50 2023-11-28T18:27:27.745 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.747 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.747 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.748 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/51 2023-11-28T18:27:27.753 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.756 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.756 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.756 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/52 2023-11-28T18:27:27.762 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.764 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.764 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.764 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/53 2023-11-28T18:27:27.770 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.772 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.773 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.773 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/54 2023-11-28T18:27:27.778 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.780 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.781 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.781 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/55 2023-11-28T18:27:27.786 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.789 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.789 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.789 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/56 2023-11-28T18:27:27.794 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.797 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.797 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.797 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/57 2023-11-28T18:27:27.803 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.805 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.805 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.805 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/58 2023-11-28T18:27:27.811 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.813 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.813 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.813 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/59 2023-11-28T18:27:27.818 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.821 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.821 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.821 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/60 2023-11-28T18:27:27.826 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.828 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.829 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.829 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/61 2023-11-28T18:27:27.834 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.837 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.837 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.837 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/62 2023-11-28T18:27:27.843 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.845 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.845 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.845 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/63 2023-11-28T18:27:27.851 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.853 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.853 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.854 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/64 2023-11-28T18:27:27.860 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.862 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.862 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.862 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/65 2023-11-28T18:27:27.868 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.870 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.870 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.870 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/66 2023-11-28T18:27:27.876 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.878 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/67 2023-11-28T18:27:27.885 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.887 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.887 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.887 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/68 2023-11-28T18:27:27.893 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.895 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.895 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.895 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/69 2023-11-28T18:27:27.901 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.903 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.903 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.904 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/70 2023-11-28T18:27:27.909 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.912 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.912 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.912 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/71 2023-11-28T18:27:27.918 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.920 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.920 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.920 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/72 2023-11-28T18:27:27.925 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.928 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.928 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.928 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/73 2023-11-28T18:27:27.934 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.936 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.936 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.936 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/74 2023-11-28T18:27:27.942 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.944 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.944 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.944 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/75 2023-11-28T18:27:27.950 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.952 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.953 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.953 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/76 2023-11-28T18:27:27.959 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.961 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.961 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.961 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/77 2023-11-28T18:27:27.967 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.969 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.969 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.969 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/78 2023-11-28T18:27:27.975 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.977 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.977 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.978 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/79 2023-11-28T18:27:27.984 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.986 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.986 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.986 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/80 2023-11-28T18:27:27.992 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:27.995 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:27.995 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:27.995 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/81 2023-11-28T18:27:28.001 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:28.003 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:28.003 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:28.003 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/82 2023-11-28T18:27:28.009 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:28.011 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:28.011 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:28.011 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/83 2023-11-28T18:27:28.017 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:28.019 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:28.019 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:28.020 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/84 2023-11-28T18:27:28.026 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:28.028 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:28.028 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:28.028 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/85 2023-11-28T18:27:28.034 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:28.036 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:28.036 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:28.036 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/86 2023-11-28T18:27:28.042 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:28.044 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:28.044 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:28.045 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/87 2023-11-28T18:27:28.050 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:28.052 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:28.053 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:28.053 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/88 2023-11-28T18:27:28.059 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:28.061 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:28.061 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:28.061 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/89 2023-11-28T18:27:28.067 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:28.070 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:28.070 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:28.070 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/90 2023-11-28T18:27:28.076 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:28.078 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:28.079 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:28.079 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/91 2023-11-28T18:27:28.084 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:28.086 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:28.086 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:28.086 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/92 2023-11-28T18:27:28.093 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:28.095 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:28.096 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:28.096 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/93 2023-11-28T18:27:28.102 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:28.104 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:28.104 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:28.104 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/94 2023-11-28T18:27:28.110 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:28.113 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:28.113 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:28.113 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/95 2023-11-28T18:27:28.120 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:28.122 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:28.123 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:28.123 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/96 2023-11-28T18:27:28.129 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:28.131 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:28.131 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:28.131 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/97 2023-11-28T18:27:28.137 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:28.139 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:28.140 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:28.140 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/98 2023-11-28T18:27:28.145 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:28.148 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:28.148 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:28.148 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/99 2023-11-28T18:27:28.155 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:27:28.157 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:27:28.157 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:27:28.159 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-11-28T18:27:28.314 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-28T18:27:28.315 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:27:28.315 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-11-28T18:27:28.470 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-28T18:27:28.470 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:27:28.471 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-11-28T18:27:28.626 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"4344","version":"4342","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1387","version":"1387","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2023-11-28T18:27:28.626 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "4344", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "4342" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "1387", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1387" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:27:28.627 INFO:tasks.cephfs.cephfs_test_case:len=0 [] 2023-11-28T18:27:31.841 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:27:31.845 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:27:33.628 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-11-28T18:27:33.793 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-28T18:27:33.794 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:27:33.794 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-11-28T18:27:33.962 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-28T18:27:33.962 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:27:33.962 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-11-28T18:27:34.119 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"4351","version":"4347","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1387","version":"1387","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2023-11-28T18:27:34.120 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 2, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "4351", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "4347" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "1387", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1387" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:27:34.120 INFO:tasks.cephfs.cephfs_test_case:len=0 [] 2023-11-28T18:27:39.121 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-11-28T18:27:39.285 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x100000004a6.101*","snapid_first":2,"projected_version":"422","version":"422","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"4353","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":3},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x100000004a6.010*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x100000004a6.110*","snapid_first":2,"projected_version":"442","version":"442","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-28T18:27:39.286 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x100000004a6.101*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "422", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "422" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "snapid_first": 2, "states": [], "version": "4353" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x100000004a6.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "426", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x100000004a6.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "442", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "442" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:27:39.286 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-11-28T18:27:39.452 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"4353","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":2},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x100000004a6.111*","snapid_first":2,"projected_version":"416","version":"416","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x100000004a6.001*","snapid_first":2,"projected_version":"428","version":"428","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-11-28T18:27:39.452 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "snapid_first": 2, "states": [], "version": "4353" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x100000004a6.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "416", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "416" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x100000004a6.001*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "428", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "428" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:27:39.453 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-11-28T18:27:39.607 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"4353","version":"4353","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":2,"2":3}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1387","version":"1387","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x100000004a6.111*","snapid_first":2,"projected_version":"416","version":"416","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x100000004a6.101*","snapid_first":2,"projected_version":"422","version":"422","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x100000004a6.100*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x100000004a6.110*","snapid_first":2,"projected_version":"442","version":"442","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x100000004a6.010*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x100000004a6.001*","snapid_first":2,"projected_version":"428","version":"428","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x100000004a6.011*","snapid_first":2,"projected_version":"430","version":"430","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x100000004a6.000*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2023-11-28T18:27:39.608 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 2, "2": 3 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "4353", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "4353" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "1387", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1387" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x100000004a6.111*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "416", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "416" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x100000004a6.101*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "422", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "422" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x100000004a6.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x100000004a6.110*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "442", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "442" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x100000004a6.010*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x100000004a6.001*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "428", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "428" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x100000004a6.011*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "430", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "430" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x100000004a6.000*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "426", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:27:39.608 INFO:tasks.cephfs.cephfs_test_case:len=8 [{'is_auth': True, 'auth_first': 2, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 2, 'dir': {'path': '/tree', 'dirfrag': '0x100000004a6.101*', 'snapid_first': 2, 'projected_version': '422', 'version': '422', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '2', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {'0': 1}}, 'replica_state': {'authority': [2, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'ptrwaiter': 0, 'child': 1, 'frozen': 0, 'subtree': 1, 'importing': 0, 'replicated': 1, 'dirty': 1}, 'nref': 4}}, {'is_auth': True, 'auth_first': 2, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 2, 'dir': {'path': '/tree', 'dirfrag': '0x100000004a6.010*', 'snapid_first': 2, 'projected_version': '426', 'version': '426', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '2', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {'0': 1}}, 'replica_state': {'authority': [2, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'ptrwaiter': 0, 'child': 1, 'frozen': 0, 'subtree': 1, 'importing': 0, 'replicated': 1, 'dirty': 1}, 'nref': 4}}, {'is_auth': True, 'auth_first': 2, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 2, 'dir': {'path': '/tree', 'dirfrag': '0x100000004a6.110*', 'snapid_first': 2, 'projected_version': '442', 'version': '442', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '2', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {'0': 1}}, 'replica_state': {'authority': [2, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'ptrwaiter': 0, 'child': 1, 'frozen': 0, 'subtree': 1, 'importing': 0, 'replicated': 1, 'dirty': 1}, 'nref': 4}}, {'is_auth': True, 'auth_first': 1, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 1, 'dir': {'path': '/tree', 'dirfrag': '0x100000004a6.111*', 'snapid_first': 2, 'projected_version': '416', 'version': '416', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '1', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {'0': 1}}, 'replica_state': {'authority': [1, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'ptrwaiter': 0, 'child': 1, 'frozen': 0, 'subtree': 1, 'importing': 0, 'replicated': 1, 'dirty': 1}, 'nref': 4}}, {'is_auth': True, 'auth_first': 1, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 1, 'dir': {'path': '/tree', 'dirfrag': '0x100000004a6.001*', 'snapid_first': 2, 'projected_version': '428', 'version': '428', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '1', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {'0': 1}}, 'replica_state': {'authority': [1, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'ptrwaiter': 0, 'child': 1, 'frozen': 0, 'subtree': 1, 'importing': 0, 'replicated': 1, 'dirty': 1}, 'nref': 4}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x100000004a6.100*', 'snapid_first': 2, 'projected_version': '426', 'version': '426', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'waiter': 0, 'authpin': 0}, 'nref': 3}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x100000004a6.011*', 'snapid_first': 2, 'projected_version': '430', 'version': '430', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x100000004a6.000*', 'snapid_first': 2, 'projected_version': '426', 'version': '426', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}] 2023-11-28T18:27:39.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:27:39.609 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin.distributed -v 0 tree) 2023-11-28T18:27:54.622 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-11-28T18:27:54.788 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"4355","version":"4355","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":2,"2":3}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1387","version":"1387","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree","dirfrag":"0x100000004a6.111*","snapid_first":2,"projected_version":"416","version":"416","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree","dirfrag":"0x100000004a6.101*","snapid_first":2,"projected_version":"422","version":"422","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree","dirfrag":"0x100000004a6.110*","snapid_first":2,"projected_version":"442","version":"442","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree","dirfrag":"0x100000004a6.010*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree","dirfrag":"0x100000004a6.001*","snapid_first":2,"projected_version":"428","version":"428","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2023-11-28T18:27:54.789 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 2, "2": 3 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "4355", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "4355" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "1387", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1387" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x100000004a6.111*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "416", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "416" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x100000004a6.101*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "422", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "422" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x100000004a6.110*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "442", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "442" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x100000004a6.010*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x100000004a6.001*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "428", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "428" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2023-11-28T18:27:54.789 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T18:27:54.790 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:27:54.836 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:27:54.885 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T18:27:54.886 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:27:54.886 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:27:54.895 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:27:55.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.066+0000 7f5f37180640 1 Processor -- start 2023-11-28T18:27:55.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.066+0000 7f5f37180640 1 -- start start 2023-11-28T18:27:55.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.070+0000 7f5f37180640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5f30071960 0x7f5f3015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:55.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.070+0000 7f5f37180640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f3015b030 0x7f5f300709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:55.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.070+0000 7f5f37180640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f300701d0 con 0x7f5f300b4290 2023-11-28T18:27:55.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.070+0000 7f5f37180640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3006df80 con 0x7f5f30071960 2023-11-28T18:27:55.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.070+0000 7f5f37180640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f30071df0 con 0x7f5f3015b030 2023-11-28T18:27:55.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.070+0000 7f5f3571d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f3015b030 0x7f5f300709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:55.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.070+0000 7f5f27fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5f30071960 0x7f5f3015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:55.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.070+0000 7f5f3571d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f3015b030 0x7f5f300709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:50596/0 (socket says 172.21.15.105:50596) 2023-11-28T18:27:55.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.070+0000 7f5f3571d640 1 -- 172.21.15.105:0/1122424351 learned_addr learned my addr 172.21.15.105:0/1122424351 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:55.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.070+0000 7f5f34f1c640 1 --1- 172.21.15.105:0/1122424351 >> v1:172.21.15.148:6789/0 conn(0x7f5f300b4290 0x7f5f300b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:43980/0 (socket says 172.21.15.105:43980) 2023-11-28T18:27:55.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.070+0000 7f5f3571d640 1 -- 172.21.15.105:0/1122424351 >> v2:172.21.15.148:3300/0 conn(0x7f5f30071960 msgr2=0x7f5f3015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:55.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.070+0000 7f5f3571d640 1 --2- 172.21.15.105:0/1122424351 >> v2:172.21.15.148:3300/0 conn(0x7f5f30071960 0x7f5f3015f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.070+0000 7f5f3571d640 1 -- 172.21.15.105:0/1122424351 >> v1:172.21.15.148:6789/0 conn(0x7f5f300b4290 legacy=0x7f5f300b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:55.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.070+0000 7f5f3571d640 1 -- 172.21.15.105:0/1122424351 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f300720d0 con 0x7f5f3015b030 2023-11-28T18:27:55.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.074+0000 7f5f3571d640 1 --2- 172.21.15.105:0/1122424351 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f3015b030 0x7f5f300709b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009e50 tx=0x7f5f2801c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7947ee36ddb2c58c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:55.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.074+0000 7f5f277fe640 1 -- 172.21.15.105:0/1122424351 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f280a57c0 con 0x7f5f3015b030 2023-11-28T18:27:55.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.074+0000 7f5f277fe640 1 -- 172.21.15.105:0/1122424351 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f280a5920 con 0x7f5f3015b030 2023-11-28T18:27:55.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.074+0000 7f5f277fe640 1 -- 172.21.15.105:0/1122424351 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f280a5cb0 con 0x7f5f3015b030 2023-11-28T18:27:55.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.074+0000 7f5f37180640 1 -- 172.21.15.105:0/1122424351 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f3015b030 msgr2=0x7f5f300709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:55.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.074+0000 7f5f37180640 1 --2- 172.21.15.105:0/1122424351 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f3015b030 0x7f5f300709b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009e50 tx=0x7f5f2801c9e0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.074+0000 7f5f37180640 1 -- 172.21.15.105:0/1122424351 shutdown_connections 2023-11-28T18:27:55.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.074+0000 7f5f37180640 1 --2- 172.21.15.105:0/1122424351 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f3015b030 0x7f5f300709b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.074+0000 7f5f37180640 1 --2- 172.21.15.105:0/1122424351 >> v2:172.21.15.148:3300/0 conn(0x7f5f30071960 0x7f5f3015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.074+0000 7f5f37180640 1 -- 172.21.15.105:0/1122424351 >> 172.21.15.105:0/1122424351 conn(0x7f5f300ceb90 msgr2=0x7f5f300cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:55.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.074+0000 7f5f37180640 1 -- 172.21.15.105:0/1122424351 shutdown_connections 2023-11-28T18:27:55.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.074+0000 7f5f37180640 1 -- 172.21.15.105:0/1122424351 wait complete. 2023-11-28T18:27:55.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f37180640 1 Processor -- start 2023-11-28T18:27:55.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f37180640 1 -- start start 2023-11-28T18:27:55.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f37180640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f300b4290 0x7f5f30140f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:55.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f37180640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f3015b030 0x7f5f301550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:55.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f37180640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f301437c0 0x7f5f30143c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:55.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f37180640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f30144190 con 0x7f5f300b4290 2023-11-28T18:27:55.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f37180640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f300af090 con 0x7f5f3015b030 2023-11-28T18:27:55.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f37180640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f30141480 con 0x7f5f301437c0 2023-11-28T18:27:55.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f34f1c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f300b4290 0x7f5f30140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:55.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f27fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f3015b030 0x7f5f301550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:55.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f34f1c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f300b4290 0x7f5f30140f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:32876/0 (socket says 172.21.15.105:32876) 2023-11-28T18:27:55.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f3571d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f301437c0 0x7f5f30143c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:55.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f34f1c640 1 -- 172.21.15.105:0/1714618713 learned_addr learned my addr 172.21.15.105:0/1714618713 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:55.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f34f1c640 1 -- 172.21.15.105:0/1714618713 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f301437c0 msgr2=0x7f5f30143c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:55.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f34f1c640 1 --2- 172.21.15.105:0/1714618713 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f301437c0 0x7f5f30143c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f34f1c640 1 -- 172.21.15.105:0/1714618713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f3015b030 msgr2=0x7f5f301550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:55.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f34f1c640 1 --2- 172.21.15.105:0/1714618713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f3015b030 0x7f5f301550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f34f1c640 1 -- 172.21.15.105:0/1714618713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f28009b30 con 0x7f5f300b4290 2023-11-28T18:27:55.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f3571d640 1 --2- 172.21.15.105:0/1714618713 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f301437c0 0x7f5f30143c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:27:55.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f27fff640 1 --2- 172.21.15.105:0/1714618713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f3015b030 0x7f5f301550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:27:55.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f34f1c640 1 --2- 172.21.15.105:0/1714618713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f300b4290 0x7f5f30140f40 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00c4c0 tx=0x7f5f2c00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:55.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f257fa640 1 -- 172.21.15.105:0/1714618713 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c007480 con 0x7f5f300b4290 2023-11-28T18:27:55.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f257fa640 1 -- 172.21.15.105:0/1714618713 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2c0075e0 con 0x7f5f300b4290 2023-11-28T18:27:55.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f257fa640 1 -- 172.21.15.105:0/1714618713 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c004a10 con 0x7f5f300b4290 2023-11-28T18:27:55.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f37180640 1 -- 172.21.15.105:0/1714618713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f301555f0 con 0x7f5f300b4290 2023-11-28T18:27:55.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f37180640 1 -- 172.21.15.105:0/1714618713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f300ae760 con 0x7f5f300b4290 2023-11-28T18:27:55.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.078+0000 7f5f37180640 1 -- 172.21.15.105:0/1714618713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f30067570 con 0x7f5f300b4290 2023-11-28T18:27:55.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.082+0000 7f5f257fa640 1 -- 172.21.15.105:0/1714618713 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5f2c004b70 con 0x7f5f300b4290 2023-11-28T18:27:55.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.082+0000 7f5f257fa640 1 --2- 172.21.15.105:0/1714618713 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f1c0668d0 0x7f5f1c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:55.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.082+0000 7f5f257fa640 1 -- 172.21.15.105:0/1714618713 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f5f2c083750 con 0x7f5f300b4290 2023-11-28T18:27:55.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.082+0000 7f5f27fff640 1 --2- 172.21.15.105:0/1714618713 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f1c0668d0 0x7f5f1c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:55.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.082+0000 7f5f27fff640 1 --2- 172.21.15.105:0/1714618713 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f1c0668d0 0x7f5f1c068d90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5f3015eb50 tx=0x7f5f20009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:55.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.086+0000 7f5f257fa640 1 -- 172.21.15.105:0/1714618713 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5f2c050860 con 0x7f5f300b4290 2023-11-28T18:27:55.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.238+0000 7f5f37180640 1 -- 172.21.15.105:0/1714618713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f301582f0 con 0x7f5f300b4290 2023-11-28T18:27:55.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.242+0000 7f5f257fa640 1 -- 172.21.15.105:0/1714618713 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+17014 (secure 0 0 0) 0x7f5f2c053e80 con 0x7f5f300b4290 2023-11-28T18:27:55.246 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:27:55.246 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":67,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:27:16.173737+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:27:03.225100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T18:27:04.478619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.e","mappings":[{"from":7,"to":2}]},{"pgid":"10.c","mappings":[{"from":7,"to":3}]},{"pgid":"10.d","mappings":[{"from":4,"to":3}]},{"pgid":"10.19","mappings":[{"from":4,"to":5}]},{"pgid":"10.1d","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:27:55.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.242+0000 7f5f37180640 1 -- 172.21.15.105:0/1714618713 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f1c0668d0 msgr2=0x7f5f1c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:55.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.242+0000 7f5f37180640 1 --2- 172.21.15.105:0/1714618713 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f1c0668d0 0x7f5f1c068d90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5f3015eb50 tx=0x7f5f20009190 comp rx=0 tx=0).stop 2023-11-28T18:27:55.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.242+0000 7f5f37180640 1 -- 172.21.15.105:0/1714618713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f300b4290 msgr2=0x7f5f30140f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:55.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.242+0000 7f5f37180640 1 --2- 172.21.15.105:0/1714618713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f300b4290 0x7f5f30140f40 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00c4c0 tx=0x7f5f2c00c980 comp rx=0 tx=0).stop 2023-11-28T18:27:55.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.242+0000 7f5f37180640 1 -- 172.21.15.105:0/1714618713 shutdown_connections 2023-11-28T18:27:55.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.242+0000 7f5f37180640 1 --2- 172.21.15.105:0/1714618713 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f301437c0 0x7f5f30143c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.242+0000 7f5f37180640 1 --2- 172.21.15.105:0/1714618713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f3015b030 0x7f5f301550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.242+0000 7f5f37180640 1 --2- 172.21.15.105:0/1714618713 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f1c0668d0 0x7f5f1c068d90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.242+0000 7f5f37180640 1 --2- 172.21.15.105:0/1714618713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f300b4290 0x7f5f30140f40 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.242+0000 7f5f37180640 1 -- 172.21.15.105:0/1714618713 >> 172.21.15.105:0/1714618713 conn(0x7f5f300ceb90 msgr2=0x7f5f300b4670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:55.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.242+0000 7f5f37180640 1 -- 172.21.15.105:0/1714618713 shutdown_connections 2023-11-28T18:27:55.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.242+0000 7f5f37180640 1 -- 172.21.15.105:0/1714618713 wait complete. 2023-11-28T18:27:55.267 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:27:03.225100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:27:04.478619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:27:55.267 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:27:55.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.402+0000 7f433ec17640 1 Processor -- start 2023-11-28T18:27:55.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.402+0000 7f433ec17640 1 -- start start 2023-11-28T18:27:55.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.406+0000 7f433ec17640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4338159f40 0x7f43380caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:55.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.406+0000 7f433ec17640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f43380cc760 0x7f43380ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:55.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.406+0000 7f433ec17640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f433806bf50 con 0x7f43380ad680 2023-11-28T18:27:55.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.406+0000 7f433ec17640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43380ccbc0 con 0x7f43380cc760 2023-11-28T18:27:55.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.406+0000 7f433ec17640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433815a320 con 0x7f4338159f40 2023-11-28T18:27:55.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.406+0000 7f4337fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f43380cc760 0x7f43380ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:55.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.406+0000 7f433d1b4640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f43380ad680 0x7f433815da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46220/0 (socket says 172.21.15.105:46220) 2023-11-28T18:27:55.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.406+0000 7f433d1b4640 1 -- 172.21.15.105:0/1819364166 learned_addr learned my addr 172.21.15.105:0/1819364166 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:55.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.406+0000 7f433c9b3640 1 --2- 172.21.15.105:0/1819364166 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4338159f40 0x7f43380caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:55.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.406+0000 7f4337fff640 1 -- 172.21.15.105:0/1819364166 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4338159f40 msgr2=0x7f43380caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:55.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.406+0000 7f4337fff640 1 --2- 172.21.15.105:0/1819364166 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4338159f40 0x7f43380caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.406+0000 7f4337fff640 1 -- 172.21.15.105:0/1819364166 >> v1:172.21.15.105:6789/0 conn(0x7f43380ad680 legacy=0x7f433815da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:55.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.406+0000 7f4337fff640 1 -- 172.21.15.105:0/1819364166 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4338071380 con 0x7f43380cc760 2023-11-28T18:27:55.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.406+0000 7f4337fff640 1 --2- 172.21.15.105:0/1819364166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43380cc760 0x7f43380ad140 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f43240097c0 tx=0x7f432401e5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85d75dfde59212d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:55.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.406+0000 7f43377fe640 1 -- 172.21.15.105:0/1819364166 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43240b2600 con 0x7f43380cc760 2023-11-28T18:27:55.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.410+0000 7f43377fe640 1 -- 172.21.15.105:0/1819364166 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43240b2760 con 0x7f43380cc760 2023-11-28T18:27:55.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.410+0000 7f43377fe640 1 -- 172.21.15.105:0/1819364166 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43240b2af0 con 0x7f43380cc760 2023-11-28T18:27:55.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.410+0000 7f433ec17640 1 -- 172.21.15.105:0/1819364166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43380cc760 msgr2=0x7f43380ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:55.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.410+0000 7f433ec17640 1 --2- 172.21.15.105:0/1819364166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43380cc760 0x7f43380ad140 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f43240097c0 tx=0x7f432401e5d0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.410+0000 7f433ec17640 1 -- 172.21.15.105:0/1819364166 shutdown_connections 2023-11-28T18:27:55.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.410+0000 7f433ec17640 1 --2- 172.21.15.105:0/1819364166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43380cc760 0x7f43380ad140 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.410+0000 7f433ec17640 1 --2- 172.21.15.105:0/1819364166 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4338159f40 0x7f43380caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.410+0000 7f433ec17640 1 -- 172.21.15.105:0/1819364166 >> 172.21.15.105:0/1819364166 conn(0x7f43380ceb90 msgr2=0x7f43380cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:55.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.410+0000 7f433ec17640 1 -- 172.21.15.105:0/1819364166 shutdown_connections 2023-11-28T18:27:55.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.410+0000 7f433ec17640 1 -- 172.21.15.105:0/1819364166 wait complete. 2023-11-28T18:27:55.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.410+0000 7f433ec17640 1 Processor -- start 2023-11-28T18:27:55.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.410+0000 7f433ec17640 1 -- start start 2023-11-28T18:27:55.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.410+0000 7f433ec17640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43380ad680 0x7f4338148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:55.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.410+0000 7f433ec17640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43380cc760 0x7f4338099e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:55.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.410+0000 7f433ec17640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4338159f40 0x7f433809a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:55.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.410+0000 7f433ec17640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433815c040 con 0x7f43380ad680 2023-11-28T18:27:55.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.410+0000 7f433ec17640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4338148d90 con 0x7f43380cc760 2023-11-28T18:27:55.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.410+0000 7f433ec17640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433815c460 con 0x7f4338159f40 2023-11-28T18:27:55.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.410+0000 7f433c9b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43380ad680 0x7f4338148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:55.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.410+0000 7f433c9b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43380ad680 0x7f4338148850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:32892/0 (socket says 172.21.15.105:32892) 2023-11-28T18:27:55.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.410+0000 7f433c9b3640 1 -- 172.21.15.105:0/1938488546 learned_addr learned my addr 172.21.15.105:0/1938488546 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:55.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.414+0000 7f433c9b3640 1 -- 172.21.15.105:0/1938488546 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4338159f40 msgr2=0x7f433809a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:55.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.414+0000 7f433c9b3640 1 --2- 172.21.15.105:0/1938488546 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4338159f40 0x7f433809a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.414+0000 7f433c9b3640 1 -- 172.21.15.105:0/1938488546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43380cc760 msgr2=0x7f4338099e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:55.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.414+0000 7f433c9b3640 1 --2- 172.21.15.105:0/1938488546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43380cc760 0x7f4338099e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.414+0000 7f433c9b3640 1 -- 172.21.15.105:0/1938488546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4324009420 con 0x7f43380ad680 2023-11-28T18:27:55.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.414+0000 7f433c9b3640 1 --2- 172.21.15.105:0/1938488546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43380ad680 0x7f4338148850 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f432c00d5f0 tx=0x7f432c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:55.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.414+0000 7f43357fa640 1 -- 172.21.15.105:0/1938488546 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c0125f0 con 0x7f43380ad680 2023-11-28T18:27:55.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.414+0000 7f43357fa640 1 -- 172.21.15.105:0/1938488546 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432c012750 con 0x7f43380ad680 2023-11-28T18:27:55.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.414+0000 7f43357fa640 1 -- 172.21.15.105:0/1938488546 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c012b00 con 0x7f43380ad680 2023-11-28T18:27:55.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.414+0000 7f433ec17640 1 -- 172.21.15.105:0/1938488546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f433815b3c0 con 0x7f43380ad680 2023-11-28T18:27:55.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.414+0000 7f433ec17640 1 -- 172.21.15.105:0/1938488546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f433815b980 con 0x7f43380ad680 2023-11-28T18:27:55.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.414+0000 7f433ec17640 1 -- 172.21.15.105:0/1938488546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4338067570 con 0x7f43380ad680 2023-11-28T18:27:55.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.414+0000 7f43357fa640 1 -- 172.21.15.105:0/1938488546 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f432c012c60 con 0x7f43380ad680 2023-11-28T18:27:55.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.414+0000 7f43357fa640 1 --2- 172.21.15.105:0/1938488546 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f431c0668d0 0x7f431c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:55.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.414+0000 7f43357fa640 1 -- 172.21.15.105:0/1938488546 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f432c084ad0 con 0x7f43380ad680 2023-11-28T18:27:55.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.414+0000 7f4337fff640 1 --2- 172.21.15.105:0/1938488546 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f431c0668d0 0x7f431c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:55.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.418+0000 7f4337fff640 1 --2- 172.21.15.105:0/1938488546 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f431c0668d0 0x7f431c068d90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f43240b1000 tx=0x7f4324004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:55.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.422+0000 7f43357fa640 1 -- 172.21.15.105:0/1938488546 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f432c051b40 con 0x7f43380ad680 2023-11-28T18:27:55.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.570+0000 7f433ec17640 1 -- 172.21.15.105:0/1938488546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f43380cf950 con 0x7f43380ad680 2023-11-28T18:27:55.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.570+0000 7f43357fa640 1 -- 172.21.15.105:0/1938488546 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) v1 ==== 93+0+10 (secure 0 0 0) 0x7f432c007380 con 0x7f43380ad680 2023-11-28T18:27:55.576 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:27:55.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.574+0000 7f433ec17640 1 -- 172.21.15.105:0/1938488546 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f431c0668d0 msgr2=0x7f431c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:55.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.574+0000 7f433ec17640 1 --2- 172.21.15.105:0/1938488546 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f431c0668d0 0x7f431c068d90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f43240b1000 tx=0x7f4324004740 comp rx=0 tx=0).stop 2023-11-28T18:27:55.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.574+0000 7f433ec17640 1 -- 172.21.15.105:0/1938488546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43380ad680 msgr2=0x7f4338148850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:55.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.574+0000 7f433ec17640 1 --2- 172.21.15.105:0/1938488546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43380ad680 0x7f4338148850 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f432c00d5f0 tx=0x7f432c00dab0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.574+0000 7f433ec17640 1 -- 172.21.15.105:0/1938488546 shutdown_connections 2023-11-28T18:27:55.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.574+0000 7f433ec17640 1 --2- 172.21.15.105:0/1938488546 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4338159f40 0x7f433809a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.574+0000 7f433ec17640 1 --2- 172.21.15.105:0/1938488546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43380cc760 0x7f4338099e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.574+0000 7f433ec17640 1 --2- 172.21.15.105:0/1938488546 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f431c0668d0 0x7f431c068d90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.574+0000 7f433ec17640 1 --2- 172.21.15.105:0/1938488546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43380ad680 0x7f4338148850 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.574+0000 7f433ec17640 1 -- 172.21.15.105:0/1938488546 >> 172.21.15.105:0/1938488546 conn(0x7f43380ceb90 msgr2=0x7f43381534c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:55.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.574+0000 7f433ec17640 1 -- 172.21.15.105:0/1938488546 shutdown_connections 2023-11-28T18:27:55.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.574+0000 7f433ec17640 1 -- 172.21.15.105:0/1938488546 wait complete. 2023-11-28T18:27:55.597 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:27:55.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.730+0000 7ff0ae5aa640 1 Processor -- start 2023-11-28T18:27:55.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.730+0000 7ff0ae5aa640 1 -- start start 2023-11-28T18:27:55.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.734+0000 7ff0ae5aa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0a8159610 0x7ff0a80ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:55.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.734+0000 7ff0ae5aa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff0a80cc760 0x7ff0a80ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:55.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.734+0000 7ff0ae5aa640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a80cde30 con 0x7ff0a806c7e0 2023-11-28T18:27:55.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.734+0000 7ff0ae5aa640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a81599f0 con 0x7ff0a80cc760 2023-11-28T18:27:55.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.734+0000 7ff0ae5aa640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a806de20 con 0x7ff0a8159610 2023-11-28T18:27:55.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.734+0000 7ff0a77fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff0a80cc760 0x7ff0a80ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:55.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.734+0000 7ff0a77fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff0a80cc760 0x7ff0a80ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:32896/0 (socket says 172.21.15.105:32896) 2023-11-28T18:27:55.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.734+0000 7ff0a77fe640 1 -- 172.21.15.105:0/2578189522 learned_addr learned my addr 172.21.15.105:0/2578189522 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:55.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.734+0000 7ff0a77fe640 1 -- 172.21.15.105:0/2578189522 >> v2:172.21.15.148:3300/0 conn(0x7ff0a8159610 msgr2=0x7ff0a80ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:55.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.734+0000 7ff0a77fe640 1 --2- 172.21.15.105:0/2578189522 >> v2:172.21.15.148:3300/0 conn(0x7ff0a8159610 0x7ff0a80ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.734+0000 7ff0a77fe640 1 -- 172.21.15.105:0/2578189522 >> v1:172.21.15.148:6789/0 conn(0x7ff0a806c7e0 legacy=0x7ff0a80abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:55.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.734+0000 7ff0a77fe640 1 -- 172.21.15.105:0/2578189522 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a80cd220 con 0x7ff0a80cc760 2023-11-28T18:27:55.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.738+0000 7ff0a77fe640 1 --2- 172.21.15.105:0/2578189522 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0a80cc760 0x7ff0a80ccbf0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7ff0a80cc6b0 tx=0x7ff0a80acca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f1c3a75902a4b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:55.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.738+0000 7ff0a6ffd640 1 -- 172.21.15.105:0/2578189522 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c0aa5f0 con 0x7ff0a80cc760 2023-11-28T18:27:55.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.738+0000 7ff0a6ffd640 1 -- 172.21.15.105:0/2578189522 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09c0aa750 con 0x7ff0a80cc760 2023-11-28T18:27:55.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.738+0000 7ff0a6ffd640 1 -- 172.21.15.105:0/2578189522 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c0aaae0 con 0x7ff0a80cc760 2023-11-28T18:27:55.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.738+0000 7ff0ae5aa640 1 -- 172.21.15.105:0/2578189522 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0a80cc760 msgr2=0x7ff0a80ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:55.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.738+0000 7ff0ae5aa640 1 --2- 172.21.15.105:0/2578189522 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0a80cc760 0x7ff0a80ccbf0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7ff0a80cc6b0 tx=0x7ff0a80acca0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.738+0000 7ff0ae5aa640 1 -- 172.21.15.105:0/2578189522 shutdown_connections 2023-11-28T18:27:55.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.738+0000 7ff0ae5aa640 1 --2- 172.21.15.105:0/2578189522 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0a80cc760 0x7ff0a80ccbf0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.738+0000 7ff0ae5aa640 1 --2- 172.21.15.105:0/2578189522 >> v2:172.21.15.148:3300/0 conn(0x7ff0a8159610 0x7ff0a80ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.738+0000 7ff0ae5aa640 1 -- 172.21.15.105:0/2578189522 >> 172.21.15.105:0/2578189522 conn(0x7ff0a80ceb90 msgr2=0x7ff0a80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:55.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.738+0000 7ff0ae5aa640 1 -- 172.21.15.105:0/2578189522 shutdown_connections 2023-11-28T18:27:55.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.738+0000 7ff0ae5aa640 1 -- 172.21.15.105:0/2578189522 wait complete. 2023-11-28T18:27:55.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.738+0000 7ff0ae5aa640 1 Processor -- start 2023-11-28T18:27:55.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.738+0000 7ff0ae5aa640 1 -- start start 2023-11-28T18:27:55.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0ae5aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a806c7e0 0x7ff0a809a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:55.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0ae5aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0a80cc760 0x7ff0a8099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:55.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0ae5aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0a8159610 0x7ff0a8152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:55.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0ae5aa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a8154e60 con 0x7ff0a80cc760 2023-11-28T18:27:55.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0ae5aa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a8154270 con 0x7ff0a806c7e0 2023-11-28T18:27:55.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0ae5aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a8099f90 con 0x7ff0a8159610 2023-11-28T18:27:55.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0a77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0a80cc760 0x7ff0a8099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:55.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0a77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0a80cc760 0x7ff0a8099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:32908/0 (socket says 172.21.15.105:32908) 2023-11-28T18:27:55.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0a77fe640 1 -- 172.21.15.105:0/4020506436 learned_addr learned my addr 172.21.15.105:0/4020506436 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:55.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0a7fff640 1 --2- 172.21.15.105:0/4020506436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a806c7e0 0x7ff0a809a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:55.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0acb47640 1 --2- 172.21.15.105:0/4020506436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0a8159610 0x7ff0a8152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:55.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0a77fe640 1 -- 172.21.15.105:0/4020506436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0a8159610 msgr2=0x7ff0a8152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:55.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0a77fe640 1 --2- 172.21.15.105:0/4020506436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0a8159610 0x7ff0a8152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0a77fe640 1 -- 172.21.15.105:0/4020506436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a806c7e0 msgr2=0x7ff0a809a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:55.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0a77fe640 1 --2- 172.21.15.105:0/4020506436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a806c7e0 0x7ff0a809a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0a77fe640 1 -- 172.21.15.105:0/4020506436 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c0094d0 con 0x7ff0a80cc760 2023-11-28T18:27:55.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0a7fff640 1 --2- 172.21.15.105:0/4020506436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a806c7e0 0x7ff0a809a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:55.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0a77fe640 1 --2- 172.21.15.105:0/4020506436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0a80cc760 0x7ff0a8099a50 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff09c01a0c0 tx=0x7ff09c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:55.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0a4ff9640 1 -- 172.21.15.105:0/4020506436 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c0b3800 con 0x7ff0a80cc760 2023-11-28T18:27:55.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0a4ff9640 1 -- 172.21.15.105:0/4020506436 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09c0b3960 con 0x7ff0a80cc760 2023-11-28T18:27:55.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0ae5aa640 1 -- 172.21.15.105:0/4020506436 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a81544c0 con 0x7ff0a80cc760 2023-11-28T18:27:55.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0a4ff9640 1 -- 172.21.15.105:0/4020506436 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c0b3d10 con 0x7ff0a80cc760 2023-11-28T18:27:55.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0ae5aa640 1 -- 172.21.15.105:0/4020506436 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a8154a80 con 0x7ff0a80cc760 2023-11-28T18:27:55.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0ae5aa640 1 -- 172.21.15.105:0/4020506436 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a8154fa0 con 0x7ff0a80cc760 2023-11-28T18:27:55.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.742+0000 7ff0a4ff9640 1 -- 172.21.15.105:0/4020506436 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff09c0b2380 con 0x7ff0a80cc760 2023-11-28T18:27:55.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.746+0000 7ff0a4ff9640 1 --2- 172.21.15.105:0/4020506436 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff0940668d0 0x7ff094068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:55.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.746+0000 7ff0a4ff9640 1 -- 172.21.15.105:0/4020506436 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7ff09c0a7070 con 0x7ff0a80cc760 2023-11-28T18:27:55.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.746+0000 7ff0a7fff640 1 --2- 172.21.15.105:0/4020506436 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff0940668d0 0x7ff094068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:55.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.746+0000 7ff0a7fff640 1 --2- 172.21.15.105:0/4020506436 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff0940668d0 0x7ff094068d90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff098006fd0 tx=0x7ff098008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:55.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.750+0000 7ff0a4ff9640 1 -- 172.21.15.105:0/4020506436 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff0a8154fa0 con 0x7ff0a80cc760 2023-11-28T18:27:55.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.898+0000 7ff0ae5aa640 1 -- 172.21.15.105:0/4020506436 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff0a8154fa0 con 0x7ff0a80cc760 2023-11-28T18:27:55.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.902+0000 7ff0a4ff9640 1 -- 172.21.15.105:0/4020506436 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v67) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff0a8154fa0 con 0x7ff0a80cc760 2023-11-28T18:27:55.906 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:27:55.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.902+0000 7ff0ae5aa640 1 -- 172.21.15.105:0/4020506436 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff0940668d0 msgr2=0x7ff094068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:55.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.902+0000 7ff0ae5aa640 1 --2- 172.21.15.105:0/4020506436 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff0940668d0 0x7ff094068d90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff098006fd0 tx=0x7ff098008040 comp rx=0 tx=0).stop 2023-11-28T18:27:55.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.902+0000 7ff0ae5aa640 1 -- 172.21.15.105:0/4020506436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0a80cc760 msgr2=0x7ff0a8099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:55.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.902+0000 7ff0ae5aa640 1 --2- 172.21.15.105:0/4020506436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0a80cc760 0x7ff0a8099a50 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff09c01a0c0 tx=0x7ff09c01acd0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.902+0000 7ff0ae5aa640 1 -- 172.21.15.105:0/4020506436 shutdown_connections 2023-11-28T18:27:55.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.902+0000 7ff0ae5aa640 1 --2- 172.21.15.105:0/4020506436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0a8159610 0x7ff0a8152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.902+0000 7ff0ae5aa640 1 --2- 172.21.15.105:0/4020506436 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff0940668d0 0x7ff094068d90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.902+0000 7ff0ae5aa640 1 --2- 172.21.15.105:0/4020506436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0a80cc760 0x7ff0a8099a50 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.902+0000 7ff0ae5aa640 1 --2- 172.21.15.105:0/4020506436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a806c7e0 0x7ff0a809a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:55.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.902+0000 7ff0ae5aa640 1 -- 172.21.15.105:0/4020506436 >> 172.21.15.105:0/4020506436 conn(0x7ff0a80ceb90 msgr2=0x7ff0a80abd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:55.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.902+0000 7ff0ae5aa640 1 -- 172.21.15.105:0/4020506436 shutdown_connections 2023-11-28T18:27:55.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:55.902+0000 7ff0ae5aa640 1 -- 172.21.15.105:0/4020506436 wait complete. 2023-11-28T18:27:55.926 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:27:56.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.062+0000 7fa18399d640 1 Processor -- start 2023-11-28T18:27:56.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.062+0000 7fa18399d640 1 -- start start 2023-11-28T18:27:56.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.062+0000 7fa18399d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa17c15d4f0 0x7fa17c0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:56.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.062+0000 7fa18399d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa17c0b25b0 con 0x7fa17c0b2d20 2023-11-28T18:27:56.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.062+0000 7fa18399d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa17c0cea30 con 0x7fa17c0d3c30 2023-11-28T18:27:56.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.062+0000 7fa18399d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa17c0b2900 con 0x7fa17c15d4f0 2023-11-28T18:27:56.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.062+0000 7fa181f3a640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa17c0b2d20 0x7fa17c0b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46228/0 (socket says 172.21.15.105:46228) 2023-11-28T18:27:56.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.062+0000 7fa181f3a640 1 -- 172.21.15.105:0/3674717072 learned_addr learned my addr 172.21.15.105:0/3674717072 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:56.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.062+0000 7fa173fff640 1 -- 172.21.15.105:0/3674717072 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1462466122 0 0) 0x7fa17c0b25b0 con 0x7fa17c0b2d20 2023-11-28T18:27:56.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.062+0000 7fa181739640 1 --2- 172.21.15.105:0/3674717072 >> v2:172.21.15.148:3300/0 conn(0x7fa17c15d4f0 0x7fa17c0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:56.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.062+0000 7fa173fff640 1 -- 172.21.15.105:0/3674717072 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa168003530 con 0x7fa17c0b2d20 2023-11-28T18:27:56.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.062+0000 7fa173fff640 1 -- 172.21.15.105:0/3674717072 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 23158806 0 0) 0x7fa168003530 con 0x7fa17c0b2d20 2023-11-28T18:27:56.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.062+0000 7fa173fff640 1 -- 172.21.15.105:0/3674717072 >> v2:172.21.15.148:3300/0 conn(0x7fa17c15d4f0 msgr2=0x7fa17c0d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:56.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.062+0000 7fa173fff640 1 --2- 172.21.15.105:0/3674717072 >> v2:172.21.15.148:3300/0 conn(0x7fa17c15d4f0 0x7fa17c0d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.062+0000 7fa173fff640 1 -- 172.21.15.105:0/3674717072 >> v1:172.21.15.148:6789/0 conn(0x7fa17c0d3c30 legacy=0x7fa17c0d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:56.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.062+0000 7fa173fff640 1 -- 172.21.15.105:0/3674717072 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c0ced10 con 0x7fa17c0b2d20 2023-11-28T18:27:56.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.062+0000 7fa173fff640 1 -- 172.21.15.105:0/3674717072 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fa174001a00 con 0x7fa17c0b2d20 2023-11-28T18:27:56.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.062+0000 7fa173fff640 1 -- 172.21.15.105:0/3674717072 >> v1:172.21.15.105:6789/0 conn(0x7fa17c0b2d20 legacy=0x7fa17c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:56.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.066+0000 7fa173fff640 1 --2- 172.21.15.105:0/3674717072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa168003980 0x7fa168003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:56.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.066+0000 7fa173fff640 1 --2- 172.21.15.105:0/3674717072 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1680047d0 0x7fa168004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:56.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.066+0000 7fa173fff640 1 --2- 172.21.15.105:0/3674717072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa168005190 0x7fa16800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:56.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.066+0000 7fa173fff640 1 -- 172.21.15.105:0/3674717072 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c0b2900 con 0x7fa168003980 2023-11-28T18:27:56.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.066+0000 7fa173fff640 1 -- 172.21.15.105:0/3674717072 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16800c130 con 0x7fa168005190 2023-11-28T18:27:56.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.066+0000 7fa173fff640 1 -- 172.21.15.105:0/3674717072 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16800c2f0 con 0x7fa1680047d0 2023-11-28T18:27:56.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.066+0000 7fa181739640 1 --2- 172.21.15.105:0/3674717072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa168003980 0x7fa168003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:56.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.066+0000 7fa181f3a640 1 --2- 172.21.15.105:0/3674717072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa168005190 0x7fa16800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:56.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.066+0000 7fa180f38640 1 --2- 172.21.15.105:0/3674717072 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1680047d0 0x7fa168004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:56.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.066+0000 7fa181739640 1 -- 172.21.15.105:0/3674717072 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1680047d0 msgr2=0x7fa168004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:56.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.066+0000 7fa181739640 1 --2- 172.21.15.105:0/3674717072 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1680047d0 0x7fa168004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.066+0000 7fa181739640 1 -- 172.21.15.105:0/3674717072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa168005190 msgr2=0x7fa16800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:56.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.066+0000 7fa181739640 1 --2- 172.21.15.105:0/3674717072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa168005190 0x7fa16800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.066+0000 7fa181739640 1 -- 172.21.15.105:0/3674717072 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa168003530 con 0x7fa168003980 2023-11-28T18:27:56.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.066+0000 7fa181f3a640 1 --2- 172.21.15.105:0/3674717072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa168005190 0x7fa16800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:56.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.066+0000 7fa181f3a640 1 -- 172.21.15.105:0/3674717072 reap_dead start 2023-11-28T18:27:56.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.066+0000 7fa181739640 1 --2- 172.21.15.105:0/3674717072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa168003980 0x7fa168003da0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa178009ae0 tx=0x7fa17801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3992618a8fa504 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:56.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.066+0000 7fa173fff640 1 -- 172.21.15.105:0/3674717072 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1780a5590 con 0x7fa168003980 2023-11-28T18:27:56.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.066+0000 7fa173fff640 1 -- 172.21.15.105:0/3674717072 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1780a56f0 con 0x7fa168003980 2023-11-28T18:27:56.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.066+0000 7fa173fff640 1 -- 172.21.15.105:0/3674717072 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1780a5aa0 con 0x7fa168003980 2023-11-28T18:27:56.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.070+0000 7fa18399d640 1 -- 172.21.15.105:0/3674717072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa168003980 msgr2=0x7fa168003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:56.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.070+0000 7fa18399d640 1 --2- 172.21.15.105:0/3674717072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa168003980 0x7fa168003da0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa178009ae0 tx=0x7fa17801ca20 comp rx=0 tx=0).stop 2023-11-28T18:27:56.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.070+0000 7fa18399d640 1 -- 172.21.15.105:0/3674717072 shutdown_connections 2023-11-28T18:27:56.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.070+0000 7fa18399d640 1 --2- 172.21.15.105:0/3674717072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa168003980 0x7fa168003da0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.070+0000 7fa18399d640 1 -- 172.21.15.105:0/3674717072 >> 172.21.15.105:0/3674717072 conn(0x7fa17c0d3030 msgr2=0x7fa17c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:56.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.070+0000 7fa18399d640 1 -- 172.21.15.105:0/3674717072 shutdown_connections 2023-11-28T18:27:56.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.070+0000 7fa18399d640 1 -- 172.21.15.105:0/3674717072 wait complete. 2023-11-28T18:27:56.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.070+0000 7fa18399d640 1 Processor -- start 2023-11-28T18:27:56.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.070+0000 7fa18399d640 1 -- start start 2023-11-28T18:27:56.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.070+0000 7fa18399d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa168003980 0x7fa17c1601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:56.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.070+0000 7fa18399d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa17c0b82f0 0x7fa17c0b8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:56.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.070+0000 7fa18399d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa17c0b5ed0 0x7fa17c0b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:56.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.070+0000 7fa18399d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c0d0a20 con 0x7fa17c0b82f0 2023-11-28T18:27:56.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.070+0000 7fa18399d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c0b6890 con 0x7fa168003980 2023-11-28T18:27:56.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.070+0000 7fa18399d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c0b06e0 con 0x7fa17c0b5ed0 2023-11-28T18:27:56.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.070+0000 7fa180f38640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa17c0b82f0 0x7fa17c0b8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:56.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.070+0000 7fa180f38640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa17c0b82f0 0x7fa17c0b8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:32922/0 (socket says 172.21.15.105:32922) 2023-11-28T18:27:56.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.070+0000 7fa180f38640 1 -- 172.21.15.105:0/1878858914 learned_addr learned my addr 172.21.15.105:0/1878858914 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:56.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.070+0000 7fa181739640 1 --2- 172.21.15.105:0/1878858914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa168003980 0x7fa17c1601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:56.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.070+0000 7fa181f3a640 1 --2- 172.21.15.105:0/1878858914 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa17c0b5ed0 0x7fa17c0b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:56.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.074+0000 7fa180f38640 1 -- 172.21.15.105:0/1878858914 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa17c0b5ed0 msgr2=0x7fa17c0b6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:56.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.074+0000 7fa180f38640 1 --2- 172.21.15.105:0/1878858914 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa17c0b5ed0 0x7fa17c0b6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.074+0000 7fa180f38640 1 -- 172.21.15.105:0/1878858914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa168003980 msgr2=0x7fa17c1601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:56.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.074+0000 7fa180f38640 1 --2- 172.21.15.105:0/1878858914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa168003980 0x7fa17c1601a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.074+0000 7fa180f38640 1 -- 172.21.15.105:0/1878858914 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c0b09c0 con 0x7fa17c0b82f0 2023-11-28T18:27:56.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.074+0000 7fa180f38640 1 --2- 172.21.15.105:0/1878858914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa17c0b82f0 0x7fa17c0b8770 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa16c001ec0 tx=0x7fa16c00e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:56.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.074+0000 7fa171ffb640 1 -- 172.21.15.105:0/1878858914 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c0052e0 con 0x7fa17c0b82f0 2023-11-28T18:27:56.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.074+0000 7fa171ffb640 1 -- 172.21.15.105:0/1878858914 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16c005440 con 0x7fa17c0b82f0 2023-11-28T18:27:56.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.074+0000 7fa171ffb640 1 -- 172.21.15.105:0/1878858914 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c003820 con 0x7fa17c0b82f0 2023-11-28T18:27:56.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.074+0000 7fa18399d640 1 -- 172.21.15.105:0/1878858914 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17c0b0ca0 con 0x7fa17c0b82f0 2023-11-28T18:27:56.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.074+0000 7fa18399d640 1 -- 172.21.15.105:0/1878858914 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17c0b1260 con 0x7fa17c0b82f0 2023-11-28T18:27:56.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.074+0000 7fa18399d640 1 -- 172.21.15.105:0/1878858914 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa17c067570 con 0x7fa17c0b82f0 2023-11-28T18:27:56.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.074+0000 7fa171ffb640 1 -- 172.21.15.105:0/1878858914 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa16c003980 con 0x7fa17c0b82f0 2023-11-28T18:27:56.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.074+0000 7fa171ffb640 1 --2- 172.21.15.105:0/1878858914 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa160066900 0x7fa160068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:56.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.074+0000 7fa171ffb640 1 -- 172.21.15.105:0/1878858914 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fa16c083be0 con 0x7fa17c0b82f0 2023-11-28T18:27:56.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.074+0000 7fa181739640 1 --2- 172.21.15.105:0/1878858914 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa160066900 0x7fa160068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:56.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.078+0000 7fa181739640 1 --2- 172.21.15.105:0/1878858914 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa160066900 0x7fa160068dc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa1780af000 tx=0x7fa1780046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:56.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.078+0000 7fa171ffb640 1 -- 172.21.15.105:0/1878858914 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa16c050d90 con 0x7fa17c0b82f0 2023-11-28T18:27:56.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.234+0000 7fa18399d640 1 -- 172.21.15.105:0/1878858914 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa17c0b3ab0 con 0x7fa17c0b82f0 2023-11-28T18:27:56.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.234+0000 7fa171ffb640 1 -- 172.21.15.105:0/1878858914 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v67) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa16c019020 con 0x7fa17c0b82f0 2023-11-28T18:27:56.238 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:27:56.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.234+0000 7fa18399d640 1 -- 172.21.15.105:0/1878858914 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa160066900 msgr2=0x7fa160068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:56.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.234+0000 7fa18399d640 1 --2- 172.21.15.105:0/1878858914 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa160066900 0x7fa160068dc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa1780af000 tx=0x7fa1780046d0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.234+0000 7fa18399d640 1 -- 172.21.15.105:0/1878858914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa17c0b82f0 msgr2=0x7fa17c0b8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:56.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.234+0000 7fa18399d640 1 --2- 172.21.15.105:0/1878858914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa17c0b82f0 0x7fa17c0b8770 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa16c001ec0 tx=0x7fa16c00e7e0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.234+0000 7fa18399d640 1 -- 172.21.15.105:0/1878858914 shutdown_connections 2023-11-28T18:27:56.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.234+0000 7fa18399d640 1 --2- 172.21.15.105:0/1878858914 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa17c0b5ed0 0x7fa17c0b6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.234+0000 7fa18399d640 1 --2- 172.21.15.105:0/1878858914 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa160066900 0x7fa160068dc0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.234+0000 7fa18399d640 1 --2- 172.21.15.105:0/1878858914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa17c0b82f0 0x7fa17c0b8770 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.234+0000 7fa18399d640 1 --2- 172.21.15.105:0/1878858914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa168003980 0x7fa17c1601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.234+0000 7fa18399d640 1 -- 172.21.15.105:0/1878858914 >> 172.21.15.105:0/1878858914 conn(0x7fa17c0d3030 msgr2=0x7fa17c0b3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:56.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.234+0000 7fa18399d640 1 -- 172.21.15.105:0/1878858914 shutdown_connections 2023-11-28T18:27:56.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.234+0000 7fa18399d640 1 -- 172.21.15.105:0/1878858914 wait complete. 2023-11-28T18:27:56.259 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T18:27:56.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.394+0000 7f65d51f8640 1 Processor -- start 2023-11-28T18:27:56.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.394+0000 7f65d51f8640 1 -- start start 2023-11-28T18:27:56.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.394+0000 7f65d51f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65d00b4290 0x7f65d00b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:56.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.394+0000 7f65d51f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f65d0071960 0x7f65d015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:56.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.394+0000 7f65d51f8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f65d015b030 0x7f65d00709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:56.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.394+0000 7f65d51f8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d00b4bb0 con 0x7f65d015b030 2023-11-28T18:27:56.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.394+0000 7f65d51f8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d00701d0 con 0x7f65d0071960 2023-11-28T18:27:56.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.394+0000 7f65d51f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d006df80 con 0x7f65d00b4290 2023-11-28T18:27:56.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.394+0000 7f65cf59e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f65d015b030 0x7f65d00709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:56.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.394+0000 7f65cf59e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f65d015b030 0x7f65d00709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:32932/0 (socket says 172.21.15.105:32932) 2023-11-28T18:27:56.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.394+0000 7f65cf59e640 1 -- 172.21.15.105:0/558475747 learned_addr learned my addr 172.21.15.105:0/558475747 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:56.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.394+0000 7f65ced9d640 1 --2- 172.21.15.105:0/558475747 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65d00b4290 0x7f65d00b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:56.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.394+0000 7f65c659c640 1 --2- 172.21.15.105:0/558475747 >> v2:172.21.15.148:3300/0 conn(0x7f65d0071960 0x7f65d015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:56.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.394+0000 7f65cf59e640 1 -- 172.21.15.105:0/558475747 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65d00b4290 msgr2=0x7f65d00b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:56.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.394+0000 7f65cf59e640 1 --2- 172.21.15.105:0/558475747 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65d00b4290 0x7f65d00b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.394+0000 7f65cf59e640 1 -- 172.21.15.105:0/558475747 >> v2:172.21.15.148:3300/0 conn(0x7f65d0071960 msgr2=0x7f65d015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:56.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.394+0000 7f65cf59e640 1 --2- 172.21.15.105:0/558475747 >> v2:172.21.15.148:3300/0 conn(0x7f65d0071960 0x7f65d015f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.394+0000 7f65cf59e640 1 -- 172.21.15.105:0/558475747 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d0071df0 con 0x7f65d015b030 2023-11-28T18:27:56.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.394+0000 7f65c659c640 1 --2- 172.21.15.105:0/558475747 >> v2:172.21.15.148:3300/0 conn(0x7f65d0071960 0x7f65d015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:56.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.398+0000 7f65cf59e640 1 --2- 172.21.15.105:0/558475747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65d015b030 0x7f65d00709b0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f65d00cc710 tx=0x7f65c801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9349ba5d4dd6917 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:56.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.398+0000 7f65ce59c640 1 -- 172.21.15.105:0/558475747 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c80ac600 con 0x7f65d015b030 2023-11-28T18:27:56.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.398+0000 7f65ce59c640 1 -- 172.21.15.105:0/558475747 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c80a8050 con 0x7f65d015b030 2023-11-28T18:27:56.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.398+0000 7f65ce59c640 1 -- 172.21.15.105:0/558475747 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c80ac980 con 0x7f65d015b030 2023-11-28T18:27:56.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.398+0000 7f65d51f8640 1 -- 172.21.15.105:0/558475747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65d015b030 msgr2=0x7f65d00709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:56.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.398+0000 7f65d51f8640 1 --2- 172.21.15.105:0/558475747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65d015b030 0x7f65d00709b0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f65d00cc710 tx=0x7f65c801c4d0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.398+0000 7f65d51f8640 1 -- 172.21.15.105:0/558475747 shutdown_connections 2023-11-28T18:27:56.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.398+0000 7f65d51f8640 1 --2- 172.21.15.105:0/558475747 >> v2:172.21.15.148:3300/0 conn(0x7f65d0071960 0x7f65d015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.398+0000 7f65d51f8640 1 --2- 172.21.15.105:0/558475747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65d015b030 0x7f65d00709b0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.398+0000 7f65d51f8640 1 --2- 172.21.15.105:0/558475747 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65d00b4290 0x7f65d00b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.398+0000 7f65d51f8640 1 -- 172.21.15.105:0/558475747 >> 172.21.15.105:0/558475747 conn(0x7f65d00ceb90 msgr2=0x7f65d00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:56.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.398+0000 7f65d51f8640 1 -- 172.21.15.105:0/558475747 shutdown_connections 2023-11-28T18:27:56.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.398+0000 7f65d51f8640 1 -- 172.21.15.105:0/558475747 wait complete. 2023-11-28T18:27:56.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.402+0000 7f65d51f8640 1 Processor -- start 2023-11-28T18:27:56.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.402+0000 7f65d51f8640 1 -- start start 2023-11-28T18:27:56.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.402+0000 7f65d51f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65d00b4290 0x7f65d01416d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:56.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.402+0000 7f65d51f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65d015b030 0x7f65d014b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:56.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.402+0000 7f65d51f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65d014bac0 0x7f65d00cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:56.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.402+0000 7f65d51f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d009aa00 con 0x7f65d014bac0 2023-11-28T18:27:56.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.402+0000 7f65d51f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d0099d80 con 0x7f65d00b4290 2023-11-28T18:27:56.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.402+0000 7f65d51f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d009a060 con 0x7f65d015b030 2023-11-28T18:27:56.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.402+0000 7f65cf59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65d014bac0 0x7f65d00cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:56.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.402+0000 7f65cf59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65d014bac0 0x7f65d00cdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:32946/0 (socket says 172.21.15.105:32946) 2023-11-28T18:27:56.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.402+0000 7f65cf59e640 1 -- 172.21.15.105:0/2515932467 learned_addr learned my addr 172.21.15.105:0/2515932467 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:56.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.402+0000 7f65ced9d640 1 --2- 172.21.15.105:0/2515932467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65d00b4290 0x7f65d01416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:56.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.402+0000 7f65c659c640 1 --2- 172.21.15.105:0/2515932467 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65d015b030 0x7f65d014b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.402+0000 7f65cf59e640 1 -- 172.21.15.105:0/2515932467 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65d015b030 msgr2=0x7f65d014b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.402+0000 7f65cf59e640 1 --2- 172.21.15.105:0/2515932467 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65d015b030 0x7f65d014b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.402+0000 7f65cf59e640 1 -- 172.21.15.105:0/2515932467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65d00b4290 msgr2=0x7f65d01416d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.402+0000 7f65cf59e640 1 --2- 172.21.15.105:0/2515932467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65d00b4290 0x7f65d01416d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.402+0000 7f65cf59e640 1 -- 172.21.15.105:0/2515932467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c80094d0 con 0x7f65d014bac0 2023-11-28T18:27:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.402+0000 7f65cf59e640 1 --2- 172.21.15.105:0/2515932467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65d014bac0 0x7f65d00cdf20 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f65c801cd70 tx=0x7f65c801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.406+0000 7f65c7fff640 1 -- 172.21.15.105:0/2515932467 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c80b7420 con 0x7f65d014bac0 2023-11-28T18:27:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.406+0000 7f65c7fff640 1 -- 172.21.15.105:0/2515932467 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c80b7580 con 0x7f65d014bac0 2023-11-28T18:27:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.406+0000 7f65c7fff640 1 -- 172.21.15.105:0/2515932467 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c80b7910 con 0x7f65d014bac0 2023-11-28T18:27:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.406+0000 7f65d51f8640 1 -- 172.21.15.105:0/2515932467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d009a340 con 0x7f65d014bac0 2023-11-28T18:27:56.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.406+0000 7f65d51f8640 1 -- 172.21.15.105:0/2515932467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d0099a50 con 0x7f65d014bac0 2023-11-28T18:27:56.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.406+0000 7f65d51f8640 1 -- 172.21.15.105:0/2515932467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d009ab40 con 0x7f65d014bac0 2023-11-28T18:27:56.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.406+0000 7f65c7fff640 1 -- 172.21.15.105:0/2515932467 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f65c80b3040 con 0x7f65d014bac0 2023-11-28T18:27:56.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.406+0000 7f65c7fff640 1 --2- 172.21.15.105:0/2515932467 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f65b0066900 0x7f65b0068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:56.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.406+0000 7f65c7fff640 1 -- 172.21.15.105:0/2515932467 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f65c811e0e0 con 0x7f65d014bac0 2023-11-28T18:27:56.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.406+0000 7f65ced9d640 1 --2- 172.21.15.105:0/2515932467 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f65b0066900 0x7f65b0068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:56.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.406+0000 7f65ced9d640 1 --2- 172.21.15.105:0/2515932467 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f65b0066900 0x7f65b0068dc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f65c00113e0 tx=0x7f65c000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:56.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.410+0000 7f65c7fff640 1 -- 172.21.15.105:0/2515932467 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f65d009ab40 con 0x7f65d014bac0 2023-11-28T18:27:56.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.570+0000 7f65d51f8640 1 -- 172.21.15.105:0/2515932467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f65d009ab40 con 0x7f65d014bac0 2023-11-28T18:27:56.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.570+0000 7f65c7fff640 1 -- 172.21.15.105:0/2515932467 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v67) v1 ==== 80+0+0 (secure 0 0 0) 0x7f65d009ab40 con 0x7f65d014bac0 2023-11-28T18:27:56.575 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2023-11-28T18:27:56.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.570+0000 7f65d51f8640 1 -- 172.21.15.105:0/2515932467 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f65b0066900 msgr2=0x7f65b0068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:56.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.570+0000 7f65d51f8640 1 --2- 172.21.15.105:0/2515932467 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f65b0066900 0x7f65b0068dc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f65c00113e0 tx=0x7f65c000b040 comp rx=0 tx=0).stop 2023-11-28T18:27:56.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.570+0000 7f65d51f8640 1 -- 172.21.15.105:0/2515932467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65d014bac0 msgr2=0x7f65d00cdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:56.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.570+0000 7f65d51f8640 1 --2- 172.21.15.105:0/2515932467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65d014bac0 0x7f65d00cdf20 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f65c801cd70 tx=0x7f65c801ad10 comp rx=0 tx=0).stop 2023-11-28T18:27:56.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.570+0000 7f65d51f8640 1 -- 172.21.15.105:0/2515932467 shutdown_connections 2023-11-28T18:27:56.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.570+0000 7f65d51f8640 1 --2- 172.21.15.105:0/2515932467 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f65b0066900 0x7f65b0068dc0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.570+0000 7f65d51f8640 1 --2- 172.21.15.105:0/2515932467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65d014bac0 0x7f65d00cdf20 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.570+0000 7f65d51f8640 1 --2- 172.21.15.105:0/2515932467 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65d015b030 0x7f65d014b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.570+0000 7f65d51f8640 1 --2- 172.21.15.105:0/2515932467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65d00b4290 0x7f65d01416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:56.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.570+0000 7f65d51f8640 1 -- 172.21.15.105:0/2515932467 >> 172.21.15.105:0/2515932467 conn(0x7f65d00ceb90 msgr2=0x7f65d00aeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:56.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.574+0000 7f65d51f8640 1 -- 172.21.15.105:0/2515932467 shutdown_connections 2023-11-28T18:27:56.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:56.574+0000 7f65d51f8640 1 -- 172.21.15.105:0/2515932467 wait complete. 2023-11-28T18:27:56.596 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2023-11-28T18:27:56.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:27:56.596 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-28T18:27:56.615 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[55430]: fuse finished with error 0 and tester_r 0 2023-11-28T18:27:56.619 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:27:56.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:27:56.619 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T18:27:56.687 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:27:56.687 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:27:56.697 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:27:56.697 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:27:56.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:27:56.698 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T18:27:56.750 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-28T18:27:56.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:27:56.751 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:27:56.751 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:27:56.760 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:27:56.760 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:27:56.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:27:56.761 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T18:27:56.812 INFO:teuthology.orchestra.run.smithi105.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-28T18:27:56.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:27:56.814 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:27:56.814 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:27:56.823 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:27:56.823 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:27:56.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:27:56.823 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T18:27:56.878 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-28T18:27:56.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:27:56.879 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:27:57.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.010+0000 7fc886a6b640 1 Processor -- start 2023-11-28T18:27:57.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.010+0000 7fc886a6b640 1 -- start start 2023-11-28T18:27:57.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.014+0000 7fc886a6b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc880159f40 0x7fc8800caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:57.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.014+0000 7fc886a6b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc8800cc760 0x7fc8800ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:57.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.014+0000 7fc886a6b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc88006bf50 con 0x7fc8800ad680 2023-11-28T18:27:57.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.014+0000 7fc886a6b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8800ccbc0 con 0x7fc8800cc760 2023-11-28T18:27:57.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.014+0000 7fc886a6b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88015a320 con 0x7fc880159f40 2023-11-28T18:27:57.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.014+0000 7fc877fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc8800cc760 0x7fc8800ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:57.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.014+0000 7fc885008640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc8800ad680 0x7fc88015da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:44006/0 (socket says 172.21.15.105:44006) 2023-11-28T18:27:57.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.014+0000 7fc885008640 1 -- 172.21.15.105:0/4142334528 learned_addr learned my addr 172.21.15.105:0/4142334528 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:57.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.014+0000 7fc884807640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc880159f40 0x7fc8800caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:57.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.014+0000 7fc8777fe640 1 -- 172.21.15.105:0/4142334528 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1509582164 0 0) 0x7fc88006bf50 con 0x7fc8800ad680 2023-11-28T18:27:57.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.014+0000 7fc8777fe640 1 -- 172.21.15.105:0/4142334528 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc868005740 con 0x7fc8800ad680 2023-11-28T18:27:57.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.014+0000 7fc884807640 1 -- 172.21.15.105:0/4142334528 >> v2:172.21.15.148:3300/0 conn(0x7fc8800cc760 msgr2=0x7fc8800ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:57.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.014+0000 7fc884807640 1 --2- 172.21.15.105:0/4142334528 >> v2:172.21.15.148:3300/0 conn(0x7fc8800cc760 0x7fc8800ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.014+0000 7fc884807640 1 -- 172.21.15.105:0/4142334528 >> v1:172.21.15.148:6789/0 conn(0x7fc8800ad680 legacy=0x7fc88015da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:57.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.014+0000 7fc884807640 1 -- 172.21.15.105:0/4142334528 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc880071380 con 0x7fc880159f40 2023-11-28T18:27:57.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.014+0000 7fc877fff640 1 --2- 172.21.15.105:0/4142334528 >> v2:172.21.15.148:3300/0 conn(0x7fc8800cc760 0x7fc8800ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:27:57.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.018+0000 7fc884807640 1 --2- 172.21.15.105:0/4142334528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc880159f40 0x7fc8800caf60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009d90 tx=0x7fc87c01c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bfde7be84fde41e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:57.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.018+0000 7fc8777fe640 1 -- 172.21.15.105:0/4142334528 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c0a5950 con 0x7fc880159f40 2023-11-28T18:27:57.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.018+0000 7fc8777fe640 1 -- 172.21.15.105:0/4142334528 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87c0a5ab0 con 0x7fc880159f40 2023-11-28T18:27:57.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.018+0000 7fc8777fe640 1 -- 172.21.15.105:0/4142334528 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c0a5e40 con 0x7fc880159f40 2023-11-28T18:27:57.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.018+0000 7fc886a6b640 1 -- 172.21.15.105:0/4142334528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc880159f40 msgr2=0x7fc8800caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:57.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.018+0000 7fc886a6b640 1 --2- 172.21.15.105:0/4142334528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc880159f40 0x7fc8800caf60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009d90 tx=0x7fc87c01c890 comp rx=0 tx=0).stop 2023-11-28T18:27:57.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.018+0000 7fc886a6b640 1 -- 172.21.15.105:0/4142334528 shutdown_connections 2023-11-28T18:27:57.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.018+0000 7fc886a6b640 1 --2- 172.21.15.105:0/4142334528 >> v2:172.21.15.148:3300/0 conn(0x7fc8800cc760 0x7fc8800ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.018+0000 7fc886a6b640 1 --2- 172.21.15.105:0/4142334528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc880159f40 0x7fc8800caf60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.018+0000 7fc886a6b640 1 -- 172.21.15.105:0/4142334528 >> 172.21.15.105:0/4142334528 conn(0x7fc8800ceb90 msgr2=0x7fc8800cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:57.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.018+0000 7fc886a6b640 1 -- 172.21.15.105:0/4142334528 shutdown_connections 2023-11-28T18:27:57.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.018+0000 7fc886a6b640 1 -- 172.21.15.105:0/4142334528 wait complete. 2023-11-28T18:27:57.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.018+0000 7fc886a6b640 1 Processor -- start 2023-11-28T18:27:57.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc886a6b640 1 -- start start 2023-11-28T18:27:57.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc886a6b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8800ad680 0x7fc88014d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:57.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc886a6b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8800cc760 0x7fc88009c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:57.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc886a6b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc880159f40 0x7fc88015c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:57.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc886a6b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88015be20 con 0x7fc880159f40 2023-11-28T18:27:57.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc886a6b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88015af30 con 0x7fc8800ad680 2023-11-28T18:27:57.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc886a6b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88015b210 con 0x7fc8800cc760 2023-11-28T18:27:57.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc885008640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc880159f40 0x7fc88015c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:57.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc885008640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc880159f40 0x7fc88015c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:32952/0 (socket says 172.21.15.105:32952) 2023-11-28T18:27:57.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc885008640 1 -- 172.21.15.105:0/3536110142 learned_addr learned my addr 172.21.15.105:0/3536110142 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:57.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc884807640 1 --2- 172.21.15.105:0/3536110142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8800ad680 0x7fc88014d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:57.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc877fff640 1 --2- 172.21.15.105:0/3536110142 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8800cc760 0x7fc88009c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:57.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc885008640 1 -- 172.21.15.105:0/3536110142 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8800cc760 msgr2=0x7fc88009c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:57.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc885008640 1 --2- 172.21.15.105:0/3536110142 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8800cc760 0x7fc88009c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc885008640 1 -- 172.21.15.105:0/3536110142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8800ad680 msgr2=0x7fc88014d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:57.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc885008640 1 --2- 172.21.15.105:0/3536110142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8800ad680 0x7fc88014d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc885008640 1 -- 172.21.15.105:0/3536110142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc870009420 con 0x7fc880159f40 2023-11-28T18:27:57.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc885008640 1 --2- 172.21.15.105:0/3536110142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc880159f40 0x7fc88015c2b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fc87800c450 tx=0x7fc87800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:57.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc8757fa640 1 -- 172.21.15.105:0/3536110142 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc878005cf0 con 0x7fc880159f40 2023-11-28T18:27:57.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc8757fa640 1 -- 172.21.15.105:0/3536110142 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc878005e50 con 0x7fc880159f40 2023-11-28T18:27:57.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc8757fa640 1 -- 172.21.15.105:0/3536110142 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8780072b0 con 0x7fc880159f40 2023-11-28T18:27:57.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc886a6b640 1 -- 172.21.15.105:0/3536110142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc87c009a70 con 0x7fc880159f40 2023-11-28T18:27:57.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc886a6b640 1 -- 172.21.15.105:0/3536110142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88015b7d0 con 0x7fc880159f40 2023-11-28T18:27:57.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.022+0000 7fc886a6b640 1 -- 172.21.15.105:0/3536110142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc88015bf90 con 0x7fc880159f40 2023-11-28T18:27:57.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.026+0000 7fc8757fa640 1 -- 172.21.15.105:0/3536110142 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc878007410 con 0x7fc880159f40 2023-11-28T18:27:57.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.026+0000 7fc8757fa640 1 --2- 172.21.15.105:0/3536110142 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc86c0668d0 0x7fc86c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:57.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.026+0000 7fc8757fa640 1 -- 172.21.15.105:0/3536110142 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fc8780837d0 con 0x7fc880159f40 2023-11-28T18:27:57.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.026+0000 7fc884807640 1 --2- 172.21.15.105:0/3536110142 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc86c0668d0 0x7fc86c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:57.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.026+0000 7fc884807640 1 --2- 172.21.15.105:0/3536110142 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc86c0668d0 0x7fc86c068d90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0af000 tx=0x7fc87c0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:57.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.030+0000 7fc8757fa640 1 -- 172.21.15.105:0/3536110142 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc878050850 con 0x7fc880159f40 2023-11-28T18:27:57.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.210+0000 7fc886a6b640 1 -- 172.21.15.105:0/3536110142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc880067570 con 0x7fc880159f40 2023-11-28T18:27:57.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.210+0000 7fc8757fa640 1 -- 172.21.15.105:0/3536110142 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4951 (secure 0 0 0) 0x7fc878053e70 con 0x7fc880159f40 2023-11-28T18:27:57.215 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:27:57.215 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":62,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/234000461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":234000461},{"type":"v1","addr":"172.21.15.148:6834","nonce":234000461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:27:05.479811+0000","modified":"2023-11-28T18:27:35.595155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6720,"mds_1":6717,"mds_2":6436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6436":{"gid":6436,"name":"b","rank":2,"incarnation":60,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6835/2396073410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2396073410},{"type":"v1","addr":"172.21.15.148:6835","nonce":2396073410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6717":{"gid":6717,"name":"a","rank":1,"incarnation":58,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6835/380088531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":380088531},{"type":"v1","addr":"172.21.15.105:6835","nonce":380088531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6720":{"gid":6720,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/3442512486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3442512486},{"type":"v1","addr":"172.21.15.105:6837","nonce":3442512486}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T18:27:57.215 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 62 2023-11-28T18:27:57.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.210+0000 7fc886a6b640 1 -- 172.21.15.105:0/3536110142 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc86c0668d0 msgr2=0x7fc86c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:57.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.210+0000 7fc886a6b640 1 --2- 172.21.15.105:0/3536110142 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc86c0668d0 0x7fc86c068d90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0af000 tx=0x7fc87c0a7070 comp rx=0 tx=0).stop 2023-11-28T18:27:57.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.210+0000 7fc886a6b640 1 -- 172.21.15.105:0/3536110142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc880159f40 msgr2=0x7fc88015c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:57.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.210+0000 7fc886a6b640 1 --2- 172.21.15.105:0/3536110142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc880159f40 0x7fc88015c2b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fc87800c450 tx=0x7fc87800c910 comp rx=0 tx=0).stop 2023-11-28T18:27:57.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.210+0000 7fc886a6b640 1 -- 172.21.15.105:0/3536110142 shutdown_connections 2023-11-28T18:27:57.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.210+0000 7fc886a6b640 1 --2- 172.21.15.105:0/3536110142 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc86c0668d0 0x7fc86c068d90 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.210+0000 7fc886a6b640 1 --2- 172.21.15.105:0/3536110142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc880159f40 0x7fc88015c2b0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.210+0000 7fc886a6b640 1 --2- 172.21.15.105:0/3536110142 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8800cc760 0x7fc88009c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.210+0000 7fc886a6b640 1 --2- 172.21.15.105:0/3536110142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8800ad680 0x7fc88014d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.210+0000 7fc886a6b640 1 -- 172.21.15.105:0/3536110142 >> 172.21.15.105:0/3536110142 conn(0x7fc8800ceb90 msgr2=0x7fc8800ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:57.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.210+0000 7fc886a6b640 1 -- 172.21.15.105:0/3536110142 shutdown_connections 2023-11-28T18:27:57.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.210+0000 7fc886a6b640 1 -- 172.21.15.105:0/3536110142 wait complete. 2023-11-28T18:27:57.236 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:27:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.370+0000 7fdd327f8640 1 Processor -- start 2023-11-28T18:27:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.370+0000 7fdd327f8640 1 -- start start 2023-11-28T18:27:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.370+0000 7fdd327f8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdd2c159f40 0x7fdd2c0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:57.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.370+0000 7fdd327f8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd2c06bf50 con 0x7fdd2c0ad850 2023-11-28T18:27:57.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.370+0000 7fdd327f8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd2c0ccbc0 con 0x7fdd2c0cc760 2023-11-28T18:27:57.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.370+0000 7fdd327f8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd2c15a320 con 0x7fdd2c159f40 2023-11-28T18:27:57.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.370+0000 7fdd2bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdd2c159f40 0x7fdd2c0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:57.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.370+0000 7fdd30d95640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fdd2c0ad850 0x7fdd2c0701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46230/0 (socket says 172.21.15.105:46230) 2023-11-28T18:27:57.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.370+0000 7fdd30d95640 1 -- 172.21.15.105:0/3475634443 learned_addr learned my addr 172.21.15.105:0/3475634443 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:57.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.370+0000 7fdd2bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdd2c159f40 0x7fdd2c0caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:32962/0 (socket says 172.21.15.105:32962) 2023-11-28T18:27:57.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.374+0000 7fdd2affd640 1 -- 172.21.15.105:0/3475634443 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1539564776 0 0) 0x7fdd2c06bf50 con 0x7fdd2c0ad850 2023-11-28T18:27:57.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.374+0000 7fdd2affd640 1 -- 172.21.15.105:0/3475634443 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd14003530 con 0x7fdd2c0ad850 2023-11-28T18:27:57.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.374+0000 7fdd2bfff640 1 -- 172.21.15.105:0/3475634443 >> v1:172.21.15.148:6789/0 conn(0x7fdd2c0cc760 legacy=0x7fdd2c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:57.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.374+0000 7fdd2bfff640 1 -- 172.21.15.105:0/3475634443 >> v1:172.21.15.105:6789/0 conn(0x7fdd2c0ad850 legacy=0x7fdd2c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:57.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.374+0000 7fdd2bfff640 1 -- 172.21.15.105:0/3475634443 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2c071380 con 0x7fdd2c159f40 2023-11-28T18:27:57.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.374+0000 7fdd2bfff640 1 --2- 172.21.15.105:0/3475634443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd2c159f40 0x7fdd2c0caf60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0097c0 tx=0x7fdd1c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e879df17906ffbf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:57.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.374+0000 7fdd2affd640 1 -- 172.21.15.105:0/3475634443 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c0ac620 con 0x7fdd2c159f40 2023-11-28T18:27:57.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.374+0000 7fdd2affd640 1 -- 172.21.15.105:0/3475634443 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1c0ac780 con 0x7fdd2c159f40 2023-11-28T18:27:57.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.374+0000 7fdd2affd640 1 -- 172.21.15.105:0/3475634443 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c0acb10 con 0x7fdd2c159f40 2023-11-28T18:27:57.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd327f8640 1 -- 172.21.15.105:0/3475634443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd2c159f40 msgr2=0x7fdd2c0caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:57.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd327f8640 1 --2- 172.21.15.105:0/3475634443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd2c159f40 0x7fdd2c0caf60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0097c0 tx=0x7fdd1c01c3c0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd327f8640 1 -- 172.21.15.105:0/3475634443 shutdown_connections 2023-11-28T18:27:57.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd327f8640 1 --2- 172.21.15.105:0/3475634443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd2c159f40 0x7fdd2c0caf60 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd327f8640 1 -- 172.21.15.105:0/3475634443 >> 172.21.15.105:0/3475634443 conn(0x7fdd2c0ceb90 msgr2=0x7fdd2c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:57.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd327f8640 1 -- 172.21.15.105:0/3475634443 shutdown_connections 2023-11-28T18:27:57.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd327f8640 1 -- 172.21.15.105:0/3475634443 wait complete. 2023-11-28T18:27:57.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd327f8640 1 Processor -- start 2023-11-28T18:27:57.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd327f8640 1 -- start start 2023-11-28T18:27:57.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd327f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd2c0ad850 0x7fdd2c152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:57.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd327f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd2c0cc760 0x7fdd2c149cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:57.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd327f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdd2c159f40 0x7fdd2c14b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:57.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd327f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c14ad80 con 0x7fdd2c0cc760 2023-11-28T18:27:57.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd327f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c0b3fd0 con 0x7fdd2c0ad850 2023-11-28T18:27:57.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd327f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c14a420 con 0x7fdd2c159f40 2023-11-28T18:27:57.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd2b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd2c0cc760 0x7fdd2c149cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:57.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd2b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd2c0cc760 0x7fdd2c149cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:32974/0 (socket says 172.21.15.105:32974) 2023-11-28T18:27:57.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd2b7fe640 1 -- 172.21.15.105:0/1768477064 learned_addr learned my addr 172.21.15.105:0/1768477064 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:57.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd2bfff640 1 --2- 172.21.15.105:0/1768477064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd2c0ad850 0x7fdd2c152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:57.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd30d95640 1 --2- 172.21.15.105:0/1768477064 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdd2c159f40 0x7fdd2c14b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:57.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd2b7fe640 1 -- 172.21.15.105:0/1768477064 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdd2c159f40 msgr2=0x7fdd2c14b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:57.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd2b7fe640 1 --2- 172.21.15.105:0/1768477064 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdd2c159f40 0x7fdd2c14b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd2b7fe640 1 -- 172.21.15.105:0/1768477064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd2c0ad850 msgr2=0x7fdd2c152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:57.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.378+0000 7fdd2b7fe640 1 --2- 172.21.15.105:0/1768477064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd2c0ad850 0x7fdd2c152470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.382+0000 7fdd2b7fe640 1 -- 172.21.15.105:0/1768477064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1c009420 con 0x7fdd2c0cc760 2023-11-28T18:27:57.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.382+0000 7fdd2b7fe640 1 --2- 172.21.15.105:0/1768477064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd2c0cc760 0x7fdd2c149cb0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000c400 tx=0x7fdd2000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:57.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.382+0000 7fdd28ff9640 1 -- 172.21.15.105:0/1768477064 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd200052d0 con 0x7fdd2c0cc760 2023-11-28T18:27:57.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.382+0000 7fdd28ff9640 1 -- 172.21.15.105:0/1768477064 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd20005430 con 0x7fdd2c0cc760 2023-11-28T18:27:57.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.382+0000 7fdd327f8640 1 -- 172.21.15.105:0/1768477064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd2c14a6c0 con 0x7fdd2c0cc760 2023-11-28T18:27:57.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.382+0000 7fdd28ff9640 1 -- 172.21.15.105:0/1768477064 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20003820 con 0x7fdd2c0cc760 2023-11-28T18:27:57.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.382+0000 7fdd327f8640 1 -- 172.21.15.105:0/1768477064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2c0b3780 con 0x7fdd2c0cc760 2023-11-28T18:27:57.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.382+0000 7fdd327f8640 1 -- 172.21.15.105:0/1768477064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd2c14aec0 con 0x7fdd2c0cc760 2023-11-28T18:27:57.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.382+0000 7fdd28ff9640 1 -- 172.21.15.105:0/1768477064 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fdd20003980 con 0x7fdd2c0cc760 2023-11-28T18:27:57.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.382+0000 7fdd28ff9640 1 --2- 172.21.15.105:0/1768477064 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdd180668d0 0x7fdd18068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:57.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.382+0000 7fdd28ff9640 1 -- 172.21.15.105:0/1768477064 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fdd20083aa0 con 0x7fdd2c0cc760 2023-11-28T18:27:57.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.382+0000 7fdd2bfff640 1 --2- 172.21.15.105:0/1768477064 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdd180668d0 0x7fdd18068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:57.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.386+0000 7fdd2bfff640 1 --2- 172.21.15.105:0/1768477064 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdd180668d0 0x7fdd18068d90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0ae000 tx=0x7fdd1c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:57.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.386+0000 7fdd28ff9640 1 -- 172.21.15.105:0/1768477064 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdd2c14aec0 con 0x7fdd2c0cc760 2023-11-28T18:27:57.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.538+0000 7fdd327f8640 1 -- 172.21.15.105:0/1768477064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd2c14aec0 con 0x7fdd2c0cc760 2023-11-28T18:27:57.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.542+0000 7fdd28ff9640 1 -- 172.21.15.105:0/1768477064 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+17014 (secure 0 0 0) 0x7fdd2c14aec0 con 0x7fdd2c0cc760 2023-11-28T18:27:57.545 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:27:57.546 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":67,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:27:16.173737+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:27:03.225100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T18:27:04.478619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.e","mappings":[{"from":7,"to":2}]},{"pgid":"10.c","mappings":[{"from":7,"to":3}]},{"pgid":"10.d","mappings":[{"from":4,"to":3}]},{"pgid":"10.19","mappings":[{"from":4,"to":5}]},{"pgid":"10.1d","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:27:57.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.542+0000 7fdd327f8640 1 -- 172.21.15.105:0/1768477064 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdd180668d0 msgr2=0x7fdd18068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:57.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.542+0000 7fdd327f8640 1 --2- 172.21.15.105:0/1768477064 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdd180668d0 0x7fdd18068d90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0ae000 tx=0x7fdd1c008040 comp rx=0 tx=0).stop 2023-11-28T18:27:57.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.542+0000 7fdd327f8640 1 -- 172.21.15.105:0/1768477064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd2c0cc760 msgr2=0x7fdd2c149cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:57.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.542+0000 7fdd327f8640 1 --2- 172.21.15.105:0/1768477064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd2c0cc760 0x7fdd2c149cb0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000c400 tx=0x7fdd2000c8c0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.542+0000 7fdd327f8640 1 -- 172.21.15.105:0/1768477064 shutdown_connections 2023-11-28T18:27:57.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.542+0000 7fdd327f8640 1 --2- 172.21.15.105:0/1768477064 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdd2c159f40 0x7fdd2c14b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.542+0000 7fdd327f8640 1 --2- 172.21.15.105:0/1768477064 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdd180668d0 0x7fdd18068d90 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.542+0000 7fdd327f8640 1 --2- 172.21.15.105:0/1768477064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd2c0cc760 0x7fdd2c149cb0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.542+0000 7fdd327f8640 1 --2- 172.21.15.105:0/1768477064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd2c0ad850 0x7fdd2c152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.542+0000 7fdd327f8640 1 -- 172.21.15.105:0/1768477064 >> 172.21.15.105:0/1768477064 conn(0x7fdd2c0ceb90 msgr2=0x7fdd2c0ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:57.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.542+0000 7fdd327f8640 1 -- 172.21.15.105:0/1768477064 shutdown_connections 2023-11-28T18:27:57.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.542+0000 7fdd327f8640 1 -- 172.21.15.105:0/1768477064 wait complete. 2023-11-28T18:27:57.567 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:27:03.225100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:27:04.478619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:27:57.569 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:27:57.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.702+0000 7f6068b1d640 1 Processor -- start 2023-11-28T18:27:57.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.702+0000 7f6068b1d640 1 -- start start 2023-11-28T18:27:57.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.706+0000 7f6068b1d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6064159f40 0x7f60640caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:57.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.706+0000 7f6068b1d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f60640ad850 0x7f60640701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:57.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.706+0000 7f6068b1d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f606406bf50 con 0x7f60640cc760 2023-11-28T18:27:57.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.706+0000 7f6068b1d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60640ccbc0 con 0x7f60640ad850 2023-11-28T18:27:57.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.706+0000 7f6068b1d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f606415a320 con 0x7f6064159f40 2023-11-28T18:27:57.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.706+0000 7f6061d9b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f60640cc760 0x7f60640ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46240/0 (socket says 172.21.15.105:46240) 2023-11-28T18:27:57.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.706+0000 7f6061d9b640 1 -- 172.21.15.105:0/3115889930 learned_addr learned my addr 172.21.15.105:0/3115889930 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:57.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.706+0000 7f6062d9d640 1 --2- 172.21.15.105:0/3115889930 >> v2:172.21.15.105:3300/0 conn(0x7f60640ad850 0x7f60640701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:57.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.706+0000 7f606259c640 1 --2- 172.21.15.105:0/3115889930 >> v2:172.21.15.148:3300/0 conn(0x7f6064159f40 0x7f60640caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:57.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.706+0000 7f606159a640 1 -- 172.21.15.105:0/3115889930 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4178121074 0 0) 0x7f606406bf50 con 0x7f60640cc760 2023-11-28T18:27:57.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.706+0000 7f6062d9d640 1 -- 172.21.15.105:0/3115889930 >> v2:172.21.15.148:3300/0 conn(0x7f6064159f40 msgr2=0x7f60640caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:57.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.706+0000 7f6062d9d640 1 --2- 172.21.15.105:0/3115889930 >> v2:172.21.15.148:3300/0 conn(0x7f6064159f40 0x7f60640caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.706+0000 7f6062d9d640 1 -- 172.21.15.105:0/3115889930 >> v1:172.21.15.105:6789/0 conn(0x7f60640cc760 legacy=0x7f60640ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:57.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.706+0000 7f6062d9d640 1 -- 172.21.15.105:0/3115889930 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6064071380 con 0x7f60640ad850 2023-11-28T18:27:57.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.710+0000 7f6062d9d640 1 --2- 172.21.15.105:0/3115889930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60640ad850 0x7f60640701d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f606406e500 tx=0x7f605c01c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325f882d724e21a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:57.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.710+0000 7f606159a640 1 -- 172.21.15.105:0/3115889930 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c0a5910 con 0x7f60640ad850 2023-11-28T18:27:57.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.710+0000 7f606159a640 1 -- 172.21.15.105:0/3115889930 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f605c0a5a70 con 0x7f60640ad850 2023-11-28T18:27:57.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.710+0000 7f606159a640 1 -- 172.21.15.105:0/3115889930 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c0a5e00 con 0x7f60640ad850 2023-11-28T18:27:57.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.710+0000 7f6068b1d640 1 -- 172.21.15.105:0/3115889930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60640ad850 msgr2=0x7f60640701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:57.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.710+0000 7f6068b1d640 1 --2- 172.21.15.105:0/3115889930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60640ad850 0x7f60640701d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f606406e500 tx=0x7f605c01c850 comp rx=0 tx=0).stop 2023-11-28T18:27:57.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.710+0000 7f6068b1d640 1 -- 172.21.15.105:0/3115889930 shutdown_connections 2023-11-28T18:27:57.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.710+0000 7f6068b1d640 1 --2- 172.21.15.105:0/3115889930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60640ad850 0x7f60640701d0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.710+0000 7f6068b1d640 1 --2- 172.21.15.105:0/3115889930 >> v2:172.21.15.148:3300/0 conn(0x7f6064159f40 0x7f60640caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.710+0000 7f6068b1d640 1 -- 172.21.15.105:0/3115889930 >> 172.21.15.105:0/3115889930 conn(0x7f60640ceb90 msgr2=0x7f60640cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:57.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.710+0000 7f6068b1d640 1 -- 172.21.15.105:0/3115889930 shutdown_connections 2023-11-28T18:27:57.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.710+0000 7f6068b1d640 1 -- 172.21.15.105:0/3115889930 wait complete. 2023-11-28T18:27:57.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.710+0000 7f6068b1d640 1 Processor -- start 2023-11-28T18:27:57.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.710+0000 7f6068b1d640 1 -- start start 2023-11-28T18:27:57.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6068b1d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60640ad850 0x7f6064152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:57.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6068b1d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60640cc760 0x7f6064155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:57.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6068b1d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6064159f40 0x7f60641f90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:57.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6068b1d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6064141e10 con 0x7f60640cc760 2023-11-28T18:27:57.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6068b1d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6064158600 con 0x7f6064159f40 2023-11-28T18:27:57.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6068b1d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6064156010 con 0x7f60640ad850 2023-11-28T18:27:57.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6061d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60640cc760 0x7f6064155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:57.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6061d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60640cc760 0x7f6064155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33006/0 (socket says 172.21.15.105:33006) 2023-11-28T18:27:57.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6061d9b640 1 -- 172.21.15.105:0/461909913 learned_addr learned my addr 172.21.15.105:0/461909913 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:57.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f606259c640 1 --2- 172.21.15.105:0/461909913 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60640ad850 0x7f6064152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:57.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6062d9d640 1 --2- 172.21.15.105:0/461909913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6064159f40 0x7f60641f90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:57.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6061d9b640 1 -- 172.21.15.105:0/461909913 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60640ad850 msgr2=0x7f6064152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:57.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6061d9b640 1 --2- 172.21.15.105:0/461909913 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60640ad850 0x7f6064152390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6061d9b640 1 -- 172.21.15.105:0/461909913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6064159f40 msgr2=0x7f60641f90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:57.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6061d9b640 1 --2- 172.21.15.105:0/461909913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6064159f40 0x7f60641f90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6061d9b640 1 -- 172.21.15.105:0/461909913 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605c009a70 con 0x7f60640cc760 2023-11-28T18:27:57.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f606259c640 1 --2- 172.21.15.105:0/461909913 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60640ad850 0x7f6064152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:57.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6061d9b640 1 --2- 172.21.15.105:0/461909913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60640cc760 0x7f6064155630 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f605800c480 tx=0x7f605800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:57.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6052ffd640 1 -- 172.21.15.105:0/461909913 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60580074e0 con 0x7f60640cc760 2023-11-28T18:27:57.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6052ffd640 1 -- 172.21.15.105:0/461909913 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6058007660 con 0x7f60640cc760 2023-11-28T18:27:57.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6068b1d640 1 -- 172.21.15.105:0/461909913 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60641588a0 con 0x7f60640cc760 2023-11-28T18:27:57.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6052ffd640 1 -- 172.21.15.105:0/461909913 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6058004a30 con 0x7f60640cc760 2023-11-28T18:27:57.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6068b1d640 1 -- 172.21.15.105:0/461909913 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6064158e60 con 0x7f60640cc760 2023-11-28T18:27:57.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6068b1d640 1 -- 172.21.15.105:0/461909913 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6064067570 con 0x7f60640cc760 2023-11-28T18:27:57.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.714+0000 7f6052ffd640 1 -- 172.21.15.105:0/461909913 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f6058004b90 con 0x7f60640cc760 2023-11-28T18:27:57.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.718+0000 7f6052ffd640 1 --2- 172.21.15.105:0/461909913 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6044066900 0x7f6044068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:57.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.718+0000 7f6052ffd640 1 -- 172.21.15.105:0/461909913 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f6058083770 con 0x7f60640cc760 2023-11-28T18:27:57.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.718+0000 7f606259c640 1 --2- 172.21.15.105:0/461909913 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6044066900 0x7f6044068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:57.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.718+0000 7f606259c640 1 --2- 172.21.15.105:0/461909913 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6044066900 0x7f6044068dc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6054009b10 tx=0x7f6054008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:57.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.722+0000 7f6052ffd640 1 -- 172.21.15.105:0/461909913 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6058054260 con 0x7f60640cc760 2023-11-28T18:27:57.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.870+0000 7f6068b1d640 1 -- 172.21.15.105:0/461909913 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f60640acce0 con 0x7f60640cc760 2023-11-28T18:27:57.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.870+0000 7f6052ffd640 1 -- 172.21.15.105:0/461909913 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6058053df0 con 0x7f60640cc760 2023-11-28T18:27:57.876 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:27:57.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.874+0000 7f6068b1d640 1 -- 172.21.15.105:0/461909913 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6044066900 msgr2=0x7f6044068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:57.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.874+0000 7f6068b1d640 1 --2- 172.21.15.105:0/461909913 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6044066900 0x7f6044068dc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6054009b10 tx=0x7f6054008040 comp rx=0 tx=0).stop 2023-11-28T18:27:57.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.874+0000 7f6068b1d640 1 -- 172.21.15.105:0/461909913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60640cc760 msgr2=0x7f6064155630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:57.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.874+0000 7f6068b1d640 1 --2- 172.21.15.105:0/461909913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60640cc760 0x7f6064155630 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f605800c480 tx=0x7f605800c940 comp rx=0 tx=0).stop 2023-11-28T18:27:57.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.874+0000 7f6068b1d640 1 -- 172.21.15.105:0/461909913 shutdown_connections 2023-11-28T18:27:57.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.874+0000 7f6068b1d640 1 --2- 172.21.15.105:0/461909913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6064159f40 0x7f60641f90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.874+0000 7f6068b1d640 1 --2- 172.21.15.105:0/461909913 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6044066900 0x7f6044068dc0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.874+0000 7f6068b1d640 1 --2- 172.21.15.105:0/461909913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60640cc760 0x7f6064155630 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.874+0000 7f6068b1d640 1 --2- 172.21.15.105:0/461909913 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60640ad850 0x7f6064152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:57.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.874+0000 7f6068b1d640 1 -- 172.21.15.105:0/461909913 >> 172.21.15.105:0/461909913 conn(0x7f60640ceb90 msgr2=0x7f60640cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:57.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.874+0000 7f6068b1d640 1 -- 172.21.15.105:0/461909913 shutdown_connections 2023-11-28T18:27:57.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:57.874+0000 7f6068b1d640 1 -- 172.21.15.105:0/461909913 wait complete. 2023-11-28T18:27:57.897 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:27:58.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.030+0000 7fc7cf1aa640 1 Processor -- start 2023-11-28T18:27:58.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.034+0000 7fc7cf1aa640 1 -- start start 2023-11-28T18:27:58.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.034+0000 7fc7cf1aa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc7c8159f40 0x7fc7c80caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:58.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.034+0000 7fc7cf1aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7c80cc760 0x7fc7c80ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:58.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.034+0000 7fc7cf1aa640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c806bf50 con 0x7fc7c80ad680 2023-11-28T18:27:58.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.034+0000 7fc7cf1aa640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c80ccbc0 con 0x7fc7c8159f40 2023-11-28T18:27:58.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.034+0000 7fc7cf1aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c815a320 con 0x7fc7c80cc760 2023-11-28T18:27:58.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.034+0000 7fc7ccf46640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc7c8159f40 0x7fc7c80caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:58.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.034+0000 7fc7bffff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7c80cc760 0x7fc7c80ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:58.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.034+0000 7fc7cd747640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc7c80ad680 0x7fc7c815da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:44028/0 (socket says 172.21.15.105:44028) 2023-11-28T18:27:58.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.034+0000 7fc7cd747640 1 -- 172.21.15.105:0/3174258975 learned_addr learned my addr 172.21.15.105:0/3174258975 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:58.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.034+0000 7fc7ccf46640 1 -- 172.21.15.105:0/3174258975 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7c80cc760 msgr2=0x7fc7c80ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:58.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.034+0000 7fc7ccf46640 1 --2- 172.21.15.105:0/3174258975 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7c80cc760 0x7fc7c80ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.034+0000 7fc7ccf46640 1 -- 172.21.15.105:0/3174258975 >> v1:172.21.15.148:6789/0 conn(0x7fc7c80ad680 legacy=0x7fc7c815da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:58.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.034+0000 7fc7ccf46640 1 -- 172.21.15.105:0/3174258975 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c8071380 con 0x7fc7c8159f40 2023-11-28T18:27:58.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.034+0000 7fc7bf7fe640 1 -- 172.21.15.105:0/3174258975 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 85680024 0 0) 0x7fc7c806bf50 con 0x7fc7c80ad680 2023-11-28T18:27:58.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.038+0000 7fc7ccf46640 1 --2- 172.21.15.105:0/3174258975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7c8159f40 0x7fc7c80caf60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40097c0 tx=0x7fc7c401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ada9c2199167e732 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:58.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.038+0000 7fc7bf7fe640 1 -- 172.21.15.105:0/3174258975 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40ac620 con 0x7fc7c8159f40 2023-11-28T18:27:58.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.038+0000 7fc7bf7fe640 1 -- 172.21.15.105:0/3174258975 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c40ac780 con 0x7fc7c8159f40 2023-11-28T18:27:58.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.038+0000 7fc7bf7fe640 1 -- 172.21.15.105:0/3174258975 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40acb10 con 0x7fc7c8159f40 2023-11-28T18:27:58.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.038+0000 7fc7cf1aa640 1 -- 172.21.15.105:0/3174258975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7c8159f40 msgr2=0x7fc7c80caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:58.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.038+0000 7fc7cf1aa640 1 --2- 172.21.15.105:0/3174258975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7c8159f40 0x7fc7c80caf60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40097c0 tx=0x7fc7c401c3c0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.038+0000 7fc7cf1aa640 1 -- 172.21.15.105:0/3174258975 shutdown_connections 2023-11-28T18:27:58.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.038+0000 7fc7cf1aa640 1 --2- 172.21.15.105:0/3174258975 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7c80cc760 0x7fc7c80ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.038+0000 7fc7cf1aa640 1 --2- 172.21.15.105:0/3174258975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7c8159f40 0x7fc7c80caf60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.038+0000 7fc7cf1aa640 1 -- 172.21.15.105:0/3174258975 >> 172.21.15.105:0/3174258975 conn(0x7fc7c80ceb90 msgr2=0x7fc7c80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:58.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.038+0000 7fc7cf1aa640 1 -- 172.21.15.105:0/3174258975 shutdown_connections 2023-11-28T18:27:58.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.038+0000 7fc7cf1aa640 1 -- 172.21.15.105:0/3174258975 wait complete. 2023-11-28T18:27:58.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.038+0000 7fc7cf1aa640 1 Processor -- start 2023-11-28T18:27:58.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.038+0000 7fc7cf1aa640 1 -- start start 2023-11-28T18:27:58.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7cf1aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7c80ad680 0x7fc7c814f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:58.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7cf1aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7c80cc760 0x7fc7c8151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:58.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7cf1aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7c8159f40 0x7fc7c80af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:58.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7cf1aa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c814f790 con 0x7fc7c80ad680 2023-11-28T18:27:58.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7cf1aa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c81501b0 con 0x7fc7c80cc760 2023-11-28T18:27:58.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7cf1aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c8151010 con 0x7fc7c8159f40 2023-11-28T18:27:58.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7ccf46640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7c80ad680 0x7fc7c814f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:58.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7ccf46640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7c80ad680 0x7fc7c814f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33020/0 (socket says 172.21.15.105:33020) 2023-11-28T18:27:58.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7ccf46640 1 -- 172.21.15.105:0/957059956 learned_addr learned my addr 172.21.15.105:0/957059956 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:58.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7bffff640 1 --2- 172.21.15.105:0/957059956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7c80cc760 0x7fc7c8151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:58.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7cd747640 1 --2- 172.21.15.105:0/957059956 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7c8159f40 0x7fc7c80af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:58.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7ccf46640 1 -- 172.21.15.105:0/957059956 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7c8159f40 msgr2=0x7fc7c80af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:58.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7ccf46640 1 --2- 172.21.15.105:0/957059956 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7c8159f40 0x7fc7c80af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7ccf46640 1 -- 172.21.15.105:0/957059956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7c80cc760 msgr2=0x7fc7c8151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:58.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7ccf46640 1 --2- 172.21.15.105:0/957059956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7c80cc760 0x7fc7c8151c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7ccf46640 1 -- 172.21.15.105:0/957059956 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c4009420 con 0x7fc7c80ad680 2023-11-28T18:27:58.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7ccf46640 1 --2- 172.21.15.105:0/957059956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7c80ad680 0x7fc7c814f250 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40097c0 tx=0x7fc7c4019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:58.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7bd7fa640 1 -- 172.21.15.105:0/957059956 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40ac620 con 0x7fc7c80ad680 2023-11-28T18:27:58.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7bd7fa640 1 -- 172.21.15.105:0/957059956 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c4004030 con 0x7fc7c80ad680 2023-11-28T18:27:58.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7bd7fa640 1 -- 172.21.15.105:0/957059956 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40043a0 con 0x7fc7c80ad680 2023-11-28T18:27:58.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7cf1aa640 1 -- 172.21.15.105:0/957059956 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c8150400 con 0x7fc7c80ad680 2023-11-28T18:27:58.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7cf1aa640 1 -- 172.21.15.105:0/957059956 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c81509c0 con 0x7fc7c80ad680 2023-11-28T18:27:58.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.042+0000 7fc7cf1aa640 1 -- 172.21.15.105:0/957059956 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c8151150 con 0x7fc7c80ad680 2023-11-28T18:27:58.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.046+0000 7fc7bd7fa640 1 -- 172.21.15.105:0/957059956 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc7c40b3040 con 0x7fc7c80ad680 2023-11-28T18:27:58.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.046+0000 7fc7bd7fa640 1 --2- 172.21.15.105:0/957059956 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc7b40668d0 0x7fc7b4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:58.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.046+0000 7fc7bd7fa640 1 -- 172.21.15.105:0/957059956 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fc7c411e4a0 con 0x7fc7c80ad680 2023-11-28T18:27:58.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.046+0000 7fc7bffff640 1 --2- 172.21.15.105:0/957059956 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc7b40668d0 0x7fc7b4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:58.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.046+0000 7fc7bffff640 1 --2- 172.21.15.105:0/957059956 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc7b40668d0 0x7fc7b4068d90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc7c80b03e0 tx=0x7fc7b8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:58.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.050+0000 7fc7bd7fa640 1 -- 172.21.15.105:0/957059956 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc7c40eb520 con 0x7fc7c80ad680 2023-11-28T18:27:58.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.202+0000 7fc7cf1aa640 1 -- 172.21.15.105:0/957059956 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc7c8067570 con 0x7fc7c80ad680 2023-11-28T18:27:58.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.206+0000 7fc7bd7fa640 1 -- 172.21.15.105:0/957059956 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v67) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc7c40eeb40 con 0x7fc7c80ad680 2023-11-28T18:27:58.210 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:27:58.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.206+0000 7fc7cf1aa640 1 -- 172.21.15.105:0/957059956 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc7b40668d0 msgr2=0x7fc7b4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:58.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.206+0000 7fc7cf1aa640 1 --2- 172.21.15.105:0/957059956 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc7b40668d0 0x7fc7b4068d90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc7c80b03e0 tx=0x7fc7b8009190 comp rx=0 tx=0).stop 2023-11-28T18:27:58.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.206+0000 7fc7cf1aa640 1 -- 172.21.15.105:0/957059956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7c80ad680 msgr2=0x7fc7c814f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:58.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.206+0000 7fc7cf1aa640 1 --2- 172.21.15.105:0/957059956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7c80ad680 0x7fc7c814f250 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40097c0 tx=0x7fc7c4019160 comp rx=0 tx=0).stop 2023-11-28T18:27:58.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.206+0000 7fc7cf1aa640 1 -- 172.21.15.105:0/957059956 shutdown_connections 2023-11-28T18:27:58.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.206+0000 7fc7cf1aa640 1 --2- 172.21.15.105:0/957059956 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7c8159f40 0x7fc7c80af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.206+0000 7fc7cf1aa640 1 --2- 172.21.15.105:0/957059956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7c80cc760 0x7fc7c8151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.206+0000 7fc7cf1aa640 1 --2- 172.21.15.105:0/957059956 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc7b40668d0 0x7fc7b4068d90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.206+0000 7fc7cf1aa640 1 --2- 172.21.15.105:0/957059956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7c80ad680 0x7fc7c814f250 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.206+0000 7fc7cf1aa640 1 -- 172.21.15.105:0/957059956 >> 172.21.15.105:0/957059956 conn(0x7fc7c80ceb90 msgr2=0x7fc7c80ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:58.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.206+0000 7fc7cf1aa640 1 -- 172.21.15.105:0/957059956 shutdown_connections 2023-11-28T18:27:58.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.206+0000 7fc7cf1aa640 1 -- 172.21.15.105:0/957059956 wait complete. 2023-11-28T18:27:58.230 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:27:58.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.366+0000 7f0b52e32640 1 Processor -- start 2023-11-28T18:27:58.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.366+0000 7f0b52e32640 1 -- start start 2023-11-28T18:27:58.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.366+0000 7f0b52e32640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0b4c0b8a20 0x7f0b4c0cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:58.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.366+0000 7f0b52e32640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0b4c0ceb40 0x7f0b4c0cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:58.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.366+0000 7f0b52e32640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0b4c0cfee0 0x7f0b4c0d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:58.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.366+0000 7f0b52e32640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b4c0d0370 con 0x7f0b4c0cfee0 2023-11-28T18:27:58.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.366+0000 7f0b52e32640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b4c0d3e60 con 0x7f0b4c0ceb40 2023-11-28T18:27:58.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.366+0000 7f0b52e32640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0b5fe0 con 0x7f0b4c0b8a20 2023-11-28T18:27:58.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.366+0000 7f0b513cf640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0b4c0cfee0 0x7f0b4c0d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:58.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.366+0000 7f0b513cf640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0b4c0cfee0 0x7f0b4c0d0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33022/0 (socket says 172.21.15.105:33022) 2023-11-28T18:27:58.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.366+0000 7f0b513cf640 1 -- 172.21.15.105:0/2173941331 learned_addr learned my addr 172.21.15.105:0/2173941331 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:58.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.366+0000 7f0b50bce640 1 --2- 172.21.15.105:0/2173941331 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0b4c0b8a20 0x7f0b4c0cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:58.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.366+0000 7f0b4bfff640 1 --2- 172.21.15.105:0/2173941331 >> v2:172.21.15.148:3300/0 conn(0x7f0b4c0ceb40 0x7f0b4c0cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:58.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.366+0000 7f0b513cf640 1 -- 172.21.15.105:0/2173941331 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0b4c0b8a20 msgr2=0x7f0b4c0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:58.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.366+0000 7f0b513cf640 1 --2- 172.21.15.105:0/2173941331 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0b4c0b8a20 0x7f0b4c0cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.366+0000 7f0b513cf640 1 -- 172.21.15.105:0/2173941331 >> v2:172.21.15.148:3300/0 conn(0x7f0b4c0ceb40 msgr2=0x7f0b4c0cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:58.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.366+0000 7f0b513cf640 1 --2- 172.21.15.105:0/2173941331 >> v2:172.21.15.148:3300/0 conn(0x7f0b4c0ceb40 0x7f0b4c0cf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.366+0000 7f0b513cf640 1 -- 172.21.15.105:0/2173941331 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4c0b62c0 con 0x7f0b4c0cfee0 2023-11-28T18:27:58.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.370+0000 7f0b513cf640 1 --2- 172.21.15.105:0/2173941331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b4c0cfee0 0x7f0b4c0d0ef0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c08c9b0 tx=0x7f0b4401c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ca0c1164962444 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:58.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.370+0000 7f0b4b7fe640 1 -- 172.21.15.105:0/2173941331 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b440a5560 con 0x7f0b4c0cfee0 2023-11-28T18:27:58.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.370+0000 7f0b4b7fe640 1 -- 172.21.15.105:0/2173941331 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b440a56c0 con 0x7f0b4c0cfee0 2023-11-28T18:27:58.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.370+0000 7f0b4b7fe640 1 -- 172.21.15.105:0/2173941331 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b440a5a50 con 0x7f0b4c0cfee0 2023-11-28T18:27:58.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.370+0000 7f0b52e32640 1 -- 172.21.15.105:0/2173941331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b4c0cfee0 msgr2=0x7f0b4c0d0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:58.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.370+0000 7f0b52e32640 1 --2- 172.21.15.105:0/2173941331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b4c0cfee0 0x7f0b4c0d0ef0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c08c9b0 tx=0x7f0b4401c6e0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.370+0000 7f0b52e32640 1 -- 172.21.15.105:0/2173941331 shutdown_connections 2023-11-28T18:27:58.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.370+0000 7f0b52e32640 1 --2- 172.21.15.105:0/2173941331 >> v2:172.21.15.148:3300/0 conn(0x7f0b4c0ceb40 0x7f0b4c0cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.370+0000 7f0b52e32640 1 --2- 172.21.15.105:0/2173941331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b4c0cfee0 0x7f0b4c0d0ef0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.370+0000 7f0b52e32640 1 --2- 172.21.15.105:0/2173941331 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0b4c0b8a20 0x7f0b4c0cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.370+0000 7f0b52e32640 1 -- 172.21.15.105:0/2173941331 >> 172.21.15.105:0/2173941331 conn(0x7f0b4c09c340 msgr2=0x7f0b4c0d2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:58.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.370+0000 7f0b52e32640 1 -- 172.21.15.105:0/2173941331 shutdown_connections 2023-11-28T18:27:58.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.370+0000 7f0b52e32640 1 -- 172.21.15.105:0/2173941331 wait complete. 2023-11-28T18:27:58.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b52e32640 1 Processor -- start 2023-11-28T18:27:58.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b52e32640 1 -- start start 2023-11-28T18:27:58.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b52e32640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b4c0b8a20 0x7f0b4c155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:58.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b52e32640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0b4c0ceb40 0x7f0b4c1583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:58.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b52e32640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b4c0cfee0 0x7f0b4c0ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:58.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b52e32640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0cd220 con 0x7f0b4c0b8a20 2023-11-28T18:27:58.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b52e32640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0b3d60 con 0x7f0b4c0cfee0 2023-11-28T18:27:58.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b52e32640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0cc980 con 0x7f0b4c0ceb40 2023-11-28T18:27:58.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b50bce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b4c0b8a20 0x7f0b4c155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:58.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b50bce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b4c0b8a20 0x7f0b4c155480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33032/0 (socket says 172.21.15.105:33032) 2023-11-28T18:27:58.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b50bce640 1 -- 172.21.15.105:0/568825188 learned_addr learned my addr 172.21.15.105:0/568825188 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:58.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b4bfff640 1 --2- 172.21.15.105:0/568825188 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0b4c0ceb40 0x7f0b4c1583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:58.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b513cf640 1 --2- 172.21.15.105:0/568825188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b4c0cfee0 0x7f0b4c0ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:58.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b50bce640 1 -- 172.21.15.105:0/568825188 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0b4c0ceb40 msgr2=0x7f0b4c1583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:58.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b50bce640 1 --2- 172.21.15.105:0/568825188 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0b4c0ceb40 0x7f0b4c1583a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b50bce640 1 -- 172.21.15.105:0/568825188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b4c0cfee0 msgr2=0x7f0b4c0ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:58.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b50bce640 1 --2- 172.21.15.105:0/568825188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b4c0cfee0 0x7f0b4c0ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b50bce640 1 -- 172.21.15.105:0/568825188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b44009b30 con 0x7f0b4c0b8a20 2023-11-28T18:27:58.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b50bce640 1 --2- 172.21.15.105:0/568825188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b4c0b8a20 0x7f0b4c155480 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400e8b0 tx=0x7f0b3400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:58.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b497fa640 1 -- 172.21.15.105:0/568825188 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3400c940 con 0x7f0b4c0b8a20 2023-11-28T18:27:58.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b497fa640 1 -- 172.21.15.105:0/568825188 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3400caa0 con 0x7f0b4c0b8a20 2023-11-28T18:27:58.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b52e32640 1 -- 172.21.15.105:0/568825188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b4c0ccc60 con 0x7f0b4c0b8a20 2023-11-28T18:27:58.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b497fa640 1 -- 172.21.15.105:0/568825188 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3400ce30 con 0x7f0b4c0b8a20 2023-11-28T18:27:58.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.374+0000 7f0b52e32640 1 -- 172.21.15.105:0/568825188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b4c155e00 con 0x7f0b4c0b8a20 2023-11-28T18:27:58.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.378+0000 7f0b52e32640 1 -- 172.21.15.105:0/568825188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b4c155c60 con 0x7f0b4c0b8a20 2023-11-28T18:27:58.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.378+0000 7f0b497fa640 1 -- 172.21.15.105:0/568825188 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f0b34013470 con 0x7f0b4c0b8a20 2023-11-28T18:27:58.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.378+0000 7f0b497fa640 1 --2- 172.21.15.105:0/568825188 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0b2c0668d0 0x7f0b2c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:58.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.378+0000 7f0b497fa640 1 -- 172.21.15.105:0/568825188 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f0b34084830 con 0x7f0b4c0b8a20 2023-11-28T18:27:58.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.378+0000 7f0b4bfff640 1 --2- 172.21.15.105:0/568825188 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0b2c0668d0 0x7f0b2c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:58.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.378+0000 7f0b4bfff640 1 --2- 172.21.15.105:0/568825188 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0b2c0668d0 0x7f0b2c068d90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c158df0 tx=0x7f0b3c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:58.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.382+0000 7f0b497fa640 1 -- 172.21.15.105:0/568825188 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0b4c155c60 con 0x7f0b4c0b8a20 2023-11-28T18:27:58.538 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.534+0000 7f0b52e32640 1 -- 172.21.15.105:0/568825188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0b4c155c60 con 0x7f0b4c0b8a20 2023-11-28T18:27:58.538 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.534+0000 7f0b497fa640 1 -- 172.21.15.105:0/568825188 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v67) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0b4c155c60 con 0x7f0b4c0b8a20 2023-11-28T18:27:58.538 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:27:58.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.534+0000 7f0b52e32640 1 -- 172.21.15.105:0/568825188 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0b2c0668d0 msgr2=0x7f0b2c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:58.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.534+0000 7f0b52e32640 1 --2- 172.21.15.105:0/568825188 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0b2c0668d0 0x7f0b2c068d90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c158df0 tx=0x7f0b3c008040 comp rx=0 tx=0).stop 2023-11-28T18:27:58.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.534+0000 7f0b52e32640 1 -- 172.21.15.105:0/568825188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b4c0b8a20 msgr2=0x7f0b4c155480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:58.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.534+0000 7f0b52e32640 1 --2- 172.21.15.105:0/568825188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b4c0b8a20 0x7f0b4c155480 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400e8b0 tx=0x7f0b3400ed70 comp rx=0 tx=0).stop 2023-11-28T18:27:58.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.534+0000 7f0b52e32640 1 -- 172.21.15.105:0/568825188 shutdown_connections 2023-11-28T18:27:58.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.534+0000 7f0b52e32640 1 --2- 172.21.15.105:0/568825188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b4c0cfee0 0x7f0b4c0ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.534+0000 7f0b52e32640 1 --2- 172.21.15.105:0/568825188 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0b4c0ceb40 0x7f0b4c1583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.534+0000 7f0b52e32640 1 --2- 172.21.15.105:0/568825188 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0b2c0668d0 0x7f0b2c068d90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.534+0000 7f0b52e32640 1 --2- 172.21.15.105:0/568825188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b4c0b8a20 0x7f0b4c155480 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.534+0000 7f0b52e32640 1 -- 172.21.15.105:0/568825188 >> 172.21.15.105:0/568825188 conn(0x7f0b4c09c340 msgr2=0x7f0b4c0b5360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:58.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.534+0000 7f0b52e32640 1 -- 172.21.15.105:0/568825188 shutdown_connections 2023-11-28T18:27:58.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.534+0000 7f0b52e32640 1 -- 172.21.15.105:0/568825188 wait complete. 2023-11-28T18:27:58.559 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:27:58.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.694+0000 7f593ee9a640 1 Processor -- start 2023-11-28T18:27:58.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.694+0000 7f593ee9a640 1 -- start start 2023-11-28T18:27:58.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.694+0000 7f593ee9a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59380d3c30 0x7f59380d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:58.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.694+0000 7f593ee9a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f59380b2d20 0x7f59380b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:58.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.694+0000 7f593ee9a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59380b25b0 con 0x7f593815d4f0 2023-11-28T18:27:58.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.694+0000 7f593ee9a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59380b2900 con 0x7f59380b2d20 2023-11-28T18:27:58.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.694+0000 7f593ee9a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59380d0a20 con 0x7f59380d3c30 2023-11-28T18:27:58.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.694+0000 7f593d437640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f59380b2d20 0x7f59380b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:58.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.694+0000 7f593d437640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f59380b2d20 0x7f59380b31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33048/0 (socket says 172.21.15.105:33048) 2023-11-28T18:27:58.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.694+0000 7f593d437640 1 -- 172.21.15.105:0/3919990620 learned_addr learned my addr 172.21.15.105:0/3919990620 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:58.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.694+0000 7f592ffff640 1 --2- 172.21.15.105:0/3919990620 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59380d3c30 0x7f59380d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:58.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.694+0000 7f593d437640 1 -- 172.21.15.105:0/3919990620 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59380d3c30 msgr2=0x7f59380d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:58.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.694+0000 7f593d437640 1 --2- 172.21.15.105:0/3919990620 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59380d3c30 0x7f59380d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.694+0000 7f593d437640 1 -- 172.21.15.105:0/3919990620 >> v1:172.21.15.148:6789/0 conn(0x7f593815d4f0 legacy=0x7f59380d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:58.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.694+0000 7f593d437640 1 -- 172.21.15.105:0/3919990620 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59380cea30 con 0x7f59380b2d20 2023-11-28T18:27:58.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.698+0000 7f593d437640 1 --2- 172.21.15.105:0/3919990620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59380b2d20 0x7f59380b31d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f5930009e50 tx=0x7f593001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b03e323ec5cca6c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:58.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.698+0000 7f592f7fe640 1 -- 172.21.15.105:0/3919990620 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59300a5810 con 0x7f59380b2d20 2023-11-28T18:27:58.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.698+0000 7f592f7fe640 1 -- 172.21.15.105:0/3919990620 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59300a5970 con 0x7f59380b2d20 2023-11-28T18:27:58.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.698+0000 7f592f7fe640 1 -- 172.21.15.105:0/3919990620 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59300a5d00 con 0x7f59380b2d20 2023-11-28T18:27:58.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.698+0000 7f593ee9a640 1 -- 172.21.15.105:0/3919990620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59380b2d20 msgr2=0x7f59380b31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:58.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.698+0000 7f593ee9a640 1 --2- 172.21.15.105:0/3919990620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59380b2d20 0x7f59380b31d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f5930009e50 tx=0x7f593001c950 comp rx=0 tx=0).stop 2023-11-28T18:27:58.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.698+0000 7f593ee9a640 1 -- 172.21.15.105:0/3919990620 shutdown_connections 2023-11-28T18:27:58.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.698+0000 7f593ee9a640 1 --2- 172.21.15.105:0/3919990620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59380b2d20 0x7f59380b31d0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.698+0000 7f593ee9a640 1 --2- 172.21.15.105:0/3919990620 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59380d3c30 0x7f59380d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.698+0000 7f593ee9a640 1 -- 172.21.15.105:0/3919990620 >> 172.21.15.105:0/3919990620 conn(0x7f59380d3030 msgr2=0x7f59380d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:58.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.698+0000 7f593ee9a640 1 -- 172.21.15.105:0/3919990620 shutdown_connections 2023-11-28T18:27:58.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.698+0000 7f593ee9a640 1 -- 172.21.15.105:0/3919990620 wait complete. 2023-11-28T18:27:58.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f593ee9a640 1 Processor -- start 2023-11-28T18:27:58.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f593ee9a640 1 -- start start 2023-11-28T18:27:58.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f593ee9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59380b2d20 0x7f593814f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:58.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f593ee9a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59380d3c30 0x7f59380ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:58.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f593ee9a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f593815d4f0 0x7f59380cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:58.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f593ee9a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59380ccf80 con 0x7f59380b2d20 2023-11-28T18:27:58.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f593ee9a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593815ca00 con 0x7f593815d4f0 2023-11-28T18:27:58.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f593ee9a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59380cc9c0 con 0x7f59380d3c30 2023-11-28T18:27:58.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f593cc36640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59380b2d20 0x7f593814f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:58.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f593cc36640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59380b2d20 0x7f593814f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33052/0 (socket says 172.21.15.105:33052) 2023-11-28T18:27:58.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f593cc36640 1 -- 172.21.15.105:0/4092694897 learned_addr learned my addr 172.21.15.105:0/4092694897 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:58.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f592ffff640 1 --2- 172.21.15.105:0/4092694897 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59380d3c30 0x7f59380ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:58.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f593d437640 1 --2- 172.21.15.105:0/4092694897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f593815d4f0 0x7f59380cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:58.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f593cc36640 1 -- 172.21.15.105:0/4092694897 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59380d3c30 msgr2=0x7f59380ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:58.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f593cc36640 1 --2- 172.21.15.105:0/4092694897 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59380d3c30 0x7f59380ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f593cc36640 1 -- 172.21.15.105:0/4092694897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f593815d4f0 msgr2=0x7f59380cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:58.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f593cc36640 1 --2- 172.21.15.105:0/4092694897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f593815d4f0 0x7f59380cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f593cc36640 1 -- 172.21.15.105:0/4092694897 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5930009b30 con 0x7f59380b2d20 2023-11-28T18:27:58.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f592ffff640 1 --2- 172.21.15.105:0/4092694897 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59380d3c30 0x7f59380ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:27:58.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f593d437640 1 --2- 172.21.15.105:0/4092694897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f593815d4f0 0x7f59380cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:27:58.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f593cc36640 1 --2- 172.21.15.105:0/4092694897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59380b2d20 0x7f593814f480 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f593400c3f0 tx=0x7f593400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:58.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f592d7fa640 1 -- 172.21.15.105:0/4092694897 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59340052d0 con 0x7f59380b2d20 2023-11-28T18:27:58.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f592d7fa640 1 -- 172.21.15.105:0/4092694897 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5934005430 con 0x7f59380b2d20 2023-11-28T18:27:58.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f593ee9a640 1 -- 172.21.15.105:0/4092694897 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59380ccca0 con 0x7f59380b2d20 2023-11-28T18:27:58.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f592d7fa640 1 -- 172.21.15.105:0/4092694897 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5934003840 con 0x7f59380b2d20 2023-11-28T18:27:58.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f593ee9a640 1 -- 172.21.15.105:0/4092694897 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f593814f9c0 con 0x7f59380b2d20 2023-11-28T18:27:58.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.702+0000 7f593ee9a640 1 -- 172.21.15.105:0/4092694897 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5938067570 con 0x7f59380b2d20 2023-11-28T18:27:58.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.706+0000 7f592d7fa640 1 -- 172.21.15.105:0/4092694897 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f59340039a0 con 0x7f59380b2d20 2023-11-28T18:27:58.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.706+0000 7f592d7fa640 1 --2- 172.21.15.105:0/4092694897 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5924066900 0x7f5924068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:58.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.706+0000 7f592d7fa640 1 -- 172.21.15.105:0/4092694897 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f5934083b10 con 0x7f59380b2d20 2023-11-28T18:27:58.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.706+0000 7f592ffff640 1 --2- 172.21.15.105:0/4092694897 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5924066900 0x7f5924068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:58.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.706+0000 7f592ffff640 1 --2- 172.21.15.105:0/4092694897 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5924066900 0x7f5924068dc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f59381f9290 tx=0x7f5928008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:58.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.710+0000 7f592d7fa640 1 -- 172.21.15.105:0/4092694897 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5934050bf0 con 0x7f59380b2d20 2023-11-28T18:27:58.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.890+0000 7f593ee9a640 1 -- 172.21.15.105:0/4092694897 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5938155620 con 0x7f59380b2d20 2023-11-28T18:27:58.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.890+0000 7f592d7fa640 1 -- 172.21.15.105:0/4092694897 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f5934017090 con 0x7f59380b2d20 2023-11-28T18:27:58.894 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:27:58.895 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":62,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/234000461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":234000461},{"type":"v1","addr":"172.21.15.148:6834","nonce":234000461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:27:05.479811+0000","modified":"2023-11-28T18:27:35.595155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6720,"mds_1":6717,"mds_2":6436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6436":{"gid":6436,"name":"b","rank":2,"incarnation":60,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6835/2396073410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2396073410},{"type":"v1","addr":"172.21.15.148:6835","nonce":2396073410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6717":{"gid":6717,"name":"a","rank":1,"incarnation":58,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6835/380088531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":380088531},{"type":"v1","addr":"172.21.15.105:6835","nonce":380088531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6720":{"gid":6720,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/3442512486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3442512486},{"type":"v1","addr":"172.21.15.105:6837","nonce":3442512486}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T18:27:58.895 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 62 2023-11-28T18:27:58.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.890+0000 7f593ee9a640 1 -- 172.21.15.105:0/4092694897 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5924066900 msgr2=0x7f5924068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:58.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.890+0000 7f593ee9a640 1 --2- 172.21.15.105:0/4092694897 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5924066900 0x7f5924068dc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f59381f9290 tx=0x7f5928008040 comp rx=0 tx=0).stop 2023-11-28T18:27:58.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.890+0000 7f593ee9a640 1 -- 172.21.15.105:0/4092694897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59380b2d20 msgr2=0x7f593814f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:58.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.890+0000 7f593ee9a640 1 --2- 172.21.15.105:0/4092694897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59380b2d20 0x7f593814f480 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f593400c3f0 tx=0x7f593400c8b0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.890+0000 7f593ee9a640 1 -- 172.21.15.105:0/4092694897 shutdown_connections 2023-11-28T18:27:58.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.890+0000 7f593ee9a640 1 --2- 172.21.15.105:0/4092694897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f593815d4f0 0x7f59380cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.890+0000 7f593ee9a640 1 --2- 172.21.15.105:0/4092694897 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59380d3c30 0x7f59380ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.890+0000 7f593ee9a640 1 --2- 172.21.15.105:0/4092694897 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5924066900 0x7f5924068dc0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.890+0000 7f593ee9a640 1 --2- 172.21.15.105:0/4092694897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59380b2d20 0x7f593814f480 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:58.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.890+0000 7f593ee9a640 1 -- 172.21.15.105:0/4092694897 >> 172.21.15.105:0/4092694897 conn(0x7f59380d3030 msgr2=0x7f593806b510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:58.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.890+0000 7f593ee9a640 1 -- 172.21.15.105:0/4092694897 shutdown_connections 2023-11-28T18:27:58.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:58.890+0000 7f593ee9a640 1 -- 172.21.15.105:0/4092694897 wait complete. 2023-11-28T18:27:58.915 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:27:59.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.046+0000 7f665069a640 1 Processor -- start 2023-11-28T18:27:59.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.046+0000 7f665069a640 1 -- start start 2023-11-28T18:27:59.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f665069a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f66480ae290 0x7f664815d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:59.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f665069a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66480cc7f0 con 0x7f6648154070 2023-11-28T18:27:59.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f665069a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664815dd10 con 0x7f664806bf50 2023-11-28T18:27:59.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f665069a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6648072540 con 0x7f66480ae290 2023-11-28T18:27:59.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664e436640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f6648154070 0x7f66480ce310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46252/0 (socket says 172.21.15.105:46252) 2023-11-28T18:27:59.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664e436640 1 -- 172.21.15.105:0/781877188 learned_addr learned my addr 172.21.15.105:0/781877188 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664d434640 1 -- 172.21.15.105:0/781877188 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918877249 0 0) 0x7f66480cc7f0 con 0x7f6648154070 2023-11-28T18:27:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664ec37640 1 --2- 172.21.15.105:0/781877188 >> v2:172.21.15.148:3300/0 conn(0x7f66480ae290 0x7f664815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664d434640 1 -- 172.21.15.105:0/781877188 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6634003530 con 0x7f6648154070 2023-11-28T18:27:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664d434640 1 -- 172.21.15.105:0/781877188 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3324168650 0 0) 0x7f6634003530 con 0x7f6648154070 2023-11-28T18:27:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664d434640 1 -- 172.21.15.105:0/781877188 >> v2:172.21.15.148:3300/0 conn(0x7f66480ae290 msgr2=0x7f664815d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664d434640 1 --2- 172.21.15.105:0/781877188 >> v2:172.21.15.148:3300/0 conn(0x7f66480ae290 0x7f664815d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664d434640 1 -- 172.21.15.105:0/781877188 >> v1:172.21.15.148:6789/0 conn(0x7f664806bf50 legacy=0x7f66480adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664d434640 1 -- 172.21.15.105:0/781877188 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664815dff0 con 0x7f6648154070 2023-11-28T18:27:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664d434640 1 -- 172.21.15.105:0/781877188 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f6644001680 con 0x7f6648154070 2023-11-28T18:27:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664d434640 1 -- 172.21.15.105:0/781877188 >> v1:172.21.15.105:6789/0 conn(0x7f6648154070 legacy=0x7f66480ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664d434640 1 --2- 172.21.15.105:0/781877188 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6634003980 0x7f6634003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664d434640 1 --2- 172.21.15.105:0/781877188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66340047d0 0x7f6634004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664d434640 1 --2- 172.21.15.105:0/781877188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6634005190 0x7f663400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664d434640 1 -- 172.21.15.105:0/781877188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6648072540 con 0x7f66340047d0 2023-11-28T18:27:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664d434640 1 -- 172.21.15.105:0/781877188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663400c030 con 0x7f6634005190 2023-11-28T18:27:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664d434640 1 -- 172.21.15.105:0/781877188 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663400c1f0 con 0x7f6634003980 2023-11-28T18:27:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664dc35640 1 --2- 172.21.15.105:0/781877188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66340047d0 0x7f6634004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664ec37640 1 --2- 172.21.15.105:0/781877188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6634005190 0x7f663400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664e436640 1 --2- 172.21.15.105:0/781877188 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6634003980 0x7f6634003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664dc35640 1 -- 172.21.15.105:0/781877188 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6634003980 msgr2=0x7f6634003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664dc35640 1 --2- 172.21.15.105:0/781877188 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6634003980 0x7f6634003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664dc35640 1 -- 172.21.15.105:0/781877188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6634005190 msgr2=0x7f663400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:59.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664dc35640 1 --2- 172.21.15.105:0/781877188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6634005190 0x7f663400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664dc35640 1 -- 172.21.15.105:0/781877188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6634003530 con 0x7f66340047d0 2023-11-28T18:27:59.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.050+0000 7f664ec37640 1 -- 172.21.15.105:0/781877188 reap_dead start 2023-11-28T18:27:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.054+0000 7f664dc35640 1 --2- 172.21.15.105:0/781877188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66340047d0 0x7f6634004c50 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f6638001780 tx=0x7f663801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a872ce02f09dcfd8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.054+0000 7f664d434640 1 -- 172.21.15.105:0/781877188 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66380052d0 con 0x7f66340047d0 2023-11-28T18:27:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.054+0000 7f664d434640 1 -- 172.21.15.105:0/781877188 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66380a7050 con 0x7f66340047d0 2023-11-28T18:27:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.054+0000 7f664d434640 1 -- 172.21.15.105:0/781877188 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6638003600 con 0x7f66340047d0 2023-11-28T18:27:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.054+0000 7f665069a640 1 -- 172.21.15.105:0/781877188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66340047d0 msgr2=0x7f6634004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.054+0000 7f665069a640 1 --2- 172.21.15.105:0/781877188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66340047d0 0x7f6634004c50 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f6638001780 tx=0x7f663801c3e0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.054+0000 7f665069a640 1 -- 172.21.15.105:0/781877188 shutdown_connections 2023-11-28T18:27:59.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.054+0000 7f665069a640 1 --2- 172.21.15.105:0/781877188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66340047d0 0x7f6634004c50 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.054+0000 7f665069a640 1 -- 172.21.15.105:0/781877188 >> 172.21.15.105:0/781877188 conn(0x7f66480ceb90 msgr2=0x7f66480cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:59.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.054+0000 7f665069a640 1 -- 172.21.15.105:0/781877188 shutdown_connections 2023-11-28T18:27:59.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.054+0000 7f665069a640 1 -- 172.21.15.105:0/781877188 wait complete. 2023-11-28T18:27:59.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f665069a640 1 Processor -- start 2023-11-28T18:27:59.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f665069a640 1 -- start start 2023-11-28T18:27:59.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f665069a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66340047d0 0x7f664813fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:59.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f665069a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f66480ad5f0 0x7f6648142630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:59.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f665069a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6648099a50 0x7f6648142b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:59.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f665069a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6648150fa0 con 0x7f6648099a50 2023-11-28T18:27:59.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f665069a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66480ad160 con 0x7f66340047d0 2023-11-28T18:27:59.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f665069a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66480ac830 con 0x7f66480ad5f0 2023-11-28T18:27:59.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f664ec37640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6648099a50 0x7f6648142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:59.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f664ec37640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6648099a50 0x7f6648142b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33068/0 (socket says 172.21.15.105:33068) 2023-11-28T18:27:59.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f664ec37640 1 -- 172.21.15.105:0/4294179916 learned_addr learned my addr 172.21.15.105:0/4294179916 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:59.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f664e436640 1 --2- 172.21.15.105:0/4294179916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66340047d0 0x7f664813fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:59.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f664dc35640 1 --2- 172.21.15.105:0/4294179916 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f66480ad5f0 0x7f6648142630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:59.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f664ec37640 1 -- 172.21.15.105:0/4294179916 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f66480ad5f0 msgr2=0x7f6648142630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:59.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f664ec37640 1 --2- 172.21.15.105:0/4294179916 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f66480ad5f0 0x7f6648142630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f664ec37640 1 -- 172.21.15.105:0/4294179916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66340047d0 msgr2=0x7f664813fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:59.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f664ec37640 1 --2- 172.21.15.105:0/4294179916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66340047d0 0x7f664813fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f664ec37640 1 -- 172.21.15.105:0/4294179916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66480acb10 con 0x7f6648099a50 2023-11-28T18:27:59.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f664ec37640 1 --2- 172.21.15.105:0/4294179916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6648099a50 0x7f6648142b70 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f6648161d90 tx=0x7f664000f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:59.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f663effd640 1 -- 172.21.15.105:0/4294179916 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664000e040 con 0x7f6648099a50 2023-11-28T18:27:59.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f663effd640 1 -- 172.21.15.105:0/4294179916 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664000f040 con 0x7f6648099a50 2023-11-28T18:27:59.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f665069a640 1 -- 172.21.15.105:0/4294179916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66480acdf0 con 0x7f6648099a50 2023-11-28T18:27:59.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f663effd640 1 -- 172.21.15.105:0/4294179916 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6640015430 con 0x7f6648099a50 2023-11-28T18:27:59.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.058+0000 7f665069a640 1 -- 172.21.15.105:0/4294179916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6648099f50 con 0x7f6648099a50 2023-11-28T18:27:59.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.062+0000 7f665069a640 1 -- 172.21.15.105:0/4294179916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6648067570 con 0x7f6648099a50 2023-11-28T18:27:59.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.062+0000 7f663effd640 1 -- 172.21.15.105:0/4294179916 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f6640015590 con 0x7f6648099a50 2023-11-28T18:27:59.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.062+0000 7f663effd640 1 --2- 172.21.15.105:0/4294179916 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f662c0668d0 0x7f662c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:59.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.062+0000 7f663effd640 1 -- 172.21.15.105:0/4294179916 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f6640085070 con 0x7f6648099a50 2023-11-28T18:27:59.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.062+0000 7f664e436640 1 --2- 172.21.15.105:0/4294179916 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f662c0668d0 0x7f662c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:59.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.062+0000 7f664e436640 1 --2- 172.21.15.105:0/4294179916 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f662c0668d0 0x7f662c068d90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6644001060 tx=0x7f6644001c20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:59.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.066+0000 7f663effd640 1 -- 172.21.15.105:0/4294179916 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f66400520f0 con 0x7f6648099a50 2023-11-28T18:27:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.218+0000 7f665069a640 1 -- 172.21.15.105:0/4294179916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66480ad2a0 con 0x7f6648099a50 2023-11-28T18:27:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.222+0000 7f663effd640 1 -- 172.21.15.105:0/4294179916 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+17014 (secure 0 0 0) 0x7f6640055710 con 0x7f6648099a50 2023-11-28T18:27:59.226 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:27:59.227 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":67,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:27:16.173737+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:27:03.225100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T18:27:04.478619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.e","mappings":[{"from":7,"to":2}]},{"pgid":"10.c","mappings":[{"from":7,"to":3}]},{"pgid":"10.d","mappings":[{"from":4,"to":3}]},{"pgid":"10.19","mappings":[{"from":4,"to":5}]},{"pgid":"10.1d","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:27:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.222+0000 7f665069a640 1 -- 172.21.15.105:0/4294179916 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f662c0668d0 msgr2=0x7f662c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.222+0000 7f665069a640 1 --2- 172.21.15.105:0/4294179916 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f662c0668d0 0x7f662c068d90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6644001060 tx=0x7f6644001c20 comp rx=0 tx=0).stop 2023-11-28T18:27:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.222+0000 7f665069a640 1 -- 172.21.15.105:0/4294179916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6648099a50 msgr2=0x7f6648142b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.222+0000 7f665069a640 1 --2- 172.21.15.105:0/4294179916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6648099a50 0x7f6648142b70 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f6648161d90 tx=0x7f664000f1b0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.222+0000 7f665069a640 1 -- 172.21.15.105:0/4294179916 shutdown_connections 2023-11-28T18:27:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.222+0000 7f665069a640 1 --2- 172.21.15.105:0/4294179916 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f662c0668d0 0x7f662c068d90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.222+0000 7f665069a640 1 --2- 172.21.15.105:0/4294179916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6648099a50 0x7f6648142b70 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.222+0000 7f665069a640 1 --2- 172.21.15.105:0/4294179916 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f66480ad5f0 0x7f6648142630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.222+0000 7f665069a640 1 --2- 172.21.15.105:0/4294179916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66340047d0 0x7f664813fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.222+0000 7f665069a640 1 -- 172.21.15.105:0/4294179916 >> 172.21.15.105:0/4294179916 conn(0x7f66480ceb90 msgr2=0x7f66480cdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.222+0000 7f665069a640 1 -- 172.21.15.105:0/4294179916 shutdown_connections 2023-11-28T18:27:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.222+0000 7f665069a640 1 -- 172.21.15.105:0/4294179916 wait complete. 2023-11-28T18:27:59.247 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T18:27:59.248 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:27:59.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35eaa6b640 1 Processor -- start 2023-11-28T18:27:59.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35eaa6b640 1 -- start start 2023-11-28T18:27:59.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35eaa6b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35e40d3c30 0x7f35e40d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:59.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35eaa6b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f35e40b2d20 0x7f35e40b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:59.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35eaa6b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e40b25b0 con 0x7f35e415d4f0 2023-11-28T18:27:59.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35eaa6b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e40b2900 con 0x7f35e40b2d20 2023-11-28T18:27:59.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35eaa6b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e40d0a20 con 0x7f35e40d3c30 2023-11-28T18:27:59.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e8807640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f35e415d4f0 0x7f35e40d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46268/0 (socket says 172.21.15.105:46268) 2023-11-28T18:27:59.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e8807640 1 -- 172.21.15.105:0/168893120 learned_addr learned my addr 172.21.15.105:0/168893120 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:59.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e3fff640 1 --2- 172.21.15.105:0/168893120 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35e40d3c30 0x7f35e40d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:59.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e9008640 1 --2- 172.21.15.105:0/168893120 >> v2:172.21.15.148:3300/0 conn(0x7f35e40b2d20 0x7f35e40b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:59.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e37fe640 1 -- 172.21.15.105:0/168893120 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3564429006 0 0) 0x7f35e40b25b0 con 0x7f35e415d4f0 2023-11-28T18:27:59.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e37fe640 1 -- 172.21.15.105:0/168893120 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35d4003530 con 0x7f35e415d4f0 2023-11-28T18:27:59.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e37fe640 1 -- 172.21.15.105:0/168893120 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1078994080 0 0) 0x7f35d4003530 con 0x7f35e415d4f0 2023-11-28T18:27:59.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e37fe640 1 -- 172.21.15.105:0/168893120 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35e40d3c30 msgr2=0x7f35e40d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:59.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e37fe640 1 --2- 172.21.15.105:0/168893120 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35e40d3c30 0x7f35e40d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e37fe640 1 -- 172.21.15.105:0/168893120 >> v2:172.21.15.148:3300/0 conn(0x7f35e40b2d20 msgr2=0x7f35e40b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:59.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e37fe640 1 --2- 172.21.15.105:0/168893120 >> v2:172.21.15.148:3300/0 conn(0x7f35e40b2d20 0x7f35e40b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e37fe640 1 -- 172.21.15.105:0/168893120 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e40cea30 con 0x7f35e415d4f0 2023-11-28T18:27:59.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e37fe640 1 -- 172.21.15.105:0/168893120 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f35d8001680 con 0x7f35e415d4f0 2023-11-28T18:27:59.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e37fe640 1 -- 172.21.15.105:0/168893120 >> v1:172.21.15.105:6789/0 conn(0x7f35e415d4f0 legacy=0x7f35e40d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:59.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e37fe640 1 --2- 172.21.15.105:0/168893120 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35e40d3c30 0x7f35d4003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:59.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e37fe640 1 --2- 172.21.15.105:0/168893120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d4003f00 0x7f35d40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:59.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e37fe640 1 --2- 172.21.15.105:0/168893120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35d4004d10 0x7f35d4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:59.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e37fe640 1 -- 172.21.15.105:0/168893120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e40b2900 con 0x7f35d4004d10 2023-11-28T18:27:59.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e37fe640 1 -- 172.21.15.105:0/168893120 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e40d0a20 con 0x7f35d4003f00 2023-11-28T18:27:59.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e37fe640 1 -- 172.21.15.105:0/168893120 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d400b9e0 con 0x7f35e40d3c30 2023-11-28T18:27:59.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e9008640 1 --2- 172.21.15.105:0/168893120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35d4004d10 0x7f35d4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:59.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e8807640 1 --2- 172.21.15.105:0/168893120 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35e40d3c30 0x7f35d4003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:59.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e3fff640 1 --2- 172.21.15.105:0/168893120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d4003f00 0x7f35d40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:59.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e9008640 1 -- 172.21.15.105:0/168893120 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35e40d3c30 msgr2=0x7f35d4003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:59.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e9008640 1 --2- 172.21.15.105:0/168893120 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35e40d3c30 0x7f35d4003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e9008640 1 -- 172.21.15.105:0/168893120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d4003f00 msgr2=0x7f35d40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:59.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e9008640 1 --2- 172.21.15.105:0/168893120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d4003f00 0x7f35d40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.382+0000 7f35e9008640 1 -- 172.21.15.105:0/168893120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d4003530 con 0x7f35d4004d10 2023-11-28T18:27:59.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.386+0000 7f35e9008640 1 --2- 172.21.15.105:0/168893120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35d4004d10 0x7f35d4005190 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00e170 tx=0x7f35dc01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4b92b2adecd121 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:59.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.386+0000 7f35e37fe640 1 -- 172.21.15.105:0/168893120 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc0a5c90 con 0x7f35d4004d10 2023-11-28T18:27:59.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.386+0000 7f35e37fe640 1 -- 172.21.15.105:0/168893120 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35dc00f040 con 0x7f35d4004d10 2023-11-28T18:27:59.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.386+0000 7f35e37fe640 1 -- 172.21.15.105:0/168893120 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc0ad440 con 0x7f35d4004d10 2023-11-28T18:27:59.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.386+0000 7f35eaa6b640 1 -- 172.21.15.105:0/168893120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35d4004d10 msgr2=0x7f35d4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:59.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.386+0000 7f35eaa6b640 1 --2- 172.21.15.105:0/168893120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35d4004d10 0x7f35d4005190 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00e170 tx=0x7f35dc01cc80 comp rx=0 tx=0).stop 2023-11-28T18:27:59.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.386+0000 7f35e9008640 1 -- 172.21.15.105:0/168893120 reap_dead start 2023-11-28T18:27:59.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.386+0000 7f35eaa6b640 1 -- 172.21.15.105:0/168893120 shutdown_connections 2023-11-28T18:27:59.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.386+0000 7f35eaa6b640 1 -- 172.21.15.105:0/168893120 >> 172.21.15.105:0/168893120 conn(0x7f35e40d3030 msgr2=0x7f35e40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:59.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.386+0000 7f35eaa6b640 1 -- 172.21.15.105:0/168893120 shutdown_connections 2023-11-28T18:27:59.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.386+0000 7f35eaa6b640 1 -- 172.21.15.105:0/168893120 wait complete. 2023-11-28T18:27:59.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35eaa6b640 1 Processor -- start 2023-11-28T18:27:59.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35eaa6b640 1 -- start start 2023-11-28T18:27:59.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35eaa6b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d4004d10 0x7f35e4158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:59.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35eaa6b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e40b7be0 0x7f35e40b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:59.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35eaa6b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35e406ba80 0x7f35e406bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:59.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35eaa6b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e406c440 con 0x7f35e406ba80 2023-11-28T18:27:59.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35eaa6b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e415df20 con 0x7f35e40b7be0 2023-11-28T18:27:59.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35eaa6b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e415e7c0 con 0x7f35d4004d10 2023-11-28T18:27:59.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35e9008640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35e406ba80 0x7f35e406bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:59.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35e9008640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35e406ba80 0x7f35e406bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33076/0 (socket says 172.21.15.105:33076) 2023-11-28T18:27:59.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35e9008640 1 -- 172.21.15.105:0/3432793616 learned_addr learned my addr 172.21.15.105:0/3432793616 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:59.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35e8807640 1 --2- 172.21.15.105:0/3432793616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d4004d10 0x7f35e4158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:59.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35e9008640 1 -- 172.21.15.105:0/3432793616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d4004d10 msgr2=0x7f35e4158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:59.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35e3fff640 1 --2- 172.21.15.105:0/3432793616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e40b7be0 0x7f35e40b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:59.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35e9008640 1 --2- 172.21.15.105:0/3432793616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d4004d10 0x7f35e4158630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35e9008640 1 -- 172.21.15.105:0/3432793616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e40b7be0 msgr2=0x7f35e40b6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:59.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35e9008640 1 --2- 172.21.15.105:0/3432793616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e40b7be0 0x7f35e40b6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35e9008640 1 -- 172.21.15.105:0/3432793616 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e40b8140 con 0x7f35e406ba80 2023-11-28T18:27:59.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35e8807640 1 --2- 172.21.15.105:0/3432793616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d4004d10 0x7f35e4158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:27:59.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35e3fff640 1 --2- 172.21.15.105:0/3432793616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e40b7be0 0x7f35e40b6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:27:59.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35e9008640 1 --2- 172.21.15.105:0/3432793616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35e406ba80 0x7f35e406bf00 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00e170 tx=0x7f35dc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:59.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35e17fa640 1 -- 172.21.15.105:0/3432793616 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc003ba0 con 0x7f35e406ba80 2023-11-28T18:27:59.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35e17fa640 1 -- 172.21.15.105:0/3432793616 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35dc003d00 con 0x7f35e406ba80 2023-11-28T18:27:59.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35e17fa640 1 -- 172.21.15.105:0/3432793616 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc0ad870 con 0x7f35e406ba80 2023-11-28T18:27:59.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.390+0000 7f35eaa6b640 1 -- 172.21.15.105:0/3432793616 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e415e170 con 0x7f35e406ba80 2023-11-28T18:27:59.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.394+0000 7f35eaa6b640 1 -- 172.21.15.105:0/3432793616 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e415dc10 con 0x7f35e406ba80 2023-11-28T18:27:59.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.394+0000 7f35eaa6b640 1 -- 172.21.15.105:0/3432793616 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e4067570 con 0x7f35e406ba80 2023-11-28T18:27:59.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.394+0000 7f35e17fa640 1 -- 172.21.15.105:0/3432793616 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f35dc0b4040 con 0x7f35e406ba80 2023-11-28T18:27:59.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.394+0000 7f35e17fa640 1 --2- 172.21.15.105:0/3432793616 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f35c8066900 0x7f35c8068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:59.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.394+0000 7f35e17fa640 1 -- 172.21.15.105:0/3432793616 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f35dc11e3a0 con 0x7f35e406ba80 2023-11-28T18:27:59.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.394+0000 7f35e8807640 1 --2- 172.21.15.105:0/3432793616 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f35c8066900 0x7f35c8068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:59.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.394+0000 7f35e8807640 1 --2- 172.21.15.105:0/3432793616 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f35c8066900 0x7f35c8068dc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f35e40b8730 tx=0x7f35d8007900 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:59.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.398+0000 7f35e17fa640 1 -- 172.21.15.105:0/3432793616 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f35dc0eb400 con 0x7f35e406ba80 2023-11-28T18:27:59.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.578+0000 7f35eaa6b640 1 -- 172.21.15.105:0/3432793616 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35e415b410 con 0x7f35e406ba80 2023-11-28T18:27:59.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.578+0000 7f35e17fa640 1 -- 172.21.15.105:0/3432793616 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f35dc0eea20 con 0x7f35e406ba80 2023-11-28T18:27:59.585 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:27:59.585 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":62,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/234000461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":234000461},{"type":"v1","addr":"172.21.15.148:6834","nonce":234000461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:27:05.479811+0000","modified":"2023-11-28T18:27:35.595155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6720,"mds_1":6717,"mds_2":6436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6436":{"gid":6436,"name":"b","rank":2,"incarnation":60,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6835/2396073410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2396073410},{"type":"v1","addr":"172.21.15.148:6835","nonce":2396073410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6717":{"gid":6717,"name":"a","rank":1,"incarnation":58,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6835/380088531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":380088531},{"type":"v1","addr":"172.21.15.105:6835","nonce":380088531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6720":{"gid":6720,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/3442512486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3442512486},{"type":"v1","addr":"172.21.15.105:6837","nonce":3442512486}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T18:27:59.585 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 62 2023-11-28T18:27:59.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.582+0000 7f35eaa6b640 1 -- 172.21.15.105:0/3432793616 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f35c8066900 msgr2=0x7f35c8068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:59.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.582+0000 7f35eaa6b640 1 --2- 172.21.15.105:0/3432793616 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f35c8066900 0x7f35c8068dc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f35e40b8730 tx=0x7f35d8007900 comp rx=0 tx=0).stop 2023-11-28T18:27:59.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.582+0000 7f35eaa6b640 1 -- 172.21.15.105:0/3432793616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35e406ba80 msgr2=0x7f35e406bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:59.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.582+0000 7f35eaa6b640 1 --2- 172.21.15.105:0/3432793616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35e406ba80 0x7f35e406bf00 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00e170 tx=0x7f35dc009750 comp rx=0 tx=0).stop 2023-11-28T18:27:59.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.582+0000 7f35eaa6b640 1 -- 172.21.15.105:0/3432793616 shutdown_connections 2023-11-28T18:27:59.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.582+0000 7f35eaa6b640 1 --2- 172.21.15.105:0/3432793616 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f35c8066900 0x7f35c8068dc0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.582+0000 7f35eaa6b640 1 --2- 172.21.15.105:0/3432793616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35e406ba80 0x7f35e406bf00 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.582+0000 7f35eaa6b640 1 --2- 172.21.15.105:0/3432793616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e40b7be0 0x7f35e40b6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.582+0000 7f35eaa6b640 1 --2- 172.21.15.105:0/3432793616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d4004d10 0x7f35e4158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.582+0000 7f35eaa6b640 1 -- 172.21.15.105:0/3432793616 >> 172.21.15.105:0/3432793616 conn(0x7f35e40d3030 msgr2=0x7f35e415c980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:59.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.582+0000 7f35eaa6b640 1 -- 172.21.15.105:0/3432793616 shutdown_connections 2023-11-28T18:27:59.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.582+0000 7f35eaa6b640 1 -- 172.21.15.105:0/3432793616 wait complete. 2023-11-28T18:27:59.606 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:27:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.738+0000 7fcab469a640 1 Processor -- start 2023-11-28T18:27:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.738+0000 7fcab469a640 1 -- start start 2023-11-28T18:27:59.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.738+0000 7fcab469a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcaac0d1b00 0x7fcaac0c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:59.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.738+0000 7fcab469a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaac0d0c60 con 0x7fcaac15d300 2023-11-28T18:27:59.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.738+0000 7fcab469a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaac0d1f90 con 0x7fcaac0d1650 2023-11-28T18:27:59.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.738+0000 7fcab469a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaac0d1470 con 0x7fcaac0d1b00 2023-11-28T18:27:59.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.738+0000 7fcab2436640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fcaac15d300 0x7fcaac0d37a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46274/0 (socket says 172.21.15.105:46274) 2023-11-28T18:27:59.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.738+0000 7fcab2436640 1 -- 172.21.15.105:0/929902392 learned_addr learned my addr 172.21.15.105:0/929902392 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:27:59.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.738+0000 7fcab2c37640 1 --2- 172.21.15.105:0/929902392 >> v2:172.21.15.105:3300/0 conn(0x7fcaac0d1b00 0x7fcaac0c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:59.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.738+0000 7fcab2c37640 1 -- 172.21.15.105:0/929902392 >> v1:172.21.15.148:6789/0 conn(0x7fcaac0d1650 legacy=0x7fcaac0c7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:27:59.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.738+0000 7fcab2c37640 1 -- 172.21.15.105:0/929902392 >> v1:172.21.15.105:6789/0 conn(0x7fcaac15d300 legacy=0x7fcaac0d37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:59.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.738+0000 7fcab2c37640 1 -- 172.21.15.105:0/929902392 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaac0d0eb0 con 0x7fcaac0d1b00 2023-11-28T18:27:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.738+0000 7fcab1434640 1 -- 172.21.15.105:0/929902392 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1097561197 0 0) 0x7fcaac0d0c60 con 0x7fcaac15d300 2023-11-28T18:27:59.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.742+0000 7fcab2c37640 1 --2- 172.21.15.105:0/929902392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcaac0d1b00 0x7fcaac0c7c20 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009d90 tx=0x7fcaa401c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58e2e497ccc694a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:59.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.742+0000 7fcab1434640 1 -- 172.21.15.105:0/929902392 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa40a5870 con 0x7fcaac0d1b00 2023-11-28T18:27:59.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.742+0000 7fcab1434640 1 -- 172.21.15.105:0/929902392 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa40a59d0 con 0x7fcaac0d1b00 2023-11-28T18:27:59.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.742+0000 7fcab1434640 1 -- 172.21.15.105:0/929902392 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa40a5d60 con 0x7fcaac0d1b00 2023-11-28T18:27:59.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.742+0000 7fcab469a640 1 -- 172.21.15.105:0/929902392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcaac0d1b00 msgr2=0x7fcaac0c7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:59.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.742+0000 7fcab469a640 1 --2- 172.21.15.105:0/929902392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcaac0d1b00 0x7fcaac0c7c20 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009d90 tx=0x7fcaa401c8c0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.742+0000 7fcab469a640 1 -- 172.21.15.105:0/929902392 shutdown_connections 2023-11-28T18:27:59.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.742+0000 7fcab469a640 1 --2- 172.21.15.105:0/929902392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcaac0d1b00 0x7fcaac0c7c20 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.742+0000 7fcab469a640 1 -- 172.21.15.105:0/929902392 >> 172.21.15.105:0/929902392 conn(0x7fcaac09c360 msgr2=0x7fcaac0d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:59.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.742+0000 7fcab469a640 1 -- 172.21.15.105:0/929902392 shutdown_connections 2023-11-28T18:27:59.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.742+0000 7fcab469a640 1 -- 172.21.15.105:0/929902392 wait complete. 2023-11-28T18:27:59.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcab469a640 1 Processor -- start 2023-11-28T18:27:59.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcab469a640 1 -- start start 2023-11-28T18:27:59.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcab469a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaac0d1650 0x7fcaac14d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:59.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcab469a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcaac0d1b00 0x7fcaac0cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:59.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcab469a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcaac15d300 0x7fcaac0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:59.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcab469a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac0cd3a0 con 0x7fcaac0d1b00 2023-11-28T18:27:59.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcab469a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac0b7990 con 0x7fcaac0d1650 2023-11-28T18:27:59.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcab469a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac0b6910 con 0x7fcaac15d300 2023-11-28T18:27:59.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcab1c35640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcaac0d1b00 0x7fcaac0cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:59.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcab1c35640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcaac0d1b00 0x7fcaac0cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33102/0 (socket says 172.21.15.105:33102) 2023-11-28T18:27:59.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcab1c35640 1 -- 172.21.15.105:0/4163149558 learned_addr learned my addr 172.21.15.105:0/4163149558 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:27:59.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcab2436640 1 --2- 172.21.15.105:0/4163149558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaac0d1650 0x7fcaac14d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:59.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcab2c37640 1 --2- 172.21.15.105:0/4163149558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcaac15d300 0x7fcaac0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:59.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcab1c35640 1 -- 172.21.15.105:0/4163149558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcaac15d300 msgr2=0x7fcaac0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:59.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcab1c35640 1 --2- 172.21.15.105:0/4163149558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcaac15d300 0x7fcaac0cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcab1c35640 1 -- 172.21.15.105:0/4163149558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaac0d1650 msgr2=0x7fcaac14d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:59.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcab1c35640 1 --2- 172.21.15.105:0/4163149558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaac0d1650 0x7fcaac14d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcab1c35640 1 -- 172.21.15.105:0/4163149558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa4009a70 con 0x7fcaac0d1b00 2023-11-28T18:27:59.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcab2c37640 1 --2- 172.21.15.105:0/4163149558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcaac15d300 0x7fcaac0cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:27:59.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcab1c35640 1 --2- 172.21.15.105:0/4163149558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcaac0d1b00 0x7fcaac0cc920 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00c3c0 tx=0x7fca9c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:59.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcaa2ffd640 1 -- 172.21.15.105:0/4163149558 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c0052d0 con 0x7fcaac0d1b00 2023-11-28T18:27:59.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcaa2ffd640 1 -- 172.21.15.105:0/4163149558 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c005430 con 0x7fcaac0d1b00 2023-11-28T18:27:59.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcaa2ffd640 1 -- 172.21.15.105:0/4163149558 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c003840 con 0x7fcaac0d1b00 2023-11-28T18:27:59.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.746+0000 7fcab469a640 1 -- 172.21.15.105:0/4163149558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaac0b6bd0 con 0x7fcaac0d1b00 2023-11-28T18:27:59.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.750+0000 7fcab469a640 1 -- 172.21.15.105:0/4163149558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaac0b7190 con 0x7fcaac0d1b00 2023-11-28T18:27:59.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.750+0000 7fcab469a640 1 -- 172.21.15.105:0/4163149558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaac067570 con 0x7fcaac0d1b00 2023-11-28T18:27:59.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.750+0000 7fcaa2ffd640 1 -- 172.21.15.105:0/4163149558 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fca9c0039a0 con 0x7fcaac0d1b00 2023-11-28T18:27:59.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.750+0000 7fcaa2ffd640 1 --2- 172.21.15.105:0/4163149558 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fca90066900 0x7fca90068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:27:59.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.750+0000 7fcaa2ffd640 1 -- 172.21.15.105:0/4163149558 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fca9c083ac0 con 0x7fcaac0d1b00 2023-11-28T18:27:59.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.750+0000 7fcab2436640 1 --2- 172.21.15.105:0/4163149558 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fca90066900 0x7fca90068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:27:59.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.750+0000 7fcab2436640 1 --2- 172.21.15.105:0/4163149558 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fca90066900 0x7fca90068dc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8000f70 tx=0x7fcaa8007240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:27:59.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.754+0000 7fcaa2ffd640 1 -- 172.21.15.105:0/4163149558 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fca9c050bd0 con 0x7fcaac0d1b00 2023-11-28T18:27:59.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.934+0000 7fcab469a640 1 -- 172.21.15.105:0/4163149558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcaac0d0070 con 0x7fcaac0d1b00 2023-11-28T18:27:59.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.934+0000 7fcaa2ffd640 1 -- 172.21.15.105:0/4163149558 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4951 (secure 0 0 0) 0x7fca9c017090 con 0x7fcaac0d1b00 2023-11-28T18:27:59.940 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:27:59.940 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":62,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/234000461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":234000461},{"type":"v1","addr":"172.21.15.148:6834","nonce":234000461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:27:05.479811+0000","modified":"2023-11-28T18:27:35.595155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6720,"mds_1":6717,"mds_2":6436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6436":{"gid":6436,"name":"b","rank":2,"incarnation":60,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6835/2396073410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2396073410},{"type":"v1","addr":"172.21.15.148:6835","nonce":2396073410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6717":{"gid":6717,"name":"a","rank":1,"incarnation":58,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6835/380088531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":380088531},{"type":"v1","addr":"172.21.15.105:6835","nonce":380088531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6720":{"gid":6720,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/3442512486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3442512486},{"type":"v1","addr":"172.21.15.105:6837","nonce":3442512486}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T18:27:59.940 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 62 2023-11-28T18:27:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.938+0000 7fcab469a640 1 -- 172.21.15.105:0/4163149558 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fca90066900 msgr2=0x7fca90068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.938+0000 7fcab469a640 1 --2- 172.21.15.105:0/4163149558 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fca90066900 0x7fca90068dc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8000f70 tx=0x7fcaa8007240 comp rx=0 tx=0).stop 2023-11-28T18:27:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.938+0000 7fcab469a640 1 -- 172.21.15.105:0/4163149558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcaac0d1b00 msgr2=0x7fcaac0cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:27:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.938+0000 7fcab469a640 1 --2- 172.21.15.105:0/4163149558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcaac0d1b00 0x7fcaac0cc920 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00c3c0 tx=0x7fca9c00c880 comp rx=0 tx=0).stop 2023-11-28T18:27:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.938+0000 7fcab469a640 1 -- 172.21.15.105:0/4163149558 shutdown_connections 2023-11-28T18:27:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.938+0000 7fcab469a640 1 --2- 172.21.15.105:0/4163149558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcaac15d300 0x7fcaac0cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.938+0000 7fcab469a640 1 --2- 172.21.15.105:0/4163149558 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fca90066900 0x7fca90068dc0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.938+0000 7fcab469a640 1 --2- 172.21.15.105:0/4163149558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcaac0d1b00 0x7fcaac0cc920 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.938+0000 7fcab469a640 1 --2- 172.21.15.105:0/4163149558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaac0d1650 0x7fcaac14d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:27:59.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.938+0000 7fcab469a640 1 -- 172.21.15.105:0/4163149558 >> 172.21.15.105:0/4163149558 conn(0x7fcaac09c360 msgr2=0x7fcaac0cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:27:59.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.938+0000 7fcab469a640 1 -- 172.21.15.105:0/4163149558 shutdown_connections 2023-11-28T18:27:59.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:27:59.938+0000 7fcab469a640 1 -- 172.21.15.105:0/4163149558 wait complete. 2023-11-28T18:27:59.961 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:28:00.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.110+0000 7efc419a7640 1 Processor -- start 2023-11-28T18:28:00.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.110+0000 7efc419a7640 1 -- start start 2023-11-28T18:28:00.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.110+0000 7efc419a7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efc3c15d2d0 0x7efc3c0cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:00.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.110+0000 7efc419a7640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3c0cfdb0 con 0x7efc3c15cef0 2023-11-28T18:28:00.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.110+0000 7efc419a7640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3c0ce840 con 0x7efc3c0cf960 2023-11-28T18:28:00.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.110+0000 7efc419a7640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc3c0b4780 con 0x7efc3c15d2d0 2023-11-28T18:28:00.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.110+0000 7efc409a5640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7efc3c15cef0 0x7efc3c0d1600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46282/0 (socket says 172.21.15.105:46282) 2023-11-28T18:28:00.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.110+0000 7efc409a5640 1 -- 172.21.15.105:0/2206579350 learned_addr learned my addr 172.21.15.105:0/2206579350 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:00.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.110+0000 7efc3bfff640 1 --2- 172.21.15.105:0/2206579350 >> v2:172.21.15.148:3300/0 conn(0x7efc3c15d2d0 0x7efc3c0cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:00.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.110+0000 7efc3b7fe640 1 -- 172.21.15.105:0/2206579350 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 270553043 0 0) 0x7efc30001120 con 0x7efc3c15cef0 2023-11-28T18:28:00.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.110+0000 7efc3b7fe640 1 -- 172.21.15.105:0/2206579350 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc2c003530 con 0x7efc3c15cef0 2023-11-28T18:28:00.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.110+0000 7efc3b7fe640 1 -- 172.21.15.105:0/2206579350 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729286008 0 0) 0x7efc3c0ce840 con 0x7efc3c0cf960 2023-11-28T18:28:00.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.110+0000 7efc3b7fe640 1 -- 172.21.15.105:0/2206579350 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc30001120 con 0x7efc3c0cf960 2023-11-28T18:28:00.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.114+0000 7efc3bfff640 1 -- 172.21.15.105:0/2206579350 >> v1:172.21.15.148:6789/0 conn(0x7efc3c0cf960 legacy=0x7efc3c166fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:00.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.114+0000 7efc3bfff640 1 -- 172.21.15.105:0/2206579350 >> v1:172.21.15.105:6789/0 conn(0x7efc3c15cef0 legacy=0x7efc3c0d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:00.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.114+0000 7efc3b7fe640 1 -- 172.21.15.105:0/2206579350 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3467911304 0 0) 0x7efc2c003530 con 0x7efc3c15cef0 2023-11-28T18:28:00.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.114+0000 7efc3bfff640 1 -- 172.21.15.105:0/2206579350 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c0cd990 con 0x7efc3c15d2d0 2023-11-28T18:28:00.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.114+0000 7efc3bfff640 1 --2- 172.21.15.105:0/2206579350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc3c15d2d0 0x7efc3c0cf420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efc280097c0 tx=0x7efc2801c2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=967c506d1d196c69 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:00.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.114+0000 7efc3b7fe640 1 -- 172.21.15.105:0/2206579350 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc30001810 con 0x7efc3c15d2d0 2023-11-28T18:28:00.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.114+0000 7efc3b7fe640 1 -- 172.21.15.105:0/2206579350 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc280a7050 con 0x7efc3c15d2d0 2023-11-28T18:28:00.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.114+0000 7efc3b7fe640 1 -- 172.21.15.105:0/2206579350 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc280ab620 con 0x7efc3c15d2d0 2023-11-28T18:28:00.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.114+0000 7efc419a7640 1 -- 172.21.15.105:0/2206579350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc3c15d2d0 msgr2=0x7efc3c0cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:00.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.114+0000 7efc419a7640 1 --2- 172.21.15.105:0/2206579350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc3c15d2d0 0x7efc3c0cf420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efc280097c0 tx=0x7efc2801c2f0 comp rx=0 tx=0).stop 2023-11-28T18:28:00.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc419a7640 1 -- 172.21.15.105:0/2206579350 shutdown_connections 2023-11-28T18:28:00.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc419a7640 1 --2- 172.21.15.105:0/2206579350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc3c15d2d0 0x7efc3c0cf420 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:00.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc419a7640 1 -- 172.21.15.105:0/2206579350 >> 172.21.15.105:0/2206579350 conn(0x7efc3c0d2e10 msgr2=0x7efc3c0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:00.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc419a7640 1 -- 172.21.15.105:0/2206579350 shutdown_connections 2023-11-28T18:28:00.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc419a7640 1 -- 172.21.15.105:0/2206579350 wait complete. 2023-11-28T18:28:00.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc419a7640 1 Processor -- start 2023-11-28T18:28:00.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc419a7640 1 -- start start 2023-11-28T18:28:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc419a7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc3c0cf960 0x7efc3c1551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc419a7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc3c15cef0 0x7efc3c155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc419a7640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc3c15d2d0 0x7efc3c0c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc419a7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c155c50 con 0x7efc3c0cf960 2023-11-28T18:28:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc419a7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c0b5a00 con 0x7efc3c15cef0 2023-11-28T18:28:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc419a7640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c0b5ce0 con 0x7efc3c15d2d0 2023-11-28T18:28:00.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc409a5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc3c0cf960 0x7efc3c1551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:00.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc409a5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc3c0cf960 0x7efc3c1551d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33108/0 (socket says 172.21.15.105:33108) 2023-11-28T18:28:00.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc409a5640 1 -- 172.21.15.105:0/273122312 learned_addr learned my addr 172.21.15.105:0/273122312 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:00.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc3bfff640 1 --2- 172.21.15.105:0/273122312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc3c15cef0 0x7efc3c155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:00.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc411a6640 1 --2- 172.21.15.105:0/273122312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc3c15d2d0 0x7efc3c0c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:00.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc409a5640 1 -- 172.21.15.105:0/273122312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc3c15d2d0 msgr2=0x7efc3c0c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:00.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc409a5640 1 --2- 172.21.15.105:0/273122312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc3c15d2d0 0x7efc3c0c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:00.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc409a5640 1 -- 172.21.15.105:0/273122312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc3c15cef0 msgr2=0x7efc3c155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:00.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc409a5640 1 --2- 172.21.15.105:0/273122312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc3c15cef0 0x7efc3c155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:00.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc409a5640 1 -- 172.21.15.105:0/273122312 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc28009420 con 0x7efc3c0cf960 2023-11-28T18:28:00.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.118+0000 7efc3bfff640 1 --2- 172.21.15.105:0/273122312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc3c15cef0 0x7efc3c155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:00.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.122+0000 7efc409a5640 1 --2- 172.21.15.105:0/273122312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc3c0cf960 0x7efc3c1551d0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7efc3000cac0 tx=0x7efc3000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:00.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.122+0000 7efc397fa640 1 -- 172.21.15.105:0/273122312 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc30005cf0 con 0x7efc3c0cf960 2023-11-28T18:28:00.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.122+0000 7efc397fa640 1 -- 172.21.15.105:0/273122312 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc30005e50 con 0x7efc3c0cf960 2023-11-28T18:28:00.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.122+0000 7efc397fa640 1 -- 172.21.15.105:0/273122312 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc300072b0 con 0x7efc3c0cf960 2023-11-28T18:28:00.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.122+0000 7efc419a7640 1 -- 172.21.15.105:0/273122312 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3c0b5fc0 con 0x7efc3c0cf960 2023-11-28T18:28:00.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.122+0000 7efc419a7640 1 -- 172.21.15.105:0/273122312 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3c0b6580 con 0x7efc3c0cf960 2023-11-28T18:28:00.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.122+0000 7efc419a7640 1 -- 172.21.15.105:0/273122312 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc3c0b6770 con 0x7efc3c0cf960 2023-11-28T18:28:00.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.122+0000 7efc397fa640 1 -- 172.21.15.105:0/273122312 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7efc30007410 con 0x7efc3c0cf960 2023-11-28T18:28:00.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.122+0000 7efc397fa640 1 --2- 172.21.15.105:0/273122312 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc200668d0 0x7efc20068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:00.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.122+0000 7efc397fa640 1 -- 172.21.15.105:0/273122312 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7efc30083e40 con 0x7efc3c0cf960 2023-11-28T18:28:00.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.122+0000 7efc3bfff640 1 --2- 172.21.15.105:0/273122312 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc200668d0 0x7efc20068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:00.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.126+0000 7efc3bfff640 1 --2- 172.21.15.105:0/273122312 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc200668d0 0x7efc20068d90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efc280ad000 tx=0x7efc280a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:00.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.130+0000 7efc397fa640 1 -- 172.21.15.105:0/273122312 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efc30050eb0 con 0x7efc3c0cf960 2023-11-28T18:28:00.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.286+0000 7efc419a7640 1 -- 172.21.15.105:0/273122312 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc3c0cfdb0 con 0x7efc3c0cf960 2023-11-28T18:28:00.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.286+0000 7efc397fa640 1 -- 172.21.15.105:0/273122312 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+17014 (secure 0 0 0) 0x7efc3c0cfdb0 con 0x7efc3c0cf960 2023-11-28T18:28:00.291 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:00.292 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":67,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:27:16.173737+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:27:03.225100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T18:27:04.478619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.e","mappings":[{"from":7,"to":2}]},{"pgid":"10.c","mappings":[{"from":7,"to":3}]},{"pgid":"10.d","mappings":[{"from":4,"to":3}]},{"pgid":"10.19","mappings":[{"from":4,"to":5}]},{"pgid":"10.1d","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:28:00.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.290+0000 7efc419a7640 1 -- 172.21.15.105:0/273122312 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc200668d0 msgr2=0x7efc20068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:00.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.290+0000 7efc419a7640 1 --2- 172.21.15.105:0/273122312 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc200668d0 0x7efc20068d90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efc280ad000 tx=0x7efc280a6000 comp rx=0 tx=0).stop 2023-11-28T18:28:00.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.290+0000 7efc419a7640 1 -- 172.21.15.105:0/273122312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc3c0cf960 msgr2=0x7efc3c1551d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:00.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.290+0000 7efc419a7640 1 --2- 172.21.15.105:0/273122312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc3c0cf960 0x7efc3c1551d0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7efc3000cac0 tx=0x7efc3000cf80 comp rx=0 tx=0).stop 2023-11-28T18:28:00.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.290+0000 7efc419a7640 1 -- 172.21.15.105:0/273122312 shutdown_connections 2023-11-28T18:28:00.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.290+0000 7efc419a7640 1 --2- 172.21.15.105:0/273122312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc3c15d2d0 0x7efc3c0c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:00.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.290+0000 7efc419a7640 1 --2- 172.21.15.105:0/273122312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc3c15cef0 0x7efc3c155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:00.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.290+0000 7efc419a7640 1 --2- 172.21.15.105:0/273122312 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc200668d0 0x7efc20068d90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:00.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.290+0000 7efc419a7640 1 --2- 172.21.15.105:0/273122312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc3c0cf960 0x7efc3c1551d0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:00.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.290+0000 7efc419a7640 1 -- 172.21.15.105:0/273122312 >> 172.21.15.105:0/273122312 conn(0x7efc3c0d2e10 msgr2=0x7efc3c06c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:00.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.290+0000 7efc419a7640 1 -- 172.21.15.105:0/273122312 shutdown_connections 2023-11-28T18:28:00.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.290+0000 7efc419a7640 1 -- 172.21.15.105:0/273122312 wait complete. 2023-11-28T18:28:00.313 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T18:28:00.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.450+0000 7ff6422fe640 1 Processor -- start 2023-11-28T18:28:00.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.450+0000 7ff6422fe640 1 -- start start 2023-11-28T18:28:00.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.450+0000 7ff6422fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff63c15d4f0 0x7ff63c0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:00.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.450+0000 7ff6422fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff63c0b25b0 con 0x7ff63c0d3c30 2023-11-28T18:28:00.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.450+0000 7ff6422fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff63c0cea30 con 0x7ff63c0b2d20 2023-11-28T18:28:00.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.450+0000 7ff6422fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff63c0b2900 con 0x7ff63c15d4f0 2023-11-28T18:28:00.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.450+0000 7ff63bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff63c15d4f0 0x7ff63c0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:00.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.450+0000 7ff63bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff63c15d4f0 0x7ff63c0d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33112/0 (socket says 172.21.15.105:33112) 2023-11-28T18:28:00.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.450+0000 7ff63b7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff63c0d3c30 0x7ff63c0d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46292/0 (socket says 172.21.15.105:46292) 2023-11-28T18:28:00.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.450+0000 7ff63bfff640 1 -- 172.21.15.105:0/1836233281 learned_addr learned my addr 172.21.15.105:0/1836233281 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:00.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.450+0000 7ff63bfff640 1 -- 172.21.15.105:0/1836233281 >> v1:172.21.15.148:6789/0 conn(0x7ff63c0b2d20 legacy=0x7ff63c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:00.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.450+0000 7ff63bfff640 1 -- 172.21.15.105:0/1836233281 >> v1:172.21.15.105:6789/0 conn(0x7ff63c0d3c30 legacy=0x7ff63c0d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:00.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.450+0000 7ff63bfff640 1 -- 172.21.15.105:0/1836233281 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63c0ced10 con 0x7ff63c15d4f0 2023-11-28T18:28:00.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.454+0000 7ff63bfff640 1 --2- 172.21.15.105:0/1836233281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff63c15d4f0 0x7ff63c0d0e10 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009870 tx=0x7ff62c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38d2173feecbc7c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:00.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.454+0000 7ff63affd640 1 -- 172.21.15.105:0/1836233281 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c0ac640 con 0x7ff63c15d4f0 2023-11-28T18:28:00.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.454+0000 7ff63affd640 1 -- 172.21.15.105:0/1836233281 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62c0ac7a0 con 0x7ff63c15d4f0 2023-11-28T18:28:00.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.454+0000 7ff63affd640 1 -- 172.21.15.105:0/1836233281 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c0acb30 con 0x7ff63c15d4f0 2023-11-28T18:28:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.454+0000 7ff6422fe640 1 -- 172.21.15.105:0/1836233281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff63c15d4f0 msgr2=0x7ff63c0d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.454+0000 7ff6422fe640 1 --2- 172.21.15.105:0/1836233281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff63c15d4f0 0x7ff63c0d0e10 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009870 tx=0x7ff62c01c440 comp rx=0 tx=0).stop 2023-11-28T18:28:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.454+0000 7ff6422fe640 1 -- 172.21.15.105:0/1836233281 shutdown_connections 2023-11-28T18:28:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.454+0000 7ff6422fe640 1 --2- 172.21.15.105:0/1836233281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff63c15d4f0 0x7ff63c0d0e10 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.454+0000 7ff6422fe640 1 -- 172.21.15.105:0/1836233281 >> 172.21.15.105:0/1836233281 conn(0x7ff63c0d3030 msgr2=0x7ff63c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.454+0000 7ff6422fe640 1 -- 172.21.15.105:0/1836233281 shutdown_connections 2023-11-28T18:28:00.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.454+0000 7ff6422fe640 1 -- 172.21.15.105:0/1836233281 wait complete. 2023-11-28T18:28:00.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.454+0000 7ff6422fe640 1 Processor -- start 2023-11-28T18:28:00.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff6422fe640 1 -- start start 2023-11-28T18:28:00.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff6422fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff63c0b2d20 0x7ff63c15db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:00.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff6422fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff63c0d3c30 0x7ff63c15e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:00.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff6422fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff63c15d4f0 0x7ff63c1f9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:00.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff6422fe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c0d0a20 con 0x7ff63c0b2d20 2023-11-28T18:28:00.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff6422fe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c15ec20 con 0x7ff63c0d3c30 2023-11-28T18:28:00.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff6422fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c152a50 con 0x7ff63c15d4f0 2023-11-28T18:28:00.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff63bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff63c0b2d20 0x7ff63c15db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:00.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff63bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff63c0b2d20 0x7ff63c15db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33114/0 (socket says 172.21.15.105:33114) 2023-11-28T18:28:00.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff63bfff640 1 -- 172.21.15.105:0/723215538 learned_addr learned my addr 172.21.15.105:0/723215538 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:00.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff63b7fe640 1 --2- 172.21.15.105:0/723215538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff63c0d3c30 0x7ff63c15e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:00.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff64089b640 1 --2- 172.21.15.105:0/723215538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff63c15d4f0 0x7ff63c1f9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:00.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff63bfff640 1 -- 172.21.15.105:0/723215538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff63c15d4f0 msgr2=0x7ff63c1f9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:00.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff63bfff640 1 --2- 172.21.15.105:0/723215538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff63c15d4f0 0x7ff63c1f9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:00.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff63bfff640 1 -- 172.21.15.105:0/723215538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff63c0d3c30 msgr2=0x7ff63c15e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:00.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff63bfff640 1 --2- 172.21.15.105:0/723215538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff63c0d3c30 0x7ff63c15e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:00.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff63bfff640 1 -- 172.21.15.105:0/723215538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62c0094d0 con 0x7ff63c0b2d20 2023-11-28T18:28:00.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff63bfff640 1 --2- 172.21.15.105:0/723215538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff63c0b2d20 0x7ff63c15db60 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7ff62c01a070 tx=0x7ff62c01ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:00.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff638ff9640 1 -- 172.21.15.105:0/723215538 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c0b7860 con 0x7ff63c0b2d20 2023-11-28T18:28:00.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff638ff9640 1 -- 172.21.15.105:0/723215538 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62c0b79c0 con 0x7ff63c0b2d20 2023-11-28T18:28:00.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff6422fe640 1 -- 172.21.15.105:0/723215538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff63c153590 con 0x7ff63c0b2d20 2023-11-28T18:28:00.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff638ff9640 1 -- 172.21.15.105:0/723215538 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c0b7d70 con 0x7ff63c0b2d20 2023-11-28T18:28:00.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff6422fe640 1 -- 172.21.15.105:0/723215538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff63c152f40 con 0x7ff63c0b2d20 2023-11-28T18:28:00.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.458+0000 7ff6422fe640 1 -- 172.21.15.105:0/723215538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff63c067570 con 0x7ff63c0b2d20 2023-11-28T18:28:00.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.462+0000 7ff638ff9640 1 -- 172.21.15.105:0/723215538 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff62c0b3070 con 0x7ff63c0b2d20 2023-11-28T18:28:00.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.462+0000 7ff638ff9640 1 --2- 172.21.15.105:0/723215538 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6280668d0 0x7ff628068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:00.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.462+0000 7ff638ff9640 1 -- 172.21.15.105:0/723215538 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6174+0+0 (secure 0 0 0) 0x7ff62c11f130 con 0x7ff63c0b2d20 2023-11-28T18:28:00.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.462+0000 7ff63b7fe640 1 --2- 172.21.15.105:0/723215538 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6280668d0 0x7ff628068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:00.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.462+0000 7ff63b7fe640 1 --2- 172.21.15.105:0/723215538 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6280668d0 0x7ff628068d90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff63c15e360 tx=0x7ff630007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:00.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.466+0000 7ff638ff9640 1 -- 172.21.15.105:0/723215538 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff62c0ec110 con 0x7ff63c0b2d20 2023-11-28T18:28:00.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.650+0000 7ff6422fe640 1 -- 172.21.15.105:0/723215538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff63c0cdc40 con 0x7ff63c0b2d20 2023-11-28T18:28:00.654 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-11-28T18:28:00.650+0000 7fd89b32a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T18:28:00.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.654+0000 7ff638ff9640 1 -- 172.21.15.105:0/723215538 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff62c0b1090 con 0x7ff63c0b2d20 2023-11-28T18:28:00.658 INFO:teuthology.orchestra.run.smithi105.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T18:28:00.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.654+0000 7ff6422fe640 1 -- 172.21.15.105:0/723215538 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6280668d0 msgr2=0x7ff628068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:00.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.654+0000 7ff6422fe640 1 --2- 172.21.15.105:0/723215538 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6280668d0 0x7ff628068d90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff63c15e360 tx=0x7ff630007c90 comp rx=0 tx=0).stop 2023-11-28T18:28:00.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.654+0000 7ff6422fe640 1 -- 172.21.15.105:0/723215538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff63c0b2d20 msgr2=0x7ff63c15db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:00.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.654+0000 7ff6422fe640 1 --2- 172.21.15.105:0/723215538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff63c0b2d20 0x7ff63c15db60 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7ff62c01a070 tx=0x7ff62c01ac80 comp rx=0 tx=0).stop 2023-11-28T18:28:00.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.654+0000 7ff6422fe640 1 -- 172.21.15.105:0/723215538 shutdown_connections 2023-11-28T18:28:00.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.654+0000 7ff6422fe640 1 --2- 172.21.15.105:0/723215538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff63c15d4f0 0x7ff63c1f9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:00.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.654+0000 7ff6422fe640 1 --2- 172.21.15.105:0/723215538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff63c0d3c30 0x7ff63c15e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:00.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.654+0000 7ff6422fe640 1 --2- 172.21.15.105:0/723215538 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6280668d0 0x7ff628068d90 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:00.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.654+0000 7ff6422fe640 1 --2- 172.21.15.105:0/723215538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff63c0b2d20 0x7ff63c15db60 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:00.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.654+0000 7ff6422fe640 1 -- 172.21.15.105:0/723215538 >> 172.21.15.105:0/723215538 conn(0x7ff63c0d3030 msgr2=0x7ff63c0afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:00.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.654+0000 7ff6422fe640 1 -- 172.21.15.105:0/723215538 shutdown_connections 2023-11-28T18:28:00.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.654+0000 7ff6422fe640 1 -- 172.21.15.105:0/723215538 wait complete. 2023-11-28T18:28:00.682 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T18:28:00.724 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2023-11-28T18:28:00.731 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2023-11-28T18:28:00.742 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2023-11-28T18:28:00.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.822+0000 7f3d35254640 1 Processor -- start 2023-11-28T18:28:00.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.822+0000 7f3d35254640 1 -- start start 2023-11-28T18:28:00.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.822+0000 7f3d35254640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3d30159f40 0x7f3d300caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:00.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.822+0000 7f3d35254640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d300cc760 0x7f3d300ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:00.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.822+0000 7f3d35254640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d3006bf50 con 0x7f3d300ad680 2023-11-28T18:28:00.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.822+0000 7f3d35254640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d300ccbc0 con 0x7f3d30159f40 2023-11-28T18:28:00.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.822+0000 7f3d35254640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3015a320 con 0x7f3d300cc760 2023-11-28T18:28:00.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.822+0000 7f3d2ed9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3d30159f40 0x7f3d300caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:00.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.826+0000 7f3d2ed9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3d30159f40 0x7f3d300caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33142/0 (socket says 172.21.15.105:33142) 2023-11-28T18:28:00.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.826+0000 7f3d2ed9d640 1 -- 172.21.15.105:0/1685163019 learned_addr learned my addr 172.21.15.105:0/1685163019 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:00.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.826+0000 7f3d2f59e640 1 --1- 172.21.15.105:0/1685163019 >> v1:172.21.15.105:6789/0 conn(0x7f3d300ad680 0x7f3d3015da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46310/0 (socket says 172.21.15.105:46310) 2023-11-28T18:28:00.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.826+0000 7f3d2659c640 1 --2- 172.21.15.105:0/1685163019 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d300cc760 0x7f3d300ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:00.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.826+0000 7f3d2ed9d640 1 -- 172.21.15.105:0/1685163019 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d300cc760 msgr2=0x7f3d300ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:00.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.826+0000 7f3d2ed9d640 1 --2- 172.21.15.105:0/1685163019 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d300cc760 0x7f3d300ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:00.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.826+0000 7f3d2ed9d640 1 -- 172.21.15.105:0/1685163019 >> v1:172.21.15.105:6789/0 conn(0x7f3d300ad680 legacy=0x7f3d3015da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:00.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.826+0000 7f3d2ed9d640 1 -- 172.21.15.105:0/1685163019 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d30071380 con 0x7f3d30159f40 2023-11-28T18:28:00.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.826+0000 7f3d2ed9d640 1 --2- 172.21.15.105:0/1685163019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d30159f40 0x7f3d300caf60 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f3d20009870 tx=0x7f3d2001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e44a0e7de42e4a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:00.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.826+0000 7f3d2e59c640 1 -- 172.21.15.105:0/1685163019 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d200ab680 con 0x7f3d30159f40 2023-11-28T18:28:00.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.826+0000 7f3d2e59c640 1 -- 172.21.15.105:0/1685163019 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d200ab7e0 con 0x7f3d30159f40 2023-11-28T18:28:00.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.826+0000 7f3d2e59c640 1 -- 172.21.15.105:0/1685163019 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d200abb70 con 0x7f3d30159f40 2023-11-28T18:28:00.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.830+0000 7f3d35254640 1 -- 172.21.15.105:0/1685163019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d30159f40 msgr2=0x7f3d300caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:00.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.830+0000 7f3d35254640 1 --2- 172.21.15.105:0/1685163019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d30159f40 0x7f3d300caf60 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f3d20009870 tx=0x7f3d2001c480 comp rx=0 tx=0).stop 2023-11-28T18:28:00.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.830+0000 7f3d35254640 1 -- 172.21.15.105:0/1685163019 shutdown_connections 2023-11-28T18:28:00.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.830+0000 7f3d35254640 1 --2- 172.21.15.105:0/1685163019 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d300cc760 0x7f3d300ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:00.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.830+0000 7f3d35254640 1 --2- 172.21.15.105:0/1685163019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d30159f40 0x7f3d300caf60 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:00.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.830+0000 7f3d35254640 1 -- 172.21.15.105:0/1685163019 >> 172.21.15.105:0/1685163019 conn(0x7f3d300ceb90 msgr2=0x7f3d300cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:00.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.830+0000 7f3d35254640 1 -- 172.21.15.105:0/1685163019 shutdown_connections 2023-11-28T18:28:00.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.830+0000 7f3d35254640 1 -- 172.21.15.105:0/1685163019 wait complete. 2023-11-28T18:28:00.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.830+0000 7f3d35254640 1 Processor -- start 2023-11-28T18:28:00.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.830+0000 7f3d35254640 1 -- start start 2023-11-28T18:28:00.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.830+0000 7f3d35254640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d300ad680 0x7f3d3014e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:00.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.830+0000 7f3d35254640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d300cc760 0x7f3d30152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:00.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.830+0000 7f3d35254640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d30159f40 0x7f3d300ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:00.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.830+0000 7f3d35254640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3015abd0 con 0x7f3d300cc760 2023-11-28T18:28:00.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.830+0000 7f3d35254640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3014ecb0 con 0x7f3d300ad680 2023-11-28T18:28:00.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.830+0000 7f3d35254640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3015a5c0 con 0x7f3d30159f40 2023-11-28T18:28:00.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.830+0000 7f3d2659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d300cc760 0x7f3d30152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:00.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.830+0000 7f3d2659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d300cc760 0x7f3d30152b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33154/0 (socket says 172.21.15.105:33154) 2023-11-28T18:28:00.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.830+0000 7f3d2659c640 1 -- 172.21.15.105:0/1360243310 learned_addr learned my addr 172.21.15.105:0/1360243310 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:00.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.834+0000 7f3d2ed9d640 1 --2- 172.21.15.105:0/1360243310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d300ad680 0x7f3d3014e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:00.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.834+0000 7f3d2f59e640 1 --2- 172.21.15.105:0/1360243310 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d30159f40 0x7f3d300ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:00.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.834+0000 7f3d2659c640 1 -- 172.21.15.105:0/1360243310 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d30159f40 msgr2=0x7f3d300ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:00.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.834+0000 7f3d2659c640 1 --2- 172.21.15.105:0/1360243310 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d30159f40 0x7f3d300ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:00.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.834+0000 7f3d2659c640 1 -- 172.21.15.105:0/1360243310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d300ad680 msgr2=0x7f3d3014e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:00.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.834+0000 7f3d2659c640 1 --2- 172.21.15.105:0/1360243310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d300ad680 0x7f3d3014e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.834+0000 7f3d2659c640 1 -- 172.21.15.105:0/1360243310 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d200094d0 con 0x7f3d300cc760 2023-11-28T18:28:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.834+0000 7f3d2f59e640 1 --2- 172.21.15.105:0/1360243310 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d30159f40 0x7f3d300ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.834+0000 7f3d2659c640 1 --2- 172.21.15.105:0/1360243310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d300cc760 0x7f3d30152b70 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800efc0 tx=0x7f3d1800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.834+0000 7f3d27fff640 1 -- 172.21.15.105:0/1360243310 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18009b80 con 0x7f3d300cc760 2023-11-28T18:28:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.834+0000 7f3d27fff640 1 -- 172.21.15.105:0/1360243310 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1800f040 con 0x7f3d300cc760 2023-11-28T18:28:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.834+0000 7f3d35254640 1 -- 172.21.15.105:0/1360243310 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d301534a0 con 0x7f3d300cc760 2023-11-28T18:28:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.834+0000 7f3d27fff640 1 -- 172.21.15.105:0/1360243310 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d180146a0 con 0x7f3d300cc760 2023-11-28T18:28:00.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.834+0000 7f3d35254640 1 -- 172.21.15.105:0/1360243310 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d30153780 con 0x7f3d300cc760 2023-11-28T18:28:00.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.834+0000 7f3d35254640 1 -- 172.21.15.105:0/1360243310 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d30067570 con 0x7f3d300cc760 2023-11-28T18:28:00.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.834+0000 7f3d27fff640 1 -- 172.21.15.105:0/1360243310 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3d18014800 con 0x7f3d300cc760 2023-11-28T18:28:00.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.834+0000 7f3d27fff640 1 --2- 172.21.15.105:0/1360243310 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d10066900 0x7f3d10068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:00.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.834+0000 7f3d27fff640 1 -- 172.21.15.105:0/1360243310 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6432+0+0 (secure 0 0 0) 0x7f3d18085480 con 0x7f3d300cc760 2023-11-28T18:28:00.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.834+0000 7f3d2ed9d640 1 --2- 172.21.15.105:0/1360243310 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d10066900 0x7f3d10068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:00.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.838+0000 7f3d2ed9d640 1 --2- 172.21.15.105:0/1360243310 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d10066900 0x7f3d10068dc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3d200af000 tx=0x7f3d200b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:00.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:00.838+0000 7f3d27fff640 1 -- 172.21.15.105:0/1360243310 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3d18052400 con 0x7f3d300cc760 2023-11-28T18:28:01.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.022+0000 7f3d35254640 1 -- 172.21.15.105:0/1360243310 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3d300cf950 con 0x7f3d300cc760 2023-11-28T18:28:01.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.654+0000 7f3d27fff640 1 -- 172.21.15.105:0/1360243310 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3d18055a20 con 0x7f3d300cc760 2023-11-28T18:28:01.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.658+0000 7f3d35254640 1 -- 172.21.15.105:0/1360243310 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d10066900 msgr2=0x7f3d10068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:01.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.658+0000 7f3d35254640 1 --2- 172.21.15.105:0/1360243310 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d10066900 0x7f3d10068dc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3d200af000 tx=0x7f3d200b6040 comp rx=0 tx=0).stop 2023-11-28T18:28:01.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.658+0000 7f3d35254640 1 -- 172.21.15.105:0/1360243310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d300cc760 msgr2=0x7f3d30152b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:01.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.658+0000 7f3d35254640 1 --2- 172.21.15.105:0/1360243310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d300cc760 0x7f3d30152b70 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800efc0 tx=0x7f3d1800c490 comp rx=0 tx=0).stop 2023-11-28T18:28:01.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.658+0000 7f3d35254640 1 -- 172.21.15.105:0/1360243310 shutdown_connections 2023-11-28T18:28:01.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.658+0000 7f3d35254640 1 --2- 172.21.15.105:0/1360243310 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d30159f40 0x7f3d300ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:01.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.658+0000 7f3d35254640 1 --2- 172.21.15.105:0/1360243310 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d10066900 0x7f3d10068dc0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:01.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.658+0000 7f3d35254640 1 --2- 172.21.15.105:0/1360243310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d300cc760 0x7f3d30152b70 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:01.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.658+0000 7f3d35254640 1 --2- 172.21.15.105:0/1360243310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d300ad680 0x7f3d3014e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:01.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.658+0000 7f3d35254640 1 -- 172.21.15.105:0/1360243310 >> 172.21.15.105:0/1360243310 conn(0x7f3d300ceb90 msgr2=0x7f3d3015d950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:01.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.658+0000 7f3d35254640 1 -- 172.21.15.105:0/1360243310 shutdown_connections 2023-11-28T18:28:01.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.658+0000 7f3d35254640 1 -- 172.21.15.105:0/1360243310 wait complete. 2023-11-28T18:28:01.686 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T18:28:01.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.826+0000 7f6cc3380640 1 Processor -- start 2023-11-28T18:28:01.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.826+0000 7f6cc3380640 1 -- start start 2023-11-28T18:28:01.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.826+0000 7f6cc3380640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6cbc06d610 0x7f6cbc0cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:01.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.826+0000 7f6cc3380640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6cbc0ce530 0x7f6cbc0ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:01.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.826+0000 7f6cc3380640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cbc162820 con 0x7f6cbc0adf30 2023-11-28T18:28:01.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.826+0000 7f6cc3380640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cbc0ae340 con 0x7f6cbc06d610 2023-11-28T18:28:01.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.826+0000 7f6cc3380640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cbc161c10 con 0x7f6cbc0ce530 2023-11-28T18:28:01.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.826+0000 7f6cc111c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6cbc06d610 0x7f6cbc0cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:01.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.826+0000 7f6cc111c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6cbc06d610 0x7f6cbc0cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48984/0 (socket says 172.21.15.105:48984) 2023-11-28T18:28:01.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.826+0000 7f6cc111c640 1 -- 172.21.15.105:0/3918242387 learned_addr learned my addr 172.21.15.105:0/3918242387 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:01.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.826+0000 7f6cc111c640 1 -- 172.21.15.105:0/3918242387 >> v2:172.21.15.148:3300/0 conn(0x7f6cbc0ce530 msgr2=0x7f6cbc0ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:01.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.826+0000 7f6cc091b640 1 --2- 172.21.15.105:0/3918242387 >> v2:172.21.15.148:3300/0 conn(0x7f6cbc0ce530 0x7f6cbc0ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:01.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.826+0000 7f6cc111c640 1 --2- 172.21.15.105:0/3918242387 >> v2:172.21.15.148:3300/0 conn(0x7f6cbc0ce530 0x7f6cbc0ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:01.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.826+0000 7f6cc111c640 1 -- 172.21.15.105:0/3918242387 >> v1:172.21.15.148:6789/0 conn(0x7f6cbc0adf30 legacy=0x7f6cbc1650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:01.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.826+0000 7f6cc111c640 1 -- 172.21.15.105:0/3918242387 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cbc161ef0 con 0x7f6cbc06d610 2023-11-28T18:28:01.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.826+0000 7f6cc091b640 1 --2- 172.21.15.105:0/3918242387 >> v2:172.21.15.148:3300/0 conn(0x7f6cbc0ce530 0x7f6cbc0ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:01.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.830+0000 7f6cc111c640 1 --2- 172.21.15.105:0/3918242387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cbc06d610 0x7f6cbc0cdff0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc073a50 tx=0x7f6cb801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b12e026fc2ec20 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:01.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.830+0000 7f6cb3fff640 1 -- 172.21.15.105:0/3918242387 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb80ac410 con 0x7f6cbc06d610 2023-11-28T18:28:01.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.830+0000 7f6cb3fff640 1 -- 172.21.15.105:0/3918242387 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb80ac570 con 0x7f6cbc06d610 2023-11-28T18:28:01.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.830+0000 7f6cb3fff640 1 -- 172.21.15.105:0/3918242387 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb80ac900 con 0x7f6cbc06d610 2023-11-28T18:28:01.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.830+0000 7f6cc3380640 1 -- 172.21.15.105:0/3918242387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cbc06d610 msgr2=0x7f6cbc0cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:01.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.830+0000 7f6cc3380640 1 --2- 172.21.15.105:0/3918242387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cbc06d610 0x7f6cbc0cdff0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc073a50 tx=0x7f6cb801c490 comp rx=0 tx=0).stop 2023-11-28T18:28:01.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.830+0000 7f6cc3380640 1 -- 172.21.15.105:0/3918242387 shutdown_connections 2023-11-28T18:28:01.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.830+0000 7f6cc3380640 1 --2- 172.21.15.105:0/3918242387 >> v2:172.21.15.148:3300/0 conn(0x7f6cbc0ce530 0x7f6cbc0ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:01.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.830+0000 7f6cc3380640 1 --2- 172.21.15.105:0/3918242387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cbc06d610 0x7f6cbc0cdff0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:01.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.830+0000 7f6cc3380640 1 -- 172.21.15.105:0/3918242387 >> 172.21.15.105:0/3918242387 conn(0x7f6cbc0ceb90 msgr2=0x7f6cbc0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:01.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.830+0000 7f6cc3380640 1 -- 172.21.15.105:0/3918242387 shutdown_connections 2023-11-28T18:28:01.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.830+0000 7f6cc3380640 1 -- 172.21.15.105:0/3918242387 wait complete. 2023-11-28T18:28:01.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cc3380640 1 Processor -- start 2023-11-28T18:28:01.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cc3380640 1 -- start start 2023-11-28T18:28:01.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cc3380640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cbc06d610 0x7f6cbc15bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:01.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cc3380640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cbc0adf30 0x7f6cbc15c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:01.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cc3380640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6cbc0ce530 0x7f6cbc15fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:01.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cc3380640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc15d630 con 0x7f6cbc06d610 2023-11-28T18:28:01.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cc3380640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc15cd90 con 0x7f6cbc0adf30 2023-11-28T18:28:01.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cc3380640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc15d070 con 0x7f6cbc0ce530 2023-11-28T18:28:01.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cc111c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cbc06d610 0x7f6cbc15bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:01.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cc111c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cbc06d610 0x7f6cbc15bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48986/0 (socket says 172.21.15.105:48986) 2023-11-28T18:28:01.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cc111c640 1 -- 172.21.15.105:0/2052377034 learned_addr learned my addr 172.21.15.105:0/2052377034 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:01.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cc091b640 1 --2- 172.21.15.105:0/2052377034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cbc0adf30 0x7f6cbc15c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:01.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cc191d640 1 --2- 172.21.15.105:0/2052377034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6cbc0ce530 0x7f6cbc15fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:01.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cc111c640 1 -- 172.21.15.105:0/2052377034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6cbc0ce530 msgr2=0x7f6cbc15fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:01.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cc111c640 1 --2- 172.21.15.105:0/2052377034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6cbc0ce530 0x7f6cbc15fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:01.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cc111c640 1 -- 172.21.15.105:0/2052377034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cbc0adf30 msgr2=0x7f6cbc15c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:01.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cc111c640 1 --2- 172.21.15.105:0/2052377034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cbc0adf30 0x7f6cbc15c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:01.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cc111c640 1 -- 172.21.15.105:0/2052377034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb80094d0 con 0x7f6cbc06d610 2023-11-28T18:28:01.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cc091b640 1 --2- 172.21.15.105:0/2052377034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cbc0adf30 0x7f6cbc15c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:01.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cc111c640 1 --2- 172.21.15.105:0/2052377034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cbc06d610 0x7f6cbc15bc50 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80af040 tx=0x7f6cb8019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:01.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cb1ffb640 1 -- 172.21.15.105:0/2052377034 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb80ac410 con 0x7f6cbc06d610 2023-11-28T18:28:01.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cb1ffb640 1 -- 172.21.15.105:0/2052377034 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb80b65f0 con 0x7f6cbc06d610 2023-11-28T18:28:01.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cc3380640 1 -- 172.21.15.105:0/2052377034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cbc15d350 con 0x7f6cbc06d610 2023-11-28T18:28:01.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cb1ffb640 1 -- 172.21.15.105:0/2052377034 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb80b69a0 con 0x7f6cbc06d610 2023-11-28T18:28:01.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.834+0000 7f6cc3380640 1 -- 172.21.15.105:0/2052377034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cbc09bb30 con 0x7f6cbc06d610 2023-11-28T18:28:01.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.838+0000 7f6cc3380640 1 -- 172.21.15.105:0/2052377034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cbc067570 con 0x7f6cbc06d610 2023-11-28T18:28:01.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.838+0000 7f6cb1ffb640 1 -- 172.21.15.105:0/2052377034 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f6cb80b6b00 con 0x7f6cbc06d610 2023-11-28T18:28:01.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.838+0000 7f6cb1ffb640 1 --2- 172.21.15.105:0/2052377034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6ca00668d0 0x7f6ca0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:01.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.838+0000 7f6cb1ffb640 1 -- 172.21.15.105:0/2052377034 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6432+0+0 (secure 0 0 0) 0x7f6cb811f140 con 0x7f6cbc06d610 2023-11-28T18:28:01.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.838+0000 7f6cc091b640 1 --2- 172.21.15.105:0/2052377034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6ca00668d0 0x7f6ca0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:01.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.838+0000 7f6cc091b640 1 --2- 172.21.15.105:0/2052377034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6ca00668d0 0x7f6ca0068d90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc15e370 tx=0x7f6cac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:01.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.842+0000 7f6cb1ffb640 1 -- 172.21.15.105:0/2052377034 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6cb80ec1e0 con 0x7f6cbc06d610 2023-11-28T18:28:01.911 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:28:01.917 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:28:02.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:01.994+0000 7f6cc3380640 1 -- 172.21.15.105:0/2052377034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6cbc070bb0 con 0x7f6cbc06d610 2023-11-28T18:28:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.658+0000 7f6cb1ffb640 1 -- 172.21.15.105:0/2052377034 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v69) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6cb80ef800 con 0x7f6cbc06d610 2023-11-28T18:28:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' removed 2023-11-28T18:28:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.662+0000 7f6cc3380640 1 -- 172.21.15.105:0/2052377034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6ca00668d0 msgr2=0x7f6ca0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.662+0000 7f6cc3380640 1 --2- 172.21.15.105:0/2052377034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6ca00668d0 0x7f6ca0068d90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc15e370 tx=0x7f6cac008040 comp rx=0 tx=0).stop 2023-11-28T18:28:02.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.662+0000 7f6cc3380640 1 -- 172.21.15.105:0/2052377034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cbc06d610 msgr2=0x7f6cbc15bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:02.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.662+0000 7f6cc3380640 1 --2- 172.21.15.105:0/2052377034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cbc06d610 0x7f6cbc15bc50 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80af040 tx=0x7f6cb8019230 comp rx=0 tx=0).stop 2023-11-28T18:28:02.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.662+0000 7f6cc3380640 1 -- 172.21.15.105:0/2052377034 shutdown_connections 2023-11-28T18:28:02.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.662+0000 7f6cc3380640 1 --2- 172.21.15.105:0/2052377034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6cbc0ce530 0x7f6cbc15fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:02.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.662+0000 7f6cc3380640 1 --2- 172.21.15.105:0/2052377034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cbc0adf30 0x7f6cbc15c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:02.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.662+0000 7f6cc3380640 1 --2- 172.21.15.105:0/2052377034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6ca00668d0 0x7f6ca0068d90 secure :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc15e370 tx=0x7f6cac008040 comp rx=0 tx=0).stop 2023-11-28T18:28:02.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.662+0000 7f6cc3380640 1 --2- 172.21.15.105:0/2052377034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cbc06d610 0x7f6cbc15bc50 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:02.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.662+0000 7f6cc3380640 1 -- 172.21.15.105:0/2052377034 >> 172.21.15.105:0/2052377034 conn(0x7f6cbc0ceb90 msgr2=0x7f6cbc0cdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:02.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.662+0000 7f6cc3380640 1 -- 172.21.15.105:0/2052377034 shutdown_connections 2023-11-28T18:28:02.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.662+0000 7f6cc3380640 1 -- 172.21.15.105:0/2052377034 wait complete. 2023-11-28T18:28:02.692 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T18:28:02.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.830+0000 7fa4067f8640 1 Processor -- start 2023-11-28T18:28:02.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.830+0000 7fa4067f8640 1 -- start start 2023-11-28T18:28:02.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.830+0000 7fa4067f8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa40015b030 0x7fa4000709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:02.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.830+0000 7fa4067f8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4000701d0 con 0x7fa400071960 2023-11-28T18:28:02.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.830+0000 7fa4067f8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa40006df80 con 0x7fa4000b4290 2023-11-28T18:28:02.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.830+0000 7fa4067f8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa400071df0 con 0x7fa40015b030 2023-11-28T18:28:02.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.830+0000 7fa3ff7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa400071960 0x7fa40015f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41974/0 (socket says 172.21.15.105:41974) 2023-11-28T18:28:02.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.830+0000 7fa3ff7fe640 1 -- 172.21.15.105:0/195519631 learned_addr learned my addr 172.21.15.105:0/195519631 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:02.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.830+0000 7fa404d95640 1 --2- 172.21.15.105:0/195519631 >> v2:172.21.15.105:3300/0 conn(0x7fa40015b030 0x7fa4000709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:02.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.830+0000 7fa3feffd640 1 -- 172.21.15.105:0/195519631 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740907968 0 0) 0x7fa4000701d0 con 0x7fa400071960 2023-11-28T18:28:02.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.830+0000 7fa404d95640 1 -- 172.21.15.105:0/195519631 >> v1:172.21.15.148:6789/0 conn(0x7fa4000b4290 legacy=0x7fa4000b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:02.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.830+0000 7fa404d95640 1 -- 172.21.15.105:0/195519631 >> v1:172.21.15.105:6789/0 conn(0x7fa400071960 legacy=0x7fa40015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:02.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.830+0000 7fa404d95640 1 -- 172.21.15.105:0/195519631 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4000720d0 con 0x7fa40015b030 2023-11-28T18:28:02.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.834+0000 7fa404d95640 1 --2- 172.21.15.105:0/195519631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40015b030 0x7fa4000709b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009d90 tx=0x7fa3f801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163f688860d2696e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:02.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.834+0000 7fa3feffd640 1 -- 172.21.15.105:0/195519631 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f80a58b0 con 0x7fa40015b030 2023-11-28T18:28:02.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.834+0000 7fa3feffd640 1 -- 172.21.15.105:0/195519631 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f80a5a10 con 0x7fa40015b030 2023-11-28T18:28:02.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.834+0000 7fa3feffd640 1 -- 172.21.15.105:0/195519631 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f80a5da0 con 0x7fa40015b030 2023-11-28T18:28:02.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.834+0000 7fa4067f8640 1 -- 172.21.15.105:0/195519631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40015b030 msgr2=0x7fa4000709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:02.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.834+0000 7fa4067f8640 1 --2- 172.21.15.105:0/195519631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40015b030 0x7fa4000709b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009d90 tx=0x7fa3f801c8e0 comp rx=0 tx=0).stop 2023-11-28T18:28:02.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.834+0000 7fa4067f8640 1 -- 172.21.15.105:0/195519631 shutdown_connections 2023-11-28T18:28:02.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.834+0000 7fa4067f8640 1 --2- 172.21.15.105:0/195519631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40015b030 0x7fa4000709b0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:02.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.834+0000 7fa4067f8640 1 -- 172.21.15.105:0/195519631 >> 172.21.15.105:0/195519631 conn(0x7fa4000ceb90 msgr2=0x7fa4000cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:02.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.834+0000 7fa4067f8640 1 -- 172.21.15.105:0/195519631 shutdown_connections 2023-11-28T18:28:02.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.834+0000 7fa4067f8640 1 -- 172.21.15.105:0/195519631 wait complete. 2023-11-28T18:28:02.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa4067f8640 1 Processor -- start 2023-11-28T18:28:02.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa4067f8640 1 -- start start 2023-11-28T18:28:02.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa4067f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4000b4290 0x7fa400165950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:02.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa4067f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40015b030 0x7fa400153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:02.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa4067f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4001558e0 0x7fa400159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:02.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa4067f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa400165e90 con 0x7fa40015b030 2023-11-28T18:28:02.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa4067f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa400099dc0 con 0x7fa4001558e0 2023-11-28T18:28:02.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa4067f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40009a0a0 con 0x7fa4000b4290 2023-11-28T18:28:02.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa3ff7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40015b030 0x7fa400153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:02.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa3ff7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40015b030 0x7fa400153a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49008/0 (socket says 172.21.15.105:49008) 2023-11-28T18:28:02.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa3ff7fe640 1 -- 172.21.15.105:0/2145727862 learned_addr learned my addr 172.21.15.105:0/2145727862 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:02.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa404d95640 1 --2- 172.21.15.105:0/2145727862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4001558e0 0x7fa400159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:02.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa3fffff640 1 --2- 172.21.15.105:0/2145727862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4000b4290 0x7fa400165950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:02.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa3ff7fe640 1 -- 172.21.15.105:0/2145727862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4000b4290 msgr2=0x7fa400165950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:02.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa3ff7fe640 1 --2- 172.21.15.105:0/2145727862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4000b4290 0x7fa400165950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:02.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa3ff7fe640 1 -- 172.21.15.105:0/2145727862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4001558e0 msgr2=0x7fa400159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:02.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa3ff7fe640 1 --2- 172.21.15.105:0/2145727862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4001558e0 0x7fa400159680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:02.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa3ff7fe640 1 -- 172.21.15.105:0/2145727862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f8009a70 con 0x7fa40015b030 2023-11-28T18:28:02.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa404d95640 1 --2- 172.21.15.105:0/2145727862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4001558e0 0x7fa400159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:02.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa3ff7fe640 1 --2- 172.21.15.105:0/2145727862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40015b030 0x7fa400153a70 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fa3f40011f0 tx=0x7fa3f400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:02.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa3fcff9640 1 -- 172.21.15.105:0/2145727862 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f40074e0 con 0x7fa40015b030 2023-11-28T18:28:02.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa3fcff9640 1 -- 172.21.15.105:0/2145727862 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f4007640 con 0x7fa40015b030 2023-11-28T18:28:02.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa3fcff9640 1 -- 172.21.15.105:0/2145727862 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4004a10 con 0x7fa40015b030 2023-11-28T18:28:02.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa4067f8640 1 -- 172.21.15.105:0/2145727862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa40009a380 con 0x7fa40015b030 2023-11-28T18:28:02.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa4067f8640 1 -- 172.21.15.105:0/2145727862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa40009a940 con 0x7fa40015b030 2023-11-28T18:28:02.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.838+0000 7fa4067f8640 1 -- 172.21.15.105:0/2145727862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa400067570 con 0x7fa40015b030 2023-11-28T18:28:02.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.842+0000 7fa3fcff9640 1 -- 172.21.15.105:0/2145727862 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa3f4004b70 con 0x7fa40015b030 2023-11-28T18:28:02.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.842+0000 7fa3fcff9640 1 --2- 172.21.15.105:0/2145727862 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa3ec0668d0 0x7fa3ec068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:02.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.842+0000 7fa3fcff9640 1 -- 172.21.15.105:0/2145727862 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5965+0+0 (secure 0 0 0) 0x7fa3f40838a0 con 0x7fa40015b030 2023-11-28T18:28:02.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.842+0000 7fa3fffff640 1 --2- 172.21.15.105:0/2145727862 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa3ec0668d0 0x7fa3ec068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:02.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.842+0000 7fa3fffff640 1 --2- 172.21.15.105:0/2145727862 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa3ec0668d0 0x7fa3ec068d90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0004e30 tx=0x7fa3f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:02.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.846+0000 7fa3fcff9640 1 -- 172.21.15.105:0/2145727862 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa3f4050940 con 0x7fa40015b030 2023-11-28T18:28:03.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:02.994+0000 7fa4067f8640 1 -- 172.21.15.105:0/2145727862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa400158310 con 0x7fa40015b030 2023-11-28T18:28:03.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.662+0000 7fa3fcff9640 1 -- 172.21.15.105:0/2145727862 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v70) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa3f4018070 con 0x7fa40015b030 2023-11-28T18:28:03.666 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' removed 2023-11-28T18:28:03.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.662+0000 7fa4067f8640 1 -- 172.21.15.105:0/2145727862 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa3ec0668d0 msgr2=0x7fa3ec068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:03.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.662+0000 7fa4067f8640 1 --2- 172.21.15.105:0/2145727862 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa3ec0668d0 0x7fa3ec068d90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0004e30 tx=0x7fa3f0009040 comp rx=0 tx=0).stop 2023-11-28T18:28:03.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.662+0000 7fa4067f8640 1 -- 172.21.15.105:0/2145727862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40015b030 msgr2=0x7fa400153a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:03.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.662+0000 7fa4067f8640 1 --2- 172.21.15.105:0/2145727862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40015b030 0x7fa400153a70 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fa3f40011f0 tx=0x7fa3f400c9b0 comp rx=0 tx=0).stop 2023-11-28T18:28:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.662+0000 7fa4067f8640 1 -- 172.21.15.105:0/2145727862 shutdown_connections 2023-11-28T18:28:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.662+0000 7fa4067f8640 1 --2- 172.21.15.105:0/2145727862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4001558e0 0x7fa400159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.662+0000 7fa4067f8640 1 --2- 172.21.15.105:0/2145727862 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa3ec0668d0 0x7fa3ec068d90 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.662+0000 7fa4067f8640 1 --2- 172.21.15.105:0/2145727862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa40015b030 0x7fa400153a70 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.662+0000 7fa4067f8640 1 --2- 172.21.15.105:0/2145727862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4000b4290 0x7fa400165950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.662+0000 7fa4067f8640 1 -- 172.21.15.105:0/2145727862 >> 172.21.15.105:0/2145727862 conn(0x7fa4000ceb90 msgr2=0x7fa4000afa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.662+0000 7fa4067f8640 1 -- 172.21.15.105:0/2145727862 shutdown_connections 2023-11-28T18:28:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.662+0000 7fa4067f8640 1 -- 172.21.15.105:0/2145727862 wait complete. 2023-11-28T18:28:03.694 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random 2023-11-28T18:28:03.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.830+0000 7f64d6baa640 1 Processor -- start 2023-11-28T18:28:03.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.830+0000 7f64d6baa640 1 -- start start 2023-11-28T18:28:03.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.830+0000 7f64d6baa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f64d0159f40 0x7f64d00caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:03.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.830+0000 7f64d6baa640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d006bf50 con 0x7f64d00ad850 2023-11-28T18:28:03.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.830+0000 7f64d6baa640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d00ccbc0 con 0x7f64d00cc760 2023-11-28T18:28:03.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.830+0000 7f64d6baa640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d015a320 con 0x7f64d0159f40 2023-11-28T18:28:03.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.830+0000 7f64d4946640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f64d0159f40 0x7f64d00caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:03.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.830+0000 7f64d5147640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f64d00ad850 0x7f64d00701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41986/0 (socket says 172.21.15.105:41986) 2023-11-28T18:28:03.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.830+0000 7f64d5147640 1 -- 172.21.15.105:0/3096624160 learned_addr learned my addr 172.21.15.105:0/3096624160 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:03.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.830+0000 7f64d4946640 1 -- 172.21.15.105:0/3096624160 >> v1:172.21.15.148:6789/0 conn(0x7f64d00cc760 legacy=0x7f64d00ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:03.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.830+0000 7f64d4946640 1 -- 172.21.15.105:0/3096624160 >> v1:172.21.15.105:6789/0 conn(0x7f64d00ad850 legacy=0x7f64d00701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:03.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.830+0000 7f64d4946640 1 -- 172.21.15.105:0/3096624160 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d0071380 con 0x7f64d0159f40 2023-11-28T18:28:03.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.830+0000 7f64cf7fe640 1 -- 172.21.15.105:0/3096624160 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82339920 0 0) 0x7f64d006bf50 con 0x7f64d00ad850 2023-11-28T18:28:03.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.834+0000 7f64d4946640 1 --2- 172.21.15.105:0/3096624160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64d0159f40 0x7f64d00caf60 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f64c40097c0 tx=0x7f64c401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3412b4831fc7ef94 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:03.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.834+0000 7f64cf7fe640 1 -- 172.21.15.105:0/3096624160 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c40ad680 con 0x7f64d0159f40 2023-11-28T18:28:03.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.834+0000 7f64cf7fe640 1 -- 172.21.15.105:0/3096624160 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c40ad7e0 con 0x7f64d0159f40 2023-11-28T18:28:03.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.834+0000 7f64cf7fe640 1 -- 172.21.15.105:0/3096624160 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c40adb70 con 0x7f64d0159f40 2023-11-28T18:28:03.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.834+0000 7f64d6baa640 1 -- 172.21.15.105:0/3096624160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64d0159f40 msgr2=0x7f64d00caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:03.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.834+0000 7f64d6baa640 1 --2- 172.21.15.105:0/3096624160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64d0159f40 0x7f64d00caf60 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f64c40097c0 tx=0x7f64c401c380 comp rx=0 tx=0).stop 2023-11-28T18:28:03.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.834+0000 7f64d6baa640 1 -- 172.21.15.105:0/3096624160 shutdown_connections 2023-11-28T18:28:03.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.834+0000 7f64d6baa640 1 --2- 172.21.15.105:0/3096624160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64d0159f40 0x7f64d00caf60 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:03.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.834+0000 7f64d6baa640 1 -- 172.21.15.105:0/3096624160 >> 172.21.15.105:0/3096624160 conn(0x7f64d00ceb90 msgr2=0x7f64d00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:03.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.834+0000 7f64d6baa640 1 -- 172.21.15.105:0/3096624160 shutdown_connections 2023-11-28T18:28:03.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.834+0000 7f64d6baa640 1 -- 172.21.15.105:0/3096624160 wait complete. 2023-11-28T18:28:03.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64d6baa640 1 Processor -- start 2023-11-28T18:28:03.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64d6baa640 1 -- start start 2023-11-28T18:28:03.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64d6baa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64d00ad850 0x7f64d014f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:03.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64d6baa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64d00cc760 0x7f64d0099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:03.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64d6baa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64d0159f40 0x7f64d00b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:03.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64d6baa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d00b19b0 con 0x7f64d00ad850 2023-11-28T18:28:03.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64d6baa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d00b3c60 con 0x7f64d0159f40 2023-11-28T18:28:03.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64d6baa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d00b1c00 con 0x7f64d00cc760 2023-11-28T18:28:03.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64d4946640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64d00ad850 0x7f64d014f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:03.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64d4946640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64d00ad850 0x7f64d014f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49020/0 (socket says 172.21.15.105:49020) 2023-11-28T18:28:03.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64d4946640 1 -- 172.21.15.105:0/639109227 learned_addr learned my addr 172.21.15.105:0/639109227 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:03.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64cffff640 1 --2- 172.21.15.105:0/639109227 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64d00cc760 0x7f64d0099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:03.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64d5147640 1 --2- 172.21.15.105:0/639109227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64d0159f40 0x7f64d00b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:03.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64d4946640 1 -- 172.21.15.105:0/639109227 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64d00cc760 msgr2=0x7f64d0099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:03.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64d4946640 1 --2- 172.21.15.105:0/639109227 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64d00cc760 0x7f64d0099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:03.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64d4946640 1 -- 172.21.15.105:0/639109227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64d0159f40 msgr2=0x7f64d00b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:03.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64d4946640 1 --2- 172.21.15.105:0/639109227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64d0159f40 0x7f64d00b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:03.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64d4946640 1 -- 172.21.15.105:0/639109227 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c4009420 con 0x7f64d00ad850 2023-11-28T18:28:03.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64cffff640 1 --2- 172.21.15.105:0/639109227 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64d00cc760 0x7f64d0099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:03.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64d4946640 1 --2- 172.21.15.105:0/639109227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64d00ad850 0x7f64d014f770 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f64c4018f90 tx=0x7f64c401abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:03.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64cd7fa640 1 -- 172.21.15.105:0/639109227 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c40ade60 con 0x7f64d00ad850 2023-11-28T18:28:03.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64cd7fa640 1 -- 172.21.15.105:0/639109227 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c40b8450 con 0x7f64d00ad850 2023-11-28T18:28:03.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64cd7fa640 1 -- 172.21.15.105:0/639109227 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c40b87e0 con 0x7f64d00ad850 2023-11-28T18:28:03.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64d6baa640 1 -- 172.21.15.105:0/639109227 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d00b1ee0 con 0x7f64d00ad850 2023-11-28T18:28:03.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64d6baa640 1 -- 172.21.15.105:0/639109227 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d00b3510 con 0x7f64d00ad850 2023-11-28T18:28:03.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.838+0000 7f64d6baa640 1 -- 172.21.15.105:0/639109227 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d0067570 con 0x7f64d00ad850 2023-11-28T18:28:03.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.842+0000 7f64cd7fa640 1 -- 172.21.15.105:0/639109227 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f64c40b4070 con 0x7f64d00ad850 2023-11-28T18:28:03.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.842+0000 7f64cd7fa640 1 --2- 172.21.15.105:0/639109227 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f64b40668d0 0x7f64b4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:03.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.842+0000 7f64cd7fa640 1 -- 172.21.15.105:0/639109227 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f64c411ec90 con 0x7f64d00ad850 2023-11-28T18:28:03.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.842+0000 7f64cffff640 1 --2- 172.21.15.105:0/639109227 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f64b40668d0 0x7f64b4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:03.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.842+0000 7f64cffff640 1 --2- 172.21.15.105:0/639109227 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f64b40668d0 0x7f64b4068d90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f64d014f0b0 tx=0x7f64bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:03.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.846+0000 7f64cd7fa640 1 -- 172.21.15.105:0/639109227 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f64c40ebf40 con 0x7f64d00ad850 2023-11-28T18:28:04.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:03.998+0000 7f64d6baa640 1 -- 172.21.15.105:0/639109227 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"} v 0) v1 -- 0x7f64d00cf950 con 0x7f64d00ad850 2023-11-28T18:28:04.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.002+0000 7f64cd7fa640 1 -- 172.21.15.105:0/639109227 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"}]=0 v31) v1 ==== 110+0+0 (secure 0 0 0) 0x7f64c40b20b0 con 0x7f64d00ad850 2023-11-28T18:28:04.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.002+0000 7f64d6baa640 1 -- 172.21.15.105:0/639109227 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f64b40668d0 msgr2=0x7f64b4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:04.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.002+0000 7f64d6baa640 1 --2- 172.21.15.105:0/639109227 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f64b40668d0 0x7f64b4068d90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f64d014f0b0 tx=0x7f64bc008040 comp rx=0 tx=0).stop 2023-11-28T18:28:04.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.002+0000 7f64d6baa640 1 -- 172.21.15.105:0/639109227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64d00ad850 msgr2=0x7f64d014f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:04.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.002+0000 7f64d6baa640 1 --2- 172.21.15.105:0/639109227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64d00ad850 0x7f64d014f770 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f64c4018f90 tx=0x7f64c401abc0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.002+0000 7f64d6baa640 1 -- 172.21.15.105:0/639109227 shutdown_connections 2023-11-28T18:28:04.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.002+0000 7f64d6baa640 1 --2- 172.21.15.105:0/639109227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64d0159f40 0x7f64d00b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.002+0000 7f64d6baa640 1 --2- 172.21.15.105:0/639109227 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64d00cc760 0x7f64d0099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.002+0000 7f64d6baa640 1 --2- 172.21.15.105:0/639109227 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f64b40668d0 0x7f64b4068d90 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.002+0000 7f64d6baa640 1 --2- 172.21.15.105:0/639109227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64d00ad850 0x7f64d014f770 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.002+0000 7f64d6baa640 1 -- 172.21.15.105:0/639109227 >> 172.21.15.105:0/639109227 conn(0x7f64d00ceb90 msgr2=0x7f64d0153450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:04.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.002+0000 7f64d6baa640 1 -- 172.21.15.105:0/639109227 shutdown_connections 2023-11-28T18:28:04.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.002+0000 7f64d6baa640 1 -- 172.21.15.105:0/639109227 wait complete. 2023-11-28T18:28:04.027 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T18:28:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.178+0000 7f46e9910640 1 Processor -- start 2023-11-28T18:28:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.178+0000 7f46e9910640 1 -- start start 2023-11-28T18:28:04.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.178+0000 7f46e9910640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46e4157910 0x7f46e40c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:04.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.178+0000 7f46e9910640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f46e406cc90 0x7f46e406d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:04.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.178+0000 7f46e9910640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e4070920 con 0x7f46e415d8d0 2023-11-28T18:28:04.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.178+0000 7f46e9910640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e4157cf0 con 0x7f46e406cc90 2023-11-28T18:28:04.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.178+0000 7f46e9910640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e406d750 con 0x7f46e4157910 2023-11-28T18:28:04.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.178+0000 7f46dbfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f46e406cc90 0x7f46e406d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:04.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.178+0000 7f46dbfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f46e406cc90 0x7f46e406d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49032/0 (socket says 172.21.15.105:49032) 2023-11-28T18:28:04.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.178+0000 7f46dbfff640 1 -- 172.21.15.105:0/4048444033 learned_addr learned my addr 172.21.15.105:0/4048444033 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:04.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.178+0000 7f46e890e640 1 --2- 172.21.15.105:0/4048444033 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46e4157910 0x7f46e40c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:04.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.182+0000 7f46dbfff640 1 -- 172.21.15.105:0/4048444033 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46e4157910 msgr2=0x7f46e40c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:04.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.182+0000 7f46dbfff640 1 --2- 172.21.15.105:0/4048444033 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46e4157910 0x7f46e40c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.182+0000 7f46dbfff640 1 -- 172.21.15.105:0/4048444033 >> v1:172.21.15.148:6789/0 conn(0x7f46e415d8d0 legacy=0x7f46e40716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:04.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.182+0000 7f46dbfff640 1 -- 172.21.15.105:0/4048444033 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e4070be0 con 0x7f46e406cc90 2023-11-28T18:28:04.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.182+0000 7f46dbfff640 1 --2- 172.21.15.105:0/4048444033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46e406cc90 0x7f46e406d0d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009870 tx=0x7f46d401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd78b700cac647a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:04.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.182+0000 7f46db7fe640 1 -- 172.21.15.105:0/4048444033 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d40aa610 con 0x7f46e406cc90 2023-11-28T18:28:04.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.182+0000 7f46db7fe640 1 -- 172.21.15.105:0/4048444033 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d40a6040 con 0x7f46e406cc90 2023-11-28T18:28:04.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.182+0000 7f46db7fe640 1 -- 172.21.15.105:0/4048444033 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d40aa990 con 0x7f46e406cc90 2023-11-28T18:28:04.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.182+0000 7f46e9910640 1 -- 172.21.15.105:0/4048444033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46e406cc90 msgr2=0x7f46e406d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:04.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.182+0000 7f46e9910640 1 --2- 172.21.15.105:0/4048444033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46e406cc90 0x7f46e406d0d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009870 tx=0x7f46d401c4b0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.182+0000 7f46e9910640 1 -- 172.21.15.105:0/4048444033 shutdown_connections 2023-11-28T18:28:04.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.182+0000 7f46e9910640 1 --2- 172.21.15.105:0/4048444033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46e406cc90 0x7f46e406d0d0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.182+0000 7f46e9910640 1 --2- 172.21.15.105:0/4048444033 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46e4157910 0x7f46e40c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.182+0000 7f46e9910640 1 -- 172.21.15.105:0/4048444033 >> 172.21.15.105:0/4048444033 conn(0x7f46e40ce9e0 msgr2=0x7f46e40cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:04.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.186+0000 7f46e9910640 1 -- 172.21.15.105:0/4048444033 shutdown_connections 2023-11-28T18:28:04.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.186+0000 7f46e9910640 1 -- 172.21.15.105:0/4048444033 wait complete. 2023-11-28T18:28:04.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.186+0000 7f46e9910640 1 Processor -- start 2023-11-28T18:28:04.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.186+0000 7f46e9910640 1 -- start start 2023-11-28T18:28:04.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.186+0000 7f46e9910640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46e406cc90 0x7f46e4166680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:04.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.186+0000 7f46e9910640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46e4157910 0x7f46e4150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:04.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.186+0000 7f46e9910640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46e415d8d0 0x7f46e40c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:04.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.186+0000 7f46e9910640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e4166bc0 con 0x7f46e4157910 2023-11-28T18:28:04.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.186+0000 7f46e9910640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e41fa900 con 0x7f46e415d8d0 2023-11-28T18:28:04.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.186+0000 7f46e9910640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e41f99a0 con 0x7f46e406cc90 2023-11-28T18:28:04.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.186+0000 7f46dbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46e4157910 0x7f46e4150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:04.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.186+0000 7f46dbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46e4157910 0x7f46e4150480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49034/0 (socket says 172.21.15.105:49034) 2023-11-28T18:28:04.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.186+0000 7f46dbfff640 1 -- 172.21.15.105:0/4029239531 learned_addr learned my addr 172.21.15.105:0/4029239531 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:04.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.186+0000 7f46dbfff640 1 -- 172.21.15.105:0/4029239531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46e406cc90 msgr2=0x7f46e4166680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:04.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.186+0000 7f46dbfff640 1 --2- 172.21.15.105:0/4029239531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46e406cc90 0x7f46e4166680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.186+0000 7f46dbfff640 1 -- 172.21.15.105:0/4029239531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46e415d8d0 msgr2=0x7f46e40c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:04.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.186+0000 7f46dbfff640 1 --2- 172.21.15.105:0/4029239531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46e415d8d0 0x7f46e40c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.186+0000 7f46dbfff640 1 -- 172.21.15.105:0/4029239531 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d40094d0 con 0x7f46e4157910 2023-11-28T18:28:04.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.186+0000 7f46dbfff640 1 --2- 172.21.15.105:0/4029239531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46e4157910 0x7f46e4150480 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f46d401a0e0 tx=0x7f46d401acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:04.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.190+0000 7f46d97fa640 1 -- 172.21.15.105:0/4029239531 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d40bd5f0 con 0x7f46e4157910 2023-11-28T18:28:04.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.190+0000 7f46d97fa640 1 -- 172.21.15.105:0/4029239531 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d40bd750 con 0x7f46e4157910 2023-11-28T18:28:04.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.190+0000 7f46d97fa640 1 -- 172.21.15.105:0/4029239531 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d40bdb00 con 0x7f46e4157910 2023-11-28T18:28:04.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.190+0000 7f46e9910640 1 -- 172.21.15.105:0/4029239531 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e415e270 con 0x7f46e4157910 2023-11-28T18:28:04.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.190+0000 7f46e9910640 1 -- 172.21.15.105:0/4029239531 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e41f9f60 con 0x7f46e4157910 2023-11-28T18:28:04.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.190+0000 7f46e9910640 1 -- 172.21.15.105:0/4029239531 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e40679d0 con 0x7f46e4157910 2023-11-28T18:28:04.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.190+0000 7f46d97fa640 1 -- 172.21.15.105:0/4029239531 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f46d40b33f0 con 0x7f46e4157910 2023-11-28T18:28:04.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.190+0000 7f46d97fa640 1 --2- 172.21.15.105:0/4029239531 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46d00668d0 0x7f46d0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:04.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.190+0000 7f46d97fa640 1 -- 172.21.15.105:0/4029239531 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f46d411f070 con 0x7f46e4157910 2023-11-28T18:28:04.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.190+0000 7f46e890e640 1 --2- 172.21.15.105:0/4029239531 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46d00668d0 0x7f46d0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:04.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.190+0000 7f46e890e640 1 --2- 172.21.15.105:0/4029239531 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46d00668d0 0x7f46d0068d90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f46e0006aa0 tx=0x7f46e0006a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.194+0000 7f46d97fa640 1 -- 172.21.15.105:0/4029239531 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f46d40ec4d0 con 0x7f46e4157910 2023-11-28T18:28:04.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.346+0000 7f46e9910640 1 -- 172.21.15.105:0/4029239531 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f46e4067e30 con 0x7f46e4157910 2023-11-28T18:28:04.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.354+0000 7f46d97fa640 1 -- 172.21.15.105:0/4029239531 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7f46e4067e30 con 0x7f46e4157910 2023-11-28T18:28:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.358+0000 7f46e9910640 1 -- 172.21.15.105:0/4029239531 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46d00668d0 msgr2=0x7f46d0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.358+0000 7f46e9910640 1 --2- 172.21.15.105:0/4029239531 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46d00668d0 0x7f46d0068d90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f46e0006aa0 tx=0x7f46e0006a30 comp rx=0 tx=0).stop 2023-11-28T18:28:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.358+0000 7f46e9910640 1 -- 172.21.15.105:0/4029239531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46e4157910 msgr2=0x7f46e4150480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.358+0000 7f46e9910640 1 --2- 172.21.15.105:0/4029239531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46e4157910 0x7f46e4150480 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f46d401a0e0 tx=0x7f46d401acf0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.358+0000 7f46e9910640 1 -- 172.21.15.105:0/4029239531 shutdown_connections 2023-11-28T18:28:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.358+0000 7f46e9910640 1 --2- 172.21.15.105:0/4029239531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46e415d8d0 0x7f46e40c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.358+0000 7f46e9910640 1 --2- 172.21.15.105:0/4029239531 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46d00668d0 0x7f46d0068d90 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.358+0000 7f46e9910640 1 --2- 172.21.15.105:0/4029239531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46e4157910 0x7f46e4150480 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.358+0000 7f46e9910640 1 --2- 172.21.15.105:0/4029239531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46e406cc90 0x7f46e4166680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.358+0000 7f46e9910640 1 -- 172.21.15.105:0/4029239531 >> 172.21.15.105:0/4029239531 conn(0x7f46e40ce9e0 msgr2=0x7f46e415c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.358+0000 7f46e9910640 1 -- 172.21.15.105:0/4029239531 shutdown_connections 2023-11-28T18:28:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.358+0000 7f46e9910640 1 -- 172.21.15.105:0/4029239531 wait complete. 2023-11-28T18:28:04.381 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_distributed 2023-11-28T18:28:04.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.514+0000 7f3fd84e3640 1 Processor -- start 2023-11-28T18:28:04.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.514+0000 7f3fd84e3640 1 -- start start 2023-11-28T18:28:04.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.518+0000 7f3fd84e3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3fd00b4290 0x7f3fd00b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:04.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.518+0000 7f3fd84e3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3fd015b030 0x7f3fd00709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:04.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.518+0000 7f3fd84e3640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd00b4bb0 con 0x7f3fd0071960 2023-11-28T18:28:04.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.518+0000 7f3fd84e3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd00701d0 con 0x7f3fd015b030 2023-11-28T18:28:04.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.518+0000 7f3fd84e3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd006df80 con 0x7f3fd00b4290 2023-11-28T18:28:04.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.518+0000 7f3fd6a80640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3fd015b030 0x7f3fd00709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:04.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.518+0000 7f3fd5a7e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3fd0071960 0x7f3fd015f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42002/0 (socket says 172.21.15.105:42002) 2023-11-28T18:28:04.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.518+0000 7f3fd5a7e640 1 -- 172.21.15.105:0/385020626 learned_addr learned my addr 172.21.15.105:0/385020626 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:04.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.518+0000 7f3fd627f640 1 --2- 172.21.15.105:0/385020626 >> v2:172.21.15.148:3300/0 conn(0x7f3fd00b4290 0x7f3fd00b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:04.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.518+0000 7f3fd6a80640 1 -- 172.21.15.105:0/385020626 >> v2:172.21.15.148:3300/0 conn(0x7f3fd00b4290 msgr2=0x7f3fd00b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:04.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.518+0000 7f3fd6a80640 1 --2- 172.21.15.105:0/385020626 >> v2:172.21.15.148:3300/0 conn(0x7f3fd00b4290 0x7f3fd00b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.518+0000 7f3fd6a80640 1 -- 172.21.15.105:0/385020626 >> v1:172.21.15.105:6789/0 conn(0x7f3fd0071960 legacy=0x7f3fd015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:04.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.518+0000 7f3fd6a80640 1 -- 172.21.15.105:0/385020626 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd0071df0 con 0x7f3fd015b030 2023-11-28T18:28:04.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.518+0000 7f3fd627f640 1 --2- 172.21.15.105:0/385020626 >> v2:172.21.15.148:3300/0 conn(0x7f3fd00b4290 0x7f3fd00b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:04.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.522+0000 7f3fd6a80640 1 --2- 172.21.15.105:0/385020626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd015b030 0x7f3fd00709b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0073a70 tx=0x7f3fc801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa50cb0db3899e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:04.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.522+0000 7f3fd527d640 1 -- 172.21.15.105:0/385020626 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc80a58b0 con 0x7f3fd015b030 2023-11-28T18:28:04.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.522+0000 7f3fd527d640 1 -- 172.21.15.105:0/385020626 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc80a5a10 con 0x7f3fd015b030 2023-11-28T18:28:04.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.522+0000 7f3fd527d640 1 -- 172.21.15.105:0/385020626 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc80a5da0 con 0x7f3fd015b030 2023-11-28T18:28:04.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.522+0000 7f3fd84e3640 1 -- 172.21.15.105:0/385020626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd015b030 msgr2=0x7f3fd00709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:04.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.522+0000 7f3fd84e3640 1 --2- 172.21.15.105:0/385020626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd015b030 0x7f3fd00709b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0073a70 tx=0x7f3fc801c8e0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.522+0000 7f3fd84e3640 1 -- 172.21.15.105:0/385020626 shutdown_connections 2023-11-28T18:28:04.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.522+0000 7f3fd84e3640 1 --2- 172.21.15.105:0/385020626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd015b030 0x7f3fd00709b0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.522+0000 7f3fd84e3640 1 --2- 172.21.15.105:0/385020626 >> v2:172.21.15.148:3300/0 conn(0x7f3fd00b4290 0x7f3fd00b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.522+0000 7f3fd84e3640 1 -- 172.21.15.105:0/385020626 >> 172.21.15.105:0/385020626 conn(0x7f3fd00ceb90 msgr2=0x7f3fd00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:04.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.522+0000 7f3fd84e3640 1 -- 172.21.15.105:0/385020626 shutdown_connections 2023-11-28T18:28:04.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.522+0000 7f3fd84e3640 1 -- 172.21.15.105:0/385020626 wait complete. 2023-11-28T18:28:04.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.522+0000 7f3fd84e3640 1 Processor -- start 2023-11-28T18:28:04.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.522+0000 7f3fd84e3640 1 -- start start 2023-11-28T18:28:04.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fd84e3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd00b4290 0x7f3fd014d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:04.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fd84e3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fd015b030 0x7f3fd00b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:04.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fd84e3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fd014fb90 0x7f3fd0159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:04.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fd84e3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd00b2d30 con 0x7f3fd00b4290 2023-11-28T18:28:04.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fd84e3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd00b3010 con 0x7f3fd015b030 2023-11-28T18:28:04.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fd84e3640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd00b32f0 con 0x7f3fd014fb90 2023-11-28T18:28:04.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fd627f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd00b4290 0x7f3fd014d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fd627f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd00b4290 0x7f3fd014d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49048/0 (socket says 172.21.15.105:49048) 2023-11-28T18:28:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fd627f640 1 -- 172.21.15.105:0/3541743298 learned_addr learned my addr 172.21.15.105:0/3541743298 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fd5a7e640 1 --2- 172.21.15.105:0/3541743298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fd015b030 0x7f3fd00b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fd6a80640 1 --2- 172.21.15.105:0/3541743298 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fd014fb90 0x7f3fd0159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fd627f640 1 -- 172.21.15.105:0/3541743298 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fd014fb90 msgr2=0x7f3fd0159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fd627f640 1 --2- 172.21.15.105:0/3541743298 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fd014fb90 0x7f3fd0159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fd627f640 1 -- 172.21.15.105:0/3541743298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fd015b030 msgr2=0x7f3fd00b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fd627f640 1 --2- 172.21.15.105:0/3541743298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fd015b030 0x7f3fd00b23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fd627f640 1 -- 172.21.15.105:0/3541743298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc8009a70 con 0x7f3fd00b4290 2023-11-28T18:28:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fd6a80640 1 --2- 172.21.15.105:0/3541743298 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fd014fb90 0x7f3fd0159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fd5a7e640 1 --2- 172.21.15.105:0/3541743298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fd015b030 0x7f3fd00b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fd627f640 1 --2- 172.21.15.105:0/3541743298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd00b4290 0x7f3fd014d610 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00c7c0 tx=0x7f3fcc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fc6ffd640 1 -- 172.21.15.105:0/3541743298 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc007a30 con 0x7f3fd00b4290 2023-11-28T18:28:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fc6ffd640 1 -- 172.21.15.105:0/3541743298 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fcc007b90 con 0x7f3fd00b4290 2023-11-28T18:28:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fd84e3640 1 -- 172.21.15.105:0/3541743298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd009a490 con 0x7f3fd00b4290 2023-11-28T18:28:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fc6ffd640 1 -- 172.21.15.105:0/3541743298 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc00ce70 con 0x7f3fd00b4290 2023-11-28T18:28:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fd84e3640 1 -- 172.21.15.105:0/3541743298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd014db50 con 0x7f3fd00b4290 2023-11-28T18:28:04.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fd84e3640 1 -- 172.21.15.105:0/3541743298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd0067570 con 0x7f3fd00b4290 2023-11-28T18:28:04.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.526+0000 7f3fc6ffd640 1 -- 172.21.15.105:0/3541743298 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3fcc0113f0 con 0x7f3fd00b4290 2023-11-28T18:28:04.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.530+0000 7f3fc6ffd640 1 --2- 172.21.15.105:0/3541743298 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3fb40668d0 0x7f3fb4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:04.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.530+0000 7f3fc6ffd640 1 -- 172.21.15.105:0/3541743298 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f3fcc084840 con 0x7f3fd00b4290 2023-11-28T18:28:04.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.530+0000 7f3fd5a7e640 1 --2- 172.21.15.105:0/3541743298 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3fb40668d0 0x7f3fb4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:04.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.530+0000 7f3fd5a7e640 1 --2- 172.21.15.105:0/3541743298 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3fb40668d0 0x7f3fb4068d90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00b4200 tx=0x7f3fc0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:04.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.534+0000 7f3fc6ffd640 1 -- 172.21.15.105:0/3541743298 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3fcc051b80 con 0x7f3fd00b4290 2023-11-28T18:28:04.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.686+0000 7f3fd84e3640 1 -- 172.21.15.105:0/3541743298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"} v 0) v1 -- 0x7f3fd00cf7f0 con 0x7f3fd00b4290 2023-11-28T18:28:04.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.690+0000 7f3fc6ffd640 1 -- 172.21.15.105:0/3541743298 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"}]=0 v33) v1 ==== 115+0+0 (secure 0 0 0) 0x7f3fcc0551a0 con 0x7f3fd00b4290 2023-11-28T18:28:04.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.694+0000 7f3fd84e3640 1 -- 172.21.15.105:0/3541743298 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3fb40668d0 msgr2=0x7f3fb4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:04.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.694+0000 7f3fd84e3640 1 --2- 172.21.15.105:0/3541743298 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3fb40668d0 0x7f3fb4068d90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00b4200 tx=0x7f3fc0009040 comp rx=0 tx=0).stop 2023-11-28T18:28:04.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.694+0000 7f3fd84e3640 1 -- 172.21.15.105:0/3541743298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd00b4290 msgr2=0x7f3fd014d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:04.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.694+0000 7f3fd84e3640 1 --2- 172.21.15.105:0/3541743298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd00b4290 0x7f3fd014d610 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00c7c0 tx=0x7f3fcc00cc80 comp rx=0 tx=0).stop 2023-11-28T18:28:04.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.694+0000 7f3fd84e3640 1 -- 172.21.15.105:0/3541743298 shutdown_connections 2023-11-28T18:28:04.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.694+0000 7f3fd84e3640 1 --2- 172.21.15.105:0/3541743298 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fd014fb90 0x7f3fd0159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.694+0000 7f3fd84e3640 1 --2- 172.21.15.105:0/3541743298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fd015b030 0x7f3fd00b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.694+0000 7f3fd84e3640 1 --2- 172.21.15.105:0/3541743298 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3fb40668d0 0x7f3fb4068d90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.694+0000 7f3fd84e3640 1 --2- 172.21.15.105:0/3541743298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd00b4290 0x7f3fd014d610 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.694+0000 7f3fd84e3640 1 -- 172.21.15.105:0/3541743298 >> 172.21.15.105:0/3541743298 conn(0x7f3fd00ceb90 msgr2=0x7f3fd0153d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:04.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.694+0000 7f3fd84e3640 1 -- 172.21.15.105:0/3541743298 shutdown_connections 2023-11-28T18:28:04.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.694+0000 7f3fd84e3640 1 -- 172.21.15.105:0/3541743298 wait complete. 2023-11-28T18:28:04.718 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random_max 2023-11-28T18:28:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.854+0000 7fa538efe640 1 Processor -- start 2023-11-28T18:28:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.854+0000 7fa538efe640 1 -- start start 2023-11-28T18:28:04.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.854+0000 7fa538efe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa534159f40 0x7fa5340caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:04.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.854+0000 7fa538efe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa5340cc760 0x7fa5340ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:04.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.854+0000 7fa538efe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa53406bf50 con 0x7fa5340ad680 2023-11-28T18:28:04.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.854+0000 7fa538efe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5340ccbc0 con 0x7fa5340cc760 2023-11-28T18:28:04.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.854+0000 7fa538efe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa53415a320 con 0x7fa534159f40 2023-11-28T18:28:04.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.854+0000 7fa529d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa5340cc760 0x7fa5340ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:04.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.854+0000 7fa529d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa5340cc760 0x7fa5340ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49062/0 (socket says 172.21.15.105:49062) 2023-11-28T18:28:04.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.854+0000 7fa529d9b640 1 -- 172.21.15.105:0/2709841171 learned_addr learned my addr 172.21.15.105:0/2709841171 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:04.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.854+0000 7fa53259c640 1 --2- 172.21.15.105:0/2709841171 >> v2:172.21.15.148:3300/0 conn(0x7fa534159f40 0x7fa5340caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:04.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.854+0000 7fa529d9b640 1 -- 172.21.15.105:0/2709841171 >> v2:172.21.15.148:3300/0 conn(0x7fa534159f40 msgr2=0x7fa5340caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:04.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.854+0000 7fa529d9b640 1 --2- 172.21.15.105:0/2709841171 >> v2:172.21.15.148:3300/0 conn(0x7fa534159f40 0x7fa5340caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.854+0000 7fa529d9b640 1 -- 172.21.15.105:0/2709841171 >> v1:172.21.15.148:6789/0 conn(0x7fa5340ad680 legacy=0x7fa53415da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:04.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.854+0000 7fa529d9b640 1 -- 172.21.15.105:0/2709841171 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa534071380 con 0x7fa5340cc760 2023-11-28T18:28:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.858+0000 7fa529d9b640 1 --2- 172.21.15.105:0/2709841171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5340cc760 0x7fa5340ad140 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa53406e500 tx=0x7fa51c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9105ce0de67240 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.858+0000 7fa531d9b640 1 -- 172.21.15.105:0/2709841171 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0a57d0 con 0x7fa5340cc760 2023-11-28T18:28:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.858+0000 7fa531d9b640 1 -- 172.21.15.105:0/2709841171 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c0a5930 con 0x7fa5340cc760 2023-11-28T18:28:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.858+0000 7fa531d9b640 1 -- 172.21.15.105:0/2709841171 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0a5cc0 con 0x7fa5340cc760 2023-11-28T18:28:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.858+0000 7fa538efe640 1 -- 172.21.15.105:0/2709841171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5340cc760 msgr2=0x7fa5340ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.858+0000 7fa538efe640 1 --2- 172.21.15.105:0/2709841171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5340cc760 0x7fa5340ad140 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa53406e500 tx=0x7fa51c01c910 comp rx=0 tx=0).stop 2023-11-28T18:28:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.858+0000 7fa538efe640 1 -- 172.21.15.105:0/2709841171 shutdown_connections 2023-11-28T18:28:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.858+0000 7fa538efe640 1 --2- 172.21.15.105:0/2709841171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5340cc760 0x7fa5340ad140 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.858+0000 7fa538efe640 1 --2- 172.21.15.105:0/2709841171 >> v2:172.21.15.148:3300/0 conn(0x7fa534159f40 0x7fa5340caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.858+0000 7fa538efe640 1 -- 172.21.15.105:0/2709841171 >> 172.21.15.105:0/2709841171 conn(0x7fa5340ceb90 msgr2=0x7fa5340cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.858+0000 7fa538efe640 1 -- 172.21.15.105:0/2709841171 shutdown_connections 2023-11-28T18:28:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.858+0000 7fa538efe640 1 -- 172.21.15.105:0/2709841171 wait complete. 2023-11-28T18:28:04.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa538efe640 1 Processor -- start 2023-11-28T18:28:04.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa538efe640 1 -- start start 2023-11-28T18:28:04.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa538efe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5340ad680 0x7fa53414c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:04.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa538efe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5340cc760 0x7fa534155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:04.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa538efe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa534159f40 0x7fa5341498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:04.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa538efe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53414cba0 con 0x7fa5340cc760 2023-11-28T18:28:04.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa538efe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53414ce80 con 0x7fa5340ad680 2023-11-28T18:28:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa538efe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53414d160 con 0x7fa534159f40 2023-11-28T18:28:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa529d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5340cc760 0x7fa534155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa529d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5340cc760 0x7fa534155b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49074/0 (socket says 172.21.15.105:49074) 2023-11-28T18:28:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa529d9b640 1 -- 172.21.15.105:0/141687874 learned_addr learned my addr 172.21.15.105:0/141687874 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa53259c640 1 --2- 172.21.15.105:0/141687874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5340ad680 0x7fa53414c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa532d9d640 1 --2- 172.21.15.105:0/141687874 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa534159f40 0x7fa5341498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa529d9b640 1 -- 172.21.15.105:0/141687874 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa534159f40 msgr2=0x7fa5341498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa529d9b640 1 --2- 172.21.15.105:0/141687874 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa534159f40 0x7fa5341498b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa529d9b640 1 -- 172.21.15.105:0/141687874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5340ad680 msgr2=0x7fa53414c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:04.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa529d9b640 1 --2- 172.21.15.105:0/141687874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5340ad680 0x7fa53414c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:04.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa529d9b640 1 -- 172.21.15.105:0/141687874 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c009b30 con 0x7fa5340cc760 2023-11-28T18:28:04.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa529d9b640 1 --2- 172.21.15.105:0/141687874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5340cc760 0x7fa534155b30 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa51c019520 tx=0x7fa51c0196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:04.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa52b7fe640 1 -- 172.21.15.105:0/141687874 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0b8670 con 0x7fa5340cc760 2023-11-28T18:28:04.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa52b7fe640 1 -- 172.21.15.105:0/141687874 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c0b87f0 con 0x7fa5340cc760 2023-11-28T18:28:04.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa52b7fe640 1 -- 172.21.15.105:0/141687874 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0b8ba0 con 0x7fa5340cc760 2023-11-28T18:28:04.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa538efe640 1 -- 172.21.15.105:0/141687874 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53414d440 con 0x7fa5340cc760 2023-11-28T18:28:04.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.862+0000 7fa538efe640 1 -- 172.21.15.105:0/141687874 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa534156070 con 0x7fa5340cc760 2023-11-28T18:28:04.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.866+0000 7fa538efe640 1 -- 172.21.15.105:0/141687874 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa53414d960 con 0x7fa5340cc760 2023-11-28T18:28:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.866+0000 7fa52b7fe640 1 -- 172.21.15.105:0/141687874 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa51c0b8d00 con 0x7fa5340cc760 2023-11-28T18:28:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.866+0000 7fa52b7fe640 1 --2- 172.21.15.105:0/141687874 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa5140668d0 0x7fa514068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.866+0000 7fa52b7fe640 1 -- 172.21.15.105:0/141687874 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fa51c11f920 con 0x7fa5340cc760 2023-11-28T18:28:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.866+0000 7fa53259c640 1 --2- 172.21.15.105:0/141687874 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa5140668d0 0x7fa514068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:04.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.866+0000 7fa53259c640 1 --2- 172.21.15.105:0/141687874 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa5140668d0 0x7fa514068d90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa524009ac0 tx=0x7fa524009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:04.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:04.870+0000 7fa52b7fe640 1 -- 172.21.15.105:0/141687874 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa53414d960 con 0x7fa5340cc760 2023-11-28T18:28:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.018+0000 7fa538efe640 1 -- 172.21.15.105:0/141687874 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"} v 0) v1 -- 0x7fa53414d960 con 0x7fa5340cc760 2023-11-28T18:28:05.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.026+0000 7fa52b7fe640 1 -- 172.21.15.105:0/141687874 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"}]=0 v34) v1 ==== 114+0+0 (secure 0 0 0) 0x7fa53414d960 con 0x7fa5340cc760 2023-11-28T18:28:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.030+0000 7fa538efe640 1 -- 172.21.15.105:0/141687874 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa5140668d0 msgr2=0x7fa514068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.030+0000 7fa538efe640 1 --2- 172.21.15.105:0/141687874 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa5140668d0 0x7fa514068d90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa524009ac0 tx=0x7fa524009190 comp rx=0 tx=0).stop 2023-11-28T18:28:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.030+0000 7fa538efe640 1 -- 172.21.15.105:0/141687874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5340cc760 msgr2=0x7fa534155b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.030+0000 7fa538efe640 1 --2- 172.21.15.105:0/141687874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5340cc760 0x7fa534155b30 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa51c019520 tx=0x7fa51c0196b0 comp rx=0 tx=0).stop 2023-11-28T18:28:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.030+0000 7fa538efe640 1 -- 172.21.15.105:0/141687874 shutdown_connections 2023-11-28T18:28:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.030+0000 7fa538efe640 1 --2- 172.21.15.105:0/141687874 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa534159f40 0x7fa5341498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.030+0000 7fa538efe640 1 --2- 172.21.15.105:0/141687874 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa5140668d0 0x7fa514068d90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.030+0000 7fa538efe640 1 --2- 172.21.15.105:0/141687874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5340cc760 0x7fa534155b30 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.030+0000 7fa538efe640 1 --2- 172.21.15.105:0/141687874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5340ad680 0x7fa53414c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.030+0000 7fa538efe640 1 -- 172.21.15.105:0/141687874 >> 172.21.15.105:0/141687874 conn(0x7fa5340ceb90 msgr2=0x7fa5340ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.030+0000 7fa538efe640 1 -- 172.21.15.105:0/141687874 shutdown_connections 2023-11-28T18:28:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.030+0000 7fa538efe640 1 -- 172.21.15.105:0/141687874 wait complete. 2023-11-28T18:28:05.054 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_off' 2023-11-28T18:28:05.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.186+0000 7fd5cf51d640 1 Processor -- start 2023-11-28T18:28:05.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.186+0000 7fd5cf51d640 1 -- start start 2023-11-28T18:28:05.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.190+0000 7fd5cf51d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd5c8159f40 0x7fd5c80caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:05.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.190+0000 7fd5cf51d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5c80cc760 0x7fd5c80ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:05.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.190+0000 7fd5cf51d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd5c80ad680 0x7fd5c815da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:05.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.190+0000 7fd5cf51d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c806bf50 con 0x7fd5c80ad680 2023-11-28T18:28:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.190+0000 7fd5cf51d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c80ccbc0 con 0x7fd5c8159f40 2023-11-28T18:28:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.190+0000 7fd5cf51d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c815a320 con 0x7fd5c80cc760 2023-11-28T18:28:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.190+0000 7fd5cdaba640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd5c80ad680 0x7fd5c815da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.190+0000 7fd5cdaba640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd5c80ad680 0x7fd5c815da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49082/0 (socket says 172.21.15.105:49082) 2023-11-28T18:28:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.190+0000 7fd5cdaba640 1 -- 172.21.15.105:0/718228758 learned_addr learned my addr 172.21.15.105:0/718228758 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.190+0000 7fd5cd2b9640 1 --2- 172.21.15.105:0/718228758 >> v2:172.21.15.148:3300/0 conn(0x7fd5c8159f40 0x7fd5c80caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.190+0000 7fd5ccab8640 1 --2- 172.21.15.105:0/718228758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5c80cc760 0x7fd5c80ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.190+0000 7fd5cdaba640 1 -- 172.21.15.105:0/718228758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5c80cc760 msgr2=0x7fd5c80ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.190+0000 7fd5cdaba640 1 --2- 172.21.15.105:0/718228758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5c80cc760 0x7fd5c80ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.190+0000 7fd5cdaba640 1 -- 172.21.15.105:0/718228758 >> v2:172.21.15.148:3300/0 conn(0x7fd5c8159f40 msgr2=0x7fd5c80caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.190+0000 7fd5cdaba640 1 --2- 172.21.15.105:0/718228758 >> v2:172.21.15.148:3300/0 conn(0x7fd5c8159f40 0x7fd5c80caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.190+0000 7fd5cdaba640 1 -- 172.21.15.105:0/718228758 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c8071380 con 0x7fd5c80ad680 2023-11-28T18:28:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.190+0000 7fd5ccab8640 1 --2- 172.21.15.105:0/718228758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5c80cc760 0x7fd5c80ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.194+0000 7fd5cdaba640 1 --2- 172.21.15.105:0/718228758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c80ad680 0x7fd5c815da20 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd5c806e500 tx=0x7fd5c001eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5c522b8ebf1633 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.194+0000 7fd5bffff640 1 -- 172.21.15.105:0/718228758 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c000b2d0 con 0x7fd5c80ad680 2023-11-28T18:28:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.194+0000 7fd5bffff640 1 -- 172.21.15.105:0/718228758 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c000b430 con 0x7fd5c80ad680 2023-11-28T18:28:05.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.194+0000 7fd5bffff640 1 -- 172.21.15.105:0/718228758 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c00a7bc0 con 0x7fd5c80ad680 2023-11-28T18:28:05.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.194+0000 7fd5cf51d640 1 -- 172.21.15.105:0/718228758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c80ad680 msgr2=0x7fd5c815da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:05.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.194+0000 7fd5cf51d640 1 --2- 172.21.15.105:0/718228758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c80ad680 0x7fd5c815da20 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd5c806e500 tx=0x7fd5c001eb10 comp rx=0 tx=0).stop 2023-11-28T18:28:05.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.194+0000 7fd5cf51d640 1 -- 172.21.15.105:0/718228758 shutdown_connections 2023-11-28T18:28:05.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.194+0000 7fd5cf51d640 1 --2- 172.21.15.105:0/718228758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c80ad680 0x7fd5c815da20 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:05.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.194+0000 7fd5cf51d640 1 --2- 172.21.15.105:0/718228758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5c80cc760 0x7fd5c80ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:05.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.194+0000 7fd5cf51d640 1 --2- 172.21.15.105:0/718228758 >> v2:172.21.15.148:3300/0 conn(0x7fd5c8159f40 0x7fd5c80caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:05.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.194+0000 7fd5cf51d640 1 -- 172.21.15.105:0/718228758 >> 172.21.15.105:0/718228758 conn(0x7fd5c80ceb90 msgr2=0x7fd5c80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:05.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.194+0000 7fd5cf51d640 1 -- 172.21.15.105:0/718228758 shutdown_connections 2023-11-28T18:28:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.194+0000 7fd5cf51d640 1 -- 172.21.15.105:0/718228758 wait complete. 2023-11-28T18:28:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.194+0000 7fd5cf51d640 1 Processor -- start 2023-11-28T18:28:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.194+0000 7fd5cf51d640 1 -- start start 2023-11-28T18:28:05.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.194+0000 7fd5cf51d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c80ad680 0x7fd5c809a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:05.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.194+0000 7fd5cf51d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5c80cc760 0x7fd5c8152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:05.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.194+0000 7fd5cf51d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c8159f40 0x7fd5c815ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.198+0000 7fd5cf51d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c809af40 con 0x7fd5c80ad680 2023-11-28T18:28:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.198+0000 7fd5cf51d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c809b220 con 0x7fd5c8159f40 2023-11-28T18:28:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.198+0000 7fd5cf51d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c809b500 con 0x7fd5c80cc760 2023-11-28T18:28:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.198+0000 7fd5cd2b9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c80ad680 0x7fd5c809a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.198+0000 7fd5cd2b9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c80ad680 0x7fd5c809a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49084/0 (socket says 172.21.15.105:49084) 2023-11-28T18:28:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.198+0000 7fd5cd2b9640 1 -- 172.21.15.105:0/2218415053 learned_addr learned my addr 172.21.15.105:0/2218415053 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.198+0000 7fd5ccab8640 1 --2- 172.21.15.105:0/2218415053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5c80cc760 0x7fd5c8152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.198+0000 7fd5cdaba640 1 --2- 172.21.15.105:0/2218415053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c8159f40 0x7fd5c815ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.198+0000 7fd5cd2b9640 1 -- 172.21.15.105:0/2218415053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5c80cc760 msgr2=0x7fd5c8152530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.198+0000 7fd5cd2b9640 1 --2- 172.21.15.105:0/2218415053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5c80cc760 0x7fd5c8152530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.198+0000 7fd5cd2b9640 1 -- 172.21.15.105:0/2218415053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c8159f40 msgr2=0x7fd5c815ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.198+0000 7fd5cd2b9640 1 --2- 172.21.15.105:0/2218415053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c8159f40 0x7fd5c815ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.198+0000 7fd5cd2b9640 1 -- 172.21.15.105:0/2218415053 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c000bd40 con 0x7fd5c80ad680 2023-11-28T18:28:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.198+0000 7fd5ccab8640 1 --2- 172.21.15.105:0/2218415053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5c80cc760 0x7fd5c8152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.198+0000 7fd5cd2b9640 1 --2- 172.21.15.105:0/2218415053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c80ad680 0x7fd5c809a960 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400e870 tx=0x7fd5c400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.198+0000 7fd5bdffb640 1 -- 172.21.15.105:0/2218415053 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4009510 con 0x7fd5c80ad680 2023-11-28T18:28:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.198+0000 7fd5bdffb640 1 -- 172.21.15.105:0/2218415053 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c400eec0 con 0x7fd5c80ad680 2023-11-28T18:28:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.198+0000 7fd5bdffb640 1 -- 172.21.15.105:0/2218415053 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c400cb40 con 0x7fd5c80ad680 2023-11-28T18:28:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.198+0000 7fd5cf51d640 1 -- 172.21.15.105:0/2218415053 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c809b7e0 con 0x7fd5c80ad680 2023-11-28T18:28:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.198+0000 7fd5cf51d640 1 -- 172.21.15.105:0/2218415053 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c809bfe0 con 0x7fd5c80ad680 2023-11-28T18:28:05.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.198+0000 7fd5cf51d640 1 -- 172.21.15.105:0/2218415053 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c8067570 con 0x7fd5c80ad680 2023-11-28T18:28:05.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.198+0000 7fd5bdffb640 1 -- 172.21.15.105:0/2218415053 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd5c400cca0 con 0x7fd5c80ad680 2023-11-28T18:28:05.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.202+0000 7fd5bdffb640 1 --2- 172.21.15.105:0/2218415053 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5ac0668d0 0x7fd5ac068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:05.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.202+0000 7fd5bdffb640 1 -- 172.21.15.105:0/2218415053 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fd5c40843d0 con 0x7fd5c80ad680 2023-11-28T18:28:05.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.202+0000 7fd5ccab8640 1 --2- 172.21.15.105:0/2218415053 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5ac0668d0 0x7fd5ac068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:05.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.202+0000 7fd5ccab8640 1 --2- 172.21.15.105:0/2218415053 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5ac0668d0 0x7fd5ac068d90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd5c815d520 tx=0x7fd5b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:05.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.206+0000 7fd5bdffb640 1 -- 172.21.15.105:0/2218415053 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd5c40517b0 con 0x7fd5c80ad680 2023-11-28T18:28:05.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.406+0000 7fd5cf51d640 1 -- 172.21.15.105:0/2218415053 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_off"]} v 0) v1 -- 0x7fd5c80cf950 con 0x7fd5c80ad680 2023-11-28T18:28:05.673 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.666+0000 7fd5bdffb640 1 -- 172.21.15.105:0/2218415053 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_off"]}]=0 v252) v1 ==== 150+0+0 (secure 0 0 0) 0x7fd5c4054dd0 con 0x7fd5c80ad680 2023-11-28T18:28:05.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.670+0000 7fd5cf51d640 1 -- 172.21.15.105:0/2218415053 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5ac0668d0 msgr2=0x7fd5ac068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:05.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.670+0000 7fd5cf51d640 1 --2- 172.21.15.105:0/2218415053 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5ac0668d0 0x7fd5ac068d90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd5c815d520 tx=0x7fd5b8008040 comp rx=0 tx=0).stop 2023-11-28T18:28:05.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.670+0000 7fd5cf51d640 1 -- 172.21.15.105:0/2218415053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c80ad680 msgr2=0x7fd5c809a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:05.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.670+0000 7fd5cf51d640 1 --2- 172.21.15.105:0/2218415053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c80ad680 0x7fd5c809a960 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400e870 tx=0x7fd5c400ec70 comp rx=0 tx=0).stop 2023-11-28T18:28:05.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.670+0000 7fd5cf51d640 1 -- 172.21.15.105:0/2218415053 shutdown_connections 2023-11-28T18:28:05.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.670+0000 7fd5cf51d640 1 --2- 172.21.15.105:0/2218415053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c8159f40 0x7fd5c815ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:05.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.670+0000 7fd5cf51d640 1 --2- 172.21.15.105:0/2218415053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5c80cc760 0x7fd5c8152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:05.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.670+0000 7fd5cf51d640 1 --2- 172.21.15.105:0/2218415053 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5ac0668d0 0x7fd5ac068d90 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:05.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.670+0000 7fd5cf51d640 1 --2- 172.21.15.105:0/2218415053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c80ad680 0x7fd5c809a960 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:05.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.670+0000 7fd5cf51d640 1 -- 172.21.15.105:0/2218415053 >> 172.21.15.105:0/2218415053 conn(0x7fd5c80ceb90 msgr2=0x7fd5c80ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:05.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.670+0000 7fd5cf51d640 1 -- 172.21.15.105:0/2218415053 shutdown_connections 2023-11-28T18:28:05.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.670+0000 7fd5cf51d640 1 -- 172.21.15.105:0/2218415053 wait complete. 2023-11-28T18:28:05.695 INFO:tasks.cephfs_test_runner:That turning off ephemeral distributed pin merges subtrees. ... ok 2023-11-28T18:28:05.696 INFO:tasks.cephfs_test_runner:Starting test: test_ephemeral_pin_dist_override (tasks.cephfs.test_exports.TestEphemeralPins) That an ephemeral distributed pin overrides a normal export pin. 2023-11-28T18:28:05.696 INFO:tasks.cephfs_test_runner:test_ephemeral_pin_dist_override (tasks.cephfs.test_exports.TestEphemeralPins) 2023-11-28T18:28:05.696 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:28:05.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.826+0000 7f1f7ccdd640 1 Processor -- start 2023-11-28T18:28:05.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.826+0000 7f1f7ccdd640 1 -- start start 2023-11-28T18:28:05.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.830+0000 7f1f7ccdd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f78159f40 0x7f1f780caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:05.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.830+0000 7f1f7ccdd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1f780cc760 0x7f1f780ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:05.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.830+0000 7f1f7ccdd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7806bf50 con 0x7f1f780ad680 2023-11-28T18:28:05.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.830+0000 7f1f7ccdd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f780ccbc0 con 0x7f1f780cc760 2023-11-28T18:28:05.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.830+0000 7f1f7ccdd640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7815a320 con 0x7f1f78159f40 2023-11-28T18:28:05.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.830+0000 7f1f6dd9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1f780cc760 0x7f1f780ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:05.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.830+0000 7f1f7659c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f78159f40 0x7f1f780caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:05.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.830+0000 7f1f6dd9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1f780cc760 0x7f1f780ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:39090/0 (socket says 172.21.15.105:39090) 2023-11-28T18:28:05.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.830+0000 7f1f6dd9b640 1 -- 172.21.15.105:0/1841666645 learned_addr learned my addr 172.21.15.105:0/1841666645 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:05.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.830+0000 7f1f6dd9b640 1 -- 172.21.15.105:0/1841666645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f78159f40 msgr2=0x7f1f780caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:05.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.830+0000 7f1f6dd9b640 1 --2- 172.21.15.105:0/1841666645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f78159f40 0x7f1f780caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:05.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.830+0000 7f1f6dd9b640 1 -- 172.21.15.105:0/1841666645 >> v1:172.21.15.148:6789/0 conn(0x7f1f780ad680 legacy=0x7f1f7815da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:05.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.830+0000 7f1f6dd9b640 1 -- 172.21.15.105:0/1841666645 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f78071380 con 0x7f1f780cc760 2023-11-28T18:28:05.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.830+0000 7f1f7659c640 1 --2- 172.21.15.105:0/1841666645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f78159f40 0x7f1f780caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:28:05.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.834+0000 7f1f6dd9b640 1 --2- 172.21.15.105:0/1841666645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f780cc760 0x7f1f780ad140 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009e50 tx=0x7f1f6001c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d82f15a295b83ff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:05.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.834+0000 7f1f75d9b640 1 -- 172.21.15.105:0/1841666645 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f600a5810 con 0x7f1f780cc760 2023-11-28T18:28:05.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.834+0000 7f1f75d9b640 1 -- 172.21.15.105:0/1841666645 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f600a5970 con 0x7f1f780cc760 2023-11-28T18:28:05.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.834+0000 7f1f75d9b640 1 -- 172.21.15.105:0/1841666645 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f600a5d00 con 0x7f1f780cc760 2023-11-28T18:28:05.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.834+0000 7f1f7ccdd640 1 -- 172.21.15.105:0/1841666645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f780cc760 msgr2=0x7f1f780ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:05.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.834+0000 7f1f7ccdd640 1 --2- 172.21.15.105:0/1841666645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f780cc760 0x7f1f780ad140 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009e50 tx=0x7f1f6001c950 comp rx=0 tx=0).stop 2023-11-28T18:28:05.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.834+0000 7f1f7ccdd640 1 -- 172.21.15.105:0/1841666645 shutdown_connections 2023-11-28T18:28:05.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.834+0000 7f1f7ccdd640 1 --2- 172.21.15.105:0/1841666645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f780cc760 0x7f1f780ad140 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:05.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.834+0000 7f1f7ccdd640 1 --2- 172.21.15.105:0/1841666645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f78159f40 0x7f1f780caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:05.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.834+0000 7f1f7ccdd640 1 -- 172.21.15.105:0/1841666645 >> 172.21.15.105:0/1841666645 conn(0x7f1f780ceb90 msgr2=0x7f1f780cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:05.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.834+0000 7f1f7ccdd640 1 -- 172.21.15.105:0/1841666645 shutdown_connections 2023-11-28T18:28:05.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.834+0000 7f1f7ccdd640 1 -- 172.21.15.105:0/1841666645 wait complete. 2023-11-28T18:28:05.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.834+0000 7f1f7ccdd640 1 Processor -- start 2023-11-28T18:28:05.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f7ccdd640 1 -- start start 2023-11-28T18:28:05.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f7ccdd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f780ad680 0x7f1f7814d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:05.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f7ccdd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f780cc760 0x7f1f780b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:05.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f7ccdd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f78159f40 0x7f1f780aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:05.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f7ccdd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f78157220 con 0x7f1f780ad680 2023-11-28T18:28:05.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f7ccdd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f780b33a0 con 0x7f1f780cc760 2023-11-28T18:28:05.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f7ccdd640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f78099a50 con 0x7f1f78159f40 2023-11-28T18:28:05.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f7659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f780ad680 0x7f1f7814d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:05.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f7659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f780ad680 0x7f1f7814d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49088/0 (socket says 172.21.15.105:49088) 2023-11-28T18:28:05.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f7659c640 1 -- 172.21.15.105:0/3015614465 learned_addr learned my addr 172.21.15.105:0/3015614465 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:05.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f6dd9b640 1 --2- 172.21.15.105:0/3015614465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f780cc760 0x7f1f780b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:05.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f76d9d640 1 --2- 172.21.15.105:0/3015614465 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f78159f40 0x7f1f780aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:05.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f7659c640 1 -- 172.21.15.105:0/3015614465 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f78159f40 msgr2=0x7f1f780aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:05.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f7659c640 1 --2- 172.21.15.105:0/3015614465 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f78159f40 0x7f1f780aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:05.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f7659c640 1 -- 172.21.15.105:0/3015614465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f780cc760 msgr2=0x7f1f780b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:05.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f7659c640 1 --2- 172.21.15.105:0/3015614465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f780cc760 0x7f1f780b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:05.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f7659c640 1 -- 172.21.15.105:0/3015614465 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f68009420 con 0x7f1f780ad680 2023-11-28T18:28:05.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f7659c640 1 --2- 172.21.15.105:0/3015614465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f780ad680 0x7f1f7814d980 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800e980 tx=0x7f1f6800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:05.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f6f7fe640 1 -- 172.21.15.105:0/3015614465 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6800ca00 con 0x7f1f780ad680 2023-11-28T18:28:05.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f6f7fe640 1 -- 172.21.15.105:0/3015614465 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6800cb60 con 0x7f1f780ad680 2023-11-28T18:28:05.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f7ccdd640 1 -- 172.21.15.105:0/3015614465 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f60009b30 con 0x7f1f780ad680 2023-11-28T18:28:05.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f6f7fe640 1 -- 172.21.15.105:0/3015614465 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f68013470 con 0x7f1f780ad680 2023-11-28T18:28:05.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f7ccdd640 1 -- 172.21.15.105:0/3015614465 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7809a010 con 0x7f1f780ad680 2023-11-28T18:28:05.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.838+0000 7f1f7ccdd640 1 -- 172.21.15.105:0/3015614465 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f78067570 con 0x7f1f780ad680 2023-11-28T18:28:05.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.842+0000 7f1f6f7fe640 1 -- 172.21.15.105:0/3015614465 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f1f680135d0 con 0x7f1f780ad680 2023-11-28T18:28:05.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.842+0000 7f1f6f7fe640 1 --2- 172.21.15.105:0/3015614465 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1f580668d0 0x7f1f58068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:05.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.842+0000 7f1f6f7fe640 1 -- 172.21.15.105:0/3015614465 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f1f68084500 con 0x7f1f780ad680 2023-11-28T18:28:05.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.842+0000 7f1f6dd9b640 1 --2- 172.21.15.105:0/3015614465 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1f580668d0 0x7f1f58068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:05.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.842+0000 7f1f6dd9b640 1 --2- 172.21.15.105:0/3015614465 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1f580668d0 0x7f1f58068d90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1f600ae000 tx=0x7f1f600a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:05.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.846+0000 7f1f6f7fe640 1 -- 172.21.15.105:0/3015614465 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1f680518f0 con 0x7f1f780ad680 2023-11-28T18:28:05.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.994+0000 7f1f7ccdd640 1 -- 172.21.15.105:0/3015614465 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f780cf6c0 con 0x7f1f780ad680 2023-11-28T18:28:05.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.994+0000 7f1f6f7fe640 1 -- 172.21.15.105:0/3015614465 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+13516 (secure 0 0 0) 0x7f1f68054f10 con 0x7f1f780ad680 2023-11-28T18:28:05.999 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:06.000 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":70,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:28:03.663612+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:6834/380088531":"2023-11-29T18:28:00.655133+0000","172.21.15.148:6835/2396073410":"2023-11-29T18:28:00.655120+0000","172.21.15.105:6835/380088531":"2023-11-29T18:28:00.655133+0000","172.21.15.148:6833/2396073410":"2023-11-29T18:28:00.655120+0000","172.21.15.105:6837/3442512486":"2023-11-29T18:28:00.655138+0000","172.21.15.105:6836/3442512486":"2023-11-29T18:28:00.655138+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:28:06.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.998+0000 7f1f7ccdd640 1 -- 172.21.15.105:0/3015614465 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1f580668d0 msgr2=0x7f1f58068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:06.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.998+0000 7f1f7ccdd640 1 --2- 172.21.15.105:0/3015614465 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1f580668d0 0x7f1f58068d90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1f600ae000 tx=0x7f1f600a7070 comp rx=0 tx=0).stop 2023-11-28T18:28:06.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.998+0000 7f1f7ccdd640 1 -- 172.21.15.105:0/3015614465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f780ad680 msgr2=0x7f1f7814d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:06.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.998+0000 7f1f7ccdd640 1 --2- 172.21.15.105:0/3015614465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f780ad680 0x7f1f7814d980 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800e980 tx=0x7f1f6800ee40 comp rx=0 tx=0).stop 2023-11-28T18:28:06.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.998+0000 7f1f7ccdd640 1 -- 172.21.15.105:0/3015614465 shutdown_connections 2023-11-28T18:28:06.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.998+0000 7f1f7ccdd640 1 --2- 172.21.15.105:0/3015614465 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f78159f40 0x7f1f780aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:06.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.998+0000 7f1f7ccdd640 1 --2- 172.21.15.105:0/3015614465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f780cc760 0x7f1f780b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:06.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.998+0000 7f1f7ccdd640 1 --2- 172.21.15.105:0/3015614465 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1f580668d0 0x7f1f58068d90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:06.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.998+0000 7f1f7ccdd640 1 --2- 172.21.15.105:0/3015614465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f780ad680 0x7f1f7814d980 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:06.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.998+0000 7f1f7ccdd640 1 -- 172.21.15.105:0/3015614465 >> 172.21.15.105:0/3015614465 conn(0x7f1f780ceb90 msgr2=0x7f1f78073020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:06.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.998+0000 7f1f7ccdd640 1 -- 172.21.15.105:0/3015614465 shutdown_connections 2023-11-28T18:28:06.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:05.998+0000 7f1f7ccdd640 1 -- 172.21.15.105:0/3015614465 wait complete. 2023-11-28T18:28:06.020 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:28:06.021 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:28:06.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.158+0000 7fe830965640 1 Processor -- start 2023-11-28T18:28:06.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.158+0000 7fe830965640 1 -- start start 2023-11-28T18:28:06.163 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.158+0000 7fe830965640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe82c159f40 0x7fe82c0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:06.163 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.158+0000 7fe830965640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe82c0cc760 0x7fe82c0ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:06.163 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.158+0000 7fe830965640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe82c0ad680 0x7fe82c15da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:06.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.158+0000 7fe830965640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe82c06bf50 con 0x7fe82c0ad680 2023-11-28T18:28:06.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.158+0000 7fe830965640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe82c0ccbc0 con 0x7fe82c159f40 2023-11-28T18:28:06.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.158+0000 7fe830965640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c15a320 con 0x7fe82c0cc760 2023-11-28T18:28:06.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.158+0000 7fe82ad9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe82c0ad680 0x7fe82c15da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:06.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.158+0000 7fe82ad9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe82c0ad680 0x7fe82c15da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49092/0 (socket says 172.21.15.105:49092) 2023-11-28T18:28:06.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.158+0000 7fe82ad9d640 1 -- 172.21.15.105:0/2368911785 learned_addr learned my addr 172.21.15.105:0/2368911785 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:06.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.158+0000 7fe82a59c640 1 --2- 172.21.15.105:0/2368911785 >> v2:172.21.15.148:3300/0 conn(0x7fe82c159f40 0x7fe82c0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:06.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.158+0000 7fe823fff640 1 --2- 172.21.15.105:0/2368911785 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe82c0cc760 0x7fe82c0ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:06.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.158+0000 7fe82ad9d640 1 -- 172.21.15.105:0/2368911785 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe82c0cc760 msgr2=0x7fe82c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:06.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.158+0000 7fe82ad9d640 1 --2- 172.21.15.105:0/2368911785 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe82c0cc760 0x7fe82c0ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:06.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.158+0000 7fe82ad9d640 1 -- 172.21.15.105:0/2368911785 >> v2:172.21.15.148:3300/0 conn(0x7fe82c159f40 msgr2=0x7fe82c0caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:06.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.158+0000 7fe82ad9d640 1 --2- 172.21.15.105:0/2368911785 >> v2:172.21.15.148:3300/0 conn(0x7fe82c159f40 0x7fe82c0caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:06.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.158+0000 7fe82ad9d640 1 -- 172.21.15.105:0/2368911785 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82c071380 con 0x7fe82c0ad680 2023-11-28T18:28:06.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.158+0000 7fe82a59c640 1 --2- 172.21.15.105:0/2368911785 >> v2:172.21.15.148:3300/0 conn(0x7fe82c159f40 0x7fe82c0caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:06.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.162+0000 7fe82ad9d640 1 --2- 172.21.15.105:0/2368911785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe82c0ad680 0x7fe82c15da20 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fe82c06e500 tx=0x7fe82401eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d83e0680419a51 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:06.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.162+0000 7fe829d9b640 1 -- 172.21.15.105:0/2368911785 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82400b2d0 con 0x7fe82c0ad680 2023-11-28T18:28:06.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.162+0000 7fe829d9b640 1 -- 172.21.15.105:0/2368911785 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82400b430 con 0x7fe82c0ad680 2023-11-28T18:28:06.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.162+0000 7fe829d9b640 1 -- 172.21.15.105:0/2368911785 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8240a7bc0 con 0x7fe82c0ad680 2023-11-28T18:28:06.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.162+0000 7fe830965640 1 -- 172.21.15.105:0/2368911785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe82c0ad680 msgr2=0x7fe82c15da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:06.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.162+0000 7fe830965640 1 --2- 172.21.15.105:0/2368911785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe82c0ad680 0x7fe82c15da20 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fe82c06e500 tx=0x7fe82401eb10 comp rx=0 tx=0).stop 2023-11-28T18:28:06.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.162+0000 7fe830965640 1 -- 172.21.15.105:0/2368911785 shutdown_connections 2023-11-28T18:28:06.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.162+0000 7fe830965640 1 --2- 172.21.15.105:0/2368911785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe82c0ad680 0x7fe82c15da20 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:06.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.162+0000 7fe830965640 1 --2- 172.21.15.105:0/2368911785 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe82c0cc760 0x7fe82c0ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:06.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.162+0000 7fe830965640 1 --2- 172.21.15.105:0/2368911785 >> v2:172.21.15.148:3300/0 conn(0x7fe82c159f40 0x7fe82c0caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:06.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.162+0000 7fe830965640 1 -- 172.21.15.105:0/2368911785 >> 172.21.15.105:0/2368911785 conn(0x7fe82c0ceb90 msgr2=0x7fe82c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:06.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.162+0000 7fe830965640 1 -- 172.21.15.105:0/2368911785 shutdown_connections 2023-11-28T18:28:06.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.162+0000 7fe830965640 1 -- 172.21.15.105:0/2368911785 wait complete. 2023-11-28T18:28:06.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe830965640 1 Processor -- start 2023-11-28T18:28:06.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe830965640 1 -- start start 2023-11-28T18:28:06.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe830965640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe82c0ad680 0x7fe82c1640c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:06.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe830965640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe82c0cc760 0x7fe82c14e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:06.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe830965640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe82c159f40 0x7fe82c14ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:06.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe830965640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c163950 con 0x7fe82c0cc760 2023-11-28T18:28:06.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe830965640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c15cc30 con 0x7fe82c159f40 2023-11-28T18:28:06.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe830965640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c15cf10 con 0x7fe82c0ad680 2023-11-28T18:28:06.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe823fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe82c0cc760 0x7fe82c14e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:06.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe823fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe82c0cc760 0x7fe82c14e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49104/0 (socket says 172.21.15.105:49104) 2023-11-28T18:28:06.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe823fff640 1 -- 172.21.15.105:0/574020295 learned_addr learned my addr 172.21.15.105:0/574020295 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:06.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe82a59c640 1 --2- 172.21.15.105:0/574020295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe82c0ad680 0x7fe82c1640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:06.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe82ad9d640 1 --2- 172.21.15.105:0/574020295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe82c159f40 0x7fe82c14ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:06.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe823fff640 1 -- 172.21.15.105:0/574020295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe82c0ad680 msgr2=0x7fe82c1640c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:06.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe823fff640 1 --2- 172.21.15.105:0/574020295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe82c0ad680 0x7fe82c1640c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:06.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe823fff640 1 -- 172.21.15.105:0/574020295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe82c159f40 msgr2=0x7fe82c14ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:06.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe823fff640 1 --2- 172.21.15.105:0/574020295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe82c159f40 0x7fe82c14ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:06.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe823fff640 1 -- 172.21.15.105:0/574020295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82400bd40 con 0x7fe82c0cc760 2023-11-28T18:28:06.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe82ad9d640 1 --2- 172.21.15.105:0/574020295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe82c159f40 0x7fe82c14ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:06.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe823fff640 1 --2- 172.21.15.105:0/574020295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe82c0cc760 0x7fe82c14e7c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fe81400d800 tx=0x7fe81400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:06.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe822ffd640 1 -- 172.21.15.105:0/574020295 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8140096a0 con 0x7fe82c0cc760 2023-11-28T18:28:06.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe822ffd640 1 -- 172.21.15.105:0/574020295 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe814009800 con 0x7fe82c0cc760 2023-11-28T18:28:06.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe822ffd640 1 -- 172.21.15.105:0/574020295 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814009b70 con 0x7fe82c0cc760 2023-11-28T18:28:06.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe830965640 1 -- 172.21.15.105:0/574020295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe82c15d1f0 con 0x7fe82c0cc760 2023-11-28T18:28:06.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe830965640 1 -- 172.21.15.105:0/574020295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe82c150880 con 0x7fe82c0cc760 2023-11-28T18:28:06.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.166+0000 7fe830965640 1 -- 172.21.15.105:0/574020295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe82c067570 con 0x7fe82c0cc760 2023-11-28T18:28:06.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.170+0000 7fe822ffd640 1 -- 172.21.15.105:0/574020295 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fe814009cd0 con 0x7fe82c0cc760 2023-11-28T18:28:06.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.170+0000 7fe822ffd640 1 --2- 172.21.15.105:0/574020295 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe80c0668d0 0x7fe80c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:06.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.170+0000 7fe822ffd640 1 -- 172.21.15.105:0/574020295 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fe8140849e0 con 0x7fe82c0cc760 2023-11-28T18:28:06.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.170+0000 7fe82a59c640 1 --2- 172.21.15.105:0/574020295 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe80c0668d0 0x7fe80c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:06.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.170+0000 7fe82a59c640 1 --2- 172.21.15.105:0/574020295 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe80c0668d0 0x7fe80c068d90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0ac860 tx=0x7fe81c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:06.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.174+0000 7fe822ffd640 1 -- 172.21.15.105:0/574020295 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe814051d30 con 0x7fe82c0cc760 2023-11-28T18:28:06.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.326+0000 7fe830965640 1 -- 172.21.15.105:0/574020295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe82c0cf950 con 0x7fe82c0cc760 2023-11-28T18:28:06.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.326+0000 7fe822ffd640 1 -- 172.21.15.105:0/574020295 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe814055350 con 0x7fe82c0cc760 2023-11-28T18:28:06.331 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:28:06.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.326+0000 7fe830965640 1 -- 172.21.15.105:0/574020295 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe80c0668d0 msgr2=0x7fe80c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:06.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.326+0000 7fe830965640 1 --2- 172.21.15.105:0/574020295 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe80c0668d0 0x7fe80c068d90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0ac860 tx=0x7fe81c008040 comp rx=0 tx=0).stop 2023-11-28T18:28:06.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.326+0000 7fe830965640 1 -- 172.21.15.105:0/574020295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe82c0cc760 msgr2=0x7fe82c14e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:06.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.326+0000 7fe830965640 1 --2- 172.21.15.105:0/574020295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe82c0cc760 0x7fe82c14e7c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fe81400d800 tx=0x7fe81400dcc0 comp rx=0 tx=0).stop 2023-11-28T18:28:06.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.326+0000 7fe830965640 1 -- 172.21.15.105:0/574020295 shutdown_connections 2023-11-28T18:28:06.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.326+0000 7fe830965640 1 --2- 172.21.15.105:0/574020295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe82c159f40 0x7fe82c14ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:06.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.326+0000 7fe830965640 1 --2- 172.21.15.105:0/574020295 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe80c0668d0 0x7fe80c068d90 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:06.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.326+0000 7fe830965640 1 --2- 172.21.15.105:0/574020295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe82c0cc760 0x7fe82c14e7c0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:06.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.326+0000 7fe830965640 1 --2- 172.21.15.105:0/574020295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe82c0ad680 0x7fe82c1640c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:06.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.326+0000 7fe830965640 1 -- 172.21.15.105:0/574020295 >> 172.21.15.105:0/574020295 conn(0x7fe82c0ceb90 msgr2=0x7fe82c070960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:06.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.326+0000 7fe830965640 1 -- 172.21.15.105:0/574020295 shutdown_connections 2023-11-28T18:28:06.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.326+0000 7fe830965640 1 -- 172.21.15.105:0/574020295 wait complete. 2023-11-28T18:28:06.352 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override' 2023-11-28T18:28:06.490 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.486+0000 7f5bee26b640 1 Processor -- start 2023-11-28T18:28:06.490 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.486+0000 7f5bee26b640 1 -- start start 2023-11-28T18:28:06.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.486+0000 7f5bee26b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be80caf60 0x7f5be80cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:06.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.486+0000 7f5bee26b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5be80ad850 0x7f5be80701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:06.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.486+0000 7f5bee26b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be806bf50 con 0x7f5be8159f40 2023-11-28T18:28:06.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.486+0000 7f5bee26b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be80cb3c0 con 0x7f5be80ad850 2023-11-28T18:28:06.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.486+0000 7f5bee26b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be815a320 con 0x7f5be80caf60 2023-11-28T18:28:06.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.486+0000 7f5be7fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5be8159f40 0x7f5be80ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42010/0 (socket says 172.21.15.105:42010) 2023-11-28T18:28:06.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.486+0000 7f5be7fff640 1 -- 172.21.15.105:0/2606665410 learned_addr learned my addr 172.21.15.105:0/2606665410 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:06.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.486+0000 7f5bec808640 1 --2- 172.21.15.105:0/2606665410 >> v2:172.21.15.105:3300/0 conn(0x7f5be80ad850 0x7f5be80701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:06.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.486+0000 7f5be77fe640 1 --2- 172.21.15.105:0/2606665410 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be80caf60 0x7f5be80cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:06.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.486+0000 7f5be6ffd640 1 -- 172.21.15.105:0/2606665410 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3809753356 0 0) 0x7f5be806bf50 con 0x7f5be8159f40 2023-11-28T18:28:06.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.486+0000 7f5be6ffd640 1 -- 172.21.15.105:0/2606665410 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bd0005740 con 0x7f5be8159f40 2023-11-28T18:28:06.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.486+0000 7f5bec808640 1 -- 172.21.15.105:0/2606665410 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be80caf60 msgr2=0x7f5be80cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:06.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.486+0000 7f5bec808640 1 --2- 172.21.15.105:0/2606665410 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be80caf60 0x7f5be80cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:06.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.486+0000 7f5bec808640 1 -- 172.21.15.105:0/2606665410 >> v1:172.21.15.105:6789/0 conn(0x7f5be8159f40 legacy=0x7f5be80ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:06.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.486+0000 7f5bec808640 1 -- 172.21.15.105:0/2606665410 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be8071380 con 0x7f5be80ad850 2023-11-28T18:28:06.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.486+0000 7f5be77fe640 1 --2- 172.21.15.105:0/2606665410 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be80caf60 0x7f5be80cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:06.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.490+0000 7f5bec808640 1 --2- 172.21.15.105:0/2606665410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be80ad850 0x7f5be80701d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009d90 tx=0x7f5be001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60132c6715453591 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:06.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.490+0000 7f5be6ffd640 1 -- 172.21.15.105:0/2606665410 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be00a5950 con 0x7f5be80ad850 2023-11-28T18:28:06.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.490+0000 7f5be6ffd640 1 -- 172.21.15.105:0/2606665410 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be00a5ab0 con 0x7f5be80ad850 2023-11-28T18:28:06.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.490+0000 7f5be6ffd640 1 -- 172.21.15.105:0/2606665410 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be00a5e20 con 0x7f5be80ad850 2023-11-28T18:28:06.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.490+0000 7f5bee26b640 1 -- 172.21.15.105:0/2606665410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be80ad850 msgr2=0x7f5be80701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:06.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.490+0000 7f5bee26b640 1 --2- 172.21.15.105:0/2606665410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be80ad850 0x7f5be80701d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009d90 tx=0x7f5be001c890 comp rx=0 tx=0).stop 2023-11-28T18:28:06.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.490+0000 7f5bee26b640 1 -- 172.21.15.105:0/2606665410 shutdown_connections 2023-11-28T18:28:06.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.490+0000 7f5bee26b640 1 --2- 172.21.15.105:0/2606665410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be80ad850 0x7f5be80701d0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:06.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.490+0000 7f5bee26b640 1 --2- 172.21.15.105:0/2606665410 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be80caf60 0x7f5be80cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:06.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.490+0000 7f5bee26b640 1 -- 172.21.15.105:0/2606665410 >> 172.21.15.105:0/2606665410 conn(0x7f5be80ceb90 msgr2=0x7f5be80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:06.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.490+0000 7f5bee26b640 1 -- 172.21.15.105:0/2606665410 shutdown_connections 2023-11-28T18:28:06.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.490+0000 7f5bee26b640 1 -- 172.21.15.105:0/2606665410 wait complete. 2023-11-28T18:28:06.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5bee26b640 1 Processor -- start 2023-11-28T18:28:06.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5bee26b640 1 -- start start 2023-11-28T18:28:06.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5bee26b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be80ad850 0x7f5be80b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:06.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5bee26b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be80caf60 0x7f5be8157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:06.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5bee26b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be8159f40 0x7f5be8158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:06.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5bee26b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be80b23f0 con 0x7f5be80caf60 2023-11-28T18:28:06.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5bee26b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be809b2a0 con 0x7f5be80ad850 2023-11-28T18:28:06.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5bee26b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be80b48d0 con 0x7f5be8159f40 2023-11-28T18:28:06.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5be77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be80caf60 0x7f5be8157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:06.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5be77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be80caf60 0x7f5be8157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49112/0 (socket says 172.21.15.105:49112) 2023-11-28T18:28:06.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5be77fe640 1 -- 172.21.15.105:0/3052959754 learned_addr learned my addr 172.21.15.105:0/3052959754 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:06.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5be7fff640 1 --2- 172.21.15.105:0/3052959754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be80ad850 0x7f5be80b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:06.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5bec808640 1 --2- 172.21.15.105:0/3052959754 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be8159f40 0x7f5be8158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:06.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5be77fe640 1 -- 172.21.15.105:0/3052959754 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be8159f40 msgr2=0x7f5be8158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:06.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5be77fe640 1 --2- 172.21.15.105:0/3052959754 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be8159f40 0x7f5be8158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:06.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5be77fe640 1 -- 172.21.15.105:0/3052959754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be80ad850 msgr2=0x7f5be80b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:06.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5be77fe640 1 --2- 172.21.15.105:0/3052959754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be80ad850 0x7f5be80b4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:06.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5be77fe640 1 -- 172.21.15.105:0/3052959754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be0009a70 con 0x7f5be80caf60 2023-11-28T18:28:06.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5bec808640 1 --2- 172.21.15.105:0/3052959754 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be8159f40 0x7f5be8158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:06.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5be77fe640 1 --2- 172.21.15.105:0/3052959754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be80caf60 0x7f5be8157ae0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00e810 tx=0x7f5bdc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:06.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5be4ff9640 1 -- 172.21.15.105:0/3052959754 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc00c8d0 con 0x7f5be80caf60 2023-11-28T18:28:06.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5be4ff9640 1 -- 172.21.15.105:0/3052959754 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bdc00ca30 con 0x7f5be80caf60 2023-11-28T18:28:06.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5be4ff9640 1 -- 172.21.15.105:0/3052959754 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc00cdc0 con 0x7f5be80caf60 2023-11-28T18:28:06.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5bee26b640 1 -- 172.21.15.105:0/3052959754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be809ca70 con 0x7f5be80caf60 2023-11-28T18:28:06.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.494+0000 7f5bee26b640 1 -- 172.21.15.105:0/3052959754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be80b1d30 con 0x7f5be80caf60 2023-11-28T18:28:06.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.498+0000 7f5bee26b640 1 -- 172.21.15.105:0/3052959754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be8067570 con 0x7f5be80caf60 2023-11-28T18:28:06.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.498+0000 7f5be4ff9640 1 -- 172.21.15.105:0/3052959754 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5bdc013410 con 0x7f5be80caf60 2023-11-28T18:28:06.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.498+0000 7f5be4ff9640 1 --2- 172.21.15.105:0/3052959754 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5bd40668d0 0x7f5bd4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:06.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.498+0000 7f5be4ff9640 1 -- 172.21.15.105:0/3052959754 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f5bdc0841a0 con 0x7f5be80caf60 2023-11-28T18:28:06.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.498+0000 7f5be7fff640 1 --2- 172.21.15.105:0/3052959754 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5bd40668d0 0x7f5bd4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:06.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.498+0000 7f5be7fff640 1 --2- 172.21.15.105:0/3052959754 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5bd40668d0 0x7f5bd4068d90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8005890 tx=0x7f5bd8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:06.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.502+0000 7f5be4ff9640 1 -- 172.21.15.105:0/3052959754 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5bdc0514f0 con 0x7f5be80caf60 2023-11-28T18:28:06.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:06.702+0000 7f5bee26b640 1 -- 172.21.15.105:0/3052959754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override"]} v 0) v1 -- 0x7f5be80cf950 con 0x7f5be80caf60 2023-11-28T18:28:07.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.678+0000 7f5be4ff9640 1 -- 172.21.15.105:0/3052959754 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override"]}]=0 v254) v1 ==== 158+0+0 (secure 0 0 0) 0x7f5bdc054b10 con 0x7f5be80caf60 2023-11-28T18:28:07.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.678+0000 7f5bee26b640 1 -- 172.21.15.105:0/3052959754 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5bd40668d0 msgr2=0x7f5bd4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:07.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.678+0000 7f5bee26b640 1 --2- 172.21.15.105:0/3052959754 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5bd40668d0 0x7f5bd4068d90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8005890 tx=0x7f5bd8009040 comp rx=0 tx=0).stop 2023-11-28T18:28:07.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.678+0000 7f5bee26b640 1 -- 172.21.15.105:0/3052959754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be80caf60 msgr2=0x7f5be8157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:07.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.678+0000 7f5bee26b640 1 --2- 172.21.15.105:0/3052959754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be80caf60 0x7f5be8157ae0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00e810 tx=0x7f5bdc00ecd0 comp rx=0 tx=0).stop 2023-11-28T18:28:07.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.678+0000 7f5bee26b640 1 -- 172.21.15.105:0/3052959754 shutdown_connections 2023-11-28T18:28:07.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.678+0000 7f5bee26b640 1 --2- 172.21.15.105:0/3052959754 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be8159f40 0x7f5be8158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:07.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.678+0000 7f5bee26b640 1 --2- 172.21.15.105:0/3052959754 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5bd40668d0 0x7f5bd4068d90 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:07.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.678+0000 7f5bee26b640 1 --2- 172.21.15.105:0/3052959754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be80caf60 0x7f5be8157ae0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:07.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.678+0000 7f5bee26b640 1 --2- 172.21.15.105:0/3052959754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be80ad850 0x7f5be80b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:07.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.678+0000 7f5bee26b640 1 -- 172.21.15.105:0/3052959754 >> 172.21.15.105:0/3052959754 conn(0x7f5be80ceb90 msgr2=0x7f5be80cd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:07.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.678+0000 7f5bee26b640 1 -- 172.21.15.105:0/3052959754 shutdown_connections 2023-11-28T18:28:07.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.678+0000 7f5bee26b640 1 -- 172.21.15.105:0/3052959754 wait complete. 2023-11-28T18:28:07.708 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T18:28:07.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.846+0000 7f6a05ac9640 1 Processor -- start 2023-11-28T18:28:07.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.846+0000 7f6a05ac9640 1 -- start start 2023-11-28T18:28:07.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.846+0000 7f6a05ac9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6a00159f40 0x7f6a000caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:07.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.846+0000 7f6a05ac9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a0006bf50 con 0x7f6a000cc760 2023-11-28T18:28:07.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.846+0000 7f6a05ac9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a000ccbc0 con 0x7f6a000ad850 2023-11-28T18:28:07.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.846+0000 7f6a05ac9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a0015a320 con 0x7f6a00159f40 2023-11-28T18:28:07.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.846+0000 7f69feffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f6a000cc760 0x7f6a000ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42022/0 (socket says 172.21.15.105:42022) 2023-11-28T18:28:07.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.846+0000 7f69feffd640 1 -- 172.21.15.105:0/534890191 learned_addr learned my addr 172.21.15.105:0/534890191 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:07.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.846+0000 7f69ff7fe640 1 --2- 172.21.15.105:0/534890191 >> v2:172.21.15.148:3300/0 conn(0x7f6a00159f40 0x7f6a000caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:07.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.846+0000 7f69fe7fc640 1 -- 172.21.15.105:0/534890191 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3035593976 0 0) 0x7f6a0006bf50 con 0x7f6a000cc760 2023-11-28T18:28:07.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.846+0000 7f69fe7fc640 1 -- 172.21.15.105:0/534890191 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69e8005740 con 0x7f6a000cc760 2023-11-28T18:28:07.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69fe7fc640 1 -- 172.21.15.105:0/534890191 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1883797820 0 0) 0x7f6a000ccbc0 con 0x7f6a000ad850 2023-11-28T18:28:07.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69fe7fc640 1 -- 172.21.15.105:0/534890191 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a0006bf50 con 0x7f6a000ad850 2023-11-28T18:28:07.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69fe7fc640 1 -- 172.21.15.105:0/534890191 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4235830128 0 0) 0x7f69e8005740 con 0x7f6a000cc760 2023-11-28T18:28:07.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69fe7fc640 1 -- 172.21.15.105:0/534890191 >> v2:172.21.15.148:3300/0 conn(0x7f6a00159f40 msgr2=0x7f6a000caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:07.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69fe7fc640 1 --2- 172.21.15.105:0/534890191 >> v2:172.21.15.148:3300/0 conn(0x7f6a00159f40 0x7f6a000caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:07.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69fe7fc640 1 -- 172.21.15.105:0/534890191 >> v1:172.21.15.148:6789/0 conn(0x7f6a000ad850 legacy=0x7f6a000701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:07.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69fe7fc640 1 -- 172.21.15.105:0/534890191 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a00071380 con 0x7f6a000cc760 2023-11-28T18:28:07.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69fe7fc640 1 -- 172.21.15.105:0/534890191 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f69f4001680 con 0x7f6a000cc760 2023-11-28T18:28:07.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69fe7fc640 1 -- 172.21.15.105:0/534890191 >> v1:172.21.15.105:6789/0 conn(0x7f6a000cc760 legacy=0x7f6a000ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:07.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69fe7fc640 1 --2- 172.21.15.105:0/534890191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69e8005d20 0x7f69e8006120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:07.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69fe7fc640 1 --2- 172.21.15.105:0/534890191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69e80069e0 0x7f69e8006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:07.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69fe7fc640 1 --2- 172.21.15.105:0/534890191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69e80073a0 0x7f69e800dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:07.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69fe7fc640 1 -- 172.21.15.105:0/534890191 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0015a320 con 0x7f69e8005d20 2023-11-28T18:28:07.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69fe7fc640 1 -- 172.21.15.105:0/534890191 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e800e2d0 con 0x7f69e80073a0 2023-11-28T18:28:07.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69fe7fc640 1 -- 172.21.15.105:0/534890191 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e800e490 con 0x7f69e80069e0 2023-11-28T18:28:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69ff7fe640 1 --2- 172.21.15.105:0/534890191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69e8005d20 0x7f69e8006120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69fffff640 1 --2- 172.21.15.105:0/534890191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69e80073a0 0x7f69e800dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69feffd640 1 --2- 172.21.15.105:0/534890191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69e80069e0 0x7f69e8006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69ff7fe640 1 -- 172.21.15.105:0/534890191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69e80069e0 msgr2=0x7f69e8006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69ff7fe640 1 --2- 172.21.15.105:0/534890191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69e80069e0 0x7f69e8006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69ff7fe640 1 -- 172.21.15.105:0/534890191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69e80073a0 msgr2=0x7f69e800dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69ff7fe640 1 --2- 172.21.15.105:0/534890191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69e80073a0 0x7f69e800dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69ff7fe640 1 -- 172.21.15.105:0/534890191 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e8005740 con 0x7f69e8005d20 2023-11-28T18:28:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69fffff640 1 --2- 172.21.15.105:0/534890191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69e80073a0 0x7f69e800dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69fffff640 1 -- 172.21.15.105:0/534890191 reap_dead start 2023-11-28T18:28:07.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.850+0000 7f69ff7fe640 1 --2- 172.21.15.105:0/534890191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69e8005d20 0x7f69e8006120 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f69f0009ae0 tx=0x7f69f001ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1409089c6a42f7e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:07.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.854+0000 7f69fe7fc640 1 -- 172.21.15.105:0/534890191 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f00a6410 con 0x7f69e8005d20 2023-11-28T18:28:07.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.854+0000 7f69fe7fc640 1 -- 172.21.15.105:0/534890191 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f00a6570 con 0x7f69e8005d20 2023-11-28T18:28:07.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.854+0000 7f69fe7fc640 1 -- 172.21.15.105:0/534890191 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f00a6920 con 0x7f69e8005d20 2023-11-28T18:28:07.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.854+0000 7f6a05ac9640 1 -- 172.21.15.105:0/534890191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69e8005d20 msgr2=0x7f69e8006120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:07.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.854+0000 7f6a05ac9640 1 --2- 172.21.15.105:0/534890191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69e8005d20 0x7f69e8006120 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f69f0009ae0 tx=0x7f69f001ca20 comp rx=0 tx=0).stop 2023-11-28T18:28:07.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.854+0000 7f6a05ac9640 1 -- 172.21.15.105:0/534890191 shutdown_connections 2023-11-28T18:28:07.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.854+0000 7f6a05ac9640 1 --2- 172.21.15.105:0/534890191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69e8005d20 0x7f69e8006120 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:07.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.854+0000 7f6a05ac9640 1 -- 172.21.15.105:0/534890191 >> 172.21.15.105:0/534890191 conn(0x7f6a000ceb90 msgr2=0x7f6a000cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:07.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.854+0000 7f6a05ac9640 1 -- 172.21.15.105:0/534890191 shutdown_connections 2023-11-28T18:28:07.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.854+0000 7f6a05ac9640 1 -- 172.21.15.105:0/534890191 wait complete. 2023-11-28T18:28:07.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.854+0000 7f6a05ac9640 1 Processor -- start 2023-11-28T18:28:07.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.854+0000 7f6a05ac9640 1 -- start start 2023-11-28T18:28:07.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.854+0000 7f6a05ac9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69e8005d20 0x7f6a00153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:07.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.854+0000 7f6a05ac9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a001f6460 0x7f6a001f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:07.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.854+0000 7f6a05ac9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a000700d0 0x7f6a00070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:07.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.854+0000 7f6a05ac9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a00070a90 con 0x7f6a001f6460 2023-11-28T18:28:07.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.854+0000 7f6a05ac9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a001f7b00 con 0x7f6a000700d0 2023-11-28T18:28:07.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.854+0000 7f6a05ac9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a00154bf0 con 0x7f69e8005d20 2023-11-28T18:28:07.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.854+0000 7f69feffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a001f6460 0x7f6a001f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:07.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.858+0000 7f69feffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a001f6460 0x7f6a001f68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49132/0 (socket says 172.21.15.105:49132) 2023-11-28T18:28:07.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.858+0000 7f69feffd640 1 -- 172.21.15.105:0/3287890566 learned_addr learned my addr 172.21.15.105:0/3287890566 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:07.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.858+0000 7f69ff7fe640 1 --2- 172.21.15.105:0/3287890566 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69e8005d20 0x7f6a00153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:07.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.858+0000 7f69fffff640 1 --2- 172.21.15.105:0/3287890566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a000700d0 0x7f6a00070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:07.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.858+0000 7f69feffd640 1 -- 172.21.15.105:0/3287890566 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69e8005d20 msgr2=0x7f6a00153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:07.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.858+0000 7f69feffd640 1 --2- 172.21.15.105:0/3287890566 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69e8005d20 0x7f6a00153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:07.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.858+0000 7f69feffd640 1 -- 172.21.15.105:0/3287890566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a000700d0 msgr2=0x7f6a00070550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:07.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.858+0000 7f69feffd640 1 --2- 172.21.15.105:0/3287890566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a000700d0 0x7f6a00070550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:07.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.858+0000 7f69feffd640 1 -- 172.21.15.105:0/3287890566 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a001f6ef0 con 0x7f6a001f6460 2023-11-28T18:28:07.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.858+0000 7f69fffff640 1 --2- 172.21.15.105:0/3287890566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a000700d0 0x7f6a00070550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:07.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.858+0000 7f69feffd640 1 --2- 172.21.15.105:0/3287890566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a001f6460 0x7f6a001f68e0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f69f400e730 tx=0x7f69f400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:07.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.858+0000 7f69e7fff640 1 -- 172.21.15.105:0/3287890566 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f400ee40 con 0x7f6a001f6460 2023-11-28T18:28:07.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.858+0000 7f69e7fff640 1 -- 172.21.15.105:0/3287890566 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f400c8d0 con 0x7f6a001f6460 2023-11-28T18:28:07.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.858+0000 7f69e7fff640 1 -- 172.21.15.105:0/3287890566 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f400cc80 con 0x7f6a001f6460 2023-11-28T18:28:07.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.858+0000 7f6a05ac9640 1 -- 172.21.15.105:0/3287890566 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a001f71d0 con 0x7f6a001f6460 2023-11-28T18:28:07.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.858+0000 7f6a05ac9640 1 -- 172.21.15.105:0/3287890566 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a001f7790 con 0x7f6a001f6460 2023-11-28T18:28:07.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.858+0000 7f6a05ac9640 1 -- 172.21.15.105:0/3287890566 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a00067570 con 0x7f6a001f6460 2023-11-28T18:28:07.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.858+0000 7f69e7fff640 1 -- 172.21.15.105:0/3287890566 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f69f401d460 con 0x7f6a001f6460 2023-11-28T18:28:07.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.858+0000 7f69e7fff640 1 --2- 172.21.15.105:0/3287890566 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f69ec0668d0 0x7f69ec068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:07.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.858+0000 7f69e7fff640 1 -- 172.21.15.105:0/3287890566 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f69f4084220 con 0x7f6a001f6460 2023-11-28T18:28:07.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.862+0000 7f69ff7fe640 1 --2- 172.21.15.105:0/3287890566 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f69ec0668d0 0x7f69ec068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:07.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.862+0000 7f69ff7fe640 1 --2- 172.21.15.105:0/3287890566 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f69ec0668d0 0x7f69ec068d90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f69f00ad000 tx=0x7f69f00b2040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:07.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:07.862+0000 7f69e7fff640 1 -- 172.21.15.105:0/3287890566 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f69f4051570 con 0x7f6a001f6460 2023-11-28T18:28:08.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.014+0000 7f6a05ac9640 1 -- 172.21.15.105:0/3287890566 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6a000cf950 con 0x7f6a001f6460 2023-11-28T18:28:08.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.018+0000 7f69e7fff640 1 -- 172.21.15.105:0/3287890566 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v35)=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7f69f4054b90 con 0x7f6a001f6460 2023-11-28T18:28:08.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.022+0000 7f6a05ac9640 1 -- 172.21.15.105:0/3287890566 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f69ec0668d0 msgr2=0x7f69ec068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:08.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.022+0000 7f6a05ac9640 1 --2- 172.21.15.105:0/3287890566 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f69ec0668d0 0x7f69ec068d90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f69f00ad000 tx=0x7f69f00b2040 comp rx=0 tx=0).stop 2023-11-28T18:28:08.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.022+0000 7f6a05ac9640 1 -- 172.21.15.105:0/3287890566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a001f6460 msgr2=0x7f6a001f68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:08.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.022+0000 7f6a05ac9640 1 --2- 172.21.15.105:0/3287890566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a001f6460 0x7f6a001f68e0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f69f400e730 tx=0x7f69f400ebf0 comp rx=0 tx=0).stop 2023-11-28T18:28:08.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.022+0000 7f6a05ac9640 1 -- 172.21.15.105:0/3287890566 shutdown_connections 2023-11-28T18:28:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.022+0000 7f6a05ac9640 1 --2- 172.21.15.105:0/3287890566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a000700d0 0x7f6a00070550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.022+0000 7f6a05ac9640 1 --2- 172.21.15.105:0/3287890566 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f69ec0668d0 0x7f69ec068d90 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.022+0000 7f6a05ac9640 1 --2- 172.21.15.105:0/3287890566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a001f6460 0x7f6a001f68e0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.022+0000 7f6a05ac9640 1 --2- 172.21.15.105:0/3287890566 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69e8005d20 0x7f6a00153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.022+0000 7f6a05ac9640 1 -- 172.21.15.105:0/3287890566 >> 172.21.15.105:0/3287890566 conn(0x7f6a000ceb90 msgr2=0x7f6a000cdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.022+0000 7f6a05ac9640 1 -- 172.21.15.105:0/3287890566 shutdown_connections 2023-11-28T18:28:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.022+0000 7f6a05ac9640 1 -- 172.21.15.105:0/3287890566 wait complete. 2023-11-28T18:28:08.046 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T18:28:08.046 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:28:08.131 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:28:08.224 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T18:28:08.224 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:28:08.225 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:28:08.234 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:28:08.234 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:28:08.234 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:28:08.244 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:28:08.244 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:28:08.244 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:28:08.285 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:28:08.285 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:28:08.285 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:28:08.294 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:28:08.295 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T18:28:08.295 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:28:08.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2d1617640 1 Processor -- start 2023-11-28T18:28:08.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2d1617640 1 -- start start 2023-11-28T18:28:08.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2d1617640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff2cc0d3c30 0x7ff2cc0d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:08.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2d1617640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2cc0b25b0 con 0x7ff2cc15d4f0 2023-11-28T18:28:08.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2d1617640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2cc0cea30 con 0x7ff2cc0b2d20 2023-11-28T18:28:08.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2d1617640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2cc0b2900 con 0x7ff2cc0d3c30 2023-11-28T18:28:08.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2caffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff2cc15d4f0 0x7ff2cc0d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42038/0 (socket says 172.21.15.105:42038) 2023-11-28T18:28:08.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2caffd640 1 -- 172.21.15.105:0/2512460037 learned_addr learned my addr 172.21.15.105:0/2512460037 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:08.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2c27fc640 1 --2- 172.21.15.105:0/2512460037 >> v2:172.21.15.148:3300/0 conn(0x7ff2cc0d3c30 0x7ff2cc0d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:08.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2ca7fc640 1 -- 172.21.15.105:0/2512460037 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2149035106 0 0) 0x7ff2cc0b25b0 con 0x7ff2cc15d4f0 2023-11-28T18:28:08.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2ca7fc640 1 -- 172.21.15.105:0/2512460037 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2b8003530 con 0x7ff2cc15d4f0 2023-11-28T18:28:08.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2ca7fc640 1 -- 172.21.15.105:0/2512460037 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1678339993 0 0) 0x7ff2b8003530 con 0x7ff2cc15d4f0 2023-11-28T18:28:08.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2ca7fc640 1 -- 172.21.15.105:0/2512460037 >> v2:172.21.15.148:3300/0 conn(0x7ff2cc0d3c30 msgr2=0x7ff2cc0d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:08.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2ca7fc640 1 --2- 172.21.15.105:0/2512460037 >> v2:172.21.15.148:3300/0 conn(0x7ff2cc0d3c30 0x7ff2cc0d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:08.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2ca7fc640 1 -- 172.21.15.105:0/2512460037 >> v1:172.21.15.148:6789/0 conn(0x7ff2cc0b2d20 legacy=0x7ff2cc0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:08.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2ca7fc640 1 -- 172.21.15.105:0/2512460037 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2cc0ced10 con 0x7ff2cc15d4f0 2023-11-28T18:28:08.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2ca7fc640 1 -- 172.21.15.105:0/2512460037 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7ff2bc001680 con 0x7ff2cc15d4f0 2023-11-28T18:28:08.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2ca7fc640 1 -- 172.21.15.105:0/2512460037 >> v1:172.21.15.105:6789/0 conn(0x7ff2cc15d4f0 legacy=0x7ff2cc0d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:08.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2ca7fc640 1 --2- 172.21.15.105:0/2512460037 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2b8003980 0x7ff2b8003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:08.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2ca7fc640 1 --2- 172.21.15.105:0/2512460037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2b8004380 0x7ff2b8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:08.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2ca7fc640 1 --2- 172.21.15.105:0/2512460037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2b80050f0 0x7ff2b800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:08.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2ca7fc640 1 -- 172.21.15.105:0/2512460037 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc0b2900 con 0x7ff2b80050f0 2023-11-28T18:28:08.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2ca7fc640 1 -- 172.21.15.105:0/2512460037 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b800c030 con 0x7ff2b8004380 2023-11-28T18:28:08.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2ca7fc640 1 -- 172.21.15.105:0/2512460037 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b800c1f0 con 0x7ff2b8003980 2023-11-28T18:28:08.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2cb7fe640 1 --2- 172.21.15.105:0/2512460037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2b80050f0 0x7ff2b800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:08.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2c27fc640 1 --2- 172.21.15.105:0/2512460037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2b8004380 0x7ff2b8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:08.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2caffd640 1 --2- 172.21.15.105:0/2512460037 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2b8003980 0x7ff2b8003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:08.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2cb7fe640 1 -- 172.21.15.105:0/2512460037 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2b8003980 msgr2=0x7ff2b8003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:08.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2cb7fe640 1 --2- 172.21.15.105:0/2512460037 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2b8003980 0x7ff2b8003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:08.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2cb7fe640 1 -- 172.21.15.105:0/2512460037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2b8004380 msgr2=0x7ff2b8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:08.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2cb7fe640 1 --2- 172.21.15.105:0/2512460037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2b8004380 0x7ff2b8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:08.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2cb7fe640 1 -- 172.21.15.105:0/2512460037 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b8003530 con 0x7ff2b80050f0 2023-11-28T18:28:08.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.462+0000 7ff2caffd640 1 --2- 172.21.15.105:0/2512460037 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2b8003980 0x7ff2b8003d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:08.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.466+0000 7ff2cb7fe640 1 -- 172.21.15.105:0/2512460037 reap_dead start 2023-11-28T18:28:08.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.466+0000 7ff2cb7fe640 1 --2- 172.21.15.105:0/2512460037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2b80050f0 0x7ff2b800b990 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4001780 tx=0x7ff2c401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b14b41fec4ec2b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:08.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.466+0000 7ff2ca7fc640 1 -- 172.21.15.105:0/2512460037 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40052d0 con 0x7ff2b80050f0 2023-11-28T18:28:08.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.466+0000 7ff2ca7fc640 1 -- 172.21.15.105:0/2512460037 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c40a7050 con 0x7ff2b80050f0 2023-11-28T18:28:08.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.466+0000 7ff2ca7fc640 1 -- 172.21.15.105:0/2512460037 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4003840 con 0x7ff2b80050f0 2023-11-28T18:28:08.479 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.466+0000 7ff2d1617640 1 -- 172.21.15.105:0/2512460037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2b80050f0 msgr2=0x7ff2b800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:08.479 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.466+0000 7ff2d1617640 1 --2- 172.21.15.105:0/2512460037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2b80050f0 0x7ff2b800b990 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4001780 tx=0x7ff2c401c400 comp rx=0 tx=0).stop 2023-11-28T18:28:08.479 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.466+0000 7ff2d1617640 1 -- 172.21.15.105:0/2512460037 shutdown_connections 2023-11-28T18:28:08.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.466+0000 7ff2d1617640 1 --2- 172.21.15.105:0/2512460037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2b80050f0 0x7ff2b800b990 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:08.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.466+0000 7ff2d1617640 1 -- 172.21.15.105:0/2512460037 >> 172.21.15.105:0/2512460037 conn(0x7ff2cc0d3030 msgr2=0x7ff2cc0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:08.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.466+0000 7ff2d1617640 1 -- 172.21.15.105:0/2512460037 shutdown_connections 2023-11-28T18:28:08.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.466+0000 7ff2d1617640 1 -- 172.21.15.105:0/2512460037 wait complete. 2023-11-28T18:28:08.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2d1617640 1 Processor -- start 2023-11-28T18:28:08.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2d1617640 1 -- start start 2023-11-28T18:28:08.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2d1617640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2b80050f0 0x7ff2cc0b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:08.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2d1617640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2cc0b4f70 0x7ff2cc159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:08.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2d1617640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2cc1e4860 0x7ff2cc1e4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:08.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2d1617640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc0d0a20 con 0x7ff2cc0b4f70 2023-11-28T18:28:08.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2d1617640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc1e5250 con 0x7ff2cc1e4860 2023-11-28T18:28:08.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2d1617640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc1f6ce0 con 0x7ff2b80050f0 2023-11-28T18:28:08.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2c27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2cc0b4f70 0x7ff2cc159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:08.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2c27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2cc0b4f70 0x7ff2cc159360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49150/0 (socket says 172.21.15.105:49150) 2023-11-28T18:28:08.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2c27fc640 1 -- 172.21.15.105:0/1582929140 learned_addr learned my addr 172.21.15.105:0/1582929140 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:08.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2c27fc640 1 -- 172.21.15.105:0/1582929140 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2b80050f0 msgr2=0x7ff2cc0b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:08.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2c27fc640 1 --2- 172.21.15.105:0/1582929140 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2b80050f0 0x7ff2cc0b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:08.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2cb7fe640 1 --2- 172.21.15.105:0/1582929140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2cc1e4860 0x7ff2cc1e4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:08.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2c27fc640 1 -- 172.21.15.105:0/1582929140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2cc1e4860 msgr2=0x7ff2cc1e4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:08.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2c27fc640 1 --2- 172.21.15.105:0/1582929140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2cc1e4860 0x7ff2cc1e4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:08.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2c27fc640 1 -- 172.21.15.105:0/1582929140 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2cc0b53f0 con 0x7ff2cc0b4f70 2023-11-28T18:28:08.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2c27fc640 1 --2- 172.21.15.105:0/1582929140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2cc0b4f70 0x7ff2cc159360 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009fc0 tx=0x7ff2b400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:08.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2c3fff640 1 -- 172.21.15.105:0/1582929140 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b400e040 con 0x7ff2cc0b4f70 2023-11-28T18:28:08.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2c3fff640 1 -- 172.21.15.105:0/1582929140 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b400f040 con 0x7ff2cc0b4f70 2023-11-28T18:28:08.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2c3fff640 1 -- 172.21.15.105:0/1582929140 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4015870 con 0x7ff2cc0b4f70 2023-11-28T18:28:08.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2d1617640 1 -- 172.21.15.105:0/1582929140 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2cc1f5f20 con 0x7ff2cc0b4f70 2023-11-28T18:28:08.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.470+0000 7ff2d1617640 1 -- 172.21.15.105:0/1582929140 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2cc1f64e0 con 0x7ff2cc0b4f70 2023-11-28T18:28:08.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.474+0000 7ff2d1617640 1 -- 172.21.15.105:0/1582929140 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2cc067570 con 0x7ff2cc0b4f70 2023-11-28T18:28:08.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.474+0000 7ff2c3fff640 1 -- 172.21.15.105:0/1582929140 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff2b40159d0 con 0x7ff2cc0b4f70 2023-11-28T18:28:08.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.474+0000 7ff2c3fff640 1 --2- 172.21.15.105:0/1582929140 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff2ac0668d0 0x7ff2ac068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:08.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.474+0000 7ff2c3fff640 1 -- 172.21.15.105:0/1582929140 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5435+0+0 (secure 0 0 0) 0x7ff2b4085080 con 0x7ff2cc0b4f70 2023-11-28T18:28:08.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.474+0000 7ff2caffd640 1 --2- 172.21.15.105:0/1582929140 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff2ac0668d0 0x7ff2ac068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:08.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.474+0000 7ff2caffd640 1 --2- 172.21.15.105:0/1582929140 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff2ac0668d0 0x7ff2ac068d90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc001210 tx=0x7ff2bc000ee0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:08.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.478+0000 7ff2c3fff640 1 -- 172.21.15.105:0/1582929140 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff2b40523d0 con 0x7ff2cc0b4f70 2023-11-28T18:28:08.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.662+0000 7ff2d1617640 1 -- 172.21.15.105:0/1582929140 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2cc06cd20 con 0x7ff2cc0b4f70 2023-11-28T18:28:08.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.662+0000 7ff2c3fff640 1 -- 172.21.15.105:0/1582929140 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+3696 (secure 0 0 0) 0x7ff2b4022e80 con 0x7ff2cc0b4f70 2023-11-28T18:28:08.666 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:08.666 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":64,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/234000461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":234000461},{"type":"v1","addr":"172.21.15.148:6834","nonce":234000461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/872137705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":872137705},{"type":"v1","addr":"172.21.15.148:6835","nonce":872137705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64},{"gid":7074,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/675328287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":675328287},{"type":"v1","addr":"172.21.15.105:6835","nonce":675328287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64},{"gid":7080,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/31391009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":31391009},{"type":"v1","addr":"172.21.15.105:6837","nonce":31391009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64}],"filesystems":[]} 2023-11-28T18:28:08.666 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 64 2023-11-28T18:28:08.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.662+0000 7ff2d1617640 1 -- 172.21.15.105:0/1582929140 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff2ac0668d0 msgr2=0x7ff2ac068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:08.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.662+0000 7ff2d1617640 1 --2- 172.21.15.105:0/1582929140 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff2ac0668d0 0x7ff2ac068d90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc001210 tx=0x7ff2bc000ee0 comp rx=0 tx=0).stop 2023-11-28T18:28:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.662+0000 7ff2d1617640 1 -- 172.21.15.105:0/1582929140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2cc0b4f70 msgr2=0x7ff2cc159360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.662+0000 7ff2d1617640 1 --2- 172.21.15.105:0/1582929140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2cc0b4f70 0x7ff2cc159360 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009fc0 tx=0x7ff2b400f580 comp rx=0 tx=0).stop 2023-11-28T18:28:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.662+0000 7ff2d1617640 1 -- 172.21.15.105:0/1582929140 shutdown_connections 2023-11-28T18:28:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.662+0000 7ff2d1617640 1 --2- 172.21.15.105:0/1582929140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2cc1e4860 0x7ff2cc1e4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.662+0000 7ff2d1617640 1 --2- 172.21.15.105:0/1582929140 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff2ac0668d0 0x7ff2ac068d90 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.662+0000 7ff2d1617640 1 --2- 172.21.15.105:0/1582929140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2cc0b4f70 0x7ff2cc159360 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.662+0000 7ff2d1617640 1 --2- 172.21.15.105:0/1582929140 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2b80050f0 0x7ff2cc0b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.662+0000 7ff2d1617640 1 -- 172.21.15.105:0/1582929140 >> 172.21.15.105:0/1582929140 conn(0x7ff2cc0d3030 msgr2=0x7ff2cc1e84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.662+0000 7ff2d1617640 1 -- 172.21.15.105:0/1582929140 shutdown_connections 2023-11-28T18:28:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.662+0000 7ff2d1617640 1 -- 172.21.15.105:0/1582929140 wait complete. 2023-11-28T18:28:08.687 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T18:28:08.687 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T18:28:08.687 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T18:28:08.687 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:28:08.687 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T18:28:08.688 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T18:28:08.688 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T18:28:08.688 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:28:08.688 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T18:28:08.688 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T18:28:08.688 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T18:28:08.688 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:28:08.689 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T18:28:08.689 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T18:28:08.689 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T18:28:08.689 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:28:08.701 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T18:28:08.701 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T18:28:08.702 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T18:28:08.702 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T18:28:08.704 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T18:28:08.704 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T18:28:08.705 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T18:28:08.705 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T18:28:08.752 INFO:tasks.ceph.mds.a:Started 2023-11-28T18:28:08.752 DEBUG:teuthology.parallel:result is None 2023-11-28T18:28:08.753 INFO:tasks.ceph.mds.b:Started 2023-11-28T18:28:08.753 DEBUG:teuthology.parallel:result is None 2023-11-28T18:28:08.754 INFO:tasks.ceph.mds.c:Started 2023-11-28T18:28:08.755 DEBUG:teuthology.parallel:result is None 2023-11-28T18:28:08.755 INFO:tasks.ceph.mds.d:Started 2023-11-28T18:28:08.755 DEBUG:teuthology.parallel:result is None 2023-11-28T18:28:08.755 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T18:28:08.841 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2023-11-28T18:28:08.842 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2023-11-28T18:28:08.847 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2023-11-28T18:28:08.848 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2023-11-28T18:28:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.898+0000 7fd9b9157640 1 Processor -- start 2023-11-28T18:28:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.898+0000 7fd9b9157640 1 -- start start 2023-11-28T18:28:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.898+0000 7fd9b9157640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9b415d4f0 0x7fd9b40d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.898+0000 7fd9b9157640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd9b40d3c30 0x7fd9b40d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.898+0000 7fd9b9157640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b40b25b0 con 0x7fd9b40d1890 2023-11-28T18:28:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.898+0000 7fd9b9157640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b40b2900 con 0x7fd9b40d3c30 2023-11-28T18:28:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.898+0000 7fd9b9157640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40cea30 con 0x7fd9b415d4f0 2023-11-28T18:28:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.898+0000 7fd9b359e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd9b40d1890 0x7fd9b40cfa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42064/0 (socket says 172.21.15.105:42064) 2023-11-28T18:28:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.898+0000 7fd9b359e640 1 -- 172.21.15.105:0/1380862061 learned_addr learned my addr 172.21.15.105:0/1380862061 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.898+0000 7fd9b2d9d640 1 --2- 172.21.15.105:0/1380862061 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9b415d4f0 0x7fd9b40d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.898+0000 7fd9aa59c640 1 --2- 172.21.15.105:0/1380862061 >> v2:172.21.15.148:3300/0 conn(0x7fd9b40d3c30 0x7fd9b40d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.898+0000 7fd9b2d9d640 1 -- 172.21.15.105:0/1380862061 >> v2:172.21.15.148:3300/0 conn(0x7fd9b40d3c30 msgr2=0x7fd9b40d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.898+0000 7fd9b2d9d640 1 --2- 172.21.15.105:0/1380862061 >> v2:172.21.15.148:3300/0 conn(0x7fd9b40d3c30 0x7fd9b40d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.898+0000 7fd9b2d9d640 1 -- 172.21.15.105:0/1380862061 >> v1:172.21.15.105:6789/0 conn(0x7fd9b40d1890 legacy=0x7fd9b40cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.898+0000 7fd9b2d9d640 1 -- 172.21.15.105:0/1380862061 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b40ced10 con 0x7fd9b415d4f0 2023-11-28T18:28:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.898+0000 7fd9b259c640 1 -- 172.21.15.105:0/1380862061 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978032668 0 0) 0x7fd9b40b25b0 con 0x7fd9b40d1890 2023-11-28T18:28:08.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.902+0000 7fd9b2d9d640 1 --2- 172.21.15.105:0/1380862061 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9b415d4f0 0x7fd9b40d0e10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40097c0 tx=0x7fd9a401c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9645260ed2a3e65 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.902+0000 7fd9b259c640 1 -- 172.21.15.105:0/1380862061 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a40ac640 con 0x7fd9b415d4f0 2023-11-28T18:28:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.902+0000 7fd9b259c640 1 -- 172.21.15.105:0/1380862061 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a40ac7a0 con 0x7fd9b415d4f0 2023-11-28T18:28:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.902+0000 7fd9b259c640 1 -- 172.21.15.105:0/1380862061 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a40acb30 con 0x7fd9b415d4f0 2023-11-28T18:28:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.902+0000 7fd9b9157640 1 -- 172.21.15.105:0/1380862061 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9b415d4f0 msgr2=0x7fd9b40d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.902+0000 7fd9b9157640 1 --2- 172.21.15.105:0/1380862061 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9b415d4f0 0x7fd9b40d0e10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40097c0 tx=0x7fd9a401c3c0 comp rx=0 tx=0).stop 2023-11-28T18:28:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.902+0000 7fd9b9157640 1 -- 172.21.15.105:0/1380862061 shutdown_connections 2023-11-28T18:28:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.902+0000 7fd9b9157640 1 --2- 172.21.15.105:0/1380862061 >> v2:172.21.15.148:3300/0 conn(0x7fd9b40d3c30 0x7fd9b40d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.902+0000 7fd9b9157640 1 --2- 172.21.15.105:0/1380862061 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9b415d4f0 0x7fd9b40d0e10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.902+0000 7fd9b9157640 1 -- 172.21.15.105:0/1380862061 >> 172.21.15.105:0/1380862061 conn(0x7fd9b40d3030 msgr2=0x7fd9b40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.902+0000 7fd9b9157640 1 -- 172.21.15.105:0/1380862061 shutdown_connections 2023-11-28T18:28:08.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.902+0000 7fd9b9157640 1 -- 172.21.15.105:0/1380862061 wait complete. 2023-11-28T18:28:08.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.906+0000 7fd9b9157640 1 Processor -- start 2023-11-28T18:28:08.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.906+0000 7fd9b9157640 1 -- start start 2023-11-28T18:28:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.906+0000 7fd9b9157640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9b40d1890 0x7fd9b40cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.906+0000 7fd9b9157640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9b40d3c30 0x7fd9b41f9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.906+0000 7fd9b9157640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9b415d4f0 0x7fd9b41f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.906+0000 7fd9b9157640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40b88e0 con 0x7fd9b40d3c30 2023-11-28T18:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.906+0000 7fd9b9157640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40b7fb0 con 0x7fd9b40d1890 2023-11-28T18:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.906+0000 7fd9b9157640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40b8290 con 0x7fd9b415d4f0 2023-11-28T18:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.906+0000 7fd9aa59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9b40d3c30 0x7fd9b41f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.906+0000 7fd9aa59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9b40d3c30 0x7fd9b41f9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49202/0 (socket says 172.21.15.105:49202) 2023-11-28T18:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.906+0000 7fd9aa59c640 1 -- 172.21.15.105:0/1906842186 learned_addr learned my addr 172.21.15.105:0/1906842186 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.906+0000 7fd9b2d9d640 1 --2- 172.21.15.105:0/1906842186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9b40d1890 0x7fd9b40cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.906+0000 7fd9b359e640 1 --2- 172.21.15.105:0/1906842186 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9b415d4f0 0x7fd9b41f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.906+0000 7fd9aa59c640 1 -- 172.21.15.105:0/1906842186 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9b415d4f0 msgr2=0x7fd9b41f96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.906+0000 7fd9aa59c640 1 --2- 172.21.15.105:0/1906842186 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9b415d4f0 0x7fd9b41f96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.906+0000 7fd9aa59c640 1 -- 172.21.15.105:0/1906842186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9b40d1890 msgr2=0x7fd9b40cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.906+0000 7fd9aa59c640 1 --2- 172.21.15.105:0/1906842186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9b40d1890 0x7fd9b40cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.906+0000 7fd9aa59c640 1 -- 172.21.15.105:0/1906842186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a4009420 con 0x7fd9b40d3c30 2023-11-28T18:28:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.906+0000 7fd9aa59c640 1 --2- 172.21.15.105:0/1906842186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9b40d3c30 0x7fd9b41f9180 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00efc0 tx=0x7fd99c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.910+0000 7fd9abfff640 1 -- 172.21.15.105:0/1906842186 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c009b80 con 0x7fd9b40d3c30 2023-11-28T18:28:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.910+0000 7fd9abfff640 1 -- 172.21.15.105:0/1906842186 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99c00f040 con 0x7fd9b40d3c30 2023-11-28T18:28:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.910+0000 7fd9abfff640 1 -- 172.21.15.105:0/1906842186 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0146a0 con 0x7fd9b40d3c30 2023-11-28T18:28:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.910+0000 7fd9b9157640 1 -- 172.21.15.105:0/1906842186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b40b8570 con 0x7fd9b40d3c30 2023-11-28T18:28:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.910+0000 7fd9b9157640 1 -- 172.21.15.105:0/1906842186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b4146550 con 0x7fd9b40d3c30 2023-11-28T18:28:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.910+0000 7fd9b9157640 1 -- 172.21.15.105:0/1906842186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b4067570 con 0x7fd9b40d3c30 2023-11-28T18:28:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.910+0000 7fd9abfff640 1 -- 172.21.15.105:0/1906842186 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd99c014800 con 0x7fd9b40d3c30 2023-11-28T18:28:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.910+0000 7fd9abfff640 1 --2- 172.21.15.105:0/1906842186 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd994066900 0x7fd994068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.910+0000 7fd9abfff640 1 -- 172.21.15.105:0/1906842186 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fd99c085140 con 0x7fd9b40d3c30 2023-11-28T18:28:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.910+0000 7fd9b2d9d640 1 --2- 172.21.15.105:0/1906842186 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd994066900 0x7fd994068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.910+0000 7fd9b2d9d640 1 --2- 172.21.15.105:0/1906842186 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd994066900 0x7fd994068dc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40ae000 tx=0x7fd9a40a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:08.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:08.914+0000 7fd9abfff640 1 -- 172.21.15.105:0/1906842186 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd99c0523f0 con 0x7fd9b40d3c30 2023-11-28T18:28:09.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.078+0000 7fd9b9157640 1 -- 172.21.15.105:0/1906842186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd9b40b2500 con 0x7fd9b40d3c30 2023-11-28T18:28:09.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.694+0000 7fd9abfff640 1 -- 172.21.15.105:0/1906842186 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v72) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd99c055a10 con 0x7fd9b40d3c30 2023-11-28T18:28:09.698 INFO:teuthology.orchestra.run.smithi105.stderr: removed all blocklist entries 2023-11-28T18:28:09.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.694+0000 7fd9b9157640 1 -- 172.21.15.105:0/1906842186 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd994066900 msgr2=0x7fd994068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:09.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.694+0000 7fd9b9157640 1 --2- 172.21.15.105:0/1906842186 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd994066900 0x7fd994068dc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40ae000 tx=0x7fd9a40a7000 comp rx=0 tx=0).stop 2023-11-28T18:28:09.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.694+0000 7fd9b9157640 1 -- 172.21.15.105:0/1906842186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9b40d3c30 msgr2=0x7fd9b41f9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:09.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.694+0000 7fd9b9157640 1 --2- 172.21.15.105:0/1906842186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9b40d3c30 0x7fd9b41f9180 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00efc0 tx=0x7fd99c00c490 comp rx=0 tx=0).stop 2023-11-28T18:28:09.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.694+0000 7fd9b9157640 1 -- 172.21.15.105:0/1906842186 shutdown_connections 2023-11-28T18:28:09.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.694+0000 7fd9b9157640 1 --2- 172.21.15.105:0/1906842186 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9b415d4f0 0x7fd9b41f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:09.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.694+0000 7fd9b9157640 1 --2- 172.21.15.105:0/1906842186 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd994066900 0x7fd994068dc0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:09.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.694+0000 7fd9b9157640 1 --2- 172.21.15.105:0/1906842186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9b40d3c30 0x7fd9b41f9180 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:09.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.694+0000 7fd9b9157640 1 --2- 172.21.15.105:0/1906842186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9b40d1890 0x7fd9b40cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:09.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.694+0000 7fd9b9157640 1 -- 172.21.15.105:0/1906842186 >> 172.21.15.105:0/1906842186 conn(0x7fd9b40d3030 msgr2=0x7fd9b406bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:09.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.694+0000 7fd9b9157640 1 -- 172.21.15.105:0/1906842186 shutdown_connections 2023-11-28T18:28:09.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.694+0000 7fd9b9157640 1 -- 172.21.15.105:0/1906842186 wait complete. 2023-11-28T18:28:09.726 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T18:28:09.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a440fe640 1 Processor -- start 2023-11-28T18:28:09.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a440fe640 1 -- start start 2023-11-28T18:28:09.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a440fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2a3c159f40 0x7f2a3c0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:09.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a440fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a3c06bf50 con 0x7f2a3c0ad850 2023-11-28T18:28:09.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a440fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a3c0ccbc0 con 0x7f2a3c0cc760 2023-11-28T18:28:09.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a440fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3c15a320 con 0x7f2a3c159f40 2023-11-28T18:28:09.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a4269b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f2a3c0ad850 0x7f2a3c0701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42080/0 (socket says 172.21.15.105:42080) 2023-11-28T18:28:09.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a4269b640 1 -- 172.21.15.105:0/1474032683 learned_addr learned my addr 172.21.15.105:0/1474032683 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:09.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a41e9a640 1 --2- 172.21.15.105:0/1474032683 >> v2:172.21.15.148:3300/0 conn(0x7f2a3c159f40 0x7f2a3c0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:09.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a40e98640 1 -- 172.21.15.105:0/1474032683 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1959249053 0 0) 0x7f2a3c06bf50 con 0x7f2a3c0ad850 2023-11-28T18:28:09.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a40e98640 1 -- 172.21.15.105:0/1474032683 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a28003530 con 0x7f2a3c0ad850 2023-11-28T18:28:09.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a40e98640 1 -- 172.21.15.105:0/1474032683 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3693509650 0 0) 0x7f2a3c0ccbc0 con 0x7f2a3c0cc760 2023-11-28T18:28:09.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a40e98640 1 -- 172.21.15.105:0/1474032683 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a3c06bf50 con 0x7f2a3c0cc760 2023-11-28T18:28:09.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a40e98640 1 -- 172.21.15.105:0/1474032683 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 953670522 0 0) 0x7f2a28003530 con 0x7f2a3c0ad850 2023-11-28T18:28:09.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a40e98640 1 -- 172.21.15.105:0/1474032683 >> v2:172.21.15.148:3300/0 conn(0x7f2a3c159f40 msgr2=0x7f2a3c0caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:09.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a40e98640 1 --2- 172.21.15.105:0/1474032683 >> v2:172.21.15.148:3300/0 conn(0x7f2a3c159f40 0x7f2a3c0caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:09.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a40e98640 1 -- 172.21.15.105:0/1474032683 >> v1:172.21.15.148:6789/0 conn(0x7f2a3c0cc760 legacy=0x7f2a3c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:09.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a40e98640 1 -- 172.21.15.105:0/1474032683 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c071380 con 0x7f2a3c0ad850 2023-11-28T18:28:09.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a40e98640 1 -- 172.21.15.105:0/1474032683 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f2a34001980 con 0x7f2a3c0ad850 2023-11-28T18:28:09.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a40e98640 1 -- 172.21.15.105:0/1474032683 >> v1:172.21.15.105:6789/0 conn(0x7f2a3c0ad850 legacy=0x7f2a3c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:09.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a40e98640 1 --2- 172.21.15.105:0/1474032683 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2a28003b60 0x7f2a28003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:09.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a40e98640 1 --2- 172.21.15.105:0/1474032683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a280047d0 0x7f2a28004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:09.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a40e98640 1 --2- 172.21.15.105:0/1474032683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a280051e0 0x7f2a2800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:09.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a40e98640 1 -- 172.21.15.105:0/1474032683 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c15a320 con 0x7f2a280051e0 2023-11-28T18:28:09.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a40e98640 1 -- 172.21.15.105:0/1474032683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2800c110 con 0x7f2a280047d0 2023-11-28T18:28:09.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.862+0000 7f2a40e98640 1 -- 172.21.15.105:0/1474032683 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2800c2d0 con 0x7f2a28003b60 2023-11-28T18:28:09.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.866+0000 7f2a4269b640 1 --2- 172.21.15.105:0/1474032683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a280051e0 0x7f2a2800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:09.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.866+0000 7f2a41699640 1 --2- 172.21.15.105:0/1474032683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a280047d0 0x7f2a28004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:09.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.866+0000 7f2a41e9a640 1 --2- 172.21.15.105:0/1474032683 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2a28003b60 0x7f2a28003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:09.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.866+0000 7f2a4269b640 1 -- 172.21.15.105:0/1474032683 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2a28003b60 msgr2=0x7f2a28003f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:09.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.866+0000 7f2a4269b640 1 --2- 172.21.15.105:0/1474032683 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2a28003b60 0x7f2a28003f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:09.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.866+0000 7f2a4269b640 1 -- 172.21.15.105:0/1474032683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a280047d0 msgr2=0x7f2a28004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:09.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.866+0000 7f2a4269b640 1 --2- 172.21.15.105:0/1474032683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a280047d0 0x7f2a28004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:09.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.866+0000 7f2a4269b640 1 -- 172.21.15.105:0/1474032683 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a28003530 con 0x7f2a280051e0 2023-11-28T18:28:09.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.866+0000 7f2a4269b640 1 -- 172.21.15.105:0/1474032683 reap_dead start 2023-11-28T18:28:09.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.866+0000 7f2a4269b640 1 --2- 172.21.15.105:0/1474032683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a280051e0 0x7f2a2800b990 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400a260 tx=0x7f2a3401f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f078e3c3620654 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:09.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.866+0000 7f2a40e98640 1 -- 172.21.15.105:0/1474032683 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a340072d0 con 0x7f2a280051e0 2023-11-28T18:28:09.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.866+0000 7f2a40e98640 1 -- 172.21.15.105:0/1474032683 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a340aa070 con 0x7f2a280051e0 2023-11-28T18:28:09.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.866+0000 7f2a40e98640 1 -- 172.21.15.105:0/1474032683 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a340076b0 con 0x7f2a280051e0 2023-11-28T18:28:09.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.866+0000 7f2a440fe640 1 -- 172.21.15.105:0/1474032683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a280051e0 msgr2=0x7f2a2800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:09.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.866+0000 7f2a440fe640 1 --2- 172.21.15.105:0/1474032683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a280051e0 0x7f2a2800b990 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400a260 tx=0x7f2a3401f390 comp rx=0 tx=0).stop 2023-11-28T18:28:09.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a440fe640 1 -- 172.21.15.105:0/1474032683 shutdown_connections 2023-11-28T18:28:09.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a440fe640 1 --2- 172.21.15.105:0/1474032683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a280051e0 0x7f2a2800b990 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:09.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a440fe640 1 -- 172.21.15.105:0/1474032683 >> 172.21.15.105:0/1474032683 conn(0x7f2a3c0ceb90 msgr2=0x7f2a3c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:09.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a440fe640 1 -- 172.21.15.105:0/1474032683 shutdown_connections 2023-11-28T18:28:09.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a440fe640 1 -- 172.21.15.105:0/1474032683 wait complete. 2023-11-28T18:28:09.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a440fe640 1 Processor -- start 2023-11-28T18:28:09.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a440fe640 1 -- start start 2023-11-28T18:28:09.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a440fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a280051e0 0x7f2a3c148610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:09.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a440fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2a3c0b19b0 0x7f2a3c09c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:09.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a440fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a3c15be10 0x7f2a3c15c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:09.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a440fe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c15c7d0 con 0x7f2a280051e0 2023-11-28T18:28:09.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a440fe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c09afe0 con 0x7f2a3c15be10 2023-11-28T18:28:09.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a440fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c09b2c0 con 0x7f2a3c0b19b0 2023-11-28T18:28:09.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a41e9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a280051e0 0x7f2a3c148610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:09.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a41e9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a280051e0 0x7f2a3c148610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49224/0 (socket says 172.21.15.105:49224) 2023-11-28T18:28:09.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a41e9a640 1 -- 172.21.15.105:0/1687384780 learned_addr learned my addr 172.21.15.105:0/1687384780 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:09.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a41699640 1 --2- 172.21.15.105:0/1687384780 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2a3c0b19b0 0x7f2a3c09c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:09.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a4269b640 1 --2- 172.21.15.105:0/1687384780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a3c15be10 0x7f2a3c15c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:09.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a41e9a640 1 -- 172.21.15.105:0/1687384780 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2a3c0b19b0 msgr2=0x7f2a3c09c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:09.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a41e9a640 1 --2- 172.21.15.105:0/1687384780 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2a3c0b19b0 0x7f2a3c09c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:09.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a41e9a640 1 -- 172.21.15.105:0/1687384780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a3c15be10 msgr2=0x7f2a3c15c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:09.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a41e9a640 1 --2- 172.21.15.105:0/1687384780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a3c15be10 0x7f2a3c15c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:09.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a41e9a640 1 -- 172.21.15.105:0/1687384780 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c09b5a0 con 0x7f2a280051e0 2023-11-28T18:28:09.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.870+0000 7f2a41e9a640 1 --2- 172.21.15.105:0/1687384780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a280051e0 0x7f2a3c148610 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f2a38011960 tx=0x7f2a38011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:09.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.874+0000 7f2a327fc640 1 -- 172.21.15.105:0/1687384780 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3800faf0 con 0x7f2a280051e0 2023-11-28T18:28:09.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.874+0000 7f2a327fc640 1 -- 172.21.15.105:0/1687384780 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3800fc50 con 0x7f2a280051e0 2023-11-28T18:28:09.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.874+0000 7f2a327fc640 1 -- 172.21.15.105:0/1687384780 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a38014440 con 0x7f2a280051e0 2023-11-28T18:28:09.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.874+0000 7f2a440fe640 1 -- 172.21.15.105:0/1687384780 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3c09b880 con 0x7f2a280051e0 2023-11-28T18:28:09.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.874+0000 7f2a440fe640 1 -- 172.21.15.105:0/1687384780 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3c09be40 con 0x7f2a280051e0 2023-11-28T18:28:09.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.874+0000 7f2a440fe640 1 -- 172.21.15.105:0/1687384780 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3c067570 con 0x7f2a280051e0 2023-11-28T18:28:09.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.874+0000 7f2a327fc640 1 -- 172.21.15.105:0/1687384780 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f2a380145a0 con 0x7f2a280051e0 2023-11-28T18:28:09.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.874+0000 7f2a327fc640 1 --2- 172.21.15.105:0/1687384780 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2a200668d0 0x7f2a20068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:09.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.874+0000 7f2a327fc640 1 -- 172.21.15.105:0/1687384780 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f2a38084970 con 0x7f2a280051e0 2023-11-28T18:28:09.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.874+0000 7f2a41699640 1 --2- 172.21.15.105:0/1687384780 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2a200668d0 0x7f2a20068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:09.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.874+0000 7f2a41699640 1 --2- 172.21.15.105:0/1687384780 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2a200668d0 0x7f2a20068d90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c15d0d0 tx=0x7f2a2c001490 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:09.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:09.878+0000 7f2a327fc640 1 -- 172.21.15.105:0/1687384780 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2a38051e60 con 0x7f2a280051e0 2023-11-28T18:28:10.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.078+0000 7f2a440fe640 1 -- 172.21.15.105:0/1687384780 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2a3c0cf950 con 0x7f2a280051e0 2023-11-28T18:28:10.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.078+0000 7f2a327fc640 1 -- 172.21.15.105:0/1687384780 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f2a38022020 con 0x7f2a280051e0 2023-11-28T18:28:10.084 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:10.084 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:28:10.085 INFO:teuthology.orchestra.run.smithi105.stdout: "auth_dump": [ 2023-11-28T18:28:10.085 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:28:10.085 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.a", 2023-11-28T18:28:10.085 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlOXSTGxAAU5kGNB3tznYsDNG51vCIZA==", 2023-11-28T18:28:10.085 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:28:10.085 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:28:10.085 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:28:10.085 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:28:10.085 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:28:10.085 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:28:10.086 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:28:10.086 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:28:10.086 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.b", 2023-11-28T18:28:10.086 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlrnbWJxAAKefl0YI2MhobnAOIj6WeaQ==", 2023-11-28T18:28:10.086 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:28:10.086 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:28:10.086 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:28:10.086 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:28:10.086 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:28:10.086 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:28:10.087 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:28:10.087 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:28:10.087 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.c", 2023-11-28T18:28:10.087 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2Zl5Y7GIhAA5OvgkNNFaI7LnzSsQVdFsQ==", 2023-11-28T18:28:10.087 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:28:10.087 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:28:10.087 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:28:10.087 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:28:10.087 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:28:10.087 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:28:10.087 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:28:10.088 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:28:10.088 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.d", 2023-11-28T18:28:10.088 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlmfYdLxAAreI7UE1YCjzkj6yl3s5Fwg==", 2023-11-28T18:28:10.088 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:28:10.088 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:28:10.088 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:28:10.088 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:28:10.088 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:28:10.088 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:28:10.089 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:28:10.089 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:28:10.089 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.0", 2023-11-28T18:28:10.089 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDtL2ZlphxQEBAAQ280OXh1zeaotIdVU8oZng==", 2023-11-28T18:28:10.089 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:28:10.089 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:28:10.089 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:28:10.090 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:28:10.090 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:28:10.090 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:28:10.090 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:28:10.090 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.1", 2023-11-28T18:28:10.090 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDwL2ZlezmOFxAARRBIOTkVJYfquIdcpW7sQQ==", 2023-11-28T18:28:10.090 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:28:10.091 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:28:10.091 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:28:10.091 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:28:10.091 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:28:10.091 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:28:10.091 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:28:10.092 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.2", 2023-11-28T18:28:10.092 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDzL2Zl4YcSHxAA2ww2F4TAD/0H/ZJ+aVMGEQ==", 2023-11-28T18:28:10.092 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:28:10.092 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:28:10.092 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:28:10.092 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:28:10.092 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:28:10.092 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:28:10.092 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:28:10.093 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.3", 2023-11-28T18:28:10.093 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD2L2ZlNxWxJRAAImp7TjTF18KwFTZ14DM6EA==", 2023-11-28T18:28:10.093 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:28:10.093 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:28:10.093 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:28:10.093 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:28:10.093 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:28:10.093 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:28:10.093 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:28:10.094 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.4", 2023-11-28T18:28:10.094 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD8L2ZlOAieLxAAz8g+anH8uJRfsttUS41GGQ==", 2023-11-28T18:28:10.094 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:28:10.094 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:28:10.094 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:28:10.094 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:28:10.094 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:28:10.094 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:28:10.094 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:28:10.094 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.5", 2023-11-28T18:28:10.094 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD/L2Zl5PKTNxAAPk+uIodLPjSd0EEMVT6iEw==", 2023-11-28T18:28:10.095 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:28:10.095 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:28:10.095 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:28:10.095 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:28:10.095 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:28:10.095 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:28:10.095 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:28:10.095 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.6", 2023-11-28T18:28:10.095 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQADMGZll+FUAxAAHlym5Z8rd/OF1911ZE2OUg==", 2023-11-28T18:28:10.095 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:28:10.096 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:28:10.096 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:28:10.096 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:28:10.096 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:28:10.096 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:28:10.096 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:28:10.096 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.7", 2023-11-28T18:28:10.096 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQAGMGZl8GO4ChAAVn3N6DnxUcs+ew09QvGo2w==", 2023-11-28T18:28:10.096 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:28:10.097 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:28:10.097 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:28:10.097 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:28:10.097 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:28:10.097 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:28:10.097 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:28:10.097 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.0", 2023-11-28T18:28:10.097 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlSt0VMxAAQ8+j1BgBPMJ2cuiZj8aG0w==", 2023-11-28T18:28:10.097 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:28:10.098 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:28:10.098 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow r", 2023-11-28T18:28:10.098 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-28T18:28:10.098 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:28:10.098 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:28:10.098 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:28:10.098 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.1", 2023-11-28T18:28:10.098 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDqL2ZlfZ1/ARAAhnJB6oj4Cc6p/oCjM6hTCw==", 2023-11-28T18:28:10.098 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:28:10.099 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:28:10.099 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:28:10.099 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:28:10.099 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:28:10.099 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:28:10.099 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:28:10.099 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:28:10.099 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.2", 2023-11-28T18:28:10.099 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlPLtyORAAU2GST6TJzlTfh6YqEb3U3w==", 2023-11-28T18:28:10.100 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:28:10.100 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:28:10.100 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:28:10.100 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:28:10.100 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:28:10.100 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:28:10.100 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:28:10.101 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:28:10.101 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.3", 2023-11-28T18:28:10.101 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDqL2ZlhiFuBxAAKQRxCdtqWpAHy1AVvdj9Sg==", 2023-11-28T18:28:10.101 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:28:10.101 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:28:10.101 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:28:10.101 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:28:10.102 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:28:10.102 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:28:10.102 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:28:10.102 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:28:10.102 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.admin", 2023-11-28T18:28:10.102 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDoL2ZlkuP2OhAAxsdZEF9122vkH53EwB8Qug==", 2023-11-28T18:28:10.103 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:28:10.103 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:28:10.103 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:28:10.103 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:28:10.103 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:28:10.103 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:28:10.103 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:28:10.103 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:28:10.103 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.x", 2023-11-28T18:28:10.103 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlEG49FxAAespsG+QK1WTpbmpV+DIwiw==", 2023-11-28T18:28:10.104 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:28:10.104 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:28:10.104 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2023-11-28T18:28:10.104 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:28:10.104 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:28:10.104 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:28:10.104 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:28:10.104 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.y", 2023-11-28T18:28:10.104 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlU89RExAAwMJp+8zZ6aFEt0KbiB6toA==", 2023-11-28T18:28:10.104 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:28:10.105 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:28:10.105 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2023-11-28T18:28:10.105 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:28:10.105 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:28:10.105 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:28:10.105 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2023-11-28T18:28:10.106 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:28:10.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.082+0000 7f2a440fe640 1 -- 172.21.15.105:0/1687384780 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2a200668d0 msgr2=0x7f2a20068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:10.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.082+0000 7f2a440fe640 1 --2- 172.21.15.105:0/1687384780 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2a200668d0 0x7f2a20068d90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c15d0d0 tx=0x7f2a2c001490 comp rx=0 tx=0).stop 2023-11-28T18:28:10.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.082+0000 7f2a440fe640 1 -- 172.21.15.105:0/1687384780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a280051e0 msgr2=0x7f2a3c148610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:10.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.082+0000 7f2a440fe640 1 --2- 172.21.15.105:0/1687384780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a280051e0 0x7f2a3c148610 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f2a38011960 tx=0x7f2a38011e20 comp rx=0 tx=0).stop 2023-11-28T18:28:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.082+0000 7f2a440fe640 1 -- 172.21.15.105:0/1687384780 shutdown_connections 2023-11-28T18:28:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.082+0000 7f2a440fe640 1 --2- 172.21.15.105:0/1687384780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a3c15be10 0x7f2a3c15c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.082+0000 7f2a440fe640 1 --2- 172.21.15.105:0/1687384780 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2a3c0b19b0 0x7f2a3c09c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.082+0000 7f2a440fe640 1 --2- 172.21.15.105:0/1687384780 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2a200668d0 0x7f2a20068d90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.082+0000 7f2a440fe640 1 --2- 172.21.15.105:0/1687384780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a280051e0 0x7f2a3c148610 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.082+0000 7f2a440fe640 1 -- 172.21.15.105:0/1687384780 >> 172.21.15.105:0/1687384780 conn(0x7f2a3c0ceb90 msgr2=0x7f2a3c0cd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.082+0000 7f2a440fe640 1 -- 172.21.15.105:0/1687384780 shutdown_connections 2023-11-28T18:28:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.082+0000 7f2a440fe640 1 -- 172.21.15.105:0/1687384780 wait complete. 2023-11-28T18:28:10.109 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:28:10.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.246+0000 7ff9d751d640 1 Processor -- start 2023-11-28T18:28:10.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.246+0000 7ff9d751d640 1 -- start start 2023-11-28T18:28:10.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.246+0000 7ff9d751d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff9d015d4f0 0x7ff9d00d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:10.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.246+0000 7ff9d751d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff9d00b2d20 0x7ff9d00b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:10.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.246+0000 7ff9d751d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d00b25b0 con 0x7ff9d00d3c30 2023-11-28T18:28:10.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.246+0000 7ff9d751d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9d00b2900 con 0x7ff9d015d4f0 2023-11-28T18:28:10.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.246+0000 7ff9d751d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9d00d0a20 con 0x7ff9d00b2d20 2023-11-28T18:28:10.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.246+0000 7ff9d52b9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff9d015d4f0 0x7ff9d00d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:10.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.246+0000 7ff9d52b9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff9d015d4f0 0x7ff9d00d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49234/0 (socket says 172.21.15.105:49234) 2023-11-28T18:28:10.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.246+0000 7ff9d52b9640 1 -- 172.21.15.105:0/3615446498 learned_addr learned my addr 172.21.15.105:0/3615446498 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:10.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.246+0000 7ff9d5aba640 1 --2- 172.21.15.105:0/3615446498 >> v2:172.21.15.148:3300/0 conn(0x7ff9d00b2d20 0x7ff9d00b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:10.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.246+0000 7ff9d52b9640 1 -- 172.21.15.105:0/3615446498 >> v2:172.21.15.148:3300/0 conn(0x7ff9d00b2d20 msgr2=0x7ff9d00b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:10.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.246+0000 7ff9d52b9640 1 --2- 172.21.15.105:0/3615446498 >> v2:172.21.15.148:3300/0 conn(0x7ff9d00b2d20 0x7ff9d00b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.246+0000 7ff9d52b9640 1 -- 172.21.15.105:0/3615446498 >> v1:172.21.15.148:6789/0 conn(0x7ff9d00d3c30 legacy=0x7ff9d00d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:10.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.246+0000 7ff9d52b9640 1 -- 172.21.15.105:0/3615446498 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d00cea30 con 0x7ff9d015d4f0 2023-11-28T18:28:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.250+0000 7ff9d52b9640 1 --2- 172.21.15.105:0/3615446498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9d015d4f0 0x7ff9d00d0e10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0160fa0 tx=0x7ff9cc01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70732201ebb0962b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.250+0000 7ff9c7fff640 1 -- 172.21.15.105:0/3615446498 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0ab600 con 0x7ff9d015d4f0 2023-11-28T18:28:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.250+0000 7ff9c7fff640 1 -- 172.21.15.105:0/3615446498 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9cc0ab760 con 0x7ff9d015d4f0 2023-11-28T18:28:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.250+0000 7ff9c7fff640 1 -- 172.21.15.105:0/3615446498 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0abaf0 con 0x7ff9d015d4f0 2023-11-28T18:28:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.250+0000 7ff9d751d640 1 -- 172.21.15.105:0/3615446498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9d015d4f0 msgr2=0x7ff9d00d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.250+0000 7ff9d751d640 1 --2- 172.21.15.105:0/3615446498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9d015d4f0 0x7ff9d00d0e10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0160fa0 tx=0x7ff9cc01c400 comp rx=0 tx=0).stop 2023-11-28T18:28:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.250+0000 7ff9d751d640 1 -- 172.21.15.105:0/3615446498 shutdown_connections 2023-11-28T18:28:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.250+0000 7ff9d751d640 1 --2- 172.21.15.105:0/3615446498 >> v2:172.21.15.148:3300/0 conn(0x7ff9d00b2d20 0x7ff9d00b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.250+0000 7ff9d751d640 1 --2- 172.21.15.105:0/3615446498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9d015d4f0 0x7ff9d00d0e10 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.250+0000 7ff9d751d640 1 -- 172.21.15.105:0/3615446498 >> 172.21.15.105:0/3615446498 conn(0x7ff9d00d3030 msgr2=0x7ff9d00d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:10.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.250+0000 7ff9d751d640 1 -- 172.21.15.105:0/3615446498 shutdown_connections 2023-11-28T18:28:10.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.250+0000 7ff9d751d640 1 -- 172.21.15.105:0/3615446498 wait complete. 2023-11-28T18:28:10.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d751d640 1 Processor -- start 2023-11-28T18:28:10.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d751d640 1 -- start start 2023-11-28T18:28:10.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d751d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9d00b2d20 0x7ff9d015c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:10.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d751d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9d00d3c30 0x7ff9d00d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:10.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d751d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9d015d4f0 0x7ff9d01539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:10.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d751d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d014c4c0 con 0x7ff9d00b2d20 2023-11-28T18:28:10.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d751d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d0154860 con 0x7ff9d015d4f0 2023-11-28T18:28:10.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d751d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d0154580 con 0x7ff9d00d3c30 2023-11-28T18:28:10.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d52b9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9d00b2d20 0x7ff9d015c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:10.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d52b9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9d00b2d20 0x7ff9d015c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49236/0 (socket says 172.21.15.105:49236) 2023-11-28T18:28:10.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d52b9640 1 -- 172.21.15.105:0/4247235752 learned_addr learned my addr 172.21.15.105:0/4247235752 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:10.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d4ab8640 1 --2- 172.21.15.105:0/4247235752 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9d00d3c30 0x7ff9d00d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:10.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d5aba640 1 --2- 172.21.15.105:0/4247235752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9d015d4f0 0x7ff9d01539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:10.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d52b9640 1 -- 172.21.15.105:0/4247235752 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9d00d3c30 msgr2=0x7ff9d00d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:10.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d52b9640 1 --2- 172.21.15.105:0/4247235752 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9d00d3c30 0x7ff9d00d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d52b9640 1 -- 172.21.15.105:0/4247235752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9d015d4f0 msgr2=0x7ff9d01539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:10.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d52b9640 1 --2- 172.21.15.105:0/4247235752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9d015d4f0 0x7ff9d01539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d52b9640 1 -- 172.21.15.105:0/4247235752 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9cc0094d0 con 0x7ff9d00b2d20 2023-11-28T18:28:10.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d5aba640 1 --2- 172.21.15.105:0/4247235752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9d015d4f0 0x7ff9d01539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:10.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d52b9640 1 --2- 172.21.15.105:0/4247235752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9d00b2d20 0x7ff9d015c620 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc0ad040 tx=0x7ff9cc01ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:10.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9c5ffb640 1 -- 172.21.15.105:0/4247235752 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0ab600 con 0x7ff9d00b2d20 2023-11-28T18:28:10.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9c5ffb640 1 -- 172.21.15.105:0/4247235752 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9cc0b4800 con 0x7ff9d00b2d20 2023-11-28T18:28:10.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d751d640 1 -- 172.21.15.105:0/4247235752 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d015bb40 con 0x7ff9d00b2d20 2023-11-28T18:28:10.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9c5ffb640 1 -- 172.21.15.105:0/4247235752 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0b4bb0 con 0x7ff9d00b2d20 2023-11-28T18:28:10.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d751d640 1 -- 172.21.15.105:0/4247235752 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d015c100 con 0x7ff9d00b2d20 2023-11-28T18:28:10.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.254+0000 7ff9d751d640 1 -- 172.21.15.105:0/4247235752 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d0067570 con 0x7ff9d00b2d20 2023-11-28T18:28:10.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.258+0000 7ff9c5ffb640 1 -- 172.21.15.105:0/4247235752 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff9cc0b4d10 con 0x7ff9d00b2d20 2023-11-28T18:28:10.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.258+0000 7ff9c5ffb640 1 --2- 172.21.15.105:0/4247235752 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff9b40668d0 0x7ff9b4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:10.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.258+0000 7ff9c5ffb640 1 -- 172.21.15.105:0/4247235752 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5177+0+0 (secure 0 0 0) 0x7ff9cc11ecb0 con 0x7ff9d00b2d20 2023-11-28T18:28:10.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.258+0000 7ff9d4ab8640 1 --2- 172.21.15.105:0/4247235752 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff9b40668d0 0x7ff9b4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:10.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.258+0000 7ff9d4ab8640 1 --2- 172.21.15.105:0/4247235752 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff9b40668d0 0x7ff9b4068d90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000b810 tx=0x7ff9c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:10.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.262+0000 7ff9c5ffb640 1 -- 172.21.15.105:0/4247235752 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff9cc0ec0f0 con 0x7ff9d00b2d20 2023-11-28T18:28:10.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.414+0000 7ff9d751d640 1 -- 172.21.15.105:0/4247235752 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9d0156200 con 0x7ff9d00b2d20 2023-11-28T18:28:10.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.414+0000 7ff9c5ffb640 1 -- 172.21.15.105:0/4247235752 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+13123 (secure 0 0 0) 0x7ff9cc0ef710 con 0x7ff9d00b2d20 2023-11-28T18:28:10.419 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:10.419 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":74,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:28:09.709713+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:28:10.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.414+0000 7ff9d751d640 1 -- 172.21.15.105:0/4247235752 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff9b40668d0 msgr2=0x7ff9b4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:10.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.414+0000 7ff9d751d640 1 --2- 172.21.15.105:0/4247235752 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff9b40668d0 0x7ff9b4068d90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000b810 tx=0x7ff9c0008040 comp rx=0 tx=0).stop 2023-11-28T18:28:10.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.414+0000 7ff9d751d640 1 -- 172.21.15.105:0/4247235752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9d00b2d20 msgr2=0x7ff9d015c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:10.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.414+0000 7ff9d751d640 1 --2- 172.21.15.105:0/4247235752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9d00b2d20 0x7ff9d015c620 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc0ad040 tx=0x7ff9cc01ac40 comp rx=0 tx=0).stop 2023-11-28T18:28:10.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.414+0000 7ff9d751d640 1 -- 172.21.15.105:0/4247235752 shutdown_connections 2023-11-28T18:28:10.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.414+0000 7ff9d751d640 1 --2- 172.21.15.105:0/4247235752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9d015d4f0 0x7ff9d01539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.414+0000 7ff9d751d640 1 --2- 172.21.15.105:0/4247235752 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9d00d3c30 0x7ff9d00d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.414+0000 7ff9d751d640 1 --2- 172.21.15.105:0/4247235752 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff9b40668d0 0x7ff9b4068d90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.414+0000 7ff9d751d640 1 --2- 172.21.15.105:0/4247235752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9d00b2d20 0x7ff9d015c620 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.414+0000 7ff9d751d640 1 -- 172.21.15.105:0/4247235752 >> 172.21.15.105:0/4247235752 conn(0x7ff9d00d3030 msgr2=0x7ff9d006c840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:10.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.414+0000 7ff9d751d640 1 -- 172.21.15.105:0/4247235752 shutdown_connections 2023-11-28T18:28:10.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.414+0000 7ff9d751d640 1 -- 172.21.15.105:0/4247235752 wait complete. 2023-11-28T18:28:10.440 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:28:10.441 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:28:10.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.574+0000 7f7642cfe640 1 Processor -- start 2023-11-28T18:28:10.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.574+0000 7f7642cfe640 1 -- start start 2023-11-28T18:28:10.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.578+0000 7f7642cfe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f763c0b4290 0x7f763c0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:10.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.578+0000 7f7642cfe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f763c071960 0x7f763c15f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:10.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.578+0000 7f7642cfe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f763c15b030 0x7f763c0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:10.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.578+0000 7f7642cfe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f763c0b4bb0 con 0x7f763c0b4290 2023-11-28T18:28:10.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.578+0000 7f7642cfe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f763c0701d0 con 0x7f763c15b030 2023-11-28T18:28:10.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.578+0000 7f7642cfe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763c06df80 con 0x7f763c071960 2023-11-28T18:28:10.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.578+0000 7f7640a9a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f763c0b4290 0x7f763c0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:10.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.578+0000 7f7640a9a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f763c0b4290 0x7f763c0b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49240/0 (socket says 172.21.15.105:49240) 2023-11-28T18:28:10.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.578+0000 7f7640a9a640 1 -- 172.21.15.105:0/816730645 learned_addr learned my addr 172.21.15.105:0/816730645 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:10.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.578+0000 7f7633fff640 1 --2- 172.21.15.105:0/816730645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f763c071960 0x7f763c15f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:10.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.578+0000 7f764129b640 1 --2- 172.21.15.105:0/816730645 >> v2:172.21.15.148:3300/0 conn(0x7f763c15b030 0x7f763c0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:10.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.578+0000 7f7640a9a640 1 -- 172.21.15.105:0/816730645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f763c071960 msgr2=0x7f763c15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:10.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.578+0000 7f7640a9a640 1 --2- 172.21.15.105:0/816730645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f763c071960 0x7f763c15f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.578+0000 7f7640a9a640 1 -- 172.21.15.105:0/816730645 >> v2:172.21.15.148:3300/0 conn(0x7f763c15b030 msgr2=0x7f763c0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:10.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.578+0000 7f7640a9a640 1 --2- 172.21.15.105:0/816730645 >> v2:172.21.15.148:3300/0 conn(0x7f763c15b030 0x7f763c0709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.578+0000 7f7640a9a640 1 -- 172.21.15.105:0/816730645 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c071df0 con 0x7f763c0b4290 2023-11-28T18:28:10.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.578+0000 7f7633fff640 1 --2- 172.21.15.105:0/816730645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f763c071960 0x7f763c15f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:10.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.582+0000 7f7640a9a640 1 --2- 172.21.15.105:0/816730645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f763c0b4290 0x7f763c0b4670 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f763c06bc50 tx=0x7f763801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32476bb214c1ee53 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:10.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.582+0000 7f76337fe640 1 -- 172.21.15.105:0/816730645 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76380ac600 con 0x7f763c0b4290 2023-11-28T18:28:10.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.582+0000 7f76337fe640 1 -- 172.21.15.105:0/816730645 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76380a8050 con 0x7f763c0b4290 2023-11-28T18:28:10.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.582+0000 7f76337fe640 1 -- 172.21.15.105:0/816730645 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76380ac980 con 0x7f763c0b4290 2023-11-28T18:28:10.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.582+0000 7f7642cfe640 1 -- 172.21.15.105:0/816730645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f763c0b4290 msgr2=0x7f763c0b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:10.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.582+0000 7f7642cfe640 1 --2- 172.21.15.105:0/816730645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f763c0b4290 0x7f763c0b4670 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f763c06bc50 tx=0x7f763801c4d0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.582+0000 7f7642cfe640 1 -- 172.21.15.105:0/816730645 shutdown_connections 2023-11-28T18:28:10.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.582+0000 7f7642cfe640 1 --2- 172.21.15.105:0/816730645 >> v2:172.21.15.148:3300/0 conn(0x7f763c15b030 0x7f763c0709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.582+0000 7f7642cfe640 1 --2- 172.21.15.105:0/816730645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f763c071960 0x7f763c15f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.582+0000 7f7642cfe640 1 --2- 172.21.15.105:0/816730645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f763c0b4290 0x7f763c0b4670 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.582+0000 7f7642cfe640 1 -- 172.21.15.105:0/816730645 >> 172.21.15.105:0/816730645 conn(0x7f763c0ceb90 msgr2=0x7f763c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:10.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.582+0000 7f7642cfe640 1 -- 172.21.15.105:0/816730645 shutdown_connections 2023-11-28T18:28:10.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.582+0000 7f7642cfe640 1 -- 172.21.15.105:0/816730645 wait complete. 2023-11-28T18:28:10.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.582+0000 7f7642cfe640 1 Processor -- start 2023-11-28T18:28:10.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.582+0000 7f7642cfe640 1 -- start start 2023-11-28T18:28:10.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.586+0000 7f7642cfe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f763c0b4290 0x7f763c09a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:10.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.586+0000 7f7642cfe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f763c15b030 0x7f763c15be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:10.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.586+0000 7f7642cfe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f763c09bb50 0x7f763c15dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:10.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.586+0000 7f7642cfe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c09b6c0 con 0x7f763c0b4290 2023-11-28T18:28:10.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.586+0000 7f7642cfe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c09aab0 con 0x7f763c15b030 2023-11-28T18:28:10.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.586+0000 7f7642cfe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763c09ad90 con 0x7f763c09bb50 2023-11-28T18:28:10.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.586+0000 7f7640a9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f763c0b4290 0x7f763c09a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:10.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.586+0000 7f7640a9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f763c0b4290 0x7f763c09a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49252/0 (socket says 172.21.15.105:49252) 2023-11-28T18:28:10.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.586+0000 7f7640a9a640 1 -- 172.21.15.105:0/1252643959 learned_addr learned my addr 172.21.15.105:0/1252643959 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:10.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.586+0000 7f7640a9a640 1 -- 172.21.15.105:0/1252643959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f763c09bb50 msgr2=0x7f763c15dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:10.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.586+0000 7f7640a9a640 1 --2- 172.21.15.105:0/1252643959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f763c09bb50 0x7f763c15dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.586+0000 7f7640a9a640 1 -- 172.21.15.105:0/1252643959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f763c15b030 msgr2=0x7f763c15be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:10.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.586+0000 7f7640a9a640 1 --2- 172.21.15.105:0/1252643959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f763c15b030 0x7f763c15be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.586+0000 7f7640a9a640 1 -- 172.21.15.105:0/1252643959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76380094d0 con 0x7f763c0b4290 2023-11-28T18:28:10.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.586+0000 7f7640a9a640 1 --2- 172.21.15.105:0/1252643959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f763c0b4290 0x7f763c09a3b0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f763801ab80 tx=0x7f763801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:10.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.586+0000 7f76317fa640 1 -- 172.21.15.105:0/1252643959 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76380b7620 con 0x7f763c0b4290 2023-11-28T18:28:10.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.586+0000 7f76317fa640 1 -- 172.21.15.105:0/1252643959 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76380b7780 con 0x7f763c0b4290 2023-11-28T18:28:10.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.586+0000 7f76317fa640 1 -- 172.21.15.105:0/1252643959 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76380b7af0 con 0x7f763c0b4290 2023-11-28T18:28:10.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.586+0000 7f7642cfe640 1 -- 172.21.15.105:0/1252643959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763c09b070 con 0x7f763c0b4290 2023-11-28T18:28:10.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.586+0000 7f7642cfe640 1 -- 172.21.15.105:0/1252643959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763c15e010 con 0x7f763c0b4290 2023-11-28T18:28:10.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.586+0000 7f7642cfe640 1 -- 172.21.15.105:0/1252643959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f763c067570 con 0x7f763c0b4290 2023-11-28T18:28:10.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.586+0000 7f76317fa640 1 -- 172.21.15.105:0/1252643959 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f76380b3040 con 0x7f763c0b4290 2023-11-28T18:28:10.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.590+0000 7f76317fa640 1 --2- 172.21.15.105:0/1252643959 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f76280668d0 0x7f7628068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:10.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.590+0000 7f76317fa640 1 -- 172.21.15.105:0/1252643959 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f763811e2e0 con 0x7f763c0b4290 2023-11-28T18:28:10.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.590+0000 7f7633fff640 1 --2- 172.21.15.105:0/1252643959 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f76280668d0 0x7f7628068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:10.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.590+0000 7f7633fff640 1 --2- 172.21.15.105:0/1252643959 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f76280668d0 0x7f7628068d90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f762c006f10 tx=0x7f762c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:10.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.594+0000 7f76317fa640 1 -- 172.21.15.105:0/1252643959 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f76380eb7c0 con 0x7f763c0b4290 2023-11-28T18:28:10.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.746+0000 7f7642cfe640 1 -- 172.21.15.105:0/1252643959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f763c0cf7f0 con 0x7f763c0b4290 2023-11-28T18:28:10.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.746+0000 7f76317fa640 1 -- 172.21.15.105:0/1252643959 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76380eede0 con 0x7f763c0b4290 2023-11-28T18:28:10.751 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:28:10.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.746+0000 7f7642cfe640 1 -- 172.21.15.105:0/1252643959 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f76280668d0 msgr2=0x7f7628068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:10.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.746+0000 7f7642cfe640 1 --2- 172.21.15.105:0/1252643959 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f76280668d0 0x7f7628068d90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f762c006f10 tx=0x7f762c008040 comp rx=0 tx=0).stop 2023-11-28T18:28:10.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.746+0000 7f7642cfe640 1 -- 172.21.15.105:0/1252643959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f763c0b4290 msgr2=0x7f763c09a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:10.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.746+0000 7f7642cfe640 1 --2- 172.21.15.105:0/1252643959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f763c0b4290 0x7f763c09a3b0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f763801ab80 tx=0x7f763801ad10 comp rx=0 tx=0).stop 2023-11-28T18:28:10.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.746+0000 7f7642cfe640 1 -- 172.21.15.105:0/1252643959 shutdown_connections 2023-11-28T18:28:10.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.746+0000 7f7642cfe640 1 --2- 172.21.15.105:0/1252643959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f763c09bb50 0x7f763c15dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.746+0000 7f7642cfe640 1 --2- 172.21.15.105:0/1252643959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f763c15b030 0x7f763c15be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.746+0000 7f7642cfe640 1 --2- 172.21.15.105:0/1252643959 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f76280668d0 0x7f7628068d90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.746+0000 7f7642cfe640 1 --2- 172.21.15.105:0/1252643959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f763c0b4290 0x7f763c09a3b0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.746+0000 7f7642cfe640 1 -- 172.21.15.105:0/1252643959 >> 172.21.15.105:0/1252643959 conn(0x7f763c0ceb90 msgr2=0x7f763c06c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:10.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.746+0000 7f7642cfe640 1 -- 172.21.15.105:0/1252643959 shutdown_connections 2023-11-28T18:28:10.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.746+0000 7f7642cfe640 1 -- 172.21.15.105:0/1252643959 wait complete. 2023-11-28T18:28:10.772 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T18:28:10.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.902+0000 7fd5ef5f5640 1 Processor -- start 2023-11-28T18:28:10.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.902+0000 7fd5ef5f5640 1 -- start start 2023-11-28T18:28:10.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.906+0000 7fd5ef5f5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd5e8159f40 0x7fd5e80caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:10.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.906+0000 7fd5ef5f5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd5e80ad850 0x7fd5e80701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.906+0000 7fd5ef5f5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e806bf50 con 0x7fd5e80cc760 2023-11-28T18:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.906+0000 7fd5ef5f5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e80ccbc0 con 0x7fd5e80ad850 2023-11-28T18:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.906+0000 7fd5ef5f5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e815a320 con 0x7fd5e8159f40 2023-11-28T18:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.906+0000 7fd5edb92640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd5e80ad850 0x7fd5e80701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.906+0000 7fd5edb92640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd5e80ad850 0x7fd5e80701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49268/0 (socket says 172.21.15.105:49268) 2023-11-28T18:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.906+0000 7fd5edb92640 1 -- 172.21.15.105:0/1007309423 learned_addr learned my addr 172.21.15.105:0/1007309423 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.906+0000 7fd5ed391640 1 --2- 172.21.15.105:0/1007309423 >> v2:172.21.15.148:3300/0 conn(0x7fd5e8159f40 0x7fd5e80caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.906+0000 7fd5edb92640 1 -- 172.21.15.105:0/1007309423 >> v2:172.21.15.148:3300/0 conn(0x7fd5e8159f40 msgr2=0x7fd5e80caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.906+0000 7fd5edb92640 1 --2- 172.21.15.105:0/1007309423 >> v2:172.21.15.148:3300/0 conn(0x7fd5e8159f40 0x7fd5e80caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.906+0000 7fd5edb92640 1 -- 172.21.15.105:0/1007309423 >> v1:172.21.15.148:6789/0 conn(0x7fd5e80cc760 legacy=0x7fd5e80ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.906+0000 7fd5edb92640 1 -- 172.21.15.105:0/1007309423 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e8071380 con 0x7fd5e80ad850 2023-11-28T18:28:10.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.910+0000 7fd5edb92640 1 --2- 172.21.15.105:0/1007309423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5e80ad850 0x7fd5e80701d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fd5e806e500 tx=0x7fd5e001eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cec4516a3d369c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:10.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.910+0000 7fd5dffff640 1 -- 172.21.15.105:0/1007309423 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e00a79f0 con 0x7fd5e80ad850 2023-11-28T18:28:10.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.910+0000 7fd5dffff640 1 -- 172.21.15.105:0/1007309423 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e00a7b50 con 0x7fd5e80ad850 2023-11-28T18:28:10.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.910+0000 7fd5dffff640 1 -- 172.21.15.105:0/1007309423 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e00af3f0 con 0x7fd5e80ad850 2023-11-28T18:28:10.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.910+0000 7fd5ef5f5640 1 -- 172.21.15.105:0/1007309423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5e80ad850 msgr2=0x7fd5e80701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:10.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.910+0000 7fd5ef5f5640 1 --2- 172.21.15.105:0/1007309423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5e80ad850 0x7fd5e80701d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fd5e806e500 tx=0x7fd5e001eb20 comp rx=0 tx=0).stop 2023-11-28T18:28:10.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.910+0000 7fd5ef5f5640 1 -- 172.21.15.105:0/1007309423 shutdown_connections 2023-11-28T18:28:10.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.910+0000 7fd5ef5f5640 1 --2- 172.21.15.105:0/1007309423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5e80ad850 0x7fd5e80701d0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.910+0000 7fd5ef5f5640 1 --2- 172.21.15.105:0/1007309423 >> v2:172.21.15.148:3300/0 conn(0x7fd5e8159f40 0x7fd5e80caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.910+0000 7fd5ef5f5640 1 -- 172.21.15.105:0/1007309423 >> 172.21.15.105:0/1007309423 conn(0x7fd5e80ceb90 msgr2=0x7fd5e80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:10.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.910+0000 7fd5ef5f5640 1 -- 172.21.15.105:0/1007309423 shutdown_connections 2023-11-28T18:28:10.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.910+0000 7fd5ef5f5640 1 -- 172.21.15.105:0/1007309423 wait complete. 2023-11-28T18:28:10.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.910+0000 7fd5ef5f5640 1 Processor -- start 2023-11-28T18:28:10.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.910+0000 7fd5ef5f5640 1 -- start start 2023-11-28T18:28:10.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5ef5f5640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5e80ad850 0x7fd5e814d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:10.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5ef5f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5e80cc760 0x7fd5e81541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:10.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5ef5f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5e8159f40 0x7fd5e809b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:10.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5ef5f5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e809b130 con 0x7fd5e8159f40 2023-11-28T18:28:10.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5ef5f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e8099a50 con 0x7fd5e80cc760 2023-11-28T18:28:10.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5ef5f5640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e814d770 con 0x7fd5e80ad850 2023-11-28T18:28:10.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5edb92640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5e8159f40 0x7fd5e809b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:10.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5edb92640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5e8159f40 0x7fd5e809b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49274/0 (socket says 172.21.15.105:49274) 2023-11-28T18:28:10.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5edb92640 1 -- 172.21.15.105:0/2541082660 learned_addr learned my addr 172.21.15.105:0/2541082660 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:10.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5ed391640 1 --2- 172.21.15.105:0/2541082660 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5e80ad850 0x7fd5e814d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:10.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5ecb90640 1 --2- 172.21.15.105:0/2541082660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5e80cc760 0x7fd5e81541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:10.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5edb92640 1 -- 172.21.15.105:0/2541082660 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5e80ad850 msgr2=0x7fd5e814d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:10.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5edb92640 1 --2- 172.21.15.105:0/2541082660 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5e80ad850 0x7fd5e814d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5edb92640 1 -- 172.21.15.105:0/2541082660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5e80cc760 msgr2=0x7fd5e81541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:10.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5edb92640 1 --2- 172.21.15.105:0/2541082660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5e80cc760 0x7fd5e81541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:10.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5edb92640 1 -- 172.21.15.105:0/2541082660 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e0009b30 con 0x7fd5e8159f40 2023-11-28T18:28:10.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5edb92640 1 --2- 172.21.15.105:0/2541082660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5e8159f40 0x7fd5e809b650 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00b1040 tx=0x7fd5e001b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:10.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5ddffb640 1 -- 172.21.15.105:0/2541082660 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e00a79f0 con 0x7fd5e8159f40 2023-11-28T18:28:10.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5ddffb640 1 -- 172.21.15.105:0/2541082660 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e000e150 con 0x7fd5e8159f40 2023-11-28T18:28:10.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5ef5f5640 1 -- 172.21.15.105:0/2541082660 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e809c4d0 con 0x7fd5e8159f40 2023-11-28T18:28:10.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5ddffb640 1 -- 172.21.15.105:0/2541082660 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0007ea0 con 0x7fd5e8159f40 2023-11-28T18:28:10.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5ef5f5640 1 -- 172.21.15.105:0/2541082660 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e8099ca0 con 0x7fd5e8159f40 2023-11-28T18:28:10.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5ef5f5640 1 -- 172.21.15.105:0/2541082660 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e80adcc0 con 0x7fd5e8159f40 2023-11-28T18:28:10.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.914+0000 7fd5ddffb640 1 -- 172.21.15.105:0/2541082660 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd5e00af3f0 con 0x7fd5e8159f40 2023-11-28T18:28:10.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.918+0000 7fd5ddffb640 1 --2- 172.21.15.105:0/2541082660 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5cc066900 0x7fd5cc068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:10.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.918+0000 7fd5ddffb640 1 -- 172.21.15.105:0/2541082660 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fd5e011e900 con 0x7fd5e8159f40 2023-11-28T18:28:10.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.918+0000 7fd5ed391640 1 --2- 172.21.15.105:0/2541082660 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5cc066900 0x7fd5cc068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:10.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.918+0000 7fd5ed391640 1 --2- 172.21.15.105:0/2541082660 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5cc066900 0x7fd5cc068dc0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd5e80afb10 tx=0x7fd5e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:10.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:10.922+0000 7fd5ddffb640 1 -- 172.21.15.105:0/2541082660 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd5e80adcc0 con 0x7fd5e8159f40 2023-11-28T18:28:11.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.070+0000 7fd5ef5f5640 1 -- 172.21.15.105:0/2541082660 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd5e80adcc0 con 0x7fd5e8159f40 2023-11-28T18:28:11.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.070+0000 7fd5ddffb640 1 -- 172.21.15.105:0/2541082660 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd5e00be060 con 0x7fd5e8159f40 2023-11-28T18:28:11.077 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:11.077 INFO:teuthology.orchestra.run.smithi105.stdout:[ 2023-11-28T18:28:11.077 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:28:11.077 INFO:teuthology.orchestra.run.smithi105.stdout: "poolnum": 1, 2023-11-28T18:28:11.077 INFO:teuthology.orchestra.run.smithi105.stdout: "poolname": ".mgr" 2023-11-28T18:28:11.077 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:28:11.077 INFO:teuthology.orchestra.run.smithi105.stdout:] 2023-11-28T18:28:11.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.074+0000 7fd5ef5f5640 1 -- 172.21.15.105:0/2541082660 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5cc066900 msgr2=0x7fd5cc068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:11.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.074+0000 7fd5ef5f5640 1 --2- 172.21.15.105:0/2541082660 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5cc066900 0x7fd5cc068dc0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd5e80afb10 tx=0x7fd5e4008040 comp rx=0 tx=0).stop 2023-11-28T18:28:11.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.074+0000 7fd5ef5f5640 1 -- 172.21.15.105:0/2541082660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5e8159f40 msgr2=0x7fd5e809b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:11.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.074+0000 7fd5ef5f5640 1 --2- 172.21.15.105:0/2541082660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5e8159f40 0x7fd5e809b650 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00b1040 tx=0x7fd5e001b8c0 comp rx=0 tx=0).stop 2023-11-28T18:28:11.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.074+0000 7fd5ef5f5640 1 -- 172.21.15.105:0/2541082660 shutdown_connections 2023-11-28T18:28:11.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.074+0000 7fd5ef5f5640 1 --2- 172.21.15.105:0/2541082660 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5cc066900 0x7fd5cc068dc0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:11.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.074+0000 7fd5ef5f5640 1 --2- 172.21.15.105:0/2541082660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5e8159f40 0x7fd5e809b650 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:11.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.074+0000 7fd5ef5f5640 1 --2- 172.21.15.105:0/2541082660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5e80cc760 0x7fd5e81541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:11.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.074+0000 7fd5ef5f5640 1 --2- 172.21.15.105:0/2541082660 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5e80ad850 0x7fd5e814d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:11.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.074+0000 7fd5ef5f5640 1 -- 172.21.15.105:0/2541082660 >> 172.21.15.105:0/2541082660 conn(0x7fd5e80ceb90 msgr2=0x7fd5e80ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:11.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.074+0000 7fd5ef5f5640 1 -- 172.21.15.105:0/2541082660 shutdown_connections 2023-11-28T18:28:11.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.074+0000 7fd5ef5f5640 1 -- 172.21.15.105:0/2541082660 wait complete. 2023-11-28T18:28:11.098 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T18:28:11.098 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T18:28:11.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbea652a640 1 Processor -- start 2023-11-28T18:28:11.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbea652a640 1 -- start start 2023-11-28T18:28:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbea652a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbea00af190 0x7fbea0166130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbea652a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbea00cfc10 con 0x7fbea00b2300 2023-11-28T18:28:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbea652a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbea00b1b60 con 0x7fbea015e6e0 2023-11-28T18:28:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbea652a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbea00b12c0 con 0x7fbea00af190 2023-11-28T18:28:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbe9f7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fbea00b2300 0x7fbea00cf500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47872/0 (socket says 172.21.15.105:47872) 2023-11-28T18:28:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbe9f7fe640 1 -- 172.21.15.105:0/3905861903 learned_addr learned my addr 172.21.15.105:0/3905861903 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbea4ac7640 1 --2- 172.21.15.105:0/3905861903 >> v2:172.21.15.148:3300/0 conn(0x7fbea00af190 0x7fbea0166130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbe9effd640 1 -- 172.21.15.105:0/3905861903 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445391322 0 0) 0x7fbea00cfc10 con 0x7fbea00b2300 2023-11-28T18:28:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbe9effd640 1 -- 172.21.15.105:0/3905861903 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe88003530 con 0x7fbea00b2300 2023-11-28T18:28:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbe9effd640 1 -- 172.21.15.105:0/3905861903 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 666872149 0 0) 0x7fbe88003530 con 0x7fbea00b2300 2023-11-28T18:28:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbe9effd640 1 -- 172.21.15.105:0/3905861903 >> v2:172.21.15.148:3300/0 conn(0x7fbea00af190 msgr2=0x7fbea0166130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbe9effd640 1 --2- 172.21.15.105:0/3905861903 >> v2:172.21.15.148:3300/0 conn(0x7fbea00af190 0x7fbea0166130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbe9effd640 1 -- 172.21.15.105:0/3905861903 >> v1:172.21.15.148:6789/0 conn(0x7fbea015e6e0 legacy=0x7fbea00cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbe9effd640 1 -- 172.21.15.105:0/3905861903 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea00b15a0 con 0x7fbea00b2300 2023-11-28T18:28:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbe9effd640 1 -- 172.21.15.105:0/3905861903 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fbe94001680 con 0x7fbea00b2300 2023-11-28T18:28:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbe9effd640 1 -- 172.21.15.105:0/3905861903 >> v1:172.21.15.105:6789/0 conn(0x7fbea00b2300 legacy=0x7fbea00cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbe9effd640 1 --2- 172.21.15.105:0/3905861903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe88003980 0x7fbe88003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbe9effd640 1 --2- 172.21.15.105:0/3905861903 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe880047d0 0x7fbe88004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbe9effd640 1 --2- 172.21.15.105:0/3905861903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe88005190 0x7fbe8800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbe9effd640 1 -- 172.21.15.105:0/3905861903 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea00b12c0 con 0x7fbe88005190 2023-11-28T18:28:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbe9effd640 1 -- 172.21.15.105:0/3905861903 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8800c040 con 0x7fbe88003980 2023-11-28T18:28:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.234+0000 7fbe9effd640 1 -- 172.21.15.105:0/3905861903 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8800c200 con 0x7fbe880047d0 2023-11-28T18:28:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.238+0000 7fbea4ac7640 1 --2- 172.21.15.105:0/3905861903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe88005190 0x7fbe8800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.238+0000 7fbe9f7fe640 1 --2- 172.21.15.105:0/3905861903 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe880047d0 0x7fbe88004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.238+0000 7fbe9ffff640 1 --2- 172.21.15.105:0/3905861903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe88003980 0x7fbe88003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.238+0000 7fbea4ac7640 1 -- 172.21.15.105:0/3905861903 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe880047d0 msgr2=0x7fbe88004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.238+0000 7fbea4ac7640 1 --2- 172.21.15.105:0/3905861903 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe880047d0 0x7fbe88004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.238+0000 7fbea4ac7640 1 -- 172.21.15.105:0/3905861903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe88003980 msgr2=0x7fbe88003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.238+0000 7fbea4ac7640 1 --2- 172.21.15.105:0/3905861903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe88003980 0x7fbe88003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.238+0000 7fbea4ac7640 1 -- 172.21.15.105:0/3905861903 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe88003530 con 0x7fbe88005190 2023-11-28T18:28:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.238+0000 7fbe9ffff640 1 --2- 172.21.15.105:0/3905861903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe88003980 0x7fbe88003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:11.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.238+0000 7fbea4ac7640 1 -- 172.21.15.105:0/3905861903 reap_dead start 2023-11-28T18:28:11.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.238+0000 7fbea4ac7640 1 --2- 172.21.15.105:0/3905861903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe88005190 0x7fbe8800b990 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009fc0 tx=0x7fbe9801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc19b5110719dea4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.238+0000 7fbe9effd640 1 -- 172.21.15.105:0/3905861903 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe980a5cb0 con 0x7fbe88005190 2023-11-28T18:28:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.238+0000 7fbe9effd640 1 -- 172.21.15.105:0/3905861903 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9800f040 con 0x7fbe88005190 2023-11-28T18:28:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.238+0000 7fbe9effd640 1 -- 172.21.15.105:0/3905861903 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe98003be0 con 0x7fbe88005190 2023-11-28T18:28:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.238+0000 7fbea652a640 1 -- 172.21.15.105:0/3905861903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe88005190 msgr2=0x7fbe8800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.238+0000 7fbea652a640 1 --2- 172.21.15.105:0/3905861903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe88005190 0x7fbe8800b990 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009fc0 tx=0x7fbe9801cca0 comp rx=0 tx=0).stop 2023-11-28T18:28:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbea652a640 1 -- 172.21.15.105:0/3905861903 shutdown_connections 2023-11-28T18:28:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbea652a640 1 --2- 172.21.15.105:0/3905861903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe88005190 0x7fbe8800b990 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbea652a640 1 -- 172.21.15.105:0/3905861903 >> 172.21.15.105:0/3905861903 conn(0x7fbea00d0df0 msgr2=0x7fbea00d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbea652a640 1 -- 172.21.15.105:0/3905861903 shutdown_connections 2023-11-28T18:28:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbea652a640 1 -- 172.21.15.105:0/3905861903 wait complete. 2023-11-28T18:28:11.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbea652a640 1 Processor -- start 2023-11-28T18:28:11.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbea652a640 1 -- start start 2023-11-28T18:28:11.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbea652a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe88005190 0x7fbea015d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:11.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbea652a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbea00b5a10 0x7fbea00b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:11.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbea652a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbea00c3b60 0x7fbea00c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:11.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbea652a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea00c4520 con 0x7fbe88005190 2023-11-28T18:28:11.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbea652a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea015de10 con 0x7fbea00c3b60 2023-11-28T18:28:11.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbea652a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea006c480 con 0x7fbea00b5a10 2023-11-28T18:28:11.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbe9ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe88005190 0x7fbea015d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:11.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbe9ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe88005190 0x7fbea015d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46248/0 (socket says 172.21.15.105:46248) 2023-11-28T18:28:11.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbe9ffff640 1 -- 172.21.15.105:0/2334628498 learned_addr learned my addr 172.21.15.105:0/2334628498 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:11.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbe9f7fe640 1 --2- 172.21.15.105:0/2334628498 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbea00b5a10 0x7fbea00b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:11.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbea4ac7640 1 --2- 172.21.15.105:0/2334628498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbea00c3b60 0x7fbea00c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:11.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbe9ffff640 1 -- 172.21.15.105:0/2334628498 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbea00b5a10 msgr2=0x7fbea00b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:11.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbe9ffff640 1 --2- 172.21.15.105:0/2334628498 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbea00b5a10 0x7fbea00b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:11.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbe9ffff640 1 -- 172.21.15.105:0/2334628498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbea00c3b60 msgr2=0x7fbea00c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:11.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbe9ffff640 1 --2- 172.21.15.105:0/2334628498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbea00c3b60 0x7fbea00c3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:11.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbe9ffff640 1 -- 172.21.15.105:0/2334628498 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea006c720 con 0x7fbe88005190 2023-11-28T18:28:11.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbe9f7fe640 1 --2- 172.21.15.105:0/2334628498 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbea00b5a10 0x7fbea00b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:11.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.242+0000 7fbe9ffff640 1 --2- 172.21.15.105:0/2334628498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe88005190 0x7fbea015d8d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000e4b0 tx=0x7fbe9000e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:11.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.246+0000 7fbe9cff9640 1 -- 172.21.15.105:0/2334628498 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe900052e0 con 0x7fbe88005190 2023-11-28T18:28:11.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.246+0000 7fbe9cff9640 1 -- 172.21.15.105:0/2334628498 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe90005440 con 0x7fbe88005190 2023-11-28T18:28:11.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.246+0000 7fbe9cff9640 1 -- 172.21.15.105:0/2334628498 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90003820 con 0x7fbe88005190 2023-11-28T18:28:11.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.246+0000 7fbea652a640 1 -- 172.21.15.105:0/2334628498 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea006ca00 con 0x7fbe88005190 2023-11-28T18:28:11.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.246+0000 7fbea652a640 1 -- 172.21.15.105:0/2334628498 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea006cfc0 con 0x7fbe88005190 2023-11-28T18:28:11.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.246+0000 7fbea652a640 1 -- 172.21.15.105:0/2334628498 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea006d180 con 0x7fbe88005190 2023-11-28T18:28:11.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.246+0000 7fbe9cff9640 1 -- 172.21.15.105:0/2334628498 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fbe90003980 con 0x7fbe88005190 2023-11-28T18:28:11.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.250+0000 7fbe9cff9640 1 --2- 172.21.15.105:0/2334628498 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbe8c0668d0 0x7fbe8c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:11.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.250+0000 7fbe9cff9640 1 -- 172.21.15.105:0/2334628498 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fbe90083e60 con 0x7fbe88005190 2023-11-28T18:28:11.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.250+0000 7fbe9f7fe640 1 --2- 172.21.15.105:0/2334628498 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbe8c0668d0 0x7fbe8c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:11.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.250+0000 7fbe9f7fe640 1 --2- 172.21.15.105:0/2334628498 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbe8c0668d0 0x7fbe8c068d90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbea00728a0 tx=0x7fbe94009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:11.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.250+0000 7fbe9cff9640 1 -- 172.21.15.105:0/2334628498 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbe90051340 con 0x7fbe88005190 2023-11-28T18:28:11.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.402+0000 7fbea652a640 1 -- 172.21.15.105:0/2334628498 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbea0067570 con 0x7fbe88005190 2023-11-28T18:28:11.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.702+0000 7fbe9cff9640 1 -- 172.21.15.105:0/2334628498 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbe90054960 con 0x7fbe88005190 2023-11-28T18:28:11.710 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' created 2023-11-28T18:28:11.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.706+0000 7fbea652a640 1 -- 172.21.15.105:0/2334628498 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbe8c0668d0 msgr2=0x7fbe8c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:11.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.706+0000 7fbea652a640 1 --2- 172.21.15.105:0/2334628498 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbe8c0668d0 0x7fbe8c068d90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbea00728a0 tx=0x7fbe94009040 comp rx=0 tx=0).stop 2023-11-28T18:28:11.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.706+0000 7fbea652a640 1 -- 172.21.15.105:0/2334628498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe88005190 msgr2=0x7fbea015d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:11.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.706+0000 7fbea652a640 1 --2- 172.21.15.105:0/2334628498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe88005190 0x7fbea015d8d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000e4b0 tx=0x7fbe9000e970 comp rx=0 tx=0).stop 2023-11-28T18:28:11.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.706+0000 7fbea652a640 1 -- 172.21.15.105:0/2334628498 shutdown_connections 2023-11-28T18:28:11.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.706+0000 7fbea652a640 1 --2- 172.21.15.105:0/2334628498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbea00c3b60 0x7fbea00c3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:11.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.706+0000 7fbea652a640 1 --2- 172.21.15.105:0/2334628498 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbea00b5a10 0x7fbea00b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:11.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.706+0000 7fbea652a640 1 --2- 172.21.15.105:0/2334628498 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbe8c0668d0 0x7fbe8c068d90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:11.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.706+0000 7fbea652a640 1 --2- 172.21.15.105:0/2334628498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe88005190 0x7fbea015d8d0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:11.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.706+0000 7fbea652a640 1 -- 172.21.15.105:0/2334628498 >> 172.21.15.105:0/2334628498 conn(0x7fbea00d0df0 msgr2=0x7fbea00cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:11.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.706+0000 7fbea652a640 1 -- 172.21.15.105:0/2334628498 shutdown_connections 2023-11-28T18:28:11.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.706+0000 7fbea652a640 1 -- 172.21.15.105:0/2334628498 wait complete. 2023-11-28T18:28:11.735 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T18:28:11.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.874+0000 7f5087757640 1 Processor -- start 2023-11-28T18:28:11.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.874+0000 7f5087757640 1 -- start start 2023-11-28T18:28:11.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.878+0000 7f5087757640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5080159f40 0x7f50800caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:11.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.878+0000 7f5087757640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f50800ad850 0x7f50800701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:11.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.878+0000 7f5087757640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508006bf50 con 0x7f50800cc760 2023-11-28T18:28:11.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.878+0000 7f5087757640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50800ccbc0 con 0x7f5080159f40 2023-11-28T18:28:11.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.878+0000 7f5087757640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508015a320 con 0x7f50800ad850 2023-11-28T18:28:11.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.878+0000 7f50854f3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5080159f40 0x7f50800caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:11.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.878+0000 7f5084cf2640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f50800cc760 0x7f50800ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47882/0 (socket says 172.21.15.105:47882) 2023-11-28T18:28:11.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.878+0000 7f5084cf2640 1 -- 172.21.15.105:0/2745798784 learned_addr learned my addr 172.21.15.105:0/2745798784 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:11.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.878+0000 7f5085cf4640 1 --2- 172.21.15.105:0/2745798784 >> v2:172.21.15.148:3300/0 conn(0x7f50800ad850 0x7f50800701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:11.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.878+0000 7f50854f3640 1 -- 172.21.15.105:0/2745798784 >> v2:172.21.15.148:3300/0 conn(0x7f50800ad850 msgr2=0x7f50800701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:11.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.878+0000 7f50854f3640 1 --2- 172.21.15.105:0/2745798784 >> v2:172.21.15.148:3300/0 conn(0x7f50800ad850 0x7f50800701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:11.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.878+0000 7f50854f3640 1 -- 172.21.15.105:0/2745798784 >> v1:172.21.15.105:6789/0 conn(0x7f50800cc760 legacy=0x7f50800ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:11.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.878+0000 7f5077fff640 1 -- 172.21.15.105:0/2745798784 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3470614412 0 0) 0x7f508006bf50 con 0x7f50800cc760 2023-11-28T18:28:11.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.878+0000 7f50854f3640 1 -- 172.21.15.105:0/2745798784 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5080071380 con 0x7f5080159f40 2023-11-28T18:28:11.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.882+0000 7f50854f3640 1 --2- 172.21.15.105:0/2745798784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5080159f40 0x7f50800caf60 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f50800b0850 tx=0x7f507c01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a14bb4ce5d74a22 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:11.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.882+0000 7f5077fff640 1 -- 172.21.15.105:0/2745798784 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c0ac600 con 0x7f5080159f40 2023-11-28T18:28:11.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.882+0000 7f5077fff640 1 -- 172.21.15.105:0/2745798784 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c0a8050 con 0x7f5080159f40 2023-11-28T18:28:11.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.882+0000 7f5077fff640 1 -- 172.21.15.105:0/2745798784 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c0ac980 con 0x7f5080159f40 2023-11-28T18:28:11.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.882+0000 7f5087757640 1 -- 172.21.15.105:0/2745798784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5080159f40 msgr2=0x7f50800caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:11.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.882+0000 7f5087757640 1 --2- 172.21.15.105:0/2745798784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5080159f40 0x7f50800caf60 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f50800b0850 tx=0x7f507c01c3a0 comp rx=0 tx=0).stop 2023-11-28T18:28:11.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.882+0000 7f5087757640 1 -- 172.21.15.105:0/2745798784 shutdown_connections 2023-11-28T18:28:11.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.882+0000 7f5087757640 1 --2- 172.21.15.105:0/2745798784 >> v2:172.21.15.148:3300/0 conn(0x7f50800ad850 0x7f50800701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:11.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.882+0000 7f5087757640 1 --2- 172.21.15.105:0/2745798784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5080159f40 0x7f50800caf60 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:11.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.882+0000 7f5087757640 1 -- 172.21.15.105:0/2745798784 >> 172.21.15.105:0/2745798784 conn(0x7f50800ceb90 msgr2=0x7f50800cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:11.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.882+0000 7f5087757640 1 -- 172.21.15.105:0/2745798784 shutdown_connections 2023-11-28T18:28:11.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.882+0000 7f5087757640 1 -- 172.21.15.105:0/2745798784 wait complete. 2023-11-28T18:28:11.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.882+0000 7f5087757640 1 Processor -- start 2023-11-28T18:28:11.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.882+0000 7f5087757640 1 -- start start 2023-11-28T18:28:11.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f5087757640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50800ad850 0x7f5080154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:11.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f5087757640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50800cc760 0x7f508014b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:11.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f5087757640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5080159f40 0x7f50801553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:11.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f5087757640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50801545f0 con 0x7f50800cc760 2023-11-28T18:28:11.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f5087757640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5080154e70 con 0x7f5080159f40 2023-11-28T18:28:11.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f5087757640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5080152e70 con 0x7f50800ad850 2023-11-28T18:28:11.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f5084cf2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50800cc760 0x7f508014b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:11.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f5084cf2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50800cc760 0x7f508014b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46270/0 (socket says 172.21.15.105:46270) 2023-11-28T18:28:11.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f5084cf2640 1 -- 172.21.15.105:0/3340178689 learned_addr learned my addr 172.21.15.105:0/3340178689 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:11.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f50854f3640 1 --2- 172.21.15.105:0/3340178689 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50800ad850 0x7f5080154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:11.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f5085cf4640 1 --2- 172.21.15.105:0/3340178689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5080159f40 0x7f50801553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:11.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f5084cf2640 1 -- 172.21.15.105:0/3340178689 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50800ad850 msgr2=0x7f5080154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:11.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f5084cf2640 1 --2- 172.21.15.105:0/3340178689 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50800ad850 0x7f5080154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:11.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f5084cf2640 1 -- 172.21.15.105:0/3340178689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5080159f40 msgr2=0x7f50801553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:11.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f5084cf2640 1 --2- 172.21.15.105:0/3340178689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5080159f40 0x7f50801553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:11.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f5084cf2640 1 -- 172.21.15.105:0/3340178689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507c009420 con 0x7f50800cc760 2023-11-28T18:28:11.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f50854f3640 1 --2- 172.21.15.105:0/3340178689 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50800ad850 0x7f5080154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:11.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f5084cf2640 1 --2- 172.21.15.105:0/3340178689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50800cc760 0x7f508014b8f0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f507000c4b0 tx=0x7f507000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:11.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f5075ffb640 1 -- 172.21.15.105:0/3340178689 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50700074e0 con 0x7f50800cc760 2023-11-28T18:28:11.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f5075ffb640 1 -- 172.21.15.105:0/3340178689 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5070007660 con 0x7f50800cc760 2023-11-28T18:28:11.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f5087757640 1 -- 172.21.15.105:0/3340178689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50800b2730 con 0x7f50800cc760 2023-11-28T18:28:11.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f5075ffb640 1 -- 172.21.15.105:0/3340178689 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070004a30 con 0x7f50800cc760 2023-11-28T18:28:11.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f5087757640 1 -- 172.21.15.105:0/3340178689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5080153f30 con 0x7f50800cc760 2023-11-28T18:28:11.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.886+0000 7f5087757640 1 -- 172.21.15.105:0/3340178689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5080067570 con 0x7f50800cc760 2023-11-28T18:28:11.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.890+0000 7f5075ffb640 1 -- 172.21.15.105:0/3340178689 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5070004b90 con 0x7f50800cc760 2023-11-28T18:28:11.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.890+0000 7f5075ffb640 1 --2- 172.21.15.105:0/3340178689 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f50640668d0 0x7f5064068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:11.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.890+0000 7f5075ffb640 1 -- 172.21.15.105:0/3340178689 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f50700838f0 con 0x7f50800cc760 2023-11-28T18:28:11.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.890+0000 7f50854f3640 1 --2- 172.21.15.105:0/3340178689 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f50640668d0 0x7f5064068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:11.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.890+0000 7f50854f3640 1 --2- 172.21.15.105:0/3340178689 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f50640668d0 0x7f5064068d90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f507c0ae000 tx=0x7f507c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:11.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:11.894+0000 7f5075ffb640 1 -- 172.21.15.105:0/3340178689 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f50700545c0 con 0x7f50800cc760 2023-11-28T18:28:12.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.046+0000 7f5087757640 1 -- 172.21.15.105:0/3340178689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f50800cf950 con 0x7f50800cc760 2023-11-28T18:28:12.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.706+0000 7f5075ffb640 1 -- 172.21.15.105:0/3340178689 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v76) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5070018020 con 0x7f50800cc760 2023-11-28T18:28:12.711 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' created 2023-11-28T18:28:12.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.710+0000 7f5087757640 1 -- 172.21.15.105:0/3340178689 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f50640668d0 msgr2=0x7f5064068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:12.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.710+0000 7f5087757640 1 --2- 172.21.15.105:0/3340178689 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f50640668d0 0x7f5064068d90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f507c0ae000 tx=0x7f507c008040 comp rx=0 tx=0).stop 2023-11-28T18:28:12.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.710+0000 7f5087757640 1 -- 172.21.15.105:0/3340178689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50800cc760 msgr2=0x7f508014b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:12.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.710+0000 7f5087757640 1 --2- 172.21.15.105:0/3340178689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50800cc760 0x7f508014b8f0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f507000c4b0 tx=0x7f507000c970 comp rx=0 tx=0).stop 2023-11-28T18:28:12.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.714+0000 7f5087757640 1 -- 172.21.15.105:0/3340178689 shutdown_connections 2023-11-28T18:28:12.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.714+0000 7f5087757640 1 --2- 172.21.15.105:0/3340178689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5080159f40 0x7f50801553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:12.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.714+0000 7f5087757640 1 --2- 172.21.15.105:0/3340178689 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f50640668d0 0x7f5064068d90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:12.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.714+0000 7f5087757640 1 --2- 172.21.15.105:0/3340178689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50800cc760 0x7f508014b8f0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:12.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.714+0000 7f5087757640 1 --2- 172.21.15.105:0/3340178689 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50800ad850 0x7f5080154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:12.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.714+0000 7f5087757640 1 -- 172.21.15.105:0/3340178689 >> 172.21.15.105:0/3340178689 conn(0x7f50800ceb90 msgr2=0x7f50800ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:12.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.714+0000 7f5087757640 1 -- 172.21.15.105:0/3340178689 shutdown_connections 2023-11-28T18:28:12.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.714+0000 7f5087757640 1 -- 172.21.15.105:0/3340178689 wait complete. 2023-11-28T18:28:12.756 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T18:28:12.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.894+0000 7f8f78f9d640 1 Processor -- start 2023-11-28T18:28:12.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.894+0000 7f8f78f9d640 1 -- start start 2023-11-28T18:28:12.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.894+0000 7f8f78f9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8f740caf60 0x7f8f740cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:12.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.894+0000 7f8f78f9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8f740ad850 0x7f8f740701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:12.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.894+0000 7f8f78f9d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f7406bf50 con 0x7f8f74159f40 2023-11-28T18:28:12.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.894+0000 7f8f78f9d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f740cb3c0 con 0x7f8f740ad850 2023-11-28T18:28:12.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.894+0000 7f8f78f9d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f7415a320 con 0x7f8f740caf60 2023-11-28T18:28:12.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.898+0000 7f8f72d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8f740ad850 0x7f8f740701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:12.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.898+0000 7f8f7259c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8f74159f40 0x7f8f740ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47890/0 (socket says 172.21.15.105:47890) 2023-11-28T18:28:12.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.898+0000 7f8f7259c640 1 -- 172.21.15.105:0/357551334 learned_addr learned my addr 172.21.15.105:0/357551334 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:12.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.898+0000 7f8f71d9b640 1 --2- 172.21.15.105:0/357551334 >> v2:172.21.15.148:3300/0 conn(0x7f8f740caf60 0x7f8f740cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:12.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.898+0000 7f8f72d9d640 1 -- 172.21.15.105:0/357551334 >> v2:172.21.15.148:3300/0 conn(0x7f8f740caf60 msgr2=0x7f8f740cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:12.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.898+0000 7f8f72d9d640 1 --2- 172.21.15.105:0/357551334 >> v2:172.21.15.148:3300/0 conn(0x7f8f740caf60 0x7f8f740cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:12.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.898+0000 7f8f72d9d640 1 -- 172.21.15.105:0/357551334 >> v1:172.21.15.105:6789/0 conn(0x7f8f74159f40 legacy=0x7f8f740ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:12.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.898+0000 7f8f7159a640 1 -- 172.21.15.105:0/357551334 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072171516 0 0) 0x7f8f7406bf50 con 0x7f8f74159f40 2023-11-28T18:28:12.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.898+0000 7f8f72d9d640 1 -- 172.21.15.105:0/357551334 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f74071380 con 0x7f8f740ad850 2023-11-28T18:28:12.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.898+0000 7f8f71d9b640 1 --2- 172.21.15.105:0/357551334 >> v2:172.21.15.148:3300/0 conn(0x7f8f740caf60 0x7f8f740cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:12.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.898+0000 7f8f72d9d640 1 --2- 172.21.15.105:0/357551334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f740ad850 0x7f8f740701d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8f7409d0d0 tx=0x7f8f6c01c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e8a0674b56be46 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:12.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.898+0000 7f8f7159a640 1 -- 172.21.15.105:0/357551334 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0a5910 con 0x7f8f740ad850 2023-11-28T18:28:12.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.898+0000 7f8f7159a640 1 -- 172.21.15.105:0/357551334 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6c0a5a70 con 0x7f8f740ad850 2023-11-28T18:28:12.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.898+0000 7f8f7159a640 1 -- 172.21.15.105:0/357551334 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0a5e00 con 0x7f8f740ad850 2023-11-28T18:28:12.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.902+0000 7f8f78f9d640 1 -- 172.21.15.105:0/357551334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f740ad850 msgr2=0x7f8f740701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:12.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.902+0000 7f8f78f9d640 1 --2- 172.21.15.105:0/357551334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f740ad850 0x7f8f740701d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8f7409d0d0 tx=0x7f8f6c01c850 comp rx=0 tx=0).stop 2023-11-28T18:28:12.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.902+0000 7f8f78f9d640 1 -- 172.21.15.105:0/357551334 shutdown_connections 2023-11-28T18:28:12.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.902+0000 7f8f78f9d640 1 --2- 172.21.15.105:0/357551334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f740ad850 0x7f8f740701d0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:12.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.902+0000 7f8f78f9d640 1 --2- 172.21.15.105:0/357551334 >> v2:172.21.15.148:3300/0 conn(0x7f8f740caf60 0x7f8f740cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:12.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.902+0000 7f8f78f9d640 1 -- 172.21.15.105:0/357551334 >> 172.21.15.105:0/357551334 conn(0x7f8f740ceb90 msgr2=0x7f8f740cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:12.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.902+0000 7f8f78f9d640 1 -- 172.21.15.105:0/357551334 shutdown_connections 2023-11-28T18:28:12.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.902+0000 7f8f78f9d640 1 -- 172.21.15.105:0/357551334 wait complete. 2023-11-28T18:28:12.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.902+0000 7f8f78f9d640 1 Processor -- start 2023-11-28T18:28:12.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.902+0000 7f8f78f9d640 1 -- start start 2023-11-28T18:28:12.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.902+0000 7f8f78f9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f740ad850 0x7f8f7414f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:12.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.902+0000 7f8f78f9d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f740caf60 0x7f8f7414cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:12.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.902+0000 7f8f78f9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f74159f40 0x7f8f741564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:12.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.902+0000 7f8f78f9d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7414ed10 con 0x7f8f74159f40 2023-11-28T18:28:12.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.902+0000 7f8f78f9d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7414fb60 con 0x7f8f740ad850 2023-11-28T18:28:12.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.902+0000 7f8f78f9d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f740c80e0 con 0x7f8f740caf60 2023-11-28T18:28:12.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.902+0000 7f8f72d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f74159f40 0x7f8f741564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:12.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.902+0000 7f8f72d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f74159f40 0x7f8f741564b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46284/0 (socket says 172.21.15.105:46284) 2023-11-28T18:28:12.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.902+0000 7f8f72d9d640 1 -- 172.21.15.105:0/2995099763 learned_addr learned my addr 172.21.15.105:0/2995099763 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:12.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.906+0000 7f8f7259c640 1 --2- 172.21.15.105:0/2995099763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f740ad850 0x7f8f7414f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:12.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.906+0000 7f8f71d9b640 1 --2- 172.21.15.105:0/2995099763 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f740caf60 0x7f8f7414cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:12.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.906+0000 7f8f72d9d640 1 -- 172.21.15.105:0/2995099763 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f740caf60 msgr2=0x7f8f7414cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:12.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.906+0000 7f8f72d9d640 1 --2- 172.21.15.105:0/2995099763 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f740caf60 0x7f8f7414cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:12.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.906+0000 7f8f72d9d640 1 -- 172.21.15.105:0/2995099763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f740ad850 msgr2=0x7f8f7414f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:12.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.906+0000 7f8f72d9d640 1 --2- 172.21.15.105:0/2995099763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f740ad850 0x7f8f7414f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:12.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.906+0000 7f8f72d9d640 1 -- 172.21.15.105:0/2995099763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6c009a70 con 0x7f8f74159f40 2023-11-28T18:28:12.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.906+0000 7f8f7259c640 1 --2- 172.21.15.105:0/2995099763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f740ad850 0x7f8f7414f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:12.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.906+0000 7f8f72d9d640 1 --2- 172.21.15.105:0/2995099763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f74159f40 0x7f8f741564b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c019460 tx=0x7f8f6c0195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:12.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.906+0000 7f8f62ffd640 1 -- 172.21.15.105:0/2995099763 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0ad850 con 0x7f8f74159f40 2023-11-28T18:28:12.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.906+0000 7f8f62ffd640 1 -- 172.21.15.105:0/2995099763 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6c0ad9b0 con 0x7f8f74159f40 2023-11-28T18:28:12.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.906+0000 7f8f78f9d640 1 -- 172.21.15.105:0/2995099763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f740c83c0 con 0x7f8f74159f40 2023-11-28T18:28:12.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.906+0000 7f8f62ffd640 1 -- 172.21.15.105:0/2995099763 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0add20 con 0x7f8f74159f40 2023-11-28T18:28:12.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.906+0000 7f8f78f9d640 1 -- 172.21.15.105:0/2995099763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f740c8980 con 0x7f8f74159f40 2023-11-28T18:28:12.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.906+0000 7f8f78f9d640 1 -- 172.21.15.105:0/2995099763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f740c8b70 con 0x7f8f74159f40 2023-11-28T18:28:12.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.906+0000 7f8f62ffd640 1 -- 172.21.15.105:0/2995099763 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8f6c0b4070 con 0x7f8f74159f40 2023-11-28T18:28:12.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.906+0000 7f8f62ffd640 1 --2- 172.21.15.105:0/2995099763 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8f540668d0 0x7f8f54068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:12.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.906+0000 7f8f62ffd640 1 -- 172.21.15.105:0/2995099763 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f8f6c11f100 con 0x7f8f74159f40 2023-11-28T18:28:12.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.910+0000 7f8f7259c640 1 --2- 172.21.15.105:0/2995099763 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8f540668d0 0x7f8f54068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:12.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.910+0000 7f8f7259c640 1 --2- 172.21.15.105:0/2995099763 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8f540668d0 0x7f8f54068d90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8f64000f70 tx=0x7f8f64007cb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:12.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:12.914+0000 7f8f62ffd640 1 -- 172.21.15.105:0/2995099763 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8f6c0ec260 con 0x7f8f74159f40 2023-11-28T18:28:13.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.090+0000 7f8f78f9d640 1 -- 172.21.15.105:0/2995099763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8f74067570 con 0x7f8f74159f40 2023-11-28T18:28:13.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.730+0000 7f8f62ffd640 1 -- 172.21.15.105:0/2995099763 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v67) v1 ==== 174+0+0 (secure 0 0 0) 0x7f8f6c0ef880 con 0x7f8f74159f40 2023-11-28T18:28:13.737 INFO:teuthology.orchestra.run.smithi105.stderr:new fs with metadata pool 11 and data pool 12 2023-11-28T18:28:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.734+0000 7f8f78f9d640 1 -- 172.21.15.105:0/2995099763 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8f540668d0 msgr2=0x7f8f54068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.734+0000 7f8f78f9d640 1 --2- 172.21.15.105:0/2995099763 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8f540668d0 0x7f8f54068d90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8f64000f70 tx=0x7f8f64007cb0 comp rx=0 tx=0).stop 2023-11-28T18:28:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.734+0000 7f8f78f9d640 1 -- 172.21.15.105:0/2995099763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f74159f40 msgr2=0x7f8f741564b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.734+0000 7f8f78f9d640 1 --2- 172.21.15.105:0/2995099763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f74159f40 0x7f8f741564b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c019460 tx=0x7f8f6c0195f0 comp rx=0 tx=0).stop 2023-11-28T18:28:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.734+0000 7f8f78f9d640 1 -- 172.21.15.105:0/2995099763 shutdown_connections 2023-11-28T18:28:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.734+0000 7f8f78f9d640 1 --2- 172.21.15.105:0/2995099763 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8f540668d0 0x7f8f54068d90 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.734+0000 7f8f78f9d640 1 --2- 172.21.15.105:0/2995099763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f74159f40 0x7f8f741564b0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.734+0000 7f8f78f9d640 1 --2- 172.21.15.105:0/2995099763 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f740caf60 0x7f8f7414cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:13.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.734+0000 7f8f78f9d640 1 --2- 172.21.15.105:0/2995099763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f740ad850 0x7f8f7414f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:13.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.734+0000 7f8f78f9d640 1 -- 172.21.15.105:0/2995099763 >> 172.21.15.105:0/2995099763 conn(0x7f8f740ceb90 msgr2=0x7f8f740cd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:13.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.734+0000 7f8f78f9d640 1 -- 172.21.15.105:0/2995099763 shutdown_connections 2023-11-28T18:28:13.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.734+0000 7f8f78f9d640 1 -- 172.21.15.105:0/2995099763 wait complete. 2023-11-28T18:28:13.768 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:28:13.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.902+0000 7f9c00780640 1 Processor -- start 2023-11-28T18:28:13.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.902+0000 7f9c00780640 1 -- start start 2023-11-28T18:28:13.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.902+0000 7f9c00780640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9bf8159f40 0x7f9bf80caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:13.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.902+0000 7f9c00780640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9bf80ad850 0x7f9bf80701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:13.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.902+0000 7f9c00780640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf806bf50 con 0x7f9bf80cc760 2023-11-28T18:28:13.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.902+0000 7f9c00780640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf80ccbc0 con 0x7f9bf8159f40 2023-11-28T18:28:13.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.902+0000 7f9c00780640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf815a320 con 0x7f9bf80ad850 2023-11-28T18:28:13.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.902+0000 7f9bfe51c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9bf8159f40 0x7f9bf80caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:13.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.902+0000 7f9bfe51c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9bf8159f40 0x7f9bf80caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46296/0 (socket says 172.21.15.105:46296) 2023-11-28T18:28:13.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.902+0000 7f9bfe51c640 1 -- 172.21.15.105:0/1360091748 learned_addr learned my addr 172.21.15.105:0/1360091748 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:13.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.906+0000 7f9bfed1d640 1 --2- 172.21.15.105:0/1360091748 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9bf80ad850 0x7f9bf80701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:13.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.906+0000 7f9bfe51c640 1 -- 172.21.15.105:0/1360091748 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9bf80ad850 msgr2=0x7f9bf80701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:13.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.906+0000 7f9bfe51c640 1 --2- 172.21.15.105:0/1360091748 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9bf80ad850 0x7f9bf80701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:13.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.906+0000 7f9bfe51c640 1 -- 172.21.15.105:0/1360091748 >> v1:172.21.15.148:6789/0 conn(0x7f9bf80cc760 legacy=0x7f9bf80ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:13.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.906+0000 7f9bfe51c640 1 -- 172.21.15.105:0/1360091748 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf8071380 con 0x7f9bf8159f40 2023-11-28T18:28:13.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.906+0000 7f9bfe51c640 1 --2- 172.21.15.105:0/1360091748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf8159f40 0x7f9bf80caf60 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009870 tx=0x7f9bf401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e5d643fd1563c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:13.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.906+0000 7f9bfd51a640 1 -- 172.21.15.105:0/1360091748 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf40ab680 con 0x7f9bf8159f40 2023-11-28T18:28:13.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.906+0000 7f9bfd51a640 1 -- 172.21.15.105:0/1360091748 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf40ab7e0 con 0x7f9bf8159f40 2023-11-28T18:28:13.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.906+0000 7f9bfd51a640 1 -- 172.21.15.105:0/1360091748 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf40abb70 con 0x7f9bf8159f40 2023-11-28T18:28:13.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.906+0000 7f9c00780640 1 -- 172.21.15.105:0/1360091748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf8159f40 msgr2=0x7f9bf80caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:13.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.906+0000 7f9c00780640 1 --2- 172.21.15.105:0/1360091748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf8159f40 0x7f9bf80caf60 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009870 tx=0x7f9bf401c480 comp rx=0 tx=0).stop 2023-11-28T18:28:13.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9c00780640 1 -- 172.21.15.105:0/1360091748 shutdown_connections 2023-11-28T18:28:13.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9c00780640 1 --2- 172.21.15.105:0/1360091748 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9bf80ad850 0x7f9bf80701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9c00780640 1 --2- 172.21.15.105:0/1360091748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf8159f40 0x7f9bf80caf60 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9c00780640 1 -- 172.21.15.105:0/1360091748 >> 172.21.15.105:0/1360091748 conn(0x7f9bf80ceb90 msgr2=0x7f9bf80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9c00780640 1 -- 172.21.15.105:0/1360091748 shutdown_connections 2023-11-28T18:28:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9c00780640 1 -- 172.21.15.105:0/1360091748 wait complete. 2023-11-28T18:28:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9c00780640 1 Processor -- start 2023-11-28T18:28:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9c00780640 1 -- start start 2023-11-28T18:28:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9c00780640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9bf80ad850 0x7f9bf8148340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9c00780640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bf80cc760 0x7f9bf8148880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9c00780640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf8159f40 0x7f9bf80b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9c00780640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf8148dc0 con 0x7f9bf8159f40 2023-11-28T18:28:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9c00780640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80b2b70 con 0x7f9bf80cc760 2023-11-28T18:28:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9c00780640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80b2e50 con 0x7f9bf80ad850 2023-11-28T18:28:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9bfed1d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf8159f40 0x7f9bf80b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9bfed1d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf8159f40 0x7f9bf80b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46298/0 (socket says 172.21.15.105:46298) 2023-11-28T18:28:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9bfed1d640 1 -- 172.21.15.105:0/1517713928 learned_addr learned my addr 172.21.15.105:0/1517713928 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9bfe51c640 1 --2- 172.21.15.105:0/1517713928 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9bf80ad850 0x7f9bf8148340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9bfdd1b640 1 --2- 172.21.15.105:0/1517713928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bf80cc760 0x7f9bf8148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9bfed1d640 1 -- 172.21.15.105:0/1517713928 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9bf80ad850 msgr2=0x7f9bf8148340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9bfed1d640 1 --2- 172.21.15.105:0/1517713928 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9bf80ad850 0x7f9bf8148340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9bfed1d640 1 -- 172.21.15.105:0/1517713928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bf80cc760 msgr2=0x7f9bf8148880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9bfed1d640 1 --2- 172.21.15.105:0/1517713928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bf80cc760 0x7f9bf8148880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9bfed1d640 1 -- 172.21.15.105:0/1517713928 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf40094d0 con 0x7f9bf8159f40 2023-11-28T18:28:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9bfdd1b640 1 --2- 172.21.15.105:0/1517713928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bf80cc760 0x7f9bf8148880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.910+0000 7f9bfed1d640 1 --2- 172.21.15.105:0/1517713928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf8159f40 0x7f9bf80b1c10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0007e40 tx=0x7f9bf000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.914+0000 7f9beeffd640 1 -- 172.21.15.105:0/1517713928 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0014620 con 0x7f9bf8159f40 2023-11-28T18:28:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.914+0000 7f9beeffd640 1 -- 172.21.15.105:0/1517713928 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf000ee90 con 0x7f9bf8159f40 2023-11-28T18:28:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.914+0000 7f9beeffd640 1 -- 172.21.15.105:0/1517713928 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00149e0 con 0x7f9bf8159f40 2023-11-28T18:28:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.914+0000 7f9c00780640 1 -- 172.21.15.105:0/1517713928 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf80b3130 con 0x7f9bf8159f40 2023-11-28T18:28:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.914+0000 7f9c00780640 1 -- 172.21.15.105:0/1517713928 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf815af80 con 0x7f9bf8159f40 2023-11-28T18:28:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.914+0000 7f9c00780640 1 -- 172.21.15.105:0/1517713928 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf815b1a0 con 0x7f9bf8159f40 2023-11-28T18:28:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.914+0000 7f9beeffd640 1 -- 172.21.15.105:0/1517713928 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f9bf0014b40 con 0x7f9bf8159f40 2023-11-28T18:28:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.914+0000 7f9beeffd640 1 --2- 172.21.15.105:0/1517713928 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9bdc0668d0 0x7f9bdc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.914+0000 7f9beeffd640 1 -- 172.21.15.105:0/1517713928 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9bf0084e20 con 0x7f9bf8159f40 2023-11-28T18:28:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.914+0000 7f9bfe51c640 1 --2- 172.21.15.105:0/1517713928 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9bdc0668d0 0x7f9bdc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:13.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.914+0000 7f9bfe51c640 1 --2- 172.21.15.105:0/1517713928 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9bdc0668d0 0x7f9bdc068d90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40af000 tx=0x7f9bf40b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:13.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:13.918+0000 7f9beeffd640 1 -- 172.21.15.105:0/1517713928 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9bf0051f20 con 0x7f9bf8159f40 2023-11-28T18:28:14.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.066+0000 7f9c00780640 1 -- 172.21.15.105:0/1517713928 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bf8067570 con 0x7f9bf8159f40 2023-11-28T18:28:14.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.070+0000 7f9beeffd640 1 -- 172.21.15.105:0/1517713928 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16850 (secure 0 0 0) 0x7f9bf0055540 con 0x7f9bf8159f40 2023-11-28T18:28:14.074 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:14.074 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":77,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:28:13.097183+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:28:11.454373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-28T18:28:12.093171+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:28:14.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.070+0000 7f9c00780640 1 -- 172.21.15.105:0/1517713928 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9bdc0668d0 msgr2=0x7f9bdc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:14.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.070+0000 7f9c00780640 1 --2- 172.21.15.105:0/1517713928 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9bdc0668d0 0x7f9bdc068d90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40af000 tx=0x7f9bf40b6040 comp rx=0 tx=0).stop 2023-11-28T18:28:14.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.070+0000 7f9c00780640 1 -- 172.21.15.105:0/1517713928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf8159f40 msgr2=0x7f9bf80b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:14.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.070+0000 7f9c00780640 1 --2- 172.21.15.105:0/1517713928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf8159f40 0x7f9bf80b1c10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0007e40 tx=0x7f9bf000e4b0 comp rx=0 tx=0).stop 2023-11-28T18:28:14.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.070+0000 7f9c00780640 1 -- 172.21.15.105:0/1517713928 shutdown_connections 2023-11-28T18:28:14.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.070+0000 7f9c00780640 1 --2- 172.21.15.105:0/1517713928 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9bdc0668d0 0x7f9bdc068d90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:14.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.070+0000 7f9c00780640 1 --2- 172.21.15.105:0/1517713928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf8159f40 0x7f9bf80b1c10 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:14.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.070+0000 7f9c00780640 1 --2- 172.21.15.105:0/1517713928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bf80cc760 0x7f9bf8148880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:14.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.070+0000 7f9c00780640 1 --2- 172.21.15.105:0/1517713928 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9bf80ad850 0x7f9bf8148340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:14.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.070+0000 7f9c00780640 1 -- 172.21.15.105:0/1517713928 >> 172.21.15.105:0/1517713928 conn(0x7f9bf80ceb90 msgr2=0x7f9bf806c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:14.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.070+0000 7f9c00780640 1 -- 172.21.15.105:0/1517713928 shutdown_connections 2023-11-28T18:28:14.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.070+0000 7f9c00780640 1 -- 172.21.15.105:0/1517713928 wait complete. 2023-11-28T18:28:14.096 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:28:14.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.230+0000 7fdec72bb640 1 Processor -- start 2023-11-28T18:28:14.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.230+0000 7fdec72bb640 1 -- start start 2023-11-28T18:28:14.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.234+0000 7fdec72bb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdec0070a10 0x7fdec0070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:14.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.234+0000 7fdec72bb640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec00cefa0 con 0x7fdec015d3c0 2023-11-28T18:28:14.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.234+0000 7fdec72bb640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec0071360 con 0x7fdec0071a10 2023-11-28T18:28:14.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.234+0000 7fdec72bb640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec015d7a0 con 0x7fdec0070a10 2023-11-28T18:28:14.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.234+0000 7fdec5858640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdec0070a10 0x7fdec0070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:14.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.234+0000 7fdec5057640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fdec015d3c0 0x7fdec0161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47898/0 (socket says 172.21.15.105:47898) 2023-11-28T18:28:14.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.234+0000 7fdec5057640 1 -- 172.21.15.105:0/3693723009 learned_addr learned my addr 172.21.15.105:0/3693723009 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:14.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.234+0000 7fdec5858640 1 -- 172.21.15.105:0/3693723009 >> v1:172.21.15.148:6789/0 conn(0x7fdec0071a10 legacy=0x7fdec00ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:14.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.234+0000 7fdec5858640 1 -- 172.21.15.105:0/3693723009 >> v1:172.21.15.105:6789/0 conn(0x7fdec015d3c0 legacy=0x7fdec0161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:14.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.234+0000 7fdec5858640 1 -- 172.21.15.105:0/3693723009 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec00ce9e0 con 0x7fdec0070a10 2023-11-28T18:28:14.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.238+0000 7fdec5858640 1 --2- 172.21.15.105:0/3693723009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec0070a10 0x7fdec0070e20 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009d90 tx=0x7fdeb801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf4cc9363756c26 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:14.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.238+0000 7fdeb7fff640 1 -- 172.21.15.105:0/3693723009 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb80a5910 con 0x7fdec0070a10 2023-11-28T18:28:14.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.238+0000 7fdeb7fff640 1 -- 172.21.15.105:0/3693723009 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb80a5a70 con 0x7fdec0070a10 2023-11-28T18:28:14.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.238+0000 7fdeb7fff640 1 -- 172.21.15.105:0/3693723009 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb80a5e00 con 0x7fdec0070a10 2023-11-28T18:28:14.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.238+0000 7fdec72bb640 1 -- 172.21.15.105:0/3693723009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec0070a10 msgr2=0x7fdec0070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:14.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.238+0000 7fdec72bb640 1 --2- 172.21.15.105:0/3693723009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec0070a10 0x7fdec0070e20 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009d90 tx=0x7fdeb801c850 comp rx=0 tx=0).stop 2023-11-28T18:28:14.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.238+0000 7fdec72bb640 1 -- 172.21.15.105:0/3693723009 shutdown_connections 2023-11-28T18:28:14.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.238+0000 7fdec72bb640 1 --2- 172.21.15.105:0/3693723009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec0070a10 0x7fdec0070e20 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:14.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.238+0000 7fdec72bb640 1 -- 172.21.15.105:0/3693723009 >> 172.21.15.105:0/3693723009 conn(0x7fdec00cf540 msgr2=0x7fdec00cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:14.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.238+0000 7fdec72bb640 1 -- 172.21.15.105:0/3693723009 shutdown_connections 2023-11-28T18:28:14.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.238+0000 7fdec72bb640 1 -- 172.21.15.105:0/3693723009 wait complete. 2023-11-28T18:28:14.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.238+0000 7fdec72bb640 1 Processor -- start 2023-11-28T18:28:14.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.238+0000 7fdec72bb640 1 -- start start 2023-11-28T18:28:14.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdec72bb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdec0070a10 0x7fdec014a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:14.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdec72bb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec0071a10 0x7fdec0148b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:14.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdec72bb640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdec015d3c0 0x7fdec00ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:14.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdec72bb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec014b490 con 0x7fdec0071a10 2023-11-28T18:28:14.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdec72bb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec015d9f0 con 0x7fdec0070a10 2023-11-28T18:28:14.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdec72bb640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec00b36e0 con 0x7fdec015d3c0 2023-11-28T18:28:14.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdec4856640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec0071a10 0x7fdec0148b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:14.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdec4856640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec0071a10 0x7fdec0148b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46316/0 (socket says 172.21.15.105:46316) 2023-11-28T18:28:14.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdec4856640 1 -- 172.21.15.105:0/6429736 learned_addr learned my addr 172.21.15.105:0/6429736 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:14.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdec5057640 1 --2- 172.21.15.105:0/6429736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdec0070a10 0x7fdec014a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:14.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdec5858640 1 --2- 172.21.15.105:0/6429736 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdec015d3c0 0x7fdec00ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:14.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdec4856640 1 -- 172.21.15.105:0/6429736 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdec015d3c0 msgr2=0x7fdec00ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:14.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdec4856640 1 --2- 172.21.15.105:0/6429736 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdec015d3c0 0x7fdec00ad570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:14.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdec4856640 1 -- 172.21.15.105:0/6429736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdec0070a10 msgr2=0x7fdec014a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:14.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdec4856640 1 --2- 172.21.15.105:0/6429736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdec0070a10 0x7fdec014a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:14.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdec4856640 1 -- 172.21.15.105:0/6429736 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb8009a70 con 0x7fdec0071a10 2023-11-28T18:28:14.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdec5858640 1 --2- 172.21.15.105:0/6429736 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdec015d3c0 0x7fdec00ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:14.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdec4856640 1 --2- 172.21.15.105:0/6429736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec0071a10 0x7fdec0148b60 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000c450 tx=0x7fdeb000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:14.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdeb5ffb640 1 -- 172.21.15.105:0/6429736 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0012830 con 0x7fdec0071a10 2023-11-28T18:28:14.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdeb5ffb640 1 -- 172.21.15.105:0/6429736 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb00129b0 con 0x7fdec0071a10 2023-11-28T18:28:14.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdec72bb640 1 -- 172.21.15.105:0/6429736 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec014aeb0 con 0x7fdec0071a10 2023-11-28T18:28:14.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdeb5ffb640 1 -- 172.21.15.105:0/6429736 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0012d60 con 0x7fdec0071a10 2023-11-28T18:28:14.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdec72bb640 1 -- 172.21.15.105:0/6429736 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec00b2ad0 con 0x7fdec0071a10 2023-11-28T18:28:14.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdec72bb640 1 -- 172.21.15.105:0/6429736 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec00cdb30 con 0x7fdec0071a10 2023-11-28T18:28:14.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.242+0000 7fdeb5ffb640 1 -- 172.21.15.105:0/6429736 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fdeb001c460 con 0x7fdec0071a10 2023-11-28T18:28:14.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.246+0000 7fdeb5ffb640 1 --2- 172.21.15.105:0/6429736 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdea40668d0 0x7fdea4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:14.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.246+0000 7fdeb5ffb640 1 -- 172.21.15.105:0/6429736 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdeb0084470 con 0x7fdec0071a10 2023-11-28T18:28:14.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.246+0000 7fdec5057640 1 --2- 172.21.15.105:0/6429736 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdea40668d0 0x7fdea4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:14.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.246+0000 7fdec5057640 1 --2- 172.21.15.105:0/6429736 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdea40668d0 0x7fdea4068d90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdebc000f70 tx=0x7fdebc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:14.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.250+0000 7fdeb5ffb640 1 -- 172.21.15.105:0/6429736 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdeb00514d0 con 0x7fdec0071a10 2023-11-28T18:28:14.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.398+0000 7fdec72bb640 1 -- 172.21.15.105:0/6429736 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdec00b3850 con 0x7fdec0071a10 2023-11-28T18:28:14.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.398+0000 7fdeb5ffb640 1 -- 172.21.15.105:0/6429736 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16837 (secure 0 0 0) 0x7fdeb0054af0 con 0x7fdec0071a10 2023-11-28T18:28:14.406 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:14.406 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":78,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:28:14.101062+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:28:11.454373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-28T18:28:12.093171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:28:14.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.402+0000 7fdec72bb640 1 -- 172.21.15.105:0/6429736 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdea40668d0 msgr2=0x7fdea4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:14.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.402+0000 7fdec72bb640 1 --2- 172.21.15.105:0/6429736 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdea40668d0 0x7fdea4068d90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdebc000f70 tx=0x7fdebc009040 comp rx=0 tx=0).stop 2023-11-28T18:28:14.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.402+0000 7fdec72bb640 1 -- 172.21.15.105:0/6429736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec0071a10 msgr2=0x7fdec0148b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:14.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.402+0000 7fdec72bb640 1 --2- 172.21.15.105:0/6429736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec0071a10 0x7fdec0148b60 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000c450 tx=0x7fdeb000c910 comp rx=0 tx=0).stop 2023-11-28T18:28:14.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.402+0000 7fdec72bb640 1 -- 172.21.15.105:0/6429736 shutdown_connections 2023-11-28T18:28:14.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.402+0000 7fdec72bb640 1 --2- 172.21.15.105:0/6429736 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdec015d3c0 0x7fdec00ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:14.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.402+0000 7fdec72bb640 1 --2- 172.21.15.105:0/6429736 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdea40668d0 0x7fdea4068d90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:14.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.402+0000 7fdec72bb640 1 --2- 172.21.15.105:0/6429736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec0071a10 0x7fdec0148b60 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:14.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.402+0000 7fdec72bb640 1 --2- 172.21.15.105:0/6429736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdec0070a10 0x7fdec014a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:14.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.402+0000 7fdec72bb640 1 -- 172.21.15.105:0/6429736 >> 172.21.15.105:0/6429736 conn(0x7fdec00cf540 msgr2=0x7fdec00b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:14.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.402+0000 7fdec72bb640 1 -- 172.21.15.105:0/6429736 shutdown_connections 2023-11-28T18:28:14.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.402+0000 7fdec72bb640 1 -- 172.21.15.105:0/6429736 wait complete. 2023-11-28T18:28:14.425 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T18:28:14.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f01a54ea640 1 Processor -- start 2023-11-28T18:28:14.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f01a54ea640 1 -- start start 2023-11-28T18:28:14.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f01a54ea640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f01a0159f40 0x7f01a00caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:14.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f01a54ea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01a00cc760 0x7f01a00ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:14.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f01a54ea640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a006bf50 con 0x7f01a00ad680 2023-11-28T18:28:14.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f01a54ea640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a00ccbc0 con 0x7f01a0159f40 2023-11-28T18:28:14.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f01a54ea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a015a320 con 0x7f01a00cc760 2023-11-28T18:28:14.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f019f7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f01a00ad680 0x7f01a015da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47912/0 (socket says 172.21.15.105:47912) 2023-11-28T18:28:14.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f019f7fe640 1 -- 172.21.15.105:0/4103642965 learned_addr learned my addr 172.21.15.105:0/4103642965 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:14.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f0197fff640 1 --2- 172.21.15.105:0/4103642965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01a00cc760 0x7f01a00ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:14.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f019effd640 1 --2- 172.21.15.105:0/4103642965 >> v2:172.21.15.148:3300/0 conn(0x7f01a0159f40 0x7f01a00caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:14.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f019e7fc640 1 -- 172.21.15.105:0/4103642965 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 693636495 0 0) 0x7f01a006bf50 con 0x7f01a00ad680 2023-11-28T18:28:14.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f019e7fc640 1 -- 172.21.15.105:0/4103642965 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f018c003530 con 0x7f01a00ad680 2023-11-28T18:28:14.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f019e7fc640 1 -- 172.21.15.105:0/4103642965 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 694130938 0 0) 0x7f018c003530 con 0x7f01a00ad680 2023-11-28T18:28:14.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f019e7fc640 1 -- 172.21.15.105:0/4103642965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01a00cc760 msgr2=0x7f01a00ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:14.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f019e7fc640 1 --2- 172.21.15.105:0/4103642965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01a00cc760 0x7f01a00ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:14.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f019e7fc640 1 -- 172.21.15.105:0/4103642965 >> v2:172.21.15.148:3300/0 conn(0x7f01a0159f40 msgr2=0x7f01a00caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:14.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f019e7fc640 1 --2- 172.21.15.105:0/4103642965 >> v2:172.21.15.148:3300/0 conn(0x7f01a0159f40 0x7f01a00caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:14.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f019e7fc640 1 -- 172.21.15.105:0/4103642965 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a0071380 con 0x7f01a00ad680 2023-11-28T18:28:14.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f019e7fc640 1 -- 172.21.15.105:0/4103642965 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f0198001a00 con 0x7f01a00ad680 2023-11-28T18:28:14.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f019e7fc640 1 -- 172.21.15.105:0/4103642965 >> v1:172.21.15.105:6789/0 conn(0x7f01a00ad680 legacy=0x7f01a015da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:14.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f019e7fc640 1 --2- 172.21.15.105:0/4103642965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f018c003980 0x7f018c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:14.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f019e7fc640 1 --2- 172.21.15.105:0/4103642965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f018c0047d0 0x7f018c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:14.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f019e7fc640 1 --2- 172.21.15.105:0/4103642965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01a00cc760 0x7f018c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:14.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f019e7fc640 1 -- 172.21.15.105:0/4103642965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a00ccbc0 con 0x7f018c0047d0 2023-11-28T18:28:14.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f019e7fc640 1 -- 172.21.15.105:0/4103642965 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a015a320 con 0x7f018c003980 2023-11-28T18:28:14.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f019e7fc640 1 -- 172.21.15.105:0/4103642965 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f018c00dba0 con 0x7f01a00cc760 2023-11-28T18:28:14.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f0197fff640 1 --2- 172.21.15.105:0/4103642965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f018c0047d0 0x7f018c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:14.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f019effd640 1 --2- 172.21.15.105:0/4103642965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f018c003980 0x7f018c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:14.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f019f7fe640 1 --2- 172.21.15.105:0/4103642965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01a00cc760 0x7f018c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:14.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f0197fff640 1 -- 172.21.15.105:0/4103642965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01a00cc760 msgr2=0x7f018c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:14.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f0197fff640 1 --2- 172.21.15.105:0/4103642965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01a00cc760 0x7f018c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:14.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f0197fff640 1 -- 172.21.15.105:0/4103642965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f018c003980 msgr2=0x7f018c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:14.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f0197fff640 1 --2- 172.21.15.105:0/4103642965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f018c003980 0x7f018c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:14.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.562+0000 7f0197fff640 1 -- 172.21.15.105:0/4103642965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018c003530 con 0x7f018c0047d0 2023-11-28T18:28:14.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.566+0000 7f0197fff640 1 --2- 172.21.15.105:0/4103642965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f018c0047d0 0x7f018c004c50 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f0188009b10 tx=0x7f018801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d22c9589951c76 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:14.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.566+0000 7f019e7fc640 1 -- 172.21.15.105:0/4103642965 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01880a5510 con 0x7f018c0047d0 2023-11-28T18:28:14.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.566+0000 7f019e7fc640 1 -- 172.21.15.105:0/4103642965 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01880a5670 con 0x7f018c0047d0 2023-11-28T18:28:14.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.566+0000 7f019e7fc640 1 -- 172.21.15.105:0/4103642965 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01880a5a20 con 0x7f018c0047d0 2023-11-28T18:28:14.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.566+0000 7f01a54ea640 1 -- 172.21.15.105:0/4103642965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f018c0047d0 msgr2=0x7f018c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:14.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.566+0000 7f01a54ea640 1 --2- 172.21.15.105:0/4103642965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f018c0047d0 0x7f018c004c50 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f0188009b10 tx=0x7f018801ca20 comp rx=0 tx=0).stop 2023-11-28T18:28:14.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.566+0000 7f019f7fe640 1 -- 172.21.15.105:0/4103642965 reap_dead start 2023-11-28T18:28:14.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.566+0000 7f01a54ea640 1 -- 172.21.15.105:0/4103642965 shutdown_connections 2023-11-28T18:28:14.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.566+0000 7f01a54ea640 1 -- 172.21.15.105:0/4103642965 >> 172.21.15.105:0/4103642965 conn(0x7f01a00ceb90 msgr2=0x7f01a00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:14.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.566+0000 7f01a54ea640 1 -- 172.21.15.105:0/4103642965 shutdown_connections 2023-11-28T18:28:14.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.566+0000 7f01a54ea640 1 -- 172.21.15.105:0/4103642965 wait complete. 2023-11-28T18:28:14.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f01a54ea640 1 Processor -- start 2023-11-28T18:28:14.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f01a54ea640 1 -- start start 2023-11-28T18:28:14.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f01a54ea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f018c0047d0 0x7f01a0155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:14.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f01a54ea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01a0158c70 0x7f01a015a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:14.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f01a54ea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01a01542a0 0x7f01a00af770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:14.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f01a54ea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a01546f0 con 0x7f01a01542a0 2023-11-28T18:28:14.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f01a54ea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a015b2d0 con 0x7f018c0047d0 2023-11-28T18:28:14.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f01a54ea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a0154940 con 0x7f01a0158c70 2023-11-28T18:28:14.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f019f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01a01542a0 0x7f01a00af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:14.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f019f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01a01542a0 0x7f01a00af770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46336/0 (socket says 172.21.15.105:46336) 2023-11-28T18:28:14.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f019f7fe640 1 -- 172.21.15.105:0/3120227610 learned_addr learned my addr 172.21.15.105:0/3120227610 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:14.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f019effd640 1 --2- 172.21.15.105:0/3120227610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f018c0047d0 0x7f01a0155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:14.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f0197fff640 1 --2- 172.21.15.105:0/3120227610 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01a0158c70 0x7f01a015a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:14.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f019f7fe640 1 -- 172.21.15.105:0/3120227610 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01a0158c70 msgr2=0x7f01a015a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:14.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f019f7fe640 1 --2- 172.21.15.105:0/3120227610 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01a0158c70 0x7f01a015a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:14.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f019f7fe640 1 -- 172.21.15.105:0/3120227610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f018c0047d0 msgr2=0x7f01a0155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:14.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f019f7fe640 1 --2- 172.21.15.105:0/3120227610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f018c0047d0 0x7f01a0155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:14.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f019f7fe640 1 -- 172.21.15.105:0/3120227610 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a0154c20 con 0x7f01a01542a0 2023-11-28T18:28:14.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f019f7fe640 1 --2- 172.21.15.105:0/3120227610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01a01542a0 0x7f01a00af770 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f01a00b4e80 tx=0x7f0198007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:14.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f01977fe640 1 -- 172.21.15.105:0/3120227610 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019800ac80 con 0x7f01a01542a0 2023-11-28T18:28:14.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f01977fe640 1 -- 172.21.15.105:0/3120227610 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019800ae20 con 0x7f01a01542a0 2023-11-28T18:28:14.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f01977fe640 1 -- 172.21.15.105:0/3120227610 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0198013630 con 0x7f01a01542a0 2023-11-28T18:28:14.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f01a54ea640 1 -- 172.21.15.105:0/3120227610 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a0154f00 con 0x7f01a01542a0 2023-11-28T18:28:14.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.570+0000 7f01a54ea640 1 -- 172.21.15.105:0/3120227610 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a015aee0 con 0x7f01a01542a0 2023-11-28T18:28:14.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.574+0000 7f01a54ea640 1 -- 172.21.15.105:0/3120227610 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a01550f0 con 0x7f01a01542a0 2023-11-28T18:28:14.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.574+0000 7f01977fe640 1 -- 172.21.15.105:0/3120227610 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f0198013790 con 0x7f01a01542a0 2023-11-28T18:28:14.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.574+0000 7f01977fe640 1 --2- 172.21.15.105:0/3120227610 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f01800668d0 0x7f0180068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:14.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.574+0000 7f01977fe640 1 -- 172.21.15.105:0/3120227610 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0198084ff0 con 0x7f01a01542a0 2023-11-28T18:28:14.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.574+0000 7f019effd640 1 --2- 172.21.15.105:0/3120227610 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f01800668d0 0x7f0180068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:14.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.574+0000 7f019effd640 1 --2- 172.21.15.105:0/3120227610 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f01800668d0 0x7f0180068d90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0190009be0 tx=0x7f0190009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:14.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.578+0000 7f01977fe640 1 -- 172.21.15.105:0/3120227610 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f01980b6280 con 0x7f01a01542a0 2023-11-28T18:28:14.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:14.766+0000 7f01a54ea640 1 -- 172.21.15.105:0/3120227610 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f01a0067570 con 0x7f01a01542a0 2023-11-28T18:28:15.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.754+0000 7f01977fe640 1 -- 172.21.15.105:0/3120227610 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v70) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0198052100 con 0x7f01a01542a0 2023-11-28T18:28:15.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.758+0000 7f01a54ea640 1 -- 172.21.15.105:0/3120227610 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f01800668d0 msgr2=0x7f0180068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:15.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.758+0000 7f01a54ea640 1 --2- 172.21.15.105:0/3120227610 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f01800668d0 0x7f0180068d90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0190009be0 tx=0x7f0190009190 comp rx=0 tx=0).stop 2023-11-28T18:28:15.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.758+0000 7f01a54ea640 1 -- 172.21.15.105:0/3120227610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01a01542a0 msgr2=0x7f01a00af770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:15.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.758+0000 7f01a54ea640 1 --2- 172.21.15.105:0/3120227610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01a01542a0 0x7f01a00af770 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f01a00b4e80 tx=0x7f0198007b10 comp rx=0 tx=0).stop 2023-11-28T18:28:15.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.758+0000 7f01a54ea640 1 -- 172.21.15.105:0/3120227610 shutdown_connections 2023-11-28T18:28:15.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.758+0000 7f01a54ea640 1 --2- 172.21.15.105:0/3120227610 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f01800668d0 0x7f0180068d90 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:15.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.758+0000 7f01a54ea640 1 --2- 172.21.15.105:0/3120227610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01a01542a0 0x7f01a00af770 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:15.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.758+0000 7f01a54ea640 1 --2- 172.21.15.105:0/3120227610 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01a0158c70 0x7f01a015a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:15.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.758+0000 7f01a54ea640 1 --2- 172.21.15.105:0/3120227610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f018c0047d0 0x7f01a0155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:15.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.758+0000 7f01a54ea640 1 -- 172.21.15.105:0/3120227610 >> 172.21.15.105:0/3120227610 conn(0x7f01a00ceb90 msgr2=0x7f01a0073160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:15.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.758+0000 7f01a54ea640 1 -- 172.21.15.105:0/3120227610 shutdown_connections 2023-11-28T18:28:15.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.758+0000 7f01a54ea640 1 -- 172.21.15.105:0/3120227610 wait complete. 2023-11-28T18:28:15.785 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T18:28:15.785 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T18:28:15.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.922+0000 7fc983baa640 1 Processor -- start 2023-11-28T18:28:15.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.926+0000 7fc983baa640 1 -- start start 2023-11-28T18:28:15.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.926+0000 7fc983baa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc97c159f40 0x7fc97c0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:15.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.926+0000 7fc983baa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc97c0ad850 0x7fc97c0701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:15.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.926+0000 7fc983baa640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc97c06bf50 con 0x7fc97c0cc760 2023-11-28T18:28:15.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.926+0000 7fc983baa640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc97c0ccbc0 con 0x7fc97c159f40 2023-11-28T18:28:15.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.926+0000 7fc983baa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c15a320 con 0x7fc97c0ad850 2023-11-28T18:28:15.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.926+0000 7fc981946640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc97c159f40 0x7fc97c0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:15.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.926+0000 7fc981145640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc97c0cc760 0x7fc97c0ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47914/0 (socket says 172.21.15.105:47914) 2023-11-28T18:28:15.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.926+0000 7fc981145640 1 -- 172.21.15.105:0/85248323 learned_addr learned my addr 172.21.15.105:0/85248323 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:15.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.926+0000 7fc982147640 1 --2- 172.21.15.105:0/85248323 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc97c0ad850 0x7fc97c0701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:15.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.926+0000 7fc981946640 1 -- 172.21.15.105:0/85248323 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc97c0ad850 msgr2=0x7fc97c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:15.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.926+0000 7fc981946640 1 --2- 172.21.15.105:0/85248323 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc97c0ad850 0x7fc97c0701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:15.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.926+0000 7fc981946640 1 -- 172.21.15.105:0/85248323 >> v1:172.21.15.105:6789/0 conn(0x7fc97c0cc760 legacy=0x7fc97c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:15.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.926+0000 7fc981946640 1 -- 172.21.15.105:0/85248323 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c071380 con 0x7fc97c159f40 2023-11-28T18:28:15.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.926+0000 7fc980944640 1 -- 172.21.15.105:0/85248323 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025464130 0 0) 0x7fc97c06bf50 con 0x7fc97c0cc760 2023-11-28T18:28:15.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.930+0000 7fc981946640 1 --2- 172.21.15.105:0/85248323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc97c159f40 0x7fc97c0caf60 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fc9780097c0 tx=0x7fc97801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71d8c726d6cefcd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:15.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.930+0000 7fc980944640 1 -- 172.21.15.105:0/85248323 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9780ac620 con 0x7fc97c159f40 2023-11-28T18:28:15.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.930+0000 7fc980944640 1 -- 172.21.15.105:0/85248323 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9780ac780 con 0x7fc97c159f40 2023-11-28T18:28:15.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.930+0000 7fc980944640 1 -- 172.21.15.105:0/85248323 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9780acb10 con 0x7fc97c159f40 2023-11-28T18:28:15.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.930+0000 7fc983baa640 1 -- 172.21.15.105:0/85248323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc97c159f40 msgr2=0x7fc97c0caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:15.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.930+0000 7fc983baa640 1 --2- 172.21.15.105:0/85248323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc97c159f40 0x7fc97c0caf60 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fc9780097c0 tx=0x7fc97801c3c0 comp rx=0 tx=0).stop 2023-11-28T18:28:15.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.930+0000 7fc983baa640 1 -- 172.21.15.105:0/85248323 shutdown_connections 2023-11-28T18:28:15.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.930+0000 7fc983baa640 1 --2- 172.21.15.105:0/85248323 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc97c0ad850 0x7fc97c0701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:15.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.930+0000 7fc983baa640 1 --2- 172.21.15.105:0/85248323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc97c159f40 0x7fc97c0caf60 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:15.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.930+0000 7fc983baa640 1 -- 172.21.15.105:0/85248323 >> 172.21.15.105:0/85248323 conn(0x7fc97c0ceb90 msgr2=0x7fc97c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:15.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.930+0000 7fc983baa640 1 -- 172.21.15.105:0/85248323 shutdown_connections 2023-11-28T18:28:15.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.930+0000 7fc983baa640 1 -- 172.21.15.105:0/85248323 wait complete. 2023-11-28T18:28:15.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.930+0000 7fc983baa640 1 Processor -- start 2023-11-28T18:28:15.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.930+0000 7fc983baa640 1 -- start start 2023-11-28T18:28:15.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc983baa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc97c0ad850 0x7fc97c1548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:15.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc983baa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc97c0cc760 0x7fc97c14ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:15.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc983baa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc97c159f40 0x7fc97c0af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:15.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc983baa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c09a2b0 con 0x7fc97c0ad850 2023-11-28T18:28:15.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc983baa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c099a50 con 0x7fc97c159f40 2023-11-28T18:28:15.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc983baa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c099cf0 con 0x7fc97c0cc760 2023-11-28T18:28:15.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc981946640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc97c0ad850 0x7fc97c1548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:15.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc981946640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc97c0ad850 0x7fc97c1548a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46364/0 (socket says 172.21.15.105:46364) 2023-11-28T18:28:15.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc981946640 1 -- 172.21.15.105:0/71660097 learned_addr learned my addr 172.21.15.105:0/71660097 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:15.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc981145640 1 --2- 172.21.15.105:0/71660097 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc97c0cc760 0x7fc97c14ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:15.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc982147640 1 --2- 172.21.15.105:0/71660097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc97c159f40 0x7fc97c0af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:15.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc981946640 1 -- 172.21.15.105:0/71660097 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc97c0cc760 msgr2=0x7fc97c14ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:15.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc981946640 1 --2- 172.21.15.105:0/71660097 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc97c0cc760 0x7fc97c14ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:15.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc981946640 1 -- 172.21.15.105:0/71660097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc97c159f40 msgr2=0x7fc97c0af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:15.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc981946640 1 --2- 172.21.15.105:0/71660097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc97c159f40 0x7fc97c0af700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:15.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc981946640 1 -- 172.21.15.105:0/71660097 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc978009420 con 0x7fc97c0ad850 2023-11-28T18:28:15.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc982147640 1 --2- 172.21.15.105:0/71660097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc97c159f40 0x7fc97c0af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:15.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc981946640 1 --2- 172.21.15.105:0/71660097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc97c0ad850 0x7fc97c1548a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fc978019ff0 tx=0x7fc978019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:15.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc9727fc640 1 -- 172.21.15.105:0/71660097 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9780b5410 con 0x7fc97c0ad850 2023-11-28T18:28:15.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc9727fc640 1 -- 172.21.15.105:0/71660097 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9780b5570 con 0x7fc97c0ad850 2023-11-28T18:28:15.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc9727fc640 1 -- 172.21.15.105:0/71660097 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9780b5920 con 0x7fc97c0ad850 2023-11-28T18:28:15.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc983baa640 1 -- 172.21.15.105:0/71660097 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc97c099fd0 con 0x7fc97c0ad850 2023-11-28T18:28:15.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc983baa640 1 -- 172.21.15.105:0/71660097 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97c09c6a0 con 0x7fc97c0ad850 2023-11-28T18:28:15.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.934+0000 7fc983baa640 1 -- 172.21.15.105:0/71660097 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc97c09c1f0 con 0x7fc97c0ad850 2023-11-28T18:28:15.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.938+0000 7fc9727fc640 1 -- 172.21.15.105:0/71660097 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc9780b3040 con 0x7fc97c0ad850 2023-11-28T18:28:15.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.938+0000 7fc9727fc640 1 --2- 172.21.15.105:0/71660097 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc9600668d0 0x7fc960068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:15.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.938+0000 7fc9727fc640 1 -- 172.21.15.105:0/71660097 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc97811e670 con 0x7fc97c0ad850 2023-11-28T18:28:15.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.938+0000 7fc981145640 1 --2- 172.21.15.105:0/71660097 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc9600668d0 0x7fc960068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:15.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.938+0000 7fc981145640 1 --2- 172.21.15.105:0/71660097 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc9600668d0 0x7fc960068d90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0b4c80 tx=0x7fc96c007c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:15.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:15.942+0000 7fc9727fc640 1 -- 172.21.15.105:0/71660097 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc9780eb640 con 0x7fc97c0ad850 2023-11-28T18:28:16.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.122+0000 7fc983baa640 1 -- 172.21.15.105:0/71660097 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc97c09c890 con 0x7fc97c0ad850 2023-11-28T18:28:16.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.758+0000 7fc9727fc640 1 -- 172.21.15.105:0/71660097 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v71) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc9780b10c0 con 0x7fc97c0ad850 2023-11-28T18:28:16.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.758+0000 7fc983baa640 1 -- 172.21.15.105:0/71660097 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc9600668d0 msgr2=0x7fc960068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.758+0000 7fc983baa640 1 --2- 172.21.15.105:0/71660097 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc9600668d0 0x7fc960068d90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0b4c80 tx=0x7fc96c007c10 comp rx=0 tx=0).stop 2023-11-28T18:28:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.758+0000 7fc983baa640 1 -- 172.21.15.105:0/71660097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc97c0ad850 msgr2=0x7fc97c1548a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.758+0000 7fc983baa640 1 --2- 172.21.15.105:0/71660097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc97c0ad850 0x7fc97c1548a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fc978019ff0 tx=0x7fc978019160 comp rx=0 tx=0).stop 2023-11-28T18:28:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.758+0000 7fc983baa640 1 -- 172.21.15.105:0/71660097 shutdown_connections 2023-11-28T18:28:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.758+0000 7fc983baa640 1 --2- 172.21.15.105:0/71660097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc97c159f40 0x7fc97c0af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.758+0000 7fc983baa640 1 --2- 172.21.15.105:0/71660097 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc97c0cc760 0x7fc97c14ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.758+0000 7fc983baa640 1 --2- 172.21.15.105:0/71660097 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc9600668d0 0x7fc960068d90 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.758+0000 7fc983baa640 1 --2- 172.21.15.105:0/71660097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc97c0ad850 0x7fc97c1548a0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.758+0000 7fc983baa640 1 -- 172.21.15.105:0/71660097 >> 172.21.15.105:0/71660097 conn(0x7fc97c0ceb90 msgr2=0x7fc97c0ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.758+0000 7fc983baa640 1 -- 172.21.15.105:0/71660097 shutdown_connections 2023-11-28T18:28:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.758+0000 7fc983baa640 1 -- 172.21.15.105:0/71660097 wait complete. 2023-11-28T18:28:16.788 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T18:28:16.788 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:28:16.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.926+0000 7f8e21cad640 1 Processor -- start 2023-11-28T18:28:16.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.926+0000 7f8e21cad640 1 -- start start 2023-11-28T18:28:16.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.926+0000 7f8e21cad640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e1c0ce2c0 0x7f8e1c159610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:16.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.926+0000 7f8e21cad640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8e1c0ccbc0 0x7f8e1c0ac490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:16.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.926+0000 7f8e21cad640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e1c0cde30 con 0x7f8e1c0cc760 2023-11-28T18:28:16.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.926+0000 7f8e21cad640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e1c0ce6a0 con 0x7f8e1c0ccbc0 2023-11-28T18:28:16.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.926+0000 7f8e21cad640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c06de20 con 0x7f8e1c0ce2c0 2023-11-28T18:28:16.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.926+0000 7f8e1bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8e1c0ccbc0 0x7f8e1c0ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:16.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.926+0000 7f8e1bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8e1c0ccbc0 0x7f8e1c0ac490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46372/0 (socket says 172.21.15.105:46372) 2023-11-28T18:28:16.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.926+0000 7f8e1bfff640 1 -- 172.21.15.105:0/4211825199 learned_addr learned my addr 172.21.15.105:0/4211825199 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:16.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.926+0000 7f8e1b7fe640 1 --2- 172.21.15.105:0/4211825199 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e1c0ce2c0 0x7f8e1c159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:16.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.926+0000 7f8e1bfff640 1 -- 172.21.15.105:0/4211825199 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e1c0ce2c0 msgr2=0x7f8e1c159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:16.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.926+0000 7f8e1bfff640 1 --2- 172.21.15.105:0/4211825199 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e1c0ce2c0 0x7f8e1c159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:16.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.926+0000 7f8e1bfff640 1 -- 172.21.15.105:0/4211825199 >> v1:172.21.15.148:6789/0 conn(0x7f8e1c0cc760 legacy=0x7f8e1c0abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:16.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.926+0000 7f8e1bfff640 1 -- 172.21.15.105:0/4211825199 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c0cd220 con 0x7f8e1c0ccbc0 2023-11-28T18:28:16.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.930+0000 7f8e1bfff640 1 --2- 172.21.15.105:0/4211825199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e1c0ccbc0 0x7f8e1c0ac490 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8e14009e50 tx=0x7f8e1401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b0543ef416edf0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:16.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.930+0000 7f8e1a7fc640 1 -- 172.21.15.105:0/4211825199 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e140a56d0 con 0x7f8e1c0ccbc0 2023-11-28T18:28:16.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.930+0000 7f8e1a7fc640 1 -- 172.21.15.105:0/4211825199 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e140a5830 con 0x7f8e1c0ccbc0 2023-11-28T18:28:16.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.930+0000 7f8e1a7fc640 1 -- 172.21.15.105:0/4211825199 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e140a5bc0 con 0x7f8e1c0ccbc0 2023-11-28T18:28:16.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.930+0000 7f8e21cad640 1 -- 172.21.15.105:0/4211825199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e1c0ccbc0 msgr2=0x7f8e1c0ac490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:16.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.930+0000 7f8e21cad640 1 --2- 172.21.15.105:0/4211825199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e1c0ccbc0 0x7f8e1c0ac490 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8e14009e50 tx=0x7f8e1401ca10 comp rx=0 tx=0).stop 2023-11-28T18:28:16.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.930+0000 7f8e21cad640 1 -- 172.21.15.105:0/4211825199 shutdown_connections 2023-11-28T18:28:16.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.930+0000 7f8e21cad640 1 --2- 172.21.15.105:0/4211825199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e1c0ccbc0 0x7f8e1c0ac490 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:16.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.930+0000 7f8e21cad640 1 --2- 172.21.15.105:0/4211825199 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e1c0ce2c0 0x7f8e1c159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:16.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.930+0000 7f8e21cad640 1 -- 172.21.15.105:0/4211825199 >> 172.21.15.105:0/4211825199 conn(0x7f8e1c0ceb90 msgr2=0x7f8e1c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:16.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.930+0000 7f8e21cad640 1 -- 172.21.15.105:0/4211825199 shutdown_connections 2023-11-28T18:28:16.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.930+0000 7f8e21cad640 1 -- 172.21.15.105:0/4211825199 wait complete. 2023-11-28T18:28:16.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e21cad640 1 Processor -- start 2023-11-28T18:28:16.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e21cad640 1 -- start start 2023-11-28T18:28:16.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e21cad640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e1c0cc760 0x7f8e1c14a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:16.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e21cad640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e1c0ccbc0 0x7f8e1c15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:16.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e21cad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e1c0ce2c0 0x7f8e1c15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:16.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e21cad640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c159ee0 con 0x7f8e1c0ccbc0 2023-11-28T18:28:16.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e21cad640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c1fb380 con 0x7f8e1c0ce2c0 2023-11-28T18:28:16.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e21cad640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c1fb5d0 con 0x7f8e1c0cc760 2023-11-28T18:28:16.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e1affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e1c0ccbc0 0x7f8e1c15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:16.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e1affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e1c0ccbc0 0x7f8e1c15aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46386/0 (socket says 172.21.15.105:46386) 2023-11-28T18:28:16.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e1affd640 1 -- 172.21.15.105:0/398304762 learned_addr learned my addr 172.21.15.105:0/398304762 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:16.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e1b7fe640 1 --2- 172.21.15.105:0/398304762 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e1c0cc760 0x7f8e1c14a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:16.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e1bfff640 1 --2- 172.21.15.105:0/398304762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e1c0ce2c0 0x7f8e1c15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:16.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e1affd640 1 -- 172.21.15.105:0/398304762 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e1c0cc760 msgr2=0x7f8e1c14a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:16.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e1affd640 1 --2- 172.21.15.105:0/398304762 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e1c0cc760 0x7f8e1c14a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:16.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e1affd640 1 -- 172.21.15.105:0/398304762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e1c0ce2c0 msgr2=0x7f8e1c15afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:16.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e1affd640 1 --2- 172.21.15.105:0/398304762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e1c0ce2c0 0x7f8e1c15afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:16.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e1affd640 1 -- 172.21.15.105:0/398304762 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e14009b30 con 0x7f8e1c0ccbc0 2023-11-28T18:28:16.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e1affd640 1 --2- 172.21.15.105:0/398304762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e1c0ccbc0 0x7f8e1c15aaa0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000c500 tx=0x7f8e1000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:16.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e03fff640 1 -- 172.21.15.105:0/398304762 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e10005220 con 0x7f8e1c0ccbc0 2023-11-28T18:28:16.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e03fff640 1 -- 172.21.15.105:0/398304762 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e10005380 con 0x7f8e1c0ccbc0 2023-11-28T18:28:16.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e03fff640 1 -- 172.21.15.105:0/398304762 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e10007270 con 0x7f8e1c0ccbc0 2023-11-28T18:28:16.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e21cad640 1 -- 172.21.15.105:0/398304762 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e1c1fb8b0 con 0x7f8e1c0ccbc0 2023-11-28T18:28:16.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.934+0000 7f8e21cad640 1 -- 172.21.15.105:0/398304762 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1c1fbe70 con 0x7f8e1c0ccbc0 2023-11-28T18:28:16.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.938+0000 7f8e21cad640 1 -- 172.21.15.105:0/398304762 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e1c067570 con 0x7f8e1c0ccbc0 2023-11-28T18:28:16.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.938+0000 7f8e03fff640 1 -- 172.21.15.105:0/398304762 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8e100073d0 con 0x7f8e1c0ccbc0 2023-11-28T18:28:16.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.938+0000 7f8e03fff640 1 --2- 172.21.15.105:0/398304762 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8e08066900 0x7f8e08068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:16.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.938+0000 7f8e03fff640 1 -- 172.21.15.105:0/398304762 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8e10083d20 con 0x7f8e1c0ccbc0 2023-11-28T18:28:16.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.938+0000 7f8e1b7fe640 1 --2- 172.21.15.105:0/398304762 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8e08066900 0x7f8e08068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:16.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.938+0000 7f8e1b7fe640 1 --2- 172.21.15.105:0/398304762 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8e08066900 0x7f8e08068dc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0079c0 tx=0x7f8e0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:16.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:16.942+0000 7f8e03fff640 1 -- 172.21.15.105:0/398304762 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8e10050ee0 con 0x7f8e1c0ccbc0 2023-11-28T18:28:17.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.122+0000 7f8e21cad640 1 -- 172.21.15.105:0/398304762 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e1c09cdd0 con 0x7f8e1c0ccbc0 2023-11-28T18:28:17.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.122+0000 7f8e03fff640 1 -- 172.21.15.105:0/398304762 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f8e10054500 con 0x7f8e1c0ccbc0 2023-11-28T18:28:17.128 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:17.128 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":71,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1455698212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":1455698212},{"type":"v1","addr":"172.21.15.105:6837","nonce":1455698212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6859,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/759522673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":759522673},{"type":"v1","addr":"172.21.15.148:6834","nonce":759522673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2489361300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2489361300},{"type":"v1","addr":"172.21.15.148:6835","nonce":2489361300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:28:13.097621+0000","modified":"2023-11-28T18:28:16.128249+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7194},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7194":{"gid":7194,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6836/188173955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":188173955},{"type":"v1","addr":"172.21.15.105:6836","nonce":188173955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-28T18:28:17.129 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 71 2023-11-28T18:28:17.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.122+0000 7f8e21cad640 1 -- 172.21.15.105:0/398304762 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8e08066900 msgr2=0x7f8e08068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:17.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.122+0000 7f8e21cad640 1 --2- 172.21.15.105:0/398304762 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8e08066900 0x7f8e08068dc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0079c0 tx=0x7f8e0c008040 comp rx=0 tx=0).stop 2023-11-28T18:28:17.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.126+0000 7f8e21cad640 1 -- 172.21.15.105:0/398304762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e1c0ccbc0 msgr2=0x7f8e1c15aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:17.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.126+0000 7f8e21cad640 1 --2- 172.21.15.105:0/398304762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e1c0ccbc0 0x7f8e1c15aaa0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000c500 tx=0x7f8e1000c9c0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.126+0000 7f8e21cad640 1 -- 172.21.15.105:0/398304762 shutdown_connections 2023-11-28T18:28:17.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.126+0000 7f8e21cad640 1 --2- 172.21.15.105:0/398304762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e1c0ce2c0 0x7f8e1c15afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.126+0000 7f8e21cad640 1 --2- 172.21.15.105:0/398304762 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8e08066900 0x7f8e08068dc0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.126+0000 7f8e21cad640 1 --2- 172.21.15.105:0/398304762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e1c0ccbc0 0x7f8e1c15aaa0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.126+0000 7f8e21cad640 1 --2- 172.21.15.105:0/398304762 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e1c0cc760 0x7f8e1c14a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.126+0000 7f8e21cad640 1 -- 172.21.15.105:0/398304762 >> 172.21.15.105:0/398304762 conn(0x7f8e1c0ceb90 msgr2=0x7f8e1c070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:17.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.126+0000 7f8e21cad640 1 -- 172.21.15.105:0/398304762 shutdown_connections 2023-11-28T18:28:17.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.126+0000 7f8e21cad640 1 -- 172.21.15.105:0/398304762 wait complete. 2023-11-28T18:28:17.148 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:28:17.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6ddefe640 1 Processor -- start 2023-11-28T18:28:17.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6ddefe640 1 -- start start 2023-11-28T18:28:17.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6ddefe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff6d80d1650 0x7ff6d80d1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:17.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6ddefe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6d80c7510 0x7ff6d80c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:17.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6ddefe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6d80d0c60 con 0x7ff6d815d300 2023-11-28T18:28:17.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6ddefe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6d80d1470 con 0x7ff6d80d1650 2023-11-28T18:28:17.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6ddefe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d80d2040 con 0x7ff6d80c7510 2023-11-28T18:28:17.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6d77fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff6d815d300 0x7ff6d80d37a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47924/0 (socket says 172.21.15.105:47924) 2023-11-28T18:28:17.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6d77fe640 1 -- 172.21.15.105:0/986241369 learned_addr learned my addr 172.21.15.105:0/986241369 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:17.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6d6ffd640 1 -- 172.21.15.105:0/986241369 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1424025824 0 0) 0x7ff6d80d0c60 con 0x7ff6d815d300 2023-11-28T18:28:17.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6d7fff640 1 --2- 172.21.15.105:0/986241369 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6d80c7510 0x7ff6d80c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:17.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6d6ffd640 1 -- 172.21.15.105:0/986241369 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6c4003530 con 0x7ff6d815d300 2023-11-28T18:28:17.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6cffff640 1 --2- 172.21.15.105:0/986241369 >> v2:172.21.15.148:3300/0 conn(0x7ff6d80d1650 0x7ff6d80d1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6d6ffd640 1 -- 172.21.15.105:0/986241369 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3162750359 0 0) 0x7ff6c4003530 con 0x7ff6d815d300 2023-11-28T18:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6d6ffd640 1 -- 172.21.15.105:0/986241369 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6d80c7510 msgr2=0x7ff6d80c79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6d6ffd640 1 --2- 172.21.15.105:0/986241369 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6d80c7510 0x7ff6d80c79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6d6ffd640 1 -- 172.21.15.105:0/986241369 >> v2:172.21.15.148:3300/0 conn(0x7ff6d80d1650 msgr2=0x7ff6d80d1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6d6ffd640 1 --2- 172.21.15.105:0/986241369 >> v2:172.21.15.148:3300/0 conn(0x7ff6d80d1650 0x7ff6d80d1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6d6ffd640 1 -- 172.21.15.105:0/986241369 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d80d0eb0 con 0x7ff6d815d300 2023-11-28T18:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6d6ffd640 1 -- 172.21.15.105:0/986241369 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7ff6c8001680 con 0x7ff6d815d300 2023-11-28T18:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6d6ffd640 1 -- 172.21.15.105:0/986241369 >> v1:172.21.15.105:6789/0 conn(0x7ff6d815d300 legacy=0x7ff6d80d37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6d6ffd640 1 --2- 172.21.15.105:0/986241369 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6d80c7510 0x7ff6c4003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6d6ffd640 1 --2- 172.21.15.105:0/986241369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6c4003f00 0x7ff6c40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6d6ffd640 1 --2- 172.21.15.105:0/986241369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6c4004d10 0x7ff6c4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:17.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6d6ffd640 1 -- 172.21.15.105:0/986241369 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d80d1470 con 0x7ff6c4003f00 2023-11-28T18:28:17.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6d6ffd640 1 -- 172.21.15.105:0/986241369 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d80d2040 con 0x7ff6c4004d10 2023-11-28T18:28:17.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6d6ffd640 1 -- 172.21.15.105:0/986241369 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c400b9e0 con 0x7ff6d80c7510 2023-11-28T18:28:17.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6cffff640 1 --2- 172.21.15.105:0/986241369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6c4003f00 0x7ff6c40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:17.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6d77fe640 1 --2- 172.21.15.105:0/986241369 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6d80c7510 0x7ff6c4003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:17.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6d7fff640 1 --2- 172.21.15.105:0/986241369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6c4004d10 0x7ff6c4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:17.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6cffff640 1 -- 172.21.15.105:0/986241369 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6d80c7510 msgr2=0x7ff6c4003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:17.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6cffff640 1 --2- 172.21.15.105:0/986241369 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6d80c7510 0x7ff6c4003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6cffff640 1 -- 172.21.15.105:0/986241369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6c4004d10 msgr2=0x7ff6c4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:17.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.282+0000 7ff6cffff640 1 --2- 172.21.15.105:0/986241369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6c4004d10 0x7ff6c4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.286+0000 7ff6cffff640 1 -- 172.21.15.105:0/986241369 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c4003530 con 0x7ff6c4003f00 2023-11-28T18:28:17.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.286+0000 7ff6cffff640 1 --2- 172.21.15.105:0/986241369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6c4003f00 0x7ff6c40047d0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009b10 tx=0x7ff6c001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caaa7c8bd306c332 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.286+0000 7ff6d6ffd640 1 -- 172.21.15.105:0/986241369 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c00a5550 con 0x7ff6c4003f00 2023-11-28T18:28:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.286+0000 7ff6d6ffd640 1 -- 172.21.15.105:0/986241369 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c00a56b0 con 0x7ff6c4003f00 2023-11-28T18:28:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.286+0000 7ff6d6ffd640 1 -- 172.21.15.105:0/986241369 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c00a5a60 con 0x7ff6c4003f00 2023-11-28T18:28:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.286+0000 7ff6ddefe640 1 -- 172.21.15.105:0/986241369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6c4003f00 msgr2=0x7ff6c40047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.286+0000 7ff6ddefe640 1 --2- 172.21.15.105:0/986241369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6c4003f00 0x7ff6c40047d0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009b10 tx=0x7ff6c001ca90 comp rx=0 tx=0).stop 2023-11-28T18:28:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.286+0000 7ff6d7fff640 1 -- 172.21.15.105:0/986241369 reap_dead start 2023-11-28T18:28:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.286+0000 7ff6ddefe640 1 -- 172.21.15.105:0/986241369 shutdown_connections 2023-11-28T18:28:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.286+0000 7ff6ddefe640 1 -- 172.21.15.105:0/986241369 >> 172.21.15.105:0/986241369 conn(0x7ff6d809c360 msgr2=0x7ff6d80d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.286+0000 7ff6ddefe640 1 -- 172.21.15.105:0/986241369 shutdown_connections 2023-11-28T18:28:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.286+0000 7ff6ddefe640 1 -- 172.21.15.105:0/986241369 wait complete. 2023-11-28T18:28:17.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6ddefe640 1 Processor -- start 2023-11-28T18:28:17.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6ddefe640 1 -- start start 2023-11-28T18:28:17.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6ddefe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6c4003f00 0x7ff6d81f5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:17.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6ddefe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d8158570 0x7ff6d80ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:17.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6ddefe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6d80cd1b0 0x7ff6d80cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:17.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6ddefe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d81f5ec0 con 0x7ff6d8158570 2023-11-28T18:28:17.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6ddefe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d8160770 con 0x7ff6c4003f00 2023-11-28T18:28:17.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6ddefe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d815fbf0 con 0x7ff6d80cd1b0 2023-11-28T18:28:17.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6cffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d8158570 0x7ff6d80ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:17.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6cffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d8158570 0x7ff6d80ccc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46402/0 (socket says 172.21.15.105:46402) 2023-11-28T18:28:17.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6cffff640 1 -- 172.21.15.105:0/3880161826 learned_addr learned my addr 172.21.15.105:0/3880161826 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:17.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6d77fe640 1 --2- 172.21.15.105:0/3880161826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6c4003f00 0x7ff6d81f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:17.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6d7fff640 1 --2- 172.21.15.105:0/3880161826 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6d80cd1b0 0x7ff6d80cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:17.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6cffff640 1 -- 172.21.15.105:0/3880161826 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6d80cd1b0 msgr2=0x7ff6d80cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:17.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6cffff640 1 --2- 172.21.15.105:0/3880161826 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6d80cd1b0 0x7ff6d80cf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6cffff640 1 -- 172.21.15.105:0/3880161826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6c4003f00 msgr2=0x7ff6d81f5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:17.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6cffff640 1 --2- 172.21.15.105:0/3880161826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6c4003f00 0x7ff6d81f5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6cffff640 1 -- 172.21.15.105:0/3880161826 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d815fed0 con 0x7ff6d8158570 2023-11-28T18:28:17.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6d77fe640 1 --2- 172.21.15.105:0/3880161826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6c4003f00 0x7ff6d81f5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:17.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6cffff640 1 --2- 172.21.15.105:0/3880161826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d8158570 0x7ff6d80ccc70 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009a00 tx=0x7ff6c4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:17.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6d4ff9640 1 -- 172.21.15.105:0/3880161826 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c00a5e60 con 0x7ff6d8158570 2023-11-28T18:28:17.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6d4ff9640 1 -- 172.21.15.105:0/3880161826 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c0004020 con 0x7ff6d8158570 2023-11-28T18:28:17.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6ddefe640 1 -- 172.21.15.105:0/3880161826 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d81601b0 con 0x7ff6d8158570 2023-11-28T18:28:17.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.290+0000 7ff6d4ff9640 1 -- 172.21.15.105:0/3880161826 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c00043b0 con 0x7ff6d8158570 2023-11-28T18:28:17.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.294+0000 7ff6ddefe640 1 -- 172.21.15.105:0/3880161826 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d81589f0 con 0x7ff6d8158570 2023-11-28T18:28:17.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.294+0000 7ff6ddefe640 1 -- 172.21.15.105:0/3880161826 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d8067570 con 0x7ff6d8158570 2023-11-28T18:28:17.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.294+0000 7ff6d4ff9640 1 -- 172.21.15.105:0/3880161826 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff6c00b4040 con 0x7ff6d8158570 2023-11-28T18:28:17.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.294+0000 7ff6d4ff9640 1 --2- 172.21.15.105:0/3880161826 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6b8066900 0x7ff6b8068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:17.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.294+0000 7ff6d4ff9640 1 -- 172.21.15.105:0/3880161826 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff6c011dcc0 con 0x7ff6d8158570 2023-11-28T18:28:17.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.294+0000 7ff6d77fe640 1 --2- 172.21.15.105:0/3880161826 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6b8066900 0x7ff6b8068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:17.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.294+0000 7ff6d77fe640 1 --2- 172.21.15.105:0/3880161826 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6b8066900 0x7ff6b8068dc0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8001440 tx=0x7ff6c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:17.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.298+0000 7ff6d4ff9640 1 -- 172.21.15.105:0/3880161826 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff6c0124080 con 0x7ff6d8158570 2023-11-28T18:28:17.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.450+0000 7ff6ddefe640 1 -- 172.21.15.105:0/3880161826 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6d80c9500 con 0x7ff6d8158570 2023-11-28T18:28:17.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.450+0000 7ff6d4ff9640 1 -- 172.21.15.105:0/3880161826 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16837 (secure 0 0 0) 0x7ff6c00be090 con 0x7ff6d8158570 2023-11-28T18:28:17.456 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:17.457 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":78,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:28:14.101062+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:28:11.454373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-28T18:28:12.093171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:28:17.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.454+0000 7ff6ddefe640 1 -- 172.21.15.105:0/3880161826 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6b8066900 msgr2=0x7ff6b8068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:17.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.454+0000 7ff6ddefe640 1 --2- 172.21.15.105:0/3880161826 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6b8066900 0x7ff6b8068dc0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8001440 tx=0x7ff6c8009040 comp rx=0 tx=0).stop 2023-11-28T18:28:17.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.454+0000 7ff6ddefe640 1 -- 172.21.15.105:0/3880161826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d8158570 msgr2=0x7ff6d80ccc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:17.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.454+0000 7ff6ddefe640 1 --2- 172.21.15.105:0/3880161826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d8158570 0x7ff6d80ccc70 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009a00 tx=0x7ff6c4005800 comp rx=0 tx=0).stop 2023-11-28T18:28:17.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.454+0000 7ff6ddefe640 1 -- 172.21.15.105:0/3880161826 shutdown_connections 2023-11-28T18:28:17.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.454+0000 7ff6ddefe640 1 --2- 172.21.15.105:0/3880161826 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6d80cd1b0 0x7ff6d80cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.454+0000 7ff6ddefe640 1 --2- 172.21.15.105:0/3880161826 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff6b8066900 0x7ff6b8068dc0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.454+0000 7ff6ddefe640 1 --2- 172.21.15.105:0/3880161826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d8158570 0x7ff6d80ccc70 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.454+0000 7ff6ddefe640 1 --2- 172.21.15.105:0/3880161826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6c4003f00 0x7ff6d81f5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.454+0000 7ff6ddefe640 1 -- 172.21.15.105:0/3880161826 >> 172.21.15.105:0/3880161826 conn(0x7ff6d809c360 msgr2=0x7ff6d806b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:17.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.454+0000 7ff6ddefe640 1 -- 172.21.15.105:0/3880161826 shutdown_connections 2023-11-28T18:28:17.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.454+0000 7ff6ddefe640 1 -- 172.21.15.105:0/3880161826 wait complete. 2023-11-28T18:28:17.478 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T18:28:17.479 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:28:17.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.610+0000 7fb79a957640 1 Processor -- start 2023-11-28T18:28:17.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.614+0000 7fb79a957640 1 -- start start 2023-11-28T18:28:17.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.614+0000 7fb79a957640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb794154070 0x7fb79415d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:17.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.614+0000 7fb79a957640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb79406bf50 0x7fb7940ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:17.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.614+0000 7fb79a957640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7940cc7f0 con 0x7fb7940adb80 2023-11-28T18:28:17.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.614+0000 7fb79a957640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb794072540 con 0x7fb794154070 2023-11-28T18:28:17.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.614+0000 7fb79a957640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79415dd10 con 0x7fb79406bf50 2023-11-28T18:28:17.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.614+0000 7fb798ef4640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb7940adb80 0x7fb7940acd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47926/0 (socket says 172.21.15.105:47926) 2023-11-28T18:28:17.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.614+0000 7fb798ef4640 1 -- 172.21.15.105:0/2906020460 learned_addr learned my addr 172.21.15.105:0/2906020460 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:17.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.614+0000 7fb7937fe640 1 --2- 172.21.15.105:0/2906020460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb79406bf50 0x7fb7940ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:17.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.614+0000 7fb793fff640 1 --2- 172.21.15.105:0/2906020460 >> v2:172.21.15.148:3300/0 conn(0x7fb794154070 0x7fb79415d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:17.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.614+0000 7fb792ffd640 1 -- 172.21.15.105:0/2906020460 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264180037 0 0) 0x7fb7940cc7f0 con 0x7fb7940adb80 2023-11-28T18:28:17.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.614+0000 7fb792ffd640 1 -- 172.21.15.105:0/2906020460 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb77c003530 con 0x7fb7940adb80 2023-11-28T18:28:17.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.614+0000 7fb7937fe640 1 -- 172.21.15.105:0/2906020460 >> v2:172.21.15.148:3300/0 conn(0x7fb794154070 msgr2=0x7fb79415d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:17.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.614+0000 7fb792ffd640 1 -- 172.21.15.105:0/2906020460 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3967804527 0 0) 0x7fb77c003530 con 0x7fb7940adb80 2023-11-28T18:28:17.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.614+0000 7fb7937fe640 1 --2- 172.21.15.105:0/2906020460 >> v2:172.21.15.148:3300/0 conn(0x7fb794154070 0x7fb79415d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.614+0000 7fb7937fe640 1 -- 172.21.15.105:0/2906020460 >> v1:172.21.15.105:6789/0 conn(0x7fb7940adb80 legacy=0x7fb7940acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:17.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.614+0000 7fb7937fe640 1 -- 172.21.15.105:0/2906020460 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79415dff0 con 0x7fb79406bf50 2023-11-28T18:28:17.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.618+0000 7fb7937fe640 1 --2- 172.21.15.105:0/2906020460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb79406bf50 0x7fb7940ce310 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb7880097c0 tx=0x7fb78801c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f77af60d7577d864 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:17.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.618+0000 7fb792ffd640 1 -- 172.21.15.105:0/2906020460 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c001a00 con 0x7fb79406bf50 2023-11-28T18:28:17.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.618+0000 7fb792ffd640 1 -- 172.21.15.105:0/2906020460 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7880a7050 con 0x7fb79406bf50 2023-11-28T18:28:17.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.618+0000 7fb792ffd640 1 -- 172.21.15.105:0/2906020460 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7880ab620 con 0x7fb79406bf50 2023-11-28T18:28:17.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.618+0000 7fb79a957640 1 -- 172.21.15.105:0/2906020460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb79406bf50 msgr2=0x7fb7940ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:17.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.618+0000 7fb79a957640 1 --2- 172.21.15.105:0/2906020460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb79406bf50 0x7fb7940ce310 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb7880097c0 tx=0x7fb78801c370 comp rx=0 tx=0).stop 2023-11-28T18:28:17.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.618+0000 7fb79a957640 1 -- 172.21.15.105:0/2906020460 shutdown_connections 2023-11-28T18:28:17.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.618+0000 7fb79a957640 1 --2- 172.21.15.105:0/2906020460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb79406bf50 0x7fb7940ce310 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.618+0000 7fb79a957640 1 --2- 172.21.15.105:0/2906020460 >> v2:172.21.15.148:3300/0 conn(0x7fb794154070 0x7fb79415d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.618+0000 7fb79a957640 1 -- 172.21.15.105:0/2906020460 >> 172.21.15.105:0/2906020460 conn(0x7fb7940ceb90 msgr2=0x7fb7940cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:17.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.618+0000 7fb79a957640 1 -- 172.21.15.105:0/2906020460 shutdown_connections 2023-11-28T18:28:17.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.618+0000 7fb79a957640 1 -- 172.21.15.105:0/2906020460 wait complete. 2023-11-28T18:28:17.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.618+0000 7fb79a957640 1 Processor -- start 2023-11-28T18:28:17.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb79a957640 1 -- start start 2023-11-28T18:28:17.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb79a957640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb79406bf50 0x7fb7940b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:17.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb79a957640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7940adb80 0x7fb79414a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:17.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb79a957640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb794154070 0x7fb79406c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:17.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb79a957640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79414e760 con 0x7fb7940adb80 2023-11-28T18:28:17.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb79a957640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7940b49a0 con 0x7fb79406bf50 2023-11-28T18:28:17.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb79a957640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79414db50 con 0x7fb794154070 2023-11-28T18:28:17.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb7937fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7940adb80 0x7fb79414a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:17.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb7937fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7940adb80 0x7fb79414a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46408/0 (socket says 172.21.15.105:46408) 2023-11-28T18:28:17.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb7937fe640 1 -- 172.21.15.105:0/870126359 learned_addr learned my addr 172.21.15.105:0/870126359 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:17.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb798ef4640 1 --2- 172.21.15.105:0/870126359 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb794154070 0x7fb79406c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:17.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb793fff640 1 --2- 172.21.15.105:0/870126359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb79406bf50 0x7fb7940b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:17.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb7937fe640 1 -- 172.21.15.105:0/870126359 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb794154070 msgr2=0x7fb79406c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:17.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb7937fe640 1 --2- 172.21.15.105:0/870126359 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb794154070 0x7fb79406c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb7937fe640 1 -- 172.21.15.105:0/870126359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb79406bf50 msgr2=0x7fb7940b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:17.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb7937fe640 1 --2- 172.21.15.105:0/870126359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb79406bf50 0x7fb7940b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb7937fe640 1 -- 172.21.15.105:0/870126359 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb788009420 con 0x7fb7940adb80 2023-11-28T18:28:17.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb7937fe640 1 --2- 172.21.15.105:0/870126359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7940adb80 0x7fb79414a9e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fb78801aa20 tx=0x7fb788019110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:17.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb790ff9640 1 -- 172.21.15.105:0/870126359 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7880b4430 con 0x7fb7940adb80 2023-11-28T18:28:17.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb790ff9640 1 -- 172.21.15.105:0/870126359 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7880b45b0 con 0x7fb7940adb80 2023-11-28T18:28:17.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb790ff9640 1 -- 172.21.15.105:0/870126359 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7880b4960 con 0x7fb7940adb80 2023-11-28T18:28:17.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb79a957640 1 -- 172.21.15.105:0/870126359 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb79414de30 con 0x7fb7940adb80 2023-11-28T18:28:17.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb79a957640 1 -- 172.21.15.105:0/870126359 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb79414e3f0 con 0x7fb7940adb80 2023-11-28T18:28:17.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.622+0000 7fb79a957640 1 -- 172.21.15.105:0/870126359 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb794067570 con 0x7fb7940adb80 2023-11-28T18:28:17.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.626+0000 7fb790ff9640 1 -- 172.21.15.105:0/870126359 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fb7880b2040 con 0x7fb7940adb80 2023-11-28T18:28:17.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.626+0000 7fb790ff9640 1 --2- 172.21.15.105:0/870126359 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb780066900 0x7fb780068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:17.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.626+0000 7fb790ff9640 1 -- 172.21.15.105:0/870126359 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb78811e170 con 0x7fb7940adb80 2023-11-28T18:28:17.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.626+0000 7fb793fff640 1 --2- 172.21.15.105:0/870126359 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb780066900 0x7fb780068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:17.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.626+0000 7fb793fff640 1 --2- 172.21.15.105:0/870126359 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb780066900 0x7fb780068dc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb7940b2a80 tx=0x7fb784009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:17.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.630+0000 7fb790ff9640 1 -- 172.21.15.105:0/870126359 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb7880eb320 con 0x7fb7940adb80 2023-11-28T18:28:17.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.778+0000 7fb79a957640 1 -- 172.21.15.105:0/870126359 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb7940ce6e0 con 0x7fb780066900 2023-11-28T18:28:17.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.786+0000 7fb790ff9640 1 -- 172.21.15.105:0/870126359 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289237 (secure 0 0 0) 0x7fb7940ce6e0 con 0x7fb780066900 2023-11-28T18:28:17.794 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:17.796 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:28:17.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.790+0000 7fb79a957640 1 -- 172.21.15.105:0/870126359 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb780066900 msgr2=0x7fb780068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:17.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.790+0000 7fb79a957640 1 --2- 172.21.15.105:0/870126359 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb780066900 0x7fb780068dc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb7940b2a80 tx=0x7fb784009190 comp rx=0 tx=0).stop 2023-11-28T18:28:17.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.790+0000 7fb79a957640 1 -- 172.21.15.105:0/870126359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7940adb80 msgr2=0x7fb79414a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:17.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.790+0000 7fb79a957640 1 --2- 172.21.15.105:0/870126359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7940adb80 0x7fb79414a9e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fb78801aa20 tx=0x7fb788019110 comp rx=0 tx=0).stop 2023-11-28T18:28:17.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.790+0000 7fb79a957640 1 -- 172.21.15.105:0/870126359 shutdown_connections 2023-11-28T18:28:17.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.790+0000 7fb79a957640 1 --2- 172.21.15.105:0/870126359 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb794154070 0x7fb79406c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.790+0000 7fb79a957640 1 --2- 172.21.15.105:0/870126359 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb780066900 0x7fb780068dc0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.790+0000 7fb79a957640 1 --2- 172.21.15.105:0/870126359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7940adb80 0x7fb79414a9e0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.790+0000 7fb79a957640 1 --2- 172.21.15.105:0/870126359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb79406bf50 0x7fb7940b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:17.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.790+0000 7fb79a957640 1 -- 172.21.15.105:0/870126359 >> 172.21.15.105:0/870126359 conn(0x7fb7940ceb90 msgr2=0x7fb7940ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:17.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.790+0000 7fb79a957640 1 -- 172.21.15.105:0/870126359 shutdown_connections 2023-11-28T18:28:17.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.790+0000 7fb79a957640 1 -- 172.21.15.105:0/870126359 wait complete. 2023-11-28T18:28:17.814 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2023-11-28T18:28:17.330002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":305312,"kb_used_data":8680,"kb_used_omap":12,"kb_used_meta":296499,"kb_avail":754669408,"statfs":{"total":773094113280,"available":772781473792,"internally_reserved":0,"allocated":8888320,"data_stored":4685796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":303615538},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.621225"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663593+0000","last_change":"2023-11-28T18:28:13.124330+0000","last_active":"2023-11-28T18:28:14.663593+0000","last_peered":"2023-11-28T18:28:14.663593+0000","last_clean":"2023-11-28T18:28:14.663593+0000","last_became_active":"2023-11-28T18:28:13.124060+0000","last_became_peered":"2023-11-28T18:28:13.124060+0000","last_unstale":"2023-11-28T18:28:14.663593+0000","last_undegraded":"2023-11-28T18:28:14.663593+0000","last_fullsized":"2023-11-28T18:28:14.663593+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:08:51.905424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.664331+0000","last_change":"2023-11-28T18:28:13.126226+0000","last_active":"2023-11-28T18:28:14.664331+0000","last_peered":"2023-11-28T18:28:14.664331+0000","last_clean":"2023-11-28T18:28:14.664331+0000","last_became_active":"2023-11-28T18:28:13.125439+0000","last_became_peered":"2023-11-28T18:28:13.125439+0000","last_unstale":"2023-11-28T18:28:14.664331+0000","last_undegraded":"2023-11-28T18:28:14.664331+0000","last_fullsized":"2023-11-28T18:28:14.664331+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:46:09.462436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.662642+0000","last_change":"2023-11-28T18:28:13.117878+0000","last_active":"2023-11-28T18:28:14.662642+0000","last_peered":"2023-11-28T18:28:14.662642+0000","last_clean":"2023-11-28T18:28:14.662642+0000","last_became_active":"2023-11-28T18:28:13.117066+0000","last_became_peered":"2023-11-28T18:28:13.117066+0000","last_unstale":"2023-11-28T18:28:14.662642+0000","last_undegraded":"2023-11-28T18:28:14.662642+0000","last_fullsized":"2023-11-28T18:28:14.662642+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:19.143572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663216+0000","last_change":"2023-11-28T18:28:13.130808+0000","last_active":"2023-11-28T18:28:14.663216+0000","last_peered":"2023-11-28T18:28:14.663216+0000","last_clean":"2023-11-28T18:28:14.663216+0000","last_became_active":"2023-11-28T18:28:13.126831+0000","last_became_peered":"2023-11-28T18:28:13.126831+0000","last_unstale":"2023-11-28T18:28:14.663216+0000","last_undegraded":"2023-11-28T18:28:14.663216+0000","last_fullsized":"2023-11-28T18:28:14.663216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:44:27.643684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663290+0000","last_change":"2023-11-28T18:28:13.122622+0000","last_active":"2023-11-28T18:28:14.663290+0000","last_peered":"2023-11-28T18:28:14.663290+0000","last_clean":"2023-11-28T18:28:14.663290+0000","last_became_active":"2023-11-28T18:28:13.122322+0000","last_became_peered":"2023-11-28T18:28:13.122322+0000","last_unstale":"2023-11-28T18:28:14.663290+0000","last_undegraded":"2023-11-28T18:28:14.663290+0000","last_fullsized":"2023-11-28T18:28:14.663290+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:59.435656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.111201+0000","last_change":"2023-11-28T18:28:13.121666+0000","last_active":"2023-11-28T18:28:14.111201+0000","last_peered":"2023-11-28T18:28:14.111201+0000","last_clean":"2023-11-28T18:28:14.111201+0000","last_became_active":"2023-11-28T18:28:13.121392+0000","last_became_peered":"2023-11-28T18:28:13.121392+0000","last_unstale":"2023-11-28T18:28:14.111201+0000","last_undegraded":"2023-11-28T18:28:14.111201+0000","last_fullsized":"2023-11-28T18:28:14.111201+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:32.852317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.114835+0000","last_change":"2023-11-28T18:28:13.132620+0000","last_active":"2023-11-28T18:28:14.114835+0000","last_peered":"2023-11-28T18:28:14.114835+0000","last_clean":"2023-11-28T18:28:14.114835+0000","last_became_active":"2023-11-28T18:28:13.132331+0000","last_became_peered":"2023-11-28T18:28:13.132331+0000","last_unstale":"2023-11-28T18:28:14.114835+0000","last_undegraded":"2023-11-28T18:28:14.114835+0000","last_fullsized":"2023-11-28T18:28:14.114835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:18:43.396576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663048+0000","last_change":"2023-11-28T18:28:13.121326+0000","last_active":"2023-11-28T18:28:14.663048+0000","last_peered":"2023-11-28T18:28:14.663048+0000","last_clean":"2023-11-28T18:28:14.663048+0000","last_became_active":"2023-11-28T18:28:13.120973+0000","last_became_peered":"2023-11-28T18:28:13.120973+0000","last_unstale":"2023-11-28T18:28:14.663048+0000","last_undegraded":"2023-11-28T18:28:14.663048+0000","last_fullsized":"2023-11-28T18:28:14.663048+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:55:15.349267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115275+0000","last_change":"2023-11-28T18:28:13.119783+0000","last_active":"2023-11-28T18:28:14.115275+0000","last_peered":"2023-11-28T18:28:14.115275+0000","last_clean":"2023-11-28T18:28:14.115275+0000","last_became_active":"2023-11-28T18:28:13.119493+0000","last_became_peered":"2023-11-28T18:28:13.119493+0000","last_unstale":"2023-11-28T18:28:14.115275+0000","last_undegraded":"2023-11-28T18:28:14.115275+0000","last_fullsized":"2023-11-28T18:28:14.115275+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:38.212829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663391+0000","last_change":"2023-11-28T18:28:13.125929+0000","last_active":"2023-11-28T18:28:14.663391+0000","last_peered":"2023-11-28T18:28:14.663391+0000","last_clean":"2023-11-28T18:28:14.663391+0000","last_became_active":"2023-11-28T18:28:13.124490+0000","last_became_peered":"2023-11-28T18:28:13.124490+0000","last_unstale":"2023-11-28T18:28:14.663391+0000","last_undegraded":"2023-11-28T18:28:14.663391+0000","last_fullsized":"2023-11-28T18:28:14.663391+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:26:14.468846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.664349+0000","last_change":"2023-11-28T18:28:13.127845+0000","last_active":"2023-11-28T18:28:14.664349+0000","last_peered":"2023-11-28T18:28:14.664349+0000","last_clean":"2023-11-28T18:28:14.664349+0000","last_became_active":"2023-11-28T18:28:13.126211+0000","last_became_peered":"2023-11-28T18:28:13.126211+0000","last_unstale":"2023-11-28T18:28:14.664349+0000","last_undegraded":"2023-11-28T18:28:14.664349+0000","last_fullsized":"2023-11-28T18:28:14.664349+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:54:39.161073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663592+0000","last_change":"2023-11-28T18:28:13.126609+0000","last_active":"2023-11-28T18:28:14.663592+0000","last_peered":"2023-11-28T18:28:14.663592+0000","last_clean":"2023-11-28T18:28:14.663592+0000","last_became_active":"2023-11-28T18:28:13.126268+0000","last_became_peered":"2023-11-28T18:28:13.126268+0000","last_unstale":"2023-11-28T18:28:14.663592+0000","last_undegraded":"2023-11-28T18:28:14.663592+0000","last_fullsized":"2023-11-28T18:28:14.663592+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:49:48.533213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663410+0000","last_change":"2023-11-28T18:28:13.119019+0000","last_active":"2023-11-28T18:28:14.663410+0000","last_peered":"2023-11-28T18:28:14.663410+0000","last_clean":"2023-11-28T18:28:14.663410+0000","last_became_active":"2023-11-28T18:28:13.118740+0000","last_became_peered":"2023-11-28T18:28:13.118740+0000","last_unstale":"2023-11-28T18:28:14.663410+0000","last_undegraded":"2023-11-28T18:28:14.663410+0000","last_fullsized":"2023-11-28T18:28:14.663410+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:06:42.585287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.111190+0000","last_change":"2023-11-28T18:28:13.121405+0000","last_active":"2023-11-28T18:28:14.111190+0000","last_peered":"2023-11-28T18:28:14.111190+0000","last_clean":"2023-11-28T18:28:14.111190+0000","last_became_active":"2023-11-28T18:28:13.121076+0000","last_became_peered":"2023-11-28T18:28:13.121076+0000","last_unstale":"2023-11-28T18:28:14.111190+0000","last_undegraded":"2023-11-28T18:28:14.111190+0000","last_fullsized":"2023-11-28T18:28:14.111190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:54:20.912473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663042+0000","last_change":"2023-11-28T18:28:13.125273+0000","last_active":"2023-11-28T18:28:14.663042+0000","last_peered":"2023-11-28T18:28:14.663042+0000","last_clean":"2023-11-28T18:28:14.663042+0000","last_became_active":"2023-11-28T18:28:13.124955+0000","last_became_peered":"2023-11-28T18:28:13.124955+0000","last_unstale":"2023-11-28T18:28:14.663042+0000","last_undegraded":"2023-11-28T18:28:14.663042+0000","last_fullsized":"2023-11-28T18:28:14.663042+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:06.468018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663053+0000","last_change":"2023-11-28T18:28:13.118087+0000","last_active":"2023-11-28T18:28:14.663053+0000","last_peered":"2023-11-28T18:28:14.663053+0000","last_clean":"2023-11-28T18:28:14.663053+0000","last_became_active":"2023-11-28T18:28:13.117810+0000","last_became_peered":"2023-11-28T18:28:13.117810+0000","last_unstale":"2023-11-28T18:28:14.663053+0000","last_undegraded":"2023-11-28T18:28:14.663053+0000","last_fullsized":"2023-11-28T18:28:14.663053+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:31:15.879055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.664158+0000","last_change":"2023-11-28T18:28:13.123501+0000","last_active":"2023-11-28T18:28:14.664158+0000","last_peered":"2023-11-28T18:28:14.664158+0000","last_clean":"2023-11-28T18:28:14.664158+0000","last_became_active":"2023-11-28T18:28:13.123240+0000","last_became_peered":"2023-11-28T18:28:13.123240+0000","last_unstale":"2023-11-28T18:28:14.664158+0000","last_undegraded":"2023-11-28T18:28:14.664158+0000","last_fullsized":"2023-11-28T18:28:14.664158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:37.908507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.662440+0000","last_change":"2023-11-28T18:28:13.122905+0000","last_active":"2023-11-28T18:28:14.662440+0000","last_peered":"2023-11-28T18:28:14.662440+0000","last_clean":"2023-11-28T18:28:14.662440+0000","last_became_active":"2023-11-28T18:28:13.122623+0000","last_became_peered":"2023-11-28T18:28:13.122623+0000","last_unstale":"2023-11-28T18:28:14.662440+0000","last_undegraded":"2023-11-28T18:28:14.662440+0000","last_fullsized":"2023-11-28T18:28:14.662440+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:18:43.396576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.111506+0000","last_change":"2023-11-28T18:28:13.128670+0000","last_active":"2023-11-28T18:28:14.111506+0000","last_peered":"2023-11-28T18:28:14.111506+0000","last_clean":"2023-11-28T18:28:14.111506+0000","last_became_active":"2023-11-28T18:28:13.128411+0000","last_became_peered":"2023-11-28T18:28:13.128411+0000","last_unstale":"2023-11-28T18:28:14.111506+0000","last_undegraded":"2023-11-28T18:28:14.111506+0000","last_fullsized":"2023-11-28T18:28:14.111506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:45:53.434006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663769+0000","last_change":"2023-11-28T18:28:13.120288+0000","last_active":"2023-11-28T18:28:14.663769+0000","last_peered":"2023-11-28T18:28:14.663769+0000","last_clean":"2023-11-28T18:28:14.663769+0000","last_became_active":"2023-11-28T18:28:13.120041+0000","last_became_peered":"2023-11-28T18:28:13.120041+0000","last_unstale":"2023-11-28T18:28:14.663769+0000","last_undegraded":"2023-11-28T18:28:14.663769+0000","last_fullsized":"2023-11-28T18:28:14.663769+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:55:42.482137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115276+0000","last_change":"2023-11-28T18:28:13.111597+0000","last_active":"2023-11-28T18:28:14.115276+0000","last_peered":"2023-11-28T18:28:14.115276+0000","last_clean":"2023-11-28T18:28:14.115276+0000","last_became_active":"2023-11-28T18:28:13.111192+0000","last_became_peered":"2023-11-28T18:28:13.111192+0000","last_unstale":"2023-11-28T18:28:14.115276+0000","last_undegraded":"2023-11-28T18:28:14.115276+0000","last_fullsized":"2023-11-28T18:28:14.115276+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:18:33.289615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663455+0000","last_change":"2023-11-28T18:28:13.119618+0000","last_active":"2023-11-28T18:28:14.663455+0000","last_peered":"2023-11-28T18:28:14.663455+0000","last_clean":"2023-11-28T18:28:14.663455+0000","last_became_active":"2023-11-28T18:28:13.119310+0000","last_became_peered":"2023-11-28T18:28:13.119310+0000","last_unstale":"2023-11-28T18:28:14.663455+0000","last_undegraded":"2023-11-28T18:28:14.663455+0000","last_fullsized":"2023-11-28T18:28:14.663455+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:41:52.691135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.114631+0000","last_change":"2023-11-28T18:28:13.115046+0000","last_active":"2023-11-28T18:28:14.114631+0000","last_peered":"2023-11-28T18:28:14.114631+0000","last_clean":"2023-11-28T18:28:14.114631+0000","last_became_active":"2023-11-28T18:28:13.114743+0000","last_became_peered":"2023-11-28T18:28:13.114743+0000","last_unstale":"2023-11-28T18:28:14.114631+0000","last_undegraded":"2023-11-28T18:28:14.114631+0000","last_fullsized":"2023-11-28T18:28:14.114631+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:22:36.702610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.662845+0000","last_change":"2023-11-28T18:28:13.118568+0000","last_active":"2023-11-28T18:28:14.662845+0000","last_peered":"2023-11-28T18:28:14.662845+0000","last_clean":"2023-11-28T18:28:14.662845+0000","last_became_active":"2023-11-28T18:28:13.118318+0000","last_became_peered":"2023-11-28T18:28:13.118318+0000","last_unstale":"2023-11-28T18:28:14.662845+0000","last_undegraded":"2023-11-28T18:28:14.662845+0000","last_fullsized":"2023-11-28T18:28:14.662845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:15:09.428663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.664309+0000","last_change":"2023-11-28T18:28:13.117927+0000","last_active":"2023-11-28T18:28:14.664309+0000","last_peered":"2023-11-28T18:28:14.664309+0000","last_clean":"2023-11-28T18:28:14.664309+0000","last_became_active":"2023-11-28T18:28:13.117623+0000","last_became_peered":"2023-11-28T18:28:13.117623+0000","last_unstale":"2023-11-28T18:28:14.664309+0000","last_undegraded":"2023-11-28T18:28:14.664309+0000","last_fullsized":"2023-11-28T18:28:14.664309+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:18:43.726285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663509+0000","last_change":"2023-11-28T18:28:13.130241+0000","last_active":"2023-11-28T18:28:14.663509+0000","last_peered":"2023-11-28T18:28:14.663509+0000","last_clean":"2023-11-28T18:28:14.663509+0000","last_became_active":"2023-11-28T18:28:13.129907+0000","last_became_peered":"2023-11-28T18:28:13.129907+0000","last_unstale":"2023-11-28T18:28:14.663509+0000","last_undegraded":"2023-11-28T18:28:14.663509+0000","last_fullsized":"2023-11-28T18:28:14.663509+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:59:27.785719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663486+0000","last_change":"2023-11-28T18:28:13.120612+0000","last_active":"2023-11-28T18:28:14.663486+0000","last_peered":"2023-11-28T18:28:14.663486+0000","last_clean":"2023-11-28T18:28:14.663486+0000","last_became_active":"2023-11-28T18:28:13.119959+0000","last_became_peered":"2023-11-28T18:28:13.119959+0000","last_unstale":"2023-11-28T18:28:14.663486+0000","last_undegraded":"2023-11-28T18:28:14.663486+0000","last_fullsized":"2023-11-28T18:28:14.663486+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:45:53.434006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663801+0000","last_change":"2023-11-28T18:28:13.125680+0000","last_active":"2023-11-28T18:28:14.663801+0000","last_peered":"2023-11-28T18:28:14.663801+0000","last_clean":"2023-11-28T18:28:14.663801+0000","last_became_active":"2023-11-28T18:28:13.125400+0000","last_became_peered":"2023-11-28T18:28:13.125400+0000","last_unstale":"2023-11-28T18:28:14.663801+0000","last_undegraded":"2023-11-28T18:28:14.663801+0000","last_fullsized":"2023-11-28T18:28:14.663801+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:34:22.888088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663761+0000","last_change":"2023-11-28T18:28:13.124552+0000","last_active":"2023-11-28T18:28:14.663761+0000","last_peered":"2023-11-28T18:28:14.663761+0000","last_clean":"2023-11-28T18:28:14.663761+0000","last_became_active":"2023-11-28T18:28:13.124244+0000","last_became_peered":"2023-11-28T18:28:13.124244+0000","last_unstale":"2023-11-28T18:28:14.663761+0000","last_undegraded":"2023-11-28T18:28:14.663761+0000","last_fullsized":"2023-11-28T18:28:14.663761+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:58:39.630155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663768+0000","last_change":"2023-11-28T18:28:13.131556+0000","last_active":"2023-11-28T18:28:14.663768+0000","last_peered":"2023-11-28T18:28:14.663768+0000","last_clean":"2023-11-28T18:28:14.663768+0000","last_became_active":"2023-11-28T18:28:13.131236+0000","last_became_peered":"2023-11-28T18:28:13.131236+0000","last_unstale":"2023-11-28T18:28:14.663768+0000","last_undegraded":"2023-11-28T18:28:14.663768+0000","last_fullsized":"2023-11-28T18:28:14.663768+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:03:43.789843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.662427+0000","last_change":"2023-11-28T18:28:13.127710+0000","last_active":"2023-11-28T18:28:14.662427+0000","last_peered":"2023-11-28T18:28:14.662427+0000","last_clean":"2023-11-28T18:28:14.662427+0000","last_became_active":"2023-11-28T18:28:13.123986+0000","last_became_peered":"2023-11-28T18:28:13.123986+0000","last_unstale":"2023-11-28T18:28:14.662427+0000","last_undegraded":"2023-11-28T18:28:14.662427+0000","last_fullsized":"2023-11-28T18:28:14.662427+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:46:03.063785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663090+0000","last_change":"2023-11-28T18:28:13.128366+0000","last_active":"2023-11-28T18:28:14.663090+0000","last_peered":"2023-11-28T18:28:14.663090+0000","last_clean":"2023-11-28T18:28:14.663090+0000","last_became_active":"2023-11-28T18:28:13.128069+0000","last_became_peered":"2023-11-28T18:28:13.128069+0000","last_unstale":"2023-11-28T18:28:14.663090+0000","last_undegraded":"2023-11-28T18:28:14.663090+0000","last_fullsized":"2023-11-28T18:28:14.663090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:24:46.417354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.662540+0000","last_change":"2023-11-28T18:28:13.127850+0000","last_active":"2023-11-28T18:28:14.662540+0000","last_peered":"2023-11-28T18:28:14.662540+0000","last_clean":"2023-11-28T18:28:14.662540+0000","last_became_active":"2023-11-28T18:28:13.127608+0000","last_became_peered":"2023-11-28T18:28:13.127608+0000","last_unstale":"2023-11-28T18:28:14.662540+0000","last_undegraded":"2023-11-28T18:28:14.662540+0000","last_fullsized":"2023-11-28T18:28:14.662540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:18:05.416644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"77'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.664186+0000","last_change":"2023-11-28T18:28:12.735435+0000","last_active":"2023-11-28T18:28:14.664186+0000","last_peered":"2023-11-28T18:28:14.664186+0000","last_clean":"2023-11-28T18:28:14.664186+0000","last_became_active":"2023-11-28T18:28:12.734785+0000","last_became_peered":"2023-11-28T18:28:12.734785+0000","last_unstale":"2023-11-28T18:28:14.664186+0000","last_undegraded":"2023-11-28T18:28:14.664186+0000","last_fullsized":"2023-11-28T18:28:14.664186+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:55:04.692978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663372+0000","last_change":"2023-11-28T18:28:13.128688+0000","last_active":"2023-11-28T18:28:14.663372+0000","last_peered":"2023-11-28T18:28:14.663372+0000","last_clean":"2023-11-28T18:28:14.663372+0000","last_became_active":"2023-11-28T18:28:13.128356+0000","last_became_peered":"2023-11-28T18:28:13.128356+0000","last_unstale":"2023-11-28T18:28:14.663372+0000","last_undegraded":"2023-11-28T18:28:14.663372+0000","last_fullsized":"2023-11-28T18:28:14.663372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:40.389583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"77'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663672+0000","last_change":"2023-11-28T18:28:12.744737+0000","last_active":"2023-11-28T18:28:14.663672+0000","last_peered":"2023-11-28T18:28:14.663672+0000","last_clean":"2023-11-28T18:28:14.663672+0000","last_became_active":"2023-11-28T18:28:12.744501+0000","last_became_peered":"2023-11-28T18:28:12.744501+0000","last_unstale":"2023-11-28T18:28:14.663672+0000","last_undegraded":"2023-11-28T18:28:14.663672+0000","last_fullsized":"2023-11-28T18:28:14.663672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:54:19.905291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.114668+0000","last_change":"2023-11-28T18:28:13.119877+0000","last_active":"2023-11-28T18:28:14.114668+0000","last_peered":"2023-11-28T18:28:14.114668+0000","last_clean":"2023-11-28T18:28:14.114668+0000","last_became_active":"2023-11-28T18:28:13.119563+0000","last_became_peered":"2023-11-28T18:28:13.119563+0000","last_unstale":"2023-11-28T18:28:14.114668+0000","last_undegraded":"2023-11-28T18:28:14.114668+0000","last_fullsized":"2023-11-28T18:28:14.114668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:09:46.561906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"77'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663495+0000","last_change":"2023-11-28T18:28:12.736812+0000","last_active":"2023-11-28T18:28:14.663495+0000","last_peered":"2023-11-28T18:28:14.663495+0000","last_clean":"2023-11-28T18:28:14.663495+0000","last_became_active":"2023-11-28T18:28:12.736525+0000","last_became_peered":"2023-11-28T18:28:12.736525+0000","last_unstale":"2023-11-28T18:28:14.663495+0000","last_undegraded":"2023-11-28T18:28:14.663495+0000","last_fullsized":"2023-11-28T18:28:14.663495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:13:57.991197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115404+0000","last_change":"2023-11-28T18:28:13.120665+0000","last_active":"2023-11-28T18:28:14.115404+0000","last_peered":"2023-11-28T18:28:14.115404+0000","last_clean":"2023-11-28T18:28:14.115404+0000","last_became_active":"2023-11-28T18:28:13.119260+0000","last_became_peered":"2023-11-28T18:28:13.119260+0000","last_unstale":"2023-11-28T18:28:14.115404+0000","last_undegraded":"2023-11-28T18:28:14.115404+0000","last_fullsized":"2023-11-28T18:28:14.115404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:59:31.174898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"77'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115295+0000","last_change":"2023-11-28T18:28:12.744170+0000","last_active":"2023-11-28T18:28:14.115295+0000","last_peered":"2023-11-28T18:28:14.115295+0000","last_clean":"2023-11-28T18:28:14.115295+0000","last_became_active":"2023-11-28T18:28:12.743538+0000","last_became_peered":"2023-11-28T18:28:12.743538+0000","last_unstale":"2023-11-28T18:28:14.115295+0000","last_undegraded":"2023-11-28T18:28:14.115295+0000","last_fullsized":"2023-11-28T18:28:14.115295+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:05:40.347837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.664018+0000","last_change":"2023-11-28T18:28:13.126337+0000","last_active":"2023-11-28T18:28:14.664018+0000","last_peered":"2023-11-28T18:28:14.664018+0000","last_clean":"2023-11-28T18:28:14.664018+0000","last_became_active":"2023-11-28T18:28:13.126065+0000","last_became_peered":"2023-11-28T18:28:13.126065+0000","last_unstale":"2023-11-28T18:28:14.664018+0000","last_undegraded":"2023-11-28T18:28:14.664018+0000","last_fullsized":"2023-11-28T18:28:14.664018+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:47:05.047585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"77'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115484+0000","last_change":"2023-11-28T18:28:12.737931+0000","last_active":"2023-11-28T18:28:14.115484+0000","last_peered":"2023-11-28T18:28:14.115484+0000","last_clean":"2023-11-28T18:28:14.115484+0000","last_became_active":"2023-11-28T18:28:12.737655+0000","last_became_peered":"2023-11-28T18:28:12.737655+0000","last_unstale":"2023-11-28T18:28:14.115484+0000","last_undegraded":"2023-11-28T18:28:14.115484+0000","last_fullsized":"2023-11-28T18:28:14.115484+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:32.590295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.664358+0000","last_change":"2023-11-28T18:28:13.127929+0000","last_active":"2023-11-28T18:28:14.664358+0000","last_peered":"2023-11-28T18:28:14.664358+0000","last_clean":"2023-11-28T18:28:14.664358+0000","last_became_active":"2023-11-28T18:28:13.126750+0000","last_became_peered":"2023-11-28T18:28:13.126750+0000","last_unstale":"2023-11-28T18:28:14.664358+0000","last_undegraded":"2023-11-28T18:28:14.664358+0000","last_fullsized":"2023-11-28T18:28:14.664358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:23:08.064510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"77'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663145+0000","last_change":"2023-11-28T18:28:12.730687+0000","last_active":"2023-11-28T18:28:14.663145+0000","last_peered":"2023-11-28T18:28:14.663145+0000","last_clean":"2023-11-28T18:28:14.663145+0000","last_became_active":"2023-11-28T18:28:12.730419+0000","last_became_peered":"2023-11-28T18:28:12.730419+0000","last_unstale":"2023-11-28T18:28:14.663145+0000","last_undegraded":"2023-11-28T18:28:14.663145+0000","last_fullsized":"2023-11-28T18:28:14.663145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:52.733948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115624+0000","last_change":"2023-11-28T18:28:13.120442+0000","last_active":"2023-11-28T18:28:14.115624+0000","last_peered":"2023-11-28T18:28:14.115624+0000","last_clean":"2023-11-28T18:28:14.115624+0000","last_became_active":"2023-11-28T18:28:13.120162+0000","last_became_peered":"2023-11-28T18:28:13.120162+0000","last_unstale":"2023-11-28T18:28:14.115624+0000","last_undegraded":"2023-11-28T18:28:14.115624+0000","last_fullsized":"2023-11-28T18:28:14.115624+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:12:18.196017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663452+0000","last_change":"2023-11-28T18:28:12.737562+0000","last_active":"2023-11-28T18:28:14.663452+0000","last_peered":"2023-11-28T18:28:14.663452+0000","last_clean":"2023-11-28T18:28:14.663452+0000","last_became_active":"2023-11-28T18:28:12.737307+0000","last_became_peered":"2023-11-28T18:28:12.737307+0000","last_unstale":"2023-11-28T18:28:14.663452+0000","last_undegraded":"2023-11-28T18:28:14.663452+0000","last_fullsized":"2023-11-28T18:28:14.663452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:35.491338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115587+0000","last_change":"2023-11-28T18:28:13.132048+0000","last_active":"2023-11-28T18:28:14.115587+0000","last_peered":"2023-11-28T18:28:14.115587+0000","last_clean":"2023-11-28T18:28:14.115587+0000","last_became_active":"2023-11-28T18:28:13.131635+0000","last_became_peered":"2023-11-28T18:28:13.131635+0000","last_unstale":"2023-11-28T18:28:14.115587+0000","last_undegraded":"2023-11-28T18:28:14.115587+0000","last_fullsized":"2023-11-28T18:28:14.115587+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:34:08.801304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663061+0000","last_change":"2023-11-28T18:28:12.730227+0000","last_active":"2023-11-28T18:28:14.663061+0000","last_peered":"2023-11-28T18:28:14.663061+0000","last_clean":"2023-11-28T18:28:14.663061+0000","last_became_active":"2023-11-28T18:28:12.729937+0000","last_became_peered":"2023-11-28T18:28:12.729937+0000","last_unstale":"2023-11-28T18:28:14.663061+0000","last_undegraded":"2023-11-28T18:28:14.663061+0000","last_fullsized":"2023-11-28T18:28:14.663061+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:16.536633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663509+0000","last_change":"2023-11-28T18:28:13.121093+0000","last_active":"2023-11-28T18:28:14.663509+0000","last_peered":"2023-11-28T18:28:14.663509+0000","last_clean":"2023-11-28T18:28:14.663509+0000","last_became_active":"2023-11-28T18:28:13.120843+0000","last_became_peered":"2023-11-28T18:28:13.120843+0000","last_unstale":"2023-11-28T18:28:14.663509+0000","last_undegraded":"2023-11-28T18:28:14.663509+0000","last_fullsized":"2023-11-28T18:28:14.663509+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:20.578566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663711+0000","last_change":"2023-11-28T18:28:12.745614+0000","last_active":"2023-11-28T18:28:14.663711+0000","last_peered":"2023-11-28T18:28:14.663711+0000","last_clean":"2023-11-28T18:28:14.663711+0000","last_became_active":"2023-11-28T18:28:12.744975+0000","last_became_peered":"2023-11-28T18:28:12.744975+0000","last_unstale":"2023-11-28T18:28:14.663711+0000","last_undegraded":"2023-11-28T18:28:14.663711+0000","last_fullsized":"2023-11-28T18:28:14.663711+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:49:47.526032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663053+0000","last_change":"2023-11-28T18:28:13.123945+0000","last_active":"2023-11-28T18:28:14.663053+0000","last_peered":"2023-11-28T18:28:14.663053+0000","last_clean":"2023-11-28T18:28:14.663053+0000","last_became_active":"2023-11-28T18:28:13.123569+0000","last_became_peered":"2023-11-28T18:28:13.123569+0000","last_unstale":"2023-11-28T18:28:14.663053+0000","last_undegraded":"2023-11-28T18:28:14.663053+0000","last_fullsized":"2023-11-28T18:28:14.663053+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:11:49.565273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663599+0000","last_change":"2023-11-28T18:28:12.741182+0000","last_active":"2023-11-28T18:28:14.663599+0000","last_peered":"2023-11-28T18:28:14.663599+0000","last_clean":"2023-11-28T18:28:14.663599+0000","last_became_active":"2023-11-28T18:28:12.740321+0000","last_became_peered":"2023-11-28T18:28:12.740321+0000","last_unstale":"2023-11-28T18:28:14.663599+0000","last_undegraded":"2023-11-28T18:28:14.663599+0000","last_fullsized":"2023-11-28T18:28:14.663599+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:36.038991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.111343+0000","last_change":"2023-11-28T18:28:13.109558+0000","last_active":"2023-11-28T18:28:14.111343+0000","last_peered":"2023-11-28T18:28:14.111343+0000","last_clean":"2023-11-28T18:28:14.111343+0000","last_became_active":"2023-11-28T18:28:13.109272+0000","last_became_peered":"2023-11-28T18:28:13.109272+0000","last_unstale":"2023-11-28T18:28:14.111343+0000","last_undegraded":"2023-11-28T18:28:14.111343+0000","last_fullsized":"2023-11-28T18:28:14.111343+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:24:46.417354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"77'3","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663987+0000","last_change":"2023-11-28T18:28:12.741910+0000","last_active":"2023-11-28T18:28:14.663987+0000","last_peered":"2023-11-28T18:28:14.663987+0000","last_clean":"2023-11-28T18:28:14.663987+0000","last_became_active":"2023-11-28T18:28:12.741176+0000","last_became_peered":"2023-11-28T18:28:12.741176+0000","last_unstale":"2023-11-28T18:28:14.663987+0000","last_undegraded":"2023-11-28T18:28:14.663987+0000","last_fullsized":"2023-11-28T18:28:14.663987+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:17.478758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.662424+0000","last_change":"2023-11-28T18:28:13.116799+0000","last_active":"2023-11-28T18:28:14.662424+0000","last_peered":"2023-11-28T18:28:14.662424+0000","last_clean":"2023-11-28T18:28:14.662424+0000","last_became_active":"2023-11-28T18:28:13.116502+0000","last_became_peered":"2023-11-28T18:28:13.116502+0000","last_unstale":"2023-11-28T18:28:14.662424+0000","last_undegraded":"2023-11-28T18:28:14.662424+0000","last_fullsized":"2023-11-28T18:28:14.662424+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:05:41.355019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"77'6","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115384+0000","last_change":"2023-11-28T18:28:12.737200+0000","last_active":"2023-11-28T18:28:14.115384+0000","last_peered":"2023-11-28T18:28:14.115384+0000","last_clean":"2023-11-28T18:28:14.115384+0000","last_became_active":"2023-11-28T18:28:12.736912+0000","last_became_peered":"2023-11-28T18:28:12.736912+0000","last_unstale":"2023-11-28T18:28:14.115384+0000","last_undegraded":"2023-11-28T18:28:14.115384+0000","last_fullsized":"2023-11-28T18:28:14.115384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:46:02.056603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663736+0000","last_change":"2023-11-28T18:28:13.125754+0000","last_active":"2023-11-28T18:28:14.663736+0000","last_peered":"2023-11-28T18:28:14.663736+0000","last_clean":"2023-11-28T18:28:14.663736+0000","last_became_active":"2023-11-28T18:28:13.125408+0000","last_became_peered":"2023-11-28T18:28:13.125408+0000","last_unstale":"2023-11-28T18:28:14.663736+0000","last_undegraded":"2023-11-28T18:28:14.663736+0000","last_fullsized":"2023-11-28T18:28:14.663736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:59:56.917810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"77'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663042+0000","last_change":"2023-11-28T18:28:12.730086+0000","last_active":"2023-11-28T18:28:14.663042+0000","last_peered":"2023-11-28T18:28:14.663042+0000","last_clean":"2023-11-28T18:28:14.663042+0000","last_became_active":"2023-11-28T18:28:12.729786+0000","last_became_peered":"2023-11-28T18:28:12.729786+0000","last_unstale":"2023-11-28T18:28:14.663042+0000","last_undegraded":"2023-11-28T18:28:14.663042+0000","last_fullsized":"2023-11-28T18:28:14.663042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:34:07.794122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115479+0000","last_change":"2023-11-28T18:28:13.122426+0000","last_active":"2023-11-28T18:28:14.115479+0000","last_peered":"2023-11-28T18:28:14.115479+0000","last_clean":"2023-11-28T18:28:14.115479+0000","last_became_active":"2023-11-28T18:28:13.122095+0000","last_became_peered":"2023-11-28T18:28:13.122095+0000","last_unstale":"2023-11-28T18:28:14.115479+0000","last_undegraded":"2023-11-28T18:28:14.115479+0000","last_fullsized":"2023-11-28T18:28:14.115479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:39:23.857851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663871+0000","last_change":"2023-11-28T18:28:12.738549+0000","last_active":"2023-11-28T18:28:14.663871+0000","last_peered":"2023-11-28T18:28:14.663871+0000","last_clean":"2023-11-28T18:28:14.663871+0000","last_became_active":"2023-11-28T18:28:12.738052+0000","last_became_peered":"2023-11-28T18:28:12.738052+0000","last_unstale":"2023-11-28T18:28:14.663871+0000","last_undegraded":"2023-11-28T18:28:14.663871+0000","last_fullsized":"2023-11-28T18:28:14.663871+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:11:48.558091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115446+0000","last_change":"2023-11-28T18:28:13.127268+0000","last_active":"2023-11-28T18:28:14.115446+0000","last_peered":"2023-11-28T18:28:14.115446+0000","last_clean":"2023-11-28T18:28:14.115446+0000","last_became_active":"2023-11-28T18:28:13.126932+0000","last_became_peered":"2023-11-28T18:28:13.126932+0000","last_unstale":"2023-11-28T18:28:14.115446+0000","last_undegraded":"2023-11-28T18:28:14.115446+0000","last_fullsized":"2023-11-28T18:28:14.115446+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:40:16.599031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663205+0000","last_change":"2023-11-28T18:28:12.733555+0000","last_active":"2023-11-28T18:28:14.663205+0000","last_peered":"2023-11-28T18:28:14.663205+0000","last_clean":"2023-11-28T18:28:14.663205+0000","last_became_active":"2023-11-28T18:28:12.733285+0000","last_became_peered":"2023-11-28T18:28:12.733285+0000","last_unstale":"2023-11-28T18:28:14.663205+0000","last_undegraded":"2023-11-28T18:28:14.663205+0000","last_fullsized":"2023-11-28T18:28:14.663205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:58.327464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115620+0000","last_change":"2023-11-28T18:28:13.129911+0000","last_active":"2023-11-28T18:28:14.115620+0000","last_peered":"2023-11-28T18:28:14.115620+0000","last_clean":"2023-11-28T18:28:14.115620+0000","last_became_active":"2023-11-28T18:28:13.129624+0000","last_became_peered":"2023-11-28T18:28:13.129624+0000","last_unstale":"2023-11-28T18:28:14.115620+0000","last_undegraded":"2023-11-28T18:28:14.115620+0000","last_fullsized":"2023-11-28T18:28:14.115620+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:21:11.948500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"77'8","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663549+0000","last_change":"2023-11-28T18:28:12.738810+0000","last_active":"2023-11-28T18:28:14.663549+0000","last_peered":"2023-11-28T18:28:14.663549+0000","last_clean":"2023-11-28T18:28:14.663549+0000","last_became_active":"2023-11-28T18:28:12.736642+0000","last_became_peered":"2023-11-28T18:28:12.736642+0000","last_unstale":"2023-11-28T18:28:14.663549+0000","last_undegraded":"2023-11-28T18:28:14.663549+0000","last_fullsized":"2023-11-28T18:28:14.663549+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:13:57.991197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.111084+0000","last_change":"2023-11-28T18:28:13.121376+0000","last_active":"2023-11-28T18:28:14.111084+0000","last_peered":"2023-11-28T18:28:14.111084+0000","last_clean":"2023-11-28T18:28:14.111084+0000","last_became_active":"2023-11-28T18:28:13.120124+0000","last_became_peered":"2023-11-28T18:28:13.120124+0000","last_unstale":"2023-11-28T18:28:14.111084+0000","last_undegraded":"2023-11-28T18:28:14.111084+0000","last_fullsized":"2023-11-28T18:28:14.111084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:38:27.289444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"77'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663753+0000","last_change":"2023-11-28T18:28:12.743909+0000","last_active":"2023-11-28T18:28:14.663753+0000","last_peered":"2023-11-28T18:28:14.663753+0000","last_clean":"2023-11-28T18:28:14.663753+0000","last_became_active":"2023-11-28T18:28:12.743628+0000","last_became_peered":"2023-11-28T18:28:12.743628+0000","last_unstale":"2023-11-28T18:28:14.663753+0000","last_undegraded":"2023-11-28T18:28:14.663753+0000","last_fullsized":"2023-11-28T18:28:14.663753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:59:55.910629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663207+0000","last_change":"2023-11-28T18:28:13.123446+0000","last_active":"2023-11-28T18:28:14.663207+0000","last_peered":"2023-11-28T18:28:14.663207+0000","last_clean":"2023-11-28T18:28:14.663207+0000","last_became_active":"2023-11-28T18:28:13.123190+0000","last_became_peered":"2023-11-28T18:28:13.123190+0000","last_unstale":"2023-11-28T18:28:14.663207+0000","last_undegraded":"2023-11-28T18:28:14.663207+0000","last_fullsized":"2023-11-28T18:28:14.663207+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:22.541327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"77'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115733+0000","last_change":"2023-11-28T18:28:12.737644+0000","last_active":"2023-11-28T18:28:14.115733+0000","last_peered":"2023-11-28T18:28:14.115733+0000","last_clean":"2023-11-28T18:28:14.115733+0000","last_became_active":"2023-11-28T18:28:12.737378+0000","last_became_peered":"2023-11-28T18:28:12.737378+0000","last_unstale":"2023-11-28T18:28:14.115733+0000","last_undegraded":"2023-11-28T18:28:14.115733+0000","last_fullsized":"2023-11-28T18:28:14.115733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:42:42.913425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663252+0000","last_change":"2023-11-28T18:28:13.129592+0000","last_active":"2023-11-28T18:28:14.663252+0000","last_peered":"2023-11-28T18:28:14.663252+0000","last_clean":"2023-11-28T18:28:14.663252+0000","last_became_active":"2023-11-28T18:28:13.129282+0000","last_became_peered":"2023-11-28T18:28:13.129282+0000","last_unstale":"2023-11-28T18:28:14.663252+0000","last_undegraded":"2023-11-28T18:28:14.663252+0000","last_fullsized":"2023-11-28T18:28:14.663252+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:14.218187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"77'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663999+0000","last_change":"2023-11-28T18:28:12.732492+0000","last_active":"2023-11-28T18:28:14.663999+0000","last_peered":"2023-11-28T18:28:14.663999+0000","last_clean":"2023-11-28T18:28:14.663999+0000","last_became_active":"2023-11-28T18:28:12.732176+0000","last_became_peered":"2023-11-28T18:28:12.732176+0000","last_unstale":"2023-11-28T18:28:14.663999+0000","last_undegraded":"2023-11-28T18:28:14.663999+0000","last_fullsized":"2023-11-28T18:28:14.663999+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:01:13.221094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.111233+0000","last_change":"2023-11-28T18:28:13.128711+0000","last_active":"2023-11-28T18:28:14.111233+0000","last_peered":"2023-11-28T18:28:14.111233+0000","last_clean":"2023-11-28T18:28:14.111233+0000","last_became_active":"2023-11-28T18:28:13.128451+0000","last_became_peered":"2023-11-28T18:28:13.128451+0000","last_unstale":"2023-11-28T18:28:14.111233+0000","last_undegraded":"2023-11-28T18:28:14.111233+0000","last_fullsized":"2023-11-28T18:28:14.111233+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:31:15.879055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.662344+0000","last_change":"2023-11-28T18:28:12.739945+0000","last_active":"2023-11-28T18:28:14.662344+0000","last_peered":"2023-11-28T18:28:14.662344+0000","last_clean":"2023-11-28T18:28:14.662344+0000","last_became_active":"2023-11-28T18:28:12.738948+0000","last_became_peered":"2023-11-28T18:28:12.738948+0000","last_unstale":"2023-11-28T18:28:14.662344+0000","last_undegraded":"2023-11-28T18:28:14.662344+0000","last_fullsized":"2023-11-28T18:28:14.662344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:59:30.167717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.664040+0000","last_change":"2023-11-28T18:28:13.124800+0000","last_active":"2023-11-28T18:28:14.664040+0000","last_peered":"2023-11-28T18:28:14.664040+0000","last_clean":"2023-11-28T18:28:14.664040+0000","last_became_active":"2023-11-28T18:28:13.124537+0000","last_became_peered":"2023-11-28T18:28:13.124537+0000","last_unstale":"2023-11-28T18:28:14.664040+0000","last_undegraded":"2023-11-28T18:28:14.664040+0000","last_fullsized":"2023-11-28T18:28:14.664040+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:10:43.594536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"77'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.111317+0000","last_change":"2023-11-28T18:28:12.737759+0000","last_active":"2023-11-28T18:28:14.111317+0000","last_peered":"2023-11-28T18:28:14.111317+0000","last_clean":"2023-11-28T18:28:14.111317+0000","last_became_active":"2023-11-28T18:28:12.737440+0000","last_became_peered":"2023-11-28T18:28:12.737440+0000","last_unstale":"2023-11-28T18:28:14.111317+0000","last_undegraded":"2023-11-28T18:28:14.111317+0000","last_fullsized":"2023-11-28T18:28:14.111317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:49:47.526032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.664396+0000","last_change":"2023-11-28T18:28:13.125784+0000","last_active":"2023-11-28T18:28:14.664396+0000","last_peered":"2023-11-28T18:28:14.664396+0000","last_clean":"2023-11-28T18:28:14.664396+0000","last_became_active":"2023-11-28T18:28:13.125077+0000","last_became_peered":"2023-11-28T18:28:13.125077+0000","last_unstale":"2023-11-28T18:28:14.664396+0000","last_undegraded":"2023-11-28T18:28:14.664396+0000","last_fullsized":"2023-11-28T18:28:14.664396+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:29:46.319741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.111425+0000","last_change":"2023-11-28T18:28:12.738291+0000","last_active":"2023-11-28T18:28:14.111425+0000","last_peered":"2023-11-28T18:28:14.111425+0000","last_clean":"2023-11-28T18:28:14.111425+0000","last_became_active":"2023-11-28T18:28:12.738025+0000","last_became_peered":"2023-11-28T18:28:12.738025+0000","last_unstale":"2023-11-28T18:28:14.111425+0000","last_undegraded":"2023-11-28T18:28:14.111425+0000","last_fullsized":"2023-11-28T18:28:14.111425+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:36.038991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115723+0000","last_change":"2023-11-28T18:28:13.128349+0000","last_active":"2023-11-28T18:28:14.115723+0000","last_peered":"2023-11-28T18:28:14.115723+0000","last_clean":"2023-11-28T18:28:14.115723+0000","last_became_active":"2023-11-28T18:28:13.127785+0000","last_became_peered":"2023-11-28T18:28:13.127785+0000","last_unstale":"2023-11-28T18:28:14.115723+0000","last_undegraded":"2023-11-28T18:28:14.115723+0000","last_fullsized":"2023-11-28T18:28:14.115723+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:48:57.176966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.111021+0000","last_change":"2023-11-28T18:28:12.738174+0000","last_active":"2023-11-28T18:28:14.111021+0000","last_peered":"2023-11-28T18:28:14.111021+0000","last_clean":"2023-11-28T18:28:14.111021+0000","last_became_active":"2023-11-28T18:28:12.737868+0000","last_became_peered":"2023-11-28T18:28:12.737868+0000","last_unstale":"2023-11-28T18:28:14.111021+0000","last_undegraded":"2023-11-28T18:28:14.111021+0000","last_fullsized":"2023-11-28T18:28:14.111021+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:15.796896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.114519+0000","last_change":"2023-11-28T18:28:13.126032+0000","last_active":"2023-11-28T18:28:14.114519+0000","last_peered":"2023-11-28T18:28:14.114519+0000","last_clean":"2023-11-28T18:28:14.114519+0000","last_became_active":"2023-11-28T18:28:13.124282+0000","last_became_peered":"2023-11-28T18:28:13.124282+0000","last_unstale":"2023-11-28T18:28:14.114519+0000","last_undegraded":"2023-11-28T18:28:14.114519+0000","last_fullsized":"2023-11-28T18:28:14.114519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:16:51.464199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"77'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663244+0000","last_change":"2023-11-28T18:28:12.742105+0000","last_active":"2023-11-28T18:28:14.663244+0000","last_peered":"2023-11-28T18:28:14.663244+0000","last_clean":"2023-11-28T18:28:14.663244+0000","last_became_active":"2023-11-28T18:28:12.741872+0000","last_became_peered":"2023-11-28T18:28:12.741872+0000","last_unstale":"2023-11-28T18:28:14.663244+0000","last_undegraded":"2023-11-28T18:28:14.663244+0000","last_fullsized":"2023-11-28T18:28:14.663244+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:15.796896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.114904+0000","last_change":"2023-11-28T18:28:13.129538+0000","last_active":"2023-11-28T18:28:14.114904+0000","last_peered":"2023-11-28T18:28:14.114904+0000","last_clean":"2023-11-28T18:28:14.114904+0000","last_became_active":"2023-11-28T18:28:13.129101+0000","last_became_peered":"2023-11-28T18:28:13.129101+0000","last_unstale":"2023-11-28T18:28:14.114904+0000","last_undegraded":"2023-11-28T18:28:14.114904+0000","last_fullsized":"2023-11-28T18:28:14.114904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:18:05.416644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.664416+0000","last_change":"2023-11-28T18:28:12.736722+0000","last_active":"2023-11-28T18:28:14.664416+0000","last_peered":"2023-11-28T18:28:14.664416+0000","last_clean":"2023-11-28T18:28:14.664416+0000","last_became_active":"2023-11-28T18:28:12.735717+0000","last_became_peered":"2023-11-28T18:28:12.735717+0000","last_unstale":"2023-11-28T18:28:14.664416+0000","last_undegraded":"2023-11-28T18:28:14.664416+0000","last_fullsized":"2023-11-28T18:28:14.664416+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:03:42.782662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.114903+0000","last_change":"2023-11-28T18:28:13.112590+0000","last_active":"2023-11-28T18:28:14.114903+0000","last_peered":"2023-11-28T18:28:14.114903+0000","last_clean":"2023-11-28T18:28:14.114903+0000","last_became_active":"2023-11-28T18:28:13.111238+0000","last_became_peered":"2023-11-28T18:28:13.111238+0000","last_unstale":"2023-11-28T18:28:14.114903+0000","last_undegraded":"2023-11-28T18:28:14.114903+0000","last_fullsized":"2023-11-28T18:28:14.114903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:01.297558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663590+0000","last_change":"2023-11-28T18:28:12.737920+0000","last_active":"2023-11-28T18:28:14.663590+0000","last_peered":"2023-11-28T18:28:14.663590+0000","last_clean":"2023-11-28T18:28:14.663590+0000","last_became_active":"2023-11-28T18:28:12.737653+0000","last_became_peered":"2023-11-28T18:28:12.737653+0000","last_unstale":"2023-11-28T18:28:14.663590+0000","last_undegraded":"2023-11-28T18:28:14.663590+0000","last_fullsized":"2023-11-28T18:28:14.663590+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:31.845136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663760+0000","last_change":"2023-11-28T18:28:13.125558+0000","last_active":"2023-11-28T18:28:14.663760+0000","last_peered":"2023-11-28T18:28:14.663760+0000","last_clean":"2023-11-28T18:28:14.663760+0000","last_became_active":"2023-11-28T18:28:13.125276+0000","last_became_peered":"2023-11-28T18:28:13.125276+0000","last_unstale":"2023-11-28T18:28:14.663760+0000","last_undegraded":"2023-11-28T18:28:14.663760+0000","last_fullsized":"2023-11-28T18:28:14.663760+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:02.561354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663368+0000","last_change":"2023-11-28T18:28:12.745317+0000","last_active":"2023-11-28T18:28:14.663368+0000","last_peered":"2023-11-28T18:28:14.663368+0000","last_clean":"2023-11-28T18:28:14.663368+0000","last_became_active":"2023-11-28T18:28:12.744923+0000","last_became_peered":"2023-11-28T18:28:12.744923+0000","last_unstale":"2023-11-28T18:28:14.663368+0000","last_undegraded":"2023-11-28T18:28:14.663368+0000","last_fullsized":"2023-11-28T18:28:14.663368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:44:26.636503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115213+0000","last_change":"2023-11-28T18:28:13.129945+0000","last_active":"2023-11-28T18:28:14.115213+0000","last_peered":"2023-11-28T18:28:14.115213+0000","last_clean":"2023-11-28T18:28:14.115213+0000","last_became_active":"2023-11-28T18:28:13.129625+0000","last_became_peered":"2023-11-28T18:28:13.129625+0000","last_unstale":"2023-11-28T18:28:14.115213+0000","last_undegraded":"2023-11-28T18:28:14.115213+0000","last_fullsized":"2023-11-28T18:28:14.115213+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:19.143572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115796+0000","last_change":"2023-11-28T18:28:12.740986+0000","last_active":"2023-11-28T18:28:14.115796+0000","last_peered":"2023-11-28T18:28:14.115796+0000","last_clean":"2023-11-28T18:28:14.115796+0000","last_became_active":"2023-11-28T18:28:12.740621+0000","last_became_peered":"2023-11-28T18:28:12.740621+0000","last_unstale":"2023-11-28T18:28:14.115796+0000","last_undegraded":"2023-11-28T18:28:14.115796+0000","last_fullsized":"2023-11-28T18:28:14.115796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:12:17.188835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115275+0000","last_change":"2023-11-28T18:28:13.130454+0000","last_active":"2023-11-28T18:28:14.115275+0000","last_peered":"2023-11-28T18:28:14.115275+0000","last_clean":"2023-11-28T18:28:14.115275+0000","last_became_active":"2023-11-28T18:28:13.130135+0000","last_became_peered":"2023-11-28T18:28:13.130135+0000","last_unstale":"2023-11-28T18:28:14.115275+0000","last_undegraded":"2023-11-28T18:28:14.115275+0000","last_fullsized":"2023-11-28T18:28:14.115275+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:55:56.581933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663532+0000","last_change":"2023-11-28T18:28:12.732044+0000","last_active":"2023-11-28T18:28:14.663532+0000","last_peered":"2023-11-28T18:28:14.663532+0000","last_clean":"2023-11-28T18:28:14.663532+0000","last_became_active":"2023-11-28T18:28:12.731779+0000","last_became_peered":"2023-11-28T18:28:12.731779+0000","last_unstale":"2023-11-28T18:28:14.663532+0000","last_undegraded":"2023-11-28T18:28:14.663532+0000","last_fullsized":"2023-11-28T18:28:14.663532+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:11:29.567488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.662740+0000","last_change":"2023-11-28T18:28:13.118250+0000","last_active":"2023-11-28T18:28:14.662740+0000","last_peered":"2023-11-28T18:28:14.662740+0000","last_clean":"2023-11-28T18:28:14.662740+0000","last_became_active":"2023-11-28T18:28:13.117939+0000","last_became_peered":"2023-11-28T18:28:13.117939+0000","last_unstale":"2023-11-28T18:28:14.662740+0000","last_undegraded":"2023-11-28T18:28:14.662740+0000","last_fullsized":"2023-11-28T18:28:14.662740+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:55:56.581933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115013+0000","last_change":"2023-11-28T18:28:12.744032+0000","last_active":"2023-11-28T18:28:14.115013+0000","last_peered":"2023-11-28T18:28:14.115013+0000","last_clean":"2023-11-28T18:28:14.115013+0000","last_became_active":"2023-11-28T18:28:12.743774+0000","last_became_peered":"2023-11-28T18:28:12.743774+0000","last_unstale":"2023-11-28T18:28:14.115013+0000","last_undegraded":"2023-11-28T18:28:14.115013+0000","last_fullsized":"2023-11-28T18:28:14.115013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:37.205648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.662749+0000","last_change":"2023-11-28T18:28:13.123603+0000","last_active":"2023-11-28T18:28:14.662749+0000","last_peered":"2023-11-28T18:28:14.662749+0000","last_clean":"2023-11-28T18:28:14.662749+0000","last_became_active":"2023-11-28T18:28:13.123068+0000","last_became_peered":"2023-11-28T18:28:13.123068+0000","last_unstale":"2023-11-28T18:28:14.662749+0000","last_undegraded":"2023-11-28T18:28:14.662749+0000","last_fullsized":"2023-11-28T18:28:14.662749+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:40:16.599031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.662271+0000","last_change":"2023-11-28T18:28:12.739581+0000","last_active":"2023-11-28T18:28:14.662271+0000","last_peered":"2023-11-28T18:28:14.662271+0000","last_clean":"2023-11-28T18:28:14.662271+0000","last_became_active":"2023-11-28T18:28:12.739164+0000","last_became_peered":"2023-11-28T18:28:12.739164+0000","last_unstale":"2023-11-28T18:28:14.662271+0000","last_undegraded":"2023-11-28T18:28:14.662271+0000","last_fullsized":"2023-11-28T18:28:14.662271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:12:17.188835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663581+0000","last_change":"2023-11-28T18:28:13.120806+0000","last_active":"2023-11-28T18:28:14.663581+0000","last_peered":"2023-11-28T18:28:14.663581+0000","last_clean":"2023-11-28T18:28:14.663581+0000","last_became_active":"2023-11-28T18:28:13.120332+0000","last_became_peered":"2023-11-28T18:28:13.120332+0000","last_unstale":"2023-11-28T18:28:14.663581+0000","last_undegraded":"2023-11-28T18:28:14.663581+0000","last_fullsized":"2023-11-28T18:28:14.663581+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:02:34.345986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.111468+0000","last_change":"2023-11-28T18:28:12.737240+0000","last_active":"2023-11-28T18:28:14.111468+0000","last_peered":"2023-11-28T18:28:14.111468+0000","last_clean":"2023-11-28T18:28:14.111468+0000","last_became_active":"2023-11-28T18:28:12.736798+0000","last_became_peered":"2023-11-28T18:28:12.736798+0000","last_unstale":"2023-11-28T18:28:14.111468+0000","last_undegraded":"2023-11-28T18:28:14.111468+0000","last_fullsized":"2023-11-28T18:28:14.111468+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:59:55.910629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":215,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-28T18:28:13.109823+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:28:13.109823+0000","last_peered":"2023-11-28T18:28:13.109823+0000","last_clean":"2023-11-28T18:28:13.109823+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:28:13.109823+0000","last_undegraded":"2023-11-28T18:28:13.109823+0000","last_fullsized":"2023-11-28T18:28:13.109823+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41016,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330824,"statfs":{"total":96636764160,"available":96594763776,"internally_reserved":0,"allocated":1552384,"data_stored":1027920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.65300000000000002,"15min":0.71699999999999997},"min":{"1min":0.39100000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.84799999999999998,"5min":2.427,"15min":2.427},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.66300000000000003,"15min":0.71999999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80700000000000005,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.93500000000000005}]},{"osd":1,"last update":"Tue Nov 28 18:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.67300000000000004,"15min":0.71899999999999997},"min":{"1min":0.41599999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76100000000000001,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.69699999999999995,"15min":0.755},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.76200000000000001,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.079}]},{"osd":2,"last update":"Tue Nov 28 18:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.71499999999999997,"15min":0.77400000000000002},"min":{"1min":0.439,"5min":0.379,"15min":0.379},"max":{"1min":0.877,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.998},{"interface":"front","average":{"1min":0.629,"5min":0.68799999999999994,"15min":0.74199999999999999},"min":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85499999999999998,"5min":2.411,"15min":2.411},"last":0.97299999999999998}]},{"osd":3,"last update":"Tue Nov 28 18:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.70299999999999996,"15min":0.76300000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81499999999999995,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.749,"15min":0.80400000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.85099999999999998,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.0389999999999999}]},{"osd":4,"last update":"Tue Nov 28 18:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58799999999999997,"15min":0.60399999999999998},"min":{"1min":0.34799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.752,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.60299999999999998,"15min":0.64100000000000001},"min":{"1min":0.33100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.83099999999999996,"5min":0.877,"15min":0.877},"last":0.64700000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65000000000000002,"15min":0.67700000000000005},"min":{"1min":0.38200000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83699999999999997,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66600000000000004,"15min":0.70199999999999996},"min":{"1min":0.39900000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93999999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.80400000000000005}]},{"osd":6,"last update":"Tue Nov 28 18:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.70499999999999996,"15min":0.74199999999999999},"min":{"1min":0.44600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.90000000000000002,"5min":1.042,"15min":1.042},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.68000000000000005,"15min":0.73299999999999998},"min":{"1min":0.36699999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80000000000000004,"5min":1.006,"15min":1.006},"last":0.77800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":966656,"data_stored":439142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.35499999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.84899999999999998,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70899999999999996,"15min":0.72899999999999998},"min":{"1min":0.498,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.93000000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.49299999999999999}]},{"osd":1,"last update":"Tue Nov 28 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.72099999999999997,"15min":0.80700000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81399999999999995,"5min":4.2389999999999999,"15min":4.2389999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.72899999999999998,"15min":0.79300000000000004},"min":{"1min":0.38200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.86599999999999999,"5min":4.218,"15min":4.218},"last":0.66400000000000003}]},{"osd":2,"last update":"Tue Nov 28 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.68300000000000005,"15min":0.71499999999999997},"min":{"1min":0.47499999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79500000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.68600000000000005,"15min":0.70299999999999996},"min":{"1min":0.42999999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.75700000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.64300000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.69599999999999995,"15min":0.72399999999999998},"min":{"1min":0.432,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.84499999999999997,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67900000000000005,"15min":0.68700000000000006},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83699999999999997,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.78900000000000003}]},{"osd":4,"last update":"Tue Nov 28 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.64400000000000002,"15min":0.66900000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89000000000000001,"5min":0.995,"15min":0.995},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.65000000000000002,"15min":0.68799999999999994},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.88200000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.81200000000000006}]},{"osd":5,"last update":"Tue Nov 28 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64800000000000002,"15min":0.68100000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82899999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.72299999999999998,"15min":0.76100000000000001},"min":{"1min":0.49199999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.871,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.84399999999999997}]},{"osd":7,"last update":"Tue Nov 28 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.73399999999999999,"15min":0.77400000000000002},"min":{"1min":0.495,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91000000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.72499999999999998,"15min":0.753},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1,"5min":1,"15min":1},"last":0.86399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":958464,"data_stored":437564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.69499999999999995,"15min":0.71199999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.75600000000000001,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.72499999999999998,"15min":0.754},"min":{"1min":0.50900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89400000000000002,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":0.61799999999999999}]},{"osd":1,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.70599999999999996,"15min":0.74199999999999999},"min":{"1min":0.44500000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80300000000000005,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.74199999999999999,"15min":0.80000000000000004},"min":{"1min":0.40200000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84599999999999997,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.77300000000000002}]},{"osd":2,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.70799999999999996,"15min":0.75900000000000001},"min":{"1min":0.42899999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77800000000000002,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.71799999999999997,"15min":0.77500000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82599999999999996,"5min":4.0149999999999997,"15min":4.0149999999999997},"last":0.63500000000000001}]},{"osd":3,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.76100000000000001,"15min":0.82799999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.86899999999999999,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.753,"15min":0.82799999999999996},"min":{"1min":0.46899999999999997,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.90100000000000002,"5min":3.9900000000000002,"15min":3.9900000000000002},"last":0.56999999999999995}]},{"osd":4,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.88200000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.497},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.68700000000000006,"15min":0.70899999999999996},"min":{"1min":0.35199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.875,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.59499999999999997}]},{"osd":6,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68300000000000005,"15min":0.68200000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.95899999999999996,"5min":1.071,"15min":1.071},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71199999999999997,"15min":0.71999999999999997},"min":{"1min":0.49199999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.92400000000000004,"5min":1.756,"15min":1.756},"last":0.54900000000000004}]},{"osd":7,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.749,"15min":0.75700000000000001},"min":{"1min":0.52000000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94999999999999996,"5min":1.282,"15min":1.282},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.753,"15min":0.75800000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.97399999999999998,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.47399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36292,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335548,"statfs":{"total":96636764160,"available":96599601152,"internally_reserved":0,"allocated":974848,"data_stored":439246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65900000000000003,"15min":0.69099999999999995},"min":{"1min":0.33900000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.91200000000000003,"5min":2.363,"15min":2.363},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64000000000000001,"15min":0.64600000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85299999999999998,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.97699999999999998}]},{"osd":1,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63400000000000001,"15min":0.624},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.92600000000000005,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.443,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.79400000000000004}]},{"osd":2,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70699999999999996,"15min":0.69699999999999995},"min":{"1min":0.45800000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.871},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71599999999999997,"15min":0.72799999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91000000000000003,"5min":1.871,"15min":1.871},"last":0.89300000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64800000000000002,"15min":0.64700000000000002},"min":{"1min":0.47399999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.48999999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.97199999999999998,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.82299999999999995}]},{"osd":5,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59099999999999997,"15min":0.58599999999999997},"min":{"1min":0.39200000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82399999999999995,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.60699999999999998,"15min":0.63500000000000001},"min":{"1min":0.41699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79200000000000004,"5min":1.79,"15min":1.79},"last":0.77200000000000002}]},{"osd":6,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.624,"15min":0.64500000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95799999999999996,"5min":1.123,"15min":1.123},"last":0.75},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.68500000000000005,"15min":0.70499999999999996},"min":{"1min":0.47299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93999999999999995,"5min":1.925,"15min":1.925},"last":1.004}]},{"osd":7,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68799999999999994,"15min":0.71599999999999997},"min":{"1min":0.41999999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.97999999999999998,"5min":1.843,"15min":1.843},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.75600000000000001,"15min":0.77700000000000002},"min":{"1min":0.59699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.998,"5min":1.593,"15min":1.593},"last":0.96299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574910,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58496,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313344,"statfs":{"total":96636764160,"available":96576864256,"internally_reserved":0,"allocated":958464,"data_stored":438112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.434,"15min":0.41699999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.48599999999999999,"15min":0.47399999999999998},"min":{"1min":0.30199999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.7080000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.34200000000000003}]},{"osd":1,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46100000000000002,"15min":0.46400000000000002},"min":{"1min":0.32300000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.57399999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.54900000000000004,"15min":0.51700000000000002},"min":{"1min":0.249,"5min":0.247,"15min":0.247},"max":{"1min":4.5599999999999996,"5min":4.5599999999999996,"15min":4.5599999999999996},"last":0.48399999999999999}]},{"osd":2,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.34699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.57399999999999995,"15min":0.56499999999999995},"min":{"1min":0.40400000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.45500000000000002}]},{"osd":4,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":0.90900000000000003,"15min":0.81999999999999995},"min":{"1min":0.47899999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":12.695,"5min":12.695,"15min":12.695},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.097,"5min":0.82699999999999996,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":6.3250000000000002,"5min":6.3250000000000002,"15min":6.3250000000000002},"last":0.751}]},{"osd":5,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.81200000000000006,"15min":0.83599999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.9430000000000001,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.81000000000000005,"15min":0.78600000000000003},"min":{"1min":0.504,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.73899999999999999}]},{"osd":6,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":0.84799999999999998,"15min":0.81899999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":6.1790000000000003,"5min":6.1790000000000003,"15min":6.1790000000000003},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.80900000000000005,"15min":0.80700000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":0.70799999999999996}]},{"osd":7,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.82499999999999996,"15min":0.82299999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":3.1709999999999998,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.0920000000000001,"5min":0.85599999999999998,"15min":0.82799999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":6.2249999999999996,"5min":6.2249999999999996,"15min":6.2249999999999996},"last":0.72499999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574911,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":966656,"data_stored":438272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:27:28 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.47499999999999998,"15min":0.499},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61399999999999999,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47699999999999998,"15min":0.505},"min":{"1min":0.30599999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.61799999999999999,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.41999999999999998}]},{"osd":1,"last update":"Tue Nov 28 18:27:28 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.502,"15min":0.52400000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65800000000000003,"5min":1.48,"15min":1.48},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.50700000000000001,"15min":0.51400000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69099999999999995,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.54200000000000004}]},{"osd":3,"last update":"Tue Nov 28 18:27:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48399999999999999,"15min":0.501},"min":{"1min":0.34000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.626,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55000000000000004,"15min":0.56699999999999995},"min":{"1min":0.34899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67800000000000005,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.45600000000000002}]},{"osd":4,"last update":"Tue Nov 28 18:27:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.77200000000000002},"min":{"1min":0.61399999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.032,"5min":1.512,"15min":1.512},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.75,"15min":0.77600000000000002},"min":{"1min":0.54000000000000004,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.96499999999999997,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.81399999999999995}]},{"osd":5,"last update":"Tue Nov 28 18:27:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73699999999999999,"15min":0.76300000000000001},"min":{"1min":0.52000000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93899999999999995,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.873,"15min":0.99199999999999999},"min":{"1min":0.52500000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97799999999999998,"5min":7.4820000000000002,"15min":7.4820000000000002},"last":0.80300000000000005}]},{"osd":6,"last update":"Tue Nov 28 18:27:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.76900000000000002,"15min":0.79500000000000004},"min":{"1min":0.58799999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.97899999999999998,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.79500000000000004,"15min":0.86099999999999999},"min":{"1min":0.55200000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.99299999999999999,"5min":3.9380000000000002,"15min":3.9380000000000002},"last":0.79100000000000004}]},{"osd":7,"last update":"Tue Nov 28 18:27:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.80500000000000005,"15min":0.83699999999999997},"min":{"1min":0.56699999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.022,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.82699999999999996,"15min":0.90900000000000003},"min":{"1min":0.51000000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0549999999999999,"5min":4.798,"15min":4.798},"last":0.83099999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36280,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335560,"statfs":{"total":96636764160,"available":96599613440,"internally_reserved":0,"allocated":962560,"data_stored":437642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.40999999999999998,"15min":0.42399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58599999999999997,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.44600000000000001,"15min":0.45000000000000001},"min":{"1min":0.27300000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.47499999999999998}]},{"osd":2,"last update":"Tue Nov 28 18:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.443,"15min":0.44800000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.59399999999999997,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47899999999999998,"15min":0.47799999999999998},"min":{"1min":0.36499999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.70499999999999996,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.73399999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52700000000000002,"15min":0.53700000000000003},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.749,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.499,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.35699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73399999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54800000000000004}]},{"osd":4,"last update":"Tue Nov 28 18:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.72699999999999998,"15min":0.747},"min":{"1min":0.441,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.004,"5min":1.512,"15min":1.512},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.752,"15min":0.76800000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0780000000000001,"5min":1.155,"15min":1.155},"last":0.59199999999999997}]},{"osd":5,"last update":"Tue Nov 28 18:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.71899999999999997,"15min":0.746},"min":{"1min":0.46999999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.872,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71399999999999997,"15min":0.745},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.90200000000000002,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.67700000000000005}]},{"osd":6,"last update":"Tue Nov 28 18:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.751,"15min":0.77700000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1279999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77200000000000002,"15min":0.80400000000000005},"min":{"1min":0.55500000000000005,"5min":0.498,"15min":0.498},"max":{"1min":1.0429999999999999,"5min":1.448,"15min":1.448},"last":0.71599999999999997}]},{"osd":7,"last update":"Tue Nov 28 18:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73099999999999998,"15min":0.749},"min":{"1min":0.53300000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.94499999999999995,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.626},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.747,"15min":0.77500000000000002},"min":{"1min":0.54500000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0109999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.80200000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36852,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334988,"statfs":{"total":96636764160,"available":96599027712,"internally_reserved":0,"allocated":1548288,"data_stored":1027898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.42799999999999999,"15min":0.442},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.54500000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.46700000000000003,"15min":0.47099999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65200000000000002,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.437}]},{"osd":2,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.497,"15min":0.50900000000000001},"min":{"1min":0.318,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61699999999999999,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.496,"15min":0.51400000000000001},"min":{"1min":0.33800000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.65000000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.54300000000000004}]},{"osd":3,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48099999999999998,"15min":0.48899999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72599999999999998,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53600000000000003,"15min":0.55300000000000005},"min":{"1min":0.36599999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.75900000000000001,"5min":1.087,"15min":1.087},"last":0.47499999999999998}]},{"osd":4,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71799999999999997,"15min":0.71499999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95299999999999996,"5min":1.131,"15min":1.131},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72299999999999998,"15min":0.71999999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88700000000000001,"5min":1.244,"15min":1.244},"last":0.66700000000000004}]},{"osd":5,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93600000000000005,"5min":1.121,"15min":1.121},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.746,"15min":0.74399999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.94299999999999995,"5min":1.075,"15min":1.075},"last":0.69299999999999995}]},{"osd":6,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.88300000000000001,"5min":1.407,"15min":1.407},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.748,"15min":0.73899999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.90400000000000003,"5min":1.089,"15min":1.089},"last":0.63800000000000001}]},{"osd":7,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.53900000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.99299999999999999,"5min":1.212,"15min":1.212},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.755,"15min":0.751},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.97699999999999998,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.76000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:28:17.821 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:28:17.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.990+0000 7ff726279640 1 Processor -- start 2023-11-28T18:28:17.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.994+0000 7ff726279640 1 -- start start 2023-11-28T18:28:17.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.994+0000 7ff726279640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff7200b4290 0x7ff7200b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:17.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.994+0000 7ff726279640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff720071960 0x7ff72015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:17.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.994+0000 7ff726279640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7200b4bb0 con 0x7ff72015b030 2023-11-28T18:28:17.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.994+0000 7ff726279640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7200701d0 con 0x7ff7200b4290 2023-11-28T18:28:17.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.994+0000 7ff726279640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff72006df80 con 0x7ff720071960 2023-11-28T18:28:17.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.994+0000 7ff71ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff7200b4290 0x7ff7200b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:17.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.994+0000 7ff724816640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff72015b030 0x7ff7200709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47934/0 (socket says 172.21.15.105:47934) 2023-11-28T18:28:17.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.994+0000 7ff724816640 1 -- 172.21.15.105:0/4051261290 learned_addr learned my addr 172.21.15.105:0/4051261290 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:17.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.994+0000 7ff71f7fe640 1 --2- 172.21.15.105:0/4051261290 >> v2:172.21.15.148:3300/0 conn(0x7ff720071960 0x7ff72015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:17.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.994+0000 7ff71ffff640 1 -- 172.21.15.105:0/4051261290 >> v2:172.21.15.148:3300/0 conn(0x7ff720071960 msgr2=0x7ff72015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:18.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.994+0000 7ff71ffff640 1 --2- 172.21.15.105:0/4051261290 >> v2:172.21.15.148:3300/0 conn(0x7ff720071960 0x7ff72015f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.994+0000 7ff71ffff640 1 -- 172.21.15.105:0/4051261290 >> v1:172.21.15.105:6789/0 conn(0x7ff72015b030 legacy=0x7ff7200709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:18.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.994+0000 7ff71effd640 1 -- 172.21.15.105:0/4051261290 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 351459488 0 0) 0x7ff7200b4bb0 con 0x7ff72015b030 2023-11-28T18:28:18.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.994+0000 7ff71ffff640 1 -- 172.21.15.105:0/4051261290 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff720071df0 con 0x7ff7200b4290 2023-11-28T18:28:18.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.998+0000 7ff71ffff640 1 --2- 172.21.15.105:0/4051261290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7200b4290 0x7ff7200b4670 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7ff72006e520 tx=0x7ff71001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9bd21fd594f48d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:18.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.998+0000 7ff71effd640 1 -- 172.21.15.105:0/4051261290 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7100ab650 con 0x7ff7200b4290 2023-11-28T18:28:18.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.998+0000 7ff71effd640 1 -- 172.21.15.105:0/4051261290 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7100ab7b0 con 0x7ff7200b4290 2023-11-28T18:28:18.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.998+0000 7ff71effd640 1 -- 172.21.15.105:0/4051261290 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7100abb40 con 0x7ff7200b4290 2023-11-28T18:28:18.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.998+0000 7ff726279640 1 -- 172.21.15.105:0/4051261290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7200b4290 msgr2=0x7ff7200b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:18.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.998+0000 7ff726279640 1 --2- 172.21.15.105:0/4051261290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7200b4290 0x7ff7200b4670 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7ff72006e520 tx=0x7ff71001c450 comp rx=0 tx=0).stop 2023-11-28T18:28:18.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.998+0000 7ff726279640 1 -- 172.21.15.105:0/4051261290 shutdown_connections 2023-11-28T18:28:18.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.998+0000 7ff726279640 1 --2- 172.21.15.105:0/4051261290 >> v2:172.21.15.148:3300/0 conn(0x7ff720071960 0x7ff72015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.998+0000 7ff726279640 1 --2- 172.21.15.105:0/4051261290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7200b4290 0x7ff7200b4670 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.998+0000 7ff726279640 1 -- 172.21.15.105:0/4051261290 >> 172.21.15.105:0/4051261290 conn(0x7ff7200ceb90 msgr2=0x7ff7200cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:18.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.998+0000 7ff726279640 1 -- 172.21.15.105:0/4051261290 shutdown_connections 2023-11-28T18:28:18.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.998+0000 7ff726279640 1 -- 172.21.15.105:0/4051261290 wait complete. 2023-11-28T18:28:18.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:17.998+0000 7ff726279640 1 Processor -- start 2023-11-28T18:28:18.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff726279640 1 -- start start 2023-11-28T18:28:18.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff726279640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7200b4290 0x7ff720155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:18.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff726279640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff72015b030 0x7ff7200aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:18.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff726279640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7200af1c0 0x7ff720156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:18.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff726279640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72009c940 con 0x7ff7200b4290 2023-11-28T18:28:18.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff726279640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7200ae860 con 0x7ff7200af1c0 2023-11-28T18:28:18.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff726279640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7200b0440 con 0x7ff72015b030 2023-11-28T18:28:18.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff71ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7200b4290 0x7ff720155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:18.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff71ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7200b4290 0x7ff720155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46428/0 (socket says 172.21.15.105:46428) 2023-11-28T18:28:18.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff71ffff640 1 -- 172.21.15.105:0/2752802099 learned_addr learned my addr 172.21.15.105:0/2752802099 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:18.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff71f7fe640 1 --2- 172.21.15.105:0/2752802099 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff72015b030 0x7ff7200aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:18.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff724816640 1 --2- 172.21.15.105:0/2752802099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7200af1c0 0x7ff720156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:18.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff71ffff640 1 -- 172.21.15.105:0/2752802099 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff72015b030 msgr2=0x7ff7200aec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:18.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff71ffff640 1 --2- 172.21.15.105:0/2752802099 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff72015b030 0x7ff7200aec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff71ffff640 1 -- 172.21.15.105:0/2752802099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7200af1c0 msgr2=0x7ff720156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:18.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff71ffff640 1 --2- 172.21.15.105:0/2752802099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7200af1c0 0x7ff720156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff71ffff640 1 -- 172.21.15.105:0/2752802099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff710009420 con 0x7ff7200b4290 2023-11-28T18:28:18.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff724816640 1 --2- 172.21.15.105:0/2752802099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7200af1c0 0x7ff720156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:18.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff71ffff640 1 --2- 172.21.15.105:0/2752802099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7200b4290 0x7ff720155a00 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7ff7100ad040 tx=0x7ff71001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:18.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff71cff9640 1 -- 172.21.15.105:0/2752802099 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7100b5810 con 0x7ff7200b4290 2023-11-28T18:28:18.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff71cff9640 1 -- 172.21.15.105:0/2752802099 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7100b59b0 con 0x7ff7200b4290 2023-11-28T18:28:18.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff71cff9640 1 -- 172.21.15.105:0/2752802099 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7100b5d60 con 0x7ff7200b4290 2023-11-28T18:28:18.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff726279640 1 -- 172.21.15.105:0/2752802099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff720155f40 con 0x7ff7200b4290 2023-11-28T18:28:18.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff726279640 1 -- 172.21.15.105:0/2752802099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7200adec0 con 0x7ff7200b4290 2023-11-28T18:28:18.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.002+0000 7ff726279640 1 -- 172.21.15.105:0/2752802099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7200cb410 con 0x7ff7200b4290 2023-11-28T18:28:18.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.006+0000 7ff71cff9640 1 -- 172.21.15.105:0/2752802099 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff7100b4390 con 0x7ff7200b4290 2023-11-28T18:28:18.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.006+0000 7ff71cff9640 1 --2- 172.21.15.105:0/2752802099 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff70c0668d0 0x7ff70c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:18.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.006+0000 7ff71cff9640 1 -- 172.21.15.105:0/2752802099 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff71011f0a0 con 0x7ff7200b4290 2023-11-28T18:28:18.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.006+0000 7ff71f7fe640 1 --2- 172.21.15.105:0/2752802099 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff70c0668d0 0x7ff70c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:18.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.006+0000 7ff71f7fe640 1 --2- 172.21.15.105:0/2752802099 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff70c0668d0 0x7ff70c068d90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff7200cb280 tx=0x7ff714008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:18.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.010+0000 7ff71cff9640 1 -- 172.21.15.105:0/2752802099 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff7100ec1b0 con 0x7ff7200b4290 2023-11-28T18:28:18.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.158+0000 7ff726279640 1 -- 172.21.15.105:0/2752802099 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff72015f8f0 con 0x7ff70c0668d0 2023-11-28T18:28:18.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.166+0000 7ff71cff9640 1 -- 172.21.15.105:0/2752802099 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289237 (secure 0 0 0) 0x7ff714010960 con 0x7ff70c0668d0 2023-11-28T18:28:18.172 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:18.173 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:28:18.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.166+0000 7ff726279640 1 -- 172.21.15.105:0/2752802099 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff70c0668d0 msgr2=0x7ff70c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:18.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.166+0000 7ff726279640 1 --2- 172.21.15.105:0/2752802099 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff70c0668d0 0x7ff70c068d90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff7200cb280 tx=0x7ff714008040 comp rx=0 tx=0).stop 2023-11-28T18:28:18.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.166+0000 7ff726279640 1 -- 172.21.15.105:0/2752802099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7200b4290 msgr2=0x7ff720155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:18.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.166+0000 7ff726279640 1 --2- 172.21.15.105:0/2752802099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7200b4290 0x7ff720155a00 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7ff7100ad040 tx=0x7ff71001ac90 comp rx=0 tx=0).stop 2023-11-28T18:28:18.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.166+0000 7ff726279640 1 -- 172.21.15.105:0/2752802099 shutdown_connections 2023-11-28T18:28:18.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.166+0000 7ff726279640 1 --2- 172.21.15.105:0/2752802099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7200af1c0 0x7ff720156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.166+0000 7ff726279640 1 --2- 172.21.15.105:0/2752802099 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff72015b030 0x7ff7200aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.166+0000 7ff726279640 1 --2- 172.21.15.105:0/2752802099 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff70c0668d0 0x7ff70c068d90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.166+0000 7ff726279640 1 --2- 172.21.15.105:0/2752802099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7200b4290 0x7ff720155a00 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.166+0000 7ff726279640 1 -- 172.21.15.105:0/2752802099 >> 172.21.15.105:0/2752802099 conn(0x7ff7200ceb90 msgr2=0x7ff72015f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:18.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.166+0000 7ff726279640 1 -- 172.21.15.105:0/2752802099 shutdown_connections 2023-11-28T18:28:18.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.170+0000 7ff726279640 1 -- 172.21.15.105:0/2752802099 wait complete. 2023-11-28T18:28:18.192 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2023-11-28T18:28:17.330002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":305312,"kb_used_data":8680,"kb_used_omap":12,"kb_used_meta":296499,"kb_avail":754669408,"statfs":{"total":773094113280,"available":772781473792,"internally_reserved":0,"allocated":8888320,"data_stored":4685796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":303615538},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.621225"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663593+0000","last_change":"2023-11-28T18:28:13.124330+0000","last_active":"2023-11-28T18:28:14.663593+0000","last_peered":"2023-11-28T18:28:14.663593+0000","last_clean":"2023-11-28T18:28:14.663593+0000","last_became_active":"2023-11-28T18:28:13.124060+0000","last_became_peered":"2023-11-28T18:28:13.124060+0000","last_unstale":"2023-11-28T18:28:14.663593+0000","last_undegraded":"2023-11-28T18:28:14.663593+0000","last_fullsized":"2023-11-28T18:28:14.663593+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:08:51.905424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.664331+0000","last_change":"2023-11-28T18:28:13.126226+0000","last_active":"2023-11-28T18:28:14.664331+0000","last_peered":"2023-11-28T18:28:14.664331+0000","last_clean":"2023-11-28T18:28:14.664331+0000","last_became_active":"2023-11-28T18:28:13.125439+0000","last_became_peered":"2023-11-28T18:28:13.125439+0000","last_unstale":"2023-11-28T18:28:14.664331+0000","last_undegraded":"2023-11-28T18:28:14.664331+0000","last_fullsized":"2023-11-28T18:28:14.664331+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:46:09.462436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.662642+0000","last_change":"2023-11-28T18:28:13.117878+0000","last_active":"2023-11-28T18:28:14.662642+0000","last_peered":"2023-11-28T18:28:14.662642+0000","last_clean":"2023-11-28T18:28:14.662642+0000","last_became_active":"2023-11-28T18:28:13.117066+0000","last_became_peered":"2023-11-28T18:28:13.117066+0000","last_unstale":"2023-11-28T18:28:14.662642+0000","last_undegraded":"2023-11-28T18:28:14.662642+0000","last_fullsized":"2023-11-28T18:28:14.662642+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:19.143572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663216+0000","last_change":"2023-11-28T18:28:13.130808+0000","last_active":"2023-11-28T18:28:14.663216+0000","last_peered":"2023-11-28T18:28:14.663216+0000","last_clean":"2023-11-28T18:28:14.663216+0000","last_became_active":"2023-11-28T18:28:13.126831+0000","last_became_peered":"2023-11-28T18:28:13.126831+0000","last_unstale":"2023-11-28T18:28:14.663216+0000","last_undegraded":"2023-11-28T18:28:14.663216+0000","last_fullsized":"2023-11-28T18:28:14.663216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:44:27.643684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663290+0000","last_change":"2023-11-28T18:28:13.122622+0000","last_active":"2023-11-28T18:28:14.663290+0000","last_peered":"2023-11-28T18:28:14.663290+0000","last_clean":"2023-11-28T18:28:14.663290+0000","last_became_active":"2023-11-28T18:28:13.122322+0000","last_became_peered":"2023-11-28T18:28:13.122322+0000","last_unstale":"2023-11-28T18:28:14.663290+0000","last_undegraded":"2023-11-28T18:28:14.663290+0000","last_fullsized":"2023-11-28T18:28:14.663290+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:59.435656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.111201+0000","last_change":"2023-11-28T18:28:13.121666+0000","last_active":"2023-11-28T18:28:14.111201+0000","last_peered":"2023-11-28T18:28:14.111201+0000","last_clean":"2023-11-28T18:28:14.111201+0000","last_became_active":"2023-11-28T18:28:13.121392+0000","last_became_peered":"2023-11-28T18:28:13.121392+0000","last_unstale":"2023-11-28T18:28:14.111201+0000","last_undegraded":"2023-11-28T18:28:14.111201+0000","last_fullsized":"2023-11-28T18:28:14.111201+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:32.852317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.114835+0000","last_change":"2023-11-28T18:28:13.132620+0000","last_active":"2023-11-28T18:28:14.114835+0000","last_peered":"2023-11-28T18:28:14.114835+0000","last_clean":"2023-11-28T18:28:14.114835+0000","last_became_active":"2023-11-28T18:28:13.132331+0000","last_became_peered":"2023-11-28T18:28:13.132331+0000","last_unstale":"2023-11-28T18:28:14.114835+0000","last_undegraded":"2023-11-28T18:28:14.114835+0000","last_fullsized":"2023-11-28T18:28:14.114835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:18:43.396576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663048+0000","last_change":"2023-11-28T18:28:13.121326+0000","last_active":"2023-11-28T18:28:14.663048+0000","last_peered":"2023-11-28T18:28:14.663048+0000","last_clean":"2023-11-28T18:28:14.663048+0000","last_became_active":"2023-11-28T18:28:13.120973+0000","last_became_peered":"2023-11-28T18:28:13.120973+0000","last_unstale":"2023-11-28T18:28:14.663048+0000","last_undegraded":"2023-11-28T18:28:14.663048+0000","last_fullsized":"2023-11-28T18:28:14.663048+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:55:15.349267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115275+0000","last_change":"2023-11-28T18:28:13.119783+0000","last_active":"2023-11-28T18:28:14.115275+0000","last_peered":"2023-11-28T18:28:14.115275+0000","last_clean":"2023-11-28T18:28:14.115275+0000","last_became_active":"2023-11-28T18:28:13.119493+0000","last_became_peered":"2023-11-28T18:28:13.119493+0000","last_unstale":"2023-11-28T18:28:14.115275+0000","last_undegraded":"2023-11-28T18:28:14.115275+0000","last_fullsized":"2023-11-28T18:28:14.115275+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:38.212829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663391+0000","last_change":"2023-11-28T18:28:13.125929+0000","last_active":"2023-11-28T18:28:14.663391+0000","last_peered":"2023-11-28T18:28:14.663391+0000","last_clean":"2023-11-28T18:28:14.663391+0000","last_became_active":"2023-11-28T18:28:13.124490+0000","last_became_peered":"2023-11-28T18:28:13.124490+0000","last_unstale":"2023-11-28T18:28:14.663391+0000","last_undegraded":"2023-11-28T18:28:14.663391+0000","last_fullsized":"2023-11-28T18:28:14.663391+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:26:14.468846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.664349+0000","last_change":"2023-11-28T18:28:13.127845+0000","last_active":"2023-11-28T18:28:14.664349+0000","last_peered":"2023-11-28T18:28:14.664349+0000","last_clean":"2023-11-28T18:28:14.664349+0000","last_became_active":"2023-11-28T18:28:13.126211+0000","last_became_peered":"2023-11-28T18:28:13.126211+0000","last_unstale":"2023-11-28T18:28:14.664349+0000","last_undegraded":"2023-11-28T18:28:14.664349+0000","last_fullsized":"2023-11-28T18:28:14.664349+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:54:39.161073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663592+0000","last_change":"2023-11-28T18:28:13.126609+0000","last_active":"2023-11-28T18:28:14.663592+0000","last_peered":"2023-11-28T18:28:14.663592+0000","last_clean":"2023-11-28T18:28:14.663592+0000","last_became_active":"2023-11-28T18:28:13.126268+0000","last_became_peered":"2023-11-28T18:28:13.126268+0000","last_unstale":"2023-11-28T18:28:14.663592+0000","last_undegraded":"2023-11-28T18:28:14.663592+0000","last_fullsized":"2023-11-28T18:28:14.663592+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:49:48.533213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663410+0000","last_change":"2023-11-28T18:28:13.119019+0000","last_active":"2023-11-28T18:28:14.663410+0000","last_peered":"2023-11-28T18:28:14.663410+0000","last_clean":"2023-11-28T18:28:14.663410+0000","last_became_active":"2023-11-28T18:28:13.118740+0000","last_became_peered":"2023-11-28T18:28:13.118740+0000","last_unstale":"2023-11-28T18:28:14.663410+0000","last_undegraded":"2023-11-28T18:28:14.663410+0000","last_fullsized":"2023-11-28T18:28:14.663410+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:06:42.585287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.111190+0000","last_change":"2023-11-28T18:28:13.121405+0000","last_active":"2023-11-28T18:28:14.111190+0000","last_peered":"2023-11-28T18:28:14.111190+0000","last_clean":"2023-11-28T18:28:14.111190+0000","last_became_active":"2023-11-28T18:28:13.121076+0000","last_became_peered":"2023-11-28T18:28:13.121076+0000","last_unstale":"2023-11-28T18:28:14.111190+0000","last_undegraded":"2023-11-28T18:28:14.111190+0000","last_fullsized":"2023-11-28T18:28:14.111190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:54:20.912473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663042+0000","last_change":"2023-11-28T18:28:13.125273+0000","last_active":"2023-11-28T18:28:14.663042+0000","last_peered":"2023-11-28T18:28:14.663042+0000","last_clean":"2023-11-28T18:28:14.663042+0000","last_became_active":"2023-11-28T18:28:13.124955+0000","last_became_peered":"2023-11-28T18:28:13.124955+0000","last_unstale":"2023-11-28T18:28:14.663042+0000","last_undegraded":"2023-11-28T18:28:14.663042+0000","last_fullsized":"2023-11-28T18:28:14.663042+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:06.468018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663053+0000","last_change":"2023-11-28T18:28:13.118087+0000","last_active":"2023-11-28T18:28:14.663053+0000","last_peered":"2023-11-28T18:28:14.663053+0000","last_clean":"2023-11-28T18:28:14.663053+0000","last_became_active":"2023-11-28T18:28:13.117810+0000","last_became_peered":"2023-11-28T18:28:13.117810+0000","last_unstale":"2023-11-28T18:28:14.663053+0000","last_undegraded":"2023-11-28T18:28:14.663053+0000","last_fullsized":"2023-11-28T18:28:14.663053+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:31:15.879055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.664158+0000","last_change":"2023-11-28T18:28:13.123501+0000","last_active":"2023-11-28T18:28:14.664158+0000","last_peered":"2023-11-28T18:28:14.664158+0000","last_clean":"2023-11-28T18:28:14.664158+0000","last_became_active":"2023-11-28T18:28:13.123240+0000","last_became_peered":"2023-11-28T18:28:13.123240+0000","last_unstale":"2023-11-28T18:28:14.664158+0000","last_undegraded":"2023-11-28T18:28:14.664158+0000","last_fullsized":"2023-11-28T18:28:14.664158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:37.908507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.662440+0000","last_change":"2023-11-28T18:28:13.122905+0000","last_active":"2023-11-28T18:28:14.662440+0000","last_peered":"2023-11-28T18:28:14.662440+0000","last_clean":"2023-11-28T18:28:14.662440+0000","last_became_active":"2023-11-28T18:28:13.122623+0000","last_became_peered":"2023-11-28T18:28:13.122623+0000","last_unstale":"2023-11-28T18:28:14.662440+0000","last_undegraded":"2023-11-28T18:28:14.662440+0000","last_fullsized":"2023-11-28T18:28:14.662440+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:18:43.396576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.111506+0000","last_change":"2023-11-28T18:28:13.128670+0000","last_active":"2023-11-28T18:28:14.111506+0000","last_peered":"2023-11-28T18:28:14.111506+0000","last_clean":"2023-11-28T18:28:14.111506+0000","last_became_active":"2023-11-28T18:28:13.128411+0000","last_became_peered":"2023-11-28T18:28:13.128411+0000","last_unstale":"2023-11-28T18:28:14.111506+0000","last_undegraded":"2023-11-28T18:28:14.111506+0000","last_fullsized":"2023-11-28T18:28:14.111506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:45:53.434006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663769+0000","last_change":"2023-11-28T18:28:13.120288+0000","last_active":"2023-11-28T18:28:14.663769+0000","last_peered":"2023-11-28T18:28:14.663769+0000","last_clean":"2023-11-28T18:28:14.663769+0000","last_became_active":"2023-11-28T18:28:13.120041+0000","last_became_peered":"2023-11-28T18:28:13.120041+0000","last_unstale":"2023-11-28T18:28:14.663769+0000","last_undegraded":"2023-11-28T18:28:14.663769+0000","last_fullsized":"2023-11-28T18:28:14.663769+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:55:42.482137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115276+0000","last_change":"2023-11-28T18:28:13.111597+0000","last_active":"2023-11-28T18:28:14.115276+0000","last_peered":"2023-11-28T18:28:14.115276+0000","last_clean":"2023-11-28T18:28:14.115276+0000","last_became_active":"2023-11-28T18:28:13.111192+0000","last_became_peered":"2023-11-28T18:28:13.111192+0000","last_unstale":"2023-11-28T18:28:14.115276+0000","last_undegraded":"2023-11-28T18:28:14.115276+0000","last_fullsized":"2023-11-28T18:28:14.115276+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:18:33.289615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663455+0000","last_change":"2023-11-28T18:28:13.119618+0000","last_active":"2023-11-28T18:28:14.663455+0000","last_peered":"2023-11-28T18:28:14.663455+0000","last_clean":"2023-11-28T18:28:14.663455+0000","last_became_active":"2023-11-28T18:28:13.119310+0000","last_became_peered":"2023-11-28T18:28:13.119310+0000","last_unstale":"2023-11-28T18:28:14.663455+0000","last_undegraded":"2023-11-28T18:28:14.663455+0000","last_fullsized":"2023-11-28T18:28:14.663455+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:41:52.691135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.114631+0000","last_change":"2023-11-28T18:28:13.115046+0000","last_active":"2023-11-28T18:28:14.114631+0000","last_peered":"2023-11-28T18:28:14.114631+0000","last_clean":"2023-11-28T18:28:14.114631+0000","last_became_active":"2023-11-28T18:28:13.114743+0000","last_became_peered":"2023-11-28T18:28:13.114743+0000","last_unstale":"2023-11-28T18:28:14.114631+0000","last_undegraded":"2023-11-28T18:28:14.114631+0000","last_fullsized":"2023-11-28T18:28:14.114631+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:22:36.702610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.662845+0000","last_change":"2023-11-28T18:28:13.118568+0000","last_active":"2023-11-28T18:28:14.662845+0000","last_peered":"2023-11-28T18:28:14.662845+0000","last_clean":"2023-11-28T18:28:14.662845+0000","last_became_active":"2023-11-28T18:28:13.118318+0000","last_became_peered":"2023-11-28T18:28:13.118318+0000","last_unstale":"2023-11-28T18:28:14.662845+0000","last_undegraded":"2023-11-28T18:28:14.662845+0000","last_fullsized":"2023-11-28T18:28:14.662845+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:15:09.428663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.664309+0000","last_change":"2023-11-28T18:28:13.117927+0000","last_active":"2023-11-28T18:28:14.664309+0000","last_peered":"2023-11-28T18:28:14.664309+0000","last_clean":"2023-11-28T18:28:14.664309+0000","last_became_active":"2023-11-28T18:28:13.117623+0000","last_became_peered":"2023-11-28T18:28:13.117623+0000","last_unstale":"2023-11-28T18:28:14.664309+0000","last_undegraded":"2023-11-28T18:28:14.664309+0000","last_fullsized":"2023-11-28T18:28:14.664309+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:18:43.726285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663509+0000","last_change":"2023-11-28T18:28:13.130241+0000","last_active":"2023-11-28T18:28:14.663509+0000","last_peered":"2023-11-28T18:28:14.663509+0000","last_clean":"2023-11-28T18:28:14.663509+0000","last_became_active":"2023-11-28T18:28:13.129907+0000","last_became_peered":"2023-11-28T18:28:13.129907+0000","last_unstale":"2023-11-28T18:28:14.663509+0000","last_undegraded":"2023-11-28T18:28:14.663509+0000","last_fullsized":"2023-11-28T18:28:14.663509+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:59:27.785719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663486+0000","last_change":"2023-11-28T18:28:13.120612+0000","last_active":"2023-11-28T18:28:14.663486+0000","last_peered":"2023-11-28T18:28:14.663486+0000","last_clean":"2023-11-28T18:28:14.663486+0000","last_became_active":"2023-11-28T18:28:13.119959+0000","last_became_peered":"2023-11-28T18:28:13.119959+0000","last_unstale":"2023-11-28T18:28:14.663486+0000","last_undegraded":"2023-11-28T18:28:14.663486+0000","last_fullsized":"2023-11-28T18:28:14.663486+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:45:53.434006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663801+0000","last_change":"2023-11-28T18:28:13.125680+0000","last_active":"2023-11-28T18:28:14.663801+0000","last_peered":"2023-11-28T18:28:14.663801+0000","last_clean":"2023-11-28T18:28:14.663801+0000","last_became_active":"2023-11-28T18:28:13.125400+0000","last_became_peered":"2023-11-28T18:28:13.125400+0000","last_unstale":"2023-11-28T18:28:14.663801+0000","last_undegraded":"2023-11-28T18:28:14.663801+0000","last_fullsized":"2023-11-28T18:28:14.663801+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:34:22.888088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663761+0000","last_change":"2023-11-28T18:28:13.124552+0000","last_active":"2023-11-28T18:28:14.663761+0000","last_peered":"2023-11-28T18:28:14.663761+0000","last_clean":"2023-11-28T18:28:14.663761+0000","last_became_active":"2023-11-28T18:28:13.124244+0000","last_became_peered":"2023-11-28T18:28:13.124244+0000","last_unstale":"2023-11-28T18:28:14.663761+0000","last_undegraded":"2023-11-28T18:28:14.663761+0000","last_fullsized":"2023-11-28T18:28:14.663761+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:58:39.630155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663768+0000","last_change":"2023-11-28T18:28:13.131556+0000","last_active":"2023-11-28T18:28:14.663768+0000","last_peered":"2023-11-28T18:28:14.663768+0000","last_clean":"2023-11-28T18:28:14.663768+0000","last_became_active":"2023-11-28T18:28:13.131236+0000","last_became_peered":"2023-11-28T18:28:13.131236+0000","last_unstale":"2023-11-28T18:28:14.663768+0000","last_undegraded":"2023-11-28T18:28:14.663768+0000","last_fullsized":"2023-11-28T18:28:14.663768+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:03:43.789843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.662427+0000","last_change":"2023-11-28T18:28:13.127710+0000","last_active":"2023-11-28T18:28:14.662427+0000","last_peered":"2023-11-28T18:28:14.662427+0000","last_clean":"2023-11-28T18:28:14.662427+0000","last_became_active":"2023-11-28T18:28:13.123986+0000","last_became_peered":"2023-11-28T18:28:13.123986+0000","last_unstale":"2023-11-28T18:28:14.662427+0000","last_undegraded":"2023-11-28T18:28:14.662427+0000","last_fullsized":"2023-11-28T18:28:14.662427+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:46:03.063785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663090+0000","last_change":"2023-11-28T18:28:13.128366+0000","last_active":"2023-11-28T18:28:14.663090+0000","last_peered":"2023-11-28T18:28:14.663090+0000","last_clean":"2023-11-28T18:28:14.663090+0000","last_became_active":"2023-11-28T18:28:13.128069+0000","last_became_peered":"2023-11-28T18:28:13.128069+0000","last_unstale":"2023-11-28T18:28:14.663090+0000","last_undegraded":"2023-11-28T18:28:14.663090+0000","last_fullsized":"2023-11-28T18:28:14.663090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:24:46.417354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.662540+0000","last_change":"2023-11-28T18:28:13.127850+0000","last_active":"2023-11-28T18:28:14.662540+0000","last_peered":"2023-11-28T18:28:14.662540+0000","last_clean":"2023-11-28T18:28:14.662540+0000","last_became_active":"2023-11-28T18:28:13.127608+0000","last_became_peered":"2023-11-28T18:28:13.127608+0000","last_unstale":"2023-11-28T18:28:14.662540+0000","last_undegraded":"2023-11-28T18:28:14.662540+0000","last_fullsized":"2023-11-28T18:28:14.662540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:18:05.416644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"77'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.664186+0000","last_change":"2023-11-28T18:28:12.735435+0000","last_active":"2023-11-28T18:28:14.664186+0000","last_peered":"2023-11-28T18:28:14.664186+0000","last_clean":"2023-11-28T18:28:14.664186+0000","last_became_active":"2023-11-28T18:28:12.734785+0000","last_became_peered":"2023-11-28T18:28:12.734785+0000","last_unstale":"2023-11-28T18:28:14.664186+0000","last_undegraded":"2023-11-28T18:28:14.664186+0000","last_fullsized":"2023-11-28T18:28:14.664186+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:55:04.692978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663372+0000","last_change":"2023-11-28T18:28:13.128688+0000","last_active":"2023-11-28T18:28:14.663372+0000","last_peered":"2023-11-28T18:28:14.663372+0000","last_clean":"2023-11-28T18:28:14.663372+0000","last_became_active":"2023-11-28T18:28:13.128356+0000","last_became_peered":"2023-11-28T18:28:13.128356+0000","last_unstale":"2023-11-28T18:28:14.663372+0000","last_undegraded":"2023-11-28T18:28:14.663372+0000","last_fullsized":"2023-11-28T18:28:14.663372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:40.389583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"77'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663672+0000","last_change":"2023-11-28T18:28:12.744737+0000","last_active":"2023-11-28T18:28:14.663672+0000","last_peered":"2023-11-28T18:28:14.663672+0000","last_clean":"2023-11-28T18:28:14.663672+0000","last_became_active":"2023-11-28T18:28:12.744501+0000","last_became_peered":"2023-11-28T18:28:12.744501+0000","last_unstale":"2023-11-28T18:28:14.663672+0000","last_undegraded":"2023-11-28T18:28:14.663672+0000","last_fullsized":"2023-11-28T18:28:14.663672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:54:19.905291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.114668+0000","last_change":"2023-11-28T18:28:13.119877+0000","last_active":"2023-11-28T18:28:14.114668+0000","last_peered":"2023-11-28T18:28:14.114668+0000","last_clean":"2023-11-28T18:28:14.114668+0000","last_became_active":"2023-11-28T18:28:13.119563+0000","last_became_peered":"2023-11-28T18:28:13.119563+0000","last_unstale":"2023-11-28T18:28:14.114668+0000","last_undegraded":"2023-11-28T18:28:14.114668+0000","last_fullsized":"2023-11-28T18:28:14.114668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:09:46.561906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"77'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663495+0000","last_change":"2023-11-28T18:28:12.736812+0000","last_active":"2023-11-28T18:28:14.663495+0000","last_peered":"2023-11-28T18:28:14.663495+0000","last_clean":"2023-11-28T18:28:14.663495+0000","last_became_active":"2023-11-28T18:28:12.736525+0000","last_became_peered":"2023-11-28T18:28:12.736525+0000","last_unstale":"2023-11-28T18:28:14.663495+0000","last_undegraded":"2023-11-28T18:28:14.663495+0000","last_fullsized":"2023-11-28T18:28:14.663495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:13:57.991197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115404+0000","last_change":"2023-11-28T18:28:13.120665+0000","last_active":"2023-11-28T18:28:14.115404+0000","last_peered":"2023-11-28T18:28:14.115404+0000","last_clean":"2023-11-28T18:28:14.115404+0000","last_became_active":"2023-11-28T18:28:13.119260+0000","last_became_peered":"2023-11-28T18:28:13.119260+0000","last_unstale":"2023-11-28T18:28:14.115404+0000","last_undegraded":"2023-11-28T18:28:14.115404+0000","last_fullsized":"2023-11-28T18:28:14.115404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:59:31.174898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"77'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115295+0000","last_change":"2023-11-28T18:28:12.744170+0000","last_active":"2023-11-28T18:28:14.115295+0000","last_peered":"2023-11-28T18:28:14.115295+0000","last_clean":"2023-11-28T18:28:14.115295+0000","last_became_active":"2023-11-28T18:28:12.743538+0000","last_became_peered":"2023-11-28T18:28:12.743538+0000","last_unstale":"2023-11-28T18:28:14.115295+0000","last_undegraded":"2023-11-28T18:28:14.115295+0000","last_fullsized":"2023-11-28T18:28:14.115295+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:05:40.347837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.664018+0000","last_change":"2023-11-28T18:28:13.126337+0000","last_active":"2023-11-28T18:28:14.664018+0000","last_peered":"2023-11-28T18:28:14.664018+0000","last_clean":"2023-11-28T18:28:14.664018+0000","last_became_active":"2023-11-28T18:28:13.126065+0000","last_became_peered":"2023-11-28T18:28:13.126065+0000","last_unstale":"2023-11-28T18:28:14.664018+0000","last_undegraded":"2023-11-28T18:28:14.664018+0000","last_fullsized":"2023-11-28T18:28:14.664018+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:47:05.047585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"77'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115484+0000","last_change":"2023-11-28T18:28:12.737931+0000","last_active":"2023-11-28T18:28:14.115484+0000","last_peered":"2023-11-28T18:28:14.115484+0000","last_clean":"2023-11-28T18:28:14.115484+0000","last_became_active":"2023-11-28T18:28:12.737655+0000","last_became_peered":"2023-11-28T18:28:12.737655+0000","last_unstale":"2023-11-28T18:28:14.115484+0000","last_undegraded":"2023-11-28T18:28:14.115484+0000","last_fullsized":"2023-11-28T18:28:14.115484+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:32.590295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.664358+0000","last_change":"2023-11-28T18:28:13.127929+0000","last_active":"2023-11-28T18:28:14.664358+0000","last_peered":"2023-11-28T18:28:14.664358+0000","last_clean":"2023-11-28T18:28:14.664358+0000","last_became_active":"2023-11-28T18:28:13.126750+0000","last_became_peered":"2023-11-28T18:28:13.126750+0000","last_unstale":"2023-11-28T18:28:14.664358+0000","last_undegraded":"2023-11-28T18:28:14.664358+0000","last_fullsized":"2023-11-28T18:28:14.664358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:23:08.064510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"77'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663145+0000","last_change":"2023-11-28T18:28:12.730687+0000","last_active":"2023-11-28T18:28:14.663145+0000","last_peered":"2023-11-28T18:28:14.663145+0000","last_clean":"2023-11-28T18:28:14.663145+0000","last_became_active":"2023-11-28T18:28:12.730419+0000","last_became_peered":"2023-11-28T18:28:12.730419+0000","last_unstale":"2023-11-28T18:28:14.663145+0000","last_undegraded":"2023-11-28T18:28:14.663145+0000","last_fullsized":"2023-11-28T18:28:14.663145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:52.733948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115624+0000","last_change":"2023-11-28T18:28:13.120442+0000","last_active":"2023-11-28T18:28:14.115624+0000","last_peered":"2023-11-28T18:28:14.115624+0000","last_clean":"2023-11-28T18:28:14.115624+0000","last_became_active":"2023-11-28T18:28:13.120162+0000","last_became_peered":"2023-11-28T18:28:13.120162+0000","last_unstale":"2023-11-28T18:28:14.115624+0000","last_undegraded":"2023-11-28T18:28:14.115624+0000","last_fullsized":"2023-11-28T18:28:14.115624+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:12:18.196017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663452+0000","last_change":"2023-11-28T18:28:12.737562+0000","last_active":"2023-11-28T18:28:14.663452+0000","last_peered":"2023-11-28T18:28:14.663452+0000","last_clean":"2023-11-28T18:28:14.663452+0000","last_became_active":"2023-11-28T18:28:12.737307+0000","last_became_peered":"2023-11-28T18:28:12.737307+0000","last_unstale":"2023-11-28T18:28:14.663452+0000","last_undegraded":"2023-11-28T18:28:14.663452+0000","last_fullsized":"2023-11-28T18:28:14.663452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:35.491338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115587+0000","last_change":"2023-11-28T18:28:13.132048+0000","last_active":"2023-11-28T18:28:14.115587+0000","last_peered":"2023-11-28T18:28:14.115587+0000","last_clean":"2023-11-28T18:28:14.115587+0000","last_became_active":"2023-11-28T18:28:13.131635+0000","last_became_peered":"2023-11-28T18:28:13.131635+0000","last_unstale":"2023-11-28T18:28:14.115587+0000","last_undegraded":"2023-11-28T18:28:14.115587+0000","last_fullsized":"2023-11-28T18:28:14.115587+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:34:08.801304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663061+0000","last_change":"2023-11-28T18:28:12.730227+0000","last_active":"2023-11-28T18:28:14.663061+0000","last_peered":"2023-11-28T18:28:14.663061+0000","last_clean":"2023-11-28T18:28:14.663061+0000","last_became_active":"2023-11-28T18:28:12.729937+0000","last_became_peered":"2023-11-28T18:28:12.729937+0000","last_unstale":"2023-11-28T18:28:14.663061+0000","last_undegraded":"2023-11-28T18:28:14.663061+0000","last_fullsized":"2023-11-28T18:28:14.663061+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:16.536633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663509+0000","last_change":"2023-11-28T18:28:13.121093+0000","last_active":"2023-11-28T18:28:14.663509+0000","last_peered":"2023-11-28T18:28:14.663509+0000","last_clean":"2023-11-28T18:28:14.663509+0000","last_became_active":"2023-11-28T18:28:13.120843+0000","last_became_peered":"2023-11-28T18:28:13.120843+0000","last_unstale":"2023-11-28T18:28:14.663509+0000","last_undegraded":"2023-11-28T18:28:14.663509+0000","last_fullsized":"2023-11-28T18:28:14.663509+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:20.578566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663711+0000","last_change":"2023-11-28T18:28:12.745614+0000","last_active":"2023-11-28T18:28:14.663711+0000","last_peered":"2023-11-28T18:28:14.663711+0000","last_clean":"2023-11-28T18:28:14.663711+0000","last_became_active":"2023-11-28T18:28:12.744975+0000","last_became_peered":"2023-11-28T18:28:12.744975+0000","last_unstale":"2023-11-28T18:28:14.663711+0000","last_undegraded":"2023-11-28T18:28:14.663711+0000","last_fullsized":"2023-11-28T18:28:14.663711+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:49:47.526032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663053+0000","last_change":"2023-11-28T18:28:13.123945+0000","last_active":"2023-11-28T18:28:14.663053+0000","last_peered":"2023-11-28T18:28:14.663053+0000","last_clean":"2023-11-28T18:28:14.663053+0000","last_became_active":"2023-11-28T18:28:13.123569+0000","last_became_peered":"2023-11-28T18:28:13.123569+0000","last_unstale":"2023-11-28T18:28:14.663053+0000","last_undegraded":"2023-11-28T18:28:14.663053+0000","last_fullsized":"2023-11-28T18:28:14.663053+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:11:49.565273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663599+0000","last_change":"2023-11-28T18:28:12.741182+0000","last_active":"2023-11-28T18:28:14.663599+0000","last_peered":"2023-11-28T18:28:14.663599+0000","last_clean":"2023-11-28T18:28:14.663599+0000","last_became_active":"2023-11-28T18:28:12.740321+0000","last_became_peered":"2023-11-28T18:28:12.740321+0000","last_unstale":"2023-11-28T18:28:14.663599+0000","last_undegraded":"2023-11-28T18:28:14.663599+0000","last_fullsized":"2023-11-28T18:28:14.663599+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:36.038991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.111343+0000","last_change":"2023-11-28T18:28:13.109558+0000","last_active":"2023-11-28T18:28:14.111343+0000","last_peered":"2023-11-28T18:28:14.111343+0000","last_clean":"2023-11-28T18:28:14.111343+0000","last_became_active":"2023-11-28T18:28:13.109272+0000","last_became_peered":"2023-11-28T18:28:13.109272+0000","last_unstale":"2023-11-28T18:28:14.111343+0000","last_undegraded":"2023-11-28T18:28:14.111343+0000","last_fullsized":"2023-11-28T18:28:14.111343+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:24:46.417354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"77'3","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663987+0000","last_change":"2023-11-28T18:28:12.741910+0000","last_active":"2023-11-28T18:28:14.663987+0000","last_peered":"2023-11-28T18:28:14.663987+0000","last_clean":"2023-11-28T18:28:14.663987+0000","last_became_active":"2023-11-28T18:28:12.741176+0000","last_became_peered":"2023-11-28T18:28:12.741176+0000","last_unstale":"2023-11-28T18:28:14.663987+0000","last_undegraded":"2023-11-28T18:28:14.663987+0000","last_fullsized":"2023-11-28T18:28:14.663987+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:17.478758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.662424+0000","last_change":"2023-11-28T18:28:13.116799+0000","last_active":"2023-11-28T18:28:14.662424+0000","last_peered":"2023-11-28T18:28:14.662424+0000","last_clean":"2023-11-28T18:28:14.662424+0000","last_became_active":"2023-11-28T18:28:13.116502+0000","last_became_peered":"2023-11-28T18:28:13.116502+0000","last_unstale":"2023-11-28T18:28:14.662424+0000","last_undegraded":"2023-11-28T18:28:14.662424+0000","last_fullsized":"2023-11-28T18:28:14.662424+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:05:41.355019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"77'6","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115384+0000","last_change":"2023-11-28T18:28:12.737200+0000","last_active":"2023-11-28T18:28:14.115384+0000","last_peered":"2023-11-28T18:28:14.115384+0000","last_clean":"2023-11-28T18:28:14.115384+0000","last_became_active":"2023-11-28T18:28:12.736912+0000","last_became_peered":"2023-11-28T18:28:12.736912+0000","last_unstale":"2023-11-28T18:28:14.115384+0000","last_undegraded":"2023-11-28T18:28:14.115384+0000","last_fullsized":"2023-11-28T18:28:14.115384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:46:02.056603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663736+0000","last_change":"2023-11-28T18:28:13.125754+0000","last_active":"2023-11-28T18:28:14.663736+0000","last_peered":"2023-11-28T18:28:14.663736+0000","last_clean":"2023-11-28T18:28:14.663736+0000","last_became_active":"2023-11-28T18:28:13.125408+0000","last_became_peered":"2023-11-28T18:28:13.125408+0000","last_unstale":"2023-11-28T18:28:14.663736+0000","last_undegraded":"2023-11-28T18:28:14.663736+0000","last_fullsized":"2023-11-28T18:28:14.663736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:59:56.917810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"77'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663042+0000","last_change":"2023-11-28T18:28:12.730086+0000","last_active":"2023-11-28T18:28:14.663042+0000","last_peered":"2023-11-28T18:28:14.663042+0000","last_clean":"2023-11-28T18:28:14.663042+0000","last_became_active":"2023-11-28T18:28:12.729786+0000","last_became_peered":"2023-11-28T18:28:12.729786+0000","last_unstale":"2023-11-28T18:28:14.663042+0000","last_undegraded":"2023-11-28T18:28:14.663042+0000","last_fullsized":"2023-11-28T18:28:14.663042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:34:07.794122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115479+0000","last_change":"2023-11-28T18:28:13.122426+0000","last_active":"2023-11-28T18:28:14.115479+0000","last_peered":"2023-11-28T18:28:14.115479+0000","last_clean":"2023-11-28T18:28:14.115479+0000","last_became_active":"2023-11-28T18:28:13.122095+0000","last_became_peered":"2023-11-28T18:28:13.122095+0000","last_unstale":"2023-11-28T18:28:14.115479+0000","last_undegraded":"2023-11-28T18:28:14.115479+0000","last_fullsized":"2023-11-28T18:28:14.115479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:39:23.857851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663871+0000","last_change":"2023-11-28T18:28:12.738549+0000","last_active":"2023-11-28T18:28:14.663871+0000","last_peered":"2023-11-28T18:28:14.663871+0000","last_clean":"2023-11-28T18:28:14.663871+0000","last_became_active":"2023-11-28T18:28:12.738052+0000","last_became_peered":"2023-11-28T18:28:12.738052+0000","last_unstale":"2023-11-28T18:28:14.663871+0000","last_undegraded":"2023-11-28T18:28:14.663871+0000","last_fullsized":"2023-11-28T18:28:14.663871+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:11:48.558091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115446+0000","last_change":"2023-11-28T18:28:13.127268+0000","last_active":"2023-11-28T18:28:14.115446+0000","last_peered":"2023-11-28T18:28:14.115446+0000","last_clean":"2023-11-28T18:28:14.115446+0000","last_became_active":"2023-11-28T18:28:13.126932+0000","last_became_peered":"2023-11-28T18:28:13.126932+0000","last_unstale":"2023-11-28T18:28:14.115446+0000","last_undegraded":"2023-11-28T18:28:14.115446+0000","last_fullsized":"2023-11-28T18:28:14.115446+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:40:16.599031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663205+0000","last_change":"2023-11-28T18:28:12.733555+0000","last_active":"2023-11-28T18:28:14.663205+0000","last_peered":"2023-11-28T18:28:14.663205+0000","last_clean":"2023-11-28T18:28:14.663205+0000","last_became_active":"2023-11-28T18:28:12.733285+0000","last_became_peered":"2023-11-28T18:28:12.733285+0000","last_unstale":"2023-11-28T18:28:14.663205+0000","last_undegraded":"2023-11-28T18:28:14.663205+0000","last_fullsized":"2023-11-28T18:28:14.663205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:58.327464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115620+0000","last_change":"2023-11-28T18:28:13.129911+0000","last_active":"2023-11-28T18:28:14.115620+0000","last_peered":"2023-11-28T18:28:14.115620+0000","last_clean":"2023-11-28T18:28:14.115620+0000","last_became_active":"2023-11-28T18:28:13.129624+0000","last_became_peered":"2023-11-28T18:28:13.129624+0000","last_unstale":"2023-11-28T18:28:14.115620+0000","last_undegraded":"2023-11-28T18:28:14.115620+0000","last_fullsized":"2023-11-28T18:28:14.115620+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:21:11.948500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"77'8","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663549+0000","last_change":"2023-11-28T18:28:12.738810+0000","last_active":"2023-11-28T18:28:14.663549+0000","last_peered":"2023-11-28T18:28:14.663549+0000","last_clean":"2023-11-28T18:28:14.663549+0000","last_became_active":"2023-11-28T18:28:12.736642+0000","last_became_peered":"2023-11-28T18:28:12.736642+0000","last_unstale":"2023-11-28T18:28:14.663549+0000","last_undegraded":"2023-11-28T18:28:14.663549+0000","last_fullsized":"2023-11-28T18:28:14.663549+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:13:57.991197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.111084+0000","last_change":"2023-11-28T18:28:13.121376+0000","last_active":"2023-11-28T18:28:14.111084+0000","last_peered":"2023-11-28T18:28:14.111084+0000","last_clean":"2023-11-28T18:28:14.111084+0000","last_became_active":"2023-11-28T18:28:13.120124+0000","last_became_peered":"2023-11-28T18:28:13.120124+0000","last_unstale":"2023-11-28T18:28:14.111084+0000","last_undegraded":"2023-11-28T18:28:14.111084+0000","last_fullsized":"2023-11-28T18:28:14.111084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:38:27.289444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"77'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663753+0000","last_change":"2023-11-28T18:28:12.743909+0000","last_active":"2023-11-28T18:28:14.663753+0000","last_peered":"2023-11-28T18:28:14.663753+0000","last_clean":"2023-11-28T18:28:14.663753+0000","last_became_active":"2023-11-28T18:28:12.743628+0000","last_became_peered":"2023-11-28T18:28:12.743628+0000","last_unstale":"2023-11-28T18:28:14.663753+0000","last_undegraded":"2023-11-28T18:28:14.663753+0000","last_fullsized":"2023-11-28T18:28:14.663753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:59:55.910629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663207+0000","last_change":"2023-11-28T18:28:13.123446+0000","last_active":"2023-11-28T18:28:14.663207+0000","last_peered":"2023-11-28T18:28:14.663207+0000","last_clean":"2023-11-28T18:28:14.663207+0000","last_became_active":"2023-11-28T18:28:13.123190+0000","last_became_peered":"2023-11-28T18:28:13.123190+0000","last_unstale":"2023-11-28T18:28:14.663207+0000","last_undegraded":"2023-11-28T18:28:14.663207+0000","last_fullsized":"2023-11-28T18:28:14.663207+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:22.541327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"77'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115733+0000","last_change":"2023-11-28T18:28:12.737644+0000","last_active":"2023-11-28T18:28:14.115733+0000","last_peered":"2023-11-28T18:28:14.115733+0000","last_clean":"2023-11-28T18:28:14.115733+0000","last_became_active":"2023-11-28T18:28:12.737378+0000","last_became_peered":"2023-11-28T18:28:12.737378+0000","last_unstale":"2023-11-28T18:28:14.115733+0000","last_undegraded":"2023-11-28T18:28:14.115733+0000","last_fullsized":"2023-11-28T18:28:14.115733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:42:42.913425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663252+0000","last_change":"2023-11-28T18:28:13.129592+0000","last_active":"2023-11-28T18:28:14.663252+0000","last_peered":"2023-11-28T18:28:14.663252+0000","last_clean":"2023-11-28T18:28:14.663252+0000","last_became_active":"2023-11-28T18:28:13.129282+0000","last_became_peered":"2023-11-28T18:28:13.129282+0000","last_unstale":"2023-11-28T18:28:14.663252+0000","last_undegraded":"2023-11-28T18:28:14.663252+0000","last_fullsized":"2023-11-28T18:28:14.663252+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:14.218187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"77'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663999+0000","last_change":"2023-11-28T18:28:12.732492+0000","last_active":"2023-11-28T18:28:14.663999+0000","last_peered":"2023-11-28T18:28:14.663999+0000","last_clean":"2023-11-28T18:28:14.663999+0000","last_became_active":"2023-11-28T18:28:12.732176+0000","last_became_peered":"2023-11-28T18:28:12.732176+0000","last_unstale":"2023-11-28T18:28:14.663999+0000","last_undegraded":"2023-11-28T18:28:14.663999+0000","last_fullsized":"2023-11-28T18:28:14.663999+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:01:13.221094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.111233+0000","last_change":"2023-11-28T18:28:13.128711+0000","last_active":"2023-11-28T18:28:14.111233+0000","last_peered":"2023-11-28T18:28:14.111233+0000","last_clean":"2023-11-28T18:28:14.111233+0000","last_became_active":"2023-11-28T18:28:13.128451+0000","last_became_peered":"2023-11-28T18:28:13.128451+0000","last_unstale":"2023-11-28T18:28:14.111233+0000","last_undegraded":"2023-11-28T18:28:14.111233+0000","last_fullsized":"2023-11-28T18:28:14.111233+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:31:15.879055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.662344+0000","last_change":"2023-11-28T18:28:12.739945+0000","last_active":"2023-11-28T18:28:14.662344+0000","last_peered":"2023-11-28T18:28:14.662344+0000","last_clean":"2023-11-28T18:28:14.662344+0000","last_became_active":"2023-11-28T18:28:12.738948+0000","last_became_peered":"2023-11-28T18:28:12.738948+0000","last_unstale":"2023-11-28T18:28:14.662344+0000","last_undegraded":"2023-11-28T18:28:14.662344+0000","last_fullsized":"2023-11-28T18:28:14.662344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:59:30.167717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.664040+0000","last_change":"2023-11-28T18:28:13.124800+0000","last_active":"2023-11-28T18:28:14.664040+0000","last_peered":"2023-11-28T18:28:14.664040+0000","last_clean":"2023-11-28T18:28:14.664040+0000","last_became_active":"2023-11-28T18:28:13.124537+0000","last_became_peered":"2023-11-28T18:28:13.124537+0000","last_unstale":"2023-11-28T18:28:14.664040+0000","last_undegraded":"2023-11-28T18:28:14.664040+0000","last_fullsized":"2023-11-28T18:28:14.664040+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:10:43.594536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"77'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.111317+0000","last_change":"2023-11-28T18:28:12.737759+0000","last_active":"2023-11-28T18:28:14.111317+0000","last_peered":"2023-11-28T18:28:14.111317+0000","last_clean":"2023-11-28T18:28:14.111317+0000","last_became_active":"2023-11-28T18:28:12.737440+0000","last_became_peered":"2023-11-28T18:28:12.737440+0000","last_unstale":"2023-11-28T18:28:14.111317+0000","last_undegraded":"2023-11-28T18:28:14.111317+0000","last_fullsized":"2023-11-28T18:28:14.111317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:49:47.526032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.664396+0000","last_change":"2023-11-28T18:28:13.125784+0000","last_active":"2023-11-28T18:28:14.664396+0000","last_peered":"2023-11-28T18:28:14.664396+0000","last_clean":"2023-11-28T18:28:14.664396+0000","last_became_active":"2023-11-28T18:28:13.125077+0000","last_became_peered":"2023-11-28T18:28:13.125077+0000","last_unstale":"2023-11-28T18:28:14.664396+0000","last_undegraded":"2023-11-28T18:28:14.664396+0000","last_fullsized":"2023-11-28T18:28:14.664396+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:29:46.319741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.111425+0000","last_change":"2023-11-28T18:28:12.738291+0000","last_active":"2023-11-28T18:28:14.111425+0000","last_peered":"2023-11-28T18:28:14.111425+0000","last_clean":"2023-11-28T18:28:14.111425+0000","last_became_active":"2023-11-28T18:28:12.738025+0000","last_became_peered":"2023-11-28T18:28:12.738025+0000","last_unstale":"2023-11-28T18:28:14.111425+0000","last_undegraded":"2023-11-28T18:28:14.111425+0000","last_fullsized":"2023-11-28T18:28:14.111425+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:36.038991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115723+0000","last_change":"2023-11-28T18:28:13.128349+0000","last_active":"2023-11-28T18:28:14.115723+0000","last_peered":"2023-11-28T18:28:14.115723+0000","last_clean":"2023-11-28T18:28:14.115723+0000","last_became_active":"2023-11-28T18:28:13.127785+0000","last_became_peered":"2023-11-28T18:28:13.127785+0000","last_unstale":"2023-11-28T18:28:14.115723+0000","last_undegraded":"2023-11-28T18:28:14.115723+0000","last_fullsized":"2023-11-28T18:28:14.115723+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:48:57.176966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.111021+0000","last_change":"2023-11-28T18:28:12.738174+0000","last_active":"2023-11-28T18:28:14.111021+0000","last_peered":"2023-11-28T18:28:14.111021+0000","last_clean":"2023-11-28T18:28:14.111021+0000","last_became_active":"2023-11-28T18:28:12.737868+0000","last_became_peered":"2023-11-28T18:28:12.737868+0000","last_unstale":"2023-11-28T18:28:14.111021+0000","last_undegraded":"2023-11-28T18:28:14.111021+0000","last_fullsized":"2023-11-28T18:28:14.111021+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:15.796896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.114519+0000","last_change":"2023-11-28T18:28:13.126032+0000","last_active":"2023-11-28T18:28:14.114519+0000","last_peered":"2023-11-28T18:28:14.114519+0000","last_clean":"2023-11-28T18:28:14.114519+0000","last_became_active":"2023-11-28T18:28:13.124282+0000","last_became_peered":"2023-11-28T18:28:13.124282+0000","last_unstale":"2023-11-28T18:28:14.114519+0000","last_undegraded":"2023-11-28T18:28:14.114519+0000","last_fullsized":"2023-11-28T18:28:14.114519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:16:51.464199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"77'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663244+0000","last_change":"2023-11-28T18:28:12.742105+0000","last_active":"2023-11-28T18:28:14.663244+0000","last_peered":"2023-11-28T18:28:14.663244+0000","last_clean":"2023-11-28T18:28:14.663244+0000","last_became_active":"2023-11-28T18:28:12.741872+0000","last_became_peered":"2023-11-28T18:28:12.741872+0000","last_unstale":"2023-11-28T18:28:14.663244+0000","last_undegraded":"2023-11-28T18:28:14.663244+0000","last_fullsized":"2023-11-28T18:28:14.663244+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:15.796896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.114904+0000","last_change":"2023-11-28T18:28:13.129538+0000","last_active":"2023-11-28T18:28:14.114904+0000","last_peered":"2023-11-28T18:28:14.114904+0000","last_clean":"2023-11-28T18:28:14.114904+0000","last_became_active":"2023-11-28T18:28:13.129101+0000","last_became_peered":"2023-11-28T18:28:13.129101+0000","last_unstale":"2023-11-28T18:28:14.114904+0000","last_undegraded":"2023-11-28T18:28:14.114904+0000","last_fullsized":"2023-11-28T18:28:14.114904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:18:05.416644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.664416+0000","last_change":"2023-11-28T18:28:12.736722+0000","last_active":"2023-11-28T18:28:14.664416+0000","last_peered":"2023-11-28T18:28:14.664416+0000","last_clean":"2023-11-28T18:28:14.664416+0000","last_became_active":"2023-11-28T18:28:12.735717+0000","last_became_peered":"2023-11-28T18:28:12.735717+0000","last_unstale":"2023-11-28T18:28:14.664416+0000","last_undegraded":"2023-11-28T18:28:14.664416+0000","last_fullsized":"2023-11-28T18:28:14.664416+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:03:42.782662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.114903+0000","last_change":"2023-11-28T18:28:13.112590+0000","last_active":"2023-11-28T18:28:14.114903+0000","last_peered":"2023-11-28T18:28:14.114903+0000","last_clean":"2023-11-28T18:28:14.114903+0000","last_became_active":"2023-11-28T18:28:13.111238+0000","last_became_peered":"2023-11-28T18:28:13.111238+0000","last_unstale":"2023-11-28T18:28:14.114903+0000","last_undegraded":"2023-11-28T18:28:14.114903+0000","last_fullsized":"2023-11-28T18:28:14.114903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:01.297558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663590+0000","last_change":"2023-11-28T18:28:12.737920+0000","last_active":"2023-11-28T18:28:14.663590+0000","last_peered":"2023-11-28T18:28:14.663590+0000","last_clean":"2023-11-28T18:28:14.663590+0000","last_became_active":"2023-11-28T18:28:12.737653+0000","last_became_peered":"2023-11-28T18:28:12.737653+0000","last_unstale":"2023-11-28T18:28:14.663590+0000","last_undegraded":"2023-11-28T18:28:14.663590+0000","last_fullsized":"2023-11-28T18:28:14.663590+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:31.845136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663760+0000","last_change":"2023-11-28T18:28:13.125558+0000","last_active":"2023-11-28T18:28:14.663760+0000","last_peered":"2023-11-28T18:28:14.663760+0000","last_clean":"2023-11-28T18:28:14.663760+0000","last_became_active":"2023-11-28T18:28:13.125276+0000","last_became_peered":"2023-11-28T18:28:13.125276+0000","last_unstale":"2023-11-28T18:28:14.663760+0000","last_undegraded":"2023-11-28T18:28:14.663760+0000","last_fullsized":"2023-11-28T18:28:14.663760+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:02.561354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663368+0000","last_change":"2023-11-28T18:28:12.745317+0000","last_active":"2023-11-28T18:28:14.663368+0000","last_peered":"2023-11-28T18:28:14.663368+0000","last_clean":"2023-11-28T18:28:14.663368+0000","last_became_active":"2023-11-28T18:28:12.744923+0000","last_became_peered":"2023-11-28T18:28:12.744923+0000","last_unstale":"2023-11-28T18:28:14.663368+0000","last_undegraded":"2023-11-28T18:28:14.663368+0000","last_fullsized":"2023-11-28T18:28:14.663368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:44:26.636503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115213+0000","last_change":"2023-11-28T18:28:13.129945+0000","last_active":"2023-11-28T18:28:14.115213+0000","last_peered":"2023-11-28T18:28:14.115213+0000","last_clean":"2023-11-28T18:28:14.115213+0000","last_became_active":"2023-11-28T18:28:13.129625+0000","last_became_peered":"2023-11-28T18:28:13.129625+0000","last_unstale":"2023-11-28T18:28:14.115213+0000","last_undegraded":"2023-11-28T18:28:14.115213+0000","last_fullsized":"2023-11-28T18:28:14.115213+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:19.143572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115796+0000","last_change":"2023-11-28T18:28:12.740986+0000","last_active":"2023-11-28T18:28:14.115796+0000","last_peered":"2023-11-28T18:28:14.115796+0000","last_clean":"2023-11-28T18:28:14.115796+0000","last_became_active":"2023-11-28T18:28:12.740621+0000","last_became_peered":"2023-11-28T18:28:12.740621+0000","last_unstale":"2023-11-28T18:28:14.115796+0000","last_undegraded":"2023-11-28T18:28:14.115796+0000","last_fullsized":"2023-11-28T18:28:14.115796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:12:17.188835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115275+0000","last_change":"2023-11-28T18:28:13.130454+0000","last_active":"2023-11-28T18:28:14.115275+0000","last_peered":"2023-11-28T18:28:14.115275+0000","last_clean":"2023-11-28T18:28:14.115275+0000","last_became_active":"2023-11-28T18:28:13.130135+0000","last_became_peered":"2023-11-28T18:28:13.130135+0000","last_unstale":"2023-11-28T18:28:14.115275+0000","last_undegraded":"2023-11-28T18:28:14.115275+0000","last_fullsized":"2023-11-28T18:28:14.115275+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:55:56.581933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663532+0000","last_change":"2023-11-28T18:28:12.732044+0000","last_active":"2023-11-28T18:28:14.663532+0000","last_peered":"2023-11-28T18:28:14.663532+0000","last_clean":"2023-11-28T18:28:14.663532+0000","last_became_active":"2023-11-28T18:28:12.731779+0000","last_became_peered":"2023-11-28T18:28:12.731779+0000","last_unstale":"2023-11-28T18:28:14.663532+0000","last_undegraded":"2023-11-28T18:28:14.663532+0000","last_fullsized":"2023-11-28T18:28:14.663532+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:11:29.567488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.662740+0000","last_change":"2023-11-28T18:28:13.118250+0000","last_active":"2023-11-28T18:28:14.662740+0000","last_peered":"2023-11-28T18:28:14.662740+0000","last_clean":"2023-11-28T18:28:14.662740+0000","last_became_active":"2023-11-28T18:28:13.117939+0000","last_became_peered":"2023-11-28T18:28:13.117939+0000","last_unstale":"2023-11-28T18:28:14.662740+0000","last_undegraded":"2023-11-28T18:28:14.662740+0000","last_fullsized":"2023-11-28T18:28:14.662740+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:55:56.581933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.115013+0000","last_change":"2023-11-28T18:28:12.744032+0000","last_active":"2023-11-28T18:28:14.115013+0000","last_peered":"2023-11-28T18:28:14.115013+0000","last_clean":"2023-11-28T18:28:14.115013+0000","last_became_active":"2023-11-28T18:28:12.743774+0000","last_became_peered":"2023-11-28T18:28:12.743774+0000","last_unstale":"2023-11-28T18:28:14.115013+0000","last_undegraded":"2023-11-28T18:28:14.115013+0000","last_fullsized":"2023-11-28T18:28:14.115013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:37.205648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.662749+0000","last_change":"2023-11-28T18:28:13.123603+0000","last_active":"2023-11-28T18:28:14.662749+0000","last_peered":"2023-11-28T18:28:14.662749+0000","last_clean":"2023-11-28T18:28:14.662749+0000","last_became_active":"2023-11-28T18:28:13.123068+0000","last_became_peered":"2023-11-28T18:28:13.123068+0000","last_unstale":"2023-11-28T18:28:14.662749+0000","last_undegraded":"2023-11-28T18:28:14.662749+0000","last_fullsized":"2023-11-28T18:28:14.662749+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:40:16.599031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.662271+0000","last_change":"2023-11-28T18:28:12.739581+0000","last_active":"2023-11-28T18:28:14.662271+0000","last_peered":"2023-11-28T18:28:14.662271+0000","last_clean":"2023-11-28T18:28:14.662271+0000","last_became_active":"2023-11-28T18:28:12.739164+0000","last_became_peered":"2023-11-28T18:28:12.739164+0000","last_unstale":"2023-11-28T18:28:14.662271+0000","last_undegraded":"2023-11-28T18:28:14.662271+0000","last_fullsized":"2023-11-28T18:28:14.662271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:12:17.188835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.663581+0000","last_change":"2023-11-28T18:28:13.120806+0000","last_active":"2023-11-28T18:28:14.663581+0000","last_peered":"2023-11-28T18:28:14.663581+0000","last_clean":"2023-11-28T18:28:14.663581+0000","last_became_active":"2023-11-28T18:28:13.120332+0000","last_became_peered":"2023-11-28T18:28:13.120332+0000","last_unstale":"2023-11-28T18:28:14.663581+0000","last_undegraded":"2023-11-28T18:28:14.663581+0000","last_fullsized":"2023-11-28T18:28:14.663581+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:12.707947+0000","last_clean_scrub_stamp":"2023-11-28T18:28:12.707947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:02:34.345986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-28T18:28:14.111468+0000","last_change":"2023-11-28T18:28:12.737240+0000","last_active":"2023-11-28T18:28:14.111468+0000","last_peered":"2023-11-28T18:28:14.111468+0000","last_clean":"2023-11-28T18:28:14.111468+0000","last_became_active":"2023-11-28T18:28:12.736798+0000","last_became_peered":"2023-11-28T18:28:12.736798+0000","last_unstale":"2023-11-28T18:28:14.111468+0000","last_undegraded":"2023-11-28T18:28:14.111468+0000","last_fullsized":"2023-11-28T18:28:14.111468+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:28:11.700766+0000","last_clean_scrub_stamp":"2023-11-28T18:28:11.700766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:59:55.910629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":215,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-28T18:28:13.109823+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:28:13.109823+0000","last_peered":"2023-11-28T18:28:13.109823+0000","last_clean":"2023-11-28T18:28:13.109823+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:28:13.109823+0000","last_undegraded":"2023-11-28T18:28:13.109823+0000","last_fullsized":"2023-11-28T18:28:13.109823+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41016,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330824,"statfs":{"total":96636764160,"available":96594763776,"internally_reserved":0,"allocated":1552384,"data_stored":1027920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.65300000000000002,"15min":0.71699999999999997},"min":{"1min":0.39100000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.84799999999999998,"5min":2.427,"15min":2.427},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.66300000000000003,"15min":0.71999999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80700000000000005,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.93500000000000005}]},{"osd":1,"last update":"Tue Nov 28 18:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.67300000000000004,"15min":0.71899999999999997},"min":{"1min":0.41599999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76100000000000001,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.69699999999999995,"15min":0.755},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.76200000000000001,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.079}]},{"osd":2,"last update":"Tue Nov 28 18:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.71499999999999997,"15min":0.77400000000000002},"min":{"1min":0.439,"5min":0.379,"15min":0.379},"max":{"1min":0.877,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.998},{"interface":"front","average":{"1min":0.629,"5min":0.68799999999999994,"15min":0.74199999999999999},"min":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85499999999999998,"5min":2.411,"15min":2.411},"last":0.97299999999999998}]},{"osd":3,"last update":"Tue Nov 28 18:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.70299999999999996,"15min":0.76300000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81499999999999995,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.749,"15min":0.80400000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.85099999999999998,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.0389999999999999}]},{"osd":4,"last update":"Tue Nov 28 18:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58799999999999997,"15min":0.60399999999999998},"min":{"1min":0.34799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.752,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.60299999999999998,"15min":0.64100000000000001},"min":{"1min":0.33100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.83099999999999996,"5min":0.877,"15min":0.877},"last":0.64700000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65000000000000002,"15min":0.67700000000000005},"min":{"1min":0.38200000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83699999999999997,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66600000000000004,"15min":0.70199999999999996},"min":{"1min":0.39900000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93999999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.80400000000000005}]},{"osd":6,"last update":"Tue Nov 28 18:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.70499999999999996,"15min":0.74199999999999999},"min":{"1min":0.44600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.90000000000000002,"5min":1.042,"15min":1.042},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.68000000000000005,"15min":0.73299999999999998},"min":{"1min":0.36699999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80000000000000004,"5min":1.006,"15min":1.006},"last":0.77800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":966656,"data_stored":439142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.35499999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.84899999999999998,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70899999999999996,"15min":0.72899999999999998},"min":{"1min":0.498,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.93000000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.49299999999999999}]},{"osd":1,"last update":"Tue Nov 28 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.72099999999999997,"15min":0.80700000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81399999999999995,"5min":4.2389999999999999,"15min":4.2389999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.72899999999999998,"15min":0.79300000000000004},"min":{"1min":0.38200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.86599999999999999,"5min":4.218,"15min":4.218},"last":0.66400000000000003}]},{"osd":2,"last update":"Tue Nov 28 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.68300000000000005,"15min":0.71499999999999997},"min":{"1min":0.47499999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79500000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.68600000000000005,"15min":0.70299999999999996},"min":{"1min":0.42999999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.75700000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.64300000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.69599999999999995,"15min":0.72399999999999998},"min":{"1min":0.432,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.84499999999999997,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67900000000000005,"15min":0.68700000000000006},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83699999999999997,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.78900000000000003}]},{"osd":4,"last update":"Tue Nov 28 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.64400000000000002,"15min":0.66900000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89000000000000001,"5min":0.995,"15min":0.995},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.65000000000000002,"15min":0.68799999999999994},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.88200000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.81200000000000006}]},{"osd":5,"last update":"Tue Nov 28 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64800000000000002,"15min":0.68100000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82899999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.72299999999999998,"15min":0.76100000000000001},"min":{"1min":0.49199999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.871,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.84399999999999997}]},{"osd":7,"last update":"Tue Nov 28 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.73399999999999999,"15min":0.77400000000000002},"min":{"1min":0.495,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91000000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.72499999999999998,"15min":0.753},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1,"5min":1,"15min":1},"last":0.86399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":958464,"data_stored":437564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.69499999999999995,"15min":0.71199999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.75600000000000001,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.72499999999999998,"15min":0.754},"min":{"1min":0.50900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89400000000000002,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":0.61799999999999999}]},{"osd":1,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.70599999999999996,"15min":0.74199999999999999},"min":{"1min":0.44500000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80300000000000005,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.74199999999999999,"15min":0.80000000000000004},"min":{"1min":0.40200000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84599999999999997,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.77300000000000002}]},{"osd":2,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.70799999999999996,"15min":0.75900000000000001},"min":{"1min":0.42899999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77800000000000002,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.71799999999999997,"15min":0.77500000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82599999999999996,"5min":4.0149999999999997,"15min":4.0149999999999997},"last":0.63500000000000001}]},{"osd":3,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.76100000000000001,"15min":0.82799999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.86899999999999999,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.753,"15min":0.82799999999999996},"min":{"1min":0.46899999999999997,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.90100000000000002,"5min":3.9900000000000002,"15min":3.9900000000000002},"last":0.56999999999999995}]},{"osd":4,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.88200000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.497},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.68700000000000006,"15min":0.70899999999999996},"min":{"1min":0.35199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.875,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.59499999999999997}]},{"osd":6,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68300000000000005,"15min":0.68200000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.95899999999999996,"5min":1.071,"15min":1.071},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71199999999999997,"15min":0.71999999999999997},"min":{"1min":0.49199999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.92400000000000004,"5min":1.756,"15min":1.756},"last":0.54900000000000004}]},{"osd":7,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.749,"15min":0.75700000000000001},"min":{"1min":0.52000000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94999999999999996,"5min":1.282,"15min":1.282},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.753,"15min":0.75800000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.97399999999999998,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.47399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36292,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335548,"statfs":{"total":96636764160,"available":96599601152,"internally_reserved":0,"allocated":974848,"data_stored":439246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65900000000000003,"15min":0.69099999999999995},"min":{"1min":0.33900000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.91200000000000003,"5min":2.363,"15min":2.363},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64000000000000001,"15min":0.64600000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85299999999999998,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.97699999999999998}]},{"osd":1,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63400000000000001,"15min":0.624},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.92600000000000005,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.443,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.79400000000000004}]},{"osd":2,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70699999999999996,"15min":0.69699999999999995},"min":{"1min":0.45800000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.871},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71599999999999997,"15min":0.72799999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91000000000000003,"5min":1.871,"15min":1.871},"last":0.89300000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64800000000000002,"15min":0.64700000000000002},"min":{"1min":0.47399999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.48999999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.97199999999999998,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.82299999999999995}]},{"osd":5,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59099999999999997,"15min":0.58599999999999997},"min":{"1min":0.39200000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82399999999999995,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.60699999999999998,"15min":0.63500000000000001},"min":{"1min":0.41699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79200000000000004,"5min":1.79,"15min":1.79},"last":0.77200000000000002}]},{"osd":6,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.624,"15min":0.64500000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95799999999999996,"5min":1.123,"15min":1.123},"last":0.75},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.68500000000000005,"15min":0.70499999999999996},"min":{"1min":0.47299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93999999999999995,"5min":1.925,"15min":1.925},"last":1.004}]},{"osd":7,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68799999999999994,"15min":0.71599999999999997},"min":{"1min":0.41999999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.97999999999999998,"5min":1.843,"15min":1.843},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.75600000000000001,"15min":0.77700000000000002},"min":{"1min":0.59699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.998,"5min":1.593,"15min":1.593},"last":0.96299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574910,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58496,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313344,"statfs":{"total":96636764160,"available":96576864256,"internally_reserved":0,"allocated":958464,"data_stored":438112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.434,"15min":0.41699999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.48599999999999999,"15min":0.47399999999999998},"min":{"1min":0.30199999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.7080000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.34200000000000003}]},{"osd":1,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46100000000000002,"15min":0.46400000000000002},"min":{"1min":0.32300000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.57399999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.54900000000000004,"15min":0.51700000000000002},"min":{"1min":0.249,"5min":0.247,"15min":0.247},"max":{"1min":4.5599999999999996,"5min":4.5599999999999996,"15min":4.5599999999999996},"last":0.48399999999999999}]},{"osd":2,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.34699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.57399999999999995,"15min":0.56499999999999995},"min":{"1min":0.40400000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.45500000000000002}]},{"osd":4,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":0.90900000000000003,"15min":0.81999999999999995},"min":{"1min":0.47899999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":12.695,"5min":12.695,"15min":12.695},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.097,"5min":0.82699999999999996,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":6.3250000000000002,"5min":6.3250000000000002,"15min":6.3250000000000002},"last":0.751}]},{"osd":5,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.81200000000000006,"15min":0.83599999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.9430000000000001,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.81000000000000005,"15min":0.78600000000000003},"min":{"1min":0.504,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.73899999999999999}]},{"osd":6,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":0.84799999999999998,"15min":0.81899999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":6.1790000000000003,"5min":6.1790000000000003,"15min":6.1790000000000003},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.80900000000000005,"15min":0.80700000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":0.70799999999999996}]},{"osd":7,"last update":"Tue Nov 28 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.82499999999999996,"15min":0.82299999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":3.1709999999999998,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.0920000000000001,"5min":0.85599999999999998,"15min":0.82799999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":6.2249999999999996,"5min":6.2249999999999996,"15min":6.2249999999999996},"last":0.72499999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574911,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":966656,"data_stored":438272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:27:28 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.47499999999999998,"15min":0.499},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61399999999999999,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47699999999999998,"15min":0.505},"min":{"1min":0.30599999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.61799999999999999,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.41999999999999998}]},{"osd":1,"last update":"Tue Nov 28 18:27:28 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.502,"15min":0.52400000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65800000000000003,"5min":1.48,"15min":1.48},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.50700000000000001,"15min":0.51400000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69099999999999995,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.54200000000000004}]},{"osd":3,"last update":"Tue Nov 28 18:27:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48399999999999999,"15min":0.501},"min":{"1min":0.34000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.626,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55000000000000004,"15min":0.56699999999999995},"min":{"1min":0.34899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67800000000000005,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.45600000000000002}]},{"osd":4,"last update":"Tue Nov 28 18:27:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.77200000000000002},"min":{"1min":0.61399999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.032,"5min":1.512,"15min":1.512},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.75,"15min":0.77600000000000002},"min":{"1min":0.54000000000000004,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.96499999999999997,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.81399999999999995}]},{"osd":5,"last update":"Tue Nov 28 18:27:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73699999999999999,"15min":0.76300000000000001},"min":{"1min":0.52000000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93899999999999995,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.873,"15min":0.99199999999999999},"min":{"1min":0.52500000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97799999999999998,"5min":7.4820000000000002,"15min":7.4820000000000002},"last":0.80300000000000005}]},{"osd":6,"last update":"Tue Nov 28 18:27:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.76900000000000002,"15min":0.79500000000000004},"min":{"1min":0.58799999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.97899999999999998,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.79500000000000004,"15min":0.86099999999999999},"min":{"1min":0.55200000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.99299999999999999,"5min":3.9380000000000002,"15min":3.9380000000000002},"last":0.79100000000000004}]},{"osd":7,"last update":"Tue Nov 28 18:27:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.80500000000000005,"15min":0.83699999999999997},"min":{"1min":0.56699999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.022,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.82699999999999996,"15min":0.90900000000000003},"min":{"1min":0.51000000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0549999999999999,"5min":4.798,"15min":4.798},"last":0.83099999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36280,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335560,"statfs":{"total":96636764160,"available":96599613440,"internally_reserved":0,"allocated":962560,"data_stored":437642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.40999999999999998,"15min":0.42399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58599999999999997,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.44600000000000001,"15min":0.45000000000000001},"min":{"1min":0.27300000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.47499999999999998}]},{"osd":2,"last update":"Tue Nov 28 18:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.443,"15min":0.44800000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.59399999999999997,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47899999999999998,"15min":0.47799999999999998},"min":{"1min":0.36499999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.70499999999999996,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.73399999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52700000000000002,"15min":0.53700000000000003},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.749,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.499,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.35699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73399999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54800000000000004}]},{"osd":4,"last update":"Tue Nov 28 18:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.72699999999999998,"15min":0.747},"min":{"1min":0.441,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.004,"5min":1.512,"15min":1.512},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.752,"15min":0.76800000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0780000000000001,"5min":1.155,"15min":1.155},"last":0.59199999999999997}]},{"osd":5,"last update":"Tue Nov 28 18:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.71899999999999997,"15min":0.746},"min":{"1min":0.46999999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.872,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71399999999999997,"15min":0.745},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.90200000000000002,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.67700000000000005}]},{"osd":6,"last update":"Tue Nov 28 18:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.751,"15min":0.77700000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1279999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77200000000000002,"15min":0.80400000000000005},"min":{"1min":0.55500000000000005,"5min":0.498,"15min":0.498},"max":{"1min":1.0429999999999999,"5min":1.448,"15min":1.448},"last":0.71599999999999997}]},{"osd":7,"last update":"Tue Nov 28 18:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73099999999999998,"15min":0.749},"min":{"1min":0.53300000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.94499999999999995,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.626},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.747,"15min":0.77500000000000002},"min":{"1min":0.54500000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0109999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.80200000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36852,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334988,"statfs":{"total":96636764160,"available":96599027712,"internally_reserved":0,"allocated":1548288,"data_stored":1027898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.42799999999999999,"15min":0.442},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.54500000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.46700000000000003,"15min":0.47099999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65200000000000002,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.437}]},{"osd":2,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.497,"15min":0.50900000000000001},"min":{"1min":0.318,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61699999999999999,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.496,"15min":0.51400000000000001},"min":{"1min":0.33800000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.65000000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.54300000000000004}]},{"osd":3,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48099999999999998,"15min":0.48899999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72599999999999998,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53600000000000003,"15min":0.55300000000000005},"min":{"1min":0.36599999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.75900000000000001,"5min":1.087,"15min":1.087},"last":0.47499999999999998}]},{"osd":4,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71799999999999997,"15min":0.71499999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95299999999999996,"5min":1.131,"15min":1.131},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72299999999999998,"15min":0.71999999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88700000000000001,"5min":1.244,"15min":1.244},"last":0.66700000000000004}]},{"osd":5,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93600000000000005,"5min":1.121,"15min":1.121},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.746,"15min":0.74399999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.94299999999999995,"5min":1.075,"15min":1.075},"last":0.69299999999999995}]},{"osd":6,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73299999999999998,"15min":0.73499999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.88300000000000001,"5min":1.407,"15min":1.407},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.748,"15min":0.73899999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.90400000000000003,"5min":1.089,"15min":1.089},"last":0.63800000000000001}]},{"osd":7,"last update":"Tue Nov 28 18:27:31 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.53900000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.99299999999999999,"5min":1.212,"15min":1.212},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.755,"15min":0.751},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.97699999999999998,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.76000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:28:18.196 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T18:28:18.197 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-28T18:28:18.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.374+0000 7f569229c640 1 Processor -- start 2023-11-28T18:28:18.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.374+0000 7f569229c640 1 -- start start 2023-11-28T18:28:18.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f569229c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f568c159610 0x7f568c0ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:18.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f569229c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f568c06cef0 0x7f568c0ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:18.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f569229c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f568c0cde30 con 0x7f568c0cc760 2023-11-28T18:28:18.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f569229c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568c1599f0 con 0x7f568c06cef0 2023-11-28T18:28:18.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f569229c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568c06de20 con 0x7f568c159610 2023-11-28T18:28:18.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568b7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f568c0cc760 0x7f568c06c7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47940/0 (socket says 172.21.15.105:47940) 2023-11-28T18:28:18.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568b7fe640 1 -- 172.21.15.105:0/173987865 learned_addr learned my addr 172.21.15.105:0/173987865 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:18.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f5690839640 1 --2- 172.21.15.105:0/173987865 >> v2:172.21.15.148:3300/0 conn(0x7f568c06cef0 0x7f568c0ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:18.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568bfff640 1 --2- 172.21.15.105:0/173987865 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f568c159610 0x7f568c0ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:18.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568affd640 1 -- 172.21.15.105:0/173987865 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1041073927 0 0) 0x7f568c0cde30 con 0x7f568c0cc760 2023-11-28T18:28:18.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568affd640 1 -- 172.21.15.105:0/173987865 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5674003530 con 0x7f568c0cc760 2023-11-28T18:28:18.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568affd640 1 -- 172.21.15.105:0/173987865 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3144401089 0 0) 0x7f5674003530 con 0x7f568c0cc760 2023-11-28T18:28:18.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568affd640 1 -- 172.21.15.105:0/173987865 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f568c159610 msgr2=0x7f568c0ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:18.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568affd640 1 --2- 172.21.15.105:0/173987865 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f568c159610 0x7f568c0ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568affd640 1 -- 172.21.15.105:0/173987865 >> v2:172.21.15.148:3300/0 conn(0x7f568c06cef0 msgr2=0x7f568c0ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:18.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568affd640 1 --2- 172.21.15.105:0/173987865 >> v2:172.21.15.148:3300/0 conn(0x7f568c06cef0 0x7f568c0ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568affd640 1 -- 172.21.15.105:0/173987865 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c0cd220 con 0x7f568c0cc760 2023-11-28T18:28:18.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568affd640 1 -- 172.21.15.105:0/173987865 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f5680001680 con 0x7f568c0cc760 2023-11-28T18:28:18.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568affd640 1 -- 172.21.15.105:0/173987865 >> v1:172.21.15.105:6789/0 conn(0x7f568c0cc760 legacy=0x7f568c06c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568affd640 1 --2- 172.21.15.105:0/173987865 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f568c159610 0x7f5674003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568affd640 1 --2- 172.21.15.105:0/173987865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5674003f50 0x7f56740047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568affd640 1 --2- 172.21.15.105:0/173987865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5674004d10 0x7f5674005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568affd640 1 -- 172.21.15.105:0/173987865 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c1599f0 con 0x7f5674003f50 2023-11-28T18:28:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568affd640 1 -- 172.21.15.105:0/173987865 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c06de20 con 0x7f5674004d10 2023-11-28T18:28:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568affd640 1 -- 172.21.15.105:0/173987865 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567400dba0 con 0x7f568c159610 2023-11-28T18:28:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568b7fe640 1 --2- 172.21.15.105:0/173987865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5674003f50 0x7f56740047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568bfff640 1 --2- 172.21.15.105:0/173987865 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f568c159610 0x7f5674003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f5690839640 1 --2- 172.21.15.105:0/173987865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5674004d10 0x7f5674005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568b7fe640 1 -- 172.21.15.105:0/173987865 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f568c159610 msgr2=0x7f5674003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:18.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568b7fe640 1 --2- 172.21.15.105:0/173987865 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f568c159610 0x7f5674003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568b7fe640 1 -- 172.21.15.105:0/173987865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5674004d10 msgr2=0x7f5674005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:18.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568b7fe640 1 --2- 172.21.15.105:0/173987865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5674004d10 0x7f5674005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f568b7fe640 1 -- 172.21.15.105:0/173987865 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5674003530 con 0x7f5674003f50 2023-11-28T18:28:18.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.378+0000 7f5690839640 1 --2- 172.21.15.105:0/173987865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5674004d10 0x7f5674005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:18.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.382+0000 7f568b7fe640 1 --2- 172.21.15.105:0/173987865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5674003f50 0x7f56740047d0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f568000c140 tx=0x7f568001f2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6c9f7acb03c518 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:18.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.382+0000 7f568affd640 1 -- 172.21.15.105:0/173987865 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5680005ae0 con 0x7f5674003f50 2023-11-28T18:28:18.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.382+0000 7f568affd640 1 -- 172.21.15.105:0/173987865 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56800ab070 con 0x7f5674003f50 2023-11-28T18:28:18.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.382+0000 7f568affd640 1 -- 172.21.15.105:0/173987865 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5680007070 con 0x7f5674003f50 2023-11-28T18:28:18.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.382+0000 7f569229c640 1 -- 172.21.15.105:0/173987865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5674003f50 msgr2=0x7f56740047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:18.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.382+0000 7f569229c640 1 --2- 172.21.15.105:0/173987865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5674003f50 0x7f56740047d0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f568000c140 tx=0x7f568001f2f0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.382+0000 7f5690839640 1 -- 172.21.15.105:0/173987865 reap_dead start 2023-11-28T18:28:18.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.382+0000 7f569229c640 1 -- 172.21.15.105:0/173987865 shutdown_connections 2023-11-28T18:28:18.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.382+0000 7f569229c640 1 -- 172.21.15.105:0/173987865 >> 172.21.15.105:0/173987865 conn(0x7f568c0ceb90 msgr2=0x7f568c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:18.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.382+0000 7f569229c640 1 -- 172.21.15.105:0/173987865 shutdown_connections 2023-11-28T18:28:18.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.382+0000 7f569229c640 1 -- 172.21.15.105:0/173987865 wait complete. 2023-11-28T18:28:18.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f569229c640 1 Processor -- start 2023-11-28T18:28:18.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f569229c640 1 -- start start 2023-11-28T18:28:18.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f569229c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f568c1507c0 0x7f568c0b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:18.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f569229c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f568c0b2e10 0x7f568c0b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:18.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f569229c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f568c0b4220 0x7f568c1fa340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:18.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f569229c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0b3440 con 0x7f568c1507c0 2023-11-28T18:28:18.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f569229c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0b46a0 con 0x7f568c0b2e10 2023-11-28T18:28:18.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f569229c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568c155ae0 con 0x7f568c0b4220 2023-11-28T18:28:18.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f568bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f568c1507c0 0x7f568c0b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:18.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f568bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f568c1507c0 0x7f568c0b28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46446/0 (socket says 172.21.15.105:46446) 2023-11-28T18:28:18.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f568bfff640 1 -- 172.21.15.105:0/2652781666 learned_addr learned my addr 172.21.15.105:0/2652781666 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:18.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f5690839640 1 --2- 172.21.15.105:0/2652781666 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f568c0b4220 0x7f568c1fa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:18.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f568b7fe640 1 --2- 172.21.15.105:0/2652781666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f568c0b2e10 0x7f568c0b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:18.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f568bfff640 1 -- 172.21.15.105:0/2652781666 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f568c0b4220 msgr2=0x7f568c1fa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:18.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f568bfff640 1 --2- 172.21.15.105:0/2652781666 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f568c0b4220 0x7f568c1fa340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f568bfff640 1 -- 172.21.15.105:0/2652781666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f568c0b2e10 msgr2=0x7f568c0b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:18.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f568bfff640 1 --2- 172.21.15.105:0/2652781666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f568c0b2e10 0x7f568c0b3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f568bfff640 1 -- 172.21.15.105:0/2652781666 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c155d80 con 0x7f568c1507c0 2023-11-28T18:28:18.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f568b7fe640 1 --2- 172.21.15.105:0/2652781666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f568c0b2e10 0x7f568c0b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:18.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f568bfff640 1 --2- 172.21.15.105:0/2652781666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f568c1507c0 0x7f568c0b28d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f567c0119d0 tx=0x7f567c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:18.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f5688ff9640 1 -- 172.21.15.105:0/2652781666 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c00fb50 con 0x7f568c1507c0 2023-11-28T18:28:18.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f5688ff9640 1 -- 172.21.15.105:0/2652781666 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c00fcb0 con 0x7f568c1507c0 2023-11-28T18:28:18.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f5688ff9640 1 -- 172.21.15.105:0/2652781666 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c014440 con 0x7f568c1507c0 2023-11-28T18:28:18.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f569229c640 1 -- 172.21.15.105:0/2652781666 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568c156060 con 0x7f568c1507c0 2023-11-28T18:28:18.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f569229c640 1 -- 172.21.15.105:0/2652781666 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568c156620 con 0x7f568c1507c0 2023-11-28T18:28:18.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.386+0000 7f569229c640 1 -- 172.21.15.105:0/2652781666 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568c0b31f0 con 0x7f568c1507c0 2023-11-28T18:28:18.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.390+0000 7f5688ff9640 1 -- 172.21.15.105:0/2652781666 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f567c0145a0 con 0x7f568c1507c0 2023-11-28T18:28:18.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.390+0000 7f5688ff9640 1 --2- 172.21.15.105:0/2652781666 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56780668d0 0x7f5678068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:18.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.390+0000 7f5688ff9640 1 -- 172.21.15.105:0/2652781666 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f567c084db0 con 0x7f568c1507c0 2023-11-28T18:28:18.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.390+0000 7f568b7fe640 1 --2- 172.21.15.105:0/2652781666 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56780668d0 0x7f5678068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:18.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.390+0000 7f568b7fe640 1 --2- 172.21.15.105:0/2652781666 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56780668d0 0x7f5678068d90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f56800ae000 tx=0x7f5680007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:18.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.394+0000 7f5688ff9640 1 -- 172.21.15.105:0/2652781666 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f567c051f60 con 0x7f568c1507c0 2023-11-28T18:28:18.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.594+0000 7f569229c640 1 -- 172.21.15.105:0/2652781666 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f568c067570 con 0x7f568c1507c0 2023-11-28T18:28:18.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.598+0000 7f5688ff9640 1 -- 172.21.15.105:0/2652781666 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f567c022020 con 0x7f568c1507c0 2023-11-28T18:28:18.603 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2023-11-28T18:28:18.603 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDpL2ZlSt0VMxAAQ8+j1BgBPMJ2cuiZj8aG0w== 2023-11-28T18:28:18.603 INFO:teuthology.orchestra.run.smithi105.stdout: caps mds = "allow" 2023-11-28T18:28:18.603 INFO:teuthology.orchestra.run.smithi105.stdout: caps mon = "allow r" 2023-11-28T18:28:18.603 INFO:teuthology.orchestra.run.smithi105.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-28T18:28:18.603 INFO:teuthology.orchestra.run.smithi105.stderr:updated caps for client.0 2023-11-28T18:28:18.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.598+0000 7f569229c640 1 -- 172.21.15.105:0/2652781666 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56780668d0 msgr2=0x7f5678068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:18.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.598+0000 7f569229c640 1 --2- 172.21.15.105:0/2652781666 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56780668d0 0x7f5678068d90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f56800ae000 tx=0x7f5680007040 comp rx=0 tx=0).stop 2023-11-28T18:28:18.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.598+0000 7f569229c640 1 -- 172.21.15.105:0/2652781666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f568c1507c0 msgr2=0x7f568c0b28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:18.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.598+0000 7f569229c640 1 --2- 172.21.15.105:0/2652781666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f568c1507c0 0x7f568c0b28d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f567c0119d0 tx=0x7f567c011e90 comp rx=0 tx=0).stop 2023-11-28T18:28:18.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.598+0000 7f569229c640 1 -- 172.21.15.105:0/2652781666 shutdown_connections 2023-11-28T18:28:18.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.598+0000 7f569229c640 1 --2- 172.21.15.105:0/2652781666 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f568c0b4220 0x7f568c1fa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.598+0000 7f569229c640 1 --2- 172.21.15.105:0/2652781666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f568c0b2e10 0x7f568c0b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.598+0000 7f569229c640 1 --2- 172.21.15.105:0/2652781666 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56780668d0 0x7f5678068d90 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.598+0000 7f569229c640 1 --2- 172.21.15.105:0/2652781666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f568c1507c0 0x7f568c0b28d0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.598+0000 7f569229c640 1 -- 172.21.15.105:0/2652781666 >> 172.21.15.105:0/2652781666 conn(0x7f568c0ceb90 msgr2=0x7f568c15d010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:18.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.602+0000 7f569229c640 1 -- 172.21.15.105:0/2652781666 shutdown_connections 2023-11-28T18:28:18.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.602+0000 7f569229c640 1 -- 172.21.15.105:0/2652781666 wait complete. 2023-11-28T18:28:18.624 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:28:18.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.758+0000 7fce9a7e6640 1 Processor -- start 2023-11-28T18:28:18.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.758+0000 7fce9a7e6640 1 -- start start 2023-11-28T18:28:18.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.758+0000 7fce9a7e6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fce940d3c30 0x7fce940d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:18.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.758+0000 7fce9a7e6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce940b25b0 con 0x7fce940b2d20 2023-11-28T18:28:18.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.758+0000 7fce9a7e6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce940cea30 con 0x7fce9415d4f0 2023-11-28T18:28:18.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.758+0000 7fce9a7e6640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce940b2900 con 0x7fce940d3c30 2023-11-28T18:28:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.758+0000 7fce98d83640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fce940b2d20 0x7fce940b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47944/0 (socket says 172.21.15.105:47944) 2023-11-28T18:28:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.758+0000 7fce98d83640 1 -- 172.21.15.105:0/1758026835 learned_addr learned my addr 172.21.15.105:0/1758026835 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.758+0000 7fce937fe640 1 --2- 172.21.15.105:0/1758026835 >> v2:172.21.15.105:3300/0 conn(0x7fce940d3c30 0x7fce940d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.758+0000 7fce937fe640 1 -- 172.21.15.105:0/1758026835 >> v1:172.21.15.148:6789/0 conn(0x7fce9415d4f0 legacy=0x7fce940d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.758+0000 7fce937fe640 1 -- 172.21.15.105:0/1758026835 >> v1:172.21.15.105:6789/0 conn(0x7fce940b2d20 legacy=0x7fce940b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.758+0000 7fce937fe640 1 -- 172.21.15.105:0/1758026835 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce940ced10 con 0x7fce940d3c30 2023-11-28T18:28:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.758+0000 7fce92ffd640 1 -- 172.21.15.105:0/1758026835 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4256513809 0 0) 0x7fce940b25b0 con 0x7fce940b2d20 2023-11-28T18:28:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.762+0000 7fce937fe640 1 --2- 172.21.15.105:0/1758026835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce940d3c30 0x7fce940d1520 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fce880097c0 tx=0x7fce8801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd45c3ac0959bc0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.762+0000 7fce92ffd640 1 -- 172.21.15.105:0/1758026835 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce880ad660 con 0x7fce940d3c30 2023-11-28T18:28:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.762+0000 7fce92ffd640 1 -- 172.21.15.105:0/1758026835 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce880ad7c0 con 0x7fce940d3c30 2023-11-28T18:28:18.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.762+0000 7fce92ffd640 1 -- 172.21.15.105:0/1758026835 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce880adb50 con 0x7fce940d3c30 2023-11-28T18:28:18.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.762+0000 7fce9a7e6640 1 -- 172.21.15.105:0/1758026835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce940d3c30 msgr2=0x7fce940d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:18.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.762+0000 7fce9a7e6640 1 --2- 172.21.15.105:0/1758026835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce940d3c30 0x7fce940d1520 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fce880097c0 tx=0x7fce8801c3c0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.762+0000 7fce9a7e6640 1 -- 172.21.15.105:0/1758026835 shutdown_connections 2023-11-28T18:28:18.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.762+0000 7fce9a7e6640 1 --2- 172.21.15.105:0/1758026835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce940d3c30 0x7fce940d1520 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.762+0000 7fce9a7e6640 1 -- 172.21.15.105:0/1758026835 >> 172.21.15.105:0/1758026835 conn(0x7fce940d3030 msgr2=0x7fce940d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:18.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.762+0000 7fce9a7e6640 1 -- 172.21.15.105:0/1758026835 shutdown_connections 2023-11-28T18:28:18.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.762+0000 7fce9a7e6640 1 -- 172.21.15.105:0/1758026835 wait complete. 2023-11-28T18:28:18.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce9a7e6640 1 Processor -- start 2023-11-28T18:28:18.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce9a7e6640 1 -- start start 2023-11-28T18:28:18.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce9a7e6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce940b2d20 0x7fce940b7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:18.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce9a7e6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce940d3c30 0x7fce940b7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:18.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce9a7e6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9415d4f0 0x7fce940cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:18.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce9a7e6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce940d0a20 con 0x7fce9415d4f0 2023-11-28T18:28:18.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce9a7e6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce940ccfc0 con 0x7fce940d3c30 2023-11-28T18:28:18.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce9a7e6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce940cc7f0 con 0x7fce940b2d20 2023-11-28T18:28:18.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce98d83640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9415d4f0 0x7fce940cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:18.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce98d83640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9415d4f0 0x7fce940cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46454/0 (socket says 172.21.15.105:46454) 2023-11-28T18:28:18.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce98d83640 1 -- 172.21.15.105:0/3385462614 learned_addr learned my addr 172.21.15.105:0/3385462614 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:18.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce937fe640 1 --2- 172.21.15.105:0/3385462614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce940d3c30 0x7fce940b7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:18.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce93fff640 1 --2- 172.21.15.105:0/3385462614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce940b2d20 0x7fce940b7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:18.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce98d83640 1 -- 172.21.15.105:0/3385462614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce940b2d20 msgr2=0x7fce940b7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:18.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce98d83640 1 --2- 172.21.15.105:0/3385462614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce940b2d20 0x7fce940b7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce98d83640 1 -- 172.21.15.105:0/3385462614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce940d3c30 msgr2=0x7fce940b7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:18.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce98d83640 1 --2- 172.21.15.105:0/3385462614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce940d3c30 0x7fce940b7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce98d83640 1 -- 172.21.15.105:0/3385462614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce88009420 con 0x7fce9415d4f0 2023-11-28T18:28:18.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce937fe640 1 --2- 172.21.15.105:0/3385462614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce940d3c30 0x7fce940b7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:18.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce93fff640 1 --2- 172.21.15.105:0/3385462614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce940b2d20 0x7fce940b7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:18.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce98d83640 1 --2- 172.21.15.105:0/3385462614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9415d4f0 0x7fce940cb4b0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00c780 tx=0x7fce8c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:18.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce90ff9640 1 -- 172.21.15.105:0/3385462614 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c00ce90 con 0x7fce9415d4f0 2023-11-28T18:28:18.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce90ff9640 1 -- 172.21.15.105:0/3385462614 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8c00a640 con 0x7fce9415d4f0 2023-11-28T18:28:18.779 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce90ff9640 1 -- 172.21.15.105:0/3385462614 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c00a9d0 con 0x7fce9415d4f0 2023-11-28T18:28:18.779 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce9a7e6640 1 -- 172.21.15.105:0/3385462614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce940cb020 con 0x7fce9415d4f0 2023-11-28T18:28:18.779 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.766+0000 7fce9a7e6640 1 -- 172.21.15.105:0/3385462614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce940ccce0 con 0x7fce9415d4f0 2023-11-28T18:28:18.779 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.770+0000 7fce9a7e6640 1 -- 172.21.15.105:0/3385462614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce940cb1b0 con 0x7fce9415d4f0 2023-11-28T18:28:18.779 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.770+0000 7fce90ff9640 1 -- 172.21.15.105:0/3385462614 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fce8c00ab30 con 0x7fce9415d4f0 2023-11-28T18:28:18.780 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.770+0000 7fce90ff9640 1 --2- 172.21.15.105:0/3385462614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fce800668d0 0x7fce80068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:18.780 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.770+0000 7fce90ff9640 1 -- 172.21.15.105:0/3385462614 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fce8c083b50 con 0x7fce9415d4f0 2023-11-28T18:28:18.781 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.770+0000 7fce93fff640 1 --2- 172.21.15.105:0/3385462614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fce800668d0 0x7fce80068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:18.781 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.770+0000 7fce93fff640 1 --2- 172.21.15.105:0/3385462614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fce800668d0 0x7fce80068d90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fce84000f70 tx=0x7fce84009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:18.781 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.774+0000 7fce90ff9640 1 -- 172.21.15.105:0/3385462614 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fce8c050bb0 con 0x7fce9415d4f0 2023-11-28T18:28:18.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.958+0000 7fce9a7e6640 1 -- 172.21.15.105:0/3385462614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce94067570 con 0x7fce9415d4f0 2023-11-28T18:28:18.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.958+0000 7fce90ff9640 1 -- 172.21.15.105:0/3385462614 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fce8c018070 con 0x7fce9415d4f0 2023-11-28T18:28:18.964 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:18.964 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":71,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1455698212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":1455698212},{"type":"v1","addr":"172.21.15.105:6837","nonce":1455698212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6859,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/759522673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":759522673},{"type":"v1","addr":"172.21.15.148:6834","nonce":759522673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2489361300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2489361300},{"type":"v1","addr":"172.21.15.148:6835","nonce":2489361300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:28:13.097621+0000","modified":"2023-11-28T18:28:16.128249+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7194},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7194":{"gid":7194,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6836/188173955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":188173955},{"type":"v1","addr":"172.21.15.105:6836","nonce":188173955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-28T18:28:18.964 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 71 2023-11-28T18:28:18.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.962+0000 7fce9a7e6640 1 -- 172.21.15.105:0/3385462614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fce800668d0 msgr2=0x7fce80068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:18.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.962+0000 7fce9a7e6640 1 --2- 172.21.15.105:0/3385462614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fce800668d0 0x7fce80068d90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fce84000f70 tx=0x7fce84009040 comp rx=0 tx=0).stop 2023-11-28T18:28:18.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.962+0000 7fce9a7e6640 1 -- 172.21.15.105:0/3385462614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9415d4f0 msgr2=0x7fce940cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:18.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.962+0000 7fce9a7e6640 1 --2- 172.21.15.105:0/3385462614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9415d4f0 0x7fce940cb4b0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00c780 tx=0x7fce8c00cc40 comp rx=0 tx=0).stop 2023-11-28T18:28:18.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.962+0000 7fce9a7e6640 1 -- 172.21.15.105:0/3385462614 shutdown_connections 2023-11-28T18:28:18.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.962+0000 7fce9a7e6640 1 --2- 172.21.15.105:0/3385462614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fce800668d0 0x7fce80068d90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.962+0000 7fce9a7e6640 1 --2- 172.21.15.105:0/3385462614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9415d4f0 0x7fce940cb4b0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.962+0000 7fce9a7e6640 1 --2- 172.21.15.105:0/3385462614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce940d3c30 0x7fce940b7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.962+0000 7fce9a7e6640 1 --2- 172.21.15.105:0/3385462614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce940b2d20 0x7fce940b7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:18.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.962+0000 7fce9a7e6640 1 -- 172.21.15.105:0/3385462614 >> 172.21.15.105:0/3385462614 conn(0x7fce940d3030 msgr2=0x7fce9406b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:18.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.962+0000 7fce9a7e6640 1 -- 172.21.15.105:0/3385462614 shutdown_connections 2023-11-28T18:28:18.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:18.962+0000 7fce9a7e6640 1 -- 172.21.15.105:0/3385462614 wait complete. 2023-11-28T18:28:19.388 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:28:13.097621+0000', 'modified': '2023-11-28T18:28:16.128249+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7194}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7194': {'gid': 7194, 'name': 'a', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6836/188173955', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 188173955}, {'type': 'v1', 'addr': '172.21.15.105:6836', 'nonce': 188173955}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:28:19.388 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T18:28:19.388 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T18:28:19.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.522+0000 7f59d026b640 1 Processor -- start 2023-11-28T18:28:19.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.522+0000 7f59d026b640 1 -- start start 2023-11-28T18:28:19.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.522+0000 7f59d026b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59c80cc760 0x7f59c8164d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.522+0000 7f59d026b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59c80ccc10 con 0x7f59c8159f40 2023-11-28T18:28:19.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.522+0000 7f59d026b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59c806bf50 con 0x7f59c80caf60 2023-11-28T18:28:19.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.522+0000 7f59d026b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c80cb3c0 con 0x7f59c80cc760 2023-11-28T18:28:19.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.522+0000 7f59ce007640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f59c8159f40 0x7f59c80ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47946/0 (socket says 172.21.15.105:47946) 2023-11-28T18:28:19.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59ce007640 1 -- 172.21.15.105:0/230090973 learned_addr learned my addr 172.21.15.105:0/230090973 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:19.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59ce808640 1 --2- 172.21.15.105:0/230090973 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59c80cc760 0x7f59c8164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59cd005640 1 -- 172.21.15.105:0/230090973 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 924959768 0 0) 0x7f59c80ccc10 con 0x7f59c8159f40 2023-11-28T18:28:19.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59cd005640 1 -- 172.21.15.105:0/230090973 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59b4003530 con 0x7f59c8159f40 2023-11-28T18:28:19.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59cd005640 1 -- 172.21.15.105:0/230090973 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2692178570 0 0) 0x7f59c806bf50 con 0x7f59c80caf60 2023-11-28T18:28:19.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59cd005640 1 -- 172.21.15.105:0/230090973 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59c80ccc10 con 0x7f59c80caf60 2023-11-28T18:28:19.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59cd005640 1 -- 172.21.15.105:0/230090973 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2431513758 0 0) 0x7f59b4003530 con 0x7f59c8159f40 2023-11-28T18:28:19.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59cd005640 1 -- 172.21.15.105:0/230090973 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59c80cc760 msgr2=0x7f59c8164d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:19.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59cd005640 1 --2- 172.21.15.105:0/230090973 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59c80cc760 0x7f59c8164d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59cd005640 1 -- 172.21.15.105:0/230090973 >> v1:172.21.15.148:6789/0 conn(0x7f59c80caf60 legacy=0x7f59c80ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:19.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59cd005640 1 -- 172.21.15.105:0/230090973 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c815a320 con 0x7f59c8159f40 2023-11-28T18:28:19.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59cd005640 1 -- 172.21.15.105:0/230090973 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f59c4001a00 con 0x7f59c8159f40 2023-11-28T18:28:19.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59cd005640 1 -- 172.21.15.105:0/230090973 >> v1:172.21.15.105:6789/0 conn(0x7f59c8159f40 legacy=0x7f59c80ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:19.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59cd005640 1 --2- 172.21.15.105:0/230090973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b4003b40 0x7f59b4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59cd005640 1 --2- 172.21.15.105:0/230090973 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59c80cc760 0x7f59b40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59cd005640 1 --2- 172.21.15.105:0/230090973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59b4004d10 0x7f59b4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59cd005640 1 -- 172.21.15.105:0/230090973 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c80cb3c0 con 0x7f59b4004d10 2023-11-28T18:28:19.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59cd005640 1 -- 172.21.15.105:0/230090973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b400dd30 con 0x7f59b4003b40 2023-11-28T18:28:19.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59cd005640 1 -- 172.21.15.105:0/230090973 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b400def0 con 0x7f59c80cc760 2023-11-28T18:28:19.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59ce808640 1 --2- 172.21.15.105:0/230090973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59b4004d10 0x7f59b4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59cd806640 1 --2- 172.21.15.105:0/230090973 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59c80cc760 0x7f59b40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59ce007640 1 --2- 172.21.15.105:0/230090973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b4003b40 0x7f59b4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59ce808640 1 -- 172.21.15.105:0/230090973 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59c80cc760 msgr2=0x7f59b40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:19.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59ce808640 1 --2- 172.21.15.105:0/230090973 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59c80cc760 0x7f59b40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59ce808640 1 -- 172.21.15.105:0/230090973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b4003b40 msgr2=0x7f59b4003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:19.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59ce808640 1 --2- 172.21.15.105:0/230090973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b4003b40 0x7f59b4003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59ce808640 1 -- 172.21.15.105:0/230090973 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b4003530 con 0x7f59b4004d10 2023-11-28T18:28:19.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.526+0000 7f59ce007640 1 --2- 172.21.15.105:0/230090973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b4003b40 0x7f59b4003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:19.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.530+0000 7f59ce808640 1 --2- 172.21.15.105:0/230090973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59b4004d10 0x7f59b4005190 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009ae0 tx=0x7f59c001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e749951002900601 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:19.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.530+0000 7f59cd005640 1 -- 172.21.15.105:0/230090973 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c00a5640 con 0x7f59b4004d10 2023-11-28T18:28:19.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.530+0000 7f59cd005640 1 -- 172.21.15.105:0/230090973 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c00a57a0 con 0x7f59b4004d10 2023-11-28T18:28:19.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.530+0000 7f59cd005640 1 -- 172.21.15.105:0/230090973 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c00a5b50 con 0x7f59b4004d10 2023-11-28T18:28:19.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.530+0000 7f59d026b640 1 -- 172.21.15.105:0/230090973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59b4004d10 msgr2=0x7f59b4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:19.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.530+0000 7f59d026b640 1 --2- 172.21.15.105:0/230090973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59b4004d10 0x7f59b4005190 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009ae0 tx=0x7f59c001ca50 comp rx=0 tx=0).stop 2023-11-28T18:28:19.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.530+0000 7f59ce808640 1 -- 172.21.15.105:0/230090973 reap_dead start 2023-11-28T18:28:19.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.530+0000 7f59d026b640 1 -- 172.21.15.105:0/230090973 shutdown_connections 2023-11-28T18:28:19.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.530+0000 7f59d026b640 1 -- 172.21.15.105:0/230090973 >> 172.21.15.105:0/230090973 conn(0x7f59c80ceb90 msgr2=0x7f59c80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:19.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.530+0000 7f59d026b640 1 -- 172.21.15.105:0/230090973 shutdown_connections 2023-11-28T18:28:19.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.530+0000 7f59d026b640 1 -- 172.21.15.105:0/230090973 wait complete. 2023-11-28T18:28:19.538 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.530+0000 7f59d026b640 1 Processor -- start 2023-11-28T18:28:19.538 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.530+0000 7f59d026b640 1 -- start start 2023-11-28T18:28:19.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59d026b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59b4004d10 0x7f59c80b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59d026b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59c80b4030 0x7f59c80ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59d026b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59c80c92f0 0x7f59c8151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59d026b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c80b4840 con 0x7f59c80c92f0 2023-11-28T18:28:19.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59d026b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c80c8530 con 0x7f59c80b4030 2023-11-28T18:28:19.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59d026b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c80c8810 con 0x7f59b4004d10 2023-11-28T18:28:19.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59ce808640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59c80c92f0 0x7f59c8151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59ce808640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59c80c92f0 0x7f59c8151210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46464/0 (socket says 172.21.15.105:46464) 2023-11-28T18:28:19.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59ce808640 1 -- 172.21.15.105:0/545894131 learned_addr learned my addr 172.21.15.105:0/545894131 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:19.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59ce007640 1 --2- 172.21.15.105:0/545894131 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59b4004d10 0x7f59c80b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59cd806640 1 --2- 172.21.15.105:0/545894131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59c80b4030 0x7f59c80ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59ce808640 1 -- 172.21.15.105:0/545894131 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59b4004d10 msgr2=0x7f59c80b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:19.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59ce808640 1 --2- 172.21.15.105:0/545894131 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59b4004d10 0x7f59c80b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59ce808640 1 -- 172.21.15.105:0/545894131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59c80b4030 msgr2=0x7f59c80ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:19.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59ce808640 1 --2- 172.21.15.105:0/545894131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59c80b4030 0x7f59c80ca610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59ce808640 1 -- 172.21.15.105:0/545894131 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c80c8af0 con 0x7f59c80c92f0 2023-11-28T18:28:19.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59cd806640 1 --2- 172.21.15.105:0/545894131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59c80b4030 0x7f59c80ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:19.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59ce808640 1 --2- 172.21.15.105:0/545894131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59c80c92f0 0x7f59c8151210 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f59c001c880 tx=0x7f59b4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:19.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59beffd640 1 -- 172.21.15.105:0/545894131 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c00a5640 con 0x7f59c80c92f0 2023-11-28T18:28:19.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59beffd640 1 -- 172.21.15.105:0/545894131 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c0004230 con 0x7f59c80c92f0 2023-11-28T18:28:19.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59d026b640 1 -- 172.21.15.105:0/545894131 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c80c8dd0 con 0x7f59c80c92f0 2023-11-28T18:28:19.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59beffd640 1 -- 172.21.15.105:0/545894131 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c0003620 con 0x7f59c80c92f0 2023-11-28T18:28:19.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59d026b640 1 -- 172.21.15.105:0/545894131 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c80cab50 con 0x7f59c80c92f0 2023-11-28T18:28:19.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.534+0000 7f59beffd640 1 -- 172.21.15.105:0/545894131 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f59c00b4070 con 0x7f59c80c92f0 2023-11-28T18:28:19.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59beffd640 1 --2- 172.21.15.105:0/545894131 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59ac066900 0x7f59ac068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59beffd640 1 -- 172.21.15.105:0/545894131 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f59b8001d30 con 0x7f59c80c92f0 2023-11-28T18:28:19.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59ce007640 1 --2- 172.21.15.105:0/545894131 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59ac066900 0x7f59ac068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59d35dc000 1 Processor -- start 2023-11-28T18:28:19.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59d35dc000 1 -- start start 2023-11-28T18:28:19.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59ce007640 1 --2- 172.21.15.105:0/545894131 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59ac066900 0x7f59ac068dc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f59c4000f90 tx=0x7f59c4002020 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:19.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59d35dc000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55af1661ad30 0x55af1661b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59d35dc000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af1661b650 0x55af165cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59d35dc000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af165ce500 0x55af165ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59d35dc000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af16497110 con 0x55af165ce500 2023-11-28T18:28:19.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59d35dc000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af164279a0 con 0x55af1661b650 2023-11-28T18:28:19.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59d35dc000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af16424ed0 con 0x55af1661ad30 2023-11-28T18:28:19.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59ce808640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af165ce500 0x55af165ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59ce808640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af165ce500 0x55af165ce8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46476/0 (socket says 172.21.15.105:46476) 2023-11-28T18:28:19.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59ce808640 1 -- 172.21.15.105:0/773384633 learned_addr learned my addr 172.21.15.105:0/773384633 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:19.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59ce007640 1 --2- 172.21.15.105:0/773384633 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55af1661ad30 0x55af1661b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59cd806640 1 --2- 172.21.15.105:0/773384633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af1661b650 0x55af165cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59ce808640 1 -- 172.21.15.105:0/773384633 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55af1661ad30 msgr2=0x55af1661b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:19.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59ce808640 1 --2- 172.21.15.105:0/773384633 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55af1661ad30 0x55af1661b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59ce808640 1 -- 172.21.15.105:0/773384633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af1661b650 msgr2=0x55af165cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:19.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59ce808640 1 --2- 172.21.15.105:0/773384633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af1661b650 0x55af165cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59ce808640 1 -- 172.21.15.105:0/773384633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af16118890 con 0x55af165ce500 2023-11-28T18:28:19.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59ce808640 1 --2- 172.21.15.105:0/773384633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af165ce500 0x55af165ce8e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f59c00b0040 tx=0x7f59c00eed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a414833ded2cdcc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:19.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59abfff640 1 -- 172.21.15.105:0/773384633 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c00f3170 con 0x55af165ce500 2023-11-28T18:28:19.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59abfff640 1 -- 172.21.15.105:0/773384633 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c00f75e0 con 0x55af165ce500 2023-11-28T18:28:19.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59abfff640 1 -- 172.21.15.105:0/773384633 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c00096b0 con 0x55af165ce500 2023-11-28T18:28:19.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59d35dc000 1 -- 172.21.15.105:0/773384633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af165ce500 msgr2=0x55af165ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:19.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59d35dc000 1 --2- 172.21.15.105:0/773384633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af165ce500 0x55af165ce8e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f59c00b0040 tx=0x7f59c00eed20 comp rx=0 tx=0).stop 2023-11-28T18:28:19.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59d35dc000 1 -- 172.21.15.105:0/773384633 shutdown_connections 2023-11-28T18:28:19.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59d35dc000 1 --2- 172.21.15.105:0/773384633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af165ce500 0x55af165ce8e0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59d35dc000 1 --2- 172.21.15.105:0/773384633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af1661b650 0x55af165cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59d35dc000 1 --2- 172.21.15.105:0/773384633 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55af1661ad30 0x55af1661b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.538+0000 7f59d35dc000 1 -- 172.21.15.105:0/773384633 >> 172.21.15.105:0/773384633 conn(0x55af165fb8c0 msgr2=0x55af165fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:19.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59d35dc000 1 -- 172.21.15.105:0/773384633 shutdown_connections 2023-11-28T18:28:19.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59d35dc000 1 -- 172.21.15.105:0/773384633 wait complete. 2023-11-28T18:28:19.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59d35dc000 1 -- start start 2023-11-28T18:28:19.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59d35dc000 1 Processor -- start 2023-11-28T18:28:19.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59d35dc000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55af165ce500 0x55af166224b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59d35dc000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af1661ad30 0x55af166229f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59d35dc000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af1661b650 0x55af165d4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59d35dc000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af164207c0 con 0x55af1661b650 2023-11-28T18:28:19.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59d35dc000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af163d8bf0 con 0x55af1661ad30 2023-11-28T18:28:19.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59d35dc000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af163dea90 con 0x55af165ce500 2023-11-28T18:28:19.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59d35dc000 10 client.0 _ll_register_callbacks cb 0x55af165bbe20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:28:19.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59ce808640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af1661b650 0x55af165d4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59ce808640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af1661b650 0x55af165d4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46484/0 (socket says 172.21.15.105:46484) 2023-11-28T18:28:19.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59ce808640 1 -- 172.21.15.105:0/520978050 learned_addr learned my addr 172.21.15.105:0/520978050 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:19.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59ce007640 1 --2- 172.21.15.105:0/520978050 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55af165ce500 0x55af166224b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59cd806640 1 --2- 172.21.15.105:0/520978050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af1661ad30 0x55af166229f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59ce808640 1 -- 172.21.15.105:0/520978050 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55af165ce500 msgr2=0x55af166224b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:19.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59ce808640 1 --2- 172.21.15.105:0/520978050 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55af165ce500 0x55af166224b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59ce808640 1 -- 172.21.15.105:0/520978050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af1661ad30 msgr2=0x55af166229f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:19.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59ce808640 1 --2- 172.21.15.105:0/520978050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af1661ad30 0x55af166229f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59ce808640 1 -- 172.21.15.105:0/520978050 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c80cab50 con 0x55af1661b650 2023-11-28T18:28:19.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59ce007640 1 --2- 172.21.15.105:0/520978050 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55af165ce500 0x55af166224b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:19.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59ce808640 1 --2- 172.21.15.105:0/520978050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af1661b650 0x55af165d4310 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f59c00ef240 tx=0x7f59c0009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670646ac7dfc6549 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:19.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:28:19.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59a97fa640 1 -- 172.21.15.105:0/520978050 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c00f75e0 con 0x55af1661b650 2023-11-28T18:28:19.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59a97fa640 1 -- 172.21.15.105:0/520978050 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f59c00ee7f0 con 0x55af1661b650 2023-11-28T18:28:19.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59d35dc000 1 -- 172.21.15.105:0/520978050 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af163e2470 con 0x55af1661b650 2023-11-28T18:28:19.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59a97fa640 1 -- 172.21.15.105:0/520978050 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59c00ee9b0 con 0x55af1661b650 2023-11-28T18:28:19.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59d35dc000 10 client.7341 fetch_fsmap learned FSMap version 71 2023-11-28T18:28:19.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59d35dc000 1 -- 172.21.15.105:0/520978050 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55af163e1190 con 0x55af1661b650 2023-11-28T18:28:19.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59a97fa640 1 -- 172.21.15.105:0/520978050 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af163e2470 con 0x55af1661b650 2023-11-28T18:28:19.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59a97fa640 1 -- 172.21.15.105:0/520978050 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f59c00f7150 con 0x55af1661b650 2023-11-28T18:28:19.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59d35dc000 10 client.7341 fetch_fsmap finished waiting for FSMap version 71 2023-11-28T18:28:19.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59d35dc000 10 client.7341 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:28:19.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59d35dc000 20 client.7341 populate_metadata read hostname 'smithi105' 2023-11-28T18:28:19.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59d35dc000 1 --2- 172.21.15.105:0/520978050 >> [v2:172.21.15.105:6834/188173955,v1:172.21.15.105:6836/188173955] conn(0x55af165d5050 0x55af165d5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59d35dc000 4 client.7341 mds_command: new command op to 7194 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:28:19.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59d35dc000 1 -- 172.21.15.105:0/520978050 --> [v2:172.21.15.105:6834/188173955,v1:172.21.15.105:6836/188173955] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af163e5b50 con 0x55af165d5050 2023-11-28T18:28:19.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.542+0000 7f59ce007640 1 --2- 172.21.15.105:0/520978050 >> [v2:172.21.15.105:6834/188173955,v1:172.21.15.105:6836/188173955] conn(0x55af165d5050 0x55af165d5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.546+0000 7f59ce007640 1 --2- 172.21.15.105:0/520978050 >> [v2:172.21.15.105:6834/188173955,v1:172.21.15.105:6836/188173955] conn(0x55af165d5050 0x55af165d5430 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7e63cda99b4c432 server_cookie=75296e0591485e83 in_seq=0 out_seq=0 2023-11-28T18:28:19.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.546+0000 7f59a97fa640 10 client.7341 ms_handle_connect on v2:172.21.15.105:6834/188173955 2023-11-28T18:28:19.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.546+0000 7f59a97fa640 1 -- 172.21.15.105:0/520978050 <== mds.0 v2:172.21.15.105:6834/188173955 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55af163e5b50 con 0x55af165d5050 2023-11-28T18:28:19.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.546+0000 7f59a97fa640 10 client.7341 handle_command_reply: tid=0 2023-11-28T18:28:19.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.546+0000 7f59d35dc000 1 client.7341 shutdown 2023-11-28T18:28:19.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.546+0000 7f59d35dc000 1 -- 172.21.15.105:0/520978050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af1661b650 msgr2=0x55af165d4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:19.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.546+0000 7f59d35dc000 1 --2- 172.21.15.105:0/520978050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af1661b650 0x55af165d4310 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f59c00ef240 tx=0x7f59c0009190 comp rx=0 tx=0).stop 2023-11-28T18:28:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.546+0000 7f59d35dc000 1 -- 172.21.15.105:0/520978050 shutdown_connections 2023-11-28T18:28:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.546+0000 7f59d35dc000 1 --2- 172.21.15.105:0/520978050 >> [v2:172.21.15.105:6834/188173955,v1:172.21.15.105:6836/188173955] conn(0x55af165d5050 0x55af165d5430 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.546+0000 7f59d35dc000 1 --2- 172.21.15.105:0/520978050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af1661b650 0x55af165d4310 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.546+0000 7f59d35dc000 1 --2- 172.21.15.105:0/520978050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af1661ad30 0x55af166229f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.546+0000 7f59d35dc000 1 --2- 172.21.15.105:0/520978050 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55af165ce500 0x55af166224b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.546+0000 7f59a97fa640 0 client.7341 ms_handle_reset on v2:172.21.15.105:6834/188173955 2023-11-28T18:28:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.546+0000 7f59d35dc000 1 -- 172.21.15.105:0/520978050 >> 172.21.15.105:0/520978050 conn(0x55af165fb8c0 msgr2=0x55af165cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.546+0000 7f59d35dc000 1 -- 172.21.15.105:0/520978050 shutdown_connections 2023-11-28T18:28:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.546+0000 7f59d35dc000 1 -- 172.21.15.105:0/520978050 wait complete. 2023-11-28T18:28:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.546+0000 7f59d35dc000 20 client.7341 trim_cache size 0 max 16384 2023-11-28T18:28:19.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59d35dc000 1 Processor -- start 2023-11-28T18:28:19.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59d35dc000 1 -- start start 2023-11-28T18:28:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59d35dc000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af165ce500 0x55af165d6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59d35dc000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55af1661ad30 0x55af165d5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59d35dc000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af1661b650 0x55af165d5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59d35dc000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af165ce370 con 0x55af165ce500 2023-11-28T18:28:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59d35dc000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af160d9940 con 0x55af1661b650 2023-11-28T18:28:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59d35dc000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af1641ac30 con 0x55af1661ad30 2023-11-28T18:28:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59ce007640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af165ce500 0x55af165d6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59ce007640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af165ce500 0x55af165d6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46494/0 (socket says 172.21.15.105:46494) 2023-11-28T18:28:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59ce007640 1 -- 172.21.15.105:0/1655144914 learned_addr learned my addr 172.21.15.105:0/1655144914 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59ce808640 1 --2- 172.21.15.105:0/1655144914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af1661b650 0x55af165d5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59cd806640 1 --2- 172.21.15.105:0/1655144914 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55af1661ad30 0x55af165d5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59ce007640 1 -- 172.21.15.105:0/1655144914 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55af1661ad30 msgr2=0x55af165d5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59ce007640 1 --2- 172.21.15.105:0/1655144914 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55af1661ad30 0x55af165d5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59ce007640 1 -- 172.21.15.105:0/1655144914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af1661b650 msgr2=0x55af165d5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59ce007640 1 --2- 172.21.15.105:0/1655144914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af1661b650 0x55af165d5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59ce007640 1 -- 172.21.15.105:0/1655144914 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af163c7490 con 0x55af165ce500 2023-11-28T18:28:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59ce007640 1 --2- 172.21.15.105:0/1655144914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af165ce500 0x55af165d6590 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f59c4001f70 tx=0x7f59c4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7f36412b8cf73 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59bcff9640 1 -- 172.21.15.105:0/1655144914 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c4015e90 con 0x55af165ce500 2023-11-28T18:28:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59bcff9640 1 -- 172.21.15.105:0/1655144914 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c40280b0 con 0x55af165ce500 2023-11-28T18:28:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59bcff9640 1 -- 172.21.15.105:0/1655144914 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c4028440 con 0x55af165ce500 2023-11-28T18:28:19.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59d35dc000 1 -- 172.21.15.105:0/1655144914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af165ce500 msgr2=0x55af165d6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:19.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59d35dc000 1 --2- 172.21.15.105:0/1655144914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af165ce500 0x55af165d6590 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f59c4001f70 tx=0x7f59c4014000 comp rx=0 tx=0).stop 2023-11-28T18:28:19.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59d35dc000 1 -- 172.21.15.105:0/1655144914 shutdown_connections 2023-11-28T18:28:19.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59d35dc000 1 --2- 172.21.15.105:0/1655144914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af1661b650 0x55af165d5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59d35dc000 1 --2- 172.21.15.105:0/1655144914 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55af1661ad30 0x55af165d5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59d35dc000 1 --2- 172.21.15.105:0/1655144914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af165ce500 0x55af165d6590 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59d35dc000 1 -- 172.21.15.105:0/1655144914 >> 172.21.15.105:0/1655144914 conn(0x55af165fb8c0 msgr2=0x55af16628b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:19.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59d35dc000 1 -- 172.21.15.105:0/1655144914 shutdown_connections 2023-11-28T18:28:19.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.562+0000 7f59d35dc000 1 -- 172.21.15.105:0/1655144914 wait complete. 2023-11-28T18:28:19.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59d35dc000 1 -- start start 2023-11-28T18:28:19.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59d35dc000 1 Processor -- start 2023-11-28T18:28:19.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59d35dc000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af165ce500 0x55af1667bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59d35dc000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af1661ad30 0x55af1667c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59d35dc000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55af1661b650 0x55af1661fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59d35dc000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af165d3d70 con 0x55af165ce500 2023-11-28T18:28:19.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59d35dc000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af163f5b70 con 0x55af1661ad30 2023-11-28T18:28:19.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59d35dc000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af164a7570 con 0x55af1661b650 2023-11-28T18:28:19.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59d35dc000 10 client.0 _ll_register_callbacks cb 0x55af16497060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:28:19.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59ce007640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af165ce500 0x55af1667bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59ce007640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af165ce500 0x55af1667bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46502/0 (socket says 172.21.15.105:46502) 2023-11-28T18:28:19.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59ce007640 1 -- 172.21.15.105:0/3477230756 learned_addr learned my addr 172.21.15.105:0/3477230756 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:19.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59cd806640 1 --2- 172.21.15.105:0/3477230756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af1661ad30 0x55af1667c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59ce808640 1 --2- 172.21.15.105:0/3477230756 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55af1661b650 0x55af1661fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59ce007640 1 -- 172.21.15.105:0/3477230756 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55af1661b650 msgr2=0x55af1661fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:19.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59ce007640 1 --2- 172.21.15.105:0/3477230756 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55af1661b650 0x55af1661fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59ce007640 1 -- 172.21.15.105:0/3477230756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af1661ad30 msgr2=0x55af1667c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:19.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59ce007640 1 --2- 172.21.15.105:0/3477230756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af1661ad30 0x55af1667c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59ce007640 1 -- 172.21.15.105:0/3477230756 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c400f040 con 0x55af165ce500 2023-11-28T18:28:19.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59ce007640 1 --2- 172.21.15.105:0/3477230756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af165ce500 0x55af1667bbf0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x55af163aa9c0 tx=0x7f59c4001cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6288235b2573769e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:19.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:28:19.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59a97fa640 1 -- 172.21.15.105:0/3477230756 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c4028920 con 0x55af165ce500 2023-11-28T18:28:19.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59a97fa640 1 -- 172.21.15.105:0/3477230756 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f59c4028e40 con 0x55af165ce500 2023-11-28T18:28:19.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59d35dc000 1 -- 172.21.15.105:0/3477230756 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af165d46e0 con 0x55af165ce500 2023-11-28T18:28:19.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59a97fa640 1 -- 172.21.15.105:0/3477230756 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59c4025400 con 0x55af165ce500 2023-11-28T18:28:19.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59d35dc000 10 client.7347 fetch_fsmap learned FSMap version 71 2023-11-28T18:28:19.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59d35dc000 1 -- 172.21.15.105:0/3477230756 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55af16118cc0 con 0x55af165ce500 2023-11-28T18:28:19.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59a97fa640 1 -- 172.21.15.105:0/3477230756 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af165d46e0 con 0x55af165ce500 2023-11-28T18:28:19.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59a97fa640 1 -- 172.21.15.105:0/3477230756 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f59c4025560 con 0x55af165ce500 2023-11-28T18:28:19.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59d35dc000 10 client.7347 fetch_fsmap finished waiting for FSMap version 71 2023-11-28T18:28:19.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59d35dc000 10 client.7347 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:28:19.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59d35dc000 20 client.7347 populate_metadata read hostname 'smithi105' 2023-11-28T18:28:19.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59d35dc000 1 --2- 172.21.15.105:0/3477230756 >> [v2:172.21.15.105:6834/188173955,v1:172.21.15.105:6836/188173955] conn(0x55af166202a0 0x55af16620680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59d35dc000 4 client.7347 mds_command: new command op to 7194 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:28:19.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59d35dc000 1 -- 172.21.15.105:0/3477230756 --> [v2:172.21.15.105:6834/188173955,v1:172.21.15.105:6836/188173955] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55af1641f110 con 0x55af166202a0 2023-11-28T18:28:19.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.566+0000 7f59cd806640 1 --2- 172.21.15.105:0/3477230756 >> [v2:172.21.15.105:6834/188173955,v1:172.21.15.105:6836/188173955] conn(0x55af166202a0 0x55af16620680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59cd806640 1 --2- 172.21.15.105:0/3477230756 >> [v2:172.21.15.105:6834/188173955,v1:172.21.15.105:6836/188173955] conn(0x55af166202a0 0x55af16620680 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d61d60fd6d232de5 server_cookie=b1a84e40fab4a536 in_seq=0 out_seq=0 2023-11-28T18:28:19.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59a97fa640 10 client.7347 ms_handle_connect on v2:172.21.15.105:6834/188173955 2023-11-28T18:28:19.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59a97fa640 1 -- 172.21.15.105:0/3477230756 <== mds.0 v2:172.21.15.105:6834/188173955 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55af1641f110 con 0x55af166202a0 2023-11-28T18:28:19.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59a97fa640 10 client.7347 handle_command_reply: tid=0 2023-11-28T18:28:19.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d35dc000 1 client.7347 shutdown 2023-11-28T18:28:19.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d35dc000 1 -- 172.21.15.105:0/3477230756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af165ce500 msgr2=0x55af1667bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:19.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d35dc000 1 --2- 172.21.15.105:0/3477230756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af165ce500 0x55af1667bbf0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x55af163aa9c0 tx=0x7f59c4001cc0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d35dc000 1 -- 172.21.15.105:0/3477230756 shutdown_connections 2023-11-28T18:28:19.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d35dc000 1 --2- 172.21.15.105:0/3477230756 >> [v2:172.21.15.105:6834/188173955,v1:172.21.15.105:6836/188173955] conn(0x55af166202a0 0x55af16620680 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d35dc000 1 --2- 172.21.15.105:0/3477230756 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55af1661b650 0x55af1661fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d35dc000 1 --2- 172.21.15.105:0/3477230756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af1661ad30 0x55af1667c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d35dc000 1 --2- 172.21.15.105:0/3477230756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af165ce500 0x55af1667bbf0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59a97fa640 0 client.7347 ms_handle_reset on v2:172.21.15.105:6834/188173955 2023-11-28T18:28:19.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d35dc000 1 -- 172.21.15.105:0/3477230756 >> 172.21.15.105:0/3477230756 conn(0x55af165fb8c0 msgr2=0x55af165cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:19.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d35dc000 1 -- 172.21.15.105:0/3477230756 shutdown_connections 2023-11-28T18:28:19.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d35dc000 1 -- 172.21.15.105:0/3477230756 wait complete. 2023-11-28T18:28:19.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d35dc000 20 client.7347 trim_cache size 0 max 16384 2023-11-28T18:28:19.579 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:28:19.579 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:28:19.580 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2023-11-28T18:28:19.580 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 7194, 2023-11-28T18:28:19.580 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:28:19.580 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:28:19.580 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:28:19.580 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 5.8360569279999996, 2023-11-28T18:28:19.580 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 71, 2023-11-28T18:28:19.580 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 78, 2023-11-28T18:28:19.580 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 78, 2023-11-28T18:28:19.580 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 10.728631173 2023-11-28T18:28:19.580 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:28:19.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d026b640 1 -- 172.21.15.105:0/545894131 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59ac066900 msgr2=0x7f59ac068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:19.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d026b640 1 --2- 172.21.15.105:0/545894131 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59ac066900 0x7f59ac068dc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f59c4000f90 tx=0x7f59c4002020 comp rx=0 tx=0).stop 2023-11-28T18:28:19.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d026b640 1 -- 172.21.15.105:0/545894131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59c80c92f0 msgr2=0x7f59c8151210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:19.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d026b640 1 --2- 172.21.15.105:0/545894131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59c80c92f0 0x7f59c8151210 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f59c001c880 tx=0x7f59b4005800 comp rx=0 tx=0).stop 2023-11-28T18:28:19.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d026b640 1 -- 172.21.15.105:0/545894131 shutdown_connections 2023-11-28T18:28:19.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d026b640 1 --2- 172.21.15.105:0/545894131 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59ac066900 0x7f59ac068dc0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d026b640 1 --2- 172.21.15.105:0/545894131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59c80c92f0 0x7f59c8151210 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d026b640 1 --2- 172.21.15.105:0/545894131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59c80b4030 0x7f59c80ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d026b640 1 --2- 172.21.15.105:0/545894131 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59b4004d10 0x7f59c80b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d026b640 1 -- 172.21.15.105:0/545894131 >> 172.21.15.105:0/545894131 conn(0x7f59c80ceb90 msgr2=0x7f59c80cdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:19.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d026b640 1 -- 172.21.15.105:0/545894131 shutdown_connections 2023-11-28T18:28:19.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.570+0000 7f59d026b640 1 -- 172.21.15.105:0/545894131 wait complete. 2023-11-28T18:28:19.593 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T18:28:19.702 INFO:tasks.ceph.mds.a.smithi105.stderr:2023-11-28T18:28:19.698+0000 7f1548db0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T18:28:19.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.726+0000 7f35de3f8640 1 Processor -- start 2023-11-28T18:28:19.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.726+0000 7f35de3f8640 1 -- start start 2023-11-28T18:28:19.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.730+0000 7f35de3f8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f35d8159f40 0x7f35d80caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.730+0000 7f35de3f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f35d80cc760 0x7f35d80ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.730+0000 7f35de3f8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d806bf50 con 0x7f35d80ad680 2023-11-28T18:28:19.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.730+0000 7f35de3f8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d80ccbc0 con 0x7f35d8159f40 2023-11-28T18:28:19.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.730+0000 7f35de3f8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d815a320 con 0x7f35d80cc760 2023-11-28T18:28:19.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.730+0000 7f35d7fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f35d8159f40 0x7f35d80caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.730+0000 7f35d7fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f35d8159f40 0x7f35d80caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46516/0 (socket says 172.21.15.105:46516) 2023-11-28T18:28:19.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.730+0000 7f35d7fff640 1 -- 172.21.15.105:0/1776826648 learned_addr learned my addr 172.21.15.105:0/1776826648 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:19.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.730+0000 7f35d7fff640 1 -- 172.21.15.105:0/1776826648 >> v2:172.21.15.148:3300/0 conn(0x7f35d80cc760 msgr2=0x7f35d80ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:19.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.730+0000 7f35d7fff640 1 --2- 172.21.15.105:0/1776826648 >> v2:172.21.15.148:3300/0 conn(0x7f35d80cc760 0x7f35d80ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.730+0000 7f35d7fff640 1 -- 172.21.15.105:0/1776826648 >> v1:172.21.15.148:6789/0 conn(0x7f35d80ad680 legacy=0x7f35d815da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:19.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.730+0000 7f35d7fff640 1 -- 172.21.15.105:0/1776826648 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d8071380 con 0x7f35d8159f40 2023-11-28T18:28:19.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.734+0000 7f35d7fff640 1 --2- 172.21.15.105:0/1776826648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35d8159f40 0x7f35d80caf60 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f35d809d0d0 tx=0x7f35c801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b82853a3df0ca0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:19.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.734+0000 7f35d6ffd640 1 -- 172.21.15.105:0/1776826648 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c80ab600 con 0x7f35d8159f40 2023-11-28T18:28:19.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.734+0000 7f35d6ffd640 1 -- 172.21.15.105:0/1776826648 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c80ab760 con 0x7f35d8159f40 2023-11-28T18:28:19.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.734+0000 7f35d6ffd640 1 -- 172.21.15.105:0/1776826648 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c80abaf0 con 0x7f35d8159f40 2023-11-28T18:28:19.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.734+0000 7f35de3f8640 1 -- 172.21.15.105:0/1776826648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35d8159f40 msgr2=0x7f35d80caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:19.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.734+0000 7f35de3f8640 1 --2- 172.21.15.105:0/1776826648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35d8159f40 0x7f35d80caf60 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f35d809d0d0 tx=0x7f35c801c400 comp rx=0 tx=0).stop 2023-11-28T18:28:19.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.734+0000 7f35de3f8640 1 -- 172.21.15.105:0/1776826648 shutdown_connections 2023-11-28T18:28:19.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.734+0000 7f35de3f8640 1 --2- 172.21.15.105:0/1776826648 >> v2:172.21.15.148:3300/0 conn(0x7f35d80cc760 0x7f35d80ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.734+0000 7f35de3f8640 1 --2- 172.21.15.105:0/1776826648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35d8159f40 0x7f35d80caf60 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.734+0000 7f35de3f8640 1 -- 172.21.15.105:0/1776826648 >> 172.21.15.105:0/1776826648 conn(0x7f35d80ceb90 msgr2=0x7f35d80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:19.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.734+0000 7f35de3f8640 1 -- 172.21.15.105:0/1776826648 shutdown_connections 2023-11-28T18:28:19.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.734+0000 7f35de3f8640 1 -- 172.21.15.105:0/1776826648 wait complete. 2023-11-28T18:28:19.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.734+0000 7f35de3f8640 1 Processor -- start 2023-11-28T18:28:19.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.734+0000 7f35de3f8640 1 -- start start 2023-11-28T18:28:19.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35de3f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d80ad680 0x7f35d809c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35de3f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d80cc760 0x7f35d8153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35de3f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35d8159f40 0x7f35d81588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35de3f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d8154f10 con 0x7f35d8159f40 2023-11-28T18:28:19.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35de3f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d8155160 con 0x7f35d80cc760 2023-11-28T18:28:19.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35de3f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d8155440 con 0x7f35d80ad680 2023-11-28T18:28:19.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35dc995640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35d8159f40 0x7f35d81588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35dc995640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35d8159f40 0x7f35d81588d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46520/0 (socket says 172.21.15.105:46520) 2023-11-28T18:28:19.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35dc995640 1 -- 172.21.15.105:0/1180579912 learned_addr learned my addr 172.21.15.105:0/1180579912 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:19.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35d7fff640 1 --2- 172.21.15.105:0/1180579912 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d80ad680 0x7f35d809c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35d77fe640 1 --2- 172.21.15.105:0/1180579912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d80cc760 0x7f35d8153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35dc995640 1 -- 172.21.15.105:0/1180579912 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d80ad680 msgr2=0x7f35d809c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:19.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35dc995640 1 --2- 172.21.15.105:0/1180579912 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d80ad680 0x7f35d809c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35dc995640 1 -- 172.21.15.105:0/1180579912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d80cc760 msgr2=0x7f35d8153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:19.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35dc995640 1 --2- 172.21.15.105:0/1180579912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d80cc760 0x7f35d8153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35dc995640 1 -- 172.21.15.105:0/1180579912 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c80094d0 con 0x7f35d8159f40 2023-11-28T18:28:19.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35dc995640 1 --2- 172.21.15.105:0/1180579912 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35d8159f40 0x7f35d81588d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f35d000b990 tx=0x7f35d000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:19.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35d4ff9640 1 -- 172.21.15.105:0/1180579912 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d000cba0 con 0x7f35d8159f40 2023-11-28T18:28:19.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35d4ff9640 1 -- 172.21.15.105:0/1180579912 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d000cd00 con 0x7f35d8159f40 2023-11-28T18:28:19.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35de3f8640 1 -- 172.21.15.105:0/1180579912 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d8155720 con 0x7f35d8159f40 2023-11-28T18:28:19.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35d4ff9640 1 -- 172.21.15.105:0/1180579912 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0014680 con 0x7f35d8159f40 2023-11-28T18:28:19.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35de3f8640 1 -- 172.21.15.105:0/1180579912 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d8155ce0 con 0x7f35d8159f40 2023-11-28T18:28:19.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35de3f8640 1 -- 172.21.15.105:0/1180579912 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d8155ed0 con 0x7f35d8159f40 2023-11-28T18:28:19.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.738+0000 7f35d4ff9640 1 -- 172.21.15.105:0/1180579912 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f35d001d3b0 con 0x7f35d8159f40 2023-11-28T18:28:19.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.742+0000 7f35d4ff9640 1 --2- 172.21.15.105:0/1180579912 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f35c40668d0 0x7f35c4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:19.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.742+0000 7f35d4ff9640 1 -- 172.21.15.105:0/1180579912 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f35d00843d0 con 0x7f35d8159f40 2023-11-28T18:28:19.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.742+0000 7f35d7fff640 1 --2- 172.21.15.105:0/1180579912 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f35c40668d0 0x7f35c4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:19.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.742+0000 7f35d7fff640 1 --2- 172.21.15.105:0/1180579912 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f35c40668d0 0x7f35c4068d90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f35c80af000 tx=0x7f35c80b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:19.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.746+0000 7f35d4ff9640 1 -- 172.21.15.105:0/1180579912 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f35d00515f0 con 0x7f35d8159f40 2023-11-28T18:28:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.926+0000 7f35de3f8640 1 -- 172.21.15.105:0/1180579912 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f35d8067570 con 0x7f35d8159f40 2023-11-28T18:28:19.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.926+0000 7f35d4ff9640 1 -- 172.21.15.105:0/1180579912 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) v1 ==== 53+0+73 (secure 0 0 0) 0x7f35d0019060 con 0x7f35d8159f40 2023-11-28T18:28:19.932 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T18:28:19.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.930+0000 7f35de3f8640 1 -- 172.21.15.105:0/1180579912 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f35c40668d0 msgr2=0x7f35c4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:19.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.930+0000 7f35de3f8640 1 --2- 172.21.15.105:0/1180579912 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f35c40668d0 0x7f35c4068d90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f35c80af000 tx=0x7f35c80b6040 comp rx=0 tx=0).stop 2023-11-28T18:28:19.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.930+0000 7f35de3f8640 1 -- 172.21.15.105:0/1180579912 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35d8159f40 msgr2=0x7f35d81588d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:19.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.930+0000 7f35de3f8640 1 --2- 172.21.15.105:0/1180579912 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35d8159f40 0x7f35d81588d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f35d000b990 tx=0x7f35d000be50 comp rx=0 tx=0).stop 2023-11-28T18:28:19.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.930+0000 7f35de3f8640 1 -- 172.21.15.105:0/1180579912 shutdown_connections 2023-11-28T18:28:19.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.930+0000 7f35de3f8640 1 --2- 172.21.15.105:0/1180579912 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f35c40668d0 0x7f35c4068d90 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.930+0000 7f35de3f8640 1 --2- 172.21.15.105:0/1180579912 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35d8159f40 0x7f35d81588d0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.930+0000 7f35de3f8640 1 --2- 172.21.15.105:0/1180579912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d80cc760 0x7f35d8153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.930+0000 7f35de3f8640 1 --2- 172.21.15.105:0/1180579912 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35d80ad680 0x7f35d809c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:19.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.930+0000 7f35de3f8640 1 -- 172.21.15.105:0/1180579912 >> 172.21.15.105:0/1180579912 conn(0x7f35d80ceb90 msgr2=0x7f35d806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:19.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.930+0000 7f35de3f8640 1 -- 172.21.15.105:0/1180579912 shutdown_connections 2023-11-28T18:28:19.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:19.930+0000 7f35de3f8640 1 -- 172.21.15.105:0/1180579912 wait complete. 2023-11-28T18:28:19.953 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T18:28:19.953 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T18:28:19.953 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2023-11-28T18:28:19.954 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T18:28:19.954 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T18:28:19.954 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T18:28:19.954 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T18:28:19.954 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T18:28:19.955 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:28:19.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:28:19.956 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2023-11-28T18:28:19.966 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T18:28:19.966 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T18:28:19.966 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T18:28:19.967 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:28:19.967 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-11-28T18:28:19.967 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:28:19.967 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:28:19.967 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:28:19.967 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2023-11-28T18:28:19.967 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:28:19.967 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:28:19.967 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2023-11-28T18:28:19.967 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:28:19.967 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:28:19.967 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T18:28:19.968 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:28:19.968 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-28T18:28:19.968 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42122sec preferred_lft 42122sec 2023-11-28T18:28:19.968 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2023-11-28T18:28:19.968 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:28:19.968 INFO:teuthology.orchestra.run.smithi105.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-28T18:28:19.968 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 02:42:cc:d4:87:d2 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:28:19.968 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-28T18:28:19.968 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:28:19.968 INFO:teuthology.orchestra.run.smithi105.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T18:28:19.968 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-28T18:28:19.968 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T18:28:19.969 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:28:19.969 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-28T18:28:19.969 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:28:19.969 INFO:teuthology.orchestra.run.smithi105.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T18:28:19.969 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:28:19.969 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-28T18:28:19.969 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:28:19.969 INFO:teuthology.orchestra.run.smithi105.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T18:28:19.969 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:28:19.969 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-28T18:28:19.969 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:28:19.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:28:19.970 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T18:28:20.031 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-11-28T18:28:20.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:28:20.033 DEBUG:teuthology.orchestra.run.smithi105:> route 2023-11-28T18:28:20.091 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2023-11-28T18:28:20.092 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T18:28:20.092 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-28T18:28:20.092 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T18:28:20.092 INFO:teuthology.orchestra.run.smithi105.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-28T18:28:20.092 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T18:28:20.092 INFO:teuthology.orchestra.run.smithi105.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:28:20.092 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:28:20.092 INFO:teuthology.orchestra.run.smithi105.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:28:20.092 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T18:28:20.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:28:20.093 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-11-28T18:28:20.093 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:28:20.093 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T18:28:20.093 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T18:28:20.093 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T18:28:20.093 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:28:20.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:28:20.184 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-11-28T18:28:20.194 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T18:28:20.194 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T18:28:20.194 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:28:20.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:28:20.195 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2023-11-28T18:28:20.256 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-28T18:28:20.257 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:28:20.257 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T18:28:20.313 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:28:20.313 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:28:20.313 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T18:28:20.364 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:28:20.431 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:28:20.486 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:28:20.487 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:28:20.487 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:28:20.487 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:28:20.487 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:28:20.487 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:28:20.487 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:28:20.487 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:28:20.487 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:28:20.487 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:28:20.487 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:28:20.488 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:28:20.488 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:28:20.488 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:28:20.488 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:28:20.488 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:28:20.488 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:28:20.488 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:28:20.488 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:28:20.488 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:28:20.488 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:28:20.488 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:28:20.488 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:28:20.489 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:28:20.489 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:28:20.489 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:28:20.489 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:28:20.489 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:28:20.489 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:28:20.489 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:28:20.489 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:28:20.489 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:28:20.489 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:28:20.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:28:20.491 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:28:20.540 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T18:28:20.540 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-28T18:28:20.583 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:28:20.605 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:28:20.658 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:28:20.659 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:28:20.659 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:28:20.659 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:28:20.659 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:28:20.659 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:28:20.660 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:28:20.660 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:28:20.660 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:28:20.660 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:28:20.660 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:28:20.661 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:28:20.661 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:28:20.661 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:28:20.661 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:28:20.661 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:28:20.662 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:28:20.662 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:28:20.662 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:28:20.662 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:28:20.662 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:28:20.663 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:28:20.663 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:28:20.663 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:28:20.663 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:28:20.663 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:28:20.663 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:28:20.663 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:28:20.663 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:28:20.663 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:28:20.664 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:28:20.664 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:28:20.664 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:28:20.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:28:20.666 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:28:20.678 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:2023-11-28T18:28:20.674+0000 7f3201e94580 -1 init, newargv = 0x55bb104c3e60 newargc=15 2023-11-28T18:28:20.679 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[58072]: starting ceph client 2023-11-28T18:28:20.685 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[58072]: starting fuse 2023-11-28T18:28:20.687 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-11-28T18:28:20.688 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-28T18:28:20.688 DEBUG:teuthology.orchestra.run.smithi105:> sudo stdin-killer -- python3 -c ' 2023-11-28T18:28:20.688 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2023-11-28T18:28:20.688 DEBUG:teuthology.orchestra.run.smithi105:> import re 2023-11-28T18:28:20.688 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-11-28T18:28:20.688 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2023-11-28T18:28:20.688 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:28:20.688 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2023-11-28T18:28:20.688 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-28T18:28:20.689 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2023-11-28T18:28:20.689 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-28T18:28:20.689 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:28:20.689 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2023-11-28T18:28:20.689 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2023-11-28T18:28:20.689 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2023-11-28T18:28:20.689 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2023-11-28T18:28:20.689 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:28:20.689 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2023-11-28T18:28:20.689 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T18:28:20.689 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T18:28:20.689 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T18:28:20.689 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2023-11-28T18:28:20.690 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2023-11-28T18:28:20.690 DEBUG:teuthology.orchestra.run.smithi105:> return f 2023-11-28T18:28:20.690 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T18:28:20.690 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:28:20.690 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.0")) 2023-11-28T18:28:20.690 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-11-28T18:28:20.815 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.0.58072.asok 2023-11-28T18:28:20.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-28T18:28:20.827 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.58072.asok 2023-11-28T18:28:20.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:28:20.827 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.58072.asok status 2023-11-28T18:28:21.001 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:28:21.002 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2023-11-28T18:28:21.002 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "47c1c547372ed5bff7984647dc200bac0e019caf", 2023-11-28T18:28:21.002 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev)", 2023-11-28T18:28:21.002 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "0", 2023-11-28T18:28:21.002 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2023-11-28T18:28:21.002 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-28T18:28:21.002 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "58072", 2023-11-28T18:28:21.002 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2023-11-28T18:28:21.002 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:28:21.003 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2023-11-28T18:28:21.003 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2023-11-28T18:28:21.003 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 7362, 2023-11-28T18:28:21.003 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2023-11-28T18:28:21.003 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2023-11-28T18:28:21.003 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2023-11-28T18:28:21.003 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 7362 2023-11-28T18:28:21.003 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:28:21.003 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-11-28T18:28:21.003 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-11-28T18:28:21.003 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-11-28T18:28:21.003 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 109544456 2023-11-28T18:28:21.004 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:28:21.004 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:28:21.004 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-11-28T18:28:21.004 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-11-28T18:28:21.004 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-11-28T18:28:21.004 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 109544456 2023-11-28T18:28:21.004 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:28:21.004 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.7362 192.168.0.1:0/109544456", 2023-11-28T18:28:21.004 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/109544456", 2023-11-28T18:28:21.004 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2023-11-28T18:28:21.004 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 71, 2023-11-28T18:28:21.004 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 78, 2023-11-28T18:28:21.005 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2023-11-28T18:28:21.005 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2023-11-28T18:28:21.005 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2023-11-28T18:28:21.005 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:28:21.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:28:21.016 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2023-11-28T18:28:21.016 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-28T18:28:21.025 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2023-11-28T18:28:21.026 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-28T18:28:21.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:28:21.026 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T18:28:21.089 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random True 2023-11-28T18:28:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.266+0000 7f5fd4d72640 1 Processor -- start 2023-11-28T18:28:21.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.266+0000 7f5fd4d72640 1 -- start start 2023-11-28T18:28:21.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fd4d72640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5fd006cef0 0x7f5fd00abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:21.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fd4d72640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd00cde30 con 0x7f5fd0159610 2023-11-28T18:28:21.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fd4d72640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd01599f0 con 0x7f5fd00ce2c0 2023-11-28T18:28:21.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fd4d72640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd006de20 con 0x7f5fd006cef0 2023-11-28T18:28:21.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fce59c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5fd0159610 0x7f5fd00cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39250/0 (socket says 172.21.15.105:39250) 2023-11-28T18:28:21.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fce59c640 1 -- 172.21.15.105:0/366161430 learned_addr learned my addr 172.21.15.105:0/366161430 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:21.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fced9d640 1 --2- 172.21.15.105:0/366161430 >> v2:172.21.15.148:3300/0 conn(0x7f5fd006cef0 0x7f5fd00abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:21.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fcdd9b640 1 -- 172.21.15.105:0/366161430 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401796178 0 0) 0x7f5fd00cde30 con 0x7f5fd0159610 2023-11-28T18:28:21.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fcdd9b640 1 -- 172.21.15.105:0/366161430 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fbc003530 con 0x7f5fd0159610 2023-11-28T18:28:21.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fcdd9b640 1 -- 172.21.15.105:0/366161430 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 290692477 0 0) 0x7f5fbc003530 con 0x7f5fd0159610 2023-11-28T18:28:21.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fcdd9b640 1 -- 172.21.15.105:0/366161430 >> v2:172.21.15.148:3300/0 conn(0x7f5fd006cef0 msgr2=0x7f5fd00abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:21.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fcdd9b640 1 --2- 172.21.15.105:0/366161430 >> v2:172.21.15.148:3300/0 conn(0x7f5fd006cef0 0x7f5fd00abd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fcdd9b640 1 -- 172.21.15.105:0/366161430 >> v1:172.21.15.148:6789/0 conn(0x7f5fd00ce2c0 legacy=0x7f5fd006c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:21.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fcdd9b640 1 -- 172.21.15.105:0/366161430 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd00cd220 con 0x7f5fd0159610 2023-11-28T18:28:21.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fcdd9b640 1 -- 172.21.15.105:0/366161430 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f5fc0001a00 con 0x7f5fd0159610 2023-11-28T18:28:21.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fcdd9b640 1 -- 172.21.15.105:0/366161430 >> v1:172.21.15.105:6789/0 conn(0x7f5fd0159610 legacy=0x7f5fd00cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:21.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fcdd9b640 1 --2- 172.21.15.105:0/366161430 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fbc003980 0x7f5fbc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:21.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fcdd9b640 1 --2- 172.21.15.105:0/366161430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fbc0047d0 0x7f5fbc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:21.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fcdd9b640 1 --2- 172.21.15.105:0/366161430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fbc005190 0x7f5fbc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:21.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fcdd9b640 1 -- 172.21.15.105:0/366161430 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd006de20 con 0x7f5fbc0047d0 2023-11-28T18:28:21.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fcdd9b640 1 -- 172.21.15.105:0/366161430 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc00c030 con 0x7f5fbc005190 2023-11-28T18:28:21.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fcdd9b640 1 -- 172.21.15.105:0/366161430 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc00c1f0 con 0x7f5fbc003980 2023-11-28T18:28:21.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fc5d9b640 1 --2- 172.21.15.105:0/366161430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fbc0047d0 0x7f5fbc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:21.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fced9d640 1 --2- 172.21.15.105:0/366161430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fbc005190 0x7f5fbc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:21.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fce59c640 1 --2- 172.21.15.105:0/366161430 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fbc003980 0x7f5fbc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:21.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fc5d9b640 1 -- 172.21.15.105:0/366161430 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fbc003980 msgr2=0x7f5fbc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:21.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fc5d9b640 1 --2- 172.21.15.105:0/366161430 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fbc003980 0x7f5fbc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fc5d9b640 1 -- 172.21.15.105:0/366161430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fbc005190 msgr2=0x7f5fbc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:21.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fc5d9b640 1 --2- 172.21.15.105:0/366161430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fbc005190 0x7f5fbc00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fc5d9b640 1 -- 172.21.15.105:0/366161430 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fbc003530 con 0x7f5fbc0047d0 2023-11-28T18:28:21.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.270+0000 7f5fced9d640 1 -- 172.21.15.105:0/366161430 reap_dead start 2023-11-28T18:28:21.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.274+0000 7f5fc5d9b640 1 --2- 172.21.15.105:0/366161430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fbc0047d0 0x7f5fbc004c50 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8001780 tx=0x7f5fb801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bfe03a5b388b8c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:21.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.274+0000 7f5fcdd9b640 1 -- 172.21.15.105:0/366161430 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb80a4ea0 con 0x7f5fbc0047d0 2023-11-28T18:28:21.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.274+0000 7f5fcdd9b640 1 -- 172.21.15.105:0/366161430 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb80a7050 con 0x7f5fbc0047d0 2023-11-28T18:28:21.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.274+0000 7f5fcdd9b640 1 -- 172.21.15.105:0/366161430 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8003600 con 0x7f5fbc0047d0 2023-11-28T18:28:21.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.274+0000 7f5fd4d72640 1 -- 172.21.15.105:0/366161430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fbc0047d0 msgr2=0x7f5fbc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:21.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.274+0000 7f5fd4d72640 1 --2- 172.21.15.105:0/366161430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fbc0047d0 0x7f5fbc004c50 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8001780 tx=0x7f5fb801c3e0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.274+0000 7f5fd4d72640 1 -- 172.21.15.105:0/366161430 shutdown_connections 2023-11-28T18:28:21.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.274+0000 7f5fd4d72640 1 --2- 172.21.15.105:0/366161430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fbc0047d0 0x7f5fbc004c50 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.274+0000 7f5fd4d72640 1 -- 172.21.15.105:0/366161430 >> 172.21.15.105:0/366161430 conn(0x7f5fd00ceb90 msgr2=0x7f5fd00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:21.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.274+0000 7f5fd4d72640 1 -- 172.21.15.105:0/366161430 shutdown_connections 2023-11-28T18:28:21.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.274+0000 7f5fd4d72640 1 -- 172.21.15.105:0/366161430 wait complete. 2023-11-28T18:28:21.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fd4d72640 1 Processor -- start 2023-11-28T18:28:21.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fd4d72640 1 -- start start 2023-11-28T18:28:21.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fd4d72640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fbc0047d0 0x7f5fd01f6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:21.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fd4d72640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fd015a500 0x7f5fd01f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:21.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fd4d72640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fd01f5a70 0x7f5fd015bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:21.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fd4d72640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd01560a0 con 0x7f5fbc0047d0 2023-11-28T18:28:21.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fd4d72640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd015aa30 con 0x7f5fd015a500 2023-11-28T18:28:21.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fd4d72640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd00c9730 con 0x7f5fd01f5a70 2023-11-28T18:28:21.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fce59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fbc0047d0 0x7f5fd01f6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:21.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fce59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fbc0047d0 0x7f5fd01f6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47528/0 (socket says 172.21.15.105:47528) 2023-11-28T18:28:21.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fce59c640 1 -- 172.21.15.105:0/3821089521 learned_addr learned my addr 172.21.15.105:0/3821089521 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:21.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fc5d9b640 1 --2- 172.21.15.105:0/3821089521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fd015a500 0x7f5fd01f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:21.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fced9d640 1 --2- 172.21.15.105:0/3821089521 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fd01f5a70 0x7f5fd015bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:21.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fce59c640 1 -- 172.21.15.105:0/3821089521 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fd01f5a70 msgr2=0x7f5fd015bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:21.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fce59c640 1 --2- 172.21.15.105:0/3821089521 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fd01f5a70 0x7f5fd015bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fce59c640 1 -- 172.21.15.105:0/3821089521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fd015a500 msgr2=0x7f5fd01f3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:21.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fce59c640 1 --2- 172.21.15.105:0/3821089521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fd015a500 0x7f5fd01f3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fce59c640 1 -- 172.21.15.105:0/3821089521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd01f7240 con 0x7f5fbc0047d0 2023-11-28T18:28:21.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fced9d640 1 --2- 172.21.15.105:0/3821089521 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fd01f5a70 0x7f5fd015bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:21.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fce59c640 1 --2- 172.21.15.105:0/3821089521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fbc0047d0 0x7f5fd01f6d00 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00118a0 tx=0x7f5fc0011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:21.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fc77fe640 1 -- 172.21.15.105:0/3821089521 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc000fb00 con 0x7f5fbc0047d0 2023-11-28T18:28:21.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fc77fe640 1 -- 172.21.15.105:0/3821089521 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc000fc80 con 0x7f5fbc0047d0 2023-11-28T18:28:21.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fd4d72640 1 -- 172.21.15.105:0/3821089521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd00c8e00 con 0x7f5fbc0047d0 2023-11-28T18:28:21.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fc77fe640 1 -- 172.21.15.105:0/3821089521 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0015410 con 0x7f5fbc0047d0 2023-11-28T18:28:21.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fd4d72640 1 -- 172.21.15.105:0/3821089521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd00c93c0 con 0x7f5fbc0047d0 2023-11-28T18:28:21.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.278+0000 7f5fd4d72640 1 -- 172.21.15.105:0/3821089521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd00ae5e0 con 0x7f5fbc0047d0 2023-11-28T18:28:21.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.282+0000 7f5fc77fe640 1 -- 172.21.15.105:0/3821089521 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5fc0015590 con 0x7f5fbc0047d0 2023-11-28T18:28:21.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.282+0000 7f5fc77fe640 1 --2- 172.21.15.105:0/3821089521 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5fb00668d0 0x7f5fb0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:21.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.282+0000 7f5fc77fe640 1 -- 172.21.15.105:0/3821089521 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5fc00876f0 con 0x7f5fbc0047d0 2023-11-28T18:28:21.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.282+0000 7f5fc5d9b640 1 --2- 172.21.15.105:0/3821089521 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5fb00668d0 0x7f5fb0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:21.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.282+0000 7f5fc5d9b640 1 --2- 172.21.15.105:0/3821089521 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5fb00668d0 0x7f5fb0068d90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8001750 tx=0x7f5fb80011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:21.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.286+0000 7f5fc77fe640 1 -- 172.21.15.105:0/3821089521 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5fc00547e0 con 0x7f5fbc0047d0 2023-11-28T18:28:21.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.438+0000 7f5fd4d72640 1 -- 172.21.15.105:0/3821089521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random}] v 0) v1 -- 0x7f5fd0067570 con 0x7f5fbc0047d0 2023-11-28T18:28:21.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.446+0000 7f5fc77fe640 1 -- 172.21.15.105:0/3821089521 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random}]=0 v36)=0 v36) v1 ==== 128+0+0 (secure 0 0 0) 0x7f5fc0085e60 con 0x7f5fbc0047d0 2023-11-28T18:28:21.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.446+0000 7f5fd4d72640 1 -- 172.21.15.105:0/3821089521 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5fb00668d0 msgr2=0x7f5fb0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:21.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.446+0000 7f5fd4d72640 1 --2- 172.21.15.105:0/3821089521 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5fb00668d0 0x7f5fb0068d90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8001750 tx=0x7f5fb80011a0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.446+0000 7f5fd4d72640 1 -- 172.21.15.105:0/3821089521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fbc0047d0 msgr2=0x7f5fd01f6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:21.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.446+0000 7f5fd4d72640 1 --2- 172.21.15.105:0/3821089521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fbc0047d0 0x7f5fd01f6d00 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00118a0 tx=0x7f5fc0011d60 comp rx=0 tx=0).stop 2023-11-28T18:28:21.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.446+0000 7f5fd4d72640 1 -- 172.21.15.105:0/3821089521 shutdown_connections 2023-11-28T18:28:21.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.446+0000 7f5fd4d72640 1 --2- 172.21.15.105:0/3821089521 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fd01f5a70 0x7f5fd015bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.446+0000 7f5fd4d72640 1 --2- 172.21.15.105:0/3821089521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fd015a500 0x7f5fd01f3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.446+0000 7f5fd4d72640 1 --2- 172.21.15.105:0/3821089521 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5fb00668d0 0x7f5fb0068d90 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.446+0000 7f5fd4d72640 1 --2- 172.21.15.105:0/3821089521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fbc0047d0 0x7f5fd01f6d00 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.446+0000 7f5fd4d72640 1 -- 172.21.15.105:0/3821089521 >> 172.21.15.105:0/3821089521 conn(0x7f5fd00ceb90 msgr2=0x7f5fd00aded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:21.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.446+0000 7f5fd4d72640 1 -- 172.21.15.105:0/3821089521 shutdown_connections 2023-11-28T18:28:21.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.446+0000 7f5fd4d72640 1 -- 172.21.15.105:0/3821089521 wait complete. 2023-11-28T18:28:21.470 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_distributed True 2023-11-28T18:28:21.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.606+0000 7fb8bb4bb640 1 Processor -- start 2023-11-28T18:28:21.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.606+0000 7fb8bb4bb640 1 -- start start 2023-11-28T18:28:21.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.606+0000 7fb8bb4bb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb8b40ad9f0 0x7fb8b40ade80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:21.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.606+0000 7fb8bb4bb640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b4162820 con 0x7fb8b406d610 2023-11-28T18:28:21.613 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.606+0000 7fb8bb4bb640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b4161c10 con 0x7fb8b40cc7d0 2023-11-28T18:28:21.613 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.606+0000 7fb8bb4bb640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b40ae3c0 con 0x7fb8b40ad9f0 2023-11-28T18:28:21.613 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.606+0000 7fb8b9257640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb8b406d610 0x7fb8b40cdff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39258/0 (socket says 172.21.15.105:39258) 2023-11-28T18:28:21.613 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.606+0000 7fb8b9257640 1 -- 172.21.15.105:0/3810891238 learned_addr learned my addr 172.21.15.105:0/3810891238 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:21.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.606+0000 7fb8b8a56640 1 --2- 172.21.15.105:0/3810891238 >> v2:172.21.15.148:3300/0 conn(0x7fb8b40ad9f0 0x7fb8b40ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:21.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.606+0000 7fb8abfff640 1 -- 172.21.15.105:0/3810891238 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2286145158 0 0) 0x7fb8b4162820 con 0x7fb8b406d610 2023-11-28T18:28:21.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.606+0000 7fb8abfff640 1 -- 172.21.15.105:0/3810891238 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8a0003530 con 0x7fb8b406d610 2023-11-28T18:28:21.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8abfff640 1 -- 172.21.15.105:0/3810891238 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 948734225 0 0) 0x7fb8a0003530 con 0x7fb8b406d610 2023-11-28T18:28:21.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8abfff640 1 -- 172.21.15.105:0/3810891238 >> v2:172.21.15.148:3300/0 conn(0x7fb8b40ad9f0 msgr2=0x7fb8b40ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:21.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8abfff640 1 --2- 172.21.15.105:0/3810891238 >> v2:172.21.15.148:3300/0 conn(0x7fb8b40ad9f0 0x7fb8b40ade80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8abfff640 1 -- 172.21.15.105:0/3810891238 >> v1:172.21.15.148:6789/0 conn(0x7fb8b40cc7d0 legacy=0x7fb8b40ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:21.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8abfff640 1 -- 172.21.15.105:0/3810891238 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b4161ef0 con 0x7fb8b406d610 2023-11-28T18:28:21.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8abfff640 1 -- 172.21.15.105:0/3810891238 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fb8b0001680 con 0x7fb8b406d610 2023-11-28T18:28:21.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8abfff640 1 -- 172.21.15.105:0/3810891238 >> v1:172.21.15.105:6789/0 conn(0x7fb8b406d610 legacy=0x7fb8b40cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:21.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8abfff640 1 --2- 172.21.15.105:0/3810891238 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8a0003980 0x7fb8a0003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:21.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8abfff640 1 --2- 172.21.15.105:0/3810891238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8a0004380 0x7fb8a0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:21.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8abfff640 1 --2- 172.21.15.105:0/3810891238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8a0005050 0x7fb8a000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:21.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8abfff640 1 -- 172.21.15.105:0/3810891238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b40ae3c0 con 0x7fb8a0005050 2023-11-28T18:28:21.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8abfff640 1 -- 172.21.15.105:0/3810891238 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a000bfb0 con 0x7fb8a0004380 2023-11-28T18:28:21.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8abfff640 1 -- 172.21.15.105:0/3810891238 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a000c170 con 0x7fb8a0003980 2023-11-28T18:28:21.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8b9a58640 1 --2- 172.21.15.105:0/3810891238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8a0005050 0x7fb8a000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:21.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8b9257640 1 --2- 172.21.15.105:0/3810891238 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8a0003980 0x7fb8a0003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:21.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8b8a56640 1 --2- 172.21.15.105:0/3810891238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8a0004380 0x7fb8a0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:21.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8b9a58640 1 -- 172.21.15.105:0/3810891238 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8a0003980 msgr2=0x7fb8a0003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:21.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8b9a58640 1 --2- 172.21.15.105:0/3810891238 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8a0003980 0x7fb8a0003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8b9a58640 1 -- 172.21.15.105:0/3810891238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8a0004380 msgr2=0x7fb8a0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:21.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8b9a58640 1 --2- 172.21.15.105:0/3810891238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8a0004380 0x7fb8a0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8b9a58640 1 -- 172.21.15.105:0/3810891238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a0003530 con 0x7fb8a0005050 2023-11-28T18:28:21.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8b8a56640 1 --2- 172.21.15.105:0/3810891238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8a0004380 0x7fb8a0004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:21.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.610+0000 7fb8b9a58640 1 -- 172.21.15.105:0/3810891238 reap_dead start 2023-11-28T18:28:21.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.614+0000 7fb8b9a58640 1 --2- 172.21.15.105:0/3810891238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8a0005050 0x7fb8a000b990 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac001f60 tx=0x7fb8ac01c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d3b8bda6bcb751 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:21.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.614+0000 7fb8abfff640 1 -- 172.21.15.105:0/3810891238 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac0a6680 con 0x7fb8a0005050 2023-11-28T18:28:21.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.614+0000 7fb8abfff640 1 -- 172.21.15.105:0/3810891238 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ac0a67e0 con 0x7fb8a0005050 2023-11-28T18:28:21.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.614+0000 7fb8abfff640 1 -- 172.21.15.105:0/3810891238 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac0a6b70 con 0x7fb8a0005050 2023-11-28T18:28:21.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.614+0000 7fb8bb4bb640 1 -- 172.21.15.105:0/3810891238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8a0005050 msgr2=0x7fb8a000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:21.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.614+0000 7fb8bb4bb640 1 --2- 172.21.15.105:0/3810891238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8a0005050 0x7fb8a000b990 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac001f60 tx=0x7fb8ac01c860 comp rx=0 tx=0).stop 2023-11-28T18:28:21.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.614+0000 7fb8bb4bb640 1 -- 172.21.15.105:0/3810891238 shutdown_connections 2023-11-28T18:28:21.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.614+0000 7fb8bb4bb640 1 --2- 172.21.15.105:0/3810891238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8a0005050 0x7fb8a000b990 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.614+0000 7fb8bb4bb640 1 -- 172.21.15.105:0/3810891238 >> 172.21.15.105:0/3810891238 conn(0x7fb8b40ceb90 msgr2=0x7fb8b40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:21.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.614+0000 7fb8bb4bb640 1 -- 172.21.15.105:0/3810891238 shutdown_connections 2023-11-28T18:28:21.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.614+0000 7fb8bb4bb640 1 -- 172.21.15.105:0/3810891238 wait complete. 2023-11-28T18:28:21.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.614+0000 7fb8bb4bb640 1 Processor -- start 2023-11-28T18:28:21.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.614+0000 7fb8bb4bb640 1 -- start start 2023-11-28T18:28:21.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.614+0000 7fb8bb4bb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8a0005050 0x7fb8b40cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:21.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.614+0000 7fb8bb4bb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8b414b2c0 0x7fb8b414db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:21.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.614+0000 7fb8bb4bb640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8b415dc20 0x7fb8b415e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:21.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.614+0000 7fb8bb4bb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b415e5e0 con 0x7fb8b414b2c0 2023-11-28T18:28:21.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.614+0000 7fb8bb4bb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b409b090 con 0x7fb8a0005050 2023-11-28T18:28:21.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.614+0000 7fb8bb4bb640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b414e0c0 con 0x7fb8b415dc20 2023-11-28T18:28:21.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.614+0000 7fb8b8a56640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8b414b2c0 0x7fb8b414db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:21.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.614+0000 7fb8b8a56640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8b414b2c0 0x7fb8b414db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47548/0 (socket says 172.21.15.105:47548) 2023-11-28T18:28:21.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.614+0000 7fb8b8a56640 1 -- 172.21.15.105:0/3542347085 learned_addr learned my addr 172.21.15.105:0/3542347085 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:21.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.618+0000 7fb8b9257640 1 --2- 172.21.15.105:0/3542347085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8a0005050 0x7fb8b40cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:21.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.618+0000 7fb8b9a58640 1 --2- 172.21.15.105:0/3542347085 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8b415dc20 0x7fb8b415e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:21.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.618+0000 7fb8b8a56640 1 -- 172.21.15.105:0/3542347085 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8b415dc20 msgr2=0x7fb8b415e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:21.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.618+0000 7fb8b8a56640 1 --2- 172.21.15.105:0/3542347085 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8b415dc20 0x7fb8b415e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.618+0000 7fb8b8a56640 1 -- 172.21.15.105:0/3542347085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8a0005050 msgr2=0x7fb8b40cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:21.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.618+0000 7fb8b8a56640 1 --2- 172.21.15.105:0/3542347085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8a0005050 0x7fb8b40cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.618+0000 7fb8b8a56640 1 -- 172.21.15.105:0/3542347085 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b414e390 con 0x7fb8b414b2c0 2023-11-28T18:28:21.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.618+0000 7fb8b8a56640 1 --2- 172.21.15.105:0/3542347085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8b414b2c0 0x7fb8b414db80 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009fd0 tx=0x7fb8a40119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:21.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.618+0000 7fb8a9ffb640 1 -- 172.21.15.105:0/3542347085 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4004220 con 0x7fb8b414b2c0 2023-11-28T18:28:21.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.618+0000 7fb8a9ffb640 1 -- 172.21.15.105:0/3542347085 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a4004380 con 0x7fb8b414b2c0 2023-11-28T18:28:21.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.618+0000 7fb8bb4bb640 1 -- 172.21.15.105:0/3542347085 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b414e670 con 0x7fb8b414b2c0 2023-11-28T18:28:21.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.618+0000 7fb8a9ffb640 1 -- 172.21.15.105:0/3542347085 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4015890 con 0x7fb8b414b2c0 2023-11-28T18:28:21.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.618+0000 7fb8bb4bb640 1 -- 172.21.15.105:0/3542347085 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b409b2e0 con 0x7fb8b414b2c0 2023-11-28T18:28:21.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.618+0000 7fb8bb4bb640 1 -- 172.21.15.105:0/3542347085 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b4067570 con 0x7fb8b414b2c0 2023-11-28T18:28:21.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.618+0000 7fb8a9ffb640 1 -- 172.21.15.105:0/3542347085 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fb8a40159f0 con 0x7fb8b414b2c0 2023-11-28T18:28:21.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.622+0000 7fb8a9ffb640 1 --2- 172.21.15.105:0/3542347085 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb8980668d0 0x7fb898068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:21.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.622+0000 7fb8a9ffb640 1 -- 172.21.15.105:0/3542347085 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb8a4084a00 con 0x7fb8b414b2c0 2023-11-28T18:28:21.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.622+0000 7fb8b9257640 1 --2- 172.21.15.105:0/3542347085 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb8980668d0 0x7fb898068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:21.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.622+0000 7fb8b9257640 1 --2- 172.21.15.105:0/3542347085 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb8980668d0 0x7fb898068d90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0000c10 tx=0x7fb8b0001440 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:21.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.626+0000 7fb8a9ffb640 1 -- 172.21.15.105:0/3542347085 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb8a4051b10 con 0x7fb8b414b2c0 2023-11-28T18:28:21.779 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.774+0000 7fb8bb4bb640 1 -- 172.21.15.105:0/3542347085 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_distributed}] v 0) v1 -- 0x7fb8b40cf950 con 0x7fb8b414b2c0 2023-11-28T18:28:21.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.778+0000 7fb8a9ffb640 1 -- 172.21.15.105:0/3542347085 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_distributed}]=0 v37)=0 v37) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb8a40220c0 con 0x7fb8b414b2c0 2023-11-28T18:28:21.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.782+0000 7fb8bb4bb640 1 -- 172.21.15.105:0/3542347085 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb8980668d0 msgr2=0x7fb898068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:21.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.782+0000 7fb8bb4bb640 1 --2- 172.21.15.105:0/3542347085 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb8980668d0 0x7fb898068d90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0000c10 tx=0x7fb8b0001440 comp rx=0 tx=0).stop 2023-11-28T18:28:21.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.782+0000 7fb8bb4bb640 1 -- 172.21.15.105:0/3542347085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8b414b2c0 msgr2=0x7fb8b414db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:21.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.782+0000 7fb8bb4bb640 1 --2- 172.21.15.105:0/3542347085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8b414b2c0 0x7fb8b414db80 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009fd0 tx=0x7fb8a40119a0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.782+0000 7fb8bb4bb640 1 -- 172.21.15.105:0/3542347085 shutdown_connections 2023-11-28T18:28:21.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.782+0000 7fb8bb4bb640 1 --2- 172.21.15.105:0/3542347085 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8b415dc20 0x7fb8b415e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.782+0000 7fb8bb4bb640 1 --2- 172.21.15.105:0/3542347085 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb8980668d0 0x7fb898068d90 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.782+0000 7fb8bb4bb640 1 --2- 172.21.15.105:0/3542347085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8b414b2c0 0x7fb8b414db80 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.782+0000 7fb8bb4bb640 1 --2- 172.21.15.105:0/3542347085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8a0005050 0x7fb8b40cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.782+0000 7fb8bb4bb640 1 -- 172.21.15.105:0/3542347085 >> 172.21.15.105:0/3542347085 conn(0x7fb8b40ceb90 msgr2=0x7fb8b40cb060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:21.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.782+0000 7fb8bb4bb640 1 -- 172.21.15.105:0/3542347085 shutdown_connections 2023-11-28T18:28:21.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.782+0000 7fb8bb4bb640 1 -- 172.21.15.105:0/3542347085 wait complete. 2023-11-28T18:28:21.805 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random_max 1.0 2023-11-28T18:28:21.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.942+0000 7fb9fadd1640 1 Processor -- start 2023-11-28T18:28:21.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.942+0000 7fb9fadd1640 1 -- start start 2023-11-28T18:28:21.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.942+0000 7fb9fadd1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb9f40ce2c0 0x7fb9f406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:21.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.942+0000 7fb9fadd1640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9f406cd20 0x7fb9f40abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:21.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.942+0000 7fb9fadd1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9f40cde30 con 0x7fb9f4159610 2023-11-28T18:28:21.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.942+0000 7fb9fadd1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9f406d1b0 con 0x7fb9f40ce2c0 2023-11-28T18:28:21.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.942+0000 7fb9fadd1640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f41599f0 con 0x7fb9f406cd20 2023-11-28T18:28:21.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.942+0000 7fb9f936e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9f406cd20 0x7fb9f40abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:21.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.942+0000 7fb9f8b6d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb9f4159610 0x7fb9f40cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:59126/0 (socket says 172.21.15.105:59126) 2023-11-28T18:28:21.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.942+0000 7fb9f8b6d640 1 -- 172.21.15.105:0/677594878 learned_addr learned my addr 172.21.15.105:0/677594878 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:21.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.942+0000 7fb9f3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb9f40ce2c0 0x7fb9f406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:21.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.942+0000 7fb9f936e640 1 -- 172.21.15.105:0/677594878 >> v2:172.21.15.148:3300/0 conn(0x7fb9f40ce2c0 msgr2=0x7fb9f406c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:21.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.942+0000 7fb9f936e640 1 --2- 172.21.15.105:0/677594878 >> v2:172.21.15.148:3300/0 conn(0x7fb9f40ce2c0 0x7fb9f406c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.942+0000 7fb9f936e640 1 -- 172.21.15.105:0/677594878 >> v1:172.21.15.148:6789/0 conn(0x7fb9f4159610 legacy=0x7fb9f40cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:21.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.942+0000 7fb9f936e640 1 -- 172.21.15.105:0/677594878 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f406de20 con 0x7fb9f406cd20 2023-11-28T18:28:21.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.942+0000 7fb9f3fff640 1 --2- 172.21.15.105:0/677594878 >> v2:172.21.15.148:3300/0 conn(0x7fb9f40ce2c0 0x7fb9f406c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:28:21.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.946+0000 7fb9f936e640 1 --2- 172.21.15.105:0/677594878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9f406cd20 0x7fb9f40abd80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec009d90 tx=0x7fb9ec01c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f623229020db7c3c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:21.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.946+0000 7fb9f37fe640 1 -- 172.21.15.105:0/677594878 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec0a5760 con 0x7fb9f406cd20 2023-11-28T18:28:21.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.946+0000 7fb9f37fe640 1 -- 172.21.15.105:0/677594878 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ec0a58c0 con 0x7fb9f406cd20 2023-11-28T18:28:21.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.946+0000 7fb9f37fe640 1 -- 172.21.15.105:0/677594878 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec0a5c50 con 0x7fb9f406cd20 2023-11-28T18:28:21.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.946+0000 7fb9fadd1640 1 -- 172.21.15.105:0/677594878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9f406cd20 msgr2=0x7fb9f40abd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:21.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.946+0000 7fb9fadd1640 1 --2- 172.21.15.105:0/677594878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9f406cd20 0x7fb9f40abd80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec009d90 tx=0x7fb9ec01c920 comp rx=0 tx=0).stop 2023-11-28T18:28:21.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.946+0000 7fb9fadd1640 1 -- 172.21.15.105:0/677594878 shutdown_connections 2023-11-28T18:28:21.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.946+0000 7fb9fadd1640 1 --2- 172.21.15.105:0/677594878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9f406cd20 0x7fb9f40abd80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.946+0000 7fb9fadd1640 1 --2- 172.21.15.105:0/677594878 >> v2:172.21.15.148:3300/0 conn(0x7fb9f40ce2c0 0x7fb9f406c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.946+0000 7fb9fadd1640 1 -- 172.21.15.105:0/677594878 >> 172.21.15.105:0/677594878 conn(0x7fb9f40ceb90 msgr2=0x7fb9f40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:21.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.946+0000 7fb9fadd1640 1 -- 172.21.15.105:0/677594878 shutdown_connections 2023-11-28T18:28:21.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.946+0000 7fb9fadd1640 1 -- 172.21.15.105:0/677594878 wait complete. 2023-11-28T18:28:21.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9fadd1640 1 Processor -- start 2023-11-28T18:28:21.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9fadd1640 1 -- start start 2023-11-28T18:28:21.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9fadd1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9f406cd20 0x7fb9f41680f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:21.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9fadd1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9f40ce2c0 0x7fb9f4150e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:21.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9fadd1640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9f4159610 0x7fb9f40b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:21.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9fadd1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f41503e0 con 0x7fb9f406cd20 2023-11-28T18:28:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9fadd1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f409c9a0 con 0x7fb9f40ce2c0 2023-11-28T18:28:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9fadd1640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f4168630 con 0x7fb9f4159610 2023-11-28T18:28:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9f8b6d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9f406cd20 0x7fb9f41680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9f8b6d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9f406cd20 0x7fb9f41680f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47552/0 (socket says 172.21.15.105:47552) 2023-11-28T18:28:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9f8b6d640 1 -- 172.21.15.105:0/1139067690 learned_addr learned my addr 172.21.15.105:0/1139067690 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9f936e640 1 --2- 172.21.15.105:0/1139067690 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9f4159610 0x7fb9f40b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9f3fff640 1 --2- 172.21.15.105:0/1139067690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9f40ce2c0 0x7fb9f4150e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9f8b6d640 1 -- 172.21.15.105:0/1139067690 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9f4159610 msgr2=0x7fb9f40b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9f8b6d640 1 --2- 172.21.15.105:0/1139067690 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9f4159610 0x7fb9f40b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9f8b6d640 1 -- 172.21.15.105:0/1139067690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9f40ce2c0 msgr2=0x7fb9f4150e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9f8b6d640 1 --2- 172.21.15.105:0/1139067690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9f40ce2c0 0x7fb9f4150e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:21.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9f8b6d640 1 -- 172.21.15.105:0/1139067690 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e0009420 con 0x7fb9f406cd20 2023-11-28T18:28:21.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9f936e640 1 --2- 172.21.15.105:0/1139067690 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9f4159610 0x7fb9f40b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:21.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9f8b6d640 1 --2- 172.21.15.105:0/1139067690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9f406cd20 0x7fb9f41680f0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800c400 tx=0x7fb9e800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:21.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9f17fa640 1 -- 172.21.15.105:0/1139067690 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8005cf0 con 0x7fb9f406cd20 2023-11-28T18:28:21.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9f17fa640 1 -- 172.21.15.105:0/1139067690 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e8005e50 con 0x7fb9f406cd20 2023-11-28T18:28:21.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9fadd1640 1 -- 172.21.15.105:0/1139067690 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ec009a70 con 0x7fb9f406cd20 2023-11-28T18:28:21.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9f17fa640 1 -- 172.21.15.105:0/1139067690 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e80072d0 con 0x7fb9f406cd20 2023-11-28T18:28:21.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.950+0000 7fb9fadd1640 1 -- 172.21.15.105:0/1139067690 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f40ccff0 con 0x7fb9f406cd20 2023-11-28T18:28:21.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.954+0000 7fb9fadd1640 1 -- 172.21.15.105:0/1139067690 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9f4067570 con 0x7fb9f406cd20 2023-11-28T18:28:21.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.954+0000 7fb9f17fa640 1 -- 172.21.15.105:0/1139067690 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fb9e8007430 con 0x7fb9f406cd20 2023-11-28T18:28:21.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.954+0000 7fb9f17fa640 1 --2- 172.21.15.105:0/1139067690 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb9d8066900 0x7fb9d8068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:21.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.954+0000 7fb9f17fa640 1 -- 172.21.15.105:0/1139067690 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb9e8083a60 con 0x7fb9f406cd20 2023-11-28T18:28:21.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.954+0000 7fb9f3fff640 1 --2- 172.21.15.105:0/1139067690 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb9d8066900 0x7fb9d8068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:21.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.954+0000 7fb9f3fff640 1 --2- 172.21.15.105:0/1139067690 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb9d8066900 0x7fb9d8068dc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb9f41f9330 tx=0x7fb9e0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:21.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:21.958+0000 7fb9f17fa640 1 -- 172.21.15.105:0/1139067690 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb9e8050c00 con 0x7fb9f406cd20 2023-11-28T18:28:22.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:22.106+0000 7fb9fadd1640 1 -- 172.21.15.105:0/1139067690 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random_max}] v 0) v1 -- 0x7fb9f40cc740 con 0x7fb9f406cd20 2023-11-28T18:28:22.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:22.110+0000 7fb9f17fa640 1 -- 172.21.15.105:0/1139067690 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random_max}]=0 v38)=0 v38) v1 ==== 131+0+0 (secure 0 0 0) 0x7fb9e80170c0 con 0x7fb9f406cd20 2023-11-28T18:28:22.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:22.114+0000 7fb9fadd1640 1 -- 172.21.15.105:0/1139067690 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb9d8066900 msgr2=0x7fb9d8068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:22.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:22.114+0000 7fb9fadd1640 1 --2- 172.21.15.105:0/1139067690 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb9d8066900 0x7fb9d8068dc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb9f41f9330 tx=0x7fb9e0009190 comp rx=0 tx=0).stop 2023-11-28T18:28:22.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:22.114+0000 7fb9fadd1640 1 -- 172.21.15.105:0/1139067690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9f406cd20 msgr2=0x7fb9f41680f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:22.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:22.114+0000 7fb9fadd1640 1 --2- 172.21.15.105:0/1139067690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9f406cd20 0x7fb9f41680f0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800c400 tx=0x7fb9e800c8c0 comp rx=0 tx=0).stop 2023-11-28T18:28:22.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:22.114+0000 7fb9fadd1640 1 -- 172.21.15.105:0/1139067690 shutdown_connections 2023-11-28T18:28:22.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:22.114+0000 7fb9fadd1640 1 --2- 172.21.15.105:0/1139067690 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9f4159610 0x7fb9f40b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:22.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:22.114+0000 7fb9fadd1640 1 --2- 172.21.15.105:0/1139067690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9f40ce2c0 0x7fb9f4150e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:22.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:22.114+0000 7fb9fadd1640 1 --2- 172.21.15.105:0/1139067690 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb9d8066900 0x7fb9d8068dc0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:22.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:22.114+0000 7fb9fadd1640 1 --2- 172.21.15.105:0/1139067690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9f406cd20 0x7fb9f41680f0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:22.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:22.114+0000 7fb9fadd1640 1 -- 172.21.15.105:0/1139067690 >> 172.21.15.105:0/1139067690 conn(0x7fb9f40ceb90 msgr2=0x7fb9f40abe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:22.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:22.114+0000 7fb9fadd1640 1 -- 172.21.15.105:0/1139067690 shutdown_connections 2023-11-28T18:28:22.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:22.114+0000 7fb9fadd1640 1 -- 172.21.15.105:0/1139067690 wait complete. 2023-11-28T18:28:22.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:28:22.138 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-28T18:28:22.139 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:28:22.139 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:28:22.139 DEBUG:teuthology.orchestra.run.smithi105:> # Use up a random number of inode numbers so the ephemeral pinning is not the same every test. 2023-11-28T18:28:22.139 DEBUG:teuthology.orchestra.run.smithi105:> mkdir .inode_number_thrash 2023-11-28T18:28:22.139 DEBUG:teuthology.orchestra.run.smithi105:> count=$((RANDOM % 1024)) 2023-11-28T18:28:22.139 DEBUG:teuthology.orchestra.run.smithi105:> for ((i = 0; i < count; i++)); do touch .inode_number_thrash/$i; done 2023-11-28T18:28:22.139 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf .inode_number_thrash 2023-11-28T18:28:22.139 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:28:32.229 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:28:32.234 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:28:33.615 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-11-28T18:28:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.754+0000 7f331649c640 1 Processor -- start 2023-11-28T18:28:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.754+0000 7f331649c640 1 -- start start 2023-11-28T18:28:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.754+0000 7f331649c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f33100d0d40 0x7f33100d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.754+0000 7f331649c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33100cf440 con 0x7f331015d1a0 2023-11-28T18:28:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.754+0000 7f331649c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33100cffc0 con 0x7f33100d0450 2023-11-28T18:28:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.754+0000 7f331649c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33100ceb40 con 0x7f33100d0d40 2023-11-28T18:28:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.754+0000 7f331549a640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f331015d1a0 0x7f33100ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41992/0 (socket says 172.21.15.105:41992) 2023-11-28T18:28:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.754+0000 7f331549a640 1 -- 172.21.15.105:0/2130025217 learned_addr learned my addr 172.21.15.105:0/2130025217 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.754+0000 7f3314c99640 1 --2- 172.21.15.105:0/2130025217 >> v2:172.21.15.148:3300/0 conn(0x7f33100d0d40 0x7f33100d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.754+0000 7f3307fff640 1 -- 172.21.15.105:0/2130025217 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1011599664 0 0) 0x7f33100cf440 con 0x7f331015d1a0 2023-11-28T18:28:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.754+0000 7f3307fff640 1 -- 172.21.15.105:0/2130025217 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32fc003530 con 0x7f331015d1a0 2023-11-28T18:28:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.754+0000 7f3307fff640 1 -- 172.21.15.105:0/2130025217 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1762319326 0 0) 0x7f32fc003530 con 0x7f331015d1a0 2023-11-28T18:28:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.754+0000 7f3307fff640 1 -- 172.21.15.105:0/2130025217 >> v2:172.21.15.148:3300/0 conn(0x7f33100d0d40 msgr2=0x7f33100d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.754+0000 7f3307fff640 1 --2- 172.21.15.105:0/2130025217 >> v2:172.21.15.148:3300/0 conn(0x7f33100d0d40 0x7f33100d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.754+0000 7f3307fff640 1 -- 172.21.15.105:0/2130025217 >> v1:172.21.15.148:6789/0 conn(0x7f33100d0450 legacy=0x7f33100b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.754+0000 7f3307fff640 1 -- 172.21.15.105:0/2130025217 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33100cf690 con 0x7f331015d1a0 2023-11-28T18:28:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.754+0000 7f3307fff640 1 -- 172.21.15.105:0/2130025217 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f330c001680 con 0x7f331015d1a0 2023-11-28T18:28:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.754+0000 7f3307fff640 1 -- 172.21.15.105:0/2130025217 >> v1:172.21.15.105:6789/0 conn(0x7f331015d1a0 legacy=0x7f33100ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.754+0000 7f3307fff640 1 --2- 172.21.15.105:0/2130025217 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32fc003980 0x7f32fc003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.754+0000 7f3307fff640 1 --2- 172.21.15.105:0/2130025217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32fc004380 0x7f32fc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.754+0000 7f3307fff640 1 --2- 172.21.15.105:0/2130025217 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32fc0050f0 0x7f32fc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.758+0000 7f3307fff640 1 -- 172.21.15.105:0/2130025217 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33100ceb40 con 0x7f32fc003980 2023-11-28T18:28:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.758+0000 7f3307fff640 1 -- 172.21.15.105:0/2130025217 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc00c030 con 0x7f32fc004380 2023-11-28T18:28:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.758+0000 7f3307fff640 1 -- 172.21.15.105:0/2130025217 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc00c1f0 con 0x7f32fc0050f0 2023-11-28T18:28:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.758+0000 7f331549a640 1 --2- 172.21.15.105:0/2130025217 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32fc003980 0x7f32fc003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.758+0000 7f3315c9b640 1 --2- 172.21.15.105:0/2130025217 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32fc0050f0 0x7f32fc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.758+0000 7f3314c99640 1 --2- 172.21.15.105:0/2130025217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32fc004380 0x7f32fc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.758+0000 7f331549a640 1 -- 172.21.15.105:0/2130025217 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32fc0050f0 msgr2=0x7f32fc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.758+0000 7f331549a640 1 --2- 172.21.15.105:0/2130025217 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32fc0050f0 0x7f32fc00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.758+0000 7f331549a640 1 -- 172.21.15.105:0/2130025217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32fc004380 msgr2=0x7f32fc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.758+0000 7f331549a640 1 --2- 172.21.15.105:0/2130025217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32fc004380 0x7f32fc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.758+0000 7f331549a640 1 -- 172.21.15.105:0/2130025217 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32fc003530 con 0x7f32fc003980 2023-11-28T18:28:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.758+0000 7f3315c9b640 1 -- 172.21.15.105:0/2130025217 reap_dead start 2023-11-28T18:28:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.758+0000 7f331549a640 1 --2- 172.21.15.105:0/2130025217 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32fc003980 0x7f32fc003d60 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f330c00c210 tx=0x7f330c01cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5861456852a3eaf6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.758+0000 7f3307fff640 1 -- 172.21.15.105:0/2130025217 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330c005ae0 con 0x7f32fc003980 2023-11-28T18:28:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.758+0000 7f3307fff640 1 -- 172.21.15.105:0/2130025217 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330c005c60 con 0x7f32fc003980 2023-11-28T18:28:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.758+0000 7f3307fff640 1 -- 172.21.15.105:0/2130025217 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330c0a59f0 con 0x7f32fc003980 2023-11-28T18:28:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.762+0000 7f331649c640 1 -- 172.21.15.105:0/2130025217 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32fc003980 msgr2=0x7f32fc003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.762+0000 7f331649c640 1 --2- 172.21.15.105:0/2130025217 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32fc003980 0x7f32fc003d60 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f330c00c210 tx=0x7f330c01cbf0 comp rx=0 tx=0).stop 2023-11-28T18:28:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.762+0000 7f331649c640 1 -- 172.21.15.105:0/2130025217 shutdown_connections 2023-11-28T18:28:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.762+0000 7f331649c640 1 --2- 172.21.15.105:0/2130025217 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32fc003980 0x7f32fc003d60 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.762+0000 7f331649c640 1 -- 172.21.15.105:0/2130025217 >> 172.21.15.105:0/2130025217 conn(0x7f33100d2e10 msgr2=0x7f33100d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.762+0000 7f331649c640 1 -- 172.21.15.105:0/2130025217 shutdown_connections 2023-11-28T18:28:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.762+0000 7f331649c640 1 -- 172.21.15.105:0/2130025217 wait complete. 2023-11-28T18:28:33.779 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.762+0000 7f331649c640 1 Processor -- start 2023-11-28T18:28:33.779 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.762+0000 7f331649c640 1 -- start start 2023-11-28T18:28:33.780 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.762+0000 7f331649c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32fc003980 0x7f33100ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:33.780 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.762+0000 7f331649c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33101f6b80 0x7f33101f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:33.780 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.762+0000 7f331649c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f331015e680 0x7f331015eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:33.780 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.762+0000 7f331649c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33101f3b10 con 0x7f331015e680 2023-11-28T18:28:33.780 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.762+0000 7f331649c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331015f040 con 0x7f33101f6b80 2023-11-28T18:28:33.780 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.762+0000 7f331649c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33101f7000 con 0x7f32fc003980 2023-11-28T18:28:33.780 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.762+0000 7f3315c9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f331015e680 0x7f331015eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:33.781 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.762+0000 7f3315c9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f331015e680 0x7f331015eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51630/0 (socket says 172.21.15.105:51630) 2023-11-28T18:28:33.781 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.762+0000 7f3315c9b640 1 -- 172.21.15.105:0/1854609661 learned_addr learned my addr 172.21.15.105:0/1854609661 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:33.781 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.762+0000 7f331549a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32fc003980 0x7f33100ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:33.781 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.762+0000 7f3314c99640 1 --2- 172.21.15.105:0/1854609661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33101f6b80 0x7f33101f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:33.781 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.762+0000 7f3315c9b640 1 -- 172.21.15.105:0/1854609661 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32fc003980 msgr2=0x7f33100ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:33.781 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.762+0000 7f3315c9b640 1 --2- 172.21.15.105:0/1854609661 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32fc003980 0x7f33100ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:33.781 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.766+0000 7f3315c9b640 1 -- 172.21.15.105:0/1854609661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33101f6b80 msgr2=0x7f33101f35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:33.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.766+0000 7f3315c9b640 1 --2- 172.21.15.105:0/1854609661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33101f6b80 0x7f33101f35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:33.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.766+0000 7f3315c9b640 1 -- 172.21.15.105:0/1854609661 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33100c9ee0 con 0x7f331015e680 2023-11-28T18:28:33.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.766+0000 7f3315c9b640 1 --2- 172.21.15.105:0/1854609661 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f331015e680 0x7f331015eb00 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f331008d3b0 tx=0x7f330800feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:33.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.766+0000 7f3305ffb640 1 -- 172.21.15.105:0/1854609661 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308006d70 con 0x7f331015e680 2023-11-28T18:28:33.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.766+0000 7f3305ffb640 1 -- 172.21.15.105:0/1854609661 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3308003980 con 0x7f331015e680 2023-11-28T18:28:33.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.766+0000 7f3305ffb640 1 -- 172.21.15.105:0/1854609661 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308003d30 con 0x7f331015e680 2023-11-28T18:28:33.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.766+0000 7f331649c640 1 -- 172.21.15.105:0/1854609661 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33100ca1c0 con 0x7f331015e680 2023-11-28T18:28:33.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.766+0000 7f331649c640 1 -- 172.21.15.105:0/1854609661 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33101f72c0 con 0x7f331015e680 2023-11-28T18:28:33.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.766+0000 7f331649c640 1 -- 172.21.15.105:0/1854609661 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3310074580 con 0x7f331015e680 2023-11-28T18:28:33.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.766+0000 7f3305ffb640 1 -- 172.21.15.105:0/1854609661 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3308005030 con 0x7f331015e680 2023-11-28T18:28:33.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.766+0000 7f3305ffb640 1 --2- 172.21.15.105:0/1854609661 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f32f4066900 0x7f32f4068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:33.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.766+0000 7f3305ffb640 1 -- 172.21.15.105:0/1854609661 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3308083660 con 0x7f331015e680 2023-11-28T18:28:33.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.766+0000 7f331549a640 1 --2- 172.21.15.105:0/1854609661 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f32f4066900 0x7f32f4068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:33.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.770+0000 7f331549a640 1 --2- 172.21.15.105:0/1854609661 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f32f4066900 0x7f32f4068dc0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f330c0ab000 tx=0x7f330c0b1040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:33.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.774+0000 7f3305ffb640 1 -- 172.21.15.105:0/1854609661 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3308050770 con 0x7f331015e680 2023-11-28T18:28:33.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:33.986+0000 7f331649c640 1 -- 172.21.15.105:0/1854609661 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f331008c5d0 con 0x7f331015e680 2023-11-28T18:28:34.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.126+0000 7f3305ffb640 1 -- 172.21.15.105:0/1854609661 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v72) v1 ==== 105+0+0 (secure 0 0 0) 0x7f3308053d90 con 0x7f331015e680 2023-11-28T18:28:34.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.126+0000 7f331649c640 1 -- 172.21.15.105:0/1854609661 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f32f4066900 msgr2=0x7f32f4068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:34.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.126+0000 7f331649c640 1 --2- 172.21.15.105:0/1854609661 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f32f4066900 0x7f32f4068dc0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f330c0ab000 tx=0x7f330c0b1040 comp rx=0 tx=0).stop 2023-11-28T18:28:34.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.126+0000 7f331649c640 1 -- 172.21.15.105:0/1854609661 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f331015e680 msgr2=0x7f331015eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:34.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.126+0000 7f331649c640 1 --2- 172.21.15.105:0/1854609661 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f331015e680 0x7f331015eb00 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f331008d3b0 tx=0x7f330800feb0 comp rx=0 tx=0).stop 2023-11-28T18:28:34.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.126+0000 7f331649c640 1 -- 172.21.15.105:0/1854609661 shutdown_connections 2023-11-28T18:28:34.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.126+0000 7f331649c640 1 --2- 172.21.15.105:0/1854609661 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f32f4066900 0x7f32f4068dc0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:34.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.126+0000 7f331649c640 1 --2- 172.21.15.105:0/1854609661 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f331015e680 0x7f331015eb00 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:34.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.126+0000 7f331649c640 1 --2- 172.21.15.105:0/1854609661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33101f6b80 0x7f33101f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:34.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.126+0000 7f331649c640 1 --2- 172.21.15.105:0/1854609661 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32fc003980 0x7f33100ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:34.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.126+0000 7f331649c640 1 -- 172.21.15.105:0/1854609661 >> 172.21.15.105:0/1854609661 conn(0x7f33100d2e10 msgr2=0x7f33100d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:34.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.126+0000 7f331649c640 1 -- 172.21.15.105:0/1854609661 shutdown_connections 2023-11-28T18:28:34.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.126+0000 7f331649c640 1 -- 172.21.15.105:0/1854609661 wait complete. 2023-11-28T18:28:34.153 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:28:34.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.286+0000 7fe72b11d640 1 Processor -- start 2023-11-28T18:28:34.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.286+0000 7fe72b11d640 1 -- start start 2023-11-28T18:28:34.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.286+0000 7fe72b11d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe72406d610 0x7fe7240cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:34.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.286+0000 7fe72b11d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe7240ce530 0x7fe7240ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:34.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.286+0000 7fe72b11d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe724162820 con 0x7fe7240adf30 2023-11-28T18:28:34.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.286+0000 7fe72b11d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7240ae340 con 0x7fe7240ce530 2023-11-28T18:28:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.286+0000 7fe72b11d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe724161c10 con 0x7fe72406d610 2023-11-28T18:28:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.286+0000 7fe723fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe7240ce530 0x7fe7240ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.286+0000 7fe7296ba640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe7240adf30 0x7fe7241650f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42006/0 (socket says 172.21.15.105:42006) 2023-11-28T18:28:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.286+0000 7fe7296ba640 1 -- 172.21.15.105:0/2075068204 learned_addr learned my addr 172.21.15.105:0/2075068204 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.290+0000 7fe728eb9640 1 --2- 172.21.15.105:0/2075068204 >> v2:172.21.15.148:3300/0 conn(0x7fe72406d610 0x7fe7240cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.290+0000 7fe723fff640 1 -- 172.21.15.105:0/2075068204 >> v2:172.21.15.148:3300/0 conn(0x7fe72406d610 msgr2=0x7fe7240cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.290+0000 7fe723fff640 1 --2- 172.21.15.105:0/2075068204 >> v2:172.21.15.148:3300/0 conn(0x7fe72406d610 0x7fe7240cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:34.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.290+0000 7fe723fff640 1 -- 172.21.15.105:0/2075068204 >> v1:172.21.15.105:6789/0 conn(0x7fe7240adf30 legacy=0x7fe7241650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:34.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.290+0000 7fe723fff640 1 -- 172.21.15.105:0/2075068204 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe724161ef0 con 0x7fe7240ce530 2023-11-28T18:28:34.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.290+0000 7fe728eb9640 1 --2- 172.21.15.105:0/2075068204 >> v2:172.21.15.148:3300/0 conn(0x7fe72406d610 0x7fe7240cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:34.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.290+0000 7fe723fff640 1 --2- 172.21.15.105:0/2075068204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7240ce530 0x7fe7240ad9f0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fe72406fa90 tx=0x7fe71001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5a86d2432bd3f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:34.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.290+0000 7fe7237fe640 1 -- 172.21.15.105:0/2075068204 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe710004230 con 0x7fe7240ce530 2023-11-28T18:28:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.290+0000 7fe7237fe640 1 -- 172.21.15.105:0/2075068204 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7100a7050 con 0x7fe7240ce530 2023-11-28T18:28:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.290+0000 7fe7237fe640 1 -- 172.21.15.105:0/2075068204 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe710003620 con 0x7fe7240ce530 2023-11-28T18:28:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.290+0000 7fe72b11d640 1 -- 172.21.15.105:0/2075068204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7240ce530 msgr2=0x7fe7240ad9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.290+0000 7fe72b11d640 1 --2- 172.21.15.105:0/2075068204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7240ce530 0x7fe7240ad9f0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fe72406fa90 tx=0x7fe71001c430 comp rx=0 tx=0).stop 2023-11-28T18:28:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe72b11d640 1 -- 172.21.15.105:0/2075068204 shutdown_connections 2023-11-28T18:28:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe72b11d640 1 --2- 172.21.15.105:0/2075068204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7240ce530 0x7fe7240ad9f0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe72b11d640 1 --2- 172.21.15.105:0/2075068204 >> v2:172.21.15.148:3300/0 conn(0x7fe72406d610 0x7fe7240cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe72b11d640 1 -- 172.21.15.105:0/2075068204 >> 172.21.15.105:0/2075068204 conn(0x7fe7240ceb90 msgr2=0x7fe7240cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe72b11d640 1 -- 172.21.15.105:0/2075068204 shutdown_connections 2023-11-28T18:28:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe72b11d640 1 -- 172.21.15.105:0/2075068204 wait complete. 2023-11-28T18:28:34.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe72b11d640 1 Processor -- start 2023-11-28T18:28:34.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe72b11d640 1 -- start start 2023-11-28T18:28:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe72b11d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe72406d610 0x7fe72409c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe72b11d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7240adf30 0x7fe7240b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe72b11d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7240ce530 0x7fe72415acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe72b11d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7240b4240 con 0x7fe7240ce530 2023-11-28T18:28:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe72b11d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7240b31a0 con 0x7fe7240adf30 2023-11-28T18:28:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe72b11d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7240b3480 con 0x7fe72406d610 2023-11-28T18:28:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe7296ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7240ce530 0x7fe72415acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe7296ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7240ce530 0x7fe72415acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51656/0 (socket says 172.21.15.105:51656) 2023-11-28T18:28:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe7296ba640 1 -- 172.21.15.105:0/3421527659 learned_addr learned my addr 172.21.15.105:0/3421527659 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe728eb9640 1 --2- 172.21.15.105:0/3421527659 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe72406d610 0x7fe72409c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe723fff640 1 --2- 172.21.15.105:0/3421527659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7240adf30 0x7fe7240b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe7296ba640 1 -- 172.21.15.105:0/3421527659 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe72406d610 msgr2=0x7fe72409c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe7296ba640 1 --2- 172.21.15.105:0/3421527659 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe72406d610 0x7fe72409c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe7296ba640 1 -- 172.21.15.105:0/3421527659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7240adf30 msgr2=0x7fe7240b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe7296ba640 1 --2- 172.21.15.105:0/3421527659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7240adf30 0x7fe7240b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe7296ba640 1 -- 172.21.15.105:0/3421527659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe710009420 con 0x7fe7240ce530 2023-11-28T18:28:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.294+0000 7fe723fff640 1 --2- 172.21.15.105:0/3421527659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7240adf30 0x7fe7240b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.298+0000 7fe7296ba640 1 --2- 172.21.15.105:0/3421527659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7240ce530 0x7fe72415acb0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fe71c00cbc0 tx=0x7fe71c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.298+0000 7fe7217fa640 1 -- 172.21.15.105:0/3421527659 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c00ad40 con 0x7fe7240ce530 2023-11-28T18:28:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.298+0000 7fe7217fa640 1 -- 172.21.15.105:0/3421527659 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71c012400 con 0x7fe7240ce530 2023-11-28T18:28:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.298+0000 7fe7217fa640 1 -- 172.21.15.105:0/3421527659 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c00aea0 con 0x7fe7240ce530 2023-11-28T18:28:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.298+0000 7fe72b11d640 1 -- 172.21.15.105:0/3421527659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7240b3760 con 0x7fe7240ce530 2023-11-28T18:28:34.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.298+0000 7fe72b11d640 1 -- 172.21.15.105:0/3421527659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7240b3d20 con 0x7fe7240ce530 2023-11-28T18:28:34.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.298+0000 7fe72b11d640 1 -- 172.21.15.105:0/3421527659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7240b3f10 con 0x7fe7240ce530 2023-11-28T18:28:34.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.298+0000 7fe7217fa640 1 -- 172.21.15.105:0/3421527659 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fe71c007070 con 0x7fe7240ce530 2023-11-28T18:28:34.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.298+0000 7fe7217fa640 1 --2- 172.21.15.105:0/3421527659 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe7080668d0 0x7fe708068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:34.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.298+0000 7fe7217fa640 1 -- 172.21.15.105:0/3421527659 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe71c0843e0 con 0x7fe7240ce530 2023-11-28T18:28:34.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.298+0000 7fe728eb9640 1 --2- 172.21.15.105:0/3421527659 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe7080668d0 0x7fe708068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:34.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.302+0000 7fe728eb9640 1 --2- 172.21.15.105:0/3421527659 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe7080668d0 0x7fe708068d90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe7240cb510 tx=0x7fe718008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:34.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.302+0000 7fe7217fa640 1 -- 172.21.15.105:0/3421527659 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe71c051580 con 0x7fe7240ce530 2023-11-28T18:28:34.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.482+0000 7fe72b11d640 1 -- 172.21.15.105:0/3421527659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe724067570 con 0x7fe7240ce530 2023-11-28T18:28:34.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.482+0000 7fe7217fa640 1 -- 172.21.15.105:0/3421527659 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4937 (secure 0 0 0) 0x7fe71c054ba0 con 0x7fe7240ce530 2023-11-28T18:28:34.488 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:34.489 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":73,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1455698212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":1455698212},{"type":"v1","addr":"172.21.15.105:6837","nonce":1455698212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6859,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/759522673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":759522673},{"type":"v1","addr":"172.21.15.148:6834","nonce":759522673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:28:13.097621+0000","modified":"2023-11-28T18:28:34.132438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1],"up":{"mds_0":7194,"mds_1":6862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6862":{"gid":6862,"name":"d","rank":1,"incarnation":73,"state":"up:creating","state_seq":1,"addr":"172.21.15.148:6835/2489361300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2489361300},{"type":"v1","addr":"172.21.15.148:6835","nonce":2489361300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7194":{"gid":7194,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6836/188173955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":188173955},{"type":"v1","addr":"172.21.15.105:6836","nonce":188173955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-28T18:28:34.490 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 73 2023-11-28T18:28:34.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.486+0000 7fe72b11d640 1 -- 172.21.15.105:0/3421527659 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe7080668d0 msgr2=0x7fe708068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:34.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.486+0000 7fe72b11d640 1 --2- 172.21.15.105:0/3421527659 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe7080668d0 0x7fe708068d90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe7240cb510 tx=0x7fe718008040 comp rx=0 tx=0).stop 2023-11-28T18:28:34.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.486+0000 7fe72b11d640 1 -- 172.21.15.105:0/3421527659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7240ce530 msgr2=0x7fe72415acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:34.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.486+0000 7fe72b11d640 1 --2- 172.21.15.105:0/3421527659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7240ce530 0x7fe72415acb0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fe71c00cbc0 tx=0x7fe71c00a430 comp rx=0 tx=0).stop 2023-11-28T18:28:34.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.486+0000 7fe72b11d640 1 -- 172.21.15.105:0/3421527659 shutdown_connections 2023-11-28T18:28:34.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.486+0000 7fe72b11d640 1 --2- 172.21.15.105:0/3421527659 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe7080668d0 0x7fe708068d90 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:34.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.486+0000 7fe72b11d640 1 --2- 172.21.15.105:0/3421527659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7240ce530 0x7fe72415acb0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:34.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.486+0000 7fe72b11d640 1 --2- 172.21.15.105:0/3421527659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7240adf30 0x7fe7240b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:34.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.486+0000 7fe72b11d640 1 --2- 172.21.15.105:0/3421527659 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe72406d610 0x7fe72409c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:34.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.486+0000 7fe72b11d640 1 -- 172.21.15.105:0/3421527659 >> 172.21.15.105:0/3421527659 conn(0x7fe7240ceb90 msgr2=0x7fe7240cc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:34.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.486+0000 7fe72b11d640 1 -- 172.21.15.105:0/3421527659 shutdown_connections 2023-11-28T18:28:34.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:34.486+0000 7fe72b11d640 1 -- 172.21.15.105:0/3421527659 wait complete. 2023-11-28T18:28:34.910 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:28:13.097621+0000', 'modified': '2023-11-28T18:28:34.132438+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 7194, 'mds_1': 6862}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6862': {'gid': 6862, 'name': 'd', 'rank': 1, 'incarnation': 73, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.148:6835/2489361300', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6833', 'nonce': 2489361300}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 2489361300}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_7194': {'gid': 7194, 'name': 'a', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6836/188173955', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 188173955}, {'type': 'v1', 'addr': '172.21.15.105:6836', 'nonce': 188173955}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:28:34.911 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6862:up:creating 2023-11-28T18:28:35.914 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:28:36.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.062+0000 7f5f4c5f8640 1 Processor -- start 2023-11-28T18:28:36.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.062+0000 7f5f4c5f8640 1 -- start start 2023-11-28T18:28:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.062+0000 7f5f4c5f8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5f440caf60 0x7f5f440cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.062+0000 7f5f4c5f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5f440ad850 0x7f5f440701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.062+0000 7f5f4c5f8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f4406bf50 con 0x7f5f44159f40 2023-11-28T18:28:36.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.062+0000 7f5f4c5f8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f440cb3c0 con 0x7f5f440caf60 2023-11-28T18:28:36.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.062+0000 7f5f4c5f8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4415a320 con 0x7f5f440ad850 2023-11-28T18:28:36.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.062+0000 7f5f49b93640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5f440caf60 0x7f5f440cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.062+0000 7f5f49b93640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5f440caf60 0x7f5f440cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51660/0 (socket says 172.21.15.105:51660) 2023-11-28T18:28:36.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.062+0000 7f5f4a394640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5f44159f40 0x7f5f440ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42008/0 (socket says 172.21.15.105:42008) 2023-11-28T18:28:36.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.062+0000 7f5f49b93640 1 -- 172.21.15.105:0/2088025628 learned_addr learned my addr 172.21.15.105:0/2088025628 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:36.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.062+0000 7f5f4ab95640 1 --2- 172.21.15.105:0/2088025628 >> v2:172.21.15.148:3300/0 conn(0x7f5f440ad850 0x7f5f440701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.062+0000 7f5f49b93640 1 -- 172.21.15.105:0/2088025628 >> v2:172.21.15.148:3300/0 conn(0x7f5f440ad850 msgr2=0x7f5f440701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:36.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.062+0000 7f5f49b93640 1 --2- 172.21.15.105:0/2088025628 >> v2:172.21.15.148:3300/0 conn(0x7f5f440ad850 0x7f5f440701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.066+0000 7f5f49b93640 1 -- 172.21.15.105:0/2088025628 >> v1:172.21.15.105:6789/0 conn(0x7f5f44159f40 legacy=0x7f5f440ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:36.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.066+0000 7f5f49b93640 1 -- 172.21.15.105:0/2088025628 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f44071380 con 0x7f5f440caf60 2023-11-28T18:28:36.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.066+0000 7f5f49392640 1 -- 172.21.15.105:0/2088025628 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 927294448 0 0) 0x7f5f4406bf50 con 0x7f5f44159f40 2023-11-28T18:28:36.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.066+0000 7f5f49b93640 1 --2- 172.21.15.105:0/2088025628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f440caf60 0x7f5f440cc760 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f5f440b0850 tx=0x7f5f3401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8d2fd11e4ad8bc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:36.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.066+0000 7f5f49392640 1 -- 172.21.15.105:0/2088025628 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f340ab640 con 0x7f5f440caf60 2023-11-28T18:28:36.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.066+0000 7f5f49392640 1 -- 172.21.15.105:0/2088025628 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f340a7070 con 0x7f5f440caf60 2023-11-28T18:28:36.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.066+0000 7f5f49392640 1 -- 172.21.15.105:0/2088025628 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f340ab9c0 con 0x7f5f440caf60 2023-11-28T18:28:36.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.066+0000 7f5f4c5f8640 1 -- 172.21.15.105:0/2088025628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f440caf60 msgr2=0x7f5f440cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.066+0000 7f5f4c5f8640 1 --2- 172.21.15.105:0/2088025628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f440caf60 0x7f5f440cc760 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f5f440b0850 tx=0x7f5f3401c440 comp rx=0 tx=0).stop 2023-11-28T18:28:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.066+0000 7f5f4c5f8640 1 -- 172.21.15.105:0/2088025628 shutdown_connections 2023-11-28T18:28:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.066+0000 7f5f4c5f8640 1 --2- 172.21.15.105:0/2088025628 >> v2:172.21.15.148:3300/0 conn(0x7f5f440ad850 0x7f5f440701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.066+0000 7f5f4c5f8640 1 --2- 172.21.15.105:0/2088025628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f440caf60 0x7f5f440cc760 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.066+0000 7f5f4c5f8640 1 -- 172.21.15.105:0/2088025628 >> 172.21.15.105:0/2088025628 conn(0x7f5f440ceb90 msgr2=0x7f5f440cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.066+0000 7f5f4c5f8640 1 -- 172.21.15.105:0/2088025628 shutdown_connections 2023-11-28T18:28:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f4c5f8640 1 -- 172.21.15.105:0/2088025628 wait complete. 2023-11-28T18:28:36.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f4c5f8640 1 Processor -- start 2023-11-28T18:28:36.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f4c5f8640 1 -- start start 2023-11-28T18:28:36.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f4c5f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f440ad850 0x7f5f440b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f4c5f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f440caf60 0x7f5f4413e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f4c5f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f44159f40 0x7f5f44156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f4c5f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f440b3bb0 con 0x7f5f440ad850 2023-11-28T18:28:36.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f4c5f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f440b4580 con 0x7f5f44159f40 2023-11-28T18:28:36.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f4c5f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f440b2fa0 con 0x7f5f440caf60 2023-11-28T18:28:36.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f4a394640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f440ad850 0x7f5f440b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f4a394640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f440ad850 0x7f5f440b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51668/0 (socket says 172.21.15.105:51668) 2023-11-28T18:28:36.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f4a394640 1 -- 172.21.15.105:0/3952171734 learned_addr learned my addr 172.21.15.105:0/3952171734 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:36.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f49b93640 1 --2- 172.21.15.105:0/3952171734 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f440caf60 0x7f5f4413e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f4ab95640 1 --2- 172.21.15.105:0/3952171734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f44159f40 0x7f5f44156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f4a394640 1 -- 172.21.15.105:0/3952171734 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f440caf60 msgr2=0x7f5f4413e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:36.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f4a394640 1 --2- 172.21.15.105:0/3952171734 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f440caf60 0x7f5f4413e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f4a394640 1 -- 172.21.15.105:0/3952171734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f44159f40 msgr2=0x7f5f44156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:36.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f4a394640 1 --2- 172.21.15.105:0/3952171734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f44159f40 0x7f5f44156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f4a394640 1 -- 172.21.15.105:0/3952171734 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f340094d0 con 0x7f5f440ad850 2023-11-28T18:28:36.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f4a394640 1 --2- 172.21.15.105:0/3952171734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f440ad850 0x7f5f440b4040 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000c480 tx=0x7f5f4000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:36.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f3affd640 1 -- 172.21.15.105:0/3952171734 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f400074e0 con 0x7f5f440ad850 2023-11-28T18:28:36.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f3affd640 1 -- 172.21.15.105:0/3952171734 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f40007660 con 0x7f5f440ad850 2023-11-28T18:28:36.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f4c5f8640 1 -- 172.21.15.105:0/3952171734 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f440b3280 con 0x7f5f440ad850 2023-11-28T18:28:36.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f3affd640 1 -- 172.21.15.105:0/3952171734 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f40004a30 con 0x7f5f440ad850 2023-11-28T18:28:36.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.070+0000 7f5f4c5f8640 1 -- 172.21.15.105:0/3952171734 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f440b3840 con 0x7f5f440ad850 2023-11-28T18:28:36.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.074+0000 7f5f4c5f8640 1 -- 172.21.15.105:0/3952171734 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f44067570 con 0x7f5f440ad850 2023-11-28T18:28:36.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.074+0000 7f5f3affd640 1 -- 172.21.15.105:0/3952171734 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5f40004b90 con 0x7f5f440ad850 2023-11-28T18:28:36.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.074+0000 7f5f3affd640 1 --2- 172.21.15.105:0/3952171734 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f28066900 0x7f5f28068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.074+0000 7f5f3affd640 1 -- 172.21.15.105:0/3952171734 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5f400837a0 con 0x7f5f440ad850 2023-11-28T18:28:36.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.074+0000 7f5f49b93640 1 --2- 172.21.15.105:0/3952171734 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f28066900 0x7f5f28068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.074+0000 7f5f49b93640 1 --2- 172.21.15.105:0/3952171734 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f28066900 0x7f5f28068dc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5f340af000 tx=0x7f5f340a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:36.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.078+0000 7f5f3affd640 1 -- 172.21.15.105:0/3952171734 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5f40054290 con 0x7f5f440ad850 2023-11-28T18:28:36.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.262+0000 7f5f4c5f8640 1 -- 172.21.15.105:0/3952171734 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f441572a0 con 0x7f5f440ad850 2023-11-28T18:28:36.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.262+0000 7f5f3affd640 1 -- 172.21.15.105:0/3952171734 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f5f40053e20 con 0x7f5f440ad850 2023-11-28T18:28:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:36.267 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":76,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1455698212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":1455698212},{"type":"v1","addr":"172.21.15.105:6837","nonce":1455698212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:28:13.097621+0000","modified":"2023-11-28T18:28:35.162563+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7194,"mds_1":6862,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"b","rank":2,"incarnation":75,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6834/759522673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":759522673},{"type":"v1","addr":"172.21.15.148:6834","nonce":759522673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6862":{"gid":6862,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6835/2489361300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2489361300},{"type":"v1","addr":"172.21.15.148:6835","nonce":2489361300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7194":{"gid":7194,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6836/188173955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":188173955},{"type":"v1","addr":"172.21.15.105:6836","nonce":188173955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-28T18:28:36.268 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 76 2023-11-28T18:28:36.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.266+0000 7f5f4c5f8640 1 -- 172.21.15.105:0/3952171734 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f28066900 msgr2=0x7f5f28068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.266+0000 7f5f4c5f8640 1 --2- 172.21.15.105:0/3952171734 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f28066900 0x7f5f28068dc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5f340af000 tx=0x7f5f340a7000 comp rx=0 tx=0).stop 2023-11-28T18:28:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.266+0000 7f5f4c5f8640 1 -- 172.21.15.105:0/3952171734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f440ad850 msgr2=0x7f5f440b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.266+0000 7f5f4c5f8640 1 --2- 172.21.15.105:0/3952171734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f440ad850 0x7f5f440b4040 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000c480 tx=0x7f5f4000c940 comp rx=0 tx=0).stop 2023-11-28T18:28:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.266+0000 7f5f4c5f8640 1 -- 172.21.15.105:0/3952171734 shutdown_connections 2023-11-28T18:28:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.266+0000 7f5f4c5f8640 1 --2- 172.21.15.105:0/3952171734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f44159f40 0x7f5f44156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.266+0000 7f5f4c5f8640 1 --2- 172.21.15.105:0/3952171734 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f440caf60 0x7f5f4413e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.266+0000 7f5f4c5f8640 1 --2- 172.21.15.105:0/3952171734 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f28066900 0x7f5f28068dc0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.266+0000 7f5f4c5f8640 1 --2- 172.21.15.105:0/3952171734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f440ad850 0x7f5f440b4040 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.266+0000 7f5f4c5f8640 1 -- 172.21.15.105:0/3952171734 >> 172.21.15.105:0/3952171734 conn(0x7f5f440ceb90 msgr2=0x7f5f440aed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.266+0000 7f5f4c5f8640 1 -- 172.21.15.105:0/3952171734 shutdown_connections 2023-11-28T18:28:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.266+0000 7f5f4c5f8640 1 -- 172.21.15.105:0/3952171734 wait complete. 2023-11-28T18:28:36.690 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:28:13.097621+0000', 'modified': '2023-11-28T18:28:35.162563+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 7194, 'mds_1': 6862, 'mds_2': 6859}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6859': {'gid': 6859, 'name': 'b', 'rank': 2, 'incarnation': 75, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.148:6834/759522673', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6832', 'nonce': 759522673}, {'type': 'v1', 'addr': '172.21.15.148:6834', 'nonce': 759522673}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_6862': {'gid': 6862, 'name': 'd', 'rank': 1, 'incarnation': 73, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.148:6835/2489361300', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6833', 'nonce': 2489361300}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 2489361300}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_7194': {'gid': 7194, 'name': 'a', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6836/188173955', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 188173955}, {'type': 'v1', 'addr': '172.21.15.105:6836', 'nonce': 188173955}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:28:36.691 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-11-28T18:28:36.691 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-28T18:28:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.834+0000 7f5640f65640 1 Processor -- start 2023-11-28T18:28:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.834+0000 7f5640f65640 1 -- start start 2023-11-28T18:28:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.834+0000 7f5640f65640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f563c071960 0x7f563c15f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.834+0000 7f5640f65640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f563c15b030 0x7f563c0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.834+0000 7f5640f65640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f563c0701d0 con 0x7f563c0b4290 2023-11-28T18:28:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.834+0000 7f5640f65640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f563c06df80 con 0x7f563c071960 2023-11-28T18:28:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.834+0000 7f5640f65640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c071df0 con 0x7f563c15b030 2023-11-28T18:28:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.834+0000 7f563ad9d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f563c15b030 0x7f563c0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.834+0000 7f5631d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f563c071960 0x7f563c15f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.834+0000 7f563a59c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f563c0b4290 0x7f563c0b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:42336/0 (socket says 172.21.15.105:42336) 2023-11-28T18:28:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.834+0000 7f563ad9d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f563c15b030 0x7f563c0709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:34682/0 (socket says 172.21.15.105:34682) 2023-11-28T18:28:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.834+0000 7f563a59c640 1 -- 172.21.15.105:0/4207432535 learned_addr learned my addr 172.21.15.105:0/4207432535 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.834+0000 7f563ad9d640 1 -- 172.21.15.105:0/4207432535 >> v2:172.21.15.148:3300/0 conn(0x7f563c071960 msgr2=0x7f563c15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.838+0000 7f563ad9d640 1 --2- 172.21.15.105:0/4207432535 >> v2:172.21.15.148:3300/0 conn(0x7f563c071960 0x7f563c15f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.838+0000 7f563ad9d640 1 -- 172.21.15.105:0/4207432535 >> v1:172.21.15.148:6789/0 conn(0x7f563c0b4290 legacy=0x7f563c0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.838+0000 7f563ad9d640 1 -- 172.21.15.105:0/4207432535 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c0720d0 con 0x7f563c15b030 2023-11-28T18:28:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.838+0000 7f5631d9b640 1 --2- 172.21.15.105:0/4207432535 >> v2:172.21.15.148:3300/0 conn(0x7f563c071960 0x7f563c15f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:28:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.838+0000 7f563ad9d640 1 --2- 172.21.15.105:0/4207432535 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f563c15b030 0x7f563c0709b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5634009d90 tx=0x7f563401c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c320c8042dec3d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.838+0000 7f5639d9b640 1 -- 172.21.15.105:0/4207432535 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56340a58f0 con 0x7f563c15b030 2023-11-28T18:28:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.838+0000 7f5639d9b640 1 -- 172.21.15.105:0/4207432535 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56340a5a50 con 0x7f563c15b030 2023-11-28T18:28:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.838+0000 7f5639d9b640 1 -- 172.21.15.105:0/4207432535 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56340a5de0 con 0x7f563c15b030 2023-11-28T18:28:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.838+0000 7f5640f65640 1 -- 172.21.15.105:0/4207432535 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f563c15b030 msgr2=0x7f563c0709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.838+0000 7f5640f65640 1 --2- 172.21.15.105:0/4207432535 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f563c15b030 0x7f563c0709b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5634009d90 tx=0x7f563401c920 comp rx=0 tx=0).stop 2023-11-28T18:28:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.838+0000 7f5640f65640 1 -- 172.21.15.105:0/4207432535 shutdown_connections 2023-11-28T18:28:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.838+0000 7f5640f65640 1 --2- 172.21.15.105:0/4207432535 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f563c15b030 0x7f563c0709b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.838+0000 7f5640f65640 1 --2- 172.21.15.105:0/4207432535 >> v2:172.21.15.148:3300/0 conn(0x7f563c071960 0x7f563c15f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.838+0000 7f5640f65640 1 -- 172.21.15.105:0/4207432535 >> 172.21.15.105:0/4207432535 conn(0x7f563c0ceb90 msgr2=0x7f563c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.838+0000 7f5640f65640 1 -- 172.21.15.105:0/4207432535 shutdown_connections 2023-11-28T18:28:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.842+0000 7f5640f65640 1 -- 172.21.15.105:0/4207432535 wait complete. 2023-11-28T18:28:36.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.842+0000 7f5640f65640 1 Processor -- start 2023-11-28T18:28:36.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.842+0000 7f5640f65640 1 -- start start 2023-11-28T18:28:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.842+0000 7f5640f65640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f563c0b4290 0x7f563c153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.842+0000 7f5640f65640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f563c15b030 0x7f563c0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.842+0000 7f5640f65640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f563c099f20 0x7f563c09a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.842+0000 7f5640f65640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c156ce0 con 0x7f563c099f20 2023-11-28T18:28:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.842+0000 7f5640f65640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0b0540 con 0x7f563c0b4290 2023-11-28T18:28:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.842+0000 7f5640f65640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c156f30 con 0x7f563c15b030 2023-11-28T18:28:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.842+0000 7f563ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f563c099f20 0x7f563c09a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.842+0000 7f563ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f563c099f20 0x7f563c09a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51676/0 (socket says 172.21.15.105:51676) 2023-11-28T18:28:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.842+0000 7f563ad9d640 1 -- 172.21.15.105:0/333453851 learned_addr learned my addr 172.21.15.105:0/333453851 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.842+0000 7f563a59c640 1 --2- 172.21.15.105:0/333453851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f563c0b4290 0x7f563c153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.842+0000 7f5631d9b640 1 --2- 172.21.15.105:0/333453851 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f563c15b030 0x7f563c0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.842+0000 7f563ad9d640 1 -- 172.21.15.105:0/333453851 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f563c15b030 msgr2=0x7f563c0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.842+0000 7f563ad9d640 1 --2- 172.21.15.105:0/333453851 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f563c15b030 0x7f563c0b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.842+0000 7f563ad9d640 1 -- 172.21.15.105:0/333453851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f563c0b4290 msgr2=0x7f563c153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.842+0000 7f563ad9d640 1 --2- 172.21.15.105:0/333453851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f563c0b4290 0x7f563c153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.842+0000 7f563ad9d640 1 -- 172.21.15.105:0/333453851 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c009420 con 0x7f563c099f20 2023-11-28T18:28:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.846+0000 7f563ad9d640 1 --2- 172.21.15.105:0/333453851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f563c099f20 0x7f563c09a3d0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f56340b0040 tx=0x7f56340196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.846+0000 7f56337fe640 1 -- 172.21.15.105:0/333453851 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56340b9860 con 0x7f563c099f20 2023-11-28T18:28:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.846+0000 7f56337fe640 1 -- 172.21.15.105:0/333453851 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56340b99c0 con 0x7f563c099f20 2023-11-28T18:28:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.846+0000 7f5640f65640 1 -- 172.21.15.105:0/333453851 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5634009a70 con 0x7f563c099f20 2023-11-28T18:28:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.846+0000 7f56337fe640 1 -- 172.21.15.105:0/333453851 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56340b9d50 con 0x7f563c099f20 2023-11-28T18:28:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.846+0000 7f5640f65640 1 -- 172.21.15.105:0/333453851 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563c1574f0 con 0x7f563c099f20 2023-11-28T18:28:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.846+0000 7f56337fe640 1 -- 172.21.15.105:0/333453851 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f56340c4460 con 0x7f563c099f20 2023-11-28T18:28:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.846+0000 7f56337fe640 1 --2- 172.21.15.105:0/333453851 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f561c0668d0 0x7f561c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.846+0000 7f56337fe640 1 -- 172.21.15.105:0/333453851 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f563411f990 con 0x7f563c099f20 2023-11-28T18:28:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.846+0000 7f563a59c640 1 --2- 172.21.15.105:0/333453851 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f561c0668d0 0x7f561c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f56442c2000 1 Processor -- start 2023-11-28T18:28:36.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f563a59c640 1 --2- 172.21.15.105:0/333453851 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f561c0668d0 0x7f561c068d90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5624005890 tx=0x7f5624009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:36.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f56442c2000 1 -- start start 2023-11-28T18:28:36.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f56442c2000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e999d30 0x55c04e99a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f56442c2000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e99a650 0x55c04e94cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f56442c2000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e94d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f56442c2000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c04e816110 con 0x55c04e94d500 2023-11-28T18:28:36.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f56442c2000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c04e7a69a0 con 0x55c04e99a650 2023-11-28T18:28:36.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f56442c2000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c04e7a3ed0 con 0x55c04e999d30 2023-11-28T18:28:36.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f563ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e94d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f563ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e94d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51690/0 (socket says 172.21.15.105:51690) 2023-11-28T18:28:36.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f563ad9d640 1 -- 172.21.15.105:0/2763344312 learned_addr learned my addr 172.21.15.105:0/2763344312 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:36.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f563a59c640 1 --2- 172.21.15.105:0/2763344312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e999d30 0x55c04e99a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f5631d9b640 1 --2- 172.21.15.105:0/2763344312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e99a650 0x55c04e94cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f563ad9d640 1 -- 172.21.15.105:0/2763344312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e999d30 msgr2=0x55c04e99a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:36.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f563ad9d640 1 --2- 172.21.15.105:0/2763344312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e999d30 0x55c04e99a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f563ad9d640 1 -- 172.21.15.105:0/2763344312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e99a650 msgr2=0x55c04e94cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:36.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f563ad9d640 1 --2- 172.21.15.105:0/2763344312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e99a650 0x55c04e94cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f563ad9d640 1 -- 172.21.15.105:0/2763344312 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c04e497890 con 0x55c04e94d500 2023-11-28T18:28:36.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f563a59c640 1 --2- 172.21.15.105:0/2763344312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e999d30 0x55c04e99a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:36.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f563ad9d640 1 --2- 172.21.15.105:0/2763344312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e94d8e0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f56340f0640 tx=0x7f56340a5930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5851d7176dbc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:36.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f561bfff640 1 -- 172.21.15.105:0/2763344312 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56340f89b0 con 0x55c04e94d500 2023-11-28T18:28:36.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f561bfff640 1 -- 172.21.15.105:0/2763344312 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563411f040 con 0x55c04e94d500 2023-11-28T18:28:36.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f561bfff640 1 -- 172.21.15.105:0/2763344312 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56340ecdc0 con 0x55c04e94d500 2023-11-28T18:28:36.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f56442c2000 1 -- 172.21.15.105:0/2763344312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 msgr2=0x55c04e94d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:36.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f56442c2000 1 --2- 172.21.15.105:0/2763344312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e94d8e0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f56340f0640 tx=0x7f56340a5930 comp rx=0 tx=0).stop 2023-11-28T18:28:36.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f56442c2000 1 -- 172.21.15.105:0/2763344312 shutdown_connections 2023-11-28T18:28:36.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f56442c2000 1 --2- 172.21.15.105:0/2763344312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e94d8e0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f56442c2000 1 --2- 172.21.15.105:0/2763344312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e99a650 0x55c04e94cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f56442c2000 1 --2- 172.21.15.105:0/2763344312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e999d30 0x55c04e99a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f56442c2000 1 -- 172.21.15.105:0/2763344312 >> 172.21.15.105:0/2763344312 conn(0x55c04e97a8c0 msgr2=0x55c04e97ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:36.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f56442c2000 1 -- 172.21.15.105:0/2763344312 shutdown_connections 2023-11-28T18:28:36.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.850+0000 7f56442c2000 1 -- 172.21.15.105:0/2763344312 wait complete. 2023-11-28T18:28:36.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56442c2000 1 -- start start 2023-11-28T18:28:36.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56442c2000 1 Processor -- start 2023-11-28T18:28:36.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56442c2000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e9534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56442c2000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e999d30 0x55c04e953a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56442c2000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e99a650 0x55c04e953f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56442c2000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c04e79f7c0 con 0x55c04e94d500 2023-11-28T18:28:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56442c2000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c04e757bf0 con 0x55c04e999d30 2023-11-28T18:28:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56442c2000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c04e75da90 con 0x55c04e99a650 2023-11-28T18:28:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56442c2000 10 client.0 _ll_register_callbacks cb 0x55c04e93ae20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:28:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f563a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e9534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f563a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e9534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51704/0 (socket says 172.21.15.105:51704) 2023-11-28T18:28:36.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f563a59c640 1 -- 172.21.15.105:0/3636466739 learned_addr learned my addr 172.21.15.105:0/3636466739 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:36.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f563ad9d640 1 --2- 172.21.15.105:0/3636466739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e99a650 0x55c04e953f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f5631d9b640 1 --2- 172.21.15.105:0/3636466739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e999d30 0x55c04e953a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f563a59c640 1 -- 172.21.15.105:0/3636466739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e99a650 msgr2=0x55c04e953f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:36.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f563a59c640 1 --2- 172.21.15.105:0/3636466739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e99a650 0x55c04e953f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f563a59c640 1 -- 172.21.15.105:0/3636466739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e999d30 msgr2=0x55c04e953a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:36.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f563a59c640 1 --2- 172.21.15.105:0/3636466739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e999d30 0x55c04e953a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f563a59c640 1 -- 172.21.15.105:0/3636466739 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563c1574f0 con 0x55c04e94d500 2023-11-28T18:28:36.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f563ad9d640 1 --2- 172.21.15.105:0/3636466739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e99a650 0x55c04e953f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:36.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f5631d9b640 1 --2- 172.21.15.105:0/3636466739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e999d30 0x55c04e953a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:36.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f563a59c640 1 --2- 172.21.15.105:0/3636466739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e9534f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f5624005ef0 tx=0x7f5624000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d66ea9b49e08c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:36.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56197fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:28:36.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56197fa640 1 -- 172.21.15.105:0/3636466739 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5624007560 con 0x55c04e94d500 2023-11-28T18:28:36.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56197fa640 1 -- 172.21.15.105:0/3636466739 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f56240076c0 con 0x55c04e94d500 2023-11-28T18:28:36.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56442c2000 1 -- 172.21.15.105:0/3636466739 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c04e761470 con 0x55c04e94d500 2023-11-28T18:28:36.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56197fa640 1 -- 172.21.15.105:0/3636466739 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5624013420 con 0x55c04e94d500 2023-11-28T18:28:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56442c2000 10 client.7410 fetch_fsmap learned FSMap version 76 2023-11-28T18:28:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56442c2000 1 -- 172.21.15.105:0/3636466739 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55c04e760190 con 0x55c04e94d500 2023-11-28T18:28:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56197fa640 1 -- 172.21.15.105:0/3636466739 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c04e761470 con 0x55c04e94d500 2023-11-28T18:28:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56197fa640 1 -- 172.21.15.105:0/3636466739 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 76) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f5624013580 con 0x55c04e94d500 2023-11-28T18:28:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56442c2000 10 client.7410 fetch_fsmap finished waiting for FSMap version 76 2023-11-28T18:28:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56442c2000 10 client.7410 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T18:28:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56442c2000 20 client.7410 populate_metadata read hostname 'smithi105' 2023-11-28T18:28:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56442c2000 1 --2- 172.21.15.105:0/3636466739 >> [v2:172.21.15.148:6832/759522673,v1:172.21.15.148:6834/759522673] conn(0x55c04e954980 0x55c04e9abff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56442c2000 4 client.7410 mds_command: new command op to 6859 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:28:36.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f56442c2000 1 -- 172.21.15.105:0/3636466739 --> [v2:172.21.15.148:6832/759522673,v1:172.21.15.148:6834/759522673] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c04e764b50 con 0x55c04e954980 2023-11-28T18:28:36.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.854+0000 7f5631d9b640 1 --2- 172.21.15.105:0/3636466739 >> [v2:172.21.15.148:6832/759522673,v1:172.21.15.148:6834/759522673] conn(0x55c04e954980 0x55c04e9abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.858+0000 7f56197fa640 10 client.7410 ms_handle_connect on v2:172.21.15.148:6832/759522673 2023-11-28T18:28:36.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.858+0000 7f5631d9b640 1 --2- 172.21.15.105:0/3636466739 >> [v2:172.21.15.148:6832/759522673,v1:172.21.15.148:6834/759522673] conn(0x55c04e954980 0x55c04e9abff0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60ebd1f4b09c79d6 server_cookie=e1c086fec6befac4 in_seq=0 out_seq=0 2023-11-28T18:28:36.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.858+0000 7f56197fa640 1 -- 172.21.15.105:0/3636466739 <== mds.2 v2:172.21.15.148:6832/759522673 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c04e764b50 con 0x55c04e954980 2023-11-28T18:28:36.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.858+0000 7f56197fa640 10 client.7410 handle_command_reply: tid=0 2023-11-28T18:28:36.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.858+0000 7f56442c2000 1 client.7410 shutdown 2023-11-28T18:28:36.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.858+0000 7f56442c2000 1 -- 172.21.15.105:0/3636466739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 msgr2=0x55c04e9534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:36.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.858+0000 7f56442c2000 1 --2- 172.21.15.105:0/3636466739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e9534f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f5624005ef0 tx=0x7f5624000ee0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.858+0000 7f56442c2000 1 -- 172.21.15.105:0/3636466739 shutdown_connections 2023-11-28T18:28:36.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.858+0000 7f56442c2000 1 --2- 172.21.15.105:0/3636466739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e99a650 0x55c04e953f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.858+0000 7f56442c2000 1 --2- 172.21.15.105:0/3636466739 >> [v2:172.21.15.148:6832/759522673,v1:172.21.15.148:6834/759522673] conn(0x55c04e954980 0x55c04e9abff0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.858+0000 7f56442c2000 1 --2- 172.21.15.105:0/3636466739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e999d30 0x55c04e953a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.858+0000 7f56442c2000 1 --2- 172.21.15.105:0/3636466739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e9534f0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.858+0000 7f56197fa640 0 client.7410 ms_handle_reset on v2:172.21.15.148:6832/759522673 2023-11-28T18:28:36.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.858+0000 7f56442c2000 1 -- 172.21.15.105:0/3636466739 >> 172.21.15.105:0/3636466739 conn(0x55c04e97a8c0 msgr2=0x55c04e94c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:36.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.858+0000 7f56442c2000 1 -- 172.21.15.105:0/3636466739 shutdown_connections 2023-11-28T18:28:36.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.858+0000 7f56442c2000 1 -- 172.21.15.105:0/3636466739 wait complete. 2023-11-28T18:28:36.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.858+0000 7f56442c2000 20 client.7410 trim_cache size 0 max 16384 2023-11-28T18:28:36.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f56442c2000 1 Processor -- start 2023-11-28T18:28:36.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f56442c2000 1 -- start start 2023-11-28T18:28:36.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f56442c2000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e9793a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f56442c2000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e999d30 0x55c04e9798e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f56442c2000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e99a650 0x55c04e9abff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f56442c2000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c04e458940 con 0x55c04e94d500 2023-11-28T18:28:36.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f56442c2000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c04e947430 con 0x55c04e99a650 2023-11-28T18:28:36.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f56442c2000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c04e799ac0 con 0x55c04e999d30 2023-11-28T18:28:36.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f563a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e9793a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f563a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e9793a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51716/0 (socket says 172.21.15.105:51716) 2023-11-28T18:28:36.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f563a59c640 1 -- 172.21.15.105:0/3474263887 learned_addr learned my addr 172.21.15.105:0/3474263887 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:36.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f563ad9d640 1 --2- 172.21.15.105:0/3474263887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e99a650 0x55c04e9abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f5631d9b640 1 --2- 172.21.15.105:0/3474263887 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e999d30 0x55c04e9798e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f563a59c640 1 -- 172.21.15.105:0/3474263887 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e999d30 msgr2=0x55c04e9798e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:36.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f563a59c640 1 --2- 172.21.15.105:0/3474263887 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e999d30 0x55c04e9798e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f563a59c640 1 -- 172.21.15.105:0/3474263887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e99a650 msgr2=0x55c04e9abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:36.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f563a59c640 1 --2- 172.21.15.105:0/3474263887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e99a650 0x55c04e9abff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f563a59c640 1 -- 172.21.15.105:0/3474263887 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c04e94d370 con 0x55c04e94d500 2023-11-28T18:28:36.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f563ad9d640 1 --2- 172.21.15.105:0/3474263887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e99a650 0x55c04e9abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:36.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f5631d9b640 1 --2- 172.21.15.105:0/3474263887 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e999d30 0x55c04e9798e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:36.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f563a59c640 1 --2- 172.21.15.105:0/3474263887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e9793a0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5624005fd0 tx=0x7f562401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4c344de2cd7363 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:36.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f5630d99640 1 -- 172.21.15.105:0/3474263887 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562400eb80 con 0x55c04e94d500 2023-11-28T18:28:36.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f5630d99640 1 -- 172.21.15.105:0/3474263887 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5624013da0 con 0x55c04e94d500 2023-11-28T18:28:36.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f5630d99640 1 -- 172.21.15.105:0/3474263887 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56240139f0 con 0x55c04e94d500 2023-11-28T18:28:36.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f56442c2000 1 -- 172.21.15.105:0/3474263887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 msgr2=0x55c04e9793a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:36.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f56442c2000 1 --2- 172.21.15.105:0/3474263887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e9793a0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5624005fd0 tx=0x7f562401f000 comp rx=0 tx=0).stop 2023-11-28T18:28:36.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f56442c2000 1 -- 172.21.15.105:0/3474263887 shutdown_connections 2023-11-28T18:28:36.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f56442c2000 1 --2- 172.21.15.105:0/3474263887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e99a650 0x55c04e9abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f56442c2000 1 --2- 172.21.15.105:0/3474263887 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e999d30 0x55c04e9798e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f56442c2000 1 --2- 172.21.15.105:0/3474263887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e9793a0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f56442c2000 1 -- 172.21.15.105:0/3474263887 >> 172.21.15.105:0/3474263887 conn(0x55c04e97a8c0 msgr2=0x55c04e94b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:36.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f56442c2000 1 -- 172.21.15.105:0/3474263887 shutdown_connections 2023-11-28T18:28:36.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.874+0000 7f56442c2000 1 -- 172.21.15.105:0/3474263887 wait complete. 2023-11-28T18:28:36.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56442c2000 1 -- start start 2023-11-28T18:28:36.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56442c2000 1 Processor -- start 2023-11-28T18:28:36.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56442c2000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e9fa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56442c2000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e999d30 0x55c04e9fad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56442c2000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e99a650 0x55c04e9b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56442c2000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c04e950de0 con 0x55c04e94d500 2023-11-28T18:28:36.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56442c2000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c04e79cfd0 con 0x55c04e99a650 2023-11-28T18:28:36.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56442c2000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c04e7692e0 con 0x55c04e999d30 2023-11-28T18:28:36.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56442c2000 10 client.0 _ll_register_callbacks cb 0x55c04e816060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:28:36.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f563a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e9fa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f563a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e9fa810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51730/0 (socket says 172.21.15.105:51730) 2023-11-28T18:28:36.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f563a59c640 1 -- 172.21.15.105:0/3660991949 learned_addr learned my addr 172.21.15.105:0/3660991949 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:36.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f5631d9b640 1 --2- 172.21.15.105:0/3660991949 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e999d30 0x55c04e9fad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f563ad9d640 1 --2- 172.21.15.105:0/3660991949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e99a650 0x55c04e9b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f563a59c640 1 -- 172.21.15.105:0/3660991949 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e999d30 msgr2=0x55c04e9fad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:36.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f563a59c640 1 --2- 172.21.15.105:0/3660991949 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e999d30 0x55c04e9fad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f563a59c640 1 -- 172.21.15.105:0/3660991949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e99a650 msgr2=0x55c04e9b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:36.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f563a59c640 1 --2- 172.21.15.105:0/3660991949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e99a650 0x55c04e9b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f563a59c640 1 -- 172.21.15.105:0/3660991949 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c04e760190 con 0x55c04e94d500 2023-11-28T18:28:36.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f563ad9d640 1 --2- 172.21.15.105:0/3660991949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e99a650 0x55c04e9b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:36.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f563a59c640 1 --2- 172.21.15.105:0/3660991949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e9fa810 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5624001560 tx=0x7f5624007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e00431d0e6d4885 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:36.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56197fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:28:36.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56197fa640 1 -- 172.21.15.105:0/3660991949 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5624013c00 con 0x55c04e94d500 2023-11-28T18:28:36.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56197fa640 1 -- 172.21.15.105:0/3660991949 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5624013420 con 0x55c04e94d500 2023-11-28T18:28:36.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56442c2000 1 -- 172.21.15.105:0/3660991949 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c04e79e110 con 0x55c04e94d500 2023-11-28T18:28:36.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56197fa640 1 -- 172.21.15.105:0/3660991949 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f562400eb80 con 0x55c04e94d500 2023-11-28T18:28:36.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56442c2000 10 client.7416 fetch_fsmap learned FSMap version 76 2023-11-28T18:28:36.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56442c2000 1 -- 172.21.15.105:0/3660991949 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55c04e746490 con 0x55c04e94d500 2023-11-28T18:28:36.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56197fa640 1 -- 172.21.15.105:0/3660991949 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c04e79e110 con 0x55c04e94d500 2023-11-28T18:28:36.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56197fa640 1 -- 172.21.15.105:0/3660991949 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 76) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f5624022450 con 0x55c04e94d500 2023-11-28T18:28:36.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56442c2000 10 client.7416 fetch_fsmap finished waiting for FSMap version 76 2023-11-28T18:28:36.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56442c2000 10 client.7416 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T18:28:36.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56442c2000 20 client.7416 populate_metadata read hostname 'smithi105' 2023-11-28T18:28:36.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56442c2000 1 --2- 172.21.15.105:0/3660991949 >> [v2:172.21.15.148:6832/759522673,v1:172.21.15.148:6834/759522673] conn(0x55c04e9b6400 0x55c04e9b67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:36.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56442c2000 4 client.7416 mds_command: new command op to 6859 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:28:36.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f56442c2000 1 -- 172.21.15.105:0/3660991949 --> [v2:172.21.15.148:6832/759522673,v1:172.21.15.148:6834/759522673] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c04e7cc650 con 0x55c04e9b6400 2023-11-28T18:28:36.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.878+0000 7f5631d9b640 1 --2- 172.21.15.105:0/3660991949 >> [v2:172.21.15.148:6832/759522673,v1:172.21.15.148:6834/759522673] conn(0x55c04e9b6400 0x55c04e9b67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:36.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f56197fa640 10 client.7416 ms_handle_connect on v2:172.21.15.148:6832/759522673 2023-11-28T18:28:36.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f5631d9b640 1 --2- 172.21.15.105:0/3660991949 >> [v2:172.21.15.148:6832/759522673,v1:172.21.15.148:6834/759522673] conn(0x55c04e9b6400 0x55c04e9b67e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9c3dfd92e2e713c server_cookie=a55979bd952be013 in_seq=0 out_seq=0 2023-11-28T18:28:36.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f56197fa640 1 -- 172.21.15.105:0/3660991949 <== mds.2 v2:172.21.15.148:6832/759522673 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55c04e7cc650 con 0x55c04e9b6400 2023-11-28T18:28:36.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f56197fa640 10 client.7416 handle_command_reply: tid=0 2023-11-28T18:28:36.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f56442c2000 1 client.7416 shutdown 2023-11-28T18:28:36.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f56442c2000 1 -- 172.21.15.105:0/3660991949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 msgr2=0x55c04e9fa810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:36.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f56442c2000 1 --2- 172.21.15.105:0/3660991949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e9fa810 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5624001560 tx=0x7f5624007000 comp rx=0 tx=0).stop 2023-11-28T18:28:36.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f56442c2000 1 -- 172.21.15.105:0/3660991949 shutdown_connections 2023-11-28T18:28:36.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f56442c2000 1 --2- 172.21.15.105:0/3660991949 >> [v2:172.21.15.148:6832/759522673,v1:172.21.15.148:6834/759522673] conn(0x55c04e9b6400 0x55c04e9b67e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f56442c2000 1 --2- 172.21.15.105:0/3660991949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c04e99a650 0x55c04e9b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f56442c2000 1 --2- 172.21.15.105:0/3660991949 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c04e999d30 0x55c04e9fad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f56442c2000 1 --2- 172.21.15.105:0/3660991949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c04e94d500 0x55c04e9fa810 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f56197fa640 0 client.7416 ms_handle_reset on v2:172.21.15.148:6832/759522673 2023-11-28T18:28:36.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f56442c2000 1 -- 172.21.15.105:0/3660991949 >> 172.21.15.105:0/3660991949 conn(0x55c04e97a8c0 msgr2=0x55c04e94bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:36.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f56442c2000 1 -- 172.21.15.105:0/3660991949 shutdown_connections 2023-11-28T18:28:36.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f56442c2000 1 -- 172.21.15.105:0/3660991949 wait complete. 2023-11-28T18:28:36.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f56442c2000 20 client.7416 trim_cache size 0 max 16384 2023-11-28T18:28:36.904 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:28:36.904 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:28:36.904 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 2, 2023-11-28T18:28:36.904 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 6859, 2023-11-28T18:28:36.904 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:28:36.904 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:28:36.904 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:28:36.904 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 1.7372534120000001, 2023-11-28T18:28:36.904 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 76, 2023-11-28T18:28:36.904 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 78, 2023-11-28T18:28:36.905 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 78, 2023-11-28T18:28:36.905 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 28.033902808000001 2023-11-28T18:28:36.905 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:28:36.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f5640f65640 1 -- 172.21.15.105:0/333453851 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f561c0668d0 msgr2=0x7f561c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:36.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f5640f65640 1 --2- 172.21.15.105:0/333453851 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f561c0668d0 0x7f561c068d90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5624005890 tx=0x7f5624009040 comp rx=0 tx=0).stop 2023-11-28T18:28:36.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f5640f65640 1 -- 172.21.15.105:0/333453851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f563c099f20 msgr2=0x7f563c09a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:36.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f5640f65640 1 --2- 172.21.15.105:0/333453851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f563c099f20 0x7f563c09a3d0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f56340b0040 tx=0x7f56340196c0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f5640f65640 1 -- 172.21.15.105:0/333453851 shutdown_connections 2023-11-28T18:28:36.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f5640f65640 1 --2- 172.21.15.105:0/333453851 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f561c0668d0 0x7f561c068d90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f5640f65640 1 --2- 172.21.15.105:0/333453851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f563c099f20 0x7f563c09a3d0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f5640f65640 1 --2- 172.21.15.105:0/333453851 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f563c15b030 0x7f563c0b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f5640f65640 1 --2- 172.21.15.105:0/333453851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f563c0b4290 0x7f563c153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:36.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f5640f65640 1 -- 172.21.15.105:0/333453851 >> 172.21.15.105:0/333453851 conn(0x7f563c0ceb90 msgr2=0x7f563c164540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:36.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f5640f65640 1 -- 172.21.15.105:0/333453851 shutdown_connections 2023-11-28T18:28:36.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:36.882+0000 7f5640f65640 1 -- 172.21.15.105:0/333453851 wait complete. 2023-11-28T18:28:36.909 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-28T18:28:37.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.042+0000 7fec7759e640 1 Processor -- start 2023-11-28T18:28:37.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.042+0000 7fec7759e640 1 -- start start 2023-11-28T18:28:37.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.042+0000 7fec7759e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec7815d1c0 0x7fec780cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.042+0000 7fec7759e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fec7806b8a0 0x7fec7806bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.042+0000 7fec7759e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fec7806c430 0x7fec780cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.042+0000 7fec7759e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec7806c8c0 con 0x7fec7806c430 2023-11-28T18:28:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.042+0000 7fec7759e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec780cb230 con 0x7fec7806b8a0 2023-11-28T18:28:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.042+0000 7fec7759e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec780b02a0 con 0x7fec7815d1c0 2023-11-28T18:28:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.046+0000 7fec76d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fec7806c430 0x7fec780cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.046+0000 7fec76d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fec7806c430 0x7fec780cd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51732/0 (socket says 172.21.15.105:51732) 2023-11-28T18:28:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.046+0000 7fec76d9d640 1 -- 172.21.15.105:0/2879758126 learned_addr learned my addr 172.21.15.105:0/2879758126 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.046+0000 7fec6dd9b640 1 --2- 172.21.15.105:0/2879758126 >> v2:172.21.15.148:3300/0 conn(0x7fec7806b8a0 0x7fec7806bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.046+0000 7fec76d9d640 1 -- 172.21.15.105:0/2879758126 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec7815d1c0 msgr2=0x7fec780cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:37.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.046+0000 7fec7659c640 1 --2- 172.21.15.105:0/2879758126 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec7815d1c0 0x7fec780cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.046+0000 7fec76d9d640 1 --2- 172.21.15.105:0/2879758126 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec7815d1c0 0x7fec780cf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.046+0000 7fec76d9d640 1 -- 172.21.15.105:0/2879758126 >> v2:172.21.15.148:3300/0 conn(0x7fec7806b8a0 msgr2=0x7fec7806bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:37.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.046+0000 7fec76d9d640 1 --2- 172.21.15.105:0/2879758126 >> v2:172.21.15.148:3300/0 conn(0x7fec7806b8a0 0x7fec7806bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.046+0000 7fec76d9d640 1 -- 172.21.15.105:0/2879758126 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec780d19a0 con 0x7fec7806c430 2023-11-28T18:28:37.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.046+0000 7fec6dd9b640 1 --2- 172.21.15.105:0/2879758126 >> v2:172.21.15.148:3300/0 conn(0x7fec7806b8a0 0x7fec7806bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:37.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.046+0000 7fec7659c640 1 --2- 172.21.15.105:0/2879758126 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec7815d1c0 0x7fec780cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:37.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.046+0000 7fec76d9d640 1 --2- 172.21.15.105:0/2879758126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7806c430 0x7fec780cd460 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fec780688f0 tx=0x7fec7001c1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734b13073e72152d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:37.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.046+0000 7fec75d9b640 1 -- 172.21.15.105:0/2879758126 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec700ab400 con 0x7fec7806c430 2023-11-28T18:28:37.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.046+0000 7fec75d9b640 1 -- 172.21.15.105:0/2879758126 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec700a7050 con 0x7fec7806c430 2023-11-28T18:28:37.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.046+0000 7fec75d9b640 1 -- 172.21.15.105:0/2879758126 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec700ab780 con 0x7fec7806c430 2023-11-28T18:28:37.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.050+0000 7fec7759e640 1 -- 172.21.15.105:0/2879758126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7806c430 msgr2=0x7fec780cd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:37.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.050+0000 7fec7759e640 1 --2- 172.21.15.105:0/2879758126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7806c430 0x7fec780cd460 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fec780688f0 tx=0x7fec7001c1b0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.050+0000 7fec7759e640 1 -- 172.21.15.105:0/2879758126 shutdown_connections 2023-11-28T18:28:37.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.050+0000 7fec7759e640 1 --2- 172.21.15.105:0/2879758126 >> v2:172.21.15.148:3300/0 conn(0x7fec7806b8a0 0x7fec7806bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.050+0000 7fec7759e640 1 --2- 172.21.15.105:0/2879758126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7806c430 0x7fec780cd460 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.050+0000 7fec7759e640 1 --2- 172.21.15.105:0/2879758126 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec7815d1c0 0x7fec780cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.050+0000 7fec7759e640 1 -- 172.21.15.105:0/2879758126 >> 172.21.15.105:0/2879758126 conn(0x7fec780d0be0 msgr2=0x7fec780d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:37.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.050+0000 7fec7759e640 1 -- 172.21.15.105:0/2879758126 shutdown_connections 2023-11-28T18:28:37.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.050+0000 7fec7759e640 1 -- 172.21.15.105:0/2879758126 wait complete. 2023-11-28T18:28:37.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.050+0000 7fec7759e640 1 Processor -- start 2023-11-28T18:28:37.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.050+0000 7fec7759e640 1 -- start start 2023-11-28T18:28:37.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.050+0000 7fec7759e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec7806b8a0 0x7fec78158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.050+0000 7fec7759e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec7806c430 0x7fec78153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.050+0000 7fec7759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7815d1c0 0x7fec780ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.050+0000 7fec7759e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec780af800 con 0x7fec7815d1c0 2023-11-28T18:28:37.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.050+0000 7fec7759e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec780c5140 con 0x7fec7806c430 2023-11-28T18:28:37.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.050+0000 7fec7759e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec781536d0 con 0x7fec7806b8a0 2023-11-28T18:28:37.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.050+0000 7fec76d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7815d1c0 0x7fec780ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.054+0000 7fec76d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7815d1c0 0x7fec780ae780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51738/0 (socket says 172.21.15.105:51738) 2023-11-28T18:28:37.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.054+0000 7fec76d9d640 1 -- 172.21.15.105:0/738360539 learned_addr learned my addr 172.21.15.105:0/738360539 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:37.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.054+0000 7fec7659c640 1 --2- 172.21.15.105:0/738360539 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec7806b8a0 0x7fec78158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.054+0000 7fec6dd9b640 1 --2- 172.21.15.105:0/738360539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec7806c430 0x7fec78153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.054+0000 7fec76d9d640 1 -- 172.21.15.105:0/738360539 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec7806b8a0 msgr2=0x7fec78158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:37.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.054+0000 7fec76d9d640 1 --2- 172.21.15.105:0/738360539 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec7806b8a0 0x7fec78158f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.054+0000 7fec76d9d640 1 -- 172.21.15.105:0/738360539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec7806c430 msgr2=0x7fec78153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:37.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.054+0000 7fec76d9d640 1 --2- 172.21.15.105:0/738360539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec7806c430 0x7fec78153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.054+0000 7fec76d9d640 1 -- 172.21.15.105:0/738360539 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec700094d0 con 0x7fec7815d1c0 2023-11-28T18:28:37.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.054+0000 7fec76d9d640 1 --2- 172.21.15.105:0/738360539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7815d1c0 0x7fec780ae780 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fec700ae040 tx=0x7fec70018f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:37.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.054+0000 7fec6f7fe640 1 -- 172.21.15.105:0/738360539 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec700a4ec0 con 0x7fec7815d1c0 2023-11-28T18:28:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.054+0000 7fec6f7fe640 1 -- 172.21.15.105:0/738360539 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec700b6470 con 0x7fec7815d1c0 2023-11-28T18:28:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.054+0000 7fec7759e640 1 -- 172.21.15.105:0/738360539 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec780c4e10 con 0x7fec7815d1c0 2023-11-28T18:28:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.054+0000 7fec6f7fe640 1 -- 172.21.15.105:0/738360539 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec700b67e0 con 0x7fec7815d1c0 2023-11-28T18:28:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.054+0000 7fec7759e640 1 -- 172.21.15.105:0/738360539 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec780aef60 con 0x7fec7815d1c0 2023-11-28T18:28:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.054+0000 7fec6f7fe640 1 -- 172.21.15.105:0/738360539 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fec700b2070 con 0x7fec7815d1c0 2023-11-28T18:28:37.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec6f7fe640 1 --2- 172.21.15.105:0/738360539 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec58066900 0x7fec58068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec6f7fe640 1 -- 172.21.15.105:0/738360539 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fec7011e8c0 con 0x7fec7815d1c0 2023-11-28T18:28:37.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec7659c640 1 --2- 172.21.15.105:0/738360539 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec58066900 0x7fec58068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec800d5000 1 Processor -- start 2023-11-28T18:28:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec7659c640 1 --2- 172.21.15.105:0/738360539 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec58066900 0x7fec58068dc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fec68006fd0 tx=0x7fec68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec800d5000 1 -- start start 2023-11-28T18:28:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec800d5000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ef4d30 0x55bdb5ef5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec800d5000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ef5650 0x55bdb5ea7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec800d5000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ea8500 0x55bdb5ea88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec800d5000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb5d71110 con 0x55bdb5ef5650 2023-11-28T18:28:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec800d5000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb5d019a0 con 0x55bdb5ea8500 2023-11-28T18:28:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec800d5000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdb5cfeed0 con 0x55bdb5ef4d30 2023-11-28T18:28:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec6dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ef5650 0x55bdb5ea7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec6dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ef5650 0x55bdb5ea7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51748/0 (socket says 172.21.15.105:51748) 2023-11-28T18:28:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec6dd9b640 1 -- 172.21.15.105:0/3730946674 learned_addr learned my addr 172.21.15.105:0/3730946674 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec76d9d640 1 --2- 172.21.15.105:0/3730946674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ea8500 0x55bdb5ea88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec7659c640 1 --2- 172.21.15.105:0/3730946674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ef4d30 0x55bdb5ef5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec6dd9b640 1 -- 172.21.15.105:0/3730946674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ef4d30 msgr2=0x55bdb5ef5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec6dd9b640 1 --2- 172.21.15.105:0/3730946674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ef4d30 0x55bdb5ef5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec6dd9b640 1 -- 172.21.15.105:0/3730946674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ea8500 msgr2=0x55bdb5ea88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:37.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec6dd9b640 1 --2- 172.21.15.105:0/3730946674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ea8500 0x55bdb5ea88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec6dd9b640 1 -- 172.21.15.105:0/3730946674 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdb59f2890 con 0x55bdb5ef5650 2023-11-28T18:28:37.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec7659c640 1 --2- 172.21.15.105:0/3730946674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ef4d30 0x55bdb5ef5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:37.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec76d9d640 1 --2- 172.21.15.105:0/3730946674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ea8500 0x55bdb5ea88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:37.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec6dd9b640 1 --2- 172.21.15.105:0/3730946674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ef5650 0x55bdb5ea7fc0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fec7806ce40 tx=0x7fec6000d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c390123240f8318a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:37.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec57fff640 1 -- 172.21.15.105:0/3730946674 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60012070 con 0x55bdb5ef5650 2023-11-28T18:28:37.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec57fff640 1 -- 172.21.15.105:0/3730946674 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6000c7b0 con 0x55bdb5ef5650 2023-11-28T18:28:37.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec57fff640 1 -- 172.21.15.105:0/3730946674 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6000cb60 con 0x55bdb5ef5650 2023-11-28T18:28:37.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec800d5000 1 -- 172.21.15.105:0/3730946674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ef5650 msgr2=0x55bdb5ea7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:37.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec800d5000 1 --2- 172.21.15.105:0/3730946674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ef5650 0x55bdb5ea7fc0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fec7806ce40 tx=0x7fec6000d9c0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec800d5000 1 -- 172.21.15.105:0/3730946674 shutdown_connections 2023-11-28T18:28:37.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec800d5000 1 --2- 172.21.15.105:0/3730946674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ea8500 0x55bdb5ea88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec800d5000 1 --2- 172.21.15.105:0/3730946674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ef5650 0x55bdb5ea7fc0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec800d5000 1 --2- 172.21.15.105:0/3730946674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ef4d30 0x55bdb5ef5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.058+0000 7fec800d5000 1 -- 172.21.15.105:0/3730946674 >> 172.21.15.105:0/3730946674 conn(0x55bdb5ed58c0 msgr2=0x55bdb5ed7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:37.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec800d5000 1 -- 172.21.15.105:0/3730946674 shutdown_connections 2023-11-28T18:28:37.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec800d5000 1 -- 172.21.15.105:0/3730946674 wait complete. 2023-11-28T18:28:37.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec800d5000 1 -- start start 2023-11-28T18:28:37.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec800d5000 1 Processor -- start 2023-11-28T18:28:37.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec800d5000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ea8500 0x55bdb5eae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec800d5000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ef4d30 0x55bdb5eaea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec800d5000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ef5650 0x55bdb5eaef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec800d5000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb5cfa7c0 con 0x55bdb5ea8500 2023-11-28T18:28:37.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec800d5000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb5cb2bf0 con 0x55bdb5ef5650 2023-11-28T18:28:37.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec800d5000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdb5cb8a90 con 0x55bdb5ef4d30 2023-11-28T18:28:37.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec800d5000 10 client.0 _ll_register_callbacks cb 0x55bdb5e95e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:28:37.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec7659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ea8500 0x55bdb5eae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec7659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ea8500 0x55bdb5eae4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51762/0 (socket says 172.21.15.105:51762) 2023-11-28T18:28:37.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec7659c640 1 -- 172.21.15.105:0/961197716 learned_addr learned my addr 172.21.15.105:0/961197716 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:37.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec76d9d640 1 --2- 172.21.15.105:0/961197716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ef5650 0x55bdb5eaef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec6dd9b640 1 --2- 172.21.15.105:0/961197716 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ef4d30 0x55bdb5eaea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec7659c640 1 -- 172.21.15.105:0/961197716 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ef4d30 msgr2=0x55bdb5eaea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:37.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec7659c640 1 --2- 172.21.15.105:0/961197716 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ef4d30 0x55bdb5eaea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec7659c640 1 -- 172.21.15.105:0/961197716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ef5650 msgr2=0x55bdb5eaef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:37.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec7659c640 1 --2- 172.21.15.105:0/961197716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ef5650 0x55bdb5eaef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec7659c640 1 -- 172.21.15.105:0/961197716 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec60007d40 con 0x55bdb5ea8500 2023-11-28T18:28:37.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec76d9d640 1 --2- 172.21.15.105:0/961197716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ef5650 0x55bdb5eaef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:37.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec6dd9b640 1 --2- 172.21.15.105:0/961197716 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ef4d30 0x55bdb5eaea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:37.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec7659c640 1 --2- 172.21.15.105:0/961197716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ea8500 0x55bdb5eae4e0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fec6800afd0 tx=0x7fec680046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59acf602ede40873 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:37.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec557fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:28:37.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec557fa640 1 -- 172.21.15.105:0/961197716 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6800fe90 con 0x55bdb5ea8500 2023-11-28T18:28:37.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec557fa640 1 -- 172.21.15.105:0/961197716 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fec6801a040 con 0x55bdb5ea8500 2023-11-28T18:28:37.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec800d5000 1 -- 172.21.15.105:0/961197716 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdb5cbc470 con 0x55bdb5ea8500 2023-11-28T18:28:37.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec557fa640 1 -- 172.21.15.105:0/961197716 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec68013b80 con 0x55bdb5ea8500 2023-11-28T18:28:37.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec800d5000 10 client.7428 fetch_fsmap learned FSMap version 76 2023-11-28T18:28:37.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec800d5000 1 -- 172.21.15.105:0/961197716 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55bdb5cbb190 con 0x55bdb5ea8500 2023-11-28T18:28:37.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.062+0000 7fec557fa640 1 -- 172.21.15.105:0/961197716 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bdb5cbc470 con 0x55bdb5ea8500 2023-11-28T18:28:37.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec557fa640 1 -- 172.21.15.105:0/961197716 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 76) v1 ==== 3318+0+0 (secure 0 0 0) 0x7fec68011040 con 0x55bdb5ea8500 2023-11-28T18:28:37.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec800d5000 10 client.7428 fetch_fsmap finished waiting for FSMap version 76 2023-11-28T18:28:37.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec800d5000 10 client.7428 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T18:28:37.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec800d5000 20 client.7428 populate_metadata read hostname 'smithi105' 2023-11-28T18:28:37.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec800d5000 1 --2- 172.21.15.105:0/961197716 >> [v2:172.21.15.148:6833/2489361300,v1:172.21.15.148:6835/2489361300] conn(0x55bdb5eaf970 0x55bdb5f06ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec800d5000 4 client.7428 mds_command: new command op to 6862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:28:37.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec800d5000 1 -- 172.21.15.105:0/961197716 --> [v2:172.21.15.148:6833/2489361300,v1:172.21.15.148:6835/2489361300] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdb5cbfb50 con 0x55bdb5eaf970 2023-11-28T18:28:37.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec6dd9b640 1 --2- 172.21.15.105:0/961197716 >> [v2:172.21.15.148:6833/2489361300,v1:172.21.15.148:6835/2489361300] conn(0x55bdb5eaf970 0x55bdb5f06ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec6dd9b640 1 --2- 172.21.15.105:0/961197716 >> [v2:172.21.15.148:6833/2489361300,v1:172.21.15.148:6835/2489361300] conn(0x55bdb5eaf970 0x55bdb5f06ff0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a721257e45ca1ce server_cookie=6187fc03c7464fcc in_seq=0 out_seq=0 2023-11-28T18:28:37.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec557fa640 10 client.7428 ms_handle_connect on v2:172.21.15.148:6833/2489361300 2023-11-28T18:28:37.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec557fa640 1 -- 172.21.15.105:0/961197716 <== mds.1 v2:172.21.15.148:6833/2489361300 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bdb5cbfb50 con 0x55bdb5eaf970 2023-11-28T18:28:37.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec557fa640 10 client.7428 handle_command_reply: tid=0 2023-11-28T18:28:37.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec800d5000 1 client.7428 shutdown 2023-11-28T18:28:37.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec800d5000 1 -- 172.21.15.105:0/961197716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ea8500 msgr2=0x55bdb5eae4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:37.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec800d5000 1 --2- 172.21.15.105:0/961197716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ea8500 0x55bdb5eae4e0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fec6800afd0 tx=0x7fec680046d0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec800d5000 1 -- 172.21.15.105:0/961197716 shutdown_connections 2023-11-28T18:28:37.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec800d5000 1 --2- 172.21.15.105:0/961197716 >> [v2:172.21.15.148:6833/2489361300,v1:172.21.15.148:6835/2489361300] conn(0x55bdb5eaf970 0x55bdb5f06ff0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec800d5000 1 --2- 172.21.15.105:0/961197716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ef5650 0x55bdb5eaef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec800d5000 1 --2- 172.21.15.105:0/961197716 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ef4d30 0x55bdb5eaea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec800d5000 1 --2- 172.21.15.105:0/961197716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ea8500 0x55bdb5eae4e0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec557fa640 0 client.7428 ms_handle_reset on v2:172.21.15.148:6833/2489361300 2023-11-28T18:28:37.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec800d5000 1 -- 172.21.15.105:0/961197716 >> 172.21.15.105:0/961197716 conn(0x55bdb5ed58c0 msgr2=0x55bdb5ea76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:37.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec800d5000 1 -- 172.21.15.105:0/961197716 shutdown_connections 2023-11-28T18:28:37.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec800d5000 1 -- 172.21.15.105:0/961197716 wait complete. 2023-11-28T18:28:37.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.066+0000 7fec800d5000 20 client.7428 trim_cache size 0 max 16384 2023-11-28T18:28:37.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec800d5000 1 Processor -- start 2023-11-28T18:28:37.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec800d5000 1 -- start start 2023-11-28T18:28:37.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec800d5000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ea8500 0x55bdb5ed43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec800d5000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ef4d30 0x55bdb5ed48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec800d5000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ef5650 0x55bdb5f06ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec800d5000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb59b3940 con 0x55bdb5ef5650 2023-11-28T18:28:37.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec800d5000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb5cf4ca0 con 0x55bdb5ef4d30 2023-11-28T18:28:37.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec800d5000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdb5ccfb70 con 0x55bdb5ea8500 2023-11-28T18:28:37.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec76d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ef5650 0x55bdb5f06ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec76d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ef5650 0x55bdb5f06ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51776/0 (socket says 172.21.15.105:51776) 2023-11-28T18:28:37.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec76d9d640 1 -- 172.21.15.105:0/2720885431 learned_addr learned my addr 172.21.15.105:0/2720885431 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:37.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec6dd9b640 1 --2- 172.21.15.105:0/2720885431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ef4d30 0x55bdb5ed48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec7659c640 1 --2- 172.21.15.105:0/2720885431 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ea8500 0x55bdb5ed43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec76d9d640 1 -- 172.21.15.105:0/2720885431 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ea8500 msgr2=0x55bdb5ed43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:37.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec76d9d640 1 --2- 172.21.15.105:0/2720885431 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ea8500 0x55bdb5ed43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec76d9d640 1 -- 172.21.15.105:0/2720885431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ef4d30 msgr2=0x55bdb5ed48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:37.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec76d9d640 1 --2- 172.21.15.105:0/2720885431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ef4d30 0x55bdb5ed48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec76d9d640 1 -- 172.21.15.105:0/2720885431 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdb5ea8370 con 0x55bdb5ef5650 2023-11-28T18:28:37.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec7659c640 1 --2- 172.21.15.105:0/2720885431 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ea8500 0x55bdb5ed43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:37.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec76d9d640 1 --2- 172.21.15.105:0/2720885431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ef5650 0x55bdb5f06ff0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fec7001c690 tx=0x7fec700ef1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f4558f4e6b3827 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:37.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec6cd99640 1 -- 172.21.15.105:0/2720885431 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec700edb50 con 0x55bdb5ef5650 2023-11-28T18:28:37.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec6cd99640 1 -- 172.21.15.105:0/2720885431 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec700ebec0 con 0x55bdb5ef5650 2023-11-28T18:28:37.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.082+0000 7fec6cd99640 1 -- 172.21.15.105:0/2720885431 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec700eeee0 con 0x55bdb5ef5650 2023-11-28T18:28:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec800d5000 1 -- 172.21.15.105:0/2720885431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ef5650 msgr2=0x55bdb5f06ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec800d5000 1 --2- 172.21.15.105:0/2720885431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ef5650 0x55bdb5f06ff0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fec7001c690 tx=0x7fec700ef1d0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec800d5000 1 -- 172.21.15.105:0/2720885431 shutdown_connections 2023-11-28T18:28:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec800d5000 1 --2- 172.21.15.105:0/2720885431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ef5650 0x55bdb5f06ff0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec800d5000 1 --2- 172.21.15.105:0/2720885431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ef4d30 0x55bdb5ed48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec800d5000 1 --2- 172.21.15.105:0/2720885431 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ea8500 0x55bdb5ed43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec800d5000 1 -- 172.21.15.105:0/2720885431 >> 172.21.15.105:0/2720885431 conn(0x55bdb5ed58c0 msgr2=0x55bdb5ea6720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec800d5000 1 -- 172.21.15.105:0/2720885431 shutdown_connections 2023-11-28T18:28:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec800d5000 1 -- 172.21.15.105:0/2720885431 wait complete. 2023-11-28T18:28:37.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec800d5000 1 -- start start 2023-11-28T18:28:37.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec800d5000 1 Processor -- start 2023-11-28T18:28:37.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec800d5000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ea8500 0x55bdb5f5e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec800d5000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ef4d30 0x55bdb5f5e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec800d5000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ef5650 0x55bdb5f19280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec800d5000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb5ea2430 con 0x55bdb5ea8500 2023-11-28T18:28:37.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec800d5000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb5cf7fd0 con 0x55bdb5ef4d30 2023-11-28T18:28:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec800d5000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdb5cfd320 con 0x55bdb5ef5650 2023-11-28T18:28:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec800d5000 10 client.0 _ll_register_callbacks cb 0x55bdb5d71060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:28:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec7659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ea8500 0x55bdb5f5e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec7659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ea8500 0x55bdb5f5e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51788/0 (socket says 172.21.15.105:51788) 2023-11-28T18:28:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec7659c640 1 -- 172.21.15.105:0/457188359 learned_addr learned my addr 172.21.15.105:0/457188359 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec76d9d640 1 --2- 172.21.15.105:0/457188359 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ef5650 0x55bdb5f19280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec6dd9b640 1 --2- 172.21.15.105:0/457188359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ef4d30 0x55bdb5f5e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec7659c640 1 -- 172.21.15.105:0/457188359 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ef5650 msgr2=0x55bdb5f19280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec7659c640 1 --2- 172.21.15.105:0/457188359 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ef5650 0x55bdb5f19280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec7659c640 1 -- 172.21.15.105:0/457188359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ef4d30 msgr2=0x55bdb5f5e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:37.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec7659c640 1 --2- 172.21.15.105:0/457188359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ef4d30 0x55bdb5f5e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec7659c640 1 -- 172.21.15.105:0/457188359 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec780aef60 con 0x55bdb5ea8500 2023-11-28T18:28:37.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec6dd9b640 1 --2- 172.21.15.105:0/457188359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ef4d30 0x55bdb5f5e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:37.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec7659c640 1 --2- 172.21.15.105:0/457188359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ea8500 0x55bdb5f5e020 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fec68029ca0 tx=0x7fec68017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c611919a3ea50c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:37.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec557fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:28:37.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec557fa640 1 -- 172.21.15.105:0/457188359 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68013b80 con 0x55bdb5ea8500 2023-11-28T18:28:37.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec557fa640 1 -- 172.21.15.105:0/457188359 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fec6803a070 con 0x55bdb5ea8500 2023-11-28T18:28:37.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec800d5000 1 -- 172.21.15.105:0/457188359 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdb5cf9110 con 0x55bdb5ea8500 2023-11-28T18:28:37.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec557fa640 1 -- 172.21.15.105:0/457188359 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec680137f0 con 0x55bdb5ea8500 2023-11-28T18:28:37.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec800d5000 10 client.7434 fetch_fsmap learned FSMap version 76 2023-11-28T18:28:37.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec800d5000 1 -- 172.21.15.105:0/457188359 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55bdb5ca1490 con 0x55bdb5ea8500 2023-11-28T18:28:37.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.086+0000 7fec557fa640 1 -- 172.21.15.105:0/457188359 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bdb5cf9110 con 0x55bdb5ea8500 2023-11-28T18:28:37.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec557fa640 1 -- 172.21.15.105:0/457188359 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 76) v1 ==== 3318+0+0 (secure 0 0 0) 0x7fec6803f480 con 0x55bdb5ea8500 2023-11-28T18:28:37.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec800d5000 10 client.7434 fetch_fsmap finished waiting for FSMap version 76 2023-11-28T18:28:37.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec800d5000 10 client.7434 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T18:28:37.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec800d5000 20 client.7434 populate_metadata read hostname 'smithi105' 2023-11-28T18:28:37.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec800d5000 1 --2- 172.21.15.105:0/457188359 >> [v2:172.21.15.148:6833/2489361300,v1:172.21.15.148:6835/2489361300] conn(0x55bdb5f64e80 0x55bdb5f67270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec800d5000 4 client.7434 mds_command: new command op to 6862 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:28:37.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec800d5000 1 -- 172.21.15.105:0/457188359 --> [v2:172.21.15.148:6833/2489361300,v1:172.21.15.148:6835/2489361300] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bdb5d27650 con 0x55bdb5f64e80 2023-11-28T18:28:37.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec6dd9b640 1 --2- 172.21.15.105:0/457188359 >> [v2:172.21.15.148:6833/2489361300,v1:172.21.15.148:6835/2489361300] conn(0x55bdb5f64e80 0x55bdb5f67270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec6dd9b640 1 --2- 172.21.15.105:0/457188359 >> [v2:172.21.15.148:6833/2489361300,v1:172.21.15.148:6835/2489361300] conn(0x55bdb5f64e80 0x55bdb5f67270 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=542d0df5632c0668 server_cookie=455264cf33685e8a in_seq=0 out_seq=0 2023-11-28T18:28:37.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec557fa640 10 client.7434 ms_handle_connect on v2:172.21.15.148:6833/2489361300 2023-11-28T18:28:37.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec557fa640 1 -- 172.21.15.105:0/457188359 <== mds.1 v2:172.21.15.148:6833/2489361300 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55bdb5d27650 con 0x55bdb5f64e80 2023-11-28T18:28:37.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec557fa640 10 client.7434 handle_command_reply: tid=0 2023-11-28T18:28:37.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec800d5000 1 client.7434 shutdown 2023-11-28T18:28:37.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec800d5000 1 -- 172.21.15.105:0/457188359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ea8500 msgr2=0x55bdb5f5e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:37.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec800d5000 1 --2- 172.21.15.105:0/457188359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ea8500 0x55bdb5f5e020 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fec68029ca0 tx=0x7fec68017000 comp rx=0 tx=0).stop 2023-11-28T18:28:37.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec800d5000 1 -- 172.21.15.105:0/457188359 shutdown_connections 2023-11-28T18:28:37.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec800d5000 1 --2- 172.21.15.105:0/457188359 >> [v2:172.21.15.148:6833/2489361300,v1:172.21.15.148:6835/2489361300] conn(0x55bdb5f64e80 0x55bdb5f67270 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec800d5000 1 --2- 172.21.15.105:0/457188359 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55bdb5ef5650 0x55bdb5f19280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec800d5000 1 --2- 172.21.15.105:0/457188359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bdb5ef4d30 0x55bdb5f5e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec800d5000 1 --2- 172.21.15.105:0/457188359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bdb5ea8500 0x55bdb5f5e020 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec557fa640 0 client.7434 ms_handle_reset on v2:172.21.15.148:6833/2489361300 2023-11-28T18:28:37.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec800d5000 1 -- 172.21.15.105:0/457188359 >> 172.21.15.105:0/457188359 conn(0x55bdb5ed58c0 msgr2=0x55bdb5ea6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:37.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec800d5000 1 -- 172.21.15.105:0/457188359 shutdown_connections 2023-11-28T18:28:37.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec800d5000 1 -- 172.21.15.105:0/457188359 wait complete. 2023-11-28T18:28:37.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec800d5000 20 client.7434 trim_cache size 0 max 16384 2023-11-28T18:28:37.107 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:28:37.107 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:28:37.107 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 1, 2023-11-28T18:28:37.107 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 6862, 2023-11-28T18:28:37.107 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:28:37.108 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:28:37.108 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:28:37.108 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 2.9565961199999999, 2023-11-28T18:28:37.108 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 76, 2023-11-28T18:28:37.108 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 78, 2023-11-28T18:28:37.108 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 78, 2023-11-28T18:28:37.108 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 28.242245262000001 2023-11-28T18:28:37.109 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:28:37.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec7759e640 1 -- 172.21.15.105:0/738360539 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec58066900 msgr2=0x7fec58068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:37.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec7759e640 1 --2- 172.21.15.105:0/738360539 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec58066900 0x7fec58068dc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fec68006fd0 tx=0x7fec68008040 comp rx=0 tx=0).stop 2023-11-28T18:28:37.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec7759e640 1 -- 172.21.15.105:0/738360539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7815d1c0 msgr2=0x7fec780ae780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:37.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec7759e640 1 --2- 172.21.15.105:0/738360539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7815d1c0 0x7fec780ae780 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fec700ae040 tx=0x7fec70018f50 comp rx=0 tx=0).stop 2023-11-28T18:28:37.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec7759e640 1 -- 172.21.15.105:0/738360539 shutdown_connections 2023-11-28T18:28:37.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec7759e640 1 --2- 172.21.15.105:0/738360539 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec58066900 0x7fec58068dc0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec7759e640 1 --2- 172.21.15.105:0/738360539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec7815d1c0 0x7fec780ae780 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec7759e640 1 --2- 172.21.15.105:0/738360539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec7806c430 0x7fec78153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec7759e640 1 --2- 172.21.15.105:0/738360539 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec7806b8a0 0x7fec78158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec7759e640 1 -- 172.21.15.105:0/738360539 >> 172.21.15.105:0/738360539 conn(0x7fec780d0be0 msgr2=0x7fec78072fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:37.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec7759e640 1 -- 172.21.15.105:0/738360539 shutdown_connections 2023-11-28T18:28:37.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.090+0000 7fec7759e640 1 -- 172.21.15.105:0/738360539 wait complete. 2023-11-28T18:28:37.113 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T18:28:37.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.250+0000 7fd389679640 1 Processor -- start 2023-11-28T18:28:37.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.250+0000 7fd389679640 1 -- start start 2023-11-28T18:28:37.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.254+0000 7fd389679640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd384159f40 0x7fd3840caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.254+0000 7fd389679640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd3840cc760 0x7fd3840ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.254+0000 7fd389679640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd3840ad680 0x7fd38415da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.254+0000 7fd389679640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38406bf50 con 0x7fd3840ad680 2023-11-28T18:28:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.254+0000 7fd389679640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3840ccbc0 con 0x7fd3840cc760 2023-11-28T18:28:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.254+0000 7fd389679640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38415a320 con 0x7fd384159f40 2023-11-28T18:28:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.254+0000 7fd3837fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd3840ad680 0x7fd38415da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.254+0000 7fd37bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd3840cc760 0x7fd3840ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.254+0000 7fd3837fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd3840ad680 0x7fd38415da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51792/0 (socket says 172.21.15.105:51792) 2023-11-28T18:28:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.254+0000 7fd3837fe640 1 -- 172.21.15.105:0/220540670 learned_addr learned my addr 172.21.15.105:0/220540670 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.254+0000 7fd382ffd640 1 --2- 172.21.15.105:0/220540670 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd384159f40 0x7fd3840caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.254+0000 7fd3837fe640 1 -- 172.21.15.105:0/220540670 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd384159f40 msgr2=0x7fd3840caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.254+0000 7fd3837fe640 1 --2- 172.21.15.105:0/220540670 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd384159f40 0x7fd3840caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.254+0000 7fd3837fe640 1 -- 172.21.15.105:0/220540670 >> v2:172.21.15.148:3300/0 conn(0x7fd3840cc760 msgr2=0x7fd3840ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.254+0000 7fd3837fe640 1 --2- 172.21.15.105:0/220540670 >> v2:172.21.15.148:3300/0 conn(0x7fd3840cc760 0x7fd3840ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.254+0000 7fd3837fe640 1 -- 172.21.15.105:0/220540670 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd384071380 con 0x7fd3840ad680 2023-11-28T18:28:37.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.258+0000 7fd3837fe640 1 --2- 172.21.15.105:0/220540670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3840ad680 0x7fd38415da20 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fd3840cfc90 tx=0x7fd37c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63be82d552067f20 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:37.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.258+0000 7fd3827fc640 1 -- 172.21.15.105:0/220540670 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0a5790 con 0x7fd3840ad680 2023-11-28T18:28:37.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.258+0000 7fd3827fc640 1 -- 172.21.15.105:0/220540670 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c0a58f0 con 0x7fd3840ad680 2023-11-28T18:28:37.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.258+0000 7fd3827fc640 1 -- 172.21.15.105:0/220540670 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0a5c80 con 0x7fd3840ad680 2023-11-28T18:28:37.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.258+0000 7fd389679640 1 -- 172.21.15.105:0/220540670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3840ad680 msgr2=0x7fd38415da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:37.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.258+0000 7fd389679640 1 --2- 172.21.15.105:0/220540670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3840ad680 0x7fd38415da20 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fd3840cfc90 tx=0x7fd37c01c910 comp rx=0 tx=0).stop 2023-11-28T18:28:37.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.258+0000 7fd389679640 1 -- 172.21.15.105:0/220540670 shutdown_connections 2023-11-28T18:28:37.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.258+0000 7fd389679640 1 --2- 172.21.15.105:0/220540670 >> v2:172.21.15.148:3300/0 conn(0x7fd3840cc760 0x7fd3840ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.258+0000 7fd389679640 1 --2- 172.21.15.105:0/220540670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3840ad680 0x7fd38415da20 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.258+0000 7fd389679640 1 --2- 172.21.15.105:0/220540670 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd384159f40 0x7fd3840caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.258+0000 7fd389679640 1 -- 172.21.15.105:0/220540670 >> 172.21.15.105:0/220540670 conn(0x7fd3840ceb90 msgr2=0x7fd3840cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:37.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.258+0000 7fd389679640 1 -- 172.21.15.105:0/220540670 shutdown_connections 2023-11-28T18:28:37.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.258+0000 7fd389679640 1 -- 172.21.15.105:0/220540670 wait complete. 2023-11-28T18:28:37.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.258+0000 7fd389679640 1 Processor -- start 2023-11-28T18:28:37.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.258+0000 7fd389679640 1 -- start start 2023-11-28T18:28:37.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.262+0000 7fd389679640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3840ad680 0x7fd384157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.262+0000 7fd389679640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3840cc760 0x7fd38414d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.262+0000 7fd389679640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd384159f40 0x7fd384153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.262+0000 7fd389679640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38414d490 con 0x7fd3840cc760 2023-11-28T18:28:37.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.262+0000 7fd389679640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38414ddd0 con 0x7fd3840ad680 2023-11-28T18:28:37.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.262+0000 7fd389679640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd384159420 con 0x7fd384159f40 2023-11-28T18:28:37.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.262+0000 7fd37bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3840cc760 0x7fd38414d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.262+0000 7fd37bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3840cc760 0x7fd38414d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51798/0 (socket says 172.21.15.105:51798) 2023-11-28T18:28:37.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.262+0000 7fd37bfff640 1 -- 172.21.15.105:0/2602772957 learned_addr learned my addr 172.21.15.105:0/2602772957 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:37.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.262+0000 7fd382ffd640 1 --2- 172.21.15.105:0/2602772957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3840ad680 0x7fd384157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.262+0000 7fd3837fe640 1 --2- 172.21.15.105:0/2602772957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd384159f40 0x7fd384153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.262+0000 7fd37bfff640 1 -- 172.21.15.105:0/2602772957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd384159f40 msgr2=0x7fd384153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:37.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.262+0000 7fd37bfff640 1 --2- 172.21.15.105:0/2602772957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd384159f40 0x7fd384153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.262+0000 7fd37bfff640 1 -- 172.21.15.105:0/2602772957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3840ad680 msgr2=0x7fd384157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:37.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.262+0000 7fd37bfff640 1 --2- 172.21.15.105:0/2602772957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3840ad680 0x7fd384157a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.262+0000 7fd37bfff640 1 -- 172.21.15.105:0/2602772957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c009b30 con 0x7fd3840cc760 2023-11-28T18:28:37.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.262+0000 7fd37bfff640 1 --2- 172.21.15.105:0/2602772957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3840cc760 0x7fd38414d890 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00e8b0 tx=0x7fd36c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:37.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.262+0000 7fd37b7fe640 1 -- 172.21.15.105:0/2602772957 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c00c940 con 0x7fd3840cc760 2023-11-28T18:28:37.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.262+0000 7fd37b7fe640 1 -- 172.21.15.105:0/2602772957 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36c00caa0 con 0x7fd3840cc760 2023-11-28T18:28:37.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.262+0000 7fd389679640 1 -- 172.21.15.105:0/2602772957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38414cdd0 con 0x7fd3840cc760 2023-11-28T18:28:37.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.262+0000 7fd37b7fe640 1 -- 172.21.15.105:0/2602772957 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c00ce30 con 0x7fd3840cc760 2023-11-28T18:28:37.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.262+0000 7fd389679640 1 -- 172.21.15.105:0/2602772957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38414c9f0 con 0x7fd3840cc760 2023-11-28T18:28:37.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd37b7fe640 1 -- 172.21.15.105:0/2602772957 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd36c013470 con 0x7fd3840cc760 2023-11-28T18:28:37.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd37b7fe640 1 --2- 172.21.15.105:0/2602772957 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd3640668d0 0x7fd364068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd37b7fe640 1 -- 172.21.15.105:0/2602772957 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd36c084670 con 0x7fd3840cc760 2023-11-28T18:28:37.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd382ffd640 1 --2- 172.21.15.105:0/2602772957 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd3640668d0 0x7fd364068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd38c9f4000 1 Processor -- start 2023-11-28T18:28:37.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd382ffd640 1 --2- 172.21.15.105:0/2602772957 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd3640668d0 0x7fd364068d90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd374009ac0 tx=0x7fd374009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:37.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd38c9f4000 1 -- start start 2023-11-28T18:28:37.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd38c9f4000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae940ffd30 0x55ae94100110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd38c9f4000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae94100650 0x55ae940b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd38c9f4000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae940b3500 0x55ae940b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd38c9f4000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae93f7c110 con 0x55ae940b3500 2023-11-28T18:28:37.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd38c9f4000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae93f0c9a0 con 0x55ae94100650 2023-11-28T18:28:37.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd38c9f4000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae93f09ed0 con 0x55ae940ffd30 2023-11-28T18:28:37.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd3837fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae940b3500 0x55ae940b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd3837fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae940b3500 0x55ae940b38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51802/0 (socket says 172.21.15.105:51802) 2023-11-28T18:28:37.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd3837fe640 1 -- 172.21.15.105:0/2449326068 learned_addr learned my addr 172.21.15.105:0/2449326068 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:37.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd37bfff640 1 --2- 172.21.15.105:0/2449326068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae94100650 0x55ae940b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd382ffd640 1 --2- 172.21.15.105:0/2449326068 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae940ffd30 0x55ae94100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd3837fe640 1 -- 172.21.15.105:0/2449326068 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae940ffd30 msgr2=0x55ae94100110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:37.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd3837fe640 1 --2- 172.21.15.105:0/2449326068 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae940ffd30 0x55ae94100110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd3837fe640 1 -- 172.21.15.105:0/2449326068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae94100650 msgr2=0x55ae940b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:37.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd3837fe640 1 --2- 172.21.15.105:0/2449326068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae94100650 0x55ae940b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd3837fe640 1 -- 172.21.15.105:0/2449326068 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae93bfd890 con 0x55ae940b3500 2023-11-28T18:28:37.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd3837fe640 1 --2- 172.21.15.105:0/2449326068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae940b3500 0x55ae940b38e0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0af040 tx=0x7fd37c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78e800d9abb4996 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:37.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd378ff9640 1 -- 172.21.15.105:0/2449326068 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0a5a60 con 0x55ae940b3500 2023-11-28T18:28:37.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd378ff9640 1 -- 172.21.15.105:0/2449326068 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c0a5bc0 con 0x55ae940b3500 2023-11-28T18:28:37.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.266+0000 7fd378ff9640 1 -- 172.21.15.105:0/2449326068 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0b63f0 con 0x55ae940b3500 2023-11-28T18:28:37.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd38c9f4000 1 -- 172.21.15.105:0/2449326068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae940b3500 msgr2=0x55ae940b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:37.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd38c9f4000 1 --2- 172.21.15.105:0/2449326068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae940b3500 0x55ae940b38e0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0af040 tx=0x7fd37c004ac0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd38c9f4000 1 -- 172.21.15.105:0/2449326068 shutdown_connections 2023-11-28T18:28:37.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd38c9f4000 1 --2- 172.21.15.105:0/2449326068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae940b3500 0x55ae940b38e0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd38c9f4000 1 --2- 172.21.15.105:0/2449326068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae94100650 0x55ae940b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd38c9f4000 1 --2- 172.21.15.105:0/2449326068 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae940ffd30 0x55ae94100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd38c9f4000 1 -- 172.21.15.105:0/2449326068 >> 172.21.15.105:0/2449326068 conn(0x55ae940e08c0 msgr2=0x55ae940e2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:37.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd38c9f4000 1 -- 172.21.15.105:0/2449326068 shutdown_connections 2023-11-28T18:28:37.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd38c9f4000 1 -- 172.21.15.105:0/2449326068 wait complete. 2023-11-28T18:28:37.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd38c9f4000 1 -- start start 2023-11-28T18:28:37.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd38c9f4000 1 Processor -- start 2023-11-28T18:28:37.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd38c9f4000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae940b3500 0x55ae940b9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd38c9f4000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae940ffd30 0x55ae940b9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd38c9f4000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae94100650 0x55ae940b9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd38c9f4000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae93f057c0 con 0x55ae940b3500 2023-11-28T18:28:37.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd38c9f4000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae93ebdbf0 con 0x55ae940ffd30 2023-11-28T18:28:37.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd38c9f4000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae93ec3a90 con 0x55ae94100650 2023-11-28T18:28:37.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd38c9f4000 10 client.0 _ll_register_callbacks cb 0x55ae940a0e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:28:37.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd382ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae940b3500 0x55ae940b9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd382ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae940b3500 0x55ae940b9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51810/0 (socket says 172.21.15.105:51810) 2023-11-28T18:28:37.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd382ffd640 1 -- 172.21.15.105:0/559684102 learned_addr learned my addr 172.21.15.105:0/559684102 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:37.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd3837fe640 1 --2- 172.21.15.105:0/559684102 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae94100650 0x55ae940b9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd37bfff640 1 --2- 172.21.15.105:0/559684102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae940ffd30 0x55ae940b9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd382ffd640 1 -- 172.21.15.105:0/559684102 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae94100650 msgr2=0x55ae940b9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:37.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd382ffd640 1 --2- 172.21.15.105:0/559684102 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae94100650 0x55ae940b9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd382ffd640 1 -- 172.21.15.105:0/559684102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae940ffd30 msgr2=0x55ae940b9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:37.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd382ffd640 1 --2- 172.21.15.105:0/559684102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae940ffd30 0x55ae940b9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd382ffd640 1 -- 172.21.15.105:0/559684102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd384071380 con 0x55ae940b3500 2023-11-28T18:28:37.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd382ffd640 1 --2- 172.21.15.105:0/559684102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae940b3500 0x55ae940b9520 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd374011f10 tx=0x7fd3740046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5fe3cf534b30dee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:37.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd361ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:28:37.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd361ffb640 1 -- 172.21.15.105:0/559684102 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37401c040 con 0x55ae940b3500 2023-11-28T18:28:37.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd361ffb640 1 -- 172.21.15.105:0/559684102 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd374014440 con 0x55ae940b3500 2023-11-28T18:28:37.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.270+0000 7fd38c9f4000 1 -- 172.21.15.105:0/559684102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae93ec7470 con 0x55ae940b3500 2023-11-28T18:28:37.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd361ffb640 1 -- 172.21.15.105:0/559684102 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd374014600 con 0x55ae940b3500 2023-11-28T18:28:37.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd38c9f4000 10 client.7446 fetch_fsmap learned FSMap version 76 2023-11-28T18:28:37.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd38c9f4000 1 -- 172.21.15.105:0/559684102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55ae93ec6190 con 0x55ae940b3500 2023-11-28T18:28:37.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd361ffb640 1 -- 172.21.15.105:0/559684102 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ae93ec7470 con 0x55ae940b3500 2023-11-28T18:28:37.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd361ffb640 1 -- 172.21.15.105:0/559684102 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 76) v1 ==== 3318+0+0 (secure 0 0 0) 0x7fd374014760 con 0x55ae940b3500 2023-11-28T18:28:37.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd38c9f4000 10 client.7446 fetch_fsmap finished waiting for FSMap version 76 2023-11-28T18:28:37.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd38c9f4000 10 client.7446 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:28:37.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd38c9f4000 20 client.7446 populate_metadata read hostname 'smithi105' 2023-11-28T18:28:37.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd38c9f4000 1 --2- 172.21.15.105:0/559684102 >> [v2:172.21.15.105:6834/188173955,v1:172.21.15.105:6836/188173955] conn(0x55ae940ba9b0 0x55ae94111ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.286 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd38c9f4000 4 client.7446 mds_command: new command op to 7194 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:28:37.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd38c9f4000 1 -- 172.21.15.105:0/559684102 --> [v2:172.21.15.105:6834/188173955,v1:172.21.15.105:6836/188173955] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae93ecab50 con 0x55ae940ba9b0 2023-11-28T18:28:37.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd37bfff640 1 --2- 172.21.15.105:0/559684102 >> [v2:172.21.15.105:6834/188173955,v1:172.21.15.105:6836/188173955] conn(0x55ae940ba9b0 0x55ae94111ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd37bfff640 1 --2- 172.21.15.105:0/559684102 >> [v2:172.21.15.105:6834/188173955,v1:172.21.15.105:6836/188173955] conn(0x55ae940ba9b0 0x55ae94111ff0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34de089ee698947d server_cookie=dd773ba087fdd18d in_seq=0 out_seq=0 2023-11-28T18:28:37.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd361ffb640 10 client.7446 ms_handle_connect on v2:172.21.15.105:6834/188173955 2023-11-28T18:28:37.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd361ffb640 1 -- 172.21.15.105:0/559684102 <== mds.0 v2:172.21.15.105:6834/188173955 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ae93ecab50 con 0x55ae940ba9b0 2023-11-28T18:28:37.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd361ffb640 10 client.7446 handle_command_reply: tid=0 2023-11-28T18:28:37.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd38c9f4000 1 client.7446 shutdown 2023-11-28T18:28:37.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd38c9f4000 1 -- 172.21.15.105:0/559684102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae940b3500 msgr2=0x55ae940b9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:37.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd38c9f4000 1 --2- 172.21.15.105:0/559684102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae940b3500 0x55ae940b9520 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd374011f10 tx=0x7fd3740046d0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd38c9f4000 1 -- 172.21.15.105:0/559684102 shutdown_connections 2023-11-28T18:28:37.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd38c9f4000 1 --2- 172.21.15.105:0/559684102 >> [v2:172.21.15.105:6834/188173955,v1:172.21.15.105:6836/188173955] conn(0x55ae940ba9b0 0x55ae94111ff0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd38c9f4000 1 --2- 172.21.15.105:0/559684102 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae94100650 0x55ae940b9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd38c9f4000 1 --2- 172.21.15.105:0/559684102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae940ffd30 0x55ae940b9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd361ffb640 0 client.7446 ms_handle_reset on v2:172.21.15.105:6834/188173955 2023-11-28T18:28:37.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd38c9f4000 1 --2- 172.21.15.105:0/559684102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae940b3500 0x55ae940b9520 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd38c9f4000 1 -- 172.21.15.105:0/559684102 >> 172.21.15.105:0/559684102 conn(0x55ae940e08c0 msgr2=0x55ae940b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:37.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd38c9f4000 1 -- 172.21.15.105:0/559684102 shutdown_connections 2023-11-28T18:28:37.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd38c9f4000 1 -- 172.21.15.105:0/559684102 wait complete. 2023-11-28T18:28:37.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.274+0000 7fd38c9f4000 20 client.7446 trim_cache size 0 max 16384 2023-11-28T18:28:37.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd38c9f4000 1 Processor -- start 2023-11-28T18:28:37.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd38c9f4000 1 -- start start 2023-11-28T18:28:37.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd38c9f4000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae940b3500 0x55ae94114430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd38c9f4000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae940ffd30 0x55ae940df3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd38c9f4000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae94100650 0x55ae940df8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd38c9f4000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae93bbe940 con 0x55ae94100650 2023-11-28T18:28:37.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd38c9f4000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae940bafc0 con 0x55ae940b3500 2023-11-28T18:28:37.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd38c9f4000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae93fa2de0 con 0x55ae940ffd30 2023-11-28T18:28:37.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd3837fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae94100650 0x55ae940df8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd3837fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae94100650 0x55ae940df8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51824/0 (socket says 172.21.15.105:51824) 2023-11-28T18:28:37.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd3837fe640 1 -- 172.21.15.105:0/995449984 learned_addr learned my addr 172.21.15.105:0/995449984 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:37.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd382ffd640 1 --2- 172.21.15.105:0/995449984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae940b3500 0x55ae94114430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd37bfff640 1 --2- 172.21.15.105:0/995449984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae940ffd30 0x55ae940df3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd3837fe640 1 -- 172.21.15.105:0/995449984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae940ffd30 msgr2=0x55ae940df3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:37.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd3837fe640 1 --2- 172.21.15.105:0/995449984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae940ffd30 0x55ae940df3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd3837fe640 1 -- 172.21.15.105:0/995449984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae940b3500 msgr2=0x55ae94114430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:37.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd3837fe640 1 --2- 172.21.15.105:0/995449984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae940b3500 0x55ae94114430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd3837fe640 1 -- 172.21.15.105:0/995449984 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae940b3370 con 0x55ae94100650 2023-11-28T18:28:37.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd37bfff640 1 --2- 172.21.15.105:0/995449984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae940ffd30 0x55ae940df3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:37.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd3837fe640 1 --2- 172.21.15.105:0/995449984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae94100650 0x55ae940df8e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fd37c01c740 tx=0x7fd37c0a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b145ce6276ccd300 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:37.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd3797fa640 1 -- 172.21.15.105:0/995449984 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0090d0 con 0x55ae94100650 2023-11-28T18:28:37.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd3797fa640 1 -- 172.21.15.105:0/995449984 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c009230 con 0x55ae94100650 2023-11-28T18:28:37.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.290+0000 7fd3797fa640 1 -- 172.21.15.105:0/995449984 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0a5ca0 con 0x55ae94100650 2023-11-28T18:28:37.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 1 -- 172.21.15.105:0/995449984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae94100650 msgr2=0x55ae940df8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:37.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 1 --2- 172.21.15.105:0/995449984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae94100650 0x55ae940df8e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fd37c01c740 tx=0x7fd37c0a7000 comp rx=0 tx=0).stop 2023-11-28T18:28:37.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 1 -- 172.21.15.105:0/995449984 shutdown_connections 2023-11-28T18:28:37.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 1 --2- 172.21.15.105:0/995449984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae94100650 0x55ae940df8e0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 1 --2- 172.21.15.105:0/995449984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae940ffd30 0x55ae940df3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 1 --2- 172.21.15.105:0/995449984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae940b3500 0x55ae94114430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 1 -- 172.21.15.105:0/995449984 >> 172.21.15.105:0/995449984 conn(0x55ae940e08c0 msgr2=0x55ae940b1720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:37.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 1 -- 172.21.15.105:0/995449984 shutdown_connections 2023-11-28T18:28:37.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 1 -- 172.21.15.105:0/995449984 wait complete. 2023-11-28T18:28:37.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 1 -- start start 2023-11-28T18:28:37.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 1 Processor -- start 2023-11-28T18:28:37.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae940b3500 0x55ae94160600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae940ffd30 0x55ae94160b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae94100650 0x55ae9411ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae940b6560 con 0x55ae940ffd30 2023-11-28T18:28:37.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae940b6e10 con 0x55ae94100650 2023-11-28T18:28:37.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae93ec29b0 con 0x55ae940b3500 2023-11-28T18:28:37.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 10 client.0 _ll_register_callbacks cb 0x55ae93f7c060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:28:37.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd37bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae940ffd30 0x55ae94160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd37bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae940ffd30 0x55ae94160b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51840/0 (socket says 172.21.15.105:51840) 2023-11-28T18:28:37.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd37bfff640 1 -- 172.21.15.105:0/763921662 learned_addr learned my addr 172.21.15.105:0/763921662 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:37.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd382ffd640 1 --2- 172.21.15.105:0/763921662 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae940b3500 0x55ae94160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd3837fe640 1 --2- 172.21.15.105:0/763921662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae94100650 0x55ae9411ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd37bfff640 1 -- 172.21.15.105:0/763921662 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae940b3500 msgr2=0x55ae94160600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:37.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd37bfff640 1 --2- 172.21.15.105:0/763921662 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae940b3500 0x55ae94160600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd37bfff640 1 -- 172.21.15.105:0/763921662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae94100650 msgr2=0x55ae9411ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:37.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd37bfff640 1 --2- 172.21.15.105:0/763921662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae94100650 0x55ae9411ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd37bfff640 1 -- 172.21.15.105:0/763921662 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37c0c4040 con 0x55ae940ffd30 2023-11-28T18:28:37.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd37bfff640 1 --2- 172.21.15.105:0/763921662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae940ffd30 0x55ae94160b40 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0551e0 tx=0x7fd36c0518a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602263a562efe78c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:37.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd361ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:28:37.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd361ffb640 1 -- 172.21.15.105:0/763921662 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c054bd0 con 0x55ae940ffd30 2023-11-28T18:28:37.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd361ffb640 1 -- 172.21.15.105:0/763921662 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd36c053940 con 0x55ae940ffd30 2023-11-28T18:28:37.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 1 -- 172.21.15.105:0/763921662 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae93f04110 con 0x55ae940ffd30 2023-11-28T18:28:37.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd361ffb640 1 -- 172.21.15.105:0/763921662 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd36c0730d0 con 0x55ae940ffd30 2023-11-28T18:28:37.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 10 client.7452 fetch_fsmap learned FSMap version 76 2023-11-28T18:28:37.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 1 -- 172.21.15.105:0/763921662 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55ae93eac490 con 0x55ae940ffd30 2023-11-28T18:28:37.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd361ffb640 1 -- 172.21.15.105:0/763921662 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36c077080 con 0x55ae940ffd30 2023-11-28T18:28:37.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd361ffb640 1 -- 172.21.15.105:0/763921662 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 76) v1 ==== 3318+0+0 (secure 0 0 0) 0x7fd36c011070 con 0x55ae940ffd30 2023-11-28T18:28:37.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 10 client.7452 fetch_fsmap finished waiting for FSMap version 76 2023-11-28T18:28:37.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 10 client.7452 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:28:37.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 20 client.7452 populate_metadata read hostname 'smithi105' 2023-11-28T18:28:37.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 1 --2- 172.21.15.105:0/763921662 >> [v2:172.21.15.105:6834/188173955,v1:172.21.15.105:6836/188173955] conn(0x55ae94167460 0x55ae94169850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:37.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 4 client.7452 mds_command: new command op to 7194 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:28:37.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd38c9f4000 1 -- 172.21.15.105:0/763921662 --> [v2:172.21.15.105:6834/188173955,v1:172.21.15.105:6836/188173955] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ae93f32650 con 0x55ae94167460 2023-11-28T18:28:37.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.294+0000 7fd382ffd640 1 --2- 172.21.15.105:0/763921662 >> [v2:172.21.15.105:6834/188173955,v1:172.21.15.105:6836/188173955] conn(0x55ae94167460 0x55ae94169850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:37.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd361ffb640 10 client.7452 ms_handle_connect on v2:172.21.15.105:6834/188173955 2023-11-28T18:28:37.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd382ffd640 1 --2- 172.21.15.105:0/763921662 >> [v2:172.21.15.105:6834/188173955,v1:172.21.15.105:6836/188173955] conn(0x55ae94167460 0x55ae94169850 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa11fd4fddd5e4c4 server_cookie=70ca798ab6babd79 in_seq=0 out_seq=0 2023-11-28T18:28:37.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd361ffb640 1 -- 172.21.15.105:0/763921662 <== mds.0 v2:172.21.15.105:6834/188173955 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55ae93f32650 con 0x55ae94167460 2023-11-28T18:28:37.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd361ffb640 10 client.7452 handle_command_reply: tid=0 2023-11-28T18:28:37.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd38c9f4000 1 client.7452 shutdown 2023-11-28T18:28:37.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd38c9f4000 1 -- 172.21.15.105:0/763921662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae940ffd30 msgr2=0x55ae94160b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:37.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd38c9f4000 1 --2- 172.21.15.105:0/763921662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae940ffd30 0x55ae94160b40 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0551e0 tx=0x7fd36c0518a0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd38c9f4000 1 -- 172.21.15.105:0/763921662 shutdown_connections 2023-11-28T18:28:37.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd38c9f4000 1 --2- 172.21.15.105:0/763921662 >> [v2:172.21.15.105:6834/188173955,v1:172.21.15.105:6836/188173955] conn(0x55ae94167460 0x55ae94169850 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd38c9f4000 1 --2- 172.21.15.105:0/763921662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae94100650 0x55ae9411ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd38c9f4000 1 --2- 172.21.15.105:0/763921662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae940ffd30 0x55ae94160b40 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd361ffb640 0 client.7452 ms_handle_reset on v2:172.21.15.105:6834/188173955 2023-11-28T18:28:37.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd38c9f4000 1 --2- 172.21.15.105:0/763921662 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae940b3500 0x55ae94160600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd38c9f4000 1 -- 172.21.15.105:0/763921662 >> 172.21.15.105:0/763921662 conn(0x55ae940e08c0 msgr2=0x55ae940b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:37.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd38c9f4000 1 -- 172.21.15.105:0/763921662 shutdown_connections 2023-11-28T18:28:37.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd38c9f4000 1 -- 172.21.15.105:0/763921662 wait complete. 2023-11-28T18:28:37.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd38c9f4000 20 client.7452 trim_cache size 0 max 16384 2023-11-28T18:28:37.308 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:28:37.308 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:28:37.308 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2023-11-28T18:28:37.309 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 7194, 2023-11-28T18:28:37.309 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:28:37.309 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:28:37.309 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:28:37.309 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 23.564104527000001, 2023-11-28T18:28:37.309 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 76, 2023-11-28T18:28:37.309 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 78, 2023-11-28T18:28:37.309 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 78, 2023-11-28T18:28:37.309 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 28.456676938000001 2023-11-28T18:28:37.309 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:28:37.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd389679640 1 -- 172.21.15.105:0/2602772957 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd3640668d0 msgr2=0x7fd364068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:37.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd389679640 1 --2- 172.21.15.105:0/2602772957 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd3640668d0 0x7fd364068d90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd374009ac0 tx=0x7fd374009190 comp rx=0 tx=0).stop 2023-11-28T18:28:37.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd389679640 1 -- 172.21.15.105:0/2602772957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3840cc760 msgr2=0x7fd38414d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:37.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd389679640 1 --2- 172.21.15.105:0/2602772957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3840cc760 0x7fd38414d890 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00e8b0 tx=0x7fd36c00ed70 comp rx=0 tx=0).stop 2023-11-28T18:28:37.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd389679640 1 -- 172.21.15.105:0/2602772957 shutdown_connections 2023-11-28T18:28:37.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd389679640 1 --2- 172.21.15.105:0/2602772957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd384159f40 0x7fd384153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd389679640 1 --2- 172.21.15.105:0/2602772957 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd3640668d0 0x7fd364068d90 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd389679640 1 --2- 172.21.15.105:0/2602772957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3840cc760 0x7fd38414d890 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd389679640 1 --2- 172.21.15.105:0/2602772957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3840ad680 0x7fd384157a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:37.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd389679640 1 -- 172.21.15.105:0/2602772957 >> 172.21.15.105:0/2602772957 conn(0x7fd3840ceb90 msgr2=0x7fd3840ac480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:37.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd389679640 1 -- 172.21.15.105:0/2602772957 shutdown_connections 2023-11-28T18:28:37.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:37.298+0000 7fd389679640 1 -- 172.21.15.105:0/2602772957 wait complete. 2023-11-28T18:28:37.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:28:37.322 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-28T18:28:37.322 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:28:37.322 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p tree 2023-11-28T18:28:37.322 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:28:37.322 DEBUG:teuthology.orchestra.run.smithi105:> setfattr -n ceph.dir.pin.distributed -v 1 tree 2023-11-28T18:28:37.322 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:28:37.322 DEBUG:teuthology.orchestra.run.smithi105:> for ((i = 0; i < 100; i++)); do 2023-11-28T18:28:37.322 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p "tree/$i" 2023-11-28T18:28:37.322 DEBUG:teuthology.orchestra.run.smithi105:> echo file > "tree/$i/file" 2023-11-28T18:28:37.322 DEBUG:teuthology.orchestra.run.smithi105:> done 2023-11-28T18:28:37.323 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:28:37.334 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree 2023-11-28T18:28:37.339 INFO:teuthology.orchestra.run.smithi105.stderr:+ setfattr -n ceph.dir.pin.distributed -v 1 tree 2023-11-28T18:28:37.343 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i = 0 )) 2023-11-28T18:28:37.343 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.344 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/0 2023-11-28T18:28:37.348 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.355 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.355 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.355 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/1 2023-11-28T18:28:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.377 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.377 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.377 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/2 2023-11-28T18:28:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.386 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.386 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.386 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/3 2023-11-28T18:28:37.393 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/4 2023-11-28T18:28:37.401 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/5 2023-11-28T18:28:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.412 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.412 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.412 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/6 2023-11-28T18:28:37.418 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.421 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.421 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.421 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/7 2023-11-28T18:28:37.427 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.430 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.430 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.430 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/8 2023-11-28T18:28:37.436 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.438 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.438 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.438 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/9 2023-11-28T18:28:37.445 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.447 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.447 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.447 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/10 2023-11-28T18:28:37.455 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.456 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.456 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.456 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/11 2023-11-28T18:28:37.462 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.464 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.465 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.465 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/12 2023-11-28T18:28:37.471 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.474 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.474 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.474 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/13 2023-11-28T18:28:37.480 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.482 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.482 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.483 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/14 2023-11-28T18:28:37.489 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.491 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.491 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.491 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/15 2023-11-28T18:28:37.498 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.500 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.500 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.500 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/16 2023-11-28T18:28:37.507 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.509 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.509 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.509 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/17 2023-11-28T18:28:37.515 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.517 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.517 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.517 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/18 2023-11-28T18:28:37.523 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.525 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.525 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.526 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/19 2023-11-28T18:28:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/20 2023-11-28T18:28:37.540 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.542 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.542 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.543 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/21 2023-11-28T18:28:37.549 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/22 2023-11-28T18:28:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.560 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.560 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.560 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/23 2023-11-28T18:28:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.568 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.568 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.568 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/24 2023-11-28T18:28:37.574 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.577 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.577 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.577 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/25 2023-11-28T18:28:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.584 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.585 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.585 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/26 2023-11-28T18:28:37.590 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/27 2023-11-28T18:28:37.599 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.601 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.601 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.601 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/28 2023-11-28T18:28:37.607 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.609 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.666 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.666 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/29 2023-11-28T18:28:37.669 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.669 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.669 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.669 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/30 2023-11-28T18:28:37.669 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.669 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.669 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.669 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/31 2023-11-28T18:28:37.669 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.670 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.670 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.670 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/32 2023-11-28T18:28:37.670 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.670 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.670 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.670 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/33 2023-11-28T18:28:37.670 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.670 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.670 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.670 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/34 2023-11-28T18:28:37.671 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.671 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.671 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.671 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/35 2023-11-28T18:28:37.671 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.671 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.671 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.671 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/36 2023-11-28T18:28:37.673 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.675 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.675 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.676 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/37 2023-11-28T18:28:37.682 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.684 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.684 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.684 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/38 2023-11-28T18:28:37.690 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.692 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.692 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.692 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/39 2023-11-28T18:28:37.699 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.701 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.701 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.701 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/40 2023-11-28T18:28:37.707 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.709 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.710 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.710 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/41 2023-11-28T18:28:37.715 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.717 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.717 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.717 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/42 2023-11-28T18:28:37.723 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.725 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.725 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.725 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/43 2023-11-28T18:28:37.732 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.734 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.735 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.735 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/44 2023-11-28T18:28:37.740 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.742 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.743 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.743 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/45 2023-11-28T18:28:37.748 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.750 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.750 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.750 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/46 2023-11-28T18:28:37.756 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/47 2023-11-28T18:28:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.766 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.766 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.766 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/48 2023-11-28T18:28:37.773 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.774 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.774 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.774 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/49 2023-11-28T18:28:37.780 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.782 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.782 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.782 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/50 2023-11-28T18:28:37.788 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.791 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.791 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.791 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/51 2023-11-28T18:28:37.797 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.799 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.799 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.799 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/52 2023-11-28T18:28:37.805 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.808 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.808 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.808 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/53 2023-11-28T18:28:37.814 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.816 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.816 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.816 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/54 2023-11-28T18:28:37.822 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.824 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.825 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.825 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/55 2023-11-28T18:28:37.831 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.833 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.833 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.833 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/56 2023-11-28T18:28:37.839 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.841 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.841 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.841 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/57 2023-11-28T18:28:37.847 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.849 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.849 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.850 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/58 2023-11-28T18:28:37.855 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.857 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.858 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.858 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/59 2023-11-28T18:28:37.864 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.866 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.866 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.866 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/60 2023-11-28T18:28:37.872 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.874 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.875 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.875 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/61 2023-11-28T18:28:37.881 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.883 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.884 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.884 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/62 2023-11-28T18:28:37.889 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.892 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.892 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.892 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/63 2023-11-28T18:28:37.899 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.902 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.902 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/64 2023-11-28T18:28:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.910 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.910 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.910 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/65 2023-11-28T18:28:37.916 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/66 2023-11-28T18:28:37.925 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.927 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.928 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.928 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/67 2023-11-28T18:28:37.934 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.936 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.936 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.936 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/68 2023-11-28T18:28:37.942 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.945 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.945 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.945 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/69 2023-11-28T18:28:37.951 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.954 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.954 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.954 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/70 2023-11-28T18:28:37.960 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.963 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.963 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.963 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/71 2023-11-28T18:28:37.969 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.971 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.972 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.972 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/72 2023-11-28T18:28:37.978 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.980 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.980 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.980 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/73 2023-11-28T18:28:37.986 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.990 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.991 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.991 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/74 2023-11-28T18:28:37.995 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:37.997 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:37.997 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:37.997 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/75 2023-11-28T18:28:38.004 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.006 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.006 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.006 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/76 2023-11-28T18:28:38.012 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.014 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.014 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.014 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/77 2023-11-28T18:28:38.020 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.023 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.023 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.023 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/78 2023-11-28T18:28:38.028 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.031 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.031 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.031 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/79 2023-11-28T18:28:38.037 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.039 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.039 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.039 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/80 2023-11-28T18:28:38.045 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.048 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.048 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.048 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/81 2023-11-28T18:28:38.054 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.056 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.056 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.056 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/82 2023-11-28T18:28:38.062 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.065 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.065 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.065 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/83 2023-11-28T18:28:38.071 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.073 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.073 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.074 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/84 2023-11-28T18:28:38.079 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.081 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.082 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.082 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/85 2023-11-28T18:28:38.088 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.090 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.090 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.090 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/86 2023-11-28T18:28:38.096 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.099 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.099 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.099 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/87 2023-11-28T18:28:38.105 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/88 2023-11-28T18:28:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.119 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/89 2023-11-28T18:28:38.124 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/90 2023-11-28T18:28:38.133 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.136 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.136 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.136 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/91 2023-11-28T18:28:38.142 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.144 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.144 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.144 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/92 2023-11-28T18:28:38.151 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.152 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.152 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.152 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/93 2023-11-28T18:28:38.158 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.161 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.161 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.162 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/94 2023-11-28T18:28:38.168 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.170 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.170 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.171 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/95 2023-11-28T18:28:38.177 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.179 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.180 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.180 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/96 2023-11-28T18:28:38.186 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.189 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.189 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.189 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/97 2023-11-28T18:28:38.195 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.198 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.198 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.198 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/98 2023-11-28T18:28:38.204 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.207 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.207 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.207 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/99 2023-11-28T18:28:38.213 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:28:38.215 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:28:38.215 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:28:38.217 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-11-28T18:28:38.378 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-28T18:28:38.379 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:28:38.380 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2023-11-28T18:28:38.538 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-28T18:28:38.538 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:28:38.539 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-11-28T18:28:38.696 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"6338","version":"6336","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"2051","version":"2051","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2023-11-28T18:28:38.696 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "6338", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6336" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "2051", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "2051" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:28:38.697 INFO:tasks.cephfs.cephfs_test_case:len=0 [] 2023-11-28T18:28:39.703 INFO:tasks.ceph.mds.a.smithi105.stderr:2023-11-28T18:28:39.698+0000 7f1548db0640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-28T18:28:39.703 INFO:tasks.ceph.mds.a.smithi105.stderr:2023-11-28T18:28:39.698+0000 7f1548db0640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-11-28T18:28:43.699 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-11-28T18:28:43.867 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-28T18:28:43.867 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:28:43.867 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2023-11-28T18:28:44.023 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-28T18:28:44.023 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:28:44.023 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-11-28T18:28:44.184 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"6347","version":"6347","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"2051","version":"2051","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x100000005f2.011*","snapid_first":2,"projected_version":"430","version":"430","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x100000005f2.110*","snapid_first":2,"projected_version":"442","version":"442","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x100000005f2.000*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x100000005f2.010*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x100000005f2.001*","snapid_first":2,"projected_version":"428","version":"428","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x100000005f2.100*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":0},"nref":3}}] 2023-11-28T18:28:44.186 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "6347", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6347" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "2051", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "2051" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x100000005f2.011*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "430", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "430" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x100000005f2.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "442", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "442" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x100000005f2.000*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "426", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x100000005f2.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "426", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x100000005f2.001*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "428", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "428" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x100000005f2.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:28:44.186 INFO:tasks.cephfs.cephfs_test_case:len=6 [{'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x100000005f2.011*', 'snapid_first': 2, 'projected_version': '430', 'version': '430', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x100000005f2.110*', 'snapid_first': 2, 'projected_version': '442', 'version': '442', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x100000005f2.000*', 'snapid_first': 2, 'projected_version': '426', 'version': '426', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x100000005f2.010*', 'snapid_first': 2, 'projected_version': '426', 'version': '426', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x100000005f2.001*', 'snapid_first': 2, 'projected_version': '428', 'version': '428', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x100000005f2.100*', 'snapid_first': 2, 'projected_version': '426', 'version': '426', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'waiter': 0, 'authpin': 0}, 'nref': 3}}] 2023-11-28T18:28:44.186 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T18:28:44.187 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:28:44.271 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:28:44.356 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T18:28:44.356 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:28:44.356 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:28:44.366 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:28:44.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.538+0000 7f3afe934640 1 Processor -- start 2023-11-28T18:28:44.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.538+0000 7f3afe934640 1 -- start start 2023-11-28T18:28:44.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.542+0000 7f3afe934640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3af80cb9c0 0x7f3af80aef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:44.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.542+0000 7f3afe934640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3af8071230 0x7f3af8161170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:44.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.542+0000 7f3afe934640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3af81616b0 0x7f3af806cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:44.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.542+0000 7f3afe934640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af806ef10 con 0x7f3af81616b0 2023-11-28T18:28:44.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.542+0000 7f3afe934640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af806e700 con 0x7f3af8071230 2023-11-28T18:28:44.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.542+0000 7f3afe934640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af806e950 con 0x7f3af80cb9c0 2023-11-28T18:28:44.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.542+0000 7f3afe133640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3af81616b0 0x7f3af806cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:44.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.542+0000 7f3afe133640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3af81616b0 0x7f3af806cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47216/0 (socket says 172.21.15.105:47216) 2023-11-28T18:28:44.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.542+0000 7f3afe133640 1 -- 172.21.15.105:0/1196715303 learned_addr learned my addr 172.21.15.105:0/1196715303 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:44.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.542+0000 7f3afd131640 1 --2- 172.21.15.105:0/1196715303 >> v2:172.21.15.148:3300/0 conn(0x7f3af8071230 0x7f3af8161170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:44.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.542+0000 7f3afd932640 1 --2- 172.21.15.105:0/1196715303 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3af80cb9c0 0x7f3af80aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:44.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.542+0000 7f3afe133640 1 -- 172.21.15.105:0/1196715303 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3af80cb9c0 msgr2=0x7f3af80aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:44.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.542+0000 7f3afe133640 1 --2- 172.21.15.105:0/1196715303 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3af80cb9c0 0x7f3af80aef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:44.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.542+0000 7f3afe133640 1 -- 172.21.15.105:0/1196715303 >> v2:172.21.15.148:3300/0 conn(0x7f3af8071230 msgr2=0x7f3af8161170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:44.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.542+0000 7f3afe133640 1 --2- 172.21.15.105:0/1196715303 >> v2:172.21.15.148:3300/0 conn(0x7f3af8071230 0x7f3af8161170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:44.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.542+0000 7f3afe133640 1 -- 172.21.15.105:0/1196715303 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af806ec30 con 0x7f3af81616b0 2023-11-28T18:28:44.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.546+0000 7f3afe133640 1 --2- 172.21.15.105:0/1196715303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3af81616b0 0x7f3af806cff0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f3af8070b60 tx=0x7f3af001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3064947d7677a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:44.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.546+0000 7f3afc930640 1 -- 172.21.15.105:0/1196715303 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af00a56a0 con 0x7f3af81616b0 2023-11-28T18:28:44.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.546+0000 7f3afc930640 1 -- 172.21.15.105:0/1196715303 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af00a5800 con 0x7f3af81616b0 2023-11-28T18:28:44.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.546+0000 7f3afc930640 1 -- 172.21.15.105:0/1196715303 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af00a5b90 con 0x7f3af81616b0 2023-11-28T18:28:44.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.546+0000 7f3afe934640 1 -- 172.21.15.105:0/1196715303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3af81616b0 msgr2=0x7f3af806cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:44.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.546+0000 7f3afe934640 1 --2- 172.21.15.105:0/1196715303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3af81616b0 0x7f3af806cff0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f3af8070b60 tx=0x7f3af001c910 comp rx=0 tx=0).stop 2023-11-28T18:28:44.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.546+0000 7f3afe934640 1 -- 172.21.15.105:0/1196715303 shutdown_connections 2023-11-28T18:28:44.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.546+0000 7f3afe934640 1 --2- 172.21.15.105:0/1196715303 >> v2:172.21.15.148:3300/0 conn(0x7f3af8071230 0x7f3af8161170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:44.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.546+0000 7f3afe934640 1 --2- 172.21.15.105:0/1196715303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3af81616b0 0x7f3af806cff0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:44.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.546+0000 7f3afe934640 1 --2- 172.21.15.105:0/1196715303 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3af80cb9c0 0x7f3af80aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:44.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.546+0000 7f3afe934640 1 -- 172.21.15.105:0/1196715303 >> 172.21.15.105:0/1196715303 conn(0x7f3af80cc500 msgr2=0x7f3af80cc910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:44.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.546+0000 7f3afe934640 1 -- 172.21.15.105:0/1196715303 shutdown_connections 2023-11-28T18:28:44.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.546+0000 7f3afe934640 1 -- 172.21.15.105:0/1196715303 wait complete. 2023-11-28T18:28:44.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.546+0000 7f3afe934640 1 Processor -- start 2023-11-28T18:28:44.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.546+0000 7f3afe934640 1 -- start start 2023-11-28T18:28:44.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3afe934640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3af8071230 0x7f3af8155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:44.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3afe934640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3af80cb9c0 0x7f3af81584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:44.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3afe934640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3af81616b0 0x7f3af8158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:44.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3afe934640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af80b2ad0 con 0x7f3af8071230 2023-11-28T18:28:44.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3afe934640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af80c78f0 con 0x7f3af81616b0 2023-11-28T18:28:44.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3afe934640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af80c7bd0 con 0x7f3af80cb9c0 2023-11-28T18:28:44.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3afd932640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3af8071230 0x7f3af8155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:44.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3afd932640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3af8071230 0x7f3af8155380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47222/0 (socket says 172.21.15.105:47222) 2023-11-28T18:28:44.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3afd932640 1 -- 172.21.15.105:0/1262357837 learned_addr learned my addr 172.21.15.105:0/1262357837 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:44.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3afe133640 1 --2- 172.21.15.105:0/1262357837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3af81616b0 0x7f3af8158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:44.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3afd131640 1 --2- 172.21.15.105:0/1262357837 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3af80cb9c0 0x7f3af81584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:44.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3afd932640 1 -- 172.21.15.105:0/1262357837 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3af80cb9c0 msgr2=0x7f3af81584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:44.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3afd932640 1 --2- 172.21.15.105:0/1262357837 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3af80cb9c0 0x7f3af81584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:44.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3afd932640 1 -- 172.21.15.105:0/1262357837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3af81616b0 msgr2=0x7f3af8158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:44.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3afd932640 1 --2- 172.21.15.105:0/1262357837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3af81616b0 0x7f3af8158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:44.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3afd932640 1 -- 172.21.15.105:0/1262357837 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af0009b30 con 0x7f3af8071230 2023-11-28T18:28:44.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3afd932640 1 --2- 172.21.15.105:0/1262357837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3af8071230 0x7f3af8155380 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f3af400e930 tx=0x7f3af400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:44.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3aee7fc640 1 -- 172.21.15.105:0/1262357837 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af4009690 con 0x7f3af8071230 2023-11-28T18:28:44.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3aee7fc640 1 -- 172.21.15.105:0/1262357837 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af400ca00 con 0x7f3af8071230 2023-11-28T18:28:44.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3aee7fc640 1 -- 172.21.15.105:0/1262357837 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af400cd70 con 0x7f3af8071230 2023-11-28T18:28:44.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3afe934640 1 -- 172.21.15.105:0/1262357837 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af80b2760 con 0x7f3af8071230 2023-11-28T18:28:44.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3afe934640 1 -- 172.21.15.105:0/1262357837 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af80c6db0 con 0x7f3af8071230 2023-11-28T18:28:44.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3afe934640 1 -- 172.21.15.105:0/1262357837 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af80c7df0 con 0x7f3af8071230 2023-11-28T18:28:44.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.550+0000 7f3aee7fc640 1 -- 172.21.15.105:0/1262357837 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3af4013470 con 0x7f3af8071230 2023-11-28T18:28:44.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.554+0000 7f3aee7fc640 1 --2- 172.21.15.105:0/1262357837 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3adc0668d0 0x7f3adc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:44.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.554+0000 7f3aee7fc640 1 -- 172.21.15.105:0/1262357837 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3af4084720 con 0x7f3af8071230 2023-11-28T18:28:44.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.554+0000 7f3afd131640 1 --2- 172.21.15.105:0/1262357837 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3adc0668d0 0x7f3adc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:44.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.554+0000 7f3afd131640 1 --2- 172.21.15.105:0/1262357837 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3adc0668d0 0x7f3adc068d90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3af8155fe0 tx=0x7f3ae8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:44.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.558+0000 7f3aee7fc640 1 -- 172.21.15.105:0/1262357837 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3af40518f0 con 0x7f3af8071230 2023-11-28T18:28:44.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.706+0000 7f3afe934640 1 -- 172.21.15.105:0/1262357837 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3af8074580 con 0x7f3af8071230 2023-11-28T18:28:44.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.706+0000 7f3aee7fc640 1 -- 172.21.15.105:0/1262357837 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16837 (secure 0 0 0) 0x7f3af4054f10 con 0x7f3af8071230 2023-11-28T18:28:44.711 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:44.711 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":78,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:28:14.101062+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:28:11.454373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-28T18:28:12.093171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:28:44.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.710+0000 7f3afe934640 1 -- 172.21.15.105:0/1262357837 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3adc0668d0 msgr2=0x7f3adc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:44.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.710+0000 7f3afe934640 1 --2- 172.21.15.105:0/1262357837 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3adc0668d0 0x7f3adc068d90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3af8155fe0 tx=0x7f3ae8009190 comp rx=0 tx=0).stop 2023-11-28T18:28:44.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.710+0000 7f3afe934640 1 -- 172.21.15.105:0/1262357837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3af8071230 msgr2=0x7f3af8155380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:44.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.710+0000 7f3afe934640 1 --2- 172.21.15.105:0/1262357837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3af8071230 0x7f3af8155380 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f3af400e930 tx=0x7f3af400edf0 comp rx=0 tx=0).stop 2023-11-28T18:28:44.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.710+0000 7f3afe934640 1 -- 172.21.15.105:0/1262357837 shutdown_connections 2023-11-28T18:28:44.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.710+0000 7f3afe934640 1 --2- 172.21.15.105:0/1262357837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3af81616b0 0x7f3af8158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:44.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.710+0000 7f3afe934640 1 --2- 172.21.15.105:0/1262357837 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3af80cb9c0 0x7f3af81584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:44.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.710+0000 7f3afe934640 1 --2- 172.21.15.105:0/1262357837 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3adc0668d0 0x7f3adc068d90 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:44.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.710+0000 7f3afe934640 1 --2- 172.21.15.105:0/1262357837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3af8071230 0x7f3af8155380 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:44.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.710+0000 7f3afe934640 1 -- 172.21.15.105:0/1262357837 >> 172.21.15.105:0/1262357837 conn(0x7f3af80cc500 msgr2=0x7f3af80ada90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:44.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.710+0000 7f3afe934640 1 -- 172.21.15.105:0/1262357837 shutdown_connections 2023-11-28T18:28:44.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.710+0000 7f3afe934640 1 -- 172.21.15.105:0/1262357837 wait complete. 2023-11-28T18:28:44.732 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:28:11.454373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:28:12.093171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:28:44.732 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:28:44.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.870+0000 7fec30a08640 1 Processor -- start 2023-11-28T18:28:44.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.870+0000 7fec30a08640 1 -- start start 2023-11-28T18:28:44.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.870+0000 7fec30a08640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fec2c157910 0x7fec2c0c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:44.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.870+0000 7fec30a08640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec2c06cc90 0x7fec2c06d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:44.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.870+0000 7fec30a08640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec2c070920 con 0x7fec2c15d8d0 2023-11-28T18:28:44.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.870+0000 7fec30a08640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec2c157cf0 con 0x7fec2c157910 2023-11-28T18:28:44.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.870+0000 7fec30a08640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c06d750 con 0x7fec2c06cc90 2023-11-28T18:28:44.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.870+0000 7fec2b59e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fec2c15d8d0 0x7fec2c0716c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43854/0 (socket says 172.21.15.105:43854) 2023-11-28T18:28:44.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.870+0000 7fec2b59e640 1 -- 172.21.15.105:0/4111862630 learned_addr learned my addr 172.21.15.105:0/4111862630 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:44.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.870+0000 7fec2ad9d640 1 --2- 172.21.15.105:0/4111862630 >> v2:172.21.15.148:3300/0 conn(0x7fec2c157910 0x7fec2c0c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:44.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.870+0000 7fec23fff640 1 --2- 172.21.15.105:0/4111862630 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec2c06cc90 0x7fec2c06d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:44.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.870+0000 7fec2ad9d640 1 -- 172.21.15.105:0/4111862630 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec2c06cc90 msgr2=0x7fec2c06d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:44.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.870+0000 7fec2ad9d640 1 --2- 172.21.15.105:0/4111862630 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec2c06cc90 0x7fec2c06d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:44.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.870+0000 7fec2ad9d640 1 -- 172.21.15.105:0/4111862630 >> v1:172.21.15.105:6789/0 conn(0x7fec2c15d8d0 legacy=0x7fec2c0716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:44.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.870+0000 7fec2a59c640 1 -- 172.21.15.105:0/4111862630 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007811818 0 0) 0x7fec2c070920 con 0x7fec2c15d8d0 2023-11-28T18:28:44.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.870+0000 7fec2ad9d640 1 -- 172.21.15.105:0/4111862630 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c070be0 con 0x7fec2c157910 2023-11-28T18:28:44.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.874+0000 7fec2ad9d640 1 --2- 172.21.15.105:0/4111862630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec2c157910 0x7fec2c0c9030 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0097c0 tx=0x7fec1c01c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ade055640eae6d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:44.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.874+0000 7fec2a59c640 1 -- 172.21.15.105:0/4111862630 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c0ab5f0 con 0x7fec2c157910 2023-11-28T18:28:44.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.874+0000 7fec2a59c640 1 -- 172.21.15.105:0/4111862630 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1c0ab750 con 0x7fec2c157910 2023-11-28T18:28:44.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.874+0000 7fec2a59c640 1 -- 172.21.15.105:0/4111862630 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c0abae0 con 0x7fec2c157910 2023-11-28T18:28:44.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.874+0000 7fec30a08640 1 -- 172.21.15.105:0/4111862630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec2c157910 msgr2=0x7fec2c0c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:44.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.874+0000 7fec30a08640 1 --2- 172.21.15.105:0/4111862630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec2c157910 0x7fec2c0c9030 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0097c0 tx=0x7fec1c01c3f0 comp rx=0 tx=0).stop 2023-11-28T18:28:44.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.874+0000 7fec30a08640 1 -- 172.21.15.105:0/4111862630 shutdown_connections 2023-11-28T18:28:44.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.874+0000 7fec30a08640 1 --2- 172.21.15.105:0/4111862630 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec2c06cc90 0x7fec2c06d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:44.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.874+0000 7fec30a08640 1 --2- 172.21.15.105:0/4111862630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec2c157910 0x7fec2c0c9030 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:44.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.874+0000 7fec30a08640 1 -- 172.21.15.105:0/4111862630 >> 172.21.15.105:0/4111862630 conn(0x7fec2c0ce9e0 msgr2=0x7fec2c0cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:44.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.874+0000 7fec30a08640 1 -- 172.21.15.105:0/4111862630 shutdown_connections 2023-11-28T18:28:44.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.874+0000 7fec30a08640 1 -- 172.21.15.105:0/4111862630 wait complete. 2023-11-28T18:28:44.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec30a08640 1 Processor -- start 2023-11-28T18:28:44.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec30a08640 1 -- start start 2023-11-28T18:28:44.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec30a08640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec2c06cc90 0x7fec2c0c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:44.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec30a08640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec2c157910 0x7fec2c0b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:44.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec30a08640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec2c15d8d0 0x7fec2c14a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:44.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec30a08640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c0b4450 con 0x7fec2c157910 2023-11-28T18:28:44.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec30a08640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c1fa890 con 0x7fec2c15d8d0 2023-11-28T18:28:44.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec30a08640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c0c8a90 con 0x7fec2c06cc90 2023-11-28T18:28:44.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec23fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec2c157910 0x7fec2c0b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:44.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec2ad9d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec2c06cc90 0x7fec2c0c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:44.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec23fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec2c157910 0x7fec2c0b3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47234/0 (socket says 172.21.15.105:47234) 2023-11-28T18:28:44.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec23fff640 1 -- 172.21.15.105:0/2782285910 learned_addr learned my addr 172.21.15.105:0/2782285910 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:44.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec2b59e640 1 --2- 172.21.15.105:0/2782285910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec2c15d8d0 0x7fec2c14a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:44.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec23fff640 1 -- 172.21.15.105:0/2782285910 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec2c06cc90 msgr2=0x7fec2c0c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:44.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec23fff640 1 --2- 172.21.15.105:0/2782285910 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec2c06cc90 0x7fec2c0c84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:44.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec23fff640 1 -- 172.21.15.105:0/2782285910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec2c15d8d0 msgr2=0x7fec2c14a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:44.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec23fff640 1 --2- 172.21.15.105:0/2782285910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec2c15d8d0 0x7fec2c14a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:44.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec23fff640 1 -- 172.21.15.105:0/2782285910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1c009420 con 0x7fec2c157910 2023-11-28T18:28:44.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec23fff640 1 --2- 172.21.15.105:0/2782285910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec2c157910 0x7fec2c0b3f10 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fec1400e8a0 tx=0x7fec1400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:44.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec237fe640 1 -- 172.21.15.105:0/2782285910 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec14009510 con 0x7fec2c157910 2023-11-28T18:28:44.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec237fe640 1 -- 172.21.15.105:0/2782285910 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1400c940 con 0x7fec2c157910 2023-11-28T18:28:44.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec237fe640 1 -- 172.21.15.105:0/2782285910 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1400ccb0 con 0x7fec2c157910 2023-11-28T18:28:44.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec30a08640 1 -- 172.21.15.105:0/2782285910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec2c072360 con 0x7fec2c157910 2023-11-28T18:28:44.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.878+0000 7fec30a08640 1 -- 172.21.15.105:0/2782285910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec2c1fa090 con 0x7fec2c157910 2023-11-28T18:28:44.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.882+0000 7fec30a08640 1 -- 172.21.15.105:0/2782285910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec2c074580 con 0x7fec2c157910 2023-11-28T18:28:44.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.882+0000 7fec237fe640 1 -- 172.21.15.105:0/2782285910 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fec14013470 con 0x7fec2c157910 2023-11-28T18:28:44.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.882+0000 7fec237fe640 1 --2- 172.21.15.105:0/2782285910 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec0c0668d0 0x7fec0c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:44.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.882+0000 7fec237fe640 1 -- 172.21.15.105:0/2782285910 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fec14084590 con 0x7fec2c157910 2023-11-28T18:28:44.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.882+0000 7fec2ad9d640 1 --2- 172.21.15.105:0/2782285910 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec0c0668d0 0x7fec0c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:44.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.882+0000 7fec2ad9d640 1 --2- 172.21.15.105:0/2782285910 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec0c0668d0 0x7fec0c068d90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0af000 tx=0x7fec1c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:44.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:44.886+0000 7fec237fe640 1 -- 172.21.15.105:0/2782285910 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fec14051730 con 0x7fec2c157910 2023-11-28T18:28:45.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.038+0000 7fec30a08640 1 -- 172.21.15.105:0/2782285910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fec2c0cf860 con 0x7fec2c157910 2023-11-28T18:28:45.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.038+0000 7fec237fe640 1 -- 172.21.15.105:0/2782285910 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec14054d50 con 0x7fec2c157910 2023-11-28T18:28:45.042 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:28:45.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.038+0000 7fec30a08640 1 -- 172.21.15.105:0/2782285910 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec0c0668d0 msgr2=0x7fec0c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:45.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.038+0000 7fec30a08640 1 --2- 172.21.15.105:0/2782285910 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec0c0668d0 0x7fec0c068d90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0af000 tx=0x7fec1c0a7000 comp rx=0 tx=0).stop 2023-11-28T18:28:45.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.038+0000 7fec30a08640 1 -- 172.21.15.105:0/2782285910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec2c157910 msgr2=0x7fec2c0b3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:45.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.038+0000 7fec30a08640 1 --2- 172.21.15.105:0/2782285910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec2c157910 0x7fec2c0b3f10 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fec1400e8a0 tx=0x7fec1400eca0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.038+0000 7fec30a08640 1 -- 172.21.15.105:0/2782285910 shutdown_connections 2023-11-28T18:28:45.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.038+0000 7fec30a08640 1 --2- 172.21.15.105:0/2782285910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec2c15d8d0 0x7fec2c14a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.038+0000 7fec30a08640 1 --2- 172.21.15.105:0/2782285910 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fec0c0668d0 0x7fec0c068d90 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.038+0000 7fec30a08640 1 --2- 172.21.15.105:0/2782285910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec2c157910 0x7fec2c0b3f10 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.038+0000 7fec30a08640 1 --2- 172.21.15.105:0/2782285910 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec2c06cc90 0x7fec2c0c84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.038+0000 7fec30a08640 1 -- 172.21.15.105:0/2782285910 >> 172.21.15.105:0/2782285910 conn(0x7fec2c0ce9e0 msgr2=0x7fec2c156ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:45.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.038+0000 7fec30a08640 1 -- 172.21.15.105:0/2782285910 shutdown_connections 2023-11-28T18:28:45.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.042+0000 7fec30a08640 1 -- 172.21.15.105:0/2782285910 wait complete. 2023-11-28T18:28:45.063 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:28:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.202+0000 7fcef350f640 1 Processor -- start 2023-11-28T18:28:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.202+0000 7fcef350f640 1 -- start start 2023-11-28T18:28:45.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.206+0000 7fcef350f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fceec071960 0x7fceec15f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:45.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.206+0000 7fcef350f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceec0701d0 con 0x7fceec0b4290 2023-11-28T18:28:45.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.206+0000 7fcef350f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceec06df80 con 0x7fceec15b030 2023-11-28T18:28:45.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.206+0000 7fcef350f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fceec071df0 con 0x7fceec071960 2023-11-28T18:28:45.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.206+0000 7fcef0aaa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fceec071960 0x7fceec15f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:45.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.206+0000 7fcef12ab640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fceec0b4290 0x7fceec0b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43856/0 (socket says 172.21.15.105:43856) 2023-11-28T18:28:45.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.206+0000 7fcef0aaa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fceec071960 0x7fceec15f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47244/0 (socket says 172.21.15.105:47244) 2023-11-28T18:28:45.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.206+0000 7fcef12ab640 1 -- 172.21.15.105:0/1118451525 learned_addr learned my addr 172.21.15.105:0/1118451525 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:45.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.206+0000 7fcef0aaa640 1 -- 172.21.15.105:0/1118451525 >> v1:172.21.15.148:6789/0 conn(0x7fceec15b030 legacy=0x7fceec0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:45.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.206+0000 7fcef0aaa640 1 -- 172.21.15.105:0/1118451525 >> v1:172.21.15.105:6789/0 conn(0x7fceec0b4290 legacy=0x7fceec0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:45.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.206+0000 7fcef0aaa640 1 -- 172.21.15.105:0/1118451525 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceec0720d0 con 0x7fceec071960 2023-11-28T18:28:45.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.210+0000 7fcef0aaa640 1 --2- 172.21.15.105:0/1118451525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceec071960 0x7fceec15f3b0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0097c0 tx=0x7fcedc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b96bc917c124ab1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:45.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.210+0000 7fcee3fff640 1 -- 172.21.15.105:0/1118451525 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc0ab650 con 0x7fceec071960 2023-11-28T18:28:45.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.210+0000 7fcee3fff640 1 -- 172.21.15.105:0/1118451525 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcedc0ab7b0 con 0x7fceec071960 2023-11-28T18:28:45.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.210+0000 7fcee3fff640 1 -- 172.21.15.105:0/1118451525 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc0abb40 con 0x7fceec071960 2023-11-28T18:28:45.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.210+0000 7fcef350f640 1 -- 172.21.15.105:0/1118451525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceec071960 msgr2=0x7fceec15f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:45.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.210+0000 7fcef350f640 1 --2- 172.21.15.105:0/1118451525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceec071960 0x7fceec15f3b0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0097c0 tx=0x7fcedc01c450 comp rx=0 tx=0).stop 2023-11-28T18:28:45.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.210+0000 7fcef350f640 1 -- 172.21.15.105:0/1118451525 shutdown_connections 2023-11-28T18:28:45.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.210+0000 7fcef350f640 1 --2- 172.21.15.105:0/1118451525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceec071960 0x7fceec15f3b0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.210+0000 7fcef350f640 1 -- 172.21.15.105:0/1118451525 >> 172.21.15.105:0/1118451525 conn(0x7fceec0ceb90 msgr2=0x7fceec0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:45.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.210+0000 7fcef350f640 1 -- 172.21.15.105:0/1118451525 shutdown_connections 2023-11-28T18:28:45.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.210+0000 7fcef350f640 1 -- 172.21.15.105:0/1118451525 wait complete. 2023-11-28T18:28:45.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.210+0000 7fcef350f640 1 Processor -- start 2023-11-28T18:28:45.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.210+0000 7fcef350f640 1 -- start start 2023-11-28T18:28:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.210+0000 7fcef350f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fceec0b4290 0x7fceec09bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.210+0000 7fcef350f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fceec15b030 0x7fceec1467a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.210+0000 7fcef350f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceec159010 0x7fceec0af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.210+0000 7fcef350f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceec159890 con 0x7fceec159010 2023-11-28T18:28:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.210+0000 7fcef350f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceec159b70 con 0x7fceec0b4290 2023-11-28T18:28:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.210+0000 7fcef350f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceec159e50 con 0x7fceec15b030 2023-11-28T18:28:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.210+0000 7fcef1aac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceec159010 0x7fceec0af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.214+0000 7fcef1aac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceec159010 0x7fceec0af5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47252/0 (socket says 172.21.15.105:47252) 2023-11-28T18:28:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.214+0000 7fcef1aac640 1 -- 172.21.15.105:0/3426294528 learned_addr learned my addr 172.21.15.105:0/3426294528 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.214+0000 7fcef0aaa640 1 --2- 172.21.15.105:0/3426294528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fceec15b030 0x7fceec1467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.214+0000 7fcef12ab640 1 --2- 172.21.15.105:0/3426294528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fceec0b4290 0x7fceec09bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.214+0000 7fcef1aac640 1 -- 172.21.15.105:0/3426294528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fceec15b030 msgr2=0x7fceec1467a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.214+0000 7fcef1aac640 1 --2- 172.21.15.105:0/3426294528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fceec15b030 0x7fceec1467a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.214+0000 7fcef1aac640 1 -- 172.21.15.105:0/3426294528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fceec0b4290 msgr2=0x7fceec09bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.214+0000 7fcef1aac640 1 --2- 172.21.15.105:0/3426294528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fceec0b4290 0x7fceec09bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.214+0000 7fcef1aac640 1 -- 172.21.15.105:0/3426294528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc009420 con 0x7fceec159010 2023-11-28T18:28:45.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.214+0000 7fcef0aaa640 1 --2- 172.21.15.105:0/3426294528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fceec15b030 0x7fceec1467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:45.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.214+0000 7fcef12ab640 1 --2- 172.21.15.105:0/3426294528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fceec0b4290 0x7fceec09bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:45.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.214+0000 7fcef1aac640 1 --2- 172.21.15.105:0/3426294528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceec159010 0x7fceec0af5e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fcee400c730 tx=0x7fcee400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:45.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.214+0000 7fcee1ffb640 1 -- 172.21.15.105:0/3426294528 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee4003b80 con 0x7fceec159010 2023-11-28T18:28:45.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.214+0000 7fcee1ffb640 1 -- 172.21.15.105:0/3426294528 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee4003ce0 con 0x7fceec159010 2023-11-28T18:28:45.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.214+0000 7fcef350f640 1 -- 172.21.15.105:0/3426294528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceec15a130 con 0x7fceec159010 2023-11-28T18:28:45.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.214+0000 7fcee1ffb640 1 -- 172.21.15.105:0/3426294528 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee4005310 con 0x7fceec159010 2023-11-28T18:28:45.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.214+0000 7fcef350f640 1 -- 172.21.15.105:0/3426294528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceec1594a0 con 0x7fceec159010 2023-11-28T18:28:45.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.214+0000 7fcef350f640 1 -- 172.21.15.105:0/3426294528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceec15a880 con 0x7fceec159010 2023-11-28T18:28:45.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.214+0000 7fcee1ffb640 1 -- 172.21.15.105:0/3426294528 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fcee4005680 con 0x7fceec159010 2023-11-28T18:28:45.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.218+0000 7fcee1ffb640 1 --2- 172.21.15.105:0/3426294528 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fced0066900 0x7fced0068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:45.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.218+0000 7fcee1ffb640 1 -- 172.21.15.105:0/3426294528 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcee4084620 con 0x7fceec159010 2023-11-28T18:28:45.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.218+0000 7fcef12ab640 1 --2- 172.21.15.105:0/3426294528 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fced0066900 0x7fced0068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:45.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.218+0000 7fcef12ab640 1 --2- 172.21.15.105:0/3426294528 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fced0066900 0x7fced0068dc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcee8005890 tx=0x7fcee8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.222+0000 7fcee1ffb640 1 -- 172.21.15.105:0/3426294528 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcee40515f0 con 0x7fceec159010 2023-11-28T18:28:45.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.370+0000 7fcef350f640 1 -- 172.21.15.105:0/3426294528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fceec067570 con 0x7fceec159010 2023-11-28T18:28:45.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.370+0000 7fcee1ffb640 1 -- 172.21.15.105:0/3426294528 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcee4054c10 con 0x7fceec159010 2023-11-28T18:28:45.374 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:28:45.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.370+0000 7fcef350f640 1 -- 172.21.15.105:0/3426294528 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fced0066900 msgr2=0x7fced0068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:45.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.370+0000 7fcef350f640 1 --2- 172.21.15.105:0/3426294528 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fced0066900 0x7fced0068dc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcee8005890 tx=0x7fcee8009040 comp rx=0 tx=0).stop 2023-11-28T18:28:45.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.370+0000 7fcef350f640 1 -- 172.21.15.105:0/3426294528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceec159010 msgr2=0x7fceec0af5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:45.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.370+0000 7fcef350f640 1 --2- 172.21.15.105:0/3426294528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceec159010 0x7fceec0af5e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fcee400c730 tx=0x7fcee400cbf0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.370+0000 7fcef350f640 1 -- 172.21.15.105:0/3426294528 shutdown_connections 2023-11-28T18:28:45.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.370+0000 7fcef350f640 1 --2- 172.21.15.105:0/3426294528 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fced0066900 0x7fced0068dc0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.370+0000 7fcef350f640 1 --2- 172.21.15.105:0/3426294528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceec159010 0x7fceec0af5e0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.370+0000 7fcef350f640 1 --2- 172.21.15.105:0/3426294528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fceec15b030 0x7fceec1467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.370+0000 7fcef350f640 1 --2- 172.21.15.105:0/3426294528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fceec0b4290 0x7fceec09bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.370+0000 7fcef350f640 1 -- 172.21.15.105:0/3426294528 >> 172.21.15.105:0/3426294528 conn(0x7fceec0ceb90 msgr2=0x7fceec0cd100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:45.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.370+0000 7fcef350f640 1 -- 172.21.15.105:0/3426294528 shutdown_connections 2023-11-28T18:28:45.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.370+0000 7fcef350f640 1 -- 172.21.15.105:0/3426294528 wait complete. 2023-11-28T18:28:45.395 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:28:45.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.530+0000 7fe4ce617640 1 Processor -- start 2023-11-28T18:28:45.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.530+0000 7fe4ce617640 1 -- start start 2023-11-28T18:28:45.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.530+0000 7fe4ce617640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4c8159610 0x7fe4c80ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:45.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.530+0000 7fe4ce617640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe4c80cc760 0x7fe4c80ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:45.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.530+0000 7fe4ce617640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c80cde30 con 0x7fe4c806c7e0 2023-11-28T18:28:45.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.530+0000 7fe4ce617640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c81599f0 con 0x7fe4c80cc760 2023-11-28T18:28:45.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.530+0000 7fe4ce617640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c806de20 con 0x7fe4c8159610 2023-11-28T18:28:45.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.530+0000 7fe4ccbb4640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe4c806c7e0 0x7fe4c80abd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43860/0 (socket says 172.21.15.105:43860) 2023-11-28T18:28:45.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.530+0000 7fe4ccbb4640 1 -- 172.21.15.105:0/1602651959 learned_addr learned my addr 172.21.15.105:0/1602651959 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:45.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.530+0000 7fe4c7fff640 1 --2- 172.21.15.105:0/1602651959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4c8159610 0x7fe4c80ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:45.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.530+0000 7fe4c77fe640 1 --2- 172.21.15.105:0/1602651959 >> v2:172.21.15.148:3300/0 conn(0x7fe4c80cc760 0x7fe4c80ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:45.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.530+0000 7fe4c6ffd640 1 -- 172.21.15.105:0/1602651959 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4023007046 0 0) 0x7fe4c80cde30 con 0x7fe4c806c7e0 2023-11-28T18:28:45.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.530+0000 7fe4c6ffd640 1 -- 172.21.15.105:0/1602651959 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4b0003530 con 0x7fe4c806c7e0 2023-11-28T18:28:45.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.530+0000 7fe4c7fff640 1 -- 172.21.15.105:0/1602651959 >> v2:172.21.15.148:3300/0 conn(0x7fe4c80cc760 msgr2=0x7fe4c80ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:45.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.530+0000 7fe4c7fff640 1 --2- 172.21.15.105:0/1602651959 >> v2:172.21.15.148:3300/0 conn(0x7fe4c80cc760 0x7fe4c80ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.530+0000 7fe4c7fff640 1 -- 172.21.15.105:0/1602651959 >> v1:172.21.15.105:6789/0 conn(0x7fe4c806c7e0 legacy=0x7fe4c80abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:45.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.530+0000 7fe4c7fff640 1 -- 172.21.15.105:0/1602651959 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c80cd220 con 0x7fe4c8159610 2023-11-28T18:28:45.539 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.534+0000 7fe4c7fff640 1 --2- 172.21.15.105:0/1602651959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4c8159610 0x7fe4c80ce2c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800a960 tx=0x7fe4c80acca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61e28c2a3937aeca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:45.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.534+0000 7fe4c6ffd640 1 -- 172.21.15.105:0/1602651959 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b80a68d0 con 0x7fe4c8159610 2023-11-28T18:28:45.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.534+0000 7fe4c6ffd640 1 -- 172.21.15.105:0/1602651959 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b80a6a30 con 0x7fe4c8159610 2023-11-28T18:28:45.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.534+0000 7fe4c6ffd640 1 -- 172.21.15.105:0/1602651959 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b80a6da0 con 0x7fe4c8159610 2023-11-28T18:28:45.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.534+0000 7fe4ce617640 1 -- 172.21.15.105:0/1602651959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4c8159610 msgr2=0x7fe4c80ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:45.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.534+0000 7fe4ce617640 1 --2- 172.21.15.105:0/1602651959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4c8159610 0x7fe4c80ce2c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800a960 tx=0x7fe4c80acca0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.534+0000 7fe4ce617640 1 -- 172.21.15.105:0/1602651959 shutdown_connections 2023-11-28T18:28:45.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.534+0000 7fe4ce617640 1 --2- 172.21.15.105:0/1602651959 >> v2:172.21.15.148:3300/0 conn(0x7fe4c80cc760 0x7fe4c80ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.534+0000 7fe4ce617640 1 --2- 172.21.15.105:0/1602651959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4c8159610 0x7fe4c80ce2c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.534+0000 7fe4ce617640 1 -- 172.21.15.105:0/1602651959 >> 172.21.15.105:0/1602651959 conn(0x7fe4c80ceb90 msgr2=0x7fe4c80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:45.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.534+0000 7fe4ce617640 1 -- 172.21.15.105:0/1602651959 shutdown_connections 2023-11-28T18:28:45.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.534+0000 7fe4ce617640 1 -- 172.21.15.105:0/1602651959 wait complete. 2023-11-28T18:28:45.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4ce617640 1 Processor -- start 2023-11-28T18:28:45.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4ce617640 1 -- start start 2023-11-28T18:28:45.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4ce617640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c806c7e0 0x7fe4c8158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:45.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4ce617640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4c80cc760 0x7fe4c8159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:45.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4ce617640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4c8159610 0x7fe4c8154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:45.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4ce617640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c815b690 con 0x7fe4c8159610 2023-11-28T18:28:45.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4ce617640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c815a280 con 0x7fe4c806c7e0 2023-11-28T18:28:45.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4ce617640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c815a790 con 0x7fe4c80cc760 2023-11-28T18:28:45.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4ccbb4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4c8159610 0x7fe4c8154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:45.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4ccbb4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4c8159610 0x7fe4c8154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47262/0 (socket says 172.21.15.105:47262) 2023-11-28T18:28:45.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4ccbb4640 1 -- 172.21.15.105:0/2819784013 learned_addr learned my addr 172.21.15.105:0/2819784013 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:45.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4c7fff640 1 --2- 172.21.15.105:0/2819784013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c806c7e0 0x7fe4c8158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:45.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4c77fe640 1 --2- 172.21.15.105:0/2819784013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4c80cc760 0x7fe4c8159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:45.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4ccbb4640 1 -- 172.21.15.105:0/2819784013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4c80cc760 msgr2=0x7fe4c8159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:45.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4ccbb4640 1 --2- 172.21.15.105:0/2819784013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4c80cc760 0x7fe4c8159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4ccbb4640 1 -- 172.21.15.105:0/2819784013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c806c7e0 msgr2=0x7fe4c8158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:45.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4ccbb4640 1 --2- 172.21.15.105:0/2819784013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c806c7e0 0x7fe4c8158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4ccbb4640 1 -- 172.21.15.105:0/2819784013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b800a5c0 con 0x7fe4c8159610 2023-11-28T18:28:45.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4ccbb4640 1 --2- 172.21.15.105:0/2819784013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4c8159610 0x7fe4c8154050 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000cb70 tx=0x7fe4c000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:45.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4c4ff9640 1 -- 172.21.15.105:0/2819784013 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c000ad40 con 0x7fe4c8159610 2023-11-28T18:28:45.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4c4ff9640 1 -- 172.21.15.105:0/2819784013 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c0012400 con 0x7fe4c8159610 2023-11-28T18:28:45.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4ce617640 1 -- 172.21.15.105:0/2819784013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c806d180 con 0x7fe4c8159610 2023-11-28T18:28:45.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4c4ff9640 1 -- 172.21.15.105:0/2819784013 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c000aea0 con 0x7fe4c8159610 2023-11-28T18:28:45.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.538+0000 7fe4ce617640 1 -- 172.21.15.105:0/2819784013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c815a4b0 con 0x7fe4c8159610 2023-11-28T18:28:45.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.542+0000 7fe4ce617640 1 -- 172.21.15.105:0/2819784013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c8067570 con 0x7fe4c8159610 2023-11-28T18:28:45.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.542+0000 7fe4c4ff9640 1 -- 172.21.15.105:0/2819784013 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fe4c0007070 con 0x7fe4c8159610 2023-11-28T18:28:45.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.542+0000 7fe4c4ff9640 1 --2- 172.21.15.105:0/2819784013 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe4b40668d0 0x7fe4b4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:45.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.542+0000 7fe4c4ff9640 1 -- 172.21.15.105:0/2819784013 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe4c0084540 con 0x7fe4c8159610 2023-11-28T18:28:45.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.542+0000 7fe4c7fff640 1 --2- 172.21.15.105:0/2819784013 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe4b40668d0 0x7fe4b4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:45.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.542+0000 7fe4c7fff640 1 --2- 172.21.15.105:0/2819784013 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe4b40668d0 0x7fe4b4068d90 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80b0000 tx=0x7fe4b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:45.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.546+0000 7fe4c4ff9640 1 -- 172.21.15.105:0/2819784013 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe4c00516e0 con 0x7fe4c8159610 2023-11-28T18:28:45.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.694+0000 7fe4ce617640 1 -- 172.21.15.105:0/2819784013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe4c80caba0 con 0x7fe4c8159610 2023-11-28T18:28:45.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.694+0000 7fe4c4ff9640 1 -- 172.21.15.105:0/2819784013 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe4c0017020 con 0x7fe4c8159610 2023-11-28T18:28:45.701 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:28:45.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.698+0000 7fe4ce617640 1 -- 172.21.15.105:0/2819784013 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe4b40668d0 msgr2=0x7fe4b4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:45.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.698+0000 7fe4ce617640 1 --2- 172.21.15.105:0/2819784013 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe4b40668d0 0x7fe4b4068d90 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80b0000 tx=0x7fe4b8008040 comp rx=0 tx=0).stop 2023-11-28T18:28:45.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.698+0000 7fe4ce617640 1 -- 172.21.15.105:0/2819784013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4c8159610 msgr2=0x7fe4c8154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:45.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.698+0000 7fe4ce617640 1 --2- 172.21.15.105:0/2819784013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4c8159610 0x7fe4c8154050 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000cb70 tx=0x7fe4c000a430 comp rx=0 tx=0).stop 2023-11-28T18:28:45.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.698+0000 7fe4ce617640 1 -- 172.21.15.105:0/2819784013 shutdown_connections 2023-11-28T18:28:45.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.698+0000 7fe4ce617640 1 --2- 172.21.15.105:0/2819784013 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe4b40668d0 0x7fe4b4068d90 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.698+0000 7fe4ce617640 1 --2- 172.21.15.105:0/2819784013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4c8159610 0x7fe4c8154050 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.698+0000 7fe4ce617640 1 --2- 172.21.15.105:0/2819784013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4c80cc760 0x7fe4c8159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.698+0000 7fe4ce617640 1 --2- 172.21.15.105:0/2819784013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c806c7e0 0x7fe4c8158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.698+0000 7fe4ce617640 1 -- 172.21.15.105:0/2819784013 >> 172.21.15.105:0/2819784013 conn(0x7fe4c80ceb90 msgr2=0x7fe4c80aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:45.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.698+0000 7fe4ce617640 1 -- 172.21.15.105:0/2819784013 shutdown_connections 2023-11-28T18:28:45.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.698+0000 7fe4ce617640 1 -- 172.21.15.105:0/2819784013 wait complete. 2023-11-28T18:28:45.722 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T18:28:45.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.854+0000 7f908cc79640 1 Processor -- start 2023-11-28T18:28:45.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.854+0000 7f908cc79640 1 -- start start 2023-11-28T18:28:45.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.858+0000 7f908cc79640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9088159610 0x7f90880ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:45.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.858+0000 7f908cc79640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f90880cc760 0x7f90880ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:45.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.858+0000 7f908cc79640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90880cde30 con 0x7f908806c7e0 2023-11-28T18:28:45.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.858+0000 7f908cc79640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90881599f0 con 0x7f9088159610 2023-11-28T18:28:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.858+0000 7f908cc79640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f908806de20 con 0x7f90880cc760 2023-11-28T18:28:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.858+0000 7f908659c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9088159610 0x7f90880ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.858+0000 7f9086d9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f908806c7e0 0x7f90880abd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43866/0 (socket says 172.21.15.105:43866) 2023-11-28T18:28:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.858+0000 7f9086d9d640 1 -- 172.21.15.105:0/4252924125 learned_addr learned my addr 172.21.15.105:0/4252924125 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.858+0000 7f907dd9b640 1 --2- 172.21.15.105:0/4252924125 >> v2:172.21.15.148:3300/0 conn(0x7f90880cc760 0x7f90880ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.858+0000 7f908659c640 1 -- 172.21.15.105:0/4252924125 >> v2:172.21.15.148:3300/0 conn(0x7f90880cc760 msgr2=0x7f90880ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.858+0000 7f9085d9b640 1 -- 172.21.15.105:0/4252924125 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330658130 0 0) 0x7f90880cde30 con 0x7f908806c7e0 2023-11-28T18:28:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.858+0000 7f908659c640 1 --2- 172.21.15.105:0/4252924125 >> v2:172.21.15.148:3300/0 conn(0x7f90880cc760 0x7f90880ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.858+0000 7f908659c640 1 -- 172.21.15.105:0/4252924125 >> v1:172.21.15.105:6789/0 conn(0x7f908806c7e0 legacy=0x7f90880abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.858+0000 7f908659c640 1 -- 172.21.15.105:0/4252924125 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90880cd220 con 0x7f9088159610 2023-11-28T18:28:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.858+0000 7f907dd9b640 1 --2- 172.21.15.105:0/4252924125 >> v2:172.21.15.148:3300/0 conn(0x7f90880cc760 0x7f90880ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.862+0000 7f908659c640 1 --2- 172.21.15.105:0/4252924125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9088159610 0x7f90880ce2c0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f90880cfc90 tx=0x7f90880acca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90c4071228834d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:45.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.862+0000 7f9085d9b640 1 -- 172.21.15.105:0/4252924125 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90780a7070 con 0x7f9088159610 2023-11-28T18:28:45.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.862+0000 7f9085d9b640 1 -- 172.21.15.105:0/4252924125 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90780ab430 con 0x7f9088159610 2023-11-28T18:28:45.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.862+0000 7f9085d9b640 1 -- 172.21.15.105:0/4252924125 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90780ab7c0 con 0x7f9088159610 2023-11-28T18:28:45.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.862+0000 7f908cc79640 1 -- 172.21.15.105:0/4252924125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9088159610 msgr2=0x7f90880ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:45.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.862+0000 7f908cc79640 1 --2- 172.21.15.105:0/4252924125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9088159610 0x7f90880ce2c0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f90880cfc90 tx=0x7f90880acca0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.862+0000 7f908cc79640 1 -- 172.21.15.105:0/4252924125 shutdown_connections 2023-11-28T18:28:45.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.862+0000 7f908cc79640 1 --2- 172.21.15.105:0/4252924125 >> v2:172.21.15.148:3300/0 conn(0x7f90880cc760 0x7f90880ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.862+0000 7f908cc79640 1 --2- 172.21.15.105:0/4252924125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9088159610 0x7f90880ce2c0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.862+0000 7f908cc79640 1 -- 172.21.15.105:0/4252924125 >> 172.21.15.105:0/4252924125 conn(0x7f90880ceb90 msgr2=0x7f90880cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:45.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.862+0000 7f908cc79640 1 -- 172.21.15.105:0/4252924125 shutdown_connections 2023-11-28T18:28:45.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.862+0000 7f908cc79640 1 -- 172.21.15.105:0/4252924125 wait complete. 2023-11-28T18:28:45.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.862+0000 7f908cc79640 1 Processor -- start 2023-11-28T18:28:45.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.862+0000 7f908cc79640 1 -- start start 2023-11-28T18:28:45.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f908cc79640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f908806c7e0 0x7f908814dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:45.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f908cc79640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90880cc760 0x7f90881507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:45.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f908cc79640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9088159610 0x7f9088150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:45.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f908cc79640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908814a670 con 0x7f908806c7e0 2023-11-28T18:28:45.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f908cc79640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90880b4500 con 0x7f90880cc760 2023-11-28T18:28:45.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f908cc79640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90880b4990 con 0x7f9088159610 2023-11-28T18:28:45.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f908659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f908806c7e0 0x7f908814dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:45.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f908659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f908806c7e0 0x7f908814dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47290/0 (socket says 172.21.15.105:47290) 2023-11-28T18:28:45.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f908659c640 1 -- 172.21.15.105:0/1400094083 learned_addr learned my addr 172.21.15.105:0/1400094083 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:45.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f907dd9b640 1 --2- 172.21.15.105:0/1400094083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90880cc760 0x7f90881507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:45.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f9086d9d640 1 --2- 172.21.15.105:0/1400094083 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9088159610 0x7f9088150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:45.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f908659c640 1 -- 172.21.15.105:0/1400094083 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9088159610 msgr2=0x7f9088150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:45.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f908659c640 1 --2- 172.21.15.105:0/1400094083 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9088159610 0x7f9088150d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f908659c640 1 -- 172.21.15.105:0/1400094083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90880cc760 msgr2=0x7f90881507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:45.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f908659c640 1 --2- 172.21.15.105:0/1400094083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90880cc760 0x7f90881507f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:45.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f908659c640 1 -- 172.21.15.105:0/1400094083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9078009420 con 0x7f908806c7e0 2023-11-28T18:28:45.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f908659c640 1 --2- 172.21.15.105:0/1400094083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f908806c7e0 0x7f908814dce0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f907801a060 tx=0x7f907801ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:45.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f907f7fe640 1 -- 172.21.15.105:0/1400094083 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90780b4800 con 0x7f908806c7e0 2023-11-28T18:28:45.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f907f7fe640 1 -- 172.21.15.105:0/1400094083 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90780abe90 con 0x7f908806c7e0 2023-11-28T18:28:45.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f908cc79640 1 -- 172.21.15.105:0/1400094083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90880b3ef0 con 0x7f908806c7e0 2023-11-28T18:28:45.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f907f7fe640 1 -- 172.21.15.105:0/1400094083 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90780b4bc0 con 0x7f908806c7e0 2023-11-28T18:28:45.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f908cc79640 1 -- 172.21.15.105:0/1400094083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90880b4190 con 0x7f908806c7e0 2023-11-28T18:28:45.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f908cc79640 1 -- 172.21.15.105:0/1400094083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90880b3d50 con 0x7f908806c7e0 2023-11-28T18:28:45.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.866+0000 7f907f7fe640 1 -- 172.21.15.105:0/1400094083 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f90780b4d20 con 0x7f908806c7e0 2023-11-28T18:28:45.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.870+0000 7f907f7fe640 1 --2- 172.21.15.105:0/1400094083 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f906c0668d0 0x7f906c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:45.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.870+0000 7f907f7fe640 1 -- 172.21.15.105:0/1400094083 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f907811dff0 con 0x7f908806c7e0 2023-11-28T18:28:45.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.870+0000 7f907dd9b640 1 --2- 172.21.15.105:0/1400094083 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f906c0668d0 0x7f906c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:45.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.870+0000 7f907dd9b640 1 --2- 172.21.15.105:0/1400094083 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f906c0668d0 0x7f906c068d90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f90880b3b40 tx=0x7f9070008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:45.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:45.874+0000 7f907f7fe640 1 -- 172.21.15.105:0/1400094083 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f90880b3d50 con 0x7f908806c7e0 2023-11-28T18:28:46.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.030+0000 7f908cc79640 1 -- 172.21.15.105:0/1400094083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f90880b3d50 con 0x7f908806c7e0 2023-11-28T18:28:46.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.030+0000 7f907f7fe640 1 -- 172.21.15.105:0/1400094083 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v78) v1 ==== 80+0+0 (secure 0 0 0) 0x7f90880b3d50 con 0x7f908806c7e0 2023-11-28T18:28:46.036 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2023-11-28T18:28:46.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.034+0000 7f908cc79640 1 -- 172.21.15.105:0/1400094083 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f906c0668d0 msgr2=0x7f906c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:46.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.034+0000 7f908cc79640 1 --2- 172.21.15.105:0/1400094083 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f906c0668d0 0x7f906c068d90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f90880b3b40 tx=0x7f9070008040 comp rx=0 tx=0).stop 2023-11-28T18:28:46.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.034+0000 7f908cc79640 1 -- 172.21.15.105:0/1400094083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f908806c7e0 msgr2=0x7f908814dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:46.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.034+0000 7f908cc79640 1 --2- 172.21.15.105:0/1400094083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f908806c7e0 0x7f908814dce0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f907801a060 tx=0x7f907801ac70 comp rx=0 tx=0).stop 2023-11-28T18:28:46.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.034+0000 7f908cc79640 1 -- 172.21.15.105:0/1400094083 shutdown_connections 2023-11-28T18:28:46.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.034+0000 7f908cc79640 1 --2- 172.21.15.105:0/1400094083 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9088159610 0x7f9088150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:46.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.034+0000 7f908cc79640 1 --2- 172.21.15.105:0/1400094083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90880cc760 0x7f90881507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:46.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.034+0000 7f908cc79640 1 --2- 172.21.15.105:0/1400094083 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f906c0668d0 0x7f906c068d90 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:46.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.034+0000 7f908cc79640 1 --2- 172.21.15.105:0/1400094083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f908806c7e0 0x7f908814dce0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:46.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.034+0000 7f908cc79640 1 -- 172.21.15.105:0/1400094083 >> 172.21.15.105:0/1400094083 conn(0x7f90880ceb90 msgr2=0x7f90880acca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:46.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.034+0000 7f908cc79640 1 -- 172.21.15.105:0/1400094083 shutdown_connections 2023-11-28T18:28:46.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.034+0000 7f908cc79640 1 -- 172.21.15.105:0/1400094083 wait complete. 2023-11-28T18:28:46.058 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2023-11-28T18:28:46.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:28:46.058 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-28T18:28:46.077 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[58072]: fuse finished with error 0 and tester_r 0 2023-11-28T18:28:46.080 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:28:46.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:28:46.081 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T18:28:46.252 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:28:46.253 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:28:46.262 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:28:46.262 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:28:46.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:28:46.262 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T18:28:46.317 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-28T18:28:46.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:28:46.318 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:28:46.318 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:28:46.327 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:28:46.327 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:28:46.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:28:46.327 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T18:28:46.376 INFO:teuthology.orchestra.run.smithi105.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-28T18:28:46.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:28:46.378 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:28:46.378 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:28:46.387 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:28:46.387 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:28:46.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:28:46.387 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T18:28:46.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:28:46.439 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-28T18:28:46.439 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:28:46.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.570+0000 7ffa7eac8640 1 Processor -- start 2023-11-28T18:28:46.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.570+0000 7ffa7eac8640 1 -- start start 2023-11-28T18:28:46.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.570+0000 7ffa7eac8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffa7815d2d0 0x7ffa780cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:46.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.570+0000 7ffa7eac8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffa780cf960 0x7ffa78166fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:46.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.570+0000 7ffa7eac8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa780cfdb0 con 0x7ffa7815cef0 2023-11-28T18:28:46.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.570+0000 7ffa7eac8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa780b4780 con 0x7ffa780cf960 2023-11-28T18:28:46.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.570+0000 7ffa7eac8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa780ce840 con 0x7ffa7815d2d0 2023-11-28T18:28:46.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.574+0000 7ffa7e2c7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffa780cf960 0x7ffa78166fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:46.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.574+0000 7ffa7e2c7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffa780cf960 0x7ffa78166fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47306/0 (socket says 172.21.15.105:47306) 2023-11-28T18:28:46.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.574+0000 7ffa7e2c7640 1 -- 172.21.15.105:0/203460697 learned_addr learned my addr 172.21.15.105:0/203460697 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:46.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.574+0000 7ffa7d2c5640 1 --2- 172.21.15.105:0/203460697 >> v2:172.21.15.148:3300/0 conn(0x7ffa7815d2d0 0x7ffa780cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:46.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.574+0000 7ffa7e2c7640 1 -- 172.21.15.105:0/203460697 >> v2:172.21.15.148:3300/0 conn(0x7ffa7815d2d0 msgr2=0x7ffa780cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:46.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.574+0000 7ffa7e2c7640 1 --2- 172.21.15.105:0/203460697 >> v2:172.21.15.148:3300/0 conn(0x7ffa7815d2d0 0x7ffa780cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:46.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.574+0000 7ffa7e2c7640 1 -- 172.21.15.105:0/203460697 >> v1:172.21.15.148:6789/0 conn(0x7ffa7815cef0 legacy=0x7ffa780d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:46.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.574+0000 7ffa7e2c7640 1 -- 172.21.15.105:0/203460697 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa780cd990 con 0x7ffa780cf960 2023-11-28T18:28:46.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.574+0000 7ffa7e2c7640 1 --2- 172.21.15.105:0/203460697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa780cf960 0x7ffa78166fd0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ffa7808bac0 tx=0x7ffa7001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366cc30de341472c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:46.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.574+0000 7ffa7cac4640 1 -- 172.21.15.105:0/203460697 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa700a57e0 con 0x7ffa780cf960 2023-11-28T18:28:46.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.574+0000 7ffa7cac4640 1 -- 172.21.15.105:0/203460697 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa700a5940 con 0x7ffa780cf960 2023-11-28T18:28:46.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.574+0000 7ffa7cac4640 1 -- 172.21.15.105:0/203460697 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa700a5cd0 con 0x7ffa780cf960 2023-11-28T18:28:46.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7eac8640 1 -- 172.21.15.105:0/203460697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa780cf960 msgr2=0x7ffa78166fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:46.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7eac8640 1 --2- 172.21.15.105:0/203460697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa780cf960 0x7ffa78166fd0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ffa7808bac0 tx=0x7ffa7001c920 comp rx=0 tx=0).stop 2023-11-28T18:28:46.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7eac8640 1 -- 172.21.15.105:0/203460697 shutdown_connections 2023-11-28T18:28:46.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7eac8640 1 --2- 172.21.15.105:0/203460697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa780cf960 0x7ffa78166fd0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:46.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7eac8640 1 --2- 172.21.15.105:0/203460697 >> v2:172.21.15.148:3300/0 conn(0x7ffa7815d2d0 0x7ffa780cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:46.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7eac8640 1 -- 172.21.15.105:0/203460697 >> 172.21.15.105:0/203460697 conn(0x7ffa780d2e10 msgr2=0x7ffa780d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:46.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7eac8640 1 -- 172.21.15.105:0/203460697 shutdown_connections 2023-11-28T18:28:46.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7eac8640 1 -- 172.21.15.105:0/203460697 wait complete. 2023-11-28T18:28:46.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7eac8640 1 Processor -- start 2023-11-28T18:28:46.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7eac8640 1 -- start start 2023-11-28T18:28:46.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7eac8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa780cf960 0x7ffa780c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:46.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7eac8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa7815cef0 0x7ffa780cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:46.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7eac8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa7815d2d0 0x7ffa7815a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:46.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7eac8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa780b83e0 con 0x7ffa7815d2d0 2023-11-28T18:28:46.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7eac8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7815c610 con 0x7ffa7815cef0 2023-11-28T18:28:46.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7eac8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa780ccd20 con 0x7ffa780cf960 2023-11-28T18:28:46.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7e2c7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa7815d2d0 0x7ffa7815a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:46.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7e2c7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa7815d2d0 0x7ffa7815a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47312/0 (socket says 172.21.15.105:47312) 2023-11-28T18:28:46.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7e2c7640 1 -- 172.21.15.105:0/1525278459 learned_addr learned my addr 172.21.15.105:0/1525278459 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:46.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7d2c5640 1 --2- 172.21.15.105:0/1525278459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa7815cef0 0x7ffa780cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:46.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7dac6640 1 --2- 172.21.15.105:0/1525278459 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa780cf960 0x7ffa780c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:46.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7e2c7640 1 -- 172.21.15.105:0/1525278459 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa780cf960 msgr2=0x7ffa780c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:46.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7e2c7640 1 --2- 172.21.15.105:0/1525278459 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa780cf960 0x7ffa780c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:46.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7e2c7640 1 -- 172.21.15.105:0/1525278459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa7815cef0 msgr2=0x7ffa780cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:46.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7e2c7640 1 --2- 172.21.15.105:0/1525278459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa7815cef0 0x7ffa780cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:46.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7e2c7640 1 -- 172.21.15.105:0/1525278459 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa70009b30 con 0x7ffa7815d2d0 2023-11-28T18:28:46.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.578+0000 7ffa7dac6640 1 --2- 172.21.15.105:0/1525278459 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa780cf960 0x7ffa780c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:46.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.582+0000 7ffa7e2c7640 1 --2- 172.21.15.105:0/1525278459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa7815d2d0 0x7ffa7815a5a0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7ffa70019530 tx=0x7ffa700196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:46.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.582+0000 7ffa6e7fc640 1 -- 172.21.15.105:0/1525278459 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa700b8670 con 0x7ffa7815d2d0 2023-11-28T18:28:46.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.582+0000 7ffa6e7fc640 1 -- 172.21.15.105:0/1525278459 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa700b87d0 con 0x7ffa7815d2d0 2023-11-28T18:28:46.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.582+0000 7ffa7eac8640 1 -- 172.21.15.105:0/1525278459 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa780b7d90 con 0x7ffa7815d2d0 2023-11-28T18:28:46.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.582+0000 7ffa6e7fc640 1 -- 172.21.15.105:0/1525278459 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa700b8b60 con 0x7ffa7815d2d0 2023-11-28T18:28:46.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.582+0000 7ffa7eac8640 1 -- 172.21.15.105:0/1525278459 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa7815aae0 con 0x7ffa7815d2d0 2023-11-28T18:28:46.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.582+0000 7ffa7eac8640 1 -- 172.21.15.105:0/1525278459 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa780b8590 con 0x7ffa7815d2d0 2023-11-28T18:28:46.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.582+0000 7ffa6e7fc640 1 -- 172.21.15.105:0/1525278459 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ffa700b8cc0 con 0x7ffa7815d2d0 2023-11-28T18:28:46.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.582+0000 7ffa6e7fc640 1 --2- 172.21.15.105:0/1525278459 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffa5c0668d0 0x7ffa5c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:46.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.582+0000 7ffa6e7fc640 1 -- 172.21.15.105:0/1525278459 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ffa7011fee0 con 0x7ffa7815d2d0 2023-11-28T18:28:46.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.582+0000 7ffa7dac6640 1 --2- 172.21.15.105:0/1525278459 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffa5c0668d0 0x7ffa5c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:46.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.586+0000 7ffa7dac6640 1 --2- 172.21.15.105:0/1525278459 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffa5c0668d0 0x7ffa5c068d90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ffa7806b4a0 tx=0x7ffa74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:46.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.586+0000 7ffa6e7fc640 1 -- 172.21.15.105:0/1525278459 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ffa700ed170 con 0x7ffa7815d2d0 2023-11-28T18:28:46.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.766+0000 7ffa7eac8640 1 -- 172.21.15.105:0/1525278459 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa780b8520 con 0x7ffa7815d2d0 2023-11-28T18:28:46.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.766+0000 7ffa6e7fc640 1 -- 172.21.15.105:0/1525278459 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4950 (secure 0 0 0) 0x7ffa700f0790 con 0x7ffa7815d2d0 2023-11-28T18:28:46.771 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:46.771 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":77,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1455698212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":1455698212},{"type":"v1","addr":"172.21.15.105:6837","nonce":1455698212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:28:13.097621+0000","modified":"2023-11-28T18:28:43.851902+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7194,"mds_1":6862,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"b","rank":2,"incarnation":75,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6834/759522673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":759522673},{"type":"v1","addr":"172.21.15.148:6834","nonce":759522673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6862":{"gid":6862,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6835/2489361300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2489361300},{"type":"v1","addr":"172.21.15.148:6835","nonce":2489361300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7194":{"gid":7194,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6836/188173955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":188173955},{"type":"v1","addr":"172.21.15.105:6836","nonce":188173955}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-28T18:28:46.772 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 77 2023-11-28T18:28:46.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.766+0000 7ffa7eac8640 1 -- 172.21.15.105:0/1525278459 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffa5c0668d0 msgr2=0x7ffa5c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:46.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.766+0000 7ffa7eac8640 1 --2- 172.21.15.105:0/1525278459 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffa5c0668d0 0x7ffa5c068d90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ffa7806b4a0 tx=0x7ffa74009040 comp rx=0 tx=0).stop 2023-11-28T18:28:46.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.766+0000 7ffa7eac8640 1 -- 172.21.15.105:0/1525278459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa7815d2d0 msgr2=0x7ffa7815a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:46.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.766+0000 7ffa7eac8640 1 --2- 172.21.15.105:0/1525278459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa7815d2d0 0x7ffa7815a5a0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7ffa70019530 tx=0x7ffa700196c0 comp rx=0 tx=0).stop 2023-11-28T18:28:46.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.766+0000 7ffa7eac8640 1 -- 172.21.15.105:0/1525278459 shutdown_connections 2023-11-28T18:28:46.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.766+0000 7ffa7eac8640 1 --2- 172.21.15.105:0/1525278459 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffa5c0668d0 0x7ffa5c068d90 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:46.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.766+0000 7ffa7eac8640 1 --2- 172.21.15.105:0/1525278459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa7815d2d0 0x7ffa7815a5a0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:46.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.766+0000 7ffa7eac8640 1 --2- 172.21.15.105:0/1525278459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa7815cef0 0x7ffa780cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:46.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.766+0000 7ffa7eac8640 1 --2- 172.21.15.105:0/1525278459 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa780cf960 0x7ffa780c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:46.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.766+0000 7ffa7eac8640 1 -- 172.21.15.105:0/1525278459 >> 172.21.15.105:0/1525278459 conn(0x7ffa780d2e10 msgr2=0x7ffa780d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:46.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.766+0000 7ffa7eac8640 1 -- 172.21.15.105:0/1525278459 shutdown_connections 2023-11-28T18:28:46.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.766+0000 7ffa7eac8640 1 -- 172.21.15.105:0/1525278459 wait complete. 2023-11-28T18:28:46.791 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:28:46.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.922+0000 7f7b6c2bb640 1 Processor -- start 2023-11-28T18:28:46.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.922+0000 7f7b6c2bb640 1 -- start start 2023-11-28T18:28:46.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.926+0000 7f7b6c2bb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7b640ce2c0 0x7f7b6406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:46.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.926+0000 7f7b6c2bb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7b6406cd20 0x7f7b640abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:46.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.926+0000 7f7b6c2bb640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b640cde30 con 0x7f7b64159610 2023-11-28T18:28:46.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.926+0000 7f7b6c2bb640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b6406d190 con 0x7f7b6406cd20 2023-11-28T18:28:46.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.926+0000 7f7b6c2bb640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b641599f0 con 0x7f7b640ce2c0 2023-11-28T18:28:46.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.926+0000 7f7b6a858640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7b6406cd20 0x7f7b640abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:46.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.926+0000 7f7b6a057640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7b64159610 0x7f7b640cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43880/0 (socket says 172.21.15.105:43880) 2023-11-28T18:28:46.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.926+0000 7f7b6a057640 1 -- 172.21.15.105:0/4207154445 learned_addr learned my addr 172.21.15.105:0/4207154445 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:46.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.926+0000 7f7b69856640 1 --2- 172.21.15.105:0/4207154445 >> v2:172.21.15.148:3300/0 conn(0x7f7b640ce2c0 0x7f7b6406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:46.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.926+0000 7f7b6a858640 1 -- 172.21.15.105:0/4207154445 >> v2:172.21.15.148:3300/0 conn(0x7f7b640ce2c0 msgr2=0x7f7b6406c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:46.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.926+0000 7f7b6a858640 1 --2- 172.21.15.105:0/4207154445 >> v2:172.21.15.148:3300/0 conn(0x7f7b640ce2c0 0x7f7b6406c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:46.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.926+0000 7f7b6a858640 1 -- 172.21.15.105:0/4207154445 >> v1:172.21.15.105:6789/0 conn(0x7f7b64159610 legacy=0x7f7b640cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:46.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.926+0000 7f7b6a858640 1 -- 172.21.15.105:0/4207154445 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6406de20 con 0x7f7b6406cd20 2023-11-28T18:28:46.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.926+0000 7f7b69055640 1 -- 172.21.15.105:0/4207154445 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810305930 0 0) 0x7f7b640cde30 con 0x7f7b64159610 2023-11-28T18:28:46.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.930+0000 7f7b6a858640 1 --2- 172.21.15.105:0/4207154445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b6406cd20 0x7f7b640abd80 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f7b640cfc90 tx=0x7f7b5c01d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8f27d178c484a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:46.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.930+0000 7f7b69055640 1 -- 172.21.15.105:0/4207154445 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c0a6960 con 0x7f7b6406cd20 2023-11-28T18:28:46.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.930+0000 7f7b69055640 1 -- 172.21.15.105:0/4207154445 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5c0a6ac0 con 0x7f7b6406cd20 2023-11-28T18:28:46.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.930+0000 7f7b69055640 1 -- 172.21.15.105:0/4207154445 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c0a6e50 con 0x7f7b6406cd20 2023-11-28T18:28:46.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.930+0000 7f7b6c2bb640 1 -- 172.21.15.105:0/4207154445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b6406cd20 msgr2=0x7f7b640abd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:46.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.930+0000 7f7b6c2bb640 1 --2- 172.21.15.105:0/4207154445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b6406cd20 0x7f7b640abd80 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f7b640cfc90 tx=0x7f7b5c01d9f0 comp rx=0 tx=0).stop 2023-11-28T18:28:46.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.930+0000 7f7b6c2bb640 1 -- 172.21.15.105:0/4207154445 shutdown_connections 2023-11-28T18:28:46.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.930+0000 7f7b6c2bb640 1 --2- 172.21.15.105:0/4207154445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b6406cd20 0x7f7b640abd80 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:46.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.930+0000 7f7b6c2bb640 1 --2- 172.21.15.105:0/4207154445 >> v2:172.21.15.148:3300/0 conn(0x7f7b640ce2c0 0x7f7b6406c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:46.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.930+0000 7f7b6c2bb640 1 -- 172.21.15.105:0/4207154445 >> 172.21.15.105:0/4207154445 conn(0x7f7b640ceb90 msgr2=0x7f7b640cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:46.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.930+0000 7f7b6c2bb640 1 -- 172.21.15.105:0/4207154445 shutdown_connections 2023-11-28T18:28:46.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.930+0000 7f7b6c2bb640 1 -- 172.21.15.105:0/4207154445 wait complete. 2023-11-28T18:28:46.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.930+0000 7f7b6c2bb640 1 Processor -- start 2023-11-28T18:28:46.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.930+0000 7f7b6c2bb640 1 -- start start 2023-11-28T18:28:46.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b6c2bb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b6406cd20 0x7f7b641f4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:46.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b6c2bb640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b640ce2c0 0x7f7b6414e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:46.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b6c2bb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b64159610 0x7f7b64151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:46.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b6c2bb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6414e660 con 0x7f7b6406cd20 2023-11-28T18:28:46.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b6c2bb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6409a3f0 con 0x7f7b64159610 2023-11-28T18:28:46.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b6c2bb640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6409a6d0 con 0x7f7b640ce2c0 2023-11-28T18:28:46.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b6a057640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b6406cd20 0x7f7b641f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:46.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b6a057640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b6406cd20 0x7f7b641f4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47332/0 (socket says 172.21.15.105:47332) 2023-11-28T18:28:46.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b6a057640 1 -- 172.21.15.105:0/4075728724 learned_addr learned my addr 172.21.15.105:0/4075728724 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:46.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b6a858640 1 --2- 172.21.15.105:0/4075728724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b64159610 0x7f7b64151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:46.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b6a057640 1 -- 172.21.15.105:0/4075728724 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b640ce2c0 msgr2=0x7f7b6414e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:46.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b69856640 1 --2- 172.21.15.105:0/4075728724 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b640ce2c0 0x7f7b6414e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:46.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b6a057640 1 --2- 172.21.15.105:0/4075728724 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b640ce2c0 0x7f7b6414e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:46.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b6a057640 1 -- 172.21.15.105:0/4075728724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b64159610 msgr2=0x7f7b64151940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:46.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b6a057640 1 --2- 172.21.15.105:0/4075728724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b64159610 0x7f7b64151940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:46.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b6a057640 1 -- 172.21.15.105:0/4075728724 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5c009a70 con 0x7f7b6406cd20 2023-11-28T18:28:46.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b6a858640 1 --2- 172.21.15.105:0/4075728724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b64159610 0x7f7b64151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:46.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b69856640 1 --2- 172.21.15.105:0/4075728724 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b640ce2c0 0x7f7b6414e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:46.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b6a057640 1 --2- 172.21.15.105:0/4075728724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b6406cd20 0x7f7b641f4300 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000c480 tx=0x7f7b6000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b5affd640 1 -- 172.21.15.105:0/4075728724 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b600074e0 con 0x7f7b6406cd20 2023-11-28T18:28:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b5affd640 1 -- 172.21.15.105:0/4075728724 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b60007660 con 0x7f7b6406cd20 2023-11-28T18:28:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b5affd640 1 -- 172.21.15.105:0/4075728724 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b60004a30 con 0x7f7b6406cd20 2023-11-28T18:28:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b6c2bb640 1 -- 172.21.15.105:0/4075728724 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b640ccff0 con 0x7f7b6406cd20 2023-11-28T18:28:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b6c2bb640 1 -- 172.21.15.105:0/4075728724 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6409ac90 con 0x7f7b6406cd20 2023-11-28T18:28:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b6c2bb640 1 -- 172.21.15.105:0/4075728724 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b64067570 con 0x7f7b6406cd20 2023-11-28T18:28:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.934+0000 7f7b5affd640 1 -- 172.21.15.105:0/4075728724 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f7b60004b90 con 0x7f7b6406cd20 2023-11-28T18:28:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.938+0000 7f7b5affd640 1 --2- 172.21.15.105:0/4075728724 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7b480668d0 0x7f7b48068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.938+0000 7f7b5affd640 1 -- 172.21.15.105:0/4075728724 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7b60083720 con 0x7f7b6406cd20 2023-11-28T18:28:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.938+0000 7f7b69856640 1 --2- 172.21.15.105:0/4075728724 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7b480668d0 0x7f7b48068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.938+0000 7f7b69856640 1 --2- 172.21.15.105:0/4075728724 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7b480668d0 0x7f7b48068d90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7b64099da0 tx=0x7f7b54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:46.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:46.942+0000 7f7b5affd640 1 -- 172.21.15.105:0/4075728724 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7b600542a0 con 0x7f7b6406cd20 2023-11-28T18:28:47.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.094+0000 7f7b6c2bb640 1 -- 172.21.15.105:0/4075728724 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b640b0630 con 0x7f7b6406cd20 2023-11-28T18:28:47.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.094+0000 7f7b5affd640 1 -- 172.21.15.105:0/4075728724 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16837 (secure 0 0 0) 0x7f7b60053e30 con 0x7f7b6406cd20 2023-11-28T18:28:47.099 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:47.100 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":78,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:28:14.101062+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:28:11.454373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-28T18:28:12.093171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:28:47.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.098+0000 7f7b6c2bb640 1 -- 172.21.15.105:0/4075728724 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7b480668d0 msgr2=0x7f7b48068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.098+0000 7f7b6c2bb640 1 --2- 172.21.15.105:0/4075728724 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7b480668d0 0x7f7b48068d90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7b64099da0 tx=0x7f7b54008040 comp rx=0 tx=0).stop 2023-11-28T18:28:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.098+0000 7f7b6c2bb640 1 -- 172.21.15.105:0/4075728724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b6406cd20 msgr2=0x7f7b641f4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.098+0000 7f7b6c2bb640 1 --2- 172.21.15.105:0/4075728724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b6406cd20 0x7f7b641f4300 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000c480 tx=0x7f7b6000c940 comp rx=0 tx=0).stop 2023-11-28T18:28:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.098+0000 7f7b6c2bb640 1 -- 172.21.15.105:0/4075728724 shutdown_connections 2023-11-28T18:28:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.098+0000 7f7b6c2bb640 1 --2- 172.21.15.105:0/4075728724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b64159610 0x7f7b64151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.098+0000 7f7b6c2bb640 1 --2- 172.21.15.105:0/4075728724 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b640ce2c0 0x7f7b6414e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.098+0000 7f7b6c2bb640 1 --2- 172.21.15.105:0/4075728724 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7b480668d0 0x7f7b48068d90 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.098+0000 7f7b6c2bb640 1 --2- 172.21.15.105:0/4075728724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b6406cd20 0x7f7b641f4300 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.098+0000 7f7b6c2bb640 1 -- 172.21.15.105:0/4075728724 >> 172.21.15.105:0/4075728724 conn(0x7f7b640ceb90 msgr2=0x7f7b640b19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.098+0000 7f7b6c2bb640 1 -- 172.21.15.105:0/4075728724 shutdown_connections 2023-11-28T18:28:47.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.098+0000 7f7b6c2bb640 1 -- 172.21.15.105:0/4075728724 wait complete. 2023-11-28T18:28:47.121 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:28:11.454373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:28:12.093171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:28:47.122 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:28:47.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.258+0000 7fb6246ea640 1 Processor -- start 2023-11-28T18:28:47.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.258+0000 7fb6246ea640 1 -- start start 2023-11-28T18:28:47.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.258+0000 7fb6246ea640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb61c159610 0x7fb61c0ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:47.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.258+0000 7fb6246ea640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb61c0cc760 0x7fb61c0ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:47.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.262+0000 7fb6246ea640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb61c0cde30 con 0x7fb61c06c7e0 2023-11-28T18:28:47.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.262+0000 7fb6246ea640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb61c1599f0 con 0x7fb61c0cc760 2023-11-28T18:28:47.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.262+0000 7fb6246ea640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb61c06de20 con 0x7fb61c159610 2023-11-28T18:28:47.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.262+0000 7fb621c85640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb61c0cc760 0x7fb61c0ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:47.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.262+0000 7fb621c85640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb61c0cc760 0x7fb61c0ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47338/0 (socket says 172.21.15.105:47338) 2023-11-28T18:28:47.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.262+0000 7fb621c85640 1 -- 172.21.15.105:0/1483882487 learned_addr learned my addr 172.21.15.105:0/1483882487 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:47.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.262+0000 7fb622486640 1 --2- 172.21.15.105:0/1483882487 >> v2:172.21.15.148:3300/0 conn(0x7fb61c159610 0x7fb61c0ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:47.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.262+0000 7fb621c85640 1 -- 172.21.15.105:0/1483882487 >> v2:172.21.15.148:3300/0 conn(0x7fb61c159610 msgr2=0x7fb61c0ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:47.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.262+0000 7fb621c85640 1 --2- 172.21.15.105:0/1483882487 >> v2:172.21.15.148:3300/0 conn(0x7fb61c159610 0x7fb61c0ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.262+0000 7fb621c85640 1 -- 172.21.15.105:0/1483882487 >> v1:172.21.15.148:6789/0 conn(0x7fb61c06c7e0 legacy=0x7fb61c0abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:47.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.262+0000 7fb621c85640 1 -- 172.21.15.105:0/1483882487 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c0cd220 con 0x7fb61c0cc760 2023-11-28T18:28:47.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.262+0000 7fb622486640 1 --2- 172.21.15.105:0/1483882487 >> v2:172.21.15.148:3300/0 conn(0x7fb61c159610 0x7fb61c0ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:47.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.262+0000 7fb621c85640 1 --2- 172.21.15.105:0/1483882487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb61c0cc760 0x7fb61c0ccbf0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fb61c0cc6b0 tx=0x7fb61c0acca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9202878f8165c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:47.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.262+0000 7fb621484640 1 -- 172.21.15.105:0/1483882487 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c0a6910 con 0x7fb61c0cc760 2023-11-28T18:28:47.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.262+0000 7fb621484640 1 -- 172.21.15.105:0/1483882487 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60c0a6a70 con 0x7fb61c0cc760 2023-11-28T18:28:47.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.266+0000 7fb621484640 1 -- 172.21.15.105:0/1483882487 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c0a6e00 con 0x7fb61c0cc760 2023-11-28T18:28:47.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.266+0000 7fb6246ea640 1 -- 172.21.15.105:0/1483882487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb61c0cc760 msgr2=0x7fb61c0ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:47.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.266+0000 7fb6246ea640 1 --2- 172.21.15.105:0/1483882487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb61c0cc760 0x7fb61c0ccbf0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fb61c0cc6b0 tx=0x7fb61c0acca0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.266+0000 7fb6246ea640 1 -- 172.21.15.105:0/1483882487 shutdown_connections 2023-11-28T18:28:47.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.266+0000 7fb6246ea640 1 --2- 172.21.15.105:0/1483882487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb61c0cc760 0x7fb61c0ccbf0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.266+0000 7fb6246ea640 1 --2- 172.21.15.105:0/1483882487 >> v2:172.21.15.148:3300/0 conn(0x7fb61c159610 0x7fb61c0ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.266+0000 7fb6246ea640 1 -- 172.21.15.105:0/1483882487 >> 172.21.15.105:0/1483882487 conn(0x7fb61c0ceb90 msgr2=0x7fb61c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:47.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.266+0000 7fb6246ea640 1 -- 172.21.15.105:0/1483882487 shutdown_connections 2023-11-28T18:28:47.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.266+0000 7fb6246ea640 1 -- 172.21.15.105:0/1483882487 wait complete. 2023-11-28T18:28:47.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.266+0000 7fb6246ea640 1 Processor -- start 2023-11-28T18:28:47.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.266+0000 7fb6246ea640 1 -- start start 2023-11-28T18:28:47.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.266+0000 7fb6246ea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb61c06c7e0 0x7fb61c09c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:47.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.266+0000 7fb6246ea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb61c0cc760 0x7fb61c1415b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:47.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.266+0000 7fb6246ea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb61c159610 0x7fb61c09a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:47.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.266+0000 7fb6246ea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c0b2be0 con 0x7fb61c06c7e0 2023-11-28T18:28:47.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.266+0000 7fb6246ea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c0b22d0 con 0x7fb61c0cc760 2023-11-28T18:28:47.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.266+0000 7fb6246ea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c141af0 con 0x7fb61c159610 2023-11-28T18:28:47.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.266+0000 7fb622486640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb61c06c7e0 0x7fb61c09c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:47.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.266+0000 7fb622486640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb61c06c7e0 0x7fb61c09c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47354/0 (socket says 172.21.15.105:47354) 2023-11-28T18:28:47.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.266+0000 7fb622486640 1 -- 172.21.15.105:0/4152367692 learned_addr learned my addr 172.21.15.105:0/4152367692 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:47.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.266+0000 7fb621c85640 1 --2- 172.21.15.105:0/4152367692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb61c0cc760 0x7fb61c1415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:47.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.266+0000 7fb622c87640 1 --2- 172.21.15.105:0/4152367692 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb61c159610 0x7fb61c09a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:47.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.270+0000 7fb622486640 1 -- 172.21.15.105:0/4152367692 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb61c159610 msgr2=0x7fb61c09a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:47.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.270+0000 7fb622486640 1 --2- 172.21.15.105:0/4152367692 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb61c159610 0x7fb61c09a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.270+0000 7fb622486640 1 -- 172.21.15.105:0/4152367692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb61c0cc760 msgr2=0x7fb61c1415b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:47.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.270+0000 7fb622486640 1 --2- 172.21.15.105:0/4152367692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb61c0cc760 0x7fb61c1415b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.270+0000 7fb622486640 1 -- 172.21.15.105:0/4152367692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60c00a670 con 0x7fb61c06c7e0 2023-11-28T18:28:47.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.270+0000 7fb622486640 1 --2- 172.21.15.105:0/4152367692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb61c06c7e0 0x7fb61c09c670 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fb61800c7f0 tx=0x7fb61800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:47.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.270+0000 7fb612ffd640 1 -- 172.21.15.105:0/4152367692 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb618007820 con 0x7fb61c06c7e0 2023-11-28T18:28:47.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.270+0000 7fb612ffd640 1 -- 172.21.15.105:0/4152367692 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb618007980 con 0x7fb61c06c7e0 2023-11-28T18:28:47.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.270+0000 7fb6246ea640 1 -- 172.21.15.105:0/4152367692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61c09cc80 con 0x7fb61c06c7e0 2023-11-28T18:28:47.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.270+0000 7fb612ffd640 1 -- 172.21.15.105:0/4152367692 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb618007d30 con 0x7fb61c06c7e0 2023-11-28T18:28:47.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.270+0000 7fb6246ea640 1 -- 172.21.15.105:0/4152367692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb61c0b2800 con 0x7fb61c06c7e0 2023-11-28T18:28:47.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.270+0000 7fb6246ea640 1 -- 172.21.15.105:0/4152367692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb61c0b2d20 con 0x7fb61c06c7e0 2023-11-28T18:28:47.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.270+0000 7fb612ffd640 1 -- 172.21.15.105:0/4152367692 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fb6180113f0 con 0x7fb61c06c7e0 2023-11-28T18:28:47.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.274+0000 7fb612ffd640 1 --2- 172.21.15.105:0/4152367692 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb6000668d0 0x7fb600068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:47.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.274+0000 7fb612ffd640 1 -- 172.21.15.105:0/4152367692 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb618084840 con 0x7fb61c06c7e0 2023-11-28T18:28:47.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.274+0000 7fb621c85640 1 --2- 172.21.15.105:0/4152367692 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb6000668d0 0x7fb600068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:47.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.274+0000 7fb621c85640 1 --2- 172.21.15.105:0/4152367692 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb6000668d0 0x7fb600068d90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0b0000 tx=0x7fb60c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:47.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.278+0000 7fb612ffd640 1 -- 172.21.15.105:0/4152367692 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb61c0b2d20 con 0x7fb61c06c7e0 2023-11-28T18:28:47.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.426+0000 7fb6246ea640 1 -- 172.21.15.105:0/4152367692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb61c0b2d20 con 0x7fb61c06c7e0 2023-11-28T18:28:47.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.426+0000 7fb612ffd640 1 -- 172.21.15.105:0/4152367692 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb61c0b2d20 con 0x7fb61c06c7e0 2023-11-28T18:28:47.433 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:28:47.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.430+0000 7fb6246ea640 1 -- 172.21.15.105:0/4152367692 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb6000668d0 msgr2=0x7fb600068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.430+0000 7fb6246ea640 1 --2- 172.21.15.105:0/4152367692 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb6000668d0 0x7fb600068d90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0b0000 tx=0x7fb60c008040 comp rx=0 tx=0).stop 2023-11-28T18:28:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.430+0000 7fb6246ea640 1 -- 172.21.15.105:0/4152367692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb61c06c7e0 msgr2=0x7fb61c09c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.430+0000 7fb6246ea640 1 --2- 172.21.15.105:0/4152367692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb61c06c7e0 0x7fb61c09c670 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fb61800c7f0 tx=0x7fb61800ccb0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.430+0000 7fb6246ea640 1 -- 172.21.15.105:0/4152367692 shutdown_connections 2023-11-28T18:28:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.430+0000 7fb6246ea640 1 --2- 172.21.15.105:0/4152367692 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb61c159610 0x7fb61c09a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.430+0000 7fb6246ea640 1 --2- 172.21.15.105:0/4152367692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb61c0cc760 0x7fb61c1415b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.430+0000 7fb6246ea640 1 --2- 172.21.15.105:0/4152367692 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb6000668d0 0x7fb600068d90 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.430+0000 7fb6246ea640 1 --2- 172.21.15.105:0/4152367692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb61c06c7e0 0x7fb61c09c670 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.430+0000 7fb6246ea640 1 -- 172.21.15.105:0/4152367692 >> 172.21.15.105:0/4152367692 conn(0x7fb61c0ceb90 msgr2=0x7fb61c071030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.430+0000 7fb6246ea640 1 -- 172.21.15.105:0/4152367692 shutdown_connections 2023-11-28T18:28:47.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.430+0000 7fb6246ea640 1 -- 172.21.15.105:0/4152367692 wait complete. 2023-11-28T18:28:47.454 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:28:47.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.590+0000 7f0b3dde4640 1 Processor -- start 2023-11-28T18:28:47.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.590+0000 7f0b3dde4640 1 -- start start 2023-11-28T18:28:47.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.590+0000 7f0b3dde4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0b380d3920 0x7f0b380cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:47.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.590+0000 7f0b3dde4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0b380b2a80 0x7f0b380b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:47.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.590+0000 7f0b3dde4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b380ce4c0 con 0x7f0b3815d000 2023-11-28T18:28:47.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.590+0000 7f0b3dde4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b380d3d80 con 0x7f0b380b2a80 2023-11-28T18:28:47.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.590+0000 7f0b3dde4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b380cedf0 con 0x7f0b380d3920 2023-11-28T18:28:47.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.590+0000 7f0b3d5e3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0b380b2a80 0x7f0b380b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:47.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.590+0000 7f0b3d5e3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0b380b2a80 0x7f0b380b2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47356/0 (socket says 172.21.15.105:47356) 2023-11-28T18:28:47.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.590+0000 7f0b3d5e3640 1 -- 172.21.15.105:0/3831956372 learned_addr learned my addr 172.21.15.105:0/3831956372 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:47.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.590+0000 7f0b37fff640 1 --2- 172.21.15.105:0/3831956372 >> v2:172.21.15.148:3300/0 conn(0x7f0b380d3920 0x7f0b380cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.590+0000 7f0b3d5e3640 1 -- 172.21.15.105:0/3831956372 >> v2:172.21.15.148:3300/0 conn(0x7f0b380d3920 msgr2=0x7f0b380cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.590+0000 7f0b3d5e3640 1 --2- 172.21.15.105:0/3831956372 >> v2:172.21.15.148:3300/0 conn(0x7f0b380d3920 0x7f0b380cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.590+0000 7f0b3d5e3640 1 -- 172.21.15.105:0/3831956372 >> v1:172.21.15.148:6789/0 conn(0x7f0b3815d000 legacy=0x7f0b380cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.590+0000 7f0b3d5e3640 1 -- 172.21.15.105:0/3831956372 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b380ce7a0 con 0x7f0b380b2a80 2023-11-28T18:28:47.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.594+0000 7f0b3d5e3640 1 --2- 172.21.15.105:0/3831956372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b380b2a80 0x7f0b380b2eb0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f0b380cd630 tx=0x7f0b3001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca41dd722b9078c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:47.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.594+0000 7f0b377fe640 1 -- 172.21.15.105:0/3831956372 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b300ac630 con 0x7f0b380b2a80 2023-11-28T18:28:47.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.594+0000 7f0b377fe640 1 -- 172.21.15.105:0/3831956372 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b300ac790 con 0x7f0b380b2a80 2023-11-28T18:28:47.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.594+0000 7f0b377fe640 1 -- 172.21.15.105:0/3831956372 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b300acb20 con 0x7f0b380b2a80 2023-11-28T18:28:47.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.594+0000 7f0b3dde4640 1 -- 172.21.15.105:0/3831956372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b380b2a80 msgr2=0x7f0b380b2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:47.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.594+0000 7f0b3dde4640 1 --2- 172.21.15.105:0/3831956372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b380b2a80 0x7f0b380b2eb0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f0b380cd630 tx=0x7f0b3001c440 comp rx=0 tx=0).stop 2023-11-28T18:28:47.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.594+0000 7f0b3dde4640 1 -- 172.21.15.105:0/3831956372 shutdown_connections 2023-11-28T18:28:47.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.594+0000 7f0b3dde4640 1 --2- 172.21.15.105:0/3831956372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b380b2a80 0x7f0b380b2eb0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.594+0000 7f0b3dde4640 1 --2- 172.21.15.105:0/3831956372 >> v2:172.21.15.148:3300/0 conn(0x7f0b380d3920 0x7f0b380cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.594+0000 7f0b3dde4640 1 -- 172.21.15.105:0/3831956372 >> 172.21.15.105:0/3831956372 conn(0x7f0b380d2e00 msgr2=0x7f0b380d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:47.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.594+0000 7f0b3dde4640 1 -- 172.21.15.105:0/3831956372 shutdown_connections 2023-11-28T18:28:47.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.594+0000 7f0b3dde4640 1 -- 172.21.15.105:0/3831956372 wait complete. 2023-11-28T18:28:47.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b3dde4640 1 Processor -- start 2023-11-28T18:28:47.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b3dde4640 1 -- start start 2023-11-28T18:28:47.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b3dde4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b380b2a80 0x7f0b38160000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:47.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b3dde4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b380d3920 0x7f0b38154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:47.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b3dde4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0b3815d000 0x7f0b3815e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:47.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b3dde4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b381546c0 con 0x7f0b380d3920 2023-11-28T18:28:47.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b3dde4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b38160540 con 0x7f0b380b2a80 2023-11-28T18:28:47.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b3dde4640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b38153a40 con 0x7f0b3815d000 2023-11-28T18:28:47.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b37fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b380d3920 0x7f0b38154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:47.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b37fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b380d3920 0x7f0b38154ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47366/0 (socket says 172.21.15.105:47366) 2023-11-28T18:28:47.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b37fff640 1 -- 172.21.15.105:0/3845524864 learned_addr learned my addr 172.21.15.105:0/3845524864 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:47.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b3cde2640 1 --2- 172.21.15.105:0/3845524864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b380b2a80 0x7f0b38160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:47.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b3d5e3640 1 --2- 172.21.15.105:0/3845524864 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0b3815d000 0x7f0b3815e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:47.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b37fff640 1 -- 172.21.15.105:0/3845524864 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0b3815d000 msgr2=0x7f0b3815e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:47.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b37fff640 1 --2- 172.21.15.105:0/3845524864 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0b3815d000 0x7f0b3815e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b37fff640 1 -- 172.21.15.105:0/3845524864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b380b2a80 msgr2=0x7f0b38160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:47.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b37fff640 1 --2- 172.21.15.105:0/3845524864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b380b2a80 0x7f0b38160000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b37fff640 1 -- 172.21.15.105:0/3845524864 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b300094d0 con 0x7f0b380d3920 2023-11-28T18:28:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b37fff640 1 --2- 172.21.15.105:0/3845524864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b380d3920 0x7f0b38154ac0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400e960 tx=0x7f0b2400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b357fa640 1 -- 172.21.15.105:0/3845524864 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b24009690 con 0x7f0b380d3920 2023-11-28T18:28:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b357fa640 1 -- 172.21.15.105:0/3845524864 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2400c9a0 con 0x7f0b380d3920 2023-11-28T18:28:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b3dde4640 1 -- 172.21.15.105:0/3845524864 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b38153d20 con 0x7f0b380d3920 2023-11-28T18:28:47.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b357fa640 1 -- 172.21.15.105:0/3845524864 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2400cd10 con 0x7f0b380d3920 2023-11-28T18:28:47.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.598+0000 7f0b3dde4640 1 -- 172.21.15.105:0/3845524864 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b38155000 con 0x7f0b380d3920 2023-11-28T18:28:47.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.602+0000 7f0b3dde4640 1 -- 172.21.15.105:0/3845524864 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b38154110 con 0x7f0b380d3920 2023-11-28T18:28:47.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.602+0000 7f0b357fa640 1 -- 172.21.15.105:0/3845524864 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f0b24013470 con 0x7f0b380d3920 2023-11-28T18:28:47.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.602+0000 7f0b357fa640 1 --2- 172.21.15.105:0/3845524864 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0b1c0668d0 0x7f0b1c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:47.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.602+0000 7f0b357fa640 1 -- 172.21.15.105:0/3845524864 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0b240846f0 con 0x7f0b380d3920 2023-11-28T18:28:47.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.602+0000 7f0b3cde2640 1 --2- 172.21.15.105:0/3845524864 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0b1c0668d0 0x7f0b1c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:47.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.602+0000 7f0b3cde2640 1 --2- 172.21.15.105:0/3845524864 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0b1c0668d0 0x7f0b1c068d90 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0012f0 tx=0x7f0b2c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:47.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.606+0000 7f0b357fa640 1 -- 172.21.15.105:0/3845524864 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0b24051890 con 0x7f0b380d3920 2023-11-28T18:28:47.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.754+0000 7f0b3dde4640 1 -- 172.21.15.105:0/3845524864 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0b38153f10 con 0x7f0b380d3920 2023-11-28T18:28:47.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.758+0000 7f0b357fa640 1 -- 172.21.15.105:0/3845524864 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0b24082e60 con 0x7f0b380d3920 2023-11-28T18:28:47.761 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:28:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.758+0000 7f0b3dde4640 1 -- 172.21.15.105:0/3845524864 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0b1c0668d0 msgr2=0x7f0b1c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.758+0000 7f0b3dde4640 1 --2- 172.21.15.105:0/3845524864 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0b1c0668d0 0x7f0b1c068d90 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0012f0 tx=0x7f0b2c009040 comp rx=0 tx=0).stop 2023-11-28T18:28:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.758+0000 7f0b3dde4640 1 -- 172.21.15.105:0/3845524864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b380d3920 msgr2=0x7f0b38154ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.758+0000 7f0b3dde4640 1 --2- 172.21.15.105:0/3845524864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b380d3920 0x7f0b38154ac0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400e960 tx=0x7f0b2400ee20 comp rx=0 tx=0).stop 2023-11-28T18:28:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.758+0000 7f0b3dde4640 1 -- 172.21.15.105:0/3845524864 shutdown_connections 2023-11-28T18:28:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.758+0000 7f0b3dde4640 1 --2- 172.21.15.105:0/3845524864 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0b3815d000 0x7f0b3815e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.758+0000 7f0b3dde4640 1 --2- 172.21.15.105:0/3845524864 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0b1c0668d0 0x7f0b1c068d90 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.758+0000 7f0b3dde4640 1 --2- 172.21.15.105:0/3845524864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b380d3920 0x7f0b38154ac0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.758+0000 7f0b3dde4640 1 --2- 172.21.15.105:0/3845524864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b380b2a80 0x7f0b38160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.758+0000 7f0b3dde4640 1 -- 172.21.15.105:0/3845524864 >> 172.21.15.105:0/3845524864 conn(0x7f0b380d2e00 msgr2=0x7f0b380afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.758+0000 7f0b3dde4640 1 -- 172.21.15.105:0/3845524864 shutdown_connections 2023-11-28T18:28:47.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.758+0000 7f0b3dde4640 1 -- 172.21.15.105:0/3845524864 wait complete. 2023-11-28T18:28:47.782 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:28:47.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.918+0000 7f4f2285f640 1 Processor -- start 2023-11-28T18:28:47.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.918+0000 7f4f2285f640 1 -- start start 2023-11-28T18:28:47.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.918+0000 7f4f2285f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4f1c071960 0x7f4f1c15f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:47.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.918+0000 7f4f2285f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4f1c15b030 0x7f4f1c0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:47.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.918+0000 7f4f2285f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f1c0701d0 con 0x7f4f1c0b4290 2023-11-28T18:28:47.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.918+0000 7f4f2285f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f1c06df80 con 0x7f4f1c15b030 2023-11-28T18:28:47.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.918+0000 7f4f2285f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f1c071df0 con 0x7f4f1c071960 2023-11-28T18:28:47.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.918+0000 7f4f20dfc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4f1c15b030 0x7f4f1c0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:47.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.918+0000 7f4f20dfc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4f1c15b030 0x7f4f1c0709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47378/0 (socket says 172.21.15.105:47378) 2023-11-28T18:28:47.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.918+0000 7f4f20dfc640 1 -- 172.21.15.105:0/3013028379 learned_addr learned my addr 172.21.15.105:0/3013028379 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:47.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.918+0000 7f4f1b7fe640 1 --2- 172.21.15.105:0/3013028379 >> v2:172.21.15.148:3300/0 conn(0x7f4f1c071960 0x7f4f1c15f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:47.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.922+0000 7f4f20dfc640 1 -- 172.21.15.105:0/3013028379 >> v2:172.21.15.148:3300/0 conn(0x7f4f1c071960 msgr2=0x7f4f1c15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:47.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.922+0000 7f4f20dfc640 1 --2- 172.21.15.105:0/3013028379 >> v2:172.21.15.148:3300/0 conn(0x7f4f1c071960 0x7f4f1c15f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.922+0000 7f4f20dfc640 1 -- 172.21.15.105:0/3013028379 >> v1:172.21.15.148:6789/0 conn(0x7f4f1c0b4290 legacy=0x7f4f1c0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:47.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.922+0000 7f4f20dfc640 1 -- 172.21.15.105:0/3013028379 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c0720d0 con 0x7f4f1c15b030 2023-11-28T18:28:47.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.922+0000 7f4f20dfc640 1 --2- 172.21.15.105:0/3013028379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f1c15b030 0x7f4f1c0709b0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c06e520 tx=0x7f4f1401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0f9c22456b9584 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:47.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.922+0000 7f4f1affd640 1 -- 172.21.15.105:0/3013028379 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f140a5770 con 0x7f4f1c15b030 2023-11-28T18:28:47.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.922+0000 7f4f1affd640 1 -- 172.21.15.105:0/3013028379 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f140a58d0 con 0x7f4f1c15b030 2023-11-28T18:28:47.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.922+0000 7f4f1affd640 1 -- 172.21.15.105:0/3013028379 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f140a5c60 con 0x7f4f1c15b030 2023-11-28T18:28:47.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.922+0000 7f4f2285f640 1 -- 172.21.15.105:0/3013028379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f1c15b030 msgr2=0x7f4f1c0709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:47.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.922+0000 7f4f2285f640 1 --2- 172.21.15.105:0/3013028379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f1c15b030 0x7f4f1c0709b0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c06e520 tx=0x7f4f1401c9a0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f2285f640 1 -- 172.21.15.105:0/3013028379 shutdown_connections 2023-11-28T18:28:47.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f2285f640 1 --2- 172.21.15.105:0/3013028379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f1c15b030 0x7f4f1c0709b0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f2285f640 1 --2- 172.21.15.105:0/3013028379 >> v2:172.21.15.148:3300/0 conn(0x7f4f1c071960 0x7f4f1c15f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f2285f640 1 -- 172.21.15.105:0/3013028379 >> 172.21.15.105:0/3013028379 conn(0x7f4f1c0ceb90 msgr2=0x7f4f1c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:47.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f2285f640 1 -- 172.21.15.105:0/3013028379 shutdown_connections 2023-11-28T18:28:47.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f2285f640 1 -- 172.21.15.105:0/3013028379 wait complete. 2023-11-28T18:28:47.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f2285f640 1 Processor -- start 2023-11-28T18:28:47.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f2285f640 1 -- start start 2023-11-28T18:28:47.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f2285f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4f1c0b4290 0x7f4f1c0cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:47.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f2285f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f1c15b030 0x7f4f1c153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:47.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f2285f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f1c1411e0 0x7f4f1c141690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:47.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f2285f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c154b80 con 0x7f4f1c15b030 2023-11-28T18:28:47.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f2285f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c0ad4b0 con 0x7f4f1c1411e0 2023-11-28T18:28:47.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f2285f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c0cd310 con 0x7f4f1c0b4290 2023-11-28T18:28:47.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f1b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f1c15b030 0x7f4f1c153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:47.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f1b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f1c15b030 0x7f4f1c153f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47392/0 (socket says 172.21.15.105:47392) 2023-11-28T18:28:47.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f1b7fe640 1 -- 172.21.15.105:0/3261166908 learned_addr learned my addr 172.21.15.105:0/3261166908 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:47.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f20dfc640 1 --2- 172.21.15.105:0/3261166908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f1c1411e0 0x7f4f1c141690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:47.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f1bfff640 1 --2- 172.21.15.105:0/3261166908 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4f1c0b4290 0x7f4f1c0cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:47.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f1b7fe640 1 -- 172.21.15.105:0/3261166908 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4f1c0b4290 msgr2=0x7f4f1c0cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:47.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f1b7fe640 1 --2- 172.21.15.105:0/3261166908 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4f1c0b4290 0x7f4f1c0cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f1b7fe640 1 -- 172.21.15.105:0/3261166908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f1c1411e0 msgr2=0x7f4f1c141690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:47.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f1b7fe640 1 --2- 172.21.15.105:0/3261166908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f1c1411e0 0x7f4f1c141690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:47.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f1b7fe640 1 -- 172.21.15.105:0/3261166908 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f14009b30 con 0x7f4f1c15b030 2023-11-28T18:28:47.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f1bfff640 1 --2- 172.21.15.105:0/3261166908 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4f1c0b4290 0x7f4f1c0cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:47.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.926+0000 7f4f1b7fe640 1 --2- 172.21.15.105:0/3261166908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f1c15b030 0x7f4f1c153f20 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000e930 tx=0x7f4f1000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:47.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.930+0000 7f4f18ff9640 1 -- 172.21.15.105:0/3261166908 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f10009690 con 0x7f4f1c15b030 2023-11-28T18:28:47.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.930+0000 7f4f18ff9640 1 -- 172.21.15.105:0/3261166908 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f1000ca00 con 0x7f4f1c15b030 2023-11-28T18:28:47.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.930+0000 7f4f2285f640 1 -- 172.21.15.105:0/3261166908 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1c0ad9e0 con 0x7f4f1c15b030 2023-11-28T18:28:47.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.930+0000 7f4f18ff9640 1 -- 172.21.15.105:0/3261166908 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1000cd70 con 0x7f4f1c15b030 2023-11-28T18:28:47.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.930+0000 7f4f2285f640 1 -- 172.21.15.105:0/3261166908 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1c0cd5b0 con 0x7f4f1c15b030 2023-11-28T18:28:47.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.930+0000 7f4f2285f640 1 -- 172.21.15.105:0/3261166908 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1c15f0d0 con 0x7f4f1c15b030 2023-11-28T18:28:47.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.930+0000 7f4f18ff9640 1 -- 172.21.15.105:0/3261166908 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f4f10013470 con 0x7f4f1c15b030 2023-11-28T18:28:47.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.930+0000 7f4f18ff9640 1 --2- 172.21.15.105:0/3261166908 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4f080668d0 0x7f4f08068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:47.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.930+0000 7f4f18ff9640 1 -- 172.21.15.105:0/3261166908 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4f100846c0 con 0x7f4f1c15b030 2023-11-28T18:28:47.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.930+0000 7f4f1bfff640 1 --2- 172.21.15.105:0/3261166908 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4f080668d0 0x7f4f08068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:47.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.930+0000 7f4f1bfff640 1 --2- 172.21.15.105:0/3261166908 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4f080668d0 0x7f4f08068d90 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c1478f0 tx=0x7f4f0c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:47.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:47.934+0000 7f4f18ff9640 1 -- 172.21.15.105:0/3261166908 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4f100519c0 con 0x7f4f1c15b030 2023-11-28T18:28:48.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.082+0000 7f4f2285f640 1 -- 172.21.15.105:0/3261166908 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4f1c067570 con 0x7f4f1c15b030 2023-11-28T18:28:48.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.082+0000 7f4f18ff9640 1 -- 172.21.15.105:0/3261166908 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4f10054fe0 con 0x7f4f1c15b030 2023-11-28T18:28:48.088 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:28:48.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.086+0000 7f4f2285f640 1 -- 172.21.15.105:0/3261166908 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4f080668d0 msgr2=0x7f4f08068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:48.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.086+0000 7f4f2285f640 1 --2- 172.21.15.105:0/3261166908 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4f080668d0 0x7f4f08068d90 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c1478f0 tx=0x7f4f0c009040 comp rx=0 tx=0).stop 2023-11-28T18:28:48.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.086+0000 7f4f2285f640 1 -- 172.21.15.105:0/3261166908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f1c15b030 msgr2=0x7f4f1c153f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:48.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.086+0000 7f4f2285f640 1 --2- 172.21.15.105:0/3261166908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f1c15b030 0x7f4f1c153f20 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000e930 tx=0x7f4f1000edf0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.086+0000 7f4f2285f640 1 -- 172.21.15.105:0/3261166908 shutdown_connections 2023-11-28T18:28:48.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.086+0000 7f4f2285f640 1 --2- 172.21.15.105:0/3261166908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f1c1411e0 0x7f4f1c141690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.086+0000 7f4f2285f640 1 --2- 172.21.15.105:0/3261166908 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4f080668d0 0x7f4f08068d90 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.086+0000 7f4f2285f640 1 --2- 172.21.15.105:0/3261166908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f1c15b030 0x7f4f1c153f20 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.086+0000 7f4f2285f640 1 --2- 172.21.15.105:0/3261166908 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4f1c0b4290 0x7f4f1c0cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.086+0000 7f4f2285f640 1 -- 172.21.15.105:0/3261166908 >> 172.21.15.105:0/3261166908 conn(0x7f4f1c0ceb90 msgr2=0x7f4f1c156e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:48.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.086+0000 7f4f2285f640 1 -- 172.21.15.105:0/3261166908 shutdown_connections 2023-11-28T18:28:48.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.086+0000 7f4f2285f640 1 -- 172.21.15.105:0/3261166908 wait complete. 2023-11-28T18:28:48.110 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:28:48.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.246+0000 7f171b59e640 1 Processor -- start 2023-11-28T18:28:48.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.246+0000 7f171b59e640 1 -- start start 2023-11-28T18:28:48.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.246+0000 7f171b59e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f171c1575b0 0x7f171c15ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:48.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.246+0000 7f171b59e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f171c157990 0x7f171c0c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:48.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.246+0000 7f171b59e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f171c15d7b0 0x7f171c06cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:48.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.246+0000 7f171b59e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f171c15e0b0 con 0x7f171c1575b0 2023-11-28T18:28:48.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.246+0000 7f171b59e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f171c15e390 con 0x7f171c157990 2023-11-28T18:28:48.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.246+0000 7f171b59e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171c15e670 con 0x7f171c15d7b0 2023-11-28T18:28:48.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.246+0000 7f171a59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f171c1575b0 0x7f171c15ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:48.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.246+0000 7f171a59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f171c1575b0 0x7f171c15ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47408/0 (socket says 172.21.15.105:47408) 2023-11-28T18:28:48.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.246+0000 7f171a59c640 1 -- 172.21.15.105:0/4269084156 learned_addr learned my addr 172.21.15.105:0/4269084156 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:48.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.246+0000 7f171ad9d640 1 --2- 172.21.15.105:0/4269084156 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f171c15d7b0 0x7f171c06cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:48.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.246+0000 7f1719d9b640 1 --2- 172.21.15.105:0/4269084156 >> v2:172.21.15.148:3300/0 conn(0x7f171c157990 0x7f171c0c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:48.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.246+0000 7f171a59c640 1 -- 172.21.15.105:0/4269084156 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f171c15d7b0 msgr2=0x7f171c06cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:48.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.246+0000 7f171a59c640 1 --2- 172.21.15.105:0/4269084156 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f171c15d7b0 0x7f171c06cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.246+0000 7f171a59c640 1 -- 172.21.15.105:0/4269084156 >> v2:172.21.15.148:3300/0 conn(0x7f171c157990 msgr2=0x7f171c0c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:48.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.246+0000 7f171a59c640 1 --2- 172.21.15.105:0/4269084156 >> v2:172.21.15.148:3300/0 conn(0x7f171c157990 0x7f171c0c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.246+0000 7f171a59c640 1 -- 172.21.15.105:0/4269084156 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171c0c9750 con 0x7f171c1575b0 2023-11-28T18:28:48.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.250+0000 7f171a59c640 1 --2- 172.21.15.105:0/4269084156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f171c1575b0 0x7f171c15ada0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f171c06f860 tx=0x7f170c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6332eea4bcd8d70c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:48.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.250+0000 7f171959a640 1 -- 172.21.15.105:0/4269084156 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c0ac410 con 0x7f171c1575b0 2023-11-28T18:28:48.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.250+0000 7f171959a640 1 -- 172.21.15.105:0/4269084156 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170c0ac570 con 0x7f171c1575b0 2023-11-28T18:28:48.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.250+0000 7f171959a640 1 -- 172.21.15.105:0/4269084156 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c0ac900 con 0x7f171c1575b0 2023-11-28T18:28:48.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.250+0000 7f171b59e640 1 -- 172.21.15.105:0/4269084156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f171c1575b0 msgr2=0x7f171c15ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:48.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.250+0000 7f171b59e640 1 --2- 172.21.15.105:0/4269084156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f171c1575b0 0x7f171c15ada0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f171c06f860 tx=0x7f170c01c4d0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.250+0000 7f171b59e640 1 -- 172.21.15.105:0/4269084156 shutdown_connections 2023-11-28T18:28:48.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.250+0000 7f171b59e640 1 --2- 172.21.15.105:0/4269084156 >> v2:172.21.15.148:3300/0 conn(0x7f171c157990 0x7f171c0c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.250+0000 7f171b59e640 1 --2- 172.21.15.105:0/4269084156 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f171c15d7b0 0x7f171c06cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.250+0000 7f171b59e640 1 --2- 172.21.15.105:0/4269084156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f171c1575b0 0x7f171c15ada0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.250+0000 7f171b59e640 1 -- 172.21.15.105:0/4269084156 >> 172.21.15.105:0/4269084156 conn(0x7f171c0b0480 msgr2=0x7f171c0b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:48.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.250+0000 7f171b59e640 1 -- 172.21.15.105:0/4269084156 shutdown_connections 2023-11-28T18:28:48.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.250+0000 7f171b59e640 1 -- 172.21.15.105:0/4269084156 wait complete. 2023-11-28T18:28:48.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f171b59e640 1 Processor -- start 2023-11-28T18:28:48.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f171b59e640 1 -- start start 2023-11-28T18:28:48.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f171b59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f171c1575b0 0x7f171c0c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:48.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f171b59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f171c157990 0x7f171c0c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:48.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f171b59e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f171c15d7b0 0x7f171c0c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:48.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f171b59e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c0c88c0 con 0x7f171c1575b0 2023-11-28T18:28:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f171b59e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c0c48d0 con 0x7f171c157990 2023-11-28T18:28:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f171b59e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171c0c6450 con 0x7f171c15d7b0 2023-11-28T18:28:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f171a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f171c1575b0 0x7f171c0c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f171a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f171c1575b0 0x7f171c0c77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47416/0 (socket says 172.21.15.105:47416) 2023-11-28T18:28:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f171a59c640 1 -- 172.21.15.105:0/1895915043 learned_addr learned my addr 172.21.15.105:0/1895915043 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f1719d9b640 1 --2- 172.21.15.105:0/1895915043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f171c157990 0x7f171c0c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f171ad9d640 1 --2- 172.21.15.105:0/1895915043 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f171c15d7b0 0x7f171c0c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f171a59c640 1 -- 172.21.15.105:0/1895915043 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f171c15d7b0 msgr2=0x7f171c0c4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f171a59c640 1 --2- 172.21.15.105:0/1895915043 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f171c15d7b0 0x7f171c0c4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f171a59c640 1 -- 172.21.15.105:0/1895915043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f171c157990 msgr2=0x7f171c0c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f171a59c640 1 --2- 172.21.15.105:0/1895915043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f171c157990 0x7f171c0c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f171a59c640 1 -- 172.21.15.105:0/1895915043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c0094d0 con 0x7f171c1575b0 2023-11-28T18:28:48.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f171a59c640 1 --2- 172.21.15.105:0/1895915043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f171c1575b0 0x7f171c0c77d0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f170c0af040 tx=0x7f170c019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:48.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f170affd640 1 -- 172.21.15.105:0/1895915043 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c0acea0 con 0x7f171c1575b0 2023-11-28T18:28:48.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f170affd640 1 -- 172.21.15.105:0/1895915043 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170c0b5630 con 0x7f171c1575b0 2023-11-28T18:28:48.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f170affd640 1 -- 172.21.15.105:0/1895915043 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c0ac410 con 0x7f171c1575b0 2023-11-28T18:28:48.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f171b59e640 1 -- 172.21.15.105:0/1895915043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171c0c6730 con 0x7f171c1575b0 2023-11-28T18:28:48.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f171b59e640 1 -- 172.21.15.105:0/1895915043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171c0c6cf0 con 0x7f171c1575b0 2023-11-28T18:28:48.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.254+0000 7f171b59e640 1 -- 172.21.15.105:0/1895915043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171c074580 con 0x7f171c1575b0 2023-11-28T18:28:48.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.258+0000 7f170affd640 1 -- 172.21.15.105:0/1895915043 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f170c0b3070 con 0x7f171c1575b0 2023-11-28T18:28:48.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.258+0000 7f170affd640 1 --2- 172.21.15.105:0/1895915043 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f16fc0668d0 0x7f16fc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:48.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.258+0000 7f170affd640 1 -- 172.21.15.105:0/1895915043 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f170c11eb70 con 0x7f171c1575b0 2023-11-28T18:28:48.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.258+0000 7f1719d9b640 1 --2- 172.21.15.105:0/1895915043 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f16fc0668d0 0x7f16fc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:48.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.258+0000 7f1719d9b640 1 --2- 172.21.15.105:0/1895915043 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f16fc0668d0 0x7f16fc068d90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f171c165a10 tx=0x7f1710008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:48.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.262+0000 7f170affd640 1 -- 172.21.15.105:0/1895915043 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f170c0ebd10 con 0x7f171c1575b0 2023-11-28T18:28:48.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.442+0000 7f171b59e640 1 -- 172.21.15.105:0/1895915043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f171c0ad4c0 con 0x7f171c1575b0 2023-11-28T18:28:48.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.442+0000 7f170affd640 1 -- 172.21.15.105:0/1895915043 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f170c0b1090 con 0x7f171c1575b0 2023-11-28T18:28:48.448 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:48.448 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":77,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1455698212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":1455698212},{"type":"v1","addr":"172.21.15.105:6837","nonce":1455698212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:28:13.097621+0000","modified":"2023-11-28T18:28:43.851902+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7194,"mds_1":6862,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"b","rank":2,"incarnation":75,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6834/759522673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":759522673},{"type":"v1","addr":"172.21.15.148:6834","nonce":759522673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6862":{"gid":6862,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6835/2489361300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2489361300},{"type":"v1","addr":"172.21.15.148:6835","nonce":2489361300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7194":{"gid":7194,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6836/188173955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":188173955},{"type":"v1","addr":"172.21.15.105:6836","nonce":188173955}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-28T18:28:48.448 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 77 2023-11-28T18:28:48.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.446+0000 7f171b59e640 1 -- 172.21.15.105:0/1895915043 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f16fc0668d0 msgr2=0x7f16fc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:48.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.446+0000 7f171b59e640 1 --2- 172.21.15.105:0/1895915043 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f16fc0668d0 0x7f16fc068d90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f171c165a10 tx=0x7f1710008040 comp rx=0 tx=0).stop 2023-11-28T18:28:48.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.446+0000 7f171b59e640 1 -- 172.21.15.105:0/1895915043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f171c1575b0 msgr2=0x7f171c0c77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:48.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.446+0000 7f171b59e640 1 --2- 172.21.15.105:0/1895915043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f171c1575b0 0x7f171c0c77d0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f170c0af040 tx=0x7f170c019270 comp rx=0 tx=0).stop 2023-11-28T18:28:48.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.446+0000 7f171b59e640 1 -- 172.21.15.105:0/1895915043 shutdown_connections 2023-11-28T18:28:48.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.446+0000 7f171b59e640 1 --2- 172.21.15.105:0/1895915043 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f171c15d7b0 0x7f171c0c4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.446+0000 7f171b59e640 1 --2- 172.21.15.105:0/1895915043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f171c157990 0x7f171c0c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.446+0000 7f171b59e640 1 --2- 172.21.15.105:0/1895915043 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f16fc0668d0 0x7f16fc068d90 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.446+0000 7f171b59e640 1 --2- 172.21.15.105:0/1895915043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f171c1575b0 0x7f171c0c77d0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.446+0000 7f171b59e640 1 -- 172.21.15.105:0/1895915043 >> 172.21.15.105:0/1895915043 conn(0x7f171c0b0480 msgr2=0x7f171c0cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:48.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.446+0000 7f171b59e640 1 -- 172.21.15.105:0/1895915043 shutdown_connections 2023-11-28T18:28:48.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.446+0000 7f171b59e640 1 -- 172.21.15.105:0/1895915043 wait complete. 2023-11-28T18:28:48.469 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:28:48.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.602+0000 7f49e9432640 1 Processor -- start 2023-11-28T18:28:48.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.602+0000 7f49e9432640 1 -- start start 2023-11-28T18:28:48.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.602+0000 7f49e9432640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f49e4071960 0x7f49e415f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:48.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.602+0000 7f49e9432640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e40701d0 con 0x7f49e415b030 2023-11-28T18:28:48.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.602+0000 7f49e9432640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e406df80 con 0x7f49e40b4290 2023-11-28T18:28:48.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.602+0000 7f49e9432640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e4071df0 con 0x7f49e4071960 2023-11-28T18:28:48.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.602+0000 7f49e2ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f49e40b4290 0x7f49e40b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:59232/0 (socket says 172.21.15.105:59232) 2023-11-28T18:28:48.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.602+0000 7f49e2ffd640 1 -- 172.21.15.105:0/1997988708 learned_addr learned my addr 172.21.15.105:0/1997988708 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:48.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.602+0000 7f49da7fc640 1 --2- 172.21.15.105:0/1997988708 >> v2:172.21.15.105:3300/0 conn(0x7f49e4071960 0x7f49e415f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:48.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.602+0000 7f49e27fc640 1 -- 172.21.15.105:0/1997988708 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2055327787 0 0) 0x7f49e40701d0 con 0x7f49e415b030 2023-11-28T18:28:48.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.602+0000 7f49e27fc640 1 -- 172.21.15.105:0/1997988708 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49d0003530 con 0x7f49e415b030 2023-11-28T18:28:48.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.602+0000 7f49e27fc640 1 -- 172.21.15.105:0/1997988708 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355378793 0 0) 0x7f49e406df80 con 0x7f49e40b4290 2023-11-28T18:28:48.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.602+0000 7f49e27fc640 1 -- 172.21.15.105:0/1997988708 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49e40701d0 con 0x7f49e40b4290 2023-11-28T18:28:48.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.602+0000 7f49da7fc640 1 -- 172.21.15.105:0/1997988708 >> v1:172.21.15.148:6789/0 conn(0x7f49e40b4290 legacy=0x7f49e40b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:48.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.602+0000 7f49da7fc640 1 -- 172.21.15.105:0/1997988708 >> v1:172.21.15.105:6789/0 conn(0x7f49e415b030 legacy=0x7f49e40709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:48.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.602+0000 7f49da7fc640 1 -- 172.21.15.105:0/1997988708 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e40720d0 con 0x7f49e4071960 2023-11-28T18:28:48.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.606+0000 7f49da7fc640 1 --2- 172.21.15.105:0/1997988708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49e4071960 0x7f49e415f3b0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0097c0 tx=0x7f49cc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352257c36ed722d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:48.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.606+0000 7f49e27fc640 1 -- 172.21.15.105:0/1997988708 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc0ab650 con 0x7f49e4071960 2023-11-28T18:28:48.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.606+0000 7f49e27fc640 1 -- 172.21.15.105:0/1997988708 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc0ab7b0 con 0x7f49e4071960 2023-11-28T18:28:48.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.606+0000 7f49e27fc640 1 -- 172.21.15.105:0/1997988708 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc0abb40 con 0x7f49e4071960 2023-11-28T18:28:48.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.606+0000 7f49e9432640 1 -- 172.21.15.105:0/1997988708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49e4071960 msgr2=0x7f49e415f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:48.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.606+0000 7f49e9432640 1 --2- 172.21.15.105:0/1997988708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49e4071960 0x7f49e415f3b0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0097c0 tx=0x7f49cc01c450 comp rx=0 tx=0).stop 2023-11-28T18:28:48.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.606+0000 7f49e9432640 1 -- 172.21.15.105:0/1997988708 shutdown_connections 2023-11-28T18:28:48.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.606+0000 7f49e9432640 1 --2- 172.21.15.105:0/1997988708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49e4071960 0x7f49e415f3b0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.606+0000 7f49e9432640 1 -- 172.21.15.105:0/1997988708 >> 172.21.15.105:0/1997988708 conn(0x7f49e40ceb90 msgr2=0x7f49e40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:48.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.606+0000 7f49e9432640 1 -- 172.21.15.105:0/1997988708 shutdown_connections 2023-11-28T18:28:48.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.606+0000 7f49e9432640 1 -- 172.21.15.105:0/1997988708 wait complete. 2023-11-28T18:28:48.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.610+0000 7f49e9432640 1 Processor -- start 2023-11-28T18:28:48.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.610+0000 7f49e9432640 1 -- start start 2023-11-28T18:28:48.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.610+0000 7f49e9432640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49e40b4290 0x7f49e4153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:48.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.610+0000 7f49e9432640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49e415b030 0x7f49e4099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:48.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.610+0000 7f49e9432640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49e4099f90 0x7f49e414d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:48.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.610+0000 7f49e9432640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e409a440 con 0x7f49e415b030 2023-11-28T18:28:48.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.610+0000 7f49e9432640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e4154560 con 0x7f49e4099f90 2023-11-28T18:28:48.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.610+0000 7f49e9432640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e4153cc0 con 0x7f49e40b4290 2023-11-28T18:28:48.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.610+0000 7f49da7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49e415b030 0x7f49e4099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:48.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.610+0000 7f49da7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49e415b030 0x7f49e4099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47440/0 (socket says 172.21.15.105:47440) 2023-11-28T18:28:48.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.610+0000 7f49da7fc640 1 -- 172.21.15.105:0/2016846892 learned_addr learned my addr 172.21.15.105:0/2016846892 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:48.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.610+0000 7f49e2ffd640 1 --2- 172.21.15.105:0/2016846892 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49e40b4290 0x7f49e4153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:48.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.610+0000 7f49e37fe640 1 --2- 172.21.15.105:0/2016846892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49e4099f90 0x7f49e414d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:48.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.610+0000 7f49da7fc640 1 -- 172.21.15.105:0/2016846892 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49e40b4290 msgr2=0x7f49e4153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:48.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.610+0000 7f49da7fc640 1 --2- 172.21.15.105:0/2016846892 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49e40b4290 0x7f49e4153430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.610+0000 7f49da7fc640 1 -- 172.21.15.105:0/2016846892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49e4099f90 msgr2=0x7f49e414d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:48.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.610+0000 7f49da7fc640 1 --2- 172.21.15.105:0/2016846892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49e4099f90 0x7f49e414d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.610+0000 7f49da7fc640 1 -- 172.21.15.105:0/2016846892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc009420 con 0x7f49e415b030 2023-11-28T18:28:48.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.610+0000 7f49da7fc640 1 --2- 172.21.15.105:0/2016846892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49e415b030 0x7f49e4099a50 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0ad040 tx=0x7f49cc01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:48.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.614+0000 7f49dbfff640 1 -- 172.21.15.105:0/2016846892 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc0bf610 con 0x7f49e415b030 2023-11-28T18:28:48.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.614+0000 7f49dbfff640 1 -- 172.21.15.105:0/2016846892 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc0bf790 con 0x7f49e415b030 2023-11-28T18:28:48.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.614+0000 7f49dbfff640 1 -- 172.21.15.105:0/2016846892 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc0bfb40 con 0x7f49e415b030 2023-11-28T18:28:48.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.614+0000 7f49e9432640 1 -- 172.21.15.105:0/2016846892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e4153fa0 con 0x7f49e415b030 2023-11-28T18:28:48.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.614+0000 7f49e9432640 1 -- 172.21.15.105:0/2016846892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e415b5a0 con 0x7f49e415b030 2023-11-28T18:28:48.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.614+0000 7f49e9432640 1 -- 172.21.15.105:0/2016846892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e4067570 con 0x7f49e415b030 2023-11-28T18:28:48.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.614+0000 7f49dbfff640 1 -- 172.21.15.105:0/2016846892 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f49cc0b5400 con 0x7f49e415b030 2023-11-28T18:28:48.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.614+0000 7f49dbfff640 1 --2- 172.21.15.105:0/2016846892 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f49c40668d0 0x7f49c4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:48.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.614+0000 7f49dbfff640 1 -- 172.21.15.105:0/2016846892 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f49cc11f150 con 0x7f49e415b030 2023-11-28T18:28:48.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.614+0000 7f49e2ffd640 1 --2- 172.21.15.105:0/2016846892 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f49c40668d0 0x7f49c4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:48.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.614+0000 7f49e2ffd640 1 --2- 172.21.15.105:0/2016846892 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f49c40668d0 0x7f49c4068d90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f49d4001570 tx=0x7f49d4001aa0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:48.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.618+0000 7f49dbfff640 1 -- 172.21.15.105:0/2016846892 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f49cc0ec390 con 0x7f49e415b030 2023-11-28T18:28:48.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.766+0000 7f49e9432640 1 -- 172.21.15.105:0/2016846892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49e40cf7f0 con 0x7f49e415b030 2023-11-28T18:28:48.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.770+0000 7f49dbfff640 1 -- 172.21.15.105:0/2016846892 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16837 (secure 0 0 0) 0x7f49cc0ef9b0 con 0x7f49e415b030 2023-11-28T18:28:48.774 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:48.774 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":78,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:28:14.101062+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:28:11.454373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-28T18:28:12.093171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:28:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.770+0000 7f49e9432640 1 -- 172.21.15.105:0/2016846892 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f49c40668d0 msgr2=0x7f49c4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.770+0000 7f49e9432640 1 --2- 172.21.15.105:0/2016846892 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f49c40668d0 0x7f49c4068d90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f49d4001570 tx=0x7f49d4001aa0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.770+0000 7f49e9432640 1 -- 172.21.15.105:0/2016846892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49e415b030 msgr2=0x7f49e4099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.770+0000 7f49e9432640 1 --2- 172.21.15.105:0/2016846892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49e415b030 0x7f49e4099a50 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0ad040 tx=0x7f49cc01ac90 comp rx=0 tx=0).stop 2023-11-28T18:28:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.770+0000 7f49e9432640 1 -- 172.21.15.105:0/2016846892 shutdown_connections 2023-11-28T18:28:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.770+0000 7f49e9432640 1 --2- 172.21.15.105:0/2016846892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49e4099f90 0x7f49e414d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.770+0000 7f49e9432640 1 --2- 172.21.15.105:0/2016846892 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f49c40668d0 0x7f49c4068d90 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.770+0000 7f49e9432640 1 --2- 172.21.15.105:0/2016846892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49e415b030 0x7f49e4099a50 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.770+0000 7f49e9432640 1 --2- 172.21.15.105:0/2016846892 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49e40b4290 0x7f49e4153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.770+0000 7f49e9432640 1 -- 172.21.15.105:0/2016846892 >> 172.21.15.105:0/2016846892 conn(0x7f49e40ceb90 msgr2=0x7f49e4157ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:48.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.770+0000 7f49e9432640 1 -- 172.21.15.105:0/2016846892 shutdown_connections 2023-11-28T18:28:48.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.770+0000 7f49e9432640 1 -- 172.21.15.105:0/2016846892 wait complete. 2023-11-28T18:28:48.795 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T18:28:48.795 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:28:48.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.930+0000 7f0fd22d6640 1 Processor -- start 2023-11-28T18:28:48.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.930+0000 7f0fd22d6640 1 -- start start 2023-11-28T18:28:48.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.930+0000 7f0fd22d6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0fcc0d3c30 0x7f0fcc0d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:48.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.930+0000 7f0fd22d6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0fcc0b2d20 0x7f0fcc0b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:48.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.930+0000 7f0fd22d6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fcc0b25b0 con 0x7f0fcc15d4f0 2023-11-28T18:28:48.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.930+0000 7f0fd22d6640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fcc0b2900 con 0x7f0fcc0b2d20 2023-11-28T18:28:48.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.930+0000 7f0fd22d6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fcc0d0a20 con 0x7f0fcc0d3c30 2023-11-28T18:28:48.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.930+0000 7f0fd0873640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0fcc0b2d20 0x7f0fcc0b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:48.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.930+0000 7f0fd0873640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0fcc0b2d20 0x7f0fcc0b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47444/0 (socket says 172.21.15.105:47444) 2023-11-28T18:28:48.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.930+0000 7f0fd0873640 1 -- 172.21.15.105:0/203993021 learned_addr learned my addr 172.21.15.105:0/203993021 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:48.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.930+0000 7f0fcb7fe640 1 --2- 172.21.15.105:0/203993021 >> v2:172.21.15.148:3300/0 conn(0x7f0fcc0d3c30 0x7f0fcc0d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:48.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.930+0000 7f0fd0873640 1 -- 172.21.15.105:0/203993021 >> v2:172.21.15.148:3300/0 conn(0x7f0fcc0d3c30 msgr2=0x7f0fcc0d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:48.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.930+0000 7f0fd0873640 1 --2- 172.21.15.105:0/203993021 >> v2:172.21.15.148:3300/0 conn(0x7f0fcc0d3c30 0x7f0fcc0d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.930+0000 7f0fd0873640 1 -- 172.21.15.105:0/203993021 >> v1:172.21.15.148:6789/0 conn(0x7f0fcc15d4f0 legacy=0x7f0fcc0d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:48.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.930+0000 7f0fd0873640 1 -- 172.21.15.105:0/203993021 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fcc0cea30 con 0x7f0fcc0b2d20 2023-11-28T18:28:48.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.934+0000 7f0fd0873640 1 --2- 172.21.15.105:0/203993021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fcc0b2d20 0x7f0fcc0b3150 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc08c430 tx=0x7f0fc401c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ffb85ceb459b66 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:48.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.934+0000 7f0fcaffd640 1 -- 172.21.15.105:0/203993021 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc40a5790 con 0x7f0fcc0b2d20 2023-11-28T18:28:48.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.934+0000 7f0fcaffd640 1 -- 172.21.15.105:0/203993021 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc40a58f0 con 0x7f0fcc0b2d20 2023-11-28T18:28:48.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.934+0000 7f0fcaffd640 1 -- 172.21.15.105:0/203993021 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc40a5c80 con 0x7f0fcc0b2d20 2023-11-28T18:28:48.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.934+0000 7f0fd22d6640 1 -- 172.21.15.105:0/203993021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fcc0b2d20 msgr2=0x7f0fcc0b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:48.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.934+0000 7f0fd22d6640 1 --2- 172.21.15.105:0/203993021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fcc0b2d20 0x7f0fcc0b3150 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc08c430 tx=0x7f0fc401c8d0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.934+0000 7f0fd22d6640 1 -- 172.21.15.105:0/203993021 shutdown_connections 2023-11-28T18:28:48.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.934+0000 7f0fd22d6640 1 --2- 172.21.15.105:0/203993021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fcc0b2d20 0x7f0fcc0b3150 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.934+0000 7f0fd22d6640 1 --2- 172.21.15.105:0/203993021 >> v2:172.21.15.148:3300/0 conn(0x7f0fcc0d3c30 0x7f0fcc0d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.934+0000 7f0fd22d6640 1 -- 172.21.15.105:0/203993021 >> 172.21.15.105:0/203993021 conn(0x7f0fcc0d3030 msgr2=0x7f0fcc0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:48.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.934+0000 7f0fd22d6640 1 -- 172.21.15.105:0/203993021 shutdown_connections 2023-11-28T18:28:48.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.934+0000 7f0fd22d6640 1 -- 172.21.15.105:0/203993021 wait complete. 2023-11-28T18:28:48.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fd22d6640 1 Processor -- start 2023-11-28T18:28:48.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fd22d6640 1 -- start start 2023-11-28T18:28:48.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fd22d6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fcc0b2d20 0x7f0fcc14bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:48.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fd22d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0fcc0d3c30 0x7f0fcc0ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:48.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fd22d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fcc15d4f0 0x7f0fcc06b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:48.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fd22d6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc154070 con 0x7f0fcc0b2d20 2023-11-28T18:28:48.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fd22d6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc15c240 con 0x7f0fcc15d4f0 2023-11-28T18:28:48.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fd22d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc153a20 con 0x7f0fcc0d3c30 2023-11-28T18:28:48.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fcbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fcc0b2d20 0x7f0fcc14bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:48.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fcbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fcc0b2d20 0x7f0fcc14bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47448/0 (socket says 172.21.15.105:47448) 2023-11-28T18:28:48.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fcbfff640 1 -- 172.21.15.105:0/3822024123 learned_addr learned my addr 172.21.15.105:0/3822024123 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:48.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fd0873640 1 --2- 172.21.15.105:0/3822024123 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fcc15d4f0 0x7f0fcc06b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:48.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fcb7fe640 1 --2- 172.21.15.105:0/3822024123 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0fcc0d3c30 0x7f0fcc0ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:48.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fcbfff640 1 -- 172.21.15.105:0/3822024123 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0fcc0d3c30 msgr2=0x7f0fcc0ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:48.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fcbfff640 1 --2- 172.21.15.105:0/3822024123 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0fcc0d3c30 0x7f0fcc0ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fcbfff640 1 -- 172.21.15.105:0/3822024123 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fcc15d4f0 msgr2=0x7f0fcc06b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:48.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fcbfff640 1 --2- 172.21.15.105:0/3822024123 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fcc15d4f0 0x7f0fcc06b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:48.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fcbfff640 1 -- 172.21.15.105:0/3822024123 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc4009b30 con 0x7f0fcc0b2d20 2023-11-28T18:28:48.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fd0873640 1 --2- 172.21.15.105:0/3822024123 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fcc15d4f0 0x7f0fcc06b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:48.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fcbfff640 1 --2- 172.21.15.105:0/3822024123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fcc0b2d20 0x7f0fcc14bc40 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00b410 tx=0x7f0fbc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:48.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fc8ff9640 1 -- 172.21.15.105:0/3822024123 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc004020 con 0x7f0fcc0b2d20 2023-11-28T18:28:48.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fc8ff9640 1 -- 172.21.15.105:0/3822024123 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fbc004180 con 0x7f0fcc0b2d20 2023-11-28T18:28:48.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fc8ff9640 1 -- 172.21.15.105:0/3822024123 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc011630 con 0x7f0fcc0b2d20 2023-11-28T18:28:48.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fd22d6640 1 -- 172.21.15.105:0/3822024123 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fcc153d00 con 0x7f0fcc0b2d20 2023-11-28T18:28:48.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.938+0000 7f0fd22d6640 1 -- 172.21.15.105:0/3822024123 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fcc0ca6b0 con 0x7f0fcc0b2d20 2023-11-28T18:28:48.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.942+0000 7f0fd22d6640 1 -- 172.21.15.105:0/3822024123 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fcc067570 con 0x7f0fcc0b2d20 2023-11-28T18:28:48.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.942+0000 7f0fc8ff9640 1 -- 172.21.15.105:0/3822024123 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f0fbc011790 con 0x7f0fcc0b2d20 2023-11-28T18:28:48.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.942+0000 7f0fc8ff9640 1 --2- 172.21.15.105:0/3822024123 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0fb80668d0 0x7f0fb8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:48.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.942+0000 7f0fc8ff9640 1 -- 172.21.15.105:0/3822024123 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0fbc084210 con 0x7f0fcc0b2d20 2023-11-28T18:28:48.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.942+0000 7f0fcb7fe640 1 --2- 172.21.15.105:0/3822024123 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0fb80668d0 0x7f0fb8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:48.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.942+0000 7f0fcb7fe640 1 --2- 172.21.15.105:0/3822024123 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0fb80668d0 0x7f0fb8068d90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc159140 tx=0x7f0fc0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:48.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:48.946+0000 7f0fc8ff9640 1 -- 172.21.15.105:0/3822024123 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0fbc0513a0 con 0x7f0fcc0b2d20 2023-11-28T18:28:49.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.122+0000 7f0fd22d6640 1 -- 172.21.15.105:0/3822024123 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fcc0cfe10 con 0x7f0fcc0b2d20 2023-11-28T18:28:49.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.122+0000 7f0fc8ff9640 1 -- 172.21.15.105:0/3822024123 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f0fbc0549c0 con 0x7f0fcc0b2d20 2023-11-28T18:28:49.129 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:49.129 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":77,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1455698212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":1455698212},{"type":"v1","addr":"172.21.15.105:6837","nonce":1455698212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:28:13.097621+0000","modified":"2023-11-28T18:28:43.851902+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7194,"mds_1":6862,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"b","rank":2,"incarnation":75,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6834/759522673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":759522673},{"type":"v1","addr":"172.21.15.148:6834","nonce":759522673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6862":{"gid":6862,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6835/2489361300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2489361300},{"type":"v1","addr":"172.21.15.148:6835","nonce":2489361300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7194":{"gid":7194,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6836/188173955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":188173955},{"type":"v1","addr":"172.21.15.105:6836","nonce":188173955}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-28T18:28:49.129 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 77 2023-11-28T18:28:49.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.126+0000 7f0fd22d6640 1 -- 172.21.15.105:0/3822024123 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0fb80668d0 msgr2=0x7f0fb8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:49.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.126+0000 7f0fd22d6640 1 --2- 172.21.15.105:0/3822024123 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0fb80668d0 0x7f0fb8068d90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc159140 tx=0x7f0fc0008040 comp rx=0 tx=0).stop 2023-11-28T18:28:49.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.126+0000 7f0fd22d6640 1 -- 172.21.15.105:0/3822024123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fcc0b2d20 msgr2=0x7f0fcc14bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:49.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.126+0000 7f0fd22d6640 1 --2- 172.21.15.105:0/3822024123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fcc0b2d20 0x7f0fcc14bc40 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00b410 tx=0x7f0fbc00b8d0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.126+0000 7f0fd22d6640 1 -- 172.21.15.105:0/3822024123 shutdown_connections 2023-11-28T18:28:49.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.126+0000 7f0fd22d6640 1 --2- 172.21.15.105:0/3822024123 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fcc15d4f0 0x7f0fcc06b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.126+0000 7f0fd22d6640 1 --2- 172.21.15.105:0/3822024123 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0fcc0d3c30 0x7f0fcc0ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.126+0000 7f0fd22d6640 1 --2- 172.21.15.105:0/3822024123 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0fb80668d0 0x7f0fb8068d90 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.126+0000 7f0fd22d6640 1 --2- 172.21.15.105:0/3822024123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fcc0b2d20 0x7f0fcc14bc40 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.126+0000 7f0fd22d6640 1 -- 172.21.15.105:0/3822024123 >> 172.21.15.105:0/3822024123 conn(0x7f0fcc0d3030 msgr2=0x7f0fcc06ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:49.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.126+0000 7f0fd22d6640 1 -- 172.21.15.105:0/3822024123 shutdown_connections 2023-11-28T18:28:49.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.126+0000 7f0fd22d6640 1 -- 172.21.15.105:0/3822024123 wait complete. 2023-11-28T18:28:49.150 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:28:49.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.282+0000 7f8cee0e3640 1 Processor -- start 2023-11-28T18:28:49.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.286+0000 7f8cee0e3640 1 -- start start 2023-11-28T18:28:49.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.286+0000 7f8cee0e3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ce80b4290 0x7f8ce80b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.286+0000 7f8cee0e3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8ce8071960 0x7f8ce815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.286+0000 7f8cee0e3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8ce815b030 0x7f8ce80709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.286+0000 7f8cee0e3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ce80b4bb0 con 0x7f8ce815b030 2023-11-28T18:28:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.286+0000 7f8cee0e3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ce80701d0 con 0x7f8ce8071960 2023-11-28T18:28:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.286+0000 7f8cee0e3640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce806df80 con 0x7f8ce80b4290 2023-11-28T18:28:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.286+0000 7f8ce7fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8ce815b030 0x7f8ce80709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.286+0000 7f8ce7fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8ce815b030 0x7f8ce80709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47456/0 (socket says 172.21.15.105:47456) 2023-11-28T18:28:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.286+0000 7f8ce7fff640 1 -- 172.21.15.105:0/2830629514 learned_addr learned my addr 172.21.15.105:0/2830629514 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.286+0000 7f8ce6ffd640 1 --2- 172.21.15.105:0/2830629514 >> v2:172.21.15.148:3300/0 conn(0x7f8ce8071960 0x7f8ce815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.286+0000 7f8ce77fe640 1 --2- 172.21.15.105:0/2830629514 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ce80b4290 0x7f8ce80b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.286+0000 7f8ce7fff640 1 -- 172.21.15.105:0/2830629514 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ce80b4290 msgr2=0x7f8ce80b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:49.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.286+0000 7f8ce7fff640 1 --2- 172.21.15.105:0/2830629514 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ce80b4290 0x7f8ce80b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.286+0000 7f8ce7fff640 1 -- 172.21.15.105:0/2830629514 >> v2:172.21.15.148:3300/0 conn(0x7f8ce8071960 msgr2=0x7f8ce815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:49.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.286+0000 7f8ce7fff640 1 --2- 172.21.15.105:0/2830629514 >> v2:172.21.15.148:3300/0 conn(0x7f8ce8071960 0x7f8ce815f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.286+0000 7f8ce7fff640 1 -- 172.21.15.105:0/2830629514 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce8071df0 con 0x7f8ce815b030 2023-11-28T18:28:49.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.286+0000 7f8ce77fe640 1 --2- 172.21.15.105:0/2830629514 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ce80b4290 0x7f8ce80b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:49.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.290+0000 7f8ce7fff640 1 --2- 172.21.15.105:0/2830629514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ce815b030 0x7f8ce80709b0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80cc710 tx=0x7f8ce001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c298c60a714768 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:49.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.290+0000 7f8ce67fc640 1 -- 172.21.15.105:0/2830629514 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce00a5730 con 0x7f8ce815b030 2023-11-28T18:28:49.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.290+0000 7f8ce67fc640 1 -- 172.21.15.105:0/2830629514 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce00a5890 con 0x7f8ce815b030 2023-11-28T18:28:49.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.290+0000 7f8ce67fc640 1 -- 172.21.15.105:0/2830629514 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce00a5c20 con 0x7f8ce815b030 2023-11-28T18:28:49.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.290+0000 7f8cee0e3640 1 -- 172.21.15.105:0/2830629514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ce815b030 msgr2=0x7f8ce80709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:49.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.290+0000 7f8cee0e3640 1 --2- 172.21.15.105:0/2830629514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ce815b030 0x7f8ce80709b0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80cc710 tx=0x7f8ce001c9a0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.290+0000 7f8cee0e3640 1 -- 172.21.15.105:0/2830629514 shutdown_connections 2023-11-28T18:28:49.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.290+0000 7f8cee0e3640 1 --2- 172.21.15.105:0/2830629514 >> v2:172.21.15.148:3300/0 conn(0x7f8ce8071960 0x7f8ce815f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.290+0000 7f8cee0e3640 1 --2- 172.21.15.105:0/2830629514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ce815b030 0x7f8ce80709b0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.290+0000 7f8cee0e3640 1 --2- 172.21.15.105:0/2830629514 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ce80b4290 0x7f8ce80b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.290+0000 7f8cee0e3640 1 -- 172.21.15.105:0/2830629514 >> 172.21.15.105:0/2830629514 conn(0x7f8ce80ceb90 msgr2=0x7f8ce80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:49.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.290+0000 7f8cee0e3640 1 -- 172.21.15.105:0/2830629514 shutdown_connections 2023-11-28T18:28:49.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.290+0000 7f8cee0e3640 1 -- 172.21.15.105:0/2830629514 wait complete. 2023-11-28T18:28:49.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.290+0000 7f8cee0e3640 1 Processor -- start 2023-11-28T18:28:49.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.290+0000 7f8cee0e3640 1 -- start start 2023-11-28T18:28:49.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8cee0e3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ce80b4290 0x7f8ce815ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:49.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8cee0e3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ce815b030 0x7f8ce8151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:49.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8cee0e3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ce8151ab0 0x7f8ce81579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:49.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8cee0e3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce815a7d0 con 0x7f8ce80b4290 2023-11-28T18:28:49.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8cee0e3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce8151f60 con 0x7f8ce8151ab0 2023-11-28T18:28:49.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8cee0e3640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce8159e30 con 0x7f8ce815b030 2023-11-28T18:28:49.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8ce77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ce80b4290 0x7f8ce815ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:49.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8ce77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ce80b4290 0x7f8ce815ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47464/0 (socket says 172.21.15.105:47464) 2023-11-28T18:28:49.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8ce77fe640 1 -- 172.21.15.105:0/2674898562 learned_addr learned my addr 172.21.15.105:0/2674898562 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:49.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8ce7fff640 1 --2- 172.21.15.105:0/2674898562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ce8151ab0 0x7f8ce81579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:49.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8ce6ffd640 1 --2- 172.21.15.105:0/2674898562 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ce815b030 0x7f8ce8151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:49.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8ce77fe640 1 -- 172.21.15.105:0/2674898562 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ce815b030 msgr2=0x7f8ce8151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:49.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8ce77fe640 1 --2- 172.21.15.105:0/2674898562 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ce815b030 0x7f8ce8151570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8ce77fe640 1 -- 172.21.15.105:0/2674898562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ce8151ab0 msgr2=0x7f8ce81579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:49.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8ce77fe640 1 --2- 172.21.15.105:0/2674898562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ce8151ab0 0x7f8ce81579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8ce77fe640 1 -- 172.21.15.105:0/2674898562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce0009b30 con 0x7f8ce80b4290 2023-11-28T18:28:49.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8ce6ffd640 1 --2- 172.21.15.105:0/2674898562 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ce815b030 0x7f8ce8151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:49.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8ce77fe640 1 --2- 172.21.15.105:0/2674898562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ce80b4290 0x7f8ce815ef60 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800e810 tx=0x7f8cd800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:49.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8ccffff640 1 -- 172.21.15.105:0/2674898562 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd800c8d0 con 0x7f8ce80b4290 2023-11-28T18:28:49.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8ccffff640 1 -- 172.21.15.105:0/2674898562 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd800ca30 con 0x7f8ce80b4290 2023-11-28T18:28:49.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8cee0e3640 1 -- 172.21.15.105:0/2674898562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce815a110 con 0x7f8ce80b4290 2023-11-28T18:28:49.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8ccffff640 1 -- 172.21.15.105:0/2674898562 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd800cde0 con 0x7f8ce80b4290 2023-11-28T18:28:49.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8cee0e3640 1 -- 172.21.15.105:0/2674898562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce8159a90 con 0x7f8ce80b4290 2023-11-28T18:28:49.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.294+0000 7f8cee0e3640 1 -- 172.21.15.105:0/2674898562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ce8067570 con 0x7f8ce80b4290 2023-11-28T18:28:49.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.298+0000 7f8ccffff640 1 -- 172.21.15.105:0/2674898562 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8cd8013410 con 0x7f8ce80b4290 2023-11-28T18:28:49.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.298+0000 7f8ccffff640 1 --2- 172.21.15.105:0/2674898562 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8cd40668d0 0x7f8cd4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:49.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.298+0000 7f8ccffff640 1 -- 172.21.15.105:0/2674898562 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8cd80846a0 con 0x7f8ce80b4290 2023-11-28T18:28:49.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.298+0000 7f8ce6ffd640 1 --2- 172.21.15.105:0/2674898562 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8cd40668d0 0x7f8cd4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:49.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.298+0000 7f8ce6ffd640 1 --2- 172.21.15.105:0/2674898562 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8cd40668d0 0x7f8cd4068d90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8ce809ca00 tx=0x7f8cdc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:49.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.302+0000 7f8ccffff640 1 -- 172.21.15.105:0/2674898562 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8cd80517b0 con 0x7f8ce80b4290 2023-11-28T18:28:49.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.482+0000 7f8cee0e3640 1 -- 172.21.15.105:0/2674898562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ce80cf7f0 con 0x7f8ce80b4290 2023-11-28T18:28:49.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.482+0000 7f8ccffff640 1 -- 172.21.15.105:0/2674898562 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f8cd8054dd0 con 0x7f8ce80b4290 2023-11-28T18:28:49.487 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:49.487 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":77,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1455698212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":1455698212},{"type":"v1","addr":"172.21.15.105:6837","nonce":1455698212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:28:13.097621+0000","modified":"2023-11-28T18:28:43.851902+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7194,"mds_1":6862,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"b","rank":2,"incarnation":75,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6834/759522673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":759522673},{"type":"v1","addr":"172.21.15.148:6834","nonce":759522673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6862":{"gid":6862,"name":"d","rank":1,"incarnation":73,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6835/2489361300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2489361300},{"type":"v1","addr":"172.21.15.148:6835","nonce":2489361300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7194":{"gid":7194,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6836/188173955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":188173955},{"type":"v1","addr":"172.21.15.105:6836","nonce":188173955}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-28T18:28:49.487 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 77 2023-11-28T18:28:49.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.486+0000 7f8cee0e3640 1 -- 172.21.15.105:0/2674898562 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8cd40668d0 msgr2=0x7f8cd4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:49.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.486+0000 7f8cee0e3640 1 --2- 172.21.15.105:0/2674898562 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8cd40668d0 0x7f8cd4068d90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8ce809ca00 tx=0x7f8cdc008040 comp rx=0 tx=0).stop 2023-11-28T18:28:49.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.486+0000 7f8cee0e3640 1 -- 172.21.15.105:0/2674898562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ce80b4290 msgr2=0x7f8ce815ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:49.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.486+0000 7f8cee0e3640 1 --2- 172.21.15.105:0/2674898562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ce80b4290 0x7f8ce815ef60 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800e810 tx=0x7f8cd800ecd0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.486+0000 7f8cee0e3640 1 -- 172.21.15.105:0/2674898562 shutdown_connections 2023-11-28T18:28:49.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.486+0000 7f8cee0e3640 1 --2- 172.21.15.105:0/2674898562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ce8151ab0 0x7f8ce81579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.486+0000 7f8cee0e3640 1 --2- 172.21.15.105:0/2674898562 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ce815b030 0x7f8ce8151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.486+0000 7f8cee0e3640 1 --2- 172.21.15.105:0/2674898562 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8cd40668d0 0x7f8cd4068d90 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.486+0000 7f8cee0e3640 1 --2- 172.21.15.105:0/2674898562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ce80b4290 0x7f8ce815ef60 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.486+0000 7f8cee0e3640 1 -- 172.21.15.105:0/2674898562 >> 172.21.15.105:0/2674898562 conn(0x7f8ce80ceb90 msgr2=0x7f8ce80b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:49.490 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.486+0000 7f8cee0e3640 1 -- 172.21.15.105:0/2674898562 shutdown_connections 2023-11-28T18:28:49.490 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.486+0000 7f8cee0e3640 1 -- 172.21.15.105:0/2674898562 wait complete. 2023-11-28T18:28:49.508 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:28:49.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.642+0000 7f335772a640 1 Processor -- start 2023-11-28T18:28:49.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.642+0000 7f335772a640 1 -- start start 2023-11-28T18:28:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.642+0000 7f335772a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3350154070 0x7f335015d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.642+0000 7f335772a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f335006bf50 0x7f33500ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.642+0000 7f335772a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f33500adb80 0x7f33500adff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.642+0000 7f335772a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3350072540 con 0x7f3350154070 2023-11-28T18:28:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.642+0000 7f335772a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33500ae530 con 0x7f33500adb80 2023-11-28T18:28:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.642+0000 7f335772a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33500cc7f0 con 0x7f335006bf50 2023-11-28T18:28:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.642+0000 7f33554c6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3350154070 0x7f335015d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.646+0000 7f33554c6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3350154070 0x7f335015d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47476/0 (socket says 172.21.15.105:47476) 2023-11-28T18:28:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.646+0000 7f33554c6640 1 -- 172.21.15.105:0/1348673326 learned_addr learned my addr 172.21.15.105:0/1348673326 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.646+0000 7f3355cc7640 1 --2- 172.21.15.105:0/1348673326 >> v2:172.21.15.148:3300/0 conn(0x7f33500adb80 0x7f33500adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:49.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.646+0000 7f3354cc5640 1 --2- 172.21.15.105:0/1348673326 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f335006bf50 0x7f33500ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:49.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.646+0000 7f33554c6640 1 -- 172.21.15.105:0/1348673326 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f335006bf50 msgr2=0x7f33500ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:49.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.646+0000 7f33554c6640 1 --2- 172.21.15.105:0/1348673326 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f335006bf50 0x7f33500ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.646+0000 7f33554c6640 1 -- 172.21.15.105:0/1348673326 >> v2:172.21.15.148:3300/0 conn(0x7f33500adb80 msgr2=0x7f33500adff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:49.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.646+0000 7f33554c6640 1 --2- 172.21.15.105:0/1348673326 >> v2:172.21.15.148:3300/0 conn(0x7f33500adb80 0x7f33500adff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.646+0000 7f33554c6640 1 -- 172.21.15.105:0/1348673326 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335015dd10 con 0x7f3350154070 2023-11-28T18:28:49.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.646+0000 7f33554c6640 1 --2- 172.21.15.105:0/1348673326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3350154070 0x7f335015d700 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f335006bc50 tx=0x7f334c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c375dece51192a87 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:49.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.646+0000 7f3347fff640 1 -- 172.21.15.105:0/1348673326 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c0ab600 con 0x7f3350154070 2023-11-28T18:28:49.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.646+0000 7f3347fff640 1 -- 172.21.15.105:0/1348673326 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334c0a7050 con 0x7f3350154070 2023-11-28T18:28:49.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.646+0000 7f3347fff640 1 -- 172.21.15.105:0/1348673326 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c0ab980 con 0x7f3350154070 2023-11-28T18:28:49.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f335772a640 1 -- 172.21.15.105:0/1348673326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3350154070 msgr2=0x7f335015d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f335772a640 1 --2- 172.21.15.105:0/1348673326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3350154070 0x7f335015d700 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f335006bc50 tx=0x7f334c01c4d0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f335772a640 1 -- 172.21.15.105:0/1348673326 shutdown_connections 2023-11-28T18:28:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f335772a640 1 --2- 172.21.15.105:0/1348673326 >> v2:172.21.15.148:3300/0 conn(0x7f33500adb80 0x7f33500adff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f335772a640 1 --2- 172.21.15.105:0/1348673326 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f335006bf50 0x7f33500ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f335772a640 1 --2- 172.21.15.105:0/1348673326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3350154070 0x7f335015d700 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f335772a640 1 -- 172.21.15.105:0/1348673326 >> 172.21.15.105:0/1348673326 conn(0x7f33500ceb90 msgr2=0x7f33500cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f335772a640 1 -- 172.21.15.105:0/1348673326 shutdown_connections 2023-11-28T18:28:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f335772a640 1 -- 172.21.15.105:0/1348673326 wait complete. 2023-11-28T18:28:49.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f335772a640 1 Processor -- start 2023-11-28T18:28:49.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f335772a640 1 -- start start 2023-11-28T18:28:49.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f335772a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f335006bf50 0x7f3350149fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:49.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f335772a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33500adb80 0x7f335014e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:49.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f335772a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3350154070 0x7f335014bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f335772a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335014a4f0 con 0x7f335006bf50 2023-11-28T18:28:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f335772a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335014f2d0 con 0x7f3350154070 2023-11-28T18:28:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f335772a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335014f5b0 con 0x7f33500adb80 2023-11-28T18:28:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f33554c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f335006bf50 0x7f3350149fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f33554c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f335006bf50 0x7f3350149fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47482/0 (socket says 172.21.15.105:47482) 2023-11-28T18:28:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f33554c6640 1 -- 172.21.15.105:0/3112997827 learned_addr learned my addr 172.21.15.105:0/3112997827 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f3354cc5640 1 --2- 172.21.15.105:0/3112997827 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33500adb80 0x7f335014e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f3355cc7640 1 --2- 172.21.15.105:0/3112997827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3350154070 0x7f335014bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f33554c6640 1 -- 172.21.15.105:0/3112997827 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33500adb80 msgr2=0x7f335014e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f33554c6640 1 --2- 172.21.15.105:0/3112997827 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33500adb80 0x7f335014e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f33554c6640 1 -- 172.21.15.105:0/3112997827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3350154070 msgr2=0x7f335014bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f33554c6640 1 --2- 172.21.15.105:0/3112997827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3350154070 0x7f335014bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f33554c6640 1 -- 172.21.15.105:0/3112997827 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334c0094d0 con 0x7f335006bf50 2023-11-28T18:28:49.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.650+0000 7f3355cc7640 1 --2- 172.21.15.105:0/3112997827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3350154070 0x7f335014bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:49.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.654+0000 7f33554c6640 1 --2- 172.21.15.105:0/3112997827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f335006bf50 0x7f3350149fb0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f334c0ae040 tx=0x7f334c019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.654+0000 7f3345ffb640 1 -- 172.21.15.105:0/3112997827 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c0b4440 con 0x7f335006bf50 2023-11-28T18:28:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.654+0000 7f3345ffb640 1 -- 172.21.15.105:0/3112997827 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334c0b45c0 con 0x7f335006bf50 2023-11-28T18:28:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.654+0000 7f3345ffb640 1 -- 172.21.15.105:0/3112997827 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c0b4970 con 0x7f335006bf50 2023-11-28T18:28:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.654+0000 7f335772a640 1 -- 172.21.15.105:0/3112997827 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f335014f890 con 0x7f335006bf50 2023-11-28T18:28:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.654+0000 7f335772a640 1 -- 172.21.15.105:0/3112997827 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335014fe50 con 0x7f335006bf50 2023-11-28T18:28:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.654+0000 7f335772a640 1 -- 172.21.15.105:0/3112997827 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3350067570 con 0x7f335006bf50 2023-11-28T18:28:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.654+0000 7f3345ffb640 1 -- 172.21.15.105:0/3112997827 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f334c0b2070 con 0x7f335006bf50 2023-11-28T18:28:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.654+0000 7f3345ffb640 1 --2- 172.21.15.105:0/3112997827 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f33340668d0 0x7f3334068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.654+0000 7f3345ffb640 1 -- 172.21.15.105:0/3112997827 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f334c11e3a0 con 0x7f335006bf50 2023-11-28T18:28:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.658+0000 7f3354cc5640 1 --2- 172.21.15.105:0/3112997827 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f33340668d0 0x7f3334068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:49.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.658+0000 7f3354cc5640 1 --2- 172.21.15.105:0/3112997827 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f33340668d0 0x7f3334068d90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f335006e3f0 tx=0x7f3340009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:49.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.658+0000 7f3345ffb640 1 -- 172.21.15.105:0/3112997827 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f334c0eb400 con 0x7f335006bf50 2023-11-28T18:28:49.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.814+0000 7f335772a640 1 -- 172.21.15.105:0/3112997827 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33500cf950 con 0x7f335006bf50 2023-11-28T18:28:49.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.814+0000 7f3345ffb640 1 -- 172.21.15.105:0/3112997827 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16837 (secure 0 0 0) 0x7f334c0eea20 con 0x7f335006bf50 2023-11-28T18:28:49.818 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:49.818 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":78,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:28:14.101062+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:28:11.454373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-28T18:28:12.093171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:28:49.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.814+0000 7f335772a640 1 -- 172.21.15.105:0/3112997827 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f33340668d0 msgr2=0x7f3334068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:49.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.814+0000 7f335772a640 1 --2- 172.21.15.105:0/3112997827 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f33340668d0 0x7f3334068d90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f335006e3f0 tx=0x7f3340009190 comp rx=0 tx=0).stop 2023-11-28T18:28:49.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.814+0000 7f335772a640 1 -- 172.21.15.105:0/3112997827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f335006bf50 msgr2=0x7f3350149fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:49.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.814+0000 7f335772a640 1 --2- 172.21.15.105:0/3112997827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f335006bf50 0x7f3350149fb0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f334c0ae040 tx=0x7f334c019270 comp rx=0 tx=0).stop 2023-11-28T18:28:49.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.814+0000 7f335772a640 1 -- 172.21.15.105:0/3112997827 shutdown_connections 2023-11-28T18:28:49.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.814+0000 7f335772a640 1 --2- 172.21.15.105:0/3112997827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3350154070 0x7f335014bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.814+0000 7f335772a640 1 --2- 172.21.15.105:0/3112997827 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33500adb80 0x7f335014e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.814+0000 7f335772a640 1 --2- 172.21.15.105:0/3112997827 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f33340668d0 0x7f3334068d90 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.814+0000 7f335772a640 1 --2- 172.21.15.105:0/3112997827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f335006bf50 0x7f3350149fb0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.814+0000 7f335772a640 1 -- 172.21.15.105:0/3112997827 >> 172.21.15.105:0/3112997827 conn(0x7f33500ceb90 msgr2=0x7f335006c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:49.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.814+0000 7f335772a640 1 -- 172.21.15.105:0/3112997827 shutdown_connections 2023-11-28T18:28:49.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.814+0000 7f335772a640 1 -- 172.21.15.105:0/3112997827 wait complete. 2023-11-28T18:28:49.840 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T18:28:49.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.974+0000 7f0e7b3df640 1 Processor -- start 2023-11-28T18:28:49.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.974+0000 7f0e7b3df640 1 -- start start 2023-11-28T18:28:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.974+0000 7f0e7b3df640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0e740caf60 0x7f0e740cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.974+0000 7f0e7b3df640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7406bf50 con 0x7f0e740ad850 2023-11-28T18:28:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.974+0000 7f0e7b3df640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7415a320 con 0x7f0e74159f40 2023-11-28T18:28:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.974+0000 7f0e7b3df640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e740cb3c0 con 0x7f0e740caf60 2023-11-28T18:28:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.974+0000 7f0e7997c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0e740ad850 0x7f0e740701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43894/0 (socket says 172.21.15.105:43894) 2023-11-28T18:28:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.974+0000 7f0e7997c640 1 -- 172.21.15.105:0/1873381401 learned_addr learned my addr 172.21.15.105:0/1873381401 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.974+0000 7f0e7897a640 1 --2- 172.21.15.105:0/1873381401 >> v2:172.21.15.148:3300/0 conn(0x7f0e740caf60 0x7f0e740cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.974+0000 7f0e6bfff640 1 -- 172.21.15.105:0/1873381401 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1287156381 0 0) 0x7f0e7406bf50 con 0x7f0e740ad850 2023-11-28T18:28:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.974+0000 7f0e6bfff640 1 -- 172.21.15.105:0/1873381401 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e60003530 con 0x7f0e740ad850 2023-11-28T18:28:49.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.974+0000 7f0e6bfff640 1 -- 172.21.15.105:0/1873381401 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1222662570 0 0) 0x7f0e60003530 con 0x7f0e740ad850 2023-11-28T18:28:49.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.974+0000 7f0e6bfff640 1 -- 172.21.15.105:0/1873381401 >> v2:172.21.15.148:3300/0 conn(0x7f0e740caf60 msgr2=0x7f0e740cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:49.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.974+0000 7f0e6bfff640 1 --2- 172.21.15.105:0/1873381401 >> v2:172.21.15.148:3300/0 conn(0x7f0e740caf60 0x7f0e740cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.974+0000 7f0e6bfff640 1 -- 172.21.15.105:0/1873381401 >> v1:172.21.15.148:6789/0 conn(0x7f0e74159f40 legacy=0x7f0e740ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:49.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.974+0000 7f0e6bfff640 1 -- 172.21.15.105:0/1873381401 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e74071380 con 0x7f0e740ad850 2023-11-28T18:28:49.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.974+0000 7f0e6bfff640 1 -- 172.21.15.105:0/1873381401 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f0e6c001a00 con 0x7f0e740ad850 2023-11-28T18:28:49.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.974+0000 7f0e6bfff640 1 -- 172.21.15.105:0/1873381401 >> v1:172.21.15.105:6789/0 conn(0x7f0e740ad850 legacy=0x7f0e740701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:49.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.974+0000 7f0e6bfff640 1 --2- 172.21.15.105:0/1873381401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e60003980 0x7f0e60003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.978+0000 7f0e6bfff640 1 --2- 172.21.15.105:0/1873381401 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e60004380 0x7f0e60004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.978+0000 7f0e6bfff640 1 --2- 172.21.15.105:0/1873381401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e600050f0 0x7f0e6000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.978+0000 7f0e6bfff640 1 -- 172.21.15.105:0/1873381401 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e740cb3c0 con 0x7f0e600050f0 2023-11-28T18:28:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.978+0000 7f0e6bfff640 1 -- 172.21.15.105:0/1873381401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6000c030 con 0x7f0e60003980 2023-11-28T18:28:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.978+0000 7f0e6bfff640 1 -- 172.21.15.105:0/1873381401 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6000c1f0 con 0x7f0e60004380 2023-11-28T18:28:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.978+0000 7f0e7997c640 1 --2- 172.21.15.105:0/1873381401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e600050f0 0x7f0e6000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.978+0000 7f0e7897a640 1 --2- 172.21.15.105:0/1873381401 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e60004380 0x7f0e60004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.978+0000 7f0e7917b640 1 --2- 172.21.15.105:0/1873381401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e60003980 0x7f0e60003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.978+0000 7f0e7997c640 1 -- 172.21.15.105:0/1873381401 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e60004380 msgr2=0x7f0e60004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.978+0000 7f0e7997c640 1 --2- 172.21.15.105:0/1873381401 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e60004380 0x7f0e60004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.978+0000 7f0e7997c640 1 -- 172.21.15.105:0/1873381401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e60003980 msgr2=0x7f0e60003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.978+0000 7f0e7997c640 1 --2- 172.21.15.105:0/1873381401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e60003980 0x7f0e60003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.978+0000 7f0e7997c640 1 -- 172.21.15.105:0/1873381401 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e60003530 con 0x7f0e600050f0 2023-11-28T18:28:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.978+0000 7f0e7997c640 1 -- 172.21.15.105:0/1873381401 reap_dead start 2023-11-28T18:28:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.978+0000 7f0e7997c640 1 --2- 172.21.15.105:0/1873381401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e600050f0 0x7f0e6000b990 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00a7d0 tx=0x7f0e6c01d290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c4596690839dd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.978+0000 7f0e6bfff640 1 -- 172.21.15.105:0/1873381401 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0ad630 con 0x7f0e600050f0 2023-11-28T18:28:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.978+0000 7f0e6bfff640 1 -- 172.21.15.105:0/1873381401 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c0a8070 con 0x7f0e600050f0 2023-11-28T18:28:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.978+0000 7f0e6bfff640 1 -- 172.21.15.105:0/1873381401 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0ad9f0 con 0x7f0e600050f0 2023-11-28T18:28:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7b3df640 1 -- 172.21.15.105:0/1873381401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e600050f0 msgr2=0x7f0e6000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7b3df640 1 --2- 172.21.15.105:0/1873381401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e600050f0 0x7f0e6000b990 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00a7d0 tx=0x7f0e6c01d290 comp rx=0 tx=0).stop 2023-11-28T18:28:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7b3df640 1 -- 172.21.15.105:0/1873381401 shutdown_connections 2023-11-28T18:28:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7b3df640 1 --2- 172.21.15.105:0/1873381401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e600050f0 0x7f0e6000b990 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7b3df640 1 -- 172.21.15.105:0/1873381401 >> 172.21.15.105:0/1873381401 conn(0x7f0e740ceb90 msgr2=0x7f0e740cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7b3df640 1 -- 172.21.15.105:0/1873381401 shutdown_connections 2023-11-28T18:28:49.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7b3df640 1 -- 172.21.15.105:0/1873381401 wait complete. 2023-11-28T18:28:49.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7b3df640 1 Processor -- start 2023-11-28T18:28:49.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7b3df640 1 -- start start 2023-11-28T18:28:49.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7b3df640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e600050f0 0x7f0e7409ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:49.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7b3df640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e7414a600 0x7f0e7413d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:49.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7b3df640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e7415a320 0x7f0e740c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:49.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7b3df640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7409b3d0 con 0x7f0e7415a320 2023-11-28T18:28:49.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7b3df640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7413d920 con 0x7f0e7414a600 2023-11-28T18:28:49.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7b3df640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7414ade0 con 0x7f0e600050f0 2023-11-28T18:28:49.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7997c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e7415a320 0x7f0e740c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:49.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7997c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e7415a320 0x7f0e740c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47494/0 (socket says 172.21.15.105:47494) 2023-11-28T18:28:49.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7997c640 1 -- 172.21.15.105:0/2851282957 learned_addr learned my addr 172.21.15.105:0/2851282957 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:49.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7897a640 1 --2- 172.21.15.105:0/2851282957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e7414a600 0x7f0e7413d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:49.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7917b640 1 --2- 172.21.15.105:0/2851282957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e600050f0 0x7f0e7409ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:49.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7997c640 1 -- 172.21.15.105:0/2851282957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e600050f0 msgr2=0x7f0e7409ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:49.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7997c640 1 --2- 172.21.15.105:0/2851282957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e600050f0 0x7f0e7409ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7997c640 1 -- 172.21.15.105:0/2851282957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e7414a600 msgr2=0x7f0e7413d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:49.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7997c640 1 --2- 172.21.15.105:0/2851282957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e7414a600 0x7f0e7413d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:49.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.982+0000 7f0e7997c640 1 -- 172.21.15.105:0/2851282957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e740c8b90 con 0x7f0e7415a320 2023-11-28T18:28:49.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.986+0000 7f0e7997c640 1 --2- 172.21.15.105:0/2851282957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e7415a320 0x7f0e740c9c30 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c01db30 tx=0x7f0e60005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:49.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.986+0000 7f0e69ffb640 1 -- 172.21.15.105:0/2851282957 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0019c0 con 0x7f0e7415a320 2023-11-28T18:28:49.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.986+0000 7f0e69ffb640 1 -- 172.21.15.105:0/2851282957 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c001b20 con 0x7f0e7415a320 2023-11-28T18:28:49.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.986+0000 7f0e7b3df640 1 -- 172.21.15.105:0/2851282957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e740c8e70 con 0x7f0e7415a320 2023-11-28T18:28:49.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.986+0000 7f0e69ffb640 1 -- 172.21.15.105:0/2851282957 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0ad630 con 0x7f0e7415a320 2023-11-28T18:28:49.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.986+0000 7f0e7b3df640 1 -- 172.21.15.105:0/2851282957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e740c9430 con 0x7f0e7415a320 2023-11-28T18:28:49.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.986+0000 7f0e7b3df640 1 -- 172.21.15.105:0/2851282957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e74067570 con 0x7f0e7415a320 2023-11-28T18:28:49.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.986+0000 7f0e69ffb640 1 -- 172.21.15.105:0/2851282957 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f0e6c0aa030 con 0x7f0e7415a320 2023-11-28T18:28:49.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.986+0000 7f0e69ffb640 1 --2- 172.21.15.105:0/2851282957 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0e580668d0 0x7f0e58068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:49.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.986+0000 7f0e69ffb640 1 -- 172.21.15.105:0/2851282957 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0e6c11de90 con 0x7f0e7415a320 2023-11-28T18:28:49.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.986+0000 7f0e7917b640 1 --2- 172.21.15.105:0/2851282957 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0e580668d0 0x7f0e58068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:49.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.990+0000 7f0e7917b640 1 --2- 172.21.15.105:0/2851282957 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0e580668d0 0x7f0e58068d90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0e70007fd0 tx=0x7f0e70009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:49.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:49.990+0000 7f0e69ffb640 1 -- 172.21.15.105:0/2851282957 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0e6c0eb070 con 0x7f0e7415a320 2023-11-28T18:28:50.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.174+0000 7f0e7b3df640 1 -- 172.21.15.105:0/2851282957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0e740cf950 con 0x7f0e7415a320 2023-11-28T18:28:50.179 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-11-28T18:28:50.174+0000 7fd89b32a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T18:28:50.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.178+0000 7f0e69ffb640 1 -- 172.21.15.105:0/2851282957 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v78) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0e6c0ee690 con 0x7f0e7415a320 2023-11-28T18:28:50.182 INFO:teuthology.orchestra.run.smithi105.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T18:28:50.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.178+0000 7f0e7b3df640 1 -- 172.21.15.105:0/2851282957 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0e580668d0 msgr2=0x7f0e58068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.178+0000 7f0e7b3df640 1 --2- 172.21.15.105:0/2851282957 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0e580668d0 0x7f0e58068d90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0e70007fd0 tx=0x7f0e70009040 comp rx=0 tx=0).stop 2023-11-28T18:28:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.178+0000 7f0e7b3df640 1 -- 172.21.15.105:0/2851282957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e7415a320 msgr2=0x7f0e740c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.178+0000 7f0e7b3df640 1 --2- 172.21.15.105:0/2851282957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e7415a320 0x7f0e740c9c30 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c01db30 tx=0x7f0e60005800 comp rx=0 tx=0).stop 2023-11-28T18:28:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.178+0000 7f0e7b3df640 1 -- 172.21.15.105:0/2851282957 shutdown_connections 2023-11-28T18:28:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.178+0000 7f0e7b3df640 1 --2- 172.21.15.105:0/2851282957 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0e580668d0 0x7f0e58068d90 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.178+0000 7f0e7b3df640 1 --2- 172.21.15.105:0/2851282957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e7415a320 0x7f0e740c9c30 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.178+0000 7f0e7b3df640 1 --2- 172.21.15.105:0/2851282957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e7414a600 0x7f0e7413d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.178+0000 7f0e7b3df640 1 --2- 172.21.15.105:0/2851282957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e600050f0 0x7f0e7409ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.178+0000 7f0e7b3df640 1 -- 172.21.15.105:0/2851282957 >> 172.21.15.105:0/2851282957 conn(0x7f0e740ceb90 msgr2=0x7f0e7406c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.178+0000 7f0e7b3df640 1 -- 172.21.15.105:0/2851282957 shutdown_connections 2023-11-28T18:28:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.178+0000 7f0e7b3df640 1 -- 172.21.15.105:0/2851282957 wait complete. 2023-11-28T18:28:50.207 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T18:28:50.234 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2023-11-28T18:28:50.244 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2023-11-28T18:28:50.255 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2023-11-28T18:28:50.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.346+0000 7f39f9a5a640 1 Processor -- start 2023-11-28T18:28:50.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.346+0000 7f39f9a5a640 1 -- start start 2023-11-28T18:28:50.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.346+0000 7f39f9a5a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f39f415d4f0 0x7f39f40d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:50.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.346+0000 7f39f9a5a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f39f40b2d20 0x7f39f40b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:50.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.346+0000 7f39f9a5a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f40b25b0 con 0x7f39f40d3c30 2023-11-28T18:28:50.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.346+0000 7f39f9a5a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f40b2900 con 0x7f39f415d4f0 2023-11-28T18:28:50.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.346+0000 7f39f9a5a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f40d0a20 con 0x7f39f40b2d20 2023-11-28T18:28:50.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.346+0000 7f39f2ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f39f415d4f0 0x7f39f40d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:50.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.346+0000 7f39f2ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f39f415d4f0 0x7f39f40d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47502/0 (socket says 172.21.15.105:47502) 2023-11-28T18:28:50.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.346+0000 7f39f2ffd640 1 -- 172.21.15.105:0/2931761281 learned_addr learned my addr 172.21.15.105:0/2931761281 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:50.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.346+0000 7f39f37fe640 1 --2- 172.21.15.105:0/2931761281 >> v2:172.21.15.148:3300/0 conn(0x7f39f40b2d20 0x7f39f40b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:50.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.346+0000 7f39f2ffd640 1 -- 172.21.15.105:0/2931761281 >> v2:172.21.15.148:3300/0 conn(0x7f39f40b2d20 msgr2=0x7f39f40b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:50.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.346+0000 7f39f2ffd640 1 --2- 172.21.15.105:0/2931761281 >> v2:172.21.15.148:3300/0 conn(0x7f39f40b2d20 0x7f39f40b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:50.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.346+0000 7f39f2ffd640 1 -- 172.21.15.105:0/2931761281 >> v1:172.21.15.148:6789/0 conn(0x7f39f40d3c30 legacy=0x7f39f40d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:50.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.346+0000 7f39f2ffd640 1 -- 172.21.15.105:0/2931761281 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f40cea30 con 0x7f39f415d4f0 2023-11-28T18:28:50.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.350+0000 7f39f2ffd640 1 --2- 172.21.15.105:0/2931761281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f415d4f0 0x7f39f40d0e10 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f39f4160fa0 tx=0x7f39e401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88789090a4d612cc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:50.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.350+0000 7f39f27fc640 1 -- 172.21.15.105:0/2931761281 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e40ab640 con 0x7f39f415d4f0 2023-11-28T18:28:50.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.350+0000 7f39f27fc640 1 -- 172.21.15.105:0/2931761281 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e40a7070 con 0x7f39f415d4f0 2023-11-28T18:28:50.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.350+0000 7f39f27fc640 1 -- 172.21.15.105:0/2931761281 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e40ab9c0 con 0x7f39f415d4f0 2023-11-28T18:28:50.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.350+0000 7f39f9a5a640 1 -- 172.21.15.105:0/2931761281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f415d4f0 msgr2=0x7f39f40d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:50.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.350+0000 7f39f9a5a640 1 --2- 172.21.15.105:0/2931761281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f415d4f0 0x7f39f40d0e10 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f39f4160fa0 tx=0x7f39e401c440 comp rx=0 tx=0).stop 2023-11-28T18:28:50.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.350+0000 7f39f9a5a640 1 -- 172.21.15.105:0/2931761281 shutdown_connections 2023-11-28T18:28:50.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.350+0000 7f39f9a5a640 1 --2- 172.21.15.105:0/2931761281 >> v2:172.21.15.148:3300/0 conn(0x7f39f40b2d20 0x7f39f40b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:50.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.350+0000 7f39f9a5a640 1 --2- 172.21.15.105:0/2931761281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f415d4f0 0x7f39f40d0e10 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:50.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.350+0000 7f39f9a5a640 1 -- 172.21.15.105:0/2931761281 >> 172.21.15.105:0/2931761281 conn(0x7f39f40d3030 msgr2=0x7f39f40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:50.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.350+0000 7f39f9a5a640 1 -- 172.21.15.105:0/2931761281 shutdown_connections 2023-11-28T18:28:50.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.350+0000 7f39f9a5a640 1 -- 172.21.15.105:0/2931761281 wait complete. 2023-11-28T18:28:50.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39f9a5a640 1 Processor -- start 2023-11-28T18:28:50.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39f9a5a640 1 -- start start 2023-11-28T18:28:50.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39f9a5a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39f40b2d20 0x7f39f414c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:50.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39f9a5a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f40d3c30 0x7f39f40cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:50.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39f9a5a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39f415d4f0 0x7f39f40d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:50.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39f9a5a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f414ca90 con 0x7f39f40d3c30 2023-11-28T18:28:50.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39f9a5a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f415a540 con 0x7f39f40b2d20 2023-11-28T18:28:50.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39f9a5a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f415ab00 con 0x7f39f415d4f0 2023-11-28T18:28:50.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39ebfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f40d3c30 0x7f39f40cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:50.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39ebfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f40d3c30 0x7f39f40cc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47512/0 (socket says 172.21.15.105:47512) 2023-11-28T18:28:50.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39ebfff640 1 -- 172.21.15.105:0/37086619 learned_addr learned my addr 172.21.15.105:0/37086619 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:50.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39f2ffd640 1 --2- 172.21.15.105:0/37086619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39f40b2d20 0x7f39f414c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:50.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39f37fe640 1 --2- 172.21.15.105:0/37086619 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39f415d4f0 0x7f39f40d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:50.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39ebfff640 1 -- 172.21.15.105:0/37086619 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39f415d4f0 msgr2=0x7f39f40d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:50.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39ebfff640 1 --2- 172.21.15.105:0/37086619 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39f415d4f0 0x7f39f40d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:50.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39ebfff640 1 -- 172.21.15.105:0/37086619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39f40b2d20 msgr2=0x7f39f414c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:50.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39ebfff640 1 --2- 172.21.15.105:0/37086619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39f40b2d20 0x7f39f414c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:50.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39ebfff640 1 -- 172.21.15.105:0/37086619 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e40094d0 con 0x7f39f40d3c30 2023-11-28T18:28:50.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39f2ffd640 1 --2- 172.21.15.105:0/37086619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39f40b2d20 0x7f39f414c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:28:50.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39ebfff640 1 --2- 172.21.15.105:0/37086619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f40d3c30 0x7f39f40cc260 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00c3f0 tx=0x7f39dc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:50.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39f0ff9640 1 -- 172.21.15.105:0/37086619 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc0052d0 con 0x7f39f40d3c30 2023-11-28T18:28:50.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39f0ff9640 1 -- 172.21.15.105:0/37086619 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39dc005430 con 0x7f39f40d3c30 2023-11-28T18:28:50.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39f0ff9640 1 -- 172.21.15.105:0/37086619 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc003840 con 0x7f39f40d3c30 2023-11-28T18:28:50.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39f9a5a640 1 -- 172.21.15.105:0/37086619 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f40ccab0 con 0x7f39f40d3c30 2023-11-28T18:28:50.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39f9a5a640 1 -- 172.21.15.105:0/37086619 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f415a790 con 0x7f39f40d3c30 2023-11-28T18:28:50.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.354+0000 7f39f9a5a640 1 -- 172.21.15.105:0/37086619 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f4067570 con 0x7f39f40d3c30 2023-11-28T18:28:50.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.358+0000 7f39f0ff9640 1 -- 172.21.15.105:0/37086619 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f39dc0039a0 con 0x7f39f40d3c30 2023-11-28T18:28:50.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.358+0000 7f39f0ff9640 1 --2- 172.21.15.105:0/37086619 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f39d40668d0 0x7f39d4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:50.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.358+0000 7f39f0ff9640 1 -- 172.21.15.105:0/37086619 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f39dc083b20 con 0x7f39f40d3c30 2023-11-28T18:28:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.358+0000 7f39f2ffd640 1 --2- 172.21.15.105:0/37086619 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f39d40668d0 0x7f39d4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.358+0000 7f39f2ffd640 1 --2- 172.21.15.105:0/37086619 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f39d40668d0 0x7f39d4068d90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f39e40af000 tx=0x7f39e40b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.362+0000 7f39f0ff9640 1 -- 172.21.15.105:0/37086619 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f39dc050bc0 con 0x7f39f40d3c30 2023-11-28T18:28:50.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:50.542+0000 7f39f9a5a640 1 -- 172.21.15.105:0/37086619 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f39f415a920 con 0x7f39f40d3c30 2023-11-28T18:28:51.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.206+0000 7f39f0ff9640 1 -- 172.21.15.105:0/37086619 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v79) v1 ==== 104+0+0 (secure 0 0 0) 0x7f39dc0541e0 con 0x7f39f40d3c30 2023-11-28T18:28:51.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.206+0000 7f39f9a5a640 1 -- 172.21.15.105:0/37086619 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f39d40668d0 msgr2=0x7f39d4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:51.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.206+0000 7f39f9a5a640 1 --2- 172.21.15.105:0/37086619 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f39d40668d0 0x7f39d4068d90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f39e40af000 tx=0x7f39e40b6040 comp rx=0 tx=0).stop 2023-11-28T18:28:51.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.206+0000 7f39f9a5a640 1 -- 172.21.15.105:0/37086619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f40d3c30 msgr2=0x7f39f40cc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:51.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.206+0000 7f39f9a5a640 1 --2- 172.21.15.105:0/37086619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f40d3c30 0x7f39f40cc260 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00c3f0 tx=0x7f39dc00c8b0 comp rx=0 tx=0).stop 2023-11-28T18:28:51.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.206+0000 7f39f9a5a640 1 -- 172.21.15.105:0/37086619 shutdown_connections 2023-11-28T18:28:51.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.206+0000 7f39f9a5a640 1 --2- 172.21.15.105:0/37086619 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39f415d4f0 0x7f39f40d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:51.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.206+0000 7f39f9a5a640 1 --2- 172.21.15.105:0/37086619 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f39d40668d0 0x7f39d4068d90 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:51.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.206+0000 7f39f9a5a640 1 --2- 172.21.15.105:0/37086619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f40d3c30 0x7f39f40cc260 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:51.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.206+0000 7f39f9a5a640 1 --2- 172.21.15.105:0/37086619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39f40b2d20 0x7f39f414c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:51.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.206+0000 7f39f9a5a640 1 -- 172.21.15.105:0/37086619 >> 172.21.15.105:0/37086619 conn(0x7f39f40d3030 msgr2=0x7f39f406c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:51.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.206+0000 7f39f9a5a640 1 -- 172.21.15.105:0/37086619 shutdown_connections 2023-11-28T18:28:51.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.206+0000 7f39f9a5a640 1 -- 172.21.15.105:0/37086619 wait complete. 2023-11-28T18:28:51.235 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T18:28:51.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.374+0000 7f3ae1e9a640 1 Processor -- start 2023-11-28T18:28:51.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.374+0000 7f3ae1e9a640 1 -- start start 2023-11-28T18:28:51.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.378+0000 7f3ae1e9a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3adc159610 0x7f3adc0ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:51.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.378+0000 7f3ae1e9a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3adc0cde30 con 0x7f3adc06cef0 2023-11-28T18:28:51.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.378+0000 7f3ae1e9a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3adc1599f0 con 0x7f3adc0cc760 2023-11-28T18:28:51.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.378+0000 7f3ae1e9a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3adc06de20 con 0x7f3adc159610 2023-11-28T18:28:51.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.378+0000 7f3adb7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3adc159610 0x7f3adc0ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:51.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.378+0000 7f3adb7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3adc159610 0x7f3adc0ce2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39128/0 (socket says 172.21.15.105:39128) 2023-11-28T18:28:51.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.378+0000 7f3adbfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3adc06cef0 0x7f3adc0abd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43678/0 (socket says 172.21.15.105:43678) 2023-11-28T18:28:51.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.378+0000 7f3adb7fe640 1 -- 172.21.15.105:0/377636930 learned_addr learned my addr 172.21.15.105:0/377636930 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:51.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.378+0000 7f3adb7fe640 1 -- 172.21.15.105:0/377636930 >> v1:172.21.15.148:6789/0 conn(0x7f3adc0cc760 legacy=0x7f3adc06c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:51.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.378+0000 7f3adb7fe640 1 -- 172.21.15.105:0/377636930 >> v1:172.21.15.105:6789/0 conn(0x7f3adc06cef0 legacy=0x7f3adc0abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:51.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.378+0000 7f3adb7fe640 1 -- 172.21.15.105:0/377636930 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3adc0cd220 con 0x7f3adc159610 2023-11-28T18:28:51.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.382+0000 7f3adb7fe640 1 --2- 172.21.15.105:0/377636930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3adc159610 0x7f3adc0ce2c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00aa10 tx=0x7f3adc0acca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00fa175642794d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:51.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.382+0000 7f3adaffd640 1 -- 172.21.15.105:0/377636930 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc0a68d0 con 0x7f3adc159610 2023-11-28T18:28:51.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.382+0000 7f3adaffd640 1 -- 172.21.15.105:0/377636930 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3acc0a6a30 con 0x7f3adc159610 2023-11-28T18:28:51.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.382+0000 7f3adaffd640 1 -- 172.21.15.105:0/377636930 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc0a6dc0 con 0x7f3adc159610 2023-11-28T18:28:51.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.382+0000 7f3ae1e9a640 1 -- 172.21.15.105:0/377636930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3adc159610 msgr2=0x7f3adc0ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:51.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.382+0000 7f3ae1e9a640 1 --2- 172.21.15.105:0/377636930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3adc159610 0x7f3adc0ce2c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00aa10 tx=0x7f3adc0acca0 comp rx=0 tx=0).stop 2023-11-28T18:28:51.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.382+0000 7f3ae1e9a640 1 -- 172.21.15.105:0/377636930 shutdown_connections 2023-11-28T18:28:51.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.382+0000 7f3ae1e9a640 1 --2- 172.21.15.105:0/377636930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3adc159610 0x7f3adc0ce2c0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:51.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.382+0000 7f3ae1e9a640 1 -- 172.21.15.105:0/377636930 >> 172.21.15.105:0/377636930 conn(0x7f3adc0ceb90 msgr2=0x7f3adc0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:51.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.382+0000 7f3ae1e9a640 1 -- 172.21.15.105:0/377636930 shutdown_connections 2023-11-28T18:28:51.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.382+0000 7f3ae1e9a640 1 -- 172.21.15.105:0/377636930 wait complete. 2023-11-28T18:28:51.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.382+0000 7f3ae1e9a640 1 Processor -- start 2023-11-28T18:28:51.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.382+0000 7f3ae1e9a640 1 -- start start 2023-11-28T18:28:51.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3ae1e9a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3adc06cef0 0x7f3adc09c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:51.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3ae1e9a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3adc0cc760 0x7f3adc0ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:51.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3ae1e9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3adc159610 0x7f3adc0aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:51.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3ae1e9a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc09c8e0 con 0x7f3adc159610 2023-11-28T18:28:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3ae1e9a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc099af0 con 0x7f3adc06cef0 2023-11-28T18:28:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3ae1e9a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc099d90 con 0x7f3adc0cc760 2023-11-28T18:28:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3adbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3adc159610 0x7f3adc0aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3adbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3adc159610 0x7f3adc0aeee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39134/0 (socket says 172.21.15.105:39134) 2023-11-28T18:28:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3adbfff640 1 -- 172.21.15.105:0/2883048382 learned_addr learned my addr 172.21.15.105:0/2883048382 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3adb7fe640 1 --2- 172.21.15.105:0/2883048382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3adc06cef0 0x7f3adc09c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3ad3fff640 1 --2- 172.21.15.105:0/2883048382 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3adc0cc760 0x7f3adc0ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3adbfff640 1 -- 172.21.15.105:0/2883048382 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3adc0cc760 msgr2=0x7f3adc0ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3adbfff640 1 --2- 172.21.15.105:0/2883048382 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3adc0cc760 0x7f3adc0ae9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3adbfff640 1 -- 172.21.15.105:0/2883048382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3adc06cef0 msgr2=0x7f3adc09c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:51.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3adbfff640 1 --2- 172.21.15.105:0/2883048382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3adc06cef0 0x7f3adc09c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:51.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3adbfff640 1 -- 172.21.15.105:0/2883048382 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3acc00a670 con 0x7f3adc159610 2023-11-28T18:28:51.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3adbfff640 1 --2- 172.21.15.105:0/2883048382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3adc159610 0x7f3adc0aeee0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400c730 tx=0x7f3ad400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:51.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3ad8ff9640 1 -- 172.21.15.105:0/2883048382 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad400ce40 con 0x7f3adc159610 2023-11-28T18:28:51.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3ad8ff9640 1 -- 172.21.15.105:0/2883048382 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad400a640 con 0x7f3adc159610 2023-11-28T18:28:51.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3ae1e9a640 1 -- 172.21.15.105:0/2883048382 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3adc09a070 con 0x7f3adc159610 2023-11-28T18:28:51.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3ad8ff9640 1 -- 172.21.15.105:0/2883048382 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad400a9f0 con 0x7f3adc159610 2023-11-28T18:28:51.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3ae1e9a640 1 -- 172.21.15.105:0/2883048382 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3adc09a630 con 0x7f3adc159610 2023-11-28T18:28:51.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3ae1e9a640 1 -- 172.21.15.105:0/2883048382 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3adc09a850 con 0x7f3adc159610 2023-11-28T18:28:51.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.386+0000 7f3ad8ff9640 1 -- 172.21.15.105:0/2883048382 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3ad400ab50 con 0x7f3adc159610 2023-11-28T18:28:51.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.390+0000 7f3ad8ff9640 1 --2- 172.21.15.105:0/2883048382 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3abc0668d0 0x7f3abc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:51.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.390+0000 7f3ad8ff9640 1 -- 172.21.15.105:0/2883048382 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f3ad4083e70 con 0x7f3adc159610 2023-11-28T18:28:51.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.390+0000 7f3adb7fe640 1 --2- 172.21.15.105:0/2883048382 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3abc0668d0 0x7f3abc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:51.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.390+0000 7f3adb7fe640 1 --2- 172.21.15.105:0/2883048382 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3abc0668d0 0x7f3abc068d90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0b0000 tx=0x7f3acc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:51.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.394+0000 7f3ad8ff9640 1 -- 172.21.15.105:0/2883048382 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3ad4050df0 con 0x7f3adc159610 2023-11-28T18:28:51.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:51.542+0000 7f3ae1e9a640 1 -- 172.21.15.105:0/2883048382 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3adc067570 con 0x7f3adc159610 2023-11-28T18:28:52.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.218+0000 7f3ad8ff9640 1 -- 172.21.15.105:0/2883048382 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v80) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3ad4017070 con 0x7f3adc159610 2023-11-28T18:28:52.223 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' removed 2023-11-28T18:28:52.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.222+0000 7f3ae1e9a640 1 -- 172.21.15.105:0/2883048382 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3abc0668d0 msgr2=0x7f3abc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:52.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.222+0000 7f3ae1e9a640 1 --2- 172.21.15.105:0/2883048382 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3abc0668d0 0x7f3abc068d90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0b0000 tx=0x7f3acc008040 comp rx=0 tx=0).stop 2023-11-28T18:28:52.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.222+0000 7f3ae1e9a640 1 -- 172.21.15.105:0/2883048382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3adc159610 msgr2=0x7f3adc0aeee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:52.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.222+0000 7f3ae1e9a640 1 --2- 172.21.15.105:0/2883048382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3adc159610 0x7f3adc0aeee0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400c730 tx=0x7f3ad400cbf0 comp rx=0 tx=0).stop 2023-11-28T18:28:52.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.222+0000 7f3ae1e9a640 1 -- 172.21.15.105:0/2883048382 shutdown_connections 2023-11-28T18:28:52.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.222+0000 7f3ae1e9a640 1 --2- 172.21.15.105:0/2883048382 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3abc0668d0 0x7f3abc068d90 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:52.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.222+0000 7f3ae1e9a640 1 --2- 172.21.15.105:0/2883048382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3adc159610 0x7f3adc0aeee0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:52.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.222+0000 7f3ae1e9a640 1 --2- 172.21.15.105:0/2883048382 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3adc0cc760 0x7f3adc0ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:52.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.222+0000 7f3ae1e9a640 1 --2- 172.21.15.105:0/2883048382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3adc06cef0 0x7f3adc09c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:52.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.222+0000 7f3ae1e9a640 1 -- 172.21.15.105:0/2883048382 >> 172.21.15.105:0/2883048382 conn(0x7f3adc0ceb90 msgr2=0x7f3adc0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:52.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.230+0000 7f3ae1e9a640 1 -- 172.21.15.105:0/2883048382 shutdown_connections 2023-11-28T18:28:52.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.230+0000 7f3ae1e9a640 1 -- 172.21.15.105:0/2883048382 wait complete. 2023-11-28T18:28:52.261 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T18:28:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.398+0000 7f075591d640 1 Processor -- start 2023-11-28T18:28:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.398+0000 7f075591d640 1 -- start start 2023-11-28T18:28:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.402+0000 7f075591d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0750169fc0 0x7f07500ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.402+0000 7f075591d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f075016dab0 con 0x7f07500cbc40 2023-11-28T18:28:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.402+0000 7f075591d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f075016dd90 con 0x7f07500abd80 2023-11-28T18:28:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.402+0000 7f075591d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f075016e070 con 0x7f0750169fc0 2023-11-28T18:28:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.402+0000 7f074f7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f07500cbc40 0x7f07500cc910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43684/0 (socket says 172.21.15.105:43684) 2023-11-28T18:28:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.402+0000 7f074f7fe640 1 -- 172.21.15.105:0/1157327268 learned_addr learned my addr 172.21.15.105:0/1157327268 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.402+0000 7f074effd640 1 --2- 172.21.15.105:0/1157327268 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0750169fc0 0x7f07500ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.402+0000 7f074e7fc640 1 -- 172.21.15.105:0/1157327268 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2409346619 0 0) 0x7f075016dab0 con 0x7f07500cbc40 2023-11-28T18:28:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.674+0000 7f074e7fc640 1 -- 172.21.15.105:0/1157327268 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f073c003510 con 0x7f07500cbc40 2023-11-28T18:28:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.674+0000 7f074e7fc640 1 -- 172.21.15.105:0/1157327268 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4088261292 0 0) 0x7f075016dd90 con 0x7f07500abd80 2023-11-28T18:28:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.674+0000 7f074e7fc640 1 -- 172.21.15.105:0/1157327268 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f075016dab0 con 0x7f07500abd80 2023-11-28T18:28:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.674+0000 7f074e7fc640 1 -- 172.21.15.105:0/1157327268 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 164807284 0 0) 0x7f073c003510 con 0x7f07500cbc40 2023-11-28T18:28:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.674+0000 7f074e7fc640 1 -- 172.21.15.105:0/1157327268 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0750169fc0 msgr2=0x7f07500ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.674+0000 7f074e7fc640 1 --2- 172.21.15.105:0/1157327268 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0750169fc0 0x7f07500ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.674+0000 7f074e7fc640 1 -- 172.21.15.105:0/1157327268 >> v1:172.21.15.148:6789/0 conn(0x7f07500abd80 legacy=0x7f07500cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.674+0000 7f074e7fc640 1 -- 172.21.15.105:0/1157327268 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075016e350 con 0x7f07500cbc40 2023-11-28T18:28:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.674+0000 7f074e7fc640 1 -- 172.21.15.105:0/1157327268 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f0748001a00 con 0x7f07500cbc40 2023-11-28T18:28:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.674+0000 7f074e7fc640 1 -- 172.21.15.105:0/1157327268 >> v1:172.21.15.105:6789/0 conn(0x7f07500cbc40 legacy=0x7f07500cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:52.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.674+0000 7f074e7fc640 1 --2- 172.21.15.105:0/1157327268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f073c003ce0 0x7f073c0040e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:52.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.674+0000 7f074e7fc640 1 --2- 172.21.15.105:0/1157327268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f073c004620 0x7f073c00d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:52.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.674+0000 7f074e7fc640 1 --2- 172.21.15.105:0/1157327268 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0750169fc0 0x7f073c00f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:52.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.674+0000 7f074e7fc640 1 -- 172.21.15.105:0/1157327268 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075016e070 con 0x7f073c003ce0 2023-11-28T18:28:52.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.674+0000 7f074e7fc640 1 -- 172.21.15.105:0/1157327268 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c00feb0 con 0x7f073c004620 2023-11-28T18:28:52.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.674+0000 7f074e7fc640 1 -- 172.21.15.105:0/1157327268 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073c010020 con 0x7f0750169fc0 2023-11-28T18:28:52.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.674+0000 7f074effd640 1 --2- 172.21.15.105:0/1157327268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f073c003ce0 0x7f073c0040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:52.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.674+0000 7f07467fc640 1 --2- 172.21.15.105:0/1157327268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f073c004620 0x7f073c00d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:52.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.674+0000 7f074f7fe640 1 --2- 172.21.15.105:0/1157327268 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0750169fc0 0x7f073c00f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:52.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.678+0000 7f074effd640 1 -- 172.21.15.105:0/1157327268 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0750169fc0 msgr2=0x7f073c00f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.678+0000 7f074effd640 1 --2- 172.21.15.105:0/1157327268 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0750169fc0 0x7f073c00f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.678+0000 7f074effd640 1 -- 172.21.15.105:0/1157327268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f073c004620 msgr2=0x7f073c00d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.678+0000 7f074effd640 1 --2- 172.21.15.105:0/1157327268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f073c004620 0x7f073c00d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.678+0000 7f074effd640 1 -- 172.21.15.105:0/1157327268 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073c003510 con 0x7f073c003ce0 2023-11-28T18:28:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.678+0000 7f074f7fe640 1 --2- 172.21.15.105:0/1157327268 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0750169fc0 0x7f073c00f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:52.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.678+0000 7f074effd640 1 --2- 172.21.15.105:0/1157327268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f073c003ce0 0x7f073c0040e0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f0738009a00 tx=0x7f073801c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88ba4050d4f670e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:52.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.678+0000 7f074e7fc640 1 -- 172.21.15.105:0/1157327268 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07380092d0 con 0x7f073c003ce0 2023-11-28T18:28:52.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.678+0000 7f074e7fc640 1 -- 172.21.15.105:0/1157327268 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f073800f040 con 0x7f073c003ce0 2023-11-28T18:28:52.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.678+0000 7f074e7fc640 1 -- 172.21.15.105:0/1157327268 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07380076d0 con 0x7f073c003ce0 2023-11-28T18:28:52.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.682+0000 7f075591d640 1 -- 172.21.15.105:0/1157327268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f073c003ce0 msgr2=0x7f073c0040e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:52.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.682+0000 7f075591d640 1 --2- 172.21.15.105:0/1157327268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f073c003ce0 0x7f073c0040e0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f0738009a00 tx=0x7f073801c680 comp rx=0 tx=0).stop 2023-11-28T18:28:52.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.682+0000 7f074f7fe640 1 -- 172.21.15.105:0/1157327268 reap_dead start 2023-11-28T18:28:52.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.682+0000 7f075591d640 1 -- 172.21.15.105:0/1157327268 shutdown_connections 2023-11-28T18:28:52.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.682+0000 7f075591d640 1 -- 172.21.15.105:0/1157327268 >> 172.21.15.105:0/1157327268 conn(0x7f07500b08c0 msgr2=0x7f07500b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:52.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.682+0000 7f075591d640 1 -- 172.21.15.105:0/1157327268 shutdown_connections 2023-11-28T18:28:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.682+0000 7f075591d640 1 -- 172.21.15.105:0/1157327268 wait complete. 2023-11-28T18:28:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.682+0000 7f075591d640 1 Processor -- start 2023-11-28T18:28:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.682+0000 7f075591d640 1 -- start start 2023-11-28T18:28:52.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f075591d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f073c003ce0 0x7f07500b4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:52.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f075591d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f07500ac6a0 0x7f075015ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:52.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f075591d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07500acb20 0x7f075016ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:52.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f075591d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075016b990 con 0x7f073c003ce0 2023-11-28T18:28:52.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f075591d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075016bc70 con 0x7f07500acb20 2023-11-28T18:28:52.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f075591d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f075016bf50 con 0x7f07500ac6a0 2023-11-28T18:28:52.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f074effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f073c003ce0 0x7f07500b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:52.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f074effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f073c003ce0 0x7f07500b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39166/0 (socket says 172.21.15.105:39166) 2023-11-28T18:28:52.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f074effd640 1 -- 172.21.15.105:0/778730476 learned_addr learned my addr 172.21.15.105:0/778730476 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:52.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f07467fc640 1 --2- 172.21.15.105:0/778730476 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f07500ac6a0 0x7f075015ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:52.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f074f7fe640 1 --2- 172.21.15.105:0/778730476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07500acb20 0x7f075016ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:52.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f074effd640 1 -- 172.21.15.105:0/778730476 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f07500ac6a0 msgr2=0x7f075015ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:52.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f074effd640 1 --2- 172.21.15.105:0/778730476 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f07500ac6a0 0x7f075015ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:52.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f074effd640 1 -- 172.21.15.105:0/778730476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07500acb20 msgr2=0x7f075016ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:52.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f074effd640 1 --2- 172.21.15.105:0/778730476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07500acb20 0x7f075016ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:52.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f074effd640 1 -- 172.21.15.105:0/778730476 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075016c230 con 0x7f073c003ce0 2023-11-28T18:28:52.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f074f7fe640 1 --2- 172.21.15.105:0/778730476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07500acb20 0x7f075016ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:28:52.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f07467fc640 1 --2- 172.21.15.105:0/778730476 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f07500ac6a0 0x7f075015ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:28:52.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f074effd640 1 --2- 172.21.15.105:0/778730476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f073c003ce0 0x7f07500b4670 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f073801c4b0 tx=0x7f0738007170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:52.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f074cff9640 1 -- 172.21.15.105:0/778730476 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0738006040 con 0x7f073c003ce0 2023-11-28T18:28:52.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f074cff9640 1 -- 172.21.15.105:0/778730476 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07380b6070 con 0x7f073c003ce0 2023-11-28T18:28:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f074cff9640 1 -- 172.21.15.105:0/778730476 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0738006400 con 0x7f073c003ce0 2023-11-28T18:28:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f075591d640 1 -- 172.21.15.105:0/778730476 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f075016c510 con 0x7f073c003ce0 2023-11-28T18:28:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f075591d640 1 -- 172.21.15.105:0/778730476 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f075015b2b0 con 0x7f073c003ce0 2023-11-28T18:28:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.686+0000 7f075591d640 1 -- 172.21.15.105:0/778730476 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0750067570 con 0x7f073c003ce0 2023-11-28T18:28:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.690+0000 7f074cff9640 1 -- 172.21.15.105:0/778730476 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f07380a6040 con 0x7f073c003ce0 2023-11-28T18:28:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.690+0000 7f074cff9640 1 --2- 172.21.15.105:0/778730476 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f07300668d0 0x7f0730068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:52.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.690+0000 7f074cff9640 1 -- 172.21.15.105:0/778730476 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f07380b8070 con 0x7f073c003ce0 2023-11-28T18:28:52.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.690+0000 7f07467fc640 1 --2- 172.21.15.105:0/778730476 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f07300668d0 0x7f0730068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:52.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.690+0000 7f07467fc640 1 --2- 172.21.15.105:0/778730476 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f07300668d0 0x7f0730068d90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0750169530 tx=0x7f0740009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:52.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.694+0000 7f074cff9640 1 -- 172.21.15.105:0/778730476 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f07380ebbd0 con 0x7f073c003ce0 2023-11-28T18:28:52.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:52.850+0000 7f075591d640 1 -- 172.21.15.105:0/778730476 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f075006d690 con 0x7f073c003ce0 2023-11-28T18:28:53.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.222+0000 7f074cff9640 1 -- 172.21.15.105:0/778730476 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v81) v1 ==== 169+0+0 (secure 0 0 0) 0x7f07380c20d0 con 0x7f073c003ce0 2023-11-28T18:28:53.226 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' removed 2023-11-28T18:28:53.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.226+0000 7f075591d640 1 -- 172.21.15.105:0/778730476 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f07300668d0 msgr2=0x7f0730068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:53.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.226+0000 7f075591d640 1 --2- 172.21.15.105:0/778730476 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f07300668d0 0x7f0730068d90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0750169530 tx=0x7f0740009040 comp rx=0 tx=0).stop 2023-11-28T18:28:53.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.226+0000 7f075591d640 1 -- 172.21.15.105:0/778730476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f073c003ce0 msgr2=0x7f07500b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:53.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.226+0000 7f075591d640 1 --2- 172.21.15.105:0/778730476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f073c003ce0 0x7f07500b4670 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f073801c4b0 tx=0x7f0738007170 comp rx=0 tx=0).stop 2023-11-28T18:28:53.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.226+0000 7f075591d640 1 -- 172.21.15.105:0/778730476 shutdown_connections 2023-11-28T18:28:53.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.226+0000 7f075591d640 1 --2- 172.21.15.105:0/778730476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07500acb20 0x7f075016ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.226+0000 7f075591d640 1 --2- 172.21.15.105:0/778730476 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f07500ac6a0 0x7f075015ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.226+0000 7f075591d640 1 --2- 172.21.15.105:0/778730476 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f07300668d0 0x7f0730068d90 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.226+0000 7f075591d640 1 --2- 172.21.15.105:0/778730476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f073c003ce0 0x7f07500b4670 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.226+0000 7f075591d640 1 -- 172.21.15.105:0/778730476 >> 172.21.15.105:0/778730476 conn(0x7f07500b08c0 msgr2=0x7f0750070820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:53.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.226+0000 7f075591d640 1 -- 172.21.15.105:0/778730476 shutdown_connections 2023-11-28T18:28:53.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.230+0000 7f075591d640 1 -- 172.21.15.105:0/778730476 wait complete. 2023-11-28T18:28:53.264 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random 2023-11-28T18:28:53.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.398+0000 7fbc15ac9640 1 Processor -- start 2023-11-28T18:28:53.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.398+0000 7fbc15ac9640 1 -- start start 2023-11-28T18:28:53.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.402+0000 7fbc15ac9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbc10159f40 0x7fbc100caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:53.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.402+0000 7fbc15ac9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc100ad850 0x7fbc100701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:53.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.402+0000 7fbc15ac9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc1006bf50 con 0x7fbc100cc760 2023-11-28T18:28:53.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.402+0000 7fbc15ac9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc100ccbc0 con 0x7fbc10159f40 2023-11-28T18:28:53.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.402+0000 7fbc15ac9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1015a320 con 0x7fbc100ad850 2023-11-28T18:28:53.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.402+0000 7fbc0f7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbc10159f40 0x7fbc100caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:53.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.402+0000 7fbc0effd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fbc100cc760 0x7fbc100ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43694/0 (socket says 172.21.15.105:43694) 2023-11-28T18:28:53.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.402+0000 7fbc0effd640 1 -- 172.21.15.105:0/3419664762 learned_addr learned my addr 172.21.15.105:0/3419664762 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:53.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.402+0000 7fbc0ffff640 1 --2- 172.21.15.105:0/3419664762 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc100ad850 0x7fbc100701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:53.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.402+0000 7fbc0e7fc640 1 -- 172.21.15.105:0/3419664762 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083489873 0 0) 0x7fbc1006bf50 con 0x7fbc100cc760 2023-11-28T18:28:53.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.402+0000 7fbc0e7fc640 1 -- 172.21.15.105:0/3419664762 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbf8003530 con 0x7fbc100cc760 2023-11-28T18:28:53.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.402+0000 7fbc0f7fe640 1 -- 172.21.15.105:0/3419664762 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc100ad850 msgr2=0x7fbc100701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:53.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.402+0000 7fbc0f7fe640 1 --2- 172.21.15.105:0/3419664762 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc100ad850 0x7fbc100701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.402+0000 7fbc0f7fe640 1 -- 172.21.15.105:0/3419664762 >> v1:172.21.15.105:6789/0 conn(0x7fbc100cc760 legacy=0x7fbc100ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:53.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.402+0000 7fbc0f7fe640 1 -- 172.21.15.105:0/3419664762 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc10071380 con 0x7fbc10159f40 2023-11-28T18:28:53.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.406+0000 7fbc0f7fe640 1 --2- 172.21.15.105:0/3419664762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc10159f40 0x7fbc100caf60 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fbc000097c0 tx=0x7fbc0001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f11e447030246f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:53.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.406+0000 7fbc0e7fc640 1 -- 172.21.15.105:0/3419664762 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc000ac620 con 0x7fbc10159f40 2023-11-28T18:28:53.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.406+0000 7fbc0e7fc640 1 -- 172.21.15.105:0/3419664762 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc000ac780 con 0x7fbc10159f40 2023-11-28T18:28:53.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.406+0000 7fbc0e7fc640 1 -- 172.21.15.105:0/3419664762 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc000acb10 con 0x7fbc10159f40 2023-11-28T18:28:53.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.406+0000 7fbc15ac9640 1 -- 172.21.15.105:0/3419664762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc10159f40 msgr2=0x7fbc100caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:53.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.406+0000 7fbc15ac9640 1 --2- 172.21.15.105:0/3419664762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc10159f40 0x7fbc100caf60 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fbc000097c0 tx=0x7fbc0001c3c0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.406+0000 7fbc15ac9640 1 -- 172.21.15.105:0/3419664762 shutdown_connections 2023-11-28T18:28:53.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.406+0000 7fbc15ac9640 1 --2- 172.21.15.105:0/3419664762 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc100ad850 0x7fbc100701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.406+0000 7fbc15ac9640 1 --2- 172.21.15.105:0/3419664762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc10159f40 0x7fbc100caf60 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.406+0000 7fbc15ac9640 1 -- 172.21.15.105:0/3419664762 >> 172.21.15.105:0/3419664762 conn(0x7fbc100ceb90 msgr2=0x7fbc100cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:53.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.406+0000 7fbc15ac9640 1 -- 172.21.15.105:0/3419664762 shutdown_connections 2023-11-28T18:28:53.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.406+0000 7fbc15ac9640 1 -- 172.21.15.105:0/3419664762 wait complete. 2023-11-28T18:28:53.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.406+0000 7fbc15ac9640 1 Processor -- start 2023-11-28T18:28:53.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.406+0000 7fbc15ac9640 1 -- start start 2023-11-28T18:28:53.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbc15ac9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc100ad850 0x7fbc1014c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:53.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbc15ac9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc100cc760 0x7fbc100b2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:53.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbc15ac9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc10159f40 0x7fbc101640c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:53.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbc15ac9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc10154810 con 0x7fbc100cc760 2023-11-28T18:28:53.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbc15ac9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1014c620 con 0x7fbc10159f40 2023-11-28T18:28:53.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbc15ac9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc100b2e50 con 0x7fbc100ad850 2023-11-28T18:28:53.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbc0effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc100cc760 0x7fbc100b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:53.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbc0effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc100cc760 0x7fbc100b2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39184/0 (socket says 172.21.15.105:39184) 2023-11-28T18:28:53.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbc0effd640 1 -- 172.21.15.105:0/1624313011 learned_addr learned my addr 172.21.15.105:0/1624313011 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:53.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbc0f7fe640 1 --2- 172.21.15.105:0/1624313011 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc100ad850 0x7fbc1014c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:53.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbc0ffff640 1 --2- 172.21.15.105:0/1624313011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc10159f40 0x7fbc101640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:53.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbc0effd640 1 -- 172.21.15.105:0/1624313011 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc100ad850 msgr2=0x7fbc1014c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:53.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbc0effd640 1 --2- 172.21.15.105:0/1624313011 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc100ad850 0x7fbc1014c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbc0effd640 1 -- 172.21.15.105:0/1624313011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc10159f40 msgr2=0x7fbc101640c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:53.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbc0effd640 1 --2- 172.21.15.105:0/1624313011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc10159f40 0x7fbc101640c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbc0effd640 1 -- 172.21.15.105:0/1624313011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc00009420 con 0x7fbc100cc760 2023-11-28T18:28:53.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbc0f7fe640 1 --2- 172.21.15.105:0/1624313011 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc100ad850 0x7fbc1014c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:53.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbc0ffff640 1 --2- 172.21.15.105:0/1624313011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc10159f40 0x7fbc101640c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:53.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbc0effd640 1 --2- 172.21.15.105:0/1624313011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc100cc760 0x7fbc100b2470 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400c450 tx=0x7fbc0400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:53.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbbf7fff640 1 -- 172.21.15.105:0/1624313011 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc04005cf0 con 0x7fbc100cc760 2023-11-28T18:28:53.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbbf7fff640 1 -- 172.21.15.105:0/1624313011 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc04005e50 con 0x7fbc100cc760 2023-11-28T18:28:53.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbc15ac9640 1 -- 172.21.15.105:0/1624313011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc10154150 con 0x7fbc100cc760 2023-11-28T18:28:53.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbbf7fff640 1 -- 172.21.15.105:0/1624313011 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc04007270 con 0x7fbc100cc760 2023-11-28T18:28:53.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbc15ac9640 1 -- 172.21.15.105:0/1624313011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc100b3130 con 0x7fbc100cc760 2023-11-28T18:28:53.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbc15ac9640 1 -- 172.21.15.105:0/1624313011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc10154950 con 0x7fbc100cc760 2023-11-28T18:28:53.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.410+0000 7fbbf7fff640 1 -- 172.21.15.105:0/1624313011 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fbc040073d0 con 0x7fbc100cc760 2023-11-28T18:28:53.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.414+0000 7fbbf7fff640 1 --2- 172.21.15.105:0/1624313011 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbbfc0668d0 0x7fbbfc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:53.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.414+0000 7fbbf7fff640 1 -- 172.21.15.105:0/1624313011 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fbc040837a0 con 0x7fbc100cc760 2023-11-28T18:28:53.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.414+0000 7fbc0f7fe640 1 --2- 172.21.15.105:0/1624313011 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbbfc0668d0 0x7fbbfc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:53.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.414+0000 7fbc0f7fe640 1 --2- 172.21.15.105:0/1624313011 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbbfc0668d0 0x7fbbfc068d90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbc000ae000 tx=0x7fbc00008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:53.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.418+0000 7fbbf7fff640 1 -- 172.21.15.105:0/1624313011 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbc10154950 con 0x7fbc100cc760 2023-11-28T18:28:53.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.578+0000 7fbc15ac9640 1 -- 172.21.15.105:0/1624313011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"} v 0) v1 -- 0x7fbc10154950 con 0x7fbc100cc760 2023-11-28T18:28:53.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.582+0000 7fbbf7fff640 1 -- 172.21.15.105:0/1624313011 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"}]=0 v39) v1 ==== 110+0+0 (secure 0 0 0) 0x7fbc10154950 con 0x7fbc100cc760 2023-11-28T18:28:53.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.582+0000 7fbc15ac9640 1 -- 172.21.15.105:0/1624313011 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbbfc0668d0 msgr2=0x7fbbfc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:53.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.582+0000 7fbc15ac9640 1 --2- 172.21.15.105:0/1624313011 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbbfc0668d0 0x7fbbfc068d90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbc000ae000 tx=0x7fbc00008040 comp rx=0 tx=0).stop 2023-11-28T18:28:53.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.582+0000 7fbc15ac9640 1 -- 172.21.15.105:0/1624313011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc100cc760 msgr2=0x7fbc100b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:53.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.582+0000 7fbc15ac9640 1 --2- 172.21.15.105:0/1624313011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc100cc760 0x7fbc100b2470 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400c450 tx=0x7fbc0400c910 comp rx=0 tx=0).stop 2023-11-28T18:28:53.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.582+0000 7fbc15ac9640 1 -- 172.21.15.105:0/1624313011 shutdown_connections 2023-11-28T18:28:53.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.582+0000 7fbc15ac9640 1 --2- 172.21.15.105:0/1624313011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc10159f40 0x7fbc101640c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.582+0000 7fbc15ac9640 1 --2- 172.21.15.105:0/1624313011 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbbfc0668d0 0x7fbbfc068d90 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.582+0000 7fbc15ac9640 1 --2- 172.21.15.105:0/1624313011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc100cc760 0x7fbc100b2470 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.582+0000 7fbc15ac9640 1 --2- 172.21.15.105:0/1624313011 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc100ad850 0x7fbc1014c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.582+0000 7fbc15ac9640 1 -- 172.21.15.105:0/1624313011 >> 172.21.15.105:0/1624313011 conn(0x7fbc100ceb90 msgr2=0x7fbc1006d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:53.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.582+0000 7fbc15ac9640 1 -- 172.21.15.105:0/1624313011 shutdown_connections 2023-11-28T18:28:53.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.582+0000 7fbc15ac9640 1 -- 172.21.15.105:0/1624313011 wait complete. 2023-11-28T18:28:53.607 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T18:28:53.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.742+0000 7effde632640 1 Processor -- start 2023-11-28T18:28:53.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.742+0000 7effde632640 1 -- start start 2023-11-28T18:28:53.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.742+0000 7effde632640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7effd80b4290 0x7effd80b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:53.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.742+0000 7effde632640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7effd8071960 0x7effd815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:53.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.742+0000 7effde632640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effd80b4bb0 con 0x7effd815b030 2023-11-28T18:28:53.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.742+0000 7effde632640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd80701d0 con 0x7effd80b4290 2023-11-28T18:28:53.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.742+0000 7effde632640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd806df80 con 0x7effd8071960 2023-11-28T18:28:53.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.742+0000 7effdcbcf640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7effd815b030 0x7effd80709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43708/0 (socket says 172.21.15.105:43708) 2023-11-28T18:28:53.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.742+0000 7effdcbcf640 1 -- 172.21.15.105:0/1960545719 learned_addr learned my addr 172.21.15.105:0/1960545719 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:53.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.742+0000 7effd77fe640 1 --2- 172.21.15.105:0/1960545719 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7effd8071960 0x7effd815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:53.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.742+0000 7effd7fff640 1 --2- 172.21.15.105:0/1960545719 >> v2:172.21.15.148:3300/0 conn(0x7effd80b4290 0x7effd80b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:53.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.746+0000 7effd6ffd640 1 -- 172.21.15.105:0/1960545719 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070474834 0 0) 0x7effd80b4bb0 con 0x7effd815b030 2023-11-28T18:28:53.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.746+0000 7effd6ffd640 1 -- 172.21.15.105:0/1960545719 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effc0003530 con 0x7effd815b030 2023-11-28T18:28:53.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.746+0000 7effd77fe640 1 -- 172.21.15.105:0/1960545719 >> v2:172.21.15.148:3300/0 conn(0x7effd80b4290 msgr2=0x7effd80b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:53.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.746+0000 7effd77fe640 1 --2- 172.21.15.105:0/1960545719 >> v2:172.21.15.148:3300/0 conn(0x7effd80b4290 0x7effd80b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.746+0000 7effd77fe640 1 -- 172.21.15.105:0/1960545719 >> v1:172.21.15.105:6789/0 conn(0x7effd815b030 legacy=0x7effd80709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:53.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.746+0000 7effd77fe640 1 -- 172.21.15.105:0/1960545719 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd8071df0 con 0x7effd8071960 2023-11-28T18:28:53.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.746+0000 7effd77fe640 1 --2- 172.21.15.105:0/1960545719 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7effd8071960 0x7effd815f3b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7effcc0097c0 tx=0x7effcc01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe4c82a928fe587b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:53.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.746+0000 7effd6ffd640 1 -- 172.21.15.105:0/1960545719 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc0ab650 con 0x7effd8071960 2023-11-28T18:28:53.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.746+0000 7effd6ffd640 1 -- 172.21.15.105:0/1960545719 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effcc0ab7b0 con 0x7effd8071960 2023-11-28T18:28:53.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.746+0000 7effd6ffd640 1 -- 172.21.15.105:0/1960545719 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc0abb40 con 0x7effd8071960 2023-11-28T18:28:53.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effde632640 1 -- 172.21.15.105:0/1960545719 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7effd8071960 msgr2=0x7effd815f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:53.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effde632640 1 --2- 172.21.15.105:0/1960545719 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7effd8071960 0x7effd815f3b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7effcc0097c0 tx=0x7effcc01c450 comp rx=0 tx=0).stop 2023-11-28T18:28:53.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effde632640 1 -- 172.21.15.105:0/1960545719 shutdown_connections 2023-11-28T18:28:53.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effde632640 1 --2- 172.21.15.105:0/1960545719 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7effd8071960 0x7effd815f3b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effde632640 1 --2- 172.21.15.105:0/1960545719 >> v2:172.21.15.148:3300/0 conn(0x7effd80b4290 0x7effd80b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effde632640 1 -- 172.21.15.105:0/1960545719 >> 172.21.15.105:0/1960545719 conn(0x7effd80ceb90 msgr2=0x7effd80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:53.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effde632640 1 -- 172.21.15.105:0/1960545719 shutdown_connections 2023-11-28T18:28:53.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effde632640 1 -- 172.21.15.105:0/1960545719 wait complete. 2023-11-28T18:28:53.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effde632640 1 Processor -- start 2023-11-28T18:28:53.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effde632640 1 -- start start 2023-11-28T18:28:53.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effde632640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd80b4290 0x7effd81f5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:53.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effde632640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7effd815b030 0x7effd81f6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:53.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effde632640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effd8099a50 0x7effd809ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:53.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effde632640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd809ba10 con 0x7effd8099a50 2023-11-28T18:28:53.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effde632640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd81f66b0 con 0x7effd80b4290 2023-11-28T18:28:53.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effde632640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd809ab90 con 0x7effd815b030 2023-11-28T18:28:53.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effdcbcf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effd8099a50 0x7effd809ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:53.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effdcbcf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effd8099a50 0x7effd809ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39194/0 (socket says 172.21.15.105:39194) 2023-11-28T18:28:53.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effdcbcf640 1 -- 172.21.15.105:0/749489510 learned_addr learned my addr 172.21.15.105:0/749489510 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:53.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effd77fe640 1 --2- 172.21.15.105:0/749489510 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7effd815b030 0x7effd81f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:53.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effd7fff640 1 --2- 172.21.15.105:0/749489510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd80b4290 0x7effd81f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:53.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effdcbcf640 1 -- 172.21.15.105:0/749489510 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7effd815b030 msgr2=0x7effd81f6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:53.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effdcbcf640 1 --2- 172.21.15.105:0/749489510 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7effd815b030 0x7effd81f6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effdcbcf640 1 -- 172.21.15.105:0/749489510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd80b4290 msgr2=0x7effd81f5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:53.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effdcbcf640 1 --2- 172.21.15.105:0/749489510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd80b4290 0x7effd81f5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.750+0000 7effdcbcf640 1 -- 172.21.15.105:0/749489510 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effcc009420 con 0x7effd8099a50 2023-11-28T18:28:53.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.754+0000 7effdcbcf640 1 --2- 172.21.15.105:0/749489510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effd8099a50 0x7effd809ade0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7effd000ca70 tx=0x7effd000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:53.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.754+0000 7effd4ff9640 1 -- 172.21.15.105:0/749489510 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd000abd0 con 0x7effd8099a50 2023-11-28T18:28:53.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.754+0000 7effd4ff9640 1 -- 172.21.15.105:0/749489510 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd000ad30 con 0x7effd8099a50 2023-11-28T18:28:53.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.754+0000 7effd4ff9640 1 -- 172.21.15.105:0/749489510 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd0012600 con 0x7effd8099a50 2023-11-28T18:28:53.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.754+0000 7effde632640 1 -- 172.21.15.105:0/749489510 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd809a010 con 0x7effd8099a50 2023-11-28T18:28:53.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.754+0000 7effde632640 1 -- 172.21.15.105:0/749489510 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd809a5d0 con 0x7effd8099a50 2023-11-28T18:28:53.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.754+0000 7effde632640 1 -- 172.21.15.105:0/749489510 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd8067570 con 0x7effd8099a50 2023-11-28T18:28:53.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.754+0000 7effd4ff9640 1 -- 172.21.15.105:0/749489510 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7effd0007070 con 0x7effd8099a50 2023-11-28T18:28:53.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.754+0000 7effd4ff9640 1 --2- 172.21.15.105:0/749489510 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7effc40668d0 0x7effc4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:53.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.754+0000 7effd4ff9640 1 -- 172.21.15.105:0/749489510 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5435+0+0 (secure 0 0 0) 0x7effd0084190 con 0x7effd8099a50 2023-11-28T18:28:53.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.754+0000 7effd7fff640 1 --2- 172.21.15.105:0/749489510 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7effc40668d0 0x7effc4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:53.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.758+0000 7effd7fff640 1 --2- 172.21.15.105:0/749489510 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7effc40668d0 0x7effc4068d90 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7effc8009b60 tx=0x7effc8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:53.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.758+0000 7effd4ff9640 1 -- 172.21.15.105:0/749489510 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7effd0051580 con 0x7effd8099a50 2023-11-28T18:28:53.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.910+0000 7effde632640 1 -- 172.21.15.105:0/749489510 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7effd815b470 con 0x7effd8099a50 2023-11-28T18:28:53.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.914+0000 7effd4ff9640 1 -- 172.21.15.105:0/749489510 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v40) v1 ==== 104+0+0 (secure 0 0 0) 0x7effd0054ba0 con 0x7effd8099a50 2023-11-28T18:28:53.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.914+0000 7effde632640 1 -- 172.21.15.105:0/749489510 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7effc40668d0 msgr2=0x7effc4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:53.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.914+0000 7effde632640 1 --2- 172.21.15.105:0/749489510 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7effc40668d0 0x7effc4068d90 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7effc8009b60 tx=0x7effc8009190 comp rx=0 tx=0).stop 2023-11-28T18:28:53.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.914+0000 7effde632640 1 -- 172.21.15.105:0/749489510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effd8099a50 msgr2=0x7effd809ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:53.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.914+0000 7effde632640 1 --2- 172.21.15.105:0/749489510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effd8099a50 0x7effd809ade0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7effd000ca70 tx=0x7effd000cf30 comp rx=0 tx=0).stop 2023-11-28T18:28:53.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.914+0000 7effde632640 1 -- 172.21.15.105:0/749489510 shutdown_connections 2023-11-28T18:28:53.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.914+0000 7effde632640 1 --2- 172.21.15.105:0/749489510 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7effc40668d0 0x7effc4068d90 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.914+0000 7effde632640 1 --2- 172.21.15.105:0/749489510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effd8099a50 0x7effd809ade0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.914+0000 7effde632640 1 --2- 172.21.15.105:0/749489510 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7effd815b030 0x7effd81f6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.914+0000 7effde632640 1 --2- 172.21.15.105:0/749489510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd80b4290 0x7effd81f5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:53.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.914+0000 7effde632640 1 -- 172.21.15.105:0/749489510 >> 172.21.15.105:0/749489510 conn(0x7effd80ceb90 msgr2=0x7effd80cb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:53.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.914+0000 7effde632640 1 -- 172.21.15.105:0/749489510 shutdown_connections 2023-11-28T18:28:53.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:53.914+0000 7effde632640 1 -- 172.21.15.105:0/749489510 wait complete. 2023-11-28T18:28:53.939 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_distributed 2023-11-28T18:28:54.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.078+0000 7f7618f84640 1 Processor -- start 2023-11-28T18:28:54.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.078+0000 7f7618f84640 1 -- start start 2023-11-28T18:28:54.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.078+0000 7f7618f84640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f761415d350 0x7f76140d38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:54.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.078+0000 7f7618f84640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f76140cfd90 0x7f76140b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:54.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.078+0000 7f7618f84640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76140cdae0 con 0x7f761415d730 2023-11-28T18:28:54.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.078+0000 7f7618f84640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76140d1f40 con 0x7f76140cfd90 2023-11-28T18:28:54.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.078+0000 7f7618f84640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76140d16a0 con 0x7f761415d350 2023-11-28T18:28:54.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.082+0000 7f76137fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f761415d350 0x7f76140d38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:54.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.082+0000 7f7613fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f76140cfd90 0x7f76140b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:54.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.082+0000 7f7612ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f761415d730 0x7f76140cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:51352/0 (socket says 172.21.15.105:51352) 2023-11-28T18:28:54.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.082+0000 7f76137fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f761415d350 0x7f76140d38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:49890/0 (socket says 172.21.15.105:49890) 2023-11-28T18:28:54.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.082+0000 7f7612ffd640 1 -- 172.21.15.105:0/692395778 learned_addr learned my addr 172.21.15.105:0/692395778 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:54.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.082+0000 7f7613fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f76140cfd90 0x7f76140b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:56992/0 (socket says 172.21.15.105:56992) 2023-11-28T18:28:54.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.082+0000 7f76137fe640 1 -- 172.21.15.105:0/692395778 >> v2:172.21.15.148:3300/0 conn(0x7f76140cfd90 msgr2=0x7f76140b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:54.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.082+0000 7f76137fe640 1 --2- 172.21.15.105:0/692395778 >> v2:172.21.15.148:3300/0 conn(0x7f76140cfd90 0x7f76140b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.082+0000 7f76137fe640 1 -- 172.21.15.105:0/692395778 >> v1:172.21.15.148:6789/0 conn(0x7f761415d730 legacy=0x7f76140cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:54.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.082+0000 7f76137fe640 1 -- 172.21.15.105:0/692395778 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76140d1980 con 0x7f761415d350 2023-11-28T18:28:54.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.082+0000 7f76137fe640 1 --2- 172.21.15.105:0/692395778 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f761415d350 0x7f76140d38b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f76040097c0 tx=0x7f760401c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72026d40329d11b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:54.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.082+0000 7f76127fc640 1 -- 172.21.15.105:0/692395778 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76040ab600 con 0x7f761415d350 2023-11-28T18:28:54.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f76127fc640 1 -- 172.21.15.105:0/692395778 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76040a7050 con 0x7f761415d350 2023-11-28T18:28:54.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f76127fc640 1 -- 172.21.15.105:0/692395778 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76040ab980 con 0x7f761415d350 2023-11-28T18:28:54.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f7618f84640 1 -- 172.21.15.105:0/692395778 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f761415d350 msgr2=0x7f76140d38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:54.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f7618f84640 1 --2- 172.21.15.105:0/692395778 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f761415d350 0x7f76140d38b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f76040097c0 tx=0x7f760401c3f0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f7618f84640 1 -- 172.21.15.105:0/692395778 shutdown_connections 2023-11-28T18:28:54.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f7618f84640 1 --2- 172.21.15.105:0/692395778 >> v2:172.21.15.148:3300/0 conn(0x7f76140cfd90 0x7f76140b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f7618f84640 1 --2- 172.21.15.105:0/692395778 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f761415d350 0x7f76140d38b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f7618f84640 1 -- 172.21.15.105:0/692395778 >> 172.21.15.105:0/692395778 conn(0x7f76140d2d90 msgr2=0x7f76140d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:54.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f7618f84640 1 -- 172.21.15.105:0/692395778 shutdown_connections 2023-11-28T18:28:54.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f7618f84640 1 -- 172.21.15.105:0/692395778 wait complete. 2023-11-28T18:28:54.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f7618f84640 1 Processor -- start 2023-11-28T18:28:54.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f7618f84640 1 -- start start 2023-11-28T18:28:54.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f7618f84640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76140cfd90 0x7f76140cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:54.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f7618f84640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f761415d350 0x7f761406c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:54.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f7618f84640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f761415d730 0x7f76140b6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:54.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f7618f84640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76140cb9e0 con 0x7f761415d730 2023-11-28T18:28:54.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f7618f84640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76140b5f20 con 0x7f76140cfd90 2023-11-28T18:28:54.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f7618f84640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76140b6200 con 0x7f761415d350 2023-11-28T18:28:54.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f7613fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f761415d730 0x7f76140b6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:54.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f7613fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f761415d730 0x7f76140b6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39210/0 (socket says 172.21.15.105:39210) 2023-11-28T18:28:54.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f7613fff640 1 -- 172.21.15.105:0/786231308 learned_addr learned my addr 172.21.15.105:0/786231308 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:54.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f76137fe640 1 --2- 172.21.15.105:0/786231308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76140cfd90 0x7f76140cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:54.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.086+0000 7f7612ffd640 1 --2- 172.21.15.105:0/786231308 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f761415d350 0x7f761406c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:54.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.090+0000 7f7613fff640 1 -- 172.21.15.105:0/786231308 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f761415d350 msgr2=0x7f761406c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:54.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.090+0000 7f7613fff640 1 --2- 172.21.15.105:0/786231308 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f761415d350 0x7f761406c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.090+0000 7f7613fff640 1 -- 172.21.15.105:0/786231308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76140cfd90 msgr2=0x7f76140cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:54.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.090+0000 7f7613fff640 1 --2- 172.21.15.105:0/786231308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76140cfd90 0x7f76140cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.090+0000 7f7613fff640 1 -- 172.21.15.105:0/786231308 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7604009420 con 0x7f761415d730 2023-11-28T18:28:54.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.090+0000 7f7613fff640 1 --2- 172.21.15.105:0/786231308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f761415d730 0x7f76140b6fc0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f760c00efc0 tx=0x7f760c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:54.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.090+0000 7f75fbfff640 1 -- 172.21.15.105:0/786231308 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c014480 con 0x7f761415d730 2023-11-28T18:28:54.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.090+0000 7f75fbfff640 1 -- 172.21.15.105:0/786231308 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760c00f040 con 0x7f761415d730 2023-11-28T18:28:54.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.090+0000 7f75fbfff640 1 -- 172.21.15.105:0/786231308 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c014820 con 0x7f761415d730 2023-11-28T18:28:54.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.090+0000 7f7618f84640 1 -- 172.21.15.105:0/786231308 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76140b64e0 con 0x7f761415d730 2023-11-28T18:28:54.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.090+0000 7f7618f84640 1 -- 172.21.15.105:0/786231308 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76140b6aa0 con 0x7f761415d730 2023-11-28T18:28:54.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.090+0000 7f7618f84640 1 -- 172.21.15.105:0/786231308 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76140b6c90 con 0x7f761415d730 2023-11-28T18:28:54.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.090+0000 7f75fbfff640 1 -- 172.21.15.105:0/786231308 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f760c014980 con 0x7f761415d730 2023-11-28T18:28:54.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.094+0000 7f75fbfff640 1 --2- 172.21.15.105:0/786231308 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7600066900 0x7f7600068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:54.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.094+0000 7f75fbfff640 1 -- 172.21.15.105:0/786231308 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f760c085180 con 0x7f761415d730 2023-11-28T18:28:54.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.094+0000 7f76137fe640 1 --2- 172.21.15.105:0/786231308 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7600066900 0x7f7600068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:54.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.094+0000 7f76137fe640 1 --2- 172.21.15.105:0/786231308 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7600066900 0x7f7600068dc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f76040ad000 tx=0x7f7604008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:54.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.098+0000 7f75fbfff640 1 -- 172.21.15.105:0/786231308 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f760c052450 con 0x7f761415d730 2023-11-28T18:28:54.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.250+0000 7f7618f84640 1 -- 172.21.15.105:0/786231308 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"} v 0) v1 -- 0x7f7614074580 con 0x7f761415d730 2023-11-28T18:28:54.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.254+0000 7f75fbfff640 1 -- 172.21.15.105:0/786231308 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"}]=0 v41) v1 ==== 115+0+0 (secure 0 0 0) 0x7f760c0180b0 con 0x7f761415d730 2023-11-28T18:28:54.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.258+0000 7f7618f84640 1 -- 172.21.15.105:0/786231308 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7600066900 msgr2=0x7f7600068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:54.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.258+0000 7f7618f84640 1 --2- 172.21.15.105:0/786231308 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7600066900 0x7f7600068dc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f76040ad000 tx=0x7f7604008040 comp rx=0 tx=0).stop 2023-11-28T18:28:54.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.258+0000 7f7618f84640 1 -- 172.21.15.105:0/786231308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f761415d730 msgr2=0x7f76140b6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:54.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.258+0000 7f7618f84640 1 --2- 172.21.15.105:0/786231308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f761415d730 0x7f76140b6fc0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f760c00efc0 tx=0x7f760c00c490 comp rx=0 tx=0).stop 2023-11-28T18:28:54.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.258+0000 7f7618f84640 1 -- 172.21.15.105:0/786231308 shutdown_connections 2023-11-28T18:28:54.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.258+0000 7f7618f84640 1 --2- 172.21.15.105:0/786231308 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7600066900 0x7f7600068dc0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.258+0000 7f7618f84640 1 --2- 172.21.15.105:0/786231308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f761415d730 0x7f76140b6fc0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.258+0000 7f7618f84640 1 --2- 172.21.15.105:0/786231308 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f761415d350 0x7f761406c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.258+0000 7f7618f84640 1 --2- 172.21.15.105:0/786231308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76140cfd90 0x7f76140cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.258+0000 7f7618f84640 1 -- 172.21.15.105:0/786231308 >> 172.21.15.105:0/786231308 conn(0x7f76140d2d90 msgr2=0x7f76140b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:54.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.258+0000 7f7618f84640 1 -- 172.21.15.105:0/786231308 shutdown_connections 2023-11-28T18:28:54.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.258+0000 7f7618f84640 1 -- 172.21.15.105:0/786231308 wait complete. 2023-11-28T18:28:54.282 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random_max 2023-11-28T18:28:54.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.418+0000 7f0256832640 1 Processor -- start 2023-11-28T18:28:54.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.418+0000 7f0256832640 1 -- start start 2023-11-28T18:28:54.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.418+0000 7f0256832640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f02500b4290 0x7f02500b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:54.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.418+0000 7f0256832640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f025015b030 0x7f02500709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:54.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.418+0000 7f0256832640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02500b4bb0 con 0x7f0250071960 2023-11-28T18:28:54.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.418+0000 7f0256832640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02500701d0 con 0x7f02500b4290 2023-11-28T18:28:54.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.418+0000 7f0256832640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025006df80 con 0x7f025015b030 2023-11-28T18:28:54.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.418+0000 7f024ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f02500b4290 0x7f02500b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:54.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.418+0000 7f024ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f02500b4290 0x7f02500b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39224/0 (socket says 172.21.15.105:39224) 2023-11-28T18:28:54.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.418+0000 7f024ffff640 1 -- 172.21.15.105:0/525959183 learned_addr learned my addr 172.21.15.105:0/525959183 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:54.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.418+0000 7f0254dcf640 1 --2- 172.21.15.105:0/525959183 >> v2:172.21.15.148:3300/0 conn(0x7f025015b030 0x7f02500709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:54.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.418+0000 7f024ffff640 1 -- 172.21.15.105:0/525959183 >> v2:172.21.15.148:3300/0 conn(0x7f025015b030 msgr2=0x7f02500709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:54.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.418+0000 7f024ffff640 1 --2- 172.21.15.105:0/525959183 >> v2:172.21.15.148:3300/0 conn(0x7f025015b030 0x7f02500709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.418+0000 7f024ffff640 1 -- 172.21.15.105:0/525959183 >> v1:172.21.15.148:6789/0 conn(0x7f0250071960 legacy=0x7f025015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:54.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.418+0000 7f024ffff640 1 -- 172.21.15.105:0/525959183 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0250071df0 con 0x7f02500b4290 2023-11-28T18:28:54.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.418+0000 7f0254dcf640 1 --2- 172.21.15.105:0/525959183 >> v2:172.21.15.148:3300/0 conn(0x7f025015b030 0x7f02500709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:28:54.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.422+0000 7f024ffff640 1 --2- 172.21.15.105:0/525959183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02500b4290 0x7f02500b4670 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f025006e370 tx=0x7f024001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93becdfed8936bce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:54.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.422+0000 7f024effd640 1 -- 172.21.15.105:0/525959183 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02400ac600 con 0x7f02500b4290 2023-11-28T18:28:54.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.422+0000 7f024effd640 1 -- 172.21.15.105:0/525959183 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02400a8050 con 0x7f02500b4290 2023-11-28T18:28:54.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.422+0000 7f024effd640 1 -- 172.21.15.105:0/525959183 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02400ac980 con 0x7f02500b4290 2023-11-28T18:28:54.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.422+0000 7f0256832640 1 -- 172.21.15.105:0/525959183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02500b4290 msgr2=0x7f02500b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:54.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.422+0000 7f0256832640 1 --2- 172.21.15.105:0/525959183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02500b4290 0x7f02500b4670 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f025006e370 tx=0x7f024001c490 comp rx=0 tx=0).stop 2023-11-28T18:28:54.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.422+0000 7f0256832640 1 -- 172.21.15.105:0/525959183 shutdown_connections 2023-11-28T18:28:54.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.422+0000 7f0256832640 1 --2- 172.21.15.105:0/525959183 >> v2:172.21.15.148:3300/0 conn(0x7f025015b030 0x7f02500709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.422+0000 7f0256832640 1 --2- 172.21.15.105:0/525959183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02500b4290 0x7f02500b4670 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.422+0000 7f0256832640 1 -- 172.21.15.105:0/525959183 >> 172.21.15.105:0/525959183 conn(0x7f02500ceb90 msgr2=0x7f02500cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:54.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.422+0000 7f0256832640 1 -- 172.21.15.105:0/525959183 shutdown_connections 2023-11-28T18:28:54.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.422+0000 7f0256832640 1 -- 172.21.15.105:0/525959183 wait complete. 2023-11-28T18:28:54.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f0256832640 1 Processor -- start 2023-11-28T18:28:54.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f0256832640 1 -- start start 2023-11-28T18:28:54.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f0256832640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02500b4290 0x7f02500cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:54.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f0256832640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f025015b030 0x7f0250149f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:54.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f0256832640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02501417c0 0x7f02500ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:54.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f0256832640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025014a480 con 0x7f02501417c0 2023-11-28T18:28:54.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f0256832640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025014c0c0 con 0x7f02500b4290 2023-11-28T18:28:54.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f0256832640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025014a6d0 con 0x7f025015b030 2023-11-28T18:28:54.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f0254dcf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02501417c0 0x7f02500ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:54.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f0254dcf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02501417c0 0x7f02500ad4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39240/0 (socket says 172.21.15.105:39240) 2023-11-28T18:28:54.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f0254dcf640 1 -- 172.21.15.105:0/234681867 learned_addr learned my addr 172.21.15.105:0/234681867 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:54.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f024ffff640 1 --2- 172.21.15.105:0/234681867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02500b4290 0x7f02500cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:54.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f024f7fe640 1 --2- 172.21.15.105:0/234681867 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f025015b030 0x7f0250149f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:54.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f0254dcf640 1 -- 172.21.15.105:0/234681867 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f025015b030 msgr2=0x7f0250149f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:54.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f0254dcf640 1 --2- 172.21.15.105:0/234681867 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f025015b030 0x7f0250149f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f0254dcf640 1 -- 172.21.15.105:0/234681867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02500b4290 msgr2=0x7f02500cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:54.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f0254dcf640 1 --2- 172.21.15.105:0/234681867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02500b4290 0x7f02500cdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f0254dcf640 1 -- 172.21.15.105:0/234681867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02400094d0 con 0x7f02501417c0 2023-11-28T18:28:54.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f0254dcf640 1 --2- 172.21.15.105:0/234681867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02501417c0 0x7f02500ad4b0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f024800ccc0 tx=0x7f024800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:54.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f024cff9640 1 -- 172.21.15.105:0/234681867 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024800f680 con 0x7f02501417c0 2023-11-28T18:28:54.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f024cff9640 1 -- 172.21.15.105:0/234681867 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024800f800 con 0x7f02501417c0 2023-11-28T18:28:54.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f024cff9640 1 -- 172.21.15.105:0/234681867 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024800fbb0 con 0x7f02501417c0 2023-11-28T18:28:54.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f0256832640 1 -- 172.21.15.105:0/234681867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f025014a9b0 con 0x7f02501417c0 2023-11-28T18:28:54.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.426+0000 7f0256832640 1 -- 172.21.15.105:0/234681867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025014ac90 con 0x7f02501417c0 2023-11-28T18:28:54.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.430+0000 7f0256832640 1 -- 172.21.15.105:0/234681867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025014c5a0 con 0x7f02501417c0 2023-11-28T18:28:54.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.430+0000 7f024cff9640 1 -- 172.21.15.105:0/234681867 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f024800b040 con 0x7f02501417c0 2023-11-28T18:28:54.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.430+0000 7f024cff9640 1 --2- 172.21.15.105:0/234681867 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f023c0668d0 0x7f023c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:54.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.430+0000 7f024cff9640 1 -- 172.21.15.105:0/234681867 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f0248084c30 con 0x7f02501417c0 2023-11-28T18:28:54.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.430+0000 7f024ffff640 1 --2- 172.21.15.105:0/234681867 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f023c0668d0 0x7f023c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:54.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.430+0000 7f024ffff640 1 --2- 172.21.15.105:0/234681867 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f023c0668d0 0x7f023c068d90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f02400ae000 tx=0x7f0240008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:54.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.434+0000 7f024cff9640 1 -- 172.21.15.105:0/234681867 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f025014c5a0 con 0x7f02501417c0 2023-11-28T18:28:54.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.586+0000 7f0256832640 1 -- 172.21.15.105:0/234681867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"} v 0) v1 -- 0x7f025014c5a0 con 0x7f02501417c0 2023-11-28T18:28:54.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.590+0000 7f024cff9640 1 -- 172.21.15.105:0/234681867 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"}]=0 v42) v1 ==== 114+0+0 (secure 0 0 0) 0x7f025014c5a0 con 0x7f02501417c0 2023-11-28T18:28:54.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.594+0000 7f0256832640 1 -- 172.21.15.105:0/234681867 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f023c0668d0 msgr2=0x7f023c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:54.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.594+0000 7f0256832640 1 --2- 172.21.15.105:0/234681867 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f023c0668d0 0x7f023c068d90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f02400ae000 tx=0x7f0240008040 comp rx=0 tx=0).stop 2023-11-28T18:28:54.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.594+0000 7f0256832640 1 -- 172.21.15.105:0/234681867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02501417c0 msgr2=0x7f02500ad4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:54.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.594+0000 7f0256832640 1 --2- 172.21.15.105:0/234681867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02501417c0 0x7f02500ad4b0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f024800ccc0 tx=0x7f024800a650 comp rx=0 tx=0).stop 2023-11-28T18:28:54.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.594+0000 7f0256832640 1 -- 172.21.15.105:0/234681867 shutdown_connections 2023-11-28T18:28:54.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.594+0000 7f0256832640 1 --2- 172.21.15.105:0/234681867 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f023c0668d0 0x7f023c068d90 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.594+0000 7f0256832640 1 --2- 172.21.15.105:0/234681867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02501417c0 0x7f02500ad4b0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.594+0000 7f0256832640 1 --2- 172.21.15.105:0/234681867 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f025015b030 0x7f0250149f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.594+0000 7f0256832640 1 --2- 172.21.15.105:0/234681867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02500b4290 0x7f02500cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.594+0000 7f0256832640 1 -- 172.21.15.105:0/234681867 >> 172.21.15.105:0/234681867 conn(0x7f02500ceb90 msgr2=0x7f0250099cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:54.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.594+0000 7f0256832640 1 -- 172.21.15.105:0/234681867 shutdown_connections 2023-11-28T18:28:54.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.594+0000 7f0256832640 1 -- 172.21.15.105:0/234681867 wait complete. 2023-11-28T18:28:54.616 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override' 2023-11-28T18:28:54.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.750+0000 7f8f564d6640 1 Processor -- start 2023-11-28T18:28:54.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.750+0000 7f8f564d6640 1 -- start start 2023-11-28T18:28:54.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.750+0000 7f8f564d6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8f5015d4f0 0x7f8f500d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:54.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.750+0000 7f8f564d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f500b2d20 0x7f8f500b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:54.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.750+0000 7f8f564d6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f500b25b0 con 0x7f8f500d3c30 2023-11-28T18:28:54.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.750+0000 7f8f564d6640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f500b2900 con 0x7f8f5015d4f0 2023-11-28T18:28:54.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.750+0000 7f8f564d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f500d0a20 con 0x7f8f500b2d20 2023-11-28T18:28:54.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.750+0000 7f8f4ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8f5015d4f0 0x7f8f500d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:54.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.750+0000 7f8f4f7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8f500d3c30 0x7f8f500d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43710/0 (socket says 172.21.15.105:43710) 2023-11-28T18:28:54.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.750+0000 7f8f4f7fe640 1 -- 172.21.15.105:0/1277118227 learned_addr learned my addr 172.21.15.105:0/1277118227 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:54.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.750+0000 7f8f54a73640 1 --2- 172.21.15.105:0/1277118227 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f500b2d20 0x7f8f500b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:54.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.750+0000 7f8f4ffff640 1 -- 172.21.15.105:0/1277118227 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f500b2d20 msgr2=0x7f8f500b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:54.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.750+0000 7f8f4ffff640 1 --2- 172.21.15.105:0/1277118227 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f500b2d20 0x7f8f500b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.750+0000 7f8f4ffff640 1 -- 172.21.15.105:0/1277118227 >> v1:172.21.15.105:6789/0 conn(0x7f8f500d3c30 legacy=0x7f8f500d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:54.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.750+0000 7f8f4ffff640 1 -- 172.21.15.105:0/1277118227 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f500cea30 con 0x7f8f5015d4f0 2023-11-28T18:28:54.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.750+0000 7f8f4effd640 1 -- 172.21.15.105:0/1277118227 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3343442101 0 0) 0x7f8f500b25b0 con 0x7f8f500d3c30 2023-11-28T18:28:54.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.754+0000 7f8f4ffff640 1 --2- 172.21.15.105:0/1277118227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f5015d4f0 0x7f8f500d0e10 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f8f400097c0 tx=0x7f8f4001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1faf6d7fedc261f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:54.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.754+0000 7f8f4effd640 1 -- 172.21.15.105:0/1277118227 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f400ac620 con 0x7f8f5015d4f0 2023-11-28T18:28:54.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.754+0000 7f8f4effd640 1 -- 172.21.15.105:0/1277118227 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f400ac780 con 0x7f8f5015d4f0 2023-11-28T18:28:54.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.754+0000 7f8f4effd640 1 -- 172.21.15.105:0/1277118227 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f400acb10 con 0x7f8f5015d4f0 2023-11-28T18:28:54.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.754+0000 7f8f564d6640 1 -- 172.21.15.105:0/1277118227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f5015d4f0 msgr2=0x7f8f500d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:54.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.754+0000 7f8f564d6640 1 --2- 172.21.15.105:0/1277118227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f5015d4f0 0x7f8f500d0e10 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f8f400097c0 tx=0x7f8f4001c3c0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.754+0000 7f8f564d6640 1 -- 172.21.15.105:0/1277118227 shutdown_connections 2023-11-28T18:28:54.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.754+0000 7f8f564d6640 1 --2- 172.21.15.105:0/1277118227 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f500b2d20 0x7f8f500b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.754+0000 7f8f564d6640 1 --2- 172.21.15.105:0/1277118227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f5015d4f0 0x7f8f500d0e10 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.754+0000 7f8f564d6640 1 -- 172.21.15.105:0/1277118227 >> 172.21.15.105:0/1277118227 conn(0x7f8f500d3030 msgr2=0x7f8f500d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:54.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.754+0000 7f8f564d6640 1 -- 172.21.15.105:0/1277118227 shutdown_connections 2023-11-28T18:28:54.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.754+0000 7f8f564d6640 1 -- 172.21.15.105:0/1277118227 wait complete. 2023-11-28T18:28:54.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f564d6640 1 Processor -- start 2023-11-28T18:28:54.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f564d6640 1 -- start start 2023-11-28T18:28:54.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f564d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f500b2d20 0x7f8f500c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:54.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f564d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f500d3c30 0x7f8f50155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:54.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f564d6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f5015d4f0 0x7f8f501559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:54.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f564d6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f50157240 con 0x7f8f5015d4f0 2023-11-28T18:28:54.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f564d6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f50157520 con 0x7f8f500d3c30 2023-11-28T18:28:54.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f564d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f50157800 con 0x7f8f500b2d20 2023-11-28T18:28:54.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f54a73640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f5015d4f0 0x7f8f501559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:54.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f54a73640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f5015d4f0 0x7f8f501559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39258/0 (socket says 172.21.15.105:39258) 2023-11-28T18:28:54.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f54a73640 1 -- 172.21.15.105:0/105690377 learned_addr learned my addr 172.21.15.105:0/105690377 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:54.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f4ffff640 1 --2- 172.21.15.105:0/105690377 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f500b2d20 0x7f8f500c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:54.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f4f7fe640 1 --2- 172.21.15.105:0/105690377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f500d3c30 0x7f8f50155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:54.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f54a73640 1 -- 172.21.15.105:0/105690377 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f500b2d20 msgr2=0x7f8f500c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:54.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f54a73640 1 --2- 172.21.15.105:0/105690377 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f500b2d20 0x7f8f500c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f54a73640 1 -- 172.21.15.105:0/105690377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f500d3c30 msgr2=0x7f8f50155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:54.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f54a73640 1 --2- 172.21.15.105:0/105690377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f500d3c30 0x7f8f50155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:54.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f54a73640 1 -- 172.21.15.105:0/105690377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f40009420 con 0x7f8f5015d4f0 2023-11-28T18:28:54.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f4ffff640 1 --2- 172.21.15.105:0/105690377 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f500b2d20 0x7f8f500c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:54.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f54a73640 1 --2- 172.21.15.105:0/105690377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f5015d4f0 0x7f8f501559b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f8f48007e60 tx=0x7f8f4800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:54.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f4cff9640 1 -- 172.21.15.105:0/105690377 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48014420 con 0x7f8f5015d4f0 2023-11-28T18:28:54.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f4cff9640 1 -- 172.21.15.105:0/105690377 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f480145a0 con 0x7f8f5015d4f0 2023-11-28T18:28:54.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f4cff9640 1 -- 172.21.15.105:0/105690377 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48014950 con 0x7f8f5015d4f0 2023-11-28T18:28:54.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f564d6640 1 -- 172.21.15.105:0/105690377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f50157ae0 con 0x7f8f5015d4f0 2023-11-28T18:28:54.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f564d6640 1 -- 172.21.15.105:0/105690377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f500c8fb0 con 0x7f8f5015d4f0 2023-11-28T18:28:54.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.758+0000 7f8f564d6640 1 -- 172.21.15.105:0/105690377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f50067570 con 0x7f8f5015d4f0 2023-11-28T18:28:54.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.762+0000 7f8f4cff9640 1 -- 172.21.15.105:0/105690377 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8f48014ab0 con 0x7f8f5015d4f0 2023-11-28T18:28:54.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.762+0000 7f8f4cff9640 1 --2- 172.21.15.105:0/105690377 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8f3c0668d0 0x7f8f3c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:54.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.762+0000 7f8f4cff9640 1 -- 172.21.15.105:0/105690377 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f8f48084c50 con 0x7f8f5015d4f0 2023-11-28T18:28:54.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.762+0000 7f8f4ffff640 1 --2- 172.21.15.105:0/105690377 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8f3c0668d0 0x7f8f3c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:54.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.762+0000 7f8f4ffff640 1 --2- 172.21.15.105:0/105690377 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8f3c0668d0 0x7f8f3c068d90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8f400ae000 tx=0x7f8f40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:54.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.766+0000 7f8f4cff9640 1 -- 172.21.15.105:0/105690377 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8f4808a020 con 0x7f8f5015d4f0 2023-11-28T18:28:54.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:54.970+0000 7f8f564d6640 1 -- 172.21.15.105:0/105690377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override"]} v 0) v1 -- 0x7f8f50157f90 con 0x7f8f5015d4f0 2023-11-28T18:28:55.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.238+0000 7f8f4cff9640 1 -- 172.21.15.105:0/105690377 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override"]}]=0 v291) v1 ==== 155+0+0 (secure 0 0 0) 0x7f8f48051fa0 con 0x7f8f5015d4f0 2023-11-28T18:28:55.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.242+0000 7f8f564d6640 1 -- 172.21.15.105:0/105690377 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8f3c0668d0 msgr2=0x7f8f3c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:55.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.242+0000 7f8f564d6640 1 --2- 172.21.15.105:0/105690377 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8f3c0668d0 0x7f8f3c068d90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8f400ae000 tx=0x7f8f40008040 comp rx=0 tx=0).stop 2023-11-28T18:28:55.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.242+0000 7f8f564d6640 1 -- 172.21.15.105:0/105690377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f5015d4f0 msgr2=0x7f8f501559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:55.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.242+0000 7f8f564d6640 1 --2- 172.21.15.105:0/105690377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f5015d4f0 0x7f8f501559b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f8f48007e60 tx=0x7f8f4800e4b0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.242+0000 7f8f564d6640 1 -- 172.21.15.105:0/105690377 shutdown_connections 2023-11-28T18:28:55.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.242+0000 7f8f564d6640 1 --2- 172.21.15.105:0/105690377 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8f3c0668d0 0x7f8f3c068d90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.242+0000 7f8f564d6640 1 --2- 172.21.15.105:0/105690377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f5015d4f0 0x7f8f501559b0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.242+0000 7f8f564d6640 1 --2- 172.21.15.105:0/105690377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f500d3c30 0x7f8f50155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.242+0000 7f8f564d6640 1 --2- 172.21.15.105:0/105690377 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f500b2d20 0x7f8f500c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.242+0000 7f8f564d6640 1 -- 172.21.15.105:0/105690377 >> 172.21.15.105:0/105690377 conn(0x7f8f500d3030 msgr2=0x7f8f500c98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:55.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.242+0000 7f8f564d6640 1 -- 172.21.15.105:0/105690377 shutdown_connections 2023-11-28T18:28:55.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.242+0000 7f8f564d6640 1 -- 172.21.15.105:0/105690377 wait complete. 2023-11-28T18:28:55.266 INFO:tasks.cephfs_test_runner:That an ephemeral distributed pin overrides a normal export pin. ... ok 2023-11-28T18:28:55.267 INFO:tasks.cephfs_test_runner:Starting test: test_ephemeral_pin_dist_override_after (tasks.cephfs.test_exports.TestEphemeralPins) That a conventional export pin overrides the distributed policy _after_ distributed policy is set. 2023-11-28T18:28:55.267 INFO:tasks.cephfs_test_runner:test_ephemeral_pin_dist_override_after (tasks.cephfs.test_exports.TestEphemeralPins) 2023-11-28T18:28:55.267 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:28:55.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.402+0000 7f5706cad640 1 Processor -- start 2023-11-28T18:28:55.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.402+0000 7f5706cad640 1 -- start start 2023-11-28T18:28:55.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.402+0000 7f5706cad640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f570015cef0 0x7f57000d1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:55.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.402+0000 7f5706cad640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f57000d1b40 0x7f57000cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:55.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.402+0000 7f5706cad640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57000ce840 con 0x7f570015d2d0 2023-11-28T18:28:55.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.402+0000 7f5706cad640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57000b4780 con 0x7f570015cef0 2023-11-28T18:28:55.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.402+0000 7f5706cad640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57000cd990 con 0x7f57000d1b40 2023-11-28T18:28:55.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.402+0000 7f5705cab640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f570015cef0 0x7f57000d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:55.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.402+0000 7f5705cab640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f570015cef0 0x7f57000d1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39266/0 (socket says 172.21.15.105:39266) 2023-11-28T18:28:55.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.402+0000 7f5705cab640 1 -- 172.21.15.105:0/1685748501 learned_addr learned my addr 172.21.15.105:0/1685748501 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:55.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.402+0000 7f57054aa640 1 --2- 172.21.15.105:0/1685748501 >> v2:172.21.15.148:3300/0 conn(0x7f57000d1b40 0x7f57000cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:55.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.402+0000 7f5705cab640 1 -- 172.21.15.105:0/1685748501 >> v2:172.21.15.148:3300/0 conn(0x7f57000d1b40 msgr2=0x7f57000cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:55.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.402+0000 7f5705cab640 1 --2- 172.21.15.105:0/1685748501 >> v2:172.21.15.148:3300/0 conn(0x7f57000d1b40 0x7f57000cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.402+0000 7f5705cab640 1 -- 172.21.15.105:0/1685748501 >> v1:172.21.15.148:6789/0 conn(0x7f570015d2d0 legacy=0x7f5700160c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:55.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.402+0000 7f5705cab640 1 -- 172.21.15.105:0/1685748501 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57000cecd0 con 0x7f570015cef0 2023-11-28T18:28:55.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.406+0000 7f5705cab640 1 --2- 172.21.15.105:0/1685748501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f570015cef0 0x7f57000d1600 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f570008bac0 tx=0x7f56fc01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d32a80ffcdb508 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:55.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.406+0000 7f5704ca9640 1 -- 172.21.15.105:0/1685748501 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc0ac640 con 0x7f570015cef0 2023-11-28T18:28:55.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.406+0000 7f5704ca9640 1 -- 172.21.15.105:0/1685748501 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56fc0ac7a0 con 0x7f570015cef0 2023-11-28T18:28:55.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.406+0000 7f5704ca9640 1 -- 172.21.15.105:0/1685748501 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc0acb30 con 0x7f570015cef0 2023-11-28T18:28:55.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.406+0000 7f5706cad640 1 -- 172.21.15.105:0/1685748501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f570015cef0 msgr2=0x7f57000d1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:55.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.406+0000 7f5706cad640 1 --2- 172.21.15.105:0/1685748501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f570015cef0 0x7f57000d1600 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f570008bac0 tx=0x7f56fc01c400 comp rx=0 tx=0).stop 2023-11-28T18:28:55.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.406+0000 7f5706cad640 1 -- 172.21.15.105:0/1685748501 shutdown_connections 2023-11-28T18:28:55.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.406+0000 7f5706cad640 1 --2- 172.21.15.105:0/1685748501 >> v2:172.21.15.148:3300/0 conn(0x7f57000d1b40 0x7f57000cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.406+0000 7f5706cad640 1 --2- 172.21.15.105:0/1685748501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f570015cef0 0x7f57000d1600 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.406+0000 7f5706cad640 1 -- 172.21.15.105:0/1685748501 >> 172.21.15.105:0/1685748501 conn(0x7f57000d2e10 msgr2=0x7f57000d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:55.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.406+0000 7f5706cad640 1 -- 172.21.15.105:0/1685748501 shutdown_connections 2023-11-28T18:28:55.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.406+0000 7f5706cad640 1 -- 172.21.15.105:0/1685748501 wait complete. 2023-11-28T18:28:55.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.406+0000 7f5706cad640 1 Processor -- start 2023-11-28T18:28:55.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.406+0000 7f5706cad640 1 -- start start 2023-11-28T18:28:55.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f5706cad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57000d1b40 0x7f570015a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:55.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f5706cad640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f570015cef0 0x7f570009ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:55.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f5706cad640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f570015d2d0 0x7f570015e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:55.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f5706cad640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570015f710 con 0x7f570015d2d0 2023-11-28T18:28:55.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f5706cad640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570015a830 con 0x7f57000d1b40 2023-11-28T18:28:55.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f5706cad640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570015ea90 con 0x7f570015cef0 2023-11-28T18:28:55.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f57064ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f570015d2d0 0x7f570015e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:55.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f5705cab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57000d1b40 0x7f570015a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:55.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f57064ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f570015d2d0 0x7f570015e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39280/0 (socket says 172.21.15.105:39280) 2023-11-28T18:28:55.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f57064ac640 1 -- 172.21.15.105:0/1299368612 learned_addr learned my addr 172.21.15.105:0/1299368612 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:55.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f5705cab640 1 --2- 172.21.15.105:0/1299368612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57000d1b40 0x7f570015a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:57046/0 (socket says 172.21.15.105:57046) 2023-11-28T18:28:55.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f57054aa640 1 --2- 172.21.15.105:0/1299368612 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f570015cef0 0x7f570009ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:55.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f57064ac640 1 -- 172.21.15.105:0/1299368612 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f570015cef0 msgr2=0x7f570009ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:55.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f57064ac640 1 --2- 172.21.15.105:0/1299368612 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f570015cef0 0x7f570009ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f57064ac640 1 -- 172.21.15.105:0/1299368612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57000d1b40 msgr2=0x7f570015a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:55.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f57064ac640 1 --2- 172.21.15.105:0/1299368612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57000d1b40 0x7f570015a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f57064ac640 1 -- 172.21.15.105:0/1299368612 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56fc0094d0 con 0x7f570015d2d0 2023-11-28T18:28:55.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f57064ac640 1 --2- 172.21.15.105:0/1299368612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f570015d2d0 0x7f570015e190 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f56f800b9a0 tx=0x7f56f800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:55.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f56f67fc640 1 -- 172.21.15.105:0/1299368612 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f800cba0 con 0x7f570015d2d0 2023-11-28T18:28:55.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f56f67fc640 1 -- 172.21.15.105:0/1299368612 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f800cd00 con 0x7f570015d2d0 2023-11-28T18:28:55.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f5706cad640 1 -- 172.21.15.105:0/1299368612 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f570015ed70 con 0x7f570015d2d0 2023-11-28T18:28:55.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f56f67fc640 1 -- 172.21.15.105:0/1299368612 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f8014680 con 0x7f570015d2d0 2023-11-28T18:28:55.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f5706cad640 1 -- 172.21.15.105:0/1299368612 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f570015f330 con 0x7f570015d2d0 2023-11-28T18:28:55.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.410+0000 7f5706cad640 1 -- 172.21.15.105:0/1299368612 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f570015f850 con 0x7f570015d2d0 2023-11-28T18:28:55.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.414+0000 7f56f67fc640 1 -- 172.21.15.105:0/1299368612 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f56f801d3b0 con 0x7f570015d2d0 2023-11-28T18:28:55.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.414+0000 7f56f67fc640 1 --2- 172.21.15.105:0/1299368612 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56e40668d0 0x7f56e4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:55.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.414+0000 7f56f67fc640 1 -- 172.21.15.105:0/1299368612 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f56f80840d0 con 0x7f570015d2d0 2023-11-28T18:28:55.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.414+0000 7f5705cab640 1 --2- 172.21.15.105:0/1299368612 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56e40668d0 0x7f56e4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:55.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.414+0000 7f5705cab640 1 --2- 172.21.15.105:0/1299368612 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56e40668d0 0x7f56e4068d90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0ae000 tx=0x7f56fc0a6070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:55.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.418+0000 7f56f67fc640 1 -- 172.21.15.105:0/1299368612 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f570015f850 con 0x7f570015d2d0 2023-11-28T18:28:55.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.566+0000 7f5706cad640 1 -- 172.21.15.105:0/1299368612 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f570015f850 con 0x7f570015d2d0 2023-11-28T18:28:55.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.566+0000 7f56f67fc640 1 -- 172.21.15.105:0/1299368612 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+13514 (secure 0 0 0) 0x7f570015f850 con 0x7f570015d2d0 2023-11-28T18:28:55.572 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:55.572 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":81,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:28:53.224207+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6835/2489361300":"2023-11-29T18:28:50.180592+0000","172.21.15.148:6833/2489361300":"2023-11-29T18:28:50.180592+0000","172.21.15.105:6836/188173955":"2023-11-29T18:28:50.180599+0000","172.21.15.105:6834/188173955":"2023-11-29T18:28:50.180599+0000","172.21.15.148:6834/759522673":"2023-11-29T18:28:50.180577+0000","172.21.15.148:6832/759522673":"2023-11-29T18:28:50.180577+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:28:55.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.570+0000 7f5706cad640 1 -- 172.21.15.105:0/1299368612 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56e40668d0 msgr2=0x7f56e4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:55.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.570+0000 7f5706cad640 1 --2- 172.21.15.105:0/1299368612 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56e40668d0 0x7f56e4068d90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0ae000 tx=0x7f56fc0a6070 comp rx=0 tx=0).stop 2023-11-28T18:28:55.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.570+0000 7f5706cad640 1 -- 172.21.15.105:0/1299368612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f570015d2d0 msgr2=0x7f570015e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:55.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.570+0000 7f5706cad640 1 --2- 172.21.15.105:0/1299368612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f570015d2d0 0x7f570015e190 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f56f800b9a0 tx=0x7f56f800be60 comp rx=0 tx=0).stop 2023-11-28T18:28:55.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.570+0000 7f5706cad640 1 -- 172.21.15.105:0/1299368612 shutdown_connections 2023-11-28T18:28:55.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.570+0000 7f5706cad640 1 --2- 172.21.15.105:0/1299368612 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56e40668d0 0x7f56e4068d90 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.570+0000 7f5706cad640 1 --2- 172.21.15.105:0/1299368612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f570015d2d0 0x7f570015e190 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.570+0000 7f5706cad640 1 --2- 172.21.15.105:0/1299368612 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f570015cef0 0x7f570009ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.570+0000 7f5706cad640 1 --2- 172.21.15.105:0/1299368612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57000d1b40 0x7f570015a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.570+0000 7f5706cad640 1 -- 172.21.15.105:0/1299368612 >> 172.21.15.105:0/1299368612 conn(0x7f57000d2e10 msgr2=0x7f570006c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:55.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.570+0000 7f5706cad640 1 -- 172.21.15.105:0/1299368612 shutdown_connections 2023-11-28T18:28:55.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.570+0000 7f5706cad640 1 -- 172.21.15.105:0/1299368612 wait complete. 2023-11-28T18:28:55.594 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:28:55.594 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:28:55.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.730+0000 7fc427f01640 1 Processor -- start 2023-11-28T18:28:55.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.730+0000 7fc427f01640 1 -- start start 2023-11-28T18:28:55.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.730+0000 7fc427f01640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc420071960 0x7fc42015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:55.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.730+0000 7fc427f01640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc42015b030 0x7fc4200709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:55.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.730+0000 7fc427f01640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4200701d0 con 0x7fc4200b4290 2023-11-28T18:28:55.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.730+0000 7fc427f01640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc42006df80 con 0x7fc420071960 2023-11-28T18:28:55.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.730+0000 7fc427f01640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc420071df0 con 0x7fc42015b030 2023-11-28T18:28:55.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.730+0000 7fc425c9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc4200b4290 0x7fc4200b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43726/0 (socket says 172.21.15.105:43726) 2023-11-28T18:28:55.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.730+0000 7fc425c9d640 1 -- 172.21.15.105:0/65658277 learned_addr learned my addr 172.21.15.105:0/65658277 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:55.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.730+0000 7fc42549c640 1 --2- 172.21.15.105:0/65658277 >> v2:172.21.15.105:3300/0 conn(0x7fc420071960 0x7fc42015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:55.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.730+0000 7fc42649e640 1 --2- 172.21.15.105:0/65658277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc42015b030 0x7fc4200709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:55.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.730+0000 7fc424c9b640 1 -- 172.21.15.105:0/65658277 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 51380752 0 0) 0x7fc4200701d0 con 0x7fc4200b4290 2023-11-28T18:28:55.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.730+0000 7fc424c9b640 1 -- 172.21.15.105:0/65658277 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc40c003530 con 0x7fc4200b4290 2023-11-28T18:28:55.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.730+0000 7fc42549c640 1 -- 172.21.15.105:0/65658277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc42015b030 msgr2=0x7fc4200709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:55.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.730+0000 7fc42549c640 1 --2- 172.21.15.105:0/65658277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc42015b030 0x7fc4200709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.730+0000 7fc42549c640 1 -- 172.21.15.105:0/65658277 >> v1:172.21.15.105:6789/0 conn(0x7fc4200b4290 legacy=0x7fc4200b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:55.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.730+0000 7fc42549c640 1 -- 172.21.15.105:0/65658277 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4200720d0 con 0x7fc420071960 2023-11-28T18:28:55.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.734+0000 7fc42549c640 1 --2- 172.21.15.105:0/65658277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc420071960 0x7fc42015f3b0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fc4100097c0 tx=0x7fc41001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c0e3b7dc4c6a03 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:55.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.734+0000 7fc424c9b640 1 -- 172.21.15.105:0/65658277 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4100ad660 con 0x7fc420071960 2023-11-28T18:28:55.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.734+0000 7fc424c9b640 1 -- 172.21.15.105:0/65658277 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4100ad7c0 con 0x7fc420071960 2023-11-28T18:28:55.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.734+0000 7fc424c9b640 1 -- 172.21.15.105:0/65658277 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4100adb50 con 0x7fc420071960 2023-11-28T18:28:55.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.734+0000 7fc427f01640 1 -- 172.21.15.105:0/65658277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc420071960 msgr2=0x7fc42015f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:55.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.734+0000 7fc427f01640 1 --2- 172.21.15.105:0/65658277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc420071960 0x7fc42015f3b0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fc4100097c0 tx=0x7fc41001c450 comp rx=0 tx=0).stop 2023-11-28T18:28:55.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.734+0000 7fc427f01640 1 -- 172.21.15.105:0/65658277 shutdown_connections 2023-11-28T18:28:55.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.734+0000 7fc427f01640 1 --2- 172.21.15.105:0/65658277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc42015b030 0x7fc4200709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.734+0000 7fc427f01640 1 --2- 172.21.15.105:0/65658277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc420071960 0x7fc42015f3b0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.734+0000 7fc427f01640 1 -- 172.21.15.105:0/65658277 >> 172.21.15.105:0/65658277 conn(0x7fc4200ceb90 msgr2=0x7fc4200cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:55.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.734+0000 7fc427f01640 1 -- 172.21.15.105:0/65658277 shutdown_connections 2023-11-28T18:28:55.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.734+0000 7fc427f01640 1 -- 172.21.15.105:0/65658277 wait complete. 2023-11-28T18:28:55.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc427f01640 1 Processor -- start 2023-11-28T18:28:55.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc427f01640 1 -- start start 2023-11-28T18:28:55.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc427f01640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4200b4290 0x7fc420152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:55.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc427f01640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc42015b030 0x7fc4200b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:55.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc427f01640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc42015d280 0x7fc42015d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:55.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc427f01640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc420153530 con 0x7fc4200b4290 2023-11-28T18:28:55.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc427f01640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42015dc50 con 0x7fc42015d280 2023-11-28T18:28:55.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc427f01640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc42015c3f0 con 0x7fc42015b030 2023-11-28T18:28:55.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc425c9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4200b4290 0x7fc420152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:55.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc425c9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4200b4290 0x7fc420152ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39292/0 (socket says 172.21.15.105:39292) 2023-11-28T18:28:55.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc425c9d640 1 -- 172.21.15.105:0/1685771436 learned_addr learned my addr 172.21.15.105:0/1685771436 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:55.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc42549c640 1 --2- 172.21.15.105:0/1685771436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc42015b030 0x7fc4200b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:55.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc42649e640 1 --2- 172.21.15.105:0/1685771436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc42015d280 0x7fc42015d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:55.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc425c9d640 1 -- 172.21.15.105:0/1685771436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc42015b030 msgr2=0x7fc4200b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:55.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc425c9d640 1 --2- 172.21.15.105:0/1685771436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc42015b030 0x7fc4200b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc425c9d640 1 -- 172.21.15.105:0/1685771436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc42015d280 msgr2=0x7fc42015d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:55.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc425c9d640 1 --2- 172.21.15.105:0/1685771436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc42015d280 0x7fc42015d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc425c9d640 1 -- 172.21.15.105:0/1685771436 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc410009420 con 0x7fc4200b4290 2023-11-28T18:28:55.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc425c9d640 1 --2- 172.21.15.105:0/1685771436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4200b4290 0x7fc420152ff0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c500 tx=0x7fc41c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:55.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc4167fc640 1 -- 172.21.15.105:0/1685771436 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c005cf0 con 0x7fc4200b4290 2023-11-28T18:28:55.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc4167fc640 1 -- 172.21.15.105:0/1685771436 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41c005e50 con 0x7fc4200b4290 2023-11-28T18:28:55.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc4167fc640 1 -- 172.21.15.105:0/1685771436 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c0072f0 con 0x7fc4200b4290 2023-11-28T18:28:55.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc427f01640 1 -- 172.21.15.105:0/1685771436 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4201fb050 con 0x7fc4200b4290 2023-11-28T18:28:55.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.738+0000 7fc427f01640 1 -- 172.21.15.105:0/1685771436 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc42015bac0 con 0x7fc4200b4290 2023-11-28T18:28:55.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.742+0000 7fc427f01640 1 -- 172.21.15.105:0/1685771436 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc420067570 con 0x7fc4200b4290 2023-11-28T18:28:55.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.742+0000 7fc4167fc640 1 -- 172.21.15.105:0/1685771436 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc41c007450 con 0x7fc4200b4290 2023-11-28T18:28:55.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.742+0000 7fc4167fc640 1 --2- 172.21.15.105:0/1685771436 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc404066900 0x7fc404068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:55.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.742+0000 7fc4167fc640 1 -- 172.21.15.105:0/1685771436 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fc41c083840 con 0x7fc4200b4290 2023-11-28T18:28:55.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.742+0000 7fc42549c640 1 --2- 172.21.15.105:0/1685771436 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc404066900 0x7fc404068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:55.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.742+0000 7fc42549c640 1 --2- 172.21.15.105:0/1685771436 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc404066900 0x7fc404068dc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc4100af000 tx=0x7fc410008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:55.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.746+0000 7fc4167fc640 1 -- 172.21.15.105:0/1685771436 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc41c050c30 con 0x7fc4200b4290 2023-11-28T18:28:55.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.894+0000 7fc427f01640 1 -- 172.21.15.105:0/1685771436 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc4200cf7f0 con 0x7fc4200b4290 2023-11-28T18:28:55.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.894+0000 7fc4167fc640 1 -- 172.21.15.105:0/1685771436 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc41c0170a0 con 0x7fc4200b4290 2023-11-28T18:28:55.900 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:28:55.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.898+0000 7fc427f01640 1 -- 172.21.15.105:0/1685771436 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc404066900 msgr2=0x7fc404068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:55.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.898+0000 7fc427f01640 1 --2- 172.21.15.105:0/1685771436 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc404066900 0x7fc404068dc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc4100af000 tx=0x7fc410008040 comp rx=0 tx=0).stop 2023-11-28T18:28:55.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.898+0000 7fc427f01640 1 -- 172.21.15.105:0/1685771436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4200b4290 msgr2=0x7fc420152ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:55.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.898+0000 7fc427f01640 1 --2- 172.21.15.105:0/1685771436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4200b4290 0x7fc420152ff0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c500 tx=0x7fc41c00c9c0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.898+0000 7fc427f01640 1 -- 172.21.15.105:0/1685771436 shutdown_connections 2023-11-28T18:28:55.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.898+0000 7fc427f01640 1 --2- 172.21.15.105:0/1685771436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc42015d280 0x7fc42015d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.898+0000 7fc427f01640 1 --2- 172.21.15.105:0/1685771436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc42015b030 0x7fc4200b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.898+0000 7fc427f01640 1 --2- 172.21.15.105:0/1685771436 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc404066900 0x7fc404068dc0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.898+0000 7fc427f01640 1 --2- 172.21.15.105:0/1685771436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4200b4290 0x7fc420152ff0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:55.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.898+0000 7fc427f01640 1 -- 172.21.15.105:0/1685771436 >> 172.21.15.105:0/1685771436 conn(0x7fc4200ceb90 msgr2=0x7fc42015f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:55.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.898+0000 7fc427f01640 1 -- 172.21.15.105:0/1685771436 shutdown_connections 2023-11-28T18:28:55.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:55.898+0000 7fc427f01640 1 -- 172.21.15.105:0/1685771436 wait complete. 2023-11-28T18:28:55.921 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_after' 2023-11-28T18:28:56.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.054+0000 7fa6aae9a640 1 Processor -- start 2023-11-28T18:28:56.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.054+0000 7fa6aae9a640 1 -- start start 2023-11-28T18:28:56.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.054+0000 7fa6aae9a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa6a40b3d30 0x7fa6a41680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:56.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.054+0000 7fa6aae9a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a40b25b0 con 0x7fa6a40d3c30 2023-11-28T18:28:56.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.054+0000 7fa6aae9a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a40d1c30 con 0x7fa6a415d4f0 2023-11-28T18:28:56.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.054+0000 7fa6aae9a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6a40b2900 con 0x7fa6a40b3d30 2023-11-28T18:28:56.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.054+0000 7fa6a9437640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa6a40b3d30 0x7fa6a41680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:56.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.054+0000 7fa69bfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa6a40d3c30 0x7fa6a40d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43734/0 (socket says 172.21.15.105:43734) 2023-11-28T18:28:56.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.054+0000 7fa69bfff640 1 -- 172.21.15.105:0/62991059 learned_addr learned my addr 172.21.15.105:0/62991059 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:56.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.054+0000 7fa6a9437640 1 -- 172.21.15.105:0/62991059 >> v1:172.21.15.148:6789/0 conn(0x7fa6a415d4f0 legacy=0x7fa6a40d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:56.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.054+0000 7fa6a9437640 1 -- 172.21.15.105:0/62991059 >> v1:172.21.15.105:6789/0 conn(0x7fa6a40d3c30 legacy=0x7fa6a40d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:56.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.054+0000 7fa6a9437640 1 -- 172.21.15.105:0/62991059 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a40cea30 con 0x7fa6a40b3d30 2023-11-28T18:28:56.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.058+0000 7fa6a9437640 1 --2- 172.21.15.105:0/62991059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6a40b3d30 0x7fa6a41680a0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009d90 tx=0x7fa69c01c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f793b09da415b7c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:56.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.058+0000 7fa69b7fe640 1 -- 172.21.15.105:0/62991059 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c0a58c0 con 0x7fa6a40b3d30 2023-11-28T18:28:56.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.058+0000 7fa69b7fe640 1 -- 172.21.15.105:0/62991059 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c0a5a20 con 0x7fa6a40b3d30 2023-11-28T18:28:56.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.058+0000 7fa69b7fe640 1 -- 172.21.15.105:0/62991059 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c0a5db0 con 0x7fa6a40b3d30 2023-11-28T18:28:56.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.058+0000 7fa6aae9a640 1 -- 172.21.15.105:0/62991059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6a40b3d30 msgr2=0x7fa6a41680a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:56.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.058+0000 7fa6aae9a640 1 --2- 172.21.15.105:0/62991059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6a40b3d30 0x7fa6a41680a0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009d90 tx=0x7fa69c01c800 comp rx=0 tx=0).stop 2023-11-28T18:28:56.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.058+0000 7fa6aae9a640 1 -- 172.21.15.105:0/62991059 shutdown_connections 2023-11-28T18:28:56.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.058+0000 7fa6aae9a640 1 --2- 172.21.15.105:0/62991059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6a40b3d30 0x7fa6a41680a0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:56.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.058+0000 7fa6aae9a640 1 -- 172.21.15.105:0/62991059 >> 172.21.15.105:0/62991059 conn(0x7fa6a40d3030 msgr2=0x7fa6a40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:56.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.058+0000 7fa6aae9a640 1 -- 172.21.15.105:0/62991059 shutdown_connections 2023-11-28T18:28:56.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.058+0000 7fa6aae9a640 1 -- 172.21.15.105:0/62991059 wait complete. 2023-11-28T18:28:56.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa6aae9a640 1 Processor -- start 2023-11-28T18:28:56.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa6aae9a640 1 -- start start 2023-11-28T18:28:56.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa6aae9a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6a40b3d30 0x7fa6a414b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:56.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa6aae9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6a40d3c30 0x7fa6a41560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:56.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa6aae9a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6a415d4f0 0x7fa6a414e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:56.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa6aae9a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a40d0a20 con 0x7fa6a40d3c30 2023-11-28T18:28:56.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa6aae9a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a414b8f0 con 0x7fa6a415d4f0 2023-11-28T18:28:56.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa6aae9a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a414f590 con 0x7fa6a40b3d30 2023-11-28T18:28:56.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa69bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6a40d3c30 0x7fa6a41560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:56.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa69bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6a40d3c30 0x7fa6a41560d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39314/0 (socket says 172.21.15.105:39314) 2023-11-28T18:28:56.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa69bfff640 1 -- 172.21.15.105:0/3453652269 learned_addr learned my addr 172.21.15.105:0/3453652269 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:56.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa6a8c36640 1 --2- 172.21.15.105:0/3453652269 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6a40b3d30 0x7fa6a414b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:56.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa6a9437640 1 --2- 172.21.15.105:0/3453652269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6a415d4f0 0x7fa6a414e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:56.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa69bfff640 1 -- 172.21.15.105:0/3453652269 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6a40b3d30 msgr2=0x7fa6a414b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:56.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa69bfff640 1 --2- 172.21.15.105:0/3453652269 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6a40b3d30 0x7fa6a414b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:56.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa69bfff640 1 -- 172.21.15.105:0/3453652269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6a415d4f0 msgr2=0x7fa6a414e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:56.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa69bfff640 1 --2- 172.21.15.105:0/3453652269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6a415d4f0 0x7fa6a414e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:56.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa69bfff640 1 -- 172.21.15.105:0/3453652269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69c009a70 con 0x7fa6a40d3c30 2023-11-28T18:28:56.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa69bfff640 1 --2- 172.21.15.105:0/3453652269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6a40d3c30 0x7fa6a41560d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fa69400c430 tx=0x7fa69400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:56.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa6997fa640 1 -- 172.21.15.105:0/3453652269 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa694005cf0 con 0x7fa6a40d3c30 2023-11-28T18:28:56.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa6997fa640 1 -- 172.21.15.105:0/3453652269 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa694005e50 con 0x7fa6a40d3c30 2023-11-28T18:28:56.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa6997fa640 1 -- 172.21.15.105:0/3453652269 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6940072d0 con 0x7fa6a40d3c30 2023-11-28T18:28:56.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa6aae9a640 1 -- 172.21.15.105:0/3453652269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a4154700 con 0x7fa6a40d3c30 2023-11-28T18:28:56.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.062+0000 7fa6aae9a640 1 -- 172.21.15.105:0/3453652269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a41549a0 con 0x7fa6a40d3c30 2023-11-28T18:28:56.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.066+0000 7fa6aae9a640 1 -- 172.21.15.105:0/3453652269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a41544c0 con 0x7fa6a40d3c30 2023-11-28T18:28:56.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.066+0000 7fa6997fa640 1 -- 172.21.15.105:0/3453652269 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa694007430 con 0x7fa6a40d3c30 2023-11-28T18:28:56.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.066+0000 7fa6997fa640 1 --2- 172.21.15.105:0/3453652269 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6900668d0 0x7fa690068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:56.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.066+0000 7fa6997fa640 1 -- 172.21.15.105:0/3453652269 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fa694083850 con 0x7fa6a40d3c30 2023-11-28T18:28:56.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.066+0000 7fa6a8c36640 1 --2- 172.21.15.105:0/3453652269 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6900668d0 0x7fa690068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:56.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.066+0000 7fa6a8c36640 1 --2- 172.21.15.105:0/3453652269 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6900668d0 0x7fa690068d90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4159200 tx=0x7fa6a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:56.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.070+0000 7fa6997fa640 1 -- 172.21.15.105:0/3453652269 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa6a41544c0 con 0x7fa6a40d3c30 2023-11-28T18:28:56.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:56.274+0000 7fa6aae9a640 1 -- 172.21.15.105:0/3453652269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_after"]} v 0) v1 -- 0x7fa6a41544c0 con 0x7fa6a40d3c30 2023-11-28T18:28:57.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.250+0000 7fa6997fa640 1 -- 172.21.15.105:0/3453652269 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_after"]}]=0 v293) v1 ==== 164+0+0 (secure 0 0 0) 0x7fa6a41544c0 con 0x7fa6a40d3c30 2023-11-28T18:28:57.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.250+0000 7fa6aae9a640 1 -- 172.21.15.105:0/3453652269 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6900668d0 msgr2=0x7fa690068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:57.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.250+0000 7fa6aae9a640 1 --2- 172.21.15.105:0/3453652269 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6900668d0 0x7fa690068d90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4159200 tx=0x7fa6a0009040 comp rx=0 tx=0).stop 2023-11-28T18:28:57.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.250+0000 7fa6aae9a640 1 -- 172.21.15.105:0/3453652269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6a40d3c30 msgr2=0x7fa6a41560d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:57.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.250+0000 7fa6aae9a640 1 --2- 172.21.15.105:0/3453652269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6a40d3c30 0x7fa6a41560d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fa69400c430 tx=0x7fa69400c8f0 comp rx=0 tx=0).stop 2023-11-28T18:28:57.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.250+0000 7fa6aae9a640 1 -- 172.21.15.105:0/3453652269 shutdown_connections 2023-11-28T18:28:57.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.250+0000 7fa6aae9a640 1 --2- 172.21.15.105:0/3453652269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6a415d4f0 0x7fa6a414e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:57.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.250+0000 7fa6aae9a640 1 --2- 172.21.15.105:0/3453652269 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6900668d0 0x7fa690068d90 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:57.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.250+0000 7fa6aae9a640 1 --2- 172.21.15.105:0/3453652269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6a40d3c30 0x7fa6a41560d0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:57.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.250+0000 7fa6aae9a640 1 --2- 172.21.15.105:0/3453652269 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6a40b3d30 0x7fa6a414b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:57.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.254+0000 7fa6aae9a640 1 -- 172.21.15.105:0/3453652269 >> 172.21.15.105:0/3453652269 conn(0x7fa6a40d3030 msgr2=0x7fa6a406c510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:57.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.254+0000 7fa6aae9a640 1 -- 172.21.15.105:0/3453652269 shutdown_connections 2023-11-28T18:28:57.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.254+0000 7fa6aae9a640 1 -- 172.21.15.105:0/3453652269 wait complete. 2023-11-28T18:28:57.280 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T18:28:57.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c7ed6640 1 Processor -- start 2023-11-28T18:28:57.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c7ed6640 1 -- start start 2023-11-28T18:28:57.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c7ed6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28c0159f40 0x7f28c00caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:57.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c7ed6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c006bf50 con 0x7f28c00ad850 2023-11-28T18:28:57.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c7ed6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c00ccbc0 con 0x7f28c00cc760 2023-11-28T18:28:57.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c7ed6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c015a320 con 0x7f28c0159f40 2023-11-28T18:28:57.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c6473640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f28c00ad850 0x7f28c00701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43744/0 (socket says 172.21.15.105:43744) 2023-11-28T18:28:57.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c6473640 1 -- 172.21.15.105:0/2171195439 learned_addr learned my addr 172.21.15.105:0/2171195439 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:28:57.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c4c70640 1 -- 172.21.15.105:0/2171195439 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619159897 0 0) 0x7f28c006bf50 con 0x7f28c00ad850 2023-11-28T18:28:57.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c5c72640 1 --2- 172.21.15.105:0/2171195439 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28c0159f40 0x7f28c00caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:57.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c4c70640 1 -- 172.21.15.105:0/2171195439 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28ac003530 con 0x7f28c00ad850 2023-11-28T18:28:57.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c4c70640 1 -- 172.21.15.105:0/2171195439 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2946822061 0 0) 0x7f28ac003530 con 0x7f28c00ad850 2023-11-28T18:28:57.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c4c70640 1 -- 172.21.15.105:0/2171195439 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28c0159f40 msgr2=0x7f28c00caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:57.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c4c70640 1 --2- 172.21.15.105:0/2171195439 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28c0159f40 0x7f28c00caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:57.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c4c70640 1 -- 172.21.15.105:0/2171195439 >> v1:172.21.15.148:6789/0 conn(0x7f28c00cc760 legacy=0x7f28c00ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:57.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c4c70640 1 -- 172.21.15.105:0/2171195439 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c0071380 con 0x7f28c00ad850 2023-11-28T18:28:57.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c4c70640 1 -- 172.21.15.105:0/2171195439 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f28b8001a00 con 0x7f28c00ad850 2023-11-28T18:28:57.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c4c70640 1 -- 172.21.15.105:0/2171195439 >> v1:172.21.15.105:6789/0 conn(0x7f28c00ad850 legacy=0x7f28c00701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:57.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c4c70640 1 --2- 172.21.15.105:0/2171195439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28ac0039b0 0x7f28ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:57.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c4c70640 1 --2- 172.21.15.105:0/2171195439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ac0047d0 0x7f28ac004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:57.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c4c70640 1 --2- 172.21.15.105:0/2171195439 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28c0159f40 0x7f28ac005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:57.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c4c70640 1 -- 172.21.15.105:0/2171195439 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c015a320 con 0x7f28ac0039b0 2023-11-28T18:28:57.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c4c70640 1 -- 172.21.15.105:0/2171195439 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac00bb20 con 0x7f28ac0047d0 2023-11-28T18:28:57.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c4c70640 1 -- 172.21.15.105:0/2171195439 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac00def0 con 0x7f28c0159f40 2023-11-28T18:28:57.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c5c72640 1 --2- 172.21.15.105:0/2171195439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28ac0039b0 0x7f28ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:57.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c6473640 1 --2- 172.21.15.105:0/2171195439 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28c0159f40 0x7f28ac005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:57.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c5471640 1 --2- 172.21.15.105:0/2171195439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ac0047d0 0x7f28ac004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:57.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c5c72640 1 -- 172.21.15.105:0/2171195439 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28c0159f40 msgr2=0x7f28ac005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:57.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c5c72640 1 --2- 172.21.15.105:0/2171195439 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28c0159f40 0x7f28ac005240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:57.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c5c72640 1 -- 172.21.15.105:0/2171195439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ac0047d0 msgr2=0x7f28ac004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:57.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c5c72640 1 --2- 172.21.15.105:0/2171195439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ac0047d0 0x7f28ac004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:57.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.422+0000 7f28c5c72640 1 -- 172.21.15.105:0/2171195439 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ac003530 con 0x7f28ac0039b0 2023-11-28T18:28:57.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.426+0000 7f28c5c72640 1 --2- 172.21.15.105:0/2171195439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28ac0039b0 0x7f28ac003dd0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009ae0 tx=0x7f28bc01ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fe41a2d517ca98 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:57.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.426+0000 7f28c4c70640 1 -- 172.21.15.105:0/2171195439 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc0a6510 con 0x7f28ac0039b0 2023-11-28T18:28:57.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.426+0000 7f28c4c70640 1 -- 172.21.15.105:0/2171195439 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc0a6670 con 0x7f28ac0039b0 2023-11-28T18:28:57.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.426+0000 7f28c4c70640 1 -- 172.21.15.105:0/2171195439 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc0a6a20 con 0x7f28ac0039b0 2023-11-28T18:28:57.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.426+0000 7f28c7ed6640 1 -- 172.21.15.105:0/2171195439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28ac0039b0 msgr2=0x7f28ac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:57.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.426+0000 7f28c7ed6640 1 --2- 172.21.15.105:0/2171195439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28ac0039b0 0x7f28ac003dd0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009ae0 tx=0x7f28bc01ca20 comp rx=0 tx=0).stop 2023-11-28T18:28:57.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.426+0000 7f28c6473640 1 -- 172.21.15.105:0/2171195439 reap_dead start 2023-11-28T18:28:57.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.426+0000 7f28c7ed6640 1 -- 172.21.15.105:0/2171195439 shutdown_connections 2023-11-28T18:28:57.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.426+0000 7f28c7ed6640 1 -- 172.21.15.105:0/2171195439 >> 172.21.15.105:0/2171195439 conn(0x7f28c00ceb90 msgr2=0x7f28c00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:57.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.426+0000 7f28c7ed6640 1 -- 172.21.15.105:0/2171195439 shutdown_connections 2023-11-28T18:28:57.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.426+0000 7f28c7ed6640 1 -- 172.21.15.105:0/2171195439 wait complete. 2023-11-28T18:28:57.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28c7ed6640 1 Processor -- start 2023-11-28T18:28:57.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28c7ed6640 1 -- start start 2023-11-28T18:28:57.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28c7ed6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28ac0039b0 0x7f28c015a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:57.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28c7ed6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28c014ed50 0x7f28c00ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:57.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28c7ed6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28c014f1b0 0x7f28c00cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:57.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28c7ed6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c00c9580 con 0x7f28c014ed50 2023-11-28T18:28:57.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28c7ed6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c0156f30 con 0x7f28c014f1b0 2023-11-28T18:28:57.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28c7ed6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c0156970 con 0x7f28ac0039b0 2023-11-28T18:28:57.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28c5471640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28c014ed50 0x7f28c00ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:57.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28c5471640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28c014ed50 0x7f28c00ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39338/0 (socket says 172.21.15.105:39338) 2023-11-28T18:28:57.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28c5471640 1 -- 172.21.15.105:0/2028823066 learned_addr learned my addr 172.21.15.105:0/2028823066 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:57.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28c5c72640 1 --2- 172.21.15.105:0/2028823066 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28ac0039b0 0x7f28c015a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:57.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28c6473640 1 --2- 172.21.15.105:0/2028823066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28c014f1b0 0x7f28c00cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:57.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28c5471640 1 -- 172.21.15.105:0/2028823066 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28ac0039b0 msgr2=0x7f28c015a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:57.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28c5471640 1 --2- 172.21.15.105:0/2028823066 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28ac0039b0 0x7f28c015a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:57.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28c5471640 1 -- 172.21.15.105:0/2028823066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28c014f1b0 msgr2=0x7f28c00cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:57.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28c5471640 1 --2- 172.21.15.105:0/2028823066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28c014f1b0 0x7f28c00cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:57.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28c5471640 1 -- 172.21.15.105:0/2028823066 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c0156c50 con 0x7f28c014ed50 2023-11-28T18:28:57.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28c6473640 1 --2- 172.21.15.105:0/2028823066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28c014f1b0 0x7f28c00cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:28:57.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28c5471640 1 --2- 172.21.15.105:0/2028823066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28c014ed50 0x7f28c00ccd50 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f28b000f5f0 tx=0x7f28b000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:57.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28b67fc640 1 -- 172.21.15.105:0/2028823066 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b00052e0 con 0x7f28c014ed50 2023-11-28T18:28:57.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28b67fc640 1 -- 172.21.15.105:0/2028823066 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b0005440 con 0x7f28c014ed50 2023-11-28T18:28:57.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28b67fc640 1 -- 172.21.15.105:0/2028823066 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b0003820 con 0x7f28c014ed50 2023-11-28T18:28:57.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28c7ed6640 1 -- 172.21.15.105:0/2028823066 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c00c91a0 con 0x7f28c014ed50 2023-11-28T18:28:57.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.430+0000 7f28c7ed6640 1 -- 172.21.15.105:0/2028823066 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c015a9a0 con 0x7f28c014ed50 2023-11-28T18:28:57.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.434+0000 7f28c7ed6640 1 -- 172.21.15.105:0/2028823066 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c0067570 con 0x7f28c014ed50 2023-11-28T18:28:57.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.434+0000 7f28b67fc640 1 -- 172.21.15.105:0/2028823066 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f28b0003980 con 0x7f28c014ed50 2023-11-28T18:28:57.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.434+0000 7f28b67fc640 1 --2- 172.21.15.105:0/2028823066 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f28a4066900 0x7f28a4068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:57.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.434+0000 7f28b67fc640 1 -- 172.21.15.105:0/2028823066 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f28b0084160 con 0x7f28c014ed50 2023-11-28T18:28:57.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.434+0000 7f28c5c72640 1 --2- 172.21.15.105:0/2028823066 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f28a4066900 0x7f28a4068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:57.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.434+0000 7f28c5c72640 1 --2- 172.21.15.105:0/2028823066 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f28a4066900 0x7f28a4068dc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009ae0 tx=0x7f28bc0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:57.443 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.438+0000 7f28b67fc640 1 -- 172.21.15.105:0/2028823066 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f28b00514a0 con 0x7f28c014ed50 2023-11-28T18:28:57.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.586+0000 7f28c7ed6640 1 -- 172.21.15.105:0/2028823066 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f28c00cf950 con 0x7f28c014ed50 2023-11-28T18:28:57.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.594+0000 7f28b67fc640 1 -- 172.21.15.105:0/2028823066 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v43)=0 v43) v1 ==== 122+0+0 (secure 0 0 0) 0x7f28b0054ac0 con 0x7f28c014ed50 2023-11-28T18:28:57.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.594+0000 7f28c7ed6640 1 -- 172.21.15.105:0/2028823066 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f28a4066900 msgr2=0x7f28a4068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:57.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.594+0000 7f28c7ed6640 1 --2- 172.21.15.105:0/2028823066 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f28a4066900 0x7f28a4068dc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009ae0 tx=0x7f28bc0046d0 comp rx=0 tx=0).stop 2023-11-28T18:28:57.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.594+0000 7f28c7ed6640 1 -- 172.21.15.105:0/2028823066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28c014ed50 msgr2=0x7f28c00ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:57.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.594+0000 7f28c7ed6640 1 --2- 172.21.15.105:0/2028823066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28c014ed50 0x7f28c00ccd50 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f28b000f5f0 tx=0x7f28b000fab0 comp rx=0 tx=0).stop 2023-11-28T18:28:57.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.594+0000 7f28c7ed6640 1 -- 172.21.15.105:0/2028823066 shutdown_connections 2023-11-28T18:28:57.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.594+0000 7f28c7ed6640 1 --2- 172.21.15.105:0/2028823066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28c014f1b0 0x7f28c00cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:57.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.594+0000 7f28c7ed6640 1 --2- 172.21.15.105:0/2028823066 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f28a4066900 0x7f28a4068dc0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:57.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.594+0000 7f28c7ed6640 1 --2- 172.21.15.105:0/2028823066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28c014ed50 0x7f28c00ccd50 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:57.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.594+0000 7f28c7ed6640 1 --2- 172.21.15.105:0/2028823066 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28ac0039b0 0x7f28c015a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:57.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.594+0000 7f28c7ed6640 1 -- 172.21.15.105:0/2028823066 >> 172.21.15.105:0/2028823066 conn(0x7f28c00ceb90 msgr2=0x7f28c00701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:57.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.594+0000 7f28c7ed6640 1 -- 172.21.15.105:0/2028823066 shutdown_connections 2023-11-28T18:28:57.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:57.594+0000 7f28c7ed6640 1 -- 172.21.15.105:0/2028823066 wait complete. 2023-11-28T18:28:57.618 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T18:28:57.619 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:28:57.707 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:28:57.824 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T18:28:57.824 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:28:57.824 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:28:57.834 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:28:57.834 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:28:57.834 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:28:57.844 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:28:57.845 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:28:57.845 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:28:57.890 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:28:57.890 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:28:57.890 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:28:57.900 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:28:57.900 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T18:28:57.900 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:28:58.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.066+0000 7f68bab1d640 1 Processor -- start 2023-11-28T18:28:58.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.066+0000 7f68bab1d640 1 -- start start 2023-11-28T18:28:58.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.066+0000 7f68bab1d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f68b415d4f0 0x7f68b40d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:58.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.066+0000 7f68bab1d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68b40d3c30 0x7f68b40d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:58.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.066+0000 7f68bab1d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b40b25b0 con 0x7f68b40d1890 2023-11-28T18:28:58.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.066+0000 7f68bab1d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b40b2900 con 0x7f68b415d4f0 2023-11-28T18:28:58.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.066+0000 7f68bab1d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40cea30 con 0x7f68b40d3c30 2023-11-28T18:28:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.066+0000 7f68abfff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68b40d3c30 0x7f68b40d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.066+0000 7f68b88b9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f68b415d4f0 0x7f68b40d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.066+0000 7f68b88b9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f68b415d4f0 0x7f68b40d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:57080/0 (socket says 172.21.15.105:57080) 2023-11-28T18:28:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.066+0000 7f68abfff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68b40d3c30 0x7f68b40d1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:49984/0 (socket says 172.21.15.105:49984) 2023-11-28T18:28:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.066+0000 7f68b88b9640 1 -- 172.21.15.105:0/1094563276 learned_addr learned my addr 172.21.15.105:0/1094563276 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.066+0000 7f68b90ba640 1 --1- 172.21.15.105:0/1094563276 >> v1:172.21.15.148:6789/0 conn(0x7f68b40d1890 0x7f68b40cfa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:51408/0 (socket says 172.21.15.105:51408) 2023-11-28T18:28:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.066+0000 7f68b88b9640 1 -- 172.21.15.105:0/1094563276 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68b40d3c30 msgr2=0x7f68b40d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:28:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.066+0000 7f68b88b9640 1 --2- 172.21.15.105:0/1094563276 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68b40d3c30 0x7f68b40d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:58.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.066+0000 7f68b88b9640 1 -- 172.21.15.105:0/1094563276 >> v1:172.21.15.148:6789/0 conn(0x7f68b40d1890 legacy=0x7f68b40cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:58.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.066+0000 7f68b88b9640 1 -- 172.21.15.105:0/1094563276 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b40ced10 con 0x7f68b415d4f0 2023-11-28T18:28:58.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.066+0000 7f68abfff640 1 --2- 172.21.15.105:0/1094563276 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68b40d3c30 0x7f68b40d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:28:58.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.070+0000 7f68b88b9640 1 --2- 172.21.15.105:0/1094563276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68b415d4f0 0x7f68b40d0e10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009870 tx=0x7f68b001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f21f87459f597515 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:58.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.070+0000 7f68ab7fe640 1 -- 172.21.15.105:0/1094563276 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b00ad640 con 0x7f68b415d4f0 2023-11-28T18:28:58.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.070+0000 7f68ab7fe640 1 -- 172.21.15.105:0/1094563276 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b00ad7a0 con 0x7f68b415d4f0 2023-11-28T18:28:58.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.070+0000 7f68ab7fe640 1 -- 172.21.15.105:0/1094563276 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b00adb30 con 0x7f68b415d4f0 2023-11-28T18:28:58.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.070+0000 7f68bab1d640 1 -- 172.21.15.105:0/1094563276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68b415d4f0 msgr2=0x7f68b40d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:58.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.070+0000 7f68bab1d640 1 --2- 172.21.15.105:0/1094563276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68b415d4f0 0x7f68b40d0e10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009870 tx=0x7f68b001c480 comp rx=0 tx=0).stop 2023-11-28T18:28:58.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.070+0000 7f68bab1d640 1 -- 172.21.15.105:0/1094563276 shutdown_connections 2023-11-28T18:28:58.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.070+0000 7f68bab1d640 1 --2- 172.21.15.105:0/1094563276 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68b40d3c30 0x7f68b40d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:58.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.070+0000 7f68bab1d640 1 --2- 172.21.15.105:0/1094563276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68b415d4f0 0x7f68b40d0e10 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:58.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.070+0000 7f68bab1d640 1 -- 172.21.15.105:0/1094563276 >> 172.21.15.105:0/1094563276 conn(0x7f68b40d3030 msgr2=0x7f68b40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:58.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.070+0000 7f68bab1d640 1 -- 172.21.15.105:0/1094563276 shutdown_connections 2023-11-28T18:28:58.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.074+0000 7f68bab1d640 1 -- 172.21.15.105:0/1094563276 wait complete. 2023-11-28T18:28:58.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.074+0000 7f68bab1d640 1 Processor -- start 2023-11-28T18:28:58.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.074+0000 7f68bab1d640 1 -- start start 2023-11-28T18:28:58.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.074+0000 7f68bab1d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68b40d1890 0x7f68b414a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:58.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.074+0000 7f68bab1d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68b40d3c30 0x7f68b40c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:58.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.074+0000 7f68bab1d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68b415d4f0 0x7f68b406b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:58.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.074+0000 7f68bab1d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40ccbe0 con 0x7f68b40d3c30 2023-11-28T18:28:58.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.074+0000 7f68bab1d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40cc8b0 con 0x7f68b415d4f0 2023-11-28T18:28:58.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.074+0000 7f68bab1d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40ca1a0 con 0x7f68b40d1890 2023-11-28T18:28:58.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.074+0000 7f68abfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68b40d3c30 0x7f68b40c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:58.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.074+0000 7f68abfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68b40d3c30 0x7f68b40c9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39344/0 (socket says 172.21.15.105:39344) 2023-11-28T18:28:58.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.074+0000 7f68abfff640 1 -- 172.21.15.105:0/529046706 learned_addr learned my addr 172.21.15.105:0/529046706 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:28:58.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.074+0000 7f68b88b9640 1 --2- 172.21.15.105:0/529046706 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68b40d1890 0x7f68b414a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:58.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.074+0000 7f68b90ba640 1 --2- 172.21.15.105:0/529046706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68b415d4f0 0x7f68b406b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:58.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.074+0000 7f68abfff640 1 -- 172.21.15.105:0/529046706 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68b40d1890 msgr2=0x7f68b414a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:58.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.074+0000 7f68abfff640 1 --2- 172.21.15.105:0/529046706 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68b40d1890 0x7f68b414a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:58.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.074+0000 7f68abfff640 1 -- 172.21.15.105:0/529046706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68b415d4f0 msgr2=0x7f68b406b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:58.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.074+0000 7f68abfff640 1 --2- 172.21.15.105:0/529046706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68b415d4f0 0x7f68b406b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:58.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.074+0000 7f68abfff640 1 -- 172.21.15.105:0/529046706 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a4009420 con 0x7f68b40d3c30 2023-11-28T18:28:58.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.078+0000 7f68abfff640 1 --2- 172.21.15.105:0/529046706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68b40d3c30 0x7f68b40c9800 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f68a400e980 tx=0x7f68a400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:58.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.078+0000 7f68a97fa640 1 -- 172.21.15.105:0/529046706 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a400ca00 con 0x7f68b40d3c30 2023-11-28T18:28:58.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.078+0000 7f68a97fa640 1 -- 172.21.15.105:0/529046706 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a400cb60 con 0x7f68b40d3c30 2023-11-28T18:28:58.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.078+0000 7f68a97fa640 1 -- 172.21.15.105:0/529046706 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4013470 con 0x7f68b40d3c30 2023-11-28T18:28:58.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.078+0000 7f68bab1d640 1 -- 172.21.15.105:0/529046706 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b00094d0 con 0x7f68b40d3c30 2023-11-28T18:28:58.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.078+0000 7f68bab1d640 1 -- 172.21.15.105:0/529046706 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b40ca760 con 0x7f68b40d3c30 2023-11-28T18:28:58.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.078+0000 7f68bab1d640 1 -- 172.21.15.105:0/529046706 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b40ccd50 con 0x7f68b40d3c30 2023-11-28T18:28:58.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.078+0000 7f68a97fa640 1 -- 172.21.15.105:0/529046706 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f68a40135d0 con 0x7f68b40d3c30 2023-11-28T18:28:58.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.078+0000 7f68a97fa640 1 --2- 172.21.15.105:0/529046706 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f68a0066900 0x7f68a0068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:28:58.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.078+0000 7f68a97fa640 1 -- 172.21.15.105:0/529046706 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f68a4084490 con 0x7f68b40d3c30 2023-11-28T18:28:58.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.078+0000 7f68b88b9640 1 --2- 172.21.15.105:0/529046706 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f68a0066900 0x7f68a0068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:28:58.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.078+0000 7f68b88b9640 1 --2- 172.21.15.105:0/529046706 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f68a0066900 0x7f68a0068dc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f68b00af000 tx=0x7f68b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:28:58.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.082+0000 7f68a97fa640 1 -- 172.21.15.105:0/529046706 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f68a4051880 con 0x7f68b40d3c30 2023-11-28T18:28:58.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.266+0000 7f68bab1d640 1 -- 172.21.15.105:0/529046706 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68b4067570 con 0x7f68b40d3c30 2023-11-28T18:28:58.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.266+0000 7f68a97fa640 1 -- 172.21.15.105:0/529046706 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f68a4021e90 con 0x7f68b40d3c30 2023-11-28T18:28:58.271 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:28:58.271 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":79,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6410,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1745286025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1745286025},{"type":"v1","addr":"172.21.15.148:6835","nonce":1745286025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":6856,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1455698212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":1455698212},{"type":"v1","addr":"172.21.15.105:6837","nonce":1455698212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7177,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/162306242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":162306242},{"type":"v1","addr":"172.21.15.148:6833","nonce":162306242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/125016080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":125016080},{"type":"v1","addr":"172.21.15.105:6836","nonce":125016080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[]} 2023-11-28T18:28:58.271 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 79 2023-11-28T18:28:58.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.270+0000 7f68bab1d640 1 -- 172.21.15.105:0/529046706 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f68a0066900 msgr2=0x7f68a0068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:58.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.270+0000 7f68bab1d640 1 --2- 172.21.15.105:0/529046706 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f68a0066900 0x7f68a0068dc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f68b00af000 tx=0x7f68b0008040 comp rx=0 tx=0).stop 2023-11-28T18:28:58.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.270+0000 7f68bab1d640 1 -- 172.21.15.105:0/529046706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68b40d3c30 msgr2=0x7f68b40c9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:28:58.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.270+0000 7f68bab1d640 1 --2- 172.21.15.105:0/529046706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68b40d3c30 0x7f68b40c9800 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f68a400e980 tx=0x7f68a400ee40 comp rx=0 tx=0).stop 2023-11-28T18:28:58.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.270+0000 7f68bab1d640 1 -- 172.21.15.105:0/529046706 shutdown_connections 2023-11-28T18:28:58.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.270+0000 7f68bab1d640 1 --2- 172.21.15.105:0/529046706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68b415d4f0 0x7f68b406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:58.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.270+0000 7f68bab1d640 1 --2- 172.21.15.105:0/529046706 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f68a0066900 0x7f68a0068dc0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:58.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.270+0000 7f68bab1d640 1 --2- 172.21.15.105:0/529046706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68b40d3c30 0x7f68b40c9800 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:58.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.270+0000 7f68bab1d640 1 --2- 172.21.15.105:0/529046706 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68b40d1890 0x7f68b414a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:28:58.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.270+0000 7f68bab1d640 1 -- 172.21.15.105:0/529046706 >> 172.21.15.105:0/529046706 conn(0x7f68b40d3030 msgr2=0x7f68b406c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:28:58.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.270+0000 7f68bab1d640 1 -- 172.21.15.105:0/529046706 shutdown_connections 2023-11-28T18:28:58.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:28:58.270+0000 7f68bab1d640 1 -- 172.21.15.105:0/529046706 wait complete. 2023-11-28T18:28:58.293 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T18:28:58.293 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T18:28:58.293 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T18:28:58.293 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:28:58.294 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T18:28:58.294 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T18:28:58.294 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T18:28:58.294 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:28:58.294 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T18:28:58.294 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T18:28:58.295 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T18:28:58.295 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:28:58.295 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T18:28:58.295 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T18:28:58.295 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T18:28:58.295 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:28:58.311 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T18:28:58.312 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T18:28:58.313 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T18:28:58.313 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T18:28:58.314 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T18:28:58.314 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T18:28:58.356 INFO:tasks.ceph.mds.c:Started 2023-11-28T18:28:58.356 DEBUG:teuthology.parallel:result is None 2023-11-28T18:28:58.360 INFO:tasks.ceph.mds.b:Started 2023-11-28T18:28:58.361 DEBUG:teuthology.parallel:result is None 2023-11-28T18:28:58.362 INFO:tasks.ceph.mds.d:Started 2023-11-28T18:28:58.363 DEBUG:teuthology.parallel:result is None 2023-11-28T18:28:58.439 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2023-11-28T18:28:58.452 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2023-11-28T18:28:58.452 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2023-11-28T18:29:02.371 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:29:02.375 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:29:04.064 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-11-28T18:29:04.312 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T18:29:04.312 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T18:29:04.316 INFO:tasks.ceph.mds.a:Started 2023-11-28T18:29:04.316 DEBUG:teuthology.parallel:result is None 2023-11-28T18:29:04.317 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T18:29:04.410 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2023-11-28T18:29:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb297646640 1 Processor -- start 2023-11-28T18:29:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb297646640 1 -- start start 2023-11-28T18:29:04.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb297646640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2900cc760 0x7fb290164d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:04.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb297646640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2900ccc10 con 0x7fb2900caf60 2023-11-28T18:29:04.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb297646640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb29006bf50 con 0x7fb290159f40 2023-11-28T18:29:04.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb297646640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2900cb3c0 con 0x7fb2900cc760 2023-11-28T18:29:04.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb294be1640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb2900caf60 0x7fb2900ad850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:38492/0 (socket says 172.21.15.105:38492) 2023-11-28T18:29:04.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb294be1640 1 -- 172.21.15.105:0/10769618 learned_addr learned my addr 172.21.15.105:0/10769618 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:04.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb287fff640 1 -- 172.21.15.105:0/10769618 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2844743599 0 0) 0x7fb2900ccc10 con 0x7fb2900caf60 2023-11-28T18:29:04.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb287fff640 1 -- 172.21.15.105:0/10769618 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb27c005740 con 0x7fb2900caf60 2023-11-28T18:29:04.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb295be3640 1 --2- 172.21.15.105:0/10769618 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2900cc760 0x7fb290164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb287fff640 1 -- 172.21.15.105:0/10769618 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 90024054 0 0) 0x7fb27c005740 con 0x7fb2900caf60 2023-11-28T18:29:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb287fff640 1 -- 172.21.15.105:0/10769618 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2900cc760 msgr2=0x7fb290164d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb287fff640 1 --2- 172.21.15.105:0/10769618 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2900cc760 0x7fb290164d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb287fff640 1 -- 172.21.15.105:0/10769618 >> v1:172.21.15.148:6789/0 conn(0x7fb290159f40 legacy=0x7fb2900ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb287fff640 1 -- 172.21.15.105:0/10769618 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29015a320 con 0x7fb2900caf60 2023-11-28T18:29:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb287fff640 1 -- 172.21.15.105:0/10769618 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fb280001680 con 0x7fb2900caf60 2023-11-28T18:29:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb287fff640 1 -- 172.21.15.105:0/10769618 >> v1:172.21.15.105:6789/0 conn(0x7fb2900caf60 legacy=0x7fb2900ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb287fff640 1 --2- 172.21.15.105:0/10769618 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2900cc760 0x7fb27c005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb287fff640 1 --2- 172.21.15.105:0/10769618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb27c006140 0x7fb27c0069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb287fff640 1 --2- 172.21.15.105:0/10769618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb27c006f20 0x7fb27c0073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb287fff640 1 -- 172.21.15.105:0/10769618 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2900cb3c0 con 0x7fb27c006f20 2023-11-28T18:29:04.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb287fff640 1 -- 172.21.15.105:0/10769618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c00dd30 con 0x7fb27c006140 2023-11-28T18:29:04.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb287fff640 1 -- 172.21.15.105:0/10769618 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c00def0 con 0x7fb2900cc760 2023-11-28T18:29:04.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb2953e2640 1 --2- 172.21.15.105:0/10769618 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2900cc760 0x7fb27c005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:04.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb294be1640 1 --2- 172.21.15.105:0/10769618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb27c006140 0x7fb27c0069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:04.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb295be3640 1 --2- 172.21.15.105:0/10769618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb27c006f20 0x7fb27c0073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:04.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb2953e2640 1 -- 172.21.15.105:0/10769618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb27c006140 msgr2=0x7fb27c0069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:04.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb2953e2640 1 --2- 172.21.15.105:0/10769618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb27c006140 0x7fb27c0069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:04.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb2953e2640 1 -- 172.21.15.105:0/10769618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb27c006f20 msgr2=0x7fb27c0073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:04.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb2953e2640 1 --2- 172.21.15.105:0/10769618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb27c006f20 0x7fb27c0073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:04.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.466+0000 7fb2953e2640 1 -- 172.21.15.105:0/10769618 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27c005740 con 0x7fb2900cc760 2023-11-28T18:29:04.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.470+0000 7fb2953e2640 1 --2- 172.21.15.105:0/10769618 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2900cc760 0x7fb27c005bc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb28c0017b0 tx=0x7fb28c01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3302861d0984543 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:04.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.470+0000 7fb287fff640 1 -- 172.21.15.105:0/10769618 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c0052d0 con 0x7fb2900cc760 2023-11-28T18:29:04.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.470+0000 7fb287fff640 1 -- 172.21.15.105:0/10769618 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28c0a7050 con 0x7fb2900cc760 2023-11-28T18:29:04.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.470+0000 7fb287fff640 1 -- 172.21.15.105:0/10769618 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c003600 con 0x7fb2900cc760 2023-11-28T18:29:04.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.470+0000 7fb297646640 1 -- 172.21.15.105:0/10769618 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2900cc760 msgr2=0x7fb27c005bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:04.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.470+0000 7fb297646640 1 --2- 172.21.15.105:0/10769618 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2900cc760 0x7fb27c005bc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb28c0017b0 tx=0x7fb28c01c3e0 comp rx=0 tx=0).stop 2023-11-28T18:29:04.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.470+0000 7fb295be3640 1 -- 172.21.15.105:0/10769618 reap_dead start 2023-11-28T18:29:04.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.470+0000 7fb297646640 1 -- 172.21.15.105:0/10769618 shutdown_connections 2023-11-28T18:29:04.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.470+0000 7fb297646640 1 -- 172.21.15.105:0/10769618 >> 172.21.15.105:0/10769618 conn(0x7fb2900ceb90 msgr2=0x7fb2900cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:04.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.470+0000 7fb297646640 1 -- 172.21.15.105:0/10769618 shutdown_connections 2023-11-28T18:29:04.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.470+0000 7fb297646640 1 -- 172.21.15.105:0/10769618 wait complete. 2023-11-28T18:29:04.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.474+0000 7fb297646640 1 Processor -- start 2023-11-28T18:29:04.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.474+0000 7fb297646640 1 -- start start 2023-11-28T18:29:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.474+0000 7fb297646640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2900cc760 0x7fb29014e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.474+0000 7fb297646640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29009bdc0 0x7fb29009c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.474+0000 7fb297646640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb29015b020 0x7fb29015b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.474+0000 7fb297646640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2901e3660 con 0x7fb2900cc760 2023-11-28T18:29:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.474+0000 7fb297646640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29015c5f0 con 0x7fb29009bdc0 2023-11-28T18:29:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.474+0000 7fb297646640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29015bd50 con 0x7fb29015b020 2023-11-28T18:29:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.474+0000 7fb2953e2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2900cc760 0x7fb29014e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.474+0000 7fb2953e2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2900cc760 0x7fb29014e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57134/0 (socket says 172.21.15.105:57134) 2023-11-28T18:29:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.474+0000 7fb2953e2640 1 -- 172.21.15.105:0/311762018 learned_addr learned my addr 172.21.15.105:0/311762018 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.474+0000 7fb294be1640 1 --2- 172.21.15.105:0/311762018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29009bdc0 0x7fb29009c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:04.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.474+0000 7fb295be3640 1 --2- 172.21.15.105:0/311762018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb29015b020 0x7fb29015b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:04.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.474+0000 7fb2953e2640 1 -- 172.21.15.105:0/311762018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb29015b020 msgr2=0x7fb29015b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:04.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.474+0000 7fb2953e2640 1 --2- 172.21.15.105:0/311762018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb29015b020 0x7fb29015b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:04.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.474+0000 7fb2953e2640 1 -- 172.21.15.105:0/311762018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29009bdc0 msgr2=0x7fb29009c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:04.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.474+0000 7fb2953e2640 1 --2- 172.21.15.105:0/311762018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29009bdc0 0x7fb29009c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:04.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.474+0000 7fb2953e2640 1 -- 172.21.15.105:0/311762018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29015c030 con 0x7fb2900cc760 2023-11-28T18:29:04.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.474+0000 7fb2953e2640 1 --2- 172.21.15.105:0/311762018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2900cc760 0x7fb29014e010 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fb28c0017b0 tx=0x7fb27c007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:04.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.474+0000 7fb285ffb640 1 -- 172.21.15.105:0/311762018 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c007250 con 0x7fb2900cc760 2023-11-28T18:29:04.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.478+0000 7fb285ffb640 1 -- 172.21.15.105:0/311762018 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28c0073b0 con 0x7fb2900cc760 2023-11-28T18:29:04.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.478+0000 7fb297646640 1 -- 172.21.15.105:0/311762018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29015c310 con 0x7fb2900cc760 2023-11-28T18:29:04.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.478+0000 7fb285ffb640 1 -- 172.21.15.105:0/311762018 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c0067e0 con 0x7fb2900cc760 2023-11-28T18:29:04.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.478+0000 7fb297646640 1 -- 172.21.15.105:0/311762018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29015b9e0 con 0x7fb2900cc760 2023-11-28T18:29:04.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.478+0000 7fb297646640 1 -- 172.21.15.105:0/311762018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2901e38d0 con 0x7fb2900cc760 2023-11-28T18:29:04.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.478+0000 7fb285ffb640 1 -- 172.21.15.105:0/311762018 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fb28c006940 con 0x7fb2900cc760 2023-11-28T18:29:04.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.478+0000 7fb285ffb640 1 --2- 172.21.15.105:0/311762018 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb2740668d0 0x7fb274068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:04.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.478+0000 7fb285ffb640 1 -- 172.21.15.105:0/311762018 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fb28c11e600 con 0x7fb2900cc760 2023-11-28T18:29:04.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.478+0000 7fb294be1640 1 --2- 172.21.15.105:0/311762018 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb2740668d0 0x7fb274068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:04.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.478+0000 7fb294be1640 1 --2- 172.21.15.105:0/311762018 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb2740668d0 0x7fb274068d90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb29009bb50 tx=0x7fb280001be0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:04.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.482+0000 7fb285ffb640 1 -- 172.21.15.105:0/311762018 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb28c0eb960 con 0x7fb2900cc760 2023-11-28T18:29:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:04.642+0000 7fb297646640 1 -- 172.21.15.105:0/311762018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb290067570 con 0x7fb2900cc760 2023-11-28T18:29:06.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.290+0000 7fb285ffb640 1 -- 172.21.15.105:0/311762018 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v86) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb28c0eef80 con 0x7fb2900cc760 2023-11-28T18:29:06.294 INFO:teuthology.orchestra.run.smithi105.stderr: removed all blocklist entries 2023-11-28T18:29:06.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.294+0000 7fb297646640 1 -- 172.21.15.105:0/311762018 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb2740668d0 msgr2=0x7fb274068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:06.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.294+0000 7fb297646640 1 --2- 172.21.15.105:0/311762018 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb2740668d0 0x7fb274068d90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb29009bb50 tx=0x7fb280001be0 comp rx=0 tx=0).stop 2023-11-28T18:29:06.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.294+0000 7fb297646640 1 -- 172.21.15.105:0/311762018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2900cc760 msgr2=0x7fb29014e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:06.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.294+0000 7fb297646640 1 --2- 172.21.15.105:0/311762018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2900cc760 0x7fb29014e010 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fb28c0017b0 tx=0x7fb27c007a10 comp rx=0 tx=0).stop 2023-11-28T18:29:06.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.294+0000 7fb297646640 1 -- 172.21.15.105:0/311762018 shutdown_connections 2023-11-28T18:29:06.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.294+0000 7fb297646640 1 --2- 172.21.15.105:0/311762018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb29015b020 0x7fb29015b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:06.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.294+0000 7fb297646640 1 --2- 172.21.15.105:0/311762018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29009bdc0 0x7fb29009c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:06.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.294+0000 7fb297646640 1 --2- 172.21.15.105:0/311762018 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb2740668d0 0x7fb274068d90 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:06.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.294+0000 7fb297646640 1 --2- 172.21.15.105:0/311762018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2900cc760 0x7fb29014e010 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:06.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.294+0000 7fb297646640 1 -- 172.21.15.105:0/311762018 >> 172.21.15.105:0/311762018 conn(0x7fb2900ceb90 msgr2=0x7fb2900ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:06.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.294+0000 7fb297646640 1 -- 172.21.15.105:0/311762018 shutdown_connections 2023-11-28T18:29:06.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.294+0000 7fb297646640 1 -- 172.21.15.105:0/311762018 wait complete. 2023-11-28T18:29:06.323 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T18:29:06.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.462+0000 7fd1d19f8640 1 Processor -- start 2023-11-28T18:29:06.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.462+0000 7fd1d19f8640 1 -- start start 2023-11-28T18:29:06.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.462+0000 7fd1d19f8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd1cc159f40 0x7fd1cc0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:06.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.462+0000 7fd1d19f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd1cc0cc760 0x7fd1cc0ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:06.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.462+0000 7fd1d19f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1cc0ad680 0x7fd1cc15da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:06.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.462+0000 7fd1d19f8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1cc06bf50 con 0x7fd1cc159f40 2023-11-28T18:29:06.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.462+0000 7fd1d19f8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1cc0ccbc0 con 0x7fd1cc0cc760 2023-11-28T18:29:06.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.462+0000 7fd1d19f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc15a320 con 0x7fd1cc0ad680 2023-11-28T18:29:06.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.462+0000 7fd1caffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd1cc159f40 0x7fd1cc0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:06.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.462+0000 7fd1caffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd1cc159f40 0x7fd1cc0caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57136/0 (socket says 172.21.15.105:57136) 2023-11-28T18:29:06.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.462+0000 7fd1caffd640 1 -- 172.21.15.105:0/469996717 learned_addr learned my addr 172.21.15.105:0/469996717 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:06.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.466+0000 7fd1caffd640 1 -- 172.21.15.105:0/469996717 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1cc0ad680 msgr2=0x7fd1cc15da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:06.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.466+0000 7fd1caffd640 1 --2- 172.21.15.105:0/469996717 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1cc0ad680 0x7fd1cc15da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:06.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.466+0000 7fd1caffd640 1 -- 172.21.15.105:0/469996717 >> v2:172.21.15.148:3300/0 conn(0x7fd1cc0cc760 msgr2=0x7fd1cc0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:06.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.466+0000 7fd1c3fff640 1 --2- 172.21.15.105:0/469996717 >> v2:172.21.15.148:3300/0 conn(0x7fd1cc0cc760 0x7fd1cc0ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:06.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.466+0000 7fd1caffd640 1 --2- 172.21.15.105:0/469996717 >> v2:172.21.15.148:3300/0 conn(0x7fd1cc0cc760 0x7fd1cc0ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:06.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.466+0000 7fd1caffd640 1 -- 172.21.15.105:0/469996717 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc071380 con 0x7fd1cc159f40 2023-11-28T18:29:06.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.466+0000 7fd1c3fff640 1 --2- 172.21.15.105:0/469996717 >> v2:172.21.15.148:3300/0 conn(0x7fd1cc0cc760 0x7fd1cc0ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:06.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.466+0000 7fd1caffd640 1 --2- 172.21.15.105:0/469996717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1cc159f40 0x7fd1cc0caf60 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc09d0d0 tx=0x7fd1bc01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de54a52995065b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:06.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.466+0000 7fd1ca7fc640 1 -- 172.21.15.105:0/469996717 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc0ab600 con 0x7fd1cc159f40 2023-11-28T18:29:06.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.466+0000 7fd1ca7fc640 1 -- 172.21.15.105:0/469996717 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1bc0ab760 con 0x7fd1cc159f40 2023-11-28T18:29:06.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.466+0000 7fd1ca7fc640 1 -- 172.21.15.105:0/469996717 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc0abaf0 con 0x7fd1cc159f40 2023-11-28T18:29:06.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1d19f8640 1 -- 172.21.15.105:0/469996717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1cc159f40 msgr2=0x7fd1cc0caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:06.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1d19f8640 1 --2- 172.21.15.105:0/469996717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1cc159f40 0x7fd1cc0caf60 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc09d0d0 tx=0x7fd1bc01c440 comp rx=0 tx=0).stop 2023-11-28T18:29:06.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1d19f8640 1 -- 172.21.15.105:0/469996717 shutdown_connections 2023-11-28T18:29:06.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1d19f8640 1 --2- 172.21.15.105:0/469996717 >> v2:172.21.15.148:3300/0 conn(0x7fd1cc0cc760 0x7fd1cc0ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:06.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1d19f8640 1 --2- 172.21.15.105:0/469996717 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1cc0ad680 0x7fd1cc15da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:06.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1d19f8640 1 --2- 172.21.15.105:0/469996717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1cc159f40 0x7fd1cc0caf60 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:06.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1d19f8640 1 -- 172.21.15.105:0/469996717 >> 172.21.15.105:0/469996717 conn(0x7fd1cc0ceb90 msgr2=0x7fd1cc0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:06.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1d19f8640 1 -- 172.21.15.105:0/469996717 shutdown_connections 2023-11-28T18:29:06.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1d19f8640 1 -- 172.21.15.105:0/469996717 wait complete. 2023-11-28T18:29:06.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1d19f8640 1 Processor -- start 2023-11-28T18:29:06.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1d19f8640 1 -- start start 2023-11-28T18:29:06.479 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1d19f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1cc0ad680 0x7fd1cc149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:06.479 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1d19f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1cc0cc760 0x7fd1cc14a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:06.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1d19f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1cc159f40 0x7fd1cc14b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:06.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1d19f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc15bab0 con 0x7fd1cc159f40 2023-11-28T18:29:06.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1d19f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc15bd90 con 0x7fd1cc0ad680 2023-11-28T18:29:06.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1d19f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc15c070 con 0x7fd1cc0cc760 2023-11-28T18:29:06.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1cb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1cc159f40 0x7fd1cc14b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:06.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1cb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1cc159f40 0x7fd1cc14b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57144/0 (socket says 172.21.15.105:57144) 2023-11-28T18:29:06.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1cb7fe640 1 -- 172.21.15.105:0/2198566990 learned_addr learned my addr 172.21.15.105:0/2198566990 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:06.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1c3fff640 1 --2- 172.21.15.105:0/2198566990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1cc0cc760 0x7fd1cc14a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:06.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1caffd640 1 --2- 172.21.15.105:0/2198566990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1cc0ad680 0x7fd1cc149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:06.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1cb7fe640 1 -- 172.21.15.105:0/2198566990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1cc0cc760 msgr2=0x7fd1cc14a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:06.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1cb7fe640 1 --2- 172.21.15.105:0/2198566990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1cc0cc760 0x7fd1cc14a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:06.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1cb7fe640 1 -- 172.21.15.105:0/2198566990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1cc0ad680 msgr2=0x7fd1cc149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:06.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1cb7fe640 1 --2- 172.21.15.105:0/2198566990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1cc0ad680 0x7fd1cc149770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:06.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1cb7fe640 1 -- 172.21.15.105:0/2198566990 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1bc0094d0 con 0x7fd1cc159f40 2023-11-28T18:29:06.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1caffd640 1 --2- 172.21.15.105:0/2198566990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1cc0ad680 0x7fd1cc149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:06.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1c3fff640 1 --2- 172.21.15.105:0/2198566990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1cc0cc760 0x7fd1cc14a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:06.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.470+0000 7fd1cb7fe640 1 --2- 172.21.15.105:0/2198566990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1cc159f40 0x7fd1cc14b0b0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400b970 tx=0x7fd1c400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:06.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.474+0000 7fd1c8ff9640 1 -- 172.21.15.105:0/2198566990 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c400cb40 con 0x7fd1cc159f40 2023-11-28T18:29:06.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.474+0000 7fd1c8ff9640 1 -- 172.21.15.105:0/2198566990 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c400ccc0 con 0x7fd1cc159f40 2023-11-28T18:29:06.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.474+0000 7fd1c8ff9640 1 -- 172.21.15.105:0/2198566990 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4014680 con 0x7fd1cc159f40 2023-11-28T18:29:06.484 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.474+0000 7fd1d19f8640 1 -- 172.21.15.105:0/2198566990 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1cc15c350 con 0x7fd1cc159f40 2023-11-28T18:29:06.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.474+0000 7fd1d19f8640 1 -- 172.21.15.105:0/2198566990 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1cc149cb0 con 0x7fd1cc159f40 2023-11-28T18:29:06.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.474+0000 7fd1d19f8640 1 -- 172.21.15.105:0/2198566990 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1cc067570 con 0x7fd1cc159f40 2023-11-28T18:29:06.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.474+0000 7fd1c8ff9640 1 -- 172.21.15.105:0/2198566990 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd1c401d3b0 con 0x7fd1cc159f40 2023-11-28T18:29:06.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.474+0000 7fd1c8ff9640 1 --2- 172.21.15.105:0/2198566990 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd1ac0668d0 0x7fd1ac068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:06.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.474+0000 7fd1c8ff9640 1 -- 172.21.15.105:0/2198566990 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fd1c4084220 con 0x7fd1cc159f40 2023-11-28T18:29:06.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.474+0000 7fd1caffd640 1 --2- 172.21.15.105:0/2198566990 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd1ac0668d0 0x7fd1ac068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:06.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.474+0000 7fd1caffd640 1 --2- 172.21.15.105:0/2198566990 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd1ac0668d0 0x7fd1ac068d90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc0af000 tx=0x7fd1bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:06.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.478+0000 7fd1c8ff9640 1 -- 172.21.15.105:0/2198566990 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd1c40515d0 con 0x7fd1cc159f40 2023-11-28T18:29:06.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.678+0000 7fd1d19f8640 1 -- 172.21.15.105:0/2198566990 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd1cc0b0480 con 0x7fd1cc159f40 2023-11-28T18:29:06.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.678+0000 7fd1c8ff9640 1 -- 172.21.15.105:0/2198566990 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd1c4054bf0 con 0x7fd1cc159f40 2023-11-28T18:29:06.683 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:29:06.684 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:29:06.684 INFO:teuthology.orchestra.run.smithi105.stdout: "auth_dump": [ 2023-11-28T18:29:06.684 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:29:06.684 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.a", 2023-11-28T18:29:06.684 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlOXSTGxAAU5kGNB3tznYsDNG51vCIZA==", 2023-11-28T18:29:06.684 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:29:06.684 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:29:06.684 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:29:06.684 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:29:06.684 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:29:06.684 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:29:06.685 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:29:06.685 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:29:06.685 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.b", 2023-11-28T18:29:06.685 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlrnbWJxAAKefl0YI2MhobnAOIj6WeaQ==", 2023-11-28T18:29:06.685 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:29:06.685 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:29:06.685 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:29:06.685 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:29:06.685 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:29:06.685 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:29:06.685 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:29:06.686 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:29:06.686 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.c", 2023-11-28T18:29:06.686 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2Zl5Y7GIhAA5OvgkNNFaI7LnzSsQVdFsQ==", 2023-11-28T18:29:06.686 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:29:06.686 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:29:06.686 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:29:06.686 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:29:06.686 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:29:06.686 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:29:06.686 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:29:06.686 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:29:06.687 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.d", 2023-11-28T18:29:06.687 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlmfYdLxAAreI7UE1YCjzkj6yl3s5Fwg==", 2023-11-28T18:29:06.687 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:29:06.687 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:29:06.687 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:29:06.687 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:29:06.687 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:29:06.687 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:29:06.687 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:29:06.687 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:29:06.687 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.0", 2023-11-28T18:29:06.688 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDtL2ZlphxQEBAAQ280OXh1zeaotIdVU8oZng==", 2023-11-28T18:29:06.688 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:29:06.688 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:29:06.688 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:29:06.688 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:29:06.688 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:29:06.688 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:29:06.688 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:29:06.688 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.1", 2023-11-28T18:29:06.688 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDwL2ZlezmOFxAARRBIOTkVJYfquIdcpW7sQQ==", 2023-11-28T18:29:06.688 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:29:06.689 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:29:06.689 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:29:06.689 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:29:06.689 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:29:06.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:29:06.689 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:29:06.689 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.2", 2023-11-28T18:29:06.689 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDzL2Zl4YcSHxAA2ww2F4TAD/0H/ZJ+aVMGEQ==", 2023-11-28T18:29:06.689 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:29:06.689 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:29:06.690 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:29:06.690 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:29:06.690 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:29:06.690 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:29:06.690 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:29:06.690 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.3", 2023-11-28T18:29:06.690 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD2L2ZlNxWxJRAAImp7TjTF18KwFTZ14DM6EA==", 2023-11-28T18:29:06.690 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:29:06.690 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:29:06.690 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:29:06.691 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:29:06.691 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:29:06.691 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:29:06.691 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:29:06.691 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.4", 2023-11-28T18:29:06.691 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD8L2ZlOAieLxAAz8g+anH8uJRfsttUS41GGQ==", 2023-11-28T18:29:06.691 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:29:06.691 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:29:06.691 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:29:06.691 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:29:06.691 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:29:06.691 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:29:06.692 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:29:06.692 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.5", 2023-11-28T18:29:06.692 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD/L2Zl5PKTNxAAPk+uIodLPjSd0EEMVT6iEw==", 2023-11-28T18:29:06.692 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:29:06.692 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:29:06.692 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:29:06.692 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:29:06.692 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:29:06.692 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:29:06.692 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:29:06.692 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.6", 2023-11-28T18:29:06.693 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQADMGZll+FUAxAAHlym5Z8rd/OF1911ZE2OUg==", 2023-11-28T18:29:06.693 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:29:06.693 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:29:06.693 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:29:06.693 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:29:06.693 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:29:06.693 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:29:06.693 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:29:06.693 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.7", 2023-11-28T18:29:06.693 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQAGMGZl8GO4ChAAVn3N6DnxUcs+ew09QvGo2w==", 2023-11-28T18:29:06.693 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:29:06.693 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:29:06.694 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:29:06.694 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:29:06.694 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:29:06.694 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:29:06.694 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:29:06.694 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.0", 2023-11-28T18:29:06.694 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlSt0VMxAAQ8+j1BgBPMJ2cuiZj8aG0w==", 2023-11-28T18:29:06.694 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:29:06.694 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:29:06.694 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow r", 2023-11-28T18:29:06.694 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-28T18:29:06.695 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:29:06.695 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:29:06.695 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:29:06.695 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.1", 2023-11-28T18:29:06.695 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDqL2ZlfZ1/ARAAhnJB6oj4Cc6p/oCjM6hTCw==", 2023-11-28T18:29:06.695 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:29:06.695 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:29:06.695 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:29:06.695 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:29:06.695 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:29:06.695 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:29:06.696 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:29:06.696 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:29:06.696 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.2", 2023-11-28T18:29:06.696 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlPLtyORAAU2GST6TJzlTfh6YqEb3U3w==", 2023-11-28T18:29:06.696 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:29:06.696 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:29:06.696 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:29:06.696 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:29:06.696 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:29:06.696 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:29:06.696 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:29:06.697 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:29:06.697 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.3", 2023-11-28T18:29:06.697 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDqL2ZlhiFuBxAAKQRxCdtqWpAHy1AVvdj9Sg==", 2023-11-28T18:29:06.697 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:29:06.697 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:29:06.697 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:29:06.697 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:29:06.697 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:29:06.697 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:29:06.697 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:29:06.697 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:29:06.698 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.admin", 2023-11-28T18:29:06.698 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDoL2ZlkuP2OhAAxsdZEF9122vkH53EwB8Qug==", 2023-11-28T18:29:06.698 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:29:06.698 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:29:06.698 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:29:06.698 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:29:06.698 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:29:06.698 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:29:06.698 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:29:06.698 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:29:06.699 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.x", 2023-11-28T18:29:06.699 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlEG49FxAAespsG+QK1WTpbmpV+DIwiw==", 2023-11-28T18:29:06.699 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:29:06.699 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:29:06.699 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2023-11-28T18:29:06.699 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:29:06.699 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:29:06.699 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:29:06.699 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:29:06.699 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.y", 2023-11-28T18:29:06.699 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlU89RExAAwMJp+8zZ6aFEt0KbiB6toA==", 2023-11-28T18:29:06.699 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:29:06.700 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:29:06.700 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2023-11-28T18:29:06.700 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:29:06.700 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:29:06.700 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:29:06.700 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2023-11-28T18:29:06.700 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:29:06.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.682+0000 7fd1d19f8640 1 -- 172.21.15.105:0/2198566990 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd1ac0668d0 msgr2=0x7fd1ac068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:06.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.682+0000 7fd1d19f8640 1 --2- 172.21.15.105:0/2198566990 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd1ac0668d0 0x7fd1ac068d90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc0af000 tx=0x7fd1bc008040 comp rx=0 tx=0).stop 2023-11-28T18:29:06.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.682+0000 7fd1d19f8640 1 -- 172.21.15.105:0/2198566990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1cc159f40 msgr2=0x7fd1cc14b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:06.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.682+0000 7fd1d19f8640 1 --2- 172.21.15.105:0/2198566990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1cc159f40 0x7fd1cc14b0b0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400b970 tx=0x7fd1c400be30 comp rx=0 tx=0).stop 2023-11-28T18:29:06.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.682+0000 7fd1d19f8640 1 -- 172.21.15.105:0/2198566990 shutdown_connections 2023-11-28T18:29:06.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.682+0000 7fd1d19f8640 1 --2- 172.21.15.105:0/2198566990 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd1ac0668d0 0x7fd1ac068d90 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:06.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.682+0000 7fd1d19f8640 1 --2- 172.21.15.105:0/2198566990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1cc159f40 0x7fd1cc14b0b0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:06.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.682+0000 7fd1d19f8640 1 --2- 172.21.15.105:0/2198566990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1cc0cc760 0x7fd1cc14a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:06.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.682+0000 7fd1d19f8640 1 --2- 172.21.15.105:0/2198566990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1cc0ad680 0x7fd1cc149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:06.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.682+0000 7fd1d19f8640 1 -- 172.21.15.105:0/2198566990 >> 172.21.15.105:0/2198566990 conn(0x7fd1cc0ceb90 msgr2=0x7fd1cc0ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:06.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.682+0000 7fd1d19f8640 1 -- 172.21.15.105:0/2198566990 shutdown_connections 2023-11-28T18:29:06.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.682+0000 7fd1d19f8640 1 -- 172.21.15.105:0/2198566990 wait complete. 2023-11-28T18:29:06.704 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:29:06.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.838+0000 7feeab679640 1 Processor -- start 2023-11-28T18:29:06.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.838+0000 7feeab679640 1 -- start start 2023-11-28T18:29:06.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.838+0000 7feeab679640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feea415d4f0 0x7feea40d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:06.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.838+0000 7feeab679640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feea40d3c30 0x7feea40d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:06.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.838+0000 7feeab679640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feea40b25b0 con 0x7feea40d1890 2023-11-28T18:29:06.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.838+0000 7feeab679640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feea40b2900 con 0x7feea40d3c30 2023-11-28T18:29:06.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.838+0000 7feeab679640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea40cea30 con 0x7feea415d4f0 2023-11-28T18:29:06.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.838+0000 7feea9415640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feea415d4f0 0x7feea40d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:06.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.838+0000 7feea9415640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feea415d4f0 0x7feea40d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:46234/0 (socket says 172.21.15.105:46234) 2023-11-28T18:29:06.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.838+0000 7feea9415640 1 -- 172.21.15.105:0/3273552936 learned_addr learned my addr 172.21.15.105:0/3273552936 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:06.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.838+0000 7feea8c14640 1 --2- 172.21.15.105:0/3273552936 >> v2:172.21.15.148:3300/0 conn(0x7feea40d3c30 0x7feea40d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:06.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.838+0000 7feea9415640 1 -- 172.21.15.105:0/3273552936 >> v2:172.21.15.148:3300/0 conn(0x7feea40d3c30 msgr2=0x7feea40d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:06.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.838+0000 7feea9415640 1 --2- 172.21.15.105:0/3273552936 >> v2:172.21.15.148:3300/0 conn(0x7feea40d3c30 0x7feea40d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:06.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.838+0000 7feea9415640 1 -- 172.21.15.105:0/3273552936 >> v1:172.21.15.148:6789/0 conn(0x7feea40d1890 legacy=0x7feea40cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:06.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.838+0000 7feea9415640 1 -- 172.21.15.105:0/3273552936 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea40ced10 con 0x7feea415d4f0 2023-11-28T18:29:06.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.842+0000 7feea9415640 1 --2- 172.21.15.105:0/3273552936 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feea415d4f0 0x7feea40d0e10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feea0009870 tx=0x7feea001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26ad51495eb28512 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:06.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.842+0000 7fee9bfff640 1 -- 172.21.15.105:0/3273552936 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea00ad640 con 0x7feea415d4f0 2023-11-28T18:29:06.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.842+0000 7fee9bfff640 1 -- 172.21.15.105:0/3273552936 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea00ad7a0 con 0x7feea415d4f0 2023-11-28T18:29:06.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.842+0000 7fee9bfff640 1 -- 172.21.15.105:0/3273552936 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea00adb30 con 0x7feea415d4f0 2023-11-28T18:29:06.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.842+0000 7feeab679640 1 -- 172.21.15.105:0/3273552936 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feea415d4f0 msgr2=0x7feea40d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:06.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.842+0000 7feeab679640 1 --2- 172.21.15.105:0/3273552936 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feea415d4f0 0x7feea40d0e10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feea0009870 tx=0x7feea001c480 comp rx=0 tx=0).stop 2023-11-28T18:29:06.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.842+0000 7feeab679640 1 -- 172.21.15.105:0/3273552936 shutdown_connections 2023-11-28T18:29:06.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.842+0000 7feeab679640 1 --2- 172.21.15.105:0/3273552936 >> v2:172.21.15.148:3300/0 conn(0x7feea40d3c30 0x7feea40d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:06.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.842+0000 7feeab679640 1 --2- 172.21.15.105:0/3273552936 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feea415d4f0 0x7feea40d0e10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:06.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.842+0000 7feeab679640 1 -- 172.21.15.105:0/3273552936 >> 172.21.15.105:0/3273552936 conn(0x7feea40d3030 msgr2=0x7feea40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:06.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.842+0000 7feeab679640 1 -- 172.21.15.105:0/3273552936 shutdown_connections 2023-11-28T18:29:06.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.842+0000 7feeab679640 1 -- 172.21.15.105:0/3273552936 wait complete. 2023-11-28T18:29:06.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.846+0000 7feeab679640 1 Processor -- start 2023-11-28T18:29:06.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.846+0000 7feeab679640 1 -- start start 2023-11-28T18:29:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.846+0000 7feeab679640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feea40d1890 0x7feea40c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.846+0000 7feeab679640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feea40d3c30 0x7feea40c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.846+0000 7feeab679640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea415d4f0 0x7feea4142410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.846+0000 7feeab679640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea40cab10 con 0x7feea415d4f0 2023-11-28T18:29:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.846+0000 7feeab679640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea40cad60 con 0x7feea40d3c30 2023-11-28T18:29:06.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.846+0000 7feeab679640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea40cb040 con 0x7feea40d1890 2023-11-28T18:29:06.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.846+0000 7feea9c16640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea415d4f0 0x7feea4142410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:06.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.846+0000 7feea9415640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feea40d1890 0x7feea40c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:06.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.846+0000 7feea9c16640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea415d4f0 0x7feea4142410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57156/0 (socket says 172.21.15.105:57156) 2023-11-28T18:29:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.846+0000 7feea9415640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feea40d1890 0x7feea40c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:46238/0 (socket says 172.21.15.105:46238) 2023-11-28T18:29:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.846+0000 7feea9c16640 1 -- 172.21.15.105:0/896716904 learned_addr learned my addr 172.21.15.105:0/896716904 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.846+0000 7feea8c14640 1 --2- 172.21.15.105:0/896716904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feea40d3c30 0x7feea40c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.846+0000 7feea9c16640 1 -- 172.21.15.105:0/896716904 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feea40d1890 msgr2=0x7feea40c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.846+0000 7feea9c16640 1 --2- 172.21.15.105:0/896716904 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feea40d1890 0x7feea40c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.846+0000 7feea9c16640 1 -- 172.21.15.105:0/896716904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feea40d3c30 msgr2=0x7feea40c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.846+0000 7feea9c16640 1 --2- 172.21.15.105:0/896716904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feea40d3c30 0x7feea40c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.846+0000 7feea9c16640 1 -- 172.21.15.105:0/896716904 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea00094d0 con 0x7feea415d4f0 2023-11-28T18:29:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.846+0000 7feea9c16640 1 --2- 172.21.15.105:0/896716904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea415d4f0 0x7feea4142410 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00c730 tx=0x7fee9c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.850+0000 7fee99ffb640 1 -- 172.21.15.105:0/896716904 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c003b80 con 0x7feea415d4f0 2023-11-28T18:29:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.850+0000 7fee99ffb640 1 -- 172.21.15.105:0/896716904 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9c003ce0 con 0x7feea415d4f0 2023-11-28T18:29:06.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.850+0000 7fee99ffb640 1 -- 172.21.15.105:0/896716904 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c00aa30 con 0x7feea415d4f0 2023-11-28T18:29:06.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.850+0000 7feeab679640 1 -- 172.21.15.105:0/896716904 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea40cb320 con 0x7feea415d4f0 2023-11-28T18:29:06.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.850+0000 7feeab679640 1 -- 172.21.15.105:0/896716904 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea4155800 con 0x7feea415d4f0 2023-11-28T18:29:06.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.850+0000 7feeab679640 1 -- 172.21.15.105:0/896716904 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea4067570 con 0x7feea415d4f0 2023-11-28T18:29:06.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.850+0000 7fee99ffb640 1 -- 172.21.15.105:0/896716904 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fee9c00ab90 con 0x7feea415d4f0 2023-11-28T18:29:06.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.850+0000 7fee99ffb640 1 --2- 172.21.15.105:0/896716904 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fee88066900 0x7fee88068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:06.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.850+0000 7fee99ffb640 1 -- 172.21.15.105:0/896716904 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fee9c083f90 con 0x7feea415d4f0 2023-11-28T18:29:06.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.850+0000 7feea9415640 1 --2- 172.21.15.105:0/896716904 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fee88066900 0x7fee88068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:06.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.850+0000 7feea9415640 1 --2- 172.21.15.105:0/896716904 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fee88066900 0x7fee88068dc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7feea00af000 tx=0x7feea00a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:06.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:06.854+0000 7fee99ffb640 1 -- 172.21.15.105:0/896716904 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fee9c051360 con 0x7feea415d4f0 2023-11-28T18:29:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.006+0000 7feeab679640 1 -- 172.21.15.105:0/896716904 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feea40b0cf0 con 0x7feea415d4f0 2023-11-28T18:29:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.006+0000 7fee99ffb640 1 -- 172.21.15.105:0/896716904 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+13123 (secure 0 0 0) 0x7fee9c054980 con 0x7feea415d4f0 2023-11-28T18:29:07.010 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:29:07.010 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":86,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:29:06.291835+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:29:07.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.006+0000 7feeab679640 1 -- 172.21.15.105:0/896716904 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fee88066900 msgr2=0x7fee88068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:07.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.006+0000 7feeab679640 1 --2- 172.21.15.105:0/896716904 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fee88066900 0x7fee88068dc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7feea00af000 tx=0x7feea00a7070 comp rx=0 tx=0).stop 2023-11-28T18:29:07.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.006+0000 7feeab679640 1 -- 172.21.15.105:0/896716904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea415d4f0 msgr2=0x7feea4142410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:07.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.006+0000 7feeab679640 1 --2- 172.21.15.105:0/896716904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea415d4f0 0x7feea4142410 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00c730 tx=0x7fee9c00cbf0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.006+0000 7feeab679640 1 -- 172.21.15.105:0/896716904 shutdown_connections 2023-11-28T18:29:07.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.006+0000 7feeab679640 1 --2- 172.21.15.105:0/896716904 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fee88066900 0x7fee88068dc0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.006+0000 7feeab679640 1 --2- 172.21.15.105:0/896716904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea415d4f0 0x7feea4142410 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.006+0000 7feeab679640 1 --2- 172.21.15.105:0/896716904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feea40d3c30 0x7feea40c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.006+0000 7feeab679640 1 --2- 172.21.15.105:0/896716904 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feea40d1890 0x7feea40c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.006+0000 7feeab679640 1 -- 172.21.15.105:0/896716904 >> 172.21.15.105:0/896716904 conn(0x7feea40d3030 msgr2=0x7feea4155d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:07.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.006+0000 7feeab679640 1 -- 172.21.15.105:0/896716904 shutdown_connections 2023-11-28T18:29:07.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.006+0000 7feeab679640 1 -- 172.21.15.105:0/896716904 wait complete. 2023-11-28T18:29:07.031 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:29:07.031 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:29:07.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.166+0000 7fc65731e640 1 Processor -- start 2023-11-28T18:29:07.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.166+0000 7fc65731e640 1 -- start start 2023-11-28T18:29:07.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.166+0000 7fc65731e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc650159710 0x7fc65006c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:07.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.166+0000 7fc65731e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc6500ce170 0x7fc65015d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:07.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.170+0000 7fc65731e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc65006d070 con 0x7fc6500cd170 2023-11-28T18:29:07.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.170+0000 7fc65731e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc65006cab0 con 0x7fc6500ce170 2023-11-28T18:29:07.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.170+0000 7fc65731e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65006cd90 con 0x7fc650159710 2023-11-28T18:29:07.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.170+0000 7fc655b1b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc6500ce170 0x7fc65015d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:07.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.170+0000 7fc655b1b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc6500ce170 0x7fc65015d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57164/0 (socket says 172.21.15.105:57164) 2023-11-28T18:29:07.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.170+0000 7fc655b1b640 1 -- 172.21.15.105:0/3814021345 learned_addr learned my addr 172.21.15.105:0/3814021345 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:07.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.170+0000 7fc65631c640 1 --2- 172.21.15.105:0/3814021345 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc650159710 0x7fc65006c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:07.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.170+0000 7fc655b1b640 1 -- 172.21.15.105:0/3814021345 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc650159710 msgr2=0x7fc65006c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:07.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.170+0000 7fc655b1b640 1 --2- 172.21.15.105:0/3814021345 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc650159710 0x7fc65006c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.170+0000 7fc655b1b640 1 -- 172.21.15.105:0/3814021345 >> v1:172.21.15.148:6789/0 conn(0x7fc6500cd170 legacy=0x7fc650070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:07.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.170+0000 7fc655b1b640 1 -- 172.21.15.105:0/3814021345 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65015d140 con 0x7fc6500ce170 2023-11-28T18:29:07.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.170+0000 7fc655b1b640 1 --2- 172.21.15.105:0/3814021345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6500ce170 0x7fc65015d940 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fc64000bb10 tx=0x7fc64001ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b2e721dffe8f34 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:07.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.170+0000 7fc65531a640 1 -- 172.21.15.105:0/3814021345 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6400a7cb0 con 0x7fc6500ce170 2023-11-28T18:29:07.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.170+0000 7fc65531a640 1 -- 172.21.15.105:0/3814021345 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6400a7e10 con 0x7fc6500ce170 2023-11-28T18:29:07.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.170+0000 7fc65531a640 1 -- 172.21.15.105:0/3814021345 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6400af660 con 0x7fc6500ce170 2023-11-28T18:29:07.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.174+0000 7fc65731e640 1 -- 172.21.15.105:0/3814021345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6500ce170 msgr2=0x7fc65015d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:07.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.174+0000 7fc65731e640 1 --2- 172.21.15.105:0/3814021345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6500ce170 0x7fc65015d940 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fc64000bb10 tx=0x7fc64001ebe0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.174+0000 7fc65731e640 1 -- 172.21.15.105:0/3814021345 shutdown_connections 2023-11-28T18:29:07.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.174+0000 7fc65731e640 1 --2- 172.21.15.105:0/3814021345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6500ce170 0x7fc65015d940 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.174+0000 7fc65731e640 1 --2- 172.21.15.105:0/3814021345 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc650159710 0x7fc65006c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.174+0000 7fc65731e640 1 -- 172.21.15.105:0/3814021345 >> 172.21.15.105:0/3814021345 conn(0x7fc6500ce9e0 msgr2=0x7fc6500cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:07.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.174+0000 7fc65731e640 1 -- 172.21.15.105:0/3814021345 shutdown_connections 2023-11-28T18:29:07.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.174+0000 7fc65731e640 1 -- 172.21.15.105:0/3814021345 wait complete. 2023-11-28T18:29:07.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.174+0000 7fc65731e640 1 Processor -- start 2023-11-28T18:29:07.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.174+0000 7fc65731e640 1 -- start start 2023-11-28T18:29:07.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.174+0000 7fc65731e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6500cd170 0x7fc6501552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:07.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.174+0000 7fc65731e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6500ce170 0x7fc65014f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:07.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.174+0000 7fc65731e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc650159710 0x7fc650147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:07.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.174+0000 7fc65731e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65014e4e0 con 0x7fc6500ce170 2023-11-28T18:29:07.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.174+0000 7fc65731e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65014e900 con 0x7fc6500cd170 2023-11-28T18:29:07.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.174+0000 7fc65731e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65014d860 con 0x7fc650159710 2023-11-28T18:29:07.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.174+0000 7fc655b1b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6500ce170 0x7fc65014f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:07.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.174+0000 7fc655b1b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6500ce170 0x7fc65014f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57172/0 (socket says 172.21.15.105:57172) 2023-11-28T18:29:07.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.174+0000 7fc655b1b640 1 -- 172.21.15.105:0/483264774 learned_addr learned my addr 172.21.15.105:0/483264774 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:07.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.174+0000 7fc65631c640 1 --2- 172.21.15.105:0/483264774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6500cd170 0x7fc6501552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:07.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.174+0000 7fc656b1d640 1 --2- 172.21.15.105:0/483264774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc650159710 0x7fc650147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:07.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.178+0000 7fc655b1b640 1 -- 172.21.15.105:0/483264774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc650159710 msgr2=0x7fc650147540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:07.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.178+0000 7fc655b1b640 1 --2- 172.21.15.105:0/483264774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc650159710 0x7fc650147540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.178+0000 7fc655b1b640 1 -- 172.21.15.105:0/483264774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6500cd170 msgr2=0x7fc6501552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:07.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.178+0000 7fc655b1b640 1 --2- 172.21.15.105:0/483264774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6500cd170 0x7fc6501552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.178+0000 7fc655b1b640 1 -- 172.21.15.105:0/483264774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64000b770 con 0x7fc6500ce170 2023-11-28T18:29:07.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.178+0000 7fc65631c640 1 --2- 172.21.15.105:0/483264774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6500cd170 0x7fc6501552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:07.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.178+0000 7fc655b1b640 1 --2- 172.21.15.105:0/483264774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6500ce170 0x7fc65014f790 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fc64000b2c0 tx=0x7fc64001b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:07.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.178+0000 7fc646ffd640 1 -- 172.21.15.105:0/483264774 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6400afd70 con 0x7fc6500ce170 2023-11-28T18:29:07.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.178+0000 7fc646ffd640 1 -- 172.21.15.105:0/483264774 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6400ba410 con 0x7fc6500ce170 2023-11-28T18:29:07.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.178+0000 7fc646ffd640 1 -- 172.21.15.105:0/483264774 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6400ba7c0 con 0x7fc6500ce170 2023-11-28T18:29:07.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.178+0000 7fc65731e640 1 -- 172.21.15.105:0/483264774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65015e000 con 0x7fc6500ce170 2023-11-28T18:29:07.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.178+0000 7fc65731e640 1 -- 172.21.15.105:0/483264774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65014de20 con 0x7fc6500ce170 2023-11-28T18:29:07.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.178+0000 7fc65731e640 1 -- 172.21.15.105:0/483264774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc650067e30 con 0x7fc6500ce170 2023-11-28T18:29:07.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.178+0000 7fc646ffd640 1 -- 172.21.15.105:0/483264774 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc6400c3450 con 0x7fc6500ce170 2023-11-28T18:29:07.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.178+0000 7fc646ffd640 1 --2- 172.21.15.105:0/483264774 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6340668d0 0x7fc634068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:07.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.178+0000 7fc646ffd640 1 -- 172.21.15.105:0/483264774 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fc64011ebd0 con 0x7fc6500ce170 2023-11-28T18:29:07.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.178+0000 7fc65631c640 1 --2- 172.21.15.105:0/483264774 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6340668d0 0x7fc634068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:07.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.182+0000 7fc65631c640 1 --2- 172.21.15.105:0/483264774 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6340668d0 0x7fc634068d90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009bb0 tx=0x7fc64c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:07.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.182+0000 7fc646ffd640 1 -- 172.21.15.105:0/483264774 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc650067e30 con 0x7fc6500ce170 2023-11-28T18:29:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.334+0000 7fc65731e640 1 -- 172.21.15.105:0/483264774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc650067e30 con 0x7fc6500ce170 2023-11-28T18:29:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.334+0000 7fc646ffd640 1 -- 172.21.15.105:0/483264774 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc650067e30 con 0x7fc6500ce170 2023-11-28T18:29:07.338 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:29:07.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.334+0000 7fc65731e640 1 -- 172.21.15.105:0/483264774 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6340668d0 msgr2=0x7fc634068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:07.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.334+0000 7fc65731e640 1 --2- 172.21.15.105:0/483264774 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6340668d0 0x7fc634068d90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009bb0 tx=0x7fc64c008040 comp rx=0 tx=0).stop 2023-11-28T18:29:07.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.334+0000 7fc65731e640 1 -- 172.21.15.105:0/483264774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6500ce170 msgr2=0x7fc65014f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:07.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.334+0000 7fc65731e640 1 --2- 172.21.15.105:0/483264774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6500ce170 0x7fc65014f790 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fc64000b2c0 tx=0x7fc64001b980 comp rx=0 tx=0).stop 2023-11-28T18:29:07.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.334+0000 7fc65731e640 1 -- 172.21.15.105:0/483264774 shutdown_connections 2023-11-28T18:29:07.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.334+0000 7fc65731e640 1 --2- 172.21.15.105:0/483264774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc650159710 0x7fc650147540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.334+0000 7fc65731e640 1 --2- 172.21.15.105:0/483264774 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6340668d0 0x7fc634068d90 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.334+0000 7fc65731e640 1 --2- 172.21.15.105:0/483264774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6500ce170 0x7fc65014f790 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.334+0000 7fc65731e640 1 --2- 172.21.15.105:0/483264774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6500cd170 0x7fc6501552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.334+0000 7fc65731e640 1 -- 172.21.15.105:0/483264774 >> 172.21.15.105:0/483264774 conn(0x7fc6500ce9e0 msgr2=0x7fc65006fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.334+0000 7fc65731e640 1 -- 172.21.15.105:0/483264774 shutdown_connections 2023-11-28T18:29:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.334+0000 7fc65731e640 1 -- 172.21.15.105:0/483264774 wait complete. 2023-11-28T18:29:07.360 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T18:29:07.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.494+0000 7f8521032640 1 Processor -- start 2023-11-28T18:29:07.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.494+0000 7f8521032640 1 -- start start 2023-11-28T18:29:07.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.494+0000 7f8521032640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f851c15d4f0 0x7f851c0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:07.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.494+0000 7f8521032640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f851c0b2d20 0x7f851c0b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:07.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.494+0000 7f8521032640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851c0b25b0 con 0x7f851c0d3c30 2023-11-28T18:29:07.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.494+0000 7f8521032640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f851c0b2900 con 0x7f851c0b2d20 2023-11-28T18:29:07.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.494+0000 7f8521032640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f851c0d0a20 con 0x7f851c15d4f0 2023-11-28T18:29:07.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.494+0000 7f851b59e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f851c0b2d20 0x7f851c0b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:07.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.494+0000 7f851a59c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f851c0d3c30 0x7f851c0d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:38500/0 (socket says 172.21.15.105:38500) 2023-11-28T18:29:07.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.494+0000 7f851a59c640 1 -- 172.21.15.105:0/4254684879 learned_addr learned my addr 172.21.15.105:0/4254684879 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:07.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.494+0000 7f851ad9d640 1 --2- 172.21.15.105:0/4254684879 >> v2:172.21.15.148:3300/0 conn(0x7f851c15d4f0 0x7f851c0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:07.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.494+0000 7f851b59e640 1 -- 172.21.15.105:0/4254684879 >> v2:172.21.15.148:3300/0 conn(0x7f851c15d4f0 msgr2=0x7f851c0d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:07.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.494+0000 7f851b59e640 1 --2- 172.21.15.105:0/4254684879 >> v2:172.21.15.148:3300/0 conn(0x7f851c15d4f0 0x7f851c0d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.494+0000 7f851b59e640 1 -- 172.21.15.105:0/4254684879 >> v1:172.21.15.105:6789/0 conn(0x7f851c0d3c30 legacy=0x7f851c0d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:07.501 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.494+0000 7f851b59e640 1 -- 172.21.15.105:0/4254684879 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c0cea30 con 0x7f851c0b2d20 2023-11-28T18:29:07.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.494+0000 7f851ad9d640 1 --2- 172.21.15.105:0/4254684879 >> v2:172.21.15.148:3300/0 conn(0x7f851c15d4f0 0x7f851c0d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:07.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.498+0000 7f851b59e640 1 --2- 172.21.15.105:0/4254684879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f851c0b2d20 0x7f851c0b3150 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f851c08bbf0 tx=0x7f851401c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f28bab5b95230c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:07.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.498+0000 7f8519d9b640 1 -- 172.21.15.105:0/4254684879 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85140a5910 con 0x7f851c0b2d20 2023-11-28T18:29:07.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.498+0000 7f8519d9b640 1 -- 172.21.15.105:0/4254684879 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85140a5a70 con 0x7f851c0b2d20 2023-11-28T18:29:07.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.498+0000 7f8519d9b640 1 -- 172.21.15.105:0/4254684879 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85140a5e00 con 0x7f851c0b2d20 2023-11-28T18:29:07.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.498+0000 7f8521032640 1 -- 172.21.15.105:0/4254684879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f851c0b2d20 msgr2=0x7f851c0b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:07.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.498+0000 7f8521032640 1 --2- 172.21.15.105:0/4254684879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f851c0b2d20 0x7f851c0b3150 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f851c08bbf0 tx=0x7f851401c850 comp rx=0 tx=0).stop 2023-11-28T18:29:07.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.498+0000 7f8521032640 1 -- 172.21.15.105:0/4254684879 shutdown_connections 2023-11-28T18:29:07.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.498+0000 7f8521032640 1 --2- 172.21.15.105:0/4254684879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f851c0b2d20 0x7f851c0b3150 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.498+0000 7f8521032640 1 --2- 172.21.15.105:0/4254684879 >> v2:172.21.15.148:3300/0 conn(0x7f851c15d4f0 0x7f851c0d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.498+0000 7f8521032640 1 -- 172.21.15.105:0/4254684879 >> 172.21.15.105:0/4254684879 conn(0x7f851c0d3030 msgr2=0x7f851c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:07.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.498+0000 7f8521032640 1 -- 172.21.15.105:0/4254684879 shutdown_connections 2023-11-28T18:29:07.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.498+0000 7f8521032640 1 -- 172.21.15.105:0/4254684879 wait complete. 2023-11-28T18:29:07.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f8521032640 1 Processor -- start 2023-11-28T18:29:07.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f8521032640 1 -- start start 2023-11-28T18:29:07.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f8521032640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f851c0b2d20 0x7f851c0cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:07.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f8521032640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851c0d3c30 0x7f851c0c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:07.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f8521032640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f851c15d4f0 0x7f851c0ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:07.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f8521032640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c155400 con 0x7f851c0b2d20 2023-11-28T18:29:07.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f8521032640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c1556e0 con 0x7f851c0d3c30 2023-11-28T18:29:07.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f8521032640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851c0b7900 con 0x7f851c15d4f0 2023-11-28T18:29:07.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f851ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f851c0b2d20 0x7f851c0cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:07.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f851ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f851c0b2d20 0x7f851c0cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57196/0 (socket says 172.21.15.105:57196) 2023-11-28T18:29:07.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f851ad9d640 1 -- 172.21.15.105:0/1648982034 learned_addr learned my addr 172.21.15.105:0/1648982034 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:07.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f851a59c640 1 --2- 172.21.15.105:0/1648982034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851c0d3c30 0x7f851c0c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:07.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f851b59e640 1 --2- 172.21.15.105:0/1648982034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f851c15d4f0 0x7f851c0ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:07.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f851ad9d640 1 -- 172.21.15.105:0/1648982034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f851c15d4f0 msgr2=0x7f851c0ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:07.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f851ad9d640 1 --2- 172.21.15.105:0/1648982034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f851c15d4f0 0x7f851c0ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f851ad9d640 1 -- 172.21.15.105:0/1648982034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851c0d3c30 msgr2=0x7f851c0c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:07.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f851ad9d640 1 --2- 172.21.15.105:0/1648982034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851c0d3c30 0x7f851c0c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f851ad9d640 1 -- 172.21.15.105:0/1648982034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8514009a70 con 0x7f851c0b2d20 2023-11-28T18:29:07.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f851ad9d640 1 --2- 172.21.15.105:0/1648982034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f851c0b2d20 0x7f851c0cc930 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f850c00e810 tx=0x7f850c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:07.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f85037fe640 1 -- 172.21.15.105:0/1648982034 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c00c8d0 con 0x7f851c0b2d20 2023-11-28T18:29:07.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f85037fe640 1 -- 172.21.15.105:0/1648982034 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850c00ca30 con 0x7f851c0b2d20 2023-11-28T18:29:07.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f8521032640 1 -- 172.21.15.105:0/1648982034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851c0b7be0 con 0x7f851c0b2d20 2023-11-28T18:29:07.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f85037fe640 1 -- 172.21.15.105:0/1648982034 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c00cdc0 con 0x7f851c0b2d20 2023-11-28T18:29:07.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f8521032640 1 -- 172.21.15.105:0/1648982034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f851c0b7520 con 0x7f851c0b2d20 2023-11-28T18:29:07.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.502+0000 7f8521032640 1 -- 172.21.15.105:0/1648982034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f851c067570 con 0x7f851c0b2d20 2023-11-28T18:29:07.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.506+0000 7f85037fe640 1 -- 172.21.15.105:0/1648982034 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f850c013410 con 0x7f851c0b2d20 2023-11-28T18:29:07.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.506+0000 7f85037fe640 1 --2- 172.21.15.105:0/1648982034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f85080668d0 0x7f8508068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:07.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.506+0000 7f85037fe640 1 -- 172.21.15.105:0/1648982034 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f850c0840d0 con 0x7f851c0b2d20 2023-11-28T18:29:07.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.506+0000 7f851a59c640 1 --2- 172.21.15.105:0/1648982034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f85080668d0 0x7f8508068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:07.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.506+0000 7f851a59c640 1 --2- 172.21.15.105:0/1648982034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f85080668d0 0x7f8508068d90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f851c156960 tx=0x7f8510009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:07.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.510+0000 7f85037fe640 1 -- 172.21.15.105:0/1648982034 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f850c051480 con 0x7f851c0b2d20 2023-11-28T18:29:07.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.662+0000 7f8521032640 1 -- 172.21.15.105:0/1648982034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f851c158f50 con 0x7f851c0b2d20 2023-11-28T18:29:07.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.662+0000 7f85037fe640 1 -- 172.21.15.105:0/1648982034 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v86) v1 ==== 84+0+65 (secure 0 0 0) 0x7f850c054aa0 con 0x7f851c0b2d20 2023-11-28T18:29:07.666 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:29:07.666 INFO:teuthology.orchestra.run.smithi105.stdout:[ 2023-11-28T18:29:07.666 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:29:07.666 INFO:teuthology.orchestra.run.smithi105.stdout: "poolnum": 1, 2023-11-28T18:29:07.667 INFO:teuthology.orchestra.run.smithi105.stdout: "poolname": ".mgr" 2023-11-28T18:29:07.667 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:29:07.667 INFO:teuthology.orchestra.run.smithi105.stdout:] 2023-11-28T18:29:07.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.662+0000 7f8521032640 1 -- 172.21.15.105:0/1648982034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f85080668d0 msgr2=0x7f8508068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:07.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.662+0000 7f8521032640 1 --2- 172.21.15.105:0/1648982034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f85080668d0 0x7f8508068d90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f851c156960 tx=0x7f8510009040 comp rx=0 tx=0).stop 2023-11-28T18:29:07.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.662+0000 7f8521032640 1 -- 172.21.15.105:0/1648982034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f851c0b2d20 msgr2=0x7f851c0cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:07.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.662+0000 7f8521032640 1 --2- 172.21.15.105:0/1648982034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f851c0b2d20 0x7f851c0cc930 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f850c00e810 tx=0x7f850c00ecd0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.662+0000 7f8521032640 1 -- 172.21.15.105:0/1648982034 shutdown_connections 2023-11-28T18:29:07.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.662+0000 7f8521032640 1 --2- 172.21.15.105:0/1648982034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f851c15d4f0 0x7f851c0ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.662+0000 7f8521032640 1 --2- 172.21.15.105:0/1648982034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851c0d3c30 0x7f851c0c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.662+0000 7f8521032640 1 --2- 172.21.15.105:0/1648982034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f85080668d0 0x7f8508068d90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.662+0000 7f8521032640 1 --2- 172.21.15.105:0/1648982034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f851c0b2d20 0x7f851c0cc930 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.662+0000 7f8521032640 1 -- 172.21.15.105:0/1648982034 >> 172.21.15.105:0/1648982034 conn(0x7f851c0d3030 msgr2=0x7f851c159300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:07.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.662+0000 7f8521032640 1 -- 172.21.15.105:0/1648982034 shutdown_connections 2023-11-28T18:29:07.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.662+0000 7f8521032640 1 -- 172.21.15.105:0/1648982034 wait complete. 2023-11-28T18:29:07.688 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T18:29:07.688 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T18:29:07.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a33dd8640 1 Processor -- start 2023-11-28T18:29:07.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a33dd8640 1 -- start start 2023-11-28T18:29:07.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a33dd8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a2c0caf60 0x7f9a2c0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:07.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a33dd8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9a2c0ad850 0x7f9a2c0701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:07.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a33dd8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a2c06bf50 con 0x7f9a2c159f40 2023-11-28T18:29:07.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a33dd8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a2c0cb3c0 con 0x7f9a2c0ad850 2023-11-28T18:29:07.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a33dd8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c15a320 con 0x7f9a2c0caf60 2023-11-28T18:29:07.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a31b74640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9a2c159f40 0x7f9a2c0ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:38504/0 (socket says 172.21.15.105:38504) 2023-11-28T18:29:07.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a31b74640 1 -- 172.21.15.105:0/2337508885 learned_addr learned my addr 172.21.15.105:0/2337508885 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:07.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a32375640 1 --2- 172.21.15.105:0/2337508885 >> v2:172.21.15.148:3300/0 conn(0x7f9a2c0ad850 0x7f9a2c0701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:07.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a31373640 1 --2- 172.21.15.105:0/2337508885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a2c0caf60 0x7f9a2c0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:07.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a30b72640 1 -- 172.21.15.105:0/2337508885 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971470928 0 0) 0x7f9a2c06bf50 con 0x7f9a2c159f40 2023-11-28T18:29:07.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a30b72640 1 -- 172.21.15.105:0/2337508885 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a18003530 con 0x7f9a2c159f40 2023-11-28T18:29:07.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a30b72640 1 -- 172.21.15.105:0/2337508885 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2789841665 0 0) 0x7f9a18003530 con 0x7f9a2c159f40 2023-11-28T18:29:07.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a30b72640 1 -- 172.21.15.105:0/2337508885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a2c0caf60 msgr2=0x7f9a2c0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:07.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a30b72640 1 --2- 172.21.15.105:0/2337508885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a2c0caf60 0x7f9a2c0cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a30b72640 1 -- 172.21.15.105:0/2337508885 >> v2:172.21.15.148:3300/0 conn(0x7f9a2c0ad850 msgr2=0x7f9a2c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:07.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a30b72640 1 --2- 172.21.15.105:0/2337508885 >> v2:172.21.15.148:3300/0 conn(0x7f9a2c0ad850 0x7f9a2c0701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a30b72640 1 -- 172.21.15.105:0/2337508885 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2c071380 con 0x7f9a2c159f40 2023-11-28T18:29:07.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a30b72640 1 -- 172.21.15.105:0/2337508885 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f9a28001680 con 0x7f9a2c159f40 2023-11-28T18:29:07.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a30b72640 1 -- 172.21.15.105:0/2337508885 >> v1:172.21.15.105:6789/0 conn(0x7f9a2c159f40 legacy=0x7f9a2c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:07.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a30b72640 1 --2- 172.21.15.105:0/2337508885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a2c0caf60 0x7f9a18003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:07.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a30b72640 1 --2- 172.21.15.105:0/2337508885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a18003f00 0x7f9a180047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:07.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a30b72640 1 --2- 172.21.15.105:0/2337508885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a18004d10 0x7f9a18005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:07.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a30b72640 1 -- 172.21.15.105:0/2337508885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c0cb3c0 con 0x7f9a18004d10 2023-11-28T18:29:07.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a30b72640 1 -- 172.21.15.105:0/2337508885 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c15a320 con 0x7f9a18003f00 2023-11-28T18:29:07.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a30b72640 1 -- 172.21.15.105:0/2337508885 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1800dbf0 con 0x7f9a2c0caf60 2023-11-28T18:29:07.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a32375640 1 --2- 172.21.15.105:0/2337508885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a18004d10 0x7f9a18005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:07.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a31b74640 1 --2- 172.21.15.105:0/2337508885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a2c0caf60 0x7f9a18003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:07.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a31373640 1 --2- 172.21.15.105:0/2337508885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a18003f00 0x7f9a180047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:07.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a32375640 1 -- 172.21.15.105:0/2337508885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a2c0caf60 msgr2=0x7f9a18003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:07.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a32375640 1 --2- 172.21.15.105:0/2337508885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a2c0caf60 0x7f9a18003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a32375640 1 -- 172.21.15.105:0/2337508885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a18003f00 msgr2=0x7f9a180047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:07.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a32375640 1 --2- 172.21.15.105:0/2337508885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a18003f00 0x7f9a180047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.826+0000 7f9a32375640 1 -- 172.21.15.105:0/2337508885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a18003530 con 0x7f9a18004d10 2023-11-28T18:29:07.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.830+0000 7f9a32375640 1 --2- 172.21.15.105:0/2337508885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a18004d10 0x7f9a18005190 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400e170 tx=0x7f9a2401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2755d8baccb8c10e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:07.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.830+0000 7f9a30b72640 1 -- 172.21.15.105:0/2337508885 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a240a5c90 con 0x7f9a18004d10 2023-11-28T18:29:07.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.830+0000 7f9a30b72640 1 -- 172.21.15.105:0/2337508885 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2400f040 con 0x7f9a18004d10 2023-11-28T18:29:07.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.830+0000 7f9a30b72640 1 -- 172.21.15.105:0/2337508885 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a240ad440 con 0x7f9a18004d10 2023-11-28T18:29:07.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.830+0000 7f9a33dd8640 1 -- 172.21.15.105:0/2337508885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a18004d10 msgr2=0x7f9a18005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:07.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.830+0000 7f9a33dd8640 1 --2- 172.21.15.105:0/2337508885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a18004d10 0x7f9a18005190 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400e170 tx=0x7f9a2401cc80 comp rx=0 tx=0).stop 2023-11-28T18:29:07.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.830+0000 7f9a32375640 1 -- 172.21.15.105:0/2337508885 reap_dead start 2023-11-28T18:29:07.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.830+0000 7f9a33dd8640 1 -- 172.21.15.105:0/2337508885 shutdown_connections 2023-11-28T18:29:07.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.830+0000 7f9a33dd8640 1 -- 172.21.15.105:0/2337508885 >> 172.21.15.105:0/2337508885 conn(0x7f9a2c0ceb90 msgr2=0x7f9a2c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:07.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.830+0000 7f9a33dd8640 1 -- 172.21.15.105:0/2337508885 shutdown_connections 2023-11-28T18:29:07.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.830+0000 7f9a33dd8640 1 -- 172.21.15.105:0/2337508885 wait complete. 2023-11-28T18:29:07.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.834+0000 7f9a33dd8640 1 Processor -- start 2023-11-28T18:29:07.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.834+0000 7f9a33dd8640 1 -- start start 2023-11-28T18:29:07.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.834+0000 7f9a33dd8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a18004d10 0x7f9a2c151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:07.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.834+0000 7f9a33dd8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a2c1590a0 0x7f9a2c09c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:07.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.834+0000 7f9a33dd8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a2c09c5f0 0x7f9a2c154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:07.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.834+0000 7f9a33dd8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c09cb00 con 0x7f9a2c09c5f0 2023-11-28T18:29:07.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.834+0000 7f9a33dd8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c151e90 con 0x7f9a18004d10 2023-11-28T18:29:07.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.834+0000 7f9a33dd8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c1f71f0 con 0x7f9a2c1590a0 2023-11-28T18:29:07.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.834+0000 7f9a32375640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a2c09c5f0 0x7f9a2c154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:07.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.834+0000 7f9a32375640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a2c09c5f0 0x7f9a2c154a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57212/0 (socket says 172.21.15.105:57212) 2023-11-28T18:29:07.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.834+0000 7f9a32375640 1 -- 172.21.15.105:0/1921870345 learned_addr learned my addr 172.21.15.105:0/1921870345 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:07.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.834+0000 7f9a32375640 1 -- 172.21.15.105:0/1921870345 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a2c1590a0 msgr2=0x7f9a2c09c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:07.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.834+0000 7f9a32375640 1 --2- 172.21.15.105:0/1921870345 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a2c1590a0 0x7f9a2c09c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.834+0000 7f9a32375640 1 -- 172.21.15.105:0/1921870345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a18004d10 msgr2=0x7f9a2c151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:07.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.834+0000 7f9a32375640 1 --2- 172.21.15.105:0/1921870345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a18004d10 0x7f9a2c151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:07.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.834+0000 7f9a32375640 1 -- 172.21.15.105:0/1921870345 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2c1f6430 con 0x7f9a2c09c5f0 2023-11-28T18:29:07.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.834+0000 7f9a32375640 1 --2- 172.21.15.105:0/1921870345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a2c09c5f0 0x7f9a2c154a50 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400e170 tx=0x7f9a24009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:07.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.834+0000 7f9a227fc640 1 -- 172.21.15.105:0/1921870345 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a240039a0 con 0x7f9a2c09c5f0 2023-11-28T18:29:07.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.834+0000 7f9a227fc640 1 -- 172.21.15.105:0/1921870345 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a24003b00 con 0x7f9a2c09c5f0 2023-11-28T18:29:07.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.834+0000 7f9a227fc640 1 -- 172.21.15.105:0/1921870345 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a24003e90 con 0x7f9a2c09c5f0 2023-11-28T18:29:07.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.834+0000 7f9a33dd8640 1 -- 172.21.15.105:0/1921870345 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a2c1f6710 con 0x7f9a2c09c5f0 2023-11-28T18:29:07.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.834+0000 7f9a33dd8640 1 -- 172.21.15.105:0/1921870345 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2c1f6cd0 con 0x7f9a2c09c5f0 2023-11-28T18:29:07.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.838+0000 7f9a33dd8640 1 -- 172.21.15.105:0/1921870345 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a2c1f6ec0 con 0x7f9a2c09c5f0 2023-11-28T18:29:07.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.838+0000 7f9a227fc640 1 -- 172.21.15.105:0/1921870345 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f9a240b4040 con 0x7f9a2c09c5f0 2023-11-28T18:29:07.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.838+0000 7f9a227fc640 1 --2- 172.21.15.105:0/1921870345 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9a100668d0 0x7f9a10068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:07.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.838+0000 7f9a227fc640 1 -- 172.21.15.105:0/1921870345 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9a2411d8a0 con 0x7f9a2c09c5f0 2023-11-28T18:29:07.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.838+0000 7f9a31b74640 1 --2- 172.21.15.105:0/1921870345 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9a100668d0 0x7f9a10068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:07.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.838+0000 7f9a31b74640 1 --2- 172.21.15.105:0/1921870345 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9a100668d0 0x7f9a10068d90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9a28001060 tx=0x7f9a28000bd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:07.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.842+0000 7f9a227fc640 1 -- 172.21.15.105:0/1921870345 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9a240ead00 con 0x7f9a2c09c5f0 2023-11-28T18:29:08.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:07.998+0000 7f9a33dd8640 1 -- 172.21.15.105:0/1921870345 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9a2c067570 con 0x7f9a2c09c5f0 2023-11-28T18:29:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.310+0000 7f9a227fc640 1 -- 172.21.15.105:0/1921870345 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v87) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9a240b20b0 con 0x7f9a2c09c5f0 2023-11-28T18:29:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' created 2023-11-28T18:29:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.314+0000 7f9a33dd8640 1 -- 172.21.15.105:0/1921870345 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9a100668d0 msgr2=0x7f9a10068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.314+0000 7f9a33dd8640 1 --2- 172.21.15.105:0/1921870345 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9a100668d0 0x7f9a10068d90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9a28001060 tx=0x7f9a28000bd0 comp rx=0 tx=0).stop 2023-11-28T18:29:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.314+0000 7f9a33dd8640 1 -- 172.21.15.105:0/1921870345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a2c09c5f0 msgr2=0x7f9a2c154a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.314+0000 7f9a33dd8640 1 --2- 172.21.15.105:0/1921870345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a2c09c5f0 0x7f9a2c154a50 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400e170 tx=0x7f9a24009750 comp rx=0 tx=0).stop 2023-11-28T18:29:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.314+0000 7f9a33dd8640 1 -- 172.21.15.105:0/1921870345 shutdown_connections 2023-11-28T18:29:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.314+0000 7f9a33dd8640 1 --2- 172.21.15.105:0/1921870345 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9a100668d0 0x7f9a10068d90 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.314+0000 7f9a33dd8640 1 --2- 172.21.15.105:0/1921870345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a2c09c5f0 0x7f9a2c154a50 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.314+0000 7f9a33dd8640 1 --2- 172.21.15.105:0/1921870345 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a2c1590a0 0x7f9a2c09c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.314+0000 7f9a33dd8640 1 --2- 172.21.15.105:0/1921870345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a18004d10 0x7f9a2c151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.314+0000 7f9a33dd8640 1 -- 172.21.15.105:0/1921870345 >> 172.21.15.105:0/1921870345 conn(0x7f9a2c0ceb90 msgr2=0x7f9a2c0aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.314+0000 7f9a33dd8640 1 -- 172.21.15.105:0/1921870345 shutdown_connections 2023-11-28T18:29:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.314+0000 7f9a33dd8640 1 -- 172.21.15.105:0/1921870345 wait complete. 2023-11-28T18:29:08.341 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T18:29:08.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.482+0000 7f4098a5a640 1 Processor -- start 2023-11-28T18:29:08.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.482+0000 7f4098a5a640 1 -- start start 2023-11-28T18:29:08.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.482+0000 7f4098a5a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f409406cef0 0x7f40940abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:08.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.482+0000 7f4098a5a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40940cde30 con 0x7f4094159610 2023-11-28T18:29:08.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.482+0000 7f4098a5a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40941599f0 con 0x7f40940ce2c0 2023-11-28T18:29:08.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.482+0000 7f4098a5a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409406de20 con 0x7f409406cef0 2023-11-28T18:29:08.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.482+0000 7f409259c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4094159610 0x7f40940cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:38510/0 (socket says 172.21.15.105:38510) 2023-11-28T18:29:08.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.482+0000 7f409259c640 1 -- 172.21.15.105:0/1876831572 learned_addr learned my addr 172.21.15.105:0/1876831572 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:08.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.482+0000 7f4092d9d640 1 --2- 172.21.15.105:0/1876831572 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f409406cef0 0x7f40940abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:08.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.482+0000 7f4091d9b640 1 -- 172.21.15.105:0/1876831572 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018404292 0 0) 0x7f40940cde30 con 0x7f4094159610 2023-11-28T18:29:08.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.482+0000 7f4091d9b640 1 -- 172.21.15.105:0/1876831572 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4080003450 con 0x7f4094159610 2023-11-28T18:29:08.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.486+0000 7f4091d9b640 1 -- 172.21.15.105:0/1876831572 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 832128297 0 0) 0x7f40941599f0 con 0x7f40940ce2c0 2023-11-28T18:29:08.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.486+0000 7f4091d9b640 1 -- 172.21.15.105:0/1876831572 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40940cde30 con 0x7f40940ce2c0 2023-11-28T18:29:08.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.486+0000 7f4092d9d640 1 -- 172.21.15.105:0/1876831572 >> v1:172.21.15.148:6789/0 conn(0x7f40940ce2c0 legacy=0x7f409406c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:08.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.486+0000 7f4092d9d640 1 -- 172.21.15.105:0/1876831572 >> v1:172.21.15.105:6789/0 conn(0x7f4094159610 legacy=0x7f40940cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:08.490 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.486+0000 7f4091d9b640 1 -- 172.21.15.105:0/1876831572 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3815296941 0 0) 0x7f4080003450 con 0x7f4094159610 2023-11-28T18:29:08.490 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.486+0000 7f4092d9d640 1 -- 172.21.15.105:0/1876831572 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40940cd220 con 0x7f409406cef0 2023-11-28T18:29:08.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.486+0000 7f4092d9d640 1 --2- 172.21.15.105:0/1876831572 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f409406cef0 0x7f40940abd80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f408c0097f0 tx=0x7f408c01c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70dc4c525abe035c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:08.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.486+0000 7f4091d9b640 1 -- 172.21.15.105:0/1876831572 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4084001a00 con 0x7f409406cef0 2023-11-28T18:29:08.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.486+0000 7f4091d9b640 1 -- 172.21.15.105:0/1876831572 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408c0a7050 con 0x7f409406cef0 2023-11-28T18:29:08.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.486+0000 7f4091d9b640 1 -- 172.21.15.105:0/1876831572 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c0ab600 con 0x7f409406cef0 2023-11-28T18:29:08.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.490+0000 7f4098a5a640 1 -- 172.21.15.105:0/1876831572 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f409406cef0 msgr2=0x7f40940abd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:08.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.490+0000 7f4098a5a640 1 --2- 172.21.15.105:0/1876831572 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f409406cef0 0x7f40940abd80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f408c0097f0 tx=0x7f408c01c3b0 comp rx=0 tx=0).stop 2023-11-28T18:29:08.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.490+0000 7f4098a5a640 1 -- 172.21.15.105:0/1876831572 shutdown_connections 2023-11-28T18:29:08.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.490+0000 7f4098a5a640 1 --2- 172.21.15.105:0/1876831572 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f409406cef0 0x7f40940abd80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:08.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.490+0000 7f4098a5a640 1 -- 172.21.15.105:0/1876831572 >> 172.21.15.105:0/1876831572 conn(0x7f40940ceb90 msgr2=0x7f40940cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:08.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.490+0000 7f4098a5a640 1 -- 172.21.15.105:0/1876831572 shutdown_connections 2023-11-28T18:29:08.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.490+0000 7f4098a5a640 1 -- 172.21.15.105:0/1876831572 wait complete. 2023-11-28T18:29:08.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.490+0000 7f4098a5a640 1 Processor -- start 2023-11-28T18:29:08.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.490+0000 7f4098a5a640 1 -- start start 2023-11-28T18:29:08.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.490+0000 7f4098a5a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409406cef0 0x7f40941f4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:08.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.490+0000 7f4098a5a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40940ce2c0 0x7f40940af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:08.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.490+0000 7f4098a5a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4094159610 0x7f4094145d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:08.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.490+0000 7f4098a5a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40941f1960 con 0x7f40940ce2c0 2023-11-28T18:29:08.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.490+0000 7f4098a5a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40941f5400 con 0x7f409406cef0 2023-11-28T18:29:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.490+0000 7f4098a5a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40941f56e0 con 0x7f4094159610 2023-11-28T18:29:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.490+0000 7f4089d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40940ce2c0 0x7f40940af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.490+0000 7f4089d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40940ce2c0 0x7f40940af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57220/0 (socket says 172.21.15.105:57220) 2023-11-28T18:29:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.490+0000 7f4089d9b640 1 -- 172.21.15.105:0/804675871 learned_addr learned my addr 172.21.15.105:0/804675871 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.490+0000 7f4092d9d640 1 --2- 172.21.15.105:0/804675871 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4094159610 0x7f4094145d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.490+0000 7f409259c640 1 --2- 172.21.15.105:0/804675871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409406cef0 0x7f40941f4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.490+0000 7f4089d9b640 1 -- 172.21.15.105:0/804675871 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4094159610 msgr2=0x7f4094145d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.490+0000 7f4089d9b640 1 --2- 172.21.15.105:0/804675871 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4094159610 0x7f4094145d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.494+0000 7f4089d9b640 1 -- 172.21.15.105:0/804675871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409406cef0 msgr2=0x7f40941f4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.494+0000 7f4089d9b640 1 --2- 172.21.15.105:0/804675871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409406cef0 0x7f40941f4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.494+0000 7f4089d9b640 1 -- 172.21.15.105:0/804675871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408c009420 con 0x7f40940ce2c0 2023-11-28T18:29:08.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.494+0000 7f409259c640 1 --2- 172.21.15.105:0/804675871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409406cef0 0x7f40941f4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:08.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.494+0000 7f4089d9b640 1 --2- 172.21.15.105:0/804675871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40940ce2c0 0x7f40940af8b0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f407c00c490 tx=0x7f407c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:08.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.494+0000 7f408b7fe640 1 -- 172.21.15.105:0/804675871 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c0052d0 con 0x7f40940ce2c0 2023-11-28T18:29:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.494+0000 7f408b7fe640 1 -- 172.21.15.105:0/804675871 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407c005430 con 0x7f40940ce2c0 2023-11-28T18:29:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.494+0000 7f408b7fe640 1 -- 172.21.15.105:0/804675871 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c012870 con 0x7f40940ce2c0 2023-11-28T18:29:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.494+0000 7f4098a5a640 1 -- 172.21.15.105:0/804675871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40940ccff0 con 0x7f40940ce2c0 2023-11-28T18:29:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.494+0000 7f4098a5a640 1 -- 172.21.15.105:0/804675871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40940afdf0 con 0x7f40940ce2c0 2023-11-28T18:29:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.494+0000 7f4098a5a640 1 -- 172.21.15.105:0/804675871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40941f1b10 con 0x7f40940ce2c0 2023-11-28T18:29:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.494+0000 7f408b7fe640 1 -- 172.21.15.105:0/804675871 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f407c0129d0 con 0x7f40940ce2c0 2023-11-28T18:29:08.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.494+0000 7f408b7fe640 1 --2- 172.21.15.105:0/804675871 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4074066920 0x7f4074068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:08.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.494+0000 7f408b7fe640 1 -- 172.21.15.105:0/804675871 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f407c083ee0 con 0x7f40940ce2c0 2023-11-28T18:29:08.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.494+0000 7f409259c640 1 --2- 172.21.15.105:0/804675871 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4074066920 0x7f4074068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:08.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.498+0000 7f409259c640 1 --2- 172.21.15.105:0/804675871 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4074066920 0x7f4074068de0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4084001570 tx=0x7f4084009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:08.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.502+0000 7f408b7fe640 1 -- 172.21.15.105:0/804675871 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f407c0511a0 con 0x7f40940ce2c0 2023-11-28T18:29:08.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:08.650+0000 7f4098a5a640 1 -- 172.21.15.105:0/804675871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4094067570 con 0x7f40940ce2c0 2023-11-28T18:29:09.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.314+0000 7f408b7fe640 1 -- 172.21.15.105:0/804675871 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v88) v1 ==== 170+0+0 (secure 0 0 0) 0x7f407c0547c0 con 0x7f40940ce2c0 2023-11-28T18:29:09.318 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' created 2023-11-28T18:29:09.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.318+0000 7f4098a5a640 1 -- 172.21.15.105:0/804675871 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4074066920 msgr2=0x7f4074068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:09.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.318+0000 7f4098a5a640 1 --2- 172.21.15.105:0/804675871 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4074066920 0x7f4074068de0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4084001570 tx=0x7f4084009040 comp rx=0 tx=0).stop 2023-11-28T18:29:09.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.318+0000 7f4098a5a640 1 -- 172.21.15.105:0/804675871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40940ce2c0 msgr2=0x7f40940af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:09.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.318+0000 7f4098a5a640 1 --2- 172.21.15.105:0/804675871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40940ce2c0 0x7f40940af8b0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f407c00c490 tx=0x7f407c00c950 comp rx=0 tx=0).stop 2023-11-28T18:29:09.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.318+0000 7f4098a5a640 1 -- 172.21.15.105:0/804675871 shutdown_connections 2023-11-28T18:29:09.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.318+0000 7f4098a5a640 1 --2- 172.21.15.105:0/804675871 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4094159610 0x7f4094145d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:09.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.318+0000 7f4098a5a640 1 --2- 172.21.15.105:0/804675871 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4074066920 0x7f4074068de0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:09.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.318+0000 7f4098a5a640 1 --2- 172.21.15.105:0/804675871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40940ce2c0 0x7f40940af8b0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:09.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.318+0000 7f4098a5a640 1 --2- 172.21.15.105:0/804675871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409406cef0 0x7f40941f4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:09.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.318+0000 7f4098a5a640 1 -- 172.21.15.105:0/804675871 >> 172.21.15.105:0/804675871 conn(0x7f40940ceb90 msgr2=0x7f40941543b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:09.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.318+0000 7f4098a5a640 1 -- 172.21.15.105:0/804675871 shutdown_connections 2023-11-28T18:29:09.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.318+0000 7f4098a5a640 1 -- 172.21.15.105:0/804675871 wait complete. 2023-11-28T18:29:09.364 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T18:29:09.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.502+0000 7f3a67980640 1 Processor -- start 2023-11-28T18:29:09.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.502+0000 7f3a67980640 1 -- start start 2023-11-28T18:29:09.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.502+0000 7f3a67980640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3a600c7490 0x7f3a600cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:09.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.502+0000 7f3a67980640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3a6006c790 0x7f3a6006b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:09.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.502+0000 7f3a67980640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a6006bbd0 0x7f3a6006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:09.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.502+0000 7f3a67980640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a600c4a00 con 0x7f3a6006c790 2023-11-28T18:29:09.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.502+0000 7f3a67980640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a600d0400 con 0x7f3a600c7490 2023-11-28T18:29:09.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.502+0000 7f3a67980640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a600c7870 con 0x7f3a6006bbd0 2023-11-28T18:29:09.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.502+0000 7f3a64f1b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3a6006c790 0x7f3a6006b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:09.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.502+0000 7f3a64f1b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3a6006c790 0x7f3a6006b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57234/0 (socket says 172.21.15.105:57234) 2023-11-28T18:29:09.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.502+0000 7f3a64f1b640 1 -- 172.21.15.105:0/2120905881 learned_addr learned my addr 172.21.15.105:0/2120905881 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:09.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.502+0000 7f3a65f1d640 1 --2- 172.21.15.105:0/2120905881 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a6006bbd0 0x7f3a6006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:09.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.502+0000 7f3a6571c640 1 --2- 172.21.15.105:0/2120905881 >> v2:172.21.15.148:3300/0 conn(0x7f3a600c7490 0x7f3a600cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:09.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.502+0000 7f3a64f1b640 1 -- 172.21.15.105:0/2120905881 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a6006bbd0 msgr2=0x7f3a6006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:09.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.502+0000 7f3a64f1b640 1 --2- 172.21.15.105:0/2120905881 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a6006bbd0 0x7f3a6006c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:09.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.502+0000 7f3a64f1b640 1 -- 172.21.15.105:0/2120905881 >> v2:172.21.15.148:3300/0 conn(0x7f3a600c7490 msgr2=0x7f3a600cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:09.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.502+0000 7f3a64f1b640 1 --2- 172.21.15.105:0/2120905881 >> v2:172.21.15.148:3300/0 conn(0x7f3a600c7490 0x7f3a600cefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:09.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.502+0000 7f3a64f1b640 1 -- 172.21.15.105:0/2120905881 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a600d0120 con 0x7f3a6006c790 2023-11-28T18:29:09.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.506+0000 7f3a64f1b640 1 --2- 172.21.15.105:0/2120905881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a6006c790 0x7f3a6006b690 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f3a600713e0 tx=0x7f3a5001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc32e7c6735839e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:09.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.506+0000 7f3a57fff640 1 -- 172.21.15.105:0/2120905881 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a500aa5f0 con 0x7f3a6006c790 2023-11-28T18:29:09.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.506+0000 7f3a57fff640 1 -- 172.21.15.105:0/2120905881 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a500aa750 con 0x7f3a6006c790 2023-11-28T18:29:09.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.506+0000 7f3a57fff640 1 -- 172.21.15.105:0/2120905881 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a500aaae0 con 0x7f3a6006c790 2023-11-28T18:29:09.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.506+0000 7f3a67980640 1 -- 172.21.15.105:0/2120905881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a6006c790 msgr2=0x7f3a6006b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:09.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.506+0000 7f3a67980640 1 --2- 172.21.15.105:0/2120905881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a6006c790 0x7f3a6006b690 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f3a600713e0 tx=0x7f3a5001c4d0 comp rx=0 tx=0).stop 2023-11-28T18:29:09.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.506+0000 7f3a67980640 1 -- 172.21.15.105:0/2120905881 shutdown_connections 2023-11-28T18:29:09.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.506+0000 7f3a67980640 1 --2- 172.21.15.105:0/2120905881 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a6006bbd0 0x7f3a6006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:09.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.506+0000 7f3a67980640 1 --2- 172.21.15.105:0/2120905881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a6006c790 0x7f3a6006b690 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:09.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.506+0000 7f3a67980640 1 --2- 172.21.15.105:0/2120905881 >> v2:172.21.15.148:3300/0 conn(0x7f3a600c7490 0x7f3a600cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:09.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.506+0000 7f3a67980640 1 -- 172.21.15.105:0/2120905881 >> 172.21.15.105:0/2120905881 conn(0x7f3a600d0eb0 msgr2=0x7f3a600d12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:09.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.506+0000 7f3a67980640 1 -- 172.21.15.105:0/2120905881 shutdown_connections 2023-11-28T18:29:09.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.506+0000 7f3a67980640 1 -- 172.21.15.105:0/2120905881 wait complete. 2023-11-28T18:29:09.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a67980640 1 Processor -- start 2023-11-28T18:29:09.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a67980640 1 -- start start 2023-11-28T18:29:09.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a67980640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a6006bbd0 0x7f3a60147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:09.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a67980640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a6006c790 0x7f3a600ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:09.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a67980640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a600c7490 0x7f3a600ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:09.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a67980640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a600af290 con 0x7f3a600c7490 2023-11-28T18:29:09.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a67980640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a600af570 con 0x7f3a6006bbd0 2023-11-28T18:29:09.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a67980640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a600af850 con 0x7f3a6006c790 2023-11-28T18:29:09.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a65f1d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a600c7490 0x7f3a600ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:09.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a65f1d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a600c7490 0x7f3a600ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57250/0 (socket says 172.21.15.105:57250) 2023-11-28T18:29:09.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a65f1d640 1 -- 172.21.15.105:0/481607513 learned_addr learned my addr 172.21.15.105:0/481607513 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:09.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a6571c640 1 --2- 172.21.15.105:0/481607513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a6006bbd0 0x7f3a60147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:09.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a64f1b640 1 --2- 172.21.15.105:0/481607513 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a6006c790 0x7f3a600ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:09.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a65f1d640 1 -- 172.21.15.105:0/481607513 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a6006c790 msgr2=0x7f3a600ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:09.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a65f1d640 1 --2- 172.21.15.105:0/481607513 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a6006c790 0x7f3a600ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:09.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a65f1d640 1 -- 172.21.15.105:0/481607513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a6006bbd0 msgr2=0x7f3a60147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:09.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a65f1d640 1 --2- 172.21.15.105:0/481607513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a6006bbd0 0x7f3a60147230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:09.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a65f1d640 1 -- 172.21.15.105:0/481607513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a500094d0 con 0x7f3a600c7490 2023-11-28T18:29:09.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a65f1d640 1 --2- 172.21.15.105:0/481607513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a600c7490 0x7f3a600ca590 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800eed0 tx=0x7f3a5800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:09.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a55ffb640 1 -- 172.21.15.105:0/481607513 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58010040 con 0x7f3a600c7490 2023-11-28T18:29:09.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a55ffb640 1 -- 172.21.15.105:0/481607513 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a58014400 con 0x7f3a600c7490 2023-11-28T18:29:09.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a55ffb640 1 -- 172.21.15.105:0/481607513 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58009ea0 con 0x7f3a600c7490 2023-11-28T18:29:09.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.510+0000 7f3a67980640 1 -- 172.21.15.105:0/481607513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a600afb30 con 0x7f3a600c7490 2023-11-28T18:29:09.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.514+0000 7f3a67980640 1 -- 172.21.15.105:0/481607513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a600aef40 con 0x7f3a600c7490 2023-11-28T18:29:09.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.514+0000 7f3a67980640 1 -- 172.21.15.105:0/481607513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a60067570 con 0x7f3a600c7490 2023-11-28T18:29:09.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.514+0000 7f3a55ffb640 1 -- 172.21.15.105:0/481607513 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3a580147b0 con 0x7f3a600c7490 2023-11-28T18:29:09.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.514+0000 7f3a55ffb640 1 --2- 172.21.15.105:0/481607513 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3a440668d0 0x7f3a44068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:09.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.514+0000 7f3a55ffb640 1 -- 172.21.15.105:0/481607513 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f3a580852b0 con 0x7f3a600c7490 2023-11-28T18:29:09.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.518+0000 7f3a6571c640 1 --2- 172.21.15.105:0/481607513 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3a440668d0 0x7f3a44068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:09.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.518+0000 7f3a6571c640 1 --2- 172.21.15.105:0/481607513 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3a440668d0 0x7f3a44068d90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009ac0 tx=0x7f3a5c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:09.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.518+0000 7f3a55ffb640 1 -- 172.21.15.105:0/481607513 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3a580524a0 con 0x7f3a600c7490 2023-11-28T18:29:09.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:09.702+0000 7f3a67980640 1 -- 172.21.15.105:0/481607513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3a600affe0 con 0x7f3a600c7490 2023-11-28T18:29:10.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.338+0000 7f3a55ffb640 1 -- 172.21.15.105:0/481607513 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v83) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3a58055ac0 con 0x7f3a600c7490 2023-11-28T18:29:10.341 INFO:teuthology.orchestra.run.smithi105.stderr:new fs with metadata pool 13 and data pool 14 2023-11-28T18:29:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.338+0000 7f3a67980640 1 -- 172.21.15.105:0/481607513 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3a440668d0 msgr2=0x7f3a44068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.338+0000 7f3a67980640 1 --2- 172.21.15.105:0/481607513 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3a440668d0 0x7f3a44068d90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009ac0 tx=0x7f3a5c009190 comp rx=0 tx=0).stop 2023-11-28T18:29:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.338+0000 7f3a67980640 1 -- 172.21.15.105:0/481607513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a600c7490 msgr2=0x7f3a600ca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.338+0000 7f3a67980640 1 --2- 172.21.15.105:0/481607513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a600c7490 0x7f3a600ca590 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800eed0 tx=0x7f3a5800b480 comp rx=0 tx=0).stop 2023-11-28T18:29:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.338+0000 7f3a67980640 1 -- 172.21.15.105:0/481607513 shutdown_connections 2023-11-28T18:29:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.338+0000 7f3a67980640 1 --2- 172.21.15.105:0/481607513 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3a440668d0 0x7f3a44068d90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.338+0000 7f3a67980640 1 --2- 172.21.15.105:0/481607513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a600c7490 0x7f3a600ca590 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.338+0000 7f3a67980640 1 --2- 172.21.15.105:0/481607513 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a6006c790 0x7f3a600ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.338+0000 7f3a67980640 1 --2- 172.21.15.105:0/481607513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a6006bbd0 0x7f3a60147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.338+0000 7f3a67980640 1 -- 172.21.15.105:0/481607513 >> 172.21.15.105:0/481607513 conn(0x7f3a600d0eb0 msgr2=0x7f3a600cc040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.342+0000 7f3a67980640 1 -- 172.21.15.105:0/481607513 shutdown_connections 2023-11-28T18:29:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.342+0000 7f3a67980640 1 -- 172.21.15.105:0/481607513 wait complete. 2023-11-28T18:29:10.399 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:29:10.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.534+0000 7fe5c3a6b640 1 Processor -- start 2023-11-28T18:29:10.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.534+0000 7fe5c3a6b640 1 -- start start 2023-11-28T18:29:10.539 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.534+0000 7fe5c3a6b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe5bc159f40 0x7fe5bc0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:10.539 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.534+0000 7fe5c3a6b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5bc0ad850 0x7fe5bc0701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:10.539 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.534+0000 7fe5c3a6b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5bc06bf50 con 0x7fe5bc0cc760 2023-11-28T18:29:10.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.534+0000 7fe5c3a6b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5bc0ccbc0 con 0x7fe5bc159f40 2023-11-28T18:29:10.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.534+0000 7fe5c3a6b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc15a320 con 0x7fe5bc0ad850 2023-11-28T18:29:10.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.534+0000 7fe5c1807640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe5bc159f40 0x7fe5bc0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:10.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.534+0000 7fe5c1807640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe5bc159f40 0x7fe5bc0caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57254/0 (socket says 172.21.15.105:57254) 2023-11-28T18:29:10.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.534+0000 7fe5c1807640 1 -- 172.21.15.105:0/1818538077 learned_addr learned my addr 172.21.15.105:0/1818538077 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:10.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.534+0000 7fe5c2008640 1 --2- 172.21.15.105:0/1818538077 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5bc0ad850 0x7fe5bc0701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:10.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.534+0000 7fe5c1807640 1 -- 172.21.15.105:0/1818538077 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5bc0ad850 msgr2=0x7fe5bc0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:10.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.534+0000 7fe5c1807640 1 --2- 172.21.15.105:0/1818538077 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5bc0ad850 0x7fe5bc0701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:10.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.534+0000 7fe5c1807640 1 -- 172.21.15.105:0/1818538077 >> v1:172.21.15.148:6789/0 conn(0x7fe5bc0cc760 legacy=0x7fe5bc0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:10.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.534+0000 7fe5c1807640 1 -- 172.21.15.105:0/1818538077 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5bc071380 con 0x7fe5bc159f40 2023-11-28T18:29:10.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.538+0000 7fe5c1807640 1 --2- 172.21.15.105:0/1818538077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5bc159f40 0x7fe5bc0caf60 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009870 tx=0x7fe5b801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ee8e6971d6796e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:10.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.538+0000 7fe5c0805640 1 -- 172.21.15.105:0/1818538077 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b80ab680 con 0x7fe5bc159f40 2023-11-28T18:29:10.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.538+0000 7fe5c0805640 1 -- 172.21.15.105:0/1818538077 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b80ab7e0 con 0x7fe5bc159f40 2023-11-28T18:29:10.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.538+0000 7fe5c0805640 1 -- 172.21.15.105:0/1818538077 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b80abb70 con 0x7fe5bc159f40 2023-11-28T18:29:10.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.538+0000 7fe5c3a6b640 1 -- 172.21.15.105:0/1818538077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5bc159f40 msgr2=0x7fe5bc0caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:10.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.538+0000 7fe5c3a6b640 1 --2- 172.21.15.105:0/1818538077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5bc159f40 0x7fe5bc0caf60 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009870 tx=0x7fe5b801c480 comp rx=0 tx=0).stop 2023-11-28T18:29:10.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.538+0000 7fe5c3a6b640 1 -- 172.21.15.105:0/1818538077 shutdown_connections 2023-11-28T18:29:10.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.538+0000 7fe5c3a6b640 1 --2- 172.21.15.105:0/1818538077 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5bc0ad850 0x7fe5bc0701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:10.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.538+0000 7fe5c3a6b640 1 --2- 172.21.15.105:0/1818538077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5bc159f40 0x7fe5bc0caf60 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:10.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.538+0000 7fe5c3a6b640 1 -- 172.21.15.105:0/1818538077 >> 172.21.15.105:0/1818538077 conn(0x7fe5bc0ceb90 msgr2=0x7fe5bc0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:10.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.538+0000 7fe5c3a6b640 1 -- 172.21.15.105:0/1818538077 shutdown_connections 2023-11-28T18:29:10.544 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.538+0000 7fe5c3a6b640 1 -- 172.21.15.105:0/1818538077 wait complete. 2023-11-28T18:29:10.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5c3a6b640 1 Processor -- start 2023-11-28T18:29:10.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5c3a6b640 1 -- start start 2023-11-28T18:29:10.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5c3a6b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5bc0ad850 0x7fe5bc14d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:10.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5c3a6b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5bc0cc760 0x7fe5bc1560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:10.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5c3a6b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5bc159f40 0x7fe5bc156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:10.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5c3a6b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc15a650 con 0x7fe5bc0cc760 2023-11-28T18:29:10.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5c3a6b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc14de50 con 0x7fe5bc159f40 2023-11-28T18:29:10.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5c3a6b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc1638d0 con 0x7fe5bc0ad850 2023-11-28T18:29:10.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5c1006640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5bc0cc760 0x7fe5bc1560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:10.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5c1006640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5bc0cc760 0x7fe5bc1560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57266/0 (socket says 172.21.15.105:57266) 2023-11-28T18:29:10.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5c1006640 1 -- 172.21.15.105:0/132906077 learned_addr learned my addr 172.21.15.105:0/132906077 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:10.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5c1807640 1 --2- 172.21.15.105:0/132906077 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5bc0ad850 0x7fe5bc14d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:10.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5c2008640 1 --2- 172.21.15.105:0/132906077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5bc159f40 0x7fe5bc156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:10.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5c1006640 1 -- 172.21.15.105:0/132906077 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5bc0ad850 msgr2=0x7fe5bc14d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:10.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5c1006640 1 --2- 172.21.15.105:0/132906077 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5bc0ad850 0x7fe5bc14d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:10.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5c1006640 1 -- 172.21.15.105:0/132906077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5bc159f40 msgr2=0x7fe5bc156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:10.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5c1006640 1 --2- 172.21.15.105:0/132906077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5bc159f40 0x7fe5bc156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:10.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5c1006640 1 -- 172.21.15.105:0/132906077 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b80094d0 con 0x7fe5bc0cc760 2023-11-28T18:29:10.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5c2008640 1 --2- 172.21.15.105:0/132906077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5bc159f40 0x7fe5bc156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:10.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5c1006640 1 --2- 172.21.15.105:0/132906077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5bc0cc760 0x7fe5bc1560c0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac00c560 tx=0x7fe5ac00ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:10.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5b27fc640 1 -- 172.21.15.105:0/132906077 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac005220 con 0x7fe5bc0cc760 2023-11-28T18:29:10.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5b27fc640 1 -- 172.21.15.105:0/132906077 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ac005380 con 0x7fe5bc0cc760 2023-11-28T18:29:10.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5b27fc640 1 -- 172.21.15.105:0/132906077 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac007270 con 0x7fe5bc0cc760 2023-11-28T18:29:10.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5c3a6b640 1 -- 172.21.15.105:0/132906077 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5bc163b50 con 0x7fe5bc0cc760 2023-11-28T18:29:10.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.542+0000 7fe5c3a6b640 1 -- 172.21.15.105:0/132906077 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5bc164110 con 0x7fe5bc0cc760 2023-11-28T18:29:10.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.546+0000 7fe5c3a6b640 1 -- 172.21.15.105:0/132906077 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5bc067570 con 0x7fe5bc0cc760 2023-11-28T18:29:10.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.546+0000 7fe5b27fc640 1 -- 172.21.15.105:0/132906077 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fe5ac0073d0 con 0x7fe5bc0cc760 2023-11-28T18:29:10.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.546+0000 7fe5b27fc640 1 --2- 172.21.15.105:0/132906077 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe5a00668d0 0x7fe5a0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:10.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.546+0000 7fe5b27fc640 1 -- 172.21.15.105:0/132906077 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe5ac083ce0 con 0x7fe5bc0cc760 2023-11-28T18:29:10.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.546+0000 7fe5c1807640 1 --2- 172.21.15.105:0/132906077 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe5a00668d0 0x7fe5a0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:10.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.546+0000 7fe5c1807640 1 --2- 172.21.15.105:0/132906077 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe5a00668d0 0x7fe5a0068d90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80af000 tx=0x7fe5b80b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:10.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.550+0000 7fe5b27fc640 1 -- 172.21.15.105:0/132906077 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe5ac050e70 con 0x7fe5bc0cc760 2023-11-28T18:29:10.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.702+0000 7fe5c3a6b640 1 -- 172.21.15.105:0/132906077 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5bc0cf950 con 0x7fe5bc0cc760 2023-11-28T18:29:10.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.702+0000 7fe5b27fc640 1 -- 172.21.15.105:0/132906077 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16790 (secure 0 0 0) 0x7fe5ac054490 con 0x7fe5bc0cc760 2023-11-28T18:29:10.708 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:29:10.708 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":89,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:29:09.706977+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:29:08.048399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-28T18:29:08.699964+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:29:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.706+0000 7fe5c3a6b640 1 -- 172.21.15.105:0/132906077 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe5a00668d0 msgr2=0x7fe5a0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.706+0000 7fe5c3a6b640 1 --2- 172.21.15.105:0/132906077 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe5a00668d0 0x7fe5a0068d90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80af000 tx=0x7fe5b80b6040 comp rx=0 tx=0).stop 2023-11-28T18:29:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.706+0000 7fe5c3a6b640 1 -- 172.21.15.105:0/132906077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5bc0cc760 msgr2=0x7fe5bc1560c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.706+0000 7fe5c3a6b640 1 --2- 172.21.15.105:0/132906077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5bc0cc760 0x7fe5bc1560c0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac00c560 tx=0x7fe5ac00ca20 comp rx=0 tx=0).stop 2023-11-28T18:29:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.706+0000 7fe5c3a6b640 1 -- 172.21.15.105:0/132906077 shutdown_connections 2023-11-28T18:29:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.706+0000 7fe5c3a6b640 1 --2- 172.21.15.105:0/132906077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5bc159f40 0x7fe5bc156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.706+0000 7fe5c3a6b640 1 --2- 172.21.15.105:0/132906077 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe5a00668d0 0x7fe5a0068d90 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.706+0000 7fe5c3a6b640 1 --2- 172.21.15.105:0/132906077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5bc0cc760 0x7fe5bc1560c0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:10.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.706+0000 7fe5c3a6b640 1 --2- 172.21.15.105:0/132906077 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5bc0ad850 0x7fe5bc14d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:10.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.706+0000 7fe5c3a6b640 1 -- 172.21.15.105:0/132906077 >> 172.21.15.105:0/132906077 conn(0x7fe5bc0ceb90 msgr2=0x7fe5bc0af860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:10.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.706+0000 7fe5c3a6b640 1 -- 172.21.15.105:0/132906077 shutdown_connections 2023-11-28T18:29:10.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.706+0000 7fe5c3a6b640 1 -- 172.21.15.105:0/132906077 wait complete. 2023-11-28T18:29:10.729 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:29:10.777 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-11-28T18:29:10.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.862+0000 7f46ef2ea640 1 Processor -- start 2023-11-28T18:29:10.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.862+0000 7f46ef2ea640 1 -- start start 2023-11-28T18:29:10.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.862+0000 7f46ef2ea640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f46e815cfc0 0x7f46e806dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:10.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.862+0000 7f46ef2ea640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f46e80c9720 0x7f46e8070660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:10.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.866+0000 7f46ef2ea640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e80cb010 con 0x7f46e806c620 2023-11-28T18:29:10.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.866+0000 7f46ef2ea640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e815e620 con 0x7f46e815cfc0 2023-11-28T18:29:10.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.866+0000 7f46ef2ea640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e80c9c20 con 0x7f46e80c9720 2023-11-28T18:29:10.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.866+0000 7f46ed086640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f46e815cfc0 0x7f46e806dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:10.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.866+0000 7f46ed086640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f46e815cfc0 0x7f46e806dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57268/0 (socket says 172.21.15.105:57268) 2023-11-28T18:29:10.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.866+0000 7f46ed086640 1 -- 172.21.15.105:0/2079394842 learned_addr learned my addr 172.21.15.105:0/2079394842 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:10.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.866+0000 7f46ec885640 1 --2- 172.21.15.105:0/2079394842 >> v2:172.21.15.148:3300/0 conn(0x7f46e80c9720 0x7f46e8070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:10.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.866+0000 7f46ed086640 1 -- 172.21.15.105:0/2079394842 >> v2:172.21.15.148:3300/0 conn(0x7f46e80c9720 msgr2=0x7f46e8070660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:10.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.866+0000 7f46ed086640 1 --2- 172.21.15.105:0/2079394842 >> v2:172.21.15.148:3300/0 conn(0x7f46e80c9720 0x7f46e8070660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:10.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.866+0000 7f46ed086640 1 -- 172.21.15.105:0/2079394842 >> v1:172.21.15.148:6789/0 conn(0x7f46e806c620 legacy=0x7f46e806ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:10.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.866+0000 7f46ed086640 1 -- 172.21.15.105:0/2079394842 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e815e340 con 0x7f46e815cfc0 2023-11-28T18:29:10.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.866+0000 7f46ec885640 1 --2- 172.21.15.105:0/2079394842 >> v2:172.21.15.148:3300/0 conn(0x7f46e80c9720 0x7f46e8070660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:10.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.866+0000 7f46ed086640 1 --2- 172.21.15.105:0/2079394842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46e815cfc0 0x7f46e806dce0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f46e806d7f0 tx=0x7f46e401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a08e2d412dc92b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:10.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.866+0000 7f46dffff640 1 -- 172.21.15.105:0/2079394842 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e40aa5f0 con 0x7f46e815cfc0 2023-11-28T18:29:10.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.866+0000 7f46dffff640 1 -- 172.21.15.105:0/2079394842 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e40aa750 con 0x7f46e815cfc0 2023-11-28T18:29:10.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.866+0000 7f46dffff640 1 -- 172.21.15.105:0/2079394842 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e40aaae0 con 0x7f46e815cfc0 2023-11-28T18:29:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.870+0000 7f46ef2ea640 1 -- 172.21.15.105:0/2079394842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46e815cfc0 msgr2=0x7f46e806dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.870+0000 7f46ef2ea640 1 --2- 172.21.15.105:0/2079394842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46e815cfc0 0x7f46e806dce0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f46e806d7f0 tx=0x7f46e401c490 comp rx=0 tx=0).stop 2023-11-28T18:29:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.870+0000 7f46ef2ea640 1 -- 172.21.15.105:0/2079394842 shutdown_connections 2023-11-28T18:29:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.870+0000 7f46ef2ea640 1 --2- 172.21.15.105:0/2079394842 >> v2:172.21.15.148:3300/0 conn(0x7f46e80c9720 0x7f46e8070660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.870+0000 7f46ef2ea640 1 --2- 172.21.15.105:0/2079394842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46e815cfc0 0x7f46e806dce0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.870+0000 7f46ef2ea640 1 -- 172.21.15.105:0/2079394842 >> 172.21.15.105:0/2079394842 conn(0x7f46e80b0c60 msgr2=0x7f46e80b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.870+0000 7f46ef2ea640 1 -- 172.21.15.105:0/2079394842 shutdown_connections 2023-11-28T18:29:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.870+0000 7f46ef2ea640 1 -- 172.21.15.105:0/2079394842 wait complete. 2023-11-28T18:29:10.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.870+0000 7f46ef2ea640 1 Processor -- start 2023-11-28T18:29:10.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.870+0000 7f46ef2ea640 1 -- start start 2023-11-28T18:29:10.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.870+0000 7f46ef2ea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46e806c620 0x7f46e80b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:10.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.870+0000 7f46ef2ea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46e80c9720 0x7f46e815bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:10.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.870+0000 7f46ef2ea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46e815cfc0 0x7f46e815b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:10.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.870+0000 7f46ef2ea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e80b2330 con 0x7f46e806c620 2023-11-28T18:29:10.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.870+0000 7f46ef2ea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e80b2610 con 0x7f46e80c9720 2023-11-28T18:29:10.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.870+0000 7f46ef2ea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e80b28f0 con 0x7f46e815cfc0 2023-11-28T18:29:10.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.870+0000 7f46ed086640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46e806c620 0x7f46e80b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:10.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.870+0000 7f46ed086640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46e806c620 0x7f46e80b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57276/0 (socket says 172.21.15.105:57276) 2023-11-28T18:29:10.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.870+0000 7f46ed086640 1 -- 172.21.15.105:0/8690847 learned_addr learned my addr 172.21.15.105:0/8690847 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:10.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.870+0000 7f46ec885640 1 --2- 172.21.15.105:0/8690847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46e80c9720 0x7f46e815bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:10.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.870+0000 7f46ed887640 1 --2- 172.21.15.105:0/8690847 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46e815cfc0 0x7f46e815b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:10.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.874+0000 7f46ed086640 1 -- 172.21.15.105:0/8690847 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46e815cfc0 msgr2=0x7f46e815b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:10.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.874+0000 7f46ed086640 1 --2- 172.21.15.105:0/8690847 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46e815cfc0 0x7f46e815b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:10.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.874+0000 7f46ed086640 1 -- 172.21.15.105:0/8690847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46e80c9720 msgr2=0x7f46e815bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:10.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.874+0000 7f46ed086640 1 --2- 172.21.15.105:0/8690847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46e80c9720 0x7f46e815bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.874+0000 7f46ed086640 1 -- 172.21.15.105:0/8690847 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e40094d0 con 0x7f46e806c620 2023-11-28T18:29:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.874+0000 7f46ed086640 1 --2- 172.21.15.105:0/8690847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46e806c620 0x7f46e80b1d50 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f46e401a0c0 tx=0x7f46e401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.874+0000 7f46ddffb640 1 -- 172.21.15.105:0/8690847 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e40b3800 con 0x7f46e806c620 2023-11-28T18:29:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.874+0000 7f46ddffb640 1 -- 172.21.15.105:0/8690847 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e40b3960 con 0x7f46e806c620 2023-11-28T18:29:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.874+0000 7f46ddffb640 1 -- 172.21.15.105:0/8690847 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e40b3d10 con 0x7f46e806c620 2023-11-28T18:29:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.874+0000 7f46ef2ea640 1 -- 172.21.15.105:0/8690847 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e80b2bd0 con 0x7f46e806c620 2023-11-28T18:29:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.874+0000 7f46ef2ea640 1 -- 172.21.15.105:0/8690847 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e815c410 con 0x7f46e806c620 2023-11-28T18:29:10.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.874+0000 7f46ef2ea640 1 -- 172.21.15.105:0/8690847 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e8067570 con 0x7f46e806c620 2023-11-28T18:29:10.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.874+0000 7f46ddffb640 1 -- 172.21.15.105:0/8690847 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f46e40c73a0 con 0x7f46e806c620 2023-11-28T18:29:10.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.874+0000 7f46ddffb640 1 --2- 172.21.15.105:0/8690847 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46cc0668d0 0x7f46cc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:10.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.874+0000 7f46ddffb640 1 -- 172.21.15.105:0/8690847 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f46e40a7070 con 0x7f46e806c620 2023-11-28T18:29:10.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.874+0000 7f46ec885640 1 --2- 172.21.15.105:0/8690847 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46cc0668d0 0x7f46cc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:10.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.878+0000 7f46ec885640 1 --2- 172.21.15.105:0/8690847 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46cc0668d0 0x7f46cc068d90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f46e815a530 tx=0x7f46d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:10.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:10.878+0000 7f46ddffb640 1 -- 172.21.15.105:0/8690847 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f46e40ec1a0 con 0x7f46e806c620 2023-11-28T18:29:11.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.030+0000 7f46ef2ea640 1 -- 172.21.15.105:0/8690847 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46e806dac0 con 0x7f46e806c620 2023-11-28T18:29:11.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.034+0000 7f46ddffb640 1 -- 172.21.15.105:0/8690847 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16790 (secure 0 0 0) 0x7f46e40ef7c0 con 0x7f46e806c620 2023-11-28T18:29:11.037 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:29:11.037 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":89,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:29:09.706977+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:29:08.048399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-28T18:29:08.699964+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:29:11.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.034+0000 7f46ef2ea640 1 -- 172.21.15.105:0/8690847 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46cc0668d0 msgr2=0x7f46cc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:11.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.034+0000 7f46ef2ea640 1 --2- 172.21.15.105:0/8690847 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46cc0668d0 0x7f46cc068d90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f46e815a530 tx=0x7f46d8008040 comp rx=0 tx=0).stop 2023-11-28T18:29:11.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.034+0000 7f46ef2ea640 1 -- 172.21.15.105:0/8690847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46e806c620 msgr2=0x7f46e80b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:11.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.034+0000 7f46ef2ea640 1 --2- 172.21.15.105:0/8690847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46e806c620 0x7f46e80b1d50 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f46e401a0c0 tx=0x7f46e401acd0 comp rx=0 tx=0).stop 2023-11-28T18:29:11.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.034+0000 7f46ef2ea640 1 -- 172.21.15.105:0/8690847 shutdown_connections 2023-11-28T18:29:11.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.034+0000 7f46ef2ea640 1 --2- 172.21.15.105:0/8690847 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46e815cfc0 0x7f46e815b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:11.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.034+0000 7f46ef2ea640 1 --2- 172.21.15.105:0/8690847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46e80c9720 0x7f46e815bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:11.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.034+0000 7f46ef2ea640 1 --2- 172.21.15.105:0/8690847 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46cc0668d0 0x7f46cc068d90 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:11.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.034+0000 7f46ef2ea640 1 --2- 172.21.15.105:0/8690847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46e806c620 0x7f46e80b1d50 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:11.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.034+0000 7f46ef2ea640 1 -- 172.21.15.105:0/8690847 >> 172.21.15.105:0/8690847 conn(0x7f46e80b0c60 msgr2=0x7f46e80b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:11.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.034+0000 7f46ef2ea640 1 -- 172.21.15.105:0/8690847 shutdown_connections 2023-11-28T18:29:11.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.034+0000 7f46ef2ea640 1 -- 172.21.15.105:0/8690847 wait complete. 2023-11-28T18:29:11.059 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T18:29:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.194+0000 7fdabf69a640 1 Processor -- start 2023-11-28T18:29:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.194+0000 7fdabf69a640 1 -- start start 2023-11-28T18:29:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.198+0000 7fdabf69a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdab8159f40 0x7fdab80caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.198+0000 7fdabf69a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdab80cc760 0x7fdab80ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.198+0000 7fdabf69a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab806bf50 con 0x7fdab80ad680 2023-11-28T18:29:11.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.198+0000 7fdabf69a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdab80ccbc0 con 0x7fdab8159f40 2023-11-28T18:29:11.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.198+0000 7fdabf69a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab815a320 con 0x7fdab80cc760 2023-11-28T18:29:11.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.198+0000 7fdabdc37640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fdab80ad680 0x7fdab815da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:39360/0 (socket says 172.21.15.105:39360) 2023-11-28T18:29:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.198+0000 7fdabdc37640 1 -- 172.21.15.105:0/2126935921 learned_addr learned my addr 172.21.15.105:0/2126935921 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.198+0000 7fdabcc35640 1 --2- 172.21.15.105:0/2126935921 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdab80cc760 0x7fdab80ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.198+0000 7fdabd436640 1 --2- 172.21.15.105:0/2126935921 >> v2:172.21.15.148:3300/0 conn(0x7fdab8159f40 0x7fdab80caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.198+0000 7fdaaffff640 1 -- 172.21.15.105:0/2126935921 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 904233438 0 0) 0x7fdab806bf50 con 0x7fdab80ad680 2023-11-28T18:29:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.198+0000 7fdaaffff640 1 -- 172.21.15.105:0/2126935921 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaa4003530 con 0x7fdab80ad680 2023-11-28T18:29:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.198+0000 7fdabd436640 1 -- 172.21.15.105:0/2126935921 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdab80cc760 msgr2=0x7fdab80ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.198+0000 7fdabd436640 1 --2- 172.21.15.105:0/2126935921 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdab80cc760 0x7fdab80ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.198+0000 7fdabd436640 1 -- 172.21.15.105:0/2126935921 >> v1:172.21.15.148:6789/0 conn(0x7fdab80ad680 legacy=0x7fdab815da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.198+0000 7fdabd436640 1 -- 172.21.15.105:0/2126935921 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab8071380 con 0x7fdab8159f40 2023-11-28T18:29:11.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.202+0000 7fdabd436640 1 --2- 172.21.15.105:0/2126935921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab8159f40 0x7fdab80caf60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdab40097c0 tx=0x7fdab401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ac43da308469400 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:11.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.202+0000 7fdaaffff640 1 -- 172.21.15.105:0/2126935921 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab40ac620 con 0x7fdab8159f40 2023-11-28T18:29:11.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.202+0000 7fdaaffff640 1 -- 172.21.15.105:0/2126935921 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab40ac780 con 0x7fdab8159f40 2023-11-28T18:29:11.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.202+0000 7fdaaffff640 1 -- 172.21.15.105:0/2126935921 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab40acaf0 con 0x7fdab8159f40 2023-11-28T18:29:11.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.202+0000 7fdabf69a640 1 -- 172.21.15.105:0/2126935921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab8159f40 msgr2=0x7fdab80caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:11.206 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.202+0000 7fdabf69a640 1 --2- 172.21.15.105:0/2126935921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab8159f40 0x7fdab80caf60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdab40097c0 tx=0x7fdab401c3c0 comp rx=0 tx=0).stop 2023-11-28T18:29:11.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.202+0000 7fdabf69a640 1 -- 172.21.15.105:0/2126935921 shutdown_connections 2023-11-28T18:29:11.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.202+0000 7fdabf69a640 1 --2- 172.21.15.105:0/2126935921 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdab80cc760 0x7fdab80ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:11.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.202+0000 7fdabf69a640 1 --2- 172.21.15.105:0/2126935921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab8159f40 0x7fdab80caf60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:11.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.202+0000 7fdabf69a640 1 -- 172.21.15.105:0/2126935921 >> 172.21.15.105:0/2126935921 conn(0x7fdab80ceb90 msgr2=0x7fdab80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:11.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.202+0000 7fdabf69a640 1 -- 172.21.15.105:0/2126935921 shutdown_connections 2023-11-28T18:29:11.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.202+0000 7fdabf69a640 1 -- 172.21.15.105:0/2126935921 wait complete. 2023-11-28T18:29:11.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.202+0000 7fdabf69a640 1 Processor -- start 2023-11-28T18:29:11.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdabf69a640 1 -- start start 2023-11-28T18:29:11.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdabf69a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdab80ad680 0x7fdab814f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:11.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdabf69a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab80cc760 0x7fdab814f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:11.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdabf69a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdab8159f40 0x7fdab80b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:11.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdabf69a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab815af10 con 0x7fdab80ad680 2023-11-28T18:29:11.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdabf69a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab815b160 con 0x7fdab80cc760 2023-11-28T18:29:11.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdabf69a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab815b440 con 0x7fdab8159f40 2023-11-28T18:29:11.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdabd436640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdab80ad680 0x7fdab814f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:11.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdabd436640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdab80ad680 0x7fdab814f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53976/0 (socket says 172.21.15.105:53976) 2023-11-28T18:29:11.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdabd436640 1 -- 172.21.15.105:0/782079397 learned_addr learned my addr 172.21.15.105:0/782079397 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:11.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdabdc37640 1 --2- 172.21.15.105:0/782079397 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdab8159f40 0x7fdab80b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:11.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdabcc35640 1 --2- 172.21.15.105:0/782079397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab80cc760 0x7fdab814f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:11.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdabd436640 1 -- 172.21.15.105:0/782079397 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdab8159f40 msgr2=0x7fdab80b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:11.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdabd436640 1 --2- 172.21.15.105:0/782079397 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdab8159f40 0x7fdab80b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:11.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdabd436640 1 -- 172.21.15.105:0/782079397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab80cc760 msgr2=0x7fdab814f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:11.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdabd436640 1 --2- 172.21.15.105:0/782079397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab80cc760 0x7fdab814f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:11.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdabd436640 1 -- 172.21.15.105:0/782079397 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab4009420 con 0x7fdab80ad680 2023-11-28T18:29:11.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdabdc37640 1 --2- 172.21.15.105:0/782079397 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdab8159f40 0x7fdab80b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:11.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdabd436640 1 --2- 172.21.15.105:0/782079397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdab80ad680 0x7fdab814f0a0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fdab401aa70 tx=0x7fdab4019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:11.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdaadffb640 1 -- 172.21.15.105:0/782079397 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab40bb070 con 0x7fdab80ad680 2023-11-28T18:29:11.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdaadffb640 1 -- 172.21.15.105:0/782079397 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab40ac420 con 0x7fdab80ad680 2023-11-28T18:29:11.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdabf69a640 1 -- 172.21.15.105:0/782079397 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab815b720 con 0x7fdab80ad680 2023-11-28T18:29:11.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdaadffb640 1 -- 172.21.15.105:0/782079397 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab40b5640 con 0x7fdab80ad680 2023-11-28T18:29:11.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdabf69a640 1 -- 172.21.15.105:0/782079397 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab815bce0 con 0x7fdab80ad680 2023-11-28T18:29:11.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.206+0000 7fdabf69a640 1 -- 172.21.15.105:0/782079397 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab815bf00 con 0x7fdab80ad680 2023-11-28T18:29:11.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.210+0000 7fdaadffb640 1 -- 172.21.15.105:0/782079397 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fdab40b3040 con 0x7fdab80ad680 2023-11-28T18:29:11.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.210+0000 7fdaadffb640 1 --2- 172.21.15.105:0/782079397 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fda9c0668d0 0x7fda9c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:11.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.210+0000 7fdaadffb640 1 -- 172.21.15.105:0/782079397 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdab411da60 con 0x7fdab80ad680 2023-11-28T18:29:11.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.210+0000 7fdabcc35640 1 --2- 172.21.15.105:0/782079397 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fda9c0668d0 0x7fda9c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:11.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.210+0000 7fdabcc35640 1 --2- 172.21.15.105:0/782079397 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fda9c0668d0 0x7fda9c068d90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdab815c220 tx=0x7fdaa8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:11.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.214+0000 7fdaadffb640 1 -- 172.21.15.105:0/782079397 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdab40eac80 con 0x7fdab80ad680 2023-11-28T18:29:11.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:11.402+0000 7fdabf69a640 1 -- 172.21.15.105:0/782079397 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdab8067570 con 0x7fdab80ad680 2023-11-28T18:29:12.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.374+0000 7fdaadffb640 1 -- 172.21.15.105:0/782079397 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v86) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdab40ee2a0 con 0x7fdab80ad680 2023-11-28T18:29:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.378+0000 7fdabf69a640 1 -- 172.21.15.105:0/782079397 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fda9c0668d0 msgr2=0x7fda9c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.378+0000 7fdabf69a640 1 --2- 172.21.15.105:0/782079397 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fda9c0668d0 0x7fda9c068d90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdab815c220 tx=0x7fdaa8009190 comp rx=0 tx=0).stop 2023-11-28T18:29:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.378+0000 7fdabf69a640 1 -- 172.21.15.105:0/782079397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdab80ad680 msgr2=0x7fdab814f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.378+0000 7fdabf69a640 1 --2- 172.21.15.105:0/782079397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdab80ad680 0x7fdab814f0a0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fdab401aa70 tx=0x7fdab4019160 comp rx=0 tx=0).stop 2023-11-28T18:29:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.378+0000 7fdabf69a640 1 -- 172.21.15.105:0/782079397 shutdown_connections 2023-11-28T18:29:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.378+0000 7fdabf69a640 1 --2- 172.21.15.105:0/782079397 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdab8159f40 0x7fdab80b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.378+0000 7fdabf69a640 1 --2- 172.21.15.105:0/782079397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdab80cc760 0x7fdab814f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.378+0000 7fdabf69a640 1 --2- 172.21.15.105:0/782079397 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fda9c0668d0 0x7fda9c068d90 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.378+0000 7fdabf69a640 1 --2- 172.21.15.105:0/782079397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdab80ad680 0x7fdab814f0a0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.378+0000 7fdabf69a640 1 -- 172.21.15.105:0/782079397 >> 172.21.15.105:0/782079397 conn(0x7fdab80ceb90 msgr2=0x7fdab80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.378+0000 7fdabf69a640 1 -- 172.21.15.105:0/782079397 shutdown_connections 2023-11-28T18:29:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.378+0000 7fdabf69a640 1 -- 172.21.15.105:0/782079397 wait complete. 2023-11-28T18:29:12.405 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T18:29:12.405 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T18:29:12.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.546+0000 7fe3f4e9c640 1 Processor -- start 2023-11-28T18:29:12.549 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.546+0000 7fe3f4e9c640 1 -- start start 2023-11-28T18:29:12.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.546+0000 7fe3f4e9c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe3f0159f40 0x7fe3f00caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:12.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.546+0000 7fe3f4e9c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3f00cc760 0x7fe3f00ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:12.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.546+0000 7fe3f4e9c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f006bf50 con 0x7fe3f00ad680 2023-11-28T18:29:12.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.546+0000 7fe3f4e9c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f00ccbc0 con 0x7fe3f0159f40 2023-11-28T18:29:12.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.546+0000 7fe3f4e9c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f015a320 con 0x7fe3f00cc760 2023-11-28T18:29:12.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.546+0000 7fe3e5d9b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3f00cc760 0x7fe3f00ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:12.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.546+0000 7fe3ee59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe3f0159f40 0x7fe3f00caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:12.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.546+0000 7fe3eed9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe3f00ad680 0x7fe3f015da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:39376/0 (socket says 172.21.15.105:39376) 2023-11-28T18:29:12.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.546+0000 7fe3eed9d640 1 -- 172.21.15.105:0/1753653678 learned_addr learned my addr 172.21.15.105:0/1753653678 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:12.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.546+0000 7fe3edd9b640 1 -- 172.21.15.105:0/1753653678 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059822822 0 0) 0x7fe3f006bf50 con 0x7fe3f00ad680 2023-11-28T18:29:12.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.546+0000 7fe3edd9b640 1 -- 172.21.15.105:0/1753653678 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3dc003530 con 0x7fe3f00ad680 2023-11-28T18:29:12.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.546+0000 7fe3ee59c640 1 -- 172.21.15.105:0/1753653678 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3f00cc760 msgr2=0x7fe3f00ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:12.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.546+0000 7fe3ee59c640 1 --2- 172.21.15.105:0/1753653678 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3f00cc760 0x7fe3f00ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:12.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.546+0000 7fe3ee59c640 1 -- 172.21.15.105:0/1753653678 >> v1:172.21.15.148:6789/0 conn(0x7fe3f00ad680 legacy=0x7fe3f015da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:12.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.546+0000 7fe3ee59c640 1 -- 172.21.15.105:0/1753653678 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f0071380 con 0x7fe3f0159f40 2023-11-28T18:29:12.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.546+0000 7fe3e5d9b640 1 --2- 172.21.15.105:0/1753653678 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3f00cc760 0x7fe3f00ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:29:12.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.550+0000 7fe3ee59c640 1 --2- 172.21.15.105:0/1753653678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3f0159f40 0x7fe3f00caf60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00097c0 tx=0x7fe3e001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34efd0bd0523c73b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:12.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.550+0000 7fe3edd9b640 1 -- 172.21.15.105:0/1753653678 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e00ac620 con 0x7fe3f0159f40 2023-11-28T18:29:12.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.550+0000 7fe3edd9b640 1 -- 172.21.15.105:0/1753653678 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e00ac780 con 0x7fe3f0159f40 2023-11-28T18:29:12.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.550+0000 7fe3edd9b640 1 -- 172.21.15.105:0/1753653678 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e00acb10 con 0x7fe3f0159f40 2023-11-28T18:29:12.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.550+0000 7fe3f4e9c640 1 -- 172.21.15.105:0/1753653678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3f0159f40 msgr2=0x7fe3f00caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:12.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.550+0000 7fe3f4e9c640 1 --2- 172.21.15.105:0/1753653678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3f0159f40 0x7fe3f00caf60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00097c0 tx=0x7fe3e001c3c0 comp rx=0 tx=0).stop 2023-11-28T18:29:12.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.550+0000 7fe3f4e9c640 1 -- 172.21.15.105:0/1753653678 shutdown_connections 2023-11-28T18:29:12.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.550+0000 7fe3f4e9c640 1 --2- 172.21.15.105:0/1753653678 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3f00cc760 0x7fe3f00ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:12.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.550+0000 7fe3f4e9c640 1 --2- 172.21.15.105:0/1753653678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3f0159f40 0x7fe3f00caf60 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:12.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.550+0000 7fe3f4e9c640 1 -- 172.21.15.105:0/1753653678 >> 172.21.15.105:0/1753653678 conn(0x7fe3f00ceb90 msgr2=0x7fe3f00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:12.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.550+0000 7fe3f4e9c640 1 -- 172.21.15.105:0/1753653678 shutdown_connections 2023-11-28T18:29:12.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.550+0000 7fe3f4e9c640 1 -- 172.21.15.105:0/1753653678 wait complete. 2023-11-28T18:29:12.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.554+0000 7fe3f4e9c640 1 Processor -- start 2023-11-28T18:29:12.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.554+0000 7fe3f4e9c640 1 -- start start 2023-11-28T18:29:12.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.554+0000 7fe3f4e9c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f00ad680 0x7fe3f014a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:12.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.554+0000 7fe3f4e9c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3f00cc760 0x7fe3f015a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:12.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.554+0000 7fe3f4e9c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3f0159f40 0x7fe3f0158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:12.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.554+0000 7fe3f4e9c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f0154e90 con 0x7fe3f00ad680 2023-11-28T18:29:12.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.554+0000 7fe3f4e9c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f014a8d0 con 0x7fe3f00cc760 2023-11-28T18:29:12.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.554+0000 7fe3f4e9c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f0166260 con 0x7fe3f0159f40 2023-11-28T18:29:12.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.554+0000 7fe3ee59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f00ad680 0x7fe3f014a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:12.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.554+0000 7fe3ee59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f00ad680 0x7fe3f014a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53988/0 (socket says 172.21.15.105:53988) 2023-11-28T18:29:12.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.554+0000 7fe3ee59c640 1 -- 172.21.15.105:0/449138339 learned_addr learned my addr 172.21.15.105:0/449138339 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:12.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.554+0000 7fe3e5d9b640 1 --2- 172.21.15.105:0/449138339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3f00cc760 0x7fe3f015a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:12.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.554+0000 7fe3eed9d640 1 --2- 172.21.15.105:0/449138339 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3f0159f40 0x7fe3f0158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:12.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.554+0000 7fe3ee59c640 1 -- 172.21.15.105:0/449138339 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3f0159f40 msgr2=0x7fe3f0158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:12.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.554+0000 7fe3ee59c640 1 --2- 172.21.15.105:0/449138339 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3f0159f40 0x7fe3f0158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:12.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.554+0000 7fe3ee59c640 1 -- 172.21.15.105:0/449138339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3f00cc760 msgr2=0x7fe3f015a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:12.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.554+0000 7fe3ee59c640 1 --2- 172.21.15.105:0/449138339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3f00cc760 0x7fe3f015a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:12.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.554+0000 7fe3ee59c640 1 -- 172.21.15.105:0/449138339 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d8009420 con 0x7fe3f00ad680 2023-11-28T18:29:12.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.554+0000 7fe3eed9d640 1 --2- 172.21.15.105:0/449138339 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3f0159f40 0x7fe3f0158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:12.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.554+0000 7fe3ee59c640 1 --2- 172.21.15.105:0/449138339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f00ad680 0x7fe3f014a390 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00af040 tx=0x7fe3e0019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:12.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.554+0000 7fe3e77fe640 1 -- 172.21.15.105:0/449138339 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e00ac620 con 0x7fe3f00ad680 2023-11-28T18:29:12.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.558+0000 7fe3e77fe640 1 -- 172.21.15.105:0/449138339 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e00b5410 con 0x7fe3f00ad680 2023-11-28T18:29:12.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.558+0000 7fe3f4e9c640 1 -- 172.21.15.105:0/449138339 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e0009420 con 0x7fe3f00ad680 2023-11-28T18:29:12.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.558+0000 7fe3e77fe640 1 -- 172.21.15.105:0/449138339 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e00b57e0 con 0x7fe3f00ad680 2023-11-28T18:29:12.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.558+0000 7fe3f4e9c640 1 -- 172.21.15.105:0/449138339 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f0166820 con 0x7fe3f00ad680 2023-11-28T18:29:12.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.558+0000 7fe3f4e9c640 1 -- 172.21.15.105:0/449138339 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f0067570 con 0x7fe3f00ad680 2023-11-28T18:29:12.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.558+0000 7fe3e77fe640 1 -- 172.21.15.105:0/449138339 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fe3e00b3040 con 0x7fe3f00ad680 2023-11-28T18:29:12.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.558+0000 7fe3e77fe640 1 --2- 172.21.15.105:0/449138339 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3d00668d0 0x7fe3d0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:12.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.558+0000 7fe3e77fe640 1 -- 172.21.15.105:0/449138339 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe3e011e5f0 con 0x7fe3f00ad680 2023-11-28T18:29:12.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.558+0000 7fe3e5d9b640 1 --2- 172.21.15.105:0/449138339 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3d00668d0 0x7fe3d0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:12.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.558+0000 7fe3e5d9b640 1 --2- 172.21.15.105:0/449138339 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3d00668d0 0x7fe3d0068d90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00cf820 tx=0x7fe3d8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:12.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.562+0000 7fe3e77fe640 1 -- 172.21.15.105:0/449138339 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe3e00eb680 con 0x7fe3f00ad680 2023-11-28T18:29:12.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:12.746+0000 7fe3f4e9c640 1 -- 172.21.15.105:0/449138339 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe3f00cf950 con 0x7fe3f00ad680 2023-11-28T18:29:13.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.386+0000 7fe3e77fe640 1 -- 172.21.15.105:0/449138339 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v87) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe3e00eeca0 con 0x7fe3f00ad680 2023-11-28T18:29:13.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.390+0000 7fe3f4e9c640 1 -- 172.21.15.105:0/449138339 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3d00668d0 msgr2=0x7fe3d0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:13.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.390+0000 7fe3f4e9c640 1 --2- 172.21.15.105:0/449138339 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3d00668d0 0x7fe3d0068d90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00cf820 tx=0x7fe3d8009190 comp rx=0 tx=0).stop 2023-11-28T18:29:13.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.390+0000 7fe3f4e9c640 1 -- 172.21.15.105:0/449138339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f00ad680 msgr2=0x7fe3f014a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:13.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.390+0000 7fe3f4e9c640 1 --2- 172.21.15.105:0/449138339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f00ad680 0x7fe3f014a390 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00af040 tx=0x7fe3e0019160 comp rx=0 tx=0).stop 2023-11-28T18:29:13.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.390+0000 7fe3f4e9c640 1 -- 172.21.15.105:0/449138339 shutdown_connections 2023-11-28T18:29:13.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.390+0000 7fe3f4e9c640 1 --2- 172.21.15.105:0/449138339 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3f0159f40 0x7fe3f0158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:13.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.390+0000 7fe3f4e9c640 1 --2- 172.21.15.105:0/449138339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3f00cc760 0x7fe3f015a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:13.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.390+0000 7fe3f4e9c640 1 --2- 172.21.15.105:0/449138339 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3d00668d0 0x7fe3d0068d90 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:13.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.390+0000 7fe3f4e9c640 1 --2- 172.21.15.105:0/449138339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f00ad680 0x7fe3f014a390 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:13.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.390+0000 7fe3f4e9c640 1 -- 172.21.15.105:0/449138339 >> 172.21.15.105:0/449138339 conn(0x7fe3f00ceb90 msgr2=0x7fe3f0073020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:13.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.390+0000 7fe3f4e9c640 1 -- 172.21.15.105:0/449138339 shutdown_connections 2023-11-28T18:29:13.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.390+0000 7fe3f4e9c640 1 -- 172.21.15.105:0/449138339 wait complete. 2023-11-28T18:29:13.416 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T18:29:13.416 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:29:13.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09d101640 1 Processor -- start 2023-11-28T18:29:13.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09d101640 1 -- start start 2023-11-28T18:29:13.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09d101640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0980caf60 0x7fd0980cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:13.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09d101640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd0980ad850 0x7fd0980701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:13.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09d101640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd09806bf50 con 0x7fd098159f40 2023-11-28T18:29:13.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09d101640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0980cb3c0 con 0x7fd0980ad850 2023-11-28T18:29:13.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09d101640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09815a320 con 0x7fd0980caf60 2023-11-28T18:29:13.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd096d9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd098159f40 0x7fd0980ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57758/0 (socket says 172.21.15.105:57758) 2023-11-28T18:29:13.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd096d9d640 1 -- 172.21.15.105:0/874332958 learned_addr learned my addr 172.21.15.105:0/874332958 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:13.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd08e59c640 1 --2- 172.21.15.105:0/874332958 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0980caf60 0x7fd0980cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:13.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09759e640 1 --2- 172.21.15.105:0/874332958 >> v2:172.21.15.148:3300/0 conn(0x7fd0980ad850 0x7fd0980701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:13.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09659c640 1 -- 172.21.15.105:0/874332958 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4232187818 0 0) 0x7fd09806bf50 con 0x7fd098159f40 2023-11-28T18:29:13.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09659c640 1 -- 172.21.15.105:0/874332958 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd084003530 con 0x7fd098159f40 2023-11-28T18:29:13.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09659c640 1 -- 172.21.15.105:0/874332958 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 436270835 0 0) 0x7fd084003530 con 0x7fd098159f40 2023-11-28T18:29:13.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09659c640 1 -- 172.21.15.105:0/874332958 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0980caf60 msgr2=0x7fd0980cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:13.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09659c640 1 --2- 172.21.15.105:0/874332958 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0980caf60 0x7fd0980cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:13.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09659c640 1 -- 172.21.15.105:0/874332958 >> v2:172.21.15.148:3300/0 conn(0x7fd0980ad850 msgr2=0x7fd0980701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:13.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09659c640 1 --2- 172.21.15.105:0/874332958 >> v2:172.21.15.148:3300/0 conn(0x7fd0980ad850 0x7fd0980701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:13.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09659c640 1 -- 172.21.15.105:0/874332958 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd098071380 con 0x7fd098159f40 2023-11-28T18:29:13.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09659c640 1 -- 172.21.15.105:0/874332958 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fd088001a00 con 0x7fd098159f40 2023-11-28T18:29:13.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09659c640 1 -- 172.21.15.105:0/874332958 >> v1:172.21.15.105:6789/0 conn(0x7fd098159f40 legacy=0x7fd0980ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:13.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09659c640 1 --2- 172.21.15.105:0/874332958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd084003980 0x7fd084003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:13.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09659c640 1 --2- 172.21.15.105:0/874332958 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0980caf60 0x7fd0840047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:13.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09659c640 1 --2- 172.21.15.105:0/874332958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd084004d10 0x7fd084005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:13.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09659c640 1 -- 172.21.15.105:0/874332958 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0980cb3c0 con 0x7fd084004d10 2023-11-28T18:29:13.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09659c640 1 -- 172.21.15.105:0/874332958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09815a320 con 0x7fd084003980 2023-11-28T18:29:13.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09659c640 1 -- 172.21.15.105:0/874332958 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08400b990 con 0x7fd0980caf60 2023-11-28T18:29:13.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd09759e640 1 --2- 172.21.15.105:0/874332958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd084004d10 0x7fd084005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:13.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd096d9d640 1 --2- 172.21.15.105:0/874332958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd084003980 0x7fd084003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:13.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.554+0000 7fd08e59c640 1 --2- 172.21.15.105:0/874332958 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0980caf60 0x7fd0840047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:13.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.558+0000 7fd09759e640 1 -- 172.21.15.105:0/874332958 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0980caf60 msgr2=0x7fd0840047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:13.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.558+0000 7fd09759e640 1 --2- 172.21.15.105:0/874332958 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0980caf60 0x7fd0840047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:13.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.558+0000 7fd09759e640 1 -- 172.21.15.105:0/874332958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd084003980 msgr2=0x7fd084003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:13.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.558+0000 7fd09759e640 1 --2- 172.21.15.105:0/874332958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd084003980 0x7fd084003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:13.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.558+0000 7fd09759e640 1 -- 172.21.15.105:0/874332958 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd084003530 con 0x7fd084004d10 2023-11-28T18:29:13.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.558+0000 7fd09759e640 1 --2- 172.21.15.105:0/874332958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd084004d10 0x7fd084005190 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fd090009670 tx=0x7fd09001ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea516d05667b80b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:13.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.558+0000 7fd09659c640 1 -- 172.21.15.105:0/874332958 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0900a77b0 con 0x7fd084004d10 2023-11-28T18:29:13.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.558+0000 7fd09659c640 1 -- 172.21.15.105:0/874332958 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0900a7910 con 0x7fd084004d10 2023-11-28T18:29:13.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.558+0000 7fd09659c640 1 -- 172.21.15.105:0/874332958 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0900a7cc0 con 0x7fd084004d10 2023-11-28T18:29:13.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.558+0000 7fd09d101640 1 -- 172.21.15.105:0/874332958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd084004d10 msgr2=0x7fd084005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:13.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.558+0000 7fd09d101640 1 --2- 172.21.15.105:0/874332958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd084004d10 0x7fd084005190 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fd090009670 tx=0x7fd09001ecc0 comp rx=0 tx=0).stop 2023-11-28T18:29:13.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.558+0000 7fd09759e640 1 -- 172.21.15.105:0/874332958 reap_dead start 2023-11-28T18:29:13.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.558+0000 7fd09d101640 1 -- 172.21.15.105:0/874332958 shutdown_connections 2023-11-28T18:29:13.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.558+0000 7fd09d101640 1 -- 172.21.15.105:0/874332958 >> 172.21.15.105:0/874332958 conn(0x7fd0980ceb90 msgr2=0x7fd0980cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:13.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.558+0000 7fd09d101640 1 -- 172.21.15.105:0/874332958 shutdown_connections 2023-11-28T18:29:13.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.558+0000 7fd09d101640 1 -- 172.21.15.105:0/874332958 wait complete. 2023-11-28T18:29:13.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.562+0000 7fd09d101640 1 Processor -- start 2023-11-28T18:29:13.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.562+0000 7fd09d101640 1 -- start start 2023-11-28T18:29:13.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.562+0000 7fd09d101640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd09814ad80 0x7fd0980aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:13.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.562+0000 7fd09d101640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd09814b160 0x7fd0980af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:13.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.562+0000 7fd09d101640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd09809ad40 0x7fd0981f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:13.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.562+0000 7fd09d101640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0980c8b30 con 0x7fd09814b160 2023-11-28T18:29:13.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.562+0000 7fd09d101640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0980c7f40 con 0x7fd09809ad40 2023-11-28T18:29:13.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.562+0000 7fd09d101640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0980c8190 con 0x7fd09814ad80 2023-11-28T18:29:13.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.562+0000 7fd08e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd09814b160 0x7fd0980af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:13.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.562+0000 7fd08e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd09814b160 0x7fd0980af2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54004/0 (socket says 172.21.15.105:54004) 2023-11-28T18:29:13.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.562+0000 7fd08e59c640 1 -- 172.21.15.105:0/3462725614 learned_addr learned my addr 172.21.15.105:0/3462725614 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:13.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.562+0000 7fd096d9d640 1 --2- 172.21.15.105:0/3462725614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd09814ad80 0x7fd0980aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:13.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.562+0000 7fd09759e640 1 --2- 172.21.15.105:0/3462725614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd09809ad40 0x7fd0981f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:13.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.562+0000 7fd08e59c640 1 -- 172.21.15.105:0/3462725614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd09814ad80 msgr2=0x7fd0980aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:13.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.562+0000 7fd08e59c640 1 --2- 172.21.15.105:0/3462725614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd09814ad80 0x7fd0980aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:13.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.562+0000 7fd08e59c640 1 -- 172.21.15.105:0/3462725614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd09809ad40 msgr2=0x7fd0981f69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:13.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.562+0000 7fd08e59c640 1 --2- 172.21.15.105:0/3462725614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd09809ad40 0x7fd0981f69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:13.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.562+0000 7fd08e59c640 1 -- 172.21.15.105:0/3462725614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0980c8470 con 0x7fd09814b160 2023-11-28T18:29:13.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.562+0000 7fd08e59c640 1 --2- 172.21.15.105:0/3462725614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd09814b160 0x7fd0980af2f0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fd0800119d0 tx=0x7fd080011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:13.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.566+0000 7fd08ffff640 1 -- 172.21.15.105:0/3462725614 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08000faf0 con 0x7fd09814b160 2023-11-28T18:29:13.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.566+0000 7fd08ffff640 1 -- 172.21.15.105:0/3462725614 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08000fc50 con 0x7fd09814b160 2023-11-28T18:29:13.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.566+0000 7fd09d101640 1 -- 172.21.15.105:0/3462725614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0980c8750 con 0x7fd09814b160 2023-11-28T18:29:13.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.566+0000 7fd08ffff640 1 -- 172.21.15.105:0/3462725614 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080014440 con 0x7fd09814b160 2023-11-28T18:29:13.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.566+0000 7fd09d101640 1 -- 172.21.15.105:0/3462725614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd09809cab0 con 0x7fd09814b160 2023-11-28T18:29:13.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.566+0000 7fd09d101640 1 -- 172.21.15.105:0/3462725614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0980c8c70 con 0x7fd09814b160 2023-11-28T18:29:13.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.566+0000 7fd08ffff640 1 -- 172.21.15.105:0/3462725614 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd0800145a0 con 0x7fd09814b160 2023-11-28T18:29:13.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.566+0000 7fd08ffff640 1 --2- 172.21.15.105:0/3462725614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd0780668d0 0x7fd078068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:13.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.566+0000 7fd08ffff640 1 -- 172.21.15.105:0/3462725614 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd080084b50 con 0x7fd09814b160 2023-11-28T18:29:13.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.566+0000 7fd096d9d640 1 --2- 172.21.15.105:0/3462725614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd0780668d0 0x7fd078068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:13.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.570+0000 7fd096d9d640 1 --2- 172.21.15.105:0/3462725614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd0780668d0 0x7fd078068d90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd088002050 tx=0x7fd08800a060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:13.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.570+0000 7fd08ffff640 1 -- 172.21.15.105:0/3462725614 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd0980c8c70 con 0x7fd09814b160 2023-11-28T18:29:13.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.750+0000 7fd09d101640 1 -- 172.21.15.105:0/3462725614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd09809cab0 con 0x7fd09814b160 2023-11-28T18:29:13.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.750+0000 7fd08ffff640 1 -- 172.21.15.105:0/3462725614 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4927 (secure 0 0 0) 0x7fd09809cab0 con 0x7fd09814b160 2023-11-28T18:29:13.756 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:29:13.756 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":87,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/1464337040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1464337040},{"type":"v1","addr":"172.21.15.148:6834","nonce":1464337040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3461832163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":3461832163},{"type":"v1","addr":"172.21.15.148:6835","nonce":3461832163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7665,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/4162499249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":4162499249},{"type":"v1","addr":"172.21.15.105:6835","nonce":4162499249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:29:09.707308+0000","modified":"2023-11-28T18:29:12.751031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7689},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7689":{"gid":7689,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/3239728352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3239728352},{"type":"v1","addr":"172.21.15.105:6837","nonce":3239728352}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-28T18:29:13.756 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 87 2023-11-28T18:29:13.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.754+0000 7fd09d101640 1 -- 172.21.15.105:0/3462725614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd0780668d0 msgr2=0x7fd078068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:13.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.754+0000 7fd09d101640 1 --2- 172.21.15.105:0/3462725614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd0780668d0 0x7fd078068d90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd088002050 tx=0x7fd08800a060 comp rx=0 tx=0).stop 2023-11-28T18:29:13.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.754+0000 7fd09d101640 1 -- 172.21.15.105:0/3462725614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd09814b160 msgr2=0x7fd0980af2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:13.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.754+0000 7fd09d101640 1 --2- 172.21.15.105:0/3462725614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd09814b160 0x7fd0980af2f0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fd0800119d0 tx=0x7fd080011dd0 comp rx=0 tx=0).stop 2023-11-28T18:29:13.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.754+0000 7fd09d101640 1 -- 172.21.15.105:0/3462725614 shutdown_connections 2023-11-28T18:29:13.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.754+0000 7fd09d101640 1 --2- 172.21.15.105:0/3462725614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd09809ad40 0x7fd0981f69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:13.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.754+0000 7fd09d101640 1 --2- 172.21.15.105:0/3462725614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd0780668d0 0x7fd078068d90 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:13.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.754+0000 7fd09d101640 1 --2- 172.21.15.105:0/3462725614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd09814b160 0x7fd0980af2f0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:13.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.754+0000 7fd09d101640 1 --2- 172.21.15.105:0/3462725614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd09814ad80 0x7fd0980aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:13.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.754+0000 7fd09d101640 1 -- 172.21.15.105:0/3462725614 >> 172.21.15.105:0/3462725614 conn(0x7fd0980ceb90 msgr2=0x7fd0980cd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:13.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.754+0000 7fd09d101640 1 -- 172.21.15.105:0/3462725614 shutdown_connections 2023-11-28T18:29:13.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.754+0000 7fd09d101640 1 -- 172.21.15.105:0/3462725614 wait complete. 2023-11-28T18:29:13.777 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:29:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.910+0000 7f4a3e780640 1 Processor -- start 2023-11-28T18:29:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.910+0000 7f4a3e780640 1 -- start start 2023-11-28T18:29:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.910+0000 7f4a3e780640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a3815d4f0 0x7f4a380d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.910+0000 7f4a3e780640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4a380d3c30 0x7f4a380d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.910+0000 7f4a3e780640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a380b25b0 con 0x7f4a380d1890 2023-11-28T18:29:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.910+0000 7f4a3e780640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a380b2900 con 0x7f4a380d3c30 2023-11-28T18:29:13.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.910+0000 7f4a3e780640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a380cea30 con 0x7f4a3815d4f0 2023-11-28T18:29:13.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.910+0000 7f4a377fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4a380d3c30 0x7f4a380d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:13.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.910+0000 7f4a377fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4a380d3c30 0x7f4a380d1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54012/0 (socket says 172.21.15.105:54012) 2023-11-28T18:29:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.910+0000 7f4a377fe640 1 -- 172.21.15.105:0/993481285 learned_addr learned my addr 172.21.15.105:0/993481285 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.910+0000 7f4a37fff640 1 --2- 172.21.15.105:0/993481285 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a3815d4f0 0x7f4a380d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.914+0000 7f4a377fe640 1 -- 172.21.15.105:0/993481285 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a3815d4f0 msgr2=0x7f4a380d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.914+0000 7f4a377fe640 1 --2- 172.21.15.105:0/993481285 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a3815d4f0 0x7f4a380d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.914+0000 7f4a377fe640 1 -- 172.21.15.105:0/993481285 >> v1:172.21.15.148:6789/0 conn(0x7f4a380d1890 legacy=0x7f4a380cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.914+0000 7f4a377fe640 1 -- 172.21.15.105:0/993481285 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a380ced10 con 0x7f4a380d3c30 2023-11-28T18:29:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.914+0000 7f4a377fe640 1 --2- 172.21.15.105:0/993481285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a380d3c30 0x7f4a380d1350 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009870 tx=0x7f4a2c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7489e9c4427f637 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.914+0000 7f4a36ffd640 1 -- 172.21.15.105:0/993481285 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c0ad640 con 0x7f4a380d3c30 2023-11-28T18:29:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.914+0000 7f4a36ffd640 1 -- 172.21.15.105:0/993481285 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2c0ad7a0 con 0x7f4a380d3c30 2023-11-28T18:29:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.914+0000 7f4a36ffd640 1 -- 172.21.15.105:0/993481285 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c0adb30 con 0x7f4a380d3c30 2023-11-28T18:29:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.914+0000 7f4a3e780640 1 -- 172.21.15.105:0/993481285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a380d3c30 msgr2=0x7f4a380d1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.914+0000 7f4a3e780640 1 --2- 172.21.15.105:0/993481285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a380d3c30 0x7f4a380d1350 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009870 tx=0x7f4a2c01c480 comp rx=0 tx=0).stop 2023-11-28T18:29:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.914+0000 7f4a3e780640 1 -- 172.21.15.105:0/993481285 shutdown_connections 2023-11-28T18:29:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.914+0000 7f4a3e780640 1 --2- 172.21.15.105:0/993481285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a380d3c30 0x7f4a380d1350 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.914+0000 7f4a3e780640 1 --2- 172.21.15.105:0/993481285 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a3815d4f0 0x7f4a380d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:13.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.914+0000 7f4a3e780640 1 -- 172.21.15.105:0/993481285 >> 172.21.15.105:0/993481285 conn(0x7f4a380d3030 msgr2=0x7f4a380d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:13.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.914+0000 7f4a3e780640 1 -- 172.21.15.105:0/993481285 shutdown_connections 2023-11-28T18:29:13.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.914+0000 7f4a3e780640 1 -- 172.21.15.105:0/993481285 wait complete. 2023-11-28T18:29:13.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.918+0000 7f4a3e780640 1 Processor -- start 2023-11-28T18:29:13.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.918+0000 7f4a3e780640 1 -- start start 2023-11-28T18:29:13.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.918+0000 7f4a3e780640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a380d1890 0x7f4a38152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:13.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.918+0000 7f4a3e780640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a380d3c30 0x7f4a380ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:13.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.918+0000 7f4a3e780640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a3815d4f0 0x7f4a380cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:13.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.918+0000 7f4a3e780640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a381542f0 con 0x7f4a380d1890 2023-11-28T18:29:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.918+0000 7f4a3e780640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3815af40 con 0x7f4a3815d4f0 2023-11-28T18:29:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.918+0000 7f4a3e780640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3815b220 con 0x7f4a380d3c30 2023-11-28T18:29:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.918+0000 7f4a37fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a380d1890 0x7f4a38152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.918+0000 7f4a37fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a380d1890 0x7f4a38152cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54018/0 (socket says 172.21.15.105:54018) 2023-11-28T18:29:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.918+0000 7f4a37fff640 1 -- 172.21.15.105:0/1340655678 learned_addr learned my addr 172.21.15.105:0/1340655678 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.918+0000 7f4a377fe640 1 --2- 172.21.15.105:0/1340655678 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a380d3c30 0x7f4a380ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.918+0000 7f4a3cd1d640 1 --2- 172.21.15.105:0/1340655678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a3815d4f0 0x7f4a380cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.918+0000 7f4a37fff640 1 -- 172.21.15.105:0/1340655678 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a380d3c30 msgr2=0x7f4a380ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.918+0000 7f4a37fff640 1 --2- 172.21.15.105:0/1340655678 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a380d3c30 0x7f4a380ca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.918+0000 7f4a37fff640 1 -- 172.21.15.105:0/1340655678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a3815d4f0 msgr2=0x7f4a380cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.918+0000 7f4a37fff640 1 --2- 172.21.15.105:0/1340655678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a3815d4f0 0x7f4a380cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.918+0000 7f4a37fff640 1 -- 172.21.15.105:0/1340655678 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2c0094d0 con 0x7f4a380d1890 2023-11-28T18:29:13.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.918+0000 7f4a37fff640 1 --2- 172.21.15.105:0/1340655678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a380d1890 0x7f4a38152cc0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800e870 tx=0x7f4a2800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:13.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.918+0000 7f4a34ff9640 1 -- 172.21.15.105:0/1340655678 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a28009510 con 0x7f4a380d1890 2023-11-28T18:29:13.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.922+0000 7f4a34ff9640 1 -- 172.21.15.105:0/1340655678 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2800eec0 con 0x7f4a380d1890 2023-11-28T18:29:13.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.922+0000 7f4a3e780640 1 -- 172.21.15.105:0/1340655678 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3815b500 con 0x7f4a380d1890 2023-11-28T18:29:13.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.922+0000 7f4a34ff9640 1 -- 172.21.15.105:0/1340655678 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2800cb40 con 0x7f4a380d1890 2023-11-28T18:29:13.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.922+0000 7f4a3e780640 1 -- 172.21.15.105:0/1340655678 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3815bac0 con 0x7f4a380d1890 2023-11-28T18:29:13.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.922+0000 7f4a3e780640 1 -- 172.21.15.105:0/1340655678 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a38067570 con 0x7f4a380d1890 2023-11-28T18:29:13.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.922+0000 7f4a34ff9640 1 -- 172.21.15.105:0/1340655678 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f4a2800cca0 con 0x7f4a380d1890 2023-11-28T18:29:13.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.922+0000 7f4a34ff9640 1 --2- 172.21.15.105:0/1340655678 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4a240668d0 0x7f4a24068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:13.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.922+0000 7f4a34ff9640 1 -- 172.21.15.105:0/1340655678 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4a280847a0 con 0x7f4a380d1890 2023-11-28T18:29:13.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.922+0000 7f4a377fe640 1 --2- 172.21.15.105:0/1340655678 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4a240668d0 0x7f4a24068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:13.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.922+0000 7f4a377fe640 1 --2- 172.21.15.105:0/1340655678 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4a240668d0 0x7f4a24068d90 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0af000 tx=0x7f4a2c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:13.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:13.926+0000 7f4a34ff9640 1 -- 172.21.15.105:0/1340655678 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4a28051800 con 0x7f4a380d1890 2023-11-28T18:29:14.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.078+0000 7f4a3e780640 1 -- 172.21.15.105:0/1340655678 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a380b1570 con 0x7f4a380d1890 2023-11-28T18:29:14.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.078+0000 7f4a34ff9640 1 -- 172.21.15.105:0/1340655678 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16777 (secure 0 0 0) 0x7f4a28054e20 con 0x7f4a380d1890 2023-11-28T18:29:14.083 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:29:14.083 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":90,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:29:11.342131+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:29:08.048399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-28T18:29:08.699964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:29:14.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.082+0000 7f4a3e780640 1 -- 172.21.15.105:0/1340655678 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4a240668d0 msgr2=0x7f4a24068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:14.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.082+0000 7f4a3e780640 1 --2- 172.21.15.105:0/1340655678 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4a240668d0 0x7f4a24068d90 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0af000 tx=0x7f4a2c008040 comp rx=0 tx=0).stop 2023-11-28T18:29:14.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.082+0000 7f4a3e780640 1 -- 172.21.15.105:0/1340655678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a380d1890 msgr2=0x7f4a38152cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:14.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.082+0000 7f4a3e780640 1 --2- 172.21.15.105:0/1340655678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a380d1890 0x7f4a38152cc0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800e870 tx=0x7f4a2800ec70 comp rx=0 tx=0).stop 2023-11-28T18:29:14.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.082+0000 7f4a3e780640 1 -- 172.21.15.105:0/1340655678 shutdown_connections 2023-11-28T18:29:14.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.082+0000 7f4a3e780640 1 --2- 172.21.15.105:0/1340655678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a3815d4f0 0x7f4a380cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:14.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.082+0000 7f4a3e780640 1 --2- 172.21.15.105:0/1340655678 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a380d3c30 0x7f4a380ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:14.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.082+0000 7f4a3e780640 1 --2- 172.21.15.105:0/1340655678 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4a240668d0 0x7f4a24068d90 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:14.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.082+0000 7f4a3e780640 1 --2- 172.21.15.105:0/1340655678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a380d1890 0x7f4a38152cc0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:14.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.082+0000 7f4a3e780640 1 -- 172.21.15.105:0/1340655678 >> 172.21.15.105:0/1340655678 conn(0x7f4a380d3030 msgr2=0x7f4a380cde80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:14.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.082+0000 7f4a3e780640 1 -- 172.21.15.105:0/1340655678 shutdown_connections 2023-11-28T18:29:14.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.082+0000 7f4a3e780640 1 -- 172.21.15.105:0/1340655678 wait complete. 2023-11-28T18:29:14.104 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T18:29:14.105 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:29:14.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b80f3e640 1 Processor -- start 2023-11-28T18:29:14.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b80f3e640 1 -- start start 2023-11-28T18:29:14.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b80f3e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b7c0b3d30 0x7f8b7c1680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:14.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b80f3e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7c0b25b0 con 0x7f8b7c15d4f0 2023-11-28T18:29:14.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b80f3e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7c0d1c30 con 0x7f8b7c0d3c30 2023-11-28T18:29:14.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b80f3e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c0b2900 con 0x7f8b7c0b3d30 2023-11-28T18:29:14.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b7a59c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8b7c15d4f0 0x7f8b7c0d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57774/0 (socket says 172.21.15.105:57774) 2023-11-28T18:29:14.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b7a59c640 1 -- 172.21.15.105:0/3529560453 learned_addr learned my addr 172.21.15.105:0/3529560453 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:14.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b7ad9d640 1 --2- 172.21.15.105:0/3529560453 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b7c0b3d30 0x7f8b7c1680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:14.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b79d9b640 1 -- 172.21.15.105:0/3529560453 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901416103 0 0) 0x7f8b7c0b25b0 con 0x7f8b7c15d4f0 2023-11-28T18:29:14.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b79d9b640 1 -- 172.21.15.105:0/3529560453 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b68003530 con 0x7f8b7c15d4f0 2023-11-28T18:29:14.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b79d9b640 1 -- 172.21.15.105:0/3529560453 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1958203190 0 0) 0x7f8b7c0d1c30 con 0x7f8b7c0d3c30 2023-11-28T18:29:14.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b79d9b640 1 -- 172.21.15.105:0/3529560453 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b7c0b25b0 con 0x7f8b7c0d3c30 2023-11-28T18:29:14.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b79d9b640 1 -- 172.21.15.105:0/3529560453 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3865993144 0 0) 0x7f8b68003530 con 0x7f8b7c15d4f0 2023-11-28T18:29:14.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b79d9b640 1 -- 172.21.15.105:0/3529560453 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b7c0b3d30 msgr2=0x7f8b7c1680a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:14.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b79d9b640 1 --2- 172.21.15.105:0/3529560453 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b7c0b3d30 0x7f8b7c1680a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:14.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b79d9b640 1 -- 172.21.15.105:0/3529560453 >> v1:172.21.15.148:6789/0 conn(0x7f8b7c0d3c30 legacy=0x7f8b7c0d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:14.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b79d9b640 1 -- 172.21.15.105:0/3529560453 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c0cea30 con 0x7f8b7c15d4f0 2023-11-28T18:29:14.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b79d9b640 1 -- 172.21.15.105:0/3529560453 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f8b64001600 con 0x7f8b7c15d4f0 2023-11-28T18:29:14.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b79d9b640 1 -- 172.21.15.105:0/3529560453 >> v1:172.21.15.105:6789/0 conn(0x7f8b7c15d4f0 legacy=0x7f8b7c0d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:14.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b79d9b640 1 --2- 172.21.15.105:0/3529560453 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b7c0b3d30 0x7f8b68003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:14.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b79d9b640 1 --2- 172.21.15.105:0/3529560453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b680040f0 0x7f8b680047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:14.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b79d9b640 1 --2- 172.21.15.105:0/3529560453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b68004d10 0x7f8b68005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:14.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b79d9b640 1 -- 172.21.15.105:0/3529560453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c0b2900 con 0x7f8b680040f0 2023-11-28T18:29:14.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b79d9b640 1 -- 172.21.15.105:0/3529560453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6800bb50 con 0x7f8b68004d10 2023-11-28T18:29:14.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b79d9b640 1 -- 172.21.15.105:0/3529560453 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6800bd10 con 0x7f8b7c0b3d30 2023-11-28T18:29:14.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.238+0000 7f8b71d9b640 1 --2- 172.21.15.105:0/3529560453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b680040f0 0x7f8b680047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:14.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.242+0000 7f8b7a59c640 1 --2- 172.21.15.105:0/3529560453 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b7c0b3d30 0x7f8b68003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:14.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.242+0000 7f8b7ad9d640 1 --2- 172.21.15.105:0/3529560453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b68004d10 0x7f8b68005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:14.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.242+0000 7f8b71d9b640 1 -- 172.21.15.105:0/3529560453 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b7c0b3d30 msgr2=0x7f8b68003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:14.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.242+0000 7f8b71d9b640 1 --2- 172.21.15.105:0/3529560453 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b7c0b3d30 0x7f8b68003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:14.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.242+0000 7f8b71d9b640 1 -- 172.21.15.105:0/3529560453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b68004d10 msgr2=0x7f8b68005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:14.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.242+0000 7f8b71d9b640 1 --2- 172.21.15.105:0/3529560453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b68004d10 0x7f8b68005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:14.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.242+0000 7f8b71d9b640 1 -- 172.21.15.105:0/3529560453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b68003530 con 0x7f8b680040f0 2023-11-28T18:29:14.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.242+0000 7f8b71d9b640 1 --2- 172.21.15.105:0/3529560453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b680040f0 0x7f8b680047d0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0018d0 tx=0x7f8b6c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58451f40e334a18 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:14.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.242+0000 7f8b79d9b640 1 -- 172.21.15.105:0/3529560453 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c0052d0 con 0x7f8b680040f0 2023-11-28T18:29:14.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.242+0000 7f8b79d9b640 1 -- 172.21.15.105:0/3529560453 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6c005450 con 0x7f8b680040f0 2023-11-28T18:29:14.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.242+0000 7f8b79d9b640 1 -- 172.21.15.105:0/3529560453 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c003840 con 0x7f8b680040f0 2023-11-28T18:29:14.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.242+0000 7f8b80f3e640 1 -- 172.21.15.105:0/3529560453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b680040f0 msgr2=0x7f8b680047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:14.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.242+0000 7f8b80f3e640 1 --2- 172.21.15.105:0/3529560453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b680040f0 0x7f8b680047d0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0018d0 tx=0x7f8b6c01c530 comp rx=0 tx=0).stop 2023-11-28T18:29:14.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.242+0000 7f8b7ad9d640 1 -- 172.21.15.105:0/3529560453 reap_dead start 2023-11-28T18:29:14.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.246+0000 7f8b80f3e640 1 -- 172.21.15.105:0/3529560453 shutdown_connections 2023-11-28T18:29:14.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.246+0000 7f8b80f3e640 1 -- 172.21.15.105:0/3529560453 >> 172.21.15.105:0/3529560453 conn(0x7f8b7c0d3030 msgr2=0x7f8b7c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:14.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.246+0000 7f8b80f3e640 1 -- 172.21.15.105:0/3529560453 shutdown_connections 2023-11-28T18:29:14.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.246+0000 7f8b80f3e640 1 -- 172.21.15.105:0/3529560453 wait complete. 2023-11-28T18:29:14.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.246+0000 7f8b80f3e640 1 Processor -- start 2023-11-28T18:29:14.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.246+0000 7f8b80f3e640 1 -- start start 2023-11-28T18:29:14.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.246+0000 7f8b80f3e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b680040f0 0x7f8b7c153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:14.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.246+0000 7f8b80f3e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b7c0b8b90 0x7f8b7c159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:14.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.246+0000 7f8b80f3e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b7c0b61f0 0x7f8b7c0b6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:14.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.246+0000 7f8b80f3e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c0d0a20 con 0x7f8b7c0b8b90 2023-11-28T18:29:14.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.246+0000 7f8b80f3e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c0c9f30 con 0x7f8b7c0b61f0 2023-11-28T18:29:14.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.246+0000 7f8b80f3e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c158d50 con 0x7f8b680040f0 2023-11-28T18:29:14.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.246+0000 7f8b71d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b7c0b8b90 0x7f8b7c159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:14.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.246+0000 7f8b71d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b7c0b8b90 0x7f8b7c159d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54036/0 (socket says 172.21.15.105:54036) 2023-11-28T18:29:14.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.246+0000 7f8b71d9b640 1 -- 172.21.15.105:0/2590353353 learned_addr learned my addr 172.21.15.105:0/2590353353 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:14.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.246+0000 7f8b7ad9d640 1 --2- 172.21.15.105:0/2590353353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b7c0b61f0 0x7f8b7c0b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:14.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.246+0000 7f8b7a59c640 1 --2- 172.21.15.105:0/2590353353 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b680040f0 0x7f8b7c153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:14.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.246+0000 7f8b71d9b640 1 -- 172.21.15.105:0/2590353353 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b680040f0 msgr2=0x7f8b7c153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:14.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.246+0000 7f8b71d9b640 1 --2- 172.21.15.105:0/2590353353 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b680040f0 0x7f8b7c153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:14.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.246+0000 7f8b71d9b640 1 -- 172.21.15.105:0/2590353353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b7c0b61f0 msgr2=0x7f8b7c0b6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:14.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.246+0000 7f8b71d9b640 1 --2- 172.21.15.105:0/2590353353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b7c0b61f0 0x7f8b7c0b6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:14.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.246+0000 7f8b71d9b640 1 -- 172.21.15.105:0/2590353353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c154390 con 0x7f8b7c0b8b90 2023-11-28T18:29:14.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.250+0000 7f8b71d9b640 1 --2- 172.21.15.105:0/2590353353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b7c0b8b90 0x7f8b7c159d80 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0ae040 tx=0x7f8b68005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:14.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.250+0000 7f8b737fe640 1 -- 172.21.15.105:0/2590353353 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c0052d0 con 0x7f8b7c0b8b90 2023-11-28T18:29:14.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.250+0000 7f8b737fe640 1 -- 172.21.15.105:0/2590353353 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6c003c80 con 0x7f8b7c0b8b90 2023-11-28T18:29:14.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.250+0000 7f8b737fe640 1 -- 172.21.15.105:0/2590353353 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c007020 con 0x7f8b7c0b8b90 2023-11-28T18:29:14.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.250+0000 7f8b80f3e640 1 -- 172.21.15.105:0/2590353353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7c158fc0 con 0x7f8b7c0b8b90 2023-11-28T18:29:14.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.250+0000 7f8b80f3e640 1 -- 172.21.15.105:0/2590353353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7c159580 con 0x7f8b7c0b8b90 2023-11-28T18:29:14.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.250+0000 7f8b80f3e640 1 -- 172.21.15.105:0/2590353353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7c067570 con 0x7f8b7c0b8b90 2023-11-28T18:29:14.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.250+0000 7f8b737fe640 1 -- 172.21.15.105:0/2590353353 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8b6c0b2040 con 0x7f8b7c0b8b90 2023-11-28T18:29:14.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.250+0000 7f8b737fe640 1 --2- 172.21.15.105:0/2590353353 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b5c066900 0x7f8b5c068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:14.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.250+0000 7f8b737fe640 1 -- 172.21.15.105:0/2590353353 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8b6c11dee0 con 0x7f8b7c0b8b90 2023-11-28T18:29:14.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.250+0000 7f8b7a59c640 1 --2- 172.21.15.105:0/2590353353 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b5c066900 0x7f8b5c068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:14.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.254+0000 7f8b7a59c640 1 --2- 172.21.15.105:0/2590353353 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b5c066900 0x7f8b5c068dc0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8b64000c10 tx=0x7f8b64001030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:14.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.254+0000 7f8b737fe640 1 -- 172.21.15.105:0/2590353353 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8b6c0eb110 con 0x7f8b7c0b8b90 2023-11-28T18:29:14.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.406+0000 7f8b80f3e640 1 -- 172.21.15.105:0/2590353353 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8b7c1688b0 con 0x7f8b5c066900 2023-11-28T18:29:14.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.414+0000 7f8b737fe640 1 -- 172.21.15.105:0/2590353353 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289011 (secure 0 0 0) 0x7f8b640070c0 con 0x7f8b5c066900 2023-11-28T18:29:14.420 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:29:14.421 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:29:14.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.418+0000 7f8b80f3e640 1 -- 172.21.15.105:0/2590353353 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b5c066900 msgr2=0x7f8b5c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:14.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.418+0000 7f8b80f3e640 1 --2- 172.21.15.105:0/2590353353 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b5c066900 0x7f8b5c068dc0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8b64000c10 tx=0x7f8b64001030 comp rx=0 tx=0).stop 2023-11-28T18:29:14.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.418+0000 7f8b80f3e640 1 -- 172.21.15.105:0/2590353353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b7c0b8b90 msgr2=0x7f8b7c159d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:14.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.418+0000 7f8b80f3e640 1 --2- 172.21.15.105:0/2590353353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b7c0b8b90 0x7f8b7c159d80 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0ae040 tx=0x7f8b68005800 comp rx=0 tx=0).stop 2023-11-28T18:29:14.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.418+0000 7f8b80f3e640 1 -- 172.21.15.105:0/2590353353 shutdown_connections 2023-11-28T18:29:14.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.418+0000 7f8b80f3e640 1 --2- 172.21.15.105:0/2590353353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b7c0b61f0 0x7f8b7c0b6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:14.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.418+0000 7f8b80f3e640 1 --2- 172.21.15.105:0/2590353353 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b5c066900 0x7f8b5c068dc0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:14.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.418+0000 7f8b80f3e640 1 --2- 172.21.15.105:0/2590353353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b7c0b8b90 0x7f8b7c159d80 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:14.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.418+0000 7f8b80f3e640 1 --2- 172.21.15.105:0/2590353353 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b680040f0 0x7f8b7c153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:14.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.418+0000 7f8b80f3e640 1 -- 172.21.15.105:0/2590353353 >> 172.21.15.105:0/2590353353 conn(0x7f8b7c0d3030 msgr2=0x7f8b7c0cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:14.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.418+0000 7f8b80f3e640 1 -- 172.21.15.105:0/2590353353 shutdown_connections 2023-11-28T18:29:14.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.418+0000 7f8b80f3e640 1 -- 172.21.15.105:0/2590353353 wait complete. 2023-11-28T18:29:14.440 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2023-11-28T18:29:13.342541+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":360576,"kb_used_data":9768,"kb_used_omap":12,"kb_used_meta":350643,"kb_avail":754614144,"statfs":{"total":773094113280,"available":772724883456,"internally_reserved":0,"allocated":10002432,"data_stored":5214493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":359058994},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.026286"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.381387+0000","last_change":"2023-11-28T18:29:10.376820+0000","last_active":"2023-11-28T18:29:10.381387+0000","last_peered":"2023-11-28T18:29:10.381387+0000","last_clean":"2023-11-28T18:29:10.381387+0000","last_became_active":"2023-11-28T18:29:10.372940+0000","last_became_peered":"2023-11-28T18:29:10.372940+0000","last_unstale":"2023-11-28T18:29:10.381387+0000","last_undegraded":"2023-11-28T18:29:10.381387+0000","last_fullsized":"2023-11-28T18:29:10.381387+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:15:04.439824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.365043+0000","last_change":"2023-11-28T18:29:10.358088+0000","last_active":"2023-11-28T18:29:10.365043+0000","last_peered":"2023-11-28T18:29:10.365043+0000","last_clean":"2023-11-28T18:29:10.365043+0000","last_became_active":"2023-11-28T18:29:10.357734+0000","last_became_peered":"2023-11-28T18:29:10.357734+0000","last_unstale":"2023-11-28T18:29:10.365043+0000","last_undegraded":"2023-11-28T18:29:10.365043+0000","last_fullsized":"2023-11-28T18:29:10.365043+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:58.005043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.363900+0000","last_change":"2023-11-28T18:29:10.354739+0000","last_active":"2023-11-28T18:29:10.363900+0000","last_peered":"2023-11-28T18:29:10.363900+0000","last_clean":"2023-11-28T18:29:10.363900+0000","last_became_active":"2023-11-28T18:29:10.354433+0000","last_became_peered":"2023-11-28T18:29:10.354433+0000","last_unstale":"2023-11-28T18:29:10.363900+0000","last_undegraded":"2023-11-28T18:29:10.363900+0000","last_fullsized":"2023-11-28T18:29:10.363900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:44.295454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.391929+0000","last_change":"2023-11-28T18:29:10.387781+0000","last_active":"2023-11-28T18:29:10.391929+0000","last_peered":"2023-11-28T18:29:10.391929+0000","last_clean":"2023-11-28T18:29:10.391929+0000","last_became_active":"2023-11-28T18:29:10.387421+0000","last_became_peered":"2023-11-28T18:29:10.387421+0000","last_unstale":"2023-11-28T18:29:10.391929+0000","last_undegraded":"2023-11-28T18:29:10.391929+0000","last_fullsized":"2023-11-28T18:29:10.391929+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:03.075548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.370346+0000","last_change":"2023-11-28T18:29:10.361908+0000","last_active":"2023-11-28T18:29:10.370346+0000","last_peered":"2023-11-28T18:29:10.370346+0000","last_clean":"2023-11-28T18:29:10.370346+0000","last_became_active":"2023-11-28T18:29:10.361582+0000","last_became_peered":"2023-11-28T18:29:10.361582+0000","last_unstale":"2023-11-28T18:29:10.370346+0000","last_undegraded":"2023-11-28T18:29:10.370346+0000","last_fullsized":"2023-11-28T18:29:10.370346+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:39:23.896975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.393604+0000","last_change":"2023-11-28T18:29:10.392610+0000","last_active":"2023-11-28T18:29:10.393604+0000","last_peered":"2023-11-28T18:29:10.393604+0000","last_clean":"2023-11-28T18:29:10.393604+0000","last_became_active":"2023-11-28T18:29:10.390791+0000","last_became_peered":"2023-11-28T18:29:10.390791+0000","last_unstale":"2023-11-28T18:29:10.393604+0000","last_undegraded":"2023-11-28T18:29:10.393604+0000","last_fullsized":"2023-11-28T18:29:10.393604+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:56:23.502494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.376368+0000","last_change":"2023-11-28T18:29:10.372170+0000","last_active":"2023-11-28T18:29:10.376368+0000","last_peered":"2023-11-28T18:29:10.376368+0000","last_clean":"2023-11-28T18:29:10.376368+0000","last_became_active":"2023-11-28T18:29:10.371891+0000","last_became_peered":"2023-11-28T18:29:10.371891+0000","last_unstale":"2023-11-28T18:29:10.376368+0000","last_undegraded":"2023-11-28T18:29:10.376368+0000","last_fullsized":"2023-11-28T18:29:10.376368+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:57:11.754150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.389158+0000","last_change":"2023-11-28T18:29:10.369093+0000","last_active":"2023-11-28T18:29:10.389158+0000","last_peered":"2023-11-28T18:29:10.389158+0000","last_clean":"2023-11-28T18:29:10.389158+0000","last_became_active":"2023-11-28T18:29:10.368378+0000","last_became_peered":"2023-11-28T18:29:10.368378+0000","last_unstale":"2023-11-28T18:29:10.389158+0000","last_undegraded":"2023-11-28T18:29:10.389158+0000","last_fullsized":"2023-11-28T18:29:10.389158+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:46:50.041536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.374418+0000","last_change":"2023-11-28T18:29:10.364179+0000","last_active":"2023-11-28T18:29:10.374418+0000","last_peered":"2023-11-28T18:29:10.374418+0000","last_clean":"2023-11-28T18:29:10.374418+0000","last_became_active":"2023-11-28T18:29:10.363779+0000","last_became_peered":"2023-11-28T18:29:10.363779+0000","last_unstale":"2023-11-28T18:29:10.374418+0000","last_undegraded":"2023-11-28T18:29:10.374418+0000","last_fullsized":"2023-11-28T18:29:10.374418+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:04:40.397373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.395405+0000","last_change":"2023-11-28T18:29:10.392418+0000","last_active":"2023-11-28T18:29:10.395405+0000","last_peered":"2023-11-28T18:29:10.395405+0000","last_clean":"2023-11-28T18:29:10.395405+0000","last_became_active":"2023-11-28T18:29:10.392120+0000","last_became_peered":"2023-11-28T18:29:10.392120+0000","last_unstale":"2023-11-28T18:29:10.395405+0000","last_undegraded":"2023-11-28T18:29:10.395405+0000","last_fullsized":"2023-11-28T18:29:10.395405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:00:27.782429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.381003+0000","last_change":"2023-11-28T18:29:10.379618+0000","last_active":"2023-11-28T18:29:10.381003+0000","last_peered":"2023-11-28T18:29:10.381003+0000","last_clean":"2023-11-28T18:29:10.381003+0000","last_became_active":"2023-11-28T18:29:10.378786+0000","last_became_peered":"2023-11-28T18:29:10.378786+0000","last_unstale":"2023-11-28T18:29:10.381003+0000","last_undegraded":"2023-11-28T18:29:10.381003+0000","last_fullsized":"2023-11-28T18:29:10.381003+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:40:59.600568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.377578+0000","last_change":"2023-11-28T18:29:10.372005+0000","last_active":"2023-11-28T18:29:10.377578+0000","last_peered":"2023-11-28T18:29:10.377578+0000","last_clean":"2023-11-28T18:29:10.377578+0000","last_became_active":"2023-11-28T18:29:10.371566+0000","last_became_peered":"2023-11-28T18:29:10.371566+0000","last_unstale":"2023-11-28T18:29:10.377578+0000","last_undegraded":"2023-11-28T18:29:10.377578+0000","last_fullsized":"2023-11-28T18:29:10.377578+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:17:48.071729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.391328+0000","last_change":"2023-11-28T18:29:10.387076+0000","last_active":"2023-11-28T18:29:10.391328+0000","last_peered":"2023-11-28T18:29:10.391328+0000","last_clean":"2023-11-28T18:29:10.391328+0000","last_became_active":"2023-11-28T18:29:10.386604+0000","last_became_peered":"2023-11-28T18:29:10.386604+0000","last_unstale":"2023-11-28T18:29:10.391328+0000","last_undegraded":"2023-11-28T18:29:10.391328+0000","last_fullsized":"2023-11-28T18:29:10.391328+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:40:59.600568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.381026+0000","last_change":"2023-11-28T18:29:10.380066+0000","last_active":"2023-11-28T18:29:10.381026+0000","last_peered":"2023-11-28T18:29:10.381026+0000","last_clean":"2023-11-28T18:29:10.381026+0000","last_became_active":"2023-11-28T18:29:10.379764+0000","last_became_peered":"2023-11-28T18:29:10.379764+0000","last_unstale":"2023-11-28T18:29:10.381026+0000","last_undegraded":"2023-11-28T18:29:10.381026+0000","last_fullsized":"2023-11-28T18:29:10.381026+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:46:50.041536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.391754+0000","last_change":"2023-11-28T18:29:10.385000+0000","last_active":"2023-11-28T18:29:10.391754+0000","last_peered":"2023-11-28T18:29:10.391754+0000","last_clean":"2023-11-28T18:29:10.391754+0000","last_became_active":"2023-11-28T18:29:10.370668+0000","last_became_peered":"2023-11-28T18:29:10.370668+0000","last_unstale":"2023-11-28T18:29:10.391754+0000","last_undegraded":"2023-11-28T18:29:10.391754+0000","last_fullsized":"2023-11-28T18:29:10.391754+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:09:48.512954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.385179+0000","last_change":"2023-11-28T18:29:10.381747+0000","last_active":"2023-11-28T18:29:10.385179+0000","last_peered":"2023-11-28T18:29:10.385179+0000","last_clean":"2023-11-28T18:29:10.385179+0000","last_became_active":"2023-11-28T18:29:10.381273+0000","last_became_peered":"2023-11-28T18:29:10.381273+0000","last_unstale":"2023-11-28T18:29:10.385179+0000","last_undegraded":"2023-11-28T18:29:10.385179+0000","last_fullsized":"2023-11-28T18:29:10.385179+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:19:58.134475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.379777+0000","last_change":"2023-11-28T18:29:10.370044+0000","last_active":"2023-11-28T18:29:10.379777+0000","last_peered":"2023-11-28T18:29:10.379777+0000","last_clean":"2023-11-28T18:29:10.379777+0000","last_became_active":"2023-11-28T18:29:10.369752+0000","last_became_peered":"2023-11-28T18:29:10.369752+0000","last_unstale":"2023-11-28T18:29:10.379777+0000","last_undegraded":"2023-11-28T18:29:10.379777+0000","last_fullsized":"2023-11-28T18:29:10.379777+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:41:34.642153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.364717+0000","last_change":"2023-11-28T18:29:10.361706+0000","last_active":"2023-11-28T18:29:10.364717+0000","last_peered":"2023-11-28T18:29:10.364717+0000","last_clean":"2023-11-28T18:29:10.364717+0000","last_became_active":"2023-11-28T18:29:10.361355+0000","last_became_peered":"2023-11-28T18:29:10.361355+0000","last_unstale":"2023-11-28T18:29:10.364717+0000","last_undegraded":"2023-11-28T18:29:10.364717+0000","last_fullsized":"2023-11-28T18:29:10.364717+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:44.282151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.377719+0000","last_change":"2023-11-28T18:29:10.374933+0000","last_active":"2023-11-28T18:29:10.377719+0000","last_peered":"2023-11-28T18:29:10.377719+0000","last_clean":"2023-11-28T18:29:10.377719+0000","last_became_active":"2023-11-28T18:29:10.374645+0000","last_became_peered":"2023-11-28T18:29:10.374645+0000","last_unstale":"2023-11-28T18:29:10.377719+0000","last_undegraded":"2023-11-28T18:29:10.377719+0000","last_fullsized":"2023-11-28T18:29:10.377719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:09:48.512954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.389578+0000","last_change":"2023-11-28T18:29:10.373800+0000","last_active":"2023-11-28T18:29:10.389578+0000","last_peered":"2023-11-28T18:29:10.389578+0000","last_clean":"2023-11-28T18:29:10.389578+0000","last_became_active":"2023-11-28T18:29:10.373373+0000","last_became_peered":"2023-11-28T18:29:10.373373+0000","last_unstale":"2023-11-28T18:29:10.389578+0000","last_undegraded":"2023-11-28T18:29:10.389578+0000","last_fullsized":"2023-11-28T18:29:10.389578+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:16:06.036193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.361781+0000","last_change":"2023-11-28T18:29:10.358406+0000","last_active":"2023-11-28T18:29:10.361781+0000","last_peered":"2023-11-28T18:29:10.361781+0000","last_clean":"2023-11-28T18:29:10.361781+0000","last_became_active":"2023-11-28T18:29:10.358065+0000","last_became_peered":"2023-11-28T18:29:10.358065+0000","last_unstale":"2023-11-28T18:29:10.361781+0000","last_undegraded":"2023-11-28T18:29:10.361781+0000","last_fullsized":"2023-11-28T18:29:10.361781+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:29.459848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.381635+0000","last_change":"2023-11-28T18:29:10.380211+0000","last_active":"2023-11-28T18:29:10.381635+0000","last_peered":"2023-11-28T18:29:10.381635+0000","last_clean":"2023-11-28T18:29:10.381635+0000","last_became_active":"2023-11-28T18:29:10.379930+0000","last_became_peered":"2023-11-28T18:29:10.379930+0000","last_unstale":"2023-11-28T18:29:10.381635+0000","last_undegraded":"2023-11-28T18:29:10.381635+0000","last_fullsized":"2023-11-28T18:29:10.381635+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:25:43.024884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.393310+0000","last_change":"2023-11-28T18:29:10.392421+0000","last_active":"2023-11-28T18:29:10.393310+0000","last_peered":"2023-11-28T18:29:10.393310+0000","last_clean":"2023-11-28T18:29:10.393310+0000","last_became_active":"2023-11-28T18:29:10.392116+0000","last_became_peered":"2023-11-28T18:29:10.392116+0000","last_unstale":"2023-11-28T18:29:10.393310+0000","last_undegraded":"2023-11-28T18:29:10.393310+0000","last_fullsized":"2023-11-28T18:29:10.393310+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:40:59.600568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.360967+0000","last_change":"2023-11-28T18:29:10.355273+0000","last_active":"2023-11-28T18:29:10.360967+0000","last_peered":"2023-11-28T18:29:10.360967+0000","last_clean":"2023-11-28T18:29:10.360967+0000","last_became_active":"2023-11-28T18:29:10.354980+0000","last_became_peered":"2023-11-28T18:29:10.354980+0000","last_unstale":"2023-11-28T18:29:10.360967+0000","last_undegraded":"2023-11-28T18:29:10.360967+0000","last_fullsized":"2023-11-28T18:29:10.360967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:58.005043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.368191+0000","last_change":"2023-11-28T18:29:10.362402+0000","last_active":"2023-11-28T18:29:10.368191+0000","last_peered":"2023-11-28T18:29:10.368191+0000","last_clean":"2023-11-28T18:29:10.368191+0000","last_became_active":"2023-11-28T18:29:10.362105+0000","last_became_peered":"2023-11-28T18:29:10.362105+0000","last_unstale":"2023-11-28T18:29:10.368191+0000","last_undegraded":"2023-11-28T18:29:10.368191+0000","last_fullsized":"2023-11-28T18:29:10.368191+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:22:06.432939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.377645+0000","last_change":"2023-11-28T18:29:10.374930+0000","last_active":"2023-11-28T18:29:10.377645+0000","last_peered":"2023-11-28T18:29:10.377645+0000","last_clean":"2023-11-28T18:29:10.377645+0000","last_became_active":"2023-11-28T18:29:10.374610+0000","last_became_peered":"2023-11-28T18:29:10.374610+0000","last_unstale":"2023-11-28T18:29:10.377645+0000","last_undegraded":"2023-11-28T18:29:10.377645+0000","last_fullsized":"2023-11-28T18:29:10.377645+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:54:33.106050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.380062+0000","last_change":"2023-11-28T18:29:10.375074+0000","last_active":"2023-11-28T18:29:10.380062+0000","last_peered":"2023-11-28T18:29:10.380062+0000","last_clean":"2023-11-28T18:29:10.380062+0000","last_became_active":"2023-11-28T18:29:10.374791+0000","last_became_peered":"2023-11-28T18:29:10.374791+0000","last_unstale":"2023-11-28T18:29:10.380062+0000","last_undegraded":"2023-11-28T18:29:10.380062+0000","last_fullsized":"2023-11-28T18:29:10.380062+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:23:33.310140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.391911+0000","last_change":"2023-11-28T18:29:10.387526+0000","last_active":"2023-11-28T18:29:10.391911+0000","last_peered":"2023-11-28T18:29:10.391911+0000","last_clean":"2023-11-28T18:29:10.391911+0000","last_became_active":"2023-11-28T18:29:10.387211+0000","last_became_peered":"2023-11-28T18:29:10.387211+0000","last_unstale":"2023-11-28T18:29:10.391911+0000","last_undegraded":"2023-11-28T18:29:10.391911+0000","last_fullsized":"2023-11-28T18:29:10.391911+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:54:33.106050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.379351+0000","last_change":"2023-11-28T18:29:10.374738+0000","last_active":"2023-11-28T18:29:10.379351+0000","last_peered":"2023-11-28T18:29:10.379351+0000","last_clean":"2023-11-28T18:29:10.379351+0000","last_became_active":"2023-11-28T18:29:10.374451+0000","last_became_peered":"2023-11-28T18:29:10.374451+0000","last_unstale":"2023-11-28T18:29:10.379351+0000","last_undegraded":"2023-11-28T18:29:10.379351+0000","last_fullsized":"2023-11-28T18:29:10.379351+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:52.925955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.370786+0000","last_change":"2023-11-28T18:29:10.362405+0000","last_active":"2023-11-28T18:29:10.370786+0000","last_peered":"2023-11-28T18:29:10.370786+0000","last_clean":"2023-11-28T18:29:10.370786+0000","last_became_active":"2023-11-28T18:29:10.362017+0000","last_became_peered":"2023-11-28T18:29:10.362017+0000","last_unstale":"2023-11-28T18:29:10.370786+0000","last_undegraded":"2023-11-28T18:29:10.370786+0000","last_fullsized":"2023-11-28T18:29:10.370786+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:55:17.520003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.372715+0000","last_change":"2023-11-28T18:29:10.366398+0000","last_active":"2023-11-28T18:29:10.372715+0000","last_peered":"2023-11-28T18:29:10.372715+0000","last_clean":"2023-11-28T18:29:10.372715+0000","last_became_active":"2023-11-28T18:29:10.366127+0000","last_became_peered":"2023-11-28T18:29:10.366127+0000","last_unstale":"2023-11-28T18:29:10.372715+0000","last_undegraded":"2023-11-28T18:29:10.372715+0000","last_fullsized":"2023-11-28T18:29:10.372715+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:26.529566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.385160+0000","last_change":"2023-11-28T18:29:10.381558+0000","last_active":"2023-11-28T18:29:10.385160+0000","last_peered":"2023-11-28T18:29:10.385160+0000","last_clean":"2023-11-28T18:29:10.385160+0000","last_became_active":"2023-11-28T18:29:10.380938+0000","last_became_peered":"2023-11-28T18:29:10.380938+0000","last_unstale":"2023-11-28T18:29:10.385160+0000","last_undegraded":"2023-11-28T18:29:10.385160+0000","last_fullsized":"2023-11-28T18:29:10.385160+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:23:43.982249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.382243+0000","last_change":"2023-11-28T18:29:10.379240+0000","last_active":"2023-11-28T18:29:10.382243+0000","last_peered":"2023-11-28T18:29:10.382243+0000","last_clean":"2023-11-28T18:29:10.382243+0000","last_became_active":"2023-11-28T18:29:10.378975+0000","last_became_peered":"2023-11-28T18:29:10.378975+0000","last_unstale":"2023-11-28T18:29:10.382243+0000","last_undegraded":"2023-11-28T18:29:10.382243+0000","last_fullsized":"2023-11-28T18:29:10.382243+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:42:32.991902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"89'4","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.381513+0000","last_change":"2023-11-28T18:29:09.348890+0000","last_active":"2023-11-28T18:29:10.381513+0000","last_peered":"2023-11-28T18:29:10.381513+0000","last_clean":"2023-11-28T18:29:10.381513+0000","last_became_active":"2023-11-28T18:29:09.348629+0000","last_became_peered":"2023-11-28T18:29:09.348629+0000","last_unstale":"2023-11-28T18:29:10.381513+0000","last_undegraded":"2023-11-28T18:29:10.381513+0000","last_fullsized":"2023-11-28T18:29:10.381513+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:02.067920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.380792+0000","last_change":"2023-11-28T18:29:10.375115+0000","last_active":"2023-11-28T18:29:10.380792+0000","last_peered":"2023-11-28T18:29:10.380792+0000","last_clean":"2023-11-28T18:29:10.380792+0000","last_became_active":"2023-11-28T18:29:10.374848+0000","last_became_peered":"2023-11-28T18:29:10.374848+0000","last_unstale":"2023-11-28T18:29:10.380792+0000","last_undegraded":"2023-11-28T18:29:10.380792+0000","last_fullsized":"2023-11-28T18:29:10.380792+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:27:01.017033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"89'1","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.374332+0000","last_change":"2023-11-28T18:29:09.335618+0000","last_active":"2023-11-28T18:29:10.374332+0000","last_peered":"2023-11-28T18:29:10.374332+0000","last_clean":"2023-11-28T18:29:10.374332+0000","last_became_active":"2023-11-28T18:29:09.335239+0000","last_became_peered":"2023-11-28T18:29:09.335239+0000","last_unstale":"2023-11-28T18:29:10.374332+0000","last_undegraded":"2023-11-28T18:29:10.374332+0000","last_fullsized":"2023-11-28T18:29:10.374332+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:58.161257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.390864+0000","last_change":"2023-11-28T18:29:10.390248+0000","last_active":"2023-11-28T18:29:10.390864+0000","last_peered":"2023-11-28T18:29:10.390864+0000","last_clean":"2023-11-28T18:29:10.390864+0000","last_became_active":"2023-11-28T18:29:10.389979+0000","last_became_peered":"2023-11-28T18:29:10.389979+0000","last_unstale":"2023-11-28T18:29:10.390864+0000","last_undegraded":"2023-11-28T18:29:10.390864+0000","last_fullsized":"2023-11-28T18:29:10.390864+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:32:45.165863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.379250+0000","last_change":"2023-11-28T18:29:09.348524+0000","last_active":"2023-11-28T18:29:10.379250+0000","last_peered":"2023-11-28T18:29:10.379250+0000","last_clean":"2023-11-28T18:29:10.379250+0000","last_became_active":"2023-11-28T18:29:09.348110+0000","last_became_peered":"2023-11-28T18:29:09.348110+0000","last_unstale":"2023-11-28T18:29:10.379250+0000","last_undegraded":"2023-11-28T18:29:10.379250+0000","last_fullsized":"2023-11-28T18:29:10.379250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:28:20.632607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.379756+0000","last_change":"2023-11-28T18:29:10.370018+0000","last_active":"2023-11-28T18:29:10.379756+0000","last_peered":"2023-11-28T18:29:10.379756+0000","last_clean":"2023-11-28T18:29:10.379756+0000","last_became_active":"2023-11-28T18:29:10.369718+0000","last_became_peered":"2023-11-28T18:29:10.369718+0000","last_unstale":"2023-11-28T18:29:10.379756+0000","last_undegraded":"2023-11-28T18:29:10.379756+0000","last_fullsized":"2023-11-28T18:29:10.379756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:32:05.253712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.392150+0000","last_change":"2023-11-28T18:29:09.340583+0000","last_active":"2023-11-28T18:29:10.392150+0000","last_peered":"2023-11-28T18:29:10.392150+0000","last_clean":"2023-11-28T18:29:10.392150+0000","last_became_active":"2023-11-28T18:29:09.340310+0000","last_became_peered":"2023-11-28T18:29:09.340310+0000","last_unstale":"2023-11-28T18:29:10.392150+0000","last_undegraded":"2023-11-28T18:29:10.392150+0000","last_fullsized":"2023-11-28T18:29:10.392150+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:16:05.028565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.389182+0000","last_change":"2023-11-28T18:29:10.369304+0000","last_active":"2023-11-28T18:29:10.389182+0000","last_peered":"2023-11-28T18:29:10.389182+0000","last_clean":"2023-11-28T18:29:10.389182+0000","last_became_active":"2023-11-28T18:29:10.368664+0000","last_became_peered":"2023-11-28T18:29:10.368664+0000","last_unstale":"2023-11-28T18:29:10.389182+0000","last_undegraded":"2023-11-28T18:29:10.389182+0000","last_fullsized":"2023-11-28T18:29:10.389182+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:03:30.953517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"89'1","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.379067+0000","last_change":"2023-11-28T18:29:09.336367+0000","last_active":"2023-11-28T18:29:10.379067+0000","last_peered":"2023-11-28T18:29:10.379067+0000","last_clean":"2023-11-28T18:29:10.379067+0000","last_became_active":"2023-11-28T18:29:09.334465+0000","last_became_peered":"2023-11-28T18:29:09.334465+0000","last_unstale":"2023-11-28T18:29:10.379067+0000","last_undegraded":"2023-11-28T18:29:10.379067+0000","last_fullsized":"2023-11-28T18:29:10.379067+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:16:05.028565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.378648+0000","last_change":"2023-11-28T18:29:10.376841+0000","last_active":"2023-11-28T18:29:10.378648+0000","last_peered":"2023-11-28T18:29:10.378648+0000","last_clean":"2023-11-28T18:29:10.378648+0000","last_became_active":"2023-11-28T18:29:10.376532+0000","last_became_peered":"2023-11-28T18:29:10.376532+0000","last_unstale":"2023-11-28T18:29:10.378648+0000","last_undegraded":"2023-11-28T18:29:10.378648+0000","last_fullsized":"2023-11-28T18:29:10.378648+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:54:33.106050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"89'1","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.389370+0000","last_change":"2023-11-28T18:29:09.351252+0000","last_active":"2023-11-28T18:29:10.389370+0000","last_peered":"2023-11-28T18:29:10.389370+0000","last_clean":"2023-11-28T18:29:10.389370+0000","last_became_active":"2023-11-28T18:29:09.350966+0000","last_became_peered":"2023-11-28T18:29:09.350966+0000","last_unstale":"2023-11-28T18:29:10.389370+0000","last_undegraded":"2023-11-28T18:29:10.389370+0000","last_fullsized":"2023-11-28T18:29:10.389370+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:58.161257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.392755+0000","last_change":"2023-11-28T18:29:10.390353+0000","last_active":"2023-11-28T18:29:10.392755+0000","last_peered":"2023-11-28T18:29:10.392755+0000","last_clean":"2023-11-28T18:29:10.392755+0000","last_became_active":"2023-11-28T18:29:10.390084+0000","last_became_peered":"2023-11-28T18:29:10.390084+0000","last_unstale":"2023-11-28T18:29:10.392755+0000","last_undegraded":"2023-11-28T18:29:10.392755+0000","last_fullsized":"2023-11-28T18:29:10.392755+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:44:39.461779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.347681+0000","last_change":"2023-11-28T18:29:09.342809+0000","last_active":"2023-11-28T18:29:10.347681+0000","last_peered":"2023-11-28T18:29:10.347681+0000","last_clean":"2023-11-28T18:29:10.347681+0000","last_became_active":"2023-11-28T18:29:09.342104+0000","last_became_peered":"2023-11-28T18:29:09.342104+0000","last_unstale":"2023-11-28T18:29:10.347681+0000","last_undegraded":"2023-11-28T18:29:10.347681+0000","last_fullsized":"2023-11-28T18:29:10.347681+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:25.521938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.391350+0000","last_change":"2023-11-28T18:29:10.387798+0000","last_active":"2023-11-28T18:29:10.391350+0000","last_peered":"2023-11-28T18:29:10.391350+0000","last_clean":"2023-11-28T18:29:10.391350+0000","last_became_active":"2023-11-28T18:29:10.387433+0000","last_became_peered":"2023-11-28T18:29:10.387433+0000","last_unstale":"2023-11-28T18:29:10.391350+0000","last_undegraded":"2023-11-28T18:29:10.391350+0000","last_fullsized":"2023-11-28T18:29:10.391350+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:04:40.397373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.352947+0000","last_change":"2023-11-28T18:29:09.349940+0000","last_active":"2023-11-28T18:29:10.352947+0000","last_peered":"2023-11-28T18:29:10.352947+0000","last_clean":"2023-11-28T18:29:10.352947+0000","last_became_active":"2023-11-28T18:29:09.349402+0000","last_became_peered":"2023-11-28T18:29:09.349402+0000","last_unstale":"2023-11-28T18:29:10.352947+0000","last_undegraded":"2023-11-28T18:29:10.352947+0000","last_fullsized":"2023-11-28T18:29:10.352947+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:16:31.549066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.360634+0000","last_change":"2023-11-28T18:29:10.355330+0000","last_active":"2023-11-28T18:29:10.360634+0000","last_peered":"2023-11-28T18:29:10.360634+0000","last_clean":"2023-11-28T18:29:10.360634+0000","last_became_active":"2023-11-28T18:29:10.354907+0000","last_became_peered":"2023-11-28T18:29:10.354907+0000","last_unstale":"2023-11-28T18:29:10.360634+0000","last_undegraded":"2023-11-28T18:29:10.360634+0000","last_fullsized":"2023-11-28T18:29:10.360634+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:03:30.953517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.354254+0000","last_change":"2023-11-28T18:29:09.349813+0000","last_active":"2023-11-28T18:29:10.354254+0000","last_peered":"2023-11-28T18:29:10.354254+0000","last_clean":"2023-11-28T18:29:10.354254+0000","last_became_active":"2023-11-28T18:29:09.349478+0000","last_became_peered":"2023-11-28T18:29:09.349478+0000","last_unstale":"2023-11-28T18:29:10.354254+0000","last_undegraded":"2023-11-28T18:29:10.354254+0000","last_fullsized":"2023-11-28T18:29:10.354254+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:15:03.432197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.373080+0000","last_change":"2023-11-28T18:29:10.366671+0000","last_active":"2023-11-28T18:29:10.373080+0000","last_peered":"2023-11-28T18:29:10.373080+0000","last_clean":"2023-11-28T18:29:10.373080+0000","last_became_active":"2023-11-28T18:29:10.365593+0000","last_became_peered":"2023-11-28T18:29:10.365593+0000","last_unstale":"2023-11-28T18:29:10.373080+0000","last_undegraded":"2023-11-28T18:29:10.373080+0000","last_fullsized":"2023-11-28T18:29:10.373080+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:08:16.247758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.348449+0000","last_change":"2023-11-28T18:29:09.344090+0000","last_active":"2023-11-28T18:29:10.348449+0000","last_peered":"2023-11-28T18:29:10.348449+0000","last_clean":"2023-11-28T18:29:10.348449+0000","last_became_active":"2023-11-28T18:29:09.342928+0000","last_became_peered":"2023-11-28T18:29:09.342928+0000","last_unstale":"2023-11-28T18:29:10.348449+0000","last_undegraded":"2023-11-28T18:29:10.348449+0000","last_fullsized":"2023-11-28T18:29:10.348449+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:57.839659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.386388+0000","last_change":"2023-11-28T18:29:10.378740+0000","last_active":"2023-11-28T18:29:10.386388+0000","last_peered":"2023-11-28T18:29:10.386388+0000","last_clean":"2023-11-28T18:29:10.386388+0000","last_became_active":"2023-11-28T18:29:10.378467+0000","last_became_peered":"2023-11-28T18:29:10.378467+0000","last_unstale":"2023-11-28T18:29:10.386388+0000","last_undegraded":"2023-11-28T18:29:10.386388+0000","last_fullsized":"2023-11-28T18:29:10.386388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:43:41.111063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"89'3","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.381846+0000","last_change":"2023-11-28T18:29:09.341105+0000","last_active":"2023-11-28T18:29:10.381846+0000","last_peered":"2023-11-28T18:29:10.381846+0000","last_clean":"2023-11-28T18:29:10.381846+0000","last_became_active":"2023-11-28T18:29:09.340804+0000","last_became_peered":"2023-11-28T18:29:09.340804+0000","last_unstale":"2023-11-28T18:29:10.381846+0000","last_undegraded":"2023-11-28T18:29:10.381846+0000","last_fullsized":"2023-11-28T18:29:10.381846+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:35.989485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.389009+0000","last_change":"2023-11-28T18:29:10.367211+0000","last_active":"2023-11-28T18:29:10.389009+0000","last_peered":"2023-11-28T18:29:10.389009+0000","last_clean":"2023-11-28T18:29:10.389009+0000","last_became_active":"2023-11-28T18:29:10.366930+0000","last_became_peered":"2023-11-28T18:29:10.366930+0000","last_unstale":"2023-11-28T18:29:10.389009+0000","last_undegraded":"2023-11-28T18:29:10.389009+0000","last_fullsized":"2023-11-28T18:29:10.389009+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:19.201563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"89'6","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.385095+0000","last_change":"2023-11-28T18:29:09.346412+0000","last_active":"2023-11-28T18:29:10.385095+0000","last_peered":"2023-11-28T18:29:10.385095+0000","last_clean":"2023-11-28T18:29:10.385095+0000","last_became_active":"2023-11-28T18:29:09.345775+0000","last_became_peered":"2023-11-28T18:29:09.345775+0000","last_unstale":"2023-11-28T18:29:10.385095+0000","last_undegraded":"2023-11-28T18:29:10.385095+0000","last_fullsized":"2023-11-28T18:29:10.385095+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:35:27.985961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.389135+0000","last_change":"2023-11-28T18:29:10.368923+0000","last_active":"2023-11-28T18:29:10.389135+0000","last_peered":"2023-11-28T18:29:10.389135+0000","last_clean":"2023-11-28T18:29:10.389135+0000","last_became_active":"2023-11-28T18:29:10.368091+0000","last_became_peered":"2023-11-28T18:29:10.368091+0000","last_unstale":"2023-11-28T18:29:10.389135+0000","last_undegraded":"2023-11-28T18:29:10.389135+0000","last_fullsized":"2023-11-28T18:29:10.389135+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:25:43.024884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.382143+0000","last_change":"2023-11-28T18:29:09.349418+0000","last_active":"2023-11-28T18:29:10.382143+0000","last_peered":"2023-11-28T18:29:10.382143+0000","last_clean":"2023-11-28T18:29:10.382143+0000","last_became_active":"2023-11-28T18:29:09.349141+0000","last_became_peered":"2023-11-28T18:29:09.349141+0000","last_unstale":"2023-11-28T18:29:10.382143+0000","last_undegraded":"2023-11-28T18:29:10.382143+0000","last_fullsized":"2023-11-28T18:29:10.382143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:27:14.392678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.370525+0000","last_change":"2023-11-28T18:29:10.361916+0000","last_active":"2023-11-28T18:29:10.370525+0000","last_peered":"2023-11-28T18:29:10.370525+0000","last_clean":"2023-11-28T18:29:10.370525+0000","last_became_active":"2023-11-28T18:29:10.361611+0000","last_became_peered":"2023-11-28T18:29:10.361611+0000","last_unstale":"2023-11-28T18:29:10.370525+0000","last_undegraded":"2023-11-28T18:29:10.370525+0000","last_fullsized":"2023-11-28T18:29:10.370525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:32:12.486586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.360852+0000","last_change":"2023-11-28T18:29:09.335403+0000","last_active":"2023-11-28T18:29:10.360852+0000","last_peered":"2023-11-28T18:29:10.360852+0000","last_clean":"2023-11-28T18:29:10.360852+0000","last_became_active":"2023-11-28T18:29:09.335093+0000","last_became_peered":"2023-11-28T18:29:09.335093+0000","last_unstale":"2023-11-28T18:29:10.360852+0000","last_undegraded":"2023-11-28T18:29:10.360852+0000","last_fullsized":"2023-11-28T18:29:10.360852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:47:46.164388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.393342+0000","last_change":"2023-11-28T18:29:10.391989+0000","last_active":"2023-11-28T18:29:10.393342+0000","last_peered":"2023-11-28T18:29:10.393342+0000","last_clean":"2023-11-28T18:29:10.393342+0000","last_became_active":"2023-11-28T18:29:10.391678+0000","last_became_peered":"2023-11-28T18:29:10.391678+0000","last_unstale":"2023-11-28T18:29:10.393342+0000","last_undegraded":"2023-11-28T18:29:10.393342+0000","last_fullsized":"2023-11-28T18:29:10.393342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:10:43.169437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.358611+0000","last_change":"2023-11-28T18:29:09.345218+0000","last_active":"2023-11-28T18:29:10.358611+0000","last_peered":"2023-11-28T18:29:10.358611+0000","last_clean":"2023-11-28T18:29:10.358611+0000","last_became_active":"2023-11-28T18:29:09.344920+0000","last_became_peered":"2023-11-28T18:29:09.344920+0000","last_unstale":"2023-11-28T18:29:10.358611+0000","last_undegraded":"2023-11-28T18:29:10.358611+0000","last_fullsized":"2023-11-28T18:29:10.358611+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:16.178469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.380424+0000","last_change":"2023-11-28T18:29:10.374300+0000","last_active":"2023-11-28T18:29:10.380424+0000","last_peered":"2023-11-28T18:29:10.380424+0000","last_clean":"2023-11-28T18:29:10.380424+0000","last_became_active":"2023-11-28T18:29:10.374070+0000","last_became_peered":"2023-11-28T18:29:10.374070+0000","last_unstale":"2023-11-28T18:29:10.380424+0000","last_undegraded":"2023-11-28T18:29:10.380424+0000","last_fullsized":"2023-11-28T18:29:10.380424+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:57.905089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"89'8","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.391774+0000","last_change":"2023-11-28T18:29:09.350701+0000","last_active":"2023-11-28T18:29:10.391774+0000","last_peered":"2023-11-28T18:29:10.391774+0000","last_clean":"2023-11-28T18:29:10.391774+0000","last_became_active":"2023-11-28T18:29:09.350408+0000","last_became_peered":"2023-11-28T18:29:09.350408+0000","last_unstale":"2023-11-28T18:29:10.391774+0000","last_undegraded":"2023-11-28T18:29:10.391774+0000","last_fullsized":"2023-11-28T18:29:10.391774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:35.989485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.382226+0000","last_change":"2023-11-28T18:29:10.379194+0000","last_active":"2023-11-28T18:29:10.382226+0000","last_peered":"2023-11-28T18:29:10.382226+0000","last_clean":"2023-11-28T18:29:10.382226+0000","last_became_active":"2023-11-28T18:29:10.378563+0000","last_became_peered":"2023-11-28T18:29:10.378563+0000","last_unstale":"2023-11-28T18:29:10.382226+0000","last_undegraded":"2023-11-28T18:29:10.382226+0000","last_fullsized":"2023-11-28T18:29:10.382226+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:29:12.192039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"89'1","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.372109+0000","last_change":"2023-11-28T18:29:09.342440+0000","last_active":"2023-11-28T18:29:10.372109+0000","last_peered":"2023-11-28T18:29:10.372109+0000","last_clean":"2023-11-28T18:29:10.372109+0000","last_became_active":"2023-11-28T18:29:09.342157+0000","last_became_peered":"2023-11-28T18:29:09.342157+0000","last_unstale":"2023-11-28T18:29:10.372109+0000","last_undegraded":"2023-11-28T18:29:10.372109+0000","last_fullsized":"2023-11-28T18:29:10.372109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:16.178469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.373530+0000","last_change":"2023-11-28T18:29:10.370630+0000","last_active":"2023-11-28T18:29:10.373530+0000","last_peered":"2023-11-28T18:29:10.373530+0000","last_clean":"2023-11-28T18:29:10.373530+0000","last_became_active":"2023-11-28T18:29:10.370343+0000","last_became_peered":"2023-11-28T18:29:10.370343+0000","last_unstale":"2023-11-28T18:29:10.373530+0000","last_undegraded":"2023-11-28T18:29:10.373530+0000","last_fullsized":"2023-11-28T18:29:10.373530+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:19.201563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.373454+0000","last_change":"2023-11-28T18:29:09.346424+0000","last_active":"2023-11-28T18:29:10.373454+0000","last_peered":"2023-11-28T18:29:10.373454+0000","last_clean":"2023-11-28T18:29:10.373454+0000","last_became_active":"2023-11-28T18:29:09.346146+0000","last_became_peered":"2023-11-28T18:29:09.346146+0000","last_unstale":"2023-11-28T18:29:10.373454+0000","last_undegraded":"2023-11-28T18:29:10.373454+0000","last_fullsized":"2023-11-28T18:29:10.373454+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:32:55.197612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.378187+0000","last_change":"2023-11-28T18:29:10.376159+0000","last_active":"2023-11-28T18:29:10.378187+0000","last_peered":"2023-11-28T18:29:10.378187+0000","last_clean":"2023-11-28T18:29:10.378187+0000","last_became_active":"2023-11-28T18:29:10.375908+0000","last_became_peered":"2023-11-28T18:29:10.375908+0000","last_unstale":"2023-11-28T18:29:10.378187+0000","last_undegraded":"2023-11-28T18:29:10.378187+0000","last_fullsized":"2023-11-28T18:29:10.378187+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:19.201563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"89'1","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.392200+0000","last_change":"2023-11-28T18:29:09.340083+0000","last_active":"2023-11-28T18:29:10.392200+0000","last_peered":"2023-11-28T18:29:10.392200+0000","last_clean":"2023-11-28T18:29:10.392200+0000","last_became_active":"2023-11-28T18:29:09.338664+0000","last_became_peered":"2023-11-28T18:29:09.338664+0000","last_unstale":"2023-11-28T18:29:10.392200+0000","last_undegraded":"2023-11-28T18:29:10.392200+0000","last_fullsized":"2023-11-28T18:29:10.392200+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:02.067920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.391781+0000","last_change":"2023-11-28T18:29:10.390287+0000","last_active":"2023-11-28T18:29:10.391781+0000","last_peered":"2023-11-28T18:29:10.391781+0000","last_clean":"2023-11-28T18:29:10.391781+0000","last_became_active":"2023-11-28T18:29:10.389978+0000","last_became_peered":"2023-11-28T18:29:10.389978+0000","last_unstale":"2023-11-28T18:29:10.391781+0000","last_undegraded":"2023-11-28T18:29:10.391781+0000","last_fullsized":"2023-11-28T18:29:10.391781+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:41:02.593427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.373105+0000","last_change":"2023-11-28T18:29:09.343699+0000","last_active":"2023-11-28T18:29:10.373105+0000","last_peered":"2023-11-28T18:29:10.373105+0000","last_clean":"2023-11-28T18:29:10.373105+0000","last_became_active":"2023-11-28T18:29:09.342820+0000","last_became_peered":"2023-11-28T18:29:09.342820+0000","last_unstale":"2023-11-28T18:29:10.373105+0000","last_undegraded":"2023-11-28T18:29:10.373105+0000","last_fullsized":"2023-11-28T18:29:10.373105+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:43.287826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.372696+0000","last_change":"2023-11-28T18:29:10.365090+0000","last_active":"2023-11-28T18:29:10.372696+0000","last_peered":"2023-11-28T18:29:10.372696+0000","last_clean":"2023-11-28T18:29:10.372696+0000","last_became_active":"2023-11-28T18:29:10.364616+0000","last_became_peered":"2023-11-28T18:29:10.364616+0000","last_unstale":"2023-11-28T18:29:10.372696+0000","last_undegraded":"2023-11-28T18:29:10.372696+0000","last_fullsized":"2023-11-28T18:29:10.372696+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:50.642782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.365963+0000","last_change":"2023-11-28T18:29:09.346916+0000","last_active":"2023-11-28T18:29:10.365963+0000","last_peered":"2023-11-28T18:29:10.365963+0000","last_clean":"2023-11-28T18:29:10.365963+0000","last_became_active":"2023-11-28T18:29:09.345562+0000","last_became_peered":"2023-11-28T18:29:09.345562+0000","last_unstale":"2023-11-28T18:29:10.365963+0000","last_undegraded":"2023-11-28T18:29:10.365963+0000","last_fullsized":"2023-11-28T18:29:10.365963+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:19:28.889518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.380061+0000","last_change":"2023-11-28T18:29:10.379022+0000","last_active":"2023-11-28T18:29:10.380061+0000","last_peered":"2023-11-28T18:29:10.380061+0000","last_clean":"2023-11-28T18:29:10.380061+0000","last_became_active":"2023-11-28T18:29:10.378672+0000","last_became_peered":"2023-11-28T18:29:10.378672+0000","last_unstale":"2023-11-28T18:29:10.380061+0000","last_undegraded":"2023-11-28T18:29:10.380061+0000","last_fullsized":"2023-11-28T18:29:10.380061+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:30:42.927271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.354042+0000","last_change":"2023-11-28T18:29:09.348033+0000","last_active":"2023-11-28T18:29:10.354042+0000","last_peered":"2023-11-28T18:29:10.354042+0000","last_clean":"2023-11-28T18:29:10.354042+0000","last_became_active":"2023-11-28T18:29:09.347758+0000","last_became_peered":"2023-11-28T18:29:09.347758+0000","last_unstale":"2023-11-28T18:29:10.354042+0000","last_undegraded":"2023-11-28T18:29:10.354042+0000","last_fullsized":"2023-11-28T18:29:10.354042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:57.839659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.381279+0000","last_change":"2023-11-28T18:29:10.376545+0000","last_active":"2023-11-28T18:29:10.381279+0000","last_peered":"2023-11-28T18:29:10.381279+0000","last_clean":"2023-11-28T18:29:10.381279+0000","last_became_active":"2023-11-28T18:29:10.374076+0000","last_became_peered":"2023-11-28T18:29:10.374076+0000","last_unstale":"2023-11-28T18:29:10.381279+0000","last_undegraded":"2023-11-28T18:29:10.381279+0000","last_fullsized":"2023-11-28T18:29:10.381279+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:32:05.660491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.364698+0000","last_change":"2023-11-28T18:29:09.344177+0000","last_active":"2023-11-28T18:29:10.364698+0000","last_peered":"2023-11-28T18:29:10.364698+0000","last_clean":"2023-11-28T18:29:10.364698+0000","last_became_active":"2023-11-28T18:29:09.341608+0000","last_became_peered":"2023-11-28T18:29:09.341608+0000","last_unstale":"2023-11-28T18:29:10.364698+0000","last_undegraded":"2023-11-28T18:29:10.364698+0000","last_fullsized":"2023-11-28T18:29:10.364698+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:33.812731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.367687+0000","last_change":"2023-11-28T18:29:10.360366+0000","last_active":"2023-11-28T18:29:10.367687+0000","last_peered":"2023-11-28T18:29:10.367687+0000","last_clean":"2023-11-28T18:29:10.367687+0000","last_became_active":"2023-11-28T18:29:10.360087+0000","last_became_peered":"2023-11-28T18:29:10.360087+0000","last_unstale":"2023-11-28T18:29:10.367687+0000","last_undegraded":"2023-11-28T18:29:10.367687+0000","last_fullsized":"2023-11-28T18:29:10.367687+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:09:48.512954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.373872+0000","last_change":"2023-11-28T18:29:09.347067+0000","last_active":"2023-11-28T18:29:10.373872+0000","last_peered":"2023-11-28T18:29:10.373872+0000","last_clean":"2023-11-28T18:29:10.373872+0000","last_became_active":"2023-11-28T18:29:09.346647+0000","last_became_peered":"2023-11-28T18:29:09.346647+0000","last_unstale":"2023-11-28T18:29:10.373872+0000","last_undegraded":"2023-11-28T18:29:10.373872+0000","last_fullsized":"2023-11-28T18:29:10.373872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:11:06.712338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.380257+0000","last_change":"2023-11-28T18:29:10.372893+0000","last_active":"2023-11-28T18:29:10.380257+0000","last_peered":"2023-11-28T18:29:10.380257+0000","last_clean":"2023-11-28T18:29:10.380257+0000","last_became_active":"2023-11-28T18:29:10.372630+0000","last_became_peered":"2023-11-28T18:29:10.372630+0000","last_unstale":"2023-11-28T18:29:10.380257+0000","last_undegraded":"2023-11-28T18:29:10.380257+0000","last_fullsized":"2023-11-28T18:29:10.380257+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:14:36.140322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.354258+0000","last_change":"2023-11-28T18:29:09.342072+0000","last_active":"2023-11-28T18:29:10.354258+0000","last_peered":"2023-11-28T18:29:10.354258+0000","last_clean":"2023-11-28T18:29:10.354258+0000","last_became_active":"2023-11-28T18:29:09.341510+0000","last_became_peered":"2023-11-28T18:29:09.341510+0000","last_unstale":"2023-11-28T18:29:10.354258+0000","last_undegraded":"2023-11-28T18:29:10.354258+0000","last_fullsized":"2023-11-28T18:29:10.354258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:59:35.230058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.382121+0000","last_change":"2023-11-28T18:29:10.377372+0000","last_active":"2023-11-28T18:29:10.382121+0000","last_peered":"2023-11-28T18:29:10.382121+0000","last_clean":"2023-11-28T18:29:10.382121+0000","last_became_active":"2023-11-28T18:29:10.377105+0000","last_became_peered":"2023-11-28T18:29:10.377105+0000","last_unstale":"2023-11-28T18:29:10.382121+0000","last_undegraded":"2023-11-28T18:29:10.382121+0000","last_fullsized":"2023-11-28T18:29:10.382121+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:28:01.196337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.354115+0000","last_change":"2023-11-28T18:29:09.340320+0000","last_active":"2023-11-28T18:29:10.354115+0000","last_peered":"2023-11-28T18:29:10.354115+0000","last_clean":"2023-11-28T18:29:10.354115+0000","last_became_active":"2023-11-28T18:29:09.339977+0000","last_became_peered":"2023-11-28T18:29:09.339977+0000","last_unstale":"2023-11-28T18:29:10.354115+0000","last_undegraded":"2023-11-28T18:29:10.354115+0000","last_fullsized":"2023-11-28T18:29:10.354115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:16.178469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.365937+0000","last_change":"2023-11-28T18:29:10.358121+0000","last_active":"2023-11-28T18:29:10.365937+0000","last_peered":"2023-11-28T18:29:10.365937+0000","last_clean":"2023-11-28T18:29:10.365937+0000","last_became_active":"2023-11-28T18:29:10.357799+0000","last_became_peered":"2023-11-28T18:29:10.357799+0000","last_unstale":"2023-11-28T18:29:10.365937+0000","last_undegraded":"2023-11-28T18:29:10.365937+0000","last_fullsized":"2023-11-28T18:29:10.365937+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:11:50.704320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.357175+0000","last_change":"2023-11-28T18:29:09.338189+0000","last_active":"2023-11-28T18:29:10.357175+0000","last_peered":"2023-11-28T18:29:10.357175+0000","last_clean":"2023-11-28T18:29:10.357175+0000","last_became_active":"2023-11-28T18:29:09.337447+0000","last_became_peered":"2023-11-28T18:29:09.337447+0000","last_unstale":"2023-11-28T18:29:10.357175+0000","last_undegraded":"2023-11-28T18:29:10.357175+0000","last_fullsized":"2023-11-28T18:29:10.357175+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:35.989485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.380210+0000","last_change":"2023-11-28T18:29:10.375886+0000","last_active":"2023-11-28T18:29:10.380210+0000","last_peered":"2023-11-28T18:29:10.380210+0000","last_clean":"2023-11-28T18:29:10.380210+0000","last_became_active":"2023-11-28T18:29:10.375232+0000","last_became_peered":"2023-11-28T18:29:10.375232+0000","last_unstale":"2023-11-28T18:29:10.380210+0000","last_undegraded":"2023-11-28T18:29:10.380210+0000","last_fullsized":"2023-11-28T18:29:10.380210+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:43:40.528137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.357726+0000","last_change":"2023-11-28T18:29:09.346493+0000","last_active":"2023-11-28T18:29:10.357726+0000","last_peered":"2023-11-28T18:29:10.357726+0000","last_clean":"2023-11-28T18:29:10.357726+0000","last_became_active":"2023-11-28T18:29:09.346237+0000","last_became_peered":"2023-11-28T18:29:09.346237+0000","last_unstale":"2023-11-28T18:29:10.357726+0000","last_undegraded":"2023-11-28T18:29:10.357726+0000","last_fullsized":"2023-11-28T18:29:10.357726+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:48.883072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.375368+0000","last_change":"2023-11-28T18:29:10.371516+0000","last_active":"2023-11-28T18:29:10.375368+0000","last_peered":"2023-11-28T18:29:10.375368+0000","last_clean":"2023-11-28T18:29:10.375368+0000","last_became_active":"2023-11-28T18:29:10.371170+0000","last_became_peered":"2023-11-28T18:29:10.371170+0000","last_unstale":"2023-11-28T18:29:10.375368+0000","last_undegraded":"2023-11-28T18:29:10.375368+0000","last_fullsized":"2023-11-28T18:29:10.375368+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:50.642782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.352398+0000","last_change":"2023-11-28T18:29:09.343728+0000","last_active":"2023-11-28T18:29:10.352398+0000","last_peered":"2023-11-28T18:29:10.352398+0000","last_clean":"2023-11-28T18:29:10.352398+0000","last_became_active":"2023-11-28T18:29:09.343451+0000","last_became_peered":"2023-11-28T18:29:09.343451+0000","last_unstale":"2023-11-28T18:29:10.352398+0000","last_undegraded":"2023-11-28T18:29:10.352398+0000","last_fullsized":"2023-11-28T18:29:10.352398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:43:40.103435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.393321+0000","last_change":"2023-11-28T18:29:10.392408+0000","last_active":"2023-11-28T18:29:10.393321+0000","last_peered":"2023-11-28T18:29:10.393321+0000","last_clean":"2023-11-28T18:29:10.393321+0000","last_became_active":"2023-11-28T18:29:10.392143+0000","last_became_peered":"2023-11-28T18:29:10.392143+0000","last_unstale":"2023-11-28T18:29:10.393321+0000","last_undegraded":"2023-11-28T18:29:10.393321+0000","last_fullsized":"2023-11-28T18:29:10.393321+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:04:40.397373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.357325+0000","last_change":"2023-11-28T18:29:09.336697+0000","last_active":"2023-11-28T18:29:10.357325+0000","last_peered":"2023-11-28T18:29:10.357325+0000","last_clean":"2023-11-28T18:29:10.357325+0000","last_became_active":"2023-11-28T18:29:09.335301+0000","last_became_peered":"2023-11-28T18:29:09.335301+0000","last_unstale":"2023-11-28T18:29:10.357325+0000","last_undegraded":"2023-11-28T18:29:10.357325+0000","last_fullsized":"2023-11-28T18:29:10.357325+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:58.161257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.381152+0000","last_change":"2023-11-28T18:29:10.371910+0000","last_active":"2023-11-28T18:29:10.381152+0000","last_peered":"2023-11-28T18:29:10.381152+0000","last_clean":"2023-11-28T18:29:10.381152+0000","last_became_active":"2023-11-28T18:29:10.371626+0000","last_became_peered":"2023-11-28T18:29:10.371626+0000","last_unstale":"2023-11-28T18:29:10.381152+0000","last_undegraded":"2023-11-28T18:29:10.381152+0000","last_fullsized":"2023-11-28T18:29:10.381152+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:36:53.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.353094+0000","last_change":"2023-11-28T18:29:09.346884+0000","last_active":"2023-11-28T18:29:10.353094+0000","last_peered":"2023-11-28T18:29:10.353094+0000","last_clean":"2023-11-28T18:29:10.353094+0000","last_became_active":"2023-11-28T18:29:09.345946+0000","last_became_peered":"2023-11-28T18:29:09.345946+0000","last_unstale":"2023-11-28T18:29:10.353094+0000","last_undegraded":"2023-11-28T18:29:10.353094+0000","last_fullsized":"2023-11-28T18:29:10.353094+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:21:04.912542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.367838+0000","last_change":"2023-11-28T18:29:10.363077+0000","last_active":"2023-11-28T18:29:10.367838+0000","last_peered":"2023-11-28T18:29:10.367838+0000","last_clean":"2023-11-28T18:29:10.367838+0000","last_became_active":"2023-11-28T18:29:10.362767+0000","last_became_peered":"2023-11-28T18:29:10.362767+0000","last_unstale":"2023-11-28T18:29:10.367838+0000","last_undegraded":"2023-11-28T18:29:10.367838+0000","last_fullsized":"2023-11-28T18:29:10.367838+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:40.038310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.362527+0000","last_change":"2023-11-28T18:29:09.347146+0000","last_active":"2023-11-28T18:29:10.362527+0000","last_peered":"2023-11-28T18:29:10.362527+0000","last_clean":"2023-11-28T18:29:10.362527+0000","last_became_active":"2023-11-28T18:29:09.346864+0000","last_became_peered":"2023-11-28T18:29:09.346864+0000","last_unstale":"2023-11-28T18:29:10.362527+0000","last_undegraded":"2023-11-28T18:29:10.362527+0000","last_fullsized":"2023-11-28T18:29:10.362527+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:59:35.230058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":225,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-28T18:29:09.335981+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:29:09.335981+0000","last_peered":"2023-11-28T18:29:09.335981+0000","last_clean":"2023-11-28T18:29:09.335981+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:29:09.335981+0000","last_undegraded":"2023-11-28T18:29:09.335981+0000","last_fullsized":"2023-11-28T18:29:09.335981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46732,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325108,"statfs":{"total":96636764160,"available":96588910592,"internally_reserved":0,"allocated":1699840,"data_stored":1095205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.64200000000000002,"15min":0.71299999999999997},"min":{"1min":0.38600000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.94899999999999995,"5min":2.427,"15min":2.427},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.71499999999999997},"min":{"1min":0.39500000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.93600000000000005,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.64400000000000002}]},{"osd":1,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66200000000000003,"15min":0.71499999999999997},"min":{"1min":0.44500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88400000000000001,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68200000000000005,"15min":0.75},"min":{"1min":0.46100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":1.079,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.80900000000000005}]},{"osd":2,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.69899999999999995,"15min":0.76900000000000002},"min":{"1min":0.42799999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.998,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67600000000000005,"15min":0.73799999999999999},"min":{"1min":0.41299999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.97299999999999998,"5min":2.411,"15min":2.411},"last":0.78800000000000003}]},{"osd":3,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.67800000000000005,"15min":0.755},"min":{"1min":0.40899999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92000000000000004,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72499999999999998,"15min":0.79600000000000004},"min":{"1min":0.49299999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.0389999999999999,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.0289999999999999}]},{"osd":4,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58299999999999996,"15min":0.60199999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83699999999999997,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58899999999999997,"15min":0.63700000000000001},"min":{"1min":0.34899999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.96299999999999997}]},{"osd":5,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.67300000000000004},"min":{"1min":0.38800000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.65000000000000002,"15min":0.69599999999999995},"min":{"1min":0.443,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92800000000000005,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.88300000000000001}]},{"osd":6,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69399999999999995,"15min":0.73799999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.998,"5min":1.042,"15min":1.042},"last":0.996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.65100000000000002,"15min":0.72299999999999998},"min":{"1min":0.40999999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":1.01}]}]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40564,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331276,"statfs":{"total":96636764160,"available":96595226624,"internally_reserved":0,"allocated":1085440,"data_stored":497686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64100000000000001,"15min":0.63700000000000001},"min":{"1min":0.42099999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.86299999999999999,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.71399999999999997,"15min":0.72999999999999998},"min":{"1min":0.49299999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.751}]},{"osd":1,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68799999999999994,"15min":0.79600000000000004},"min":{"1min":0.46800000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96899999999999997,"5min":4.2389999999999999,"15min":4.2389999999999999},"last":0.624},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70899999999999996,"15min":0.78600000000000003},"min":{"1min":0.39500000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.94499999999999995,"5min":4.218,"15min":4.218},"last":0.69099999999999995}]},{"osd":2,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67700000000000005,"15min":0.71299999999999997},"min":{"1min":0.373,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84999999999999998,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67700000000000005,"15min":0.69999999999999996},"min":{"1min":0.48599999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.86099999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69599999999999995,"15min":0.72399999999999998},"min":{"1min":0.52600000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91900000000000004,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.68200000000000005,"15min":0.68799999999999994},"min":{"1min":0.50700000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91900000000000004,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.72999999999999998}]},{"osd":4,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.624,"15min":0.66200000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.63400000000000001,"15min":0.68200000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.80200000000000005}]},{"osd":5,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63400000000000001,"15min":0.67600000000000005},"min":{"1min":0.36199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.69999999999999996,"15min":0.754},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95099999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.94599999999999995}]},{"osd":7,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71299999999999997,"15min":0.76700000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.002,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.70399999999999996,"15min":0.746},"min":{"1min":0.443,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90200000000000002,"5min":1,"15min":1},"last":0.78800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574921,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":504781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69199999999999995,"15min":0.71099999999999997},"min":{"1min":0.34399999999999997,"5min":0.31,"15min":0.31},"max":{"1min":2.1299999999999999,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70799999999999996,"15min":0.748},"min":{"1min":0.52200000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89500000000000002,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":0.72699999999999998}]},{"osd":1,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.68600000000000005,"15min":0.73499999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92400000000000004,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72899999999999998,"15min":0.79500000000000004},"min":{"1min":0.45800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.2109999999999999,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.82699999999999996}]},{"osd":2,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67900000000000005,"15min":0.749},"min":{"1min":0.40899999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84599999999999997,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.624},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69299999999999995,"15min":0.76700000000000002},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.014,"5min":4.0149999999999997,"15min":4.0149999999999997},"last":0.65000000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.73999999999999999,"15min":0.82099999999999995},"min":{"1min":0.48999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.254,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71299999999999997,"15min":0.81499999999999995},"min":{"1min":0.46300000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.98599999999999999,"5min":3.9900000000000002,"15min":3.9900000000000002},"last":0.56200000000000006}]},{"osd":4,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61499999999999999,"15min":0.60899999999999999},"min":{"1min":0.32900000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68400000000000005,"15min":0.70799999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.68700000000000006}]},{"osd":6,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67300000000000004,"15min":0.67900000000000005},"min":{"1min":0.35499999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70299999999999996,"15min":0.71699999999999997},"min":{"1min":0.438,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.262,"5min":1.756,"15min":1.756},"last":0.747}]},{"osd":7,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.74299999999999999,"15min":0.755},"min":{"1min":0.42399999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.74199999999999999,"15min":0.754},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.76000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68360,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303480,"statfs":{"total":96636764160,"available":96566763520,"internally_reserved":0,"allocated":1105920,"data_stored":504941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63900000000000001,"15min":0.68400000000000005},"min":{"1min":0.32900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.82799999999999996,"5min":2.363,"15min":2.363},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.43099999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.73399999999999999}]},{"osd":1,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63600000000000001,"15min":0.625},"min":{"1min":0.441,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93200000000000005,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67100000000000004,"15min":0.65600000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.86799999999999999}]},{"osd":2,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71099999999999997,"15min":0.69899999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93100000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71199999999999997,"15min":0.72599999999999998},"min":{"1min":0.50900000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94399999999999995,"5min":1.871,"15min":1.871},"last":0.753}]},{"osd":3,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.46400000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67800000000000005,"15min":0.68300000000000005},"min":{"1min":0.47699999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96699999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.79300000000000004}]},{"osd":5,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.94299999999999995,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.443},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59499999999999997,"15min":0.63100000000000001},"min":{"1min":0.33400000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80300000000000005,"5min":1.79,"15min":1.79},"last":0.39100000000000001}]},{"osd":6,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61799999999999999,"15min":0.64300000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91500000000000004,"5min":1.123,"15min":1.123},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68400000000000005,"15min":0.70499999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0800000000000001,"5min":1.925,"15min":1.925},"last":0.64000000000000001}]},{"osd":7,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68000000000000005,"15min":0.71299999999999997},"min":{"1min":0.49399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88400000000000001,"5min":1.843,"15min":1.843},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.747,"15min":0.77400000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96299999999999997,"5min":1.593,"15min":1.593},"last":0.89000000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574921,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62784,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309056,"statfs":{"total":96636764160,"available":96572473344,"internally_reserved":0,"allocated":1097728,"data_stored":504781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.434,"15min":0.41699999999999998},"min":{"1min":0.27100000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.80900000000000005,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48899999999999999,"15min":0.47499999999999998},"min":{"1min":0.308,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.96399999999999997,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.41099999999999998}]},{"osd":1,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.45700000000000002,"15min":0.46300000000000002},"min":{"1min":0.28799999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.73399999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.54400000000000004,"15min":0.51600000000000001},"min":{"1min":0.318,"5min":0.247,"15min":0.247},"max":{"1min":0.75600000000000001,"5min":4.5599999999999996,"15min":4.5599999999999996},"last":0.433}]},{"osd":2,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54800000000000004,"15min":0.53000000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2110000000000001,"5min":2.48,"15min":2.48},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.437,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.68700000000000006,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.52000000000000002}]},{"osd":4,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.89400000000000002,"15min":0.81499999999999995},"min":{"1min":0.441,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0189999999999999,"5min":12.695,"15min":12.695},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.80900000000000005,"15min":0.78700000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.052,"5min":6.3250000000000002,"15min":6.3250000000000002},"last":0.56999999999999995}]},{"osd":5,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.78600000000000003,"15min":0.82799999999999996},"min":{"1min":0.49299999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0309999999999999,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.79900000000000004,"15min":0.78200000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0840000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.59399999999999997}]},{"osd":6,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.83699999999999997,"15min":0.81499999999999995},"min":{"1min":0.53100000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97199999999999998,"5min":6.1790000000000003,"15min":6.1790000000000003},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.79500000000000004,"15min":0.80200000000000005},"min":{"1min":0.53300000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0680000000000001,"5min":1.984,"15min":1.984},"last":0.64200000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.81299999999999994,"15min":0.81899999999999995},"min":{"1min":0.55400000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1319999999999999,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.84299999999999997,"15min":0.82299999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.008,"5min":6.2249999999999996,"15min":6.2249999999999996},"last":0.73199999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1097728,"data_stored":504781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.47799999999999998,"15min":0.5},"min":{"1min":0.28699999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.47099999999999997,"15min":0.503},"min":{"1min":0.29799999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66800000000000004,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.42499999999999999}]},{"osd":1,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.496,"15min":0.52200000000000002},"min":{"1min":0.36599999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70999999999999996,"5min":1.48,"15min":1.48},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.872,"5min":0.57799999999999996,"15min":0.53800000000000003},"min":{"1min":0.34200000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":8.3539999999999992,"5min":8.3539999999999992,"15min":8.3539999999999992},"last":0.38700000000000001}]},{"osd":3,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.501,"15min":0.50700000000000001},"min":{"1min":0.34499999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":3.1549999999999998,"5min":3.1549999999999998,"15min":3.1549999999999998},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.56200000000000006,"15min":0.57099999999999995},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.4790000000000001,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":0.40600000000000003}]},{"osd":4,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.77300000000000002,"15min":0.77500000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.5179999999999998,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.76000000000000001,"15min":0.77900000000000003},"min":{"1min":0.51700000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.8170000000000002,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.78600000000000003}]},{"osd":5,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.75800000000000001,"15min":0.77000000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.0709999999999997,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.80800000000000005,"15min":0.96999999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0780000000000001,"5min":7.4820000000000002,"15min":7.4820000000000002},"last":0.76500000000000001}]},{"osd":6,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":0.85599999999999998,"15min":0.82399999999999995},"min":{"1min":0.53100000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":12.019,"5min":12.019,"15min":12.019},"last":0.877},{"interface":"front","average":{"1min":1.1220000000000001,"5min":0.83999999999999997,"15min":0.876},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":8.7400000000000002,"5min":8.7400000000000002,"15min":8.7400000000000002},"last":0.71099999999999997}]},{"osd":7,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":0.85599999999999998,"15min":0.85399999999999998},"min":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":8.327,"5min":8.327,"15min":8.327},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.121,"5min":0.86099999999999999,"15min":0.92000000000000004},"min":{"1min":0.57499999999999996,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":8.609,"5min":8.609,"15min":8.609},"last":0.82899999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40588,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331252,"statfs":{"total":96636764160,"available":96595202048,"internally_reserved":0,"allocated":1101824,"data_stored":504851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.41199999999999998,"15min":0.42499999999999999},"min":{"1min":0.29999999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.56100000000000005,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.372},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.29899999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.65000000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.503}]},{"osd":2,"last update":"Tue Nov 28 18:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.44800000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67400000000000004,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.48599999999999999,"15min":0.47999999999999998},"min":{"1min":0.38800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.73399999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.41499999999999998}]},{"osd":3,"last update":"Tue Nov 28 18:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.74199999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.502,"15min":0.49199999999999999},"min":{"1min":0.34899999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71599999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.46600000000000003}]},{"osd":4,"last update":"Tue Nov 28 18:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72399999999999998,"15min":0.746},"min":{"1min":0.42699999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.032,"5min":1.155,"15min":1.155},"last":0.78200000000000003}]},{"osd":5,"last update":"Tue Nov 28 18:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71099999999999997,"15min":0.74299999999999999},"min":{"1min":0.54100000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93700000000000006,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.70999999999999996,"15min":0.74399999999999999},"min":{"1min":0.502,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.63300000000000001}]},{"osd":6,"last update":"Tue Nov 28 18:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.747,"15min":0.77500000000000002},"min":{"1min":0.57299999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0549999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76800000000000002,"15min":0.80300000000000005},"min":{"1min":0.55900000000000005,"5min":0.498,"15min":0.498},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.77100000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72699999999999998,"15min":0.748},"min":{"1min":0.56000000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0669999999999999,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.74099999999999999,"15min":0.77300000000000002},"min":{"1min":0.58399999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97599999999999998,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.64500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574921,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37028,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334812,"statfs":{"total":96636764160,"available":96598847488,"internally_reserved":0,"allocated":1716224,"data_stored":1097467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 18:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.41999999999999998,"15min":0.44},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.628,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.33000000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71299999999999997,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.55100000000000005}]},{"osd":2,"last update":"Tue Nov 28 18:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.48899999999999999,"15min":0.50600000000000001},"min":{"1min":0.34599999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.629,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.495,"15min":0.51300000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.754,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.496}]},{"osd":3,"last update":"Tue Nov 28 18:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47899999999999998,"15min":0.48899999999999999},"min":{"1min":0.32400000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78600000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.443},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52900000000000003,"15min":0.55100000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.69899999999999995,"5min":1.087,"15min":1.087},"last":0.51800000000000002}]},{"osd":4,"last update":"Tue Nov 28 18:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.56499999999999995,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86899999999999999,"5min":1.131,"15min":1.131},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.46899999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.995,"5min":1.244,"15min":1.244},"last":0.86099999999999999}]},{"osd":5,"last update":"Tue Nov 28 18:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72699999999999998,"15min":0.72799999999999998},"min":{"1min":0.48299999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.88600000000000001,"5min":1.121,"15min":1.121},"last":0.873},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.73699999999999999,"15min":0.74099999999999999},"min":{"1min":0.54700000000000004,"5min":0.502,"15min":0.502},"max":{"1min":0.97799999999999998,"5min":1.075,"15min":1.075},"last":0.71399999999999997}]},{"osd":6,"last update":"Tue Nov 28 18:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.72799999999999998,"15min":0.73299999999999998},"min":{"1min":0.51400000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.92900000000000005,"5min":1.407,"15min":1.407},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.74299999999999999,"15min":0.73699999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96099999999999997,"5min":1.089,"15min":1.089},"last":0.81999999999999995}]},{"osd":7,"last update":"Tue Nov 28 18:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.75700000000000001,"15min":0.755},"min":{"1min":0.56899999999999995,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.054,"5min":1.212,"15min":1.212},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.754,"15min":0.75},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.94699999999999995,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.84199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:29:14.445 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:29:14.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.618+0000 7fdb98993640 1 Processor -- start 2023-11-28T18:29:14.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.618+0000 7fdb98993640 1 -- start start 2023-11-28T18:29:14.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.618+0000 7fdb98993640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdb940d1c00 0x7fdb940ca000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:14.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.618+0000 7fdb98993640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdb940d3a50 0x7fdb940ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:14.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.618+0000 7fdb98993640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb940ce530 con 0x7fdb941607b0 2023-11-28T18:29:14.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.618+0000 7fdb98993640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb940d0eb0 con 0x7fdb940d3a50 2023-11-28T18:29:14.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.618+0000 7fdb98993640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb940d1150 con 0x7fdb940d1c00 2023-11-28T18:29:14.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.618+0000 7fdb937fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fdb941607b0 0x7fdb940d14f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57776/0 (socket says 172.21.15.105:57776) 2023-11-28T18:29:14.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.618+0000 7fdb937fe640 1 -- 172.21.15.105:0/513648535 learned_addr learned my addr 172.21.15.105:0/513648535 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:14.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.618+0000 7fdb93fff640 1 --2- 172.21.15.105:0/513648535 >> v2:172.21.15.148:3300/0 conn(0x7fdb940d3a50 0x7fdb940ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:14.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.618+0000 7fdb92ffd640 1 --2- 172.21.15.105:0/513648535 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdb940d1c00 0x7fdb940ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:14.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.618+0000 7fdb927fc640 1 -- 172.21.15.105:0/513648535 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2110799036 0 0) 0x7fdb940ce530 con 0x7fdb941607b0 2023-11-28T18:29:14.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.618+0000 7fdb927fc640 1 -- 172.21.15.105:0/513648535 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb7c003530 con 0x7fdb941607b0 2023-11-28T18:29:14.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.618+0000 7fdb93fff640 1 -- 172.21.15.105:0/513648535 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdb940d1c00 msgr2=0x7fdb940ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:14.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.618+0000 7fdb93fff640 1 --2- 172.21.15.105:0/513648535 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdb940d1c00 0x7fdb940ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:14.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.618+0000 7fdb93fff640 1 -- 172.21.15.105:0/513648535 >> v1:172.21.15.105:6789/0 conn(0x7fdb941607b0 legacy=0x7fdb940d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:14.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.618+0000 7fdb93fff640 1 -- 172.21.15.105:0/513648535 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb940ce780 con 0x7fdb940d3a50 2023-11-28T18:29:14.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.622+0000 7fdb93fff640 1 --2- 172.21.15.105:0/513648535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb940d3a50 0x7fdb940ca540 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009d90 tx=0x7fdb8c01c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cfe8ab7a292adc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:14.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.622+0000 7fdb927fc640 1 -- 172.21.15.105:0/513648535 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c0a5970 con 0x7fdb940d3a50 2023-11-28T18:29:14.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.622+0000 7fdb927fc640 1 -- 172.21.15.105:0/513648535 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8c0a5ad0 con 0x7fdb940d3a50 2023-11-28T18:29:14.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.622+0000 7fdb927fc640 1 -- 172.21.15.105:0/513648535 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c0a5e60 con 0x7fdb940d3a50 2023-11-28T18:29:14.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.622+0000 7fdb98993640 1 -- 172.21.15.105:0/513648535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb940d3a50 msgr2=0x7fdb940ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:14.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.622+0000 7fdb98993640 1 --2- 172.21.15.105:0/513648535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb940d3a50 0x7fdb940ca540 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009d90 tx=0x7fdb8c01c890 comp rx=0 tx=0).stop 2023-11-28T18:29:14.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.622+0000 7fdb98993640 1 -- 172.21.15.105:0/513648535 shutdown_connections 2023-11-28T18:29:14.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.622+0000 7fdb98993640 1 --2- 172.21.15.105:0/513648535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb940d3a50 0x7fdb940ca540 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:14.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.622+0000 7fdb98993640 1 --2- 172.21.15.105:0/513648535 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdb940d1c00 0x7fdb940ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:14.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.622+0000 7fdb98993640 1 -- 172.21.15.105:0/513648535 >> 172.21.15.105:0/513648535 conn(0x7fdb940d2f30 msgr2=0x7fdb940d3340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:14.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.622+0000 7fdb98993640 1 -- 172.21.15.105:0/513648535 shutdown_connections 2023-11-28T18:29:14.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.622+0000 7fdb98993640 1 -- 172.21.15.105:0/513648535 wait complete. 2023-11-28T18:29:14.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb98993640 1 Processor -- start 2023-11-28T18:29:14.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb98993640 1 -- start start 2023-11-28T18:29:14.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb98993640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdb940d1c00 0x7fdb9415a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:14.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb98993640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb940d3a50 0x7fdb940b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:14.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb98993640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb941607b0 0x7fdb9415d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:14.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb98993640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb940b2f30 con 0x7fdb940d3a50 2023-11-28T18:29:14.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb98993640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb940b3350 con 0x7fdb941607b0 2023-11-28T18:29:14.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb98993640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9415ad90 con 0x7fdb940d1c00 2023-11-28T18:29:14.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb92ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb940d3a50 0x7fdb940b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:14.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb92ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb940d3a50 0x7fdb940b1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54048/0 (socket says 172.21.15.105:54048) 2023-11-28T18:29:14.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb92ffd640 1 -- 172.21.15.105:0/2067994463 learned_addr learned my addr 172.21.15.105:0/2067994463 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:14.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb937fe640 1 --2- 172.21.15.105:0/2067994463 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdb940d1c00 0x7fdb9415a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:14.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb93fff640 1 --2- 172.21.15.105:0/2067994463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb941607b0 0x7fdb9415d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:14.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb92ffd640 1 -- 172.21.15.105:0/2067994463 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdb940d1c00 msgr2=0x7fdb9415a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:14.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb92ffd640 1 --2- 172.21.15.105:0/2067994463 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdb940d1c00 0x7fdb9415a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:14.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb92ffd640 1 -- 172.21.15.105:0/2067994463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb941607b0 msgr2=0x7fdb9415d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:14.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb92ffd640 1 --2- 172.21.15.105:0/2067994463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb941607b0 0x7fdb9415d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:14.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb92ffd640 1 -- 172.21.15.105:0/2067994463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8c009a70 con 0x7fdb940d3a50 2023-11-28T18:29:14.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb92ffd640 1 --2- 172.21.15.105:0/2067994463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb940d3a50 0x7fdb940b1940 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800e870 tx=0x7fdb8800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:14.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb7bfff640 1 -- 172.21.15.105:0/2067994463 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb88009510 con 0x7fdb940d3a50 2023-11-28T18:29:14.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb7bfff640 1 -- 172.21.15.105:0/2067994463 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8800eec0 con 0x7fdb940d3a50 2023-11-28T18:29:14.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb7bfff640 1 -- 172.21.15.105:0/2067994463 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8800cb40 con 0x7fdb940d3a50 2023-11-28T18:29:14.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb98993640 1 -- 172.21.15.105:0/2067994463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb940cf100 con 0x7fdb940d3a50 2023-11-28T18:29:14.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.626+0000 7fdb98993640 1 -- 172.21.15.105:0/2067994463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb940b2590 con 0x7fdb940d3a50 2023-11-28T18:29:14.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.630+0000 7fdb98993640 1 -- 172.21.15.105:0/2067994463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb940b3070 con 0x7fdb940d3a50 2023-11-28T18:29:14.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.630+0000 7fdb7bfff640 1 -- 172.21.15.105:0/2067994463 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fdb8800cca0 con 0x7fdb940d3a50 2023-11-28T18:29:14.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.630+0000 7fdb7bfff640 1 --2- 172.21.15.105:0/2067994463 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdb800668d0 0x7fdb80068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:14.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.630+0000 7fdb7bfff640 1 -- 172.21.15.105:0/2067994463 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdb88084600 con 0x7fdb940d3a50 2023-11-28T18:29:14.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.630+0000 7fdb937fe640 1 --2- 172.21.15.105:0/2067994463 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdb800668d0 0x7fdb80068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:14.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.630+0000 7fdb937fe640 1 --2- 172.21.15.105:0/2067994463 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdb800668d0 0x7fdb80068d90 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdb840011f0 tx=0x7fdb84007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:14.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.634+0000 7fdb7bfff640 1 -- 172.21.15.105:0/2067994463 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdb940b3070 con 0x7fdb940d3a50 2023-11-28T18:29:14.785 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.782+0000 7fdb98993640 1 -- 172.21.15.105:0/2067994463 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdb9408cf60 con 0x7fdb800668d0 2023-11-28T18:29:14.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.786+0000 7fdb7bfff640 1 -- 172.21.15.105:0/2067994463 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289011 (secure 0 0 0) 0x7fdb9408cf60 con 0x7fdb800668d0 2023-11-28T18:29:14.799 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:29:14.801 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:29:14.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.790+0000 7fdb98993640 1 -- 172.21.15.105:0/2067994463 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdb800668d0 msgr2=0x7fdb80068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:14.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.790+0000 7fdb98993640 1 --2- 172.21.15.105:0/2067994463 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdb800668d0 0x7fdb80068d90 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdb840011f0 tx=0x7fdb84007d40 comp rx=0 tx=0).stop 2023-11-28T18:29:14.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.790+0000 7fdb98993640 1 -- 172.21.15.105:0/2067994463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb940d3a50 msgr2=0x7fdb940b1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:14.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.790+0000 7fdb98993640 1 --2- 172.21.15.105:0/2067994463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb940d3a50 0x7fdb940b1940 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800e870 tx=0x7fdb8800ec70 comp rx=0 tx=0).stop 2023-11-28T18:29:14.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.790+0000 7fdb98993640 1 -- 172.21.15.105:0/2067994463 shutdown_connections 2023-11-28T18:29:14.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.790+0000 7fdb98993640 1 --2- 172.21.15.105:0/2067994463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb941607b0 0x7fdb9415d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:14.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.790+0000 7fdb98993640 1 --2- 172.21.15.105:0/2067994463 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdb800668d0 0x7fdb80068d90 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:14.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.790+0000 7fdb98993640 1 --2- 172.21.15.105:0/2067994463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb940d3a50 0x7fdb940b1940 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:14.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.790+0000 7fdb98993640 1 --2- 172.21.15.105:0/2067994463 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdb940d1c00 0x7fdb9415a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:14.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.790+0000 7fdb98993640 1 -- 172.21.15.105:0/2067994463 >> 172.21.15.105:0/2067994463 conn(0x7fdb940d2f30 msgr2=0x7fdb9406c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:14.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.790+0000 7fdb98993640 1 -- 172.21.15.105:0/2067994463 shutdown_connections 2023-11-28T18:29:14.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.790+0000 7fdb98993640 1 -- 172.21.15.105:0/2067994463 wait complete. 2023-11-28T18:29:14.817 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2023-11-28T18:29:13.342541+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":360576,"kb_used_data":9768,"kb_used_omap":12,"kb_used_meta":350643,"kb_avail":754614144,"statfs":{"total":773094113280,"available":772724883456,"internally_reserved":0,"allocated":10002432,"data_stored":5214493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":359058994},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.026286"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.381387+0000","last_change":"2023-11-28T18:29:10.376820+0000","last_active":"2023-11-28T18:29:10.381387+0000","last_peered":"2023-11-28T18:29:10.381387+0000","last_clean":"2023-11-28T18:29:10.381387+0000","last_became_active":"2023-11-28T18:29:10.372940+0000","last_became_peered":"2023-11-28T18:29:10.372940+0000","last_unstale":"2023-11-28T18:29:10.381387+0000","last_undegraded":"2023-11-28T18:29:10.381387+0000","last_fullsized":"2023-11-28T18:29:10.381387+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:15:04.439824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.365043+0000","last_change":"2023-11-28T18:29:10.358088+0000","last_active":"2023-11-28T18:29:10.365043+0000","last_peered":"2023-11-28T18:29:10.365043+0000","last_clean":"2023-11-28T18:29:10.365043+0000","last_became_active":"2023-11-28T18:29:10.357734+0000","last_became_peered":"2023-11-28T18:29:10.357734+0000","last_unstale":"2023-11-28T18:29:10.365043+0000","last_undegraded":"2023-11-28T18:29:10.365043+0000","last_fullsized":"2023-11-28T18:29:10.365043+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:58.005043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.363900+0000","last_change":"2023-11-28T18:29:10.354739+0000","last_active":"2023-11-28T18:29:10.363900+0000","last_peered":"2023-11-28T18:29:10.363900+0000","last_clean":"2023-11-28T18:29:10.363900+0000","last_became_active":"2023-11-28T18:29:10.354433+0000","last_became_peered":"2023-11-28T18:29:10.354433+0000","last_unstale":"2023-11-28T18:29:10.363900+0000","last_undegraded":"2023-11-28T18:29:10.363900+0000","last_fullsized":"2023-11-28T18:29:10.363900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:44.295454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.391929+0000","last_change":"2023-11-28T18:29:10.387781+0000","last_active":"2023-11-28T18:29:10.391929+0000","last_peered":"2023-11-28T18:29:10.391929+0000","last_clean":"2023-11-28T18:29:10.391929+0000","last_became_active":"2023-11-28T18:29:10.387421+0000","last_became_peered":"2023-11-28T18:29:10.387421+0000","last_unstale":"2023-11-28T18:29:10.391929+0000","last_undegraded":"2023-11-28T18:29:10.391929+0000","last_fullsized":"2023-11-28T18:29:10.391929+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:03.075548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.370346+0000","last_change":"2023-11-28T18:29:10.361908+0000","last_active":"2023-11-28T18:29:10.370346+0000","last_peered":"2023-11-28T18:29:10.370346+0000","last_clean":"2023-11-28T18:29:10.370346+0000","last_became_active":"2023-11-28T18:29:10.361582+0000","last_became_peered":"2023-11-28T18:29:10.361582+0000","last_unstale":"2023-11-28T18:29:10.370346+0000","last_undegraded":"2023-11-28T18:29:10.370346+0000","last_fullsized":"2023-11-28T18:29:10.370346+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:39:23.896975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.393604+0000","last_change":"2023-11-28T18:29:10.392610+0000","last_active":"2023-11-28T18:29:10.393604+0000","last_peered":"2023-11-28T18:29:10.393604+0000","last_clean":"2023-11-28T18:29:10.393604+0000","last_became_active":"2023-11-28T18:29:10.390791+0000","last_became_peered":"2023-11-28T18:29:10.390791+0000","last_unstale":"2023-11-28T18:29:10.393604+0000","last_undegraded":"2023-11-28T18:29:10.393604+0000","last_fullsized":"2023-11-28T18:29:10.393604+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:56:23.502494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.376368+0000","last_change":"2023-11-28T18:29:10.372170+0000","last_active":"2023-11-28T18:29:10.376368+0000","last_peered":"2023-11-28T18:29:10.376368+0000","last_clean":"2023-11-28T18:29:10.376368+0000","last_became_active":"2023-11-28T18:29:10.371891+0000","last_became_peered":"2023-11-28T18:29:10.371891+0000","last_unstale":"2023-11-28T18:29:10.376368+0000","last_undegraded":"2023-11-28T18:29:10.376368+0000","last_fullsized":"2023-11-28T18:29:10.376368+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:57:11.754150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.389158+0000","last_change":"2023-11-28T18:29:10.369093+0000","last_active":"2023-11-28T18:29:10.389158+0000","last_peered":"2023-11-28T18:29:10.389158+0000","last_clean":"2023-11-28T18:29:10.389158+0000","last_became_active":"2023-11-28T18:29:10.368378+0000","last_became_peered":"2023-11-28T18:29:10.368378+0000","last_unstale":"2023-11-28T18:29:10.389158+0000","last_undegraded":"2023-11-28T18:29:10.389158+0000","last_fullsized":"2023-11-28T18:29:10.389158+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:46:50.041536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.374418+0000","last_change":"2023-11-28T18:29:10.364179+0000","last_active":"2023-11-28T18:29:10.374418+0000","last_peered":"2023-11-28T18:29:10.374418+0000","last_clean":"2023-11-28T18:29:10.374418+0000","last_became_active":"2023-11-28T18:29:10.363779+0000","last_became_peered":"2023-11-28T18:29:10.363779+0000","last_unstale":"2023-11-28T18:29:10.374418+0000","last_undegraded":"2023-11-28T18:29:10.374418+0000","last_fullsized":"2023-11-28T18:29:10.374418+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:04:40.397373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.395405+0000","last_change":"2023-11-28T18:29:10.392418+0000","last_active":"2023-11-28T18:29:10.395405+0000","last_peered":"2023-11-28T18:29:10.395405+0000","last_clean":"2023-11-28T18:29:10.395405+0000","last_became_active":"2023-11-28T18:29:10.392120+0000","last_became_peered":"2023-11-28T18:29:10.392120+0000","last_unstale":"2023-11-28T18:29:10.395405+0000","last_undegraded":"2023-11-28T18:29:10.395405+0000","last_fullsized":"2023-11-28T18:29:10.395405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:00:27.782429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.381003+0000","last_change":"2023-11-28T18:29:10.379618+0000","last_active":"2023-11-28T18:29:10.381003+0000","last_peered":"2023-11-28T18:29:10.381003+0000","last_clean":"2023-11-28T18:29:10.381003+0000","last_became_active":"2023-11-28T18:29:10.378786+0000","last_became_peered":"2023-11-28T18:29:10.378786+0000","last_unstale":"2023-11-28T18:29:10.381003+0000","last_undegraded":"2023-11-28T18:29:10.381003+0000","last_fullsized":"2023-11-28T18:29:10.381003+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:40:59.600568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.377578+0000","last_change":"2023-11-28T18:29:10.372005+0000","last_active":"2023-11-28T18:29:10.377578+0000","last_peered":"2023-11-28T18:29:10.377578+0000","last_clean":"2023-11-28T18:29:10.377578+0000","last_became_active":"2023-11-28T18:29:10.371566+0000","last_became_peered":"2023-11-28T18:29:10.371566+0000","last_unstale":"2023-11-28T18:29:10.377578+0000","last_undegraded":"2023-11-28T18:29:10.377578+0000","last_fullsized":"2023-11-28T18:29:10.377578+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:17:48.071729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.391328+0000","last_change":"2023-11-28T18:29:10.387076+0000","last_active":"2023-11-28T18:29:10.391328+0000","last_peered":"2023-11-28T18:29:10.391328+0000","last_clean":"2023-11-28T18:29:10.391328+0000","last_became_active":"2023-11-28T18:29:10.386604+0000","last_became_peered":"2023-11-28T18:29:10.386604+0000","last_unstale":"2023-11-28T18:29:10.391328+0000","last_undegraded":"2023-11-28T18:29:10.391328+0000","last_fullsized":"2023-11-28T18:29:10.391328+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:40:59.600568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.381026+0000","last_change":"2023-11-28T18:29:10.380066+0000","last_active":"2023-11-28T18:29:10.381026+0000","last_peered":"2023-11-28T18:29:10.381026+0000","last_clean":"2023-11-28T18:29:10.381026+0000","last_became_active":"2023-11-28T18:29:10.379764+0000","last_became_peered":"2023-11-28T18:29:10.379764+0000","last_unstale":"2023-11-28T18:29:10.381026+0000","last_undegraded":"2023-11-28T18:29:10.381026+0000","last_fullsized":"2023-11-28T18:29:10.381026+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:46:50.041536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.391754+0000","last_change":"2023-11-28T18:29:10.385000+0000","last_active":"2023-11-28T18:29:10.391754+0000","last_peered":"2023-11-28T18:29:10.391754+0000","last_clean":"2023-11-28T18:29:10.391754+0000","last_became_active":"2023-11-28T18:29:10.370668+0000","last_became_peered":"2023-11-28T18:29:10.370668+0000","last_unstale":"2023-11-28T18:29:10.391754+0000","last_undegraded":"2023-11-28T18:29:10.391754+0000","last_fullsized":"2023-11-28T18:29:10.391754+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:09:48.512954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.385179+0000","last_change":"2023-11-28T18:29:10.381747+0000","last_active":"2023-11-28T18:29:10.385179+0000","last_peered":"2023-11-28T18:29:10.385179+0000","last_clean":"2023-11-28T18:29:10.385179+0000","last_became_active":"2023-11-28T18:29:10.381273+0000","last_became_peered":"2023-11-28T18:29:10.381273+0000","last_unstale":"2023-11-28T18:29:10.385179+0000","last_undegraded":"2023-11-28T18:29:10.385179+0000","last_fullsized":"2023-11-28T18:29:10.385179+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:19:58.134475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.379777+0000","last_change":"2023-11-28T18:29:10.370044+0000","last_active":"2023-11-28T18:29:10.379777+0000","last_peered":"2023-11-28T18:29:10.379777+0000","last_clean":"2023-11-28T18:29:10.379777+0000","last_became_active":"2023-11-28T18:29:10.369752+0000","last_became_peered":"2023-11-28T18:29:10.369752+0000","last_unstale":"2023-11-28T18:29:10.379777+0000","last_undegraded":"2023-11-28T18:29:10.379777+0000","last_fullsized":"2023-11-28T18:29:10.379777+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:41:34.642153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.364717+0000","last_change":"2023-11-28T18:29:10.361706+0000","last_active":"2023-11-28T18:29:10.364717+0000","last_peered":"2023-11-28T18:29:10.364717+0000","last_clean":"2023-11-28T18:29:10.364717+0000","last_became_active":"2023-11-28T18:29:10.361355+0000","last_became_peered":"2023-11-28T18:29:10.361355+0000","last_unstale":"2023-11-28T18:29:10.364717+0000","last_undegraded":"2023-11-28T18:29:10.364717+0000","last_fullsized":"2023-11-28T18:29:10.364717+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:44.282151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.377719+0000","last_change":"2023-11-28T18:29:10.374933+0000","last_active":"2023-11-28T18:29:10.377719+0000","last_peered":"2023-11-28T18:29:10.377719+0000","last_clean":"2023-11-28T18:29:10.377719+0000","last_became_active":"2023-11-28T18:29:10.374645+0000","last_became_peered":"2023-11-28T18:29:10.374645+0000","last_unstale":"2023-11-28T18:29:10.377719+0000","last_undegraded":"2023-11-28T18:29:10.377719+0000","last_fullsized":"2023-11-28T18:29:10.377719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:09:48.512954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.389578+0000","last_change":"2023-11-28T18:29:10.373800+0000","last_active":"2023-11-28T18:29:10.389578+0000","last_peered":"2023-11-28T18:29:10.389578+0000","last_clean":"2023-11-28T18:29:10.389578+0000","last_became_active":"2023-11-28T18:29:10.373373+0000","last_became_peered":"2023-11-28T18:29:10.373373+0000","last_unstale":"2023-11-28T18:29:10.389578+0000","last_undegraded":"2023-11-28T18:29:10.389578+0000","last_fullsized":"2023-11-28T18:29:10.389578+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:16:06.036193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.361781+0000","last_change":"2023-11-28T18:29:10.358406+0000","last_active":"2023-11-28T18:29:10.361781+0000","last_peered":"2023-11-28T18:29:10.361781+0000","last_clean":"2023-11-28T18:29:10.361781+0000","last_became_active":"2023-11-28T18:29:10.358065+0000","last_became_peered":"2023-11-28T18:29:10.358065+0000","last_unstale":"2023-11-28T18:29:10.361781+0000","last_undegraded":"2023-11-28T18:29:10.361781+0000","last_fullsized":"2023-11-28T18:29:10.361781+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:29.459848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.381635+0000","last_change":"2023-11-28T18:29:10.380211+0000","last_active":"2023-11-28T18:29:10.381635+0000","last_peered":"2023-11-28T18:29:10.381635+0000","last_clean":"2023-11-28T18:29:10.381635+0000","last_became_active":"2023-11-28T18:29:10.379930+0000","last_became_peered":"2023-11-28T18:29:10.379930+0000","last_unstale":"2023-11-28T18:29:10.381635+0000","last_undegraded":"2023-11-28T18:29:10.381635+0000","last_fullsized":"2023-11-28T18:29:10.381635+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:25:43.024884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.393310+0000","last_change":"2023-11-28T18:29:10.392421+0000","last_active":"2023-11-28T18:29:10.393310+0000","last_peered":"2023-11-28T18:29:10.393310+0000","last_clean":"2023-11-28T18:29:10.393310+0000","last_became_active":"2023-11-28T18:29:10.392116+0000","last_became_peered":"2023-11-28T18:29:10.392116+0000","last_unstale":"2023-11-28T18:29:10.393310+0000","last_undegraded":"2023-11-28T18:29:10.393310+0000","last_fullsized":"2023-11-28T18:29:10.393310+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:40:59.600568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.360967+0000","last_change":"2023-11-28T18:29:10.355273+0000","last_active":"2023-11-28T18:29:10.360967+0000","last_peered":"2023-11-28T18:29:10.360967+0000","last_clean":"2023-11-28T18:29:10.360967+0000","last_became_active":"2023-11-28T18:29:10.354980+0000","last_became_peered":"2023-11-28T18:29:10.354980+0000","last_unstale":"2023-11-28T18:29:10.360967+0000","last_undegraded":"2023-11-28T18:29:10.360967+0000","last_fullsized":"2023-11-28T18:29:10.360967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:58.005043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.368191+0000","last_change":"2023-11-28T18:29:10.362402+0000","last_active":"2023-11-28T18:29:10.368191+0000","last_peered":"2023-11-28T18:29:10.368191+0000","last_clean":"2023-11-28T18:29:10.368191+0000","last_became_active":"2023-11-28T18:29:10.362105+0000","last_became_peered":"2023-11-28T18:29:10.362105+0000","last_unstale":"2023-11-28T18:29:10.368191+0000","last_undegraded":"2023-11-28T18:29:10.368191+0000","last_fullsized":"2023-11-28T18:29:10.368191+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:22:06.432939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.377645+0000","last_change":"2023-11-28T18:29:10.374930+0000","last_active":"2023-11-28T18:29:10.377645+0000","last_peered":"2023-11-28T18:29:10.377645+0000","last_clean":"2023-11-28T18:29:10.377645+0000","last_became_active":"2023-11-28T18:29:10.374610+0000","last_became_peered":"2023-11-28T18:29:10.374610+0000","last_unstale":"2023-11-28T18:29:10.377645+0000","last_undegraded":"2023-11-28T18:29:10.377645+0000","last_fullsized":"2023-11-28T18:29:10.377645+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:54:33.106050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.380062+0000","last_change":"2023-11-28T18:29:10.375074+0000","last_active":"2023-11-28T18:29:10.380062+0000","last_peered":"2023-11-28T18:29:10.380062+0000","last_clean":"2023-11-28T18:29:10.380062+0000","last_became_active":"2023-11-28T18:29:10.374791+0000","last_became_peered":"2023-11-28T18:29:10.374791+0000","last_unstale":"2023-11-28T18:29:10.380062+0000","last_undegraded":"2023-11-28T18:29:10.380062+0000","last_fullsized":"2023-11-28T18:29:10.380062+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:23:33.310140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.391911+0000","last_change":"2023-11-28T18:29:10.387526+0000","last_active":"2023-11-28T18:29:10.391911+0000","last_peered":"2023-11-28T18:29:10.391911+0000","last_clean":"2023-11-28T18:29:10.391911+0000","last_became_active":"2023-11-28T18:29:10.387211+0000","last_became_peered":"2023-11-28T18:29:10.387211+0000","last_unstale":"2023-11-28T18:29:10.391911+0000","last_undegraded":"2023-11-28T18:29:10.391911+0000","last_fullsized":"2023-11-28T18:29:10.391911+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:54:33.106050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.379351+0000","last_change":"2023-11-28T18:29:10.374738+0000","last_active":"2023-11-28T18:29:10.379351+0000","last_peered":"2023-11-28T18:29:10.379351+0000","last_clean":"2023-11-28T18:29:10.379351+0000","last_became_active":"2023-11-28T18:29:10.374451+0000","last_became_peered":"2023-11-28T18:29:10.374451+0000","last_unstale":"2023-11-28T18:29:10.379351+0000","last_undegraded":"2023-11-28T18:29:10.379351+0000","last_fullsized":"2023-11-28T18:29:10.379351+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:52.925955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.370786+0000","last_change":"2023-11-28T18:29:10.362405+0000","last_active":"2023-11-28T18:29:10.370786+0000","last_peered":"2023-11-28T18:29:10.370786+0000","last_clean":"2023-11-28T18:29:10.370786+0000","last_became_active":"2023-11-28T18:29:10.362017+0000","last_became_peered":"2023-11-28T18:29:10.362017+0000","last_unstale":"2023-11-28T18:29:10.370786+0000","last_undegraded":"2023-11-28T18:29:10.370786+0000","last_fullsized":"2023-11-28T18:29:10.370786+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:55:17.520003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.372715+0000","last_change":"2023-11-28T18:29:10.366398+0000","last_active":"2023-11-28T18:29:10.372715+0000","last_peered":"2023-11-28T18:29:10.372715+0000","last_clean":"2023-11-28T18:29:10.372715+0000","last_became_active":"2023-11-28T18:29:10.366127+0000","last_became_peered":"2023-11-28T18:29:10.366127+0000","last_unstale":"2023-11-28T18:29:10.372715+0000","last_undegraded":"2023-11-28T18:29:10.372715+0000","last_fullsized":"2023-11-28T18:29:10.372715+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:26.529566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.385160+0000","last_change":"2023-11-28T18:29:10.381558+0000","last_active":"2023-11-28T18:29:10.385160+0000","last_peered":"2023-11-28T18:29:10.385160+0000","last_clean":"2023-11-28T18:29:10.385160+0000","last_became_active":"2023-11-28T18:29:10.380938+0000","last_became_peered":"2023-11-28T18:29:10.380938+0000","last_unstale":"2023-11-28T18:29:10.385160+0000","last_undegraded":"2023-11-28T18:29:10.385160+0000","last_fullsized":"2023-11-28T18:29:10.385160+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:23:43.982249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.382243+0000","last_change":"2023-11-28T18:29:10.379240+0000","last_active":"2023-11-28T18:29:10.382243+0000","last_peered":"2023-11-28T18:29:10.382243+0000","last_clean":"2023-11-28T18:29:10.382243+0000","last_became_active":"2023-11-28T18:29:10.378975+0000","last_became_peered":"2023-11-28T18:29:10.378975+0000","last_unstale":"2023-11-28T18:29:10.382243+0000","last_undegraded":"2023-11-28T18:29:10.382243+0000","last_fullsized":"2023-11-28T18:29:10.382243+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:42:32.991902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"89'4","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.381513+0000","last_change":"2023-11-28T18:29:09.348890+0000","last_active":"2023-11-28T18:29:10.381513+0000","last_peered":"2023-11-28T18:29:10.381513+0000","last_clean":"2023-11-28T18:29:10.381513+0000","last_became_active":"2023-11-28T18:29:09.348629+0000","last_became_peered":"2023-11-28T18:29:09.348629+0000","last_unstale":"2023-11-28T18:29:10.381513+0000","last_undegraded":"2023-11-28T18:29:10.381513+0000","last_fullsized":"2023-11-28T18:29:10.381513+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:02.067920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.380792+0000","last_change":"2023-11-28T18:29:10.375115+0000","last_active":"2023-11-28T18:29:10.380792+0000","last_peered":"2023-11-28T18:29:10.380792+0000","last_clean":"2023-11-28T18:29:10.380792+0000","last_became_active":"2023-11-28T18:29:10.374848+0000","last_became_peered":"2023-11-28T18:29:10.374848+0000","last_unstale":"2023-11-28T18:29:10.380792+0000","last_undegraded":"2023-11-28T18:29:10.380792+0000","last_fullsized":"2023-11-28T18:29:10.380792+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:27:01.017033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"89'1","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.374332+0000","last_change":"2023-11-28T18:29:09.335618+0000","last_active":"2023-11-28T18:29:10.374332+0000","last_peered":"2023-11-28T18:29:10.374332+0000","last_clean":"2023-11-28T18:29:10.374332+0000","last_became_active":"2023-11-28T18:29:09.335239+0000","last_became_peered":"2023-11-28T18:29:09.335239+0000","last_unstale":"2023-11-28T18:29:10.374332+0000","last_undegraded":"2023-11-28T18:29:10.374332+0000","last_fullsized":"2023-11-28T18:29:10.374332+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:58.161257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.390864+0000","last_change":"2023-11-28T18:29:10.390248+0000","last_active":"2023-11-28T18:29:10.390864+0000","last_peered":"2023-11-28T18:29:10.390864+0000","last_clean":"2023-11-28T18:29:10.390864+0000","last_became_active":"2023-11-28T18:29:10.389979+0000","last_became_peered":"2023-11-28T18:29:10.389979+0000","last_unstale":"2023-11-28T18:29:10.390864+0000","last_undegraded":"2023-11-28T18:29:10.390864+0000","last_fullsized":"2023-11-28T18:29:10.390864+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:32:45.165863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.379250+0000","last_change":"2023-11-28T18:29:09.348524+0000","last_active":"2023-11-28T18:29:10.379250+0000","last_peered":"2023-11-28T18:29:10.379250+0000","last_clean":"2023-11-28T18:29:10.379250+0000","last_became_active":"2023-11-28T18:29:09.348110+0000","last_became_peered":"2023-11-28T18:29:09.348110+0000","last_unstale":"2023-11-28T18:29:10.379250+0000","last_undegraded":"2023-11-28T18:29:10.379250+0000","last_fullsized":"2023-11-28T18:29:10.379250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:28:20.632607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.379756+0000","last_change":"2023-11-28T18:29:10.370018+0000","last_active":"2023-11-28T18:29:10.379756+0000","last_peered":"2023-11-28T18:29:10.379756+0000","last_clean":"2023-11-28T18:29:10.379756+0000","last_became_active":"2023-11-28T18:29:10.369718+0000","last_became_peered":"2023-11-28T18:29:10.369718+0000","last_unstale":"2023-11-28T18:29:10.379756+0000","last_undegraded":"2023-11-28T18:29:10.379756+0000","last_fullsized":"2023-11-28T18:29:10.379756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:32:05.253712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.392150+0000","last_change":"2023-11-28T18:29:09.340583+0000","last_active":"2023-11-28T18:29:10.392150+0000","last_peered":"2023-11-28T18:29:10.392150+0000","last_clean":"2023-11-28T18:29:10.392150+0000","last_became_active":"2023-11-28T18:29:09.340310+0000","last_became_peered":"2023-11-28T18:29:09.340310+0000","last_unstale":"2023-11-28T18:29:10.392150+0000","last_undegraded":"2023-11-28T18:29:10.392150+0000","last_fullsized":"2023-11-28T18:29:10.392150+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:16:05.028565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.389182+0000","last_change":"2023-11-28T18:29:10.369304+0000","last_active":"2023-11-28T18:29:10.389182+0000","last_peered":"2023-11-28T18:29:10.389182+0000","last_clean":"2023-11-28T18:29:10.389182+0000","last_became_active":"2023-11-28T18:29:10.368664+0000","last_became_peered":"2023-11-28T18:29:10.368664+0000","last_unstale":"2023-11-28T18:29:10.389182+0000","last_undegraded":"2023-11-28T18:29:10.389182+0000","last_fullsized":"2023-11-28T18:29:10.389182+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:03:30.953517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"89'1","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.379067+0000","last_change":"2023-11-28T18:29:09.336367+0000","last_active":"2023-11-28T18:29:10.379067+0000","last_peered":"2023-11-28T18:29:10.379067+0000","last_clean":"2023-11-28T18:29:10.379067+0000","last_became_active":"2023-11-28T18:29:09.334465+0000","last_became_peered":"2023-11-28T18:29:09.334465+0000","last_unstale":"2023-11-28T18:29:10.379067+0000","last_undegraded":"2023-11-28T18:29:10.379067+0000","last_fullsized":"2023-11-28T18:29:10.379067+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:16:05.028565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.378648+0000","last_change":"2023-11-28T18:29:10.376841+0000","last_active":"2023-11-28T18:29:10.378648+0000","last_peered":"2023-11-28T18:29:10.378648+0000","last_clean":"2023-11-28T18:29:10.378648+0000","last_became_active":"2023-11-28T18:29:10.376532+0000","last_became_peered":"2023-11-28T18:29:10.376532+0000","last_unstale":"2023-11-28T18:29:10.378648+0000","last_undegraded":"2023-11-28T18:29:10.378648+0000","last_fullsized":"2023-11-28T18:29:10.378648+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:54:33.106050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"89'1","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.389370+0000","last_change":"2023-11-28T18:29:09.351252+0000","last_active":"2023-11-28T18:29:10.389370+0000","last_peered":"2023-11-28T18:29:10.389370+0000","last_clean":"2023-11-28T18:29:10.389370+0000","last_became_active":"2023-11-28T18:29:09.350966+0000","last_became_peered":"2023-11-28T18:29:09.350966+0000","last_unstale":"2023-11-28T18:29:10.389370+0000","last_undegraded":"2023-11-28T18:29:10.389370+0000","last_fullsized":"2023-11-28T18:29:10.389370+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:58.161257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.392755+0000","last_change":"2023-11-28T18:29:10.390353+0000","last_active":"2023-11-28T18:29:10.392755+0000","last_peered":"2023-11-28T18:29:10.392755+0000","last_clean":"2023-11-28T18:29:10.392755+0000","last_became_active":"2023-11-28T18:29:10.390084+0000","last_became_peered":"2023-11-28T18:29:10.390084+0000","last_unstale":"2023-11-28T18:29:10.392755+0000","last_undegraded":"2023-11-28T18:29:10.392755+0000","last_fullsized":"2023-11-28T18:29:10.392755+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:44:39.461779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.347681+0000","last_change":"2023-11-28T18:29:09.342809+0000","last_active":"2023-11-28T18:29:10.347681+0000","last_peered":"2023-11-28T18:29:10.347681+0000","last_clean":"2023-11-28T18:29:10.347681+0000","last_became_active":"2023-11-28T18:29:09.342104+0000","last_became_peered":"2023-11-28T18:29:09.342104+0000","last_unstale":"2023-11-28T18:29:10.347681+0000","last_undegraded":"2023-11-28T18:29:10.347681+0000","last_fullsized":"2023-11-28T18:29:10.347681+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:25.521938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.391350+0000","last_change":"2023-11-28T18:29:10.387798+0000","last_active":"2023-11-28T18:29:10.391350+0000","last_peered":"2023-11-28T18:29:10.391350+0000","last_clean":"2023-11-28T18:29:10.391350+0000","last_became_active":"2023-11-28T18:29:10.387433+0000","last_became_peered":"2023-11-28T18:29:10.387433+0000","last_unstale":"2023-11-28T18:29:10.391350+0000","last_undegraded":"2023-11-28T18:29:10.391350+0000","last_fullsized":"2023-11-28T18:29:10.391350+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:04:40.397373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.352947+0000","last_change":"2023-11-28T18:29:09.349940+0000","last_active":"2023-11-28T18:29:10.352947+0000","last_peered":"2023-11-28T18:29:10.352947+0000","last_clean":"2023-11-28T18:29:10.352947+0000","last_became_active":"2023-11-28T18:29:09.349402+0000","last_became_peered":"2023-11-28T18:29:09.349402+0000","last_unstale":"2023-11-28T18:29:10.352947+0000","last_undegraded":"2023-11-28T18:29:10.352947+0000","last_fullsized":"2023-11-28T18:29:10.352947+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:16:31.549066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.360634+0000","last_change":"2023-11-28T18:29:10.355330+0000","last_active":"2023-11-28T18:29:10.360634+0000","last_peered":"2023-11-28T18:29:10.360634+0000","last_clean":"2023-11-28T18:29:10.360634+0000","last_became_active":"2023-11-28T18:29:10.354907+0000","last_became_peered":"2023-11-28T18:29:10.354907+0000","last_unstale":"2023-11-28T18:29:10.360634+0000","last_undegraded":"2023-11-28T18:29:10.360634+0000","last_fullsized":"2023-11-28T18:29:10.360634+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:03:30.953517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.354254+0000","last_change":"2023-11-28T18:29:09.349813+0000","last_active":"2023-11-28T18:29:10.354254+0000","last_peered":"2023-11-28T18:29:10.354254+0000","last_clean":"2023-11-28T18:29:10.354254+0000","last_became_active":"2023-11-28T18:29:09.349478+0000","last_became_peered":"2023-11-28T18:29:09.349478+0000","last_unstale":"2023-11-28T18:29:10.354254+0000","last_undegraded":"2023-11-28T18:29:10.354254+0000","last_fullsized":"2023-11-28T18:29:10.354254+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:15:03.432197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.373080+0000","last_change":"2023-11-28T18:29:10.366671+0000","last_active":"2023-11-28T18:29:10.373080+0000","last_peered":"2023-11-28T18:29:10.373080+0000","last_clean":"2023-11-28T18:29:10.373080+0000","last_became_active":"2023-11-28T18:29:10.365593+0000","last_became_peered":"2023-11-28T18:29:10.365593+0000","last_unstale":"2023-11-28T18:29:10.373080+0000","last_undegraded":"2023-11-28T18:29:10.373080+0000","last_fullsized":"2023-11-28T18:29:10.373080+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:08:16.247758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.348449+0000","last_change":"2023-11-28T18:29:09.344090+0000","last_active":"2023-11-28T18:29:10.348449+0000","last_peered":"2023-11-28T18:29:10.348449+0000","last_clean":"2023-11-28T18:29:10.348449+0000","last_became_active":"2023-11-28T18:29:09.342928+0000","last_became_peered":"2023-11-28T18:29:09.342928+0000","last_unstale":"2023-11-28T18:29:10.348449+0000","last_undegraded":"2023-11-28T18:29:10.348449+0000","last_fullsized":"2023-11-28T18:29:10.348449+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:57.839659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.386388+0000","last_change":"2023-11-28T18:29:10.378740+0000","last_active":"2023-11-28T18:29:10.386388+0000","last_peered":"2023-11-28T18:29:10.386388+0000","last_clean":"2023-11-28T18:29:10.386388+0000","last_became_active":"2023-11-28T18:29:10.378467+0000","last_became_peered":"2023-11-28T18:29:10.378467+0000","last_unstale":"2023-11-28T18:29:10.386388+0000","last_undegraded":"2023-11-28T18:29:10.386388+0000","last_fullsized":"2023-11-28T18:29:10.386388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:43:41.111063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"89'3","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.381846+0000","last_change":"2023-11-28T18:29:09.341105+0000","last_active":"2023-11-28T18:29:10.381846+0000","last_peered":"2023-11-28T18:29:10.381846+0000","last_clean":"2023-11-28T18:29:10.381846+0000","last_became_active":"2023-11-28T18:29:09.340804+0000","last_became_peered":"2023-11-28T18:29:09.340804+0000","last_unstale":"2023-11-28T18:29:10.381846+0000","last_undegraded":"2023-11-28T18:29:10.381846+0000","last_fullsized":"2023-11-28T18:29:10.381846+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:35.989485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.389009+0000","last_change":"2023-11-28T18:29:10.367211+0000","last_active":"2023-11-28T18:29:10.389009+0000","last_peered":"2023-11-28T18:29:10.389009+0000","last_clean":"2023-11-28T18:29:10.389009+0000","last_became_active":"2023-11-28T18:29:10.366930+0000","last_became_peered":"2023-11-28T18:29:10.366930+0000","last_unstale":"2023-11-28T18:29:10.389009+0000","last_undegraded":"2023-11-28T18:29:10.389009+0000","last_fullsized":"2023-11-28T18:29:10.389009+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:19.201563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"89'6","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.385095+0000","last_change":"2023-11-28T18:29:09.346412+0000","last_active":"2023-11-28T18:29:10.385095+0000","last_peered":"2023-11-28T18:29:10.385095+0000","last_clean":"2023-11-28T18:29:10.385095+0000","last_became_active":"2023-11-28T18:29:09.345775+0000","last_became_peered":"2023-11-28T18:29:09.345775+0000","last_unstale":"2023-11-28T18:29:10.385095+0000","last_undegraded":"2023-11-28T18:29:10.385095+0000","last_fullsized":"2023-11-28T18:29:10.385095+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:35:27.985961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.389135+0000","last_change":"2023-11-28T18:29:10.368923+0000","last_active":"2023-11-28T18:29:10.389135+0000","last_peered":"2023-11-28T18:29:10.389135+0000","last_clean":"2023-11-28T18:29:10.389135+0000","last_became_active":"2023-11-28T18:29:10.368091+0000","last_became_peered":"2023-11-28T18:29:10.368091+0000","last_unstale":"2023-11-28T18:29:10.389135+0000","last_undegraded":"2023-11-28T18:29:10.389135+0000","last_fullsized":"2023-11-28T18:29:10.389135+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:25:43.024884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.382143+0000","last_change":"2023-11-28T18:29:09.349418+0000","last_active":"2023-11-28T18:29:10.382143+0000","last_peered":"2023-11-28T18:29:10.382143+0000","last_clean":"2023-11-28T18:29:10.382143+0000","last_became_active":"2023-11-28T18:29:09.349141+0000","last_became_peered":"2023-11-28T18:29:09.349141+0000","last_unstale":"2023-11-28T18:29:10.382143+0000","last_undegraded":"2023-11-28T18:29:10.382143+0000","last_fullsized":"2023-11-28T18:29:10.382143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:27:14.392678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.370525+0000","last_change":"2023-11-28T18:29:10.361916+0000","last_active":"2023-11-28T18:29:10.370525+0000","last_peered":"2023-11-28T18:29:10.370525+0000","last_clean":"2023-11-28T18:29:10.370525+0000","last_became_active":"2023-11-28T18:29:10.361611+0000","last_became_peered":"2023-11-28T18:29:10.361611+0000","last_unstale":"2023-11-28T18:29:10.370525+0000","last_undegraded":"2023-11-28T18:29:10.370525+0000","last_fullsized":"2023-11-28T18:29:10.370525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:32:12.486586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.360852+0000","last_change":"2023-11-28T18:29:09.335403+0000","last_active":"2023-11-28T18:29:10.360852+0000","last_peered":"2023-11-28T18:29:10.360852+0000","last_clean":"2023-11-28T18:29:10.360852+0000","last_became_active":"2023-11-28T18:29:09.335093+0000","last_became_peered":"2023-11-28T18:29:09.335093+0000","last_unstale":"2023-11-28T18:29:10.360852+0000","last_undegraded":"2023-11-28T18:29:10.360852+0000","last_fullsized":"2023-11-28T18:29:10.360852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:47:46.164388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.393342+0000","last_change":"2023-11-28T18:29:10.391989+0000","last_active":"2023-11-28T18:29:10.393342+0000","last_peered":"2023-11-28T18:29:10.393342+0000","last_clean":"2023-11-28T18:29:10.393342+0000","last_became_active":"2023-11-28T18:29:10.391678+0000","last_became_peered":"2023-11-28T18:29:10.391678+0000","last_unstale":"2023-11-28T18:29:10.393342+0000","last_undegraded":"2023-11-28T18:29:10.393342+0000","last_fullsized":"2023-11-28T18:29:10.393342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:10:43.169437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.358611+0000","last_change":"2023-11-28T18:29:09.345218+0000","last_active":"2023-11-28T18:29:10.358611+0000","last_peered":"2023-11-28T18:29:10.358611+0000","last_clean":"2023-11-28T18:29:10.358611+0000","last_became_active":"2023-11-28T18:29:09.344920+0000","last_became_peered":"2023-11-28T18:29:09.344920+0000","last_unstale":"2023-11-28T18:29:10.358611+0000","last_undegraded":"2023-11-28T18:29:10.358611+0000","last_fullsized":"2023-11-28T18:29:10.358611+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:16.178469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.380424+0000","last_change":"2023-11-28T18:29:10.374300+0000","last_active":"2023-11-28T18:29:10.380424+0000","last_peered":"2023-11-28T18:29:10.380424+0000","last_clean":"2023-11-28T18:29:10.380424+0000","last_became_active":"2023-11-28T18:29:10.374070+0000","last_became_peered":"2023-11-28T18:29:10.374070+0000","last_unstale":"2023-11-28T18:29:10.380424+0000","last_undegraded":"2023-11-28T18:29:10.380424+0000","last_fullsized":"2023-11-28T18:29:10.380424+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:57.905089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"89'8","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.391774+0000","last_change":"2023-11-28T18:29:09.350701+0000","last_active":"2023-11-28T18:29:10.391774+0000","last_peered":"2023-11-28T18:29:10.391774+0000","last_clean":"2023-11-28T18:29:10.391774+0000","last_became_active":"2023-11-28T18:29:09.350408+0000","last_became_peered":"2023-11-28T18:29:09.350408+0000","last_unstale":"2023-11-28T18:29:10.391774+0000","last_undegraded":"2023-11-28T18:29:10.391774+0000","last_fullsized":"2023-11-28T18:29:10.391774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:35.989485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.382226+0000","last_change":"2023-11-28T18:29:10.379194+0000","last_active":"2023-11-28T18:29:10.382226+0000","last_peered":"2023-11-28T18:29:10.382226+0000","last_clean":"2023-11-28T18:29:10.382226+0000","last_became_active":"2023-11-28T18:29:10.378563+0000","last_became_peered":"2023-11-28T18:29:10.378563+0000","last_unstale":"2023-11-28T18:29:10.382226+0000","last_undegraded":"2023-11-28T18:29:10.382226+0000","last_fullsized":"2023-11-28T18:29:10.382226+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:29:12.192039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"89'1","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.372109+0000","last_change":"2023-11-28T18:29:09.342440+0000","last_active":"2023-11-28T18:29:10.372109+0000","last_peered":"2023-11-28T18:29:10.372109+0000","last_clean":"2023-11-28T18:29:10.372109+0000","last_became_active":"2023-11-28T18:29:09.342157+0000","last_became_peered":"2023-11-28T18:29:09.342157+0000","last_unstale":"2023-11-28T18:29:10.372109+0000","last_undegraded":"2023-11-28T18:29:10.372109+0000","last_fullsized":"2023-11-28T18:29:10.372109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:16.178469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.373530+0000","last_change":"2023-11-28T18:29:10.370630+0000","last_active":"2023-11-28T18:29:10.373530+0000","last_peered":"2023-11-28T18:29:10.373530+0000","last_clean":"2023-11-28T18:29:10.373530+0000","last_became_active":"2023-11-28T18:29:10.370343+0000","last_became_peered":"2023-11-28T18:29:10.370343+0000","last_unstale":"2023-11-28T18:29:10.373530+0000","last_undegraded":"2023-11-28T18:29:10.373530+0000","last_fullsized":"2023-11-28T18:29:10.373530+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:19.201563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.373454+0000","last_change":"2023-11-28T18:29:09.346424+0000","last_active":"2023-11-28T18:29:10.373454+0000","last_peered":"2023-11-28T18:29:10.373454+0000","last_clean":"2023-11-28T18:29:10.373454+0000","last_became_active":"2023-11-28T18:29:09.346146+0000","last_became_peered":"2023-11-28T18:29:09.346146+0000","last_unstale":"2023-11-28T18:29:10.373454+0000","last_undegraded":"2023-11-28T18:29:10.373454+0000","last_fullsized":"2023-11-28T18:29:10.373454+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:32:55.197612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.378187+0000","last_change":"2023-11-28T18:29:10.376159+0000","last_active":"2023-11-28T18:29:10.378187+0000","last_peered":"2023-11-28T18:29:10.378187+0000","last_clean":"2023-11-28T18:29:10.378187+0000","last_became_active":"2023-11-28T18:29:10.375908+0000","last_became_peered":"2023-11-28T18:29:10.375908+0000","last_unstale":"2023-11-28T18:29:10.378187+0000","last_undegraded":"2023-11-28T18:29:10.378187+0000","last_fullsized":"2023-11-28T18:29:10.378187+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:19.201563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"89'1","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.392200+0000","last_change":"2023-11-28T18:29:09.340083+0000","last_active":"2023-11-28T18:29:10.392200+0000","last_peered":"2023-11-28T18:29:10.392200+0000","last_clean":"2023-11-28T18:29:10.392200+0000","last_became_active":"2023-11-28T18:29:09.338664+0000","last_became_peered":"2023-11-28T18:29:09.338664+0000","last_unstale":"2023-11-28T18:29:10.392200+0000","last_undegraded":"2023-11-28T18:29:10.392200+0000","last_fullsized":"2023-11-28T18:29:10.392200+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:02.067920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.391781+0000","last_change":"2023-11-28T18:29:10.390287+0000","last_active":"2023-11-28T18:29:10.391781+0000","last_peered":"2023-11-28T18:29:10.391781+0000","last_clean":"2023-11-28T18:29:10.391781+0000","last_became_active":"2023-11-28T18:29:10.389978+0000","last_became_peered":"2023-11-28T18:29:10.389978+0000","last_unstale":"2023-11-28T18:29:10.391781+0000","last_undegraded":"2023-11-28T18:29:10.391781+0000","last_fullsized":"2023-11-28T18:29:10.391781+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:41:02.593427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.373105+0000","last_change":"2023-11-28T18:29:09.343699+0000","last_active":"2023-11-28T18:29:10.373105+0000","last_peered":"2023-11-28T18:29:10.373105+0000","last_clean":"2023-11-28T18:29:10.373105+0000","last_became_active":"2023-11-28T18:29:09.342820+0000","last_became_peered":"2023-11-28T18:29:09.342820+0000","last_unstale":"2023-11-28T18:29:10.373105+0000","last_undegraded":"2023-11-28T18:29:10.373105+0000","last_fullsized":"2023-11-28T18:29:10.373105+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:43.287826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.372696+0000","last_change":"2023-11-28T18:29:10.365090+0000","last_active":"2023-11-28T18:29:10.372696+0000","last_peered":"2023-11-28T18:29:10.372696+0000","last_clean":"2023-11-28T18:29:10.372696+0000","last_became_active":"2023-11-28T18:29:10.364616+0000","last_became_peered":"2023-11-28T18:29:10.364616+0000","last_unstale":"2023-11-28T18:29:10.372696+0000","last_undegraded":"2023-11-28T18:29:10.372696+0000","last_fullsized":"2023-11-28T18:29:10.372696+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:50.642782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.365963+0000","last_change":"2023-11-28T18:29:09.346916+0000","last_active":"2023-11-28T18:29:10.365963+0000","last_peered":"2023-11-28T18:29:10.365963+0000","last_clean":"2023-11-28T18:29:10.365963+0000","last_became_active":"2023-11-28T18:29:09.345562+0000","last_became_peered":"2023-11-28T18:29:09.345562+0000","last_unstale":"2023-11-28T18:29:10.365963+0000","last_undegraded":"2023-11-28T18:29:10.365963+0000","last_fullsized":"2023-11-28T18:29:10.365963+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:19:28.889518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.380061+0000","last_change":"2023-11-28T18:29:10.379022+0000","last_active":"2023-11-28T18:29:10.380061+0000","last_peered":"2023-11-28T18:29:10.380061+0000","last_clean":"2023-11-28T18:29:10.380061+0000","last_became_active":"2023-11-28T18:29:10.378672+0000","last_became_peered":"2023-11-28T18:29:10.378672+0000","last_unstale":"2023-11-28T18:29:10.380061+0000","last_undegraded":"2023-11-28T18:29:10.380061+0000","last_fullsized":"2023-11-28T18:29:10.380061+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:30:42.927271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.354042+0000","last_change":"2023-11-28T18:29:09.348033+0000","last_active":"2023-11-28T18:29:10.354042+0000","last_peered":"2023-11-28T18:29:10.354042+0000","last_clean":"2023-11-28T18:29:10.354042+0000","last_became_active":"2023-11-28T18:29:09.347758+0000","last_became_peered":"2023-11-28T18:29:09.347758+0000","last_unstale":"2023-11-28T18:29:10.354042+0000","last_undegraded":"2023-11-28T18:29:10.354042+0000","last_fullsized":"2023-11-28T18:29:10.354042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:57.839659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.381279+0000","last_change":"2023-11-28T18:29:10.376545+0000","last_active":"2023-11-28T18:29:10.381279+0000","last_peered":"2023-11-28T18:29:10.381279+0000","last_clean":"2023-11-28T18:29:10.381279+0000","last_became_active":"2023-11-28T18:29:10.374076+0000","last_became_peered":"2023-11-28T18:29:10.374076+0000","last_unstale":"2023-11-28T18:29:10.381279+0000","last_undegraded":"2023-11-28T18:29:10.381279+0000","last_fullsized":"2023-11-28T18:29:10.381279+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:32:05.660491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.364698+0000","last_change":"2023-11-28T18:29:09.344177+0000","last_active":"2023-11-28T18:29:10.364698+0000","last_peered":"2023-11-28T18:29:10.364698+0000","last_clean":"2023-11-28T18:29:10.364698+0000","last_became_active":"2023-11-28T18:29:09.341608+0000","last_became_peered":"2023-11-28T18:29:09.341608+0000","last_unstale":"2023-11-28T18:29:10.364698+0000","last_undegraded":"2023-11-28T18:29:10.364698+0000","last_fullsized":"2023-11-28T18:29:10.364698+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:59:33.812731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.367687+0000","last_change":"2023-11-28T18:29:10.360366+0000","last_active":"2023-11-28T18:29:10.367687+0000","last_peered":"2023-11-28T18:29:10.367687+0000","last_clean":"2023-11-28T18:29:10.367687+0000","last_became_active":"2023-11-28T18:29:10.360087+0000","last_became_peered":"2023-11-28T18:29:10.360087+0000","last_unstale":"2023-11-28T18:29:10.367687+0000","last_undegraded":"2023-11-28T18:29:10.367687+0000","last_fullsized":"2023-11-28T18:29:10.367687+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:09:48.512954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.373872+0000","last_change":"2023-11-28T18:29:09.347067+0000","last_active":"2023-11-28T18:29:10.373872+0000","last_peered":"2023-11-28T18:29:10.373872+0000","last_clean":"2023-11-28T18:29:10.373872+0000","last_became_active":"2023-11-28T18:29:09.346647+0000","last_became_peered":"2023-11-28T18:29:09.346647+0000","last_unstale":"2023-11-28T18:29:10.373872+0000","last_undegraded":"2023-11-28T18:29:10.373872+0000","last_fullsized":"2023-11-28T18:29:10.373872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:11:06.712338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.380257+0000","last_change":"2023-11-28T18:29:10.372893+0000","last_active":"2023-11-28T18:29:10.380257+0000","last_peered":"2023-11-28T18:29:10.380257+0000","last_clean":"2023-11-28T18:29:10.380257+0000","last_became_active":"2023-11-28T18:29:10.372630+0000","last_became_peered":"2023-11-28T18:29:10.372630+0000","last_unstale":"2023-11-28T18:29:10.380257+0000","last_undegraded":"2023-11-28T18:29:10.380257+0000","last_fullsized":"2023-11-28T18:29:10.380257+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:14:36.140322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.354258+0000","last_change":"2023-11-28T18:29:09.342072+0000","last_active":"2023-11-28T18:29:10.354258+0000","last_peered":"2023-11-28T18:29:10.354258+0000","last_clean":"2023-11-28T18:29:10.354258+0000","last_became_active":"2023-11-28T18:29:09.341510+0000","last_became_peered":"2023-11-28T18:29:09.341510+0000","last_unstale":"2023-11-28T18:29:10.354258+0000","last_undegraded":"2023-11-28T18:29:10.354258+0000","last_fullsized":"2023-11-28T18:29:10.354258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:59:35.230058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.382121+0000","last_change":"2023-11-28T18:29:10.377372+0000","last_active":"2023-11-28T18:29:10.382121+0000","last_peered":"2023-11-28T18:29:10.382121+0000","last_clean":"2023-11-28T18:29:10.382121+0000","last_became_active":"2023-11-28T18:29:10.377105+0000","last_became_peered":"2023-11-28T18:29:10.377105+0000","last_unstale":"2023-11-28T18:29:10.382121+0000","last_undegraded":"2023-11-28T18:29:10.382121+0000","last_fullsized":"2023-11-28T18:29:10.382121+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:28:01.196337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.354115+0000","last_change":"2023-11-28T18:29:09.340320+0000","last_active":"2023-11-28T18:29:10.354115+0000","last_peered":"2023-11-28T18:29:10.354115+0000","last_clean":"2023-11-28T18:29:10.354115+0000","last_became_active":"2023-11-28T18:29:09.339977+0000","last_became_peered":"2023-11-28T18:29:09.339977+0000","last_unstale":"2023-11-28T18:29:10.354115+0000","last_undegraded":"2023-11-28T18:29:10.354115+0000","last_fullsized":"2023-11-28T18:29:10.354115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:16.178469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.365937+0000","last_change":"2023-11-28T18:29:10.358121+0000","last_active":"2023-11-28T18:29:10.365937+0000","last_peered":"2023-11-28T18:29:10.365937+0000","last_clean":"2023-11-28T18:29:10.365937+0000","last_became_active":"2023-11-28T18:29:10.357799+0000","last_became_peered":"2023-11-28T18:29:10.357799+0000","last_unstale":"2023-11-28T18:29:10.365937+0000","last_undegraded":"2023-11-28T18:29:10.365937+0000","last_fullsized":"2023-11-28T18:29:10.365937+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:11:50.704320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.357175+0000","last_change":"2023-11-28T18:29:09.338189+0000","last_active":"2023-11-28T18:29:10.357175+0000","last_peered":"2023-11-28T18:29:10.357175+0000","last_clean":"2023-11-28T18:29:10.357175+0000","last_became_active":"2023-11-28T18:29:09.337447+0000","last_became_peered":"2023-11-28T18:29:09.337447+0000","last_unstale":"2023-11-28T18:29:10.357175+0000","last_undegraded":"2023-11-28T18:29:10.357175+0000","last_fullsized":"2023-11-28T18:29:10.357175+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:35.989485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.380210+0000","last_change":"2023-11-28T18:29:10.375886+0000","last_active":"2023-11-28T18:29:10.380210+0000","last_peered":"2023-11-28T18:29:10.380210+0000","last_clean":"2023-11-28T18:29:10.380210+0000","last_became_active":"2023-11-28T18:29:10.375232+0000","last_became_peered":"2023-11-28T18:29:10.375232+0000","last_unstale":"2023-11-28T18:29:10.380210+0000","last_undegraded":"2023-11-28T18:29:10.380210+0000","last_fullsized":"2023-11-28T18:29:10.380210+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:43:40.528137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.357726+0000","last_change":"2023-11-28T18:29:09.346493+0000","last_active":"2023-11-28T18:29:10.357726+0000","last_peered":"2023-11-28T18:29:10.357726+0000","last_clean":"2023-11-28T18:29:10.357726+0000","last_became_active":"2023-11-28T18:29:09.346237+0000","last_became_peered":"2023-11-28T18:29:09.346237+0000","last_unstale":"2023-11-28T18:29:10.357726+0000","last_undegraded":"2023-11-28T18:29:10.357726+0000","last_fullsized":"2023-11-28T18:29:10.357726+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:48.883072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.375368+0000","last_change":"2023-11-28T18:29:10.371516+0000","last_active":"2023-11-28T18:29:10.375368+0000","last_peered":"2023-11-28T18:29:10.375368+0000","last_clean":"2023-11-28T18:29:10.375368+0000","last_became_active":"2023-11-28T18:29:10.371170+0000","last_became_peered":"2023-11-28T18:29:10.371170+0000","last_unstale":"2023-11-28T18:29:10.375368+0000","last_undegraded":"2023-11-28T18:29:10.375368+0000","last_fullsized":"2023-11-28T18:29:10.375368+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:50.642782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.352398+0000","last_change":"2023-11-28T18:29:09.343728+0000","last_active":"2023-11-28T18:29:10.352398+0000","last_peered":"2023-11-28T18:29:10.352398+0000","last_clean":"2023-11-28T18:29:10.352398+0000","last_became_active":"2023-11-28T18:29:09.343451+0000","last_became_peered":"2023-11-28T18:29:09.343451+0000","last_unstale":"2023-11-28T18:29:10.352398+0000","last_undegraded":"2023-11-28T18:29:10.352398+0000","last_fullsized":"2023-11-28T18:29:10.352398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:43:40.103435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.393321+0000","last_change":"2023-11-28T18:29:10.392408+0000","last_active":"2023-11-28T18:29:10.393321+0000","last_peered":"2023-11-28T18:29:10.393321+0000","last_clean":"2023-11-28T18:29:10.393321+0000","last_became_active":"2023-11-28T18:29:10.392143+0000","last_became_peered":"2023-11-28T18:29:10.392143+0000","last_unstale":"2023-11-28T18:29:10.393321+0000","last_undegraded":"2023-11-28T18:29:10.393321+0000","last_fullsized":"2023-11-28T18:29:10.393321+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:04:40.397373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.357325+0000","last_change":"2023-11-28T18:29:09.336697+0000","last_active":"2023-11-28T18:29:10.357325+0000","last_peered":"2023-11-28T18:29:10.357325+0000","last_clean":"2023-11-28T18:29:10.357325+0000","last_became_active":"2023-11-28T18:29:09.335301+0000","last_became_peered":"2023-11-28T18:29:09.335301+0000","last_unstale":"2023-11-28T18:29:10.357325+0000","last_undegraded":"2023-11-28T18:29:10.357325+0000","last_fullsized":"2023-11-28T18:29:10.357325+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:58.161257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.381152+0000","last_change":"2023-11-28T18:29:10.371910+0000","last_active":"2023-11-28T18:29:10.381152+0000","last_peered":"2023-11-28T18:29:10.381152+0000","last_clean":"2023-11-28T18:29:10.381152+0000","last_became_active":"2023-11-28T18:29:10.371626+0000","last_became_peered":"2023-11-28T18:29:10.371626+0000","last_unstale":"2023-11-28T18:29:10.381152+0000","last_undegraded":"2023-11-28T18:29:10.381152+0000","last_fullsized":"2023-11-28T18:29:10.381152+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:36:53.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.353094+0000","last_change":"2023-11-28T18:29:09.346884+0000","last_active":"2023-11-28T18:29:10.353094+0000","last_peered":"2023-11-28T18:29:10.353094+0000","last_clean":"2023-11-28T18:29:10.353094+0000","last_became_active":"2023-11-28T18:29:09.345946+0000","last_became_peered":"2023-11-28T18:29:09.345946+0000","last_unstale":"2023-11-28T18:29:10.353094+0000","last_undegraded":"2023-11-28T18:29:10.353094+0000","last_fullsized":"2023-11-28T18:29:10.353094+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:21:04.912542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.367838+0000","last_change":"2023-11-28T18:29:10.363077+0000","last_active":"2023-11-28T18:29:10.367838+0000","last_peered":"2023-11-28T18:29:10.367838+0000","last_clean":"2023-11-28T18:29:10.367838+0000","last_became_active":"2023-11-28T18:29:10.362767+0000","last_became_peered":"2023-11-28T18:29:10.362767+0000","last_unstale":"2023-11-28T18:29:10.367838+0000","last_undegraded":"2023-11-28T18:29:10.367838+0000","last_fullsized":"2023-11-28T18:29:10.367838+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:09.315478+0000","last_clean_scrub_stamp":"2023-11-28T18:29:09.315478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:33:40.038310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-28T18:29:10.362527+0000","last_change":"2023-11-28T18:29:09.347146+0000","last_active":"2023-11-28T18:29:10.362527+0000","last_peered":"2023-11-28T18:29:10.362527+0000","last_clean":"2023-11-28T18:29:10.362527+0000","last_became_active":"2023-11-28T18:29:09.346864+0000","last_became_peered":"2023-11-28T18:29:09.346864+0000","last_unstale":"2023-11-28T18:29:10.362527+0000","last_undegraded":"2023-11-28T18:29:10.362527+0000","last_fullsized":"2023-11-28T18:29:10.362527+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:29:08.307850+0000","last_clean_scrub_stamp":"2023-11-28T18:29:08.307850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:59:35.230058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":225,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-28T18:29:09.335981+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:29:09.335981+0000","last_peered":"2023-11-28T18:29:09.335981+0000","last_clean":"2023-11-28T18:29:09.335981+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:29:09.335981+0000","last_undegraded":"2023-11-28T18:29:09.335981+0000","last_fullsized":"2023-11-28T18:29:09.335981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46732,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325108,"statfs":{"total":96636764160,"available":96588910592,"internally_reserved":0,"allocated":1699840,"data_stored":1095205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.64200000000000002,"15min":0.71299999999999997},"min":{"1min":0.38600000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.94899999999999995,"5min":2.427,"15min":2.427},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.71499999999999997},"min":{"1min":0.39500000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.93600000000000005,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.64400000000000002}]},{"osd":1,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66200000000000003,"15min":0.71499999999999997},"min":{"1min":0.44500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88400000000000001,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68200000000000005,"15min":0.75},"min":{"1min":0.46100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":1.079,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.80900000000000005}]},{"osd":2,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.69899999999999995,"15min":0.76900000000000002},"min":{"1min":0.42799999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.998,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67600000000000005,"15min":0.73799999999999999},"min":{"1min":0.41299999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.97299999999999998,"5min":2.411,"15min":2.411},"last":0.78800000000000003}]},{"osd":3,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.67800000000000005,"15min":0.755},"min":{"1min":0.40899999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92000000000000004,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72499999999999998,"15min":0.79600000000000004},"min":{"1min":0.49299999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.0389999999999999,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.0289999999999999}]},{"osd":4,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58299999999999996,"15min":0.60199999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83699999999999997,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58899999999999997,"15min":0.63700000000000001},"min":{"1min":0.34899999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.96299999999999997}]},{"osd":5,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.67300000000000004},"min":{"1min":0.38800000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.65000000000000002,"15min":0.69599999999999995},"min":{"1min":0.443,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92800000000000005,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.88300000000000001}]},{"osd":6,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69399999999999995,"15min":0.73799999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.998,"5min":1.042,"15min":1.042},"last":0.996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.65100000000000002,"15min":0.72299999999999998},"min":{"1min":0.40999999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":1.01}]}]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40564,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331276,"statfs":{"total":96636764160,"available":96595226624,"internally_reserved":0,"allocated":1085440,"data_stored":497686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64100000000000001,"15min":0.63700000000000001},"min":{"1min":0.42099999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.86299999999999999,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.71399999999999997,"15min":0.72999999999999998},"min":{"1min":0.49299999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.751}]},{"osd":1,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68799999999999994,"15min":0.79600000000000004},"min":{"1min":0.46800000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96899999999999997,"5min":4.2389999999999999,"15min":4.2389999999999999},"last":0.624},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70899999999999996,"15min":0.78600000000000003},"min":{"1min":0.39500000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.94499999999999995,"5min":4.218,"15min":4.218},"last":0.69099999999999995}]},{"osd":2,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67700000000000005,"15min":0.71299999999999997},"min":{"1min":0.373,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84999999999999998,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67700000000000005,"15min":0.69999999999999996},"min":{"1min":0.48599999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.86099999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69599999999999995,"15min":0.72399999999999998},"min":{"1min":0.52600000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91900000000000004,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.68200000000000005,"15min":0.68799999999999994},"min":{"1min":0.50700000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91900000000000004,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.72999999999999998}]},{"osd":4,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.624,"15min":0.66200000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.63400000000000001,"15min":0.68200000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.80200000000000005}]},{"osd":5,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63400000000000001,"15min":0.67600000000000005},"min":{"1min":0.36199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.69999999999999996,"15min":0.754},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95099999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.94599999999999995}]},{"osd":7,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71299999999999997,"15min":0.76700000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.002,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.70399999999999996,"15min":0.746},"min":{"1min":0.443,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90200000000000002,"5min":1,"15min":1},"last":0.78800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574921,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":504781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69199999999999995,"15min":0.71099999999999997},"min":{"1min":0.34399999999999997,"5min":0.31,"15min":0.31},"max":{"1min":2.1299999999999999,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70799999999999996,"15min":0.748},"min":{"1min":0.52200000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89500000000000002,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":0.72699999999999998}]},{"osd":1,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.68600000000000005,"15min":0.73499999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92400000000000004,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72899999999999998,"15min":0.79500000000000004},"min":{"1min":0.45800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.2109999999999999,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.82699999999999996}]},{"osd":2,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67900000000000005,"15min":0.749},"min":{"1min":0.40899999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84599999999999997,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.624},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69299999999999995,"15min":0.76700000000000002},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.014,"5min":4.0149999999999997,"15min":4.0149999999999997},"last":0.65000000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.73999999999999999,"15min":0.82099999999999995},"min":{"1min":0.48999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.254,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71299999999999997,"15min":0.81499999999999995},"min":{"1min":0.46300000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.98599999999999999,"5min":3.9900000000000002,"15min":3.9900000000000002},"last":0.56200000000000006}]},{"osd":4,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61499999999999999,"15min":0.60899999999999999},"min":{"1min":0.32900000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68400000000000005,"15min":0.70799999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.68700000000000006}]},{"osd":6,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67300000000000004,"15min":0.67900000000000005},"min":{"1min":0.35499999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70299999999999996,"15min":0.71699999999999997},"min":{"1min":0.438,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.262,"5min":1.756,"15min":1.756},"last":0.747}]},{"osd":7,"last update":"Tue Nov 28 18:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.74299999999999999,"15min":0.755},"min":{"1min":0.42399999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.74199999999999999,"15min":0.754},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.76000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68360,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303480,"statfs":{"total":96636764160,"available":96566763520,"internally_reserved":0,"allocated":1105920,"data_stored":504941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63900000000000001,"15min":0.68400000000000005},"min":{"1min":0.32900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.82799999999999996,"5min":2.363,"15min":2.363},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.43099999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.73399999999999999}]},{"osd":1,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63600000000000001,"15min":0.625},"min":{"1min":0.441,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93200000000000005,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67100000000000004,"15min":0.65600000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.86799999999999999}]},{"osd":2,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71099999999999997,"15min":0.69899999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93100000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71199999999999997,"15min":0.72599999999999998},"min":{"1min":0.50900000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94399999999999995,"5min":1.871,"15min":1.871},"last":0.753}]},{"osd":3,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.46400000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67800000000000005,"15min":0.68300000000000005},"min":{"1min":0.47699999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96699999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.79300000000000004}]},{"osd":5,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.94299999999999995,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.443},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59499999999999997,"15min":0.63100000000000001},"min":{"1min":0.33400000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80300000000000005,"5min":1.79,"15min":1.79},"last":0.39100000000000001}]},{"osd":6,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61799999999999999,"15min":0.64300000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91500000000000004,"5min":1.123,"15min":1.123},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68400000000000005,"15min":0.70499999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0800000000000001,"5min":1.925,"15min":1.925},"last":0.64000000000000001}]},{"osd":7,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68000000000000005,"15min":0.71299999999999997},"min":{"1min":0.49399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88400000000000001,"5min":1.843,"15min":1.843},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.747,"15min":0.77400000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96299999999999997,"5min":1.593,"15min":1.593},"last":0.89000000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574921,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62784,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309056,"statfs":{"total":96636764160,"available":96572473344,"internally_reserved":0,"allocated":1097728,"data_stored":504781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.434,"15min":0.41699999999999998},"min":{"1min":0.27100000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.80900000000000005,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48899999999999999,"15min":0.47499999999999998},"min":{"1min":0.308,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.96399999999999997,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.41099999999999998}]},{"osd":1,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.45700000000000002,"15min":0.46300000000000002},"min":{"1min":0.28799999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.73399999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.54400000000000004,"15min":0.51600000000000001},"min":{"1min":0.318,"5min":0.247,"15min":0.247},"max":{"1min":0.75600000000000001,"5min":4.5599999999999996,"15min":4.5599999999999996},"last":0.433}]},{"osd":2,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54800000000000004,"15min":0.53000000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2110000000000001,"5min":2.48,"15min":2.48},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.437,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.68700000000000006,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.52000000000000002}]},{"osd":4,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.89400000000000002,"15min":0.81499999999999995},"min":{"1min":0.441,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0189999999999999,"5min":12.695,"15min":12.695},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.80900000000000005,"15min":0.78700000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.052,"5min":6.3250000000000002,"15min":6.3250000000000002},"last":0.56999999999999995}]},{"osd":5,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.78600000000000003,"15min":0.82799999999999996},"min":{"1min":0.49299999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0309999999999999,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.79900000000000004,"15min":0.78200000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0840000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.59399999999999997}]},{"osd":6,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.83699999999999997,"15min":0.81499999999999995},"min":{"1min":0.53100000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97199999999999998,"5min":6.1790000000000003,"15min":6.1790000000000003},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.79500000000000004,"15min":0.80200000000000005},"min":{"1min":0.53300000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0680000000000001,"5min":1.984,"15min":1.984},"last":0.64200000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.81299999999999994,"15min":0.81899999999999995},"min":{"1min":0.55400000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1319999999999999,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.84299999999999997,"15min":0.82299999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.008,"5min":6.2249999999999996,"15min":6.2249999999999996},"last":0.73199999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1097728,"data_stored":504781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.47799999999999998,"15min":0.5},"min":{"1min":0.28699999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.47099999999999997,"15min":0.503},"min":{"1min":0.29799999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66800000000000004,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.42499999999999999}]},{"osd":1,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.496,"15min":0.52200000000000002},"min":{"1min":0.36599999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70999999999999996,"5min":1.48,"15min":1.48},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.872,"5min":0.57799999999999996,"15min":0.53800000000000003},"min":{"1min":0.34200000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":8.3539999999999992,"5min":8.3539999999999992,"15min":8.3539999999999992},"last":0.38700000000000001}]},{"osd":3,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.501,"15min":0.50700000000000001},"min":{"1min":0.34499999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":3.1549999999999998,"5min":3.1549999999999998,"15min":3.1549999999999998},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.56200000000000006,"15min":0.57099999999999995},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.4790000000000001,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":0.40600000000000003}]},{"osd":4,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.77300000000000002,"15min":0.77500000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.5179999999999998,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.76000000000000001,"15min":0.77900000000000003},"min":{"1min":0.51700000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.8170000000000002,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.78600000000000003}]},{"osd":5,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.75800000000000001,"15min":0.77000000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.0709999999999997,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.80800000000000005,"15min":0.96999999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0780000000000001,"5min":7.4820000000000002,"15min":7.4820000000000002},"last":0.76500000000000001}]},{"osd":6,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":0.85599999999999998,"15min":0.82399999999999995},"min":{"1min":0.53100000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":12.019,"5min":12.019,"15min":12.019},"last":0.877},{"interface":"front","average":{"1min":1.1220000000000001,"5min":0.83999999999999997,"15min":0.876},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":8.7400000000000002,"5min":8.7400000000000002,"15min":8.7400000000000002},"last":0.71099999999999997}]},{"osd":7,"last update":"Tue Nov 28 18:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":0.85599999999999998,"15min":0.85399999999999998},"min":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":8.327,"5min":8.327,"15min":8.327},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.121,"5min":0.86099999999999999,"15min":0.92000000000000004},"min":{"1min":0.57499999999999996,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":8.609,"5min":8.609,"15min":8.609},"last":0.82899999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40588,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331252,"statfs":{"total":96636764160,"available":96595202048,"internally_reserved":0,"allocated":1101824,"data_stored":504851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.41199999999999998,"15min":0.42499999999999999},"min":{"1min":0.29999999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.56100000000000005,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.372},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.29899999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.65000000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.503}]},{"osd":2,"last update":"Tue Nov 28 18:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.44800000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67400000000000004,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.48599999999999999,"15min":0.47999999999999998},"min":{"1min":0.38800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.73399999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.41499999999999998}]},{"osd":3,"last update":"Tue Nov 28 18:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.74199999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.502,"15min":0.49199999999999999},"min":{"1min":0.34899999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71599999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.46600000000000003}]},{"osd":4,"last update":"Tue Nov 28 18:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72399999999999998,"15min":0.746},"min":{"1min":0.42699999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.032,"5min":1.155,"15min":1.155},"last":0.78200000000000003}]},{"osd":5,"last update":"Tue Nov 28 18:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71099999999999997,"15min":0.74299999999999999},"min":{"1min":0.54100000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93700000000000006,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.70999999999999996,"15min":0.74399999999999999},"min":{"1min":0.502,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.63300000000000001}]},{"osd":6,"last update":"Tue Nov 28 18:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.747,"15min":0.77500000000000002},"min":{"1min":0.57299999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0549999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76800000000000002,"15min":0.80300000000000005},"min":{"1min":0.55900000000000005,"5min":0.498,"15min":0.498},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.77100000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72699999999999998,"15min":0.748},"min":{"1min":0.56000000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0669999999999999,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.74099999999999999,"15min":0.77300000000000002},"min":{"1min":0.58399999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97599999999999998,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.64500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574921,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37028,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334812,"statfs":{"total":96636764160,"available":96598847488,"internally_reserved":0,"allocated":1716224,"data_stored":1097467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 18:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.41999999999999998,"15min":0.44},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.628,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.33000000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71299999999999997,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.55100000000000005}]},{"osd":2,"last update":"Tue Nov 28 18:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.48899999999999999,"15min":0.50600000000000001},"min":{"1min":0.34599999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.629,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.495,"15min":0.51300000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.754,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.496}]},{"osd":3,"last update":"Tue Nov 28 18:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47899999999999998,"15min":0.48899999999999999},"min":{"1min":0.32400000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78600000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.443},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52900000000000003,"15min":0.55100000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.69899999999999995,"5min":1.087,"15min":1.087},"last":0.51800000000000002}]},{"osd":4,"last update":"Tue Nov 28 18:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.56499999999999995,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86899999999999999,"5min":1.131,"15min":1.131},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.46899999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.995,"5min":1.244,"15min":1.244},"last":0.86099999999999999}]},{"osd":5,"last update":"Tue Nov 28 18:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72699999999999998,"15min":0.72799999999999998},"min":{"1min":0.48299999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.88600000000000001,"5min":1.121,"15min":1.121},"last":0.873},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.73699999999999999,"15min":0.74099999999999999},"min":{"1min":0.54700000000000004,"5min":0.502,"15min":0.502},"max":{"1min":0.97799999999999998,"5min":1.075,"15min":1.075},"last":0.71399999999999997}]},{"osd":6,"last update":"Tue Nov 28 18:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.72799999999999998,"15min":0.73299999999999998},"min":{"1min":0.51400000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.92900000000000005,"5min":1.407,"15min":1.407},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.74299999999999999,"15min":0.73699999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96099999999999997,"5min":1.089,"15min":1.089},"last":0.81999999999999995}]},{"osd":7,"last update":"Tue Nov 28 18:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.75700000000000001,"15min":0.755},"min":{"1min":0.56899999999999995,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.054,"5min":1.212,"15min":1.212},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.754,"15min":0.75},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.94699999999999995,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.84199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:29:14.832 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T18:29:14.832 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-28T18:29:15.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:14.998+0000 7fd644abb640 1 Processor -- start 2023-11-28T18:29:15.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.002+0000 7fd644abb640 1 -- start start 2023-11-28T18:29:15.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.002+0000 7fd644abb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd64015b030 0x7fd6400709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:15.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.002+0000 7fd644abb640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6400701d0 con 0x7fd640071960 2023-11-28T18:29:15.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.002+0000 7fd644abb640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd64006df80 con 0x7fd6400b4290 2023-11-28T18:29:15.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.002+0000 7fd644abb640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd640071df0 con 0x7fd64015b030 2023-11-28T18:29:15.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.002+0000 7fd635d9b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd640071960 0x7fd64015f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57784/0 (socket says 172.21.15.105:57784) 2023-11-28T18:29:15.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.002+0000 7fd635d9b640 1 -- 172.21.15.105:0/2020183161 learned_addr learned my addr 172.21.15.105:0/2020183161 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:15.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.002+0000 7fd63ed9d640 1 --2- 172.21.15.105:0/2020183161 >> v2:172.21.15.148:3300/0 conn(0x7fd64015b030 0x7fd6400709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:15.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.002+0000 7fd63dd9b640 1 -- 172.21.15.105:0/2020183161 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1790598355 0 0) 0x7fd6400701d0 con 0x7fd640071960 2023-11-28T18:29:15.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.002+0000 7fd63dd9b640 1 -- 172.21.15.105:0/2020183161 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd62c003530 con 0x7fd640071960 2023-11-28T18:29:15.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.002+0000 7fd63dd9b640 1 -- 172.21.15.105:0/2020183161 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020968054 0 0) 0x7fd64006df80 con 0x7fd6400b4290 2023-11-28T18:29:15.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.002+0000 7fd63ed9d640 1 -- 172.21.15.105:0/2020183161 >> v1:172.21.15.148:6789/0 conn(0x7fd6400b4290 legacy=0x7fd6400b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:15.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.002+0000 7fd63ed9d640 1 -- 172.21.15.105:0/2020183161 >> v1:172.21.15.105:6789/0 conn(0x7fd640071960 legacy=0x7fd64015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:15.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.002+0000 7fd63ed9d640 1 -- 172.21.15.105:0/2020183161 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6400720d0 con 0x7fd64015b030 2023-11-28T18:29:15.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.006+0000 7fd63ed9d640 1 --2- 172.21.15.105:0/2020183161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd64015b030 0x7fd6400709b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd638009d90 tx=0x7fd63801c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b2fcba6f10802df server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:15.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.006+0000 7fd63dd9b640 1 -- 172.21.15.105:0/2020183161 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6380a58f0 con 0x7fd64015b030 2023-11-28T18:29:15.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.006+0000 7fd63dd9b640 1 -- 172.21.15.105:0/2020183161 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6380a5a50 con 0x7fd64015b030 2023-11-28T18:29:15.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.006+0000 7fd63dd9b640 1 -- 172.21.15.105:0/2020183161 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6380a5de0 con 0x7fd64015b030 2023-11-28T18:29:15.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.006+0000 7fd644abb640 1 -- 172.21.15.105:0/2020183161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd64015b030 msgr2=0x7fd6400709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:15.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.006+0000 7fd644abb640 1 --2- 172.21.15.105:0/2020183161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd64015b030 0x7fd6400709b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd638009d90 tx=0x7fd63801c920 comp rx=0 tx=0).stop 2023-11-28T18:29:15.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.006+0000 7fd644abb640 1 -- 172.21.15.105:0/2020183161 shutdown_connections 2023-11-28T18:29:15.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.006+0000 7fd644abb640 1 --2- 172.21.15.105:0/2020183161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd64015b030 0x7fd6400709b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:15.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.006+0000 7fd644abb640 1 -- 172.21.15.105:0/2020183161 >> 172.21.15.105:0/2020183161 conn(0x7fd6400ceb90 msgr2=0x7fd6400cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:15.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.006+0000 7fd644abb640 1 -- 172.21.15.105:0/2020183161 shutdown_connections 2023-11-28T18:29:15.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.006+0000 7fd644abb640 1 -- 172.21.15.105:0/2020183161 wait complete. 2023-11-28T18:29:15.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd644abb640 1 Processor -- start 2023-11-28T18:29:15.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd644abb640 1 -- start start 2023-11-28T18:29:15.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd644abb640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6400b4290 0x7fd640154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:15.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd644abb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd64015b030 0x7fd6401f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:15.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd644abb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6401f7630 0x7fd6401587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:15.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd644abb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd640154cc0 con 0x7fd64015b030 2023-11-28T18:29:15.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd644abb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6400b2d60 con 0x7fd6401f7630 2023-11-28T18:29:15.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd644abb640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6401f6a10 con 0x7fd6400b4290 2023-11-28T18:29:15.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd635d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd64015b030 0x7fd6401f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:15.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd635d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd64015b030 0x7fd6401f61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54050/0 (socket says 172.21.15.105:54050) 2023-11-28T18:29:15.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd635d9b640 1 -- 172.21.15.105:0/3611876751 learned_addr learned my addr 172.21.15.105:0/3611876751 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:15.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd63e59c640 1 --2- 172.21.15.105:0/3611876751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6400b4290 0x7fd640154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:15.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd63ed9d640 1 --2- 172.21.15.105:0/3611876751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6401f7630 0x7fd6401587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:15.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd635d9b640 1 -- 172.21.15.105:0/3611876751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6400b4290 msgr2=0x7fd640154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:15.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd635d9b640 1 --2- 172.21.15.105:0/3611876751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6400b4290 0x7fd640154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:15.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd635d9b640 1 -- 172.21.15.105:0/3611876751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6401f7630 msgr2=0x7fd6401587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:15.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd635d9b640 1 --2- 172.21.15.105:0/3611876751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6401f7630 0x7fd6401587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:15.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd635d9b640 1 -- 172.21.15.105:0/3611876751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd638009a70 con 0x7fd64015b030 2023-11-28T18:29:15.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd635d9b640 1 --2- 172.21.15.105:0/3611876751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd64015b030 0x7fd6401f61c0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fd62800c450 tx=0x7fd62800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:15.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd6377fe640 1 -- 172.21.15.105:0/3611876751 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628005cf0 con 0x7fd64015b030 2023-11-28T18:29:15.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd6377fe640 1 -- 172.21.15.105:0/3611876751 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd628005e50 con 0x7fd64015b030 2023-11-28T18:29:15.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd644abb640 1 -- 172.21.15.105:0/3611876751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6401f6cf0 con 0x7fd64015b030 2023-11-28T18:29:15.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd6377fe640 1 -- 172.21.15.105:0/3611876751 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628007290 con 0x7fd64015b030 2023-11-28T18:29:15.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd644abb640 1 -- 172.21.15.105:0/3611876751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6401f6700 con 0x7fd64015b030 2023-11-28T18:29:15.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.010+0000 7fd644abb640 1 -- 172.21.15.105:0/3611876751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6400b2f60 con 0x7fd64015b030 2023-11-28T18:29:15.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.014+0000 7fd6377fe640 1 -- 172.21.15.105:0/3611876751 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd6280073f0 con 0x7fd64015b030 2023-11-28T18:29:15.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.014+0000 7fd6377fe640 1 --2- 172.21.15.105:0/3611876751 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd6200668d0 0x7fd620068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:15.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.014+0000 7fd6377fe640 1 -- 172.21.15.105:0/3611876751 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd628083b10 con 0x7fd64015b030 2023-11-28T18:29:15.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.014+0000 7fd63e59c640 1 --2- 172.21.15.105:0/3611876751 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd6200668d0 0x7fd620068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:15.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.014+0000 7fd63e59c640 1 --2- 172.21.15.105:0/3611876751 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd6200668d0 0x7fd620068d90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd630000f70 tx=0x7fd630009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:15.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.018+0000 7fd6377fe640 1 -- 172.21.15.105:0/3611876751 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd6400b2f60 con 0x7fd64015b030 2023-11-28T18:29:15.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.222+0000 7fd644abb640 1 -- 172.21.15.105:0/3611876751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd6400b2f20 con 0x7fd64015b030 2023-11-28T18:29:15.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.226+0000 7fd6377fe640 1 -- 172.21.15.105:0/3611876751 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7fd628017070 con 0x7fd64015b030 2023-11-28T18:29:15.229 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2023-11-28T18:29:15.230 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDpL2ZlSt0VMxAAQ8+j1BgBPMJ2cuiZj8aG0w== 2023-11-28T18:29:15.230 INFO:teuthology.orchestra.run.smithi105.stdout: caps mds = "allow" 2023-11-28T18:29:15.230 INFO:teuthology.orchestra.run.smithi105.stdout: caps mon = "allow r" 2023-11-28T18:29:15.230 INFO:teuthology.orchestra.run.smithi105.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-28T18:29:15.230 INFO:teuthology.orchestra.run.smithi105.stderr:updated caps for client.0 2023-11-28T18:29:15.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.226+0000 7fd644abb640 1 -- 172.21.15.105:0/3611876751 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd6200668d0 msgr2=0x7fd620068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:15.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.226+0000 7fd644abb640 1 --2- 172.21.15.105:0/3611876751 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd6200668d0 0x7fd620068d90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd630000f70 tx=0x7fd630009040 comp rx=0 tx=0).stop 2023-11-28T18:29:15.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.226+0000 7fd644abb640 1 -- 172.21.15.105:0/3611876751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd64015b030 msgr2=0x7fd6401f61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:15.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.226+0000 7fd644abb640 1 --2- 172.21.15.105:0/3611876751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd64015b030 0x7fd6401f61c0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fd62800c450 tx=0x7fd62800c910 comp rx=0 tx=0).stop 2023-11-28T18:29:15.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.226+0000 7fd644abb640 1 -- 172.21.15.105:0/3611876751 shutdown_connections 2023-11-28T18:29:15.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.226+0000 7fd644abb640 1 --2- 172.21.15.105:0/3611876751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6401f7630 0x7fd6401587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:15.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.226+0000 7fd644abb640 1 --2- 172.21.15.105:0/3611876751 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd6200668d0 0x7fd620068d90 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:15.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.226+0000 7fd644abb640 1 --2- 172.21.15.105:0/3611876751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd64015b030 0x7fd6401f61c0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:15.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.226+0000 7fd644abb640 1 --2- 172.21.15.105:0/3611876751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6400b4290 0x7fd640154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:15.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.226+0000 7fd644abb640 1 -- 172.21.15.105:0/3611876751 >> 172.21.15.105:0/3611876751 conn(0x7fd6400ceb90 msgr2=0x7fd640072fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:15.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.226+0000 7fd644abb640 1 -- 172.21.15.105:0/3611876751 shutdown_connections 2023-11-28T18:29:15.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.226+0000 7fd644abb640 1 -- 172.21.15.105:0/3611876751 wait complete. 2023-11-28T18:29:15.251 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:29:15.300 INFO:tasks.ceph.mds.a.smithi105.stderr:2023-11-28T18:29:15.294+0000 7f2ea22d3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T18:29:15.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.386+0000 7fcd33c79640 1 Processor -- start 2023-11-28T18:29:15.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.386+0000 7fcd33c79640 1 -- start start 2023-11-28T18:29:15.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.386+0000 7fcd33c79640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcd2c0b2db0 0x7fcd2c0ae660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:15.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.386+0000 7fcd33c79640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd2c0ad670 con 0x7fcd2c159e30 2023-11-28T18:29:15.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.386+0000 7fcd33c79640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd2c0b3210 con 0x7fcd2c0712c0 2023-11-28T18:29:15.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.386+0000 7fcd33c79640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c06ef80 con 0x7fcd2c0b2db0 2023-11-28T18:29:15.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.386+0000 7fcd31a15640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fcd2c159e30 0x7fcd2c06e7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57800/0 (socket says 172.21.15.105:57800) 2023-11-28T18:29:15.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.386+0000 7fcd31a15640 1 -- 172.21.15.105:0/844468448 learned_addr learned my addr 172.21.15.105:0/844468448 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:15.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.390+0000 7fcd31214640 1 --2- 172.21.15.105:0/844468448 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcd2c0b2db0 0x7fcd2c0ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:15.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.390+0000 7fcd30a13640 1 -- 172.21.15.105:0/844468448 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697849753 0 0) 0x7fcd2c0ad670 con 0x7fcd2c159e30 2023-11-28T18:29:15.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.390+0000 7fcd30a13640 1 -- 172.21.15.105:0/844468448 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd18003530 con 0x7fcd2c159e30 2023-11-28T18:29:15.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.390+0000 7fcd30a13640 1 -- 172.21.15.105:0/844468448 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1699524249 0 0) 0x7fcd240014a0 con 0x7fcd2c0712c0 2023-11-28T18:29:15.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.390+0000 7fcd30a13640 1 -- 172.21.15.105:0/844468448 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd2c0ad670 con 0x7fcd2c0712c0 2023-11-28T18:29:15.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.390+0000 7fcd31214640 1 -- 172.21.15.105:0/844468448 >> v1:172.21.15.148:6789/0 conn(0x7fcd2c0712c0 legacy=0x7fcd2c15a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:15.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.390+0000 7fcd31214640 1 -- 172.21.15.105:0/844468448 >> v1:172.21.15.105:6789/0 conn(0x7fcd2c159e30 legacy=0x7fcd2c06e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:15.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.390+0000 7fcd30a13640 1 -- 172.21.15.105:0/844468448 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3661318316 0 0) 0x7fcd18003530 con 0x7fcd2c159e30 2023-11-28T18:29:15.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.390+0000 7fcd31214640 1 -- 172.21.15.105:0/844468448 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2c0ad8a0 con 0x7fcd2c0b2db0 2023-11-28T18:29:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.390+0000 7fcd31214640 1 --2- 172.21.15.105:0/844468448 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcd2c0b2db0 0x7fcd2c0ae660 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0097f0 tx=0x7fcd1c01c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec555537fb4a0ebe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:15.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.394+0000 7fcd30a13640 1 -- 172.21.15.105:0/844468448 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd28001680 con 0x7fcd2c0b2db0 2023-11-28T18:29:15.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.394+0000 7fcd30a13640 1 -- 172.21.15.105:0/844468448 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1c0a7050 con 0x7fcd2c0b2db0 2023-11-28T18:29:15.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.394+0000 7fcd30a13640 1 -- 172.21.15.105:0/844468448 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c0ab620 con 0x7fcd2c0b2db0 2023-11-28T18:29:15.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.394+0000 7fcd33c79640 1 -- 172.21.15.105:0/844468448 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcd2c0b2db0 msgr2=0x7fcd2c0ae660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:15.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.394+0000 7fcd33c79640 1 --2- 172.21.15.105:0/844468448 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcd2c0b2db0 0x7fcd2c0ae660 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0097f0 tx=0x7fcd1c01c3b0 comp rx=0 tx=0).stop 2023-11-28T18:29:15.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.394+0000 7fcd33c79640 1 -- 172.21.15.105:0/844468448 shutdown_connections 2023-11-28T18:29:15.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.394+0000 7fcd33c79640 1 --2- 172.21.15.105:0/844468448 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcd2c0b2db0 0x7fcd2c0ae660 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:15.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.394+0000 7fcd33c79640 1 -- 172.21.15.105:0/844468448 >> 172.21.15.105:0/844468448 conn(0x7fcd2c06b860 msgr2=0x7fcd2c06bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:15.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.394+0000 7fcd33c79640 1 -- 172.21.15.105:0/844468448 shutdown_connections 2023-11-28T18:29:15.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.394+0000 7fcd33c79640 1 -- 172.21.15.105:0/844468448 wait complete. 2023-11-28T18:29:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd33c79640 1 Processor -- start 2023-11-28T18:29:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd33c79640 1 -- start start 2023-11-28T18:29:15.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd33c79640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcd2c0712c0 0x7fcd2c142670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:15.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd33c79640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd2c0b2db0 0x7fcd2c0cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:15.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd33c79640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd2c159e30 0x7fcd2c0cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:15.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd33c79640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c1463b0 con 0x7fcd2c159e30 2023-11-28T18:29:15.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd33c79640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c0cbcb0 con 0x7fcd2c0b2db0 2023-11-28T18:29:15.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd33c79640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c143fe0 con 0x7fcd2c0712c0 2023-11-28T18:29:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd32216640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd2c159e30 0x7fcd2c0cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd31a15640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcd2c0712c0 0x7fcd2c142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd32216640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd2c159e30 0x7fcd2c0cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54058/0 (socket says 172.21.15.105:54058) 2023-11-28T18:29:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd32216640 1 -- 172.21.15.105:0/2627164967 learned_addr learned my addr 172.21.15.105:0/2627164967 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd31214640 1 --2- 172.21.15.105:0/2627164967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd2c0b2db0 0x7fcd2c0cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd32216640 1 -- 172.21.15.105:0/2627164967 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcd2c0712c0 msgr2=0x7fcd2c142670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd32216640 1 --2- 172.21.15.105:0/2627164967 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcd2c0712c0 0x7fcd2c142670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd32216640 1 -- 172.21.15.105:0/2627164967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd2c0b2db0 msgr2=0x7fcd2c0cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd32216640 1 --2- 172.21.15.105:0/2627164967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd2c0b2db0 0x7fcd2c0cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd32216640 1 -- 172.21.15.105:0/2627164967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c009420 con 0x7fcd2c159e30 2023-11-28T18:29:15.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd32216640 1 --2- 172.21.15.105:0/2627164967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd2c159e30 0x7fcd2c0cb650 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400c790 tx=0x7fcd2400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:15.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd227fc640 1 -- 172.21.15.105:0/2627164967 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2400cea0 con 0x7fcd2c159e30 2023-11-28T18:29:15.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd227fc640 1 -- 172.21.15.105:0/2627164967 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd24003ba0 con 0x7fcd2c159e30 2023-11-28T18:29:15.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd33c79640 1 -- 172.21.15.105:0/2627164967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2c071c60 con 0x7fcd2c159e30 2023-11-28T18:29:15.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd227fc640 1 -- 172.21.15.105:0/2627164967 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2400a850 con 0x7fcd2c159e30 2023-11-28T18:29:15.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd33c79640 1 -- 172.21.15.105:0/2627164967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2c1445a0 con 0x7fcd2c159e30 2023-11-28T18:29:15.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.398+0000 7fcd33c79640 1 -- 172.21.15.105:0/2627164967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd2c067570 con 0x7fcd2c159e30 2023-11-28T18:29:15.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.402+0000 7fcd227fc640 1 -- 172.21.15.105:0/2627164967 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fcd2400a9b0 con 0x7fcd2c159e30 2023-11-28T18:29:15.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.402+0000 7fcd227fc640 1 --2- 172.21.15.105:0/2627164967 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcd100668d0 0x7fcd10068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:15.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.402+0000 7fcd227fc640 1 -- 172.21.15.105:0/2627164967 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcd24084210 con 0x7fcd2c159e30 2023-11-28T18:29:15.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.402+0000 7fcd31a15640 1 --2- 172.21.15.105:0/2627164967 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcd100668d0 0x7fcd10068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:15.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.402+0000 7fcd31a15640 1 --2- 172.21.15.105:0/2627164967 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcd100668d0 0x7fcd10068d90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fcd280011f0 tx=0x7fcd28001cc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:15.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.410+0000 7fcd227fc640 1 -- 172.21.15.105:0/2627164967 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcd24051320 con 0x7fcd2c159e30 2023-11-28T18:29:15.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.594+0000 7fcd33c79640 1 -- 172.21.15.105:0/2627164967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd2c0b3210 con 0x7fcd2c159e30 2023-11-28T18:29:15.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.594+0000 7fcd227fc640 1 -- 172.21.15.105:0/2627164967 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4927 (secure 0 0 0) 0x7fcd2c0b3210 con 0x7fcd2c159e30 2023-11-28T18:29:15.598 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:29:15.599 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":87,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/1464337040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1464337040},{"type":"v1","addr":"172.21.15.148:6834","nonce":1464337040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3461832163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":3461832163},{"type":"v1","addr":"172.21.15.148:6835","nonce":3461832163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7665,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/4162499249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":4162499249},{"type":"v1","addr":"172.21.15.105:6835","nonce":4162499249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:29:09.707308+0000","modified":"2023-11-28T18:29:12.751031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7689},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7689":{"gid":7689,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/3239728352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3239728352},{"type":"v1","addr":"172.21.15.105:6837","nonce":3239728352}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-28T18:29:15.599 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 87 2023-11-28T18:29:15.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.594+0000 7fcd33c79640 1 -- 172.21.15.105:0/2627164967 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcd100668d0 msgr2=0x7fcd10068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:15.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.594+0000 7fcd33c79640 1 --2- 172.21.15.105:0/2627164967 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcd100668d0 0x7fcd10068d90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fcd280011f0 tx=0x7fcd28001cc0 comp rx=0 tx=0).stop 2023-11-28T18:29:15.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.594+0000 7fcd33c79640 1 -- 172.21.15.105:0/2627164967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd2c159e30 msgr2=0x7fcd2c0cb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:15.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.594+0000 7fcd33c79640 1 --2- 172.21.15.105:0/2627164967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd2c159e30 0x7fcd2c0cb650 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400c790 tx=0x7fcd2400cc50 comp rx=0 tx=0).stop 2023-11-28T18:29:15.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.594+0000 7fcd33c79640 1 -- 172.21.15.105:0/2627164967 shutdown_connections 2023-11-28T18:29:15.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.594+0000 7fcd33c79640 1 --2- 172.21.15.105:0/2627164967 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcd100668d0 0x7fcd10068d90 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:15.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.594+0000 7fcd33c79640 1 --2- 172.21.15.105:0/2627164967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd2c159e30 0x7fcd2c0cb650 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:15.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.594+0000 7fcd33c79640 1 --2- 172.21.15.105:0/2627164967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd2c0b2db0 0x7fcd2c0cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:15.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.594+0000 7fcd33c79640 1 --2- 172.21.15.105:0/2627164967 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcd2c0712c0 0x7fcd2c142670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:15.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.594+0000 7fcd33c79640 1 -- 172.21.15.105:0/2627164967 >> 172.21.15.105:0/2627164967 conn(0x7fcd2c06b860 msgr2=0x7fcd2c0b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:15.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.598+0000 7fcd33c79640 1 -- 172.21.15.105:0/2627164967 shutdown_connections 2023-11-28T18:29:15.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:15.598+0000 7fcd33c79640 1 -- 172.21.15.105:0/2627164967 wait complete. 2023-11-28T18:29:16.026 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:29:09.707308+0000', 'modified': '2023-11-28T18:29:12.751031+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7689}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7689': {'gid': 7689, 'name': 'a', 'rank': 0, 'incarnation': 83, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6837/3239728352', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 3239728352}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 3239728352}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:29:16.027 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T18:29:16.027 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T18:29:16.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.162+0000 7feae919d640 1 Processor -- start 2023-11-28T18:29:16.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.162+0000 7feae919d640 1 -- start start 2023-11-28T18:29:16.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.166+0000 7feae919d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feae4159f40 0x7feae40caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.166+0000 7feae919d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feae40cc760 0x7feae40ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.166+0000 7feae919d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae406bf50 con 0x7feae40ad680 2023-11-28T18:29:16.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.166+0000 7feae919d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae40ccbc0 con 0x7feae4159f40 2023-11-28T18:29:16.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.166+0000 7feae919d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae415a320 con 0x7feae40cc760 2023-11-28T18:29:16.170 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.166+0000 7feada59c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feae40cc760 0x7feae40ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.170 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.166+0000 7feae2d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feae4159f40 0x7feae40caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.170 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.166+0000 7feae359e640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7feae40ad680 0x7feae415da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:39422/0 (socket says 172.21.15.105:39422) 2023-11-28T18:29:16.170 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.166+0000 7feae359e640 1 -- 172.21.15.105:0/557050989 learned_addr learned my addr 172.21.15.105:0/557050989 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:16.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.166+0000 7feada59c640 1 -- 172.21.15.105:0/557050989 >> v2:172.21.15.148:3300/0 conn(0x7feae4159f40 msgr2=0x7feae40caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:16.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.166+0000 7feada59c640 1 --2- 172.21.15.105:0/557050989 >> v2:172.21.15.148:3300/0 conn(0x7feae4159f40 0x7feae40caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.166+0000 7feada59c640 1 -- 172.21.15.105:0/557050989 >> v1:172.21.15.148:6789/0 conn(0x7feae40ad680 legacy=0x7feae415da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:16.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.166+0000 7feada59c640 1 -- 172.21.15.105:0/557050989 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae4071380 con 0x7feae40cc760 2023-11-28T18:29:16.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.170+0000 7feada59c640 1 --2- 172.21.15.105:0/557050989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feae40cc760 0x7feae40ad140 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feacc009d90 tx=0x7feacc01eaa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8c3e32185dd8d14 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:16.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.170+0000 7feae259c640 1 -- 172.21.15.105:0/557050989 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc0a7b70 con 0x7feae40cc760 2023-11-28T18:29:16.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.170+0000 7feae259c640 1 -- 172.21.15.105:0/557050989 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feacc0a7cd0 con 0x7feae40cc760 2023-11-28T18:29:16.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.170+0000 7feae259c640 1 -- 172.21.15.105:0/557050989 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc0af660 con 0x7feae40cc760 2023-11-28T18:29:16.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.170+0000 7feae919d640 1 -- 172.21.15.105:0/557050989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feae40cc760 msgr2=0x7feae40ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:16.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.170+0000 7feae919d640 1 --2- 172.21.15.105:0/557050989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feae40cc760 0x7feae40ad140 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feacc009d90 tx=0x7feacc01eaa0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.170+0000 7feae919d640 1 -- 172.21.15.105:0/557050989 shutdown_connections 2023-11-28T18:29:16.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.170+0000 7feae919d640 1 --2- 172.21.15.105:0/557050989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feae40cc760 0x7feae40ad140 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.170+0000 7feae919d640 1 --2- 172.21.15.105:0/557050989 >> v2:172.21.15.148:3300/0 conn(0x7feae4159f40 0x7feae40caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.170+0000 7feae919d640 1 -- 172.21.15.105:0/557050989 >> 172.21.15.105:0/557050989 conn(0x7feae40ceb90 msgr2=0x7feae40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:16.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.170+0000 7feae919d640 1 -- 172.21.15.105:0/557050989 shutdown_connections 2023-11-28T18:29:16.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.170+0000 7feae919d640 1 -- 172.21.15.105:0/557050989 wait complete. 2023-11-28T18:29:16.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.170+0000 7feae919d640 1 Processor -- start 2023-11-28T18:29:16.175 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.170+0000 7feae919d640 1 -- start start 2023-11-28T18:29:16.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feae919d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feae40ad680 0x7feae4152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feae919d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feae40cc760 0x7feae4147960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feae919d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feae4159f40 0x7feae414c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feae919d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae41fa630 con 0x7feae4159f40 2023-11-28T18:29:16.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feae919d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae415b260 con 0x7feae40ad680 2023-11-28T18:29:16.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feae919d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae4147ea0 con 0x7feae40cc760 2023-11-28T18:29:16.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feae359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feae4159f40 0x7feae414c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feae359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feae4159f40 0x7feae414c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54070/0 (socket says 172.21.15.105:54070) 2023-11-28T18:29:16.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feae359e640 1 -- 172.21.15.105:0/3148970570 learned_addr learned my addr 172.21.15.105:0/3148970570 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:16.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feae2d9d640 1 --2- 172.21.15.105:0/3148970570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feae40ad680 0x7feae4152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feada59c640 1 --2- 172.21.15.105:0/3148970570 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feae40cc760 0x7feae4147960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feae359e640 1 -- 172.21.15.105:0/3148970570 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feae40cc760 msgr2=0x7feae4147960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:16.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feae359e640 1 --2- 172.21.15.105:0/3148970570 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feae40cc760 0x7feae4147960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feae359e640 1 -- 172.21.15.105:0/3148970570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feae40ad680 msgr2=0x7feae4152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:16.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feae359e640 1 --2- 172.21.15.105:0/3148970570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feae40ad680 0x7feae4152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feae359e640 1 -- 172.21.15.105:0/3148970570 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feacc009a70 con 0x7feae4159f40 2023-11-28T18:29:16.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feae359e640 1 --2- 172.21.15.105:0/3148970570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feae4159f40 0x7feae414c9f0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7feadc00c430 tx=0x7feadc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:16.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feadbfff640 1 -- 172.21.15.105:0/3148970570 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc005cf0 con 0x7feae4159f40 2023-11-28T18:29:16.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feadbfff640 1 -- 172.21.15.105:0/3148970570 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc005e50 con 0x7feae4159f40 2023-11-28T18:29:16.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feadbfff640 1 -- 172.21.15.105:0/3148970570 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc0072d0 con 0x7feae4159f40 2023-11-28T18:29:16.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feae919d640 1 -- 172.21.15.105:0/3148970570 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae41530e0 con 0x7feae4159f40 2023-11-28T18:29:16.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feae919d640 1 -- 172.21.15.105:0/3148970570 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae41536a0 con 0x7feae4159f40 2023-11-28T18:29:16.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.174+0000 7feadbfff640 1 -- 172.21.15.105:0/3148970570 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7feadc007430 con 0x7feae4159f40 2023-11-28T18:29:16.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feadbfff640 1 --2- 172.21.15.105:0/3148970570 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feac40668d0 0x7feac4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feadbfff640 1 -- 172.21.15.105:0/3148970570 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6145+0+0 (secure 0 0 0) 0x7feadc083ae0 con 0x7feae4159f40 2023-11-28T18:29:16.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feae2d9d640 1 --2- 172.21.15.105:0/3148970570 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feac40668d0 0x7feac4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feaec500000 1 Processor -- start 2023-11-28T18:29:16.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feaec500000 1 -- start start 2023-11-28T18:29:16.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feae2d9d640 1 --2- 172.21.15.105:0/3148970570 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feac40668d0 0x7feac4068d90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fead4009ac0 tx=0x7fead4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:16.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feaec500000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312accd30 0x562312acd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feaec500000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562312acd650 0x562312a7ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feaec500000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312a80500 0x562312a808e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feaec500000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x562312949110 con 0x562312a80500 2023-11-28T18:29:16.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feaec500000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623128d99a0 con 0x562312accd30 2023-11-28T18:29:16.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feaec500000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623128d6ed0 con 0x562312acd650 2023-11-28T18:29:16.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feae359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312a80500 0x562312a808e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feae359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312a80500 0x562312a808e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54082/0 (socket says 172.21.15.105:54082) 2023-11-28T18:29:16.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feae359e640 1 -- 172.21.15.105:0/4082654413 learned_addr learned my addr 172.21.15.105:0/4082654413 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:16.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feae2d9d640 1 --2- 172.21.15.105:0/4082654413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312accd30 0x562312acd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feada59c640 1 --2- 172.21.15.105:0/4082654413 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562312acd650 0x562312a7ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feae359e640 1 -- 172.21.15.105:0/4082654413 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562312acd650 msgr2=0x562312a7ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:16.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feae359e640 1 --2- 172.21.15.105:0/4082654413 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562312acd650 0x562312a7ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feae359e640 1 -- 172.21.15.105:0/4082654413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312accd30 msgr2=0x562312acd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:16.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feae359e640 1 --2- 172.21.15.105:0/4082654413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312accd30 0x562312acd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feae359e640 1 -- 172.21.15.105:0/4082654413 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623125ca890 con 0x562312a80500 2023-11-28T18:29:16.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7feae359e640 1 --2- 172.21.15.105:0/4082654413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312a80500 0x562312a808e0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7feadc05c550 tx=0x7feadc054830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1eeba8918b71ae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:16.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7fead8d99640 1 -- 172.21.15.105:0/4082654413 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc05ca80 con 0x562312a80500 2023-11-28T18:29:16.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7fead8d99640 1 -- 172.21.15.105:0/4082654413 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc052970 con 0x562312a80500 2023-11-28T18:29:16.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.178+0000 7fead8d99640 1 -- 172.21.15.105:0/4082654413 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc050f40 con 0x562312a80500 2023-11-28T18:29:16.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 1 -- 172.21.15.105:0/4082654413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312a80500 msgr2=0x562312a808e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:16.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 1 --2- 172.21.15.105:0/4082654413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312a80500 0x562312a808e0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7feadc05c550 tx=0x7feadc054830 comp rx=0 tx=0).stop 2023-11-28T18:29:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 1 -- 172.21.15.105:0/4082654413 shutdown_connections 2023-11-28T18:29:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 1 --2- 172.21.15.105:0/4082654413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312a80500 0x562312a808e0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 1 --2- 172.21.15.105:0/4082654413 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562312acd650 0x562312a7ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 1 --2- 172.21.15.105:0/4082654413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312accd30 0x562312acd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 1 -- 172.21.15.105:0/4082654413 >> 172.21.15.105:0/4082654413 conn(0x562312aad8c0 msgr2=0x562312aafcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 1 -- 172.21.15.105:0/4082654413 shutdown_connections 2023-11-28T18:29:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 1 -- 172.21.15.105:0/4082654413 wait complete. 2023-11-28T18:29:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 1 -- start start 2023-11-28T18:29:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 1 Processor -- start 2023-11-28T18:29:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312a80500 0x562312a86520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562312accd30 0x562312a86a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312acd650 0x562312a86fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623128d27c0 con 0x562312acd650 2023-11-28T18:29:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56231288abf0 con 0x562312a80500 2023-11-28T18:29:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x562312890a90 con 0x562312accd30 2023-11-28T18:29:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 10 client.0 _ll_register_callbacks cb 0x562312a6de20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:29:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feae359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312acd650 0x562312a86fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feae359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312acd650 0x562312a86fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54096/0 (socket says 172.21.15.105:54096) 2023-11-28T18:29:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feae359e640 1 -- 172.21.15.105:0/2364372360 learned_addr learned my addr 172.21.15.105:0/2364372360 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feada59c640 1 --2- 172.21.15.105:0/2364372360 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562312accd30 0x562312a86a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feae2d9d640 1 --2- 172.21.15.105:0/2364372360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312a80500 0x562312a86520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feae359e640 1 -- 172.21.15.105:0/2364372360 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562312accd30 msgr2=0x562312a86a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feae359e640 1 --2- 172.21.15.105:0/2364372360 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562312accd30 0x562312a86a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feae359e640 1 -- 172.21.15.105:0/2364372360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312a80500 msgr2=0x562312a86520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feae359e640 1 --2- 172.21.15.105:0/2364372360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312a80500 0x562312a86520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feae359e640 1 -- 172.21.15.105:0/2364372360 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae41536a0 con 0x562312acd650 2023-11-28T18:29:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feae2d9d640 1 --2- 172.21.15.105:0/2364372360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312a80500 0x562312a86520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feae359e640 1 --2- 172.21.15.105:0/2364372360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312acd650 0x562312a86fa0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7feadc054870 tx=0x7feadc01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9640c13511985dfc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feac1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:29:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feac1ffb640 1 -- 172.21.15.105:0/2364372360 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc052bd0 con 0x562312acd650 2023-11-28T18:29:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feac1ffb640 1 -- 172.21.15.105:0/2364372360 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 6145+0+0 (secure 0 0 0) 0x7feadc05c5f0 con 0x562312acd650 2023-11-28T18:29:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 1 -- 172.21.15.105:0/2364372360 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562312894470 con 0x562312acd650 2023-11-28T18:29:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feac1ffb640 1 -- 172.21.15.105:0/2364372360 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7feadc05c7b0 con 0x562312acd650 2023-11-28T18:29:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 10 client.7800 fetch_fsmap learned FSMap version 87 2023-11-28T18:29:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 1 -- 172.21.15.105:0/2364372360 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x562312893190 con 0x562312acd650 2023-11-28T18:29:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feac1ffb640 1 -- 172.21.15.105:0/2364372360 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562312894470 con 0x562312acd650 2023-11-28T18:29:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feac1ffb640 1 -- 172.21.15.105:0/2364372360 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 87) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feadc077050 con 0x562312acd650 2023-11-28T18:29:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 10 client.7800 fetch_fsmap finished waiting for FSMap version 87 2023-11-28T18:29:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 10 client.7800 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:29:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 20 client.7800 populate_metadata read hostname 'smithi105' 2023-11-28T18:29:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 1 --2- 172.21.15.105:0/2364372360 >> [v2:172.21.15.105:6836/3239728352,v1:172.21.15.105:6837/3239728352] conn(0x562312a879b0 0x562312adeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 4 client.7800 mds_command: new command op to 7689 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:29:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.182+0000 7feaec500000 1 -- 172.21.15.105:0/2364372360 --> [v2:172.21.15.105:6836/3239728352,v1:172.21.15.105:6837/3239728352] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562312897b50 con 0x562312a879b0 2023-11-28T18:29:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.186+0000 7feae2d9d640 1 --2- 172.21.15.105:0/2364372360 >> [v2:172.21.15.105:6836/3239728352,v1:172.21.15.105:6837/3239728352] conn(0x562312a879b0 0x562312adeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.186+0000 7feac1ffb640 10 client.7800 ms_handle_connect on v2:172.21.15.105:6836/3239728352 2023-11-28T18:29:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.186+0000 7feae2d9d640 1 --2- 172.21.15.105:0/2364372360 >> [v2:172.21.15.105:6836/3239728352,v1:172.21.15.105:6837/3239728352] conn(0x562312a879b0 0x562312adeff0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8505a3555638098f server_cookie=71bc6f4bcd6dcce1 in_seq=0 out_seq=0 2023-11-28T18:29:16.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.186+0000 7feac1ffb640 1 -- 172.21.15.105:0/2364372360 <== mds.0 v2:172.21.15.105:6836/3239728352 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562312897b50 con 0x562312a879b0 2023-11-28T18:29:16.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.186+0000 7feac1ffb640 10 client.7800 handle_command_reply: tid=0 2023-11-28T18:29:16.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.186+0000 7feaec500000 1 client.7800 shutdown 2023-11-28T18:29:16.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.186+0000 7feaec500000 1 -- 172.21.15.105:0/2364372360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312acd650 msgr2=0x562312a86fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:16.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.186+0000 7feaec500000 1 --2- 172.21.15.105:0/2364372360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312acd650 0x562312a86fa0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7feadc054870 tx=0x7feadc01e040 comp rx=0 tx=0).stop 2023-11-28T18:29:16.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.186+0000 7feaec500000 1 -- 172.21.15.105:0/2364372360 shutdown_connections 2023-11-28T18:29:16.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.186+0000 7feaec500000 1 --2- 172.21.15.105:0/2364372360 >> [v2:172.21.15.105:6836/3239728352,v1:172.21.15.105:6837/3239728352] conn(0x562312a879b0 0x562312adeff0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.186+0000 7feaec500000 1 --2- 172.21.15.105:0/2364372360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312acd650 0x562312a86fa0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.186+0000 7feaec500000 1 --2- 172.21.15.105:0/2364372360 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562312accd30 0x562312a86a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.186+0000 7feac1ffb640 0 client.7800 ms_handle_reset on v2:172.21.15.105:6836/3239728352 2023-11-28T18:29:16.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.186+0000 7feaec500000 1 --2- 172.21.15.105:0/2364372360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312a80500 0x562312a86520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.186+0000 7feaec500000 1 -- 172.21.15.105:0/2364372360 >> 172.21.15.105:0/2364372360 conn(0x562312aad8c0 msgr2=0x562312a7f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:16.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.186+0000 7feaec500000 1 -- 172.21.15.105:0/2364372360 shutdown_connections 2023-11-28T18:29:16.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.186+0000 7feaec500000 1 -- 172.21.15.105:0/2364372360 wait complete. 2023-11-28T18:29:16.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.186+0000 7feaec500000 20 client.7800 trim_cache size 0 max 16384 2023-11-28T18:29:16.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feaec500000 1 Processor -- start 2023-11-28T18:29:16.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feaec500000 1 -- start start 2023-11-28T18:29:16.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feaec500000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312aad8c0 0x562312aac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feaec500000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562312a80500 0x562312a7e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feaec500000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312accd30 0x562312a7ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feaec500000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x56231258b940 con 0x562312accd30 2023-11-28T18:29:16.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feaec500000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623128d67c0 con 0x562312aad8c0 2023-11-28T18:29:16.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feaec500000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x56231289b910 con 0x562312a80500 2023-11-28T18:29:16.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feae359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312accd30 0x562312a7ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feae359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312accd30 0x562312a7ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54108/0 (socket says 172.21.15.105:54108) 2023-11-28T18:29:16.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feae359e640 1 -- 172.21.15.105:0/3696936427 learned_addr learned my addr 172.21.15.105:0/3696936427 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:16.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feada59c640 1 --2- 172.21.15.105:0/3696936427 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562312a80500 0x562312a7e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feae2d9d640 1 --2- 172.21.15.105:0/3696936427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312aad8c0 0x562312aac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feae359e640 1 -- 172.21.15.105:0/3696936427 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562312a80500 msgr2=0x562312a7e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:16.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feae359e640 1 --2- 172.21.15.105:0/3696936427 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562312a80500 0x562312a7e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feae359e640 1 -- 172.21.15.105:0/3696936427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312aad8c0 msgr2=0x562312aac9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:16.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feae359e640 1 --2- 172.21.15.105:0/3696936427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312aad8c0 0x562312aac9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feae359e640 1 -- 172.21.15.105:0/3696936427 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562312879490 con 0x562312accd30 2023-11-28T18:29:16.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feae2d9d640 1 --2- 172.21.15.105:0/3696936427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312aad8c0 0x562312aac9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:16.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feae359e640 1 --2- 172.21.15.105:0/3696936427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312accd30 0x562312a7ec60 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7feadc0507e0 tx=0x7feadc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53958ff331e69339 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:16.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7fead959a640 1 -- 172.21.15.105:0/3696936427 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc050620 con 0x562312accd30 2023-11-28T18:29:16.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7fead959a640 1 -- 172.21.15.105:0/3696936427 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc07f070 con 0x562312accd30 2023-11-28T18:29:16.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7fead959a640 1 -- 172.21.15.105:0/3696936427 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc00e040 con 0x562312accd30 2023-11-28T18:29:16.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feaec500000 1 -- 172.21.15.105:0/3696936427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312accd30 msgr2=0x562312a7ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:16.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feaec500000 1 --2- 172.21.15.105:0/3696936427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312accd30 0x562312a7ec60 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7feadc0507e0 tx=0x7feadc017000 comp rx=0 tx=0).stop 2023-11-28T18:29:16.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feaec500000 1 -- 172.21.15.105:0/3696936427 shutdown_connections 2023-11-28T18:29:16.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feaec500000 1 --2- 172.21.15.105:0/3696936427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312accd30 0x562312a7ec60 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feaec500000 1 --2- 172.21.15.105:0/3696936427 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562312a80500 0x562312a7e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feaec500000 1 --2- 172.21.15.105:0/3696936427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312aad8c0 0x562312aac9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feaec500000 1 -- 172.21.15.105:0/3696936427 >> 172.21.15.105:0/3696936427 conn(0x562312a88100 msgr2=0x562312a7f670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:16.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feaec500000 1 -- 172.21.15.105:0/3696936427 shutdown_connections 2023-11-28T18:29:16.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.202+0000 7feaec500000 1 -- 172.21.15.105:0/3696936427 wait complete. 2023-11-28T18:29:16.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feaec500000 1 -- start start 2023-11-28T18:29:16.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feaec500000 1 Processor -- start 2023-11-28T18:29:16.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feaec500000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312a80500 0x562312b2d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feaec500000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562312aad8c0 0x562312b2db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feaec500000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312accd30 0x562312ae8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feaec500000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x562312890a90 con 0x562312a80500 2023-11-28T18:29:16.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feaec500000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56231289b910 con 0x562312accd30 2023-11-28T18:29:16.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feaec500000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623128a7b70 con 0x562312aad8c0 2023-11-28T18:29:16.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feaec500000 10 client.0 _ll_register_callbacks cb 0x562312949060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:29:16.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feae2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312a80500 0x562312b2d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feae2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312a80500 0x562312b2d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54114/0 (socket says 172.21.15.105:54114) 2023-11-28T18:29:16.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feae2d9d640 1 -- 172.21.15.105:0/3667899405 learned_addr learned my addr 172.21.15.105:0/3667899405 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:16.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feae359e640 1 --2- 172.21.15.105:0/3667899405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312accd30 0x562312ae8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feada59c640 1 --2- 172.21.15.105:0/3667899405 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562312aad8c0 0x562312b2db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feae2d9d640 1 -- 172.21.15.105:0/3667899405 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562312aad8c0 msgr2=0x562312b2db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:16.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feae2d9d640 1 --2- 172.21.15.105:0/3667899405 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562312aad8c0 0x562312b2db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feae2d9d640 1 -- 172.21.15.105:0/3667899405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312accd30 msgr2=0x562312ae8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:16.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feae2d9d640 1 --2- 172.21.15.105:0/3667899405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312accd30 0x562312ae8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feae2d9d640 1 -- 172.21.15.105:0/3667899405 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562312893190 con 0x562312a80500 2023-11-28T18:29:16.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feae359e640 1 --2- 172.21.15.105:0/3667899405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312accd30 0x562312ae8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:16.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feada59c640 1 --2- 172.21.15.105:0/3667899405 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562312aad8c0 0x562312b2db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:16.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feae2d9d640 1 --2- 172.21.15.105:0/3667899405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312a80500 0x562312b2d600 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fead4009540 tx=0x7fead4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b4557bebe19206 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:16.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feac1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:29:16.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feac1ffb640 1 -- 172.21.15.105:0/3667899405 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead401c040 con 0x562312a80500 2023-11-28T18:29:16.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feaec500000 1 -- 172.21.15.105:0/3667899405 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623128d1110 con 0x562312a80500 2023-11-28T18:29:16.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feac1ffb640 1 -- 172.21.15.105:0/3667899405 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fead402ab00 con 0x562312a80500 2023-11-28T18:29:16.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feac1ffb640 1 -- 172.21.15.105:0/3667899405 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fead402ad00 con 0x562312a80500 2023-11-28T18:29:16.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feaec500000 10 client.7806 fetch_fsmap learned FSMap version 87 2023-11-28T18:29:16.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feaec500000 1 -- 172.21.15.105:0/3667899405 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x5623125cacc0 con 0x562312a80500 2023-11-28T18:29:16.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feac1ffb640 1 -- 172.21.15.105:0/3667899405 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5623128d1110 con 0x562312a80500 2023-11-28T18:29:16.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feac1ffb640 1 -- 172.21.15.105:0/3667899405 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 87) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fead4030020 con 0x562312a80500 2023-11-28T18:29:16.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feaec500000 10 client.7806 fetch_fsmap finished waiting for FSMap version 87 2023-11-28T18:29:16.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feaec500000 10 client.7806 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:29:16.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feaec500000 20 client.7806 populate_metadata read hostname 'smithi105' 2023-11-28T18:29:16.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feaec500000 1 --2- 172.21.15.105:0/3667899405 >> [v2:172.21.15.105:6836/3239728352,v1:172.21.15.105:6837/3239728352] conn(0x562312acd650 0x562312b36470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feaec500000 4 client.7806 mds_command: new command op to 7689 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:29:16.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feaec500000 1 -- 172.21.15.105:0/3667899405 --> [v2:172.21.15.105:6836/3239728352,v1:172.21.15.105:6837/3239728352] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5623128ff650 con 0x562312acd650 2023-11-28T18:29:16.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.206+0000 7feada59c640 1 --2- 172.21.15.105:0/3667899405 >> [v2:172.21.15.105:6836/3239728352,v1:172.21.15.105:6837/3239728352] conn(0x562312acd650 0x562312b36470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feac1ffb640 10 client.7806 ms_handle_connect on v2:172.21.15.105:6836/3239728352 2023-11-28T18:29:16.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feada59c640 1 --2- 172.21.15.105:0/3667899405 >> [v2:172.21.15.105:6836/3239728352,v1:172.21.15.105:6837/3239728352] conn(0x562312acd650 0x562312b36470 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=404d19a20a6aa4a server_cookie=1a9120027154e520 in_seq=0 out_seq=0 2023-11-28T18:29:16.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feac1ffb640 1 -- 172.21.15.105:0/3667899405 <== mds.0 v2:172.21.15.105:6836/3239728352 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5623128ff650 con 0x562312acd650 2023-11-28T18:29:16.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feac1ffb640 10 client.7806 handle_command_reply: tid=0 2023-11-28T18:29:16.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feaec500000 1 client.7806 shutdown 2023-11-28T18:29:16.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feaec500000 1 -- 172.21.15.105:0/3667899405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312a80500 msgr2=0x562312b2d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:16.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feaec500000 1 --2- 172.21.15.105:0/3667899405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312a80500 0x562312b2d600 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fead4009540 tx=0x7fead4004740 comp rx=0 tx=0).stop 2023-11-28T18:29:16.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feaec500000 1 -- 172.21.15.105:0/3667899405 shutdown_connections 2023-11-28T18:29:16.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feaec500000 1 --2- 172.21.15.105:0/3667899405 >> [v2:172.21.15.105:6836/3239728352,v1:172.21.15.105:6837/3239728352] conn(0x562312acd650 0x562312b36470 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feaec500000 1 --2- 172.21.15.105:0/3667899405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562312accd30 0x562312ae8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feaec500000 1 --2- 172.21.15.105:0/3667899405 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562312aad8c0 0x562312b2db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feac1ffb640 0 client.7806 ms_handle_reset on v2:172.21.15.105:6836/3239728352 2023-11-28T18:29:16.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feaec500000 1 --2- 172.21.15.105:0/3667899405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562312a80500 0x562312b2d600 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feaec500000 1 -- 172.21.15.105:0/3667899405 >> 172.21.15.105:0/3667899405 conn(0x562312a88100 msgr2=0x562312a7e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:16.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feaec500000 1 -- 172.21.15.105:0/3667899405 shutdown_connections 2023-11-28T18:29:16.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feaec500000 1 -- 172.21.15.105:0/3667899405 wait complete. 2023-11-28T18:29:16.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feaec500000 20 client.7806 trim_cache size 0 max 16384 2023-11-28T18:29:16.220 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:29:16.220 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:29:16.220 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2023-11-28T18:29:16.220 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 7689, 2023-11-28T18:29:16.220 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:29:16.220 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:29:16.220 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:29:16.220 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 5.8709108719999996, 2023-11-28T18:29:16.220 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 87, 2023-11-28T18:29:16.220 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 91, 2023-11-28T18:29:16.220 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 90, 2023-11-28T18:29:16.221 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 11.799424224999999 2023-11-28T18:29:16.221 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:29:16.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feae919d640 1 -- 172.21.15.105:0/3148970570 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feac40668d0 msgr2=0x7feac4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:16.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feae919d640 1 --2- 172.21.15.105:0/3148970570 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feac40668d0 0x7feac4068d90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fead4009ac0 tx=0x7fead4009190 comp rx=0 tx=0).stop 2023-11-28T18:29:16.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feae919d640 1 -- 172.21.15.105:0/3148970570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feae4159f40 msgr2=0x7feae414c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:16.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feae919d640 1 --2- 172.21.15.105:0/3148970570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feae4159f40 0x7feae414c9f0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7feadc00c430 tx=0x7feadc00c8f0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feae919d640 1 -- 172.21.15.105:0/3148970570 shutdown_connections 2023-11-28T18:29:16.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feae919d640 1 --2- 172.21.15.105:0/3148970570 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feac40668d0 0x7feac4068d90 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feae919d640 1 --2- 172.21.15.105:0/3148970570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feae4159f40 0x7feae414c9f0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feae919d640 1 --2- 172.21.15.105:0/3148970570 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feae40cc760 0x7feae4147960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feae919d640 1 --2- 172.21.15.105:0/3148970570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feae40ad680 0x7feae4152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feae919d640 1 -- 172.21.15.105:0/3148970570 >> 172.21.15.105:0/3148970570 conn(0x7feae40ceb90 msgr2=0x7feae4071aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:16.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feae919d640 1 -- 172.21.15.105:0/3148970570 shutdown_connections 2023-11-28T18:29:16.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.210+0000 7feae919d640 1 -- 172.21.15.105:0/3148970570 wait complete. 2023-11-28T18:29:16.232 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T18:29:16.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.366+0000 7fc28859d640 1 Processor -- start 2023-11-28T18:29:16.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.366+0000 7fc28859d640 1 -- start start 2023-11-28T18:29:16.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.366+0000 7fc28859d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc280071a10 0x7fc2800ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.366+0000 7fc28859d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc280070a10 0x7fc280070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.366+0000 7fc28859d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2800713a0 con 0x7fc28015d3c0 2023-11-28T18:29:16.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.366+0000 7fc28859d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2800cefa0 con 0x7fc280071a10 2023-11-28T18:29:16.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.366+0000 7fc28859d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc28015d7a0 con 0x7fc280070a10 2023-11-28T18:29:16.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.366+0000 7fc286b3a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc280070a10 0x7fc280070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.366+0000 7fc285b38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc280071a10 0x7fc2800ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.366+0000 7fc286339640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc28015d3c0 0x7fc280161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:39436/0 (socket says 172.21.15.105:39436) 2023-11-28T18:29:16.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.366+0000 7fc286339640 1 -- 172.21.15.105:0/3282659475 learned_addr learned my addr 172.21.15.105:0/3282659475 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:16.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.370+0000 7fc286b3a640 1 -- 172.21.15.105:0/3282659475 >> v2:172.21.15.148:3300/0 conn(0x7fc280071a10 msgr2=0x7fc2800ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:16.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.370+0000 7fc286b3a640 1 --2- 172.21.15.105:0/3282659475 >> v2:172.21.15.148:3300/0 conn(0x7fc280071a10 0x7fc2800ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.370+0000 7fc286b3a640 1 -- 172.21.15.105:0/3282659475 >> v1:172.21.15.148:6789/0 conn(0x7fc28015d3c0 legacy=0x7fc280161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:16.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.370+0000 7fc286b3a640 1 -- 172.21.15.105:0/3282659475 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2800ce9e0 con 0x7fc280070a10 2023-11-28T18:29:16.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.370+0000 7fc285b38640 1 --2- 172.21.15.105:0/3282659475 >> v2:172.21.15.148:3300/0 conn(0x7fc280071a10 0x7fc2800ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:29:16.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.370+0000 7fc286b3a640 1 --2- 172.21.15.105:0/3282659475 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc280070a10 0x7fc280070e60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc278009d90 tx=0x7fc27801c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfb777df6d636aa9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:16.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.370+0000 7fc285337640 1 -- 172.21.15.105:0/3282659475 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2780a5860 con 0x7fc280070a10 2023-11-28T18:29:16.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.370+0000 7fc285337640 1 -- 172.21.15.105:0/3282659475 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2780a59c0 con 0x7fc280070a10 2023-11-28T18:29:16.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.370+0000 7fc285337640 1 -- 172.21.15.105:0/3282659475 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2780a5d50 con 0x7fc280070a10 2023-11-28T18:29:16.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.370+0000 7fc28859d640 1 -- 172.21.15.105:0/3282659475 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc280070a10 msgr2=0x7fc280070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:16.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.370+0000 7fc28859d640 1 --2- 172.21.15.105:0/3282659475 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc280070a10 0x7fc280070e60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc278009d90 tx=0x7fc27801c890 comp rx=0 tx=0).stop 2023-11-28T18:29:16.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc28859d640 1 -- 172.21.15.105:0/3282659475 shutdown_connections 2023-11-28T18:29:16.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc28859d640 1 --2- 172.21.15.105:0/3282659475 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc280070a10 0x7fc280070e60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc28859d640 1 --2- 172.21.15.105:0/3282659475 >> v2:172.21.15.148:3300/0 conn(0x7fc280071a10 0x7fc2800ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc28859d640 1 -- 172.21.15.105:0/3282659475 >> 172.21.15.105:0/3282659475 conn(0x7fc2800cf540 msgr2=0x7fc2800cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:16.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc28859d640 1 -- 172.21.15.105:0/3282659475 shutdown_connections 2023-11-28T18:29:16.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc28859d640 1 -- 172.21.15.105:0/3282659475 wait complete. 2023-11-28T18:29:16.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc28859d640 1 Processor -- start 2023-11-28T18:29:16.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc28859d640 1 -- start start 2023-11-28T18:29:16.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc28859d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc280070a10 0x7fc280160360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc28859d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc280071a10 0x7fc28015f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc28859d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc28015d3c0 0x7fc280146fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc28859d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28015fe80 con 0x7fc280070a10 2023-11-28T18:29:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc28859d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28015edf0 con 0x7fc280071a10 2023-11-28T18:29:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc28859d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc28015f280 con 0x7fc28015d3c0 2023-11-28T18:29:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc286339640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc280070a10 0x7fc280160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc286339640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc280070a10 0x7fc280160360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54118/0 (socket says 172.21.15.105:54118) 2023-11-28T18:29:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc286339640 1 -- 172.21.15.105:0/412664 learned_addr learned my addr 172.21.15.105:0/412664 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc286b3a640 1 --2- 172.21.15.105:0/412664 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc28015d3c0 0x7fc280146fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc285b38640 1 --2- 172.21.15.105:0/412664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc280071a10 0x7fc28015f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc286339640 1 -- 172.21.15.105:0/412664 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc28015d3c0 msgr2=0x7fc280146fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc286339640 1 --2- 172.21.15.105:0/412664 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc28015d3c0 0x7fc280146fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc286339640 1 -- 172.21.15.105:0/412664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc280071a10 msgr2=0x7fc28015f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc286339640 1 --2- 172.21.15.105:0/412664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc280071a10 0x7fc28015f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc286339640 1 -- 172.21.15.105:0/412664 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc270009420 con 0x7fc280070a10 2023-11-28T18:29:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.374+0000 7fc286339640 1 --2- 172.21.15.105:0/412664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc280070a10 0x7fc280160360 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00c400 tx=0x7fc27c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.378+0000 7fc276ffd640 1 -- 172.21.15.105:0/412664 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c005cf0 con 0x7fc280070a10 2023-11-28T18:29:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.378+0000 7fc276ffd640 1 -- 172.21.15.105:0/412664 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27c005e50 con 0x7fc280070a10 2023-11-28T18:29:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.378+0000 7fc276ffd640 1 -- 172.21.15.105:0/412664 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c0072d0 con 0x7fc280070a10 2023-11-28T18:29:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.378+0000 7fc28859d640 1 -- 172.21.15.105:0/412664 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc278009a70 con 0x7fc280070a10 2023-11-28T18:29:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.378+0000 7fc28859d640 1 -- 172.21.15.105:0/412664 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc28015e4c0 con 0x7fc280070a10 2023-11-28T18:29:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.378+0000 7fc28859d640 1 -- 172.21.15.105:0/412664 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc280067570 con 0x7fc280070a10 2023-11-28T18:29:16.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.378+0000 7fc276ffd640 1 -- 172.21.15.105:0/412664 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc27c007430 con 0x7fc280070a10 2023-11-28T18:29:16.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.378+0000 7fc276ffd640 1 --2- 172.21.15.105:0/412664 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc2640668d0 0x7fc264068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:16.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.378+0000 7fc276ffd640 1 -- 172.21.15.105:0/412664 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fc27c083960 con 0x7fc280070a10 2023-11-28T18:29:16.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.378+0000 7fc285b38640 1 --2- 172.21.15.105:0/412664 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc2640668d0 0x7fc264068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:16.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.378+0000 7fc285b38640 1 --2- 172.21.15.105:0/412664 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc2640668d0 0x7fc264068d90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc280071290 tx=0x7fc270009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:16.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.382+0000 7fc276ffd640 1 -- 172.21.15.105:0/412664 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc27c0509f0 con 0x7fc280070a10 2023-11-28T18:29:16.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.574+0000 7fc28859d640 1 -- 172.21.15.105:0/412664 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc2800713a0 con 0x7fc280070a10 2023-11-28T18:29:16.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.574+0000 7fc276ffd640 1 -- 172.21.15.105:0/412664 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v87) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc2800713a0 con 0x7fc280070a10 2023-11-28T18:29:16.579 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T18:29:16.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.578+0000 7fc28859d640 1 -- 172.21.15.105:0/412664 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc2640668d0 msgr2=0x7fc264068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:16.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.578+0000 7fc28859d640 1 --2- 172.21.15.105:0/412664 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc2640668d0 0x7fc264068d90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc280071290 tx=0x7fc270009190 comp rx=0 tx=0).stop 2023-11-28T18:29:16.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.578+0000 7fc28859d640 1 -- 172.21.15.105:0/412664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc280070a10 msgr2=0x7fc280160360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:16.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.578+0000 7fc28859d640 1 --2- 172.21.15.105:0/412664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc280070a10 0x7fc280160360 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00c400 tx=0x7fc27c00c8c0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.578+0000 7fc28859d640 1 -- 172.21.15.105:0/412664 shutdown_connections 2023-11-28T18:29:16.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.578+0000 7fc28859d640 1 --2- 172.21.15.105:0/412664 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc28015d3c0 0x7fc280146fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.578+0000 7fc28859d640 1 --2- 172.21.15.105:0/412664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc280071a10 0x7fc28015f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.578+0000 7fc28859d640 1 --2- 172.21.15.105:0/412664 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc2640668d0 0x7fc264068d90 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.578+0000 7fc28859d640 1 --2- 172.21.15.105:0/412664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc280070a10 0x7fc280160360 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:16.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.578+0000 7fc28859d640 1 -- 172.21.15.105:0/412664 >> 172.21.15.105:0/412664 conn(0x7fc2800cf540 msgr2=0x7fc2800cf920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:16.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.578+0000 7fc28859d640 1 -- 172.21.15.105:0/412664 shutdown_connections 2023-11-28T18:29:16.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:16.578+0000 7fc28859d640 1 -- 172.21.15.105:0/412664 wait complete. 2023-11-28T18:29:16.601 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T18:29:16.601 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T18:29:16.601 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2023-11-28T18:29:16.601 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T18:29:16.601 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T18:29:16.601 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T18:29:16.601 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T18:29:16.601 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T18:29:16.601 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:29:16.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:16.602 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2023-11-28T18:29:16.614 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T18:29:16.614 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T18:29:16.614 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T18:29:16.614 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:29:16.614 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-11-28T18:29:16.614 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:29:16.614 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:29:16.614 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:29:16.615 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2023-11-28T18:29:16.615 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:29:16.615 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:29:16.615 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2023-11-28T18:29:16.615 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:29:16.615 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:29:16.615 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T18:29:16.615 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:29:16.615 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-28T18:29:16.615 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42065sec preferred_lft 42065sec 2023-11-28T18:29:16.615 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2023-11-28T18:29:16.615 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:29:16.616 INFO:teuthology.orchestra.run.smithi105.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-28T18:29:16.616 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 02:42:cc:d4:87:d2 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:29:16.616 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-28T18:29:16.616 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:29:16.616 INFO:teuthology.orchestra.run.smithi105.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T18:29:16.616 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-28T18:29:16.616 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T18:29:16.616 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:29:16.616 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-28T18:29:16.616 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:29:16.616 INFO:teuthology.orchestra.run.smithi105.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T18:29:16.616 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:29:16.616 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-28T18:29:16.617 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:29:16.617 INFO:teuthology.orchestra.run.smithi105.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T18:29:16.617 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:29:16.617 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-28T18:29:16.617 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:29:16.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:16.617 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T18:29:16.674 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-11-28T18:29:16.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:16.676 DEBUG:teuthology.orchestra.run.smithi105:> route 2023-11-28T18:29:16.731 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2023-11-28T18:29:16.731 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T18:29:16.731 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-28T18:29:16.731 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T18:29:16.732 INFO:teuthology.orchestra.run.smithi105.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-28T18:29:16.732 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T18:29:16.732 INFO:teuthology.orchestra.run.smithi105.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:29:16.732 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:29:16.732 INFO:teuthology.orchestra.run.smithi105.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:29:16.732 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T18:29:16.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:16.732 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-11-28T18:29:16.732 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:29:16.733 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T18:29:16.733 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T18:29:16.733 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T18:29:16.733 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:29:16.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:16.826 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-11-28T18:29:16.836 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T18:29:16.836 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T18:29:16.836 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:29:16.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:16.837 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2023-11-28T18:29:16.901 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-28T18:29:16.901 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:29:16.901 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T18:29:16.957 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:29:16.958 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:29:16.958 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T18:29:17.008 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:29:17.072 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:29:17.129 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:29:17.130 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:29:17.130 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:29:17.130 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:29:17.130 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:29:17.130 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:29:17.130 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:29:17.130 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:29:17.130 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:29:17.130 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:29:17.130 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:29:17.131 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:29:17.131 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:29:17.131 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:29:17.131 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:29:17.131 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:29:17.131 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:29:17.131 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:29:17.131 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:29:17.131 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:29:17.131 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:29:17.131 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:29:17.131 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:29:17.132 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:29:17.132 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:29:17.132 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:29:17.132 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:29:17.132 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:29:17.132 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:29:17.132 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:29:17.132 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:29:17.132 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:29:17.132 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:29:17.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:17.134 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:29:17.184 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T18:29:17.184 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-28T18:29:17.235 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:29:17.256 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:29:17.309 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:29:17.310 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:29:17.310 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:29:17.310 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:29:17.310 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:29:17.310 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:29:17.310 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:29:17.310 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:29:17.310 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:29:17.310 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:29:17.310 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:29:17.311 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:29:17.311 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:29:17.311 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:29:17.311 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:29:17.311 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:29:17.311 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:29:17.311 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:29:17.311 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:29:17.311 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:29:17.311 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:29:17.311 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:29:17.311 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:29:17.311 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:29:17.312 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:29:17.312 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:29:17.312 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:29:17.312 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:29:17.312 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:29:17.312 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:29:17.312 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:29:17.312 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:29:17.312 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:29:17.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:17.313 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:29:17.330 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:2023-11-28T18:29:17.325+0000 7f3f355a0580 -1 init, newargv = 0x55771190cdf0 newargc=15 2023-11-28T18:29:17.331 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[60998]: starting ceph client 2023-11-28T18:29:17.337 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[60998]: starting fuse 2023-11-28T18:29:17.340 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-11-28T18:29:17.341 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-28T18:29:17.341 DEBUG:teuthology.orchestra.run.smithi105:> sudo stdin-killer -- python3 -c ' 2023-11-28T18:29:17.341 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2023-11-28T18:29:17.341 DEBUG:teuthology.orchestra.run.smithi105:> import re 2023-11-28T18:29:17.341 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-11-28T18:29:17.341 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2023-11-28T18:29:17.341 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:29:17.341 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2023-11-28T18:29:17.341 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-28T18:29:17.342 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2023-11-28T18:29:17.342 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-28T18:29:17.342 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:29:17.342 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2023-11-28T18:29:17.342 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2023-11-28T18:29:17.342 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2023-11-28T18:29:17.342 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2023-11-28T18:29:17.342 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:29:17.342 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2023-11-28T18:29:17.342 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T18:29:17.342 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T18:29:17.343 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T18:29:17.343 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2023-11-28T18:29:17.343 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2023-11-28T18:29:17.343 DEBUG:teuthology.orchestra.run.smithi105:> return f 2023-11-28T18:29:17.343 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T18:29:17.343 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:29:17.343 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.0")) 2023-11-28T18:29:17.343 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-11-28T18:29:17.465 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.0.60998.asok 2023-11-28T18:29:17.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-28T18:29:17.477 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.60998.asok 2023-11-28T18:29:17.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:17.477 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.60998.asok status 2023-11-28T18:29:17.648 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:29:17.649 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2023-11-28T18:29:17.649 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "47c1c547372ed5bff7984647dc200bac0e019caf", 2023-11-28T18:29:17.649 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev)", 2023-11-28T18:29:17.649 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "0", 2023-11-28T18:29:17.649 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2023-11-28T18:29:17.649 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-28T18:29:17.649 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "60998", 2023-11-28T18:29:17.650 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2023-11-28T18:29:17.650 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:29:17.650 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2023-11-28T18:29:17.650 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2023-11-28T18:29:17.650 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 7818, 2023-11-28T18:29:17.650 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2023-11-28T18:29:17.650 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2023-11-28T18:29:17.650 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2023-11-28T18:29:17.650 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 7818 2023-11-28T18:29:17.650 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:29:17.651 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-11-28T18:29:17.651 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-11-28T18:29:17.651 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-11-28T18:29:17.651 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 1935327954 2023-11-28T18:29:17.651 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:29:17.651 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:29:17.651 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-11-28T18:29:17.651 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-11-28T18:29:17.651 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-11-28T18:29:17.651 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 1935327954 2023-11-28T18:29:17.652 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:29:17.652 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.7818 192.168.0.1:0/1935327954", 2023-11-28T18:29:17.652 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/1935327954", 2023-11-28T18:29:17.652 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2023-11-28T18:29:17.652 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 87, 2023-11-28T18:29:17.652 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 92, 2023-11-28T18:29:17.652 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2023-11-28T18:29:17.652 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2023-11-28T18:29:17.652 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2023-11-28T18:29:17.653 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:29:17.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:17.664 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2023-11-28T18:29:17.664 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-28T18:29:17.674 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2023-11-28T18:29:17.674 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-28T18:29:17.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:17.675 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T18:29:17.736 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random True 2023-11-28T18:29:17.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.913+0000 7fd81018d640 1 Processor -- start 2023-11-28T18:29:17.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.913+0000 7fd81018d640 1 -- start start 2023-11-28T18:29:17.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.913+0000 7fd81018d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd80806bf50 0x7fd80815d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:17.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.913+0000 7fd81018d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd8080adb80 0x7fd8080adf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:17.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.913+0000 7fd81018d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8080cc7f0 con 0x7fd808154070 2023-11-28T18:29:17.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.913+0000 7fd81018d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd808072540 con 0x7fd80806bf50 2023-11-28T18:29:17.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.913+0000 7fd81018d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8080ae4d0 con 0x7fd8080adb80 2023-11-28T18:29:17.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.913+0000 7fd80d728640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd80806bf50 0x7fd80815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:17.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.913+0000 7fd80d728640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd80806bf50 0x7fd80815d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54126/0 (socket says 172.21.15.105:54126) 2023-11-28T18:29:17.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.913+0000 7fd80d728640 1 -- 172.21.15.105:0/2624145376 learned_addr learned my addr 172.21.15.105:0/2624145376 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:17.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.913+0000 7fd80e72a640 1 --2- 172.21.15.105:0/2624145376 >> v2:172.21.15.148:3300/0 conn(0x7fd8080adb80 0x7fd8080adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:17.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.917+0000 7fd80d728640 1 -- 172.21.15.105:0/2624145376 >> v2:172.21.15.148:3300/0 conn(0x7fd8080adb80 msgr2=0x7fd8080adf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:17.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.917+0000 7fd80d728640 1 --2- 172.21.15.105:0/2624145376 >> v2:172.21.15.148:3300/0 conn(0x7fd8080adb80 0x7fd8080adf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:17.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.917+0000 7fd80d728640 1 -- 172.21.15.105:0/2624145376 >> v1:172.21.15.148:6789/0 conn(0x7fd808154070 legacy=0x7fd8080ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:17.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.917+0000 7fd80d728640 1 -- 172.21.15.105:0/2624145376 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80815dd10 con 0x7fd80806bf50 2023-11-28T18:29:17.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.917+0000 7fd80d728640 1 --2- 172.21.15.105:0/2624145376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd80806bf50 0x7fd80815d700 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd808067c50 tx=0x7fd7f801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ae265cd9fb1f45 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:17.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.917+0000 7fd80cf27640 1 -- 172.21.15.105:0/2624145376 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80ab600 con 0x7fd80806bf50 2023-11-28T18:29:17.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.917+0000 7fd80cf27640 1 -- 172.21.15.105:0/2624145376 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f80a7050 con 0x7fd80806bf50 2023-11-28T18:29:17.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.917+0000 7fd80cf27640 1 -- 172.21.15.105:0/2624145376 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80ab980 con 0x7fd80806bf50 2023-11-28T18:29:17.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.917+0000 7fd81018d640 1 -- 172.21.15.105:0/2624145376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd80806bf50 msgr2=0x7fd80815d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:17.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.917+0000 7fd81018d640 1 --2- 172.21.15.105:0/2624145376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd80806bf50 0x7fd80815d700 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd808067c50 tx=0x7fd7f801c4d0 comp rx=0 tx=0).stop 2023-11-28T18:29:17.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.917+0000 7fd81018d640 1 -- 172.21.15.105:0/2624145376 shutdown_connections 2023-11-28T18:29:17.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.917+0000 7fd81018d640 1 --2- 172.21.15.105:0/2624145376 >> v2:172.21.15.148:3300/0 conn(0x7fd8080adb80 0x7fd8080adf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:17.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.917+0000 7fd81018d640 1 --2- 172.21.15.105:0/2624145376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd80806bf50 0x7fd80815d700 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:17.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.917+0000 7fd81018d640 1 -- 172.21.15.105:0/2624145376 >> 172.21.15.105:0/2624145376 conn(0x7fd8080ceb90 msgr2=0x7fd8080cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:17.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd81018d640 1 -- 172.21.15.105:0/2624145376 shutdown_connections 2023-11-28T18:29:17.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd81018d640 1 -- 172.21.15.105:0/2624145376 wait complete. 2023-11-28T18:29:17.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd81018d640 1 Processor -- start 2023-11-28T18:29:17.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd81018d640 1 -- start start 2023-11-28T18:29:17.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd81018d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd80806bf50 0x7fd8080abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:17.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd81018d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8080adb80 0x7fd80814e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:17.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd81018d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd808154070 0x7fd808147000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:17.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd81018d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd808150090 con 0x7fd8080adb80 2023-11-28T18:29:17.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd81018d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80814f280 con 0x7fd808154070 2023-11-28T18:29:17.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd81018d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd80814f560 con 0x7fd80806bf50 2023-11-28T18:29:17.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd80d728640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8080adb80 0x7fd80814e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd80d728640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8080adb80 0x7fd80814e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54140/0 (socket says 172.21.15.105:54140) 2023-11-28T18:29:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd80d728640 1 -- 172.21.15.105:0/736231796 learned_addr learned my addr 172.21.15.105:0/736231796 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd80df29640 1 --2- 172.21.15.105:0/736231796 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd80806bf50 0x7fd8080abfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd80e72a640 1 --2- 172.21.15.105:0/736231796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd808154070 0x7fd808147000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd80d728640 1 -- 172.21.15.105:0/736231796 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd80806bf50 msgr2=0x7fd8080abfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd80d728640 1 --2- 172.21.15.105:0/736231796 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd80806bf50 0x7fd8080abfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd80d728640 1 -- 172.21.15.105:0/736231796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd808154070 msgr2=0x7fd808147000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd80d728640 1 --2- 172.21.15.105:0/736231796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd808154070 0x7fd808147000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd80d728640 1 -- 172.21.15.105:0/736231796 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f80094d0 con 0x7fd8080adb80 2023-11-28T18:29:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd80e72a640 1 --2- 172.21.15.105:0/736231796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd808154070 0x7fd808147000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd80d728640 1 --2- 172.21.15.105:0/736231796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8080adb80 0x7fd80814e8e0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80190e0 tx=0x7fd7f8019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:17.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd7fe7fc640 1 -- 172.21.15.105:0/736231796 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80b37d0 con 0x7fd8080adb80 2023-11-28T18:29:17.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd7fe7fc640 1 -- 172.21.15.105:0/736231796 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f80b3930 con 0x7fd8080adb80 2023-11-28T18:29:17.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd7fe7fc640 1 -- 172.21.15.105:0/736231796 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80b3ca0 con 0x7fd8080adb80 2023-11-28T18:29:17.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.921+0000 7fd81018d640 1 -- 172.21.15.105:0/736231796 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd80814f840 con 0x7fd8080adb80 2023-11-28T18:29:17.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.925+0000 7fd81018d640 1 -- 172.21.15.105:0/736231796 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80814d9d0 con 0x7fd8080adb80 2023-11-28T18:29:17.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.925+0000 7fd81018d640 1 -- 172.21.15.105:0/736231796 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd80814dbc0 con 0x7fd8080adb80 2023-11-28T18:29:17.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.925+0000 7fd7fe7fc640 1 -- 172.21.15.105:0/736231796 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd7f80b2070 con 0x7fd8080adb80 2023-11-28T18:29:17.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.925+0000 7fd7fe7fc640 1 --2- 172.21.15.105:0/736231796 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd7ec0668d0 0x7fd7ec068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:17.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.925+0000 7fd7fe7fc640 1 -- 172.21.15.105:0/736231796 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fd7f811e970 con 0x7fd8080adb80 2023-11-28T18:29:17.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.925+0000 7fd80df29640 1 --2- 172.21.15.105:0/736231796 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd7ec0668d0 0x7fd7ec068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:17.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.925+0000 7fd80df29640 1 --2- 172.21.15.105:0/736231796 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd7ec0668d0 0x7fd7ec068d90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd804007fd0 tx=0x7fd804009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:17.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:17.929+0000 7fd7fe7fc640 1 -- 172.21.15.105:0/736231796 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd7f80eb950 con 0x7fd8080adb80 2023-11-28T18:29:18.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.081+0000 7fd81018d640 1 -- 172.21.15.105:0/736231796 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random}] v 0) v1 -- 0x7fd808067570 con 0x7fd8080adb80 2023-11-28T18:29:18.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.085+0000 7fd7fe7fc640 1 -- 172.21.15.105:0/736231796 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random}]=0 v44)=0 v44) v1 ==== 128+0+0 (secure 0 0 0) 0x7fd7f80b00c0 con 0x7fd8080adb80 2023-11-28T18:29:18.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.089+0000 7fd81018d640 1 -- 172.21.15.105:0/736231796 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd7ec0668d0 msgr2=0x7fd7ec068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:18.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.089+0000 7fd81018d640 1 --2- 172.21.15.105:0/736231796 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd7ec0668d0 0x7fd7ec068d90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd804007fd0 tx=0x7fd804009040 comp rx=0 tx=0).stop 2023-11-28T18:29:18.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.089+0000 7fd81018d640 1 -- 172.21.15.105:0/736231796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8080adb80 msgr2=0x7fd80814e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.089+0000 7fd81018d640 1 --2- 172.21.15.105:0/736231796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8080adb80 0x7fd80814e8e0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80190e0 tx=0x7fd7f8019270 comp rx=0 tx=0).stop 2023-11-28T18:29:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.089+0000 7fd81018d640 1 -- 172.21.15.105:0/736231796 shutdown_connections 2023-11-28T18:29:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.089+0000 7fd81018d640 1 --2- 172.21.15.105:0/736231796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd808154070 0x7fd808147000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.089+0000 7fd81018d640 1 --2- 172.21.15.105:0/736231796 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd7ec0668d0 0x7fd7ec068d90 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.089+0000 7fd81018d640 1 --2- 172.21.15.105:0/736231796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8080adb80 0x7fd80814e8e0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.089+0000 7fd81018d640 1 --2- 172.21.15.105:0/736231796 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd80806bf50 0x7fd8080abfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.089+0000 7fd81018d640 1 -- 172.21.15.105:0/736231796 >> 172.21.15.105:0/736231796 conn(0x7fd8080ceb90 msgr2=0x7fd8080ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.089+0000 7fd81018d640 1 -- 172.21.15.105:0/736231796 shutdown_connections 2023-11-28T18:29:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.089+0000 7fd81018d640 1 -- 172.21.15.105:0/736231796 wait complete. 2023-11-28T18:29:18.113 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_distributed True 2023-11-28T18:29:18.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.249+0000 7f518a454640 1 Processor -- start 2023-11-28T18:29:18.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.249+0000 7f518a454640 1 -- start start 2023-11-28T18:29:18.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.249+0000 7f518a454640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51840caf60 0x7f51840cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:18.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.249+0000 7f518a454640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f51840ad850 0x7f51840701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:18.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.249+0000 7f518a454640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f518406bf50 con 0x7f5184159f40 2023-11-28T18:29:18.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.249+0000 7f518a454640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51840cb3c0 con 0x7f51840ad850 2023-11-28T18:29:18.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.249+0000 7f518a454640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518415a320 con 0x7f51840caf60 2023-11-28T18:29:18.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.249+0000 7f51889f1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f51840ad850 0x7f51840701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:18.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.249+0000 7f5183fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5184159f40 0x7f51840ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57804/0 (socket says 172.21.15.105:57804) 2023-11-28T18:29:18.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.249+0000 7f5183fff640 1 -- 172.21.15.105:0/2035943543 learned_addr learned my addr 172.21.15.105:0/2035943543 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:18.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.249+0000 7f51837fe640 1 --2- 172.21.15.105:0/2035943543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51840caf60 0x7f51840cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:18.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.249+0000 7f51889f1640 1 -- 172.21.15.105:0/2035943543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51840caf60 msgr2=0x7f51840cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:18.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.249+0000 7f51889f1640 1 --2- 172.21.15.105:0/2035943543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51840caf60 0x7f51840cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:18.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.249+0000 7f51889f1640 1 -- 172.21.15.105:0/2035943543 >> v1:172.21.15.105:6789/0 conn(0x7f5184159f40 legacy=0x7f51840ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:18.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.249+0000 7f51889f1640 1 -- 172.21.15.105:0/2035943543 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5184071380 con 0x7f51840ad850 2023-11-28T18:29:18.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.249+0000 7f5182ffd640 1 -- 172.21.15.105:0/2035943543 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837615896 0 0) 0x7f518406bf50 con 0x7f5184159f40 2023-11-28T18:29:18.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.253+0000 7f51889f1640 1 --2- 172.21.15.105:0/2035943543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51840ad850 0x7f51840701d0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f517c009d90 tx=0x7f517c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ff0370827af2ee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:18.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.253+0000 7f5182ffd640 1 -- 172.21.15.105:0/2035943543 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c0a5950 con 0x7f51840ad850 2023-11-28T18:29:18.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.253+0000 7f5182ffd640 1 -- 172.21.15.105:0/2035943543 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517c0a5ab0 con 0x7f51840ad850 2023-11-28T18:29:18.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.253+0000 7f5182ffd640 1 -- 172.21.15.105:0/2035943543 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c0a5e40 con 0x7f51840ad850 2023-11-28T18:29:18.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.253+0000 7f518a454640 1 -- 172.21.15.105:0/2035943543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51840ad850 msgr2=0x7f51840701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:18.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.253+0000 7f518a454640 1 --2- 172.21.15.105:0/2035943543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51840ad850 0x7f51840701d0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f517c009d90 tx=0x7f517c01c890 comp rx=0 tx=0).stop 2023-11-28T18:29:18.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.253+0000 7f518a454640 1 -- 172.21.15.105:0/2035943543 shutdown_connections 2023-11-28T18:29:18.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.253+0000 7f518a454640 1 --2- 172.21.15.105:0/2035943543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51840ad850 0x7f51840701d0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:18.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.253+0000 7f518a454640 1 --2- 172.21.15.105:0/2035943543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51840caf60 0x7f51840cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:18.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.253+0000 7f518a454640 1 -- 172.21.15.105:0/2035943543 >> 172.21.15.105:0/2035943543 conn(0x7f51840ceb90 msgr2=0x7f51840cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:18.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.253+0000 7f518a454640 1 -- 172.21.15.105:0/2035943543 shutdown_connections 2023-11-28T18:29:18.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.253+0000 7f518a454640 1 -- 172.21.15.105:0/2035943543 wait complete. 2023-11-28T18:29:18.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f518a454640 1 Processor -- start 2023-11-28T18:29:18.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f518a454640 1 -- start start 2023-11-28T18:29:18.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f518a454640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51840ad850 0x7f51841f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:18.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f518a454640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51840caf60 0x7f5184147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:18.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f518a454640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5184159f40 0x7f5184146610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:18.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f518a454640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5184147ef0 con 0x7f5184159f40 2023-11-28T18:29:18.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f518a454640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51841f4310 con 0x7f51840ad850 2023-11-28T18:29:18.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f518a454640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51841f5300 con 0x7f51840caf60 2023-11-28T18:29:18.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f51889f1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5184159f40 0x7f5184146610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:18.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f51889f1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5184159f40 0x7f5184146610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54158/0 (socket says 172.21.15.105:54158) 2023-11-28T18:29:18.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f51889f1640 1 -- 172.21.15.105:0/1100792644 learned_addr learned my addr 172.21.15.105:0/1100792644 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:18.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f51889f1640 1 -- 172.21.15.105:0/1100792644 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51840caf60 msgr2=0x7f5184147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:18.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f5183fff640 1 --2- 172.21.15.105:0/1100792644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51840ad850 0x7f51841f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:18.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f51889f1640 1 --2- 172.21.15.105:0/1100792644 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51840caf60 0x7f5184147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:18.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f51889f1640 1 -- 172.21.15.105:0/1100792644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51840ad850 msgr2=0x7f51841f4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:18.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f51889f1640 1 --2- 172.21.15.105:0/1100792644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51840ad850 0x7f51841f4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:18.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f51889f1640 1 -- 172.21.15.105:0/1100792644 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517c009a70 con 0x7f5184159f40 2023-11-28T18:29:18.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f5183fff640 1 --2- 172.21.15.105:0/1100792644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51840ad850 0x7f51841f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:18.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f51889f1640 1 --2- 172.21.15.105:0/1100792644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5184159f40 0x7f5184146610 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f517c0b0040 tx=0x7f517c019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:18.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f5180ff9640 1 -- 172.21.15.105:0/1100792644 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c0ade40 con 0x7f5184159f40 2023-11-28T18:29:18.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f5180ff9640 1 -- 172.21.15.105:0/1100792644 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517c009450 con 0x7f5184159f40 2023-11-28T18:29:18.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f518a454640 1 -- 172.21.15.105:0/1100792644 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5184147930 con 0x7f5184159f40 2023-11-28T18:29:18.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f5180ff9640 1 -- 172.21.15.105:0/1100792644 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c0b9690 con 0x7f5184159f40 2023-11-28T18:29:18.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.257+0000 7f518a454640 1 -- 172.21.15.105:0/1100792644 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51841f45c0 con 0x7f5184159f40 2023-11-28T18:29:18.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.261+0000 7f518a454640 1 -- 172.21.15.105:0/1100792644 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5184067570 con 0x7f5184159f40 2023-11-28T18:29:18.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.261+0000 7f5180ff9640 1 -- 172.21.15.105:0/1100792644 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f517c0b97f0 con 0x7f5184159f40 2023-11-28T18:29:18.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.261+0000 7f5180ff9640 1 --2- 172.21.15.105:0/1100792644 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f51700668d0 0x7f5170068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:18.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.261+0000 7f5180ff9640 1 -- 172.21.15.105:0/1100792644 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f517c11feb0 con 0x7f5184159f40 2023-11-28T18:29:18.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.261+0000 7f5183fff640 1 --2- 172.21.15.105:0/1100792644 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f51700668d0 0x7f5170068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:18.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.261+0000 7f5183fff640 1 --2- 172.21.15.105:0/1100792644 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f51700668d0 0x7f5170068d90 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5174000f70 tx=0x7f5174009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:18.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.265+0000 7f5180ff9640 1 -- 172.21.15.105:0/1100792644 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f517c0ece90 con 0x7f5184159f40 2023-11-28T18:29:18.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.425+0000 7f518a454640 1 -- 172.21.15.105:0/1100792644 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_distributed}] v 0) v1 -- 0x7f51840cf950 con 0x7f5184159f40 2023-11-28T18:29:18.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.429+0000 7f5180ff9640 1 -- 172.21.15.105:0/1100792644 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_distributed}]=0 v45)=0 v45) v1 ==== 133+0+0 (secure 0 0 0) 0x7f517c0f04b0 con 0x7f5184159f40 2023-11-28T18:29:18.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.433+0000 7f518a454640 1 -- 172.21.15.105:0/1100792644 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f51700668d0 msgr2=0x7f5170068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:18.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.433+0000 7f518a454640 1 --2- 172.21.15.105:0/1100792644 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f51700668d0 0x7f5170068d90 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5174000f70 tx=0x7f5174009040 comp rx=0 tx=0).stop 2023-11-28T18:29:18.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.433+0000 7f518a454640 1 -- 172.21.15.105:0/1100792644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5184159f40 msgr2=0x7f5184146610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:18.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.433+0000 7f518a454640 1 --2- 172.21.15.105:0/1100792644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5184159f40 0x7f5184146610 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f517c0b0040 tx=0x7f517c019630 comp rx=0 tx=0).stop 2023-11-28T18:29:18.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.433+0000 7f518a454640 1 -- 172.21.15.105:0/1100792644 shutdown_connections 2023-11-28T18:29:18.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.433+0000 7f518a454640 1 --2- 172.21.15.105:0/1100792644 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f51700668d0 0x7f5170068d90 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:18.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.433+0000 7f518a454640 1 --2- 172.21.15.105:0/1100792644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5184159f40 0x7f5184146610 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:18.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.433+0000 7f518a454640 1 --2- 172.21.15.105:0/1100792644 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51840caf60 0x7f5184147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:18.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.433+0000 7f518a454640 1 --2- 172.21.15.105:0/1100792644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51840ad850 0x7f51841f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:18.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.433+0000 7f518a454640 1 -- 172.21.15.105:0/1100792644 >> 172.21.15.105:0/1100792644 conn(0x7f51840ceb90 msgr2=0x7f518406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:18.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.433+0000 7f518a454640 1 -- 172.21.15.105:0/1100792644 shutdown_connections 2023-11-28T18:29:18.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.433+0000 7f518a454640 1 -- 172.21.15.105:0/1100792644 wait complete. 2023-11-28T18:29:18.456 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random_max 1.0 2023-11-28T18:29:18.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.593+0000 7fd15889c640 1 Processor -- start 2023-11-28T18:29:18.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.593+0000 7fd15889c640 1 -- start start 2023-11-28T18:29:18.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.593+0000 7fd15889c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd15415d4f0 0x7fd1540d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:18.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.593+0000 7fd15889c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd1540b2d20 0x7fd1540b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:18.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.593+0000 7fd15889c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1540b25b0 con 0x7fd1540d3c30 2023-11-28T18:29:18.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.593+0000 7fd15889c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1540b2900 con 0x7fd1540b2d20 2023-11-28T18:29:18.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.593+0000 7fd15889c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1540d0a20 con 0x7fd15415d4f0 2023-11-28T18:29:18.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.593+0000 7fd152d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd1540b2d20 0x7fd1540b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:18.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.593+0000 7fd152d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd1540b2d20 0x7fd1540b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54162/0 (socket says 172.21.15.105:54162) 2023-11-28T18:29:18.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.593+0000 7fd152d9d640 1 -- 172.21.15.105:0/3515942248 learned_addr learned my addr 172.21.15.105:0/3515942248 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:18.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.593+0000 7fd15259c640 1 --2- 172.21.15.105:0/3515942248 >> v2:172.21.15.148:3300/0 conn(0x7fd15415d4f0 0x7fd1540d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:18.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.593+0000 7fd152d9d640 1 -- 172.21.15.105:0/3515942248 >> v2:172.21.15.148:3300/0 conn(0x7fd15415d4f0 msgr2=0x7fd1540d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:18.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.593+0000 7fd152d9d640 1 --2- 172.21.15.105:0/3515942248 >> v2:172.21.15.148:3300/0 conn(0x7fd15415d4f0 0x7fd1540d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:18.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.593+0000 7fd152d9d640 1 -- 172.21.15.105:0/3515942248 >> v1:172.21.15.148:6789/0 conn(0x7fd1540d3c30 legacy=0x7fd1540d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:18.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.593+0000 7fd152d9d640 1 -- 172.21.15.105:0/3515942248 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1540cea30 con 0x7fd1540b2d20 2023-11-28T18:29:18.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.597+0000 7fd152d9d640 1 --2- 172.21.15.105:0/3515942248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1540b2d20 0x7fd1540b3150 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fd15408bbf0 tx=0x7fd14c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f9f0536a3d4da server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:18.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.597+0000 7fd151d9b640 1 -- 172.21.15.105:0/3515942248 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c0a57d0 con 0x7fd1540b2d20 2023-11-28T18:29:18.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.597+0000 7fd151d9b640 1 -- 172.21.15.105:0/3515942248 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14c0a5930 con 0x7fd1540b2d20 2023-11-28T18:29:18.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.597+0000 7fd151d9b640 1 -- 172.21.15.105:0/3515942248 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c0a5cc0 con 0x7fd1540b2d20 2023-11-28T18:29:18.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.597+0000 7fd15889c640 1 -- 172.21.15.105:0/3515942248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1540b2d20 msgr2=0x7fd1540b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:18.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.597+0000 7fd15889c640 1 --2- 172.21.15.105:0/3515942248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1540b2d20 0x7fd1540b3150 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fd15408bbf0 tx=0x7fd14c01c910 comp rx=0 tx=0).stop 2023-11-28T18:29:18.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.597+0000 7fd15889c640 1 -- 172.21.15.105:0/3515942248 shutdown_connections 2023-11-28T18:29:18.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.597+0000 7fd15889c640 1 --2- 172.21.15.105:0/3515942248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1540b2d20 0x7fd1540b3150 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:18.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.597+0000 7fd15889c640 1 --2- 172.21.15.105:0/3515942248 >> v2:172.21.15.148:3300/0 conn(0x7fd15415d4f0 0x7fd1540d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:18.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.597+0000 7fd15889c640 1 -- 172.21.15.105:0/3515942248 >> 172.21.15.105:0/3515942248 conn(0x7fd1540d3030 msgr2=0x7fd1540d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:18.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.597+0000 7fd15889c640 1 -- 172.21.15.105:0/3515942248 shutdown_connections 2023-11-28T18:29:18.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.597+0000 7fd15889c640 1 -- 172.21.15.105:0/3515942248 wait complete. 2023-11-28T18:29:18.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd15889c640 1 Processor -- start 2023-11-28T18:29:18.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd15889c640 1 -- start start 2023-11-28T18:29:18.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd15889c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1540b2d20 0x7fd15415b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:18.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd15889c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1540d3c30 0x7fd1540c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:18.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd15889c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd15415d4f0 0x7fd1540ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:18.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd15889c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1540c9ea0 con 0x7fd1540b2d20 2023-11-28T18:29:18.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd15889c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1540cab70 con 0x7fd1540d3c30 2023-11-28T18:29:18.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd15889c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd154158a80 con 0x7fd15415d4f0 2023-11-28T18:29:18.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd15259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1540b2d20 0x7fd15415b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:18.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd15259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1540b2d20 0x7fd15415b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54176/0 (socket says 172.21.15.105:54176) 2023-11-28T18:29:18.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd15259c640 1 -- 172.21.15.105:0/207765038 learned_addr learned my addr 172.21.15.105:0/207765038 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:18.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd15259c640 1 -- 172.21.15.105:0/207765038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd15415d4f0 msgr2=0x7fd1540ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:18.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd152d9d640 1 --2- 172.21.15.105:0/207765038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd15415d4f0 0x7fd1540ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:18.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd149d9b640 1 --2- 172.21.15.105:0/207765038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1540d3c30 0x7fd1540c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:18.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd15259c640 1 --2- 172.21.15.105:0/207765038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd15415d4f0 0x7fd1540ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:18.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd15259c640 1 -- 172.21.15.105:0/207765038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1540d3c30 msgr2=0x7fd1540c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:18.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd15259c640 1 --2- 172.21.15.105:0/207765038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1540d3c30 0x7fd1540c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:18.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd15259c640 1 -- 172.21.15.105:0/207765038 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c009b30 con 0x7fd1540b2d20 2023-11-28T18:29:18.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd152d9d640 1 --2- 172.21.15.105:0/207765038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd15415d4f0 0x7fd1540ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:18.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd15259c640 1 --2- 172.21.15.105:0/207765038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1540b2d20 0x7fd15415b830 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd14400e870 tx=0x7fd14400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:18.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd14b7fe640 1 -- 172.21.15.105:0/207765038 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd144009510 con 0x7fd1540b2d20 2023-11-28T18:29:18.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd14b7fe640 1 -- 172.21.15.105:0/207765038 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14400eec0 con 0x7fd1540b2d20 2023-11-28T18:29:18.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd14b7fe640 1 -- 172.21.15.105:0/207765038 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14400cb40 con 0x7fd1540b2d20 2023-11-28T18:29:18.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd15889c640 1 -- 172.21.15.105:0/207765038 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd154158d60 con 0x7fd1540b2d20 2023-11-28T18:29:18.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.601+0000 7fd15889c640 1 -- 172.21.15.105:0/207765038 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd15415b030 con 0x7fd1540b2d20 2023-11-28T18:29:18.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.605+0000 7fd15889c640 1 -- 172.21.15.105:0/207765038 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd154067570 con 0x7fd1540b2d20 2023-11-28T18:29:18.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.605+0000 7fd14b7fe640 1 -- 172.21.15.105:0/207765038 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd14400cca0 con 0x7fd1540b2d20 2023-11-28T18:29:18.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.605+0000 7fd14b7fe640 1 --2- 172.21.15.105:0/207765038 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd134066900 0x7fd134068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:18.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.605+0000 7fd14b7fe640 1 -- 172.21.15.105:0/207765038 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fd144084770 con 0x7fd1540b2d20 2023-11-28T18:29:18.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.605+0000 7fd149d9b640 1 --2- 172.21.15.105:0/207765038 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd134066900 0x7fd134068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:18.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.605+0000 7fd149d9b640 1 --2- 172.21.15.105:0/207765038 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd134066900 0x7fd134068dc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd1540ca8b0 tx=0x7fd13c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:18.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.609+0000 7fd14b7fe640 1 -- 172.21.15.105:0/207765038 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd144051750 con 0x7fd1540b2d20 2023-11-28T18:29:18.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.761+0000 7fd15889c640 1 -- 172.21.15.105:0/207765038 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random_max}] v 0) v1 -- 0x7fd1540b0070 con 0x7fd1540b2d20 2023-11-28T18:29:18.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.765+0000 7fd14b7fe640 1 -- 172.21.15.105:0/207765038 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random_max}]=0 v46)=0 v46) v1 ==== 131+0+0 (secure 0 0 0) 0x7fd144082e60 con 0x7fd1540b2d20 2023-11-28T18:29:18.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.765+0000 7fd15889c640 1 -- 172.21.15.105:0/207765038 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd134066900 msgr2=0x7fd134068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:18.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.765+0000 7fd15889c640 1 --2- 172.21.15.105:0/207765038 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd134066900 0x7fd134068dc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd1540ca8b0 tx=0x7fd13c009040 comp rx=0 tx=0).stop 2023-11-28T18:29:18.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.765+0000 7fd15889c640 1 -- 172.21.15.105:0/207765038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1540b2d20 msgr2=0x7fd15415b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:18.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.765+0000 7fd15889c640 1 --2- 172.21.15.105:0/207765038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1540b2d20 0x7fd15415b830 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd14400e870 tx=0x7fd14400ec70 comp rx=0 tx=0).stop 2023-11-28T18:29:18.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.765+0000 7fd15889c640 1 -- 172.21.15.105:0/207765038 shutdown_connections 2023-11-28T18:29:18.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.765+0000 7fd15889c640 1 --2- 172.21.15.105:0/207765038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd15415d4f0 0x7fd1540ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:18.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.765+0000 7fd15889c640 1 --2- 172.21.15.105:0/207765038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1540d3c30 0x7fd1540c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:18.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.765+0000 7fd15889c640 1 --2- 172.21.15.105:0/207765038 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd134066900 0x7fd134068dc0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:18.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.765+0000 7fd15889c640 1 --2- 172.21.15.105:0/207765038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1540b2d20 0x7fd15415b830 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:18.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.765+0000 7fd15889c640 1 -- 172.21.15.105:0/207765038 >> 172.21.15.105:0/207765038 conn(0x7fd1540d3030 msgr2=0x7fd15406c850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:18.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.765+0000 7fd15889c640 1 -- 172.21.15.105:0/207765038 shutdown_connections 2023-11-28T18:29:18.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:18.765+0000 7fd15889c640 1 -- 172.21.15.105:0/207765038 wait complete. 2023-11-28T18:29:18.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:18.790 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-28T18:29:18.791 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:29:18.791 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:29:18.791 DEBUG:teuthology.orchestra.run.smithi105:> # Use up a random number of inode numbers so the ephemeral pinning is not the same every test. 2023-11-28T18:29:18.791 DEBUG:teuthology.orchestra.run.smithi105:> mkdir .inode_number_thrash 2023-11-28T18:29:18.791 DEBUG:teuthology.orchestra.run.smithi105:> count=$((RANDOM % 1024)) 2023-11-28T18:29:18.791 DEBUG:teuthology.orchestra.run.smithi105:> for ((i = 0; i < count; i++)); do touch .inode_number_thrash/$i; done 2023-11-28T18:29:18.791 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf .inode_number_thrash 2023-11-28T18:29:18.791 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:29:19.469 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-11-28T18:29:19.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.605+0000 7f5045d2a640 1 Processor -- start 2023-11-28T18:29:19.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.605+0000 7f5045d2a640 1 -- start start 2023-11-28T18:29:19.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.605+0000 7f5045d2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5040071960 0x7f504015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:19.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.605+0000 7f5045d2a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f504015b030 0x7f50400709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:19.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.605+0000 7f5045d2a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50400701d0 con 0x7f50400b4290 2023-11-28T18:29:19.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.605+0000 7f5045d2a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f504006df80 con 0x7f504015b030 2023-11-28T18:29:19.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.605+0000 7f5045d2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5040071df0 con 0x7f5040071960 2023-11-28T18:29:19.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.605+0000 7f503ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f504015b030 0x7f50400709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:19.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.605+0000 7f503f7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f50400b4290 0x7f50400b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57816/0 (socket says 172.21.15.105:57816) 2023-11-28T18:29:19.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.605+0000 7f503f7fe640 1 -- 172.21.15.105:0/2758235332 learned_addr learned my addr 172.21.15.105:0/2758235332 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:19.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.605+0000 7f503effd640 1 --2- 172.21.15.105:0/2758235332 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5040071960 0x7f504015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:19.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.605+0000 7f503ffff640 1 -- 172.21.15.105:0/2758235332 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5040071960 msgr2=0x7f504015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:19.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.605+0000 7f503ffff640 1 --2- 172.21.15.105:0/2758235332 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5040071960 0x7f504015f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:19.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.605+0000 7f503ffff640 1 -- 172.21.15.105:0/2758235332 >> v1:172.21.15.105:6789/0 conn(0x7f50400b4290 legacy=0x7f50400b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:19.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.605+0000 7f503ffff640 1 -- 172.21.15.105:0/2758235332 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50400720d0 con 0x7f504015b030 2023-11-28T18:29:19.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.605+0000 7f503e7fc640 1 -- 172.21.15.105:0/2758235332 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3674610036 0 0) 0x7f50400701d0 con 0x7f50400b4290 2023-11-28T18:29:19.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.605+0000 7f503effd640 1 --2- 172.21.15.105:0/2758235332 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5040071960 0x7f504015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:19.613 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.609+0000 7f503ffff640 1 --2- 172.21.15.105:0/2758235332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f504015b030 0x7f50400709b0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f5038009d90 tx=0x7f503801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b41fedde696141 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:19.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.609+0000 7f503e7fc640 1 -- 172.21.15.105:0/2758235332 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50380a58f0 con 0x7f504015b030 2023-11-28T18:29:19.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.609+0000 7f503e7fc640 1 -- 172.21.15.105:0/2758235332 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50380a5a50 con 0x7f504015b030 2023-11-28T18:29:19.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.609+0000 7f503e7fc640 1 -- 172.21.15.105:0/2758235332 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50380a5de0 con 0x7f504015b030 2023-11-28T18:29:19.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.609+0000 7f5045d2a640 1 -- 172.21.15.105:0/2758235332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f504015b030 msgr2=0x7f50400709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:19.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.609+0000 7f5045d2a640 1 --2- 172.21.15.105:0/2758235332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f504015b030 0x7f50400709b0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f5038009d90 tx=0x7f503801c920 comp rx=0 tx=0).stop 2023-11-28T18:29:19.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.609+0000 7f5045d2a640 1 -- 172.21.15.105:0/2758235332 shutdown_connections 2023-11-28T18:29:19.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.609+0000 7f5045d2a640 1 --2- 172.21.15.105:0/2758235332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f504015b030 0x7f50400709b0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:19.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.609+0000 7f5045d2a640 1 --2- 172.21.15.105:0/2758235332 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5040071960 0x7f504015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:19.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.609+0000 7f5045d2a640 1 -- 172.21.15.105:0/2758235332 >> 172.21.15.105:0/2758235332 conn(0x7f50400ceb90 msgr2=0x7f50400cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:19.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.609+0000 7f5045d2a640 1 -- 172.21.15.105:0/2758235332 shutdown_connections 2023-11-28T18:29:19.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.609+0000 7f5045d2a640 1 -- 172.21.15.105:0/2758235332 wait complete. 2023-11-28T18:29:19.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f5045d2a640 1 Processor -- start 2023-11-28T18:29:19.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f5045d2a640 1 -- start start 2023-11-28T18:29:19.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f5045d2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50400b4290 0x7f50400caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:19.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f5045d2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f504015b030 0x7f50401522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:19.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f5045d2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5040155e70 0x7f5040148660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:19.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f5045d2a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5040155a50 con 0x7f504015b030 2023-11-28T18:29:19.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f5045d2a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5040155420 con 0x7f50400b4290 2023-11-28T18:29:19.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f5045d2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5040149a00 con 0x7f5040155e70 2023-11-28T18:29:19.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f503effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f504015b030 0x7f50401522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:19.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f503effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f504015b030 0x7f50401522f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54200/0 (socket says 172.21.15.105:54200) 2023-11-28T18:29:19.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f503effd640 1 -- 172.21.15.105:0/2559698523 learned_addr learned my addr 172.21.15.105:0/2559698523 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:19.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f503ffff640 1 --2- 172.21.15.105:0/2559698523 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5040155e70 0x7f5040148660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:19.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f503f7fe640 1 --2- 172.21.15.105:0/2559698523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50400b4290 0x7f50400caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:19.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f503effd640 1 -- 172.21.15.105:0/2559698523 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5040155e70 msgr2=0x7f5040148660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:19.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f503effd640 1 --2- 172.21.15.105:0/2559698523 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5040155e70 0x7f5040148660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:19.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f503effd640 1 -- 172.21.15.105:0/2559698523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50400b4290 msgr2=0x7f50400caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:19.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f503effd640 1 --2- 172.21.15.105:0/2559698523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50400b4290 0x7f50400caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:19.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f503effd640 1 -- 172.21.15.105:0/2559698523 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5038009a70 con 0x7f504015b030 2023-11-28T18:29:19.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f503f7fe640 1 --2- 172.21.15.105:0/2559698523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50400b4290 0x7f50400caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:19.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f503effd640 1 --2- 172.21.15.105:0/2559698523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f504015b030 0x7f50401522f0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f503400e810 tx=0x7f503400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:19.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f5027fff640 1 -- 172.21.15.105:0/2559698523 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503400c8d0 con 0x7f504015b030 2023-11-28T18:29:19.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f5027fff640 1 -- 172.21.15.105:0/2559698523 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503400ca30 con 0x7f504015b030 2023-11-28T18:29:19.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f5027fff640 1 -- 172.21.15.105:0/2559698523 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503400cde0 con 0x7f504015b030 2023-11-28T18:29:19.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f5045d2a640 1 -- 172.21.15.105:0/2559698523 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50400cd810 con 0x7f504015b030 2023-11-28T18:29:19.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.613+0000 7f5045d2a640 1 -- 172.21.15.105:0/2559698523 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5040155670 con 0x7f504015b030 2023-11-28T18:29:19.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.617+0000 7f5045d2a640 1 -- 172.21.15.105:0/2559698523 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5040067570 con 0x7f504015b030 2023-11-28T18:29:19.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.617+0000 7f5027fff640 1 -- 172.21.15.105:0/2559698523 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5034013410 con 0x7f504015b030 2023-11-28T18:29:19.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.617+0000 7f5027fff640 1 --2- 172.21.15.105:0/2559698523 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f502c066900 0x7f502c068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:19.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.617+0000 7f5027fff640 1 -- 172.21.15.105:0/2559698523 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f50340842e0 con 0x7f504015b030 2023-11-28T18:29:19.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.617+0000 7f503f7fe640 1 --2- 172.21.15.105:0/2559698523 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f502c066900 0x7f502c068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:19.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.617+0000 7f503f7fe640 1 --2- 172.21.15.105:0/2559698523 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f502c066900 0x7f502c068dc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5030000f70 tx=0x7f5030007c40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:19.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.621+0000 7f5027fff640 1 -- 172.21.15.105:0/2559698523 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5034051400 con 0x7f504015b030 2023-11-28T18:29:19.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:19.801+0000 7f5045d2a640 1 -- 172.21.15.105:0/2559698523 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f5040071410 con 0x7f504015b030 2023-11-28T18:29:20.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.093+0000 7f5027fff640 1 -- 172.21.15.105:0/2559698523 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v88) v1 ==== 105+0+0 (secure 0 0 0) 0x7f5034054a20 con 0x7f504015b030 2023-11-28T18:29:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.093+0000 7f5045d2a640 1 -- 172.21.15.105:0/2559698523 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f502c066900 msgr2=0x7f502c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.093+0000 7f5045d2a640 1 --2- 172.21.15.105:0/2559698523 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f502c066900 0x7f502c068dc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5030000f70 tx=0x7f5030007c40 comp rx=0 tx=0).stop 2023-11-28T18:29:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.093+0000 7f5045d2a640 1 -- 172.21.15.105:0/2559698523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f504015b030 msgr2=0x7f50401522f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.093+0000 7f5045d2a640 1 --2- 172.21.15.105:0/2559698523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f504015b030 0x7f50401522f0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f503400e810 tx=0x7f503400ecd0 comp rx=0 tx=0).stop 2023-11-28T18:29:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.093+0000 7f5045d2a640 1 -- 172.21.15.105:0/2559698523 shutdown_connections 2023-11-28T18:29:20.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.093+0000 7f5045d2a640 1 --2- 172.21.15.105:0/2559698523 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5040155e70 0x7f5040148660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:20.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.093+0000 7f5045d2a640 1 --2- 172.21.15.105:0/2559698523 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f502c066900 0x7f502c068dc0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:20.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.093+0000 7f5045d2a640 1 --2- 172.21.15.105:0/2559698523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f504015b030 0x7f50401522f0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:20.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.093+0000 7f5045d2a640 1 --2- 172.21.15.105:0/2559698523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50400b4290 0x7f50400caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:20.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.093+0000 7f5045d2a640 1 -- 172.21.15.105:0/2559698523 >> 172.21.15.105:0/2559698523 conn(0x7f50400ceb90 msgr2=0x7f50400cb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:20.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.093+0000 7f5045d2a640 1 -- 172.21.15.105:0/2559698523 shutdown_connections 2023-11-28T18:29:20.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.093+0000 7f5045d2a640 1 -- 172.21.15.105:0/2559698523 wait complete. 2023-11-28T18:29:20.124 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:29:20.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.257+0000 7f4760032640 1 Processor -- start 2023-11-28T18:29:20.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.257+0000 7f4760032640 1 -- start start 2023-11-28T18:29:20.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.257+0000 7f4760032640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f475815d4f0 0x7f47580d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:20.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.257+0000 7f4760032640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f47580b2d20 0x7f47580b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:20.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.257+0000 7f4760032640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47580b25b0 con 0x7f47580d3c30 2023-11-28T18:29:20.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.257+0000 7f4760032640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47580b2900 con 0x7f47580b2d20 2023-11-28T18:29:20.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.257+0000 7f4760032640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47580d0a20 con 0x7f475815d4f0 2023-11-28T18:29:20.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.257+0000 7f475e5cf640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f47580b2d20 0x7f47580b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:20.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.257+0000 7f475d5cd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f47580d3c30 0x7f47580d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57822/0 (socket says 172.21.15.105:57822) 2023-11-28T18:29:20.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.257+0000 7f475d5cd640 1 -- 172.21.15.105:0/2267350048 learned_addr learned my addr 172.21.15.105:0/2267350048 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:20.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.257+0000 7f475ddce640 1 --2- 172.21.15.105:0/2267350048 >> v2:172.21.15.148:3300/0 conn(0x7f475815d4f0 0x7f47580d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:20.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.257+0000 7f475cdcc640 1 -- 172.21.15.105:0/2267350048 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112494763 0 0) 0x7f47580b25b0 con 0x7f47580d3c30 2023-11-28T18:29:20.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.257+0000 7f475cdcc640 1 -- 172.21.15.105:0/2267350048 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4744003530 con 0x7f47580d3c30 2023-11-28T18:29:20.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.257+0000 7f475e5cf640 1 -- 172.21.15.105:0/2267350048 >> v2:172.21.15.148:3300/0 conn(0x7f475815d4f0 msgr2=0x7f47580d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:20.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.257+0000 7f475e5cf640 1 --2- 172.21.15.105:0/2267350048 >> v2:172.21.15.148:3300/0 conn(0x7f475815d4f0 0x7f47580d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:20.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.257+0000 7f475e5cf640 1 -- 172.21.15.105:0/2267350048 >> v1:172.21.15.105:6789/0 conn(0x7f47580d3c30 legacy=0x7f47580d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:20.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.257+0000 7f475e5cf640 1 -- 172.21.15.105:0/2267350048 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47580cea30 con 0x7f47580b2d20 2023-11-28T18:29:20.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.261+0000 7f475e5cf640 1 --2- 172.21.15.105:0/2267350048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47580b2d20 0x7f47580b3150 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f475808bbf0 tx=0x7f475001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fffc5fda246bf8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:20.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.261+0000 7f475cdcc640 1 -- 172.21.15.105:0/2267350048 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47500a5910 con 0x7f47580b2d20 2023-11-28T18:29:20.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.261+0000 7f475cdcc640 1 -- 172.21.15.105:0/2267350048 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47500a5a70 con 0x7f47580b2d20 2023-11-28T18:29:20.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.261+0000 7f475cdcc640 1 -- 172.21.15.105:0/2267350048 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47500a5de0 con 0x7f47580b2d20 2023-11-28T18:29:20.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.261+0000 7f4760032640 1 -- 172.21.15.105:0/2267350048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47580b2d20 msgr2=0x7f47580b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:20.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.261+0000 7f4760032640 1 --2- 172.21.15.105:0/2267350048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47580b2d20 0x7f47580b3150 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f475808bbf0 tx=0x7f475001c850 comp rx=0 tx=0).stop 2023-11-28T18:29:20.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.261+0000 7f4760032640 1 -- 172.21.15.105:0/2267350048 shutdown_connections 2023-11-28T18:29:20.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.261+0000 7f4760032640 1 --2- 172.21.15.105:0/2267350048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47580b2d20 0x7f47580b3150 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:20.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.261+0000 7f4760032640 1 --2- 172.21.15.105:0/2267350048 >> v2:172.21.15.148:3300/0 conn(0x7f475815d4f0 0x7f47580d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:20.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.261+0000 7f4760032640 1 -- 172.21.15.105:0/2267350048 >> 172.21.15.105:0/2267350048 conn(0x7f47580d3030 msgr2=0x7f47580d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:20.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.261+0000 7f4760032640 1 -- 172.21.15.105:0/2267350048 shutdown_connections 2023-11-28T18:29:20.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.261+0000 7f4760032640 1 -- 172.21.15.105:0/2267350048 wait complete. 2023-11-28T18:29:20.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f4760032640 1 Processor -- start 2023-11-28T18:29:20.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f4760032640 1 -- start start 2023-11-28T18:29:20.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f4760032640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47580b2d20 0x7f47580cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:20.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f4760032640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47580d3c30 0x7f47580c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:20.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f4760032640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f475815d4f0 0x7f475806c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:20.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f4760032640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47580b60c0 con 0x7f47580b2d20 2023-11-28T18:29:20.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f4760032640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47580b63a0 con 0x7f475815d4f0 2023-11-28T18:29:20.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f4760032640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47580b6680 con 0x7f47580d3c30 2023-11-28T18:29:20.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f475ddce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47580b2d20 0x7f47580cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:20.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f475ddce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47580b2d20 0x7f47580cca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54216/0 (socket says 172.21.15.105:54216) 2023-11-28T18:29:20.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f475ddce640 1 -- 172.21.15.105:0/2751244569 learned_addr learned my addr 172.21.15.105:0/2751244569 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:20.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f475ddce640 1 -- 172.21.15.105:0/2751244569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47580d3c30 msgr2=0x7f47580c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:20.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f475d5cd640 1 --2- 172.21.15.105:0/2751244569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47580d3c30 0x7f47580c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:20.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f475e5cf640 1 --2- 172.21.15.105:0/2751244569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f475815d4f0 0x7f475806c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:20.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f475ddce640 1 --2- 172.21.15.105:0/2751244569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47580d3c30 0x7f47580c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:20.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f475ddce640 1 -- 172.21.15.105:0/2751244569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f475815d4f0 msgr2=0x7f475806c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:20.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f475ddce640 1 --2- 172.21.15.105:0/2751244569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f475815d4f0 0x7f475806c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:20.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f475ddce640 1 -- 172.21.15.105:0/2751244569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4750009a70 con 0x7f47580b2d20 2023-11-28T18:29:20.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f475e5cf640 1 --2- 172.21.15.105:0/2751244569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f475815d4f0 0x7f475806c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:20.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f475d5cd640 1 --2- 172.21.15.105:0/2751244569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47580d3c30 0x7f47580c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:20.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f475ddce640 1 --2- 172.21.15.105:0/2751244569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47580b2d20 0x7f47580cca80 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f475400e930 tx=0x7f475400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:20.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f474e7fc640 1 -- 172.21.15.105:0/2751244569 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754009690 con 0x7f47580b2d20 2023-11-28T18:29:20.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f474e7fc640 1 -- 172.21.15.105:0/2751244569 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475400ca00 con 0x7f47580b2d20 2023-11-28T18:29:20.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f474e7fc640 1 -- 172.21.15.105:0/2751244569 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475400cd70 con 0x7f47580b2d20 2023-11-28T18:29:20.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f4760032640 1 -- 172.21.15.105:0/2751244569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47580b6960 con 0x7f47580b2d20 2023-11-28T18:29:20.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.265+0000 7f4760032640 1 -- 172.21.15.105:0/2751244569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47581565d0 con 0x7f47580b2d20 2023-11-28T18:29:20.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.269+0000 7f4760032640 1 -- 172.21.15.105:0/2751244569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4758067570 con 0x7f47580b2d20 2023-11-28T18:29:20.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.269+0000 7f474e7fc640 1 -- 172.21.15.105:0/2751244569 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f4754013470 con 0x7f47580b2d20 2023-11-28T18:29:20.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.269+0000 7f474e7fc640 1 --2- 172.21.15.105:0/2751244569 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f473c0668d0 0x7f473c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:20.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.269+0000 7f474e7fc640 1 -- 172.21.15.105:0/2751244569 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f4754084920 con 0x7f47580b2d20 2023-11-28T18:29:20.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.269+0000 7f475d5cd640 1 --2- 172.21.15.105:0/2751244569 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f473c0668d0 0x7f473c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:20.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.269+0000 7f475d5cd640 1 --2- 172.21.15.105:0/2751244569 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f473c0668d0 0x7f473c068d90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f47581f9300 tx=0x7f4748009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:20.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.273+0000 7f474e7fc640 1 -- 172.21.15.105:0/2751244569 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f47540519c0 con 0x7f47580b2d20 2023-11-28T18:29:20.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.461+0000 7f4760032640 1 -- 172.21.15.105:0/2751244569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f475806b670 con 0x7f47580b2d20 2023-11-28T18:29:20.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.461+0000 7f474e7fc640 1 -- 172.21.15.105:0/2751244569 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f4754017090 con 0x7f47580b2d20 2023-11-28T18:29:20.467 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:29:20.467 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":89,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/1464337040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1464337040},{"type":"v1","addr":"172.21.15.148:6834","nonce":1464337040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3461832163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":3461832163},{"type":"v1","addr":"172.21.15.148:6835","nonce":3461832163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:29:09.707308+0000","modified":"2023-11-28T18:29:20.100850+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1],"up":{"mds_0":7689,"mds_1":7665},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7665":{"gid":7665,"name":"c","rank":1,"incarnation":89,"state":"up:creating","state_seq":1,"addr":"172.21.15.105:6835/4162499249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":4162499249},{"type":"v1","addr":"172.21.15.105:6835","nonce":4162499249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7689":{"gid":7689,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/3239728352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3239728352},{"type":"v1","addr":"172.21.15.105:6837","nonce":3239728352}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-28T18:29:20.468 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 89 2023-11-28T18:29:20.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.465+0000 7f4760032640 1 -- 172.21.15.105:0/2751244569 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f473c0668d0 msgr2=0x7f473c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:20.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.465+0000 7f4760032640 1 --2- 172.21.15.105:0/2751244569 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f473c0668d0 0x7f473c068d90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f47581f9300 tx=0x7f4748009040 comp rx=0 tx=0).stop 2023-11-28T18:29:20.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.465+0000 7f4760032640 1 -- 172.21.15.105:0/2751244569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47580b2d20 msgr2=0x7f47580cca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:20.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.465+0000 7f4760032640 1 --2- 172.21.15.105:0/2751244569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47580b2d20 0x7f47580cca80 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f475400e930 tx=0x7f475400edf0 comp rx=0 tx=0).stop 2023-11-28T18:29:20.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.465+0000 7f4760032640 1 -- 172.21.15.105:0/2751244569 shutdown_connections 2023-11-28T18:29:20.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.465+0000 7f4760032640 1 --2- 172.21.15.105:0/2751244569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f475815d4f0 0x7f475806c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:20.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.465+0000 7f4760032640 1 --2- 172.21.15.105:0/2751244569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47580d3c30 0x7f47580c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:20.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.465+0000 7f4760032640 1 --2- 172.21.15.105:0/2751244569 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f473c0668d0 0x7f473c068d90 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:20.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.465+0000 7f4760032640 1 --2- 172.21.15.105:0/2751244569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47580b2d20 0x7f47580cca80 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:20.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.465+0000 7f4760032640 1 -- 172.21.15.105:0/2751244569 >> 172.21.15.105:0/2751244569 conn(0x7f47580d3030 msgr2=0x7f47580cf490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:20.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.465+0000 7f4760032640 1 -- 172.21.15.105:0/2751244569 shutdown_connections 2023-11-28T18:29:20.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:20.465+0000 7f4760032640 1 -- 172.21.15.105:0/2751244569 wait complete. 2023-11-28T18:29:20.890 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:29:09.707308+0000', 'modified': '2023-11-28T18:29:20.100850+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 7689, 'mds_1': 7665}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7665': {'gid': 7665, 'name': 'c', 'rank': 1, 'incarnation': 89, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.105:6835/4162499249', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 4162499249}, {'type': 'v1', 'addr': '172.21.15.105:6835', 'nonce': 4162499249}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_7689': {'gid': 7689, 'name': 'a', 'rank': 0, 'incarnation': 83, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6837/3239728352', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 3239728352}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 3239728352}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:29:20.890 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7665:up:creating 2023-11-28T18:29:21.891 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:29:22.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.033+0000 7fb57813e640 1 Processor -- start 2023-11-28T18:29:22.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.033+0000 7fb57813e640 1 -- start start 2023-11-28T18:29:22.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.037+0000 7fb57813e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb5700b4290 0x7fb5700b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.037+0000 7fb57813e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5700b4bb0 con 0x7fb570071960 2023-11-28T18:29:22.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.037+0000 7fb57813e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5700701d0 con 0x7fb57015b030 2023-11-28T18:29:22.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.037+0000 7fb57813e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb57006df80 con 0x7fb5700b4290 2023-11-28T18:29:22.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.037+0000 7fb575eda640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb5700b4290 0x7fb5700b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.037+0000 7fb5756d9640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb570071960 0x7fb57015f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40190/0 (socket says 172.21.15.105:40190) 2023-11-28T18:29:22.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.037+0000 7fb5756d9640 1 -- 172.21.15.105:0/2001001106 learned_addr learned my addr 172.21.15.105:0/2001001106 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:22.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.037+0000 7fb575eda640 1 -- 172.21.15.105:0/2001001106 >> v1:172.21.15.148:6789/0 conn(0x7fb57015b030 legacy=0x7fb5700709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:22.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.037+0000 7fb575eda640 1 -- 172.21.15.105:0/2001001106 >> v1:172.21.15.105:6789/0 conn(0x7fb570071960 legacy=0x7fb57015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:22.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.037+0000 7fb575eda640 1 -- 172.21.15.105:0/2001001106 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb570071df0 con 0x7fb5700b4290 2023-11-28T18:29:22.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.037+0000 7fb575eda640 1 --2- 172.21.15.105:0/2001001106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5700b4290 0x7fb5700b4670 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fb56c0097c0 tx=0x7fb56c01c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b201d86468aade server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:22.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.037+0000 7fb574ed8640 1 -- 172.21.15.105:0/2001001106 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c0ab600 con 0x7fb5700b4290 2023-11-28T18:29:22.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.037+0000 7fb574ed8640 1 -- 172.21.15.105:0/2001001106 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56c0a7050 con 0x7fb5700b4290 2023-11-28T18:29:22.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.037+0000 7fb574ed8640 1 -- 172.21.15.105:0/2001001106 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c0ab980 con 0x7fb5700b4290 2023-11-28T18:29:22.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.041+0000 7fb57813e640 1 -- 172.21.15.105:0/2001001106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5700b4290 msgr2=0x7fb5700b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:22.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.041+0000 7fb57813e640 1 --2- 172.21.15.105:0/2001001106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5700b4290 0x7fb5700b4670 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fb56c0097c0 tx=0x7fb56c01c2d0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.041+0000 7fb57813e640 1 -- 172.21.15.105:0/2001001106 shutdown_connections 2023-11-28T18:29:22.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.041+0000 7fb57813e640 1 --2- 172.21.15.105:0/2001001106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5700b4290 0x7fb5700b4670 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.041+0000 7fb57813e640 1 -- 172.21.15.105:0/2001001106 >> 172.21.15.105:0/2001001106 conn(0x7fb5700ceb90 msgr2=0x7fb5700cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:22.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.041+0000 7fb57813e640 1 -- 172.21.15.105:0/2001001106 shutdown_connections 2023-11-28T18:29:22.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.041+0000 7fb57813e640 1 -- 172.21.15.105:0/2001001106 wait complete. 2023-11-28T18:29:22.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.041+0000 7fb57813e640 1 Processor -- start 2023-11-28T18:29:22.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.041+0000 7fb57813e640 1 -- start start 2023-11-28T18:29:22.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.041+0000 7fb57813e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5700b4290 0x7fb57015dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.041+0000 7fb57813e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb57015b030 0x7fb5700cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.041+0000 7fb57813e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57015b470 0x7fb57015b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.041+0000 7fb57813e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5700c9290 con 0x7fb57015b470 2023-11-28T18:29:22.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.041+0000 7fb57813e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57015be60 con 0x7fb57015b030 2023-11-28T18:29:22.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.041+0000 7fb57813e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5700c8330 con 0x7fb5700b4290 2023-11-28T18:29:22.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.041+0000 7fb5766db640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57015b470 0x7fb57015b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.041+0000 7fb5766db640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57015b470 0x7fb57015b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46594/0 (socket says 172.21.15.105:46594) 2023-11-28T18:29:22.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.041+0000 7fb5766db640 1 -- 172.21.15.105:0/948920577 learned_addr learned my addr 172.21.15.105:0/948920577 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:22.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.045+0000 7fb575eda640 1 --2- 172.21.15.105:0/948920577 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5700b4290 0x7fb57015dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.045+0000 7fb5756d9640 1 --2- 172.21.15.105:0/948920577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb57015b030 0x7fb5700cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.045+0000 7fb5766db640 1 -- 172.21.15.105:0/948920577 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5700b4290 msgr2=0x7fb57015dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:22.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.045+0000 7fb5766db640 1 --2- 172.21.15.105:0/948920577 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5700b4290 0x7fb57015dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.045+0000 7fb5766db640 1 -- 172.21.15.105:0/948920577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb57015b030 msgr2=0x7fb5700cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:22.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.045+0000 7fb5766db640 1 --2- 172.21.15.105:0/948920577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb57015b030 0x7fb5700cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.045+0000 7fb5766db640 1 -- 172.21.15.105:0/948920577 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c009420 con 0x7fb57015b470 2023-11-28T18:29:22.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.045+0000 7fb5756d9640 1 --2- 172.21.15.105:0/948920577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb57015b030 0x7fb5700cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:22.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.045+0000 7fb5766db640 1 --2- 172.21.15.105:0/948920577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57015b470 0x7fb57015b920 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fb56800be50 tx=0x7fb56800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:22.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.045+0000 7fb5667fc640 1 -- 172.21.15.105:0/948920577 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb568014440 con 0x7fb57015b470 2023-11-28T18:29:22.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.045+0000 7fb5667fc640 1 -- 172.21.15.105:0/948920577 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5680145a0 con 0x7fb57015b470 2023-11-28T18:29:22.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.045+0000 7fb57813e640 1 -- 172.21.15.105:0/948920577 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5700c8610 con 0x7fb57015b470 2023-11-28T18:29:22.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.045+0000 7fb5667fc640 1 -- 172.21.15.105:0/948920577 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb568014990 con 0x7fb57015b470 2023-11-28T18:29:22.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.045+0000 7fb57813e640 1 -- 172.21.15.105:0/948920577 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5700c8bd0 con 0x7fb57015b470 2023-11-28T18:29:22.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.045+0000 7fb57813e640 1 -- 172.21.15.105:0/948920577 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5700c93d0 con 0x7fb57015b470 2023-11-28T18:29:22.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.045+0000 7fb5667fc640 1 -- 172.21.15.105:0/948920577 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fb56801d3b0 con 0x7fb57015b470 2023-11-28T18:29:22.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.049+0000 7fb5667fc640 1 --2- 172.21.15.105:0/948920577 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb5540668d0 0x7fb554068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.049+0000 7fb5667fc640 1 -- 172.21.15.105:0/948920577 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fb568084780 con 0x7fb57015b470 2023-11-28T18:29:22.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.049+0000 7fb575eda640 1 --2- 172.21.15.105:0/948920577 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb5540668d0 0x7fb554068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.049+0000 7fb575eda640 1 --2- 172.21.15.105:0/948920577 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb5540668d0 0x7fb554068d90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb56c0ad000 tx=0x7fb56c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:22.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.053+0000 7fb5667fc640 1 -- 172.21.15.105:0/948920577 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb5700c93d0 con 0x7fb57015b470 2023-11-28T18:29:22.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.233+0000 7fb57813e640 1 -- 172.21.15.105:0/948920577 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5700c93d0 con 0x7fb57015b470 2023-11-28T18:29:22.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.233+0000 7fb5667fc640 1 -- 172.21.15.105:0/948920577 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4955 (secure 0 0 0) 0x7fb5700c93d0 con 0x7fb57015b470 2023-11-28T18:29:22.240 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:29:22.240 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/1464337040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1464337040},{"type":"v1","addr":"172.21.15.148:6834","nonce":1464337040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:29:09.707308+0000","modified":"2023-11-28T18:29:21.127984+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7689,"mds_1":7665,"mds_2":6485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6485":{"gid":6485,"name":"b","rank":2,"incarnation":91,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6835/3461832163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":3461832163},{"type":"v1","addr":"172.21.15.148:6835","nonce":3461832163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7665":{"gid":7665,"name":"c","rank":1,"incarnation":89,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6835/4162499249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":4162499249},{"type":"v1","addr":"172.21.15.105:6835","nonce":4162499249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7689":{"gid":7689,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/3239728352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3239728352},{"type":"v1","addr":"172.21.15.105:6837","nonce":3239728352}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-28T18:29:22.241 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 92 2023-11-28T18:29:22.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.237+0000 7fb57813e640 1 -- 172.21.15.105:0/948920577 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb5540668d0 msgr2=0x7fb554068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:22.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.237+0000 7fb57813e640 1 --2- 172.21.15.105:0/948920577 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb5540668d0 0x7fb554068d90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb56c0ad000 tx=0x7fb56c0a6000 comp rx=0 tx=0).stop 2023-11-28T18:29:22.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.237+0000 7fb57813e640 1 -- 172.21.15.105:0/948920577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57015b470 msgr2=0x7fb57015b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:22.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.237+0000 7fb57813e640 1 --2- 172.21.15.105:0/948920577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57015b470 0x7fb57015b920 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fb56800be50 tx=0x7fb56800bf40 comp rx=0 tx=0).stop 2023-11-28T18:29:22.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.237+0000 7fb57813e640 1 -- 172.21.15.105:0/948920577 shutdown_connections 2023-11-28T18:29:22.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.237+0000 7fb57813e640 1 --2- 172.21.15.105:0/948920577 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb5540668d0 0x7fb554068d90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.237+0000 7fb57813e640 1 --2- 172.21.15.105:0/948920577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57015b470 0x7fb57015b920 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.237+0000 7fb57813e640 1 --2- 172.21.15.105:0/948920577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb57015b030 0x7fb5700cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.237+0000 7fb57813e640 1 --2- 172.21.15.105:0/948920577 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5700b4290 0x7fb57015dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.237+0000 7fb57813e640 1 -- 172.21.15.105:0/948920577 >> 172.21.15.105:0/948920577 conn(0x7fb5700ceb90 msgr2=0x7fb5700ad4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:22.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.237+0000 7fb57813e640 1 -- 172.21.15.105:0/948920577 shutdown_connections 2023-11-28T18:29:22.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.237+0000 7fb57813e640 1 -- 172.21.15.105:0/948920577 wait complete. 2023-11-28T18:29:22.663 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 92, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:29:09.707308+0000', 'modified': '2023-11-28T18:29:21.127984+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 7689, 'mds_1': 7665, 'mds_2': 6485}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6485': {'gid': 6485, 'name': 'b', 'rank': 2, 'incarnation': 91, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.148:6835/3461832163', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6833', 'nonce': 3461832163}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 3461832163}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_7665': {'gid': 7665, 'name': 'c', 'rank': 1, 'incarnation': 89, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.105:6835/4162499249', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 4162499249}, {'type': 'v1', 'addr': '172.21.15.105:6835', 'nonce': 4162499249}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_7689': {'gid': 7689, 'name': 'a', 'rank': 0, 'incarnation': 83, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6837/3239728352', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 3239728352}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 3239728352}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:29:22.663 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-11-28T18:29:22.663 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-28T18:29:22.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.797+0000 7fe3fec32640 1 Processor -- start 2023-11-28T18:29:22.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.797+0000 7fe3fec32640 1 -- start start 2023-11-28T18:29:22.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.797+0000 7fe3fec32640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe3f80b4290 0x7fe3f80b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.797+0000 7fe3fec32640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3f8071960 0x7fe3f815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.797+0000 7fe3fec32640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe3f815b030 0x7fe3f80709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.797+0000 7fe3fec32640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f80b4bb0 con 0x7fe3f815b030 2023-11-28T18:29:22.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.797+0000 7fe3fec32640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f80701d0 con 0x7fe3f80b4290 2023-11-28T18:29:22.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.797+0000 7fe3fec32640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f806df80 con 0x7fe3f8071960 2023-11-28T18:29:22.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.797+0000 7fe3fd1cf640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe3f815b030 0x7fe3f80709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.797+0000 7fe3fd1cf640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe3f815b030 0x7fe3f80709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46598/0 (socket says 172.21.15.105:46598) 2023-11-28T18:29:22.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.797+0000 7fe3fd1cf640 1 -- 172.21.15.105:0/237600969 learned_addr learned my addr 172.21.15.105:0/237600969 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:22.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.797+0000 7fe3effff640 1 --2- 172.21.15.105:0/237600969 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3f8071960 0x7fe3f815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.797+0000 7fe3fc9ce640 1 --2- 172.21.15.105:0/237600969 >> v2:172.21.15.148:3300/0 conn(0x7fe3f80b4290 0x7fe3f80b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.797+0000 7fe3fd1cf640 1 -- 172.21.15.105:0/237600969 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3f8071960 msgr2=0x7fe3f815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:22.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.797+0000 7fe3fd1cf640 1 --2- 172.21.15.105:0/237600969 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3f8071960 0x7fe3f815f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.797+0000 7fe3fd1cf640 1 -- 172.21.15.105:0/237600969 >> v2:172.21.15.148:3300/0 conn(0x7fe3f80b4290 msgr2=0x7fe3f80b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:22.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.797+0000 7fe3fd1cf640 1 --2- 172.21.15.105:0/237600969 >> v2:172.21.15.148:3300/0 conn(0x7fe3f80b4290 0x7fe3f80b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.797+0000 7fe3fd1cf640 1 -- 172.21.15.105:0/237600969 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f8071df0 con 0x7fe3f815b030 2023-11-28T18:29:22.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.801+0000 7fe3fd1cf640 1 --2- 172.21.15.105:0/237600969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f815b030 0x7fe3f80709b0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8073a70 tx=0x7fe3f001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a5a739727518ed server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:22.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.801+0000 7fe3ef7fe640 1 -- 172.21.15.105:0/237600969 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f00a5730 con 0x7fe3f815b030 2023-11-28T18:29:22.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.801+0000 7fe3ef7fe640 1 -- 172.21.15.105:0/237600969 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f00a5890 con 0x7fe3f815b030 2023-11-28T18:29:22.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.801+0000 7fe3ef7fe640 1 -- 172.21.15.105:0/237600969 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f00a5c20 con 0x7fe3f815b030 2023-11-28T18:29:22.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.801+0000 7fe3fec32640 1 -- 172.21.15.105:0/237600969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f815b030 msgr2=0x7fe3f80709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:22.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.801+0000 7fe3fec32640 1 --2- 172.21.15.105:0/237600969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f815b030 0x7fe3f80709b0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8073a70 tx=0x7fe3f001c9a0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.801+0000 7fe3fec32640 1 -- 172.21.15.105:0/237600969 shutdown_connections 2023-11-28T18:29:22.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.801+0000 7fe3fec32640 1 --2- 172.21.15.105:0/237600969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f815b030 0x7fe3f80709b0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.801+0000 7fe3fec32640 1 --2- 172.21.15.105:0/237600969 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3f8071960 0x7fe3f815f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.801+0000 7fe3fec32640 1 --2- 172.21.15.105:0/237600969 >> v2:172.21.15.148:3300/0 conn(0x7fe3f80b4290 0x7fe3f80b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.801+0000 7fe3fec32640 1 -- 172.21.15.105:0/237600969 >> 172.21.15.105:0/237600969 conn(0x7fe3f80ceb90 msgr2=0x7fe3f80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:22.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.801+0000 7fe3fec32640 1 -- 172.21.15.105:0/237600969 shutdown_connections 2023-11-28T18:29:22.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.801+0000 7fe3fec32640 1 -- 172.21.15.105:0/237600969 wait complete. 2023-11-28T18:29:22.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.805+0000 7fe3fec32640 1 Processor -- start 2023-11-28T18:29:22.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.805+0000 7fe3fec32640 1 -- start start 2023-11-28T18:29:22.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.805+0000 7fe3fec32640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f80b4290 0x7fe3f814cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.805+0000 7fe3fec32640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3f815b030 0x7fe3f814a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.805+0000 7fe3fec32640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3f8137ef0 0x7fe3f806c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.805+0000 7fe3fec32640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f814aca0 con 0x7fe3f80b4290 2023-11-28T18:29:22.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.805+0000 7fe3fec32640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f814e000 con 0x7fe3f8137ef0 2023-11-28T18:29:22.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.805+0000 7fe3fec32640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f814d3f0 con 0x7fe3f815b030 2023-11-28T18:29:22.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.805+0000 7fe3fc9ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f80b4290 0x7fe3f814cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.805+0000 7fe3fc9ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f80b4290 0x7fe3f814cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46610/0 (socket says 172.21.15.105:46610) 2023-11-28T18:29:22.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.805+0000 7fe3fc9ce640 1 -- 172.21.15.105:0/2103442585 learned_addr learned my addr 172.21.15.105:0/2103442585 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:22.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.805+0000 7fe3effff640 1 --2- 172.21.15.105:0/2103442585 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3f815b030 0x7fe3f814a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.805+0000 7fe3fd1cf640 1 --2- 172.21.15.105:0/2103442585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3f8137ef0 0x7fe3f806c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.805+0000 7fe3fc9ce640 1 -- 172.21.15.105:0/2103442585 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3f815b030 msgr2=0x7fe3f814a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:22.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.805+0000 7fe3fc9ce640 1 --2- 172.21.15.105:0/2103442585 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3f815b030 0x7fe3f814a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.805+0000 7fe3fc9ce640 1 -- 172.21.15.105:0/2103442585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3f8137ef0 msgr2=0x7fe3f806c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:22.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.805+0000 7fe3fc9ce640 1 --2- 172.21.15.105:0/2103442585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3f8137ef0 0x7fe3f806c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.805+0000 7fe3fc9ce640 1 -- 172.21.15.105:0/2103442585 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f0009b30 con 0x7fe3f80b4290 2023-11-28T18:29:22.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.805+0000 7fe3fc9ce640 1 --2- 172.21.15.105:0/2103442585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f80b4290 0x7fe3f814cd10 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400d800 tx=0x7fe3f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:22.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.805+0000 7fe3ed7fa640 1 -- 172.21.15.105:0/2103442585 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f400b6c0 con 0x7fe3f80b4290 2023-11-28T18:29:22.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.805+0000 7fe3ed7fa640 1 -- 172.21.15.105:0/2103442585 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f400b820 con 0x7fe3f80b4290 2023-11-28T18:29:22.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.809+0000 7fe3fec32640 1 -- 172.21.15.105:0/2103442585 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f814d6d0 con 0x7fe3f80b4290 2023-11-28T18:29:22.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.809+0000 7fe3ed7fa640 1 -- 172.21.15.105:0/2103442585 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f400bb90 con 0x7fe3f80b4290 2023-11-28T18:29:22.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.809+0000 7fe3fec32640 1 -- 172.21.15.105:0/2103442585 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f814dc90 con 0x7fe3f80b4290 2023-11-28T18:29:22.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.809+0000 7fe3ed7fa640 1 -- 172.21.15.105:0/2103442585 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fe3f400bcf0 con 0x7fe3f80b4290 2023-11-28T18:29:22.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.809+0000 7fe3ed7fa640 1 --2- 172.21.15.105:0/2103442585 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3e40668d0 0x7fe3e4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.809+0000 7fe3ed7fa640 1 -- 172.21.15.105:0/2103442585 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fe3f4084a90 con 0x7fe3f80b4290 2023-11-28T18:29:22.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.809+0000 7fe3effff640 1 --2- 172.21.15.105:0/2103442585 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3e40668d0 0x7fe3e4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.809+0000 7fe401f9c000 1 Processor -- start 2023-11-28T18:29:22.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.809+0000 7fe401f9c000 1 -- start start 2023-11-28T18:29:22.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe3effff640 1 --2- 172.21.15.105:0/2103442585 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3e40668d0 0x7fe3e4068d90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe3f815aa00 tx=0x7fe3e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:22.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe401f9c000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d492d40 0x564e8d493120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe401f9c000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d493660 0x564e8d445fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe401f9c000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d446510 0x564e8d4468f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe401f9c000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e8d30f120 con 0x564e8d493660 2023-11-28T18:29:22.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe401f9c000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e8d27f9a0 con 0x564e8d446510 2023-11-28T18:29:22.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe401f9c000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e8d27ced0 con 0x564e8d492d40 2023-11-28T18:29:22.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe3effff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d493660 0x564e8d445fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe3fc9ce640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d492d40 0x564e8d493120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe3effff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d493660 0x564e8d445fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46618/0 (socket says 172.21.15.105:46618) 2023-11-28T18:29:22.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe3effff640 1 -- 172.21.15.105:0/1371370654 learned_addr learned my addr 172.21.15.105:0/1371370654 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:22.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe3fd1cf640 1 --2- 172.21.15.105:0/1371370654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d446510 0x564e8d4468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe3fc9ce640 1 -- 172.21.15.105:0/1371370654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d446510 msgr2=0x564e8d4468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:22.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe3fc9ce640 1 --2- 172.21.15.105:0/1371370654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d446510 0x564e8d4468f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe3fc9ce640 1 -- 172.21.15.105:0/1371370654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d493660 msgr2=0x564e8d445fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:22.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe3fc9ce640 1 --2- 172.21.15.105:0/1371370654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d493660 0x564e8d445fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe3fc9ce640 1 -- 172.21.15.105:0/1371370654 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e8cf70890 con 0x564e8d492d40 2023-11-28T18:29:22.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe3effff640 1 --2- 172.21.15.105:0/1371370654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d493660 0x564e8d445fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:29:22.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe3fc9ce640 1 --2- 172.21.15.105:0/1371370654 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d492d40 0x564e8d493120 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4082e50 tx=0x7fe3f40557f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=146ae27b18cabf26 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:22.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe3de7fc640 1 -- 172.21.15.105:0/1371370654 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f405dad0 con 0x564e8d492d40 2023-11-28T18:29:22.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe3de7fc640 1 -- 172.21.15.105:0/1371370654 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f4053b90 con 0x564e8d492d40 2023-11-28T18:29:22.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe3de7fc640 1 -- 172.21.15.105:0/1371370654 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f405d640 con 0x564e8d492d40 2023-11-28T18:29:22.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe401f9c000 1 -- 172.21.15.105:0/1371370654 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d492d40 msgr2=0x564e8d493120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:22.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe401f9c000 1 --2- 172.21.15.105:0/1371370654 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d492d40 0x564e8d493120 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4082e50 tx=0x7fe3f40557f0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe401f9c000 1 -- 172.21.15.105:0/1371370654 shutdown_connections 2023-11-28T18:29:22.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe401f9c000 1 --2- 172.21.15.105:0/1371370654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d446510 0x564e8d4468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe401f9c000 1 --2- 172.21.15.105:0/1371370654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d493660 0x564e8d445fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe401f9c000 1 --2- 172.21.15.105:0/1371370654 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d492d40 0x564e8d493120 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe401f9c000 1 -- 172.21.15.105:0/1371370654 >> 172.21.15.105:0/1371370654 conn(0x564e8d4738d0 msgr2=0x564e8d475cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:22.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe401f9c000 1 -- 172.21.15.105:0/1371370654 shutdown_connections 2023-11-28T18:29:22.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe401f9c000 1 -- 172.21.15.105:0/1371370654 wait complete. 2023-11-28T18:29:22.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe401f9c000 1 -- start start 2023-11-28T18:29:22.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.813+0000 7fe401f9c000 1 Processor -- start 2023-11-28T18:29:22.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe401f9c000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d446510 0x564e8d49e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe401f9c000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d492d40 0x564e8d49e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe401f9c000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d493660 0x564e8d4655e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe401f9c000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e8d2787c0 con 0x564e8d492d40 2023-11-28T18:29:22.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe401f9c000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e8d230bf0 con 0x564e8d493660 2023-11-28T18:29:22.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe401f9c000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e8d236a90 con 0x564e8d446510 2023-11-28T18:29:22.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe401f9c000 10 client.0 _ll_register_callbacks cb 0x564e8d433e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:29:22.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe3effff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d492d40 0x564e8d49e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe3effff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d492d40 0x564e8d49e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46630/0 (socket says 172.21.15.105:46630) 2023-11-28T18:29:22.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe3effff640 1 -- 172.21.15.105:0/3382815409 learned_addr learned my addr 172.21.15.105:0/3382815409 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:22.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe3fc9ce640 1 --2- 172.21.15.105:0/3382815409 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d446510 0x564e8d49e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe3effff640 1 -- 172.21.15.105:0/3382815409 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d446510 msgr2=0x564e8d49e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:22.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe3effff640 1 --2- 172.21.15.105:0/3382815409 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d446510 0x564e8d49e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe3fd1cf640 1 --2- 172.21.15.105:0/3382815409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d493660 0x564e8d4655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe3effff640 1 -- 172.21.15.105:0/3382815409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d493660 msgr2=0x564e8d4655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:22.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe3effff640 1 --2- 172.21.15.105:0/3382815409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d493660 0x564e8d4655e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe3effff640 1 -- 172.21.15.105:0/3382815409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e800e040 con 0x564e8d492d40 2023-11-28T18:29:22.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe3fc9ce640 1 --2- 172.21.15.105:0/3382815409 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d446510 0x564e8d49e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:22.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe3fd1cf640 1 --2- 172.21.15.105:0/3382815409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d493660 0x564e8d4655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:22.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe3effff640 1 --2- 172.21.15.105:0/3382815409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d492d40 0x564e8d49e620 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8016800 tx=0x7fe3e80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b690813cb8c55b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:22.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe3d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:29:22.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe3d77fe640 1 -- 172.21.15.105:0/3382815409 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8004020 con 0x564e8d492d40 2023-11-28T18:29:22.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe401f9c000 1 -- 172.21.15.105:0/3382815409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e8d23a470 con 0x564e8d492d40 2023-11-28T18:29:22.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe3d77fe640 1 -- 172.21.15.105:0/3382815409 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fe3e8004180 con 0x564e8d492d40 2023-11-28T18:29:22.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe3d77fe640 1 -- 172.21.15.105:0/3382815409 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3e8004340 con 0x564e8d492d40 2023-11-28T18:29:22.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe401f9c000 10 client.7878 fetch_fsmap learned FSMap version 92 2023-11-28T18:29:22.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe401f9c000 1 -- 172.21.15.105:0/3382815409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) v3 -- 0x7fe3f814dc90 con 0x564e8d492d40 2023-11-28T18:29:22.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe3d77fe640 1 -- 172.21.15.105:0/3382815409 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564e8d23a470 con 0x564e8d492d40 2023-11-28T18:29:22.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe3d77fe640 1 -- 172.21.15.105:0/3382815409 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 92) v1 ==== 3318+0+0 (secure 0 0 0) 0x7fe3e80149b0 con 0x564e8d492d40 2023-11-28T18:29:22.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe401f9c000 10 client.7878 fetch_fsmap finished waiting for FSMap version 92 2023-11-28T18:29:22.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe401f9c000 10 client.7878 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T18:29:22.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe401f9c000 20 client.7878 populate_metadata read hostname 'smithi105' 2023-11-28T18:29:22.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe401f9c000 1 --2- 172.21.15.105:0/3382815409 >> [v2:172.21.15.148:6833/3461832163,v1:172.21.15.148:6835/3461832163] conn(0x564e8d466380 0x564e8d466760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe401f9c000 4 client.7878 mds_command: new command op to 6485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:29:22.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe401f9c000 1 -- 172.21.15.105:0/3382815409 --> [v2:172.21.15.148:6833/3461832163,v1:172.21.15.148:6835/3461832163] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e8d23db50 con 0x564e8d466380 2023-11-28T18:29:22.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe3fc9ce640 1 --2- 172.21.15.105:0/3382815409 >> [v2:172.21.15.148:6833/3461832163,v1:172.21.15.148:6835/3461832163] conn(0x564e8d466380 0x564e8d466760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe3d77fe640 10 client.7878 ms_handle_connect on v2:172.21.15.148:6833/3461832163 2023-11-28T18:29:22.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.817+0000 7fe3fc9ce640 1 --2- 172.21.15.105:0/3382815409 >> [v2:172.21.15.148:6833/3461832163,v1:172.21.15.148:6835/3461832163] conn(0x564e8d466380 0x564e8d466760 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=38b181b738b122ff server_cookie=cfb3c93c09f544e2 in_seq=0 out_seq=0 2023-11-28T18:29:22.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.821+0000 7fe3d77fe640 1 -- 172.21.15.105:0/3382815409 <== mds.2 v2:172.21.15.148:6833/3461832163 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564e8d23db50 con 0x564e8d466380 2023-11-28T18:29:22.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.821+0000 7fe3d77fe640 10 client.7878 handle_command_reply: tid=0 2023-11-28T18:29:22.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.821+0000 7fe401f9c000 1 client.7878 shutdown 2023-11-28T18:29:22.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.821+0000 7fe401f9c000 1 -- 172.21.15.105:0/3382815409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d492d40 msgr2=0x564e8d49e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:22.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.821+0000 7fe401f9c000 1 --2- 172.21.15.105:0/3382815409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d492d40 0x564e8d49e620 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8016800 tx=0x7fe3e80046d0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.821+0000 7fe401f9c000 1 -- 172.21.15.105:0/3382815409 shutdown_connections 2023-11-28T18:29:22.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.821+0000 7fe401f9c000 1 --2- 172.21.15.105:0/3382815409 >> [v2:172.21.15.148:6833/3461832163,v1:172.21.15.148:6835/3461832163] conn(0x564e8d466380 0x564e8d466760 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.821+0000 7fe401f9c000 1 --2- 172.21.15.105:0/3382815409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d493660 0x564e8d4655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.821+0000 7fe401f9c000 1 --2- 172.21.15.105:0/3382815409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d492d40 0x564e8d49e620 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.821+0000 7fe401f9c000 1 --2- 172.21.15.105:0/3382815409 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d446510 0x564e8d49e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.821+0000 7fe3d77fe640 0 client.7878 ms_handle_reset on v2:172.21.15.148:6833/3461832163 2023-11-28T18:29:22.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.821+0000 7fe401f9c000 1 -- 172.21.15.105:0/3382815409 >> 172.21.15.105:0/3382815409 conn(0x564e8d4738d0 msgr2=0x564e8d475cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:22.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.821+0000 7fe401f9c000 1 -- 172.21.15.105:0/3382815409 shutdown_connections 2023-11-28T18:29:22.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.821+0000 7fe401f9c000 1 -- 172.21.15.105:0/3382815409 wait complete. 2023-11-28T18:29:22.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.821+0000 7fe401f9c000 20 client.7878 trim_cache size 0 max 16384 2023-11-28T18:29:22.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe401f9c000 1 Processor -- start 2023-11-28T18:29:22.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe401f9c000 1 -- start start 2023-11-28T18:29:22.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe401f9c000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d446510 0x564e8d4aa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe401f9c000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d492d40 0x564e8d4aa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe401f9c000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d493660 0x564e8d4aace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe401f9c000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e8cf31940 con 0x564e8d493660 2023-11-28T18:29:22.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe401f9c000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e8d27c7c0 con 0x564e8d446510 2023-11-28T18:29:22.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe401f9c000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e8d450520 con 0x564e8d492d40 2023-11-28T18:29:22.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe3fd1cf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d493660 0x564e8d4aace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe3fd1cf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d493660 0x564e8d4aace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46634/0 (socket says 172.21.15.105:46634) 2023-11-28T18:29:22.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe3fd1cf640 1 -- 172.21.15.105:0/2193325598 learned_addr learned my addr 172.21.15.105:0/2193325598 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe3effff640 1 --2- 172.21.15.105:0/2193325598 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d492d40 0x564e8d4aa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe3fd1cf640 1 -- 172.21.15.105:0/2193325598 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d492d40 msgr2=0x564e8d4aa7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe3fd1cf640 1 --2- 172.21.15.105:0/2193325598 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d492d40 0x564e8d4aa7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe3fd1cf640 1 -- 172.21.15.105:0/2193325598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d446510 msgr2=0x564e8d4aa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe3fc9ce640 1 --2- 172.21.15.105:0/2193325598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d446510 0x564e8d4aa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe3fd1cf640 1 --2- 172.21.15.105:0/2193325598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d446510 0x564e8d4aa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe3fd1cf640 1 -- 172.21.15.105:0/2193325598 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e8d239190 con 0x564e8d493660 2023-11-28T18:29:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe3fc9ce640 1 --2- 172.21.15.105:0/2193325598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d446510 0x564e8d4aa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe3effff640 1 --2- 172.21.15.105:0/2193325598 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d492d40 0x564e8d4aa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe3fd1cf640 1 --2- 172.21.15.105:0/2193325598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d493660 0x564e8d4aace0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fe3f001c9e0 tx=0x7fe3f00a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab48cf65c53a245 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe3deffd640 1 -- 172.21.15.105:0/2193325598 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f00a5c00 con 0x564e8d493660 2023-11-28T18:29:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe3deffd640 1 -- 172.21.15.105:0/2193325598 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f00a5d60 con 0x564e8d493660 2023-11-28T18:29:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe3deffd640 1 -- 172.21.15.105:0/2193325598 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f00c0620 con 0x564e8d493660 2023-11-28T18:29:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe401f9c000 1 -- 172.21.15.105:0/2193325598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d493660 msgr2=0x564e8d4aace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:22.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe401f9c000 1 --2- 172.21.15.105:0/2193325598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d493660 0x564e8d4aace0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fe3f001c9e0 tx=0x7fe3f00a7070 comp rx=0 tx=0).stop 2023-11-28T18:29:22.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe401f9c000 1 -- 172.21.15.105:0/2193325598 shutdown_connections 2023-11-28T18:29:22.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe401f9c000 1 --2- 172.21.15.105:0/2193325598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d493660 0x564e8d4aace0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe401f9c000 1 --2- 172.21.15.105:0/2193325598 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d492d40 0x564e8d4aa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe401f9c000 1 --2- 172.21.15.105:0/2193325598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d446510 0x564e8d4aa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe401f9c000 1 -- 172.21.15.105:0/2193325598 >> 172.21.15.105:0/2193325598 conn(0x564e8d4738d0 msgr2=0x564e8d444fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:22.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe401f9c000 1 -- 172.21.15.105:0/2193325598 shutdown_connections 2023-11-28T18:29:22.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.837+0000 7fe401f9c000 1 -- 172.21.15.105:0/2193325598 wait complete. 2023-11-28T18:29:22.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe401f9c000 1 -- start start 2023-11-28T18:29:22.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe401f9c000 1 Processor -- start 2023-11-28T18:29:22.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe401f9c000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d446510 0x564e8d4f7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe401f9c000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d492d40 0x564e8d4f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe401f9c000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d493660 0x564e8d4b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe401f9c000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e8d275fd0 con 0x564e8d493660 2023-11-28T18:29:22.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe401f9c000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e8d317580 con 0x564e8d446510 2023-11-28T18:29:22.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe401f9c000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e8d274130 con 0x564e8d492d40 2023-11-28T18:29:22.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe401f9c000 10 client.0 _ll_register_callbacks cb 0x564e8d30f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:29:22.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe3fd1cf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d493660 0x564e8d4b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe3fd1cf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d493660 0x564e8d4b2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46636/0 (socket says 172.21.15.105:46636) 2023-11-28T18:29:22.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe3fd1cf640 1 -- 172.21.15.105:0/2433341014 learned_addr learned my addr 172.21.15.105:0/2433341014 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:22.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe3fc9ce640 1 --2- 172.21.15.105:0/2433341014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d446510 0x564e8d4f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe3effff640 1 --2- 172.21.15.105:0/2433341014 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d492d40 0x564e8d4f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe3fd1cf640 1 -- 172.21.15.105:0/2433341014 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d492d40 msgr2=0x564e8d4f7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:22.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe3fd1cf640 1 --2- 172.21.15.105:0/2433341014 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d492d40 0x564e8d4f7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe3fd1cf640 1 -- 172.21.15.105:0/2433341014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d446510 msgr2=0x564e8d4f7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:22.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe3fd1cf640 1 --2- 172.21.15.105:0/2433341014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d446510 0x564e8d4f7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe3fd1cf640 1 -- 172.21.15.105:0/2433341014 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f8071df0 con 0x564e8d493660 2023-11-28T18:29:22.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe3fc9ce640 1 --2- 172.21.15.105:0/2433341014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d446510 0x564e8d4f7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:22.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe3fd1cf640 1 --2- 172.21.15.105:0/2433341014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d493660 0x564e8d4b2790 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00c2040 tx=0x7fe3f001b1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6f58c912a0e0b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:22.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe3d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:29:22.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe3d77fe640 1 -- 172.21.15.105:0/2433341014 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0003c60 con 0x564e8d493660 2023-11-28T18:29:22.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe3d77fe640 1 -- 172.21.15.105:0/2433341014 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fe3f00bb070 con 0x564e8d493660 2023-11-28T18:29:22.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe401f9c000 1 -- 172.21.15.105:0/2433341014 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e8d473760 con 0x564e8d493660 2023-11-28T18:29:22.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe3d77fe640 1 -- 172.21.15.105:0/2433341014 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3f0003dc0 con 0x564e8d493660 2023-11-28T18:29:22.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe401f9c000 10 client.7884 fetch_fsmap learned FSMap version 92 2023-11-28T18:29:22.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe401f9c000 1 -- 172.21.15.105:0/2433341014 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) v3 -- 0x564e8d21f490 con 0x564e8d493660 2023-11-28T18:29:22.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe3d77fe640 1 -- 172.21.15.105:0/2433341014 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564e8d473760 con 0x564e8d493660 2023-11-28T18:29:22.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe3d77fe640 1 -- 172.21.15.105:0/2433341014 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 92) v1 ==== 3318+0+0 (secure 0 0 0) 0x7fe3f00d43f0 con 0x564e8d493660 2023-11-28T18:29:22.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe401f9c000 10 client.7884 fetch_fsmap finished waiting for FSMap version 92 2023-11-28T18:29:22.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe401f9c000 10 client.7884 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T18:29:22.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe401f9c000 20 client.7884 populate_metadata read hostname 'smithi105' 2023-11-28T18:29:22.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe401f9c000 1 --2- 172.21.15.105:0/2433341014 >> [v2:172.21.15.148:6833/3461832163,v1:172.21.15.148:6835/3461832163] conn(0x564e8d4b3150 0x564e8d4b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:22.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe401f9c000 4 client.7884 mds_command: new command op to 6485 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:29:22.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe401f9c000 1 -- 172.21.15.105:0/2433341014 --> [v2:172.21.15.148:6833/3461832163,v1:172.21.15.148:6835/3461832163] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564e8d277110 con 0x564e8d4b3150 2023-11-28T18:29:22.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.841+0000 7fe3fc9ce640 1 --2- 172.21.15.105:0/2433341014 >> [v2:172.21.15.148:6833/3461832163,v1:172.21.15.148:6835/3461832163] conn(0x564e8d4b3150 0x564e8d4b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:22.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe3fc9ce640 1 --2- 172.21.15.105:0/2433341014 >> [v2:172.21.15.148:6833/3461832163,v1:172.21.15.148:6835/3461832163] conn(0x564e8d4b3150 0x564e8d4b3530 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6758e7025dea7ed4 server_cookie=f5052686f638057d in_seq=0 out_seq=0 2023-11-28T18:29:22.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe3d77fe640 10 client.7884 ms_handle_connect on v2:172.21.15.148:6833/3461832163 2023-11-28T18:29:22.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe3d77fe640 1 -- 172.21.15.105:0/2433341014 <== mds.2 v2:172.21.15.148:6833/3461832163 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x564e8d277110 con 0x564e8d4b3150 2023-11-28T18:29:22.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe3d77fe640 10 client.7884 handle_command_reply: tid=0 2023-11-28T18:29:22.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe401f9c000 1 client.7884 shutdown 2023-11-28T18:29:22.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe401f9c000 1 -- 172.21.15.105:0/2433341014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d493660 msgr2=0x564e8d4b2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:22.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe401f9c000 1 --2- 172.21.15.105:0/2433341014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d493660 0x564e8d4b2790 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00c2040 tx=0x7fe3f001b1e0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe401f9c000 1 -- 172.21.15.105:0/2433341014 shutdown_connections 2023-11-28T18:29:22.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe401f9c000 1 --2- 172.21.15.105:0/2433341014 >> [v2:172.21.15.148:6833/3461832163,v1:172.21.15.148:6835/3461832163] conn(0x564e8d4b3150 0x564e8d4b3530 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe401f9c000 1 --2- 172.21.15.105:0/2433341014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e8d493660 0x564e8d4b2790 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe401f9c000 1 --2- 172.21.15.105:0/2433341014 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564e8d492d40 0x564e8d4f7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe401f9c000 1 --2- 172.21.15.105:0/2433341014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e8d446510 0x564e8d4f7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe3d77fe640 0 client.7884 ms_handle_reset on v2:172.21.15.148:6833/3461832163 2023-11-28T18:29:22.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe401f9c000 1 -- 172.21.15.105:0/2433341014 >> 172.21.15.105:0/2433341014 conn(0x564e8d4738d0 msgr2=0x564e8d445870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:22.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe401f9c000 1 -- 172.21.15.105:0/2433341014 shutdown_connections 2023-11-28T18:29:22.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe401f9c000 1 -- 172.21.15.105:0/2433341014 wait complete. 2023-11-28T18:29:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe401f9c000 20 client.7884 trim_cache size 0 max 16384 2023-11-28T18:29:22.855 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:29:22.855 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:29:22.855 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 2, 2023-11-28T18:29:22.855 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 6485, 2023-11-28T18:29:22.855 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:29:22.855 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:29:22.855 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:29:22.855 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 1.736918961, 2023-11-28T18:29:22.856 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 92, 2023-11-28T18:29:22.856 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 92, 2023-11-28T18:29:22.856 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 92, 2023-11-28T18:29:22.856 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 24.394036437 2023-11-28T18:29:22.856 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:29:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe3fec32640 1 -- 172.21.15.105:0/2103442585 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3e40668d0 msgr2=0x7fe3e4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe3fec32640 1 --2- 172.21.15.105:0/2103442585 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3e40668d0 0x7fe3e4068d90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe3f815aa00 tx=0x7fe3e8008040 comp rx=0 tx=0).stop 2023-11-28T18:29:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe3fec32640 1 -- 172.21.15.105:0/2103442585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f80b4290 msgr2=0x7fe3f814cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe3fec32640 1 --2- 172.21.15.105:0/2103442585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f80b4290 0x7fe3f814cd10 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400d800 tx=0x7fe3f400dcc0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe3fec32640 1 -- 172.21.15.105:0/2103442585 shutdown_connections 2023-11-28T18:29:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe3fec32640 1 --2- 172.21.15.105:0/2103442585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3f8137ef0 0x7fe3f806c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe3fec32640 1 --2- 172.21.15.105:0/2103442585 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3f815b030 0x7fe3f814a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe3fec32640 1 --2- 172.21.15.105:0/2103442585 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3e40668d0 0x7fe3e4068d90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe3fec32640 1 --2- 172.21.15.105:0/2103442585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f80b4290 0x7fe3f814cd10 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe3fec32640 1 -- 172.21.15.105:0/2103442585 >> 172.21.15.105:0/2103442585 conn(0x7fe3f80ceb90 msgr2=0x7fe3f8153b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:22.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe3fec32640 1 -- 172.21.15.105:0/2103442585 shutdown_connections 2023-11-28T18:29:22.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:22.845+0000 7fe3fec32640 1 -- 172.21.15.105:0/2103442585 wait complete. 2023-11-28T18:29:22.869 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T18:29:23.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.001+0000 7f2ed89b8640 1 Processor -- start 2023-11-28T18:29:23.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.001+0000 7f2ed89b8640 1 -- start start 2023-11-28T18:29:23.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.005+0000 7f2ed89b8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ed406bf50 0x7f2ed415d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.005+0000 7f2ed89b8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2ed40adb80 0x7f2ed40adff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.005+0000 7f2ed89b8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ed40cc7f0 con 0x7f2ed4154070 2023-11-28T18:29:23.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.005+0000 7f2ed89b8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ed4072540 con 0x7f2ed40adb80 2023-11-28T18:29:23.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.005+0000 7f2ed89b8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed40ae530 con 0x7f2ed406bf50 2023-11-28T18:29:23.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.005+0000 7f2ed259c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f2ed4154070 0x7f2ed40ce310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40196/0 (socket says 172.21.15.105:40196) 2023-11-28T18:29:23.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.005+0000 7f2ed259c640 1 -- 172.21.15.105:0/2431314774 learned_addr learned my addr 172.21.15.105:0/2431314774 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:23.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.005+0000 7f2ecbfff640 1 --2- 172.21.15.105:0/2431314774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ed406bf50 0x7f2ed415d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.005+0000 7f2ed2d9d640 1 --2- 172.21.15.105:0/2431314774 >> v2:172.21.15.148:3300/0 conn(0x7f2ed40adb80 0x7f2ed40adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.005+0000 7f2ed1d9b640 1 -- 172.21.15.105:0/2431314774 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1627169230 0 0) 0x7f2ed40cc7f0 con 0x7f2ed4154070 2023-11-28T18:29:23.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.005+0000 7f2ed1d9b640 1 -- 172.21.15.105:0/2431314774 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ec0003530 con 0x7f2ed4154070 2023-11-28T18:29:23.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.005+0000 7f2ed1d9b640 1 -- 172.21.15.105:0/2431314774 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2179082085 0 0) 0x7f2ec0003530 con 0x7f2ed4154070 2023-11-28T18:29:23.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.005+0000 7f2ecbfff640 1 -- 172.21.15.105:0/2431314774 >> v2:172.21.15.148:3300/0 conn(0x7f2ed40adb80 msgr2=0x7f2ed40adff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:23.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.005+0000 7f2ecbfff640 1 --2- 172.21.15.105:0/2431314774 >> v2:172.21.15.148:3300/0 conn(0x7f2ed40adb80 0x7f2ed40adff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.005+0000 7f2ecbfff640 1 -- 172.21.15.105:0/2431314774 >> v1:172.21.15.105:6789/0 conn(0x7f2ed4154070 legacy=0x7f2ed40ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:23.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.005+0000 7f2ecbfff640 1 -- 172.21.15.105:0/2431314774 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed415dd10 con 0x7f2ed406bf50 2023-11-28T18:29:23.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.005+0000 7f2ed2d9d640 1 --2- 172.21.15.105:0/2431314774 >> v2:172.21.15.148:3300/0 conn(0x7f2ed40adb80 0x7f2ed40adff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:29:23.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.009+0000 7f2ecbfff640 1 --2- 172.21.15.105:0/2431314774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ed406bf50 0x7f2ed415d700 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0097c0 tx=0x7f2ebc01c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=689c235ea33d485d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:23.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.009+0000 7f2ed1d9b640 1 -- 172.21.15.105:0/2431314774 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4001680 con 0x7f2ed406bf50 2023-11-28T18:29:23.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.009+0000 7f2ed1d9b640 1 -- 172.21.15.105:0/2431314774 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ebc0a4c90 con 0x7f2ed406bf50 2023-11-28T18:29:23.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.009+0000 7f2ed1d9b640 1 -- 172.21.15.105:0/2431314774 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc0ac440 con 0x7f2ed406bf50 2023-11-28T18:29:23.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.009+0000 7f2ed89b8640 1 -- 172.21.15.105:0/2431314774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ed406bf50 msgr2=0x7f2ed415d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:23.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.009+0000 7f2ed89b8640 1 --2- 172.21.15.105:0/2431314774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ed406bf50 0x7f2ed415d700 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0097c0 tx=0x7f2ebc01c370 comp rx=0 tx=0).stop 2023-11-28T18:29:23.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.009+0000 7f2ed89b8640 1 -- 172.21.15.105:0/2431314774 shutdown_connections 2023-11-28T18:29:23.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.009+0000 7f2ed89b8640 1 --2- 172.21.15.105:0/2431314774 >> v2:172.21.15.148:3300/0 conn(0x7f2ed40adb80 0x7f2ed40adff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.009+0000 7f2ed89b8640 1 --2- 172.21.15.105:0/2431314774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ed406bf50 0x7f2ed415d700 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.009+0000 7f2ed89b8640 1 -- 172.21.15.105:0/2431314774 >> 172.21.15.105:0/2431314774 conn(0x7f2ed40ceb90 msgr2=0x7f2ed40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:23.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.009+0000 7f2ed89b8640 1 -- 172.21.15.105:0/2431314774 shutdown_connections 2023-11-28T18:29:23.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.009+0000 7f2ed89b8640 1 -- 172.21.15.105:0/2431314774 wait complete. 2023-11-28T18:29:23.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.009+0000 7f2ed89b8640 1 Processor -- start 2023-11-28T18:29:23.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.009+0000 7f2ed89b8640 1 -- start start 2023-11-28T18:29:23.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ed89b8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ed406bf50 0x7f2ed4144d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ed89b8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ed40adb80 0x7f2ed40b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ed89b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed4154070 0x7f2ed40abd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ed89b8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed4145260 con 0x7f2ed406bf50 2023-11-28T18:29:23.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ed89b8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed4152980 con 0x7f2ed4154070 2023-11-28T18:29:23.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ed89b8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed4152c60 con 0x7f2ed40adb80 2023-11-28T18:29:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ed259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ed406bf50 0x7f2ed4144d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ed259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ed406bf50 0x7f2ed4144d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46646/0 (socket says 172.21.15.105:46646) 2023-11-28T18:29:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ed259c640 1 -- 172.21.15.105:0/2676213291 learned_addr learned my addr 172.21.15.105:0/2676213291 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ecbfff640 1 --2- 172.21.15.105:0/2676213291 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ed40adb80 0x7f2ed40b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ed2d9d640 1 --2- 172.21.15.105:0/2676213291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed4154070 0x7f2ed40abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ed259c640 1 -- 172.21.15.105:0/2676213291 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ed40adb80 msgr2=0x7f2ed40b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ed259c640 1 --2- 172.21.15.105:0/2676213291 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ed40adb80 0x7f2ed40b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ed259c640 1 -- 172.21.15.105:0/2676213291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed4154070 msgr2=0x7f2ed40abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ed259c640 1 --2- 172.21.15.105:0/2676213291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed4154070 0x7f2ed40abd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ed259c640 1 -- 172.21.15.105:0/2676213291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ecc009a70 con 0x7f2ed406bf50 2023-11-28T18:29:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ed2d9d640 1 --2- 172.21.15.105:0/2676213291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed4154070 0x7f2ed40abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:23.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ed259c640 1 --2- 172.21.15.105:0/2676213291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ed406bf50 0x7f2ed4144d20 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400c750 tx=0x7f2ec400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:23.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ecaffd640 1 -- 172.21.15.105:0/2676213291 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4005ae0 con 0x7f2ed406bf50 2023-11-28T18:29:23.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ecaffd640 1 -- 172.21.15.105:0/2676213291 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec4005c40 con 0x7f2ed406bf50 2023-11-28T18:29:23.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ed89b8640 1 -- 172.21.15.105:0/2676213291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ebc009420 con 0x7f2ed406bf50 2023-11-28T18:29:23.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ecaffd640 1 -- 172.21.15.105:0/2676213291 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4007070 con 0x7f2ed406bf50 2023-11-28T18:29:23.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.013+0000 7f2ed89b8640 1 -- 172.21.15.105:0/2676213291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed4153220 con 0x7f2ed406bf50 2023-11-28T18:29:23.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2ecaffd640 1 -- 172.21.15.105:0/2676213291 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f2ec40071f0 con 0x7f2ed406bf50 2023-11-28T18:29:23.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2ecaffd640 1 --2- 172.21.15.105:0/2676213291 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2eb40668d0 0x7f2eb4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2ecaffd640 1 -- 172.21.15.105:0/2676213291 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f2ec40843f0 con 0x7f2ed406bf50 2023-11-28T18:29:23.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2ecbfff640 1 --2- 172.21.15.105:0/2676213291 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2eb40668d0 0x7f2eb4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2edbd22000 1 Processor -- start 2023-11-28T18:29:23.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2ecbfff640 1 --2- 172.21.15.105:0/2676213291 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2eb40668d0 0x7f2eb4068d90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0ae000 tx=0x7f2ebc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:23.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2edbd22000 1 -- start start 2023-11-28T18:29:23.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2edbd22000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b065ad30 0x5586b065b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2edbd22000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b065b650 0x5586b060dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2edbd22000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b060e500 0x5586b060e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2edbd22000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586b04d7110 con 0x5586b065b650 2023-11-28T18:29:23.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2edbd22000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586b04679a0 con 0x5586b060e500 2023-11-28T18:29:23.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2edbd22000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586b0464ed0 con 0x5586b065ad30 2023-11-28T18:29:23.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2ecbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b065b650 0x5586b060dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2ecbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b065b650 0x5586b060dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46658/0 (socket says 172.21.15.105:46658) 2023-11-28T18:29:23.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2ecbfff640 1 -- 172.21.15.105:0/3734027299 learned_addr learned my addr 172.21.15.105:0/3734027299 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:23.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2ed259c640 1 --2- 172.21.15.105:0/3734027299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b065ad30 0x5586b065b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2ed2d9d640 1 --2- 172.21.15.105:0/3734027299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b060e500 0x5586b060e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2ecbfff640 1 -- 172.21.15.105:0/3734027299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b065ad30 msgr2=0x5586b065b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:23.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2ecbfff640 1 --2- 172.21.15.105:0/3734027299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b065ad30 0x5586b065b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2ecbfff640 1 -- 172.21.15.105:0/3734027299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b060e500 msgr2=0x5586b060e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:23.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2ecbfff640 1 --2- 172.21.15.105:0/3734027299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b060e500 0x5586b060e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2ecbfff640 1 -- 172.21.15.105:0/3734027299 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586b0158890 con 0x5586b065b650 2023-11-28T18:29:23.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.017+0000 7f2ed259c640 1 --2- 172.21.15.105:0/3734027299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b065ad30 0x5586b065b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:23.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2ecbfff640 1 --2- 172.21.15.105:0/3734027299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b065b650 0x5586b060dfc0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0af040 tx=0x7f2ebc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152ad6ae8a3db14e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:23.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2eb3fff640 1 -- 172.21.15.105:0/3734027299 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc0acd50 con 0x5586b065b650 2023-11-28T18:29:23.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2eb3fff640 1 -- 172.21.15.105:0/3734027299 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ebc0bd070 con 0x5586b065b650 2023-11-28T18:29:23.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2eb3fff640 1 -- 172.21.15.105:0/3734027299 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc0c3630 con 0x5586b065b650 2023-11-28T18:29:23.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2edbd22000 1 -- 172.21.15.105:0/3734027299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b065b650 msgr2=0x5586b060dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:23.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2edbd22000 1 --2- 172.21.15.105:0/3734027299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b065b650 0x5586b060dfc0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0af040 tx=0x7f2ebc004740 comp rx=0 tx=0).stop 2023-11-28T18:29:23.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2edbd22000 1 -- 172.21.15.105:0/3734027299 shutdown_connections 2023-11-28T18:29:23.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2edbd22000 1 --2- 172.21.15.105:0/3734027299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b060e500 0x5586b060e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2edbd22000 1 --2- 172.21.15.105:0/3734027299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b065b650 0x5586b060dfc0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2edbd22000 1 --2- 172.21.15.105:0/3734027299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b065ad30 0x5586b065b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2edbd22000 1 -- 172.21.15.105:0/3734027299 >> 172.21.15.105:0/3734027299 conn(0x5586b063b8c0 msgr2=0x5586b063dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:23.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2edbd22000 1 -- 172.21.15.105:0/3734027299 shutdown_connections 2023-11-28T18:29:23.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2edbd22000 1 -- 172.21.15.105:0/3734027299 wait complete. 2023-11-28T18:29:23.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2edbd22000 1 -- start start 2023-11-28T18:29:23.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2edbd22000 1 Processor -- start 2023-11-28T18:29:23.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2edbd22000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b060e500 0x5586b0614520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2edbd22000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b065ad30 0x5586b0614a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2edbd22000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b065b650 0x5586b0614fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2edbd22000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586b04607c0 con 0x5586b060e500 2023-11-28T18:29:23.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2edbd22000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586b0418bf0 con 0x5586b065ad30 2023-11-28T18:29:23.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2edbd22000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586b041ea90 con 0x5586b065b650 2023-11-28T18:29:23.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2edbd22000 10 client.0 _ll_register_callbacks cb 0x5586b05fbe20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:29:23.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2ed259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b060e500 0x5586b0614520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2ed259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b060e500 0x5586b0614520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46664/0 (socket says 172.21.15.105:46664) 2023-11-28T18:29:23.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2ed259c640 1 -- 172.21.15.105:0/646611838 learned_addr learned my addr 172.21.15.105:0/646611838 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:23.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2ecbfff640 1 --2- 172.21.15.105:0/646611838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b065ad30 0x5586b0614a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.021+0000 7f2ed2d9d640 1 --2- 172.21.15.105:0/646611838 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b065b650 0x5586b0614fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2ed259c640 1 -- 172.21.15.105:0/646611838 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b065b650 msgr2=0x5586b0614fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:23.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2ed259c640 1 --2- 172.21.15.105:0/646611838 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b065b650 0x5586b0614fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2ed259c640 1 -- 172.21.15.105:0/646611838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b065ad30 msgr2=0x5586b0614a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:23.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2ed259c640 1 --2- 172.21.15.105:0/646611838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b065ad30 0x5586b0614a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2ed259c640 1 -- 172.21.15.105:0/646611838 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed415dd10 con 0x5586b060e500 2023-11-28T18:29:23.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2ed259c640 1 --2- 172.21.15.105:0/646611838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b060e500 0x5586b0614520 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4051630 tx=0x7f2ec4059280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be84859ce2b0c6ff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:23.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2eb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:29:23.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2eb17fa640 1 -- 172.21.15.105:0/646611838 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4054620 con 0x5586b060e500 2023-11-28T18:29:23.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2eb17fa640 1 -- 172.21.15.105:0/646611838 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f2ec4054780 con 0x5586b060e500 2023-11-28T18:29:23.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2edbd22000 1 -- 172.21.15.105:0/646611838 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586b0422470 con 0x5586b060e500 2023-11-28T18:29:23.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2eb17fa640 1 -- 172.21.15.105:0/646611838 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ec4082220 con 0x5586b060e500 2023-11-28T18:29:23.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2edbd22000 10 client.7896 fetch_fsmap learned FSMap version 92 2023-11-28T18:29:23.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2edbd22000 1 -- 172.21.15.105:0/646611838 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) v3 -- 0x5586b0421190 con 0x5586b060e500 2023-11-28T18:29:23.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2eb17fa640 1 -- 172.21.15.105:0/646611838 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5586b0422470 con 0x5586b060e500 2023-11-28T18:29:23.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2eb17fa640 1 -- 172.21.15.105:0/646611838 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 92) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f2ec4019020 con 0x5586b060e500 2023-11-28T18:29:23.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2edbd22000 10 client.7896 fetch_fsmap finished waiting for FSMap version 92 2023-11-28T18:29:23.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2edbd22000 10 client.7896 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:29:23.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2edbd22000 20 client.7896 populate_metadata read hostname 'smithi105' 2023-11-28T18:29:23.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2edbd22000 1 --2- 172.21.15.105:0/646611838 >> [v2:172.21.15.105:6834/4162499249,v1:172.21.15.105:6835/4162499249] conn(0x5586b06159b0 0x5586b066cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2edbd22000 4 client.7896 mds_command: new command op to 7665 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:29:23.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2edbd22000 1 -- 172.21.15.105:0/646611838 --> [v2:172.21.15.105:6834/4162499249,v1:172.21.15.105:6835/4162499249] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586b0425b50 con 0x5586b06159b0 2023-11-28T18:29:23.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2ecbfff640 1 --2- 172.21.15.105:0/646611838 >> [v2:172.21.15.105:6834/4162499249,v1:172.21.15.105:6835/4162499249] conn(0x5586b06159b0 0x5586b066cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2ecbfff640 1 --2- 172.21.15.105:0/646611838 >> [v2:172.21.15.105:6834/4162499249,v1:172.21.15.105:6835/4162499249] conn(0x5586b06159b0 0x5586b066cff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4f6d1237c615bc2 server_cookie=f592840f75f1e903 in_seq=0 out_seq=0 2023-11-28T18:29:23.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2eb17fa640 10 client.7896 ms_handle_connect on v2:172.21.15.105:6834/4162499249 2023-11-28T18:29:23.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2eb17fa640 1 -- 172.21.15.105:0/646611838 <== mds.1 v2:172.21.15.105:6834/4162499249 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5586b0425b50 con 0x5586b06159b0 2023-11-28T18:29:23.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2eb17fa640 10 client.7896 handle_command_reply: tid=0 2023-11-28T18:29:23.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2edbd22000 1 client.7896 shutdown 2023-11-28T18:29:23.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2edbd22000 1 -- 172.21.15.105:0/646611838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b060e500 msgr2=0x5586b0614520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:23.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2edbd22000 1 --2- 172.21.15.105:0/646611838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b060e500 0x5586b0614520 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4051630 tx=0x7f2ec4059280 comp rx=0 tx=0).stop 2023-11-28T18:29:23.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2edbd22000 1 -- 172.21.15.105:0/646611838 shutdown_connections 2023-11-28T18:29:23.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2edbd22000 1 --2- 172.21.15.105:0/646611838 >> [v2:172.21.15.105:6834/4162499249,v1:172.21.15.105:6835/4162499249] conn(0x5586b06159b0 0x5586b066cff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2edbd22000 1 --2- 172.21.15.105:0/646611838 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b065b650 0x5586b0614fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2edbd22000 1 --2- 172.21.15.105:0/646611838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b065ad30 0x5586b0614a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2eb17fa640 0 client.7896 ms_handle_reset on v2:172.21.15.105:6834/4162499249 2023-11-28T18:29:23.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2edbd22000 1 --2- 172.21.15.105:0/646611838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b060e500 0x5586b0614520 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2edbd22000 1 -- 172.21.15.105:0/646611838 >> 172.21.15.105:0/646611838 conn(0x5586b063b8c0 msgr2=0x5586b060d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:23.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2edbd22000 1 -- 172.21.15.105:0/646611838 shutdown_connections 2023-11-28T18:29:23.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2edbd22000 1 -- 172.21.15.105:0/646611838 wait complete. 2023-11-28T18:29:23.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.025+0000 7f2edbd22000 20 client.7896 trim_cache size 0 max 16384 2023-11-28T18:29:23.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.041+0000 7f2edbd22000 1 Processor -- start 2023-11-28T18:29:23.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.041+0000 7f2edbd22000 1 -- start start 2023-11-28T18:29:23.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.041+0000 7f2edbd22000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b060e500 0x5586b063a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.041+0000 7f2edbd22000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b065ad30 0x5586b063a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.041+0000 7f2edbd22000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b065b650 0x5586b066cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.041+0000 7f2edbd22000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586b0119940 con 0x5586b065b650 2023-11-28T18:29:23.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.041+0000 7f2edbd22000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586b041f8f0 con 0x5586b060e500 2023-11-28T18:29:23.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.041+0000 7f2edbd22000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586b040d5b0 con 0x5586b065ad30 2023-11-28T18:29:23.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.041+0000 7f2ed2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b065b650 0x5586b066cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.041+0000 7f2ed2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b065b650 0x5586b066cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46676/0 (socket says 172.21.15.105:46676) 2023-11-28T18:29:23.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.041+0000 7f2ed2d9d640 1 -- 172.21.15.105:0/2929690488 learned_addr learned my addr 172.21.15.105:0/2929690488 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:23.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2ed259c640 1 --2- 172.21.15.105:0/2929690488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b060e500 0x5586b063a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2ecbfff640 1 --2- 172.21.15.105:0/2929690488 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b065ad30 0x5586b063a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2ed2d9d640 1 -- 172.21.15.105:0/2929690488 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b065ad30 msgr2=0x5586b063a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:23.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2ed2d9d640 1 --2- 172.21.15.105:0/2929690488 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b065ad30 0x5586b063a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2ed2d9d640 1 -- 172.21.15.105:0/2929690488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b060e500 msgr2=0x5586b063a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:23.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2ed2d9d640 1 --2- 172.21.15.105:0/2929690488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b060e500 0x5586b063a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2ed2d9d640 1 -- 172.21.15.105:0/2929690488 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586b060e370 con 0x5586b065b650 2023-11-28T18:29:23.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2ecbfff640 1 --2- 172.21.15.105:0/2929690488 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b065ad30 0x5586b063a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:23.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2ed2d9d640 1 --2- 172.21.15.105:0/2929690488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b065b650 0x5586b066cff0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc007850 tx=0x7f2ecc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aaadb1223813bb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:23.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2ec8ff9640 1 -- 172.21.15.105:0/2929690488 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc00e040 con 0x5586b065b650 2023-11-28T18:29:23.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2ec8ff9640 1 -- 172.21.15.105:0/2929690488 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ecc00f040 con 0x5586b065b650 2023-11-28T18:29:23.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2ec8ff9640 1 -- 172.21.15.105:0/2929690488 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc015ee0 con 0x5586b065b650 2023-11-28T18:29:23.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2edbd22000 1 -- 172.21.15.105:0/2929690488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b065b650 msgr2=0x5586b066cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:23.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2edbd22000 1 --2- 172.21.15.105:0/2929690488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b065b650 0x5586b066cff0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc007850 tx=0x7f2ecc009750 comp rx=0 tx=0).stop 2023-11-28T18:29:23.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2edbd22000 1 -- 172.21.15.105:0/2929690488 shutdown_connections 2023-11-28T18:29:23.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2edbd22000 1 --2- 172.21.15.105:0/2929690488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b065b650 0x5586b066cff0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2edbd22000 1 --2- 172.21.15.105:0/2929690488 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b065ad30 0x5586b063a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2edbd22000 1 --2- 172.21.15.105:0/2929690488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b060e500 0x5586b063a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2edbd22000 1 -- 172.21.15.105:0/2929690488 >> 172.21.15.105:0/2929690488 conn(0x5586b063b8c0 msgr2=0x5586b060c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2edbd22000 1 -- 172.21.15.105:0/2929690488 shutdown_connections 2023-11-28T18:29:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2edbd22000 1 -- 172.21.15.105:0/2929690488 wait complete. 2023-11-28T18:29:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2edbd22000 1 -- start start 2023-11-28T18:29:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2edbd22000 1 Processor -- start 2023-11-28T18:29:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2edbd22000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b060e500 0x5586b06bb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2edbd22000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b065ad30 0x5586b06bbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2edbd22000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b065b650 0x5586b0676a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2edbd22000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586b045c5c0 con 0x5586b065b650 2023-11-28T18:29:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2edbd22000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586b042a6c0 con 0x5586b065ad30 2023-11-28T18:29:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2edbd22000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586b045ffd0 con 0x5586b060e500 2023-11-28T18:29:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2edbd22000 10 client.0 _ll_register_callbacks cb 0x5586b04d7060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:29:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2ed259c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b060e500 0x5586b06bb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2ed2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b065b650 0x5586b0676a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2ed259c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b060e500 0x5586b06bb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:42732/0 (socket says 172.21.15.105:42732) 2023-11-28T18:29:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2ed259c640 1 -- 172.21.15.105:0/2302363500 learned_addr learned my addr 172.21.15.105:0/2302363500 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.045+0000 7f2ecbfff640 1 --2- 172.21.15.105:0/2302363500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b065ad30 0x5586b06bbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2ed2d9d640 1 -- 172.21.15.105:0/2302363500 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b060e500 msgr2=0x5586b06bb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2ed2d9d640 1 --2- 172.21.15.105:0/2302363500 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b060e500 0x5586b06bb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2ed2d9d640 1 -- 172.21.15.105:0/2302363500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b065ad30 msgr2=0x5586b06bbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:23.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2ed2d9d640 1 --2- 172.21.15.105:0/2302363500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b065ad30 0x5586b06bbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2ed2d9d640 1 -- 172.21.15.105:0/2302363500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ecc007500 con 0x5586b065b650 2023-11-28T18:29:23.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2ed259c640 1 --2- 172.21.15.105:0/2302363500 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b060e500 0x5586b06bb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:29:23.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2ecbfff640 1 --2- 172.21.15.105:0/2302363500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b065ad30 0x5586b06bbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:29:23.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2ed2d9d640 1 --2- 172.21.15.105:0/2302363500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b065b650 0x5586b0676a70 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc007820 tx=0x7f2ecc004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6387b42db66f0bfd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:23.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2eb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:29:23.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2eb17fa640 1 -- 172.21.15.105:0/2302363500 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc00e050 con 0x5586b065b650 2023-11-28T18:29:23.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2eb17fa640 1 -- 172.21.15.105:0/2302363500 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f2ecc02ccb0 con 0x5586b065b650 2023-11-28T18:29:23.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2edbd22000 1 -- 172.21.15.105:0/2302363500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586b045f110 con 0x5586b065b650 2023-11-28T18:29:23.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2eb17fa640 1 -- 172.21.15.105:0/2302363500 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ecc015ec0 con 0x5586b065b650 2023-11-28T18:29:23.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2edbd22000 10 client.7902 fetch_fsmap learned FSMap version 92 2023-11-28T18:29:23.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2edbd22000 1 -- 172.21.15.105:0/2302363500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) v3 -- 0x5586b0407490 con 0x5586b065b650 2023-11-28T18:29:23.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2eb17fa640 1 -- 172.21.15.105:0/2302363500 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5586b045f110 con 0x5586b065b650 2023-11-28T18:29:23.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2eb17fa640 1 -- 172.21.15.105:0/2302363500 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 92) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f2ecc014040 con 0x5586b065b650 2023-11-28T18:29:23.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2edbd22000 10 client.7902 fetch_fsmap finished waiting for FSMap version 92 2023-11-28T18:29:23.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2edbd22000 10 client.7902 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:29:23.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2edbd22000 20 client.7902 populate_metadata read hostname 'smithi105' 2023-11-28T18:29:23.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2edbd22000 1 --2- 172.21.15.105:0/2302363500 >> [v2:172.21.15.105:6834/4162499249,v1:172.21.15.105:6835/4162499249] conn(0x5586b06781a0 0x5586b06c4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2edbd22000 4 client.7902 mds_command: new command op to 7665 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:29:23.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2edbd22000 1 -- 172.21.15.105:0/2302363500 --> [v2:172.21.15.105:6834/4162499249,v1:172.21.15.105:6835/4162499249] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5586b048d650 con 0x5586b06781a0 2023-11-28T18:29:23.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2ed259c640 1 --2- 172.21.15.105:0/2302363500 >> [v2:172.21.15.105:6834/4162499249,v1:172.21.15.105:6835/4162499249] conn(0x5586b06781a0 0x5586b06c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2eb17fa640 10 client.7902 ms_handle_connect on v2:172.21.15.105:6834/4162499249 2023-11-28T18:29:23.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2ed259c640 1 --2- 172.21.15.105:0/2302363500 >> [v2:172.21.15.105:6834/4162499249,v1:172.21.15.105:6835/4162499249] conn(0x5586b06781a0 0x5586b06c4680 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d08a3076fec24dcc server_cookie=c0586c0b58dc31f7 in_seq=0 out_seq=0 2023-11-28T18:29:23.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2eb17fa640 1 -- 172.21.15.105:0/2302363500 <== mds.1 v2:172.21.15.105:6834/4162499249 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x5586b048d650 con 0x5586b06781a0 2023-11-28T18:29:23.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2eb17fa640 10 client.7902 handle_command_reply: tid=0 2023-11-28T18:29:23.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2edbd22000 1 client.7902 shutdown 2023-11-28T18:29:23.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2edbd22000 1 -- 172.21.15.105:0/2302363500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b065b650 msgr2=0x5586b0676a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:23.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2edbd22000 1 --2- 172.21.15.105:0/2302363500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b065b650 0x5586b0676a70 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc007820 tx=0x7f2ecc004ac0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2edbd22000 1 -- 172.21.15.105:0/2302363500 shutdown_connections 2023-11-28T18:29:23.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2edbd22000 1 --2- 172.21.15.105:0/2302363500 >> [v2:172.21.15.105:6834/4162499249,v1:172.21.15.105:6835/4162499249] conn(0x5586b06781a0 0x5586b06c4680 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2edbd22000 1 --2- 172.21.15.105:0/2302363500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586b065b650 0x5586b0676a70 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2edbd22000 1 --2- 172.21.15.105:0/2302363500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b065ad30 0x5586b06bbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2edbd22000 1 --2- 172.21.15.105:0/2302363500 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5586b060e500 0x5586b06bb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2edbd22000 1 -- 172.21.15.105:0/2302363500 >> 172.21.15.105:0/2302363500 conn(0x5586b063b8c0 msgr2=0x5586b060cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2eb17fa640 0 client.7902 ms_handle_reset on v2:172.21.15.105:6834/4162499249 2023-11-28T18:29:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2edbd22000 1 -- 172.21.15.105:0/2302363500 shutdown_connections 2023-11-28T18:29:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2edbd22000 1 -- 172.21.15.105:0/2302363500 wait complete. 2023-11-28T18:29:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2edbd22000 20 client.7902 trim_cache size 0 max 16384 2023-11-28T18:29:23.062 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:29:23.062 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:29:23.062 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 1, 2023-11-28T18:29:23.062 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 7665, 2023-11-28T18:29:23.062 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:29:23.062 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:29:23.063 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:29:23.063 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 2.952782622, 2023-11-28T18:29:23.063 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 92, 2023-11-28T18:29:23.063 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 92, 2023-11-28T18:29:23.063 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 92, 2023-11-28T18:29:23.063 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 24.612562039 2023-11-28T18:29:23.063 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:29:23.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2ed89b8640 1 -- 172.21.15.105:0/2676213291 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2eb40668d0 msgr2=0x7f2eb4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:23.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2ed89b8640 1 --2- 172.21.15.105:0/2676213291 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2eb40668d0 0x7f2eb4068d90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0ae000 tx=0x7f2ebc008040 comp rx=0 tx=0).stop 2023-11-28T18:29:23.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2ed89b8640 1 -- 172.21.15.105:0/2676213291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ed406bf50 msgr2=0x7f2ed4144d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:23.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2ed89b8640 1 --2- 172.21.15.105:0/2676213291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ed406bf50 0x7f2ed4144d20 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400c750 tx=0x7f2ec400cc10 comp rx=0 tx=0).stop 2023-11-28T18:29:23.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2ed89b8640 1 -- 172.21.15.105:0/2676213291 shutdown_connections 2023-11-28T18:29:23.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2ed89b8640 1 --2- 172.21.15.105:0/2676213291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed4154070 0x7f2ed40abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2ed89b8640 1 --2- 172.21.15.105:0/2676213291 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ed40adb80 0x7f2ed40b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2ed89b8640 1 --2- 172.21.15.105:0/2676213291 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2eb40668d0 0x7f2eb4068d90 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2ed89b8640 1 --2- 172.21.15.105:0/2676213291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ed406bf50 0x7f2ed4144d20 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2ed89b8640 1 -- 172.21.15.105:0/2676213291 >> 172.21.15.105:0/2676213291 conn(0x7f2ed40ceb90 msgr2=0x7f2ed40708b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:23.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2ed89b8640 1 -- 172.21.15.105:0/2676213291 shutdown_connections 2023-11-28T18:29:23.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.049+0000 7f2ed89b8640 1 -- 172.21.15.105:0/2676213291 wait complete. 2023-11-28T18:29:23.074 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T18:29:23.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.209+0000 7f9aed580640 1 Processor -- start 2023-11-28T18:29:23.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.209+0000 7f9aed580640 1 -- start start 2023-11-28T18:29:23.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.209+0000 7f9aed580640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9ae806cef0 0x7f9ae80abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.209+0000 7f9aed580640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae80cde30 con 0x7f9ae80ce2c0 2023-11-28T18:29:23.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.209+0000 7f9aed580640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae81599f0 con 0x7f9ae8159610 2023-11-28T18:29:23.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9aed580640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae806de20 con 0x7f9ae806cef0 2023-11-28T18:29:23.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ade7fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9ae80ce2c0 0x7f9ae806c7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40200/0 (socket says 172.21.15.105:40200) 2023-11-28T18:29:23.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ade7fc640 1 -- 172.21.15.105:0/168888677 learned_addr learned my addr 172.21.15.105:0/168888677 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:23.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae77fe640 1 --2- 172.21.15.105:0/168888677 >> v2:172.21.15.148:3300/0 conn(0x7f9ae806cef0 0x7f9ae80abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae67fc640 1 -- 172.21.15.105:0/168888677 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2246744652 0 0) 0x7f9ae80cde30 con 0x7f9ae80ce2c0 2023-11-28T18:29:23.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae67fc640 1 -- 172.21.15.105:0/168888677 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ad4003530 con 0x7f9ae80ce2c0 2023-11-28T18:29:23.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae67fc640 1 -- 172.21.15.105:0/168888677 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 937077725 0 0) 0x7f9ae81599f0 con 0x7f9ae8159610 2023-11-28T18:29:23.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae67fc640 1 -- 172.21.15.105:0/168888677 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ae80cde30 con 0x7f9ae8159610 2023-11-28T18:29:23.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae67fc640 1 -- 172.21.15.105:0/168888677 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2635142094 0 0) 0x7f9ad4003530 con 0x7f9ae80ce2c0 2023-11-28T18:29:23.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae67fc640 1 -- 172.21.15.105:0/168888677 >> v2:172.21.15.148:3300/0 conn(0x7f9ae806cef0 msgr2=0x7f9ae80abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:23.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae67fc640 1 --2- 172.21.15.105:0/168888677 >> v2:172.21.15.148:3300/0 conn(0x7f9ae806cef0 0x7f9ae80abd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae67fc640 1 -- 172.21.15.105:0/168888677 >> v1:172.21.15.148:6789/0 conn(0x7f9ae8159610 legacy=0x7f9ae80cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:23.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae67fc640 1 -- 172.21.15.105:0/168888677 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae80cd220 con 0x7f9ae80ce2c0 2023-11-28T18:29:23.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae67fc640 1 -- 172.21.15.105:0/168888677 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f9ad0001680 con 0x7f9ae80ce2c0 2023-11-28T18:29:23.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae77fe640 1 --2- 172.21.15.105:0/168888677 >> v2:172.21.15.148:3300/0 conn(0x7f9ae806cef0 0x7f9ae80abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:29:23.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae67fc640 1 -- 172.21.15.105:0/168888677 >> v1:172.21.15.105:6789/0 conn(0x7f9ae80ce2c0 legacy=0x7f9ae806c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:23.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae67fc640 1 --2- 172.21.15.105:0/168888677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ad4003b10 0x7f9ad4003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae67fc640 1 --2- 172.21.15.105:0/168888677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ad40047d0 0x7f9ad4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae67fc640 1 --2- 172.21.15.105:0/168888677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ad4005190 0x7f9ad400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae67fc640 1 -- 172.21.15.105:0/168888677 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae806de20 con 0x7f9ad40047d0 2023-11-28T18:29:23.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae67fc640 1 -- 172.21.15.105:0/168888677 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad400c010 con 0x7f9ad4005190 2023-11-28T18:29:23.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae67fc640 1 -- 172.21.15.105:0/168888677 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad400c1d0 con 0x7f9ad4003b10 2023-11-28T18:29:23.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ade7fc640 1 --2- 172.21.15.105:0/168888677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ad40047d0 0x7f9ad4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae77fe640 1 --2- 172.21.15.105:0/168888677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ad4005190 0x7f9ad400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae6ffd640 1 --2- 172.21.15.105:0/168888677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ad4003b10 0x7f9ad4003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ade7fc640 1 -- 172.21.15.105:0/168888677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ad4003b10 msgr2=0x7f9ad4003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ade7fc640 1 --2- 172.21.15.105:0/168888677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ad4003b10 0x7f9ad4003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ade7fc640 1 -- 172.21.15.105:0/168888677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ad4005190 msgr2=0x7f9ad400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ade7fc640 1 --2- 172.21.15.105:0/168888677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ad4005190 0x7f9ad400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ade7fc640 1 -- 172.21.15.105:0/168888677 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad4003530 con 0x7f9ad40047d0 2023-11-28T18:29:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae77fe640 1 --2- 172.21.15.105:0/168888677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ad4005190 0x7f9ad400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.213+0000 7f9ae77fe640 1 -- 172.21.15.105:0/168888677 reap_dead start 2023-11-28T18:29:23.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.217+0000 7f9ade7fc640 1 --2- 172.21.15.105:0/168888677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ad40047d0 0x7f9ad4004c50 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000c160 tx=0x7f9ad001cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c5857332667e2d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:23.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.217+0000 7f9ae67fc640 1 -- 172.21.15.105:0/168888677 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad00058e0 con 0x7f9ad40047d0 2023-11-28T18:29:23.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.217+0000 7f9ae67fc640 1 -- 172.21.15.105:0/168888677 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad0005a60 con 0x7f9ad40047d0 2023-11-28T18:29:23.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.217+0000 7f9ae67fc640 1 -- 172.21.15.105:0/168888677 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0005e50 con 0x7f9ad40047d0 2023-11-28T18:29:23.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.217+0000 7f9aed580640 1 -- 172.21.15.105:0/168888677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ad40047d0 msgr2=0x7f9ad4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:23.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.217+0000 7f9aed580640 1 --2- 172.21.15.105:0/168888677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ad40047d0 0x7f9ad4004c50 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000c160 tx=0x7f9ad001cba0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.217+0000 7f9aed580640 1 -- 172.21.15.105:0/168888677 shutdown_connections 2023-11-28T18:29:23.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.217+0000 7f9aed580640 1 --2- 172.21.15.105:0/168888677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ad40047d0 0x7f9ad4004c50 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.217+0000 7f9aed580640 1 -- 172.21.15.105:0/168888677 >> 172.21.15.105:0/168888677 conn(0x7f9ae80ceb90 msgr2=0x7f9ae80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:23.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.217+0000 7f9aed580640 1 -- 172.21.15.105:0/168888677 shutdown_connections 2023-11-28T18:29:23.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.217+0000 7f9aed580640 1 -- 172.21.15.105:0/168888677 wait complete. 2023-11-28T18:29:23.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.217+0000 7f9aed580640 1 Processor -- start 2023-11-28T18:29:23.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.217+0000 7f9aed580640 1 -- start start 2023-11-28T18:29:23.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9aed580640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ad40047d0 0x7f9ae814f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9aed580640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ae80708f0 0x7f9ae8070d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9aed580640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ae80ca280 0x7f9ae80ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9aed580640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae80712b0 con 0x7f9ad40047d0 2023-11-28T18:29:23.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9aed580640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae80b2ef0 con 0x7f9ae80708f0 2023-11-28T18:29:23.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9aed580640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae80cac50 con 0x7f9ae80ca280 2023-11-28T18:29:23.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9ae6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ad40047d0 0x7f9ae814f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9ae6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ad40047d0 0x7f9ae814f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46704/0 (socket says 172.21.15.105:46704) 2023-11-28T18:29:23.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9ae6ffd640 1 -- 172.21.15.105:0/953638595 learned_addr learned my addr 172.21.15.105:0/953638595 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:23.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9ae77fe640 1 --2- 172.21.15.105:0/953638595 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ae80ca280 0x7f9ae80ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9ade7fc640 1 --2- 172.21.15.105:0/953638595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ae80708f0 0x7f9ae8070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9ae6ffd640 1 -- 172.21.15.105:0/953638595 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ae80ca280 msgr2=0x7f9ae80ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:23.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9ae6ffd640 1 --2- 172.21.15.105:0/953638595 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ae80ca280 0x7f9ae80ca710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9ae6ffd640 1 -- 172.21.15.105:0/953638595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ae80708f0 msgr2=0x7f9ae8070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:23.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9ae6ffd640 1 --2- 172.21.15.105:0/953638595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ae80708f0 0x7f9ae8070d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9ae6ffd640 1 -- 172.21.15.105:0/953638595 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae80b3140 con 0x7f9ad40047d0 2023-11-28T18:29:23.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9ae6ffd640 1 --2- 172.21.15.105:0/953638595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ad40047d0 0x7f9ae814f710 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8011ee0 tx=0x7f9ad800f540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:23.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9adffff640 1 -- 172.21.15.105:0/953638595 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8015610 con 0x7f9ad40047d0 2023-11-28T18:29:23.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9adffff640 1 -- 172.21.15.105:0/953638595 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad8015770 con 0x7f9ad40047d0 2023-11-28T18:29:23.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9aed580640 1 -- 172.21.15.105:0/953638595 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae80b3420 con 0x7f9ad40047d0 2023-11-28T18:29:23.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9adffff640 1 -- 172.21.15.105:0/953638595 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8015b20 con 0x7f9ad40047d0 2023-11-28T18:29:23.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9aed580640 1 -- 172.21.15.105:0/953638595 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae80b39e0 con 0x7f9ad40047d0 2023-11-28T18:29:23.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.221+0000 7f9adffff640 1 -- 172.21.15.105:0/953638595 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f9ad8015c80 con 0x7f9ad40047d0 2023-11-28T18:29:23.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9adffff640 1 --2- 172.21.15.105:0/953638595 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9ac80668d0 0x7f9ac8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9adffff640 1 -- 172.21.15.105:0/953638595 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f9ad8087e50 con 0x7f9ad40047d0 2023-11-28T18:29:23.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9ade7fc640 1 --2- 172.21.15.105:0/953638595 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9ac80668d0 0x7f9ac8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9af08e3000 1 Processor -- start 2023-11-28T18:29:23.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9af08e3000 1 -- start start 2023-11-28T18:29:23.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9ade7fc640 1 --2- 172.21.15.105:0/953638595 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9ac80668d0 0x7f9ac8068d90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9ad00aa000 tx=0x7f9ad0001a00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:23.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9af08e3000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca51c5d30 0x560ca51c6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9af08e3000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560ca51c6650 0x560ca5178fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9af08e3000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca5179500 0x560ca51798e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9af08e3000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca5042110 con 0x560ca5179500 2023-11-28T18:29:23.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9af08e3000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca4fd29a0 con 0x560ca51c5d30 2023-11-28T18:29:23.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9af08e3000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ca4fcfed0 con 0x560ca51c6650 2023-11-28T18:29:23.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9ae77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca5179500 0x560ca51798e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9ae77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca5179500 0x560ca51798e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46710/0 (socket says 172.21.15.105:46710) 2023-11-28T18:29:23.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9ae77fe640 1 -- 172.21.15.105:0/2135017396 learned_addr learned my addr 172.21.15.105:0/2135017396 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:23.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9ade7fc640 1 --2- 172.21.15.105:0/2135017396 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560ca51c6650 0x560ca5178fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9ae6ffd640 1 --2- 172.21.15.105:0/2135017396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca51c5d30 0x560ca51c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9ae77fe640 1 -- 172.21.15.105:0/2135017396 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560ca51c6650 msgr2=0x560ca5178fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:23.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9ae77fe640 1 --2- 172.21.15.105:0/2135017396 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560ca51c6650 0x560ca5178fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9ae77fe640 1 -- 172.21.15.105:0/2135017396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca51c5d30 msgr2=0x560ca51c6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:23.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9ae77fe640 1 --2- 172.21.15.105:0/2135017396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca51c5d30 0x560ca51c6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9ae77fe640 1 -- 172.21.15.105:0/2135017396 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ca4cc3890 con 0x560ca5179500 2023-11-28T18:29:23.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9ae6ffd640 1 --2- 172.21.15.105:0/2135017396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca51c5d30 0x560ca51c6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:23.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9ae77fe640 1 --2- 172.21.15.105:0/2135017396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca5179500 0x560ca51798e0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8067c70 tx=0x7f9ae000a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbf9d82dcfdf002 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:23.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9adcff9640 1 -- 172.21.15.105:0/2135017396 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000f040 con 0x560ca5179500 2023-11-28T18:29:23.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9adcff9640 1 -- 172.21.15.105:0/2135017396 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae0016830 con 0x560ca5179500 2023-11-28T18:29:23.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.225+0000 7f9adcff9640 1 -- 172.21.15.105:0/2135017396 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0016be0 con 0x560ca5179500 2023-11-28T18:29:23.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9af08e3000 1 -- 172.21.15.105:0/2135017396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca5179500 msgr2=0x560ca51798e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:23.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9af08e3000 1 --2- 172.21.15.105:0/2135017396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca5179500 0x560ca51798e0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8067c70 tx=0x7f9ae000a860 comp rx=0 tx=0).stop 2023-11-28T18:29:23.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9af08e3000 1 -- 172.21.15.105:0/2135017396 shutdown_connections 2023-11-28T18:29:23.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9af08e3000 1 --2- 172.21.15.105:0/2135017396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca5179500 0x560ca51798e0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9af08e3000 1 --2- 172.21.15.105:0/2135017396 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560ca51c6650 0x560ca5178fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9af08e3000 1 --2- 172.21.15.105:0/2135017396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca51c5d30 0x560ca51c6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9af08e3000 1 -- 172.21.15.105:0/2135017396 >> 172.21.15.105:0/2135017396 conn(0x560ca51a68c0 msgr2=0x560ca51a8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:23.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9af08e3000 1 -- 172.21.15.105:0/2135017396 shutdown_connections 2023-11-28T18:29:23.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9af08e3000 1 -- 172.21.15.105:0/2135017396 wait complete. 2023-11-28T18:29:23.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9af08e3000 1 -- start start 2023-11-28T18:29:23.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9af08e3000 1 Processor -- start 2023-11-28T18:29:23.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9af08e3000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca5179500 0x560ca517f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9af08e3000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560ca51c5d30 0x560ca517f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9af08e3000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca51c6650 0x560ca517ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9af08e3000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca4fcb7c0 con 0x560ca51c6650 2023-11-28T18:29:23.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9af08e3000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca4f83bf0 con 0x560ca5179500 2023-11-28T18:29:23.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9af08e3000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ca4f89a90 con 0x560ca51c5d30 2023-11-28T18:29:23.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9af08e3000 10 client.0 _ll_register_callbacks cb 0x560ca5166e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:29:23.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9ae77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca51c6650 0x560ca517ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9ae77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca51c6650 0x560ca517ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46722/0 (socket says 172.21.15.105:46722) 2023-11-28T18:29:23.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9ae77fe640 1 -- 172.21.15.105:0/746136588 learned_addr learned my addr 172.21.15.105:0/746136588 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:23.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9ade7fc640 1 --2- 172.21.15.105:0/746136588 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560ca51c5d30 0x560ca517f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9ae6ffd640 1 --2- 172.21.15.105:0/746136588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca5179500 0x560ca517f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9ae77fe640 1 -- 172.21.15.105:0/746136588 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560ca51c5d30 msgr2=0x560ca517f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:23.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9ae77fe640 1 --2- 172.21.15.105:0/746136588 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560ca51c5d30 0x560ca517f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9ae77fe640 1 -- 172.21.15.105:0/746136588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca5179500 msgr2=0x560ca517f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:23.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9ae77fe640 1 --2- 172.21.15.105:0/746136588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca5179500 0x560ca517f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9ae77fe640 1 -- 172.21.15.105:0/746136588 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae000ab80 con 0x560ca51c6650 2023-11-28T18:29:23.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9ae77fe640 1 --2- 172.21.15.105:0/746136588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca51c6650 0x560ca517ff30 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000a8a0 tx=0x7f9ae0004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762ad30b9261a547 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:23.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9ac5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:29:23.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9ac5ffb640 1 -- 172.21.15.105:0/746136588 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000f030 con 0x560ca51c6650 2023-11-28T18:29:23.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9ac5ffb640 1 -- 172.21.15.105:0/746136588 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f9ae0013e80 con 0x560ca51c6650 2023-11-28T18:29:23.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9af08e3000 1 -- 172.21.15.105:0/746136588 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ca4f8d470 con 0x560ca51c6650 2023-11-28T18:29:23.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9ac5ffb640 1 -- 172.21.15.105:0/746136588 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ae0020660 con 0x560ca51c6650 2023-11-28T18:29:23.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9af08e3000 10 client.7914 fetch_fsmap learned FSMap version 92 2023-11-28T18:29:23.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.229+0000 7f9af08e3000 1 -- 172.21.15.105:0/746136588 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) v3 -- 0x560ca4f8c190 con 0x560ca51c6650 2023-11-28T18:29:23.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9ac5ffb640 1 -- 172.21.15.105:0/746136588 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560ca4f8d470 con 0x560ca51c6650 2023-11-28T18:29:23.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9ac5ffb640 1 -- 172.21.15.105:0/746136588 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 92) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f9ae00207c0 con 0x560ca51c6650 2023-11-28T18:29:23.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9af08e3000 10 client.7914 fetch_fsmap finished waiting for FSMap version 92 2023-11-28T18:29:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9af08e3000 10 client.7914 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:29:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9af08e3000 20 client.7914 populate_metadata read hostname 'smithi105' 2023-11-28T18:29:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9af08e3000 1 --2- 172.21.15.105:0/746136588 >> [v2:172.21.15.105:6836/3239728352,v1:172.21.15.105:6837/3239728352] conn(0x560ca5180940 0x560ca51d7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9af08e3000 4 client.7914 mds_command: new command op to 7689 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:29:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9af08e3000 1 -- 172.21.15.105:0/746136588 --> [v2:172.21.15.105:6836/3239728352,v1:172.21.15.105:6837/3239728352] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ca4f90b50 con 0x560ca5180940 2023-11-28T18:29:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9ae6ffd640 1 --2- 172.21.15.105:0/746136588 >> [v2:172.21.15.105:6836/3239728352,v1:172.21.15.105:6837/3239728352] conn(0x560ca5180940 0x560ca51d7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9ae6ffd640 1 --2- 172.21.15.105:0/746136588 >> [v2:172.21.15.105:6836/3239728352,v1:172.21.15.105:6837/3239728352] conn(0x560ca5180940 0x560ca51d7ff0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49b431bb038a6341 server_cookie=da5f25c7cfd9b44c in_seq=0 out_seq=0 2023-11-28T18:29:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9ac5ffb640 10 client.7914 ms_handle_connect on v2:172.21.15.105:6836/3239728352 2023-11-28T18:29:23.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9ac5ffb640 1 -- 172.21.15.105:0/746136588 <== mds.0 v2:172.21.15.105:6836/3239728352 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f9ad807fe90 con 0x560ca5180940 2023-11-28T18:29:23.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9ac5ffb640 10 client.7914 handle_command_reply: tid=0 2023-11-28T18:29:23.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9af08e3000 1 client.7914 shutdown 2023-11-28T18:29:23.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9af08e3000 1 -- 172.21.15.105:0/746136588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca51c6650 msgr2=0x560ca517ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:23.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9af08e3000 1 --2- 172.21.15.105:0/746136588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca51c6650 0x560ca517ff30 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000a8a0 tx=0x7f9ae0004a50 comp rx=0 tx=0).stop 2023-11-28T18:29:23.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9af08e3000 1 -- 172.21.15.105:0/746136588 shutdown_connections 2023-11-28T18:29:23.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9af08e3000 1 --2- 172.21.15.105:0/746136588 >> [v2:172.21.15.105:6836/3239728352,v1:172.21.15.105:6837/3239728352] conn(0x560ca5180940 0x560ca51d7ff0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9af08e3000 1 --2- 172.21.15.105:0/746136588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca51c6650 0x560ca517ff30 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9af08e3000 1 --2- 172.21.15.105:0/746136588 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560ca51c5d30 0x560ca517f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9af08e3000 1 --2- 172.21.15.105:0/746136588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca5179500 0x560ca517f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9ac5ffb640 0 client.7914 ms_handle_reset on v2:172.21.15.105:6836/3239728352 2023-11-28T18:29:23.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9af08e3000 1 -- 172.21.15.105:0/746136588 >> 172.21.15.105:0/746136588 conn(0x560ca51a68c0 msgr2=0x560ca51786b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:23.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9af08e3000 1 -- 172.21.15.105:0/746136588 shutdown_connections 2023-11-28T18:29:23.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9af08e3000 1 -- 172.21.15.105:0/746136588 wait complete. 2023-11-28T18:29:23.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.233+0000 7f9af08e3000 20 client.7914 trim_cache size 0 max 16384 2023-11-28T18:29:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9af08e3000 1 Processor -- start 2023-11-28T18:29:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9af08e3000 1 -- start start 2023-11-28T18:29:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9af08e3000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca5179500 0x560ca51a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9af08e3000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560ca51c5d30 0x560ca51a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9af08e3000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca51c6650 0x560ca51d7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9af08e3000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca4c84940 con 0x560ca51c6650 2023-11-28T18:29:23.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9af08e3000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca517c100 con 0x560ca5179500 2023-11-28T18:29:23.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9af08e3000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ca4f8a8f0 con 0x560ca51c5d30 2023-11-28T18:29:23.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9ae77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca51c6650 0x560ca51d7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9ae77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca51c6650 0x560ca51d7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46724/0 (socket says 172.21.15.105:46724) 2023-11-28T18:29:23.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9ae77fe640 1 -- 172.21.15.105:0/1951107818 learned_addr learned my addr 172.21.15.105:0/1951107818 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:23.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9ae6ffd640 1 --2- 172.21.15.105:0/1951107818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca5179500 0x560ca51a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9ade7fc640 1 --2- 172.21.15.105:0/1951107818 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560ca51c5d30 0x560ca51a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9ae77fe640 1 -- 172.21.15.105:0/1951107818 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560ca51c5d30 msgr2=0x560ca51a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:23.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9ae77fe640 1 --2- 172.21.15.105:0/1951107818 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560ca51c5d30 0x560ca51a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9ae77fe640 1 -- 172.21.15.105:0/1951107818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca5179500 msgr2=0x560ca51a53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:23.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9ae77fe640 1 --2- 172.21.15.105:0/1951107818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca5179500 0x560ca51a53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9ae77fe640 1 -- 172.21.15.105:0/1951107818 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ca4f72490 con 0x560ca51c6650 2023-11-28T18:29:23.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9ade7fc640 1 --2- 172.21.15.105:0/1951107818 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560ca51c5d30 0x560ca51a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9ae77fe640 1 --2- 172.21.15.105:0/1951107818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca51c6650 0x560ca51d7ff0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000c010 tx=0x7f9ae0023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa71d828a6bb4b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9add7fa640 1 -- 172.21.15.105:0/1951107818 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0038070 con 0x560ca51c6650 2023-11-28T18:29:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9add7fa640 1 -- 172.21.15.105:0/1951107818 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae0013dc0 con 0x560ca51c6650 2023-11-28T18:29:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.249+0000 7f9add7fa640 1 -- 172.21.15.105:0/1951107818 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000f070 con 0x560ca51c6650 2023-11-28T18:29:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9af08e3000 1 -- 172.21.15.105:0/1951107818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca51c6650 msgr2=0x560ca51d7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9af08e3000 1 --2- 172.21.15.105:0/1951107818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca51c6650 0x560ca51d7ff0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000c010 tx=0x7f9ae0023040 comp rx=0 tx=0).stop 2023-11-28T18:29:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9af08e3000 1 -- 172.21.15.105:0/1951107818 shutdown_connections 2023-11-28T18:29:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9af08e3000 1 --2- 172.21.15.105:0/1951107818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca51c6650 0x560ca51d7ff0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9af08e3000 1 --2- 172.21.15.105:0/1951107818 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560ca51c5d30 0x560ca51a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9af08e3000 1 --2- 172.21.15.105:0/1951107818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca5179500 0x560ca51a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9af08e3000 1 -- 172.21.15.105:0/1951107818 >> 172.21.15.105:0/1951107818 conn(0x560ca51a68c0 msgr2=0x560ca5177720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:23.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9af08e3000 1 -- 172.21.15.105:0/1951107818 shutdown_connections 2023-11-28T18:29:23.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9af08e3000 1 -- 172.21.15.105:0/1951107818 wait complete. 2023-11-28T18:29:23.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9af08e3000 1 -- start start 2023-11-28T18:29:23.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9af08e3000 1 Processor -- start 2023-11-28T18:29:23.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9af08e3000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca5179500 0x560ca5226810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9af08e3000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560ca51c5d30 0x560ca5226d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9af08e3000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca51c6650 0x560ca51e1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9af08e3000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca51a6780 con 0x560ca51c6650 2023-11-28T18:29:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9af08e3000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca5180f50 con 0x560ca5179500 2023-11-28T18:29:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9af08e3000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ca4fa0b70 con 0x560ca51c5d30 2023-11-28T18:29:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9af08e3000 10 client.0 _ll_register_callbacks cb 0x560ca5042060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:29:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9ae77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca51c6650 0x560ca51e1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9ae77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca51c6650 0x560ca51e1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46728/0 (socket says 172.21.15.105:46728) 2023-11-28T18:29:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9ae77fe640 1 -- 172.21.15.105:0/1983214720 learned_addr learned my addr 172.21.15.105:0/1983214720 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9ae6ffd640 1 --2- 172.21.15.105:0/1983214720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca5179500 0x560ca5226810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9ade7fc640 1 --2- 172.21.15.105:0/1983214720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560ca51c5d30 0x560ca5226d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9ae77fe640 1 -- 172.21.15.105:0/1983214720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560ca51c5d30 msgr2=0x560ca5226d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9ae77fe640 1 --2- 172.21.15.105:0/1983214720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560ca51c5d30 0x560ca5226d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9ae77fe640 1 -- 172.21.15.105:0/1983214720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca5179500 msgr2=0x560ca5226810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9ae77fe640 1 --2- 172.21.15.105:0/1983214720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca5179500 0x560ca5226810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9ae77fe640 1 -- 172.21.15.105:0/1983214720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ca4f8c190 con 0x560ca51c6650 2023-11-28T18:29:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9ae77fe640 1 --2- 172.21.15.105:0/1983214720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca51c6650 0x560ca51e1a70 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0022040 tx=0x7f9ae000c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40668f2b19229a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9ac5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:29:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9ac5ffb640 1 -- 172.21.15.105:0/1983214720 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0038070 con 0x560ca51c6650 2023-11-28T18:29:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9ac5ffb640 1 -- 172.21.15.105:0/1983214720 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f9ae002b8a0 con 0x560ca51c6650 2023-11-28T18:29:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9af08e3000 1 -- 172.21.15.105:0/1983214720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ca4fca110 con 0x560ca51c6650 2023-11-28T18:29:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9ac5ffb640 1 -- 172.21.15.105:0/1983214720 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ae002ba60 con 0x560ca51c6650 2023-11-28T18:29:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9af08e3000 10 client.7920 fetch_fsmap learned FSMap version 92 2023-11-28T18:29:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9af08e3000 1 -- 172.21.15.105:0/1983214720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) v3 -- 0x560ca4cc3cc0 con 0x560ca51c6650 2023-11-28T18:29:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9ac5ffb640 1 -- 172.21.15.105:0/1983214720 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560ca4fca110 con 0x560ca51c6650 2023-11-28T18:29:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.253+0000 7f9ac5ffb640 1 -- 172.21.15.105:0/1983214720 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 92) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f9ae0044070 con 0x560ca51c6650 2023-11-28T18:29:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9af08e3000 10 client.7920 fetch_fsmap finished waiting for FSMap version 92 2023-11-28T18:29:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9af08e3000 10 client.7920 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:29:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9af08e3000 20 client.7920 populate_metadata read hostname 'smithi105' 2023-11-28T18:29:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9af08e3000 1 --2- 172.21.15.105:0/1983214720 >> [v2:172.21.15.105:6836/3239728352,v1:172.21.15.105:6837/3239728352] conn(0x560ca51e2430 0x560ca51e2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9af08e3000 4 client.7920 mds_command: new command op to 7689 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:29:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9af08e3000 1 -- 172.21.15.105:0/1983214720 --> [v2:172.21.15.105:6836/3239728352,v1:172.21.15.105:6837/3239728352] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560ca4ff8650 con 0x560ca51e2430 2023-11-28T18:29:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9ae6ffd640 1 --2- 172.21.15.105:0/1983214720 >> [v2:172.21.15.105:6836/3239728352,v1:172.21.15.105:6837/3239728352] conn(0x560ca51e2430 0x560ca51e2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9ae6ffd640 1 --2- 172.21.15.105:0/1983214720 >> [v2:172.21.15.105:6836/3239728352,v1:172.21.15.105:6837/3239728352] conn(0x560ca51e2430 0x560ca51e2810 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adb32af064c63a62 server_cookie=d2987e036893a005 in_seq=0 out_seq=0 2023-11-28T18:29:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9ac5ffb640 10 client.7920 ms_handle_connect on v2:172.21.15.105:6836/3239728352 2023-11-28T18:29:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9ac5ffb640 1 -- 172.21.15.105:0/1983214720 <== mds.0 v2:172.21.15.105:6836/3239728352 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x560ca4ff8650 con 0x560ca51e2430 2023-11-28T18:29:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9ac5ffb640 10 client.7920 handle_command_reply: tid=0 2023-11-28T18:29:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9af08e3000 1 client.7920 shutdown 2023-11-28T18:29:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9af08e3000 1 -- 172.21.15.105:0/1983214720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca51c6650 msgr2=0x560ca51e1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9af08e3000 1 --2- 172.21.15.105:0/1983214720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca51c6650 0x560ca51e1a70 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0022040 tx=0x7f9ae000c040 comp rx=0 tx=0).stop 2023-11-28T18:29:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9af08e3000 1 -- 172.21.15.105:0/1983214720 shutdown_connections 2023-11-28T18:29:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9af08e3000 1 --2- 172.21.15.105:0/1983214720 >> [v2:172.21.15.105:6836/3239728352,v1:172.21.15.105:6837/3239728352] conn(0x560ca51e2430 0x560ca51e2810 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9af08e3000 1 --2- 172.21.15.105:0/1983214720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ca51c6650 0x560ca51e1a70 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9af08e3000 1 --2- 172.21.15.105:0/1983214720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560ca51c5d30 0x560ca5226d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9af08e3000 1 --2- 172.21.15.105:0/1983214720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca5179500 0x560ca5226810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9ac5ffb640 0 client.7920 ms_handle_reset on v2:172.21.15.105:6836/3239728352 2023-11-28T18:29:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9af08e3000 1 -- 172.21.15.105:0/1983214720 >> 172.21.15.105:0/1983214720 conn(0x560ca51a68c0 msgr2=0x560ca5177fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9af08e3000 1 -- 172.21.15.105:0/1983214720 shutdown_connections 2023-11-28T18:29:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9af08e3000 1 -- 172.21.15.105:0/1983214720 wait complete. 2023-11-28T18:29:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9af08e3000 20 client.7920 trim_cache size 0 max 16384 2023-11-28T18:29:23.268 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:29:23.268 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:29:23.269 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2023-11-28T18:29:23.269 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 7689, 2023-11-28T18:29:23.269 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:29:23.269 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:29:23.269 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:29:23.269 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 12.919393613, 2023-11-28T18:29:23.269 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 92, 2023-11-28T18:29:23.269 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 92, 2023-11-28T18:29:23.269 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 92, 2023-11-28T18:29:23.269 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 18.847906885 2023-11-28T18:29:23.269 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:29:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9aed580640 1 -- 172.21.15.105:0/953638595 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9ac80668d0 msgr2=0x7f9ac8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9aed580640 1 --2- 172.21.15.105:0/953638595 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9ac80668d0 0x7f9ac8068d90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9ad00aa000 tx=0x7f9ad0001a00 comp rx=0 tx=0).stop 2023-11-28T18:29:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9aed580640 1 -- 172.21.15.105:0/953638595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ad40047d0 msgr2=0x7f9ae814f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9aed580640 1 --2- 172.21.15.105:0/953638595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ad40047d0 0x7f9ae814f710 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8011ee0 tx=0x7f9ad800f540 comp rx=0 tx=0).stop 2023-11-28T18:29:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9aed580640 1 -- 172.21.15.105:0/953638595 shutdown_connections 2023-11-28T18:29:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9aed580640 1 --2- 172.21.15.105:0/953638595 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ae80ca280 0x7f9ae80ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9aed580640 1 --2- 172.21.15.105:0/953638595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ae80708f0 0x7f9ae8070d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9aed580640 1 --2- 172.21.15.105:0/953638595 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9ac80668d0 0x7f9ac8068d90 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9aed580640 1 --2- 172.21.15.105:0/953638595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ad40047d0 0x7f9ae814f710 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9aed580640 1 -- 172.21.15.105:0/953638595 >> 172.21.15.105:0/953638595 conn(0x7f9ae80ceb90 msgr2=0x7f9ae8158760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9aed580640 1 -- 172.21.15.105:0/953638595 shutdown_connections 2023-11-28T18:29:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:23.257+0000 7f9aed580640 1 -- 172.21.15.105:0/953638595 wait complete. 2023-11-28T18:29:23.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:23.280 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-28T18:29:23.281 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:29:23.281 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p tree 2023-11-28T18:29:23.281 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:29:23.281 DEBUG:teuthology.orchestra.run.smithi105:> setfattr -n ceph.dir.pin.distributed -v 1 tree 2023-11-28T18:29:23.281 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:29:23.281 DEBUG:teuthology.orchestra.run.smithi105:> for ((i = 0; i < 100; i++)); do 2023-11-28T18:29:23.281 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p "tree/$i" 2023-11-28T18:29:23.281 DEBUG:teuthology.orchestra.run.smithi105:> echo file > "tree/$i/file" 2023-11-28T18:29:23.281 DEBUG:teuthology.orchestra.run.smithi105:> done 2023-11-28T18:29:23.281 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:29:23.295 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree 2023-11-28T18:29:23.303 INFO:teuthology.orchestra.run.smithi105.stderr:+ setfattr -n ceph.dir.pin.distributed -v 1 tree 2023-11-28T18:29:23.307 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i = 0 )) 2023-11-28T18:29:23.307 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.308 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/0 2023-11-28T18:29:23.312 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.317 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.317 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.317 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/1 2023-11-28T18:29:23.326 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.328 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.329 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.329 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/2 2023-11-28T18:29:23.335 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.337 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.337 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.337 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/3 2023-11-28T18:29:23.343 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.346 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.346 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.346 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/4 2023-11-28T18:29:23.352 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.354 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.354 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.354 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/5 2023-11-28T18:29:23.360 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.363 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.363 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.363 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/6 2023-11-28T18:29:23.369 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.371 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.371 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.372 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/7 2023-11-28T18:29:23.378 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.380 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.380 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.380 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/8 2023-11-28T18:29:23.386 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.388 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.388 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.388 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/9 2023-11-28T18:29:23.394 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.397 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.397 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.397 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/10 2023-11-28T18:29:23.403 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.405 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.405 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.405 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/11 2023-11-28T18:29:23.411 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.413 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.414 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.414 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/12 2023-11-28T18:29:23.420 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.422 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.422 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/13 2023-11-28T18:29:23.428 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.431 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.431 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.431 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/14 2023-11-28T18:29:23.436 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.438 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.438 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.439 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/15 2023-11-28T18:29:23.444 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.447 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.447 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.447 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/16 2023-11-28T18:29:23.453 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.455 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.456 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.456 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/17 2023-11-28T18:29:23.462 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.464 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.464 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.464 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/18 2023-11-28T18:29:23.470 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.472 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.472 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.472 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/19 2023-11-28T18:29:23.479 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.481 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.481 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.481 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/20 2023-11-28T18:29:23.487 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.489 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.489 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.489 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/21 2023-11-28T18:29:23.495 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.498 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.498 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.498 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/22 2023-11-28T18:29:23.503 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.505 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.506 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.506 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/23 2023-11-28T18:29:23.511 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.514 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/24 2023-11-28T18:29:23.519 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.522 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.522 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.522 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/25 2023-11-28T18:29:23.528 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.530 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.530 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.530 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/26 2023-11-28T18:29:23.536 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.538 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.538 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.538 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/27 2023-11-28T18:29:23.544 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.546 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.546 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.546 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/28 2023-11-28T18:29:23.552 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.554 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.555 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.555 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/29 2023-11-28T18:29:23.561 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.563 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.563 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.563 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/30 2023-11-28T18:29:23.569 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.571 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.571 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.572 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/31 2023-11-28T18:29:23.577 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.579 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.580 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.580 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/32 2023-11-28T18:29:23.585 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.587 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.588 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.588 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/33 2023-11-28T18:29:23.594 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.596 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.596 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.596 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/34 2023-11-28T18:29:23.602 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.604 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.604 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.604 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/35 2023-11-28T18:29:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/36 2023-11-28T18:29:23.621 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.621 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.622 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.622 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/37 2023-11-28T18:29:23.626 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.628 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.628 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.629 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/38 2023-11-28T18:29:23.635 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.637 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.637 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.637 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/39 2023-11-28T18:29:23.642 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.645 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.645 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.645 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/40 2023-11-28T18:29:23.650 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.653 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.653 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.653 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/41 2023-11-28T18:29:23.659 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.661 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.661 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.661 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/42 2023-11-28T18:29:23.668 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.670 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.670 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.670 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/43 2023-11-28T18:29:23.676 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.678 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.678 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.679 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/44 2023-11-28T18:29:23.684 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.687 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.687 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.687 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/45 2023-11-28T18:29:23.693 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.695 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.695 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.695 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/46 2023-11-28T18:29:23.701 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.704 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.704 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.704 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/47 2023-11-28T18:29:23.710 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.712 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.712 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.712 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/48 2023-11-28T18:29:23.718 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.720 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.720 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.720 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/49 2023-11-28T18:29:23.726 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.729 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.729 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.729 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/50 2023-11-28T18:29:23.734 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.737 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.737 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.737 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/51 2023-11-28T18:29:23.742 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.745 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.745 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.745 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/52 2023-11-28T18:29:23.751 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.753 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.753 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.753 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/53 2023-11-28T18:29:23.759 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.761 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.761 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.761 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/54 2023-11-28T18:29:23.767 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.769 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.769 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.770 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/55 2023-11-28T18:29:23.775 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.778 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.778 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.778 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/56 2023-11-28T18:29:23.784 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.786 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.786 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.786 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/57 2023-11-28T18:29:23.792 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/58 2023-11-28T18:29:23.800 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.802 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.802 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.802 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/59 2023-11-28T18:29:23.808 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.810 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.811 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.811 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/60 2023-11-28T18:29:23.816 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.819 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.819 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.819 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/61 2023-11-28T18:29:23.825 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.827 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.827 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.827 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/62 2023-11-28T18:29:23.833 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.835 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.835 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.836 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/63 2023-11-28T18:29:23.841 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.844 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.844 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.844 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/64 2023-11-28T18:29:23.850 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.852 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.852 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.852 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/65 2023-11-28T18:29:23.858 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.860 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.860 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.860 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/66 2023-11-28T18:29:23.866 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.868 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.868 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.869 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/67 2023-11-28T18:29:23.874 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.876 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.876 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.876 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/68 2023-11-28T18:29:23.882 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.885 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.885 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.885 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/69 2023-11-28T18:29:23.893 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.893 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.893 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.894 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/70 2023-11-28T18:29:23.899 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.901 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.901 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.901 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/71 2023-11-28T18:29:23.907 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.910 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.910 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.910 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/72 2023-11-28T18:29:23.915 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.918 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.918 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.918 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/73 2023-11-28T18:29:23.923 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.926 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.926 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.926 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/74 2023-11-28T18:29:23.932 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.934 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.934 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.934 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/75 2023-11-28T18:29:23.940 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.942 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.942 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.942 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/76 2023-11-28T18:29:23.948 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.950 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.950 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.950 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/77 2023-11-28T18:29:23.956 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.958 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.958 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.959 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/78 2023-11-28T18:29:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.967 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.967 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.967 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/79 2023-11-28T18:29:23.973 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.975 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.975 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.975 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/80 2023-11-28T18:29:23.981 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.983 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.983 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.983 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/81 2023-11-28T18:29:23.989 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:23.991 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:23.991 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:23.991 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/82 2023-11-28T18:29:23.998 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:24.000 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:24.000 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:24.000 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/83 2023-11-28T18:29:24.006 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:24.008 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:24.008 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:24.008 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/84 2023-11-28T18:29:24.014 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:24.016 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:24.016 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:24.016 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/85 2023-11-28T18:29:24.023 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:24.025 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:24.025 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:24.025 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/86 2023-11-28T18:29:24.031 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:24.033 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:24.033 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:24.033 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/87 2023-11-28T18:29:24.039 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:24.041 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:24.041 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:24.041 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/88 2023-11-28T18:29:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:24.050 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:24.050 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:24.050 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/89 2023-11-28T18:29:24.056 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:24.058 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:24.059 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:24.059 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/90 2023-11-28T18:29:24.065 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:24.067 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:24.067 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:24.067 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/91 2023-11-28T18:29:24.073 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:24.077 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:24.077 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:24.077 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/92 2023-11-28T18:29:24.082 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:24.084 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:24.084 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:24.085 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/93 2023-11-28T18:29:24.090 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:24.092 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:24.092 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:24.093 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/94 2023-11-28T18:29:24.099 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:24.100 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:24.101 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:24.101 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/95 2023-11-28T18:29:24.107 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:24.110 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:24.110 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:24.110 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/96 2023-11-28T18:29:24.116 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:24.119 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:24.119 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:24.119 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/97 2023-11-28T18:29:24.125 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:24.128 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:24.128 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:24.128 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/98 2023-11-28T18:29:24.134 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:24.136 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:24.137 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:24.137 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/99 2023-11-28T18:29:24.143 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:29:24.145 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:29:24.145 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:29:24.147 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-11-28T18:29:24.302 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-28T18:29:24.302 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:24.303 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-11-28T18:29:24.464 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-28T18:29:24.464 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:24.465 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-11-28T18:29:24.619 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"566","version":"566","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"123","version":"123","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2023-11-28T18:29:24.620 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "566", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "566" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "123", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "123" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:24.620 INFO:tasks.cephfs.cephfs_test_case:len=0 [] 2023-11-28T18:29:25.300 INFO:tasks.ceph.mds.a.smithi105.stderr:2023-11-28T18:29:25.297+0000 7f2ea22d3640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-11-28T18:29:29.622 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-11-28T18:29:29.788 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-28T18:29:29.788 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:29.789 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-11-28T18:29:29.953 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-28T18:29:29.954 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:29.955 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-11-28T18:29:30.110 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"572","version":"572","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"123","version":"123","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.101*","snapid_first":2,"projected_version":"422","version":"422","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.001*","snapid_first":2,"projected_version":"428","version":"428","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.010*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.110*","snapid_first":2,"projected_version":"442","version":"442","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.111*","snapid_first":2,"projected_version":"416","version":"416","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2023-11-28T18:29:30.111 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "572", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "572" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "123", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "123" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.101*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "422", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "422" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.001*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "428", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "428" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "426", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "442", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "442" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "416", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "416" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:30.111 INFO:tasks.cephfs.cephfs_test_case:len=5 [{'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x10000000230.101*', 'snapid_first': 2, 'projected_version': '422', 'version': '422', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x10000000230.001*', 'snapid_first': 2, 'projected_version': '428', 'version': '428', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x10000000230.010*', 'snapid_first': 2, 'projected_version': '426', 'version': '426', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x10000000230.110*', 'snapid_first': 2, 'projected_version': '442', 'version': '442', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x10000000230.111*', 'snapid_first': 2, 'projected_version': '416', 'version': '416', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}] 2023-11-28T18:29:32.436 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:29:32.441 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:29:35.112 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-11-28T18:29:35.280 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"572","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000230.011*","snapid_first":2,"projected_version":"430","version":"430","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-11-28T18:29:35.280 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "572" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x10000000230.011*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "430", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "430" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:35.281 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-11-28T18:29:35.446 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"572","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":2},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.100*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.000*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-11-28T18:29:35.446 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "snapid_first": 2, "states": [], "version": "572" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "426", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.000*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "426", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:35.447 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-11-28T18:29:35.601 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"572","version":"572","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":2,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"123","version":"123","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.000*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.101*","snapid_first":2,"projected_version":"422","version":"422","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.001*","snapid_first":2,"projected_version":"428","version":"428","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.010*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.100*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000230.011*","snapid_first":2,"projected_version":"430","version":"430","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.110*","snapid_first":2,"projected_version":"442","version":"442","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.111*","snapid_first":2,"projected_version":"416","version":"416","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2023-11-28T18:29:35.602 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 2, "2": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "572", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "572" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "123", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "123" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.000*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.101*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "422", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "422" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.001*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "428", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "428" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "426", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x10000000230.011*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "430", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "430" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "442", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "442" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "416", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "416" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:35.602 INFO:tasks.cephfs.cephfs_test_case:len=8 [{'is_auth': True, 'auth_first': 2, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 2, 'dir': {'path': '/tree', 'dirfrag': '0x10000000230.011*', 'snapid_first': 2, 'projected_version': '430', 'version': '430', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '2', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {'0': 1}}, 'replica_state': {'authority': [2, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'ptrwaiter': 0, 'child': 1, 'frozen': 0, 'subtree': 1, 'importing': 0, 'replicated': 1, 'dirty': 1}, 'nref': 4}}, {'is_auth': True, 'auth_first': 1, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 1, 'dir': {'path': '/tree', 'dirfrag': '0x10000000230.100*', 'snapid_first': 2, 'projected_version': '426', 'version': '426', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '1', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {'0': 1}}, 'replica_state': {'authority': [1, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'ptrwaiter': 0, 'child': 1, 'frozen': 0, 'subtree': 1, 'importing': 0, 'replicated': 1, 'dirty': 1}, 'nref': 4}}, {'is_auth': True, 'auth_first': 1, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 1, 'dir': {'path': '/tree', 'dirfrag': '0x10000000230.000*', 'snapid_first': 2, 'projected_version': '426', 'version': '426', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '1', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {'0': 1}}, 'replica_state': {'authority': [1, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'ptrwaiter': 0, 'child': 1, 'frozen': 0, 'subtree': 1, 'importing': 0, 'replicated': 1, 'dirty': 1}, 'nref': 4}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x10000000230.101*', 'snapid_first': 2, 'projected_version': '422', 'version': '422', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x10000000230.001*', 'snapid_first': 2, 'projected_version': '428', 'version': '428', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x10000000230.010*', 'snapid_first': 2, 'projected_version': '426', 'version': '426', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x10000000230.110*', 'snapid_first': 2, 'projected_version': '442', 'version': '442', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x10000000230.111*', 'snapid_first': 2, 'projected_version': '416', 'version': '416', 'committing_version': '401', 'committed_version': '401', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 0}, 'nref': 3}}] 2023-11-28T18:29:35.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:35.603 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/0) 2023-11-28T18:29:35.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:35.622 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/1) 2023-11-28T18:29:35.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:35.679 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/2) 2023-11-28T18:29:35.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:35.734 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/3) 2023-11-28T18:29:35.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:35.787 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/4) 2023-11-28T18:29:35.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:35.840 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/5) 2023-11-28T18:29:35.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:35.895 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/6) 2023-11-28T18:29:35.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:35.951 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/7) 2023-11-28T18:29:36.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:36.003 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/8) 2023-11-28T18:29:36.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:36.059 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/9) 2023-11-28T18:29:36.115 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-11-28T18:29:36.274 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"572","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000230.011*","snapid_first":2,"projected_version":"437","version":"430","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":4,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}}] 2023-11-28T18:29:36.274 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "572" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 4, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x10000000230.011*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/tree", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "437", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "430" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:36.275 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-11-28T18:29:36.431 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"572","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":2},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.100*","snapid_first":2,"projected_version":"429","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.000*","snapid_first":2,"projected_version":"433","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":4,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}}] 2023-11-28T18:29:36.432 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "snapid_first": 2, "states": [], "version": "572" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 2, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/tree", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "429", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 4, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.000*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/tree", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "433", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:36.432 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-11-28T18:29:36.590 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"580","version":"574","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":2,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":3,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"123","version":"123","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.000*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.101*","snapid_first":2,"projected_version":"423","version":"423","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.001*","snapid_first":2,"projected_version":"429","version":"429","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.010*","snapid_first":2,"projected_version":"427","version":"427","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.100*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000230.011*","snapid_first":2,"projected_version":"430","version":"430","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.110*","snapid_first":2,"projected_version":"447","version":"443","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.111*","snapid_first":2,"projected_version":"419","version":"417","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":1},"nref":4}}] 2023-11-28T18:29:36.591 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 3, "auth_state": { "replicas": { "1": 2, "2": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 5, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "580", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "574" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "123", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "123" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.000*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.101*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "423", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "423" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.001*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "429", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "429" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "427", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "427" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x10000000230.011*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "430", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "430" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 2, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "447", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "443" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "419", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "417" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:36.592 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/tree/0', 1), ('/tree/1', 1), ('/tree/2', 1), ('/tree/3', 1), ('/tree/4', 1), ('/tree/5', 1), ('/tree/6', 1), ('/tree/7', 1), ('/tree/8', 1), ('/tree/9', 1)] 2023-11-28T18:29:38.593 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-11-28T18:29:38.760 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"572","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000230.011*","snapid_first":2,"projected_version":"437","version":"437","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}}] 2023-11-28T18:29:38.761 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "572" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x10000000230.011*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "437", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "437" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:38.761 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-11-28T18:29:38.926 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"572","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":2},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.100*","snapid_first":2,"projected_version":"429","version":"429","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.000*","snapid_first":2,"projected_version":"433","version":"433","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}}] 2023-11-28T18:29:38.927 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "snapid_first": 2, "states": [], "version": "572" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "429", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "429" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.000*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "433", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "433" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:38.927 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-11-28T18:29:39.083 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"580","version":"574","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":2,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":3,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"123","version":"123","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.000*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.101*","snapid_first":2,"projected_version":"423","version":"423","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.001*","snapid_first":2,"projected_version":"429","version":"429","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.010*","snapid_first":2,"projected_version":"427","version":"427","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.100*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000230.011*","snapid_first":2,"projected_version":"430","version":"430","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.110*","snapid_first":2,"projected_version":"447","version":"443","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.111*","snapid_first":2,"projected_version":"419","version":"417","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":1},"nref":4}}] 2023-11-28T18:29:39.084 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 3, "auth_state": { "replicas": { "1": 2, "2": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 5, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "580", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "574" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "123", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "123" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.000*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.101*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "423", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "423" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.001*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "429", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "429" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "427", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "427" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x10000000230.011*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "430", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "430" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 2, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "447", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "443" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "419", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "417" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:39.084 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/tree/0', 1), ('/tree/1', 1), ('/tree/2', 1), ('/tree/3', 1), ('/tree/4', 1), ('/tree/5', 1), ('/tree/6', 1), ('/tree/7', 1), ('/tree/8', 1), ('/tree/9', 1)] 2023-11-28T18:29:41.085 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-11-28T18:29:41.251 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"572","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000230.011*","snapid_first":2,"projected_version":"437","version":"437","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}}] 2023-11-28T18:29:41.252 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "572" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x10000000230.011*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "437", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "437" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:41.252 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-11-28T18:29:41.422 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"572","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":2},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.100*","snapid_first":2,"projected_version":"429","version":"429","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.000*","snapid_first":2,"projected_version":"433","version":"433","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}}] 2023-11-28T18:29:41.423 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "snapid_first": 2, "states": [], "version": "572" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "429", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "429" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.000*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "433", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "433" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:41.423 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-11-28T18:29:41.580 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"580","version":"580","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":2,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"123","version":"123","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.000*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.101*","snapid_first":2,"projected_version":"423","version":"423","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.001*","snapid_first":2,"projected_version":"429","version":"429","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.010*","snapid_first":2,"projected_version":"427","version":"427","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.100*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000230.011*","snapid_first":2,"projected_version":"430","version":"430","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.110*","snapid_first":2,"projected_version":"447","version":"447","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.111*","snapid_first":2,"projected_version":"419","version":"419","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2023-11-28T18:29:41.581 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 2, "2": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "580", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "580" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "123", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "123" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.000*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.101*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "423", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "423" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.001*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "429", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "429" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "427", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "427" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x10000000230.011*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "430", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "430" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "447", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "447" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "419", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "419" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:41.581 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/tree/0', 1), ('/tree/1', 1), ('/tree/2', 1), ('/tree/3', 1), ('/tree/4', 1), ('/tree/5', 1), ('/tree/6', 1), ('/tree/7', 1), ('/tree/8', 1), ('/tree/9', 1)] 2023-11-28T18:29:43.582 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-11-28T18:29:43.747 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"572","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000230.011*","snapid_first":2,"projected_version":"438","version":"438","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}}] 2023-11-28T18:29:43.747 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "572" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x10000000230.011*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "438", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "438" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:43.748 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-11-28T18:29:43.915 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"572","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":2},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.100*","snapid_first":2,"projected_version":"430","version":"429","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.000*","snapid_first":2,"projected_version":"434","version":"433","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/8","dirfrag":"0x10000000241","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1},"nref":3}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/3","dirfrag":"0x10000000237","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1},"nref":3}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/9","dirfrag":"0x10000000243","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1},"nref":3}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/0","dirfrag":"0x10000000231","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1},"nref":3}}] 2023-11-28T18:29:43.916 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "snapid_first": 2, "states": [], "version": "572" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/tree", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "430", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "429" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.000*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/tree", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "434", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "433" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000241", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree/8", "pins": { "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000237", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree/3", "pins": { "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000243", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree/9", "pins": { "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000231", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree/0", "pins": { "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:43.916 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-11-28T18:29:44.070 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"582","version":"582","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":2,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"123","version":"123","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.000*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.101*","snapid_first":2,"projected_version":"424","version":"424","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.001*","snapid_first":2,"projected_version":"430","version":"430","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.010*","snapid_first":2,"projected_version":"428","version":"428","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.100*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000230.011*","snapid_first":2,"projected_version":"430","version":"430","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.110*","snapid_first":2,"projected_version":"448","version":"448","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.111*","snapid_first":2,"projected_version":"420","version":"420","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2023-11-28T18:29:44.071 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 2, "2": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "582", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "582" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "123", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "123" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.000*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.101*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "424", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "424" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.001*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "430", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "430" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "428", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "428" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x10000000230.011*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "430", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "430" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "448", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "448" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "420", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "420" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:44.072 INFO:tasks.cephfs.cephfs_test_case:[('/tree/0', 1), ('/tree/3', 1), ('/tree/8', 1), ('/tree/9', 1)] =?= [('/tree/0', 1), ('/tree/1', 1), ('/tree/2', 1), ('/tree/3', 1), ('/tree/4', 1), ('/tree/5', 1), ('/tree/6', 1), ('/tree/7', 1), ('/tree/8', 1), ('/tree/9', 1)] 2023-11-28T18:29:46.072 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-11-28T18:29:46.240 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"572","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000230.011*","snapid_first":2,"projected_version":"438","version":"438","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"1":2}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/2","dirfrag":"0x10000000235","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/4","dirfrag":"0x10000000239","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/1","dirfrag":"0x10000000233","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2023-11-28T18:29:46.241 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "572" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "1": 2 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x10000000230.011*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "438", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "438" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000235", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/2", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000239", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/4", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000233", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/1", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2023-11-28T18:29:46.241 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-11-28T18:29:46.407 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"572","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":2},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.100*","snapid_first":2,"projected_version":"430","version":"430","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.000*","snapid_first":2,"projected_version":"434","version":"434","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/8","dirfrag":"0x10000000241","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1},"nref":3}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/3","dirfrag":"0x10000000237","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1},"nref":3}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/9","dirfrag":"0x10000000243","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1},"nref":3}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/0","dirfrag":"0x10000000231","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1},"nref":3}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000230.011*","snapid_first":2,"projected_version":"0","version":"438","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":2},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/4","dirfrag":"0x10000000239","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/6","dirfrag":"0x1000000023d","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/7","dirfrag":"0x1000000023f","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/2","dirfrag":"0x10000000235","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/1","dirfrag":"0x10000000233","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/5","dirfrag":"0x1000000023b","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-11-28T18:29:46.408 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "snapid_first": 2, "states": [], "version": "572" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "430", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "430" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.000*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "434", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "434" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000241", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree/8", "pins": { "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000237", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree/3", "pins": { "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000243", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree/9", "pins": { "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000231", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree/0", "pins": { "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000230.011*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/tree", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 2 }, "snapid_first": 2, "states": [], "version": "438" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000239", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000023d", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000023f", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/7", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000235", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/2", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000233", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/1", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000023b", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/5", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:46.409 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-11-28T18:29:46.565 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"583","version":"583","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":2,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"123","version":"123","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/5","dirfrag":"0x1000000023b","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/6","dirfrag":"0x1000000023d","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/7","dirfrag":"0x1000000023f","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.000*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.101*","snapid_first":2,"projected_version":"424","version":"424","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.001*","snapid_first":2,"projected_version":"430","version":"430","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.010*","snapid_first":2,"projected_version":"428","version":"428","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000230.100*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000230.011*","snapid_first":2,"projected_version":"430","version":"430","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.110*","snapid_first":2,"projected_version":"448","version":"448","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":2}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000230.111*","snapid_first":2,"projected_version":"420","version":"420","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}}] 2023-11-28T18:29:46.567 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 2, "2": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "583", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "583" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "123", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "123" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000023b", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/5", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000023d", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000023f", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/7", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.000*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.101*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "424", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "424" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.001*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "430", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "430" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "428", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "428" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x10000000230.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x10000000230.011*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "430", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "430" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 2 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "replicated": 1, "subtree": 1 }, "projected_version": "448", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "448" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x10000000230.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "replicated": 1, "subtree": 1 }, "projected_version": "420", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "420" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:29:46.567 INFO:tasks.cephfs.cephfs_test_case:[('/tree/0', 1), ('/tree/1', 1), ('/tree/2', 1), ('/tree/3', 1), ('/tree/4', 1), ('/tree/5', 1), ('/tree/6', 1), ('/tree/7', 1), ('/tree/8', 1), ('/tree/9', 1)] =?= [('/tree/0', 1), ('/tree/1', 1), ('/tree/2', 1), ('/tree/3', 1), ('/tree/4', 1), ('/tree/5', 1), ('/tree/6', 1), ('/tree/7', 1), ('/tree/8', 1), ('/tree/9', 1)] 2023-11-28T18:29:46.567 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T18:29:46.567 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:29:46.629 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:29:46.707 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T18:29:46.707 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:29:46.707 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:29:46.717 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:29:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.889+0000 7f761f217640 1 Processor -- start 2023-11-28T18:29:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.889+0000 7f761f217640 1 -- start start 2023-11-28T18:29:46.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.893+0000 7f761f217640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7618159f40 0x7f76180caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:46.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.893+0000 7f761f217640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f76180ad850 0x7f76180701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:46.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.893+0000 7f761f217640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f761806bf50 con 0x7f76180cc760 2023-11-28T18:29:46.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.893+0000 7f761f217640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76180ccbc0 con 0x7f7618159f40 2023-11-28T18:29:46.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.893+0000 7f761f217640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f761815a320 con 0x7f76180ad850 2023-11-28T18:29:46.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.893+0000 7f761cfb3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7618159f40 0x7f76180caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:46.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.893+0000 7f761cfb3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7618159f40 0x7f76180caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38320/0 (socket says 172.21.15.105:38320) 2023-11-28T18:29:46.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.893+0000 7f761cfb3640 1 -- 172.21.15.105:0/2125753735 learned_addr learned my addr 172.21.15.105:0/2125753735 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:46.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.893+0000 7f761d7b4640 1 --2- 172.21.15.105:0/2125753735 >> v2:172.21.15.148:3300/0 conn(0x7f76180ad850 0x7f76180701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:46.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.893+0000 7f761cfb3640 1 -- 172.21.15.105:0/2125753735 >> v2:172.21.15.148:3300/0 conn(0x7f76180ad850 msgr2=0x7f76180701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:46.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.893+0000 7f761cfb3640 1 --2- 172.21.15.105:0/2125753735 >> v2:172.21.15.148:3300/0 conn(0x7f76180ad850 0x7f76180701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:46.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.893+0000 7f761cfb3640 1 -- 172.21.15.105:0/2125753735 >> v1:172.21.15.148:6789/0 conn(0x7f76180cc760 legacy=0x7f76180ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:46.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.893+0000 7f761cfb3640 1 -- 172.21.15.105:0/2125753735 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7618071380 con 0x7f7618159f40 2023-11-28T18:29:46.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.897+0000 7f761cfb3640 1 --2- 172.21.15.105:0/2125753735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7618159f40 0x7f76180caf60 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f76180b0850 tx=0x7f761401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b0016d699dd94a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:46.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.897+0000 7f760f7fe640 1 -- 172.21.15.105:0/2125753735 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76140ab640 con 0x7f7618159f40 2023-11-28T18:29:46.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.897+0000 7f760f7fe640 1 -- 172.21.15.105:0/2125753735 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76140a7070 con 0x7f7618159f40 2023-11-28T18:29:46.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.897+0000 7f760f7fe640 1 -- 172.21.15.105:0/2125753735 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76140ab9c0 con 0x7f7618159f40 2023-11-28T18:29:46.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.897+0000 7f761f217640 1 -- 172.21.15.105:0/2125753735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7618159f40 msgr2=0x7f76180caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:46.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.897+0000 7f761f217640 1 --2- 172.21.15.105:0/2125753735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7618159f40 0x7f76180caf60 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f76180b0850 tx=0x7f761401c440 comp rx=0 tx=0).stop 2023-11-28T18:29:46.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.897+0000 7f761f217640 1 -- 172.21.15.105:0/2125753735 shutdown_connections 2023-11-28T18:29:46.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.897+0000 7f761f217640 1 --2- 172.21.15.105:0/2125753735 >> v2:172.21.15.148:3300/0 conn(0x7f76180ad850 0x7f76180701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:46.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.897+0000 7f761f217640 1 --2- 172.21.15.105:0/2125753735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7618159f40 0x7f76180caf60 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:46.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.897+0000 7f761f217640 1 -- 172.21.15.105:0/2125753735 >> 172.21.15.105:0/2125753735 conn(0x7f76180ceb90 msgr2=0x7f76180cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:46.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.897+0000 7f761f217640 1 -- 172.21.15.105:0/2125753735 shutdown_connections 2023-11-28T18:29:46.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.897+0000 7f761f217640 1 -- 172.21.15.105:0/2125753735 wait complete. 2023-11-28T18:29:46.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.897+0000 7f761f217640 1 Processor -- start 2023-11-28T18:29:46.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.897+0000 7f761f217640 1 -- start start 2023-11-28T18:29:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f761f217640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76180ad850 0x7f7618155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f761f217640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76180cc760 0x7f76180c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f761f217640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7618159f40 0x7f7618150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f761f217640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76181562a0 con 0x7f76180ad850 2023-11-28T18:29:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f761f217640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7618149c50 con 0x7f76180cc760 2023-11-28T18:29:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f761f217640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7618154f30 con 0x7f7618159f40 2023-11-28T18:29:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f761cfb3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76180ad850 0x7f7618155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f761cfb3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76180ad850 0x7f7618155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38326/0 (socket says 172.21.15.105:38326) 2023-11-28T18:29:46.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f761cfb3640 1 -- 172.21.15.105:0/2809635211 learned_addr learned my addr 172.21.15.105:0/2809635211 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:46.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f760ffff640 1 --2- 172.21.15.105:0/2809635211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76180cc760 0x7f76180c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:46.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f761d7b4640 1 --2- 172.21.15.105:0/2809635211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7618159f40 0x7f7618150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:46.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f761cfb3640 1 -- 172.21.15.105:0/2809635211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7618159f40 msgr2=0x7f7618150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:46.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f761cfb3640 1 --2- 172.21.15.105:0/2809635211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7618159f40 0x7f7618150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:46.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f761cfb3640 1 -- 172.21.15.105:0/2809635211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76180cc760 msgr2=0x7f76180c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:46.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f761cfb3640 1 --2- 172.21.15.105:0/2809635211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76180cc760 0x7f76180c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:46.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f761cfb3640 1 -- 172.21.15.105:0/2809635211 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76140094d0 con 0x7f76180ad850 2023-11-28T18:29:46.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f761cfb3640 1 --2- 172.21.15.105:0/2809635211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76180ad850 0x7f7618155d60 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f7614019050 tx=0x7f761401ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:46.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f760d7fa640 1 -- 172.21.15.105:0/2809635211 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76140be3f0 con 0x7f76180ad850 2023-11-28T18:29:46.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f760d7fa640 1 -- 172.21.15.105:0/2809635211 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76140be550 con 0x7f76180ad850 2023-11-28T18:29:46.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f760d7fa640 1 -- 172.21.15.105:0/2809635211 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76140be900 con 0x7f76180ad850 2023-11-28T18:29:46.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f761f217640 1 -- 172.21.15.105:0/2809635211 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76180c8e60 con 0x7f76180ad850 2023-11-28T18:29:46.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f761f217640 1 -- 172.21.15.105:0/2809635211 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76180c9420 con 0x7f76180ad850 2023-11-28T18:29:46.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f761f217640 1 -- 172.21.15.105:0/2809635211 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76180c9610 con 0x7f76180ad850 2023-11-28T18:29:46.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.901+0000 7f760d7fa640 1 -- 172.21.15.105:0/2809635211 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f76140b43f0 con 0x7f76180ad850 2023-11-28T18:29:46.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.905+0000 7f760d7fa640 1 --2- 172.21.15.105:0/2809635211 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7604066900 0x7f7604068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:46.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.905+0000 7f760d7fa640 1 -- 172.21.15.105:0/2809635211 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f761411ea40 con 0x7f76180ad850 2023-11-28T18:29:46.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.905+0000 7f760ffff640 1 --2- 172.21.15.105:0/2809635211 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7604066900 0x7f7604068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:46.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.905+0000 7f760ffff640 1 --2- 172.21.15.105:0/2809635211 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7604066900 0x7f7604068dc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7618155210 tx=0x7f7608009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:46.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:46.909+0000 7f760d7fa640 1 -- 172.21.15.105:0/2809635211 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f76140ebad0 con 0x7f76180ad850 2023-11-28T18:29:47.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.061+0000 7f761f217640 1 -- 172.21.15.105:0/2809635211 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7618067570 con 0x7f76180ad850 2023-11-28T18:29:47.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.061+0000 7f760d7fa640 1 -- 172.21.15.105:0/2809635211 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16971 (secure 0 0 0) 0x7f76140c4090 con 0x7f76180ad850 2023-11-28T18:29:47.066 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:29:47.066 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":92,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:29:16.406699+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:29:08.048399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-28T18:29:08.699964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":5}]},{"pgid":"14.17","mappings":[{"from":1,"to":2}]},{"pgid":"14.21","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:29:47.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.061+0000 7f761f217640 1 -- 172.21.15.105:0/2809635211 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7604066900 msgr2=0x7f7604068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:47.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.061+0000 7f761f217640 1 --2- 172.21.15.105:0/2809635211 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7604066900 0x7f7604068dc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7618155210 tx=0x7f7608009040 comp rx=0 tx=0).stop 2023-11-28T18:29:47.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.061+0000 7f761f217640 1 -- 172.21.15.105:0/2809635211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76180ad850 msgr2=0x7f7618155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:47.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.061+0000 7f761f217640 1 --2- 172.21.15.105:0/2809635211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76180ad850 0x7f7618155d60 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f7614019050 tx=0x7f761401ac80 comp rx=0 tx=0).stop 2023-11-28T18:29:47.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.065+0000 7f761f217640 1 -- 172.21.15.105:0/2809635211 shutdown_connections 2023-11-28T18:29:47.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.065+0000 7f761f217640 1 --2- 172.21.15.105:0/2809635211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7618159f40 0x7f7618150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.065+0000 7f761f217640 1 --2- 172.21.15.105:0/2809635211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76180cc760 0x7f76180c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.065+0000 7f761f217640 1 --2- 172.21.15.105:0/2809635211 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7604066900 0x7f7604068dc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.065+0000 7f761f217640 1 --2- 172.21.15.105:0/2809635211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76180ad850 0x7f7618155d60 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.065+0000 7f761f217640 1 -- 172.21.15.105:0/2809635211 >> 172.21.15.105:0/2809635211 conn(0x7f76180ceb90 msgr2=0x7f76180cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:47.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.065+0000 7f761f217640 1 -- 172.21.15.105:0/2809635211 shutdown_connections 2023-11-28T18:29:47.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.065+0000 7f761f217640 1 -- 172.21.15.105:0/2809635211 wait complete. 2023-11-28T18:29:47.087 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:29:08.048399+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:29:08.699964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:29:47.088 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:29:47.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.225+0000 7ff049c5a640 1 Processor -- start 2023-11-28T18:29:47.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.225+0000 7ff049c5a640 1 -- start start 2023-11-28T18:29:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.225+0000 7ff049c5a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0440caf60 0x7ff0440cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.225+0000 7ff049c5a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff0440ad850 0x7ff0440701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.225+0000 7ff049c5a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff04406bf50 con 0x7ff044159f40 2023-11-28T18:29:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.225+0000 7ff049c5a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0440cb3c0 con 0x7ff0440ad850 2023-11-28T18:29:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.225+0000 7ff049c5a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff04415a320 con 0x7ff0440caf60 2023-11-28T18:29:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.225+0000 7ff0437fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff044159f40 0x7ff0440ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40438/0 (socket says 172.21.15.105:40438) 2023-11-28T18:29:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.225+0000 7ff0437fe640 1 -- 172.21.15.105:0/1680321507 learned_addr learned my addr 172.21.15.105:0/1680321507 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.225+0000 7ff043fff640 1 --2- 172.21.15.105:0/1680321507 >> v2:172.21.15.105:3300/0 conn(0x7ff0440ad850 0x7ff0440701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.225+0000 7ff042ffd640 1 --2- 172.21.15.105:0/1680321507 >> v2:172.21.15.148:3300/0 conn(0x7ff0440caf60 0x7ff0440cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:47.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.225+0000 7ff0427fc640 1 -- 172.21.15.105:0/1680321507 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 297208299 0 0) 0x7ff04406bf50 con 0x7ff044159f40 2023-11-28T18:29:47.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.225+0000 7ff0427fc640 1 -- 172.21.15.105:0/1680321507 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff02c003530 con 0x7ff044159f40 2023-11-28T18:29:47.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.225+0000 7ff043fff640 1 -- 172.21.15.105:0/1680321507 >> v2:172.21.15.148:3300/0 conn(0x7ff0440caf60 msgr2=0x7ff0440cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:47.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.225+0000 7ff043fff640 1 --2- 172.21.15.105:0/1680321507 >> v2:172.21.15.148:3300/0 conn(0x7ff0440caf60 0x7ff0440cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.225+0000 7ff043fff640 1 -- 172.21.15.105:0/1680321507 >> v1:172.21.15.105:6789/0 conn(0x7ff044159f40 legacy=0x7ff0440ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:47.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.225+0000 7ff043fff640 1 -- 172.21.15.105:0/1680321507 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff044071380 con 0x7ff0440ad850 2023-11-28T18:29:47.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.225+0000 7ff042ffd640 1 --2- 172.21.15.105:0/1680321507 >> v2:172.21.15.148:3300/0 conn(0x7ff0440caf60 0x7ff0440cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:47.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.229+0000 7ff043fff640 1 --2- 172.21.15.105:0/1680321507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0440ad850 0x7ff0440701d0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ff04409d0d0 tx=0x7ff03c01c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7aee064f5644bfb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:47.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.229+0000 7ff0427fc640 1 -- 172.21.15.105:0/1680321507 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c0a5910 con 0x7ff0440ad850 2023-11-28T18:29:47.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.229+0000 7ff0427fc640 1 -- 172.21.15.105:0/1680321507 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03c0a5a70 con 0x7ff0440ad850 2023-11-28T18:29:47.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.229+0000 7ff0427fc640 1 -- 172.21.15.105:0/1680321507 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c0a5e00 con 0x7ff0440ad850 2023-11-28T18:29:47.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.229+0000 7ff049c5a640 1 -- 172.21.15.105:0/1680321507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0440ad850 msgr2=0x7ff0440701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:47.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.229+0000 7ff049c5a640 1 --2- 172.21.15.105:0/1680321507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0440ad850 0x7ff0440701d0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ff04409d0d0 tx=0x7ff03c01c850 comp rx=0 tx=0).stop 2023-11-28T18:29:47.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.229+0000 7ff049c5a640 1 -- 172.21.15.105:0/1680321507 shutdown_connections 2023-11-28T18:29:47.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.229+0000 7ff049c5a640 1 --2- 172.21.15.105:0/1680321507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0440ad850 0x7ff0440701d0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.229+0000 7ff049c5a640 1 --2- 172.21.15.105:0/1680321507 >> v2:172.21.15.148:3300/0 conn(0x7ff0440caf60 0x7ff0440cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.229+0000 7ff049c5a640 1 -- 172.21.15.105:0/1680321507 >> 172.21.15.105:0/1680321507 conn(0x7ff0440ceb90 msgr2=0x7ff0440cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:47.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.229+0000 7ff049c5a640 1 -- 172.21.15.105:0/1680321507 shutdown_connections 2023-11-28T18:29:47.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.229+0000 7ff049c5a640 1 -- 172.21.15.105:0/1680321507 wait complete. 2023-11-28T18:29:47.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff049c5a640 1 Processor -- start 2023-11-28T18:29:47.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff049c5a640 1 -- start start 2023-11-28T18:29:47.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff049c5a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0440ad850 0x7ff04414ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:47.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff049c5a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0440caf60 0x7ff044099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:47.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff049c5a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff044159f40 0x7ff044099f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:47.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff049c5a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04409a8b0 con 0x7ff0440caf60 2023-11-28T18:29:47.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff049c5a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04409b680 con 0x7ff0440ad850 2023-11-28T18:29:47.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff049c5a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04409b8d0 con 0x7ff044159f40 2023-11-28T18:29:47.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff042ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0440caf60 0x7ff044099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:47.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff042ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0440caf60 0x7ff044099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38344/0 (socket says 172.21.15.105:38344) 2023-11-28T18:29:47.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff042ffd640 1 -- 172.21.15.105:0/4101591062 learned_addr learned my addr 172.21.15.105:0/4101591062 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:47.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff0437fe640 1 --2- 172.21.15.105:0/4101591062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0440ad850 0x7ff04414ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:47.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff043fff640 1 --2- 172.21.15.105:0/4101591062 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff044159f40 0x7ff044099f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:47.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff042ffd640 1 -- 172.21.15.105:0/4101591062 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff044159f40 msgr2=0x7ff044099f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:47.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff042ffd640 1 --2- 172.21.15.105:0/4101591062 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff044159f40 0x7ff044099f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff042ffd640 1 -- 172.21.15.105:0/4101591062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0440ad850 msgr2=0x7ff04414ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:47.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff042ffd640 1 --2- 172.21.15.105:0/4101591062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0440ad850 0x7ff04414ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff042ffd640 1 -- 172.21.15.105:0/4101591062 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03c009a70 con 0x7ff0440caf60 2023-11-28T18:29:47.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff043fff640 1 --2- 172.21.15.105:0/4101591062 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff044159f40 0x7ff044099f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:47.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff042ffd640 1 --2- 172.21.15.105:0/4101591062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0440caf60 0x7ff044099a50 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff03800e840 tx=0x7ff03800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:47.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff02bfff640 1 -- 172.21.15.105:0/4101591062 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03800c8d0 con 0x7ff0440caf60 2023-11-28T18:29:47.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff02bfff640 1 -- 172.21.15.105:0/4101591062 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03800ca30 con 0x7ff0440caf60 2023-11-28T18:29:47.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff02bfff640 1 -- 172.21.15.105:0/4101591062 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03800cdc0 con 0x7ff0440caf60 2023-11-28T18:29:47.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff049c5a640 1 -- 172.21.15.105:0/4101591062 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04409bbb0 con 0x7ff0440caf60 2023-11-28T18:29:47.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.233+0000 7ff049c5a640 1 -- 172.21.15.105:0/4101591062 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04409c170 con 0x7ff0440caf60 2023-11-28T18:29:47.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.237+0000 7ff049c5a640 1 -- 172.21.15.105:0/4101591062 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff044067570 con 0x7ff0440caf60 2023-11-28T18:29:47.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.237+0000 7ff02bfff640 1 -- 172.21.15.105:0/4101591062 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff038013410 con 0x7ff0440caf60 2023-11-28T18:29:47.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.237+0000 7ff02bfff640 1 --2- 172.21.15.105:0/4101591062 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff030066900 0x7ff030068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:47.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.237+0000 7ff02bfff640 1 -- 172.21.15.105:0/4101591062 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7ff038084220 con 0x7ff0440caf60 2023-11-28T18:29:47.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.237+0000 7ff0437fe640 1 --2- 172.21.15.105:0/4101591062 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff030066900 0x7ff030068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:47.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.237+0000 7ff0437fe640 1 --2- 172.21.15.105:0/4101591062 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff030066900 0x7ff030068dc0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff0340011f0 tx=0x7ff034007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:47.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.241+0000 7ff02bfff640 1 -- 172.21.15.105:0/4101591062 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff0380513e0 con 0x7ff0440caf60 2023-11-28T18:29:47.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.393+0000 7ff049c5a640 1 -- 172.21.15.105:0/4101591062 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff0440cf950 con 0x7ff0440caf60 2023-11-28T18:29:47.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.393+0000 7ff02bfff640 1 -- 172.21.15.105:0/4101591062 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff038054a00 con 0x7ff0440caf60 2023-11-28T18:29:47.398 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:29:47.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.397+0000 7ff049c5a640 1 -- 172.21.15.105:0/4101591062 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff030066900 msgr2=0x7ff030068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:47.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.397+0000 7ff049c5a640 1 --2- 172.21.15.105:0/4101591062 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff030066900 0x7ff030068dc0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff0340011f0 tx=0x7ff034007d40 comp rx=0 tx=0).stop 2023-11-28T18:29:47.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.397+0000 7ff049c5a640 1 -- 172.21.15.105:0/4101591062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0440caf60 msgr2=0x7ff044099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:47.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.397+0000 7ff049c5a640 1 --2- 172.21.15.105:0/4101591062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0440caf60 0x7ff044099a50 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff03800e840 tx=0x7ff03800ed00 comp rx=0 tx=0).stop 2023-11-28T18:29:47.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.397+0000 7ff049c5a640 1 -- 172.21.15.105:0/4101591062 shutdown_connections 2023-11-28T18:29:47.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.397+0000 7ff049c5a640 1 --2- 172.21.15.105:0/4101591062 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff044159f40 0x7ff044099f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.397+0000 7ff049c5a640 1 --2- 172.21.15.105:0/4101591062 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff030066900 0x7ff030068dc0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.397+0000 7ff049c5a640 1 --2- 172.21.15.105:0/4101591062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0440caf60 0x7ff044099a50 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.397+0000 7ff049c5a640 1 --2- 172.21.15.105:0/4101591062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0440ad850 0x7ff04414ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.397+0000 7ff049c5a640 1 -- 172.21.15.105:0/4101591062 >> 172.21.15.105:0/4101591062 conn(0x7ff0440ceb90 msgr2=0x7ff0440ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:47.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.397+0000 7ff049c5a640 1 -- 172.21.15.105:0/4101591062 shutdown_connections 2023-11-28T18:29:47.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.397+0000 7ff049c5a640 1 -- 172.21.15.105:0/4101591062 wait complete. 2023-11-28T18:29:47.419 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:29:47.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.553+0000 7f78eb180640 1 Processor -- start 2023-11-28T18:29:47.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.553+0000 7f78eb180640 1 -- start start 2023-11-28T18:29:47.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.557+0000 7f78eb180640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f78e40b4290 0x7f78e40b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:47.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.557+0000 7f78eb180640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78e4071960 0x7f78e415f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:47.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.557+0000 7f78eb180640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e40b4bb0 con 0x7f78e415b030 2023-11-28T18:29:47.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.557+0000 7f78eb180640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e40701d0 con 0x7f78e40b4290 2023-11-28T18:29:47.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.557+0000 7f78eb180640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e406df80 con 0x7f78e4071960 2023-11-28T18:29:47.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.557+0000 7f78dbfff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78e4071960 0x7f78e415f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:47.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.557+0000 7f78e8f1c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f78e40b4290 0x7f78e40b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:47.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.557+0000 7f78e971d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f78e415b030 0x7f78e40709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:54714/0 (socket says 172.21.15.105:54714) 2023-11-28T18:29:47.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.557+0000 7f78dbfff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78e4071960 0x7f78e415f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:52418/0 (socket says 172.21.15.105:52418) 2023-11-28T18:29:47.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.557+0000 7f78e971d640 1 -- 172.21.15.105:0/2559440421 learned_addr learned my addr 172.21.15.105:0/2559440421 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:47.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.557+0000 7f78dbfff640 1 -- 172.21.15.105:0/2559440421 >> v2:172.21.15.148:3300/0 conn(0x7f78e40b4290 msgr2=0x7f78e40b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:47.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.557+0000 7f78dbfff640 1 --2- 172.21.15.105:0/2559440421 >> v2:172.21.15.148:3300/0 conn(0x7f78e40b4290 0x7f78e40b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.557+0000 7f78dbfff640 1 -- 172.21.15.105:0/2559440421 >> v1:172.21.15.148:6789/0 conn(0x7f78e415b030 legacy=0x7f78e40709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:47.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.557+0000 7f78dbfff640 1 -- 172.21.15.105:0/2559440421 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e4071df0 con 0x7f78e4071960 2023-11-28T18:29:47.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.557+0000 7f78e8f1c640 1 --2- 172.21.15.105:0/2559440421 >> v2:172.21.15.148:3300/0 conn(0x7f78e40b4290 0x7f78e40b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:29:47.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.561+0000 7f78dbfff640 1 --2- 172.21.15.105:0/2559440421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78e4071960 0x7f78e415f3b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f78d40097c0 tx=0x7f78d401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=324cc87254c53f7e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:47.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.561+0000 7f78db7fe640 1 -- 172.21.15.105:0/2559440421 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d40ab650 con 0x7f78e4071960 2023-11-28T18:29:47.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.561+0000 7f78db7fe640 1 -- 172.21.15.105:0/2559440421 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d40ab7b0 con 0x7f78e4071960 2023-11-28T18:29:47.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.561+0000 7f78db7fe640 1 -- 172.21.15.105:0/2559440421 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d40abb40 con 0x7f78e4071960 2023-11-28T18:29:47.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.561+0000 7f78eb180640 1 -- 172.21.15.105:0/2559440421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78e4071960 msgr2=0x7f78e415f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:47.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.561+0000 7f78eb180640 1 --2- 172.21.15.105:0/2559440421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78e4071960 0x7f78e415f3b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f78d40097c0 tx=0x7f78d401c450 comp rx=0 tx=0).stop 2023-11-28T18:29:47.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.561+0000 7f78eb180640 1 -- 172.21.15.105:0/2559440421 shutdown_connections 2023-11-28T18:29:47.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.561+0000 7f78eb180640 1 --2- 172.21.15.105:0/2559440421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78e4071960 0x7f78e415f3b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.561+0000 7f78eb180640 1 --2- 172.21.15.105:0/2559440421 >> v2:172.21.15.148:3300/0 conn(0x7f78e40b4290 0x7f78e40b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.561+0000 7f78eb180640 1 -- 172.21.15.105:0/2559440421 >> 172.21.15.105:0/2559440421 conn(0x7f78e40ceb90 msgr2=0x7f78e40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:47.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.561+0000 7f78eb180640 1 -- 172.21.15.105:0/2559440421 shutdown_connections 2023-11-28T18:29:47.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.561+0000 7f78eb180640 1 -- 172.21.15.105:0/2559440421 wait complete. 2023-11-28T18:29:47.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.561+0000 7f78eb180640 1 Processor -- start 2023-11-28T18:29:47.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.561+0000 7f78eb180640 1 -- start start 2023-11-28T18:29:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78eb180640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e40b4290 0x7f78e415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78eb180640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78e415b030 0x7f78e4158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78eb180640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78e41588a0 0x7f78e4144bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78eb180640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e4143b10 con 0x7f78e40b4290 2023-11-28T18:29:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78eb180640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e4143df0 con 0x7f78e41588a0 2023-11-28T18:29:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78eb180640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e41440d0 con 0x7f78e415b030 2023-11-28T18:29:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78e8f1c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e40b4290 0x7f78e415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78e8f1c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e40b4290 0x7f78e415ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38346/0 (socket says 172.21.15.105:38346) 2023-11-28T18:29:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78e8f1c640 1 -- 172.21.15.105:0/3086977344 learned_addr learned my addr 172.21.15.105:0/3086977344 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78e971d640 1 --2- 172.21.15.105:0/3086977344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78e41588a0 0x7f78e4144bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78dbfff640 1 --2- 172.21.15.105:0/3086977344 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78e415b030 0x7f78e4158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78e8f1c640 1 -- 172.21.15.105:0/3086977344 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78e415b030 msgr2=0x7f78e4158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78e8f1c640 1 --2- 172.21.15.105:0/3086977344 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78e415b030 0x7f78e4158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78e8f1c640 1 -- 172.21.15.105:0/3086977344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78e41588a0 msgr2=0x7f78e4144bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:47.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78e8f1c640 1 --2- 172.21.15.105:0/3086977344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78e41588a0 0x7f78e4144bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78e8f1c640 1 -- 172.21.15.105:0/3086977344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e0009420 con 0x7f78e40b4290 2023-11-28T18:29:47.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78dbfff640 1 --2- 172.21.15.105:0/3086977344 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78e415b030 0x7f78e4158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:47.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78e8f1c640 1 --2- 172.21.15.105:0/3086977344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e40b4290 0x7f78e415ced0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f78e000ebe0 tx=0x7f78e000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:47.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78d97fa640 1 -- 172.21.15.105:0/3086977344 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e000eee0 con 0x7f78e40b4290 2023-11-28T18:29:47.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78d97fa640 1 -- 172.21.15.105:0/3086977344 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e000cc30 con 0x7f78e40b4290 2023-11-28T18:29:47.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78d97fa640 1 -- 172.21.15.105:0/3086977344 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e0013470 con 0x7f78e40b4290 2023-11-28T18:29:47.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78eb180640 1 -- 172.21.15.105:0/3086977344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d4009420 con 0x7f78e40b4290 2023-11-28T18:29:47.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78eb180640 1 -- 172.21.15.105:0/3086977344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e4144690 con 0x7f78e40b4290 2023-11-28T18:29:47.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.565+0000 7f78eb180640 1 -- 172.21.15.105:0/3086977344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e40cb410 con 0x7f78e40b4290 2023-11-28T18:29:47.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.569+0000 7f78d97fa640 1 -- 172.21.15.105:0/3086977344 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f78e00135d0 con 0x7f78e40b4290 2023-11-28T18:29:47.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.569+0000 7f78d97fa640 1 --2- 172.21.15.105:0/3086977344 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78d00668d0 0x7f78d0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:47.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.569+0000 7f78d97fa640 1 -- 172.21.15.105:0/3086977344 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f78e00849a0 con 0x7f78e40b4290 2023-11-28T18:29:47.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.569+0000 7f78dbfff640 1 --2- 172.21.15.105:0/3086977344 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78d00668d0 0x7f78d0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:47.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.569+0000 7f78dbfff640 1 --2- 172.21.15.105:0/3086977344 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78d00668d0 0x7f78d0068d90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f78d40af000 tx=0x7f78d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:47.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.573+0000 7f78d97fa640 1 -- 172.21.15.105:0/3086977344 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f78e0051980 con 0x7f78e40b4290 2023-11-28T18:29:47.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.725+0000 7f78eb180640 1 -- 172.21.15.105:0/3086977344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78e4067570 con 0x7f78e40b4290 2023-11-28T18:29:47.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.725+0000 7f78d97fa640 1 -- 172.21.15.105:0/3086977344 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f78e0017020 con 0x7f78e40b4290 2023-11-28T18:29:47.729 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:29:47.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.725+0000 7f78eb180640 1 -- 172.21.15.105:0/3086977344 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78d00668d0 msgr2=0x7f78d0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:47.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.725+0000 7f78eb180640 1 --2- 172.21.15.105:0/3086977344 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78d00668d0 0x7f78d0068d90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f78d40af000 tx=0x7f78d4008040 comp rx=0 tx=0).stop 2023-11-28T18:29:47.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.725+0000 7f78eb180640 1 -- 172.21.15.105:0/3086977344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e40b4290 msgr2=0x7f78e415ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:47.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.725+0000 7f78eb180640 1 --2- 172.21.15.105:0/3086977344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e40b4290 0x7f78e415ced0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f78e000ebe0 tx=0x7f78e000c520 comp rx=0 tx=0).stop 2023-11-28T18:29:47.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.725+0000 7f78eb180640 1 -- 172.21.15.105:0/3086977344 shutdown_connections 2023-11-28T18:29:47.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.725+0000 7f78eb180640 1 --2- 172.21.15.105:0/3086977344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78e41588a0 0x7f78e4144bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.725+0000 7f78eb180640 1 --2- 172.21.15.105:0/3086977344 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78e415b030 0x7f78e4158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.725+0000 7f78eb180640 1 --2- 172.21.15.105:0/3086977344 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78d00668d0 0x7f78d0068d90 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.725+0000 7f78eb180640 1 --2- 172.21.15.105:0/3086977344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e40b4290 0x7f78e415ced0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.725+0000 7f78eb180640 1 -- 172.21.15.105:0/3086977344 >> 172.21.15.105:0/3086977344 conn(0x7f78e40ceb90 msgr2=0x7f78e40cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:47.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.725+0000 7f78eb180640 1 -- 172.21.15.105:0/3086977344 shutdown_connections 2023-11-28T18:29:47.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.725+0000 7f78eb180640 1 -- 172.21.15.105:0/3086977344 wait complete. 2023-11-28T18:29:47.749 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:29:47.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.885+0000 7fbc222d6640 1 Processor -- start 2023-11-28T18:29:47.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.885+0000 7fbc222d6640 1 -- start start 2023-11-28T18:29:47.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.885+0000 7fbc222d6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbc1c0caf60 0x7fbc1c0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:47.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.885+0000 7fbc222d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc1c0ad850 0x7fbc1c0701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:47.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.885+0000 7fbc222d6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc1c06bf50 con 0x7fbc1c159f40 2023-11-28T18:29:47.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.885+0000 7fbc222d6640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc1c0cb3c0 con 0x7fbc1c0caf60 2023-11-28T18:29:47.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.885+0000 7fbc222d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c15a320 con 0x7fbc1c0ad850 2023-11-28T18:29:47.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.885+0000 7fbc1b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbc1c0caf60 0x7fbc1c0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:47.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.885+0000 7fbc1bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbc1c159f40 0x7fbc1c0ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:54720/0 (socket says 172.21.15.105:54720) 2023-11-28T18:29:47.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.885+0000 7fbc1bfff640 1 -- 172.21.15.105:0/3728857941 learned_addr learned my addr 172.21.15.105:0/3728857941 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:47.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.885+0000 7fbc20873640 1 --2- 172.21.15.105:0/3728857941 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc1c0ad850 0x7fbc1c0701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:47.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.885+0000 7fbc1b7fe640 1 -- 172.21.15.105:0/3728857941 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc1c0ad850 msgr2=0x7fbc1c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:47.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.885+0000 7fbc1b7fe640 1 --2- 172.21.15.105:0/3728857941 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc1c0ad850 0x7fbc1c0701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.885+0000 7fbc1b7fe640 1 -- 172.21.15.105:0/3728857941 >> v1:172.21.15.148:6789/0 conn(0x7fbc1c159f40 legacy=0x7fbc1c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:47.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.885+0000 7fbc1b7fe640 1 -- 172.21.15.105:0/3728857941 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1c071380 con 0x7fbc1c0caf60 2023-11-28T18:29:47.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.889+0000 7fbc1b7fe640 1 --2- 172.21.15.105:0/3728857941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc1c0caf60 0x7fbc1c0cc760 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fbc100097c0 tx=0x7fbc1001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850d029c1e1b2ad server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:47.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.889+0000 7fbc1affd640 1 -- 172.21.15.105:0/3728857941 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc100ac620 con 0x7fbc1c0caf60 2023-11-28T18:29:47.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.889+0000 7fbc1affd640 1 -- 172.21.15.105:0/3728857941 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc100ac780 con 0x7fbc1c0caf60 2023-11-28T18:29:47.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.889+0000 7fbc1affd640 1 -- 172.21.15.105:0/3728857941 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc100acb10 con 0x7fbc1c0caf60 2023-11-28T18:29:47.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.889+0000 7fbc222d6640 1 -- 172.21.15.105:0/3728857941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc1c0caf60 msgr2=0x7fbc1c0cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:47.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.889+0000 7fbc222d6640 1 --2- 172.21.15.105:0/3728857941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc1c0caf60 0x7fbc1c0cc760 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fbc100097c0 tx=0x7fbc1001c3c0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.889+0000 7fbc222d6640 1 -- 172.21.15.105:0/3728857941 shutdown_connections 2023-11-28T18:29:47.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.889+0000 7fbc222d6640 1 --2- 172.21.15.105:0/3728857941 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc1c0ad850 0x7fbc1c0701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.889+0000 7fbc222d6640 1 --2- 172.21.15.105:0/3728857941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc1c0caf60 0x7fbc1c0cc760 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.889+0000 7fbc222d6640 1 -- 172.21.15.105:0/3728857941 >> 172.21.15.105:0/3728857941 conn(0x7fbc1c0ceb90 msgr2=0x7fbc1c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:47.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.889+0000 7fbc222d6640 1 -- 172.21.15.105:0/3728857941 shutdown_connections 2023-11-28T18:29:47.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.889+0000 7fbc222d6640 1 -- 172.21.15.105:0/3728857941 wait complete. 2023-11-28T18:29:47.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc222d6640 1 Processor -- start 2023-11-28T18:29:47.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc222d6640 1 -- start start 2023-11-28T18:29:47.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc222d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc1c0ad850 0x7fbc1c13e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:47.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc222d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc1c0caf60 0x7fbc1c0aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:47.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc222d6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc1c159f40 0x7fbc1c0af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:47.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc222d6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c1629b0 con 0x7fbc1c159f40 2023-11-28T18:29:47.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc222d6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c158230 con 0x7fbc1c0ad850 2023-11-28T18:29:47.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc222d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c158510 con 0x7fbc1c0caf60 2023-11-28T18:29:47.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc20873640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc1c159f40 0x7fbc1c0af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:47.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc20873640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc1c159f40 0x7fbc1c0af350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38368/0 (socket says 172.21.15.105:38368) 2023-11-28T18:29:47.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc20873640 1 -- 172.21.15.105:0/3472813189 learned_addr learned my addr 172.21.15.105:0/3472813189 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:47.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc1bfff640 1 --2- 172.21.15.105:0/3472813189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc1c0ad850 0x7fbc1c13e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:47.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc1b7fe640 1 --2- 172.21.15.105:0/3472813189 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc1c0caf60 0x7fbc1c0aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:47.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc20873640 1 -- 172.21.15.105:0/3472813189 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc1c0caf60 msgr2=0x7fbc1c0aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:47.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc20873640 1 --2- 172.21.15.105:0/3472813189 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc1c0caf60 0x7fbc1c0aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc20873640 1 -- 172.21.15.105:0/3472813189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc1c0ad850 msgr2=0x7fbc1c13e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:47.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc20873640 1 --2- 172.21.15.105:0/3472813189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc1c0ad850 0x7fbc1c13e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:47.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc20873640 1 -- 172.21.15.105:0/3472813189 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc10009420 con 0x7fbc1c159f40 2023-11-28T18:29:47.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc20873640 1 --2- 172.21.15.105:0/3472813189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc1c159f40 0x7fbc1c0af350 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400efc0 tx=0x7fbc1400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:47.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc18ff9640 1 -- 172.21.15.105:0/3472813189 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc14009ba0 con 0x7fbc1c159f40 2023-11-28T18:29:47.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc18ff9640 1 -- 172.21.15.105:0/3472813189 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1400f040 con 0x7fbc1c159f40 2023-11-28T18:29:47.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc18ff9640 1 -- 172.21.15.105:0/3472813189 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc140146c0 con 0x7fbc1c159f40 2023-11-28T18:29:47.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc222d6640 1 -- 172.21.15.105:0/3472813189 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1c1587f0 con 0x7fbc1c159f40 2023-11-28T18:29:47.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.893+0000 7fbc222d6640 1 -- 172.21.15.105:0/3472813189 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1c13e9a0 con 0x7fbc1c159f40 2023-11-28T18:29:47.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.897+0000 7fbc222d6640 1 -- 172.21.15.105:0/3472813189 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc1c067570 con 0x7fbc1c159f40 2023-11-28T18:29:47.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.897+0000 7fbc18ff9640 1 -- 172.21.15.105:0/3472813189 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fbc14014820 con 0x7fbc1c159f40 2023-11-28T18:29:47.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.897+0000 7fbc18ff9640 1 --2- 172.21.15.105:0/3472813189 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbc080668d0 0x7fbc08068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:47.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.897+0000 7fbc18ff9640 1 -- 172.21.15.105:0/3472813189 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fbc14084f30 con 0x7fbc1c159f40 2023-11-28T18:29:47.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.897+0000 7fbc1bfff640 1 --2- 172.21.15.105:0/3472813189 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbc080668d0 0x7fbc08068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:47.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.897+0000 7fbc1bfff640 1 --2- 172.21.15.105:0/3472813189 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbc080668d0 0x7fbc08068d90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c005890 tx=0x7fbc0c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:47.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:47.901+0000 7fbc18ff9640 1 -- 172.21.15.105:0/3472813189 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbc140520f0 con 0x7fbc1c159f40 2023-11-28T18:29:48.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.049+0000 7fbc222d6640 1 -- 172.21.15.105:0/3472813189 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbc1c0cf950 con 0x7fbc1c159f40 2023-11-28T18:29:48.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.049+0000 7fbc18ff9640 1 -- 172.21.15.105:0/3472813189 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbc140180a0 con 0x7fbc1c159f40 2023-11-28T18:29:48.055 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:29:48.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.053+0000 7fbc222d6640 1 -- 172.21.15.105:0/3472813189 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbc080668d0 msgr2=0x7fbc08068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:48.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.053+0000 7fbc222d6640 1 --2- 172.21.15.105:0/3472813189 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbc080668d0 0x7fbc08068d90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c005890 tx=0x7fbc0c009040 comp rx=0 tx=0).stop 2023-11-28T18:29:48.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.053+0000 7fbc222d6640 1 -- 172.21.15.105:0/3472813189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc1c159f40 msgr2=0x7fbc1c0af350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:48.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.053+0000 7fbc222d6640 1 --2- 172.21.15.105:0/3472813189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc1c159f40 0x7fbc1c0af350 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400efc0 tx=0x7fbc1400c490 comp rx=0 tx=0).stop 2023-11-28T18:29:48.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.053+0000 7fbc222d6640 1 -- 172.21.15.105:0/3472813189 shutdown_connections 2023-11-28T18:29:48.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.053+0000 7fbc222d6640 1 --2- 172.21.15.105:0/3472813189 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbc080668d0 0x7fbc08068d90 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:48.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.053+0000 7fbc222d6640 1 --2- 172.21.15.105:0/3472813189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc1c159f40 0x7fbc1c0af350 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:48.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.053+0000 7fbc222d6640 1 --2- 172.21.15.105:0/3472813189 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc1c0caf60 0x7fbc1c0aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:48.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.053+0000 7fbc222d6640 1 --2- 172.21.15.105:0/3472813189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc1c0ad850 0x7fbc1c13e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:48.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.053+0000 7fbc222d6640 1 -- 172.21.15.105:0/3472813189 >> 172.21.15.105:0/3472813189 conn(0x7fbc1c0ceb90 msgr2=0x7fbc1c152b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:48.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.053+0000 7fbc222d6640 1 -- 172.21.15.105:0/3472813189 shutdown_connections 2023-11-28T18:29:48.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.053+0000 7fbc222d6640 1 -- 172.21.15.105:0/3472813189 wait complete. 2023-11-28T18:29:48.076 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T18:29:48.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.217+0000 7f364d356640 1 Processor -- start 2023-11-28T18:29:48.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.217+0000 7f364d356640 1 -- start start 2023-11-28T18:29:48.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.217+0000 7f364d356640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f364815d2d0 0x7f36480cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:48.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.217+0000 7f364d356640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36480cf960 0x7f3648166fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:48.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.217+0000 7f364d356640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36480ce840 con 0x7f364815cef0 2023-11-28T18:29:48.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.217+0000 7f364d356640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36480cfdf0 con 0x7f364815d2d0 2023-11-28T18:29:48.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.217+0000 7f364d356640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36480b4780 con 0x7f36480cf960 2023-11-28T18:29:48.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.217+0000 7f363ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f364815d2d0 0x7f36480cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:48.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.217+0000 7f363ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f364815d2d0 0x7f36480cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38376/0 (socket says 172.21.15.105:38376) 2023-11-28T18:29:48.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.217+0000 7f363ffff640 1 -- 172.21.15.105:0/685696962 learned_addr learned my addr 172.21.15.105:0/685696962 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:48.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.217+0000 7f364cb55640 1 --2- 172.21.15.105:0/685696962 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36480cf960 0x7f3648166fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:48.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.217+0000 7f363ffff640 1 -- 172.21.15.105:0/685696962 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36480cf960 msgr2=0x7f3648166fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:48.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.217+0000 7f363ffff640 1 --2- 172.21.15.105:0/685696962 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36480cf960 0x7f3648166fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:48.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.217+0000 7f363ffff640 1 -- 172.21.15.105:0/685696962 >> v1:172.21.15.148:6789/0 conn(0x7f364815cef0 legacy=0x7f36480d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:48.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.217+0000 7f363ffff640 1 -- 172.21.15.105:0/685696962 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36480cd990 con 0x7f364815d2d0 2023-11-28T18:29:48.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.221+0000 7f363ffff640 1 --2- 172.21.15.105:0/685696962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f364815d2d0 0x7f36480cf420 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f36300098a0 tx=0x7f363001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c747091af4cca0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:48.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.221+0000 7f36477fe640 1 -- 172.21.15.105:0/685696962 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36300ab670 con 0x7f364815d2d0 2023-11-28T18:29:48.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.221+0000 7f36477fe640 1 -- 172.21.15.105:0/685696962 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36300ab7d0 con 0x7f364815d2d0 2023-11-28T18:29:48.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.221+0000 7f36477fe640 1 -- 172.21.15.105:0/685696962 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36300abb60 con 0x7f364815d2d0 2023-11-28T18:29:48.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.221+0000 7f364d356640 1 -- 172.21.15.105:0/685696962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f364815d2d0 msgr2=0x7f36480cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:48.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.221+0000 7f364d356640 1 --2- 172.21.15.105:0/685696962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f364815d2d0 0x7f36480cf420 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f36300098a0 tx=0x7f363001c470 comp rx=0 tx=0).stop 2023-11-28T18:29:48.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.221+0000 7f364d356640 1 -- 172.21.15.105:0/685696962 shutdown_connections 2023-11-28T18:29:48.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.221+0000 7f364d356640 1 --2- 172.21.15.105:0/685696962 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36480cf960 0x7f3648166fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:48.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.221+0000 7f364d356640 1 --2- 172.21.15.105:0/685696962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f364815d2d0 0x7f36480cf420 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:48.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.221+0000 7f364d356640 1 -- 172.21.15.105:0/685696962 >> 172.21.15.105:0/685696962 conn(0x7f36480d2e10 msgr2=0x7f36480d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.221+0000 7f364d356640 1 -- 172.21.15.105:0/685696962 shutdown_connections 2023-11-28T18:29:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.221+0000 7f364d356640 1 -- 172.21.15.105:0/685696962 wait complete. 2023-11-28T18:29:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f364d356640 1 Processor -- start 2023-11-28T18:29:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f364d356640 1 -- start start 2023-11-28T18:29:48.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f364d356640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36480cf960 0x7f364815bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:48.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f364d356640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f364815cef0 0x7f364815c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:48.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f364d356640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f364815d2d0 0x7f36480b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:48.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f364d356640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36481f5bb0 con 0x7f364815cef0 2023-11-28T18:29:48.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f364d356640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36480c6ae0 con 0x7f364815d2d0 2023-11-28T18:29:48.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f364d356640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36480c6d80 con 0x7f36480cf960 2023-11-28T18:29:48.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f363ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f364815cef0 0x7f364815c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:48.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f363ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f364815cef0 0x7f364815c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38378/0 (socket says 172.21.15.105:38378) 2023-11-28T18:29:48.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f363ffff640 1 -- 172.21.15.105:0/787124759 learned_addr learned my addr 172.21.15.105:0/787124759 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:48.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f364cb55640 1 --2- 172.21.15.105:0/787124759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f364815d2d0 0x7f36480b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:48.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f3647fff640 1 --2- 172.21.15.105:0/787124759 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36480cf960 0x7f364815bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:48.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f363ffff640 1 -- 172.21.15.105:0/787124759 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36480cf960 msgr2=0x7f364815bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:48.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f363ffff640 1 --2- 172.21.15.105:0/787124759 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36480cf960 0x7f364815bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:48.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f363ffff640 1 -- 172.21.15.105:0/787124759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f364815d2d0 msgr2=0x7f36480b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:48.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f363ffff640 1 --2- 172.21.15.105:0/787124759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f364815d2d0 0x7f36480b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:48.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f363ffff640 1 -- 172.21.15.105:0/787124759 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36300094d0 con 0x7f364815cef0 2023-11-28T18:29:48.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f363ffff640 1 --2- 172.21.15.105:0/787124759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f364815cef0 0x7f364815c460 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f36300ad040 tx=0x7f3630019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:48.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f36457fa640 1 -- 172.21.15.105:0/787124759 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36300b5810 con 0x7f364815cef0 2023-11-28T18:29:48.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f36457fa640 1 -- 172.21.15.105:0/787124759 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36300b5990 con 0x7f364815cef0 2023-11-28T18:29:48.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f364d356640 1 -- 172.21.15.105:0/787124759 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36480c7060 con 0x7f364815cef0 2023-11-28T18:29:48.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f36457fa640 1 -- 172.21.15.105:0/787124759 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36300b5d40 con 0x7f364815cef0 2023-11-28T18:29:48.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.225+0000 7f364d356640 1 -- 172.21.15.105:0/787124759 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36480c7620 con 0x7f364815cef0 2023-11-28T18:29:48.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.229+0000 7f364d356640 1 -- 172.21.15.105:0/787124759 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36480c7810 con 0x7f364815cef0 2023-11-28T18:29:48.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.229+0000 7f36457fa640 1 -- 172.21.15.105:0/787124759 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f36300c83f0 con 0x7f364815cef0 2023-11-28T18:29:48.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.229+0000 7f36457fa640 1 --2- 172.21.15.105:0/787124759 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3628066900 0x7f3628068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:48.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.229+0000 7f36457fa640 1 -- 172.21.15.105:0/787124759 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f363011f290 con 0x7f364815cef0 2023-11-28T18:29:48.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.229+0000 7f3647fff640 1 --2- 172.21.15.105:0/787124759 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3628066900 0x7f3628068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:48.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.229+0000 7f3647fff640 1 --2- 172.21.15.105:0/787124759 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3628066900 0x7f3628068dc0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3638007fd0 tx=0x7f3638009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:48.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.233+0000 7f36457fa640 1 -- 172.21.15.105:0/787124759 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f36300ec3b0 con 0x7f364815cef0 2023-11-28T18:29:48.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.393+0000 7f364d356640 1 -- 172.21.15.105:0/787124759 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f36480c6ec0 con 0x7f364815cef0 2023-11-28T18:29:48.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.393+0000 7f36457fa640 1 -- 172.21.15.105:0/787124759 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v92) v1 ==== 80+0+0 (secure 0 0 0) 0x7f36480c6ec0 con 0x7f364815cef0 2023-11-28T18:29:48.399 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2023-11-28T18:29:48.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.397+0000 7f364d356640 1 -- 172.21.15.105:0/787124759 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3628066900 msgr2=0x7f3628068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:48.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.397+0000 7f364d356640 1 --2- 172.21.15.105:0/787124759 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3628066900 0x7f3628068dc0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3638007fd0 tx=0x7f3638009040 comp rx=0 tx=0).stop 2023-11-28T18:29:48.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.397+0000 7f364d356640 1 -- 172.21.15.105:0/787124759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f364815cef0 msgr2=0x7f364815c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:48.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.397+0000 7f364d356640 1 --2- 172.21.15.105:0/787124759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f364815cef0 0x7f364815c460 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f36300ad040 tx=0x7f3630019210 comp rx=0 tx=0).stop 2023-11-28T18:29:48.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.397+0000 7f364d356640 1 -- 172.21.15.105:0/787124759 shutdown_connections 2023-11-28T18:29:48.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.397+0000 7f364d356640 1 --2- 172.21.15.105:0/787124759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f364815d2d0 0x7f36480b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:48.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.397+0000 7f364d356640 1 --2- 172.21.15.105:0/787124759 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3628066900 0x7f3628068dc0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:48.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.397+0000 7f364d356640 1 --2- 172.21.15.105:0/787124759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f364815cef0 0x7f364815c460 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:48.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.397+0000 7f364d356640 1 --2- 172.21.15.105:0/787124759 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36480cf960 0x7f364815bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:48.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.397+0000 7f364d356640 1 -- 172.21.15.105:0/787124759 >> 172.21.15.105:0/787124759 conn(0x7f36480d2e10 msgr2=0x7f36480d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:48.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.397+0000 7f364d356640 1 -- 172.21.15.105:0/787124759 shutdown_connections 2023-11-28T18:29:48.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.397+0000 7f364d356640 1 -- 172.21.15.105:0/787124759 wait complete. 2023-11-28T18:29:48.420 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2023-11-28T18:29:48.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:48.420 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-28T18:29:48.439 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[60998]: fuse finished with error 0 and tester_r 0 2023-11-28T18:29:48.443 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:29:48.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:48.443 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T18:29:48.554 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:29:48.554 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:29:48.563 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:29:48.564 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:29:48.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:48.564 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T18:29:48.616 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-28T18:29:48.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:29:48.618 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:29:48.618 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:29:48.628 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:29:48.628 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:29:48.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:48.628 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T18:29:48.679 INFO:teuthology.orchestra.run.smithi105.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-28T18:29:48.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:29:48.680 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:29:48.681 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:29:48.691 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:29:48.691 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:29:48.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:29:48.692 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T18:29:48.744 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-28T18:29:48.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:29:48.745 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:29:48.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.881+0000 7f44e192a640 1 Processor -- start 2023-11-28T18:29:48.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.881+0000 7f44e192a640 1 -- start start 2023-11-28T18:29:48.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.881+0000 7f44e192a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f44dc0cdd80 0x7f44dc06c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:48.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.881+0000 7f44e192a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f44dc0ce230 0x7f44dc0cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:48.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.881+0000 7f44e192a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44dc0b3c40 con 0x7f44dc15d320 2023-11-28T18:29:48.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.881+0000 7f44e192a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44dc06ca80 con 0x7f44dc0ce230 2023-11-28T18:29:48.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.881+0000 7f44e192a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc15d700 con 0x7f44dc0cdd80 2023-11-28T18:29:48.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.881+0000 7f44daffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f44dc15d320 0x7f44dc0cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40442/0 (socket says 172.21.15.105:40442) 2023-11-28T18:29:48.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.881+0000 7f44daffd640 1 -- 172.21.15.105:0/1791710672 learned_addr learned my addr 172.21.15.105:0/1791710672 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:48.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.881+0000 7f44db7fe640 1 --2- 172.21.15.105:0/1791710672 >> v2:172.21.15.105:3300/0 conn(0x7f44dc0ce230 0x7f44dc0cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:48.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.881+0000 7f44d27fc640 1 --2- 172.21.15.105:0/1791710672 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f44dc0cdd80 0x7f44dc06c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:48.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.881+0000 7f44db7fe640 1 -- 172.21.15.105:0/1791710672 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f44dc0cdd80 msgr2=0x7f44dc06c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:48.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.881+0000 7f44da7fc640 1 -- 172.21.15.105:0/1791710672 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446156670 0 0) 0x7f44dc0b3c40 con 0x7f44dc15d320 2023-11-28T18:29:48.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.881+0000 7f44db7fe640 1 --2- 172.21.15.105:0/1791710672 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f44dc0cdd80 0x7f44dc06c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:48.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.881+0000 7f44db7fe640 1 -- 172.21.15.105:0/1791710672 >> v1:172.21.15.105:6789/0 conn(0x7f44dc15d320 legacy=0x7f44dc0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:48.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.881+0000 7f44db7fe640 1 -- 172.21.15.105:0/1791710672 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44dc06c7a0 con 0x7f44dc0ce230 2023-11-28T18:29:48.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.885+0000 7f44db7fe640 1 --2- 172.21.15.105:0/1791710672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44dc0ce230 0x7f44dc0cc0c0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009d90 tx=0x7f44d401c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984b2fad7f23f869 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:48.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.885+0000 7f44da7fc640 1 -- 172.21.15.105:0/1791710672 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d40a5980 con 0x7f44dc0ce230 2023-11-28T18:29:48.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.885+0000 7f44da7fc640 1 -- 172.21.15.105:0/1791710672 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d40a5ae0 con 0x7f44dc0ce230 2023-11-28T18:29:48.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.885+0000 7f44da7fc640 1 -- 172.21.15.105:0/1791710672 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d40a5e70 con 0x7f44dc0ce230 2023-11-28T18:29:48.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.885+0000 7f44e192a640 1 -- 172.21.15.105:0/1791710672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44dc0ce230 msgr2=0x7f44dc0cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:48.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.885+0000 7f44e192a640 1 --2- 172.21.15.105:0/1791710672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44dc0ce230 0x7f44dc0cc0c0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009d90 tx=0x7f44d401c8c0 comp rx=0 tx=0).stop 2023-11-28T18:29:48.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.885+0000 7f44e192a640 1 -- 172.21.15.105:0/1791710672 shutdown_connections 2023-11-28T18:29:48.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.885+0000 7f44e192a640 1 --2- 172.21.15.105:0/1791710672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44dc0ce230 0x7f44dc0cc0c0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:48.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.885+0000 7f44e192a640 1 --2- 172.21.15.105:0/1791710672 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f44dc0cdd80 0x7f44dc06c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:48.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.885+0000 7f44e192a640 1 -- 172.21.15.105:0/1791710672 >> 172.21.15.105:0/1791710672 conn(0x7f44dc0d0df0 msgr2=0x7f44dc0d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:48.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.885+0000 7f44e192a640 1 -- 172.21.15.105:0/1791710672 shutdown_connections 2023-11-28T18:29:48.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.885+0000 7f44e192a640 1 -- 172.21.15.105:0/1791710672 wait complete. 2023-11-28T18:29:48.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44e192a640 1 Processor -- start 2023-11-28T18:29:48.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44e192a640 1 -- start start 2023-11-28T18:29:48.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44e192a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44dc0cdd80 0x7f44dc0b6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:48.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44e192a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f44dc0ce230 0x7f44dc0c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:48.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44e192a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44dc15d320 0x7f44dc0ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:48.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44e192a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc0c9f50 con 0x7f44dc0cdd80 2023-11-28T18:29:48.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44e192a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc155120 con 0x7f44dc15d320 2023-11-28T18:29:48.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44e192a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc0c98e0 con 0x7f44dc0ce230 2023-11-28T18:29:48.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44daffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44dc0cdd80 0x7f44dc0b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:48.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44daffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44dc0cdd80 0x7f44dc0b6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38398/0 (socket says 172.21.15.105:38398) 2023-11-28T18:29:48.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44daffd640 1 -- 172.21.15.105:0/1928025393 learned_addr learned my addr 172.21.15.105:0/1928025393 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:48.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44d27fc640 1 --2- 172.21.15.105:0/1928025393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f44dc0ce230 0x7f44dc0c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:48.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44db7fe640 1 --2- 172.21.15.105:0/1928025393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44dc15d320 0x7f44dc0ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:48.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44daffd640 1 -- 172.21.15.105:0/1928025393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f44dc0ce230 msgr2=0x7f44dc0c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:48.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44daffd640 1 --2- 172.21.15.105:0/1928025393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f44dc0ce230 0x7f44dc0c93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:48.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44daffd640 1 -- 172.21.15.105:0/1928025393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44dc15d320 msgr2=0x7f44dc0ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:48.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44daffd640 1 --2- 172.21.15.105:0/1928025393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44dc15d320 0x7f44dc0ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:48.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44daffd640 1 -- 172.21.15.105:0/1928025393 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d4009a70 con 0x7f44dc0cdd80 2023-11-28T18:29:48.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44db7fe640 1 --2- 172.21.15.105:0/1928025393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44dc15d320 0x7f44dc0ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:48.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44d27fc640 1 --2- 172.21.15.105:0/1928025393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f44dc0ce230 0x7f44dc0c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:48.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44daffd640 1 --2- 172.21.15.105:0/1928025393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44dc0cdd80 0x7f44dc0b6890 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f44c400c480 tx=0x7f44c400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:48.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44d8ff9640 1 -- 172.21.15.105:0/1928025393 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c40074e0 con 0x7f44dc0cdd80 2023-11-28T18:29:48.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44d8ff9640 1 -- 172.21.15.105:0/1928025393 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c4007660 con 0x7f44dc0cdd80 2023-11-28T18:29:48.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44d8ff9640 1 -- 172.21.15.105:0/1928025393 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c4004a30 con 0x7f44dc0cdd80 2023-11-28T18:29:48.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44e192a640 1 -- 172.21.15.105:0/1928025393 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44dc0ca8b0 con 0x7f44dc0cdd80 2023-11-28T18:29:48.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.889+0000 7f44e192a640 1 -- 172.21.15.105:0/1928025393 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44dc0adf40 con 0x7f44dc0cdd80 2023-11-28T18:29:48.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.893+0000 7f44e192a640 1 -- 172.21.15.105:0/1928025393 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44dc0ca1c0 con 0x7f44dc0cdd80 2023-11-28T18:29:48.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.893+0000 7f44d8ff9640 1 -- 172.21.15.105:0/1928025393 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f44c4004b90 con 0x7f44dc0cdd80 2023-11-28T18:29:48.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.893+0000 7f44d8ff9640 1 --2- 172.21.15.105:0/1928025393 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f44bc0668d0 0x7f44bc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:48.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.893+0000 7f44d8ff9640 1 -- 172.21.15.105:0/1928025393 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f44c40837a0 con 0x7f44dc0cdd80 2023-11-28T18:29:48.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.893+0000 7f44d27fc640 1 --2- 172.21.15.105:0/1928025393 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f44bc0668d0 0x7f44bc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:48.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.893+0000 7f44d27fc640 1 --2- 172.21.15.105:0/1928025393 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f44bc0668d0 0x7f44bc068d90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0b3da0 tx=0x7f44cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:48.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:48.897+0000 7f44d8ff9640 1 -- 172.21.15.105:0/1928025393 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f44c40542a0 con 0x7f44dc0cdd80 2023-11-28T18:29:49.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.081+0000 7f44e192a640 1 -- 172.21.15.105:0/1928025393 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44dc067570 con 0x7f44dc0cdd80 2023-11-28T18:29:49.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.081+0000 7f44d8ff9640 1 -- 172.21.15.105:0/1928025393 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4959 (secure 0 0 0) 0x7f44c4053e30 con 0x7f44dc0cdd80 2023-11-28T18:29:49.088 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:29:49.089 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":94,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/1464337040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1464337040},{"type":"v1","addr":"172.21.15.148:6834","nonce":1464337040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:29:09.707308+0000","modified":"2023-11-28T18:29:43.461020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7689,"mds_1":7665,"mds_2":6485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6485":{"gid":6485,"name":"b","rank":2,"incarnation":91,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6835/3461832163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":3461832163},{"type":"v1","addr":"172.21.15.148:6835","nonce":3461832163}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7665":{"gid":7665,"name":"c","rank":1,"incarnation":89,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6835/4162499249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":4162499249},{"type":"v1","addr":"172.21.15.105:6835","nonce":4162499249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7689":{"gid":7689,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/3239728352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3239728352},{"type":"v1","addr":"172.21.15.105:6837","nonce":3239728352}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-28T18:29:49.090 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 94 2023-11-28T18:29:49.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.085+0000 7f44e192a640 1 -- 172.21.15.105:0/1928025393 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f44bc0668d0 msgr2=0x7f44bc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:49.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.085+0000 7f44e192a640 1 --2- 172.21.15.105:0/1928025393 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f44bc0668d0 0x7f44bc068d90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0b3da0 tx=0x7f44cc008040 comp rx=0 tx=0).stop 2023-11-28T18:29:49.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.085+0000 7f44e192a640 1 -- 172.21.15.105:0/1928025393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44dc0cdd80 msgr2=0x7f44dc0b6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:49.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.085+0000 7f44e192a640 1 --2- 172.21.15.105:0/1928025393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44dc0cdd80 0x7f44dc0b6890 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f44c400c480 tx=0x7f44c400c940 comp rx=0 tx=0).stop 2023-11-28T18:29:49.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.085+0000 7f44e192a640 1 -- 172.21.15.105:0/1928025393 shutdown_connections 2023-11-28T18:29:49.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.085+0000 7f44e192a640 1 --2- 172.21.15.105:0/1928025393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44dc15d320 0x7f44dc0ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.085+0000 7f44e192a640 1 --2- 172.21.15.105:0/1928025393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f44dc0ce230 0x7f44dc0c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.085+0000 7f44e192a640 1 --2- 172.21.15.105:0/1928025393 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f44bc0668d0 0x7f44bc068d90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.085+0000 7f44e192a640 1 --2- 172.21.15.105:0/1928025393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44dc0cdd80 0x7f44dc0b6890 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.085+0000 7f44e192a640 1 -- 172.21.15.105:0/1928025393 >> 172.21.15.105:0/1928025393 conn(0x7f44dc0d0df0 msgr2=0x7f44dc06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:49.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.085+0000 7f44e192a640 1 -- 172.21.15.105:0/1928025393 shutdown_connections 2023-11-28T18:29:49.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.085+0000 7f44e192a640 1 -- 172.21.15.105:0/1928025393 wait complete. 2023-11-28T18:29:49.109 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:29:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.241+0000 7fe3ce09a640 1 Processor -- start 2023-11-28T18:29:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.241+0000 7fe3ce09a640 1 -- start start 2023-11-28T18:29:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3ce09a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe3c80af340 0x7fe3c80721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3ce09a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c806f190 con 0x7fe3c806a580 2023-11-28T18:29:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3ce09a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c80b2ee0 con 0x7fe3c806ed00 2023-11-28T18:29:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3ce09a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c80b34a0 con 0x7fe3c80af340 2023-11-28T18:29:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c7fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe3c806a580 0x7fe3c806d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40446/0 (socket says 172.21.15.105:40446) 2023-11-28T18:29:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c7fff640 1 -- 172.21.15.105:0/1135248749 learned_addr learned my addr 172.21.15.105:0/1135248749 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c77fe640 1 --2- 172.21.15.105:0/1135248749 >> v2:172.21.15.148:3300/0 conn(0x7fe3c80af340 0x7fe3c80721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c6ffd640 1 -- 172.21.15.105:0/1135248749 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 652123879 0 0) 0x7fe3c806f190 con 0x7fe3c806a580 2023-11-28T18:29:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c6ffd640 1 -- 172.21.15.105:0/1135248749 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3b4003530 con 0x7fe3c806a580 2023-11-28T18:29:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c6ffd640 1 -- 172.21.15.105:0/1135248749 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3087253341 0 0) 0x7fe3c80b2ee0 con 0x7fe3c806ed00 2023-11-28T18:29:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c6ffd640 1 -- 172.21.15.105:0/1135248749 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3c806f190 con 0x7fe3c806ed00 2023-11-28T18:29:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c6ffd640 1 -- 172.21.15.105:0/1135248749 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3387547241 0 0) 0x7fe3b4003530 con 0x7fe3c806a580 2023-11-28T18:29:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c6ffd640 1 -- 172.21.15.105:0/1135248749 >> v2:172.21.15.148:3300/0 conn(0x7fe3c80af340 msgr2=0x7fe3c80721d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c6ffd640 1 --2- 172.21.15.105:0/1135248749 >> v2:172.21.15.148:3300/0 conn(0x7fe3c80af340 0x7fe3c80721d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c6ffd640 1 -- 172.21.15.105:0/1135248749 >> v1:172.21.15.148:6789/0 conn(0x7fe3c806ed00 legacy=0x7fe3c80d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c6ffd640 1 -- 172.21.15.105:0/1135248749 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c80b31c0 con 0x7fe3c806a580 2023-11-28T18:29:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c6ffd640 1 -- 172.21.15.105:0/1135248749 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fe3c0001a00 con 0x7fe3c806a580 2023-11-28T18:29:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c6ffd640 1 -- 172.21.15.105:0/1135248749 >> v1:172.21.15.105:6789/0 conn(0x7fe3c806a580 legacy=0x7fe3c806d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c6ffd640 1 --2- 172.21.15.105:0/1135248749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3b4003b10 0x7fe3b4003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c6ffd640 1 --2- 172.21.15.105:0/1135248749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3b40047d0 0x7fe3b4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c6ffd640 1 --2- 172.21.15.105:0/1135248749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3b4005190 0x7fe3b400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c6ffd640 1 -- 172.21.15.105:0/1135248749 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c80b34a0 con 0x7fe3b4005190 2023-11-28T18:29:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c6ffd640 1 -- 172.21.15.105:0/1135248749 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b400c010 con 0x7fe3b40047d0 2023-11-28T18:29:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c6ffd640 1 -- 172.21.15.105:0/1135248749 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b400c1d0 con 0x7fe3b4003b10 2023-11-28T18:29:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c7fff640 1 --2- 172.21.15.105:0/1135248749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3b4005190 0x7fe3b400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3bffff640 1 --2- 172.21.15.105:0/1135248749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3b40047d0 0x7fe3b4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:49.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c77fe640 1 --2- 172.21.15.105:0/1135248749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3b4003b10 0x7fe3b4003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c7fff640 1 -- 172.21.15.105:0/1135248749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3b4003b10 msgr2=0x7fe3b4003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c7fff640 1 --2- 172.21.15.105:0/1135248749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3b4003b10 0x7fe3b4003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c7fff640 1 -- 172.21.15.105:0/1135248749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3b40047d0 msgr2=0x7fe3b4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c7fff640 1 --2- 172.21.15.105:0/1135248749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3b40047d0 0x7fe3b4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.245+0000 7fe3c7fff640 1 -- 172.21.15.105:0/1135248749 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b4003530 con 0x7fe3b4005190 2023-11-28T18:29:49.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.249+0000 7fe3c7fff640 1 -- 172.21.15.105:0/1135248749 reap_dead start 2023-11-28T18:29:49.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.249+0000 7fe3c7fff640 1 --2- 172.21.15.105:0/1135248749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3b4005190 0x7fe3b400b990 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000a7a0 tx=0x7fe3c001fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4736ade79102a3a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:49.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.249+0000 7fe3c6ffd640 1 -- 172.21.15.105:0/1135248749 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c00aa630 con 0x7fe3b4005190 2023-11-28T18:29:49.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.249+0000 7fe3c6ffd640 1 -- 172.21.15.105:0/1135248749 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c00aa7b0 con 0x7fe3b4005190 2023-11-28T18:29:49.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.249+0000 7fe3c6ffd640 1 -- 172.21.15.105:0/1135248749 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c00aab60 con 0x7fe3b4005190 2023-11-28T18:29:49.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.249+0000 7fe3ce09a640 1 -- 172.21.15.105:0/1135248749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3b4005190 msgr2=0x7fe3b400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:49.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.249+0000 7fe3ce09a640 1 --2- 172.21.15.105:0/1135248749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3b4005190 0x7fe3b400b990 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000a7a0 tx=0x7fe3c001fa00 comp rx=0 tx=0).stop 2023-11-28T18:29:49.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.249+0000 7fe3ce09a640 1 -- 172.21.15.105:0/1135248749 shutdown_connections 2023-11-28T18:29:49.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.249+0000 7fe3ce09a640 1 --2- 172.21.15.105:0/1135248749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3b4005190 0x7fe3b400b990 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.249+0000 7fe3ce09a640 1 -- 172.21.15.105:0/1135248749 >> 172.21.15.105:0/1135248749 conn(0x7fe3c806b860 msgr2=0x7fe3c806bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.249+0000 7fe3ce09a640 1 -- 172.21.15.105:0/1135248749 shutdown_connections 2023-11-28T18:29:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.249+0000 7fe3ce09a640 1 -- 172.21.15.105:0/1135248749 wait complete. 2023-11-28T18:29:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3ce09a640 1 Processor -- start 2023-11-28T18:29:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3ce09a640 1 -- start start 2023-11-28T18:29:49.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3ce09a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3b4005190 0x7fe3c815c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:49.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3ce09a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3c81f2840 0x7fe3c81f2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:49.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3ce09a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3c81f37c0 0x7fe3c80c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:49.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3ce09a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c80c5a60 con 0x7fe3c81f2840 2023-11-28T18:29:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3ce09a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c80c4de0 con 0x7fe3b4005190 2023-11-28T18:29:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3ce09a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c80c50c0 con 0x7fe3c81f37c0 2023-11-28T18:29:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3bffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3c81f2840 0x7fe3c81f2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3bffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3c81f2840 0x7fe3c81f2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38416/0 (socket says 172.21.15.105:38416) 2023-11-28T18:29:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3bffff640 1 -- 172.21.15.105:0/3157694650 learned_addr learned my addr 172.21.15.105:0/3157694650 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3c7fff640 1 --2- 172.21.15.105:0/3157694650 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3c81f37c0 0x7fe3c80c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3c77fe640 1 --2- 172.21.15.105:0/3157694650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3b4005190 0x7fe3c815c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3bffff640 1 -- 172.21.15.105:0/3157694650 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3c81f37c0 msgr2=0x7fe3c80c9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3bffff640 1 --2- 172.21.15.105:0/3157694650 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3c81f37c0 0x7fe3c80c9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3bffff640 1 -- 172.21.15.105:0/3157694650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3b4005190 msgr2=0x7fe3c815c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3bffff640 1 --2- 172.21.15.105:0/3157694650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3b4005190 0x7fe3c815c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3bffff640 1 -- 172.21.15.105:0/3157694650 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c80c53a0 con 0x7fe3c81f2840 2023-11-28T18:29:49.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3bffff640 1 --2- 172.21.15.105:0/3157694650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3c81f2840 0x7fe3c81f2f50 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000f770 tx=0x7fe3b000fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:49.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3c4ff9640 1 -- 172.21.15.105:0/3157694650 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0005320 con 0x7fe3c81f2840 2023-11-28T18:29:49.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3c4ff9640 1 -- 172.21.15.105:0/3157694650 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b0005480 con 0x7fe3c81f2840 2023-11-28T18:29:49.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3c4ff9640 1 -- 172.21.15.105:0/3157694650 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0006a20 con 0x7fe3c81f2840 2023-11-28T18:29:49.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3ce09a640 1 -- 172.21.15.105:0/3157694650 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c80c5680 con 0x7fe3c81f2840 2023-11-28T18:29:49.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.253+0000 7fe3ce09a640 1 -- 172.21.15.105:0/3157694650 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c81f3490 con 0x7fe3c81f2840 2023-11-28T18:29:49.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.257+0000 7fe3ce09a640 1 -- 172.21.15.105:0/3157694650 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c80c5ba0 con 0x7fe3c81f2840 2023-11-28T18:29:49.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.257+0000 7fe3c4ff9640 1 -- 172.21.15.105:0/3157694650 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fe3b0006b80 con 0x7fe3c81f2840 2023-11-28T18:29:49.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.257+0000 7fe3c4ff9640 1 --2- 172.21.15.105:0/3157694650 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3a80668d0 0x7fe3a8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:49.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.257+0000 7fe3c4ff9640 1 -- 172.21.15.105:0/3157694650 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fe3b0083c60 con 0x7fe3c81f2840 2023-11-28T18:29:49.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.257+0000 7fe3c77fe640 1 --2- 172.21.15.105:0/3157694650 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3a80668d0 0x7fe3a8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:49.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.257+0000 7fe3c77fe640 1 --2- 172.21.15.105:0/3157694650 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3a80668d0 0x7fe3a8068d90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80b4ad0 tx=0x7fe3b8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:49.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.261+0000 7fe3c4ff9640 1 -- 172.21.15.105:0/3157694650 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe3c80c5ba0 con 0x7fe3c81f2840 2023-11-28T18:29:49.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.417+0000 7fe3ce09a640 1 -- 172.21.15.105:0/3157694650 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3c80c5ba0 con 0x7fe3c81f2840 2023-11-28T18:29:49.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.417+0000 7fe3c4ff9640 1 -- 172.21.15.105:0/3157694650 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16971 (secure 0 0 0) 0x7fe3c80c5ba0 con 0x7fe3c81f2840 2023-11-28T18:29:49.422 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:29:49.422 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":92,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:29:16.406699+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:29:08.048399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-28T18:29:08.699964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":5}]},{"pgid":"14.17","mappings":[{"from":1,"to":2}]},{"pgid":"14.21","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:29:49.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.417+0000 7fe3ce09a640 1 -- 172.21.15.105:0/3157694650 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3a80668d0 msgr2=0x7fe3a8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:49.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.417+0000 7fe3ce09a640 1 --2- 172.21.15.105:0/3157694650 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3a80668d0 0x7fe3a8068d90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80b4ad0 tx=0x7fe3b8009190 comp rx=0 tx=0).stop 2023-11-28T18:29:49.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.417+0000 7fe3ce09a640 1 -- 172.21.15.105:0/3157694650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3c81f2840 msgr2=0x7fe3c81f2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:49.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.417+0000 7fe3ce09a640 1 --2- 172.21.15.105:0/3157694650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3c81f2840 0x7fe3c81f2f50 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000f770 tx=0x7fe3b000fb70 comp rx=0 tx=0).stop 2023-11-28T18:29:49.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.417+0000 7fe3ce09a640 1 -- 172.21.15.105:0/3157694650 shutdown_connections 2023-11-28T18:29:49.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.417+0000 7fe3ce09a640 1 --2- 172.21.15.105:0/3157694650 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3c81f37c0 0x7fe3c80c9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.417+0000 7fe3ce09a640 1 --2- 172.21.15.105:0/3157694650 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe3a80668d0 0x7fe3a8068d90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.417+0000 7fe3ce09a640 1 --2- 172.21.15.105:0/3157694650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3c81f2840 0x7fe3c81f2f50 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.417+0000 7fe3ce09a640 1 --2- 172.21.15.105:0/3157694650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3b4005190 0x7fe3c815c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.417+0000 7fe3ce09a640 1 -- 172.21.15.105:0/3157694650 >> 172.21.15.105:0/3157694650 conn(0x7fe3c806b860 msgr2=0x7fe3c806bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:49.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.417+0000 7fe3ce09a640 1 -- 172.21.15.105:0/3157694650 shutdown_connections 2023-11-28T18:29:49.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.421+0000 7fe3ce09a640 1 -- 172.21.15.105:0/3157694650 wait complete. 2023-11-28T18:29:49.443 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:29:08.048399+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:29:08.699964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:29:49.444 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:29:49.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.577+0000 7f9fdc357640 1 Processor -- start 2023-11-28T18:29:49.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.581+0000 7f9fdc357640 1 -- start start 2023-11-28T18:29:49.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.581+0000 7f9fdc357640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9fd4071960 0x7f9fd415f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:49.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.581+0000 7f9fdc357640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9fd415b030 0x7f9fd40709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:49.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.581+0000 7f9fdc357640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd40701d0 con 0x7f9fd40b4290 2023-11-28T18:29:49.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.581+0000 7f9fdc357640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd406df80 con 0x7f9fd4071960 2023-11-28T18:29:49.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.581+0000 7f9fdc357640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd4071df0 con 0x7f9fd415b030 2023-11-28T18:29:49.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.581+0000 7f9fd98f2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9fd4071960 0x7f9fd415f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:49.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.581+0000 7f9fd98f2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9fd4071960 0x7f9fd415f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38426/0 (socket says 172.21.15.105:38426) 2023-11-28T18:29:49.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.581+0000 7f9fd98f2640 1 -- 172.21.15.105:0/2873528225 learned_addr learned my addr 172.21.15.105:0/2873528225 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:49.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.581+0000 7f9fd98f2640 1 -- 172.21.15.105:0/2873528225 >> v2:172.21.15.148:3300/0 conn(0x7f9fd415b030 msgr2=0x7f9fd40709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:49.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.581+0000 7f9fda8f4640 1 --2- 172.21.15.105:0/2873528225 >> v2:172.21.15.148:3300/0 conn(0x7f9fd415b030 0x7f9fd40709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:49.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.581+0000 7f9fd98f2640 1 --2- 172.21.15.105:0/2873528225 >> v2:172.21.15.148:3300/0 conn(0x7f9fd415b030 0x7f9fd40709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.581+0000 7f9fd98f2640 1 -- 172.21.15.105:0/2873528225 >> v1:172.21.15.148:6789/0 conn(0x7f9fd40b4290 legacy=0x7f9fd40b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:49.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.581+0000 7f9fd98f2640 1 -- 172.21.15.105:0/2873528225 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd40720d0 con 0x7f9fd4071960 2023-11-28T18:29:49.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.581+0000 7f9fda8f4640 1 --2- 172.21.15.105:0/2873528225 >> v2:172.21.15.148:3300/0 conn(0x7f9fd415b030 0x7f9fd40709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:49.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.585+0000 7f9fd98f2640 1 --2- 172.21.15.105:0/2873528225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fd4071960 0x7f9fd415f3b0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4067c50 tx=0x7f9fc401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7105a1c30d69478b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:49.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.585+0000 7f9fd90f1640 1 -- 172.21.15.105:0/2873528225 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc40ac600 con 0x7f9fd4071960 2023-11-28T18:29:49.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.585+0000 7f9fd90f1640 1 -- 172.21.15.105:0/2873528225 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc40a8050 con 0x7f9fd4071960 2023-11-28T18:29:49.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.585+0000 7f9fd90f1640 1 -- 172.21.15.105:0/2873528225 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc40ac980 con 0x7f9fd4071960 2023-11-28T18:29:49.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.585+0000 7f9fdc357640 1 -- 172.21.15.105:0/2873528225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fd4071960 msgr2=0x7f9fd415f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.585+0000 7f9fdc357640 1 --2- 172.21.15.105:0/2873528225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fd4071960 0x7f9fd415f3b0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4067c50 tx=0x7f9fc401c490 comp rx=0 tx=0).stop 2023-11-28T18:29:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.585+0000 7f9fdc357640 1 -- 172.21.15.105:0/2873528225 shutdown_connections 2023-11-28T18:29:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.585+0000 7f9fdc357640 1 --2- 172.21.15.105:0/2873528225 >> v2:172.21.15.148:3300/0 conn(0x7f9fd415b030 0x7f9fd40709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.585+0000 7f9fdc357640 1 --2- 172.21.15.105:0/2873528225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fd4071960 0x7f9fd415f3b0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.585+0000 7f9fdc357640 1 -- 172.21.15.105:0/2873528225 >> 172.21.15.105:0/2873528225 conn(0x7f9fd40ceb90 msgr2=0x7f9fd40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.585+0000 7f9fdc357640 1 -- 172.21.15.105:0/2873528225 shutdown_connections 2023-11-28T18:29:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.585+0000 7f9fdc357640 1 -- 172.21.15.105:0/2873528225 wait complete. 2023-11-28T18:29:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.585+0000 7f9fdc357640 1 Processor -- start 2023-11-28T18:29:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.585+0000 7f9fdc357640 1 -- start start 2023-11-28T18:29:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fdc357640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fd40b4290 0x7f9fd4099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fdc357640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fd415b030 0x7f9fd409b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fdc357640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9fd409b8f0 0x7f9fd409bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fdc357640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd4099f90 con 0x7f9fd415b030 2023-11-28T18:29:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fdc357640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd409c2c0 con 0x7f9fd40b4290 2023-11-28T18:29:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fdc357640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd40b3f50 con 0x7f9fd409b8f0 2023-11-28T18:29:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fd98f2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fd415b030 0x7f9fd409b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fd98f2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fd415b030 0x7f9fd409b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38442/0 (socket says 172.21.15.105:38442) 2023-11-28T18:29:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fd98f2640 1 -- 172.21.15.105:0/1571085546 learned_addr learned my addr 172.21.15.105:0/1571085546 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fda0f3640 1 --2- 172.21.15.105:0/1571085546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fd40b4290 0x7f9fd4099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fda8f4640 1 --2- 172.21.15.105:0/1571085546 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9fd409b8f0 0x7f9fd409bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fd98f2640 1 -- 172.21.15.105:0/1571085546 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9fd409b8f0 msgr2=0x7f9fd409bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fd98f2640 1 --2- 172.21.15.105:0/1571085546 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9fd409b8f0 0x7f9fd409bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fd98f2640 1 -- 172.21.15.105:0/1571085546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fd40b4290 msgr2=0x7f9fd4099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fd98f2640 1 --2- 172.21.15.105:0/1571085546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fd40b4290 0x7f9fd4099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fd98f2640 1 -- 172.21.15.105:0/1571085546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc40094d0 con 0x7f9fd415b030 2023-11-28T18:29:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fd98f2640 1 --2- 172.21.15.105:0/1571085546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fd415b030 0x7f9fd409b3b0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f9fc401a0c0 tx=0x7f9fc401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fcaffd640 1 -- 172.21.15.105:0/1571085546 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc40bd070 con 0x7f9fd415b030 2023-11-28T18:29:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fcaffd640 1 -- 172.21.15.105:0/1571085546 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc40b7420 con 0x7f9fd415b030 2023-11-28T18:29:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fcaffd640 1 -- 172.21.15.105:0/1571085546 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc40b7790 con 0x7f9fd415b030 2023-11-28T18:29:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fdc357640 1 -- 172.21.15.105:0/1571085546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd40b23c0 con 0x7f9fd415b030 2023-11-28T18:29:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fdc357640 1 -- 172.21.15.105:0/1571085546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd40b2980 con 0x7f9fd415b030 2023-11-28T18:29:49.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.589+0000 7f9fdc357640 1 -- 172.21.15.105:0/1571085546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd406e380 con 0x7f9fd415b030 2023-11-28T18:29:49.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.593+0000 7f9fcaffd640 1 -- 172.21.15.105:0/1571085546 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f9fc40b3040 con 0x7f9fd415b030 2023-11-28T18:29:49.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.593+0000 7f9fcaffd640 1 --2- 172.21.15.105:0/1571085546 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9fb80668d0 0x7f9fb8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:49.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.593+0000 7f9fcaffd640 1 -- 172.21.15.105:0/1571085546 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f9fc411e0e0 con 0x7f9fd415b030 2023-11-28T18:29:49.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.593+0000 7f9fda0f3640 1 --2- 172.21.15.105:0/1571085546 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9fb80668d0 0x7f9fb8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:49.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.593+0000 7f9fda0f3640 1 --2- 172.21.15.105:0/1571085546 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9fb80668d0 0x7f9fb8068d90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0007fd0 tx=0x7f9fd0007e40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:49.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.597+0000 7f9fcaffd640 1 -- 172.21.15.105:0/1571085546 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9fc40eb200 con 0x7f9fd415b030 2023-11-28T18:29:49.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.745+0000 7f9fdc357640 1 -- 172.21.15.105:0/1571085546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9fd4067570 con 0x7f9fd415b030 2023-11-28T18:29:49.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.749+0000 7f9fcaffd640 1 -- 172.21.15.105:0/1571085546 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9fc40ee820 con 0x7f9fd415b030 2023-11-28T18:29:49.752 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:29:49.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.749+0000 7f9fdc357640 1 -- 172.21.15.105:0/1571085546 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9fb80668d0 msgr2=0x7f9fb8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:49.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.749+0000 7f9fdc357640 1 --2- 172.21.15.105:0/1571085546 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9fb80668d0 0x7f9fb8068d90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0007fd0 tx=0x7f9fd0007e40 comp rx=0 tx=0).stop 2023-11-28T18:29:49.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.749+0000 7f9fdc357640 1 -- 172.21.15.105:0/1571085546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fd415b030 msgr2=0x7f9fd409b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:49.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.749+0000 7f9fdc357640 1 --2- 172.21.15.105:0/1571085546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fd415b030 0x7f9fd409b3b0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f9fc401a0c0 tx=0x7f9fc401acd0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.749+0000 7f9fdc357640 1 -- 172.21.15.105:0/1571085546 shutdown_connections 2023-11-28T18:29:49.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.749+0000 7f9fdc357640 1 --2- 172.21.15.105:0/1571085546 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9fd409b8f0 0x7f9fd409bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.749+0000 7f9fdc357640 1 --2- 172.21.15.105:0/1571085546 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9fb80668d0 0x7f9fb8068d90 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.749+0000 7f9fdc357640 1 --2- 172.21.15.105:0/1571085546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fd415b030 0x7f9fd409b3b0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.749+0000 7f9fdc357640 1 --2- 172.21.15.105:0/1571085546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fd40b4290 0x7f9fd4099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.749+0000 7f9fdc357640 1 -- 172.21.15.105:0/1571085546 >> 172.21.15.105:0/1571085546 conn(0x7f9fd40ceb90 msgr2=0x7f9fd40cb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:49.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.749+0000 7f9fdc357640 1 -- 172.21.15.105:0/1571085546 shutdown_connections 2023-11-28T18:29:49.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.749+0000 7f9fdc357640 1 -- 172.21.15.105:0/1571085546 wait complete. 2023-11-28T18:29:49.773 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:29:49.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.909+0000 7f469f996640 1 Processor -- start 2023-11-28T18:29:49.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.909+0000 7f469f996640 1 -- start start 2023-11-28T18:29:49.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.909+0000 7f469f996640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f469806d610 0x7f46980cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:49.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.909+0000 7f469f996640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4698162820 con 0x7f46980ce530 2023-11-28T18:29:49.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.909+0000 7f469f996640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4698161c10 con 0x7f46980ae100 2023-11-28T18:29:49.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.909+0000 7f469f996640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4698161ef0 con 0x7f469806d610 2023-11-28T18:29:49.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.909+0000 7f469cf31640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f46980ce530 0x7f46980ad9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40460/0 (socket says 172.21.15.105:40460) 2023-11-28T18:29:49.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.909+0000 7f469cf31640 1 -- 172.21.15.105:0/3637653115 learned_addr learned my addr 172.21.15.105:0/3637653115 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:49.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.909+0000 7f469d732640 1 --2- 172.21.15.105:0/3637653115 >> v2:172.21.15.148:3300/0 conn(0x7f469806d610 0x7f46980cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:49.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.909+0000 7f468ffff640 1 -- 172.21.15.105:0/3637653115 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244072455 0 0) 0x7f4698162820 con 0x7f46980ce530 2023-11-28T18:29:49.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.909+0000 7f468ffff640 1 -- 172.21.15.105:0/3637653115 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4684003530 con 0x7f46980ce530 2023-11-28T18:29:49.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.913+0000 7f469d732640 1 -- 172.21.15.105:0/3637653115 >> v1:172.21.15.148:6789/0 conn(0x7f46980ae100 legacy=0x7f46980cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:49.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.913+0000 7f469d732640 1 -- 172.21.15.105:0/3637653115 >> v1:172.21.15.105:6789/0 conn(0x7f46980ce530 legacy=0x7f46980ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:49.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.913+0000 7f469d732640 1 -- 172.21.15.105:0/3637653115 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46981621d0 con 0x7f469806d610 2023-11-28T18:29:49.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.913+0000 7f469d732640 1 --2- 172.21.15.105:0/3637653115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f469806d610 0x7f46980cdff0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f46940097c0 tx=0x7f469401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69aff42016caa5e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:49.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.913+0000 7f468ffff640 1 -- 172.21.15.105:0/3637653115 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46940ab670 con 0x7f469806d610 2023-11-28T18:29:49.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.913+0000 7f468ffff640 1 -- 172.21.15.105:0/3637653115 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46940ab7d0 con 0x7f469806d610 2023-11-28T18:29:49.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.913+0000 7f468ffff640 1 -- 172.21.15.105:0/3637653115 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46940abb60 con 0x7f469806d610 2023-11-28T18:29:49.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.913+0000 7f469f996640 1 -- 172.21.15.105:0/3637653115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f469806d610 msgr2=0x7f46980cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:49.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.913+0000 7f469f996640 1 --2- 172.21.15.105:0/3637653115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f469806d610 0x7f46980cdff0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f46940097c0 tx=0x7f469401c450 comp rx=0 tx=0).stop 2023-11-28T18:29:49.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.913+0000 7f469f996640 1 -- 172.21.15.105:0/3637653115 shutdown_connections 2023-11-28T18:29:49.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.913+0000 7f469f996640 1 --2- 172.21.15.105:0/3637653115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f469806d610 0x7f46980cdff0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.913+0000 7f469f996640 1 -- 172.21.15.105:0/3637653115 >> 172.21.15.105:0/3637653115 conn(0x7f46980ceb90 msgr2=0x7f46980cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:49.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f469f996640 1 -- 172.21.15.105:0/3637653115 shutdown_connections 2023-11-28T18:29:49.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f469f996640 1 -- 172.21.15.105:0/3637653115 wait complete. 2023-11-28T18:29:49.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f469f996640 1 Processor -- start 2023-11-28T18:29:49.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f469f996640 1 -- start start 2023-11-28T18:29:49.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f469f996640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f469806d610 0x7f4698157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:49.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f469f996640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46980ae100 0x7f469809c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:49.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f469f996640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46980ce530 0x7f4698164b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:49.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f469f996640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469809b110 con 0x7f46980ae100 2023-11-28T18:29:49.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f469f996640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469809b5a0 con 0x7f469806d610 2023-11-28T18:29:49.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f469f996640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469809a500 con 0x7f46980ce530 2023-11-28T18:29:49.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f469cf31640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46980ae100 0x7f469809c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:49.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f469cf31640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46980ae100 0x7f469809c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38452/0 (socket says 172.21.15.105:38452) 2023-11-28T18:29:49.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f469cf31640 1 -- 172.21.15.105:0/3611386127 learned_addr learned my addr 172.21.15.105:0/3611386127 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:49.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f469d732640 1 --2- 172.21.15.105:0/3611386127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f469806d610 0x7f4698157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:49.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f469df33640 1 --2- 172.21.15.105:0/3611386127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46980ce530 0x7f4698164b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:49.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f469cf31640 1 -- 172.21.15.105:0/3611386127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46980ce530 msgr2=0x7f4698164b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:49.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f469cf31640 1 --2- 172.21.15.105:0/3611386127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46980ce530 0x7f4698164b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f469cf31640 1 -- 172.21.15.105:0/3611386127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f469806d610 msgr2=0x7f4698157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:49.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f469cf31640 1 --2- 172.21.15.105:0/3611386127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f469806d610 0x7f4698157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:49.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f469cf31640 1 -- 172.21.15.105:0/3611386127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4694009420 con 0x7f46980ae100 2023-11-28T18:29:49.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f469df33640 1 --2- 172.21.15.105:0/3611386127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46980ce530 0x7f4698164b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:49.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f469cf31640 1 --2- 172.21.15.105:0/3611386127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46980ae100 0x7f469809c1b0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f468800c420 tx=0x7f468800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:49.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.917+0000 7f468dffb640 1 -- 172.21.15.105:0/3611386127 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688005cf0 con 0x7f46980ae100 2023-11-28T18:29:49.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.921+0000 7f469f996640 1 -- 172.21.15.105:0/3611386127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469809a7e0 con 0x7f46980ae100 2023-11-28T18:29:49.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.921+0000 7f468dffb640 1 -- 172.21.15.105:0/3611386127 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4688005e50 con 0x7f46980ae100 2023-11-28T18:29:49.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.921+0000 7f468dffb640 1 -- 172.21.15.105:0/3611386127 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46880072d0 con 0x7f46980ae100 2023-11-28T18:29:49.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.921+0000 7f469f996640 1 -- 172.21.15.105:0/3611386127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469809ada0 con 0x7f46980ae100 2023-11-28T18:29:49.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.921+0000 7f469f996640 1 -- 172.21.15.105:0/3611386127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f469809b280 con 0x7f46980ae100 2023-11-28T18:29:49.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.921+0000 7f468dffb640 1 -- 172.21.15.105:0/3611386127 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f4688007430 con 0x7f46980ae100 2023-11-28T18:29:49.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.921+0000 7f468dffb640 1 --2- 172.21.15.105:0/3611386127 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f467c0668d0 0x7f467c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:49.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.921+0000 7f468dffb640 1 -- 172.21.15.105:0/3611386127 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f4688083c20 con 0x7f46980ae100 2023-11-28T18:29:49.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.921+0000 7f469d732640 1 --2- 172.21.15.105:0/3611386127 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f467c0668d0 0x7f467c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:49.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.921+0000 7f469d732640 1 --2- 172.21.15.105:0/3611386127 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f467c0668d0 0x7f467c068d90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f46940af000 tx=0x7f46940a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:49.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:49.925+0000 7f468dffb640 1 -- 172.21.15.105:0/3611386127 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4688050cb0 con 0x7f46980ae100 2023-11-28T18:29:50.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.077+0000 7f469f996640 1 -- 172.21.15.105:0/3611386127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4698067570 con 0x7f46980ae100 2023-11-28T18:29:50.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.077+0000 7f468dffb640 1 -- 172.21.15.105:0/3611386127 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f46880170a0 con 0x7f46980ae100 2023-11-28T18:29:50.082 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:29:50.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.077+0000 7f469f996640 1 -- 172.21.15.105:0/3611386127 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f467c0668d0 msgr2=0x7f467c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.077+0000 7f469f996640 1 --2- 172.21.15.105:0/3611386127 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f467c0668d0 0x7f467c068d90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f46940af000 tx=0x7f46940a7000 comp rx=0 tx=0).stop 2023-11-28T18:29:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.077+0000 7f469f996640 1 -- 172.21.15.105:0/3611386127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46980ae100 msgr2=0x7f469809c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.077+0000 7f469f996640 1 --2- 172.21.15.105:0/3611386127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46980ae100 0x7f469809c1b0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f468800c420 tx=0x7f468800c8e0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.077+0000 7f469f996640 1 -- 172.21.15.105:0/3611386127 shutdown_connections 2023-11-28T18:29:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.077+0000 7f469f996640 1 --2- 172.21.15.105:0/3611386127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46980ce530 0x7f4698164b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.077+0000 7f469f996640 1 --2- 172.21.15.105:0/3611386127 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f467c0668d0 0x7f467c068d90 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.081+0000 7f469f996640 1 --2- 172.21.15.105:0/3611386127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46980ae100 0x7f469809c1b0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.081+0000 7f469f996640 1 --2- 172.21.15.105:0/3611386127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f469806d610 0x7f4698157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.081+0000 7f469f996640 1 -- 172.21.15.105:0/3611386127 >> 172.21.15.105:0/3611386127 conn(0x7f46980ceb90 msgr2=0x7f46980af0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.081+0000 7f469f996640 1 -- 172.21.15.105:0/3611386127 shutdown_connections 2023-11-28T18:29:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.081+0000 7f469f996640 1 -- 172.21.15.105:0/3611386127 wait complete. 2023-11-28T18:29:50.103 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:29:50.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.241+0000 7f6a6f5df640 1 Processor -- start 2023-11-28T18:29:50.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.241+0000 7f6a6f5df640 1 -- start start 2023-11-28T18:29:50.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.241+0000 7f6a6f5df640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a6815d320 0x7f6a6806c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:50.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.241+0000 7f6a6f5df640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6a680ce230 0x7f6a680cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:50.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.241+0000 7f6a6f5df640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6a680cdbb0 0x7f6a680cef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:50.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.241+0000 7f6a6f5df640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a680b3c40 con 0x7f6a680ce230 2023-11-28T18:29:50.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.241+0000 7f6a6f5df640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a6806ca80 con 0x7f6a680cdbb0 2023-11-28T18:29:50.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.241+0000 7f6a6f5df640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6815d700 con 0x7f6a6815d320 2023-11-28T18:29:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.241+0000 7f6a6cb7a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6a680ce230 0x7f6a680cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.241+0000 7f6a6cb7a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6a680ce230 0x7f6a680cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38458/0 (socket says 172.21.15.105:38458) 2023-11-28T18:29:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.241+0000 7f6a6cb7a640 1 -- 172.21.15.105:0/822886686 learned_addr learned my addr 172.21.15.105:0/822886686 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.241+0000 7f6a6db7c640 1 --2- 172.21.15.105:0/822886686 >> v2:172.21.15.148:3300/0 conn(0x7f6a680cdbb0 0x7f6a680cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.241+0000 7f6a6d37b640 1 --2- 172.21.15.105:0/822886686 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a6815d320 0x7f6a6806c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.241+0000 7f6a6cb7a640 1 -- 172.21.15.105:0/822886686 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a6815d320 msgr2=0x7f6a6806c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.241+0000 7f6a6cb7a640 1 --2- 172.21.15.105:0/822886686 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a6815d320 0x7f6a6806c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.241+0000 7f6a6cb7a640 1 -- 172.21.15.105:0/822886686 >> v2:172.21.15.148:3300/0 conn(0x7f6a680cdbb0 msgr2=0x7f6a680cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.241+0000 7f6a6cb7a640 1 --2- 172.21.15.105:0/822886686 >> v2:172.21.15.148:3300/0 conn(0x7f6a680cdbb0 0x7f6a680cef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.241+0000 7f6a6cb7a640 1 -- 172.21.15.105:0/822886686 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a680cdfc0 con 0x7f6a680ce230 2023-11-28T18:29:50.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.245+0000 7f6a6cb7a640 1 --2- 172.21.15.105:0/822886686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a680ce230 0x7f6a680cd670 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f6a6806d850 tx=0x7f6a5801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5dbed869c34c17 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:50.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.245+0000 7f6a5ffff640 1 -- 172.21.15.105:0/822886686 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a580a7070 con 0x7f6a680ce230 2023-11-28T18:29:50.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.245+0000 7f6a5ffff640 1 -- 172.21.15.105:0/822886686 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a580ab430 con 0x7f6a680ce230 2023-11-28T18:29:50.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.245+0000 7f6a5ffff640 1 -- 172.21.15.105:0/822886686 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a580ab7c0 con 0x7f6a680ce230 2023-11-28T18:29:50.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.245+0000 7f6a6f5df640 1 -- 172.21.15.105:0/822886686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a680ce230 msgr2=0x7f6a680cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:50.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.245+0000 7f6a6f5df640 1 --2- 172.21.15.105:0/822886686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a680ce230 0x7f6a680cd670 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f6a6806d850 tx=0x7f6a5801c470 comp rx=0 tx=0).stop 2023-11-28T18:29:50.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.245+0000 7f6a6f5df640 1 -- 172.21.15.105:0/822886686 shutdown_connections 2023-11-28T18:29:50.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.245+0000 7f6a6f5df640 1 --2- 172.21.15.105:0/822886686 >> v2:172.21.15.148:3300/0 conn(0x7f6a680cdbb0 0x7f6a680cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.245+0000 7f6a6f5df640 1 --2- 172.21.15.105:0/822886686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a680ce230 0x7f6a680cd670 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.245+0000 7f6a6f5df640 1 --2- 172.21.15.105:0/822886686 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a6815d320 0x7f6a6806c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.245+0000 7f6a6f5df640 1 -- 172.21.15.105:0/822886686 >> 172.21.15.105:0/822886686 conn(0x7f6a680d0df0 msgr2=0x7f6a680d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:50.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.245+0000 7f6a6f5df640 1 -- 172.21.15.105:0/822886686 shutdown_connections 2023-11-28T18:29:50.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.245+0000 7f6a6f5df640 1 -- 172.21.15.105:0/822886686 wait complete. 2023-11-28T18:29:50.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a6f5df640 1 Processor -- start 2023-11-28T18:29:50.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a6f5df640 1 -- start start 2023-11-28T18:29:50.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a6f5df640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a680cdbb0 0x7f6a6815f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a6f5df640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a680ce230 0x7f6a68151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a6f5df640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a6815d320 0x7f6a6815ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a6f5df640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6815e3b0 con 0x7f6a680ce230 2023-11-28T18:29:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a6f5df640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a68151bb0 con 0x7f6a6815d320 2023-11-28T18:29:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a6f5df640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6815f900 con 0x7f6a680cdbb0 2023-11-28T18:29:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a6cb7a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a680ce230 0x7f6a68151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a6cb7a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a680ce230 0x7f6a68151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38464/0 (socket says 172.21.15.105:38464) 2023-11-28T18:29:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a6cb7a640 1 -- 172.21.15.105:0/3708718285 learned_addr learned my addr 172.21.15.105:0/3708718285 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:50.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a6d37b640 1 --2- 172.21.15.105:0/3708718285 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a680cdbb0 0x7f6a6815f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:50.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a6db7c640 1 --2- 172.21.15.105:0/3708718285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a6815d320 0x7f6a6815ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:50.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a6cb7a640 1 -- 172.21.15.105:0/3708718285 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a680cdbb0 msgr2=0x7f6a6815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:50.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a6cb7a640 1 --2- 172.21.15.105:0/3708718285 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a680cdbb0 0x7f6a6815f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a6cb7a640 1 -- 172.21.15.105:0/3708718285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a6815d320 msgr2=0x7f6a6815ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:50.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a6cb7a640 1 --2- 172.21.15.105:0/3708718285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a6815d320 0x7f6a6815ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a6cb7a640 1 -- 172.21.15.105:0/3708718285 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a580094d0 con 0x7f6a680ce230 2023-11-28T18:29:50.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a6d37b640 1 --2- 172.21.15.105:0/3708718285 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a680cdbb0 0x7f6a6815f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:50.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a6cb7a640 1 --2- 172.21.15.105:0/3708718285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a680ce230 0x7f6a68151150 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f6a5801ab20 tx=0x7f6a58019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:50.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a5dffb640 1 -- 172.21.15.105:0/3708718285 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a580ba070 con 0x7f6a680ce230 2023-11-28T18:29:50.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a5dffb640 1 -- 172.21.15.105:0/3708718285 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a580b4420 con 0x7f6a680ce230 2023-11-28T18:29:50.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a5dffb640 1 -- 172.21.15.105:0/3708718285 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a580b47b0 con 0x7f6a680ce230 2023-11-28T18:29:50.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a6f5df640 1 -- 172.21.15.105:0/3708718285 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6815e600 con 0x7f6a680ce230 2023-11-28T18:29:50.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.249+0000 7f6a6f5df640 1 -- 172.21.15.105:0/3708718285 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6815ebc0 con 0x7f6a680ce230 2023-11-28T18:29:50.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.253+0000 7f6a6f5df640 1 -- 172.21.15.105:0/3708718285 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a68067570 con 0x7f6a680ce230 2023-11-28T18:29:50.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.253+0000 7f6a5dffb640 1 -- 172.21.15.105:0/3708718285 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f6a580b4910 con 0x7f6a680ce230 2023-11-28T18:29:50.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.253+0000 7f6a5dffb640 1 --2- 172.21.15.105:0/3708718285 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6a4c0668d0 0x7f6a4c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:50.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.253+0000 7f6a5dffb640 1 -- 172.21.15.105:0/3708718285 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f6a5811de50 con 0x7f6a680ce230 2023-11-28T18:29:50.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.253+0000 7f6a6d37b640 1 --2- 172.21.15.105:0/3708718285 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6a4c0668d0 0x7f6a4c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:50.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.253+0000 7f6a6d37b640 1 --2- 172.21.15.105:0/3708718285 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6a4c0668d0 0x7f6a4c068d90 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6a680cb5b0 tx=0x7f6a64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:50.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.257+0000 7f6a5dffb640 1 -- 172.21.15.105:0/3708718285 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6a580eaff0 con 0x7f6a680ce230 2023-11-28T18:29:50.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.413+0000 7f6a6f5df640 1 -- 172.21.15.105:0/3708718285 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6a680d0910 con 0x7f6a680ce230 2023-11-28T18:29:50.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.413+0000 7f6a5dffb640 1 -- 172.21.15.105:0/3708718285 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6a580ee610 con 0x7f6a680ce230 2023-11-28T18:29:50.418 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:29:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.413+0000 7f6a6f5df640 1 -- 172.21.15.105:0/3708718285 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6a4c0668d0 msgr2=0x7f6a4c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.413+0000 7f6a6f5df640 1 --2- 172.21.15.105:0/3708718285 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6a4c0668d0 0x7f6a4c068d90 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6a680cb5b0 tx=0x7f6a64008040 comp rx=0 tx=0).stop 2023-11-28T18:29:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.413+0000 7f6a6f5df640 1 -- 172.21.15.105:0/3708718285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a680ce230 msgr2=0x7f6a68151150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.413+0000 7f6a6f5df640 1 --2- 172.21.15.105:0/3708718285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a680ce230 0x7f6a68151150 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f6a5801ab20 tx=0x7f6a58019210 comp rx=0 tx=0).stop 2023-11-28T18:29:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.417+0000 7f6a6f5df640 1 -- 172.21.15.105:0/3708718285 shutdown_connections 2023-11-28T18:29:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.417+0000 7f6a6f5df640 1 --2- 172.21.15.105:0/3708718285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a6815d320 0x7f6a6815ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.417+0000 7f6a6f5df640 1 --2- 172.21.15.105:0/3708718285 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6a4c0668d0 0x7f6a4c068d90 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.417+0000 7f6a6f5df640 1 --2- 172.21.15.105:0/3708718285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a680ce230 0x7f6a68151150 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.417+0000 7f6a6f5df640 1 --2- 172.21.15.105:0/3708718285 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a680cdbb0 0x7f6a6815f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.417+0000 7f6a6f5df640 1 -- 172.21.15.105:0/3708718285 >> 172.21.15.105:0/3708718285 conn(0x7f6a680d0df0 msgr2=0x7f6a6815d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.417+0000 7f6a6f5df640 1 -- 172.21.15.105:0/3708718285 shutdown_connections 2023-11-28T18:29:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.417+0000 7f6a6f5df640 1 -- 172.21.15.105:0/3708718285 wait complete. 2023-11-28T18:29:50.439 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:29:50.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.573+0000 7f4bcca32640 1 Processor -- start 2023-11-28T18:29:50.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.573+0000 7f4bcca32640 1 -- start start 2023-11-28T18:29:50.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.573+0000 7f4bcca32640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4bc80b3d30 0x7f4bc81680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:50.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.573+0000 7f4bcca32640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc80b25b0 con 0x7f4bc80d3c30 2023-11-28T18:29:50.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.573+0000 7f4bcca32640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc80d1c30 con 0x7f4bc815d4f0 2023-11-28T18:29:50.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.573+0000 7f4bcca32640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc80b2900 con 0x7f4bc80b3d30 2023-11-28T18:29:50.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.573+0000 7f4bc6d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4bc80b3d30 0x7f4bc81680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:50.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.573+0000 7f4bc659c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4bc815d4f0 0x7f4bc80d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:54756/0 (socket says 172.21.15.105:54756) 2023-11-28T18:29:50.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.573+0000 7f4bc659c640 1 -- 172.21.15.105:0/4252343716 learned_addr learned my addr 172.21.15.105:0/4252343716 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:50.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.573+0000 7f4bc6d9d640 1 -- 172.21.15.105:0/4252343716 >> v1:172.21.15.148:6789/0 conn(0x7f4bc815d4f0 legacy=0x7f4bc80d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:50.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.573+0000 7f4bc6d9d640 1 -- 172.21.15.105:0/4252343716 >> v1:172.21.15.105:6789/0 conn(0x7f4bc80d3c30 legacy=0x7f4bc80d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:50.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.573+0000 7f4bc6d9d640 1 -- 172.21.15.105:0/4252343716 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc80cea30 con 0x7f4bc80b3d30 2023-11-28T18:29:50.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.577+0000 7f4bc6d9d640 1 --2- 172.21.15.105:0/4252343716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc80b3d30 0x7f4bc81680a0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00097c0 tx=0x7f4bc001c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9910b2784a29a8e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:50.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.577+0000 7f4bc5d9b640 1 -- 172.21.15.105:0/4252343716 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc00ab600 con 0x7f4bc80b3d30 2023-11-28T18:29:50.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.577+0000 7f4bc5d9b640 1 -- 172.21.15.105:0/4252343716 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc00a7050 con 0x7f4bc80b3d30 2023-11-28T18:29:50.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.577+0000 7f4bc5d9b640 1 -- 172.21.15.105:0/4252343716 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc00ab980 con 0x7f4bc80b3d30 2023-11-28T18:29:50.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.577+0000 7f4bcca32640 1 -- 172.21.15.105:0/4252343716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc80b3d30 msgr2=0x7f4bc81680a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:50.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.577+0000 7f4bcca32640 1 --2- 172.21.15.105:0/4252343716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc80b3d30 0x7f4bc81680a0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00097c0 tx=0x7f4bc001c330 comp rx=0 tx=0).stop 2023-11-28T18:29:50.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.577+0000 7f4bcca32640 1 -- 172.21.15.105:0/4252343716 shutdown_connections 2023-11-28T18:29:50.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.577+0000 7f4bcca32640 1 --2- 172.21.15.105:0/4252343716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc80b3d30 0x7f4bc81680a0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.577+0000 7f4bcca32640 1 -- 172.21.15.105:0/4252343716 >> 172.21.15.105:0/4252343716 conn(0x7f4bc80d3030 msgr2=0x7f4bc80d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:50.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.577+0000 7f4bcca32640 1 -- 172.21.15.105:0/4252343716 shutdown_connections 2023-11-28T18:29:50.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.577+0000 7f4bcca32640 1 -- 172.21.15.105:0/4252343716 wait complete. 2023-11-28T18:29:50.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bcca32640 1 Processor -- start 2023-11-28T18:29:50.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bcca32640 1 -- start start 2023-11-28T18:29:50.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bcca32640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc80b3d30 0x7f4bc814b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:50.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bcca32640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bc80d3c30 0x7f4bc81560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:50.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bcca32640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4bc815d4f0 0x7f4bc814e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:50.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bcca32640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc80d0a20 con 0x7f4bc80b3d30 2023-11-28T18:29:50.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bcca32640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc814b8f0 con 0x7f4bc80d3c30 2023-11-28T18:29:50.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bcca32640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc814f590 con 0x7f4bc815d4f0 2023-11-28T18:29:50.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bc659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc80b3d30 0x7f4bc814b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:50.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bc659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc80b3d30 0x7f4bc814b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38480/0 (socket says 172.21.15.105:38480) 2023-11-28T18:29:50.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bc659c640 1 -- 172.21.15.105:0/2792901982 learned_addr learned my addr 172.21.15.105:0/2792901982 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:50.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bc6d9d640 1 --2- 172.21.15.105:0/2792901982 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4bc815d4f0 0x7f4bc814e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:50.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bbdd9b640 1 --2- 172.21.15.105:0/2792901982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bc80d3c30 0x7f4bc81560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:50.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bc659c640 1 -- 172.21.15.105:0/2792901982 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4bc815d4f0 msgr2=0x7f4bc814e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:50.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bc659c640 1 --2- 172.21.15.105:0/2792901982 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4bc815d4f0 0x7f4bc814e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bc659c640 1 -- 172.21.15.105:0/2792901982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bc80d3c30 msgr2=0x7f4bc81560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:50.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bc659c640 1 --2- 172.21.15.105:0/2792901982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bc80d3c30 0x7f4bc81560d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bc659c640 1 -- 172.21.15.105:0/2792901982 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc0009420 con 0x7f4bc80b3d30 2023-11-28T18:29:50.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bc659c640 1 --2- 172.21.15.105:0/2792901982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc80b3d30 0x7f4bc814b3b0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f4bb800cbf0 tx=0x7f4bb800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:50.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bbf7fe640 1 -- 172.21.15.105:0/2792901982 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb800aca0 con 0x7f4bc80b3d30 2023-11-28T18:29:50.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bbf7fe640 1 -- 172.21.15.105:0/2792901982 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb800ae00 con 0x7f4bc80b3d30 2023-11-28T18:29:50.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bbf7fe640 1 -- 172.21.15.105:0/2792901982 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8012600 con 0x7f4bc80b3d30 2023-11-28T18:29:50.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bcca32640 1 -- 172.21.15.105:0/2792901982 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc8154700 con 0x7f4bc80b3d30 2023-11-28T18:29:50.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.581+0000 7f4bcca32640 1 -- 172.21.15.105:0/2792901982 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc81549a0 con 0x7f4bc80b3d30 2023-11-28T18:29:50.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.585+0000 7f4bcca32640 1 -- 172.21.15.105:0/2792901982 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc8067570 con 0x7f4bc80b3d30 2023-11-28T18:29:50.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.585+0000 7f4bbf7fe640 1 -- 172.21.15.105:0/2792901982 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f4bb8007070 con 0x7f4bc80b3d30 2023-11-28T18:29:50.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.585+0000 7f4bbf7fe640 1 --2- 172.21.15.105:0/2792901982 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4ba80668d0 0x7f4ba8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:50.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.585+0000 7f4bbf7fe640 1 -- 172.21.15.105:0/2792901982 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f4bb80844d0 con 0x7f4bc80b3d30 2023-11-28T18:29:50.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.585+0000 7f4bbdd9b640 1 --2- 172.21.15.105:0/2792901982 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4ba80668d0 0x7f4ba8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:50.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.585+0000 7f4bbdd9b640 1 --2- 172.21.15.105:0/2792901982 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4ba80668d0 0x7f4ba8068d90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4bc815cef0 tx=0x7f4bb0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:50.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.589+0000 7f4bbf7fe640 1 -- 172.21.15.105:0/2792901982 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4bb8051560 con 0x7f4bc80b3d30 2023-11-28T18:29:50.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.769+0000 7f4bcca32640 1 -- 172.21.15.105:0/2792901982 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4bc80ce270 con 0x7f4bc80b3d30 2023-11-28T18:29:50.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.769+0000 7f4bbf7fe640 1 -- 172.21.15.105:0/2792901982 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4959 (secure 0 0 0) 0x7f4bb8054b80 con 0x7f4bc80b3d30 2023-11-28T18:29:50.775 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:29:50.775 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":94,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/1464337040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1464337040},{"type":"v1","addr":"172.21.15.148:6834","nonce":1464337040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:29:09.707308+0000","modified":"2023-11-28T18:29:43.461020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7689,"mds_1":7665,"mds_2":6485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6485":{"gid":6485,"name":"b","rank":2,"incarnation":91,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6835/3461832163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":3461832163},{"type":"v1","addr":"172.21.15.148:6835","nonce":3461832163}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7665":{"gid":7665,"name":"c","rank":1,"incarnation":89,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6835/4162499249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":4162499249},{"type":"v1","addr":"172.21.15.105:6835","nonce":4162499249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7689":{"gid":7689,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/3239728352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3239728352},{"type":"v1","addr":"172.21.15.105:6837","nonce":3239728352}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-28T18:29:50.775 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 94 2023-11-28T18:29:50.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.773+0000 7f4bcca32640 1 -- 172.21.15.105:0/2792901982 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4ba80668d0 msgr2=0x7f4ba8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:50.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.773+0000 7f4bcca32640 1 --2- 172.21.15.105:0/2792901982 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4ba80668d0 0x7f4ba8068d90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4bc815cef0 tx=0x7f4bb0009040 comp rx=0 tx=0).stop 2023-11-28T18:29:50.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.773+0000 7f4bcca32640 1 -- 172.21.15.105:0/2792901982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc80b3d30 msgr2=0x7f4bc814b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:50.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.773+0000 7f4bcca32640 1 --2- 172.21.15.105:0/2792901982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc80b3d30 0x7f4bc814b3b0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f4bb800cbf0 tx=0x7f4bb800a060 comp rx=0 tx=0).stop 2023-11-28T18:29:50.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.773+0000 7f4bcca32640 1 -- 172.21.15.105:0/2792901982 shutdown_connections 2023-11-28T18:29:50.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.773+0000 7f4bcca32640 1 --2- 172.21.15.105:0/2792901982 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4bc815d4f0 0x7f4bc814e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.773+0000 7f4bcca32640 1 --2- 172.21.15.105:0/2792901982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bc80d3c30 0x7f4bc81560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.773+0000 7f4bcca32640 1 --2- 172.21.15.105:0/2792901982 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4ba80668d0 0x7f4ba8068d90 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.773+0000 7f4bcca32640 1 --2- 172.21.15.105:0/2792901982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc80b3d30 0x7f4bc814b3b0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.773+0000 7f4bcca32640 1 -- 172.21.15.105:0/2792901982 >> 172.21.15.105:0/2792901982 conn(0x7f4bc80d3030 msgr2=0x7f4bc806c510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:50.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.773+0000 7f4bcca32640 1 -- 172.21.15.105:0/2792901982 shutdown_connections 2023-11-28T18:29:50.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.773+0000 7f4bcca32640 1 -- 172.21.15.105:0/2792901982 wait complete. 2023-11-28T18:29:50.796 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:29:50.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79b518d640 1 Processor -- start 2023-11-28T18:29:50.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79b518d640 1 -- start start 2023-11-28T18:29:50.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79b518d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b00cc760 0x7f79b0164d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:50.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79b518d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b00ccc10 con 0x7f79b00caf60 2023-11-28T18:29:50.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79b518d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b006bf50 con 0x7f79b0159f40 2023-11-28T18:29:50.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79b518d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b00cb3c0 con 0x7f79b00cc760 2023-11-28T18:29:50.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79a659c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f79b00caf60 0x7f79b00ad850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40472/0 (socket says 172.21.15.105:40472) 2023-11-28T18:29:50.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79a659c640 1 -- 172.21.15.105:0/3116508146 learned_addr learned my addr 172.21.15.105:0/3116508146 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:50.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79ae59c640 1 -- 172.21.15.105:0/3116508146 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1633395343 0 0) 0x7f79b00ccc10 con 0x7f79b00caf60 2023-11-28T18:29:50.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79af59e640 1 --2- 172.21.15.105:0/3116508146 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b00cc760 0x7f79b0164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:50.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79ae59c640 1 -- 172.21.15.105:0/3116508146 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f799c003530 con 0x7f79b00caf60 2023-11-28T18:29:50.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79ae59c640 1 -- 172.21.15.105:0/3116508146 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 618778744 0 0) 0x7f799c003530 con 0x7f79b00caf60 2023-11-28T18:29:50.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79ae59c640 1 -- 172.21.15.105:0/3116508146 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b00cc760 msgr2=0x7f79b0164d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:50.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79ae59c640 1 --2- 172.21.15.105:0/3116508146 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b00cc760 0x7f79b0164d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79ae59c640 1 -- 172.21.15.105:0/3116508146 >> v1:172.21.15.148:6789/0 conn(0x7f79b0159f40 legacy=0x7f79b00ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:50.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79ae59c640 1 -- 172.21.15.105:0/3116508146 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b015a320 con 0x7f79b00caf60 2023-11-28T18:29:50.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79ae59c640 1 -- 172.21.15.105:0/3116508146 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f7998001a00 con 0x7f79b00caf60 2023-11-28T18:29:50.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79ae59c640 1 -- 172.21.15.105:0/3116508146 >> v1:172.21.15.105:6789/0 conn(0x7f79b00caf60 legacy=0x7f79b00ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:50.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79ae59c640 1 --2- 172.21.15.105:0/3116508146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f799c0039b0 0x7f799c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:50.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79ae59c640 1 --2- 172.21.15.105:0/3116508146 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b00cc760 0x7f799c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:50.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79ae59c640 1 --2- 172.21.15.105:0/3116508146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f799c004d10 0x7f799c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:50.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79ae59c640 1 -- 172.21.15.105:0/3116508146 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b00cb3c0 con 0x7f799c0039b0 2023-11-28T18:29:50.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79ae59c640 1 -- 172.21.15.105:0/3116508146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c00baa0 con 0x7f799c004d10 2023-11-28T18:29:50.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.929+0000 7f79ae59c640 1 -- 172.21.15.105:0/3116508146 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799c00de70 con 0x7f79b00cc760 2023-11-28T18:29:50.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.933+0000 7f79aed9d640 1 --2- 172.21.15.105:0/3116508146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f799c0039b0 0x7f799c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:50.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.933+0000 7f79a659c640 1 --2- 172.21.15.105:0/3116508146 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b00cc760 0x7f799c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:50.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.933+0000 7f79af59e640 1 --2- 172.21.15.105:0/3116508146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f799c004d10 0x7f799c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:50.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.933+0000 7f79aed9d640 1 -- 172.21.15.105:0/3116508146 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b00cc760 msgr2=0x7f799c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:50.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.933+0000 7f79aed9d640 1 --2- 172.21.15.105:0/3116508146 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b00cc760 0x7f799c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.933+0000 7f79aed9d640 1 -- 172.21.15.105:0/3116508146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f799c004d10 msgr2=0x7f799c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:50.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.933+0000 7f79aed9d640 1 --2- 172.21.15.105:0/3116508146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f799c004d10 0x7f799c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.933+0000 7f79aed9d640 1 -- 172.21.15.105:0/3116508146 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c003530 con 0x7f799c0039b0 2023-11-28T18:29:50.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.933+0000 7f79aed9d640 1 --2- 172.21.15.105:0/3116508146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f799c0039b0 0x7f799c003d90 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f79a0001860 tx=0x7f79a001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c64bceec1cf560d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:50.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.933+0000 7f79ae59c640 1 -- 172.21.15.105:0/3116508146 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a00ab600 con 0x7f799c0039b0 2023-11-28T18:29:50.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.933+0000 7f79ae59c640 1 -- 172.21.15.105:0/3116508146 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a00a7050 con 0x7f799c0039b0 2023-11-28T18:29:50.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.933+0000 7f79ae59c640 1 -- 172.21.15.105:0/3116508146 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a00ab9a0 con 0x7f799c0039b0 2023-11-28T18:29:50.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79b518d640 1 -- 172.21.15.105:0/3116508146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f799c0039b0 msgr2=0x7f799c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:50.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79b518d640 1 --2- 172.21.15.105:0/3116508146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f799c0039b0 0x7f799c003d90 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f79a0001860 tx=0x7f79a001c4d0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79af59e640 1 -- 172.21.15.105:0/3116508146 reap_dead start 2023-11-28T18:29:50.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79b518d640 1 -- 172.21.15.105:0/3116508146 shutdown_connections 2023-11-28T18:29:50.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79b518d640 1 -- 172.21.15.105:0/3116508146 >> 172.21.15.105:0/3116508146 conn(0x7f79b00ceb90 msgr2=0x7f79b00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:50.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79b518d640 1 -- 172.21.15.105:0/3116508146 shutdown_connections 2023-11-28T18:29:50.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79b518d640 1 -- 172.21.15.105:0/3116508146 wait complete. 2023-11-28T18:29:50.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79b518d640 1 Processor -- start 2023-11-28T18:29:50.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79b518d640 1 -- start start 2023-11-28T18:29:50.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79b518d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f799c0039b0 0x7f79b0153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:50.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79b518d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b0147200 0x7f79b013c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:50.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79b518d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79b0158c10 0x7f79b0159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:50.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79b518d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b01595d0 con 0x7f79b0158c10 2023-11-28T18:29:50.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79b518d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b01523c0 con 0x7f799c0039b0 2023-11-28T18:29:50.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79b518d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b0151320 con 0x7f79b0147200 2023-11-28T18:29:50.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79af59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79b0158c10 0x7f79b0159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:50.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79af59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79b0158c10 0x7f79b0159090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38498/0 (socket says 172.21.15.105:38498) 2023-11-28T18:29:50.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79af59e640 1 -- 172.21.15.105:0/3157052466 learned_addr learned my addr 172.21.15.105:0/3157052466 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:50.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79aed9d640 1 --2- 172.21.15.105:0/3157052466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f799c0039b0 0x7f79b0153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:50.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79a659c640 1 --2- 172.21.15.105:0/3157052466 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b0147200 0x7f79b013c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:50.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79af59e640 1 -- 172.21.15.105:0/3157052466 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b0147200 msgr2=0x7f79b013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:50.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79af59e640 1 --2- 172.21.15.105:0/3157052466 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b0147200 0x7f79b013c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79af59e640 1 -- 172.21.15.105:0/3157052466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f799c0039b0 msgr2=0x7f79b0153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:50.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.937+0000 7f79af59e640 1 --2- 172.21.15.105:0/3157052466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f799c0039b0 0x7f79b0153040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:50.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.941+0000 7f79af59e640 1 -- 172.21.15.105:0/3157052466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b0151600 con 0x7f79b0158c10 2023-11-28T18:29:50.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.941+0000 7f79af59e640 1 --2- 172.21.15.105:0/3157052466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79b0158c10 0x7f79b0159090 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f79b006fbe0 tx=0x7f79a8011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:50.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.941+0000 7f79a7fff640 1 -- 172.21.15.105:0/3157052466 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a8004240 con 0x7f79b0158c10 2023-11-28T18:29:50.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.941+0000 7f79a7fff640 1 -- 172.21.15.105:0/3157052466 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a80043a0 con 0x7f79b0158c10 2023-11-28T18:29:50.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.941+0000 7f79b518d640 1 -- 172.21.15.105:0/3157052466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b01518e0 con 0x7f79b0158c10 2023-11-28T18:29:50.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.941+0000 7f79a7fff640 1 -- 172.21.15.105:0/3157052466 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a800f9b0 con 0x7f79b0158c10 2023-11-28T18:29:50.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.941+0000 7f79b518d640 1 -- 172.21.15.105:0/3157052466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b0151ea0 con 0x7f79b0158c10 2023-11-28T18:29:50.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.941+0000 7f79b518d640 1 -- 172.21.15.105:0/3157052466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b01520c0 con 0x7f79b0158c10 2023-11-28T18:29:50.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.941+0000 7f79a7fff640 1 -- 172.21.15.105:0/3157052466 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f79a800fb10 con 0x7f79b0158c10 2023-11-28T18:29:50.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.941+0000 7f79a7fff640 1 --2- 172.21.15.105:0/3157052466 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f79900668d0 0x7f7990068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.941+0000 7f79a7fff640 1 -- 172.21.15.105:0/3157052466 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f79a80846a0 con 0x7f79b0158c10 2023-11-28T18:29:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.941+0000 7f79aed9d640 1 --2- 172.21.15.105:0/3157052466 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f79900668d0 0x7f7990068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.945+0000 7f79aed9d640 1 --2- 172.21.15.105:0/3157052466 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f79900668d0 0x7f7990068d90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f79a00ad000 tx=0x7f79a00a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:50.945+0000 7f79a7fff640 1 -- 172.21.15.105:0/3157052466 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f79a8051730 con 0x7f79b0158c10 2023-11-28T18:29:51.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.097+0000 7f79b518d640 1 -- 172.21.15.105:0/3157052466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79b0067570 con 0x7f79b0158c10 2023-11-28T18:29:51.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.097+0000 7f79a7fff640 1 -- 172.21.15.105:0/3157052466 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16971 (secure 0 0 0) 0x7f79a8054d50 con 0x7f79b0158c10 2023-11-28T18:29:51.102 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:29:51.102 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":92,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:29:16.406699+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:29:08.048399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-28T18:29:08.699964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":5}]},{"pgid":"14.17","mappings":[{"from":1,"to":2}]},{"pgid":"14.21","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:29:51.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.097+0000 7f79b518d640 1 -- 172.21.15.105:0/3157052466 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f79900668d0 msgr2=0x7f7990068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:51.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.097+0000 7f79b518d640 1 --2- 172.21.15.105:0/3157052466 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f79900668d0 0x7f7990068d90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f79a00ad000 tx=0x7f79a00a6000 comp rx=0 tx=0).stop 2023-11-28T18:29:51.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.097+0000 7f79b518d640 1 -- 172.21.15.105:0/3157052466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79b0158c10 msgr2=0x7f79b0159090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:51.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.097+0000 7f79b518d640 1 --2- 172.21.15.105:0/3157052466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79b0158c10 0x7f79b0159090 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f79b006fbe0 tx=0x7f79a8011cf0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.097+0000 7f79b518d640 1 -- 172.21.15.105:0/3157052466 shutdown_connections 2023-11-28T18:29:51.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.097+0000 7f79b518d640 1 --2- 172.21.15.105:0/3157052466 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f79900668d0 0x7f7990068d90 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.097+0000 7f79b518d640 1 --2- 172.21.15.105:0/3157052466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79b0158c10 0x7f79b0159090 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.097+0000 7f79b518d640 1 --2- 172.21.15.105:0/3157052466 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79b0147200 0x7f79b013c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.097+0000 7f79b518d640 1 --2- 172.21.15.105:0/3157052466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f799c0039b0 0x7f79b0153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.097+0000 7f79b518d640 1 -- 172.21.15.105:0/3157052466 >> 172.21.15.105:0/3157052466 conn(0x7f79b00ceb90 msgr2=0x7f79b006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:51.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.101+0000 7f79b518d640 1 -- 172.21.15.105:0/3157052466 shutdown_connections 2023-11-28T18:29:51.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.101+0000 7f79b518d640 1 -- 172.21.15.105:0/3157052466 wait complete. 2023-11-28T18:29:51.123 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T18:29:51.124 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:29:51.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa5002d6640 1 Processor -- start 2023-11-28T18:29:51.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa5002d6640 1 -- start start 2023-11-28T18:29:51.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa5002d6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa4f80b3d30 0x7fa4f81680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:51.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa5002d6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f80b25b0 con 0x7fa4f815d4f0 2023-11-28T18:29:51.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa5002d6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f80d1c30 con 0x7fa4f80d3c30 2023-11-28T18:29:51.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa5002d6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f80b2900 con 0x7fa4f80b3d30 2023-11-28T18:29:51.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa4fe072640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa4f815d4f0 0x7fa4f80d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56658/0 (socket says 172.21.15.105:56658) 2023-11-28T18:29:51.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa4fe072640 1 -- 172.21.15.105:0/1652295648 learned_addr learned my addr 172.21.15.105:0/1652295648 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:51.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa4fe873640 1 --2- 172.21.15.105:0/1652295648 >> v2:172.21.15.148:3300/0 conn(0x7fa4f80b3d30 0x7fa4f81680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:51.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa4fd070640 1 -- 172.21.15.105:0/1652295648 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1148450492 0 0) 0x7fa4f80b25b0 con 0x7fa4f815d4f0 2023-11-28T18:29:51.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa4fd070640 1 -- 172.21.15.105:0/1652295648 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4e4003530 con 0x7fa4f815d4f0 2023-11-28T18:29:51.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa4fd070640 1 -- 172.21.15.105:0/1652295648 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2646344871 0 0) 0x7fa4e4003530 con 0x7fa4f815d4f0 2023-11-28T18:29:51.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa4fd070640 1 -- 172.21.15.105:0/1652295648 >> v2:172.21.15.148:3300/0 conn(0x7fa4f80b3d30 msgr2=0x7fa4f81680a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:51.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa4fd070640 1 --2- 172.21.15.105:0/1652295648 >> v2:172.21.15.148:3300/0 conn(0x7fa4f80b3d30 0x7fa4f81680a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa4fd070640 1 -- 172.21.15.105:0/1652295648 >> v1:172.21.15.148:6789/0 conn(0x7fa4f80d3c30 legacy=0x7fa4f80d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:51.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa4fd070640 1 -- 172.21.15.105:0/1652295648 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f80cea30 con 0x7fa4f815d4f0 2023-11-28T18:29:51.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa4fd070640 1 -- 172.21.15.105:0/1652295648 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fa4f4001680 con 0x7fa4f815d4f0 2023-11-28T18:29:51.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa4fd070640 1 -- 172.21.15.105:0/1652295648 >> v1:172.21.15.105:6789/0 conn(0x7fa4f815d4f0 legacy=0x7fa4f80d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:51.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa4fd070640 1 --2- 172.21.15.105:0/1652295648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e40039d0 0x7fa4e4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:51.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa4fd070640 1 --2- 172.21.15.105:0/1652295648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4e40047d0 0x7fa4e4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:51.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa4fd070640 1 --2- 172.21.15.105:0/1652295648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e40052b0 0x7fa4e400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:51.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa4fd070640 1 -- 172.21.15.105:0/1652295648 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f80b2900 con 0x7fa4e40047d0 2023-11-28T18:29:51.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa4fd070640 1 -- 172.21.15.105:0/1652295648 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e400c1b0 con 0x7fa4e40052b0 2023-11-28T18:29:51.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.257+0000 7fa4fd070640 1 -- 172.21.15.105:0/1652295648 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e400c370 con 0x7fa4e40039d0 2023-11-28T18:29:51.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.261+0000 7fa4fd871640 1 --2- 172.21.15.105:0/1652295648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4e40047d0 0x7fa4e4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:51.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.261+0000 7fa4fe072640 1 --2- 172.21.15.105:0/1652295648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e40039d0 0x7fa4e4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.261+0000 7fa4fe873640 1 --2- 172.21.15.105:0/1652295648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e40052b0 0x7fa4e400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.261+0000 7fa4fd871640 1 -- 172.21.15.105:0/1652295648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e40039d0 msgr2=0x7fa4e4003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.261+0000 7fa4fd871640 1 --2- 172.21.15.105:0/1652295648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e40039d0 0x7fa4e4003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.261+0000 7fa4fd871640 1 -- 172.21.15.105:0/1652295648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e40052b0 msgr2=0x7fa4e400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.261+0000 7fa4fd871640 1 --2- 172.21.15.105:0/1652295648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e40052b0 0x7fa4e400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.261+0000 7fa4fd871640 1 -- 172.21.15.105:0/1652295648 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e4003530 con 0x7fa4e40047d0 2023-11-28T18:29:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.261+0000 7fa4fe873640 1 -- 172.21.15.105:0/1652295648 reap_dead start 2023-11-28T18:29:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.261+0000 7fa4fd871640 1 --2- 172.21.15.105:0/1652295648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4e40047d0 0x7fa4e4004c50 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8001780 tx=0x7fa4e801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23bd98e05a6962 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.261+0000 7fa4fd070640 1 -- 172.21.15.105:0/1652295648 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e80052d0 con 0x7fa4e40047d0 2023-11-28T18:29:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.261+0000 7fa4fd070640 1 -- 172.21.15.105:0/1652295648 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e80a7050 con 0x7fa4e40047d0 2023-11-28T18:29:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.261+0000 7fa4fd070640 1 -- 172.21.15.105:0/1652295648 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8003600 con 0x7fa4e40047d0 2023-11-28T18:29:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa5002d6640 1 -- 172.21.15.105:0/1652295648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4e40047d0 msgr2=0x7fa4e4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa5002d6640 1 --2- 172.21.15.105:0/1652295648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4e40047d0 0x7fa4e4004c50 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8001780 tx=0x7fa4e801c3b0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa5002d6640 1 -- 172.21.15.105:0/1652295648 shutdown_connections 2023-11-28T18:29:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa5002d6640 1 --2- 172.21.15.105:0/1652295648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4e40047d0 0x7fa4e4004c50 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa5002d6640 1 -- 172.21.15.105:0/1652295648 >> 172.21.15.105:0/1652295648 conn(0x7fa4f80d3030 msgr2=0x7fa4f80d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa5002d6640 1 -- 172.21.15.105:0/1652295648 shutdown_connections 2023-11-28T18:29:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa5002d6640 1 -- 172.21.15.105:0/1652295648 wait complete. 2023-11-28T18:29:51.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa5002d6640 1 Processor -- start 2023-11-28T18:29:51.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa5002d6640 1 -- start start 2023-11-28T18:29:51.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa5002d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e40047d0 0x7fa4f815bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:51.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa5002d6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f8156160 0x7fa4f8157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:51.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa5002d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4f81565e0 0x7fa4f8158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:51.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa5002d6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f80d0a20 con 0x7fa4f8156160 2023-11-28T18:29:51.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa5002d6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f80ccce0 con 0x7fa4f81565e0 2023-11-28T18:29:51.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa5002d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f8158950 con 0x7fa4e40047d0 2023-11-28T18:29:51.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa4fd871640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f8156160 0x7fa4f8157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:51.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa4fd871640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f8156160 0x7fa4f8157de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44286/0 (socket says 172.21.15.105:44286) 2023-11-28T18:29:51.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa4fd871640 1 -- 172.21.15.105:0/336795584 learned_addr learned my addr 172.21.15.105:0/336795584 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:51.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa4fe072640 1 --2- 172.21.15.105:0/336795584 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e40047d0 0x7fa4f815bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa4fe873640 1 --2- 172.21.15.105:0/336795584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4f81565e0 0x7fa4f8158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa4fd871640 1 -- 172.21.15.105:0/336795584 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e40047d0 msgr2=0x7fa4f815bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa4fd871640 1 --2- 172.21.15.105:0/336795584 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e40047d0 0x7fa4f815bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa4fd871640 1 -- 172.21.15.105:0/336795584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4f81565e0 msgr2=0x7fa4f8158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa4fd871640 1 --2- 172.21.15.105:0/336795584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4f81565e0 0x7fa4f8158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa4fd871640 1 -- 172.21.15.105:0/336795584 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f8158c30 con 0x7fa4f8156160 2023-11-28T18:29:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.265+0000 7fa4fd871640 1 --2- 172.21.15.105:0/336795584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f8156160 0x7fa4f8157de0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8000c10 tx=0x7fa4e4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.269+0000 7fa4eeffd640 1 -- 172.21.15.105:0/336795584 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e80072f0 con 0x7fa4f8156160 2023-11-28T18:29:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.269+0000 7fa4eeffd640 1 -- 172.21.15.105:0/336795584 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e8007450 con 0x7fa4f8156160 2023-11-28T18:29:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.269+0000 7fa4eeffd640 1 -- 172.21.15.105:0/336795584 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e80067e0 con 0x7fa4f8156160 2023-11-28T18:29:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.269+0000 7fa5002d6640 1 -- 172.21.15.105:0/336795584 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f80cb1f0 con 0x7fa4f8156160 2023-11-28T18:29:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.269+0000 7fa5002d6640 1 -- 172.21.15.105:0/336795584 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f8158f10 con 0x7fa4f8156160 2023-11-28T18:29:51.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.269+0000 7fa5002d6640 1 -- 172.21.15.105:0/336795584 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f8067570 con 0x7fa4f8156160 2023-11-28T18:29:51.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.269+0000 7fa4eeffd640 1 -- 172.21.15.105:0/336795584 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa4e8006940 con 0x7fa4f8156160 2023-11-28T18:29:51.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.269+0000 7fa4eeffd640 1 --2- 172.21.15.105:0/336795584 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa4dc0668d0 0x7fa4dc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:51.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.269+0000 7fa4eeffd640 1 -- 172.21.15.105:0/336795584 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fa4e811e6d0 con 0x7fa4f8156160 2023-11-28T18:29:51.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.269+0000 7fa4fe072640 1 --2- 172.21.15.105:0/336795584 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa4dc0668d0 0x7fa4dc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:51.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.269+0000 7fa4fe072640 1 --2- 172.21.15.105:0/336795584 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa4dc0668d0 0x7fa4dc068d90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4001ec0 tx=0x7fa4f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:51.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.273+0000 7fa4eeffd640 1 -- 172.21.15.105:0/336795584 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa4e80eb6b0 con 0x7fa4f8156160 2023-11-28T18:29:51.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.457+0000 7fa5002d6640 1 -- 172.21.15.105:0/336795584 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4f8067570 con 0x7fa4f8156160 2023-11-28T18:29:51.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.457+0000 7fa4eeffd640 1 -- 172.21.15.105:0/336795584 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4959 (secure 0 0 0) 0x7fa4e80eecd0 con 0x7fa4f8156160 2023-11-28T18:29:51.462 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:29:51.462 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":94,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/1464337040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1464337040},{"type":"v1","addr":"172.21.15.148:6834","nonce":1464337040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:29:09.707308+0000","modified":"2023-11-28T18:29:43.461020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7689,"mds_1":7665,"mds_2":6485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6485":{"gid":6485,"name":"b","rank":2,"incarnation":91,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6835/3461832163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":3461832163},{"type":"v1","addr":"172.21.15.148:6835","nonce":3461832163}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7665":{"gid":7665,"name":"c","rank":1,"incarnation":89,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6835/4162499249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":4162499249},{"type":"v1","addr":"172.21.15.105:6835","nonce":4162499249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7689":{"gid":7689,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/3239728352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3239728352},{"type":"v1","addr":"172.21.15.105:6837","nonce":3239728352}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-28T18:29:51.462 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 94 2023-11-28T18:29:51.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.457+0000 7fa5002d6640 1 -- 172.21.15.105:0/336795584 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa4dc0668d0 msgr2=0x7fa4dc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:51.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.457+0000 7fa5002d6640 1 --2- 172.21.15.105:0/336795584 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa4dc0668d0 0x7fa4dc068d90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4001ec0 tx=0x7fa4f4009040 comp rx=0 tx=0).stop 2023-11-28T18:29:51.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.457+0000 7fa5002d6640 1 -- 172.21.15.105:0/336795584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f8156160 msgr2=0x7fa4f8157de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:51.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.457+0000 7fa5002d6640 1 --2- 172.21.15.105:0/336795584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f8156160 0x7fa4f8157de0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8000c10 tx=0x7fa4e4005800 comp rx=0 tx=0).stop 2023-11-28T18:29:51.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.457+0000 7fa5002d6640 1 -- 172.21.15.105:0/336795584 shutdown_connections 2023-11-28T18:29:51.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.457+0000 7fa5002d6640 1 --2- 172.21.15.105:0/336795584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4f81565e0 0x7fa4f8158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.457+0000 7fa5002d6640 1 --2- 172.21.15.105:0/336795584 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa4dc0668d0 0x7fa4dc068d90 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.457+0000 7fa5002d6640 1 --2- 172.21.15.105:0/336795584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f8156160 0x7fa4f8157de0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.457+0000 7fa5002d6640 1 --2- 172.21.15.105:0/336795584 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e40047d0 0x7fa4f815bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.457+0000 7fa5002d6640 1 -- 172.21.15.105:0/336795584 >> 172.21.15.105:0/336795584 conn(0x7fa4f80d3030 msgr2=0x7fa4f806b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:51.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.457+0000 7fa5002d6640 1 -- 172.21.15.105:0/336795584 shutdown_connections 2023-11-28T18:29:51.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.457+0000 7fa5002d6640 1 -- 172.21.15.105:0/336795584 wait complete. 2023-11-28T18:29:51.483 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:29:51.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.617+0000 7fd12e32a640 1 Processor -- start 2023-11-28T18:29:51.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.617+0000 7fd12e32a640 1 -- start start 2023-11-28T18:29:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.617+0000 7fd12e32a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1280b4140 0x7fd1280d3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.617+0000 7fd12e32a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd128160530 0x7fd128160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.617+0000 7fd12e32a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1280d0820 con 0x7fd1280d10e0 2023-11-28T18:29:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.617+0000 7fd12e32a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd128160ec0 con 0x7fd128160530 2023-11-28T18:29:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.617+0000 7fd12e32a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1280d1d20 con 0x7fd1280b4140 2023-11-28T18:29:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.617+0000 7fd127fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd1280d10e0 0x7fd1280d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56672/0 (socket says 172.21.15.105:56672) 2023-11-28T18:29:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.617+0000 7fd127fff640 1 -- 172.21.15.105:0/3398781464 learned_addr learned my addr 172.21.15.105:0/3398781464 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.617+0000 7fd12c8c7640 1 --2- 172.21.15.105:0/3398781464 >> v2:172.21.15.105:3300/0 conn(0x7fd128160530 0x7fd128160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.617+0000 7fd1277fe640 1 --2- 172.21.15.105:0/3398781464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1280b4140 0x7fd1280d3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.617+0000 7fd12c8c7640 1 -- 172.21.15.105:0/3398781464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1280b4140 msgr2=0x7fd1280d3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.617+0000 7fd12c8c7640 1 --2- 172.21.15.105:0/3398781464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1280b4140 0x7fd1280d3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.617+0000 7fd12c8c7640 1 -- 172.21.15.105:0/3398781464 >> v1:172.21.15.105:6789/0 conn(0x7fd1280d10e0 legacy=0x7fd1280d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:51.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.617+0000 7fd126ffd640 1 -- 172.21.15.105:0/3398781464 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1331825299 0 0) 0x7fd1280d0820 con 0x7fd1280d10e0 2023-11-28T18:29:51.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.617+0000 7fd12c8c7640 1 -- 172.21.15.105:0/3398781464 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1280d01d0 con 0x7fd128160530 2023-11-28T18:29:51.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.621+0000 7fd12c8c7640 1 --2- 172.21.15.105:0/3398781464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd128160530 0x7fd128160980 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fd120009d90 tx=0x7fd12001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecddf0c10a5e1fe4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:51.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.621+0000 7fd126ffd640 1 -- 172.21.15.105:0/3398781464 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1200a5930 con 0x7fd128160530 2023-11-28T18:29:51.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.621+0000 7fd126ffd640 1 -- 172.21.15.105:0/3398781464 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1200a5a90 con 0x7fd128160530 2023-11-28T18:29:51.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.621+0000 7fd126ffd640 1 -- 172.21.15.105:0/3398781464 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1200a5e20 con 0x7fd128160530 2023-11-28T18:29:51.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.621+0000 7fd12e32a640 1 -- 172.21.15.105:0/3398781464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd128160530 msgr2=0x7fd128160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:51.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.621+0000 7fd12e32a640 1 --2- 172.21.15.105:0/3398781464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd128160530 0x7fd128160980 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fd120009d90 tx=0x7fd12001c890 comp rx=0 tx=0).stop 2023-11-28T18:29:51.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.621+0000 7fd12e32a640 1 -- 172.21.15.105:0/3398781464 shutdown_connections 2023-11-28T18:29:51.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.621+0000 7fd12e32a640 1 --2- 172.21.15.105:0/3398781464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd128160530 0x7fd128160980 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.621+0000 7fd12e32a640 1 --2- 172.21.15.105:0/3398781464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1280b4140 0x7fd1280d3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.621+0000 7fd12e32a640 1 -- 172.21.15.105:0/3398781464 >> 172.21.15.105:0/3398781464 conn(0x7fd1280d30a0 msgr2=0x7fd1280d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:51.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.621+0000 7fd12e32a640 1 -- 172.21.15.105:0/3398781464 shutdown_connections 2023-11-28T18:29:51.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.621+0000 7fd12e32a640 1 -- 172.21.15.105:0/3398781464 wait complete. 2023-11-28T18:29:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd12e32a640 1 Processor -- start 2023-11-28T18:29:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd12e32a640 1 -- start start 2023-11-28T18:29:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd12e32a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1280b4140 0x7fd12814b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd12e32a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1280d10e0 0x7fd1280ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd12e32a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd128160530 0x7fd12815d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd12e32a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12814beb0 con 0x7fd1280d10e0 2023-11-28T18:29:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd12e32a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12814e590 con 0x7fd128160530 2023-11-28T18:29:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd12e32a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12814e870 con 0x7fd1280b4140 2023-11-28T18:29:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd1277fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1280d10e0 0x7fd1280ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd1277fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1280d10e0 0x7fd1280ca170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44306/0 (socket says 172.21.15.105:44306) 2023-11-28T18:29:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd1277fe640 1 -- 172.21.15.105:0/1428111165 learned_addr learned my addr 172.21.15.105:0/1428111165 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:51.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd127fff640 1 --2- 172.21.15.105:0/1428111165 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1280b4140 0x7fd12814b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:51.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd12c8c7640 1 --2- 172.21.15.105:0/1428111165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd128160530 0x7fd12815d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:51.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd1277fe640 1 -- 172.21.15.105:0/1428111165 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1280b4140 msgr2=0x7fd12814b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:51.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd1277fe640 1 --2- 172.21.15.105:0/1428111165 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1280b4140 0x7fd12814b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd1277fe640 1 -- 172.21.15.105:0/1428111165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd128160530 msgr2=0x7fd12815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:51.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd1277fe640 1 --2- 172.21.15.105:0/1428111165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd128160530 0x7fd12815d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd1277fe640 1 -- 172.21.15.105:0/1428111165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd120009a70 con 0x7fd1280d10e0 2023-11-28T18:29:51.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd127fff640 1 --2- 172.21.15.105:0/1428111165 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1280b4140 0x7fd12814b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:51.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd12c8c7640 1 --2- 172.21.15.105:0/1428111165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd128160530 0x7fd12815d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:51.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd1277fe640 1 --2- 172.21.15.105:0/1428111165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1280d10e0 0x7fd1280ca170 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00d800 tx=0x7fd11c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:51.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd124ff9640 1 -- 172.21.15.105:0/1428111165 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c00b6c0 con 0x7fd1280d10e0 2023-11-28T18:29:51.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd124ff9640 1 -- 172.21.15.105:0/1428111165 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11c00b820 con 0x7fd1280d10e0 2023-11-28T18:29:51.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd12e32a640 1 -- 172.21.15.105:0/1428111165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12814eb50 con 0x7fd1280d10e0 2023-11-28T18:29:51.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd124ff9640 1 -- 172.21.15.105:0/1428111165 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c00bb90 con 0x7fd1280d10e0 2023-11-28T18:29:51.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.625+0000 7fd12e32a640 1 -- 172.21.15.105:0/1428111165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12814f110 con 0x7fd1280d10e0 2023-11-28T18:29:51.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.629+0000 7fd12e32a640 1 -- 172.21.15.105:0/1428111165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1280745f0 con 0x7fd1280d10e0 2023-11-28T18:29:51.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.629+0000 7fd124ff9640 1 -- 172.21.15.105:0/1428111165 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd11c00bcf0 con 0x7fd1280d10e0 2023-11-28T18:29:51.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.629+0000 7fd124ff9640 1 --2- 172.21.15.105:0/1428111165 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd1140668d0 0x7fd114068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:51.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.629+0000 7fd124ff9640 1 -- 172.21.15.105:0/1428111165 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fd11c084c30 con 0x7fd1280d10e0 2023-11-28T18:29:51.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.629+0000 7fd127fff640 1 --2- 172.21.15.105:0/1428111165 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd1140668d0 0x7fd114068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:51.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.629+0000 7fd127fff640 1 --2- 172.21.15.105:0/1428111165 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd1140668d0 0x7fd114068d90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd118000f70 tx=0x7fd118007c40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:51.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.633+0000 7fd124ff9640 1 -- 172.21.15.105:0/1428111165 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd11c051c10 con 0x7fd1280d10e0 2023-11-28T18:29:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.813+0000 7fd12e32a640 1 -- 172.21.15.105:0/1428111165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd128159700 con 0x7fd1280d10e0 2023-11-28T18:29:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.813+0000 7fd124ff9640 1 -- 172.21.15.105:0/1428111165 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4959 (secure 0 0 0) 0x7fd11c011340 con 0x7fd1280d10e0 2023-11-28T18:29:51.818 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:29:51.818 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":94,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/1464337040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1464337040},{"type":"v1","addr":"172.21.15.148:6834","nonce":1464337040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:29:09.707308+0000","modified":"2023-11-28T18:29:43.461020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7689,"mds_1":7665,"mds_2":6485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6485":{"gid":6485,"name":"b","rank":2,"incarnation":91,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6835/3461832163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":3461832163},{"type":"v1","addr":"172.21.15.148:6835","nonce":3461832163}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7665":{"gid":7665,"name":"c","rank":1,"incarnation":89,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6835/4162499249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":4162499249},{"type":"v1","addr":"172.21.15.105:6835","nonce":4162499249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7689":{"gid":7689,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/3239728352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3239728352},{"type":"v1","addr":"172.21.15.105:6837","nonce":3239728352}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-28T18:29:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 94 2023-11-28T18:29:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.813+0000 7fd12e32a640 1 -- 172.21.15.105:0/1428111165 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd1140668d0 msgr2=0x7fd114068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.813+0000 7fd12e32a640 1 --2- 172.21.15.105:0/1428111165 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd1140668d0 0x7fd114068d90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd118000f70 tx=0x7fd118007c40 comp rx=0 tx=0).stop 2023-11-28T18:29:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.813+0000 7fd12e32a640 1 -- 172.21.15.105:0/1428111165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1280d10e0 msgr2=0x7fd1280ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.813+0000 7fd12e32a640 1 --2- 172.21.15.105:0/1428111165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1280d10e0 0x7fd1280ca170 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00d800 tx=0x7fd11c00dcc0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.813+0000 7fd12e32a640 1 -- 172.21.15.105:0/1428111165 shutdown_connections 2023-11-28T18:29:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.813+0000 7fd12e32a640 1 --2- 172.21.15.105:0/1428111165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd128160530 0x7fd12815d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.813+0000 7fd12e32a640 1 --2- 172.21.15.105:0/1428111165 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd1140668d0 0x7fd114068d90 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.813+0000 7fd12e32a640 1 --2- 172.21.15.105:0/1428111165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1280d10e0 0x7fd1280ca170 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.813+0000 7fd12e32a640 1 --2- 172.21.15.105:0/1428111165 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1280b4140 0x7fd12814b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.813+0000 7fd12e32a640 1 -- 172.21.15.105:0/1428111165 >> 172.21.15.105:0/1428111165 conn(0x7fd1280d30a0 msgr2=0x7fd12806b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:51.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.817+0000 7fd12e32a640 1 -- 172.21.15.105:0/1428111165 shutdown_connections 2023-11-28T18:29:51.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.817+0000 7fd12e32a640 1 -- 172.21.15.105:0/1428111165 wait complete. 2023-11-28T18:29:51.839 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:29:51.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.973+0000 7f9509879640 1 Processor -- start 2023-11-28T18:29:51.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.973+0000 7f9509879640 1 -- start start 2023-11-28T18:29:51.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.973+0000 7f9509879640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f95040d3c30 0x7f95040d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:51.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.973+0000 7f9509879640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95040b25b0 con 0x7f95040b2d20 2023-11-28T18:29:51.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.973+0000 7f9509879640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95040cea30 con 0x7f950415d4f0 2023-11-28T18:29:51.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.973+0000 7f9509879640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95040b2900 con 0x7f95040d3c30 2023-11-28T18:29:51.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.973+0000 7f95037fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f95040b2d20 0x7f95040b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56680/0 (socket says 172.21.15.105:56680) 2023-11-28T18:29:51.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.973+0000 7f95037fe640 1 -- 172.21.15.105:0/697305237 learned_addr learned my addr 172.21.15.105:0/697305237 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:51.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.973+0000 7f94fa7fc640 1 --2- 172.21.15.105:0/697305237 >> v2:172.21.15.148:3300/0 conn(0x7f95040d3c30 0x7f95040d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:51.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.973+0000 7f95027fc640 1 -- 172.21.15.105:0/697305237 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983638377 0 0) 0x7f95040b25b0 con 0x7f95040b2d20 2023-11-28T18:29:51.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.973+0000 7f95027fc640 1 -- 172.21.15.105:0/697305237 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94f0003530 con 0x7f95040b2d20 2023-11-28T18:29:51.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.973+0000 7f94fa7fc640 1 -- 172.21.15.105:0/697305237 >> v1:172.21.15.148:6789/0 conn(0x7f950415d4f0 legacy=0x7f95040d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:51.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.973+0000 7f94fa7fc640 1 -- 172.21.15.105:0/697305237 >> v1:172.21.15.105:6789/0 conn(0x7f95040b2d20 legacy=0x7f95040b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:51.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.973+0000 7f95027fc640 1 -- 172.21.15.105:0/697305237 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2097874382 0 0) 0x7f94f0003530 con 0x7f95040b2d20 2023-11-28T18:29:51.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.973+0000 7f94fa7fc640 1 -- 172.21.15.105:0/697305237 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95040ced10 con 0x7f95040d3c30 2023-11-28T18:29:51.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.977+0000 7f94fa7fc640 1 --2- 172.21.15.105:0/697305237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95040d3c30 0x7f95040d1520 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f94f40097c0 tx=0x7f94f401c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32e551628eefa57a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.977+0000 7f95027fc640 1 -- 172.21.15.105:0/697305237 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc001a00 con 0x7f95040d3c30 2023-11-28T18:29:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.977+0000 7f95027fc640 1 -- 172.21.15.105:0/697305237 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f40a7050 con 0x7f95040d3c30 2023-11-28T18:29:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.977+0000 7f95027fc640 1 -- 172.21.15.105:0/697305237 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f40ab600 con 0x7f95040d3c30 2023-11-28T18:29:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.977+0000 7f9509879640 1 -- 172.21.15.105:0/697305237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95040d3c30 msgr2=0x7f95040d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.977+0000 7f9509879640 1 --2- 172.21.15.105:0/697305237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95040d3c30 0x7f95040d1520 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f94f40097c0 tx=0x7f94f401c2e0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.977+0000 7f9509879640 1 -- 172.21.15.105:0/697305237 shutdown_connections 2023-11-28T18:29:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.977+0000 7f9509879640 1 --2- 172.21.15.105:0/697305237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95040d3c30 0x7f95040d1520 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.977+0000 7f9509879640 1 -- 172.21.15.105:0/697305237 >> 172.21.15.105:0/697305237 conn(0x7f95040d3030 msgr2=0x7f95040d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.977+0000 7f9509879640 1 -- 172.21.15.105:0/697305237 shutdown_connections 2023-11-28T18:29:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.977+0000 7f9509879640 1 -- 172.21.15.105:0/697305237 wait complete. 2023-11-28T18:29:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f9509879640 1 Processor -- start 2023-11-28T18:29:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f9509879640 1 -- start start 2023-11-28T18:29:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f9509879640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95040b2d20 0x7f9504153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:51.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f9509879640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95040d3c30 0x7f950415db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:51.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f9509879640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f950415d4f0 0x7f950415f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:51.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f9509879640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95040d0a20 con 0x7f95040d3c30 2023-11-28T18:29:51.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f9509879640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950415f400 con 0x7f95040b2d20 2023-11-28T18:29:51.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f9509879640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f950415e520 con 0x7f950415d4f0 2023-11-28T18:29:51.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f94fa7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95040d3c30 0x7f950415db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:51.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f94fa7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95040d3c30 0x7f950415db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44308/0 (socket says 172.21.15.105:44308) 2023-11-28T18:29:51.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f94fa7fc640 1 -- 172.21.15.105:0/3732406193 learned_addr learned my addr 172.21.15.105:0/3732406193 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:51.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f9502ffd640 1 --2- 172.21.15.105:0/3732406193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95040b2d20 0x7f9504153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:51.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f95037fe640 1 --2- 172.21.15.105:0/3732406193 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f950415d4f0 0x7f950415f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:51.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f94fa7fc640 1 -- 172.21.15.105:0/3732406193 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f950415d4f0 msgr2=0x7f950415f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:51.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f94fa7fc640 1 --2- 172.21.15.105:0/3732406193 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f950415d4f0 0x7f950415f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f94fa7fc640 1 -- 172.21.15.105:0/3732406193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95040b2d20 msgr2=0x7f9504153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:51.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f94fa7fc640 1 --2- 172.21.15.105:0/3732406193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95040b2d20 0x7f9504153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:51.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f94fa7fc640 1 -- 172.21.15.105:0/3732406193 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f4009420 con 0x7f95040d3c30 2023-11-28T18:29:51.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f94fa7fc640 1 --2- 172.21.15.105:0/3732406193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95040d3c30 0x7f950415db60 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f94f4018ef0 tx=0x7f94f4019080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:51.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f9500ff9640 1 -- 172.21.15.105:0/3732406193 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f4006e20 con 0x7f95040d3c30 2023-11-28T18:29:51.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f9500ff9640 1 -- 172.21.15.105:0/3732406193 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f40a4e90 con 0x7f95040d3c30 2023-11-28T18:29:51.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f9500ff9640 1 -- 172.21.15.105:0/3732406193 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f40b4600 con 0x7f95040d3c30 2023-11-28T18:29:51.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.981+0000 7f9509879640 1 -- 172.21.15.105:0/3732406193 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f950415e800 con 0x7f95040d3c30 2023-11-28T18:29:51.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.985+0000 7f9509879640 1 -- 172.21.15.105:0/3732406193 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f950415f020 con 0x7f95040d3c30 2023-11-28T18:29:51.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.985+0000 7f9509879640 1 -- 172.21.15.105:0/3732406193 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9504067570 con 0x7f95040d3c30 2023-11-28T18:29:51.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.985+0000 7f9500ff9640 1 -- 172.21.15.105:0/3732406193 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f94f40b2040 con 0x7f95040d3c30 2023-11-28T18:29:51.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.985+0000 7f9500ff9640 1 --2- 172.21.15.105:0/3732406193 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f94e40668d0 0x7f94e4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:51.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.985+0000 7f9500ff9640 1 -- 172.21.15.105:0/3732406193 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f94f411e360 con 0x7f95040d3c30 2023-11-28T18:29:51.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.985+0000 7f9502ffd640 1 --2- 172.21.15.105:0/3732406193 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f94e40668d0 0x7f94e4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:51.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.985+0000 7f9502ffd640 1 --2- 172.21.15.105:0/3732406193 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f94e40668d0 0x7f94e4068d90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f94ec007fd0 tx=0x7f94ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:51.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:51.989+0000 7f9500ff9640 1 -- 172.21.15.105:0/3732406193 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f94f40eb3d0 con 0x7f95040d3c30 2023-11-28T18:29:52.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.137+0000 7f9509879640 1 -- 172.21.15.105:0/3732406193 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f950406b8b0 con 0x7f95040d3c30 2023-11-28T18:29:52.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.141+0000 7f9500ff9640 1 -- 172.21.15.105:0/3732406193 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16971 (secure 0 0 0) 0x7f94f40ee9f0 con 0x7f95040d3c30 2023-11-28T18:29:52.145 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:29:52.145 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":92,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:29:16.406699+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:29:08.048399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-28T18:29:08.699964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":5}]},{"pgid":"14.17","mappings":[{"from":1,"to":2}]},{"pgid":"14.21","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:29:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.141+0000 7f9509879640 1 -- 172.21.15.105:0/3732406193 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f94e40668d0 msgr2=0x7f94e4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.141+0000 7f9509879640 1 --2- 172.21.15.105:0/3732406193 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f94e40668d0 0x7f94e4068d90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f94ec007fd0 tx=0x7f94ec009040 comp rx=0 tx=0).stop 2023-11-28T18:29:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.141+0000 7f9509879640 1 -- 172.21.15.105:0/3732406193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95040d3c30 msgr2=0x7f950415db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.141+0000 7f9509879640 1 --2- 172.21.15.105:0/3732406193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95040d3c30 0x7f950415db60 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f94f4018ef0 tx=0x7f94f4019080 comp rx=0 tx=0).stop 2023-11-28T18:29:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.141+0000 7f9509879640 1 -- 172.21.15.105:0/3732406193 shutdown_connections 2023-11-28T18:29:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.141+0000 7f9509879640 1 --2- 172.21.15.105:0/3732406193 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f950415d4f0 0x7f950415f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.141+0000 7f9509879640 1 --2- 172.21.15.105:0/3732406193 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f94e40668d0 0x7f94e4068d90 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.141+0000 7f9509879640 1 --2- 172.21.15.105:0/3732406193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95040d3c30 0x7f950415db60 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.141+0000 7f9509879640 1 --2- 172.21.15.105:0/3732406193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95040b2d20 0x7f9504153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:52.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.141+0000 7f9509879640 1 -- 172.21.15.105:0/3732406193 >> 172.21.15.105:0/3732406193 conn(0x7f95040d3030 msgr2=0x7f95040c9940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:52.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.141+0000 7f9509879640 1 -- 172.21.15.105:0/3732406193 shutdown_connections 2023-11-28T18:29:52.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.141+0000 7f9509879640 1 -- 172.21.15.105:0/3732406193 wait complete. 2023-11-28T18:29:52.166 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T18:29:52.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.301+0000 7f753c217640 1 Processor -- start 2023-11-28T18:29:52.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.301+0000 7f753c217640 1 -- start start 2023-11-28T18:29:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.301+0000 7f753c217640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f75340ad9f0 0x7f75340ade80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.301+0000 7f753c217640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7534162820 con 0x7f753406d610 2023-11-28T18:29:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.301+0000 7f753c217640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7534161c10 con 0x7f75340cc7d0 2023-11-28T18:29:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.301+0000 7f753c217640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75340ae3c0 con 0x7f75340ad9f0 2023-11-28T18:29:52.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f7539fb3640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f753406d610 0x7f75340cdff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56688/0 (socket says 172.21.15.105:56688) 2023-11-28T18:29:52.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f7539fb3640 1 -- 172.21.15.105:0/2120689075 learned_addr learned my addr 172.21.15.105:0/2120689075 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:52.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f75397b2640 1 --2- 172.21.15.105:0/2120689075 >> v2:172.21.15.148:3300/0 conn(0x7f75340ad9f0 0x7f75340ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:52.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f7538fb1640 1 -- 172.21.15.105:0/2120689075 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2748001947 0 0) 0x7f7534162820 con 0x7f753406d610 2023-11-28T18:29:52.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f7538fb1640 1 -- 172.21.15.105:0/2120689075 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7520003530 con 0x7f753406d610 2023-11-28T18:29:52.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f7538fb1640 1 -- 172.21.15.105:0/2120689075 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 614206932 0 0) 0x7f7520003530 con 0x7f753406d610 2023-11-28T18:29:52.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f7538fb1640 1 -- 172.21.15.105:0/2120689075 >> v2:172.21.15.148:3300/0 conn(0x7f75340ad9f0 msgr2=0x7f75340ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:52.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f7538fb1640 1 --2- 172.21.15.105:0/2120689075 >> v2:172.21.15.148:3300/0 conn(0x7f75340ad9f0 0x7f75340ade80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:52.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f7538fb1640 1 -- 172.21.15.105:0/2120689075 >> v1:172.21.15.148:6789/0 conn(0x7f75340cc7d0 legacy=0x7f75340ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:52.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f7538fb1640 1 -- 172.21.15.105:0/2120689075 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7534161ef0 con 0x7f753406d610 2023-11-28T18:29:52.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f7538fb1640 1 -- 172.21.15.105:0/2120689075 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f7530001680 con 0x7f753406d610 2023-11-28T18:29:52.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f7538fb1640 1 -- 172.21.15.105:0/2120689075 >> v1:172.21.15.105:6789/0 conn(0x7f753406d610 legacy=0x7f75340cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:52.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f7538fb1640 1 --2- 172.21.15.105:0/2120689075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7520003980 0x7f7520003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:52.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f7538fb1640 1 --2- 172.21.15.105:0/2120689075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75200047d0 0x7f7520004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:52.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f7538fb1640 1 --2- 172.21.15.105:0/2120689075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7520005190 0x7f752000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:52.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f7538fb1640 1 -- 172.21.15.105:0/2120689075 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75340ae3c0 con 0x7f7520005190 2023-11-28T18:29:52.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f7538fb1640 1 -- 172.21.15.105:0/2120689075 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752000c030 con 0x7f7520003980 2023-11-28T18:29:52.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f7538fb1640 1 -- 172.21.15.105:0/2120689075 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752000c1f0 con 0x7f75200047d0 2023-11-28T18:29:52.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f753a7b4640 1 --2- 172.21.15.105:0/2120689075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7520005190 0x7f752000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:52.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f7539fb3640 1 --2- 172.21.15.105:0/2120689075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7520003980 0x7f7520003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:52.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f75397b2640 1 --2- 172.21.15.105:0/2120689075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75200047d0 0x7f7520004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:52.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f753a7b4640 1 -- 172.21.15.105:0/2120689075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75200047d0 msgr2=0x7f7520004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:52.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f753a7b4640 1 --2- 172.21.15.105:0/2120689075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75200047d0 0x7f7520004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:52.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f753a7b4640 1 -- 172.21.15.105:0/2120689075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7520003980 msgr2=0x7f7520003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:52.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f753a7b4640 1 --2- 172.21.15.105:0/2120689075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7520003980 0x7f7520003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:52.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.305+0000 7f753a7b4640 1 -- 172.21.15.105:0/2120689075 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7520003530 con 0x7f7520005190 2023-11-28T18:29:52.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.309+0000 7f753a7b4640 1 -- 172.21.15.105:0/2120689075 reap_dead start 2023-11-28T18:29:52.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.309+0000 7f753a7b4640 1 --2- 172.21.15.105:0/2120689075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7520005190 0x7f752000b990 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f752c001e70 tx=0x7f752c01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae263731e55371e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:52.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.309+0000 7f7538fb1640 1 -- 172.21.15.105:0/2120689075 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c003b80 con 0x7f7520005190 2023-11-28T18:29:52.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.309+0000 7f7538fb1640 1 -- 172.21.15.105:0/2120689075 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752c003d00 con 0x7f7520005190 2023-11-28T18:29:52.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.309+0000 7f7538fb1640 1 -- 172.21.15.105:0/2120689075 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c005260 con 0x7f7520005190 2023-11-28T18:29:52.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.309+0000 7f753c217640 1 -- 172.21.15.105:0/2120689075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7520005190 msgr2=0x7f752000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:52.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.309+0000 7f753c217640 1 --2- 172.21.15.105:0/2120689075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7520005190 0x7f752000b990 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f752c001e70 tx=0x7f752c01c760 comp rx=0 tx=0).stop 2023-11-28T18:29:52.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.309+0000 7f753c217640 1 -- 172.21.15.105:0/2120689075 shutdown_connections 2023-11-28T18:29:52.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.309+0000 7f753c217640 1 --2- 172.21.15.105:0/2120689075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7520005190 0x7f752000b990 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:52.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.309+0000 7f753c217640 1 -- 172.21.15.105:0/2120689075 >> 172.21.15.105:0/2120689075 conn(0x7f75340ceb90 msgr2=0x7f75340cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:52.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.309+0000 7f753c217640 1 -- 172.21.15.105:0/2120689075 shutdown_connections 2023-11-28T18:29:52.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.309+0000 7f753c217640 1 -- 172.21.15.105:0/2120689075 wait complete. 2023-11-28T18:29:52.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.309+0000 7f753c217640 1 Processor -- start 2023-11-28T18:29:52.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.309+0000 7f753c217640 1 -- start start 2023-11-28T18:29:52.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f753c217640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7520005190 0x7f753409a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:52.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f753c217640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7534099d80 0x7f75340b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:52.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f753c217640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75340af880 0x7f75340afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:52.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f753c217640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75340b0270 con 0x7f7534099d80 2023-11-28T18:29:52.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f753c217640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7534162b00 con 0x7f75340af880 2023-11-28T18:29:52.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f753c217640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7534162de0 con 0x7f7520005190 2023-11-28T18:29:52.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f75397b2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7534099d80 0x7f75340b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:52.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f75397b2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7534099d80 0x7f75340b3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44334/0 (socket says 172.21.15.105:44334) 2023-11-28T18:29:52.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f75397b2640 1 -- 172.21.15.105:0/1077230912 learned_addr learned my addr 172.21.15.105:0/1077230912 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:52.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f753a7b4640 1 --2- 172.21.15.105:0/1077230912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75340af880 0x7f75340afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:52.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f7539fb3640 1 --2- 172.21.15.105:0/1077230912 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7520005190 0x7f753409a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:52.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f75397b2640 1 -- 172.21.15.105:0/1077230912 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7520005190 msgr2=0x7f753409a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:52.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f75397b2640 1 --2- 172.21.15.105:0/1077230912 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7520005190 0x7f753409a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:52.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f75397b2640 1 -- 172.21.15.105:0/1077230912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75340af880 msgr2=0x7f75340afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:52.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f75397b2640 1 --2- 172.21.15.105:0/1077230912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75340af880 0x7f75340afd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:52.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f75397b2640 1 -- 172.21.15.105:0/1077230912 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75341630c0 con 0x7f7534099d80 2023-11-28T18:29:52.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f7539fb3640 1 --2- 172.21.15.105:0/1077230912 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7520005190 0x7f753409a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:52.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f75397b2640 1 --2- 172.21.15.105:0/1077230912 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7534099d80 0x7f75340b3ed0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f7524011960 tx=0x7f7524011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:52.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f752a7fc640 1 -- 172.21.15.105:0/1077230912 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752400faf0 con 0x7f7534099d80 2023-11-28T18:29:52.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f752a7fc640 1 -- 172.21.15.105:0/1077230912 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752400fc50 con 0x7f7534099d80 2023-11-28T18:29:52.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f752a7fc640 1 -- 172.21.15.105:0/1077230912 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7524014440 con 0x7f7534099d80 2023-11-28T18:29:52.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f753c217640 1 -- 172.21.15.105:0/1077230912 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75341633a0 con 0x7f7534099d80 2023-11-28T18:29:52.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f753c217640 1 -- 172.21.15.105:0/1077230912 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753409a980 con 0x7f7534099d80 2023-11-28T18:29:52.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f753c217640 1 -- 172.21.15.105:0/1077230912 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f753409cf30 con 0x7f7534099d80 2023-11-28T18:29:52.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.313+0000 7f752a7fc640 1 -- 172.21.15.105:0/1077230912 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f75240145a0 con 0x7f7534099d80 2023-11-28T18:29:52.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.317+0000 7f752a7fc640 1 --2- 172.21.15.105:0/1077230912 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7518066900 0x7f7518068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:52.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.317+0000 7f752a7fc640 1 -- 172.21.15.105:0/1077230912 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f7524084c50 con 0x7f7534099d80 2023-11-28T18:29:52.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.317+0000 7f7539fb3640 1 --2- 172.21.15.105:0/1077230912 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7518066900 0x7f7518068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:52.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.317+0000 7f7539fb3640 1 --2- 172.21.15.105:0/1077230912 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7518066900 0x7f7518068dc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f75300017b0 tx=0x7f7530009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:52.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.321+0000 7f752a7fc640 1 -- 172.21.15.105:0/1077230912 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f752408a020 con 0x7f7534099d80 2023-11-28T18:29:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.501+0000 7f753c217640 1 -- 172.21.15.105:0/1077230912 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7534067570 con 0x7f7534099d80 2023-11-28T18:29:52.505 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-11-28T18:29:52.501+0000 7fd89b32a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T18:29:52.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.505+0000 7f752a7fc640 1 -- 172.21.15.105:0/1077230912 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v95) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7524051d60 con 0x7f7534099d80 2023-11-28T18:29:52.508 INFO:teuthology.orchestra.run.smithi105.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T18:29:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.505+0000 7f753c217640 1 -- 172.21.15.105:0/1077230912 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7518066900 msgr2=0x7f7518068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.505+0000 7f753c217640 1 --2- 172.21.15.105:0/1077230912 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7518066900 0x7f7518068dc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f75300017b0 tx=0x7f7530009040 comp rx=0 tx=0).stop 2023-11-28T18:29:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.505+0000 7f753c217640 1 -- 172.21.15.105:0/1077230912 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7534099d80 msgr2=0x7f75340b3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.505+0000 7f753c217640 1 --2- 172.21.15.105:0/1077230912 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7534099d80 0x7f75340b3ed0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f7524011960 tx=0x7f7524011e20 comp rx=0 tx=0).stop 2023-11-28T18:29:52.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.505+0000 7f753c217640 1 -- 172.21.15.105:0/1077230912 shutdown_connections 2023-11-28T18:29:52.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.505+0000 7f753c217640 1 --2- 172.21.15.105:0/1077230912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75340af880 0x7f75340afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:52.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.505+0000 7f753c217640 1 --2- 172.21.15.105:0/1077230912 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7518066900 0x7f7518068dc0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:52.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.505+0000 7f753c217640 1 --2- 172.21.15.105:0/1077230912 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7534099d80 0x7f75340b3ed0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:52.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.505+0000 7f753c217640 1 --2- 172.21.15.105:0/1077230912 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7520005190 0x7f753409a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:52.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.505+0000 7f753c217640 1 -- 172.21.15.105:0/1077230912 >> 172.21.15.105:0/1077230912 conn(0x7f75340ceb90 msgr2=0x7f75340cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:52.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.509+0000 7f753c217640 1 -- 172.21.15.105:0/1077230912 shutdown_connections 2023-11-28T18:29:52.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.509+0000 7f753c217640 1 -- 172.21.15.105:0/1077230912 wait complete. 2023-11-28T18:29:52.536 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T18:29:52.571 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2023-11-28T18:29:52.578 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2023-11-28T18:29:52.581 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2023-11-28T18:29:52.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.677+0000 7f8b74bdf640 1 Processor -- start 2023-11-28T18:29:52.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.677+0000 7f8b74bdf640 1 -- start start 2023-11-28T18:29:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b74bdf640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b700cc760 0x7f8b70164d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:52.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b74bdf640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b700ccc10 con 0x7f8b700caf60 2023-11-28T18:29:52.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b74bdf640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7006bf50 con 0x7f8b70159f40 2023-11-28T18:29:52.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b74bdf640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700cb3c0 con 0x7f8b700cc760 2023-11-28T18:29:52.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b65d9b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8b700caf60 0x7f8b700ad850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56692/0 (socket says 172.21.15.105:56692) 2023-11-28T18:29:52.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b65d9b640 1 -- 172.21.15.105:0/360418556 learned_addr learned my addr 172.21.15.105:0/360418556 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6ed9d640 1 --2- 172.21.15.105:0/360418556 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b700cc760 0x7f8b70164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6dd9b640 1 -- 172.21.15.105:0/360418556 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2579606374 0 0) 0x7f8b700ccc10 con 0x7f8b700caf60 2023-11-28T18:29:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6dd9b640 1 -- 172.21.15.105:0/360418556 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b5c005740 con 0x7f8b700caf60 2023-11-28T18:29:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6dd9b640 1 -- 172.21.15.105:0/360418556 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2024523141 0 0) 0x7f8b7006bf50 con 0x7f8b70159f40 2023-11-28T18:29:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6dd9b640 1 -- 172.21.15.105:0/360418556 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b700ccc10 con 0x7f8b70159f40 2023-11-28T18:29:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6dd9b640 1 -- 172.21.15.105:0/360418556 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3884660921 0 0) 0x7f8b5c005740 con 0x7f8b700caf60 2023-11-28T18:29:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6dd9b640 1 -- 172.21.15.105:0/360418556 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b700cc760 msgr2=0x7f8b70164d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6dd9b640 1 --2- 172.21.15.105:0/360418556 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b700cc760 0x7f8b70164d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6dd9b640 1 -- 172.21.15.105:0/360418556 >> v1:172.21.15.148:6789/0 conn(0x7f8b70159f40 legacy=0x7f8b700ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6dd9b640 1 -- 172.21.15.105:0/360418556 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7015a320 con 0x7f8b700caf60 2023-11-28T18:29:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6dd9b640 1 -- 172.21.15.105:0/360418556 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f8b58001680 con 0x7f8b700caf60 2023-11-28T18:29:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6dd9b640 1 -- 172.21.15.105:0/360418556 >> v1:172.21.15.105:6789/0 conn(0x7f8b700caf60 legacy=0x7f8b700ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6dd9b640 1 --2- 172.21.15.105:0/360418556 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b700cc760 0x7f8b5c005d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6dd9b640 1 --2- 172.21.15.105:0/360418556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b5c006300 0x7f8b5c0069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6dd9b640 1 --2- 172.21.15.105:0/360418556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b5c006f20 0x7f8b5c0073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6dd9b640 1 -- 172.21.15.105:0/360418556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700cb3c0 con 0x7f8b5c006f20 2023-11-28T18:29:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6dd9b640 1 -- 172.21.15.105:0/360418556 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c00dd60 con 0x7f8b5c006300 2023-11-28T18:29:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6dd9b640 1 -- 172.21.15.105:0/360418556 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c00df20 con 0x7f8b700cc760 2023-11-28T18:29:52.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6ed9d640 1 --2- 172.21.15.105:0/360418556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b5c006f20 0x7f8b5c0073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:52.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6e59c640 1 --2- 172.21.15.105:0/360418556 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b700cc760 0x7f8b5c005d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:52.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b65d9b640 1 --2- 172.21.15.105:0/360418556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b5c006300 0x7f8b5c0069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:52.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6ed9d640 1 -- 172.21.15.105:0/360418556 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b700cc760 msgr2=0x7f8b5c005d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:52.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6ed9d640 1 --2- 172.21.15.105:0/360418556 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b700cc760 0x7f8b5c005d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:52.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6ed9d640 1 -- 172.21.15.105:0/360418556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b5c006300 msgr2=0x7f8b5c0069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:52.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6ed9d640 1 --2- 172.21.15.105:0/360418556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b5c006300 0x7f8b5c0069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:52.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.681+0000 7f8b6ed9d640 1 -- 172.21.15.105:0/360418556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5c005740 con 0x7f8b5c006f20 2023-11-28T18:29:52.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.685+0000 7f8b6ed9d640 1 --2- 172.21.15.105:0/360418556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b5c006f20 0x7f8b5c0073a0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009fc0 tx=0x7f8b6801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f90a8d406535a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:52.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.685+0000 7f8b6dd9b640 1 -- 172.21.15.105:0/360418556 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b680a5c70 con 0x7f8b5c006f20 2023-11-28T18:29:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.685+0000 7f8b6dd9b640 1 -- 172.21.15.105:0/360418556 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6800f040 con 0x7f8b5c006f20 2023-11-28T18:29:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.685+0000 7f8b6dd9b640 1 -- 172.21.15.105:0/360418556 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b680ad440 con 0x7f8b5c006f20 2023-11-28T18:29:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.685+0000 7f8b74bdf640 1 -- 172.21.15.105:0/360418556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b5c006f20 msgr2=0x7f8b5c0073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.685+0000 7f8b74bdf640 1 --2- 172.21.15.105:0/360418556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b5c006f20 0x7f8b5c0073a0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009fc0 tx=0x7f8b6801cc60 comp rx=0 tx=0).stop 2023-11-28T18:29:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.685+0000 7f8b6ed9d640 1 -- 172.21.15.105:0/360418556 reap_dead start 2023-11-28T18:29:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.685+0000 7f8b74bdf640 1 -- 172.21.15.105:0/360418556 shutdown_connections 2023-11-28T18:29:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.685+0000 7f8b74bdf640 1 -- 172.21.15.105:0/360418556 >> 172.21.15.105:0/360418556 conn(0x7f8b700ceb90 msgr2=0x7f8b700cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.685+0000 7f8b74bdf640 1 -- 172.21.15.105:0/360418556 shutdown_connections 2023-11-28T18:29:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.685+0000 7f8b74bdf640 1 -- 172.21.15.105:0/360418556 wait complete. 2023-11-28T18:29:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b74bdf640 1 Processor -- start 2023-11-28T18:29:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b74bdf640 1 -- start start 2023-11-28T18:29:52.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b74bdf640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b5c006f20 0x7f8b700b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:52.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b74bdf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b700af030 0x7f8b700af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:52.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b74bdf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b701f37b0 0x7f8b701f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:52.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b74bdf640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700af9f0 con 0x7f8b701f37b0 2023-11-28T18:29:52.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b74bdf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700c95a0 con 0x7f8b700af030 2023-11-28T18:29:52.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b74bdf640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700c9880 con 0x7f8b5c006f20 2023-11-28T18:29:52.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b6ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b701f37b0 0x7f8b701f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:52.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b6ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b701f37b0 0x7f8b701f3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44368/0 (socket says 172.21.15.105:44368) 2023-11-28T18:29:52.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b6ed9d640 1 -- 172.21.15.105:0/2810152957 learned_addr learned my addr 172.21.15.105:0/2810152957 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:52.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b6e59c640 1 --2- 172.21.15.105:0/2810152957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b5c006f20 0x7f8b700b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:52.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b65d9b640 1 --2- 172.21.15.105:0/2810152957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b700af030 0x7f8b700af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:52.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b6ed9d640 1 -- 172.21.15.105:0/2810152957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b5c006f20 msgr2=0x7f8b700b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:52.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b6ed9d640 1 --2- 172.21.15.105:0/2810152957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b5c006f20 0x7f8b700b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:52.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b6ed9d640 1 -- 172.21.15.105:0/2810152957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b700af030 msgr2=0x7f8b700af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:52.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b6ed9d640 1 --2- 172.21.15.105:0/2810152957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b700af030 0x7f8b700af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:52.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b6ed9d640 1 -- 172.21.15.105:0/2810152957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b700c9b60 con 0x7f8b701f37b0 2023-11-28T18:29:52.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b6ed9d640 1 --2- 172.21.15.105:0/2810152957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b701f37b0 0x7f8b701f3c60 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f8b680b0040 tx=0x7f8b68009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:52.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b677fe640 1 -- 172.21.15.105:0/2810152957 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b680039a0 con 0x7f8b701f37b0 2023-11-28T18:29:52.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b677fe640 1 -- 172.21.15.105:0/2810152957 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b68003b00 con 0x7f8b701f37b0 2023-11-28T18:29:52.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b74bdf640 1 -- 172.21.15.105:0/2810152957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b700c9e40 con 0x7f8b701f37b0 2023-11-28T18:29:52.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b677fe640 1 -- 172.21.15.105:0/2810152957 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b68003e90 con 0x7f8b701f37b0 2023-11-28T18:29:52.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b74bdf640 1 -- 172.21.15.105:0/2810152957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b701f41a0 con 0x7f8b701f37b0 2023-11-28T18:29:52.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.689+0000 7f8b74bdf640 1 -- 172.21.15.105:0/2810152957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b700ce9a0 con 0x7f8b701f37b0 2023-11-28T18:29:52.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.693+0000 7f8b677fe640 1 -- 172.21.15.105:0/2810152957 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8b680b4040 con 0x7f8b701f37b0 2023-11-28T18:29:52.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.693+0000 7f8b677fe640 1 --2- 172.21.15.105:0/2810152957 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b50066900 0x7f8b50068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:52.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.693+0000 7f8b677fe640 1 -- 172.21.15.105:0/2810152957 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6403+0+0 (secure 0 0 0) 0x7f8b60001d30 con 0x7f8b701f37b0 2023-11-28T18:29:52.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.693+0000 7f8b6e59c640 1 --2- 172.21.15.105:0/2810152957 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b50066900 0x7f8b50068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:52.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.693+0000 7f8b6e59c640 1 --2- 172.21.15.105:0/2810152957 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b50066900 0x7f8b50068dc0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8b7006e1b0 tx=0x7f8b60001480 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:52.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.697+0000 7f8b677fe640 1 -- 172.21.15.105:0/2810152957 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8b680eb370 con 0x7f8b701f37b0 2023-11-28T18:29:52.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:52.877+0000 7f8b74bdf640 1 -- 172.21.15.105:0/2810152957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8b70067570 con 0x7f8b701f37b0 2023-11-28T18:29:53.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.509+0000 7f8b677fe640 1 -- 172.21.15.105:0/2810152957 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v96) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8b680ee990 con 0x7f8b701f37b0 2023-11-28T18:29:53.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.513+0000 7f8b74bdf640 1 -- 172.21.15.105:0/2810152957 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b50066900 msgr2=0x7f8b50068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:53.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.513+0000 7f8b74bdf640 1 --2- 172.21.15.105:0/2810152957 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b50066900 0x7f8b50068dc0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8b7006e1b0 tx=0x7f8b60001480 comp rx=0 tx=0).stop 2023-11-28T18:29:53.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.513+0000 7f8b74bdf640 1 -- 172.21.15.105:0/2810152957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b701f37b0 msgr2=0x7f8b701f3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:53.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.513+0000 7f8b74bdf640 1 --2- 172.21.15.105:0/2810152957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b701f37b0 0x7f8b701f3c60 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f8b680b0040 tx=0x7f8b68009750 comp rx=0 tx=0).stop 2023-11-28T18:29:53.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.513+0000 7f8b74bdf640 1 -- 172.21.15.105:0/2810152957 shutdown_connections 2023-11-28T18:29:53.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.513+0000 7f8b74bdf640 1 --2- 172.21.15.105:0/2810152957 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b50066900 0x7f8b50068dc0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:53.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.513+0000 7f8b74bdf640 1 --2- 172.21.15.105:0/2810152957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b701f37b0 0x7f8b701f3c60 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:53.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.513+0000 7f8b74bdf640 1 --2- 172.21.15.105:0/2810152957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b700af030 0x7f8b700af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:53.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.513+0000 7f8b74bdf640 1 --2- 172.21.15.105:0/2810152957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b5c006f20 0x7f8b700b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:53.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.513+0000 7f8b74bdf640 1 -- 172.21.15.105:0/2810152957 >> 172.21.15.105:0/2810152957 conn(0x7f8b700ceb90 msgr2=0x7f8b70154970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:53.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.513+0000 7f8b74bdf640 1 -- 172.21.15.105:0/2810152957 shutdown_connections 2023-11-28T18:29:53.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.513+0000 7f8b74bdf640 1 -- 172.21.15.105:0/2810152957 wait complete. 2023-11-28T18:29:53.542 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T18:29:53.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.725+0000 7f3fc1617640 1 Processor -- start 2023-11-28T18:29:53.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.725+0000 7f3fc1617640 1 -- start start 2023-11-28T18:29:53.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.725+0000 7f3fc1617640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3fbc159610 0x7f3fbc0ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:53.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.725+0000 7f3fc1617640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fbc0cc760 0x7f3fbc0ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:53.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.725+0000 7f3fc1617640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fbc0cde30 con 0x7f3fbc06c7e0 2023-11-28T18:29:53.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.725+0000 7f3fc1617640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fbc1599f0 con 0x7f3fbc159610 2023-11-28T18:29:53.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.725+0000 7f3fc1617640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc06de20 con 0x7f3fbc0cc760 2023-11-28T18:29:53.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.725+0000 7f3fbb7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3fbc06c7e0 0x7f3fbc0abd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56700/0 (socket says 172.21.15.105:56700) 2023-11-28T18:29:53.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.725+0000 7f3fbb7fe640 1 -- 172.21.15.105:0/1495833532 learned_addr learned my addr 172.21.15.105:0/1495833532 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:53.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.725+0000 7f3fbaffd640 1 --2- 172.21.15.105:0/1495833532 >> v2:172.21.15.148:3300/0 conn(0x7f3fbc159610 0x7f3fbc0ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:53.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.725+0000 7f3fb27fc640 1 --2- 172.21.15.105:0/1495833532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fbc0cc760 0x7f3fbc0ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:53.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.725+0000 7f3fba7fc640 1 -- 172.21.15.105:0/1495833532 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345919100 0 0) 0x7f3fbc0cde30 con 0x7f3fbc06c7e0 2023-11-28T18:29:53.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.725+0000 7f3fba7fc640 1 -- 172.21.15.105:0/1495833532 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fa8003450 con 0x7f3fbc06c7e0 2023-11-28T18:29:53.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fba7fc640 1 -- 172.21.15.105:0/1495833532 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 517074761 0 0) 0x7f3fa8003450 con 0x7f3fbc06c7e0 2023-11-28T18:29:53.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fba7fc640 1 -- 172.21.15.105:0/1495833532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fbc0cc760 msgr2=0x7f3fbc0ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:53.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fba7fc640 1 --2- 172.21.15.105:0/1495833532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fbc0cc760 0x7f3fbc0ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:53.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fba7fc640 1 -- 172.21.15.105:0/1495833532 >> v2:172.21.15.148:3300/0 conn(0x7f3fbc159610 msgr2=0x7f3fbc0ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:53.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fba7fc640 1 --2- 172.21.15.105:0/1495833532 >> v2:172.21.15.148:3300/0 conn(0x7f3fbc159610 0x7f3fbc0ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:53.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fba7fc640 1 -- 172.21.15.105:0/1495833532 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fbc0cd220 con 0x7f3fbc06c7e0 2023-11-28T18:29:53.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fba7fc640 1 -- 172.21.15.105:0/1495833532 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f3fb4001680 con 0x7f3fbc06c7e0 2023-11-28T18:29:53.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fba7fc640 1 -- 172.21.15.105:0/1495833532 >> v1:172.21.15.105:6789/0 conn(0x7f3fbc06c7e0 legacy=0x7f3fbc0abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:53.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fba7fc640 1 --2- 172.21.15.105:0/1495833532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fa8003a70 0x7f3fa8003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:53.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fba7fc640 1 --2- 172.21.15.105:0/1495833532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fbc0cc760 0x7f3fa800a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:53.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fba7fc640 1 --2- 172.21.15.105:0/1495833532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fa800ae00 0x7f3fa800d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:53.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fba7fc640 1 -- 172.21.15.105:0/1495833532 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc1599f0 con 0x7f3fa8003a70 2023-11-28T18:29:53.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fba7fc640 1 -- 172.21.15.105:0/1495833532 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc06de20 con 0x7f3fa800ae00 2023-11-28T18:29:53.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fba7fc640 1 -- 172.21.15.105:0/1495833532 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa800daf0 con 0x7f3fbc0cc760 2023-11-28T18:29:53.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fbaffd640 1 --2- 172.21.15.105:0/1495833532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fa8003a70 0x7f3fa8003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:53.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fbaffd640 1 -- 172.21.15.105:0/1495833532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fbc0cc760 msgr2=0x7f3fa800a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:53.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fbaffd640 1 --2- 172.21.15.105:0/1495833532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fbc0cc760 0x7f3fa800a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:53.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fbaffd640 1 -- 172.21.15.105:0/1495833532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fa800ae00 msgr2=0x7f3fa800d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:53.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fbaffd640 1 --2- 172.21.15.105:0/1495833532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fa800ae00 0x7f3fa800d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:53.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fbaffd640 1 -- 172.21.15.105:0/1495833532 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa8003450 con 0x7f3fa8003a70 2023-11-28T18:29:53.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fbaffd640 1 --2- 172.21.15.105:0/1495833532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fa8003a70 0x7f3fa8003e90 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f3fac009f90 tx=0x7f3fac01cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0b1b138a139c36 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:53.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fba7fc640 1 -- 172.21.15.105:0/1495833532 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac0a5c00 con 0x7f3fa8003a70 2023-11-28T18:29:53.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fba7fc640 1 -- 172.21.15.105:0/1495833532 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fac00f040 con 0x7f3fa8003a70 2023-11-28T18:29:53.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.729+0000 7f3fba7fc640 1 -- 172.21.15.105:0/1495833532 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac0ad440 con 0x7f3fa8003a70 2023-11-28T18:29:53.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fc1617640 1 -- 172.21.15.105:0/1495833532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fa8003a70 msgr2=0x7f3fa8003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:53.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fc1617640 1 --2- 172.21.15.105:0/1495833532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fa8003a70 0x7f3fa8003e90 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f3fac009f90 tx=0x7f3fac01cc10 comp rx=0 tx=0).stop 2023-11-28T18:29:53.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fbb7fe640 1 -- 172.21.15.105:0/1495833532 reap_dead start 2023-11-28T18:29:53.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fc1617640 1 -- 172.21.15.105:0/1495833532 shutdown_connections 2023-11-28T18:29:53.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fc1617640 1 -- 172.21.15.105:0/1495833532 >> 172.21.15.105:0/1495833532 conn(0x7f3fbc0ceb90 msgr2=0x7f3fbc0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:53.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fc1617640 1 -- 172.21.15.105:0/1495833532 shutdown_connections 2023-11-28T18:29:53.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fc1617640 1 -- 172.21.15.105:0/1495833532 wait complete. 2023-11-28T18:29:53.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fc1617640 1 Processor -- start 2023-11-28T18:29:53.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fc1617640 1 -- start start 2023-11-28T18:29:53.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fc1617640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fa8003a70 0x7f3fbc156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:53.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fc1617640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fbc15a990 0x7f3fbc15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:53.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fc1617640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fbc15da00 0x7f3fbc0afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:53.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fc1617640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc1f9920 con 0x7f3fbc15a990 2023-11-28T18:29:53.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fc1617640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc1f9c00 con 0x7f3fbc15da00 2023-11-28T18:29:53.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fc1617640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc1f9ee0 con 0x7f3fa8003a70 2023-11-28T18:29:53.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fb27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fbc15a990 0x7f3fbc15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:53.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fb27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fbc15a990 0x7f3fbc15d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44388/0 (socket says 172.21.15.105:44388) 2023-11-28T18:29:53.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fb27fc640 1 -- 172.21.15.105:0/3452632496 learned_addr learned my addr 172.21.15.105:0/3452632496 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:53.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fbaffd640 1 --2- 172.21.15.105:0/3452632496 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fa8003a70 0x7f3fbc156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:53.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fbb7fe640 1 --2- 172.21.15.105:0/3452632496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fbc15da00 0x7f3fbc0afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:53.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fb27fc640 1 -- 172.21.15.105:0/3452632496 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fa8003a70 msgr2=0x7f3fbc156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:53.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fb27fc640 1 --2- 172.21.15.105:0/3452632496 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fa8003a70 0x7f3fbc156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:53.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.733+0000 7f3fb27fc640 1 -- 172.21.15.105:0/3452632496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fbc15da00 msgr2=0x7f3fbc0afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:53.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.737+0000 7f3fb27fc640 1 --2- 172.21.15.105:0/3452632496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fbc15da00 0x7f3fbc0afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:53.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.737+0000 7f3fb27fc640 1 -- 172.21.15.105:0/3452632496 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fbc1fa1c0 con 0x7f3fbc15a990 2023-11-28T18:29:53.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.737+0000 7f3fbaffd640 1 --2- 172.21.15.105:0/3452632496 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fa8003a70 0x7f3fbc156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:53.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.737+0000 7f3fbb7fe640 1 --2- 172.21.15.105:0/3452632496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fbc15da00 0x7f3fbc0afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:53.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.737+0000 7f3fb27fc640 1 --2- 172.21.15.105:0/3452632496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fbc15a990 0x7f3fbc15d4c0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400e7f0 tx=0x7f3fa400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:53.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.737+0000 7f3fb3fff640 1 -- 172.21.15.105:0/3452632496 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa400c940 con 0x7f3fbc15a990 2023-11-28T18:29:53.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.737+0000 7f3fb3fff640 1 -- 172.21.15.105:0/3452632496 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa400caa0 con 0x7f3fbc15a990 2023-11-28T18:29:53.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.737+0000 7f3fb3fff640 1 -- 172.21.15.105:0/3452632496 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa400ce50 con 0x7f3fbc15a990 2023-11-28T18:29:53.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.737+0000 7f3fc1617640 1 -- 172.21.15.105:0/3452632496 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fbc1fa4a0 con 0x7f3fbc15a990 2023-11-28T18:29:53.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.737+0000 7f3fc1617640 1 -- 172.21.15.105:0/3452632496 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fbc0aff90 con 0x7f3fbc15a990 2023-11-28T18:29:53.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.737+0000 7f3fc1617640 1 -- 172.21.15.105:0/3452632496 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fbc067570 con 0x7f3fbc15a990 2023-11-28T18:29:53.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.737+0000 7f3fb3fff640 1 -- 172.21.15.105:0/3452632496 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3fa4013470 con 0x7f3fbc15a990 2023-11-28T18:29:53.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.737+0000 7f3fb3fff640 1 --2- 172.21.15.105:0/3452632496 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3f9c0668d0 0x7f3f9c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:53.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.737+0000 7f3fb3fff640 1 -- 172.21.15.105:0/3452632496 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6403+0+0 (secure 0 0 0) 0x7f3fa4084840 con 0x7f3fbc15a990 2023-11-28T18:29:53.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.737+0000 7f3fbaffd640 1 --2- 172.21.15.105:0/3452632496 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3f9c0668d0 0x7f3f9c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:53.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.741+0000 7f3fbaffd640 1 --2- 172.21.15.105:0/3452632496 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3f9c0668d0 0x7f3f9c068d90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3fac0af000 tx=0x7f3fac0a7040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:53.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.741+0000 7f3fb3fff640 1 -- 172.21.15.105:0/3452632496 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3fa4051720 con 0x7f3fbc15a990 2023-11-28T18:29:53.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:53.893+0000 7f3fc1617640 1 -- 172.21.15.105:0/3452632496 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3fbc0cf950 con 0x7f3fbc15a990 2023-11-28T18:29:54.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.513+0000 7f3fb3fff640 1 -- 172.21.15.105:0/3452632496 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3fa4054d40 con 0x7f3fbc15a990 2023-11-28T18:29:54.516 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' removed 2023-11-28T18:29:54.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.513+0000 7f3fc1617640 1 -- 172.21.15.105:0/3452632496 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3f9c0668d0 msgr2=0x7f3f9c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:54.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.513+0000 7f3fc1617640 1 --2- 172.21.15.105:0/3452632496 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3f9c0668d0 0x7f3f9c068d90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3fac0af000 tx=0x7f3fac0a7040 comp rx=0 tx=0).stop 2023-11-28T18:29:54.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.513+0000 7f3fc1617640 1 -- 172.21.15.105:0/3452632496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fbc15a990 msgr2=0x7f3fbc15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:54.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.513+0000 7f3fc1617640 1 --2- 172.21.15.105:0/3452632496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fbc15a990 0x7f3fbc15d4c0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400e7f0 tx=0x7f3fa400ecb0 comp rx=0 tx=0).stop 2023-11-28T18:29:54.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.513+0000 7f3fc1617640 1 -- 172.21.15.105:0/3452632496 shutdown_connections 2023-11-28T18:29:54.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.513+0000 7f3fc1617640 1 --2- 172.21.15.105:0/3452632496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3fbc15da00 0x7f3fbc0afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:54.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.513+0000 7f3fc1617640 1 --2- 172.21.15.105:0/3452632496 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3f9c0668d0 0x7f3f9c068d90 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:54.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.513+0000 7f3fc1617640 1 --2- 172.21.15.105:0/3452632496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fbc15a990 0x7f3fbc15d4c0 secure :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400e7f0 tx=0x7f3fa400ecb0 comp rx=0 tx=0).stop 2023-11-28T18:29:54.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.513+0000 7f3fc1617640 1 --2- 172.21.15.105:0/3452632496 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3fa8003a70 0x7f3fbc156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:54.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.513+0000 7f3fc1617640 1 -- 172.21.15.105:0/3452632496 >> 172.21.15.105:0/3452632496 conn(0x7f3fbc0ceb90 msgr2=0x7f3fbc158630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:54.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.513+0000 7f3fc1617640 1 -- 172.21.15.105:0/3452632496 shutdown_connections 2023-11-28T18:29:54.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.517+0000 7f3fc1617640 1 -- 172.21.15.105:0/3452632496 wait complete. 2023-11-28T18:29:54.544 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T18:29:54.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a759e640 1 Processor -- start 2023-11-28T18:29:54.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a759e640 1 -- start start 2023-11-28T18:29:54.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a759e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f74a80d0d40 0x7f74a80d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:54.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a759e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a80cf440 con 0x7f74a815d1a0 2023-11-28T18:29:54.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a759e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a80cffc0 con 0x7f74a80d0450 2023-11-28T18:29:54.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a759e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74a80ceb40 con 0x7f74a80d0d40 2023-11-28T18:29:54.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a659c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f74a815d1a0 0x7f74a80ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56716/0 (socket says 172.21.15.105:56716) 2023-11-28T18:29:54.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a659c640 1 -- 172.21.15.105:0/3272428038 learned_addr learned my addr 172.21.15.105:0/3272428038 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:54.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a5d9b640 1 -- 172.21.15.105:0/3272428038 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082759929 0 0) 0x7f74a80cf440 con 0x7f74a815d1a0 2023-11-28T18:29:54.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a5d9b640 1 -- 172.21.15.105:0/3272428038 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7494003530 con 0x7f74a815d1a0 2023-11-28T18:29:54.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f749dd9b640 1 --2- 172.21.15.105:0/3272428038 >> v2:172.21.15.148:3300/0 conn(0x7f74a80d0d40 0x7f74a80d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:54.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a5d9b640 1 -- 172.21.15.105:0/3272428038 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 274437028 0 0) 0x7f7494003530 con 0x7f74a815d1a0 2023-11-28T18:29:54.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a5d9b640 1 -- 172.21.15.105:0/3272428038 >> v2:172.21.15.148:3300/0 conn(0x7f74a80d0d40 msgr2=0x7f74a80d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:54.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a5d9b640 1 --2- 172.21.15.105:0/3272428038 >> v2:172.21.15.148:3300/0 conn(0x7f74a80d0d40 0x7f74a80d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:54.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a5d9b640 1 -- 172.21.15.105:0/3272428038 >> v1:172.21.15.148:6789/0 conn(0x7f74a80d0450 legacy=0x7f74a80b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:54.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a5d9b640 1 -- 172.21.15.105:0/3272428038 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a80cf690 con 0x7f74a815d1a0 2023-11-28T18:29:54.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a5d9b640 1 -- 172.21.15.105:0/3272428038 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f7498001680 con 0x7f74a815d1a0 2023-11-28T18:29:54.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a5d9b640 1 -- 172.21.15.105:0/3272428038 >> v1:172.21.15.105:6789/0 conn(0x7f74a815d1a0 legacy=0x7f74a80ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:54.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a5d9b640 1 --2- 172.21.15.105:0/3272428038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74940039d0 0x7f7494003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:54.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a5d9b640 1 --2- 172.21.15.105:0/3272428038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f74940047d0 0x7f7494004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:54.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a5d9b640 1 --2- 172.21.15.105:0/3272428038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74940052b0 0x7f749400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:54.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a5d9b640 1 -- 172.21.15.105:0/3272428038 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80ceb40 con 0x7f74940052b0 2023-11-28T18:29:54.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a5d9b640 1 -- 172.21.15.105:0/3272428038 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749400c1d0 con 0x7f74940039d0 2023-11-28T18:29:54.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a5d9b640 1 -- 172.21.15.105:0/3272428038 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749400c390 con 0x7f74940047d0 2023-11-28T18:29:54.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a6d9d640 1 --2- 172.21.15.105:0/3272428038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74940052b0 0x7f749400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:54.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f749dd9b640 1 --2- 172.21.15.105:0/3272428038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f74940047d0 0x7f7494004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:54.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a659c640 1 --2- 172.21.15.105:0/3272428038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74940039d0 0x7f7494003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:54.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a6d9d640 1 -- 172.21.15.105:0/3272428038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f74940047d0 msgr2=0x7f7494004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:54.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a6d9d640 1 --2- 172.21.15.105:0/3272428038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f74940047d0 0x7f7494004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:54.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a6d9d640 1 -- 172.21.15.105:0/3272428038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74940039d0 msgr2=0x7f7494003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:54.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a6d9d640 1 --2- 172.21.15.105:0/3272428038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74940039d0 0x7f7494003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:54.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a6d9d640 1 -- 172.21.15.105:0/3272428038 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7494003530 con 0x7f74940052b0 2023-11-28T18:29:54.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.685+0000 7f74a659c640 1 --2- 172.21.15.105:0/3272428038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74940039d0 0x7f7494003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:54.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.689+0000 7f74a6d9d640 1 -- 172.21.15.105:0/3272428038 reap_dead start 2023-11-28T18:29:54.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.689+0000 7f74a6d9d640 1 --2- 172.21.15.105:0/3272428038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74940052b0 0x7f749400b990 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f74a0001780 tx=0x7f74a001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70b4e031232ce67 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:54.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.689+0000 7f74a5d9b640 1 -- 172.21.15.105:0/3272428038 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a00052d0 con 0x7f74940052b0 2023-11-28T18:29:54.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.689+0000 7f74a5d9b640 1 -- 172.21.15.105:0/3272428038 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a00a7050 con 0x7f74940052b0 2023-11-28T18:29:54.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.689+0000 7f74a5d9b640 1 -- 172.21.15.105:0/3272428038 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a0003840 con 0x7f74940052b0 2023-11-28T18:29:54.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.689+0000 7f74a759e640 1 -- 172.21.15.105:0/3272428038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74940052b0 msgr2=0x7f749400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:54.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.689+0000 7f74a759e640 1 --2- 172.21.15.105:0/3272428038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74940052b0 0x7f749400b990 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f74a0001780 tx=0x7f74a001c400 comp rx=0 tx=0).stop 2023-11-28T18:29:54.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.689+0000 7f74a759e640 1 -- 172.21.15.105:0/3272428038 shutdown_connections 2023-11-28T18:29:54.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.689+0000 7f74a759e640 1 --2- 172.21.15.105:0/3272428038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74940052b0 0x7f749400b990 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:54.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.689+0000 7f74a759e640 1 -- 172.21.15.105:0/3272428038 >> 172.21.15.105:0/3272428038 conn(0x7f74a80d2e10 msgr2=0x7f74a80d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:54.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.689+0000 7f74a759e640 1 -- 172.21.15.105:0/3272428038 shutdown_connections 2023-11-28T18:29:54.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.689+0000 7f74a759e640 1 -- 172.21.15.105:0/3272428038 wait complete. 2023-11-28T18:29:54.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f74a759e640 1 Processor -- start 2023-11-28T18:29:54.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f74a759e640 1 -- start start 2023-11-28T18:29:54.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f74a759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74940052b0 0x7f74a81f2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:54.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f74a759e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74a80b71d0 0x7f74a806c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:54.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f74a759e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f74a80b7650 0x7f74a806c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:54.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f74a759e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80cb680 con 0x7f74940052b0 2023-11-28T18:29:54.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f74a759e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80cb960 con 0x7f74a80b71d0 2023-11-28T18:29:54.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f74a759e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80cbc40 con 0x7f74a80b7650 2023-11-28T18:29:54.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f74a659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74940052b0 0x7f74a81f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:54.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f74a659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74940052b0 0x7f74a81f2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44402/0 (socket says 172.21.15.105:44402) 2023-11-28T18:29:54.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f74a659c640 1 -- 172.21.15.105:0/1458078283 learned_addr learned my addr 172.21.15.105:0/1458078283 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:54.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f749dd9b640 1 --2- 172.21.15.105:0/1458078283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74a80b71d0 0x7f74a806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:54.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f74a6d9d640 1 --2- 172.21.15.105:0/1458078283 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f74a80b7650 0x7f74a806c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:54.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f74a659c640 1 -- 172.21.15.105:0/1458078283 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f74a80b7650 msgr2=0x7f74a806c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:54.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f74a659c640 1 --2- 172.21.15.105:0/1458078283 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f74a80b7650 0x7f74a806c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:54.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f74a659c640 1 -- 172.21.15.105:0/1458078283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74a80b71d0 msgr2=0x7f74a806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:54.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f74a659c640 1 --2- 172.21.15.105:0/1458078283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74a80b71d0 0x7f74a806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:54.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f74a659c640 1 -- 172.21.15.105:0/1458078283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a80cec80 con 0x7f74940052b0 2023-11-28T18:29:54.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f74a659c640 1 --2- 172.21.15.105:0/1458078283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74940052b0 0x7f74a81f2950 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f7498011840 tx=0x7f7498011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:54.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f749f7fe640 1 -- 172.21.15.105:0/1458078283 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749800fb60 con 0x7f74940052b0 2023-11-28T18:29:54.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f749f7fe640 1 -- 172.21.15.105:0/1458078283 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749800fcc0 con 0x7f74940052b0 2023-11-28T18:29:54.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f749f7fe640 1 -- 172.21.15.105:0/1458078283 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7498015630 con 0x7f74940052b0 2023-11-28T18:29:54.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f74a759e640 1 -- 172.21.15.105:0/1458078283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a80cbf20 con 0x7f74940052b0 2023-11-28T18:29:54.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.693+0000 7f74a759e640 1 -- 172.21.15.105:0/1458078283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a80b82e0 con 0x7f74940052b0 2023-11-28T18:29:54.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.697+0000 7f74a759e640 1 -- 172.21.15.105:0/1458078283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a80cc3d0 con 0x7f74940052b0 2023-11-28T18:29:54.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.697+0000 7f749f7fe640 1 -- 172.21.15.105:0/1458078283 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f7498015790 con 0x7f74940052b0 2023-11-28T18:29:54.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.697+0000 7f749f7fe640 1 --2- 172.21.15.105:0/1458078283 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f74880668d0 0x7f7488068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:54.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.697+0000 7f749f7fe640 1 -- 172.21.15.105:0/1458078283 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5965+0+0 (secure 0 0 0) 0x7f7498087710 con 0x7f74940052b0 2023-11-28T18:29:54.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.697+0000 7f749dd9b640 1 --2- 172.21.15.105:0/1458078283 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f74880668d0 0x7f7488068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:54.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.697+0000 7f749dd9b640 1 --2- 172.21.15.105:0/1458078283 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f74880668d0 0x7f7488068d90 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f74a8160060 tx=0x7f7490009760 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:54.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.701+0000 7f749f7fe640 1 -- 172.21.15.105:0/1458078283 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f74a80cc3d0 con 0x7f74940052b0 2023-11-28T18:29:54.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:54.849+0000 7f74a759e640 1 -- 172.21.15.105:0/1458078283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f74a80cc3d0 con 0x7f74940052b0 2023-11-28T18:29:55.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.513+0000 7f749f7fe640 1 -- 172.21.15.105:0/1458078283 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) v1 ==== 169+0+0 (secure 0 0 0) 0x7f74a80cc3d0 con 0x7f74940052b0 2023-11-28T18:29:55.518 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' removed 2023-11-28T18:29:55.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.517+0000 7f74a759e640 1 -- 172.21.15.105:0/1458078283 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f74880668d0 msgr2=0x7f7488068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:55.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.517+0000 7f74a759e640 1 --2- 172.21.15.105:0/1458078283 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f74880668d0 0x7f7488068d90 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f74a8160060 tx=0x7f7490009760 comp rx=0 tx=0).stop 2023-11-28T18:29:55.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.517+0000 7f74a759e640 1 -- 172.21.15.105:0/1458078283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74940052b0 msgr2=0x7f74a81f2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:55.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.517+0000 7f74a759e640 1 --2- 172.21.15.105:0/1458078283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74940052b0 0x7f74a81f2950 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f7498011840 tx=0x7f7498011d00 comp rx=0 tx=0).stop 2023-11-28T18:29:55.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.517+0000 7f74a759e640 1 -- 172.21.15.105:0/1458078283 shutdown_connections 2023-11-28T18:29:55.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.517+0000 7f74a759e640 1 --2- 172.21.15.105:0/1458078283 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f74a80b7650 0x7f74a806c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:55.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.517+0000 7f74a759e640 1 --2- 172.21.15.105:0/1458078283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74a80b71d0 0x7f74a806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:55.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.517+0000 7f74a759e640 1 --2- 172.21.15.105:0/1458078283 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f74880668d0 0x7f7488068d90 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:55.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.517+0000 7f74a759e640 1 --2- 172.21.15.105:0/1458078283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74940052b0 0x7f74a81f2950 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:55.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.517+0000 7f74a759e640 1 -- 172.21.15.105:0/1458078283 >> 172.21.15.105:0/1458078283 conn(0x7f74a80d2e10 msgr2=0x7f74a8072f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:55.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.517+0000 7f74a759e640 1 -- 172.21.15.105:0/1458078283 shutdown_connections 2023-11-28T18:29:55.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.517+0000 7f74a759e640 1 -- 172.21.15.105:0/1458078283 wait complete. 2023-11-28T18:29:55.553 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random 2023-11-28T18:29:55.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.693+0000 7fbafefc4640 1 Processor -- start 2023-11-28T18:29:55.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.693+0000 7fbafefc4640 1 -- start start 2023-11-28T18:29:55.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.693+0000 7fbafefc4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbaf815d3c0 0x7fbaf8071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:55.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.693+0000 7fbafefc4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbaf8070a10 0x7fbaf8070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:55.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.693+0000 7fbafefc4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaf80713a0 con 0x7fbaf8161870 2023-11-28T18:29:55.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.693+0000 7fbafefc4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaf80cefa0 con 0x7fbaf815d3c0 2023-11-28T18:29:55.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.693+0000 7fbafefc4640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf815d7a0 con 0x7fbaf8070a10 2023-11-28T18:29:55.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.693+0000 7fbafd561640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbaf8070a10 0x7fbaf8070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:55.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.693+0000 7fbafcd60640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbaf815d3c0 0x7fbaf8071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:55.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.693+0000 7fbafcd60640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbaf815d3c0 0x7fbaf8071a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:50372/0 (socket says 172.21.15.105:50372) 2023-11-28T18:29:55.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.693+0000 7fbafcd60640 1 -- 172.21.15.105:0/662412983 learned_addr learned my addr 172.21.15.105:0/662412983 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:55.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.693+0000 7fbafcd60640 1 -- 172.21.15.105:0/662412983 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbaf8070a10 msgr2=0x7fbaf8070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:55.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.693+0000 7fbafcd60640 1 --2- 172.21.15.105:0/662412983 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbaf8070a10 0x7fbaf8070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:55.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.693+0000 7fbafcd60640 1 -- 172.21.15.105:0/662412983 >> v1:172.21.15.148:6789/0 conn(0x7fbaf8161870 legacy=0x7fbaf80ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:55.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.693+0000 7fbafcd60640 1 -- 172.21.15.105:0/662412983 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf80ce9e0 con 0x7fbaf815d3c0 2023-11-28T18:29:55.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.693+0000 7fbafd561640 1 --2- 172.21.15.105:0/662412983 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbaf8070a10 0x7fbaf8070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:29:55.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.693+0000 7fbafcd60640 1 --2- 172.21.15.105:0/662412983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbaf815d3c0 0x7fbaf8071a10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009870 tx=0x7fbaf401c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b28120d9ad32207 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:55.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.693+0000 7fbaef7fe640 1 -- 172.21.15.105:0/662412983 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf40aa670 con 0x7fbaf815d3c0 2023-11-28T18:29:55.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.693+0000 7fbaef7fe640 1 -- 172.21.15.105:0/662412983 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf40aa7d0 con 0x7fbaf815d3c0 2023-11-28T18:29:55.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.693+0000 7fbaef7fe640 1 -- 172.21.15.105:0/662412983 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf40aab60 con 0x7fbaf815d3c0 2023-11-28T18:29:55.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafefc4640 1 -- 172.21.15.105:0/662412983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbaf815d3c0 msgr2=0x7fbaf8071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:55.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafefc4640 1 --2- 172.21.15.105:0/662412983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbaf815d3c0 0x7fbaf8071a10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009870 tx=0x7fbaf401c510 comp rx=0 tx=0).stop 2023-11-28T18:29:55.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafefc4640 1 -- 172.21.15.105:0/662412983 shutdown_connections 2023-11-28T18:29:55.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafefc4640 1 --2- 172.21.15.105:0/662412983 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbaf8070a10 0x7fbaf8070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:55.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafefc4640 1 --2- 172.21.15.105:0/662412983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbaf815d3c0 0x7fbaf8071a10 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:55.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafefc4640 1 -- 172.21.15.105:0/662412983 >> 172.21.15.105:0/662412983 conn(0x7fbaf80cf540 msgr2=0x7fbaf80cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:55.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafefc4640 1 -- 172.21.15.105:0/662412983 shutdown_connections 2023-11-28T18:29:55.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafefc4640 1 -- 172.21.15.105:0/662412983 wait complete. 2023-11-28T18:29:55.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafefc4640 1 Processor -- start 2023-11-28T18:29:55.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafefc4640 1 -- start start 2023-11-28T18:29:55.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafefc4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbaf8070a10 0x7fbaf815c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:55.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafefc4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbaf815d3c0 0x7fbaf8152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:55.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafefc4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbaf8161870 0x7fbaf814fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:55.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafefc4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf815c920 con 0x7fbaf8161870 2023-11-28T18:29:55.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafefc4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf814ece0 con 0x7fbaf8070a10 2023-11-28T18:29:55.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafefc4640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf814f1e0 con 0x7fbaf815d3c0 2023-11-28T18:29:55.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafd561640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbaf8161870 0x7fbaf814fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:55.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafd561640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbaf8161870 0x7fbaf814fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44404/0 (socket says 172.21.15.105:44404) 2023-11-28T18:29:55.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafd561640 1 -- 172.21.15.105:0/2824001189 learned_addr learned my addr 172.21.15.105:0/2824001189 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:55.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafcd60640 1 --2- 172.21.15.105:0/2824001189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbaf8070a10 0x7fbaf815c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:55.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbaeffff640 1 --2- 172.21.15.105:0/2824001189 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbaf815d3c0 0x7fbaf8152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:55.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafd561640 1 -- 172.21.15.105:0/2824001189 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbaf815d3c0 msgr2=0x7fbaf8152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:55.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafd561640 1 --2- 172.21.15.105:0/2824001189 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbaf815d3c0 0x7fbaf8152840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:55.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafd561640 1 -- 172.21.15.105:0/2824001189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbaf8070a10 msgr2=0x7fbaf815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:55.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafd561640 1 --2- 172.21.15.105:0/2824001189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbaf8070a10 0x7fbaf815c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:55.705 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.697+0000 7fbafd561640 1 -- 172.21.15.105:0/2824001189 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf0009a70 con 0x7fbaf8161870 2023-11-28T18:29:55.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.701+0000 7fbafd561640 1 --2- 172.21.15.105:0/2824001189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbaf8161870 0x7fbaf814fd00 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000efc0 tx=0x7fbaf0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:55.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.701+0000 7fbaed7fa640 1 -- 172.21.15.105:0/2824001189 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0009d70 con 0x7fbaf8161870 2023-11-28T18:29:55.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.701+0000 7fbaed7fa640 1 -- 172.21.15.105:0/2824001189 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf000f040 con 0x7fbaf8161870 2023-11-28T18:29:55.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.701+0000 7fbaed7fa640 1 -- 172.21.15.105:0/2824001189 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf00039a0 con 0x7fbaf8161870 2023-11-28T18:29:55.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.701+0000 7fbafefc4640 1 -- 172.21.15.105:0/2824001189 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf40094d0 con 0x7fbaf8161870 2023-11-28T18:29:55.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.701+0000 7fbafefc4640 1 -- 172.21.15.105:0/2824001189 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf814dc40 con 0x7fbaf8161870 2023-11-28T18:29:55.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.701+0000 7fbafefc4640 1 -- 172.21.15.105:0/2824001189 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaf8067570 con 0x7fbaf8161870 2023-11-28T18:29:55.706 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.701+0000 7fbaed7fa640 1 -- 172.21.15.105:0/2824001189 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fbaf00144a0 con 0x7fbaf8161870 2023-11-28T18:29:55.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.701+0000 7fbaed7fa640 1 --2- 172.21.15.105:0/2824001189 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbae40668d0 0x7fbae4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:55.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.701+0000 7fbaed7fa640 1 -- 172.21.15.105:0/2824001189 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fbaf0084f20 con 0x7fbaf8161870 2023-11-28T18:29:55.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.701+0000 7fbafcd60640 1 --2- 172.21.15.105:0/2824001189 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbae40668d0 0x7fbae4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:55.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.705+0000 7fbafcd60640 1 --2- 172.21.15.105:0/2824001189 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbae40668d0 0x7fbae4068d90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40ae000 tx=0x7fbaf40b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:55.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.705+0000 7fbaed7fa640 1 -- 172.21.15.105:0/2824001189 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbaf0052280 con 0x7fbaf8161870 2023-11-28T18:29:55.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.857+0000 7fbafefc4640 1 -- 172.21.15.105:0/2824001189 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"} v 0) v1 -- 0x7fbaf8066960 con 0x7fbaf8161870 2023-11-28T18:29:55.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.861+0000 7fbaed7fa640 1 -- 172.21.15.105:0/2824001189 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"}]=0 v47) v1 ==== 110+0+0 (secure 0 0 0) 0x7fbaf00558a0 con 0x7fbaf8161870 2023-11-28T18:29:55.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.861+0000 7fbafefc4640 1 -- 172.21.15.105:0/2824001189 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbae40668d0 msgr2=0x7fbae4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:55.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.861+0000 7fbafefc4640 1 --2- 172.21.15.105:0/2824001189 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbae40668d0 0x7fbae4068d90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40ae000 tx=0x7fbaf40b5040 comp rx=0 tx=0).stop 2023-11-28T18:29:55.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.861+0000 7fbafefc4640 1 -- 172.21.15.105:0/2824001189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbaf8161870 msgr2=0x7fbaf814fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:55.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.861+0000 7fbafefc4640 1 --2- 172.21.15.105:0/2824001189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbaf8161870 0x7fbaf814fd00 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000efc0 tx=0x7fbaf0009f90 comp rx=0 tx=0).stop 2023-11-28T18:29:55.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.861+0000 7fbafefc4640 1 -- 172.21.15.105:0/2824001189 shutdown_connections 2023-11-28T18:29:55.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.861+0000 7fbafefc4640 1 --2- 172.21.15.105:0/2824001189 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbae40668d0 0x7fbae4068d90 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:55.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.861+0000 7fbafefc4640 1 --2- 172.21.15.105:0/2824001189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbaf8161870 0x7fbaf814fd00 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:55.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.861+0000 7fbafefc4640 1 --2- 172.21.15.105:0/2824001189 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbaf815d3c0 0x7fbaf8152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:55.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.861+0000 7fbafefc4640 1 --2- 172.21.15.105:0/2824001189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbaf8070a10 0x7fbaf815c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:55.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.861+0000 7fbafefc4640 1 -- 172.21.15.105:0/2824001189 >> 172.21.15.105:0/2824001189 conn(0x7fbaf80cf540 msgr2=0x7fbaf80b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:55.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.861+0000 7fbafefc4640 1 -- 172.21.15.105:0/2824001189 shutdown_connections 2023-11-28T18:29:55.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:55.861+0000 7fbafefc4640 1 -- 172.21.15.105:0/2824001189 wait complete. 2023-11-28T18:29:55.887 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T18:29:56.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc34b1e6640 1 Processor -- start 2023-11-28T18:29:56.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc34b1e6640 1 -- start start 2023-11-28T18:29:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc34b1e6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc344159f40 0x7fc3440caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc34b1e6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3440cc760 0x7fc3440ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc34b1e6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc34406bf50 con 0x7fc3440ad680 2023-11-28T18:29:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc34b1e6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3440ccbc0 con 0x7fc344159f40 2023-11-28T18:29:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc34b1e6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc34415a320 con 0x7fc3440cc760 2023-11-28T18:29:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc349783640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc3440ad680 0x7fc34415da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56732/0 (socket says 172.21.15.105:56732) 2023-11-28T18:29:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc349783640 1 -- 172.21.15.105:0/499474970 learned_addr learned my addr 172.21.15.105:0/499474970 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc348f82640 1 --2- 172.21.15.105:0/499474970 >> v2:172.21.15.148:3300/0 conn(0x7fc344159f40 0x7fc3440caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc343fff640 1 --2- 172.21.15.105:0/499474970 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3440cc760 0x7fc3440ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc3437fe640 1 -- 172.21.15.105:0/499474970 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973274208 0 0) 0x7fc34406bf50 con 0x7fc3440ad680 2023-11-28T18:29:56.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc3437fe640 1 -- 172.21.15.105:0/499474970 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc334003530 con 0x7fc3440ad680 2023-11-28T18:29:56.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc3437fe640 1 -- 172.21.15.105:0/499474970 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 900797408 0 0) 0x7fc334003530 con 0x7fc3440ad680 2023-11-28T18:29:56.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc3437fe640 1 -- 172.21.15.105:0/499474970 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3440cc760 msgr2=0x7fc3440ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:56.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc3437fe640 1 --2- 172.21.15.105:0/499474970 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3440cc760 0x7fc3440ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc3437fe640 1 -- 172.21.15.105:0/499474970 >> v2:172.21.15.148:3300/0 conn(0x7fc344159f40 msgr2=0x7fc3440caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:56.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc3437fe640 1 --2- 172.21.15.105:0/499474970 >> v2:172.21.15.148:3300/0 conn(0x7fc344159f40 0x7fc3440caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc3437fe640 1 -- 172.21.15.105:0/499474970 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc344071380 con 0x7fc3440ad680 2023-11-28T18:29:56.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc3437fe640 1 -- 172.21.15.105:0/499474970 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fc33c001a00 con 0x7fc3440ad680 2023-11-28T18:29:56.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc343fff640 1 --2- 172.21.15.105:0/499474970 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3440cc760 0x7fc3440ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:29:56.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc3437fe640 1 -- 172.21.15.105:0/499474970 >> v1:172.21.15.105:6789/0 conn(0x7fc3440ad680 legacy=0x7fc34415da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:56.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc3437fe640 1 --2- 172.21.15.105:0/499474970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc334003980 0x7fc334003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc3437fe640 1 --2- 172.21.15.105:0/499474970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3340047d0 0x7fc334004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc3437fe640 1 --2- 172.21.15.105:0/499474970 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3440cc760 0x7fc334005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc3437fe640 1 -- 172.21.15.105:0/499474970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3440ccbc0 con 0x7fc3340047d0 2023-11-28T18:29:56.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc3437fe640 1 -- 172.21.15.105:0/499474970 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34415a320 con 0x7fc334003980 2023-11-28T18:29:56.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc3437fe640 1 -- 172.21.15.105:0/499474970 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc33400dba0 con 0x7fc3440cc760 2023-11-28T18:29:56.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc348f82640 1 --2- 172.21.15.105:0/499474970 >> v2:172.21.15.148:3300/0 conn(0x7fc344159f40 0x7fc3440caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:29:56.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc343fff640 1 --2- 172.21.15.105:0/499474970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3340047d0 0x7fc334004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc349783640 1 --2- 172.21.15.105:0/499474970 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3440cc760 0x7fc334005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc348f82640 1 --2- 172.21.15.105:0/499474970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc334003980 0x7fc334003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc343fff640 1 -- 172.21.15.105:0/499474970 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3440cc760 msgr2=0x7fc334005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc343fff640 1 --2- 172.21.15.105:0/499474970 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3440cc760 0x7fc334005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc343fff640 1 -- 172.21.15.105:0/499474970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc334003980 msgr2=0x7fc334003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc343fff640 1 --2- 172.21.15.105:0/499474970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc334003980 0x7fc334003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.025+0000 7fc343fff640 1 -- 172.21.15.105:0/499474970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc330009420 con 0x7fc3340047d0 2023-11-28T18:29:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.029+0000 7fc343fff640 1 --2- 172.21.15.105:0/499474970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3340047d0 0x7fc334004c50 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fc330009cb0 tx=0x7fc33001cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c66402e08688344 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.029+0000 7fc3437fe640 1 -- 172.21.15.105:0/499474970 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3300a5780 con 0x7fc3340047d0 2023-11-28T18:29:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.029+0000 7fc3437fe640 1 -- 172.21.15.105:0/499474970 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3300a58e0 con 0x7fc3340047d0 2023-11-28T18:29:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.029+0000 7fc3437fe640 1 -- 172.21.15.105:0/499474970 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3300a5c90 con 0x7fc3340047d0 2023-11-28T18:29:56.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.029+0000 7fc34b1e6640 1 -- 172.21.15.105:0/499474970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3340047d0 msgr2=0x7fc334004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:56.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.029+0000 7fc34b1e6640 1 --2- 172.21.15.105:0/499474970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3340047d0 0x7fc334004c50 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fc330009cb0 tx=0x7fc33001cbc0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.029+0000 7fc349783640 1 -- 172.21.15.105:0/499474970 reap_dead start 2023-11-28T18:29:56.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.029+0000 7fc34b1e6640 1 -- 172.21.15.105:0/499474970 shutdown_connections 2023-11-28T18:29:56.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.029+0000 7fc34b1e6640 1 -- 172.21.15.105:0/499474970 >> 172.21.15.105:0/499474970 conn(0x7fc3440ceb90 msgr2=0x7fc3440cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:56.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.029+0000 7fc34b1e6640 1 -- 172.21.15.105:0/499474970 shutdown_connections 2023-11-28T18:29:56.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.029+0000 7fc34b1e6640 1 -- 172.21.15.105:0/499474970 wait complete. 2023-11-28T18:29:56.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.029+0000 7fc34b1e6640 1 Processor -- start 2023-11-28T18:29:56.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.029+0000 7fc34b1e6640 1 -- start start 2023-11-28T18:29:56.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc34b1e6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3340047d0 0x7fc34414c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc34b1e6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc34414cf50 0x7fc3440ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc34b1e6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc34414d4a0 0x7fc3440ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc34b1e6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3441594f0 con 0x7fc34414d4a0 2023-11-28T18:29:56.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc34b1e6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc344158680 con 0x7fc34414cf50 2023-11-28T18:29:56.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc34b1e6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc344158920 con 0x7fc3340047d0 2023-11-28T18:29:56.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc349783640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc34414d4a0 0x7fc3440ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc348f82640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3340047d0 0x7fc34414c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc349783640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc34414d4a0 0x7fc3440ca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44430/0 (socket says 172.21.15.105:44430) 2023-11-28T18:29:56.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc349783640 1 -- 172.21.15.105:0/121826844 learned_addr learned my addr 172.21.15.105:0/121826844 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc348f82640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3340047d0 0x7fc34414c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:45436/0 (socket says 172.21.15.105:45436) 2023-11-28T18:29:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc343fff640 1 --2- 172.21.15.105:0/121826844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc34414cf50 0x7fc3440ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc349783640 1 -- 172.21.15.105:0/121826844 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3340047d0 msgr2=0x7fc34414c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc349783640 1 --2- 172.21.15.105:0/121826844 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3340047d0 0x7fc34414c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc349783640 1 -- 172.21.15.105:0/121826844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc34414cf50 msgr2=0x7fc3440ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc349783640 1 --2- 172.21.15.105:0/121826844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc34414cf50 0x7fc3440ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc349783640 1 -- 172.21.15.105:0/121826844 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc344158bc0 con 0x7fc34414d4a0 2023-11-28T18:29:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc349783640 1 --2- 172.21.15.105:0/121826844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc34414d4a0 0x7fc3440ca630 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0013e0 tx=0x7fc33c00a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc3417fa640 1 -- 172.21.15.105:0/121826844 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c00aca0 con 0x7fc34414d4a0 2023-11-28T18:29:56.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc3417fa640 1 -- 172.21.15.105:0/121826844 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33c00ae20 con 0x7fc34414d4a0 2023-11-28T18:29:56.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc34b1e6640 1 -- 172.21.15.105:0/121826844 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc344158ea0 con 0x7fc34414d4a0 2023-11-28T18:29:56.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc3417fa640 1 -- 172.21.15.105:0/121826844 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c013630 con 0x7fc34414d4a0 2023-11-28T18:29:56.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc34b1e6640 1 -- 172.21.15.105:0/121826844 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc34414dc90 con 0x7fc34414d4a0 2023-11-28T18:29:56.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc34b1e6640 1 -- 172.21.15.105:0/121826844 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc344067570 con 0x7fc34414d4a0 2023-11-28T18:29:56.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.033+0000 7fc3417fa640 1 -- 172.21.15.105:0/121826844 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc33c013790 con 0x7fc34414d4a0 2023-11-28T18:29:56.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.037+0000 7fc3417fa640 1 --2- 172.21.15.105:0/121826844 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc3280668d0 0x7fc328068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.037+0000 7fc3417fa640 1 -- 172.21.15.105:0/121826844 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fc33c084d10 con 0x7fc34414d4a0 2023-11-28T18:29:56.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.037+0000 7fc348f82640 1 --2- 172.21.15.105:0/121826844 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc3280668d0 0x7fc328068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.037+0000 7fc348f82640 1 --2- 172.21.15.105:0/121826844 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc3280668d0 0x7fc328068d90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc3380093f0 tx=0x7fc338009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:56.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.041+0000 7fc3417fa640 1 -- 172.21.15.105:0/121826844 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc33c052100 con 0x7fc34414d4a0 2023-11-28T18:29:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.189+0000 7fc34b1e6640 1 -- 172.21.15.105:0/121826844 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc344160b50 con 0x7fc34414d4a0 2023-11-28T18:29:56.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.197+0000 7fc3417fa640 1 -- 172.21.15.105:0/121826844 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v48) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc33c055720 con 0x7fc34414d4a0 2023-11-28T18:29:56.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.197+0000 7fc34b1e6640 1 -- 172.21.15.105:0/121826844 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc3280668d0 msgr2=0x7fc328068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:56.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.197+0000 7fc34b1e6640 1 --2- 172.21.15.105:0/121826844 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc3280668d0 0x7fc328068d90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc3380093f0 tx=0x7fc338009190 comp rx=0 tx=0).stop 2023-11-28T18:29:56.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.197+0000 7fc34b1e6640 1 -- 172.21.15.105:0/121826844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc34414d4a0 msgr2=0x7fc3440ca630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:56.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.197+0000 7fc34b1e6640 1 --2- 172.21.15.105:0/121826844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc34414d4a0 0x7fc3440ca630 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0013e0 tx=0x7fc33c00a640 comp rx=0 tx=0).stop 2023-11-28T18:29:56.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.197+0000 7fc34b1e6640 1 -- 172.21.15.105:0/121826844 shutdown_connections 2023-11-28T18:29:56.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.197+0000 7fc34b1e6640 1 --2- 172.21.15.105:0/121826844 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc3280668d0 0x7fc328068d90 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.197+0000 7fc34b1e6640 1 --2- 172.21.15.105:0/121826844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc34414d4a0 0x7fc3440ca630 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.197+0000 7fc34b1e6640 1 --2- 172.21.15.105:0/121826844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc34414cf50 0x7fc3440ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.197+0000 7fc34b1e6640 1 --2- 172.21.15.105:0/121826844 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3340047d0 0x7fc34414c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.197+0000 7fc34b1e6640 1 -- 172.21.15.105:0/121826844 >> 172.21.15.105:0/121826844 conn(0x7fc3440ceb90 msgr2=0x7fc3440ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:56.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.197+0000 7fc34b1e6640 1 -- 172.21.15.105:0/121826844 shutdown_connections 2023-11-28T18:29:56.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.197+0000 7fc34b1e6640 1 -- 172.21.15.105:0/121826844 wait complete. 2023-11-28T18:29:56.221 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_distributed 2023-11-28T18:29:56.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.361+0000 7f0d205f8640 1 Processor -- start 2023-11-28T18:29:56.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.361+0000 7f0d205f8640 1 -- start start 2023-11-28T18:29:56.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.361+0000 7f0d205f8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0d180b4290 0x7f0d180b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.361+0000 7f0d205f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1815b030 0x7f0d180709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.361+0000 7f0d205f8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d180b4bb0 con 0x7f0d18071960 2023-11-28T18:29:56.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.361+0000 7f0d205f8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d180701d0 con 0x7f0d180b4290 2023-11-28T18:29:56.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.361+0000 7f0d205f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1806df80 con 0x7f0d1815b030 2023-11-28T18:29:56.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.361+0000 7f0d1e394640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0d180b4290 0x7f0d180b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.365+0000 7f0d1e394640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0d180b4290 0x7f0d180b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44432/0 (socket says 172.21.15.105:44432) 2023-11-28T18:29:56.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.365+0000 7f0d1e394640 1 -- 172.21.15.105:0/17800918 learned_addr learned my addr 172.21.15.105:0/17800918 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:56.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.365+0000 7f0d1eb95640 1 --2- 172.21.15.105:0/17800918 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1815b030 0x7f0d180709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.365+0000 7f0d1e394640 1 -- 172.21.15.105:0/17800918 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1815b030 msgr2=0x7f0d180709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:56.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.365+0000 7f0d1e394640 1 --2- 172.21.15.105:0/17800918 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1815b030 0x7f0d180709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.365+0000 7f0d1e394640 1 -- 172.21.15.105:0/17800918 >> v1:172.21.15.148:6789/0 conn(0x7f0d18071960 legacy=0x7f0d1815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:56.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.365+0000 7f0d1e394640 1 -- 172.21.15.105:0/17800918 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d18071df0 con 0x7f0d180b4290 2023-11-28T18:29:56.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.365+0000 7f0d1e394640 1 --2- 172.21.15.105:0/17800918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d180b4290 0x7f0d180b4670 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009e50 tx=0x7f0d1401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8ae541c778623e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:56.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.365+0000 7f0d1d392640 1 -- 172.21.15.105:0/17800918 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d140a57c0 con 0x7f0d180b4290 2023-11-28T18:29:56.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.365+0000 7f0d1d392640 1 -- 172.21.15.105:0/17800918 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d140a5920 con 0x7f0d180b4290 2023-11-28T18:29:56.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.365+0000 7f0d1d392640 1 -- 172.21.15.105:0/17800918 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d140a5cb0 con 0x7f0d180b4290 2023-11-28T18:29:56.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d205f8640 1 -- 172.21.15.105:0/17800918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d180b4290 msgr2=0x7f0d180b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:56.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d205f8640 1 --2- 172.21.15.105:0/17800918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d180b4290 0x7f0d180b4670 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009e50 tx=0x7f0d1401c9e0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d205f8640 1 -- 172.21.15.105:0/17800918 shutdown_connections 2023-11-28T18:29:56.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d205f8640 1 --2- 172.21.15.105:0/17800918 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1815b030 0x7f0d180709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d205f8640 1 --2- 172.21.15.105:0/17800918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d180b4290 0x7f0d180b4670 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d205f8640 1 -- 172.21.15.105:0/17800918 >> 172.21.15.105:0/17800918 conn(0x7f0d180ceb90 msgr2=0x7f0d180cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:56.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d205f8640 1 -- 172.21.15.105:0/17800918 shutdown_connections 2023-11-28T18:29:56.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d205f8640 1 -- 172.21.15.105:0/17800918 wait complete. 2023-11-28T18:29:56.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d205f8640 1 Processor -- start 2023-11-28T18:29:56.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d205f8640 1 -- start start 2023-11-28T18:29:56.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d205f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d180b4290 0x7f0d181654c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d205f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d1815b030 0x7f0d180ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d205f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1815d3f0 0x7f0d18156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d205f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1815e9e0 con 0x7f0d1815b030 2023-11-28T18:29:56.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d205f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1815d940 con 0x7f0d180b4290 2023-11-28T18:29:56.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d205f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1815dc20 con 0x7f0d1815d3f0 2023-11-28T18:29:56.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d1db93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d1815b030 0x7f0d180ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d1db93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d1815b030 0x7f0d180ae230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44444/0 (socket says 172.21.15.105:44444) 2023-11-28T18:29:56.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d1db93640 1 -- 172.21.15.105:0/3966671673 learned_addr learned my addr 172.21.15.105:0/3966671673 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:56.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d1e394640 1 --2- 172.21.15.105:0/3966671673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d180b4290 0x7f0d181654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d1eb95640 1 --2- 172.21.15.105:0/3966671673 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1815d3f0 0x7f0d18156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d1db93640 1 -- 172.21.15.105:0/3966671673 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1815d3f0 msgr2=0x7f0d18156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:56.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d1db93640 1 --2- 172.21.15.105:0/3966671673 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1815d3f0 0x7f0d18156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d1db93640 1 -- 172.21.15.105:0/3966671673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d180b4290 msgr2=0x7f0d181654c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:56.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d1db93640 1 --2- 172.21.15.105:0/3966671673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d180b4290 0x7f0d181654c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d1db93640 1 -- 172.21.15.105:0/3966671673 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d14009b30 con 0x7f0d1815b030 2023-11-28T18:29:56.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.369+0000 7f0d1db93640 1 --2- 172.21.15.105:0/3966671673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d1815b030 0x7f0d180ae230 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800c4d0 tx=0x7f0d0800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:56.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.373+0000 7f0d0effd640 1 -- 172.21.15.105:0/3966671673 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d080052d0 con 0x7f0d1815b030 2023-11-28T18:29:56.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.373+0000 7f0d0effd640 1 -- 172.21.15.105:0/3966671673 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d08005430 con 0x7f0d1815b030 2023-11-28T18:29:56.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.373+0000 7f0d205f8640 1 -- 172.21.15.105:0/3966671673 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1815df00 con 0x7f0d1815b030 2023-11-28T18:29:56.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.373+0000 7f0d0effd640 1 -- 172.21.15.105:0/3966671673 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d08003800 con 0x7f0d1815b030 2023-11-28T18:29:56.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.373+0000 7f0d205f8640 1 -- 172.21.15.105:0/3966671673 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d1815e4c0 con 0x7f0d1815b030 2023-11-28T18:29:56.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.373+0000 7f0d205f8640 1 -- 172.21.15.105:0/3966671673 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d18067570 con 0x7f0d1815b030 2023-11-28T18:29:56.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.373+0000 7f0d0effd640 1 -- 172.21.15.105:0/3966671673 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f0d08003960 con 0x7f0d1815b030 2023-11-28T18:29:56.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.373+0000 7f0d0effd640 1 --2- 172.21.15.105:0/3966671673 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cfc066900 0x7f0cfc068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.373+0000 7f0d0effd640 1 -- 172.21.15.105:0/3966671673 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f0d080837f0 con 0x7f0d1815b030 2023-11-28T18:29:56.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.373+0000 7f0d1e394640 1 --2- 172.21.15.105:0/3966671673 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cfc066900 0x7f0cfc068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.373+0000 7f0d1e394640 1 --2- 172.21.15.105:0/3966671673 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cfc066900 0x7f0cfc068dc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0d140ae000 tx=0x7f0d140a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:56.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.377+0000 7f0d0effd640 1 -- 172.21.15.105:0/3966671673 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0d08050b50 con 0x7f0d1815b030 2023-11-28T18:29:56.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.529+0000 7f0d205f8640 1 -- 172.21.15.105:0/3966671673 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"} v 0) v1 -- 0x7f0d1815e680 con 0x7f0d1815b030 2023-11-28T18:29:56.538 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.533+0000 7f0d0effd640 1 -- 172.21.15.105:0/3966671673 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"}]=0 v49) v1 ==== 115+0+0 (secure 0 0 0) 0x7f0d08054170 con 0x7f0d1815b030 2023-11-28T18:29:56.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.537+0000 7f0d205f8640 1 -- 172.21.15.105:0/3966671673 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cfc066900 msgr2=0x7f0cfc068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:56.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.537+0000 7f0d205f8640 1 --2- 172.21.15.105:0/3966671673 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cfc066900 0x7f0cfc068dc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0d140ae000 tx=0x7f0d140a7070 comp rx=0 tx=0).stop 2023-11-28T18:29:56.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.537+0000 7f0d205f8640 1 -- 172.21.15.105:0/3966671673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d1815b030 msgr2=0x7f0d180ae230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:56.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.537+0000 7f0d205f8640 1 --2- 172.21.15.105:0/3966671673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d1815b030 0x7f0d180ae230 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800c4d0 tx=0x7f0d0800c8d0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.537+0000 7f0d205f8640 1 -- 172.21.15.105:0/3966671673 shutdown_connections 2023-11-28T18:29:56.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.537+0000 7f0d205f8640 1 --2- 172.21.15.105:0/3966671673 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1815d3f0 0x7f0d18156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.537+0000 7f0d205f8640 1 --2- 172.21.15.105:0/3966671673 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0cfc066900 0x7f0cfc068dc0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.537+0000 7f0d205f8640 1 --2- 172.21.15.105:0/3966671673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d1815b030 0x7f0d180ae230 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.537+0000 7f0d205f8640 1 --2- 172.21.15.105:0/3966671673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d180b4290 0x7f0d181654c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.537+0000 7f0d205f8640 1 -- 172.21.15.105:0/3966671673 >> 172.21.15.105:0/3966671673 conn(0x7f0d180ceb90 msgr2=0x7f0d1809ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:56.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.537+0000 7f0d205f8640 1 -- 172.21.15.105:0/3966671673 shutdown_connections 2023-11-28T18:29:56.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.537+0000 7f0d205f8640 1 -- 172.21.15.105:0/3966671673 wait complete. 2023-11-28T18:29:56.560 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random_max 2023-11-28T18:29:56.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f465b454640 1 Processor -- start 2023-11-28T18:29:56.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f465b454640 1 -- start start 2023-11-28T18:29:56.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f465b454640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f46540cc760 0x7f4654164d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f465b454640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46540ccc10 con 0x7f4654159f40 2023-11-28T18:29:56.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f465b454640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f465406bf50 con 0x7f46540caf60 2023-11-28T18:29:56.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f465b454640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46540cb3c0 con 0x7f46540cc760 2023-11-28T18:29:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f46591f0640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4654159f40 0x7f46540ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56734/0 (socket says 172.21.15.105:56734) 2023-11-28T18:29:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f46591f0640 1 -- 172.21.15.105:0/2100540346 learned_addr learned my addr 172.21.15.105:0/2100540346 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f46599f1640 1 --2- 172.21.15.105:0/2100540346 >> v2:172.21.15.148:3300/0 conn(0x7f46540cc760 0x7f4654164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f464bfff640 1 -- 172.21.15.105:0/2100540346 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068227471 0 0) 0x7f46540ccc10 con 0x7f4654159f40 2023-11-28T18:29:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f464bfff640 1 -- 172.21.15.105:0/2100540346 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4640003530 con 0x7f4654159f40 2023-11-28T18:29:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f464bfff640 1 -- 172.21.15.105:0/2100540346 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2790423987 0 0) 0x7f4640003530 con 0x7f4654159f40 2023-11-28T18:29:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f464bfff640 1 -- 172.21.15.105:0/2100540346 >> v2:172.21.15.148:3300/0 conn(0x7f46540cc760 msgr2=0x7f4654164d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f464bfff640 1 --2- 172.21.15.105:0/2100540346 >> v2:172.21.15.148:3300/0 conn(0x7f46540cc760 0x7f4654164d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f464bfff640 1 -- 172.21.15.105:0/2100540346 >> v1:172.21.15.148:6789/0 conn(0x7f46540caf60 legacy=0x7f46540ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f464bfff640 1 -- 172.21.15.105:0/2100540346 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465415a320 con 0x7f4654159f40 2023-11-28T18:29:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f464bfff640 1 -- 172.21.15.105:0/2100540346 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f4650001680 con 0x7f4654159f40 2023-11-28T18:29:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f464bfff640 1 -- 172.21.15.105:0/2100540346 >> v1:172.21.15.105:6789/0 conn(0x7f4654159f40 legacy=0x7f46540ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:56.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f464bfff640 1 --2- 172.21.15.105:0/2100540346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4640003980 0x7f4640003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f464bfff640 1 --2- 172.21.15.105:0/2100540346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46400047d0 0x7f4640004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f464bfff640 1 --2- 172.21.15.105:0/2100540346 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4640005190 0x7f464000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f464bfff640 1 -- 172.21.15.105:0/2100540346 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46540cb3c0 con 0x7f4640003980 2023-11-28T18:29:56.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f464bfff640 1 -- 172.21.15.105:0/2100540346 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464000c130 con 0x7f46400047d0 2023-11-28T18:29:56.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f464bfff640 1 -- 172.21.15.105:0/2100540346 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f464000c2f0 con 0x7f4640005190 2023-11-28T18:29:56.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f46591f0640 1 --2- 172.21.15.105:0/2100540346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4640003980 0x7f4640003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f46599f1640 1 --2- 172.21.15.105:0/2100540346 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4640005190 0x7f464000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f46589ef640 1 --2- 172.21.15.105:0/2100540346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46400047d0 0x7f4640004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f46591f0640 1 -- 172.21.15.105:0/2100540346 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4640005190 msgr2=0x7f464000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:56.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f46591f0640 1 --2- 172.21.15.105:0/2100540346 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4640005190 0x7f464000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f46591f0640 1 -- 172.21.15.105:0/2100540346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46400047d0 msgr2=0x7f4640004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:56.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f46591f0640 1 --2- 172.21.15.105:0/2100540346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46400047d0 0x7f4640004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f46591f0640 1 -- 172.21.15.105:0/2100540346 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4640003530 con 0x7f4640003980 2023-11-28T18:29:56.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.705+0000 7f46599f1640 1 -- 172.21.15.105:0/2100540346 reap_dead start 2023-11-28T18:29:56.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.709+0000 7f46591f0640 1 --2- 172.21.15.105:0/2100540346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4640003980 0x7f4640003da0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f465000c240 tx=0x7f465001cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca70d1c13ff5c4ee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:56.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.709+0000 7f464bfff640 1 -- 172.21.15.105:0/2100540346 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4650005ae0 con 0x7f4640003980 2023-11-28T18:29:56.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.709+0000 7f464bfff640 1 -- 172.21.15.105:0/2100540346 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4650005c40 con 0x7f4640003980 2023-11-28T18:29:56.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.709+0000 7f464bfff640 1 -- 172.21.15.105:0/2100540346 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46500072b0 con 0x7f4640003980 2023-11-28T18:29:56.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.709+0000 7f465b454640 1 -- 172.21.15.105:0/2100540346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4640003980 msgr2=0x7f4640003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:56.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.709+0000 7f465b454640 1 --2- 172.21.15.105:0/2100540346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4640003980 0x7f4640003da0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f465000c240 tx=0x7f465001cb90 comp rx=0 tx=0).stop 2023-11-28T18:29:56.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.709+0000 7f465b454640 1 -- 172.21.15.105:0/2100540346 shutdown_connections 2023-11-28T18:29:56.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.709+0000 7f465b454640 1 --2- 172.21.15.105:0/2100540346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4640003980 0x7f4640003da0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.709+0000 7f465b454640 1 -- 172.21.15.105:0/2100540346 >> 172.21.15.105:0/2100540346 conn(0x7f46540ceb90 msgr2=0x7f46540cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:56.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.709+0000 7f465b454640 1 -- 172.21.15.105:0/2100540346 shutdown_connections 2023-11-28T18:29:56.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.709+0000 7f465b454640 1 -- 172.21.15.105:0/2100540346 wait complete. 2023-11-28T18:29:56.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.709+0000 7f465b454640 1 Processor -- start 2023-11-28T18:29:56.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f465b454640 1 -- start start 2023-11-28T18:29:56.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f465b454640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4640003980 0x7f4654158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f465b454640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46540b03f0 0x7f465414b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f465b454640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46540c8b30 0x7f46540af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f465b454640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46541597c0 con 0x7f4640003980 2023-11-28T18:29:56.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f465b454640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46541594e0 con 0x7f46540b03f0 2023-11-28T18:29:56.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f465b454640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46540c8050 con 0x7f46540c8b30 2023-11-28T18:29:56.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f46591f0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4640003980 0x7f4654158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f46591f0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4640003980 0x7f4654158c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44464/0 (socket says 172.21.15.105:44464) 2023-11-28T18:29:56.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f46591f0640 1 -- 172.21.15.105:0/1264689081 learned_addr learned my addr 172.21.15.105:0/1264689081 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:56.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f46599f1640 1 --2- 172.21.15.105:0/1264689081 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46540c8b30 0x7f46540af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f46589ef640 1 --2- 172.21.15.105:0/1264689081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46540b03f0 0x7f465414b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f46591f0640 1 -- 172.21.15.105:0/1264689081 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46540c8b30 msgr2=0x7f46540af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:56.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f46591f0640 1 --2- 172.21.15.105:0/1264689081 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46540c8b30 0x7f46540af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f46591f0640 1 -- 172.21.15.105:0/1264689081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46540b03f0 msgr2=0x7f465414b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:56.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f46591f0640 1 --2- 172.21.15.105:0/1264689081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46540b03f0 0x7f465414b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f46591f0640 1 -- 172.21.15.105:0/1264689081 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46540c8330 con 0x7f4640003980 2023-11-28T18:29:56.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f46589ef640 1 --2- 172.21.15.105:0/1264689081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46540b03f0 0x7f465414b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:56.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f46591f0640 1 --2- 172.21.15.105:0/1264689081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4640003980 0x7f4654158c20 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f465001c9c0 tx=0x7f4640005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:56.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f4649ffb640 1 -- 172.21.15.105:0/1264689081 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46500b4070 con 0x7f4640003980 2023-11-28T18:29:56.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f4649ffb640 1 -- 172.21.15.105:0/1264689081 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4650001480 con 0x7f4640003980 2023-11-28T18:29:56.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f465b454640 1 -- 172.21.15.105:0/1264689081 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46540c8610 con 0x7f4640003980 2023-11-28T18:29:56.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f4649ffb640 1 -- 172.21.15.105:0/1264689081 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4650007270 con 0x7f4640003980 2023-11-28T18:29:56.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f465b454640 1 -- 172.21.15.105:0/1264689081 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465414b970 con 0x7f4640003980 2023-11-28T18:29:56.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.713+0000 7f465b454640 1 -- 172.21.15.105:0/1264689081 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4654067570 con 0x7f4640003980 2023-11-28T18:29:56.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.717+0000 7f4649ffb640 1 -- 172.21.15.105:0/1264689081 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f46500073d0 con 0x7f4640003980 2023-11-28T18:29:56.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.717+0000 7f4649ffb640 1 --2- 172.21.15.105:0/1264689081 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4638066900 0x7f4638068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:56.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.717+0000 7f4649ffb640 1 -- 172.21.15.105:0/1264689081 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f465011dfa0 con 0x7f4640003980 2023-11-28T18:29:56.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.717+0000 7f46589ef640 1 --2- 172.21.15.105:0/1264689081 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4638066900 0x7f4638068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:56.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.717+0000 7f46589ef640 1 --2- 172.21.15.105:0/1264689081 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4638066900 0x7f4638068dc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f46540b0890 tx=0x7f4644009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:56.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.721+0000 7f4649ffb640 1 -- 172.21.15.105:0/1264689081 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f46500eb2e0 con 0x7f4640003980 2023-11-28T18:29:56.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.873+0000 7f465b454640 1 -- 172.21.15.105:0/1264689081 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"} v 0) v1 -- 0x7f46540cf950 con 0x7f4640003980 2023-11-28T18:29:56.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.881+0000 7f4649ffb640 1 -- 172.21.15.105:0/1264689081 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"}]=0 v50) v1 ==== 114+0+0 (secure 0 0 0) 0x7f46500ac370 con 0x7f4640003980 2023-11-28T18:29:56.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.881+0000 7f465b454640 1 -- 172.21.15.105:0/1264689081 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4638066900 msgr2=0x7f4638068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:56.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.881+0000 7f465b454640 1 --2- 172.21.15.105:0/1264689081 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4638066900 0x7f4638068dc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f46540b0890 tx=0x7f4644009040 comp rx=0 tx=0).stop 2023-11-28T18:29:56.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.881+0000 7f465b454640 1 -- 172.21.15.105:0/1264689081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4640003980 msgr2=0x7f4654158c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:56.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.881+0000 7f465b454640 1 --2- 172.21.15.105:0/1264689081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4640003980 0x7f4654158c20 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f465001c9c0 tx=0x7f4640005800 comp rx=0 tx=0).stop 2023-11-28T18:29:56.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.881+0000 7f465b454640 1 -- 172.21.15.105:0/1264689081 shutdown_connections 2023-11-28T18:29:56.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.881+0000 7f465b454640 1 --2- 172.21.15.105:0/1264689081 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46540c8b30 0x7f46540af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.881+0000 7f465b454640 1 --2- 172.21.15.105:0/1264689081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46540b03f0 0x7f465414b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.881+0000 7f465b454640 1 --2- 172.21.15.105:0/1264689081 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4638066900 0x7f4638068dc0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.881+0000 7f465b454640 1 --2- 172.21.15.105:0/1264689081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4640003980 0x7f4654158c20 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:56.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.881+0000 7f465b454640 1 -- 172.21.15.105:0/1264689081 >> 172.21.15.105:0/1264689081 conn(0x7f46540ceb90 msgr2=0x7f46541558c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:56.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.881+0000 7f465b454640 1 -- 172.21.15.105:0/1264689081 shutdown_connections 2023-11-28T18:29:56.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:56.881+0000 7f465b454640 1 -- 172.21.15.105:0/1264689081 wait complete. 2023-11-28T18:29:56.902 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_after' 2023-11-28T18:29:57.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.037+0000 7f8b34f76640 1 Processor -- start 2023-11-28T18:29:57.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.037+0000 7f8b34f76640 1 -- start start 2023-11-28T18:29:57.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.037+0000 7f8b34f76640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8b3006c600 0x7f8b300cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:57.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.037+0000 7f8b34f76640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b3006b8a0 0x7f8b300cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:57.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.037+0000 7f8b34f76640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b300cb230 con 0x7f8b3015d1c0 2023-11-28T18:29:57.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.037+0000 7f8b34f76640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b300b02a0 con 0x7f8b3006c600 2023-11-28T18:29:57.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.037+0000 7f8b34f76640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b300d19a0 con 0x7f8b3006b8a0 2023-11-28T18:29:57.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.037+0000 7f8b27fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8b3006c600 0x7f8b300cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:57.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.037+0000 7f8b2f7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8b3015d1c0 0x7f8b3006bef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56738/0 (socket says 172.21.15.105:56738) 2023-11-28T18:29:57.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.037+0000 7f8b2f7fe640 1 -- 172.21.15.105:0/3767331450 learned_addr learned my addr 172.21.15.105:0/3767331450 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:57.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.037+0000 7f8b2ffff640 1 --2- 172.21.15.105:0/3767331450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b3006b8a0 0x7f8b300cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:57.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.037+0000 7f8b27fff640 1 -- 172.21.15.105:0/3767331450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b3006b8a0 msgr2=0x7f8b300cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:57.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.037+0000 7f8b27fff640 1 --2- 172.21.15.105:0/3767331450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b3006b8a0 0x7f8b300cbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:57.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.037+0000 7f8b27fff640 1 -- 172.21.15.105:0/3767331450 >> v1:172.21.15.105:6789/0 conn(0x7f8b3015d1c0 legacy=0x7f8b3006bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:57.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.037+0000 7f8b27fff640 1 -- 172.21.15.105:0/3767331450 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b300d0010 con 0x7f8b3006c600 2023-11-28T18:29:57.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.037+0000 7f8b2effd640 1 -- 172.21.15.105:0/3767331450 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799241115 0 0) 0x7f8b300cb230 con 0x7f8b3015d1c0 2023-11-28T18:29:57.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.041+0000 7f8b27fff640 1 --2- 172.21.15.105:0/3767331450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b3006c600 0x7f8b300cf910 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f8b180097c0 tx=0x7f8b1801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440eee0dd02304cf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:57.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.041+0000 7f8b2effd640 1 -- 172.21.15.105:0/3767331450 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b180ab5f0 con 0x7f8b3006c600 2023-11-28T18:29:57.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.041+0000 7f8b2effd640 1 -- 172.21.15.105:0/3767331450 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b180ab750 con 0x7f8b3006c600 2023-11-28T18:29:57.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.041+0000 7f8b2effd640 1 -- 172.21.15.105:0/3767331450 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b180abae0 con 0x7f8b3006c600 2023-11-28T18:29:57.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.041+0000 7f8b34f76640 1 -- 172.21.15.105:0/3767331450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b3006c600 msgr2=0x7f8b300cf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:57.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.041+0000 7f8b34f76640 1 --2- 172.21.15.105:0/3767331450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b3006c600 0x7f8b300cf910 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f8b180097c0 tx=0x7f8b1801c3f0 comp rx=0 tx=0).stop 2023-11-28T18:29:57.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.041+0000 7f8b34f76640 1 -- 172.21.15.105:0/3767331450 shutdown_connections 2023-11-28T18:29:57.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.041+0000 7f8b34f76640 1 --2- 172.21.15.105:0/3767331450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b3006b8a0 0x7f8b300cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:57.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.041+0000 7f8b34f76640 1 --2- 172.21.15.105:0/3767331450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b3006c600 0x7f8b300cf910 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:57.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.041+0000 7f8b34f76640 1 -- 172.21.15.105:0/3767331450 >> 172.21.15.105:0/3767331450 conn(0x7f8b300d0be0 msgr2=0x7f8b300d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:57.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.041+0000 7f8b34f76640 1 -- 172.21.15.105:0/3767331450 shutdown_connections 2023-11-28T18:29:57.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.041+0000 7f8b34f76640 1 -- 172.21.15.105:0/3767331450 wait complete. 2023-11-28T18:29:57.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b34f76640 1 Processor -- start 2023-11-28T18:29:57.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b34f76640 1 -- start start 2023-11-28T18:29:57.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b34f76640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b3006b8a0 0x7f8b30154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:57.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b34f76640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b3006c600 0x7f8b30153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:57.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b34f76640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b3015d1c0 0x7f8b3015b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:57.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b34f76640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b300afdc0 con 0x7f8b3006b8a0 2023-11-28T18:29:57.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b34f76640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3015d8c0 con 0x7f8b3006c600 2023-11-28T18:29:57.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b34f76640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b300ae500 con 0x7f8b3015d1c0 2023-11-28T18:29:57.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b2f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b3006b8a0 0x7f8b30154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:57.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b2f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b3006b8a0 0x7f8b30154d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44480/0 (socket says 172.21.15.105:44480) 2023-11-28T18:29:57.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b2f7fe640 1 -- 172.21.15.105:0/2298474130 learned_addr learned my addr 172.21.15.105:0/2298474130 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:57.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b27fff640 1 --2- 172.21.15.105:0/2298474130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b3006c600 0x7f8b30153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:57.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b2ffff640 1 --2- 172.21.15.105:0/2298474130 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b3015d1c0 0x7f8b3015b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:57.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b2f7fe640 1 -- 172.21.15.105:0/2298474130 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b3015d1c0 msgr2=0x7f8b3015b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:57.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b2f7fe640 1 --2- 172.21.15.105:0/2298474130 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b3015d1c0 0x7f8b3015b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:57.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b2f7fe640 1 -- 172.21.15.105:0/2298474130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b3006c600 msgr2=0x7f8b30153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:57.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b2f7fe640 1 --2- 172.21.15.105:0/2298474130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b3006c600 0x7f8b30153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:57.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b2f7fe640 1 -- 172.21.15.105:0/2298474130 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b18009420 con 0x7f8b3006b8a0 2023-11-28T18:29:57.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b2f7fe640 1 --2- 172.21.15.105:0/2298474130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b3006b8a0 0x7f8b30154d70 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f8b200011f0 tx=0x7f8b2000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:57.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b2cff9640 1 -- 172.21.15.105:0/2298474130 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b200074e0 con 0x7f8b3006b8a0 2023-11-28T18:29:57.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b2cff9640 1 -- 172.21.15.105:0/2298474130 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b20007640 con 0x7f8b3006b8a0 2023-11-28T18:29:57.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b2cff9640 1 -- 172.21.15.105:0/2298474130 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b20004a10 con 0x7f8b3006b8a0 2023-11-28T18:29:57.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b34f76640 1 -- 172.21.15.105:0/2298474130 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b300ae7e0 con 0x7f8b3006b8a0 2023-11-28T18:29:57.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b34f76640 1 -- 172.21.15.105:0/2298474130 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b300aeda0 con 0x7f8b3006b8a0 2023-11-28T18:29:57.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.045+0000 7f8b34f76640 1 -- 172.21.15.105:0/2298474130 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b30074580 con 0x7f8b3006b8a0 2023-11-28T18:29:57.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.049+0000 7f8b2cff9640 1 -- 172.21.15.105:0/2298474130 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8b20004b70 con 0x7f8b3006b8a0 2023-11-28T18:29:57.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.049+0000 7f8b2cff9640 1 --2- 172.21.15.105:0/2298474130 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b100668d0 0x7f8b10068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:57.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.049+0000 7f8b2cff9640 1 -- 172.21.15.105:0/2298474130 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f8b20083760 con 0x7f8b3006b8a0 2023-11-28T18:29:57.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.049+0000 7f8b27fff640 1 --2- 172.21.15.105:0/2298474130 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b100668d0 0x7f8b10068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:57.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.049+0000 7f8b27fff640 1 --2- 172.21.15.105:0/2298474130 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b100668d0 0x7f8b10068d90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8b180af000 tx=0x7f8b180a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:57.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.053+0000 7f8b2cff9640 1 -- 172.21.15.105:0/2298474130 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8b20050980 con 0x7f8b3006b8a0 2023-11-28T18:29:57.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.253+0000 7f8b34f76640 1 -- 172.21.15.105:0/2298474130 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_after"]} v 0) v1 -- 0x7f8b30067780 con 0x7f8b3006b8a0 2023-11-28T18:29:57.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.517+0000 7f8b2cff9640 1 -- 172.21.15.105:0/2298474130 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_after"]}]=0 v338) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8b20018070 con 0x7f8b3006b8a0 2023-11-28T18:29:57.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.521+0000 7f8b34f76640 1 -- 172.21.15.105:0/2298474130 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b100668d0 msgr2=0x7f8b10068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:57.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.521+0000 7f8b34f76640 1 --2- 172.21.15.105:0/2298474130 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b100668d0 0x7f8b10068d90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8b180af000 tx=0x7f8b180a7000 comp rx=0 tx=0).stop 2023-11-28T18:29:57.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.521+0000 7f8b34f76640 1 -- 172.21.15.105:0/2298474130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b3006b8a0 msgr2=0x7f8b30154d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:57.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.521+0000 7f8b34f76640 1 --2- 172.21.15.105:0/2298474130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b3006b8a0 0x7f8b30154d70 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f8b200011f0 tx=0x7f8b2000c9b0 comp rx=0 tx=0).stop 2023-11-28T18:29:57.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.521+0000 7f8b34f76640 1 -- 172.21.15.105:0/2298474130 shutdown_connections 2023-11-28T18:29:57.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.521+0000 7f8b34f76640 1 --2- 172.21.15.105:0/2298474130 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b3015d1c0 0x7f8b3015b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:57.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.521+0000 7f8b34f76640 1 --2- 172.21.15.105:0/2298474130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b3006c600 0x7f8b30153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:57.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.521+0000 7f8b34f76640 1 --2- 172.21.15.105:0/2298474130 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8b100668d0 0x7f8b10068d90 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:57.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.521+0000 7f8b34f76640 1 --2- 172.21.15.105:0/2298474130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b3006b8a0 0x7f8b30154d70 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:57.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.521+0000 7f8b34f76640 1 -- 172.21.15.105:0/2298474130 >> 172.21.15.105:0/2298474130 conn(0x7f8b300d0be0 msgr2=0x7f8b300b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:57.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.521+0000 7f8b34f76640 1 -- 172.21.15.105:0/2298474130 shutdown_connections 2023-11-28T18:29:57.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.521+0000 7f8b34f76640 1 -- 172.21.15.105:0/2298474130 wait complete. 2023-11-28T18:29:57.544 INFO:tasks.cephfs_test_runner:That a conventional export pin overrides the distributed policy _after_ distributed policy is set. ... ok 2023-11-28T18:29:57.545 INFO:tasks.cephfs_test_runner:Starting test: test_ephemeral_pin_dist_override_before (tasks.cephfs.test_exports.TestEphemeralPins) That a conventional export pin overrides the distributed policy _before_ distributed policy is set. 2023-11-28T18:29:57.545 INFO:tasks.cephfs_test_runner:test_ephemeral_pin_dist_override_before (tasks.cephfs.test_exports.TestEphemeralPins) 2023-11-28T18:29:57.545 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:29:57.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.681+0000 7fce6c0c9640 1 Processor -- start 2023-11-28T18:29:57.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.681+0000 7fce6c0c9640 1 -- start start 2023-11-28T18:29:57.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.681+0000 7fce6c0c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce6406d610 0x7fce640cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:57.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.681+0000 7fce6c0c9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fce640ce530 0x7fce640ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:57.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.681+0000 7fce6c0c9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce640ae380 con 0x7fce640adf30 2023-11-28T18:29:57.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.681+0000 7fce6c0c9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce64162820 con 0x7fce640ce530 2023-11-28T18:29:57.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.681+0000 7fce6c0c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce64161c10 con 0x7fce6406d610 2023-11-28T18:29:57.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.681+0000 7fce69664640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fce640ce530 0x7fce640ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:57.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.681+0000 7fce69664640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fce640ce530 0x7fce640ad9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44482/0 (socket says 172.21.15.105:44482) 2023-11-28T18:29:57.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.681+0000 7fce69664640 1 -- 172.21.15.105:0/4019088255 learned_addr learned my addr 172.21.15.105:0/4019088255 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:57.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.681+0000 7fce69e65640 1 --2- 172.21.15.105:0/4019088255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce6406d610 0x7fce640cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:57.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.681+0000 7fce69664640 1 -- 172.21.15.105:0/4019088255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce6406d610 msgr2=0x7fce640cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:57.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.681+0000 7fce69664640 1 --2- 172.21.15.105:0/4019088255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce6406d610 0x7fce640cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:57.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.681+0000 7fce69664640 1 -- 172.21.15.105:0/4019088255 >> v1:172.21.15.148:6789/0 conn(0x7fce640adf30 legacy=0x7fce641650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:57.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.681+0000 7fce69664640 1 -- 172.21.15.105:0/4019088255 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce64161ef0 con 0x7fce640ce530 2023-11-28T18:29:57.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.681+0000 7fce69e65640 1 --2- 172.21.15.105:0/4019088255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce6406d610 0x7fce640cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:57.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.681+0000 7fce69664640 1 --2- 172.21.15.105:0/4019088255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce640ce530 0x7fce640ad9f0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fce54009870 tx=0x7fce5401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2bc41129fc1085 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:57.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.681+0000 7fce68e63640 1 -- 172.21.15.105:0/4019088255 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce540ac410 con 0x7fce640ce530 2023-11-28T18:29:57.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.681+0000 7fce68e63640 1 -- 172.21.15.105:0/4019088255 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce540ac570 con 0x7fce640ce530 2023-11-28T18:29:57.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.681+0000 7fce68e63640 1 -- 172.21.15.105:0/4019088255 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce540ac900 con 0x7fce640ce530 2023-11-28T18:29:57.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.685+0000 7fce6c0c9640 1 -- 172.21.15.105:0/4019088255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce640ce530 msgr2=0x7fce640ad9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:57.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.685+0000 7fce6c0c9640 1 --2- 172.21.15.105:0/4019088255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce640ce530 0x7fce640ad9f0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fce54009870 tx=0x7fce5401c4d0 comp rx=0 tx=0).stop 2023-11-28T18:29:57.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.685+0000 7fce6c0c9640 1 -- 172.21.15.105:0/4019088255 shutdown_connections 2023-11-28T18:29:57.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.685+0000 7fce6c0c9640 1 --2- 172.21.15.105:0/4019088255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce640ce530 0x7fce640ad9f0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:57.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.685+0000 7fce6c0c9640 1 --2- 172.21.15.105:0/4019088255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce6406d610 0x7fce640cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:57.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.685+0000 7fce6c0c9640 1 -- 172.21.15.105:0/4019088255 >> 172.21.15.105:0/4019088255 conn(0x7fce640ceb90 msgr2=0x7fce640cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:57.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.685+0000 7fce6c0c9640 1 -- 172.21.15.105:0/4019088255 shutdown_connections 2023-11-28T18:29:57.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.685+0000 7fce6c0c9640 1 -- 172.21.15.105:0/4019088255 wait complete. 2023-11-28T18:29:57.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.685+0000 7fce6c0c9640 1 Processor -- start 2023-11-28T18:29:57.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.685+0000 7fce6c0c9640 1 -- start start 2023-11-28T18:29:57.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.685+0000 7fce6c0c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce6406d610 0x7fce641608d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:57.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.685+0000 7fce6c0c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce640adf30 0x7fce640afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:57.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.685+0000 7fce6c0c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce640ce530 0x7fce640b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:57.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.685+0000 7fce6c0c9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce641512c0 con 0x7fce640ce530 2023-11-28T18:29:57.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.685+0000 7fce6c0c9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce64160e10 con 0x7fce640adf30 2023-11-28T18:29:57.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.685+0000 7fce6c0c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6414ff10 con 0x7fce6406d610 2023-11-28T18:29:57.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.685+0000 7fce6a666640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce640ce530 0x7fce640b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:57.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.685+0000 7fce6a666640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce640ce530 0x7fce640b0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44492/0 (socket says 172.21.15.105:44492) 2023-11-28T18:29:57.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.685+0000 7fce6a666640 1 -- 172.21.15.105:0/1776242942 learned_addr learned my addr 172.21.15.105:0/1776242942 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:57.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.685+0000 7fce69e65640 1 --2- 172.21.15.105:0/1776242942 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce6406d610 0x7fce641608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:57.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.685+0000 7fce69664640 1 --2- 172.21.15.105:0/1776242942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce640adf30 0x7fce640afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:57.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.689+0000 7fce6a666640 1 -- 172.21.15.105:0/1776242942 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce6406d610 msgr2=0x7fce641608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:57.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.689+0000 7fce6a666640 1 --2- 172.21.15.105:0/1776242942 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce6406d610 0x7fce641608d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:57.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.689+0000 7fce6a666640 1 -- 172.21.15.105:0/1776242942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce640adf30 msgr2=0x7fce640afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:57.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.689+0000 7fce6a666640 1 --2- 172.21.15.105:0/1776242942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce640adf30 0x7fce640afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:57.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.689+0000 7fce6a666640 1 -- 172.21.15.105:0/1776242942 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce540094d0 con 0x7fce640ce530 2023-11-28T18:29:57.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.689+0000 7fce6a666640 1 --2- 172.21.15.105:0/1776242942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce640ce530 0x7fce640b0120 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00c780 tx=0x7fce5c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:57.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.689+0000 7fce5a7fc640 1 -- 172.21.15.105:0/1776242942 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c004b70 con 0x7fce640ce530 2023-11-28T18:29:57.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.689+0000 7fce5a7fc640 1 -- 172.21.15.105:0/1776242942 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5c00ce90 con 0x7fce640ce530 2023-11-28T18:29:57.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.689+0000 7fce5a7fc640 1 -- 172.21.15.105:0/1776242942 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c00a850 con 0x7fce640ce530 2023-11-28T18:29:57.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.689+0000 7fce6c0c9640 1 -- 172.21.15.105:0/1776242942 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce641501f0 con 0x7fce640ce530 2023-11-28T18:29:57.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.689+0000 7fce6c0c9640 1 -- 172.21.15.105:0/1776242942 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce641507b0 con 0x7fce640ce530 2023-11-28T18:29:57.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.689+0000 7fce6c0c9640 1 -- 172.21.15.105:0/1776242942 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce64067570 con 0x7fce640ce530 2023-11-28T18:29:57.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.689+0000 7fce5a7fc640 1 -- 172.21.15.105:0/1776242942 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fce5c00a9b0 con 0x7fce640ce530 2023-11-28T18:29:57.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.689+0000 7fce5a7fc640 1 --2- 172.21.15.105:0/1776242942 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fce48066900 0x7fce48068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:57.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.689+0000 7fce5a7fc640 1 -- 172.21.15.105:0/1776242942 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fce5c084410 con 0x7fce640ce530 2023-11-28T18:29:57.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.689+0000 7fce69e65640 1 --2- 172.21.15.105:0/1776242942 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fce48066900 0x7fce48068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:57.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.693+0000 7fce69e65640 1 --2- 172.21.15.105:0/1776242942 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fce48066900 0x7fce48068dc0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fce6000ab60 tx=0x7fce60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:57.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.693+0000 7fce5a7fc640 1 -- 172.21.15.105:0/1776242942 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fce5c051770 con 0x7fce640ce530 2023-11-28T18:29:57.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.845+0000 7fce6c0c9640 1 -- 172.21.15.105:0/1776242942 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce641514e0 con 0x7fce640ce530 2023-11-28T18:29:57.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.845+0000 7fce5a7fc640 1 -- 172.21.15.105:0/1776242942 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+13518 (secure 0 0 0) 0x7fce5c054d90 con 0x7fce640ce530 2023-11-28T18:29:57.850 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:29:57.850 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":95,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:29:55.516853+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6835/3461832163":"2023-11-29T18:29:52.506853+0000","172.21.15.148:6833/3461832163":"2023-11-29T18:29:52.506853+0000","172.21.15.105:6837/3239728352":"2023-11-29T18:29:52.506879+0000","172.21.15.105:6836/3239728352":"2023-11-29T18:29:52.506879+0000","172.21.15.105:6835/4162499249":"2023-11-29T18:29:52.506871+0000","172.21.15.105:6834/4162499249":"2023-11-29T18:29:52.506871+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:29:57.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.845+0000 7fce6c0c9640 1 -- 172.21.15.105:0/1776242942 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fce48066900 msgr2=0x7fce48068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:57.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.845+0000 7fce6c0c9640 1 --2- 172.21.15.105:0/1776242942 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fce48066900 0x7fce48068dc0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fce6000ab60 tx=0x7fce60008040 comp rx=0 tx=0).stop 2023-11-28T18:29:57.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.845+0000 7fce6c0c9640 1 -- 172.21.15.105:0/1776242942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce640ce530 msgr2=0x7fce640b0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:57.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.845+0000 7fce6c0c9640 1 --2- 172.21.15.105:0/1776242942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce640ce530 0x7fce640b0120 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00c780 tx=0x7fce5c00cc40 comp rx=0 tx=0).stop 2023-11-28T18:29:57.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.845+0000 7fce6c0c9640 1 -- 172.21.15.105:0/1776242942 shutdown_connections 2023-11-28T18:29:57.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.845+0000 7fce6c0c9640 1 --2- 172.21.15.105:0/1776242942 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fce48066900 0x7fce48068dc0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:57.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.845+0000 7fce6c0c9640 1 --2- 172.21.15.105:0/1776242942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce640ce530 0x7fce640b0120 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:57.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.845+0000 7fce6c0c9640 1 --2- 172.21.15.105:0/1776242942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce640adf30 0x7fce640afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:57.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.845+0000 7fce6c0c9640 1 --2- 172.21.15.105:0/1776242942 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce6406d610 0x7fce641608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:57.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.845+0000 7fce6c0c9640 1 -- 172.21.15.105:0/1776242942 >> 172.21.15.105:0/1776242942 conn(0x7fce640ceb90 msgr2=0x7fce640aeaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:57.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.845+0000 7fce6c0c9640 1 -- 172.21.15.105:0/1776242942 shutdown_connections 2023-11-28T18:29:57.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:57.845+0000 7fce6c0c9640 1 -- 172.21.15.105:0/1776242942 wait complete. 2023-11-28T18:29:57.871 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:29:57.872 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:29:58.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdeffb8640 1 Processor -- start 2023-11-28T18:29:58.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdeffb8640 1 -- start start 2023-11-28T18:29:58.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdeffb8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcde815d4f0 0x7fcde80d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:58.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdeffb8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde80b25b0 con 0x7fcde80b2d20 2023-11-28T18:29:58.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdeffb8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde80cea30 con 0x7fcde80d3c30 2023-11-28T18:29:58.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdeffb8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde80b2900 con 0x7fcde815d4f0 2023-11-28T18:29:58.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdedd54640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcde815d4f0 0x7fcde80d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:58.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdedd54640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcde815d4f0 0x7fcde80d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44498/0 (socket says 172.21.15.105:44498) 2023-11-28T18:29:58.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdee555640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fcde80b2d20 0x7fcde80b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56742/0 (socket says 172.21.15.105:56742) 2023-11-28T18:29:58.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdedd54640 1 -- 172.21.15.105:0/449746209 learned_addr learned my addr 172.21.15.105:0/449746209 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:58.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdedd54640 1 -- 172.21.15.105:0/449746209 >> v1:172.21.15.148:6789/0 conn(0x7fcde80d3c30 legacy=0x7fcde80d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:58.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdedd54640 1 -- 172.21.15.105:0/449746209 >> v1:172.21.15.105:6789/0 conn(0x7fcde80b2d20 legacy=0x7fcde80b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:58.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdecd52640 1 -- 172.21.15.105:0/449746209 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512190335 0 0) 0x7fcde80b25b0 con 0x7fcde80b2d20 2023-11-28T18:29:58.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdedd54640 1 -- 172.21.15.105:0/449746209 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde80ced10 con 0x7fcde815d4f0 2023-11-28T18:29:58.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdedd54640 1 --2- 172.21.15.105:0/449746209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde815d4f0 0x7fcde80d0e10 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009870 tx=0x7fcde401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee371d8c90113699 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:58.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdecd52640 1 -- 172.21.15.105:0/449746209 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde40ab620 con 0x7fcde815d4f0 2023-11-28T18:29:58.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdecd52640 1 -- 172.21.15.105:0/449746209 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde40ab780 con 0x7fcde815d4f0 2023-11-28T18:29:58.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdecd52640 1 -- 172.21.15.105:0/449746209 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde40abb10 con 0x7fcde815d4f0 2023-11-28T18:29:58.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdeffb8640 1 -- 172.21.15.105:0/449746209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde815d4f0 msgr2=0x7fcde80d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:58.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdeffb8640 1 --2- 172.21.15.105:0/449746209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde815d4f0 0x7fcde80d0e10 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009870 tx=0x7fcde401c480 comp rx=0 tx=0).stop 2023-11-28T18:29:58.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdeffb8640 1 -- 172.21.15.105:0/449746209 shutdown_connections 2023-11-28T18:29:58.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdeffb8640 1 --2- 172.21.15.105:0/449746209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde815d4f0 0x7fcde80d0e10 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:58.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdeffb8640 1 -- 172.21.15.105:0/449746209 >> 172.21.15.105:0/449746209 conn(0x7fcde80d3030 msgr2=0x7fcde80d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:58.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdeffb8640 1 -- 172.21.15.105:0/449746209 shutdown_connections 2023-11-28T18:29:58.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.013+0000 7fcdeffb8640 1 -- 172.21.15.105:0/449746209 wait complete. 2023-11-28T18:29:58.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdeffb8640 1 Processor -- start 2023-11-28T18:29:58.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdeffb8640 1 -- start start 2023-11-28T18:29:58.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdeffb8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde80b2d20 0x7fcde8150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:58.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdeffb8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcde80d3c30 0x7fcde81f9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:58.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdeffb8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcde815d4f0 0x7fcde815f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:58.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdeffb8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80d0a20 con 0x7fcde80b2d20 2023-11-28T18:29:58.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdeffb8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde81f9b40 con 0x7fcde815d4f0 2023-11-28T18:29:58.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdeffb8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80cc0c0 con 0x7fcde80d3c30 2023-11-28T18:29:58.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdedd54640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde80b2d20 0x7fcde8150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:58.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdedd54640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde80b2d20 0x7fcde8150e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44508/0 (socket says 172.21.15.105:44508) 2023-11-28T18:29:58.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdedd54640 1 -- 172.21.15.105:0/862932878 learned_addr learned my addr 172.21.15.105:0/862932878 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:58.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdee555640 1 --2- 172.21.15.105:0/862932878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcde815d4f0 0x7fcde815f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:58.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcded553640 1 --2- 172.21.15.105:0/862932878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcde80d3c30 0x7fcde81f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:58.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdedd54640 1 -- 172.21.15.105:0/862932878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcde80d3c30 msgr2=0x7fcde81f9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:58.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdedd54640 1 --2- 172.21.15.105:0/862932878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcde80d3c30 0x7fcde81f9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:58.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdedd54640 1 -- 172.21.15.105:0/862932878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcde815d4f0 msgr2=0x7fcde815f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:58.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdedd54640 1 --2- 172.21.15.105:0/862932878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcde815d4f0 0x7fcde815f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:58.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdedd54640 1 -- 172.21.15.105:0/862932878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde40094d0 con 0x7fcde80b2d20 2023-11-28T18:29:58.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdedd54640 1 --2- 172.21.15.105:0/862932878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde80b2d20 0x7fcde8150e20 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fcde40ad040 tx=0x7fcde401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:58.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdde7fc640 1 -- 172.21.15.105:0/862932878 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde40abdc0 con 0x7fcde80b2d20 2023-11-28T18:29:58.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdde7fc640 1 -- 172.21.15.105:0/862932878 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde40b8070 con 0x7fcde80b2d20 2023-11-28T18:29:58.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdde7fc640 1 -- 172.21.15.105:0/862932878 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4004230 con 0x7fcde80b2d20 2023-11-28T18:29:58.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdeffb8640 1 -- 172.21.15.105:0/862932878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde80ca100 con 0x7fcde80b2d20 2023-11-28T18:29:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.017+0000 7fcdeffb8640 1 -- 172.21.15.105:0/862932878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde80cc5f0 con 0x7fcde80b2d20 2023-11-28T18:29:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.021+0000 7fcdeffb8640 1 -- 172.21.15.105:0/862932878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde8067570 con 0x7fcde80b2d20 2023-11-28T18:29:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.021+0000 7fcdde7fc640 1 -- 172.21.15.105:0/862932878 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fcde4004390 con 0x7fcde80b2d20 2023-11-28T18:29:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.021+0000 7fcdde7fc640 1 --2- 172.21.15.105:0/862932878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcdcc0668d0 0x7fcdcc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.021+0000 7fcdde7fc640 1 -- 172.21.15.105:0/862932878 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fcde411e880 con 0x7fcde80b2d20 2023-11-28T18:29:58.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.021+0000 7fcded553640 1 --2- 172.21.15.105:0/862932878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcdcc0668d0 0x7fcdcc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:58.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.021+0000 7fcded553640 1 --2- 172.21.15.105:0/862932878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcdcc0668d0 0x7fcdcc068d90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcde806b890 tx=0x7fcdd8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:58.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.025+0000 7fcdde7fc640 1 -- 172.21.15.105:0/862932878 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcde40ebb30 con 0x7fcde80b2d20 2023-11-28T18:29:58.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.177+0000 7fcdeffb8640 1 -- 172.21.15.105:0/862932878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcde80cc7e0 con 0x7fcde80b2d20 2023-11-28T18:29:58.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.177+0000 7fcdde7fc640 1 -- 172.21.15.105:0/862932878 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcde40ef150 con 0x7fcde80b2d20 2023-11-28T18:29:58.183 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:29:58.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.181+0000 7fcdeffb8640 1 -- 172.21.15.105:0/862932878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcdcc0668d0 msgr2=0x7fcdcc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:58.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.181+0000 7fcdeffb8640 1 --2- 172.21.15.105:0/862932878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcdcc0668d0 0x7fcdcc068d90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcde806b890 tx=0x7fcdd8009040 comp rx=0 tx=0).stop 2023-11-28T18:29:58.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.181+0000 7fcdeffb8640 1 -- 172.21.15.105:0/862932878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde80b2d20 msgr2=0x7fcde8150e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:58.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.181+0000 7fcdeffb8640 1 --2- 172.21.15.105:0/862932878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde80b2d20 0x7fcde8150e20 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fcde40ad040 tx=0x7fcde401acc0 comp rx=0 tx=0).stop 2023-11-28T18:29:58.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.181+0000 7fcdeffb8640 1 -- 172.21.15.105:0/862932878 shutdown_connections 2023-11-28T18:29:58.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.181+0000 7fcdeffb8640 1 --2- 172.21.15.105:0/862932878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcde815d4f0 0x7fcde815f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:58.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.181+0000 7fcdeffb8640 1 --2- 172.21.15.105:0/862932878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcde80d3c30 0x7fcde81f9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:58.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.181+0000 7fcdeffb8640 1 --2- 172.21.15.105:0/862932878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fcdcc0668d0 0x7fcdcc068d90 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:58.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.181+0000 7fcdeffb8640 1 --2- 172.21.15.105:0/862932878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde80b2d20 0x7fcde8150e20 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:58.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.181+0000 7fcdeffb8640 1 -- 172.21.15.105:0/862932878 >> 172.21.15.105:0/862932878 conn(0x7fcde80d3030 msgr2=0x7fcde8154d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:58.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.181+0000 7fcdeffb8640 1 -- 172.21.15.105:0/862932878 shutdown_connections 2023-11-28T18:29:58.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.181+0000 7fcdeffb8640 1 -- 172.21.15.105:0/862932878 wait complete. 2023-11-28T18:29:58.204 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_before' 2023-11-28T18:29:58.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.341+0000 7f685609f640 1 Processor -- start 2023-11-28T18:29:58.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.341+0000 7f685609f640 1 -- start start 2023-11-28T18:29:58.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.341+0000 7f685609f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f685015d2d0 0x7f68500cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:58.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.341+0000 7f685609f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f68500cf960 0x7f6850166fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:58.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.341+0000 7f685609f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68500ce840 con 0x7f685015cef0 2023-11-28T18:29:58.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.341+0000 7f685609f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68500cfe10 con 0x7f68500cf960 2023-11-28T18:29:58.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.341+0000 7f685609f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68500b4780 con 0x7f685015d2d0 2023-11-28T18:29:58.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.341+0000 7f685509d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f685015cef0 0x7f68500d1600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56748/0 (socket says 172.21.15.105:56748) 2023-11-28T18:29:58.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.341+0000 7f685509d640 1 -- 172.21.15.105:0/315136391 learned_addr learned my addr 172.21.15.105:0/315136391 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:58.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.341+0000 7f685589e640 1 --2- 172.21.15.105:0/315136391 >> v2:172.21.15.105:3300/0 conn(0x7f68500cf960 0x7f6850166fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:58.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.341+0000 7f685489c640 1 --2- 172.21.15.105:0/315136391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f685015d2d0 0x7f68500cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:58.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.341+0000 7f6847fff640 1 -- 172.21.15.105:0/315136391 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2424943054 0 0) 0x7f68500ce840 con 0x7f685015cef0 2023-11-28T18:29:58.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.341+0000 7f6847fff640 1 -- 172.21.15.105:0/315136391 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f683c003530 con 0x7f685015cef0 2023-11-28T18:29:58.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.341+0000 7f685589e640 1 -- 172.21.15.105:0/315136391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f685015d2d0 msgr2=0x7f68500cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:58.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.341+0000 7f685589e640 1 --2- 172.21.15.105:0/315136391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f685015d2d0 0x7f68500cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:58.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.341+0000 7f685589e640 1 -- 172.21.15.105:0/315136391 >> v1:172.21.15.105:6789/0 conn(0x7f685015cef0 legacy=0x7f68500d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:58.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.341+0000 7f685589e640 1 -- 172.21.15.105:0/315136391 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68500cd990 con 0x7f68500cf960 2023-11-28T18:29:58.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.341+0000 7f685589e640 1 --2- 172.21.15.105:0/315136391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68500cf960 0x7f6850166fd0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f6848009d90 tx=0x7f684801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcdf4000564c972a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:58.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.341+0000 7f6847fff640 1 -- 172.21.15.105:0/315136391 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68480a5960 con 0x7f68500cf960 2023-11-28T18:29:58.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.341+0000 7f6847fff640 1 -- 172.21.15.105:0/315136391 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68480a5ac0 con 0x7f68500cf960 2023-11-28T18:29:58.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.341+0000 7f6847fff640 1 -- 172.21.15.105:0/315136391 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68480a5e50 con 0x7f68500cf960 2023-11-28T18:29:58.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685609f640 1 -- 172.21.15.105:0/315136391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68500cf960 msgr2=0x7f6850166fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:58.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685609f640 1 --2- 172.21.15.105:0/315136391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68500cf960 0x7f6850166fd0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f6848009d90 tx=0x7f684801c8a0 comp rx=0 tx=0).stop 2023-11-28T18:29:58.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685609f640 1 -- 172.21.15.105:0/315136391 shutdown_connections 2023-11-28T18:29:58.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685609f640 1 --2- 172.21.15.105:0/315136391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68500cf960 0x7f6850166fd0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:58.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685609f640 1 --2- 172.21.15.105:0/315136391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f685015d2d0 0x7f68500cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:58.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685609f640 1 -- 172.21.15.105:0/315136391 >> 172.21.15.105:0/315136391 conn(0x7f68500d2e10 msgr2=0x7f68500d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:58.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685609f640 1 -- 172.21.15.105:0/315136391 shutdown_connections 2023-11-28T18:29:58.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685609f640 1 -- 172.21.15.105:0/315136391 wait complete. 2023-11-28T18:29:58.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685609f640 1 Processor -- start 2023-11-28T18:29:58.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685609f640 1 -- start start 2023-11-28T18:29:58.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685609f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68500cf960 0x7f6850153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:58.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685609f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f685015cef0 0x7f68500ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:58.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685609f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f685015d2d0 0x7f68500cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:58.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685609f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68500cce60 con 0x7f68500cf960 2023-11-28T18:29:58.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685609f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68500b3900 con 0x7f685015cef0 2023-11-28T18:29:58.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685609f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68500ccb80 con 0x7f685015d2d0 2023-11-28T18:29:58.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685509d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68500cf960 0x7f6850153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:58.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685509d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68500cf960 0x7f6850153f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44530/0 (socket says 172.21.15.105:44530) 2023-11-28T18:29:58.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685509d640 1 -- 172.21.15.105:0/1505132101 learned_addr learned my addr 172.21.15.105:0/1505132101 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:58.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685589e640 1 --2- 172.21.15.105:0/1505132101 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f685015d2d0 0x7f68500cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:58.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685489c640 1 --2- 172.21.15.105:0/1505132101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f685015cef0 0x7f68500ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:58.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685509d640 1 -- 172.21.15.105:0/1505132101 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f685015d2d0 msgr2=0x7f68500cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:58.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685509d640 1 --2- 172.21.15.105:0/1505132101 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f685015d2d0 0x7f68500cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:58.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685509d640 1 -- 172.21.15.105:0/1505132101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f685015cef0 msgr2=0x7f68500ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:58.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685509d640 1 --2- 172.21.15.105:0/1505132101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f685015cef0 0x7f68500ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:58.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685509d640 1 -- 172.21.15.105:0/1505132101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6848009a70 con 0x7f68500cf960 2023-11-28T18:29:58.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685489c640 1 --2- 172.21.15.105:0/1505132101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f685015cef0 0x7f68500ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:58.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.345+0000 7f685589e640 1 --2- 172.21.15.105:0/1505132101 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f685015d2d0 0x7f68500cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:29:58.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.349+0000 7f685509d640 1 --2- 172.21.15.105:0/1505132101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68500cf960 0x7f6850153f20 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f684c00c450 tx=0x7f684c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.349+0000 7f6845ffb640 1 -- 172.21.15.105:0/1505132101 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c005cf0 con 0x7f68500cf960 2023-11-28T18:29:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.349+0000 7f6845ffb640 1 -- 172.21.15.105:0/1505132101 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684c005e50 con 0x7f68500cf960 2023-11-28T18:29:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.349+0000 7f6845ffb640 1 -- 172.21.15.105:0/1505132101 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c007290 con 0x7f68500cf960 2023-11-28T18:29:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.349+0000 7f685609f640 1 -- 172.21.15.105:0/1505132101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68500b32b0 con 0x7f68500cf960 2023-11-28T18:29:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.349+0000 7f685609f640 1 -- 172.21.15.105:0/1505132101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68500cc830 con 0x7f68500cf960 2023-11-28T18:29:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.349+0000 7f685609f640 1 -- 172.21.15.105:0/1505132101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68500d03c0 con 0x7f68500cf960 2023-11-28T18:29:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.349+0000 7f6845ffb640 1 -- 172.21.15.105:0/1505132101 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f684c0073f0 con 0x7f68500cf960 2023-11-28T18:29:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.349+0000 7f6845ffb640 1 --2- 172.21.15.105:0/1505132101 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f68340668d0 0x7f6834068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.349+0000 7f6845ffb640 1 -- 172.21.15.105:0/1505132101 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f684c083920 con 0x7f68500cf960 2023-11-28T18:29:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.349+0000 7f685489c640 1 --2- 172.21.15.105:0/1505132101 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f68340668d0 0x7f6834068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.353+0000 7f685489c640 1 --2- 172.21.15.105:0/1505132101 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f68340668d0 0x7f6834068d90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f68500d01e0 tx=0x7f6840008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:58.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.353+0000 7f6845ffb640 1 -- 172.21.15.105:0/1505132101 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f684c050c60 con 0x7f68500cf960 2023-11-28T18:29:58.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:58.561+0000 7f685609f640 1 -- 172.21.15.105:0/1505132101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_before"]} v 0) v1 -- 0x7f68500b0450 con 0x7f68500cf960 2023-11-28T18:29:59.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.533+0000 7f6845ffb640 1 -- 172.21.15.105:0/1505132101 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_before"]}]=0 v340) v1 ==== 165+0+0 (secure 0 0 0) 0x7f684c017070 con 0x7f68500cf960 2023-11-28T18:29:59.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.537+0000 7f685609f640 1 -- 172.21.15.105:0/1505132101 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f68340668d0 msgr2=0x7f6834068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:59.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.537+0000 7f685609f640 1 --2- 172.21.15.105:0/1505132101 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f68340668d0 0x7f6834068d90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f68500d01e0 tx=0x7f6840008040 comp rx=0 tx=0).stop 2023-11-28T18:29:59.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.537+0000 7f685609f640 1 -- 172.21.15.105:0/1505132101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68500cf960 msgr2=0x7f6850153f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:59.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.537+0000 7f685609f640 1 --2- 172.21.15.105:0/1505132101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68500cf960 0x7f6850153f20 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f684c00c450 tx=0x7f684c00c910 comp rx=0 tx=0).stop 2023-11-28T18:29:59.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.537+0000 7f685609f640 1 -- 172.21.15.105:0/1505132101 shutdown_connections 2023-11-28T18:29:59.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.537+0000 7f685609f640 1 --2- 172.21.15.105:0/1505132101 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f685015d2d0 0x7f68500cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:59.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.537+0000 7f685609f640 1 --2- 172.21.15.105:0/1505132101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f685015cef0 0x7f68500ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:59.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.537+0000 7f685609f640 1 --2- 172.21.15.105:0/1505132101 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f68340668d0 0x7f6834068d90 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:59.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.537+0000 7f685609f640 1 --2- 172.21.15.105:0/1505132101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68500cf960 0x7f6850153f20 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:59.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.537+0000 7f685609f640 1 -- 172.21.15.105:0/1505132101 >> 172.21.15.105:0/1505132101 conn(0x7f68500d2e10 msgr2=0x7f685006bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:59.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.537+0000 7f685609f640 1 -- 172.21.15.105:0/1505132101 shutdown_connections 2023-11-28T18:29:59.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.537+0000 7f685609f640 1 -- 172.21.15.105:0/1505132101 wait complete. 2023-11-28T18:29:59.684 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T18:29:59.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.829+0000 7fb582674640 1 Processor -- start 2023-11-28T18:29:59.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.829+0000 7fb582674640 1 -- start start 2023-11-28T18:29:59.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.833+0000 7fb582674640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb57c0cd170 0x7fb57c0cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:59.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.833+0000 7fb582674640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb57c06d070 con 0x7fb57c0ce170 2023-11-28T18:29:59.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.833+0000 7fb582674640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb57c06cab0 con 0x7fb57c159710 2023-11-28T18:29:59.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.833+0000 7fb582674640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb57c15e050 con 0x7fb57c0cd170 2023-11-28T18:29:59.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.833+0000 7fb580e71640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb57c0ce170 0x7fb57c15d940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56762/0 (socket says 172.21.15.105:56762) 2023-11-28T18:29:59.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.833+0000 7fb580e71640 1 -- 172.21.15.105:0/2316479610 learned_addr learned my addr 172.21.15.105:0/2316479610 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:29:59.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.833+0000 7fb581e73640 1 --2- 172.21.15.105:0/2316479610 >> v2:172.21.15.105:3300/0 conn(0x7fb57c0cd170 0x7fb57c0cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:59.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.833+0000 7fb581e73640 1 -- 172.21.15.105:0/2316479610 >> v1:172.21.15.148:6789/0 conn(0x7fb57c159710 legacy=0x7fb57c06c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:29:59.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.833+0000 7fb581e73640 1 -- 172.21.15.105:0/2316479610 >> v1:172.21.15.105:6789/0 conn(0x7fb57c0ce170 legacy=0x7fb57c15d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:59.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.833+0000 7fb581e73640 1 -- 172.21.15.105:0/2316479610 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb57c06cd90 con 0x7fb57c0cd170 2023-11-28T18:29:59.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.837+0000 7fb581e73640 1 --2- 172.21.15.105:0/2316479610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57c0cd170 0x7fb57c0cd600 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fb574009d90 tx=0x7fb57401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545b240305c8d8f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:59.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.837+0000 7fb573fff640 1 -- 172.21.15.105:0/2316479610 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5740a58e0 con 0x7fb57c0cd170 2023-11-28T18:29:59.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.837+0000 7fb573fff640 1 -- 172.21.15.105:0/2316479610 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5740a5a40 con 0x7fb57c0cd170 2023-11-28T18:29:59.836 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.837+0000 7fb573fff640 1 -- 172.21.15.105:0/2316479610 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5740a5dd0 con 0x7fb57c0cd170 2023-11-28T18:29:59.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.837+0000 7fb582674640 1 -- 172.21.15.105:0/2316479610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57c0cd170 msgr2=0x7fb57c0cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:59.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.837+0000 7fb582674640 1 --2- 172.21.15.105:0/2316479610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57c0cd170 0x7fb57c0cd600 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fb574009d90 tx=0x7fb57401c910 comp rx=0 tx=0).stop 2023-11-28T18:29:59.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.837+0000 7fb582674640 1 -- 172.21.15.105:0/2316479610 shutdown_connections 2023-11-28T18:29:59.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.837+0000 7fb582674640 1 --2- 172.21.15.105:0/2316479610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57c0cd170 0x7fb57c0cd600 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:59.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.837+0000 7fb582674640 1 -- 172.21.15.105:0/2316479610 >> 172.21.15.105:0/2316479610 conn(0x7fb57c0ce9e0 msgr2=0x7fb57c0cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:29:59.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.837+0000 7fb582674640 1 -- 172.21.15.105:0/2316479610 shutdown_connections 2023-11-28T18:29:59.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.837+0000 7fb582674640 1 -- 172.21.15.105:0/2316479610 wait complete. 2023-11-28T18:29:59.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.837+0000 7fb582674640 1 Processor -- start 2023-11-28T18:29:59.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.837+0000 7fb582674640 1 -- start start 2023-11-28T18:29:59.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb582674640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb57c0cd170 0x7fb57c13f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:59.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb582674640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb57c0ce170 0x7fb57c070d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:59.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb582674640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57c159710 0x7fb57c0712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:59.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb582674640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c0c9fb0 con 0x7fb57c159710 2023-11-28T18:29:59.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb582674640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c12c390 con 0x7fb57c0cd170 2023-11-28T18:29:59.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb582674640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c0c93a0 con 0x7fb57c0ce170 2023-11-28T18:29:59.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb581e73640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57c159710 0x7fb57c0712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:59.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb581e73640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57c159710 0x7fb57c0712b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44536/0 (socket says 172.21.15.105:44536) 2023-11-28T18:29:59.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb581e73640 1 -- 172.21.15.105:0/1729709932 learned_addr learned my addr 172.21.15.105:0/1729709932 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:29:59.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb581672640 1 --2- 172.21.15.105:0/1729709932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb57c0cd170 0x7fb57c13f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:59.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb580e71640 1 --2- 172.21.15.105:0/1729709932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb57c0ce170 0x7fb57c070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:59.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb581e73640 1 -- 172.21.15.105:0/1729709932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb57c0ce170 msgr2=0x7fb57c070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:59.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb581e73640 1 --2- 172.21.15.105:0/1729709932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb57c0ce170 0x7fb57c070d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:59.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb581e73640 1 -- 172.21.15.105:0/1729709932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb57c0cd170 msgr2=0x7fb57c13f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:29:59.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb581e73640 1 --2- 172.21.15.105:0/1729709932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb57c0cd170 0x7fb57c13f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:29:59.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb581e73640 1 -- 172.21.15.105:0/1729709932 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb574009a70 con 0x7fb57c159710 2023-11-28T18:29:59.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb580e71640 1 --2- 172.21.15.105:0/1729709932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb57c0ce170 0x7fb57c070d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:29:59.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb581e73640 1 --2- 172.21.15.105:0/1729709932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57c159710 0x7fb57c0712b0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb57401a540 tx=0x7fb5740196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:59.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb571ffb640 1 -- 172.21.15.105:0/1729709932 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5740ad950 con 0x7fb57c159710 2023-11-28T18:29:59.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb571ffb640 1 -- 172.21.15.105:0/1729709932 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5740adab0 con 0x7fb57c159710 2023-11-28T18:29:59.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb571ffb640 1 -- 172.21.15.105:0/1729709932 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5740ade40 con 0x7fb57c159710 2023-11-28T18:29:59.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb582674640 1 -- 172.21.15.105:0/1729709932 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb57c0c9680 con 0x7fb57c159710 2023-11-28T18:29:59.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb582674640 1 -- 172.21.15.105:0/1729709932 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb57c0c9c40 con 0x7fb57c159710 2023-11-28T18:29:59.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb582674640 1 -- 172.21.15.105:0/1729709932 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb57c074580 con 0x7fb57c159710 2023-11-28T18:29:59.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.841+0000 7fb571ffb640 1 -- 172.21.15.105:0/1729709932 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fb5740b4040 con 0x7fb57c159710 2023-11-28T18:29:59.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.845+0000 7fb571ffb640 1 --2- 172.21.15.105:0/1729709932 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb560066950 0x7fb560068e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:29:59.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.845+0000 7fb571ffb640 1 -- 172.21.15.105:0/1729709932 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fb57411f260 con 0x7fb57c159710 2023-11-28T18:29:59.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.845+0000 7fb581672640 1 --2- 172.21.15.105:0/1729709932 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb560066950 0x7fb560068e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:29:59.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.845+0000 7fb581672640 1 --2- 172.21.15.105:0/1729709932 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb560066950 0x7fb560068e10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb5780098e0 tx=0x7fb578009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:29:59.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.849+0000 7fb571ffb640 1 -- 172.21.15.105:0/1729709932 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb5740ec500 con 0x7fb57c159710 2023-11-28T18:29:59.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:29:59.997+0000 7fb582674640 1 -- 172.21.15.105:0/1729709932 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb57c074580 con 0x7fb57c159710 2023-11-28T18:30:00.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.001+0000 7fb571ffb640 1 -- 172.21.15.105:0/1729709932 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v51)=0 v51) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb5740efb20 con 0x7fb57c159710 2023-11-28T18:30:00.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.005+0000 7fb582674640 1 -- 172.21.15.105:0/1729709932 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb560066950 msgr2=0x7fb560068e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:00.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.005+0000 7fb582674640 1 --2- 172.21.15.105:0/1729709932 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb560066950 0x7fb560068e10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb5780098e0 tx=0x7fb578009060 comp rx=0 tx=0).stop 2023-11-28T18:30:00.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.005+0000 7fb582674640 1 -- 172.21.15.105:0/1729709932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57c159710 msgr2=0x7fb57c0712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:00.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.005+0000 7fb582674640 1 --2- 172.21.15.105:0/1729709932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57c159710 0x7fb57c0712b0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb57401a540 tx=0x7fb5740196b0 comp rx=0 tx=0).stop 2023-11-28T18:30:00.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.005+0000 7fb582674640 1 -- 172.21.15.105:0/1729709932 shutdown_connections 2023-11-28T18:30:00.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.005+0000 7fb582674640 1 --2- 172.21.15.105:0/1729709932 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb560066950 0x7fb560068e10 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:00.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.005+0000 7fb582674640 1 --2- 172.21.15.105:0/1729709932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57c159710 0x7fb57c0712b0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:00.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.005+0000 7fb582674640 1 --2- 172.21.15.105:0/1729709932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb57c0ce170 0x7fb57c070d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:00.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.005+0000 7fb582674640 1 --2- 172.21.15.105:0/1729709932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb57c0cd170 0x7fb57c13f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:00.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.005+0000 7fb582674640 1 -- 172.21.15.105:0/1729709932 >> 172.21.15.105:0/1729709932 conn(0x7fb57c0ce9e0 msgr2=0x7fb57c15d940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:00.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.005+0000 7fb582674640 1 -- 172.21.15.105:0/1729709932 shutdown_connections 2023-11-28T18:30:00.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.005+0000 7fb582674640 1 -- 172.21.15.105:0/1729709932 wait complete. 2023-11-28T18:30:00.025 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T18:30:00.025 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:30:00.110 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:30:00.195 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T18:30:00.195 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:30:00.195 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:30:00.204 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:30:00.204 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:30:00.205 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:30:00.215 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:30:00.215 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:30:00.216 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:30:00.260 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:30:00.260 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:30:00.260 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:30:00.270 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:30:00.271 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T18:30:00.271 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:30:00.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.441+0000 7fd524f0f640 1 Processor -- start 2023-11-28T18:30:00.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.441+0000 7fd524f0f640 1 -- start start 2023-11-28T18:30:00.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.441+0000 7fd524f0f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd52015d3c0 0x7fd520071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:00.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.441+0000 7fd524f0f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd520161870 0x7fd5200ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:00.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.441+0000 7fd524f0f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5200cefa0 con 0x7fd520070a10 2023-11-28T18:30:00.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.441+0000 7fd524f0f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd52015d7a0 con 0x7fd52015d3c0 2023-11-28T18:30:00.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.441+0000 7fd524f0f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5200ce9e0 con 0x7fd520161870 2023-11-28T18:30:00.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.441+0000 7fd51e59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd52015d3c0 0x7fd520071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:00.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.441+0000 7fd51e59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd52015d3c0 0x7fd520071a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44540/0 (socket says 172.21.15.105:44540) 2023-11-28T18:30:00.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.441+0000 7fd51e59c640 1 -- 172.21.15.105:0/1142117215 learned_addr learned my addr 172.21.15.105:0/1142117215 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:00.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.441+0000 7fd515d9b640 1 --2- 172.21.15.105:0/1142117215 >> v2:172.21.15.148:3300/0 conn(0x7fd520161870 0x7fd5200ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:00.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.441+0000 7fd51e59c640 1 -- 172.21.15.105:0/1142117215 >> v2:172.21.15.148:3300/0 conn(0x7fd520161870 msgr2=0x7fd5200ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:00.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.441+0000 7fd51e59c640 1 --2- 172.21.15.105:0/1142117215 >> v2:172.21.15.148:3300/0 conn(0x7fd520161870 0x7fd5200ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:00.446 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.441+0000 7fd51e59c640 1 -- 172.21.15.105:0/1142117215 >> v1:172.21.15.148:6789/0 conn(0x7fd520070a10 legacy=0x7fd520070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:00.446 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.441+0000 7fd51e59c640 1 -- 172.21.15.105:0/1142117215 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5200cecc0 con 0x7fd52015d3c0 2023-11-28T18:30:00.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.445+0000 7fd51e59c640 1 --2- 172.21.15.105:0/1142117215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd52015d3c0 0x7fd520071a10 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fd520160e30 tx=0x7fd51001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e76c74bd65349ef server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:00.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.445+0000 7fd51dd9b640 1 -- 172.21.15.105:0/1142117215 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5100aa430 con 0x7fd52015d3c0 2023-11-28T18:30:00.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.445+0000 7fd51dd9b640 1 -- 172.21.15.105:0/1142117215 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5100aa590 con 0x7fd52015d3c0 2023-11-28T18:30:00.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.445+0000 7fd51dd9b640 1 -- 172.21.15.105:0/1142117215 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5100aa920 con 0x7fd52015d3c0 2023-11-28T18:30:00.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.445+0000 7fd524f0f640 1 -- 172.21.15.105:0/1142117215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd52015d3c0 msgr2=0x7fd520071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:00.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.445+0000 7fd524f0f640 1 --2- 172.21.15.105:0/1142117215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd52015d3c0 0x7fd520071a10 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fd520160e30 tx=0x7fd51001c4d0 comp rx=0 tx=0).stop 2023-11-28T18:30:00.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.445+0000 7fd524f0f640 1 -- 172.21.15.105:0/1142117215 shutdown_connections 2023-11-28T18:30:00.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.445+0000 7fd524f0f640 1 --2- 172.21.15.105:0/1142117215 >> v2:172.21.15.148:3300/0 conn(0x7fd520161870 0x7fd5200ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:00.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.445+0000 7fd524f0f640 1 --2- 172.21.15.105:0/1142117215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd52015d3c0 0x7fd520071a10 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:00.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.445+0000 7fd524f0f640 1 -- 172.21.15.105:0/1142117215 >> 172.21.15.105:0/1142117215 conn(0x7fd5200cf540 msgr2=0x7fd5200cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:00.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.445+0000 7fd524f0f640 1 -- 172.21.15.105:0/1142117215 shutdown_connections 2023-11-28T18:30:00.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.449+0000 7fd524f0f640 1 -- 172.21.15.105:0/1142117215 wait complete. 2023-11-28T18:30:00.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.449+0000 7fd524f0f640 1 Processor -- start 2023-11-28T18:30:00.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.449+0000 7fd524f0f640 1 -- start start 2023-11-28T18:30:00.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.449+0000 7fd524f0f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd520070a10 0x7fd520144fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:00.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.449+0000 7fd524f0f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd52015d3c0 0x7fd520154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:00.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.449+0000 7fd524f0f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd520161870 0x7fd5201f95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:00.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.449+0000 7fd524f0f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52014f2f0 con 0x7fd520070a10 2023-11-28T18:30:00.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.449+0000 7fd524f0f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd520155510 con 0x7fd520161870 2023-11-28T18:30:00.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.449+0000 7fd524f0f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52014ea70 con 0x7fd52015d3c0 2023-11-28T18:30:00.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.449+0000 7fd51e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd520070a10 0x7fd520144fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:00.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.449+0000 7fd51e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd520070a10 0x7fd520144fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44548/0 (socket says 172.21.15.105:44548) 2023-11-28T18:30:00.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.449+0000 7fd51e59c640 1 -- 172.21.15.105:0/4107944003 learned_addr learned my addr 172.21.15.105:0/4107944003 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:00.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.449+0000 7fd515d9b640 1 --2- 172.21.15.105:0/4107944003 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd52015d3c0 0x7fd520154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:00.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.449+0000 7fd51ed9d640 1 --2- 172.21.15.105:0/4107944003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd520161870 0x7fd5201f95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:00.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.449+0000 7fd51e59c640 1 -- 172.21.15.105:0/4107944003 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd52015d3c0 msgr2=0x7fd520154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:00.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.449+0000 7fd51e59c640 1 --2- 172.21.15.105:0/4107944003 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd52015d3c0 0x7fd520154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.449+0000 7fd51e59c640 1 -- 172.21.15.105:0/4107944003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd520161870 msgr2=0x7fd5201f95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.449+0000 7fd51e59c640 1 --2- 172.21.15.105:0/4107944003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd520161870 0x7fd5201f95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.449+0000 7fd51e59c640 1 -- 172.21.15.105:0/4107944003 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5100094d0 con 0x7fd520070a10 2023-11-28T18:30:00.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.449+0000 7fd51ed9d640 1 --2- 172.21.15.105:0/4107944003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd520161870 0x7fd5201f95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:00.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.449+0000 7fd51e59c640 1 --2- 172.21.15.105:0/4107944003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd520070a10 0x7fd520144fa0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fd5100190e0 tx=0x7fd51001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:00.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.453+0000 7fd5177fe640 1 -- 172.21.15.105:0/4107944003 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5100aa430 con 0x7fd520070a10 2023-11-28T18:30:00.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.453+0000 7fd5177fe640 1 -- 172.21.15.105:0/4107944003 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5100b3800 con 0x7fd520070a10 2023-11-28T18:30:00.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.453+0000 7fd524f0f640 1 -- 172.21.15.105:0/4107944003 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52014ed30 con 0x7fd520070a10 2023-11-28T18:30:00.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.453+0000 7fd5177fe640 1 -- 172.21.15.105:0/4107944003 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5100b3b90 con 0x7fd520070a10 2023-11-28T18:30:00.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.453+0000 7fd524f0f640 1 -- 172.21.15.105:0/4107944003 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5201454e0 con 0x7fd520070a10 2023-11-28T18:30:00.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.453+0000 7fd524f0f640 1 -- 172.21.15.105:0/4107944003 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd520067570 con 0x7fd520070a10 2023-11-28T18:30:00.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.453+0000 7fd5177fe640 1 -- 172.21.15.105:0/4107944003 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd5100b3cf0 con 0x7fd520070a10 2023-11-28T18:30:00.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.453+0000 7fd5177fe640 1 --2- 172.21.15.105:0/4107944003 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5000668d0 0x7fd500068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:00.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.453+0000 7fd5177fe640 1 -- 172.21.15.105:0/4107944003 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fd51011ed00 con 0x7fd520070a10 2023-11-28T18:30:00.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.453+0000 7fd515d9b640 1 --2- 172.21.15.105:0/4107944003 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5000668d0 0x7fd500068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:00.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.457+0000 7fd515d9b640 1 --2- 172.21.15.105:0/4107944003 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5000668d0 0x7fd500068d90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd520152fe0 tx=0x7fd508009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:00.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.457+0000 7fd5177fe640 1 -- 172.21.15.105:0/4107944003 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd5100ec0f0 con 0x7fd520070a10 2023-11-28T18:30:00.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.637+0000 7fd524f0f640 1 -- 172.21.15.105:0/4107944003 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5200b0b50 con 0x7fd520070a10 2023-11-28T18:30:00.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.637+0000 7fd5177fe640 1 -- 172.21.15.105:0/4107944003 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+3705 (secure 0 0 0) 0x7fd5100ef710 con 0x7fd520070a10 2023-11-28T18:30:00.639 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:00.639 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":96,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/1464337040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1464337040},{"type":"v1","addr":"172.21.15.148:6834","nonce":1464337040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6722,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/266240730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":266240730},{"type":"v1","addr":"172.21.15.148:6835","nonce":266240730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":8037,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1311588272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1311588272},{"type":"v1","addr":"172.21.15.105:6835","nonce":1311588272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":8040,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/647926356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":647926356},{"type":"v1","addr":"172.21.15.105:6837","nonce":647926356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[]} 2023-11-28T18:30:00.639 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 96 2023-11-28T18:30:00.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.641+0000 7fd524f0f640 1 -- 172.21.15.105:0/4107944003 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5000668d0 msgr2=0x7fd500068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:00.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.641+0000 7fd524f0f640 1 --2- 172.21.15.105:0/4107944003 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5000668d0 0x7fd500068d90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd520152fe0 tx=0x7fd508009190 comp rx=0 tx=0).stop 2023-11-28T18:30:00.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.641+0000 7fd524f0f640 1 -- 172.21.15.105:0/4107944003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd520070a10 msgr2=0x7fd520144fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:00.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.641+0000 7fd524f0f640 1 --2- 172.21.15.105:0/4107944003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd520070a10 0x7fd520144fa0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fd5100190e0 tx=0x7fd51001ad10 comp rx=0 tx=0).stop 2023-11-28T18:30:00.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.641+0000 7fd524f0f640 1 -- 172.21.15.105:0/4107944003 shutdown_connections 2023-11-28T18:30:00.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.641+0000 7fd524f0f640 1 --2- 172.21.15.105:0/4107944003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd520161870 0x7fd5201f95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:00.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.641+0000 7fd524f0f640 1 --2- 172.21.15.105:0/4107944003 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd52015d3c0 0x7fd520154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:00.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.641+0000 7fd524f0f640 1 --2- 172.21.15.105:0/4107944003 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd5000668d0 0x7fd500068d90 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:00.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.641+0000 7fd524f0f640 1 --2- 172.21.15.105:0/4107944003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd520070a10 0x7fd520144fa0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:00.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.641+0000 7fd524f0f640 1 -- 172.21.15.105:0/4107944003 >> 172.21.15.105:0/4107944003 conn(0x7fd5200cf540 msgr2=0x7fd5200cf920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:00.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.641+0000 7fd524f0f640 1 -- 172.21.15.105:0/4107944003 shutdown_connections 2023-11-28T18:30:00.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.641+0000 7fd524f0f640 1 -- 172.21.15.105:0/4107944003 wait complete. 2023-11-28T18:30:00.661 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T18:30:00.661 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T18:30:00.661 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T18:30:00.661 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:30:00.662 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T18:30:00.662 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T18:30:00.662 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T18:30:00.662 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:30:00.662 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T18:30:00.662 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T18:30:00.663 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T18:30:00.663 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:30:00.663 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T18:30:00.663 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T18:30:00.663 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T18:30:00.663 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:30:00.676 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T18:30:00.676 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T18:30:00.677 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T18:30:00.677 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T18:30:00.678 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T18:30:00.678 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T18:30:00.679 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T18:30:00.679 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T18:30:00.727 INFO:tasks.ceph.mds.a:Started 2023-11-28T18:30:00.727 DEBUG:teuthology.parallel:result is None 2023-11-28T18:30:00.727 INFO:tasks.ceph.mds.b:Started 2023-11-28T18:30:00.728 DEBUG:teuthology.parallel:result is None 2023-11-28T18:30:00.728 INFO:tasks.ceph.mds.c:Started 2023-11-28T18:30:00.729 DEBUG:teuthology.parallel:result is None 2023-11-28T18:30:00.729 INFO:tasks.ceph.mds.d:Started 2023-11-28T18:30:00.729 DEBUG:teuthology.parallel:result is None 2023-11-28T18:30:00.730 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T18:30:00.817 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2023-11-28T18:30:00.818 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2023-11-28T18:30:00.820 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2023-11-28T18:30:00.821 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2023-11-28T18:30:00.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.873+0000 7fd30fb0f640 1 Processor -- start 2023-11-28T18:30:00.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.873+0000 7fd30fb0f640 1 -- start start 2023-11-28T18:30:00.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.873+0000 7fd30fb0f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd30806d610 0x7fd3080cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:00.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.873+0000 7fd30fb0f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd3080ae100 0x7fd3080cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:00.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.873+0000 7fd30fb0f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd308162820 con 0x7fd3080ce530 2023-11-28T18:30:00.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.873+0000 7fd30fb0f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd308161c10 con 0x7fd3080ae100 2023-11-28T18:30:00.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.873+0000 7fd30fb0f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd308161ef0 con 0x7fd30806d610 2023-11-28T18:30:00.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.873+0000 7fd30e0ac640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd3080ae100 0x7fd3080cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:00.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.873+0000 7fd30e0ac640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd3080ae100 0x7fd3080cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44586/0 (socket says 172.21.15.105:44586) 2023-11-28T18:30:00.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.873+0000 7fd30e0ac640 1 -- 172.21.15.105:0/1415837561 learned_addr learned my addr 172.21.15.105:0/1415837561 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:00.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.873+0000 7fd30d8ab640 1 --2- 172.21.15.105:0/1415837561 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd30806d610 0x7fd3080cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:00.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.877+0000 7fd30e0ac640 1 -- 172.21.15.105:0/1415837561 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd30806d610 msgr2=0x7fd3080cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:00.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.877+0000 7fd30e0ac640 1 --2- 172.21.15.105:0/1415837561 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd30806d610 0x7fd3080cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:00.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.877+0000 7fd30e0ac640 1 -- 172.21.15.105:0/1415837561 >> v1:172.21.15.105:6789/0 conn(0x7fd3080ce530 legacy=0x7fd3080ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:00.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.877+0000 7fd30e0ac640 1 -- 172.21.15.105:0/1415837561 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3081621d0 con 0x7fd3080ae100 2023-11-28T18:30:00.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.877+0000 7fd30d8ab640 1 --2- 172.21.15.105:0/1415837561 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd30806d610 0x7fd3080cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.877+0000 7fd30e0ac640 1 --2- 172.21.15.105:0/1415837561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3080ae100 0x7fd3080cc7d0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd300009e50 tx=0x7fd30001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9403e52cc8503a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.877+0000 7fd30c8a9640 1 -- 172.21.15.105:0/1415837561 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3000a58d0 con 0x7fd3080ae100 2023-11-28T18:30:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.877+0000 7fd30c8a9640 1 -- 172.21.15.105:0/1415837561 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3000a5a30 con 0x7fd3080ae100 2023-11-28T18:30:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.877+0000 7fd30c8a9640 1 -- 172.21.15.105:0/1415837561 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3000a5dc0 con 0x7fd3080ae100 2023-11-28T18:30:00.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.877+0000 7fd30fb0f640 1 -- 172.21.15.105:0/1415837561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3080ae100 msgr2=0x7fd3080cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:00.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.877+0000 7fd30fb0f640 1 --2- 172.21.15.105:0/1415837561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3080ae100 0x7fd3080cc7d0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd300009e50 tx=0x7fd30001c9e0 comp rx=0 tx=0).stop 2023-11-28T18:30:00.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30fb0f640 1 -- 172.21.15.105:0/1415837561 shutdown_connections 2023-11-28T18:30:00.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30fb0f640 1 --2- 172.21.15.105:0/1415837561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3080ae100 0x7fd3080cc7d0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:00.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30fb0f640 1 --2- 172.21.15.105:0/1415837561 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd30806d610 0x7fd3080cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:00.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30fb0f640 1 -- 172.21.15.105:0/1415837561 >> 172.21.15.105:0/1415837561 conn(0x7fd3080ceb90 msgr2=0x7fd3080cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:00.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30fb0f640 1 -- 172.21.15.105:0/1415837561 shutdown_connections 2023-11-28T18:30:00.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30fb0f640 1 -- 172.21.15.105:0/1415837561 wait complete. 2023-11-28T18:30:00.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30fb0f640 1 Processor -- start 2023-11-28T18:30:00.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30fb0f640 1 -- start start 2023-11-28T18:30:00.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30fb0f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd30806d610 0x7fd30815c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:00.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30fb0f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3080ae100 0x7fd30806bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:00.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30fb0f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3080ce530 0x7fd30806c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:00.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30fb0f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3080b4a40 con 0x7fd3080ae100 2023-11-28T18:30:00.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30fb0f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30815c5a0 con 0x7fd3080ce530 2023-11-28T18:30:00.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30fb0f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd308099b80 con 0x7fd30806d610 2023-11-28T18:30:00.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30d0aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3080ae100 0x7fd30806bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:00.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30d0aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3080ae100 0x7fd30806bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44590/0 (socket says 172.21.15.105:44590) 2023-11-28T18:30:00.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30d0aa640 1 -- 172.21.15.105:0/2455811764 learned_addr learned my addr 172.21.15.105:0/2455811764 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:00.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30e0ac640 1 --2- 172.21.15.105:0/2455811764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3080ce530 0x7fd30806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30d8ab640 1 --2- 172.21.15.105:0/2455811764 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd30806d610 0x7fd30815c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30d0aa640 1 -- 172.21.15.105:0/2455811764 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd30806d610 msgr2=0x7fd30815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30d0aa640 1 --2- 172.21.15.105:0/2455811764 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd30806d610 0x7fd30815c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30d0aa640 1 -- 172.21.15.105:0/2455811764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3080ce530 msgr2=0x7fd30806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30d0aa640 1 --2- 172.21.15.105:0/2455811764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3080ce530 0x7fd30806c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30d0aa640 1 -- 172.21.15.105:0/2455811764 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd300009b30 con 0x7fd3080ae100 2023-11-28T18:30:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd30d0aa640 1 --2- 172.21.15.105:0/2455811764 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3080ae100 0x7fd30806bf50 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800c3f0 tx=0x7fd2f800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.881+0000 7fd2fe7fc640 1 -- 172.21.15.105:0/2455811764 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f80052d0 con 0x7fd3080ae100 2023-11-28T18:30:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.885+0000 7fd2fe7fc640 1 -- 172.21.15.105:0/2455811764 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f8005430 con 0x7fd3080ae100 2023-11-28T18:30:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.885+0000 7fd30fb0f640 1 -- 172.21.15.105:0/2455811764 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd308165090 con 0x7fd3080ae100 2023-11-28T18:30:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.885+0000 7fd2fe7fc640 1 -- 172.21.15.105:0/2455811764 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8003840 con 0x7fd3080ae100 2023-11-28T18:30:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.885+0000 7fd30fb0f640 1 -- 172.21.15.105:0/2455811764 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd308165740 con 0x7fd3080ae100 2023-11-28T18:30:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.885+0000 7fd30fb0f640 1 -- 172.21.15.105:0/2455811764 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd308099cf0 con 0x7fd3080ae100 2023-11-28T18:30:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.885+0000 7fd2fe7fc640 1 -- 172.21.15.105:0/2455811764 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd2f80039a0 con 0x7fd3080ae100 2023-11-28T18:30:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.885+0000 7fd2fe7fc640 1 --2- 172.21.15.105:0/2455811764 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd2ec066900 0x7fd2ec068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.885+0000 7fd2fe7fc640 1 -- 172.21.15.105:0/2455811764 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fd2f8083940 con 0x7fd3080ae100 2023-11-28T18:30:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.885+0000 7fd30d8ab640 1 --2- 172.21.15.105:0/2455811764 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd2ec066900 0x7fd2ec068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.885+0000 7fd30d8ab640 1 --2- 172.21.15.105:0/2455811764 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd2ec066900 0x7fd2ec068dc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd30400c900 tx=0x7fd304008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:00.889+0000 7fd2fe7fc640 1 -- 172.21.15.105:0/2455811764 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd2f8050bf0 con 0x7fd3080ae100 2023-11-28T18:30:01.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.049+0000 7fd30fb0f640 1 -- 172.21.15.105:0/2455811764 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd308067570 con 0x7fd3080ae100 2023-11-28T18:30:01.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.553+0000 7fd2fe7fc640 1 -- 172.21.15.105:0/2455811764 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v97) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd2f8017080 con 0x7fd3080ae100 2023-11-28T18:30:01.555 INFO:teuthology.orchestra.run.smithi105.stderr: removed all blocklist entries 2023-11-28T18:30:01.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.557+0000 7fd30fb0f640 1 -- 172.21.15.105:0/2455811764 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd2ec066900 msgr2=0x7fd2ec068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:01.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.557+0000 7fd30fb0f640 1 --2- 172.21.15.105:0/2455811764 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd2ec066900 0x7fd2ec068dc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd30400c900 tx=0x7fd304008040 comp rx=0 tx=0).stop 2023-11-28T18:30:01.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.557+0000 7fd30fb0f640 1 -- 172.21.15.105:0/2455811764 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3080ae100 msgr2=0x7fd30806bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:01.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.557+0000 7fd30fb0f640 1 --2- 172.21.15.105:0/2455811764 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3080ae100 0x7fd30806bf50 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800c3f0 tx=0x7fd2f800c8b0 comp rx=0 tx=0).stop 2023-11-28T18:30:01.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.557+0000 7fd30fb0f640 1 -- 172.21.15.105:0/2455811764 shutdown_connections 2023-11-28T18:30:01.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.557+0000 7fd30fb0f640 1 --2- 172.21.15.105:0/2455811764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3080ce530 0x7fd30806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:01.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.557+0000 7fd30fb0f640 1 --2- 172.21.15.105:0/2455811764 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd2ec066900 0x7fd2ec068dc0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:01.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.557+0000 7fd30fb0f640 1 --2- 172.21.15.105:0/2455811764 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3080ae100 0x7fd30806bf50 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:01.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.557+0000 7fd30fb0f640 1 --2- 172.21.15.105:0/2455811764 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd30806d610 0x7fd30815c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:01.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.557+0000 7fd30fb0f640 1 -- 172.21.15.105:0/2455811764 >> 172.21.15.105:0/2455811764 conn(0x7fd3080ceb90 msgr2=0x7fd30815f690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:01.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.557+0000 7fd30fb0f640 1 -- 172.21.15.105:0/2455811764 shutdown_connections 2023-11-28T18:30:01.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.557+0000 7fd30fb0f640 1 -- 172.21.15.105:0/2455811764 wait complete. 2023-11-28T18:30:01.580 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T18:30:01.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c3edb8640 1 Processor -- start 2023-11-28T18:30:01.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c3edb8640 1 -- start start 2023-11-28T18:30:01.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c3edb8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5c380d3c30 0x7f5c380d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:01.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c3edb8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c380b25b0 con 0x7f5c3815d4f0 2023-11-28T18:30:01.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c3edb8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c380cea30 con 0x7f5c380b2d20 2023-11-28T18:30:01.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c3edb8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c380b2900 con 0x7f5c380d3c30 2023-11-28T18:30:01.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c3cb54640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5c3815d4f0 0x7f5c380d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40146/0 (socket says 172.21.15.105:40146) 2023-11-28T18:30:01.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c3cb54640 1 -- 172.21.15.105:0/134685300 learned_addr learned my addr 172.21.15.105:0/134685300 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:01.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c37fff640 1 --2- 172.21.15.105:0/134685300 >> v2:172.21.15.148:3300/0 conn(0x7f5c380d3c30 0x7f5c380d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:01.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c377fe640 1 -- 172.21.15.105:0/134685300 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331385123 0 0) 0x7f5c380b25b0 con 0x7f5c3815d4f0 2023-11-28T18:30:01.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c377fe640 1 -- 172.21.15.105:0/134685300 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c28003530 con 0x7f5c3815d4f0 2023-11-28T18:30:01.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c377fe640 1 -- 172.21.15.105:0/134685300 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2745439950 0 0) 0x7f5c28003530 con 0x7f5c3815d4f0 2023-11-28T18:30:01.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c377fe640 1 -- 172.21.15.105:0/134685300 >> v2:172.21.15.148:3300/0 conn(0x7f5c380d3c30 msgr2=0x7f5c380d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:01.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c377fe640 1 --2- 172.21.15.105:0/134685300 >> v2:172.21.15.148:3300/0 conn(0x7f5c380d3c30 0x7f5c380d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:01.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c377fe640 1 -- 172.21.15.105:0/134685300 >> v1:172.21.15.148:6789/0 conn(0x7f5c380b2d20 legacy=0x7f5c380b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:01.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c377fe640 1 -- 172.21.15.105:0/134685300 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c380ced10 con 0x7f5c3815d4f0 2023-11-28T18:30:01.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c377fe640 1 -- 172.21.15.105:0/134685300 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f5c2c001680 con 0x7f5c3815d4f0 2023-11-28T18:30:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c377fe640 1 -- 172.21.15.105:0/134685300 >> v1:172.21.15.105:6789/0 conn(0x7f5c3815d4f0 legacy=0x7f5c380d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c377fe640 1 --2- 172.21.15.105:0/134685300 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c280039d0 0x7f5c28003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c377fe640 1 --2- 172.21.15.105:0/134685300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c280047d0 0x7f5c28004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c377fe640 1 --2- 172.21.15.105:0/134685300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c280051e0 0x7f5c2800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c377fe640 1 -- 172.21.15.105:0/134685300 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c380b2900 con 0x7f5c280047d0 2023-11-28T18:30:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c377fe640 1 -- 172.21.15.105:0/134685300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2800c130 con 0x7f5c280051e0 2023-11-28T18:30:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c377fe640 1 -- 172.21.15.105:0/134685300 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2800c2f0 con 0x7f5c280039d0 2023-11-28T18:30:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.717+0000 7f5c37fff640 1 --2- 172.21.15.105:0/134685300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c280047d0 0x7f5c28004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.721+0000 7f5c3cb54640 1 --2- 172.21.15.105:0/134685300 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c280039d0 0x7f5c28003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.721+0000 7f5c3d355640 1 --2- 172.21.15.105:0/134685300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c280051e0 0x7f5c2800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:01.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.721+0000 7f5c37fff640 1 -- 172.21.15.105:0/134685300 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c280039d0 msgr2=0x7f5c28003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:01.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.721+0000 7f5c37fff640 1 --2- 172.21.15.105:0/134685300 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c280039d0 0x7f5c28003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:01.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.721+0000 7f5c37fff640 1 -- 172.21.15.105:0/134685300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c280051e0 msgr2=0x7f5c2800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:01.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.721+0000 7f5c37fff640 1 --2- 172.21.15.105:0/134685300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c280051e0 0x7f5c2800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:01.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.721+0000 7f5c37fff640 1 -- 172.21.15.105:0/134685300 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c28003530 con 0x7f5c280047d0 2023-11-28T18:30:01.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.721+0000 7f5c3d355640 1 -- 172.21.15.105:0/134685300 reap_dead start 2023-11-28T18:30:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.721+0000 7f5c37fff640 1 --2- 172.21.15.105:0/134685300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c280047d0 0x7f5c28004c50 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009ab0 tx=0x7f5c2401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50655d511c0ec57a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.721+0000 7f5c377fe640 1 -- 172.21.15.105:0/134685300 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c240a64f0 con 0x7f5c280047d0 2023-11-28T18:30:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.721+0000 7f5c377fe640 1 -- 172.21.15.105:0/134685300 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c240a6650 con 0x7f5c280047d0 2023-11-28T18:30:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.721+0000 7f5c377fe640 1 -- 172.21.15.105:0/134685300 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c240a6a00 con 0x7f5c280047d0 2023-11-28T18:30:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.721+0000 7f5c3edb8640 1 -- 172.21.15.105:0/134685300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c280047d0 msgr2=0x7f5c28004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.721+0000 7f5c3edb8640 1 --2- 172.21.15.105:0/134685300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c280047d0 0x7f5c28004c50 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009ab0 tx=0x7f5c2401ca00 comp rx=0 tx=0).stop 2023-11-28T18:30:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c3edb8640 1 -- 172.21.15.105:0/134685300 shutdown_connections 2023-11-28T18:30:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c3edb8640 1 --2- 172.21.15.105:0/134685300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c280047d0 0x7f5c28004c50 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:01.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c3edb8640 1 -- 172.21.15.105:0/134685300 >> 172.21.15.105:0/134685300 conn(0x7f5c380d3030 msgr2=0x7f5c380d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:01.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c3edb8640 1 -- 172.21.15.105:0/134685300 shutdown_connections 2023-11-28T18:30:01.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c3edb8640 1 -- 172.21.15.105:0/134685300 wait complete. 2023-11-28T18:30:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c3edb8640 1 Processor -- start 2023-11-28T18:30:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c3edb8640 1 -- start start 2023-11-28T18:30:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c3edb8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c280047d0 0x7f5c380cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c3edb8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c380b7580 0x7f5c380b7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c3edb8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c38150f80 0x7f5c38151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c3edb8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c380d0a20 con 0x7f5c380b7580 2023-11-28T18:30:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c3edb8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c380b8300 con 0x7f5c280047d0 2023-11-28T18:30:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c3edb8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c38151940 con 0x7f5c38150f80 2023-11-28T18:30:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c37fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c380b7580 0x7f5c380b7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c37fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c380b7580 0x7f5c380b7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49998/0 (socket says 172.21.15.105:49998) 2023-11-28T18:30:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c37fff640 1 -- 172.21.15.105:0/2030191297 learned_addr learned my addr 172.21.15.105:0/2030191297 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c3cb54640 1 --2- 172.21.15.105:0/2030191297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c280047d0 0x7f5c380cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c3d355640 1 --2- 172.21.15.105:0/2030191297 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c38150f80 0x7f5c38151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c37fff640 1 -- 172.21.15.105:0/2030191297 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c38150f80 msgr2=0x7f5c38151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c37fff640 1 --2- 172.21.15.105:0/2030191297 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c38150f80 0x7f5c38151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c37fff640 1 -- 172.21.15.105:0/2030191297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c280047d0 msgr2=0x7f5c380cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c37fff640 1 --2- 172.21.15.105:0/2030191297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c280047d0 0x7f5c380cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c37fff640 1 -- 172.21.15.105:0/2030191297 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c380b8020 con 0x7f5c380b7580 2023-11-28T18:30:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c3d355640 1 --2- 172.21.15.105:0/2030191297 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c38150f80 0x7f5c38151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.725+0000 7f5c37fff640 1 --2- 172.21.15.105:0/2030191297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c380b7580 0x7f5c380b7a00 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f5c240b0040 tx=0x7f5c240197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:01.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.729+0000 7f5c357fa640 1 -- 172.21.15.105:0/2030191297 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c240bd070 con 0x7f5c380b7580 2023-11-28T18:30:01.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.729+0000 7f5c357fa640 1 -- 172.21.15.105:0/2030191297 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2400f000 con 0x7f5c380b7580 2023-11-28T18:30:01.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.729+0000 7f5c357fa640 1 -- 172.21.15.105:0/2030191297 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c240a67c0 con 0x7f5c380b7580 2023-11-28T18:30:01.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.729+0000 7f5c3edb8640 1 -- 172.21.15.105:0/2030191297 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c3815dc80 con 0x7f5c380b7580 2023-11-28T18:30:01.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.729+0000 7f5c3edb8640 1 -- 172.21.15.105:0/2030191297 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3815e240 con 0x7f5c380b7580 2023-11-28T18:30:01.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.729+0000 7f5c3edb8640 1 -- 172.21.15.105:0/2030191297 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c3815dae0 con 0x7f5c380b7580 2023-11-28T18:30:01.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.729+0000 7f5c357fa640 1 -- 172.21.15.105:0/2030191297 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5c240a6920 con 0x7f5c380b7580 2023-11-28T18:30:01.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.729+0000 7f5c357fa640 1 --2- 172.21.15.105:0/2030191297 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5c1c0668d0 0x7f5c1c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:01.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.729+0000 7f5c357fa640 1 -- 172.21.15.105:0/2030191297 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f5c240aa070 con 0x7f5c380b7580 2023-11-28T18:30:01.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.729+0000 7f5c3cb54640 1 --2- 172.21.15.105:0/2030191297 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5c1c0668d0 0x7f5c1c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:01.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.729+0000 7f5c3cb54640 1 --2- 172.21.15.105:0/2030191297 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5c1c0668d0 0x7f5c1c068d90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c001060 tx=0x7f5c2c000f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:01.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.733+0000 7f5c357fa640 1 -- 172.21.15.105:0/2030191297 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5c3815dae0 con 0x7f5c380b7580 2023-11-28T18:30:01.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.933+0000 7f5c3edb8640 1 -- 172.21.15.105:0/2030191297 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5c3815dae0 con 0x7f5c380b7580 2023-11-28T18:30:01.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.933+0000 7f5c357fa640 1 -- 172.21.15.105:0/2030191297 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5c3815dae0 con 0x7f5c380b7580 2023-11-28T18:30:01.933 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:01.933 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:30:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: "auth_dump": [ 2023-11-28T18:30:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.a", 2023-11-28T18:30:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlOXSTGxAAU5kGNB3tznYsDNG51vCIZA==", 2023-11-28T18:30:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:30:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:30:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:30:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.b", 2023-11-28T18:30:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlrnbWJxAAKefl0YI2MhobnAOIj6WeaQ==", 2023-11-28T18:30:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:30:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:30:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:30:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.c", 2023-11-28T18:30:01.936 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2Zl5Y7GIhAA5OvgkNNFaI7LnzSsQVdFsQ==", 2023-11-28T18:30:01.936 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:01.936 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:30:01.936 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:30:01.936 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:30:01.936 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:01.936 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:01.936 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:01.936 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:01.936 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.d", 2023-11-28T18:30:01.936 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlmfYdLxAAreI7UE1YCjzkj6yl3s5Fwg==", 2023-11-28T18:30:01.936 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:01.936 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:30:01.937 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:30:01.937 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:30:01.937 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:01.937 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:01.937 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:01.937 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:01.937 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.0", 2023-11-28T18:30:01.937 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDtL2ZlphxQEBAAQ280OXh1zeaotIdVU8oZng==", 2023-11-28T18:30:01.937 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:01.937 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:30:01.937 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:30:01.937 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:01.938 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:01.938 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:01.938 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:01.938 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.1", 2023-11-28T18:30:01.938 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDwL2ZlezmOFxAARRBIOTkVJYfquIdcpW7sQQ==", 2023-11-28T18:30:01.938 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:01.938 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:30:01.938 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:30:01.938 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:01.938 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:01.938 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:01.938 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:01.939 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.2", 2023-11-28T18:30:01.939 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDzL2Zl4YcSHxAA2ww2F4TAD/0H/ZJ+aVMGEQ==", 2023-11-28T18:30:01.939 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:01.939 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:30:01.939 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:30:01.939 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:01.939 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:01.939 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:01.939 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:01.939 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.3", 2023-11-28T18:30:01.939 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD2L2ZlNxWxJRAAImp7TjTF18KwFTZ14DM6EA==", 2023-11-28T18:30:01.939 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:01.940 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:30:01.940 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:30:01.940 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:01.940 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:01.940 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:01.940 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:01.940 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.4", 2023-11-28T18:30:01.940 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD8L2ZlOAieLxAAz8g+anH8uJRfsttUS41GGQ==", 2023-11-28T18:30:01.940 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:01.940 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:30:01.940 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:30:01.940 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:01.940 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:01.941 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:01.941 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:01.941 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.5", 2023-11-28T18:30:01.941 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD/L2Zl5PKTNxAAPk+uIodLPjSd0EEMVT6iEw==", 2023-11-28T18:30:01.941 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:01.941 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:30:01.941 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:30:01.941 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:01.941 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:01.941 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:01.941 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:01.941 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.6", 2023-11-28T18:30:01.942 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQADMGZll+FUAxAAHlym5Z8rd/OF1911ZE2OUg==", 2023-11-28T18:30:01.942 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:01.942 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:30:01.942 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:30:01.942 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:01.942 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:01.942 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:01.942 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:01.942 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.7", 2023-11-28T18:30:01.942 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQAGMGZl8GO4ChAAVn3N6DnxUcs+ew09QvGo2w==", 2023-11-28T18:30:01.942 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:01.942 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:30:01.943 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:30:01.943 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:01.943 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:01.943 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:01.943 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:01.943 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.0", 2023-11-28T18:30:01.943 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlSt0VMxAAQ8+j1BgBPMJ2cuiZj8aG0w==", 2023-11-28T18:30:01.943 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:01.943 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:30:01.943 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow r", 2023-11-28T18:30:01.943 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-28T18:30:01.943 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:01.944 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:01.944 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:01.944 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.1", 2023-11-28T18:30:01.944 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDqL2ZlfZ1/ARAAhnJB6oj4Cc6p/oCjM6hTCw==", 2023-11-28T18:30:01.944 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:01.944 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:30:01.944 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:30:01.944 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:30:01.944 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:30:01.944 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:01.944 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:01.944 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:01.945 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.2", 2023-11-28T18:30:01.945 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlPLtyORAAU2GST6TJzlTfh6YqEb3U3w==", 2023-11-28T18:30:01.945 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:01.945 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:30:01.945 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:30:01.945 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:30:01.945 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:30:01.945 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:01.945 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:01.945 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:01.945 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.3", 2023-11-28T18:30:01.945 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDqL2ZlhiFuBxAAKQRxCdtqWpAHy1AVvdj9Sg==", 2023-11-28T18:30:01.946 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:01.946 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:30:01.946 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:30:01.946 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:30:01.946 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:30:01.946 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:01.946 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:01.946 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:01.946 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.admin", 2023-11-28T18:30:01.946 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDoL2ZlkuP2OhAAxsdZEF9122vkH53EwB8Qug==", 2023-11-28T18:30:01.946 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:01.946 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:30:01.947 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:30:01.947 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:30:01.947 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:01.947 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:01.947 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:01.947 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:01.947 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.x", 2023-11-28T18:30:01.947 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlEG49FxAAespsG+QK1WTpbmpV+DIwiw==", 2023-11-28T18:30:01.947 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:01.947 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:30:01.947 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2023-11-28T18:30:01.947 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:01.948 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:01.948 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:01.948 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:01.948 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.y", 2023-11-28T18:30:01.948 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlU89RExAAwMJp+8zZ6aFEt0KbiB6toA==", 2023-11-28T18:30:01.948 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:01.948 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:30:01.948 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2023-11-28T18:30:01.948 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:01.948 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:01.948 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:01.948 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2023-11-28T18:30:01.949 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:30:01.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.933+0000 7f5c3edb8640 1 -- 172.21.15.105:0/2030191297 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5c1c0668d0 msgr2=0x7f5c1c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:01.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.933+0000 7f5c3edb8640 1 --2- 172.21.15.105:0/2030191297 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5c1c0668d0 0x7f5c1c068d90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c001060 tx=0x7f5c2c000f90 comp rx=0 tx=0).stop 2023-11-28T18:30:01.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.933+0000 7f5c3edb8640 1 -- 172.21.15.105:0/2030191297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c380b7580 msgr2=0x7f5c380b7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:01.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.933+0000 7f5c3edb8640 1 --2- 172.21.15.105:0/2030191297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c380b7580 0x7f5c380b7a00 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f5c240b0040 tx=0x7f5c240197a0 comp rx=0 tx=0).stop 2023-11-28T18:30:01.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.933+0000 7f5c3edb8640 1 -- 172.21.15.105:0/2030191297 shutdown_connections 2023-11-28T18:30:01.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.933+0000 7f5c3edb8640 1 --2- 172.21.15.105:0/2030191297 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c38150f80 0x7f5c38151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:01.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.933+0000 7f5c3edb8640 1 --2- 172.21.15.105:0/2030191297 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5c1c0668d0 0x7f5c1c068d90 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:01.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.933+0000 7f5c3edb8640 1 --2- 172.21.15.105:0/2030191297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c380b7580 0x7f5c380b7a00 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:01.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.933+0000 7f5c3edb8640 1 --2- 172.21.15.105:0/2030191297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c280047d0 0x7f5c380cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:01.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.933+0000 7f5c3edb8640 1 -- 172.21.15.105:0/2030191297 >> 172.21.15.105:0/2030191297 conn(0x7f5c380d3030 msgr2=0x7f5c3806b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:01.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.933+0000 7f5c3edb8640 1 -- 172.21.15.105:0/2030191297 shutdown_connections 2023-11-28T18:30:01.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:01.933+0000 7f5c3edb8640 1 -- 172.21.15.105:0/2030191297 wait complete. 2023-11-28T18:30:01.955 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:30:02.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.089+0000 7f6123b2a640 1 Processor -- start 2023-11-28T18:30:02.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.089+0000 7f6123b2a640 1 -- start start 2023-11-28T18:30:02.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.093+0000 7f6123b2a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f611c0b4290 0x7f611c0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:02.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.093+0000 7f6123b2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f611c15b030 0x7f611c0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:02.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.093+0000 7f6123b2a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f611c0b4bb0 con 0x7f611c071960 2023-11-28T18:30:02.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.093+0000 7f6123b2a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f611c0701d0 con 0x7f611c0b4290 2023-11-28T18:30:02.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.093+0000 7f6123b2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611c06df80 con 0x7f611c15b030 2023-11-28T18:30:02.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.093+0000 7f61218c6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f611c0b4290 0x7f611c0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:02.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.093+0000 7f61218c6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f611c0b4290 0x7f611c0b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50002/0 (socket says 172.21.15.105:50002) 2023-11-28T18:30:02.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.093+0000 7f61218c6640 1 -- 172.21.15.105:0/2102327419 learned_addr learned my addr 172.21.15.105:0/2102327419 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:02.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.093+0000 7f61220c7640 1 --2- 172.21.15.105:0/2102327419 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f611c15b030 0x7f611c0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:02.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.093+0000 7f61218c6640 1 -- 172.21.15.105:0/2102327419 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f611c15b030 msgr2=0x7f611c0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:02.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.093+0000 7f61218c6640 1 --2- 172.21.15.105:0/2102327419 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f611c15b030 0x7f611c0709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.093+0000 7f61218c6640 1 -- 172.21.15.105:0/2102327419 >> v1:172.21.15.148:6789/0 conn(0x7f611c071960 legacy=0x7f611c15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.093+0000 7f61218c6640 1 -- 172.21.15.105:0/2102327419 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611c071df0 con 0x7f611c0b4290 2023-11-28T18:30:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.093+0000 7f61220c7640 1 --2- 172.21.15.105:0/2102327419 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f611c15b030 0x7f611c0709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:02.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.097+0000 7f61218c6640 1 --2- 172.21.15.105:0/2102327419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f611c0b4290 0x7f611c0b4670 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f6118009870 tx=0x7f611801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7d289b93f7eac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.097+0000 7f61208c4640 1 -- 172.21.15.105:0/2102327419 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61180ab620 con 0x7f611c0b4290 2023-11-28T18:30:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.097+0000 7f61208c4640 1 -- 172.21.15.105:0/2102327419 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61180ab780 con 0x7f611c0b4290 2023-11-28T18:30:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.097+0000 7f61208c4640 1 -- 172.21.15.105:0/2102327419 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61180abb10 con 0x7f611c0b4290 2023-11-28T18:30:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.097+0000 7f6123b2a640 1 -- 172.21.15.105:0/2102327419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f611c0b4290 msgr2=0x7f611c0b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.097+0000 7f6123b2a640 1 --2- 172.21.15.105:0/2102327419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f611c0b4290 0x7f611c0b4670 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f6118009870 tx=0x7f611801c510 comp rx=0 tx=0).stop 2023-11-28T18:30:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.097+0000 7f6123b2a640 1 -- 172.21.15.105:0/2102327419 shutdown_connections 2023-11-28T18:30:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.097+0000 7f6123b2a640 1 --2- 172.21.15.105:0/2102327419 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f611c15b030 0x7f611c0709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.097+0000 7f6123b2a640 1 --2- 172.21.15.105:0/2102327419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f611c0b4290 0x7f611c0b4670 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.097+0000 7f6123b2a640 1 -- 172.21.15.105:0/2102327419 >> 172.21.15.105:0/2102327419 conn(0x7f611c0ceb90 msgr2=0x7f611c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.097+0000 7f6123b2a640 1 -- 172.21.15.105:0/2102327419 shutdown_connections 2023-11-28T18:30:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.097+0000 7f6123b2a640 1 -- 172.21.15.105:0/2102327419 wait complete. 2023-11-28T18:30:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.097+0000 7f6123b2a640 1 Processor -- start 2023-11-28T18:30:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.097+0000 7f6123b2a640 1 -- start start 2023-11-28T18:30:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f6123b2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f611c0b4290 0x7f611c14b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f6123b2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f611c15b030 0x7f611c0cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f6123b2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f611c0cde60 0x7f611c164830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f6123b2a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c154130 con 0x7f611c0b4290 2023-11-28T18:30:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f6123b2a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c0ce310 con 0x7f611c15b030 2023-11-28T18:30:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f6123b2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611c153d90 con 0x7f611c0cde60 2023-11-28T18:30:02.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f61218c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f611c0b4290 0x7f611c14b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:02.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f61218c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f611c0b4290 0x7f611c14b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50004/0 (socket says 172.21.15.105:50004) 2023-11-28T18:30:02.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f61218c6640 1 -- 172.21.15.105:0/3064706772 learned_addr learned my addr 172.21.15.105:0/3064706772 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:02.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f61210c5640 1 --2- 172.21.15.105:0/3064706772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f611c15b030 0x7f611c0cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:02.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f61220c7640 1 --2- 172.21.15.105:0/3064706772 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f611c0cde60 0x7f611c164830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:02.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f61218c6640 1 -- 172.21.15.105:0/3064706772 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f611c0cde60 msgr2=0x7f611c164830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:02.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f61218c6640 1 --2- 172.21.15.105:0/3064706772 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f611c0cde60 0x7f611c164830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f61218c6640 1 -- 172.21.15.105:0/3064706772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f611c15b030 msgr2=0x7f611c0cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:02.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f61218c6640 1 --2- 172.21.15.105:0/3064706772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f611c15b030 0x7f611c0cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f61218c6640 1 -- 172.21.15.105:0/3064706772 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61180094d0 con 0x7f611c0b4290 2023-11-28T18:30:02.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f61218c6640 1 --2- 172.21.15.105:0/3064706772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f611c0b4290 0x7f611c14b8f0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f61180ad040 tx=0x7f611801ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:02.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f61127fc640 1 -- 172.21.15.105:0/3064706772 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61180be3f0 con 0x7f611c0b4290 2023-11-28T18:30:02.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f61127fc640 1 -- 172.21.15.105:0/3064706772 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61180be570 con 0x7f611c0b4290 2023-11-28T18:30:02.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f6123b2a640 1 -- 172.21.15.105:0/3064706772 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f611c165560 con 0x7f611c0b4290 2023-11-28T18:30:02.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f61127fc640 1 -- 172.21.15.105:0/3064706772 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61180be920 con 0x7f611c0b4290 2023-11-28T18:30:02.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f6123b2a640 1 -- 172.21.15.105:0/3064706772 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f611c165b20 con 0x7f611c0b4290 2023-11-28T18:30:02.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f6123b2a640 1 -- 172.21.15.105:0/3064706772 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f611c153bf0 con 0x7f611c0b4290 2023-11-28T18:30:02.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.101+0000 7f61127fc640 1 -- 172.21.15.105:0/3064706772 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f61180c7440 con 0x7f611c0b4290 2023-11-28T18:30:02.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.105+0000 7f61127fc640 1 --2- 172.21.15.105:0/3064706772 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f61000668d0 0x7f6100068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:02.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.105+0000 7f61127fc640 1 -- 172.21.15.105:0/3064706772 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f611811eaf0 con 0x7f611c0b4290 2023-11-28T18:30:02.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.105+0000 7f61210c5640 1 --2- 172.21.15.105:0/3064706772 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f61000668d0 0x7f6100068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:02.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.105+0000 7f61210c5640 1 --2- 172.21.15.105:0/3064706772 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f61000668d0 0x7f6100068d90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f611c06dd70 tx=0x7f610c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:02.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.109+0000 7f61127fc640 1 -- 172.21.15.105:0/3064706772 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f611c153bf0 con 0x7f611c0b4290 2023-11-28T18:30:02.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.257+0000 7f6123b2a640 1 -- 172.21.15.105:0/3064706772 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f611c154270 con 0x7f611c0b4290 2023-11-28T18:30:02.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.257+0000 7f61127fc640 1 -- 172.21.15.105:0/3064706772 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+13123 (secure 0 0 0) 0x7f611c154270 con 0x7f611c0b4290 2023-11-28T18:30:02.259 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:02.260 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":99,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:30:01.568601+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:30:02.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.261+0000 7f6123b2a640 1 -- 172.21.15.105:0/3064706772 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f61000668d0 msgr2=0x7f6100068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:02.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.261+0000 7f6123b2a640 1 --2- 172.21.15.105:0/3064706772 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f61000668d0 0x7f6100068d90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f611c06dd70 tx=0x7f610c009040 comp rx=0 tx=0).stop 2023-11-28T18:30:02.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.261+0000 7f6123b2a640 1 -- 172.21.15.105:0/3064706772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f611c0b4290 msgr2=0x7f611c14b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:02.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.261+0000 7f6123b2a640 1 --2- 172.21.15.105:0/3064706772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f611c0b4290 0x7f611c14b8f0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f61180ad040 tx=0x7f611801ad50 comp rx=0 tx=0).stop 2023-11-28T18:30:02.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.261+0000 7f6123b2a640 1 -- 172.21.15.105:0/3064706772 shutdown_connections 2023-11-28T18:30:02.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.261+0000 7f6123b2a640 1 --2- 172.21.15.105:0/3064706772 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f611c0cde60 0x7f611c164830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.261+0000 7f6123b2a640 1 --2- 172.21.15.105:0/3064706772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f611c15b030 0x7f611c0cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.261+0000 7f6123b2a640 1 --2- 172.21.15.105:0/3064706772 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f61000668d0 0x7f6100068d90 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.261+0000 7f6123b2a640 1 --2- 172.21.15.105:0/3064706772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f611c0b4290 0x7f611c14b8f0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.261+0000 7f6123b2a640 1 -- 172.21.15.105:0/3064706772 >> 172.21.15.105:0/3064706772 conn(0x7f611c0ceb90 msgr2=0x7f611c0b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:02.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.261+0000 7f6123b2a640 1 -- 172.21.15.105:0/3064706772 shutdown_connections 2023-11-28T18:30:02.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.261+0000 7f6123b2a640 1 -- 172.21.15.105:0/3064706772 wait complete. 2023-11-28T18:30:02.281 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:30:02.282 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:30:02.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.421+0000 7f4c4f380640 1 Processor -- start 2023-11-28T18:30:02.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.421+0000 7f4c4f380640 1 -- start start 2023-11-28T18:30:02.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.421+0000 7f4c4f380640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c48154070 0x7f4c4815d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:02.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.421+0000 7f4c4f380640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4c4806bf50 0x7f4c480ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:02.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.421+0000 7f4c4f380640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c480cc7f0 con 0x7f4c480adb80 2023-11-28T18:30:02.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.421+0000 7f4c4f380640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c48072540 con 0x7f4c4806bf50 2023-11-28T18:30:02.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.421+0000 7f4c4f380640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4815dd10 con 0x7f4c48154070 2023-11-28T18:30:02.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.421+0000 7f4c4c91b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4c4806bf50 0x7f4c480ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:02.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.425+0000 7f4c4c91b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4c4806bf50 0x7f4c480ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50018/0 (socket says 172.21.15.105:50018) 2023-11-28T18:30:02.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.425+0000 7f4c4c91b640 1 -- 172.21.15.105:0/3344752911 learned_addr learned my addr 172.21.15.105:0/3344752911 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:02.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.425+0000 7f4c4d11c640 1 --2- 172.21.15.105:0/3344752911 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c48154070 0x7f4c4815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:02.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.425+0000 7f4c4c91b640 1 -- 172.21.15.105:0/3344752911 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c48154070 msgr2=0x7f4c4815d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:02.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.425+0000 7f4c4c91b640 1 --2- 172.21.15.105:0/3344752911 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c48154070 0x7f4c4815d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.425+0000 7f4c4c91b640 1 -- 172.21.15.105:0/3344752911 >> v1:172.21.15.148:6789/0 conn(0x7f4c480adb80 legacy=0x7f4c480acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:02.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.425+0000 7f4c4c91b640 1 -- 172.21.15.105:0/3344752911 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4815dff0 con 0x7f4c4806bf50 2023-11-28T18:30:02.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.425+0000 7f4c4c91b640 1 --2- 172.21.15.105:0/3344752911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4806bf50 0x7f4c480ce310 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009870 tx=0x7f4c3801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18b30258be2e53d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:02.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.425+0000 7f4c3ffff640 1 -- 172.21.15.105:0/3344752911 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c380aa670 con 0x7f4c4806bf50 2023-11-28T18:30:02.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.425+0000 7f4c3ffff640 1 -- 172.21.15.105:0/3344752911 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c380aa7d0 con 0x7f4c4806bf50 2023-11-28T18:30:02.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.425+0000 7f4c3ffff640 1 -- 172.21.15.105:0/3344752911 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c380aab60 con 0x7f4c4806bf50 2023-11-28T18:30:02.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4f380640 1 -- 172.21.15.105:0/3344752911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4806bf50 msgr2=0x7f4c480ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:02.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4f380640 1 --2- 172.21.15.105:0/3344752911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4806bf50 0x7f4c480ce310 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009870 tx=0x7f4c3801c510 comp rx=0 tx=0).stop 2023-11-28T18:30:02.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4f380640 1 -- 172.21.15.105:0/3344752911 shutdown_connections 2023-11-28T18:30:02.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4f380640 1 --2- 172.21.15.105:0/3344752911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4806bf50 0x7f4c480ce310 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4f380640 1 --2- 172.21.15.105:0/3344752911 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c48154070 0x7f4c4815d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4f380640 1 -- 172.21.15.105:0/3344752911 >> 172.21.15.105:0/3344752911 conn(0x7f4c480ceb90 msgr2=0x7f4c480cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:02.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4f380640 1 -- 172.21.15.105:0/3344752911 shutdown_connections 2023-11-28T18:30:02.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4f380640 1 -- 172.21.15.105:0/3344752911 wait complete. 2023-11-28T18:30:02.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4f380640 1 Processor -- start 2023-11-28T18:30:02.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4f380640 1 -- start start 2023-11-28T18:30:02.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4f380640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4806bf50 0x7f4c481451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:02.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4f380640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c480adb80 0x7f4c4814b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:02.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4f380640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c48154070 0x7f4c4814bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:02.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4f380640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c48152ce0 con 0x7f4c4806bf50 2023-11-28T18:30:02.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4f380640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c48152fc0 con 0x7f4c480adb80 2023-11-28T18:30:02.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4f380640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c481532a0 con 0x7f4c48154070 2023-11-28T18:30:02.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4d11c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4806bf50 0x7f4c481451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:02.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4d11c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4806bf50 0x7f4c481451b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50020/0 (socket says 172.21.15.105:50020) 2023-11-28T18:30:02.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4d11c640 1 -- 172.21.15.105:0/798124427 learned_addr learned my addr 172.21.15.105:0/798124427 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:02.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4c91b640 1 --2- 172.21.15.105:0/798124427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c480adb80 0x7f4c4814b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:02.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4d91d640 1 --2- 172.21.15.105:0/798124427 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c48154070 0x7f4c4814bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:02.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4d11c640 1 -- 172.21.15.105:0/798124427 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c48154070 msgr2=0x7f4c4814bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:02.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4d11c640 1 --2- 172.21.15.105:0/798124427 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c48154070 0x7f4c4814bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4d11c640 1 -- 172.21.15.105:0/798124427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c480adb80 msgr2=0x7f4c4814b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:02.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4d11c640 1 --2- 172.21.15.105:0/798124427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c480adb80 0x7f4c4814b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4d11c640 1 -- 172.21.15.105:0/798124427 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c380094d0 con 0x7f4c4806bf50 2023-11-28T18:30:02.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.429+0000 7f4c4d91d640 1 --2- 172.21.15.105:0/798124427 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c48154070 0x7f4c4814bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:02.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.433+0000 7f4c4d11c640 1 --2- 172.21.15.105:0/798124427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4806bf50 0x7f4c481451b0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400e870 tx=0x7f4c4400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:02.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.433+0000 7f4c3dffb640 1 -- 172.21.15.105:0/798124427 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c44009510 con 0x7f4c4806bf50 2023-11-28T18:30:02.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.433+0000 7f4c3dffb640 1 -- 172.21.15.105:0/798124427 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4400eec0 con 0x7f4c4806bf50 2023-11-28T18:30:02.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.433+0000 7f4c3dffb640 1 -- 172.21.15.105:0/798124427 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4400cb40 con 0x7f4c4806bf50 2023-11-28T18:30:02.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.433+0000 7f4c4f380640 1 -- 172.21.15.105:0/798124427 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c480ae520 con 0x7f4c4806bf50 2023-11-28T18:30:02.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.433+0000 7f4c4f380640 1 -- 172.21.15.105:0/798124427 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c48152990 con 0x7f4c4806bf50 2023-11-28T18:30:02.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.433+0000 7f4c4f380640 1 -- 172.21.15.105:0/798124427 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c48067570 con 0x7f4c4806bf50 2023-11-28T18:30:02.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.433+0000 7f4c3dffb640 1 -- 172.21.15.105:0/798124427 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f4c4400cca0 con 0x7f4c4806bf50 2023-11-28T18:30:02.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.433+0000 7f4c3dffb640 1 --2- 172.21.15.105:0/798124427 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4c2c0668d0 0x7f4c2c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:02.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.433+0000 7f4c3dffb640 1 -- 172.21.15.105:0/798124427 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4c44084310 con 0x7f4c4806bf50 2023-11-28T18:30:02.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.433+0000 7f4c4c91b640 1 --2- 172.21.15.105:0/798124427 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4c2c0668d0 0x7f4c2c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:02.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.437+0000 7f4c4c91b640 1 --2- 172.21.15.105:0/798124427 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4c2c0668d0 0x7f4c2c068d90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4c380ae000 tx=0x7f4c380b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:02.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.437+0000 7f4c3dffb640 1 -- 172.21.15.105:0/798124427 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4c44051800 con 0x7f4c4806bf50 2023-11-28T18:30:02.494 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:30:02.497 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:30:02.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.593+0000 7f4c4f380640 1 -- 172.21.15.105:0/798124427 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4c48152e20 con 0x7f4c4806bf50 2023-11-28T18:30:02.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.597+0000 7f4c3dffb640 1 -- 172.21.15.105:0/798124427 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4c48152e20 con 0x7f4c4806bf50 2023-11-28T18:30:02.596 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:30:02.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.597+0000 7f4c4f380640 1 -- 172.21.15.105:0/798124427 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4c2c0668d0 msgr2=0x7f4c2c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:02.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.597+0000 7f4c4f380640 1 --2- 172.21.15.105:0/798124427 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4c2c0668d0 0x7f4c2c068d90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4c380ae000 tx=0x7f4c380b5040 comp rx=0 tx=0).stop 2023-11-28T18:30:02.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.597+0000 7f4c4f380640 1 -- 172.21.15.105:0/798124427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4806bf50 msgr2=0x7f4c481451b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:02.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.597+0000 7f4c4f380640 1 --2- 172.21.15.105:0/798124427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4806bf50 0x7f4c481451b0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400e870 tx=0x7f4c4400ec70 comp rx=0 tx=0).stop 2023-11-28T18:30:02.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.597+0000 7f4c4f380640 1 -- 172.21.15.105:0/798124427 shutdown_connections 2023-11-28T18:30:02.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.597+0000 7f4c4f380640 1 --2- 172.21.15.105:0/798124427 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c48154070 0x7f4c4814bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.597+0000 7f4c4f380640 1 --2- 172.21.15.105:0/798124427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c480adb80 0x7f4c4814b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.597+0000 7f4c4f380640 1 --2- 172.21.15.105:0/798124427 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4c2c0668d0 0x7f4c2c068d90 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.597+0000 7f4c4f380640 1 --2- 172.21.15.105:0/798124427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4806bf50 0x7f4c481451b0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.597+0000 7f4c4f380640 1 -- 172.21.15.105:0/798124427 >> 172.21.15.105:0/798124427 conn(0x7f4c480ceb90 msgr2=0x7f4c480ac800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:02.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.597+0000 7f4c4f380640 1 -- 172.21.15.105:0/798124427 shutdown_connections 2023-11-28T18:30:02.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.597+0000 7f4c4f380640 1 -- 172.21.15.105:0/798124427 wait complete. 2023-11-28T18:30:02.617 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T18:30:02.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.797+0000 7f99fdd93640 1 Processor -- start 2023-11-28T18:30:02.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.797+0000 7f99fdd93640 1 -- start start 2023-11-28T18:30:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.797+0000 7f99fdd93640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99f81607b0 0x7f99f80d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.797+0000 7f99fdd93640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f99f80b3eb0 0x7f99f80d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.797+0000 7f99fdd93640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f80ce530 con 0x7f99f80d3a50 2023-11-28T18:30:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.797+0000 7f99fdd93640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f80d0eb0 con 0x7f99f80b3eb0 2023-11-28T18:30:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.797+0000 7f99fdd93640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f80d1150 con 0x7f99f81607b0 2023-11-28T18:30:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.797+0000 7f99fcd91640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99f81607b0 0x7f99f80d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.797+0000 7f99fd592640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f99f80b3eb0 0x7f99f80d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.797+0000 7f99fcd91640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99f81607b0 0x7f99f80d14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:57610/0 (socket says 172.21.15.105:57610) 2023-11-28T18:30:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.797+0000 7f99fcd91640 1 -- 172.21.15.105:0/2348538951 learned_addr learned my addr 172.21.15.105:0/2348538951 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.797+0000 7f99fcd91640 1 -- 172.21.15.105:0/2348538951 >> v2:172.21.15.148:3300/0 conn(0x7f99f80b3eb0 msgr2=0x7f99f80d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.797+0000 7f99fcd91640 1 --2- 172.21.15.105:0/2348538951 >> v2:172.21.15.148:3300/0 conn(0x7f99f80b3eb0 0x7f99f80d1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.797+0000 7f99fcd91640 1 -- 172.21.15.105:0/2348538951 >> v1:172.21.15.148:6789/0 conn(0x7f99f80d3a50 legacy=0x7f99f80cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.797+0000 7f99fcd91640 1 -- 172.21.15.105:0/2348538951 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f80ce780 con 0x7f99f81607b0 2023-11-28T18:30:02.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.801+0000 7f99fcd91640 1 --2- 172.21.15.105:0/2348538951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99f81607b0 0x7f99f80d14f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009870 tx=0x7f99ec01c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1a0713ef257c332 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:02.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.801+0000 7f99f77fe640 1 -- 172.21.15.105:0/2348538951 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec0ab600 con 0x7f99f81607b0 2023-11-28T18:30:02.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.801+0000 7f99f77fe640 1 -- 172.21.15.105:0/2348538951 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec0a7050 con 0x7f99f81607b0 2023-11-28T18:30:02.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.801+0000 7f99f77fe640 1 -- 172.21.15.105:0/2348538951 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec0ab980 con 0x7f99f81607b0 2023-11-28T18:30:02.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.801+0000 7f99fdd93640 1 -- 172.21.15.105:0/2348538951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99f81607b0 msgr2=0x7f99f80d14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:02.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.801+0000 7f99fdd93640 1 --2- 172.21.15.105:0/2348538951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99f81607b0 0x7f99f80d14f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009870 tx=0x7f99ec01c4b0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.801+0000 7f99fdd93640 1 -- 172.21.15.105:0/2348538951 shutdown_connections 2023-11-28T18:30:02.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.801+0000 7f99fdd93640 1 --2- 172.21.15.105:0/2348538951 >> v2:172.21.15.148:3300/0 conn(0x7f99f80b3eb0 0x7f99f80d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.801+0000 7f99fdd93640 1 --2- 172.21.15.105:0/2348538951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99f81607b0 0x7f99f80d14f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.801+0000 7f99fdd93640 1 -- 172.21.15.105:0/2348538951 >> 172.21.15.105:0/2348538951 conn(0x7f99f80d2f30 msgr2=0x7f99f80d3340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:02.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.801+0000 7f99fdd93640 1 -- 172.21.15.105:0/2348538951 shutdown_connections 2023-11-28T18:30:02.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.801+0000 7f99fdd93640 1 -- 172.21.15.105:0/2348538951 wait complete. 2023-11-28T18:30:02.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99fdd93640 1 Processor -- start 2023-11-28T18:30:02.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99fdd93640 1 -- start start 2023-11-28T18:30:02.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99fdd93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99f80b3eb0 0x7f99f815d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:02.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99fdd93640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99f80d3a50 0x7f99f81543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:02.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99fdd93640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99f81607b0 0x7f99f80c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:02.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99fdd93640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f8154b70 con 0x7f99f80b3eb0 2023-11-28T18:30:02.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99fdd93640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f8154dc0 con 0x7f99f81607b0 2023-11-28T18:30:02.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99fdd93640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f81550a0 con 0x7f99f80d3a50 2023-11-28T18:30:02.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99fcd91640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99f80b3eb0 0x7f99f815d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:02.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99fcd91640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99f80b3eb0 0x7f99f815d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50022/0 (socket says 172.21.15.105:50022) 2023-11-28T18:30:02.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99fcd91640 1 -- 172.21.15.105:0/529848024 learned_addr learned my addr 172.21.15.105:0/529848024 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:02.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99fd592640 1 --2- 172.21.15.105:0/529848024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99f81607b0 0x7f99f80c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:02.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99f7fff640 1 --2- 172.21.15.105:0/529848024 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99f80d3a50 0x7f99f81543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:02.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99fcd91640 1 -- 172.21.15.105:0/529848024 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99f80d3a50 msgr2=0x7f99f81543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:02.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99fcd91640 1 --2- 172.21.15.105:0/529848024 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99f80d3a50 0x7f99f81543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99fcd91640 1 -- 172.21.15.105:0/529848024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99f81607b0 msgr2=0x7f99f80c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:02.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99fcd91640 1 --2- 172.21.15.105:0/529848024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99f81607b0 0x7f99f80c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99fcd91640 1 -- 172.21.15.105:0/529848024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ec0094d0 con 0x7f99f80b3eb0 2023-11-28T18:30:02.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99fcd91640 1 --2- 172.21.15.105:0/529848024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99f80b3eb0 0x7f99f815d1b0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f99ec01a0e0 tx=0x7f99ec019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:02.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99f57fa640 1 -- 172.21.15.105:0/529848024 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec0abd10 con 0x7f99f80b3eb0 2023-11-28T18:30:02.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99f57fa640 1 -- 172.21.15.105:0/529848024 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec0be410 con 0x7f99f80b3eb0 2023-11-28T18:30:02.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99fdd93640 1 -- 172.21.15.105:0/529848024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f8155380 con 0x7f99f80b3eb0 2023-11-28T18:30:02.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99f57fa640 1 -- 172.21.15.105:0/529848024 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec0be7c0 con 0x7f99f80b3eb0 2023-11-28T18:30:02.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.805+0000 7f99fdd93640 1 -- 172.21.15.105:0/529848024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f8154910 con 0x7f99f80b3eb0 2023-11-28T18:30:02.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.809+0000 7f99fdd93640 1 -- 172.21.15.105:0/529848024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f81551e0 con 0x7f99f80b3eb0 2023-11-28T18:30:02.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.809+0000 7f99f57fa640 1 -- 172.21.15.105:0/529848024 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f99ec0b2070 con 0x7f99f80b3eb0 2023-11-28T18:30:02.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.809+0000 7f99f57fa640 1 --2- 172.21.15.105:0/529848024 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99dc0668d0 0x7f99dc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:02.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.809+0000 7f99f57fa640 1 -- 172.21.15.105:0/529848024 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f99ec11e0b0 con 0x7f99f80b3eb0 2023-11-28T18:30:02.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.809+0000 7f99f7fff640 1 --2- 172.21.15.105:0/529848024 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99dc0668d0 0x7f99dc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:02.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.809+0000 7f99f7fff640 1 --2- 172.21.15.105:0/529848024 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99dc0668d0 0x7f99dc068d90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f99f80cb490 tx=0x7f99e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:02.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.813+0000 7f99f57fa640 1 -- 172.21.15.105:0/529848024 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f99f81551e0 con 0x7f99f80b3eb0 2023-11-28T18:30:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.961+0000 7f99fdd93640 1 -- 172.21.15.105:0/529848024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f99f81551e0 con 0x7f99f80b3eb0 2023-11-28T18:30:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.965+0000 7f99f57fa640 1 -- 172.21.15.105:0/529848024 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v99) v1 ==== 84+0+65 (secure 0 0 0) 0x7f99f81551e0 con 0x7f99f80b3eb0 2023-11-28T18:30:02.964 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:02.964 INFO:teuthology.orchestra.run.smithi105.stdout:[ 2023-11-28T18:30:02.965 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:02.965 INFO:teuthology.orchestra.run.smithi105.stdout: "poolnum": 1, 2023-11-28T18:30:02.965 INFO:teuthology.orchestra.run.smithi105.stdout: "poolname": ".mgr" 2023-11-28T18:30:02.965 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:02.965 INFO:teuthology.orchestra.run.smithi105.stdout:] 2023-11-28T18:30:02.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.965+0000 7f99fdd93640 1 -- 172.21.15.105:0/529848024 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99dc0668d0 msgr2=0x7f99dc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:02.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.965+0000 7f99fdd93640 1 --2- 172.21.15.105:0/529848024 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99dc0668d0 0x7f99dc068d90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f99f80cb490 tx=0x7f99e4009040 comp rx=0 tx=0).stop 2023-11-28T18:30:02.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.965+0000 7f99fdd93640 1 -- 172.21.15.105:0/529848024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99f80b3eb0 msgr2=0x7f99f815d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:02.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.965+0000 7f99fdd93640 1 --2- 172.21.15.105:0/529848024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99f80b3eb0 0x7f99f815d1b0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f99ec01a0e0 tx=0x7f99ec019250 comp rx=0 tx=0).stop 2023-11-28T18:30:02.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.965+0000 7f99fdd93640 1 -- 172.21.15.105:0/529848024 shutdown_connections 2023-11-28T18:30:02.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.965+0000 7f99fdd93640 1 --2- 172.21.15.105:0/529848024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99f81607b0 0x7f99f80c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.965+0000 7f99fdd93640 1 --2- 172.21.15.105:0/529848024 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99f80d3a50 0x7f99f81543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.965+0000 7f99fdd93640 1 --2- 172.21.15.105:0/529848024 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99dc0668d0 0x7f99dc068d90 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.965+0000 7f99fdd93640 1 --2- 172.21.15.105:0/529848024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99f80b3eb0 0x7f99f815d1b0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.965+0000 7f99fdd93640 1 -- 172.21.15.105:0/529848024 >> 172.21.15.105:0/529848024 conn(0x7f99f80d2f30 msgr2=0x7f99f815d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.965+0000 7f99fdd93640 1 -- 172.21.15.105:0/529848024 shutdown_connections 2023-11-28T18:30:02.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:02.965+0000 7f99fdd93640 1 -- 172.21.15.105:0/529848024 wait complete. 2023-11-28T18:30:02.986 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T18:30:02.987 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T18:30:03.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.125+0000 7f5aee12a640 1 Processor -- start 2023-11-28T18:30:03.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.125+0000 7f5aee12a640 1 -- start start 2023-11-28T18:30:03.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.129+0000 7f5aee12a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5ae815d300 0x7f5ae80d37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:03.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.129+0000 7f5aee12a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ae80d3ce0 0x7f5ae80d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:03.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.129+0000 7f5aee12a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae80d0c60 con 0x7f5ae80d1b90 2023-11-28T18:30:03.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.129+0000 7f5aee12a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae80d1470 con 0x7f5ae815d300 2023-11-28T18:30:03.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.129+0000 7f5aee12a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae80d2040 con 0x7f5ae80d3ce0 2023-11-28T18:30:03.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.129+0000 7f5ae7fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5ae80d1b90 0x7f5ae80cdb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40160/0 (socket says 172.21.15.105:40160) 2023-11-28T18:30:03.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.129+0000 7f5ae7fff640 1 -- 172.21.15.105:0/2092596277 learned_addr learned my addr 172.21.15.105:0/2092596277 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:03.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.129+0000 7f5ae6ffd640 1 --2- 172.21.15.105:0/2092596277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ae80d3ce0 0x7f5ae80d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:03.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.129+0000 7f5ae77fe640 1 --2- 172.21.15.105:0/2092596277 >> v2:172.21.15.148:3300/0 conn(0x7f5ae815d300 0x7f5ae80d37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:03.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.129+0000 7f5ae67fc640 1 -- 172.21.15.105:0/2092596277 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2094331842 0 0) 0x7f5ae80d0c60 con 0x7f5ae80d1b90 2023-11-28T18:30:03.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.129+0000 7f5ae6ffd640 1 -- 172.21.15.105:0/2092596277 >> v2:172.21.15.148:3300/0 conn(0x7f5ae815d300 msgr2=0x7f5ae80d37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:03.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.129+0000 7f5ae6ffd640 1 --2- 172.21.15.105:0/2092596277 >> v2:172.21.15.148:3300/0 conn(0x7f5ae815d300 0x7f5ae80d37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:03.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.129+0000 7f5ae6ffd640 1 -- 172.21.15.105:0/2092596277 >> v1:172.21.15.105:6789/0 conn(0x7f5ae80d1b90 legacy=0x7f5ae80cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:03.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.129+0000 7f5ae6ffd640 1 -- 172.21.15.105:0/2092596277 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae80d0eb0 con 0x7f5ae80d3ce0 2023-11-28T18:30:03.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.133+0000 7f5ae6ffd640 1 --2- 172.21.15.105:0/2092596277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ae80d3ce0 0x7f5ae80d1650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0097c0 tx=0x7f5adc01c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=808fcabd3e51a10c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:03.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.133+0000 7f5ae67fc640 1 -- 172.21.15.105:0/2092596277 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc0ab600 con 0x7f5ae80d3ce0 2023-11-28T18:30:03.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.133+0000 7f5ae67fc640 1 -- 172.21.15.105:0/2092596277 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5adc0a7050 con 0x7f5ae80d3ce0 2023-11-28T18:30:03.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.133+0000 7f5ae67fc640 1 -- 172.21.15.105:0/2092596277 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc0ab980 con 0x7f5ae80d3ce0 2023-11-28T18:30:03.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.133+0000 7f5aee12a640 1 -- 172.21.15.105:0/2092596277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ae80d3ce0 msgr2=0x7f5ae80d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:03.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.133+0000 7f5aee12a640 1 --2- 172.21.15.105:0/2092596277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ae80d3ce0 0x7f5ae80d1650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0097c0 tx=0x7f5adc01c3f0 comp rx=0 tx=0).stop 2023-11-28T18:30:03.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.133+0000 7f5aee12a640 1 -- 172.21.15.105:0/2092596277 shutdown_connections 2023-11-28T18:30:03.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.133+0000 7f5aee12a640 1 --2- 172.21.15.105:0/2092596277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ae80d3ce0 0x7f5ae80d1650 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:03.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.133+0000 7f5aee12a640 1 --2- 172.21.15.105:0/2092596277 >> v2:172.21.15.148:3300/0 conn(0x7f5ae815d300 0x7f5ae80d37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:03.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.133+0000 7f5aee12a640 1 -- 172.21.15.105:0/2092596277 >> 172.21.15.105:0/2092596277 conn(0x7f5ae809c360 msgr2=0x7f5ae80d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:03.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.133+0000 7f5aee12a640 1 -- 172.21.15.105:0/2092596277 shutdown_connections 2023-11-28T18:30:03.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.133+0000 7f5aee12a640 1 -- 172.21.15.105:0/2092596277 wait complete. 2023-11-28T18:30:03.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.133+0000 7f5aee12a640 1 Processor -- start 2023-11-28T18:30:03.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5aee12a640 1 -- start start 2023-11-28T18:30:03.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5aee12a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ae80d1b90 0x7f5ae8155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:03.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5aee12a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ae80d3ce0 0x7f5ae8160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:03.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5aee12a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ae815d300 0x7f5ae8158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:03.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5aee12a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae8155b30 con 0x7f5ae80d1b90 2023-11-28T18:30:03.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5aee12a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae8155200 con 0x7f5ae80d3ce0 2023-11-28T18:30:03.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5aee12a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae81554e0 con 0x7f5ae815d300 2023-11-28T18:30:03.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5ae77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ae80d1b90 0x7f5ae8155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:03.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5ae77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ae80d1b90 0x7f5ae8155fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50026/0 (socket says 172.21.15.105:50026) 2023-11-28T18:30:03.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5ae77fe640 1 -- 172.21.15.105:0/1755513655 learned_addr learned my addr 172.21.15.105:0/1755513655 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:03.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5ae6ffd640 1 --2- 172.21.15.105:0/1755513655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ae80d3ce0 0x7f5ae8160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:03.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5ae7fff640 1 --2- 172.21.15.105:0/1755513655 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ae815d300 0x7f5ae8158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:03.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5ae77fe640 1 -- 172.21.15.105:0/1755513655 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ae815d300 msgr2=0x7f5ae8158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5ae77fe640 1 --2- 172.21.15.105:0/1755513655 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ae815d300 0x7f5ae8158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5ae77fe640 1 -- 172.21.15.105:0/1755513655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ae80d3ce0 msgr2=0x7f5ae8160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5ae77fe640 1 --2- 172.21.15.105:0/1755513655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ae80d3ce0 0x7f5ae8160310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5ae77fe640 1 -- 172.21.15.105:0/1755513655 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5adc009420 con 0x7f5ae80d1b90 2023-11-28T18:30:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5ae77fe640 1 --2- 172.21.15.105:0/1755513655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ae80d1b90 0x7f5ae8155fc0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800e930 tx=0x7f5ad800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5acffff640 1 -- 172.21.15.105:0/1755513655 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8009690 con 0x7f5ae80d1b90 2023-11-28T18:30:03.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5acffff640 1 -- 172.21.15.105:0/1755513655 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad800ca00 con 0x7f5ae80d1b90 2023-11-28T18:30:03.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5acffff640 1 -- 172.21.15.105:0/1755513655 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad800cd70 con 0x7f5ae80d1b90 2023-11-28T18:30:03.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5aee12a640 1 -- 172.21.15.105:0/1755513655 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae81557c0 con 0x7f5ae80d1b90 2023-11-28T18:30:03.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5aee12a640 1 -- 172.21.15.105:0/1755513655 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae8158690 con 0x7f5ae80d1b90 2023-11-28T18:30:03.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.137+0000 7f5aee12a640 1 -- 172.21.15.105:0/1755513655 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae8067570 con 0x7f5ae80d1b90 2023-11-28T18:30:03.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.141+0000 7f5acffff640 1 -- 172.21.15.105:0/1755513655 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5ad8013470 con 0x7f5ae80d1b90 2023-11-28T18:30:03.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.141+0000 7f5acffff640 1 --2- 172.21.15.105:0/1755513655 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5ad4066900 0x7f5ad4068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:03.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.141+0000 7f5acffff640 1 -- 172.21.15.105:0/1755513655 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f5ad8084450 con 0x7f5ae80d1b90 2023-11-28T18:30:03.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.141+0000 7f5ae6ffd640 1 --2- 172.21.15.105:0/1755513655 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5ad4066900 0x7f5ad4068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:03.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.141+0000 7f5ae6ffd640 1 --2- 172.21.15.105:0/1755513655 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5ad4066900 0x7f5ad4068dc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0ad000 tx=0x7f5adc0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:03.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.145+0000 7f5acffff640 1 -- 172.21.15.105:0/1755513655 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5ad8051970 con 0x7f5ae80d1b90 2023-11-28T18:30:03.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:03.293+0000 7f5aee12a640 1 -- 172.21.15.105:0/1755513655 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5ae80c9b60 con 0x7f5ae80d1b90 2023-11-28T18:30:04.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.133+0000 7f5acffff640 1 -- 172.21.15.105:0/1755513655 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v100) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5ad8054f90 con 0x7f5ae80d1b90 2023-11-28T18:30:04.135 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' created 2023-11-28T18:30:04.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.137+0000 7f5aee12a640 1 -- 172.21.15.105:0/1755513655 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5ad4066900 msgr2=0x7f5ad4068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:04.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.137+0000 7f5aee12a640 1 --2- 172.21.15.105:0/1755513655 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5ad4066900 0x7f5ad4068dc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0ad000 tx=0x7f5adc0a6000 comp rx=0 tx=0).stop 2023-11-28T18:30:04.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.137+0000 7f5aee12a640 1 -- 172.21.15.105:0/1755513655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ae80d1b90 msgr2=0x7f5ae8155fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:04.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.137+0000 7f5aee12a640 1 --2- 172.21.15.105:0/1755513655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ae80d1b90 0x7f5ae8155fc0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800e930 tx=0x7f5ad800edf0 comp rx=0 tx=0).stop 2023-11-28T18:30:04.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.137+0000 7f5aee12a640 1 -- 172.21.15.105:0/1755513655 shutdown_connections 2023-11-28T18:30:04.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.137+0000 7f5aee12a640 1 --2- 172.21.15.105:0/1755513655 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ae815d300 0x7f5ae8158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:04.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.137+0000 7f5aee12a640 1 --2- 172.21.15.105:0/1755513655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ae80d3ce0 0x7f5ae8160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:04.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.137+0000 7f5aee12a640 1 --2- 172.21.15.105:0/1755513655 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5ad4066900 0x7f5ad4068dc0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:04.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.137+0000 7f5aee12a640 1 --2- 172.21.15.105:0/1755513655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ae80d1b90 0x7f5ae8155fc0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:04.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.137+0000 7f5aee12a640 1 -- 172.21.15.105:0/1755513655 >> 172.21.15.105:0/1755513655 conn(0x7f5ae809c360 msgr2=0x7f5ae806b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:04.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.137+0000 7f5aee12a640 1 -- 172.21.15.105:0/1755513655 shutdown_connections 2023-11-28T18:30:04.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.137+0000 7f5aee12a640 1 -- 172.21.15.105:0/1755513655 wait complete. 2023-11-28T18:30:04.166 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T18:30:04.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.313+0000 7fde1666b640 1 Processor -- start 2023-11-28T18:30:04.314 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.313+0000 7fde1666b640 1 -- start start 2023-11-28T18:30:04.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.313+0000 7fde1666b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fde100caf60 0x7fde100cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:04.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.313+0000 7fde1666b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde1006bf50 con 0x7fde10159f40 2023-11-28T18:30:04.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.313+0000 7fde1666b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde1015a320 con 0x7fde100ad850 2023-11-28T18:30:04.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.313+0000 7fde1666b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde100cb3c0 con 0x7fde100caf60 2023-11-28T18:30:04.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde0ffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fde10159f40 0x7fde100ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40166/0 (socket says 172.21.15.105:40166) 2023-11-28T18:30:04.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde0ffff640 1 -- 172.21.15.105:0/2589692845 learned_addr learned my addr 172.21.15.105:0/2589692845 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:04.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde0f7fe640 1 --2- 172.21.15.105:0/2589692845 >> v2:172.21.15.148:3300/0 conn(0x7fde100caf60 0x7fde100cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:04.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde0effd640 1 -- 172.21.15.105:0/2589692845 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491339727 0 0) 0x7fde1006bf50 con 0x7fde10159f40 2023-11-28T18:30:04.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde0effd640 1 -- 172.21.15.105:0/2589692845 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddf8003530 con 0x7fde10159f40 2023-11-28T18:30:04.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde0effd640 1 -- 172.21.15.105:0/2589692845 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 704723756 0 0) 0x7fddf8003530 con 0x7fde10159f40 2023-11-28T18:30:04.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde0effd640 1 -- 172.21.15.105:0/2589692845 >> v2:172.21.15.148:3300/0 conn(0x7fde100caf60 msgr2=0x7fde100cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:04.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde0effd640 1 --2- 172.21.15.105:0/2589692845 >> v2:172.21.15.148:3300/0 conn(0x7fde100caf60 0x7fde100cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde0effd640 1 -- 172.21.15.105:0/2589692845 >> v1:172.21.15.148:6789/0 conn(0x7fde100ad850 legacy=0x7fde100701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde0effd640 1 -- 172.21.15.105:0/2589692845 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde10071380 con 0x7fde10159f40 2023-11-28T18:30:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde0effd640 1 -- 172.21.15.105:0/2589692845 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fde00001680 con 0x7fde10159f40 2023-11-28T18:30:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde0effd640 1 -- 172.21.15.105:0/2589692845 >> v1:172.21.15.105:6789/0 conn(0x7fde10159f40 legacy=0x7fde100ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde0effd640 1 --2- 172.21.15.105:0/2589692845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddf8003980 0x7fddf8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde0effd640 1 --2- 172.21.15.105:0/2589692845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fddf80047d0 0x7fddf8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde0effd640 1 --2- 172.21.15.105:0/2589692845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddf8005190 0x7fddf800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde0effd640 1 -- 172.21.15.105:0/2589692845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde100cb3c0 con 0x7fddf8005190 2023-11-28T18:30:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde0effd640 1 -- 172.21.15.105:0/2589692845 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf800c130 con 0x7fddf8003980 2023-11-28T18:30:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde0effd640 1 -- 172.21.15.105:0/2589692845 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf800e500 con 0x7fddf80047d0 2023-11-28T18:30:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde14c08640 1 --2- 172.21.15.105:0/2589692845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddf8005190 0x7fddf800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:04.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde0ffff640 1 --2- 172.21.15.105:0/2589692845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddf8003980 0x7fddf8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:04.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde0f7fe640 1 --2- 172.21.15.105:0/2589692845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fddf80047d0 0x7fddf8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:04.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde14c08640 1 -- 172.21.15.105:0/2589692845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fddf80047d0 msgr2=0x7fddf8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:04.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde14c08640 1 --2- 172.21.15.105:0/2589692845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fddf80047d0 0x7fddf8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:04.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde14c08640 1 -- 172.21.15.105:0/2589692845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddf8003980 msgr2=0x7fddf8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:04.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde14c08640 1 --2- 172.21.15.105:0/2589692845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddf8003980 0x7fddf8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:04.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde14c08640 1 -- 172.21.15.105:0/2589692845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf8003530 con 0x7fddf8005190 2023-11-28T18:30:04.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde0f7fe640 1 --2- 172.21.15.105:0/2589692845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fddf80047d0 0x7fddf8004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:04.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.317+0000 7fde0ffff640 1 --2- 172.21.15.105:0/2589692845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddf8003980 0x7fddf8003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:04.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.321+0000 7fde14c08640 1 -- 172.21.15.105:0/2589692845 reap_dead start 2023-11-28T18:30:04.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.321+0000 7fde14c08640 1 --2- 172.21.15.105:0/2589692845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddf8005190 0x7fddf800b990 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fde08001e70 tx=0x7fde0801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377d313e9a63f3c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.321+0000 7fde0effd640 1 -- 172.21.15.105:0/2589692845 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde08003b80 con 0x7fddf8005190 2023-11-28T18:30:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.321+0000 7fde0effd640 1 -- 172.21.15.105:0/2589692845 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde08003d20 con 0x7fddf8005190 2023-11-28T18:30:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.321+0000 7fde0effd640 1 -- 172.21.15.105:0/2589692845 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde08005260 con 0x7fddf8005190 2023-11-28T18:30:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.321+0000 7fde1666b640 1 -- 172.21.15.105:0/2589692845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddf8005190 msgr2=0x7fddf800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.321+0000 7fde1666b640 1 --2- 172.21.15.105:0/2589692845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddf8005190 0x7fddf800b990 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fde08001e70 tx=0x7fde0801c760 comp rx=0 tx=0).stop 2023-11-28T18:30:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.321+0000 7fde1666b640 1 -- 172.21.15.105:0/2589692845 shutdown_connections 2023-11-28T18:30:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.321+0000 7fde1666b640 1 --2- 172.21.15.105:0/2589692845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddf8005190 0x7fddf800b990 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.321+0000 7fde1666b640 1 -- 172.21.15.105:0/2589692845 >> 172.21.15.105:0/2589692845 conn(0x7fde100ceb90 msgr2=0x7fde100cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.321+0000 7fde1666b640 1 -- 172.21.15.105:0/2589692845 shutdown_connections 2023-11-28T18:30:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.321+0000 7fde1666b640 1 -- 172.21.15.105:0/2589692845 wait complete. 2023-11-28T18:30:04.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.321+0000 7fde1666b640 1 Processor -- start 2023-11-28T18:30:04.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.321+0000 7fde1666b640 1 -- start start 2023-11-28T18:30:04.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde1666b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddf8005190 0x7fde100b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:04.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde1666b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde10148f90 0x7fde100b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:04.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde1666b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde101493f0 0x7fde100c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:04.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde1666b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde100b3080 con 0x7fddf8005190 2023-11-28T18:30:04.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde1666b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde100b2400 con 0x7fde10148f90 2023-11-28T18:30:04.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde1666b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde100b26e0 con 0x7fde101493f0 2023-11-28T18:30:04.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde0ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddf8005190 0x7fde100b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:04.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde0ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddf8005190 0x7fde100b34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50050/0 (socket says 172.21.15.105:50050) 2023-11-28T18:30:04.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde0ffff640 1 -- 172.21.15.105:0/2573367056 learned_addr learned my addr 172.21.15.105:0/2573367056 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:04.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde14c08640 1 --2- 172.21.15.105:0/2573367056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde101493f0 0x7fde100c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:04.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde0ffff640 1 -- 172.21.15.105:0/2573367056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde101493f0 msgr2=0x7fde100c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:04.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde0f7fe640 1 --2- 172.21.15.105:0/2573367056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde10148f90 0x7fde100b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:04.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde0ffff640 1 --2- 172.21.15.105:0/2573367056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde101493f0 0x7fde100c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:04.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde0ffff640 1 -- 172.21.15.105:0/2573367056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde10148f90 msgr2=0x7fde100b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:04.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde0ffff640 1 --2- 172.21.15.105:0/2573367056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde10148f90 0x7fde100b19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde0ffff640 1 -- 172.21.15.105:0/2573367056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde100b29c0 con 0x7fddf8005190 2023-11-28T18:30:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde14c08640 1 --2- 172.21.15.105:0/2573367056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde101493f0 0x7fde100c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde0ffff640 1 --2- 172.21.15.105:0/2573367056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddf8005190 0x7fde100b34a0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fde00011840 tx=0x7fde00011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde0cff9640 1 -- 172.21.15.105:0/2573367056 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0000fb60 con 0x7fddf8005190 2023-11-28T18:30:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde0cff9640 1 -- 172.21.15.105:0/2573367056 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0000fcc0 con 0x7fddf8005190 2023-11-28T18:30:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde1666b640 1 -- 172.21.15.105:0/2573367056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde100b2ca0 con 0x7fddf8005190 2023-11-28T18:30:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde0cff9640 1 -- 172.21.15.105:0/2573367056 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde00015610 con 0x7fddf8005190 2023-11-28T18:30:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde1666b640 1 -- 172.21.15.105:0/2573367056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde100b1ef0 con 0x7fddf8005190 2023-11-28T18:30:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde1666b640 1 -- 172.21.15.105:0/2573367056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde10067570 con 0x7fddf8005190 2023-11-28T18:30:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.325+0000 7fde0cff9640 1 -- 172.21.15.105:0/2573367056 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fde00015770 con 0x7fddf8005190 2023-11-28T18:30:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.329+0000 7fde0cff9640 1 --2- 172.21.15.105:0/2573367056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fddfc0668d0 0x7fddfc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.329+0000 7fde0cff9640 1 -- 172.21.15.105:0/2573367056 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5547+0+0 (secure 0 0 0) 0x7fde00087720 con 0x7fddf8005190 2023-11-28T18:30:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.329+0000 7fde0f7fe640 1 --2- 172.21.15.105:0/2573367056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fddfc0668d0 0x7fddfc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.329+0000 7fde0f7fe640 1 --2- 172.21.15.105:0/2573367056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fddfc0668d0 0x7fddfc068d90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fde1015bf20 tx=0x7fde04008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:04.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.333+0000 7fde0cff9640 1 -- 172.21.15.105:0/2573367056 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fde00054960 con 0x7fddf8005190 2023-11-28T18:30:04.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:04.481+0000 7fde1666b640 1 -- 172.21.15.105:0/2573367056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fde10156d90 con 0x7fddf8005190 2023-11-28T18:30:05.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.137+0000 7fde0cff9640 1 -- 172.21.15.105:0/2573367056 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v101) v1 ==== 170+0+0 (secure 0 0 0) 0x7fde00057f80 con 0x7fddf8005190 2023-11-28T18:30:05.140 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' created 2023-11-28T18:30:05.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.141+0000 7fde1666b640 1 -- 172.21.15.105:0/2573367056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fddfc0668d0 msgr2=0x7fddfc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:05.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.141+0000 7fde1666b640 1 --2- 172.21.15.105:0/2573367056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fddfc0668d0 0x7fddfc068d90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fde1015bf20 tx=0x7fde04008040 comp rx=0 tx=0).stop 2023-11-28T18:30:05.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.141+0000 7fde1666b640 1 -- 172.21.15.105:0/2573367056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddf8005190 msgr2=0x7fde100b34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:05.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.141+0000 7fde1666b640 1 --2- 172.21.15.105:0/2573367056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddf8005190 0x7fde100b34a0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fde00011840 tx=0x7fde00011d00 comp rx=0 tx=0).stop 2023-11-28T18:30:05.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.141+0000 7fde1666b640 1 -- 172.21.15.105:0/2573367056 shutdown_connections 2023-11-28T18:30:05.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.141+0000 7fde1666b640 1 --2- 172.21.15.105:0/2573367056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde101493f0 0x7fde100c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:05.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.141+0000 7fde1666b640 1 --2- 172.21.15.105:0/2573367056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde10148f90 0x7fde100b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:05.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.141+0000 7fde1666b640 1 --2- 172.21.15.105:0/2573367056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fddfc0668d0 0x7fddfc068d90 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:05.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.141+0000 7fde1666b640 1 --2- 172.21.15.105:0/2573367056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddf8005190 0x7fde100b34a0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:05.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.141+0000 7fde1666b640 1 -- 172.21.15.105:0/2573367056 >> 172.21.15.105:0/2573367056 conn(0x7fde100ceb90 msgr2=0x7fde10072eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:05.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.141+0000 7fde1666b640 1 -- 172.21.15.105:0/2573367056 shutdown_connections 2023-11-28T18:30:05.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.141+0000 7fde1666b640 1 -- 172.21.15.105:0/2573367056 wait complete. 2023-11-28T18:30:05.183 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T18:30:05.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.325+0000 7fd33eedd640 1 Processor -- start 2023-11-28T18:30:05.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.325+0000 7fd33eedd640 1 -- start start 2023-11-28T18:30:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.325+0000 7fd33eedd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3380b4290 0x7fd3380b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.325+0000 7fd33eedd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd338071960 0x7fd33815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.325+0000 7fd33eedd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3380b4bb0 con 0x7fd33815b030 2023-11-28T18:30:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.325+0000 7fd33eedd640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3380701d0 con 0x7fd338071960 2023-11-28T18:30:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.325+0000 7fd33eedd640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33806df80 con 0x7fd3380b4290 2023-11-28T18:30:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.325+0000 7fd337fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd338071960 0x7fd33815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.325+0000 7fd337fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd338071960 0x7fd33815f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50060/0 (socket says 172.21.15.105:50060) 2023-11-28T18:30:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.325+0000 7fd337fff640 1 -- 172.21.15.105:0/4009853660 learned_addr learned my addr 172.21.15.105:0/4009853660 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.325+0000 7fd337fff640 1 -- 172.21.15.105:0/4009853660 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3380b4290 msgr2=0x7fd3380b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.325+0000 7fd337fff640 1 --2- 172.21.15.105:0/4009853660 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3380b4290 0x7fd3380b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.325+0000 7fd337fff640 1 -- 172.21.15.105:0/4009853660 >> v1:172.21.15.148:6789/0 conn(0x7fd33815b030 legacy=0x7fd3380709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.325+0000 7fd337fff640 1 -- 172.21.15.105:0/4009853660 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd338071df0 con 0x7fd338071960 2023-11-28T18:30:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.329+0000 7fd337fff640 1 --2- 172.21.15.105:0/4009853660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd338071960 0x7fd33815f3b0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fd324009870 tx=0x7fd32401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3acb3bfafe718729 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.329+0000 7fd3377fe640 1 -- 172.21.15.105:0/4009853660 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3240aa5f0 con 0x7fd338071960 2023-11-28T18:30:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.329+0000 7fd3377fe640 1 -- 172.21.15.105:0/4009853660 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3240aa750 con 0x7fd338071960 2023-11-28T18:30:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.329+0000 7fd3377fe640 1 -- 172.21.15.105:0/4009853660 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3240aaae0 con 0x7fd338071960 2023-11-28T18:30:05.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.329+0000 7fd33eedd640 1 -- 172.21.15.105:0/4009853660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd338071960 msgr2=0x7fd33815f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:05.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.329+0000 7fd33eedd640 1 --2- 172.21.15.105:0/4009853660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd338071960 0x7fd33815f3b0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fd324009870 tx=0x7fd32401c4d0 comp rx=0 tx=0).stop 2023-11-28T18:30:05.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.329+0000 7fd33eedd640 1 -- 172.21.15.105:0/4009853660 shutdown_connections 2023-11-28T18:30:05.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.329+0000 7fd33eedd640 1 --2- 172.21.15.105:0/4009853660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd338071960 0x7fd33815f3b0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:05.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.329+0000 7fd33eedd640 1 --2- 172.21.15.105:0/4009853660 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3380b4290 0x7fd3380b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:05.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.329+0000 7fd33eedd640 1 -- 172.21.15.105:0/4009853660 >> 172.21.15.105:0/4009853660 conn(0x7fd3380ceb90 msgr2=0x7fd3380cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:05.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.329+0000 7fd33eedd640 1 -- 172.21.15.105:0/4009853660 shutdown_connections 2023-11-28T18:30:05.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.329+0000 7fd33eedd640 1 -- 172.21.15.105:0/4009853660 wait complete. 2023-11-28T18:30:05.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd33eedd640 1 Processor -- start 2023-11-28T18:30:05.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd33eedd640 1 -- start start 2023-11-28T18:30:05.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd33eedd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3380b4290 0x7fd3380b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:05.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd33eedd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd33815b030 0x7fd338159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:05.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd33eedd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd338159580 0x7fd338153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:05.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd33eedd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd338159a30 con 0x7fd3380b4290 2023-11-28T18:30:05.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd33eedd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33809c920 con 0x7fd338159580 2023-11-28T18:30:05.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd33eedd640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33814c7c0 con 0x7fd33815b030 2023-11-28T18:30:05.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd33cc79640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3380b4290 0x7fd3380b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:05.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd33cc79640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3380b4290 0x7fd3380b2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50068/0 (socket says 172.21.15.105:50068) 2023-11-28T18:30:05.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd33cc79640 1 -- 172.21.15.105:0/2974594262 learned_addr learned my addr 172.21.15.105:0/2974594262 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd337fff640 1 --2- 172.21.15.105:0/2974594262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd33815b030 0x7fd338159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd33d47a640 1 --2- 172.21.15.105:0/2974594262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd338159580 0x7fd338153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd33cc79640 1 -- 172.21.15.105:0/2974594262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd33815b030 msgr2=0x7fd338159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd33cc79640 1 --2- 172.21.15.105:0/2974594262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd33815b030 0x7fd338159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd33cc79640 1 -- 172.21.15.105:0/2974594262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd338159580 msgr2=0x7fd338153650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd33cc79640 1 --2- 172.21.15.105:0/2974594262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd338159580 0x7fd338153650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd33cc79640 1 -- 172.21.15.105:0/2974594262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3240094d0 con 0x7fd3380b4290 2023-11-28T18:30:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd33cc79640 1 --2- 172.21.15.105:0/2974594262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3380b4290 0x7fd3380b2c60 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fd32c00b550 tx=0x7fd32c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd3357fa640 1 -- 172.21.15.105:0/2974594262 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c010870 con 0x7fd3380b4290 2023-11-28T18:30:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd3357fa640 1 -- 172.21.15.105:0/2974594262 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32c0109f0 con 0x7fd3380b4290 2023-11-28T18:30:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd3357fa640 1 -- 172.21.15.105:0/2974594262 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c010da0 con 0x7fd3380b4290 2023-11-28T18:30:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd33eedd640 1 -- 172.21.15.105:0/2974594262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd338159c20 con 0x7fd3380b4290 2023-11-28T18:30:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.333+0000 7fd33eedd640 1 -- 172.21.15.105:0/2974594262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd338152b70 con 0x7fd3380b4290 2023-11-28T18:30:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.337+0000 7fd33eedd640 1 -- 172.21.15.105:0/2974594262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd338067570 con 0x7fd3380b4290 2023-11-28T18:30:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.337+0000 7fd3357fa640 1 -- 172.21.15.105:0/2974594262 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd32c023480 con 0x7fd3380b4290 2023-11-28T18:30:05.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.337+0000 7fd3357fa640 1 --2- 172.21.15.105:0/2974594262 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd31c0668d0 0x7fd31c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:05.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.337+0000 7fd3357fa640 1 -- 172.21.15.105:0/2974594262 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5929+0+0 (secure 0 0 0) 0x7fd32c0849a0 con 0x7fd3380b4290 2023-11-28T18:30:05.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.337+0000 7fd337fff640 1 --2- 172.21.15.105:0/2974594262 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd31c0668d0 0x7fd31c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:05.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.337+0000 7fd337fff640 1 --2- 172.21.15.105:0/2974594262 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd31c0668d0 0x7fd31c068d90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd3240ae000 tx=0x7fd3240a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:05.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.341+0000 7fd3357fa640 1 -- 172.21.15.105:0/2974594262 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd32c051ae0 con 0x7fd3380b4290 2023-11-28T18:30:05.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:05.521+0000 7fd33eedd640 1 -- 172.21.15.105:0/2974594262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd33815a860 con 0x7fd3380b4290 2023-11-28T18:30:06.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.153+0000 7fd3357fa640 1 -- 172.21.15.105:0/2974594262 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v99) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd32c055100 con 0x7fd3380b4290 2023-11-28T18:30:06.154 INFO:teuthology.orchestra.run.smithi105.stderr:new fs with metadata pool 15 and data pool 16 2023-11-28T18:30:06.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.157+0000 7fd33eedd640 1 -- 172.21.15.105:0/2974594262 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd31c0668d0 msgr2=0x7fd31c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:06.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.157+0000 7fd33eedd640 1 --2- 172.21.15.105:0/2974594262 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd31c0668d0 0x7fd31c068d90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd3240ae000 tx=0x7fd3240a7000 comp rx=0 tx=0).stop 2023-11-28T18:30:06.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.157+0000 7fd33eedd640 1 -- 172.21.15.105:0/2974594262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3380b4290 msgr2=0x7fd3380b2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:06.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.157+0000 7fd33eedd640 1 --2- 172.21.15.105:0/2974594262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3380b4290 0x7fd3380b2c60 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fd32c00b550 tx=0x7fd32c00ba10 comp rx=0 tx=0).stop 2023-11-28T18:30:06.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.157+0000 7fd33eedd640 1 -- 172.21.15.105:0/2974594262 shutdown_connections 2023-11-28T18:30:06.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.157+0000 7fd33eedd640 1 --2- 172.21.15.105:0/2974594262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd338159580 0x7fd338153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.157+0000 7fd33eedd640 1 --2- 172.21.15.105:0/2974594262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd33815b030 0x7fd338159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.157+0000 7fd33eedd640 1 --2- 172.21.15.105:0/2974594262 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd31c0668d0 0x7fd31c068d90 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.157+0000 7fd33eedd640 1 --2- 172.21.15.105:0/2974594262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3380b4290 0x7fd3380b2c60 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.157+0000 7fd33eedd640 1 -- 172.21.15.105:0/2974594262 >> 172.21.15.105:0/2974594262 conn(0x7fd3380ceb90 msgr2=0x7fd3380ade40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:06.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.161+0000 7fd33eedd640 1 -- 172.21.15.105:0/2974594262 shutdown_connections 2023-11-28T18:30:06.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.161+0000 7fd33eedd640 1 -- 172.21.15.105:0/2974594262 wait complete. 2023-11-28T18:30:06.206 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:30:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.345+0000 7fac7929f640 1 Processor -- start 2023-11-28T18:30:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.345+0000 7fac7929f640 1 -- start start 2023-11-28T18:30:06.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.345+0000 7fac7929f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fac74157990 0x7fac7415ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:06.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.345+0000 7fac7929f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fac7415d7b0 0x7fac7406cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:06.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.345+0000 7fac7929f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac7415e0b0 con 0x7fac741575b0 2023-11-28T18:30:06.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.345+0000 7fac7929f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac740ca660 con 0x7fac7415d7b0 2023-11-28T18:30:06.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.345+0000 7fac7929f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac7415e390 con 0x7fac74157990 2023-11-28T18:30:06.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.349+0000 7fac73fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fac741575b0 0x7fac740c9f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:48578/0 (socket says 172.21.15.105:48578) 2023-11-28T18:30:06.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.349+0000 7fac737fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fac74157990 0x7fac7415ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:06.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.349+0000 7fac73fff640 1 -- 172.21.15.105:0/2670283107 learned_addr learned my addr 172.21.15.105:0/2670283107 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:06.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.349+0000 7fac78a9e640 1 --2- 172.21.15.105:0/2670283107 >> v2:172.21.15.105:3300/0 conn(0x7fac7415d7b0 0x7fac7406cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:06.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.349+0000 7fac72ffd640 1 -- 172.21.15.105:0/2670283107 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911703318 0 0) 0x7fac7415e0b0 con 0x7fac741575b0 2023-11-28T18:30:06.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.349+0000 7fac72ffd640 1 -- 172.21.15.105:0/2670283107 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac5c003530 con 0x7fac741575b0 2023-11-28T18:30:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.349+0000 7fac737fe640 1 -- 172.21.15.105:0/2670283107 >> v2:172.21.15.105:3300/0 conn(0x7fac7415d7b0 msgr2=0x7fac7406cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.349+0000 7fac737fe640 1 --2- 172.21.15.105:0/2670283107 >> v2:172.21.15.105:3300/0 conn(0x7fac7415d7b0 0x7fac7406cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.349+0000 7fac737fe640 1 -- 172.21.15.105:0/2670283107 >> v1:172.21.15.148:6789/0 conn(0x7fac741575b0 legacy=0x7fac740c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.349+0000 7fac737fe640 1 -- 172.21.15.105:0/2670283107 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7415e670 con 0x7fac74157990 2023-11-28T18:30:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.349+0000 7fac737fe640 1 --2- 172.21.15.105:0/2670283107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fac74157990 0x7fac7415ada0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fac680097c0 tx=0x7fac6801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9ffd4e20de5eae5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:06.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.353+0000 7fac72ffd640 1 -- 172.21.15.105:0/2670283107 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac680ab650 con 0x7fac74157990 2023-11-28T18:30:06.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.353+0000 7fac72ffd640 1 -- 172.21.15.105:0/2670283107 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac680ab7b0 con 0x7fac74157990 2023-11-28T18:30:06.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.353+0000 7fac72ffd640 1 -- 172.21.15.105:0/2670283107 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac680abb20 con 0x7fac74157990 2023-11-28T18:30:06.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.353+0000 7fac7929f640 1 -- 172.21.15.105:0/2670283107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fac74157990 msgr2=0x7fac7415ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:06.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.353+0000 7fac7929f640 1 --2- 172.21.15.105:0/2670283107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fac74157990 0x7fac7415ada0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fac680097c0 tx=0x7fac6801c450 comp rx=0 tx=0).stop 2023-11-28T18:30:06.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.353+0000 7fac7929f640 1 -- 172.21.15.105:0/2670283107 shutdown_connections 2023-11-28T18:30:06.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.353+0000 7fac7929f640 1 --2- 172.21.15.105:0/2670283107 >> v2:172.21.15.105:3300/0 conn(0x7fac7415d7b0 0x7fac7406cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.353+0000 7fac7929f640 1 --2- 172.21.15.105:0/2670283107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fac74157990 0x7fac7415ada0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.353+0000 7fac7929f640 1 -- 172.21.15.105:0/2670283107 >> 172.21.15.105:0/2670283107 conn(0x7fac740b0480 msgr2=0x7fac740b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:06.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.353+0000 7fac7929f640 1 -- 172.21.15.105:0/2670283107 shutdown_connections 2023-11-28T18:30:06.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.353+0000 7fac7929f640 1 -- 172.21.15.105:0/2670283107 wait complete. 2023-11-28T18:30:06.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.353+0000 7fac7929f640 1 Processor -- start 2023-11-28T18:30:06.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.353+0000 7fac7929f640 1 -- start start 2023-11-28T18:30:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.353+0000 7fac7929f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fac741575b0 0x7fac7414ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.353+0000 7fac7929f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac74157990 0x7fac74155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.353+0000 7fac7929f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac7415d7b0 0x7fac74157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.353+0000 7fac7929f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac74155d20 con 0x7fac74157990 2023-11-28T18:30:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.353+0000 7fac7929f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac74156000 con 0x7fac7415d7b0 2023-11-28T18:30:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.353+0000 7fac7929f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac741562e0 con 0x7fac741575b0 2023-11-28T18:30:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.353+0000 7fac737fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac74157990 0x7fac74155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.353+0000 7fac737fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac74157990 0x7fac74155690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50092/0 (socket says 172.21.15.105:50092) 2023-11-28T18:30:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.357+0000 7fac737fe640 1 -- 172.21.15.105:0/2373035664 learned_addr learned my addr 172.21.15.105:0/2373035664 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.357+0000 7fac73fff640 1 --2- 172.21.15.105:0/2373035664 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fac741575b0 0x7fac7414ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.357+0000 7fac78a9e640 1 --2- 172.21.15.105:0/2373035664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac7415d7b0 0x7fac74157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.357+0000 7fac737fe640 1 -- 172.21.15.105:0/2373035664 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fac741575b0 msgr2=0x7fac7414ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.357+0000 7fac737fe640 1 --2- 172.21.15.105:0/2373035664 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fac741575b0 0x7fac7414ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.357+0000 7fac737fe640 1 -- 172.21.15.105:0/2373035664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac7415d7b0 msgr2=0x7fac74157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.357+0000 7fac737fe640 1 --2- 172.21.15.105:0/2373035664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac7415d7b0 0x7fac74157fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.357+0000 7fac737fe640 1 -- 172.21.15.105:0/2373035664 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac68009420 con 0x7fac74157990 2023-11-28T18:30:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.357+0000 7fac78a9e640 1 --2- 172.21.15.105:0/2373035664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac7415d7b0 0x7fac74157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.357+0000 7fac73fff640 1 --2- 172.21.15.105:0/2373035664 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fac741575b0 0x7fac7414ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.357+0000 7fac737fe640 1 --2- 172.21.15.105:0/2373035664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac74157990 0x7fac74155690 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fac680ad040 tx=0x7fac6801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.357+0000 7fac70ff9640 1 -- 172.21.15.105:0/2373035664 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac680bf610 con 0x7fac74157990 2023-11-28T18:30:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.357+0000 7fac70ff9640 1 -- 172.21.15.105:0/2373035664 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac680bf770 con 0x7fac74157990 2023-11-28T18:30:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.357+0000 7fac70ff9640 1 -- 172.21.15.105:0/2373035664 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac680bfae0 con 0x7fac74157990 2023-11-28T18:30:06.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.357+0000 7fac7929f640 1 -- 172.21.15.105:0/2373035664 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac741565c0 con 0x7fac74157990 2023-11-28T18:30:06.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.357+0000 7fac7929f640 1 -- 172.21.15.105:0/2373035664 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac741591f0 con 0x7fac74157990 2023-11-28T18:30:06.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.357+0000 7fac7929f640 1 -- 172.21.15.105:0/2373035664 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac74074580 con 0x7fac74157990 2023-11-28T18:30:06.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.357+0000 7fac70ff9640 1 -- 172.21.15.105:0/2373035664 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fac680bfc40 con 0x7fac74157990 2023-11-28T18:30:06.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.357+0000 7fac70ff9640 1 --2- 172.21.15.105:0/2373035664 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fac600668d0 0x7fac60068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:06.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.361+0000 7fac70ff9640 1 -- 172.21.15.105:0/2373035664 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fac6811efe0 con 0x7fac74157990 2023-11-28T18:30:06.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.361+0000 7fac73fff640 1 --2- 172.21.15.105:0/2373035664 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fac600668d0 0x7fac60068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:06.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.361+0000 7fac73fff640 1 --2- 172.21.15.105:0/2373035664 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fac600668d0 0x7fac60068d90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fac640011f0 tx=0x7fac64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:06.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.361+0000 7fac70ff9640 1 -- 172.21.15.105:0/2373035664 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fac680ec0b0 con 0x7fac74157990 2023-11-28T18:30:06.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.513+0000 7fac7929f640 1 -- 172.21.15.105:0/2373035664 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac74156420 con 0x7fac74157990 2023-11-28T18:30:06.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.517+0000 7fac70ff9640 1 -- 172.21.15.105:0/2373035664 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16789 (secure 0 0 0) 0x7fac74156420 con 0x7fac74157990 2023-11-28T18:30:06.517 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:06.517 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":102,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:30:05.524791+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:30:03.341424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-28T18:30:04.527931+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:30:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.517+0000 7fac7929f640 1 -- 172.21.15.105:0/2373035664 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fac600668d0 msgr2=0x7fac60068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.517+0000 7fac7929f640 1 --2- 172.21.15.105:0/2373035664 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fac600668d0 0x7fac60068d90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fac640011f0 tx=0x7fac64009040 comp rx=0 tx=0).stop 2023-11-28T18:30:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.517+0000 7fac7929f640 1 -- 172.21.15.105:0/2373035664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac74157990 msgr2=0x7fac74155690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.517+0000 7fac7929f640 1 --2- 172.21.15.105:0/2373035664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac74157990 0x7fac74155690 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fac680ad040 tx=0x7fac6801ac90 comp rx=0 tx=0).stop 2023-11-28T18:30:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.517+0000 7fac7929f640 1 -- 172.21.15.105:0/2373035664 shutdown_connections 2023-11-28T18:30:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.517+0000 7fac7929f640 1 --2- 172.21.15.105:0/2373035664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac7415d7b0 0x7fac74157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.517+0000 7fac7929f640 1 --2- 172.21.15.105:0/2373035664 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fac600668d0 0x7fac60068d90 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.517+0000 7fac7929f640 1 --2- 172.21.15.105:0/2373035664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac74157990 0x7fac74155690 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.517+0000 7fac7929f640 1 --2- 172.21.15.105:0/2373035664 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fac741575b0 0x7fac7414ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.517+0000 7fac7929f640 1 -- 172.21.15.105:0/2373035664 >> 172.21.15.105:0/2373035664 conn(0x7fac740b0480 msgr2=0x7fac740cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.517+0000 7fac7929f640 1 -- 172.21.15.105:0/2373035664 shutdown_connections 2023-11-28T18:30:06.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.517+0000 7fac7929f640 1 -- 172.21.15.105:0/2373035664 wait complete. 2023-11-28T18:30:06.538 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:30:06.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.677+0000 7f642605a640 1 Processor -- start 2023-11-28T18:30:06.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.677+0000 7f642605a640 1 -- start start 2023-11-28T18:30:06.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.677+0000 7f642605a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f64200caf60 0x7f64200cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:06.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.677+0000 7f642605a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64200ad850 0x7f64200701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:06.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.677+0000 7f642605a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f642006bf50 con 0x7f6420159f40 2023-11-28T18:30:06.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.677+0000 7f642605a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64200cb3c0 con 0x7f64200caf60 2023-11-28T18:30:06.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.677+0000 7f642605a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642015a320 con 0x7f64200ad850 2023-11-28T18:30:06.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.677+0000 7f6417fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f64200caf60 0x7f64200cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:06.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.677+0000 7f6417fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f64200caf60 0x7f64200cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50102/0 (socket says 172.21.15.105:50102) 2023-11-28T18:30:06.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.677+0000 7f6417fff640 1 -- 172.21.15.105:0/3327538284 learned_addr learned my addr 172.21.15.105:0/3327538284 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:06.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.677+0000 7f641f7fe640 1 --1- 172.21.15.105:0/3327538284 >> v1:172.21.15.105:6789/0 conn(0x7f6420159f40 0x7f64200ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40174/0 (socket says 172.21.15.105:40174) 2023-11-28T18:30:06.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.677+0000 7f641ffff640 1 --2- 172.21.15.105:0/3327538284 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64200ad850 0x7f64200701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:06.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.677+0000 7f6417fff640 1 -- 172.21.15.105:0/3327538284 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64200ad850 msgr2=0x7f64200701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:06.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.677+0000 7f6417fff640 1 --2- 172.21.15.105:0/3327538284 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64200ad850 0x7f64200701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.677+0000 7f6417fff640 1 -- 172.21.15.105:0/3327538284 >> v1:172.21.15.105:6789/0 conn(0x7f6420159f40 legacy=0x7f64200ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:06.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.677+0000 7f6417fff640 1 -- 172.21.15.105:0/3327538284 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6420071380 con 0x7f64200caf60 2023-11-28T18:30:06.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.677+0000 7f641effd640 1 -- 172.21.15.105:0/3327538284 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1202184891 0 0) 0x7f642006bf50 con 0x7f6420159f40 2023-11-28T18:30:06.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.681+0000 7f6417fff640 1 --2- 172.21.15.105:0/3327538284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64200caf60 0x7f64200cc760 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f6408009870 tx=0x7f640801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853fef38bbe7640e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:06.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.681+0000 7f641effd640 1 -- 172.21.15.105:0/3327538284 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64080ab680 con 0x7f64200caf60 2023-11-28T18:30:06.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.681+0000 7f641effd640 1 -- 172.21.15.105:0/3327538284 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64080ab7e0 con 0x7f64200caf60 2023-11-28T18:30:06.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.681+0000 7f641effd640 1 -- 172.21.15.105:0/3327538284 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64080abb70 con 0x7f64200caf60 2023-11-28T18:30:06.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.681+0000 7f642605a640 1 -- 172.21.15.105:0/3327538284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64200caf60 msgr2=0x7f64200cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:06.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.681+0000 7f642605a640 1 --2- 172.21.15.105:0/3327538284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64200caf60 0x7f64200cc760 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f6408009870 tx=0x7f640801c480 comp rx=0 tx=0).stop 2023-11-28T18:30:06.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.681+0000 7f642605a640 1 -- 172.21.15.105:0/3327538284 shutdown_connections 2023-11-28T18:30:06.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.681+0000 7f642605a640 1 --2- 172.21.15.105:0/3327538284 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64200ad850 0x7f64200701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.681+0000 7f642605a640 1 --2- 172.21.15.105:0/3327538284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64200caf60 0x7f64200cc760 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.681+0000 7f642605a640 1 -- 172.21.15.105:0/3327538284 >> 172.21.15.105:0/3327538284 conn(0x7f64200ceb90 msgr2=0x7f64200cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:06.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.681+0000 7f642605a640 1 -- 172.21.15.105:0/3327538284 shutdown_connections 2023-11-28T18:30:06.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.681+0000 7f642605a640 1 -- 172.21.15.105:0/3327538284 wait complete. 2023-11-28T18:30:06.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.681+0000 7f642605a640 1 Processor -- start 2023-11-28T18:30:06.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.681+0000 7f642605a640 1 -- start start 2023-11-28T18:30:06.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f642605a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64200ad850 0x7f6420155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:06.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f642605a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64200caf60 0x7f64201487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:06.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f642605a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6420159f40 0x7f64200b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:06.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f642605a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6420155960 con 0x7f6420159f40 2023-11-28T18:30:06.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f642605a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64201549d0 con 0x7f64200ad850 2023-11-28T18:30:06.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f642605a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6420156910 con 0x7f64200caf60 2023-11-28T18:30:06.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f641ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6420159f40 0x7f64200b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:06.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f641ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6420159f40 0x7f64200b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50114/0 (socket says 172.21.15.105:50114) 2023-11-28T18:30:06.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f641ffff640 1 -- 172.21.15.105:0/1330695690 learned_addr learned my addr 172.21.15.105:0/1330695690 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:06.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f6417fff640 1 --2- 172.21.15.105:0/1330695690 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64200caf60 0x7f64201487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:06.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f641f7fe640 1 --2- 172.21.15.105:0/1330695690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64200ad850 0x7f6420155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:06.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f641ffff640 1 -- 172.21.15.105:0/1330695690 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64200caf60 msgr2=0x7f64201487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:06.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f641ffff640 1 --2- 172.21.15.105:0/1330695690 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64200caf60 0x7f64201487c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f641ffff640 1 -- 172.21.15.105:0/1330695690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64200ad850 msgr2=0x7f6420155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:06.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f641ffff640 1 --2- 172.21.15.105:0/1330695690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64200ad850 0x7f6420155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f641ffff640 1 -- 172.21.15.105:0/1330695690 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64080094d0 con 0x7f6420159f40 2023-11-28T18:30:06.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f641f7fe640 1 --2- 172.21.15.105:0/1330695690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64200ad850 0x7f6420155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:06.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f641ffff640 1 --2- 172.21.15.105:0/1330695690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6420159f40 0x7f64200b2850 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f6418007e40 tx=0x7f641800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:06.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f641cff9640 1 -- 172.21.15.105:0/1330695690 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418014620 con 0x7f6420159f40 2023-11-28T18:30:06.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f641cff9640 1 -- 172.21.15.105:0/1330695690 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641800ee90 con 0x7f6420159f40 2023-11-28T18:30:06.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f641cff9640 1 -- 172.21.15.105:0/1330695690 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64180149e0 con 0x7f6420159f40 2023-11-28T18:30:06.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f642605a640 1 -- 172.21.15.105:0/1330695690 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6420156bb0 con 0x7f6420159f40 2023-11-28T18:30:06.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f642605a640 1 -- 172.21.15.105:0/1330695690 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6420154c20 con 0x7f6420159f40 2023-11-28T18:30:06.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.685+0000 7f642605a640 1 -- 172.21.15.105:0/1330695690 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6420067570 con 0x7f6420159f40 2023-11-28T18:30:06.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.689+0000 7f641cff9640 1 -- 172.21.15.105:0/1330695690 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f6418014b40 con 0x7f6420159f40 2023-11-28T18:30:06.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.689+0000 7f641cff9640 1 --2- 172.21.15.105:0/1330695690 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f64000668d0 0x7f6400068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:06.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.689+0000 7f641cff9640 1 -- 172.21.15.105:0/1330695690 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6418084f10 con 0x7f6420159f40 2023-11-28T18:30:06.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.689+0000 7f641f7fe640 1 --2- 172.21.15.105:0/1330695690 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f64000668d0 0x7f6400068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:06.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.689+0000 7f641f7fe640 1 --2- 172.21.15.105:0/1330695690 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f64000668d0 0x7f6400068d90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6410000f70 tx=0x7f6410007cb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:06.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.693+0000 7f641cff9640 1 -- 172.21.15.105:0/1330695690 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6418051f70 con 0x7f6420159f40 2023-11-28T18:30:06.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.841+0000 7f642605a640 1 -- 172.21.15.105:0/1330695690 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64200ce850 con 0x7f6420159f40 2023-11-28T18:30:06.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.841+0000 7f641cff9640 1 -- 172.21.15.105:0/1330695690 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16789 (secure 0 0 0) 0x7f64200ce850 con 0x7f6420159f40 2023-11-28T18:30:06.843 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:06.843 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":102,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:30:05.524791+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:30:03.341424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-28T18:30:04.527931+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:30:06.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.845+0000 7f642605a640 1 -- 172.21.15.105:0/1330695690 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f64000668d0 msgr2=0x7f6400068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:06.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.845+0000 7f642605a640 1 --2- 172.21.15.105:0/1330695690 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f64000668d0 0x7f6400068d90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6410000f70 tx=0x7f6410007cb0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.845+0000 7f642605a640 1 -- 172.21.15.105:0/1330695690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6420159f40 msgr2=0x7f64200b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:06.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.845+0000 7f642605a640 1 --2- 172.21.15.105:0/1330695690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6420159f40 0x7f64200b2850 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f6418007e40 tx=0x7f641800e4b0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.845+0000 7f642605a640 1 -- 172.21.15.105:0/1330695690 shutdown_connections 2023-11-28T18:30:06.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.845+0000 7f642605a640 1 --2- 172.21.15.105:0/1330695690 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f64000668d0 0x7f6400068d90 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.845+0000 7f642605a640 1 --2- 172.21.15.105:0/1330695690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6420159f40 0x7f64200b2850 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.845+0000 7f642605a640 1 --2- 172.21.15.105:0/1330695690 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64200caf60 0x7f64201487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.845+0000 7f642605a640 1 --2- 172.21.15.105:0/1330695690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64200ad850 0x7f6420155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:06.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.845+0000 7f642605a640 1 -- 172.21.15.105:0/1330695690 >> 172.21.15.105:0/1330695690 conn(0x7f64200ceb90 msgr2=0x7f642006cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:06.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.845+0000 7f642605a640 1 -- 172.21.15.105:0/1330695690 shutdown_connections 2023-11-28T18:30:06.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:06.845+0000 7f642605a640 1 -- 172.21.15.105:0/1330695690 wait complete. 2023-11-28T18:30:06.865 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T18:30:07.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.005+0000 7fa7e705f640 1 Processor -- start 2023-11-28T18:30:07.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.005+0000 7fa7e705f640 1 -- start start 2023-11-28T18:30:07.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.005+0000 7fa7e705f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa7e0159f40 0x7fa7e00caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:07.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.005+0000 7fa7e705f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7e00cc760 0x7fa7e00ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:07.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.005+0000 7fa7e705f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e006bf50 con 0x7fa7e00ad680 2023-11-28T18:30:07.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.005+0000 7fa7e705f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e00ccbc0 con 0x7fa7e0159f40 2023-11-28T18:30:07.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.005+0000 7fa7e705f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e015a320 con 0x7fa7e00cc760 2023-11-28T18:30:07.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.005+0000 7fa7e55fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa7e00ad680 0x7fa7e015da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40180/0 (socket says 172.21.15.105:40180) 2023-11-28T18:30:07.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.005+0000 7fa7e55fc640 1 -- 172.21.15.105:0/2670213316 learned_addr learned my addr 172.21.15.105:0/2670213316 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.005+0000 7fa7e4dfb640 1 --2- 172.21.15.105:0/2670213316 >> v2:172.21.15.148:3300/0 conn(0x7fa7e0159f40 0x7fa7e00caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.005+0000 7fa7dffff640 1 --2- 172.21.15.105:0/2670213316 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7e00cc760 0x7fa7e00ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.005+0000 7fa7df7fe640 1 -- 172.21.15.105:0/2670213316 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704913598 0 0) 0x7fa7e006bf50 con 0x7fa7e00ad680 2023-11-28T18:30:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.005+0000 7fa7df7fe640 1 -- 172.21.15.105:0/2670213316 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7d0003530 con 0x7fa7e00ad680 2023-11-28T18:30:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.009+0000 7fa7e4dfb640 1 -- 172.21.15.105:0/2670213316 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7e00cc760 msgr2=0x7fa7e00ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.009+0000 7fa7df7fe640 1 -- 172.21.15.105:0/2670213316 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 610209650 0 0) 0x7fa7d0003530 con 0x7fa7e00ad680 2023-11-28T18:30:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.009+0000 7fa7e4dfb640 1 --2- 172.21.15.105:0/2670213316 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7e00cc760 0x7fa7e00ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:07.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.009+0000 7fa7e4dfb640 1 -- 172.21.15.105:0/2670213316 >> v1:172.21.15.105:6789/0 conn(0x7fa7e00ad680 legacy=0x7fa7e015da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:07.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.009+0000 7fa7e4dfb640 1 -- 172.21.15.105:0/2670213316 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e0071380 con 0x7fa7e0159f40 2023-11-28T18:30:07.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.009+0000 7fa7dffff640 1 --2- 172.21.15.105:0/2670213316 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7e00cc760 0x7fa7e00ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:30:07.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.009+0000 7fa7e4dfb640 1 --2- 172.21.15.105:0/2670213316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7e0159f40 0x7fa7e00caf60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40097c0 tx=0x7fa7d401c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96614ddf42fc46e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:07.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.009+0000 7fa7df7fe640 1 -- 172.21.15.105:0/2670213316 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8001a00 con 0x7fa7e0159f40 2023-11-28T18:30:07.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.009+0000 7fa7df7fe640 1 -- 172.21.15.105:0/2670213316 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d40a7050 con 0x7fa7e0159f40 2023-11-28T18:30:07.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.009+0000 7fa7df7fe640 1 -- 172.21.15.105:0/2670213316 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40ab600 con 0x7fa7e0159f40 2023-11-28T18:30:07.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7e705f640 1 -- 172.21.15.105:0/2670213316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7e0159f40 msgr2=0x7fa7e00caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:07.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7e705f640 1 --2- 172.21.15.105:0/2670213316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7e0159f40 0x7fa7e00caf60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40097c0 tx=0x7fa7d401c2e0 comp rx=0 tx=0).stop 2023-11-28T18:30:07.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7e705f640 1 -- 172.21.15.105:0/2670213316 shutdown_connections 2023-11-28T18:30:07.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7e705f640 1 --2- 172.21.15.105:0/2670213316 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7e00cc760 0x7fa7e00ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:07.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7e705f640 1 --2- 172.21.15.105:0/2670213316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7e0159f40 0x7fa7e00caf60 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:07.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7e705f640 1 -- 172.21.15.105:0/2670213316 >> 172.21.15.105:0/2670213316 conn(0x7fa7e00ceb90 msgr2=0x7fa7e00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:07.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7e705f640 1 -- 172.21.15.105:0/2670213316 shutdown_connections 2023-11-28T18:30:07.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7e705f640 1 -- 172.21.15.105:0/2670213316 wait complete. 2023-11-28T18:30:07.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7e705f640 1 Processor -- start 2023-11-28T18:30:07.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7e705f640 1 -- start start 2023-11-28T18:30:07.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7e705f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7e00ad680 0x7fa7e014cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:07.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7e705f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7e00cc760 0x7fa7e00b3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:07.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7e705f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7e0159f40 0x7fa7e00ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:07.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7e705f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e015a520 con 0x7fa7e00cc760 2023-11-28T18:30:07.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7e705f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e015cef0 con 0x7fa7e0159f40 2023-11-28T18:30:07.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7e705f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e015d1d0 con 0x7fa7e00ad680 2023-11-28T18:30:07.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7dffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7e00cc760 0x7fa7e00b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:07.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7dffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7e00cc760 0x7fa7e00b3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50118/0 (socket says 172.21.15.105:50118) 2023-11-28T18:30:07.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7dffff640 1 -- 172.21.15.105:0/1500730528 learned_addr learned my addr 172.21.15.105:0/1500730528 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:07.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7e4dfb640 1 --2- 172.21.15.105:0/1500730528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7e00ad680 0x7fa7e014cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:07.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7e55fc640 1 --2- 172.21.15.105:0/1500730528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7e0159f40 0x7fa7e00ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:07.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7dffff640 1 -- 172.21.15.105:0/1500730528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7e00ad680 msgr2=0x7fa7e014cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:07.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7dffff640 1 --2- 172.21.15.105:0/1500730528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7e00ad680 0x7fa7e014cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:07.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7dffff640 1 -- 172.21.15.105:0/1500730528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7e0159f40 msgr2=0x7fa7e00ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:07.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7dffff640 1 --2- 172.21.15.105:0/1500730528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7e0159f40 0x7fa7e00ae700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:07.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7dffff640 1 -- 172.21.15.105:0/1500730528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7cc009420 con 0x7fa7e00cc760 2023-11-28T18:30:07.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.013+0000 7fa7e55fc640 1 --2- 172.21.15.105:0/1500730528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7e0159f40 0x7fa7e00ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:07.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.017+0000 7fa7dffff640 1 --2- 172.21.15.105:0/1500730528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7e00cc760 0x7fa7e00b3030 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00ebe0 tx=0x7fa7cc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:07.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.017+0000 7fa7dd7fa640 1 -- 172.21.15.105:0/1500730528 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc00eee0 con 0x7fa7e00cc760 2023-11-28T18:30:07.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.017+0000 7fa7dd7fa640 1 -- 172.21.15.105:0/1500730528 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7cc00cc30 con 0x7fa7e00cc760 2023-11-28T18:30:07.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.017+0000 7fa7dd7fa640 1 -- 172.21.15.105:0/1500730528 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc013470 con 0x7fa7e00cc760 2023-11-28T18:30:07.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.017+0000 7fa7e705f640 1 -- 172.21.15.105:0/1500730528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d4009420 con 0x7fa7e00cc760 2023-11-28T18:30:07.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.017+0000 7fa7e705f640 1 -- 172.21.15.105:0/1500730528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e015aa50 con 0x7fa7e00cc760 2023-11-28T18:30:07.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.017+0000 7fa7e705f640 1 -- 172.21.15.105:0/1500730528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e0067570 con 0x7fa7e00cc760 2023-11-28T18:30:07.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.017+0000 7fa7dd7fa640 1 -- 172.21.15.105:0/1500730528 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa7cc0135d0 con 0x7fa7e00cc760 2023-11-28T18:30:07.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.017+0000 7fa7dd7fa640 1 --2- 172.21.15.105:0/1500730528 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa7c40668d0 0x7fa7c4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:07.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.017+0000 7fa7dd7fa640 1 -- 172.21.15.105:0/1500730528 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa7cc084870 con 0x7fa7e00cc760 2023-11-28T18:30:07.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.017+0000 7fa7e4dfb640 1 --2- 172.21.15.105:0/1500730528 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa7c40668d0 0x7fa7c4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:07.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.021+0000 7fa7e4dfb640 1 --2- 172.21.15.105:0/1500730528 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa7c40668d0 0x7fa7c4068d90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40ad000 tx=0x7fa7d40a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:07.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.021+0000 7fa7dd7fa640 1 -- 172.21.15.105:0/1500730528 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa7cc051980 con 0x7fa7e00cc760 2023-11-28T18:30:07.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:07.209+0000 7fa7e705f640 1 -- 172.21.15.105:0/1500730528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa7e006d1f0 con 0x7fa7e00cc760 2023-11-28T18:30:07.555 INFO:tasks.ceph.mds.c.smithi105.stderr:2023-11-28T18:30:07.553+0000 7f31b1335640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T18:30:08.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.193+0000 7fa7dd7fa640 1 -- 172.21.15.105:0/1500730528 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v102) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa7cc054fa0 con 0x7fa7e00cc760 2023-11-28T18:30:08.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.193+0000 7fa7e705f640 1 -- 172.21.15.105:0/1500730528 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa7c40668d0 msgr2=0x7fa7c4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:08.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.193+0000 7fa7e705f640 1 --2- 172.21.15.105:0/1500730528 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa7c40668d0 0x7fa7c4068d90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40ad000 tx=0x7fa7d40a6000 comp rx=0 tx=0).stop 2023-11-28T18:30:08.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.193+0000 7fa7e705f640 1 -- 172.21.15.105:0/1500730528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7e00cc760 msgr2=0x7fa7e00b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:08.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.193+0000 7fa7e705f640 1 --2- 172.21.15.105:0/1500730528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7e00cc760 0x7fa7e00b3030 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00ebe0 tx=0x7fa7cc00c520 comp rx=0 tx=0).stop 2023-11-28T18:30:08.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.197+0000 7fa7e705f640 1 -- 172.21.15.105:0/1500730528 shutdown_connections 2023-11-28T18:30:08.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.197+0000 7fa7e705f640 1 --2- 172.21.15.105:0/1500730528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7e0159f40 0x7fa7e00ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:08.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.197+0000 7fa7e705f640 1 --2- 172.21.15.105:0/1500730528 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa7c40668d0 0x7fa7c4068d90 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:08.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.197+0000 7fa7e705f640 1 --2- 172.21.15.105:0/1500730528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7e00cc760 0x7fa7e00b3030 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:08.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.197+0000 7fa7e705f640 1 --2- 172.21.15.105:0/1500730528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7e00ad680 0x7fa7e014cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:08.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.197+0000 7fa7e705f640 1 -- 172.21.15.105:0/1500730528 >> 172.21.15.105:0/1500730528 conn(0x7fa7e00ceb90 msgr2=0x7fa7e0163040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:08.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.197+0000 7fa7e705f640 1 -- 172.21.15.105:0/1500730528 shutdown_connections 2023-11-28T18:30:08.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.197+0000 7fa7e705f640 1 -- 172.21.15.105:0/1500730528 wait complete. 2023-11-28T18:30:08.219 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T18:30:08.220 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T18:30:08.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.365+0000 7f7f0405f640 1 Processor -- start 2023-11-28T18:30:08.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.365+0000 7f7f0405f640 1 -- start start 2023-11-28T18:30:08.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.365+0000 7f7f0405f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7efc15d300 0x7f7efc0d37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:08.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.365+0000 7f7f0405f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7efc0d0c60 con 0x7f7efc0d1d60 2023-11-28T18:30:08.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.365+0000 7f7f0405f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7efc0d0eb0 con 0x7f7efc0d3ce0 2023-11-28T18:30:08.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.365+0000 7f7f0405f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7efc0d1470 con 0x7f7efc15d300 2023-11-28T18:30:08.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.365+0000 7f7f01dfb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7efc15d300 0x7f7efc0d37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:08.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.365+0000 7f7f025fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7efc0d1d60 0x7f7efc0c7510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40186/0 (socket says 172.21.15.105:40186) 2023-11-28T18:30:08.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.365+0000 7f7f025fc640 1 -- 172.21.15.105:0/4045848682 learned_addr learned my addr 172.21.15.105:0/4045848682 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:08.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.365+0000 7f7f01dfb640 1 -- 172.21.15.105:0/4045848682 >> v1:172.21.15.148:6789/0 conn(0x7f7efc0d3ce0 legacy=0x7f7efc0d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:08.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.365+0000 7f7f01dfb640 1 -- 172.21.15.105:0/4045848682 >> v1:172.21.15.105:6789/0 conn(0x7f7efc0d1d60 legacy=0x7f7efc0c7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:08.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.365+0000 7f7f01dfb640 1 -- 172.21.15.105:0/4045848682 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc0d1190 con 0x7f7efc15d300 2023-11-28T18:30:08.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.365+0000 7f7f00df9640 1 -- 172.21.15.105:0/4045848682 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792469363 0 0) 0x7f7efc0d0c60 con 0x7f7efc0d1d60 2023-11-28T18:30:08.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.369+0000 7f7f01dfb640 1 --2- 172.21.15.105:0/4045848682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7efc15d300 0x7f7efc0d37a0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f7ef80097c0 tx=0x7f7ef801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1060e31e503a94c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:08.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.369+0000 7f7f00df9640 1 -- 172.21.15.105:0/4045848682 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef80ab600 con 0x7f7efc15d300 2023-11-28T18:30:08.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.369+0000 7f7f00df9640 1 -- 172.21.15.105:0/4045848682 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef80a7050 con 0x7f7efc15d300 2023-11-28T18:30:08.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.369+0000 7f7f00df9640 1 -- 172.21.15.105:0/4045848682 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef80ab980 con 0x7f7efc15d300 2023-11-28T18:30:08.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.369+0000 7f7f0405f640 1 -- 172.21.15.105:0/4045848682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7efc15d300 msgr2=0x7f7efc0d37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:08.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.369+0000 7f7f0405f640 1 --2- 172.21.15.105:0/4045848682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7efc15d300 0x7f7efc0d37a0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f7ef80097c0 tx=0x7f7ef801c3f0 comp rx=0 tx=0).stop 2023-11-28T18:30:08.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.369+0000 7f7f0405f640 1 -- 172.21.15.105:0/4045848682 shutdown_connections 2023-11-28T18:30:08.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.369+0000 7f7f0405f640 1 --2- 172.21.15.105:0/4045848682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7efc15d300 0x7f7efc0d37a0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:08.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.369+0000 7f7f0405f640 1 -- 172.21.15.105:0/4045848682 >> 172.21.15.105:0/4045848682 conn(0x7f7efc09c360 msgr2=0x7f7efc0d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:08.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.369+0000 7f7f0405f640 1 -- 172.21.15.105:0/4045848682 shutdown_connections 2023-11-28T18:30:08.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.369+0000 7f7f0405f640 1 -- 172.21.15.105:0/4045848682 wait complete. 2023-11-28T18:30:08.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7f0405f640 1 Processor -- start 2023-11-28T18:30:08.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7f0405f640 1 -- start start 2023-11-28T18:30:08.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7f0405f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7efc0d1d60 0x7f7efc0b75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:08.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7f0405f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7efc0d3ce0 0x7f7efc0cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:08.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7f0405f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7efc15d300 0x7f7efc0cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:08.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7f0405f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc1591f0 con 0x7f7efc0d1d60 2023-11-28T18:30:08.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7f0405f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc0cae60 con 0x7f7efc15d300 2023-11-28T18:30:08.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7f0405f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc0ca290 con 0x7f7efc0d3ce0 2023-11-28T18:30:08.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7f01dfb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7efc0d1d60 0x7f7efc0b75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:08.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7f01dfb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7efc0d1d60 0x7f7efc0b75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50126/0 (socket says 172.21.15.105:50126) 2023-11-28T18:30:08.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7f01dfb640 1 -- 172.21.15.105:0/1391879856 learned_addr learned my addr 172.21.15.105:0/1391879856 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:08.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7f015fa640 1 --2- 172.21.15.105:0/1391879856 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7efc0d3ce0 0x7f7efc0cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:08.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7f025fc640 1 --2- 172.21.15.105:0/1391879856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7efc15d300 0x7f7efc0cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:08.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7f01dfb640 1 -- 172.21.15.105:0/1391879856 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7efc0d3ce0 msgr2=0x7f7efc0cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:08.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7f01dfb640 1 --2- 172.21.15.105:0/1391879856 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7efc0d3ce0 0x7f7efc0cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:08.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7f01dfb640 1 -- 172.21.15.105:0/1391879856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7efc15d300 msgr2=0x7f7efc0cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:08.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7f01dfb640 1 --2- 172.21.15.105:0/1391879856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7efc15d300 0x7f7efc0cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:08.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7f01dfb640 1 -- 172.21.15.105:0/1391879856 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef8009420 con 0x7f7efc0d1d60 2023-11-28T18:30:08.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7f01dfb640 1 --2- 172.21.15.105:0/1391879856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7efc0d1d60 0x7f7efc0b75d0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f7ef801a020 tx=0x7f7ef801ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:08.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7ef27fc640 1 -- 172.21.15.105:0/1391879856 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8004020 con 0x7f7efc0d1d60 2023-11-28T18:30:08.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7ef27fc640 1 -- 172.21.15.105:0/1391879856 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef8004180 con 0x7f7efc0d1d60 2023-11-28T18:30:08.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7ef27fc640 1 -- 172.21.15.105:0/1391879856 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef80044f0 con 0x7f7efc0d1d60 2023-11-28T18:30:08.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7f0405f640 1 -- 172.21.15.105:0/1391879856 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7efc0cc480 con 0x7f7efc0d1d60 2023-11-28T18:30:08.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.373+0000 7f7f0405f640 1 -- 172.21.15.105:0/1391879856 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7efc0ca810 con 0x7f7efc0d1d60 2023-11-28T18:30:08.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.377+0000 7f7f0405f640 1 -- 172.21.15.105:0/1391879856 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7efc067570 con 0x7f7efc0d1d60 2023-11-28T18:30:08.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.377+0000 7f7ef27fc640 1 -- 172.21.15.105:0/1391879856 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f7ef8003620 con 0x7f7efc0d1d60 2023-11-28T18:30:08.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.377+0000 7f7ef27fc640 1 --2- 172.21.15.105:0/1391879856 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7ee0066900 0x7f7ee0068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:08.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.377+0000 7f7ef27fc640 1 -- 172.21.15.105:0/1391879856 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7ef811ead0 con 0x7f7efc0d1d60 2023-11-28T18:30:08.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.377+0000 7f7f015fa640 1 --2- 172.21.15.105:0/1391879856 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7ee0066900 0x7f7ee0068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:08.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.377+0000 7f7f015fa640 1 --2- 172.21.15.105:0/1391879856 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7ee0066900 0x7f7ee0068dc0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7efc1f9420 tx=0x7f7eec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:08.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.381+0000 7f7ef27fc640 1 -- 172.21.15.105:0/1391879856 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7ef80ebb30 con 0x7f7efc0d1d60 2023-11-28T18:30:08.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:08.565+0000 7f7f0405f640 1 -- 172.21.15.105:0/1391879856 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7efc06c0c0 con 0x7f7efc0d1d60 2023-11-28T18:30:09.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.193+0000 7f7ef27fc640 1 -- 172.21.15.105:0/1391879856 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v103) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7ef80b00c0 con 0x7f7efc0d1d60 2023-11-28T18:30:09.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.197+0000 7f7f0405f640 1 -- 172.21.15.105:0/1391879856 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7ee0066900 msgr2=0x7f7ee0068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:09.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.197+0000 7f7f0405f640 1 --2- 172.21.15.105:0/1391879856 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7ee0066900 0x7f7ee0068dc0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7efc1f9420 tx=0x7f7eec009040 comp rx=0 tx=0).stop 2023-11-28T18:30:09.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.197+0000 7f7f0405f640 1 -- 172.21.15.105:0/1391879856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7efc0d1d60 msgr2=0x7f7efc0b75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:09.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.197+0000 7f7f0405f640 1 --2- 172.21.15.105:0/1391879856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7efc0d1d60 0x7f7efc0b75d0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f7ef801a020 tx=0x7f7ef801ac30 comp rx=0 tx=0).stop 2023-11-28T18:30:09.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.197+0000 7f7f0405f640 1 -- 172.21.15.105:0/1391879856 shutdown_connections 2023-11-28T18:30:09.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.197+0000 7f7f0405f640 1 --2- 172.21.15.105:0/1391879856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7efc15d300 0x7f7efc0cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.197+0000 7f7f0405f640 1 --2- 172.21.15.105:0/1391879856 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7efc0d3ce0 0x7f7efc0cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.197+0000 7f7f0405f640 1 --2- 172.21.15.105:0/1391879856 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7ee0066900 0x7f7ee0068dc0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.197+0000 7f7f0405f640 1 --2- 172.21.15.105:0/1391879856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7efc0d1d60 0x7f7efc0b75d0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.197+0000 7f7f0405f640 1 -- 172.21.15.105:0/1391879856 >> 172.21.15.105:0/1391879856 conn(0x7f7efc09c360 msgr2=0x7f7efc0b5140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:09.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.197+0000 7f7f0405f640 1 -- 172.21.15.105:0/1391879856 shutdown_connections 2023-11-28T18:30:09.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.197+0000 7f7f0405f640 1 -- 172.21.15.105:0/1391879856 wait complete. 2023-11-28T18:30:09.218 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T18:30:09.218 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:30:09.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.357+0000 7f1061879640 1 Processor -- start 2023-11-28T18:30:09.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.361+0000 7f1061879640 1 -- start start 2023-11-28T18:30:09.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.361+0000 7f1061879640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f105c0b4290 0x7f105c0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:09.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.361+0000 7f1061879640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f105c15b030 0x7f105c0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:09.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.361+0000 7f1061879640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f105c0b4bb0 con 0x7f105c071960 2023-11-28T18:30:09.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.361+0000 7f1061879640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105c0701d0 con 0x7f105c15b030 2023-11-28T18:30:09.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.361+0000 7f1061879640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105c06df80 con 0x7f105c0b4290 2023-11-28T18:30:09.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.361+0000 7f105b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f105c15b030 0x7f105c0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:09.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.361+0000 7f105b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f105c15b030 0x7f105c0709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50140/0 (socket says 172.21.15.105:50140) 2023-11-28T18:30:09.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.361+0000 7f105b7fe640 1 -- 172.21.15.105:0/569659487 learned_addr learned my addr 172.21.15.105:0/569659487 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:09.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.361+0000 7f105affd640 1 --2- 172.21.15.105:0/569659487 >> v2:172.21.15.148:3300/0 conn(0x7f105c0b4290 0x7f105c0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:09.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.361+0000 7f105b7fe640 1 -- 172.21.15.105:0/569659487 >> v2:172.21.15.148:3300/0 conn(0x7f105c0b4290 msgr2=0x7f105c0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:09.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.361+0000 7f105b7fe640 1 --2- 172.21.15.105:0/569659487 >> v2:172.21.15.148:3300/0 conn(0x7f105c0b4290 0x7f105c0b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.361+0000 7f105b7fe640 1 -- 172.21.15.105:0/569659487 >> v1:172.21.15.148:6789/0 conn(0x7f105c071960 legacy=0x7f105c15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:09.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.361+0000 7f105b7fe640 1 -- 172.21.15.105:0/569659487 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c071df0 con 0x7f105c15b030 2023-11-28T18:30:09.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.365+0000 7f105b7fe640 1 --2- 172.21.15.105:0/569659487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c15b030 0x7f105c0709b0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f105c073a70 tx=0x7f105401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aec0a93d1a6911d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:09.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.365+0000 7f105a7fc640 1 -- 172.21.15.105:0/569659487 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10540a5730 con 0x7f105c15b030 2023-11-28T18:30:09.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.365+0000 7f105a7fc640 1 -- 172.21.15.105:0/569659487 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10540a5890 con 0x7f105c15b030 2023-11-28T18:30:09.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.365+0000 7f105a7fc640 1 -- 172.21.15.105:0/569659487 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10540a5c20 con 0x7f105c15b030 2023-11-28T18:30:09.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.365+0000 7f1061879640 1 -- 172.21.15.105:0/569659487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c15b030 msgr2=0x7f105c0709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:09.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.365+0000 7f1061879640 1 --2- 172.21.15.105:0/569659487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c15b030 0x7f105c0709b0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f105c073a70 tx=0x7f105401c960 comp rx=0 tx=0).stop 2023-11-28T18:30:09.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.365+0000 7f1061879640 1 -- 172.21.15.105:0/569659487 shutdown_connections 2023-11-28T18:30:09.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.365+0000 7f1061879640 1 --2- 172.21.15.105:0/569659487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c15b030 0x7f105c0709b0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.365+0000 7f1061879640 1 --2- 172.21.15.105:0/569659487 >> v2:172.21.15.148:3300/0 conn(0x7f105c0b4290 0x7f105c0b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.365+0000 7f1061879640 1 -- 172.21.15.105:0/569659487 >> 172.21.15.105:0/569659487 conn(0x7f105c0ceb90 msgr2=0x7f105c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:09.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.365+0000 7f1061879640 1 -- 172.21.15.105:0/569659487 shutdown_connections 2023-11-28T18:30:09.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.365+0000 7f1061879640 1 -- 172.21.15.105:0/569659487 wait complete. 2023-11-28T18:30:09.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.365+0000 7f1061879640 1 Processor -- start 2023-11-28T18:30:09.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.365+0000 7f1061879640 1 -- start start 2023-11-28T18:30:09.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f1061879640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f105c0b4290 0x7f105c0b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:09.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f1061879640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c15b030 0x7f105c149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:09.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f1061879640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f105c14b920 0x7f105c1503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:09.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f1061879640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c0b38b0 con 0x7f105c15b030 2023-11-28T18:30:09.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f1061879640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c14a4f0 con 0x7f105c0b4290 2023-11-28T18:30:09.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f1061879640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105c14a7d0 con 0x7f105c14b920 2023-11-28T18:30:09.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f1053fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c15b030 0x7f105c149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:09.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f105affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f105c0b4290 0x7f105c0b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:09.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f1053fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c15b030 0x7f105c149b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50154/0 (socket says 172.21.15.105:50154) 2023-11-28T18:30:09.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f1053fff640 1 -- 172.21.15.105:0/3482909865 learned_addr learned my addr 172.21.15.105:0/3482909865 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:09.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f105b7fe640 1 --2- 172.21.15.105:0/3482909865 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f105c14b920 0x7f105c1503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:09.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f1053fff640 1 -- 172.21.15.105:0/3482909865 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f105c14b920 msgr2=0x7f105c1503f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:09.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f1053fff640 1 --2- 172.21.15.105:0/3482909865 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f105c14b920 0x7f105c1503f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f1053fff640 1 -- 172.21.15.105:0/3482909865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f105c0b4290 msgr2=0x7f105c0b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:09.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f1053fff640 1 --2- 172.21.15.105:0/3482909865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f105c0b4290 0x7f105c0b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f1053fff640 1 -- 172.21.15.105:0/3482909865 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1054009b30 con 0x7f105c15b030 2023-11-28T18:30:09.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f1053fff640 1 --2- 172.21.15.105:0/3482909865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c15b030 0x7f105c149b90 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f104400c450 tx=0x7f104400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:09.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f1058ff9640 1 -- 172.21.15.105:0/3482909865 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1044012830 con 0x7f105c15b030 2023-11-28T18:30:09.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f1058ff9640 1 -- 172.21.15.105:0/3482909865 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10440129b0 con 0x7f105c15b030 2023-11-28T18:30:09.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f1058ff9640 1 -- 172.21.15.105:0/3482909865 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1044012d60 con 0x7f105c15b030 2023-11-28T18:30:09.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f1061879640 1 -- 172.21.15.105:0/3482909865 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105c14aab0 con 0x7f105c15b030 2023-11-28T18:30:09.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f1061879640 1 -- 172.21.15.105:0/3482909865 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105c14b070 con 0x7f105c15b030 2023-11-28T18:30:09.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f1061879640 1 -- 172.21.15.105:0/3482909865 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105c14b290 con 0x7f105c15b030 2023-11-28T18:30:09.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.369+0000 7f1058ff9640 1 -- 172.21.15.105:0/3482909865 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f104401c460 con 0x7f105c15b030 2023-11-28T18:30:09.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.373+0000 7f1058ff9640 1 --2- 172.21.15.105:0/3482909865 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f103c066900 0x7f103c068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:09.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.373+0000 7f1058ff9640 1 -- 172.21.15.105:0/3482909865 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f10440842b0 con 0x7f105c15b030 2023-11-28T18:30:09.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.373+0000 7f105affd640 1 --2- 172.21.15.105:0/3482909865 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f103c066900 0x7f103c068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:09.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.373+0000 7f105affd640 1 --2- 172.21.15.105:0/3482909865 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f103c066900 0x7f103c068dc0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f105c1f77e0 tx=0x7f104c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:09.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.377+0000 7f1058ff9640 1 -- 172.21.15.105:0/3482909865 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1044051440 con 0x7f105c15b030 2023-11-28T18:30:09.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.557+0000 7f1061879640 1 -- 172.21.15.105:0/3482909865 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f105c067570 con 0x7f105c15b030 2023-11-28T18:30:09.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.557+0000 7f1058ff9640 1 -- 172.21.15.105:0/3482909865 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4929 (secure 0 0 0) 0x7f1044054a60 con 0x7f105c15b030 2023-11-28T18:30:09.559 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:09.559 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":103,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/2603493310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2603493310},{"type":"v1","addr":"172.21.15.148:6834","nonce":2603493310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":6803,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3664281230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":3664281230},{"type":"v1","addr":"172.21.15.148:6835","nonce":3664281230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":8142,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3968059262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3968059262},{"type":"v1","addr":"172.21.15.105:6836","nonce":3968059262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:30:05.525117+0000","modified":"2023-11-28T18:30:08.567624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8145":{"gid":8145,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2782729992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2782729992},{"type":"v1","addr":"172.21.15.105:6837","nonce":2782729992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-11-28T18:30:09.559 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 103 2023-11-28T18:30:09.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.561+0000 7f1061879640 1 -- 172.21.15.105:0/3482909865 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f103c066900 msgr2=0x7f103c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:09.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.561+0000 7f1061879640 1 --2- 172.21.15.105:0/3482909865 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f103c066900 0x7f103c068dc0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f105c1f77e0 tx=0x7f104c008040 comp rx=0 tx=0).stop 2023-11-28T18:30:09.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.561+0000 7f1061879640 1 -- 172.21.15.105:0/3482909865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c15b030 msgr2=0x7f105c149b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:09.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.561+0000 7f1061879640 1 --2- 172.21.15.105:0/3482909865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c15b030 0x7f105c149b90 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f104400c450 tx=0x7f104400c910 comp rx=0 tx=0).stop 2023-11-28T18:30:09.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.561+0000 7f1061879640 1 -- 172.21.15.105:0/3482909865 shutdown_connections 2023-11-28T18:30:09.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.561+0000 7f1061879640 1 --2- 172.21.15.105:0/3482909865 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f105c14b920 0x7f105c1503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.561+0000 7f1061879640 1 --2- 172.21.15.105:0/3482909865 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f103c066900 0x7f103c068dc0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.561+0000 7f1061879640 1 --2- 172.21.15.105:0/3482909865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f105c15b030 0x7f105c149b90 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.561+0000 7f1061879640 1 --2- 172.21.15.105:0/3482909865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f105c0b4290 0x7f105c0b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.561+0000 7f1061879640 1 -- 172.21.15.105:0/3482909865 >> 172.21.15.105:0/3482909865 conn(0x7f105c0ceb90 msgr2=0x7f105c152070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:09.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.561+0000 7f1061879640 1 -- 172.21.15.105:0/3482909865 shutdown_connections 2023-11-28T18:30:09.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.561+0000 7f1061879640 1 -- 172.21.15.105:0/3482909865 wait complete. 2023-11-28T18:30:09.580 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:30:09.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.717+0000 7ff4fd196640 1 Processor -- start 2023-11-28T18:30:09.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.717+0000 7ff4fd196640 1 -- start start 2023-11-28T18:30:09.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.717+0000 7ff4fd196640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff4f80cc760 0x7ff4f8164d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:09.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.717+0000 7ff4fd196640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f80ccc10 con 0x7ff4f80caf60 2023-11-28T18:30:09.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.717+0000 7ff4fd196640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f806bf50 con 0x7ff4f8159f40 2023-11-28T18:30:09.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.717+0000 7ff4fd196640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f80cb3c0 con 0x7ff4f80cc760 2023-11-28T18:30:09.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.717+0000 7ff4ee59c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff4f80caf60 0x7ff4f80ad850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40200/0 (socket says 172.21.15.105:40200) 2023-11-28T18:30:09.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.717+0000 7ff4ee59c640 1 -- 172.21.15.105:0/261798726 learned_addr learned my addr 172.21.15.105:0/261798726 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.717+0000 7ff4f759e640 1 --2- 172.21.15.105:0/261798726 >> v2:172.21.15.148:3300/0 conn(0x7ff4f80cc760 0x7ff4f8164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.717+0000 7ff4f659c640 1 -- 172.21.15.105:0/261798726 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2590725539 0 0) 0x7ff4f80ccc10 con 0x7ff4f80caf60 2023-11-28T18:30:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.717+0000 7ff4f659c640 1 -- 172.21.15.105:0/261798726 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4e4003530 con 0x7ff4f80caf60 2023-11-28T18:30:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.717+0000 7ff4f659c640 1 -- 172.21.15.105:0/261798726 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2448926372 0 0) 0x7ff4e4003530 con 0x7ff4f80caf60 2023-11-28T18:30:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.717+0000 7ff4f659c640 1 -- 172.21.15.105:0/261798726 >> v2:172.21.15.148:3300/0 conn(0x7ff4f80cc760 msgr2=0x7ff4f8164d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.717+0000 7ff4f659c640 1 --2- 172.21.15.105:0/261798726 >> v2:172.21.15.148:3300/0 conn(0x7ff4f80cc760 0x7ff4f8164d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.717+0000 7ff4f659c640 1 -- 172.21.15.105:0/261798726 >> v1:172.21.15.148:6789/0 conn(0x7ff4f8159f40 legacy=0x7ff4f80ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:09.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.717+0000 7ff4f659c640 1 -- 172.21.15.105:0/261798726 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f815a320 con 0x7ff4f80caf60 2023-11-28T18:30:09.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.717+0000 7ff4f659c640 1 -- 172.21.15.105:0/261798726 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7ff4e0001680 con 0x7ff4f80caf60 2023-11-28T18:30:09.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.717+0000 7ff4f659c640 1 -- 172.21.15.105:0/261798726 >> v1:172.21.15.105:6789/0 conn(0x7ff4f80caf60 legacy=0x7ff4f80ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:09.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.717+0000 7ff4f659c640 1 --2- 172.21.15.105:0/261798726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4e4003980 0x7ff4e4003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:09.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.721+0000 7ff4f659c640 1 --2- 172.21.15.105:0/261798726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4e4004380 0x7ff4e4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:09.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.721+0000 7ff4f659c640 1 --2- 172.21.15.105:0/261798726 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4e40050f0 0x7ff4e400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:09.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.721+0000 7ff4f659c640 1 -- 172.21.15.105:0/261798726 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80cb3c0 con 0x7ff4e4004380 2023-11-28T18:30:09.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.721+0000 7ff4f659c640 1 -- 172.21.15.105:0/261798726 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e400c030 con 0x7ff4e4003980 2023-11-28T18:30:09.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.721+0000 7ff4f659c640 1 -- 172.21.15.105:0/261798726 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e400c1f0 con 0x7ff4e40050f0 2023-11-28T18:30:09.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.721+0000 7ff4ee59c640 1 --2- 172.21.15.105:0/261798726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4e4004380 0x7ff4e4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:09.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.721+0000 7ff4f6d9d640 1 --2- 172.21.15.105:0/261798726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4e4003980 0x7ff4e4003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:09.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.721+0000 7ff4f759e640 1 --2- 172.21.15.105:0/261798726 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4e40050f0 0x7ff4e400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:09.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.721+0000 7ff4ee59c640 1 -- 172.21.15.105:0/261798726 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4e40050f0 msgr2=0x7ff4e400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:09.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.721+0000 7ff4ee59c640 1 --2- 172.21.15.105:0/261798726 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4e40050f0 0x7ff4e400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.721+0000 7ff4ee59c640 1 -- 172.21.15.105:0/261798726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4e4003980 msgr2=0x7ff4e4003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:09.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.721+0000 7ff4ee59c640 1 --2- 172.21.15.105:0/261798726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4e4003980 0x7ff4e4003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.721+0000 7ff4ee59c640 1 -- 172.21.15.105:0/261798726 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e4003530 con 0x7ff4e4004380 2023-11-28T18:30:09.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.721+0000 7ff4f759e640 1 -- 172.21.15.105:0/261798726 reap_dead start 2023-11-28T18:30:09.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.721+0000 7ff4ee59c640 1 --2- 172.21.15.105:0/261798726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4e4004380 0x7ff4e4004850 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000c190 tx=0x7ff4e001cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d0e2c0286b8b91 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:09.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.721+0000 7ff4f659c640 1 -- 172.21.15.105:0/261798726 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0005ae0 con 0x7ff4e4004380 2023-11-28T18:30:09.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.721+0000 7ff4f659c640 1 -- 172.21.15.105:0/261798726 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e0005c60 con 0x7ff4e4004380 2023-11-28T18:30:09.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.721+0000 7ff4f659c640 1 -- 172.21.15.105:0/261798726 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e00072b0 con 0x7ff4e4004380 2023-11-28T18:30:09.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.725+0000 7ff4fd196640 1 -- 172.21.15.105:0/261798726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4e4004380 msgr2=0x7ff4e4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:09.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.725+0000 7ff4fd196640 1 --2- 172.21.15.105:0/261798726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4e4004380 0x7ff4e4004850 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000c190 tx=0x7ff4e001cb10 comp rx=0 tx=0).stop 2023-11-28T18:30:09.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.725+0000 7ff4fd196640 1 -- 172.21.15.105:0/261798726 shutdown_connections 2023-11-28T18:30:09.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.725+0000 7ff4fd196640 1 --2- 172.21.15.105:0/261798726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4e4004380 0x7ff4e4004850 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.725+0000 7ff4fd196640 1 -- 172.21.15.105:0/261798726 >> 172.21.15.105:0/261798726 conn(0x7ff4f80ceb90 msgr2=0x7ff4f80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:09.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.725+0000 7ff4fd196640 1 -- 172.21.15.105:0/261798726 shutdown_connections 2023-11-28T18:30:09.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.725+0000 7ff4fd196640 1 -- 172.21.15.105:0/261798726 wait complete. 2023-11-28T18:30:09.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.725+0000 7ff4fd196640 1 Processor -- start 2023-11-28T18:30:09.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.725+0000 7ff4fd196640 1 -- start start 2023-11-28T18:30:09.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.725+0000 7ff4fd196640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4e4004380 0x7ff4f81f8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:09.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.725+0000 7ff4fd196640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4f81e8a30 0x7ff4f80af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:09.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.725+0000 7ff4fd196640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4f81e8e90 0x7ff4f80afc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:09.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.725+0000 7ff4fd196640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f81e7cb0 con 0x7ff4f81e8a30 2023-11-28T18:30:09.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.725+0000 7ff4fd196640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f81e7f50 con 0x7ff4f81e8e90 2023-11-28T18:30:09.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.725+0000 7ff4fd196640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f81e8230 con 0x7ff4e4004380 2023-11-28T18:30:09.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.725+0000 7ff4ee59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4f81e8a30 0x7ff4f80af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:09.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.725+0000 7ff4ee59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4f81e8a30 0x7ff4f80af750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50174/0 (socket says 172.21.15.105:50174) 2023-11-28T18:30:09.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.725+0000 7ff4ee59c640 1 -- 172.21.15.105:0/2525112836 learned_addr learned my addr 172.21.15.105:0/2525112836 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:09.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.725+0000 7ff4f6d9d640 1 --2- 172.21.15.105:0/2525112836 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4e4004380 0x7ff4f81f8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:09.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.725+0000 7ff4f759e640 1 --2- 172.21.15.105:0/2525112836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4f81e8e90 0x7ff4f80afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:09.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.729+0000 7ff4ee59c640 1 -- 172.21.15.105:0/2525112836 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4e4004380 msgr2=0x7ff4f81f8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:09.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.729+0000 7ff4ee59c640 1 --2- 172.21.15.105:0/2525112836 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4e4004380 0x7ff4f81f8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.729+0000 7ff4ee59c640 1 -- 172.21.15.105:0/2525112836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4f81e8e90 msgr2=0x7ff4f80afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:09.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.729+0000 7ff4ee59c640 1 --2- 172.21.15.105:0/2525112836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4f81e8e90 0x7ff4f80afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.729+0000 7ff4ee59c640 1 -- 172.21.15.105:0/2525112836 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f81e8510 con 0x7ff4f81e8a30 2023-11-28T18:30:09.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.729+0000 7ff4ee59c640 1 --2- 172.21.15.105:0/2525112836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4f81e8a30 0x7ff4f80af750 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7ff4e001c940 tx=0x7ff4e4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:09.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.729+0000 7ff4effff640 1 -- 172.21.15.105:0/2525112836 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e00b4070 con 0x7ff4f81e8a30 2023-11-28T18:30:09.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.729+0000 7ff4effff640 1 -- 172.21.15.105:0/2525112836 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e0001540 con 0x7ff4f81e8a30 2023-11-28T18:30:09.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.729+0000 7ff4effff640 1 -- 172.21.15.105:0/2525112836 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0007270 con 0x7ff4f81e8a30 2023-11-28T18:30:09.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.729+0000 7ff4fd196640 1 -- 172.21.15.105:0/2525112836 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f81f8d60 con 0x7ff4f81e8a30 2023-11-28T18:30:09.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.729+0000 7ff4fd196640 1 -- 172.21.15.105:0/2525112836 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f80c9820 con 0x7ff4f81e8a30 2023-11-28T18:30:09.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.729+0000 7ff4fd196640 1 -- 172.21.15.105:0/2525112836 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f8067570 con 0x7ff4f81e8a30 2023-11-28T18:30:09.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.729+0000 7ff4effff640 1 -- 172.21.15.105:0/2525112836 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff4e00073d0 con 0x7ff4f81e8a30 2023-11-28T18:30:09.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.729+0000 7ff4effff640 1 --2- 172.21.15.105:0/2525112836 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff4d80668c0 0x7ff4d8068d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:09.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.733+0000 7ff4effff640 1 -- 172.21.15.105:0/2525112836 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff4e011dc20 con 0x7ff4f81e8a30 2023-11-28T18:30:09.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.733+0000 7ff4f6d9d640 1 --2- 172.21.15.105:0/2525112836 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff4d80668c0 0x7ff4d8068d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:09.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.733+0000 7ff4f6d9d640 1 --2- 172.21.15.105:0/2525112836 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff4d80668c0 0x7ff4d8068d80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff4f80501d0 tx=0x7ff4e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:09.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.733+0000 7ff4effff640 1 -- 172.21.15.105:0/2525112836 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff4e00ead30 con 0x7ff4f81e8a30 2023-11-28T18:30:09.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.885+0000 7ff4fd196640 1 -- 172.21.15.105:0/2525112836 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4f80cf950 con 0x7ff4f81e8a30 2023-11-28T18:30:09.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.889+0000 7ff4effff640 1 -- 172.21.15.105:0/2525112836 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16776 (secure 0 0 0) 0x7ff4e00bf090 con 0x7ff4f81e8a30 2023-11-28T18:30:09.888 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:09.888 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":103,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:30:07.171066+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:30:03.341424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-28T18:30:04.527931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:30:09.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.889+0000 7ff4fd196640 1 -- 172.21.15.105:0/2525112836 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff4d80668c0 msgr2=0x7ff4d8068d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:09.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.889+0000 7ff4fd196640 1 --2- 172.21.15.105:0/2525112836 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff4d80668c0 0x7ff4d8068d80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff4f80501d0 tx=0x7ff4e8009040 comp rx=0 tx=0).stop 2023-11-28T18:30:09.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.889+0000 7ff4fd196640 1 -- 172.21.15.105:0/2525112836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4f81e8a30 msgr2=0x7ff4f80af750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:09.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.889+0000 7ff4fd196640 1 --2- 172.21.15.105:0/2525112836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4f81e8a30 0x7ff4f80af750 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7ff4e001c940 tx=0x7ff4e4005800 comp rx=0 tx=0).stop 2023-11-28T18:30:09.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.889+0000 7ff4fd196640 1 -- 172.21.15.105:0/2525112836 shutdown_connections 2023-11-28T18:30:09.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.889+0000 7ff4fd196640 1 --2- 172.21.15.105:0/2525112836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4f81e8e90 0x7ff4f80afc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.889+0000 7ff4fd196640 1 --2- 172.21.15.105:0/2525112836 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff4d80668c0 0x7ff4d8068d80 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.889+0000 7ff4fd196640 1 --2- 172.21.15.105:0/2525112836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4f81e8a30 0x7ff4f80af750 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.889+0000 7ff4fd196640 1 --2- 172.21.15.105:0/2525112836 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4e4004380 0x7ff4f81f8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:09.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.889+0000 7ff4fd196640 1 -- 172.21.15.105:0/2525112836 >> 172.21.15.105:0/2525112836 conn(0x7ff4f80ceb90 msgr2=0x7ff4f8155620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:09.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.889+0000 7ff4fd196640 1 -- 172.21.15.105:0/2525112836 shutdown_connections 2023-11-28T18:30:09.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:09.889+0000 7ff4fd196640 1 -- 172.21.15.105:0/2525112836 wait complete. 2023-11-28T18:30:09.909 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T18:30:09.909 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:30:10.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.049+0000 7f6310209640 1 Processor -- start 2023-11-28T18:30:10.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.049+0000 7f6310209640 1 -- start start 2023-11-28T18:30:10.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.049+0000 7f6310209640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63080d1650 0x7f63080d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:10.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.049+0000 7f6310209640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f63080c7510 0x7f63080c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:10.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.049+0000 7f6310209640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63080d0c60 con 0x7f630815d300 2023-11-28T18:30:10.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.049+0000 7f6310209640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63080d1470 con 0x7f63080c7510 2023-11-28T18:30:10.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.049+0000 7f6310209640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63080cefa0 con 0x7f63080d1650 2023-11-28T18:30:10.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.049+0000 7f630dfa5640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f630815d300 0x7f63080d37a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40202/0 (socket says 172.21.15.105:40202) 2023-11-28T18:30:10.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.049+0000 7f630dfa5640 1 -- 172.21.15.105:0/1389607107 learned_addr learned my addr 172.21.15.105:0/1389607107 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:10.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.049+0000 7f630d7a4640 1 --2- 172.21.15.105:0/1389607107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63080d1650 0x7f63080d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:10.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.049+0000 7f630e7a6640 1 --2- 172.21.15.105:0/1389607107 >> v2:172.21.15.148:3300/0 conn(0x7f63080c7510 0x7f63080c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:10.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.049+0000 7f630cfa3640 1 -- 172.21.15.105:0/1389607107 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1445200271 0 0) 0x7f63080d0c60 con 0x7f630815d300 2023-11-28T18:30:10.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.049+0000 7f630cfa3640 1 -- 172.21.15.105:0/1389607107 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62f4003530 con 0x7f630815d300 2023-11-28T18:30:10.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.049+0000 7f630cfa3640 1 -- 172.21.15.105:0/1389607107 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3268372141 0 0) 0x7f62f4003530 con 0x7f630815d300 2023-11-28T18:30:10.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.049+0000 7f630cfa3640 1 -- 172.21.15.105:0/1389607107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63080d1650 msgr2=0x7f63080d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:10.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.049+0000 7f630cfa3640 1 --2- 172.21.15.105:0/1389607107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63080d1650 0x7f63080d1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.049+0000 7f630cfa3640 1 -- 172.21.15.105:0/1389607107 >> v2:172.21.15.148:3300/0 conn(0x7f63080c7510 msgr2=0x7f63080c79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:10.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.049+0000 7f630cfa3640 1 --2- 172.21.15.105:0/1389607107 >> v2:172.21.15.148:3300/0 conn(0x7f63080c7510 0x7f63080c79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.049+0000 7f630cfa3640 1 -- 172.21.15.105:0/1389607107 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63080d0eb0 con 0x7f630815d300 2023-11-28T18:30:10.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.049+0000 7f630cfa3640 1 -- 172.21.15.105:0/1389607107 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f6304001680 con 0x7f630815d300 2023-11-28T18:30:10.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.049+0000 7f630cfa3640 1 -- 172.21.15.105:0/1389607107 >> v1:172.21.15.105:6789/0 conn(0x7f630815d300 legacy=0x7f63080d37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:10.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.053+0000 7f630cfa3640 1 --2- 172.21.15.105:0/1389607107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63080d1650 0x7f62f4003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:10.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.053+0000 7f630cfa3640 1 --2- 172.21.15.105:0/1389607107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62f4003f00 0x7f62f40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:10.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.053+0000 7f630cfa3640 1 --2- 172.21.15.105:0/1389607107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62f4004d10 0x7f62f4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:10.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.053+0000 7f630cfa3640 1 -- 172.21.15.105:0/1389607107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63080d1470 con 0x7f62f4004d10 2023-11-28T18:30:10.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.053+0000 7f630cfa3640 1 -- 172.21.15.105:0/1389607107 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63080cefa0 con 0x7f62f4003f00 2023-11-28T18:30:10.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.053+0000 7f630cfa3640 1 -- 172.21.15.105:0/1389607107 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62f400b9e0 con 0x7f63080d1650 2023-11-28T18:30:10.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.053+0000 7f630e7a6640 1 --2- 172.21.15.105:0/1389607107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62f4004d10 0x7f62f4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:10.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.053+0000 7f630dfa5640 1 --2- 172.21.15.105:0/1389607107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63080d1650 0x7f62f4003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:10.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.053+0000 7f630d7a4640 1 --2- 172.21.15.105:0/1389607107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62f4003f00 0x7f62f40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:10.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.053+0000 7f630e7a6640 1 -- 172.21.15.105:0/1389607107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63080d1650 msgr2=0x7f62f4003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:10.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.053+0000 7f630e7a6640 1 --2- 172.21.15.105:0/1389607107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63080d1650 0x7f62f4003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.053+0000 7f630e7a6640 1 -- 172.21.15.105:0/1389607107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62f4003f00 msgr2=0x7f62f40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:10.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.053+0000 7f630e7a6640 1 --2- 172.21.15.105:0/1389607107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62f4003f00 0x7f62f40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.053+0000 7f630e7a6640 1 -- 172.21.15.105:0/1389607107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f4003530 con 0x7f62f4004d10 2023-11-28T18:30:10.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.053+0000 7f630e7a6640 1 --2- 172.21.15.105:0/1389607107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62f4004d10 0x7f62f4005190 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f630000e170 tx=0x7f630001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44cb91d00e76e34a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:10.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.053+0000 7f630cfa3640 1 -- 172.21.15.105:0/1389607107 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63000a5c90 con 0x7f62f4004d10 2023-11-28T18:30:10.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.053+0000 7f630cfa3640 1 -- 172.21.15.105:0/1389607107 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630000f040 con 0x7f62f4004d10 2023-11-28T18:30:10.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.053+0000 7f630cfa3640 1 -- 172.21.15.105:0/1389607107 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63000ad440 con 0x7f62f4004d10 2023-11-28T18:30:10.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.057+0000 7f6310209640 1 -- 172.21.15.105:0/1389607107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62f4004d10 msgr2=0x7f62f4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:10.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.057+0000 7f6310209640 1 --2- 172.21.15.105:0/1389607107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62f4004d10 0x7f62f4005190 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f630000e170 tx=0x7f630001cc80 comp rx=0 tx=0).stop 2023-11-28T18:30:10.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.057+0000 7f630e7a6640 1 -- 172.21.15.105:0/1389607107 reap_dead start 2023-11-28T18:30:10.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.057+0000 7f6310209640 1 -- 172.21.15.105:0/1389607107 shutdown_connections 2023-11-28T18:30:10.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.057+0000 7f6310209640 1 -- 172.21.15.105:0/1389607107 >> 172.21.15.105:0/1389607107 conn(0x7f630809c360 msgr2=0x7f63080d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:10.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.057+0000 7f6310209640 1 -- 172.21.15.105:0/1389607107 shutdown_connections 2023-11-28T18:30:10.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.057+0000 7f6310209640 1 -- 172.21.15.105:0/1389607107 wait complete. 2023-11-28T18:30:10.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.057+0000 7f6310209640 1 Processor -- start 2023-11-28T18:30:10.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.057+0000 7f6310209640 1 -- start start 2023-11-28T18:30:10.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.057+0000 7f6310209640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62f4004d10 0x7f630814a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:10.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.057+0000 7f6310209640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f630815a370 0x7f630815a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:10.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.057+0000 7f6310209640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63080cc230 0x7f63080cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:10.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.057+0000 7f6310209640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63081561c0 con 0x7f63080cc230 2023-11-28T18:30:10.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.057+0000 7f6310209640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63081592d0 con 0x7f630815a370 2023-11-28T18:30:10.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.057+0000 7f6310209640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63080ccbf0 con 0x7f62f4004d10 2023-11-28T18:30:10.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.057+0000 7f630e7a6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63080cc230 0x7f63080cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:10.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.057+0000 7f630e7a6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63080cc230 0x7f63080cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50188/0 (socket says 172.21.15.105:50188) 2023-11-28T18:30:10.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.057+0000 7f630e7a6640 1 -- 172.21.15.105:0/3312336865 learned_addr learned my addr 172.21.15.105:0/3312336865 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:10.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.057+0000 7f630dfa5640 1 --2- 172.21.15.105:0/3312336865 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62f4004d10 0x7f630814a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:10.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.057+0000 7f630d7a4640 1 --2- 172.21.15.105:0/3312336865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f630815a370 0x7f630815a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:10.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.061+0000 7f630e7a6640 1 -- 172.21.15.105:0/3312336865 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62f4004d10 msgr2=0x7f630814a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:10.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.061+0000 7f630e7a6640 1 --2- 172.21.15.105:0/3312336865 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62f4004d10 0x7f630814a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.061+0000 7f630e7a6640 1 -- 172.21.15.105:0/3312336865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f630815a370 msgr2=0x7f630815a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:10.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.061+0000 7f630e7a6640 1 --2- 172.21.15.105:0/3312336865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f630815a370 0x7f630815a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.061+0000 7f630e7a6640 1 -- 172.21.15.105:0/3312336865 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6308156cd0 con 0x7f63080cc230 2023-11-28T18:30:10.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.061+0000 7f630dfa5640 1 --2- 172.21.15.105:0/3312336865 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62f4004d10 0x7f630814a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:10.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.061+0000 7f630e7a6640 1 --2- 172.21.15.105:0/3312336865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63080cc230 0x7f63080cc6b0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f630000e170 tx=0x7f6300009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:10.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.061+0000 7f62fe7fc640 1 -- 172.21.15.105:0/3312336865 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63000ba070 con 0x7f63080cc230 2023-11-28T18:30:10.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.061+0000 7f62fe7fc640 1 -- 172.21.15.105:0/3312336865 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63000039a0 con 0x7f63080cc230 2023-11-28T18:30:10.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.061+0000 7f62fe7fc640 1 -- 172.21.15.105:0/3312336865 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300003b30 con 0x7f63080cc230 2023-11-28T18:30:10.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.061+0000 7f6310209640 1 -- 172.21.15.105:0/3312336865 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63081563a0 con 0x7f63080cc230 2023-11-28T18:30:10.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.061+0000 7f6310209640 1 -- 172.21.15.105:0/3312336865 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6308156960 con 0x7f63080cc230 2023-11-28T18:30:10.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.061+0000 7f6310209640 1 -- 172.21.15.105:0/3312336865 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6308067570 con 0x7f63080cc230 2023-11-28T18:30:10.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.061+0000 7f62fe7fc640 1 -- 172.21.15.105:0/3312336865 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f63000b4040 con 0x7f63080cc230 2023-11-28T18:30:10.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.061+0000 7f62fe7fc640 1 --2- 172.21.15.105:0/3312336865 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f62ec0668d0 0x7f62ec068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:10.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.061+0000 7f62fe7fc640 1 -- 172.21.15.105:0/3312336865 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f630011df40 con 0x7f63080cc230 2023-11-28T18:30:10.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.061+0000 7f630dfa5640 1 --2- 172.21.15.105:0/3312336865 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f62ec0668d0 0x7f62ec068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:10.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.065+0000 7f630dfa5640 1 --2- 172.21.15.105:0/3312336865 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f62ec0668d0 0x7f62ec068d90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f63040052a0 tx=0x7f6304009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:10.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.065+0000 7f62fe7fc640 1 -- 172.21.15.105:0/3312336865 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f63000eb020 con 0x7f63080cc230 2023-11-28T18:30:10.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.213+0000 7f6310209640 1 -- 172.21.15.105:0/3312336865 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f63080c6ff0 con 0x7f62ec0668d0 2023-11-28T18:30:10.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.221+0000 7f62fe7fc640 1 -- 172.21.15.105:0/3312336865 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289487 (secure 0 0 0) 0x7f6304014960 con 0x7f62ec0668d0 2023-11-28T18:30:10.223 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:10.225 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:30:10.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.225+0000 7f6310209640 1 -- 172.21.15.105:0/3312336865 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f62ec0668d0 msgr2=0x7f62ec068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:10.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.225+0000 7f6310209640 1 --2- 172.21.15.105:0/3312336865 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f62ec0668d0 0x7f62ec068d90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f63040052a0 tx=0x7f6304009040 comp rx=0 tx=0).stop 2023-11-28T18:30:10.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.225+0000 7f6310209640 1 -- 172.21.15.105:0/3312336865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63080cc230 msgr2=0x7f63080cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:10.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.225+0000 7f6310209640 1 --2- 172.21.15.105:0/3312336865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63080cc230 0x7f63080cc6b0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f630000e170 tx=0x7f6300009750 comp rx=0 tx=0).stop 2023-11-28T18:30:10.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.225+0000 7f6310209640 1 -- 172.21.15.105:0/3312336865 shutdown_connections 2023-11-28T18:30:10.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.225+0000 7f6310209640 1 --2- 172.21.15.105:0/3312336865 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f62ec0668d0 0x7f62ec068d90 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.225+0000 7f6310209640 1 --2- 172.21.15.105:0/3312336865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63080cc230 0x7f63080cc6b0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.225+0000 7f6310209640 1 --2- 172.21.15.105:0/3312336865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f630815a370 0x7f630815a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.225+0000 7f6310209640 1 --2- 172.21.15.105:0/3312336865 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62f4004d10 0x7f630814a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.225+0000 7f6310209640 1 -- 172.21.15.105:0/3312336865 >> 172.21.15.105:0/3312336865 conn(0x7f630809c360 msgr2=0x7f630806b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:10.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.225+0000 7f6310209640 1 -- 172.21.15.105:0/3312336865 shutdown_connections 2023-11-28T18:30:10.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.225+0000 7f6310209640 1 -- 172.21.15.105:0/3312336865 wait complete. 2023-11-28T18:30:10.243 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2023-11-28T18:30:09.354332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":365888,"kb_used_data":10952,"kb_used_omap":12,"kb_used_meta":354803,"kb_avail":754608832,"statfs":{"total":773094113280,"available":772719443968,"internally_reserved":0,"allocated":11214848,"data_stored":5809024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":363318834},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.214344"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.186798+0000","last_change":"2023-11-28T18:30:06.176504+0000","last_active":"2023-11-28T18:30:06.186798+0000","last_peered":"2023-11-28T18:30:06.186798+0000","last_clean":"2023-11-28T18:30:06.186798+0000","last_became_active":"2023-11-28T18:30:06.176036+0000","last_became_peered":"2023-11-28T18:30:06.176036+0000","last_unstale":"2023-11-28T18:30:06.186798+0000","last_undegraded":"2023-11-28T18:30:06.186798+0000","last_fullsized":"2023-11-28T18:30:06.186798+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:16:51.197942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.195995+0000","last_change":"2023-11-28T18:30:06.192559+0000","last_active":"2023-11-28T18:30:06.195995+0000","last_peered":"2023-11-28T18:30:06.195995+0000","last_clean":"2023-11-28T18:30:06.195995+0000","last_became_active":"2023-11-28T18:30:06.192301+0000","last_became_peered":"2023-11-28T18:30:06.192301+0000","last_unstale":"2023-11-28T18:30:06.195995+0000","last_undegraded":"2023-11-28T18:30:06.195995+0000","last_fullsized":"2023-11-28T18:30:06.195995+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:15:31.961878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.198998+0000","last_change":"2023-11-28T18:30:06.192473+0000","last_active":"2023-11-28T18:30:06.198998+0000","last_peered":"2023-11-28T18:30:06.198998+0000","last_clean":"2023-11-28T18:30:06.198998+0000","last_became_active":"2023-11-28T18:30:06.192177+0000","last_became_peered":"2023-11-28T18:30:06.192177+0000","last_unstale":"2023-11-28T18:30:06.198998+0000","last_undegraded":"2023-11-28T18:30:06.198998+0000","last_fullsized":"2023-11-28T18:30:06.198998+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:11.772188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.197187+0000","last_change":"2023-11-28T18:30:06.192719+0000","last_active":"2023-11-28T18:30:06.197187+0000","last_peered":"2023-11-28T18:30:06.197187+0000","last_clean":"2023-11-28T18:30:06.197187+0000","last_became_active":"2023-11-28T18:30:06.192375+0000","last_became_peered":"2023-11-28T18:30:06.192375+0000","last_unstale":"2023-11-28T18:30:06.197187+0000","last_undegraded":"2023-11-28T18:30:06.197187+0000","last_fullsized":"2023-11-28T18:30:06.197187+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:48:42.993573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.202475+0000","last_change":"2023-11-28T18:30:06.197410+0000","last_active":"2023-11-28T18:30:06.202475+0000","last_peered":"2023-11-28T18:30:06.202475+0000","last_clean":"2023-11-28T18:30:06.202475+0000","last_became_active":"2023-11-28T18:30:06.197152+0000","last_became_peered":"2023-11-28T18:30:06.197152+0000","last_unstale":"2023-11-28T18:30:06.202475+0000","last_undegraded":"2023-11-28T18:30:06.202475+0000","last_fullsized":"2023-11-28T18:30:06.202475+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:24:39.803805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.194557+0000","last_change":"2023-11-28T18:30:06.189834+0000","last_active":"2023-11-28T18:30:06.194557+0000","last_peered":"2023-11-28T18:30:06.194557+0000","last_clean":"2023-11-28T18:30:06.194557+0000","last_became_active":"2023-11-28T18:30:06.189483+0000","last_became_peered":"2023-11-28T18:30:06.189483+0000","last_unstale":"2023-11-28T18:30:06.194557+0000","last_undegraded":"2023-11-28T18:30:06.194557+0000","last_fullsized":"2023-11-28T18:30:06.194557+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:19.324050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.194522+0000","last_change":"2023-11-28T18:30:06.190513+0000","last_active":"2023-11-28T18:30:06.194522+0000","last_peered":"2023-11-28T18:30:06.194522+0000","last_clean":"2023-11-28T18:30:06.194522+0000","last_became_active":"2023-11-28T18:30:06.190244+0000","last_became_peered":"2023-11-28T18:30:06.190244+0000","last_unstale":"2023-11-28T18:30:06.194522+0000","last_undegraded":"2023-11-28T18:30:06.194522+0000","last_fullsized":"2023-11-28T18:30:06.194522+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:30.463710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.200758+0000","last_change":"2023-11-28T18:30:06.196229+0000","last_active":"2023-11-28T18:30:06.200758+0000","last_peered":"2023-11-28T18:30:06.200758+0000","last_clean":"2023-11-28T18:30:06.200758+0000","last_became_active":"2023-11-28T18:30:06.195463+0000","last_became_peered":"2023-11-28T18:30:06.195463+0000","last_unstale":"2023-11-28T18:30:06.200758+0000","last_undegraded":"2023-11-28T18:30:06.200758+0000","last_fullsized":"2023-11-28T18:30:06.200758+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:34:54.990441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.193177+0000","last_change":"2023-11-28T18:30:06.187187+0000","last_active":"2023-11-28T18:30:06.193177+0000","last_peered":"2023-11-28T18:30:06.193177+0000","last_clean":"2023-11-28T18:30:06.193177+0000","last_became_active":"2023-11-28T18:30:06.186849+0000","last_became_peered":"2023-11-28T18:30:06.186849+0000","last_unstale":"2023-11-28T18:30:06.193177+0000","last_undegraded":"2023-11-28T18:30:06.193177+0000","last_fullsized":"2023-11-28T18:30:06.193177+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:24:39.803805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.202018+0000","last_change":"2023-11-28T18:30:06.193867+0000","last_active":"2023-11-28T18:30:06.202018+0000","last_peered":"2023-11-28T18:30:06.202018+0000","last_clean":"2023-11-28T18:30:06.202018+0000","last_became_active":"2023-11-28T18:30:06.193585+0000","last_became_peered":"2023-11-28T18:30:06.193585+0000","last_unstale":"2023-11-28T18:30:06.202018+0000","last_undegraded":"2023-11-28T18:30:06.202018+0000","last_fullsized":"2023-11-28T18:30:06.202018+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:45.712257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.194502+0000","last_change":"2023-11-28T18:30:06.190803+0000","last_active":"2023-11-28T18:30:06.194502+0000","last_peered":"2023-11-28T18:30:06.194502+0000","last_clean":"2023-11-28T18:30:06.194502+0000","last_became_active":"2023-11-28T18:30:06.190511+0000","last_became_peered":"2023-11-28T18:30:06.190511+0000","last_unstale":"2023-11-28T18:30:06.194502+0000","last_undegraded":"2023-11-28T18:30:06.194502+0000","last_fullsized":"2023-11-28T18:30:06.194502+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:23:48.056361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.196787+0000","last_change":"2023-11-28T18:30:06.190247+0000","last_active":"2023-11-28T18:30:06.196787+0000","last_peered":"2023-11-28T18:30:06.196787+0000","last_clean":"2023-11-28T18:30:06.196787+0000","last_became_active":"2023-11-28T18:30:06.189861+0000","last_became_peered":"2023-11-28T18:30:06.189861+0000","last_unstale":"2023-11-28T18:30:06.196787+0000","last_undegraded":"2023-11-28T18:30:06.196787+0000","last_fullsized":"2023-11-28T18:30:06.196787+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:20:53.956031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.180979+0000","last_change":"2023-11-28T18:30:06.177229+0000","last_active":"2023-11-28T18:30:06.180979+0000","last_peered":"2023-11-28T18:30:06.180979+0000","last_clean":"2023-11-28T18:30:06.180979+0000","last_became_active":"2023-11-28T18:30:06.176971+0000","last_became_peered":"2023-11-28T18:30:06.176971+0000","last_unstale":"2023-11-28T18:30:06.180979+0000","last_undegraded":"2023-11-28T18:30:06.180979+0000","last_fullsized":"2023-11-28T18:30:06.180979+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:43:28.813459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.196018+0000","last_change":"2023-11-28T18:30:06.180198+0000","last_active":"2023-11-28T18:30:06.196018+0000","last_peered":"2023-11-28T18:30:06.196018+0000","last_clean":"2023-11-28T18:30:06.196018+0000","last_became_active":"2023-11-28T18:30:06.179931+0000","last_became_peered":"2023-11-28T18:30:06.179931+0000","last_unstale":"2023-11-28T18:30:06.196018+0000","last_undegraded":"2023-11-28T18:30:06.196018+0000","last_fullsized":"2023-11-28T18:30:06.196018+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:03:50.027926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.196011+0000","last_change":"2023-11-28T18:30:06.182089+0000","last_active":"2023-11-28T18:30:06.196011+0000","last_peered":"2023-11-28T18:30:06.196011+0000","last_clean":"2023-11-28T18:30:06.196011+0000","last_became_active":"2023-11-28T18:30:06.181761+0000","last_became_peered":"2023-11-28T18:30:06.181761+0000","last_unstale":"2023-11-28T18:30:06.196011+0000","last_undegraded":"2023-11-28T18:30:06.196011+0000","last_fullsized":"2023-11-28T18:30:06.196011+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:21:51.623438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.204332+0000","last_change":"2023-11-28T18:30:06.202313+0000","last_active":"2023-11-28T18:30:06.204332+0000","last_peered":"2023-11-28T18:30:06.204332+0000","last_clean":"2023-11-28T18:30:06.204332+0000","last_became_active":"2023-11-28T18:30:06.201852+0000","last_became_peered":"2023-11-28T18:30:06.201852+0000","last_unstale":"2023-11-28T18:30:06.204332+0000","last_undegraded":"2023-11-28T18:30:06.204332+0000","last_fullsized":"2023-11-28T18:30:06.204332+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:51:02.819791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.192015+0000","last_change":"2023-11-28T18:30:06.188582+0000","last_active":"2023-11-28T18:30:06.192015+0000","last_peered":"2023-11-28T18:30:06.192015+0000","last_clean":"2023-11-28T18:30:06.192015+0000","last_became_active":"2023-11-28T18:30:06.188316+0000","last_became_peered":"2023-11-28T18:30:06.188316+0000","last_unstale":"2023-11-28T18:30:06.192015+0000","last_undegraded":"2023-11-28T18:30:06.192015+0000","last_fullsized":"2023-11-28T18:30:06.192015+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:10.321867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.195977+0000","last_change":"2023-11-28T18:30:06.191486+0000","last_active":"2023-11-28T18:30:06.195977+0000","last_peered":"2023-11-28T18:30:06.195977+0000","last_clean":"2023-11-28T18:30:06.195977+0000","last_became_active":"2023-11-28T18:30:06.191194+0000","last_became_peered":"2023-11-28T18:30:06.191194+0000","last_unstale":"2023-11-28T18:30:06.195977+0000","last_undegraded":"2023-11-28T18:30:06.195977+0000","last_fullsized":"2023-11-28T18:30:06.195977+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:33:01.075269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.199034+0000","last_change":"2023-11-28T18:30:06.196246+0000","last_active":"2023-11-28T18:30:06.199034+0000","last_peered":"2023-11-28T18:30:06.199034+0000","last_clean":"2023-11-28T18:30:06.199034+0000","last_became_active":"2023-11-28T18:30:06.195769+0000","last_became_peered":"2023-11-28T18:30:06.195769+0000","last_unstale":"2023-11-28T18:30:06.199034+0000","last_undegraded":"2023-11-28T18:30:06.199034+0000","last_fullsized":"2023-11-28T18:30:06.199034+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:47:25.744265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.201222+0000","last_change":"2023-11-28T18:30:06.198305+0000","last_active":"2023-11-28T18:30:06.201222+0000","last_peered":"2023-11-28T18:30:06.201222+0000","last_clean":"2023-11-28T18:30:06.201222+0000","last_became_active":"2023-11-28T18:30:06.198044+0000","last_became_peered":"2023-11-28T18:30:06.198044+0000","last_unstale":"2023-11-28T18:30:06.201222+0000","last_undegraded":"2023-11-28T18:30:06.201222+0000","last_fullsized":"2023-11-28T18:30:06.201222+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:19.324050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.199787+0000","last_change":"2023-11-28T18:30:06.194442+0000","last_active":"2023-11-28T18:30:06.199787+0000","last_peered":"2023-11-28T18:30:06.199787+0000","last_clean":"2023-11-28T18:30:06.199787+0000","last_became_active":"2023-11-28T18:30:06.194159+0000","last_became_peered":"2023-11-28T18:30:06.194159+0000","last_unstale":"2023-11-28T18:30:06.199787+0000","last_undegraded":"2023-11-28T18:30:06.199787+0000","last_fullsized":"2023-11-28T18:30:06.199787+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:20:36.155372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.201871+0000","last_change":"2023-11-28T18:30:06.190286+0000","last_active":"2023-11-28T18:30:06.201871+0000","last_peered":"2023-11-28T18:30:06.201871+0000","last_clean":"2023-11-28T18:30:06.201871+0000","last_became_active":"2023-11-28T18:30:06.190021+0000","last_became_peered":"2023-11-28T18:30:06.190021+0000","last_unstale":"2023-11-28T18:30:06.201871+0000","last_undegraded":"2023-11-28T18:30:06.201871+0000","last_fullsized":"2023-11-28T18:30:06.201871+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:19.324050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.199420+0000","last_change":"2023-11-28T18:30:06.191890+0000","last_active":"2023-11-28T18:30:06.199420+0000","last_peered":"2023-11-28T18:30:06.199420+0000","last_clean":"2023-11-28T18:30:06.199420+0000","last_became_active":"2023-11-28T18:30:06.191565+0000","last_became_peered":"2023-11-28T18:30:06.191565+0000","last_unstale":"2023-11-28T18:30:06.199420+0000","last_undegraded":"2023-11-28T18:30:06.199420+0000","last_fullsized":"2023-11-28T18:30:06.199420+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:34:16.945951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.201905+0000","last_change":"2023-11-28T18:30:06.189574+0000","last_active":"2023-11-28T18:30:06.201905+0000","last_peered":"2023-11-28T18:30:06.201905+0000","last_clean":"2023-11-28T18:30:06.201905+0000","last_became_active":"2023-11-28T18:30:06.189266+0000","last_became_peered":"2023-11-28T18:30:06.189266+0000","last_unstale":"2023-11-28T18:30:06.201905+0000","last_undegraded":"2023-11-28T18:30:06.201905+0000","last_fullsized":"2023-11-28T18:30:06.201905+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:15:31.961878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.196579+0000","last_change":"2023-11-28T18:30:06.192657+0000","last_active":"2023-11-28T18:30:06.196579+0000","last_peered":"2023-11-28T18:30:06.196579+0000","last_clean":"2023-11-28T18:30:06.196579+0000","last_became_active":"2023-11-28T18:30:06.192327+0000","last_became_peered":"2023-11-28T18:30:06.192327+0000","last_unstale":"2023-11-28T18:30:06.196579+0000","last_undegraded":"2023-11-28T18:30:06.196579+0000","last_fullsized":"2023-11-28T18:30:06.196579+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:45.712257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.196581+0000","last_change":"2023-11-28T18:30:06.193105+0000","last_active":"2023-11-28T18:30:06.196581+0000","last_peered":"2023-11-28T18:30:06.196581+0000","last_clean":"2023-11-28T18:30:06.196581+0000","last_became_active":"2023-11-28T18:30:06.192815+0000","last_became_peered":"2023-11-28T18:30:06.192815+0000","last_unstale":"2023-11-28T18:30:06.196581+0000","last_undegraded":"2023-11-28T18:30:06.196581+0000","last_fullsized":"2023-11-28T18:30:06.196581+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:45:35.283335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.184372+0000","last_change":"2023-11-28T18:30:06.180332+0000","last_active":"2023-11-28T18:30:06.184372+0000","last_peered":"2023-11-28T18:30:06.184372+0000","last_clean":"2023-11-28T18:30:06.184372+0000","last_became_active":"2023-11-28T18:30:06.180033+0000","last_became_peered":"2023-11-28T18:30:06.180033+0000","last_unstale":"2023-11-28T18:30:06.184372+0000","last_undegraded":"2023-11-28T18:30:06.184372+0000","last_fullsized":"2023-11-28T18:30:06.184372+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:01:03.780112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.191751+0000","last_change":"2023-11-28T18:30:06.186408+0000","last_active":"2023-11-28T18:30:06.191751+0000","last_peered":"2023-11-28T18:30:06.191751+0000","last_clean":"2023-11-28T18:30:06.191751+0000","last_became_active":"2023-11-28T18:30:06.186127+0000","last_became_peered":"2023-11-28T18:30:06.186127+0000","last_unstale":"2023-11-28T18:30:06.191751+0000","last_undegraded":"2023-11-28T18:30:06.191751+0000","last_fullsized":"2023-11-28T18:30:06.191751+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:45.712257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.205155+0000","last_change":"2023-11-28T18:30:06.200264+0000","last_active":"2023-11-28T18:30:06.205155+0000","last_peered":"2023-11-28T18:30:06.205155+0000","last_clean":"2023-11-28T18:30:06.205155+0000","last_became_active":"2023-11-28T18:30:06.200020+0000","last_became_peered":"2023-11-28T18:30:06.200020+0000","last_unstale":"2023-11-28T18:30:06.205155+0000","last_undegraded":"2023-11-28T18:30:06.205155+0000","last_fullsized":"2023-11-28T18:30:06.205155+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:00:32.059242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.197048+0000","last_change":"2023-11-28T18:30:06.193909+0000","last_active":"2023-11-28T18:30:06.197048+0000","last_peered":"2023-11-28T18:30:06.197048+0000","last_clean":"2023-11-28T18:30:06.197048+0000","last_became_active":"2023-11-28T18:30:06.193651+0000","last_became_peered":"2023-11-28T18:30:06.193651+0000","last_unstale":"2023-11-28T18:30:06.197048+0000","last_undegraded":"2023-11-28T18:30:06.197048+0000","last_fullsized":"2023-11-28T18:30:06.197048+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:24:39.803805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.191655+0000","last_change":"2023-11-28T18:30:06.185088+0000","last_active":"2023-11-28T18:30:06.191655+0000","last_peered":"2023-11-28T18:30:06.191655+0000","last_clean":"2023-11-28T18:30:06.191655+0000","last_became_active":"2023-11-28T18:30:06.184796+0000","last_became_peered":"2023-11-28T18:30:06.184796+0000","last_unstale":"2023-11-28T18:30:06.191655+0000","last_undegraded":"2023-11-28T18:30:06.191655+0000","last_fullsized":"2023-11-28T18:30:06.191655+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:56.618131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.204512+0000","last_change":"2023-11-28T18:30:06.198428+0000","last_active":"2023-11-28T18:30:06.204512+0000","last_peered":"2023-11-28T18:30:06.204512+0000","last_clean":"2023-11-28T18:30:06.204512+0000","last_became_active":"2023-11-28T18:30:06.198146+0000","last_became_peered":"2023-11-28T18:30:06.198146+0000","last_unstale":"2023-11-28T18:30:06.204512+0000","last_undegraded":"2023-11-28T18:30:06.204512+0000","last_fullsized":"2023-11-28T18:30:06.204512+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:13.007653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.202330+0000","last_change":"2023-11-28T18:30:06.196978+0000","last_active":"2023-11-28T18:30:06.202330+0000","last_peered":"2023-11-28T18:30:06.202330+0000","last_clean":"2023-11-28T18:30:06.202330+0000","last_became_active":"2023-11-28T18:30:06.196715+0000","last_became_peered":"2023-11-28T18:30:06.196715+0000","last_unstale":"2023-11-28T18:30:06.202330+0000","last_undegraded":"2023-11-28T18:30:06.202330+0000","last_fullsized":"2023-11-28T18:30:06.202330+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:23.494330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"102'4","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.191463+0000","last_change":"2023-11-28T18:30:05.170363+0000","last_active":"2023-11-28T18:30:06.191463+0000","last_peered":"2023-11-28T18:30:06.191463+0000","last_clean":"2023-11-28T18:30:06.191463+0000","last_became_active":"2023-11-28T18:30:05.170040+0000","last_became_peered":"2023-11-28T18:30:05.170040+0000","last_unstale":"2023-11-28T18:30:06.191463+0000","last_undegraded":"2023-11-28T18:30:06.191463+0000","last_fullsized":"2023-11-28T18:30:06.191463+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:52:59.377449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.203189+0000","last_change":"2023-11-28T18:30:06.196183+0000","last_active":"2023-11-28T18:30:06.203189+0000","last_peered":"2023-11-28T18:30:06.203189+0000","last_clean":"2023-11-28T18:30:06.203189+0000","last_became_active":"2023-11-28T18:30:06.195890+0000","last_became_peered":"2023-11-28T18:30:06.195890+0000","last_unstale":"2023-11-28T18:30:06.203189+0000","last_undegraded":"2023-11-28T18:30:06.203189+0000","last_fullsized":"2023-11-28T18:30:06.203189+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:58.897105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.197893+0000","last_change":"2023-11-28T18:30:05.164703+0000","last_active":"2023-11-28T18:30:06.197893+0000","last_peered":"2023-11-28T18:30:06.197893+0000","last_clean":"2023-11-28T18:30:06.197893+0000","last_became_active":"2023-11-28T18:30:05.164446+0000","last_became_peered":"2023-11-28T18:30:05.164446+0000","last_unstale":"2023-11-28T18:30:06.197893+0000","last_undegraded":"2023-11-28T18:30:06.197893+0000","last_fullsized":"2023-11-28T18:30:06.197893+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:14.015784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.197030+0000","last_change":"2023-11-28T18:30:06.188269+0000","last_active":"2023-11-28T18:30:06.197030+0000","last_peered":"2023-11-28T18:30:06.197030+0000","last_clean":"2023-11-28T18:30:06.197030+0000","last_became_active":"2023-11-28T18:30:06.187981+0000","last_became_peered":"2023-11-28T18:30:06.187981+0000","last_unstale":"2023-11-28T18:30:06.197030+0000","last_undegraded":"2023-11-28T18:30:06.197030+0000","last_fullsized":"2023-11-28T18:30:06.197030+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:41:58.414983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.197601+0000","last_change":"2023-11-28T18:30:05.162404+0000","last_active":"2023-11-28T18:30:06.197601+0000","last_peered":"2023-11-28T18:30:06.197601+0000","last_clean":"2023-11-28T18:30:06.197601+0000","last_became_active":"2023-11-28T18:30:05.161794+0000","last_became_peered":"2023-11-28T18:30:05.161794+0000","last_unstale":"2023-11-28T18:30:06.197601+0000","last_undegraded":"2023-11-28T18:30:06.197601+0000","last_fullsized":"2023-11-28T18:30:06.197601+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:27:55.831255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.202643+0000","last_change":"2023-11-28T18:30:06.196752+0000","last_active":"2023-11-28T18:30:06.202643+0000","last_peered":"2023-11-28T18:30:06.202643+0000","last_clean":"2023-11-28T18:30:06.202643+0000","last_became_active":"2023-11-28T18:30:06.194764+0000","last_became_peered":"2023-11-28T18:30:06.194764+0000","last_unstale":"2023-11-28T18:30:06.202643+0000","last_undegraded":"2023-11-28T18:30:06.202643+0000","last_fullsized":"2023-11-28T18:30:06.202643+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:48:42.993573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.197716+0000","last_change":"2023-11-28T18:30:05.158127+0000","last_active":"2023-11-28T18:30:06.197716+0000","last_peered":"2023-11-28T18:30:06.197716+0000","last_clean":"2023-11-28T18:30:06.197716+0000","last_became_active":"2023-11-28T18:30:05.157850+0000","last_became_peered":"2023-11-28T18:30:05.157850+0000","last_unstale":"2023-11-28T18:30:06.197716+0000","last_undegraded":"2023-11-28T18:30:06.197716+0000","last_fullsized":"2023-11-28T18:30:06.197716+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:07.006261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.188387+0000","last_change":"2023-11-28T18:30:06.186064+0000","last_active":"2023-11-28T18:30:06.188387+0000","last_peered":"2023-11-28T18:30:06.188387+0000","last_clean":"2023-11-28T18:30:06.188387+0000","last_became_active":"2023-11-28T18:30:06.185779+0000","last_became_peered":"2023-11-28T18:30:06.185779+0000","last_unstale":"2023-11-28T18:30:06.188387+0000","last_undegraded":"2023-11-28T18:30:06.188387+0000","last_fullsized":"2023-11-28T18:30:06.188387+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:36:24.815145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.190187+0000","last_change":"2023-11-28T18:30:05.171666+0000","last_active":"2023-11-28T18:30:06.190187+0000","last_peered":"2023-11-28T18:30:06.190187+0000","last_clean":"2023-11-28T18:30:06.190187+0000","last_became_active":"2023-11-28T18:30:05.171426+0000","last_became_peered":"2023-11-28T18:30:05.171426+0000","last_unstale":"2023-11-28T18:30:06.190187+0000","last_undegraded":"2023-11-28T18:30:06.190187+0000","last_fullsized":"2023-11-28T18:30:06.190187+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:52:03.835518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.199017+0000","last_change":"2023-11-28T18:30:06.196037+0000","last_active":"2023-11-28T18:30:06.199017+0000","last_peered":"2023-11-28T18:30:06.199017+0000","last_clean":"2023-11-28T18:30:06.199017+0000","last_became_active":"2023-11-28T18:30:06.195416+0000","last_became_peered":"2023-11-28T18:30:06.195416+0000","last_unstale":"2023-11-28T18:30:06.199017+0000","last_undegraded":"2023-11-28T18:30:06.199017+0000","last_fullsized":"2023-11-28T18:30:06.199017+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:55:39.257881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.194553+0000","last_change":"2023-11-28T18:30:05.160156+0000","last_active":"2023-11-28T18:30:06.194553+0000","last_peered":"2023-11-28T18:30:06.194553+0000","last_clean":"2023-11-28T18:30:06.194553+0000","last_became_active":"2023-11-28T18:30:05.159877+0000","last_became_peered":"2023-11-28T18:30:05.159877+0000","last_unstale":"2023-11-28T18:30:06.194553+0000","last_undegraded":"2023-11-28T18:30:06.194553+0000","last_fullsized":"2023-11-28T18:30:06.194553+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:38:12.960970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.194582+0000","last_change":"2023-11-28T18:30:06.185704+0000","last_active":"2023-11-28T18:30:06.194582+0000","last_peered":"2023-11-28T18:30:06.194582+0000","last_clean":"2023-11-28T18:30:06.194582+0000","last_became_active":"2023-11-28T18:30:06.185405+0000","last_became_peered":"2023-11-28T18:30:06.185405+0000","last_unstale":"2023-11-28T18:30:06.194582+0000","last_undegraded":"2023-11-28T18:30:06.194582+0000","last_fullsized":"2023-11-28T18:30:06.194582+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:12:03.541522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.178522+0000","last_change":"2023-11-28T18:30:05.157935+0000","last_active":"2023-11-28T18:30:06.178522+0000","last_peered":"2023-11-28T18:30:06.178522+0000","last_clean":"2023-11-28T18:30:06.178522+0000","last_became_active":"2023-11-28T18:30:05.157274+0000","last_became_peered":"2023-11-28T18:30:05.157274+0000","last_unstale":"2023-11-28T18:30:06.178522+0000","last_undegraded":"2023-11-28T18:30:06.178522+0000","last_fullsized":"2023-11-28T18:30:06.178522+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:48:41.986238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.182639+0000","last_change":"2023-11-28T18:30:06.180445+0000","last_active":"2023-11-28T18:30:06.182639+0000","last_peered":"2023-11-28T18:30:06.182639+0000","last_clean":"2023-11-28T18:30:06.182639+0000","last_became_active":"2023-11-28T18:30:06.179442+0000","last_became_peered":"2023-11-28T18:30:06.179442+0000","last_unstale":"2023-11-28T18:30:06.182639+0000","last_undegraded":"2023-11-28T18:30:06.182639+0000","last_fullsized":"2023-11-28T18:30:06.182639+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:01:03.780112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.159406+0000","last_change":"2023-11-28T18:30:05.153674+0000","last_active":"2023-11-28T18:30:06.159406+0000","last_peered":"2023-11-28T18:30:06.159406+0000","last_clean":"2023-11-28T18:30:06.159406+0000","last_became_active":"2023-11-28T18:30:05.153327+0000","last_became_peered":"2023-11-28T18:30:05.153327+0000","last_unstale":"2023-11-28T18:30:06.159406+0000","last_undegraded":"2023-11-28T18:30:06.159406+0000","last_fullsized":"2023-11-28T18:30:06.159406+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:38:12.960970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.199111+0000","last_change":"2023-11-28T18:30:06.196453+0000","last_active":"2023-11-28T18:30:06.199111+0000","last_peered":"2023-11-28T18:30:06.199111+0000","last_clean":"2023-11-28T18:30:06.199111+0000","last_became_active":"2023-11-28T18:30:06.194959+0000","last_became_peered":"2023-11-28T18:30:06.194959+0000","last_unstale":"2023-11-28T18:30:06.199111+0000","last_undegraded":"2023-11-28T18:30:06.199111+0000","last_fullsized":"2023-11-28T18:30:06.199111+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:23:24.963867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.179527+0000","last_change":"2023-11-28T18:30:05.162132+0000","last_active":"2023-11-28T18:30:06.179527+0000","last_peered":"2023-11-28T18:30:06.179527+0000","last_clean":"2023-11-28T18:30:06.179527+0000","last_became_active":"2023-11-28T18:30:05.161844+0000","last_became_peered":"2023-11-28T18:30:05.161844+0000","last_unstale":"2023-11-28T18:30:06.179527+0000","last_undegraded":"2023-11-28T18:30:06.179527+0000","last_fullsized":"2023-11-28T18:30:06.179527+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:43:27.806124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.204480+0000","last_change":"2023-11-28T18:30:06.202617+0000","last_active":"2023-11-28T18:30:06.204480+0000","last_peered":"2023-11-28T18:30:06.204480+0000","last_clean":"2023-11-28T18:30:06.204480+0000","last_became_active":"2023-11-28T18:30:06.202320+0000","last_became_peered":"2023-11-28T18:30:06.202320+0000","last_unstale":"2023-11-28T18:30:06.204480+0000","last_undegraded":"2023-11-28T18:30:06.204480+0000","last_fullsized":"2023-11-28T18:30:06.204480+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:33:01.482047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.170921+0000","last_change":"2023-11-28T18:30:05.168611+0000","last_active":"2023-11-28T18:30:06.170921+0000","last_peered":"2023-11-28T18:30:06.170921+0000","last_clean":"2023-11-28T18:30:06.170921+0000","last_became_active":"2023-11-28T18:30:05.168350+0000","last_became_peered":"2023-11-28T18:30:05.168350+0000","last_unstale":"2023-11-28T18:30:06.170921+0000","last_undegraded":"2023-11-28T18:30:06.170921+0000","last_fullsized":"2023-11-28T18:30:06.170921+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:21:09.627822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.200357+0000","last_change":"2023-11-28T18:30:06.195887+0000","last_active":"2023-11-28T18:30:06.200357+0000","last_peered":"2023-11-28T18:30:06.200357+0000","last_clean":"2023-11-28T18:30:06.200357+0000","last_became_active":"2023-11-28T18:30:06.193400+0000","last_became_peered":"2023-11-28T18:30:06.193400+0000","last_unstale":"2023-11-28T18:30:06.200357+0000","last_undegraded":"2023-11-28T18:30:06.200357+0000","last_fullsized":"2023-11-28T18:30:06.200357+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:25:00.493596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"102'3","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.190509+0000","last_change":"2023-11-28T18:30:05.171478+0000","last_active":"2023-11-28T18:30:06.190509+0000","last_peered":"2023-11-28T18:30:06.190509+0000","last_clean":"2023-11-28T18:30:06.190509+0000","last_became_active":"2023-11-28T18:30:05.170871+0000","last_became_peered":"2023-11-28T18:30:05.170871+0000","last_unstale":"2023-11-28T18:30:06.190509+0000","last_undegraded":"2023-11-28T18:30:06.190509+0000","last_fullsized":"2023-11-28T18:30:06.190509+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:47:29.877689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.203160+0000","last_change":"2023-11-28T18:30:06.200566+0000","last_active":"2023-11-28T18:30:06.203160+0000","last_peered":"2023-11-28T18:30:06.203160+0000","last_clean":"2023-11-28T18:30:06.203160+0000","last_became_active":"2023-11-28T18:30:06.200255+0000","last_became_peered":"2023-11-28T18:30:06.200255+0000","last_unstale":"2023-11-28T18:30:06.203160+0000","last_undegraded":"2023-11-28T18:30:06.203160+0000","last_fullsized":"2023-11-28T18:30:06.203160+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:27:56.838590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"102'6","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.204576+0000","last_change":"2023-11-28T18:30:05.173650+0000","last_active":"2023-11-28T18:30:06.204576+0000","last_peered":"2023-11-28T18:30:06.204576+0000","last_clean":"2023-11-28T18:30:06.204576+0000","last_became_active":"2023-11-28T18:30:05.173175+0000","last_became_peered":"2023-11-28T18:30:05.173175+0000","last_unstale":"2023-11-28T18:30:06.204576+0000","last_undegraded":"2023-11-28T18:30:06.204576+0000","last_fullsized":"2023-11-28T18:30:06.204576+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:41:57.407648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.184517+0000","last_change":"2023-11-28T18:30:06.179449+0000","last_active":"2023-11-28T18:30:06.184517+0000","last_peered":"2023-11-28T18:30:06.184517+0000","last_clean":"2023-11-28T18:30:06.184517+0000","last_became_active":"2023-11-28T18:30:06.179135+0000","last_became_peered":"2023-11-28T18:30:06.179135+0000","last_unstale":"2023-11-28T18:30:06.184517+0000","last_undegraded":"2023-11-28T18:30:06.184517+0000","last_fullsized":"2023-11-28T18:30:06.184517+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:36:24.815145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.199561+0000","last_change":"2023-11-28T18:30:05.167772+0000","last_active":"2023-11-28T18:30:06.199561+0000","last_peered":"2023-11-28T18:30:06.199561+0000","last_clean":"2023-11-28T18:30:06.199561+0000","last_became_active":"2023-11-28T18:30:05.167500+0000","last_became_peered":"2023-11-28T18:30:05.167500+0000","last_unstale":"2023-11-28T18:30:06.199561+0000","last_undegraded":"2023-11-28T18:30:06.199561+0000","last_fullsized":"2023-11-28T18:30:06.199561+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:20:52.948696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.194013+0000","last_change":"2023-11-28T18:30:06.188105+0000","last_active":"2023-11-28T18:30:06.194013+0000","last_peered":"2023-11-28T18:30:06.194013+0000","last_clean":"2023-11-28T18:30:06.194013+0000","last_became_active":"2023-11-28T18:30:06.187701+0000","last_became_peered":"2023-11-28T18:30:06.187701+0000","last_unstale":"2023-11-28T18:30:06.194013+0000","last_undegraded":"2023-11-28T18:30:06.194013+0000","last_fullsized":"2023-11-28T18:30:06.194013+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:08.013596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.158851+0000","last_change":"2023-11-28T18:30:05.157249+0000","last_active":"2023-11-28T18:30:06.158851+0000","last_peered":"2023-11-28T18:30:06.158851+0000","last_clean":"2023-11-28T18:30:06.158851+0000","last_became_active":"2023-11-28T18:30:05.156416+0000","last_became_peered":"2023-11-28T18:30:05.156416+0000","last_unstale":"2023-11-28T18:30:06.158851+0000","last_undegraded":"2023-11-28T18:30:06.158851+0000","last_fullsized":"2023-11-28T18:30:06.158851+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:45:34.276000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.204419+0000","last_change":"2023-11-28T18:30:06.197709+0000","last_active":"2023-11-28T18:30:06.204419+0000","last_peered":"2023-11-28T18:30:06.204419+0000","last_clean":"2023-11-28T18:30:06.204419+0000","last_became_active":"2023-11-28T18:30:06.196045+0000","last_became_peered":"2023-11-28T18:30:06.196045+0000","last_unstale":"2023-11-28T18:30:06.204419+0000","last_undegraded":"2023-11-28T18:30:06.204419+0000","last_fullsized":"2023-11-28T18:30:06.204419+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:41:55.422124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.182280+0000","last_change":"2023-11-28T18:30:05.168287+0000","last_active":"2023-11-28T18:30:06.182280+0000","last_peered":"2023-11-28T18:30:06.182280+0000","last_clean":"2023-11-28T18:30:06.182280+0000","last_became_active":"2023-11-28T18:30:05.168010+0000","last_became_peered":"2023-11-28T18:30:05.168010+0000","last_unstale":"2023-11-28T18:30:06.182280+0000","last_undegraded":"2023-11-28T18:30:06.182280+0000","last_fullsized":"2023-11-28T18:30:06.182280+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:48:00.884187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.192034+0000","last_change":"2023-11-28T18:30:06.189209+0000","last_active":"2023-11-28T18:30:06.192034+0000","last_peered":"2023-11-28T18:30:06.192034+0000","last_clean":"2023-11-28T18:30:06.192034+0000","last_became_active":"2023-11-28T18:30:06.188073+0000","last_became_peered":"2023-11-28T18:30:06.188073+0000","last_unstale":"2023-11-28T18:30:06.192034+0000","last_undegraded":"2023-11-28T18:30:06.192034+0000","last_fullsized":"2023-11-28T18:30:06.192034+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:27:13.812700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"102'8","reported_seq":19,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.199192+0000","last_change":"2023-11-28T18:30:05.158153+0000","last_active":"2023-11-28T18:30:06.199192+0000","last_peered":"2023-11-28T18:30:06.199192+0000","last_clean":"2023-11-28T18:30:06.199192+0000","last_became_active":"2023-11-28T18:30:05.157810+0000","last_became_peered":"2023-11-28T18:30:05.157810+0000","last_unstale":"2023-11-28T18:30:06.199192+0000","last_undegraded":"2023-11-28T18:30:06.199192+0000","last_fullsized":"2023-11-28T18:30:06.199192+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:33:00.474713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.204561+0000","last_change":"2023-11-28T18:30:06.199260+0000","last_active":"2023-11-28T18:30:06.204561+0000","last_peered":"2023-11-28T18:30:06.204561+0000","last_clean":"2023-11-28T18:30:06.204561+0000","last_became_active":"2023-11-28T18:30:06.195752+0000","last_became_peered":"2023-11-28T18:30:06.195752+0000","last_unstale":"2023-11-28T18:30:06.204561+0000","last_undegraded":"2023-11-28T18:30:06.204561+0000","last_fullsized":"2023-11-28T18:30:06.204561+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:07:32.818669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.198463+0000","last_change":"2023-11-28T18:30:05.169718+0000","last_active":"2023-11-28T18:30:06.198463+0000","last_peered":"2023-11-28T18:30:06.198463+0000","last_clean":"2023-11-28T18:30:06.198463+0000","last_became_active":"2023-11-28T18:30:05.169321+0000","last_became_peered":"2023-11-28T18:30:05.169321+0000","last_unstale":"2023-11-28T18:30:06.198463+0000","last_undegraded":"2023-11-28T18:30:06.198463+0000","last_fullsized":"2023-11-28T18:30:06.198463+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:20:52.948696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.193163+0000","last_change":"2023-11-28T18:30:06.187217+0000","last_active":"2023-11-28T18:30:06.193163+0000","last_peered":"2023-11-28T18:30:06.193163+0000","last_clean":"2023-11-28T18:30:06.193163+0000","last_became_active":"2023-11-28T18:30:06.186852+0000","last_became_peered":"2023-11-28T18:30:06.186852+0000","last_unstale":"2023-11-28T18:30:06.193163+0000","last_undegraded":"2023-11-28T18:30:06.193163+0000","last_fullsized":"2023-11-28T18:30:06.193163+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:27:56.838590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.182178+0000","last_change":"2023-11-28T18:30:05.155867+0000","last_active":"2023-11-28T18:30:06.182178+0000","last_peered":"2023-11-28T18:30:06.182178+0000","last_clean":"2023-11-28T18:30:06.182178+0000","last_became_active":"2023-11-28T18:30:05.155572+0000","last_became_peered":"2023-11-28T18:30:05.155572+0000","last_unstale":"2023-11-28T18:30:06.182178+0000","last_undegraded":"2023-11-28T18:30:06.182178+0000","last_fullsized":"2023-11-28T18:30:06.182178+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:49:07.897264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.202493+0000","last_change":"2023-11-28T18:30:06.197574+0000","last_active":"2023-11-28T18:30:06.202493+0000","last_peered":"2023-11-28T18:30:06.202493+0000","last_clean":"2023-11-28T18:30:06.202493+0000","last_became_active":"2023-11-28T18:30:06.197315+0000","last_became_peered":"2023-11-28T18:30:06.197315+0000","last_unstale":"2023-11-28T18:30:06.202493+0000","last_undegraded":"2023-11-28T18:30:06.202493+0000","last_fullsized":"2023-11-28T18:30:06.202493+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:41:58.414983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.170845+0000","last_change":"2023-11-28T18:30:05.170900+0000","last_active":"2023-11-28T18:30:06.170845+0000","last_peered":"2023-11-28T18:30:06.170845+0000","last_clean":"2023-11-28T18:30:06.170845+0000","last_became_active":"2023-11-28T18:30:05.170594+0000","last_became_peered":"2023-11-28T18:30:05.170594+0000","last_unstale":"2023-11-28T18:30:06.170845+0000","last_undegraded":"2023-11-28T18:30:06.170845+0000","last_fullsized":"2023-11-28T18:30:06.170845+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:52:59.377449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.179586+0000","last_change":"2023-11-28T18:30:06.172634+0000","last_active":"2023-11-28T18:30:06.179586+0000","last_peered":"2023-11-28T18:30:06.179586+0000","last_clean":"2023-11-28T18:30:06.179586+0000","last_became_active":"2023-11-28T18:30:06.172347+0000","last_became_peered":"2023-11-28T18:30:06.172347+0000","last_unstale":"2023-11-28T18:30:06.179586+0000","last_undegraded":"2023-11-28T18:30:06.179586+0000","last_fullsized":"2023-11-28T18:30:06.179586+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:00:14.663059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.199705+0000","last_change":"2023-11-28T18:30:05.175506+0000","last_active":"2023-11-28T18:30:06.199705+0000","last_peered":"2023-11-28T18:30:06.199705+0000","last_clean":"2023-11-28T18:30:06.199705+0000","last_became_active":"2023-11-28T18:30:05.175079+0000","last_became_peered":"2023-11-28T18:30:05.175079+0000","last_unstale":"2023-11-28T18:30:06.199705+0000","last_undegraded":"2023-11-28T18:30:06.199705+0000","last_fullsized":"2023-11-28T18:30:06.199705+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:56:30.582825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.189225+0000","last_change":"2023-11-28T18:30:06.186382+0000","last_active":"2023-11-28T18:30:06.189225+0000","last_peered":"2023-11-28T18:30:06.189225+0000","last_clean":"2023-11-28T18:30:06.189225+0000","last_became_active":"2023-11-28T18:30:06.186094+0000","last_became_peered":"2023-11-28T18:30:06.186094+0000","last_unstale":"2023-11-28T18:30:06.189225+0000","last_undegraded":"2023-11-28T18:30:06.189225+0000","last_fullsized":"2023-11-28T18:30:06.189225+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:06:08.004603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.184888+0000","last_change":"2023-11-28T18:30:05.168522+0000","last_active":"2023-11-28T18:30:06.184888+0000","last_peered":"2023-11-28T18:30:06.184888+0000","last_clean":"2023-11-28T18:30:06.184888+0000","last_became_active":"2023-11-28T18:30:05.167218+0000","last_became_peered":"2023-11-28T18:30:05.167218+0000","last_unstale":"2023-11-28T18:30:06.184888+0000","last_undegraded":"2023-11-28T18:30:06.184888+0000","last_fullsized":"2023-11-28T18:30:06.184888+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:31:37.741493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.197238+0000","last_change":"2023-11-28T18:30:06.191690+0000","last_active":"2023-11-28T18:30:06.197238+0000","last_peered":"2023-11-28T18:30:06.197238+0000","last_clean":"2023-11-28T18:30:06.197238+0000","last_became_active":"2023-11-28T18:30:06.191057+0000","last_became_peered":"2023-11-28T18:30:06.191057+0000","last_unstale":"2023-11-28T18:30:06.197238+0000","last_undegraded":"2023-11-28T18:30:06.197238+0000","last_fullsized":"2023-11-28T18:30:06.197238+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:23:48.056361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.166727+0000","last_change":"2023-11-28T18:30:05.168933+0000","last_active":"2023-11-28T18:30:06.166727+0000","last_peered":"2023-11-28T18:30:06.166727+0000","last_clean":"2023-11-28T18:30:06.166727+0000","last_became_active":"2023-11-28T18:30:05.168436+0000","last_became_peered":"2023-11-28T18:30:05.168436+0000","last_unstale":"2023-11-28T18:30:06.166727+0000","last_undegraded":"2023-11-28T18:30:06.166727+0000","last_fullsized":"2023-11-28T18:30:06.166727+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:37:47.950433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.204380+0000","last_change":"2023-11-28T18:30:06.197151+0000","last_active":"2023-11-28T18:30:06.204380+0000","last_peered":"2023-11-28T18:30:06.204380+0000","last_clean":"2023-11-28T18:30:06.204380+0000","last_became_active":"2023-11-28T18:30:06.193062+0000","last_became_peered":"2023-11-28T18:30:06.193062+0000","last_unstale":"2023-11-28T18:30:06.204380+0000","last_undegraded":"2023-11-28T18:30:06.204380+0000","last_fullsized":"2023-11-28T18:30:06.204380+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:17:01.857749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.181967+0000","last_change":"2023-11-28T18:30:05.161460+0000","last_active":"2023-11-28T18:30:06.181967+0000","last_peered":"2023-11-28T18:30:06.181967+0000","last_clean":"2023-11-28T18:30:06.181967+0000","last_became_active":"2023-11-28T18:30:05.161023+0000","last_became_peered":"2023-11-28T18:30:05.161023+0000","last_unstale":"2023-11-28T18:30:06.181967+0000","last_undegraded":"2023-11-28T18:30:06.181967+0000","last_fullsized":"2023-11-28T18:30:06.181967+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:56.469337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.182516+0000","last_change":"2023-11-28T18:30:06.175722+0000","last_active":"2023-11-28T18:30:06.182516+0000","last_peered":"2023-11-28T18:30:06.182516+0000","last_clean":"2023-11-28T18:30:06.182516+0000","last_became_active":"2023-11-28T18:30:06.175438+0000","last_became_peered":"2023-11-28T18:30:06.175438+0000","last_unstale":"2023-11-28T18:30:06.182516+0000","last_undegraded":"2023-11-28T18:30:06.182516+0000","last_fullsized":"2023-11-28T18:30:06.182516+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:50:09.060450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.202633+0000","last_change":"2023-11-28T18:30:05.164247+0000","last_active":"2023-11-28T18:30:06.202633+0000","last_peered":"2023-11-28T18:30:06.202633+0000","last_clean":"2023-11-28T18:30:06.202633+0000","last_became_active":"2023-11-28T18:30:05.163970+0000","last_became_peered":"2023-11-28T18:30:05.163970+0000","last_unstale":"2023-11-28T18:30:06.202633+0000","last_undegraded":"2023-11-28T18:30:06.202633+0000","last_fullsized":"2023-11-28T18:30:06.202633+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:45:34.276000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.196039+0000","last_change":"2023-11-28T18:30:06.190076+0000","last_active":"2023-11-28T18:30:06.196039+0000","last_peered":"2023-11-28T18:30:06.196039+0000","last_clean":"2023-11-28T18:30:06.196039+0000","last_became_active":"2023-11-28T18:30:06.189787+0000","last_became_peered":"2023-11-28T18:30:06.189787+0000","last_unstale":"2023-11-28T18:30:06.196039+0000","last_undegraded":"2023-11-28T18:30:06.196039+0000","last_fullsized":"2023-11-28T18:30:06.196039+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:08.013596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.171188+0000","last_change":"2023-11-28T18:30:05.172663+0000","last_active":"2023-11-28T18:30:06.171188+0000","last_peered":"2023-11-28T18:30:06.171188+0000","last_clean":"2023-11-28T18:30:06.171188+0000","last_became_active":"2023-11-28T18:30:05.172344+0000","last_became_peered":"2023-11-28T18:30:05.172344+0000","last_unstale":"2023-11-28T18:30:06.171188+0000","last_undegraded":"2023-11-28T18:30:06.171188+0000","last_fullsized":"2023-11-28T18:30:06.171188+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:40:33.167683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.197493+0000","last_change":"2023-11-28T18:30:06.192022+0000","last_active":"2023-11-28T18:30:06.197493+0000","last_peered":"2023-11-28T18:30:06.197493+0000","last_clean":"2023-11-28T18:30:06.197493+0000","last_became_active":"2023-11-28T18:30:06.191766+0000","last_became_peered":"2023-11-28T18:30:06.191766+0000","last_unstale":"2023-11-28T18:30:06.197493+0000","last_undegraded":"2023-11-28T18:30:06.197493+0000","last_fullsized":"2023-11-28T18:30:06.197493+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:34:38.117919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.172327+0000","last_change":"2023-11-28T18:30:05.166981+0000","last_active":"2023-11-28T18:30:06.172327+0000","last_peered":"2023-11-28T18:30:06.172327+0000","last_clean":"2023-11-28T18:30:06.172327+0000","last_became_active":"2023-11-28T18:30:05.166641+0000","last_became_peered":"2023-11-28T18:30:05.166641+0000","last_unstale":"2023-11-28T18:30:06.172327+0000","last_undegraded":"2023-11-28T18:30:06.172327+0000","last_fullsized":"2023-11-28T18:30:06.172327+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:28:56.010559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.194439+0000","last_change":"2023-11-28T18:30:06.189742+0000","last_active":"2023-11-28T18:30:06.194439+0000","last_peered":"2023-11-28T18:30:06.194439+0000","last_clean":"2023-11-28T18:30:06.194439+0000","last_became_active":"2023-11-28T18:30:06.189489+0000","last_became_peered":"2023-11-28T18:30:06.189489+0000","last_unstale":"2023-11-28T18:30:06.194439+0000","last_undegraded":"2023-11-28T18:30:06.194439+0000","last_fullsized":"2023-11-28T18:30:06.194439+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:43:28.813459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.181706+0000","last_change":"2023-11-28T18:30:05.170517+0000","last_active":"2023-11-28T18:30:06.181706+0000","last_peered":"2023-11-28T18:30:06.181706+0000","last_clean":"2023-11-28T18:30:06.181706+0000","last_became_active":"2023-11-28T18:30:05.170185+0000","last_became_peered":"2023-11-28T18:30:05.170185+0000","last_unstale":"2023-11-28T18:30:06.181706+0000","last_undegraded":"2023-11-28T18:30:06.181706+0000","last_fullsized":"2023-11-28T18:30:06.181706+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:28:56.010559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.204532+0000","last_change":"2023-11-28T18:30:06.198229+0000","last_active":"2023-11-28T18:30:06.204532+0000","last_peered":"2023-11-28T18:30:06.204532+0000","last_clean":"2023-11-28T18:30:06.204532+0000","last_became_active":"2023-11-28T18:30:06.196839+0000","last_became_peered":"2023-11-28T18:30:06.196839+0000","last_unstale":"2023-11-28T18:30:06.204532+0000","last_undegraded":"2023-11-28T18:30:06.204532+0000","last_fullsized":"2023-11-28T18:30:06.204532+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:10:44.334510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.178035+0000","last_change":"2023-11-28T18:30:05.167244+0000","last_active":"2023-11-28T18:30:06.178035+0000","last_peered":"2023-11-28T18:30:06.178035+0000","last_clean":"2023-11-28T18:30:06.178035+0000","last_became_active":"2023-11-28T18:30:05.165852+0000","last_became_peered":"2023-11-28T18:30:05.165852+0000","last_unstale":"2023-11-28T18:30:06.178035+0000","last_undegraded":"2023-11-28T18:30:06.178035+0000","last_fullsized":"2023-11-28T18:30:06.178035+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:28:56.010559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.193577+0000","last_change":"2023-11-28T18:30:06.190372+0000","last_active":"2023-11-28T18:30:06.193577+0000","last_peered":"2023-11-28T18:30:06.193577+0000","last_clean":"2023-11-28T18:30:06.193577+0000","last_became_active":"2023-11-28T18:30:06.190100+0000","last_became_peered":"2023-11-28T18:30:06.190100+0000","last_unstale":"2023-11-28T18:30:06.193577+0000","last_undegraded":"2023-11-28T18:30:06.193577+0000","last_fullsized":"2023-11-28T18:30:06.193577+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:05:36.218930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.185021+0000","last_change":"2023-11-28T18:30:05.168365+0000","last_active":"2023-11-28T18:30:06.185021+0000","last_peered":"2023-11-28T18:30:06.185021+0000","last_clean":"2023-11-28T18:30:06.185021+0000","last_became_active":"2023-11-28T18:30:05.168090+0000","last_became_peered":"2023-11-28T18:30:05.168090+0000","last_unstale":"2023-11-28T18:30:06.185021+0000","last_undegraded":"2023-11-28T18:30:06.185021+0000","last_fullsized":"2023-11-28T18:30:06.185021+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:12:45.518541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.196169+0000","last_change":"2023-11-28T18:30:06.181894+0000","last_active":"2023-11-28T18:30:06.196169+0000","last_peered":"2023-11-28T18:30:06.196169+0000","last_clean":"2023-11-28T18:30:06.196169+0000","last_became_active":"2023-11-28T18:30:06.181571+0000","last_became_peered":"2023-11-28T18:30:06.181571+0000","last_unstale":"2023-11-28T18:30:06.196169+0000","last_undegraded":"2023-11-28T18:30:06.196169+0000","last_fullsized":"2023-11-28T18:30:06.196169+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:36:22.366120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.166856+0000","last_change":"2023-11-28T18:30:05.168740+0000","last_active":"2023-11-28T18:30:06.166856+0000","last_peered":"2023-11-28T18:30:06.166856+0000","last_clean":"2023-11-28T18:30:06.166856+0000","last_became_active":"2023-11-28T18:30:05.168082+0000","last_became_peered":"2023-11-28T18:30:05.168082+0000","last_unstale":"2023-11-28T18:30:06.166856+0000","last_undegraded":"2023-11-28T18:30:06.166856+0000","last_fullsized":"2023-11-28T18:30:06.166856+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:33:00.474713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.188537+0000","last_change":"2023-11-28T18:30:06.181114+0000","last_active":"2023-11-28T18:30:06.188537+0000","last_peered":"2023-11-28T18:30:06.188537+0000","last_clean":"2023-11-28T18:30:06.188537+0000","last_became_active":"2023-11-28T18:30:06.180806+0000","last_became_peered":"2023-11-28T18:30:06.180806+0000","last_unstale":"2023-11-28T18:30:06.188537+0000","last_undegraded":"2023-11-28T18:30:06.188537+0000","last_fullsized":"2023-11-28T18:30:06.188537+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:34:38.117919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.182102+0000","last_change":"2023-11-28T18:30:05.161265+0000","last_active":"2023-11-28T18:30:06.182102+0000","last_peered":"2023-11-28T18:30:06.182102+0000","last_clean":"2023-11-28T18:30:06.182102+0000","last_became_active":"2023-11-28T18:30:05.160714+0000","last_became_peered":"2023-11-28T18:30:05.160714+0000","last_unstale":"2023-11-28T18:30:06.182102+0000","last_undegraded":"2023-11-28T18:30:06.182102+0000","last_fullsized":"2023-11-28T18:30:06.182102+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:12:35.016288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.181312+0000","last_change":"2023-11-28T18:30:06.178208+0000","last_active":"2023-11-28T18:30:06.181312+0000","last_peered":"2023-11-28T18:30:06.181312+0000","last_clean":"2023-11-28T18:30:06.181312+0000","last_became_active":"2023-11-28T18:30:06.175565+0000","last_became_peered":"2023-11-28T18:30:06.175565+0000","last_unstale":"2023-11-28T18:30:06.181312+0000","last_undegraded":"2023-11-28T18:30:06.181312+0000","last_fullsized":"2023-11-28T18:30:06.181312+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:45:35.283335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.195860+0000","last_change":"2023-11-28T18:30:05.164191+0000","last_active":"2023-11-28T18:30:06.195860+0000","last_peered":"2023-11-28T18:30:06.195860+0000","last_clean":"2023-11-28T18:30:06.195860+0000","last_became_active":"2023-11-28T18:30:05.163923+0000","last_became_peered":"2023-11-28T18:30:05.163923+0000","last_unstale":"2023-11-28T18:30:06.195860+0000","last_undegraded":"2023-11-28T18:30:06.195860+0000","last_fullsized":"2023-11-28T18:30:06.195860+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:55:27.920271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":236,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-28T18:30:05.148201+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:30:05.148201+0000","last_peered":"2023-11-28T18:30:05.148201+0000","last_clean":"2023-11-28T18:30:05.148201+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:30:05.148201+0000","last_undegraded":"2023-11-28T18:30:05.148201+0000","last_fullsized":"2023-11-28T18:30:05.148201+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574933,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46876,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324964,"statfs":{"total":96636764160,"available":96588763136,"internally_reserved":0,"allocated":1847296,"data_stored":1168904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.628,"15min":0.70799999999999996},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.91900000000000004,"5min":0.98099999999999998,"15min":2.427},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.63300000000000001,"15min":0.70999999999999996},"min":{"1min":0.36699999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":2.2509999999999999},"last":0.74199999999999999}]},{"osd":1,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65400000000000003,"15min":0.71199999999999997},"min":{"1min":0.433,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.94099999999999995,"5min":0.96099999999999997,"15min":2.3849999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66200000000000003,"15min":0.74299999999999999},"min":{"1min":0.46500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.94499999999999995,"5min":1.079,"15min":2.3130000000000002},"last":0.878}]},{"osd":2,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67900000000000005,"15min":0.76200000000000001},"min":{"1min":0.45900000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.93100000000000005,"5min":0.998,"15min":2.4590000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.67000000000000004,"15min":0.73599999999999999},"min":{"1min":0.42199999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.022,"5min":1.022,"15min":2.411},"last":0.82199999999999995}]},{"osd":3,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65900000000000003,"15min":0.749},"min":{"1min":0.504,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90700000000000003,"5min":0.92000000000000004,"15min":2.2730000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.71399999999999997,"15min":0.79200000000000004},"min":{"1min":0.52900000000000003,"5min":0.432,"15min":0.432},"max":{"1min":1.0289999999999999,"5min":1.0389999999999999,"15min":2.3650000000000002},"last":0.84499999999999997}]},{"osd":4,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.42099999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.502},{"interface":"front","average":{"1min":0.63,"5min":0.58299999999999996,"15min":0.63500000000000001},"min":{"1min":0.442,"5min":0.315,"15min":0.315},"max":{"1min":0.96299999999999997,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.60799999999999998}]},{"osd":5,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63400000000000001,"15min":0.67200000000000004},"min":{"1min":0.44,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.65000000000000002,"15min":0.69599999999999995},"min":{"1min":0.48099999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68700000000000006}]},{"osd":6,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.69499999999999995,"15min":0.73899999999999999},"min":{"1min":0.47799999999999998,"5min":0.375,"15min":0.375},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.63700000000000001,"15min":0.71899999999999997},"min":{"1min":0.41299999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.71799999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40728,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331112,"statfs":{"total":96636764160,"available":96595058688,"internally_reserved":0,"allocated":1253376,"data_stored":578536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.65900000000000003,"15min":0.64300000000000002},"min":{"1min":0.433,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70899999999999996,"15min":0.72899999999999998},"min":{"1min":0.51300000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.98799999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.624}]},{"osd":1,"last update":"Tue Nov 28 18:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64800000000000002,"15min":0.78300000000000003},"min":{"1min":0.36599999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84999999999999998,"5min":0.96899999999999997,"15min":4.2389999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.77600000000000002},"min":{"1min":0.46600000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.91200000000000003,"5min":0.94499999999999995,"15min":4.218},"last":0.56799999999999995}]},{"osd":2,"last update":"Tue Nov 28 18:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66300000000000003,"15min":0.70799999999999996},"min":{"1min":0.41199999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.67900000000000005,"15min":0.70099999999999996},"min":{"1min":0.54900000000000004,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92100000000000004,"5min":1.012,"15min":1.012},"last":0.754}]},{"osd":3,"last update":"Tue Nov 28 18:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.69299999999999995,"15min":0.72299999999999998},"min":{"1min":0.59899999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.67500000000000004,"15min":0.68500000000000005},"min":{"1min":0.47699999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.84099999999999997,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.83299999999999996}]},{"osd":4,"last update":"Tue Nov 28 18:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61399999999999999,"15min":0.65900000000000003},"min":{"1min":0.376,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.95599999999999996,"5min":1.02,"15min":1.02},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.624,"5min":0.61799999999999999,"15min":0.67700000000000005},"min":{"1min":0.41599999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.90400000000000003,"5min":1.038,"15min":1.038},"last":0.77000000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61599999999999999,"15min":0.67000000000000004},"min":{"1min":0.47699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81999999999999995,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67600000000000005,"15min":0.746},"min":{"1min":0.443,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":1.0449999999999999},"last":0.85299999999999998}]},{"osd":7,"last update":"Tue Nov 28 18:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68899999999999995,"15min":0.75900000000000001},"min":{"1min":0.52200000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92300000000000004,"5min":1.002,"15min":1.0269999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67900000000000005,"15min":0.73699999999999999},"min":{"1min":0.46200000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78800000000000003,"5min":1,"15min":1},"last":0.86799999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1253376,"data_stored":578536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.68700000000000006,"15min":0.70899999999999996},"min":{"1min":0.35299999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.88800000000000001,"5min":2.1299999999999999,"15min":2.8929999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69999999999999996,"15min":0.746},"min":{"1min":0.39200000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92100000000000004,"5min":0.97799999999999998,"15min":2.9140000000000001},"last":0.50600000000000001}]},{"osd":1,"last update":"Tue Nov 28 18:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.67700000000000005,"15min":0.73199999999999998},"min":{"1min":0.42799999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90100000000000002,"5min":0.93000000000000005,"15min":3.4809999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.70299999999999996,"15min":0.78700000000000003},"min":{"1min":0.54000000000000004,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93700000000000006,"5min":2.2109999999999999,"15min":3.4580000000000002},"last":0.67200000000000004}]},{"osd":2,"last update":"Tue Nov 28 18:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65300000000000002,"15min":0.74099999999999999},"min":{"1min":0.48699999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84299999999999997,"5min":0.93200000000000005,"15min":3.9510000000000001},"last":0.626},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67400000000000004,"15min":0.76000000000000001},"min":{"1min":0.56799999999999995,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95999999999999996,"5min":1.014,"15min":4.0149999999999997},"last":0.48699999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.70499999999999996,"15min":0.80900000000000005},"min":{"1min":0.52600000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84699999999999998,"5min":2.254,"15min":4.0869999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68000000000000005,"15min":0.80400000000000005},"min":{"1min":0.48799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84199999999999997,"5min":0.98599999999999999,"15min":3.9900000000000002},"last":0.68899999999999995}]},{"osd":4,"last update":"Tue Nov 28 18:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.35999999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.90000000000000002,"5min":1.696,"15min":1.696},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68000000000000005,"15min":0.70599999999999996},"min":{"1min":0.47899999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.84099999999999997,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.52700000000000002}]},{"osd":6,"last update":"Tue Nov 28 18:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66400000000000003,"15min":0.67600000000000005},"min":{"1min":0.33900000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.93700000000000006,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.69399999999999995,"15min":0.71399999999999997},"min":{"1min":0.46700000000000003,"5min":0.438,"15min":0.42399999999999999},"max":{"1min":0.89200000000000002,"5min":1.262,"15min":1.756},"last":0.54900000000000004}]},{"osd":7,"last update":"Tue Nov 28 18:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.72999999999999998,"15min":0.75},"min":{"1min":0.42999999999999999,"5min":0.42399999999999999,"15min":0.41699999999999998},"max":{"1min":0.93400000000000005,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.73399999999999999,"15min":0.752},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.96299999999999997,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.73599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72664,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299176,"statfs":{"total":96636764160,"available":96562356224,"internally_reserved":0,"allocated":1253376,"data_stored":578536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.628,"15min":0.68000000000000005},"min":{"1min":0.36599999999999999,"5min":0.32900000000000001,"15min":0.315},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":2.363},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64400000000000002,"15min":0.64700000000000002},"min":{"1min":0.52900000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92300000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.71299999999999997}]},{"osd":1,"last update":"Tue Nov 28 18:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64300000000000002,"15min":0.627},"min":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.98599999999999999,"5min":1.026,"15min":1.0629999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68200000000000005,"15min":0.66000000000000003},"min":{"1min":0.45000000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.64800000000000002}]},{"osd":2,"last update":"Tue Nov 28 18:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72099999999999997,"15min":0.70199999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.72199999999999998,"15min":0.72999999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.091,"5min":1.091,"15min":1.871},"last":0.73299999999999998}]},{"osd":3,"last update":"Tue Nov 28 18:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66400000000000003,"15min":0.65200000000000002},"min":{"1min":0.45700000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89600000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68300000000000005,"15min":0.68500000000000005},"min":{"1min":0.47799999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.748}]},{"osd":5,"last update":"Tue Nov 28 18:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60699999999999998,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.315,"15min":0.315},"max":{"1min":1.03,"5min":1.03,"15min":1.1799999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.59899999999999998,"15min":0.63300000000000001},"min":{"1min":0.39100000000000001,"5min":0.33400000000000002,"15min":0.32300000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.79},"last":0.56399999999999995}]},{"osd":6,"last update":"Tue Nov 28 18:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.624,"15min":0.64500000000000002},"min":{"1min":0.433,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.123},"last":0.629},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.69599999999999995,"15min":0.70899999999999996},"min":{"1min":0.57999999999999996,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.925},"last":0.76200000000000001}]},{"osd":7,"last update":"Tue Nov 28 18:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.68400000000000005,"15min":0.71399999999999997},"min":{"1min":0.58699999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.843},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.755,"15min":0.77700000000000002},"min":{"1min":0.54600000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.593},"last":0.88100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574932,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62936,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308904,"statfs":{"total":96636764160,"available":96572317696,"internally_reserved":0,"allocated":1253376,"data_stored":578536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.441,"15min":0.41899999999999998},"min":{"1min":0.255,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.622,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.48799999999999999,"15min":0.47399999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.65600000000000003,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.626}]},{"osd":1,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.45100000000000001,"15min":0.46100000000000002},"min":{"1min":0.29599999999999999,"5min":0.28799999999999998,"15min":0.27300000000000002},"max":{"1min":0.68999999999999995,"5min":0.73399999999999999,"15min":0.96099999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.54300000000000004,"15min":0.51500000000000001},"min":{"1min":0.29699999999999999,"5min":0.249,"15min":0.247},"max":{"1min":0.79500000000000004,"5min":4.5599999999999996,"15min":4.5599999999999996},"last":0.50900000000000001}]},{"osd":2,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.54200000000000004,"15min":0.52800000000000002},"min":{"1min":0.38200000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74299999999999999,"5min":2.48,"15min":2.48},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92800000000000005,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.58699999999999997}]},{"osd":4,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.88800000000000001,"15min":0.81299999999999994},"min":{"1min":0.56000000000000005,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97599999999999998,"5min":12.695,"15min":12.695},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.79500000000000004,"15min":0.78200000000000003},"min":{"1min":0.51600000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.875,"5min":6.3250000000000002,"15min":6.3250000000000002},"last":0.872}]},{"osd":5,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76400000000000001,"15min":0.81999999999999995},"min":{"1min":0.54700000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95099999999999996,"5min":1.9430000000000001,"15min":2.5019999999999998},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.79000000000000004,"15min":0.77900000000000003},"min":{"1min":0.52100000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97399999999999998,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.92800000000000005}]},{"osd":6,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.82299999999999995,"15min":0.81100000000000005},"min":{"1min":0.54000000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.91200000000000003,"5min":6.1790000000000003,"15min":6.1790000000000003},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78600000000000003,"15min":0.79900000000000004},"min":{"1min":0.504,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.022,"5min":1.984,"15min":1.984},"last":0.85599999999999998}]},{"osd":7,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.79800000000000004,"15min":0.81399999999999995},"min":{"1min":0.53100000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94299999999999995,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.83099999999999996,"15min":0.81899999999999995},"min":{"1min":0.55600000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97599999999999998,"5min":6.2249999999999996,"15min":6.2249999999999996},"last":1.002}]}]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1253376,"data_stored":578536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:29:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.45700000000000002,"15min":0.49299999999999999},"min":{"1min":0.27300000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.629,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.44900000000000001,"15min":0.495},"min":{"1min":0.26700000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63300000000000001,"5min":0.66800000000000004,"15min":2.0150000000000001},"last":0.29399999999999998}]},{"osd":1,"last update":"Tue Nov 28 18:29:29 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.48799999999999999,"15min":0.51900000000000002},"min":{"1min":0.38400000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67500000000000004,"5min":1.48,"15min":1.48},"last":0.38},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.57299999999999995,"15min":0.53600000000000003},"min":{"1min":0.36699999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69099999999999995,"5min":8.3539999999999992,"15min":8.3539999999999992},"last":0.436}]},{"osd":3,"last update":"Tue Nov 28 18:29:29 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.48999999999999999,"15min":0.503},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76900000000000002,"5min":3.1549999999999998,"15min":3.1549999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55000000000000004,"15min":0.56699999999999995},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.754,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":0.49399999999999999}]},{"osd":4,"last update":"Tue Nov 28 18:29:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.75800000000000001,"15min":0.77000000000000002},"min":{"1min":0.55100000000000005,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92900000000000005,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.74199999999999999,"15min":0.77300000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.032,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.58999999999999997}]},{"osd":5,"last update":"Tue Nov 28 18:29:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.73999999999999999,"15min":0.76400000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94099999999999995,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.74299999999999999,"15min":0.94899999999999995},"min":{"1min":0.496,"5min":0.495,"15min":0.495},"max":{"1min":0.95599999999999996,"5min":1.5449999999999999,"15min":7.4820000000000002},"last":0.58299999999999996}]},{"osd":6,"last update":"Tue Nov 28 18:29:29 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.84299999999999997,"15min":0.81999999999999995},"min":{"1min":0.54700000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.95399999999999996,"5min":12.019,"15min":12.019},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.80600000000000005,"15min":0.86499999999999999},"min":{"1min":0.48999999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0149999999999999,"5min":8.7400000000000002,"15min":8.7400000000000002},"last":0.65200000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:29:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.83099999999999996,"15min":0.84599999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.98799999999999999,"5min":8.327,"15min":8.327},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.81599999999999995,"15min":0.90500000000000003},"min":{"1min":0.441,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97399999999999998,"5min":8.609,"15min":8.609},"last":0.56999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40724,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331116,"statfs":{"total":96636764160,"available":96595062784,"internally_reserved":0,"allocated":1253376,"data_stored":578536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40699999999999997,"15min":0.42299999999999999},"min":{"1min":0.29299999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.60799999999999998,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.46200000000000002,"15min":0.45500000000000002},"min":{"1min":0.371,"5min":0.27300000000000002,"15min":0.26600000000000001},"max":{"1min":0.66000000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.51500000000000001}]},{"osd":2,"last update":"Tue Nov 28 18:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.441,"15min":0.44700000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.75700000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48899999999999999,"15min":0.48099999999999998},"min":{"1min":0.35099999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.71499999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.56100000000000005}]},{"osd":3,"last update":"Tue Nov 28 18:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.42599999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.505,"15min":0.49299999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32200000000000001},"max":{"1min":0.73899999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54800000000000004}]},{"osd":4,"last update":"Tue Nov 28 18:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71599999999999997,"15min":0.74299999999999999},"min":{"1min":0.44,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0409999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.749,"15min":0.76700000000000002},"min":{"1min":0.56200000000000006,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0629999999999999,"5min":1.0780000000000001,"15min":1.155},"last":0.80200000000000005}]},{"osd":5,"last update":"Tue Nov 28 18:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70199999999999996,"15min":0.73999999999999999},"min":{"1min":0.504,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.997,"5min":0.997,"15min":1.3799999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70199999999999996,"15min":0.74099999999999999},"min":{"1min":0.56100000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96999999999999997,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.76800000000000002}]},{"osd":6,"last update":"Tue Nov 28 18:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.74299999999999999,"15min":0.77400000000000002},"min":{"1min":0.54800000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0349999999999999,"5min":1.1279999999999999,"15min":1.2969999999999999},"last":0.625},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.76100000000000001,"15min":0.80100000000000005},"min":{"1min":0.53300000000000003,"5min":0.498,"15min":0.498},"max":{"1min":1.028,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.71899999999999997}]},{"osd":7,"last update":"Tue Nov 28 18:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.72899999999999998,"15min":0.749},"min":{"1min":0.59599999999999997,"5min":0.51900000000000002,"15min":0.47899999999999998},"max":{"1min":0.98999999999999999,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.75},{"interface":"front","average":{"1min":0.746,"5min":0.73199999999999998,"15min":0.77000000000000002},"min":{"1min":0.53100000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0069999999999999,"5min":1.0109999999999999,"15min":1.4239999999999999},"last":0.78500000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574932,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37148,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334692,"statfs":{"total":96636764160,"available":96598724608,"internally_reserved":0,"allocated":1847296,"data_stored":1168904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 18:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41699999999999998,"15min":0.439},"min":{"1min":0.28100000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.28999999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.45300000000000001}]},{"osd":2,"last update":"Tue Nov 28 18:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.48099999999999998,"15min":0.503},"min":{"1min":0.318,"5min":0.30599999999999999,"15min":0.30499999999999999},"max":{"1min":0.59299999999999997,"5min":0.629,"15min":0.92900000000000005},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.48499999999999999,"15min":0.51000000000000001},"min":{"1min":0.38300000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63500000000000001,"5min":0.77800000000000002,"15min":0.91200000000000003},"last":0.40200000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47299999999999998,"15min":0.48699999999999999},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60199999999999998,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52400000000000002,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78500000000000003,"5min":0.80000000000000004,"15min":1.087},"last":0.502}]},{"osd":4,"last update":"Tue Nov 28 18:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.51900000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.94399999999999995,"5min":1.131,"15min":1.131},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71599999999999997,"15min":0.71799999999999997},"min":{"1min":0.53200000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88200000000000001,"5min":1.244,"15min":1.244},"last":0.72399999999999998}]},{"osd":5,"last update":"Tue Nov 28 18:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72399999999999998,"15min":0.72699999999999998},"min":{"1min":0.48399999999999999,"5min":0.436,"15min":0.436},"max":{"1min":0.92100000000000004,"5min":1.121,"15min":1.121},"last":0.625},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.86799999999999999,"5min":1.075,"15min":1.075},"last":0.745}]},{"osd":6,"last update":"Tue Nov 28 18:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72599999999999998,"15min":0.73199999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.93799999999999994,"5min":1.1140000000000001,"15min":1.407},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95599999999999996,"5min":1.087,"15min":1.089},"last":0.78800000000000003}]},{"osd":7,"last update":"Tue Nov 28 18:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.75600000000000001,"15min":0.754},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97099999999999997,"5min":1.212,"15min":1.212},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.74399999999999999,"15min":0.747},"min":{"1min":0.54900000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.84199999999999997,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.69999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:30:10.251 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:30:10.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.425+0000 7fdf7e6aa640 1 Processor -- start 2023-11-28T18:30:10.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.425+0000 7fdf7e6aa640 1 -- start start 2023-11-28T18:30:10.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.425+0000 7fdf7e6aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf780d3c30 0x7fdf780d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:10.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.425+0000 7fdf7e6aa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdf780b2d20 0x7fdf780b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:10.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.425+0000 7fdf7e6aa640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf780b25b0 con 0x7fdf7815d4f0 2023-11-28T18:30:10.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.425+0000 7fdf7e6aa640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf780b2900 con 0x7fdf780b2d20 2023-11-28T18:30:10.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.425+0000 7fdf7e6aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf780d0a20 con 0x7fdf780d3c30 2023-11-28T18:30:10.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.425+0000 7fdf7cc47640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdf780b2d20 0x7fdf780b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:10.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.425+0000 7fdf777fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf780d3c30 0x7fdf780d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:10.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.425+0000 7fdf7cc47640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdf780b2d20 0x7fdf780b31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:53174/0 (socket says 172.21.15.105:53174) 2023-11-28T18:30:10.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.425+0000 7fdf7cc47640 1 -- 172.21.15.105:0/4188640527 learned_addr learned my addr 172.21.15.105:0/4188640527 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:10.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.425+0000 7fdf7cc47640 1 -- 172.21.15.105:0/4188640527 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf780d3c30 msgr2=0x7fdf780d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:10.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.425+0000 7fdf7cc47640 1 --2- 172.21.15.105:0/4188640527 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf780d3c30 0x7fdf780d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.425+0000 7fdf7cc47640 1 -- 172.21.15.105:0/4188640527 >> v1:172.21.15.148:6789/0 conn(0x7fdf7815d4f0 legacy=0x7fdf780d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:10.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.425+0000 7fdf7cc47640 1 -- 172.21.15.105:0/4188640527 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf780cea30 con 0x7fdf780b2d20 2023-11-28T18:30:10.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.425+0000 7fdf777fe640 1 --2- 172.21.15.105:0/4188640527 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf780d3c30 0x7fdf780d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:30:10.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.429+0000 7fdf7cc47640 1 --2- 172.21.15.105:0/4188640527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf780b2d20 0x7fdf780b31d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009e50 tx=0x7fdf7001c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b41e91cbb57a768 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:10.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.429+0000 7fdf76ffd640 1 -- 172.21.15.105:0/4188640527 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf700a5810 con 0x7fdf780b2d20 2023-11-28T18:30:10.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.429+0000 7fdf76ffd640 1 -- 172.21.15.105:0/4188640527 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf700a5970 con 0x7fdf780b2d20 2023-11-28T18:30:10.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.429+0000 7fdf76ffd640 1 -- 172.21.15.105:0/4188640527 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf700a5d00 con 0x7fdf780b2d20 2023-11-28T18:30:10.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.429+0000 7fdf7e6aa640 1 -- 172.21.15.105:0/4188640527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf780b2d20 msgr2=0x7fdf780b31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:10.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.429+0000 7fdf7e6aa640 1 --2- 172.21.15.105:0/4188640527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf780b2d20 0x7fdf780b31d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009e50 tx=0x7fdf7001c950 comp rx=0 tx=0).stop 2023-11-28T18:30:10.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.429+0000 7fdf7e6aa640 1 -- 172.21.15.105:0/4188640527 shutdown_connections 2023-11-28T18:30:10.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.429+0000 7fdf7e6aa640 1 --2- 172.21.15.105:0/4188640527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf780b2d20 0x7fdf780b31d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.429+0000 7fdf7e6aa640 1 --2- 172.21.15.105:0/4188640527 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf780d3c30 0x7fdf780d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.429+0000 7fdf7e6aa640 1 -- 172.21.15.105:0/4188640527 >> 172.21.15.105:0/4188640527 conn(0x7fdf780d3030 msgr2=0x7fdf780d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:10.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.429+0000 7fdf7e6aa640 1 -- 172.21.15.105:0/4188640527 shutdown_connections 2023-11-28T18:30:10.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.429+0000 7fdf7e6aa640 1 -- 172.21.15.105:0/4188640527 wait complete. 2023-11-28T18:30:10.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf7e6aa640 1 Processor -- start 2023-11-28T18:30:10.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf7e6aa640 1 -- start start 2023-11-28T18:30:10.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf7e6aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf780b2d20 0x7fdf78154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:10.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf7e6aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf780d3c30 0x7fdf781576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:10.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf7e6aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf7815d4f0 0x7fdf7815c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:10.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf7e6aa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf78156490 con 0x7fdf780b2d20 2023-11-28T18:30:10.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf7e6aa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf78156d40 con 0x7fdf7815d4f0 2023-11-28T18:30:10.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf7e6aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf780b8b30 con 0x7fdf780d3c30 2023-11-28T18:30:10.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf77fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf780b2d20 0x7fdf78154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:10.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf77fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf780b2d20 0x7fdf78154380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50194/0 (socket says 172.21.15.105:50194) 2023-11-28T18:30:10.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf77fff640 1 -- 172.21.15.105:0/3498098273 learned_addr learned my addr 172.21.15.105:0/3498098273 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:10.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf7cc47640 1 --2- 172.21.15.105:0/3498098273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf7815d4f0 0x7fdf7815c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:10.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf777fe640 1 --2- 172.21.15.105:0/3498098273 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf780d3c30 0x7fdf781576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:10.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf77fff640 1 -- 172.21.15.105:0/3498098273 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf780d3c30 msgr2=0x7fdf781576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:10.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf77fff640 1 --2- 172.21.15.105:0/3498098273 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf780d3c30 0x7fdf781576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf77fff640 1 -- 172.21.15.105:0/3498098273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf7815d4f0 msgr2=0x7fdf7815c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:10.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf77fff640 1 --2- 172.21.15.105:0/3498098273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf7815d4f0 0x7fdf7815c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf77fff640 1 -- 172.21.15.105:0/3498098273 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6c009420 con 0x7fdf780b2d20 2023-11-28T18:30:10.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf777fe640 1 --2- 172.21.15.105:0/3498098273 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf780d3c30 0x7fdf781576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:10.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf77fff640 1 --2- 172.21.15.105:0/3498098273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf780b2d20 0x7fdf78154380 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fdf6800c370 tx=0x7fdf6800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:10.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf74ff9640 1 -- 172.21.15.105:0/3498098273 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf680052d0 con 0x7fdf780b2d20 2023-11-28T18:30:10.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf74ff9640 1 -- 172.21.15.105:0/3498098273 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf68005430 con 0x7fdf780b2d20 2023-11-28T18:30:10.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf7e6aa640 1 -- 172.21.15.105:0/3498098273 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf70009b30 con 0x7fdf780b2d20 2023-11-28T18:30:10.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf74ff9640 1 -- 172.21.15.105:0/3498098273 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf68007260 con 0x7fdf780b2d20 2023-11-28T18:30:10.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.433+0000 7fdf7e6aa640 1 -- 172.21.15.105:0/3498098273 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf78157000 con 0x7fdf780b2d20 2023-11-28T18:30:10.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.437+0000 7fdf7e6aa640 1 -- 172.21.15.105:0/3498098273 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf78067570 con 0x7fdf780b2d20 2023-11-28T18:30:10.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.437+0000 7fdf74ff9640 1 -- 172.21.15.105:0/3498098273 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fdf680073c0 con 0x7fdf780b2d20 2023-11-28T18:30:10.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.437+0000 7fdf74ff9640 1 --2- 172.21.15.105:0/3498098273 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdf64066900 0x7fdf64068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:10.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.437+0000 7fdf74ff9640 1 -- 172.21.15.105:0/3498098273 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdf68083990 con 0x7fdf780b2d20 2023-11-28T18:30:10.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.437+0000 7fdf777fe640 1 --2- 172.21.15.105:0/3498098273 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdf64066900 0x7fdf64068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:10.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.437+0000 7fdf777fe640 1 --2- 172.21.15.105:0/3498098273 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdf64066900 0x7fdf64068dc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fdf781590a0 tx=0x7fdf6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:10.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.441+0000 7fdf74ff9640 1 -- 172.21.15.105:0/3498098273 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdf680509f0 con 0x7fdf780b2d20 2023-11-28T18:30:10.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.589+0000 7fdf7e6aa640 1 -- 172.21.15.105:0/3498098273 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdf780d1a60 con 0x7fdf64066900 2023-11-28T18:30:10.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.597+0000 7fdf74ff9640 1 -- 172.21.15.105:0/3498098273 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289487 (secure 0 0 0) 0x7fdf6c0109c0 con 0x7fdf64066900 2023-11-28T18:30:10.599 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:10.601 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:30:10.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.601+0000 7fdf7e6aa640 1 -- 172.21.15.105:0/3498098273 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdf64066900 msgr2=0x7fdf64068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:10.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.601+0000 7fdf7e6aa640 1 --2- 172.21.15.105:0/3498098273 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdf64066900 0x7fdf64068dc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fdf781590a0 tx=0x7fdf6c008040 comp rx=0 tx=0).stop 2023-11-28T18:30:10.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.601+0000 7fdf7e6aa640 1 -- 172.21.15.105:0/3498098273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf780b2d20 msgr2=0x7fdf78154380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:10.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.601+0000 7fdf7e6aa640 1 --2- 172.21.15.105:0/3498098273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf780b2d20 0x7fdf78154380 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fdf6800c370 tx=0x7fdf6800c830 comp rx=0 tx=0).stop 2023-11-28T18:30:10.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.601+0000 7fdf7e6aa640 1 -- 172.21.15.105:0/3498098273 shutdown_connections 2023-11-28T18:30:10.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.601+0000 7fdf7e6aa640 1 --2- 172.21.15.105:0/3498098273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf7815d4f0 0x7fdf7815c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.601+0000 7fdf7e6aa640 1 --2- 172.21.15.105:0/3498098273 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf780d3c30 0x7fdf781576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.601+0000 7fdf7e6aa640 1 --2- 172.21.15.105:0/3498098273 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdf64066900 0x7fdf64068dc0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.601+0000 7fdf7e6aa640 1 --2- 172.21.15.105:0/3498098273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf780b2d20 0x7fdf78154380 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.601+0000 7fdf7e6aa640 1 -- 172.21.15.105:0/3498098273 >> 172.21.15.105:0/3498098273 conn(0x7fdf780d3030 msgr2=0x7fdf780b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:10.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.601+0000 7fdf7e6aa640 1 -- 172.21.15.105:0/3498098273 shutdown_connections 2023-11-28T18:30:10.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.601+0000 7fdf7e6aa640 1 -- 172.21.15.105:0/3498098273 wait complete. 2023-11-28T18:30:10.620 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2023-11-28T18:30:09.354332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":365888,"kb_used_data":10952,"kb_used_omap":12,"kb_used_meta":354803,"kb_avail":754608832,"statfs":{"total":773094113280,"available":772719443968,"internally_reserved":0,"allocated":11214848,"data_stored":5809024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":363318834},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.214344"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.186798+0000","last_change":"2023-11-28T18:30:06.176504+0000","last_active":"2023-11-28T18:30:06.186798+0000","last_peered":"2023-11-28T18:30:06.186798+0000","last_clean":"2023-11-28T18:30:06.186798+0000","last_became_active":"2023-11-28T18:30:06.176036+0000","last_became_peered":"2023-11-28T18:30:06.176036+0000","last_unstale":"2023-11-28T18:30:06.186798+0000","last_undegraded":"2023-11-28T18:30:06.186798+0000","last_fullsized":"2023-11-28T18:30:06.186798+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:16:51.197942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.195995+0000","last_change":"2023-11-28T18:30:06.192559+0000","last_active":"2023-11-28T18:30:06.195995+0000","last_peered":"2023-11-28T18:30:06.195995+0000","last_clean":"2023-11-28T18:30:06.195995+0000","last_became_active":"2023-11-28T18:30:06.192301+0000","last_became_peered":"2023-11-28T18:30:06.192301+0000","last_unstale":"2023-11-28T18:30:06.195995+0000","last_undegraded":"2023-11-28T18:30:06.195995+0000","last_fullsized":"2023-11-28T18:30:06.195995+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:15:31.961878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.198998+0000","last_change":"2023-11-28T18:30:06.192473+0000","last_active":"2023-11-28T18:30:06.198998+0000","last_peered":"2023-11-28T18:30:06.198998+0000","last_clean":"2023-11-28T18:30:06.198998+0000","last_became_active":"2023-11-28T18:30:06.192177+0000","last_became_peered":"2023-11-28T18:30:06.192177+0000","last_unstale":"2023-11-28T18:30:06.198998+0000","last_undegraded":"2023-11-28T18:30:06.198998+0000","last_fullsized":"2023-11-28T18:30:06.198998+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:11.772188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.197187+0000","last_change":"2023-11-28T18:30:06.192719+0000","last_active":"2023-11-28T18:30:06.197187+0000","last_peered":"2023-11-28T18:30:06.197187+0000","last_clean":"2023-11-28T18:30:06.197187+0000","last_became_active":"2023-11-28T18:30:06.192375+0000","last_became_peered":"2023-11-28T18:30:06.192375+0000","last_unstale":"2023-11-28T18:30:06.197187+0000","last_undegraded":"2023-11-28T18:30:06.197187+0000","last_fullsized":"2023-11-28T18:30:06.197187+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:48:42.993573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.202475+0000","last_change":"2023-11-28T18:30:06.197410+0000","last_active":"2023-11-28T18:30:06.202475+0000","last_peered":"2023-11-28T18:30:06.202475+0000","last_clean":"2023-11-28T18:30:06.202475+0000","last_became_active":"2023-11-28T18:30:06.197152+0000","last_became_peered":"2023-11-28T18:30:06.197152+0000","last_unstale":"2023-11-28T18:30:06.202475+0000","last_undegraded":"2023-11-28T18:30:06.202475+0000","last_fullsized":"2023-11-28T18:30:06.202475+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:24:39.803805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.194557+0000","last_change":"2023-11-28T18:30:06.189834+0000","last_active":"2023-11-28T18:30:06.194557+0000","last_peered":"2023-11-28T18:30:06.194557+0000","last_clean":"2023-11-28T18:30:06.194557+0000","last_became_active":"2023-11-28T18:30:06.189483+0000","last_became_peered":"2023-11-28T18:30:06.189483+0000","last_unstale":"2023-11-28T18:30:06.194557+0000","last_undegraded":"2023-11-28T18:30:06.194557+0000","last_fullsized":"2023-11-28T18:30:06.194557+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:19.324050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.194522+0000","last_change":"2023-11-28T18:30:06.190513+0000","last_active":"2023-11-28T18:30:06.194522+0000","last_peered":"2023-11-28T18:30:06.194522+0000","last_clean":"2023-11-28T18:30:06.194522+0000","last_became_active":"2023-11-28T18:30:06.190244+0000","last_became_peered":"2023-11-28T18:30:06.190244+0000","last_unstale":"2023-11-28T18:30:06.194522+0000","last_undegraded":"2023-11-28T18:30:06.194522+0000","last_fullsized":"2023-11-28T18:30:06.194522+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:30.463710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.200758+0000","last_change":"2023-11-28T18:30:06.196229+0000","last_active":"2023-11-28T18:30:06.200758+0000","last_peered":"2023-11-28T18:30:06.200758+0000","last_clean":"2023-11-28T18:30:06.200758+0000","last_became_active":"2023-11-28T18:30:06.195463+0000","last_became_peered":"2023-11-28T18:30:06.195463+0000","last_unstale":"2023-11-28T18:30:06.200758+0000","last_undegraded":"2023-11-28T18:30:06.200758+0000","last_fullsized":"2023-11-28T18:30:06.200758+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:34:54.990441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.193177+0000","last_change":"2023-11-28T18:30:06.187187+0000","last_active":"2023-11-28T18:30:06.193177+0000","last_peered":"2023-11-28T18:30:06.193177+0000","last_clean":"2023-11-28T18:30:06.193177+0000","last_became_active":"2023-11-28T18:30:06.186849+0000","last_became_peered":"2023-11-28T18:30:06.186849+0000","last_unstale":"2023-11-28T18:30:06.193177+0000","last_undegraded":"2023-11-28T18:30:06.193177+0000","last_fullsized":"2023-11-28T18:30:06.193177+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:24:39.803805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.202018+0000","last_change":"2023-11-28T18:30:06.193867+0000","last_active":"2023-11-28T18:30:06.202018+0000","last_peered":"2023-11-28T18:30:06.202018+0000","last_clean":"2023-11-28T18:30:06.202018+0000","last_became_active":"2023-11-28T18:30:06.193585+0000","last_became_peered":"2023-11-28T18:30:06.193585+0000","last_unstale":"2023-11-28T18:30:06.202018+0000","last_undegraded":"2023-11-28T18:30:06.202018+0000","last_fullsized":"2023-11-28T18:30:06.202018+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:45.712257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.194502+0000","last_change":"2023-11-28T18:30:06.190803+0000","last_active":"2023-11-28T18:30:06.194502+0000","last_peered":"2023-11-28T18:30:06.194502+0000","last_clean":"2023-11-28T18:30:06.194502+0000","last_became_active":"2023-11-28T18:30:06.190511+0000","last_became_peered":"2023-11-28T18:30:06.190511+0000","last_unstale":"2023-11-28T18:30:06.194502+0000","last_undegraded":"2023-11-28T18:30:06.194502+0000","last_fullsized":"2023-11-28T18:30:06.194502+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:23:48.056361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.196787+0000","last_change":"2023-11-28T18:30:06.190247+0000","last_active":"2023-11-28T18:30:06.196787+0000","last_peered":"2023-11-28T18:30:06.196787+0000","last_clean":"2023-11-28T18:30:06.196787+0000","last_became_active":"2023-11-28T18:30:06.189861+0000","last_became_peered":"2023-11-28T18:30:06.189861+0000","last_unstale":"2023-11-28T18:30:06.196787+0000","last_undegraded":"2023-11-28T18:30:06.196787+0000","last_fullsized":"2023-11-28T18:30:06.196787+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:20:53.956031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.180979+0000","last_change":"2023-11-28T18:30:06.177229+0000","last_active":"2023-11-28T18:30:06.180979+0000","last_peered":"2023-11-28T18:30:06.180979+0000","last_clean":"2023-11-28T18:30:06.180979+0000","last_became_active":"2023-11-28T18:30:06.176971+0000","last_became_peered":"2023-11-28T18:30:06.176971+0000","last_unstale":"2023-11-28T18:30:06.180979+0000","last_undegraded":"2023-11-28T18:30:06.180979+0000","last_fullsized":"2023-11-28T18:30:06.180979+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:43:28.813459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.196018+0000","last_change":"2023-11-28T18:30:06.180198+0000","last_active":"2023-11-28T18:30:06.196018+0000","last_peered":"2023-11-28T18:30:06.196018+0000","last_clean":"2023-11-28T18:30:06.196018+0000","last_became_active":"2023-11-28T18:30:06.179931+0000","last_became_peered":"2023-11-28T18:30:06.179931+0000","last_unstale":"2023-11-28T18:30:06.196018+0000","last_undegraded":"2023-11-28T18:30:06.196018+0000","last_fullsized":"2023-11-28T18:30:06.196018+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:03:50.027926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.196011+0000","last_change":"2023-11-28T18:30:06.182089+0000","last_active":"2023-11-28T18:30:06.196011+0000","last_peered":"2023-11-28T18:30:06.196011+0000","last_clean":"2023-11-28T18:30:06.196011+0000","last_became_active":"2023-11-28T18:30:06.181761+0000","last_became_peered":"2023-11-28T18:30:06.181761+0000","last_unstale":"2023-11-28T18:30:06.196011+0000","last_undegraded":"2023-11-28T18:30:06.196011+0000","last_fullsized":"2023-11-28T18:30:06.196011+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:21:51.623438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.204332+0000","last_change":"2023-11-28T18:30:06.202313+0000","last_active":"2023-11-28T18:30:06.204332+0000","last_peered":"2023-11-28T18:30:06.204332+0000","last_clean":"2023-11-28T18:30:06.204332+0000","last_became_active":"2023-11-28T18:30:06.201852+0000","last_became_peered":"2023-11-28T18:30:06.201852+0000","last_unstale":"2023-11-28T18:30:06.204332+0000","last_undegraded":"2023-11-28T18:30:06.204332+0000","last_fullsized":"2023-11-28T18:30:06.204332+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:51:02.819791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.192015+0000","last_change":"2023-11-28T18:30:06.188582+0000","last_active":"2023-11-28T18:30:06.192015+0000","last_peered":"2023-11-28T18:30:06.192015+0000","last_clean":"2023-11-28T18:30:06.192015+0000","last_became_active":"2023-11-28T18:30:06.188316+0000","last_became_peered":"2023-11-28T18:30:06.188316+0000","last_unstale":"2023-11-28T18:30:06.192015+0000","last_undegraded":"2023-11-28T18:30:06.192015+0000","last_fullsized":"2023-11-28T18:30:06.192015+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:10.321867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.195977+0000","last_change":"2023-11-28T18:30:06.191486+0000","last_active":"2023-11-28T18:30:06.195977+0000","last_peered":"2023-11-28T18:30:06.195977+0000","last_clean":"2023-11-28T18:30:06.195977+0000","last_became_active":"2023-11-28T18:30:06.191194+0000","last_became_peered":"2023-11-28T18:30:06.191194+0000","last_unstale":"2023-11-28T18:30:06.195977+0000","last_undegraded":"2023-11-28T18:30:06.195977+0000","last_fullsized":"2023-11-28T18:30:06.195977+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:33:01.075269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.199034+0000","last_change":"2023-11-28T18:30:06.196246+0000","last_active":"2023-11-28T18:30:06.199034+0000","last_peered":"2023-11-28T18:30:06.199034+0000","last_clean":"2023-11-28T18:30:06.199034+0000","last_became_active":"2023-11-28T18:30:06.195769+0000","last_became_peered":"2023-11-28T18:30:06.195769+0000","last_unstale":"2023-11-28T18:30:06.199034+0000","last_undegraded":"2023-11-28T18:30:06.199034+0000","last_fullsized":"2023-11-28T18:30:06.199034+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:47:25.744265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.201222+0000","last_change":"2023-11-28T18:30:06.198305+0000","last_active":"2023-11-28T18:30:06.201222+0000","last_peered":"2023-11-28T18:30:06.201222+0000","last_clean":"2023-11-28T18:30:06.201222+0000","last_became_active":"2023-11-28T18:30:06.198044+0000","last_became_peered":"2023-11-28T18:30:06.198044+0000","last_unstale":"2023-11-28T18:30:06.201222+0000","last_undegraded":"2023-11-28T18:30:06.201222+0000","last_fullsized":"2023-11-28T18:30:06.201222+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:19.324050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.199787+0000","last_change":"2023-11-28T18:30:06.194442+0000","last_active":"2023-11-28T18:30:06.199787+0000","last_peered":"2023-11-28T18:30:06.199787+0000","last_clean":"2023-11-28T18:30:06.199787+0000","last_became_active":"2023-11-28T18:30:06.194159+0000","last_became_peered":"2023-11-28T18:30:06.194159+0000","last_unstale":"2023-11-28T18:30:06.199787+0000","last_undegraded":"2023-11-28T18:30:06.199787+0000","last_fullsized":"2023-11-28T18:30:06.199787+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:20:36.155372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.201871+0000","last_change":"2023-11-28T18:30:06.190286+0000","last_active":"2023-11-28T18:30:06.201871+0000","last_peered":"2023-11-28T18:30:06.201871+0000","last_clean":"2023-11-28T18:30:06.201871+0000","last_became_active":"2023-11-28T18:30:06.190021+0000","last_became_peered":"2023-11-28T18:30:06.190021+0000","last_unstale":"2023-11-28T18:30:06.201871+0000","last_undegraded":"2023-11-28T18:30:06.201871+0000","last_fullsized":"2023-11-28T18:30:06.201871+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:19.324050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.199420+0000","last_change":"2023-11-28T18:30:06.191890+0000","last_active":"2023-11-28T18:30:06.199420+0000","last_peered":"2023-11-28T18:30:06.199420+0000","last_clean":"2023-11-28T18:30:06.199420+0000","last_became_active":"2023-11-28T18:30:06.191565+0000","last_became_peered":"2023-11-28T18:30:06.191565+0000","last_unstale":"2023-11-28T18:30:06.199420+0000","last_undegraded":"2023-11-28T18:30:06.199420+0000","last_fullsized":"2023-11-28T18:30:06.199420+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:34:16.945951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.201905+0000","last_change":"2023-11-28T18:30:06.189574+0000","last_active":"2023-11-28T18:30:06.201905+0000","last_peered":"2023-11-28T18:30:06.201905+0000","last_clean":"2023-11-28T18:30:06.201905+0000","last_became_active":"2023-11-28T18:30:06.189266+0000","last_became_peered":"2023-11-28T18:30:06.189266+0000","last_unstale":"2023-11-28T18:30:06.201905+0000","last_undegraded":"2023-11-28T18:30:06.201905+0000","last_fullsized":"2023-11-28T18:30:06.201905+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:15:31.961878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.196579+0000","last_change":"2023-11-28T18:30:06.192657+0000","last_active":"2023-11-28T18:30:06.196579+0000","last_peered":"2023-11-28T18:30:06.196579+0000","last_clean":"2023-11-28T18:30:06.196579+0000","last_became_active":"2023-11-28T18:30:06.192327+0000","last_became_peered":"2023-11-28T18:30:06.192327+0000","last_unstale":"2023-11-28T18:30:06.196579+0000","last_undegraded":"2023-11-28T18:30:06.196579+0000","last_fullsized":"2023-11-28T18:30:06.196579+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:45.712257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.196581+0000","last_change":"2023-11-28T18:30:06.193105+0000","last_active":"2023-11-28T18:30:06.196581+0000","last_peered":"2023-11-28T18:30:06.196581+0000","last_clean":"2023-11-28T18:30:06.196581+0000","last_became_active":"2023-11-28T18:30:06.192815+0000","last_became_peered":"2023-11-28T18:30:06.192815+0000","last_unstale":"2023-11-28T18:30:06.196581+0000","last_undegraded":"2023-11-28T18:30:06.196581+0000","last_fullsized":"2023-11-28T18:30:06.196581+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:45:35.283335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.184372+0000","last_change":"2023-11-28T18:30:06.180332+0000","last_active":"2023-11-28T18:30:06.184372+0000","last_peered":"2023-11-28T18:30:06.184372+0000","last_clean":"2023-11-28T18:30:06.184372+0000","last_became_active":"2023-11-28T18:30:06.180033+0000","last_became_peered":"2023-11-28T18:30:06.180033+0000","last_unstale":"2023-11-28T18:30:06.184372+0000","last_undegraded":"2023-11-28T18:30:06.184372+0000","last_fullsized":"2023-11-28T18:30:06.184372+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:01:03.780112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.191751+0000","last_change":"2023-11-28T18:30:06.186408+0000","last_active":"2023-11-28T18:30:06.191751+0000","last_peered":"2023-11-28T18:30:06.191751+0000","last_clean":"2023-11-28T18:30:06.191751+0000","last_became_active":"2023-11-28T18:30:06.186127+0000","last_became_peered":"2023-11-28T18:30:06.186127+0000","last_unstale":"2023-11-28T18:30:06.191751+0000","last_undegraded":"2023-11-28T18:30:06.191751+0000","last_fullsized":"2023-11-28T18:30:06.191751+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:45.712257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.205155+0000","last_change":"2023-11-28T18:30:06.200264+0000","last_active":"2023-11-28T18:30:06.205155+0000","last_peered":"2023-11-28T18:30:06.205155+0000","last_clean":"2023-11-28T18:30:06.205155+0000","last_became_active":"2023-11-28T18:30:06.200020+0000","last_became_peered":"2023-11-28T18:30:06.200020+0000","last_unstale":"2023-11-28T18:30:06.205155+0000","last_undegraded":"2023-11-28T18:30:06.205155+0000","last_fullsized":"2023-11-28T18:30:06.205155+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:00:32.059242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.197048+0000","last_change":"2023-11-28T18:30:06.193909+0000","last_active":"2023-11-28T18:30:06.197048+0000","last_peered":"2023-11-28T18:30:06.197048+0000","last_clean":"2023-11-28T18:30:06.197048+0000","last_became_active":"2023-11-28T18:30:06.193651+0000","last_became_peered":"2023-11-28T18:30:06.193651+0000","last_unstale":"2023-11-28T18:30:06.197048+0000","last_undegraded":"2023-11-28T18:30:06.197048+0000","last_fullsized":"2023-11-28T18:30:06.197048+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:24:39.803805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.191655+0000","last_change":"2023-11-28T18:30:06.185088+0000","last_active":"2023-11-28T18:30:06.191655+0000","last_peered":"2023-11-28T18:30:06.191655+0000","last_clean":"2023-11-28T18:30:06.191655+0000","last_became_active":"2023-11-28T18:30:06.184796+0000","last_became_peered":"2023-11-28T18:30:06.184796+0000","last_unstale":"2023-11-28T18:30:06.191655+0000","last_undegraded":"2023-11-28T18:30:06.191655+0000","last_fullsized":"2023-11-28T18:30:06.191655+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:56.618131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.204512+0000","last_change":"2023-11-28T18:30:06.198428+0000","last_active":"2023-11-28T18:30:06.204512+0000","last_peered":"2023-11-28T18:30:06.204512+0000","last_clean":"2023-11-28T18:30:06.204512+0000","last_became_active":"2023-11-28T18:30:06.198146+0000","last_became_peered":"2023-11-28T18:30:06.198146+0000","last_unstale":"2023-11-28T18:30:06.204512+0000","last_undegraded":"2023-11-28T18:30:06.204512+0000","last_fullsized":"2023-11-28T18:30:06.204512+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:13.007653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.202330+0000","last_change":"2023-11-28T18:30:06.196978+0000","last_active":"2023-11-28T18:30:06.202330+0000","last_peered":"2023-11-28T18:30:06.202330+0000","last_clean":"2023-11-28T18:30:06.202330+0000","last_became_active":"2023-11-28T18:30:06.196715+0000","last_became_peered":"2023-11-28T18:30:06.196715+0000","last_unstale":"2023-11-28T18:30:06.202330+0000","last_undegraded":"2023-11-28T18:30:06.202330+0000","last_fullsized":"2023-11-28T18:30:06.202330+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:23.494330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"102'4","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.191463+0000","last_change":"2023-11-28T18:30:05.170363+0000","last_active":"2023-11-28T18:30:06.191463+0000","last_peered":"2023-11-28T18:30:06.191463+0000","last_clean":"2023-11-28T18:30:06.191463+0000","last_became_active":"2023-11-28T18:30:05.170040+0000","last_became_peered":"2023-11-28T18:30:05.170040+0000","last_unstale":"2023-11-28T18:30:06.191463+0000","last_undegraded":"2023-11-28T18:30:06.191463+0000","last_fullsized":"2023-11-28T18:30:06.191463+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:52:59.377449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.203189+0000","last_change":"2023-11-28T18:30:06.196183+0000","last_active":"2023-11-28T18:30:06.203189+0000","last_peered":"2023-11-28T18:30:06.203189+0000","last_clean":"2023-11-28T18:30:06.203189+0000","last_became_active":"2023-11-28T18:30:06.195890+0000","last_became_peered":"2023-11-28T18:30:06.195890+0000","last_unstale":"2023-11-28T18:30:06.203189+0000","last_undegraded":"2023-11-28T18:30:06.203189+0000","last_fullsized":"2023-11-28T18:30:06.203189+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:58.897105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.197893+0000","last_change":"2023-11-28T18:30:05.164703+0000","last_active":"2023-11-28T18:30:06.197893+0000","last_peered":"2023-11-28T18:30:06.197893+0000","last_clean":"2023-11-28T18:30:06.197893+0000","last_became_active":"2023-11-28T18:30:05.164446+0000","last_became_peered":"2023-11-28T18:30:05.164446+0000","last_unstale":"2023-11-28T18:30:06.197893+0000","last_undegraded":"2023-11-28T18:30:06.197893+0000","last_fullsized":"2023-11-28T18:30:06.197893+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:14.015784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.197030+0000","last_change":"2023-11-28T18:30:06.188269+0000","last_active":"2023-11-28T18:30:06.197030+0000","last_peered":"2023-11-28T18:30:06.197030+0000","last_clean":"2023-11-28T18:30:06.197030+0000","last_became_active":"2023-11-28T18:30:06.187981+0000","last_became_peered":"2023-11-28T18:30:06.187981+0000","last_unstale":"2023-11-28T18:30:06.197030+0000","last_undegraded":"2023-11-28T18:30:06.197030+0000","last_fullsized":"2023-11-28T18:30:06.197030+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:41:58.414983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.197601+0000","last_change":"2023-11-28T18:30:05.162404+0000","last_active":"2023-11-28T18:30:06.197601+0000","last_peered":"2023-11-28T18:30:06.197601+0000","last_clean":"2023-11-28T18:30:06.197601+0000","last_became_active":"2023-11-28T18:30:05.161794+0000","last_became_peered":"2023-11-28T18:30:05.161794+0000","last_unstale":"2023-11-28T18:30:06.197601+0000","last_undegraded":"2023-11-28T18:30:06.197601+0000","last_fullsized":"2023-11-28T18:30:06.197601+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:27:55.831255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.202643+0000","last_change":"2023-11-28T18:30:06.196752+0000","last_active":"2023-11-28T18:30:06.202643+0000","last_peered":"2023-11-28T18:30:06.202643+0000","last_clean":"2023-11-28T18:30:06.202643+0000","last_became_active":"2023-11-28T18:30:06.194764+0000","last_became_peered":"2023-11-28T18:30:06.194764+0000","last_unstale":"2023-11-28T18:30:06.202643+0000","last_undegraded":"2023-11-28T18:30:06.202643+0000","last_fullsized":"2023-11-28T18:30:06.202643+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:48:42.993573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.197716+0000","last_change":"2023-11-28T18:30:05.158127+0000","last_active":"2023-11-28T18:30:06.197716+0000","last_peered":"2023-11-28T18:30:06.197716+0000","last_clean":"2023-11-28T18:30:06.197716+0000","last_became_active":"2023-11-28T18:30:05.157850+0000","last_became_peered":"2023-11-28T18:30:05.157850+0000","last_unstale":"2023-11-28T18:30:06.197716+0000","last_undegraded":"2023-11-28T18:30:06.197716+0000","last_fullsized":"2023-11-28T18:30:06.197716+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:07.006261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.188387+0000","last_change":"2023-11-28T18:30:06.186064+0000","last_active":"2023-11-28T18:30:06.188387+0000","last_peered":"2023-11-28T18:30:06.188387+0000","last_clean":"2023-11-28T18:30:06.188387+0000","last_became_active":"2023-11-28T18:30:06.185779+0000","last_became_peered":"2023-11-28T18:30:06.185779+0000","last_unstale":"2023-11-28T18:30:06.188387+0000","last_undegraded":"2023-11-28T18:30:06.188387+0000","last_fullsized":"2023-11-28T18:30:06.188387+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:36:24.815145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.190187+0000","last_change":"2023-11-28T18:30:05.171666+0000","last_active":"2023-11-28T18:30:06.190187+0000","last_peered":"2023-11-28T18:30:06.190187+0000","last_clean":"2023-11-28T18:30:06.190187+0000","last_became_active":"2023-11-28T18:30:05.171426+0000","last_became_peered":"2023-11-28T18:30:05.171426+0000","last_unstale":"2023-11-28T18:30:06.190187+0000","last_undegraded":"2023-11-28T18:30:06.190187+0000","last_fullsized":"2023-11-28T18:30:06.190187+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:52:03.835518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.199017+0000","last_change":"2023-11-28T18:30:06.196037+0000","last_active":"2023-11-28T18:30:06.199017+0000","last_peered":"2023-11-28T18:30:06.199017+0000","last_clean":"2023-11-28T18:30:06.199017+0000","last_became_active":"2023-11-28T18:30:06.195416+0000","last_became_peered":"2023-11-28T18:30:06.195416+0000","last_unstale":"2023-11-28T18:30:06.199017+0000","last_undegraded":"2023-11-28T18:30:06.199017+0000","last_fullsized":"2023-11-28T18:30:06.199017+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:55:39.257881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.194553+0000","last_change":"2023-11-28T18:30:05.160156+0000","last_active":"2023-11-28T18:30:06.194553+0000","last_peered":"2023-11-28T18:30:06.194553+0000","last_clean":"2023-11-28T18:30:06.194553+0000","last_became_active":"2023-11-28T18:30:05.159877+0000","last_became_peered":"2023-11-28T18:30:05.159877+0000","last_unstale":"2023-11-28T18:30:06.194553+0000","last_undegraded":"2023-11-28T18:30:06.194553+0000","last_fullsized":"2023-11-28T18:30:06.194553+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:38:12.960970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.194582+0000","last_change":"2023-11-28T18:30:06.185704+0000","last_active":"2023-11-28T18:30:06.194582+0000","last_peered":"2023-11-28T18:30:06.194582+0000","last_clean":"2023-11-28T18:30:06.194582+0000","last_became_active":"2023-11-28T18:30:06.185405+0000","last_became_peered":"2023-11-28T18:30:06.185405+0000","last_unstale":"2023-11-28T18:30:06.194582+0000","last_undegraded":"2023-11-28T18:30:06.194582+0000","last_fullsized":"2023-11-28T18:30:06.194582+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:12:03.541522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.178522+0000","last_change":"2023-11-28T18:30:05.157935+0000","last_active":"2023-11-28T18:30:06.178522+0000","last_peered":"2023-11-28T18:30:06.178522+0000","last_clean":"2023-11-28T18:30:06.178522+0000","last_became_active":"2023-11-28T18:30:05.157274+0000","last_became_peered":"2023-11-28T18:30:05.157274+0000","last_unstale":"2023-11-28T18:30:06.178522+0000","last_undegraded":"2023-11-28T18:30:06.178522+0000","last_fullsized":"2023-11-28T18:30:06.178522+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:48:41.986238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.182639+0000","last_change":"2023-11-28T18:30:06.180445+0000","last_active":"2023-11-28T18:30:06.182639+0000","last_peered":"2023-11-28T18:30:06.182639+0000","last_clean":"2023-11-28T18:30:06.182639+0000","last_became_active":"2023-11-28T18:30:06.179442+0000","last_became_peered":"2023-11-28T18:30:06.179442+0000","last_unstale":"2023-11-28T18:30:06.182639+0000","last_undegraded":"2023-11-28T18:30:06.182639+0000","last_fullsized":"2023-11-28T18:30:06.182639+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:01:03.780112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.159406+0000","last_change":"2023-11-28T18:30:05.153674+0000","last_active":"2023-11-28T18:30:06.159406+0000","last_peered":"2023-11-28T18:30:06.159406+0000","last_clean":"2023-11-28T18:30:06.159406+0000","last_became_active":"2023-11-28T18:30:05.153327+0000","last_became_peered":"2023-11-28T18:30:05.153327+0000","last_unstale":"2023-11-28T18:30:06.159406+0000","last_undegraded":"2023-11-28T18:30:06.159406+0000","last_fullsized":"2023-11-28T18:30:06.159406+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:38:12.960970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.199111+0000","last_change":"2023-11-28T18:30:06.196453+0000","last_active":"2023-11-28T18:30:06.199111+0000","last_peered":"2023-11-28T18:30:06.199111+0000","last_clean":"2023-11-28T18:30:06.199111+0000","last_became_active":"2023-11-28T18:30:06.194959+0000","last_became_peered":"2023-11-28T18:30:06.194959+0000","last_unstale":"2023-11-28T18:30:06.199111+0000","last_undegraded":"2023-11-28T18:30:06.199111+0000","last_fullsized":"2023-11-28T18:30:06.199111+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:23:24.963867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.179527+0000","last_change":"2023-11-28T18:30:05.162132+0000","last_active":"2023-11-28T18:30:06.179527+0000","last_peered":"2023-11-28T18:30:06.179527+0000","last_clean":"2023-11-28T18:30:06.179527+0000","last_became_active":"2023-11-28T18:30:05.161844+0000","last_became_peered":"2023-11-28T18:30:05.161844+0000","last_unstale":"2023-11-28T18:30:06.179527+0000","last_undegraded":"2023-11-28T18:30:06.179527+0000","last_fullsized":"2023-11-28T18:30:06.179527+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:43:27.806124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.204480+0000","last_change":"2023-11-28T18:30:06.202617+0000","last_active":"2023-11-28T18:30:06.204480+0000","last_peered":"2023-11-28T18:30:06.204480+0000","last_clean":"2023-11-28T18:30:06.204480+0000","last_became_active":"2023-11-28T18:30:06.202320+0000","last_became_peered":"2023-11-28T18:30:06.202320+0000","last_unstale":"2023-11-28T18:30:06.204480+0000","last_undegraded":"2023-11-28T18:30:06.204480+0000","last_fullsized":"2023-11-28T18:30:06.204480+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:33:01.482047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.170921+0000","last_change":"2023-11-28T18:30:05.168611+0000","last_active":"2023-11-28T18:30:06.170921+0000","last_peered":"2023-11-28T18:30:06.170921+0000","last_clean":"2023-11-28T18:30:06.170921+0000","last_became_active":"2023-11-28T18:30:05.168350+0000","last_became_peered":"2023-11-28T18:30:05.168350+0000","last_unstale":"2023-11-28T18:30:06.170921+0000","last_undegraded":"2023-11-28T18:30:06.170921+0000","last_fullsized":"2023-11-28T18:30:06.170921+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:21:09.627822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.200357+0000","last_change":"2023-11-28T18:30:06.195887+0000","last_active":"2023-11-28T18:30:06.200357+0000","last_peered":"2023-11-28T18:30:06.200357+0000","last_clean":"2023-11-28T18:30:06.200357+0000","last_became_active":"2023-11-28T18:30:06.193400+0000","last_became_peered":"2023-11-28T18:30:06.193400+0000","last_unstale":"2023-11-28T18:30:06.200357+0000","last_undegraded":"2023-11-28T18:30:06.200357+0000","last_fullsized":"2023-11-28T18:30:06.200357+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:25:00.493596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"102'3","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.190509+0000","last_change":"2023-11-28T18:30:05.171478+0000","last_active":"2023-11-28T18:30:06.190509+0000","last_peered":"2023-11-28T18:30:06.190509+0000","last_clean":"2023-11-28T18:30:06.190509+0000","last_became_active":"2023-11-28T18:30:05.170871+0000","last_became_peered":"2023-11-28T18:30:05.170871+0000","last_unstale":"2023-11-28T18:30:06.190509+0000","last_undegraded":"2023-11-28T18:30:06.190509+0000","last_fullsized":"2023-11-28T18:30:06.190509+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:47:29.877689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.203160+0000","last_change":"2023-11-28T18:30:06.200566+0000","last_active":"2023-11-28T18:30:06.203160+0000","last_peered":"2023-11-28T18:30:06.203160+0000","last_clean":"2023-11-28T18:30:06.203160+0000","last_became_active":"2023-11-28T18:30:06.200255+0000","last_became_peered":"2023-11-28T18:30:06.200255+0000","last_unstale":"2023-11-28T18:30:06.203160+0000","last_undegraded":"2023-11-28T18:30:06.203160+0000","last_fullsized":"2023-11-28T18:30:06.203160+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:27:56.838590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"102'6","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.204576+0000","last_change":"2023-11-28T18:30:05.173650+0000","last_active":"2023-11-28T18:30:06.204576+0000","last_peered":"2023-11-28T18:30:06.204576+0000","last_clean":"2023-11-28T18:30:06.204576+0000","last_became_active":"2023-11-28T18:30:05.173175+0000","last_became_peered":"2023-11-28T18:30:05.173175+0000","last_unstale":"2023-11-28T18:30:06.204576+0000","last_undegraded":"2023-11-28T18:30:06.204576+0000","last_fullsized":"2023-11-28T18:30:06.204576+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:41:57.407648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.184517+0000","last_change":"2023-11-28T18:30:06.179449+0000","last_active":"2023-11-28T18:30:06.184517+0000","last_peered":"2023-11-28T18:30:06.184517+0000","last_clean":"2023-11-28T18:30:06.184517+0000","last_became_active":"2023-11-28T18:30:06.179135+0000","last_became_peered":"2023-11-28T18:30:06.179135+0000","last_unstale":"2023-11-28T18:30:06.184517+0000","last_undegraded":"2023-11-28T18:30:06.184517+0000","last_fullsized":"2023-11-28T18:30:06.184517+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:36:24.815145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.199561+0000","last_change":"2023-11-28T18:30:05.167772+0000","last_active":"2023-11-28T18:30:06.199561+0000","last_peered":"2023-11-28T18:30:06.199561+0000","last_clean":"2023-11-28T18:30:06.199561+0000","last_became_active":"2023-11-28T18:30:05.167500+0000","last_became_peered":"2023-11-28T18:30:05.167500+0000","last_unstale":"2023-11-28T18:30:06.199561+0000","last_undegraded":"2023-11-28T18:30:06.199561+0000","last_fullsized":"2023-11-28T18:30:06.199561+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:20:52.948696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.194013+0000","last_change":"2023-11-28T18:30:06.188105+0000","last_active":"2023-11-28T18:30:06.194013+0000","last_peered":"2023-11-28T18:30:06.194013+0000","last_clean":"2023-11-28T18:30:06.194013+0000","last_became_active":"2023-11-28T18:30:06.187701+0000","last_became_peered":"2023-11-28T18:30:06.187701+0000","last_unstale":"2023-11-28T18:30:06.194013+0000","last_undegraded":"2023-11-28T18:30:06.194013+0000","last_fullsized":"2023-11-28T18:30:06.194013+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:08.013596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.158851+0000","last_change":"2023-11-28T18:30:05.157249+0000","last_active":"2023-11-28T18:30:06.158851+0000","last_peered":"2023-11-28T18:30:06.158851+0000","last_clean":"2023-11-28T18:30:06.158851+0000","last_became_active":"2023-11-28T18:30:05.156416+0000","last_became_peered":"2023-11-28T18:30:05.156416+0000","last_unstale":"2023-11-28T18:30:06.158851+0000","last_undegraded":"2023-11-28T18:30:06.158851+0000","last_fullsized":"2023-11-28T18:30:06.158851+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:45:34.276000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.204419+0000","last_change":"2023-11-28T18:30:06.197709+0000","last_active":"2023-11-28T18:30:06.204419+0000","last_peered":"2023-11-28T18:30:06.204419+0000","last_clean":"2023-11-28T18:30:06.204419+0000","last_became_active":"2023-11-28T18:30:06.196045+0000","last_became_peered":"2023-11-28T18:30:06.196045+0000","last_unstale":"2023-11-28T18:30:06.204419+0000","last_undegraded":"2023-11-28T18:30:06.204419+0000","last_fullsized":"2023-11-28T18:30:06.204419+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:41:55.422124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.182280+0000","last_change":"2023-11-28T18:30:05.168287+0000","last_active":"2023-11-28T18:30:06.182280+0000","last_peered":"2023-11-28T18:30:06.182280+0000","last_clean":"2023-11-28T18:30:06.182280+0000","last_became_active":"2023-11-28T18:30:05.168010+0000","last_became_peered":"2023-11-28T18:30:05.168010+0000","last_unstale":"2023-11-28T18:30:06.182280+0000","last_undegraded":"2023-11-28T18:30:06.182280+0000","last_fullsized":"2023-11-28T18:30:06.182280+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:48:00.884187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.192034+0000","last_change":"2023-11-28T18:30:06.189209+0000","last_active":"2023-11-28T18:30:06.192034+0000","last_peered":"2023-11-28T18:30:06.192034+0000","last_clean":"2023-11-28T18:30:06.192034+0000","last_became_active":"2023-11-28T18:30:06.188073+0000","last_became_peered":"2023-11-28T18:30:06.188073+0000","last_unstale":"2023-11-28T18:30:06.192034+0000","last_undegraded":"2023-11-28T18:30:06.192034+0000","last_fullsized":"2023-11-28T18:30:06.192034+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:27:13.812700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"102'8","reported_seq":19,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.199192+0000","last_change":"2023-11-28T18:30:05.158153+0000","last_active":"2023-11-28T18:30:06.199192+0000","last_peered":"2023-11-28T18:30:06.199192+0000","last_clean":"2023-11-28T18:30:06.199192+0000","last_became_active":"2023-11-28T18:30:05.157810+0000","last_became_peered":"2023-11-28T18:30:05.157810+0000","last_unstale":"2023-11-28T18:30:06.199192+0000","last_undegraded":"2023-11-28T18:30:06.199192+0000","last_fullsized":"2023-11-28T18:30:06.199192+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:33:00.474713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.204561+0000","last_change":"2023-11-28T18:30:06.199260+0000","last_active":"2023-11-28T18:30:06.204561+0000","last_peered":"2023-11-28T18:30:06.204561+0000","last_clean":"2023-11-28T18:30:06.204561+0000","last_became_active":"2023-11-28T18:30:06.195752+0000","last_became_peered":"2023-11-28T18:30:06.195752+0000","last_unstale":"2023-11-28T18:30:06.204561+0000","last_undegraded":"2023-11-28T18:30:06.204561+0000","last_fullsized":"2023-11-28T18:30:06.204561+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:07:32.818669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.198463+0000","last_change":"2023-11-28T18:30:05.169718+0000","last_active":"2023-11-28T18:30:06.198463+0000","last_peered":"2023-11-28T18:30:06.198463+0000","last_clean":"2023-11-28T18:30:06.198463+0000","last_became_active":"2023-11-28T18:30:05.169321+0000","last_became_peered":"2023-11-28T18:30:05.169321+0000","last_unstale":"2023-11-28T18:30:06.198463+0000","last_undegraded":"2023-11-28T18:30:06.198463+0000","last_fullsized":"2023-11-28T18:30:06.198463+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:20:52.948696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.193163+0000","last_change":"2023-11-28T18:30:06.187217+0000","last_active":"2023-11-28T18:30:06.193163+0000","last_peered":"2023-11-28T18:30:06.193163+0000","last_clean":"2023-11-28T18:30:06.193163+0000","last_became_active":"2023-11-28T18:30:06.186852+0000","last_became_peered":"2023-11-28T18:30:06.186852+0000","last_unstale":"2023-11-28T18:30:06.193163+0000","last_undegraded":"2023-11-28T18:30:06.193163+0000","last_fullsized":"2023-11-28T18:30:06.193163+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:27:56.838590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.182178+0000","last_change":"2023-11-28T18:30:05.155867+0000","last_active":"2023-11-28T18:30:06.182178+0000","last_peered":"2023-11-28T18:30:06.182178+0000","last_clean":"2023-11-28T18:30:06.182178+0000","last_became_active":"2023-11-28T18:30:05.155572+0000","last_became_peered":"2023-11-28T18:30:05.155572+0000","last_unstale":"2023-11-28T18:30:06.182178+0000","last_undegraded":"2023-11-28T18:30:06.182178+0000","last_fullsized":"2023-11-28T18:30:06.182178+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:49:07.897264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.202493+0000","last_change":"2023-11-28T18:30:06.197574+0000","last_active":"2023-11-28T18:30:06.202493+0000","last_peered":"2023-11-28T18:30:06.202493+0000","last_clean":"2023-11-28T18:30:06.202493+0000","last_became_active":"2023-11-28T18:30:06.197315+0000","last_became_peered":"2023-11-28T18:30:06.197315+0000","last_unstale":"2023-11-28T18:30:06.202493+0000","last_undegraded":"2023-11-28T18:30:06.202493+0000","last_fullsized":"2023-11-28T18:30:06.202493+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:41:58.414983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.170845+0000","last_change":"2023-11-28T18:30:05.170900+0000","last_active":"2023-11-28T18:30:06.170845+0000","last_peered":"2023-11-28T18:30:06.170845+0000","last_clean":"2023-11-28T18:30:06.170845+0000","last_became_active":"2023-11-28T18:30:05.170594+0000","last_became_peered":"2023-11-28T18:30:05.170594+0000","last_unstale":"2023-11-28T18:30:06.170845+0000","last_undegraded":"2023-11-28T18:30:06.170845+0000","last_fullsized":"2023-11-28T18:30:06.170845+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:52:59.377449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.179586+0000","last_change":"2023-11-28T18:30:06.172634+0000","last_active":"2023-11-28T18:30:06.179586+0000","last_peered":"2023-11-28T18:30:06.179586+0000","last_clean":"2023-11-28T18:30:06.179586+0000","last_became_active":"2023-11-28T18:30:06.172347+0000","last_became_peered":"2023-11-28T18:30:06.172347+0000","last_unstale":"2023-11-28T18:30:06.179586+0000","last_undegraded":"2023-11-28T18:30:06.179586+0000","last_fullsized":"2023-11-28T18:30:06.179586+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:00:14.663059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.199705+0000","last_change":"2023-11-28T18:30:05.175506+0000","last_active":"2023-11-28T18:30:06.199705+0000","last_peered":"2023-11-28T18:30:06.199705+0000","last_clean":"2023-11-28T18:30:06.199705+0000","last_became_active":"2023-11-28T18:30:05.175079+0000","last_became_peered":"2023-11-28T18:30:05.175079+0000","last_unstale":"2023-11-28T18:30:06.199705+0000","last_undegraded":"2023-11-28T18:30:06.199705+0000","last_fullsized":"2023-11-28T18:30:06.199705+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:56:30.582825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.189225+0000","last_change":"2023-11-28T18:30:06.186382+0000","last_active":"2023-11-28T18:30:06.189225+0000","last_peered":"2023-11-28T18:30:06.189225+0000","last_clean":"2023-11-28T18:30:06.189225+0000","last_became_active":"2023-11-28T18:30:06.186094+0000","last_became_peered":"2023-11-28T18:30:06.186094+0000","last_unstale":"2023-11-28T18:30:06.189225+0000","last_undegraded":"2023-11-28T18:30:06.189225+0000","last_fullsized":"2023-11-28T18:30:06.189225+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:06:08.004603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.184888+0000","last_change":"2023-11-28T18:30:05.168522+0000","last_active":"2023-11-28T18:30:06.184888+0000","last_peered":"2023-11-28T18:30:06.184888+0000","last_clean":"2023-11-28T18:30:06.184888+0000","last_became_active":"2023-11-28T18:30:05.167218+0000","last_became_peered":"2023-11-28T18:30:05.167218+0000","last_unstale":"2023-11-28T18:30:06.184888+0000","last_undegraded":"2023-11-28T18:30:06.184888+0000","last_fullsized":"2023-11-28T18:30:06.184888+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:31:37.741493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.197238+0000","last_change":"2023-11-28T18:30:06.191690+0000","last_active":"2023-11-28T18:30:06.197238+0000","last_peered":"2023-11-28T18:30:06.197238+0000","last_clean":"2023-11-28T18:30:06.197238+0000","last_became_active":"2023-11-28T18:30:06.191057+0000","last_became_peered":"2023-11-28T18:30:06.191057+0000","last_unstale":"2023-11-28T18:30:06.197238+0000","last_undegraded":"2023-11-28T18:30:06.197238+0000","last_fullsized":"2023-11-28T18:30:06.197238+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:23:48.056361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.166727+0000","last_change":"2023-11-28T18:30:05.168933+0000","last_active":"2023-11-28T18:30:06.166727+0000","last_peered":"2023-11-28T18:30:06.166727+0000","last_clean":"2023-11-28T18:30:06.166727+0000","last_became_active":"2023-11-28T18:30:05.168436+0000","last_became_peered":"2023-11-28T18:30:05.168436+0000","last_unstale":"2023-11-28T18:30:06.166727+0000","last_undegraded":"2023-11-28T18:30:06.166727+0000","last_fullsized":"2023-11-28T18:30:06.166727+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:37:47.950433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.204380+0000","last_change":"2023-11-28T18:30:06.197151+0000","last_active":"2023-11-28T18:30:06.204380+0000","last_peered":"2023-11-28T18:30:06.204380+0000","last_clean":"2023-11-28T18:30:06.204380+0000","last_became_active":"2023-11-28T18:30:06.193062+0000","last_became_peered":"2023-11-28T18:30:06.193062+0000","last_unstale":"2023-11-28T18:30:06.204380+0000","last_undegraded":"2023-11-28T18:30:06.204380+0000","last_fullsized":"2023-11-28T18:30:06.204380+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:17:01.857749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.181967+0000","last_change":"2023-11-28T18:30:05.161460+0000","last_active":"2023-11-28T18:30:06.181967+0000","last_peered":"2023-11-28T18:30:06.181967+0000","last_clean":"2023-11-28T18:30:06.181967+0000","last_became_active":"2023-11-28T18:30:05.161023+0000","last_became_peered":"2023-11-28T18:30:05.161023+0000","last_unstale":"2023-11-28T18:30:06.181967+0000","last_undegraded":"2023-11-28T18:30:06.181967+0000","last_fullsized":"2023-11-28T18:30:06.181967+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:56.469337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.182516+0000","last_change":"2023-11-28T18:30:06.175722+0000","last_active":"2023-11-28T18:30:06.182516+0000","last_peered":"2023-11-28T18:30:06.182516+0000","last_clean":"2023-11-28T18:30:06.182516+0000","last_became_active":"2023-11-28T18:30:06.175438+0000","last_became_peered":"2023-11-28T18:30:06.175438+0000","last_unstale":"2023-11-28T18:30:06.182516+0000","last_undegraded":"2023-11-28T18:30:06.182516+0000","last_fullsized":"2023-11-28T18:30:06.182516+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:50:09.060450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.202633+0000","last_change":"2023-11-28T18:30:05.164247+0000","last_active":"2023-11-28T18:30:06.202633+0000","last_peered":"2023-11-28T18:30:06.202633+0000","last_clean":"2023-11-28T18:30:06.202633+0000","last_became_active":"2023-11-28T18:30:05.163970+0000","last_became_peered":"2023-11-28T18:30:05.163970+0000","last_unstale":"2023-11-28T18:30:06.202633+0000","last_undegraded":"2023-11-28T18:30:06.202633+0000","last_fullsized":"2023-11-28T18:30:06.202633+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:45:34.276000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.196039+0000","last_change":"2023-11-28T18:30:06.190076+0000","last_active":"2023-11-28T18:30:06.196039+0000","last_peered":"2023-11-28T18:30:06.196039+0000","last_clean":"2023-11-28T18:30:06.196039+0000","last_became_active":"2023-11-28T18:30:06.189787+0000","last_became_peered":"2023-11-28T18:30:06.189787+0000","last_unstale":"2023-11-28T18:30:06.196039+0000","last_undegraded":"2023-11-28T18:30:06.196039+0000","last_fullsized":"2023-11-28T18:30:06.196039+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:08.013596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.171188+0000","last_change":"2023-11-28T18:30:05.172663+0000","last_active":"2023-11-28T18:30:06.171188+0000","last_peered":"2023-11-28T18:30:06.171188+0000","last_clean":"2023-11-28T18:30:06.171188+0000","last_became_active":"2023-11-28T18:30:05.172344+0000","last_became_peered":"2023-11-28T18:30:05.172344+0000","last_unstale":"2023-11-28T18:30:06.171188+0000","last_undegraded":"2023-11-28T18:30:06.171188+0000","last_fullsized":"2023-11-28T18:30:06.171188+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:40:33.167683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.197493+0000","last_change":"2023-11-28T18:30:06.192022+0000","last_active":"2023-11-28T18:30:06.197493+0000","last_peered":"2023-11-28T18:30:06.197493+0000","last_clean":"2023-11-28T18:30:06.197493+0000","last_became_active":"2023-11-28T18:30:06.191766+0000","last_became_peered":"2023-11-28T18:30:06.191766+0000","last_unstale":"2023-11-28T18:30:06.197493+0000","last_undegraded":"2023-11-28T18:30:06.197493+0000","last_fullsized":"2023-11-28T18:30:06.197493+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:34:38.117919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.172327+0000","last_change":"2023-11-28T18:30:05.166981+0000","last_active":"2023-11-28T18:30:06.172327+0000","last_peered":"2023-11-28T18:30:06.172327+0000","last_clean":"2023-11-28T18:30:06.172327+0000","last_became_active":"2023-11-28T18:30:05.166641+0000","last_became_peered":"2023-11-28T18:30:05.166641+0000","last_unstale":"2023-11-28T18:30:06.172327+0000","last_undegraded":"2023-11-28T18:30:06.172327+0000","last_fullsized":"2023-11-28T18:30:06.172327+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:28:56.010559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.194439+0000","last_change":"2023-11-28T18:30:06.189742+0000","last_active":"2023-11-28T18:30:06.194439+0000","last_peered":"2023-11-28T18:30:06.194439+0000","last_clean":"2023-11-28T18:30:06.194439+0000","last_became_active":"2023-11-28T18:30:06.189489+0000","last_became_peered":"2023-11-28T18:30:06.189489+0000","last_unstale":"2023-11-28T18:30:06.194439+0000","last_undegraded":"2023-11-28T18:30:06.194439+0000","last_fullsized":"2023-11-28T18:30:06.194439+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:43:28.813459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.181706+0000","last_change":"2023-11-28T18:30:05.170517+0000","last_active":"2023-11-28T18:30:06.181706+0000","last_peered":"2023-11-28T18:30:06.181706+0000","last_clean":"2023-11-28T18:30:06.181706+0000","last_became_active":"2023-11-28T18:30:05.170185+0000","last_became_peered":"2023-11-28T18:30:05.170185+0000","last_unstale":"2023-11-28T18:30:06.181706+0000","last_undegraded":"2023-11-28T18:30:06.181706+0000","last_fullsized":"2023-11-28T18:30:06.181706+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:28:56.010559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.204532+0000","last_change":"2023-11-28T18:30:06.198229+0000","last_active":"2023-11-28T18:30:06.204532+0000","last_peered":"2023-11-28T18:30:06.204532+0000","last_clean":"2023-11-28T18:30:06.204532+0000","last_became_active":"2023-11-28T18:30:06.196839+0000","last_became_peered":"2023-11-28T18:30:06.196839+0000","last_unstale":"2023-11-28T18:30:06.204532+0000","last_undegraded":"2023-11-28T18:30:06.204532+0000","last_fullsized":"2023-11-28T18:30:06.204532+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:10:44.334510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.178035+0000","last_change":"2023-11-28T18:30:05.167244+0000","last_active":"2023-11-28T18:30:06.178035+0000","last_peered":"2023-11-28T18:30:06.178035+0000","last_clean":"2023-11-28T18:30:06.178035+0000","last_became_active":"2023-11-28T18:30:05.165852+0000","last_became_peered":"2023-11-28T18:30:05.165852+0000","last_unstale":"2023-11-28T18:30:06.178035+0000","last_undegraded":"2023-11-28T18:30:06.178035+0000","last_fullsized":"2023-11-28T18:30:06.178035+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:28:56.010559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.193577+0000","last_change":"2023-11-28T18:30:06.190372+0000","last_active":"2023-11-28T18:30:06.193577+0000","last_peered":"2023-11-28T18:30:06.193577+0000","last_clean":"2023-11-28T18:30:06.193577+0000","last_became_active":"2023-11-28T18:30:06.190100+0000","last_became_peered":"2023-11-28T18:30:06.190100+0000","last_unstale":"2023-11-28T18:30:06.193577+0000","last_undegraded":"2023-11-28T18:30:06.193577+0000","last_fullsized":"2023-11-28T18:30:06.193577+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:05:36.218930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.185021+0000","last_change":"2023-11-28T18:30:05.168365+0000","last_active":"2023-11-28T18:30:06.185021+0000","last_peered":"2023-11-28T18:30:06.185021+0000","last_clean":"2023-11-28T18:30:06.185021+0000","last_became_active":"2023-11-28T18:30:05.168090+0000","last_became_peered":"2023-11-28T18:30:05.168090+0000","last_unstale":"2023-11-28T18:30:06.185021+0000","last_undegraded":"2023-11-28T18:30:06.185021+0000","last_fullsized":"2023-11-28T18:30:06.185021+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:12:45.518541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.196169+0000","last_change":"2023-11-28T18:30:06.181894+0000","last_active":"2023-11-28T18:30:06.196169+0000","last_peered":"2023-11-28T18:30:06.196169+0000","last_clean":"2023-11-28T18:30:06.196169+0000","last_became_active":"2023-11-28T18:30:06.181571+0000","last_became_peered":"2023-11-28T18:30:06.181571+0000","last_unstale":"2023-11-28T18:30:06.196169+0000","last_undegraded":"2023-11-28T18:30:06.196169+0000","last_fullsized":"2023-11-28T18:30:06.196169+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:36:22.366120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.166856+0000","last_change":"2023-11-28T18:30:05.168740+0000","last_active":"2023-11-28T18:30:06.166856+0000","last_peered":"2023-11-28T18:30:06.166856+0000","last_clean":"2023-11-28T18:30:06.166856+0000","last_became_active":"2023-11-28T18:30:05.168082+0000","last_became_peered":"2023-11-28T18:30:05.168082+0000","last_unstale":"2023-11-28T18:30:06.166856+0000","last_undegraded":"2023-11-28T18:30:06.166856+0000","last_fullsized":"2023-11-28T18:30:06.166856+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:33:00.474713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.188537+0000","last_change":"2023-11-28T18:30:06.181114+0000","last_active":"2023-11-28T18:30:06.188537+0000","last_peered":"2023-11-28T18:30:06.188537+0000","last_clean":"2023-11-28T18:30:06.188537+0000","last_became_active":"2023-11-28T18:30:06.180806+0000","last_became_peered":"2023-11-28T18:30:06.180806+0000","last_unstale":"2023-11-28T18:30:06.188537+0000","last_undegraded":"2023-11-28T18:30:06.188537+0000","last_fullsized":"2023-11-28T18:30:06.188537+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:34:38.117919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.182102+0000","last_change":"2023-11-28T18:30:05.161265+0000","last_active":"2023-11-28T18:30:06.182102+0000","last_peered":"2023-11-28T18:30:06.182102+0000","last_clean":"2023-11-28T18:30:06.182102+0000","last_became_active":"2023-11-28T18:30:05.160714+0000","last_became_peered":"2023-11-28T18:30:05.160714+0000","last_unstale":"2023-11-28T18:30:06.182102+0000","last_undegraded":"2023-11-28T18:30:06.182102+0000","last_fullsized":"2023-11-28T18:30:06.182102+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:12:35.016288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.181312+0000","last_change":"2023-11-28T18:30:06.178208+0000","last_active":"2023-11-28T18:30:06.181312+0000","last_peered":"2023-11-28T18:30:06.181312+0000","last_clean":"2023-11-28T18:30:06.181312+0000","last_became_active":"2023-11-28T18:30:06.175565+0000","last_became_peered":"2023-11-28T18:30:06.175565+0000","last_unstale":"2023-11-28T18:30:06.181312+0000","last_undegraded":"2023-11-28T18:30:06.181312+0000","last_fullsized":"2023-11-28T18:30:06.181312+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:05.137034+0000","last_clean_scrub_stamp":"2023-11-28T18:30:05.137034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:45:35.283335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-28T18:30:06.195860+0000","last_change":"2023-11-28T18:30:05.164191+0000","last_active":"2023-11-28T18:30:06.195860+0000","last_peered":"2023-11-28T18:30:06.195860+0000","last_clean":"2023-11-28T18:30:06.195860+0000","last_became_active":"2023-11-28T18:30:05.163923+0000","last_became_peered":"2023-11-28T18:30:05.163923+0000","last_unstale":"2023-11-28T18:30:06.195860+0000","last_undegraded":"2023-11-28T18:30:06.195860+0000","last_fullsized":"2023-11-28T18:30:06.195860+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:30:04.129699+0000","last_clean_scrub_stamp":"2023-11-28T18:30:04.129699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:55:27.920271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":236,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-28T18:30:05.148201+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:30:05.148201+0000","last_peered":"2023-11-28T18:30:05.148201+0000","last_clean":"2023-11-28T18:30:05.148201+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:30:05.148201+0000","last_undegraded":"2023-11-28T18:30:05.148201+0000","last_fullsized":"2023-11-28T18:30:05.148201+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574933,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46876,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324964,"statfs":{"total":96636764160,"available":96588763136,"internally_reserved":0,"allocated":1847296,"data_stored":1168904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.628,"15min":0.70799999999999996},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.91900000000000004,"5min":0.98099999999999998,"15min":2.427},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.63300000000000001,"15min":0.70999999999999996},"min":{"1min":0.36699999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":2.2509999999999999},"last":0.74199999999999999}]},{"osd":1,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65400000000000003,"15min":0.71199999999999997},"min":{"1min":0.433,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.94099999999999995,"5min":0.96099999999999997,"15min":2.3849999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66200000000000003,"15min":0.74299999999999999},"min":{"1min":0.46500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.94499999999999995,"5min":1.079,"15min":2.3130000000000002},"last":0.878}]},{"osd":2,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67900000000000005,"15min":0.76200000000000001},"min":{"1min":0.45900000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.93100000000000005,"5min":0.998,"15min":2.4590000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.67000000000000004,"15min":0.73599999999999999},"min":{"1min":0.42199999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.022,"5min":1.022,"15min":2.411},"last":0.82199999999999995}]},{"osd":3,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65900000000000003,"15min":0.749},"min":{"1min":0.504,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90700000000000003,"5min":0.92000000000000004,"15min":2.2730000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.71399999999999997,"15min":0.79200000000000004},"min":{"1min":0.52900000000000003,"5min":0.432,"15min":0.432},"max":{"1min":1.0289999999999999,"5min":1.0389999999999999,"15min":2.3650000000000002},"last":0.84499999999999997}]},{"osd":4,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.42099999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.502},{"interface":"front","average":{"1min":0.63,"5min":0.58299999999999996,"15min":0.63500000000000001},"min":{"1min":0.442,"5min":0.315,"15min":0.315},"max":{"1min":0.96299999999999997,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.60799999999999998}]},{"osd":5,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63400000000000001,"15min":0.67200000000000004},"min":{"1min":0.44,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.65000000000000002,"15min":0.69599999999999995},"min":{"1min":0.48099999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68700000000000006}]},{"osd":6,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.69499999999999995,"15min":0.73899999999999999},"min":{"1min":0.47799999999999998,"5min":0.375,"15min":0.375},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.63700000000000001,"15min":0.71899999999999997},"min":{"1min":0.41299999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.71799999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40728,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331112,"statfs":{"total":96636764160,"available":96595058688,"internally_reserved":0,"allocated":1253376,"data_stored":578536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.65900000000000003,"15min":0.64300000000000002},"min":{"1min":0.433,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70899999999999996,"15min":0.72899999999999998},"min":{"1min":0.51300000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.98799999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.624}]},{"osd":1,"last update":"Tue Nov 28 18:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64800000000000002,"15min":0.78300000000000003},"min":{"1min":0.36599999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84999999999999998,"5min":0.96899999999999997,"15min":4.2389999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.77600000000000002},"min":{"1min":0.46600000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.91200000000000003,"5min":0.94499999999999995,"15min":4.218},"last":0.56799999999999995}]},{"osd":2,"last update":"Tue Nov 28 18:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66300000000000003,"15min":0.70799999999999996},"min":{"1min":0.41199999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.67900000000000005,"15min":0.70099999999999996},"min":{"1min":0.54900000000000004,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92100000000000004,"5min":1.012,"15min":1.012},"last":0.754}]},{"osd":3,"last update":"Tue Nov 28 18:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.69299999999999995,"15min":0.72299999999999998},"min":{"1min":0.59899999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.67500000000000004,"15min":0.68500000000000005},"min":{"1min":0.47699999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.84099999999999997,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.83299999999999996}]},{"osd":4,"last update":"Tue Nov 28 18:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61399999999999999,"15min":0.65900000000000003},"min":{"1min":0.376,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.95599999999999996,"5min":1.02,"15min":1.02},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.624,"5min":0.61799999999999999,"15min":0.67700000000000005},"min":{"1min":0.41599999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.90400000000000003,"5min":1.038,"15min":1.038},"last":0.77000000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61599999999999999,"15min":0.67000000000000004},"min":{"1min":0.47699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81999999999999995,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67600000000000005,"15min":0.746},"min":{"1min":0.443,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":1.0449999999999999},"last":0.85299999999999998}]},{"osd":7,"last update":"Tue Nov 28 18:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68899999999999995,"15min":0.75900000000000001},"min":{"1min":0.52200000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92300000000000004,"5min":1.002,"15min":1.0269999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67900000000000005,"15min":0.73699999999999999},"min":{"1min":0.46200000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78800000000000003,"5min":1,"15min":1},"last":0.86799999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1253376,"data_stored":578536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.68700000000000006,"15min":0.70899999999999996},"min":{"1min":0.35299999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.88800000000000001,"5min":2.1299999999999999,"15min":2.8929999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69999999999999996,"15min":0.746},"min":{"1min":0.39200000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92100000000000004,"5min":0.97799999999999998,"15min":2.9140000000000001},"last":0.50600000000000001}]},{"osd":1,"last update":"Tue Nov 28 18:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.67700000000000005,"15min":0.73199999999999998},"min":{"1min":0.42799999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90100000000000002,"5min":0.93000000000000005,"15min":3.4809999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.70299999999999996,"15min":0.78700000000000003},"min":{"1min":0.54000000000000004,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93700000000000006,"5min":2.2109999999999999,"15min":3.4580000000000002},"last":0.67200000000000004}]},{"osd":2,"last update":"Tue Nov 28 18:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65300000000000002,"15min":0.74099999999999999},"min":{"1min":0.48699999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84299999999999997,"5min":0.93200000000000005,"15min":3.9510000000000001},"last":0.626},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67400000000000004,"15min":0.76000000000000001},"min":{"1min":0.56799999999999995,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95999999999999996,"5min":1.014,"15min":4.0149999999999997},"last":0.48699999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.70499999999999996,"15min":0.80900000000000005},"min":{"1min":0.52600000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84699999999999998,"5min":2.254,"15min":4.0869999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68000000000000005,"15min":0.80400000000000005},"min":{"1min":0.48799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84199999999999997,"5min":0.98599999999999999,"15min":3.9900000000000002},"last":0.68899999999999995}]},{"osd":4,"last update":"Tue Nov 28 18:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.35999999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.90000000000000002,"5min":1.696,"15min":1.696},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68000000000000005,"15min":0.70599999999999996},"min":{"1min":0.47899999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.84099999999999997,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.52700000000000002}]},{"osd":6,"last update":"Tue Nov 28 18:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66400000000000003,"15min":0.67600000000000005},"min":{"1min":0.33900000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.93700000000000006,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.69399999999999995,"15min":0.71399999999999997},"min":{"1min":0.46700000000000003,"5min":0.438,"15min":0.42399999999999999},"max":{"1min":0.89200000000000002,"5min":1.262,"15min":1.756},"last":0.54900000000000004}]},{"osd":7,"last update":"Tue Nov 28 18:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.72999999999999998,"15min":0.75},"min":{"1min":0.42999999999999999,"5min":0.42399999999999999,"15min":0.41699999999999998},"max":{"1min":0.93400000000000005,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.73399999999999999,"15min":0.752},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.96299999999999997,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.73599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72664,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299176,"statfs":{"total":96636764160,"available":96562356224,"internally_reserved":0,"allocated":1253376,"data_stored":578536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.628,"15min":0.68000000000000005},"min":{"1min":0.36599999999999999,"5min":0.32900000000000001,"15min":0.315},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":2.363},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64400000000000002,"15min":0.64700000000000002},"min":{"1min":0.52900000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92300000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.71299999999999997}]},{"osd":1,"last update":"Tue Nov 28 18:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64300000000000002,"15min":0.627},"min":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.98599999999999999,"5min":1.026,"15min":1.0629999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68200000000000005,"15min":0.66000000000000003},"min":{"1min":0.45000000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.64800000000000002}]},{"osd":2,"last update":"Tue Nov 28 18:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72099999999999997,"15min":0.70199999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.72199999999999998,"15min":0.72999999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.091,"5min":1.091,"15min":1.871},"last":0.73299999999999998}]},{"osd":3,"last update":"Tue Nov 28 18:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66400000000000003,"15min":0.65200000000000002},"min":{"1min":0.45700000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89600000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68300000000000005,"15min":0.68500000000000005},"min":{"1min":0.47799999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.748}]},{"osd":5,"last update":"Tue Nov 28 18:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60699999999999998,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.315,"15min":0.315},"max":{"1min":1.03,"5min":1.03,"15min":1.1799999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.59899999999999998,"15min":0.63300000000000001},"min":{"1min":0.39100000000000001,"5min":0.33400000000000002,"15min":0.32300000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.79},"last":0.56399999999999995}]},{"osd":6,"last update":"Tue Nov 28 18:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.624,"15min":0.64500000000000002},"min":{"1min":0.433,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.123},"last":0.629},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.69599999999999995,"15min":0.70899999999999996},"min":{"1min":0.57999999999999996,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.925},"last":0.76200000000000001}]},{"osd":7,"last update":"Tue Nov 28 18:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.68400000000000005,"15min":0.71399999999999997},"min":{"1min":0.58699999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.843},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.755,"15min":0.77700000000000002},"min":{"1min":0.54600000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.593},"last":0.88100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574932,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62936,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308904,"statfs":{"total":96636764160,"available":96572317696,"internally_reserved":0,"allocated":1253376,"data_stored":578536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.441,"15min":0.41899999999999998},"min":{"1min":0.255,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.622,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.48799999999999999,"15min":0.47399999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.65600000000000003,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.626}]},{"osd":1,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.45100000000000001,"15min":0.46100000000000002},"min":{"1min":0.29599999999999999,"5min":0.28799999999999998,"15min":0.27300000000000002},"max":{"1min":0.68999999999999995,"5min":0.73399999999999999,"15min":0.96099999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.54300000000000004,"15min":0.51500000000000001},"min":{"1min":0.29699999999999999,"5min":0.249,"15min":0.247},"max":{"1min":0.79500000000000004,"5min":4.5599999999999996,"15min":4.5599999999999996},"last":0.50900000000000001}]},{"osd":2,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.54200000000000004,"15min":0.52800000000000002},"min":{"1min":0.38200000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74299999999999999,"5min":2.48,"15min":2.48},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92800000000000005,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.58699999999999997}]},{"osd":4,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.88800000000000001,"15min":0.81299999999999994},"min":{"1min":0.56000000000000005,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97599999999999998,"5min":12.695,"15min":12.695},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.79500000000000004,"15min":0.78200000000000003},"min":{"1min":0.51600000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.875,"5min":6.3250000000000002,"15min":6.3250000000000002},"last":0.872}]},{"osd":5,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76400000000000001,"15min":0.81999999999999995},"min":{"1min":0.54700000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95099999999999996,"5min":1.9430000000000001,"15min":2.5019999999999998},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.79000000000000004,"15min":0.77900000000000003},"min":{"1min":0.52100000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97399999999999998,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.92800000000000005}]},{"osd":6,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.82299999999999995,"15min":0.81100000000000005},"min":{"1min":0.54000000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.91200000000000003,"5min":6.1790000000000003,"15min":6.1790000000000003},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78600000000000003,"15min":0.79900000000000004},"min":{"1min":0.504,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.022,"5min":1.984,"15min":1.984},"last":0.85599999999999998}]},{"osd":7,"last update":"Tue Nov 28 18:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.79800000000000004,"15min":0.81399999999999995},"min":{"1min":0.53100000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94299999999999995,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.83099999999999996,"15min":0.81899999999999995},"min":{"1min":0.55600000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97599999999999998,"5min":6.2249999999999996,"15min":6.2249999999999996},"last":1.002}]}]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1253376,"data_stored":578536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:29:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.45700000000000002,"15min":0.49299999999999999},"min":{"1min":0.27300000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.629,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.44900000000000001,"15min":0.495},"min":{"1min":0.26700000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63300000000000001,"5min":0.66800000000000004,"15min":2.0150000000000001},"last":0.29399999999999998}]},{"osd":1,"last update":"Tue Nov 28 18:29:29 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.48799999999999999,"15min":0.51900000000000002},"min":{"1min":0.38400000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67500000000000004,"5min":1.48,"15min":1.48},"last":0.38},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.57299999999999995,"15min":0.53600000000000003},"min":{"1min":0.36699999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69099999999999995,"5min":8.3539999999999992,"15min":8.3539999999999992},"last":0.436}]},{"osd":3,"last update":"Tue Nov 28 18:29:29 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.48999999999999999,"15min":0.503},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76900000000000002,"5min":3.1549999999999998,"15min":3.1549999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55000000000000004,"15min":0.56699999999999995},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.754,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":0.49399999999999999}]},{"osd":4,"last update":"Tue Nov 28 18:29:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.75800000000000001,"15min":0.77000000000000002},"min":{"1min":0.55100000000000005,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92900000000000005,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.74199999999999999,"15min":0.77300000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.032,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.58999999999999997}]},{"osd":5,"last update":"Tue Nov 28 18:29:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.73999999999999999,"15min":0.76400000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94099999999999995,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.74299999999999999,"15min":0.94899999999999995},"min":{"1min":0.496,"5min":0.495,"15min":0.495},"max":{"1min":0.95599999999999996,"5min":1.5449999999999999,"15min":7.4820000000000002},"last":0.58299999999999996}]},{"osd":6,"last update":"Tue Nov 28 18:29:29 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.84299999999999997,"15min":0.81999999999999995},"min":{"1min":0.54700000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.95399999999999996,"5min":12.019,"15min":12.019},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.80600000000000005,"15min":0.86499999999999999},"min":{"1min":0.48999999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0149999999999999,"5min":8.7400000000000002,"15min":8.7400000000000002},"last":0.65200000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:29:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.83099999999999996,"15min":0.84599999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.98799999999999999,"5min":8.327,"15min":8.327},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.81599999999999995,"15min":0.90500000000000003},"min":{"1min":0.441,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97399999999999998,"5min":8.609,"15min":8.609},"last":0.56999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40724,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331116,"statfs":{"total":96636764160,"available":96595062784,"internally_reserved":0,"allocated":1253376,"data_stored":578536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40699999999999997,"15min":0.42299999999999999},"min":{"1min":0.29299999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.60799999999999998,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.46200000000000002,"15min":0.45500000000000002},"min":{"1min":0.371,"5min":0.27300000000000002,"15min":0.26600000000000001},"max":{"1min":0.66000000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.51500000000000001}]},{"osd":2,"last update":"Tue Nov 28 18:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.441,"15min":0.44700000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.75700000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48899999999999999,"15min":0.48099999999999998},"min":{"1min":0.35099999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.71499999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.56100000000000005}]},{"osd":3,"last update":"Tue Nov 28 18:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.42599999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.505,"15min":0.49299999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32200000000000001},"max":{"1min":0.73899999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54800000000000004}]},{"osd":4,"last update":"Tue Nov 28 18:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71599999999999997,"15min":0.74299999999999999},"min":{"1min":0.44,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0409999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.749,"15min":0.76700000000000002},"min":{"1min":0.56200000000000006,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0629999999999999,"5min":1.0780000000000001,"15min":1.155},"last":0.80200000000000005}]},{"osd":5,"last update":"Tue Nov 28 18:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70199999999999996,"15min":0.73999999999999999},"min":{"1min":0.504,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.997,"5min":0.997,"15min":1.3799999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70199999999999996,"15min":0.74099999999999999},"min":{"1min":0.56100000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96999999999999997,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.76800000000000002}]},{"osd":6,"last update":"Tue Nov 28 18:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.74299999999999999,"15min":0.77400000000000002},"min":{"1min":0.54800000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0349999999999999,"5min":1.1279999999999999,"15min":1.2969999999999999},"last":0.625},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.76100000000000001,"15min":0.80100000000000005},"min":{"1min":0.53300000000000003,"5min":0.498,"15min":0.498},"max":{"1min":1.028,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.71899999999999997}]},{"osd":7,"last update":"Tue Nov 28 18:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.72899999999999998,"15min":0.749},"min":{"1min":0.59599999999999997,"5min":0.51900000000000002,"15min":0.47899999999999998},"max":{"1min":0.98999999999999999,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.75},{"interface":"front","average":{"1min":0.746,"5min":0.73199999999999998,"15min":0.77000000000000002},"min":{"1min":0.53100000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0069999999999999,"5min":1.0109999999999999,"15min":1.4239999999999999},"last":0.78500000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574932,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37148,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334692,"statfs":{"total":96636764160,"available":96598724608,"internally_reserved":0,"allocated":1847296,"data_stored":1168904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 18:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41699999999999998,"15min":0.439},"min":{"1min":0.28100000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.28999999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.45300000000000001}]},{"osd":2,"last update":"Tue Nov 28 18:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.48099999999999998,"15min":0.503},"min":{"1min":0.318,"5min":0.30599999999999999,"15min":0.30499999999999999},"max":{"1min":0.59299999999999997,"5min":0.629,"15min":0.92900000000000005},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.48499999999999999,"15min":0.51000000000000001},"min":{"1min":0.38300000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63500000000000001,"5min":0.77800000000000002,"15min":0.91200000000000003},"last":0.40200000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47299999999999998,"15min":0.48699999999999999},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60199999999999998,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52400000000000002,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78500000000000003,"5min":0.80000000000000004,"15min":1.087},"last":0.502}]},{"osd":4,"last update":"Tue Nov 28 18:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.51900000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.94399999999999995,"5min":1.131,"15min":1.131},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71599999999999997,"15min":0.71799999999999997},"min":{"1min":0.53200000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88200000000000001,"5min":1.244,"15min":1.244},"last":0.72399999999999998}]},{"osd":5,"last update":"Tue Nov 28 18:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72399999999999998,"15min":0.72699999999999998},"min":{"1min":0.48399999999999999,"5min":0.436,"15min":0.436},"max":{"1min":0.92100000000000004,"5min":1.121,"15min":1.121},"last":0.625},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.86799999999999999,"5min":1.075,"15min":1.075},"last":0.745}]},{"osd":6,"last update":"Tue Nov 28 18:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72599999999999998,"15min":0.73199999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.93799999999999994,"5min":1.1140000000000001,"15min":1.407},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95599999999999996,"5min":1.087,"15min":1.089},"last":0.78800000000000003}]},{"osd":7,"last update":"Tue Nov 28 18:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.75600000000000001,"15min":0.754},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97099999999999997,"5min":1.212,"15min":1.212},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.74399999999999999,"15min":0.747},"min":{"1min":0.54900000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.84199999999999997,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.69999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:30:10.626 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T18:30:10.626 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-28T18:30:10.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.805+0000 7fecbef80640 1 Processor -- start 2023-11-28T18:30:10.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.805+0000 7fecbef80640 1 -- start start 2023-11-28T18:30:10.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.809+0000 7fecbef80640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fecb8071960 0x7fecb815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:10.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.809+0000 7fecbef80640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fecb815b030 0x7fecb80709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:10.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.809+0000 7fecbef80640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb80701d0 con 0x7fecb80b4290 2023-11-28T18:30:10.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.809+0000 7fecbef80640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb806df80 con 0x7fecb815b030 2023-11-28T18:30:10.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.809+0000 7fecbef80640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb8071df0 con 0x7fecb8071960 2023-11-28T18:30:10.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.809+0000 7fecbd51d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fecb815b030 0x7fecb80709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:10.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.809+0000 7fecbd51d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fecb815b030 0x7fecb80709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50200/0 (socket says 172.21.15.105:50200) 2023-11-28T18:30:10.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.809+0000 7fecbd51d640 1 -- 172.21.15.105:0/870854100 learned_addr learned my addr 172.21.15.105:0/870854100 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:10.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.809+0000 7fecaffff640 1 --2- 172.21.15.105:0/870854100 >> v2:172.21.15.148:3300/0 conn(0x7fecb8071960 0x7fecb815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:10.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.809+0000 7fecbd51d640 1 -- 172.21.15.105:0/870854100 >> v2:172.21.15.148:3300/0 conn(0x7fecb8071960 msgr2=0x7fecb815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:10.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.809+0000 7fecbd51d640 1 --2- 172.21.15.105:0/870854100 >> v2:172.21.15.148:3300/0 conn(0x7fecb8071960 0x7fecb815f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.809+0000 7fecbd51d640 1 -- 172.21.15.105:0/870854100 >> v1:172.21.15.148:6789/0 conn(0x7fecb80b4290 legacy=0x7fecb80b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:10.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.809+0000 7fecbd51d640 1 -- 172.21.15.105:0/870854100 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb80720d0 con 0x7fecb815b030 2023-11-28T18:30:10.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.813+0000 7fecbd51d640 1 --2- 172.21.15.105:0/870854100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb815b030 0x7fecb80709b0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fecb806e520 tx=0x7fecb001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b1869e47361960 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:10.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.813+0000 7fecaf7fe640 1 -- 172.21.15.105:0/870854100 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb00a5730 con 0x7fecb815b030 2023-11-28T18:30:10.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.813+0000 7fecaf7fe640 1 -- 172.21.15.105:0/870854100 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb00a5890 con 0x7fecb815b030 2023-11-28T18:30:10.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.813+0000 7fecaf7fe640 1 -- 172.21.15.105:0/870854100 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb00a5c20 con 0x7fecb815b030 2023-11-28T18:30:10.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.813+0000 7fecbef80640 1 -- 172.21.15.105:0/870854100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb815b030 msgr2=0x7fecb80709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:10.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.813+0000 7fecbef80640 1 --2- 172.21.15.105:0/870854100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb815b030 0x7fecb80709b0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fecb806e520 tx=0x7fecb001c960 comp rx=0 tx=0).stop 2023-11-28T18:30:10.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.813+0000 7fecbef80640 1 -- 172.21.15.105:0/870854100 shutdown_connections 2023-11-28T18:30:10.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.813+0000 7fecbef80640 1 --2- 172.21.15.105:0/870854100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb815b030 0x7fecb80709b0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.813+0000 7fecbef80640 1 --2- 172.21.15.105:0/870854100 >> v2:172.21.15.148:3300/0 conn(0x7fecb8071960 0x7fecb815f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.813+0000 7fecbef80640 1 -- 172.21.15.105:0/870854100 >> 172.21.15.105:0/870854100 conn(0x7fecb80ceb90 msgr2=0x7fecb80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:10.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.813+0000 7fecbef80640 1 -- 172.21.15.105:0/870854100 shutdown_connections 2023-11-28T18:30:10.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.813+0000 7fecbef80640 1 -- 172.21.15.105:0/870854100 wait complete. 2023-11-28T18:30:10.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.813+0000 7fecbef80640 1 Processor -- start 2023-11-28T18:30:10.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.813+0000 7fecbef80640 1 -- start start 2023-11-28T18:30:10.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecbef80640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecb80b4290 0x7fecb8140f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:10.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecbef80640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecb815b030 0x7fecb81550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:10.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecbef80640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb81555f0 0x7fecb809aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:10.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecbef80640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb809c330 con 0x7fecb81555f0 2023-11-28T18:30:10.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecbef80640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb8154bc0 con 0x7fecb815b030 2023-11-28T18:30:10.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecbef80640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb8155a80 con 0x7fecb80b4290 2023-11-28T18:30:10.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecbd51d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb81555f0 0x7fecb809aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:10.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecbd51d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb81555f0 0x7fecb809aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50208/0 (socket says 172.21.15.105:50208) 2023-11-28T18:30:10.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecbd51d640 1 -- 172.21.15.105:0/556869280 learned_addr learned my addr 172.21.15.105:0/556869280 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:10.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecbcd1c640 1 --2- 172.21.15.105:0/556869280 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecb80b4290 0x7fecb8140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:10.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecaffff640 1 --2- 172.21.15.105:0/556869280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecb815b030 0x7fecb81550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:10.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecbd51d640 1 -- 172.21.15.105:0/556869280 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecb80b4290 msgr2=0x7fecb8140f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:10.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecbd51d640 1 --2- 172.21.15.105:0/556869280 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecb80b4290 0x7fecb8140f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecbd51d640 1 -- 172.21.15.105:0/556869280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecb815b030 msgr2=0x7fecb81550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:10.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecbd51d640 1 --2- 172.21.15.105:0/556869280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecb815b030 0x7fecb81550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:10.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecbd51d640 1 -- 172.21.15.105:0/556869280 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb0009b30 con 0x7fecb81555f0 2023-11-28T18:30:10.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecaffff640 1 --2- 172.21.15.105:0/556869280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecb815b030 0x7fecb81550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:10.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecbd51d640 1 --2- 172.21.15.105:0/556869280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb81555f0 0x7fecb809aaa0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fecb001a590 tx=0x7fecb0019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:10.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecad7fa640 1 -- 172.21.15.105:0/556869280 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb00b8670 con 0x7fecb81555f0 2023-11-28T18:30:10.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecad7fa640 1 -- 172.21.15.105:0/556869280 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb00b87d0 con 0x7fecb81555f0 2023-11-28T18:30:10.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecad7fa640 1 -- 172.21.15.105:0/556869280 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb00b8b80 con 0x7fecb81555f0 2023-11-28T18:30:10.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecbef80640 1 -- 172.21.15.105:0/556869280 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb8141480 con 0x7fecb81555f0 2023-11-28T18:30:10.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecbef80640 1 -- 172.21.15.105:0/556869280 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb809b990 con 0x7fecb81555f0 2023-11-28T18:30:10.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecbef80640 1 -- 172.21.15.105:0/556869280 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb815f0d0 con 0x7fecb81555f0 2023-11-28T18:30:10.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.817+0000 7fecad7fa640 1 -- 172.21.15.105:0/556869280 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fecb00b8ce0 con 0x7fecb81555f0 2023-11-28T18:30:10.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.821+0000 7fecad7fa640 1 --2- 172.21.15.105:0/556869280 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feca40668d0 0x7feca4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:10.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.821+0000 7fecad7fa640 1 -- 172.21.15.105:0/556869280 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fecb011fb60 con 0x7fecb81555f0 2023-11-28T18:30:10.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.821+0000 7fecbcd1c640 1 --2- 172.21.15.105:0/556869280 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feca40668d0 0x7feca4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:10.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.821+0000 7fecbcd1c640 1 --2- 172.21.15.105:0/556869280 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feca40668d0 0x7feca4068d90 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fecb4007fd0 tx=0x7fecb4007e40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:10.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:10.825+0000 7fecad7fa640 1 -- 172.21.15.105:0/556869280 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fecb00ecce0 con 0x7fecb81555f0 2023-11-28T18:30:11.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.025+0000 7fecbef80640 1 -- 172.21.15.105:0/556869280 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fecb815f0d0 con 0x7fecb81555f0 2023-11-28T18:30:11.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.029+0000 7fecad7fa640 1 -- 172.21.15.105:0/556869280 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7fecb00f0300 con 0x7fecb81555f0 2023-11-28T18:30:11.028 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2023-11-28T18:30:11.028 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDpL2ZlSt0VMxAAQ8+j1BgBPMJ2cuiZj8aG0w== 2023-11-28T18:30:11.028 INFO:teuthology.orchestra.run.smithi105.stdout: caps mds = "allow" 2023-11-28T18:30:11.028 INFO:teuthology.orchestra.run.smithi105.stdout: caps mon = "allow r" 2023-11-28T18:30:11.029 INFO:teuthology.orchestra.run.smithi105.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-28T18:30:11.029 INFO:teuthology.orchestra.run.smithi105.stderr:updated caps for client.0 2023-11-28T18:30:11.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.029+0000 7fecbef80640 1 -- 172.21.15.105:0/556869280 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feca40668d0 msgr2=0x7feca4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:11.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.029+0000 7fecbef80640 1 --2- 172.21.15.105:0/556869280 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feca40668d0 0x7feca4068d90 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fecb4007fd0 tx=0x7fecb4007e40 comp rx=0 tx=0).stop 2023-11-28T18:30:11.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.029+0000 7fecbef80640 1 -- 172.21.15.105:0/556869280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb81555f0 msgr2=0x7fecb809aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:11.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.029+0000 7fecbef80640 1 --2- 172.21.15.105:0/556869280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb81555f0 0x7fecb809aaa0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fecb001a590 tx=0x7fecb0019700 comp rx=0 tx=0).stop 2023-11-28T18:30:11.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.029+0000 7fecbef80640 1 -- 172.21.15.105:0/556869280 shutdown_connections 2023-11-28T18:30:11.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.029+0000 7fecbef80640 1 --2- 172.21.15.105:0/556869280 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feca40668d0 0x7feca4068d90 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.029+0000 7fecbef80640 1 --2- 172.21.15.105:0/556869280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb81555f0 0x7fecb809aaa0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.029+0000 7fecbef80640 1 --2- 172.21.15.105:0/556869280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecb815b030 0x7fecb81550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.029+0000 7fecbef80640 1 --2- 172.21.15.105:0/556869280 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecb80b4290 0x7fecb8140f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.029+0000 7fecbef80640 1 -- 172.21.15.105:0/556869280 >> 172.21.15.105:0/556869280 conn(0x7fecb80ceb90 msgr2=0x7fecb80b39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:11.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.029+0000 7fecbef80640 1 -- 172.21.15.105:0/556869280 shutdown_connections 2023-11-28T18:30:11.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.029+0000 7fecbef80640 1 -- 172.21.15.105:0/556869280 wait complete. 2023-11-28T18:30:11.049 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:30:11.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.189+0000 7fa8d239d640 1 Processor -- start 2023-11-28T18:30:11.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.189+0000 7fa8d239d640 1 -- start start 2023-11-28T18:30:11.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.189+0000 7fa8d239d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8cc0d00b0 0x7fa8cc0cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:11.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.189+0000 7fa8d239d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8cc0ceb40 0x7fa8cc0b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.189+0000 7fa8d239d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8cc0d3e60 con 0x7fa8cc0b8a20 2023-11-28T18:30:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.189+0000 7fa8d239d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8cc0b5fe0 con 0x7fa8cc0d00b0 2023-11-28T18:30:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.189+0000 7fa8d239d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc0b62c0 con 0x7fa8cc0ceb40 2023-11-28T18:30:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.189+0000 7fa8cbfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa8cc0b8a20 0x7fa8cc0cf9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39034/0 (socket says 172.21.15.105:39034) 2023-11-28T18:30:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.189+0000 7fa8cbfff640 1 -- 172.21.15.105:0/3864158328 learned_addr learned my addr 172.21.15.105:0/3864158328 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.189+0000 7fa8d093a640 1 --2- 172.21.15.105:0/3864158328 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8cc0ceb40 0x7fa8cc0b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.189+0000 7fa8cb7fe640 1 --2- 172.21.15.105:0/3864158328 >> v2:172.21.15.148:3300/0 conn(0x7fa8cc0d00b0 0x7fa8cc0cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.189+0000 7fa8caffd640 1 -- 172.21.15.105:0/3864158328 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 776259440 0 0) 0x7fa8cc0d3e60 con 0x7fa8cc0b8a20 2023-11-28T18:30:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.189+0000 7fa8caffd640 1 -- 172.21.15.105:0/3864158328 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8b4003530 con 0x7fa8cc0b8a20 2023-11-28T18:30:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.189+0000 7fa8caffd640 1 -- 172.21.15.105:0/3864158328 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 751241049 0 0) 0x7fa8b4003530 con 0x7fa8cc0b8a20 2023-11-28T18:30:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.189+0000 7fa8caffd640 1 -- 172.21.15.105:0/3864158328 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8cc0ceb40 msgr2=0x7fa8cc0b6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.189+0000 7fa8caffd640 1 --2- 172.21.15.105:0/3864158328 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8cc0ceb40 0x7fa8cc0b6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.189+0000 7fa8caffd640 1 -- 172.21.15.105:0/3864158328 >> v2:172.21.15.148:3300/0 conn(0x7fa8cc0d00b0 msgr2=0x7fa8cc0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.189+0000 7fa8caffd640 1 --2- 172.21.15.105:0/3864158328 >> v2:172.21.15.148:3300/0 conn(0x7fa8cc0d00b0 0x7fa8cc0cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.189+0000 7fa8caffd640 1 -- 172.21.15.105:0/3864158328 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8cc0b65a0 con 0x7fa8cc0b8a20 2023-11-28T18:30:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.189+0000 7fa8caffd640 1 -- 172.21.15.105:0/3864158328 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fa8bc001680 con 0x7fa8cc0b8a20 2023-11-28T18:30:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.189+0000 7fa8caffd640 1 -- 172.21.15.105:0/3864158328 >> v1:172.21.15.105:6789/0 conn(0x7fa8cc0b8a20 legacy=0x7fa8cc0cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.189+0000 7fa8caffd640 1 --2- 172.21.15.105:0/3864158328 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8b4003980 0x7fa8b4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.193+0000 7fa8caffd640 1 --2- 172.21.15.105:0/3864158328 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8cc0ceb40 0x7fa8b40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.193+0000 7fa8caffd640 1 --2- 172.21.15.105:0/3864158328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b4004d10 0x7fa8b4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.193+0000 7fa8caffd640 1 -- 172.21.15.105:0/3864158328 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc0b5fe0 con 0x7fa8b4003980 2023-11-28T18:30:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.193+0000 7fa8caffd640 1 -- 172.21.15.105:0/3864158328 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc0b62c0 con 0x7fa8b4004d10 2023-11-28T18:30:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.193+0000 7fa8caffd640 1 -- 172.21.15.105:0/3864158328 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b400b990 con 0x7fa8cc0ceb40 2023-11-28T18:30:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.193+0000 7fa8cbfff640 1 --2- 172.21.15.105:0/3864158328 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8b4003980 0x7fa8b4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.193+0000 7fa8cb7fe640 1 --2- 172.21.15.105:0/3864158328 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8cc0ceb40 0x7fa8b40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.193+0000 7fa8d093a640 1 --2- 172.21.15.105:0/3864158328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b4004d10 0x7fa8b4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.193+0000 7fa8cbfff640 1 -- 172.21.15.105:0/3864158328 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8cc0ceb40 msgr2=0x7fa8b40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.193+0000 7fa8cbfff640 1 --2- 172.21.15.105:0/3864158328 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8cc0ceb40 0x7fa8b40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.193+0000 7fa8cbfff640 1 -- 172.21.15.105:0/3864158328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b4004d10 msgr2=0x7fa8b4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.193+0000 7fa8cbfff640 1 --2- 172.21.15.105:0/3864158328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b4004d10 0x7fa8b4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.193+0000 7fa8cbfff640 1 -- 172.21.15.105:0/3864158328 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b4003530 con 0x7fa8b4003980 2023-11-28T18:30:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.193+0000 7fa8d093a640 1 --2- 172.21.15.105:0/3864158328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b4004d10 0x7fa8b4005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:11.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.193+0000 7fa8cbfff640 1 --2- 172.21.15.105:0/3864158328 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8b4003980 0x7fa8b4003da0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00c240 tx=0x7fa8bc01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd25203cac63cf2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:11.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.193+0000 7fa8caffd640 1 -- 172.21.15.105:0/3864158328 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc0058e0 con 0x7fa8b4003980 2023-11-28T18:30:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.193+0000 7fa8caffd640 1 -- 172.21.15.105:0/3864158328 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8bc005a40 con 0x7fa8b4003980 2023-11-28T18:30:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.193+0000 7fa8caffd640 1 -- 172.21.15.105:0/3864158328 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc005e10 con 0x7fa8b4003980 2023-11-28T18:30:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8d239d640 1 -- 172.21.15.105:0/3864158328 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8b4003980 msgr2=0x7fa8b4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8d239d640 1 --2- 172.21.15.105:0/3864158328 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8b4003980 0x7fa8b4003da0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00c240 tx=0x7fa8bc01cc30 comp rx=0 tx=0).stop 2023-11-28T18:30:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8d093a640 1 -- 172.21.15.105:0/3864158328 reap_dead start 2023-11-28T18:30:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8d239d640 1 -- 172.21.15.105:0/3864158328 shutdown_connections 2023-11-28T18:30:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8d239d640 1 -- 172.21.15.105:0/3864158328 >> 172.21.15.105:0/3864158328 conn(0x7fa8cc09c340 msgr2=0x7fa8cc0d2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8d239d640 1 -- 172.21.15.105:0/3864158328 shutdown_connections 2023-11-28T18:30:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8d239d640 1 -- 172.21.15.105:0/3864158328 wait complete. 2023-11-28T18:30:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8d239d640 1 Processor -- start 2023-11-28T18:30:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8d239d640 1 -- start start 2023-11-28T18:30:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8d239d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b4003980 0x7fa8cc0ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8d239d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8cc15e940 0x7fa8cc1522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8d239d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8cc06c170 0x7fa8cc0c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8d239d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc15ee20 con 0x7fa8cc15e940 2023-11-28T18:30:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8d239d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc15f100 con 0x7fa8b4003980 2023-11-28T18:30:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8d239d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc15f3e0 con 0x7fa8cc06c170 2023-11-28T18:30:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8cb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8cc15e940 0x7fa8cc1522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8cb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8cc15e940 0x7fa8cc1522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46608/0 (socket says 172.21.15.105:46608) 2023-11-28T18:30:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8cb7fe640 1 -- 172.21.15.105:0/3881600467 learned_addr learned my addr 172.21.15.105:0/3881600467 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8cbfff640 1 --2- 172.21.15.105:0/3881600467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b4003980 0x7fa8cc0ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8d093a640 1 --2- 172.21.15.105:0/3881600467 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8cc06c170 0x7fa8cc0c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8cb7fe640 1 -- 172.21.15.105:0/3881600467 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8cc06c170 msgr2=0x7fa8cc0c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8cb7fe640 1 --2- 172.21.15.105:0/3881600467 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8cc06c170 0x7fa8cc0c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8cb7fe640 1 -- 172.21.15.105:0/3881600467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b4003980 msgr2=0x7fa8cc0ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.197+0000 7fa8cb7fe640 1 --2- 172.21.15.105:0/3881600467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b4003980 0x7fa8cc0ca0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.201+0000 7fa8cb7fe640 1 -- 172.21.15.105:0/3881600467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8cc15f6c0 con 0x7fa8cc15e940 2023-11-28T18:30:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.201+0000 7fa8cb7fe640 1 --2- 172.21.15.105:0/3881600467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8cc15e940 0x7fa8cc1522b0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0011aa0 tx=0x7fa8c0011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:11.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.201+0000 7fa8c8ff9640 1 -- 172.21.15.105:0/3881600467 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c000fb80 con 0x7fa8cc15e940 2023-11-28T18:30:11.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.201+0000 7fa8c8ff9640 1 -- 172.21.15.105:0/3881600467 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c000fce0 con 0x7fa8cc15e940 2023-11-28T18:30:11.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.201+0000 7fa8d239d640 1 -- 172.21.15.105:0/3881600467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8cc15f9a0 con 0x7fa8cc15e940 2023-11-28T18:30:11.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.201+0000 7fa8c8ff9640 1 -- 172.21.15.105:0/3881600467 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0014680 con 0x7fa8cc15e940 2023-11-28T18:30:11.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.201+0000 7fa8d239d640 1 -- 172.21.15.105:0/3881600467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8cc06c620 con 0x7fa8cc15e940 2023-11-28T18:30:11.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.201+0000 7fa8d239d640 1 -- 172.21.15.105:0/3881600467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8cc067570 con 0x7fa8cc15e940 2023-11-28T18:30:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.201+0000 7fa8c8ff9640 1 -- 172.21.15.105:0/3881600467 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa8c00147e0 con 0x7fa8cc15e940 2023-11-28T18:30:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.201+0000 7fa8c8ff9640 1 --2- 172.21.15.105:0/3881600467 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa8b8066900 0x7fa8b8068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.201+0000 7fa8c8ff9640 1 -- 172.21.15.105:0/3881600467 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa8c0084d70 con 0x7fa8cc15e940 2023-11-28T18:30:11.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.205+0000 7fa8cbfff640 1 --2- 172.21.15.105:0/3881600467 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa8b8066900 0x7fa8b8068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:11.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.205+0000 7fa8cbfff640 1 --2- 172.21.15.105:0/3881600467 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa8b8066900 0x7fa8b8068dc0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0aa000 tx=0x7fa8bc001a80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:11.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.209+0000 7fa8c8ff9640 1 -- 172.21.15.105:0/3881600467 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa8c0051f00 con 0x7fa8cc15e940 2023-11-28T18:30:11.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.393+0000 7fa8d239d640 1 -- 172.21.15.105:0/3881600467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8cc15da10 con 0x7fa8cc15e940 2023-11-28T18:30:11.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.397+0000 7fa8c8ff9640 1 -- 172.21.15.105:0/3881600467 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4929 (secure 0 0 0) 0x7fa8c0022020 con 0x7fa8cc15e940 2023-11-28T18:30:11.396 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:11.396 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":103,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/2603493310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2603493310},{"type":"v1","addr":"172.21.15.148:6834","nonce":2603493310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":6803,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3664281230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":3664281230},{"type":"v1","addr":"172.21.15.148:6835","nonce":3664281230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":8142,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3968059262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3968059262},{"type":"v1","addr":"172.21.15.105:6836","nonce":3968059262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:30:05.525117+0000","modified":"2023-11-28T18:30:08.567624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8145":{"gid":8145,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2782729992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2782729992},{"type":"v1","addr":"172.21.15.105:6837","nonce":2782729992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-11-28T18:30:11.396 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 103 2023-11-28T18:30:11.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.397+0000 7fa8d239d640 1 -- 172.21.15.105:0/3881600467 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa8b8066900 msgr2=0x7fa8b8068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:11.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.397+0000 7fa8d239d640 1 --2- 172.21.15.105:0/3881600467 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa8b8066900 0x7fa8b8068dc0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0aa000 tx=0x7fa8bc001a80 comp rx=0 tx=0).stop 2023-11-28T18:30:11.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.397+0000 7fa8d239d640 1 -- 172.21.15.105:0/3881600467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8cc15e940 msgr2=0x7fa8cc1522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:11.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.397+0000 7fa8d239d640 1 --2- 172.21.15.105:0/3881600467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8cc15e940 0x7fa8cc1522b0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0011aa0 tx=0x7fa8c0011f60 comp rx=0 tx=0).stop 2023-11-28T18:30:11.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.397+0000 7fa8d239d640 1 -- 172.21.15.105:0/3881600467 shutdown_connections 2023-11-28T18:30:11.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.397+0000 7fa8d239d640 1 --2- 172.21.15.105:0/3881600467 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8cc06c170 0x7fa8cc0c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.397+0000 7fa8d239d640 1 --2- 172.21.15.105:0/3881600467 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa8b8066900 0x7fa8b8068dc0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.397+0000 7fa8d239d640 1 --2- 172.21.15.105:0/3881600467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8cc15e940 0x7fa8cc1522b0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.397+0000 7fa8d239d640 1 --2- 172.21.15.105:0/3881600467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b4003980 0x7fa8cc0ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.397+0000 7fa8d239d640 1 -- 172.21.15.105:0/3881600467 >> 172.21.15.105:0/3881600467 conn(0x7fa8cc09c340 msgr2=0x7fa8cc0b7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:11.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.397+0000 7fa8d239d640 1 -- 172.21.15.105:0/3881600467 shutdown_connections 2023-11-28T18:30:11.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.397+0000 7fa8d239d640 1 -- 172.21.15.105:0/3881600467 wait complete. 2023-11-28T18:30:11.820 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:30:05.525117+0000', 'modified': '2023-11-28T18:30:08.567624+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8145}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8145': {'gid': 8145, 'name': 'c', 'rank': 0, 'incarnation': 99, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6837/2782729992', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6835', 'nonce': 2782729992}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 2782729992}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:30:11.820 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T18:30:11.820 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T18:30:11.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.961+0000 7f56cb8fe640 1 Processor -- start 2023-11-28T18:30:11.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.961+0000 7f56cb8fe640 1 -- start start 2023-11-28T18:30:11.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.965+0000 7f56cb8fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f56c40ad9f0 0x7f56c40ade80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:11.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.965+0000 7f56cb8fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f56c40cc7d0 0x7f56c40ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:11.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.965+0000 7f56cb8fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c4162820 con 0x7f56c406d610 2023-11-28T18:30:11.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.965+0000 7f56cb8fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c40ae3c0 con 0x7f56c40ad9f0 2023-11-28T18:30:11.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.965+0000 7f56cb8fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c4161c10 con 0x7f56c40cc7d0 2023-11-28T18:30:11.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.965+0000 7f56c8e99640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f56c40ad9f0 0x7f56c40ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:11.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.965+0000 7f56c969a640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f56c406d610 0x7f56c40cdff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39050/0 (socket says 172.21.15.105:39050) 2023-11-28T18:30:11.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.965+0000 7f56c969a640 1 -- 172.21.15.105:0/1907895641 learned_addr learned my addr 172.21.15.105:0/1907895641 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:11.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.965+0000 7f56c8e99640 1 -- 172.21.15.105:0/1907895641 >> v2:172.21.15.148:3300/0 conn(0x7f56c40cc7d0 msgr2=0x7f56c40ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:11.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.965+0000 7f56c9e9b640 1 --2- 172.21.15.105:0/1907895641 >> v2:172.21.15.148:3300/0 conn(0x7f56c40cc7d0 0x7f56c40ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:11.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.965+0000 7f56bbfff640 1 -- 172.21.15.105:0/1907895641 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799057820 0 0) 0x7f56c4162820 con 0x7f56c406d610 2023-11-28T18:30:11.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.965+0000 7f56c8e99640 1 --2- 172.21.15.105:0/1907895641 >> v2:172.21.15.148:3300/0 conn(0x7f56c40cc7d0 0x7f56c40ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.965+0000 7f56c8e99640 1 -- 172.21.15.105:0/1907895641 >> v1:172.21.15.105:6789/0 conn(0x7f56c406d610 legacy=0x7f56c40cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:11.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.965+0000 7f56c8e99640 1 -- 172.21.15.105:0/1907895641 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c4161ef0 con 0x7f56c40ad9f0 2023-11-28T18:30:11.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.965+0000 7f56c9e9b640 1 --2- 172.21.15.105:0/1907895641 >> v2:172.21.15.148:3300/0 conn(0x7f56c40cc7d0 0x7f56c40ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.965+0000 7f56c8e99640 1 --2- 172.21.15.105:0/1907895641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56c40ad9f0 0x7f56c40ade80 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f56c4067c50 tx=0x7f56b401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac05a1829b5f845 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.969+0000 7f56bbfff640 1 -- 172.21.15.105:0/1907895641 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b40ab600 con 0x7f56c40ad9f0 2023-11-28T18:30:11.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.969+0000 7f56bbfff640 1 -- 172.21.15.105:0/1907895641 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b40a7050 con 0x7f56c40ad9f0 2023-11-28T18:30:11.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.969+0000 7f56bbfff640 1 -- 172.21.15.105:0/1907895641 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b40ab980 con 0x7f56c40ad9f0 2023-11-28T18:30:11.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.969+0000 7f56cb8fe640 1 -- 172.21.15.105:0/1907895641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56c40ad9f0 msgr2=0x7f56c40ade80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:11.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.969+0000 7f56cb8fe640 1 --2- 172.21.15.105:0/1907895641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56c40ad9f0 0x7f56c40ade80 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f56c4067c50 tx=0x7f56b401c430 comp rx=0 tx=0).stop 2023-11-28T18:30:11.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.969+0000 7f56cb8fe640 1 -- 172.21.15.105:0/1907895641 shutdown_connections 2023-11-28T18:30:11.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.969+0000 7f56cb8fe640 1 --2- 172.21.15.105:0/1907895641 >> v2:172.21.15.148:3300/0 conn(0x7f56c40cc7d0 0x7f56c40ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.969+0000 7f56cb8fe640 1 --2- 172.21.15.105:0/1907895641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56c40ad9f0 0x7f56c40ade80 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.969+0000 7f56cb8fe640 1 -- 172.21.15.105:0/1907895641 >> 172.21.15.105:0/1907895641 conn(0x7f56c40ceb90 msgr2=0x7f56c40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:11.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.969+0000 7f56cb8fe640 1 -- 172.21.15.105:0/1907895641 shutdown_connections 2023-11-28T18:30:11.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.969+0000 7f56cb8fe640 1 -- 172.21.15.105:0/1907895641 wait complete. 2023-11-28T18:30:11.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.969+0000 7f56cb8fe640 1 Processor -- start 2023-11-28T18:30:11.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.969+0000 7f56cb8fe640 1 -- start start 2023-11-28T18:30:11.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.969+0000 7f56cb8fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56c406d610 0x7f56c4154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:11.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.969+0000 7f56cb8fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56c40ad9f0 0x7f56c415cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:11.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.969+0000 7f56cb8fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56c40cc7d0 0x7f56c415d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:11.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.969+0000 7f56cb8fe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c41579f0 con 0x7f56c406d610 2023-11-28T18:30:11.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.969+0000 7f56cb8fe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c4157560 con 0x7f56c40ad9f0 2023-11-28T18:30:11.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.969+0000 7f56cb8fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c4156c30 con 0x7f56c40cc7d0 2023-11-28T18:30:11.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.969+0000 7f56c969a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56c406d610 0x7f56c4154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:11.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.969+0000 7f56c969a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56c406d610 0x7f56c4154b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46618/0 (socket says 172.21.15.105:46618) 2023-11-28T18:30:11.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.969+0000 7f56c969a640 1 -- 172.21.15.105:0/2621317120 learned_addr learned my addr 172.21.15.105:0/2621317120 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:11.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.973+0000 7f56c8e99640 1 --2- 172.21.15.105:0/2621317120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56c40ad9f0 0x7f56c415cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:11.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.973+0000 7f56c9e9b640 1 --2- 172.21.15.105:0/2621317120 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56c40cc7d0 0x7f56c415d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:11.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.973+0000 7f56c969a640 1 -- 172.21.15.105:0/2621317120 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56c40cc7d0 msgr2=0x7f56c415d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:11.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.973+0000 7f56c969a640 1 --2- 172.21.15.105:0/2621317120 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56c40cc7d0 0x7f56c415d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.973+0000 7f56c969a640 1 -- 172.21.15.105:0/2621317120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56c40ad9f0 msgr2=0x7f56c415cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.973+0000 7f56c969a640 1 --2- 172.21.15.105:0/2621317120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56c40ad9f0 0x7f56c415cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.973+0000 7f56c969a640 1 -- 172.21.15.105:0/2621317120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b4009420 con 0x7f56c406d610 2023-11-28T18:30:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.973+0000 7f56c9e9b640 1 --2- 172.21.15.105:0/2621317120 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56c40cc7d0 0x7f56c415d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.973+0000 7f56c969a640 1 --2- 172.21.15.105:0/2621317120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56c406d610 0x7f56c4154b40 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f56c000c480 tx=0x7f56c000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.973+0000 7f56b9ffb640 1 -- 172.21.15.105:0/2621317120 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c00074e0 con 0x7f56c406d610 2023-11-28T18:30:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.973+0000 7f56b9ffb640 1 -- 172.21.15.105:0/2621317120 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c0007660 con 0x7f56c406d610 2023-11-28T18:30:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.973+0000 7f56b9ffb640 1 -- 172.21.15.105:0/2621317120 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c0004a30 con 0x7f56c406d610 2023-11-28T18:30:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.973+0000 7f56cb8fe640 1 -- 172.21.15.105:0/2621317120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c4156f10 con 0x7f56c406d610 2023-11-28T18:30:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.973+0000 7f56cb8fe640 1 -- 172.21.15.105:0/2621317120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c415dd10 con 0x7f56c406d610 2023-11-28T18:30:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.973+0000 7f56b9ffb640 1 -- 172.21.15.105:0/2621317120 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f56c0004b90 con 0x7f56c406d610 2023-11-28T18:30:11.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56b9ffb640 1 --2- 172.21.15.105:0/2621317120 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56a80668d0 0x7f56a8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:11.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56b9ffb640 1 -- 172.21.15.105:0/2621317120 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f56c0083700 con 0x7f56c406d610 2023-11-28T18:30:11.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56c8e99640 1 --2- 172.21.15.105:0/2621317120 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56a80668d0 0x7f56a8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:11.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56cec67000 1 Processor -- start 2023-11-28T18:30:11.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56cec67000 1 -- start start 2023-11-28T18:30:11.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56c8e99640 1 --2- 172.21.15.105:0/2621317120 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56a80668d0 0x7f56a8068d90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f56b40ad000 tx=0x7f56b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:11.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56cec67000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56433583cd30 0x56433583d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:11.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56cec67000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56433583d650 0x5643357effc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:11.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56cec67000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643357f0500 0x5643357f08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:11.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56cec67000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643356b9110 con 0x5643357f0500 2023-11-28T18:30:11.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56cec67000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643356499a0 con 0x56433583cd30 2023-11-28T18:30:11.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56cec67000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x564335646ed0 con 0x56433583d650 2023-11-28T18:30:11.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56c9e9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643357f0500 0x5643357f08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:11.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56c9e9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643357f0500 0x5643357f08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46622/0 (socket says 172.21.15.105:46622) 2023-11-28T18:30:11.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56c9e9b640 1 -- 172.21.15.105:0/3071122738 learned_addr learned my addr 172.21.15.105:0/3071122738 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:11.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56c8e99640 1 --2- 172.21.15.105:0/3071122738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56433583d650 0x5643357effc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:11.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56c969a640 1 --2- 172.21.15.105:0/3071122738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56433583cd30 0x56433583d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:11.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56c9e9b640 1 -- 172.21.15.105:0/3071122738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56433583d650 msgr2=0x5643357effc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:11.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56c9e9b640 1 --2- 172.21.15.105:0/3071122738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56433583d650 0x5643357effc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56c9e9b640 1 -- 172.21.15.105:0/3071122738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56433583cd30 msgr2=0x56433583d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:11.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56c9e9b640 1 --2- 172.21.15.105:0/3071122738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56433583cd30 0x56433583d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56c9e9b640 1 -- 172.21.15.105:0/3071122738 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56433533a890 con 0x5643357f0500 2023-11-28T18:30:11.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56c969a640 1 --2- 172.21.15.105:0/3071122738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56433583cd30 0x56433583d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:11.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56c9e9b640 1 --2- 172.21.15.105:0/3071122738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643357f0500 0x5643357f08e0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00d460 tx=0x7f56bc00d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6429bd9a50184b80 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:11.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56a6ffd640 1 -- 172.21.15.105:0/3071122738 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc00b740 con 0x5643357f0500 2023-11-28T18:30:11.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56a6ffd640 1 -- 172.21.15.105:0/3071122738 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc00b8c0 con 0x5643357f0500 2023-11-28T18:30:11.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56a6ffd640 1 -- 172.21.15.105:0/3071122738 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc00bc70 con 0x5643357f0500 2023-11-28T18:30:11.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56cec67000 1 -- 172.21.15.105:0/3071122738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643357f0500 msgr2=0x5643357f08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:11.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56cec67000 1 --2- 172.21.15.105:0/3071122738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643357f0500 0x5643357f08e0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00d460 tx=0x7f56bc00d920 comp rx=0 tx=0).stop 2023-11-28T18:30:11.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56cec67000 1 -- 172.21.15.105:0/3071122738 shutdown_connections 2023-11-28T18:30:11.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56cec67000 1 --2- 172.21.15.105:0/3071122738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643357f0500 0x5643357f08e0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56cec67000 1 --2- 172.21.15.105:0/3071122738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56433583d650 0x5643357effc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56cec67000 1 --2- 172.21.15.105:0/3071122738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56433583cd30 0x56433583d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56cec67000 1 -- 172.21.15.105:0/3071122738 >> 172.21.15.105:0/3071122738 conn(0x56433581d8c0 msgr2=0x56433581fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:11.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56cec67000 1 -- 172.21.15.105:0/3071122738 shutdown_connections 2023-11-28T18:30:11.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.977+0000 7f56cec67000 1 -- 172.21.15.105:0/3071122738 wait complete. 2023-11-28T18:30:11.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56cec67000 1 -- start start 2023-11-28T18:30:11.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56cec67000 1 Processor -- start 2023-11-28T18:30:11.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56cec67000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643357f0500 0x5643357f64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:11.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56cec67000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56433583cd30 0x5643357f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:11.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56cec67000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56433583d650 0x5643357f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56cec67000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643356427c0 con 0x56433583cd30 2023-11-28T18:30:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56cec67000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643355fabf0 con 0x5643357f0500 2023-11-28T18:30:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56cec67000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x564335600a90 con 0x56433583d650 2023-11-28T18:30:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56cec67000 10 client.0 _ll_register_callbacks cb 0x5643357dde20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:30:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56c8e99640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56433583cd30 0x5643357f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56c8e99640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56433583cd30 0x5643357f6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46638/0 (socket says 172.21.15.105:46638) 2023-11-28T18:30:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56c8e99640 1 -- 172.21.15.105:0/1364348065 learned_addr learned my addr 172.21.15.105:0/1364348065 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56c9e9b640 1 --2- 172.21.15.105:0/1364348065 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56433583d650 0x5643357f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56c969a640 1 --2- 172.21.15.105:0/1364348065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643357f0500 0x5643357f64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56c8e99640 1 -- 172.21.15.105:0/1364348065 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56433583d650 msgr2=0x5643357f6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56c8e99640 1 --2- 172.21.15.105:0/1364348065 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56433583d650 0x5643357f6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56c8e99640 1 -- 172.21.15.105:0/1364348065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643357f0500 msgr2=0x5643357f64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:11.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56c8e99640 1 --2- 172.21.15.105:0/1364348065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643357f0500 0x5643357f64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56c8e99640 1 -- 172.21.15.105:0/1364348065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56bc007c50 con 0x56433583cd30 2023-11-28T18:30:11.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56c969a640 1 --2- 172.21.15.105:0/1364348065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643357f0500 0x5643357f64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:11.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56c9e9b640 1 --2- 172.21.15.105:0/1364348065 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56433583d650 0x5643357f6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:11.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56c8e99640 1 --2- 172.21.15.105:0/1364348065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56433583cd30 0x5643357f6a20 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f56b4097f00 tx=0x7f56b401ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497ee5892e24c696 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:11.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f569ffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:30:11.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f569ffff640 1 -- 172.21.15.105:0/1364348065 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b40bb050 con 0x56433583cd30 2023-11-28T18:30:11.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f569ffff640 1 -- 172.21.15.105:0/1364348065 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f56b40b5ae0 con 0x56433583cd30 2023-11-28T18:30:11.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56cec67000 1 -- 172.21.15.105:0/1364348065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564335604470 con 0x56433583cd30 2023-11-28T18:30:11.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f569ffff640 1 -- 172.21.15.105:0/1364348065 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56b40abe90 con 0x56433583cd30 2023-11-28T18:30:11.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56cec67000 10 client.8277 fetch_fsmap learned FSMap version 103 2023-11-28T18:30:11.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56cec67000 1 -- 172.21.15.105:0/1364348065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x564335603190 con 0x56433583cd30 2023-11-28T18:30:11.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f569ffff640 1 -- 172.21.15.105:0/1364348065 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564335604470 con 0x56433583cd30 2023-11-28T18:30:11.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f569ffff640 1 -- 172.21.15.105:0/1364348065 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 103) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f56b40b5ca0 con 0x56433583cd30 2023-11-28T18:30:11.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56cec67000 10 client.8277 fetch_fsmap finished waiting for FSMap version 103 2023-11-28T18:30:11.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56cec67000 10 client.8277 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:30:11.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56cec67000 20 client.8277 populate_metadata read hostname 'smithi105' 2023-11-28T18:30:11.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56cec67000 1 --2- 172.21.15.105:0/1364348065 >> [v2:172.21.15.105:6835/2782729992,v1:172.21.15.105:6837/2782729992] conn(0x5643357f7970 0x56433584eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:11.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56cec67000 4 client.8277 mds_command: new command op to 8145 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:30:11.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56cec67000 1 -- 172.21.15.105:0/1364348065 --> [v2:172.21.15.105:6835/2782729992,v1:172.21.15.105:6837/2782729992] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564335607b50 con 0x5643357f7970 2023-11-28T18:30:11.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.981+0000 7f56c969a640 1 --2- 172.21.15.105:0/1364348065 >> [v2:172.21.15.105:6835/2782729992,v1:172.21.15.105:6837/2782729992] conn(0x5643357f7970 0x56433584eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:11.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.985+0000 7f569ffff640 10 client.8277 ms_handle_connect on v2:172.21.15.105:6835/2782729992 2023-11-28T18:30:11.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.985+0000 7f56c969a640 1 --2- 172.21.15.105:0/1364348065 >> [v2:172.21.15.105:6835/2782729992,v1:172.21.15.105:6837/2782729992] conn(0x5643357f7970 0x56433584eff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=502c829f73b6b971 server_cookie=d81d18a35c0837ee in_seq=0 out_seq=0 2023-11-28T18:30:11.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.985+0000 7f569ffff640 1 -- 172.21.15.105:0/1364348065 <== mds.0 v2:172.21.15.105:6835/2782729992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564335607b50 con 0x5643357f7970 2023-11-28T18:30:11.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.985+0000 7f569ffff640 10 client.8277 handle_command_reply: tid=0 2023-11-28T18:30:11.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.985+0000 7f56cec67000 1 client.8277 shutdown 2023-11-28T18:30:11.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.985+0000 7f56cec67000 1 -- 172.21.15.105:0/1364348065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56433583cd30 msgr2=0x5643357f6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:11.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.985+0000 7f56cec67000 1 --2- 172.21.15.105:0/1364348065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56433583cd30 0x5643357f6a20 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f56b4097f00 tx=0x7f56b401ac70 comp rx=0 tx=0).stop 2023-11-28T18:30:11.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.985+0000 7f56cec67000 1 -- 172.21.15.105:0/1364348065 shutdown_connections 2023-11-28T18:30:11.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.985+0000 7f56cec67000 1 --2- 172.21.15.105:0/1364348065 >> [v2:172.21.15.105:6835/2782729992,v1:172.21.15.105:6837/2782729992] conn(0x5643357f7970 0x56433584eff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.985+0000 7f56cec67000 1 --2- 172.21.15.105:0/1364348065 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56433583d650 0x5643357f6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.985+0000 7f56cec67000 1 --2- 172.21.15.105:0/1364348065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56433583cd30 0x5643357f6a20 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.985+0000 7f56cec67000 1 --2- 172.21.15.105:0/1364348065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643357f0500 0x5643357f64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:11.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.985+0000 7f569ffff640 0 client.8277 ms_handle_reset on v2:172.21.15.105:6835/2782729992 2023-11-28T18:30:11.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.985+0000 7f56cec67000 1 -- 172.21.15.105:0/1364348065 >> 172.21.15.105:0/1364348065 conn(0x56433581d8c0 msgr2=0x5643357ef6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:11.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.985+0000 7f56cec67000 1 -- 172.21.15.105:0/1364348065 shutdown_connections 2023-11-28T18:30:11.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.985+0000 7f56cec67000 1 -- 172.21.15.105:0/1364348065 wait complete. 2023-11-28T18:30:11.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:11.985+0000 7f56cec67000 20 client.8277 trim_cache size 0 max 16384 2023-11-28T18:30:12.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56cec67000 1 Processor -- start 2023-11-28T18:30:12.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56cec67000 1 -- start start 2023-11-28T18:30:12.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56cec67000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643357f0500 0x56433581c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:12.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56cec67000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56433583cd30 0x56433581c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:12.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56cec67000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56433583d650 0x56433584eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:12.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56cec67000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643352fb940 con 0x56433583d650 2023-11-28T18:30:12.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56cec67000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643355fe680 con 0x5643357f0500 2023-11-28T18:30:12.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56cec67000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x56433583cbe0 con 0x56433583cd30 2023-11-28T18:30:12.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56c9e9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56433583d650 0x56433584eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:12.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56c9e9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56433583d650 0x56433584eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46650/0 (socket says 172.21.15.105:46650) 2023-11-28T18:30:12.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56c9e9b640 1 -- 172.21.15.105:0/3044328547 learned_addr learned my addr 172.21.15.105:0/3044328547 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:12.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56c8e99640 1 --2- 172.21.15.105:0/3044328547 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56433583cd30 0x56433581c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:12.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56c969a640 1 --2- 172.21.15.105:0/3044328547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643357f0500 0x56433581c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:12.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56c9e9b640 1 -- 172.21.15.105:0/3044328547 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56433583cd30 msgr2=0x56433581c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:12.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56c9e9b640 1 --2- 172.21.15.105:0/3044328547 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56433583cd30 0x56433581c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56c9e9b640 1 -- 172.21.15.105:0/3044328547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643357f0500 msgr2=0x56433581c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:12.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56c9e9b640 1 --2- 172.21.15.105:0/3044328547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643357f0500 0x56433581c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56c9e9b640 1 -- 172.21.15.105:0/3044328547 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643355e9490 con 0x56433583d650 2023-11-28T18:30:12.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56c8e99640 1 --2- 172.21.15.105:0/3044328547 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56433583cd30 0x56433581c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:12.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56c969a640 1 --2- 172.21.15.105:0/3044328547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643357f0500 0x56433581c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:12.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56c9e9b640 1 --2- 172.21.15.105:0/3044328547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56433583d650 0x56433584eff0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f56bc007f40 tx=0x7f56bc010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ace5e5559f9b18 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56a77fe640 1 -- 172.21.15.105:0/3044328547 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc00b940 con 0x56433583d650 2023-11-28T18:30:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56a77fe640 1 -- 172.21.15.105:0/3044328547 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc00bac0 con 0x56433583d650 2023-11-28T18:30:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56a77fe640 1 -- 172.21.15.105:0/3044328547 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc00be70 con 0x56433583d650 2023-11-28T18:30:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56cec67000 1 -- 172.21.15.105:0/3044328547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56433583d650 msgr2=0x56433584eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56cec67000 1 --2- 172.21.15.105:0/3044328547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56433583d650 0x56433584eff0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f56bc007f40 tx=0x7f56bc010000 comp rx=0 tx=0).stop 2023-11-28T18:30:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56cec67000 1 -- 172.21.15.105:0/3044328547 shutdown_connections 2023-11-28T18:30:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56cec67000 1 --2- 172.21.15.105:0/3044328547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56433583d650 0x56433584eff0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56cec67000 1 --2- 172.21.15.105:0/3044328547 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56433583cd30 0x56433581c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56cec67000 1 --2- 172.21.15.105:0/3044328547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643357f0500 0x56433581c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56cec67000 1 -- 172.21.15.105:0/3044328547 >> 172.21.15.105:0/3044328547 conn(0x56433581d8c0 msgr2=0x5643357ee720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:12.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56cec67000 1 -- 172.21.15.105:0/3044328547 shutdown_connections 2023-11-28T18:30:12.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.001+0000 7f56cec67000 1 -- 172.21.15.105:0/3044328547 wait complete. 2023-11-28T18:30:12.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56cec67000 1 -- start start 2023-11-28T18:30:12.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56cec67000 1 Processor -- start 2023-11-28T18:30:12.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56cec67000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643357f0500 0x56433589d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:12.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56cec67000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56433583cd30 0x56433589dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:12.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56cec67000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56433583d650 0x564335858a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:12.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56cec67000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x56433570b7d0 con 0x56433583d650 2023-11-28T18:30:12.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56cec67000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643355ff9b0 con 0x5643357f0500 2023-11-28T18:30:12.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56cec67000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x56433564a0c0 con 0x56433583cd30 2023-11-28T18:30:12.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56cec67000 10 client.0 _ll_register_callbacks cb 0x5643356b9060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:30:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56c9e9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56433583d650 0x564335858a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56c9e9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56433583d650 0x564335858a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46656/0 (socket says 172.21.15.105:46656) 2023-11-28T18:30:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56c9e9b640 1 -- 172.21.15.105:0/2213674480 learned_addr learned my addr 172.21.15.105:0/2213674480 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56c9e9b640 1 -- 172.21.15.105:0/2213674480 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56433583cd30 msgr2=0x56433589dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56c9e9b640 1 --2- 172.21.15.105:0/2213674480 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56433583cd30 0x56433589dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56c9e9b640 1 -- 172.21.15.105:0/2213674480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643357f0500 msgr2=0x56433589d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56c9e9b640 1 --2- 172.21.15.105:0/2213674480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643357f0500 0x56433589d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56c9e9b640 1 -- 172.21.15.105:0/2213674480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56433533a890 con 0x56433583d650 2023-11-28T18:30:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56c9e9b640 1 --2- 172.21.15.105:0/2213674480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56433583d650 0x564335858a70 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f56bc015000 tx=0x7f56bc004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef21605a33f7af70 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f569ffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:30:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f569ffff640 1 -- 172.21.15.105:0/2213674480 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc00d240 con 0x56433583d650 2023-11-28T18:30:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f569ffff640 1 -- 172.21.15.105:0/2213674480 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f56bc01f440 con 0x56433583d650 2023-11-28T18:30:12.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56cec67000 1 -- 172.21.15.105:0/2213674480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564335641110 con 0x56433583d650 2023-11-28T18:30:12.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f569ffff640 1 -- 172.21.15.105:0/2213674480 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56bc01f620 con 0x56433583d650 2023-11-28T18:30:12.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56cec67000 10 client.8283 fetch_fsmap learned FSMap version 103 2023-11-28T18:30:12.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56cec67000 1 -- 172.21.15.105:0/2213674480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x56433533acc0 con 0x56433583d650 2023-11-28T18:30:12.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f569ffff640 1 -- 172.21.15.105:0/2213674480 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564335641110 con 0x56433583d650 2023-11-28T18:30:12.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f569ffff640 1 -- 172.21.15.105:0/2213674480 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 103) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f56bc008040 con 0x56433583d650 2023-11-28T18:30:12.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56cec67000 10 client.8283 fetch_fsmap finished waiting for FSMap version 103 2023-11-28T18:30:12.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56cec67000 10 client.8283 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:30:12.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56cec67000 20 client.8283 populate_metadata read hostname 'smithi105' 2023-11-28T18:30:12.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56cec67000 1 --2- 172.21.15.105:0/2213674480 >> [v2:172.21.15.105:6835/2782729992,v1:172.21.15.105:6837/2782729992] conn(0x564335859430 0x564335859810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:12.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56cec67000 4 client.8283 mds_command: new command op to 8145 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:30:12.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56cec67000 1 -- 172.21.15.105:0/2213674480 --> [v2:172.21.15.105:6835/2782729992,v1:172.21.15.105:6837/2782729992] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56433566f650 con 0x564335859430 2023-11-28T18:30:12.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56c969a640 1 --2- 172.21.15.105:0/2213674480 >> [v2:172.21.15.105:6835/2782729992,v1:172.21.15.105:6837/2782729992] conn(0x564335859430 0x564335859810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:12.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f569ffff640 10 client.8283 ms_handle_connect on v2:172.21.15.105:6835/2782729992 2023-11-28T18:30:12.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.005+0000 7f56c969a640 1 --2- 172.21.15.105:0/2213674480 >> [v2:172.21.15.105:6835/2782729992,v1:172.21.15.105:6837/2782729992] conn(0x564335859430 0x564335859810 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b862ac0fcab7973 server_cookie=30fa8cc15e7469f9 in_seq=0 out_seq=0 2023-11-28T18:30:12.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f569ffff640 1 -- 172.21.15.105:0/2213674480 <== mds.0 v2:172.21.15.105:6835/2782729992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x56433566f650 con 0x564335859430 2023-11-28T18:30:12.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f569ffff640 10 client.8283 handle_command_reply: tid=0 2023-11-28T18:30:12.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cec67000 1 client.8283 shutdown 2023-11-28T18:30:12.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cec67000 1 -- 172.21.15.105:0/2213674480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56433583d650 msgr2=0x564335858a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:12.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cec67000 1 --2- 172.21.15.105:0/2213674480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56433583d650 0x564335858a70 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f56bc015000 tx=0x7f56bc004ac0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cec67000 1 -- 172.21.15.105:0/2213674480 shutdown_connections 2023-11-28T18:30:12.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cec67000 1 --2- 172.21.15.105:0/2213674480 >> [v2:172.21.15.105:6835/2782729992,v1:172.21.15.105:6837/2782729992] conn(0x564335859430 0x564335859810 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cec67000 1 --2- 172.21.15.105:0/2213674480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56433583d650 0x564335858a70 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cec67000 1 --2- 172.21.15.105:0/2213674480 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56433583cd30 0x56433589dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f569ffff640 0 client.8283 ms_handle_reset on v2:172.21.15.105:6835/2782729992 2023-11-28T18:30:12.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cec67000 1 --2- 172.21.15.105:0/2213674480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643357f0500 0x56433589d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cec67000 1 -- 172.21.15.105:0/2213674480 >> 172.21.15.105:0/2213674480 conn(0x56433581d8c0 msgr2=0x5643357eefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:12.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cec67000 1 -- 172.21.15.105:0/2213674480 shutdown_connections 2023-11-28T18:30:12.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cec67000 1 -- 172.21.15.105:0/2213674480 wait complete. 2023-11-28T18:30:12.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cec67000 20 client.8283 trim_cache size 0 max 16384 2023-11-28T18:30:12.016 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:30:12.016 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:30:12.016 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2023-11-28T18:30:12.016 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 8145, 2023-11-28T18:30:12.017 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:30:12.017 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:30:12.017 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:30:12.017 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 5.852634353, 2023-11-28T18:30:12.017 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 103, 2023-11-28T18:30:12.017 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 103, 2023-11-28T18:30:12.017 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 103, 2023-11-28T18:30:12.017 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 11.186855294000001 2023-11-28T18:30:12.017 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:30:12.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cb8fe640 1 -- 172.21.15.105:0/2621317120 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56a80668d0 msgr2=0x7f56a8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:12.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cb8fe640 1 --2- 172.21.15.105:0/2621317120 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56a80668d0 0x7f56a8068d90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f56b40ad000 tx=0x7f56b4008040 comp rx=0 tx=0).stop 2023-11-28T18:30:12.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cb8fe640 1 -- 172.21.15.105:0/2621317120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56c406d610 msgr2=0x7f56c4154b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:12.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cb8fe640 1 --2- 172.21.15.105:0/2621317120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56c406d610 0x7f56c4154b40 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f56c000c480 tx=0x7f56c000c940 comp rx=0 tx=0).stop 2023-11-28T18:30:12.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cb8fe640 1 -- 172.21.15.105:0/2621317120 shutdown_connections 2023-11-28T18:30:12.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cb8fe640 1 --2- 172.21.15.105:0/2621317120 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56c40cc7d0 0x7f56c415d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cb8fe640 1 --2- 172.21.15.105:0/2621317120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56c40ad9f0 0x7f56c415cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cb8fe640 1 --2- 172.21.15.105:0/2621317120 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56a80668d0 0x7f56a8068d90 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cb8fe640 1 --2- 172.21.15.105:0/2621317120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56c406d610 0x7f56c4154b40 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cb8fe640 1 -- 172.21.15.105:0/2621317120 >> 172.21.15.105:0/2621317120 conn(0x7f56c40ceb90 msgr2=0x7f56c4071af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:12.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cb8fe640 1 -- 172.21.15.105:0/2621317120 shutdown_connections 2023-11-28T18:30:12.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.009+0000 7f56cb8fe640 1 -- 172.21.15.105:0/2621317120 wait complete. 2023-11-28T18:30:12.028 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T18:30:12.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.165+0000 7fecdc232640 1 Processor -- start 2023-11-28T18:30:12.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.165+0000 7fecdc232640 1 -- start start 2023-11-28T18:30:12.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.165+0000 7fecdc232640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fecd415d3c0 0x7fecd4071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:12.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.165+0000 7fecdc232640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd40cefa0 con 0x7fecd4070a10 2023-11-28T18:30:12.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.165+0000 7fecdc232640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd415d7a0 con 0x7fecd4161870 2023-11-28T18:30:12.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.165+0000 7fecdc232640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd40ce9e0 con 0x7fecd415d3c0 2023-11-28T18:30:12.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.165+0000 7fecd9fce640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fecd415d3c0 0x7fecd4071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:12.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.165+0000 7fecda7cf640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fecd4070a10 0x7fecd4070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39060/0 (socket says 172.21.15.105:39060) 2023-11-28T18:30:12.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.165+0000 7fecda7cf640 1 -- 172.21.15.105:0/2483808556 learned_addr learned my addr 172.21.15.105:0/2483808556 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:12.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.165+0000 7fecd9fce640 1 -- 172.21.15.105:0/2483808556 >> v1:172.21.15.148:6789/0 conn(0x7fecd4161870 legacy=0x7fecd40ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:12.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.165+0000 7fecd9fce640 1 -- 172.21.15.105:0/2483808556 >> v1:172.21.15.105:6789/0 conn(0x7fecd4070a10 legacy=0x7fecd4070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:12.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.165+0000 7fecd9fce640 1 -- 172.21.15.105:0/2483808556 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd40cecc0 con 0x7fecd415d3c0 2023-11-28T18:30:12.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.169+0000 7fecd9fce640 1 --2- 172.21.15.105:0/2483808556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd415d3c0 0x7fecd4071a10 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fecd00097c0 tx=0x7fecd001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cb574fd2759171 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:12.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.169+0000 7fecd8fcc640 1 -- 172.21.15.105:0/2483808556 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd00ab610 con 0x7fecd415d3c0 2023-11-28T18:30:12.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.169+0000 7fecd8fcc640 1 -- 172.21.15.105:0/2483808556 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd00ab770 con 0x7fecd415d3c0 2023-11-28T18:30:12.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.169+0000 7fecd8fcc640 1 -- 172.21.15.105:0/2483808556 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd00abb00 con 0x7fecd415d3c0 2023-11-28T18:30:12.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.169+0000 7fecdc232640 1 -- 172.21.15.105:0/2483808556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd415d3c0 msgr2=0x7fecd4071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:12.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.169+0000 7fecdc232640 1 --2- 172.21.15.105:0/2483808556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd415d3c0 0x7fecd4071a10 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fecd00097c0 tx=0x7fecd001c410 comp rx=0 tx=0).stop 2023-11-28T18:30:12.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.169+0000 7fecdc232640 1 -- 172.21.15.105:0/2483808556 shutdown_connections 2023-11-28T18:30:12.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.169+0000 7fecdc232640 1 --2- 172.21.15.105:0/2483808556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd415d3c0 0x7fecd4071a10 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.169+0000 7fecdc232640 1 -- 172.21.15.105:0/2483808556 >> 172.21.15.105:0/2483808556 conn(0x7fecd40cf540 msgr2=0x7fecd40cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:12.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.169+0000 7fecdc232640 1 -- 172.21.15.105:0/2483808556 shutdown_connections 2023-11-28T18:30:12.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.169+0000 7fecdc232640 1 -- 172.21.15.105:0/2483808556 wait complete. 2023-11-28T18:30:12.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecdc232640 1 Processor -- start 2023-11-28T18:30:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecdc232640 1 -- start start 2023-11-28T18:30:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecdc232640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd4070a10 0x7fecd415c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecdc232640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecd415d3c0 0x7fecd40b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecdc232640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecd4161870 0x7fecd4152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecdc232640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd4160840 con 0x7fecd4070a10 2023-11-28T18:30:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecdc232640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd41504e0 con 0x7fecd4161870 2023-11-28T18:30:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecdc232640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd415ffa0 con 0x7fecd415d3c0 2023-11-28T18:30:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecd9fce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd4070a10 0x7fecd415c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecd9fce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd4070a10 0x7fecd415c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46678/0 (socket says 172.21.15.105:46678) 2023-11-28T18:30:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecd9fce640 1 -- 172.21.15.105:0/1966024907 learned_addr learned my addr 172.21.15.105:0/1966024907 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecda7cf640 1 --2- 172.21.15.105:0/1966024907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecd4161870 0x7fecd4152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecd97cd640 1 --2- 172.21.15.105:0/1966024907 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecd415d3c0 0x7fecd40b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecd9fce640 1 -- 172.21.15.105:0/1966024907 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecd415d3c0 msgr2=0x7fecd40b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:12.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecd9fce640 1 --2- 172.21.15.105:0/1966024907 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecd415d3c0 0x7fecd40b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecd9fce640 1 -- 172.21.15.105:0/1966024907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecd4161870 msgr2=0x7fecd4152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:12.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecd9fce640 1 --2- 172.21.15.105:0/1966024907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecd4161870 0x7fecd4152840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecd9fce640 1 -- 172.21.15.105:0/1966024907 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd0009420 con 0x7fecd4070a10 2023-11-28T18:30:12.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecda7cf640 1 --2- 172.21.15.105:0/1966024907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecd4161870 0x7fecd4152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:12.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecd9fce640 1 --2- 172.21.15.105:0/1966024907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd4070a10 0x7fecd415c1c0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fecd00ad040 tx=0x7fecd001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:12.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecca7fc640 1 -- 172.21.15.105:0/1966024907 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd00b5810 con 0x7fecd4070a10 2023-11-28T18:30:12.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecca7fc640 1 -- 172.21.15.105:0/1966024907 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd00b5990 con 0x7fecd4070a10 2023-11-28T18:30:12.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecca7fc640 1 -- 172.21.15.105:0/1966024907 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd00b5d40 con 0x7fecd4070a10 2023-11-28T18:30:12.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecdc232640 1 -- 172.21.15.105:0/1966024907 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd4160280 con 0x7fecd4070a10 2023-11-28T18:30:12.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.173+0000 7fecdc232640 1 -- 172.21.15.105:0/1966024907 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd415c700 con 0x7fecd4070a10 2023-11-28T18:30:12.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.177+0000 7fecdc232640 1 -- 172.21.15.105:0/1966024907 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd41502b0 con 0x7fecd4070a10 2023-11-28T18:30:12.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.177+0000 7fecca7fc640 1 -- 172.21.15.105:0/1966024907 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fecd00bf410 con 0x7fecd4070a10 2023-11-28T18:30:12.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.177+0000 7fecca7fc640 1 --2- 172.21.15.105:0/1966024907 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fecb80668d0 0x7fecb8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:12.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.177+0000 7fecca7fc640 1 -- 172.21.15.105:0/1966024907 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fecd011f060 con 0x7fecd4070a10 2023-11-28T18:30:12.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.177+0000 7fecd97cd640 1 --2- 172.21.15.105:0/1966024907 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fecb80668d0 0x7fecb8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:12.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.177+0000 7fecd97cd640 1 --2- 172.21.15.105:0/1966024907 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fecb80668d0 0x7fecb8068d90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fecd4154930 tx=0x7fecc4007e40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:12.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.181+0000 7fecca7fc640 1 -- 172.21.15.105:0/1966024907 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fecd41502b0 con 0x7fecd4070a10 2023-11-28T18:30:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.365+0000 7fecdc232640 1 -- 172.21.15.105:0/1966024907 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fecd41502b0 con 0x7fecd4070a10 2023-11-28T18:30:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.365+0000 7fecca7fc640 1 -- 172.21.15.105:0/1966024907 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v103) v1 ==== 53+0+73 (secure 0 0 0) 0x7fecd41502b0 con 0x7fecd4070a10 2023-11-28T18:30:12.364 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T18:30:12.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.365+0000 7fecdc232640 1 -- 172.21.15.105:0/1966024907 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fecb80668d0 msgr2=0x7fecb8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:12.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.365+0000 7fecdc232640 1 --2- 172.21.15.105:0/1966024907 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fecb80668d0 0x7fecb8068d90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fecd4154930 tx=0x7fecc4007e40 comp rx=0 tx=0).stop 2023-11-28T18:30:12.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.365+0000 7fecdc232640 1 -- 172.21.15.105:0/1966024907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd4070a10 msgr2=0x7fecd415c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:12.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.365+0000 7fecdc232640 1 --2- 172.21.15.105:0/1966024907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd4070a10 0x7fecd415c1c0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fecd00ad040 tx=0x7fecd001ac50 comp rx=0 tx=0).stop 2023-11-28T18:30:12.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.365+0000 7fecdc232640 1 -- 172.21.15.105:0/1966024907 shutdown_connections 2023-11-28T18:30:12.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.365+0000 7fecdc232640 1 --2- 172.21.15.105:0/1966024907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecd4161870 0x7fecd4152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.365+0000 7fecdc232640 1 --2- 172.21.15.105:0/1966024907 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecd415d3c0 0x7fecd40b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.365+0000 7fecdc232640 1 --2- 172.21.15.105:0/1966024907 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fecb80668d0 0x7fecb8068d90 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.365+0000 7fecdc232640 1 --2- 172.21.15.105:0/1966024907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd4070a10 0x7fecd415c1c0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:12.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.365+0000 7fecdc232640 1 -- 172.21.15.105:0/1966024907 >> 172.21.15.105:0/1966024907 conn(0x7fecd40cf540 msgr2=0x7fecd40b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:12.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.365+0000 7fecdc232640 1 -- 172.21.15.105:0/1966024907 shutdown_connections 2023-11-28T18:30:12.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:12.365+0000 7fecdc232640 1 -- 172.21.15.105:0/1966024907 wait complete. 2023-11-28T18:30:12.385 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T18:30:12.385 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T18:30:12.385 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2023-11-28T18:30:12.385 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T18:30:12.385 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T18:30:12.385 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T18:30:12.385 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T18:30:12.386 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T18:30:12.386 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:30:12.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:12.386 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2023-11-28T18:30:12.395 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T18:30:12.396 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T18:30:12.396 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T18:30:12.396 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:30:12.396 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-11-28T18:30:12.396 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:30:12.396 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:30:12.396 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:30:12.396 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2023-11-28T18:30:12.396 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:30:12.396 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:30:12.396 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2023-11-28T18:30:12.397 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:30:12.397 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:30:12.397 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T18:30:12.397 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:30:12.397 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-28T18:30:12.397 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42009sec preferred_lft 42009sec 2023-11-28T18:30:12.397 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2023-11-28T18:30:12.397 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:30:12.397 INFO:teuthology.orchestra.run.smithi105.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-28T18:30:12.397 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 02:42:cc:d4:87:d2 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:30:12.397 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-28T18:30:12.397 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:30:12.397 INFO:teuthology.orchestra.run.smithi105.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T18:30:12.397 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-28T18:30:12.398 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T18:30:12.398 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:30:12.398 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-28T18:30:12.398 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:30:12.398 INFO:teuthology.orchestra.run.smithi105.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T18:30:12.398 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:30:12.398 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-28T18:30:12.398 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:30:12.398 INFO:teuthology.orchestra.run.smithi105.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T18:30:12.398 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:30:12.398 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-28T18:30:12.399 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:30:12.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:12.399 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T18:30:12.462 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-11-28T18:30:12.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:12.465 DEBUG:teuthology.orchestra.run.smithi105:> route 2023-11-28T18:30:12.518 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2023-11-28T18:30:12.518 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T18:30:12.518 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-28T18:30:12.518 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T18:30:12.518 INFO:teuthology.orchestra.run.smithi105.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-28T18:30:12.518 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T18:30:12.518 INFO:teuthology.orchestra.run.smithi105.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:30:12.518 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:30:12.518 INFO:teuthology.orchestra.run.smithi105.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:30:12.518 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T18:30:12.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:12.519 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-11-28T18:30:12.519 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:30:12.519 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T18:30:12.519 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T18:30:12.519 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T18:30:12.519 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:30:12.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:12.612 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-11-28T18:30:12.622 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T18:30:12.622 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T18:30:12.622 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:30:12.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:12.623 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2023-11-28T18:30:12.683 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-28T18:30:12.684 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:30:12.684 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T18:30:12.735 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:30:12.736 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:30:12.736 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T18:30:12.791 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:30:12.860 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:30:12.913 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:30:12.913 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:30:12.914 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:30:12.914 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:30:12.914 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:30:12.914 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:30:12.914 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:30:12.915 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:30:12.915 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:30:12.915 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:30:12.915 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:30:12.915 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:30:12.915 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:30:12.916 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:30:12.916 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:30:12.916 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:30:12.916 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:30:12.916 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:30:12.917 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:30:12.917 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:30:12.917 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:30:12.918 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:30:12.918 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:30:12.918 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:30:12.918 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:30:12.918 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:30:12.919 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:30:12.919 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:30:12.919 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:30:12.919 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:30:12.919 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:30:12.919 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:30:12.919 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:30:12.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:12.920 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:30:12.971 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T18:30:12.971 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-28T18:30:13.018 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:30:13.040 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:30:13.094 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:30:13.097 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:30:13.097 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:30:13.097 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:30:13.097 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:30:13.097 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:30:13.097 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:30:13.097 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:30:13.097 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:30:13.098 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:30:13.098 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:30:13.098 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:30:13.098 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:30:13.098 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:30:13.098 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:30:13.098 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:30:13.098 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:30:13.098 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:30:13.098 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:30:13.098 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:30:13.098 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:30:13.099 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:30:13.099 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:30:13.099 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:30:13.099 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:30:13.099 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:30:13.099 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:30:13.099 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:30:13.099 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:30:13.099 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:30:13.099 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:30:13.099 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:30:13.100 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:30:13.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:13.102 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:30:13.115 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:2023-11-28T18:30:13.113+0000 7ff71837a580 -1 init, newargv = 0x5606b667dd00 newargc=15 2023-11-28T18:30:13.115 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[63065]: starting ceph client 2023-11-28T18:30:13.122 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[63065]: starting fuse 2023-11-28T18:30:13.124 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-11-28T18:30:13.124 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-28T18:30:13.124 DEBUG:teuthology.orchestra.run.smithi105:> sudo stdin-killer -- python3 -c ' 2023-11-28T18:30:13.124 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2023-11-28T18:30:13.124 DEBUG:teuthology.orchestra.run.smithi105:> import re 2023-11-28T18:30:13.125 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-11-28T18:30:13.125 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2023-11-28T18:30:13.125 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:30:13.125 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2023-11-28T18:30:13.125 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-28T18:30:13.125 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2023-11-28T18:30:13.125 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-28T18:30:13.125 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:30:13.125 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2023-11-28T18:30:13.125 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2023-11-28T18:30:13.125 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2023-11-28T18:30:13.125 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2023-11-28T18:30:13.125 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:30:13.125 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2023-11-28T18:30:13.126 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T18:30:13.126 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T18:30:13.126 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T18:30:13.126 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2023-11-28T18:30:13.126 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2023-11-28T18:30:13.126 DEBUG:teuthology.orchestra.run.smithi105:> return f 2023-11-28T18:30:13.126 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T18:30:13.126 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:30:13.126 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.0")) 2023-11-28T18:30:13.126 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-11-28T18:30:13.253 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.0.63065.asok 2023-11-28T18:30:13.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-28T18:30:13.264 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.63065.asok 2023-11-28T18:30:13.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:13.265 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.63065.asok status 2023-11-28T18:30:13.440 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:30:13.441 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2023-11-28T18:30:13.441 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "47c1c547372ed5bff7984647dc200bac0e019caf", 2023-11-28T18:30:13.441 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev)", 2023-11-28T18:30:13.441 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "0", 2023-11-28T18:30:13.441 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2023-11-28T18:30:13.441 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-28T18:30:13.441 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "63065", 2023-11-28T18:30:13.441 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2023-11-28T18:30:13.441 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:13.442 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2023-11-28T18:30:13.442 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2023-11-28T18:30:13.442 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 8301, 2023-11-28T18:30:13.442 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2023-11-28T18:30:13.442 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2023-11-28T18:30:13.442 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2023-11-28T18:30:13.442 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 8301 2023-11-28T18:30:13.442 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:13.442 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-11-28T18:30:13.442 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-11-28T18:30:13.442 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-11-28T18:30:13.442 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 3273945629 2023-11-28T18:30:13.443 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:13.443 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:13.443 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-11-28T18:30:13.443 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-11-28T18:30:13.443 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-11-28T18:30:13.443 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 3273945629 2023-11-28T18:30:13.443 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:13.443 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.8301 192.168.0.1:0/3273945629", 2023-11-28T18:30:13.443 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/3273945629", 2023-11-28T18:30:13.443 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2023-11-28T18:30:13.443 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 103, 2023-11-28T18:30:13.444 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 103, 2023-11-28T18:30:13.444 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2023-11-28T18:30:13.444 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2023-11-28T18:30:13.444 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2023-11-28T18:30:13.444 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:30:13.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:13.456 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2023-11-28T18:30:13.456 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-28T18:30:13.465 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2023-11-28T18:30:13.465 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-28T18:30:13.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:13.466 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T18:30:13.528 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random True 2023-11-28T18:30:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.713+0000 7f633d432640 1 Processor -- start 2023-11-28T18:30:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.713+0000 7f633d432640 1 -- start start 2023-11-28T18:30:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.713+0000 7f633d432640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f63380cdd80 0x7f633806c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.713+0000 7f633d432640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63380ce230 0x7f63380cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.713+0000 7f633d432640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63380b3c40 con 0x7f633815d320 2023-11-28T18:30:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.713+0000 7f633d432640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633806ca80 con 0x7f63380cdd80 2023-11-28T18:30:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.713+0000 7f633d432640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633815d700 con 0x7f63380ce230 2023-11-28T18:30:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.713+0000 7f63377fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63380ce230 0x7f63380cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.713+0000 7f63377fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63380ce230 0x7f63380cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:57932/0 (socket says 172.21.15.105:57932) 2023-11-28T18:30:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.713+0000 7f63377fe640 1 -- 172.21.15.105:0/1489676491 learned_addr learned my addr 172.21.15.105:0/1489676491 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.713+0000 7f632ffff640 1 --2- 172.21.15.105:0/1489676491 >> v2:172.21.15.148:3300/0 conn(0x7f63380cdd80 0x7f633806c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.713+0000 7f63377fe640 1 -- 172.21.15.105:0/1489676491 >> v2:172.21.15.148:3300/0 conn(0x7f63380cdd80 msgr2=0x7f633806c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.713+0000 7f63377fe640 1 --2- 172.21.15.105:0/1489676491 >> v2:172.21.15.148:3300/0 conn(0x7f63380cdd80 0x7f633806c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.713+0000 7f63377fe640 1 -- 172.21.15.105:0/1489676491 >> v1:172.21.15.148:6789/0 conn(0x7f633815d320 legacy=0x7f63380cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.713+0000 7f63377fe640 1 -- 172.21.15.105:0/1489676491 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633806c7a0 con 0x7f63380ce230 2023-11-28T18:30:13.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.717+0000 7f63377fe640 1 --2- 172.21.15.105:0/1489676491 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63380ce230 0x7f63380cc0c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6330009e50 tx=0x7f633001c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45f45a1442bb9422 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:13.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.717+0000 7f63367fc640 1 -- 172.21.15.105:0/1489676491 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63300a5840 con 0x7f63380ce230 2023-11-28T18:30:13.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.717+0000 7f63367fc640 1 -- 172.21.15.105:0/1489676491 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63300a59a0 con 0x7f63380ce230 2023-11-28T18:30:13.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.717+0000 7f63367fc640 1 -- 172.21.15.105:0/1489676491 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63300a5d30 con 0x7f63380ce230 2023-11-28T18:30:13.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.717+0000 7f633d432640 1 -- 172.21.15.105:0/1489676491 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63380ce230 msgr2=0x7f63380cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:13.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.717+0000 7f633d432640 1 --2- 172.21.15.105:0/1489676491 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63380ce230 0x7f63380cc0c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6330009e50 tx=0x7f633001c980 comp rx=0 tx=0).stop 2023-11-28T18:30:13.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.717+0000 7f633d432640 1 -- 172.21.15.105:0/1489676491 shutdown_connections 2023-11-28T18:30:13.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.717+0000 7f633d432640 1 --2- 172.21.15.105:0/1489676491 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63380ce230 0x7f63380cc0c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:13.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.717+0000 7f633d432640 1 --2- 172.21.15.105:0/1489676491 >> v2:172.21.15.148:3300/0 conn(0x7f63380cdd80 0x7f633806c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:13.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.717+0000 7f633d432640 1 -- 172.21.15.105:0/1489676491 >> 172.21.15.105:0/1489676491 conn(0x7f63380d0df0 msgr2=0x7f63380d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:13.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.717+0000 7f633d432640 1 -- 172.21.15.105:0/1489676491 shutdown_connections 2023-11-28T18:30:13.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.717+0000 7f633d432640 1 -- 172.21.15.105:0/1489676491 wait complete. 2023-11-28T18:30:13.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f633d432640 1 Processor -- start 2023-11-28T18:30:13.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f633d432640 1 -- start start 2023-11-28T18:30:13.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f633d432640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63380cdd80 0x7f6338153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:13.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f633d432640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63380ce230 0x7f633815de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:13.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f633d432640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633815d320 0x7f633815e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:13.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f633d432640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6338156c80 con 0x7f633815d320 2023-11-28T18:30:13.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f633d432640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63380b4190 con 0x7f63380ce230 2023-11-28T18:30:13.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f633d432640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63380b4470 con 0x7f63380cdd80 2023-11-28T18:30:13.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f63377fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633815d320 0x7f633815e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:13.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f63377fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633815d320 0x7f633815e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46686/0 (socket says 172.21.15.105:46686) 2023-11-28T18:30:13.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f63377fe640 1 -- 172.21.15.105:0/1465420703 learned_addr learned my addr 172.21.15.105:0/1465420703 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:13.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f632ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63380ce230 0x7f633815de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:13.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f6336ffd640 1 --2- 172.21.15.105:0/1465420703 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63380cdd80 0x7f6338153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:13.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f63377fe640 1 -- 172.21.15.105:0/1465420703 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63380cdd80 msgr2=0x7f6338153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:13.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f63377fe640 1 --2- 172.21.15.105:0/1465420703 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63380cdd80 0x7f6338153280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:13.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f63377fe640 1 -- 172.21.15.105:0/1465420703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63380ce230 msgr2=0x7f633815de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:13.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f63377fe640 1 --2- 172.21.15.105:0/1465420703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63380ce230 0x7f633815de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:13.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f63377fe640 1 -- 172.21.15.105:0/1465420703 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6330009b30 con 0x7f633815d320 2023-11-28T18:30:13.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f63377fe640 1 --2- 172.21.15.105:0/1465420703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633815d320 0x7f633815e360 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f633001a5b0 tx=0x7f6330019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:13.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f632f7fe640 1 -- 172.21.15.105:0/1465420703 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63300b8670 con 0x7f633815d320 2023-11-28T18:30:13.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f632f7fe640 1 -- 172.21.15.105:0/1465420703 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63300b87d0 con 0x7f633815d320 2023-11-28T18:30:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f632f7fe640 1 -- 172.21.15.105:0/1465420703 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63300b8b80 con 0x7f633815d320 2023-11-28T18:30:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f633d432640 1 -- 172.21.15.105:0/1465420703 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6338156630 con 0x7f633815d320 2023-11-28T18:30:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f633d432640 1 -- 172.21.15.105:0/1465420703 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63380b3dd0 con 0x7f633815d320 2023-11-28T18:30:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.721+0000 7f633d432640 1 -- 172.21.15.105:0/1465420703 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6338156490 con 0x7f633815d320 2023-11-28T18:30:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.725+0000 7f632f7fe640 1 -- 172.21.15.105:0/1465420703 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f63300b8ce0 con 0x7f633815d320 2023-11-28T18:30:13.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.725+0000 7f632f7fe640 1 --2- 172.21.15.105:0/1465420703 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f63180668d0 0x7f6318068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:13.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.725+0000 7f632f7fe640 1 -- 172.21.15.105:0/1465420703 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f633011f960 con 0x7f633815d320 2023-11-28T18:30:13.727 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.725+0000 7f6336ffd640 1 --2- 172.21.15.105:0/1465420703 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f63180668d0 0x7f6318068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:13.727 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.725+0000 7f6336ffd640 1 --2- 172.21.15.105:0/1465420703 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f63180668d0 0x7f6318068d90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6328001770 tx=0x7f6328009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.729+0000 7f632f7fe640 1 -- 172.21.15.105:0/1465420703 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6338156490 con 0x7f633815d320 2023-11-28T18:30:13.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.881+0000 7f633d432640 1 -- 172.21.15.105:0/1465420703 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random}] v 0) v1 -- 0x7f63381579f0 con 0x7f633815d320 2023-11-28T18:30:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.885+0000 7f632f7fe640 1 -- 172.21.15.105:0/1465420703 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random}]=0 v52)=0 v52) v1 ==== 128+0+0 (secure 0 0 0) 0x7f63381579f0 con 0x7f633815d320 2023-11-28T18:30:13.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.885+0000 7f633d432640 1 -- 172.21.15.105:0/1465420703 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f63180668d0 msgr2=0x7f6318068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:13.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.885+0000 7f633d432640 1 --2- 172.21.15.105:0/1465420703 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f63180668d0 0x7f6318068d90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6328001770 tx=0x7f6328009040 comp rx=0 tx=0).stop 2023-11-28T18:30:13.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.885+0000 7f633d432640 1 -- 172.21.15.105:0/1465420703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633815d320 msgr2=0x7f633815e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:13.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.885+0000 7f633d432640 1 --2- 172.21.15.105:0/1465420703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633815d320 0x7f633815e360 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f633001a5b0 tx=0x7f6330019720 comp rx=0 tx=0).stop 2023-11-28T18:30:13.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.885+0000 7f633d432640 1 -- 172.21.15.105:0/1465420703 shutdown_connections 2023-11-28T18:30:13.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.885+0000 7f633d432640 1 --2- 172.21.15.105:0/1465420703 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f63180668d0 0x7f6318068d90 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:13.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.885+0000 7f633d432640 1 --2- 172.21.15.105:0/1465420703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633815d320 0x7f633815e360 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:13.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.885+0000 7f633d432640 1 --2- 172.21.15.105:0/1465420703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63380ce230 0x7f633815de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:13.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.885+0000 7f633d432640 1 --2- 172.21.15.105:0/1465420703 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63380cdd80 0x7f6338153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:13.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.885+0000 7f633d432640 1 -- 172.21.15.105:0/1465420703 >> 172.21.15.105:0/1465420703 conn(0x7f63380d0df0 msgr2=0x7f63380cd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:13.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.885+0000 7f633d432640 1 -- 172.21.15.105:0/1465420703 shutdown_connections 2023-11-28T18:30:13.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:13.885+0000 7f633d432640 1 -- 172.21.15.105:0/1465420703 wait complete. 2023-11-28T18:30:13.906 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_distributed True 2023-11-28T18:30:14.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.045+0000 7f1f8469c640 1 Processor -- start 2023-11-28T18:30:14.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.045+0000 7f1f8469c640 1 -- start start 2023-11-28T18:30:14.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.049+0000 7f1f8469c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1f7c0cc760 0x7f1f7c164d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:14.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.049+0000 7f1f8469c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7c0ccc10 con 0x7f1f7c159f40 2023-11-28T18:30:14.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.049+0000 7f1f8469c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7c06bf50 con 0x7f1f7c0caf60 2023-11-28T18:30:14.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.049+0000 7f1f8469c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f7c0cb3c0 con 0x7f1f7c0cc760 2023-11-28T18:30:14.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.049+0000 7f1f82438640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f1f7c159f40 0x7f1f7c0ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39068/0 (socket says 172.21.15.105:39068) 2023-11-28T18:30:14.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.049+0000 7f1f82438640 1 -- 172.21.15.105:0/672090117 learned_addr learned my addr 172.21.15.105:0/672090117 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:14.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.049+0000 7f1f82c39640 1 --2- 172.21.15.105:0/672090117 >> v2:172.21.15.105:3300/0 conn(0x7f1f7c0cc760 0x7f1f7c164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:14.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.049+0000 7f1f81436640 1 -- 172.21.15.105:0/672090117 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1943094528 0 0) 0x7f1f7c0ccc10 con 0x7f1f7c159f40 2023-11-28T18:30:14.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.049+0000 7f1f82c39640 1 -- 172.21.15.105:0/672090117 >> v1:172.21.15.148:6789/0 conn(0x7f1f7c0caf60 legacy=0x7f1f7c0ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:14.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.049+0000 7f1f82c39640 1 -- 172.21.15.105:0/672090117 >> v1:172.21.15.105:6789/0 conn(0x7f1f7c159f40 legacy=0x7f1f7c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:14.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.049+0000 7f1f82c39640 1 -- 172.21.15.105:0/672090117 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c15a320 con 0x7f1f7c0cc760 2023-11-28T18:30:14.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.049+0000 7f1f82c39640 1 --2- 172.21.15.105:0/672090117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f7c0cc760 0x7f1f7c164d90 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009d90 tx=0x7f1f7401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9c2f6c276afb4f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:14.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.053+0000 7f1f81436640 1 -- 172.21.15.105:0/672090117 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f740a5950 con 0x7f1f7c0cc760 2023-11-28T18:30:14.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.053+0000 7f1f81436640 1 -- 172.21.15.105:0/672090117 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f740a5ab0 con 0x7f1f7c0cc760 2023-11-28T18:30:14.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.053+0000 7f1f81436640 1 -- 172.21.15.105:0/672090117 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f740a5e40 con 0x7f1f7c0cc760 2023-11-28T18:30:14.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.053+0000 7f1f8469c640 1 -- 172.21.15.105:0/672090117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f7c0cc760 msgr2=0x7f1f7c164d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:14.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.053+0000 7f1f8469c640 1 --2- 172.21.15.105:0/672090117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f7c0cc760 0x7f1f7c164d90 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009d90 tx=0x7f1f7401c890 comp rx=0 tx=0).stop 2023-11-28T18:30:14.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.053+0000 7f1f8469c640 1 -- 172.21.15.105:0/672090117 shutdown_connections 2023-11-28T18:30:14.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.053+0000 7f1f8469c640 1 --2- 172.21.15.105:0/672090117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f7c0cc760 0x7f1f7c164d90 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:14.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.053+0000 7f1f8469c640 1 -- 172.21.15.105:0/672090117 >> 172.21.15.105:0/672090117 conn(0x7f1f7c0ceb90 msgr2=0x7f1f7c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:14.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.053+0000 7f1f8469c640 1 -- 172.21.15.105:0/672090117 shutdown_connections 2023-11-28T18:30:14.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.053+0000 7f1f8469c640 1 -- 172.21.15.105:0/672090117 wait complete. 2023-11-28T18:30:14.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.053+0000 7f1f8469c640 1 Processor -- start 2023-11-28T18:30:14.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.053+0000 7f1f8469c640 1 -- start start 2023-11-28T18:30:14.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.053+0000 7f1f8469c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f7c0caf60 0x7f1f7c158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:14.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.053+0000 7f1f8469c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f7c0cc760 0x7f1f7c143190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:14.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.053+0000 7f1f8469c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f7c159f40 0x7f1f7c14b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:14.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.053+0000 7f1f8469c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c157f10 con 0x7f1f7c159f40 2023-11-28T18:30:14.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.053+0000 7f1f8469c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c1436d0 con 0x7f1f7c0cc760 2023-11-28T18:30:14.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.053+0000 7f1f8469c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c159570 con 0x7f1f7c0caf60 2023-11-28T18:30:14.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.053+0000 7f1f82c39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f7c159f40 0x7f1f7c14b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:14.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.053+0000 7f1f82c39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f7c159f40 0x7f1f7c14b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46712/0 (socket says 172.21.15.105:46712) 2023-11-28T18:30:14.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.053+0000 7f1f82c39640 1 -- 172.21.15.105:0/3531058535 learned_addr learned my addr 172.21.15.105:0/3531058535 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:14.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.057+0000 7f1f82438640 1 --2- 172.21.15.105:0/3531058535 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f7c0caf60 0x7f1f7c158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:14.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.057+0000 7f1f81c37640 1 --2- 172.21.15.105:0/3531058535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f7c0cc760 0x7f1f7c143190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:14.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.057+0000 7f1f82c39640 1 -- 172.21.15.105:0/3531058535 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f7c0caf60 msgr2=0x7f1f7c158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:14.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.057+0000 7f1f82c39640 1 --2- 172.21.15.105:0/3531058535 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f7c0caf60 0x7f1f7c158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:14.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.057+0000 7f1f82c39640 1 -- 172.21.15.105:0/3531058535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f7c0cc760 msgr2=0x7f1f7c143190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:14.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.057+0000 7f1f82c39640 1 --2- 172.21.15.105:0/3531058535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f7c0cc760 0x7f1f7c143190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:14.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.057+0000 7f1f82c39640 1 -- 172.21.15.105:0/3531058535 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f74009a70 con 0x7f1f7c159f40 2023-11-28T18:30:14.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.057+0000 7f1f82438640 1 --2- 172.21.15.105:0/3531058535 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f7c0caf60 0x7f1f7c158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:14.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.057+0000 7f1f81c37640 1 --2- 172.21.15.105:0/3531058535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f7c0cc760 0x7f1f7c143190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:14.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.057+0000 7f1f82c39640 1 --2- 172.21.15.105:0/3531058535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f7c159f40 0x7f1f7c14b650 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f1f740b0040 tx=0x7f1f74019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:14.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.057+0000 7f1f72ffd640 1 -- 172.21.15.105:0/3531058535 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f740ade40 con 0x7f1f7c159f40 2023-11-28T18:30:14.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.057+0000 7f1f72ffd640 1 -- 172.21.15.105:0/3531058535 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f740b9450 con 0x7f1f7c159f40 2023-11-28T18:30:14.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.057+0000 7f1f8469c640 1 -- 172.21.15.105:0/3531058535 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7c1575e0 con 0x7f1f7c159f40 2023-11-28T18:30:14.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.057+0000 7f1f72ffd640 1 -- 172.21.15.105:0/3531058535 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f740b9800 con 0x7f1f7c159f40 2023-11-28T18:30:14.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.057+0000 7f1f8469c640 1 -- 172.21.15.105:0/3531058535 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7c157ba0 con 0x7f1f7c159f40 2023-11-28T18:30:14.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.057+0000 7f1f8469c640 1 -- 172.21.15.105:0/3531058535 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f7c158080 con 0x7f1f7c159f40 2023-11-28T18:30:14.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.057+0000 7f1f72ffd640 1 -- 172.21.15.105:0/3531058535 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f1f740b9960 con 0x7f1f7c159f40 2023-11-28T18:30:14.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.057+0000 7f1f72ffd640 1 --2- 172.21.15.105:0/3531058535 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1f600668d0 0x7f1f60068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:14.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.057+0000 7f1f72ffd640 1 -- 172.21.15.105:0/3531058535 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1f7411fec0 con 0x7f1f7c159f40 2023-11-28T18:30:14.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.061+0000 7f1f82438640 1 --2- 172.21.15.105:0/3531058535 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1f600668d0 0x7f1f60068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:14.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.061+0000 7f1f82438640 1 --2- 172.21.15.105:0/3531058535 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1f600668d0 0x7f1f60068d90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1f78000f70 tx=0x7f1f78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:14.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.061+0000 7f1f72ffd640 1 -- 172.21.15.105:0/3531058535 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1f740ecfe0 con 0x7f1f7c159f40 2023-11-28T18:30:14.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.221+0000 7f1f8469c640 1 -- 172.21.15.105:0/3531058535 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_distributed}] v 0) v1 -- 0x7f1f7c067570 con 0x7f1f7c159f40 2023-11-28T18:30:14.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.221+0000 7f1f72ffd640 1 -- 172.21.15.105:0/3531058535 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_distributed}]=0 v53)=0 v53) v1 ==== 133+0+0 (secure 0 0 0) 0x7f1f740f0600 con 0x7f1f7c159f40 2023-11-28T18:30:14.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.225+0000 7f1f8469c640 1 -- 172.21.15.105:0/3531058535 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1f600668d0 msgr2=0x7f1f60068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.225+0000 7f1f8469c640 1 --2- 172.21.15.105:0/3531058535 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1f600668d0 0x7f1f60068d90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1f78000f70 tx=0x7f1f78009040 comp rx=0 tx=0).stop 2023-11-28T18:30:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.225+0000 7f1f8469c640 1 -- 172.21.15.105:0/3531058535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f7c159f40 msgr2=0x7f1f7c14b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.225+0000 7f1f8469c640 1 --2- 172.21.15.105:0/3531058535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f7c159f40 0x7f1f7c14b650 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f1f740b0040 tx=0x7f1f74019630 comp rx=0 tx=0).stop 2023-11-28T18:30:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.225+0000 7f1f8469c640 1 -- 172.21.15.105:0/3531058535 shutdown_connections 2023-11-28T18:30:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.225+0000 7f1f8469c640 1 --2- 172.21.15.105:0/3531058535 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1f600668d0 0x7f1f60068d90 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.225+0000 7f1f8469c640 1 --2- 172.21.15.105:0/3531058535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f7c159f40 0x7f1f7c14b650 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.225+0000 7f1f8469c640 1 --2- 172.21.15.105:0/3531058535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f7c0cc760 0x7f1f7c143190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.225+0000 7f1f8469c640 1 --2- 172.21.15.105:0/3531058535 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f7c0caf60 0x7f1f7c158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.225+0000 7f1f8469c640 1 -- 172.21.15.105:0/3531058535 >> 172.21.15.105:0/3531058535 conn(0x7f1f7c0ceb90 msgr2=0x7f1f7c0cdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.225+0000 7f1f8469c640 1 -- 172.21.15.105:0/3531058535 shutdown_connections 2023-11-28T18:30:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.225+0000 7f1f8469c640 1 -- 172.21.15.105:0/3531058535 wait complete. 2023-11-28T18:30:14.244 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random_max 1.0 2023-11-28T18:30:14.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.385+0000 7fc62c817640 1 Processor -- start 2023-11-28T18:30:14.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.385+0000 7fc62c817640 1 -- start start 2023-11-28T18:30:14.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.385+0000 7fc62c817640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc628159f40 0x7fc6280caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:14.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.385+0000 7fc62c817640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc6280cc760 0x7fc6280ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.385+0000 7fc62c817640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6280ad680 0x7fc62815da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.385+0000 7fc62c817640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc62806bf50 con 0x7fc6280cc760 2023-11-28T18:30:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.385+0000 7fc62c817640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6280ccbc0 con 0x7fc628159f40 2023-11-28T18:30:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.385+0000 7fc62c817640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc62815a320 con 0x7fc6280ad680 2023-11-28T18:30:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.385+0000 7fc61dd9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc6280cc760 0x7fc6280ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.385+0000 7fc61dd9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc6280cc760 0x7fc6280ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46726/0 (socket says 172.21.15.105:46726) 2023-11-28T18:30:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.385+0000 7fc61dd9b640 1 -- 172.21.15.105:0/22527210 learned_addr learned my addr 172.21.15.105:0/22527210 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.385+0000 7fc626d9d640 1 --2- 172.21.15.105:0/22527210 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6280ad680 0x7fc62815da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.385+0000 7fc62659c640 1 --2- 172.21.15.105:0/22527210 >> v2:172.21.15.148:3300/0 conn(0x7fc628159f40 0x7fc6280caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.385+0000 7fc61dd9b640 1 -- 172.21.15.105:0/22527210 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6280ad680 msgr2=0x7fc62815da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:14.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.385+0000 7fc61dd9b640 1 --2- 172.21.15.105:0/22527210 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6280ad680 0x7fc62815da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:14.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.385+0000 7fc61dd9b640 1 -- 172.21.15.105:0/22527210 >> v2:172.21.15.148:3300/0 conn(0x7fc628159f40 msgr2=0x7fc6280caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:14.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.385+0000 7fc61dd9b640 1 --2- 172.21.15.105:0/22527210 >> v2:172.21.15.148:3300/0 conn(0x7fc628159f40 0x7fc6280caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:14.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.385+0000 7fc61dd9b640 1 -- 172.21.15.105:0/22527210 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc628071380 con 0x7fc6280cc760 2023-11-28T18:30:14.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.389+0000 7fc61dd9b640 1 --2- 172.21.15.105:0/22527210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6280cc760 0x7fc6280ad140 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fc62806e500 tx=0x7fc61001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee0bdbaee0ff791 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:14.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.389+0000 7fc625d9b640 1 -- 172.21.15.105:0/22527210 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6100a5790 con 0x7fc6280cc760 2023-11-28T18:30:14.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.389+0000 7fc625d9b640 1 -- 172.21.15.105:0/22527210 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6100a58f0 con 0x7fc6280cc760 2023-11-28T18:30:14.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.389+0000 7fc625d9b640 1 -- 172.21.15.105:0/22527210 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6100a5c80 con 0x7fc6280cc760 2023-11-28T18:30:14.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.389+0000 7fc62c817640 1 -- 172.21.15.105:0/22527210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6280cc760 msgr2=0x7fc6280ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:14.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.389+0000 7fc62c817640 1 --2- 172.21.15.105:0/22527210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6280cc760 0x7fc6280ad140 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fc62806e500 tx=0x7fc61001c910 comp rx=0 tx=0).stop 2023-11-28T18:30:14.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.389+0000 7fc62c817640 1 -- 172.21.15.105:0/22527210 shutdown_connections 2023-11-28T18:30:14.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.389+0000 7fc62c817640 1 --2- 172.21.15.105:0/22527210 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6280ad680 0x7fc62815da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:14.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.389+0000 7fc62c817640 1 --2- 172.21.15.105:0/22527210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6280cc760 0x7fc6280ad140 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:14.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.389+0000 7fc62c817640 1 --2- 172.21.15.105:0/22527210 >> v2:172.21.15.148:3300/0 conn(0x7fc628159f40 0x7fc6280caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:14.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.389+0000 7fc62c817640 1 -- 172.21.15.105:0/22527210 >> 172.21.15.105:0/22527210 conn(0x7fc6280ceb90 msgr2=0x7fc6280cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:14.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.389+0000 7fc62c817640 1 -- 172.21.15.105:0/22527210 shutdown_connections 2023-11-28T18:30:14.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.389+0000 7fc62c817640 1 -- 172.21.15.105:0/22527210 wait complete. 2023-11-28T18:30:14.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc62c817640 1 Processor -- start 2023-11-28T18:30:14.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc62c817640 1 -- start start 2023-11-28T18:30:14.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc62c817640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6280ad680 0x7fc628147460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:14.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc62c817640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6280cc760 0x7fc6280af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:14.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc62c817640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc628159f40 0x7fc628147f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:14.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc62c817640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6280aeec0 con 0x7fc6280cc760 2023-11-28T18:30:14.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc62c817640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc628158d70 con 0x7fc628159f40 2023-11-28T18:30:14.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc62c817640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6281584c0 con 0x7fc6280ad680 2023-11-28T18:30:14.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc61dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6280cc760 0x7fc6280af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:14.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc61dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6280cc760 0x7fc6280af110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46734/0 (socket says 172.21.15.105:46734) 2023-11-28T18:30:14.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc61dd9b640 1 -- 172.21.15.105:0/31352950 learned_addr learned my addr 172.21.15.105:0/31352950 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:14.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc626d9d640 1 --2- 172.21.15.105:0/31352950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc628159f40 0x7fc628147f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:14.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc62659c640 1 --2- 172.21.15.105:0/31352950 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6280ad680 0x7fc628147460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:14.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc61dd9b640 1 -- 172.21.15.105:0/31352950 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6280ad680 msgr2=0x7fc628147460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:14.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc61dd9b640 1 --2- 172.21.15.105:0/31352950 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6280ad680 0x7fc628147460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:14.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc61dd9b640 1 -- 172.21.15.105:0/31352950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc628159f40 msgr2=0x7fc628147f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:14.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc61dd9b640 1 --2- 172.21.15.105:0/31352950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc628159f40 0x7fc628147f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:14.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc61dd9b640 1 -- 172.21.15.105:0/31352950 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc610009b30 con 0x7fc6280cc760 2023-11-28T18:30:14.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc61dd9b640 1 --2- 172.21.15.105:0/31352950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6280cc760 0x7fc6280af110 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fc61001afc0 tx=0x7fc6100196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:14.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc61f7fe640 1 -- 172.21.15.105:0/31352950 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6100b8670 con 0x7fc6280cc760 2023-11-28T18:30:14.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc61f7fe640 1 -- 172.21.15.105:0/31352950 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6100b87d0 con 0x7fc6280cc760 2023-11-28T18:30:14.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc61f7fe640 1 -- 172.21.15.105:0/31352950 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6100b8b80 con 0x7fc6280cc760 2023-11-28T18:30:14.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc62c817640 1 -- 172.21.15.105:0/31352950 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6280aebe0 con 0x7fc6280cc760 2023-11-28T18:30:14.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.393+0000 7fc62c817640 1 -- 172.21.15.105:0/31352950 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc628158990 con 0x7fc6280cc760 2023-11-28T18:30:14.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.397+0000 7fc62c817640 1 -- 172.21.15.105:0/31352950 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc628067570 con 0x7fc6280cc760 2023-11-28T18:30:14.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.397+0000 7fc61f7fe640 1 -- 172.21.15.105:0/31352950 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc6100b8ce0 con 0x7fc6280cc760 2023-11-28T18:30:14.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.397+0000 7fc61f7fe640 1 --2- 172.21.15.105:0/31352950 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6080668d0 0x7fc608068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:14.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.397+0000 7fc61f7fe640 1 -- 172.21.15.105:0/31352950 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc61011fb00 con 0x7fc6280cc760 2023-11-28T18:30:14.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.397+0000 7fc62659c640 1 --2- 172.21.15.105:0/31352950 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6080668d0 0x7fc608068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:14.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.397+0000 7fc62659c640 1 --2- 172.21.15.105:0/31352950 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6080668d0 0x7fc608068d90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc6180079c0 tx=0x7fc618008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:14.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.401+0000 7fc61f7fe640 1 -- 172.21.15.105:0/31352950 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc610121e60 con 0x7fc6280cc760 2023-11-28T18:30:14.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.549+0000 7fc62c817640 1 -- 172.21.15.105:0/31352950 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random_max}] v 0) v1 -- 0x7fc6280cf950 con 0x7fc6280cc760 2023-11-28T18:30:14.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.557+0000 7fc61f7fe640 1 -- 172.21.15.105:0/31352950 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random_max}]=0 v54)=0 v54) v1 ==== 131+0+0 (secure 0 0 0) 0x7fc6100f0850 con 0x7fc6280cc760 2023-11-28T18:30:14.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.557+0000 7fc62c817640 1 -- 172.21.15.105:0/31352950 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6080668d0 msgr2=0x7fc608068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:14.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.557+0000 7fc62c817640 1 --2- 172.21.15.105:0/31352950 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6080668d0 0x7fc608068d90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc6180079c0 tx=0x7fc618008040 comp rx=0 tx=0).stop 2023-11-28T18:30:14.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.557+0000 7fc62c817640 1 -- 172.21.15.105:0/31352950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6280cc760 msgr2=0x7fc6280af110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:14.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.557+0000 7fc62c817640 1 --2- 172.21.15.105:0/31352950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6280cc760 0x7fc6280af110 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fc61001afc0 tx=0x7fc6100196b0 comp rx=0 tx=0).stop 2023-11-28T18:30:14.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.557+0000 7fc62c817640 1 -- 172.21.15.105:0/31352950 shutdown_connections 2023-11-28T18:30:14.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.557+0000 7fc62c817640 1 --2- 172.21.15.105:0/31352950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc628159f40 0x7fc628147f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:14.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.557+0000 7fc62c817640 1 --2- 172.21.15.105:0/31352950 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc6080668d0 0x7fc608068d90 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:14.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.557+0000 7fc62c817640 1 --2- 172.21.15.105:0/31352950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6280cc760 0x7fc6280af110 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:14.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.557+0000 7fc62c817640 1 --2- 172.21.15.105:0/31352950 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6280ad680 0x7fc628147460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:14.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.557+0000 7fc62c817640 1 -- 172.21.15.105:0/31352950 >> 172.21.15.105:0/31352950 conn(0x7fc6280ceb90 msgr2=0x7fc6280cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:14.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.557+0000 7fc62c817640 1 -- 172.21.15.105:0/31352950 shutdown_connections 2023-11-28T18:30:14.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:14.557+0000 7fc62c817640 1 -- 172.21.15.105:0/31352950 wait complete. 2023-11-28T18:30:14.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:14.579 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-28T18:30:14.579 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:30:14.580 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:30:14.580 DEBUG:teuthology.orchestra.run.smithi105:> # Use up a random number of inode numbers so the ephemeral pinning is not the same every test. 2023-11-28T18:30:14.580 DEBUG:teuthology.orchestra.run.smithi105:> mkdir .inode_number_thrash 2023-11-28T18:30:14.580 DEBUG:teuthology.orchestra.run.smithi105:> count=$((RANDOM % 1024)) 2023-11-28T18:30:14.580 DEBUG:teuthology.orchestra.run.smithi105:> for ((i = 0; i < count; i++)); do touch .inode_number_thrash/$i; done 2023-11-28T18:30:14.580 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf .inode_number_thrash 2023-11-28T18:30:14.580 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:30:23.596 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-11-28T18:30:23.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.737+0000 7fa63be32640 1 Processor -- start 2023-11-28T18:30:23.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.737+0000 7fa63be32640 1 -- start start 2023-11-28T18:30:23.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.737+0000 7fa63be32640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa634159f40 0x7fa6340caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:23.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.737+0000 7fa63be32640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa63406bf50 con 0x7fa6340ad850 2023-11-28T18:30:23.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.737+0000 7fa63be32640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6340ccbc0 con 0x7fa6340cc760 2023-11-28T18:30:23.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.737+0000 7fa63be32640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63415a320 con 0x7fa634159f40 2023-11-28T18:30:23.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.737+0000 7fa63a3cf640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa6340ad850 0x7fa6340701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49118/0 (socket says 172.21.15.105:49118) 2023-11-28T18:30:23.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.737+0000 7fa63a3cf640 1 -- 172.21.15.105:0/2294197294 learned_addr learned my addr 172.21.15.105:0/2294197294 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:23.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.737+0000 7fa639bce640 1 --2- 172.21.15.105:0/2294197294 >> v2:172.21.15.148:3300/0 conn(0x7fa634159f40 0x7fa6340caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:23.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.737+0000 7fa638bcc640 1 -- 172.21.15.105:0/2294197294 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 295088860 0 0) 0x7fa63406bf50 con 0x7fa6340ad850 2023-11-28T18:30:23.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.737+0000 7fa638bcc640 1 -- 172.21.15.105:0/2294197294 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa620003530 con 0x7fa6340ad850 2023-11-28T18:30:23.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.737+0000 7fa639bce640 1 -- 172.21.15.105:0/2294197294 >> v1:172.21.15.148:6789/0 conn(0x7fa6340cc760 legacy=0x7fa6340ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:23.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.737+0000 7fa639bce640 1 -- 172.21.15.105:0/2294197294 >> v1:172.21.15.105:6789/0 conn(0x7fa6340ad850 legacy=0x7fa6340701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:23.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.737+0000 7fa639bce640 1 -- 172.21.15.105:0/2294197294 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa634071380 con 0x7fa634159f40 2023-11-28T18:30:23.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.741+0000 7fa639bce640 1 --2- 172.21.15.105:0/2294197294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa634159f40 0x7fa6340caf60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa6300097c0 tx=0x7fa63001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97fcdea45508f07f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:23.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.741+0000 7fa638bcc640 1 -- 172.21.15.105:0/2294197294 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6300ac620 con 0x7fa634159f40 2023-11-28T18:30:23.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.741+0000 7fa638bcc640 1 -- 172.21.15.105:0/2294197294 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6300ac780 con 0x7fa634159f40 2023-11-28T18:30:23.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.741+0000 7fa638bcc640 1 -- 172.21.15.105:0/2294197294 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6300acaf0 con 0x7fa634159f40 2023-11-28T18:30:23.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.741+0000 7fa63be32640 1 -- 172.21.15.105:0/2294197294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa634159f40 msgr2=0x7fa6340caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:23.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.741+0000 7fa63be32640 1 --2- 172.21.15.105:0/2294197294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa634159f40 0x7fa6340caf60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa6300097c0 tx=0x7fa63001c3c0 comp rx=0 tx=0).stop 2023-11-28T18:30:23.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.741+0000 7fa63be32640 1 -- 172.21.15.105:0/2294197294 shutdown_connections 2023-11-28T18:30:23.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.741+0000 7fa63be32640 1 --2- 172.21.15.105:0/2294197294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa634159f40 0x7fa6340caf60 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:23.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.741+0000 7fa63be32640 1 -- 172.21.15.105:0/2294197294 >> 172.21.15.105:0/2294197294 conn(0x7fa6340ceb90 msgr2=0x7fa6340cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:23.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.741+0000 7fa63be32640 1 -- 172.21.15.105:0/2294197294 shutdown_connections 2023-11-28T18:30:23.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.741+0000 7fa63be32640 1 -- 172.21.15.105:0/2294197294 wait complete. 2023-11-28T18:30:23.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa63be32640 1 Processor -- start 2023-11-28T18:30:23.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa63be32640 1 -- start start 2023-11-28T18:30:23.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa63be32640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6340ad850 0x7fa63415cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:23.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa63be32640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6340cc760 0x7fa6340c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:23.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa63be32640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa634159f40 0x7fa6340b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:23.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa63be32640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6340b2aa0 con 0x7fa6340cc760 2023-11-28T18:30:23.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa63be32640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63409c180 con 0x7fa634159f40 2023-11-28T18:30:23.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa63be32640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6340b1e20 con 0x7fa6340ad850 2023-11-28T18:30:23.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa6393cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6340cc760 0x7fa6340c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:23.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa6393cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6340cc760 0x7fa6340c91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55246/0 (socket says 172.21.15.105:55246) 2023-11-28T18:30:23.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa6393cd640 1 -- 172.21.15.105:0/381744291 learned_addr learned my addr 172.21.15.105:0/381744291 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:23.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa63a3cf640 1 --2- 172.21.15.105:0/381744291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa634159f40 0x7fa6340b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:23.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa6393cd640 1 -- 172.21.15.105:0/381744291 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6340ad850 msgr2=0x7fa63415cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:23.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa6393cd640 1 --2- 172.21.15.105:0/381744291 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6340ad850 0x7fa63415cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:23.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa6393cd640 1 -- 172.21.15.105:0/381744291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa634159f40 msgr2=0x7fa6340b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:23.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa6393cd640 1 --2- 172.21.15.105:0/381744291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa634159f40 0x7fa6340b2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:23.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa6393cd640 1 -- 172.21.15.105:0/381744291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa630009420 con 0x7fa6340cc760 2023-11-28T18:30:23.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa6393cd640 1 --2- 172.21.15.105:0/381744291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6340cc760 0x7fa6340c91f0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fa62400c420 tx=0x7fa62400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:23.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa62a7fc640 1 -- 172.21.15.105:0/381744291 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6240052d0 con 0x7fa6340cc760 2023-11-28T18:30:23.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa62a7fc640 1 -- 172.21.15.105:0/381744291 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa624005430 con 0x7fa6340cc760 2023-11-28T18:30:23.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa63be32640 1 -- 172.21.15.105:0/381744291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6340b2160 con 0x7fa6340cc760 2023-11-28T18:30:23.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa62a7fc640 1 -- 172.21.15.105:0/381744291 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa624007260 con 0x7fa6340cc760 2023-11-28T18:30:23.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa63be32640 1 -- 172.21.15.105:0/381744291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63415c210 con 0x7fa6340cc760 2023-11-28T18:30:23.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.745+0000 7fa63be32640 1 -- 172.21.15.105:0/381744291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6340b2be0 con 0x7fa6340cc760 2023-11-28T18:30:23.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.749+0000 7fa62a7fc640 1 -- 172.21.15.105:0/381744291 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa6240073c0 con 0x7fa6340cc760 2023-11-28T18:30:23.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.749+0000 7fa62a7fc640 1 --2- 172.21.15.105:0/381744291 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6180668d0 0x7fa618068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:23.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.749+0000 7fa62a7fc640 1 -- 172.21.15.105:0/381744291 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa624083a50 con 0x7fa6340cc760 2023-11-28T18:30:23.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.749+0000 7fa639bce640 1 --2- 172.21.15.105:0/381744291 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6180668d0 0x7fa618068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:23.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.749+0000 7fa639bce640 1 --2- 172.21.15.105:0/381744291 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6180668d0 0x7fa618068d90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa6300ae000 tx=0x7fa630004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:23.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.753+0000 7fa62a7fc640 1 -- 172.21.15.105:0/381744291 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa6340b2be0 con 0x7fa6340cc760 2023-11-28T18:30:23.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:23.933+0000 7fa63be32640 1 -- 172.21.15.105:0/381744291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7fa6340b2be0 con 0x7fa6340cc760 2023-11-28T18:30:24.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.149+0000 7fa62a7fc640 1 -- 172.21.15.105:0/381744291 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v104) v1 ==== 105+0+0 (secure 0 0 0) 0x7fa6340b2be0 con 0x7fa6340cc760 2023-11-28T18:30:24.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.153+0000 7fa63be32640 1 -- 172.21.15.105:0/381744291 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6180668d0 msgr2=0x7fa618068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:24.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.153+0000 7fa63be32640 1 --2- 172.21.15.105:0/381744291 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6180668d0 0x7fa618068d90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa6300ae000 tx=0x7fa630004740 comp rx=0 tx=0).stop 2023-11-28T18:30:24.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.153+0000 7fa63be32640 1 -- 172.21.15.105:0/381744291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6340cc760 msgr2=0x7fa6340c91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:24.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.153+0000 7fa63be32640 1 --2- 172.21.15.105:0/381744291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6340cc760 0x7fa6340c91f0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fa62400c420 tx=0x7fa62400c8e0 comp rx=0 tx=0).stop 2023-11-28T18:30:24.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.153+0000 7fa63be32640 1 -- 172.21.15.105:0/381744291 shutdown_connections 2023-11-28T18:30:24.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.153+0000 7fa63be32640 1 --2- 172.21.15.105:0/381744291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa634159f40 0x7fa6340b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:24.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.153+0000 7fa63be32640 1 --2- 172.21.15.105:0/381744291 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa6180668d0 0x7fa618068d90 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:24.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.153+0000 7fa63be32640 1 --2- 172.21.15.105:0/381744291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6340cc760 0x7fa6340c91f0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:24.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.153+0000 7fa63be32640 1 --2- 172.21.15.105:0/381744291 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6340ad850 0x7fa63415cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:24.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.153+0000 7fa63be32640 1 -- 172.21.15.105:0/381744291 >> 172.21.15.105:0/381744291 conn(0x7fa6340ceb90 msgr2=0x7fa6340cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:24.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.153+0000 7fa63be32640 1 -- 172.21.15.105:0/381744291 shutdown_connections 2023-11-28T18:30:24.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.153+0000 7fa63be32640 1 -- 172.21.15.105:0/381744291 wait complete. 2023-11-28T18:30:24.179 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:30:24.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.345+0000 7f98c6cea640 1 Processor -- start 2023-11-28T18:30:24.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.345+0000 7f98c6cea640 1 -- start start 2023-11-28T18:30:24.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.345+0000 7f98c6cea640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f98c0071960 0x7f98c015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:24.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.345+0000 7f98c6cea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98c015b030 0x7f98c00709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:24.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.345+0000 7f98c6cea640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c00701d0 con 0x7f98c00b4290 2023-11-28T18:30:24.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.345+0000 7f98c6cea640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c006df80 con 0x7f98c0071960 2023-11-28T18:30:24.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.345+0000 7f98c6cea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c0071df0 con 0x7f98c015b030 2023-11-28T18:30:24.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.349+0000 7f98b7fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f98c0071960 0x7f98c015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:24.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.349+0000 7f98b7fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f98c0071960 0x7f98c015f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55258/0 (socket says 172.21.15.105:55258) 2023-11-28T18:30:24.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.349+0000 7f98b7fff640 1 -- 172.21.15.105:0/2728603851 learned_addr learned my addr 172.21.15.105:0/2728603851 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:24.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.349+0000 7f98c5287640 1 --2- 172.21.15.105:0/2728603851 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98c015b030 0x7f98c00709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:24.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.349+0000 7f98b7fff640 1 -- 172.21.15.105:0/2728603851 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98c015b030 msgr2=0x7f98c00709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:24.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.349+0000 7f98b7fff640 1 --2- 172.21.15.105:0/2728603851 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98c015b030 0x7f98c00709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:24.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.349+0000 7f98b7fff640 1 -- 172.21.15.105:0/2728603851 >> v1:172.21.15.148:6789/0 conn(0x7f98c00b4290 legacy=0x7f98c00b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:24.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.349+0000 7f98b7fff640 1 -- 172.21.15.105:0/2728603851 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c00720d0 con 0x7f98c0071960 2023-11-28T18:30:24.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.349+0000 7f98b7fff640 1 --2- 172.21.15.105:0/2728603851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98c0071960 0x7f98c015f3b0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f98b0009870 tx=0x7f98b001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d2b8d0f77d1bf9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:24.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.349+0000 7f98b77fe640 1 -- 172.21.15.105:0/2728603851 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b00ab620 con 0x7f98c0071960 2023-11-28T18:30:24.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.349+0000 7f98b77fe640 1 -- 172.21.15.105:0/2728603851 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b00ab780 con 0x7f98c0071960 2023-11-28T18:30:24.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.349+0000 7f98b77fe640 1 -- 172.21.15.105:0/2728603851 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b00abb10 con 0x7f98c0071960 2023-11-28T18:30:24.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.353+0000 7f98c6cea640 1 -- 172.21.15.105:0/2728603851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98c0071960 msgr2=0x7f98c015f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:24.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.353+0000 7f98c6cea640 1 --2- 172.21.15.105:0/2728603851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98c0071960 0x7f98c015f3b0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f98b0009870 tx=0x7f98b001c510 comp rx=0 tx=0).stop 2023-11-28T18:30:24.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.353+0000 7f98c6cea640 1 -- 172.21.15.105:0/2728603851 shutdown_connections 2023-11-28T18:30:24.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.353+0000 7f98c6cea640 1 --2- 172.21.15.105:0/2728603851 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98c015b030 0x7f98c00709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:24.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.353+0000 7f98c6cea640 1 --2- 172.21.15.105:0/2728603851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98c0071960 0x7f98c015f3b0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:24.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.353+0000 7f98c6cea640 1 -- 172.21.15.105:0/2728603851 >> 172.21.15.105:0/2728603851 conn(0x7f98c00ceb90 msgr2=0x7f98c00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:24.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.353+0000 7f98c6cea640 1 -- 172.21.15.105:0/2728603851 shutdown_connections 2023-11-28T18:30:24.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.353+0000 7f98c6cea640 1 -- 172.21.15.105:0/2728603851 wait complete. 2023-11-28T18:30:24.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.353+0000 7f98c6cea640 1 Processor -- start 2023-11-28T18:30:24.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.353+0000 7f98c6cea640 1 -- start start 2023-11-28T18:30:24.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.353+0000 7f98c6cea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98c00b4290 0x7f98c0150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:24.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.353+0000 7f98c6cea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98c015b030 0x7f98c0099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:24.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.353+0000 7f98c6cea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c00cd8e0 0x7f98c0159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:24.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.353+0000 7f98c6cea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c0151590 con 0x7f98c00b4290 2023-11-28T18:30:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.353+0000 7f98c6cea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c0150cf0 con 0x7f98c00cd8e0 2023-11-28T18:30:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.353+0000 7f98c6cea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c0150fd0 con 0x7f98c015b030 2023-11-28T18:30:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.353+0000 7f98c4a86640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98c00b4290 0x7f98c0150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.353+0000 7f98c4a86640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98c00b4290 0x7f98c0150630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55274/0 (socket says 172.21.15.105:55274) 2023-11-28T18:30:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.353+0000 7f98c4a86640 1 -- 172.21.15.105:0/3625438128 learned_addr learned my addr 172.21.15.105:0/3625438128 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.353+0000 7f98b7fff640 1 --2- 172.21.15.105:0/3625438128 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98c015b030 0x7f98c0099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.353+0000 7f98c5287640 1 --2- 172.21.15.105:0/3625438128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c00cd8e0 0x7f98c0159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.357+0000 7f98c4a86640 1 -- 172.21.15.105:0/3625438128 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98c015b030 msgr2=0x7f98c0099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.357+0000 7f98c4a86640 1 --2- 172.21.15.105:0/3625438128 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98c015b030 0x7f98c0099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.357+0000 7f98c4a86640 1 -- 172.21.15.105:0/3625438128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c00cd8e0 msgr2=0x7f98c0159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.357+0000 7f98c4a86640 1 --2- 172.21.15.105:0/3625438128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c00cd8e0 0x7f98c0159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.357+0000 7f98c4a86640 1 -- 172.21.15.105:0/3625438128 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b00094d0 con 0x7f98c00b4290 2023-11-28T18:30:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.357+0000 7f98c5287640 1 --2- 172.21.15.105:0/3625438128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c00cd8e0 0x7f98c0159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.357+0000 7f98c4a86640 1 --2- 172.21.15.105:0/3625438128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98c00b4290 0x7f98c0150630 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00c500 tx=0x7f98bc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.357+0000 7f98b57fa640 1 -- 172.21.15.105:0/3625438128 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc005220 con 0x7f98c00b4290 2023-11-28T18:30:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.357+0000 7f98b57fa640 1 -- 172.21.15.105:0/3625438128 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98bc005380 con 0x7f98c00b4290 2023-11-28T18:30:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.357+0000 7f98b57fa640 1 -- 172.21.15.105:0/3625438128 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc007270 con 0x7f98c00b4290 2023-11-28T18:30:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.357+0000 7f98c6cea640 1 -- 172.21.15.105:0/3625438128 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c01512b0 con 0x7f98c00b4290 2023-11-28T18:30:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.357+0000 7f98c6cea640 1 -- 172.21.15.105:0/3625438128 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c00cdd90 con 0x7f98c00b4290 2023-11-28T18:30:24.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.357+0000 7f98c6cea640 1 -- 172.21.15.105:0/3625438128 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c0152800 con 0x7f98c00b4290 2023-11-28T18:30:24.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.357+0000 7f98b57fa640 1 -- 172.21.15.105:0/3625438128 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f98bc0073d0 con 0x7f98c00b4290 2023-11-28T18:30:24.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.357+0000 7f98b57fa640 1 --2- 172.21.15.105:0/3625438128 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f98ac0668d0 0x7f98ac068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:24.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.357+0000 7f98b57fa640 1 -- 172.21.15.105:0/3625438128 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f98bc083ee0 con 0x7f98c00b4290 2023-11-28T18:30:24.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.361+0000 7f98b7fff640 1 --2- 172.21.15.105:0/3625438128 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f98ac0668d0 0x7f98ac068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:24.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.361+0000 7f98b7fff640 1 --2- 172.21.15.105:0/3625438128 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f98ac0668d0 0x7f98ac068d90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f98b00af000 tx=0x7f98b00b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:24.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.361+0000 7f98b57fa640 1 -- 172.21.15.105:0/3625438128 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f98c0152800 con 0x7f98c00b4290 2023-11-28T18:30:24.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.541+0000 7f98c6cea640 1 -- 172.21.15.105:0/3625438128 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98c0152800 con 0x7f98c00b4290 2023-11-28T18:30:24.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.541+0000 7f98b57fa640 1 -- 172.21.15.105:0/3625438128 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4946 (secure 0 0 0) 0x7f98c0152800 con 0x7f98c00b4290 2023-11-28T18:30:24.543 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:24.543 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":105,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/2603493310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2603493310},{"type":"v1","addr":"172.21.15.148:6834","nonce":2603493310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":6803,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3664281230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":3664281230},{"type":"v1","addr":"172.21.15.148:6835","nonce":3664281230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:30:05.525117+0000","modified":"2023-11-28T18:30:24.155609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1],"up":{"mds_0":8145,"mds_1":8142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8142":{"gid":8142,"name":"a","rank":1,"incarnation":105,"state":"up:creating","state_seq":1,"addr":"172.21.15.105:6836/3968059262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3968059262},{"type":"v1","addr":"172.21.15.105:6836","nonce":3968059262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8145":{"gid":8145,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2782729992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2782729992},{"type":"v1","addr":"172.21.15.105:6837","nonce":2782729992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-11-28T18:30:24.543 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 105 2023-11-28T18:30:24.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.545+0000 7f98c6cea640 1 -- 172.21.15.105:0/3625438128 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f98ac0668d0 msgr2=0x7f98ac068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:24.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.545+0000 7f98c6cea640 1 --2- 172.21.15.105:0/3625438128 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f98ac0668d0 0x7f98ac068d90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f98b00af000 tx=0x7f98b00b6040 comp rx=0 tx=0).stop 2023-11-28T18:30:24.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.545+0000 7f98c6cea640 1 -- 172.21.15.105:0/3625438128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98c00b4290 msgr2=0x7f98c0150630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:24.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.545+0000 7f98c6cea640 1 --2- 172.21.15.105:0/3625438128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98c00b4290 0x7f98c0150630 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00c500 tx=0x7f98bc00c9c0 comp rx=0 tx=0).stop 2023-11-28T18:30:24.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.545+0000 7f98c6cea640 1 -- 172.21.15.105:0/3625438128 shutdown_connections 2023-11-28T18:30:24.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.545+0000 7f98c6cea640 1 --2- 172.21.15.105:0/3625438128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c00cd8e0 0x7f98c0159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:24.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.545+0000 7f98c6cea640 1 --2- 172.21.15.105:0/3625438128 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98c015b030 0x7f98c0099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:24.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.545+0000 7f98c6cea640 1 --2- 172.21.15.105:0/3625438128 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f98ac0668d0 0x7f98ac068d90 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:24.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.545+0000 7f98c6cea640 1 --2- 172.21.15.105:0/3625438128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98c00b4290 0x7f98c0150630 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:24.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.545+0000 7f98c6cea640 1 -- 172.21.15.105:0/3625438128 >> 172.21.15.105:0/3625438128 conn(0x7f98c00ceb90 msgr2=0x7f98c00ad5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:24.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.545+0000 7f98c6cea640 1 -- 172.21.15.105:0/3625438128 shutdown_connections 2023-11-28T18:30:24.546 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:24.545+0000 7f98c6cea640 1 -- 172.21.15.105:0/3625438128 wait complete. 2023-11-28T18:30:24.966 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:30:05.525117+0000', 'modified': '2023-11-28T18:30:24.155609+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 8145, 'mds_1': 8142}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8142': {'gid': 8142, 'name': 'a', 'rank': 1, 'incarnation': 105, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.105:6836/3968059262', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 3968059262}, {'type': 'v1', 'addr': '172.21.15.105:6836', 'nonce': 3968059262}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_8145': {'gid': 8145, 'name': 'c', 'rank': 0, 'incarnation': 99, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6837/2782729992', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6835', 'nonce': 2782729992}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 2782729992}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:30:24.966 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8142:up:creating 2023-11-28T18:30:25.967 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:30:26.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda8ac5a640 1 Processor -- start 2023-11-28T18:30:26.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda8ac5a640 1 -- start start 2023-11-28T18:30:26.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda8ac5a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda840d3c30 0x7fda840d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda8ac5a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda840b25b0 con 0x7fda8415d4f0 2023-11-28T18:30:26.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda8ac5a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda840cea30 con 0x7fda840b2d20 2023-11-28T18:30:26.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda8ac5a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda840b2900 con 0x7fda840d3c30 2023-11-28T18:30:26.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda889f6640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fda8415d4f0 0x7fda840d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49134/0 (socket says 172.21.15.105:49134) 2023-11-28T18:30:26.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda889f6640 1 -- 172.21.15.105:0/3738768462 learned_addr learned my addr 172.21.15.105:0/3738768462 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:26.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda83fff640 1 --2- 172.21.15.105:0/3738768462 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda840d3c30 0x7fda840d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda837fe640 1 -- 172.21.15.105:0/3738768462 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3797307454 0 0) 0x7fda840b25b0 con 0x7fda8415d4f0 2023-11-28T18:30:26.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda837fe640 1 -- 172.21.15.105:0/3738768462 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda74003530 con 0x7fda8415d4f0 2023-11-28T18:30:26.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda837fe640 1 -- 172.21.15.105:0/3738768462 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389449901 0 0) 0x7fda840cea30 con 0x7fda840b2d20 2023-11-28T18:30:26.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda837fe640 1 -- 172.21.15.105:0/3738768462 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda840b25b0 con 0x7fda840b2d20 2023-11-28T18:30:26.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda837fe640 1 -- 172.21.15.105:0/3738768462 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 33496203 0 0) 0x7fda74003530 con 0x7fda8415d4f0 2023-11-28T18:30:26.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda837fe640 1 -- 172.21.15.105:0/3738768462 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda840d3c30 msgr2=0x7fda840d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:26.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda837fe640 1 --2- 172.21.15.105:0/3738768462 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda840d3c30 0x7fda840d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda837fe640 1 -- 172.21.15.105:0/3738768462 >> v1:172.21.15.148:6789/0 conn(0x7fda840b2d20 legacy=0x7fda840b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:26.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda837fe640 1 -- 172.21.15.105:0/3738768462 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda840ced10 con 0x7fda8415d4f0 2023-11-28T18:30:26.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda837fe640 1 -- 172.21.15.105:0/3738768462 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fda78001680 con 0x7fda8415d4f0 2023-11-28T18:30:26.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda837fe640 1 -- 172.21.15.105:0/3738768462 >> v1:172.21.15.105:6789/0 conn(0x7fda8415d4f0 legacy=0x7fda840d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:26.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda837fe640 1 --2- 172.21.15.105:0/3738768462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda74003b40 0x7fda74003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda837fe640 1 --2- 172.21.15.105:0/3738768462 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda740047d0 0x7fda74004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda837fe640 1 --2- 172.21.15.105:0/3738768462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda74005190 0x7fda7400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda837fe640 1 -- 172.21.15.105:0/3738768462 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda840b2900 con 0x7fda74003b40 2023-11-28T18:30:26.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda837fe640 1 -- 172.21.15.105:0/3738768462 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7400c060 con 0x7fda74005190 2023-11-28T18:30:26.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda837fe640 1 -- 172.21.15.105:0/3738768462 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7400c220 con 0x7fda740047d0 2023-11-28T18:30:26.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda889f6640 1 --2- 172.21.15.105:0/3738768462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda74003b40 0x7fda74003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda83fff640 1 --2- 172.21.15.105:0/3738768462 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda740047d0 0x7fda74004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda891f7640 1 --2- 172.21.15.105:0/3738768462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda74005190 0x7fda7400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda889f6640 1 -- 172.21.15.105:0/3738768462 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda740047d0 msgr2=0x7fda74004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:26.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda889f6640 1 --2- 172.21.15.105:0/3738768462 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda740047d0 0x7fda74004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda889f6640 1 -- 172.21.15.105:0/3738768462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda74005190 msgr2=0x7fda7400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:26.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda889f6640 1 --2- 172.21.15.105:0/3738768462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda74005190 0x7fda7400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.117+0000 7fda889f6640 1 -- 172.21.15.105:0/3738768462 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda74003530 con 0x7fda74003b40 2023-11-28T18:30:26.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.121+0000 7fda889f6640 1 --2- 172.21.15.105:0/3738768462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda74003b40 0x7fda74003f40 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fda78007fb0 tx=0x7fda7801caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c70ae6541e83ff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:26.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.121+0000 7fda837fe640 1 -- 172.21.15.105:0/3738768462 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda78005ae0 con 0x7fda74003b40 2023-11-28T18:30:26.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.121+0000 7fda837fe640 1 -- 172.21.15.105:0/3738768462 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda78005c40 con 0x7fda74003b40 2023-11-28T18:30:26.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.121+0000 7fda837fe640 1 -- 172.21.15.105:0/3738768462 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda78007070 con 0x7fda74003b40 2023-11-28T18:30:26.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.121+0000 7fda8ac5a640 1 -- 172.21.15.105:0/3738768462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda74003b40 msgr2=0x7fda74003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:26.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.121+0000 7fda8ac5a640 1 --2- 172.21.15.105:0/3738768462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda74003b40 0x7fda74003f40 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fda78007fb0 tx=0x7fda7801caf0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.121+0000 7fda891f7640 1 -- 172.21.15.105:0/3738768462 reap_dead start 2023-11-28T18:30:26.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.121+0000 7fda8ac5a640 1 -- 172.21.15.105:0/3738768462 shutdown_connections 2023-11-28T18:30:26.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.121+0000 7fda8ac5a640 1 -- 172.21.15.105:0/3738768462 >> 172.21.15.105:0/3738768462 conn(0x7fda840d3030 msgr2=0x7fda840d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:26.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.121+0000 7fda8ac5a640 1 -- 172.21.15.105:0/3738768462 shutdown_connections 2023-11-28T18:30:26.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.121+0000 7fda8ac5a640 1 -- 172.21.15.105:0/3738768462 wait complete. 2023-11-28T18:30:26.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda8ac5a640 1 Processor -- start 2023-11-28T18:30:26.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda8ac5a640 1 -- start start 2023-11-28T18:30:26.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda8ac5a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda74003b40 0x7fda84156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda8ac5a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda84152fe0 0x7fda841527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda8ac5a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda841f6020 0x7fda841f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda8ac5a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda840d0a20 con 0x7fda841f6020 2023-11-28T18:30:26.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda8ac5a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda840cb2e0 con 0x7fda84152fe0 2023-11-28T18:30:26.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda8ac5a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda840cbe60 con 0x7fda74003b40 2023-11-28T18:30:26.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda891f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda841f6020 0x7fda841f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda891f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda841f6020 0x7fda841f64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55298/0 (socket says 172.21.15.105:55298) 2023-11-28T18:30:26.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda891f7640 1 -- 172.21.15.105:0/161754647 learned_addr learned my addr 172.21.15.105:0/161754647 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:26.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda889f6640 1 --2- 172.21.15.105:0/161754647 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda74003b40 0x7fda84156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda83fff640 1 --2- 172.21.15.105:0/161754647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda84152fe0 0x7fda841527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda891f7640 1 -- 172.21.15.105:0/161754647 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda74003b40 msgr2=0x7fda84156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:26.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda891f7640 1 --2- 172.21.15.105:0/161754647 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda74003b40 0x7fda84156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda891f7640 1 -- 172.21.15.105:0/161754647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda84152fe0 msgr2=0x7fda841527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:26.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda891f7640 1 --2- 172.21.15.105:0/161754647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda84152fe0 0x7fda841527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda891f7640 1 -- 172.21.15.105:0/161754647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda840cb530 con 0x7fda841f6020 2023-11-28T18:30:26.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda891f7640 1 --2- 172.21.15.105:0/161754647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda841f6020 0x7fda841f64a0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fda7c007fd0 tx=0x7fda7c00ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:26.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda817fa640 1 -- 172.21.15.105:0/161754647 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c006d70 con 0x7fda841f6020 2023-11-28T18:30:26.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda817fa640 1 -- 172.21.15.105:0/161754647 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7c003980 con 0x7fda841f6020 2023-11-28T18:30:26.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda8ac5a640 1 -- 172.21.15.105:0/161754647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda840cb810 con 0x7fda841f6020 2023-11-28T18:30:26.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda817fa640 1 -- 172.21.15.105:0/161754647 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c003d30 con 0x7fda841f6020 2023-11-28T18:30:26.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.125+0000 7fda8ac5a640 1 -- 172.21.15.105:0/161754647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda841537c0 con 0x7fda841f6020 2023-11-28T18:30:26.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.129+0000 7fda8ac5a640 1 -- 172.21.15.105:0/161754647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda840cbff0 con 0x7fda841f6020 2023-11-28T18:30:26.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.129+0000 7fda817fa640 1 -- 172.21.15.105:0/161754647 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fda7c0050d0 con 0x7fda841f6020 2023-11-28T18:30:26.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.129+0000 7fda817fa640 1 --2- 172.21.15.105:0/161754647 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fda68066900 0x7fda68068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.129+0000 7fda817fa640 1 -- 172.21.15.105:0/161754647 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fda7c083950 con 0x7fda841f6020 2023-11-28T18:30:26.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.129+0000 7fda889f6640 1 --2- 172.21.15.105:0/161754647 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fda68066900 0x7fda68068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.129+0000 7fda889f6640 1 --2- 172.21.15.105:0/161754647 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fda68066900 0x7fda68068dc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fda780ab000 tx=0x7fda780a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:26.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.133+0000 7fda817fa640 1 -- 172.21.15.105:0/161754647 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fda7c050ad0 con 0x7fda841f6020 2023-11-28T18:30:26.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.313+0000 7fda8ac5a640 1 -- 172.21.15.105:0/161754647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda84067570 con 0x7fda841f6020 2023-11-28T18:30:26.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.317+0000 7fda817fa640 1 -- 172.21.15.105:0/161754647 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+4959 (secure 0 0 0) 0x7fda7c019090 con 0x7fda841f6020 2023-11-28T18:30:26.316 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:26.316 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":108,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/2603493310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2603493310},{"type":"v1","addr":"172.21.15.148:6834","nonce":2603493310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:30:05.525117+0000","modified":"2023-11-28T18:30:25.187092+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8145,"mds_1":8142,"mds_2":6803},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6803":{"gid":6803,"name":"d","rank":2,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6835/3664281230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":3664281230},{"type":"v1","addr":"172.21.15.148:6835","nonce":3664281230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8142":{"gid":8142,"name":"a","rank":1,"incarnation":105,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6836/3968059262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3968059262},{"type":"v1","addr":"172.21.15.105:6836","nonce":3968059262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8145":{"gid":8145,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2782729992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2782729992},{"type":"v1","addr":"172.21.15.105:6837","nonce":2782729992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-11-28T18:30:26.317 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 108 2023-11-28T18:30:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.317+0000 7fda8ac5a640 1 -- 172.21.15.105:0/161754647 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fda68066900 msgr2=0x7fda68068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.317+0000 7fda8ac5a640 1 --2- 172.21.15.105:0/161754647 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fda68066900 0x7fda68068dc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fda780ab000 tx=0x7fda780a7070 comp rx=0 tx=0).stop 2023-11-28T18:30:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.317+0000 7fda8ac5a640 1 -- 172.21.15.105:0/161754647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda841f6020 msgr2=0x7fda841f64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.317+0000 7fda8ac5a640 1 --2- 172.21.15.105:0/161754647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda841f6020 0x7fda841f64a0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fda7c007fd0 tx=0x7fda7c00ff70 comp rx=0 tx=0).stop 2023-11-28T18:30:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.317+0000 7fda8ac5a640 1 -- 172.21.15.105:0/161754647 shutdown_connections 2023-11-28T18:30:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.317+0000 7fda8ac5a640 1 --2- 172.21.15.105:0/161754647 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fda68066900 0x7fda68068dc0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.317+0000 7fda8ac5a640 1 --2- 172.21.15.105:0/161754647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda841f6020 0x7fda841f64a0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.317+0000 7fda8ac5a640 1 --2- 172.21.15.105:0/161754647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda84152fe0 0x7fda841527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.317+0000 7fda8ac5a640 1 --2- 172.21.15.105:0/161754647 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda74003b40 0x7fda84156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.317+0000 7fda8ac5a640 1 -- 172.21.15.105:0/161754647 >> 172.21.15.105:0/161754647 conn(0x7fda840d3030 msgr2=0x7fda840cdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.317+0000 7fda8ac5a640 1 -- 172.21.15.105:0/161754647 shutdown_connections 2023-11-28T18:30:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.317+0000 7fda8ac5a640 1 -- 172.21.15.105:0/161754647 wait complete. 2023-11-28T18:30:26.739 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 108, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:30:05.525117+0000', 'modified': '2023-11-28T18:30:25.187092+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 8145, 'mds_1': 8142, 'mds_2': 6803}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6803': {'gid': 6803, 'name': 'd', 'rank': 2, 'incarnation': 107, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.148:6835/3664281230', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6833', 'nonce': 3664281230}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 3664281230}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_8142': {'gid': 8142, 'name': 'a', 'rank': 1, 'incarnation': 105, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.105:6836/3968059262', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 3968059262}, {'type': 'v1', 'addr': '172.21.15.105:6836', 'nonce': 3968059262}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_8145': {'gid': 8145, 'name': 'c', 'rank': 0, 'incarnation': 99, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6837/2782729992', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6835', 'nonce': 2782729992}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 2782729992}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:30:26.739 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-11-28T18:30:26.740 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-28T18:30:26.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2daa4fe640 1 Processor -- start 2023-11-28T18:30:26.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2daa4fe640 1 -- start start 2023-11-28T18:30:26.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2daa4fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2da415d320 0x7f2da406c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2daa4fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2da40ce230 0x7f2da40cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2daa4fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da40b3c40 con 0x7f2da40cdbb0 2023-11-28T18:30:26.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2daa4fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da406ca80 con 0x7f2da415d320 2023-11-28T18:30:26.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2daa4fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da415d700 con 0x7f2da40ce230 2023-11-28T18:30:26.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da8a9b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f2da40cdbb0 0x7f2da40cef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49148/0 (socket says 172.21.15.105:49148) 2023-11-28T18:30:26.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da8a9b640 1 -- 172.21.15.105:0/1288479869 learned_addr learned my addr 172.21.15.105:0/1288479869 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:26.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da3fff640 1 --2- 172.21.15.105:0/1288479869 >> v2:172.21.15.148:3300/0 conn(0x7f2da415d320 0x7f2da406c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da37fe640 1 --2- 172.21.15.105:0/1288479869 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2da40ce230 0x7f2da40cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da2ffd640 1 -- 172.21.15.105:0/1288479869 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1177077971 0 0) 0x7f2da40b3c40 con 0x7f2da40cdbb0 2023-11-28T18:30:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da2ffd640 1 -- 172.21.15.105:0/1288479869 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d8c003530 con 0x7f2da40cdbb0 2023-11-28T18:30:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da2ffd640 1 -- 172.21.15.105:0/1288479869 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1978244484 0 0) 0x7f2d8c003530 con 0x7f2da40cdbb0 2023-11-28T18:30:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da2ffd640 1 -- 172.21.15.105:0/1288479869 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2da40ce230 msgr2=0x7f2da40cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da2ffd640 1 --2- 172.21.15.105:0/1288479869 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2da40ce230 0x7f2da40cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da2ffd640 1 -- 172.21.15.105:0/1288479869 >> v2:172.21.15.148:3300/0 conn(0x7f2da415d320 msgr2=0x7f2da406c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da2ffd640 1 --2- 172.21.15.105:0/1288479869 >> v2:172.21.15.148:3300/0 conn(0x7f2da415d320 0x7f2da406c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da2ffd640 1 -- 172.21.15.105:0/1288479869 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da40cdfc0 con 0x7f2da40cdbb0 2023-11-28T18:30:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da2ffd640 1 -- 172.21.15.105:0/1288479869 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f2d9c001a00 con 0x7f2da40cdbb0 2023-11-28T18:30:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da3fff640 1 --2- 172.21.15.105:0/1288479869 >> v2:172.21.15.148:3300/0 conn(0x7f2da415d320 0x7f2da406c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:30:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da2ffd640 1 -- 172.21.15.105:0/1288479869 >> v1:172.21.15.105:6789/0 conn(0x7f2da40cdbb0 legacy=0x7f2da40cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da2ffd640 1 --2- 172.21.15.105:0/1288479869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d8c003980 0x7f2d8c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da2ffd640 1 --2- 172.21.15.105:0/1288479869 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2da40ce230 0x7f2d8c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da2ffd640 1 --2- 172.21.15.105:0/1288479869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d8c004d10 0x7f2d8c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da2ffd640 1 -- 172.21.15.105:0/1288479869 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da406ca80 con 0x7f2d8c004d10 2023-11-28T18:30:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da2ffd640 1 -- 172.21.15.105:0/1288479869 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da415d700 con 0x7f2d8c003980 2023-11-28T18:30:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da2ffd640 1 -- 172.21.15.105:0/1288479869 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c00b990 con 0x7f2da40ce230 2023-11-28T18:30:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da8a9b640 1 --2- 172.21.15.105:0/1288479869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d8c004d10 0x7f2d8c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da3fff640 1 --2- 172.21.15.105:0/1288479869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d8c003980 0x7f2d8c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.881+0000 7f2da37fe640 1 --2- 172.21.15.105:0/1288479869 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2da40ce230 0x7f2d8c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.885+0000 7f2da8a9b640 1 -- 172.21.15.105:0/1288479869 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2da40ce230 msgr2=0x7f2d8c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.885+0000 7f2da8a9b640 1 --2- 172.21.15.105:0/1288479869 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2da40ce230 0x7f2d8c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.885+0000 7f2da8a9b640 1 -- 172.21.15.105:0/1288479869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d8c003980 msgr2=0x7f2d8c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:26.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.885+0000 7f2da8a9b640 1 --2- 172.21.15.105:0/1288479869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d8c003980 0x7f2d8c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.885+0000 7f2da8a9b640 1 -- 172.21.15.105:0/1288479869 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c003530 con 0x7f2d8c004d10 2023-11-28T18:30:26.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.885+0000 7f2da8a9b640 1 --2- 172.21.15.105:0/1288479869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d8c004d10 0x7f2d8c005190 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00a860 tx=0x7f2d9c01fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5435e2a5eacc580 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:26.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.885+0000 7f2da2ffd640 1 -- 172.21.15.105:0/1288479869 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c0aa6f0 con 0x7f2d8c004d10 2023-11-28T18:30:26.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.885+0000 7f2da2ffd640 1 -- 172.21.15.105:0/1288479869 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9c0aa850 con 0x7f2d8c004d10 2023-11-28T18:30:26.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.885+0000 7f2da2ffd640 1 -- 172.21.15.105:0/1288479869 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c0aac20 con 0x7f2d8c004d10 2023-11-28T18:30:26.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.885+0000 7f2daa4fe640 1 -- 172.21.15.105:0/1288479869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d8c004d10 msgr2=0x7f2d8c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:26.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.885+0000 7f2daa4fe640 1 --2- 172.21.15.105:0/1288479869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d8c004d10 0x7f2d8c005190 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00a860 tx=0x7f2d9c01fac0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.885+0000 7f2da8a9b640 1 -- 172.21.15.105:0/1288479869 reap_dead start 2023-11-28T18:30:26.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.885+0000 7f2daa4fe640 1 -- 172.21.15.105:0/1288479869 shutdown_connections 2023-11-28T18:30:26.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.885+0000 7f2daa4fe640 1 -- 172.21.15.105:0/1288479869 >> 172.21.15.105:0/1288479869 conn(0x7f2da40d0df0 msgr2=0x7f2da40d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:26.890 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.889+0000 7f2daa4fe640 1 -- 172.21.15.105:0/1288479869 shutdown_connections 2023-11-28T18:30:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.889+0000 7f2daa4fe640 1 -- 172.21.15.105:0/1288479869 wait complete. 2023-11-28T18:30:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.889+0000 7f2daa4fe640 1 Processor -- start 2023-11-28T18:30:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.889+0000 7f2daa4fe640 1 -- start start 2023-11-28T18:30:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.889+0000 7f2daa4fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d8c004d10 0x7f2da40c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.889+0000 7f2daa4fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2da40cad20 0x7f2da41512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.889+0000 7f2daa4fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2da409c4a0 0x7f2da409c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.889+0000 7f2daa4fe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da40c93f0 con 0x7f2d8c004d10 2023-11-28T18:30:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.889+0000 7f2daa4fe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da40c96d0 con 0x7f2da409c4a0 2023-11-28T18:30:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.889+0000 7f2daa4fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da40c99b0 con 0x7f2da40cad20 2023-11-28T18:30:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.889+0000 7f2da3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d8c004d10 0x7f2da40c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.889+0000 7f2da3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d8c004d10 0x7f2da40c8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55314/0 (socket says 172.21.15.105:55314) 2023-11-28T18:30:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.889+0000 7f2da3fff640 1 -- 172.21.15.105:0/2247016437 learned_addr learned my addr 172.21.15.105:0/2247016437 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.889+0000 7f2da37fe640 1 --2- 172.21.15.105:0/2247016437 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2da40cad20 0x7f2da41512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.889+0000 7f2da8a9b640 1 --2- 172.21.15.105:0/2247016437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2da409c4a0 0x7f2da409c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.889+0000 7f2da3fff640 1 -- 172.21.15.105:0/2247016437 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2da40cad20 msgr2=0x7f2da41512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.889+0000 7f2da3fff640 1 --2- 172.21.15.105:0/2247016437 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2da40cad20 0x7f2da41512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.889+0000 7f2da3fff640 1 -- 172.21.15.105:0/2247016437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2da409c4a0 msgr2=0x7f2da409c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.889+0000 7f2da3fff640 1 --2- 172.21.15.105:0/2247016437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2da409c4a0 0x7f2da409c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.889+0000 7f2da3fff640 1 -- 172.21.15.105:0/2247016437 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da40c9c90 con 0x7f2d8c004d10 2023-11-28T18:30:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.889+0000 7f2da37fe640 1 --2- 172.21.15.105:0/2247016437 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2da40cad20 0x7f2da41512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.889+0000 7f2da3fff640 1 --2- 172.21.15.105:0/2247016437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d8c004d10 0x7f2da40c8d50 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f2d94011cd0 tx=0x7f2d9400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.893+0000 7f2da0ff9640 1 -- 172.21.15.105:0/2247016437 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9400fd20 con 0x7f2d8c004d10 2023-11-28T18:30:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.893+0000 7f2da0ff9640 1 -- 172.21.15.105:0/2247016437 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9400fe80 con 0x7f2d8c004d10 2023-11-28T18:30:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.893+0000 7f2daa4fe640 1 -- 172.21.15.105:0/2247016437 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da40c9f70 con 0x7f2d8c004d10 2023-11-28T18:30:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.893+0000 7f2da0ff9640 1 -- 172.21.15.105:0/2247016437 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94004270 con 0x7f2d8c004d10 2023-11-28T18:30:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.893+0000 7f2daa4fe640 1 -- 172.21.15.105:0/2247016437 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da4151830 con 0x7f2d8c004d10 2023-11-28T18:30:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.893+0000 7f2da0ff9640 1 -- 172.21.15.105:0/2247016437 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f2d940043d0 con 0x7f2d8c004d10 2023-11-28T18:30:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.893+0000 7f2da0ff9640 1 --2- 172.21.15.105:0/2247016437 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2d90066900 0x7f2d90068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2da0ff9640 1 -- 172.21.15.105:0/2247016437 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2d94084cf0 con 0x7f2d8c004d10 2023-11-28T18:30:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2da37fe640 1 --2- 172.21.15.105:0/2247016437 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2d90066900 0x7f2d90068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2dad85f000 1 Processor -- start 2023-11-28T18:30:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2dad85f000 1 -- start start 2023-11-28T18:30:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2da37fe640 1 --2- 172.21.15.105:0/2247016437 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2d90066900 0x7f2d90068dc0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2da40b6930 tx=0x7f2d98008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2dad85f000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d055135d30 0x55d055136110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2dad85f000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d055136650 0x55d0550e8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2dad85f000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0550e9500 0x55d0550e98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2dad85f000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d054fb2110 con 0x55d0550e9500 2023-11-28T18:30:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2dad85f000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d054f429a0 con 0x55d055136650 2023-11-28T18:30:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2dad85f000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d054f3fed0 con 0x55d055135d30 2023-11-28T18:30:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2da8a9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0550e9500 0x55d0550e98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2da8a9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0550e9500 0x55d0550e98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55318/0 (socket says 172.21.15.105:55318) 2023-11-28T18:30:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2da8a9b640 1 -- 172.21.15.105:0/2042335748 learned_addr learned my addr 172.21.15.105:0/2042335748 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2da8a9b640 1 -- 172.21.15.105:0/2042335748 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d055135d30 msgr2=0x55d055136110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2da3fff640 1 --2- 172.21.15.105:0/2042335748 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d055135d30 0x55d055136110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2da8a9b640 1 --2- 172.21.15.105:0/2042335748 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d055135d30 0x55d055136110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2da8a9b640 1 -- 172.21.15.105:0/2042335748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d055136650 msgr2=0x55d0550e8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2da8a9b640 1 --2- 172.21.15.105:0/2042335748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d055136650 0x55d0550e8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2da8a9b640 1 -- 172.21.15.105:0/2042335748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d054c33890 con 0x55d0550e9500 2023-11-28T18:30:26.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2da8a9b640 1 --2- 172.21.15.105:0/2042335748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0550e9500 0x55d0550e98e0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00a860 tx=0x7f2d9c001e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861b828257be94ae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:26.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2d89ffb640 1 -- 172.21.15.105:0/2042335748 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c0b9040 con 0x55d0550e9500 2023-11-28T18:30:26.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2d89ffb640 1 -- 172.21.15.105:0/2042335748 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9c001fe0 con 0x55d0550e9500 2023-11-28T18:30:26.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2d89ffb640 1 -- 172.21.15.105:0/2042335748 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c0aa6f0 con 0x55d0550e9500 2023-11-28T18:30:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2dad85f000 1 -- 172.21.15.105:0/2042335748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0550e9500 msgr2=0x55d0550e98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2dad85f000 1 --2- 172.21.15.105:0/2042335748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0550e9500 0x55d0550e98e0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00a860 tx=0x7f2d9c001e00 comp rx=0 tx=0).stop 2023-11-28T18:30:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2dad85f000 1 -- 172.21.15.105:0/2042335748 shutdown_connections 2023-11-28T18:30:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2dad85f000 1 --2- 172.21.15.105:0/2042335748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0550e9500 0x55d0550e98e0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2dad85f000 1 --2- 172.21.15.105:0/2042335748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d055136650 0x55d0550e8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2dad85f000 1 --2- 172.21.15.105:0/2042335748 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d055135d30 0x55d055136110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2dad85f000 1 -- 172.21.15.105:0/2042335748 >> 172.21.15.105:0/2042335748 conn(0x55d0551168c0 msgr2=0x55d055118cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2dad85f000 1 -- 172.21.15.105:0/2042335748 shutdown_connections 2023-11-28T18:30:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.897+0000 7f2dad85f000 1 -- 172.21.15.105:0/2042335748 wait complete. 2023-11-28T18:30:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2dad85f000 1 -- start start 2023-11-28T18:30:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2dad85f000 1 Processor -- start 2023-11-28T18:30:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2dad85f000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0550e9500 0x55d0550ef520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2dad85f000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d055135d30 0x55d0550efa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2dad85f000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d055136650 0x55d0550effa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2dad85f000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d054f3b7c0 con 0x55d0550e9500 2023-11-28T18:30:26.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2dad85f000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d054ef3bf0 con 0x55d055136650 2023-11-28T18:30:26.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2dad85f000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d054ef9a90 con 0x55d055135d30 2023-11-28T18:30:26.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2dad85f000 10 client.0 _ll_register_callbacks cb 0x55d0550d6e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:30:26.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2da3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0550e9500 0x55d0550ef520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2da3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0550e9500 0x55d0550ef520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55334/0 (socket says 172.21.15.105:55334) 2023-11-28T18:30:26.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2da3fff640 1 -- 172.21.15.105:0/1259103272 learned_addr learned my addr 172.21.15.105:0/1259103272 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:26.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2da8a9b640 1 --2- 172.21.15.105:0/1259103272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d055136650 0x55d0550effa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2da37fe640 1 --2- 172.21.15.105:0/1259103272 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d055135d30 0x55d0550efa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2da3fff640 1 -- 172.21.15.105:0/1259103272 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d055135d30 msgr2=0x55d0550efa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:26.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2da3fff640 1 --2- 172.21.15.105:0/1259103272 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d055135d30 0x55d0550efa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2da3fff640 1 -- 172.21.15.105:0/1259103272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d055136650 msgr2=0x55d0550effa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:26.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2da3fff640 1 --2- 172.21.15.105:0/1259103272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d055136650 0x55d0550effa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2da3fff640 1 -- 172.21.15.105:0/1259103272 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d94009420 con 0x55d0550e9500 2023-11-28T18:30:26.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2da3fff640 1 --2- 172.21.15.105:0/1259103272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0550e9500 0x55d0550ef520 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f2d94052020 tx=0x7f2d94055c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c50abb5f75d457 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:26.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2d82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:30:26.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2d82ffd640 1 -- 172.21.15.105:0/1259103272 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94053f80 con 0x55d0550e9500 2023-11-28T18:30:26.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2d82ffd640 1 -- 172.21.15.105:0/1259103272 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2d9406d1b0 con 0x55d0550e9500 2023-11-28T18:30:26.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2dad85f000 1 -- 172.21.15.105:0/1259103272 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d054efd470 con 0x55d0550e9500 2023-11-28T18:30:26.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2d82ffd640 1 -- 172.21.15.105:0/1259103272 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d9406d390 con 0x55d0550e9500 2023-11-28T18:30:26.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2dad85f000 10 client.8349 fetch_fsmap learned FSMap version 108 2023-11-28T18:30:26.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2dad85f000 1 -- 172.21.15.105:0/1259103272 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x55d054efc190 con 0x55d0550e9500 2023-11-28T18:30:26.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2d82ffd640 1 -- 172.21.15.105:0/1259103272 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d054efd470 con 0x55d0550e9500 2023-11-28T18:30:26.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2d82ffd640 1 -- 172.21.15.105:0/1259103272 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 108) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f2d94055010 con 0x55d0550e9500 2023-11-28T18:30:26.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2dad85f000 10 client.8349 fetch_fsmap finished waiting for FSMap version 108 2023-11-28T18:30:26.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2dad85f000 10 client.8349 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T18:30:26.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2dad85f000 20 client.8349 populate_metadata read hostname 'smithi105' 2023-11-28T18:30:26.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2dad85f000 1 --2- 172.21.15.105:0/1259103272 >> [v2:172.21.15.148:6833/3664281230,v1:172.21.15.148:6835/3664281230] conn(0x55d0550f09b0 0x55d055147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2dad85f000 4 client.8349 mds_command: new command op to 6803 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:30:26.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2dad85f000 1 -- 172.21.15.105:0/1259103272 --> [v2:172.21.15.148:6833/3664281230,v1:172.21.15.148:6835/3664281230] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d054f00b50 con 0x55d0550f09b0 2023-11-28T18:30:26.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.901+0000 7f2da37fe640 1 --2- 172.21.15.105:0/1259103272 >> [v2:172.21.15.148:6833/3664281230,v1:172.21.15.148:6835/3664281230] conn(0x55d0550f09b0 0x55d055147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.905+0000 7f2d82ffd640 10 client.8349 ms_handle_connect on v2:172.21.15.148:6833/3664281230 2023-11-28T18:30:26.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.905+0000 7f2da37fe640 1 --2- 172.21.15.105:0/1259103272 >> [v2:172.21.15.148:6833/3664281230,v1:172.21.15.148:6835/3664281230] conn(0x55d0550f09b0 0x55d055147ff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e0e66fcb6dd4ea81 server_cookie=1ac56c4ec37e18f3 in_seq=0 out_seq=0 2023-11-28T18:30:26.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.905+0000 7f2d82ffd640 1 -- 172.21.15.105:0/1259103272 <== mds.2 v2:172.21.15.148:6833/3664281230 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d054f00b50 con 0x55d0550f09b0 2023-11-28T18:30:26.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.905+0000 7f2d82ffd640 10 client.8349 handle_command_reply: tid=0 2023-11-28T18:30:26.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.905+0000 7f2dad85f000 1 client.8349 shutdown 2023-11-28T18:30:26.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.905+0000 7f2dad85f000 1 -- 172.21.15.105:0/1259103272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0550e9500 msgr2=0x55d0550ef520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:26.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.905+0000 7f2dad85f000 1 --2- 172.21.15.105:0/1259103272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0550e9500 0x55d0550ef520 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f2d94052020 tx=0x7f2d94055c60 comp rx=0 tx=0).stop 2023-11-28T18:30:26.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.905+0000 7f2dad85f000 1 -- 172.21.15.105:0/1259103272 shutdown_connections 2023-11-28T18:30:26.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.905+0000 7f2dad85f000 1 --2- 172.21.15.105:0/1259103272 >> [v2:172.21.15.148:6833/3664281230,v1:172.21.15.148:6835/3664281230] conn(0x55d0550f09b0 0x55d055147ff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.905+0000 7f2dad85f000 1 --2- 172.21.15.105:0/1259103272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d055136650 0x55d0550effa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.905+0000 7f2dad85f000 1 --2- 172.21.15.105:0/1259103272 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d055135d30 0x55d0550efa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.905+0000 7f2dad85f000 1 --2- 172.21.15.105:0/1259103272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0550e9500 0x55d0550ef520 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.905+0000 7f2dad85f000 1 -- 172.21.15.105:0/1259103272 >> 172.21.15.105:0/1259103272 conn(0x55d0551168c0 msgr2=0x55d0550e86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:26.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.905+0000 7f2d82ffd640 0 client.8349 ms_handle_reset on v2:172.21.15.148:6833/3664281230 2023-11-28T18:30:26.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.905+0000 7f2dad85f000 1 -- 172.21.15.105:0/1259103272 shutdown_connections 2023-11-28T18:30:26.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.905+0000 7f2dad85f000 1 -- 172.21.15.105:0/1259103272 wait complete. 2023-11-28T18:30:26.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.905+0000 7f2dad85f000 20 client.8349 trim_cache size 0 max 16384 2023-11-28T18:30:26.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2dad85f000 1 Processor -- start 2023-11-28T18:30:26.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2dad85f000 1 -- start start 2023-11-28T18:30:26.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2dad85f000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d0550e9500 0x55d0551153a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2dad85f000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d055135d30 0x55d0551158e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2dad85f000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d055136650 0x55d055147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2dad85f000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d054bf4940 con 0x55d055135d30 2023-11-28T18:30:26.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2dad85f000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d054f04910 con 0x55d0550e9500 2023-11-28T18:30:26.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2dad85f000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0550f3510 con 0x55d055136650 2023-11-28T18:30:26.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2da37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d055135d30 0x55d0551158e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2da37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d055135d30 0x55d0551158e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55338/0 (socket says 172.21.15.105:55338) 2023-11-28T18:30:26.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2da37fe640 1 -- 172.21.15.105:0/2538971482 learned_addr learned my addr 172.21.15.105:0/2538971482 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:26.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2da3fff640 1 --2- 172.21.15.105:0/2538971482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d0550e9500 0x55d0551153a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2da8a9b640 1 --2- 172.21.15.105:0/2538971482 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d055136650 0x55d055147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2da37fe640 1 -- 172.21.15.105:0/2538971482 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d055136650 msgr2=0x55d055147ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:26.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2da37fe640 1 --2- 172.21.15.105:0/2538971482 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d055136650 0x55d055147ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2da37fe640 1 -- 172.21.15.105:0/2538971482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d0550e9500 msgr2=0x55d0551153a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:26.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2da37fe640 1 --2- 172.21.15.105:0/2538971482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d0550e9500 0x55d0551153a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2da37fe640 1 -- 172.21.15.105:0/2538971482 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d054ee2490 con 0x55d055135d30 2023-11-28T18:30:26.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2da37fe640 1 --2- 172.21.15.105:0/2538971482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d055135d30 0x55d0551158e0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f2d98014fc0 tx=0x7f2d98017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd605b88acb4b21c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:26.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2d8a7fc640 1 -- 172.21.15.105:0/2538971482 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98029040 con 0x55d055135d30 2023-11-28T18:30:26.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2d8a7fc640 1 -- 172.21.15.105:0/2538971482 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d98025be0 con 0x55d055135d30 2023-11-28T18:30:26.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2d8a7fc640 1 -- 172.21.15.105:0/2538971482 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9802c350 con 0x55d055135d30 2023-11-28T18:30:26.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2dad85f000 1 -- 172.21.15.105:0/2538971482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d055135d30 msgr2=0x55d0551158e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:26.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2dad85f000 1 --2- 172.21.15.105:0/2538971482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d055135d30 0x55d0551158e0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f2d98014fc0 tx=0x7f2d98017000 comp rx=0 tx=0).stop 2023-11-28T18:30:26.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2dad85f000 1 -- 172.21.15.105:0/2538971482 shutdown_connections 2023-11-28T18:30:26.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2dad85f000 1 --2- 172.21.15.105:0/2538971482 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d055136650 0x55d055147ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2dad85f000 1 --2- 172.21.15.105:0/2538971482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d055135d30 0x55d0551158e0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2dad85f000 1 --2- 172.21.15.105:0/2538971482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d0550e9500 0x55d0551153a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2dad85f000 1 -- 172.21.15.105:0/2538971482 >> 172.21.15.105:0/2538971482 conn(0x55d0551168c0 msgr2=0x55d0550e7720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:26.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2dad85f000 1 -- 172.21.15.105:0/2538971482 shutdown_connections 2023-11-28T18:30:26.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.921+0000 7f2dad85f000 1 -- 172.21.15.105:0/2538971482 wait complete. 2023-11-28T18:30:26.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2dad85f000 1 -- start start 2023-11-28T18:30:26.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2dad85f000 1 Processor -- start 2023-11-28T18:30:26.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2dad85f000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0550e9500 0x55d055196810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2dad85f000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d055135d30 0x55d055196d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2dad85f000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d055136650 0x55d055151a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2dad85f000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d054f10b70 con 0x55d0550e9500 2023-11-28T18:30:26.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2dad85f000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0550ece10 con 0x55d055135d30 2023-11-28T18:30:26.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2dad85f000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d054f039f0 con 0x55d055136650 2023-11-28T18:30:26.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2dad85f000 10 client.0 _ll_register_callbacks cb 0x55d054fb2060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:30:26.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2da3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0550e9500 0x55d055196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2da3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0550e9500 0x55d055196810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55342/0 (socket says 172.21.15.105:55342) 2023-11-28T18:30:26.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2da3fff640 1 -- 172.21.15.105:0/153079581 learned_addr learned my addr 172.21.15.105:0/153079581 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:26.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2da8a9b640 1 --2- 172.21.15.105:0/153079581 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d055136650 0x55d055151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2da37fe640 1 --2- 172.21.15.105:0/153079581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d055135d30 0x55d055196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2da3fff640 1 -- 172.21.15.105:0/153079581 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d055136650 msgr2=0x55d055151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:26.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2da3fff640 1 --2- 172.21.15.105:0/153079581 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d055136650 0x55d055151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2da3fff640 1 -- 172.21.15.105:0/153079581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d055135d30 msgr2=0x55d055196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:26.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2da3fff640 1 --2- 172.21.15.105:0/153079581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d055135d30 0x55d055196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2da3fff640 1 -- 172.21.15.105:0/153079581 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d054c33cc0 con 0x55d0550e9500 2023-11-28T18:30:26.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2da3fff640 1 --2- 172.21.15.105:0/153079581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0550e9500 0x55d055196810 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f2d940830f0 tx=0x7f2d94082f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450a9a5f621089e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:26.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2d82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:30:26.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2d82ffd640 1 -- 172.21.15.105:0/153079581 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9405db80 con 0x55d0550e9500 2023-11-28T18:30:26.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2d82ffd640 1 -- 172.21.15.105:0/153079581 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2d94053f80 con 0x55d0550e9500 2023-11-28T18:30:26.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2dad85f000 1 -- 172.21.15.105:0/153079581 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d054f3a110 con 0x55d0550e9500 2023-11-28T18:30:26.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2d82ffd640 1 -- 172.21.15.105:0/153079581 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d940092a0 con 0x55d0550e9500 2023-11-28T18:30:26.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2dad85f000 10 client.8355 fetch_fsmap learned FSMap version 108 2023-11-28T18:30:26.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2dad85f000 1 -- 172.21.15.105:0/153079581 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x55d054e64a50 con 0x55d0550e9500 2023-11-28T18:30:26.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2d82ffd640 1 -- 172.21.15.105:0/153079581 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d054f3a110 con 0x55d0550e9500 2023-11-28T18:30:26.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2d82ffd640 1 -- 172.21.15.105:0/153079581 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 108) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f2d94051910 con 0x55d0550e9500 2023-11-28T18:30:26.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2dad85f000 10 client.8355 fetch_fsmap finished waiting for FSMap version 108 2023-11-28T18:30:26.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2dad85f000 10 client.8355 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T18:30:26.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2dad85f000 20 client.8355 populate_metadata read hostname 'smithi105' 2023-11-28T18:30:26.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2dad85f000 1 --2- 172.21.15.105:0/153079581 >> [v2:172.21.15.148:6833/3664281230,v1:172.21.15.148:6835/3664281230] conn(0x55d055152370 0x55d055152750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:26.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2dad85f000 4 client.8355 mds_command: new command op to 6803 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:30:26.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2dad85f000 1 -- 172.21.15.105:0/153079581 --> [v2:172.21.15.148:6833/3664281230,v1:172.21.15.148:6835/3664281230] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d054f68650 con 0x55d055152370 2023-11-28T18:30:26.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.925+0000 7f2da37fe640 1 --2- 172.21.15.105:0/153079581 >> [v2:172.21.15.148:6833/3664281230,v1:172.21.15.148:6835/3664281230] conn(0x55d055152370 0x55d055152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:26.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2da37fe640 1 --2- 172.21.15.105:0/153079581 >> [v2:172.21.15.148:6833/3664281230,v1:172.21.15.148:6835/3664281230] conn(0x55d055152370 0x55d055152750 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a2d28bd04e6a8e54 server_cookie=20289c5d13c16e2b in_seq=0 out_seq=0 2023-11-28T18:30:26.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2d82ffd640 10 client.8355 ms_handle_connect on v2:172.21.15.148:6833/3664281230 2023-11-28T18:30:26.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2d82ffd640 1 -- 172.21.15.105:0/153079581 <== mds.2 v2:172.21.15.148:6833/3664281230 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55d054f68650 con 0x55d055152370 2023-11-28T18:30:26.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2d82ffd640 10 client.8355 handle_command_reply: tid=0 2023-11-28T18:30:26.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2dad85f000 1 client.8355 shutdown 2023-11-28T18:30:26.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2dad85f000 1 -- 172.21.15.105:0/153079581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0550e9500 msgr2=0x55d055196810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:26.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2dad85f000 1 --2- 172.21.15.105:0/153079581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0550e9500 0x55d055196810 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f2d940830f0 tx=0x7f2d94082f70 comp rx=0 tx=0).stop 2023-11-28T18:30:26.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2dad85f000 1 -- 172.21.15.105:0/153079581 shutdown_connections 2023-11-28T18:30:26.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2dad85f000 1 --2- 172.21.15.105:0/153079581 >> [v2:172.21.15.148:6833/3664281230,v1:172.21.15.148:6835/3664281230] conn(0x55d055152370 0x55d055152750 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2dad85f000 1 --2- 172.21.15.105:0/153079581 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d055136650 0x55d055151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2dad85f000 1 --2- 172.21.15.105:0/153079581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d055135d30 0x55d055196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2d82ffd640 0 client.8355 ms_handle_reset on v2:172.21.15.148:6833/3664281230 2023-11-28T18:30:26.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2dad85f000 1 --2- 172.21.15.105:0/153079581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0550e9500 0x55d055196810 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2dad85f000 1 -- 172.21.15.105:0/153079581 >> 172.21.15.105:0/153079581 conn(0x55d0551168c0 msgr2=0x55d0550e7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:26.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2dad85f000 1 -- 172.21.15.105:0/153079581 shutdown_connections 2023-11-28T18:30:26.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2dad85f000 1 -- 172.21.15.105:0/153079581 wait complete. 2023-11-28T18:30:26.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2dad85f000 20 client.8355 trim_cache size 0 max 16384 2023-11-28T18:30:26.935 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:30:26.936 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:30:26.936 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 2, 2023-11-28T18:30:26.936 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 6803, 2023-11-28T18:30:26.936 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:30:26.936 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:30:26.936 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:30:26.936 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 1.758706965, 2023-11-28T18:30:26.936 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 108, 2023-11-28T18:30:26.936 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 105, 2023-11-28T18:30:26.936 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 105, 2023-11-28T18:30:26.937 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 26.104206858000001 2023-11-28T18:30:26.937 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:30:26.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2daa4fe640 1 -- 172.21.15.105:0/2247016437 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2d90066900 msgr2=0x7f2d90068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:26.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2daa4fe640 1 --2- 172.21.15.105:0/2247016437 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2d90066900 0x7f2d90068dc0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2da40b6930 tx=0x7f2d98008040 comp rx=0 tx=0).stop 2023-11-28T18:30:26.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2daa4fe640 1 -- 172.21.15.105:0/2247016437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d8c004d10 msgr2=0x7f2da40c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:26.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2daa4fe640 1 --2- 172.21.15.105:0/2247016437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d8c004d10 0x7f2da40c8d50 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f2d94011cd0 tx=0x7f2d9400f610 comp rx=0 tx=0).stop 2023-11-28T18:30:26.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2daa4fe640 1 -- 172.21.15.105:0/2247016437 shutdown_connections 2023-11-28T18:30:26.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2daa4fe640 1 --2- 172.21.15.105:0/2247016437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2da409c4a0 0x7f2da409c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2daa4fe640 1 --2- 172.21.15.105:0/2247016437 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2da40cad20 0x7f2da41512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2daa4fe640 1 --2- 172.21.15.105:0/2247016437 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2d90066900 0x7f2d90068dc0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2daa4fe640 1 --2- 172.21.15.105:0/2247016437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d8c004d10 0x7f2da40c8d50 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:26.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2daa4fe640 1 -- 172.21.15.105:0/2247016437 >> 172.21.15.105:0/2247016437 conn(0x7f2da40d0df0 msgr2=0x7f2da406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:26.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2daa4fe640 1 -- 172.21.15.105:0/2247016437 shutdown_connections 2023-11-28T18:30:26.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:26.929+0000 7f2daa4fe640 1 -- 172.21.15.105:0/2247016437 wait complete. 2023-11-28T18:30:26.947 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T18:30:27.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.085+0000 7f5931355640 1 Processor -- start 2023-11-28T18:30:27.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.085+0000 7f5931355640 1 -- start start 2023-11-28T18:30:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.089+0000 7f5931355640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f592c0d00b0 0x7f592c0cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.089+0000 7f5931355640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f592c0d3e60 con 0x7f592c0b8a20 2023-11-28T18:30:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.089+0000 7f5931355640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f592c0b5fe0 con 0x7f592c0ceb40 2023-11-28T18:30:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.089+0000 7f5931355640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f592c0b62c0 con 0x7f592c0d00b0 2023-11-28T18:30:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.089+0000 7f592affd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f592c0b8a20 0x7f592c0cf9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49160/0 (socket says 172.21.15.105:49160) 2023-11-28T18:30:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.089+0000 7f592affd640 1 -- 172.21.15.105:0/632258827 learned_addr learned my addr 172.21.15.105:0/632258827 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.089+0000 7f592a7fc640 1 --2- 172.21.15.105:0/632258827 >> v2:172.21.15.148:3300/0 conn(0x7f592c0d00b0 0x7f592c0cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.089+0000 7f5929ffb640 1 -- 172.21.15.105:0/632258827 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4052895636 0 0) 0x7f592c0d3e60 con 0x7f592c0b8a20 2023-11-28T18:30:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.089+0000 7f5929ffb640 1 -- 172.21.15.105:0/632258827 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5914003530 con 0x7f592c0b8a20 2023-11-28T18:30:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.089+0000 7f592a7fc640 1 -- 172.21.15.105:0/632258827 >> v1:172.21.15.148:6789/0 conn(0x7f592c0ceb40 legacy=0x7f592c0b6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.089+0000 7f592a7fc640 1 -- 172.21.15.105:0/632258827 >> v1:172.21.15.105:6789/0 conn(0x7f592c0b8a20 legacy=0x7f592c0cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.089+0000 7f592a7fc640 1 -- 172.21.15.105:0/632258827 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592c0b65a0 con 0x7f592c0d00b0 2023-11-28T18:30:27.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.093+0000 7f592a7fc640 1 --2- 172.21.15.105:0/632258827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f592c0d00b0 0x7f592c0cdb50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f59200097c0 tx=0x7f592001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5a516307c160308 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.093+0000 7f5929ffb640 1 -- 172.21.15.105:0/632258827 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59200ac410 con 0x7f592c0d00b0 2023-11-28T18:30:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.093+0000 7f5929ffb640 1 -- 172.21.15.105:0/632258827 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59200ac570 con 0x7f592c0d00b0 2023-11-28T18:30:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.093+0000 7f5929ffb640 1 -- 172.21.15.105:0/632258827 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59200ac900 con 0x7f592c0d00b0 2023-11-28T18:30:27.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.093+0000 7f5931355640 1 -- 172.21.15.105:0/632258827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f592c0d00b0 msgr2=0x7f592c0cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:27.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.093+0000 7f5931355640 1 --2- 172.21.15.105:0/632258827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f592c0d00b0 0x7f592c0cdb50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f59200097c0 tx=0x7f592001c450 comp rx=0 tx=0).stop 2023-11-28T18:30:27.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.093+0000 7f5931355640 1 -- 172.21.15.105:0/632258827 shutdown_connections 2023-11-28T18:30:27.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.093+0000 7f5931355640 1 --2- 172.21.15.105:0/632258827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f592c0d00b0 0x7f592c0cdb50 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.093+0000 7f5931355640 1 -- 172.21.15.105:0/632258827 >> 172.21.15.105:0/632258827 conn(0x7f592c09c340 msgr2=0x7f592c0d2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:27.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.093+0000 7f5931355640 1 -- 172.21.15.105:0/632258827 shutdown_connections 2023-11-28T18:30:27.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.093+0000 7f5931355640 1 -- 172.21.15.105:0/632258827 wait complete. 2023-11-28T18:30:27.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.093+0000 7f5931355640 1 Processor -- start 2023-11-28T18:30:27.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.093+0000 7f5931355640 1 -- start start 2023-11-28T18:30:27.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.093+0000 7f5931355640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f592c0b8a20 0x7f592c15bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.093+0000 7f5931355640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f592c0ceb40 0x7f592c1f96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.093+0000 7f5931355640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f592c0d00b0 0x7f592c1f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.093+0000 7f5931355640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c15ba80 con 0x7f592c0b8a20 2023-11-28T18:30:27.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.093+0000 7f5931355640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c15ae70 con 0x7f592c0ceb40 2023-11-28T18:30:27.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.093+0000 7f5931355640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592c15b150 con 0x7f592c0d00b0 2023-11-28T18:30:27.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.093+0000 7f592affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f592c0b8a20 0x7f592c15bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.097+0000 7f592affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f592c0b8a20 0x7f592c15bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55350/0 (socket says 172.21.15.105:55350) 2023-11-28T18:30:27.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.097+0000 7f592affd640 1 -- 172.21.15.105:0/845346645 learned_addr learned my addr 172.21.15.105:0/845346645 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:27.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.097+0000 7f592a7fc640 1 --2- 172.21.15.105:0/845346645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f592c0ceb40 0x7f592c1f96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.097+0000 7f592b7fe640 1 --2- 172.21.15.105:0/845346645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f592c0d00b0 0x7f592c1f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.097+0000 7f592affd640 1 -- 172.21.15.105:0/845346645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f592c0d00b0 msgr2=0x7f592c1f9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:27.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.097+0000 7f592affd640 1 --2- 172.21.15.105:0/845346645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f592c0d00b0 0x7f592c1f9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.097+0000 7f592affd640 1 -- 172.21.15.105:0/845346645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f592c0ceb40 msgr2=0x7f592c1f96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:27.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.097+0000 7f592affd640 1 --2- 172.21.15.105:0/845346645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f592c0ceb40 0x7f592c1f96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.097+0000 7f592affd640 1 -- 172.21.15.105:0/845346645 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5920009420 con 0x7f592c0b8a20 2023-11-28T18:30:27.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.097+0000 7f592a7fc640 1 --2- 172.21.15.105:0/845346645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f592c0ceb40 0x7f592c1f96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:27.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.097+0000 7f592affd640 1 --2- 172.21.15.105:0/845346645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f592c0b8a20 0x7f592c15bf10 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f591c00c420 tx=0x7f591c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:27.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.097+0000 7f59137fe640 1 -- 172.21.15.105:0/845346645 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c005cf0 con 0x7f592c0b8a20 2023-11-28T18:30:27.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.097+0000 7f59137fe640 1 -- 172.21.15.105:0/845346645 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591c005e50 con 0x7f592c0b8a20 2023-11-28T18:30:27.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.097+0000 7f59137fe640 1 -- 172.21.15.105:0/845346645 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c0072d0 con 0x7f592c0b8a20 2023-11-28T18:30:27.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.097+0000 7f5931355640 1 -- 172.21.15.105:0/845346645 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c15b430 con 0x7f592c0b8a20 2023-11-28T18:30:27.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.097+0000 7f5931355640 1 -- 172.21.15.105:0/845346645 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c1fa170 con 0x7f592c0b8a20 2023-11-28T18:30:27.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.097+0000 7f59137fe640 1 -- 172.21.15.105:0/845346645 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f591c007430 con 0x7f592c0b8a20 2023-11-28T18:30:27.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f59137fe640 1 --2- 172.21.15.105:0/845346645 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5918066900 0x7f5918068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f59137fe640 1 -- 172.21.15.105:0/845346645 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f591c083ae0 con 0x7f592c0b8a20 2023-11-28T18:30:27.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f592a7fc640 1 --2- 172.21.15.105:0/845346645 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5918066900 0x7f5918068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f59346ba000 1 Processor -- start 2023-11-28T18:30:27.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f59346ba000 1 -- start start 2023-11-28T18:30:27.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f592a7fc640 1 --2- 172.21.15.105:0/845346645 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5918066900 0x7f5918068dc0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f59200ae000 tx=0x7f59200a6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:27.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f59346ba000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1871d30 0x5599a1872110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f59346ba000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1872650 0x5599a1824fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f59346ba000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599a1825500 0x5599a18258e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f59346ba000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a16ee110 con 0x5599a1872650 2023-11-28T18:30:27.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f59346ba000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a167e9a0 con 0x5599a1825500 2023-11-28T18:30:27.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f59346ba000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599a167bed0 con 0x5599a1871d30 2023-11-28T18:30:27.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f592a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1872650 0x5599a1824fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f592a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1872650 0x5599a1824fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55360/0 (socket says 172.21.15.105:55360) 2023-11-28T18:30:27.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f592a7fc640 1 -- 172.21.15.105:0/2786462790 learned_addr learned my addr 172.21.15.105:0/2786462790 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:27.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f592affd640 1 --2- 172.21.15.105:0/2786462790 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1871d30 0x5599a1872110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f592b7fe640 1 --2- 172.21.15.105:0/2786462790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599a1825500 0x5599a18258e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f592a7fc640 1 -- 172.21.15.105:0/2786462790 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1871d30 msgr2=0x5599a1872110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f592a7fc640 1 --2- 172.21.15.105:0/2786462790 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1871d30 0x5599a1872110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f592a7fc640 1 -- 172.21.15.105:0/2786462790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599a1825500 msgr2=0x5599a18258e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f592a7fc640 1 --2- 172.21.15.105:0/2786462790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599a1825500 0x5599a18258e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f592a7fc640 1 -- 172.21.15.105:0/2786462790 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599a136f890 con 0x5599a1872650 2023-11-28T18:30:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f592b7fe640 1 --2- 172.21.15.105:0/2786462790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599a1825500 0x5599a18258e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f592a7fc640 1 --2- 172.21.15.105:0/2786462790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1872650 0x5599a1824fc0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f592001a080 tx=0x7f5920004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e2e5126cd062ae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f5910ff9640 1 -- 172.21.15.105:0/2786462790 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59200ace10 con 0x5599a1872650 2023-11-28T18:30:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f5910ff9640 1 -- 172.21.15.105:0/2786462790 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59200bd070 con 0x5599a1872650 2023-11-28T18:30:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.101+0000 7f5910ff9640 1 -- 172.21.15.105:0/2786462790 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59200b7bd0 con 0x5599a1872650 2023-11-28T18:30:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f59346ba000 1 -- 172.21.15.105:0/2786462790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1872650 msgr2=0x5599a1824fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:27.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f59346ba000 1 --2- 172.21.15.105:0/2786462790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1872650 0x5599a1824fc0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f592001a080 tx=0x7f5920004740 comp rx=0 tx=0).stop 2023-11-28T18:30:27.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f59346ba000 1 -- 172.21.15.105:0/2786462790 shutdown_connections 2023-11-28T18:30:27.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f59346ba000 1 --2- 172.21.15.105:0/2786462790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599a1825500 0x5599a18258e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f59346ba000 1 --2- 172.21.15.105:0/2786462790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1872650 0x5599a1824fc0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f59346ba000 1 --2- 172.21.15.105:0/2786462790 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1871d30 0x5599a1872110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f59346ba000 1 -- 172.21.15.105:0/2786462790 >> 172.21.15.105:0/2786462790 conn(0x5599a18528c0 msgr2=0x5599a1854cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:27.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f59346ba000 1 -- 172.21.15.105:0/2786462790 shutdown_connections 2023-11-28T18:30:27.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f59346ba000 1 -- 172.21.15.105:0/2786462790 wait complete. 2023-11-28T18:30:27.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f59346ba000 1 -- start start 2023-11-28T18:30:27.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f59346ba000 1 Processor -- start 2023-11-28T18:30:27.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f59346ba000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1825500 0x5599a182b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f59346ba000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599a1871d30 0x5599a182ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f59346ba000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1872650 0x5599a182bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f59346ba000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a16777c0 con 0x5599a1825500 2023-11-28T18:30:27.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f59346ba000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a162fbf0 con 0x5599a1871d30 2023-11-28T18:30:27.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f59346ba000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599a1635a90 con 0x5599a1872650 2023-11-28T18:30:27.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f59346ba000 10 client.0 _ll_register_callbacks cb 0x5599a1812e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:30:27.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f592affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1825500 0x5599a182b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f592affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1825500 0x5599a182b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55372/0 (socket says 172.21.15.105:55372) 2023-11-28T18:30:27.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f592affd640 1 -- 172.21.15.105:0/2052146649 learned_addr learned my addr 172.21.15.105:0/2052146649 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:27.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f592a7fc640 1 --2- 172.21.15.105:0/2052146649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599a1871d30 0x5599a182ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f592b7fe640 1 --2- 172.21.15.105:0/2052146649 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1872650 0x5599a182bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f592affd640 1 -- 172.21.15.105:0/2052146649 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1872650 msgr2=0x5599a182bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:27.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f592affd640 1 --2- 172.21.15.105:0/2052146649 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1872650 0x5599a182bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f592affd640 1 -- 172.21.15.105:0/2052146649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599a1871d30 msgr2=0x5599a182ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:27.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f592affd640 1 --2- 172.21.15.105:0/2052146649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599a1871d30 0x5599a182ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f592affd640 1 -- 172.21.15.105:0/2052146649 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c0b65a0 con 0x5599a1825500 2023-11-28T18:30:27.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f592b7fe640 1 --2- 172.21.15.105:0/2052146649 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1872650 0x5599a182bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:27.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f592a7fc640 1 --2- 172.21.15.105:0/2052146649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599a1871d30 0x5599a182ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:27.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f592affd640 1 --2- 172.21.15.105:0/2052146649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1825500 0x5599a182b530 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f591c054820 tx=0x7f591c05cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6202fd415c772491 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:27.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f5909ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:30:27.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f5909ffb640 1 -- 172.21.15.105:0/2052146649 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c052cf0 con 0x5599a1825500 2023-11-28T18:30:27.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f59346ba000 1 -- 172.21.15.105:0/2052146649 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599a1639470 con 0x5599a1825500 2023-11-28T18:30:27.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.105+0000 7f5909ffb640 1 -- 172.21.15.105:0/2052146649 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f591c055b10 con 0x5599a1825500 2023-11-28T18:30:27.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f5909ffb640 1 -- 172.21.15.105:0/2052146649 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7f591c055cd0 con 0x5599a1825500 2023-11-28T18:30:27.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f59346ba000 10 client.8367 fetch_fsmap learned FSMap version 108 2023-11-28T18:30:27.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f59346ba000 1 -- 172.21.15.105:0/2052146649 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x5599a1638190 con 0x5599a1825500 2023-11-28T18:30:27.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f5909ffb640 1 -- 172.21.15.105:0/2052146649 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5599a1639470 con 0x5599a1825500 2023-11-28T18:30:27.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f5909ffb640 1 -- 172.21.15.105:0/2052146649 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 108) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f591c07baf0 con 0x5599a1825500 2023-11-28T18:30:27.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f59346ba000 10 client.8367 fetch_fsmap finished waiting for FSMap version 108 2023-11-28T18:30:27.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f59346ba000 10 client.8367 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:30:27.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f59346ba000 20 client.8367 populate_metadata read hostname 'smithi105' 2023-11-28T18:30:27.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f59346ba000 1 --2- 172.21.15.105:0/2052146649 >> [v2:172.21.15.105:6834/3968059262,v1:172.21.15.105:6836/3968059262] conn(0x5599a182c9c0 0x5599a1883ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f59346ba000 4 client.8367 mds_command: new command op to 8142 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:30:27.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f59346ba000 1 -- 172.21.15.105:0/2052146649 --> [v2:172.21.15.105:6834/3968059262,v1:172.21.15.105:6836/3968059262] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599a163cb50 con 0x5599a182c9c0 2023-11-28T18:30:27.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f592a7fc640 1 --2- 172.21.15.105:0/2052146649 >> [v2:172.21.15.105:6834/3968059262,v1:172.21.15.105:6836/3968059262] conn(0x5599a182c9c0 0x5599a1883ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f5909ffb640 10 client.8367 ms_handle_connect on v2:172.21.15.105:6834/3968059262 2023-11-28T18:30:27.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f592a7fc640 1 --2- 172.21.15.105:0/2052146649 >> [v2:172.21.15.105:6834/3968059262,v1:172.21.15.105:6836/3968059262] conn(0x5599a182c9c0 0x5599a1883ff0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b81f8a2d5cd9593 server_cookie=27d9b68fed5b21cd in_seq=0 out_seq=0 2023-11-28T18:30:27.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f5909ffb640 1 -- 172.21.15.105:0/2052146649 <== mds.1 v2:172.21.15.105:6834/3968059262 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5599a163cb50 con 0x5599a182c9c0 2023-11-28T18:30:27.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f5909ffb640 10 client.8367 handle_command_reply: tid=0 2023-11-28T18:30:27.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f59346ba000 1 client.8367 shutdown 2023-11-28T18:30:27.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f59346ba000 1 -- 172.21.15.105:0/2052146649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1825500 msgr2=0x5599a182b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:27.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f59346ba000 1 --2- 172.21.15.105:0/2052146649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1825500 0x5599a182b530 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f591c054820 tx=0x7f591c05cc20 comp rx=0 tx=0).stop 2023-11-28T18:30:27.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f59346ba000 1 -- 172.21.15.105:0/2052146649 shutdown_connections 2023-11-28T18:30:27.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f59346ba000 1 --2- 172.21.15.105:0/2052146649 >> [v2:172.21.15.105:6834/3968059262,v1:172.21.15.105:6836/3968059262] conn(0x5599a182c9c0 0x5599a1883ff0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f59346ba000 1 --2- 172.21.15.105:0/2052146649 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1872650 0x5599a182bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f59346ba000 1 --2- 172.21.15.105:0/2052146649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599a1871d30 0x5599a182ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f5909ffb640 0 client.8367 ms_handle_reset on v2:172.21.15.105:6834/3968059262 2023-11-28T18:30:27.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f59346ba000 1 --2- 172.21.15.105:0/2052146649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1825500 0x5599a182b530 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f59346ba000 1 -- 172.21.15.105:0/2052146649 >> 172.21.15.105:0/2052146649 conn(0x5599a18528c0 msgr2=0x5599a18246b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:27.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f59346ba000 1 -- 172.21.15.105:0/2052146649 shutdown_connections 2023-11-28T18:30:27.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f59346ba000 1 -- 172.21.15.105:0/2052146649 wait complete. 2023-11-28T18:30:27.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.109+0000 7f59346ba000 20 client.8367 trim_cache size 0 max 16384 2023-11-28T18:30:27.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.125+0000 7f59346ba000 1 Processor -- start 2023-11-28T18:30:27.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.125+0000 7f59346ba000 1 -- start start 2023-11-28T18:30:27.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.125+0000 7f59346ba000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599a1825500 0x5599a1886430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.125+0000 7f59346ba000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1871d30 0x5599a18513a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.125+0000 7f59346ba000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1872650 0x5599a18518e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.125+0000 7f59346ba000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a1330940 con 0x5599a1872650 2023-11-28T18:30:27.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.125+0000 7f59346ba000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a1828e20 con 0x5599a1825500 2023-11-28T18:30:27.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.125+0000 7f59346ba000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599a16b8440 con 0x5599a1871d30 2023-11-28T18:30:27.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.125+0000 7f592b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1872650 0x5599a18518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.125+0000 7f592b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1872650 0x5599a18518e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55386/0 (socket says 172.21.15.105:55386) 2023-11-28T18:30:27.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.125+0000 7f592b7fe640 1 -- 172.21.15.105:0/1234852346 learned_addr learned my addr 172.21.15.105:0/1234852346 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:27.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.125+0000 7f592affd640 1 --2- 172.21.15.105:0/1234852346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599a1825500 0x5599a1886430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.125+0000 7f592a7fc640 1 --2- 172.21.15.105:0/1234852346 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1871d30 0x5599a18513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.125+0000 7f592b7fe640 1 -- 172.21.15.105:0/1234852346 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1871d30 msgr2=0x5599a18513a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:27.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.125+0000 7f592b7fe640 1 --2- 172.21.15.105:0/1234852346 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1871d30 0x5599a18513a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.125+0000 7f592b7fe640 1 -- 172.21.15.105:0/1234852346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599a1825500 msgr2=0x5599a1886430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:27.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.125+0000 7f592b7fe640 1 --2- 172.21.15.105:0/1234852346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599a1825500 0x5599a1886430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.125+0000 7f592b7fe640 1 -- 172.21.15.105:0/1234852346 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599a1825370 con 0x5599a1872650 2023-11-28T18:30:27.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.125+0000 7f592a7fc640 1 --2- 172.21.15.105:0/1234852346 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1871d30 0x5599a18513a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:27.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.125+0000 7f592b7fe640 1 --2- 172.21.15.105:0/1234852346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1872650 0x5599a18518e0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f59240051b0 tx=0x7f5924003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe4f1edd7fe7ce7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:27.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.125+0000 7f59117fa640 1 -- 172.21.15.105:0/1234852346 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5924005850 con 0x5599a1872650 2023-11-28T18:30:27.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59117fa640 1 -- 172.21.15.105:0/1234852346 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5924006b60 con 0x5599a1872650 2023-11-28T18:30:27.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59117fa640 1 -- 172.21.15.105:0/1234852346 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5924012840 con 0x5599a1872650 2023-11-28T18:30:27.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59346ba000 1 -- 172.21.15.105:0/1234852346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1872650 msgr2=0x5599a18518e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:27.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59346ba000 1 --2- 172.21.15.105:0/1234852346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1872650 0x5599a18518e0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f59240051b0 tx=0x7f5924003f90 comp rx=0 tx=0).stop 2023-11-28T18:30:27.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59346ba000 1 -- 172.21.15.105:0/1234852346 shutdown_connections 2023-11-28T18:30:27.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59346ba000 1 --2- 172.21.15.105:0/1234852346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1872650 0x5599a18518e0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59346ba000 1 --2- 172.21.15.105:0/1234852346 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1871d30 0x5599a18513a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59346ba000 1 --2- 172.21.15.105:0/1234852346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599a1825500 0x5599a1886430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59346ba000 1 -- 172.21.15.105:0/1234852346 >> 172.21.15.105:0/1234852346 conn(0x5599a18528c0 msgr2=0x5599a1823720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:27.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59346ba000 1 -- 172.21.15.105:0/1234852346 shutdown_connections 2023-11-28T18:30:27.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59346ba000 1 -- 172.21.15.105:0/1234852346 wait complete. 2023-11-28T18:30:27.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59346ba000 1 -- start start 2023-11-28T18:30:27.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59346ba000 1 Processor -- start 2023-11-28T18:30:27.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59346ba000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1825500 0x5599a18d2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59346ba000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599a1871d30 0x5599a18d2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59346ba000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1872650 0x5599a188da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59346ba000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a167b0a0 con 0x5599a1825500 2023-11-28T18:30:27.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59346ba000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a182cfd0 con 0x5599a1871d30 2023-11-28T18:30:27.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59346ba000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599a164cb70 con 0x5599a1872650 2023-11-28T18:30:27.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59346ba000 10 client.0 _ll_register_callbacks cb 0x5599a16ee060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:30:27.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f592affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1825500 0x5599a18d2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f592affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1825500 0x5599a18d2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55400/0 (socket says 172.21.15.105:55400) 2023-11-28T18:30:27.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f592affd640 1 -- 172.21.15.105:0/2638573807 learned_addr learned my addr 172.21.15.105:0/2638573807 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:27.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f592a7fc640 1 --2- 172.21.15.105:0/2638573807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599a1871d30 0x5599a18d2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f592b7fe640 1 --2- 172.21.15.105:0/2638573807 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1872650 0x5599a188da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f592affd640 1 -- 172.21.15.105:0/2638573807 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1872650 msgr2=0x5599a188da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:27.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f592affd640 1 --2- 172.21.15.105:0/2638573807 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1872650 0x5599a188da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f592affd640 1 -- 172.21.15.105:0/2638573807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599a1871d30 msgr2=0x5599a18d2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:27.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f592affd640 1 --2- 172.21.15.105:0/2638573807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599a1871d30 0x5599a18d2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f592affd640 1 -- 172.21.15.105:0/2638573807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592400f040 con 0x5599a1825500 2023-11-28T18:30:27.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f592b7fe640 1 --2- 172.21.15.105:0/2638573807 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1872650 0x5599a188da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:27.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f592affd640 1 --2- 172.21.15.105:0/2638573807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1825500 0x5599a18d2600 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f591c0544f0 tx=0x7f591c057070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c3eccb9ad1371 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:27.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f5909ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:30:27.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f5909ffb640 1 -- 172.21.15.105:0/2638573807 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c054920 con 0x5599a1825500 2023-11-28T18:30:27.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f5909ffb640 1 -- 172.21.15.105:0/2638573807 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f591c052cf0 con 0x5599a1825500 2023-11-28T18:30:27.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59346ba000 1 -- 172.21.15.105:0/2638573807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599a1676110 con 0x5599a1825500 2023-11-28T18:30:27.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f5909ffb640 1 -- 172.21.15.105:0/2638573807 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7f591c00e040 con 0x5599a1825500 2023-11-28T18:30:27.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59346ba000 10 client.8373 fetch_fsmap learned FSMap version 108 2023-11-28T18:30:27.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f59346ba000 1 -- 172.21.15.105:0/2638573807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x5599a161e490 con 0x5599a1825500 2023-11-28T18:30:27.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.129+0000 7f5909ffb640 1 -- 172.21.15.105:0/2638573807 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591c076020 con 0x5599a1825500 2023-11-28T18:30:27.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f5909ffb640 1 -- 172.21.15.105:0/2638573807 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 108) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f591c050880 con 0x5599a1825500 2023-11-28T18:30:27.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f59346ba000 10 client.8373 fetch_fsmap finished waiting for FSMap version 108 2023-11-28T18:30:27.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f59346ba000 10 client.8373 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:30:27.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f59346ba000 20 client.8373 populate_metadata read hostname 'smithi105' 2023-11-28T18:30:27.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f59346ba000 1 --2- 172.21.15.105:0/2638573807 >> [v2:172.21.15.105:6834/3968059262,v1:172.21.15.105:6836/3968059262] conn(0x5599a188e370 0x5599a188e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f59346ba000 4 client.8373 mds_command: new command op to 8142 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:30:27.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f59346ba000 1 -- 172.21.15.105:0/2638573807 --> [v2:172.21.15.105:6834/3968059262,v1:172.21.15.105:6836/3968059262] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5599a16a4650 con 0x5599a188e370 2023-11-28T18:30:27.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f592a7fc640 1 --2- 172.21.15.105:0/2638573807 >> [v2:172.21.15.105:6834/3968059262,v1:172.21.15.105:6836/3968059262] conn(0x5599a188e370 0x5599a188e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f592a7fc640 1 --2- 172.21.15.105:0/2638573807 >> [v2:172.21.15.105:6834/3968059262,v1:172.21.15.105:6836/3968059262] conn(0x5599a188e370 0x5599a188e750 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=239f86e1ee230985 server_cookie=df8a4e8c642f4b0 in_seq=0 out_seq=0 2023-11-28T18:30:27.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f5909ffb640 10 client.8373 ms_handle_connect on v2:172.21.15.105:6834/3968059262 2023-11-28T18:30:27.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f5909ffb640 1 -- 172.21.15.105:0/2638573807 <== mds.1 v2:172.21.15.105:6834/3968059262 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x5599a16a4650 con 0x5599a188e370 2023-11-28T18:30:27.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f5909ffb640 10 client.8373 handle_command_reply: tid=0 2023-11-28T18:30:27.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f59346ba000 1 client.8373 shutdown 2023-11-28T18:30:27.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f59346ba000 1 -- 172.21.15.105:0/2638573807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1825500 msgr2=0x5599a18d2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:27.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f59346ba000 1 --2- 172.21.15.105:0/2638573807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1825500 0x5599a18d2600 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f591c0544f0 tx=0x7f591c057070 comp rx=0 tx=0).stop 2023-11-28T18:30:27.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f59346ba000 1 -- 172.21.15.105:0/2638573807 shutdown_connections 2023-11-28T18:30:27.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f59346ba000 1 --2- 172.21.15.105:0/2638573807 >> [v2:172.21.15.105:6834/3968059262,v1:172.21.15.105:6836/3968059262] conn(0x5599a188e370 0x5599a188e750 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f59346ba000 1 --2- 172.21.15.105:0/2638573807 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5599a1872650 0x5599a188da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f59346ba000 1 --2- 172.21.15.105:0/2638573807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599a1871d30 0x5599a18d2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f5909ffb640 0 client.8373 ms_handle_reset on v2:172.21.15.105:6834/3968059262 2023-11-28T18:30:27.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f59346ba000 1 --2- 172.21.15.105:0/2638573807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599a1825500 0x5599a18d2600 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f59346ba000 1 -- 172.21.15.105:0/2638573807 >> 172.21.15.105:0/2638573807 conn(0x5599a18528c0 msgr2=0x5599a1823fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:27.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f59346ba000 1 -- 172.21.15.105:0/2638573807 shutdown_connections 2023-11-28T18:30:27.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f59346ba000 1 -- 172.21.15.105:0/2638573807 wait complete. 2023-11-28T18:30:27.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f59346ba000 20 client.8373 trim_cache size 0 max 16384 2023-11-28T18:30:27.141 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:30:27.141 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:30:27.141 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 1, 2023-11-28T18:30:27.141 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 8142, 2023-11-28T18:30:27.141 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:30:27.141 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:30:27.143 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:30:27.143 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 2.9741770619999999, 2023-11-28T18:30:27.143 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 108, 2023-11-28T18:30:27.143 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 105, 2023-11-28T18:30:27.143 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 105, 2023-11-28T18:30:27.143 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 26.312092168 2023-11-28T18:30:27.143 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:30:27.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f5931355640 1 -- 172.21.15.105:0/845346645 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5918066900 msgr2=0x7f5918068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:27.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f5931355640 1 --2- 172.21.15.105:0/845346645 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5918066900 0x7f5918068dc0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f59200ae000 tx=0x7f59200a6040 comp rx=0 tx=0).stop 2023-11-28T18:30:27.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f5931355640 1 -- 172.21.15.105:0/845346645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f592c0b8a20 msgr2=0x7f592c15bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:27.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f5931355640 1 --2- 172.21.15.105:0/845346645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f592c0b8a20 0x7f592c15bf10 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f591c00c420 tx=0x7f591c00c8e0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f5931355640 1 -- 172.21.15.105:0/845346645 shutdown_connections 2023-11-28T18:30:27.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f5931355640 1 --2- 172.21.15.105:0/845346645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f592c0d00b0 0x7f592c1f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f5931355640 1 --2- 172.21.15.105:0/845346645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f592c0ceb40 0x7f592c1f96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f5931355640 1 --2- 172.21.15.105:0/845346645 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5918066900 0x7f5918068dc0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f5931355640 1 --2- 172.21.15.105:0/845346645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f592c0b8a20 0x7f592c15bf10 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f5931355640 1 -- 172.21.15.105:0/845346645 >> 172.21.15.105:0/845346645 conn(0x7f592c09c340 msgr2=0x7f592c15dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:27.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f5931355640 1 -- 172.21.15.105:0/845346645 shutdown_connections 2023-11-28T18:30:27.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.133+0000 7f5931355640 1 -- 172.21.15.105:0/845346645 wait complete. 2023-11-28T18:30:27.153 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T18:30:27.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.293+0000 7f32f8e6b640 1 Processor -- start 2023-11-28T18:30:27.295 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.293+0000 7f32f8e6b640 1 -- start start 2023-11-28T18:30:27.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f8e6b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32f4071960 0x7f32f415f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f8e6b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f32f415b030 0x7f32f40709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f8e6b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f40701d0 con 0x7f32f40b4290 2023-11-28T18:30:27.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f8e6b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f406df80 con 0x7f32f415b030 2023-11-28T18:30:27.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f8e6b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f4071df0 con 0x7f32f4071960 2023-11-28T18:30:27.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f259c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f32f40b4290 0x7f32f40b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49172/0 (socket says 172.21.15.105:49172) 2023-11-28T18:30:27.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f259c640 1 -- 172.21.15.105:0/3612983895 learned_addr learned my addr 172.21.15.105:0/3612983895 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:27.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f1d9b640 1 -- 172.21.15.105:0/3612983895 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 107333088 0 0) 0x7f32f40701d0 con 0x7f32f40b4290 2023-11-28T18:30:27.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f1d9b640 1 -- 172.21.15.105:0/3612983895 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32e0003530 con 0x7f32f40b4290 2023-11-28T18:30:27.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32e9d9b640 1 --2- 172.21.15.105:0/3612983895 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32f4071960 0x7f32f415f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f2d9d640 1 --2- 172.21.15.105:0/3612983895 >> v2:172.21.15.148:3300/0 conn(0x7f32f415b030 0x7f32f40709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f1d9b640 1 -- 172.21.15.105:0/3612983895 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1233682428 0 0) 0x7f32e0003530 con 0x7f32f40b4290 2023-11-28T18:30:27.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f1d9b640 1 -- 172.21.15.105:0/3612983895 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32f4071960 msgr2=0x7f32f415f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:27.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f1d9b640 1 --2- 172.21.15.105:0/3612983895 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32f4071960 0x7f32f415f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f1d9b640 1 -- 172.21.15.105:0/3612983895 >> v2:172.21.15.148:3300/0 conn(0x7f32f415b030 msgr2=0x7f32f40709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:27.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f1d9b640 1 --2- 172.21.15.105:0/3612983895 >> v2:172.21.15.148:3300/0 conn(0x7f32f415b030 0x7f32f40709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f1d9b640 1 -- 172.21.15.105:0/3612983895 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f40720d0 con 0x7f32f40b4290 2023-11-28T18:30:27.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f1d9b640 1 -- 172.21.15.105:0/3612983895 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f32e4001a00 con 0x7f32f40b4290 2023-11-28T18:30:27.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f1d9b640 1 -- 172.21.15.105:0/3612983895 >> v1:172.21.15.105:6789/0 conn(0x7f32f40b4290 legacy=0x7f32f40b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:27.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f1d9b640 1 --2- 172.21.15.105:0/3612983895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e0003980 0x7f32e0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f1d9b640 1 --2- 172.21.15.105:0/3612983895 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32e00047d0 0x7f32e0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f1d9b640 1 --2- 172.21.15.105:0/3612983895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32e0005150 0x7f32e000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f1d9b640 1 -- 172.21.15.105:0/3612983895 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f406df80 con 0x7f32e0003980 2023-11-28T18:30:27.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f1d9b640 1 -- 172.21.15.105:0/3612983895 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f4071df0 con 0x7f32e0005150 2023-11-28T18:30:27.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f1d9b640 1 -- 172.21.15.105:0/3612983895 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e00045d0 con 0x7f32e00047d0 2023-11-28T18:30:27.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f259c640 1 --2- 172.21.15.105:0/3612983895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e0003980 0x7f32e0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f2d9d640 1 --2- 172.21.15.105:0/3612983895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32e0005150 0x7f32e000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32e9d9b640 1 --2- 172.21.15.105:0/3612983895 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32e00047d0 0x7f32e0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f259c640 1 -- 172.21.15.105:0/3612983895 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32e00047d0 msgr2=0x7f32e0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:27.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f259c640 1 --2- 172.21.15.105:0/3612983895 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32e00047d0 0x7f32e0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f259c640 1 -- 172.21.15.105:0/3612983895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32e0005150 msgr2=0x7f32e000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:27.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f259c640 1 --2- 172.21.15.105:0/3612983895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32e0005150 0x7f32e000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32f259c640 1 -- 172.21.15.105:0/3612983895 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e0003530 con 0x7f32e0003980 2023-11-28T18:30:27.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.297+0000 7f32e9d9b640 1 --2- 172.21.15.105:0/3612983895 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32e00047d0 0x7f32e0004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:27.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.301+0000 7f32f259c640 1 --2- 172.21.15.105:0/3612983895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e0003980 0x7f32e0003da0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f32e400a830 tx=0x7f32e401d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc2f6c3f4a3fc96 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:27.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.301+0000 7f32f1d9b640 1 -- 172.21.15.105:0/3612983895 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e40ad430 con 0x7f32e0003980 2023-11-28T18:30:27.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.301+0000 7f32f1d9b640 1 -- 172.21.15.105:0/3612983895 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e40a8070 con 0x7f32e0003980 2023-11-28T18:30:27.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.301+0000 7f32f1d9b640 1 -- 172.21.15.105:0/3612983895 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e40ad7f0 con 0x7f32e0003980 2023-11-28T18:30:27.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.301+0000 7f32f8e6b640 1 -- 172.21.15.105:0/3612983895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e0003980 msgr2=0x7f32e0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:27.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.301+0000 7f32f8e6b640 1 --2- 172.21.15.105:0/3612983895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e0003980 0x7f32e0003da0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f32e400a830 tx=0x7f32e401d330 comp rx=0 tx=0).stop 2023-11-28T18:30:27.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.301+0000 7f32f2d9d640 1 -- 172.21.15.105:0/3612983895 reap_dead start 2023-11-28T18:30:27.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.301+0000 7f32f8e6b640 1 -- 172.21.15.105:0/3612983895 shutdown_connections 2023-11-28T18:30:27.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.301+0000 7f32f8e6b640 1 -- 172.21.15.105:0/3612983895 >> 172.21.15.105:0/3612983895 conn(0x7f32f40ceb90 msgr2=0x7f32f40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:27.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.301+0000 7f32f8e6b640 1 -- 172.21.15.105:0/3612983895 shutdown_connections 2023-11-28T18:30:27.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.301+0000 7f32f8e6b640 1 -- 172.21.15.105:0/3612983895 wait complete. 2023-11-28T18:30:27.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32f8e6b640 1 Processor -- start 2023-11-28T18:30:27.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32f8e6b640 1 -- start start 2023-11-28T18:30:27.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32f8e6b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32e0003980 0x7f32f4150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32f8e6b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32f415a530 0x7f32f409bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32f8e6b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32f40c7f40 0x7f32f4160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32f8e6b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f4150ff0 con 0x7f32f40c7f40 2023-11-28T18:30:27.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32f8e6b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f4164a80 con 0x7f32e0003980 2023-11-28T18:30:27.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32f8e6b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f409c500 con 0x7f32f415a530 2023-11-28T18:30:27.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32f2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32f40c7f40 0x7f32f4160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32f2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32f40c7f40 0x7f32f4160ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55420/0 (socket says 172.21.15.105:55420) 2023-11-28T18:30:27.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32f2d9d640 1 -- 172.21.15.105:0/1621956949 learned_addr learned my addr 172.21.15.105:0/1621956949 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:27.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32e9d9b640 1 --2- 172.21.15.105:0/1621956949 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32f415a530 0x7f32f409bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32f2d9d640 1 -- 172.21.15.105:0/1621956949 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32f415a530 msgr2=0x7f32f409bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:27.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32f2d9d640 1 --2- 172.21.15.105:0/1621956949 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32f415a530 0x7f32f409bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32f2d9d640 1 -- 172.21.15.105:0/1621956949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32e0003980 msgr2=0x7f32f4150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:27.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32f259c640 1 --2- 172.21.15.105:0/1621956949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32e0003980 0x7f32f4150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32f2d9d640 1 --2- 172.21.15.105:0/1621956949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32e0003980 0x7f32f4150ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32f2d9d640 1 -- 172.21.15.105:0/1621956949 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f409c7a0 con 0x7f32f40c7f40 2023-11-28T18:30:27.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32e9d9b640 1 --2- 172.21.15.105:0/1621956949 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32f415a530 0x7f32f409bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:27.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32f259c640 1 --2- 172.21.15.105:0/1621956949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32e0003980 0x7f32f4150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:27.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32f2d9d640 1 --2- 172.21.15.105:0/1621956949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32f40c7f40 0x7f32f4160ca0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f32f409d0d0 tx=0x7f32ec011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:27.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32eb7fe640 1 -- 172.21.15.105:0/1621956949 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec004240 con 0x7f32f40c7f40 2023-11-28T18:30:27.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32eb7fe640 1 -- 172.21.15.105:0/1621956949 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ec0043a0 con 0x7f32f40c7f40 2023-11-28T18:30:27.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32eb7fe640 1 -- 172.21.15.105:0/1621956949 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec00f930 con 0x7f32f40c7f40 2023-11-28T18:30:27.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32f8e6b640 1 -- 172.21.15.105:0/1621956949 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f4158630 con 0x7f32f40c7f40 2023-11-28T18:30:27.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.305+0000 7f32f8e6b640 1 -- 172.21.15.105:0/1621956949 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f4158bf0 con 0x7f32f40c7f40 2023-11-28T18:30:27.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.309+0000 7f32eb7fe640 1 -- 172.21.15.105:0/1621956949 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f32ec00fa90 con 0x7f32f40c7f40 2023-11-28T18:30:27.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.309+0000 7f32eb7fe640 1 --2- 172.21.15.105:0/1621956949 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f32d4066900 0x7f32d4068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.309+0000 7f32eb7fe640 1 -- 172.21.15.105:0/1621956949 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f32ec0847e0 con 0x7f32f40c7f40 2023-11-28T18:30:27.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.309+0000 7f32f259c640 1 --2- 172.21.15.105:0/1621956949 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f32d4066900 0x7f32d4068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.309+0000 7f32fc1d7000 1 Processor -- start 2023-11-28T18:30:27.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.309+0000 7f32fc1d7000 1 -- start start 2023-11-28T18:30:27.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32f259c640 1 --2- 172.21.15.105:0/1621956949 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f32d4066900 0x7f32d4068dc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f32e40ad000 tx=0x7f32e4001d90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:27.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32fc1d7000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55589b6aad30 0x55589b6ab110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32fc1d7000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55589b6ab650 0x55589b65dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32fc1d7000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b65e500 0x55589b65e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32fc1d7000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55589b527110 con 0x55589b65e500 2023-11-28T18:30:27.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32fc1d7000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55589b4b79a0 con 0x55589b6aad30 2023-11-28T18:30:27.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32fc1d7000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55589b4b4ed0 con 0x55589b6ab650 2023-11-28T18:30:27.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32f2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b65e500 0x55589b65e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32f2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b65e500 0x55589b65e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55436/0 (socket says 172.21.15.105:55436) 2023-11-28T18:30:27.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32f2d9d640 1 -- 172.21.15.105:0/4165171819 learned_addr learned my addr 172.21.15.105:0/4165171819 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:27.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32e9d9b640 1 --2- 172.21.15.105:0/4165171819 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55589b6ab650 0x55589b65dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32f259c640 1 --2- 172.21.15.105:0/4165171819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55589b6aad30 0x55589b6ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32f2d9d640 1 -- 172.21.15.105:0/4165171819 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55589b6ab650 msgr2=0x55589b65dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:27.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32f2d9d640 1 --2- 172.21.15.105:0/4165171819 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55589b6ab650 0x55589b65dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32f2d9d640 1 -- 172.21.15.105:0/4165171819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55589b6aad30 msgr2=0x55589b6ab110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:27.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32f2d9d640 1 --2- 172.21.15.105:0/4165171819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55589b6aad30 0x55589b6ab110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32f2d9d640 1 -- 172.21.15.105:0/4165171819 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55589b1a8890 con 0x55589b65e500 2023-11-28T18:30:27.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32e9d9b640 1 --2- 172.21.15.105:0/4165171819 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55589b6ab650 0x55589b65dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:27.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32f2d9d640 1 --2- 172.21.15.105:0/4165171819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b65e500 0x55589b65e8e0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f32ec05d350 tx=0x7f32ec055660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3880a6d22d7388 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:27.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32d3fff640 1 -- 172.21.15.105:0/4165171819 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec0536e0 con 0x55589b65e500 2023-11-28T18:30:27.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32d3fff640 1 -- 172.21.15.105:0/4165171819 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ec053940 con 0x55589b65e500 2023-11-28T18:30:27.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32d3fff640 1 -- 172.21.15.105:0/4165171819 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec051cb0 con 0x55589b65e500 2023-11-28T18:30:27.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32fc1d7000 1 -- 172.21.15.105:0/4165171819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b65e500 msgr2=0x55589b65e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:27.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32fc1d7000 1 --2- 172.21.15.105:0/4165171819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b65e500 0x55589b65e8e0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f32ec05d350 tx=0x7f32ec055660 comp rx=0 tx=0).stop 2023-11-28T18:30:27.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32fc1d7000 1 -- 172.21.15.105:0/4165171819 shutdown_connections 2023-11-28T18:30:27.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32fc1d7000 1 --2- 172.21.15.105:0/4165171819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b65e500 0x55589b65e8e0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32fc1d7000 1 --2- 172.21.15.105:0/4165171819 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55589b6ab650 0x55589b65dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32fc1d7000 1 --2- 172.21.15.105:0/4165171819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55589b6aad30 0x55589b6ab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32fc1d7000 1 -- 172.21.15.105:0/4165171819 >> 172.21.15.105:0/4165171819 conn(0x55589b68b8c0 msgr2=0x55589b68dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:27.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32fc1d7000 1 -- 172.21.15.105:0/4165171819 shutdown_connections 2023-11-28T18:30:27.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32fc1d7000 1 -- 172.21.15.105:0/4165171819 wait complete. 2023-11-28T18:30:27.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32fc1d7000 1 -- start start 2023-11-28T18:30:27.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.313+0000 7f32fc1d7000 1 Processor -- start 2023-11-28T18:30:27.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32fc1d7000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b65e500 0x55589b6644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32fc1d7000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55589b6aad30 0x55589b664a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32fc1d7000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55589b6ab650 0x55589b664f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32fc1d7000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55589b4b07c0 con 0x55589b65e500 2023-11-28T18:30:27.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32fc1d7000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55589b468bf0 con 0x55589b6ab650 2023-11-28T18:30:27.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32fc1d7000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55589b46ea90 con 0x55589b6aad30 2023-11-28T18:30:27.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32fc1d7000 10 client.0 _ll_register_callbacks cb 0x55589b64be20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:30:27.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32f259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b65e500 0x55589b6644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32f259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b65e500 0x55589b6644e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55446/0 (socket says 172.21.15.105:55446) 2023-11-28T18:30:27.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32f259c640 1 -- 172.21.15.105:0/2107774703 learned_addr learned my addr 172.21.15.105:0/2107774703 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:27.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32e9d9b640 1 --2- 172.21.15.105:0/2107774703 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55589b6aad30 0x55589b664a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32f2d9d640 1 --2- 172.21.15.105:0/2107774703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55589b6ab650 0x55589b664f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32f259c640 1 -- 172.21.15.105:0/2107774703 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55589b6aad30 msgr2=0x55589b664a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:27.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32f259c640 1 --2- 172.21.15.105:0/2107774703 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55589b6aad30 0x55589b664a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32f259c640 1 -- 172.21.15.105:0/2107774703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55589b6ab650 msgr2=0x55589b664f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:27.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32f259c640 1 --2- 172.21.15.105:0/2107774703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55589b6ab650 0x55589b664f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32f259c640 1 -- 172.21.15.105:0/2107774703 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f4158bf0 con 0x55589b65e500 2023-11-28T18:30:27.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32f259c640 1 --2- 172.21.15.105:0/2107774703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b65e500 0x55589b6644e0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f32e401cda0 tx=0x7f32e401a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6522298db75657f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:27.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:30:27.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32d17fa640 1 -- 172.21.15.105:0/2107774703 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e40c9070 con 0x55589b65e500 2023-11-28T18:30:27.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32d17fa640 1 -- 172.21.15.105:0/2107774703 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f32e40ad980 con 0x55589b65e500 2023-11-28T18:30:27.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32fc1d7000 1 -- 172.21.15.105:0/2107774703 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55589b472470 con 0x55589b65e500 2023-11-28T18:30:27.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32d17fa640 1 -- 172.21.15.105:0/2107774703 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32e40adb40 con 0x55589b65e500 2023-11-28T18:30:27.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32fc1d7000 10 client.8385 fetch_fsmap learned FSMap version 108 2023-11-28T18:30:27.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32fc1d7000 1 -- 172.21.15.105:0/2107774703 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x55589b471190 con 0x55589b65e500 2023-11-28T18:30:27.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32d17fa640 1 -- 172.21.15.105:0/2107774703 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55589b472470 con 0x55589b65e500 2023-11-28T18:30:27.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32d17fa640 1 -- 172.21.15.105:0/2107774703 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 108) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f32e40cc020 con 0x55589b65e500 2023-11-28T18:30:27.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32fc1d7000 10 client.8385 fetch_fsmap finished waiting for FSMap version 108 2023-11-28T18:30:27.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32fc1d7000 10 client.8385 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:30:27.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32fc1d7000 20 client.8385 populate_metadata read hostname 'smithi105' 2023-11-28T18:30:27.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32fc1d7000 1 --2- 172.21.15.105:0/2107774703 >> [v2:172.21.15.105:6835/2782729992,v1:172.21.15.105:6837/2782729992] conn(0x55589b665970 0x55589b6bcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32fc1d7000 4 client.8385 mds_command: new command op to 8145 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:30:27.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32fc1d7000 1 -- 172.21.15.105:0/2107774703 --> [v2:172.21.15.105:6835/2782729992,v1:172.21.15.105:6837/2782729992] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55589b475b50 con 0x55589b665970 2023-11-28T18:30:27.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32e9d9b640 1 --2- 172.21.15.105:0/2107774703 >> [v2:172.21.15.105:6835/2782729992,v1:172.21.15.105:6837/2782729992] conn(0x55589b665970 0x55589b6bcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32d17fa640 10 client.8385 ms_handle_connect on v2:172.21.15.105:6835/2782729992 2023-11-28T18:30:27.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32e9d9b640 1 --2- 172.21.15.105:0/2107774703 >> [v2:172.21.15.105:6835/2782729992,v1:172.21.15.105:6837/2782729992] conn(0x55589b665970 0x55589b6bcff0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eb340bed9f337d3 server_cookie=22ecff8adf4664a8 in_seq=0 out_seq=0 2023-11-28T18:30:27.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32d17fa640 1 -- 172.21.15.105:0/2107774703 <== mds.0 v2:172.21.15.105:6835/2782729992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55589b475b50 con 0x55589b665970 2023-11-28T18:30:27.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32d17fa640 10 client.8385 handle_command_reply: tid=0 2023-11-28T18:30:27.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.317+0000 7f32fc1d7000 1 client.8385 shutdown 2023-11-28T18:30:27.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.321+0000 7f32fc1d7000 1 -- 172.21.15.105:0/2107774703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b65e500 msgr2=0x55589b6644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:27.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.321+0000 7f32fc1d7000 1 --2- 172.21.15.105:0/2107774703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b65e500 0x55589b6644e0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f32e401cda0 tx=0x7f32e401a0d0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.321+0000 7f32fc1d7000 1 -- 172.21.15.105:0/2107774703 shutdown_connections 2023-11-28T18:30:27.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.321+0000 7f32fc1d7000 1 --2- 172.21.15.105:0/2107774703 >> [v2:172.21.15.105:6835/2782729992,v1:172.21.15.105:6837/2782729992] conn(0x55589b665970 0x55589b6bcff0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.321+0000 7f32fc1d7000 1 --2- 172.21.15.105:0/2107774703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55589b6ab650 0x55589b664f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.321+0000 7f32d17fa640 0 client.8385 ms_handle_reset on v2:172.21.15.105:6835/2782729992 2023-11-28T18:30:27.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.321+0000 7f32fc1d7000 1 --2- 172.21.15.105:0/2107774703 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55589b6aad30 0x55589b664a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.321+0000 7f32fc1d7000 1 --2- 172.21.15.105:0/2107774703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b65e500 0x55589b6644e0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.321+0000 7f32fc1d7000 1 -- 172.21.15.105:0/2107774703 >> 172.21.15.105:0/2107774703 conn(0x55589b68b8c0 msgr2=0x55589b65d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:27.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.321+0000 7f32fc1d7000 1 -- 172.21.15.105:0/2107774703 shutdown_connections 2023-11-28T18:30:27.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.321+0000 7f32fc1d7000 1 -- 172.21.15.105:0/2107774703 wait complete. 2023-11-28T18:30:27.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.321+0000 7f32fc1d7000 20 client.8385 trim_cache size 0 max 16384 2023-11-28T18:30:27.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.333+0000 7f32fc1d7000 1 Processor -- start 2023-11-28T18:30:27.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32fc1d7000 1 -- start start 2023-11-28T18:30:27.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32fc1d7000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55589b65e500 0x55589b68a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32fc1d7000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b6aad30 0x55589b68a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32fc1d7000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55589b6ab650 0x55589b6bcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32fc1d7000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55589b169940 con 0x55589b6aad30 2023-11-28T18:30:27.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32fc1d7000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55589b485b70 con 0x55589b65e500 2023-11-28T18:30:27.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32fc1d7000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55589b1a91b0 con 0x55589b6ab650 2023-11-28T18:30:27.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32e9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b6aad30 0x55589b68a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32e9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b6aad30 0x55589b68a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55448/0 (socket says 172.21.15.105:55448) 2023-11-28T18:30:27.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32e9d9b640 1 -- 172.21.15.105:0/4287195476 learned_addr learned my addr 172.21.15.105:0/4287195476 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:27.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32e9d9b640 1 -- 172.21.15.105:0/4287195476 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55589b6ab650 msgr2=0x55589b6bcff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:27.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32e9d9b640 1 --2- 172.21.15.105:0/4287195476 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55589b6ab650 0x55589b6bcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32e9d9b640 1 -- 172.21.15.105:0/4287195476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55589b65e500 msgr2=0x55589b68a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:27.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32e9d9b640 1 --2- 172.21.15.105:0/4287195476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55589b65e500 0x55589b68a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32e9d9b640 1 -- 172.21.15.105:0/4287195476 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55589b65e370 con 0x55589b6aad30 2023-11-28T18:30:27.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32e9d9b640 1 --2- 172.21.15.105:0/4287195476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b6aad30 0x55589b68a8e0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x55589b44f6b0 tx=0x7f32dc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3368a7baea15e516 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:27.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32e8d99640 1 -- 172.21.15.105:0/4287195476 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc01c040 con 0x55589b6aad30 2023-11-28T18:30:27.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32e8d99640 1 -- 172.21.15.105:0/4287195476 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc01ed70 con 0x55589b6aad30 2023-11-28T18:30:27.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32e8d99640 1 -- 172.21.15.105:0/4287195476 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc003d40 con 0x55589b6aad30 2023-11-28T18:30:27.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32fc1d7000 1 -- 172.21.15.105:0/4287195476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b6aad30 msgr2=0x55589b68a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:27.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32fc1d7000 1 --2- 172.21.15.105:0/4287195476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b6aad30 0x55589b68a8e0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x55589b44f6b0 tx=0x7f32dc008040 comp rx=0 tx=0).stop 2023-11-28T18:30:27.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32fc1d7000 1 -- 172.21.15.105:0/4287195476 shutdown_connections 2023-11-28T18:30:27.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32fc1d7000 1 --2- 172.21.15.105:0/4287195476 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55589b6ab650 0x55589b6bcff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32fc1d7000 1 --2- 172.21.15.105:0/4287195476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b6aad30 0x55589b68a8e0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32fc1d7000 1 --2- 172.21.15.105:0/4287195476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55589b65e500 0x55589b68a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32fc1d7000 1 -- 172.21.15.105:0/4287195476 >> 172.21.15.105:0/4287195476 conn(0x55589b68b8c0 msgr2=0x55589b65c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:27.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32fc1d7000 1 -- 172.21.15.105:0/4287195476 shutdown_connections 2023-11-28T18:30:27.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32fc1d7000 1 -- 172.21.15.105:0/4287195476 wait complete. 2023-11-28T18:30:27.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32fc1d7000 1 -- start start 2023-11-28T18:30:27.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.337+0000 7f32fc1d7000 1 Processor -- start 2023-11-28T18:30:27.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55589b65e500 0x55589b70b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b6aad30 0x55589b70bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55589b6ab650 0x55589b6c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55589b4aac30 con 0x55589b6aad30 2023-11-28T18:30:27.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55589b46d9b0 con 0x55589b6ab650 2023-11-28T18:30:27.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55589b4ac5c0 con 0x55589b65e500 2023-11-28T18:30:27.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 10 client.0 _ll_register_callbacks cb 0x55589b527060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:30:27.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32e9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b6aad30 0x55589b70bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32e9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b6aad30 0x55589b70bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55460/0 (socket says 172.21.15.105:55460) 2023-11-28T18:30:27.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32e9d9b640 1 -- 172.21.15.105:0/1009734003 learned_addr learned my addr 172.21.15.105:0/1009734003 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:27.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32f259c640 1 --2- 172.21.15.105:0/1009734003 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55589b65e500 0x55589b70b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32f2d9d640 1 --2- 172.21.15.105:0/1009734003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55589b6ab650 0x55589b6c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32e9d9b640 1 -- 172.21.15.105:0/1009734003 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55589b65e500 msgr2=0x55589b70b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:27.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32e9d9b640 1 --2- 172.21.15.105:0/1009734003 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55589b65e500 0x55589b70b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32e9d9b640 1 -- 172.21.15.105:0/1009734003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55589b6ab650 msgr2=0x55589b6c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:27.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32e9d9b640 1 --2- 172.21.15.105:0/1009734003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55589b6ab650 0x55589b6c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32e9d9b640 1 -- 172.21.15.105:0/1009734003 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32dc009890 con 0x55589b6aad30 2023-11-28T18:30:27.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32f2d9d640 1 --2- 172.21.15.105:0/1009734003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55589b6ab650 0x55589b6c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:27.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32e9d9b640 1 --2- 172.21.15.105:0/1009734003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b6aad30 0x55589b70bd50 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f32dc004710 tx=0x7f32f406e370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5242c854f3d2c28c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:27.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:30:27.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32d17fa640 1 -- 172.21.15.105:0/1009734003 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc01e6d0 con 0x55589b6aad30 2023-11-28T18:30:27.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 1 -- 172.21.15.105:0/1009734003 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55589b4af110 con 0x55589b6aad30 2023-11-28T18:30:27.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32d17fa640 1 -- 172.21.15.105:0/1009734003 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f32dc01e830 con 0x55589b6aad30 2023-11-28T18:30:27.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32d17fa640 1 -- 172.21.15.105:0/1009734003 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32dc004590 con 0x55589b6aad30 2023-11-28T18:30:27.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 10 client.8391 fetch_fsmap learned FSMap version 108 2023-11-28T18:30:27.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 1 -- 172.21.15.105:0/1009734003 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x55589b457490 con 0x55589b6aad30 2023-11-28T18:30:27.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32d17fa640 1 -- 172.21.15.105:0/1009734003 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55589b4af110 con 0x55589b6aad30 2023-11-28T18:30:27.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32d17fa640 1 -- 172.21.15.105:0/1009734003 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 108) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f32dc01ea30 con 0x55589b6aad30 2023-11-28T18:30:27.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 10 client.8391 fetch_fsmap finished waiting for FSMap version 108 2023-11-28T18:30:27.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 10 client.8391 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:30:27.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 20 client.8391 populate_metadata read hostname 'smithi105' 2023-11-28T18:30:27.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 1 --2- 172.21.15.105:0/1009734003 >> [v2:172.21.15.105:6835/2782729992,v1:172.21.15.105:6837/2782729992] conn(0x55589b6c8180 0x55589b714680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:27.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 4 client.8391 mds_command: new command op to 8145 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:30:27.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 1 -- 172.21.15.105:0/1009734003 --> [v2:172.21.15.105:6835/2782729992,v1:172.21.15.105:6837/2782729992] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55589b4dd650 con 0x55589b6c8180 2023-11-28T18:30:27.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32f259c640 1 --2- 172.21.15.105:0/1009734003 >> [v2:172.21.15.105:6835/2782729992,v1:172.21.15.105:6837/2782729992] conn(0x55589b6c8180 0x55589b714680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:27.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32f259c640 1 --2- 172.21.15.105:0/1009734003 >> [v2:172.21.15.105:6835/2782729992,v1:172.21.15.105:6837/2782729992] conn(0x55589b6c8180 0x55589b714680 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16c2c0713cc60e4f server_cookie=edb2b4e0283aaeb4 in_seq=0 out_seq=0 2023-11-28T18:30:27.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32d17fa640 10 client.8391 ms_handle_connect on v2:172.21.15.105:6835/2782729992 2023-11-28T18:30:27.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32d17fa640 1 -- 172.21.15.105:0/1009734003 <== mds.0 v2:172.21.15.105:6835/2782729992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55589b4dd650 con 0x55589b6c8180 2023-11-28T18:30:27.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32d17fa640 10 client.8391 handle_command_reply: tid=0 2023-11-28T18:30:27.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 1 client.8391 shutdown 2023-11-28T18:30:27.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 1 -- 172.21.15.105:0/1009734003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b6aad30 msgr2=0x55589b70bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:27.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 1 --2- 172.21.15.105:0/1009734003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b6aad30 0x55589b70bd50 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f32dc004710 tx=0x7f32f406e370 comp rx=0 tx=0).stop 2023-11-28T18:30:27.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 1 -- 172.21.15.105:0/1009734003 shutdown_connections 2023-11-28T18:30:27.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 1 --2- 172.21.15.105:0/1009734003 >> [v2:172.21.15.105:6835/2782729992,v1:172.21.15.105:6837/2782729992] conn(0x55589b6c8180 0x55589b714680 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 1 --2- 172.21.15.105:0/1009734003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55589b6ab650 0x55589b6c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 1 --2- 172.21.15.105:0/1009734003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55589b6aad30 0x55589b70bd50 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 1 --2- 172.21.15.105:0/1009734003 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55589b65e500 0x55589b70b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32d17fa640 0 client.8391 ms_handle_reset on v2:172.21.15.105:6835/2782729992 2023-11-28T18:30:27.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 1 -- 172.21.15.105:0/1009734003 >> 172.21.15.105:0/1009734003 conn(0x55589b68b8c0 msgr2=0x55589b65cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:27.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 1 -- 172.21.15.105:0/1009734003 shutdown_connections 2023-11-28T18:30:27.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.341+0000 7f32fc1d7000 1 -- 172.21.15.105:0/1009734003 wait complete. 2023-11-28T18:30:27.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.345+0000 7f32fc1d7000 20 client.8391 trim_cache size 0 max 16384 2023-11-28T18:30:27.351 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:30:27.351 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:30:27.351 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2023-11-28T18:30:27.351 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 8145, 2023-11-28T18:30:27.351 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:30:27.351 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:30:27.352 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:30:27.352 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 21.188060755999999, 2023-11-28T18:30:27.352 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 108, 2023-11-28T18:30:27.352 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 105, 2023-11-28T18:30:27.352 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 105, 2023-11-28T18:30:27.352 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 26.522283256000001 2023-11-28T18:30:27.352 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:30:27.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.345+0000 7f32f8e6b640 1 -- 172.21.15.105:0/1621956949 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f32d4066900 msgr2=0x7f32d4068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:27.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.345+0000 7f32f8e6b640 1 --2- 172.21.15.105:0/1621956949 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f32d4066900 0x7f32d4068dc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f32e40ad000 tx=0x7f32e4001d90 comp rx=0 tx=0).stop 2023-11-28T18:30:27.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.345+0000 7f32f8e6b640 1 -- 172.21.15.105:0/1621956949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32f40c7f40 msgr2=0x7f32f4160ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:27.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.345+0000 7f32f8e6b640 1 --2- 172.21.15.105:0/1621956949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32f40c7f40 0x7f32f4160ca0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f32f409d0d0 tx=0x7f32ec011cf0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.345+0000 7f32f8e6b640 1 -- 172.21.15.105:0/1621956949 shutdown_connections 2023-11-28T18:30:27.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.345+0000 7f32f8e6b640 1 --2- 172.21.15.105:0/1621956949 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f32d4066900 0x7f32d4068dc0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.345+0000 7f32f8e6b640 1 --2- 172.21.15.105:0/1621956949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32f40c7f40 0x7f32f4160ca0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.345+0000 7f32f8e6b640 1 --2- 172.21.15.105:0/1621956949 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32f415a530 0x7f32f409bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.345+0000 7f32f8e6b640 1 --2- 172.21.15.105:0/1621956949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32e0003980 0x7f32f4150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:27.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.345+0000 7f32f8e6b640 1 -- 172.21.15.105:0/1621956949 >> 172.21.15.105:0/1621956949 conn(0x7f32f40ceb90 msgr2=0x7f32f4071410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:27.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.345+0000 7f32f8e6b640 1 -- 172.21.15.105:0/1621956949 shutdown_connections 2023-11-28T18:30:27.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:27.345+0000 7f32f8e6b640 1 -- 172.21.15.105:0/1621956949 wait complete. 2023-11-28T18:30:27.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:27.363 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-28T18:30:27.363 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:30:27.363 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p tree 2023-11-28T18:30:27.363 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:30:27.363 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:30:27.363 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:30:27.363 DEBUG:teuthology.orchestra.run.smithi105:> for ((i = 0; i < 10; i++)); do 2023-11-28T18:30:27.363 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p "tree/$i" 2023-11-28T18:30:27.364 DEBUG:teuthology.orchestra.run.smithi105:> echo file > "tree/$i/file" 2023-11-28T18:30:27.364 DEBUG:teuthology.orchestra.run.smithi105:> done 2023-11-28T18:30:27.364 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:30:27.375 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree 2023-11-28T18:30:27.379 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i = 0 )) 2023-11-28T18:30:27.379 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 10 )) 2023-11-28T18:30:27.379 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/0 2023-11-28T18:30:27.383 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:30:27.390 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:30:27.390 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 10 )) 2023-11-28T18:30:27.390 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/1 2023-11-28T18:30:27.398 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:30:27.401 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:30:27.402 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 10 )) 2023-11-28T18:30:27.402 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/2 2023-11-28T18:30:27.407 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:30:27.409 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:30:27.410 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 10 )) 2023-11-28T18:30:27.410 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/3 2023-11-28T18:30:27.416 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:30:27.418 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:30:27.418 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 10 )) 2023-11-28T18:30:27.418 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/4 2023-11-28T18:30:27.424 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:30:27.426 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:30:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 10 )) 2023-11-28T18:30:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/5 2023-11-28T18:30:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:30:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:30:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 10 )) 2023-11-28T18:30:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/6 2023-11-28T18:30:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:30:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:30:27.444 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 10 )) 2023-11-28T18:30:27.444 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/7 2023-11-28T18:30:27.449 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:30:27.453 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:30:27.454 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 10 )) 2023-11-28T18:30:27.454 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/8 2023-11-28T18:30:27.458 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:30:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:30:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 10 )) 2023-11-28T18:30:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/9 2023-11-28T18:30:27.466 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:30:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:30:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 10 )) 2023-11-28T18:30:27.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:27.469 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/0) 2023-11-28T18:30:27.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:27.527 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/1) 2023-11-28T18:30:27.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:27.582 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/2) 2023-11-28T18:30:27.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:27.638 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/3) 2023-11-28T18:30:27.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:27.694 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/4) 2023-11-28T18:30:27.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:27.750 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/5) 2023-11-28T18:30:27.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:27.802 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/6) 2023-11-28T18:30:27.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:27.854 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/7) 2023-11-28T18:30:27.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:27.906 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/8) 2023-11-28T18:30:27.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:27.959 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/9) 2023-11-28T18:30:28.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:28.013 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin.distributed -v 1 tree) 2023-11-28T18:30:32.578 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:30:32.582 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:30:43.066 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2023-11-28T18:30:43.235 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"4911","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":2},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000051b.010*","snapid_first":2,"projected_version":"64","version":"64","committing_version":"64","committed_version":"64","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"1":2}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000051b.000*","snapid_first":2,"projected_version":"68","version":"68","committing_version":"64","committed_version":"64","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"1":2}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/3","dirfrag":"0x10000000522","snapid_first":2,"projected_version":"0","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"subtree":1,"importbound":0,"sticky":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/8","dirfrag":"0x1000000052c","snapid_first":2,"projected_version":"0","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"subtree":1,"importbound":0,"sticky":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/9","dirfrag":"0x1000000052e","snapid_first":2,"projected_version":"0","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"subtree":1,"importbound":0,"sticky":0},"nref":1}}] 2023-11-28T18:30:43.236 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "snapid_first": 2, "states": [], "version": "4911" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": { "0": 1, "1": 2 } }, "committed_version": "64", "committing_version": "64", "dir_auth": "2", "dirfrag": "0x1000000051b.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "request": 1, "subtree": 1 }, "projected_version": "64", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "64" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "1": 2 } }, "committed_version": "64", "committing_version": "64", "dir_auth": "2", "dirfrag": "0x1000000051b.000*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "68", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "68" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000522", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/3", "pins": { "importbound": 0, "sticky": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000052c", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/8", "pins": { "importbound": 0, "sticky": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000052e", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/9", "pins": { "importbound": 0, "sticky": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2023-11-28T18:30:43.237 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-11-28T18:30:43.405 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"4887","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/1","dirfrag":"0x1000000051e","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/8","dirfrag":"0x1000000052c","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/7","dirfrag":"0x1000000052a","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"importbound":0,"sticky":0,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/6","dirfrag":"0x10000000528","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/0","dirfrag":"0x1000000051c","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"importbound":0,"sticky":0,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/5","dirfrag":"0x10000000526","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"importbound":0,"sticky":0,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/4","dirfrag":"0x10000000524","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/3","dirfrag":"0x10000000522","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/2","dirfrag":"0x10000000520","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000051b.000*","snapid_first":2,"projected_version":"61","version":"64","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":2},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000051b.010*","snapid_first":2,"projected_version":"61","version":"64","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":2},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"subtree":1},"nref":1}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000051b.100*","snapid_first":2,"projected_version":"66","version":"66","committing_version":"64","committed_version":"64","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000051b.110*","snapid_first":2,"projected_version":"68","version":"68","committing_version":"64","committed_version":"64","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/9","dirfrag":"0x1000000052e","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-11-28T18:30:43.407 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "4887" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000051e", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/1", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000052c", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/8", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000052a", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree/7", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importbound": 0, "importing": 0, "ptrwaiter": 0, "replicated": 0, "sticky": 0, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000528", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/6", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000051c", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree/0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importbound": 0, "importing": 0, "ptrwaiter": 0, "replicated": 0, "sticky": 0, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000526", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree/5", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importbound": 0, "importing": 0, "ptrwaiter": 0, "replicated": 0, "sticky": 0, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000524", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/4", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000522", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/3", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000520", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000051b.000*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/tree", "pins": { "child": 1, "subtree": 1 }, "projected_version": "61", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 2 }, "snapid_first": 2, "states": [], "version": "64" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000051b.010*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "subtree": 1 }, "projected_version": "61", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 2 }, "snapid_first": 2, "states": [], "version": "64" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "64", "committing_version": "64", "dir_auth": "1", "dirfrag": "0x1000000051b.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "66", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "66" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "64", "committing_version": "64", "dir_auth": "1", "dirfrag": "0x1000000051b.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "68", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "68" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000052e", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/9", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:30:43.408 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-11-28T18:30:43.564 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"4911","version":"4911","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1,"2":2}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1621","version":"1621","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x1000000051b.101*","snapid_first":2,"projected_version":"64","version":"64","committing_version":"64","committed_version":"64","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{"1":2}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"frozen":0,"subtree":1,"replicated":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000051b.010*","snapid_first":2,"projected_version":"64","version":"64","committing_version":"64","committed_version":"64","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000051b.100*","snapid_first":2,"projected_version":"66","version":"66","committing_version":"64","committed_version":"64","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x1000000051b.001*","snapid_first":2,"projected_version":"64","version":"64","committing_version":"64","committed_version":"64","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{"1":2}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"frozen":0,"subtree":1,"replicated":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000051b.110*","snapid_first":2,"projected_version":"68","version":"68","committing_version":"64","committed_version":"64","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x1000000051b.011*","snapid_first":2,"projected_version":"70","version":"70","committing_version":"64","committed_version":"64","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":2}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000051b.000*","snapid_first":2,"projected_version":"68","version":"68","committing_version":"64","committed_version":"64","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x1000000051b.111*","snapid_first":2,"projected_version":"66","version":"66","committing_version":"64","committed_version":"64","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":2}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/1","dirfrag":"0x1000000051e","snapid_first":2,"projected_version":"3","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/2","dirfrag":"0x10000000520","snapid_first":2,"projected_version":"3","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/4","dirfrag":"0x10000000524","snapid_first":2,"projected_version":"3","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/6","dirfrag":"0x10000000528","snapid_first":2,"projected_version":"3","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2023-11-28T18:30:43.565 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1, "2": 2 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "4911", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "4911" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "1621", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1621" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 2 } }, "committed_version": "64", "committing_version": "64", "dir_auth": "0", "dirfrag": "0x1000000051b.101*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/tree", "pins": { "authpin": 0, "dirty": 0, "frozen": 0, "replicated": 1, "subtree": 1 }, "projected_version": "64", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "64" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "64", "committing_version": "64", "dir_auth": "2", "dirfrag": "0x1000000051b.010*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "64", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "64" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "64", "committing_version": "64", "dir_auth": "1", "dirfrag": "0x1000000051b.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "66", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "66" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 2 } }, "committed_version": "64", "committing_version": "64", "dir_auth": "0", "dirfrag": "0x1000000051b.001*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/tree", "pins": { "authpin": 0, "dirty": 0, "frozen": 0, "replicated": 1, "subtree": 1 }, "projected_version": "64", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "64" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "64", "committing_version": "64", "dir_auth": "1", "dirfrag": "0x1000000051b.110*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "68", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "68" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 2 } }, "committed_version": "64", "committing_version": "64", "dir_auth": "0", "dirfrag": "0x1000000051b.011*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "replicated": 1, "subtree": 1 }, "projected_version": "70", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "70" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "64", "committing_version": "64", "dir_auth": "2", "dirfrag": "0x1000000051b.000*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "68", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "68" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 2 } }, "committed_version": "64", "committing_version": "64", "dir_auth": "0", "dirfrag": "0x1000000051b.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "replicated": 1, "subtree": 1 }, "projected_version": "66", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "66" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000051e", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/1", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "3", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000520", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/2", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "3", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000524", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/4", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "3", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000528", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "3", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2023-11-28T18:30:43.565 INFO:tasks.cephfs.cephfs_test_case:[('/tree/0', 1), ('/tree/1', 1), ('/tree/2', 1), ('/tree/3', 1), ('/tree/4', 1), ('/tree/5', 1), ('/tree/6', 1), ('/tree/7', 1), ('/tree/8', 1), ('/tree/9', 1)] =?= [('/tree/0', 1), ('/tree/1', 1), ('/tree/2', 1), ('/tree/3', 1), ('/tree/4', 1), ('/tree/5', 1), ('/tree/6', 1), ('/tree/7', 1), ('/tree/8', 1), ('/tree/9', 1)] 2023-11-28T18:30:43.565 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T18:30:43.566 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:30:43.657 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:30:43.738 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T18:30:43.739 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:30:43.739 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:30:43.748 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:30:43.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.933+0000 7f5f8e83a640 1 Processor -- start 2023-11-28T18:30:43.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.933+0000 7f5f8e83a640 1 -- start start 2023-11-28T18:30:43.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.933+0000 7f5f8e83a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5f8815d2d0 0x7f5f880cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:43.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.933+0000 7f5f8e83a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5f880cf960 0x7f5f88166fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:43.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.933+0000 7f5f8e83a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f880cfdb0 con 0x7f5f8815cef0 2023-11-28T18:30:43.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.933+0000 7f5f8e83a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f880b4780 con 0x7f5f880cf960 2023-11-28T18:30:43.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.933+0000 7f5f8e83a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f880ce840 con 0x7f5f8815d2d0 2023-11-28T18:30:43.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.933+0000 7f5f8e039640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5f880cf960 0x7f5f88166fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:43.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.933+0000 7f5f8e039640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5f880cf960 0x7f5f88166fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35062/0 (socket says 172.21.15.105:35062) 2023-11-28T18:30:43.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.933+0000 7f5f8e039640 1 -- 172.21.15.105:0/1266559533 learned_addr learned my addr 172.21.15.105:0/1266559533 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:43.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.933+0000 7f5f8d037640 1 --2- 172.21.15.105:0/1266559533 >> v2:172.21.15.148:3300/0 conn(0x7f5f8815d2d0 0x7f5f880cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:43.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.933+0000 7f5f8e039640 1 -- 172.21.15.105:0/1266559533 >> v2:172.21.15.148:3300/0 conn(0x7f5f8815d2d0 msgr2=0x7f5f880cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:43.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.933+0000 7f5f8e039640 1 --2- 172.21.15.105:0/1266559533 >> v2:172.21.15.148:3300/0 conn(0x7f5f8815d2d0 0x7f5f880cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:43.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.933+0000 7f5f8e039640 1 -- 172.21.15.105:0/1266559533 >> v1:172.21.15.148:6789/0 conn(0x7f5f8815cef0 legacy=0x7f5f880d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:43.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.933+0000 7f5f8e039640 1 -- 172.21.15.105:0/1266559533 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f880cd990 con 0x7f5f880cf960 2023-11-28T18:30:43.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.937+0000 7f5f8e039640 1 --2- 172.21.15.105:0/1266559533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f880cf960 0x7f5f88166fd0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f5f8808bac0 tx=0x7f5f8001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692b3e24874e582c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:43.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.937+0000 7f5f8c836640 1 -- 172.21.15.105:0/1266559533 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f800a57e0 con 0x7f5f880cf960 2023-11-28T18:30:43.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.937+0000 7f5f8c836640 1 -- 172.21.15.105:0/1266559533 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f800a5940 con 0x7f5f880cf960 2023-11-28T18:30:43.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.937+0000 7f5f8c836640 1 -- 172.21.15.105:0/1266559533 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f800a5cd0 con 0x7f5f880cf960 2023-11-28T18:30:43.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.937+0000 7f5f8e83a640 1 -- 172.21.15.105:0/1266559533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f880cf960 msgr2=0x7f5f88166fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:43.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.937+0000 7f5f8e83a640 1 --2- 172.21.15.105:0/1266559533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f880cf960 0x7f5f88166fd0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f5f8808bac0 tx=0x7f5f8001c920 comp rx=0 tx=0).stop 2023-11-28T18:30:43.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.937+0000 7f5f8e83a640 1 -- 172.21.15.105:0/1266559533 shutdown_connections 2023-11-28T18:30:43.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.937+0000 7f5f8e83a640 1 --2- 172.21.15.105:0/1266559533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f880cf960 0x7f5f88166fd0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:43.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.937+0000 7f5f8e83a640 1 --2- 172.21.15.105:0/1266559533 >> v2:172.21.15.148:3300/0 conn(0x7f5f8815d2d0 0x7f5f880cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:43.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.937+0000 7f5f8e83a640 1 -- 172.21.15.105:0/1266559533 >> 172.21.15.105:0/1266559533 conn(0x7f5f880d2e10 msgr2=0x7f5f880d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:43.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.937+0000 7f5f8e83a640 1 -- 172.21.15.105:0/1266559533 shutdown_connections 2023-11-28T18:30:43.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.937+0000 7f5f8e83a640 1 -- 172.21.15.105:0/1266559533 wait complete. 2023-11-28T18:30:43.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.937+0000 7f5f8e83a640 1 Processor -- start 2023-11-28T18:30:43.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f8e83a640 1 -- start start 2023-11-28T18:30:43.941 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f8e83a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f880cf960 0x7f5f88153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:43.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f8e83a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f8815cef0 0x7f5f880b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:43.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f8e83a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f8815d2d0 0x7f5f88154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:43.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f8e83a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f88160320 con 0x7f5f8815cef0 2023-11-28T18:30:43.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f8e83a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f880b0af0 con 0x7f5f8815d2d0 2023-11-28T18:30:43.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f8e83a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f880b0dd0 con 0x7f5f880cf960 2023-11-28T18:30:43.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f8d037640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f8815cef0 0x7f5f880b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:43.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f8d037640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f8815cef0 0x7f5f880b12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35074/0 (socket says 172.21.15.105:35074) 2023-11-28T18:30:43.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f8d037640 1 -- 172.21.15.105:0/1455083360 learned_addr learned my addr 172.21.15.105:0/1455083360 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:43.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f8e039640 1 --2- 172.21.15.105:0/1455083360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f8815d2d0 0x7f5f88154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:43.942 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f8d838640 1 --2- 172.21.15.105:0/1455083360 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f880cf960 0x7f5f88153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:43.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f8d037640 1 -- 172.21.15.105:0/1455083360 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f880cf960 msgr2=0x7f5f88153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:43.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f8d037640 1 --2- 172.21.15.105:0/1455083360 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f880cf960 0x7f5f88153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:43.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f8d037640 1 -- 172.21.15.105:0/1455083360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f8815d2d0 msgr2=0x7f5f88154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:43.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f8d037640 1 --2- 172.21.15.105:0/1455083360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f8815d2d0 0x7f5f88154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:43.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f8d037640 1 -- 172.21.15.105:0/1455083360 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f80009b30 con 0x7f5f8815cef0 2023-11-28T18:30:43.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f8d037640 1 --2- 172.21.15.105:0/1455083360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f8815cef0 0x7f5f880b12f0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800e9b0 tx=0x7f5f7800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:43.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f7e7fc640 1 -- 172.21.15.105:0/1455083360 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f780096b0 con 0x7f5f8815cef0 2023-11-28T18:30:43.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f7e7fc640 1 -- 172.21.15.105:0/1455083360 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7800c9a0 con 0x7f5f8815cef0 2023-11-28T18:30:43.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f8e83a640 1 -- 172.21.15.105:0/1455083360 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f88160600 con 0x7f5f8815cef0 2023-11-28T18:30:43.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f7e7fc640 1 -- 172.21.15.105:0/1455083360 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7800cd30 con 0x7f5f8815cef0 2023-11-28T18:30:43.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f8e83a640 1 -- 172.21.15.105:0/1455083360 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f880b1830 con 0x7f5f8815cef0 2023-11-28T18:30:43.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.941+0000 7f5f8e83a640 1 -- 172.21.15.105:0/1455083360 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f880b0fa0 con 0x7f5f8815cef0 2023-11-28T18:30:43.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.945+0000 7f5f7e7fc640 1 -- 172.21.15.105:0/1455083360 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5f78013470 con 0x7f5f8815cef0 2023-11-28T18:30:43.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.945+0000 7f5f7e7fc640 1 --2- 172.21.15.105:0/1455083360 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f6c0668d0 0x7f5f6c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:43.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.945+0000 7f5f7e7fc640 1 -- 172.21.15.105:0/1455083360 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5f780847c0 con 0x7f5f8815cef0 2023-11-28T18:30:43.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.945+0000 7f5f8d838640 1 --2- 172.21.15.105:0/1455083360 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f6c0668d0 0x7f5f6c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:43.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.945+0000 7f5f8d838640 1 --2- 172.21.15.105:0/1455083360 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f6c0668d0 0x7f5f6c068d90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5f88154c60 tx=0x7f5f84009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:43.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:43.949+0000 7f5f7e7fc640 1 -- 172.21.15.105:0/1455083360 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5f78051940 con 0x7f5f8815cef0 2023-11-28T18:30:44.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.097+0000 7f5f8e83a640 1 -- 172.21.15.105:0/1455083360 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f88160460 con 0x7f5f8815cef0 2023-11-28T18:30:44.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.097+0000 7f5f7e7fc640 1 -- 172.21.15.105:0/1455083360 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16826 (secure 0 0 0) 0x7f5f88160460 con 0x7f5f8815cef0 2023-11-28T18:30:44.099 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:44.099 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":105,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:30:16.894705+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:30:03.341424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-28T18:30:04.527931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1b","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:30:44.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.101+0000 7f5f8e83a640 1 -- 172.21.15.105:0/1455083360 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f6c0668d0 msgr2=0x7f5f6c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:44.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.101+0000 7f5f8e83a640 1 --2- 172.21.15.105:0/1455083360 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f6c0668d0 0x7f5f6c068d90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5f88154c60 tx=0x7f5f84009040 comp rx=0 tx=0).stop 2023-11-28T18:30:44.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.101+0000 7f5f8e83a640 1 -- 172.21.15.105:0/1455083360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f8815cef0 msgr2=0x7f5f880b12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:44.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.101+0000 7f5f8e83a640 1 --2- 172.21.15.105:0/1455083360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f8815cef0 0x7f5f880b12f0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800e9b0 tx=0x7f5f7800ee70 comp rx=0 tx=0).stop 2023-11-28T18:30:44.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.101+0000 7f5f8e83a640 1 -- 172.21.15.105:0/1455083360 shutdown_connections 2023-11-28T18:30:44.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.101+0000 7f5f8e83a640 1 --2- 172.21.15.105:0/1455083360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f8815d2d0 0x7f5f88154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.101+0000 7f5f8e83a640 1 --2- 172.21.15.105:0/1455083360 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5f6c0668d0 0x7f5f6c068d90 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.101+0000 7f5f8e83a640 1 --2- 172.21.15.105:0/1455083360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f8815cef0 0x7f5f880b12f0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.101+0000 7f5f8e83a640 1 --2- 172.21.15.105:0/1455083360 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f880cf960 0x7f5f88153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.101+0000 7f5f8e83a640 1 -- 172.21.15.105:0/1455083360 >> 172.21.15.105:0/1455083360 conn(0x7f5f880d2e10 msgr2=0x7f5f880c9690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:44.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.101+0000 7f5f8e83a640 1 -- 172.21.15.105:0/1455083360 shutdown_connections 2023-11-28T18:30:44.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.101+0000 7f5f8e83a640 1 -- 172.21.15.105:0/1455083360 wait complete. 2023-11-28T18:30:44.120 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:30:03.341424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:30:04.527931+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:30:44.121 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:30:44.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.265+0000 7fea75b96640 1 Processor -- start 2023-11-28T18:30:44.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.265+0000 7fea75b96640 1 -- start start 2023-11-28T18:30:44.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.265+0000 7fea75b96640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fea700d3c30 0x7fea700d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:44.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.265+0000 7fea75b96640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fea700b2d20 0x7fea700b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:44.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.265+0000 7fea75b96640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea700b25b0 con 0x7fea7015d4f0 2023-11-28T18:30:44.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.265+0000 7fea75b96640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea700b2900 con 0x7fea700d3c30 2023-11-28T18:30:44.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.265+0000 7fea75b96640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea700d0a20 con 0x7fea700b2d20 2023-11-28T18:30:44.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.265+0000 7fea6f7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fea7015d4f0 0x7fea700d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33656/0 (socket says 172.21.15.105:33656) 2023-11-28T18:30:44.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.265+0000 7fea6f7fe640 1 -- 172.21.15.105:0/801801964 learned_addr learned my addr 172.21.15.105:0/801801964 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:44.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.265+0000 7fea67fff640 1 --2- 172.21.15.105:0/801801964 >> v2:172.21.15.105:3300/0 conn(0x7fea700d3c30 0x7fea700d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:44.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.265+0000 7fea6ffff640 1 --2- 172.21.15.105:0/801801964 >> v2:172.21.15.148:3300/0 conn(0x7fea700b2d20 0x7fea700b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:44.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.265+0000 7fea6effd640 1 -- 172.21.15.105:0/801801964 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 781311635 0 0) 0x7fea700b25b0 con 0x7fea7015d4f0 2023-11-28T18:30:44.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.265+0000 7fea6effd640 1 -- 172.21.15.105:0/801801964 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea5c003530 con 0x7fea7015d4f0 2023-11-28T18:30:44.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.265+0000 7fea67fff640 1 -- 172.21.15.105:0/801801964 >> v2:172.21.15.148:3300/0 conn(0x7fea700b2d20 msgr2=0x7fea700b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:44.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.265+0000 7fea67fff640 1 --2- 172.21.15.105:0/801801964 >> v2:172.21.15.148:3300/0 conn(0x7fea700b2d20 0x7fea700b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.265+0000 7fea67fff640 1 -- 172.21.15.105:0/801801964 >> v1:172.21.15.105:6789/0 conn(0x7fea7015d4f0 legacy=0x7fea700d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:44.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.265+0000 7fea67fff640 1 -- 172.21.15.105:0/801801964 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea700cea30 con 0x7fea700d3c30 2023-11-28T18:30:44.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.269+0000 7fea67fff640 1 --2- 172.21.15.105:0/801801964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea700d3c30 0x7fea700d1520 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fea70160fa0 tx=0x7fea5801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a87add4d92cfde8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:44.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.269+0000 7fea6effd640 1 -- 172.21.15.105:0/801801964 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea580ad680 con 0x7fea700d3c30 2023-11-28T18:30:44.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.269+0000 7fea6effd640 1 -- 172.21.15.105:0/801801964 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea580ad7e0 con 0x7fea700d3c30 2023-11-28T18:30:44.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.269+0000 7fea6effd640 1 -- 172.21.15.105:0/801801964 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea580adb70 con 0x7fea700d3c30 2023-11-28T18:30:44.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.269+0000 7fea75b96640 1 -- 172.21.15.105:0/801801964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea700d3c30 msgr2=0x7fea700d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:44.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.269+0000 7fea75b96640 1 --2- 172.21.15.105:0/801801964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea700d3c30 0x7fea700d1520 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fea70160fa0 tx=0x7fea5801c380 comp rx=0 tx=0).stop 2023-11-28T18:30:44.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.269+0000 7fea75b96640 1 -- 172.21.15.105:0/801801964 shutdown_connections 2023-11-28T18:30:44.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.269+0000 7fea75b96640 1 --2- 172.21.15.105:0/801801964 >> v2:172.21.15.148:3300/0 conn(0x7fea700b2d20 0x7fea700b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.269+0000 7fea75b96640 1 --2- 172.21.15.105:0/801801964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea700d3c30 0x7fea700d1520 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.269+0000 7fea75b96640 1 -- 172.21.15.105:0/801801964 >> 172.21.15.105:0/801801964 conn(0x7fea700d3030 msgr2=0x7fea700d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:44.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.269+0000 7fea75b96640 1 -- 172.21.15.105:0/801801964 shutdown_connections 2023-11-28T18:30:44.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.269+0000 7fea75b96640 1 -- 172.21.15.105:0/801801964 wait complete. 2023-11-28T18:30:44.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.269+0000 7fea75b96640 1 Processor -- start 2023-11-28T18:30:44.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea75b96640 1 -- start start 2023-11-28T18:30:44.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea75b96640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea700b2d20 0x7fea7015c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:44.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea75b96640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea700d3c30 0x7fea700cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:44.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea75b96640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea7015d4f0 0x7fea700c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:44.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea75b96640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea700caad0 con 0x7fea700d3c30 2023-11-28T18:30:44.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea75b96640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea70159ee0 con 0x7fea700b2d20 2023-11-28T18:30:44.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea75b96640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea7015c630 con 0x7fea7015d4f0 2023-11-28T18:30:44.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea67fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea700d3c30 0x7fea700cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:44.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea67fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea700d3c30 0x7fea700cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35106/0 (socket says 172.21.15.105:35106) 2023-11-28T18:30:44.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea67fff640 1 -- 172.21.15.105:0/1693886171 learned_addr learned my addr 172.21.15.105:0/1693886171 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:44.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea6f7fe640 1 --2- 172.21.15.105:0/1693886171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea700b2d20 0x7fea7015c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:44.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea6ffff640 1 --2- 172.21.15.105:0/1693886171 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea7015d4f0 0x7fea700c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:44.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea67fff640 1 -- 172.21.15.105:0/1693886171 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea7015d4f0 msgr2=0x7fea700c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:44.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea67fff640 1 --2- 172.21.15.105:0/1693886171 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea7015d4f0 0x7fea700c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea67fff640 1 -- 172.21.15.105:0/1693886171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea700b2d20 msgr2=0x7fea7015c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:44.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea67fff640 1 --2- 172.21.15.105:0/1693886171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea700b2d20 0x7fea7015c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea67fff640 1 -- 172.21.15.105:0/1693886171 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea58009420 con 0x7fea700d3c30 2023-11-28T18:30:44.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea67fff640 1 --2- 172.21.15.105:0/1693886171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea700d3c30 0x7fea700cbd80 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fea58018f90 tx=0x7fea5801abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:44.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea6cff9640 1 -- 172.21.15.105:0/1693886171 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea580add60 con 0x7fea700d3c30 2023-11-28T18:30:44.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea6cff9640 1 -- 172.21.15.105:0/1693886171 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea580b6460 con 0x7fea700d3c30 2023-11-28T18:30:44.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea6cff9640 1 -- 172.21.15.105:0/1693886171 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea580b6810 con 0x7fea700d3c30 2023-11-28T18:30:44.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea75b96640 1 -- 172.21.15.105:0/1693886171 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea700cb300 con 0x7fea700d3c30 2023-11-28T18:30:44.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea75b96640 1 -- 172.21.15.105:0/1693886171 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea700cacb0 con 0x7fea700d3c30 2023-11-28T18:30:44.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.273+0000 7fea75b96640 1 -- 172.21.15.105:0/1693886171 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea700cb490 con 0x7fea700d3c30 2023-11-28T18:30:44.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.277+0000 7fea6cff9640 1 -- 172.21.15.105:0/1693886171 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fea580b4070 con 0x7fea700d3c30 2023-11-28T18:30:44.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.277+0000 7fea6cff9640 1 --2- 172.21.15.105:0/1693886171 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fea500668d0 0x7fea50068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:44.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.277+0000 7fea6cff9640 1 -- 172.21.15.105:0/1693886171 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fea5811ded0 con 0x7fea700d3c30 2023-11-28T18:30:44.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.277+0000 7fea6f7fe640 1 --2- 172.21.15.105:0/1693886171 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fea500668d0 0x7fea50068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:44.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.277+0000 7fea6f7fe640 1 --2- 172.21.15.105:0/1693886171 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fea500668d0 0x7fea50068d90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fea600011f0 tx=0x7fea60007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:44.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.281+0000 7fea6cff9640 1 -- 172.21.15.105:0/1693886171 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fea580eafc0 con 0x7fea700d3c30 2023-11-28T18:30:44.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.433+0000 7fea75b96640 1 -- 172.21.15.105:0/1693886171 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fea70067570 con 0x7fea700d3c30 2023-11-28T18:30:44.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.433+0000 7fea6cff9640 1 -- 172.21.15.105:0/1693886171 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7fea580b2020 con 0x7fea700d3c30 2023-11-28T18:30:44.434 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:30:44.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.437+0000 7fea75b96640 1 -- 172.21.15.105:0/1693886171 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fea500668d0 msgr2=0x7fea50068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:44.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.437+0000 7fea75b96640 1 --2- 172.21.15.105:0/1693886171 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fea500668d0 0x7fea50068d90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fea600011f0 tx=0x7fea60007d40 comp rx=0 tx=0).stop 2023-11-28T18:30:44.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.437+0000 7fea75b96640 1 -- 172.21.15.105:0/1693886171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea700d3c30 msgr2=0x7fea700cbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:44.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.437+0000 7fea75b96640 1 --2- 172.21.15.105:0/1693886171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea700d3c30 0x7fea700cbd80 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fea58018f90 tx=0x7fea5801abc0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.437+0000 7fea75b96640 1 -- 172.21.15.105:0/1693886171 shutdown_connections 2023-11-28T18:30:44.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.437+0000 7fea75b96640 1 --2- 172.21.15.105:0/1693886171 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea7015d4f0 0x7fea700c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.437+0000 7fea75b96640 1 --2- 172.21.15.105:0/1693886171 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fea500668d0 0x7fea50068d90 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.437+0000 7fea75b96640 1 --2- 172.21.15.105:0/1693886171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea700d3c30 0x7fea700cbd80 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.437+0000 7fea75b96640 1 --2- 172.21.15.105:0/1693886171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea700b2d20 0x7fea7015c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.437+0000 7fea75b96640 1 -- 172.21.15.105:0/1693886171 >> 172.21.15.105:0/1693886171 conn(0x7fea700d3030 msgr2=0x7fea700afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:44.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.437+0000 7fea75b96640 1 -- 172.21.15.105:0/1693886171 shutdown_connections 2023-11-28T18:30:44.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.437+0000 7fea75b96640 1 -- 172.21.15.105:0/1693886171 wait complete. 2023-11-28T18:30:44.455 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:30:44.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.593+0000 7f43ed980640 1 Processor -- start 2023-11-28T18:30:44.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.593+0000 7f43ed980640 1 -- start start 2023-11-28T18:30:44.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.597+0000 7f43ed980640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43e815d4f0 0x7f43e80d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:44.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.597+0000 7f43ed980640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f43e80d3c30 0x7f43e80d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:44.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.597+0000 7f43ed980640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e80b25b0 con 0x7f43e80d1890 2023-11-28T18:30:44.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.597+0000 7f43ed980640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e80b2900 con 0x7f43e80d3c30 2023-11-28T18:30:44.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.597+0000 7f43ed980640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e80cea30 con 0x7f43e815d4f0 2023-11-28T18:30:44.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.597+0000 7f43dffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f43e80d3c30 0x7f43e80d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:44.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.597+0000 7f43e77fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f43e80d1890 0x7f43e80cfa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33662/0 (socket says 172.21.15.105:33662) 2023-11-28T18:30:44.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.597+0000 7f43e77fe640 1 -- 172.21.15.105:0/1312891368 learned_addr learned my addr 172.21.15.105:0/1312891368 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:44.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.597+0000 7f43e6ffd640 1 --2- 172.21.15.105:0/1312891368 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43e815d4f0 0x7f43e80d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:44.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.597+0000 7f43dffff640 1 -- 172.21.15.105:0/1312891368 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43e815d4f0 msgr2=0x7f43e80d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:44.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.597+0000 7f43dffff640 1 --2- 172.21.15.105:0/1312891368 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43e815d4f0 0x7f43e80d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.597+0000 7f43dffff640 1 -- 172.21.15.105:0/1312891368 >> v1:172.21.15.105:6789/0 conn(0x7f43e80d1890 legacy=0x7f43e80cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:44.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.597+0000 7f43dffff640 1 -- 172.21.15.105:0/1312891368 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e80ced10 con 0x7f43e80d3c30 2023-11-28T18:30:44.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.597+0000 7f43e67fc640 1 -- 172.21.15.105:0/1312891368 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 77286473 0 0) 0x7f43e80b25b0 con 0x7f43e80d1890 2023-11-28T18:30:44.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.597+0000 7f43e6ffd640 1 --2- 172.21.15.105:0/1312891368 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43e815d4f0 0x7f43e80d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:44.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.601+0000 7f43dffff640 1 --2- 172.21.15.105:0/1312891368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43e80d3c30 0x7f43e80d1350 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f43d00097c0 tx=0x7f43d001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6377292eba866136 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:44.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.601+0000 7f43e67fc640 1 -- 172.21.15.105:0/1312891368 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d00ac640 con 0x7f43e80d3c30 2023-11-28T18:30:44.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.601+0000 7f43e67fc640 1 -- 172.21.15.105:0/1312891368 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d00ac7a0 con 0x7f43e80d3c30 2023-11-28T18:30:44.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.601+0000 7f43e67fc640 1 -- 172.21.15.105:0/1312891368 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d00acb30 con 0x7f43e80d3c30 2023-11-28T18:30:44.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.601+0000 7f43ed980640 1 -- 172.21.15.105:0/1312891368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43e80d3c30 msgr2=0x7f43e80d1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:44.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.601+0000 7f43ed980640 1 --2- 172.21.15.105:0/1312891368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43e80d3c30 0x7f43e80d1350 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f43d00097c0 tx=0x7f43d001c3c0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.601+0000 7f43ed980640 1 -- 172.21.15.105:0/1312891368 shutdown_connections 2023-11-28T18:30:44.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.601+0000 7f43ed980640 1 --2- 172.21.15.105:0/1312891368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43e80d3c30 0x7f43e80d1350 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.601+0000 7f43ed980640 1 --2- 172.21.15.105:0/1312891368 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43e815d4f0 0x7f43e80d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.601+0000 7f43ed980640 1 -- 172.21.15.105:0/1312891368 >> 172.21.15.105:0/1312891368 conn(0x7f43e80d3030 msgr2=0x7f43e80d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:44.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.601+0000 7f43ed980640 1 -- 172.21.15.105:0/1312891368 shutdown_connections 2023-11-28T18:30:44.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.601+0000 7f43ed980640 1 -- 172.21.15.105:0/1312891368 wait complete. 2023-11-28T18:30:44.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.601+0000 7f43ed980640 1 Processor -- start 2023-11-28T18:30:44.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.601+0000 7f43ed980640 1 -- start start 2023-11-28T18:30:44.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43ed980640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43e80d1890 0x7f43e81608c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:44.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43ed980640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43e80d3c30 0x7f43e814c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:44.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43ed980640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43e815d4f0 0x7f43e815b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:44.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43ed980640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e80b5f20 con 0x7f43e815d4f0 2023-11-28T18:30:44.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43ed980640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e80b6200 con 0x7f43e80d3c30 2023-11-28T18:30:44.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43ed980640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e80b64e0 con 0x7f43e80d1890 2023-11-28T18:30:44.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43e77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43e815d4f0 0x7f43e815b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:44.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43e77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43e815d4f0 0x7f43e815b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35120/0 (socket says 172.21.15.105:35120) 2023-11-28T18:30:44.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43e77fe640 1 -- 172.21.15.105:0/3054307951 learned_addr learned my addr 172.21.15.105:0/3054307951 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:44.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43dffff640 1 --2- 172.21.15.105:0/3054307951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43e80d3c30 0x7f43e814c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:44.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43e6ffd640 1 --2- 172.21.15.105:0/3054307951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43e80d1890 0x7f43e81608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:44.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43e77fe640 1 -- 172.21.15.105:0/3054307951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43e80d1890 msgr2=0x7f43e81608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:44.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43e77fe640 1 --2- 172.21.15.105:0/3054307951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43e80d1890 0x7f43e81608c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43e77fe640 1 -- 172.21.15.105:0/3054307951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43e80d3c30 msgr2=0x7f43e814c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:44.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43e77fe640 1 --2- 172.21.15.105:0/3054307951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43e80d3c30 0x7f43e814c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43e77fe640 1 -- 172.21.15.105:0/3054307951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d0009420 con 0x7f43e815d4f0 2023-11-28T18:30:44.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43e6ffd640 1 --2- 172.21.15.105:0/3054307951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43e80d1890 0x7f43e81608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:44.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43e77fe640 1 --2- 172.21.15.105:0/3054307951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43e815d4f0 0x7f43e815b260 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f43e000c780 tx=0x7f43e000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:44.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43e4ff9640 1 -- 172.21.15.105:0/3054307951 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e000ce90 con 0x7f43e815d4f0 2023-11-28T18:30:44.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43e4ff9640 1 -- 172.21.15.105:0/3054307951 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e000a640 con 0x7f43e815d4f0 2023-11-28T18:30:44.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43ed980640 1 -- 172.21.15.105:0/3054307951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e80b67c0 con 0x7f43e815d4f0 2023-11-28T18:30:44.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43e4ff9640 1 -- 172.21.15.105:0/3054307951 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e000a9d0 con 0x7f43e815d4f0 2023-11-28T18:30:44.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43ed980640 1 -- 172.21.15.105:0/3054307951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e80b5bb0 con 0x7f43e815d4f0 2023-11-28T18:30:44.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43ed980640 1 -- 172.21.15.105:0/3054307951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e8067570 con 0x7f43e815d4f0 2023-11-28T18:30:44.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.605+0000 7f43e4ff9640 1 -- 172.21.15.105:0/3054307951 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f43e000ab30 con 0x7f43e815d4f0 2023-11-28T18:30:44.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.609+0000 7f43e4ff9640 1 --2- 172.21.15.105:0/3054307951 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f43c8066900 0x7f43c8068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:44.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.609+0000 7f43e4ff9640 1 -- 172.21.15.105:0/3054307951 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f43e0083ce0 con 0x7f43e815d4f0 2023-11-28T18:30:44.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.609+0000 7f43e6ffd640 1 --2- 172.21.15.105:0/3054307951 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f43c8066900 0x7f43c8068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:44.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.609+0000 7f43e6ffd640 1 --2- 172.21.15.105:0/3054307951 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f43c8066900 0x7f43c8068dc0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f43e81f9990 tx=0x7f43d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:44.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.613+0000 7f43e4ff9640 1 -- 172.21.15.105:0/3054307951 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f43e0050dd0 con 0x7f43e815d4f0 2023-11-28T18:30:44.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.761+0000 7f43ed980640 1 -- 172.21.15.105:0/3054307951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f43e80cb850 con 0x7f43e815d4f0 2023-11-28T18:30:44.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.761+0000 7f43e4ff9640 1 -- 172.21.15.105:0/3054307951 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) v1 ==== 104+0+11 (secure 0 0 0) 0x7f43e00180a0 con 0x7f43e815d4f0 2023-11-28T18:30:44.760 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:30:44.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.761+0000 7f43ed980640 1 -- 172.21.15.105:0/3054307951 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f43c8066900 msgr2=0x7f43c8068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:44.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.761+0000 7f43ed980640 1 --2- 172.21.15.105:0/3054307951 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f43c8066900 0x7f43c8068dc0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f43e81f9990 tx=0x7f43d8008040 comp rx=0 tx=0).stop 2023-11-28T18:30:44.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.761+0000 7f43ed980640 1 -- 172.21.15.105:0/3054307951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43e815d4f0 msgr2=0x7f43e815b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:44.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.761+0000 7f43ed980640 1 --2- 172.21.15.105:0/3054307951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43e815d4f0 0x7f43e815b260 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f43e000c780 tx=0x7f43e000cc40 comp rx=0 tx=0).stop 2023-11-28T18:30:44.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.761+0000 7f43ed980640 1 -- 172.21.15.105:0/3054307951 shutdown_connections 2023-11-28T18:30:44.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.761+0000 7f43ed980640 1 --2- 172.21.15.105:0/3054307951 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f43c8066900 0x7f43c8068dc0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.761+0000 7f43ed980640 1 --2- 172.21.15.105:0/3054307951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43e815d4f0 0x7f43e815b260 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.761+0000 7f43ed980640 1 --2- 172.21.15.105:0/3054307951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43e80d3c30 0x7f43e814c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.761+0000 7f43ed980640 1 --2- 172.21.15.105:0/3054307951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43e80d1890 0x7f43e81608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.761+0000 7f43ed980640 1 -- 172.21.15.105:0/3054307951 >> 172.21.15.105:0/3054307951 conn(0x7f43e80d3030 msgr2=0x7f43e815c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:44.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.761+0000 7f43ed980640 1 -- 172.21.15.105:0/3054307951 shutdown_connections 2023-11-28T18:30:44.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.761+0000 7f43ed980640 1 -- 172.21.15.105:0/3054307951 wait complete. 2023-11-28T18:30:44.781 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:30:44.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.921+0000 7f156dcfe640 1 Processor -- start 2023-11-28T18:30:44.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.921+0000 7f156dcfe640 1 -- start start 2023-11-28T18:30:44.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.921+0000 7f156dcfe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f156815d3c0 0x7f1568071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:44.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.921+0000 7f156dcfe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1568070a10 0x7f1568070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:44.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.921+0000 7f156dcfe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15680713a0 con 0x7f1568161870 2023-11-28T18:30:44.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.921+0000 7f156dcfe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15680cefa0 con 0x7f156815d3c0 2023-11-28T18:30:44.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.921+0000 7f156dcfe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156815d7a0 con 0x7f1568070a10 2023-11-28T18:30:44.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.921+0000 7f15677fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f156815d3c0 0x7f1568071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:44.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.921+0000 7f155ffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f1568161870 0x7f15680ce190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33676/0 (socket says 172.21.15.105:33676) 2023-11-28T18:30:44.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.921+0000 7f155ffff640 1 -- 172.21.15.105:0/3232879560 learned_addr learned my addr 172.21.15.105:0/3232879560 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:44.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.925+0000 7f1567fff640 1 --2- 172.21.15.105:0/3232879560 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1568070a10 0x7f1568070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:44.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.925+0000 7f15677fe640 1 -- 172.21.15.105:0/3232879560 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1568070a10 msgr2=0x7f1568070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:44.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.925+0000 7f15677fe640 1 --2- 172.21.15.105:0/3232879560 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1568070a10 0x7f1568070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.925+0000 7f15677fe640 1 -- 172.21.15.105:0/3232879560 >> v1:172.21.15.105:6789/0 conn(0x7f1568161870 legacy=0x7f15680ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:44.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.925+0000 7f15677fe640 1 -- 172.21.15.105:0/3232879560 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15680ce9e0 con 0x7f156815d3c0 2023-11-28T18:30:44.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.925+0000 7f1567fff640 1 --2- 172.21.15.105:0/3232879560 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1568070a10 0x7f1568070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:44.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.925+0000 7f15677fe640 1 --2- 172.21.15.105:0/3232879560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f156815d3c0 0x7f1568071a10 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f15580097c0 tx=0x7f155801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789b9631fad6c469 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:44.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.925+0000 7f1566ffd640 1 -- 172.21.15.105:0/3232879560 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15580ab650 con 0x7f156815d3c0 2023-11-28T18:30:44.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.925+0000 7f1566ffd640 1 -- 172.21.15.105:0/3232879560 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15580ab7b0 con 0x7f156815d3c0 2023-11-28T18:30:44.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.925+0000 7f1566ffd640 1 -- 172.21.15.105:0/3232879560 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15580abb40 con 0x7f156815d3c0 2023-11-28T18:30:44.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.925+0000 7f156dcfe640 1 -- 172.21.15.105:0/3232879560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f156815d3c0 msgr2=0x7f1568071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:44.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.925+0000 7f156dcfe640 1 --2- 172.21.15.105:0/3232879560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f156815d3c0 0x7f1568071a10 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f15580097c0 tx=0x7f155801c450 comp rx=0 tx=0).stop 2023-11-28T18:30:44.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.925+0000 7f156dcfe640 1 -- 172.21.15.105:0/3232879560 shutdown_connections 2023-11-28T18:30:44.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.925+0000 7f156dcfe640 1 --2- 172.21.15.105:0/3232879560 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1568070a10 0x7f1568070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.925+0000 7f156dcfe640 1 --2- 172.21.15.105:0/3232879560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f156815d3c0 0x7f1568071a10 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.925+0000 7f156dcfe640 1 -- 172.21.15.105:0/3232879560 >> 172.21.15.105:0/3232879560 conn(0x7f15680cf540 msgr2=0x7f15680cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:44.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.929+0000 7f156dcfe640 1 -- 172.21.15.105:0/3232879560 shutdown_connections 2023-11-28T18:30:44.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.929+0000 7f156dcfe640 1 -- 172.21.15.105:0/3232879560 wait complete. 2023-11-28T18:30:44.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.929+0000 7f156dcfe640 1 Processor -- start 2023-11-28T18:30:44.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.929+0000 7f156dcfe640 1 -- start start 2023-11-28T18:30:44.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.929+0000 7f156dcfe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1568070a10 0x7f1568154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:44.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.929+0000 7f156dcfe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f156815d3c0 0x7f156814d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:44.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.929+0000 7f156dcfe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1568161870 0x7f15680b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:44.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.929+0000 7f156dcfe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156814e8c0 con 0x7f1568070a10 2023-11-28T18:30:44.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.929+0000 7f156dcfe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156814dcb0 con 0x7f156815d3c0 2023-11-28T18:30:44.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.929+0000 7f156dcfe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156814df90 con 0x7f1568161870 2023-11-28T18:30:44.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.929+0000 7f15677fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1568070a10 0x7f1568154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:44.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.929+0000 7f15677fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1568070a10 0x7f1568154e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35144/0 (socket says 172.21.15.105:35144) 2023-11-28T18:30:44.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.929+0000 7f15677fe640 1 -- 172.21.15.105:0/563284972 learned_addr learned my addr 172.21.15.105:0/563284972 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:44.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.929+0000 7f155ffff640 1 --2- 172.21.15.105:0/563284972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f156815d3c0 0x7f156814d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:44.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.929+0000 7f1567fff640 1 --2- 172.21.15.105:0/563284972 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1568161870 0x7f15680b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:44.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.929+0000 7f15677fe640 1 -- 172.21.15.105:0/563284972 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1568161870 msgr2=0x7f15680b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:44.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.929+0000 7f15677fe640 1 --2- 172.21.15.105:0/563284972 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1568161870 0x7f15680b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.929+0000 7f15677fe640 1 -- 172.21.15.105:0/563284972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f156815d3c0 msgr2=0x7f156814d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:44.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.929+0000 7f15677fe640 1 --2- 172.21.15.105:0/563284972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f156815d3c0 0x7f156814d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:44.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.929+0000 7f15677fe640 1 -- 172.21.15.105:0/563284972 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1558009420 con 0x7f1568070a10 2023-11-28T18:30:44.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.929+0000 7f15677fe640 1 --2- 172.21.15.105:0/563284972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1568070a10 0x7f1568154e20 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f15580ad040 tx=0x7f155801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:44.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.929+0000 7f1564ff9640 1 -- 172.21.15.105:0/563284972 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15580b5810 con 0x7f1568070a10 2023-11-28T18:30:44.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.933+0000 7f1564ff9640 1 -- 172.21.15.105:0/563284972 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15580b5990 con 0x7f1568070a10 2023-11-28T18:30:44.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.933+0000 7f1564ff9640 1 -- 172.21.15.105:0/563284972 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15580b5d40 con 0x7f1568070a10 2023-11-28T18:30:44.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.933+0000 7f156dcfe640 1 -- 172.21.15.105:0/563284972 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156814e270 con 0x7f1568070a10 2023-11-28T18:30:44.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.933+0000 7f156dcfe640 1 -- 172.21.15.105:0/563284972 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156814d910 con 0x7f1568070a10 2023-11-28T18:30:44.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.933+0000 7f156dcfe640 1 -- 172.21.15.105:0/563284972 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1568067570 con 0x7f1568070a10 2023-11-28T18:30:44.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.933+0000 7f1564ff9640 1 -- 172.21.15.105:0/563284972 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f15580b4390 con 0x7f1568070a10 2023-11-28T18:30:44.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.933+0000 7f1564ff9640 1 --2- 172.21.15.105:0/563284972 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15480668d0 0x7f1548068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:44.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.933+0000 7f1564ff9640 1 -- 172.21.15.105:0/563284972 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f155811f1a0 con 0x7f1568070a10 2023-11-28T18:30:44.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.933+0000 7f155ffff640 1 --2- 172.21.15.105:0/563284972 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15480668d0 0x7f1548068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:44.935 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.933+0000 7f155ffff640 1 --2- 172.21.15.105:0/563284972 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15480668d0 0x7f1548068d90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1568152c20 tx=0x7f1550009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:44.938 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:44.937+0000 7f1564ff9640 1 -- 172.21.15.105:0/563284972 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f15580ec290 con 0x7f1568070a10 2023-11-28T18:30:45.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.089+0000 7f156dcfe640 1 -- 172.21.15.105:0/563284972 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f156814e0d0 con 0x7f1568070a10 2023-11-28T18:30:45.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.089+0000 7f1564ff9640 1 -- 172.21.15.105:0/563284972 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) v1 ==== 100+0+11 (secure 0 0 0) 0x7f156814e0d0 con 0x7f1568070a10 2023-11-28T18:30:45.090 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:30:45.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.093+0000 7f156dcfe640 1 -- 172.21.15.105:0/563284972 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15480668d0 msgr2=0x7f1548068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:45.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.093+0000 7f156dcfe640 1 --2- 172.21.15.105:0/563284972 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15480668d0 0x7f1548068d90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1568152c20 tx=0x7f1550009040 comp rx=0 tx=0).stop 2023-11-28T18:30:45.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.093+0000 7f156dcfe640 1 -- 172.21.15.105:0/563284972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1568070a10 msgr2=0x7f1568154e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:45.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.093+0000 7f156dcfe640 1 --2- 172.21.15.105:0/563284972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1568070a10 0x7f1568154e20 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f15580ad040 tx=0x7f155801ac90 comp rx=0 tx=0).stop 2023-11-28T18:30:45.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.093+0000 7f156dcfe640 1 -- 172.21.15.105:0/563284972 shutdown_connections 2023-11-28T18:30:45.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.093+0000 7f156dcfe640 1 --2- 172.21.15.105:0/563284972 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1568161870 0x7f15680b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:45.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.093+0000 7f156dcfe640 1 --2- 172.21.15.105:0/563284972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f156815d3c0 0x7f156814d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:45.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.093+0000 7f156dcfe640 1 --2- 172.21.15.105:0/563284972 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15480668d0 0x7f1548068d90 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:45.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.093+0000 7f156dcfe640 1 --2- 172.21.15.105:0/563284972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1568070a10 0x7f1568154e20 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:45.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.093+0000 7f156dcfe640 1 -- 172.21.15.105:0/563284972 >> 172.21.15.105:0/563284972 conn(0x7f15680cf540 msgr2=0x7f15680af160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:45.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.093+0000 7f156dcfe640 1 -- 172.21.15.105:0/563284972 shutdown_connections 2023-11-28T18:30:45.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.093+0000 7f156dcfe640 1 -- 172.21.15.105:0/563284972 wait complete. 2023-11-28T18:30:45.111 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T18:30:45.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.253+0000 7f6b1f165640 1 Processor -- start 2023-11-28T18:30:45.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.253+0000 7f6b1f165640 1 -- start start 2023-11-28T18:30:45.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.253+0000 7f6b1f165640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6b180d3c30 0x7f6b180d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:45.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.253+0000 7f6b1f165640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b180b2d20 0x7f6b180b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:45.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.253+0000 7f6b1f165640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b180b25b0 con 0x7f6b1815d4f0 2023-11-28T18:30:45.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.253+0000 7f6b1f165640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b180b2900 con 0x7f6b180d3c30 2023-11-28T18:30:45.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.253+0000 7f6b1f165640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b180d0a20 con 0x7f6b180b2d20 2023-11-28T18:30:45.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.253+0000 7f6b1cf01640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f6b1815d4f0 0x7f6b180d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33688/0 (socket says 172.21.15.105:33688) 2023-11-28T18:30:45.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.253+0000 7f6b1cf01640 1 -- 172.21.15.105:0/411239674 learned_addr learned my addr 172.21.15.105:0/411239674 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:45.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.253+0000 7f6b0ffff640 1 --2- 172.21.15.105:0/411239674 >> v2:172.21.15.148:3300/0 conn(0x7f6b180d3c30 0x7f6b180d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:45.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.253+0000 7f6b1d702640 1 --2- 172.21.15.105:0/411239674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b180b2d20 0x7f6b180b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:45.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.253+0000 7f6b0ffff640 1 -- 172.21.15.105:0/411239674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b180b2d20 msgr2=0x7f6b180b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:45.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.253+0000 7f6b0ffff640 1 --2- 172.21.15.105:0/411239674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b180b2d20 0x7f6b180b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:45.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.253+0000 7f6b0ffff640 1 -- 172.21.15.105:0/411239674 >> v1:172.21.15.105:6789/0 conn(0x7f6b1815d4f0 legacy=0x7f6b180d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:45.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.253+0000 7f6b0ffff640 1 -- 172.21.15.105:0/411239674 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b180cea30 con 0x7f6b180d3c30 2023-11-28T18:30:45.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.257+0000 7f6b0ffff640 1 --2- 172.21.15.105:0/411239674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b180d3c30 0x7f6b180d1520 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6b080097c0 tx=0x7f6b0801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d956bc2c356099ca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:45.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.257+0000 7f6b0f7fe640 1 -- 172.21.15.105:0/411239674 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b080ac620 con 0x7f6b180d3c30 2023-11-28T18:30:45.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.257+0000 7f6b0f7fe640 1 -- 172.21.15.105:0/411239674 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b080ac780 con 0x7f6b180d3c30 2023-11-28T18:30:45.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.257+0000 7f6b0f7fe640 1 -- 172.21.15.105:0/411239674 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b080acb10 con 0x7f6b180d3c30 2023-11-28T18:30:45.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.257+0000 7f6b1f165640 1 -- 172.21.15.105:0/411239674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b180d3c30 msgr2=0x7f6b180d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:45.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.257+0000 7f6b1f165640 1 --2- 172.21.15.105:0/411239674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b180d3c30 0x7f6b180d1520 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6b080097c0 tx=0x7f6b0801c3c0 comp rx=0 tx=0).stop 2023-11-28T18:30:45.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.257+0000 7f6b1f165640 1 -- 172.21.15.105:0/411239674 shutdown_connections 2023-11-28T18:30:45.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.257+0000 7f6b1f165640 1 --2- 172.21.15.105:0/411239674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b180b2d20 0x7f6b180b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:45.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.257+0000 7f6b1f165640 1 --2- 172.21.15.105:0/411239674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b180d3c30 0x7f6b180d1520 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:45.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.257+0000 7f6b1f165640 1 -- 172.21.15.105:0/411239674 >> 172.21.15.105:0/411239674 conn(0x7f6b180d3030 msgr2=0x7f6b180d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:45.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.257+0000 7f6b1f165640 1 -- 172.21.15.105:0/411239674 shutdown_connections 2023-11-28T18:30:45.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.257+0000 7f6b1f165640 1 -- 172.21.15.105:0/411239674 wait complete. 2023-11-28T18:30:45.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b1f165640 1 Processor -- start 2023-11-28T18:30:45.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b1f165640 1 -- start start 2023-11-28T18:30:45.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b1f165640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b180b2d20 0x7f6b18155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:45.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b1f165640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b180d3c30 0x7f6b18153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:45.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b1f165640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1815d4f0 0x7f6b18153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:45.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b1f165640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b18153a50 con 0x7f6b1815d4f0 2023-11-28T18:30:45.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b1f165640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b181542f0 con 0x7f6b180d3c30 2023-11-28T18:30:45.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b1f165640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b181545d0 con 0x7f6b180b2d20 2023-11-28T18:30:45.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b1d702640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1815d4f0 0x7f6b18153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:45.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b1d702640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1815d4f0 0x7f6b18153510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35154/0 (socket says 172.21.15.105:35154) 2023-11-28T18:30:45.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b1d702640 1 -- 172.21.15.105:0/3423354846 learned_addr learned my addr 172.21.15.105:0/3423354846 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:45.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b0ffff640 1 --2- 172.21.15.105:0/3423354846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b180d3c30 0x7f6b18153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:45.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b1cf01640 1 --2- 172.21.15.105:0/3423354846 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b180b2d20 0x7f6b18155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:45.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b1d702640 1 -- 172.21.15.105:0/3423354846 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b180b2d20 msgr2=0x7f6b18155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:45.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b1d702640 1 --2- 172.21.15.105:0/3423354846 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b180b2d20 0x7f6b18155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:45.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b1d702640 1 -- 172.21.15.105:0/3423354846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b180d3c30 msgr2=0x7f6b18153c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:45.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b1d702640 1 --2- 172.21.15.105:0/3423354846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b180d3c30 0x7f6b18153c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:45.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b1d702640 1 -- 172.21.15.105:0/3423354846 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b08009420 con 0x7f6b1815d4f0 2023-11-28T18:30:45.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b1d702640 1 --2- 172.21.15.105:0/3423354846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1815d4f0 0x7f6b18153510 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f6b1000efc0 tx=0x7f6b1000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:45.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b0d7fa640 1 -- 172.21.15.105:0/3423354846 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b10009ba0 con 0x7f6b1815d4f0 2023-11-28T18:30:45.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b0d7fa640 1 -- 172.21.15.105:0/3423354846 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1000f040 con 0x7f6b1815d4f0 2023-11-28T18:30:45.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b0d7fa640 1 -- 172.21.15.105:0/3423354846 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b100146c0 con 0x7f6b1815d4f0 2023-11-28T18:30:45.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b1f165640 1 -- 172.21.15.105:0/3423354846 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b181548b0 con 0x7f6b1815d4f0 2023-11-28T18:30:45.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.261+0000 7f6b1f165640 1 -- 172.21.15.105:0/3423354846 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b18154e70 con 0x7f6b1815d4f0 2023-11-28T18:30:45.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.265+0000 7f6b1f165640 1 -- 172.21.15.105:0/3423354846 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b180b7250 con 0x7f6b1815d4f0 2023-11-28T18:30:45.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.265+0000 7f6b0d7fa640 1 -- 172.21.15.105:0/3423354846 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f6b10014820 con 0x7f6b1815d4f0 2023-11-28T18:30:45.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.265+0000 7f6b0d7fa640 1 --2- 172.21.15.105:0/3423354846 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b040668d0 0x7f6b04068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:45.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.265+0000 7f6b0d7fa640 1 -- 172.21.15.105:0/3423354846 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6b10084f00 con 0x7f6b1815d4f0 2023-11-28T18:30:45.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.265+0000 7f6b1cf01640 1 --2- 172.21.15.105:0/3423354846 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b040668d0 0x7f6b04068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:45.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.265+0000 7f6b1cf01640 1 --2- 172.21.15.105:0/3423354846 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b040668d0 0x7f6b04068d90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f6b14005890 tx=0x7f6b14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:45.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.269+0000 7f6b0d7fa640 1 -- 172.21.15.105:0/3423354846 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6b10052070 con 0x7f6b1815d4f0 2023-11-28T18:30:45.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.433+0000 7f6b1f165640 1 -- 172.21.15.105:0/3423354846 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6b18067570 con 0x7f6b1815d4f0 2023-11-28T18:30:45.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.433+0000 7f6b0d7fa640 1 -- 172.21.15.105:0/3423354846 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v105) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6b10018080 con 0x7f6b1815d4f0 2023-11-28T18:30:45.433 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2023-11-28T18:30:45.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.433+0000 7f6b1f165640 1 -- 172.21.15.105:0/3423354846 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b040668d0 msgr2=0x7f6b04068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:45.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.433+0000 7f6b1f165640 1 --2- 172.21.15.105:0/3423354846 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b040668d0 0x7f6b04068d90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f6b14005890 tx=0x7f6b14009040 comp rx=0 tx=0).stop 2023-11-28T18:30:45.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.433+0000 7f6b1f165640 1 -- 172.21.15.105:0/3423354846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1815d4f0 msgr2=0x7f6b18153510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:45.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.433+0000 7f6b1f165640 1 --2- 172.21.15.105:0/3423354846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1815d4f0 0x7f6b18153510 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f6b1000efc0 tx=0x7f6b1000c490 comp rx=0 tx=0).stop 2023-11-28T18:30:45.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.433+0000 7f6b1f165640 1 -- 172.21.15.105:0/3423354846 shutdown_connections 2023-11-28T18:30:45.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.433+0000 7f6b1f165640 1 --2- 172.21.15.105:0/3423354846 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b040668d0 0x7f6b04068d90 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:45.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.433+0000 7f6b1f165640 1 --2- 172.21.15.105:0/3423354846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1815d4f0 0x7f6b18153510 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:45.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.433+0000 7f6b1f165640 1 --2- 172.21.15.105:0/3423354846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b180d3c30 0x7f6b18153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:45.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.433+0000 7f6b1f165640 1 --2- 172.21.15.105:0/3423354846 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b180b2d20 0x7f6b18155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:45.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.433+0000 7f6b1f165640 1 -- 172.21.15.105:0/3423354846 >> 172.21.15.105:0/3423354846 conn(0x7f6b180d3030 msgr2=0x7f6b1806c750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:45.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.433+0000 7f6b1f165640 1 -- 172.21.15.105:0/3423354846 shutdown_connections 2023-11-28T18:30:45.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.433+0000 7f6b1f165640 1 -- 172.21.15.105:0/3423354846 wait complete. 2023-11-28T18:30:45.455 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2023-11-28T18:30:45.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:45.456 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-28T18:30:45.475 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[63065]: fuse finished with error 0 and tester_r 0 2023-11-28T18:30:45.478 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:30:45.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:45.478 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T18:30:45.653 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:30:45.653 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:30:45.663 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:30:45.663 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:30:45.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:45.664 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T18:30:45.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:30:45.722 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-28T18:30:45.722 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:30:45.722 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:30:45.731 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:30:45.732 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:30:45.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:45.733 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T18:30:45.787 INFO:teuthology.orchestra.run.smithi105.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-28T18:30:45.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:30:45.789 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:30:45.789 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:30:45.798 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:30:45.798 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:30:45.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:30:45.799 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T18:30:45.852 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-28T18:30:45.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:30:45.853 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:30:45.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.989+0000 7efc40a9c640 1 Processor -- start 2023-11-28T18:30:45.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.989+0000 7efc40a9c640 1 -- start start 2023-11-28T18:30:45.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.993+0000 7efc40a9c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efc3c15d320 0x7efc3c06c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:45.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.993+0000 7efc40a9c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efc3c0ce230 0x7efc3c0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:45.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.993+0000 7efc40a9c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3c0b3c40 con 0x7efc3c0cdbb0 2023-11-28T18:30:45.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.993+0000 7efc40a9c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc3c06ca80 con 0x7efc3c0ce230 2023-11-28T18:30:45.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.993+0000 7efc40a9c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc3c15d700 con 0x7efc3c15d320 2023-11-28T18:30:45.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.993+0000 7efc33fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efc3c0ce230 0x7efc3c0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:45.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.993+0000 7efc3ad9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7efc3c0cdbb0 0x7efc3c0cef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33692/0 (socket says 172.21.15.105:33692) 2023-11-28T18:30:45.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.993+0000 7efc3ad9d640 1 -- 172.21.15.105:0/121450649 learned_addr learned my addr 172.21.15.105:0/121450649 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:45.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.993+0000 7efc3a59c640 1 --2- 172.21.15.105:0/121450649 >> v2:172.21.15.148:3300/0 conn(0x7efc3c15d320 0x7efc3c06c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:45.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.993+0000 7efc33fff640 1 -- 172.21.15.105:0/121450649 >> v2:172.21.15.148:3300/0 conn(0x7efc3c15d320 msgr2=0x7efc3c06c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:45.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.993+0000 7efc33fff640 1 --2- 172.21.15.105:0/121450649 >> v2:172.21.15.148:3300/0 conn(0x7efc3c15d320 0x7efc3c06c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:45.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.993+0000 7efc33fff640 1 -- 172.21.15.105:0/121450649 >> v1:172.21.15.105:6789/0 conn(0x7efc3c0cdbb0 legacy=0x7efc3c0cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:45.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.993+0000 7efc33fff640 1 -- 172.21.15.105:0/121450649 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c0cdfc0 con 0x7efc3c0ce230 2023-11-28T18:30:45.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.993+0000 7efc39d9b640 1 -- 172.21.15.105:0/121450649 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2878028035 0 0) 0x7efc3c0b3c40 con 0x7efc3c0cdbb0 2023-11-28T18:30:45.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.993+0000 7efc3a59c640 1 --2- 172.21.15.105:0/121450649 >> v2:172.21.15.148:3300/0 conn(0x7efc3c15d320 0x7efc3c06c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:45.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.997+0000 7efc33fff640 1 --2- 172.21.15.105:0/121450649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc3c0ce230 0x7efc3c0cd670 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7efc3c071d40 tx=0x7efc2401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee38aa7fdbccd25 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:45.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.997+0000 7efc39d9b640 1 -- 172.21.15.105:0/121450649 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc240ac630 con 0x7efc3c0ce230 2023-11-28T18:30:45.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.997+0000 7efc39d9b640 1 -- 172.21.15.105:0/121450649 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc240ac790 con 0x7efc3c0ce230 2023-11-28T18:30:45.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.997+0000 7efc39d9b640 1 -- 172.21.15.105:0/121450649 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc240acb20 con 0x7efc3c0ce230 2023-11-28T18:30:45.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.997+0000 7efc40a9c640 1 -- 172.21.15.105:0/121450649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc3c0ce230 msgr2=0x7efc3c0cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:45.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.997+0000 7efc40a9c640 1 --2- 172.21.15.105:0/121450649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc3c0ce230 0x7efc3c0cd670 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7efc3c071d40 tx=0x7efc2401c3d0 comp rx=0 tx=0).stop 2023-11-28T18:30:45.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.997+0000 7efc40a9c640 1 -- 172.21.15.105:0/121450649 shutdown_connections 2023-11-28T18:30:45.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.997+0000 7efc40a9c640 1 --2- 172.21.15.105:0/121450649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc3c0ce230 0x7efc3c0cd670 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:45.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.997+0000 7efc40a9c640 1 --2- 172.21.15.105:0/121450649 >> v2:172.21.15.148:3300/0 conn(0x7efc3c15d320 0x7efc3c06c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:45.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.997+0000 7efc40a9c640 1 -- 172.21.15.105:0/121450649 >> 172.21.15.105:0/121450649 conn(0x7efc3c0d0df0 msgr2=0x7efc3c0d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:45.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.997+0000 7efc40a9c640 1 -- 172.21.15.105:0/121450649 shutdown_connections 2023-11-28T18:30:45.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.997+0000 7efc40a9c640 1 -- 172.21.15.105:0/121450649 wait complete. 2023-11-28T18:30:45.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.997+0000 7efc40a9c640 1 Processor -- start 2023-11-28T18:30:45.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:45.997+0000 7efc40a9c640 1 -- start start 2023-11-28T18:30:46.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc40a9c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc3c0cdbb0 0x7efc3c151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:46.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc40a9c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc3c0ce230 0x7efc3c152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:46.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc40a9c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc3c15d320 0x7efc3c15f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:46.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc40a9c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c15f3d0 con 0x7efc3c15d320 2023-11-28T18:30:46.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc40a9c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c1537c0 con 0x7efc3c0cdbb0 2023-11-28T18:30:46.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc40a9c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c15eaa0 con 0x7efc3c0ce230 2023-11-28T18:30:46.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc3ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc3c15d320 0x7efc3c15f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:46.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc3ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc3c15d320 0x7efc3c15f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35172/0 (socket says 172.21.15.105:35172) 2023-11-28T18:30:46.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc3ad9d640 1 -- 172.21.15.105:0/552397748 learned_addr learned my addr 172.21.15.105:0/552397748 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:46.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc3a59c640 1 --2- 172.21.15.105:0/552397748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc3c0cdbb0 0x7efc3c151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:46.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc33fff640 1 --2- 172.21.15.105:0/552397748 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc3c0ce230 0x7efc3c152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:46.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc3ad9d640 1 -- 172.21.15.105:0/552397748 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc3c0ce230 msgr2=0x7efc3c152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:46.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc3ad9d640 1 --2- 172.21.15.105:0/552397748 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc3c0ce230 0x7efc3c152e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc3ad9d640 1 -- 172.21.15.105:0/552397748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc3c0cdbb0 msgr2=0x7efc3c151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:46.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc3ad9d640 1 --2- 172.21.15.105:0/552397748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc3c0cdbb0 0x7efc3c151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc3ad9d640 1 -- 172.21.15.105:0/552397748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc24009420 con 0x7efc3c15d320 2023-11-28T18:30:46.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc3a59c640 1 --2- 172.21.15.105:0/552397748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc3c0cdbb0 0x7efc3c151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:46.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc33fff640 1 --2- 172.21.15.105:0/552397748 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc3c0ce230 0x7efc3c152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:46.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc3ad9d640 1 --2- 172.21.15.105:0/552397748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc3c15d320 0x7efc3c15f860 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7efc34001570 tx=0x7efc3400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:46.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc32ffd640 1 -- 172.21.15.105:0/552397748 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3400a640 con 0x7efc3c15d320 2023-11-28T18:30:46.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc32ffd640 1 -- 172.21.15.105:0/552397748 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3400a7c0 con 0x7efc3c15d320 2023-11-28T18:30:46.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc40a9c640 1 -- 172.21.15.105:0/552397748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3c15ed80 con 0x7efc3c15d320 2023-11-28T18:30:46.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc32ffd640 1 -- 172.21.15.105:0/552397748 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3400ab70 con 0x7efc3c15d320 2023-11-28T18:30:46.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc40a9c640 1 -- 172.21.15.105:0/552397748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3c153390 con 0x7efc3c15d320 2023-11-28T18:30:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc40a9c640 1 -- 172.21.15.105:0/552397748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc3c067570 con 0x7efc3c15d320 2023-11-28T18:30:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.001+0000 7efc32ffd640 1 -- 172.21.15.105:0/552397748 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7efc3400acd0 con 0x7efc3c15d320 2023-11-28T18:30:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.005+0000 7efc32ffd640 1 --2- 172.21.15.105:0/552397748 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc1c0668d0 0x7efc1c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.005+0000 7efc32ffd640 1 -- 172.21.15.105:0/552397748 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efc34083e60 con 0x7efc3c15d320 2023-11-28T18:30:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.005+0000 7efc3a59c640 1 --2- 172.21.15.105:0/552397748 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc1c0668d0 0x7efc1c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:46.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.005+0000 7efc3a59c640 1 --2- 172.21.15.105:0/552397748 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc1c0668d0 0x7efc1c068d90 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0097a0 tx=0x7efc2c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:46.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.009+0000 7efc32ffd640 1 -- 172.21.15.105:0/552397748 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efc34050f40 con 0x7efc3c15d320 2023-11-28T18:30:46.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.185+0000 7efc40a9c640 1 -- 172.21.15.105:0/552397748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc3c06b220 con 0x7efc3c15d320 2023-11-28T18:30:46.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.189+0000 7efc32ffd640 1 -- 172.21.15.105:0/552397748 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4962 (secure 0 0 0) 0x7efc34018020 con 0x7efc3c15d320 2023-11-28T18:30:46.188 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:46.188 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":110,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/2603493310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2603493310},{"type":"v1","addr":"172.21.15.148:6834","nonce":2603493310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:30:05.525117+0000","modified":"2023-11-28T18:30:40.827658+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8145,"mds_1":8142,"mds_2":6803},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6803":{"gid":6803,"name":"d","rank":2,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6835/3664281230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":3664281230},{"type":"v1","addr":"172.21.15.148:6835","nonce":3664281230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8142":{"gid":8142,"name":"a","rank":1,"incarnation":105,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6836/3968059262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3968059262},{"type":"v1","addr":"172.21.15.105:6836","nonce":3968059262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8145":{"gid":8145,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2782729992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2782729992},{"type":"v1","addr":"172.21.15.105:6837","nonce":2782729992}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-11-28T18:30:46.188 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 110 2023-11-28T18:30:46.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.189+0000 7efc40a9c640 1 -- 172.21.15.105:0/552397748 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc1c0668d0 msgr2=0x7efc1c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:46.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.189+0000 7efc40a9c640 1 --2- 172.21.15.105:0/552397748 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc1c0668d0 0x7efc1c068d90 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0097a0 tx=0x7efc2c008040 comp rx=0 tx=0).stop 2023-11-28T18:30:46.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.189+0000 7efc40a9c640 1 -- 172.21.15.105:0/552397748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc3c15d320 msgr2=0x7efc3c15f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:46.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.189+0000 7efc40a9c640 1 --2- 172.21.15.105:0/552397748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc3c15d320 0x7efc3c15f860 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7efc34001570 tx=0x7efc3400cca0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.189+0000 7efc40a9c640 1 -- 172.21.15.105:0/552397748 shutdown_connections 2023-11-28T18:30:46.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.189+0000 7efc40a9c640 1 --2- 172.21.15.105:0/552397748 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efc1c0668d0 0x7efc1c068d90 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.189+0000 7efc40a9c640 1 --2- 172.21.15.105:0/552397748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc3c15d320 0x7efc3c15f860 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.189+0000 7efc40a9c640 1 --2- 172.21.15.105:0/552397748 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc3c0ce230 0x7efc3c152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.189+0000 7efc40a9c640 1 --2- 172.21.15.105:0/552397748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc3c0cdbb0 0x7efc3c151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.189+0000 7efc40a9c640 1 -- 172.21.15.105:0/552397748 >> 172.21.15.105:0/552397748 conn(0x7efc3c0d0df0 msgr2=0x7efc3c072fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:46.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.189+0000 7efc40a9c640 1 -- 172.21.15.105:0/552397748 shutdown_connections 2023-11-28T18:30:46.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.189+0000 7efc40a9c640 1 -- 172.21.15.105:0/552397748 wait complete. 2023-11-28T18:30:46.210 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:30:46.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.349+0000 7fe65f8c9640 1 Processor -- start 2023-11-28T18:30:46.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.349+0000 7fe65f8c9640 1 -- start start 2023-11-28T18:30:46.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.353+0000 7fe65f8c9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe6580d3c30 0x7fe6580d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:46.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.353+0000 7fe65f8c9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6580b25b0 con 0x7fe65815d4f0 2023-11-28T18:30:46.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.353+0000 7fe65f8c9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6580cea30 con 0x7fe6580b2d20 2023-11-28T18:30:46.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.353+0000 7fe65f8c9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6580b2900 con 0x7fe6580d3c30 2023-11-28T18:30:46.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.353+0000 7fe65d665640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe65815d4f0 0x7fe6580d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33704/0 (socket says 172.21.15.105:33704) 2023-11-28T18:30:46.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.353+0000 7fe65d665640 1 -- 172.21.15.105:0/3309441860 learned_addr learned my addr 172.21.15.105:0/3309441860 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:46.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.353+0000 7fe65ce64640 1 --2- 172.21.15.105:0/3309441860 >> v2:172.21.15.105:3300/0 conn(0x7fe6580d3c30 0x7fe6580d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:46.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.353+0000 7fe64ffff640 1 -- 172.21.15.105:0/3309441860 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2579345650 0 0) 0x7fe6580b25b0 con 0x7fe65815d4f0 2023-11-28T18:30:46.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.353+0000 7fe64ffff640 1 -- 172.21.15.105:0/3309441860 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe644003530 con 0x7fe65815d4f0 2023-11-28T18:30:46.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.353+0000 7fe65ce64640 1 -- 172.21.15.105:0/3309441860 >> v1:172.21.15.148:6789/0 conn(0x7fe6580b2d20 legacy=0x7fe6580b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:46.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.353+0000 7fe65ce64640 1 -- 172.21.15.105:0/3309441860 >> v1:172.21.15.105:6789/0 conn(0x7fe65815d4f0 legacy=0x7fe6580d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:46.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.353+0000 7fe65ce64640 1 -- 172.21.15.105:0/3309441860 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6580ced10 con 0x7fe6580d3c30 2023-11-28T18:30:46.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.357+0000 7fe65ce64640 1 --2- 172.21.15.105:0/3309441860 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6580d3c30 0x7fe6580d1520 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fe6480097c0 tx=0x7fe64801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5e0bb07f2d7ee0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:46.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.357+0000 7fe64ffff640 1 -- 172.21.15.105:0/3309441860 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6480ab600 con 0x7fe6580d3c30 2023-11-28T18:30:46.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.357+0000 7fe64ffff640 1 -- 172.21.15.105:0/3309441860 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6480a7050 con 0x7fe6580d3c30 2023-11-28T18:30:46.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.357+0000 7fe64ffff640 1 -- 172.21.15.105:0/3309441860 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6480ab980 con 0x7fe6580d3c30 2023-11-28T18:30:46.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.357+0000 7fe65f8c9640 1 -- 172.21.15.105:0/3309441860 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6580d3c30 msgr2=0x7fe6580d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:46.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.357+0000 7fe65f8c9640 1 --2- 172.21.15.105:0/3309441860 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6580d3c30 0x7fe6580d1520 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fe6480097c0 tx=0x7fe64801c380 comp rx=0 tx=0).stop 2023-11-28T18:30:46.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.357+0000 7fe65f8c9640 1 -- 172.21.15.105:0/3309441860 shutdown_connections 2023-11-28T18:30:46.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.357+0000 7fe65f8c9640 1 --2- 172.21.15.105:0/3309441860 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6580d3c30 0x7fe6580d1520 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.357+0000 7fe65f8c9640 1 -- 172.21.15.105:0/3309441860 >> 172.21.15.105:0/3309441860 conn(0x7fe6580d3030 msgr2=0x7fe6580d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:46.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.357+0000 7fe65f8c9640 1 -- 172.21.15.105:0/3309441860 shutdown_connections 2023-11-28T18:30:46.357 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.357+0000 7fe65f8c9640 1 -- 172.21.15.105:0/3309441860 wait complete. 2023-11-28T18:30:46.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.357+0000 7fe65f8c9640 1 Processor -- start 2023-11-28T18:30:46.358 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.357+0000 7fe65f8c9640 1 -- start start 2023-11-28T18:30:46.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.357+0000 7fe65f8c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6580b2d20 0x7fe6581545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:46.360 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe65f8c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6580d3c30 0x7fe658155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:46.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe65f8c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe65815d4f0 0x7fe65806c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:46.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe65f8c9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6580d0a20 con 0x7fe6580d3c30 2023-11-28T18:30:46.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe65f8c9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65815fb10 con 0x7fe6580b2d20 2023-11-28T18:30:46.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe65f8c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe658154af0 con 0x7fe65815d4f0 2023-11-28T18:30:46.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe65ce64640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6580d3c30 0x7fe658155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:46.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe65ce64640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6580d3c30 0x7fe658155520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35184/0 (socket says 172.21.15.105:35184) 2023-11-28T18:30:46.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe65ce64640 1 -- 172.21.15.105:0/2909790576 learned_addr learned my addr 172.21.15.105:0/2909790576 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:46.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe65d665640 1 --2- 172.21.15.105:0/2909790576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6580b2d20 0x7fe6581545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:46.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe65de66640 1 --2- 172.21.15.105:0/2909790576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe65815d4f0 0x7fe65806c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:46.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe65ce64640 1 -- 172.21.15.105:0/2909790576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe65815d4f0 msgr2=0x7fe65806c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:46.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe65ce64640 1 --2- 172.21.15.105:0/2909790576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe65815d4f0 0x7fe65806c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe65ce64640 1 -- 172.21.15.105:0/2909790576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6580b2d20 msgr2=0x7fe6581545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:46.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe65ce64640 1 --2- 172.21.15.105:0/2909790576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6580b2d20 0x7fe6581545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe65ce64640 1 -- 172.21.15.105:0/2909790576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe648009420 con 0x7fe6580d3c30 2023-11-28T18:30:46.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe65d665640 1 --2- 172.21.15.105:0/2909790576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6580b2d20 0x7fe6581545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:46.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe65ce64640 1 --2- 172.21.15.105:0/2909790576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6580d3c30 0x7fe658155520 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fe6480ae040 tx=0x7fe648019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:46.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe64dffb640 1 -- 172.21.15.105:0/2909790576 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6480b4430 con 0x7fe6580d3c30 2023-11-28T18:30:46.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe64dffb640 1 -- 172.21.15.105:0/2909790576 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6480b45b0 con 0x7fe6580d3c30 2023-11-28T18:30:46.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe64dffb640 1 -- 172.21.15.105:0/2909790576 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6480b4960 con 0x7fe6580d3c30 2023-11-28T18:30:46.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe65f8c9640 1 -- 172.21.15.105:0/2909790576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe65815dd00 con 0x7fe6580d3c30 2023-11-28T18:30:46.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe65f8c9640 1 -- 172.21.15.105:0/2909790576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65815e2c0 con 0x7fe6580d3c30 2023-11-28T18:30:46.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe65f8c9640 1 -- 172.21.15.105:0/2909790576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe658067570 con 0x7fe6580d3c30 2023-11-28T18:30:46.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.361+0000 7fe64dffb640 1 -- 172.21.15.105:0/2909790576 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fe6480b2040 con 0x7fe6580d3c30 2023-11-28T18:30:46.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.365+0000 7fe64dffb640 1 --2- 172.21.15.105:0/2909790576 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe63c066900 0x7fe63c068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:46.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.365+0000 7fe64dffb640 1 -- 172.21.15.105:0/2909790576 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe64811e370 con 0x7fe6580d3c30 2023-11-28T18:30:46.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.365+0000 7fe65d665640 1 --2- 172.21.15.105:0/2909790576 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe63c066900 0x7fe63c068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:46.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.365+0000 7fe65d665640 1 --2- 172.21.15.105:0/2909790576 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe63c066900 0x7fe63c068dc0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe6540011f0 tx=0x7fe654009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:46.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.369+0000 7fe64dffb640 1 -- 172.21.15.105:0/2909790576 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe6480eb440 con 0x7fe6580d3c30 2023-11-28T18:30:46.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.517+0000 7fe65f8c9640 1 -- 172.21.15.105:0/2909790576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6580733e0 con 0x7fe6580d3c30 2023-11-28T18:30:46.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.521+0000 7fe64dffb640 1 -- 172.21.15.105:0/2909790576 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16826 (secure 0 0 0) 0x7fe6480eea60 con 0x7fe6580d3c30 2023-11-28T18:30:46.520 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:46.520 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":105,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:30:16.894705+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:30:03.341424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-28T18:30:04.527931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1b","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:30:46.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.521+0000 7fe65f8c9640 1 -- 172.21.15.105:0/2909790576 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe63c066900 msgr2=0x7fe63c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:46.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.521+0000 7fe65f8c9640 1 --2- 172.21.15.105:0/2909790576 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe63c066900 0x7fe63c068dc0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe6540011f0 tx=0x7fe654009040 comp rx=0 tx=0).stop 2023-11-28T18:30:46.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.521+0000 7fe65f8c9640 1 -- 172.21.15.105:0/2909790576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6580d3c30 msgr2=0x7fe658155520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:46.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.521+0000 7fe65f8c9640 1 --2- 172.21.15.105:0/2909790576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6580d3c30 0x7fe658155520 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fe6480ae040 tx=0x7fe648019120 comp rx=0 tx=0).stop 2023-11-28T18:30:46.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.521+0000 7fe65f8c9640 1 -- 172.21.15.105:0/2909790576 shutdown_connections 2023-11-28T18:30:46.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.521+0000 7fe65f8c9640 1 --2- 172.21.15.105:0/2909790576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe65815d4f0 0x7fe65806c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.521+0000 7fe65f8c9640 1 --2- 172.21.15.105:0/2909790576 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe63c066900 0x7fe63c068dc0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.521+0000 7fe65f8c9640 1 --2- 172.21.15.105:0/2909790576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6580d3c30 0x7fe658155520 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.521+0000 7fe65f8c9640 1 --2- 172.21.15.105:0/2909790576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6580b2d20 0x7fe6581545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.521+0000 7fe65f8c9640 1 -- 172.21.15.105:0/2909790576 >> 172.21.15.105:0/2909790576 conn(0x7fe6580d3030 msgr2=0x7fe658158900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:46.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.521+0000 7fe65f8c9640 1 -- 172.21.15.105:0/2909790576 shutdown_connections 2023-11-28T18:30:46.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.521+0000 7fe65f8c9640 1 -- 172.21.15.105:0/2909790576 wait complete. 2023-11-28T18:30:46.542 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:30:03.341424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:30:04.527931+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:30:46.543 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:30:46.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.685+0000 7f1a9929a640 1 Processor -- start 2023-11-28T18:30:46.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.685+0000 7f1a9929a640 1 -- start start 2023-11-28T18:30:46.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.685+0000 7f1a9929a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1a9415d3c0 0x7f1a94071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:46.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.685+0000 7f1a9929a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1a94070a10 0x7f1a94070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:46.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.685+0000 7f1a9929a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a940cefa0 con 0x7f1a94161870 2023-11-28T18:30:46.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.685+0000 7f1a9929a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a94071360 con 0x7f1a94070a10 2023-11-28T18:30:46.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.685+0000 7f1a9929a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a9415d7a0 con 0x7f1a9415d3c0 2023-11-28T18:30:46.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.685+0000 7f1a937fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1a94070a10 0x7f1a94070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:46.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.685+0000 7f1a937fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1a94070a10 0x7f1a94070e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35194/0 (socket says 172.21.15.105:35194) 2023-11-28T18:30:46.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.685+0000 7f1a937fe640 1 -- 172.21.15.105:0/2442126027 learned_addr learned my addr 172.21.15.105:0/2442126027 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:46.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.685+0000 7f1a92ffd640 1 --2- 172.21.15.105:0/2442126027 >> v2:172.21.15.148:3300/0 conn(0x7f1a9415d3c0 0x7f1a94071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:46.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.685+0000 7f1a937fe640 1 -- 172.21.15.105:0/2442126027 >> v2:172.21.15.148:3300/0 conn(0x7f1a9415d3c0 msgr2=0x7f1a94071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:46.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.685+0000 7f1a937fe640 1 --2- 172.21.15.105:0/2442126027 >> v2:172.21.15.148:3300/0 conn(0x7f1a9415d3c0 0x7f1a94071a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.685+0000 7f1a937fe640 1 -- 172.21.15.105:0/2442126027 >> v1:172.21.15.148:6789/0 conn(0x7f1a94161870 legacy=0x7f1a940ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:46.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.685+0000 7f1a937fe640 1 -- 172.21.15.105:0/2442126027 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a940ce9e0 con 0x7f1a94070a10 2023-11-28T18:30:46.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.689+0000 7f1a937fe640 1 --2- 172.21.15.105:0/2442126027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a94070a10 0x7f1a94070e20 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f1a9406ba90 tx=0x7f1a8c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6dcca35d74fe4f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:46.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.689+0000 7f1a927fc640 1 -- 172.21.15.105:0/2442126027 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c0ab670 con 0x7f1a94070a10 2023-11-28T18:30:46.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.689+0000 7f1a927fc640 1 -- 172.21.15.105:0/2442126027 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c0ab7d0 con 0x7f1a94070a10 2023-11-28T18:30:46.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.689+0000 7f1a927fc640 1 -- 172.21.15.105:0/2442126027 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c0abb60 con 0x7f1a94070a10 2023-11-28T18:30:46.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.689+0000 7f1a9929a640 1 -- 172.21.15.105:0/2442126027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a94070a10 msgr2=0x7f1a94070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:46.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.689+0000 7f1a9929a640 1 --2- 172.21.15.105:0/2442126027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a94070a10 0x7f1a94070e20 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f1a9406ba90 tx=0x7f1a8c01c470 comp rx=0 tx=0).stop 2023-11-28T18:30:46.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.689+0000 7f1a9929a640 1 -- 172.21.15.105:0/2442126027 shutdown_connections 2023-11-28T18:30:46.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.689+0000 7f1a9929a640 1 --2- 172.21.15.105:0/2442126027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a94070a10 0x7f1a94070e20 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.689+0000 7f1a9929a640 1 --2- 172.21.15.105:0/2442126027 >> v2:172.21.15.148:3300/0 conn(0x7f1a9415d3c0 0x7f1a94071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.689+0000 7f1a9929a640 1 -- 172.21.15.105:0/2442126027 >> 172.21.15.105:0/2442126027 conn(0x7f1a940cf540 msgr2=0x7f1a940cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:46.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.689+0000 7f1a9929a640 1 -- 172.21.15.105:0/2442126027 shutdown_connections 2023-11-28T18:30:46.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.689+0000 7f1a9929a640 1 -- 172.21.15.105:0/2442126027 wait complete. 2023-11-28T18:30:46.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a9929a640 1 Processor -- start 2023-11-28T18:30:46.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a9929a640 1 -- start start 2023-11-28T18:30:46.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a9929a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a94070a10 0x7f1a941587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:46.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a9929a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a9415d3c0 0x7f1a94143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:46.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a9929a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a94161870 0x7f1a94156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:46.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a9929a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a94143660 con 0x7f1a94161870 2023-11-28T18:30:46.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a9929a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a94158360 con 0x7f1a9415d3c0 2023-11-28T18:30:46.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a9929a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a94157750 con 0x7f1a94070a10 2023-11-28T18:30:46.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a937fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a94161870 0x7f1a94156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:46.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a937fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a94161870 0x7f1a94156cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35210/0 (socket says 172.21.15.105:35210) 2023-11-28T18:30:46.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a937fe640 1 -- 172.21.15.105:0/4114918399 learned_addr learned my addr 172.21.15.105:0/4114918399 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:46.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a92ffd640 1 --2- 172.21.15.105:0/4114918399 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a94070a10 0x7f1a941587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:46.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a8a7fc640 1 --2- 172.21.15.105:0/4114918399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a9415d3c0 0x7f1a94143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:46.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a937fe640 1 -- 172.21.15.105:0/4114918399 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a94070a10 msgr2=0x7f1a941587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:46.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a937fe640 1 --2- 172.21.15.105:0/4114918399 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a94070a10 0x7f1a941587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a937fe640 1 -- 172.21.15.105:0/4114918399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a9415d3c0 msgr2=0x7f1a94143120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:46.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a937fe640 1 --2- 172.21.15.105:0/4114918399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a9415d3c0 0x7f1a94143120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a937fe640 1 -- 172.21.15.105:0/4114918399 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8c0094d0 con 0x7f1a94161870 2023-11-28T18:30:46.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a937fe640 1 --2- 172.21.15.105:0/4114918399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a94161870 0x7f1a94156cb0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c01ab20 tx=0x7f1a8c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:46.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a8bfff640 1 -- 172.21.15.105:0/4114918399 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c0b4800 con 0x7f1a94161870 2023-11-28T18:30:46.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a8bfff640 1 -- 172.21.15.105:0/4114918399 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c0b4960 con 0x7f1a94161870 2023-11-28T18:30:46.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a8bfff640 1 -- 172.21.15.105:0/4114918399 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c0b4d10 con 0x7f1a94161870 2023-11-28T18:30:46.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a9929a640 1 -- 172.21.15.105:0/4114918399 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a94161e80 con 0x7f1a94161870 2023-11-28T18:30:46.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.693+0000 7f1a9929a640 1 -- 172.21.15.105:0/4114918399 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a94157d10 con 0x7f1a94161870 2023-11-28T18:30:46.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.697+0000 7f1a9929a640 1 -- 172.21.15.105:0/4114918399 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a94161ce0 con 0x7f1a94161870 2023-11-28T18:30:46.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.697+0000 7f1a8bfff640 1 -- 172.21.15.105:0/4114918399 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f1a8c0bd390 con 0x7f1a94161870 2023-11-28T18:30:46.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.697+0000 7f1a8bfff640 1 --2- 172.21.15.105:0/4114918399 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1a740668d0 0x7f1a74068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:46.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.697+0000 7f1a8bfff640 1 -- 172.21.15.105:0/4114918399 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1a8c11ed60 con 0x7f1a94161870 2023-11-28T18:30:46.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.697+0000 7f1a92ffd640 1 --2- 172.21.15.105:0/4114918399 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1a740668d0 0x7f1a74068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:46.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.697+0000 7f1a92ffd640 1 --2- 172.21.15.105:0/4114918399 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1a740668d0 0x7f1a74068d90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1a84007a60 tx=0x7f1a84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:46.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.701+0000 7f1a8bfff640 1 -- 172.21.15.105:0/4114918399 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1a94161ce0 con 0x7f1a94161870 2023-11-28T18:30:46.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.853+0000 7f1a9929a640 1 -- 172.21.15.105:0/4114918399 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1a94161ce0 con 0x7f1a94161870 2023-11-28T18:30:46.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.853+0000 7f1a8bfff640 1 -- 172.21.15.105:0/4114918399 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1a94161ce0 con 0x7f1a94161870 2023-11-28T18:30:46.855 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:30:46.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.857+0000 7f1a9929a640 1 -- 172.21.15.105:0/4114918399 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1a740668d0 msgr2=0x7f1a74068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:46.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.857+0000 7f1a9929a640 1 --2- 172.21.15.105:0/4114918399 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1a740668d0 0x7f1a74068d90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1a84007a60 tx=0x7f1a84008040 comp rx=0 tx=0).stop 2023-11-28T18:30:46.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.857+0000 7f1a9929a640 1 -- 172.21.15.105:0/4114918399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a94161870 msgr2=0x7f1a94156cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:46.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.857+0000 7f1a9929a640 1 --2- 172.21.15.105:0/4114918399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a94161870 0x7f1a94156cb0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c01ab20 tx=0x7f1a8c01acb0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.857+0000 7f1a9929a640 1 -- 172.21.15.105:0/4114918399 shutdown_connections 2023-11-28T18:30:46.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.857+0000 7f1a9929a640 1 --2- 172.21.15.105:0/4114918399 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1a740668d0 0x7f1a74068d90 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.857+0000 7f1a9929a640 1 --2- 172.21.15.105:0/4114918399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a94161870 0x7f1a94156cb0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.857+0000 7f1a9929a640 1 --2- 172.21.15.105:0/4114918399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a9415d3c0 0x7f1a94143120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.857+0000 7f1a9929a640 1 --2- 172.21.15.105:0/4114918399 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a94070a10 0x7f1a941587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:46.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.857+0000 7f1a9929a640 1 -- 172.21.15.105:0/4114918399 >> 172.21.15.105:0/4114918399 conn(0x7f1a940cf540 msgr2=0x7f1a940af550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:46.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.857+0000 7f1a9929a640 1 -- 172.21.15.105:0/4114918399 shutdown_connections 2023-11-28T18:30:46.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:46.857+0000 7f1a9929a640 1 -- 172.21.15.105:0/4114918399 wait complete. 2023-11-28T18:30:46.876 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:30:47.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.017+0000 7f1522f2a640 1 Processor -- start 2023-11-28T18:30:47.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.017+0000 7f1522f2a640 1 -- start start 2023-11-28T18:30:47.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.017+0000 7f1522f2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f151c159f40 0x7f151c0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:47.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.017+0000 7f1522f2a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f151c0cc760 0x7f151c0ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:47.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.021+0000 7f1522f2a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f151c0ad680 0x7f151c15da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:47.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.021+0000 7f1522f2a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f151c06bf50 con 0x7f151c0ad680 2023-11-28T18:30:47.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.021+0000 7f1522f2a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f151c0ccbc0 con 0x7f151c0cc760 2023-11-28T18:30:47.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.021+0000 7f1522f2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151c15a320 con 0x7f151c159f40 2023-11-28T18:30:47.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.021+0000 7f15214c7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f151c0ad680 0x7f151c15da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:47.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.021+0000 7f15214c7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f151c0ad680 0x7f151c15da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35222/0 (socket says 172.21.15.105:35222) 2023-11-28T18:30:47.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.021+0000 7f15214c7640 1 -- 172.21.15.105:0/524643104 learned_addr learned my addr 172.21.15.105:0/524643104 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:47.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.021+0000 7f151bfff640 1 --2- 172.21.15.105:0/524643104 >> v2:172.21.15.148:3300/0 conn(0x7f151c0cc760 0x7f151c0ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:47.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.021+0000 7f1520cc6640 1 --2- 172.21.15.105:0/524643104 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f151c159f40 0x7f151c0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:47.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.021+0000 7f15214c7640 1 -- 172.21.15.105:0/524643104 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f151c159f40 msgr2=0x7f151c0caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:47.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.021+0000 7f15214c7640 1 --2- 172.21.15.105:0/524643104 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f151c159f40 0x7f151c0caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.021+0000 7f15214c7640 1 -- 172.21.15.105:0/524643104 >> v2:172.21.15.148:3300/0 conn(0x7f151c0cc760 msgr2=0x7f151c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:47.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.021+0000 7f15214c7640 1 --2- 172.21.15.105:0/524643104 >> v2:172.21.15.148:3300/0 conn(0x7f151c0cc760 0x7f151c0ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.021+0000 7f15214c7640 1 -- 172.21.15.105:0/524643104 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151c071380 con 0x7f151c0ad680 2023-11-28T18:30:47.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.021+0000 7f151bfff640 1 --2- 172.21.15.105:0/524643104 >> v2:172.21.15.148:3300/0 conn(0x7f151c0cc760 0x7f151c0ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:47.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.021+0000 7f15214c7640 1 --2- 172.21.15.105:0/524643104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f151c0ad680 0x7f151c15da20 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f151c0cfc90 tx=0x7f151401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e01087e53a5e28e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:47.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.021+0000 7f151b7fe640 1 -- 172.21.15.105:0/524643104 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15140a5790 con 0x7f151c0ad680 2023-11-28T18:30:47.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.021+0000 7f151b7fe640 1 -- 172.21.15.105:0/524643104 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15140a58f0 con 0x7f151c0ad680 2023-11-28T18:30:47.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.021+0000 7f151b7fe640 1 -- 172.21.15.105:0/524643104 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15140a5c80 con 0x7f151c0ad680 2023-11-28T18:30:47.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.025+0000 7f1522f2a640 1 -- 172.21.15.105:0/524643104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f151c0ad680 msgr2=0x7f151c15da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:47.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.025+0000 7f1522f2a640 1 --2- 172.21.15.105:0/524643104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f151c0ad680 0x7f151c15da20 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f151c0cfc90 tx=0x7f151401c910 comp rx=0 tx=0).stop 2023-11-28T18:30:47.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.025+0000 7f1522f2a640 1 -- 172.21.15.105:0/524643104 shutdown_connections 2023-11-28T18:30:47.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.025+0000 7f1522f2a640 1 --2- 172.21.15.105:0/524643104 >> v2:172.21.15.148:3300/0 conn(0x7f151c0cc760 0x7f151c0ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.025+0000 7f1522f2a640 1 --2- 172.21.15.105:0/524643104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f151c0ad680 0x7f151c15da20 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.025+0000 7f1522f2a640 1 --2- 172.21.15.105:0/524643104 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f151c159f40 0x7f151c0caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.025+0000 7f1522f2a640 1 -- 172.21.15.105:0/524643104 >> 172.21.15.105:0/524643104 conn(0x7f151c0ceb90 msgr2=0x7f151c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:47.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.025+0000 7f1522f2a640 1 -- 172.21.15.105:0/524643104 shutdown_connections 2023-11-28T18:30:47.024 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.025+0000 7f1522f2a640 1 -- 172.21.15.105:0/524643104 wait complete. 2023-11-28T18:30:47.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.025+0000 7f1522f2a640 1 Processor -- start 2023-11-28T18:30:47.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.025+0000 7f1522f2a640 1 -- start start 2023-11-28T18:30:47.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.025+0000 7f1522f2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f151c0ad680 0x7f151c154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:47.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.025+0000 7f1522f2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f151c0cc760 0x7f151c152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:47.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.025+0000 7f1522f2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f151c159f40 0x7f151c0ac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:47.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.025+0000 7f1522f2a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c0b4500 con 0x7f151c159f40 2023-11-28T18:30:47.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.025+0000 7f1522f2a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c154cb0 con 0x7f151c0cc760 2023-11-28T18:30:47.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.025+0000 7f1522f2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151c0b1b30 con 0x7f151c0ad680 2023-11-28T18:30:47.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.025+0000 7f15214c7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f151c159f40 0x7f151c0ac3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:47.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.025+0000 7f15214c7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f151c159f40 0x7f151c0ac3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35224/0 (socket says 172.21.15.105:35224) 2023-11-28T18:30:47.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.025+0000 7f15214c7640 1 -- 172.21.15.105:0/2938845029 learned_addr learned my addr 172.21.15.105:0/2938845029 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:47.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.025+0000 7f1520cc6640 1 --2- 172.21.15.105:0/2938845029 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f151c0ad680 0x7f151c154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:47.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.025+0000 7f151bfff640 1 --2- 172.21.15.105:0/2938845029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f151c0cc760 0x7f151c152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:47.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.029+0000 7f15214c7640 1 -- 172.21.15.105:0/2938845029 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f151c0ad680 msgr2=0x7f151c154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:47.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.029+0000 7f15214c7640 1 --2- 172.21.15.105:0/2938845029 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f151c0ad680 0x7f151c154770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.029+0000 7f15214c7640 1 -- 172.21.15.105:0/2938845029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f151c0cc760 msgr2=0x7f151c152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:47.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.029+0000 7f15214c7640 1 --2- 172.21.15.105:0/2938845029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f151c0cc760 0x7f151c152660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.029+0000 7f15214c7640 1 -- 172.21.15.105:0/2938845029 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1514009b30 con 0x7f151c159f40 2023-11-28T18:30:47.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.029+0000 7f15214c7640 1 --2- 172.21.15.105:0/2938845029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f151c159f40 0x7f151c0ac3f0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f151401afc0 tx=0x7f15140196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:47.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.029+0000 7f15197fa640 1 -- 172.21.15.105:0/2938845029 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15140b8670 con 0x7f151c159f40 2023-11-28T18:30:47.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.029+0000 7f15197fa640 1 -- 172.21.15.105:0/2938845029 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15140b87d0 con 0x7f151c159f40 2023-11-28T18:30:47.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.029+0000 7f15197fa640 1 -- 172.21.15.105:0/2938845029 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15140b8b80 con 0x7f151c159f40 2023-11-28T18:30:47.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.029+0000 7f1522f2a640 1 -- 172.21.15.105:0/2938845029 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151c0b3c60 con 0x7f151c159f40 2023-11-28T18:30:47.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.029+0000 7f1522f2a640 1 -- 172.21.15.105:0/2938845029 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151c0b4220 con 0x7f151c159f40 2023-11-28T18:30:47.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.029+0000 7f1522f2a640 1 -- 172.21.15.105:0/2938845029 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f151c0b1cf0 con 0x7f151c159f40 2023-11-28T18:30:47.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.029+0000 7f15197fa640 1 -- 172.21.15.105:0/2938845029 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f15140b8ce0 con 0x7f151c159f40 2023-11-28T18:30:47.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.033+0000 7f15197fa640 1 --2- 172.21.15.105:0/2938845029 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15000668d0 0x7f1500068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:47.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.033+0000 7f15197fa640 1 -- 172.21.15.105:0/2938845029 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f151411faf0 con 0x7f151c159f40 2023-11-28T18:30:47.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.033+0000 7f1520cc6640 1 --2- 172.21.15.105:0/2938845029 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15000668d0 0x7f1500068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:47.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.033+0000 7f1520cc6640 1 --2- 172.21.15.105:0/2938845029 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15000668d0 0x7f1500068d90 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f151c1f96e0 tx=0x7f1510008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:47.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.037+0000 7f15197fa640 1 -- 172.21.15.105:0/2938845029 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f15140ecda0 con 0x7f151c159f40 2023-11-28T18:30:47.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.185+0000 7f1522f2a640 1 -- 172.21.15.105:0/2938845029 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f151c067570 con 0x7f151c159f40 2023-11-28T18:30:47.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.185+0000 7f15197fa640 1 -- 172.21.15.105:0/2938845029 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) v1 ==== 104+0+11 (secure 0 0 0) 0x7f15140bc020 con 0x7f151c159f40 2023-11-28T18:30:47.188 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:30:47.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.189+0000 7f1522f2a640 1 -- 172.21.15.105:0/2938845029 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15000668d0 msgr2=0x7f1500068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:47.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.189+0000 7f1522f2a640 1 --2- 172.21.15.105:0/2938845029 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15000668d0 0x7f1500068d90 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f151c1f96e0 tx=0x7f1510008040 comp rx=0 tx=0).stop 2023-11-28T18:30:47.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.189+0000 7f1522f2a640 1 -- 172.21.15.105:0/2938845029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f151c159f40 msgr2=0x7f151c0ac3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:47.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.189+0000 7f1522f2a640 1 --2- 172.21.15.105:0/2938845029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f151c159f40 0x7f151c0ac3f0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f151401afc0 tx=0x7f15140196b0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.189+0000 7f1522f2a640 1 -- 172.21.15.105:0/2938845029 shutdown_connections 2023-11-28T18:30:47.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.189+0000 7f1522f2a640 1 --2- 172.21.15.105:0/2938845029 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15000668d0 0x7f1500068d90 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.189+0000 7f1522f2a640 1 --2- 172.21.15.105:0/2938845029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f151c159f40 0x7f151c0ac3f0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.189+0000 7f1522f2a640 1 --2- 172.21.15.105:0/2938845029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f151c0cc760 0x7f151c152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.189+0000 7f1522f2a640 1 --2- 172.21.15.105:0/2938845029 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f151c0ad680 0x7f151c154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.189+0000 7f1522f2a640 1 -- 172.21.15.105:0/2938845029 >> 172.21.15.105:0/2938845029 conn(0x7f151c0ceb90 msgr2=0x7f151c0ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:47.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.189+0000 7f1522f2a640 1 -- 172.21.15.105:0/2938845029 shutdown_connections 2023-11-28T18:30:47.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.189+0000 7f1522f2a640 1 -- 172.21.15.105:0/2938845029 wait complete. 2023-11-28T18:30:47.208 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:30:47.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.349+0000 7f7f4dc32640 1 Processor -- start 2023-11-28T18:30:47.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.349+0000 7f7f4dc32640 1 -- start start 2023-11-28T18:30:47.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.349+0000 7f7f4dc32640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7f4815d4f0 0x7f7f480d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:47.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.349+0000 7f7f4dc32640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f480b25b0 con 0x7f7f480b2d20 2023-11-28T18:30:47.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.349+0000 7f7f4dc32640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f480cea30 con 0x7f7f480d3c30 2023-11-28T18:30:47.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.349+0000 7f7f4dc32640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f480b2900 con 0x7f7f4815d4f0 2023-11-28T18:30:47.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.349+0000 7f7f47fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7f480b2d20 0x7f7f480b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33714/0 (socket says 172.21.15.105:33714) 2023-11-28T18:30:47.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.349+0000 7f7f47fff640 1 -- 172.21.15.105:0/896532467 learned_addr learned my addr 172.21.15.105:0/896532467 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:47.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.349+0000 7f7f477fe640 1 --2- 172.21.15.105:0/896532467 >> v2:172.21.15.148:3300/0 conn(0x7f7f4815d4f0 0x7f7f480d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:47.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.349+0000 7f7f467fc640 1 -- 172.21.15.105:0/896532467 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3438509898 0 0) 0x7f7f480b25b0 con 0x7f7f480b2d20 2023-11-28T18:30:47.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.349+0000 7f7f467fc640 1 -- 172.21.15.105:0/896532467 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f30003530 con 0x7f7f480b2d20 2023-11-28T18:30:47.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.349+0000 7f7f477fe640 1 -- 172.21.15.105:0/896532467 >> v1:172.21.15.148:6789/0 conn(0x7f7f480d3c30 legacy=0x7f7f480d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:47.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.349+0000 7f7f477fe640 1 -- 172.21.15.105:0/896532467 >> v1:172.21.15.105:6789/0 conn(0x7f7f480b2d20 legacy=0x7f7f480b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:47.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.349+0000 7f7f467fc640 1 -- 172.21.15.105:0/896532467 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4070478771 0 0) 0x7f7f30003530 con 0x7f7f480b2d20 2023-11-28T18:30:47.352 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.349+0000 7f7f477fe640 1 -- 172.21.15.105:0/896532467 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f480ced10 con 0x7f7f4815d4f0 2023-11-28T18:30:47.353 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.353+0000 7f7f477fe640 1 --2- 172.21.15.105:0/896532467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f4815d4f0 0x7f7f480d0e10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7f380097c0 tx=0x7f7f3801c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1add450d7e4c3b9c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:47.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.353+0000 7f7f467fc640 1 -- 172.21.15.105:0/896532467 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f40001a00 con 0x7f7f4815d4f0 2023-11-28T18:30:47.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.353+0000 7f7f467fc640 1 -- 172.21.15.105:0/896532467 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f380a7050 con 0x7f7f4815d4f0 2023-11-28T18:30:47.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.353+0000 7f7f467fc640 1 -- 172.21.15.105:0/896532467 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f380ab600 con 0x7f7f4815d4f0 2023-11-28T18:30:47.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.353+0000 7f7f4dc32640 1 -- 172.21.15.105:0/896532467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f4815d4f0 msgr2=0x7f7f480d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:47.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.353+0000 7f7f4dc32640 1 --2- 172.21.15.105:0/896532467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f4815d4f0 0x7f7f480d0e10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7f380097c0 tx=0x7f7f3801c2e0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.353+0000 7f7f4dc32640 1 -- 172.21.15.105:0/896532467 shutdown_connections 2023-11-28T18:30:47.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.353+0000 7f7f4dc32640 1 --2- 172.21.15.105:0/896532467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f4815d4f0 0x7f7f480d0e10 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.353+0000 7f7f4dc32640 1 -- 172.21.15.105:0/896532467 >> 172.21.15.105:0/896532467 conn(0x7f7f480d3030 msgr2=0x7f7f480d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:47.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.353+0000 7f7f4dc32640 1 -- 172.21.15.105:0/896532467 shutdown_connections 2023-11-28T18:30:47.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.353+0000 7f7f4dc32640 1 -- 172.21.15.105:0/896532467 wait complete. 2023-11-28T18:30:47.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f4dc32640 1 Processor -- start 2023-11-28T18:30:47.356 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f4dc32640 1 -- start start 2023-11-28T18:30:47.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f4dc32640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f480b2d20 0x7f7f4815dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:47.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f4dc32640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7f480d3c30 0x7f7f4814b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:47.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f4dc32640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f4815d4f0 0x7f7f4806c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:47.359 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f4dc32640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f480d0a20 con 0x7f7f480b2d20 2023-11-28T18:30:47.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f4dc32640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4815e2b0 con 0x7f7f4815d4f0 2023-11-28T18:30:47.361 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f4dc32640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f480cb010 con 0x7f7f480d3c30 2023-11-28T18:30:47.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f477fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f480b2d20 0x7f7f4815dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:47.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f477fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f480b2d20 0x7f7f4815dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35228/0 (socket says 172.21.15.105:35228) 2023-11-28T18:30:47.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f477fe640 1 -- 172.21.15.105:0/3496902885 learned_addr learned my addr 172.21.15.105:0/3496902885 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:47.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f47fff640 1 --2- 172.21.15.105:0/3496902885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f4815d4f0 0x7f7f4806c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:47.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f46ffd640 1 --2- 172.21.15.105:0/3496902885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7f480d3c30 0x7f7f4814b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:47.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f477fe640 1 -- 172.21.15.105:0/3496902885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7f480d3c30 msgr2=0x7f7f4814b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:47.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f477fe640 1 --2- 172.21.15.105:0/3496902885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7f480d3c30 0x7f7f4814b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f477fe640 1 -- 172.21.15.105:0/3496902885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f4815d4f0 msgr2=0x7f7f4806c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:47.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f477fe640 1 --2- 172.21.15.105:0/3496902885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f4815d4f0 0x7f7f4806c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f477fe640 1 -- 172.21.15.105:0/3496902885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f38009420 con 0x7f7f480b2d20 2023-11-28T18:30:47.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f47fff640 1 --2- 172.21.15.105:0/3496902885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f4815d4f0 0x7f7f4806c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:47.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f477fe640 1 --2- 172.21.15.105:0/3496902885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f480b2d20 0x7f7f4815dd70 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f7f380097c0 tx=0x7f7f38019080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:47.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f2ffff640 1 -- 172.21.15.105:0/3496902885 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38006e20 con 0x7f7f480b2d20 2023-11-28T18:30:47.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f2ffff640 1 -- 172.21.15.105:0/3496902885 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f380a4e90 con 0x7f7f480b2d20 2023-11-28T18:30:47.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f2ffff640 1 -- 172.21.15.105:0/3496902885 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f380b4600 con 0x7f7f480b2d20 2023-11-28T18:30:47.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f4dc32640 1 -- 172.21.15.105:0/3496902885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f480cace0 con 0x7f7f480b2d20 2023-11-28T18:30:47.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.357+0000 7f7f4dc32640 1 -- 172.21.15.105:0/3496902885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f4815e4e0 con 0x7f7f480b2d20 2023-11-28T18:30:47.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.361+0000 7f7f4dc32640 1 -- 172.21.15.105:0/3496902885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f480cab40 con 0x7f7f480b2d20 2023-11-28T18:30:47.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.361+0000 7f7f2ffff640 1 -- 172.21.15.105:0/3496902885 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f7f380b2040 con 0x7f7f480b2d20 2023-11-28T18:30:47.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.361+0000 7f7f2ffff640 1 --2- 172.21.15.105:0/3496902885 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7f34066900 0x7f7f34068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:47.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.361+0000 7f7f2ffff640 1 -- 172.21.15.105:0/3496902885 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7f3811ddb0 con 0x7f7f480b2d20 2023-11-28T18:30:47.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.361+0000 7f7f46ffd640 1 --2- 172.21.15.105:0/3496902885 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7f34066900 0x7f7f34068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:47.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.361+0000 7f7f46ffd640 1 --2- 172.21.15.105:0/3496902885 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7f34066900 0x7f7f34068dc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7f480b90a0 tx=0x7f7f3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:47.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.365+0000 7f7f2ffff640 1 -- 172.21.15.105:0/3496902885 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7f480cab40 con 0x7f7f480b2d20 2023-11-28T18:30:47.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.517+0000 7f7f4dc32640 1 -- 172.21.15.105:0/3496902885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7f480cab40 con 0x7f7f480b2d20 2023-11-28T18:30:47.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.517+0000 7f7f2ffff640 1 -- 172.21.15.105:0/3496902885 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7f480cab40 con 0x7f7f480b2d20 2023-11-28T18:30:47.516 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:30:47.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.517+0000 7f7f4dc32640 1 -- 172.21.15.105:0/3496902885 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7f34066900 msgr2=0x7f7f34068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:47.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.517+0000 7f7f4dc32640 1 --2- 172.21.15.105:0/3496902885 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7f34066900 0x7f7f34068dc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7f480b90a0 tx=0x7f7f3c009040 comp rx=0 tx=0).stop 2023-11-28T18:30:47.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.517+0000 7f7f4dc32640 1 -- 172.21.15.105:0/3496902885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f480b2d20 msgr2=0x7f7f4815dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:47.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.517+0000 7f7f4dc32640 1 --2- 172.21.15.105:0/3496902885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f480b2d20 0x7f7f4815dd70 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f7f380097c0 tx=0x7f7f38019080 comp rx=0 tx=0).stop 2023-11-28T18:30:47.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.517+0000 7f7f4dc32640 1 -- 172.21.15.105:0/3496902885 shutdown_connections 2023-11-28T18:30:47.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.517+0000 7f7f4dc32640 1 --2- 172.21.15.105:0/3496902885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f4815d4f0 0x7f7f4806c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.517+0000 7f7f4dc32640 1 --2- 172.21.15.105:0/3496902885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7f480d3c30 0x7f7f4814b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.517+0000 7f7f4dc32640 1 --2- 172.21.15.105:0/3496902885 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7f34066900 0x7f7f34068dc0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.517+0000 7f7f4dc32640 1 --2- 172.21.15.105:0/3496902885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f480b2d20 0x7f7f4815dd70 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.517+0000 7f7f4dc32640 1 -- 172.21.15.105:0/3496902885 >> 172.21.15.105:0/3496902885 conn(0x7f7f480d3030 msgr2=0x7f7f480affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:47.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.517+0000 7f7f4dc32640 1 -- 172.21.15.105:0/3496902885 shutdown_connections 2023-11-28T18:30:47.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.517+0000 7f7f4dc32640 1 -- 172.21.15.105:0/3496902885 wait complete. 2023-11-28T18:30:47.538 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:30:47.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.677+0000 7fb5ce365640 1 Processor -- start 2023-11-28T18:30:47.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.677+0000 7fb5ce365640 1 -- start start 2023-11-28T18:30:47.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.677+0000 7fb5ce365640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb5c80caf60 0x7fb5c80cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:47.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.677+0000 7fb5ce365640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5c80ad850 0x7fb5c80701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:47.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.677+0000 7fb5ce365640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c806bf50 con 0x7fb5c8159f40 2023-11-28T18:30:47.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.677+0000 7fb5ce365640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c80cb3c0 con 0x7fb5c80caf60 2023-11-28T18:30:47.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.677+0000 7fb5ce365640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c815a320 con 0x7fb5c80ad850 2023-11-28T18:30:47.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.677+0000 7fb5c77fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb5c80caf60 0x7fb5c80cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:47.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.677+0000 7fb5c77fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb5c80caf60 0x7fb5c80cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35244/0 (socket says 172.21.15.105:35244) 2023-11-28T18:30:47.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.677+0000 7fb5c77fe640 1 -- 172.21.15.105:0/1591322729 learned_addr learned my addr 172.21.15.105:0/1591322729 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:47.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.677+0000 7fb5c7fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb5c8159f40 0x7fb5c80ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33718/0 (socket says 172.21.15.105:33718) 2023-11-28T18:30:47.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.677+0000 7fb5cc902640 1 --2- 172.21.15.105:0/1591322729 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5c80ad850 0x7fb5c80701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:47.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.677+0000 7fb5c77fe640 1 -- 172.21.15.105:0/1591322729 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5c80ad850 msgr2=0x7fb5c80701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:47.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.677+0000 7fb5c77fe640 1 --2- 172.21.15.105:0/1591322729 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5c80ad850 0x7fb5c80701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.677+0000 7fb5c77fe640 1 -- 172.21.15.105:0/1591322729 >> v1:172.21.15.105:6789/0 conn(0x7fb5c8159f40 legacy=0x7fb5c80ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:47.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.677+0000 7fb5c77fe640 1 -- 172.21.15.105:0/1591322729 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c8071380 con 0x7fb5c80caf60 2023-11-28T18:30:47.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.681+0000 7fb5c77fe640 1 --2- 172.21.15.105:0/1591322729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5c80caf60 0x7fb5c80cc760 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009870 tx=0x7fb5bc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c930ca2a233741 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:47.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.681+0000 7fb5c6ffd640 1 -- 172.21.15.105:0/1591322729 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc0ab680 con 0x7fb5c80caf60 2023-11-28T18:30:47.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.681+0000 7fb5c6ffd640 1 -- 172.21.15.105:0/1591322729 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc0ab7e0 con 0x7fb5c80caf60 2023-11-28T18:30:47.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.681+0000 7fb5c6ffd640 1 -- 172.21.15.105:0/1591322729 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc0abb70 con 0x7fb5c80caf60 2023-11-28T18:30:47.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.681+0000 7fb5ce365640 1 -- 172.21.15.105:0/1591322729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5c80caf60 msgr2=0x7fb5c80cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:47.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.681+0000 7fb5ce365640 1 --2- 172.21.15.105:0/1591322729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5c80caf60 0x7fb5c80cc760 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009870 tx=0x7fb5bc01c480 comp rx=0 tx=0).stop 2023-11-28T18:30:47.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.681+0000 7fb5ce365640 1 -- 172.21.15.105:0/1591322729 shutdown_connections 2023-11-28T18:30:47.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.681+0000 7fb5ce365640 1 --2- 172.21.15.105:0/1591322729 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5c80ad850 0x7fb5c80701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.681+0000 7fb5ce365640 1 --2- 172.21.15.105:0/1591322729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5c80caf60 0x7fb5c80cc760 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.681+0000 7fb5ce365640 1 -- 172.21.15.105:0/1591322729 >> 172.21.15.105:0/1591322729 conn(0x7fb5c80ceb90 msgr2=0x7fb5c80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:47.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.681+0000 7fb5ce365640 1 -- 172.21.15.105:0/1591322729 shutdown_connections 2023-11-28T18:30:47.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.681+0000 7fb5ce365640 1 -- 172.21.15.105:0/1591322729 wait complete. 2023-11-28T18:30:47.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5ce365640 1 Processor -- start 2023-11-28T18:30:47.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5ce365640 1 -- start start 2023-11-28T18:30:47.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5ce365640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5c80ad850 0x7fb5c813f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:47.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5ce365640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5c80caf60 0x7fb5c8163950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:47.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5ce365640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5c8159f40 0x7fb5c81624a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:47.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5ce365640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c8164810 con 0x7fb5c80caf60 2023-11-28T18:30:47.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5ce365640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c8164250 con 0x7fb5c8159f40 2023-11-28T18:30:47.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5ce365640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c8164530 con 0x7fb5c80ad850 2023-11-28T18:30:47.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5c77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5c80caf60 0x7fb5c8163950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:47.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5c77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5c80caf60 0x7fb5c8163950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35260/0 (socket says 172.21.15.105:35260) 2023-11-28T18:30:47.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5c77fe640 1 -- 172.21.15.105:0/3354846878 learned_addr learned my addr 172.21.15.105:0/3354846878 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:47.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5c7fff640 1 --2- 172.21.15.105:0/3354846878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5c80ad850 0x7fb5c813f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:47.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5cc902640 1 --2- 172.21.15.105:0/3354846878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5c8159f40 0x7fb5c81624a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:47.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5c77fe640 1 -- 172.21.15.105:0/3354846878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5c80ad850 msgr2=0x7fb5c813f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:47.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5c77fe640 1 --2- 172.21.15.105:0/3354846878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5c80ad850 0x7fb5c813f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5c77fe640 1 -- 172.21.15.105:0/3354846878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5c8159f40 msgr2=0x7fb5c81624a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:47.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5c77fe640 1 --2- 172.21.15.105:0/3354846878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5c8159f40 0x7fb5c81624a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5c77fe640 1 -- 172.21.15.105:0/3354846878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5bc0094d0 con 0x7fb5c80caf60 2023-11-28T18:30:47.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5c77fe640 1 --2- 172.21.15.105:0/3354846878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5c80caf60 0x7fb5c8163950 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc01a0b0 tx=0x7fb5bc01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:47.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5c4ff9640 1 -- 172.21.15.105:0/3354846878 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc0b4800 con 0x7fb5c80caf60 2023-11-28T18:30:47.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5c4ff9640 1 -- 172.21.15.105:0/3354846878 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc0b4960 con 0x7fb5c80caf60 2023-11-28T18:30:47.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5ce365640 1 -- 172.21.15.105:0/3354846878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c8161ca0 con 0x7fb5c80caf60 2023-11-28T18:30:47.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5c4ff9640 1 -- 172.21.15.105:0/3354846878 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc0b4d10 con 0x7fb5c80caf60 2023-11-28T18:30:47.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5ce365640 1 -- 172.21.15.105:0/3354846878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c8163e90 con 0x7fb5c80caf60 2023-11-28T18:30:47.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.685+0000 7fb5ce365640 1 -- 172.21.15.105:0/3354846878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c8067570 con 0x7fb5c80caf60 2023-11-28T18:30:47.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.689+0000 7fb5c4ff9640 1 -- 172.21.15.105:0/3354846878 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fb5bc0c7390 con 0x7fb5c80caf60 2023-11-28T18:30:47.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.689+0000 7fb5c4ff9640 1 --2- 172.21.15.105:0/3354846878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb5b40668d0 0x7fb5b4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:47.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.689+0000 7fb5c4ff9640 1 -- 172.21.15.105:0/3354846878 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb5bc11f400 con 0x7fb5c80caf60 2023-11-28T18:30:47.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.689+0000 7fb5c7fff640 1 --2- 172.21.15.105:0/3354846878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb5b40668d0 0x7fb5b4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:47.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.689+0000 7fb5c7fff640 1 --2- 172.21.15.105:0/3354846878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb5b40668d0 0x7fb5b4068d90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8000f70 tx=0x7fb5b8007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:47.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.693+0000 7fb5c4ff9640 1 -- 172.21.15.105:0/3354846878 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb5bc0ec440 con 0x7fb5c80caf60 2023-11-28T18:30:47.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.869+0000 7fb5ce365640 1 -- 172.21.15.105:0/3354846878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5c8164670 con 0x7fb5c80caf60 2023-11-28T18:30:47.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.873+0000 7fb5c4ff9640 1 -- 172.21.15.105:0/3354846878 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4963 (secure 0 0 0) 0x7fb5c8164670 con 0x7fb5c80caf60 2023-11-28T18:30:47.871 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:47.871 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":111,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/2603493310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2603493310},{"type":"v1","addr":"172.21.15.148:6834","nonce":2603493310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:30:05.525117+0000","modified":"2023-11-28T18:30:45.830630+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8145,"mds_1":8142,"mds_2":6803},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6803":{"gid":6803,"name":"d","rank":2,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6835/3664281230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":3664281230},{"type":"v1","addr":"172.21.15.148:6835","nonce":3664281230}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8142":{"gid":8142,"name":"a","rank":1,"incarnation":105,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6836/3968059262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3968059262},{"type":"v1","addr":"172.21.15.105:6836","nonce":3968059262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8145":{"gid":8145,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2782729992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2782729992},{"type":"v1","addr":"172.21.15.105:6837","nonce":2782729992}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-11-28T18:30:47.872 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 111 2023-11-28T18:30:47.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.873+0000 7fb5ce365640 1 -- 172.21.15.105:0/3354846878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb5b40668d0 msgr2=0x7fb5b4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:47.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.873+0000 7fb5ce365640 1 --2- 172.21.15.105:0/3354846878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb5b40668d0 0x7fb5b4068d90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8000f70 tx=0x7fb5b8007c90 comp rx=0 tx=0).stop 2023-11-28T18:30:47.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.873+0000 7fb5ce365640 1 -- 172.21.15.105:0/3354846878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5c80caf60 msgr2=0x7fb5c8163950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:47.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.873+0000 7fb5ce365640 1 --2- 172.21.15.105:0/3354846878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5c80caf60 0x7fb5c8163950 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc01a0b0 tx=0x7fb5bc01acc0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.873+0000 7fb5ce365640 1 -- 172.21.15.105:0/3354846878 shutdown_connections 2023-11-28T18:30:47.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.873+0000 7fb5ce365640 1 --2- 172.21.15.105:0/3354846878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5c8159f40 0x7fb5c81624a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.873+0000 7fb5ce365640 1 --2- 172.21.15.105:0/3354846878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb5b40668d0 0x7fb5b4068d90 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.873+0000 7fb5ce365640 1 --2- 172.21.15.105:0/3354846878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5c80caf60 0x7fb5c8163950 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.873+0000 7fb5ce365640 1 --2- 172.21.15.105:0/3354846878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5c80ad850 0x7fb5c813f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:47.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.873+0000 7fb5ce365640 1 -- 172.21.15.105:0/3354846878 >> 172.21.15.105:0/3354846878 conn(0x7fb5c80ceb90 msgr2=0x7fb5c80cc710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:47.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.873+0000 7fb5ce365640 1 -- 172.21.15.105:0/3354846878 shutdown_connections 2023-11-28T18:30:47.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:47.873+0000 7fb5ce365640 1 -- 172.21.15.105:0/3354846878 wait complete. 2023-11-28T18:30:47.893 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:30:48.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.029+0000 7f966bc6b640 1 Processor -- start 2023-11-28T18:30:48.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.029+0000 7f966bc6b640 1 -- start start 2023-11-28T18:30:48.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.033+0000 7f966bc6b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f96640b4290 0x7f96640b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:48.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.033+0000 7f966bc6b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9664071960 0x7f966415f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:48.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.033+0000 7f966bc6b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f966415b030 0x7f96640709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:48.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.033+0000 7f966bc6b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96640b4bb0 con 0x7f96640b4290 2023-11-28T18:30:48.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.033+0000 7f966bc6b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96640701d0 con 0x7f966415b030 2023-11-28T18:30:48.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.033+0000 7f966bc6b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f966406df80 con 0x7f9664071960 2023-11-28T18:30:48.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.033+0000 7f9669a07640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f96640b4290 0x7f96640b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:48.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.033+0000 7f9669a07640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f96640b4290 0x7f96640b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35264/0 (socket says 172.21.15.105:35264) 2023-11-28T18:30:48.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.033+0000 7f9669a07640 1 -- 172.21.15.105:0/2203195903 learned_addr learned my addr 172.21.15.105:0/2203195903 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:48.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.033+0000 7f9669206640 1 --2- 172.21.15.105:0/2203195903 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9664071960 0x7f966415f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:48.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.033+0000 7f966a208640 1 --2- 172.21.15.105:0/2203195903 >> v2:172.21.15.148:3300/0 conn(0x7f966415b030 0x7f96640709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:48.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.033+0000 7f9669a07640 1 -- 172.21.15.105:0/2203195903 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9664071960 msgr2=0x7f966415f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:48.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.033+0000 7f9669a07640 1 --2- 172.21.15.105:0/2203195903 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9664071960 0x7f966415f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.033+0000 7f9669a07640 1 -- 172.21.15.105:0/2203195903 >> v2:172.21.15.148:3300/0 conn(0x7f966415b030 msgr2=0x7f96640709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:48.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.033+0000 7f9669a07640 1 --2- 172.21.15.105:0/2203195903 >> v2:172.21.15.148:3300/0 conn(0x7f966415b030 0x7f96640709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.033+0000 7f9669a07640 1 -- 172.21.15.105:0/2203195903 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9664071df0 con 0x7f96640b4290 2023-11-28T18:30:48.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.033+0000 7f966a208640 1 --2- 172.21.15.105:0/2203195903 >> v2:172.21.15.148:3300/0 conn(0x7f966415b030 0x7f96640709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:48.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.033+0000 7f9669a07640 1 --2- 172.21.15.105:0/2203195903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96640b4290 0x7f96640b4670 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f966406bc50 tx=0x7f966001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa039d181e73cbd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:48.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.037+0000 7f9668a05640 1 -- 172.21.15.105:0/2203195903 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96600ac600 con 0x7f96640b4290 2023-11-28T18:30:48.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.037+0000 7f9668a05640 1 -- 172.21.15.105:0/2203195903 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96600a8050 con 0x7f96640b4290 2023-11-28T18:30:48.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.037+0000 7f9668a05640 1 -- 172.21.15.105:0/2203195903 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96600ac980 con 0x7f96640b4290 2023-11-28T18:30:48.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.037+0000 7f966bc6b640 1 -- 172.21.15.105:0/2203195903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96640b4290 msgr2=0x7f96640b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:48.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.037+0000 7f966bc6b640 1 --2- 172.21.15.105:0/2203195903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96640b4290 0x7f96640b4670 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f966406bc50 tx=0x7f966001c4d0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.037+0000 7f966bc6b640 1 -- 172.21.15.105:0/2203195903 shutdown_connections 2023-11-28T18:30:48.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.037+0000 7f966bc6b640 1 --2- 172.21.15.105:0/2203195903 >> v2:172.21.15.148:3300/0 conn(0x7f966415b030 0x7f96640709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.037+0000 7f966bc6b640 1 --2- 172.21.15.105:0/2203195903 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9664071960 0x7f966415f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.037+0000 7f966bc6b640 1 --2- 172.21.15.105:0/2203195903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96640b4290 0x7f96640b4670 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.037+0000 7f966bc6b640 1 -- 172.21.15.105:0/2203195903 >> 172.21.15.105:0/2203195903 conn(0x7f96640ceb90 msgr2=0x7f96640cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:48.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.037+0000 7f966bc6b640 1 -- 172.21.15.105:0/2203195903 shutdown_connections 2023-11-28T18:30:48.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.037+0000 7f966bc6b640 1 -- 172.21.15.105:0/2203195903 wait complete. 2023-11-28T18:30:48.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.037+0000 7f966bc6b640 1 Processor -- start 2023-11-28T18:30:48.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.037+0000 7f966bc6b640 1 -- start start 2023-11-28T18:30:48.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.037+0000 7f966bc6b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96640b4290 0x7f96640cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:48.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.037+0000 7f966bc6b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f966415b030 0x7f9664149fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:48.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.037+0000 7f966bc6b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f966414a810 0x7f9664159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:48.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.037+0000 7f966bc6b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96640ce7d0 con 0x7f966414a810 2023-11-28T18:30:48.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.037+0000 7f966bc6b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966409bf80 con 0x7f96640b4290 2023-11-28T18:30:48.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.037+0000 7f966bc6b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f966409b650 con 0x7f966415b030 2023-11-28T18:30:48.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.041+0000 7f966a208640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f966414a810 0x7f9664159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:48.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.041+0000 7f966a208640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f966414a810 0x7f9664159330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35266/0 (socket says 172.21.15.105:35266) 2023-11-28T18:30:48.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.041+0000 7f966a208640 1 -- 172.21.15.105:0/1789785091 learned_addr learned my addr 172.21.15.105:0/1789785091 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:48.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.041+0000 7f9669a07640 1 --2- 172.21.15.105:0/1789785091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96640b4290 0x7f96640cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:48.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.041+0000 7f9669206640 1 --2- 172.21.15.105:0/1789785091 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f966415b030 0x7f9664149fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:48.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.041+0000 7f966a208640 1 -- 172.21.15.105:0/1789785091 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f966415b030 msgr2=0x7f9664149fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:48.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.041+0000 7f966a208640 1 --2- 172.21.15.105:0/1789785091 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f966415b030 0x7f9664149fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.041+0000 7f966a208640 1 -- 172.21.15.105:0/1789785091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96640b4290 msgr2=0x7f96640cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:48.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.041+0000 7f966a208640 1 --2- 172.21.15.105:0/1789785091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96640b4290 0x7f96640cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.041+0000 7f966a208640 1 -- 172.21.15.105:0/1789785091 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96600094d0 con 0x7f966414a810 2023-11-28T18:30:48.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.041+0000 7f966a208640 1 --2- 172.21.15.105:0/1789785091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f966414a810 0x7f9664159330 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f965c007e40 tx=0x7f965c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:48.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.041+0000 7f965a7fc640 1 -- 172.21.15.105:0/1789785091 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c003ba0 con 0x7f966414a810 2023-11-28T18:30:48.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.041+0000 7f965a7fc640 1 -- 172.21.15.105:0/1789785091 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965c00f040 con 0x7f966414a810 2023-11-28T18:30:48.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.041+0000 7f966bc6b640 1 -- 172.21.15.105:0/1789785091 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f966409b930 con 0x7f966414a810 2023-11-28T18:30:48.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.041+0000 7f965a7fc640 1 -- 172.21.15.105:0/1789785091 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c00de20 con 0x7f966414a810 2023-11-28T18:30:48.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.041+0000 7f966bc6b640 1 -- 172.21.15.105:0/1789785091 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96640ce4c0 con 0x7f966414a810 2023-11-28T18:30:48.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.041+0000 7f966bc6b640 1 -- 172.21.15.105:0/1789785091 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9664067570 con 0x7f966414a810 2023-11-28T18:30:48.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.041+0000 7f965a7fc640 1 -- 172.21.15.105:0/1789785091 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f965c004030 con 0x7f966414a810 2023-11-28T18:30:48.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.041+0000 7f965a7fc640 1 --2- 172.21.15.105:0/1789785091 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f96480668d0 0x7f9648068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:48.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.041+0000 7f965a7fc640 1 -- 172.21.15.105:0/1789785091 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f965c084840 con 0x7f966414a810 2023-11-28T18:30:48.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.045+0000 7f9669a07640 1 --2- 172.21.15.105:0/1789785091 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f96480668d0 0x7f9648068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:48.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.045+0000 7f9669a07640 1 --2- 172.21.15.105:0/1789785091 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f96480668d0 0x7f9648068d90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f96600ae000 tx=0x7f96600a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:48.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.045+0000 7f965a7fc640 1 -- 172.21.15.105:0/1789785091 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f965c051930 con 0x7f966414a810 2023-11-28T18:30:48.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.201+0000 7f966bc6b640 1 -- 172.21.15.105:0/1789785091 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f966409c0c0 con 0x7f966414a810 2023-11-28T18:30:48.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.201+0000 7f965a7fc640 1 -- 172.21.15.105:0/1789785091 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16826 (secure 0 0 0) 0x7f965c054f50 con 0x7f966414a810 2023-11-28T18:30:48.200 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:48.201 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":105,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:30:16.894705+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:30:03.341424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-28T18:30:04.527931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1b","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:30:48.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.201+0000 7f966bc6b640 1 -- 172.21.15.105:0/1789785091 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f96480668d0 msgr2=0x7f9648068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:48.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.201+0000 7f966bc6b640 1 --2- 172.21.15.105:0/1789785091 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f96480668d0 0x7f9648068d90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f96600ae000 tx=0x7f96600a7000 comp rx=0 tx=0).stop 2023-11-28T18:30:48.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.201+0000 7f966bc6b640 1 -- 172.21.15.105:0/1789785091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f966414a810 msgr2=0x7f9664159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:48.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.201+0000 7f966bc6b640 1 --2- 172.21.15.105:0/1789785091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f966414a810 0x7f9664159330 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f965c007e40 tx=0x7f965c00d4a0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.201+0000 7f966bc6b640 1 -- 172.21.15.105:0/1789785091 shutdown_connections 2023-11-28T18:30:48.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.201+0000 7f966bc6b640 1 --2- 172.21.15.105:0/1789785091 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f96480668d0 0x7f9648068d90 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.201+0000 7f966bc6b640 1 --2- 172.21.15.105:0/1789785091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f966414a810 0x7f9664159330 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.201+0000 7f966bc6b640 1 --2- 172.21.15.105:0/1789785091 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f966415b030 0x7f9664149fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.201+0000 7f966bc6b640 1 --2- 172.21.15.105:0/1789785091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96640b4290 0x7f96640cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.201+0000 7f966bc6b640 1 -- 172.21.15.105:0/1789785091 >> 172.21.15.105:0/1789785091 conn(0x7f96640ceb90 msgr2=0x7f96640b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:48.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.201+0000 7f966bc6b640 1 -- 172.21.15.105:0/1789785091 shutdown_connections 2023-11-28T18:30:48.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.201+0000 7f966bc6b640 1 -- 172.21.15.105:0/1789785091 wait complete. 2023-11-28T18:30:48.222 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T18:30:48.223 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:30:48.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.361+0000 7f6d4f6c9640 1 Processor -- start 2023-11-28T18:30:48.362 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.361+0000 7f6d4f6c9640 1 -- start start 2023-11-28T18:30:48.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.361+0000 7f6d4f6c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d4806d610 0x7f6d480cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:48.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.361+0000 7f6d4f6c9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6d480ce530 0x7f6d480ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:48.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.361+0000 7f6d4f6c9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d480ae380 con 0x7f6d480adf30 2023-11-28T18:30:48.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.361+0000 7f6d4f6c9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d48162820 con 0x7f6d480ce530 2023-11-28T18:30:48.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.361+0000 7f6d4f6c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d48161c10 con 0x7f6d4806d610 2023-11-28T18:30:48.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.365+0000 7f6d4d465640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d4806d610 0x7f6d480cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:48.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.365+0000 7f6d4d465640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d4806d610 0x7f6d480cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:38904/0 (socket says 172.21.15.105:38904) 2023-11-28T18:30:48.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.365+0000 7f6d4d465640 1 -- 172.21.15.105:0/1650758469 learned_addr learned my addr 172.21.15.105:0/1650758469 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:48.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.365+0000 7f6d4cc64640 1 --2- 172.21.15.105:0/1650758469 >> v2:172.21.15.148:3300/0 conn(0x7f6d480ce530 0x7f6d480ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:48.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.365+0000 7f6d4d465640 1 -- 172.21.15.105:0/1650758469 >> v2:172.21.15.148:3300/0 conn(0x7f6d480ce530 msgr2=0x7f6d480ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:48.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.365+0000 7f6d4d465640 1 --2- 172.21.15.105:0/1650758469 >> v2:172.21.15.148:3300/0 conn(0x7f6d480ce530 0x7f6d480ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.365+0000 7f6d4d465640 1 -- 172.21.15.105:0/1650758469 >> v1:172.21.15.148:6789/0 conn(0x7f6d480adf30 legacy=0x7f6d481650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:48.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.365+0000 7f6d4d465640 1 -- 172.21.15.105:0/1650758469 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d48161ef0 con 0x7f6d4806d610 2023-11-28T18:30:48.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.365+0000 7f6d4d465640 1 --2- 172.21.15.105:0/1650758469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d4806d610 0x7f6d480cdff0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6d44009870 tx=0x7f6d4401c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c0fcf6ff3ba0de0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:48.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.369+0000 7f6d3ffff640 1 -- 172.21.15.105:0/1650758469 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d440a7070 con 0x7f6d4806d610 2023-11-28T18:30:48.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.369+0000 7f6d3ffff640 1 -- 172.21.15.105:0/1650758469 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d440ab430 con 0x7f6d4806d610 2023-11-28T18:30:48.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.369+0000 7f6d3ffff640 1 -- 172.21.15.105:0/1650758469 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d440ab7c0 con 0x7f6d4806d610 2023-11-28T18:30:48.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.369+0000 7f6d4f6c9640 1 -- 172.21.15.105:0/1650758469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d4806d610 msgr2=0x7f6d480cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:48.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.369+0000 7f6d4f6c9640 1 --2- 172.21.15.105:0/1650758469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d4806d610 0x7f6d480cdff0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6d44009870 tx=0x7f6d4401c510 comp rx=0 tx=0).stop 2023-11-28T18:30:48.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.369+0000 7f6d4f6c9640 1 -- 172.21.15.105:0/1650758469 shutdown_connections 2023-11-28T18:30:48.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.369+0000 7f6d4f6c9640 1 --2- 172.21.15.105:0/1650758469 >> v2:172.21.15.148:3300/0 conn(0x7f6d480ce530 0x7f6d480ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.369+0000 7f6d4f6c9640 1 --2- 172.21.15.105:0/1650758469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d4806d610 0x7f6d480cdff0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.369+0000 7f6d4f6c9640 1 -- 172.21.15.105:0/1650758469 >> 172.21.15.105:0/1650758469 conn(0x7f6d480ceb90 msgr2=0x7f6d480cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:48.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.369+0000 7f6d4f6c9640 1 -- 172.21.15.105:0/1650758469 shutdown_connections 2023-11-28T18:30:48.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.369+0000 7f6d4f6c9640 1 -- 172.21.15.105:0/1650758469 wait complete. 2023-11-28T18:30:48.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.369+0000 7f6d4f6c9640 1 Processor -- start 2023-11-28T18:30:48.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.369+0000 7f6d4f6c9640 1 -- start start 2023-11-28T18:30:48.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.369+0000 7f6d4f6c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d4806d610 0x7f6d48151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:48.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.369+0000 7f6d4f6c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d480adf30 0x7f6d480acf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:48.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.369+0000 7f6d4f6c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d480ce530 0x7f6d480ad490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:48.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.369+0000 7f6d4f6c9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d48164810 con 0x7f6d4806d610 2023-11-28T18:30:48.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.369+0000 7f6d4f6c9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d48152460 con 0x7f6d480adf30 2023-11-28T18:30:48.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.369+0000 7f6d4f6c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d48161220 con 0x7f6d480ce530 2023-11-28T18:30:48.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.369+0000 7f6d4d465640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d4806d610 0x7f6d48151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:48.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.373+0000 7f6d4d465640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d4806d610 0x7f6d48151c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35278/0 (socket says 172.21.15.105:35278) 2023-11-28T18:30:48.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.373+0000 7f6d4d465640 1 -- 172.21.15.105:0/3319338674 learned_addr learned my addr 172.21.15.105:0/3319338674 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:48.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.373+0000 7f6d4dc66640 1 --2- 172.21.15.105:0/3319338674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d480ce530 0x7f6d480ad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:48.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.373+0000 7f6d4cc64640 1 --2- 172.21.15.105:0/3319338674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d480adf30 0x7f6d480acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:48.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.373+0000 7f6d4d465640 1 -- 172.21.15.105:0/3319338674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d480ce530 msgr2=0x7f6d480ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:48.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.373+0000 7f6d4d465640 1 --2- 172.21.15.105:0/3319338674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d480ce530 0x7f6d480ad490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.373+0000 7f6d4d465640 1 -- 172.21.15.105:0/3319338674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d480adf30 msgr2=0x7f6d480acf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:48.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.373+0000 7f6d4d465640 1 --2- 172.21.15.105:0/3319338674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d480adf30 0x7f6d480acf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.373+0000 7f6d4d465640 1 -- 172.21.15.105:0/3319338674 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d440094d0 con 0x7f6d4806d610 2023-11-28T18:30:48.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.373+0000 7f6d4d465640 1 --2- 172.21.15.105:0/3319338674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d4806d610 0x7f6d48151c90 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f6d440ad040 tx=0x7f6d440192b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:48.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.373+0000 7f6d3dffb640 1 -- 172.21.15.105:0/3319338674 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d440b4620 con 0x7f6d4806d610 2023-11-28T18:30:48.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.373+0000 7f6d3dffb640 1 -- 172.21.15.105:0/3319338674 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d440b4780 con 0x7f6d4806d610 2023-11-28T18:30:48.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.373+0000 7f6d4f6c9640 1 -- 172.21.15.105:0/3319338674 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d48164a60 con 0x7f6d4806d610 2023-11-28T18:30:48.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.373+0000 7f6d3dffb640 1 -- 172.21.15.105:0/3319338674 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d440b4b30 con 0x7f6d4806d610 2023-11-28T18:30:48.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.373+0000 7f6d4f6c9640 1 -- 172.21.15.105:0/3319338674 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d481521d0 con 0x7f6d4806d610 2023-11-28T18:30:48.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.373+0000 7f6d4f6c9640 1 -- 172.21.15.105:0/3319338674 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d481525d0 con 0x7f6d4806d610 2023-11-28T18:30:48.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.373+0000 7f6d3dffb640 1 -- 172.21.15.105:0/3319338674 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f6d440b4c90 con 0x7f6d4806d610 2023-11-28T18:30:48.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.373+0000 7f6d3dffb640 1 --2- 172.21.15.105:0/3319338674 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6d2c0668d0 0x7f6d2c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:48.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.373+0000 7f6d3dffb640 1 -- 172.21.15.105:0/3319338674 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6d4411ea90 con 0x7f6d4806d610 2023-11-28T18:30:48.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.377+0000 7f6d4cc64640 1 --2- 172.21.15.105:0/3319338674 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6d2c0668d0 0x7f6d2c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:48.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.377+0000 7f6d4cc64640 1 --2- 172.21.15.105:0/3319338674 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6d2c0668d0 0x7f6d2c068d90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6d4815ebb0 tx=0x7f6d38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:48.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.377+0000 7f6d3dffb640 1 -- 172.21.15.105:0/3319338674 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6d440ebc00 con 0x7f6d4806d610 2023-11-28T18:30:48.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.557+0000 7f6d4f6c9640 1 -- 172.21.15.105:0/3319338674 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d48067570 con 0x7f6d4806d610 2023-11-28T18:30:48.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.557+0000 7f6d3dffb640 1 -- 172.21.15.105:0/3319338674 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4963 (secure 0 0 0) 0x7f6d440c4020 con 0x7f6d4806d610 2023-11-28T18:30:48.559 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:48.559 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":111,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/2603493310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2603493310},{"type":"v1","addr":"172.21.15.148:6834","nonce":2603493310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:30:05.525117+0000","modified":"2023-11-28T18:30:45.830630+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8145,"mds_1":8142,"mds_2":6803},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6803":{"gid":6803,"name":"d","rank":2,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6835/3664281230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":3664281230},{"type":"v1","addr":"172.21.15.148:6835","nonce":3664281230}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8142":{"gid":8142,"name":"a","rank":1,"incarnation":105,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6836/3968059262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3968059262},{"type":"v1","addr":"172.21.15.105:6836","nonce":3968059262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8145":{"gid":8145,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2782729992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2782729992},{"type":"v1","addr":"172.21.15.105:6837","nonce":2782729992}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-11-28T18:30:48.559 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 111 2023-11-28T18:30:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.561+0000 7f6d4f6c9640 1 -- 172.21.15.105:0/3319338674 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6d2c0668d0 msgr2=0x7f6d2c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.561+0000 7f6d4f6c9640 1 --2- 172.21.15.105:0/3319338674 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6d2c0668d0 0x7f6d2c068d90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6d4815ebb0 tx=0x7f6d38008040 comp rx=0 tx=0).stop 2023-11-28T18:30:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.561+0000 7f6d4f6c9640 1 -- 172.21.15.105:0/3319338674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d4806d610 msgr2=0x7f6d48151c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.561+0000 7f6d4f6c9640 1 --2- 172.21.15.105:0/3319338674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d4806d610 0x7f6d48151c90 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f6d440ad040 tx=0x7f6d440192b0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.561+0000 7f6d4f6c9640 1 -- 172.21.15.105:0/3319338674 shutdown_connections 2023-11-28T18:30:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.561+0000 7f6d4f6c9640 1 --2- 172.21.15.105:0/3319338674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6d480ce530 0x7f6d480ad490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.561+0000 7f6d4f6c9640 1 --2- 172.21.15.105:0/3319338674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d480adf30 0x7f6d480acf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.561+0000 7f6d4f6c9640 1 --2- 172.21.15.105:0/3319338674 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6d2c0668d0 0x7f6d2c068d90 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.561+0000 7f6d4f6c9640 1 --2- 172.21.15.105:0/3319338674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d4806d610 0x7f6d48151c90 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.561+0000 7f6d4f6c9640 1 -- 172.21.15.105:0/3319338674 >> 172.21.15.105:0/3319338674 conn(0x7f6d480ceb90 msgr2=0x7f6d481650a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:48.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.561+0000 7f6d4f6c9640 1 -- 172.21.15.105:0/3319338674 shutdown_connections 2023-11-28T18:30:48.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.561+0000 7f6d4f6c9640 1 -- 172.21.15.105:0/3319338674 wait complete. 2023-11-28T18:30:48.580 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:30:48.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.717+0000 7f15d7cc9640 1 Processor -- start 2023-11-28T18:30:48.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.721+0000 7f15d7cc9640 1 -- start start 2023-11-28T18:30:48.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.721+0000 7f15d7cc9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15d00d3c30 0x7f15d00d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:48.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.721+0000 7f15d7cc9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d00b25b0 con 0x7f15d015d4f0 2023-11-28T18:30:48.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.721+0000 7f15d7cc9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d00cea30 con 0x7f15d00b2d20 2023-11-28T18:30:48.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.721+0000 7f15d7cc9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d00b2900 con 0x7f15d00d3c30 2023-11-28T18:30:48.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.721+0000 7f15d5a65640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f15d015d4f0 0x7f15d00d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33724/0 (socket says 172.21.15.105:33724) 2023-11-28T18:30:48.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.721+0000 7f15d5a65640 1 -- 172.21.15.105:0/492042595 learned_addr learned my addr 172.21.15.105:0/492042595 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:48.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.721+0000 7f15d5264640 1 --2- 172.21.15.105:0/492042595 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15d00d3c30 0x7f15d00d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:48.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.721+0000 7f15d4a63640 1 -- 172.21.15.105:0/492042595 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761619909 0 0) 0x7f15d00b25b0 con 0x7f15d015d4f0 2023-11-28T18:30:48.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.721+0000 7f15d4a63640 1 -- 172.21.15.105:0/492042595 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15bc003530 con 0x7f15d015d4f0 2023-11-28T18:30:48.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.721+0000 7f15d5264640 1 -- 172.21.15.105:0/492042595 >> v1:172.21.15.148:6789/0 conn(0x7f15d00b2d20 legacy=0x7f15d00b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:48.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.721+0000 7f15d5264640 1 -- 172.21.15.105:0/492042595 >> v1:172.21.15.105:6789/0 conn(0x7f15d015d4f0 legacy=0x7f15d00d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:48.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.721+0000 7f15d5264640 1 -- 172.21.15.105:0/492042595 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d00ced10 con 0x7f15d00d3c30 2023-11-28T18:30:48.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.725+0000 7f15d5264640 1 --2- 172.21.15.105:0/492042595 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15d00d3c30 0x7f15d00d1520 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f15c00097f0 tx=0x7f15c001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dcca2bd146ec5f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:48.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.725+0000 7f15d4a63640 1 -- 172.21.15.105:0/492042595 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c00ab600 con 0x7f15d00d3c30 2023-11-28T18:30:48.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.725+0000 7f15d4a63640 1 -- 172.21.15.105:0/492042595 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c00a7050 con 0x7f15d00d3c30 2023-11-28T18:30:48.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.725+0000 7f15d4a63640 1 -- 172.21.15.105:0/492042595 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c00ab980 con 0x7f15d00d3c30 2023-11-28T18:30:48.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.725+0000 7f15d7cc9640 1 -- 172.21.15.105:0/492042595 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15d00d3c30 msgr2=0x7f15d00d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:48.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.725+0000 7f15d7cc9640 1 --2- 172.21.15.105:0/492042595 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15d00d3c30 0x7f15d00d1520 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f15c00097f0 tx=0x7f15c001c400 comp rx=0 tx=0).stop 2023-11-28T18:30:48.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.725+0000 7f15d7cc9640 1 -- 172.21.15.105:0/492042595 shutdown_connections 2023-11-28T18:30:48.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.725+0000 7f15d7cc9640 1 --2- 172.21.15.105:0/492042595 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15d00d3c30 0x7f15d00d1520 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.725+0000 7f15d7cc9640 1 -- 172.21.15.105:0/492042595 >> 172.21.15.105:0/492042595 conn(0x7f15d00d3030 msgr2=0x7f15d00d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:48.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.725+0000 7f15d7cc9640 1 -- 172.21.15.105:0/492042595 shutdown_connections 2023-11-28T18:30:48.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.725+0000 7f15d7cc9640 1 -- 172.21.15.105:0/492042595 wait complete. 2023-11-28T18:30:48.727 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15d7cc9640 1 Processor -- start 2023-11-28T18:30:48.727 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15d7cc9640 1 -- start start 2023-11-28T18:30:48.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15d7cc9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d00b2d20 0x7f15d01540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:48.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15d7cc9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15d00d3c30 0x7f15d0151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:48.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15d7cc9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15d015d4f0 0x7f15d015b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:48.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15d7cc9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d00d0a20 con 0x7f15d015d4f0 2023-11-28T18:30:48.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15d7cc9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d00cac40 con 0x7f15d00b2d20 2023-11-28T18:30:48.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15d7cc9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d015b8a0 con 0x7f15d00d3c30 2023-11-28T18:30:48.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15d6266640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15d015d4f0 0x7f15d015b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:48.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15d6266640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15d015d4f0 0x7f15d015b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35294/0 (socket says 172.21.15.105:35294) 2023-11-28T18:30:48.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15d6266640 1 -- 172.21.15.105:0/37462005 learned_addr learned my addr 172.21.15.105:0/37462005 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:48.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15d5264640 1 --2- 172.21.15.105:0/37462005 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15d00d3c30 0x7f15d0151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:48.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15d5a65640 1 --2- 172.21.15.105:0/37462005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d00b2d20 0x7f15d01540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:48.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15d6266640 1 -- 172.21.15.105:0/37462005 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15d00d3c30 msgr2=0x7f15d0151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:48.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15d6266640 1 --2- 172.21.15.105:0/37462005 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15d00d3c30 0x7f15d0151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15d6266640 1 -- 172.21.15.105:0/37462005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d00b2d20 msgr2=0x7f15d01540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:48.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15d6266640 1 --2- 172.21.15.105:0/37462005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d00b2d20 0x7f15d01540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15d6266640 1 -- 172.21.15.105:0/37462005 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c0009420 con 0x7f15d015d4f0 2023-11-28T18:30:48.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15d6266640 1 --2- 172.21.15.105:0/37462005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15d015d4f0 0x7f15d015b050 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f15c800c730 tx=0x7f15c800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:48.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15c67fc640 1 -- 172.21.15.105:0/37462005 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8003b80 con 0x7f15d015d4f0 2023-11-28T18:30:48.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15c67fc640 1 -- 172.21.15.105:0/37462005 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c8003ce0 con 0x7f15d015d4f0 2023-11-28T18:30:48.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15d7cc9640 1 -- 172.21.15.105:0/37462005 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d00ca2a0 con 0x7f15d015d4f0 2023-11-28T18:30:48.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15c67fc640 1 -- 172.21.15.105:0/37462005 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c800aa30 con 0x7f15d015d4f0 2023-11-28T18:30:48.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15d7cc9640 1 -- 172.21.15.105:0/37462005 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d00ca860 con 0x7f15d015d4f0 2023-11-28T18:30:48.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.729+0000 7f15d7cc9640 1 -- 172.21.15.105:0/37462005 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d00cad80 con 0x7f15d015d4f0 2023-11-28T18:30:48.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.733+0000 7f15c67fc640 1 -- 172.21.15.105:0/37462005 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f15c800ab90 con 0x7f15d015d4f0 2023-11-28T18:30:48.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.733+0000 7f15c67fc640 1 --2- 172.21.15.105:0/37462005 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15b40668d0 0x7f15b4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:48.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.733+0000 7f15c67fc640 1 -- 172.21.15.105:0/37462005 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f15c8084220 con 0x7f15d015d4f0 2023-11-28T18:30:48.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.733+0000 7f15d5a65640 1 --2- 172.21.15.105:0/37462005 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15b40668d0 0x7f15b4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:48.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.733+0000 7f15d5a65640 1 --2- 172.21.15.105:0/37462005 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15b40668d0 0x7f15b4068d90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0011f0 tx=0x7f15cc007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:48.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.737+0000 7f15c67fc640 1 -- 172.21.15.105:0/37462005 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f15d00cad80 con 0x7f15d015d4f0 2023-11-28T18:30:48.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.917+0000 7f15d7cc9640 1 -- 172.21.15.105:0/37462005 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15d00cad80 con 0x7f15d015d4f0 2023-11-28T18:30:48.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.917+0000 7f15c67fc640 1 -- 172.21.15.105:0/37462005 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4963 (secure 0 0 0) 0x7f15d00cad80 con 0x7f15d015d4f0 2023-11-28T18:30:48.917 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:48.917 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":111,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/2603493310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2603493310},{"type":"v1","addr":"172.21.15.148:6834","nonce":2603493310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:30:05.525117+0000","modified":"2023-11-28T18:30:45.830630+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8145,"mds_1":8142,"mds_2":6803},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6803":{"gid":6803,"name":"d","rank":2,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6835/3664281230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":3664281230},{"type":"v1","addr":"172.21.15.148:6835","nonce":3664281230}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8142":{"gid":8142,"name":"a","rank":1,"incarnation":105,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6836/3968059262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3968059262},{"type":"v1","addr":"172.21.15.105:6836","nonce":3968059262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8145":{"gid":8145,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2782729992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2782729992},{"type":"v1","addr":"172.21.15.105:6837","nonce":2782729992}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-11-28T18:30:48.917 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 111 2023-11-28T18:30:48.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.917+0000 7f15d7cc9640 1 -- 172.21.15.105:0/37462005 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15b40668d0 msgr2=0x7f15b4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:48.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.917+0000 7f15d7cc9640 1 --2- 172.21.15.105:0/37462005 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15b40668d0 0x7f15b4068d90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0011f0 tx=0x7f15cc007d40 comp rx=0 tx=0).stop 2023-11-28T18:30:48.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.917+0000 7f15d7cc9640 1 -- 172.21.15.105:0/37462005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15d015d4f0 msgr2=0x7f15d015b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:48.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.917+0000 7f15d7cc9640 1 --2- 172.21.15.105:0/37462005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15d015d4f0 0x7f15d015b050 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f15c800c730 tx=0x7f15c800cbf0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.917+0000 7f15d7cc9640 1 -- 172.21.15.105:0/37462005 shutdown_connections 2023-11-28T18:30:48.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.917+0000 7f15d7cc9640 1 --2- 172.21.15.105:0/37462005 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15b40668d0 0x7f15b4068d90 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.917+0000 7f15d7cc9640 1 --2- 172.21.15.105:0/37462005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15d015d4f0 0x7f15d015b050 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.917+0000 7f15d7cc9640 1 --2- 172.21.15.105:0/37462005 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15d00d3c30 0x7f15d0151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.917+0000 7f15d7cc9640 1 --2- 172.21.15.105:0/37462005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d00b2d20 0x7f15d01540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:48.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.917+0000 7f15d7cc9640 1 -- 172.21.15.105:0/37462005 >> 172.21.15.105:0/37462005 conn(0x7f15d00d3030 msgr2=0x7f15d00d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:48.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.917+0000 7f15d7cc9640 1 -- 172.21.15.105:0/37462005 shutdown_connections 2023-11-28T18:30:48.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:48.917+0000 7f15d7cc9640 1 -- 172.21.15.105:0/37462005 wait complete. 2023-11-28T18:30:48.938 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:30:49.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.077+0000 7fadf3157640 1 Processor -- start 2023-11-28T18:30:49.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.077+0000 7fadf3157640 1 -- start start 2023-11-28T18:30:49.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.077+0000 7fadf3157640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fadec0cdd80 0x7fadec06c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:49.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.077+0000 7fadf3157640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fadec0ce230 0x7fadec0cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:49.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.077+0000 7fadf3157640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadec0b3c40 con 0x7fadec15d320 2023-11-28T18:30:49.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.077+0000 7fadf3157640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadec06ca80 con 0x7fadec0ce230 2023-11-28T18:30:49.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.077+0000 7fadf3157640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadec15d700 con 0x7fadec0cdd80 2023-11-28T18:30:49.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.077+0000 7fadf16f4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fadec0ce230 0x7fadec0cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:49.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.077+0000 7fadf16f4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fadec0ce230 0x7fadec0cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35302/0 (socket says 172.21.15.105:35302) 2023-11-28T18:30:49.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.077+0000 7fadf16f4640 1 -- 172.21.15.105:0/3097850200 learned_addr learned my addr 172.21.15.105:0/3097850200 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.077+0000 7fadebfff640 1 --2- 172.21.15.105:0/3097850200 >> v2:172.21.15.148:3300/0 conn(0x7fadec0cdd80 0x7fadec06c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.077+0000 7fadf16f4640 1 -- 172.21.15.105:0/3097850200 >> v2:172.21.15.148:3300/0 conn(0x7fadec0cdd80 msgr2=0x7fadec06c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.077+0000 7fadf16f4640 1 --2- 172.21.15.105:0/3097850200 >> v2:172.21.15.148:3300/0 conn(0x7fadec0cdd80 0x7fadec06c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.077+0000 7fadf16f4640 1 -- 172.21.15.105:0/3097850200 >> v1:172.21.15.148:6789/0 conn(0x7fadec15d320 legacy=0x7fadec0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.077+0000 7fadf16f4640 1 -- 172.21.15.105:0/3097850200 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec06c7a0 con 0x7fadec0ce230 2023-11-28T18:30:49.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.077+0000 7fadebfff640 1 --2- 172.21.15.105:0/3097850200 >> v2:172.21.15.148:3300/0 conn(0x7fadec0cdd80 0x7fadec06c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:49.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.081+0000 7fadf16f4640 1 --2- 172.21.15.105:0/3097850200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadec0ce230 0x7fadec0cc0c0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fadec073a90 tx=0x7fade401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c918b011c360d05 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:49.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.081+0000 7fadeb7fe640 1 -- 172.21.15.105:0/3097850200 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade40a57c0 con 0x7fadec0ce230 2023-11-28T18:30:49.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.081+0000 7fadeb7fe640 1 -- 172.21.15.105:0/3097850200 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade40a5920 con 0x7fadec0ce230 2023-11-28T18:30:49.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.081+0000 7fadeb7fe640 1 -- 172.21.15.105:0/3097850200 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade40a5cb0 con 0x7fadec0ce230 2023-11-28T18:30:49.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.081+0000 7fadf3157640 1 -- 172.21.15.105:0/3097850200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadec0ce230 msgr2=0x7fadec0cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:49.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.081+0000 7fadf3157640 1 --2- 172.21.15.105:0/3097850200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadec0ce230 0x7fadec0cc0c0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fadec073a90 tx=0x7fade401c900 comp rx=0 tx=0).stop 2023-11-28T18:30:49.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.081+0000 7fadf3157640 1 -- 172.21.15.105:0/3097850200 shutdown_connections 2023-11-28T18:30:49.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.081+0000 7fadf3157640 1 --2- 172.21.15.105:0/3097850200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadec0ce230 0x7fadec0cc0c0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.081+0000 7fadf3157640 1 --2- 172.21.15.105:0/3097850200 >> v2:172.21.15.148:3300/0 conn(0x7fadec0cdd80 0x7fadec06c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.081+0000 7fadf3157640 1 -- 172.21.15.105:0/3097850200 >> 172.21.15.105:0/3097850200 conn(0x7fadec0d0df0 msgr2=0x7fadec0d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:49.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.081+0000 7fadf3157640 1 -- 172.21.15.105:0/3097850200 shutdown_connections 2023-11-28T18:30:49.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.081+0000 7fadf3157640 1 -- 172.21.15.105:0/3097850200 wait complete. 2023-11-28T18:30:49.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fadf3157640 1 Processor -- start 2023-11-28T18:30:49.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fadf3157640 1 -- start start 2023-11-28T18:30:49.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fadf3157640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadec0cdd80 0x7fadec1552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:49.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fadf3157640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadec0ce230 0x7fadec153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:49.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fadf3157640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadec15d320 0x7fadec155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:49.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fadf3157640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec160200 con 0x7fadec0ce230 2023-11-28T18:30:49.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fadf3157640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec0b3f00 con 0x7fadec15d320 2023-11-28T18:30:49.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fadf3157640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadec0b41e0 con 0x7fadec0cdd80 2023-11-28T18:30:49.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fadebfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadec0ce230 0x7fadec153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:49.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fadebfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadec0ce230 0x7fadec153760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35310/0 (socket says 172.21.15.105:35310) 2023-11-28T18:30:49.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fadebfff640 1 -- 172.21.15.105:0/2467175878 learned_addr learned my addr 172.21.15.105:0/2467175878 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:49.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fadf16f4640 1 --2- 172.21.15.105:0/2467175878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadec15d320 0x7fadec155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:49.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fadf0ef3640 1 --2- 172.21.15.105:0/2467175878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadec0cdd80 0x7fadec1552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:49.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fadebfff640 1 -- 172.21.15.105:0/2467175878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadec0cdd80 msgr2=0x7fadec1552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:49.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fadebfff640 1 --2- 172.21.15.105:0/2467175878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadec0cdd80 0x7fadec1552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fadebfff640 1 -- 172.21.15.105:0/2467175878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadec15d320 msgr2=0x7fadec155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:49.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fadebfff640 1 --2- 172.21.15.105:0/2467175878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadec15d320 0x7fadec155c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fadebfff640 1 -- 172.21.15.105:0/2467175878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade4009b30 con 0x7fadec0ce230 2023-11-28T18:30:49.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fadebfff640 1 --2- 172.21.15.105:0/2467175878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadec0ce230 0x7fadec153760 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fadd800e7e0 tx=0x7fadd800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:49.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fade97fa640 1 -- 172.21.15.105:0/2467175878 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd800c8d0 con 0x7fadec0ce230 2023-11-28T18:30:49.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fade97fa640 1 -- 172.21.15.105:0/2467175878 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd800ee90 con 0x7fadec0ce230 2023-11-28T18:30:49.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fade97fa640 1 -- 172.21.15.105:0/2467175878 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd800cc30 con 0x7fadec0ce230 2023-11-28T18:30:49.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fadf3157640 1 -- 172.21.15.105:0/2467175878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadec0b44c0 con 0x7fadec0ce230 2023-11-28T18:30:49.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fadf3157640 1 -- 172.21.15.105:0/2467175878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadec1558f0 con 0x7fadec0ce230 2023-11-28T18:30:49.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.085+0000 7fadf3157640 1 -- 172.21.15.105:0/2467175878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadec0b1d00 con 0x7fadec0ce230 2023-11-28T18:30:49.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.089+0000 7fade97fa640 1 -- 172.21.15.105:0/2467175878 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fadd8013410 con 0x7fadec0ce230 2023-11-28T18:30:49.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.089+0000 7fade97fa640 1 --2- 172.21.15.105:0/2467175878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fadd00668d0 0x7fadd0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:49.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.089+0000 7fade97fa640 1 -- 172.21.15.105:0/2467175878 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fadd8084290 con 0x7fadec0ce230 2023-11-28T18:30:49.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.089+0000 7fadf0ef3640 1 --2- 172.21.15.105:0/2467175878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fadd00668d0 0x7fadd0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:49.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.089+0000 7fadf0ef3640 1 --2- 172.21.15.105:0/2467175878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fadd00668d0 0x7fadd0068d90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fadec15f520 tx=0x7fade0009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:49.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.093+0000 7fade97fa640 1 -- 172.21.15.105:0/2467175878 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fadd8051410 con 0x7fadec0ce230 2023-11-28T18:30:49.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.245+0000 7fadf3157640 1 -- 172.21.15.105:0/2467175878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadec067570 con 0x7fadec0ce230 2023-11-28T18:30:49.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.245+0000 7fade97fa640 1 -- 172.21.15.105:0/2467175878 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16826 (secure 0 0 0) 0x7fadd8054a30 con 0x7fadec0ce230 2023-11-28T18:30:49.248 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:49.248 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":105,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:30:16.894705+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:30:03.341424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-28T18:30:04.527931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1b","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:30:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.249+0000 7fadf3157640 1 -- 172.21.15.105:0/2467175878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fadd00668d0 msgr2=0x7fadd0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.249+0000 7fadf3157640 1 --2- 172.21.15.105:0/2467175878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fadd00668d0 0x7fadd0068d90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fadec15f520 tx=0x7fade0009060 comp rx=0 tx=0).stop 2023-11-28T18:30:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.249+0000 7fadf3157640 1 -- 172.21.15.105:0/2467175878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadec0ce230 msgr2=0x7fadec153760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.249+0000 7fadf3157640 1 --2- 172.21.15.105:0/2467175878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadec0ce230 0x7fadec153760 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fadd800e7e0 tx=0x7fadd800eca0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.249+0000 7fadf3157640 1 -- 172.21.15.105:0/2467175878 shutdown_connections 2023-11-28T18:30:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.249+0000 7fadf3157640 1 --2- 172.21.15.105:0/2467175878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadec15d320 0x7fadec155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.249+0000 7fadf3157640 1 --2- 172.21.15.105:0/2467175878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fadd00668d0 0x7fadd0068d90 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.249+0000 7fadf3157640 1 --2- 172.21.15.105:0/2467175878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadec0ce230 0x7fadec153760 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.249+0000 7fadf3157640 1 --2- 172.21.15.105:0/2467175878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadec0cdd80 0x7fadec1552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.249+0000 7fadf3157640 1 -- 172.21.15.105:0/2467175878 >> 172.21.15.105:0/2467175878 conn(0x7fadec0d0df0 msgr2=0x7fadec06fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.249+0000 7fadf3157640 1 -- 172.21.15.105:0/2467175878 shutdown_connections 2023-11-28T18:30:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.249+0000 7fadf3157640 1 -- 172.21.15.105:0/2467175878 wait complete. 2023-11-28T18:30:49.267 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T18:30:49.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.405+0000 7f7b4c557640 1 Processor -- start 2023-11-28T18:30:49.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.409+0000 7f7b4c557640 1 -- start start 2023-11-28T18:30:49.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.409+0000 7f7b4c557640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7b440caf60 0x7f7b440cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:49.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.409+0000 7f7b4c557640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4406bf50 con 0x7f7b440ad850 2023-11-28T18:30:49.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.409+0000 7f7b4c557640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4415a320 con 0x7f7b44159f40 2023-11-28T18:30:49.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.409+0000 7f7b4c557640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b440cb3c0 con 0x7f7b440caf60 2023-11-28T18:30:49.410 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.409+0000 7f7b4aaf4640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7b440ad850 0x7f7b440701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33726/0 (socket says 172.21.15.105:33726) 2023-11-28T18:30:49.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.409+0000 7f7b4aaf4640 1 -- 172.21.15.105:0/2033933526 learned_addr learned my addr 172.21.15.105:0/2033933526 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:49.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.409+0000 7f7b49af2640 1 --2- 172.21.15.105:0/2033933526 >> v2:172.21.15.148:3300/0 conn(0x7f7b440caf60 0x7f7b440cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:49.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.409+0000 7f7b492f1640 1 -- 172.21.15.105:0/2033933526 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4187885281 0 0) 0x7f7b4406bf50 con 0x7f7b440ad850 2023-11-28T18:30:49.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.409+0000 7f7b492f1640 1 -- 172.21.15.105:0/2033933526 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b30003530 con 0x7f7b440ad850 2023-11-28T18:30:49.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.409+0000 7f7b49af2640 1 -- 172.21.15.105:0/2033933526 >> v1:172.21.15.148:6789/0 conn(0x7f7b44159f40 legacy=0x7f7b440ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:49.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.409+0000 7f7b492f1640 1 -- 172.21.15.105:0/2033933526 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2571664775 0 0) 0x7f7b30003530 con 0x7f7b440ad850 2023-11-28T18:30:49.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.409+0000 7f7b49af2640 1 -- 172.21.15.105:0/2033933526 >> v1:172.21.15.105:6789/0 conn(0x7f7b440ad850 legacy=0x7f7b440701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:49.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.409+0000 7f7b49af2640 1 -- 172.21.15.105:0/2033933526 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b44071380 con 0x7f7b440caf60 2023-11-28T18:30:49.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.413+0000 7f7b49af2640 1 --2- 172.21.15.105:0/2033933526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b440caf60 0x7f7b440cc760 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7b340097c0 tx=0x7f7b3401c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=535a6433d968ef21 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:49.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.413+0000 7f7b492f1640 1 -- 172.21.15.105:0/2033933526 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c001a00 con 0x7f7b440caf60 2023-11-28T18:30:49.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.413+0000 7f7b492f1640 1 -- 172.21.15.105:0/2033933526 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b340a7050 con 0x7f7b440caf60 2023-11-28T18:30:49.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.413+0000 7f7b492f1640 1 -- 172.21.15.105:0/2033933526 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b340ab620 con 0x7f7b440caf60 2023-11-28T18:30:49.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.413+0000 7f7b4c557640 1 -- 172.21.15.105:0/2033933526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b440caf60 msgr2=0x7f7b440cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:49.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.413+0000 7f7b4c557640 1 --2- 172.21.15.105:0/2033933526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b440caf60 0x7f7b440cc760 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7b340097c0 tx=0x7f7b3401c2e0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.413+0000 7f7b4c557640 1 -- 172.21.15.105:0/2033933526 shutdown_connections 2023-11-28T18:30:49.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.413+0000 7f7b4c557640 1 --2- 172.21.15.105:0/2033933526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b440caf60 0x7f7b440cc760 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.413+0000 7f7b4c557640 1 -- 172.21.15.105:0/2033933526 >> 172.21.15.105:0/2033933526 conn(0x7f7b440ceb90 msgr2=0x7f7b440cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:49.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.413+0000 7f7b4c557640 1 -- 172.21.15.105:0/2033933526 shutdown_connections 2023-11-28T18:30:49.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.413+0000 7f7b4c557640 1 -- 172.21.15.105:0/2033933526 wait complete. 2023-11-28T18:30:49.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b4c557640 1 Processor -- start 2023-11-28T18:30:49.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b4c557640 1 -- start start 2023-11-28T18:30:49.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b4c557640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b440ad850 0x7f7b4414def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:49.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b4c557640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b440caf60 0x7f7b4415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:49.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b4c557640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b44159f40 0x7f7b440c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:49.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b4c557640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4415bd20 con 0x7f7b440caf60 2023-11-28T18:30:49.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b4c557640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4415c000 con 0x7f7b440ad850 2023-11-28T18:30:49.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b4c557640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4415c2e0 con 0x7f7b44159f40 2023-11-28T18:30:49.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b49af2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b440caf60 0x7f7b4415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:49.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b49af2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b440caf60 0x7f7b4415cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35318/0 (socket says 172.21.15.105:35318) 2023-11-28T18:30:49.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b49af2640 1 -- 172.21.15.105:0/1986103373 learned_addr learned my addr 172.21.15.105:0/1986103373 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:49.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b4a2f3640 1 --2- 172.21.15.105:0/1986103373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b440ad850 0x7f7b4414def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:49.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b4aaf4640 1 --2- 172.21.15.105:0/1986103373 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b44159f40 0x7f7b440c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:49.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b49af2640 1 -- 172.21.15.105:0/1986103373 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b44159f40 msgr2=0x7f7b440c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:49.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b49af2640 1 --2- 172.21.15.105:0/1986103373 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b44159f40 0x7f7b440c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b49af2640 1 -- 172.21.15.105:0/1986103373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b440ad850 msgr2=0x7f7b4414def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:49.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b49af2640 1 --2- 172.21.15.105:0/1986103373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b440ad850 0x7f7b4414def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b49af2640 1 -- 172.21.15.105:0/1986103373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b34009420 con 0x7f7b440caf60 2023-11-28T18:30:49.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b4aaf4640 1 --2- 172.21.15.105:0/1986103373 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b44159f40 0x7f7b440c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:49.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b49af2640 1 --2- 172.21.15.105:0/1986103373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b440caf60 0x7f7b4415cdc0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f7b340097c0 tx=0x7f7b34019080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:49.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b3affd640 1 -- 172.21.15.105:0/1986103373 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b34004020 con 0x7f7b440caf60 2023-11-28T18:30:49.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b3affd640 1 -- 172.21.15.105:0/1986103373 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b34004180 con 0x7f7b440caf60 2023-11-28T18:30:49.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b3affd640 1 -- 172.21.15.105:0/1986103373 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b34004510 con 0x7f7b440caf60 2023-11-28T18:30:49.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b4c557640 1 -- 172.21.15.105:0/1986103373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b4415c5c0 con 0x7f7b440caf60 2023-11-28T18:30:49.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b4c557640 1 -- 172.21.15.105:0/1986103373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b4415d300 con 0x7f7b440caf60 2023-11-28T18:30:49.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.417+0000 7f7b4c557640 1 -- 172.21.15.105:0/1986103373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b44067570 con 0x7f7b440caf60 2023-11-28T18:30:49.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.421+0000 7f7b3affd640 1 -- 172.21.15.105:0/1986103373 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f7b340b2040 con 0x7f7b440caf60 2023-11-28T18:30:49.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.421+0000 7f7b3affd640 1 --2- 172.21.15.105:0/1986103373 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7b280668d0 0x7f7b28068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:49.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.421+0000 7f7b3affd640 1 -- 172.21.15.105:0/1986103373 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7b3411e660 con 0x7f7b440caf60 2023-11-28T18:30:49.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.421+0000 7f7b4a2f3640 1 --2- 172.21.15.105:0/1986103373 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7b280668d0 0x7f7b28068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:49.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.421+0000 7f7b4a2f3640 1 --2- 172.21.15.105:0/1986103373 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7b280668d0 0x7f7b28068d90 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7b40000f70 tx=0x7f7b40009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:49.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.425+0000 7f7b3affd640 1 -- 172.21.15.105:0/1986103373 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7b340eb6a0 con 0x7f7b440caf60 2023-11-28T18:30:49.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.605+0000 7f7b4c557640 1 -- 172.21.15.105:0/1986103373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7b4415ca70 con 0x7f7b440caf60 2023-11-28T18:30:49.609 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-11-28T18:30:49.609+0000 7fd89b32a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T18:30:49.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.609+0000 7f7b3affd640 1 -- 172.21.15.105:0/1986103373 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v112) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7b340eecc0 con 0x7f7b440caf60 2023-11-28T18:30:49.610 INFO:teuthology.orchestra.run.smithi105.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T18:30:49.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.613+0000 7f7b4c557640 1 -- 172.21.15.105:0/1986103373 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7b280668d0 msgr2=0x7f7b28068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:49.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.613+0000 7f7b4c557640 1 --2- 172.21.15.105:0/1986103373 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7b280668d0 0x7f7b28068d90 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7b40000f70 tx=0x7f7b40009040 comp rx=0 tx=0).stop 2023-11-28T18:30:49.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.613+0000 7f7b4c557640 1 -- 172.21.15.105:0/1986103373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b440caf60 msgr2=0x7f7b4415cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:49.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.613+0000 7f7b4c557640 1 --2- 172.21.15.105:0/1986103373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b440caf60 0x7f7b4415cdc0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f7b340097c0 tx=0x7f7b34019080 comp rx=0 tx=0).stop 2023-11-28T18:30:49.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.613+0000 7f7b4c557640 1 -- 172.21.15.105:0/1986103373 shutdown_connections 2023-11-28T18:30:49.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.613+0000 7f7b4c557640 1 --2- 172.21.15.105:0/1986103373 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b44159f40 0x7f7b440c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.614 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.613+0000 7f7b4c557640 1 --2- 172.21.15.105:0/1986103373 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7b280668d0 0x7f7b28068d90 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.613+0000 7f7b4c557640 1 --2- 172.21.15.105:0/1986103373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b440caf60 0x7f7b4415cdc0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.613+0000 7f7b4c557640 1 --2- 172.21.15.105:0/1986103373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b440ad850 0x7f7b4414def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.613+0000 7f7b4c557640 1 -- 172.21.15.105:0/1986103373 >> 172.21.15.105:0/1986103373 conn(0x7f7b440ceb90 msgr2=0x7f7b44152fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:49.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.613+0000 7f7b4c557640 1 -- 172.21.15.105:0/1986103373 shutdown_connections 2023-11-28T18:30:49.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.613+0000 7f7b4c557640 1 -- 172.21.15.105:0/1986103373 wait complete. 2023-11-28T18:30:49.635 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T18:30:49.665 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2023-11-28T18:30:49.677 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2023-11-28T18:30:49.690 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2023-11-28T18:30:49.781 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.781+0000 7f959e9c4640 1 Processor -- start 2023-11-28T18:30:49.781 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.781+0000 7f959e9c4640 1 -- start start 2023-11-28T18:30:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.781+0000 7f959e9c4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9598159f40 0x7f95980caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.781+0000 7f959e9c4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f95980ad850 0x7f95980701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.781+0000 7f959e9c4640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f959806bf50 con 0x7f95980cc760 2023-11-28T18:30:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.781+0000 7f959e9c4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95980ccbc0 con 0x7f9598159f40 2023-11-28T18:30:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.781+0000 7f959e9c4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f959815a320 con 0x7f95980ad850 2023-11-28T18:30:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.781+0000 7f9597fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9598159f40 0x7f95980caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.781+0000 7f9597fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9598159f40 0x7f95980caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35354/0 (socket says 172.21.15.105:35354) 2023-11-28T18:30:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.781+0000 7f9597fff640 1 -- 172.21.15.105:0/3441859492 learned_addr learned my addr 172.21.15.105:0/3441859492 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.781+0000 7f958ffff640 1 --1- 172.21.15.105:0/3441859492 >> v1:172.21.15.105:6789/0 conn(0x7f95980cc760 0x7f95980ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33754/0 (socket says 172.21.15.105:33754) 2023-11-28T18:30:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.785+0000 7f9597fff640 1 -- 172.21.15.105:0/3441859492 >> v2:172.21.15.148:3300/0 conn(0x7f95980ad850 msgr2=0x7f95980701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.785+0000 7f9597fff640 1 --2- 172.21.15.105:0/3441859492 >> v2:172.21.15.148:3300/0 conn(0x7f95980ad850 0x7f95980701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.785+0000 7f9597fff640 1 -- 172.21.15.105:0/3441859492 >> v1:172.21.15.105:6789/0 conn(0x7f95980cc760 legacy=0x7f95980ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.785+0000 7f9597fff640 1 -- 172.21.15.105:0/3441859492 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9598071380 con 0x7f9598159f40 2023-11-28T18:30:49.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.785+0000 7f9597fff640 1 --2- 172.21.15.105:0/3441859492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9598159f40 0x7f95980caf60 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f95980b0850 tx=0x7f958801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8acbd2d06dad03 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:49.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.785+0000 7f95977fe640 1 -- 172.21.15.105:0/3441859492 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95880ab640 con 0x7f9598159f40 2023-11-28T18:30:49.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.785+0000 7f95977fe640 1 -- 172.21.15.105:0/3441859492 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95880a7070 con 0x7f9598159f40 2023-11-28T18:30:49.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.785+0000 7f95977fe640 1 -- 172.21.15.105:0/3441859492 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95880ab9c0 con 0x7f9598159f40 2023-11-28T18:30:49.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.785+0000 7f959e9c4640 1 -- 172.21.15.105:0/3441859492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9598159f40 msgr2=0x7f95980caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:49.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.785+0000 7f959e9c4640 1 --2- 172.21.15.105:0/3441859492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9598159f40 0x7f95980caf60 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f95980b0850 tx=0x7f958801c440 comp rx=0 tx=0).stop 2023-11-28T18:30:49.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.785+0000 7f959e9c4640 1 -- 172.21.15.105:0/3441859492 shutdown_connections 2023-11-28T18:30:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.785+0000 7f959e9c4640 1 --2- 172.21.15.105:0/3441859492 >> v2:172.21.15.148:3300/0 conn(0x7f95980ad850 0x7f95980701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.785+0000 7f959e9c4640 1 --2- 172.21.15.105:0/3441859492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9598159f40 0x7f95980caf60 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.785+0000 7f959e9c4640 1 -- 172.21.15.105:0/3441859492 >> 172.21.15.105:0/3441859492 conn(0x7f95980ceb90 msgr2=0x7f95980cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f959e9c4640 1 -- 172.21.15.105:0/3441859492 shutdown_connections 2023-11-28T18:30:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f959e9c4640 1 -- 172.21.15.105:0/3441859492 wait complete. 2023-11-28T18:30:49.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f959e9c4640 1 Processor -- start 2023-11-28T18:30:49.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f959e9c4640 1 -- start start 2023-11-28T18:30:49.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f959e9c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95980ad850 0x7f95980b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:49.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f959e9c4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95980cc760 0x7f95980b3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:49.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f959e9c4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9598159f40 0x7f95981f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:49.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f959e9c4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959809c430 con 0x7f95980cc760 2023-11-28T18:30:49.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f959e9c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959814f050 con 0x7f95980ad850 2023-11-28T18:30:49.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f959e9c4640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f959814f330 con 0x7f9598159f40 2023-11-28T18:30:49.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f958ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95980cc760 0x7f95980b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:49.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f958ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95980cc760 0x7f95980b3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35366/0 (socket says 172.21.15.105:35366) 2023-11-28T18:30:49.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f958ffff640 1 -- 172.21.15.105:0/2819663720 learned_addr learned my addr 172.21.15.105:0/2819663720 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:49.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f9597fff640 1 --2- 172.21.15.105:0/2819663720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95980ad850 0x7f95980b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:49.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f959cf61640 1 --2- 172.21.15.105:0/2819663720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9598159f40 0x7f95981f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:49.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f958ffff640 1 -- 172.21.15.105:0/2819663720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9598159f40 msgr2=0x7f95981f7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:49.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f958ffff640 1 --2- 172.21.15.105:0/2819663720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9598159f40 0x7f95981f7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f958ffff640 1 -- 172.21.15.105:0/2819663720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95980ad850 msgr2=0x7f95980b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:49.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f958ffff640 1 --2- 172.21.15.105:0/2819663720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95980ad850 0x7f95980b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:49.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f958ffff640 1 -- 172.21.15.105:0/2819663720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95880094d0 con 0x7f95980cc760 2023-11-28T18:30:49.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f958ffff640 1 --2- 172.21.15.105:0/2819663720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95980cc760 0x7f95980b3910 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f958000c490 tx=0x7f958000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:49.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f95957fa640 1 -- 172.21.15.105:0/2819663720 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9580007480 con 0x7f95980cc760 2023-11-28T18:30:49.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f95957fa640 1 -- 172.21.15.105:0/2819663720 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95800075e0 con 0x7f95980cc760 2023-11-28T18:30:49.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f95957fa640 1 -- 172.21.15.105:0/2819663720 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9580004a10 con 0x7f95980cc760 2023-11-28T18:30:49.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f959e9c4640 1 -- 172.21.15.105:0/2819663720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f959814f610 con 0x7f95980cc760 2023-11-28T18:30:49.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.789+0000 7f959e9c4640 1 -- 172.21.15.105:0/2819663720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95981f4570 con 0x7f95980cc760 2023-11-28T18:30:49.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.793+0000 7f959e9c4640 1 -- 172.21.15.105:0/2819663720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9598067570 con 0x7f95980cc760 2023-11-28T18:30:49.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.793+0000 7f95957fa640 1 -- 172.21.15.105:0/2819663720 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f9580004b70 con 0x7f95980cc760 2023-11-28T18:30:49.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.793+0000 7f95957fa640 1 --2- 172.21.15.105:0/2819663720 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f95780668d0 0x7f9578068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:49.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.793+0000 7f95957fa640 1 -- 172.21.15.105:0/2819663720 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f9580083910 con 0x7f95980cc760 2023-11-28T18:30:49.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.793+0000 7f9597fff640 1 --2- 172.21.15.105:0/2819663720 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f95780668d0 0x7f9578068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:49.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.793+0000 7f9597fff640 1 --2- 172.21.15.105:0/2819663720 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f95780668d0 0x7f9578068d90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f95880af000 tx=0x7f95880b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:49.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.797+0000 7f95957fa640 1 -- 172.21.15.105:0/2819663720 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9580050870 con 0x7f95980cc760 2023-11-28T18:30:49.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:49.977+0000 7f959e9c4640 1 -- 172.21.15.105:0/2819663720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9598165b60 con 0x7f95980cc760 2023-11-28T18:30:50.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.613+0000 7f95957fa640 1 -- 172.21.15.105:0/2819663720 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v113) v1 ==== 104+0+0 (secure 0 0 0) 0x7f95800180a0 con 0x7f95980cc760 2023-11-28T18:30:50.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.613+0000 7f959e9c4640 1 -- 172.21.15.105:0/2819663720 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f95780668d0 msgr2=0x7f9578068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:50.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.613+0000 7f959e9c4640 1 --2- 172.21.15.105:0/2819663720 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f95780668d0 0x7f9578068d90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f95880af000 tx=0x7f95880b6040 comp rx=0 tx=0).stop 2023-11-28T18:30:50.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.617+0000 7f959e9c4640 1 -- 172.21.15.105:0/2819663720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95980cc760 msgr2=0x7f95980b3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:50.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.617+0000 7f959e9c4640 1 --2- 172.21.15.105:0/2819663720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95980cc760 0x7f95980b3910 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f958000c490 tx=0x7f958000c950 comp rx=0 tx=0).stop 2023-11-28T18:30:50.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.617+0000 7f959e9c4640 1 -- 172.21.15.105:0/2819663720 shutdown_connections 2023-11-28T18:30:50.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.617+0000 7f959e9c4640 1 --2- 172.21.15.105:0/2819663720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9598159f40 0x7f95981f7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:50.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.617+0000 7f959e9c4640 1 --2- 172.21.15.105:0/2819663720 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f95780668d0 0x7f9578068d90 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:50.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.617+0000 7f959e9c4640 1 --2- 172.21.15.105:0/2819663720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95980cc760 0x7f95980b3910 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:50.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.617+0000 7f959e9c4640 1 --2- 172.21.15.105:0/2819663720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95980ad850 0x7f95980b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:50.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.617+0000 7f959e9c4640 1 -- 172.21.15.105:0/2819663720 >> 172.21.15.105:0/2819663720 conn(0x7f95980ceb90 msgr2=0x7f95980ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:50.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.617+0000 7f959e9c4640 1 -- 172.21.15.105:0/2819663720 shutdown_connections 2023-11-28T18:30:50.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.617+0000 7f959e9c4640 1 -- 172.21.15.105:0/2819663720 wait complete. 2023-11-28T18:30:50.641 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T18:30:50.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.829+0000 7f0a0a1f8640 1 Processor -- start 2023-11-28T18:30:50.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.829+0000 7f0a0a1f8640 1 -- start start 2023-11-28T18:30:50.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.829+0000 7f0a0a1f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0a0415af10 0x7f0a040d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:50.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.829+0000 7f0a0a1f8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0a040b0240 0x7f0a040ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:50.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.829+0000 7f0a0a1f8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a040ae350 con 0x7f0a040b3f00 2023-11-28T18:30:50.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.829+0000 7f0a0a1f8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a040d0fd0 con 0x7f0a040b0240 2023-11-28T18:30:50.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.829+0000 7f0a0a1f8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a040d12b0 con 0x7f0a0415af10 2023-11-28T18:30:50.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.829+0000 7f0a099f7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0a040b0240 0x7f0a040ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:50.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.829+0000 7f0a089f5640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0a040b3f00 0x7f0a040afb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33766/0 (socket says 172.21.15.105:33766) 2023-11-28T18:30:50.831 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.829+0000 7f0a089f5640 1 -- 172.21.15.105:0/2448124483 learned_addr learned my addr 172.21.15.105:0/2448124483 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:50.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.829+0000 7f0a091f6640 1 --2- 172.21.15.105:0/2448124483 >> v2:172.21.15.148:3300/0 conn(0x7f0a0415af10 0x7f0a040d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:50.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.829+0000 7f0a099f7640 1 -- 172.21.15.105:0/2448124483 >> v2:172.21.15.148:3300/0 conn(0x7f0a0415af10 msgr2=0x7f0a040d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:50.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.829+0000 7f0a099f7640 1 --2- 172.21.15.105:0/2448124483 >> v2:172.21.15.148:3300/0 conn(0x7f0a0415af10 0x7f0a040d0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:50.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.829+0000 7f09fbfff640 1 -- 172.21.15.105:0/2448124483 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1222327686 0 0) 0x7f0a040ae350 con 0x7f0a040b3f00 2023-11-28T18:30:50.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.829+0000 7f0a099f7640 1 -- 172.21.15.105:0/2448124483 >> v1:172.21.15.105:6789/0 conn(0x7f0a040b3f00 legacy=0x7f0a040afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:50.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.829+0000 7f0a099f7640 1 -- 172.21.15.105:0/2448124483 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a040d1590 con 0x7f0a040b0240 2023-11-28T18:30:50.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.829+0000 7f0a091f6640 1 --2- 172.21.15.105:0/2448124483 >> v2:172.21.15.148:3300/0 conn(0x7f0a0415af10 0x7f0a040d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:50.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.833+0000 7f0a099f7640 1 --2- 172.21.15.105:0/2448124483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a040b0240 0x7f0a040ae7c0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f0a040b3940 tx=0x7f09fc01c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fffa0969f11bcf58 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:50.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.833+0000 7f09fbfff640 1 -- 172.21.15.105:0/2448124483 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc0a5960 con 0x7f0a040b0240 2023-11-28T18:30:50.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.833+0000 7f09fbfff640 1 -- 172.21.15.105:0/2448124483 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09fc0a5ac0 con 0x7f0a040b0240 2023-11-28T18:30:50.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.833+0000 7f09fbfff640 1 -- 172.21.15.105:0/2448124483 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc0a5e50 con 0x7f0a040b0240 2023-11-28T18:30:50.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.833+0000 7f0a0a1f8640 1 -- 172.21.15.105:0/2448124483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a040b0240 msgr2=0x7f0a040ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:50.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.833+0000 7f0a0a1f8640 1 --2- 172.21.15.105:0/2448124483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a040b0240 0x7f0a040ae7c0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f0a040b3940 tx=0x7f09fc01c880 comp rx=0 tx=0).stop 2023-11-28T18:30:50.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.833+0000 7f0a0a1f8640 1 -- 172.21.15.105:0/2448124483 shutdown_connections 2023-11-28T18:30:50.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.833+0000 7f0a0a1f8640 1 --2- 172.21.15.105:0/2448124483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a040b0240 0x7f0a040ae7c0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:50.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.833+0000 7f0a0a1f8640 1 --2- 172.21.15.105:0/2448124483 >> v2:172.21.15.148:3300/0 conn(0x7f0a0415af10 0x7f0a040d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:50.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.833+0000 7f0a0a1f8640 1 -- 172.21.15.105:0/2448124483 >> 172.21.15.105:0/2448124483 conn(0x7f0a04099a50 msgr2=0x7f0a04099e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:50.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.833+0000 7f0a0a1f8640 1 -- 172.21.15.105:0/2448124483 shutdown_connections 2023-11-28T18:30:50.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.833+0000 7f0a0a1f8640 1 -- 172.21.15.105:0/2448124483 wait complete. 2023-11-28T18:30:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f0a0a1f8640 1 Processor -- start 2023-11-28T18:30:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f0a0a1f8640 1 -- start start 2023-11-28T18:30:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f0a0a1f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a040b0240 0x7f0a0406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f0a0a1f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0a040b3f00 0x7f0a04159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f0a0a1f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a0415af10 0x7f0a0415a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f0a0a1f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a04148da0 con 0x7f0a040b0240 2023-11-28T18:30:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f0a0a1f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a041462f0 con 0x7f0a0415af10 2023-11-28T18:30:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f0a0a1f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a040b0750 con 0x7f0a040b3f00 2023-11-28T18:30:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f0a091f6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a040b0240 0x7f0a0406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f0a091f6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a040b0240 0x7f0a0406d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35384/0 (socket says 172.21.15.105:35384) 2023-11-28T18:30:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f0a091f6640 1 -- 172.21.15.105:0/3138605108 learned_addr learned my addr 172.21.15.105:0/3138605108 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f0a089f5640 1 --2- 172.21.15.105:0/3138605108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0a040b3f00 0x7f0a04159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f0a099f7640 1 --2- 172.21.15.105:0/3138605108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a0415af10 0x7f0a0415a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f0a091f6640 1 -- 172.21.15.105:0/3138605108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0a040b3f00 msgr2=0x7f0a04159d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f0a091f6640 1 --2- 172.21.15.105:0/3138605108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0a040b3f00 0x7f0a04159d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f0a091f6640 1 -- 172.21.15.105:0/3138605108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a0415af10 msgr2=0x7f0a0415a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f0a091f6640 1 --2- 172.21.15.105:0/3138605108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a0415af10 0x7f0a0415a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:50.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f0a091f6640 1 -- 172.21.15.105:0/3138605108 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc009a70 con 0x7f0a040b0240 2023-11-28T18:30:50.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f0a091f6640 1 --2- 172.21.15.105:0/3138605108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a040b0240 0x7f0a0406d150 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000c7f0 tx=0x7f0a0000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:50.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f09f9ffb640 1 -- 172.21.15.105:0/3138605108 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00007820 con 0x7f0a040b0240 2023-11-28T18:30:50.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f09f9ffb640 1 -- 172.21.15.105:0/3138605108 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a00007980 con 0x7f0a040b0240 2023-11-28T18:30:50.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f0a0a1f8640 1 -- 172.21.15.105:0/3138605108 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0415cfc0 con 0x7f0a040b0240 2023-11-28T18:30:50.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f09f9ffb640 1 -- 172.21.15.105:0/3138605108 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00007d30 con 0x7f0a040b0240 2023-11-28T18:30:50.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f0a0a1f8640 1 -- 172.21.15.105:0/3138605108 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0415d580 con 0x7f0a040b0240 2023-11-28T18:30:50.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.837+0000 7f0a0a1f8640 1 -- 172.21.15.105:0/3138605108 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a04074580 con 0x7f0a040b0240 2023-11-28T18:30:50.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.841+0000 7f09f9ffb640 1 -- 172.21.15.105:0/3138605108 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f0a000113f0 con 0x7f0a040b0240 2023-11-28T18:30:50.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.841+0000 7f09f9ffb640 1 --2- 172.21.15.105:0/3138605108 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f09e8066920 0x7f09e8068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:50.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.841+0000 7f09f9ffb640 1 -- 172.21.15.105:0/3138605108 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f0a00084a10 con 0x7f0a040b0240 2023-11-28T18:30:50.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.841+0000 7f0a089f5640 1 --2- 172.21.15.105:0/3138605108 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f09e8066920 0x7f09e8068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:50.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.841+0000 7f0a089f5640 1 --2- 172.21.15.105:0/3138605108 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f09e8066920 0x7f09e8068de0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0a0414c4c0 tx=0x7f09f4007b60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:50.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.845+0000 7f09f9ffb640 1 -- 172.21.15.105:0/3138605108 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0a000519d0 con 0x7f0a040b0240 2023-11-28T18:30:50.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:50.993+0000 7f0a0a1f8640 1 -- 172.21.15.105:0/3138605108 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0a0406efb0 con 0x7f0a040b0240 2023-11-28T18:30:51.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.613+0000 7f09f9ffb640 1 -- 172.21.15.105:0/3138605108 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v107) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0a00054ff0 con 0x7f0a040b0240 2023-11-28T18:30:51.615 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' removed 2023-11-28T18:30:51.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.617+0000 7f0a0a1f8640 1 -- 172.21.15.105:0/3138605108 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f09e8066920 msgr2=0x7f09e8068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:51.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.617+0000 7f0a0a1f8640 1 --2- 172.21.15.105:0/3138605108 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f09e8066920 0x7f09e8068de0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0a0414c4c0 tx=0x7f09f4007b60 comp rx=0 tx=0).stop 2023-11-28T18:30:51.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.617+0000 7f0a0a1f8640 1 -- 172.21.15.105:0/3138605108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a040b0240 msgr2=0x7f0a0406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:51.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.617+0000 7f0a0a1f8640 1 --2- 172.21.15.105:0/3138605108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a040b0240 0x7f0a0406d150 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000c7f0 tx=0x7f0a0000ccb0 comp rx=0 tx=0).stop 2023-11-28T18:30:51.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.617+0000 7f0a0a1f8640 1 -- 172.21.15.105:0/3138605108 shutdown_connections 2023-11-28T18:30:51.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.617+0000 7f0a0a1f8640 1 --2- 172.21.15.105:0/3138605108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a0415af10 0x7f0a0415a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:51.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.617+0000 7f0a0a1f8640 1 --2- 172.21.15.105:0/3138605108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0a040b3f00 0x7f0a04159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:51.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.617+0000 7f0a0a1f8640 1 --2- 172.21.15.105:0/3138605108 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f09e8066920 0x7f09e8068de0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:51.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.617+0000 7f0a0a1f8640 1 --2- 172.21.15.105:0/3138605108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a040b0240 0x7f0a0406d150 secure :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000c7f0 tx=0x7f0a0000ccb0 comp rx=0 tx=0).stop 2023-11-28T18:30:51.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.617+0000 7f0a0a1f8640 1 -- 172.21.15.105:0/3138605108 >> 172.21.15.105:0/3138605108 conn(0x7f0a04099a50 msgr2=0x7f0a040cacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:51.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.617+0000 7f0a0a1f8640 1 -- 172.21.15.105:0/3138605108 shutdown_connections 2023-11-28T18:30:51.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.617+0000 7f0a0a1f8640 1 -- 172.21.15.105:0/3138605108 wait complete. 2023-11-28T18:30:51.642 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T18:30:51.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.789+0000 7f17869e6640 1 Processor -- start 2023-11-28T18:30:51.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.789+0000 7f17869e6640 1 -- start start 2023-11-28T18:30:51.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.793+0000 7f17869e6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f17800cc760 0x7f1780164d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:51.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.793+0000 7f17869e6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17800ccc10 con 0x7f17800caf60 2023-11-28T18:30:51.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.793+0000 7f17869e6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f178006bf50 con 0x7f1780159f40 2023-11-28T18:30:51.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.793+0000 7f17869e6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17800cb3c0 con 0x7f17800cc760 2023-11-28T18:30:51.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.793+0000 7f177f7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f17800caf60 0x7f17800ad850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42122/0 (socket says 172.21.15.105:42122) 2023-11-28T18:30:51.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.793+0000 7f177f7fe640 1 -- 172.21.15.105:0/2932068442 learned_addr learned my addr 172.21.15.105:0/2932068442 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:51.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.793+0000 7f1784f83640 1 --2- 172.21.15.105:0/2932068442 >> v2:172.21.15.148:3300/0 conn(0x7f17800cc760 0x7f1780164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:51.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.793+0000 7f177effd640 1 -- 172.21.15.105:0/2932068442 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1482617559 0 0) 0x7f17800ccc10 con 0x7f17800caf60 2023-11-28T18:30:51.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.793+0000 7f177effd640 1 -- 172.21.15.105:0/2932068442 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1768005740 con 0x7f17800caf60 2023-11-28T18:30:51.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.793+0000 7f177effd640 1 -- 172.21.15.105:0/2932068442 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540607838 0 0) 0x7f178006bf50 con 0x7f1780159f40 2023-11-28T18:30:51.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.793+0000 7f177effd640 1 -- 172.21.15.105:0/2932068442 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17800ccc10 con 0x7f1780159f40 2023-11-28T18:30:51.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.793+0000 7f177effd640 1 -- 172.21.15.105:0/2932068442 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1644433214 0 0) 0x7f1768005740 con 0x7f17800caf60 2023-11-28T18:30:51.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.793+0000 7f1784f83640 1 -- 172.21.15.105:0/2932068442 >> v1:172.21.15.148:6789/0 conn(0x7f1780159f40 legacy=0x7f17800ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:51.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.793+0000 7f1784f83640 1 -- 172.21.15.105:0/2932068442 >> v1:172.21.15.105:6789/0 conn(0x7f17800caf60 legacy=0x7f17800ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:51.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.793+0000 7f1784f83640 1 -- 172.21.15.105:0/2932068442 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178015a320 con 0x7f17800cc760 2023-11-28T18:30:51.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.797+0000 7f1784f83640 1 --2- 172.21.15.105:0/2932068442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17800cc760 0x7f1780164d90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1778009d90 tx=0x7f177801c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43c3cce1c871e9ca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:51.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.797+0000 7f177effd640 1 -- 172.21.15.105:0/2932068442 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1774001680 con 0x7f17800cc760 2023-11-28T18:30:51.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.797+0000 7f177effd640 1 -- 172.21.15.105:0/2932068442 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17780a6500 con 0x7f17800cc760 2023-11-28T18:30:51.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.797+0000 7f177effd640 1 -- 172.21.15.105:0/2932068442 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17780a6870 con 0x7f17800cc760 2023-11-28T18:30:51.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.797+0000 7f17869e6640 1 -- 172.21.15.105:0/2932068442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17800cc760 msgr2=0x7f1780164d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:51.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.797+0000 7f17869e6640 1 --2- 172.21.15.105:0/2932068442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17800cc760 0x7f1780164d90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1778009d90 tx=0x7f177801c7b0 comp rx=0 tx=0).stop 2023-11-28T18:30:51.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.797+0000 7f17869e6640 1 -- 172.21.15.105:0/2932068442 shutdown_connections 2023-11-28T18:30:51.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.797+0000 7f17869e6640 1 --2- 172.21.15.105:0/2932068442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17800cc760 0x7f1780164d90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:51.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.797+0000 7f17869e6640 1 -- 172.21.15.105:0/2932068442 >> 172.21.15.105:0/2932068442 conn(0x7f17800ceb90 msgr2=0x7f17800cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:51.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.797+0000 7f17869e6640 1 -- 172.21.15.105:0/2932068442 shutdown_connections 2023-11-28T18:30:51.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.797+0000 7f17869e6640 1 -- 172.21.15.105:0/2932068442 wait complete. 2023-11-28T18:30:51.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.797+0000 7f17869e6640 1 Processor -- start 2023-11-28T18:30:51.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.797+0000 7f17869e6640 1 -- start start 2023-11-28T18:30:51.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f17869e6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17800caf60 0x7f178015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:51.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f17869e6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17800cc760 0x7f17801450e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:51.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f17869e6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1780159f40 0x7f178015aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:51.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f17869e6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178014ab90 con 0x7f17800caf60 2023-11-28T18:30:51.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f17869e6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1780149f80 con 0x7f17800cc760 2023-11-28T18:30:51.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f17869e6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178014a260 con 0x7f1780159f40 2023-11-28T18:30:51.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f177ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17800caf60 0x7f178015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:51.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f177ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17800caf60 0x7f178015a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59724/0 (socket says 172.21.15.105:59724) 2023-11-28T18:30:51.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f177ffff640 1 -- 172.21.15.105:0/1953934488 learned_addr learned my addr 172.21.15.105:0/1953934488 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:51.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f177f7fe640 1 --2- 172.21.15.105:0/1953934488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17800cc760 0x7f17801450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f1784f83640 1 --2- 172.21.15.105:0/1953934488 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1780159f40 0x7f178015aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f177ffff640 1 -- 172.21.15.105:0/1953934488 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1780159f40 msgr2=0x7f178015aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f177ffff640 1 --2- 172.21.15.105:0/1953934488 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1780159f40 0x7f178015aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f177ffff640 1 -- 172.21.15.105:0/1953934488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17800cc760 msgr2=0x7f17801450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f177ffff640 1 --2- 172.21.15.105:0/1953934488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17800cc760 0x7f17801450e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f177ffff640 1 -- 172.21.15.105:0/1953934488 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1778009a70 con 0x7f17800caf60 2023-11-28T18:30:51.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f177ffff640 1 --2- 172.21.15.105:0/1953934488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17800caf60 0x7f178015a4b0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f177000c430 tx=0x7f177000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:51.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f177cff9640 1 -- 172.21.15.105:0/1953934488 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17700052d0 con 0x7f17800caf60 2023-11-28T18:30:51.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f177cff9640 1 -- 172.21.15.105:0/1953934488 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1770005450 con 0x7f17800caf60 2023-11-28T18:30:51.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f177cff9640 1 -- 172.21.15.105:0/1953934488 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1770003840 con 0x7f17800caf60 2023-11-28T18:30:51.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f17869e6640 1 -- 172.21.15.105:0/1953934488 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178014a540 con 0x7f17800caf60 2023-11-28T18:30:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f17869e6640 1 -- 172.21.15.105:0/1953934488 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1780145620 con 0x7f17800caf60 2023-11-28T18:30:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f17869e6640 1 -- 172.21.15.105:0/1953934488 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1780067570 con 0x7f17800caf60 2023-11-28T18:30:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.801+0000 7f177cff9640 1 -- 172.21.15.105:0/1953934488 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f17700039a0 con 0x7f17800caf60 2023-11-28T18:30:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.805+0000 7f177cff9640 1 --2- 172.21.15.105:0/1953934488 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f176c0668d0 0x7f176c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.805+0000 7f177cff9640 1 -- 172.21.15.105:0/1953934488 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5878+0+0 (secure 0 0 0) 0x7f1770083b40 con 0x7f17800caf60 2023-11-28T18:30:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.805+0000 7f177f7fe640 1 --2- 172.21.15.105:0/1953934488 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f176c0668d0 0x7f176c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.805+0000 7f177f7fe640 1 --2- 172.21.15.105:0/1953934488 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f176c0668d0 0x7f176c068d90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f17800cd520 tx=0x7f1774009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:51.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.809+0000 7f177cff9640 1 -- 172.21.15.105:0/1953934488 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1770050cc0 con 0x7f17800caf60 2023-11-28T18:30:51.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:51.957+0000 7f17869e6640 1 -- 172.21.15.105:0/1953934488 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f17800cf950 con 0x7f17800caf60 2023-11-28T18:30:52.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.625+0000 7f177cff9640 1 -- 172.21.15.105:0/1953934488 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v108) v1 ==== 169+0+0 (secure 0 0 0) 0x7f17700170d0 con 0x7f17800caf60 2023-11-28T18:30:52.625 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' removed 2023-11-28T18:30:52.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.625+0000 7f17869e6640 1 -- 172.21.15.105:0/1953934488 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f176c0668d0 msgr2=0x7f176c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:52.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.625+0000 7f17869e6640 1 --2- 172.21.15.105:0/1953934488 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f176c0668d0 0x7f176c068d90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f17800cd520 tx=0x7f1774009040 comp rx=0 tx=0).stop 2023-11-28T18:30:52.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.625+0000 7f17869e6640 1 -- 172.21.15.105:0/1953934488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17800caf60 msgr2=0x7f178015a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:52.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.625+0000 7f17869e6640 1 --2- 172.21.15.105:0/1953934488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17800caf60 0x7f178015a4b0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f177000c430 tx=0x7f177000c8f0 comp rx=0 tx=0).stop 2023-11-28T18:30:52.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.625+0000 7f17869e6640 1 -- 172.21.15.105:0/1953934488 shutdown_connections 2023-11-28T18:30:52.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.625+0000 7f17869e6640 1 --2- 172.21.15.105:0/1953934488 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1780159f40 0x7f178015aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:52.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.625+0000 7f17869e6640 1 --2- 172.21.15.105:0/1953934488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17800cc760 0x7f17801450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:52.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.625+0000 7f17869e6640 1 --2- 172.21.15.105:0/1953934488 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f176c0668d0 0x7f176c068d90 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:52.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.625+0000 7f17869e6640 1 --2- 172.21.15.105:0/1953934488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17800caf60 0x7f178015a4b0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:52.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.625+0000 7f17869e6640 1 -- 172.21.15.105:0/1953934488 >> 172.21.15.105:0/1953934488 conn(0x7f17800ceb90 msgr2=0x7f1780070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:52.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.629+0000 7f17869e6640 1 -- 172.21.15.105:0/1953934488 shutdown_connections 2023-11-28T18:30:52.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.629+0000 7f17869e6640 1 -- 172.21.15.105:0/1953934488 wait complete. 2023-11-28T18:30:52.656 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random 2023-11-28T18:30:52.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.797+0000 7f52c8aad640 1 Processor -- start 2023-11-28T18:30:52.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.797+0000 7f52c8aad640 1 -- start start 2023-11-28T18:30:52.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.797+0000 7f52c8aad640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f52c40d1b00 0x7f52c4168090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:52.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.797+0000 7f52c8aad640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c40cf440 con 0x7f52c40d0d40 2023-11-28T18:30:52.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.797+0000 7f52c8aad640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c40cffc0 con 0x7f52c415d1a0 2023-11-28T18:30:52.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.797+0000 7f52c8aad640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c40ceb40 con 0x7f52c40d1b00 2023-11-28T18:30:52.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.797+0000 7f52c3fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f52c40d1b00 0x7f52c4168090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:52.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.797+0000 7f52c3fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f52c40d1b00 0x7f52c4168090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59730/0 (socket says 172.21.15.105:59730) 2023-11-28T18:30:52.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.797+0000 7f52c3fff640 1 -- 172.21.15.105:0/3249580083 learned_addr learned my addr 172.21.15.105:0/3249580083 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:52.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.797+0000 7f52c2ffd640 1 --1- 172.21.15.105:0/3249580083 >> v1:172.21.15.105:6789/0 conn(0x7f52c40d0d40 0x7f52c40d13f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42126/0 (socket says 172.21.15.105:42126) 2023-11-28T18:30:52.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.797+0000 7f52c3fff640 1 -- 172.21.15.105:0/3249580083 >> v1:172.21.15.148:6789/0 conn(0x7f52c415d1a0 legacy=0x7f52c40ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:52.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.797+0000 7f52c27fc640 1 -- 172.21.15.105:0/3249580083 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230109633 0 0) 0x7f52c40cf440 con 0x7f52c40d0d40 2023-11-28T18:30:52.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.797+0000 7f52c3fff640 1 -- 172.21.15.105:0/3249580083 >> v1:172.21.15.105:6789/0 conn(0x7f52c40d0d40 legacy=0x7f52c40d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:52.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.797+0000 7f52c3fff640 1 -- 172.21.15.105:0/3249580083 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c40cf690 con 0x7f52c40d1b00 2023-11-28T18:30:52.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.801+0000 7f52c3fff640 1 --2- 172.21.15.105:0/3249580083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c40d1b00 0x7f52c4168090 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009e50 tx=0x7f52bc01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1bcf951b4381d46 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:52.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.801+0000 7f52c27fc640 1 -- 172.21.15.105:0/3249580083 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc0a5810 con 0x7f52c40d1b00 2023-11-28T18:30:52.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.801+0000 7f52c27fc640 1 -- 172.21.15.105:0/3249580083 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc0a5970 con 0x7f52c40d1b00 2023-11-28T18:30:52.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.801+0000 7f52c27fc640 1 -- 172.21.15.105:0/3249580083 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc0a5d00 con 0x7f52c40d1b00 2023-11-28T18:30:52.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.801+0000 7f52c8aad640 1 -- 172.21.15.105:0/3249580083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c40d1b00 msgr2=0x7f52c4168090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:52.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.801+0000 7f52c8aad640 1 --2- 172.21.15.105:0/3249580083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c40d1b00 0x7f52c4168090 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009e50 tx=0x7f52bc01c950 comp rx=0 tx=0).stop 2023-11-28T18:30:52.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.801+0000 7f52c8aad640 1 -- 172.21.15.105:0/3249580083 shutdown_connections 2023-11-28T18:30:52.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.801+0000 7f52c8aad640 1 --2- 172.21.15.105:0/3249580083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c40d1b00 0x7f52c4168090 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:52.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.801+0000 7f52c8aad640 1 -- 172.21.15.105:0/3249580083 >> 172.21.15.105:0/3249580083 conn(0x7f52c40d2e10 msgr2=0x7f52c40d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:52.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.801+0000 7f52c8aad640 1 -- 172.21.15.105:0/3249580083 shutdown_connections 2023-11-28T18:30:52.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.801+0000 7f52c8aad640 1 -- 172.21.15.105:0/3249580083 wait complete. 2023-11-28T18:30:52.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.805+0000 7f52c8aad640 1 Processor -- start 2023-11-28T18:30:52.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.805+0000 7f52c8aad640 1 -- start start 2023-11-28T18:30:52.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.805+0000 7f52c8aad640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52c40d0d40 0x7f52c41530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:52.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.805+0000 7f52c8aad640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c40d1b00 0x7f52c4154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:52.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.805+0000 7f52c8aad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52c415d1a0 0x7f52c415b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:52.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.805+0000 7f52c8aad640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c415b370 con 0x7f52c40d1b00 2023-11-28T18:30:52.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.805+0000 7f52c8aad640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c415a780 con 0x7f52c415d1a0 2023-11-28T18:30:52.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.805+0000 7f52c8aad640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c415a9d0 con 0x7f52c40d0d40 2023-11-28T18:30:52.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.805+0000 7f52c2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c40d1b00 0x7f52c4154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:52.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.805+0000 7f52c2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c40d1b00 0x7f52c4154f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59740/0 (socket says 172.21.15.105:59740) 2023-11-28T18:30:52.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.805+0000 7f52c2ffd640 1 -- 172.21.15.105:0/600646848 learned_addr learned my addr 172.21.15.105:0/600646848 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:52.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.805+0000 7f52c2ffd640 1 -- 172.21.15.105:0/600646848 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52c40d0d40 msgr2=0x7f52c41530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:52.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.805+0000 7f52c2ffd640 1 --2- 172.21.15.105:0/600646848 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52c40d0d40 0x7f52c41530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:52.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.805+0000 7f52c2ffd640 1 -- 172.21.15.105:0/600646848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52c415d1a0 msgr2=0x7f52c415b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:52.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.805+0000 7f52c2ffd640 1 --2- 172.21.15.105:0/600646848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52c415d1a0 0x7f52c415b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:52.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.805+0000 7f52c2ffd640 1 -- 172.21.15.105:0/600646848 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52bc009b30 con 0x7f52c40d1b00 2023-11-28T18:30:52.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.805+0000 7f52c2ffd640 1 --2- 172.21.15.105:0/600646848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c40d1b00 0x7f52c4154f30 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f52b800c4a0 tx=0x7f52b800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:52.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.805+0000 7f52abfff640 1 -- 172.21.15.105:0/600646848 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b80074e0 con 0x7f52c40d1b00 2023-11-28T18:30:52.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.805+0000 7f52abfff640 1 -- 172.21.15.105:0/600646848 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b8007640 con 0x7f52c40d1b00 2023-11-28T18:30:52.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.805+0000 7f52c8aad640 1 -- 172.21.15.105:0/600646848 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c415acb0 con 0x7f52c40d1b00 2023-11-28T18:30:52.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.805+0000 7f52abfff640 1 -- 172.21.15.105:0/600646848 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b8004a30 con 0x7f52c40d1b00 2023-11-28T18:30:52.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.805+0000 7f52c8aad640 1 -- 172.21.15.105:0/600646848 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c41535e0 con 0x7f52c40d1b00 2023-11-28T18:30:52.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.809+0000 7f52c8aad640 1 -- 172.21.15.105:0/600646848 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c4074580 con 0x7f52c40d1b00 2023-11-28T18:30:52.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.809+0000 7f52abfff640 1 -- 172.21.15.105:0/600646848 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f52b8004b90 con 0x7f52c40d1b00 2023-11-28T18:30:52.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.809+0000 7f52abfff640 1 --2- 172.21.15.105:0/600646848 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52b0066900 0x7f52b0068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:52.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.809+0000 7f52abfff640 1 -- 172.21.15.105:0/600646848 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f52b8083500 con 0x7f52c40d1b00 2023-11-28T18:30:52.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.809+0000 7f52c37fe640 1 --2- 172.21.15.105:0/600646848 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52b0066900 0x7f52b0068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:52.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.809+0000 7f52c37fe640 1 --2- 172.21.15.105:0/600646848 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52b0066900 0x7f52b0068dc0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f52b4000f70 tx=0x7f52b40079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:52.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.813+0000 7f52abfff640 1 -- 172.21.15.105:0/600646848 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f52b8050840 con 0x7f52c40d1b00 2023-11-28T18:30:52.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.973+0000 7f52c8aad640 1 -- 172.21.15.105:0/600646848 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"} v 0) v1 -- 0x7f52c40b5730 con 0x7f52c40d1b00 2023-11-28T18:30:52.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.977+0000 7f52abfff640 1 -- 172.21.15.105:0/600646848 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"}]=0 v55) v1 ==== 110+0+0 (secure 0 0 0) 0x7f52b8053e60 con 0x7f52c40d1b00 2023-11-28T18:30:52.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.981+0000 7f52c8aad640 1 -- 172.21.15.105:0/600646848 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52b0066900 msgr2=0x7f52b0068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:52.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.981+0000 7f52c8aad640 1 --2- 172.21.15.105:0/600646848 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52b0066900 0x7f52b0068dc0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f52b4000f70 tx=0x7f52b40079c0 comp rx=0 tx=0).stop 2023-11-28T18:30:52.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.981+0000 7f52c8aad640 1 -- 172.21.15.105:0/600646848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c40d1b00 msgr2=0x7f52c4154f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:52.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.981+0000 7f52c8aad640 1 --2- 172.21.15.105:0/600646848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c40d1b00 0x7f52c4154f30 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f52b800c4a0 tx=0x7f52b800c960 comp rx=0 tx=0).stop 2023-11-28T18:30:52.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.981+0000 7f52c8aad640 1 -- 172.21.15.105:0/600646848 shutdown_connections 2023-11-28T18:30:52.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.981+0000 7f52c8aad640 1 --2- 172.21.15.105:0/600646848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52c415d1a0 0x7f52c415b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:52.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.981+0000 7f52c8aad640 1 --2- 172.21.15.105:0/600646848 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f52b0066900 0x7f52b0068dc0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:52.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.981+0000 7f52c8aad640 1 --2- 172.21.15.105:0/600646848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c40d1b00 0x7f52c4154f30 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:52.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.981+0000 7f52c8aad640 1 --2- 172.21.15.105:0/600646848 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52c40d0d40 0x7f52c41530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:52.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.981+0000 7f52c8aad640 1 -- 172.21.15.105:0/600646848 >> 172.21.15.105:0/600646848 conn(0x7f52c40d2e10 msgr2=0x7f52c40afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:52.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.981+0000 7f52c8aad640 1 -- 172.21.15.105:0/600646848 shutdown_connections 2023-11-28T18:30:52.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:52.981+0000 7f52c8aad640 1 -- 172.21.15.105:0/600646848 wait complete. 2023-11-28T18:30:53.000 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T18:30:53.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.141+0000 7fc17010f640 1 Processor -- start 2023-11-28T18:30:53.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.141+0000 7fc17010f640 1 -- start start 2023-11-28T18:30:53.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.141+0000 7fc17010f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc16815d4f0 0x7fc1680d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:53.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.141+0000 7fc17010f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc1680d3c30 0x7fc1680d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:53.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.141+0000 7fc17010f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1680b25b0 con 0x7fc1680d1890 2023-11-28T18:30:53.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.141+0000 7fc17010f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1680b2900 con 0x7fc1680d3c30 2023-11-28T18:30:53.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.141+0000 7fc17010f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1680cea30 con 0x7fc16815d4f0 2023-11-28T18:30:53.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.141+0000 7fc16d6aa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc1680d3c30 0x7fc1680d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:53.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.141+0000 7fc16d6aa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc1680d3c30 0x7fc1680d1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59744/0 (socket says 172.21.15.105:59744) 2023-11-28T18:30:53.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.141+0000 7fc16d6aa640 1 -- 172.21.15.105:0/4097354505 learned_addr learned my addr 172.21.15.105:0/4097354505 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:53.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.141+0000 7fc16deab640 1 --2- 172.21.15.105:0/4097354505 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc16815d4f0 0x7fc1680d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:53.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.141+0000 7fc16d6aa640 1 -- 172.21.15.105:0/4097354505 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc16815d4f0 msgr2=0x7fc1680d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:53.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.141+0000 7fc16d6aa640 1 --2- 172.21.15.105:0/4097354505 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc16815d4f0 0x7fc1680d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.141+0000 7fc16d6aa640 1 -- 172.21.15.105:0/4097354505 >> v1:172.21.15.105:6789/0 conn(0x7fc1680d1890 legacy=0x7fc1680cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:53.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.141+0000 7fc16d6aa640 1 -- 172.21.15.105:0/4097354505 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1680ced10 con 0x7fc1680d3c30 2023-11-28T18:30:53.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.145+0000 7fc16d6aa640 1 --2- 172.21.15.105:0/4097354505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1680d3c30 0x7fc1680d1350 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fc158009870 tx=0x7fc15801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f86d2aaeac69b48c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:53.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.145+0000 7fc16cea9640 1 -- 172.21.15.105:0/4097354505 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1580ad640 con 0x7fc1680d3c30 2023-11-28T18:30:53.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.145+0000 7fc16cea9640 1 -- 172.21.15.105:0/4097354505 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1580ad7a0 con 0x7fc1680d3c30 2023-11-28T18:30:53.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.145+0000 7fc16cea9640 1 -- 172.21.15.105:0/4097354505 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1580adb30 con 0x7fc1680d3c30 2023-11-28T18:30:53.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.145+0000 7fc17010f640 1 -- 172.21.15.105:0/4097354505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1680d3c30 msgr2=0x7fc1680d1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:53.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.145+0000 7fc17010f640 1 --2- 172.21.15.105:0/4097354505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1680d3c30 0x7fc1680d1350 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fc158009870 tx=0x7fc15801c480 comp rx=0 tx=0).stop 2023-11-28T18:30:53.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.145+0000 7fc17010f640 1 -- 172.21.15.105:0/4097354505 shutdown_connections 2023-11-28T18:30:53.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.145+0000 7fc17010f640 1 --2- 172.21.15.105:0/4097354505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1680d3c30 0x7fc1680d1350 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.145+0000 7fc17010f640 1 --2- 172.21.15.105:0/4097354505 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc16815d4f0 0x7fc1680d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.145+0000 7fc17010f640 1 -- 172.21.15.105:0/4097354505 >> 172.21.15.105:0/4097354505 conn(0x7fc1680d3030 msgr2=0x7fc1680d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:53.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.145+0000 7fc17010f640 1 -- 172.21.15.105:0/4097354505 shutdown_connections 2023-11-28T18:30:53.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.145+0000 7fc17010f640 1 -- 172.21.15.105:0/4097354505 wait complete. 2023-11-28T18:30:53.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc17010f640 1 Processor -- start 2023-11-28T18:30:53.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc17010f640 1 -- start start 2023-11-28T18:30:53.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc17010f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1680d1890 0x7fc16815db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:53.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc17010f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1680d3c30 0x7fc16814f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:53.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc17010f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc16815d4f0 0x7fc168154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:53.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc17010f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1681547c0 con 0x7fc16815d4f0 2023-11-28T18:30:53.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc17010f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1681541b0 con 0x7fc1680d1890 2023-11-28T18:30:53.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc17010f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1680cba90 con 0x7fc1680d3c30 2023-11-28T18:30:53.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc16e6ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc16815d4f0 0x7fc168154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:53.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc16e6ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc16815d4f0 0x7fc168154c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59754/0 (socket says 172.21.15.105:59754) 2023-11-28T18:30:53.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc16e6ac640 1 -- 172.21.15.105:0/3358054651 learned_addr learned my addr 172.21.15.105:0/3358054651 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:53.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc16deab640 1 --2- 172.21.15.105:0/3358054651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1680d1890 0x7fc16815db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:53.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc16d6aa640 1 --2- 172.21.15.105:0/3358054651 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1680d3c30 0x7fc16814f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:53.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc16e6ac640 1 -- 172.21.15.105:0/3358054651 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1680d3c30 msgr2=0x7fc16814f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:53.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc16e6ac640 1 --2- 172.21.15.105:0/3358054651 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1680d3c30 0x7fc16814f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc16e6ac640 1 -- 172.21.15.105:0/3358054651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1680d1890 msgr2=0x7fc16815db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:53.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc16e6ac640 1 --2- 172.21.15.105:0/3358054651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1680d1890 0x7fc16815db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc16e6ac640 1 -- 172.21.15.105:0/3358054651 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1580094d0 con 0x7fc16815d4f0 2023-11-28T18:30:53.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc16deab640 1 --2- 172.21.15.105:0/3358054651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1680d1890 0x7fc16815db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:53.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc16e6ac640 1 --2- 172.21.15.105:0/3358054651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc16815d4f0 0x7fc168154c50 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fc16000c730 tx=0x7fc16000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:53.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc15e7fc640 1 -- 172.21.15.105:0/3358054651 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc160003b80 con 0x7fc16815d4f0 2023-11-28T18:30:53.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc15e7fc640 1 -- 172.21.15.105:0/3358054651 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc160003ce0 con 0x7fc16815d4f0 2023-11-28T18:30:53.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc17010f640 1 -- 172.21.15.105:0/3358054651 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16814fd50 con 0x7fc16815d4f0 2023-11-28T18:30:53.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc15e7fc640 1 -- 172.21.15.105:0/3358054651 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16000aa30 con 0x7fc16815d4f0 2023-11-28T18:30:53.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc17010f640 1 -- 172.21.15.105:0/3358054651 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc168154450 con 0x7fc16815d4f0 2023-11-28T18:30:53.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.149+0000 7fc17010f640 1 -- 172.21.15.105:0/3358054651 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc168154010 con 0x7fc16815d4f0 2023-11-28T18:30:53.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.153+0000 7fc15e7fc640 1 -- 172.21.15.105:0/3358054651 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc16000ab90 con 0x7fc16815d4f0 2023-11-28T18:30:53.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.153+0000 7fc15e7fc640 1 --2- 172.21.15.105:0/3358054651 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc14c0668d0 0x7fc14c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:53.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.153+0000 7fc15e7fc640 1 -- 172.21.15.105:0/3358054651 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fc160084120 con 0x7fc16815d4f0 2023-11-28T18:30:53.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.153+0000 7fc16deab640 1 --2- 172.21.15.105:0/3358054651 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc14c0668d0 0x7fc14c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:53.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.153+0000 7fc16deab640 1 --2- 172.21.15.105:0/3358054651 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc14c0668d0 0x7fc14c068d90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc1680c98f0 tx=0x7fc164008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:53.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.157+0000 7fc15e7fc640 1 -- 172.21.15.105:0/3358054651 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc168154010 con 0x7fc16815d4f0 2023-11-28T18:30:53.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.309+0000 7fc17010f640 1 -- 172.21.15.105:0/3358054651 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc168154010 con 0x7fc16815d4f0 2023-11-28T18:30:53.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.313+0000 7fc15e7fc640 1 -- 172.21.15.105:0/3358054651 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v56) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc168154010 con 0x7fc16815d4f0 2023-11-28T18:30:53.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.313+0000 7fc17010f640 1 -- 172.21.15.105:0/3358054651 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc14c0668d0 msgr2=0x7fc14c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:53.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.313+0000 7fc17010f640 1 --2- 172.21.15.105:0/3358054651 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc14c0668d0 0x7fc14c068d90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc1680c98f0 tx=0x7fc164008040 comp rx=0 tx=0).stop 2023-11-28T18:30:53.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.313+0000 7fc17010f640 1 -- 172.21.15.105:0/3358054651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc16815d4f0 msgr2=0x7fc168154c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:53.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.313+0000 7fc17010f640 1 --2- 172.21.15.105:0/3358054651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc16815d4f0 0x7fc168154c50 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fc16000c730 tx=0x7fc16000cbf0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.317+0000 7fc17010f640 1 -- 172.21.15.105:0/3358054651 shutdown_connections 2023-11-28T18:30:53.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.317+0000 7fc17010f640 1 --2- 172.21.15.105:0/3358054651 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc14c0668d0 0x7fc14c068d90 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.317+0000 7fc17010f640 1 --2- 172.21.15.105:0/3358054651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc16815d4f0 0x7fc168154c50 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.317+0000 7fc17010f640 1 --2- 172.21.15.105:0/3358054651 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1680d3c30 0x7fc16814f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.317+0000 7fc17010f640 1 --2- 172.21.15.105:0/3358054651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1680d1890 0x7fc16815db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.317+0000 7fc17010f640 1 -- 172.21.15.105:0/3358054651 >> 172.21.15.105:0/3358054651 conn(0x7fc1680d3030 msgr2=0x7fc1680cde80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:53.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.317+0000 7fc17010f640 1 -- 172.21.15.105:0/3358054651 shutdown_connections 2023-11-28T18:30:53.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.317+0000 7fc17010f640 1 -- 172.21.15.105:0/3358054651 wait complete. 2023-11-28T18:30:53.334 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_distributed 2023-11-28T18:30:53.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.481+0000 7fc607757640 1 Processor -- start 2023-11-28T18:30:53.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.481+0000 7fc607757640 1 -- start start 2023-11-28T18:30:53.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.481+0000 7fc607757640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc60015d3c0 0x7fc600071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:53.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.481+0000 7fc607757640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc600070a10 0x7fc600070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:53.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.481+0000 7fc607757640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6000cefa0 con 0x7fc600161870 2023-11-28T18:30:53.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.481+0000 7fc607757640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc600071360 con 0x7fc600070a10 2023-11-28T18:30:53.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.481+0000 7fc607757640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc60015d7a0 con 0x7fc60015d3c0 2023-11-28T18:30:53.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.481+0000 7fc605cf4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc600070a10 0x7fc600070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:53.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.481+0000 7fc605cf4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc600070a10 0x7fc600070e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59760/0 (socket says 172.21.15.105:59760) 2023-11-28T18:30:53.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.481+0000 7fc605cf4640 1 -- 172.21.15.105:0/284982637 learned_addr learned my addr 172.21.15.105:0/284982637 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:53.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.481+0000 7fc605cf4640 1 -- 172.21.15.105:0/284982637 >> v2:172.21.15.148:3300/0 conn(0x7fc60015d3c0 msgr2=0x7fc600071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:53.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.481+0000 7fc605cf4640 1 --2- 172.21.15.105:0/284982637 >> v2:172.21.15.148:3300/0 conn(0x7fc60015d3c0 0x7fc600071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.481+0000 7fc605cf4640 1 -- 172.21.15.105:0/284982637 >> v1:172.21.15.148:6789/0 conn(0x7fc600161870 legacy=0x7fc6000ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:53.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.481+0000 7fc605cf4640 1 -- 172.21.15.105:0/284982637 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6000ce9e0 con 0x7fc600070a10 2023-11-28T18:30:53.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.485+0000 7fc605cf4640 1 --2- 172.21.15.105:0/284982637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc600070a10 0x7fc600070e20 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fc60006ba90 tx=0x7fc5f801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a79c715b851d7a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:53.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.485+0000 7fc5f7fff640 1 -- 172.21.15.105:0/284982637 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f80a5820 con 0x7fc600070a10 2023-11-28T18:30:53.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.485+0000 7fc5f7fff640 1 -- 172.21.15.105:0/284982637 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f80a5980 con 0x7fc600070a10 2023-11-28T18:30:53.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.485+0000 7fc5f7fff640 1 -- 172.21.15.105:0/284982637 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f80a5d10 con 0x7fc600070a10 2023-11-28T18:30:53.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.485+0000 7fc607757640 1 -- 172.21.15.105:0/284982637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc600070a10 msgr2=0x7fc600070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:53.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.485+0000 7fc607757640 1 --2- 172.21.15.105:0/284982637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc600070a10 0x7fc600070e20 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fc60006ba90 tx=0x7fc5f801c960 comp rx=0 tx=0).stop 2023-11-28T18:30:53.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.485+0000 7fc607757640 1 -- 172.21.15.105:0/284982637 shutdown_connections 2023-11-28T18:30:53.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.485+0000 7fc607757640 1 --2- 172.21.15.105:0/284982637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc600070a10 0x7fc600070e20 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.485+0000 7fc607757640 1 --2- 172.21.15.105:0/284982637 >> v2:172.21.15.148:3300/0 conn(0x7fc60015d3c0 0x7fc600071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.485+0000 7fc607757640 1 -- 172.21.15.105:0/284982637 >> 172.21.15.105:0/284982637 conn(0x7fc6000cf540 msgr2=0x7fc6000cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:53.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.485+0000 7fc607757640 1 -- 172.21.15.105:0/284982637 shutdown_connections 2023-11-28T18:30:53.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.485+0000 7fc607757640 1 -- 172.21.15.105:0/284982637 wait complete. 2023-11-28T18:30:53.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc607757640 1 Processor -- start 2023-11-28T18:30:53.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc607757640 1 -- start start 2023-11-28T18:30:53.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc607757640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc600070a10 0x7fc600156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:53.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc607757640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc60015d3c0 0x7fc60013fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:53.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc607757640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc600161870 0x7fc600155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:53.491 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc607757640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc600156940 con 0x7fc600161870 2023-11-28T18:30:53.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc607757640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60015aaf0 con 0x7fc60015d3c0 2023-11-28T18:30:53.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc607757640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60015add0 con 0x7fc600070a10 2023-11-28T18:30:53.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc605cf4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc600161870 0x7fc600155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:53.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc605cf4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc600161870 0x7fc600155ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59776/0 (socket says 172.21.15.105:59776) 2023-11-28T18:30:53.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc605cf4640 1 -- 172.21.15.105:0/1587343805 learned_addr learned my addr 172.21.15.105:0/1587343805 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:53.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc6054f3640 1 --2- 172.21.15.105:0/1587343805 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc600070a10 0x7fc600156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:53.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc604cf2640 1 --2- 172.21.15.105:0/1587343805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc60015d3c0 0x7fc60013fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:53.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc605cf4640 1 -- 172.21.15.105:0/1587343805 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc600070a10 msgr2=0x7fc600156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:53.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc605cf4640 1 --2- 172.21.15.105:0/1587343805 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc600070a10 0x7fc600156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.492 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc605cf4640 1 -- 172.21.15.105:0/1587343805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc60015d3c0 msgr2=0x7fc60013fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:53.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc605cf4640 1 --2- 172.21.15.105:0/1587343805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc60015d3c0 0x7fc60013fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc605cf4640 1 -- 172.21.15.105:0/1587343805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f8009b30 con 0x7fc600161870 2023-11-28T18:30:53.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc605cf4640 1 --2- 172.21.15.105:0/1587343805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc600161870 0x7fc600155ed0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fc5f801a590 tx=0x7fc5f8019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:53.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc5f5ffb640 1 -- 172.21.15.105:0/1587343805 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f80b8880 con 0x7fc600161870 2023-11-28T18:30:53.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc5f5ffb640 1 -- 172.21.15.105:0/1587343805 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f80b8a00 con 0x7fc600161870 2023-11-28T18:30:53.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc607757640 1 -- 172.21.15.105:0/1587343805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc600161e80 con 0x7fc600161870 2023-11-28T18:30:53.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc5f5ffb640 1 -- 172.21.15.105:0/1587343805 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f80b8db0 con 0x7fc600161870 2023-11-28T18:30:53.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.489+0000 7fc607757640 1 -- 172.21.15.105:0/1587343805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc60015b390 con 0x7fc600161870 2023-11-28T18:30:53.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.493+0000 7fc607757640 1 -- 172.21.15.105:0/1587343805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc60015b580 con 0x7fc600161870 2023-11-28T18:30:53.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.493+0000 7fc5f5ffb640 1 -- 172.21.15.105:0/1587343805 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc5f80b6440 con 0x7fc600161870 2023-11-28T18:30:53.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.493+0000 7fc5f5ffb640 1 --2- 172.21.15.105:0/1587343805 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc5e40668d0 0x7fc5e4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:53.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.493+0000 7fc5f5ffb640 1 -- 172.21.15.105:0/1587343805 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fc5f811faf0 con 0x7fc600161870 2023-11-28T18:30:53.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.493+0000 7fc6054f3640 1 --2- 172.21.15.105:0/1587343805 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc5e40668d0 0x7fc5e4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:53.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.493+0000 7fc6054f3640 1 --2- 172.21.15.105:0/1587343805 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc5e40668d0 0x7fc5e4068d90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc006fd0 tx=0x7fc5fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:53.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.497+0000 7fc5f5ffb640 1 -- 172.21.15.105:0/1587343805 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc5f80ecf50 con 0x7fc600161870 2023-11-28T18:30:53.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.649+0000 7fc607757640 1 -- 172.21.15.105:0/1587343805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"} v 0) v1 -- 0x7fc600067570 con 0x7fc600161870 2023-11-28T18:30:53.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.653+0000 7fc5f5ffb640 1 -- 172.21.15.105:0/1587343805 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"}]=0 v57) v1 ==== 115+0+0 (secure 0 0 0) 0x7fc5f80bc020 con 0x7fc600161870 2023-11-28T18:30:53.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.657+0000 7fc607757640 1 -- 172.21.15.105:0/1587343805 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc5e40668d0 msgr2=0x7fc5e4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:53.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.657+0000 7fc607757640 1 --2- 172.21.15.105:0/1587343805 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc5e40668d0 0x7fc5e4068d90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc006fd0 tx=0x7fc5fc008040 comp rx=0 tx=0).stop 2023-11-28T18:30:53.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.657+0000 7fc607757640 1 -- 172.21.15.105:0/1587343805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc600161870 msgr2=0x7fc600155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:53.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.657+0000 7fc607757640 1 --2- 172.21.15.105:0/1587343805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc600161870 0x7fc600155ed0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fc5f801a590 tx=0x7fc5f8019700 comp rx=0 tx=0).stop 2023-11-28T18:30:53.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.657+0000 7fc607757640 1 -- 172.21.15.105:0/1587343805 shutdown_connections 2023-11-28T18:30:53.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.657+0000 7fc607757640 1 --2- 172.21.15.105:0/1587343805 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc5e40668d0 0x7fc5e4068d90 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.657+0000 7fc607757640 1 --2- 172.21.15.105:0/1587343805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc600161870 0x7fc600155ed0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.657+0000 7fc607757640 1 --2- 172.21.15.105:0/1587343805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc60015d3c0 0x7fc60013fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.657+0000 7fc607757640 1 --2- 172.21.15.105:0/1587343805 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc600070a10 0x7fc600156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.657+0000 7fc607757640 1 -- 172.21.15.105:0/1587343805 >> 172.21.15.105:0/1587343805 conn(0x7fc6000cf540 msgr2=0x7fc6000af880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:53.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.657+0000 7fc607757640 1 -- 172.21.15.105:0/1587343805 shutdown_connections 2023-11-28T18:30:53.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.657+0000 7fc607757640 1 -- 172.21.15.105:0/1587343805 wait complete. 2023-11-28T18:30:53.674 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random_max 2023-11-28T18:30:53.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.813+0000 7fae9a6aa640 1 Processor -- start 2023-11-28T18:30:53.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.813+0000 7fae9a6aa640 1 -- start start 2023-11-28T18:30:53.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.813+0000 7fae9a6aa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fae9415d3c0 0x7fae94071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:53.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.813+0000 7fae9a6aa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fae94161870 0x7fae940ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:53.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.813+0000 7fae9a6aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae94070a10 0x7fae94070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:53.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.813+0000 7fae9a6aa640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae940713a0 con 0x7fae9415d3c0 2023-11-28T18:30:53.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.813+0000 7fae9a6aa640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae940cefa0 con 0x7fae94161870 2023-11-28T18:30:53.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.813+0000 7fae9a6aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9415d7a0 con 0x7fae94070a10 2023-11-28T18:30:53.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.813+0000 7fae93fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fae9415d3c0 0x7fae94071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:53.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.813+0000 7fae93fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fae9415d3c0 0x7fae94071a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59792/0 (socket says 172.21.15.105:59792) 2023-11-28T18:30:53.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.813+0000 7fae93fff640 1 -- 172.21.15.105:0/3174819455 learned_addr learned my addr 172.21.15.105:0/3174819455 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:53.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.817+0000 7fae937fe640 1 --2- 172.21.15.105:0/3174819455 >> v2:172.21.15.148:3300/0 conn(0x7fae94161870 0x7fae940ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:53.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.817+0000 7fae98c47640 1 --2- 172.21.15.105:0/3174819455 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae94070a10 0x7fae94070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:53.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.817+0000 7fae93fff640 1 -- 172.21.15.105:0/3174819455 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae94070a10 msgr2=0x7fae94070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:53.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.817+0000 7fae93fff640 1 --2- 172.21.15.105:0/3174819455 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae94070a10 0x7fae94070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.817+0000 7fae93fff640 1 -- 172.21.15.105:0/3174819455 >> v2:172.21.15.148:3300/0 conn(0x7fae94161870 msgr2=0x7fae940ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:53.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.817+0000 7fae93fff640 1 --2- 172.21.15.105:0/3174819455 >> v2:172.21.15.148:3300/0 conn(0x7fae94161870 0x7fae940ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.817+0000 7fae93fff640 1 -- 172.21.15.105:0/3174819455 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae940ce9e0 con 0x7fae9415d3c0 2023-11-28T18:30:53.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.817+0000 7fae937fe640 1 --2- 172.21.15.105:0/3174819455 >> v2:172.21.15.148:3300/0 conn(0x7fae94161870 0x7fae940ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:53.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.817+0000 7fae98c47640 1 --2- 172.21.15.105:0/3174819455 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae94070a10 0x7fae94070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:53.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.817+0000 7fae93fff640 1 --2- 172.21.15.105:0/3174819455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae9415d3c0 0x7fae94071a10 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fae94160e30 tx=0x7fae8401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f334b9bb484c6e4e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:53.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.817+0000 7fae92ffd640 1 -- 172.21.15.105:0/3174819455 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae840aa5f0 con 0x7fae9415d3c0 2023-11-28T18:30:53.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.817+0000 7fae92ffd640 1 -- 172.21.15.105:0/3174819455 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae840aa750 con 0x7fae9415d3c0 2023-11-28T18:30:53.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.817+0000 7fae92ffd640 1 -- 172.21.15.105:0/3174819455 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae840aaae0 con 0x7fae9415d3c0 2023-11-28T18:30:53.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.817+0000 7fae9a6aa640 1 -- 172.21.15.105:0/3174819455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae9415d3c0 msgr2=0x7fae94071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:53.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.817+0000 7fae9a6aa640 1 --2- 172.21.15.105:0/3174819455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae9415d3c0 0x7fae94071a10 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fae94160e30 tx=0x7fae8401c4d0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae9a6aa640 1 -- 172.21.15.105:0/3174819455 shutdown_connections 2023-11-28T18:30:53.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae9a6aa640 1 --2- 172.21.15.105:0/3174819455 >> v2:172.21.15.148:3300/0 conn(0x7fae94161870 0x7fae940ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae9a6aa640 1 --2- 172.21.15.105:0/3174819455 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae94070a10 0x7fae94070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae9a6aa640 1 --2- 172.21.15.105:0/3174819455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae9415d3c0 0x7fae94071a10 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae9a6aa640 1 -- 172.21.15.105:0/3174819455 >> 172.21.15.105:0/3174819455 conn(0x7fae940cf540 msgr2=0x7fae940cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:53.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae9a6aa640 1 -- 172.21.15.105:0/3174819455 shutdown_connections 2023-11-28T18:30:53.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae9a6aa640 1 -- 172.21.15.105:0/3174819455 wait complete. 2023-11-28T18:30:53.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae9a6aa640 1 Processor -- start 2023-11-28T18:30:53.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae9a6aa640 1 -- start start 2023-11-28T18:30:53.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae9a6aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae94070a10 0x7fae94154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:53.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae9a6aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae9415d3c0 0x7fae941678b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:53.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae9a6aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae94161870 0x7fae9414e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae9a6aa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae940ad390 con 0x7fae94161870 2023-11-28T18:30:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae9a6aa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9414ec50 con 0x7fae9415d3c0 2023-11-28T18:30:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae9a6aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae940ac9f0 con 0x7fae94070a10 2023-11-28T18:30:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae98c47640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae94161870 0x7fae9414e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae98c47640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae94161870 0x7fae9414e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59802/0 (socket says 172.21.15.105:59802) 2023-11-28T18:30:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae98c47640 1 -- 172.21.15.105:0/4165084976 learned_addr learned my addr 172.21.15.105:0/4165084976 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae937fe640 1 --2- 172.21.15.105:0/4165084976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae9415d3c0 0x7fae941678b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae93fff640 1 --2- 172.21.15.105:0/4165084976 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae94070a10 0x7fae94154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae98c47640 1 -- 172.21.15.105:0/4165084976 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae94070a10 msgr2=0x7fae94154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae98c47640 1 --2- 172.21.15.105:0/4165084976 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae94070a10 0x7fae94154260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae98c47640 1 -- 172.21.15.105:0/4165084976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae9415d3c0 msgr2=0x7fae941678b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:53.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae98c47640 1 --2- 172.21.15.105:0/4165084976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae9415d3c0 0x7fae941678b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae98c47640 1 -- 172.21.15.105:0/4165084976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae840094d0 con 0x7fae94161870 2023-11-28T18:30:53.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.821+0000 7fae98c47640 1 --2- 172.21.15.105:0/4165084976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae94161870 0x7fae9414e300 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00cd10 tx=0x7fae8c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:53.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.825+0000 7fae90ff9640 1 -- 172.21.15.105:0/4165084976 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c00f680 con 0x7fae94161870 2023-11-28T18:30:53.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.825+0000 7fae90ff9640 1 -- 172.21.15.105:0/4165084976 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8c00f800 con 0x7fae94161870 2023-11-28T18:30:53.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.825+0000 7fae90ff9640 1 -- 172.21.15.105:0/4165084976 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c00fbb0 con 0x7fae94161870 2023-11-28T18:30:53.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.825+0000 7fae9a6aa640 1 -- 172.21.15.105:0/4165084976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae940accd0 con 0x7fae94161870 2023-11-28T18:30:53.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.825+0000 7fae9a6aa640 1 -- 172.21.15.105:0/4165084976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae941547a0 con 0x7fae94161870 2023-11-28T18:30:53.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.825+0000 7fae9a6aa640 1 -- 172.21.15.105:0/4165084976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae940ad4d0 con 0x7fae94161870 2023-11-28T18:30:53.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.825+0000 7fae90ff9640 1 -- 172.21.15.105:0/4165084976 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fae8c00b040 con 0x7fae94161870 2023-11-28T18:30:53.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.825+0000 7fae90ff9640 1 --2- 172.21.15.105:0/4165084976 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae800668d0 0x7fae80068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:53.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.825+0000 7fae90ff9640 1 -- 172.21.15.105:0/4165084976 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fae8c084b90 con 0x7fae94161870 2023-11-28T18:30:53.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.825+0000 7fae93fff640 1 --2- 172.21.15.105:0/4165084976 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae800668d0 0x7fae80068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:53.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.825+0000 7fae93fff640 1 --2- 172.21.15.105:0/4165084976 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae800668d0 0x7fae80068d90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fae840ae000 tx=0x7fae840b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:53.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.829+0000 7fae90ff9640 1 -- 172.21.15.105:0/4165084976 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fae940ad4d0 con 0x7fae94161870 2023-11-28T18:30:53.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.981+0000 7fae9a6aa640 1 -- 172.21.15.105:0/4165084976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"} v 0) v1 -- 0x7fae940ad4d0 con 0x7fae94161870 2023-11-28T18:30:53.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.989+0000 7fae90ff9640 1 -- 172.21.15.105:0/4165084976 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"}]=0 v58) v1 ==== 114+0+0 (secure 0 0 0) 0x7fae940ad4d0 con 0x7fae94161870 2023-11-28T18:30:53.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.989+0000 7fae9a6aa640 1 -- 172.21.15.105:0/4165084976 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae800668d0 msgr2=0x7fae80068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:53.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.989+0000 7fae9a6aa640 1 --2- 172.21.15.105:0/4165084976 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae800668d0 0x7fae80068d90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fae840ae000 tx=0x7fae840b5040 comp rx=0 tx=0).stop 2023-11-28T18:30:53.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.989+0000 7fae9a6aa640 1 -- 172.21.15.105:0/4165084976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae94161870 msgr2=0x7fae9414e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:53.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.989+0000 7fae9a6aa640 1 --2- 172.21.15.105:0/4165084976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae94161870 0x7fae9414e300 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00cd10 tx=0x7fae8c00a650 comp rx=0 tx=0).stop 2023-11-28T18:30:53.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.989+0000 7fae9a6aa640 1 -- 172.21.15.105:0/4165084976 shutdown_connections 2023-11-28T18:30:53.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.989+0000 7fae9a6aa640 1 --2- 172.21.15.105:0/4165084976 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fae800668d0 0x7fae80068d90 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.989+0000 7fae9a6aa640 1 --2- 172.21.15.105:0/4165084976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae94161870 0x7fae9414e300 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.989+0000 7fae9a6aa640 1 --2- 172.21.15.105:0/4165084976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae9415d3c0 0x7fae941678b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.989+0000 7fae9a6aa640 1 --2- 172.21.15.105:0/4165084976 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae94070a10 0x7fae94154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:53.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.989+0000 7fae9a6aa640 1 -- 172.21.15.105:0/4165084976 >> 172.21.15.105:0/4165084976 conn(0x7fae940cf540 msgr2=0x7fae940b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:53.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.989+0000 7fae9a6aa640 1 -- 172.21.15.105:0/4165084976 shutdown_connections 2023-11-28T18:30:53.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:53.989+0000 7fae9a6aa640 1 -- 172.21.15.105:0/4165084976 wait complete. 2023-11-28T18:30:54.010 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_before' 2023-11-28T18:30:54.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.145+0000 7f5e783d1640 1 Processor -- start 2023-11-28T18:30:54.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.145+0000 7f5e783d1640 1 -- start start 2023-11-28T18:30:54.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.149+0000 7f5e783d1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5e70159f40 0x7f5e700caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:54.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.149+0000 7f5e783d1640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e700cc760 0x7f5e700ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:54.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.149+0000 7f5e783d1640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e7006bf50 con 0x7f5e700ad680 2023-11-28T18:30:54.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.149+0000 7f5e783d1640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e700ccbc0 con 0x7f5e70159f40 2023-11-28T18:30:54.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.149+0000 7f5e783d1640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7015a320 con 0x7f5e700cc760 2023-11-28T18:30:54.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.149+0000 7f5e7616d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5e70159f40 0x7f5e700caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:54.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.149+0000 7f5e7616d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5e70159f40 0x7f5e700caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59810/0 (socket says 172.21.15.105:59810) 2023-11-28T18:30:54.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.149+0000 7f5e7616d640 1 -- 172.21.15.105:0/3047558805 learned_addr learned my addr 172.21.15.105:0/3047558805 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:54.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.149+0000 7f5e7596c640 1 --2- 172.21.15.105:0/3047558805 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e700cc760 0x7f5e700ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:54.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.149+0000 7f5e7616d640 1 -- 172.21.15.105:0/3047558805 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e700cc760 msgr2=0x7f5e700ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:54.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.149+0000 7f5e7616d640 1 --2- 172.21.15.105:0/3047558805 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e700cc760 0x7f5e700ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:54.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.149+0000 7f5e7616d640 1 -- 172.21.15.105:0/3047558805 >> v1:172.21.15.105:6789/0 conn(0x7f5e700ad680 legacy=0x7f5e7015da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:54.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.149+0000 7f5e7616d640 1 -- 172.21.15.105:0/3047558805 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e70071380 con 0x7f5e70159f40 2023-11-28T18:30:54.151 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.153+0000 7f5e7616d640 1 --2- 172.21.15.105:0/3047558805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e70159f40 0x7f5e700caf60 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009870 tx=0x7f5e6c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27fdcee6f03eb018 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:54.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.153+0000 7f5e7516b640 1 -- 172.21.15.105:0/3047558805 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c0ab680 con 0x7f5e70159f40 2023-11-28T18:30:54.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.153+0000 7f5e7516b640 1 -- 172.21.15.105:0/3047558805 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c0ab7e0 con 0x7f5e70159f40 2023-11-28T18:30:54.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.153+0000 7f5e7516b640 1 -- 172.21.15.105:0/3047558805 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c0abb70 con 0x7f5e70159f40 2023-11-28T18:30:54.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.153+0000 7f5e783d1640 1 -- 172.21.15.105:0/3047558805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e70159f40 msgr2=0x7f5e700caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:54.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.153+0000 7f5e783d1640 1 --2- 172.21.15.105:0/3047558805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e70159f40 0x7f5e700caf60 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009870 tx=0x7f5e6c01c480 comp rx=0 tx=0).stop 2023-11-28T18:30:54.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.153+0000 7f5e783d1640 1 -- 172.21.15.105:0/3047558805 shutdown_connections 2023-11-28T18:30:54.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.153+0000 7f5e783d1640 1 --2- 172.21.15.105:0/3047558805 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e700cc760 0x7f5e700ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:54.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.153+0000 7f5e783d1640 1 --2- 172.21.15.105:0/3047558805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e70159f40 0x7f5e700caf60 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:54.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.153+0000 7f5e783d1640 1 -- 172.21.15.105:0/3047558805 >> 172.21.15.105:0/3047558805 conn(0x7f5e700ceb90 msgr2=0x7f5e700cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:54.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.153+0000 7f5e783d1640 1 -- 172.21.15.105:0/3047558805 shutdown_connections 2023-11-28T18:30:54.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.153+0000 7f5e783d1640 1 -- 172.21.15.105:0/3047558805 wait complete. 2023-11-28T18:30:54.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.153+0000 7f5e783d1640 1 Processor -- start 2023-11-28T18:30:54.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.153+0000 7f5e783d1640 1 -- start start 2023-11-28T18:30:54.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e783d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e700ad680 0x7f5e7015b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:54.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e783d1640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e700cc760 0x7f5e701521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:54.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e783d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e70159f40 0x7f5e7009c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:54.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e783d1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7014cad0 con 0x7f5e70159f40 2023-11-28T18:30:54.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e783d1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7009afd0 con 0x7f5e700ad680 2023-11-28T18:30:54.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e783d1640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7009b2b0 con 0x7f5e700cc760 2023-11-28T18:30:54.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e7696e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e70159f40 0x7f5e7009c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:54.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e7696e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e70159f40 0x7f5e7009c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59814/0 (socket says 172.21.15.105:59814) 2023-11-28T18:30:54.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e7696e640 1 -- 172.21.15.105:0/3587694180 learned_addr learned my addr 172.21.15.105:0/3587694180 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:54.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e7596c640 1 --2- 172.21.15.105:0/3587694180 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e700cc760 0x7f5e701521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:54.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e7616d640 1 --2- 172.21.15.105:0/3587694180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e700ad680 0x7f5e7015b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:54.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e7696e640 1 -- 172.21.15.105:0/3587694180 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e700cc760 msgr2=0x7f5e701521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:54.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e7696e640 1 --2- 172.21.15.105:0/3587694180 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e700cc760 0x7f5e701521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:54.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e7696e640 1 -- 172.21.15.105:0/3587694180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e700ad680 msgr2=0x7f5e7015b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:54.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e7696e640 1 --2- 172.21.15.105:0/3587694180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e700ad680 0x7f5e7015b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:54.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e7696e640 1 -- 172.21.15.105:0/3587694180 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6c0094d0 con 0x7f5e70159f40 2023-11-28T18:30:54.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e7696e640 1 --2- 172.21.15.105:0/3587694180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e70159f40 0x7f5e7009c070 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f5e6800c730 tx=0x7f5e6800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:54.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e66ffd640 1 -- 172.21.15.105:0/3587694180 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e68003b80 con 0x7f5e70159f40 2023-11-28T18:30:54.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e66ffd640 1 -- 172.21.15.105:0/3587694180 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e68003ce0 con 0x7f5e70159f40 2023-11-28T18:30:54.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e783d1640 1 -- 172.21.15.105:0/3587694180 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7009b590 con 0x7f5e70159f40 2023-11-28T18:30:54.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e66ffd640 1 -- 172.21.15.105:0/3587694180 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6800aa30 con 0x7f5e70159f40 2023-11-28T18:30:54.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e783d1640 1 -- 172.21.15.105:0/3587694180 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7009bb50 con 0x7f5e70159f40 2023-11-28T18:30:54.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.157+0000 7f5e783d1640 1 -- 172.21.15.105:0/3587694180 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e70067570 con 0x7f5e70159f40 2023-11-28T18:30:54.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.161+0000 7f5e66ffd640 1 -- 172.21.15.105:0/3587694180 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5e6800ab90 con 0x7f5e70159f40 2023-11-28T18:30:54.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.161+0000 7f5e66ffd640 1 --2- 172.21.15.105:0/3587694180 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5e540668d0 0x7f5e54068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:54.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.161+0000 7f5e66ffd640 1 -- 172.21.15.105:0/3587694180 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f5e680840e0 con 0x7f5e70159f40 2023-11-28T18:30:54.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.161+0000 7f5e7616d640 1 --2- 172.21.15.105:0/3587694180 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5e540668d0 0x7f5e54068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:54.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.161+0000 7f5e7616d640 1 --2- 172.21.15.105:0/3587694180 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5e540668d0 0x7f5e54068d90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0af000 tx=0x7f5e6c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:54.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.165+0000 7f5e66ffd640 1 -- 172.21.15.105:0/3587694180 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5e68051420 con 0x7f5e70159f40 2023-11-28T18:30:54.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.365+0000 7f5e783d1640 1 -- 172.21.15.105:0/3587694180 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_before"]} v 0) v1 -- 0x7f5e70070ec0 con 0x7f5e70159f40 2023-11-28T18:30:54.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.629+0000 7f5e66ffd640 1 -- 172.21.15.105:0/3587694180 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_before"]}]=0 v385) v1 ==== 162+0+0 (secure 0 0 0) 0x7f5e68054a40 con 0x7f5e70159f40 2023-11-28T18:30:54.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.633+0000 7f5e783d1640 1 -- 172.21.15.105:0/3587694180 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5e540668d0 msgr2=0x7f5e54068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:54.631 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.633+0000 7f5e783d1640 1 --2- 172.21.15.105:0/3587694180 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5e540668d0 0x7f5e54068d90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0af000 tx=0x7f5e6c0a7000 comp rx=0 tx=0).stop 2023-11-28T18:30:54.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.633+0000 7f5e783d1640 1 -- 172.21.15.105:0/3587694180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e70159f40 msgr2=0x7f5e7009c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:54.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.633+0000 7f5e783d1640 1 --2- 172.21.15.105:0/3587694180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e70159f40 0x7f5e7009c070 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f5e6800c730 tx=0x7f5e6800cbf0 comp rx=0 tx=0).stop 2023-11-28T18:30:54.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.633+0000 7f5e783d1640 1 -- 172.21.15.105:0/3587694180 shutdown_connections 2023-11-28T18:30:54.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.633+0000 7f5e783d1640 1 --2- 172.21.15.105:0/3587694180 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5e540668d0 0x7f5e54068d90 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:54.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.633+0000 7f5e783d1640 1 --2- 172.21.15.105:0/3587694180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e70159f40 0x7f5e7009c070 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:54.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.633+0000 7f5e783d1640 1 --2- 172.21.15.105:0/3587694180 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e700cc760 0x7f5e701521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:54.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.633+0000 7f5e783d1640 1 --2- 172.21.15.105:0/3587694180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e700ad680 0x7f5e7015b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:54.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.633+0000 7f5e783d1640 1 -- 172.21.15.105:0/3587694180 >> 172.21.15.105:0/3587694180 conn(0x7f5e700ceb90 msgr2=0x7f5e700ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:54.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.633+0000 7f5e783d1640 1 -- 172.21.15.105:0/3587694180 shutdown_connections 2023-11-28T18:30:54.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.633+0000 7f5e783d1640 1 -- 172.21.15.105:0/3587694180 wait complete. 2023-11-28T18:30:54.651 INFO:tasks.cephfs_test_runner:That a conventional export pin overrides the distributed policy _before_ distributed policy is set. ... ok 2023-11-28T18:30:54.651 INFO:tasks.cephfs_test_runner:Starting test: test_ephemeral_pin_dist_override_pin (tasks.cephfs.test_exports.TestEphemeralPins) That an export pin overrides an ephemerally pinned directory. 2023-11-28T18:30:54.652 INFO:tasks.cephfs_test_runner:test_ephemeral_pin_dist_override_pin (tasks.cephfs.test_exports.TestEphemeralPins) 2023-11-28T18:30:54.652 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:30:54.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.789+0000 7ff998446640 1 Processor -- start 2023-11-28T18:30:54.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.789+0000 7ff998446640 1 -- start start 2023-11-28T18:30:54.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.789+0000 7ff998446640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff9900b4290 0x7ff9900b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:54.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.789+0000 7ff998446640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff99015b030 0x7ff9900709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:54.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.789+0000 7ff998446640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9900b4bb0 con 0x7ff990071960 2023-11-28T18:30:54.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.789+0000 7ff998446640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9900701d0 con 0x7ff99015b030 2023-11-28T18:30:54.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.789+0000 7ff998446640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff99006df80 con 0x7ff9900b4290 2023-11-28T18:30:54.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.793+0000 7ff9969e3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff99015b030 0x7ff9900709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:54.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.793+0000 7ff9959e1640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff990071960 0x7ff99015f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42160/0 (socket says 172.21.15.105:42160) 2023-11-28T18:30:54.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.793+0000 7ff9959e1640 1 -- 172.21.15.105:0/2711267238 learned_addr learned my addr 172.21.15.105:0/2711267238 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:54.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.793+0000 7ff9961e2640 1 --2- 172.21.15.105:0/2711267238 >> v2:172.21.15.148:3300/0 conn(0x7ff9900b4290 0x7ff9900b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:54.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.793+0000 7ff9969e3640 1 -- 172.21.15.105:0/2711267238 >> v2:172.21.15.148:3300/0 conn(0x7ff9900b4290 msgr2=0x7ff9900b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:54.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.793+0000 7ff9969e3640 1 --2- 172.21.15.105:0/2711267238 >> v2:172.21.15.148:3300/0 conn(0x7ff9900b4290 0x7ff9900b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:54.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.793+0000 7ff9969e3640 1 -- 172.21.15.105:0/2711267238 >> v1:172.21.15.105:6789/0 conn(0x7ff990071960 legacy=0x7ff99015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:54.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.793+0000 7ff9969e3640 1 -- 172.21.15.105:0/2711267238 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff990071df0 con 0x7ff99015b030 2023-11-28T18:30:54.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.793+0000 7ff9951e0640 1 -- 172.21.15.105:0/2711267238 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 396861064 0 0) 0x7ff9900b4bb0 con 0x7ff990071960 2023-11-28T18:30:54.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.793+0000 7ff9969e3640 1 --2- 172.21.15.105:0/2711267238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff99015b030 0x7ff9900709b0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7ff990073a70 tx=0x7ff98801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a752edd7cdc3780 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:54.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.793+0000 7ff9951e0640 1 -- 172.21.15.105:0/2711267238 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9880a58b0 con 0x7ff99015b030 2023-11-28T18:30:54.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.793+0000 7ff9951e0640 1 -- 172.21.15.105:0/2711267238 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9880a5a10 con 0x7ff99015b030 2023-11-28T18:30:54.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.793+0000 7ff9951e0640 1 -- 172.21.15.105:0/2711267238 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9880a5da0 con 0x7ff99015b030 2023-11-28T18:30:54.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.797+0000 7ff998446640 1 -- 172.21.15.105:0/2711267238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff99015b030 msgr2=0x7ff9900709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:54.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.797+0000 7ff998446640 1 --2- 172.21.15.105:0/2711267238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff99015b030 0x7ff9900709b0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7ff990073a70 tx=0x7ff98801c8e0 comp rx=0 tx=0).stop 2023-11-28T18:30:54.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.797+0000 7ff998446640 1 -- 172.21.15.105:0/2711267238 shutdown_connections 2023-11-28T18:30:54.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.797+0000 7ff998446640 1 --2- 172.21.15.105:0/2711267238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff99015b030 0x7ff9900709b0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:54.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.797+0000 7ff998446640 1 --2- 172.21.15.105:0/2711267238 >> v2:172.21.15.148:3300/0 conn(0x7ff9900b4290 0x7ff9900b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:54.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.797+0000 7ff998446640 1 -- 172.21.15.105:0/2711267238 >> 172.21.15.105:0/2711267238 conn(0x7ff9900ceb90 msgr2=0x7ff9900cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:54.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.797+0000 7ff998446640 1 -- 172.21.15.105:0/2711267238 shutdown_connections 2023-11-28T18:30:54.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.797+0000 7ff998446640 1 -- 172.21.15.105:0/2711267238 wait complete. 2023-11-28T18:30:54.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.797+0000 7ff998446640 1 Processor -- start 2023-11-28T18:30:54.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.797+0000 7ff998446640 1 -- start start 2023-11-28T18:30:54.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.797+0000 7ff998446640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9900b4290 0x7ff9900ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:54.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.797+0000 7ff998446640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff99015b030 0x7ff9900ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:54.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.797+0000 7ff998446640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff990157a20 0x7ff9901f91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:54.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.797+0000 7ff998446640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9900af140 con 0x7ff990157a20 2023-11-28T18:30:54.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.797+0000 7ff998446640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9900af420 con 0x7ff9900b4290 2023-11-28T18:30:54.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.797+0000 7ff998446640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9900af700 con 0x7ff99015b030 2023-11-28T18:30:54.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.797+0000 7ff9969e3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff990157a20 0x7ff9901f91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:54.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.797+0000 7ff9969e3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff990157a20 0x7ff9901f91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59826/0 (socket says 172.21.15.105:59826) 2023-11-28T18:30:54.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.797+0000 7ff9969e3640 1 -- 172.21.15.105:0/3320034743 learned_addr learned my addr 172.21.15.105:0/3320034743 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:54.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.797+0000 7ff9969e3640 1 -- 172.21.15.105:0/3320034743 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff99015b030 msgr2=0x7ff9900ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:54.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.797+0000 7ff9969e3640 1 --2- 172.21.15.105:0/3320034743 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff99015b030 0x7ff9900ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:54.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.801+0000 7ff9969e3640 1 -- 172.21.15.105:0/3320034743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9900b4290 msgr2=0x7ff9900ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:54.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.801+0000 7ff9969e3640 1 --2- 172.21.15.105:0/3320034743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9900b4290 0x7ff9900ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:54.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.801+0000 7ff9969e3640 1 -- 172.21.15.105:0/3320034743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff988009a70 con 0x7ff990157a20 2023-11-28T18:30:54.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.801+0000 7ff9969e3640 1 --2- 172.21.15.105:0/3320034743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff990157a20 0x7ff9901f91d0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7ff9880194f0 tx=0x7ff988019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:54.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.801+0000 7ff986ffd640 1 -- 172.21.15.105:0/3320034743 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9880bd070 con 0x7ff990157a20 2023-11-28T18:30:54.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.801+0000 7ff986ffd640 1 -- 172.21.15.105:0/3320034743 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9880ad650 con 0x7ff990157a20 2023-11-28T18:30:54.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.801+0000 7ff998446640 1 -- 172.21.15.105:0/3320034743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9900af9e0 con 0x7ff990157a20 2023-11-28T18:30:54.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.801+0000 7ff986ffd640 1 -- 172.21.15.105:0/3320034743 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9880ad9c0 con 0x7ff990157a20 2023-11-28T18:30:54.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.801+0000 7ff998446640 1 -- 172.21.15.105:0/3320034743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9900aee30 con 0x7ff990157a20 2023-11-28T18:30:54.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.801+0000 7ff998446640 1 -- 172.21.15.105:0/3320034743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff990067570 con 0x7ff990157a20 2023-11-28T18:30:54.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.801+0000 7ff986ffd640 1 -- 172.21.15.105:0/3320034743 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff9880b4070 con 0x7ff990157a20 2023-11-28T18:30:54.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.801+0000 7ff986ffd640 1 --2- 172.21.15.105:0/3320034743 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff974066900 0x7ff974068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:54.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.801+0000 7ff986ffd640 1 -- 172.21.15.105:0/3320034743 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5435+0+0 (secure 0 0 0) 0x7ff98811ea50 con 0x7ff990157a20 2023-11-28T18:30:54.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.801+0000 7ff9961e2640 1 --2- 172.21.15.105:0/3320034743 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff974066900 0x7ff974068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:54.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.805+0000 7ff9961e2640 1 --2- 172.21.15.105:0/3320034743 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff974066900 0x7ff974068dc0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0098f0 tx=0x7ff98c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:54.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.805+0000 7ff986ffd640 1 -- 172.21.15.105:0/3320034743 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff9880ebd00 con 0x7ff990157a20 2023-11-28T18:30:54.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.957+0000 7ff998446640 1 -- 172.21.15.105:0/3320034743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9900cf7f0 con 0x7ff990157a20 2023-11-28T18:30:54.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.957+0000 7ff986ffd640 1 -- 172.21.15.105:0/3320034743 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+13527 (secure 0 0 0) 0x7ff9880b2020 con 0x7ff990157a20 2023-11-28T18:30:54.958 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:54.958 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":108,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:30:52.617335+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6835/3664281230":"2023-11-29T18:30:49.609422+0000","172.21.15.148:6833/3664281230":"2023-11-29T18:30:49.609422+0000","172.21.15.105:6834/3968059262":"2023-11-29T18:30:49.609435+0000","172.21.15.105:6837/2782729992":"2023-11-29T18:30:49.609439+0000","172.21.15.105:6836/3968059262":"2023-11-29T18:30:49.609435+0000","172.21.15.105:6835/2782729992":"2023-11-29T18:30:49.609439+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:30:54.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.961+0000 7ff998446640 1 -- 172.21.15.105:0/3320034743 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff974066900 msgr2=0x7ff974068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:54.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.961+0000 7ff998446640 1 --2- 172.21.15.105:0/3320034743 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff974066900 0x7ff974068dc0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0098f0 tx=0x7ff98c008040 comp rx=0 tx=0).stop 2023-11-28T18:30:54.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.961+0000 7ff998446640 1 -- 172.21.15.105:0/3320034743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff990157a20 msgr2=0x7ff9901f91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:54.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.961+0000 7ff998446640 1 --2- 172.21.15.105:0/3320034743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff990157a20 0x7ff9901f91d0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7ff9880194f0 tx=0x7ff988019680 comp rx=0 tx=0).stop 2023-11-28T18:30:54.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.961+0000 7ff998446640 1 -- 172.21.15.105:0/3320034743 shutdown_connections 2023-11-28T18:30:54.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.961+0000 7ff998446640 1 --2- 172.21.15.105:0/3320034743 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff974066900 0x7ff974068dc0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:54.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.961+0000 7ff998446640 1 --2- 172.21.15.105:0/3320034743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff990157a20 0x7ff9901f91d0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:54.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.961+0000 7ff998446640 1 --2- 172.21.15.105:0/3320034743 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff99015b030 0x7ff9900ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:54.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.961+0000 7ff998446640 1 --2- 172.21.15.105:0/3320034743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9900b4290 0x7ff9900ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:54.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.961+0000 7ff998446640 1 -- 172.21.15.105:0/3320034743 >> 172.21.15.105:0/3320034743 conn(0x7ff9900ceb90 msgr2=0x7ff990165070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:54.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.961+0000 7ff998446640 1 -- 172.21.15.105:0/3320034743 shutdown_connections 2023-11-28T18:30:54.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:54.961+0000 7ff998446640 1 -- 172.21.15.105:0/3320034743 wait complete. 2023-11-28T18:30:54.979 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:30:54.979 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:30:55.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.121+0000 7fd7edac9640 1 Processor -- start 2023-11-28T18:30:55.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.121+0000 7fd7edac9640 1 -- start start 2023-11-28T18:30:55.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.121+0000 7fd7edac9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7e8159f40 0x7fd7e80caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:55.122 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.121+0000 7fd7edac9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd7e80cc760 0x7fd7e80ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:55.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.121+0000 7fd7edac9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e806bf50 con 0x7fd7e80ad680 2023-11-28T18:30:55.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.121+0000 7fd7edac9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e80ccbc0 con 0x7fd7e80cc760 2023-11-28T18:30:55.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.121+0000 7fd7edac9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e815a320 con 0x7fd7e8159f40 2023-11-28T18:30:55.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.121+0000 7fd7e7fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd7e80ad680 0x7fd7e815da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42174/0 (socket says 172.21.15.105:42174) 2023-11-28T18:30:55.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.121+0000 7fd7e7fff640 1 -- 172.21.15.105:0/1196148558 learned_addr learned my addr 172.21.15.105:0/1196148558 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:55.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.121+0000 7fd7e77fe640 1 --2- 172.21.15.105:0/1196148558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7e8159f40 0x7fd7e80caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:55.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.121+0000 7fd7dffff640 1 --2- 172.21.15.105:0/1196148558 >> v2:172.21.15.148:3300/0 conn(0x7fd7e80cc760 0x7fd7e80ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:55.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.121+0000 7fd7e6ffd640 1 -- 172.21.15.105:0/1196148558 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 496363965 0 0) 0x7fd7e806bf50 con 0x7fd7e80ad680 2023-11-28T18:30:55.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.121+0000 7fd7e6ffd640 1 -- 172.21.15.105:0/1196148558 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7d4003530 con 0x7fd7e80ad680 2023-11-28T18:30:55.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.121+0000 7fd7dffff640 1 -- 172.21.15.105:0/1196148558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7e8159f40 msgr2=0x7fd7e80caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:55.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.121+0000 7fd7dffff640 1 --2- 172.21.15.105:0/1196148558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7e8159f40 0x7fd7e80caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:55.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.121+0000 7fd7dffff640 1 -- 172.21.15.105:0/1196148558 >> v1:172.21.15.105:6789/0 conn(0x7fd7e80ad680 legacy=0x7fd7e815da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:55.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.121+0000 7fd7e6ffd640 1 -- 172.21.15.105:0/1196148558 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1965590945 0 0) 0x7fd7d4003530 con 0x7fd7e80ad680 2023-11-28T18:30:55.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.121+0000 7fd7dffff640 1 -- 172.21.15.105:0/1196148558 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e8071380 con 0x7fd7e80cc760 2023-11-28T18:30:55.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.121+0000 7fd7e77fe640 1 --2- 172.21.15.105:0/1196148558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7e8159f40 0x7fd7e80caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:30:55.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.125+0000 7fd7dffff640 1 --2- 172.21.15.105:0/1196148558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7e80cc760 0x7fd7e80ad140 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00097c0 tx=0x7fd7d001c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e50f6f951c933d2a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:55.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.125+0000 7fd7e6ffd640 1 -- 172.21.15.105:0/1196148558 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0001a00 con 0x7fd7e80cc760 2023-11-28T18:30:55.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.125+0000 7fd7e6ffd640 1 -- 172.21.15.105:0/1196148558 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d00a7050 con 0x7fd7e80cc760 2023-11-28T18:30:55.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.125+0000 7fd7e6ffd640 1 -- 172.21.15.105:0/1196148558 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d00ab620 con 0x7fd7e80cc760 2023-11-28T18:30:55.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.125+0000 7fd7edac9640 1 -- 172.21.15.105:0/1196148558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7e80cc760 msgr2=0x7fd7e80ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:55.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.125+0000 7fd7edac9640 1 --2- 172.21.15.105:0/1196148558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7e80cc760 0x7fd7e80ad140 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00097c0 tx=0x7fd7d001c2e0 comp rx=0 tx=0).stop 2023-11-28T18:30:55.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.125+0000 7fd7edac9640 1 -- 172.21.15.105:0/1196148558 shutdown_connections 2023-11-28T18:30:55.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.125+0000 7fd7edac9640 1 --2- 172.21.15.105:0/1196148558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7e80cc760 0x7fd7e80ad140 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:55.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.125+0000 7fd7edac9640 1 --2- 172.21.15.105:0/1196148558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7e8159f40 0x7fd7e80caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:55.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.125+0000 7fd7edac9640 1 -- 172.21.15.105:0/1196148558 >> 172.21.15.105:0/1196148558 conn(0x7fd7e80ceb90 msgr2=0x7fd7e80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:55.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.125+0000 7fd7edac9640 1 -- 172.21.15.105:0/1196148558 shutdown_connections 2023-11-28T18:30:55.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.125+0000 7fd7edac9640 1 -- 172.21.15.105:0/1196148558 wait complete. 2023-11-28T18:30:55.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7edac9640 1 Processor -- start 2023-11-28T18:30:55.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7edac9640 1 -- start start 2023-11-28T18:30:55.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7edac9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7e80ad680 0x7fd7e815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:55.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7edac9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7e80cc760 0x7fd7e815b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:55.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7edac9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7e8159f40 0x7fd7e8152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:55.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7edac9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e815bb90 con 0x7fd7e80cc760 2023-11-28T18:30:55.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7edac9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e813f950 con 0x7fd7e80ad680 2023-11-28T18:30:55.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7edac9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e815be10 con 0x7fd7e8159f40 2023-11-28T18:30:55.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7dffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7e80cc760 0x7fd7e815b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:55.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7dffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7e80cc760 0x7fd7e815b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59830/0 (socket says 172.21.15.105:59830) 2023-11-28T18:30:55.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7dffff640 1 -- 172.21.15.105:0/486694006 learned_addr learned my addr 172.21.15.105:0/486694006 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:55.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7dffff640 1 -- 172.21.15.105:0/486694006 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7e8159f40 msgr2=0x7fd7e8152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:55.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7dffff640 1 --2- 172.21.15.105:0/486694006 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7e8159f40 0x7fd7e8152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:55.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7dffff640 1 -- 172.21.15.105:0/486694006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7e80ad680 msgr2=0x7fd7e815ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:55.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7dffff640 1 --2- 172.21.15.105:0/486694006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7e80ad680 0x7fd7e815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:55.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7dffff640 1 -- 172.21.15.105:0/486694006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d8009420 con 0x7fd7e80cc760 2023-11-28T18:30:55.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7dffff640 1 --2- 172.21.15.105:0/486694006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7e80cc760 0x7fd7e815b650 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00ae040 tx=0x7fd7d0019080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:55.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7e4ff9640 1 -- 172.21.15.105:0/486694006 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d00b4430 con 0x7fd7e80cc760 2023-11-28T18:30:55.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7e4ff9640 1 -- 172.21.15.105:0/486694006 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d00b45b0 con 0x7fd7e80cc760 2023-11-28T18:30:55.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7edac9640 1 -- 172.21.15.105:0/486694006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d0009420 con 0x7fd7e80cc760 2023-11-28T18:30:55.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7e4ff9640 1 -- 172.21.15.105:0/486694006 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d00b4960 con 0x7fd7e80cc760 2023-11-28T18:30:55.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7edac9640 1 -- 172.21.15.105:0/486694006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e815c3d0 con 0x7fd7e80cc760 2023-11-28T18:30:55.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.129+0000 7fd7edac9640 1 -- 172.21.15.105:0/486694006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e8067570 con 0x7fd7e80cc760 2023-11-28T18:30:55.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.133+0000 7fd7e4ff9640 1 -- 172.21.15.105:0/486694006 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd7d00b2040 con 0x7fd7e80cc760 2023-11-28T18:30:55.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.133+0000 7fd7e4ff9640 1 --2- 172.21.15.105:0/486694006 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd7c80668d0 0x7fd7c8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:55.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.133+0000 7fd7e4ff9640 1 -- 172.21.15.105:0/486694006 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fd7d011e1a0 con 0x7fd7e80cc760 2023-11-28T18:30:55.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.133+0000 7fd7e77fe640 1 --2- 172.21.15.105:0/486694006 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd7c80668d0 0x7fd7c8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:55.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.133+0000 7fd7e77fe640 1 --2- 172.21.15.105:0/486694006 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd7c80668d0 0x7fd7c8068d90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8006e90 tx=0x7fd7d8006de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:55.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.137+0000 7fd7e4ff9640 1 -- 172.21.15.105:0/486694006 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd7d00eb4e0 con 0x7fd7e80cc760 2023-11-28T18:30:55.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.285+0000 7fd7edac9640 1 -- 172.21.15.105:0/486694006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd7e81572a0 con 0x7fd7e80cc760 2023-11-28T18:30:55.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.289+0000 7fd7e4ff9640 1 -- 172.21.15.105:0/486694006 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd7d00eeb00 con 0x7fd7e80cc760 2023-11-28T18:30:55.288 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:30:55.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.289+0000 7fd7edac9640 1 -- 172.21.15.105:0/486694006 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd7c80668d0 msgr2=0x7fd7c8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:55.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.289+0000 7fd7edac9640 1 --2- 172.21.15.105:0/486694006 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd7c80668d0 0x7fd7c8068d90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8006e90 tx=0x7fd7d8006de0 comp rx=0 tx=0).stop 2023-11-28T18:30:55.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.289+0000 7fd7edac9640 1 -- 172.21.15.105:0/486694006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7e80cc760 msgr2=0x7fd7e815b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:55.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.289+0000 7fd7edac9640 1 --2- 172.21.15.105:0/486694006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7e80cc760 0x7fd7e815b650 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00ae040 tx=0x7fd7d0019080 comp rx=0 tx=0).stop 2023-11-28T18:30:55.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.289+0000 7fd7edac9640 1 -- 172.21.15.105:0/486694006 shutdown_connections 2023-11-28T18:30:55.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.289+0000 7fd7edac9640 1 --2- 172.21.15.105:0/486694006 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7e8159f40 0x7fd7e8152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:55.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.289+0000 7fd7edac9640 1 --2- 172.21.15.105:0/486694006 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd7c80668d0 0x7fd7c8068d90 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:55.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.289+0000 7fd7edac9640 1 --2- 172.21.15.105:0/486694006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7e80cc760 0x7fd7e815b650 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:55.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.289+0000 7fd7edac9640 1 --2- 172.21.15.105:0/486694006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7e80ad680 0x7fd7e815ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:55.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.289+0000 7fd7edac9640 1 -- 172.21.15.105:0/486694006 >> 172.21.15.105:0/486694006 conn(0x7fd7e80ceb90 msgr2=0x7fd7e806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:55.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.289+0000 7fd7edac9640 1 -- 172.21.15.105:0/486694006 shutdown_connections 2023-11-28T18:30:55.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.289+0000 7fd7edac9640 1 -- 172.21.15.105:0/486694006 wait complete. 2023-11-28T18:30:55.308 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_pin' 2023-11-28T18:30:55.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.445+0000 7f0068e17640 1 Processor -- start 2023-11-28T18:30:55.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.445+0000 7f0068e17640 1 -- start start 2023-11-28T18:30:55.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.449+0000 7f0068e17640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0064159f40 0x7f00640caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:55.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.449+0000 7f0068e17640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f00640ad850 0x7f00640701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:55.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.449+0000 7f0068e17640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f006406bf50 con 0x7f00640cc760 2023-11-28T18:30:55.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.449+0000 7f0068e17640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00640ccbc0 con 0x7f00640ad850 2023-11-28T18:30:55.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.449+0000 7f0068e17640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006415a320 con 0x7f0064159f40 2023-11-28T18:30:55.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.449+0000 7f0059d9b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f00640cc760 0x7f00640ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42188/0 (socket says 172.21.15.105:42188) 2023-11-28T18:30:55.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.449+0000 7f0059d9b640 1 -- 172.21.15.105:0/4115844048 learned_addr learned my addr 172.21.15.105:0/4115844048 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:55.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.449+0000 7f0062d9d640 1 --2- 172.21.15.105:0/4115844048 >> v2:172.21.15.105:3300/0 conn(0x7f00640ad850 0x7f00640701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:55.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.449+0000 7f006259c640 1 --2- 172.21.15.105:0/4115844048 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0064159f40 0x7f00640caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:55.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.449+0000 7f0062d9d640 1 -- 172.21.15.105:0/4115844048 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0064159f40 msgr2=0x7f00640caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:55.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.449+0000 7f0062d9d640 1 --2- 172.21.15.105:0/4115844048 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0064159f40 0x7f00640caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:55.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.449+0000 7f0062d9d640 1 -- 172.21.15.105:0/4115844048 >> v1:172.21.15.105:6789/0 conn(0x7f00640cc760 legacy=0x7f00640ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:55.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.449+0000 7f0062d9d640 1 -- 172.21.15.105:0/4115844048 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0064071380 con 0x7f00640ad850 2023-11-28T18:30:55.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.449+0000 7f006259c640 1 --2- 172.21.15.105:0/4115844048 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0064159f40 0x7f00640caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:55.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.453+0000 7f0062d9d640 1 --2- 172.21.15.105:0/4115844048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00640ad850 0x7f00640701d0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f005c0097c0 tx=0x7f005c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a06ac7b905fa3b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:55.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.453+0000 7f0061d9b640 1 -- 172.21.15.105:0/4115844048 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c0ac620 con 0x7f00640ad850 2023-11-28T18:30:55.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.453+0000 7f0061d9b640 1 -- 172.21.15.105:0/4115844048 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005c0ac780 con 0x7f00640ad850 2023-11-28T18:30:55.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.453+0000 7f0061d9b640 1 -- 172.21.15.105:0/4115844048 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c0acb10 con 0x7f00640ad850 2023-11-28T18:30:55.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.453+0000 7f0068e17640 1 -- 172.21.15.105:0/4115844048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00640ad850 msgr2=0x7f00640701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:55.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.453+0000 7f0068e17640 1 --2- 172.21.15.105:0/4115844048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00640ad850 0x7f00640701d0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f005c0097c0 tx=0x7f005c01c3c0 comp rx=0 tx=0).stop 2023-11-28T18:30:55.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.453+0000 7f0068e17640 1 -- 172.21.15.105:0/4115844048 shutdown_connections 2023-11-28T18:30:55.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.453+0000 7f0068e17640 1 --2- 172.21.15.105:0/4115844048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00640ad850 0x7f00640701d0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:55.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.453+0000 7f0068e17640 1 --2- 172.21.15.105:0/4115844048 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0064159f40 0x7f00640caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:55.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.453+0000 7f0068e17640 1 -- 172.21.15.105:0/4115844048 >> 172.21.15.105:0/4115844048 conn(0x7f00640ceb90 msgr2=0x7f00640cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:55.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.453+0000 7f0068e17640 1 -- 172.21.15.105:0/4115844048 shutdown_connections 2023-11-28T18:30:55.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.453+0000 7f0068e17640 1 -- 172.21.15.105:0/4115844048 wait complete. 2023-11-28T18:30:55.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.453+0000 7f0068e17640 1 Processor -- start 2023-11-28T18:30:55.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.453+0000 7f0068e17640 1 -- start start 2023-11-28T18:30:55.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f0068e17640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00640ad850 0x7f00640b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:55.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f0068e17640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00640cc760 0x7f006414eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:55.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f0068e17640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0064159f40 0x7f0064167440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:55.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f0068e17640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00640c9720 con 0x7f00640cc760 2023-11-28T18:30:55.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f0068e17640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0064166450 con 0x7f0064159f40 2023-11-28T18:30:55.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f0068e17640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0064166680 con 0x7f00640ad850 2023-11-28T18:30:55.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f0059d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00640cc760 0x7f006414eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:55.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f0059d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00640cc760 0x7f006414eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59850/0 (socket says 172.21.15.105:59850) 2023-11-28T18:30:55.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f0059d9b640 1 -- 172.21.15.105:0/173454507 learned_addr learned my addr 172.21.15.105:0/173454507 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:55.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f0062d9d640 1 --2- 172.21.15.105:0/173454507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0064159f40 0x7f0064167440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:55.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f006259c640 1 --2- 172.21.15.105:0/173454507 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00640ad850 0x7f00640b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:55.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f0059d9b640 1 -- 172.21.15.105:0/173454507 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00640ad850 msgr2=0x7f00640b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:55.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f0059d9b640 1 --2- 172.21.15.105:0/173454507 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00640ad850 0x7f00640b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:55.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f0059d9b640 1 -- 172.21.15.105:0/173454507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0064159f40 msgr2=0x7f0064167440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:55.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f0059d9b640 1 --2- 172.21.15.105:0/173454507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0064159f40 0x7f0064167440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:55.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f0059d9b640 1 -- 172.21.15.105:0/173454507 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005c009420 con 0x7f00640cc760 2023-11-28T18:30:55.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f0062d9d640 1 --2- 172.21.15.105:0/173454507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0064159f40 0x7f0064167440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:55.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f006259c640 1 --2- 172.21.15.105:0/173454507 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00640ad850 0x7f00640b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:30:55.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f0059d9b640 1 --2- 172.21.15.105:0/173454507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00640cc760 0x7f006414eb30 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f004c00cba0 tx=0x7f004c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:55.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f005b7fe640 1 -- 172.21.15.105:0/173454507 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c00ad40 con 0x7f00640cc760 2023-11-28T18:30:55.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f005b7fe640 1 -- 172.21.15.105:0/173454507 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004c012400 con 0x7f00640cc760 2023-11-28T18:30:55.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f0068e17640 1 -- 172.21.15.105:0/173454507 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0064166960 con 0x7f00640cc760 2023-11-28T18:30:55.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f005b7fe640 1 -- 172.21.15.105:0/173454507 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c00aea0 con 0x7f00640cc760 2023-11-28T18:30:55.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f0068e17640 1 -- 172.21.15.105:0/173454507 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0064166f20 con 0x7f00640cc760 2023-11-28T18:30:55.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f0068e17640 1 -- 172.21.15.105:0/173454507 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00640723f0 con 0x7f00640cc760 2023-11-28T18:30:55.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.457+0000 7f005b7fe640 1 -- 172.21.15.105:0/173454507 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f004c007070 con 0x7f00640cc760 2023-11-28T18:30:55.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.461+0000 7f005b7fe640 1 --2- 172.21.15.105:0/173454507 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00440668d0 0x7f0044068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:55.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.461+0000 7f005b7fe640 1 -- 172.21.15.105:0/173454507 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f004c0841c0 con 0x7f00640cc760 2023-11-28T18:30:55.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.461+0000 7f006259c640 1 --2- 172.21.15.105:0/173454507 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00440668d0 0x7f0044068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:55.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.461+0000 7f006259c640 1 --2- 172.21.15.105:0/173454507 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00440668d0 0x7f0044068d90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0054008890 tx=0x7f0054008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:55.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.465+0000 7f005b7fe640 1 -- 172.21.15.105:0/173454507 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f004c0515b0 con 0x7f00640cc760 2023-11-28T18:30:55.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:55.669+0000 7f0068e17640 1 -- 172.21.15.105:0/173454507 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_pin"]} v 0) v1 -- 0x7f0064167140 con 0x7f00640cc760 2023-11-28T18:30:56.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.637+0000 7f005b7fe640 1 -- 172.21.15.105:0/173454507 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_pin"]}]=0 v387) v1 ==== 162+0+0 (secure 0 0 0) 0x7f004c054bd0 con 0x7f00640cc760 2023-11-28T18:30:56.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.641+0000 7f0068e17640 1 -- 172.21.15.105:0/173454507 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00440668d0 msgr2=0x7f0044068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:56.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.641+0000 7f0068e17640 1 --2- 172.21.15.105:0/173454507 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00440668d0 0x7f0044068d90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0054008890 tx=0x7f0054008040 comp rx=0 tx=0).stop 2023-11-28T18:30:56.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.641+0000 7f0068e17640 1 -- 172.21.15.105:0/173454507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00640cc760 msgr2=0x7f006414eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:56.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.641+0000 7f0068e17640 1 --2- 172.21.15.105:0/173454507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00640cc760 0x7f006414eb30 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f004c00cba0 tx=0x7f004c00a430 comp rx=0 tx=0).stop 2023-11-28T18:30:56.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.641+0000 7f0068e17640 1 -- 172.21.15.105:0/173454507 shutdown_connections 2023-11-28T18:30:56.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.641+0000 7f0068e17640 1 --2- 172.21.15.105:0/173454507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0064159f40 0x7f0064167440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:56.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.641+0000 7f0068e17640 1 --2- 172.21.15.105:0/173454507 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00440668d0 0x7f0044068d90 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:56.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.641+0000 7f0068e17640 1 --2- 172.21.15.105:0/173454507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00640cc760 0x7f006414eb30 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:56.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.641+0000 7f0068e17640 1 --2- 172.21.15.105:0/173454507 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00640ad850 0x7f00640b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:56.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.641+0000 7f0068e17640 1 -- 172.21.15.105:0/173454507 >> 172.21.15.105:0/173454507 conn(0x7f00640ceb90 msgr2=0x7f0064071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:56.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.641+0000 7f0068e17640 1 -- 172.21.15.105:0/173454507 shutdown_connections 2023-11-28T18:30:56.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.641+0000 7f0068e17640 1 -- 172.21.15.105:0/173454507 wait complete. 2023-11-28T18:30:56.664 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T18:30:56.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.809+0000 7f2d824bb640 1 Processor -- start 2023-11-28T18:30:56.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.809+0000 7f2d824bb640 1 -- start start 2023-11-28T18:30:56.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.809+0000 7f2d824bb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2d7c15d1a0 0x7f2d7c0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:56.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.809+0000 7f2d824bb640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d7c0cf440 con 0x7f2d7c0ce970 2023-11-28T18:30:56.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.809+0000 7f2d824bb640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d7c0cffc0 con 0x7f2d7c0d0d40 2023-11-28T18:30:56.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.809+0000 7f2d824bb640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d7c0cf690 con 0x7f2d7c15d1a0 2023-11-28T18:30:56.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.809+0000 7f2d814b9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2d7c15d1a0 0x7f2d7c0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:56.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.809+0000 7f2d81cba640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f2d7c0ce970 0x7f2d7c0b2260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42190/0 (socket says 172.21.15.105:42190) 2023-11-28T18:30:56.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.809+0000 7f2d814b9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2d7c15d1a0 0x7f2d7c0ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59862/0 (socket says 172.21.15.105:59862) 2023-11-28T18:30:56.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.809+0000 7f2d81cba640 1 -- 172.21.15.105:0/2631182180 learned_addr learned my addr 172.21.15.105:0/2631182180 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:56.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.813+0000 7f2d814b9640 1 -- 172.21.15.105:0/2631182180 >> v1:172.21.15.148:6789/0 conn(0x7f2d7c0d0d40 legacy=0x7f2d7c0d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:56.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.813+0000 7f2d814b9640 1 -- 172.21.15.105:0/2631182180 >> v1:172.21.15.105:6789/0 conn(0x7f2d7c0ce970 legacy=0x7f2d7c0b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:56.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.813+0000 7f2d814b9640 1 -- 172.21.15.105:0/2631182180 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7c0cf970 con 0x7f2d7c15d1a0 2023-11-28T18:30:56.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.813+0000 7f2d73fff640 1 -- 172.21.15.105:0/2631182180 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 915206081 0 0) 0x7f2d7c0cf440 con 0x7f2d7c0ce970 2023-11-28T18:30:56.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.813+0000 7f2d814b9640 1 --2- 172.21.15.105:0/2631182180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d7c15d1a0 0x7f2d7c0ce430 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f2d780097c0 tx=0x7f2d7801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995e163ffa97975c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:56.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.813+0000 7f2d73fff640 1 -- 172.21.15.105:0/2631182180 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d780ac640 con 0x7f2d7c15d1a0 2023-11-28T18:30:56.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.813+0000 7f2d73fff640 1 -- 172.21.15.105:0/2631182180 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d780ac7a0 con 0x7f2d7c15d1a0 2023-11-28T18:30:56.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.813+0000 7f2d73fff640 1 -- 172.21.15.105:0/2631182180 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d780acb30 con 0x7f2d7c15d1a0 2023-11-28T18:30:56.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.813+0000 7f2d824bb640 1 -- 172.21.15.105:0/2631182180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d7c15d1a0 msgr2=0x7f2d7c0ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:56.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.813+0000 7f2d824bb640 1 --2- 172.21.15.105:0/2631182180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d7c15d1a0 0x7f2d7c0ce430 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f2d780097c0 tx=0x7f2d7801c3c0 comp rx=0 tx=0).stop 2023-11-28T18:30:56.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.813+0000 7f2d824bb640 1 -- 172.21.15.105:0/2631182180 shutdown_connections 2023-11-28T18:30:56.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.813+0000 7f2d824bb640 1 --2- 172.21.15.105:0/2631182180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d7c15d1a0 0x7f2d7c0ce430 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:56.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.813+0000 7f2d824bb640 1 -- 172.21.15.105:0/2631182180 >> 172.21.15.105:0/2631182180 conn(0x7f2d7c0d2e10 msgr2=0x7f2d7c0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:56.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.817+0000 7f2d824bb640 1 -- 172.21.15.105:0/2631182180 shutdown_connections 2023-11-28T18:30:56.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.817+0000 7f2d824bb640 1 -- 172.21.15.105:0/2631182180 wait complete. 2023-11-28T18:30:56.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.817+0000 7f2d824bb640 1 Processor -- start 2023-11-28T18:30:56.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.817+0000 7f2d824bb640 1 -- start start 2023-11-28T18:30:56.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.817+0000 7f2d824bb640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d7c0ce970 0x7f2d7c158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:56.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.817+0000 7f2d824bb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d7c0d0d40 0x7f2d7c0b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:56.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.817+0000 7f2d824bb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d7c15d1a0 0x7f2d7c0b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:56.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.817+0000 7f2d824bb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c15a500 con 0x7f2d7c15d1a0 2023-11-28T18:30:56.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.817+0000 7f2d824bb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c15a7e0 con 0x7f2d7c0d0d40 2023-11-28T18:30:56.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.817+0000 7f2d824bb640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c15aac0 con 0x7f2d7c0ce970 2023-11-28T18:30:56.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.817+0000 7f2d81cba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d7c15d1a0 0x7f2d7c0b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:56.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.817+0000 7f2d81cba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d7c15d1a0 0x7f2d7c0b7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59872/0 (socket says 172.21.15.105:59872) 2023-11-28T18:30:56.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.817+0000 7f2d81cba640 1 -- 172.21.15.105:0/2923840377 learned_addr learned my addr 172.21.15.105:0/2923840377 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:56.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.817+0000 7f2d814b9640 1 --2- 172.21.15.105:0/2923840377 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d7c0ce970 0x7f2d7c158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:56.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.817+0000 7f2d80cb8640 1 --2- 172.21.15.105:0/2923840377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d7c0d0d40 0x7f2d7c0b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:56.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.817+0000 7f2d81cba640 1 -- 172.21.15.105:0/2923840377 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d7c0ce970 msgr2=0x7f2d7c158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:56.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.817+0000 7f2d81cba640 1 --2- 172.21.15.105:0/2923840377 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d7c0ce970 0x7f2d7c158130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:56.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.817+0000 7f2d81cba640 1 -- 172.21.15.105:0/2923840377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d7c0d0d40 msgr2=0x7f2d7c0b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:56.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.817+0000 7f2d81cba640 1 --2- 172.21.15.105:0/2923840377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d7c0d0d40 0x7f2d7c0b7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:56.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.817+0000 7f2d81cba640 1 -- 172.21.15.105:0/2923840377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d78009420 con 0x7f2d7c15d1a0 2023-11-28T18:30:56.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.817+0000 7f2d81cba640 1 --2- 172.21.15.105:0/2923840377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d7c15d1a0 0x7f2d7c0b7fb0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400c780 tx=0x7f2d7400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:56.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.821+0000 7f2d71ffb640 1 -- 172.21.15.105:0/2923840377 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7400ce90 con 0x7f2d7c15d1a0 2023-11-28T18:30:56.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.821+0000 7f2d71ffb640 1 -- 172.21.15.105:0/2923840377 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7400a640 con 0x7f2d7c15d1a0 2023-11-28T18:30:56.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.821+0000 7f2d71ffb640 1 -- 172.21.15.105:0/2923840377 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7400aa70 con 0x7f2d7c15d1a0 2023-11-28T18:30:56.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.821+0000 7f2d824bb640 1 -- 172.21.15.105:0/2923840377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7c15ada0 con 0x7f2d7c15d1a0 2023-11-28T18:30:56.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.821+0000 7f2d824bb640 1 -- 172.21.15.105:0/2923840377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7c0ca0c0 con 0x7f2d7c15d1a0 2023-11-28T18:30:56.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.821+0000 7f2d824bb640 1 -- 172.21.15.105:0/2923840377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d7c0ca340 con 0x7f2d7c15d1a0 2023-11-28T18:30:56.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.821+0000 7f2d71ffb640 1 -- 172.21.15.105:0/2923840377 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f2d7400abd0 con 0x7f2d7c15d1a0 2023-11-28T18:30:56.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.821+0000 7f2d71ffb640 1 --2- 172.21.15.105:0/2923840377 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2d600668d0 0x7f2d60068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:56.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.821+0000 7f2d71ffb640 1 -- 172.21.15.105:0/2923840377 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f2d740838d0 con 0x7f2d7c15d1a0 2023-11-28T18:30:56.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.821+0000 7f2d814b9640 1 --2- 172.21.15.105:0/2923840377 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2d600668d0 0x7f2d60068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:56.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.825+0000 7f2d814b9640 1 --2- 172.21.15.105:0/2923840377 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2d600668d0 0x7f2d60068d90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2d780ae000 tx=0x7f2d780a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:56.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.825+0000 7f2d71ffb640 1 -- 172.21.15.105:0/2923840377 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2d74050c10 con 0x7f2d7c15d1a0 2023-11-28T18:30:56.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.977+0000 7f2d824bb640 1 -- 172.21.15.105:0/2923840377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2d7c074580 con 0x7f2d7c15d1a0 2023-11-28T18:30:56.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.981+0000 7f2d71ffb640 1 -- 172.21.15.105:0/2923840377 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v59)=0 v59) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2d74018070 con 0x7f2d7c15d1a0 2023-11-28T18:30:56.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.985+0000 7f2d824bb640 1 -- 172.21.15.105:0/2923840377 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2d600668d0 msgr2=0x7f2d60068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:56.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.985+0000 7f2d824bb640 1 --2- 172.21.15.105:0/2923840377 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2d600668d0 0x7f2d60068d90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2d780ae000 tx=0x7f2d780a7000 comp rx=0 tx=0).stop 2023-11-28T18:30:56.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.985+0000 7f2d824bb640 1 -- 172.21.15.105:0/2923840377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d7c15d1a0 msgr2=0x7f2d7c0b7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:56.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.985+0000 7f2d824bb640 1 --2- 172.21.15.105:0/2923840377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d7c15d1a0 0x7f2d7c0b7fb0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400c780 tx=0x7f2d7400cc40 comp rx=0 tx=0).stop 2023-11-28T18:30:56.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.985+0000 7f2d824bb640 1 -- 172.21.15.105:0/2923840377 shutdown_connections 2023-11-28T18:30:56.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.985+0000 7f2d824bb640 1 --2- 172.21.15.105:0/2923840377 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2d600668d0 0x7f2d60068d90 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:56.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.985+0000 7f2d824bb640 1 --2- 172.21.15.105:0/2923840377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d7c15d1a0 0x7f2d7c0b7fb0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:56.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.985+0000 7f2d824bb640 1 --2- 172.21.15.105:0/2923840377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d7c0d0d40 0x7f2d7c0b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:56.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.985+0000 7f2d824bb640 1 --2- 172.21.15.105:0/2923840377 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d7c0ce970 0x7f2d7c158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:56.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.985+0000 7f2d824bb640 1 -- 172.21.15.105:0/2923840377 >> 172.21.15.105:0/2923840377 conn(0x7f2d7c0d2e10 msgr2=0x7f2d7c15c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:56.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.985+0000 7f2d824bb640 1 -- 172.21.15.105:0/2923840377 shutdown_connections 2023-11-28T18:30:56.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:56.985+0000 7f2d824bb640 1 -- 172.21.15.105:0/2923840377 wait complete. 2023-11-28T18:30:57.004 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T18:30:57.004 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:30:57.106 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:30:57.170 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T18:30:57.171 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:30:57.171 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:30:57.180 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:30:57.180 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:30:57.180 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:30:57.190 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:30:57.190 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:30:57.190 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:30:57.231 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:30:57.231 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:30:57.231 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:30:57.241 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:30:57.241 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T18:30:57.242 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:30:57.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.413+0000 7f20ca9d1640 1 Processor -- start 2023-11-28T18:30:57.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.413+0000 7f20ca9d1640 1 -- start start 2023-11-28T18:30:57.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.413+0000 7f20ca9d1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f20c415d4f0 0x7f20c40d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:57.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.413+0000 7f20ca9d1640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20c40d3c30 0x7f20c40d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:57.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.413+0000 7f20ca9d1640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c40b25b0 con 0x7f20c40d1890 2023-11-28T18:30:57.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.413+0000 7f20ca9d1640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c40b2900 con 0x7f20c415d4f0 2023-11-28T18:30:57.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.413+0000 7f20ca9d1640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40cea30 con 0x7f20c40d3c30 2023-11-28T18:30:57.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.413+0000 7f20c8f6e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f20c40d1890 0x7f20c40cfa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42198/0 (socket says 172.21.15.105:42198) 2023-11-28T18:30:57.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.413+0000 7f20c3fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f20c415d4f0 0x7f20c40d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:57.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.413+0000 7f20c8f6e640 1 -- 172.21.15.105:0/3326978647 learned_addr learned my addr 172.21.15.105:0/3326978647 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:57.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.413+0000 7f20c37fe640 1 --2- 172.21.15.105:0/3326978647 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20c40d3c30 0x7f20c40d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:57.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.413+0000 7f20c3fff640 1 -- 172.21.15.105:0/3326978647 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20c40d3c30 msgr2=0x7f20c40d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:57.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.413+0000 7f20c3fff640 1 --2- 172.21.15.105:0/3326978647 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20c40d3c30 0x7f20c40d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:57.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.413+0000 7f20c3fff640 1 -- 172.21.15.105:0/3326978647 >> v1:172.21.15.105:6789/0 conn(0x7f20c40d1890 legacy=0x7f20c40cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:57.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.413+0000 7f20c3fff640 1 -- 172.21.15.105:0/3326978647 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c40ced10 con 0x7f20c415d4f0 2023-11-28T18:30:57.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.417+0000 7f20c3fff640 1 --2- 172.21.15.105:0/3326978647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20c415d4f0 0x7f20c40d0e10 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f20b40097c0 tx=0x7f20b401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332b323ae5316dd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:57.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.417+0000 7f20c2ffd640 1 -- 172.21.15.105:0/3326978647 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b40ab600 con 0x7f20c415d4f0 2023-11-28T18:30:57.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.417+0000 7f20c2ffd640 1 -- 172.21.15.105:0/3326978647 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b40a7050 con 0x7f20c415d4f0 2023-11-28T18:30:57.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.417+0000 7f20c2ffd640 1 -- 172.21.15.105:0/3326978647 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b40ab980 con 0x7f20c415d4f0 2023-11-28T18:30:57.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.417+0000 7f20ca9d1640 1 -- 172.21.15.105:0/3326978647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20c415d4f0 msgr2=0x7f20c40d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:57.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.417+0000 7f20ca9d1640 1 --2- 172.21.15.105:0/3326978647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20c415d4f0 0x7f20c40d0e10 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f20b40097c0 tx=0x7f20b401c3c0 comp rx=0 tx=0).stop 2023-11-28T18:30:57.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.417+0000 7f20ca9d1640 1 -- 172.21.15.105:0/3326978647 shutdown_connections 2023-11-28T18:30:57.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.417+0000 7f20ca9d1640 1 --2- 172.21.15.105:0/3326978647 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20c40d3c30 0x7f20c40d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:57.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.417+0000 7f20ca9d1640 1 --2- 172.21.15.105:0/3326978647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20c415d4f0 0x7f20c40d0e10 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:57.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.417+0000 7f20ca9d1640 1 -- 172.21.15.105:0/3326978647 >> 172.21.15.105:0/3326978647 conn(0x7f20c40d3030 msgr2=0x7f20c40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:57.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.417+0000 7f20ca9d1640 1 -- 172.21.15.105:0/3326978647 shutdown_connections 2023-11-28T18:30:57.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.417+0000 7f20ca9d1640 1 -- 172.21.15.105:0/3326978647 wait complete. 2023-11-28T18:30:57.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20ca9d1640 1 Processor -- start 2023-11-28T18:30:57.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20ca9d1640 1 -- start start 2023-11-28T18:30:57.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20ca9d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c40d1890 0x7f20c414a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:57.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20ca9d1640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20c40d3c30 0x7f20c40c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:57.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20ca9d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20c415d4f0 0x7f20c406b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:57.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20ca9d1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40ccbe0 con 0x7f20c415d4f0 2023-11-28T18:30:57.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20ca9d1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40cc8b0 con 0x7f20c40d1890 2023-11-28T18:30:57.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20ca9d1640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40ca1a0 con 0x7f20c40d3c30 2023-11-28T18:30:57.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20c8f6e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20c415d4f0 0x7f20c406b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:57.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20c8f6e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20c415d4f0 0x7f20c406b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59888/0 (socket says 172.21.15.105:59888) 2023-11-28T18:30:57.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20c8f6e640 1 -- 172.21.15.105:0/2429790034 learned_addr learned my addr 172.21.15.105:0/2429790034 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:57.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20c3fff640 1 --2- 172.21.15.105:0/2429790034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c40d1890 0x7f20c414a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:57.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20c37fe640 1 --2- 172.21.15.105:0/2429790034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20c40d3c30 0x7f20c40c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:57.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20c8f6e640 1 -- 172.21.15.105:0/2429790034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20c40d3c30 msgr2=0x7f20c40c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:57.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20c8f6e640 1 --2- 172.21.15.105:0/2429790034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20c40d3c30 0x7f20c40c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:57.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20c8f6e640 1 -- 172.21.15.105:0/2429790034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c40d1890 msgr2=0x7f20c414a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:57.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20c8f6e640 1 --2- 172.21.15.105:0/2429790034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c40d1890 0x7f20c414a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:57.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20c8f6e640 1 -- 172.21.15.105:0/2429790034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b4009420 con 0x7f20c415d4f0 2023-11-28T18:30:57.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20c3fff640 1 --2- 172.21.15.105:0/2429790034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c40d1890 0x7f20c414a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:57.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20c37fe640 1 --2- 172.21.15.105:0/2429790034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20c40d3c30 0x7f20c40c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:30:57.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20c8f6e640 1 --2- 172.21.15.105:0/2429790034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20c415d4f0 0x7f20c406b6a0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00ca80 tx=0x7f20bc00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:57.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20c0ff9640 1 -- 172.21.15.105:0/2429790034 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc00ac30 con 0x7f20c415d4f0 2023-11-28T18:30:57.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20c0ff9640 1 -- 172.21.15.105:0/2429790034 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20bc00ad90 con 0x7f20c415d4f0 2023-11-28T18:30:57.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20ca9d1640 1 -- 172.21.15.105:0/2429790034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c40ca480 con 0x7f20c415d4f0 2023-11-28T18:30:57.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20c0ff9640 1 -- 172.21.15.105:0/2429790034 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc012600 con 0x7f20c415d4f0 2023-11-28T18:30:57.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.421+0000 7f20ca9d1640 1 -- 172.21.15.105:0/2429790034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c40c9d40 con 0x7f20c415d4f0 2023-11-28T18:30:57.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.425+0000 7f20ca9d1640 1 -- 172.21.15.105:0/2429790034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c40ccd50 con 0x7f20c415d4f0 2023-11-28T18:30:57.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.425+0000 7f20c0ff9640 1 -- 172.21.15.105:0/2429790034 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f20bc007070 con 0x7f20c415d4f0 2023-11-28T18:30:57.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.425+0000 7f20c0ff9640 1 --2- 172.21.15.105:0/2429790034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f20b0066900 0x7f20b0068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:57.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.425+0000 7f20c0ff9640 1 -- 172.21.15.105:0/2429790034 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f20bc0840d0 con 0x7f20c415d4f0 2023-11-28T18:30:57.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.425+0000 7f20c3fff640 1 --2- 172.21.15.105:0/2429790034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f20b0066900 0x7f20b0068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:57.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.425+0000 7f20c3fff640 1 --2- 172.21.15.105:0/2429790034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f20b0066900 0x7f20b0068dc0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f20b40ad000 tx=0x7f20b40a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:57.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.429+0000 7f20c0ff9640 1 -- 172.21.15.105:0/2429790034 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f20bc0514c0 con 0x7f20c415d4f0 2023-11-28T18:30:57.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.609+0000 7f20ca9d1640 1 -- 172.21.15.105:0/2429790034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20c4067570 con 0x7f20c415d4f0 2023-11-28T18:30:57.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.609+0000 7f20c0ff9640 1 -- 172.21.15.105:0/2429790034 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+3715 (secure 0 0 0) 0x7f20bc054ae0 con 0x7f20c415d4f0 2023-11-28T18:30:57.609 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:57.609 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":113,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/2603493310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2603493310},{"type":"v1","addr":"172.21.15.148:6834","nonce":2603493310}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":8035,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2675884191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":2675884191},{"type":"v1","addr":"172.21.15.148:6835","nonce":2675884191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113},{"gid":8514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2386171834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2386171834},{"type":"v1","addr":"172.21.15.105:6835","nonce":2386171834}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113},{"gid":8520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/3012020791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3012020791},{"type":"v1","addr":"172.21.15.105:6837","nonce":3012020791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113}],"filesystems":[]} 2023-11-28T18:30:57.610 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 113 2023-11-28T18:30:57.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.609+0000 7f20ca9d1640 1 -- 172.21.15.105:0/2429790034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f20b0066900 msgr2=0x7f20b0068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:57.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.609+0000 7f20ca9d1640 1 --2- 172.21.15.105:0/2429790034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f20b0066900 0x7f20b0068dc0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f20b40ad000 tx=0x7f20b40a6000 comp rx=0 tx=0).stop 2023-11-28T18:30:57.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.609+0000 7f20ca9d1640 1 -- 172.21.15.105:0/2429790034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20c415d4f0 msgr2=0x7f20c406b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:57.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.609+0000 7f20ca9d1640 1 --2- 172.21.15.105:0/2429790034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20c415d4f0 0x7f20c406b6a0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00ca80 tx=0x7f20bc00cf40 comp rx=0 tx=0).stop 2023-11-28T18:30:57.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.609+0000 7f20ca9d1640 1 -- 172.21.15.105:0/2429790034 shutdown_connections 2023-11-28T18:30:57.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.609+0000 7f20ca9d1640 1 --2- 172.21.15.105:0/2429790034 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f20b0066900 0x7f20b0068dc0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:57.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.609+0000 7f20ca9d1640 1 --2- 172.21.15.105:0/2429790034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20c415d4f0 0x7f20c406b6a0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:57.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.609+0000 7f20ca9d1640 1 --2- 172.21.15.105:0/2429790034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20c40d3c30 0x7f20c40c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:57.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.609+0000 7f20ca9d1640 1 --2- 172.21.15.105:0/2429790034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c40d1890 0x7f20c414a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:57.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.609+0000 7f20ca9d1640 1 -- 172.21.15.105:0/2429790034 >> 172.21.15.105:0/2429790034 conn(0x7f20c40d3030 msgr2=0x7f20c406c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:57.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.609+0000 7f20ca9d1640 1 -- 172.21.15.105:0/2429790034 shutdown_connections 2023-11-28T18:30:57.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.609+0000 7f20ca9d1640 1 -- 172.21.15.105:0/2429790034 wait complete. 2023-11-28T18:30:57.631 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T18:30:57.631 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T18:30:57.631 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T18:30:57.631 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:30:57.631 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T18:30:57.631 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T18:30:57.632 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T18:30:57.632 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:30:57.632 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T18:30:57.632 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T18:30:57.632 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T18:30:57.632 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:30:57.633 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T18:30:57.633 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T18:30:57.633 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T18:30:57.633 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:30:57.646 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T18:30:57.646 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T18:30:57.647 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T18:30:57.648 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T18:30:57.650 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T18:30:57.651 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T18:30:57.652 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T18:30:57.652 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T18:30:57.694 INFO:tasks.ceph.mds.c:Started 2023-11-28T18:30:57.694 DEBUG:teuthology.parallel:result is None 2023-11-28T18:30:57.695 INFO:tasks.ceph.mds.d:Started 2023-11-28T18:30:57.695 DEBUG:teuthology.parallel:result is None 2023-11-28T18:30:57.696 INFO:tasks.ceph.mds.a:Started 2023-11-28T18:30:57.696 DEBUG:teuthology.parallel:result is None 2023-11-28T18:30:57.697 INFO:tasks.ceph.mds.b:Started 2023-11-28T18:30:57.698 DEBUG:teuthology.parallel:result is None 2023-11-28T18:30:57.699 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T18:30:57.783 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2023-11-28T18:30:57.784 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2023-11-28T18:30:57.789 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2023-11-28T18:30:57.789 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2023-11-28T18:30:57.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.845+0000 7f4296db8640 1 Processor -- start 2023-11-28T18:30:57.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.845+0000 7f4296db8640 1 -- start start 2023-11-28T18:30:57.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.845+0000 7f4296db8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f42900d3c30 0x7f42900d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:57.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.845+0000 7f4296db8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f42900b2d20 0x7f42900b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:57.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.845+0000 7f4296db8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42900b25b0 con 0x7f429015d4f0 2023-11-28T18:30:57.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.845+0000 7f4296db8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42900b2900 con 0x7f42900b2d20 2023-11-28T18:30:57.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.845+0000 7f4296db8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42900d0a20 con 0x7f42900d3c30 2023-11-28T18:30:57.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.845+0000 7f4295355640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f42900b2d20 0x7f42900b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:57.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.845+0000 7f4294b54640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f429015d4f0 0x7f42900d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42238/0 (socket says 172.21.15.105:42238) 2023-11-28T18:30:57.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.845+0000 7f4294b54640 1 -- 172.21.15.105:0/2588362695 learned_addr learned my addr 172.21.15.105:0/2588362695 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:57.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.845+0000 7f428ffff640 1 --2- 172.21.15.105:0/2588362695 >> v2:172.21.15.148:3300/0 conn(0x7f42900d3c30 0x7f42900d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:57.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.845+0000 7f4295355640 1 -- 172.21.15.105:0/2588362695 >> v2:172.21.15.148:3300/0 conn(0x7f42900d3c30 msgr2=0x7f42900d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:57.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.845+0000 7f4295355640 1 --2- 172.21.15.105:0/2588362695 >> v2:172.21.15.148:3300/0 conn(0x7f42900d3c30 0x7f42900d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:57.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.845+0000 7f4295355640 1 -- 172.21.15.105:0/2588362695 >> v1:172.21.15.105:6789/0 conn(0x7f429015d4f0 legacy=0x7f42900d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:57.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.845+0000 7f4295355640 1 -- 172.21.15.105:0/2588362695 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42900cea30 con 0x7f42900b2d20 2023-11-28T18:30:57.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.845+0000 7f428f7fe640 1 -- 172.21.15.105:0/2588362695 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3522808726 0 0) 0x7f42900b25b0 con 0x7f429015d4f0 2023-11-28T18:30:57.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.849+0000 7f4295355640 1 --2- 172.21.15.105:0/2588362695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42900b2d20 0x7f42900b3150 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f429008c430 tx=0x7f428801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe672102cfb7eaf5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:57.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.849+0000 7f428f7fe640 1 -- 172.21.15.105:0/2588362695 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42880a5910 con 0x7f42900b2d20 2023-11-28T18:30:57.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.849+0000 7f428f7fe640 1 -- 172.21.15.105:0/2588362695 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42880a5a70 con 0x7f42900b2d20 2023-11-28T18:30:57.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.849+0000 7f428f7fe640 1 -- 172.21.15.105:0/2588362695 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42880a5e00 con 0x7f42900b2d20 2023-11-28T18:30:57.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.849+0000 7f4296db8640 1 -- 172.21.15.105:0/2588362695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42900b2d20 msgr2=0x7f42900b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:57.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.849+0000 7f4296db8640 1 --2- 172.21.15.105:0/2588362695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42900b2d20 0x7f42900b3150 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f429008c430 tx=0x7f428801c850 comp rx=0 tx=0).stop 2023-11-28T18:30:57.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.849+0000 7f4296db8640 1 -- 172.21.15.105:0/2588362695 shutdown_connections 2023-11-28T18:30:57.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.849+0000 7f4296db8640 1 --2- 172.21.15.105:0/2588362695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42900b2d20 0x7f42900b3150 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:57.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.849+0000 7f4296db8640 1 --2- 172.21.15.105:0/2588362695 >> v2:172.21.15.148:3300/0 conn(0x7f42900d3c30 0x7f42900d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:57.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.849+0000 7f4296db8640 1 -- 172.21.15.105:0/2588362695 >> 172.21.15.105:0/2588362695 conn(0x7f42900d3030 msgr2=0x7f42900d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:57.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.849+0000 7f4296db8640 1 -- 172.21.15.105:0/2588362695 shutdown_connections 2023-11-28T18:30:57.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.849+0000 7f4296db8640 1 -- 172.21.15.105:0/2588362695 wait complete. 2023-11-28T18:30:57.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.849+0000 7f4296db8640 1 Processor -- start 2023-11-28T18:30:57.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f4296db8640 1 -- start start 2023-11-28T18:30:57.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f4296db8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42900b2d20 0x7f4290155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:57.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f4296db8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42900d3c30 0x7f4290154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:57.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f4296db8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f429015d4f0 0x7f42900c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:57.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f4296db8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42900cb8c0 con 0x7f429015d4f0 2023-11-28T18:30:57.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f4296db8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4290155aa0 con 0x7f42900d3c30 2023-11-28T18:30:57.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f4296db8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4290154e90 con 0x7f42900b2d20 2023-11-28T18:30:57.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f4295355640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f429015d4f0 0x7f42900c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:57.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f4295355640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f429015d4f0 0x7f42900c8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59912/0 (socket says 172.21.15.105:59912) 2023-11-28T18:30:57.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f4295355640 1 -- 172.21.15.105:0/3916671444 learned_addr learned my addr 172.21.15.105:0/3916671444 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:57.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f4294b54640 1 --2- 172.21.15.105:0/3916671444 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42900b2d20 0x7f4290155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:57.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f428ffff640 1 --2- 172.21.15.105:0/3916671444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42900d3c30 0x7f4290154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:57.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f4295355640 1 -- 172.21.15.105:0/3916671444 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42900b2d20 msgr2=0x7f4290155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:57.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f4295355640 1 --2- 172.21.15.105:0/3916671444 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42900b2d20 0x7f4290155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:57.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f4295355640 1 -- 172.21.15.105:0/3916671444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42900d3c30 msgr2=0x7f4290154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:57.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f4295355640 1 --2- 172.21.15.105:0/3916671444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42900d3c30 0x7f4290154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:57.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f4295355640 1 -- 172.21.15.105:0/3916671444 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4288009a70 con 0x7f429015d4f0 2023-11-28T18:30:57.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f4294b54640 1 --2- 172.21.15.105:0/3916671444 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42900b2d20 0x7f4290155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:30:57.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f4295355640 1 --2- 172.21.15.105:0/3916671444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f429015d4f0 0x7f42900c8c10 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f4288019460 tx=0x7f42880195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:57.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f428d7fa640 1 -- 172.21.15.105:0/3916671444 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42880ad850 con 0x7f429015d4f0 2023-11-28T18:30:57.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f428d7fa640 1 -- 172.21.15.105:0/3916671444 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42880ad9b0 con 0x7f429015d4f0 2023-11-28T18:30:57.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f428d7fa640 1 -- 172.21.15.105:0/3916671444 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42880add20 con 0x7f429015d4f0 2023-11-28T18:30:57.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f4296db8640 1 -- 172.21.15.105:0/3916671444 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4290155170 con 0x7f429015d4f0 2023-11-28T18:30:57.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f4296db8640 1 -- 172.21.15.105:0/3916671444 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4290155730 con 0x7f429015d4f0 2023-11-28T18:30:57.856 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-11-28T18:30:57.856 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-11-28T18:30:57.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.853+0000 7f4296db8640 1 -- 172.21.15.105:0/3916671444 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4290155c10 con 0x7f429015d4f0 2023-11-28T18:30:57.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.857+0000 7f428d7fa640 1 -- 172.21.15.105:0/3916671444 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f42880b4070 con 0x7f429015d4f0 2023-11-28T18:30:57.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.857+0000 7f428d7fa640 1 --2- 172.21.15.105:0/3916671444 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f42740668d0 0x7f4274068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.857+0000 7f428d7fa640 1 -- 172.21.15.105:0/3916671444 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f428811ef30 con 0x7f429015d4f0 2023-11-28T18:30:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.857+0000 7f4294b54640 1 --2- 172.21.15.105:0/3916671444 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f42740668d0 0x7f4274068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.857+0000 7f4294b54640 1 --2- 172.21.15.105:0/3916671444 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f42740668d0 0x7f4274068d90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f4284007fd0 tx=0x7f4284007d60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:57.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:57.861+0000 7f428d7fa640 1 -- 172.21.15.105:0/3916671444 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f42880ec290 con 0x7f429015d4f0 2023-11-28T18:30:58.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:58.033+0000 7f4296db8640 1 -- 172.21.15.105:0/3916671444 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4290067570 con 0x7f429015d4f0 2023-11-28T18:30:59.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.009+0000 7f428d7fa640 1 -- 172.21.15.105:0/3916671444 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v113) v1 ==== 97+0+0 (secure 0 0 0) 0x7f42880ef8b0 con 0x7f429015d4f0 2023-11-28T18:30:59.011 INFO:teuthology.orchestra.run.smithi105.stderr: removed all blocklist entries 2023-11-28T18:30:59.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.013+0000 7f4296db8640 1 -- 172.21.15.105:0/3916671444 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f42740668d0 msgr2=0x7f4274068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:59.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.013+0000 7f4296db8640 1 --2- 172.21.15.105:0/3916671444 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f42740668d0 0x7f4274068d90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f4284007fd0 tx=0x7f4284007d60 comp rx=0 tx=0).stop 2023-11-28T18:30:59.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.013+0000 7f4296db8640 1 -- 172.21.15.105:0/3916671444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f429015d4f0 msgr2=0x7f42900c8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:59.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.013+0000 7f4296db8640 1 --2- 172.21.15.105:0/3916671444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f429015d4f0 0x7f42900c8c10 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f4288019460 tx=0x7f42880195f0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.013+0000 7f4296db8640 1 -- 172.21.15.105:0/3916671444 shutdown_connections 2023-11-28T18:30:59.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.013+0000 7f4296db8640 1 --2- 172.21.15.105:0/3916671444 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f42740668d0 0x7f4274068d90 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.013+0000 7f4296db8640 1 --2- 172.21.15.105:0/3916671444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f429015d4f0 0x7f42900c8c10 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.013+0000 7f4296db8640 1 --2- 172.21.15.105:0/3916671444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42900d3c30 0x7f4290154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.013+0000 7f4296db8640 1 --2- 172.21.15.105:0/3916671444 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42900b2d20 0x7f4290155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.013+0000 7f4296db8640 1 -- 172.21.15.105:0/3916671444 >> 172.21.15.105:0/3916671444 conn(0x7f42900d3030 msgr2=0x7f4290072eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:59.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.013+0000 7f4296db8640 1 -- 172.21.15.105:0/3916671444 shutdown_connections 2023-11-28T18:30:59.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.013+0000 7f4296db8640 1 -- 172.21.15.105:0/3916671444 wait complete. 2023-11-28T18:30:59.037 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T18:30:59.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.177+0000 7f1741baa640 1 Processor -- start 2023-11-28T18:30:59.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.177+0000 7f1741baa640 1 -- start start 2023-11-28T18:30:59.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.181+0000 7f1741baa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f173c0af340 0x7f173c0721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:59.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.181+0000 7f1741baa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f173c06ed00 0x7f173c0d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:59.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.181+0000 7f1741baa640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f173c06f160 con 0x7f173c0d08b0 2023-11-28T18:30:59.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.181+0000 7f1741baa640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f173c0b34a0 con 0x7f173c0af340 2023-11-28T18:30:59.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.181+0000 7f1741baa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173c0b2ee0 con 0x7f173c06ed00 2023-11-28T18:30:59.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.181+0000 7f173bfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f173c0d08b0 0x7f173c06a580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42240/0 (socket says 172.21.15.105:42240) 2023-11-28T18:30:59.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.181+0000 7f173bfff640 1 -- 172.21.15.105:0/2103272831 learned_addr learned my addr 172.21.15.105:0/2103272831 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:59.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.181+0000 7f1733fff640 1 --2- 172.21.15.105:0/2103272831 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f173c06ed00 0x7f173c0d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:59.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.181+0000 7f173b7fe640 1 --2- 172.21.15.105:0/2103272831 >> v2:172.21.15.148:3300/0 conn(0x7f173c0af340 0x7f173c0721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:59.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.181+0000 7f173affd640 1 -- 172.21.15.105:0/2103272831 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007008141 0 0) 0x7f173c06f160 con 0x7f173c0d08b0 2023-11-28T18:30:59.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.181+0000 7f173affd640 1 -- 172.21.15.105:0/2103272831 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1728003530 con 0x7f173c0d08b0 2023-11-28T18:30:59.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.181+0000 7f173affd640 1 -- 172.21.15.105:0/2103272831 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1274238603 0 0) 0x7f1728003530 con 0x7f173c0d08b0 2023-11-28T18:30:59.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.181+0000 7f1733fff640 1 -- 172.21.15.105:0/2103272831 >> v2:172.21.15.148:3300/0 conn(0x7f173c0af340 msgr2=0x7f173c0721d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:59.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.181+0000 7f1733fff640 1 --2- 172.21.15.105:0/2103272831 >> v2:172.21.15.148:3300/0 conn(0x7f173c0af340 0x7f173c0721d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.181+0000 7f1733fff640 1 -- 172.21.15.105:0/2103272831 >> v1:172.21.15.105:6789/0 conn(0x7f173c0d08b0 legacy=0x7f173c06a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:59.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.181+0000 7f1733fff640 1 -- 172.21.15.105:0/2103272831 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173c0b31c0 con 0x7f173c06ed00 2023-11-28T18:30:59.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.185+0000 7f1733fff640 1 --2- 172.21.15.105:0/2103272831 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f173c06ed00 0x7f173c0d0370 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f17240097c0 tx=0x7f172401c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb8c739215c02d6b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:59.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.185+0000 7f173affd640 1 -- 172.21.15.105:0/2103272831 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1734001a00 con 0x7f173c06ed00 2023-11-28T18:30:59.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.185+0000 7f173affd640 1 -- 172.21.15.105:0/2103272831 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17240a7050 con 0x7f173c06ed00 2023-11-28T18:30:59.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.185+0000 7f173affd640 1 -- 172.21.15.105:0/2103272831 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17240ab600 con 0x7f173c06ed00 2023-11-28T18:30:59.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.185+0000 7f1741baa640 1 -- 172.21.15.105:0/2103272831 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f173c06ed00 msgr2=0x7f173c0d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:59.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.185+0000 7f1741baa640 1 --2- 172.21.15.105:0/2103272831 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f173c06ed00 0x7f173c0d0370 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f17240097c0 tx=0x7f172401c370 comp rx=0 tx=0).stop 2023-11-28T18:30:59.185 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.185+0000 7f1741baa640 1 -- 172.21.15.105:0/2103272831 shutdown_connections 2023-11-28T18:30:59.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.185+0000 7f1741baa640 1 --2- 172.21.15.105:0/2103272831 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f173c06ed00 0x7f173c0d0370 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.185+0000 7f1741baa640 1 --2- 172.21.15.105:0/2103272831 >> v2:172.21.15.148:3300/0 conn(0x7f173c0af340 0x7f173c0721d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.185+0000 7f1741baa640 1 -- 172.21.15.105:0/2103272831 >> 172.21.15.105:0/2103272831 conn(0x7f173c06b860 msgr2=0x7f173c06bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:59.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.185+0000 7f1741baa640 1 -- 172.21.15.105:0/2103272831 shutdown_connections 2023-11-28T18:30:59.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.185+0000 7f1741baa640 1 -- 172.21.15.105:0/2103272831 wait complete. 2023-11-28T18:30:59.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1741baa640 1 Processor -- start 2023-11-28T18:30:59.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1741baa640 1 -- start start 2023-11-28T18:30:59.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1741baa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f173c06ed00 0x7f173c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:59.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1741baa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f173c0af340 0x7f173c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:59.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1741baa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f173c0d08b0 0x7f173c143150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:59.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1741baa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c15f110 con 0x7f173c0af340 2023-11-28T18:30:59.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1741baa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c15eae0 con 0x7f173c06ed00 2023-11-28T18:30:59.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1741baa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173c143b80 con 0x7f173c0d08b0 2023-11-28T18:30:59.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1733fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f173c0af340 0x7f173c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:59.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1733fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f173c0af340 0x7f173c15f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59918/0 (socket says 172.21.15.105:59918) 2023-11-28T18:30:59.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1733fff640 1 -- 172.21.15.105:0/3965292144 learned_addr learned my addr 172.21.15.105:0/3965292144 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:59.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f173b7fe640 1 --2- 172.21.15.105:0/3965292144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f173c06ed00 0x7f173c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:59.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f173bfff640 1 --2- 172.21.15.105:0/3965292144 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f173c0d08b0 0x7f173c143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:59.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1733fff640 1 -- 172.21.15.105:0/3965292144 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f173c0d08b0 msgr2=0x7f173c143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:59.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1733fff640 1 --2- 172.21.15.105:0/3965292144 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f173c0d08b0 0x7f173c143150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1733fff640 1 -- 172.21.15.105:0/3965292144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f173c06ed00 msgr2=0x7f173c15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:59.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1733fff640 1 --2- 172.21.15.105:0/3965292144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f173c06ed00 0x7f173c15aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1733fff640 1 -- 172.21.15.105:0/3965292144 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1724009420 con 0x7f173c0af340 2023-11-28T18:30:59.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1733fff640 1 --2- 172.21.15.105:0/3965292144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f173c0af340 0x7f173c15f5a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f1724000c10 tx=0x7f1724019110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:59.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1738ff9640 1 -- 172.21.15.105:0/3965292144 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724003ab0 con 0x7f173c0af340 2023-11-28T18:30:59.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1738ff9640 1 -- 172.21.15.105:0/3965292144 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1724003c10 con 0x7f173c0af340 2023-11-28T18:30:59.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1738ff9640 1 -- 172.21.15.105:0/3965292144 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17240ab400 con 0x7f173c0af340 2023-11-28T18:30:59.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1741baa640 1 -- 172.21.15.105:0/3965292144 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f173c143e60 con 0x7f173c0af340 2023-11-28T18:30:59.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1741baa640 1 -- 172.21.15.105:0/3965292144 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173c15afc0 con 0x7f173c0af340 2023-11-28T18:30:59.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.189+0000 7f1741baa640 1 -- 172.21.15.105:0/3965292144 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f173c067570 con 0x7f173c0af340 2023-11-28T18:30:59.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.193+0000 7f1738ff9640 1 -- 172.21.15.105:0/3965292144 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f17240b2040 con 0x7f173c0af340 2023-11-28T18:30:59.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.193+0000 7f1738ff9640 1 --2- 172.21.15.105:0/3965292144 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f171c0668d0 0x7f171c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:59.192 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.193+0000 7f1738ff9640 1 -- 172.21.15.105:0/3965292144 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f172411e070 con 0x7f173c0af340 2023-11-28T18:30:59.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.193+0000 7f173b7fe640 1 --2- 172.21.15.105:0/3965292144 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f171c0668d0 0x7f171c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:59.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.193+0000 7f173b7fe640 1 --2- 172.21.15.105:0/3965292144 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f171c0668d0 0x7f171c068d90 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f172c009b60 tx=0x7f172c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:59.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.197+0000 7f1738ff9640 1 -- 172.21.15.105:0/3965292144 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f17240eb4b0 con 0x7f173c0af340 2023-11-28T18:30:59.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.401+0000 7f1741baa640 1 -- 172.21.15.105:0/3965292144 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f173c143ff0 con 0x7f173c0af340 2023-11-28T18:30:59.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.401+0000 7f1738ff9640 1 -- 172.21.15.105:0/3965292144 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f17240eead0 con 0x7f173c0af340 2023-11-28T18:30:59.400 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:59.400 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:30:59.400 INFO:teuthology.orchestra.run.smithi105.stdout: "auth_dump": [ 2023-11-28T18:30:59.400 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:59.400 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.a", 2023-11-28T18:30:59.400 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlOXSTGxAAU5kGNB3tznYsDNG51vCIZA==", 2023-11-28T18:30:59.400 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:59.400 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:30:59.401 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:30:59.402 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:30:59.402 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:59.402 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:59.402 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:59.402 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:59.403 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.b", 2023-11-28T18:30:59.403 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlrnbWJxAAKefl0YI2MhobnAOIj6WeaQ==", 2023-11-28T18:30:59.403 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:59.403 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:30:59.403 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:30:59.403 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:30:59.403 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:59.403 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:59.403 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:59.404 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:59.404 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.c", 2023-11-28T18:30:59.404 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2Zl5Y7GIhAA5OvgkNNFaI7LnzSsQVdFsQ==", 2023-11-28T18:30:59.404 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:59.404 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:30:59.404 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:30:59.404 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:30:59.404 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:59.405 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:59.405 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:59.405 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:59.405 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.d", 2023-11-28T18:30:59.405 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlmfYdLxAAreI7UE1YCjzkj6yl3s5Fwg==", 2023-11-28T18:30:59.405 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:59.405 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:30:59.405 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:30:59.405 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:30:59.405 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:59.405 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:59.406 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:59.406 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:59.406 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.0", 2023-11-28T18:30:59.406 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDtL2ZlphxQEBAAQ280OXh1zeaotIdVU8oZng==", 2023-11-28T18:30:59.406 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:59.406 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:30:59.406 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:30:59.406 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:59.406 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:59.406 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:59.406 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:59.406 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.1", 2023-11-28T18:30:59.407 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDwL2ZlezmOFxAARRBIOTkVJYfquIdcpW7sQQ==", 2023-11-28T18:30:59.407 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:59.407 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:30:59.407 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:30:59.407 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:59.407 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:59.407 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:59.407 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:59.407 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.2", 2023-11-28T18:30:59.407 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDzL2Zl4YcSHxAA2ww2F4TAD/0H/ZJ+aVMGEQ==", 2023-11-28T18:30:59.408 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:59.408 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:30:59.408 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:30:59.408 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:59.408 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:59.408 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:59.408 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:59.408 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.3", 2023-11-28T18:30:59.409 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD2L2ZlNxWxJRAAImp7TjTF18KwFTZ14DM6EA==", 2023-11-28T18:30:59.409 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:59.409 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:30:59.409 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:30:59.409 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:59.409 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:59.409 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:59.409 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:59.409 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.4", 2023-11-28T18:30:59.409 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD8L2ZlOAieLxAAz8g+anH8uJRfsttUS41GGQ==", 2023-11-28T18:30:59.409 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:59.409 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:30:59.410 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:30:59.410 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:59.410 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:59.410 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:59.410 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:59.410 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.5", 2023-11-28T18:30:59.410 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD/L2Zl5PKTNxAAPk+uIodLPjSd0EEMVT6iEw==", 2023-11-28T18:30:59.410 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:59.410 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:30:59.410 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:30:59.410 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:59.411 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:59.411 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:59.411 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:59.411 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.6", 2023-11-28T18:30:59.411 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQADMGZll+FUAxAAHlym5Z8rd/OF1911ZE2OUg==", 2023-11-28T18:30:59.411 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:59.411 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:30:59.411 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:30:59.411 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:59.412 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:59.412 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:59.412 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:59.412 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.7", 2023-11-28T18:30:59.412 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQAGMGZl8GO4ChAAVn3N6DnxUcs+ew09QvGo2w==", 2023-11-28T18:30:59.412 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:59.412 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:30:59.412 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:30:59.412 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:59.412 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:59.412 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:59.412 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:59.413 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.0", 2023-11-28T18:30:59.413 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlSt0VMxAAQ8+j1BgBPMJ2cuiZj8aG0w==", 2023-11-28T18:30:59.413 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:59.413 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:30:59.413 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow r", 2023-11-28T18:30:59.413 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-28T18:30:59.413 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:59.414 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:59.414 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:59.414 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.1", 2023-11-28T18:30:59.414 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDqL2ZlfZ1/ARAAhnJB6oj4Cc6p/oCjM6hTCw==", 2023-11-28T18:30:59.414 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:59.414 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:30:59.414 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:30:59.414 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:30:59.414 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:30:59.414 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:59.415 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:59.415 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:59.415 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.2", 2023-11-28T18:30:59.415 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlPLtyORAAU2GST6TJzlTfh6YqEb3U3w==", 2023-11-28T18:30:59.415 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:59.415 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:30:59.415 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:30:59.415 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:30:59.415 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:30:59.415 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:59.415 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:59.415 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:59.416 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.3", 2023-11-28T18:30:59.416 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDqL2ZlhiFuBxAAKQRxCdtqWpAHy1AVvdj9Sg==", 2023-11-28T18:30:59.416 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:59.416 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:30:59.416 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:30:59.416 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:30:59.416 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:30:59.416 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:59.416 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:59.416 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:59.417 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.admin", 2023-11-28T18:30:59.417 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDoL2ZlkuP2OhAAxsdZEF9122vkH53EwB8Qug==", 2023-11-28T18:30:59.417 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:59.417 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:30:59.417 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:30:59.417 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:30:59.417 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:59.417 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:59.417 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:59.417 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:59.417 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.x", 2023-11-28T18:30:59.418 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlEG49FxAAespsG+QK1WTpbmpV+DIwiw==", 2023-11-28T18:30:59.418 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:59.418 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:30:59.418 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2023-11-28T18:30:59.418 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:59.418 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:59.418 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:30:59.418 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:30:59.418 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.y", 2023-11-28T18:30:59.418 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlU89RExAAwMJp+8zZ6aFEt0KbiB6toA==", 2023-11-28T18:30:59.418 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:30:59.419 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:30:59.419 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2023-11-28T18:30:59.419 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:30:59.419 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:59.419 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:30:59.419 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2023-11-28T18:30:59.419 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:30:59.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.401+0000 7f1741baa640 1 -- 172.21.15.105:0/3965292144 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f171c0668d0 msgr2=0x7f171c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:59.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.401+0000 7f1741baa640 1 --2- 172.21.15.105:0/3965292144 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f171c0668d0 0x7f171c068d90 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f172c009b60 tx=0x7f172c009190 comp rx=0 tx=0).stop 2023-11-28T18:30:59.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.401+0000 7f1741baa640 1 -- 172.21.15.105:0/3965292144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f173c0af340 msgr2=0x7f173c15f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:59.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.401+0000 7f1741baa640 1 --2- 172.21.15.105:0/3965292144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f173c0af340 0x7f173c15f5a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f1724000c10 tx=0x7f1724019110 comp rx=0 tx=0).stop 2023-11-28T18:30:59.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.401+0000 7f1741baa640 1 -- 172.21.15.105:0/3965292144 shutdown_connections 2023-11-28T18:30:59.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.401+0000 7f1741baa640 1 --2- 172.21.15.105:0/3965292144 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f173c0d08b0 0x7f173c143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.401+0000 7f1741baa640 1 --2- 172.21.15.105:0/3965292144 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f171c0668d0 0x7f171c068d90 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.401+0000 7f1741baa640 1 --2- 172.21.15.105:0/3965292144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f173c0af340 0x7f173c15f5a0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.401+0000 7f1741baa640 1 --2- 172.21.15.105:0/3965292144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f173c06ed00 0x7f173c15aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.401+0000 7f1741baa640 1 -- 172.21.15.105:0/3965292144 >> 172.21.15.105:0/3965292144 conn(0x7f173c06b860 msgr2=0x7f173c0b4930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:59.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.401+0000 7f1741baa640 1 -- 172.21.15.105:0/3965292144 shutdown_connections 2023-11-28T18:30:59.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.401+0000 7f1741baa640 1 -- 172.21.15.105:0/3965292144 wait complete. 2023-11-28T18:30:59.422 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:30:59.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.561+0000 7f3679017640 1 Processor -- start 2023-11-28T18:30:59.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.561+0000 7f3679017640 1 -- start start 2023-11-28T18:30:59.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.561+0000 7f3679017640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36740d1b00 0x7f36740c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:59.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.561+0000 7f3679017640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36740d0c60 con 0x7f36740d1650 2023-11-28T18:30:59.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.561+0000 7f3679017640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36740d1f90 con 0x7f367415d300 2023-11-28T18:30:59.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.561+0000 7f3679017640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36740d1470 con 0x7f36740d1b00 2023-11-28T18:30:59.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.561+0000 7f366a59c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f36740d1650 0x7f36740c7510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42254/0 (socket says 172.21.15.105:42254) 2023-11-28T18:30:59.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.561+0000 7f366a59c640 1 -- 172.21.15.105:0/3781130494 learned_addr learned my addr 172.21.15.105:0/3781130494 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:30:59.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.561+0000 7f367359e640 1 --2- 172.21.15.105:0/3781130494 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36740d1b00 0x7f36740c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:59.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.561+0000 7f367259c640 1 -- 172.21.15.105:0/3781130494 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 838893450 0 0) 0x7f36740d0c60 con 0x7f36740d1650 2023-11-28T18:30:59.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.561+0000 7f367259c640 1 -- 172.21.15.105:0/3781130494 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3660003530 con 0x7f36740d1650 2023-11-28T18:30:59.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.561+0000 7f367259c640 1 -- 172.21.15.105:0/3781130494 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3069536225 0 0) 0x7f36740d1f90 con 0x7f367415d300 2023-11-28T18:30:59.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.561+0000 7f367259c640 1 -- 172.21.15.105:0/3781130494 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36740d0c60 con 0x7f367415d300 2023-11-28T18:30:59.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.561+0000 7f367359e640 1 -- 172.21.15.105:0/3781130494 >> v1:172.21.15.148:6789/0 conn(0x7f367415d300 legacy=0x7f36740d37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:59.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.561+0000 7f367359e640 1 -- 172.21.15.105:0/3781130494 >> v1:172.21.15.105:6789/0 conn(0x7f36740d1650 legacy=0x7f36740c7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:59.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.561+0000 7f367259c640 1 -- 172.21.15.105:0/3781130494 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1701029422 0 0) 0x7f3660003530 con 0x7f36740d1650 2023-11-28T18:30:59.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.561+0000 7f367359e640 1 -- 172.21.15.105:0/3781130494 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36740d0eb0 con 0x7f36740d1b00 2023-11-28T18:30:59.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.565+0000 7f367359e640 1 --2- 172.21.15.105:0/3781130494 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36740d1b00 0x7f36740c7c20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f366c009dc0 tx=0x7f366c01c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bc1b30ae49e295e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:59.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.565+0000 7f367259c640 1 -- 172.21.15.105:0/3781130494 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c001680 con 0x7f36740d1b00 2023-11-28T18:30:59.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.565+0000 7f367259c640 1 -- 172.21.15.105:0/3781130494 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366c0a5530 con 0x7f36740d1b00 2023-11-28T18:30:59.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.565+0000 7f367259c640 1 -- 172.21.15.105:0/3781130494 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c0a58c0 con 0x7f36740d1b00 2023-11-28T18:30:59.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.565+0000 7f3679017640 1 -- 172.21.15.105:0/3781130494 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36740d1b00 msgr2=0x7f36740c7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:59.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.565+0000 7f3679017640 1 --2- 172.21.15.105:0/3781130494 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36740d1b00 0x7f36740c7c20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f366c009dc0 tx=0x7f366c01c800 comp rx=0 tx=0).stop 2023-11-28T18:30:59.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.565+0000 7f3679017640 1 -- 172.21.15.105:0/3781130494 shutdown_connections 2023-11-28T18:30:59.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.565+0000 7f3679017640 1 --2- 172.21.15.105:0/3781130494 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36740d1b00 0x7f36740c7c20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.565+0000 7f3679017640 1 -- 172.21.15.105:0/3781130494 >> 172.21.15.105:0/3781130494 conn(0x7f367409c360 msgr2=0x7f36740d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:59.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.565+0000 7f3679017640 1 -- 172.21.15.105:0/3781130494 shutdown_connections 2023-11-28T18:30:59.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.565+0000 7f3679017640 1 -- 172.21.15.105:0/3781130494 wait complete. 2023-11-28T18:30:59.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f3679017640 1 Processor -- start 2023-11-28T18:30:59.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f3679017640 1 -- start start 2023-11-28T18:30:59.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f3679017640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36740d1650 0x7f367415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:59.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f3679017640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36740d1b00 0x7f367415f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:59.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f3679017640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f367415d300 0x7f36741502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:59.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f3679017640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367415f990 con 0x7f36740d1b00 2023-11-28T18:30:59.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f3679017640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367415d990 con 0x7f367415d300 2023-11-28T18:30:59.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f3679017640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f367415dc70 con 0x7f36740d1650 2023-11-28T18:30:59.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f366a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36740d1b00 0x7f367415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:59.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f366a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36740d1b00 0x7f367415f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59932/0 (socket says 172.21.15.105:59932) 2023-11-28T18:30:59.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f366a59c640 1 -- 172.21.15.105:0/3439969537 learned_addr learned my addr 172.21.15.105:0/3439969537 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:59.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f3672d9d640 1 --2- 172.21.15.105:0/3439969537 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36740d1650 0x7f367415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:59.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f367359e640 1 --2- 172.21.15.105:0/3439969537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f367415d300 0x7f36741502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:59.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f366a59c640 1 -- 172.21.15.105:0/3439969537 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36740d1650 msgr2=0x7f367415beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:59.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f366a59c640 1 --2- 172.21.15.105:0/3439969537 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36740d1650 0x7f367415beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f366a59c640 1 -- 172.21.15.105:0/3439969537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f367415d300 msgr2=0x7f36741502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:59.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f366a59c640 1 --2- 172.21.15.105:0/3439969537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f367415d300 0x7f36741502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f366a59c640 1 -- 172.21.15.105:0/3439969537 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366c009a70 con 0x7f36740d1b00 2023-11-28T18:30:59.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f366a59c640 1 --2- 172.21.15.105:0/3439969537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36740d1b00 0x7f367415f450 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f365c00c780 tx=0x7f365c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:59.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f366bfff640 1 -- 172.21.15.105:0/3439969537 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c005ae0 con 0x7f36740d1b00 2023-11-28T18:30:59.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f3679017640 1 -- 172.21.15.105:0/3439969537 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f367415df50 con 0x7f36740d1b00 2023-11-28T18:30:59.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f366bfff640 1 -- 172.21.15.105:0/3439969537 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365c005c40 con 0x7f36740d1b00 2023-11-28T18:30:59.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f366bfff640 1 -- 172.21.15.105:0/3439969537 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c007070 con 0x7f36740d1b00 2023-11-28T18:30:59.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.569+0000 7f3679017640 1 -- 172.21.15.105:0/3439969537 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f367415e510 con 0x7f36740d1b00 2023-11-28T18:30:59.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.573+0000 7f3679017640 1 -- 172.21.15.105:0/3439969537 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f367415e700 con 0x7f36740d1b00 2023-11-28T18:30:59.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.573+0000 7f366bfff640 1 -- 172.21.15.105:0/3439969537 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f365c0071f0 con 0x7f36740d1b00 2023-11-28T18:30:59.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.573+0000 7f366bfff640 1 --2- 172.21.15.105:0/3439969537 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f36540668d0 0x7f3654068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:59.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.573+0000 7f366bfff640 1 -- 172.21.15.105:0/3439969537 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f365c083930 con 0x7f36740d1b00 2023-11-28T18:30:59.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.573+0000 7f3672d9d640 1 --2- 172.21.15.105:0/3439969537 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f36540668d0 0x7f3654068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:59.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.573+0000 7f3672d9d640 1 --2- 172.21.15.105:0/3439969537 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f36540668d0 0x7f3654068d90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f36640011f0 tx=0x7f3664009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.577+0000 7f366bfff640 1 -- 172.21.15.105:0/3439969537 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f365c050da0 con 0x7f36740d1b00 2023-11-28T18:30:59.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.733+0000 7f3679017640 1 -- 172.21.15.105:0/3439969537 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3674067570 con 0x7f36740d1b00 2023-11-28T18:30:59.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.733+0000 7f366bfff640 1 -- 172.21.15.105:0/3439969537 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+13132 (secure 0 0 0) 0x7f365c017090 con 0x7f36740d1b00 2023-11-28T18:30:59.733 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:30:59.733 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":113,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:30:59.010309+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:30:59.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.733+0000 7f3679017640 1 -- 172.21.15.105:0/3439969537 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f36540668d0 msgr2=0x7f3654068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:59.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.733+0000 7f3679017640 1 --2- 172.21.15.105:0/3439969537 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f36540668d0 0x7f3654068d90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f36640011f0 tx=0x7f3664009040 comp rx=0 tx=0).stop 2023-11-28T18:30:59.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.733+0000 7f3679017640 1 -- 172.21.15.105:0/3439969537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36740d1b00 msgr2=0x7f367415f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:59.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.733+0000 7f3679017640 1 --2- 172.21.15.105:0/3439969537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36740d1b00 0x7f367415f450 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f365c00c780 tx=0x7f365c00cc40 comp rx=0 tx=0).stop 2023-11-28T18:30:59.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.733+0000 7f3679017640 1 -- 172.21.15.105:0/3439969537 shutdown_connections 2023-11-28T18:30:59.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.733+0000 7f3679017640 1 --2- 172.21.15.105:0/3439969537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f367415d300 0x7f36741502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.733+0000 7f3679017640 1 --2- 172.21.15.105:0/3439969537 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f36540668d0 0x7f3654068d90 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.733+0000 7f3679017640 1 --2- 172.21.15.105:0/3439969537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36740d1b00 0x7f367415f450 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.733+0000 7f3679017640 1 --2- 172.21.15.105:0/3439969537 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36740d1650 0x7f367415beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.733+0000 7f3679017640 1 -- 172.21.15.105:0/3439969537 >> 172.21.15.105:0/3439969537 conn(0x7f367409c360 msgr2=0x7f367406b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:59.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.733+0000 7f3679017640 1 -- 172.21.15.105:0/3439969537 shutdown_connections 2023-11-28T18:30:59.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.733+0000 7f3679017640 1 -- 172.21.15.105:0/3439969537 wait complete. 2023-11-28T18:30:59.754 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:30:59.755 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:30:59.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.897+0000 7f327f59d640 1 Processor -- start 2023-11-28T18:30:59.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.897+0000 7f327f59d640 1 -- start start 2023-11-28T18:30:59.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.897+0000 7f327f59d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f327806d610 0x7f32780cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:59.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.897+0000 7f327f59d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f32780ae100 0x7f32780cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:59.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.897+0000 7f327f59d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3278162820 con 0x7f32780ce530 2023-11-28T18:30:59.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.897+0000 7f327f59d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3278161c10 con 0x7f32780ae100 2023-11-28T18:30:59.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.897+0000 7f327f59d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3278161ef0 con 0x7f327806d610 2023-11-28T18:30:59.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.897+0000 7f327db3a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f32780ae100 0x7f32780cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:59.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.897+0000 7f327db3a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f32780ae100 0x7f32780cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59934/0 (socket says 172.21.15.105:59934) 2023-11-28T18:30:59.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.897+0000 7f327db3a640 1 -- 172.21.15.105:0/1432516671 learned_addr learned my addr 172.21.15.105:0/1432516671 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:59.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.897+0000 7f327cb38640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f32780ce530 0x7f32780ad9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42264/0 (socket says 172.21.15.105:42264) 2023-11-28T18:30:59.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.897+0000 7f327d339640 1 --2- 172.21.15.105:0/1432516671 >> v2:172.21.15.148:3300/0 conn(0x7f327806d610 0x7f32780cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:59.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.897+0000 7f327db3a640 1 -- 172.21.15.105:0/1432516671 >> v2:172.21.15.148:3300/0 conn(0x7f327806d610 msgr2=0x7f32780cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:30:59.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.897+0000 7f327db3a640 1 --2- 172.21.15.105:0/1432516671 >> v2:172.21.15.148:3300/0 conn(0x7f327806d610 0x7f32780cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.897+0000 7f327db3a640 1 -- 172.21.15.105:0/1432516671 >> v1:172.21.15.105:6789/0 conn(0x7f32780ce530 legacy=0x7f32780ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:59.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.897+0000 7f327db3a640 1 -- 172.21.15.105:0/1432516671 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32781621d0 con 0x7f32780ae100 2023-11-28T18:30:59.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.897+0000 7f326ffff640 1 -- 172.21.15.105:0/1432516671 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146851978 0 0) 0x7f3278162820 con 0x7f32780ce530 2023-11-28T18:30:59.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.901+0000 7f327db3a640 1 --2- 172.21.15.105:0/1432516671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32780ae100 0x7f32780cc7d0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f327806fa90 tx=0x7f327001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51a9c5c77b74f32 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:59.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.901+0000 7f326ffff640 1 -- 172.21.15.105:0/1432516671 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32700a5880 con 0x7f32780ae100 2023-11-28T18:30:59.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.901+0000 7f326ffff640 1 -- 172.21.15.105:0/1432516671 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32700a59e0 con 0x7f32780ae100 2023-11-28T18:30:59.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.901+0000 7f326ffff640 1 -- 172.21.15.105:0/1432516671 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32700a5d70 con 0x7f32780ae100 2023-11-28T18:30:59.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.901+0000 7f327f59d640 1 -- 172.21.15.105:0/1432516671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32780ae100 msgr2=0x7f32780cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:59.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.901+0000 7f327f59d640 1 --2- 172.21.15.105:0/1432516671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32780ae100 0x7f32780cc7d0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f327806fa90 tx=0x7f327001c9a0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.901+0000 7f327f59d640 1 -- 172.21.15.105:0/1432516671 shutdown_connections 2023-11-28T18:30:59.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.901+0000 7f327f59d640 1 --2- 172.21.15.105:0/1432516671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32780ae100 0x7f32780cc7d0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.901+0000 7f327f59d640 1 --2- 172.21.15.105:0/1432516671 >> v2:172.21.15.148:3300/0 conn(0x7f327806d610 0x7f32780cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.901+0000 7f327f59d640 1 -- 172.21.15.105:0/1432516671 >> 172.21.15.105:0/1432516671 conn(0x7f32780ceb90 msgr2=0x7f32780cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:30:59.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.901+0000 7f327f59d640 1 -- 172.21.15.105:0/1432516671 shutdown_connections 2023-11-28T18:30:59.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.901+0000 7f327f59d640 1 -- 172.21.15.105:0/1432516671 wait complete. 2023-11-28T18:30:59.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f327f59d640 1 Processor -- start 2023-11-28T18:30:59.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f327f59d640 1 -- start start 2023-11-28T18:30:59.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f327f59d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f327806d610 0x7f327814ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:59.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f327f59d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32780ae100 0x7f3278165240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:59.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f327f59d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32780ce530 0x7f3278165780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:59.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f327f59d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327815e370 con 0x7f32780ae100 2023-11-28T18:30:59.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f327f59d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3278162c00 con 0x7f32780ce530 2023-11-28T18:30:59.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f327f59d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3278162ee0 con 0x7f327806d610 2023-11-28T18:30:59.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f327cb38640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32780ae100 0x7f3278165240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:59.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f327cb38640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32780ae100 0x7f3278165240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59936/0 (socket says 172.21.15.105:59936) 2023-11-28T18:30:59.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f327cb38640 1 -- 172.21.15.105:0/4166684165 learned_addr learned my addr 172.21.15.105:0/4166684165 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:30:59.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f327d339640 1 --2- 172.21.15.105:0/4166684165 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f327806d610 0x7f327814ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:59.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f327db3a640 1 --2- 172.21.15.105:0/4166684165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32780ce530 0x7f3278165780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:59.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f327cb38640 1 -- 172.21.15.105:0/4166684165 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f327806d610 msgr2=0x7f327814ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:59.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f327cb38640 1 --2- 172.21.15.105:0/4166684165 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f327806d610 0x7f327814ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f327cb38640 1 -- 172.21.15.105:0/4166684165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32780ce530 msgr2=0x7f3278165780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:30:59.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f327cb38640 1 --2- 172.21.15.105:0/4166684165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32780ce530 0x7f3278165780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:30:59.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f327cb38640 1 -- 172.21.15.105:0/4166684165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3270009b30 con 0x7f32780ae100 2023-11-28T18:30:59.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f327cb38640 1 --2- 172.21.15.105:0/4166684165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32780ae100 0x7f3278165240 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f326800c520 tx=0x7f326800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:59.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f326dffb640 1 -- 172.21.15.105:0/4166684165 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32680074e0 con 0x7f32780ae100 2023-11-28T18:30:59.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f326dffb640 1 -- 172.21.15.105:0/4166684165 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3268007640 con 0x7f32780ae100 2023-11-28T18:30:59.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f327f59d640 1 -- 172.21.15.105:0/4166684165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327809a6a0 con 0x7f32780ae100 2023-11-28T18:30:59.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f326dffb640 1 -- 172.21.15.105:0/4166684165 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32680049f0 con 0x7f32780ae100 2023-11-28T18:30:59.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.905+0000 7f327f59d640 1 -- 172.21.15.105:0/4166684165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327809ac60 con 0x7f32780ae100 2023-11-28T18:30:59.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.909+0000 7f327f59d640 1 -- 172.21.15.105:0/4166684165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3278067570 con 0x7f32780ae100 2023-11-28T18:30:59.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.909+0000 7f326dffb640 1 -- 172.21.15.105:0/4166684165 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3268004b50 con 0x7f32780ae100 2023-11-28T18:30:59.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.909+0000 7f326dffb640 1 --2- 172.21.15.105:0/4166684165 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f325c066900 0x7f325c068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:30:59.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.909+0000 7f326dffb640 1 -- 172.21.15.105:0/4166684165 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f3268083380 con 0x7f32780ae100 2023-11-28T18:30:59.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.909+0000 7f327d339640 1 --2- 172.21.15.105:0/4166684165 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f325c066900 0x7f325c068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:30:59.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.909+0000 7f327d339640 1 --2- 172.21.15.105:0/4166684165 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f325c066900 0x7f325c068dc0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f32781651d0 tx=0x7f3274008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:30:59.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:30:59.913+0000 7f326dffb640 1 -- 172.21.15.105:0/4166684165 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3268050870 con 0x7f32780ae100 2023-11-28T18:31:00.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.065+0000 7f327f59d640 1 -- 172.21.15.105:0/4166684165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f32780cf950 con 0x7f32780ae100 2023-11-28T18:31:00.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.065+0000 7f326dffb640 1 -- 172.21.15.105:0/4166684165 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f32680180a0 con 0x7f32780ae100 2023-11-28T18:31:00.065 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:31:00.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.069+0000 7f327f59d640 1 -- 172.21.15.105:0/4166684165 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f325c066900 msgr2=0x7f325c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:00.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.069+0000 7f327f59d640 1 --2- 172.21.15.105:0/4166684165 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f325c066900 0x7f325c068dc0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f32781651d0 tx=0x7f3274008040 comp rx=0 tx=0).stop 2023-11-28T18:31:00.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.069+0000 7f327f59d640 1 -- 172.21.15.105:0/4166684165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32780ae100 msgr2=0x7f3278165240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:00.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.069+0000 7f327f59d640 1 --2- 172.21.15.105:0/4166684165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32780ae100 0x7f3278165240 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f326800c520 tx=0x7f326800c9e0 comp rx=0 tx=0).stop 2023-11-28T18:31:00.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.069+0000 7f327f59d640 1 -- 172.21.15.105:0/4166684165 shutdown_connections 2023-11-28T18:31:00.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.069+0000 7f327f59d640 1 --2- 172.21.15.105:0/4166684165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32780ce530 0x7f3278165780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:00.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.069+0000 7f327f59d640 1 --2- 172.21.15.105:0/4166684165 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f325c066900 0x7f325c068dc0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:00.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.069+0000 7f327f59d640 1 --2- 172.21.15.105:0/4166684165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32780ae100 0x7f3278165240 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:00.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.069+0000 7f327f59d640 1 --2- 172.21.15.105:0/4166684165 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f327806d610 0x7f327814ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:00.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.069+0000 7f327f59d640 1 -- 172.21.15.105:0/4166684165 >> 172.21.15.105:0/4166684165 conn(0x7f32780ceb90 msgr2=0x7f32780cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:00.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.069+0000 7f327f59d640 1 -- 172.21.15.105:0/4166684165 shutdown_connections 2023-11-28T18:31:00.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.069+0000 7f327f59d640 1 -- 172.21.15.105:0/4166684165 wait complete. 2023-11-28T18:31:00.094 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T18:31:00.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.229+0000 7f534192a640 1 Processor -- start 2023-11-28T18:31:00.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.229+0000 7f534192a640 1 -- start start 2023-11-28T18:31:00.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f534192a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f533c0ce2c0 0x7f533c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:00.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f534192a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f533c0cde30 con 0x7f533c159610 2023-11-28T18:31:00.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f534192a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f533c1599f0 con 0x7f533c06cd20 2023-11-28T18:31:00.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f534192a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533c06d1b0 con 0x7f533c0ce2c0 2023-11-28T18:31:00.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533affd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f533c159610 0x7f533c0cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42272/0 (socket says 172.21.15.105:42272) 2023-11-28T18:31:00.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533affd640 1 -- 172.21.15.105:0/1754439468 learned_addr learned my addr 172.21.15.105:0/1754439468 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:00.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f53327fc640 1 --2- 172.21.15.105:0/1754439468 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f533c0ce2c0 0x7f533c06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:00.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533a7fc640 1 -- 172.21.15.105:0/1754439468 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 692740701 0 0) 0x7f533c0cde30 con 0x7f533c159610 2023-11-28T18:31:00.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533a7fc640 1 -- 172.21.15.105:0/1754439468 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5328003530 con 0x7f533c159610 2023-11-28T18:31:00.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533a7fc640 1 -- 172.21.15.105:0/1754439468 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3282954677 0 0) 0x7f533c1599f0 con 0x7f533c06cd20 2023-11-28T18:31:00.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533a7fc640 1 -- 172.21.15.105:0/1754439468 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f533c0cde30 con 0x7f533c06cd20 2023-11-28T18:31:00.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533a7fc640 1 -- 172.21.15.105:0/1754439468 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2712782810 0 0) 0x7f5328003530 con 0x7f533c159610 2023-11-28T18:31:00.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533a7fc640 1 -- 172.21.15.105:0/1754439468 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f533c0ce2c0 msgr2=0x7f533c06c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:00.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533a7fc640 1 --2- 172.21.15.105:0/1754439468 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f533c0ce2c0 0x7f533c06c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:00.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533a7fc640 1 -- 172.21.15.105:0/1754439468 >> v1:172.21.15.148:6789/0 conn(0x7f533c06cd20 legacy=0x7f533c0abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:00.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533a7fc640 1 -- 172.21.15.105:0/1754439468 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533c06de20 con 0x7f533c159610 2023-11-28T18:31:00.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533a7fc640 1 -- 172.21.15.105:0/1754439468 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f532c001a00 con 0x7f533c159610 2023-11-28T18:31:00.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533a7fc640 1 -- 172.21.15.105:0/1754439468 >> v1:172.21.15.105:6789/0 conn(0x7f533c159610 legacy=0x7f533c0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:00.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533a7fc640 1 --2- 172.21.15.105:0/1754439468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328003b40 0x7f5328003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:00.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533a7fc640 1 --2- 172.21.15.105:0/1754439468 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f533c0ce2c0 0x7f53280047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:00.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533a7fc640 1 --2- 172.21.15.105:0/1754439468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5328004d10 0x7f5328005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:00.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533a7fc640 1 -- 172.21.15.105:0/1754439468 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c06d1b0 con 0x7f5328003b40 2023-11-28T18:31:00.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533a7fc640 1 -- 172.21.15.105:0/1754439468 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532800ba40 con 0x7f5328004d10 2023-11-28T18:31:00.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533a7fc640 1 -- 172.21.15.105:0/1754439468 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532800bbb0 con 0x7f533c0ce2c0 2023-11-28T18:31:00.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533affd640 1 --2- 172.21.15.105:0/1754439468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328003b40 0x7f5328003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:00.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f53327fc640 1 --2- 172.21.15.105:0/1754439468 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f533c0ce2c0 0x7f53280047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:00.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533b7fe640 1 --2- 172.21.15.105:0/1754439468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5328004d10 0x7f5328005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:00.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533affd640 1 -- 172.21.15.105:0/1754439468 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f533c0ce2c0 msgr2=0x7f53280047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:00.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533affd640 1 --2- 172.21.15.105:0/1754439468 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f533c0ce2c0 0x7f53280047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:00.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533affd640 1 -- 172.21.15.105:0/1754439468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5328004d10 msgr2=0x7f5328005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:00.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533affd640 1 --2- 172.21.15.105:0/1754439468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5328004d10 0x7f5328005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:00.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533affd640 1 -- 172.21.15.105:0/1754439468 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5328003530 con 0x7f5328003b40 2023-11-28T18:31:00.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f53327fc640 1 --2- 172.21.15.105:0/1754439468 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f533c0ce2c0 0x7f53280047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:00.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.233+0000 7f533b7fe640 1 --2- 172.21.15.105:0/1754439468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5328004d10 0x7f5328005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:00.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.237+0000 7f533affd640 1 --2- 172.21.15.105:0/1754439468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328003b40 0x7f5328003f40 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f532c00a7d0 tx=0x7f532c01d290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19956801596930d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:00.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.237+0000 7f533a7fc640 1 -- 172.21.15.105:0/1754439468 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c0072d0 con 0x7f5328003b40 2023-11-28T18:31:00.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.237+0000 7f533a7fc640 1 -- 172.21.15.105:0/1754439468 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532c0a8070 con 0x7f5328003b40 2023-11-28T18:31:00.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.237+0000 7f533a7fc640 1 -- 172.21.15.105:0/1754439468 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c007690 con 0x7f5328003b40 2023-11-28T18:31:00.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.237+0000 7f534192a640 1 -- 172.21.15.105:0/1754439468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328003b40 msgr2=0x7f5328003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:00.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.237+0000 7f534192a640 1 --2- 172.21.15.105:0/1754439468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328003b40 0x7f5328003f40 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f532c00a7d0 tx=0x7f532c01d290 comp rx=0 tx=0).stop 2023-11-28T18:31:00.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.237+0000 7f533b7fe640 1 -- 172.21.15.105:0/1754439468 reap_dead start 2023-11-28T18:31:00.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.237+0000 7f534192a640 1 -- 172.21.15.105:0/1754439468 shutdown_connections 2023-11-28T18:31:00.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.237+0000 7f534192a640 1 -- 172.21.15.105:0/1754439468 >> 172.21.15.105:0/1754439468 conn(0x7f533c0ceb90 msgr2=0x7f533c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:00.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.237+0000 7f534192a640 1 -- 172.21.15.105:0/1754439468 shutdown_connections 2023-11-28T18:31:00.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.237+0000 7f534192a640 1 -- 172.21.15.105:0/1754439468 wait complete. 2023-11-28T18:31:00.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.237+0000 7f534192a640 1 Processor -- start 2023-11-28T18:31:00.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f534192a640 1 -- start start 2023-11-28T18:31:00.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f534192a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328003b40 0x7f533c151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:00.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f534192a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f533c152f50 0x7f533c1586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:00.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f534192a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f533c1533d0 0x7f533c158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:00.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f534192a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c0b2de0 con 0x7f5328003b40 2023-11-28T18:31:00.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f534192a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c152940 con 0x7f533c1533d0 2023-11-28T18:31:00.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f534192a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533c153940 con 0x7f533c152f50 2023-11-28T18:31:00.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f533affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328003b40 0x7f533c151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:00.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f533affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328003b40 0x7f533c151e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59952/0 (socket says 172.21.15.105:59952) 2023-11-28T18:31:00.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f533affd640 1 -- 172.21.15.105:0/2999095353 learned_addr learned my addr 172.21.15.105:0/2999095353 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:00.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f533b7fe640 1 --2- 172.21.15.105:0/2999095353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f533c1533d0 0x7f533c158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:00.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f53327fc640 1 --2- 172.21.15.105:0/2999095353 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f533c152f50 0x7f533c1586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:00.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f533affd640 1 -- 172.21.15.105:0/2999095353 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f533c152f50 msgr2=0x7f533c1586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:00.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f533affd640 1 --2- 172.21.15.105:0/2999095353 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f533c152f50 0x7f533c1586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:00.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f533affd640 1 -- 172.21.15.105:0/2999095353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f533c1533d0 msgr2=0x7f533c158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:00.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f533affd640 1 --2- 172.21.15.105:0/2999095353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f533c1533d0 0x7f533c158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:00.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f533affd640 1 -- 172.21.15.105:0/2999095353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533c0ccff0 con 0x7f5328003b40 2023-11-28T18:31:00.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f533b7fe640 1 --2- 172.21.15.105:0/2999095353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f533c1533d0 0x7f533c158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:00.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f533affd640 1 --2- 172.21.15.105:0/2999095353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328003b40 0x7f533c151e30 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f532c00a7d0 tx=0x7f5328005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:00.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f5338ff9640 1 -- 172.21.15.105:0/2999095353 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c0b4070 con 0x7f5328003b40 2023-11-28T18:31:00.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f5338ff9640 1 -- 172.21.15.105:0/2999095353 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532c0018c0 con 0x7f5328003b40 2023-11-28T18:31:00.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f5338ff9640 1 -- 172.21.15.105:0/2999095353 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c004d70 con 0x7f5328003b40 2023-11-28T18:31:00.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f534192a640 1 -- 172.21.15.105:0/2999095353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f533c153c20 con 0x7f5328003b40 2023-11-28T18:31:00.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f534192a640 1 -- 172.21.15.105:0/2999095353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533c1541e0 con 0x7f5328003b40 2023-11-28T18:31:00.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.241+0000 7f534192a640 1 -- 172.21.15.105:0/2999095353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f533c0b3050 con 0x7f5328003b40 2023-11-28T18:31:00.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.245+0000 7f5338ff9640 1 -- 172.21.15.105:0/2999095353 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f532c0c1440 con 0x7f5328003b40 2023-11-28T18:31:00.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.245+0000 7f5338ff9640 1 --2- 172.21.15.105:0/2999095353 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f531c0668d0 0x7f531c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:00.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.245+0000 7f5338ff9640 1 -- 172.21.15.105:0/2999095353 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f532c11df20 con 0x7f5328003b40 2023-11-28T18:31:00.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.245+0000 7f53327fc640 1 --2- 172.21.15.105:0/2999095353 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f531c0668d0 0x7f531c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:00.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.245+0000 7f53327fc640 1 --2- 172.21.15.105:0/2999095353 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f531c0668d0 0x7f531c068d90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f533c0ccc60 tx=0x7f5324008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:00.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.249+0000 7f5338ff9640 1 -- 172.21.15.105:0/2999095353 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f532c0eb360 con 0x7f5328003b40 2023-11-28T18:31:00.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.401+0000 7f534192a640 1 -- 172.21.15.105:0/2999095353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f533c067570 con 0x7f5328003b40 2023-11-28T18:31:00.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.401+0000 7f5338ff9640 1 -- 172.21.15.105:0/2999095353 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v113) v1 ==== 84+0+65 (secure 0 0 0) 0x7f532c0ee980 con 0x7f5328003b40 2023-11-28T18:31:00.400 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:00.401 INFO:teuthology.orchestra.run.smithi105.stdout:[ 2023-11-28T18:31:00.401 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:31:00.401 INFO:teuthology.orchestra.run.smithi105.stdout: "poolnum": 1, 2023-11-28T18:31:00.401 INFO:teuthology.orchestra.run.smithi105.stdout: "poolname": ".mgr" 2023-11-28T18:31:00.401 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:00.401 INFO:teuthology.orchestra.run.smithi105.stdout:] 2023-11-28T18:31:00.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.401+0000 7f534192a640 1 -- 172.21.15.105:0/2999095353 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f531c0668d0 msgr2=0x7f531c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:00.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.401+0000 7f534192a640 1 --2- 172.21.15.105:0/2999095353 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f531c0668d0 0x7f531c068d90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f533c0ccc60 tx=0x7f5324008040 comp rx=0 tx=0).stop 2023-11-28T18:31:00.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.401+0000 7f534192a640 1 -- 172.21.15.105:0/2999095353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328003b40 msgr2=0x7f533c151e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:00.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.401+0000 7f534192a640 1 --2- 172.21.15.105:0/2999095353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328003b40 0x7f533c151e30 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f532c00a7d0 tx=0x7f5328005800 comp rx=0 tx=0).stop 2023-11-28T18:31:00.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.401+0000 7f534192a640 1 -- 172.21.15.105:0/2999095353 shutdown_connections 2023-11-28T18:31:00.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.401+0000 7f534192a640 1 --2- 172.21.15.105:0/2999095353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f533c1533d0 0x7f533c158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:00.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.401+0000 7f534192a640 1 --2- 172.21.15.105:0/2999095353 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f533c152f50 0x7f533c1586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:00.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.401+0000 7f534192a640 1 --2- 172.21.15.105:0/2999095353 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f531c0668d0 0x7f531c068d90 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:00.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.401+0000 7f534192a640 1 --2- 172.21.15.105:0/2999095353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328003b40 0x7f533c151e30 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:00.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.401+0000 7f534192a640 1 -- 172.21.15.105:0/2999095353 >> 172.21.15.105:0/2999095353 conn(0x7f533c0ceb90 msgr2=0x7f533c0ad720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:00.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.401+0000 7f534192a640 1 -- 172.21.15.105:0/2999095353 shutdown_connections 2023-11-28T18:31:00.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.401+0000 7f534192a640 1 -- 172.21.15.105:0/2999095353 wait complete. 2023-11-28T18:31:00.421 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T18:31:00.421 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T18:31:00.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.561+0000 7f92ef5aa640 1 Processor -- start 2023-11-28T18:31:00.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.561+0000 7f92ef5aa640 1 -- start start 2023-11-28T18:31:00.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.565+0000 7f92ef5aa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f92e8071960 0x7f92e815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:00.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.565+0000 7f92ef5aa640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e80701d0 con 0x7f92e815b030 2023-11-28T18:31:00.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.565+0000 7f92ef5aa640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e806df80 con 0x7f92e80b4290 2023-11-28T18:31:00.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.565+0000 7f92ef5aa640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e8071df0 con 0x7f92e8071960 2023-11-28T18:31:00.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.565+0000 7f92edb47640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f92e815b030 0x7f92e80709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42280/0 (socket says 172.21.15.105:42280) 2023-11-28T18:31:00.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.565+0000 7f92edb47640 1 -- 172.21.15.105:0/4050172610 learned_addr learned my addr 172.21.15.105:0/4050172610 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.565+0000 7f92ecb45640 1 --2- 172.21.15.105:0/4050172610 >> v2:172.21.15.148:3300/0 conn(0x7f92e8071960 0x7f92e815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.565+0000 7f92dffff640 1 -- 172.21.15.105:0/4050172610 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308864682 0 0) 0x7f92e80701d0 con 0x7f92e815b030 2023-11-28T18:31:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.565+0000 7f92dffff640 1 -- 172.21.15.105:0/4050172610 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92d4003530 con 0x7f92e815b030 2023-11-28T18:31:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.565+0000 7f92dffff640 1 -- 172.21.15.105:0/4050172610 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3510029927 0 0) 0x7f92d4003530 con 0x7f92e815b030 2023-11-28T18:31:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.565+0000 7f92ecb45640 1 -- 172.21.15.105:0/4050172610 >> v1:172.21.15.148:6789/0 conn(0x7f92e80b4290 legacy=0x7f92e80b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.565+0000 7f92ecb45640 1 -- 172.21.15.105:0/4050172610 >> v1:172.21.15.105:6789/0 conn(0x7f92e815b030 legacy=0x7f92e80709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.565+0000 7f92ecb45640 1 -- 172.21.15.105:0/4050172610 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e80720d0 con 0x7f92e8071960 2023-11-28T18:31:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.569+0000 7f92ecb45640 1 --2- 172.21.15.105:0/4050172610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92e8071960 0x7f92e815f3b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f92d80097c0 tx=0x7f92d801c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7687690e0de849df server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:00.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.569+0000 7f92dffff640 1 -- 172.21.15.105:0/4050172610 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e0001a00 con 0x7f92e8071960 2023-11-28T18:31:00.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.569+0000 7f92dffff640 1 -- 172.21.15.105:0/4050172610 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d80a7050 con 0x7f92e8071960 2023-11-28T18:31:00.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.569+0000 7f92dffff640 1 -- 172.21.15.105:0/4050172610 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d80ab600 con 0x7f92e8071960 2023-11-28T18:31:00.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.569+0000 7f92ef5aa640 1 -- 172.21.15.105:0/4050172610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92e8071960 msgr2=0x7f92e815f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.569+0000 7f92ef5aa640 1 --2- 172.21.15.105:0/4050172610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92e8071960 0x7f92e815f3b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f92d80097c0 tx=0x7f92d801c370 comp rx=0 tx=0).stop 2023-11-28T18:31:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.569+0000 7f92ef5aa640 1 -- 172.21.15.105:0/4050172610 shutdown_connections 2023-11-28T18:31:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.569+0000 7f92ef5aa640 1 --2- 172.21.15.105:0/4050172610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92e8071960 0x7f92e815f3b0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.569+0000 7f92ef5aa640 1 -- 172.21.15.105:0/4050172610 >> 172.21.15.105:0/4050172610 conn(0x7f92e80ceb90 msgr2=0x7f92e80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.569+0000 7f92ef5aa640 1 -- 172.21.15.105:0/4050172610 shutdown_connections 2023-11-28T18:31:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.569+0000 7f92ef5aa640 1 -- 172.21.15.105:0/4050172610 wait complete. 2023-11-28T18:31:00.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.569+0000 7f92ef5aa640 1 Processor -- start 2023-11-28T18:31:00.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.569+0000 7f92ef5aa640 1 -- start start 2023-11-28T18:31:00.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92ef5aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92e80b4290 0x7f92e8153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:00.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92ef5aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92e815b030 0x7f92e81648e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:00.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92ef5aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92e8156d10 0x7f92e8164e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:00.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92ef5aa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e81568f0 con 0x7f92e80b4290 2023-11-28T18:31:00.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92ef5aa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e8155fc0 con 0x7f92e8156d10 2023-11-28T18:31:00.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92ef5aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e8156230 con 0x7f92e815b030 2023-11-28T18:31:00.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92ed346640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92e80b4290 0x7f92e8153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:00.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92ed346640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92e80b4290 0x7f92e8153900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59964/0 (socket says 172.21.15.105:59964) 2023-11-28T18:31:00.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92ed346640 1 -- 172.21.15.105:0/4077512557 learned_addr learned my addr 172.21.15.105:0/4077512557 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:00.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92ecb45640 1 --2- 172.21.15.105:0/4077512557 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92e815b030 0x7f92e81648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:00.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92edb47640 1 --2- 172.21.15.105:0/4077512557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92e8156d10 0x7f92e8164e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:00.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92ed346640 1 -- 172.21.15.105:0/4077512557 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92e815b030 msgr2=0x7f92e81648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:00.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92ed346640 1 --2- 172.21.15.105:0/4077512557 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92e815b030 0x7f92e81648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:00.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92ed346640 1 -- 172.21.15.105:0/4077512557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92e8156d10 msgr2=0x7f92e8164e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:00.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92ed346640 1 --2- 172.21.15.105:0/4077512557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92e8156d10 0x7f92e8164e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:00.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92ed346640 1 -- 172.21.15.105:0/4077512557 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d8009420 con 0x7f92e80b4290 2023-11-28T18:31:00.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92edb47640 1 --2- 172.21.15.105:0/4077512557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92e8156d10 0x7f92e8164e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:31:00.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92ed346640 1 --2- 172.21.15.105:0/4077512557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92e80b4290 0x7f92e8153900 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f92e400c530 tx=0x7f92e400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:00.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92ddffb640 1 -- 172.21.15.105:0/4077512557 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4005220 con 0x7f92e80b4290 2023-11-28T18:31:00.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92ddffb640 1 -- 172.21.15.105:0/4077512557 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e4005380 con 0x7f92e80b4290 2023-11-28T18:31:00.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92ef5aa640 1 -- 172.21.15.105:0/4077512557 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e8156510 con 0x7f92e80b4290 2023-11-28T18:31:00.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92ddffb640 1 -- 172.21.15.105:0/4077512557 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4007270 con 0x7f92e80b4290 2023-11-28T18:31:00.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92ef5aa640 1 -- 172.21.15.105:0/4077512557 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e8153e40 con 0x7f92e80b4290 2023-11-28T18:31:00.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.573+0000 7f92ef5aa640 1 -- 172.21.15.105:0/4077512557 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e80cbbd0 con 0x7f92e80b4290 2023-11-28T18:31:00.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.577+0000 7f92ddffb640 1 -- 172.21.15.105:0/4077512557 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f92e40073d0 con 0x7f92e80b4290 2023-11-28T18:31:00.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.577+0000 7f92ddffb640 1 --2- 172.21.15.105:0/4077512557 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f92cc0668d0 0x7f92cc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:00.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.577+0000 7f92ddffb640 1 -- 172.21.15.105:0/4077512557 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f92e4083b40 con 0x7f92e80b4290 2023-11-28T18:31:00.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.577+0000 7f92ecb45640 1 --2- 172.21.15.105:0/4077512557 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f92cc0668d0 0x7f92cc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:00.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.577+0000 7f92ecb45640 1 --2- 172.21.15.105:0/4077512557 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f92cc0668d0 0x7f92cc068d90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f92d80ad000 tx=0x7f92d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:00.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.581+0000 7f92ddffb640 1 -- 172.21.15.105:0/4077512557 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f92e4050f80 con 0x7f92e80b4290 2023-11-28T18:31:00.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:00.733+0000 7f92ef5aa640 1 -- 172.21.15.105:0/4077512557 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f92e8156b60 con 0x7f92e80b4290 2023-11-28T18:31:01.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.665+0000 7f92ddffb640 1 -- 172.21.15.105:0/4077512557 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v114) v1 ==== 138+0+0 (secure 0 0 0) 0x7f92e40545a0 con 0x7f92e80b4290 2023-11-28T18:31:01.664 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' created 2023-11-28T18:31:01.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.665+0000 7f92ef5aa640 1 -- 172.21.15.105:0/4077512557 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f92cc0668d0 msgr2=0x7f92cc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:01.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.665+0000 7f92ef5aa640 1 --2- 172.21.15.105:0/4077512557 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f92cc0668d0 0x7f92cc068d90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f92d80ad000 tx=0x7f92d8008040 comp rx=0 tx=0).stop 2023-11-28T18:31:01.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.665+0000 7f92ef5aa640 1 -- 172.21.15.105:0/4077512557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92e80b4290 msgr2=0x7f92e8153900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:01.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.665+0000 7f92ef5aa640 1 --2- 172.21.15.105:0/4077512557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92e80b4290 0x7f92e8153900 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f92e400c530 tx=0x7f92e400c9f0 comp rx=0 tx=0).stop 2023-11-28T18:31:01.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.665+0000 7f92ef5aa640 1 -- 172.21.15.105:0/4077512557 shutdown_connections 2023-11-28T18:31:01.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.665+0000 7f92ef5aa640 1 --2- 172.21.15.105:0/4077512557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92e8156d10 0x7f92e8164e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:01.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.665+0000 7f92ef5aa640 1 --2- 172.21.15.105:0/4077512557 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92e815b030 0x7f92e81648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:01.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.665+0000 7f92ef5aa640 1 --2- 172.21.15.105:0/4077512557 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f92cc0668d0 0x7f92cc068d90 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:01.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.665+0000 7f92ef5aa640 1 --2- 172.21.15.105:0/4077512557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92e80b4290 0x7f92e8153900 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:01.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.665+0000 7f92ef5aa640 1 -- 172.21.15.105:0/4077512557 >> 172.21.15.105:0/4077512557 conn(0x7f92e80ceb90 msgr2=0x7f92e815f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:01.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.669+0000 7f92ef5aa640 1 -- 172.21.15.105:0/4077512557 shutdown_connections 2023-11-28T18:31:01.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.669+0000 7f92ef5aa640 1 -- 172.21.15.105:0/4077512557 wait complete. 2023-11-28T18:31:01.695 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T18:31:01.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.845+0000 7f64c85c4640 1 Processor -- start 2023-11-28T18:31:01.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.845+0000 7f64c85c4640 1 -- start start 2023-11-28T18:31:01.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.845+0000 7f64c85c4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f64c0071960 0x7f64c015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:01.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.845+0000 7f64c85c4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f64c015b030 0x7f64c00709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:01.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.845+0000 7f64c85c4640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c00701d0 con 0x7f64c00b4290 2023-11-28T18:31:01.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.845+0000 7f64c85c4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c006df80 con 0x7f64c015b030 2023-11-28T18:31:01.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.845+0000 7f64c85c4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c0071df0 con 0x7f64c0071960 2023-11-28T18:31:01.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.845+0000 7f64c6b61640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f64c015b030 0x7f64c00709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:01.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.845+0000 7f64c6360640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f64c00b4290 0x7f64c00b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59546/0 (socket says 172.21.15.105:59546) 2023-11-28T18:31:01.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.845+0000 7f64c6360640 1 -- 172.21.15.105:0/1573271957 learned_addr learned my addr 172.21.15.105:0/1573271957 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:01.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.845+0000 7f64c5b5f640 1 --2- 172.21.15.105:0/1573271957 >> v2:172.21.15.148:3300/0 conn(0x7f64c0071960 0x7f64c015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:01.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.845+0000 7f64c6b61640 1 -- 172.21.15.105:0/1573271957 >> v2:172.21.15.148:3300/0 conn(0x7f64c0071960 msgr2=0x7f64c015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:01.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.845+0000 7f64c6b61640 1 --2- 172.21.15.105:0/1573271957 >> v2:172.21.15.148:3300/0 conn(0x7f64c0071960 0x7f64c015f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:01.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.845+0000 7f64c6b61640 1 -- 172.21.15.105:0/1573271957 >> v1:172.21.15.105:6789/0 conn(0x7f64c00b4290 legacy=0x7f64c00b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:01.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.845+0000 7f64c6b61640 1 -- 172.21.15.105:0/1573271957 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c00720d0 con 0x7f64c015b030 2023-11-28T18:31:01.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.849+0000 7f64c6b61640 1 --2- 172.21.15.105:0/1573271957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64c015b030 0x7f64c00709b0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f64c006e520 tx=0x7f64b801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85d8d7e0a7620e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:01.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.849+0000 7f64c535e640 1 -- 172.21.15.105:0/1573271957 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b80a58b0 con 0x7f64c015b030 2023-11-28T18:31:01.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.849+0000 7f64c535e640 1 -- 172.21.15.105:0/1573271957 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b80a5a10 con 0x7f64c015b030 2023-11-28T18:31:01.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.849+0000 7f64c535e640 1 -- 172.21.15.105:0/1573271957 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b80a5da0 con 0x7f64c015b030 2023-11-28T18:31:01.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.849+0000 7f64c85c4640 1 -- 172.21.15.105:0/1573271957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64c015b030 msgr2=0x7f64c00709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:01.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.849+0000 7f64c85c4640 1 --2- 172.21.15.105:0/1573271957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64c015b030 0x7f64c00709b0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f64c006e520 tx=0x7f64b801c8e0 comp rx=0 tx=0).stop 2023-11-28T18:31:01.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.849+0000 7f64c85c4640 1 -- 172.21.15.105:0/1573271957 shutdown_connections 2023-11-28T18:31:01.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.849+0000 7f64c85c4640 1 --2- 172.21.15.105:0/1573271957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64c015b030 0x7f64c00709b0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:01.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.849+0000 7f64c85c4640 1 --2- 172.21.15.105:0/1573271957 >> v2:172.21.15.148:3300/0 conn(0x7f64c0071960 0x7f64c015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:01.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.849+0000 7f64c85c4640 1 -- 172.21.15.105:0/1573271957 >> 172.21.15.105:0/1573271957 conn(0x7f64c00ceb90 msgr2=0x7f64c00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:01.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.849+0000 7f64c85c4640 1 -- 172.21.15.105:0/1573271957 shutdown_connections 2023-11-28T18:31:01.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.849+0000 7f64c85c4640 1 -- 172.21.15.105:0/1573271957 wait complete. 2023-11-28T18:31:01.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.849+0000 7f64c85c4640 1 Processor -- start 2023-11-28T18:31:01.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64c85c4640 1 -- start start 2023-11-28T18:31:01.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64c85c4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64c00b4290 0x7f64c0154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:01.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64c85c4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64c015b030 0x7f64c009b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:01.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64c85c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64c009c580 0x7f64c009bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:01.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64c85c4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c014bb00 con 0x7f64c015b030 2023-11-28T18:31:01.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64c85c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c0152790 con 0x7f64c009c580 2023-11-28T18:31:01.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64c85c4640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c014bcf0 con 0x7f64c00b4290 2023-11-28T18:31:01.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64c5b5f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64c015b030 0x7f64c009b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:01.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64c5b5f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64c015b030 0x7f64c009b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38162/0 (socket says 172.21.15.105:38162) 2023-11-28T18:31:01.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64c5b5f640 1 -- 172.21.15.105:0/1653961330 learned_addr learned my addr 172.21.15.105:0/1653961330 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:01.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64c5b5f640 1 -- 172.21.15.105:0/1653961330 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64c00b4290 msgr2=0x7f64c0154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:01.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64c5b5f640 1 --2- 172.21.15.105:0/1653961330 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64c00b4290 0x7f64c0154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:01.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64c6b61640 1 --2- 172.21.15.105:0/1653961330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64c009c580 0x7f64c009bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:01.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64c5b5f640 1 -- 172.21.15.105:0/1653961330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64c009c580 msgr2=0x7f64c009bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:01.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64c5b5f640 1 --2- 172.21.15.105:0/1653961330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64c009c580 0x7f64c009bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:01.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64c5b5f640 1 -- 172.21.15.105:0/1653961330 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b8009a70 con 0x7f64c015b030 2023-11-28T18:31:01.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64c6b61640 1 --2- 172.21.15.105:0/1653961330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64c009c580 0x7f64c009bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:31:01.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64c5b5f640 1 --2- 172.21.15.105:0/1653961330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64c015b030 0x7f64c009b6d0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f64b000d800 tx=0x7f64b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:01.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64b6ffd640 1 -- 172.21.15.105:0/1653961330 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b000b6c0 con 0x7f64c015b030 2023-11-28T18:31:01.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64b6ffd640 1 -- 172.21.15.105:0/1653961330 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b000b820 con 0x7f64c015b030 2023-11-28T18:31:01.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64b6ffd640 1 -- 172.21.15.105:0/1653961330 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b000bb90 con 0x7f64c015b030 2023-11-28T18:31:01.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64c85c4640 1 -- 172.21.15.105:0/1653961330 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c014bfd0 con 0x7f64c015b030 2023-11-28T18:31:01.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64c85c4640 1 -- 172.21.15.105:0/1653961330 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c01529e0 con 0x7f64c015b030 2023-11-28T18:31:01.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.853+0000 7f64c85c4640 1 -- 172.21.15.105:0/1653961330 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c014c4f0 con 0x7f64c015b030 2023-11-28T18:31:01.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.857+0000 7f64b6ffd640 1 -- 172.21.15.105:0/1653961330 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f64b000bcf0 con 0x7f64c015b030 2023-11-28T18:31:01.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.857+0000 7f64b6ffd640 1 --2- 172.21.15.105:0/1653961330 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f64a4066900 0x7f64a4068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:01.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.857+0000 7f64b6ffd640 1 -- 172.21.15.105:0/1653961330 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f64b00848c0 con 0x7f64c015b030 2023-11-28T18:31:01.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.857+0000 7f64c6360640 1 --2- 172.21.15.105:0/1653961330 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f64a4066900 0x7f64a4068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:01.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.857+0000 7f64c6360640 1 --2- 172.21.15.105:0/1653961330 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f64a4066900 0x7f64a4068dc0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f64c015ece0 tx=0x7f64bc007a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:01.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:01.861+0000 7f64b6ffd640 1 -- 172.21.15.105:0/1653961330 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f64b0051ba0 con 0x7f64c015b030 2023-11-28T18:31:02.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.009+0000 7f64c85c4640 1 -- 172.21.15.105:0/1653961330 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f64c0067570 con 0x7f64c015b030 2023-11-28T18:31:02.660 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:31:02.665 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:31:02.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.673+0000 7f64b6ffd640 1 -- 172.21.15.105:0/1653961330 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v115) v1 ==== 170+0+0 (secure 0 0 0) 0x7f64b00551c0 con 0x7f64c015b030 2023-11-28T18:31:02.672 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' created 2023-11-28T18:31:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.677+0000 7f64c85c4640 1 -- 172.21.15.105:0/1653961330 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f64a4066900 msgr2=0x7f64a4068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.677+0000 7f64c85c4640 1 --2- 172.21.15.105:0/1653961330 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f64a4066900 0x7f64a4068dc0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f64c015ece0 tx=0x7f64bc007a50 comp rx=0 tx=0).stop 2023-11-28T18:31:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.677+0000 7f64c85c4640 1 -- 172.21.15.105:0/1653961330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64c015b030 msgr2=0x7f64c009b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.677+0000 7f64c85c4640 1 --2- 172.21.15.105:0/1653961330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64c015b030 0x7f64c009b6d0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f64b000d800 tx=0x7f64b000dcc0 comp rx=0 tx=0).stop 2023-11-28T18:31:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.677+0000 7f64c85c4640 1 -- 172.21.15.105:0/1653961330 shutdown_connections 2023-11-28T18:31:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.677+0000 7f64c85c4640 1 --2- 172.21.15.105:0/1653961330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64c009c580 0x7f64c009bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.677+0000 7f64c85c4640 1 --2- 172.21.15.105:0/1653961330 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f64a4066900 0x7f64a4068dc0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.677+0000 7f64c85c4640 1 --2- 172.21.15.105:0/1653961330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64c015b030 0x7f64c009b6d0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.677+0000 7f64c85c4640 1 --2- 172.21.15.105:0/1653961330 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64c00b4290 0x7f64c0154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.677+0000 7f64c85c4640 1 -- 172.21.15.105:0/1653961330 >> 172.21.15.105:0/1653961330 conn(0x7f64c00ceb90 msgr2=0x7f64c00ae590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.677+0000 7f64c85c4640 1 -- 172.21.15.105:0/1653961330 shutdown_connections 2023-11-28T18:31:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.677+0000 7f64c85c4640 1 -- 172.21.15.105:0/1653961330 wait complete. 2023-11-28T18:31:02.719 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T18:31:02.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.921+0000 7fc5d0b0f640 1 Processor -- start 2023-11-28T18:31:02.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.921+0000 7fc5d0b0f640 1 -- start start 2023-11-28T18:31:02.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.921+0000 7fc5d0b0f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc5cc159f40 0x7fc5cc0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:02.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.921+0000 7fc5d0b0f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5cc0cc760 0x7fc5cc0ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:02.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.921+0000 7fc5d0b0f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5cc06bf50 con 0x7fc5cc0ad680 2023-11-28T18:31:02.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.921+0000 7fc5d0b0f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5cc0ccbc0 con 0x7fc5cc159f40 2023-11-28T18:31:02.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.921+0000 7fc5d0b0f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc15a320 con 0x7fc5cc0cc760 2023-11-28T18:31:02.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.921+0000 7fc5c1d9b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5cc0cc760 0x7fc5cc0ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:02.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.921+0000 7fc5ca59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc5cc159f40 0x7fc5cc0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:02.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.921+0000 7fc5c1d9b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5cc0cc760 0x7fc5cc0ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:57726/0 (socket says 172.21.15.105:57726) 2023-11-28T18:31:02.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.921+0000 7fc5c1d9b640 1 -- 172.21.15.105:0/905059232 learned_addr learned my addr 172.21.15.105:0/905059232 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:02.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.921+0000 7fc5ca59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc5cc159f40 0x7fc5cc0caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:33782/0 (socket says 172.21.15.105:33782) 2023-11-28T18:31:02.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.921+0000 7fc5cad9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc5cc0ad680 0x7fc5cc15da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:44534/0 (socket says 172.21.15.105:44534) 2023-11-28T18:31:02.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.921+0000 7fc5ca59c640 1 -- 172.21.15.105:0/905059232 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5cc0cc760 msgr2=0x7fc5cc0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:02.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.921+0000 7fc5ca59c640 1 --2- 172.21.15.105:0/905059232 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5cc0cc760 0x7fc5cc0ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:02.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.921+0000 7fc5ca59c640 1 -- 172.21.15.105:0/905059232 >> v1:172.21.15.148:6789/0 conn(0x7fc5cc0ad680 legacy=0x7fc5cc15da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:02.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.921+0000 7fc5ca59c640 1 -- 172.21.15.105:0/905059232 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5cc071380 con 0x7fc5cc159f40 2023-11-28T18:31:02.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.921+0000 7fc5c1d9b640 1 --2- 172.21.15.105:0/905059232 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5cc0cc760 0x7fc5cc0ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:31:02.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.925+0000 7fc5ca59c640 1 --2- 172.21.15.105:0/905059232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5cc159f40 0x7fc5cc0caf60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009d90 tx=0x7fc5bc01c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a065b3f66888d43 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:02.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.925+0000 7fc5c9d9b640 1 -- 172.21.15.105:0/905059232 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc0a5950 con 0x7fc5cc159f40 2023-11-28T18:31:02.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.925+0000 7fc5c9d9b640 1 -- 172.21.15.105:0/905059232 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5bc0a5ab0 con 0x7fc5cc159f40 2023-11-28T18:31:02.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.925+0000 7fc5c9d9b640 1 -- 172.21.15.105:0/905059232 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc0a5e40 con 0x7fc5cc159f40 2023-11-28T18:31:02.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.925+0000 7fc5d0b0f640 1 -- 172.21.15.105:0/905059232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5cc159f40 msgr2=0x7fc5cc0caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:02.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.925+0000 7fc5d0b0f640 1 --2- 172.21.15.105:0/905059232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5cc159f40 0x7fc5cc0caf60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009d90 tx=0x7fc5bc01c890 comp rx=0 tx=0).stop 2023-11-28T18:31:02.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.925+0000 7fc5d0b0f640 1 -- 172.21.15.105:0/905059232 shutdown_connections 2023-11-28T18:31:02.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.925+0000 7fc5d0b0f640 1 --2- 172.21.15.105:0/905059232 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5cc0cc760 0x7fc5cc0ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:02.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.925+0000 7fc5d0b0f640 1 --2- 172.21.15.105:0/905059232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5cc159f40 0x7fc5cc0caf60 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:02.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.925+0000 7fc5d0b0f640 1 -- 172.21.15.105:0/905059232 >> 172.21.15.105:0/905059232 conn(0x7fc5cc0ceb90 msgr2=0x7fc5cc0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:02.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.925+0000 7fc5d0b0f640 1 -- 172.21.15.105:0/905059232 shutdown_connections 2023-11-28T18:31:02.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.925+0000 7fc5d0b0f640 1 -- 172.21.15.105:0/905059232 wait complete. 2023-11-28T18:31:02.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5d0b0f640 1 Processor -- start 2023-11-28T18:31:02.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5d0b0f640 1 -- start start 2023-11-28T18:31:02.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5d0b0f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5cc0ad680 0x7fc5cc153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:02.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5d0b0f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5cc0cc760 0x7fc5cc14ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:02.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5d0b0f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5cc159f40 0x7fc5cc158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:02.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5d0b0f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0b19b0 con 0x7fc5cc159f40 2023-11-28T18:31:02.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5d0b0f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0b48e0 con 0x7fc5cc0ad680 2023-11-28T18:31:02.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5d0b0f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc14b170 con 0x7fc5cc0cc760 2023-11-28T18:31:02.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5ca59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5cc0ad680 0x7fc5cc153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:02.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5cad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5cc159f40 0x7fc5cc158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:02.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5ca59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5cc0ad680 0x7fc5cc153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:33792/0 (socket says 172.21.15.105:33792) 2023-11-28T18:31:02.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5ca59c640 1 -- 172.21.15.105:0/2292046660 learned_addr learned my addr 172.21.15.105:0/2292046660 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:02.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5c1d9b640 1 --2- 172.21.15.105:0/2292046660 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5cc0cc760 0x7fc5cc14ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:02.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5cad9d640 1 -- 172.21.15.105:0/2292046660 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5cc0cc760 msgr2=0x7fc5cc14ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:02.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5cad9d640 1 --2- 172.21.15.105:0/2292046660 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5cc0cc760 0x7fc5cc14ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:02.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5cad9d640 1 -- 172.21.15.105:0/2292046660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5cc0ad680 msgr2=0x7fc5cc153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:02.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5cad9d640 1 --2- 172.21.15.105:0/2292046660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5cc0ad680 0x7fc5cc153550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:02.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5cad9d640 1 -- 172.21.15.105:0/2292046660 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b40094d0 con 0x7fc5cc159f40 2023-11-28T18:31:02.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5cad9d640 1 --2- 172.21.15.105:0/2292046660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5cc159f40 0x7fc5cc158a20 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400c3f0 tx=0x7fc5c400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:02.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5c37fe640 1 -- 172.21.15.105:0/2292046660 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c40074e0 con 0x7fc5cc159f40 2023-11-28T18:31:02.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5c37fe640 1 -- 172.21.15.105:0/2292046660 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c4007640 con 0x7fc5cc159f40 2023-11-28T18:31:02.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5d0b0f640 1 -- 172.21.15.105:0/2292046660 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5bc009a70 con 0x7fc5cc159f40 2023-11-28T18:31:02.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5c37fe640 1 -- 172.21.15.105:0/2292046660 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4004a10 con 0x7fc5cc159f40 2023-11-28T18:31:02.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.929+0000 7fc5d0b0f640 1 -- 172.21.15.105:0/2292046660 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5cc15a930 con 0x7fc5cc159f40 2023-11-28T18:31:02.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.933+0000 7fc5d0b0f640 1 -- 172.21.15.105:0/2292046660 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5cc067570 con 0x7fc5cc159f40 2023-11-28T18:31:02.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.933+0000 7fc5c37fe640 1 -- 172.21.15.105:0/2292046660 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc5c4004b70 con 0x7fc5cc159f40 2023-11-28T18:31:02.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.933+0000 7fc5c37fe640 1 --2- 172.21.15.105:0/2292046660 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc5ac0668d0 0x7fc5ac068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:02.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.933+0000 7fc5c37fe640 1 -- 172.21.15.105:0/2292046660 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5929+0+0 (secure 0 0 0) 0x7fc5c4083c70 con 0x7fc5cc159f40 2023-11-28T18:31:02.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.933+0000 7fc5ca59c640 1 --2- 172.21.15.105:0/2292046660 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc5ac0668d0 0x7fc5ac068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:02.934 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.933+0000 7fc5ca59c640 1 --2- 172.21.15.105:0/2292046660 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc5ac0668d0 0x7fc5ac068d90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009d90 tx=0x7fc5bc0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:02.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:02.937+0000 7fc5c37fe640 1 -- 172.21.15.105:0/2292046660 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc5c4050e70 con 0x7fc5cc159f40 2023-11-28T18:31:03.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.117+0000 7fc5d0b0f640 1 -- 172.21.15.105:0/2292046660 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc5cc14be00 con 0x7fc5cc159f40 2023-11-28T18:31:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.677+0000 7fc5c37fe640 1 -- 172.21.15.105:0/2292046660 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v115) v1 ==== 174+0+0 (secure 0 0 0) 0x7fc5c4054490 con 0x7fc5cc159f40 2023-11-28T18:31:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:new fs with metadata pool 17 and data pool 18 2023-11-28T18:31:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.681+0000 7fc5d0b0f640 1 -- 172.21.15.105:0/2292046660 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc5ac0668d0 msgr2=0x7fc5ac068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.681+0000 7fc5d0b0f640 1 --2- 172.21.15.105:0/2292046660 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc5ac0668d0 0x7fc5ac068d90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009d90 tx=0x7fc5bc0a7070 comp rx=0 tx=0).stop 2023-11-28T18:31:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.681+0000 7fc5d0b0f640 1 -- 172.21.15.105:0/2292046660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5cc159f40 msgr2=0x7fc5cc158a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.681+0000 7fc5d0b0f640 1 --2- 172.21.15.105:0/2292046660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5cc159f40 0x7fc5cc158a20 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400c3f0 tx=0x7fc5c400c8b0 comp rx=0 tx=0).stop 2023-11-28T18:31:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.681+0000 7fc5d0b0f640 1 -- 172.21.15.105:0/2292046660 shutdown_connections 2023-11-28T18:31:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.681+0000 7fc5d0b0f640 1 --2- 172.21.15.105:0/2292046660 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc5ac0668d0 0x7fc5ac068d90 secure :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009d90 tx=0x7fc5bc0a7070 comp rx=0 tx=0).stop 2023-11-28T18:31:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.681+0000 7fc5d0b0f640 1 --2- 172.21.15.105:0/2292046660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5cc159f40 0x7fc5cc158a20 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.681+0000 7fc5d0b0f640 1 --2- 172.21.15.105:0/2292046660 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5cc0cc760 0x7fc5cc14ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.681+0000 7fc5d0b0f640 1 --2- 172.21.15.105:0/2292046660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5cc0ad680 0x7fc5cc153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.681+0000 7fc5d0b0f640 1 -- 172.21.15.105:0/2292046660 >> 172.21.15.105:0/2292046660 conn(0x7fc5cc0ceb90 msgr2=0x7fc5cc0700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.681+0000 7fc5d0b0f640 1 -- 172.21.15.105:0/2292046660 shutdown_connections 2023-11-28T18:31:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.681+0000 7fc5d0b0f640 1 -- 172.21.15.105:0/2292046660 wait complete. 2023-11-28T18:31:03.707 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:31:03.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabdf0e3640 1 Processor -- start 2023-11-28T18:31:03.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabdf0e3640 1 -- start start 2023-11-28T18:31:03.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabdf0e3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fabd80d3c30 0x7fabd80d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:03.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabdf0e3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fabd80b2d20 0x7fabd80b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:03.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabdf0e3640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd80b25b0 con 0x7fabd815d4f0 2023-11-28T18:31:03.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabdf0e3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd80b2900 con 0x7fabd80d3c30 2023-11-28T18:31:03.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabdf0e3640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd80d0a20 con 0x7fabd80b2d20 2023-11-28T18:31:03.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabdce7f640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fabd815d4f0 0x7fabd80d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59556/0 (socket says 172.21.15.105:59556) 2023-11-28T18:31:03.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabdce7f640 1 -- 172.21.15.105:0/214365311 learned_addr learned my addr 172.21.15.105:0/214365311 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:03.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabcffff640 1 --2- 172.21.15.105:0/214365311 >> v2:172.21.15.148:3300/0 conn(0x7fabd80d3c30 0x7fabd80d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:03.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabdd680640 1 --2- 172.21.15.105:0/214365311 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fabd80b2d20 0x7fabd80b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:03.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabcf7fe640 1 -- 172.21.15.105:0/214365311 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3328927360 0 0) 0x7fabd80b25b0 con 0x7fabd815d4f0 2023-11-28T18:31:03.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabcf7fe640 1 -- 172.21.15.105:0/214365311 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabc0003530 con 0x7fabd815d4f0 2023-11-28T18:31:03.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabcf7fe640 1 -- 172.21.15.105:0/214365311 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2133518321 0 0) 0x7fabc0003530 con 0x7fabd815d4f0 2023-11-28T18:31:03.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabcf7fe640 1 -- 172.21.15.105:0/214365311 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fabd80b2d20 msgr2=0x7fabd80b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:03.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabcf7fe640 1 --2- 172.21.15.105:0/214365311 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fabd80b2d20 0x7fabd80b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:03.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabcf7fe640 1 -- 172.21.15.105:0/214365311 >> v2:172.21.15.148:3300/0 conn(0x7fabd80d3c30 msgr2=0x7fabd80d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:03.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabcf7fe640 1 --2- 172.21.15.105:0/214365311 >> v2:172.21.15.148:3300/0 conn(0x7fabd80d3c30 0x7fabd80d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:03.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabcf7fe640 1 -- 172.21.15.105:0/214365311 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd80cea30 con 0x7fabd815d4f0 2023-11-28T18:31:03.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabcf7fe640 1 -- 172.21.15.105:0/214365311 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fabd4001a00 con 0x7fabd815d4f0 2023-11-28T18:31:03.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabcf7fe640 1 -- 172.21.15.105:0/214365311 >> v1:172.21.15.105:6789/0 conn(0x7fabd815d4f0 legacy=0x7fabd80d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:03.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabcf7fe640 1 --2- 172.21.15.105:0/214365311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabc0003980 0x7fabc0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:03.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabcf7fe640 1 --2- 172.21.15.105:0/214365311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc00047d0 0x7fabc0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:03.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabcf7fe640 1 --2- 172.21.15.105:0/214365311 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fabd80b2d20 0x7fabc0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:03.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabcf7fe640 1 -- 172.21.15.105:0/214365311 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd80b2900 con 0x7fabc0003980 2023-11-28T18:31:03.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabcf7fe640 1 -- 172.21.15.105:0/214365311 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd80d0a20 con 0x7fabc00047d0 2023-11-28T18:31:03.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabcf7fe640 1 -- 172.21.15.105:0/214365311 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc000b9e0 con 0x7fabd80b2d20 2023-11-28T18:31:03.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabdce7f640 1 --2- 172.21.15.105:0/214365311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabc0003980 0x7fabc0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:03.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabcffff640 1 --2- 172.21.15.105:0/214365311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc00047d0 0x7fabc0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:03.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabdd680640 1 --2- 172.21.15.105:0/214365311 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fabd80b2d20 0x7fabc0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:03.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabdce7f640 1 -- 172.21.15.105:0/214365311 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fabd80b2d20 msgr2=0x7fabc0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:03.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabdce7f640 1 --2- 172.21.15.105:0/214365311 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fabd80b2d20 0x7fabc0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:03.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabdce7f640 1 -- 172.21.15.105:0/214365311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc00047d0 msgr2=0x7fabc0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:03.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.845+0000 7fabdce7f640 1 --2- 172.21.15.105:0/214365311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc00047d0 0x7fabc0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:03.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.849+0000 7fabdce7f640 1 -- 172.21.15.105:0/214365311 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc0003530 con 0x7fabc0003980 2023-11-28T18:31:03.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.849+0000 7fabdce7f640 1 --2- 172.21.15.105:0/214365311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabc0003980 0x7fabc0003da0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fabd400a7c0 tx=0x7fabd401d250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78be75635d219d5c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:03.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.849+0000 7fabcf7fe640 1 -- 172.21.15.105:0/214365311 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd40ad630 con 0x7fabc0003980 2023-11-28T18:31:03.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.849+0000 7fabcf7fe640 1 -- 172.21.15.105:0/214365311 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd40a8070 con 0x7fabc0003980 2023-11-28T18:31:03.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.849+0000 7fabcf7fe640 1 -- 172.21.15.105:0/214365311 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd40ad9f0 con 0x7fabc0003980 2023-11-28T18:31:03.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.849+0000 7fabdf0e3640 1 -- 172.21.15.105:0/214365311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabc0003980 msgr2=0x7fabc0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:03.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.849+0000 7fabdf0e3640 1 --2- 172.21.15.105:0/214365311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabc0003980 0x7fabc0003da0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fabd400a7c0 tx=0x7fabd401d250 comp rx=0 tx=0).stop 2023-11-28T18:31:03.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.849+0000 7fabdd680640 1 -- 172.21.15.105:0/214365311 reap_dead start 2023-11-28T18:31:03.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.849+0000 7fabdf0e3640 1 -- 172.21.15.105:0/214365311 shutdown_connections 2023-11-28T18:31:03.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.849+0000 7fabdf0e3640 1 -- 172.21.15.105:0/214365311 >> 172.21.15.105:0/214365311 conn(0x7fabd80d3030 msgr2=0x7fabd80d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:03.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.849+0000 7fabdf0e3640 1 -- 172.21.15.105:0/214365311 shutdown_connections 2023-11-28T18:31:03.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.849+0000 7fabdf0e3640 1 -- 172.21.15.105:0/214365311 wait complete. 2023-11-28T18:31:03.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.853+0000 7fabdf0e3640 1 Processor -- start 2023-11-28T18:31:03.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.853+0000 7fabdf0e3640 1 -- start start 2023-11-28T18:31:03.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.853+0000 7fabdf0e3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fabc0003980 0x7fabd815dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:03.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.853+0000 7fabdf0e3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabd80cbc90 0x7fabd80b8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:03.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.853+0000 7fabdf0e3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabd815f2f0 0x7fabd80b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:03.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.853+0000 7fabdf0e3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd815e020 con 0x7fabd80cbc90 2023-11-28T18:31:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.853+0000 7fabdf0e3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd815e250 con 0x7fabd815f2f0 2023-11-28T18:31:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.853+0000 7fabdf0e3640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd815e530 con 0x7fabc0003980 2023-11-28T18:31:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.853+0000 7fabcffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabd80cbc90 0x7fabd80b8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.853+0000 7fabcffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabd80cbc90 0x7fabd80b8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38190/0 (socket says 172.21.15.105:38190) 2023-11-28T18:31:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.853+0000 7fabcffff640 1 -- 172.21.15.105:0/755831724 learned_addr learned my addr 172.21.15.105:0/755831724 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.853+0000 7fabcffff640 1 -- 172.21.15.105:0/755831724 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fabc0003980 msgr2=0x7fabd815dae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T18:31:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.853+0000 7fabcffff640 1 --2- 172.21.15.105:0/755831724 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fabc0003980 0x7fabd815dae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.853+0000 7fabcffff640 1 -- 172.21.15.105:0/755831724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabd815f2f0 msgr2=0x7fabd80b0640 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T18:31:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.853+0000 7fabcffff640 1 --2- 172.21.15.105:0/755831724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabd815f2f0 0x7fabd80b0640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.853+0000 7fabcffff640 1 -- 172.21.15.105:0/755831724 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd815e810 con 0x7fabd80cbc90 2023-11-28T18:31:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.853+0000 7fabcffff640 1 --2- 172.21.15.105:0/755831724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabd80cbc90 0x7fabd80b8080 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fabc80119d0 tx=0x7fabc8011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.853+0000 7fabcd7fa640 1 -- 172.21.15.105:0/755831724 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc800fad0 con 0x7fabd80cbc90 2023-11-28T18:31:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.853+0000 7fabcd7fa640 1 -- 172.21.15.105:0/755831724 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc800fc30 con 0x7fabd80cbc90 2023-11-28T18:31:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.853+0000 7fabcd7fa640 1 -- 172.21.15.105:0/755831724 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc8014440 con 0x7fabd80cbc90 2023-11-28T18:31:03.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.853+0000 7fabdf0e3640 1 -- 172.21.15.105:0/755831724 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd815eaf0 con 0x7fabd80cbc90 2023-11-28T18:31:03.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.857+0000 7fabdf0e3640 1 -- 172.21.15.105:0/755831724 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd80b85c0 con 0x7fabd80cbc90 2023-11-28T18:31:03.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.857+0000 7fabdf0e3640 1 -- 172.21.15.105:0/755831724 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd8067570 con 0x7fabd80cbc90 2023-11-28T18:31:03.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.857+0000 7fabcd7fa640 1 -- 172.21.15.105:0/755831724 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fabc80145a0 con 0x7fabd80cbc90 2023-11-28T18:31:03.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.857+0000 7fabcd7fa640 1 --2- 172.21.15.105:0/755831724 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fabc4066900 0x7fabc4068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:03.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.857+0000 7fabcd7fa640 1 -- 172.21.15.105:0/755831724 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fabc8084dc0 con 0x7fabd80cbc90 2023-11-28T18:31:03.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.857+0000 7fabdce7f640 1 --2- 172.21.15.105:0/755831724 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fabc4066900 0x7fabc4068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:03.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.857+0000 7fabdce7f640 1 --2- 172.21.15.105:0/755831724 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fabc4066900 0x7fabc4068dc0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fabd40ad000 tx=0x7fabd4001d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:03.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:03.861+0000 7fabcd7fa640 1 -- 172.21.15.105:0/755831724 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fabc8051e50 con 0x7fabd80cbc90 2023-11-28T18:31:04.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.013+0000 7fabdf0e3640 1 -- 172.21.15.105:0/755831724 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabd80b1f20 con 0x7fabd80cbc90 2023-11-28T18:31:04.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.013+0000 7fabcd7fa640 1 -- 172.21.15.105:0/755831724 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16857 (secure 0 0 0) 0x7fabc8022070 con 0x7fabd80cbc90 2023-11-28T18:31:04.015 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:04.015 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":116,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:31:03.119734+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:31:00.777530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-28T18:31:02.057361+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:31:04.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.017+0000 7fabdf0e3640 1 -- 172.21.15.105:0/755831724 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fabc4066900 msgr2=0x7fabc4068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:04.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.017+0000 7fabdf0e3640 1 --2- 172.21.15.105:0/755831724 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fabc4066900 0x7fabc4068dc0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fabd40ad000 tx=0x7fabd4001d30 comp rx=0 tx=0).stop 2023-11-28T18:31:04.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.017+0000 7fabdf0e3640 1 -- 172.21.15.105:0/755831724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabd80cbc90 msgr2=0x7fabd80b8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:04.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.017+0000 7fabdf0e3640 1 --2- 172.21.15.105:0/755831724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabd80cbc90 0x7fabd80b8080 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fabc80119d0 tx=0x7fabc8011e90 comp rx=0 tx=0).stop 2023-11-28T18:31:04.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.017+0000 7fabdf0e3640 1 -- 172.21.15.105:0/755831724 shutdown_connections 2023-11-28T18:31:04.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.017+0000 7fabdf0e3640 1 --2- 172.21.15.105:0/755831724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabd815f2f0 0x7fabd80b0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:04.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.017+0000 7fabdf0e3640 1 --2- 172.21.15.105:0/755831724 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fabc4066900 0x7fabc4068dc0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:04.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.017+0000 7fabdf0e3640 1 --2- 172.21.15.105:0/755831724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabd80cbc90 0x7fabd80b8080 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:04.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.017+0000 7fabdf0e3640 1 --2- 172.21.15.105:0/755831724 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fabc0003980 0x7fabd815dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:04.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.017+0000 7fabdf0e3640 1 -- 172.21.15.105:0/755831724 >> 172.21.15.105:0/755831724 conn(0x7fabd80d3030 msgr2=0x7fabd806b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:04.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.017+0000 7fabdf0e3640 1 -- 172.21.15.105:0/755831724 shutdown_connections 2023-11-28T18:31:04.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.017+0000 7fabdf0e3640 1 -- 172.21.15.105:0/755831724 wait complete. 2023-11-28T18:31:04.036 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:31:04.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.177+0000 7f6df958d640 1 Processor -- start 2023-11-28T18:31:04.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.177+0000 7f6df958d640 1 -- start start 2023-11-28T18:31:04.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.177+0000 7f6df958d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6df40d3c30 0x7f6df40d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:04.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.177+0000 7f6df958d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6df40b2d20 0x7f6df40b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:04.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.177+0000 7f6df958d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df40b25b0 con 0x7f6df415d4f0 2023-11-28T18:31:04.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.177+0000 7f6df958d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df40b2900 con 0x7f6df40b2d20 2023-11-28T18:31:04.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.177+0000 7f6df958d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df40d0a20 con 0x7f6df40d3c30 2023-11-28T18:31:04.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.177+0000 7f6dea7fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6df40d3c30 0x7f6df40d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:04.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.177+0000 7f6df37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6df40b2d20 0x7f6df40b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:04.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.177+0000 7f6df37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6df40b2d20 0x7f6df40b31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:33802/0 (socket says 172.21.15.105:33802) 2023-11-28T18:31:04.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.177+0000 7f6df2ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6df415d4f0 0x7f6df40d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:44538/0 (socket says 172.21.15.105:44538) 2023-11-28T18:31:04.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.177+0000 7f6dea7fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6df40d3c30 0x7f6df40d1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:57772/0 (socket says 172.21.15.105:57772) 2023-11-28T18:31:04.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.177+0000 7f6df37fe640 1 -- 172.21.15.105:0/3446507401 learned_addr learned my addr 172.21.15.105:0/3446507401 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:04.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.177+0000 7f6dea7fc640 1 -- 172.21.15.105:0/3446507401 >> v2:172.21.15.148:3300/0 conn(0x7f6df40b2d20 msgr2=0x7f6df40b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:04.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.177+0000 7f6dea7fc640 1 --2- 172.21.15.105:0/3446507401 >> v2:172.21.15.148:3300/0 conn(0x7f6df40b2d20 0x7f6df40b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:04.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.177+0000 7f6dea7fc640 1 -- 172.21.15.105:0/3446507401 >> v1:172.21.15.148:6789/0 conn(0x7f6df415d4f0 legacy=0x7f6df40d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:04.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.177+0000 7f6dea7fc640 1 -- 172.21.15.105:0/3446507401 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df40cea30 con 0x7f6df40d3c30 2023-11-28T18:31:04.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.177+0000 7f6df37fe640 1 --2- 172.21.15.105:0/3446507401 >> v2:172.21.15.148:3300/0 conn(0x7f6df40b2d20 0x7f6df40b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:31:04.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.181+0000 7f6dea7fc640 1 --2- 172.21.15.105:0/3446507401 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6df40d3c30 0x7f6df40d1520 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0097c0 tx=0x7f6ddc01c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8db6dcb989d11707 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:04.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.181+0000 7f6df27fc640 1 -- 172.21.15.105:0/3446507401 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc0ac620 con 0x7f6df40d3c30 2023-11-28T18:31:04.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.181+0000 7f6df27fc640 1 -- 172.21.15.105:0/3446507401 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc0ac780 con 0x7f6df40d3c30 2023-11-28T18:31:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.181+0000 7f6df27fc640 1 -- 172.21.15.105:0/3446507401 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc0acb10 con 0x7f6df40d3c30 2023-11-28T18:31:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.181+0000 7f6df958d640 1 -- 172.21.15.105:0/3446507401 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6df40d3c30 msgr2=0x7f6df40d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.181+0000 7f6df958d640 1 --2- 172.21.15.105:0/3446507401 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6df40d3c30 0x7f6df40d1520 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0097c0 tx=0x7f6ddc01c3c0 comp rx=0 tx=0).stop 2023-11-28T18:31:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.181+0000 7f6df958d640 1 -- 172.21.15.105:0/3446507401 shutdown_connections 2023-11-28T18:31:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.181+0000 7f6df958d640 1 --2- 172.21.15.105:0/3446507401 >> v2:172.21.15.148:3300/0 conn(0x7f6df40b2d20 0x7f6df40b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.181+0000 7f6df958d640 1 --2- 172.21.15.105:0/3446507401 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6df40d3c30 0x7f6df40d1520 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.181+0000 7f6df958d640 1 -- 172.21.15.105:0/3446507401 >> 172.21.15.105:0/3446507401 conn(0x7f6df40d3030 msgr2=0x7f6df40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.181+0000 7f6df958d640 1 -- 172.21.15.105:0/3446507401 shutdown_connections 2023-11-28T18:31:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.181+0000 7f6df958d640 1 -- 172.21.15.105:0/3446507401 wait complete. 2023-11-28T18:31:04.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6df958d640 1 Processor -- start 2023-11-28T18:31:04.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6df958d640 1 -- start start 2023-11-28T18:31:04.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6df958d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6df40b2d20 0x7f6df414c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:04.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6df958d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6df40d3c30 0x7f6df415c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:04.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6df958d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6df415d4f0 0x7f6df40d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:04.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6df958d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df414ca00 con 0x7f6df40d3c30 2023-11-28T18:31:04.186 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6df958d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df415a960 con 0x7f6df415d4f0 2023-11-28T18:31:04.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6df958d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df415ac40 con 0x7f6df40b2d20 2023-11-28T18:31:04.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6dea7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6df40d3c30 0x7f6df415c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:04.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6dea7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6df40d3c30 0x7f6df415c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38202/0 (socket says 172.21.15.105:38202) 2023-11-28T18:31:04.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6dea7fc640 1 -- 172.21.15.105:0/1865117559 learned_addr learned my addr 172.21.15.105:0/1865117559 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:04.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6df2ffd640 1 --2- 172.21.15.105:0/1865117559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6df40b2d20 0x7f6df414c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:04.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6df37fe640 1 --2- 172.21.15.105:0/1865117559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6df415d4f0 0x7f6df40d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:04.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6dea7fc640 1 -- 172.21.15.105:0/1865117559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6df40b2d20 msgr2=0x7f6df414c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:04.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6dea7fc640 1 --2- 172.21.15.105:0/1865117559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6df40b2d20 0x7f6df414c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:04.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6dea7fc640 1 -- 172.21.15.105:0/1865117559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6df415d4f0 msgr2=0x7f6df40d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:04.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6dea7fc640 1 --2- 172.21.15.105:0/1865117559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6df415d4f0 0x7f6df40d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:04.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6dea7fc640 1 -- 172.21.15.105:0/1865117559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec009b30 con 0x7f6df40d3c30 2023-11-28T18:31:04.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6df2ffd640 1 --2- 172.21.15.105:0/1865117559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6df40b2d20 0x7f6df414c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:04.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6df37fe640 1 --2- 172.21.15.105:0/1865117559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6df415d4f0 0x7f6df40d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:04.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6dea7fc640 1 --2- 172.21.15.105:0/1865117559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6df40d3c30 0x7f6df415c0d0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0af040 tx=0x7f6ddc01ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:04.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6debfff640 1 -- 172.21.15.105:0/1865117559 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc0b8450 con 0x7f6df40d3c30 2023-11-28T18:31:04.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6debfff640 1 -- 172.21.15.105:0/1865117559 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc0b85d0 con 0x7f6df40d3c30 2023-11-28T18:31:04.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6debfff640 1 -- 172.21.15.105:0/1865117559 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc0b8980 con 0x7f6df40d3c30 2023-11-28T18:31:04.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6df958d640 1 -- 172.21.15.105:0/1865117559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ddc009420 con 0x7f6df40d3c30 2023-11-28T18:31:04.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.185+0000 7f6df958d640 1 -- 172.21.15.105:0/1865117559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df415b200 con 0x7f6df40d3c30 2023-11-28T18:31:04.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.189+0000 7f6df958d640 1 -- 172.21.15.105:0/1865117559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df4067570 con 0x7f6df40d3c30 2023-11-28T18:31:04.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.189+0000 7f6debfff640 1 -- 172.21.15.105:0/1865117559 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f6ddc0aa070 con 0x7f6df40d3c30 2023-11-28T18:31:04.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.189+0000 7f6debfff640 1 --2- 172.21.15.105:0/1865117559 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6dd4066900 0x7f6dd4068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:04.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.189+0000 7f6debfff640 1 -- 172.21.15.105:0/1865117559 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6ddc11eb80 con 0x7f6df40d3c30 2023-11-28T18:31:04.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.189+0000 7f6df2ffd640 1 --2- 172.21.15.105:0/1865117559 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6dd4066900 0x7f6dd4068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:04.191 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.189+0000 7f6df2ffd640 1 --2- 172.21.15.105:0/1865117559 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6dd4066900 0x7f6dd4068dc0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6de4000f70 tx=0x7f6de4007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:04.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.193+0000 7f6debfff640 1 -- 172.21.15.105:0/1865117559 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6ddc0ebd00 con 0x7f6df40d3c30 2023-11-28T18:31:04.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.345+0000 7f6df958d640 1 -- 172.21.15.105:0/1865117559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6df4156040 con 0x7f6df40d3c30 2023-11-28T18:31:04.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.345+0000 7f6debfff640 1 -- 172.21.15.105:0/1865117559 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16844 (secure 0 0 0) 0x7f6ddc0b1090 con 0x7f6df40d3c30 2023-11-28T18:31:04.346 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:04.346 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":117,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:31:04.144749+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:31:00.777530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-28T18:31:02.057361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:31:04.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.349+0000 7f6df958d640 1 -- 172.21.15.105:0/1865117559 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6dd4066900 msgr2=0x7f6dd4068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:04.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.349+0000 7f6df958d640 1 --2- 172.21.15.105:0/1865117559 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6dd4066900 0x7f6dd4068dc0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6de4000f70 tx=0x7f6de4007d50 comp rx=0 tx=0).stop 2023-11-28T18:31:04.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.349+0000 7f6df958d640 1 -- 172.21.15.105:0/1865117559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6df40d3c30 msgr2=0x7f6df415c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:04.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.349+0000 7f6df958d640 1 --2- 172.21.15.105:0/1865117559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6df40d3c30 0x7f6df415c0d0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0af040 tx=0x7f6ddc01ac00 comp rx=0 tx=0).stop 2023-11-28T18:31:04.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.349+0000 7f6df958d640 1 -- 172.21.15.105:0/1865117559 shutdown_connections 2023-11-28T18:31:04.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.349+0000 7f6df958d640 1 --2- 172.21.15.105:0/1865117559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6df415d4f0 0x7f6df40d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:04.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.349+0000 7f6df958d640 1 --2- 172.21.15.105:0/1865117559 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6dd4066900 0x7f6dd4068dc0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:04.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.349+0000 7f6df958d640 1 --2- 172.21.15.105:0/1865117559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6df40d3c30 0x7f6df415c0d0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:04.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.349+0000 7f6df958d640 1 --2- 172.21.15.105:0/1865117559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6df40b2d20 0x7f6df414c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:04.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.349+0000 7f6df958d640 1 -- 172.21.15.105:0/1865117559 >> 172.21.15.105:0/1865117559 conn(0x7f6df40d3030 msgr2=0x7f6df40b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:04.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.349+0000 7f6df958d640 1 -- 172.21.15.105:0/1865117559 shutdown_connections 2023-11-28T18:31:04.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.349+0000 7f6df958d640 1 -- 172.21.15.105:0/1865117559 wait complete. 2023-11-28T18:31:04.367 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T18:31:04.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f415f09c640 1 Processor -- start 2023-11-28T18:31:04.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f415f09c640 1 -- start start 2023-11-28T18:31:04.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f415f09c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f415815b030 0x7f41580709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:04.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f415f09c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41580701d0 con 0x7f41580b4290 2023-11-28T18:31:04.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f415f09c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f415806df80 con 0x7f4158071960 2023-11-28T18:31:04.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f415f09c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4158071df0 con 0x7f415815b030 2023-11-28T18:31:04.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f415ce38640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f41580b4290 0x7f41580b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59560/0 (socket says 172.21.15.105:59560) 2023-11-28T18:31:04.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f415ce38640 1 -- 172.21.15.105:0/1505801071 learned_addr learned my addr 172.21.15.105:0/1505801071 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:04.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f415d639640 1 --2- 172.21.15.105:0/1505801071 >> v2:172.21.15.148:3300/0 conn(0x7f415815b030 0x7f41580709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:04.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f41577fe640 1 -- 172.21.15.105:0/1505801071 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3186895274 0 0) 0x7f41580701d0 con 0x7f41580b4290 2023-11-28T18:31:04.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f41577fe640 1 -- 172.21.15.105:0/1505801071 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4148003530 con 0x7f41580b4290 2023-11-28T18:31:04.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f41577fe640 1 -- 172.21.15.105:0/1505801071 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3109713239 0 0) 0x7f4148003530 con 0x7f41580b4290 2023-11-28T18:31:04.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f41577fe640 1 -- 172.21.15.105:0/1505801071 >> v2:172.21.15.148:3300/0 conn(0x7f415815b030 msgr2=0x7f41580709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:04.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f41577fe640 1 --2- 172.21.15.105:0/1505801071 >> v2:172.21.15.148:3300/0 conn(0x7f415815b030 0x7f41580709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:04.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f41577fe640 1 -- 172.21.15.105:0/1505801071 >> v1:172.21.15.148:6789/0 conn(0x7f4158071960 legacy=0x7f415815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:04.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f41577fe640 1 -- 172.21.15.105:0/1505801071 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41580720d0 con 0x7f41580b4290 2023-11-28T18:31:04.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f41577fe640 1 -- 172.21.15.105:0/1505801071 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f414c001680 con 0x7f41580b4290 2023-11-28T18:31:04.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f41577fe640 1 -- 172.21.15.105:0/1505801071 >> v1:172.21.15.105:6789/0 conn(0x7f41580b4290 legacy=0x7f41580b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:04.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f41577fe640 1 --2- 172.21.15.105:0/1505801071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41480039d0 0x7f4148003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:04.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f41577fe640 1 --2- 172.21.15.105:0/1505801071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41480047d0 0x7f4148004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:04.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f41577fe640 1 --2- 172.21.15.105:0/1505801071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4148005190 0x7f414800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:04.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f41577fe640 1 -- 172.21.15.105:0/1505801071 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4158071df0 con 0x7f41480047d0 2023-11-28T18:31:04.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f41577fe640 1 -- 172.21.15.105:0/1505801071 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414800c090 con 0x7f4148005190 2023-11-28T18:31:04.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f41577fe640 1 -- 172.21.15.105:0/1505801071 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414800c250 con 0x7f41480039d0 2023-11-28T18:31:04.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f4157fff640 1 --2- 172.21.15.105:0/1505801071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41480047d0 0x7f4148004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:04.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f415ce38640 1 --2- 172.21.15.105:0/1505801071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41480039d0 0x7f4148003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:04.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.509+0000 7f415d639640 1 --2- 172.21.15.105:0/1505801071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4148005190 0x7f414800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:04.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.513+0000 7f4157fff640 1 -- 172.21.15.105:0/1505801071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41480039d0 msgr2=0x7f4148003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:04.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.513+0000 7f4157fff640 1 --2- 172.21.15.105:0/1505801071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41480039d0 0x7f4148003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:04.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.513+0000 7f4157fff640 1 -- 172.21.15.105:0/1505801071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4148005190 msgr2=0x7f414800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:04.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.513+0000 7f4157fff640 1 --2- 172.21.15.105:0/1505801071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4148005190 0x7f414800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:04.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.513+0000 7f4157fff640 1 -- 172.21.15.105:0/1505801071 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4148003530 con 0x7f41480047d0 2023-11-28T18:31:04.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.513+0000 7f415d639640 1 --2- 172.21.15.105:0/1505801071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4148005190 0x7f414800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:04.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.513+0000 7f415d639640 1 -- 172.21.15.105:0/1505801071 reap_dead start 2023-11-28T18:31:04.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.513+0000 7f4157fff640 1 --2- 172.21.15.105:0/1505801071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41480047d0 0x7f4148004c50 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f4144001780 tx=0x7f414401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6009e224d3dacf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:04.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.513+0000 7f41577fe640 1 -- 172.21.15.105:0/1505801071 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41440a4ea0 con 0x7f41480047d0 2023-11-28T18:31:04.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.513+0000 7f41577fe640 1 -- 172.21.15.105:0/1505801071 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41440a7050 con 0x7f41480047d0 2023-11-28T18:31:04.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.513+0000 7f41577fe640 1 -- 172.21.15.105:0/1505801071 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4144003600 con 0x7f41480047d0 2023-11-28T18:31:04.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.513+0000 7f415f09c640 1 -- 172.21.15.105:0/1505801071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41480047d0 msgr2=0x7f4148004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:04.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.513+0000 7f415f09c640 1 --2- 172.21.15.105:0/1505801071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41480047d0 0x7f4148004c50 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f4144001780 tx=0x7f414401c3e0 comp rx=0 tx=0).stop 2023-11-28T18:31:04.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.513+0000 7f415f09c640 1 -- 172.21.15.105:0/1505801071 shutdown_connections 2023-11-28T18:31:04.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.513+0000 7f415f09c640 1 --2- 172.21.15.105:0/1505801071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41480047d0 0x7f4148004c50 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:04.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.513+0000 7f415f09c640 1 -- 172.21.15.105:0/1505801071 >> 172.21.15.105:0/1505801071 conn(0x7f41580ceb90 msgr2=0x7f41580cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:04.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.513+0000 7f415f09c640 1 -- 172.21.15.105:0/1505801071 shutdown_connections 2023-11-28T18:31:04.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.513+0000 7f415f09c640 1 -- 172.21.15.105:0/1505801071 wait complete. 2023-11-28T18:31:04.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.517+0000 7f415f09c640 1 Processor -- start 2023-11-28T18:31:04.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.517+0000 7f415f09c640 1 -- start start 2023-11-28T18:31:04.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.517+0000 7f415f09c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41480047d0 0x7f41581efd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:04.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.517+0000 7f415f09c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41580b23c0 0x7f41580b2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:04.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.517+0000 7f415f09c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41581f5ff0 0x7f41581f6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:04.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.517+0000 7f415f09c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41581f69b0 con 0x7f41480047d0 2023-11-28T18:31:04.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.517+0000 7f415f09c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4158153e90 con 0x7f41581f5ff0 2023-11-28T18:31:04.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.517+0000 7f415f09c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4158154170 con 0x7f41580b23c0 2023-11-28T18:31:04.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.517+0000 7f415ce38640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41480047d0 0x7f41581efd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:04.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.517+0000 7f415ce38640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41480047d0 0x7f41581efd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38224/0 (socket says 172.21.15.105:38224) 2023-11-28T18:31:04.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.517+0000 7f415ce38640 1 -- 172.21.15.105:0/172875173 learned_addr learned my addr 172.21.15.105:0/172875173 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:04.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.517+0000 7f4157fff640 1 --2- 172.21.15.105:0/172875173 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41580b23c0 0x7f41580b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:04.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.517+0000 7f415d639640 1 --2- 172.21.15.105:0/172875173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41581f5ff0 0x7f41581f6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:04.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.517+0000 7f415ce38640 1 -- 172.21.15.105:0/172875173 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41580b23c0 msgr2=0x7f41580b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:04.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.517+0000 7f415ce38640 1 --2- 172.21.15.105:0/172875173 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41580b23c0 0x7f41580b2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:04.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.517+0000 7f415ce38640 1 -- 172.21.15.105:0/172875173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41581f5ff0 msgr2=0x7f41581f6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:04.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.517+0000 7f415ce38640 1 --2- 172.21.15.105:0/172875173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41581f5ff0 0x7f41581f6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:04.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.517+0000 7f415ce38640 1 -- 172.21.15.105:0/172875173 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4158154450 con 0x7f41480047d0 2023-11-28T18:31:04.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.517+0000 7f415ce38640 1 --2- 172.21.15.105:0/172875173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41480047d0 0x7f41581efd70 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f414c012470 tx=0x7f414c012930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:04.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.517+0000 7f41557fa640 1 -- 172.21.15.105:0/172875173 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c014810 con 0x7f41480047d0 2023-11-28T18:31:04.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.521+0000 7f41557fa640 1 -- 172.21.15.105:0/172875173 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414c014990 con 0x7f41480047d0 2023-11-28T18:31:04.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.521+0000 7f415f09c640 1 -- 172.21.15.105:0/172875173 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4158154730 con 0x7f41480047d0 2023-11-28T18:31:04.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.521+0000 7f41557fa640 1 -- 172.21.15.105:0/172875173 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c014d40 con 0x7f41480047d0 2023-11-28T18:31:04.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.521+0000 7f415f09c640 1 -- 172.21.15.105:0/172875173 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41581f02b0 con 0x7f41480047d0 2023-11-28T18:31:04.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.521+0000 7f415f09c640 1 -- 172.21.15.105:0/172875173 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41581548c0 con 0x7f41480047d0 2023-11-28T18:31:04.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.521+0000 7f41557fa640 1 -- 172.21.15.105:0/172875173 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f414c0153e0 con 0x7f41480047d0 2023-11-28T18:31:04.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.521+0000 7f41557fa640 1 --2- 172.21.15.105:0/172875173 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f413c066900 0x7f413c068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:04.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.521+0000 7f41557fa640 1 -- 172.21.15.105:0/172875173 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f414c086f70 con 0x7f41480047d0 2023-11-28T18:31:04.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.521+0000 7f4157fff640 1 --2- 172.21.15.105:0/172875173 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f413c066900 0x7f413c068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:04.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.525+0000 7f4157fff640 1 --2- 172.21.15.105:0/172875173 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f413c066900 0x7f413c068dc0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4144001750 tx=0x7f41440011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:04.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.525+0000 7f41557fa640 1 -- 172.21.15.105:0/172875173 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f414c057aa0 con 0x7f41480047d0 2023-11-28T18:31:04.569 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-11-28T18:31:04.569 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-11-28T18:31:04.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:04.713+0000 7f415f09c640 1 -- 172.21.15.105:0/172875173 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4158067570 con 0x7f41480047d0 2023-11-28T18:31:05.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.701+0000 7f41557fa640 1 -- 172.21.15.105:0/172875173 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v118) v1 ==== 118+0+0 (secure 0 0 0) 0x7f414c01c0b0 con 0x7f41480047d0 2023-11-28T18:31:05.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.701+0000 7f415f09c640 1 -- 172.21.15.105:0/172875173 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f413c066900 msgr2=0x7f413c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:05.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.701+0000 7f415f09c640 1 --2- 172.21.15.105:0/172875173 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f413c066900 0x7f413c068dc0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4144001750 tx=0x7f41440011a0 comp rx=0 tx=0).stop 2023-11-28T18:31:05.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.701+0000 7f415f09c640 1 -- 172.21.15.105:0/172875173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41480047d0 msgr2=0x7f41581efd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:05.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.701+0000 7f415f09c640 1 --2- 172.21.15.105:0/172875173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41480047d0 0x7f41581efd70 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f414c012470 tx=0x7f414c012930 comp rx=0 tx=0).stop 2023-11-28T18:31:05.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.701+0000 7f415f09c640 1 -- 172.21.15.105:0/172875173 shutdown_connections 2023-11-28T18:31:05.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.701+0000 7f415f09c640 1 --2- 172.21.15.105:0/172875173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41581f5ff0 0x7f41581f6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:05.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.701+0000 7f415f09c640 1 --2- 172.21.15.105:0/172875173 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41580b23c0 0x7f41580b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:05.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.701+0000 7f415f09c640 1 --2- 172.21.15.105:0/172875173 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f413c066900 0x7f413c068dc0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:05.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.701+0000 7f415f09c640 1 --2- 172.21.15.105:0/172875173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41480047d0 0x7f41581efd70 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:05.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.701+0000 7f415f09c640 1 -- 172.21.15.105:0/172875173 >> 172.21.15.105:0/172875173 conn(0x7f41580ceb90 msgr2=0x7f41580af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:05.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.701+0000 7f415f09c640 1 -- 172.21.15.105:0/172875173 shutdown_connections 2023-11-28T18:31:05.704 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.701+0000 7f415f09c640 1 -- 172.21.15.105:0/172875173 wait complete. 2023-11-28T18:31:05.725 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T18:31:05.725 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T18:31:05.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.869+0000 7f33830c9640 1 Processor -- start 2023-11-28T18:31:05.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.869+0000 7f33830c9640 1 -- start start 2023-11-28T18:31:05.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.869+0000 7f33830c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f337c0b4290 0x7f337c0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:05.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.869+0000 7f33830c9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f337c0b4bb0 con 0x7f337c15b030 2023-11-28T18:31:05.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.869+0000 7f33830c9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f337c0701d0 con 0x7f337c071960 2023-11-28T18:31:05.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.869+0000 7f33830c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f337c06df80 con 0x7f337c0b4290 2023-11-28T18:31:05.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.873+0000 7f3381666640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f337c15b030 0x7f337c0709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59564/0 (socket says 172.21.15.105:59564) 2023-11-28T18:31:05.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.873+0000 7f3381666640 1 -- 172.21.15.105:0/3898363905 learned_addr learned my addr 172.21.15.105:0/3898363905 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:05.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.873+0000 7f3380e65640 1 --2- 172.21.15.105:0/3898363905 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f337c0b4290 0x7f337c0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:05.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.873+0000 7f33737fe640 1 -- 172.21.15.105:0/3898363905 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1309586921 0 0) 0x7f337c0b4bb0 con 0x7f337c15b030 2023-11-28T18:31:05.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.873+0000 7f33737fe640 1 -- 172.21.15.105:0/3898363905 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3364003530 con 0x7f337c15b030 2023-11-28T18:31:05.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.873+0000 7f33737fe640 1 -- 172.21.15.105:0/3898363905 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082011604 0 0) 0x7f337c0701d0 con 0x7f337c071960 2023-11-28T18:31:05.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.873+0000 7f33737fe640 1 -- 172.21.15.105:0/3898363905 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f337c0b4bb0 con 0x7f337c071960 2023-11-28T18:31:05.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.873+0000 7f33737fe640 1 -- 172.21.15.105:0/3898363905 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1923238833 0 0) 0x7f3364003530 con 0x7f337c15b030 2023-11-28T18:31:05.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.873+0000 7f3380e65640 1 -- 172.21.15.105:0/3898363905 >> v1:172.21.15.148:6789/0 conn(0x7f337c071960 legacy=0x7f337c15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:05.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.873+0000 7f3380e65640 1 -- 172.21.15.105:0/3898363905 >> v1:172.21.15.105:6789/0 conn(0x7f337c15b030 legacy=0x7f337c0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:05.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.873+0000 7f3380e65640 1 -- 172.21.15.105:0/3898363905 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f337c071df0 con 0x7f337c0b4290 2023-11-28T18:31:05.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.873+0000 7f3380e65640 1 --2- 172.21.15.105:0/3898363905 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f337c0b4290 0x7f337c0b4670 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f33780097f0 tx=0x7f337801c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39e310416cf4f61c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:05.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.877+0000 7f33737fe640 1 -- 172.21.15.105:0/3898363905 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3374001a00 con 0x7f337c0b4290 2023-11-28T18:31:05.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.877+0000 7f33737fe640 1 -- 172.21.15.105:0/3898363905 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33780a7050 con 0x7f337c0b4290 2023-11-28T18:31:05.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.877+0000 7f33737fe640 1 -- 172.21.15.105:0/3898363905 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33780ab620 con 0x7f337c0b4290 2023-11-28T18:31:05.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.877+0000 7f33830c9640 1 -- 172.21.15.105:0/3898363905 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f337c0b4290 msgr2=0x7f337c0b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:05.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.877+0000 7f33830c9640 1 --2- 172.21.15.105:0/3898363905 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f337c0b4290 0x7f337c0b4670 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f33780097f0 tx=0x7f337801c3b0 comp rx=0 tx=0).stop 2023-11-28T18:31:05.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.877+0000 7f33830c9640 1 -- 172.21.15.105:0/3898363905 shutdown_connections 2023-11-28T18:31:05.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.877+0000 7f33830c9640 1 --2- 172.21.15.105:0/3898363905 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f337c0b4290 0x7f337c0b4670 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:05.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.877+0000 7f33830c9640 1 -- 172.21.15.105:0/3898363905 >> 172.21.15.105:0/3898363905 conn(0x7f337c0ceb90 msgr2=0x7f337c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:05.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.877+0000 7f33830c9640 1 -- 172.21.15.105:0/3898363905 shutdown_connections 2023-11-28T18:31:05.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.877+0000 7f33830c9640 1 -- 172.21.15.105:0/3898363905 wait complete. 2023-11-28T18:31:05.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.877+0000 7f33830c9640 1 Processor -- start 2023-11-28T18:31:05.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.877+0000 7f33830c9640 1 -- start start 2023-11-28T18:31:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.877+0000 7f33830c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f337c0b4290 0x7f337c158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.877+0000 7f33830c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f337c15b030 0x7f337c0ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.877+0000 7f33830c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f337c154ed0 0x7f337c0cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.877+0000 7f33830c9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337c1586e0 con 0x7f337c0b4290 2023-11-28T18:31:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.877+0000 7f33830c9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337c155a40 con 0x7f337c154ed0 2023-11-28T18:31:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.877+0000 7f33830c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f337c157db0 con 0x7f337c15b030 2023-11-28T18:31:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.877+0000 7f3380e65640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f337c0b4290 0x7f337c158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.877+0000 7f3380e65640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f337c0b4290 0x7f337c158b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38234/0 (socket says 172.21.15.105:38234) 2023-11-28T18:31:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.877+0000 7f3380e65640 1 -- 172.21.15.105:0/438509596 learned_addr learned my addr 172.21.15.105:0/438509596 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.881+0000 7f3373fff640 1 --2- 172.21.15.105:0/438509596 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f337c15b030 0x7f337c0ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.881+0000 7f3381666640 1 --2- 172.21.15.105:0/438509596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f337c154ed0 0x7f337c0cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.881+0000 7f3380e65640 1 -- 172.21.15.105:0/438509596 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f337c15b030 msgr2=0x7f337c0ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.881+0000 7f3380e65640 1 --2- 172.21.15.105:0/438509596 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f337c15b030 0x7f337c0ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.881+0000 7f3380e65640 1 -- 172.21.15.105:0/438509596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f337c154ed0 msgr2=0x7f337c0cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.881+0000 7f3380e65640 1 --2- 172.21.15.105:0/438509596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f337c154ed0 0x7f337c0cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.881+0000 7f3380e65640 1 -- 172.21.15.105:0/438509596 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3378009420 con 0x7f337c0b4290 2023-11-28T18:31:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.881+0000 7f3380e65640 1 --2- 172.21.15.105:0/438509596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f337c0b4290 0x7f337c158b70 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f33780ae040 tx=0x7f3378019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.881+0000 7f33717fa640 1 -- 172.21.15.105:0/438509596 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33780bb070 con 0x7f337c0b4290 2023-11-28T18:31:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.881+0000 7f33717fa640 1 -- 172.21.15.105:0/438509596 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33780abc90 con 0x7f337c0b4290 2023-11-28T18:31:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.881+0000 7f33717fa640 1 -- 172.21.15.105:0/438509596 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33780b4410 con 0x7f337c0b4290 2023-11-28T18:31:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.881+0000 7f33830c9640 1 -- 172.21.15.105:0/438509596 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f337c158090 con 0x7f337c0b4290 2023-11-28T18:31:05.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.881+0000 7f33830c9640 1 -- 172.21.15.105:0/438509596 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f337c09bd50 con 0x7f337c0b4290 2023-11-28T18:31:05.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.881+0000 7f33830c9640 1 -- 172.21.15.105:0/438509596 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f337c067570 con 0x7f337c0b4290 2023-11-28T18:31:05.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.881+0000 7f33717fa640 1 -- 172.21.15.105:0/438509596 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f33780b2040 con 0x7f337c0b4290 2023-11-28T18:31:05.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.881+0000 7f33717fa640 1 --2- 172.21.15.105:0/438509596 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3368066900 0x7f3368068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:05.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.881+0000 7f33717fa640 1 -- 172.21.15.105:0/438509596 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f337811d910 con 0x7f337c0b4290 2023-11-28T18:31:05.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.885+0000 7f3373fff640 1 --2- 172.21.15.105:0/438509596 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3368066900 0x7f3368068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:05.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.885+0000 7f3373fff640 1 --2- 172.21.15.105:0/438509596 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3368066900 0x7f3368068dc0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f337c1590b0 tx=0x7f336c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:05.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:05.885+0000 7f33717fa640 1 -- 172.21.15.105:0/438509596 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f33780eaaa0 con 0x7f337c0b4290 2023-11-28T18:31:06.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.065+0000 7f33830c9640 1 -- 172.21.15.105:0/438509596 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f337c0cf7f0 con 0x7f337c0b4290 2023-11-28T18:31:06.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.709+0000 7f33717fa640 1 -- 172.21.15.105:0/438509596 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v119) v1 ==== 122+0+0 (secure 0 0 0) 0x7f33780b0020 con 0x7f337c0b4290 2023-11-28T18:31:06.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.713+0000 7f33830c9640 1 -- 172.21.15.105:0/438509596 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3368066900 msgr2=0x7f3368068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:06.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.713+0000 7f33830c9640 1 --2- 172.21.15.105:0/438509596 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3368066900 0x7f3368068dc0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f337c1590b0 tx=0x7f336c009040 comp rx=0 tx=0).stop 2023-11-28T18:31:06.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.713+0000 7f33830c9640 1 -- 172.21.15.105:0/438509596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f337c0b4290 msgr2=0x7f337c158b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:06.712 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.713+0000 7f33830c9640 1 --2- 172.21.15.105:0/438509596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f337c0b4290 0x7f337c158b70 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f33780ae040 tx=0x7f3378019150 comp rx=0 tx=0).stop 2023-11-28T18:31:06.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.713+0000 7f33830c9640 1 -- 172.21.15.105:0/438509596 shutdown_connections 2023-11-28T18:31:06.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.713+0000 7f33830c9640 1 --2- 172.21.15.105:0/438509596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f337c154ed0 0x7f337c0cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:06.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.713+0000 7f33830c9640 1 --2- 172.21.15.105:0/438509596 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f337c15b030 0x7f337c0ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:06.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.713+0000 7f33830c9640 1 --2- 172.21.15.105:0/438509596 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3368066900 0x7f3368068dc0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:06.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.713+0000 7f33830c9640 1 --2- 172.21.15.105:0/438509596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f337c0b4290 0x7f337c158b70 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:06.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.713+0000 7f33830c9640 1 -- 172.21.15.105:0/438509596 >> 172.21.15.105:0/438509596 conn(0x7f337c0ceb90 msgr2=0x7f337c0cadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:06.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.713+0000 7f33830c9640 1 -- 172.21.15.105:0/438509596 shutdown_connections 2023-11-28T18:31:06.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.713+0000 7f33830c9640 1 -- 172.21.15.105:0/438509596 wait complete. 2023-11-28T18:31:06.733 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T18:31:06.734 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:31:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.873+0000 7fb43acaa640 1 Processor -- start 2023-11-28T18:31:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.873+0000 7fb43acaa640 1 -- start start 2023-11-28T18:31:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.877+0000 7fb43acaa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb434071960 0x7fb43415f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.877+0000 7fb43acaa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb43415b030 0x7fb4340709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.877+0000 7fb43acaa640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4340701d0 con 0x7fb4340b4290 2023-11-28T18:31:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.877+0000 7fb43acaa640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb43406df80 con 0x7fb43415b030 2023-11-28T18:31:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.877+0000 7fb43acaa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb434071df0 con 0x7fb434071960 2023-11-28T18:31:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.877+0000 7fb438a46640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb4340b4290 0x7fb4340b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59574/0 (socket says 172.21.15.105:59574) 2023-11-28T18:31:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.877+0000 7fb438a46640 1 -- 172.21.15.105:0/3375777653 learned_addr learned my addr 172.21.15.105:0/3375777653 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.877+0000 7fb439247640 1 --2- 172.21.15.105:0/3375777653 >> v2:172.21.15.105:3300/0 conn(0x7fb43415b030 0x7fb4340709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.877+0000 7fb433fff640 1 --2- 172.21.15.105:0/3375777653 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb434071960 0x7fb43415f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.877+0000 7fb4337fe640 1 -- 172.21.15.105:0/3375777653 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 699615487 0 0) 0x7fb4340701d0 con 0x7fb4340b4290 2023-11-28T18:31:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.877+0000 7fb4337fe640 1 -- 172.21.15.105:0/3375777653 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb424003530 con 0x7fb4340b4290 2023-11-28T18:31:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.877+0000 7fb439247640 1 -- 172.21.15.105:0/3375777653 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb434071960 msgr2=0x7fb43415f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.877+0000 7fb439247640 1 --2- 172.21.15.105:0/3375777653 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb434071960 0x7fb43415f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.877+0000 7fb439247640 1 -- 172.21.15.105:0/3375777653 >> v1:172.21.15.105:6789/0 conn(0x7fb4340b4290 legacy=0x7fb4340b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.877+0000 7fb439247640 1 -- 172.21.15.105:0/3375777653 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4340720d0 con 0x7fb43415b030 2023-11-28T18:31:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.881+0000 7fb439247640 1 --2- 172.21.15.105:0/3375777653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb43415b030 0x7fb4340709b0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009d90 tx=0x7fb42c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e63ea5dca078db server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.881+0000 7fb4337fe640 1 -- 172.21.15.105:0/3375777653 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c0a58f0 con 0x7fb43415b030 2023-11-28T18:31:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.881+0000 7fb4337fe640 1 -- 172.21.15.105:0/3375777653 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42c0a5a50 con 0x7fb43415b030 2023-11-28T18:31:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.881+0000 7fb4337fe640 1 -- 172.21.15.105:0/3375777653 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c0a5dc0 con 0x7fb43415b030 2023-11-28T18:31:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.881+0000 7fb43acaa640 1 -- 172.21.15.105:0/3375777653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb43415b030 msgr2=0x7fb4340709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.881+0000 7fb43acaa640 1 --2- 172.21.15.105:0/3375777653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb43415b030 0x7fb4340709b0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009d90 tx=0x7fb42c01c920 comp rx=0 tx=0).stop 2023-11-28T18:31:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.881+0000 7fb43acaa640 1 -- 172.21.15.105:0/3375777653 shutdown_connections 2023-11-28T18:31:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.881+0000 7fb43acaa640 1 --2- 172.21.15.105:0/3375777653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb43415b030 0x7fb4340709b0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:06.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.881+0000 7fb43acaa640 1 --2- 172.21.15.105:0/3375777653 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb434071960 0x7fb43415f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:06.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.881+0000 7fb43acaa640 1 -- 172.21.15.105:0/3375777653 >> 172.21.15.105:0/3375777653 conn(0x7fb4340ceb90 msgr2=0x7fb4340cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:06.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.881+0000 7fb43acaa640 1 -- 172.21.15.105:0/3375777653 shutdown_connections 2023-11-28T18:31:06.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.881+0000 7fb43acaa640 1 -- 172.21.15.105:0/3375777653 wait complete. 2023-11-28T18:31:06.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.881+0000 7fb43acaa640 1 Processor -- start 2023-11-28T18:31:06.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.881+0000 7fb43acaa640 1 -- start start 2023-11-28T18:31:06.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb43acaa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4340b4290 0x7fb43414d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:06.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb43acaa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb43415b030 0x7fb434163f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:06.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb43acaa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb43414c250 0x7fb43414b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:06.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb43acaa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4341f61c0 con 0x7fb43415b030 2023-11-28T18:31:06.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb43acaa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43414cb50 con 0x7fb43414c250 2023-11-28T18:31:06.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb43acaa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb434164450 con 0x7fb4340b4290 2023-11-28T18:31:06.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb433fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb43415b030 0x7fb434163f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:06.885 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb433fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb43415b030 0x7fb434163f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38254/0 (socket says 172.21.15.105:38254) 2023-11-28T18:31:06.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb433fff640 1 -- 172.21.15.105:0/1380074852 learned_addr learned my addr 172.21.15.105:0/1380074852 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:06.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb438a46640 1 --2- 172.21.15.105:0/1380074852 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4340b4290 0x7fb43414d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:06.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb439247640 1 --2- 172.21.15.105:0/1380074852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb43414c250 0x7fb43414b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:06.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb433fff640 1 -- 172.21.15.105:0/1380074852 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4340b4290 msgr2=0x7fb43414d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:06.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb433fff640 1 --2- 172.21.15.105:0/1380074852 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4340b4290 0x7fb43414d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:06.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb433fff640 1 -- 172.21.15.105:0/1380074852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb43414c250 msgr2=0x7fb43414b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:06.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb433fff640 1 --2- 172.21.15.105:0/1380074852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb43414c250 0x7fb43414b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:06.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb433fff640 1 -- 172.21.15.105:0/1380074852 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42c009a70 con 0x7fb43415b030 2023-11-28T18:31:06.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb433fff640 1 --2- 172.21.15.105:0/1380074852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb43415b030 0x7fb434163f10 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fb42000e960 tx=0x7fb42000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:06.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb4317fa640 1 -- 172.21.15.105:0/1380074852 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb420009690 con 0x7fb43415b030 2023-11-28T18:31:06.886 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb4317fa640 1 -- 172.21.15.105:0/1380074852 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42000ca00 con 0x7fb43415b030 2023-11-28T18:31:06.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb43acaa640 1 -- 172.21.15.105:0/1380074852 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb434153cd0 con 0x7fb43415b030 2023-11-28T18:31:06.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb4317fa640 1 -- 172.21.15.105:0/1380074852 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42000cd70 con 0x7fb43415b030 2023-11-28T18:31:06.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb43acaa640 1 -- 172.21.15.105:0/1380074852 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb434154290 con 0x7fb43415b030 2023-11-28T18:31:06.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb43acaa640 1 -- 172.21.15.105:0/1380074852 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4340cbbd0 con 0x7fb43415b030 2023-11-28T18:31:06.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.885+0000 7fb4317fa640 1 -- 172.21.15.105:0/1380074852 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fb420013470 con 0x7fb43415b030 2023-11-28T18:31:06.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.889+0000 7fb4317fa640 1 --2- 172.21.15.105:0/1380074852 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb418066900 0x7fb418068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:06.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.889+0000 7fb4317fa640 1 -- 172.21.15.105:0/1380074852 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb420084820 con 0x7fb43415b030 2023-11-28T18:31:06.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.889+0000 7fb438a46640 1 --2- 172.21.15.105:0/1380074852 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb418066900 0x7fb418068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:06.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.889+0000 7fb438a46640 1 --2- 172.21.15.105:0/1380074852 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb418066900 0x7fb418068dc0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb428005890 tx=0x7fb428009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:06.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:06.893+0000 7fb4317fa640 1 -- 172.21.15.105:0/1380074852 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb420051880 con 0x7fb43415b030 2023-11-28T18:31:07.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.073+0000 7fb43acaa640 1 -- 172.21.15.105:0/1380074852 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4341f6300 con 0x7fb43415b030 2023-11-28T18:31:07.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.073+0000 7fb4317fa640 1 -- 172.21.15.105:0/1380074852 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4927 (secure 0 0 0) 0x7fb4341f6300 con 0x7fb43415b030 2023-11-28T18:31:07.074 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:07.074 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":119,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/2073622668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2073622668},{"type":"v1","addr":"172.21.15.148:6834","nonce":2073622668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":7085,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/270170970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":270170970},{"type":"v1","addr":"172.21.15.148:6835","nonce":270170970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":8640,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3373504883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3373504883},{"type":"v1","addr":"172.21.15.105:6836","nonce":3373504883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:31:03.120082+0000","modified":"2023-11-28T18:31:06.069874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8643},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8643":{"gid":8643,"name":"a","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/764864050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":764864050},{"type":"v1","addr":"172.21.15.105:6837","nonce":764864050}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-28T18:31:07.075 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 119 2023-11-28T18:31:07.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.077+0000 7fb43acaa640 1 -- 172.21.15.105:0/1380074852 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb418066900 msgr2=0x7fb418068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:07.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.077+0000 7fb43acaa640 1 --2- 172.21.15.105:0/1380074852 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb418066900 0x7fb418068dc0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb428005890 tx=0x7fb428009040 comp rx=0 tx=0).stop 2023-11-28T18:31:07.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.077+0000 7fb43acaa640 1 -- 172.21.15.105:0/1380074852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb43415b030 msgr2=0x7fb434163f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:07.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.077+0000 7fb43acaa640 1 --2- 172.21.15.105:0/1380074852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb43415b030 0x7fb434163f10 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fb42000e960 tx=0x7fb42000ee20 comp rx=0 tx=0).stop 2023-11-28T18:31:07.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.077+0000 7fb43acaa640 1 -- 172.21.15.105:0/1380074852 shutdown_connections 2023-11-28T18:31:07.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.077+0000 7fb43acaa640 1 --2- 172.21.15.105:0/1380074852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb43414c250 0x7fb43414b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.077+0000 7fb43acaa640 1 --2- 172.21.15.105:0/1380074852 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb418066900 0x7fb418068dc0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.077+0000 7fb43acaa640 1 --2- 172.21.15.105:0/1380074852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb43415b030 0x7fb434163f10 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.077+0000 7fb43acaa640 1 --2- 172.21.15.105:0/1380074852 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4340b4290 0x7fb43414d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.077+0000 7fb43acaa640 1 -- 172.21.15.105:0/1380074852 >> 172.21.15.105:0/1380074852 conn(0x7fb4340ceb90 msgr2=0x7fb43415f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:07.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.077+0000 7fb43acaa640 1 -- 172.21.15.105:0/1380074852 shutdown_connections 2023-11-28T18:31:07.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.077+0000 7fb43acaa640 1 -- 172.21.15.105:0/1380074852 wait complete. 2023-11-28T18:31:07.096 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:31:07.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.233+0000 7fd631409640 1 Processor -- start 2023-11-28T18:31:07.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.233+0000 7fd631409640 1 -- start start 2023-11-28T18:31:07.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.233+0000 7fd631409640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd62c159f40 0x7fd62c0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:07.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.233+0000 7fd631409640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd62c0cc760 0x7fd62c0ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:07.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.233+0000 7fd631409640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd62c06bf50 con 0x7fd62c0ad680 2023-11-28T18:31:07.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.233+0000 7fd631409640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd62c0ccbc0 con 0x7fd62c159f40 2023-11-28T18:31:07.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.233+0000 7fd631409640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c15a320 con 0x7fd62c0cc760 2023-11-28T18:31:07.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.237+0000 7fd62affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd62c159f40 0x7fd62c0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:07.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.237+0000 7fd62affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd62c159f40 0x7fd62c0caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38258/0 (socket says 172.21.15.105:38258) 2023-11-28T18:31:07.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.237+0000 7fd62affd640 1 -- 172.21.15.105:0/376855268 learned_addr learned my addr 172.21.15.105:0/376855268 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:07.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.237+0000 7fd6227fc640 1 --2- 172.21.15.105:0/376855268 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd62c0cc760 0x7fd62c0ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:07.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.237+0000 7fd62affd640 1 -- 172.21.15.105:0/376855268 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd62c0cc760 msgr2=0x7fd62c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:07.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.237+0000 7fd62affd640 1 --2- 172.21.15.105:0/376855268 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd62c0cc760 0x7fd62c0ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.237+0000 7fd62affd640 1 -- 172.21.15.105:0/376855268 >> v1:172.21.15.105:6789/0 conn(0x7fd62c0ad680 legacy=0x7fd62c15da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:07.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.237+0000 7fd62a7fc640 1 -- 172.21.15.105:0/376855268 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4146115995 0 0) 0x7fd62c06bf50 con 0x7fd62c0ad680 2023-11-28T18:31:07.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.237+0000 7fd62affd640 1 -- 172.21.15.105:0/376855268 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c071380 con 0x7fd62c159f40 2023-11-28T18:31:07.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.237+0000 7fd62affd640 1 --2- 172.21.15.105:0/376855268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd62c159f40 0x7fd62c0caf60 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009870 tx=0x7fd61c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee87a4e65c9b64f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:07.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.237+0000 7fd62a7fc640 1 -- 172.21.15.105:0/376855268 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0ab680 con 0x7fd62c159f40 2023-11-28T18:31:07.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.237+0000 7fd62a7fc640 1 -- 172.21.15.105:0/376855268 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61c0ab7e0 con 0x7fd62c159f40 2023-11-28T18:31:07.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.237+0000 7fd62a7fc640 1 -- 172.21.15.105:0/376855268 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0abb70 con 0x7fd62c159f40 2023-11-28T18:31:07.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd631409640 1 -- 172.21.15.105:0/376855268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd62c159f40 msgr2=0x7fd62c0caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:07.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd631409640 1 --2- 172.21.15.105:0/376855268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd62c159f40 0x7fd62c0caf60 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009870 tx=0x7fd61c01c480 comp rx=0 tx=0).stop 2023-11-28T18:31:07.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd631409640 1 -- 172.21.15.105:0/376855268 shutdown_connections 2023-11-28T18:31:07.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd631409640 1 --2- 172.21.15.105:0/376855268 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd62c0cc760 0x7fd62c0ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd631409640 1 --2- 172.21.15.105:0/376855268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd62c159f40 0x7fd62c0caf60 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd631409640 1 -- 172.21.15.105:0/376855268 >> 172.21.15.105:0/376855268 conn(0x7fd62c0ceb90 msgr2=0x7fd62c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:07.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd631409640 1 -- 172.21.15.105:0/376855268 shutdown_connections 2023-11-28T18:31:07.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd631409640 1 -- 172.21.15.105:0/376855268 wait complete. 2023-11-28T18:31:07.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd631409640 1 Processor -- start 2023-11-28T18:31:07.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd631409640 1 -- start start 2023-11-28T18:31:07.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd631409640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd62c0ad680 0x7fd62c09bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:07.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd631409640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd62c0cc760 0x7fd62c154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:07.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd631409640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd62c159f40 0x7fd62c09c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:07.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd631409640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c0b1a50 con 0x7fd62c0ad680 2023-11-28T18:31:07.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd631409640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c0b1d30 con 0x7fd62c0cc760 2023-11-28T18:31:07.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd631409640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c0b2010 con 0x7fd62c159f40 2023-11-28T18:31:07.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd62affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd62c0ad680 0x7fd62c09bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:07.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd62affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd62c0ad680 0x7fd62c09bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38260/0 (socket says 172.21.15.105:38260) 2023-11-28T18:31:07.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd62affd640 1 -- 172.21.15.105:0/3001296723 learned_addr learned my addr 172.21.15.105:0/3001296723 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:07.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd6227fc640 1 --2- 172.21.15.105:0/3001296723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd62c0cc760 0x7fd62c154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:07.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd62b7fe640 1 --2- 172.21.15.105:0/3001296723 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd62c159f40 0x7fd62c09c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:07.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd62affd640 1 -- 172.21.15.105:0/3001296723 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd62c159f40 msgr2=0x7fd62c09c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:07.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd62affd640 1 --2- 172.21.15.105:0/3001296723 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd62c159f40 0x7fd62c09c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd62affd640 1 -- 172.21.15.105:0/3001296723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd62c0cc760 msgr2=0x7fd62c154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:07.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd62affd640 1 --2- 172.21.15.105:0/3001296723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd62c0cc760 0x7fd62c154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.241+0000 7fd62affd640 1 -- 172.21.15.105:0/3001296723 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c0094d0 con 0x7fd62c0ad680 2023-11-28T18:31:07.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.245+0000 7fd62b7fe640 1 --2- 172.21.15.105:0/3001296723 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd62c159f40 0x7fd62c09c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:07.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.245+0000 7fd62affd640 1 --2- 172.21.15.105:0/3001296723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd62c0ad680 0x7fd62c09bda0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fd61c01a0b0 tx=0x7fd61c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:07.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.245+0000 7fd623fff640 1 -- 172.21.15.105:0/3001296723 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0b4800 con 0x7fd62c0ad680 2023-11-28T18:31:07.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.245+0000 7fd623fff640 1 -- 172.21.15.105:0/3001296723 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61c0b4980 con 0x7fd62c0ad680 2023-11-28T18:31:07.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.245+0000 7fd631409640 1 -- 172.21.15.105:0/3001296723 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd62c0b22f0 con 0x7fd62c0ad680 2023-11-28T18:31:07.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.245+0000 7fd623fff640 1 -- 172.21.15.105:0/3001296723 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0b4d30 con 0x7fd62c0ad680 2023-11-28T18:31:07.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.245+0000 7fd631409640 1 -- 172.21.15.105:0/3001296723 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62c159720 con 0x7fd62c0ad680 2023-11-28T18:31:07.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.245+0000 7fd631409640 1 -- 172.21.15.105:0/3001296723 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd62c067570 con 0x7fd62c0ad680 2023-11-28T18:31:07.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.245+0000 7fd623fff640 1 -- 172.21.15.105:0/3001296723 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd61c0c7390 con 0x7fd62c0ad680 2023-11-28T18:31:07.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.245+0000 7fd623fff640 1 --2- 172.21.15.105:0/3001296723 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd60c0668d0 0x7fd60c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:07.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.245+0000 7fd623fff640 1 -- 172.21.15.105:0/3001296723 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd61c11f210 con 0x7fd62c0ad680 2023-11-28T18:31:07.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.245+0000 7fd6227fc640 1 --2- 172.21.15.105:0/3001296723 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd60c0668d0 0x7fd60c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:07.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.249+0000 7fd6227fc640 1 --2- 172.21.15.105:0/3001296723 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd60c0668d0 0x7fd60c068d90 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd62c158d20 tx=0x7fd614009830 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:07.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.249+0000 7fd623fff640 1 -- 172.21.15.105:0/3001296723 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd61c0ec3a0 con 0x7fd62c0ad680 2023-11-28T18:31:07.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.401+0000 7fd631409640 1 -- 172.21.15.105:0/3001296723 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd62c0b1b90 con 0x7fd62c0ad680 2023-11-28T18:31:07.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.401+0000 7fd623fff640 1 -- 172.21.15.105:0/3001296723 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16844 (secure 0 0 0) 0x7fd62c0b1b90 con 0x7fd62c0ad680 2023-11-28T18:31:07.403 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:07.403 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":117,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:31:04.144749+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:31:00.777530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-28T18:31:02.057361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:31:07.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.405+0000 7fd631409640 1 -- 172.21.15.105:0/3001296723 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd60c0668d0 msgr2=0x7fd60c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:07.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.405+0000 7fd631409640 1 --2- 172.21.15.105:0/3001296723 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd60c0668d0 0x7fd60c068d90 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd62c158d20 tx=0x7fd614009830 comp rx=0 tx=0).stop 2023-11-28T18:31:07.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.405+0000 7fd631409640 1 -- 172.21.15.105:0/3001296723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd62c0ad680 msgr2=0x7fd62c09bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:07.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.405+0000 7fd631409640 1 --2- 172.21.15.105:0/3001296723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd62c0ad680 0x7fd62c09bda0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fd61c01a0b0 tx=0x7fd61c01acc0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.405+0000 7fd631409640 1 -- 172.21.15.105:0/3001296723 shutdown_connections 2023-11-28T18:31:07.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.405+0000 7fd631409640 1 --2- 172.21.15.105:0/3001296723 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd62c159f40 0x7fd62c09c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.405+0000 7fd631409640 1 --2- 172.21.15.105:0/3001296723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd62c0cc760 0x7fd62c154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.405+0000 7fd631409640 1 --2- 172.21.15.105:0/3001296723 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd60c0668d0 0x7fd60c068d90 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.405+0000 7fd631409640 1 --2- 172.21.15.105:0/3001296723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd62c0ad680 0x7fd62c09bda0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.405+0000 7fd631409640 1 -- 172.21.15.105:0/3001296723 >> 172.21.15.105:0/3001296723 conn(0x7fd62c0ceb90 msgr2=0x7fd62c06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:07.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.405+0000 7fd631409640 1 -- 172.21.15.105:0/3001296723 shutdown_connections 2023-11-28T18:31:07.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.405+0000 7fd631409640 1 -- 172.21.15.105:0/3001296723 wait complete. 2023-11-28T18:31:07.426 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T18:31:07.426 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:31:07.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.565+0000 7f6b42aaa640 1 Processor -- start 2023-11-28T18:31:07.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.565+0000 7f6b42aaa640 1 -- start start 2023-11-28T18:31:07.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.565+0000 7f6b42aaa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6b3c15d4f0 0x7f6b3c0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:07.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.565+0000 7f6b42aaa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b3c0b2d20 0x7f6b3c0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:07.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.565+0000 7f6b42aaa640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b3c0b25b0 con 0x7f6b3c0d3c30 2023-11-28T18:31:07.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.565+0000 7f6b42aaa640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b3c0b2900 con 0x7f6b3c15d4f0 2023-11-28T18:31:07.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.565+0000 7f6b42aaa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c0d0a20 con 0x7f6b3c0b2d20 2023-11-28T18:31:07.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.565+0000 7f6b41047640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b3c0b2d20 0x7f6b3c0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:07.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.565+0000 7f6b41047640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b3c0b2d20 0x7f6b3c0b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:57876/0 (socket says 172.21.15.105:57876) 2023-11-28T18:31:07.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.565+0000 7f6b41047640 1 -- 172.21.15.105:0/271487982 learned_addr learned my addr 172.21.15.105:0/271487982 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:07.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.565+0000 7f6b40846640 1 --2- 172.21.15.105:0/271487982 >> v2:172.21.15.148:3300/0 conn(0x7f6b3c15d4f0 0x7f6b3c0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:07.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.565+0000 7f6b40846640 1 -- 172.21.15.105:0/271487982 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b3c0b2d20 msgr2=0x7f6b3c0b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:07.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.565+0000 7f6b40846640 1 --2- 172.21.15.105:0/271487982 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b3c0b2d20 0x7f6b3c0b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.565+0000 7f6b40846640 1 -- 172.21.15.105:0/271487982 >> v1:172.21.15.148:6789/0 conn(0x7f6b3c0d3c30 legacy=0x7f6b3c0d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:07.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.565+0000 7f6b40846640 1 -- 172.21.15.105:0/271487982 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b3c0cea30 con 0x7f6b3c15d4f0 2023-11-28T18:31:07.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.565+0000 7f6b41047640 1 --2- 172.21.15.105:0/271487982 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b3c0b2d20 0x7f6b3c0b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:31:07.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.569+0000 7f6b40846640 1 --2- 172.21.15.105:0/271487982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b3c15d4f0 0x7f6b3c0d0e10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6b300097c0 tx=0x7f6b3001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5fbc9fa532607b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:07.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.569+0000 7f6b3b7fe640 1 -- 172.21.15.105:0/271487982 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b300ac620 con 0x7f6b3c15d4f0 2023-11-28T18:31:07.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.569+0000 7f6b3b7fe640 1 -- 172.21.15.105:0/271487982 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b300ac780 con 0x7f6b3c15d4f0 2023-11-28T18:31:07.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.569+0000 7f6b3b7fe640 1 -- 172.21.15.105:0/271487982 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b300acb10 con 0x7f6b3c15d4f0 2023-11-28T18:31:07.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.569+0000 7f6b42aaa640 1 -- 172.21.15.105:0/271487982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b3c15d4f0 msgr2=0x7f6b3c0d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:07.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.569+0000 7f6b42aaa640 1 --2- 172.21.15.105:0/271487982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b3c15d4f0 0x7f6b3c0d0e10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6b300097c0 tx=0x7f6b3001c3c0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.569+0000 7f6b42aaa640 1 -- 172.21.15.105:0/271487982 shutdown_connections 2023-11-28T18:31:07.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.569+0000 7f6b42aaa640 1 --2- 172.21.15.105:0/271487982 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b3c0b2d20 0x7f6b3c0b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.569+0000 7f6b42aaa640 1 --2- 172.21.15.105:0/271487982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b3c15d4f0 0x7f6b3c0d0e10 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.569+0000 7f6b42aaa640 1 -- 172.21.15.105:0/271487982 >> 172.21.15.105:0/271487982 conn(0x7f6b3c0d3030 msgr2=0x7f6b3c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:07.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.569+0000 7f6b42aaa640 1 -- 172.21.15.105:0/271487982 shutdown_connections 2023-11-28T18:31:07.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.569+0000 7f6b42aaa640 1 -- 172.21.15.105:0/271487982 wait complete. 2023-11-28T18:31:07.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b42aaa640 1 Processor -- start 2023-11-28T18:31:07.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b42aaa640 1 -- start start 2023-11-28T18:31:07.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b42aaa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b3c0b2d20 0x7f6b3c0cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:07.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b42aaa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b3c0d3c30 0x7f6b3c14ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:07.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b42aaa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b3c15d4f0 0x7f6b3c14b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:07.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b42aaa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c0ccfd0 con 0x7f6b3c0b2d20 2023-11-28T18:31:07.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b42aaa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c1599c0 con 0x7f6b3c15d4f0 2023-11-28T18:31:07.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b42aaa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c156f30 con 0x7f6b3c0d3c30 2023-11-28T18:31:07.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b40846640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b3c0b2d20 0x7f6b3c0cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:07.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b40846640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b3c0b2d20 0x7f6b3c0cca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38262/0 (socket says 172.21.15.105:38262) 2023-11-28T18:31:07.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b40846640 1 -- 172.21.15.105:0/1240295188 learned_addr learned my addr 172.21.15.105:0/1240295188 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:07.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b3bfff640 1 --2- 172.21.15.105:0/1240295188 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b3c0d3c30 0x7f6b3c14ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:07.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b41047640 1 --2- 172.21.15.105:0/1240295188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b3c15d4f0 0x7f6b3c14b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:07.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b40846640 1 -- 172.21.15.105:0/1240295188 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b3c0d3c30 msgr2=0x7f6b3c14ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:07.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b40846640 1 --2- 172.21.15.105:0/1240295188 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b3c0d3c30 0x7f6b3c14ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b40846640 1 -- 172.21.15.105:0/1240295188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b3c15d4f0 msgr2=0x7f6b3c14b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:07.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b40846640 1 --2- 172.21.15.105:0/1240295188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b3c15d4f0 0x7f6b3c14b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b40846640 1 -- 172.21.15.105:0/1240295188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b34009b30 con 0x7f6b3c0b2d20 2023-11-28T18:31:07.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b41047640 1 --2- 172.21.15.105:0/1240295188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b3c15d4f0 0x7f6b3c14b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:07.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b40846640 1 --2- 172.21.15.105:0/1240295188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b3c0b2d20 0x7f6b3c0cca90 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f6b30019ff0 tx=0x7f6b30019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:07.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b397fa640 1 -- 172.21.15.105:0/1240295188 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b300b5410 con 0x7f6b3c0b2d20 2023-11-28T18:31:07.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b397fa640 1 -- 172.21.15.105:0/1240295188 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b300b5590 con 0x7f6b3c0b2d20 2023-11-28T18:31:07.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b42aaa640 1 -- 172.21.15.105:0/1240295188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b30009420 con 0x7f6b3c0b2d20 2023-11-28T18:31:07.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b397fa640 1 -- 172.21.15.105:0/1240295188 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b300b5940 con 0x7f6b3c0b2d20 2023-11-28T18:31:07.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b42aaa640 1 -- 172.21.15.105:0/1240295188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b3c159370 con 0x7f6b3c0b2d20 2023-11-28T18:31:07.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.573+0000 7f6b42aaa640 1 -- 172.21.15.105:0/1240295188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b3c067570 con 0x7f6b3c0b2d20 2023-11-28T18:31:07.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.577+0000 7f6b397fa640 1 -- 172.21.15.105:0/1240295188 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f6b300b3040 con 0x7f6b3c0b2d20 2023-11-28T18:31:07.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.577+0000 7f6b397fa640 1 --2- 172.21.15.105:0/1240295188 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b200668d0 0x7f6b20068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:07.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.577+0000 7f6b397fa640 1 -- 172.21.15.105:0/1240295188 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6b3011e770 con 0x7f6b3c0b2d20 2023-11-28T18:31:07.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.577+0000 7f6b3bfff640 1 --2- 172.21.15.105:0/1240295188 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b200668d0 0x7f6b20068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:07.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.577+0000 7f6b3bfff640 1 --2- 172.21.15.105:0/1240295188 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b200668d0 0x7f6b20068d90 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c1538c0 tx=0x7f6b28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:07.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.581+0000 7f6b397fa640 1 -- 172.21.15.105:0/1240295188 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6b300eb990 con 0x7f6b3c0b2d20 2023-11-28T18:31:07.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.729+0000 7f6b42aaa640 1 -- 172.21.15.105:0/1240295188 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6b3c0d1a60 con 0x7f6b200668d0 2023-11-28T18:31:07.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.737+0000 7f6b397fa640 1 -- 172.21.15.105:0/1240295188 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289447 (secure 0 0 0) 0x7f6b28001450 con 0x7f6b200668d0 2023-11-28T18:31:07.741 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:07.743 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:31:07.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.741+0000 7f6b42aaa640 1 -- 172.21.15.105:0/1240295188 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b200668d0 msgr2=0x7f6b20068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:07.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.741+0000 7f6b42aaa640 1 --2- 172.21.15.105:0/1240295188 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b200668d0 0x7f6b20068d90 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c1538c0 tx=0x7f6b28009040 comp rx=0 tx=0).stop 2023-11-28T18:31:07.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.741+0000 7f6b42aaa640 1 -- 172.21.15.105:0/1240295188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b3c0b2d20 msgr2=0x7f6b3c0cca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:07.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.741+0000 7f6b42aaa640 1 --2- 172.21.15.105:0/1240295188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b3c0b2d20 0x7f6b3c0cca90 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f6b30019ff0 tx=0x7f6b30019160 comp rx=0 tx=0).stop 2023-11-28T18:31:07.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.741+0000 7f6b42aaa640 1 -- 172.21.15.105:0/1240295188 shutdown_connections 2023-11-28T18:31:07.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.741+0000 7f6b42aaa640 1 --2- 172.21.15.105:0/1240295188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b3c15d4f0 0x7f6b3c14b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.741+0000 7f6b42aaa640 1 --2- 172.21.15.105:0/1240295188 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b3c0d3c30 0x7f6b3c14ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.741+0000 7f6b42aaa640 1 --2- 172.21.15.105:0/1240295188 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f6b200668d0 0x7f6b20068d90 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.741+0000 7f6b42aaa640 1 --2- 172.21.15.105:0/1240295188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b3c0b2d20 0x7f6b3c0cca90 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.741+0000 7f6b42aaa640 1 -- 172.21.15.105:0/1240295188 >> 172.21.15.105:0/1240295188 conn(0x7f6b3c0d3030 msgr2=0x7f6b3c0cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:07.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.741+0000 7f6b42aaa640 1 -- 172.21.15.105:0/1240295188 shutdown_connections 2023-11-28T18:31:07.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.741+0000 7f6b42aaa640 1 -- 172.21.15.105:0/1240295188 wait complete. 2023-11-28T18:31:07.762 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2023-11-28T18:31:07.366768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":405068,"kb_used_data":12616,"kb_used_omap":12,"kb_used_meta":392307,"kb_avail":754569652,"statfs":{"total":773094113280,"available":772679323648,"internally_reserved":0,"allocated":12918784,"data_stored":6615556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":401722930},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.700357"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358229+0000","last_change":"2023-11-28T18:31:03.165164+0000","last_active":"2023-11-28T18:31:05.358229+0000","last_peered":"2023-11-28T18:31:05.358229+0000","last_clean":"2023-11-28T18:31:05.358229+0000","last_became_active":"2023-11-28T18:31:03.164853+0000","last_became_peered":"2023-11-28T18:31:03.164853+0000","last_unstale":"2023-11-28T18:31:05.358229+0000","last_undegraded":"2023-11-28T18:31:05.358229+0000","last_fullsized":"2023-11-28T18:31:05.358229+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:26:09.301219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154547+0000","last_change":"2023-11-28T18:31:03.176466+0000","last_active":"2023-11-28T18:31:04.154547+0000","last_peered":"2023-11-28T18:31:04.154547+0000","last_clean":"2023-11-28T18:31:04.154547+0000","last_became_active":"2023-11-28T18:31:03.176150+0000","last_became_peered":"2023-11-28T18:31:03.176150+0000","last_unstale":"2023-11-28T18:31:04.154547+0000","last_undegraded":"2023-11-28T18:31:04.154547+0000","last_fullsized":"2023-11-28T18:31:04.154547+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:49:08.879513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154621+0000","last_change":"2023-11-28T18:31:03.171440+0000","last_active":"2023-11-28T18:31:04.154621+0000","last_peered":"2023-11-28T18:31:04.154621+0000","last_clean":"2023-11-28T18:31:04.154621+0000","last_became_active":"2023-11-28T18:31:03.171184+0000","last_became_peered":"2023-11-28T18:31:03.171184+0000","last_unstale":"2023-11-28T18:31:04.154621+0000","last_undegraded":"2023-11-28T18:31:04.154621+0000","last_fullsized":"2023-11-28T18:31:04.154621+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:14:45.424633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.130936+0000","last_change":"2023-11-28T18:31:03.176828+0000","last_active":"2023-11-28T18:31:05.130936+0000","last_peered":"2023-11-28T18:31:05.130936+0000","last_clean":"2023-11-28T18:31:05.130936+0000","last_became_active":"2023-11-28T18:31:03.176561+0000","last_became_peered":"2023-11-28T18:31:03.176561+0000","last_unstale":"2023-11-28T18:31:05.130936+0000","last_undegraded":"2023-11-28T18:31:05.130936+0000","last_fullsized":"2023-11-28T18:31:05.130936+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:44:54.950658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.357388+0000","last_change":"2023-11-28T18:31:03.166770+0000","last_active":"2023-11-28T18:31:05.357388+0000","last_peered":"2023-11-28T18:31:05.357388+0000","last_clean":"2023-11-28T18:31:05.357388+0000","last_became_active":"2023-11-28T18:31:03.165977+0000","last_became_peered":"2023-11-28T18:31:03.165977+0000","last_unstale":"2023-11-28T18:31:05.357388+0000","last_undegraded":"2023-11-28T18:31:05.357388+0000","last_fullsized":"2023-11-28T18:31:05.357388+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:48.726973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356047+0000","last_change":"2023-11-28T18:31:03.171056+0000","last_active":"2023-11-28T18:31:05.356047+0000","last_peered":"2023-11-28T18:31:05.356047+0000","last_clean":"2023-11-28T18:31:05.356047+0000","last_became_active":"2023-11-28T18:31:03.170642+0000","last_became_peered":"2023-11-28T18:31:03.170642+0000","last_unstale":"2023-11-28T18:31:05.356047+0000","last_undegraded":"2023-11-28T18:31:05.356047+0000","last_fullsized":"2023-11-28T18:31:05.356047+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:32:04.317161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356044+0000","last_change":"2023-11-28T18:31:03.174604+0000","last_active":"2023-11-28T18:31:05.356044+0000","last_peered":"2023-11-28T18:31:05.356044+0000","last_clean":"2023-11-28T18:31:05.356044+0000","last_became_active":"2023-11-28T18:31:03.171130+0000","last_became_peered":"2023-11-28T18:31:03.171130+0000","last_unstale":"2023-11-28T18:31:05.356044+0000","last_undegraded":"2023-11-28T18:31:05.356044+0000","last_fullsized":"2023-11-28T18:31:05.356044+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:13:45.614265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.153997+0000","last_change":"2023-11-28T18:31:03.169122+0000","last_active":"2023-11-28T18:31:04.153997+0000","last_peered":"2023-11-28T18:31:04.153997+0000","last_clean":"2023-11-28T18:31:04.153997+0000","last_became_active":"2023-11-28T18:31:03.168849+0000","last_became_peered":"2023-11-28T18:31:03.168849+0000","last_unstale":"2023-11-28T18:31:04.153997+0000","last_undegraded":"2023-11-28T18:31:04.153997+0000","last_fullsized":"2023-11-28T18:31:04.153997+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:35:14.378872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.131126+0000","last_change":"2023-11-28T18:31:03.178668+0000","last_active":"2023-11-28T18:31:05.131126+0000","last_peered":"2023-11-28T18:31:05.131126+0000","last_clean":"2023-11-28T18:31:05.131126+0000","last_became_active":"2023-11-28T18:31:03.178386+0000","last_became_peered":"2023-11-28T18:31:03.178386+0000","last_unstale":"2023-11-28T18:31:05.131126+0000","last_undegraded":"2023-11-28T18:31:05.131126+0000","last_fullsized":"2023-11-28T18:31:05.131126+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:04:54.885976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157778+0000","last_change":"2023-11-28T18:31:03.173327+0000","last_active":"2023-11-28T18:31:04.157778+0000","last_peered":"2023-11-28T18:31:04.157778+0000","last_clean":"2023-11-28T18:31:04.157778+0000","last_became_active":"2023-11-28T18:31:03.173048+0000","last_became_peered":"2023-11-28T18:31:03.173048+0000","last_unstale":"2023-11-28T18:31:04.157778+0000","last_undegraded":"2023-11-28T18:31:04.157778+0000","last_fullsized":"2023-11-28T18:31:04.157778+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:23.273296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358016+0000","last_change":"2023-11-28T18:31:03.159540+0000","last_active":"2023-11-28T18:31:05.358016+0000","last_peered":"2023-11-28T18:31:05.358016+0000","last_clean":"2023-11-28T18:31:05.358016+0000","last_became_active":"2023-11-28T18:31:03.159198+0000","last_became_peered":"2023-11-28T18:31:03.159198+0000","last_unstale":"2023-11-28T18:31:05.358016+0000","last_undegraded":"2023-11-28T18:31:05.358016+0000","last_fullsized":"2023-11-28T18:31:05.358016+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:23.273296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356362+0000","last_change":"2023-11-28T18:31:03.172083+0000","last_active":"2023-11-28T18:31:05.356362+0000","last_peered":"2023-11-28T18:31:05.356362+0000","last_clean":"2023-11-28T18:31:05.356362+0000","last_became_active":"2023-11-28T18:31:03.171134+0000","last_became_peered":"2023-11-28T18:31:03.171134+0000","last_unstale":"2023-11-28T18:31:05.356362+0000","last_undegraded":"2023-11-28T18:31:05.356362+0000","last_fullsized":"2023-11-28T18:31:05.356362+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:58:39.627376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157615+0000","last_change":"2023-11-28T18:31:03.163845+0000","last_active":"2023-11-28T18:31:04.157615+0000","last_peered":"2023-11-28T18:31:04.157615+0000","last_clean":"2023-11-28T18:31:04.157615+0000","last_became_active":"2023-11-28T18:31:03.163564+0000","last_became_peered":"2023-11-28T18:31:03.163564+0000","last_unstale":"2023-11-28T18:31:04.157615+0000","last_undegraded":"2023-11-28T18:31:04.157615+0000","last_fullsized":"2023-11-28T18:31:04.157615+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:46:07.850898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154205+0000","last_change":"2023-11-28T18:31:03.165835+0000","last_active":"2023-11-28T18:31:04.154205+0000","last_peered":"2023-11-28T18:31:04.154205+0000","last_clean":"2023-11-28T18:31:04.154205+0000","last_became_active":"2023-11-28T18:31:03.165516+0000","last_became_peered":"2023-11-28T18:31:03.165516+0000","last_unstale":"2023-11-28T18:31:04.154205+0000","last_undegraded":"2023-11-28T18:31:04.154205+0000","last_fullsized":"2023-11-28T18:31:04.154205+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:42:10.190912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.357221+0000","last_change":"2023-11-28T18:31:03.161357+0000","last_active":"2023-11-28T18:31:05.357221+0000","last_peered":"2023-11-28T18:31:05.357221+0000","last_clean":"2023-11-28T18:31:05.357221+0000","last_became_active":"2023-11-28T18:31:03.160135+0000","last_became_peered":"2023-11-28T18:31:03.160135+0000","last_unstale":"2023-11-28T18:31:05.357221+0000","last_undegraded":"2023-11-28T18:31:05.357221+0000","last_fullsized":"2023-11-28T18:31:05.357221+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:56:36.786912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.355990+0000","last_change":"2023-11-28T18:31:03.166047+0000","last_active":"2023-11-28T18:31:05.355990+0000","last_peered":"2023-11-28T18:31:05.355990+0000","last_clean":"2023-11-28T18:31:05.355990+0000","last_became_active":"2023-11-28T18:31:03.165744+0000","last_became_peered":"2023-11-28T18:31:03.165744+0000","last_unstale":"2023-11-28T18:31:05.355990+0000","last_undegraded":"2023-11-28T18:31:05.355990+0000","last_fullsized":"2023-11-28T18:31:05.355990+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:03:34.675681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157890+0000","last_change":"2023-11-28T18:31:03.167959+0000","last_active":"2023-11-28T18:31:04.157890+0000","last_peered":"2023-11-28T18:31:04.157890+0000","last_clean":"2023-11-28T18:31:04.157890+0000","last_became_active":"2023-11-28T18:31:03.167523+0000","last_became_peered":"2023-11-28T18:31:03.167523+0000","last_unstale":"2023-11-28T18:31:04.157890+0000","last_undegraded":"2023-11-28T18:31:04.157890+0000","last_fullsized":"2023-11-28T18:31:04.157890+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:22:08.164188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154611+0000","last_change":"2023-11-28T18:31:03.174970+0000","last_active":"2023-11-28T18:31:04.154611+0000","last_peered":"2023-11-28T18:31:04.154611+0000","last_clean":"2023-11-28T18:31:04.154611+0000","last_became_active":"2023-11-28T18:31:03.174695+0000","last_became_peered":"2023-11-28T18:31:03.174695+0000","last_unstale":"2023-11-28T18:31:04.154611+0000","last_undegraded":"2023-11-28T18:31:04.154611+0000","last_fullsized":"2023-11-28T18:31:04.154611+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:15:27.520824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.160514+0000","last_change":"2023-11-28T18:31:03.172486+0000","last_active":"2023-11-28T18:31:04.160514+0000","last_peered":"2023-11-28T18:31:04.160514+0000","last_clean":"2023-11-28T18:31:04.160514+0000","last_became_active":"2023-11-28T18:31:03.172212+0000","last_became_peered":"2023-11-28T18:31:03.172212+0000","last_unstale":"2023-11-28T18:31:04.160514+0000","last_undegraded":"2023-11-28T18:31:04.160514+0000","last_fullsized":"2023-11-28T18:31:04.160514+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:56:36.786912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.389471+0000","last_change":"2023-11-28T18:31:03.173970+0000","last_active":"2023-11-28T18:31:04.389471+0000","last_peered":"2023-11-28T18:31:04.389471+0000","last_clean":"2023-11-28T18:31:04.389471+0000","last_became_active":"2023-11-28T18:31:03.173642+0000","last_became_peered":"2023-11-28T18:31:03.173642+0000","last_unstale":"2023-11-28T18:31:04.389471+0000","last_undegraded":"2023-11-28T18:31:04.389471+0000","last_fullsized":"2023-11-28T18:31:04.389471+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:35:35.646951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.130899+0000","last_change":"2023-11-28T18:31:03.177854+0000","last_active":"2023-11-28T18:31:05.130899+0000","last_peered":"2023-11-28T18:31:05.130899+0000","last_clean":"2023-11-28T18:31:05.130899+0000","last_became_active":"2023-11-28T18:31:03.176607+0000","last_became_peered":"2023-11-28T18:31:03.176607+0000","last_unstale":"2023-11-28T18:31:05.130899+0000","last_undegraded":"2023-11-28T18:31:05.130899+0000","last_fullsized":"2023-11-28T18:31:05.130899+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:04.428310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.357489+0000","last_change":"2023-11-28T18:31:03.154435+0000","last_active":"2023-11-28T18:31:05.357489+0000","last_peered":"2023-11-28T18:31:05.357489+0000","last_clean":"2023-11-28T18:31:05.357489+0000","last_became_active":"2023-11-28T18:31:03.154116+0000","last_became_peered":"2023-11-28T18:31:03.154116+0000","last_unstale":"2023-11-28T18:31:05.357489+0000","last_undegraded":"2023-11-28T18:31:05.357489+0000","last_fullsized":"2023-11-28T18:31:05.357489+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:07:05.533635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.131169+0000","last_change":"2023-11-28T18:31:03.177952+0000","last_active":"2023-11-28T18:31:05.131169+0000","last_peered":"2023-11-28T18:31:05.131169+0000","last_clean":"2023-11-28T18:31:05.131169+0000","last_became_active":"2023-11-28T18:31:03.177631+0000","last_became_peered":"2023-11-28T18:31:03.177631+0000","last_unstale":"2023-11-28T18:31:05.131169+0000","last_undegraded":"2023-11-28T18:31:05.131169+0000","last_fullsized":"2023-11-28T18:31:05.131169+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:08:24.255143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.357619+0000","last_change":"2023-11-28T18:31:03.170656+0000","last_active":"2023-11-28T18:31:05.357619+0000","last_peered":"2023-11-28T18:31:05.357619+0000","last_clean":"2023-11-28T18:31:05.357619+0000","last_became_active":"2023-11-28T18:31:03.170150+0000","last_became_peered":"2023-11-28T18:31:03.170150+0000","last_unstale":"2023-11-28T18:31:05.357619+0000","last_undegraded":"2023-11-28T18:31:05.357619+0000","last_fullsized":"2023-11-28T18:31:05.357619+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:48:09.007039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358493+0000","last_change":"2023-11-28T18:31:03.155722+0000","last_active":"2023-11-28T18:31:05.358493+0000","last_peered":"2023-11-28T18:31:05.358493+0000","last_clean":"2023-11-28T18:31:05.358493+0000","last_became_active":"2023-11-28T18:31:03.155394+0000","last_became_peered":"2023-11-28T18:31:03.155394+0000","last_unstale":"2023-11-28T18:31:05.358493+0000","last_undegraded":"2023-11-28T18:31:05.358493+0000","last_fullsized":"2023-11-28T18:31:05.358493+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:42:10.190912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.389622+0000","last_change":"2023-11-28T18:31:03.164633+0000","last_active":"2023-11-28T18:31:04.389622+0000","last_peered":"2023-11-28T18:31:04.389622+0000","last_clean":"2023-11-28T18:31:04.389622+0000","last_became_active":"2023-11-28T18:31:03.164372+0000","last_became_peered":"2023-11-28T18:31:03.164372+0000","last_unstale":"2023-11-28T18:31:04.389622+0000","last_undegraded":"2023-11-28T18:31:04.389622+0000","last_fullsized":"2023-11-28T18:31:04.389622+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:02:01.309143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.160634+0000","last_change":"2023-11-28T18:31:03.172868+0000","last_active":"2023-11-28T18:31:04.160634+0000","last_peered":"2023-11-28T18:31:04.160634+0000","last_clean":"2023-11-28T18:31:04.160634+0000","last_became_active":"2023-11-28T18:31:03.172575+0000","last_became_peered":"2023-11-28T18:31:03.172575+0000","last_unstale":"2023-11-28T18:31:04.160634+0000","last_undegraded":"2023-11-28T18:31:04.160634+0000","last_fullsized":"2023-11-28T18:31:04.160634+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:48.726973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356367+0000","last_change":"2023-11-28T18:31:03.172288+0000","last_active":"2023-11-28T18:31:05.356367+0000","last_peered":"2023-11-28T18:31:05.356367+0000","last_clean":"2023-11-28T18:31:05.356367+0000","last_became_active":"2023-11-28T18:31:03.171687+0000","last_became_peered":"2023-11-28T18:31:03.171687+0000","last_unstale":"2023-11-28T18:31:05.356367+0000","last_undegraded":"2023-11-28T18:31:05.356367+0000","last_fullsized":"2023-11-28T18:31:05.356367+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:55:40.593795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.389484+0000","last_change":"2023-11-28T18:31:03.161400+0000","last_active":"2023-11-28T18:31:04.389484+0000","last_peered":"2023-11-28T18:31:04.389484+0000","last_clean":"2023-11-28T18:31:04.389484+0000","last_became_active":"2023-11-28T18:31:03.160718+0000","last_became_peered":"2023-11-28T18:31:03.160718+0000","last_unstale":"2023-11-28T18:31:04.389484+0000","last_undegraded":"2023-11-28T18:31:04.389484+0000","last_fullsized":"2023-11-28T18:31:04.389484+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:44:26.342490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154282+0000","last_change":"2023-11-28T18:31:03.157973+0000","last_active":"2023-11-28T18:31:04.154282+0000","last_peered":"2023-11-28T18:31:04.154282+0000","last_clean":"2023-11-28T18:31:04.154282+0000","last_became_active":"2023-11-28T18:31:03.157690+0000","last_became_peered":"2023-11-28T18:31:03.157690+0000","last_unstale":"2023-11-28T18:31:04.154282+0000","last_undegraded":"2023-11-28T18:31:04.154282+0000","last_fullsized":"2023-11-28T18:31:04.154282+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:28:49.821579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.357822+0000","last_change":"2023-11-28T18:31:03.163986+0000","last_active":"2023-11-28T18:31:05.357822+0000","last_peered":"2023-11-28T18:31:05.357822+0000","last_clean":"2023-11-28T18:31:05.357822+0000","last_became_active":"2023-11-28T18:31:03.163600+0000","last_became_peered":"2023-11-28T18:31:03.163600+0000","last_unstale":"2023-11-28T18:31:05.357822+0000","last_undegraded":"2023-11-28T18:31:05.357822+0000","last_fullsized":"2023-11-28T18:31:05.357822+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:55:32.753849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.357622+0000","last_change":"2023-11-28T18:31:03.170675+0000","last_active":"2023-11-28T18:31:05.357622+0000","last_peered":"2023-11-28T18:31:05.357622+0000","last_clean":"2023-11-28T18:31:05.357622+0000","last_became_active":"2023-11-28T18:31:03.169960+0000","last_became_peered":"2023-11-28T18:31:03.169960+0000","last_unstale":"2023-11-28T18:31:05.357622+0000","last_undegraded":"2023-11-28T18:31:05.357622+0000","last_fullsized":"2023-11-28T18:31:05.357622+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:08:18.381736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157837+0000","last_change":"2023-11-28T18:31:03.175773+0000","last_active":"2023-11-28T18:31:04.157837+0000","last_peered":"2023-11-28T18:31:04.157837+0000","last_clean":"2023-11-28T18:31:04.157837+0000","last_became_active":"2023-11-28T18:31:03.175492+0000","last_became_peered":"2023-11-28T18:31:03.175492+0000","last_unstale":"2023-11-28T18:31:04.157837+0000","last_undegraded":"2023-11-28T18:31:04.157837+0000","last_fullsized":"2023-11-28T18:31:04.157837+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:35:14.474983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"116'4","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.160552+0000","last_change":"2023-11-28T18:31:02.695616+0000","last_active":"2023-11-28T18:31:04.160552+0000","last_peered":"2023-11-28T18:31:04.160552+0000","last_clean":"2023-11-28T18:31:04.160552+0000","last_became_active":"2023-11-28T18:31:02.695272+0000","last_became_peered":"2023-11-28T18:31:02.695272+0000","last_unstale":"2023-11-28T18:31:04.160552+0000","last_undegraded":"2023-11-28T18:31:04.160552+0000","last_fullsized":"2023-11-28T18:31:04.160552+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:24:21.484867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356560+0000","last_change":"2023-11-28T18:31:03.172403+0000","last_active":"2023-11-28T18:31:05.356560+0000","last_peered":"2023-11-28T18:31:05.356560+0000","last_clean":"2023-11-28T18:31:05.356560+0000","last_became_active":"2023-11-28T18:31:03.170743+0000","last_became_peered":"2023-11-28T18:31:03.170743+0000","last_unstale":"2023-11-28T18:31:05.356560+0000","last_undegraded":"2023-11-28T18:31:05.356560+0000","last_fullsized":"2023-11-28T18:31:05.356560+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:36:23.211383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"116'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157883+0000","last_change":"2023-11-28T18:31:02.695849+0000","last_active":"2023-11-28T18:31:04.157883+0000","last_peered":"2023-11-28T18:31:04.157883+0000","last_clean":"2023-11-28T18:31:04.157883+0000","last_became_active":"2023-11-28T18:31:02.695517+0000","last_became_peered":"2023-11-28T18:31:02.695517+0000","last_unstale":"2023-11-28T18:31:04.157883+0000","last_undegraded":"2023-11-28T18:31:04.157883+0000","last_fullsized":"2023-11-28T18:31:04.157883+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:01:11.184059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358241+0000","last_change":"2023-11-28T18:31:03.174369+0000","last_active":"2023-11-28T18:31:05.358241+0000","last_peered":"2023-11-28T18:31:05.358241+0000","last_clean":"2023-11-28T18:31:05.358241+0000","last_became_active":"2023-11-28T18:31:03.174053+0000","last_became_peered":"2023-11-28T18:31:03.174053+0000","last_unstale":"2023-11-28T18:31:05.358241+0000","last_undegraded":"2023-11-28T18:31:05.358241+0000","last_fullsized":"2023-11-28T18:31:05.358241+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:35:14.378872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.131207+0000","last_change":"2023-11-28T18:31:02.700671+0000","last_active":"2023-11-28T18:31:05.131207+0000","last_peered":"2023-11-28T18:31:05.131207+0000","last_clean":"2023-11-28T18:31:05.131207+0000","last_became_active":"2023-11-28T18:31:02.700392+0000","last_became_peered":"2023-11-28T18:31:02.700392+0000","last_unstale":"2023-11-28T18:31:05.131207+0000","last_undegraded":"2023-11-28T18:31:05.131207+0000","last_fullsized":"2023-11-28T18:31:05.131207+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:28:48.813547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.131242+0000","last_change":"2023-11-28T18:31:03.175584+0000","last_active":"2023-11-28T18:31:05.131242+0000","last_peered":"2023-11-28T18:31:05.131242+0000","last_clean":"2023-11-28T18:31:05.131242+0000","last_became_active":"2023-11-28T18:31:03.175322+0000","last_became_peered":"2023-11-28T18:31:03.175322+0000","last_unstale":"2023-11-28T18:31:05.131242+0000","last_undegraded":"2023-11-28T18:31:05.131242+0000","last_fullsized":"2023-11-28T18:31:05.131242+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:23:29.439976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157922+0000","last_change":"2023-11-28T18:31:02.699895+0000","last_active":"2023-11-28T18:31:04.157922+0000","last_peered":"2023-11-28T18:31:04.157922+0000","last_clean":"2023-11-28T18:31:04.157922+0000","last_became_active":"2023-11-28T18:31:02.699627+0000","last_became_peered":"2023-11-28T18:31:02.699627+0000","last_unstale":"2023-11-28T18:31:04.157922+0000","last_undegraded":"2023-11-28T18:31:04.157922+0000","last_fullsized":"2023-11-28T18:31:04.157922+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:30:38.565540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157572+0000","last_change":"2023-11-28T18:31:03.171333+0000","last_active":"2023-11-28T18:31:04.157572+0000","last_peered":"2023-11-28T18:31:04.157572+0000","last_clean":"2023-11-28T18:31:04.157572+0000","last_became_active":"2023-11-28T18:31:03.171028+0000","last_became_peered":"2023-11-28T18:31:03.171028+0000","last_unstale":"2023-11-28T18:31:04.157572+0000","last_undegraded":"2023-11-28T18:31:04.157572+0000","last_fullsized":"2023-11-28T18:31:04.157572+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:51:06.589481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"116'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.357399+0000","last_change":"2023-11-28T18:31:02.702240+0000","last_active":"2023-11-28T18:31:05.357399+0000","last_peered":"2023-11-28T18:31:05.357399+0000","last_clean":"2023-11-28T18:31:05.357399+0000","last_became_active":"2023-11-28T18:31:02.701944+0000","last_became_peered":"2023-11-28T18:31:02.701944+0000","last_unstale":"2023-11-28T18:31:05.357399+0000","last_undegraded":"2023-11-28T18:31:05.357399+0000","last_fullsized":"2023-11-28T18:31:05.357399+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:30:38.565540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.357198+0000","last_change":"2023-11-28T18:31:03.166490+0000","last_active":"2023-11-28T18:31:05.357198+0000","last_peered":"2023-11-28T18:31:05.357198+0000","last_clean":"2023-11-28T18:31:05.357198+0000","last_became_active":"2023-11-28T18:31:03.166205+0000","last_became_peered":"2023-11-28T18:31:03.166205+0000","last_unstale":"2023-11-28T18:31:05.357198+0000","last_undegraded":"2023-11-28T18:31:05.357198+0000","last_fullsized":"2023-11-28T18:31:05.357198+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:46:07.850898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"116'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356092+0000","last_change":"2023-11-28T18:31:02.702744+0000","last_active":"2023-11-28T18:31:05.356092+0000","last_peered":"2023-11-28T18:31:05.356092+0000","last_clean":"2023-11-28T18:31:05.356092+0000","last_became_active":"2023-11-28T18:31:02.702458+0000","last_became_peered":"2023-11-28T18:31:02.702458+0000","last_unstale":"2023-11-28T18:31:05.356092+0000","last_undegraded":"2023-11-28T18:31:05.356092+0000","last_fullsized":"2023-11-28T18:31:05.356092+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:39:06.258713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154330+0000","last_change":"2023-11-28T18:31:03.162176+0000","last_active":"2023-11-28T18:31:04.154330+0000","last_peered":"2023-11-28T18:31:04.154330+0000","last_clean":"2023-11-28T18:31:04.154330+0000","last_became_active":"2023-11-28T18:31:03.161868+0000","last_became_peered":"2023-11-28T18:31:03.161868+0000","last_unstale":"2023-11-28T18:31:04.154330+0000","last_undegraded":"2023-11-28T18:31:04.154330+0000","last_fullsized":"2023-11-28T18:31:04.154330+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:10.291448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.357772+0000","last_change":"2023-11-28T18:31:02.704113+0000","last_active":"2023-11-28T18:31:05.357772+0000","last_peered":"2023-11-28T18:31:05.357772+0000","last_clean":"2023-11-28T18:31:05.357772+0000","last_became_active":"2023-11-28T18:31:02.703873+0000","last_became_peered":"2023-11-28T18:31:02.703873+0000","last_unstale":"2023-11-28T18:31:05.357772+0000","last_undegraded":"2023-11-28T18:31:05.357772+0000","last_fullsized":"2023-11-28T18:31:05.357772+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:56:35.778880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157759+0000","last_change":"2023-11-28T18:31:03.161690+0000","last_active":"2023-11-28T18:31:04.157759+0000","last_peered":"2023-11-28T18:31:04.157759+0000","last_clean":"2023-11-28T18:31:04.157759+0000","last_became_active":"2023-11-28T18:31:03.161405+0000","last_became_peered":"2023-11-28T18:31:03.161405+0000","last_unstale":"2023-11-28T18:31:04.157759+0000","last_undegraded":"2023-11-28T18:31:04.157759+0000","last_fullsized":"2023-11-28T18:31:04.157759+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:07:05.533635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356147+0000","last_change":"2023-11-28T18:31:02.704407+0000","last_active":"2023-11-28T18:31:05.356147+0000","last_peered":"2023-11-28T18:31:05.356147+0000","last_clean":"2023-11-28T18:31:05.356147+0000","last_became_active":"2023-11-28T18:31:02.704098+0000","last_became_peered":"2023-11-28T18:31:02.704098+0000","last_unstale":"2023-11-28T18:31:05.356147+0000","last_undegraded":"2023-11-28T18:31:05.356147+0000","last_fullsized":"2023-11-28T18:31:05.356147+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:05:18.687677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.160336+0000","last_change":"2023-11-28T18:31:03.169360+0000","last_active":"2023-11-28T18:31:04.160336+0000","last_peered":"2023-11-28T18:31:04.160336+0000","last_clean":"2023-11-28T18:31:04.160336+0000","last_became_active":"2023-11-28T18:31:03.168937+0000","last_became_peered":"2023-11-28T18:31:03.168937+0000","last_unstale":"2023-11-28T18:31:04.160336+0000","last_undegraded":"2023-11-28T18:31:04.160336+0000","last_fullsized":"2023-11-28T18:31:04.160336+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:46:07.850898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.160539+0000","last_change":"2023-11-28T18:31:02.710256+0000","last_active":"2023-11-28T18:31:04.160539+0000","last_peered":"2023-11-28T18:31:04.160539+0000","last_clean":"2023-11-28T18:31:04.160539+0000","last_became_active":"2023-11-28T18:31:02.709881+0000","last_became_peered":"2023-11-28T18:31:02.709881+0000","last_unstale":"2023-11-28T18:31:04.160539+0000","last_undegraded":"2023-11-28T18:31:04.160539+0000","last_fullsized":"2023-11-28T18:31:04.160539+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:20.015329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358304+0000","last_change":"2023-11-28T18:31:03.155535+0000","last_active":"2023-11-28T18:31:05.358304+0000","last_peered":"2023-11-28T18:31:05.358304+0000","last_clean":"2023-11-28T18:31:05.358304+0000","last_became_active":"2023-11-28T18:31:03.155215+0000","last_became_peered":"2023-11-28T18:31:03.155215+0000","last_unstale":"2023-11-28T18:31:05.358304+0000","last_undegraded":"2023-11-28T18:31:05.358304+0000","last_fullsized":"2023-11-28T18:31:05.358304+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:55.346235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358206+0000","last_change":"2023-11-28T18:31:02.709245+0000","last_active":"2023-11-28T18:31:05.358206+0000","last_peered":"2023-11-28T18:31:05.358206+0000","last_clean":"2023-11-28T18:31:05.358206+0000","last_became_active":"2023-11-28T18:31:02.708549+0000","last_became_peered":"2023-11-28T18:31:02.708549+0000","last_unstale":"2023-11-28T18:31:05.358206+0000","last_undegraded":"2023-11-28T18:31:05.358206+0000","last_fullsized":"2023-11-28T18:31:05.358206+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:41:30.696017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154661+0000","last_change":"2023-11-28T18:31:03.165827+0000","last_active":"2023-11-28T18:31:04.154661+0000","last_peered":"2023-11-28T18:31:04.154661+0000","last_clean":"2023-11-28T18:31:04.154661+0000","last_became_active":"2023-11-28T18:31:03.165485+0000","last_became_peered":"2023-11-28T18:31:03.165485+0000","last_unstale":"2023-11-28T18:31:04.154661+0000","last_undegraded":"2023-11-28T18:31:04.154661+0000","last_fullsized":"2023-11-28T18:31:04.154661+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:07:46.210330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"116'3","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358407+0000","last_change":"2023-11-28T18:31:02.701153+0000","last_active":"2023-11-28T18:31:05.358407+0000","last_peered":"2023-11-28T18:31:05.358407+0000","last_clean":"2023-11-28T18:31:05.358407+0000","last_became_active":"2023-11-28T18:31:02.700500+0000","last_became_peered":"2023-11-28T18:31:02.700500+0000","last_unstale":"2023-11-28T18:31:05.358407+0000","last_undegraded":"2023-11-28T18:31:05.358407+0000","last_fullsized":"2023-11-28T18:31:05.358407+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:35:13.466951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358482+0000","last_change":"2023-11-28T18:31:03.166517+0000","last_active":"2023-11-28T18:31:05.358482+0000","last_peered":"2023-11-28T18:31:05.358482+0000","last_clean":"2023-11-28T18:31:05.358482+0000","last_became_active":"2023-11-28T18:31:03.166189+0000","last_became_peered":"2023-11-28T18:31:03.166189+0000","last_unstale":"2023-11-28T18:31:05.358482+0000","last_undegraded":"2023-11-28T18:31:05.358482+0000","last_fullsized":"2023-11-28T18:31:05.358482+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:44:23.208354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"116'6","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154436+0000","last_change":"2023-11-28T18:31:02.684346+0000","last_active":"2023-11-28T18:31:04.154436+0000","last_peered":"2023-11-28T18:31:04.154436+0000","last_clean":"2023-11-28T18:31:04.154436+0000","last_became_active":"2023-11-28T18:31:02.683995+0000","last_became_peered":"2023-11-28T18:31:02.683995+0000","last_unstale":"2023-11-28T18:31:04.154436+0000","last_undegraded":"2023-11-28T18:31:04.154436+0000","last_fullsized":"2023-11-28T18:31:04.154436+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:35:13.466951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.160303+0000","last_change":"2023-11-28T18:31:03.169942+0000","last_active":"2023-11-28T18:31:04.160303+0000","last_peered":"2023-11-28T18:31:04.160303+0000","last_clean":"2023-11-28T18:31:04.160303+0000","last_became_active":"2023-11-28T18:31:03.168800+0000","last_became_peered":"2023-11-28T18:31:03.168800+0000","last_unstale":"2023-11-28T18:31:04.160303+0000","last_undegraded":"2023-11-28T18:31:04.160303+0000","last_fullsized":"2023-11-28T18:31:04.160303+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:51:06.589481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154317+0000","last_change":"2023-11-28T18:31:02.694377+0000","last_active":"2023-11-28T18:31:04.154317+0000","last_peered":"2023-11-28T18:31:04.154317+0000","last_clean":"2023-11-28T18:31:04.154317+0000","last_became_active":"2023-11-28T18:31:02.694098+0000","last_became_peered":"2023-11-28T18:31:02.694098+0000","last_unstale":"2023-11-28T18:31:04.154317+0000","last_undegraded":"2023-11-28T18:31:04.154317+0000","last_fullsized":"2023-11-28T18:31:04.154317+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:22.265264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157624+0000","last_change":"2023-11-28T18:31:03.172885+0000","last_active":"2023-11-28T18:31:04.157624+0000","last_peered":"2023-11-28T18:31:04.157624+0000","last_clean":"2023-11-28T18:31:04.157624+0000","last_became_active":"2023-11-28T18:31:03.172590+0000","last_became_peered":"2023-11-28T18:31:03.172590+0000","last_unstale":"2023-11-28T18:31:04.157624+0000","last_undegraded":"2023-11-28T18:31:04.157624+0000","last_fullsized":"2023-11-28T18:31:04.157624+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:56:36.786912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154448+0000","last_change":"2023-11-28T18:31:02.703039+0000","last_active":"2023-11-28T18:31:04.154448+0000","last_peered":"2023-11-28T18:31:04.154448+0000","last_clean":"2023-11-28T18:31:04.154448+0000","last_became_active":"2023-11-28T18:31:02.702767+0000","last_became_peered":"2023-11-28T18:31:02.702767+0000","last_unstale":"2023-11-28T18:31:04.154448+0000","last_undegraded":"2023-11-28T18:31:04.154448+0000","last_fullsized":"2023-11-28T18:31:04.154448+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:55:34.529650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.389479+0000","last_change":"2023-11-28T18:31:03.160992+0000","last_active":"2023-11-28T18:31:04.389479+0000","last_peered":"2023-11-28T18:31:04.389479+0000","last_clean":"2023-11-28T18:31:04.389479+0000","last_became_active":"2023-11-28T18:31:03.160743+0000","last_became_peered":"2023-11-28T18:31:03.160743+0000","last_unstale":"2023-11-28T18:31:04.389479+0000","last_undegraded":"2023-11-28T18:31:04.389479+0000","last_fullsized":"2023-11-28T18:31:04.389479+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:22.344176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154096+0000","last_change":"2023-11-28T18:31:02.685729+0000","last_active":"2023-11-28T18:31:04.154096+0000","last_peered":"2023-11-28T18:31:04.154096+0000","last_clean":"2023-11-28T18:31:04.154096+0000","last_became_active":"2023-11-28T18:31:02.685434+0000","last_became_peered":"2023-11-28T18:31:02.685434+0000","last_unstale":"2023-11-28T18:31:04.154096+0000","last_undegraded":"2023-11-28T18:31:04.154096+0000","last_fullsized":"2023-11-28T18:31:04.154096+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:07:04.525603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358130+0000","last_change":"2023-11-28T18:31:03.173704+0000","last_active":"2023-11-28T18:31:05.358130+0000","last_peered":"2023-11-28T18:31:05.358130+0000","last_clean":"2023-11-28T18:31:05.358130+0000","last_became_active":"2023-11-28T18:31:03.173331+0000","last_became_peered":"2023-11-28T18:31:03.173331+0000","last_unstale":"2023-11-28T18:31:05.358130+0000","last_undegraded":"2023-11-28T18:31:05.358130+0000","last_fullsized":"2023-11-28T18:31:05.358130+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:24.944277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"116'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157877+0000","last_change":"2023-11-28T18:31:02.706740+0000","last_active":"2023-11-28T18:31:04.157877+0000","last_peered":"2023-11-28T18:31:04.157877+0000","last_clean":"2023-11-28T18:31:04.157877+0000","last_became_active":"2023-11-28T18:31:02.706422+0000","last_became_peered":"2023-11-28T18:31:02.706422+0000","last_unstale":"2023-11-28T18:31:04.157877+0000","last_undegraded":"2023-11-28T18:31:04.157877+0000","last_fullsized":"2023-11-28T18:31:04.157877+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:24:21.484867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358102+0000","last_change":"2023-11-28T18:31:03.154396+0000","last_active":"2023-11-28T18:31:05.358102+0000","last_peered":"2023-11-28T18:31:05.358102+0000","last_clean":"2023-11-28T18:31:05.358102+0000","last_became_active":"2023-11-28T18:31:03.154096+0000","last_became_peered":"2023-11-28T18:31:03.154096+0000","last_unstale":"2023-11-28T18:31:05.358102+0000","last_undegraded":"2023-11-28T18:31:05.358102+0000","last_fullsized":"2023-11-28T18:31:05.358102+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:35:14.474983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.389602+0000","last_change":"2023-11-28T18:31:02.705096+0000","last_active":"2023-11-28T18:31:04.389602+0000","last_peered":"2023-11-28T18:31:04.389602+0000","last_clean":"2023-11-28T18:31:04.389602+0000","last_became_active":"2023-11-28T18:31:02.704703+0000","last_became_peered":"2023-11-28T18:31:02.704703+0000","last_unstale":"2023-11-28T18:31:04.389602+0000","last_undegraded":"2023-11-28T18:31:04.389602+0000","last_fullsized":"2023-11-28T18:31:04.389602+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:31.804335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356441+0000","last_change":"2023-11-28T18:31:03.173462+0000","last_active":"2023-11-28T18:31:05.356441+0000","last_peered":"2023-11-28T18:31:05.356441+0000","last_clean":"2023-11-28T18:31:05.356441+0000","last_became_active":"2023-11-28T18:31:03.173174+0000","last_became_peered":"2023-11-28T18:31:03.173174+0000","last_unstale":"2023-11-28T18:31:05.356441+0000","last_undegraded":"2023-11-28T18:31:05.356441+0000","last_fullsized":"2023-11-28T18:31:05.356441+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:33:07.941786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"116'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.131304+0000","last_change":"2023-11-28T18:31:02.700108+0000","last_active":"2023-11-28T18:31:05.131304+0000","last_peered":"2023-11-28T18:31:05.131304+0000","last_clean":"2023-11-28T18:31:05.131304+0000","last_became_active":"2023-11-28T18:31:02.699813+0000","last_became_peered":"2023-11-28T18:31:02.699813+0000","last_unstale":"2023-11-28T18:31:05.131304+0000","last_undegraded":"2023-11-28T18:31:05.131304+0000","last_fullsized":"2023-11-28T18:31:05.131304+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:09.283416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358567+0000","last_change":"2023-11-28T18:31:03.179578+0000","last_active":"2023-11-28T18:31:05.358567+0000","last_peered":"2023-11-28T18:31:05.358567+0000","last_clean":"2023-11-28T18:31:05.358567+0000","last_became_active":"2023-11-28T18:31:03.179043+0000","last_became_peered":"2023-11-28T18:31:03.179043+0000","last_unstale":"2023-11-28T18:31:05.358567+0000","last_undegraded":"2023-11-28T18:31:05.358567+0000","last_fullsized":"2023-11-28T18:31:05.358567+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:28:49.821579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.131084+0000","last_change":"2023-11-28T18:31:02.706160+0000","last_active":"2023-11-28T18:31:05.131084+0000","last_peered":"2023-11-28T18:31:05.131084+0000","last_clean":"2023-11-28T18:31:05.131084+0000","last_became_active":"2023-11-28T18:31:02.705836+0000","last_became_peered":"2023-11-28T18:31:02.705836+0000","last_unstale":"2023-11-28T18:31:05.131084+0000","last_undegraded":"2023-11-28T18:31:05.131084+0000","last_fullsized":"2023-11-28T18:31:05.131084+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:35:13.370840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.355989+0000","last_change":"2023-11-28T18:31:03.174068+0000","last_active":"2023-11-28T18:31:05.355989+0000","last_peered":"2023-11-28T18:31:05.355989+0000","last_clean":"2023-11-28T18:31:05.355989+0000","last_became_active":"2023-11-28T18:31:03.172701+0000","last_became_peered":"2023-11-28T18:31:03.172701+0000","last_unstale":"2023-11-28T18:31:05.355989+0000","last_undegraded":"2023-11-28T18:31:05.355989+0000","last_fullsized":"2023-11-28T18:31:05.355989+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:31:51.703770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356144+0000","last_change":"2023-11-28T18:31:02.706556+0000","last_active":"2023-11-28T18:31:05.356144+0000","last_peered":"2023-11-28T18:31:05.356144+0000","last_clean":"2023-11-28T18:31:05.356144+0000","last_became_active":"2023-11-28T18:31:02.706288+0000","last_became_peered":"2023-11-28T18:31:02.706288+0000","last_unstale":"2023-11-28T18:31:05.356144+0000","last_undegraded":"2023-11-28T18:31:05.356144+0000","last_fullsized":"2023-11-28T18:31:05.356144+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:06.509719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154008+0000","last_change":"2023-11-28T18:31:03.154716+0000","last_active":"2023-11-28T18:31:04.154008+0000","last_peered":"2023-11-28T18:31:04.154008+0000","last_clean":"2023-11-28T18:31:04.154008+0000","last_became_active":"2023-11-28T18:31:03.154419+0000","last_became_peered":"2023-11-28T18:31:03.154419+0000","last_unstale":"2023-11-28T18:31:04.154008+0000","last_undegraded":"2023-11-28T18:31:04.154008+0000","last_fullsized":"2023-11-28T18:31:04.154008+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:55.346235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.130718+0000","last_change":"2023-11-28T18:31:02.702584+0000","last_active":"2023-11-28T18:31:05.130718+0000","last_peered":"2023-11-28T18:31:05.130718+0000","last_clean":"2023-11-28T18:31:05.130718+0000","last_became_active":"2023-11-28T18:31:02.702054+0000","last_became_peered":"2023-11-28T18:31:02.702054+0000","last_unstale":"2023-11-28T18:31:05.130718+0000","last_undegraded":"2023-11-28T18:31:05.130718+0000","last_fullsized":"2023-11-28T18:31:05.130718+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:54.338203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154597+0000","last_change":"2023-11-28T18:31:03.170182+0000","last_active":"2023-11-28T18:31:04.154597+0000","last_peered":"2023-11-28T18:31:04.154597+0000","last_clean":"2023-11-28T18:31:04.154597+0000","last_became_active":"2023-11-28T18:31:03.169522+0000","last_became_peered":"2023-11-28T18:31:03.169522+0000","last_unstale":"2023-11-28T18:31:04.154597+0000","last_undegraded":"2023-11-28T18:31:04.154597+0000","last_fullsized":"2023-11-28T18:31:04.154597+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:41:36.177742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154194+0000","last_change":"2023-11-28T18:31:02.705758+0000","last_active":"2023-11-28T18:31:04.154194+0000","last_peered":"2023-11-28T18:31:04.154194+0000","last_clean":"2023-11-28T18:31:04.154194+0000","last_became_active":"2023-11-28T18:31:02.705435+0000","last_became_peered":"2023-11-28T18:31:02.705435+0000","last_unstale":"2023-11-28T18:31:04.154194+0000","last_undegraded":"2023-11-28T18:31:04.154194+0000","last_fullsized":"2023-11-28T18:31:04.154194+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:47.718941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358049+0000","last_change":"2023-11-28T18:31:03.167051+0000","last_active":"2023-11-28T18:31:05.358049+0000","last_peered":"2023-11-28T18:31:05.358049+0000","last_clean":"2023-11-28T18:31:05.358049+0000","last_became_active":"2023-11-28T18:31:03.166707+0000","last_became_peered":"2023-11-28T18:31:03.166707+0000","last_unstale":"2023-11-28T18:31:05.358049+0000","last_undegraded":"2023-11-28T18:31:05.358049+0000","last_fullsized":"2023-11-28T18:31:05.358049+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:55:35.537682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.130486+0000","last_change":"2023-11-28T18:31:02.694696+0000","last_active":"2023-11-28T18:31:05.130486+0000","last_peered":"2023-11-28T18:31:05.130486+0000","last_clean":"2023-11-28T18:31:05.130486+0000","last_became_active":"2023-11-28T18:31:02.694413+0000","last_became_peered":"2023-11-28T18:31:02.694413+0000","last_unstale":"2023-11-28T18:31:05.130486+0000","last_undegraded":"2023-11-28T18:31:05.130486+0000","last_fullsized":"2023-11-28T18:31:05.130486+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:55:31.745817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356547+0000","last_change":"2023-11-28T18:31:03.170901+0000","last_active":"2023-11-28T18:31:05.356547+0000","last_peered":"2023-11-28T18:31:05.356547+0000","last_clean":"2023-11-28T18:31:05.356547+0000","last_became_active":"2023-11-28T18:31:03.170600+0000","last_became_peered":"2023-11-28T18:31:03.170600+0000","last_unstale":"2023-11-28T18:31:05.356547+0000","last_undegraded":"2023-11-28T18:31:05.356547+0000","last_fullsized":"2023-11-28T18:31:05.356547+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:54:02.678471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356490+0000","last_change":"2023-11-28T18:31:02.702698+0000","last_active":"2023-11-28T18:31:05.356490+0000","last_peered":"2023-11-28T18:31:05.356490+0000","last_clean":"2023-11-28T18:31:05.356490+0000","last_became_active":"2023-11-28T18:31:02.702423+0000","last_became_peered":"2023-11-28T18:31:02.702423+0000","last_unstale":"2023-11-28T18:31:05.356490+0000","last_undegraded":"2023-11-28T18:31:05.356490+0000","last_fullsized":"2023-11-28T18:31:05.356490+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:17:56.982900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157674+0000","last_change":"2023-11-28T18:31:03.174085+0000","last_active":"2023-11-28T18:31:04.157674+0000","last_peered":"2023-11-28T18:31:04.157674+0000","last_clean":"2023-11-28T18:31:04.157674+0000","last_became_active":"2023-11-28T18:31:03.173581+0000","last_became_peered":"2023-11-28T18:31:03.173581+0000","last_unstale":"2023-11-28T18:31:04.157674+0000","last_undegraded":"2023-11-28T18:31:04.157674+0000","last_fullsized":"2023-11-28T18:31:04.157674+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:48.726973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.160527+0000","last_change":"2023-11-28T18:31:02.708037+0000","last_active":"2023-11-28T18:31:04.160527+0000","last_peered":"2023-11-28T18:31:04.160527+0000","last_clean":"2023-11-28T18:31:04.160527+0000","last_became_active":"2023-11-28T18:31:02.707577+0000","last_became_peered":"2023-11-28T18:31:02.707577+0000","last_unstale":"2023-11-28T18:31:04.160527+0000","last_undegraded":"2023-11-28T18:31:04.160527+0000","last_fullsized":"2023-11-28T18:31:04.160527+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:28:10.333699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.131218+0000","last_change":"2023-11-28T18:31:03.178585+0000","last_active":"2023-11-28T18:31:05.131218+0000","last_peered":"2023-11-28T18:31:05.131218+0000","last_clean":"2023-11-28T18:31:05.131218+0000","last_became_active":"2023-11-28T18:31:03.177218+0000","last_became_peered":"2023-11-28T18:31:03.177218+0000","last_unstale":"2023-11-28T18:31:05.131218+0000","last_undegraded":"2023-11-28T18:31:05.131218+0000","last_fullsized":"2023-11-28T18:31:05.131218+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:24:58.809372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.160602+0000","last_change":"2023-11-28T18:31:02.708035+0000","last_active":"2023-11-28T18:31:04.160602+0000","last_peered":"2023-11-28T18:31:04.160602+0000","last_clean":"2023-11-28T18:31:04.160602+0000","last_became_active":"2023-11-28T18:31:02.707715+0000","last_became_peered":"2023-11-28T18:31:02.707715+0000","last_unstale":"2023-11-28T18:31:04.160602+0000","last_undegraded":"2023-11-28T18:31:04.160602+0000","last_fullsized":"2023-11-28T18:31:04.160602+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:01:11.184059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.389655+0000","last_change":"2023-11-28T18:31:03.167260+0000","last_active":"2023-11-28T18:31:04.389655+0000","last_peered":"2023-11-28T18:31:04.389655+0000","last_clean":"2023-11-28T18:31:04.389655+0000","last_became_active":"2023-11-28T18:31:03.166321+0000","last_became_peered":"2023-11-28T18:31:03.166321+0000","last_unstale":"2023-11-28T18:31:04.389655+0000","last_undegraded":"2023-11-28T18:31:04.389655+0000","last_fullsized":"2023-11-28T18:31:04.389655+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:45.585392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356568+0000","last_change":"2023-11-28T18:31:02.707644+0000","last_active":"2023-11-28T18:31:05.356568+0000","last_peered":"2023-11-28T18:31:05.356568+0000","last_clean":"2023-11-28T18:31:05.356568+0000","last_became_active":"2023-11-28T18:31:02.707392+0000","last_became_peered":"2023-11-28T18:31:02.707392+0000","last_unstale":"2023-11-28T18:31:05.356568+0000","last_undegraded":"2023-11-28T18:31:05.356568+0000","last_fullsized":"2023-11-28T18:31:05.356568+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:05.933002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356268+0000","last_change":"2023-11-28T18:31:03.164309+0000","last_active":"2023-11-28T18:31:05.356268+0000","last_peered":"2023-11-28T18:31:05.356268+0000","last_clean":"2023-11-28T18:31:05.356268+0000","last_became_active":"2023-11-28T18:31:03.163966+0000","last_became_peered":"2023-11-28T18:31:03.163966+0000","last_unstale":"2023-11-28T18:31:05.356268+0000","last_undegraded":"2023-11-28T18:31:05.356268+0000","last_fullsized":"2023-11-28T18:31:05.356268+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:06:57.073736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.357755+0000","last_change":"2023-11-28T18:31:02.691320+0000","last_active":"2023-11-28T18:31:05.357755+0000","last_peered":"2023-11-28T18:31:05.357755+0000","last_clean":"2023-11-28T18:31:05.357755+0000","last_became_active":"2023-11-28T18:31:02.691018+0000","last_became_peered":"2023-11-28T18:31:02.691018+0000","last_unstale":"2023-11-28T18:31:05.357755+0000","last_undegraded":"2023-11-28T18:31:05.357755+0000","last_fullsized":"2023-11-28T18:31:05.357755+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:01:11.184059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154091+0000","last_change":"2023-11-28T18:31:03.166813+0000","last_active":"2023-11-28T18:31:04.154091+0000","last_peered":"2023-11-28T18:31:04.154091+0000","last_clean":"2023-11-28T18:31:04.154091+0000","last_became_active":"2023-11-28T18:31:03.166039+0000","last_became_peered":"2023-11-28T18:31:03.166039+0000","last_unstale":"2023-11-28T18:31:04.154091+0000","last_undegraded":"2023-11-28T18:31:04.154091+0000","last_fullsized":"2023-11-28T18:31:04.154091+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:44:23.208354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.357458+0000","last_change":"2023-11-28T18:31:02.703970+0000","last_active":"2023-11-28T18:31:05.357458+0000","last_peered":"2023-11-28T18:31:05.357458+0000","last_clean":"2023-11-28T18:31:05.357458+0000","last_became_active":"2023-11-28T18:31:02.703679+0000","last_became_peered":"2023-11-28T18:31:02.703679+0000","last_unstale":"2023-11-28T18:31:05.357458+0000","last_undegraded":"2023-11-28T18:31:05.357458+0000","last_fullsized":"2023-11-28T18:31:05.357458+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:57:11.690622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.130736+0000","last_change":"2023-11-28T18:31:03.174732+0000","last_active":"2023-11-28T18:31:05.130736+0000","last_peered":"2023-11-28T18:31:05.130736+0000","last_clean":"2023-11-28T18:31:05.130736+0000","last_became_active":"2023-11-28T18:31:03.174463+0000","last_became_peered":"2023-11-28T18:31:03.174463+0000","last_unstale":"2023-11-28T18:31:05.130736+0000","last_undegraded":"2023-11-28T18:31:05.130736+0000","last_fullsized":"2023-11-28T18:31:05.130736+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:07:46.210330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356115+0000","last_change":"2023-11-28T18:31:02.703636+0000","last_active":"2023-11-28T18:31:05.356115+0000","last_peered":"2023-11-28T18:31:05.356115+0000","last_clean":"2023-11-28T18:31:05.356115+0000","last_became_active":"2023-11-28T18:31:02.703302+0000","last_became_peered":"2023-11-28T18:31:02.703302+0000","last_unstale":"2023-11-28T18:31:05.356115+0000","last_undegraded":"2023-11-28T18:31:05.356115+0000","last_fullsized":"2023-11-28T18:31:05.356115+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:49:43.542415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157885+0000","last_change":"2023-11-28T18:31:03.166640+0000","last_active":"2023-11-28T18:31:04.157885+0000","last_peered":"2023-11-28T18:31:04.157885+0000","last_clean":"2023-11-28T18:31:04.157885+0000","last_became_active":"2023-11-28T18:31:03.166389+0000","last_became_peered":"2023-11-28T18:31:03.166389+0000","last_unstale":"2023-11-28T18:31:04.157885+0000","last_undegraded":"2023-11-28T18:31:04.157885+0000","last_fullsized":"2023-11-28T18:31:04.157885+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:26:09.301219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.130530+0000","last_change":"2023-11-28T18:31:02.695278+0000","last_active":"2023-11-28T18:31:05.130530+0000","last_peered":"2023-11-28T18:31:05.130530+0000","last_clean":"2023-11-28T18:31:05.130530+0000","last_became_active":"2023-11-28T18:31:02.694979+0000","last_became_peered":"2023-11-28T18:31:02.694979+0000","last_unstale":"2023-11-28T18:31:05.130530+0000","last_undegraded":"2023-11-28T18:31:05.130530+0000","last_fullsized":"2023-11-28T18:31:05.130530+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:55:34.529650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.131189+0000","last_change":"2023-11-28T18:31:03.178449+0000","last_active":"2023-11-28T18:31:05.131189+0000","last_peered":"2023-11-28T18:31:05.131189+0000","last_clean":"2023-11-28T18:31:05.131189+0000","last_became_active":"2023-11-28T18:31:03.178172+0000","last_became_peered":"2023-11-28T18:31:03.178172+0000","last_unstale":"2023-11-28T18:31:05.131189+0000","last_undegraded":"2023-11-28T18:31:05.131189+0000","last_fullsized":"2023-11-28T18:31:05.131189+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:23:13.307954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"116'8","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154020+0000","last_change":"2023-11-28T18:31:02.695116+0000","last_active":"2023-11-28T18:31:04.154020+0000","last_peered":"2023-11-28T18:31:04.154020+0000","last_clean":"2023-11-28T18:31:04.154020+0000","last_became_active":"2023-11-28T18:31:02.694855+0000","last_became_peered":"2023-11-28T18:31:02.694855+0000","last_unstale":"2023-11-28T18:31:04.154020+0000","last_undegraded":"2023-11-28T18:31:04.154020+0000","last_fullsized":"2023-11-28T18:31:04.154020+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:24:21.484867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":248,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-11-28T18:31:03.166133+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:31:03.166133+0000","last_peered":"2023-11-28T18:31:03.166133+0000","last_clean":"2023-11-28T18:31:03.166133+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:31:03.166133+0000","last_undegraded":"2023-11-28T18:31:03.166133+0000","last_fullsized":"2023-11-28T18:31:03.166133+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574945,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51248,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320592,"statfs":{"total":96636764160,"available":96584286208,"internally_reserved":0,"allocated":2068480,"data_stored":1269894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63700000000000001,"15min":0.70099999999999996},"min":{"1min":0.32600000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.96999999999999997,"5min":0.98099999999999998,"15min":2.427},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.64200000000000002,"15min":0.70199999999999996},"min":{"1min":0.40999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.82799999999999996,"5min":0.98199999999999998,"15min":2.2509999999999999},"last":1.0449999999999999}]},{"osd":1,"last update":"Tue Nov 28 18:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66700000000000004,"15min":0.70699999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.34699999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":2.3849999999999998},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.65800000000000003,"15min":0.73199999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.006,"5min":1.079,"15min":2.3130000000000002},"last":0.82899999999999996}]},{"osd":2,"last update":"Tue Nov 28 18:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67800000000000005,"15min":0.752},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.91500000000000004,"5min":0.998,"15min":2.4590000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.67400000000000004,"15min":0.72699999999999998},"min":{"1min":0.499,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82199999999999995,"5min":1.022,"15min":2.411},"last":0.999}]},{"osd":3,"last update":"Tue Nov 28 18:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65900000000000003,"15min":0.73899999999999999},"min":{"1min":0.45700000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80800000000000005,"5min":0.92000000000000004,"15min":2.2730000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71099999999999997,"15min":0.78300000000000003},"min":{"1min":0.48699999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.873,"5min":1.0389999999999999,"15min":2.3650000000000002},"last":0.98399999999999999}]},{"osd":4,"last update":"Tue Nov 28 18:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59099999999999997,"15min":0.60299999999999998},"min":{"1min":0.40100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86299999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.63200000000000001},"min":{"1min":0.38600000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.91800000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.749}]},{"osd":5,"last update":"Tue Nov 28 18:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.628,"15min":0.66700000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85599999999999998,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.64500000000000002,"15min":0.68999999999999995},"min":{"1min":0.40000000000000002,"5min":0.39900000000000002,"15min":0.39300000000000002},"max":{"1min":0.85099999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.875}]},{"osd":6,"last update":"Tue Nov 28 18:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70199999999999996,"15min":0.73499999999999999},"min":{"1min":0.54500000000000004,"5min":0.375,"15min":0.375},"max":{"1min":0.97699999999999998,"5min":1.131,"15min":1.131},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63600000000000001,"15min":0.71199999999999997},"min":{"1min":0.44700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88900000000000001,"5min":1.081,"15min":1.081},"last":1.02}]}]},{"osd":6,"up_from":13,"seq":55834574944,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72868,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298972,"statfs":{"total":96636764160,"available":96562147328,"internally_reserved":0,"allocated":1462272,"data_stored":679332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66800000000000004,"15min":0.64500000000000002},"min":{"1min":0.44800000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.94499999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":1.331},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71399999999999997,"15min":0.72599999999999998},"min":{"1min":0.40699999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.88700000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":1.673}]},{"osd":1,"last update":"Tue Nov 28 18:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65600000000000003,"15min":0.77100000000000002},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90800000000000003,"5min":0.96899999999999997,"15min":4.2389999999999999},"last":1.2809999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68200000000000005,"15min":0.76700000000000002},"min":{"1min":0.44500000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.82599999999999996,"5min":0.94499999999999995,"15min":4.218},"last":0.81000000000000005}]},{"osd":2,"last update":"Tue Nov 28 18:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66100000000000003,"15min":0.70099999999999996},"min":{"1min":0.39300000000000002,"5min":0.373,"15min":0.36299999999999999},"max":{"1min":0.90500000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":2.7599999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.68300000000000005,"15min":0.70099999999999996},"min":{"1min":0.59699999999999998,"5min":0.42999999999999999,"15min":0.40100000000000002},"max":{"1min":0.90500000000000003,"5min":1.012,"15min":1.012},"last":1.7849999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69999999999999996,"15min":0.72199999999999998},"min":{"1min":0.46300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.96499999999999997,"5min":1.016,"15min":1.016},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67500000000000004,"15min":0.68500000000000005},"min":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.1100000000000001},"last":2.8090000000000002}]},{"osd":4,"last update":"Tue Nov 28 18:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60299999999999998,"15min":0.65500000000000003},"min":{"1min":0.44500000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83599999999999997,"5min":1.02,"15min":1.02},"last":1.2310000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.621,"15min":0.67300000000000004},"min":{"1min":0.45000000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.91100000000000003,"5min":1.038,"15min":1.038},"last":2.1640000000000001}]},{"osd":5,"last update":"Tue Nov 28 18:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.622,"15min":0.66800000000000004},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.877,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":3.0110000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.74099999999999999},"min":{"1min":0.44700000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":1.0449999999999999},"last":1.3740000000000001}]},{"osd":7,"last update":"Tue Nov 28 18:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68500000000000005,"15min":0.751},"min":{"1min":0.48699999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88200000000000001,"5min":1.002,"15min":1.0269999999999999},"last":1.722},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.68200000000000005,"15min":0.73499999999999999},"min":{"1min":0.42799999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":2.8490000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36776,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335064,"statfs":{"total":96636764160,"available":96599105536,"internally_reserved":0,"allocated":1466368,"data_stored":678910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.68500000000000005,"15min":0.70399999999999996},"min":{"1min":0.36099999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.90600000000000003,"5min":2.1299999999999999,"15min":2.8929999999999998},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.71299999999999997,"15min":0.74299999999999999},"min":{"1min":0.434,"5min":0.39200000000000002,"15min":0.33900000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":2.9140000000000001},"last":1.0169999999999999}]},{"osd":1,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68100000000000005,"15min":0.72799999999999998},"min":{"1min":0.505,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":3.4809999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.77900000000000003},"min":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.39100000000000001},"max":{"1min":0.96299999999999997,"5min":2.2109999999999999,"15min":3.4580000000000002},"last":0.93100000000000005}]},{"osd":2,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.66100000000000003,"15min":0.73499999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":3.9510000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66700000000000004,"15min":0.75},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86299999999999999,"5min":1.014,"15min":4.0149999999999997},"last":0.91300000000000003}]},{"osd":3,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70999999999999996,"15min":0.79800000000000004},"min":{"1min":0.44700000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1,"5min":2.254,"15min":4.0869999999999997},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68999999999999995,"15min":0.79100000000000004},"min":{"1min":0.46400000000000002,"5min":0.46300000000000002,"15min":0.41299999999999998},"max":{"1min":0.89800000000000002,"5min":0.98599999999999999,"15min":3.9900000000000002},"last":0.874}]},{"osd":4,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.33500000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79300000000000004,"5min":1.696,"15min":1.696},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.66700000000000004,"15min":0.69999999999999996},"min":{"1min":0.45900000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.90800000000000003,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.58499999999999996}]},{"osd":6,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65600000000000003,"15min":0.67400000000000004},"min":{"1min":0.42899999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0600000000000001,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68899999999999995,"15min":0.70999999999999996},"min":{"1min":0.48299999999999998,"5min":0.438,"15min":0.42399999999999999},"max":{"1min":0.89200000000000002,"5min":1.262,"15min":1.756},"last":0.68899999999999995}]},{"osd":7,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72499999999999998,"15min":0.745},"min":{"1min":0.50700000000000001,"5min":0.42399999999999999,"15min":0.41699999999999998},"max":{"1min":0.91400000000000003,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72799999999999998,"15min":0.748},"min":{"1min":0.502,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1259999999999999,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.71899999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574945,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72876,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298964,"statfs":{"total":96636764160,"available":96562139136,"internally_reserved":0,"allocated":1470464,"data_stored":680362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.64200000000000002,"15min":0.67600000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.315},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":2.363},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.39200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.91700000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.99099999999999999}]},{"osd":1,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.64200000000000002,"15min":0.628},"min":{"1min":0.41399999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0629999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68500000000000005,"15min":0.66000000000000003},"min":{"1min":0.47199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.98899999999999999,"5min":1.02,"15min":1.02},"last":0.81200000000000006}]},{"osd":2,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.71799999999999997,"15min":0.70099999999999996},"min":{"1min":0.48499999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96699999999999997,"5min":1.056,"15min":1.056},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72599999999999998,"15min":0.72899999999999998},"min":{"1min":0.54800000000000004,"5min":0.434,"15min":0.434},"max":{"1min":1.024,"5min":1.091,"15min":1.871},"last":0.90900000000000003}]},{"osd":3,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.65300000000000002},"min":{"1min":0.50600000000000001,"5min":0.45700000000000002,"15min":0.40899999999999997},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68899999999999995,"15min":0.68500000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.91700000000000004,"5min":1.014,"15min":1.014},"last":0.83499999999999996}]},{"osd":5,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60099999999999998,"15min":0.58999999999999997},"min":{"1min":0.33100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.82599999999999996,"5min":1.03,"15min":1.1799999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60199999999999998,"15min":0.629},"min":{"1min":0.39600000000000002,"5min":0.33400000000000002,"15min":0.32300000000000001},"max":{"1min":0.86899999999999999,"5min":1.05,"15min":1.79},"last":0.77700000000000002}]},{"osd":6,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.61299999999999999,"15min":0.63900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.91400000000000003,"5min":1.0680000000000001,"15min":1.123},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.68999999999999995,"15min":0.70399999999999996},"min":{"1min":0.41799999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92000000000000004,"5min":1.1970000000000001,"15min":1.925},"last":0.747}]},{"osd":7,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68000000000000005,"15min":0.70799999999999996},"min":{"1min":0.45900000000000002,"5min":0.41999999999999998,"15min":0.36399999999999999},"max":{"1min":0.93200000000000005,"5min":1.1040000000000001,"15min":1.843},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.749,"15min":0.77100000000000002},"min":{"1min":0.47799999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94399999999999995,"5min":1.1299999999999999,"15min":1.593},"last":1.0149999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574944,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46516,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325324,"statfs":{"total":96636764160,"available":96589131776,"internally_reserved":0,"allocated":1474560,"data_stored":680396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.439,"15min":0.41999999999999998},"min":{"1min":0.28399999999999997,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.57799999999999996,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.5,"15min":0.47599999999999998},"min":{"1min":0.33100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.65000000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.40200000000000002}]},{"osd":1,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45300000000000001,"15min":0.45900000000000002},"min":{"1min":0.32000000000000001,"5min":0.28799999999999998,"15min":0.27300000000000002},"max":{"1min":0.67000000000000004,"5min":0.73399999999999999,"15min":0.96099999999999997},"last":0.498},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55400000000000005,"15min":0.51700000000000002},"min":{"1min":0.375,"5min":0.249,"15min":0.247},"max":{"1min":0.72599999999999998,"5min":4.5599999999999996,"15min":4.5599999999999996},"last":0.51500000000000001}]},{"osd":2,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55400000000000005,"15min":0.52900000000000003},"min":{"1min":0.377,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.97899999999999998,"5min":2.48,"15min":2.48},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.439,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81399999999999995,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.47899999999999998}]},{"osd":4,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.89600000000000002,"15min":0.81200000000000006},"min":{"1min":0.50700000000000001,"5min":0.441,"15min":0.39800000000000002},"max":{"1min":1.1870000000000001,"5min":12.695,"15min":12.695},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.80100000000000005,"15min":0.78000000000000003},"min":{"1min":0.47899999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0349999999999999,"5min":6.3250000000000002,"15min":6.3250000000000002},"last":0.89100000000000001}]},{"osd":5,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76800000000000002,"15min":0.81499999999999995},"min":{"1min":0.55200000000000005,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94099999999999995,"5min":1.9430000000000001,"15min":2.5019999999999998},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.80400000000000005,"15min":0.78100000000000003},"min":{"1min":0.58699999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0269999999999999,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.109}]},{"osd":6,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.82899999999999996,"15min":0.80800000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1439999999999999,"5min":6.1790000000000003,"15min":6.1790000000000003},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.52200000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0009999999999999,"5min":1.984,"15min":1.984},"last":0.94199999999999995}]},{"osd":7,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.80700000000000005,"15min":0.81000000000000005},"min":{"1min":0.54400000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.069,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.83999999999999997,"15min":0.81799999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0640000000000001,"5min":6.2249999999999996,"15min":6.2249999999999996},"last":1.0349999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574945,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36772,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335068,"statfs":{"total":96636764160,"available":96599109632,"internally_reserved":0,"allocated":1458176,"data_stored":678750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.437,"15min":0.48499999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.56299999999999994,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.45100000000000001,"15min":0.48999999999999999},"min":{"1min":0.26800000000000002,"5min":0.26700000000000002,"15min":0.26500000000000001},"max":{"1min":0.66300000000000003,"5min":0.66800000000000004,"15min":2.0150000000000001},"last":0.47099999999999997}]},{"osd":1,"last update":"Tue Nov 28 18:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.47299999999999998,"15min":0.51200000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.63200000000000001,"5min":0.70999999999999996,"15min":1.48},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.56999999999999995,"15min":0.53200000000000003},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.56599999999999995,"5min":8.3539999999999992,"15min":8.3539999999999992},"last":0.51700000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.48799999999999999,"15min":0.499},"min":{"1min":0.34799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.56799999999999995,"5min":3.1549999999999998,"15min":3.1549999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.499,"5min":0.53800000000000003,"15min":0.56200000000000006},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66500000000000004,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":0.53500000000000003}]},{"osd":4,"last update":"Tue Nov 28 18:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.74199999999999999,"15min":0.76500000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.04,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.749,"15min":0.76900000000000002},"min":{"1min":0.45700000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0600000000000001,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.68999999999999995}]},{"osd":5,"last update":"Tue Nov 28 18:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.749,"15min":0.76200000000000001},"min":{"1min":0.47399999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1040000000000001,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72799999999999998,"15min":0.92600000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97199999999999998,"5min":1.0780000000000001,"15min":7.4820000000000002},"last":0.68000000000000005}]},{"osd":6,"last update":"Tue Nov 28 18:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.84699999999999998,"15min":0.81499999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.129,"5min":12.019,"15min":12.019},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81299999999999994,"15min":0.85399999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0349999999999999,"5min":8.7400000000000002,"15min":8.7400000000000002},"last":0.71899999999999997}]},{"osd":7,"last update":"Tue Nov 28 18:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91700000000000004,"5min":8.327,"15min":8.327},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.81899999999999995,"15min":0.89200000000000002},"min":{"1min":0.504,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1479999999999999,"5min":8.609,"15min":8.609},"last":0.628}]}]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40936,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330904,"statfs":{"total":96636764160,"available":96594845696,"internally_reserved":0,"allocated":1462272,"data_stored":678772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.41499999999999998,"15min":0.42299999999999999},"min":{"1min":0.248,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58299999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.47399999999999998,"15min":0.45800000000000002},"min":{"1min":0.29599999999999999,"5min":0.27300000000000002,"15min":0.26600000000000001},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.36899999999999999}]},{"osd":2,"last update":"Tue Nov 28 18:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.44600000000000001,"15min":0.44900000000000001},"min":{"1min":0.32300000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.63100000000000001,"5min":0.69499999999999995,"15min":0.75700000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.65100000000000002,"5min":0.73399999999999999,"15min":0.79200000000000004},"last":0.44}]},{"osd":3,"last update":"Tue Nov 28 18:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54000000000000004,"15min":0.53900000000000003},"min":{"1min":0.39400000000000002,"5min":0.376,"15min":0.36699999999999999},"max":{"1min":0.71599999999999997,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.505},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51100000000000001,"15min":0.496},"min":{"1min":0.371,"5min":0.33600000000000002,"15min":0.32200000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.49199999999999999}]},{"osd":4,"last update":"Tue Nov 28 18:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.70999999999999996,"15min":0.73599999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91000000000000003,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.747,"15min":0.76300000000000001},"min":{"1min":0.52400000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.81799999999999995,"5min":1.0780000000000001,"15min":1.155},"last":0.73099999999999998}]},{"osd":5,"last update":"Tue Nov 28 18:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.69499999999999995,"15min":0.73399999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92500000000000004,"5min":0.997,"15min":1.3799999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70099999999999996,"15min":0.73599999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.98999999999999999,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.66000000000000003}]},{"osd":6,"last update":"Tue Nov 28 18:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.73599999999999999,"15min":0.76600000000000001},"min":{"1min":0.48599999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.83599999999999997,"5min":1.1279999999999999,"15min":1.2969999999999999},"last":0.753},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.75700000000000001,"15min":0.79300000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.95499999999999996,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.71299999999999997}]},{"osd":7,"last update":"Tue Nov 28 18:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.72699999999999998,"15min":0.746},"min":{"1min":0.503,"5min":0.503,"15min":0.47899999999999998},"max":{"1min":0.90600000000000003,"5min":1.0669999999999999,"15min":1.2410000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73499999999999999,"15min":0.76500000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.48499999999999999},"max":{"1min":0.96999999999999997,"5min":1.0109999999999999,"15min":1.4239999999999999},"last":0.67900000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574944,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47076,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324764,"statfs":{"total":96636764160,"available":96588558336,"internally_reserved":0,"allocated":2056192,"data_stored":1269140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 18:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41699999999999998,"15min":0.435},"min":{"1min":0.252,"5min":0.248,"15min":0.248},"max":{"1min":0.61199999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.441},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47699999999999998,"15min":0.47199999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.69199999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.45500000000000002}]},{"osd":2,"last update":"Tue Nov 28 18:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48399999999999999,"15min":0.502},"min":{"1min":0.33000000000000002,"5min":0.318,"15min":0.30499999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.92900000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.48899999999999999,"15min":0.50700000000000001},"min":{"1min":0.316,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65000000000000002,"5min":0.77800000000000002,"15min":0.91200000000000003},"last":0.48999999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47599999999999998,"15min":0.48599999999999999},"min":{"1min":0.373,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65500000000000003,"5min":0.78600000000000003,"15min":0.86499999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53700000000000003,"15min":0.54800000000000004},"min":{"1min":0.38800000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77200000000000002,"5min":0.80000000000000004,"15min":1.087},"last":0.39300000000000002}]},{"osd":4,"last update":"Tue Nov 28 18:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71999999999999997,"15min":0.71499999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.97699999999999998,"5min":1.131,"15min":1.131},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71499999999999997,"15min":0.71699999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.95099999999999996,"5min":0.996,"15min":1.244},"last":0.65100000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71899999999999997,"15min":0.72499999999999998},"min":{"1min":0.46600000000000003,"5min":0.436,"15min":0.436},"max":{"1min":0.91900000000000004,"5min":1.0489999999999999,"15min":1.121},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71599999999999997,"15min":0.73399999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.94099999999999995,"5min":1.0329999999999999,"15min":1.075},"last":0.66500000000000004}]},{"osd":6,"last update":"Tue Nov 28 18:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73299999999999998,"15min":0.73199999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.97599999999999998,"5min":1.1140000000000001,"15min":1.407},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.73099999999999998,"15min":0.73399999999999999},"min":{"1min":0.46800000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88,"5min":1.087,"15min":1.089},"last":0.61499999999999999}]},{"osd":7,"last update":"Tue Nov 28 18:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.749,"15min":0.752},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.91600000000000004,"5min":1.054,"15min":1.212},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.73399999999999999,"15min":0.74399999999999999},"min":{"1min":0.54100000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.89900000000000002,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.625}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:31:07.770 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:31:07.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.941+0000 7f593a755640 1 Processor -- start 2023-11-28T18:31:07.943 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.945+0000 7f593a755640 1 -- start start 2023-11-28T18:31:07.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.945+0000 7f593a755640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5934159610 0x7f59340abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.945+0000 7f593a755640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f59340ac2c0 0x7f59340ac730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.945+0000 7f593a755640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59340cde30 con 0x7f59340ce2c0 2023-11-28T18:31:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.945+0000 7f593a755640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59340ce6a0 con 0x7f59340ac2c0 2023-11-28T18:31:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.945+0000 7f593a755640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f593406de20 con 0x7f5934159610 2023-11-28T18:31:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.945+0000 7f5938cf2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f59340ac2c0 0x7f59340ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.945+0000 7f5938cf2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f59340ac2c0 0x7f59340ac730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38266/0 (socket says 172.21.15.105:38266) 2023-11-28T18:31:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.945+0000 7f5938cf2640 1 -- 172.21.15.105:0/1454191919 learned_addr learned my addr 172.21.15.105:0/1454191919 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.945+0000 7f59337fe640 1 --2- 172.21.15.105:0/1454191919 >> v2:172.21.15.148:3300/0 conn(0x7f5934159610 0x7f59340abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:07.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.945+0000 7f5938cf2640 1 -- 172.21.15.105:0/1454191919 >> v2:172.21.15.148:3300/0 conn(0x7f5934159610 msgr2=0x7f59340abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:07.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.945+0000 7f5938cf2640 1 --2- 172.21.15.105:0/1454191919 >> v2:172.21.15.148:3300/0 conn(0x7f5934159610 0x7f59340abd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.945+0000 7f5938cf2640 1 -- 172.21.15.105:0/1454191919 >> v1:172.21.15.148:6789/0 conn(0x7f59340ce2c0 legacy=0x7f59340cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:07.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.945+0000 7f5938cf2640 1 -- 172.21.15.105:0/1454191919 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59340cd220 con 0x7f59340ac2c0 2023-11-28T18:31:07.947 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.945+0000 7f59337fe640 1 --2- 172.21.15.105:0/1454191919 >> v2:172.21.15.148:3300/0 conn(0x7f5934159610 0x7f59340abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:07.948 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.949+0000 7f5938cf2640 1 --2- 172.21.15.105:0/1454191919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59340ac2c0 0x7f59340ac730 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f59340cfc90 tx=0x7f592c01ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6779fe425660f776 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.949+0000 7f5932ffd640 1 -- 172.21.15.105:0/1454191919 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c00b2d0 con 0x7f59340ac2c0 2023-11-28T18:31:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.949+0000 7f5932ffd640 1 -- 172.21.15.105:0/1454191919 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592c00b430 con 0x7f59340ac2c0 2023-11-28T18:31:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.949+0000 7f5932ffd640 1 -- 172.21.15.105:0/1454191919 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c0a7ab0 con 0x7f59340ac2c0 2023-11-28T18:31:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.949+0000 7f593a755640 1 -- 172.21.15.105:0/1454191919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59340ac2c0 msgr2=0x7f59340ac730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.949+0000 7f593a755640 1 --2- 172.21.15.105:0/1454191919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59340ac2c0 0x7f59340ac730 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f59340cfc90 tx=0x7f592c01ebd0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.949+0000 7f593a755640 1 -- 172.21.15.105:0/1454191919 shutdown_connections 2023-11-28T18:31:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.949+0000 7f593a755640 1 --2- 172.21.15.105:0/1454191919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59340ac2c0 0x7f59340ac730 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.949+0000 7f593a755640 1 --2- 172.21.15.105:0/1454191919 >> v2:172.21.15.148:3300/0 conn(0x7f5934159610 0x7f59340abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.949+0000 7f593a755640 1 -- 172.21.15.105:0/1454191919 >> 172.21.15.105:0/1454191919 conn(0x7f59340ceb90 msgr2=0x7f59340cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.949+0000 7f593a755640 1 -- 172.21.15.105:0/1454191919 shutdown_connections 2023-11-28T18:31:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.949+0000 7f593a755640 1 -- 172.21.15.105:0/1454191919 wait complete. 2023-11-28T18:31:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.949+0000 7f593a755640 1 Processor -- start 2023-11-28T18:31:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f593a755640 1 -- start start 2023-11-28T18:31:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f593a755640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59340ac2c0 0x7f5934154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f593a755640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59340ce2c0 0x7f59340b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f593a755640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5934159610 0x7f59340b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f593a755640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593415a040 con 0x7f59340ac2c0 2023-11-28T18:31:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f593a755640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593415a320 con 0x7f5934159610 2023-11-28T18:31:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f593a755640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593415a600 con 0x7f59340ce2c0 2023-11-28T18:31:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f5933fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59340ac2c0 0x7f5934154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f5933fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59340ac2c0 0x7f5934154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38272/0 (socket says 172.21.15.105:38272) 2023-11-28T18:31:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f5933fff640 1 -- 172.21.15.105:0/2052514908 learned_addr learned my addr 172.21.15.105:0/2052514908 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f5938cf2640 1 --2- 172.21.15.105:0/2052514908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5934159610 0x7f59340b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f5933fff640 1 -- 172.21.15.105:0/2052514908 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59340ce2c0 msgr2=0x7f59340b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f5933fff640 1 --2- 172.21.15.105:0/2052514908 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59340ce2c0 0x7f59340b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f5933fff640 1 -- 172.21.15.105:0/2052514908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5934159610 msgr2=0x7f59340b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f5933fff640 1 --2- 172.21.15.105:0/2052514908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5934159610 0x7f59340b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f5933fff640 1 -- 172.21.15.105:0/2052514908 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592c00bd40 con 0x7f59340ac2c0 2023-11-28T18:31:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f5938cf2640 1 --2- 172.21.15.105:0/2052514908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5934159610 0x7f59340b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:31:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f5933fff640 1 --2- 172.21.15.105:0/2052514908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59340ac2c0 0x7f5934154a20 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f592400c500 tx=0x7f592400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:07.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f5930ff9640 1 -- 172.21.15.105:0/2052514908 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5924005220 con 0x7f59340ac2c0 2023-11-28T18:31:07.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f5930ff9640 1 -- 172.21.15.105:0/2052514908 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5924005380 con 0x7f59340ac2c0 2023-11-28T18:31:07.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f5930ff9640 1 -- 172.21.15.105:0/2052514908 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5924007270 con 0x7f59340ac2c0 2023-11-28T18:31:07.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f593a755640 1 -- 172.21.15.105:0/2052514908 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59340ccff0 con 0x7f59340ac2c0 2023-11-28T18:31:07.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f593a755640 1 -- 172.21.15.105:0/2052514908 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f593415abc0 con 0x7f59340ac2c0 2023-11-28T18:31:07.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.953+0000 7f593a755640 1 -- 172.21.15.105:0/2052514908 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5934067570 con 0x7f59340ac2c0 2023-11-28T18:31:07.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.957+0000 7f5930ff9640 1 -- 172.21.15.105:0/2052514908 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f59240073d0 con 0x7f59340ac2c0 2023-11-28T18:31:07.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.957+0000 7f5930ff9640 1 --2- 172.21.15.105:0/2052514908 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59200668d0 0x7f5920068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:07.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.957+0000 7f5930ff9640 1 -- 172.21.15.105:0/2052514908 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5924083ef0 con 0x7f59340ac2c0 2023-11-28T18:31:07.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.957+0000 7f59337fe640 1 --2- 172.21.15.105:0/2052514908 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59200668d0 0x7f5920068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:07.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.957+0000 7f59337fe640 1 --2- 172.21.15.105:0/2052514908 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59200668d0 0x7f5920068d90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5928006f10 tx=0x7f5928008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:07.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:07.961+0000 7f5930ff9640 1 -- 172.21.15.105:0/2052514908 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5924050f60 con 0x7f59340ac2c0 2023-11-28T18:31:08.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.105+0000 7f593a755640 1 -- 172.21.15.105:0/2052514908 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f59340cce70 con 0x7f59200668d0 2023-11-28T18:31:08.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.113+0000 7f5930ff9640 1 -- 172.21.15.105:0/2052514908 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289447 (secure 0 0 0) 0x7f592800be40 con 0x7f59200668d0 2023-11-28T18:31:08.117 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:31:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.117+0000 7f593a755640 1 -- 172.21.15.105:0/2052514908 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59200668d0 msgr2=0x7f5920068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.117+0000 7f593a755640 1 --2- 172.21.15.105:0/2052514908 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59200668d0 0x7f5920068d90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5928006f10 tx=0x7f5928008040 comp rx=0 tx=0).stop 2023-11-28T18:31:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.117+0000 7f593a755640 1 -- 172.21.15.105:0/2052514908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59340ac2c0 msgr2=0x7f5934154a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.117+0000 7f593a755640 1 --2- 172.21.15.105:0/2052514908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59340ac2c0 0x7f5934154a20 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f592400c500 tx=0x7f592400c9c0 comp rx=0 tx=0).stop 2023-11-28T18:31:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.117+0000 7f593a755640 1 -- 172.21.15.105:0/2052514908 shutdown_connections 2023-11-28T18:31:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.117+0000 7f593a755640 1 --2- 172.21.15.105:0/2052514908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5934159610 0x7f59340b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.117+0000 7f593a755640 1 --2- 172.21.15.105:0/2052514908 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59340ce2c0 0x7f59340b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.117+0000 7f593a755640 1 --2- 172.21.15.105:0/2052514908 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f59200668d0 0x7f5920068d90 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.117+0000 7f593a755640 1 --2- 172.21.15.105:0/2052514908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59340ac2c0 0x7f5934154a20 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.117+0000 7f593a755640 1 -- 172.21.15.105:0/2052514908 >> 172.21.15.105:0/2052514908 conn(0x7f59340ceb90 msgr2=0x7f59340b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.117+0000 7f593a755640 1 -- 172.21.15.105:0/2052514908 shutdown_connections 2023-11-28T18:31:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.117+0000 7f593a755640 1 -- 172.21.15.105:0/2052514908 wait complete. 2023-11-28T18:31:08.137 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2023-11-28T18:31:07.366768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":405068,"kb_used_data":12616,"kb_used_omap":12,"kb_used_meta":392307,"kb_avail":754569652,"statfs":{"total":773094113280,"available":772679323648,"internally_reserved":0,"allocated":12918784,"data_stored":6615556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":401722930},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.700357"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358229+0000","last_change":"2023-11-28T18:31:03.165164+0000","last_active":"2023-11-28T18:31:05.358229+0000","last_peered":"2023-11-28T18:31:05.358229+0000","last_clean":"2023-11-28T18:31:05.358229+0000","last_became_active":"2023-11-28T18:31:03.164853+0000","last_became_peered":"2023-11-28T18:31:03.164853+0000","last_unstale":"2023-11-28T18:31:05.358229+0000","last_undegraded":"2023-11-28T18:31:05.358229+0000","last_fullsized":"2023-11-28T18:31:05.358229+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:26:09.301219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154547+0000","last_change":"2023-11-28T18:31:03.176466+0000","last_active":"2023-11-28T18:31:04.154547+0000","last_peered":"2023-11-28T18:31:04.154547+0000","last_clean":"2023-11-28T18:31:04.154547+0000","last_became_active":"2023-11-28T18:31:03.176150+0000","last_became_peered":"2023-11-28T18:31:03.176150+0000","last_unstale":"2023-11-28T18:31:04.154547+0000","last_undegraded":"2023-11-28T18:31:04.154547+0000","last_fullsized":"2023-11-28T18:31:04.154547+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:49:08.879513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154621+0000","last_change":"2023-11-28T18:31:03.171440+0000","last_active":"2023-11-28T18:31:04.154621+0000","last_peered":"2023-11-28T18:31:04.154621+0000","last_clean":"2023-11-28T18:31:04.154621+0000","last_became_active":"2023-11-28T18:31:03.171184+0000","last_became_peered":"2023-11-28T18:31:03.171184+0000","last_unstale":"2023-11-28T18:31:04.154621+0000","last_undegraded":"2023-11-28T18:31:04.154621+0000","last_fullsized":"2023-11-28T18:31:04.154621+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:14:45.424633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.130936+0000","last_change":"2023-11-28T18:31:03.176828+0000","last_active":"2023-11-28T18:31:05.130936+0000","last_peered":"2023-11-28T18:31:05.130936+0000","last_clean":"2023-11-28T18:31:05.130936+0000","last_became_active":"2023-11-28T18:31:03.176561+0000","last_became_peered":"2023-11-28T18:31:03.176561+0000","last_unstale":"2023-11-28T18:31:05.130936+0000","last_undegraded":"2023-11-28T18:31:05.130936+0000","last_fullsized":"2023-11-28T18:31:05.130936+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:44:54.950658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.357388+0000","last_change":"2023-11-28T18:31:03.166770+0000","last_active":"2023-11-28T18:31:05.357388+0000","last_peered":"2023-11-28T18:31:05.357388+0000","last_clean":"2023-11-28T18:31:05.357388+0000","last_became_active":"2023-11-28T18:31:03.165977+0000","last_became_peered":"2023-11-28T18:31:03.165977+0000","last_unstale":"2023-11-28T18:31:05.357388+0000","last_undegraded":"2023-11-28T18:31:05.357388+0000","last_fullsized":"2023-11-28T18:31:05.357388+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:48.726973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356047+0000","last_change":"2023-11-28T18:31:03.171056+0000","last_active":"2023-11-28T18:31:05.356047+0000","last_peered":"2023-11-28T18:31:05.356047+0000","last_clean":"2023-11-28T18:31:05.356047+0000","last_became_active":"2023-11-28T18:31:03.170642+0000","last_became_peered":"2023-11-28T18:31:03.170642+0000","last_unstale":"2023-11-28T18:31:05.356047+0000","last_undegraded":"2023-11-28T18:31:05.356047+0000","last_fullsized":"2023-11-28T18:31:05.356047+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:32:04.317161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356044+0000","last_change":"2023-11-28T18:31:03.174604+0000","last_active":"2023-11-28T18:31:05.356044+0000","last_peered":"2023-11-28T18:31:05.356044+0000","last_clean":"2023-11-28T18:31:05.356044+0000","last_became_active":"2023-11-28T18:31:03.171130+0000","last_became_peered":"2023-11-28T18:31:03.171130+0000","last_unstale":"2023-11-28T18:31:05.356044+0000","last_undegraded":"2023-11-28T18:31:05.356044+0000","last_fullsized":"2023-11-28T18:31:05.356044+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:13:45.614265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.153997+0000","last_change":"2023-11-28T18:31:03.169122+0000","last_active":"2023-11-28T18:31:04.153997+0000","last_peered":"2023-11-28T18:31:04.153997+0000","last_clean":"2023-11-28T18:31:04.153997+0000","last_became_active":"2023-11-28T18:31:03.168849+0000","last_became_peered":"2023-11-28T18:31:03.168849+0000","last_unstale":"2023-11-28T18:31:04.153997+0000","last_undegraded":"2023-11-28T18:31:04.153997+0000","last_fullsized":"2023-11-28T18:31:04.153997+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:35:14.378872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.131126+0000","last_change":"2023-11-28T18:31:03.178668+0000","last_active":"2023-11-28T18:31:05.131126+0000","last_peered":"2023-11-28T18:31:05.131126+0000","last_clean":"2023-11-28T18:31:05.131126+0000","last_became_active":"2023-11-28T18:31:03.178386+0000","last_became_peered":"2023-11-28T18:31:03.178386+0000","last_unstale":"2023-11-28T18:31:05.131126+0000","last_undegraded":"2023-11-28T18:31:05.131126+0000","last_fullsized":"2023-11-28T18:31:05.131126+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:04:54.885976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157778+0000","last_change":"2023-11-28T18:31:03.173327+0000","last_active":"2023-11-28T18:31:04.157778+0000","last_peered":"2023-11-28T18:31:04.157778+0000","last_clean":"2023-11-28T18:31:04.157778+0000","last_became_active":"2023-11-28T18:31:03.173048+0000","last_became_peered":"2023-11-28T18:31:03.173048+0000","last_unstale":"2023-11-28T18:31:04.157778+0000","last_undegraded":"2023-11-28T18:31:04.157778+0000","last_fullsized":"2023-11-28T18:31:04.157778+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:23.273296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358016+0000","last_change":"2023-11-28T18:31:03.159540+0000","last_active":"2023-11-28T18:31:05.358016+0000","last_peered":"2023-11-28T18:31:05.358016+0000","last_clean":"2023-11-28T18:31:05.358016+0000","last_became_active":"2023-11-28T18:31:03.159198+0000","last_became_peered":"2023-11-28T18:31:03.159198+0000","last_unstale":"2023-11-28T18:31:05.358016+0000","last_undegraded":"2023-11-28T18:31:05.358016+0000","last_fullsized":"2023-11-28T18:31:05.358016+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:23.273296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356362+0000","last_change":"2023-11-28T18:31:03.172083+0000","last_active":"2023-11-28T18:31:05.356362+0000","last_peered":"2023-11-28T18:31:05.356362+0000","last_clean":"2023-11-28T18:31:05.356362+0000","last_became_active":"2023-11-28T18:31:03.171134+0000","last_became_peered":"2023-11-28T18:31:03.171134+0000","last_unstale":"2023-11-28T18:31:05.356362+0000","last_undegraded":"2023-11-28T18:31:05.356362+0000","last_fullsized":"2023-11-28T18:31:05.356362+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:58:39.627376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157615+0000","last_change":"2023-11-28T18:31:03.163845+0000","last_active":"2023-11-28T18:31:04.157615+0000","last_peered":"2023-11-28T18:31:04.157615+0000","last_clean":"2023-11-28T18:31:04.157615+0000","last_became_active":"2023-11-28T18:31:03.163564+0000","last_became_peered":"2023-11-28T18:31:03.163564+0000","last_unstale":"2023-11-28T18:31:04.157615+0000","last_undegraded":"2023-11-28T18:31:04.157615+0000","last_fullsized":"2023-11-28T18:31:04.157615+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:46:07.850898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154205+0000","last_change":"2023-11-28T18:31:03.165835+0000","last_active":"2023-11-28T18:31:04.154205+0000","last_peered":"2023-11-28T18:31:04.154205+0000","last_clean":"2023-11-28T18:31:04.154205+0000","last_became_active":"2023-11-28T18:31:03.165516+0000","last_became_peered":"2023-11-28T18:31:03.165516+0000","last_unstale":"2023-11-28T18:31:04.154205+0000","last_undegraded":"2023-11-28T18:31:04.154205+0000","last_fullsized":"2023-11-28T18:31:04.154205+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:42:10.190912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.357221+0000","last_change":"2023-11-28T18:31:03.161357+0000","last_active":"2023-11-28T18:31:05.357221+0000","last_peered":"2023-11-28T18:31:05.357221+0000","last_clean":"2023-11-28T18:31:05.357221+0000","last_became_active":"2023-11-28T18:31:03.160135+0000","last_became_peered":"2023-11-28T18:31:03.160135+0000","last_unstale":"2023-11-28T18:31:05.357221+0000","last_undegraded":"2023-11-28T18:31:05.357221+0000","last_fullsized":"2023-11-28T18:31:05.357221+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:56:36.786912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.355990+0000","last_change":"2023-11-28T18:31:03.166047+0000","last_active":"2023-11-28T18:31:05.355990+0000","last_peered":"2023-11-28T18:31:05.355990+0000","last_clean":"2023-11-28T18:31:05.355990+0000","last_became_active":"2023-11-28T18:31:03.165744+0000","last_became_peered":"2023-11-28T18:31:03.165744+0000","last_unstale":"2023-11-28T18:31:05.355990+0000","last_undegraded":"2023-11-28T18:31:05.355990+0000","last_fullsized":"2023-11-28T18:31:05.355990+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:03:34.675681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157890+0000","last_change":"2023-11-28T18:31:03.167959+0000","last_active":"2023-11-28T18:31:04.157890+0000","last_peered":"2023-11-28T18:31:04.157890+0000","last_clean":"2023-11-28T18:31:04.157890+0000","last_became_active":"2023-11-28T18:31:03.167523+0000","last_became_peered":"2023-11-28T18:31:03.167523+0000","last_unstale":"2023-11-28T18:31:04.157890+0000","last_undegraded":"2023-11-28T18:31:04.157890+0000","last_fullsized":"2023-11-28T18:31:04.157890+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:22:08.164188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154611+0000","last_change":"2023-11-28T18:31:03.174970+0000","last_active":"2023-11-28T18:31:04.154611+0000","last_peered":"2023-11-28T18:31:04.154611+0000","last_clean":"2023-11-28T18:31:04.154611+0000","last_became_active":"2023-11-28T18:31:03.174695+0000","last_became_peered":"2023-11-28T18:31:03.174695+0000","last_unstale":"2023-11-28T18:31:04.154611+0000","last_undegraded":"2023-11-28T18:31:04.154611+0000","last_fullsized":"2023-11-28T18:31:04.154611+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:15:27.520824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.160514+0000","last_change":"2023-11-28T18:31:03.172486+0000","last_active":"2023-11-28T18:31:04.160514+0000","last_peered":"2023-11-28T18:31:04.160514+0000","last_clean":"2023-11-28T18:31:04.160514+0000","last_became_active":"2023-11-28T18:31:03.172212+0000","last_became_peered":"2023-11-28T18:31:03.172212+0000","last_unstale":"2023-11-28T18:31:04.160514+0000","last_undegraded":"2023-11-28T18:31:04.160514+0000","last_fullsized":"2023-11-28T18:31:04.160514+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:56:36.786912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.389471+0000","last_change":"2023-11-28T18:31:03.173970+0000","last_active":"2023-11-28T18:31:04.389471+0000","last_peered":"2023-11-28T18:31:04.389471+0000","last_clean":"2023-11-28T18:31:04.389471+0000","last_became_active":"2023-11-28T18:31:03.173642+0000","last_became_peered":"2023-11-28T18:31:03.173642+0000","last_unstale":"2023-11-28T18:31:04.389471+0000","last_undegraded":"2023-11-28T18:31:04.389471+0000","last_fullsized":"2023-11-28T18:31:04.389471+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:35:35.646951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.130899+0000","last_change":"2023-11-28T18:31:03.177854+0000","last_active":"2023-11-28T18:31:05.130899+0000","last_peered":"2023-11-28T18:31:05.130899+0000","last_clean":"2023-11-28T18:31:05.130899+0000","last_became_active":"2023-11-28T18:31:03.176607+0000","last_became_peered":"2023-11-28T18:31:03.176607+0000","last_unstale":"2023-11-28T18:31:05.130899+0000","last_undegraded":"2023-11-28T18:31:05.130899+0000","last_fullsized":"2023-11-28T18:31:05.130899+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:04.428310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.357489+0000","last_change":"2023-11-28T18:31:03.154435+0000","last_active":"2023-11-28T18:31:05.357489+0000","last_peered":"2023-11-28T18:31:05.357489+0000","last_clean":"2023-11-28T18:31:05.357489+0000","last_became_active":"2023-11-28T18:31:03.154116+0000","last_became_peered":"2023-11-28T18:31:03.154116+0000","last_unstale":"2023-11-28T18:31:05.357489+0000","last_undegraded":"2023-11-28T18:31:05.357489+0000","last_fullsized":"2023-11-28T18:31:05.357489+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:07:05.533635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.131169+0000","last_change":"2023-11-28T18:31:03.177952+0000","last_active":"2023-11-28T18:31:05.131169+0000","last_peered":"2023-11-28T18:31:05.131169+0000","last_clean":"2023-11-28T18:31:05.131169+0000","last_became_active":"2023-11-28T18:31:03.177631+0000","last_became_peered":"2023-11-28T18:31:03.177631+0000","last_unstale":"2023-11-28T18:31:05.131169+0000","last_undegraded":"2023-11-28T18:31:05.131169+0000","last_fullsized":"2023-11-28T18:31:05.131169+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:08:24.255143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.357619+0000","last_change":"2023-11-28T18:31:03.170656+0000","last_active":"2023-11-28T18:31:05.357619+0000","last_peered":"2023-11-28T18:31:05.357619+0000","last_clean":"2023-11-28T18:31:05.357619+0000","last_became_active":"2023-11-28T18:31:03.170150+0000","last_became_peered":"2023-11-28T18:31:03.170150+0000","last_unstale":"2023-11-28T18:31:05.357619+0000","last_undegraded":"2023-11-28T18:31:05.357619+0000","last_fullsized":"2023-11-28T18:31:05.357619+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:48:09.007039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358493+0000","last_change":"2023-11-28T18:31:03.155722+0000","last_active":"2023-11-28T18:31:05.358493+0000","last_peered":"2023-11-28T18:31:05.358493+0000","last_clean":"2023-11-28T18:31:05.358493+0000","last_became_active":"2023-11-28T18:31:03.155394+0000","last_became_peered":"2023-11-28T18:31:03.155394+0000","last_unstale":"2023-11-28T18:31:05.358493+0000","last_undegraded":"2023-11-28T18:31:05.358493+0000","last_fullsized":"2023-11-28T18:31:05.358493+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:42:10.190912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.389622+0000","last_change":"2023-11-28T18:31:03.164633+0000","last_active":"2023-11-28T18:31:04.389622+0000","last_peered":"2023-11-28T18:31:04.389622+0000","last_clean":"2023-11-28T18:31:04.389622+0000","last_became_active":"2023-11-28T18:31:03.164372+0000","last_became_peered":"2023-11-28T18:31:03.164372+0000","last_unstale":"2023-11-28T18:31:04.389622+0000","last_undegraded":"2023-11-28T18:31:04.389622+0000","last_fullsized":"2023-11-28T18:31:04.389622+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:02:01.309143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.160634+0000","last_change":"2023-11-28T18:31:03.172868+0000","last_active":"2023-11-28T18:31:04.160634+0000","last_peered":"2023-11-28T18:31:04.160634+0000","last_clean":"2023-11-28T18:31:04.160634+0000","last_became_active":"2023-11-28T18:31:03.172575+0000","last_became_peered":"2023-11-28T18:31:03.172575+0000","last_unstale":"2023-11-28T18:31:04.160634+0000","last_undegraded":"2023-11-28T18:31:04.160634+0000","last_fullsized":"2023-11-28T18:31:04.160634+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:48.726973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356367+0000","last_change":"2023-11-28T18:31:03.172288+0000","last_active":"2023-11-28T18:31:05.356367+0000","last_peered":"2023-11-28T18:31:05.356367+0000","last_clean":"2023-11-28T18:31:05.356367+0000","last_became_active":"2023-11-28T18:31:03.171687+0000","last_became_peered":"2023-11-28T18:31:03.171687+0000","last_unstale":"2023-11-28T18:31:05.356367+0000","last_undegraded":"2023-11-28T18:31:05.356367+0000","last_fullsized":"2023-11-28T18:31:05.356367+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:55:40.593795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.389484+0000","last_change":"2023-11-28T18:31:03.161400+0000","last_active":"2023-11-28T18:31:04.389484+0000","last_peered":"2023-11-28T18:31:04.389484+0000","last_clean":"2023-11-28T18:31:04.389484+0000","last_became_active":"2023-11-28T18:31:03.160718+0000","last_became_peered":"2023-11-28T18:31:03.160718+0000","last_unstale":"2023-11-28T18:31:04.389484+0000","last_undegraded":"2023-11-28T18:31:04.389484+0000","last_fullsized":"2023-11-28T18:31:04.389484+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:44:26.342490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154282+0000","last_change":"2023-11-28T18:31:03.157973+0000","last_active":"2023-11-28T18:31:04.154282+0000","last_peered":"2023-11-28T18:31:04.154282+0000","last_clean":"2023-11-28T18:31:04.154282+0000","last_became_active":"2023-11-28T18:31:03.157690+0000","last_became_peered":"2023-11-28T18:31:03.157690+0000","last_unstale":"2023-11-28T18:31:04.154282+0000","last_undegraded":"2023-11-28T18:31:04.154282+0000","last_fullsized":"2023-11-28T18:31:04.154282+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:28:49.821579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.357822+0000","last_change":"2023-11-28T18:31:03.163986+0000","last_active":"2023-11-28T18:31:05.357822+0000","last_peered":"2023-11-28T18:31:05.357822+0000","last_clean":"2023-11-28T18:31:05.357822+0000","last_became_active":"2023-11-28T18:31:03.163600+0000","last_became_peered":"2023-11-28T18:31:03.163600+0000","last_unstale":"2023-11-28T18:31:05.357822+0000","last_undegraded":"2023-11-28T18:31:05.357822+0000","last_fullsized":"2023-11-28T18:31:05.357822+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:55:32.753849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.357622+0000","last_change":"2023-11-28T18:31:03.170675+0000","last_active":"2023-11-28T18:31:05.357622+0000","last_peered":"2023-11-28T18:31:05.357622+0000","last_clean":"2023-11-28T18:31:05.357622+0000","last_became_active":"2023-11-28T18:31:03.169960+0000","last_became_peered":"2023-11-28T18:31:03.169960+0000","last_unstale":"2023-11-28T18:31:05.357622+0000","last_undegraded":"2023-11-28T18:31:05.357622+0000","last_fullsized":"2023-11-28T18:31:05.357622+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:08:18.381736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157837+0000","last_change":"2023-11-28T18:31:03.175773+0000","last_active":"2023-11-28T18:31:04.157837+0000","last_peered":"2023-11-28T18:31:04.157837+0000","last_clean":"2023-11-28T18:31:04.157837+0000","last_became_active":"2023-11-28T18:31:03.175492+0000","last_became_peered":"2023-11-28T18:31:03.175492+0000","last_unstale":"2023-11-28T18:31:04.157837+0000","last_undegraded":"2023-11-28T18:31:04.157837+0000","last_fullsized":"2023-11-28T18:31:04.157837+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:35:14.474983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"116'4","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.160552+0000","last_change":"2023-11-28T18:31:02.695616+0000","last_active":"2023-11-28T18:31:04.160552+0000","last_peered":"2023-11-28T18:31:04.160552+0000","last_clean":"2023-11-28T18:31:04.160552+0000","last_became_active":"2023-11-28T18:31:02.695272+0000","last_became_peered":"2023-11-28T18:31:02.695272+0000","last_unstale":"2023-11-28T18:31:04.160552+0000","last_undegraded":"2023-11-28T18:31:04.160552+0000","last_fullsized":"2023-11-28T18:31:04.160552+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:24:21.484867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356560+0000","last_change":"2023-11-28T18:31:03.172403+0000","last_active":"2023-11-28T18:31:05.356560+0000","last_peered":"2023-11-28T18:31:05.356560+0000","last_clean":"2023-11-28T18:31:05.356560+0000","last_became_active":"2023-11-28T18:31:03.170743+0000","last_became_peered":"2023-11-28T18:31:03.170743+0000","last_unstale":"2023-11-28T18:31:05.356560+0000","last_undegraded":"2023-11-28T18:31:05.356560+0000","last_fullsized":"2023-11-28T18:31:05.356560+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:36:23.211383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"116'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157883+0000","last_change":"2023-11-28T18:31:02.695849+0000","last_active":"2023-11-28T18:31:04.157883+0000","last_peered":"2023-11-28T18:31:04.157883+0000","last_clean":"2023-11-28T18:31:04.157883+0000","last_became_active":"2023-11-28T18:31:02.695517+0000","last_became_peered":"2023-11-28T18:31:02.695517+0000","last_unstale":"2023-11-28T18:31:04.157883+0000","last_undegraded":"2023-11-28T18:31:04.157883+0000","last_fullsized":"2023-11-28T18:31:04.157883+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:01:11.184059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358241+0000","last_change":"2023-11-28T18:31:03.174369+0000","last_active":"2023-11-28T18:31:05.358241+0000","last_peered":"2023-11-28T18:31:05.358241+0000","last_clean":"2023-11-28T18:31:05.358241+0000","last_became_active":"2023-11-28T18:31:03.174053+0000","last_became_peered":"2023-11-28T18:31:03.174053+0000","last_unstale":"2023-11-28T18:31:05.358241+0000","last_undegraded":"2023-11-28T18:31:05.358241+0000","last_fullsized":"2023-11-28T18:31:05.358241+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:35:14.378872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.131207+0000","last_change":"2023-11-28T18:31:02.700671+0000","last_active":"2023-11-28T18:31:05.131207+0000","last_peered":"2023-11-28T18:31:05.131207+0000","last_clean":"2023-11-28T18:31:05.131207+0000","last_became_active":"2023-11-28T18:31:02.700392+0000","last_became_peered":"2023-11-28T18:31:02.700392+0000","last_unstale":"2023-11-28T18:31:05.131207+0000","last_undegraded":"2023-11-28T18:31:05.131207+0000","last_fullsized":"2023-11-28T18:31:05.131207+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:28:48.813547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.131242+0000","last_change":"2023-11-28T18:31:03.175584+0000","last_active":"2023-11-28T18:31:05.131242+0000","last_peered":"2023-11-28T18:31:05.131242+0000","last_clean":"2023-11-28T18:31:05.131242+0000","last_became_active":"2023-11-28T18:31:03.175322+0000","last_became_peered":"2023-11-28T18:31:03.175322+0000","last_unstale":"2023-11-28T18:31:05.131242+0000","last_undegraded":"2023-11-28T18:31:05.131242+0000","last_fullsized":"2023-11-28T18:31:05.131242+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:23:29.439976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157922+0000","last_change":"2023-11-28T18:31:02.699895+0000","last_active":"2023-11-28T18:31:04.157922+0000","last_peered":"2023-11-28T18:31:04.157922+0000","last_clean":"2023-11-28T18:31:04.157922+0000","last_became_active":"2023-11-28T18:31:02.699627+0000","last_became_peered":"2023-11-28T18:31:02.699627+0000","last_unstale":"2023-11-28T18:31:04.157922+0000","last_undegraded":"2023-11-28T18:31:04.157922+0000","last_fullsized":"2023-11-28T18:31:04.157922+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:30:38.565540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157572+0000","last_change":"2023-11-28T18:31:03.171333+0000","last_active":"2023-11-28T18:31:04.157572+0000","last_peered":"2023-11-28T18:31:04.157572+0000","last_clean":"2023-11-28T18:31:04.157572+0000","last_became_active":"2023-11-28T18:31:03.171028+0000","last_became_peered":"2023-11-28T18:31:03.171028+0000","last_unstale":"2023-11-28T18:31:04.157572+0000","last_undegraded":"2023-11-28T18:31:04.157572+0000","last_fullsized":"2023-11-28T18:31:04.157572+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:51:06.589481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"116'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.357399+0000","last_change":"2023-11-28T18:31:02.702240+0000","last_active":"2023-11-28T18:31:05.357399+0000","last_peered":"2023-11-28T18:31:05.357399+0000","last_clean":"2023-11-28T18:31:05.357399+0000","last_became_active":"2023-11-28T18:31:02.701944+0000","last_became_peered":"2023-11-28T18:31:02.701944+0000","last_unstale":"2023-11-28T18:31:05.357399+0000","last_undegraded":"2023-11-28T18:31:05.357399+0000","last_fullsized":"2023-11-28T18:31:05.357399+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:30:38.565540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.357198+0000","last_change":"2023-11-28T18:31:03.166490+0000","last_active":"2023-11-28T18:31:05.357198+0000","last_peered":"2023-11-28T18:31:05.357198+0000","last_clean":"2023-11-28T18:31:05.357198+0000","last_became_active":"2023-11-28T18:31:03.166205+0000","last_became_peered":"2023-11-28T18:31:03.166205+0000","last_unstale":"2023-11-28T18:31:05.357198+0000","last_undegraded":"2023-11-28T18:31:05.357198+0000","last_fullsized":"2023-11-28T18:31:05.357198+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:46:07.850898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"116'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356092+0000","last_change":"2023-11-28T18:31:02.702744+0000","last_active":"2023-11-28T18:31:05.356092+0000","last_peered":"2023-11-28T18:31:05.356092+0000","last_clean":"2023-11-28T18:31:05.356092+0000","last_became_active":"2023-11-28T18:31:02.702458+0000","last_became_peered":"2023-11-28T18:31:02.702458+0000","last_unstale":"2023-11-28T18:31:05.356092+0000","last_undegraded":"2023-11-28T18:31:05.356092+0000","last_fullsized":"2023-11-28T18:31:05.356092+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:39:06.258713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154330+0000","last_change":"2023-11-28T18:31:03.162176+0000","last_active":"2023-11-28T18:31:04.154330+0000","last_peered":"2023-11-28T18:31:04.154330+0000","last_clean":"2023-11-28T18:31:04.154330+0000","last_became_active":"2023-11-28T18:31:03.161868+0000","last_became_peered":"2023-11-28T18:31:03.161868+0000","last_unstale":"2023-11-28T18:31:04.154330+0000","last_undegraded":"2023-11-28T18:31:04.154330+0000","last_fullsized":"2023-11-28T18:31:04.154330+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:10.291448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.357772+0000","last_change":"2023-11-28T18:31:02.704113+0000","last_active":"2023-11-28T18:31:05.357772+0000","last_peered":"2023-11-28T18:31:05.357772+0000","last_clean":"2023-11-28T18:31:05.357772+0000","last_became_active":"2023-11-28T18:31:02.703873+0000","last_became_peered":"2023-11-28T18:31:02.703873+0000","last_unstale":"2023-11-28T18:31:05.357772+0000","last_undegraded":"2023-11-28T18:31:05.357772+0000","last_fullsized":"2023-11-28T18:31:05.357772+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:56:35.778880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157759+0000","last_change":"2023-11-28T18:31:03.161690+0000","last_active":"2023-11-28T18:31:04.157759+0000","last_peered":"2023-11-28T18:31:04.157759+0000","last_clean":"2023-11-28T18:31:04.157759+0000","last_became_active":"2023-11-28T18:31:03.161405+0000","last_became_peered":"2023-11-28T18:31:03.161405+0000","last_unstale":"2023-11-28T18:31:04.157759+0000","last_undegraded":"2023-11-28T18:31:04.157759+0000","last_fullsized":"2023-11-28T18:31:04.157759+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:07:05.533635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356147+0000","last_change":"2023-11-28T18:31:02.704407+0000","last_active":"2023-11-28T18:31:05.356147+0000","last_peered":"2023-11-28T18:31:05.356147+0000","last_clean":"2023-11-28T18:31:05.356147+0000","last_became_active":"2023-11-28T18:31:02.704098+0000","last_became_peered":"2023-11-28T18:31:02.704098+0000","last_unstale":"2023-11-28T18:31:05.356147+0000","last_undegraded":"2023-11-28T18:31:05.356147+0000","last_fullsized":"2023-11-28T18:31:05.356147+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:05:18.687677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.160336+0000","last_change":"2023-11-28T18:31:03.169360+0000","last_active":"2023-11-28T18:31:04.160336+0000","last_peered":"2023-11-28T18:31:04.160336+0000","last_clean":"2023-11-28T18:31:04.160336+0000","last_became_active":"2023-11-28T18:31:03.168937+0000","last_became_peered":"2023-11-28T18:31:03.168937+0000","last_unstale":"2023-11-28T18:31:04.160336+0000","last_undegraded":"2023-11-28T18:31:04.160336+0000","last_fullsized":"2023-11-28T18:31:04.160336+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:46:07.850898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.160539+0000","last_change":"2023-11-28T18:31:02.710256+0000","last_active":"2023-11-28T18:31:04.160539+0000","last_peered":"2023-11-28T18:31:04.160539+0000","last_clean":"2023-11-28T18:31:04.160539+0000","last_became_active":"2023-11-28T18:31:02.709881+0000","last_became_peered":"2023-11-28T18:31:02.709881+0000","last_unstale":"2023-11-28T18:31:04.160539+0000","last_undegraded":"2023-11-28T18:31:04.160539+0000","last_fullsized":"2023-11-28T18:31:04.160539+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:20.015329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358304+0000","last_change":"2023-11-28T18:31:03.155535+0000","last_active":"2023-11-28T18:31:05.358304+0000","last_peered":"2023-11-28T18:31:05.358304+0000","last_clean":"2023-11-28T18:31:05.358304+0000","last_became_active":"2023-11-28T18:31:03.155215+0000","last_became_peered":"2023-11-28T18:31:03.155215+0000","last_unstale":"2023-11-28T18:31:05.358304+0000","last_undegraded":"2023-11-28T18:31:05.358304+0000","last_fullsized":"2023-11-28T18:31:05.358304+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:55.346235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358206+0000","last_change":"2023-11-28T18:31:02.709245+0000","last_active":"2023-11-28T18:31:05.358206+0000","last_peered":"2023-11-28T18:31:05.358206+0000","last_clean":"2023-11-28T18:31:05.358206+0000","last_became_active":"2023-11-28T18:31:02.708549+0000","last_became_peered":"2023-11-28T18:31:02.708549+0000","last_unstale":"2023-11-28T18:31:05.358206+0000","last_undegraded":"2023-11-28T18:31:05.358206+0000","last_fullsized":"2023-11-28T18:31:05.358206+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:41:30.696017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154661+0000","last_change":"2023-11-28T18:31:03.165827+0000","last_active":"2023-11-28T18:31:04.154661+0000","last_peered":"2023-11-28T18:31:04.154661+0000","last_clean":"2023-11-28T18:31:04.154661+0000","last_became_active":"2023-11-28T18:31:03.165485+0000","last_became_peered":"2023-11-28T18:31:03.165485+0000","last_unstale":"2023-11-28T18:31:04.154661+0000","last_undegraded":"2023-11-28T18:31:04.154661+0000","last_fullsized":"2023-11-28T18:31:04.154661+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:07:46.210330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"116'3","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358407+0000","last_change":"2023-11-28T18:31:02.701153+0000","last_active":"2023-11-28T18:31:05.358407+0000","last_peered":"2023-11-28T18:31:05.358407+0000","last_clean":"2023-11-28T18:31:05.358407+0000","last_became_active":"2023-11-28T18:31:02.700500+0000","last_became_peered":"2023-11-28T18:31:02.700500+0000","last_unstale":"2023-11-28T18:31:05.358407+0000","last_undegraded":"2023-11-28T18:31:05.358407+0000","last_fullsized":"2023-11-28T18:31:05.358407+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:35:13.466951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358482+0000","last_change":"2023-11-28T18:31:03.166517+0000","last_active":"2023-11-28T18:31:05.358482+0000","last_peered":"2023-11-28T18:31:05.358482+0000","last_clean":"2023-11-28T18:31:05.358482+0000","last_became_active":"2023-11-28T18:31:03.166189+0000","last_became_peered":"2023-11-28T18:31:03.166189+0000","last_unstale":"2023-11-28T18:31:05.358482+0000","last_undegraded":"2023-11-28T18:31:05.358482+0000","last_fullsized":"2023-11-28T18:31:05.358482+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:44:23.208354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"116'6","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154436+0000","last_change":"2023-11-28T18:31:02.684346+0000","last_active":"2023-11-28T18:31:04.154436+0000","last_peered":"2023-11-28T18:31:04.154436+0000","last_clean":"2023-11-28T18:31:04.154436+0000","last_became_active":"2023-11-28T18:31:02.683995+0000","last_became_peered":"2023-11-28T18:31:02.683995+0000","last_unstale":"2023-11-28T18:31:04.154436+0000","last_undegraded":"2023-11-28T18:31:04.154436+0000","last_fullsized":"2023-11-28T18:31:04.154436+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:35:13.466951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.160303+0000","last_change":"2023-11-28T18:31:03.169942+0000","last_active":"2023-11-28T18:31:04.160303+0000","last_peered":"2023-11-28T18:31:04.160303+0000","last_clean":"2023-11-28T18:31:04.160303+0000","last_became_active":"2023-11-28T18:31:03.168800+0000","last_became_peered":"2023-11-28T18:31:03.168800+0000","last_unstale":"2023-11-28T18:31:04.160303+0000","last_undegraded":"2023-11-28T18:31:04.160303+0000","last_fullsized":"2023-11-28T18:31:04.160303+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:51:06.589481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154317+0000","last_change":"2023-11-28T18:31:02.694377+0000","last_active":"2023-11-28T18:31:04.154317+0000","last_peered":"2023-11-28T18:31:04.154317+0000","last_clean":"2023-11-28T18:31:04.154317+0000","last_became_active":"2023-11-28T18:31:02.694098+0000","last_became_peered":"2023-11-28T18:31:02.694098+0000","last_unstale":"2023-11-28T18:31:04.154317+0000","last_undegraded":"2023-11-28T18:31:04.154317+0000","last_fullsized":"2023-11-28T18:31:04.154317+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:48:22.265264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157624+0000","last_change":"2023-11-28T18:31:03.172885+0000","last_active":"2023-11-28T18:31:04.157624+0000","last_peered":"2023-11-28T18:31:04.157624+0000","last_clean":"2023-11-28T18:31:04.157624+0000","last_became_active":"2023-11-28T18:31:03.172590+0000","last_became_peered":"2023-11-28T18:31:03.172590+0000","last_unstale":"2023-11-28T18:31:04.157624+0000","last_undegraded":"2023-11-28T18:31:04.157624+0000","last_fullsized":"2023-11-28T18:31:04.157624+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:56:36.786912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154448+0000","last_change":"2023-11-28T18:31:02.703039+0000","last_active":"2023-11-28T18:31:04.154448+0000","last_peered":"2023-11-28T18:31:04.154448+0000","last_clean":"2023-11-28T18:31:04.154448+0000","last_became_active":"2023-11-28T18:31:02.702767+0000","last_became_peered":"2023-11-28T18:31:02.702767+0000","last_unstale":"2023-11-28T18:31:04.154448+0000","last_undegraded":"2023-11-28T18:31:04.154448+0000","last_fullsized":"2023-11-28T18:31:04.154448+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:55:34.529650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.389479+0000","last_change":"2023-11-28T18:31:03.160992+0000","last_active":"2023-11-28T18:31:04.389479+0000","last_peered":"2023-11-28T18:31:04.389479+0000","last_clean":"2023-11-28T18:31:04.389479+0000","last_became_active":"2023-11-28T18:31:03.160743+0000","last_became_peered":"2023-11-28T18:31:03.160743+0000","last_unstale":"2023-11-28T18:31:04.389479+0000","last_undegraded":"2023-11-28T18:31:04.389479+0000","last_fullsized":"2023-11-28T18:31:04.389479+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:22.344176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154096+0000","last_change":"2023-11-28T18:31:02.685729+0000","last_active":"2023-11-28T18:31:04.154096+0000","last_peered":"2023-11-28T18:31:04.154096+0000","last_clean":"2023-11-28T18:31:04.154096+0000","last_became_active":"2023-11-28T18:31:02.685434+0000","last_became_peered":"2023-11-28T18:31:02.685434+0000","last_unstale":"2023-11-28T18:31:04.154096+0000","last_undegraded":"2023-11-28T18:31:04.154096+0000","last_fullsized":"2023-11-28T18:31:04.154096+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:07:04.525603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358130+0000","last_change":"2023-11-28T18:31:03.173704+0000","last_active":"2023-11-28T18:31:05.358130+0000","last_peered":"2023-11-28T18:31:05.358130+0000","last_clean":"2023-11-28T18:31:05.358130+0000","last_became_active":"2023-11-28T18:31:03.173331+0000","last_became_peered":"2023-11-28T18:31:03.173331+0000","last_unstale":"2023-11-28T18:31:05.358130+0000","last_undegraded":"2023-11-28T18:31:05.358130+0000","last_fullsized":"2023-11-28T18:31:05.358130+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:24.944277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"116'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157877+0000","last_change":"2023-11-28T18:31:02.706740+0000","last_active":"2023-11-28T18:31:04.157877+0000","last_peered":"2023-11-28T18:31:04.157877+0000","last_clean":"2023-11-28T18:31:04.157877+0000","last_became_active":"2023-11-28T18:31:02.706422+0000","last_became_peered":"2023-11-28T18:31:02.706422+0000","last_unstale":"2023-11-28T18:31:04.157877+0000","last_undegraded":"2023-11-28T18:31:04.157877+0000","last_fullsized":"2023-11-28T18:31:04.157877+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:24:21.484867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358102+0000","last_change":"2023-11-28T18:31:03.154396+0000","last_active":"2023-11-28T18:31:05.358102+0000","last_peered":"2023-11-28T18:31:05.358102+0000","last_clean":"2023-11-28T18:31:05.358102+0000","last_became_active":"2023-11-28T18:31:03.154096+0000","last_became_peered":"2023-11-28T18:31:03.154096+0000","last_unstale":"2023-11-28T18:31:05.358102+0000","last_undegraded":"2023-11-28T18:31:05.358102+0000","last_fullsized":"2023-11-28T18:31:05.358102+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:35:14.474983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.389602+0000","last_change":"2023-11-28T18:31:02.705096+0000","last_active":"2023-11-28T18:31:04.389602+0000","last_peered":"2023-11-28T18:31:04.389602+0000","last_clean":"2023-11-28T18:31:04.389602+0000","last_became_active":"2023-11-28T18:31:02.704703+0000","last_became_peered":"2023-11-28T18:31:02.704703+0000","last_unstale":"2023-11-28T18:31:04.389602+0000","last_undegraded":"2023-11-28T18:31:04.389602+0000","last_fullsized":"2023-11-28T18:31:04.389602+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:31.804335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356441+0000","last_change":"2023-11-28T18:31:03.173462+0000","last_active":"2023-11-28T18:31:05.356441+0000","last_peered":"2023-11-28T18:31:05.356441+0000","last_clean":"2023-11-28T18:31:05.356441+0000","last_became_active":"2023-11-28T18:31:03.173174+0000","last_became_peered":"2023-11-28T18:31:03.173174+0000","last_unstale":"2023-11-28T18:31:05.356441+0000","last_undegraded":"2023-11-28T18:31:05.356441+0000","last_fullsized":"2023-11-28T18:31:05.356441+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:33:07.941786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"116'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.131304+0000","last_change":"2023-11-28T18:31:02.700108+0000","last_active":"2023-11-28T18:31:05.131304+0000","last_peered":"2023-11-28T18:31:05.131304+0000","last_clean":"2023-11-28T18:31:05.131304+0000","last_became_active":"2023-11-28T18:31:02.699813+0000","last_became_peered":"2023-11-28T18:31:02.699813+0000","last_unstale":"2023-11-28T18:31:05.131304+0000","last_undegraded":"2023-11-28T18:31:05.131304+0000","last_fullsized":"2023-11-28T18:31:05.131304+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:09.283416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358567+0000","last_change":"2023-11-28T18:31:03.179578+0000","last_active":"2023-11-28T18:31:05.358567+0000","last_peered":"2023-11-28T18:31:05.358567+0000","last_clean":"2023-11-28T18:31:05.358567+0000","last_became_active":"2023-11-28T18:31:03.179043+0000","last_became_peered":"2023-11-28T18:31:03.179043+0000","last_unstale":"2023-11-28T18:31:05.358567+0000","last_undegraded":"2023-11-28T18:31:05.358567+0000","last_fullsized":"2023-11-28T18:31:05.358567+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:28:49.821579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.131084+0000","last_change":"2023-11-28T18:31:02.706160+0000","last_active":"2023-11-28T18:31:05.131084+0000","last_peered":"2023-11-28T18:31:05.131084+0000","last_clean":"2023-11-28T18:31:05.131084+0000","last_became_active":"2023-11-28T18:31:02.705836+0000","last_became_peered":"2023-11-28T18:31:02.705836+0000","last_unstale":"2023-11-28T18:31:05.131084+0000","last_undegraded":"2023-11-28T18:31:05.131084+0000","last_fullsized":"2023-11-28T18:31:05.131084+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:35:13.370840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.355989+0000","last_change":"2023-11-28T18:31:03.174068+0000","last_active":"2023-11-28T18:31:05.355989+0000","last_peered":"2023-11-28T18:31:05.355989+0000","last_clean":"2023-11-28T18:31:05.355989+0000","last_became_active":"2023-11-28T18:31:03.172701+0000","last_became_peered":"2023-11-28T18:31:03.172701+0000","last_unstale":"2023-11-28T18:31:05.355989+0000","last_undegraded":"2023-11-28T18:31:05.355989+0000","last_fullsized":"2023-11-28T18:31:05.355989+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:31:51.703770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356144+0000","last_change":"2023-11-28T18:31:02.706556+0000","last_active":"2023-11-28T18:31:05.356144+0000","last_peered":"2023-11-28T18:31:05.356144+0000","last_clean":"2023-11-28T18:31:05.356144+0000","last_became_active":"2023-11-28T18:31:02.706288+0000","last_became_peered":"2023-11-28T18:31:02.706288+0000","last_unstale":"2023-11-28T18:31:05.356144+0000","last_undegraded":"2023-11-28T18:31:05.356144+0000","last_fullsized":"2023-11-28T18:31:05.356144+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:06.509719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154008+0000","last_change":"2023-11-28T18:31:03.154716+0000","last_active":"2023-11-28T18:31:04.154008+0000","last_peered":"2023-11-28T18:31:04.154008+0000","last_clean":"2023-11-28T18:31:04.154008+0000","last_became_active":"2023-11-28T18:31:03.154419+0000","last_became_peered":"2023-11-28T18:31:03.154419+0000","last_unstale":"2023-11-28T18:31:04.154008+0000","last_undegraded":"2023-11-28T18:31:04.154008+0000","last_fullsized":"2023-11-28T18:31:04.154008+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:55.346235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.130718+0000","last_change":"2023-11-28T18:31:02.702584+0000","last_active":"2023-11-28T18:31:05.130718+0000","last_peered":"2023-11-28T18:31:05.130718+0000","last_clean":"2023-11-28T18:31:05.130718+0000","last_became_active":"2023-11-28T18:31:02.702054+0000","last_became_peered":"2023-11-28T18:31:02.702054+0000","last_unstale":"2023-11-28T18:31:05.130718+0000","last_undegraded":"2023-11-28T18:31:05.130718+0000","last_fullsized":"2023-11-28T18:31:05.130718+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:54.338203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154597+0000","last_change":"2023-11-28T18:31:03.170182+0000","last_active":"2023-11-28T18:31:04.154597+0000","last_peered":"2023-11-28T18:31:04.154597+0000","last_clean":"2023-11-28T18:31:04.154597+0000","last_became_active":"2023-11-28T18:31:03.169522+0000","last_became_peered":"2023-11-28T18:31:03.169522+0000","last_unstale":"2023-11-28T18:31:04.154597+0000","last_undegraded":"2023-11-28T18:31:04.154597+0000","last_fullsized":"2023-11-28T18:31:04.154597+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:41:36.177742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154194+0000","last_change":"2023-11-28T18:31:02.705758+0000","last_active":"2023-11-28T18:31:04.154194+0000","last_peered":"2023-11-28T18:31:04.154194+0000","last_clean":"2023-11-28T18:31:04.154194+0000","last_became_active":"2023-11-28T18:31:02.705435+0000","last_became_peered":"2023-11-28T18:31:02.705435+0000","last_unstale":"2023-11-28T18:31:04.154194+0000","last_undegraded":"2023-11-28T18:31:04.154194+0000","last_fullsized":"2023-11-28T18:31:04.154194+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:47.718941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.358049+0000","last_change":"2023-11-28T18:31:03.167051+0000","last_active":"2023-11-28T18:31:05.358049+0000","last_peered":"2023-11-28T18:31:05.358049+0000","last_clean":"2023-11-28T18:31:05.358049+0000","last_became_active":"2023-11-28T18:31:03.166707+0000","last_became_peered":"2023-11-28T18:31:03.166707+0000","last_unstale":"2023-11-28T18:31:05.358049+0000","last_undegraded":"2023-11-28T18:31:05.358049+0000","last_fullsized":"2023-11-28T18:31:05.358049+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:55:35.537682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.130486+0000","last_change":"2023-11-28T18:31:02.694696+0000","last_active":"2023-11-28T18:31:05.130486+0000","last_peered":"2023-11-28T18:31:05.130486+0000","last_clean":"2023-11-28T18:31:05.130486+0000","last_became_active":"2023-11-28T18:31:02.694413+0000","last_became_peered":"2023-11-28T18:31:02.694413+0000","last_unstale":"2023-11-28T18:31:05.130486+0000","last_undegraded":"2023-11-28T18:31:05.130486+0000","last_fullsized":"2023-11-28T18:31:05.130486+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:55:31.745817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356547+0000","last_change":"2023-11-28T18:31:03.170901+0000","last_active":"2023-11-28T18:31:05.356547+0000","last_peered":"2023-11-28T18:31:05.356547+0000","last_clean":"2023-11-28T18:31:05.356547+0000","last_became_active":"2023-11-28T18:31:03.170600+0000","last_became_peered":"2023-11-28T18:31:03.170600+0000","last_unstale":"2023-11-28T18:31:05.356547+0000","last_undegraded":"2023-11-28T18:31:05.356547+0000","last_fullsized":"2023-11-28T18:31:05.356547+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:54:02.678471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356490+0000","last_change":"2023-11-28T18:31:02.702698+0000","last_active":"2023-11-28T18:31:05.356490+0000","last_peered":"2023-11-28T18:31:05.356490+0000","last_clean":"2023-11-28T18:31:05.356490+0000","last_became_active":"2023-11-28T18:31:02.702423+0000","last_became_peered":"2023-11-28T18:31:02.702423+0000","last_unstale":"2023-11-28T18:31:05.356490+0000","last_undegraded":"2023-11-28T18:31:05.356490+0000","last_fullsized":"2023-11-28T18:31:05.356490+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:17:56.982900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157674+0000","last_change":"2023-11-28T18:31:03.174085+0000","last_active":"2023-11-28T18:31:04.157674+0000","last_peered":"2023-11-28T18:31:04.157674+0000","last_clean":"2023-11-28T18:31:04.157674+0000","last_became_active":"2023-11-28T18:31:03.173581+0000","last_became_peered":"2023-11-28T18:31:03.173581+0000","last_unstale":"2023-11-28T18:31:04.157674+0000","last_undegraded":"2023-11-28T18:31:04.157674+0000","last_fullsized":"2023-11-28T18:31:04.157674+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:48.726973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.160527+0000","last_change":"2023-11-28T18:31:02.708037+0000","last_active":"2023-11-28T18:31:04.160527+0000","last_peered":"2023-11-28T18:31:04.160527+0000","last_clean":"2023-11-28T18:31:04.160527+0000","last_became_active":"2023-11-28T18:31:02.707577+0000","last_became_peered":"2023-11-28T18:31:02.707577+0000","last_unstale":"2023-11-28T18:31:04.160527+0000","last_undegraded":"2023-11-28T18:31:04.160527+0000","last_fullsized":"2023-11-28T18:31:04.160527+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:28:10.333699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.131218+0000","last_change":"2023-11-28T18:31:03.178585+0000","last_active":"2023-11-28T18:31:05.131218+0000","last_peered":"2023-11-28T18:31:05.131218+0000","last_clean":"2023-11-28T18:31:05.131218+0000","last_became_active":"2023-11-28T18:31:03.177218+0000","last_became_peered":"2023-11-28T18:31:03.177218+0000","last_unstale":"2023-11-28T18:31:05.131218+0000","last_undegraded":"2023-11-28T18:31:05.131218+0000","last_fullsized":"2023-11-28T18:31:05.131218+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:24:58.809372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.160602+0000","last_change":"2023-11-28T18:31:02.708035+0000","last_active":"2023-11-28T18:31:04.160602+0000","last_peered":"2023-11-28T18:31:04.160602+0000","last_clean":"2023-11-28T18:31:04.160602+0000","last_became_active":"2023-11-28T18:31:02.707715+0000","last_became_peered":"2023-11-28T18:31:02.707715+0000","last_unstale":"2023-11-28T18:31:04.160602+0000","last_undegraded":"2023-11-28T18:31:04.160602+0000","last_fullsized":"2023-11-28T18:31:04.160602+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:01:11.184059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.389655+0000","last_change":"2023-11-28T18:31:03.167260+0000","last_active":"2023-11-28T18:31:04.389655+0000","last_peered":"2023-11-28T18:31:04.389655+0000","last_clean":"2023-11-28T18:31:04.389655+0000","last_became_active":"2023-11-28T18:31:03.166321+0000","last_became_peered":"2023-11-28T18:31:03.166321+0000","last_unstale":"2023-11-28T18:31:04.389655+0000","last_undegraded":"2023-11-28T18:31:04.389655+0000","last_fullsized":"2023-11-28T18:31:04.389655+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:24:45.585392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356568+0000","last_change":"2023-11-28T18:31:02.707644+0000","last_active":"2023-11-28T18:31:05.356568+0000","last_peered":"2023-11-28T18:31:05.356568+0000","last_clean":"2023-11-28T18:31:05.356568+0000","last_became_active":"2023-11-28T18:31:02.707392+0000","last_became_peered":"2023-11-28T18:31:02.707392+0000","last_unstale":"2023-11-28T18:31:05.356568+0000","last_undegraded":"2023-11-28T18:31:05.356568+0000","last_fullsized":"2023-11-28T18:31:05.356568+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:05.933002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356268+0000","last_change":"2023-11-28T18:31:03.164309+0000","last_active":"2023-11-28T18:31:05.356268+0000","last_peered":"2023-11-28T18:31:05.356268+0000","last_clean":"2023-11-28T18:31:05.356268+0000","last_became_active":"2023-11-28T18:31:03.163966+0000","last_became_peered":"2023-11-28T18:31:03.163966+0000","last_unstale":"2023-11-28T18:31:05.356268+0000","last_undegraded":"2023-11-28T18:31:05.356268+0000","last_fullsized":"2023-11-28T18:31:05.356268+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:06:57.073736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.357755+0000","last_change":"2023-11-28T18:31:02.691320+0000","last_active":"2023-11-28T18:31:05.357755+0000","last_peered":"2023-11-28T18:31:05.357755+0000","last_clean":"2023-11-28T18:31:05.357755+0000","last_became_active":"2023-11-28T18:31:02.691018+0000","last_became_peered":"2023-11-28T18:31:02.691018+0000","last_unstale":"2023-11-28T18:31:05.357755+0000","last_undegraded":"2023-11-28T18:31:05.357755+0000","last_fullsized":"2023-11-28T18:31:05.357755+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:01:11.184059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154091+0000","last_change":"2023-11-28T18:31:03.166813+0000","last_active":"2023-11-28T18:31:04.154091+0000","last_peered":"2023-11-28T18:31:04.154091+0000","last_clean":"2023-11-28T18:31:04.154091+0000","last_became_active":"2023-11-28T18:31:03.166039+0000","last_became_peered":"2023-11-28T18:31:03.166039+0000","last_unstale":"2023-11-28T18:31:04.154091+0000","last_undegraded":"2023-11-28T18:31:04.154091+0000","last_fullsized":"2023-11-28T18:31:04.154091+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:44:23.208354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.357458+0000","last_change":"2023-11-28T18:31:02.703970+0000","last_active":"2023-11-28T18:31:05.357458+0000","last_peered":"2023-11-28T18:31:05.357458+0000","last_clean":"2023-11-28T18:31:05.357458+0000","last_became_active":"2023-11-28T18:31:02.703679+0000","last_became_peered":"2023-11-28T18:31:02.703679+0000","last_unstale":"2023-11-28T18:31:05.357458+0000","last_undegraded":"2023-11-28T18:31:05.357458+0000","last_fullsized":"2023-11-28T18:31:05.357458+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:57:11.690622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.130736+0000","last_change":"2023-11-28T18:31:03.174732+0000","last_active":"2023-11-28T18:31:05.130736+0000","last_peered":"2023-11-28T18:31:05.130736+0000","last_clean":"2023-11-28T18:31:05.130736+0000","last_became_active":"2023-11-28T18:31:03.174463+0000","last_became_peered":"2023-11-28T18:31:03.174463+0000","last_unstale":"2023-11-28T18:31:05.130736+0000","last_undegraded":"2023-11-28T18:31:05.130736+0000","last_fullsized":"2023-11-28T18:31:05.130736+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:07:46.210330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.356115+0000","last_change":"2023-11-28T18:31:02.703636+0000","last_active":"2023-11-28T18:31:05.356115+0000","last_peered":"2023-11-28T18:31:05.356115+0000","last_clean":"2023-11-28T18:31:05.356115+0000","last_became_active":"2023-11-28T18:31:02.703302+0000","last_became_peered":"2023-11-28T18:31:02.703302+0000","last_unstale":"2023-11-28T18:31:05.356115+0000","last_undegraded":"2023-11-28T18:31:05.356115+0000","last_fullsized":"2023-11-28T18:31:05.356115+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:49:43.542415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.157885+0000","last_change":"2023-11-28T18:31:03.166640+0000","last_active":"2023-11-28T18:31:04.157885+0000","last_peered":"2023-11-28T18:31:04.157885+0000","last_clean":"2023-11-28T18:31:04.157885+0000","last_became_active":"2023-11-28T18:31:03.166389+0000","last_became_peered":"2023-11-28T18:31:03.166389+0000","last_unstale":"2023-11-28T18:31:04.157885+0000","last_undegraded":"2023-11-28T18:31:04.157885+0000","last_fullsized":"2023-11-28T18:31:04.157885+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:26:09.301219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.130530+0000","last_change":"2023-11-28T18:31:02.695278+0000","last_active":"2023-11-28T18:31:05.130530+0000","last_peered":"2023-11-28T18:31:05.130530+0000","last_clean":"2023-11-28T18:31:05.130530+0000","last_became_active":"2023-11-28T18:31:02.694979+0000","last_became_peered":"2023-11-28T18:31:02.694979+0000","last_unstale":"2023-11-28T18:31:05.130530+0000","last_undegraded":"2023-11-28T18:31:05.130530+0000","last_fullsized":"2023-11-28T18:31:05.130530+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:55:34.529650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:05.131189+0000","last_change":"2023-11-28T18:31:03.178449+0000","last_active":"2023-11-28T18:31:05.131189+0000","last_peered":"2023-11-28T18:31:05.131189+0000","last_clean":"2023-11-28T18:31:05.131189+0000","last_became_active":"2023-11-28T18:31:03.178172+0000","last_became_peered":"2023-11-28T18:31:03.178172+0000","last_unstale":"2023-11-28T18:31:05.131189+0000","last_undegraded":"2023-11-28T18:31:05.131189+0000","last_fullsized":"2023-11-28T18:31:05.131189+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:02.666065+0000","last_clean_scrub_stamp":"2023-11-28T18:31:02.666065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:23:13.307954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"116'8","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-11-28T18:31:04.154020+0000","last_change":"2023-11-28T18:31:02.695116+0000","last_active":"2023-11-28T18:31:04.154020+0000","last_peered":"2023-11-28T18:31:04.154020+0000","last_clean":"2023-11-28T18:31:04.154020+0000","last_became_active":"2023-11-28T18:31:02.694855+0000","last_became_peered":"2023-11-28T18:31:02.694855+0000","last_unstale":"2023-11-28T18:31:04.154020+0000","last_undegraded":"2023-11-28T18:31:04.154020+0000","last_fullsized":"2023-11-28T18:31:04.154020+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:01.658033+0000","last_clean_scrub_stamp":"2023-11-28T18:31:01.658033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:24:21.484867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":248,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-11-28T18:31:03.166133+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:31:03.166133+0000","last_peered":"2023-11-28T18:31:03.166133+0000","last_clean":"2023-11-28T18:31:03.166133+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:31:03.166133+0000","last_undegraded":"2023-11-28T18:31:03.166133+0000","last_fullsized":"2023-11-28T18:31:03.166133+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574945,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51248,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320592,"statfs":{"total":96636764160,"available":96584286208,"internally_reserved":0,"allocated":2068480,"data_stored":1269894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63700000000000001,"15min":0.70099999999999996},"min":{"1min":0.32600000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.96999999999999997,"5min":0.98099999999999998,"15min":2.427},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.64200000000000002,"15min":0.70199999999999996},"min":{"1min":0.40999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.82799999999999996,"5min":0.98199999999999998,"15min":2.2509999999999999},"last":1.0449999999999999}]},{"osd":1,"last update":"Tue Nov 28 18:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66700000000000004,"15min":0.70699999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.34699999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":2.3849999999999998},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.65800000000000003,"15min":0.73199999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.006,"5min":1.079,"15min":2.3130000000000002},"last":0.82899999999999996}]},{"osd":2,"last update":"Tue Nov 28 18:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67800000000000005,"15min":0.752},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.91500000000000004,"5min":0.998,"15min":2.4590000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.67400000000000004,"15min":0.72699999999999998},"min":{"1min":0.499,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82199999999999995,"5min":1.022,"15min":2.411},"last":0.999}]},{"osd":3,"last update":"Tue Nov 28 18:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65900000000000003,"15min":0.73899999999999999},"min":{"1min":0.45700000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80800000000000005,"5min":0.92000000000000004,"15min":2.2730000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71099999999999997,"15min":0.78300000000000003},"min":{"1min":0.48699999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.873,"5min":1.0389999999999999,"15min":2.3650000000000002},"last":0.98399999999999999}]},{"osd":4,"last update":"Tue Nov 28 18:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59099999999999997,"15min":0.60299999999999998},"min":{"1min":0.40100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86299999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.63200000000000001},"min":{"1min":0.38600000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.91800000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.749}]},{"osd":5,"last update":"Tue Nov 28 18:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.628,"15min":0.66700000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85599999999999998,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.64500000000000002,"15min":0.68999999999999995},"min":{"1min":0.40000000000000002,"5min":0.39900000000000002,"15min":0.39300000000000002},"max":{"1min":0.85099999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.875}]},{"osd":6,"last update":"Tue Nov 28 18:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70199999999999996,"15min":0.73499999999999999},"min":{"1min":0.54500000000000004,"5min":0.375,"15min":0.375},"max":{"1min":0.97699999999999998,"5min":1.131,"15min":1.131},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63600000000000001,"15min":0.71199999999999997},"min":{"1min":0.44700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88900000000000001,"5min":1.081,"15min":1.081},"last":1.02}]}]},{"osd":6,"up_from":13,"seq":55834574944,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72868,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298972,"statfs":{"total":96636764160,"available":96562147328,"internally_reserved":0,"allocated":1462272,"data_stored":679332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66800000000000004,"15min":0.64500000000000002},"min":{"1min":0.44800000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.94499999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":1.331},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71399999999999997,"15min":0.72599999999999998},"min":{"1min":0.40699999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.88700000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":1.673}]},{"osd":1,"last update":"Tue Nov 28 18:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65600000000000003,"15min":0.77100000000000002},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90800000000000003,"5min":0.96899999999999997,"15min":4.2389999999999999},"last":1.2809999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68200000000000005,"15min":0.76700000000000002},"min":{"1min":0.44500000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.82599999999999996,"5min":0.94499999999999995,"15min":4.218},"last":0.81000000000000005}]},{"osd":2,"last update":"Tue Nov 28 18:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66100000000000003,"15min":0.70099999999999996},"min":{"1min":0.39300000000000002,"5min":0.373,"15min":0.36299999999999999},"max":{"1min":0.90500000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":2.7599999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.68300000000000005,"15min":0.70099999999999996},"min":{"1min":0.59699999999999998,"5min":0.42999999999999999,"15min":0.40100000000000002},"max":{"1min":0.90500000000000003,"5min":1.012,"15min":1.012},"last":1.7849999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69999999999999996,"15min":0.72199999999999998},"min":{"1min":0.46300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.96499999999999997,"5min":1.016,"15min":1.016},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67500000000000004,"15min":0.68500000000000005},"min":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.1100000000000001},"last":2.8090000000000002}]},{"osd":4,"last update":"Tue Nov 28 18:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60299999999999998,"15min":0.65500000000000003},"min":{"1min":0.44500000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83599999999999997,"5min":1.02,"15min":1.02},"last":1.2310000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.621,"15min":0.67300000000000004},"min":{"1min":0.45000000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.91100000000000003,"5min":1.038,"15min":1.038},"last":2.1640000000000001}]},{"osd":5,"last update":"Tue Nov 28 18:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.622,"15min":0.66800000000000004},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.877,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":3.0110000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.74099999999999999},"min":{"1min":0.44700000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":1.0449999999999999},"last":1.3740000000000001}]},{"osd":7,"last update":"Tue Nov 28 18:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68500000000000005,"15min":0.751},"min":{"1min":0.48699999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88200000000000001,"5min":1.002,"15min":1.0269999999999999},"last":1.722},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.68200000000000005,"15min":0.73499999999999999},"min":{"1min":0.42799999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":2.8490000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36776,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335064,"statfs":{"total":96636764160,"available":96599105536,"internally_reserved":0,"allocated":1466368,"data_stored":678910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.68500000000000005,"15min":0.70399999999999996},"min":{"1min":0.36099999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.90600000000000003,"5min":2.1299999999999999,"15min":2.8929999999999998},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.71299999999999997,"15min":0.74299999999999999},"min":{"1min":0.434,"5min":0.39200000000000002,"15min":0.33900000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":2.9140000000000001},"last":1.0169999999999999}]},{"osd":1,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68100000000000005,"15min":0.72799999999999998},"min":{"1min":0.505,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":3.4809999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.77900000000000003},"min":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.39100000000000001},"max":{"1min":0.96299999999999997,"5min":2.2109999999999999,"15min":3.4580000000000002},"last":0.93100000000000005}]},{"osd":2,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.66100000000000003,"15min":0.73499999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":3.9510000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66700000000000004,"15min":0.75},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86299999999999999,"5min":1.014,"15min":4.0149999999999997},"last":0.91300000000000003}]},{"osd":3,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70999999999999996,"15min":0.79800000000000004},"min":{"1min":0.44700000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1,"5min":2.254,"15min":4.0869999999999997},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68999999999999995,"15min":0.79100000000000004},"min":{"1min":0.46400000000000002,"5min":0.46300000000000002,"15min":0.41299999999999998},"max":{"1min":0.89800000000000002,"5min":0.98599999999999999,"15min":3.9900000000000002},"last":0.874}]},{"osd":4,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.33500000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79300000000000004,"5min":1.696,"15min":1.696},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.66700000000000004,"15min":0.69999999999999996},"min":{"1min":0.45900000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.90800000000000003,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.58499999999999996}]},{"osd":6,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65600000000000003,"15min":0.67400000000000004},"min":{"1min":0.42899999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0600000000000001,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68899999999999995,"15min":0.70999999999999996},"min":{"1min":0.48299999999999998,"5min":0.438,"15min":0.42399999999999999},"max":{"1min":0.89200000000000002,"5min":1.262,"15min":1.756},"last":0.68899999999999995}]},{"osd":7,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72499999999999998,"15min":0.745},"min":{"1min":0.50700000000000001,"5min":0.42399999999999999,"15min":0.41699999999999998},"max":{"1min":0.91400000000000003,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72799999999999998,"15min":0.748},"min":{"1min":0.502,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1259999999999999,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.71899999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574945,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72876,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298964,"statfs":{"total":96636764160,"available":96562139136,"internally_reserved":0,"allocated":1470464,"data_stored":680362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.64200000000000002,"15min":0.67600000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.315},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":2.363},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.39200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.91700000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.99099999999999999}]},{"osd":1,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.64200000000000002,"15min":0.628},"min":{"1min":0.41399999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0629999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68500000000000005,"15min":0.66000000000000003},"min":{"1min":0.47199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.98899999999999999,"5min":1.02,"15min":1.02},"last":0.81200000000000006}]},{"osd":2,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.71799999999999997,"15min":0.70099999999999996},"min":{"1min":0.48499999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96699999999999997,"5min":1.056,"15min":1.056},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72599999999999998,"15min":0.72899999999999998},"min":{"1min":0.54800000000000004,"5min":0.434,"15min":0.434},"max":{"1min":1.024,"5min":1.091,"15min":1.871},"last":0.90900000000000003}]},{"osd":3,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.65300000000000002},"min":{"1min":0.50600000000000001,"5min":0.45700000000000002,"15min":0.40899999999999997},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68899999999999995,"15min":0.68500000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.91700000000000004,"5min":1.014,"15min":1.014},"last":0.83499999999999996}]},{"osd":5,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60099999999999998,"15min":0.58999999999999997},"min":{"1min":0.33100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.82599999999999996,"5min":1.03,"15min":1.1799999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60199999999999998,"15min":0.629},"min":{"1min":0.39600000000000002,"5min":0.33400000000000002,"15min":0.32300000000000001},"max":{"1min":0.86899999999999999,"5min":1.05,"15min":1.79},"last":0.77700000000000002}]},{"osd":6,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.61299999999999999,"15min":0.63900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.91400000000000003,"5min":1.0680000000000001,"15min":1.123},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.68999999999999995,"15min":0.70399999999999996},"min":{"1min":0.41799999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92000000000000004,"5min":1.1970000000000001,"15min":1.925},"last":0.747}]},{"osd":7,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68000000000000005,"15min":0.70799999999999996},"min":{"1min":0.45900000000000002,"5min":0.41999999999999998,"15min":0.36399999999999999},"max":{"1min":0.93200000000000005,"5min":1.1040000000000001,"15min":1.843},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.749,"15min":0.77100000000000002},"min":{"1min":0.47799999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94399999999999995,"5min":1.1299999999999999,"15min":1.593},"last":1.0149999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574944,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46516,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325324,"statfs":{"total":96636764160,"available":96589131776,"internally_reserved":0,"allocated":1474560,"data_stored":680396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.439,"15min":0.41999999999999998},"min":{"1min":0.28399999999999997,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.57799999999999996,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.5,"15min":0.47599999999999998},"min":{"1min":0.33100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.65000000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.40200000000000002}]},{"osd":1,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45300000000000001,"15min":0.45900000000000002},"min":{"1min":0.32000000000000001,"5min":0.28799999999999998,"15min":0.27300000000000002},"max":{"1min":0.67000000000000004,"5min":0.73399999999999999,"15min":0.96099999999999997},"last":0.498},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55400000000000005,"15min":0.51700000000000002},"min":{"1min":0.375,"5min":0.249,"15min":0.247},"max":{"1min":0.72599999999999998,"5min":4.5599999999999996,"15min":4.5599999999999996},"last":0.51500000000000001}]},{"osd":2,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55400000000000005,"15min":0.52900000000000003},"min":{"1min":0.377,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.97899999999999998,"5min":2.48,"15min":2.48},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.439,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81399999999999995,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.47899999999999998}]},{"osd":4,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.89600000000000002,"15min":0.81200000000000006},"min":{"1min":0.50700000000000001,"5min":0.441,"15min":0.39800000000000002},"max":{"1min":1.1870000000000001,"5min":12.695,"15min":12.695},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.80100000000000005,"15min":0.78000000000000003},"min":{"1min":0.47899999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0349999999999999,"5min":6.3250000000000002,"15min":6.3250000000000002},"last":0.89100000000000001}]},{"osd":5,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76800000000000002,"15min":0.81499999999999995},"min":{"1min":0.55200000000000005,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94099999999999995,"5min":1.9430000000000001,"15min":2.5019999999999998},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.80400000000000005,"15min":0.78100000000000003},"min":{"1min":0.58699999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0269999999999999,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.109}]},{"osd":6,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.82899999999999996,"15min":0.80800000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1439999999999999,"5min":6.1790000000000003,"15min":6.1790000000000003},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.52200000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0009999999999999,"5min":1.984,"15min":1.984},"last":0.94199999999999995}]},{"osd":7,"last update":"Tue Nov 28 18:30:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.80700000000000005,"15min":0.81000000000000005},"min":{"1min":0.54400000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.069,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.83999999999999997,"15min":0.81799999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0640000000000001,"5min":6.2249999999999996,"15min":6.2249999999999996},"last":1.0349999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574945,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36772,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335068,"statfs":{"total":96636764160,"available":96599109632,"internally_reserved":0,"allocated":1458176,"data_stored":678750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.437,"15min":0.48499999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.56299999999999994,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.45100000000000001,"15min":0.48999999999999999},"min":{"1min":0.26800000000000002,"5min":0.26700000000000002,"15min":0.26500000000000001},"max":{"1min":0.66300000000000003,"5min":0.66800000000000004,"15min":2.0150000000000001},"last":0.47099999999999997}]},{"osd":1,"last update":"Tue Nov 28 18:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.47299999999999998,"15min":0.51200000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.63200000000000001,"5min":0.70999999999999996,"15min":1.48},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.56999999999999995,"15min":0.53200000000000003},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.56599999999999995,"5min":8.3539999999999992,"15min":8.3539999999999992},"last":0.51700000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.48799999999999999,"15min":0.499},"min":{"1min":0.34799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.56799999999999995,"5min":3.1549999999999998,"15min":3.1549999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.499,"5min":0.53800000000000003,"15min":0.56200000000000006},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66500000000000004,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":0.53500000000000003}]},{"osd":4,"last update":"Tue Nov 28 18:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.74199999999999999,"15min":0.76500000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.04,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.749,"15min":0.76900000000000002},"min":{"1min":0.45700000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0600000000000001,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.68999999999999995}]},{"osd":5,"last update":"Tue Nov 28 18:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.749,"15min":0.76200000000000001},"min":{"1min":0.47399999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1040000000000001,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72799999999999998,"15min":0.92600000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97199999999999998,"5min":1.0780000000000001,"15min":7.4820000000000002},"last":0.68000000000000005}]},{"osd":6,"last update":"Tue Nov 28 18:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.84699999999999998,"15min":0.81499999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.129,"5min":12.019,"15min":12.019},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81299999999999994,"15min":0.85399999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0349999999999999,"5min":8.7400000000000002,"15min":8.7400000000000002},"last":0.71899999999999997}]},{"osd":7,"last update":"Tue Nov 28 18:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91700000000000004,"5min":8.327,"15min":8.327},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.81899999999999995,"15min":0.89200000000000002},"min":{"1min":0.504,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1479999999999999,"5min":8.609,"15min":8.609},"last":0.628}]}]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40936,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330904,"statfs":{"total":96636764160,"available":96594845696,"internally_reserved":0,"allocated":1462272,"data_stored":678772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.41499999999999998,"15min":0.42299999999999999},"min":{"1min":0.248,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58299999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.47399999999999998,"15min":0.45800000000000002},"min":{"1min":0.29599999999999999,"5min":0.27300000000000002,"15min":0.26600000000000001},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.36899999999999999}]},{"osd":2,"last update":"Tue Nov 28 18:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.44600000000000001,"15min":0.44900000000000001},"min":{"1min":0.32300000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.63100000000000001,"5min":0.69499999999999995,"15min":0.75700000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.65100000000000002,"5min":0.73399999999999999,"15min":0.79200000000000004},"last":0.44}]},{"osd":3,"last update":"Tue Nov 28 18:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54000000000000004,"15min":0.53900000000000003},"min":{"1min":0.39400000000000002,"5min":0.376,"15min":0.36699999999999999},"max":{"1min":0.71599999999999997,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.505},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51100000000000001,"15min":0.496},"min":{"1min":0.371,"5min":0.33600000000000002,"15min":0.32200000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.49199999999999999}]},{"osd":4,"last update":"Tue Nov 28 18:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.70999999999999996,"15min":0.73599999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91000000000000003,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.747,"15min":0.76300000000000001},"min":{"1min":0.52400000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.81799999999999995,"5min":1.0780000000000001,"15min":1.155},"last":0.73099999999999998}]},{"osd":5,"last update":"Tue Nov 28 18:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.69499999999999995,"15min":0.73399999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92500000000000004,"5min":0.997,"15min":1.3799999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70099999999999996,"15min":0.73599999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.98999999999999999,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.66000000000000003}]},{"osd":6,"last update":"Tue Nov 28 18:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.73599999999999999,"15min":0.76600000000000001},"min":{"1min":0.48599999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.83599999999999997,"5min":1.1279999999999999,"15min":1.2969999999999999},"last":0.753},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.75700000000000001,"15min":0.79300000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.95499999999999996,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.71299999999999997}]},{"osd":7,"last update":"Tue Nov 28 18:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.72699999999999998,"15min":0.746},"min":{"1min":0.503,"5min":0.503,"15min":0.47899999999999998},"max":{"1min":0.90600000000000003,"5min":1.0669999999999999,"15min":1.2410000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73499999999999999,"15min":0.76500000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.48499999999999999},"max":{"1min":0.96999999999999997,"5min":1.0109999999999999,"15min":1.4239999999999999},"last":0.67900000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574944,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47076,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324764,"statfs":{"total":96636764160,"available":96588558336,"internally_reserved":0,"allocated":2056192,"data_stored":1269140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 18:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41699999999999998,"15min":0.435},"min":{"1min":0.252,"5min":0.248,"15min":0.248},"max":{"1min":0.61199999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.441},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47699999999999998,"15min":0.47199999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.69199999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.45500000000000002}]},{"osd":2,"last update":"Tue Nov 28 18:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48399999999999999,"15min":0.502},"min":{"1min":0.33000000000000002,"5min":0.318,"15min":0.30499999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.92900000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.48899999999999999,"15min":0.50700000000000001},"min":{"1min":0.316,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65000000000000002,"5min":0.77800000000000002,"15min":0.91200000000000003},"last":0.48999999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47599999999999998,"15min":0.48599999999999999},"min":{"1min":0.373,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65500000000000003,"5min":0.78600000000000003,"15min":0.86499999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53700000000000003,"15min":0.54800000000000004},"min":{"1min":0.38800000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77200000000000002,"5min":0.80000000000000004,"15min":1.087},"last":0.39300000000000002}]},{"osd":4,"last update":"Tue Nov 28 18:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71999999999999997,"15min":0.71499999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.97699999999999998,"5min":1.131,"15min":1.131},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71499999999999997,"15min":0.71699999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.95099999999999996,"5min":0.996,"15min":1.244},"last":0.65100000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71899999999999997,"15min":0.72499999999999998},"min":{"1min":0.46600000000000003,"5min":0.436,"15min":0.436},"max":{"1min":0.91900000000000004,"5min":1.0489999999999999,"15min":1.121},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71599999999999997,"15min":0.73399999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.94099999999999995,"5min":1.0329999999999999,"15min":1.075},"last":0.66500000000000004}]},{"osd":6,"last update":"Tue Nov 28 18:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73299999999999998,"15min":0.73199999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.97599999999999998,"5min":1.1140000000000001,"15min":1.407},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.73099999999999998,"15min":0.73399999999999999},"min":{"1min":0.46800000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88,"5min":1.087,"15min":1.089},"last":0.61499999999999999}]},{"osd":7,"last update":"Tue Nov 28 18:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.749,"15min":0.752},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.91600000000000004,"5min":1.054,"15min":1.212},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.73399999999999999,"15min":0.74399999999999999},"min":{"1min":0.54100000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.89900000000000002,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.625}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:31:08.142 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T18:31:08.143 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-28T18:31:08.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.325+0000 7f8a5bd0f640 1 Processor -- start 2023-11-28T18:31:08.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.325+0000 7f8a5bd0f640 1 -- start start 2023-11-28T18:31:08.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.329+0000 7f8a5bd0f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a54159f40 0x7f8a540caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:08.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.329+0000 7f8a5bd0f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a5406bf50 con 0x7f8a540cc760 2023-11-28T18:31:08.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.329+0000 7f8a5bd0f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a540ccbc0 con 0x7f8a540ad850 2023-11-28T18:31:08.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.329+0000 7f8a5bd0f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a5415a320 con 0x7f8a54159f40 2023-11-28T18:31:08.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.329+0000 7f8a59aab640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a54159f40 0x7f8a540caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:08.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.329+0000 7f8a59aab640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a54159f40 0x7f8a540caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38278/0 (socket says 172.21.15.105:38278) 2023-11-28T18:31:08.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.329+0000 7f8a59aab640 1 -- 172.21.15.105:0/3149074210 learned_addr learned my addr 172.21.15.105:0/3149074210 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:08.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.329+0000 7f8a592aa640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8a540cc760 0x7f8a540ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59590/0 (socket says 172.21.15.105:59590) 2023-11-28T18:31:08.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.329+0000 7f8a59aab640 1 -- 172.21.15.105:0/3149074210 >> v1:172.21.15.148:6789/0 conn(0x7f8a540ad850 legacy=0x7f8a540701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:08.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.329+0000 7f8a59aab640 1 -- 172.21.15.105:0/3149074210 >> v1:172.21.15.105:6789/0 conn(0x7f8a540cc760 legacy=0x7f8a540ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:08.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.329+0000 7f8a59aab640 1 -- 172.21.15.105:0/3149074210 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a54071380 con 0x7f8a54159f40 2023-11-28T18:31:08.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.329+0000 7f8a58aa9640 1 -- 172.21.15.105:0/3149074210 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2616484199 0 0) 0x7f8a5406bf50 con 0x7f8a540cc760 2023-11-28T18:31:08.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.333+0000 7f8a59aab640 1 --2- 172.21.15.105:0/3149074210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a54159f40 0x7f8a540caf60 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000bb10 tx=0x7f8a5001eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caaadca4844f556d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:08.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.333+0000 7f8a58aa9640 1 -- 172.21.15.105:0/3149074210 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a500a7be0 con 0x7f8a54159f40 2023-11-28T18:31:08.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.333+0000 7f8a58aa9640 1 -- 172.21.15.105:0/3149074210 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a500a7d40 con 0x7f8a54159f40 2023-11-28T18:31:08.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.333+0000 7f8a58aa9640 1 -- 172.21.15.105:0/3149074210 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a500af660 con 0x7f8a54159f40 2023-11-28T18:31:08.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.333+0000 7f8a5bd0f640 1 -- 172.21.15.105:0/3149074210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a54159f40 msgr2=0x7f8a540caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:08.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.333+0000 7f8a5bd0f640 1 --2- 172.21.15.105:0/3149074210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a54159f40 0x7f8a540caf60 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000bb10 tx=0x7f8a5001eb10 comp rx=0 tx=0).stop 2023-11-28T18:31:08.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.333+0000 7f8a5bd0f640 1 -- 172.21.15.105:0/3149074210 shutdown_connections 2023-11-28T18:31:08.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.333+0000 7f8a5bd0f640 1 --2- 172.21.15.105:0/3149074210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a54159f40 0x7f8a540caf60 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:08.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.333+0000 7f8a5bd0f640 1 -- 172.21.15.105:0/3149074210 >> 172.21.15.105:0/3149074210 conn(0x7f8a540ceb90 msgr2=0x7f8a540cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:08.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.333+0000 7f8a5bd0f640 1 -- 172.21.15.105:0/3149074210 shutdown_connections 2023-11-28T18:31:08.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.333+0000 7f8a5bd0f640 1 -- 172.21.15.105:0/3149074210 wait complete. 2023-11-28T18:31:08.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.333+0000 7f8a5bd0f640 1 Processor -- start 2023-11-28T18:31:08.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.333+0000 7f8a5bd0f640 1 -- start start 2023-11-28T18:31:08.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a5bd0f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a540ad850 0x7f8a5409c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:08.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a5bd0f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a540cc760 0x7f8a5409a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:08.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a5bd0f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a54159f40 0x7f8a54149b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:08.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a5bd0f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5409bfc0 con 0x7f8a540ad850 2023-11-28T18:31:08.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a5bd0f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5415b730 con 0x7f8a540cc760 2023-11-28T18:31:08.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a5bd0f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5409b020 con 0x7f8a54159f40 2023-11-28T18:31:08.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a59aab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a540ad850 0x7f8a5409c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:08.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a59aab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a540ad850 0x7f8a5409c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38282/0 (socket says 172.21.15.105:38282) 2023-11-28T18:31:08.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a59aab640 1 -- 172.21.15.105:0/3995139881 learned_addr learned my addr 172.21.15.105:0/3995139881 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:08.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a592aa640 1 --2- 172.21.15.105:0/3995139881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a540cc760 0x7f8a5409a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:08.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a5a2ac640 1 --2- 172.21.15.105:0/3995139881 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a54159f40 0x7f8a54149b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:08.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a59aab640 1 -- 172.21.15.105:0/3995139881 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a54159f40 msgr2=0x7f8a54149b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:08.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a59aab640 1 --2- 172.21.15.105:0/3995139881 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a54159f40 0x7f8a54149b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:08.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a59aab640 1 -- 172.21.15.105:0/3995139881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a540cc760 msgr2=0x7f8a5409a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:08.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a59aab640 1 --2- 172.21.15.105:0/3995139881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a540cc760 0x7f8a5409a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:08.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a59aab640 1 -- 172.21.15.105:0/3995139881 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5000b770 con 0x7f8a540ad850 2023-11-28T18:31:08.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a592aa640 1 --2- 172.21.15.105:0/3995139881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a540cc760 0x7f8a5409a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:08.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a59aab640 1 --2- 172.21.15.105:0/3995139881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a540ad850 0x7f8a5409c4c0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000b2c0 tx=0x7f8a5001b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:08.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a4a7fc640 1 -- 172.21.15.105:0/3995139881 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a500a7be0 con 0x7f8a540ad850 2023-11-28T18:31:08.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a4a7fc640 1 -- 172.21.15.105:0/3995139881 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a500ba420 con 0x7f8a540ad850 2023-11-28T18:31:08.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a5bd0f640 1 -- 172.21.15.105:0/3995139881 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a5415b9d0 con 0x7f8a540ad850 2023-11-28T18:31:08.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a4a7fc640 1 -- 172.21.15.105:0/3995139881 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a500ba7f0 con 0x7f8a540ad850 2023-11-28T18:31:08.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a5bd0f640 1 -- 172.21.15.105:0/3995139881 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a5415bf90 con 0x7f8a540ad850 2023-11-28T18:31:08.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a5bd0f640 1 -- 172.21.15.105:0/3995139881 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a5415c180 con 0x7f8a540ad850 2023-11-28T18:31:08.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.337+0000 7f8a4a7fc640 1 -- 172.21.15.105:0/3995139881 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8a500b6040 con 0x7f8a540ad850 2023-11-28T18:31:08.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.341+0000 7f8a4a7fc640 1 --2- 172.21.15.105:0/3995139881 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a380668d0 0x7f8a38068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:08.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.341+0000 7f8a4a7fc640 1 -- 172.21.15.105:0/3995139881 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8a501219c0 con 0x7f8a540ad850 2023-11-28T18:31:08.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.341+0000 7f8a592aa640 1 --2- 172.21.15.105:0/3995139881 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a380668d0 0x7f8a38068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:08.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.341+0000 7f8a592aa640 1 --2- 172.21.15.105:0/3995139881 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a380668d0 0x7f8a38068d90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8a44007fd0 tx=0x7f8a44007c20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:08.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.345+0000 7f8a4a7fc640 1 -- 172.21.15.105:0/3995139881 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8a500eeab0 con 0x7f8a540ad850 2023-11-28T18:31:08.543 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.545+0000 7f8a5bd0f640 1 -- 172.21.15.105:0/3995139881 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8a54067570 con 0x7f8a540ad850 2023-11-28T18:31:08.547 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.549+0000 7f8a4a7fc640 1 -- 172.21.15.105:0/3995139881 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f8a500b40e0 con 0x7f8a540ad850 2023-11-28T18:31:08.547 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2023-11-28T18:31:08.547 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDpL2ZlSt0VMxAAQ8+j1BgBPMJ2cuiZj8aG0w== 2023-11-28T18:31:08.548 INFO:teuthology.orchestra.run.smithi105.stdout: caps mds = "allow" 2023-11-28T18:31:08.548 INFO:teuthology.orchestra.run.smithi105.stdout: caps mon = "allow r" 2023-11-28T18:31:08.548 INFO:teuthology.orchestra.run.smithi105.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-28T18:31:08.549 INFO:teuthology.orchestra.run.smithi105.stderr:updated caps for client.0 2023-11-28T18:31:08.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.549+0000 7f8a5bd0f640 1 -- 172.21.15.105:0/3995139881 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a380668d0 msgr2=0x7f8a38068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:08.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.549+0000 7f8a5bd0f640 1 --2- 172.21.15.105:0/3995139881 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a380668d0 0x7f8a38068d90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8a44007fd0 tx=0x7f8a44007c20 comp rx=0 tx=0).stop 2023-11-28T18:31:08.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.549+0000 7f8a5bd0f640 1 -- 172.21.15.105:0/3995139881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a540ad850 msgr2=0x7f8a5409c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:08.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.549+0000 7f8a5bd0f640 1 --2- 172.21.15.105:0/3995139881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a540ad850 0x7f8a5409c4c0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000b2c0 tx=0x7f8a5001b8b0 comp rx=0 tx=0).stop 2023-11-28T18:31:08.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.549+0000 7f8a5bd0f640 1 -- 172.21.15.105:0/3995139881 shutdown_connections 2023-11-28T18:31:08.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.549+0000 7f8a5bd0f640 1 --2- 172.21.15.105:0/3995139881 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a54159f40 0x7f8a54149b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:08.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.549+0000 7f8a5bd0f640 1 --2- 172.21.15.105:0/3995139881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a540cc760 0x7f8a5409a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:08.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.549+0000 7f8a5bd0f640 1 --2- 172.21.15.105:0/3995139881 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a380668d0 0x7f8a38068d90 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:08.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.549+0000 7f8a5bd0f640 1 --2- 172.21.15.105:0/3995139881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a540ad850 0x7f8a5409c4c0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:08.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.549+0000 7f8a5bd0f640 1 -- 172.21.15.105:0/3995139881 >> 172.21.15.105:0/3995139881 conn(0x7f8a540ceb90 msgr2=0x7f8a54071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:08.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.549+0000 7f8a5bd0f640 1 -- 172.21.15.105:0/3995139881 shutdown_connections 2023-11-28T18:31:08.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.549+0000 7f8a5bd0f640 1 -- 172.21.15.105:0/3995139881 wait complete. 2023-11-28T18:31:08.570 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:31:08.658 INFO:tasks.ceph.mds.a.smithi105.stderr:2023-11-28T18:31:08.657+0000 7f4d2e9cd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T18:31:08.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.709+0000 7fbf654c9640 1 Processor -- start 2023-11-28T18:31:08.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.709+0000 7fbf654c9640 1 -- start start 2023-11-28T18:31:08.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.709+0000 7fbf654c9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbf600d3c30 0x7fbf600d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:08.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.709+0000 7fbf654c9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf600b25b0 con 0x7fbf600b2d20 2023-11-28T18:31:08.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.709+0000 7fbf654c9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf600cea30 con 0x7fbf6015d4f0 2023-11-28T18:31:08.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.709+0000 7fbf654c9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf600b2900 con 0x7fbf600d3c30 2023-11-28T18:31:08.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.709+0000 7fbf5f7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fbf600b2d20 0x7fbf600b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59592/0 (socket says 172.21.15.105:59592) 2023-11-28T18:31:08.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.709+0000 7fbf5f7fe640 1 -- 172.21.15.105:0/2867520908 learned_addr learned my addr 172.21.15.105:0/2867520908 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:08.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.709+0000 7fbf567fc640 1 --2- 172.21.15.105:0/2867520908 >> v2:172.21.15.148:3300/0 conn(0x7fbf600d3c30 0x7fbf600d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:08.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.709+0000 7fbf5e7fc640 1 -- 172.21.15.105:0/2867520908 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857955724 0 0) 0x7fbf600b25b0 con 0x7fbf600b2d20 2023-11-28T18:31:08.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.709+0000 7fbf5e7fc640 1 -- 172.21.15.105:0/2867520908 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf4c003530 con 0x7fbf600b2d20 2023-11-28T18:31:08.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.709+0000 7fbf567fc640 1 -- 172.21.15.105:0/2867520908 >> v1:172.21.15.148:6789/0 conn(0x7fbf6015d4f0 legacy=0x7fbf600d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:08.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.709+0000 7fbf567fc640 1 -- 172.21.15.105:0/2867520908 >> v1:172.21.15.105:6789/0 conn(0x7fbf600b2d20 legacy=0x7fbf600b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:08.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.709+0000 7fbf5e7fc640 1 -- 172.21.15.105:0/2867520908 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 943034824 0 0) 0x7fbf4c003530 con 0x7fbf600b2d20 2023-11-28T18:31:08.711 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.709+0000 7fbf567fc640 1 -- 172.21.15.105:0/2867520908 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf600ced10 con 0x7fbf600d3c30 2023-11-28T18:31:08.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.713+0000 7fbf567fc640 1 --2- 172.21.15.105:0/2867520908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf600d3c30 0x7fbf600d1520 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009d90 tx=0x7fbf4801c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=291db3e695a5a19a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:08.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.713+0000 7fbf5e7fc640 1 -- 172.21.15.105:0/2867520908 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58001680 con 0x7fbf600d3c30 2023-11-28T18:31:08.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.713+0000 7fbf5e7fc640 1 -- 172.21.15.105:0/2867520908 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf480a5590 con 0x7fbf600d3c30 2023-11-28T18:31:08.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.713+0000 7fbf5e7fc640 1 -- 172.21.15.105:0/2867520908 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf480a5920 con 0x7fbf600d3c30 2023-11-28T18:31:08.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.713+0000 7fbf654c9640 1 -- 172.21.15.105:0/2867520908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf600d3c30 msgr2=0x7fbf600d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:08.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.713+0000 7fbf654c9640 1 --2- 172.21.15.105:0/2867520908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf600d3c30 0x7fbf600d1520 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009d90 tx=0x7fbf4801c7b0 comp rx=0 tx=0).stop 2023-11-28T18:31:08.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.713+0000 7fbf654c9640 1 -- 172.21.15.105:0/2867520908 shutdown_connections 2023-11-28T18:31:08.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.713+0000 7fbf654c9640 1 --2- 172.21.15.105:0/2867520908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf600d3c30 0x7fbf600d1520 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:08.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.713+0000 7fbf654c9640 1 -- 172.21.15.105:0/2867520908 >> 172.21.15.105:0/2867520908 conn(0x7fbf600d3030 msgr2=0x7fbf600d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:08.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.713+0000 7fbf654c9640 1 -- 172.21.15.105:0/2867520908 shutdown_connections 2023-11-28T18:31:08.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.713+0000 7fbf654c9640 1 -- 172.21.15.105:0/2867520908 wait complete. 2023-11-28T18:31:08.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf654c9640 1 Processor -- start 2023-11-28T18:31:08.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf654c9640 1 -- start start 2023-11-28T18:31:08.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf654c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf600b2d20 0x7fbf60155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:08.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf654c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf600d3c30 0x7fbf600cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:08.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf654c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbf6015d4f0 0x7fbf600cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:08.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf654c9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf600d0a20 con 0x7fbf600b2d20 2023-11-28T18:31:08.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf654c9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6015dc70 con 0x7fbf600d3c30 2023-11-28T18:31:08.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf654c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6015df50 con 0x7fbf6015d4f0 2023-11-28T18:31:08.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf5effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf600b2d20 0x7fbf60155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:08.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf5effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf600b2d20 0x7fbf60155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38290/0 (socket says 172.21.15.105:38290) 2023-11-28T18:31:08.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf5effd640 1 -- 172.21.15.105:0/2675176444 learned_addr learned my addr 172.21.15.105:0/2675176444 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:08.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf5effd640 1 -- 172.21.15.105:0/2675176444 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbf6015d4f0 msgr2=0x7fbf600cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:08.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf5effd640 1 --2- 172.21.15.105:0/2675176444 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbf6015d4f0 0x7fbf600cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:08.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf5effd640 1 -- 172.21.15.105:0/2675176444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf600d3c30 msgr2=0x7fbf600cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:08.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf5effd640 1 --2- 172.21.15.105:0/2675176444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf600d3c30 0x7fbf600cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:08.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf5effd640 1 -- 172.21.15.105:0/2675176444 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf48009a70 con 0x7fbf600b2d20 2023-11-28T18:31:08.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf5effd640 1 --2- 172.21.15.105:0/2675176444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf600b2d20 0x7fbf60155450 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fbf5000c3c0 tx=0x7fbf5000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:08.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf57fff640 1 -- 172.21.15.105:0/2675176444 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf500052d0 con 0x7fbf600b2d20 2023-11-28T18:31:08.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf57fff640 1 -- 172.21.15.105:0/2675176444 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf50005430 con 0x7fbf600b2d20 2023-11-28T18:31:08.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf57fff640 1 -- 172.21.15.105:0/2675176444 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf50003840 con 0x7fbf600b2d20 2023-11-28T18:31:08.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf654c9640 1 -- 172.21.15.105:0/2675176444 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf6015e230 con 0x7fbf600b2d20 2023-11-28T18:31:08.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf654c9640 1 -- 172.21.15.105:0/2675176444 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf6015e7f0 con 0x7fbf600b2d20 2023-11-28T18:31:08.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.717+0000 7fbf654c9640 1 -- 172.21.15.105:0/2675176444 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf6015e9e0 con 0x7fbf600b2d20 2023-11-28T18:31:08.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.721+0000 7fbf57fff640 1 -- 172.21.15.105:0/2675176444 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fbf500039a0 con 0x7fbf600b2d20 2023-11-28T18:31:08.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.721+0000 7fbf57fff640 1 --2- 172.21.15.105:0/2675176444 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbf40066900 0x7fbf40068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:08.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.721+0000 7fbf57fff640 1 -- 172.21.15.105:0/2675176444 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbf50083b00 con 0x7fbf600b2d20 2023-11-28T18:31:08.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.721+0000 7fbf567fc640 1 --2- 172.21.15.105:0/2675176444 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbf40066900 0x7fbf40068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:08.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.721+0000 7fbf567fc640 1 --2- 172.21.15.105:0/2675176444 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbf40066900 0x7fbf40068dc0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fbf480af000 tx=0x7fbf48004ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:08.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.725+0000 7fbf57fff640 1 -- 172.21.15.105:0/2675176444 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbf50085e60 con 0x7fbf600b2d20 2023-11-28T18:31:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.905+0000 7fbf654c9640 1 -- 172.21.15.105:0/2675176444 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf60067570 con 0x7fbf600b2d20 2023-11-28T18:31:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.905+0000 7fbf57fff640 1 -- 172.21.15.105:0/2675176444 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4927 (secure 0 0 0) 0x7fbf50017090 con 0x7fbf600b2d20 2023-11-28T18:31:08.906 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:08.906 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":119,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/2073622668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2073622668},{"type":"v1","addr":"172.21.15.148:6834","nonce":2073622668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":7085,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/270170970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":270170970},{"type":"v1","addr":"172.21.15.148:6835","nonce":270170970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":8640,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/3373504883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3373504883},{"type":"v1","addr":"172.21.15.105:6836","nonce":3373504883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:31:03.120082+0000","modified":"2023-11-28T18:31:06.069874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8643},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8643":{"gid":8643,"name":"a","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/764864050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":764864050},{"type":"v1","addr":"172.21.15.105:6837","nonce":764864050}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-28T18:31:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 119 2023-11-28T18:31:08.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.909+0000 7fbf654c9640 1 -- 172.21.15.105:0/2675176444 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbf40066900 msgr2=0x7fbf40068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:08.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.909+0000 7fbf654c9640 1 --2- 172.21.15.105:0/2675176444 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbf40066900 0x7fbf40068dc0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fbf480af000 tx=0x7fbf48004ac0 comp rx=0 tx=0).stop 2023-11-28T18:31:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.909+0000 7fbf654c9640 1 -- 172.21.15.105:0/2675176444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf600b2d20 msgr2=0x7fbf60155450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.909+0000 7fbf654c9640 1 --2- 172.21.15.105:0/2675176444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf600b2d20 0x7fbf60155450 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fbf5000c3c0 tx=0x7fbf5000c880 comp rx=0 tx=0).stop 2023-11-28T18:31:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.909+0000 7fbf654c9640 1 -- 172.21.15.105:0/2675176444 shutdown_connections 2023-11-28T18:31:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.909+0000 7fbf654c9640 1 --2- 172.21.15.105:0/2675176444 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbf6015d4f0 0x7fbf600cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.909+0000 7fbf654c9640 1 --2- 172.21.15.105:0/2675176444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf600d3c30 0x7fbf600cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.909+0000 7fbf654c9640 1 --2- 172.21.15.105:0/2675176444 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbf40066900 0x7fbf40068dc0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.909+0000 7fbf654c9640 1 --2- 172.21.15.105:0/2675176444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf600b2d20 0x7fbf60155450 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.909+0000 7fbf654c9640 1 -- 172.21.15.105:0/2675176444 >> 172.21.15.105:0/2675176444 conn(0x7fbf600d3030 msgr2=0x7fbf60158520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.909+0000 7fbf654c9640 1 -- 172.21.15.105:0/2675176444 shutdown_connections 2023-11-28T18:31:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:08.909+0000 7fbf654c9640 1 -- 172.21.15.105:0/2675176444 wait complete. 2023-11-28T18:31:09.330 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 119, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:31:03.120082+0000', 'modified': '2023-11-28T18:31:06.069874+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8643}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8643': {'gid': 8643, 'name': 'a', 'rank': 0, 'incarnation': 115, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6837/764864050', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6835', 'nonce': 764864050}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 764864050}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:31:09.330 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T18:31:09.330 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T18:31:09.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.468+0000 7f46daf01640 1 Processor -- start 2023-11-28T18:31:09.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.468+0000 7f46daf01640 1 -- start start 2023-11-28T18:31:09.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.468+0000 7f46daf01640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46d415b030 0x7f46d40709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.468+0000 7f46daf01640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d40701d0 con 0x7f46d40b4290 2023-11-28T18:31:09.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.468+0000 7f46daf01640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d406df80 con 0x7f46d4071960 2023-11-28T18:31:09.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.468+0000 7f46daf01640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d4071df0 con 0x7f46d415b030 2023-11-28T18:31:09.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.468+0000 7f46d8c9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f46d40b4290 0x7f46d40b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59608/0 (socket says 172.21.15.105:59608) 2023-11-28T18:31:09.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.468+0000 7f46d8c9d640 1 -- 172.21.15.105:0/2627740955 learned_addr learned my addr 172.21.15.105:0/2627740955 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:09.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46d949e640 1 --2- 172.21.15.105:0/2627740955 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46d415b030 0x7f46d40709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46cb7fe640 1 -- 172.21.15.105:0/2627740955 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4280282296 0 0) 0x7f46d40701d0 con 0x7f46d40b4290 2023-11-28T18:31:09.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46cb7fe640 1 -- 172.21.15.105:0/2627740955 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46bc003530 con 0x7f46d40b4290 2023-11-28T18:31:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46cb7fe640 1 -- 172.21.15.105:0/2627740955 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 77032727 0 0) 0x7f46d406df80 con 0x7f46d4071960 2023-11-28T18:31:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46cb7fe640 1 -- 172.21.15.105:0/2627740955 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46d40701d0 con 0x7f46d4071960 2023-11-28T18:31:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46cb7fe640 1 -- 172.21.15.105:0/2627740955 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1324562276 0 0) 0x7f46bc003530 con 0x7f46d40b4290 2023-11-28T18:31:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46cb7fe640 1 -- 172.21.15.105:0/2627740955 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46d415b030 msgr2=0x7f46d40709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46cb7fe640 1 --2- 172.21.15.105:0/2627740955 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46d415b030 0x7f46d40709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46cb7fe640 1 -- 172.21.15.105:0/2627740955 >> v1:172.21.15.148:6789/0 conn(0x7f46d4071960 legacy=0x7f46d415f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46cb7fe640 1 -- 172.21.15.105:0/2627740955 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d40720d0 con 0x7f46d40b4290 2023-11-28T18:31:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46cb7fe640 1 -- 172.21.15.105:0/2627740955 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f46d0001680 con 0x7f46d40b4290 2023-11-28T18:31:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46d949e640 1 --2- 172.21.15.105:0/2627740955 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46d415b030 0x7f46d40709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:31:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46cb7fe640 1 -- 172.21.15.105:0/2627740955 >> v1:172.21.15.105:6789/0 conn(0x7f46d40b4290 legacy=0x7f46d40b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46cb7fe640 1 --2- 172.21.15.105:0/2627740955 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46d415b030 0x7f46bc003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46cb7fe640 1 --2- 172.21.15.105:0/2627740955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46bc004080 0x7f46bc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46cb7fe640 1 --2- 172.21.15.105:0/2627740955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46bc004d10 0x7f46bc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46cb7fe640 1 -- 172.21.15.105:0/2627740955 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d4071df0 con 0x7f46bc004d10 2023-11-28T18:31:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46cb7fe640 1 -- 172.21.15.105:0/2627740955 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46bc00ba40 con 0x7f46bc004080 2023-11-28T18:31:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46cb7fe640 1 -- 172.21.15.105:0/2627740955 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46bc00bc00 con 0x7f46d415b030 2023-11-28T18:31:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46d949e640 1 --2- 172.21.15.105:0/2627740955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46bc004d10 0x7f46bc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46d8c9d640 1 --2- 172.21.15.105:0/2627740955 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46d415b030 0x7f46bc003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46cbfff640 1 --2- 172.21.15.105:0/2627740955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46bc004080 0x7f46bc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46d949e640 1 -- 172.21.15.105:0/2627740955 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46d415b030 msgr2=0x7f46bc003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46d949e640 1 --2- 172.21.15.105:0/2627740955 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46d415b030 0x7f46bc003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46d949e640 1 -- 172.21.15.105:0/2627740955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46bc004080 msgr2=0x7f46bc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46d949e640 1 --2- 172.21.15.105:0/2627740955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46bc004080 0x7f46bc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.472+0000 7f46d949e640 1 -- 172.21.15.105:0/2627740955 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc009a70 con 0x7f46bc004d10 2023-11-28T18:31:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.476+0000 7f46d949e640 1 --2- 172.21.15.105:0/2627740955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46bc004d10 0x7f46bc005190 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00e320 tx=0x7f46cc01cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae3ee6b48e605f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.476+0000 7f46cb7fe640 1 -- 172.21.15.105:0/2627740955 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc0a5db0 con 0x7f46bc004d10 2023-11-28T18:31:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.476+0000 7f46cb7fe640 1 -- 172.21.15.105:0/2627740955 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46cc00f040 con 0x7f46bc004d10 2023-11-28T18:31:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.476+0000 7f46cb7fe640 1 -- 172.21.15.105:0/2627740955 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc003ba0 con 0x7f46bc004d10 2023-11-28T18:31:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.476+0000 7f46daf01640 1 -- 172.21.15.105:0/2627740955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46bc004d10 msgr2=0x7f46bc005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.476+0000 7f46daf01640 1 --2- 172.21.15.105:0/2627740955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46bc004d10 0x7f46bc005190 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00e320 tx=0x7f46cc01cda0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.476+0000 7f46d949e640 1 -- 172.21.15.105:0/2627740955 reap_dead start 2023-11-28T18:31:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.476+0000 7f46daf01640 1 -- 172.21.15.105:0/2627740955 shutdown_connections 2023-11-28T18:31:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.476+0000 7f46daf01640 1 -- 172.21.15.105:0/2627740955 >> 172.21.15.105:0/2627740955 conn(0x7f46d40ceb90 msgr2=0x7f46d40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.476+0000 7f46daf01640 1 -- 172.21.15.105:0/2627740955 shutdown_connections 2023-11-28T18:31:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.476+0000 7f46daf01640 1 -- 172.21.15.105:0/2627740955 wait complete. 2023-11-28T18:31:09.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.476+0000 7f46daf01640 1 Processor -- start 2023-11-28T18:31:09.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.476+0000 7f46daf01640 1 -- start start 2023-11-28T18:31:09.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46daf01640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46d4149840 0x7f46d40b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46daf01640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46d409b970 0x7f46d40b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46daf01640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46d4099b70 0x7f46d409a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46daf01640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d409a8d0 con 0x7f46d4099b70 2023-11-28T18:31:09.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46daf01640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d409abb0 con 0x7f46d409b970 2023-11-28T18:31:09.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46daf01640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d409ae90 con 0x7f46d4149840 2023-11-28T18:31:09.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46d949e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46d4099b70 0x7f46d409a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46d949e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46d4099b70 0x7f46d409a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38302/0 (socket says 172.21.15.105:38302) 2023-11-28T18:31:09.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46d8c9d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46d4149840 0x7f46d40b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46d949e640 1 -- 172.21.15.105:0/3465565041 learned_addr learned my addr 172.21.15.105:0/3465565041 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:09.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46cbfff640 1 --2- 172.21.15.105:0/3465565041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46d409b970 0x7f46d40b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46d949e640 1 -- 172.21.15.105:0/3465565041 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46d4149840 msgr2=0x7f46d40b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:09.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46d949e640 1 --2- 172.21.15.105:0/3465565041 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46d4149840 0x7f46d40b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46d949e640 1 -- 172.21.15.105:0/3465565041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46d409b970 msgr2=0x7f46d40b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:09.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46d949e640 1 --2- 172.21.15.105:0/3465565041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46d409b970 0x7f46d40b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46d949e640 1 -- 172.21.15.105:0/3465565041 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d409b170 con 0x7f46d4099b70 2023-11-28T18:31:09.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46d949e640 1 --2- 172.21.15.105:0/3465565041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46d4099b70 0x7f46d409a020 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f46cc01d640 tx=0x7f46cc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:09.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46c97fa640 1 -- 172.21.15.105:0/3465565041 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc0bc070 con 0x7f46d4099b70 2023-11-28T18:31:09.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46c97fa640 1 -- 172.21.15.105:0/3465565041 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46cc0a5dc0 con 0x7f46d4099b70 2023-11-28T18:31:09.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46daf01640 1 -- 172.21.15.105:0/3465565041 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d409b450 con 0x7f46d4099b70 2023-11-28T18:31:09.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46c97fa640 1 -- 172.21.15.105:0/3465565041 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc0039a0 con 0x7f46d4099b70 2023-11-28T18:31:09.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46daf01640 1 -- 172.21.15.105:0/3465565041 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d4149c20 con 0x7f46d4099b70 2023-11-28T18:31:09.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.480+0000 7f46c97fa640 1 -- 172.21.15.105:0/3465565041 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f46cc003b00 con 0x7f46d4099b70 2023-11-28T18:31:09.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46c97fa640 1 --2- 172.21.15.105:0/3465565041 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46c0066900 0x7f46c0068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46c97fa640 1 -- 172.21.15.105:0/3465565041 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f46cc11e550 con 0x7f46d4099b70 2023-11-28T18:31:09.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46d8c9d640 1 --2- 172.21.15.105:0/3465565041 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46c0066900 0x7f46c0068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46de270000 1 Processor -- start 2023-11-28T18:31:09.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46de270000 1 -- start start 2023-11-28T18:31:09.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46d8c9d640 1 --2- 172.21.15.105:0/3465565041 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46c0066900 0x7f46c0068dc0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f46d0005260 tx=0x7f46d0001e20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:09.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46de270000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6de330 0x55fb7c6de710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46de270000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6dec50 0x55fb7c6c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46de270000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb7c6c32a0 0x55fb7c6c3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46de270000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7c556110 con 0x55fb7c6dec50 2023-11-28T18:31:09.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46de270000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7c4e69a0 con 0x55fb7c6de330 2023-11-28T18:31:09.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46de270000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb7c4e3ed0 con 0x55fb7c6c32a0 2023-11-28T18:31:09.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46cbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6dec50 0x55fb7c6c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46cbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6dec50 0x55fb7c6c2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38318/0 (socket says 172.21.15.105:38318) 2023-11-28T18:31:09.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46cbfff640 1 -- 172.21.15.105:0/2643923055 learned_addr learned my addr 172.21.15.105:0/2643923055 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:09.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46d8c9d640 1 --2- 172.21.15.105:0/2643923055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6de330 0x55fb7c6de710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46d949e640 1 --2- 172.21.15.105:0/2643923055 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb7c6c32a0 0x55fb7c6c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46cbfff640 1 -- 172.21.15.105:0/2643923055 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb7c6c32a0 msgr2=0x55fb7c6c3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:09.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46cbfff640 1 --2- 172.21.15.105:0/2643923055 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb7c6c32a0 0x55fb7c6c3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46cbfff640 1 -- 172.21.15.105:0/2643923055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6de330 msgr2=0x55fb7c6de710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:09.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46cbfff640 1 --2- 172.21.15.105:0/2643923055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6de330 0x55fb7c6de710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46cbfff640 1 -- 172.21.15.105:0/2643923055 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb7c1d7890 con 0x55fb7c6dec50 2023-11-28T18:31:09.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46d949e640 1 --2- 172.21.15.105:0/2643923055 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb7c6c32a0 0x55fb7c6c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:09.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46d8c9d640 1 --2- 172.21.15.105:0/2643923055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6de330 0x55fb7c6de710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:09.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46cbfff640 1 --2- 172.21.15.105:0/2643923055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6dec50 0x55fb7c6c2d60 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f46c4001770 tx=0x7f46c4001c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a6dfad74b325db server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:09.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46ba7fc640 1 -- 172.21.15.105:0/2643923055 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c40053a0 con 0x55fb7c6dec50 2023-11-28T18:31:09.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46ba7fc640 1 -- 172.21.15.105:0/2643923055 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c4012840 con 0x55fb7c6dec50 2023-11-28T18:31:09.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.484+0000 7f46ba7fc640 1 -- 172.21.15.105:0/2643923055 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c4005500 con 0x55fb7c6dec50 2023-11-28T18:31:09.490 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46de270000 1 -- 172.21.15.105:0/2643923055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6dec50 msgr2=0x55fb7c6c2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:09.490 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46de270000 1 --2- 172.21.15.105:0/2643923055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6dec50 0x55fb7c6c2d60 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f46c4001770 tx=0x7f46c4001c30 comp rx=0 tx=0).stop 2023-11-28T18:31:09.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46de270000 1 -- 172.21.15.105:0/2643923055 shutdown_connections 2023-11-28T18:31:09.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46de270000 1 --2- 172.21.15.105:0/2643923055 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb7c6c32a0 0x55fb7c6c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46de270000 1 --2- 172.21.15.105:0/2643923055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6dec50 0x55fb7c6c2d60 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46de270000 1 --2- 172.21.15.105:0/2643923055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6de330 0x55fb7c6de710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46de270000 1 -- 172.21.15.105:0/2643923055 >> 172.21.15.105:0/2643923055 conn(0x55fb7c6ba8c0 msgr2=0x55fb7c6bccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:09.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46de270000 1 -- 172.21.15.105:0/2643923055 shutdown_connections 2023-11-28T18:31:09.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46de270000 1 -- 172.21.15.105:0/2643923055 wait complete. 2023-11-28T18:31:09.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46de270000 1 -- start start 2023-11-28T18:31:09.494 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46de270000 1 Processor -- start 2023-11-28T18:31:09.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46de270000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb7c6c32a0 0x55fb7c693da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46de270000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6de330 0x55fb7c6942e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46de270000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6dec50 0x55fb7c694820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46de270000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7c4df7c0 con 0x55fb7c6de330 2023-11-28T18:31:09.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46de270000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7c497bf0 con 0x55fb7c6dec50 2023-11-28T18:31:09.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46de270000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb7c49da90 con 0x55fb7c6c32a0 2023-11-28T18:31:09.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46de270000 10 client.0 _ll_register_callbacks cb 0x55fb7c67ae20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:31:09.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46cbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6de330 0x55fb7c6942e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46cbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6de330 0x55fb7c6942e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38324/0 (socket says 172.21.15.105:38324) 2023-11-28T18:31:09.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46cbfff640 1 -- 172.21.15.105:0/641066013 learned_addr learned my addr 172.21.15.105:0/641066013 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:09.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46d8c9d640 1 --2- 172.21.15.105:0/641066013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb7c6c32a0 0x55fb7c693da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46d949e640 1 --2- 172.21.15.105:0/641066013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6dec50 0x55fb7c694820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46cbfff640 1 -- 172.21.15.105:0/641066013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb7c6c32a0 msgr2=0x55fb7c693da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:09.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46cbfff640 1 --2- 172.21.15.105:0/641066013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb7c6c32a0 0x55fb7c693da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46cbfff640 1 -- 172.21.15.105:0/641066013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6dec50 msgr2=0x55fb7c694820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:09.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46cbfff640 1 --2- 172.21.15.105:0/641066013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6dec50 0x55fb7c694820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46cbfff640 1 -- 172.21.15.105:0/641066013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d40701d0 con 0x55fb7c6de330 2023-11-28T18:31:09.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46cbfff640 1 --2- 172.21.15.105:0/641066013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6de330 0x55fb7c6942e0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x55fb7c47d340 tx=0x7f46c40012e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b97753b3ac61620 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:09.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:31:09.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46b37fe640 1 -- 172.21.15.105:0/641066013 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c4007220 con 0x55fb7c6de330 2023-11-28T18:31:09.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46de270000 1 -- 172.21.15.105:0/641066013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb7c4a1470 con 0x55fb7c6de330 2023-11-28T18:31:09.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.488+0000 7f46b37fe640 1 -- 172.21.15.105:0/641066013 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f46c4007380 con 0x55fb7c6de330 2023-11-28T18:31:09.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46b37fe640 1 -- 172.21.15.105:0/641066013 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46c4007540 con 0x55fb7c6de330 2023-11-28T18:31:09.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46de270000 10 client.8778 fetch_fsmap learned FSMap version 119 2023-11-28T18:31:09.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46de270000 1 -- 172.21.15.105:0/641066013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x55fb7c4a0190 con 0x55fb7c6de330 2023-11-28T18:31:09.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46b37fe640 1 -- 172.21.15.105:0/641066013 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb7c4a1470 con 0x55fb7c6de330 2023-11-28T18:31:09.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46b37fe640 1 -- 172.21.15.105:0/641066013 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f46c40067e0 con 0x55fb7c6de330 2023-11-28T18:31:09.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46de270000 10 client.8778 fetch_fsmap finished waiting for FSMap version 119 2023-11-28T18:31:09.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46de270000 10 client.8778 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:31:09.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46de270000 20 client.8778 populate_metadata read hostname 'smithi105' 2023-11-28T18:31:09.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46de270000 1 --2- 172.21.15.105:0/641066013 >> [v2:172.21.15.105:6835/764864050,v1:172.21.15.105:6837/764864050] conn(0x55fb7c695230 0x55fb7c6ef5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46de270000 4 client.8778 mds_command: new command op to 8643 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:31:09.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46de270000 1 -- 172.21.15.105:0/641066013 --> [v2:172.21.15.105:6835/764864050,v1:172.21.15.105:6837/764864050] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb7c4a4b50 con 0x55fb7c695230 2023-11-28T18:31:09.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46d8c9d640 1 --2- 172.21.15.105:0/641066013 >> [v2:172.21.15.105:6835/764864050,v1:172.21.15.105:6837/764864050] conn(0x55fb7c695230 0x55fb7c6ef5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46d8c9d640 1 --2- 172.21.15.105:0/641066013 >> [v2:172.21.15.105:6835/764864050,v1:172.21.15.105:6837/764864050] conn(0x55fb7c695230 0x55fb7c6ef5e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbc0061e08a5ec72 server_cookie=6d24019ef93e8d28 in_seq=0 out_seq=0 2023-11-28T18:31:09.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46b37fe640 10 client.8778 ms_handle_connect on v2:172.21.15.105:6835/764864050 2023-11-28T18:31:09.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46b37fe640 1 -- 172.21.15.105:0/641066013 <== mds.0 v2:172.21.15.105:6835/764864050 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fb7c4a4b50 con 0x55fb7c695230 2023-11-28T18:31:09.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46b37fe640 10 client.8778 handle_command_reply: tid=0 2023-11-28T18:31:09.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46de270000 1 client.8778 shutdown 2023-11-28T18:31:09.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46de270000 1 -- 172.21.15.105:0/641066013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6de330 msgr2=0x55fb7c6942e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:09.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46de270000 1 --2- 172.21.15.105:0/641066013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6de330 0x55fb7c6942e0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x55fb7c47d340 tx=0x7f46c40012e0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46de270000 1 -- 172.21.15.105:0/641066013 shutdown_connections 2023-11-28T18:31:09.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46de270000 1 --2- 172.21.15.105:0/641066013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6dec50 0x55fb7c694820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46de270000 1 --2- 172.21.15.105:0/641066013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6de330 0x55fb7c6942e0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46de270000 1 --2- 172.21.15.105:0/641066013 >> [v2:172.21.15.105:6835/764864050,v1:172.21.15.105:6837/764864050] conn(0x55fb7c695230 0x55fb7c6ef5e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46de270000 1 --2- 172.21.15.105:0/641066013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb7c6c32a0 0x55fb7c693da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46de270000 1 -- 172.21.15.105:0/641066013 >> 172.21.15.105:0/641066013 conn(0x55fb7c6ba8c0 msgr2=0x55fb7c68c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:09.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46b37fe640 0 client.8778 ms_handle_reset on v2:172.21.15.105:6835/764864050 2023-11-28T18:31:09.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46de270000 1 -- 172.21.15.105:0/641066013 shutdown_connections 2023-11-28T18:31:09.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46de270000 1 -- 172.21.15.105:0/641066013 wait complete. 2023-11-28T18:31:09.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.492+0000 7f46de270000 20 client.8778 trim_cache size 0 max 16384 2023-11-28T18:31:09.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.508+0000 7f46de270000 1 Processor -- start 2023-11-28T18:31:09.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.508+0000 7f46de270000 1 -- start start 2023-11-28T18:31:09.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.508+0000 7f46de270000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb7c6c32a0 0x55fb7c73a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.508+0000 7f46de270000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6de330 0x55fb7c73abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.508+0000 7f46de270000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6dec50 0x55fb7c73b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.508+0000 7f46de270000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7c198940 con 0x55fb7c6dec50 2023-11-28T18:31:09.512 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.508+0000 7f46de270000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7c4e2320 con 0x55fb7c6de330 2023-11-28T18:31:09.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.508+0000 7f46de270000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb7c4db130 con 0x55fb7c6c32a0 2023-11-28T18:31:09.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.508+0000 7f46d949e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6dec50 0x55fb7c73b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.508+0000 7f46d949e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6dec50 0x55fb7c73b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38340/0 (socket says 172.21.15.105:38340) 2023-11-28T18:31:09.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.508+0000 7f46d949e640 1 -- 172.21.15.105:0/1123399700 learned_addr learned my addr 172.21.15.105:0/1123399700 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:09.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.508+0000 7f46d949e640 1 -- 172.21.15.105:0/1123399700 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb7c6c32a0 msgr2=0x55fb7c73a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:09.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.508+0000 7f46d949e640 1 --2- 172.21.15.105:0/1123399700 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb7c6c32a0 0x55fb7c73a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.508+0000 7f46cbfff640 1 --2- 172.21.15.105:0/1123399700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6de330 0x55fb7c73abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.508+0000 7f46d949e640 1 -- 172.21.15.105:0/1123399700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6de330 msgr2=0x55fb7c73abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:09.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.508+0000 7f46d949e640 1 --2- 172.21.15.105:0/1123399700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6de330 0x55fb7c73abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.508+0000 7f46d949e640 1 -- 172.21.15.105:0/1123399700 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb7c486490 con 0x55fb7c6dec50 2023-11-28T18:31:09.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.508+0000 7f46cbfff640 1 --2- 172.21.15.105:0/1123399700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6de330 0x55fb7c73abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:31:09.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.508+0000 7f46d949e640 1 --2- 172.21.15.105:0/1123399700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6dec50 0x55fb7c73b100 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f46cc002240 tx=0x7f46cc11e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e849b298dbd743a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:09.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.508+0000 7f46baffd640 1 -- 172.21.15.105:0/1123399700 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc0ed470 con 0x55fb7c6dec50 2023-11-28T18:31:09.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.508+0000 7f46baffd640 1 -- 172.21.15.105:0/1123399700 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46cc009dc0 con 0x55fb7c6dec50 2023-11-28T18:31:09.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.508+0000 7f46baffd640 1 -- 172.21.15.105:0/1123399700 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc0ef290 con 0x55fb7c6dec50 2023-11-28T18:31:09.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46de270000 1 -- 172.21.15.105:0/1123399700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6dec50 msgr2=0x55fb7c73b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:09.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46de270000 1 --2- 172.21.15.105:0/1123399700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6dec50 0x55fb7c73b100 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f46cc002240 tx=0x7f46cc11e040 comp rx=0 tx=0).stop 2023-11-28T18:31:09.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46de270000 1 -- 172.21.15.105:0/1123399700 shutdown_connections 2023-11-28T18:31:09.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46de270000 1 --2- 172.21.15.105:0/1123399700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6dec50 0x55fb7c73b100 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46de270000 1 --2- 172.21.15.105:0/1123399700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6de330 0x55fb7c73abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46de270000 1 --2- 172.21.15.105:0/1123399700 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb7c6c32a0 0x55fb7c73a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46de270000 1 -- 172.21.15.105:0/1123399700 >> 172.21.15.105:0/1123399700 conn(0x55fb7c6ba8c0 msgr2=0x55fb7c68c630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:09.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46de270000 1 -- 172.21.15.105:0/1123399700 shutdown_connections 2023-11-28T18:31:09.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46de270000 1 -- 172.21.15.105:0/1123399700 wait complete. 2023-11-28T18:31:09.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46de270000 1 -- start start 2023-11-28T18:31:09.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46de270000 1 Processor -- start 2023-11-28T18:31:09.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46de270000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6c32a0 0x55fb7c6c2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46de270000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6de330 0x55fb7c6fa0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46de270000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb7c6dec50 0x55fb7c6fa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46de270000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7c520440 con 0x55fb7c6de330 2023-11-28T18:31:09.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46de270000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7c4db130 con 0x55fb7c6c32a0 2023-11-28T18:31:09.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46de270000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb7c49c9b0 con 0x55fb7c6dec50 2023-11-28T18:31:09.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46de270000 10 client.0 _ll_register_callbacks cb 0x55fb7c556060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:31:09.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46cbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6de330 0x55fb7c6fa0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46cbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6de330 0x55fb7c6fa0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38346/0 (socket says 172.21.15.105:38346) 2023-11-28T18:31:09.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46cbfff640 1 -- 172.21.15.105:0/31833208 learned_addr learned my addr 172.21.15.105:0/31833208 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:09.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46d8c9d640 1 --2- 172.21.15.105:0/31833208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6c32a0 0x55fb7c6c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46d949e640 1 --2- 172.21.15.105:0/31833208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb7c6dec50 0x55fb7c6fa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46cbfff640 1 -- 172.21.15.105:0/31833208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb7c6dec50 msgr2=0x55fb7c6fa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:09.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46cbfff640 1 --2- 172.21.15.105:0/31833208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb7c6dec50 0x55fb7c6fa5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46cbfff640 1 -- 172.21.15.105:0/31833208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6c32a0 msgr2=0x55fb7c6c2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:09.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46cbfff640 1 --2- 172.21.15.105:0/31833208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6c32a0 0x55fb7c6c2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46cbfff640 1 -- 172.21.15.105:0/31833208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d4149c20 con 0x55fb7c6de330 2023-11-28T18:31:09.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46d8c9d640 1 --2- 172.21.15.105:0/31833208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6c32a0 0x55fb7c6c2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:09.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46cbfff640 1 --2- 172.21.15.105:0/31833208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6de330 0x55fb7c6fa0b0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f46c4001ec0 tx=0x7f46c4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df856ea1e432d9f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:09.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:31:09.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46b37fe640 1 -- 172.21.15.105:0/31833208 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c4012b50 con 0x55fb7c6de330 2023-11-28T18:31:09.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46de270000 1 -- 172.21.15.105:0/31833208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb7c4de110 con 0x55fb7c6de330 2023-11-28T18:31:09.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46b37fe640 1 -- 172.21.15.105:0/31833208 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f46c4003600 con 0x55fb7c6de330 2023-11-28T18:31:09.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46b37fe640 1 -- 172.21.15.105:0/31833208 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46c40037c0 con 0x55fb7c6de330 2023-11-28T18:31:09.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46de270000 10 client.8784 fetch_fsmap learned FSMap version 119 2023-11-28T18:31:09.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.512+0000 7f46de270000 1 -- 172.21.15.105:0/31833208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x55fb7c1d7cc0 con 0x55fb7c6de330 2023-11-28T18:31:09.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46b37fe640 1 -- 172.21.15.105:0/31833208 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb7c4de110 con 0x55fb7c6de330 2023-11-28T18:31:09.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46b37fe640 1 -- 172.21.15.105:0/31833208 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f46c4003920 con 0x55fb7c6de330 2023-11-28T18:31:09.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46de270000 10 client.8784 fetch_fsmap finished waiting for FSMap version 119 2023-11-28T18:31:09.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46de270000 10 client.8784 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:31:09.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46de270000 20 client.8784 populate_metadata read hostname 'smithi105' 2023-11-28T18:31:09.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46de270000 1 --2- 172.21.15.105:0/31833208 >> [v2:172.21.15.105:6835/764864050,v1:172.21.15.105:6837/764864050] conn(0x55fb7c73dd20 0x55fb7c73e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46de270000 4 client.8784 mds_command: new command op to 8643 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:31:09.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46de270000 1 -- 172.21.15.105:0/31833208 --> [v2:172.21.15.105:6835/764864050,v1:172.21.15.105:6837/764864050] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fb7c50c650 con 0x55fb7c73dd20 2023-11-28T18:31:09.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46d8c9d640 1 --2- 172.21.15.105:0/31833208 >> [v2:172.21.15.105:6835/764864050,v1:172.21.15.105:6837/764864050] conn(0x55fb7c73dd20 0x55fb7c73e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46d8c9d640 1 --2- 172.21.15.105:0/31833208 >> [v2:172.21.15.105:6835/764864050,v1:172.21.15.105:6837/764864050] conn(0x55fb7c73dd20 0x55fb7c73e100 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a0f7978f3a4718c server_cookie=c18801f48a54052e in_seq=0 out_seq=0 2023-11-28T18:31:09.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46b37fe640 10 client.8784 ms_handle_connect on v2:172.21.15.105:6835/764864050 2023-11-28T18:31:09.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46b37fe640 1 -- 172.21.15.105:0/31833208 <== mds.0 v2:172.21.15.105:6835/764864050 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55fb7c50c650 con 0x55fb7c73dd20 2023-11-28T18:31:09.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46b37fe640 10 client.8784 handle_command_reply: tid=0 2023-11-28T18:31:09.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46de270000 1 client.8784 shutdown 2023-11-28T18:31:09.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46de270000 1 -- 172.21.15.105:0/31833208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6de330 msgr2=0x55fb7c6fa0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:09.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46de270000 1 --2- 172.21.15.105:0/31833208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6de330 0x55fb7c6fa0b0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f46c4001ec0 tx=0x7f46c4009040 comp rx=0 tx=0).stop 2023-11-28T18:31:09.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46de270000 1 -- 172.21.15.105:0/31833208 shutdown_connections 2023-11-28T18:31:09.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46de270000 1 --2- 172.21.15.105:0/31833208 >> [v2:172.21.15.105:6835/764864050,v1:172.21.15.105:6837/764864050] conn(0x55fb7c73dd20 0x55fb7c73e100 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46de270000 1 --2- 172.21.15.105:0/31833208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fb7c6dec50 0x55fb7c6fa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46de270000 1 --2- 172.21.15.105:0/31833208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fb7c6de330 0x55fb7c6fa0b0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46b37fe640 0 client.8784 ms_handle_reset on v2:172.21.15.105:6835/764864050 2023-11-28T18:31:09.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46de270000 1 --2- 172.21.15.105:0/31833208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb7c6c32a0 0x55fb7c6c2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46de270000 1 -- 172.21.15.105:0/31833208 >> 172.21.15.105:0/31833208 conn(0x55fb7c6ba8c0 msgr2=0x55fb7c6b8390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:09.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46de270000 1 -- 172.21.15.105:0/31833208 shutdown_connections 2023-11-28T18:31:09.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46de270000 1 -- 172.21.15.105:0/31833208 wait complete. 2023-11-28T18:31:09.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46de270000 20 client.8784 trim_cache size 0 max 16384 2023-11-28T18:31:09.529 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:31:09.529 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:31:09.529 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2023-11-28T18:31:09.529 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 8643, 2023-11-28T18:31:09.529 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:31:09.530 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:31:09.530 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:31:09.530 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 5.8382448389999997, 2023-11-28T18:31:09.530 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 119, 2023-11-28T18:31:09.530 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 117, 2023-11-28T18:31:09.530 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 117, 2023-11-28T18:31:09.530 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 11.729149184000001 2023-11-28T18:31:09.530 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:31:09.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46daf01640 1 -- 172.21.15.105:0/3465565041 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46c0066900 msgr2=0x7f46c0068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:09.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46daf01640 1 --2- 172.21.15.105:0/3465565041 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46c0066900 0x7f46c0068dc0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f46d0005260 tx=0x7f46d0001e20 comp rx=0 tx=0).stop 2023-11-28T18:31:09.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46daf01640 1 -- 172.21.15.105:0/3465565041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46d4099b70 msgr2=0x7f46d409a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:09.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46daf01640 1 --2- 172.21.15.105:0/3465565041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46d4099b70 0x7f46d409a020 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f46cc01d640 tx=0x7f46cc009750 comp rx=0 tx=0).stop 2023-11-28T18:31:09.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46daf01640 1 -- 172.21.15.105:0/3465565041 shutdown_connections 2023-11-28T18:31:09.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46daf01640 1 --2- 172.21.15.105:0/3465565041 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46c0066900 0x7f46c0068dc0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46daf01640 1 --2- 172.21.15.105:0/3465565041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46d4099b70 0x7f46d409a020 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46daf01640 1 --2- 172.21.15.105:0/3465565041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46d409b970 0x7f46d40b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46daf01640 1 --2- 172.21.15.105:0/3465565041 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46d4149840 0x7f46d40b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46daf01640 1 -- 172.21.15.105:0/3465565041 >> 172.21.15.105:0/3465565041 conn(0x7f46d40ceb90 msgr2=0x7f46d40ae830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:09.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46daf01640 1 -- 172.21.15.105:0/3465565041 shutdown_connections 2023-11-28T18:31:09.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.516+0000 7f46daf01640 1 -- 172.21.15.105:0/3465565041 wait complete. 2023-11-28T18:31:09.536 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T18:31:09.674 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.672+0000 7f8ea8032640 1 Processor -- start 2023-11-28T18:31:09.674 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.672+0000 7f8ea8032640 1 -- start start 2023-11-28T18:31:09.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.672+0000 7f8ea8032640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8ea00b4290 0x7f8ea00b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.676+0000 7f8ea8032640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ea0071960 0x7f8ea015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.676+0000 7f8ea8032640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea00b4bb0 con 0x7f8ea015b030 2023-11-28T18:31:09.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.676+0000 7f8ea8032640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea00701d0 con 0x7f8ea00b4290 2023-11-28T18:31:09.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.676+0000 7f8ea8032640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea006df80 con 0x7f8ea0071960 2023-11-28T18:31:09.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.676+0000 7f8ea65cf640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8ea015b030 0x7f8ea00709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59612/0 (socket says 172.21.15.105:59612) 2023-11-28T18:31:09.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.676+0000 7f8ea65cf640 1 -- 172.21.15.105:0/397550767 learned_addr learned my addr 172.21.15.105:0/397550767 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:09.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.676+0000 7f8ea5dce640 1 --2- 172.21.15.105:0/397550767 >> v2:172.21.15.148:3300/0 conn(0x7f8ea00b4290 0x7f8ea00b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.676+0000 7f8ea55cd640 1 --2- 172.21.15.105:0/397550767 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ea0071960 0x7f8ea015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.676+0000 7f8ea4dcc640 1 -- 172.21.15.105:0/397550767 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1534216050 0 0) 0x7f8ea00b4bb0 con 0x7f8ea015b030 2023-11-28T18:31:09.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.676+0000 7f8ea4dcc640 1 -- 172.21.15.105:0/397550767 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e8c003530 con 0x7f8ea015b030 2023-11-28T18:31:09.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.676+0000 7f8ea4dcc640 1 -- 172.21.15.105:0/397550767 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2304600887 0 0) 0x7f8e8c003530 con 0x7f8ea015b030 2023-11-28T18:31:09.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.676+0000 7f8ea55cd640 1 -- 172.21.15.105:0/397550767 >> v2:172.21.15.148:3300/0 conn(0x7f8ea00b4290 msgr2=0x7f8ea00b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:09.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.676+0000 7f8ea55cd640 1 --2- 172.21.15.105:0/397550767 >> v2:172.21.15.148:3300/0 conn(0x7f8ea00b4290 0x7f8ea00b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.676+0000 7f8ea55cd640 1 -- 172.21.15.105:0/397550767 >> v1:172.21.15.105:6789/0 conn(0x7f8ea015b030 legacy=0x7f8ea00709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:09.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.676+0000 7f8ea55cd640 1 -- 172.21.15.105:0/397550767 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea0071df0 con 0x7f8ea0071960 2023-11-28T18:31:09.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.676+0000 7f8ea5dce640 1 --2- 172.21.15.105:0/397550767 >> v2:172.21.15.148:3300/0 conn(0x7f8ea00b4290 0x7f8ea00b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:31:09.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.680+0000 7f8ea55cd640 1 --2- 172.21.15.105:0/397550767 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ea0071960 0x7f8ea015f3b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8e900097c0 tx=0x7f8e9001c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac7fd9624934ba75 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:09.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.680+0000 7f8ea4dcc640 1 -- 172.21.15.105:0/397550767 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98001a00 con 0x7f8ea0071960 2023-11-28T18:31:09.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.680+0000 7f8ea4dcc640 1 -- 172.21.15.105:0/397550767 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e900a7050 con 0x7f8ea0071960 2023-11-28T18:31:09.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.680+0000 7f8ea4dcc640 1 -- 172.21.15.105:0/397550767 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e900ab620 con 0x7f8ea0071960 2023-11-28T18:31:09.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.680+0000 7f8ea8032640 1 -- 172.21.15.105:0/397550767 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ea0071960 msgr2=0x7f8ea015f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:09.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.680+0000 7f8ea8032640 1 --2- 172.21.15.105:0/397550767 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ea0071960 0x7f8ea015f3b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8e900097c0 tx=0x7f8e9001c370 comp rx=0 tx=0).stop 2023-11-28T18:31:09.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.680+0000 7f8ea8032640 1 -- 172.21.15.105:0/397550767 shutdown_connections 2023-11-28T18:31:09.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.680+0000 7f8ea8032640 1 --2- 172.21.15.105:0/397550767 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ea0071960 0x7f8ea015f3b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.680+0000 7f8ea8032640 1 --2- 172.21.15.105:0/397550767 >> v2:172.21.15.148:3300/0 conn(0x7f8ea00b4290 0x7f8ea00b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.680+0000 7f8ea8032640 1 -- 172.21.15.105:0/397550767 >> 172.21.15.105:0/397550767 conn(0x7f8ea00ceb90 msgr2=0x7f8ea00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:09.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.680+0000 7f8ea8032640 1 -- 172.21.15.105:0/397550767 shutdown_connections 2023-11-28T18:31:09.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.680+0000 7f8ea8032640 1 -- 172.21.15.105:0/397550767 wait complete. 2023-11-28T18:31:09.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.680+0000 7f8ea8032640 1 Processor -- start 2023-11-28T18:31:09.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.680+0000 7f8ea8032640 1 -- start start 2023-11-28T18:31:09.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.680+0000 7f8ea8032640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ea00b4290 0x7f8ea009b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.680+0000 7f8ea8032640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ea015b030 0x7f8ea00cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.680+0000 7f8ea8032640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ea014af30 0x7f8ea00b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.680+0000 7f8ea8032640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea0152a40 con 0x7f8ea015b030 2023-11-28T18:31:09.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.680+0000 7f8ea8032640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea015b7c0 con 0x7f8ea014af30 2023-11-28T18:31:09.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.680+0000 7f8ea8032640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea014b3c0 con 0x7f8ea00b4290 2023-11-28T18:31:09.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.684+0000 7f8ea55cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ea015b030 0x7f8ea00cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.684+0000 7f8ea55cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ea015b030 0x7f8ea00cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38348/0 (socket says 172.21.15.105:38348) 2023-11-28T18:31:09.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.684+0000 7f8ea55cd640 1 -- 172.21.15.105:0/3982048117 learned_addr learned my addr 172.21.15.105:0/3982048117 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:09.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.684+0000 7f8ea5dce640 1 --2- 172.21.15.105:0/3982048117 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ea00b4290 0x7f8ea009b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.684+0000 7f8ea65cf640 1 --2- 172.21.15.105:0/3982048117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ea014af30 0x7f8ea00b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.684+0000 7f8ea55cd640 1 -- 172.21.15.105:0/3982048117 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ea00b4290 msgr2=0x7f8ea009b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:09.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.684+0000 7f8ea55cd640 1 --2- 172.21.15.105:0/3982048117 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ea00b4290 0x7f8ea009b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.684+0000 7f8ea55cd640 1 -- 172.21.15.105:0/3982048117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ea014af30 msgr2=0x7f8ea00b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:09.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.684+0000 7f8ea55cd640 1 --2- 172.21.15.105:0/3982048117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ea014af30 0x7f8ea00b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.684+0000 7f8ea55cd640 1 -- 172.21.15.105:0/3982048117 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9c009420 con 0x7f8ea015b030 2023-11-28T18:31:09.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.684+0000 7f8ea65cf640 1 --2- 172.21.15.105:0/3982048117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ea014af30 0x7f8ea00b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:09.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.684+0000 7f8ea55cd640 1 --2- 172.21.15.105:0/3982048117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ea015b030 0x7f8ea00cadb0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f8e9001aa20 tx=0x7f8e9001abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:09.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.684+0000 7f8e967fc640 1 -- 172.21.15.105:0/3982048117 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e900b7450 con 0x7f8ea015b030 2023-11-28T18:31:09.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.684+0000 7f8e967fc640 1 -- 172.21.15.105:0/3982048117 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e900b75d0 con 0x7f8ea015b030 2023-11-28T18:31:09.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.684+0000 7f8e967fc640 1 -- 172.21.15.105:0/3982048117 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e900b7980 con 0x7f8ea015b030 2023-11-28T18:31:09.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.684+0000 7f8ea8032640 1 -- 172.21.15.105:0/3982048117 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e90009420 con 0x7f8ea015b030 2023-11-28T18:31:09.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.684+0000 7f8ea8032640 1 -- 172.21.15.105:0/3982048117 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea0152f70 con 0x7f8ea015b030 2023-11-28T18:31:09.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.684+0000 7f8ea8032640 1 -- 172.21.15.105:0/3982048117 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea0067570 con 0x7f8ea015b030 2023-11-28T18:31:09.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.684+0000 7f8e967fc640 1 -- 172.21.15.105:0/3982048117 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8e900b7ae0 con 0x7f8ea015b030 2023-11-28T18:31:09.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.684+0000 7f8e967fc640 1 --2- 172.21.15.105:0/3982048117 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8e840668d0 0x7f8e84068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:09.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.684+0000 7f8e967fc640 1 -- 172.21.15.105:0/3982048117 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8e9011ecd0 con 0x7f8ea015b030 2023-11-28T18:31:09.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.688+0000 7f8ea5dce640 1 --2- 172.21.15.105:0/3982048117 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8e840668d0 0x7f8e84068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:09.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.688+0000 7f8ea5dce640 1 --2- 172.21.15.105:0/3982048117 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8e840668d0 0x7f8e84068d90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0155f70 tx=0x7f8e9c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:09.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.688+0000 7f8e967fc640 1 -- 172.21.15.105:0/3982048117 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8e900ebd30 con 0x7f8ea015b030 2023-11-28T18:31:09.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.872+0000 7f8ea8032640 1 -- 172.21.15.105:0/3982048117 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8ea00cf7f0 con 0x7f8ea015b030 2023-11-28T18:31:09.872 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.872+0000 7f8e967fc640 1 -- 172.21.15.105:0/3982048117 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v119) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8e900b0070 con 0x7f8ea015b030 2023-11-28T18:31:09.873 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T18:31:09.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.872+0000 7f8ea8032640 1 -- 172.21.15.105:0/3982048117 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8e840668d0 msgr2=0x7f8e84068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:09.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.872+0000 7f8ea8032640 1 --2- 172.21.15.105:0/3982048117 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8e840668d0 0x7f8e84068d90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0155f70 tx=0x7f8e9c009190 comp rx=0 tx=0).stop 2023-11-28T18:31:09.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.872+0000 7f8ea8032640 1 -- 172.21.15.105:0/3982048117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ea015b030 msgr2=0x7f8ea00cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:09.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.872+0000 7f8ea8032640 1 --2- 172.21.15.105:0/3982048117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ea015b030 0x7f8ea00cadb0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f8e9001aa20 tx=0x7f8e9001abb0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.872+0000 7f8ea8032640 1 -- 172.21.15.105:0/3982048117 shutdown_connections 2023-11-28T18:31:09.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.872+0000 7f8ea8032640 1 --2- 172.21.15.105:0/3982048117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ea014af30 0x7f8ea00b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.872+0000 7f8ea8032640 1 --2- 172.21.15.105:0/3982048117 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8e840668d0 0x7f8e84068d90 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.872+0000 7f8ea8032640 1 --2- 172.21.15.105:0/3982048117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ea015b030 0x7f8ea00cadb0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.872+0000 7f8ea8032640 1 --2- 172.21.15.105:0/3982048117 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ea00b4290 0x7f8ea009b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:09.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.872+0000 7f8ea8032640 1 -- 172.21.15.105:0/3982048117 >> 172.21.15.105:0/3982048117 conn(0x7f8ea00ceb90 msgr2=0x7f8ea00aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:09.876 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.872+0000 7f8ea8032640 1 -- 172.21.15.105:0/3982048117 shutdown_connections 2023-11-28T18:31:09.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:09.872+0000 7f8ea8032640 1 -- 172.21.15.105:0/3982048117 wait complete. 2023-11-28T18:31:09.894 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T18:31:09.894 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T18:31:09.895 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2023-11-28T18:31:09.895 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T18:31:09.895 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T18:31:09.895 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T18:31:09.896 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T18:31:09.896 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T18:31:09.896 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:31:09.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:31:09.896 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2023-11-28T18:31:09.906 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T18:31:09.907 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T18:31:09.907 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T18:31:09.907 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:31:09.907 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-11-28T18:31:09.907 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:31:09.907 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:31:09.907 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:31:09.907 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2023-11-28T18:31:09.907 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:31:09.907 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:31:09.907 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2023-11-28T18:31:09.908 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:31:09.908 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:31:09.908 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T18:31:09.908 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:31:09.908 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-28T18:31:09.908 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 41952sec preferred_lft 41952sec 2023-11-28T18:31:09.908 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2023-11-28T18:31:09.908 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:31:09.908 INFO:teuthology.orchestra.run.smithi105.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-28T18:31:09.908 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 02:42:cc:d4:87:d2 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:31:09.908 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-28T18:31:09.908 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:31:09.909 INFO:teuthology.orchestra.run.smithi105.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T18:31:09.909 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-28T18:31:09.909 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T18:31:09.909 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:31:09.909 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-28T18:31:09.909 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:31:09.909 INFO:teuthology.orchestra.run.smithi105.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T18:31:09.909 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:31:09.909 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-28T18:31:09.909 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:31:09.909 INFO:teuthology.orchestra.run.smithi105.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T18:31:09.909 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:31:09.910 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-28T18:31:09.910 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:31:09.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:31:09.910 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T18:31:09.973 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-11-28T18:31:09.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:31:09.976 DEBUG:teuthology.orchestra.run.smithi105:> route 2023-11-28T18:31:10.029 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2023-11-28T18:31:10.030 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T18:31:10.030 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-28T18:31:10.030 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T18:31:10.030 INFO:teuthology.orchestra.run.smithi105.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-28T18:31:10.030 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T18:31:10.031 INFO:teuthology.orchestra.run.smithi105.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:31:10.031 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:31:10.031 INFO:teuthology.orchestra.run.smithi105.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:31:10.031 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T18:31:10.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:31:10.032 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-11-28T18:31:10.032 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:31:10.032 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T18:31:10.032 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T18:31:10.033 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T18:31:10.033 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:31:10.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:31:10.126 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-11-28T18:31:10.136 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T18:31:10.136 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T18:31:10.136 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:31:10.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:31:10.136 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2023-11-28T18:31:10.199 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-28T18:31:10.199 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:31:10.200 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T18:31:10.255 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:31:10.255 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:31:10.255 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T18:31:10.306 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:31:10.367 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:31:10.420 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:31:10.420 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:31:10.420 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:31:10.421 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:31:10.421 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:31:10.421 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:31:10.421 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:31:10.421 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:31:10.421 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:31:10.421 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:31:10.421 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:31:10.421 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:31:10.421 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:31:10.421 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:31:10.421 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:31:10.422 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:31:10.422 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:31:10.422 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:31:10.422 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:31:10.422 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:31:10.422 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:31:10.422 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:31:10.422 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:31:10.422 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:31:10.422 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:31:10.422 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:31:10.423 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:31:10.423 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:31:10.423 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:31:10.423 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:31:10.423 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:31:10.423 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:31:10.423 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:31:10.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:31:10.424 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:31:10.477 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T18:31:10.478 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-28T18:31:10.525 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:31:10.547 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:31:10.601 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:31:10.601 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:31:10.601 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:31:10.601 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:31:10.601 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:31:10.601 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:31:10.601 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:31:10.601 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:31:10.601 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:31:10.601 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:31:10.602 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:31:10.602 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:31:10.602 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:31:10.602 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:31:10.602 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:31:10.602 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:31:10.602 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:31:10.602 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:31:10.602 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:31:10.602 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:31:10.603 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:31:10.603 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:31:10.603 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:31:10.603 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:31:10.603 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:31:10.603 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:31:10.603 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:31:10.603 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:31:10.603 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:31:10.603 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:31:10.603 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:31:10.603 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:31:10.603 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:31:10.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:31:10.605 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:31:10.620 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:2023-11-28T18:31:10.620+0000 7fb9d91db580 -1 init, newargv = 0x5599f79c3e50 newargc=15 2023-11-28T18:31:10.620 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[65733]: starting ceph client 2023-11-28T18:31:10.627 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[65733]: starting fuse 2023-11-28T18:31:10.630 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-11-28T18:31:10.631 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-28T18:31:10.631 DEBUG:teuthology.orchestra.run.smithi105:> sudo stdin-killer -- python3 -c ' 2023-11-28T18:31:10.631 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2023-11-28T18:31:10.631 DEBUG:teuthology.orchestra.run.smithi105:> import re 2023-11-28T18:31:10.632 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-11-28T18:31:10.632 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2023-11-28T18:31:10.632 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:31:10.632 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2023-11-28T18:31:10.632 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-28T18:31:10.633 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2023-11-28T18:31:10.633 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-28T18:31:10.633 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:31:10.633 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2023-11-28T18:31:10.633 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2023-11-28T18:31:10.633 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2023-11-28T18:31:10.633 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2023-11-28T18:31:10.633 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:31:10.633 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2023-11-28T18:31:10.633 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T18:31:10.633 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T18:31:10.634 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T18:31:10.634 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2023-11-28T18:31:10.634 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2023-11-28T18:31:10.634 DEBUG:teuthology.orchestra.run.smithi105:> return f 2023-11-28T18:31:10.634 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T18:31:10.634 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:31:10.634 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.0")) 2023-11-28T18:31:10.634 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-11-28T18:31:10.761 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.0.65733.asok 2023-11-28T18:31:10.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-28T18:31:10.772 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.65733.asok 2023-11-28T18:31:10.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:31:10.772 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.65733.asok status 2023-11-28T18:31:10.947 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:31:10.947 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2023-11-28T18:31:10.947 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "47c1c547372ed5bff7984647dc200bac0e019caf", 2023-11-28T18:31:10.947 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev)", 2023-11-28T18:31:10.947 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "0", 2023-11-28T18:31:10.947 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2023-11-28T18:31:10.947 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-28T18:31:10.948 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "65733", 2023-11-28T18:31:10.948 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2023-11-28T18:31:10.948 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:31:10.948 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2023-11-28T18:31:10.948 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2023-11-28T18:31:10.948 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 8796, 2023-11-28T18:31:10.948 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2023-11-28T18:31:10.948 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2023-11-28T18:31:10.948 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2023-11-28T18:31:10.948 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 8796 2023-11-28T18:31:10.948 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:31:10.949 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-11-28T18:31:10.949 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-11-28T18:31:10.949 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-11-28T18:31:10.949 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 2782626755 2023-11-28T18:31:10.949 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:10.949 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:31:10.949 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-11-28T18:31:10.949 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-11-28T18:31:10.949 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-11-28T18:31:10.949 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 2782626755 2023-11-28T18:31:10.949 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:31:10.950 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.8796 192.168.0.1:0/2782626755", 2023-11-28T18:31:10.950 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/2782626755", 2023-11-28T18:31:10.950 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2023-11-28T18:31:10.950 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 119, 2023-11-28T18:31:10.950 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 117, 2023-11-28T18:31:10.950 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2023-11-28T18:31:10.950 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2023-11-28T18:31:10.950 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2023-11-28T18:31:10.950 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:31:10.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:31:10.962 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2023-11-28T18:31:10.962 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-28T18:31:10.972 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2023-11-28T18:31:10.972 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-28T18:31:10.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:31:10.972 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T18:31:11.035 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random True 2023-11-28T18:31:11.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.220+0000 7fbe22632640 1 Processor -- start 2023-11-28T18:31:11.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.220+0000 7fbe22632640 1 -- start start 2023-11-28T18:31:11.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.224+0000 7fbe22632640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe1c0ce2c0 0x7fbe1c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:11.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.224+0000 7fbe22632640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbe1c06cd20 0x7fbe1c0abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:11.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.224+0000 7fbe22632640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe1c0cde30 con 0x7fbe1c159610 2023-11-28T18:31:11.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.224+0000 7fbe22632640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe1c06d1b0 con 0x7fbe1c06cd20 2023-11-28T18:31:11.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.224+0000 7fbe22632640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c1599f0 con 0x7fbe1c0ce2c0 2023-11-28T18:31:11.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.224+0000 7fbe20bcf640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbe1c06cd20 0x7fbe1c0abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:11.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.224+0000 7fbe20bcf640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbe1c06cd20 0x7fbe1c0abd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55030/0 (socket says 172.21.15.105:55030) 2023-11-28T18:31:11.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.224+0000 7fbe1bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbe1c159610 0x7fbe1c0cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:47768/0 (socket says 172.21.15.105:47768) 2023-11-28T18:31:11.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.224+0000 7fbe20bcf640 1 -- 172.21.15.105:0/3599052142 learned_addr learned my addr 172.21.15.105:0/3599052142 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:11.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.224+0000 7fbe1b7fe640 1 --2- 172.21.15.105:0/3599052142 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe1c0ce2c0 0x7fbe1c06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:11.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.224+0000 7fbe20bcf640 1 -- 172.21.15.105:0/3599052142 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe1c0ce2c0 msgr2=0x7fbe1c06c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:11.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.224+0000 7fbe20bcf640 1 --2- 172.21.15.105:0/3599052142 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe1c0ce2c0 0x7fbe1c06c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.224+0000 7fbe20bcf640 1 -- 172.21.15.105:0/3599052142 >> v1:172.21.15.148:6789/0 conn(0x7fbe1c159610 legacy=0x7fbe1c0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:11.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.224+0000 7fbe20bcf640 1 -- 172.21.15.105:0/3599052142 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1c06de20 con 0x7fbe1c06cd20 2023-11-28T18:31:11.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.224+0000 7fbe20bcf640 1 --2- 172.21.15.105:0/3599052142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe1c06cd20 0x7fbe1c0abd80 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009e50 tx=0x7fbe1401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95520b08255ead05 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:11.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.228+0000 7fbe1affd640 1 -- 172.21.15.105:0/3599052142 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe140a5630 con 0x7fbe1c06cd20 2023-11-28T18:31:11.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.228+0000 7fbe1affd640 1 -- 172.21.15.105:0/3599052142 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe140a5790 con 0x7fbe1c06cd20 2023-11-28T18:31:11.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.228+0000 7fbe1affd640 1 -- 172.21.15.105:0/3599052142 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe140a5b20 con 0x7fbe1c06cd20 2023-11-28T18:31:11.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.228+0000 7fbe22632640 1 -- 172.21.15.105:0/3599052142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe1c06cd20 msgr2=0x7fbe1c0abd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:11.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.228+0000 7fbe22632640 1 --2- 172.21.15.105:0/3599052142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe1c06cd20 0x7fbe1c0abd80 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009e50 tx=0x7fbe1401c9e0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.228+0000 7fbe22632640 1 -- 172.21.15.105:0/3599052142 shutdown_connections 2023-11-28T18:31:11.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.228+0000 7fbe22632640 1 --2- 172.21.15.105:0/3599052142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe1c06cd20 0x7fbe1c0abd80 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.228+0000 7fbe22632640 1 --2- 172.21.15.105:0/3599052142 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe1c0ce2c0 0x7fbe1c06c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.228+0000 7fbe22632640 1 -- 172.21.15.105:0/3599052142 >> 172.21.15.105:0/3599052142 conn(0x7fbe1c0ceb90 msgr2=0x7fbe1c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:11.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.228+0000 7fbe22632640 1 -- 172.21.15.105:0/3599052142 shutdown_connections 2023-11-28T18:31:11.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.228+0000 7fbe22632640 1 -- 172.21.15.105:0/3599052142 wait complete. 2023-11-28T18:31:11.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.228+0000 7fbe22632640 1 Processor -- start 2023-11-28T18:31:11.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.228+0000 7fbe22632640 1 -- start start 2023-11-28T18:31:11.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.228+0000 7fbe22632640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe1c06cd20 0x7fbe1c0ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:11.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.228+0000 7fbe22632640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe1c0ce2c0 0x7fbe1c0adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:11.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.228+0000 7fbe22632640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe1c159610 0x7fbe1c09a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:11.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.228+0000 7fbe22632640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c09c850 con 0x7fbe1c06cd20 2023-11-28T18:31:11.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.228+0000 7fbe22632640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c0acf10 con 0x7fbe1c0ce2c0 2023-11-28T18:31:11.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.228+0000 7fbe22632640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c0ad1f0 con 0x7fbe1c159610 2023-11-28T18:31:11.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.228+0000 7fbe1bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe1c06cd20 0x7fbe1c0ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:11.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.228+0000 7fbe1bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe1c06cd20 0x7fbe1c0ac850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55038/0 (socket says 172.21.15.105:55038) 2023-11-28T18:31:11.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.228+0000 7fbe1bfff640 1 -- 172.21.15.105:0/3441306735 learned_addr learned my addr 172.21.15.105:0/3441306735 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:11.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.232+0000 7fbe1b7fe640 1 --2- 172.21.15.105:0/3441306735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe1c0ce2c0 0x7fbe1c0adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:11.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.232+0000 7fbe20bcf640 1 --2- 172.21.15.105:0/3441306735 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe1c159610 0x7fbe1c09a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:11.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.232+0000 7fbe1bfff640 1 -- 172.21.15.105:0/3441306735 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe1c159610 msgr2=0x7fbe1c09a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:11.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.232+0000 7fbe1bfff640 1 --2- 172.21.15.105:0/3441306735 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe1c159610 0x7fbe1c09a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.232+0000 7fbe1bfff640 1 -- 172.21.15.105:0/3441306735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe1c0ce2c0 msgr2=0x7fbe1c0adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:11.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.232+0000 7fbe1bfff640 1 --2- 172.21.15.105:0/3441306735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe1c0ce2c0 0x7fbe1c0adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.232+0000 7fbe1bfff640 1 -- 172.21.15.105:0/3441306735 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe14009b30 con 0x7fbe1c06cd20 2023-11-28T18:31:11.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.232+0000 7fbe1b7fe640 1 --2- 172.21.15.105:0/3441306735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe1c0ce2c0 0x7fbe1c0adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:11.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.232+0000 7fbe20bcf640 1 --2- 172.21.15.105:0/3441306735 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe1c159610 0x7fbe1c09a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:11.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.232+0000 7fbe1bfff640 1 --2- 172.21.15.105:0/3441306735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe1c06cd20 0x7fbe1c0ac850 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00c490 tx=0x7fbe0c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:11.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.232+0000 7fbe18ff9640 1 -- 172.21.15.105:0/3441306735 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c007480 con 0x7fbe1c06cd20 2023-11-28T18:31:11.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.232+0000 7fbe18ff9640 1 -- 172.21.15.105:0/3441306735 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0c0075e0 con 0x7fbe1c06cd20 2023-11-28T18:31:11.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.232+0000 7fbe18ff9640 1 -- 172.21.15.105:0/3441306735 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c004a10 con 0x7fbe1c06cd20 2023-11-28T18:31:11.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.232+0000 7fbe22632640 1 -- 172.21.15.105:0/3441306735 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe1c0ccff0 con 0x7fbe1c06cd20 2023-11-28T18:31:11.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.232+0000 7fbe22632640 1 -- 172.21.15.105:0/3441306735 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe1c0ad7b0 con 0x7fbe1c06cd20 2023-11-28T18:31:11.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.232+0000 7fbe22632640 1 -- 172.21.15.105:0/3441306735 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe1c067570 con 0x7fbe1c06cd20 2023-11-28T18:31:11.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.232+0000 7fbe18ff9640 1 -- 172.21.15.105:0/3441306735 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fbe0c004b70 con 0x7fbe1c06cd20 2023-11-28T18:31:11.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.232+0000 7fbe18ff9640 1 --2- 172.21.15.105:0/3441306735 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbe080668d0 0x7fbe08068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:11.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.232+0000 7fbe18ff9640 1 -- 172.21.15.105:0/3441306735 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbe0c083770 con 0x7fbe1c06cd20 2023-11-28T18:31:11.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.232+0000 7fbe1b7fe640 1 --2- 172.21.15.105:0/3441306735 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbe080668d0 0x7fbe08068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:11.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.236+0000 7fbe1b7fe640 1 --2- 172.21.15.105:0/3441306735 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbe080668d0 0x7fbe08068d90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c09a230 tx=0x7fbe10008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:11.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.236+0000 7fbe18ff9640 1 -- 172.21.15.105:0/3441306735 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbe0c050860 con 0x7fbe1c06cd20 2023-11-28T18:31:11.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.392+0000 7fbe22632640 1 -- 172.21.15.105:0/3441306735 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random}] v 0) v1 -- 0x7fbe1c0cf950 con 0x7fbe1c06cd20 2023-11-28T18:31:11.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.396+0000 7fbe18ff9640 1 -- 172.21.15.105:0/3441306735 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random}]=0 v60)=0 v60) v1 ==== 128+0+0 (secure 0 0 0) 0x7fbe0c053e80 con 0x7fbe1c06cd20 2023-11-28T18:31:11.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.400+0000 7fbe22632640 1 -- 172.21.15.105:0/3441306735 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbe080668d0 msgr2=0x7fbe08068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:11.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.400+0000 7fbe22632640 1 --2- 172.21.15.105:0/3441306735 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbe080668d0 0x7fbe08068d90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c09a230 tx=0x7fbe10008040 comp rx=0 tx=0).stop 2023-11-28T18:31:11.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.400+0000 7fbe22632640 1 -- 172.21.15.105:0/3441306735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe1c06cd20 msgr2=0x7fbe1c0ac850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:11.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.400+0000 7fbe22632640 1 --2- 172.21.15.105:0/3441306735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe1c06cd20 0x7fbe1c0ac850 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00c490 tx=0x7fbe0c00c950 comp rx=0 tx=0).stop 2023-11-28T18:31:11.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.400+0000 7fbe22632640 1 -- 172.21.15.105:0/3441306735 shutdown_connections 2023-11-28T18:31:11.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.400+0000 7fbe22632640 1 --2- 172.21.15.105:0/3441306735 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe1c159610 0x7fbe1c09a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.400+0000 7fbe22632640 1 --2- 172.21.15.105:0/3441306735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe1c0ce2c0 0x7fbe1c0adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.400+0000 7fbe22632640 1 --2- 172.21.15.105:0/3441306735 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbe080668d0 0x7fbe08068d90 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.400+0000 7fbe22632640 1 --2- 172.21.15.105:0/3441306735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe1c06cd20 0x7fbe1c0ac850 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.400+0000 7fbe22632640 1 -- 172.21.15.105:0/3441306735 >> 172.21.15.105:0/3441306735 conn(0x7fbe1c0ceb90 msgr2=0x7fbe1c0cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:11.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.400+0000 7fbe22632640 1 -- 172.21.15.105:0/3441306735 shutdown_connections 2023-11-28T18:31:11.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.400+0000 7fbe22632640 1 -- 172.21.15.105:0/3441306735 wait complete. 2023-11-28T18:31:11.420 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_distributed True 2023-11-28T18:31:11.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.560+0000 7f19f091d640 1 Processor -- start 2023-11-28T18:31:11.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.560+0000 7f19f091d640 1 -- start start 2023-11-28T18:31:11.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.560+0000 7f19f091d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f19ec159f40 0x7f19ec0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:11.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.560+0000 7f19f091d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19ec0cc760 0x7f19ec0ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:11.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.560+0000 7f19f091d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f19ec0ad680 0x7f19ec15da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:11.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.560+0000 7f19f091d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ec06bf50 con 0x7f19ec159f40 2023-11-28T18:31:11.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.560+0000 7f19f091d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ec0ccbc0 con 0x7f19ec0ad680 2023-11-28T18:31:11.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.560+0000 7f19f091d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec15a320 con 0x7f19ec0cc760 2023-11-28T18:31:11.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.560+0000 7f19ea59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f19ec159f40 0x7f19ec0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.560+0000 7f19ea59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f19ec159f40 0x7f19ec0caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55046/0 (socket says 172.21.15.105:55046) 2023-11-28T18:31:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.560+0000 7f19ea59c640 1 -- 172.21.15.105:0/3434758461 learned_addr learned my addr 172.21.15.105:0/3434758461 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.560+0000 7f19e1d9b640 1 --2- 172.21.15.105:0/3434758461 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19ec0cc760 0x7f19ec0ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.560+0000 7f19ead9d640 1 --2- 172.21.15.105:0/3434758461 >> v2:172.21.15.148:3300/0 conn(0x7f19ec0ad680 0x7f19ec15da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.560+0000 7f19ea59c640 1 -- 172.21.15.105:0/3434758461 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19ec0cc760 msgr2=0x7f19ec0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.560+0000 7f19ea59c640 1 --2- 172.21.15.105:0/3434758461 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19ec0cc760 0x7f19ec0ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.560+0000 7f19ea59c640 1 -- 172.21.15.105:0/3434758461 >> v2:172.21.15.148:3300/0 conn(0x7f19ec0ad680 msgr2=0x7f19ec15da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.560+0000 7f19ea59c640 1 --2- 172.21.15.105:0/3434758461 >> v2:172.21.15.148:3300/0 conn(0x7f19ec0ad680 0x7f19ec15da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.560+0000 7f19ea59c640 1 -- 172.21.15.105:0/3434758461 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec071380 con 0x7f19ec159f40 2023-11-28T18:31:11.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.564+0000 7f19ea59c640 1 --2- 172.21.15.105:0/3434758461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19ec159f40 0x7f19ec0caf60 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0724d0 tx=0x7f19dc01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7115d1c05aa5df63 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:11.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.564+0000 7f19e9d9b640 1 -- 172.21.15.105:0/3434758461 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc0ab600 con 0x7f19ec159f40 2023-11-28T18:31:11.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.564+0000 7f19e9d9b640 1 -- 172.21.15.105:0/3434758461 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc0ab760 con 0x7f19ec159f40 2023-11-28T18:31:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.564+0000 7f19e9d9b640 1 -- 172.21.15.105:0/3434758461 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc0abaf0 con 0x7f19ec159f40 2023-11-28T18:31:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.564+0000 7f19f091d640 1 -- 172.21.15.105:0/3434758461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19ec159f40 msgr2=0x7f19ec0caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.564+0000 7f19f091d640 1 --2- 172.21.15.105:0/3434758461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19ec159f40 0x7f19ec0caf60 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0724d0 tx=0x7f19dc01c440 comp rx=0 tx=0).stop 2023-11-28T18:31:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.564+0000 7f19f091d640 1 -- 172.21.15.105:0/3434758461 shutdown_connections 2023-11-28T18:31:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.564+0000 7f19f091d640 1 --2- 172.21.15.105:0/3434758461 >> v2:172.21.15.148:3300/0 conn(0x7f19ec0ad680 0x7f19ec15da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.564+0000 7f19f091d640 1 --2- 172.21.15.105:0/3434758461 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19ec0cc760 0x7f19ec0ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.564+0000 7f19f091d640 1 --2- 172.21.15.105:0/3434758461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19ec159f40 0x7f19ec0caf60 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.564+0000 7f19f091d640 1 -- 172.21.15.105:0/3434758461 >> 172.21.15.105:0/3434758461 conn(0x7f19ec0ceb90 msgr2=0x7f19ec0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.564+0000 7f19f091d640 1 -- 172.21.15.105:0/3434758461 shutdown_connections 2023-11-28T18:31:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.564+0000 7f19f091d640 1 -- 172.21.15.105:0/3434758461 wait complete. 2023-11-28T18:31:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19f091d640 1 Processor -- start 2023-11-28T18:31:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19f091d640 1 -- start start 2023-11-28T18:31:11.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19f091d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19ec0ad680 0x7f19ec14ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:11.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19f091d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19ec0cc760 0x7f19ec151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:11.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19f091d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19ec159f40 0x7f19ec163ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:11.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19f091d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec152e70 con 0x7f19ec0cc760 2023-11-28T18:31:11.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19f091d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0b1b70 con 0x7f19ec159f40 2023-11-28T18:31:11.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19f091d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec1525d0 con 0x7f19ec0ad680 2023-11-28T18:31:11.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19e1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19ec0cc760 0x7f19ec151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:11.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19e1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19ec0cc760 0x7f19ec151b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55050/0 (socket says 172.21.15.105:55050) 2023-11-28T18:31:11.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19e1d9b640 1 -- 172.21.15.105:0/685488599 learned_addr learned my addr 172.21.15.105:0/685488599 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:11.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19ea59c640 1 --2- 172.21.15.105:0/685488599 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19ec0ad680 0x7f19ec14ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:11.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19ead9d640 1 --2- 172.21.15.105:0/685488599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19ec159f40 0x7f19ec163ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:11.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19e1d9b640 1 -- 172.21.15.105:0/685488599 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19ec0ad680 msgr2=0x7f19ec14ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:11.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19e1d9b640 1 --2- 172.21.15.105:0/685488599 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19ec0ad680 0x7f19ec14ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19e1d9b640 1 -- 172.21.15.105:0/685488599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19ec159f40 msgr2=0x7f19ec163ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:11.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19e1d9b640 1 --2- 172.21.15.105:0/685488599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19ec159f40 0x7f19ec163ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19e1d9b640 1 -- 172.21.15.105:0/685488599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19dc0094d0 con 0x7f19ec0cc760 2023-11-28T18:31:11.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19e1d9b640 1 --2- 172.21.15.105:0/685488599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19ec0cc760 0x7f19ec151b10 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f19d400d800 tx=0x7f19d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19e37fe640 1 -- 172.21.15.105:0/685488599 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d400b6c0 con 0x7f19ec0cc760 2023-11-28T18:31:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19e37fe640 1 -- 172.21.15.105:0/685488599 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d400b820 con 0x7f19ec0cc760 2023-11-28T18:31:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19f091d640 1 -- 172.21.15.105:0/685488599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec1528b0 con 0x7f19ec0cc760 2023-11-28T18:31:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19e37fe640 1 -- 172.21.15.105:0/685488599 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d400bb90 con 0x7f19ec0cc760 2023-11-28T18:31:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.568+0000 7f19f091d640 1 -- 172.21.15.105:0/685488599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec14e2f0 con 0x7f19ec0cc760 2023-11-28T18:31:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.572+0000 7f19f091d640 1 -- 172.21.15.105:0/685488599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ec067570 con 0x7f19ec0cc760 2023-11-28T18:31:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.572+0000 7f19e37fe640 1 -- 172.21.15.105:0/685488599 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f19d400bcf0 con 0x7f19ec0cc760 2023-11-28T18:31:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.572+0000 7f19e37fe640 1 --2- 172.21.15.105:0/685488599 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f19cc066900 0x7f19cc068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.572+0000 7f19e37fe640 1 -- 172.21.15.105:0/685488599 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f19d4084920 con 0x7f19ec0cc760 2023-11-28T18:31:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.572+0000 7f19ea59c640 1 --2- 172.21.15.105:0/685488599 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f19cc066900 0x7f19cc068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.572+0000 7f19ea59c640 1 --2- 172.21.15.105:0/685488599 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f19cc066900 0x7f19cc068dc0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0af000 tx=0x7f19dc0b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:11.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.576+0000 7f19e37fe640 1 -- 172.21.15.105:0/685488599 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f19d4051ac0 con 0x7f19ec0cc760 2023-11-28T18:31:11.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.728+0000 7f19f091d640 1 -- 172.21.15.105:0/685488599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_distributed}] v 0) v1 -- 0x7f19ec159770 con 0x7f19ec0cc760 2023-11-28T18:31:11.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.732+0000 7f19e37fe640 1 -- 172.21.15.105:0/685488599 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_distributed}]=0 v61)=0 v61) v1 ==== 133+0+0 (secure 0 0 0) 0x7f19d40550e0 con 0x7f19ec0cc760 2023-11-28T18:31:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.732+0000 7f19f091d640 1 -- 172.21.15.105:0/685488599 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f19cc066900 msgr2=0x7f19cc068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.732+0000 7f19f091d640 1 --2- 172.21.15.105:0/685488599 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f19cc066900 0x7f19cc068dc0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0af000 tx=0x7f19dc0b6040 comp rx=0 tx=0).stop 2023-11-28T18:31:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.732+0000 7f19f091d640 1 -- 172.21.15.105:0/685488599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19ec0cc760 msgr2=0x7f19ec151b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.732+0000 7f19f091d640 1 --2- 172.21.15.105:0/685488599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19ec0cc760 0x7f19ec151b10 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f19d400d800 tx=0x7f19d400dcc0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.732+0000 7f19f091d640 1 -- 172.21.15.105:0/685488599 shutdown_connections 2023-11-28T18:31:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.732+0000 7f19f091d640 1 --2- 172.21.15.105:0/685488599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19ec159f40 0x7f19ec163ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.732+0000 7f19f091d640 1 --2- 172.21.15.105:0/685488599 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f19cc066900 0x7f19cc068dc0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.732+0000 7f19f091d640 1 --2- 172.21.15.105:0/685488599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19ec0cc760 0x7f19ec151b10 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.732+0000 7f19f091d640 1 --2- 172.21.15.105:0/685488599 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19ec0ad680 0x7f19ec14ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.732+0000 7f19f091d640 1 -- 172.21.15.105:0/685488599 >> 172.21.15.105:0/685488599 conn(0x7f19ec0ceb90 msgr2=0x7f19ec06d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.736+0000 7f19f091d640 1 -- 172.21.15.105:0/685488599 shutdown_connections 2023-11-28T18:31:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.736+0000 7f19f091d640 1 -- 172.21.15.105:0/685488599 wait complete. 2023-11-28T18:31:11.754 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random_max 1.0 2023-11-28T18:31:11.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd39086b640 1 Processor -- start 2023-11-28T18:31:11.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd39086b640 1 -- start start 2023-11-28T18:31:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd39086b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd38c154070 0x7fd38c15d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd39086b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38c0cc7f0 con 0x7fd38c06bf50 2023-11-28T18:31:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd39086b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38c15dd10 con 0x7fd38c0adb80 2023-11-28T18:31:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd39086b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c072540 con 0x7fd38c154070 2023-11-28T18:31:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd389d9b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd38c06bf50 0x7fd38c0ce310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50358/0 (socket says 172.21.15.105:50358) 2023-11-28T18:31:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd389d9b640 1 -- 172.21.15.105:0/520778508 learned_addr learned my addr 172.21.15.105:0/520778508 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:11.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd38a59c640 1 --2- 172.21.15.105:0/520778508 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd38c154070 0x7fd38c15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:11.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd38959a640 1 -- 172.21.15.105:0/520778508 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975014044 0 0) 0x7fd38c0cc7f0 con 0x7fd38c06bf50 2023-11-28T18:31:11.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd38959a640 1 -- 172.21.15.105:0/520778508 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd374003530 con 0x7fd38c06bf50 2023-11-28T18:31:11.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd38959a640 1 -- 172.21.15.105:0/520778508 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2926566160 0 0) 0x7fd38c15dd10 con 0x7fd38c0adb80 2023-11-28T18:31:11.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd38959a640 1 -- 172.21.15.105:0/520778508 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd38c0cc7f0 con 0x7fd38c0adb80 2023-11-28T18:31:11.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd38959a640 1 -- 172.21.15.105:0/520778508 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 577016421 0 0) 0x7fd374003530 con 0x7fd38c06bf50 2023-11-28T18:31:11.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd38959a640 1 -- 172.21.15.105:0/520778508 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd38c154070 msgr2=0x7fd38c15d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:11.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd38959a640 1 --2- 172.21.15.105:0/520778508 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd38c154070 0x7fd38c15d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd38959a640 1 -- 172.21.15.105:0/520778508 >> v1:172.21.15.148:6789/0 conn(0x7fd38c0adb80 legacy=0x7fd38c0acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:11.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd38959a640 1 -- 172.21.15.105:0/520778508 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c15dff0 con 0x7fd38c06bf50 2023-11-28T18:31:11.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd38959a640 1 -- 172.21.15.105:0/520778508 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fd380001680 con 0x7fd38c06bf50 2023-11-28T18:31:11.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd38959a640 1 -- 172.21.15.105:0/520778508 >> v1:172.21.15.105:6789/0 conn(0x7fd38c06bf50 legacy=0x7fd38c0ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:11.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd38959a640 1 --2- 172.21.15.105:0/520778508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd374003b40 0x7fd374003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:11.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd38959a640 1 --2- 172.21.15.105:0/520778508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3740047d0 0x7fd374004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:11.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd38959a640 1 --2- 172.21.15.105:0/520778508 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd38c154070 0x7fd374005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:11.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd38959a640 1 -- 172.21.15.105:0/520778508 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c072540 con 0x7fd3740047d0 2023-11-28T18:31:11.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd38959a640 1 -- 172.21.15.105:0/520778508 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37400ba40 con 0x7fd374003b40 2023-11-28T18:31:11.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd38959a640 1 -- 172.21.15.105:0/520778508 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37400bbd0 con 0x7fd38c154070 2023-11-28T18:31:11.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd389d9b640 1 --2- 172.21.15.105:0/520778508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3740047d0 0x7fd374004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:11.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd38a59c640 1 --2- 172.21.15.105:0/520778508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd374003b40 0x7fd374003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:11.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd389d9b640 1 -- 172.21.15.105:0/520778508 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd38c154070 msgr2=0x7fd374005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:11.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd389d9b640 1 --2- 172.21.15.105:0/520778508 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd38c154070 0x7fd374005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd389d9b640 1 -- 172.21.15.105:0/520778508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd374003b40 msgr2=0x7fd374003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:11.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd389d9b640 1 --2- 172.21.15.105:0/520778508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd374003b40 0x7fd374003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.896+0000 7fd389d9b640 1 -- 172.21.15.105:0/520778508 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd374003530 con 0x7fd3740047d0 2023-11-28T18:31:11.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.900+0000 7fd389d9b640 1 --2- 172.21.15.105:0/520778508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3740047d0 0x7fd374004c50 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fd38c06c370 tx=0x7fd38001cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3be0493314792b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:11.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.900+0000 7fd38959a640 1 -- 172.21.15.105:0/520778508 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3800058e0 con 0x7fd3740047d0 2023-11-28T18:31:11.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.900+0000 7fd38959a640 1 -- 172.21.15.105:0/520778508 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd380005a60 con 0x7fd3740047d0 2023-11-28T18:31:11.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.900+0000 7fd38959a640 1 -- 172.21.15.105:0/520778508 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380005e10 con 0x7fd3740047d0 2023-11-28T18:31:11.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.900+0000 7fd39086b640 1 -- 172.21.15.105:0/520778508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3740047d0 msgr2=0x7fd374004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:11.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.900+0000 7fd39086b640 1 --2- 172.21.15.105:0/520778508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3740047d0 0x7fd374004c50 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fd38c06c370 tx=0x7fd38001cb40 comp rx=0 tx=0).stop 2023-11-28T18:31:11.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.900+0000 7fd38ad9d640 1 -- 172.21.15.105:0/520778508 reap_dead start 2023-11-28T18:31:11.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.900+0000 7fd39086b640 1 -- 172.21.15.105:0/520778508 shutdown_connections 2023-11-28T18:31:11.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.900+0000 7fd39086b640 1 -- 172.21.15.105:0/520778508 >> 172.21.15.105:0/520778508 conn(0x7fd38c0ceb90 msgr2=0x7fd38c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:11.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.904+0000 7fd39086b640 1 -- 172.21.15.105:0/520778508 shutdown_connections 2023-11-28T18:31:11.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.904+0000 7fd39086b640 1 -- 172.21.15.105:0/520778508 wait complete. 2023-11-28T18:31:11.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.904+0000 7fd39086b640 1 Processor -- start 2023-11-28T18:31:11.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.904+0000 7fd39086b640 1 -- start start 2023-11-28T18:31:11.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.904+0000 7fd39086b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3740047d0 0x7fd38c13fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:11.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.904+0000 7fd39086b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38c151db0 0x7fd38c0c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:11.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.904+0000 7fd39086b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd38c0c8510 0x7fd38c142790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:11.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.904+0000 7fd39086b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c140700 con 0x7fd3740047d0 2023-11-28T18:31:11.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.904+0000 7fd39086b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c1403e0 con 0x7fd38c151db0 2023-11-28T18:31:11.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.904+0000 7fd39086b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c1522b0 con 0x7fd38c0c8510 2023-11-28T18:31:11.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.904+0000 7fd38a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3740047d0 0x7fd38c13fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:11.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.904+0000 7fd38a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3740047d0 0x7fd38c13fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55056/0 (socket says 172.21.15.105:55056) 2023-11-28T18:31:11.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.904+0000 7fd38a59c640 1 -- 172.21.15.105:0/1136321203 learned_addr learned my addr 172.21.15.105:0/1136321203 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:11.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.904+0000 7fd38ad9d640 1 --2- 172.21.15.105:0/1136321203 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd38c0c8510 0x7fd38c142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:11.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.904+0000 7fd389d9b640 1 --2- 172.21.15.105:0/1136321203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38c151db0 0x7fd38c0c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:11.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.904+0000 7fd38a59c640 1 -- 172.21.15.105:0/1136321203 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd38c0c8510 msgr2=0x7fd38c142790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:11.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.904+0000 7fd38a59c640 1 --2- 172.21.15.105:0/1136321203 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd38c0c8510 0x7fd38c142790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.904+0000 7fd38a59c640 1 -- 172.21.15.105:0/1136321203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38c151db0 msgr2=0x7fd38c0c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:11.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.904+0000 7fd38a59c640 1 --2- 172.21.15.105:0/1136321203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38c151db0 0x7fd38c0c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:11.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.904+0000 7fd38a59c640 1 -- 172.21.15.105:0/1136321203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c152590 con 0x7fd3740047d0 2023-11-28T18:31:11.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.904+0000 7fd38a59c640 1 --2- 172.21.15.105:0/1136321203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3740047d0 0x7fd38c13fea0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fd37c011600 tx=0x7fd37c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.908+0000 7fd37affd640 1 -- 172.21.15.105:0/1136321203 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c004220 con 0x7fd3740047d0 2023-11-28T18:31:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.908+0000 7fd37affd640 1 -- 172.21.15.105:0/1136321203 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c004380 con 0x7fd3740047d0 2023-11-28T18:31:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.908+0000 7fd39086b640 1 -- 172.21.15.105:0/1136321203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38c152870 con 0x7fd3740047d0 2023-11-28T18:31:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.908+0000 7fd37affd640 1 -- 172.21.15.105:0/1136321203 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c00f920 con 0x7fd3740047d0 2023-11-28T18:31:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.908+0000 7fd39086b640 1 -- 172.21.15.105:0/1136321203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38c142cd0 con 0x7fd3740047d0 2023-11-28T18:31:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.908+0000 7fd39086b640 1 -- 172.21.15.105:0/1136321203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38c152d20 con 0x7fd3740047d0 2023-11-28T18:31:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.908+0000 7fd37affd640 1 -- 172.21.15.105:0/1136321203 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd37c00fa80 con 0x7fd3740047d0 2023-11-28T18:31:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.908+0000 7fd37affd640 1 --2- 172.21.15.105:0/1136321203 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd36c066900 0x7fd36c068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.908+0000 7fd37affd640 1 -- 172.21.15.105:0/1136321203 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd37c0845c0 con 0x7fd3740047d0 2023-11-28T18:31:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.908+0000 7fd389d9b640 1 --2- 172.21.15.105:0/1136321203 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd36c066900 0x7fd36c068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.912+0000 7fd389d9b640 1 --2- 172.21.15.105:0/1136321203 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd36c066900 0x7fd36c068dc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd3800aa000 tx=0x7fd380001a00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:11.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:11.912+0000 7fd37affd640 1 -- 172.21.15.105:0/1136321203 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd37c051770 con 0x7fd3740047d0 2023-11-28T18:31:12.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:12.068+0000 7fd39086b640 1 -- 172.21.15.105:0/1136321203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random_max}] v 0) v1 -- 0x7fd38c067570 con 0x7fd3740047d0 2023-11-28T18:31:12.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:12.072+0000 7fd37affd640 1 -- 172.21.15.105:0/1136321203 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random_max}]=0 v62)=0 v62) v1 ==== 131+0+0 (secure 0 0 0) 0x7fd37c054d90 con 0x7fd3740047d0 2023-11-28T18:31:12.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:12.076+0000 7fd39086b640 1 -- 172.21.15.105:0/1136321203 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd36c066900 msgr2=0x7fd36c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:12.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:12.076+0000 7fd39086b640 1 --2- 172.21.15.105:0/1136321203 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd36c066900 0x7fd36c068dc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd3800aa000 tx=0x7fd380001a00 comp rx=0 tx=0).stop 2023-11-28T18:31:12.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:12.076+0000 7fd39086b640 1 -- 172.21.15.105:0/1136321203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3740047d0 msgr2=0x7fd38c13fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:12.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:12.076+0000 7fd39086b640 1 --2- 172.21.15.105:0/1136321203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3740047d0 0x7fd38c13fea0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fd37c011600 tx=0x7fd37c011ac0 comp rx=0 tx=0).stop 2023-11-28T18:31:12.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:12.076+0000 7fd39086b640 1 -- 172.21.15.105:0/1136321203 shutdown_connections 2023-11-28T18:31:12.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:12.076+0000 7fd39086b640 1 --2- 172.21.15.105:0/1136321203 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd38c0c8510 0x7fd38c142790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:12.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:12.076+0000 7fd39086b640 1 --2- 172.21.15.105:0/1136321203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38c151db0 0x7fd38c0c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:12.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:12.076+0000 7fd39086b640 1 --2- 172.21.15.105:0/1136321203 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd36c066900 0x7fd36c068dc0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:12.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:12.076+0000 7fd39086b640 1 --2- 172.21.15.105:0/1136321203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3740047d0 0x7fd38c13fea0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:12.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:12.076+0000 7fd39086b640 1 -- 172.21.15.105:0/1136321203 >> 172.21.15.105:0/1136321203 conn(0x7fd38c0ceb90 msgr2=0x7fd38c0cdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:12.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:12.076+0000 7fd39086b640 1 -- 172.21.15.105:0/1136321203 shutdown_connections 2023-11-28T18:31:12.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:12.076+0000 7fd39086b640 1 -- 172.21.15.105:0/1136321203 wait complete. 2023-11-28T18:31:12.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:31:12.095 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-28T18:31:12.095 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:31:12.095 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:31:12.096 DEBUG:teuthology.orchestra.run.smithi105:> # Use up a random number of inode numbers so the ephemeral pinning is not the same every test. 2023-11-28T18:31:12.096 DEBUG:teuthology.orchestra.run.smithi105:> mkdir .inode_number_thrash 2023-11-28T18:31:12.096 DEBUG:teuthology.orchestra.run.smithi105:> count=$((RANDOM % 1024)) 2023-11-28T18:31:12.096 DEBUG:teuthology.orchestra.run.smithi105:> for ((i = 0; i < count; i++)); do touch .inode_number_thrash/$i; done 2023-11-28T18:31:12.096 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf .inode_number_thrash 2023-11-28T18:31:12.096 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:31:13.380 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-11-28T18:31:13.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.520+0000 7f7dda432640 1 Processor -- start 2023-11-28T18:31:13.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.520+0000 7f7dda432640 1 -- start start 2023-11-28T18:31:13.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.520+0000 7f7dda432640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7dd4159f40 0x7f7dd40caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:13.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.520+0000 7f7dda432640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7dd40cc760 0x7f7dd40ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:13.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.520+0000 7f7dda432640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dd406bf50 con 0x7f7dd40ad680 2023-11-28T18:31:13.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.520+0000 7f7dda432640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd40ccbc0 con 0x7f7dd40cc760 2023-11-28T18:31:13.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.520+0000 7f7dda432640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd415a320 con 0x7f7dd4159f40 2023-11-28T18:31:13.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.520+0000 7f7dd37fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7dd40cc760 0x7f7dd40ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:13.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.520+0000 7f7dd37fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7dd40cc760 0x7f7dd40ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55066/0 (socket says 172.21.15.105:55066) 2023-11-28T18:31:13.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.520+0000 7f7dd37fe640 1 -- 172.21.15.105:0/450636997 learned_addr learned my addr 172.21.15.105:0/450636997 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:13.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.524+0000 7f7dd3fff640 1 --2- 172.21.15.105:0/450636997 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7dd4159f40 0x7f7dd40caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:13.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.524+0000 7f7dd37fe640 1 -- 172.21.15.105:0/450636997 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7dd4159f40 msgr2=0x7f7dd40caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:13.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.524+0000 7f7dd37fe640 1 --2- 172.21.15.105:0/450636997 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7dd4159f40 0x7f7dd40caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:13.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.524+0000 7f7dd37fe640 1 -- 172.21.15.105:0/450636997 >> v1:172.21.15.148:6789/0 conn(0x7f7dd40ad680 legacy=0x7f7dd415da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:13.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.524+0000 7f7dd37fe640 1 -- 172.21.15.105:0/450636997 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd4071380 con 0x7f7dd40cc760 2023-11-28T18:31:13.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.524+0000 7f7dd2ffd640 1 -- 172.21.15.105:0/450636997 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1130171444 0 0) 0x7f7dd406bf50 con 0x7f7dd40ad680 2023-11-28T18:31:13.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.524+0000 7f7dd37fe640 1 --2- 172.21.15.105:0/450636997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7dd40cc760 0x7f7dd40ad140 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800bb10 tx=0x7f7dc801eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2786dbd171ecf4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:13.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.524+0000 7f7dd2ffd640 1 -- 172.21.15.105:0/450636997 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc80a7c20 con 0x7f7dd40cc760 2023-11-28T18:31:13.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.524+0000 7f7dd2ffd640 1 -- 172.21.15.105:0/450636997 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc80a7d80 con 0x7f7dd40cc760 2023-11-28T18:31:13.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.524+0000 7f7dd2ffd640 1 -- 172.21.15.105:0/450636997 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc80af660 con 0x7f7dd40cc760 2023-11-28T18:31:13.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dda432640 1 -- 172.21.15.105:0/450636997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7dd40cc760 msgr2=0x7f7dd40ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:13.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dda432640 1 --2- 172.21.15.105:0/450636997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7dd40cc760 0x7f7dd40ad140 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800bb10 tx=0x7f7dc801eb50 comp rx=0 tx=0).stop 2023-11-28T18:31:13.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dda432640 1 -- 172.21.15.105:0/450636997 shutdown_connections 2023-11-28T18:31:13.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dda432640 1 --2- 172.21.15.105:0/450636997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7dd40cc760 0x7f7dd40ad140 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:13.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dda432640 1 --2- 172.21.15.105:0/450636997 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7dd4159f40 0x7f7dd40caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:13.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dda432640 1 -- 172.21.15.105:0/450636997 >> 172.21.15.105:0/450636997 conn(0x7f7dd40ceb90 msgr2=0x7f7dd40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:13.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dda432640 1 -- 172.21.15.105:0/450636997 shutdown_connections 2023-11-28T18:31:13.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dda432640 1 -- 172.21.15.105:0/450636997 wait complete. 2023-11-28T18:31:13.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dda432640 1 Processor -- start 2023-11-28T18:31:13.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dda432640 1 -- start start 2023-11-28T18:31:13.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dda432640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dd40ad680 0x7f7dd4156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:13.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dda432640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7dd40cc760 0x7f7dd4153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:13.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dda432640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7dd4159f40 0x7f7dd413be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:13.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dda432640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd40afb50 con 0x7f7dd40cc760 2023-11-28T18:31:13.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dda432640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd414ec20 con 0x7f7dd40ad680 2023-11-28T18:31:13.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dda432640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd40af1b0 con 0x7f7dd4159f40 2023-11-28T18:31:13.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dd37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7dd40cc760 0x7f7dd4153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:13.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dd37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7dd40cc760 0x7f7dd4153670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55076/0 (socket says 172.21.15.105:55076) 2023-11-28T18:31:13.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dd37fe640 1 -- 172.21.15.105:0/3292437350 learned_addr learned my addr 172.21.15.105:0/3292437350 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:13.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dd3fff640 1 --2- 172.21.15.105:0/3292437350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dd40ad680 0x7f7dd4156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:13.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dd89cf640 1 --2- 172.21.15.105:0/3292437350 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7dd4159f40 0x7f7dd413be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:13.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dd3fff640 1 -- 172.21.15.105:0/3292437350 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7dd4159f40 msgr2=0x7f7dd413be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:13.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dd3fff640 1 --2- 172.21.15.105:0/3292437350 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7dd4159f40 0x7f7dd413be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:13.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dd3fff640 1 -- 172.21.15.105:0/3292437350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7dd40cc760 msgr2=0x7f7dd4153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:13.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dd3fff640 1 --2- 172.21.15.105:0/3292437350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7dd40cc760 0x7f7dd4153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:13.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dd3fff640 1 -- 172.21.15.105:0/3292437350 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc800b770 con 0x7f7dd40ad680 2023-11-28T18:31:13.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.528+0000 7f7dd37fe640 1 --2- 172.21.15.105:0/3292437350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7dd40cc760 0x7f7dd4153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:31:13.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.532+0000 7f7dd3fff640 1 --2- 172.21.15.105:0/3292437350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dd40ad680 0x7f7dd4156060 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400e870 tx=0x7f7dc400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:13.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.532+0000 7f7dd0ff9640 1 -- 172.21.15.105:0/3292437350 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc400ee60 con 0x7f7dd40ad680 2023-11-28T18:31:13.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.532+0000 7f7dd0ff9640 1 -- 172.21.15.105:0/3292437350 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc400c940 con 0x7f7dd40ad680 2023-11-28T18:31:13.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.532+0000 7f7dda432640 1 -- 172.21.15.105:0/3292437350 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd40af490 con 0x7f7dd40ad680 2023-11-28T18:31:13.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.532+0000 7f7dd0ff9640 1 -- 172.21.15.105:0/3292437350 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc400ccd0 con 0x7f7dd40ad680 2023-11-28T18:31:13.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.532+0000 7f7dda432640 1 -- 172.21.15.105:0/3292437350 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd4156720 con 0x7f7dd40ad680 2023-11-28T18:31:13.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.532+0000 7f7dda432640 1 -- 172.21.15.105:0/3292437350 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd4067570 con 0x7f7dd40ad680 2023-11-28T18:31:13.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.532+0000 7f7dd0ff9640 1 -- 172.21.15.105:0/3292437350 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f7dc4013470 con 0x7f7dd40ad680 2023-11-28T18:31:13.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.532+0000 7f7dd0ff9640 1 --2- 172.21.15.105:0/3292437350 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7dc00668d0 0x7f7dc0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:13.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.532+0000 7f7dd0ff9640 1 -- 172.21.15.105:0/3292437350 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7dc4084590 con 0x7f7dd40ad680 2023-11-28T18:31:13.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.532+0000 7f7dd37fe640 1 --2- 172.21.15.105:0/3292437350 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7dc00668d0 0x7f7dc0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:13.536 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.532+0000 7f7dd37fe640 1 --2- 172.21.15.105:0/3292437350 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7dc00668d0 0x7f7dc0068d90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8000c10 tx=0x7f7dc80a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:13.538 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.536+0000 7f7dd0ff9640 1 -- 172.21.15.105:0/3292437350 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7dc40516a0 con 0x7f7dd40ad680 2023-11-28T18:31:13.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:13.720+0000 7f7dda432640 1 -- 172.21.15.105:0/3292437350 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f7dd40cf950 con 0x7f7dd40ad680 2023-11-28T18:31:14.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.148+0000 7f7dd0ff9640 1 -- 172.21.15.105:0/3292437350 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v120) v1 ==== 105+0+0 (secure 0 0 0) 0x7f7dc4054cc0 con 0x7f7dd40ad680 2023-11-28T18:31:14.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.152+0000 7f7dda432640 1 -- 172.21.15.105:0/3292437350 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7dc00668d0 msgr2=0x7f7dc0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:14.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.152+0000 7f7dda432640 1 --2- 172.21.15.105:0/3292437350 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7dc00668d0 0x7f7dc0068d90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8000c10 tx=0x7f7dc80a9070 comp rx=0 tx=0).stop 2023-11-28T18:31:14.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.152+0000 7f7dda432640 1 -- 172.21.15.105:0/3292437350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dd40ad680 msgr2=0x7f7dd4156060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:14.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.152+0000 7f7dda432640 1 --2- 172.21.15.105:0/3292437350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dd40ad680 0x7f7dd4156060 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400e870 tx=0x7f7dc400ed30 comp rx=0 tx=0).stop 2023-11-28T18:31:14.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.152+0000 7f7dda432640 1 -- 172.21.15.105:0/3292437350 shutdown_connections 2023-11-28T18:31:14.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.152+0000 7f7dda432640 1 --2- 172.21.15.105:0/3292437350 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7dd4159f40 0x7f7dd413be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:14.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.152+0000 7f7dda432640 1 --2- 172.21.15.105:0/3292437350 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7dc00668d0 0x7f7dc0068d90 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:14.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.152+0000 7f7dda432640 1 --2- 172.21.15.105:0/3292437350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7dd40cc760 0x7f7dd4153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:14.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.152+0000 7f7dda432640 1 --2- 172.21.15.105:0/3292437350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dd40ad680 0x7f7dd4156060 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:14.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.152+0000 7f7dda432640 1 -- 172.21.15.105:0/3292437350 >> 172.21.15.105:0/3292437350 conn(0x7f7dd40ceb90 msgr2=0x7f7dd4071aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:14.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.152+0000 7f7dda432640 1 -- 172.21.15.105:0/3292437350 shutdown_connections 2023-11-28T18:31:14.154 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.152+0000 7f7dda432640 1 -- 172.21.15.105:0/3292437350 wait complete. 2023-11-28T18:31:14.179 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:31:14.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.316+0000 7f40cedb8640 1 Processor -- start 2023-11-28T18:31:14.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.316+0000 7f40cedb8640 1 -- start start 2023-11-28T18:31:14.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.320+0000 7f40cedb8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f40c80b4290 0x7f40c80b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:14.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.320+0000 7f40cedb8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f40c815b030 0x7f40c80709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:14.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.320+0000 7f40cedb8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c80b4bb0 con 0x7f40c8071960 2023-11-28T18:31:14.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.320+0000 7f40cedb8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c80701d0 con 0x7f40c815b030 2023-11-28T18:31:14.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.320+0000 7f40cedb8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c806df80 con 0x7f40c80b4290 2023-11-28T18:31:14.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.320+0000 7f40cd355640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f40c815b030 0x7f40c80709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:14.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.320+0000 7f40bffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f40c8071960 0x7f40c815f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50374/0 (socket says 172.21.15.105:50374) 2023-11-28T18:31:14.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.320+0000 7f40bffff640 1 -- 172.21.15.105:0/2983916905 learned_addr learned my addr 172.21.15.105:0/2983916905 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:14.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.320+0000 7f40ccb54640 1 --2- 172.21.15.105:0/2983916905 >> v2:172.21.15.148:3300/0 conn(0x7f40c80b4290 0x7f40c80b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:14.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.320+0000 7f40bf7fe640 1 -- 172.21.15.105:0/2983916905 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1788432068 0 0) 0x7f40c80b4bb0 con 0x7f40c8071960 2023-11-28T18:31:14.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.320+0000 7f40bf7fe640 1 -- 172.21.15.105:0/2983916905 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40b0003530 con 0x7f40c8071960 2023-11-28T18:31:14.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.320+0000 7f40cd355640 1 -- 172.21.15.105:0/2983916905 >> v2:172.21.15.148:3300/0 conn(0x7f40c80b4290 msgr2=0x7f40c80b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:14.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.320+0000 7f40cd355640 1 --2- 172.21.15.105:0/2983916905 >> v2:172.21.15.148:3300/0 conn(0x7f40c80b4290 0x7f40c80b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:14.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.320+0000 7f40cd355640 1 -- 172.21.15.105:0/2983916905 >> v1:172.21.15.105:6789/0 conn(0x7f40c8071960 legacy=0x7f40c815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:14.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.320+0000 7f40cd355640 1 -- 172.21.15.105:0/2983916905 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c8071df0 con 0x7f40c815b030 2023-11-28T18:31:14.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.320+0000 7f40ccb54640 1 --2- 172.21.15.105:0/2983916905 >> v2:172.21.15.148:3300/0 conn(0x7f40c80b4290 0x7f40c80b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:14.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.320+0000 7f40cd355640 1 --2- 172.21.15.105:0/2983916905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40c815b030 0x7f40c80709b0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f40c8073a70 tx=0x7f40c001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b4748fffd32cb7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:14.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.324+0000 7f40bf7fe640 1 -- 172.21.15.105:0/2983916905 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c00ab600 con 0x7f40c815b030 2023-11-28T18:31:14.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.324+0000 7f40bf7fe640 1 -- 172.21.15.105:0/2983916905 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c00a7050 con 0x7f40c815b030 2023-11-28T18:31:14.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.324+0000 7f40bf7fe640 1 -- 172.21.15.105:0/2983916905 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c00ab980 con 0x7f40c815b030 2023-11-28T18:31:14.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.324+0000 7f40cedb8640 1 -- 172.21.15.105:0/2983916905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40c815b030 msgr2=0x7f40c80709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:14.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.324+0000 7f40cedb8640 1 --2- 172.21.15.105:0/2983916905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40c815b030 0x7f40c80709b0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f40c8073a70 tx=0x7f40c001c430 comp rx=0 tx=0).stop 2023-11-28T18:31:14.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.324+0000 7f40cedb8640 1 -- 172.21.15.105:0/2983916905 shutdown_connections 2023-11-28T18:31:14.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.324+0000 7f40cedb8640 1 --2- 172.21.15.105:0/2983916905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40c815b030 0x7f40c80709b0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:14.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.324+0000 7f40cedb8640 1 --2- 172.21.15.105:0/2983916905 >> v2:172.21.15.148:3300/0 conn(0x7f40c80b4290 0x7f40c80b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:14.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.324+0000 7f40cedb8640 1 -- 172.21.15.105:0/2983916905 >> 172.21.15.105:0/2983916905 conn(0x7f40c80ceb90 msgr2=0x7f40c80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:14.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.324+0000 7f40cedb8640 1 -- 172.21.15.105:0/2983916905 shutdown_connections 2023-11-28T18:31:14.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.324+0000 7f40cedb8640 1 -- 172.21.15.105:0/2983916905 wait complete. 2023-11-28T18:31:14.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.324+0000 7f40cedb8640 1 Processor -- start 2023-11-28T18:31:14.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.324+0000 7f40cedb8640 1 -- start start 2023-11-28T18:31:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.324+0000 7f40cedb8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40c80b4290 0x7f40c8157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.324+0000 7f40cedb8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f40c815b030 0x7f40c8157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.324+0000 7f40cedb8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c80b2a00 0x7f40c8143080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.324+0000 7f40cedb8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c809af90 con 0x7f40c80b4290 2023-11-28T18:31:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.324+0000 7f40cedb8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c80b3a80 con 0x7f40c80b2a00 2023-11-28T18:31:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.324+0000 7f40cedb8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c80b2f00 con 0x7f40c815b030 2023-11-28T18:31:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.324+0000 7f40ccb54640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40c80b4290 0x7f40c8157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.328+0000 7f40ccb54640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40c80b4290 0x7f40c8157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55106/0 (socket says 172.21.15.105:55106) 2023-11-28T18:31:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.328+0000 7f40ccb54640 1 -- 172.21.15.105:0/4068452246 learned_addr learned my addr 172.21.15.105:0/4068452246 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.328+0000 7f40bffff640 1 --2- 172.21.15.105:0/4068452246 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f40c815b030 0x7f40c8157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.328+0000 7f40cd355640 1 --2- 172.21.15.105:0/4068452246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c80b2a00 0x7f40c8143080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.328+0000 7f40ccb54640 1 -- 172.21.15.105:0/4068452246 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f40c815b030 msgr2=0x7f40c8157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.328+0000 7f40ccb54640 1 --2- 172.21.15.105:0/4068452246 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f40c815b030 0x7f40c8157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.328+0000 7f40ccb54640 1 -- 172.21.15.105:0/4068452246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c80b2a00 msgr2=0x7f40c8143080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.328+0000 7f40ccb54640 1 --2- 172.21.15.105:0/4068452246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c80b2a00 0x7f40c8143080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.328+0000 7f40ccb54640 1 -- 172.21.15.105:0/4068452246 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c0009420 con 0x7f40c80b4290 2023-11-28T18:31:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.328+0000 7f40bffff640 1 --2- 172.21.15.105:0/4068452246 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f40c815b030 0x7f40c8157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.328+0000 7f40ccb54640 1 --2- 172.21.15.105:0/4068452246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40c80b4290 0x7f40c8157940 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009fc0 tx=0x7f40c400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.328+0000 7f40bd7fa640 1 -- 172.21.15.105:0/4068452246 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c4003ba0 con 0x7f40c80b4290 2023-11-28T18:31:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.328+0000 7f40bd7fa640 1 -- 172.21.15.105:0/4068452246 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c400f040 con 0x7f40c80b4290 2023-11-28T18:31:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.328+0000 7f40cedb8640 1 -- 172.21.15.105:0/4068452246 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c80b31e0 con 0x7f40c80b4290 2023-11-28T18:31:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.328+0000 7f40bd7fa640 1 -- 172.21.15.105:0/4068452246 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c4004030 con 0x7f40c80b4290 2023-11-28T18:31:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.328+0000 7f40cedb8640 1 -- 172.21.15.105:0/4068452246 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c80b37a0 con 0x7f40c80b4290 2023-11-28T18:31:14.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.328+0000 7f40cedb8640 1 -- 172.21.15.105:0/4068452246 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c8067570 con 0x7f40c80b4290 2023-11-28T18:31:14.330 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.328+0000 7f40bd7fa640 1 -- 172.21.15.105:0/4068452246 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f40c4004190 con 0x7f40c80b4290 2023-11-28T18:31:14.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.332+0000 7f40bd7fa640 1 --2- 172.21.15.105:0/4068452246 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f40b4066900 0x7f40b4068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:14.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.332+0000 7f40bd7fa640 1 -- 172.21.15.105:0/4068452246 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f40c4084900 con 0x7f40c80b4290 2023-11-28T18:31:14.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.332+0000 7f40bffff640 1 --2- 172.21.15.105:0/4068452246 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f40b4066900 0x7f40b4068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:14.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.332+0000 7f40bffff640 1 --2- 172.21.15.105:0/4068452246 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f40b4066900 0x7f40b4068dc0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f40c815e440 tx=0x7f40b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:14.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.336+0000 7f40bd7fa640 1 -- 172.21.15.105:0/4068452246 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f40c4051980 con 0x7f40c80b4290 2023-11-28T18:31:14.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.520+0000 7f40cedb8640 1 -- 172.21.15.105:0/4068452246 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40c815b710 con 0x7f40c80b4290 2023-11-28T18:31:14.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.520+0000 7f40bd7fa640 1 -- 172.21.15.105:0/4068452246 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4943 (secure 0 0 0) 0x7f40c401a020 con 0x7f40c80b4290 2023-11-28T18:31:14.520 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:14.520 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":121,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/2073622668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2073622668},{"type":"v1","addr":"172.21.15.148:6834","nonce":2073622668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":7085,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/270170970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":270170970},{"type":"v1","addr":"172.21.15.148:6835","nonce":270170970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:31:03.120082+0000","modified":"2023-11-28T18:31:14.150787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1],"up":{"mds_0":8643,"mds_1":8640},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8640":{"gid":8640,"name":"c","rank":1,"incarnation":121,"state":"up:creating","state_seq":1,"addr":"172.21.15.105:6836/3373504883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3373504883},{"type":"v1","addr":"172.21.15.105:6836","nonce":3373504883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8643":{"gid":8643,"name":"a","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/764864050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":764864050},{"type":"v1","addr":"172.21.15.105:6837","nonce":764864050}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-28T18:31:14.521 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 121 2023-11-28T18:31:14.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.520+0000 7f40cedb8640 1 -- 172.21.15.105:0/4068452246 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f40b4066900 msgr2=0x7f40b4068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:14.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.520+0000 7f40cedb8640 1 --2- 172.21.15.105:0/4068452246 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f40b4066900 0x7f40b4068dc0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f40c815e440 tx=0x7f40b8009040 comp rx=0 tx=0).stop 2023-11-28T18:31:14.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.520+0000 7f40cedb8640 1 -- 172.21.15.105:0/4068452246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40c80b4290 msgr2=0x7f40c8157940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:14.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.520+0000 7f40cedb8640 1 --2- 172.21.15.105:0/4068452246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40c80b4290 0x7f40c8157940 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009fc0 tx=0x7f40c400d560 comp rx=0 tx=0).stop 2023-11-28T18:31:14.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.520+0000 7f40cedb8640 1 -- 172.21.15.105:0/4068452246 shutdown_connections 2023-11-28T18:31:14.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.520+0000 7f40cedb8640 1 --2- 172.21.15.105:0/4068452246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c80b2a00 0x7f40c8143080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:14.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.520+0000 7f40cedb8640 1 --2- 172.21.15.105:0/4068452246 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f40c815b030 0x7f40c8157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:14.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.520+0000 7f40cedb8640 1 --2- 172.21.15.105:0/4068452246 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f40b4066900 0x7f40b4068dc0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:14.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.520+0000 7f40cedb8640 1 --2- 172.21.15.105:0/4068452246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40c80b4290 0x7f40c8157940 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:14.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.520+0000 7f40cedb8640 1 -- 172.21.15.105:0/4068452246 >> 172.21.15.105:0/4068452246 conn(0x7f40c80ceb90 msgr2=0x7f40c806c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:14.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.520+0000 7f40cedb8640 1 -- 172.21.15.105:0/4068452246 shutdown_connections 2023-11-28T18:31:14.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:14.520+0000 7f40cedb8640 1 -- 172.21.15.105:0/4068452246 wait complete. 2023-11-28T18:31:14.943 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 121, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:31:03.120082+0000', 'modified': '2023-11-28T18:31:14.150787+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 8643, 'mds_1': 8640}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8640': {'gid': 8640, 'name': 'c', 'rank': 1, 'incarnation': 121, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.105:6836/3373504883', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 3373504883}, {'type': 'v1', 'addr': '172.21.15.105:6836', 'nonce': 3373504883}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_8643': {'gid': 8643, 'name': 'a', 'rank': 0, 'incarnation': 115, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6837/764864050', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6835', 'nonce': 764864050}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 764864050}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:31:14.943 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8640:up:creating 2023-11-28T18:31:15.945 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:31:16.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.088+0000 7f4329701640 1 Processor -- start 2023-11-28T18:31:16.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.088+0000 7f4329701640 1 -- start start 2023-11-28T18:31:16.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.088+0000 7f4329701640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f43240d00b0 0x7f43240cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.088+0000 7f4329701640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f43240ceb40 0x7f43240b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.088+0000 7f4329701640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43240d3e60 con 0x7f43240b8a20 2023-11-28T18:31:16.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.088+0000 7f4329701640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43240b5fe0 con 0x7f43240ceb40 2023-11-28T18:31:16.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.088+0000 7f4329701640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43240b62c0 con 0x7f43240d00b0 2023-11-28T18:31:16.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.088+0000 7f43237fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f43240ceb40 0x7f43240b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.088+0000 7f43237fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f43240ceb40 0x7f43240b6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55110/0 (socket says 172.21.15.105:55110) 2023-11-28T18:31:16.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.088+0000 7f43237fe640 1 -- 172.21.15.105:0/1034990126 learned_addr learned my addr 172.21.15.105:0/1034990126 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:16.090 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.088+0000 7f43227fc640 1 --2- 172.21.15.105:0/1034990126 >> v2:172.21.15.148:3300/0 conn(0x7f43240d00b0 0x7f43240cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.088+0000 7f43237fe640 1 -- 172.21.15.105:0/1034990126 >> v2:172.21.15.148:3300/0 conn(0x7f43240d00b0 msgr2=0x7f43240cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:16.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.088+0000 7f43237fe640 1 --2- 172.21.15.105:0/1034990126 >> v2:172.21.15.148:3300/0 conn(0x7f43240d00b0 0x7f43240cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.088+0000 7f43237fe640 1 -- 172.21.15.105:0/1034990126 >> v1:172.21.15.148:6789/0 conn(0x7f43240b8a20 legacy=0x7f43240cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:16.091 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.088+0000 7f43237fe640 1 -- 172.21.15.105:0/1034990126 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43240b65a0 con 0x7f43240ceb40 2023-11-28T18:31:16.092 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.092+0000 7f43237fe640 1 --2- 172.21.15.105:0/1034990126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43240ceb40 0x7f43240b6da0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f432408d350 tx=0x7f431c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e696c41dcf4064 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:16.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.092+0000 7f4321ffb640 1 -- 172.21.15.105:0/1034990126 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c0a5800 con 0x7f43240ceb40 2023-11-28T18:31:16.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.092+0000 7f4321ffb640 1 -- 172.21.15.105:0/1034990126 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c0a5960 con 0x7f43240ceb40 2023-11-28T18:31:16.093 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.092+0000 7f4321ffb640 1 -- 172.21.15.105:0/1034990126 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c0a5cf0 con 0x7f43240ceb40 2023-11-28T18:31:16.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.092+0000 7f4329701640 1 -- 172.21.15.105:0/1034990126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43240ceb40 msgr2=0x7f43240b6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:16.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.092+0000 7f4329701640 1 --2- 172.21.15.105:0/1034990126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43240ceb40 0x7f43240b6da0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f432408d350 tx=0x7f431c01c9a0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.092+0000 7f4329701640 1 -- 172.21.15.105:0/1034990126 shutdown_connections 2023-11-28T18:31:16.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.092+0000 7f4329701640 1 --2- 172.21.15.105:0/1034990126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43240ceb40 0x7f43240b6da0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.092+0000 7f4329701640 1 --2- 172.21.15.105:0/1034990126 >> v2:172.21.15.148:3300/0 conn(0x7f43240d00b0 0x7f43240cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.094 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.092+0000 7f4329701640 1 -- 172.21.15.105:0/1034990126 >> 172.21.15.105:0/1034990126 conn(0x7f432409c340 msgr2=0x7f43240d2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:16.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.092+0000 7f4329701640 1 -- 172.21.15.105:0/1034990126 shutdown_connections 2023-11-28T18:31:16.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.092+0000 7f4329701640 1 -- 172.21.15.105:0/1034990126 wait complete. 2023-11-28T18:31:16.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f4329701640 1 Processor -- start 2023-11-28T18:31:16.095 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f4329701640 1 -- start start 2023-11-28T18:31:16.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f4329701640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43240b8a20 0x7f4324157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f4329701640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43240ceb40 0x7f43240caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f4329701640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43240d00b0 0x7f432406b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f4329701640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43240cb550 con 0x7f43240ceb40 2023-11-28T18:31:16.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f4329701640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43240cb830 con 0x7f43240d00b0 2023-11-28T18:31:16.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f4329701640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43240cbb10 con 0x7f43240b8a20 2023-11-28T18:31:16.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f43227fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43240ceb40 0x7f43240caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f43227fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43240ceb40 0x7f43240caad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55114/0 (socket says 172.21.15.105:55114) 2023-11-28T18:31:16.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f43227fc640 1 -- 172.21.15.105:0/4254786828 learned_addr learned my addr 172.21.15.105:0/4254786828 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:16.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f43237fe640 1 --2- 172.21.15.105:0/4254786828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43240d00b0 0x7f432406b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f4322ffd640 1 --2- 172.21.15.105:0/4254786828 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43240b8a20 0x7f4324157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f43227fc640 1 -- 172.21.15.105:0/4254786828 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43240b8a20 msgr2=0x7f4324157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:16.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f43227fc640 1 --2- 172.21.15.105:0/4254786828 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43240b8a20 0x7f4324157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f43227fc640 1 -- 172.21.15.105:0/4254786828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43240d00b0 msgr2=0x7f432406b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:16.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f43227fc640 1 --2- 172.21.15.105:0/4254786828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43240d00b0 0x7f432406b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f43227fc640 1 -- 172.21.15.105:0/4254786828 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c009b30 con 0x7f43240ceb40 2023-11-28T18:31:16.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f43227fc640 1 --2- 172.21.15.105:0/4254786828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43240ceb40 0x7f43240caad0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f431800d5f0 tx=0x7f431800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:16.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f430b7fe640 1 -- 172.21.15.105:0/4254786828 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43180125f0 con 0x7f43240ceb40 2023-11-28T18:31:16.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f430b7fe640 1 -- 172.21.15.105:0/4254786828 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4318012750 con 0x7f43240ceb40 2023-11-28T18:31:16.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f4329701640 1 -- 172.21.15.105:0/4254786828 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43240cbdf0 con 0x7f43240ceb40 2023-11-28T18:31:16.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f430b7fe640 1 -- 172.21.15.105:0/4254786828 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318012ae0 con 0x7f43240ceb40 2023-11-28T18:31:16.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.096+0000 7f4329701640 1 -- 172.21.15.105:0/4254786828 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432415cb40 con 0x7f43240ceb40 2023-11-28T18:31:16.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.100+0000 7f4329701640 1 -- 172.21.15.105:0/4254786828 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43240c70c0 con 0x7f43240ceb40 2023-11-28T18:31:16.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.100+0000 7f430b7fe640 1 -- 172.21.15.105:0/4254786828 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f4318012c40 con 0x7f43240ceb40 2023-11-28T18:31:16.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.100+0000 7f430b7fe640 1 --2- 172.21.15.105:0/4254786828 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f43100668d0 0x7f4310068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.100+0000 7f430b7fe640 1 -- 172.21.15.105:0/4254786828 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4318084a60 con 0x7f43240ceb40 2023-11-28T18:31:16.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.100+0000 7f4322ffd640 1 --2- 172.21.15.105:0/4254786828 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f43100668d0 0x7f4310068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.100+0000 7f4322ffd640 1 --2- 172.21.15.105:0/4254786828 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f43100668d0 0x7f4310068d90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f43240c6eb0 tx=0x7f4314009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:16.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.104+0000 7f430b7fe640 1 -- 172.21.15.105:0/4254786828 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4318051ac0 con 0x7f43240ceb40 2023-11-28T18:31:16.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.288+0000 7f4329701640 1 -- 172.21.15.105:0/4254786828 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4324067570 con 0x7f43240ceb40 2023-11-28T18:31:16.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.288+0000 7f430b7fe640 1 -- 172.21.15.105:0/4254786828 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+4955 (secure 0 0 0) 0x7f4318007380 con 0x7f43240ceb40 2023-11-28T18:31:16.288 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:16.288 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":124,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/2073622668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2073622668},{"type":"v1","addr":"172.21.15.148:6834","nonce":2073622668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:31:03.120082+0000","modified":"2023-11-28T18:31:15.191869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8643,"mds_1":8640,"mds_2":7085},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7085":{"gid":7085,"name":"d","rank":2,"incarnation":123,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6835/270170970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":270170970},{"type":"v1","addr":"172.21.15.148:6835","nonce":270170970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8640":{"gid":8640,"name":"c","rank":1,"incarnation":121,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6836/3373504883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3373504883},{"type":"v1","addr":"172.21.15.105:6836","nonce":3373504883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8643":{"gid":8643,"name":"a","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/764864050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":764864050},{"type":"v1","addr":"172.21.15.105:6837","nonce":764864050}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-28T18:31:16.289 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 124 2023-11-28T18:31:16.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.288+0000 7f4329701640 1 -- 172.21.15.105:0/4254786828 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f43100668d0 msgr2=0x7f4310068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:16.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.288+0000 7f4329701640 1 --2- 172.21.15.105:0/4254786828 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f43100668d0 0x7f4310068d90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f43240c6eb0 tx=0x7f4314009040 comp rx=0 tx=0).stop 2023-11-28T18:31:16.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.288+0000 7f4329701640 1 -- 172.21.15.105:0/4254786828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43240ceb40 msgr2=0x7f43240caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:16.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.288+0000 7f4329701640 1 --2- 172.21.15.105:0/4254786828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43240ceb40 0x7f43240caad0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f431800d5f0 tx=0x7f431800dab0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.288+0000 7f4329701640 1 -- 172.21.15.105:0/4254786828 shutdown_connections 2023-11-28T18:31:16.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.288+0000 7f4329701640 1 --2- 172.21.15.105:0/4254786828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43240d00b0 0x7f432406b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.288+0000 7f4329701640 1 --2- 172.21.15.105:0/4254786828 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f43100668d0 0x7f4310068d90 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.288+0000 7f4329701640 1 --2- 172.21.15.105:0/4254786828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43240ceb40 0x7f43240caad0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.288+0000 7f4329701640 1 --2- 172.21.15.105:0/4254786828 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43240b8a20 0x7f4324157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.288+0000 7f4329701640 1 -- 172.21.15.105:0/4254786828 >> 172.21.15.105:0/4254786828 conn(0x7f432409c340 msgr2=0x7f4324160530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:16.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.288+0000 7f4329701640 1 -- 172.21.15.105:0/4254786828 shutdown_connections 2023-11-28T18:31:16.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.288+0000 7f4329701640 1 -- 172.21.15.105:0/4254786828 wait complete. 2023-11-28T18:31:16.712 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:31:03.120082+0000', 'modified': '2023-11-28T18:31:15.191869+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 8643, 'mds_1': 8640, 'mds_2': 7085}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7085': {'gid': 7085, 'name': 'd', 'rank': 2, 'incarnation': 123, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6835/270170970', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6833', 'nonce': 270170970}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 270170970}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_8640': {'gid': 8640, 'name': 'c', 'rank': 1, 'incarnation': 121, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.105:6836/3373504883', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 3373504883}, {'type': 'v1', 'addr': '172.21.15.105:6836', 'nonce': 3373504883}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_8643': {'gid': 8643, 'name': 'a', 'rank': 0, 'incarnation': 115, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6837/764864050', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6835', 'nonce': 764864050}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 764864050}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:31:16.712 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-11-28T18:31:16.713 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-28T18:31:16.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f15480fe640 1 Processor -- start 2023-11-28T18:31:16.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f15480fe640 1 -- start start 2023-11-28T18:31:16.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f15480fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f154006d610 0x7f15400cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f15480fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1540162820 con 0x7f15400ce530 2023-11-28T18:31:16.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f15480fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1540161c10 con 0x7f15400ae100 2023-11-28T18:31:16.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f15480fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1540161ef0 con 0x7f154006d610 2023-11-28T18:31:16.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1545699640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f15400ce530 0x7f15400ad9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50390/0 (socket says 172.21.15.105:50390) 2023-11-28T18:31:16.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1545699640 1 -- 172.21.15.105:0/1416790100 learned_addr learned my addr 172.21.15.105:0/1416790100 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:16.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1545e9a640 1 --2- 172.21.15.105:0/1416790100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f154006d610 0x7f15400cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1544e98640 1 -- 172.21.15.105:0/1416790100 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 612762962 0 0) 0x7f1540162820 con 0x7f15400ce530 2023-11-28T18:31:16.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1544e98640 1 -- 172.21.15.105:0/1416790100 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f152c003530 con 0x7f15400ce530 2023-11-28T18:31:16.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1544e98640 1 -- 172.21.15.105:0/1416790100 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1047809525 0 0) 0x7f152c003530 con 0x7f15400ce530 2023-11-28T18:31:16.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1544e98640 1 -- 172.21.15.105:0/1416790100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f154006d610 msgr2=0x7f15400cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:16.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1544e98640 1 --2- 172.21.15.105:0/1416790100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f154006d610 0x7f15400cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1544e98640 1 -- 172.21.15.105:0/1416790100 >> v1:172.21.15.148:6789/0 conn(0x7f15400ae100 legacy=0x7f15400cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:16.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1544e98640 1 -- 172.21.15.105:0/1416790100 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15401621d0 con 0x7f15400ce530 2023-11-28T18:31:16.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1544e98640 1 -- 172.21.15.105:0/1416790100 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f1530001680 con 0x7f15400ce530 2023-11-28T18:31:16.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1544e98640 1 -- 172.21.15.105:0/1416790100 >> v1:172.21.15.105:6789/0 conn(0x7f15400ce530 legacy=0x7f15400ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:16.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1544e98640 1 --2- 172.21.15.105:0/1416790100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f152c0039b0 0x7f152c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.860 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1544e98640 1 --2- 172.21.15.105:0/1416790100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f154006d610 0x7f152c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1544e98640 1 --2- 172.21.15.105:0/1416790100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f152c004d10 0x7f152c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1544e98640 1 -- 172.21.15.105:0/1416790100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1540161ef0 con 0x7f152c0039b0 2023-11-28T18:31:16.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1544e98640 1 -- 172.21.15.105:0/1416790100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c00b990 con 0x7f152c004d10 2023-11-28T18:31:16.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1544e98640 1 -- 172.21.15.105:0/1416790100 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152c00bb50 con 0x7f154006d610 2023-11-28T18:31:16.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1545e9a640 1 --2- 172.21.15.105:0/1416790100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f152c0039b0 0x7f152c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1545699640 1 --2- 172.21.15.105:0/1416790100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f154006d610 0x7f152c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f154669b640 1 --2- 172.21.15.105:0/1416790100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f152c004d10 0x7f152c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1545e9a640 1 -- 172.21.15.105:0/1416790100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f154006d610 msgr2=0x7f152c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:16.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1545e9a640 1 --2- 172.21.15.105:0/1416790100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f154006d610 0x7f152c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1545e9a640 1 -- 172.21.15.105:0/1416790100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f152c004d10 msgr2=0x7f152c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:16.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1545e9a640 1 --2- 172.21.15.105:0/1416790100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f152c004d10 0x7f152c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.856+0000 7f1545e9a640 1 -- 172.21.15.105:0/1416790100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152c003530 con 0x7f152c0039b0 2023-11-28T18:31:16.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.860+0000 7f1545e9a640 1 --2- 172.21.15.105:0/1416790100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f152c0039b0 0x7f152c003d90 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f153c009ae0 tx=0x7f153c01c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47272292180203c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:16.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.860+0000 7f1544e98640 1 -- 172.21.15.105:0/1416790100 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c0a5580 con 0x7f152c0039b0 2023-11-28T18:31:16.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.860+0000 7f1544e98640 1 -- 172.21.15.105:0/1416790100 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153c0a56e0 con 0x7f152c0039b0 2023-11-28T18:31:16.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.860+0000 7f1544e98640 1 -- 172.21.15.105:0/1416790100 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c0a5a90 con 0x7f152c0039b0 2023-11-28T18:31:16.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.860+0000 7f15480fe640 1 -- 172.21.15.105:0/1416790100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f152c0039b0 msgr2=0x7f152c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:16.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.860+0000 7f15480fe640 1 --2- 172.21.15.105:0/1416790100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f152c0039b0 0x7f152c003d90 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f153c009ae0 tx=0x7f153c01c990 comp rx=0 tx=0).stop 2023-11-28T18:31:16.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.860+0000 7f154669b640 1 -- 172.21.15.105:0/1416790100 reap_dead start 2023-11-28T18:31:16.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.860+0000 7f15480fe640 1 -- 172.21.15.105:0/1416790100 shutdown_connections 2023-11-28T18:31:16.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.860+0000 7f15480fe640 1 -- 172.21.15.105:0/1416790100 >> 172.21.15.105:0/1416790100 conn(0x7f15400ceb90 msgr2=0x7f15400cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:16.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.860+0000 7f15480fe640 1 -- 172.21.15.105:0/1416790100 shutdown_connections 2023-11-28T18:31:16.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.860+0000 7f15480fe640 1 -- 172.21.15.105:0/1416790100 wait complete. 2023-11-28T18:31:16.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f15480fe640 1 Processor -- start 2023-11-28T18:31:16.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f15480fe640 1 -- start start 2023-11-28T18:31:16.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f15480fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f152c0039b0 0x7f154015bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f15480fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f154009c440 0x7f154009aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f15480fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15400b3c60 0x7f15400ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f15480fe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154009bfb0 con 0x7f15400b3c60 2023-11-28T18:31:16.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f15480fe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154009b3a0 con 0x7f152c0039b0 2023-11-28T18:31:16.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f15480fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154009b680 con 0x7f154009c440 2023-11-28T18:31:16.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f154669b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15400b3c60 0x7f15400ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f154669b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15400b3c60 0x7f15400ad2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55130/0 (socket says 172.21.15.105:55130) 2023-11-28T18:31:16.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f154669b640 1 -- 172.21.15.105:0/3086320817 learned_addr learned my addr 172.21.15.105:0/3086320817 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:16.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f1545e9a640 1 --2- 172.21.15.105:0/3086320817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f152c0039b0 0x7f154015bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f1545699640 1 --2- 172.21.15.105:0/3086320817 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f154009c440 0x7f154009aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f154669b640 1 -- 172.21.15.105:0/3086320817 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f154009c440 msgr2=0x7f154009aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:16.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f154669b640 1 --2- 172.21.15.105:0/3086320817 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f154009c440 0x7f154009aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f154669b640 1 -- 172.21.15.105:0/3086320817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f152c0039b0 msgr2=0x7f154015bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:16.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f154669b640 1 --2- 172.21.15.105:0/3086320817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f152c0039b0 0x7f154015bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f154669b640 1 -- 172.21.15.105:0/3086320817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154009b960 con 0x7f15400b3c60 2023-11-28T18:31:16.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f154669b640 1 --2- 172.21.15.105:0/3086320817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15400b3c60 0x7f15400ad2b0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f154006ba30 tx=0x7f1538010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:16.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f15367fc640 1 -- 172.21.15.105:0/3086320817 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1538003d90 con 0x7f15400b3c60 2023-11-28T18:31:16.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f15367fc640 1 -- 172.21.15.105:0/3086320817 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153800d070 con 0x7f15400b3c60 2023-11-28T18:31:16.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f15480fe640 1 -- 172.21.15.105:0/3086320817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154009bc40 con 0x7f15400b3c60 2023-11-28T18:31:16.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f15367fc640 1 -- 172.21.15.105:0/3086320817 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153800ead0 con 0x7f15400b3c60 2023-11-28T18:31:16.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.864+0000 7f15480fe640 1 -- 172.21.15.105:0/3086320817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15400ad7f0 con 0x7f15400b3c60 2023-11-28T18:31:16.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.868+0000 7f15367fc640 1 -- 172.21.15.105:0/3086320817 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f153800ec30 con 0x7f15400b3c60 2023-11-28T18:31:16.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.868+0000 7f15367fc640 1 --2- 172.21.15.105:0/3086320817 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15240668d0 0x7f1524068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.869 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.868+0000 7f15367fc640 1 -- 172.21.15.105:0/3086320817 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1538087730 con 0x7f15400b3c60 2023-11-28T18:31:16.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.868+0000 7f1545e9a640 1 --2- 172.21.15.105:0/3086320817 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15240668d0 0x7f1524068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.868+0000 7f154b46c000 1 Processor -- start 2023-11-28T18:31:16.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.868+0000 7f154b46c000 1 -- start start 2023-11-28T18:31:16.870 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.868+0000 7f1545e9a640 1 --2- 172.21.15.105:0/3086320817 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15240668d0 0x7f1524068d90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f153c0af000 tx=0x7f153c0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:16.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f154b46c000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e93d30 0x561525e94110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f154b46c000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561525e94650 0x561525e46fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f154b46c000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561525e47500 0x561525e478e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f154b46c000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x561525d10110 con 0x561525e93d30 2023-11-28T18:31:16.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f154b46c000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x561525ca09a0 con 0x561525e47500 2023-11-28T18:31:16.871 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f154b46c000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x561525c9ded0 con 0x561525e94650 2023-11-28T18:31:16.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f1545e9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e93d30 0x561525e94110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.873 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f1545e9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e93d30 0x561525e94110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55144/0 (socket says 172.21.15.105:55144) 2023-11-28T18:31:16.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f1545e9a640 1 -- 172.21.15.105:0/3169589631 learned_addr learned my addr 172.21.15.105:0/3169589631 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:16.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f154669b640 1 --2- 172.21.15.105:0/3169589631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561525e47500 0x561525e478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f1545699640 1 --2- 172.21.15.105:0/3169589631 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561525e94650 0x561525e46fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f1545e9a640 1 -- 172.21.15.105:0/3169589631 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561525e94650 msgr2=0x561525e46fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:16.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f1545e9a640 1 --2- 172.21.15.105:0/3169589631 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561525e94650 0x561525e46fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f1545e9a640 1 -- 172.21.15.105:0/3169589631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561525e47500 msgr2=0x561525e478e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:16.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f1545e9a640 1 --2- 172.21.15.105:0/3169589631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561525e47500 0x561525e478e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f1545e9a640 1 -- 172.21.15.105:0/3169589631 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561525991890 con 0x561525e93d30 2023-11-28T18:31:16.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f154669b640 1 --2- 172.21.15.105:0/3169589631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561525e47500 0x561525e478e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:31:16.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f1545e9a640 1 --2- 172.21.15.105:0/3169589631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e93d30 0x561525e94110 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f153c01c9d0 tx=0x7f153c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4968b286cb9fc046 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:16.874 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f15237fe640 1 -- 172.21.15.105:0/3169589631 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c0ad440 con 0x561525e93d30 2023-11-28T18:31:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f15237fe640 1 -- 172.21.15.105:0/3169589631 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153c0ad5a0 con 0x561525e93d30 2023-11-28T18:31:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f15237fe640 1 -- 172.21.15.105:0/3169589631 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c0ad910 con 0x561525e93d30 2023-11-28T18:31:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f154b46c000 1 -- 172.21.15.105:0/3169589631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e93d30 msgr2=0x561525e94110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f154b46c000 1 --2- 172.21.15.105:0/3169589631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e93d30 0x561525e94110 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f153c01c9d0 tx=0x7f153c004740 comp rx=0 tx=0).stop 2023-11-28T18:31:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f154b46c000 1 -- 172.21.15.105:0/3169589631 shutdown_connections 2023-11-28T18:31:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f154b46c000 1 --2- 172.21.15.105:0/3169589631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561525e47500 0x561525e478e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f154b46c000 1 --2- 172.21.15.105:0/3169589631 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561525e94650 0x561525e46fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f154b46c000 1 --2- 172.21.15.105:0/3169589631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e93d30 0x561525e94110 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f154b46c000 1 -- 172.21.15.105:0/3169589631 >> 172.21.15.105:0/3169589631 conn(0x561525e748c0 msgr2=0x561525e76cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f154b46c000 1 -- 172.21.15.105:0/3169589631 shutdown_connections 2023-11-28T18:31:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f154b46c000 1 -- 172.21.15.105:0/3169589631 wait complete. 2023-11-28T18:31:16.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f154b46c000 1 -- start start 2023-11-28T18:31:16.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.872+0000 7f154b46c000 1 Processor -- start 2023-11-28T18:31:16.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f154b46c000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e47500 0x561525e4d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f154b46c000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561525e93d30 0x561525e4da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f154b46c000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561525e94650 0x561525e4dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f154b46c000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x561525c997c0 con 0x561525e47500 2023-11-28T18:31:16.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f154b46c000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x561525c51bf0 con 0x561525e94650 2023-11-28T18:31:16.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f154b46c000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x561525c57a90 con 0x561525e93d30 2023-11-28T18:31:16.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f154b46c000 10 client.0 _ll_register_callbacks cb 0x561525e34e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:31:16.879 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f1545e9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e47500 0x561525e4d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f1545e9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e47500 0x561525e4d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55160/0 (socket says 172.21.15.105:55160) 2023-11-28T18:31:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f1545e9a640 1 -- 172.21.15.105:0/4250698028 learned_addr learned my addr 172.21.15.105:0/4250698028 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f1545699640 1 --2- 172.21.15.105:0/4250698028 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561525e93d30 0x561525e4da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f154669b640 1 --2- 172.21.15.105:0/4250698028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561525e94650 0x561525e4dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f1545e9a640 1 -- 172.21.15.105:0/4250698028 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561525e93d30 msgr2=0x561525e4da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f1545e9a640 1 --2- 172.21.15.105:0/4250698028 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561525e93d30 0x561525e4da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f1545e9a640 1 -- 172.21.15.105:0/4250698028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561525e94650 msgr2=0x561525e4dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f1545e9a640 1 --2- 172.21.15.105:0/4250698028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561525e94650 0x561525e4dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f1545e9a640 1 -- 172.21.15.105:0/4250698028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15400ad7f0 con 0x561525e47500 2023-11-28T18:31:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f1545e9a640 1 --2- 172.21.15.105:0/4250698028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e47500 0x561525e4d520 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f153c00f210 tx=0x7f153c019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ada816453619ad0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f1520ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:31:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f1520ff9640 1 -- 172.21.15.105:0/4250698028 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c0a5ea0 con 0x561525e47500 2023-11-28T18:31:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f1520ff9640 1 -- 172.21.15.105:0/4250698028 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f153c0add60 con 0x561525e47500 2023-11-28T18:31:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f154b46c000 1 -- 172.21.15.105:0/4250698028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561525c5b470 con 0x561525e47500 2023-11-28T18:31:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f1520ff9640 1 -- 172.21.15.105:0/4250698028 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7f153c0c2440 con 0x561525e47500 2023-11-28T18:31:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f154b46c000 10 client.8847 fetch_fsmap learned FSMap version 124 2023-11-28T18:31:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f154b46c000 1 -- 172.21.15.105:0/4250698028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x7f152c003530 con 0x561525e47500 2023-11-28T18:31:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f1520ff9640 1 -- 172.21.15.105:0/4250698028 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561525c5b470 con 0x561525e47500 2023-11-28T18:31:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f1520ff9640 1 -- 172.21.15.105:0/4250698028 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 124) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f153c0d6050 con 0x561525e47500 2023-11-28T18:31:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f154b46c000 10 client.8847 fetch_fsmap finished waiting for FSMap version 124 2023-11-28T18:31:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f154b46c000 10 client.8847 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T18:31:16.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f154b46c000 20 client.8847 populate_metadata read hostname 'smithi105' 2023-11-28T18:31:16.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f154b46c000 1 --2- 172.21.15.105:0/4250698028 >> [v2:172.21.15.148:6833/270170970,v1:172.21.15.148:6835/270170970] conn(0x561525e4e9b0 0x561525ea5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f154b46c000 4 client.8847 mds_command: new command op to 7085 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:31:16.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f154b46c000 1 -- 172.21.15.105:0/4250698028 --> [v2:172.21.15.148:6833/270170970,v1:172.21.15.148:6835/270170970] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561525c5eb50 con 0x561525e4e9b0 2023-11-28T18:31:16.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f1545699640 1 --2- 172.21.15.105:0/4250698028 >> [v2:172.21.15.148:6833/270170970,v1:172.21.15.148:6835/270170970] conn(0x561525e4e9b0 0x561525ea5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f1545699640 1 --2- 172.21.15.105:0/4250698028 >> [v2:172.21.15.148:6833/270170970,v1:172.21.15.148:6835/270170970] conn(0x561525e4e9b0 0x561525ea5ff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=403a06b34234d138 server_cookie=969fa03b392529bf in_seq=0 out_seq=0 2023-11-28T18:31:16.882 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.876+0000 7f1520ff9640 10 client.8847 ms_handle_connect on v2:172.21.15.148:6833/270170970 2023-11-28T18:31:16.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.880+0000 7f1520ff9640 1 -- 172.21.15.105:0/4250698028 <== mds.2 v2:172.21.15.148:6833/270170970 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561525c5eb50 con 0x561525e4e9b0 2023-11-28T18:31:16.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.880+0000 7f1520ff9640 10 client.8847 handle_command_reply: tid=0 2023-11-28T18:31:16.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.880+0000 7f154b46c000 1 client.8847 shutdown 2023-11-28T18:31:16.883 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.880+0000 7f154b46c000 1 -- 172.21.15.105:0/4250698028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e47500 msgr2=0x561525e4d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:16.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.880+0000 7f154b46c000 1 --2- 172.21.15.105:0/4250698028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e47500 0x561525e4d520 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f153c00f210 tx=0x7f153c019730 comp rx=0 tx=0).stop 2023-11-28T18:31:16.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.880+0000 7f154b46c000 1 -- 172.21.15.105:0/4250698028 shutdown_connections 2023-11-28T18:31:16.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.880+0000 7f154b46c000 1 --2- 172.21.15.105:0/4250698028 >> [v2:172.21.15.148:6833/270170970,v1:172.21.15.148:6835/270170970] conn(0x561525e4e9b0 0x561525ea5ff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.880+0000 7f154b46c000 1 --2- 172.21.15.105:0/4250698028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561525e94650 0x561525e4dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.880+0000 7f154b46c000 1 --2- 172.21.15.105:0/4250698028 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561525e93d30 0x561525e4da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.880+0000 7f154b46c000 1 --2- 172.21.15.105:0/4250698028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e47500 0x561525e4d520 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.880+0000 7f1520ff9640 0 client.8847 ms_handle_reset on v2:172.21.15.148:6833/270170970 2023-11-28T18:31:16.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.880+0000 7f154b46c000 1 -- 172.21.15.105:0/4250698028 >> 172.21.15.105:0/4250698028 conn(0x561525e748c0 msgr2=0x561525e466b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:16.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.880+0000 7f154b46c000 1 -- 172.21.15.105:0/4250698028 shutdown_connections 2023-11-28T18:31:16.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.880+0000 7f154b46c000 1 -- 172.21.15.105:0/4250698028 wait complete. 2023-11-28T18:31:16.884 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.880+0000 7f154b46c000 20 client.8847 trim_cache size 0 max 16384 2023-11-28T18:31:16.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f154b46c000 1 Processor -- start 2023-11-28T18:31:16.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f154b46c000 1 -- start start 2023-11-28T18:31:16.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f154b46c000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561525e47500 0x561525ea8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f154b46c000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e93d30 0x561525e733a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f154b46c000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561525e94650 0x561525e738e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f154b46c000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x561525952940 con 0x561525e93d30 2023-11-28T18:31:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f154b46c000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x561525e74780 con 0x561525e94650 2023-11-28T18:31:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f154b46c000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x561525c6eb70 con 0x561525e47500 2023-11-28T18:31:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f1545699640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e93d30 0x561525e733a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f1545699640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e93d30 0x561525e733a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55164/0 (socket says 172.21.15.105:55164) 2023-11-28T18:31:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f1545699640 1 -- 172.21.15.105:0/2004608361 learned_addr learned my addr 172.21.15.105:0/2004608361 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f154669b640 1 --2- 172.21.15.105:0/2004608361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561525e94650 0x561525e738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f1545699640 1 -- 172.21.15.105:0/2004608361 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561525e47500 msgr2=0x561525ea8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:16.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f1545e9a640 1 --2- 172.21.15.105:0/2004608361 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561525e47500 0x561525ea8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f1545699640 1 --2- 172.21.15.105:0/2004608361 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561525e47500 0x561525ea8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f1545699640 1 -- 172.21.15.105:0/2004608361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561525e94650 msgr2=0x561525e738e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:16.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f1545699640 1 --2- 172.21.15.105:0/2004608361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561525e94650 0x561525e738e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f1545699640 1 -- 172.21.15.105:0/2004608361 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561525c5a190 con 0x561525e93d30 2023-11-28T18:31:16.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f154669b640 1 --2- 172.21.15.105:0/2004608361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561525e94650 0x561525e738e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:31:16.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f1545699640 1 --2- 172.21.15.105:0/2004608361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e93d30 0x561525e733a0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f15400cc130 tx=0x7f154006e500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5959815c21131c7d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:16.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f1523fff640 1 -- 172.21.15.105:0/2004608361 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15300190d0 con 0x561525e93d30 2023-11-28T18:31:16.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f1523fff640 1 -- 172.21.15.105:0/2004608361 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1530021070 con 0x561525e93d30 2023-11-28T18:31:16.901 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f1523fff640 1 -- 172.21.15.105:0/2004608361 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1530019900 con 0x561525e93d30 2023-11-28T18:31:16.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f154b46c000 1 -- 172.21.15.105:0/2004608361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e93d30 msgr2=0x561525e733a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:16.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f154b46c000 1 --2- 172.21.15.105:0/2004608361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e93d30 0x561525e733a0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f15400cc130 tx=0x7f154006e500 comp rx=0 tx=0).stop 2023-11-28T18:31:16.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f154b46c000 1 -- 172.21.15.105:0/2004608361 shutdown_connections 2023-11-28T18:31:16.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f154b46c000 1 --2- 172.21.15.105:0/2004608361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561525e94650 0x561525e738e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f154b46c000 1 --2- 172.21.15.105:0/2004608361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e93d30 0x561525e733a0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f154b46c000 1 --2- 172.21.15.105:0/2004608361 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561525e47500 0x561525ea8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f154b46c000 1 -- 172.21.15.105:0/2004608361 >> 172.21.15.105:0/2004608361 conn(0x561525e748c0 msgr2=0x561525e45720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:16.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f154b46c000 1 -- 172.21.15.105:0/2004608361 shutdown_connections 2023-11-28T18:31:16.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.896+0000 7f154b46c000 1 -- 172.21.15.105:0/2004608361 wait complete. 2023-11-28T18:31:16.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154b46c000 1 -- start start 2023-11-28T18:31:16.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154b46c000 1 Processor -- start 2023-11-28T18:31:16.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154b46c000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561525e47500 0x561525ef4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154b46c000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561525e93d30 0x561525ef4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154b46c000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e94650 0x561525eafa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154b46c000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x561525c443b0 con 0x561525e94650 2023-11-28T18:31:16.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154b46c000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x561525c98fd0 con 0x561525e47500 2023-11-28T18:31:16.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154b46c000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x561525c636c0 con 0x561525e93d30 2023-11-28T18:31:16.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154b46c000 10 client.0 _ll_register_callbacks cb 0x561525d10060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:31:16.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154669b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e94650 0x561525eafa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154669b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e94650 0x561525eafa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55168/0 (socket says 172.21.15.105:55168) 2023-11-28T18:31:16.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154669b640 1 -- 172.21.15.105:0/4101238436 learned_addr learned my addr 172.21.15.105:0/4101238436 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:16.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f1545699640 1 --2- 172.21.15.105:0/4101238436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561525e93d30 0x561525ef4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154669b640 1 -- 172.21.15.105:0/4101238436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561525e93d30 msgr2=0x561525ef4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:16.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154669b640 1 --2- 172.21.15.105:0/4101238436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561525e93d30 0x561525ef4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154669b640 1 -- 172.21.15.105:0/4101238436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561525e47500 msgr2=0x561525ef4600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T18:31:16.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154669b640 1 --2- 172.21.15.105:0/4101238436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561525e47500 0x561525ef4600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154669b640 1 -- 172.21.15.105:0/4101238436 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f153000f040 con 0x561525e94650 2023-11-28T18:31:16.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154669b640 1 --2- 172.21.15.105:0/4101238436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e94650 0x561525eafa70 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f15380549e0 tx=0x7f1538058370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53736cbab002399b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:16.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f1520ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:31:16.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f1520ff9640 1 -- 172.21.15.105:0/4101238436 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1538060550 con 0x561525e94650 2023-11-28T18:31:16.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f1520ff9640 1 -- 172.21.15.105:0/4101238436 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f153805c630 con 0x561525e94650 2023-11-28T18:31:16.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154b46c000 1 -- 172.21.15.105:0/4101238436 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561525c98110 con 0x561525e94650 2023-11-28T18:31:16.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f1520ff9640 1 -- 172.21.15.105:0/4101238436 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15380607e0 con 0x561525e94650 2023-11-28T18:31:16.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154b46c000 10 client.8853 fetch_fsmap learned FSMap version 124 2023-11-28T18:31:16.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154b46c000 1 -- 172.21.15.105:0/4101238436 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x561525c40490 con 0x561525e94650 2023-11-28T18:31:16.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f1520ff9640 1 -- 172.21.15.105:0/4101238436 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561525c98110 con 0x561525e94650 2023-11-28T18:31:16.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f1520ff9640 1 -- 172.21.15.105:0/4101238436 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 124) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f1538076170 con 0x561525e94650 2023-11-28T18:31:16.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154b46c000 10 client.8853 fetch_fsmap finished waiting for FSMap version 124 2023-11-28T18:31:16.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154b46c000 10 client.8853 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T18:31:16.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154b46c000 20 client.8853 populate_metadata read hostname 'smithi105' 2023-11-28T18:31:16.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154b46c000 1 --2- 172.21.15.105:0/4101238436 >> [v2:172.21.15.148:6833/270170970,v1:172.21.15.148:6835/270170970] conn(0x561525efb460 0x561525efd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:16.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154b46c000 4 client.8853 mds_command: new command op to 7085 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:31:16.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f154b46c000 1 -- 172.21.15.105:0/4101238436 --> [v2:172.21.15.148:6833/270170970,v1:172.21.15.148:6835/270170970] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561525cc6650 con 0x561525efb460 2023-11-28T18:31:16.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.900+0000 7f1545e9a640 1 --2- 172.21.15.105:0/4101238436 >> [v2:172.21.15.148:6833/270170970,v1:172.21.15.148:6835/270170970] conn(0x561525efb460 0x561525efd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:16.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f1545e9a640 1 --2- 172.21.15.105:0/4101238436 >> [v2:172.21.15.148:6833/270170970,v1:172.21.15.148:6835/270170970] conn(0x561525efb460 0x561525efd850 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f9e0fa4a5b36dc9 server_cookie=cbd2052d635f2936 in_seq=0 out_seq=0 2023-11-28T18:31:16.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f1520ff9640 10 client.8853 ms_handle_connect on v2:172.21.15.148:6833/270170970 2023-11-28T18:31:16.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f1520ff9640 1 -- 172.21.15.105:0/4101238436 <== mds.2 v2:172.21.15.148:6833/270170970 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x561525cc6650 con 0x561525efb460 2023-11-28T18:31:16.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f1520ff9640 10 client.8853 handle_command_reply: tid=0 2023-11-28T18:31:16.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f154b46c000 1 client.8853 shutdown 2023-11-28T18:31:16.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f154b46c000 1 -- 172.21.15.105:0/4101238436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e94650 msgr2=0x561525eafa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:16.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f154b46c000 1 --2- 172.21.15.105:0/4101238436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e94650 0x561525eafa70 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f15380549e0 tx=0x7f1538058370 comp rx=0 tx=0).stop 2023-11-28T18:31:16.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f154b46c000 1 -- 172.21.15.105:0/4101238436 shutdown_connections 2023-11-28T18:31:16.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f154b46c000 1 --2- 172.21.15.105:0/4101238436 >> [v2:172.21.15.148:6833/270170970,v1:172.21.15.148:6835/270170970] conn(0x561525efb460 0x561525efd850 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f154b46c000 1 --2- 172.21.15.105:0/4101238436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561525e94650 0x561525eafa70 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f154b46c000 1 --2- 172.21.15.105:0/4101238436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561525e93d30 0x561525ef4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f154b46c000 1 --2- 172.21.15.105:0/4101238436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561525e47500 0x561525ef4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f154b46c000 1 -- 172.21.15.105:0/4101238436 >> 172.21.15.105:0/4101238436 conn(0x561525e748c0 msgr2=0x561525e45fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:16.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f1520ff9640 0 client.8853 ms_handle_reset on v2:172.21.15.148:6833/270170970 2023-11-28T18:31:16.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f154b46c000 1 -- 172.21.15.105:0/4101238436 shutdown_connections 2023-11-28T18:31:16.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f154b46c000 1 -- 172.21.15.105:0/4101238436 wait complete. 2023-11-28T18:31:16.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f154b46c000 20 client.8853 trim_cache size 0 max 16384 2023-11-28T18:31:16.912 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:31:16.912 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:31:16.912 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 2, 2023-11-28T18:31:16.912 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 7085, 2023-11-28T18:31:16.912 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:31:16.912 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:31:16.912 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:31:16.912 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 1.7309971319999999, 2023-11-28T18:31:16.913 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 124, 2023-11-28T18:31:16.913 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 117, 2023-11-28T18:31:16.913 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 117, 2023-11-28T18:31:16.913 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 19.110626372999999 2023-11-28T18:31:16.913 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:31:16.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f15480fe640 1 -- 172.21.15.105:0/3086320817 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15240668d0 msgr2=0x7f1524068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:16.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f15480fe640 1 --2- 172.21.15.105:0/3086320817 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15240668d0 0x7f1524068d90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f153c0af000 tx=0x7f153c0046d0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f15480fe640 1 -- 172.21.15.105:0/3086320817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15400b3c60 msgr2=0x7f15400ad2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:16.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f15480fe640 1 --2- 172.21.15.105:0/3086320817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15400b3c60 0x7f15400ad2b0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f154006ba30 tx=0x7f1538010f20 comp rx=0 tx=0).stop 2023-11-28T18:31:16.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f15480fe640 1 -- 172.21.15.105:0/3086320817 shutdown_connections 2023-11-28T18:31:16.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f15480fe640 1 --2- 172.21.15.105:0/3086320817 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f15240668d0 0x7f1524068d90 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f15480fe640 1 --2- 172.21.15.105:0/3086320817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15400b3c60 0x7f15400ad2b0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f15480fe640 1 --2- 172.21.15.105:0/3086320817 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f154009c440 0x7f154009aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f15480fe640 1 --2- 172.21.15.105:0/3086320817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f152c0039b0 0x7f154015bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:16.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f15480fe640 1 -- 172.21.15.105:0/3086320817 >> 172.21.15.105:0/3086320817 conn(0x7f15400ceb90 msgr2=0x7f15400abf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:16.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f15480fe640 1 -- 172.21.15.105:0/3086320817 shutdown_connections 2023-11-28T18:31:16.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:16.904+0000 7f15480fe640 1 -- 172.21.15.105:0/3086320817 wait complete. 2023-11-28T18:31:16.923 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T18:31:17.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.060+0000 7f980659d640 1 Processor -- start 2023-11-28T18:31:17.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.060+0000 7f980659d640 1 -- start start 2023-11-28T18:31:17.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.060+0000 7f980659d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9800159f40 0x7f98000caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.060+0000 7f980659d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f98000cc760 0x7f98000ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.060+0000 7f980659d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f980006bf50 con 0x7f98000ad680 2023-11-28T18:31:17.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.060+0000 7f980659d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98000ccbc0 con 0x7f98000cc760 2023-11-28T18:31:17.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.060+0000 7f980659d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980015a320 con 0x7f9800159f40 2023-11-28T18:31:17.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.060+0000 7f9804b3a640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f98000ad680 0x7f980015da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50396/0 (socket says 172.21.15.105:50396) 2023-11-28T18:31:17.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.060+0000 7f9804b3a640 1 -- 172.21.15.105:0/3969719547 learned_addr learned my addr 172.21.15.105:0/3969719547 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:17.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.060+0000 7f97ff7fe640 1 --2- 172.21.15.105:0/3969719547 >> v2:172.21.15.105:3300/0 conn(0x7f98000cc760 0x7f98000ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.064+0000 7f97fffff640 1 --2- 172.21.15.105:0/3969719547 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9800159f40 0x7f98000caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.064+0000 7f97ff7fe640 1 -- 172.21.15.105:0/3969719547 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9800159f40 msgr2=0x7f98000caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:17.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.064+0000 7f97ff7fe640 1 --2- 172.21.15.105:0/3969719547 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9800159f40 0x7f98000caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.064+0000 7f97ff7fe640 1 -- 172.21.15.105:0/3969719547 >> v1:172.21.15.105:6789/0 conn(0x7f98000ad680 legacy=0x7f980015da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:17.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.064+0000 7f97feffd640 1 -- 172.21.15.105:0/3969719547 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1292997049 0 0) 0x7f980006bf50 con 0x7f98000ad680 2023-11-28T18:31:17.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.064+0000 7f97ff7fe640 1 -- 172.21.15.105:0/3969719547 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9800071380 con 0x7f98000cc760 2023-11-28T18:31:17.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.064+0000 7f97fffff640 1 --2- 172.21.15.105:0/3969719547 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9800159f40 0x7f98000caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:17.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.064+0000 7f97ff7fe640 1 --2- 172.21.15.105:0/3969719547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98000cc760 0x7f98000ad140 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f97f40097c0 tx=0x7f97f401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22875ccc3f9dddaf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:17.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.064+0000 7f97feffd640 1 -- 172.21.15.105:0/3969719547 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f40ac620 con 0x7f98000cc760 2023-11-28T18:31:17.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.064+0000 7f97feffd640 1 -- 172.21.15.105:0/3969719547 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f40ac780 con 0x7f98000cc760 2023-11-28T18:31:17.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.064+0000 7f97feffd640 1 -- 172.21.15.105:0/3969719547 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f40acb10 con 0x7f98000cc760 2023-11-28T18:31:17.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f980659d640 1 -- 172.21.15.105:0/3969719547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98000cc760 msgr2=0x7f98000ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:17.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f980659d640 1 --2- 172.21.15.105:0/3969719547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98000cc760 0x7f98000ad140 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f97f40097c0 tx=0x7f97f401c3c0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f980659d640 1 -- 172.21.15.105:0/3969719547 shutdown_connections 2023-11-28T18:31:17.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f980659d640 1 --2- 172.21.15.105:0/3969719547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98000cc760 0x7f98000ad140 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f980659d640 1 --2- 172.21.15.105:0/3969719547 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9800159f40 0x7f98000caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f980659d640 1 -- 172.21.15.105:0/3969719547 >> 172.21.15.105:0/3969719547 conn(0x7f98000ceb90 msgr2=0x7f98000cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:17.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f980659d640 1 -- 172.21.15.105:0/3969719547 shutdown_connections 2023-11-28T18:31:17.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f980659d640 1 -- 172.21.15.105:0/3969719547 wait complete. 2023-11-28T18:31:17.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f980659d640 1 Processor -- start 2023-11-28T18:31:17.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f980659d640 1 -- start start 2023-11-28T18:31:17.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f980659d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98000ad680 0x7f98000af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f980659d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98000cc760 0x7f980015cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f980659d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9800159f40 0x7f98001484e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f980659d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980015c930 con 0x7f98000ad680 2023-11-28T18:31:17.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f980659d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980015c2e0 con 0x7f98000cc760 2023-11-28T18:31:17.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f980659d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980015a890 con 0x7f9800159f40 2023-11-28T18:31:17.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f97fffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98000ad680 0x7f98000af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f97fffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98000ad680 0x7f98000af0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55178/0 (socket says 172.21.15.105:55178) 2023-11-28T18:31:17.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f97fffff640 1 -- 172.21.15.105:0/146115616 learned_addr learned my addr 172.21.15.105:0/146115616 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:17.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f9804b3a640 1 --2- 172.21.15.105:0/146115616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9800159f40 0x7f98001484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f97ff7fe640 1 --2- 172.21.15.105:0/146115616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98000cc760 0x7f980015cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f97fffff640 1 -- 172.21.15.105:0/146115616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9800159f40 msgr2=0x7f98001484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:17.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f97fffff640 1 --2- 172.21.15.105:0/146115616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9800159f40 0x7f98001484e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f97fffff640 1 -- 172.21.15.105:0/146115616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98000cc760 msgr2=0x7f980015cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:17.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f97fffff640 1 --2- 172.21.15.105:0/146115616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98000cc760 0x7f980015cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.068+0000 7f97fffff640 1 -- 172.21.15.105:0/146115616 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f4009420 con 0x7f98000ad680 2023-11-28T18:31:17.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.072+0000 7f97fffff640 1 --2- 172.21.15.105:0/146115616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98000ad680 0x7f98000af0e0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f97f000d830 tx=0x7f97f000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:17.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.072+0000 7f97fcff9640 1 -- 172.21.15.105:0/146115616 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f00096c0 con 0x7f98000ad680 2023-11-28T18:31:17.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.072+0000 7f97fcff9640 1 -- 172.21.15.105:0/146115616 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f0009820 con 0x7f98000ad680 2023-11-28T18:31:17.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.072+0000 7f97fcff9640 1 -- 172.21.15.105:0/146115616 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0009bd0 con 0x7f98000ad680 2023-11-28T18:31:17.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.072+0000 7f980659d640 1 -- 172.21.15.105:0/146115616 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980015ab70 con 0x7f98000ad680 2023-11-28T18:31:17.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.072+0000 7f980659d640 1 -- 172.21.15.105:0/146115616 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98000af620 con 0x7f98000ad680 2023-11-28T18:31:17.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.072+0000 7f97fcff9640 1 -- 172.21.15.105:0/146115616 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f97f0014410 con 0x7f98000ad680 2023-11-28T18:31:17.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.072+0000 7f97fcff9640 1 --2- 172.21.15.105:0/146115616 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f97ec0668d0 0x7f97ec068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.072+0000 7f97fcff9640 1 -- 172.21.15.105:0/146115616 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f97f0084f10 con 0x7f98000ad680 2023-11-28T18:31:17.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.072+0000 7f97ff7fe640 1 --2- 172.21.15.105:0/146115616 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f97ec0668d0 0x7f97ec068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.072+0000 7f9809902000 1 Processor -- start 2023-11-28T18:31:17.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f9809902000 1 -- start start 2023-11-28T18:31:17.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f97ff7fe640 1 --2- 172.21.15.105:0/146115616 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f97ec0668d0 0x7f97ec068d90 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f97f40ae000 tx=0x7f97f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:17.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f9809902000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e2026d30 0x55b1e2027110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f9809902000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1e2027650 0x55b1e1fd9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f9809902000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1e1fda500 0x55b1e1fda8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f9809902000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e1ea3110 con 0x55b1e2026d30 2023-11-28T18:31:17.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f9809902000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e1e339a0 con 0x55b1e2027650 2023-11-28T18:31:17.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f9809902000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1e1e30ed0 con 0x55b1e1fda500 2023-11-28T18:31:17.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f97fffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e2026d30 0x55b1e2027110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f97fffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e2026d30 0x55b1e2027110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55190/0 (socket says 172.21.15.105:55190) 2023-11-28T18:31:17.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f97fffff640 1 -- 172.21.15.105:0/2205451196 learned_addr learned my addr 172.21.15.105:0/2205451196 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:17.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f97ff7fe640 1 --2- 172.21.15.105:0/2205451196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1e2027650 0x55b1e1fd9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f9804b3a640 1 --2- 172.21.15.105:0/2205451196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1e1fda500 0x55b1e1fda8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f97fffff640 1 -- 172.21.15.105:0/2205451196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1e1fda500 msgr2=0x55b1e1fda8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:17.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f97fffff640 1 --2- 172.21.15.105:0/2205451196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1e1fda500 0x55b1e1fda8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f97fffff640 1 -- 172.21.15.105:0/2205451196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1e2027650 msgr2=0x55b1e1fd9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:17.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f97fffff640 1 --2- 172.21.15.105:0/2205451196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1e2027650 0x55b1e1fd9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f97fffff640 1 -- 172.21.15.105:0/2205451196 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1e1b24890 con 0x55b1e2026d30 2023-11-28T18:31:17.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f97ff7fe640 1 --2- 172.21.15.105:0/2205451196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1e2027650 0x55b1e1fd9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:17.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f97fffff640 1 --2- 172.21.15.105:0/2205451196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e2026d30 0x55b1e2027110 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f97f0083230 tx=0x7f97f0055900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947a5c49889aad97 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:17.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f97e67fc640 1 -- 172.21.15.105:0/2205451196 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0059e10 con 0x55b1e2026d30 2023-11-28T18:31:17.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f97e67fc640 1 -- 172.21.15.105:0/2205451196 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f005dfc0 con 0x55b1e2026d30 2023-11-28T18:31:17.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f97e67fc640 1 -- 172.21.15.105:0/2205451196 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f00523d0 con 0x55b1e2026d30 2023-11-28T18:31:17.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f9809902000 1 -- 172.21.15.105:0/2205451196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e2026d30 msgr2=0x55b1e2027110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:17.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f9809902000 1 --2- 172.21.15.105:0/2205451196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e2026d30 0x55b1e2027110 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f97f0083230 tx=0x7f97f0055900 comp rx=0 tx=0).stop 2023-11-28T18:31:17.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f9809902000 1 -- 172.21.15.105:0/2205451196 shutdown_connections 2023-11-28T18:31:17.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f9809902000 1 --2- 172.21.15.105:0/2205451196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1e1fda500 0x55b1e1fda8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f9809902000 1 --2- 172.21.15.105:0/2205451196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1e2027650 0x55b1e1fd9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f9809902000 1 --2- 172.21.15.105:0/2205451196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e2026d30 0x55b1e2027110 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f9809902000 1 -- 172.21.15.105:0/2205451196 >> 172.21.15.105:0/2205451196 conn(0x55b1e20078c0 msgr2=0x55b1e2009cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:17.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f9809902000 1 -- 172.21.15.105:0/2205451196 shutdown_connections 2023-11-28T18:31:17.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f9809902000 1 -- 172.21.15.105:0/2205451196 wait complete. 2023-11-28T18:31:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f9809902000 1 -- start start 2023-11-28T18:31:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.076+0000 7f9809902000 1 Processor -- start 2023-11-28T18:31:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f9809902000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1e1fda500 0x55b1e1fe04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f9809902000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e2026d30 0x55b1e1fe0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f9809902000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1e2027650 0x55b1e1fe0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f9809902000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e1e2c7c0 con 0x55b1e2026d30 2023-11-28T18:31:17.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f9809902000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e1de4bf0 con 0x55b1e1fda500 2023-11-28T18:31:17.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f9809902000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1e1deaa90 con 0x55b1e2027650 2023-11-28T18:31:17.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f9809902000 10 client.0 _ll_register_callbacks cb 0x55b1e1fc7e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:31:17.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f97ff7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e2026d30 0x55b1e1fe0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f97ff7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e2026d30 0x55b1e1fe0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55204/0 (socket says 172.21.15.105:55204) 2023-11-28T18:31:17.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f97ff7fe640 1 -- 172.21.15.105:0/2587549320 learned_addr learned my addr 172.21.15.105:0/2587549320 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:17.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f97fffff640 1 --2- 172.21.15.105:0/2587549320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1e1fda500 0x55b1e1fe04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f9804b3a640 1 --2- 172.21.15.105:0/2587549320 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1e2027650 0x55b1e1fe0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f97ff7fe640 1 -- 172.21.15.105:0/2587549320 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1e2027650 msgr2=0x55b1e1fe0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:17.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f97ff7fe640 1 --2- 172.21.15.105:0/2587549320 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1e2027650 0x55b1e1fe0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f97ff7fe640 1 -- 172.21.15.105:0/2587549320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1e1fda500 msgr2=0x55b1e1fe04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:17.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f97ff7fe640 1 --2- 172.21.15.105:0/2587549320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1e1fda500 0x55b1e1fe04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f97ff7fe640 1 -- 172.21.15.105:0/2587549320 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98000af620 con 0x55b1e2026d30 2023-11-28T18:31:17.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f9804b3a640 1 --2- 172.21.15.105:0/2587549320 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1e2027650 0x55b1e1fe0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:17.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f97ff7fe640 1 --2- 172.21.15.105:0/2587549320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e2026d30 0x55b1e1fe0a20 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f97f40097c0 tx=0x7f97f401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2487ceb20f9707 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:17.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f97deffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:31:17.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f97deffd640 1 -- 172.21.15.105:0/2587549320 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f40c33f0 con 0x55b1e2026d30 2023-11-28T18:31:17.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f9809902000 1 -- 172.21.15.105:0/2587549320 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1e1dee470 con 0x55b1e2026d30 2023-11-28T18:31:17.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f97deffd640 1 -- 172.21.15.105:0/2587549320 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f97f40c3550 con 0x55b1e2026d30 2023-11-28T18:31:17.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f97deffd640 1 -- 172.21.15.105:0/2587549320 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97f40c3710 con 0x55b1e2026d30 2023-11-28T18:31:17.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f9809902000 10 client.8868 fetch_fsmap learned FSMap version 124 2023-11-28T18:31:17.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f9809902000 1 -- 172.21.15.105:0/2587549320 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x55b1e1ded190 con 0x55b1e2026d30 2023-11-28T18:31:17.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f97deffd640 1 -- 172.21.15.105:0/2587549320 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b1e1dee470 con 0x55b1e2026d30 2023-11-28T18:31:17.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f97deffd640 1 -- 172.21.15.105:0/2587549320 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 124) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f97f40c2070 con 0x55b1e2026d30 2023-11-28T18:31:17.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f9809902000 10 client.8868 fetch_fsmap finished waiting for FSMap version 124 2023-11-28T18:31:17.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f9809902000 10 client.8868 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:31:17.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f9809902000 20 client.8868 populate_metadata read hostname 'smithi105' 2023-11-28T18:31:17.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f9809902000 1 --2- 172.21.15.105:0/2587549320 >> [v2:172.21.15.105:6834/3373504883,v1:172.21.15.105:6836/3373504883] conn(0x55b1e1fe1970 0x55b1e2038ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f9809902000 4 client.8868 mds_command: new command op to 8640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:31:17.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f9809902000 1 -- 172.21.15.105:0/2587549320 --> [v2:172.21.15.105:6834/3373504883,v1:172.21.15.105:6836/3373504883] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1e1df1b50 con 0x55b1e1fe1970 2023-11-28T18:31:17.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f97fffff640 1 --2- 172.21.15.105:0/2587549320 >> [v2:172.21.15.105:6834/3373504883,v1:172.21.15.105:6836/3373504883] conn(0x55b1e1fe1970 0x55b1e2038ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f97fffff640 1 --2- 172.21.15.105:0/2587549320 >> [v2:172.21.15.105:6834/3373504883,v1:172.21.15.105:6836/3373504883] conn(0x55b1e1fe1970 0x55b1e2038ff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=288265dc6fd1f417 server_cookie=3ad34db421d89351 in_seq=0 out_seq=0 2023-11-28T18:31:17.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.080+0000 7f97deffd640 10 client.8868 ms_handle_connect on v2:172.21.15.105:6834/3373504883 2023-11-28T18:31:17.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.084+0000 7f97deffd640 1 -- 172.21.15.105:0/2587549320 <== mds.1 v2:172.21.15.105:6834/3373504883 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b1e1df1b50 con 0x55b1e1fe1970 2023-11-28T18:31:17.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.084+0000 7f97deffd640 10 client.8868 handle_command_reply: tid=0 2023-11-28T18:31:17.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.084+0000 7f9809902000 1 client.8868 shutdown 2023-11-28T18:31:17.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.084+0000 7f9809902000 1 -- 172.21.15.105:0/2587549320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e2026d30 msgr2=0x55b1e1fe0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:17.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.084+0000 7f9809902000 1 --2- 172.21.15.105:0/2587549320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e2026d30 0x55b1e1fe0a20 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f97f40097c0 tx=0x7f97f401ac00 comp rx=0 tx=0).stop 2023-11-28T18:31:17.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.084+0000 7f9809902000 1 -- 172.21.15.105:0/2587549320 shutdown_connections 2023-11-28T18:31:17.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.084+0000 7f9809902000 1 --2- 172.21.15.105:0/2587549320 >> [v2:172.21.15.105:6834/3373504883,v1:172.21.15.105:6836/3373504883] conn(0x55b1e1fe1970 0x55b1e2038ff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.084+0000 7f9809902000 1 --2- 172.21.15.105:0/2587549320 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1e2027650 0x55b1e1fe0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.084+0000 7f9809902000 1 --2- 172.21.15.105:0/2587549320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e2026d30 0x55b1e1fe0a20 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.084+0000 7f97deffd640 0 client.8868 ms_handle_reset on v2:172.21.15.105:6834/3373504883 2023-11-28T18:31:17.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.084+0000 7f9809902000 1 --2- 172.21.15.105:0/2587549320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1e1fda500 0x55b1e1fe04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.084+0000 7f9809902000 1 -- 172.21.15.105:0/2587549320 >> 172.21.15.105:0/2587549320 conn(0x55b1e20078c0 msgr2=0x55b1e1fd96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:17.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.084+0000 7f9809902000 1 -- 172.21.15.105:0/2587549320 shutdown_connections 2023-11-28T18:31:17.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.084+0000 7f9809902000 1 -- 172.21.15.105:0/2587549320 wait complete. 2023-11-28T18:31:17.089 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.084+0000 7f9809902000 20 client.8868 trim_cache size 0 max 16384 2023-11-28T18:31:17.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f9809902000 1 Processor -- start 2023-11-28T18:31:17.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f9809902000 1 -- start start 2023-11-28T18:31:17.100 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f9809902000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e1fda500 0x55b1e20063a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f9809902000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1e2026d30 0x55b1e20068e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f9809902000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1e2027650 0x55b1e2038ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f9809902000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e1ae5940 con 0x55b1e1fda500 2023-11-28T18:31:17.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f9809902000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e1fdd100 con 0x55b1e2026d30 2023-11-28T18:31:17.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f9809902000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1e1deb8f0 con 0x55b1e2027650 2023-11-28T18:31:17.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f97fffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e1fda500 0x55b1e20063a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f97fffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e1fda500 0x55b1e20063a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55218/0 (socket says 172.21.15.105:55218) 2023-11-28T18:31:17.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f97fffff640 1 -- 172.21.15.105:0/2315171881 learned_addr learned my addr 172.21.15.105:0/2315171881 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:17.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f97fffff640 1 -- 172.21.15.105:0/2315171881 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1e2027650 msgr2=0x55b1e2038ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:17.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f9804b3a640 1 --2- 172.21.15.105:0/2315171881 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1e2027650 0x55b1e2038ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f97fffff640 1 --2- 172.21.15.105:0/2315171881 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1e2027650 0x55b1e2038ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f97fffff640 1 -- 172.21.15.105:0/2315171881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1e2026d30 msgr2=0x55b1e20068e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:17.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f97ff7fe640 1 --2- 172.21.15.105:0/2315171881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1e2026d30 0x55b1e20068e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f97fffff640 1 --2- 172.21.15.105:0/2315171881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1e2026d30 0x55b1e20068e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f97fffff640 1 -- 172.21.15.105:0/2315171881 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1e1dd3490 con 0x55b1e1fda500 2023-11-28T18:31:17.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f97ff7fe640 1 --2- 172.21.15.105:0/2315171881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1e2026d30 0x55b1e20068e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:31:17.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f97fffff640 1 --2- 172.21.15.105:0/2315171881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e1fda500 0x55b1e20063a0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f97f00832f0 tx=0x7f97f0052130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea99742256ccfc40 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:17.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f97e6ffd640 1 -- 172.21.15.105:0/2315171881 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0059e10 con 0x55b1e1fda500 2023-11-28T18:31:17.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f97e6ffd640 1 -- 172.21.15.105:0/2315171881 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f0054060 con 0x55b1e1fda500 2023-11-28T18:31:17.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f97e6ffd640 1 -- 172.21.15.105:0/2315171881 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f005db30 con 0x55b1e1fda500 2023-11-28T18:31:17.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f9809902000 1 -- 172.21.15.105:0/2315171881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e1fda500 msgr2=0x55b1e20063a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:17.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f9809902000 1 --2- 172.21.15.105:0/2315171881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e1fda500 0x55b1e20063a0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f97f00832f0 tx=0x7f97f0052130 comp rx=0 tx=0).stop 2023-11-28T18:31:17.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f9809902000 1 -- 172.21.15.105:0/2315171881 shutdown_connections 2023-11-28T18:31:17.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f9809902000 1 --2- 172.21.15.105:0/2315171881 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1e2027650 0x55b1e2038ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f9809902000 1 --2- 172.21.15.105:0/2315171881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1e2026d30 0x55b1e20068e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f9809902000 1 --2- 172.21.15.105:0/2315171881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e1fda500 0x55b1e20063a0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f9809902000 1 -- 172.21.15.105:0/2315171881 >> 172.21.15.105:0/2315171881 conn(0x55b1e20078c0 msgr2=0x55b1e1fd8720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:17.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f9809902000 1 -- 172.21.15.105:0/2315171881 shutdown_connections 2023-11-28T18:31:17.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f9809902000 1 -- 172.21.15.105:0/2315171881 wait complete. 2023-11-28T18:31:17.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f9809902000 1 -- start start 2023-11-28T18:31:17.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.100+0000 7f9809902000 1 Processor -- start 2023-11-28T18:31:17.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e1fda500 0x55b1e2087810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1e2026d30 0x55b1e2087d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1e2027650 0x55b1e2042a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e1e28130 con 0x55b1e1fda500 2023-11-28T18:31:17.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e1fd4430 con 0x55b1e2026d30 2023-11-28T18:31:17.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1e1e2b650 con 0x55b1e2027650 2023-11-28T18:31:17.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 10 client.0 _ll_register_callbacks cb 0x55b1e1ea3060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:31:17.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f97fffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e1fda500 0x55b1e2087810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f97fffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e1fda500 0x55b1e2087810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55226/0 (socket says 172.21.15.105:55226) 2023-11-28T18:31:17.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f97fffff640 1 -- 172.21.15.105:0/3224256896 learned_addr learned my addr 172.21.15.105:0/3224256896 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:17.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f97ff7fe640 1 --2- 172.21.15.105:0/3224256896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1e2026d30 0x55b1e2087d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9804b3a640 1 --2- 172.21.15.105:0/3224256896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1e2027650 0x55b1e2042a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f97fffff640 1 -- 172.21.15.105:0/3224256896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1e2027650 msgr2=0x55b1e2042a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:17.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f97fffff640 1 --2- 172.21.15.105:0/3224256896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1e2027650 0x55b1e2042a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f97fffff640 1 -- 172.21.15.105:0/3224256896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1e2026d30 msgr2=0x55b1e2087d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:17.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f97fffff640 1 --2- 172.21.15.105:0/3224256896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1e2026d30 0x55b1e2087d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f97fffff640 1 -- 172.21.15.105:0/3224256896 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b1e1b24890 con 0x55b1e1fda500 2023-11-28T18:31:17.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f97fffff640 1 --2- 172.21.15.105:0/3224256896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e1fda500 0x55b1e2087810 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f97f00521c0 tx=0x7f97f000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11af0d01b0762b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:17.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f97deffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:31:17.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f97deffd640 1 -- 172.21.15.105:0/3224256896 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0078070 con 0x55b1e1fda500 2023-11-28T18:31:17.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f97deffd640 1 -- 172.21.15.105:0/3224256896 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f97f0059e10 con 0x55b1e1fda500 2023-11-28T18:31:17.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 1 -- 172.21.15.105:0/3224256896 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1e1e2b110 con 0x55b1e1fda500 2023-11-28T18:31:17.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f97deffd640 1 -- 172.21.15.105:0/3224256896 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97f00523d0 con 0x55b1e1fda500 2023-11-28T18:31:17.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 10 client.8874 fetch_fsmap learned FSMap version 124 2023-11-28T18:31:17.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 1 -- 172.21.15.105:0/3224256896 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x55b1e1b24cc0 con 0x55b1e1fda500 2023-11-28T18:31:17.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f97deffd640 1 -- 172.21.15.105:0/3224256896 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b1e1e2b110 con 0x55b1e1fda500 2023-11-28T18:31:17.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f97deffd640 1 -- 172.21.15.105:0/3224256896 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 124) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f97f007b030 con 0x55b1e1fda500 2023-11-28T18:31:17.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 10 client.8874 fetch_fsmap finished waiting for FSMap version 124 2023-11-28T18:31:17.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 10 client.8874 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:31:17.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 20 client.8874 populate_metadata read hostname 'smithi105' 2023-11-28T18:31:17.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 1 --2- 172.21.15.105:0/3224256896 >> [v2:172.21.15.105:6834/3373504883,v1:172.21.15.105:6836/3373504883] conn(0x55b1e20433d0 0x55b1e20437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 4 client.8874 mds_command: new command op to 8640 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:31:17.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 1 -- 172.21.15.105:0/3224256896 --> [v2:172.21.15.105:6834/3373504883,v1:172.21.15.105:6836/3373504883] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b1e1e59650 con 0x55b1e20433d0 2023-11-28T18:31:17.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f97ff7fe640 1 --2- 172.21.15.105:0/3224256896 >> [v2:172.21.15.105:6834/3373504883,v1:172.21.15.105:6836/3373504883] conn(0x55b1e20433d0 0x55b1e20437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f97ff7fe640 1 --2- 172.21.15.105:0/3224256896 >> [v2:172.21.15.105:6834/3373504883,v1:172.21.15.105:6836/3373504883] conn(0x55b1e20433d0 0x55b1e20437b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a354bba712876800 server_cookie=5e8434a7f92a981c in_seq=0 out_seq=0 2023-11-28T18:31:17.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f97deffd640 10 client.8874 ms_handle_connect on v2:172.21.15.105:6834/3373504883 2023-11-28T18:31:17.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f97deffd640 1 -- 172.21.15.105:0/3224256896 <== mds.1 v2:172.21.15.105:6834/3373504883 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x55b1e1e59650 con 0x55b1e20433d0 2023-11-28T18:31:17.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f97deffd640 10 client.8874 handle_command_reply: tid=0 2023-11-28T18:31:17.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 1 client.8874 shutdown 2023-11-28T18:31:17.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 1 -- 172.21.15.105:0/3224256896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e1fda500 msgr2=0x55b1e2087810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:17.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 1 --2- 172.21.15.105:0/3224256896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e1fda500 0x55b1e2087810 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f97f00521c0 tx=0x7f97f000bfd0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 1 -- 172.21.15.105:0/3224256896 shutdown_connections 2023-11-28T18:31:17.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 1 --2- 172.21.15.105:0/3224256896 >> [v2:172.21.15.105:6834/3373504883,v1:172.21.15.105:6836/3373504883] conn(0x55b1e20433d0 0x55b1e20437b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 1 --2- 172.21.15.105:0/3224256896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1e2027650 0x55b1e2042a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 1 --2- 172.21.15.105:0/3224256896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1e2026d30 0x55b1e2087d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 1 --2- 172.21.15.105:0/3224256896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1e1fda500 0x55b1e2087810 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 1 -- 172.21.15.105:0/3224256896 >> 172.21.15.105:0/3224256896 conn(0x55b1e20078c0 msgr2=0x55b1e1fd8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:17.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f97deffd640 0 client.8874 ms_handle_reset on v2:172.21.15.105:6834/3373504883 2023-11-28T18:31:17.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 1 -- 172.21.15.105:0/3224256896 shutdown_connections 2023-11-28T18:31:17.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 1 -- 172.21.15.105:0/3224256896 wait complete. 2023-11-28T18:31:17.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.104+0000 7f9809902000 20 client.8874 trim_cache size 0 max 16384 2023-11-28T18:31:17.113 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:31:17.113 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:31:17.113 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 1, 2023-11-28T18:31:17.113 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 8640, 2023-11-28T18:31:17.113 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:31:17.113 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:31:17.113 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:31:17.113 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 2.9532621429999999, 2023-11-28T18:31:17.113 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 124, 2023-11-28T18:31:17.113 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 117, 2023-11-28T18:31:17.113 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 117, 2023-11-28T18:31:17.114 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 19.319136385 2023-11-28T18:31:17.114 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:31:17.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.108+0000 7f980659d640 1 -- 172.21.15.105:0/146115616 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f97ec0668d0 msgr2=0x7f97ec068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:17.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.108+0000 7f980659d640 1 --2- 172.21.15.105:0/146115616 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f97ec0668d0 0x7f97ec068d90 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f97f40ae000 tx=0x7f97f4008040 comp rx=0 tx=0).stop 2023-11-28T18:31:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.108+0000 7f980659d640 1 -- 172.21.15.105:0/146115616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98000ad680 msgr2=0x7f98000af0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.108+0000 7f980659d640 1 --2- 172.21.15.105:0/146115616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98000ad680 0x7f98000af0e0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f97f000d830 tx=0x7f97f000dcf0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.108+0000 7f980659d640 1 -- 172.21.15.105:0/146115616 shutdown_connections 2023-11-28T18:31:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.108+0000 7f980659d640 1 --2- 172.21.15.105:0/146115616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9800159f40 0x7f98001484e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.108+0000 7f980659d640 1 --2- 172.21.15.105:0/146115616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98000cc760 0x7f980015cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.108+0000 7f980659d640 1 --2- 172.21.15.105:0/146115616 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f97ec0668d0 0x7f97ec068d90 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.108+0000 7f980659d640 1 --2- 172.21.15.105:0/146115616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98000ad680 0x7f98000af0e0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.108+0000 7f980659d640 1 -- 172.21.15.105:0/146115616 >> 172.21.15.105:0/146115616 conn(0x7f98000ceb90 msgr2=0x7f98000ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.108+0000 7f980659d640 1 -- 172.21.15.105:0/146115616 shutdown_connections 2023-11-28T18:31:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.108+0000 7f980659d640 1 -- 172.21.15.105:0/146115616 wait complete. 2023-11-28T18:31:17.126 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T18:31:17.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.264+0000 7fa84038d640 1 Processor -- start 2023-11-28T18:31:17.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.264+0000 7fa84038d640 1 -- start start 2023-11-28T18:31:17.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.268+0000 7fa84038d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa8380b4290 0x7fa8380b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.268+0000 7fa84038d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa838071960 0x7fa83815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.268+0000 7fa84038d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8380b4bb0 con 0x7fa83815b030 2023-11-28T18:31:17.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.268+0000 7fa84038d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8380701d0 con 0x7fa8380b4290 2023-11-28T18:31:17.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.268+0000 7fa84038d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83806df80 con 0x7fa838071960 2023-11-28T18:31:17.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.268+0000 7fa83e129640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa8380b4290 0x7fa8380b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.268+0000 7fa83e129640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa8380b4290 0x7fa8380b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55236/0 (socket says 172.21.15.105:55236) 2023-11-28T18:31:17.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.268+0000 7fa83e129640 1 -- 172.21.15.105:0/3283755534 learned_addr learned my addr 172.21.15.105:0/3283755534 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:17.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.268+0000 7fa83d928640 1 --2- 172.21.15.105:0/3283755534 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa838071960 0x7fa83815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.268+0000 7fa83e129640 1 -- 172.21.15.105:0/3283755534 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa838071960 msgr2=0x7fa83815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:17.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.268+0000 7fa83e129640 1 --2- 172.21.15.105:0/3283755534 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa838071960 0x7fa83815f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.268+0000 7fa83e129640 1 -- 172.21.15.105:0/3283755534 >> v1:172.21.15.148:6789/0 conn(0x7fa83815b030 legacy=0x7fa8380709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:17.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.268+0000 7fa83e129640 1 -- 172.21.15.105:0/3283755534 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa838071df0 con 0x7fa8380b4290 2023-11-28T18:31:17.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.268+0000 7fa83e129640 1 --2- 172.21.15.105:0/3283755534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8380b4290 0x7fa8380b4670 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fa834009870 tx=0x7fa83401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e4961a42e4b34f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:17.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.268+0000 7fa83d127640 1 -- 172.21.15.105:0/3283755534 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8340aa670 con 0x7fa8380b4290 2023-11-28T18:31:17.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa83d127640 1 -- 172.21.15.105:0/3283755534 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8340aa7d0 con 0x7fa8380b4290 2023-11-28T18:31:17.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa83d127640 1 -- 172.21.15.105:0/3283755534 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8340aab60 con 0x7fa8380b4290 2023-11-28T18:31:17.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa84038d640 1 -- 172.21.15.105:0/3283755534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8380b4290 msgr2=0x7fa8380b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:17.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa84038d640 1 --2- 172.21.15.105:0/3283755534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8380b4290 0x7fa8380b4670 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fa834009870 tx=0x7fa83401c510 comp rx=0 tx=0).stop 2023-11-28T18:31:17.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa84038d640 1 -- 172.21.15.105:0/3283755534 shutdown_connections 2023-11-28T18:31:17.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa84038d640 1 --2- 172.21.15.105:0/3283755534 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa838071960 0x7fa83815f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa84038d640 1 --2- 172.21.15.105:0/3283755534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8380b4290 0x7fa8380b4670 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa84038d640 1 -- 172.21.15.105:0/3283755534 >> 172.21.15.105:0/3283755534 conn(0x7fa8380ceb90 msgr2=0x7fa8380cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:17.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa84038d640 1 -- 172.21.15.105:0/3283755534 shutdown_connections 2023-11-28T18:31:17.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa84038d640 1 -- 172.21.15.105:0/3283755534 wait complete. 2023-11-28T18:31:17.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa84038d640 1 Processor -- start 2023-11-28T18:31:17.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa84038d640 1 -- start start 2023-11-28T18:31:17.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa84038d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8380b4290 0x7fa83814b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa84038d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa83815b030 0x7fa838153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa84038d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa83814bb70 0x7fa838159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa84038d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8381549f0 con 0x7fa8380b4290 2023-11-28T18:31:17.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa84038d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa838154e80 con 0x7fa83815b030 2023-11-28T18:31:17.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa84038d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa838153de0 con 0x7fa83814bb70 2023-11-28T18:31:17.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa83e129640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8380b4290 0x7fa83814b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa83e129640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8380b4290 0x7fa83814b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55252/0 (socket says 172.21.15.105:55252) 2023-11-28T18:31:17.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa83d928640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa83815b030 0x7fa838153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa83e129640 1 -- 172.21.15.105:0/2122908129 learned_addr learned my addr 172.21.15.105:0/2122908129 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:17.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.272+0000 7fa83e92a640 1 --2- 172.21.15.105:0/2122908129 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa83814bb70 0x7fa838159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.276+0000 7fa83e129640 1 -- 172.21.15.105:0/2122908129 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa83814bb70 msgr2=0x7fa838159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:17.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.276+0000 7fa83e129640 1 --2- 172.21.15.105:0/2122908129 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa83814bb70 0x7fa838159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.276+0000 7fa83e129640 1 -- 172.21.15.105:0/2122908129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa83815b030 msgr2=0x7fa838153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:17.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.276+0000 7fa83e129640 1 --2- 172.21.15.105:0/2122908129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa83815b030 0x7fa838153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.276+0000 7fa83e129640 1 -- 172.21.15.105:0/2122908129 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8340094d0 con 0x7fa8380b4290 2023-11-28T18:31:17.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.276+0000 7fa83e92a640 1 --2- 172.21.15.105:0/2122908129 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa83814bb70 0x7fa838159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:17.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.276+0000 7fa83e129640 1 --2- 172.21.15.105:0/2122908129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8380b4290 0x7fa83814b1f0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fa8340ac040 tx=0x7fa83401ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:17.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.276+0000 7fa82effd640 1 -- 172.21.15.105:0/2122908129 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8340aa670 con 0x7fa8380b4290 2023-11-28T18:31:17.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.276+0000 7fa82effd640 1 -- 172.21.15.105:0/2122908129 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8340b3820 con 0x7fa8380b4290 2023-11-28T18:31:17.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.276+0000 7fa82effd640 1 -- 172.21.15.105:0/2122908129 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8340b3bd0 con 0x7fa8380b4290 2023-11-28T18:31:17.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.276+0000 7fa84038d640 1 -- 172.21.15.105:0/2122908129 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8381540c0 con 0x7fa8380b4290 2023-11-28T18:31:17.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.276+0000 7fa84038d640 1 -- 172.21.15.105:0/2122908129 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa838154680 con 0x7fa8380b4290 2023-11-28T18:31:17.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.276+0000 7fa82effd640 1 -- 172.21.15.105:0/2122908129 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa8340b3d30 con 0x7fa8380b4290 2023-11-28T18:31:17.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa82effd640 1 --2- 172.21.15.105:0/2122908129 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa81c0668d0 0x7fa81c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa82effd640 1 -- 172.21.15.105:0/2122908129 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa83411ee50 con 0x7fa8380b4290 2023-11-28T18:31:17.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa83d928640 1 --2- 172.21.15.105:0/2122908129 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa81c0668d0 0x7fa81c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa8436ea000 1 Processor -- start 2023-11-28T18:31:17.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa8436ea000 1 -- start start 2023-11-28T18:31:17.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa83d928640 1 --2- 172.21.15.105:0/2122908129 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa81c0668d0 0x7fa81c068d90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa8380cc420 tx=0x7fa828009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:17.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa8436ea000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c147330 0x558b7c147710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa8436ea000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b7c147c50 0x558b7c12bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa8436ea000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558b7c12c2a0 0x558b7c12c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa8436ea000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7bfbf110 con 0x558b7c147330 2023-11-28T18:31:17.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa8436ea000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7bf4f9a0 con 0x558b7c147c50 2023-11-28T18:31:17.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa8436ea000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b7bf4ced0 con 0x558b7c12c2a0 2023-11-28T18:31:17.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa83e129640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c147330 0x558b7c147710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa83e129640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c147330 0x558b7c147710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55266/0 (socket says 172.21.15.105:55266) 2023-11-28T18:31:17.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa83e129640 1 -- 172.21.15.105:0/1607047225 learned_addr learned my addr 172.21.15.105:0/1607047225 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:17.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa83d928640 1 --2- 172.21.15.105:0/1607047225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b7c147c50 0x558b7c12bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa83e92a640 1 --2- 172.21.15.105:0/1607047225 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558b7c12c2a0 0x558b7c12c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa83e129640 1 -- 172.21.15.105:0/1607047225 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558b7c12c2a0 msgr2=0x558b7c12c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:17.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa83e129640 1 --2- 172.21.15.105:0/1607047225 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558b7c12c2a0 0x558b7c12c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa83e129640 1 -- 172.21.15.105:0/1607047225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b7c147c50 msgr2=0x558b7c12bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:17.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa83e129640 1 --2- 172.21.15.105:0/1607047225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b7c147c50 0x558b7c12bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa83e129640 1 -- 172.21.15.105:0/1607047225 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b7bc40890 con 0x558b7c147330 2023-11-28T18:31:17.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa83e129640 1 --2- 172.21.15.105:0/1607047225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c147330 0x558b7c147710 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fa834009280 tx=0x7fa8340192b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a51468e329ab3ef server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:17.283 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa81bfff640 1 -- 172.21.15.105:0/1607047225 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8340f0ea0 con 0x558b7c147330 2023-11-28T18:31:17.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa81bfff640 1 -- 172.21.15.105:0/1607047225 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8340f7e70 con 0x558b7c147330 2023-11-28T18:31:17.284 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa81bfff640 1 -- 172.21.15.105:0/1607047225 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8340ec330 con 0x558b7c147330 2023-11-28T18:31:17.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa8436ea000 1 -- 172.21.15.105:0/1607047225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c147330 msgr2=0x558b7c147710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:17.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa8436ea000 1 --2- 172.21.15.105:0/1607047225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c147330 0x558b7c147710 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fa834009280 tx=0x7fa8340192b0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa8436ea000 1 -- 172.21.15.105:0/1607047225 shutdown_connections 2023-11-28T18:31:17.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa8436ea000 1 --2- 172.21.15.105:0/1607047225 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558b7c12c2a0 0x558b7c12c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa8436ea000 1 --2- 172.21.15.105:0/1607047225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b7c147c50 0x558b7c12bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa8436ea000 1 --2- 172.21.15.105:0/1607047225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c147330 0x558b7c147710 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.280+0000 7fa8436ea000 1 -- 172.21.15.105:0/1607047225 >> 172.21.15.105:0/1607047225 conn(0x558b7c1238c0 msgr2=0x558b7c125cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:17.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8436ea000 1 -- 172.21.15.105:0/1607047225 shutdown_connections 2023-11-28T18:31:17.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8436ea000 1 -- 172.21.15.105:0/1607047225 wait complete. 2023-11-28T18:31:17.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8436ea000 1 -- start start 2023-11-28T18:31:17.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8436ea000 1 Processor -- start 2023-11-28T18:31:17.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8436ea000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c12c2a0 0x558b7c0fccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8436ea000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b7c147330 0x558b7c0fd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8436ea000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558b7c147c50 0x558b7c0fd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8436ea000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7bf487c0 con 0x558b7c12c2a0 2023-11-28T18:31:17.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8436ea000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7bf00bf0 con 0x558b7c147330 2023-11-28T18:31:17.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8436ea000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b7bf06a90 con 0x558b7c147c50 2023-11-28T18:31:17.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8436ea000 10 client.0 _ll_register_callbacks cb 0x558b7c0e3e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:31:17.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa83e129640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c12c2a0 0x558b7c0fccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa83e129640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c12c2a0 0x558b7c0fccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55284/0 (socket says 172.21.15.105:55284) 2023-11-28T18:31:17.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa83e129640 1 -- 172.21.15.105:0/2982013521 learned_addr learned my addr 172.21.15.105:0/2982013521 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:17.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa83e92a640 1 --2- 172.21.15.105:0/2982013521 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558b7c147c50 0x558b7c0fd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa83d928640 1 --2- 172.21.15.105:0/2982013521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b7c147330 0x558b7c0fd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa83e129640 1 -- 172.21.15.105:0/2982013521 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558b7c147c50 msgr2=0x558b7c0fd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:17.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa83e129640 1 --2- 172.21.15.105:0/2982013521 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558b7c147c50 0x558b7c0fd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa83e129640 1 -- 172.21.15.105:0/2982013521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b7c147330 msgr2=0x558b7c0fd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:17.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa83e129640 1 --2- 172.21.15.105:0/2982013521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b7c147330 0x558b7c0fd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa83e129640 1 -- 172.21.15.105:0/2982013521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa838154680 con 0x558b7c12c2a0 2023-11-28T18:31:17.289 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa83e129640 1 --2- 172.21.15.105:0/2982013521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c12c2a0 0x558b7c0fccb0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fa8340ef920 tx=0x7fa83401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebfcb81e894aa5c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8197fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:31:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8197fa640 1 -- 172.21.15.105:0/2982013521 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8340f7be0 con 0x558b7c12c2a0 2023-11-28T18:31:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8197fa640 1 -- 172.21.15.105:0/2982013521 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa8340f7e70 con 0x558b7c12c2a0 2023-11-28T18:31:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8436ea000 1 -- 172.21.15.105:0/2982013521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b7bf0a470 con 0x558b7c12c2a0 2023-11-28T18:31:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8197fa640 1 -- 172.21.15.105:0/2982013521 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa834107170 con 0x558b7c12c2a0 2023-11-28T18:31:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8436ea000 10 client.8886 fetch_fsmap learned FSMap version 124 2023-11-28T18:31:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8436ea000 1 -- 172.21.15.105:0/2982013521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x558b7bf09190 con 0x558b7c12c2a0 2023-11-28T18:31:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8197fa640 1 -- 172.21.15.105:0/2982013521 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558b7bf0a470 con 0x558b7c12c2a0 2023-11-28T18:31:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8197fa640 1 -- 172.21.15.105:0/2982013521 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 124) v1 ==== 3318+0+0 (secure 0 0 0) 0x7fa8340a8050 con 0x558b7c12c2a0 2023-11-28T18:31:17.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8436ea000 10 client.8886 fetch_fsmap finished waiting for FSMap version 124 2023-11-28T18:31:17.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8436ea000 10 client.8886 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:31:17.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8436ea000 20 client.8886 populate_metadata read hostname 'smithi105' 2023-11-28T18:31:17.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8436ea000 1 --2- 172.21.15.105:0/2982013521 >> [v2:172.21.15.105:6835/764864050,v1:172.21.15.105:6837/764864050] conn(0x558b7c0fe140 0x558b7c1585e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8436ea000 4 client.8886 mds_command: new command op to 8643 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:31:17.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa8436ea000 1 -- 172.21.15.105:0/2982013521 --> [v2:172.21.15.105:6835/764864050,v1:172.21.15.105:6837/764864050] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b7bf0db50 con 0x558b7c0fe140 2023-11-28T18:31:17.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.284+0000 7fa83d928640 1 --2- 172.21.15.105:0/2982013521 >> [v2:172.21.15.105:6835/764864050,v1:172.21.15.105:6837/764864050] conn(0x558b7c0fe140 0x558b7c1585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.288+0000 7fa83d928640 1 --2- 172.21.15.105:0/2982013521 >> [v2:172.21.15.105:6835/764864050,v1:172.21.15.105:6837/764864050] conn(0x558b7c0fe140 0x558b7c1585e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=856dc9a3bfed482b server_cookie=44291f82bf042fbc in_seq=0 out_seq=0 2023-11-28T18:31:17.291 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.288+0000 7fa8197fa640 10 client.8886 ms_handle_connect on v2:172.21.15.105:6835/764864050 2023-11-28T18:31:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.288+0000 7fa8197fa640 1 -- 172.21.15.105:0/2982013521 <== mds.0 v2:172.21.15.105:6835/764864050 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558b7bf0db50 con 0x558b7c0fe140 2023-11-28T18:31:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.288+0000 7fa8197fa640 10 client.8886 handle_command_reply: tid=0 2023-11-28T18:31:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.288+0000 7fa8436ea000 1 client.8886 shutdown 2023-11-28T18:31:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.288+0000 7fa8436ea000 1 -- 172.21.15.105:0/2982013521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c12c2a0 msgr2=0x558b7c0fccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.288+0000 7fa8436ea000 1 --2- 172.21.15.105:0/2982013521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c12c2a0 0x558b7c0fccb0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fa8340ef920 tx=0x7fa83401c510 comp rx=0 tx=0).stop 2023-11-28T18:31:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.288+0000 7fa8436ea000 1 -- 172.21.15.105:0/2982013521 shutdown_connections 2023-11-28T18:31:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.288+0000 7fa8436ea000 1 --2- 172.21.15.105:0/2982013521 >> [v2:172.21.15.105:6835/764864050,v1:172.21.15.105:6837/764864050] conn(0x558b7c0fe140 0x558b7c1585e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.288+0000 7fa8436ea000 1 --2- 172.21.15.105:0/2982013521 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558b7c147c50 0x558b7c0fd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.288+0000 7fa8436ea000 1 --2- 172.21.15.105:0/2982013521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b7c147330 0x558b7c0fd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.288+0000 7fa8197fa640 0 client.8886 ms_handle_reset on v2:172.21.15.105:6835/764864050 2023-11-28T18:31:17.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.288+0000 7fa8436ea000 1 --2- 172.21.15.105:0/2982013521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c12c2a0 0x558b7c0fccb0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.288+0000 7fa8436ea000 1 -- 172.21.15.105:0/2982013521 >> 172.21.15.105:0/2982013521 conn(0x558b7c1238c0 msgr2=0x558b7c0f56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:17.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.288+0000 7fa8436ea000 1 -- 172.21.15.105:0/2982013521 shutdown_connections 2023-11-28T18:31:17.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.288+0000 7fa8436ea000 1 -- 172.21.15.105:0/2982013521 wait complete. 2023-11-28T18:31:17.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.288+0000 7fa8436ea000 20 client.8886 trim_cache size 0 max 16384 2023-11-28T18:31:17.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa8436ea000 1 Processor -- start 2023-11-28T18:31:17.304 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa8436ea000 1 -- start start 2023-11-28T18:31:17.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa8436ea000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c1238c0 0x558b7c0f7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa8436ea000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b7c12c2a0 0x558b7c0f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa8436ea000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558b7c147330 0x558b7c0f80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa8436ea000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7bc01940 con 0x558b7c1238c0 2023-11-28T18:31:17.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa8436ea000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7bf11540 con 0x558b7c12c2a0 2023-11-28T18:31:17.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa8436ea000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b7bfcf570 con 0x558b7c147330 2023-11-28T18:31:17.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa83e129640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c1238c0 0x558b7c0f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa83e129640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c1238c0 0x558b7c0f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55290/0 (socket says 172.21.15.105:55290) 2023-11-28T18:31:17.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa83e129640 1 -- 172.21.15.105:0/77123074 learned_addr learned my addr 172.21.15.105:0/77123074 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:17.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa83d928640 1 --2- 172.21.15.105:0/77123074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b7c12c2a0 0x558b7c0f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa83e92a640 1 --2- 172.21.15.105:0/77123074 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558b7c147330 0x558b7c0f80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa83e129640 1 -- 172.21.15.105:0/77123074 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558b7c147330 msgr2=0x558b7c0f80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:17.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa83e129640 1 --2- 172.21.15.105:0/77123074 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558b7c147330 0x558b7c0f80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa83e129640 1 -- 172.21.15.105:0/77123074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b7c12c2a0 msgr2=0x558b7c0f7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:17.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa83e129640 1 --2- 172.21.15.105:0/77123074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b7c12c2a0 0x558b7c0f7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa83e129640 1 -- 172.21.15.105:0/77123074 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b7beef490 con 0x558b7c1238c0 2023-11-28T18:31:17.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa83e129640 1 --2- 172.21.15.105:0/77123074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c1238c0 0x558b7c0f7640 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fa8340f4b90 tx=0x7fa834105040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3342fb0c1a29ef server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:17.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa82cff9640 1 -- 172.21.15.105:0/77123074 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8340f7be0 con 0x558b7c1238c0 2023-11-28T18:31:17.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa82cff9640 1 -- 172.21.15.105:0/77123074 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa834116e80 con 0x558b7c1238c0 2023-11-28T18:31:17.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa82cff9640 1 -- 172.21.15.105:0/77123074 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8341073d0 con 0x558b7c1238c0 2023-11-28T18:31:17.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa8436ea000 1 -- 172.21.15.105:0/77123074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c1238c0 msgr2=0x558b7c0f7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:17.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa8436ea000 1 --2- 172.21.15.105:0/77123074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c1238c0 0x558b7c0f7640 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fa8340f4b90 tx=0x7fa834105040 comp rx=0 tx=0).stop 2023-11-28T18:31:17.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa8436ea000 1 -- 172.21.15.105:0/77123074 shutdown_connections 2023-11-28T18:31:17.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa8436ea000 1 --2- 172.21.15.105:0/77123074 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558b7c147330 0x558b7c0f80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa8436ea000 1 --2- 172.21.15.105:0/77123074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b7c12c2a0 0x558b7c0f7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa8436ea000 1 --2- 172.21.15.105:0/77123074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c1238c0 0x558b7c0f7640 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.304+0000 7fa8436ea000 1 -- 172.21.15.105:0/77123074 >> 172.21.15.105:0/77123074 conn(0x558b7c15cbc0 msgr2=0x558b7c121390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:17.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8436ea000 1 -- 172.21.15.105:0/77123074 shutdown_connections 2023-11-28T18:31:17.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8436ea000 1 -- 172.21.15.105:0/77123074 wait complete. 2023-11-28T18:31:17.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8436ea000 1 -- start start 2023-11-28T18:31:17.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8436ea000 1 Processor -- start 2023-11-28T18:31:17.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8436ea000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b7c1238c0 0x558b7c1abab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8436ea000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c12c2a0 0x558b7c1abff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8436ea000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558b7c147330 0x558b7c1284c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8436ea000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7bf45fd0 con 0x558b7c12c2a0 2023-11-28T18:31:17.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8436ea000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7bfbf110 con 0x558b7c1238c0 2023-11-28T18:31:17.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8436ea000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b7bf448b0 con 0x558b7c147330 2023-11-28T18:31:17.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8436ea000 10 client.0 _ll_register_callbacks cb 0x558b7bfbf060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:31:17.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa83d928640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c12c2a0 0x558b7c1abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa83d928640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c12c2a0 0x558b7c1abff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55304/0 (socket says 172.21.15.105:55304) 2023-11-28T18:31:17.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa83d928640 1 -- 172.21.15.105:0/3532201413 learned_addr learned my addr 172.21.15.105:0/3532201413 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:17.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa83e92a640 1 --2- 172.21.15.105:0/3532201413 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558b7c147330 0x558b7c1284c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa83e129640 1 --2- 172.21.15.105:0/3532201413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b7c1238c0 0x558b7c1abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.311 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa83d928640 1 -- 172.21.15.105:0/3532201413 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558b7c147330 msgr2=0x558b7c1284c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:17.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa83d928640 1 --2- 172.21.15.105:0/3532201413 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558b7c147330 0x558b7c1284c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa83d928640 1 -- 172.21.15.105:0/3532201413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b7c1238c0 msgr2=0x558b7c1abab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:17.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa83d928640 1 --2- 172.21.15.105:0/3532201413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b7c1238c0 0x558b7c1abab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa83d928640 1 -- 172.21.15.105:0/3532201413 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b7bf09190 con 0x558b7c12c2a0 2023-11-28T18:31:17.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa83d928640 1 --2- 172.21.15.105:0/3532201413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c12c2a0 0x558b7c1abff0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fa8280096c0 tx=0x7fa828004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a770784474108a3e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:17.312 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8197fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:31:17.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8197fa640 1 -- 172.21.15.105:0/3532201413 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82801c040 con 0x558b7c12c2a0 2023-11-28T18:31:17.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8197fa640 1 -- 172.21.15.105:0/3532201413 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa82802ab00 con 0x558b7c12c2a0 2023-11-28T18:31:17.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8436ea000 1 -- 172.21.15.105:0/3532201413 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b7bf75650 con 0x558b7c12c2a0 2023-11-28T18:31:17.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8197fa640 1 -- 172.21.15.105:0/3532201413 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa82802acc0 con 0x558b7c12c2a0 2023-11-28T18:31:17.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8436ea000 10 client.8892 fetch_fsmap learned FSMap version 124 2023-11-28T18:31:17.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8436ea000 1 -- 172.21.15.105:0/3532201413 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x558b7bc40cc0 con 0x558b7c12c2a0 2023-11-28T18:31:17.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8197fa640 1 -- 172.21.15.105:0/3532201413 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558b7bf75650 con 0x558b7c12c2a0 2023-11-28T18:31:17.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8197fa640 1 -- 172.21.15.105:0/3532201413 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 124) v1 ==== 3318+0+0 (secure 0 0 0) 0x7fa828030020 con 0x558b7c12c2a0 2023-11-28T18:31:17.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8436ea000 10 client.8892 fetch_fsmap finished waiting for FSMap version 124 2023-11-28T18:31:17.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8436ea000 10 client.8892 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T18:31:17.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8436ea000 20 client.8892 populate_metadata read hostname 'smithi105' 2023-11-28T18:31:17.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8436ea000 1 --2- 172.21.15.105:0/3532201413 >> [v2:172.21.15.105:6835/764864050,v1:172.21.15.105:6837/764864050] conn(0x558b7c147c50 0x558b7c1b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:17.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8436ea000 4 client.8892 mds_command: new command op to 8643 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:31:17.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.308+0000 7fa8436ea000 1 -- 172.21.15.105:0/3532201413 --> [v2:172.21.15.105:6835/764864050,v1:172.21.15.105:6837/764864050] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558b7bf417a0 con 0x558b7c147c50 2023-11-28T18:31:17.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa83e129640 1 --2- 172.21.15.105:0/3532201413 >> [v2:172.21.15.105:6835/764864050,v1:172.21.15.105:6837/764864050] conn(0x558b7c147c50 0x558b7c1b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:17.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa8197fa640 10 client.8892 ms_handle_connect on v2:172.21.15.105:6835/764864050 2023-11-28T18:31:17.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa83e129640 1 --2- 172.21.15.105:0/3532201413 >> [v2:172.21.15.105:6835/764864050,v1:172.21.15.105:6837/764864050] conn(0x558b7c147c50 0x558b7c1b4920 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dfad63746221c03 server_cookie=9340e3c545524a2e in_seq=0 out_seq=0 2023-11-28T18:31:17.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa8197fa640 1 -- 172.21.15.105:0/3532201413 <== mds.0 v2:172.21.15.105:6835/764864050 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x558b7bf417a0 con 0x558b7c147c50 2023-11-28T18:31:17.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa8197fa640 10 client.8892 handle_command_reply: tid=0 2023-11-28T18:31:17.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa8436ea000 1 client.8892 shutdown 2023-11-28T18:31:17.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa8436ea000 1 -- 172.21.15.105:0/3532201413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c12c2a0 msgr2=0x558b7c1abff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:17.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa8436ea000 1 --2- 172.21.15.105:0/3532201413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c12c2a0 0x558b7c1abff0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fa8280096c0 tx=0x7fa828004740 comp rx=0 tx=0).stop 2023-11-28T18:31:17.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa8436ea000 1 -- 172.21.15.105:0/3532201413 shutdown_connections 2023-11-28T18:31:17.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa8436ea000 1 --2- 172.21.15.105:0/3532201413 >> [v2:172.21.15.105:6835/764864050,v1:172.21.15.105:6837/764864050] conn(0x558b7c147c50 0x558b7c1b4920 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa8436ea000 1 --2- 172.21.15.105:0/3532201413 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558b7c147330 0x558b7c1284c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa8436ea000 1 --2- 172.21.15.105:0/3532201413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b7c12c2a0 0x558b7c1abff0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa8197fa640 0 client.8892 ms_handle_reset on v2:172.21.15.105:6835/764864050 2023-11-28T18:31:17.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa8436ea000 1 --2- 172.21.15.105:0/3532201413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b7c1238c0 0x558b7c1abab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa8436ea000 1 -- 172.21.15.105:0/3532201413 >> 172.21.15.105:0/3532201413 conn(0x558b7c15cbc0 msgr2=0x558b7c122320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:17.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa8436ea000 1 -- 172.21.15.105:0/3532201413 shutdown_connections 2023-11-28T18:31:17.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa8436ea000 1 -- 172.21.15.105:0/3532201413 wait complete. 2023-11-28T18:31:17.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa8436ea000 20 client.8892 trim_cache size 0 max 16384 2023-11-28T18:31:17.318 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:31:17.318 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:31:17.318 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2023-11-28T18:31:17.318 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 8643, 2023-11-28T18:31:17.318 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:31:17.318 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:31:17.318 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:31:17.319 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 13.63398031, 2023-11-28T18:31:17.319 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 124, 2023-11-28T18:31:17.319 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 117, 2023-11-28T18:31:17.319 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 117, 2023-11-28T18:31:17.319 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 19.524886145 2023-11-28T18:31:17.319 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:31:17.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa84038d640 1 -- 172.21.15.105:0/2122908129 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa81c0668d0 msgr2=0x7fa81c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:17.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa84038d640 1 --2- 172.21.15.105:0/2122908129 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa81c0668d0 0x7fa81c068d90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa8380cc420 tx=0x7fa828009190 comp rx=0 tx=0).stop 2023-11-28T18:31:17.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa84038d640 1 -- 172.21.15.105:0/2122908129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8380b4290 msgr2=0x7fa83814b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:17.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa84038d640 1 --2- 172.21.15.105:0/2122908129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8380b4290 0x7fa83814b1f0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fa8340ac040 tx=0x7fa83401ad50 comp rx=0 tx=0).stop 2023-11-28T18:31:17.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa84038d640 1 -- 172.21.15.105:0/2122908129 shutdown_connections 2023-11-28T18:31:17.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa84038d640 1 --2- 172.21.15.105:0/2122908129 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa83814bb70 0x7fa838159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa84038d640 1 --2- 172.21.15.105:0/2122908129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa83815b030 0x7fa838153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa84038d640 1 --2- 172.21.15.105:0/2122908129 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa81c0668d0 0x7fa81c068d90 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa84038d640 1 --2- 172.21.15.105:0/2122908129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8380b4290 0x7fa83814b1f0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:17.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa84038d640 1 -- 172.21.15.105:0/2122908129 >> 172.21.15.105:0/2122908129 conn(0x7fa8380ceb90 msgr2=0x7fa8380b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:17.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa84038d640 1 -- 172.21.15.105:0/2122908129 shutdown_connections 2023-11-28T18:31:17.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:17.312+0000 7fa84038d640 1 -- 172.21.15.105:0/2122908129 wait complete. 2023-11-28T18:31:17.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:31:17.331 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-28T18:31:17.332 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:31:17.332 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p tree 2023-11-28T18:31:17.332 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:31:17.332 DEBUG:teuthology.orchestra.run.smithi105:> setfattr -n ceph.dir.pin.distributed -v 1 tree 2023-11-28T18:31:17.332 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:31:17.332 DEBUG:teuthology.orchestra.run.smithi105:> for ((i = 0; i < 100; i++)); do 2023-11-28T18:31:17.332 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p "tree/$i" 2023-11-28T18:31:17.332 DEBUG:teuthology.orchestra.run.smithi105:> echo file > "tree/$i/file" 2023-11-28T18:31:17.332 DEBUG:teuthology.orchestra.run.smithi105:> done 2023-11-28T18:31:17.332 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:31:17.348 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree 2023-11-28T18:31:17.356 INFO:teuthology.orchestra.run.smithi105.stderr:+ setfattr -n ceph.dir.pin.distributed -v 1 tree 2023-11-28T18:31:17.360 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i = 0 )) 2023-11-28T18:31:17.360 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.361 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/0 2023-11-28T18:31:17.365 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.371 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.371 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.371 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/1 2023-11-28T18:31:17.380 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.382 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.382 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.383 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/2 2023-11-28T18:31:17.389 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.391 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.391 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.391 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/3 2023-11-28T18:31:17.397 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.399 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.399 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.400 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/4 2023-11-28T18:31:17.406 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.408 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.408 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.408 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/5 2023-11-28T18:31:17.414 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.416 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.417 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.417 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/6 2023-11-28T18:31:17.426 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.426 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.426 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.426 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/7 2023-11-28T18:31:17.431 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.436 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.437 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.437 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/8 2023-11-28T18:31:17.439 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.442 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.442 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.443 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/9 2023-11-28T18:31:17.448 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.450 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.450 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.450 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/10 2023-11-28T18:31:17.456 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.459 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.459 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.459 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/11 2023-11-28T18:31:17.466 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.467 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.467 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.467 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/12 2023-11-28T18:31:17.473 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.475 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.475 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.475 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/13 2023-11-28T18:31:17.481 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.484 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.484 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.484 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/14 2023-11-28T18:31:17.490 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.492 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.492 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.492 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/15 2023-11-28T18:31:17.498 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.501 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.501 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.501 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/16 2023-11-28T18:31:17.507 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.509 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.509 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.509 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/17 2023-11-28T18:31:17.515 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.517 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.517 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.517 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/18 2023-11-28T18:31:17.523 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.526 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.526 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.526 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/19 2023-11-28T18:31:17.532 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.534 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.534 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.534 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/20 2023-11-28T18:31:17.540 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.542 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.542 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.542 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/21 2023-11-28T18:31:17.548 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.550 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.550 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.550 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/22 2023-11-28T18:31:17.556 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.559 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.559 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.559 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/23 2023-11-28T18:31:17.565 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.567 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.567 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.567 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/24 2023-11-28T18:31:17.572 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.574 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.575 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.575 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/25 2023-11-28T18:31:17.580 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.583 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.583 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.583 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/26 2023-11-28T18:31:17.588 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.591 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.591 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.591 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/27 2023-11-28T18:31:17.596 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.599 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.599 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.599 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/28 2023-11-28T18:31:17.604 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.607 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.607 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.607 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/29 2023-11-28T18:31:17.612 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.615 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.615 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.615 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/30 2023-11-28T18:31:17.621 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.623 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.623 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.623 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/31 2023-11-28T18:31:17.630 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.632 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.632 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.632 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/32 2023-11-28T18:31:17.638 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.640 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.641 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.641 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/33 2023-11-28T18:31:17.647 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.649 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.649 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.649 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/34 2023-11-28T18:31:17.654 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.657 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.657 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.657 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/35 2023-11-28T18:31:17.663 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.665 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.665 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.666 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/36 2023-11-28T18:31:17.671 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.673 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.674 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.674 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/37 2023-11-28T18:31:17.682 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.683 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.683 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.683 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/38 2023-11-28T18:31:17.688 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.690 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.690 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.690 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/39 2023-11-28T18:31:17.696 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.700 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.700 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.700 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/40 2023-11-28T18:31:17.706 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.708 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.708 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.709 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/41 2023-11-28T18:31:17.715 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.717 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.717 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.717 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/42 2023-11-28T18:31:17.723 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.725 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.725 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.726 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/43 2023-11-28T18:31:17.731 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.733 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.734 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.734 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/44 2023-11-28T18:31:17.740 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.742 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.742 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.742 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/45 2023-11-28T18:31:17.748 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.750 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.750 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.751 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/46 2023-11-28T18:31:17.756 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.758 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.758 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.759 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/47 2023-11-28T18:31:17.764 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.766 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.766 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.766 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/48 2023-11-28T18:31:17.772 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.774 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.775 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.775 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/49 2023-11-28T18:31:17.781 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.783 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.783 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.783 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/50 2023-11-28T18:31:17.789 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.793 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.793 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.793 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/51 2023-11-28T18:31:17.804 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.806 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.806 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.806 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/52 2023-11-28T18:31:17.813 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.816 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.816 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.816 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/53 2023-11-28T18:31:17.821 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.824 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.824 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.824 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/54 2023-11-28T18:31:17.829 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.832 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.832 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.832 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/55 2023-11-28T18:31:17.837 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.840 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.840 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.840 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/56 2023-11-28T18:31:17.846 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.848 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.848 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.848 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/57 2023-11-28T18:31:17.854 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.856 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.856 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.856 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/58 2023-11-28T18:31:17.862 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.865 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.865 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.865 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/59 2023-11-28T18:31:17.870 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.873 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.873 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.873 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/60 2023-11-28T18:31:17.878 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.880 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.881 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.881 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/61 2023-11-28T18:31:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/62 2023-11-28T18:31:17.895 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.897 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.897 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.897 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/63 2023-11-28T18:31:17.903 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.905 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.905 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.905 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/64 2023-11-28T18:31:17.911 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.913 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.914 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.914 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/65 2023-11-28T18:31:17.919 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.921 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.921 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.921 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/66 2023-11-28T18:31:17.927 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.930 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.930 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.930 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/67 2023-11-28T18:31:17.935 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.938 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.938 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.938 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/68 2023-11-28T18:31:17.943 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.946 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.946 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.946 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/69 2023-11-28T18:31:17.952 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.954 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.954 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.954 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/70 2023-11-28T18:31:17.959 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.962 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.962 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.963 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/71 2023-11-28T18:31:17.967 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.969 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.970 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.970 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/72 2023-11-28T18:31:17.976 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.978 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.978 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.978 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/73 2023-11-28T18:31:17.984 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.986 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.986 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.986 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/74 2023-11-28T18:31:17.991 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:17.994 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:17.994 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:17.994 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/75 2023-11-28T18:31:18.000 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.002 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.002 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.002 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/76 2023-11-28T18:31:18.008 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.010 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.010 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.010 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/77 2023-11-28T18:31:18.015 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.018 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.018 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.018 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/78 2023-11-28T18:31:18.024 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.026 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.026 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.026 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/79 2023-11-28T18:31:18.032 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.034 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.034 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.034 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/80 2023-11-28T18:31:18.040 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.042 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.042 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.042 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/81 2023-11-28T18:31:18.048 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.050 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.051 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.051 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/82 2023-11-28T18:31:18.056 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.059 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.059 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.059 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/83 2023-11-28T18:31:18.065 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.068 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.068 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.068 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/84 2023-11-28T18:31:18.074 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.077 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.077 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.077 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/85 2023-11-28T18:31:18.084 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.087 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.087 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.087 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/86 2023-11-28T18:31:18.093 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.096 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.096 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.096 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/87 2023-11-28T18:31:18.102 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.105 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.105 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.106 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/88 2023-11-28T18:31:18.111 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.114 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.114 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.114 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/89 2023-11-28T18:31:18.120 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.123 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.123 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.123 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/90 2023-11-28T18:31:18.130 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.132 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.132 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.132 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/91 2023-11-28T18:31:18.139 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.141 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.142 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.142 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/92 2023-11-28T18:31:18.148 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.151 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.151 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.151 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/93 2023-11-28T18:31:18.157 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.159 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.159 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.160 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/94 2023-11-28T18:31:18.166 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.169 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.169 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.169 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/95 2023-11-28T18:31:18.174 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.176 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.176 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.176 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/96 2023-11-28T18:31:18.182 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.185 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.185 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.185 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/97 2023-11-28T18:31:18.191 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.193 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.193 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.193 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/98 2023-11-28T18:31:18.199 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.202 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.202 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.202 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/99 2023-11-28T18:31:18.207 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:31:18.210 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:31:18.210 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 100 )) 2023-11-28T18:31:18.211 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2023-11-28T18:31:18.367 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-28T18:31:18.368 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:31:18.368 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-11-28T18:31:18.522 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-28T18:31:18.523 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:31:18.523 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-11-28T18:31:18.659 INFO:tasks.ceph.mds.a.smithi105.stderr:2023-11-28T18:31:18.656+0000 7f4d2e9cd640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-28T18:31:18.659 INFO:tasks.ceph.mds.a.smithi105.stderr:2023-11-28T18:31:18.656+0000 7f4d2e9cd640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-11-28T18:31:18.681 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"956","version":"954","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"251","version":"251","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2023-11-28T18:31:18.681 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "956", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "954" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "251", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "251" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:31:18.681 INFO:tasks.cephfs.cephfs_test_case:len=0 [] 2023-11-28T18:31:23.683 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2023-11-28T18:31:23.849 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"960","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":2},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000271.101*","snapid_first":2,"projected_version":"250","version":"232","committing_version":"205","committed_version":"205","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":9,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000271.011*","snapid_first":2,"projected_version":"258","version":"240","committing_version":"205","committed_version":"205","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":9,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}}] 2023-11-28T18:31:23.850 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "snapid_first": 2, "states": [], "version": "960" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 9, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "205", "committing_version": "205", "dir_auth": "2", "dirfrag": "0x10000000271.101*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/tree", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "250", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "232" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 9, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "205", "committing_version": "205", "dir_auth": "2", "dirfrag": "0x10000000271.011*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/tree", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "258", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "240" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:31:23.851 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-11-28T18:31:24.016 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"960","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":3},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000271.110*","snapid_first":2,"projected_version":"274","version":"252","committing_version":"205","committed_version":"205","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":11,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000271.001*","snapid_first":2,"projected_version":"268","version":"246","committing_version":"205","committed_version":"205","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":11,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000271.000*","snapid_first":2,"projected_version":"250","version":"240","committing_version":"205","committed_version":"205","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":5,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-28T18:31:24.017 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "snapid_first": 2, "states": [], "version": "960" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 11, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "205", "committing_version": "205", "dir_auth": "1", "dirfrag": "0x10000000271.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/tree", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "274", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "252" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 11, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "205", "committing_version": "205", "dir_auth": "1", "dirfrag": "0x10000000271.001*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/tree", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "268", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "246" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 5, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "205", "committing_version": "205", "dir_auth": "1", "dirfrag": "0x10000000271.000*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/tree", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "250", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "240" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:31:24.017 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-11-28T18:31:24.172 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"962","version":"960","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":3,"2":2}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"253","version":"253","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000271.011*","snapid_first":2,"projected_version":"240","version":"240","committing_version":"205","committed_version":"205","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000271.010*","snapid_first":2,"projected_version":"258","version":"236","committing_version":"205","committed_version":"205","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":11,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000271.101*","snapid_first":2,"projected_version":"232","version":"232","committing_version":"205","committed_version":"205","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000271.111*","snapid_first":2,"projected_version":"228","version":"218","committing_version":"205","committed_version":"205","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":5,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":1},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000271.000*","snapid_first":2,"projected_version":"240","version":"240","committing_version":"205","committed_version":"205","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000271.100*","snapid_first":2,"projected_version":"258","version":"242","committing_version":"205","committed_version":"205","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":8,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":1},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000271.001*","snapid_first":2,"projected_version":"246","version":"246","committing_version":"205","committed_version":"205","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000271.110*","snapid_first":2,"projected_version":"252","version":"252","committing_version":"205","committed_version":"205","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2023-11-28T18:31:24.173 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": { "1": 3, "2": 2 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 5, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "962", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "960" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "253", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "253" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "205", "committing_version": "205", "dir_auth": "2", "dirfrag": "0x10000000271.011*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "240", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "240" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 11, "auth_state": { "replicas": {} }, "committed_version": "205", "committing_version": "205", "dir_auth": "0", "dirfrag": "0x10000000271.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "258", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "236" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "205", "committing_version": "205", "dir_auth": "2", "dirfrag": "0x10000000271.101*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "232", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "232" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 5, "auth_state": { "replicas": {} }, "committed_version": "205", "committing_version": "205", "dir_auth": "0", "dirfrag": "0x10000000271.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "228", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "218" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "205", "committing_version": "205", "dir_auth": "1", "dirfrag": "0x10000000271.000*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "240", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "240" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 8, "auth_state": { "replicas": {} }, "committed_version": "205", "committing_version": "205", "dir_auth": "0", "dirfrag": "0x10000000271.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "258", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "242" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "205", "committing_version": "205", "dir_auth": "1", "dirfrag": "0x10000000271.001*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "246", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "246" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "205", "committing_version": "205", "dir_auth": "1", "dirfrag": "0x10000000271.110*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "252", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "252" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2023-11-28T18:31:24.174 INFO:tasks.cephfs.cephfs_test_case:len=8 [{'is_auth': True, 'auth_first': 2, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 2, 'dir': {'path': '/tree', 'dirfrag': '0x10000000271.101*', 'snapid_first': 2, 'projected_version': '250', 'version': '232', 'committing_version': '205', 'committed_version': '205', 'is_rep': False, 'dir_auth': '2', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {'0': 1}}, 'replica_state': {'authority': [2, -2], 'replica_nonce': 0}, 'auth_pins': 9, 'is_frozen': False, 'is_freezing': False, 'pins': {'ptrwaiter': 0, 'child': 1, 'frozen': 0, 'subtree': 1, 'importing': 0, 'replicated': 1, 'dirty': 1, 'authpin': 1}, 'nref': 5}}, {'is_auth': True, 'auth_first': 2, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 2, 'dir': {'path': '/tree', 'dirfrag': '0x10000000271.011*', 'snapid_first': 2, 'projected_version': '258', 'version': '240', 'committing_version': '205', 'committed_version': '205', 'is_rep': False, 'dir_auth': '2', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {'0': 1}}, 'replica_state': {'authority': [2, -2], 'replica_nonce': 0}, 'auth_pins': 9, 'is_frozen': False, 'is_freezing': False, 'pins': {'ptrwaiter': 0, 'child': 1, 'frozen': 0, 'subtree': 1, 'importing': 0, 'replicated': 1, 'dirty': 1, 'authpin': 1}, 'nref': 5}}, {'is_auth': True, 'auth_first': 1, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 1, 'dir': {'path': '/tree', 'dirfrag': '0x10000000271.110*', 'snapid_first': 2, 'projected_version': '274', 'version': '252', 'committing_version': '205', 'committed_version': '205', 'is_rep': False, 'dir_auth': '1', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {'0': 1}}, 'replica_state': {'authority': [1, -2], 'replica_nonce': 0}, 'auth_pins': 11, 'is_frozen': False, 'is_freezing': False, 'pins': {'ptrwaiter': 0, 'child': 1, 'frozen': 0, 'subtree': 1, 'importing': 0, 'replicated': 1, 'dirty': 1, 'authpin': 1}, 'nref': 5}}, {'is_auth': True, 'auth_first': 1, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 1, 'dir': {'path': '/tree', 'dirfrag': '0x10000000271.001*', 'snapid_first': 2, 'projected_version': '268', 'version': '246', 'committing_version': '205', 'committed_version': '205', 'is_rep': False, 'dir_auth': '1', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {'0': 1}}, 'replica_state': {'authority': [1, -2], 'replica_nonce': 0}, 'auth_pins': 11, 'is_frozen': False, 'is_freezing': False, 'pins': {'ptrwaiter': 0, 'child': 1, 'frozen': 0, 'subtree': 1, 'importing': 0, 'replicated': 1, 'dirty': 1, 'authpin': 1}, 'nref': 5}}, {'is_auth': True, 'auth_first': 1, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 1, 'dir': {'path': '/tree', 'dirfrag': '0x10000000271.000*', 'snapid_first': 2, 'projected_version': '250', 'version': '240', 'committing_version': '205', 'committed_version': '205', 'is_rep': False, 'dir_auth': '1', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {'0': 1}}, 'replica_state': {'authority': [1, -2], 'replica_nonce': 0}, 'auth_pins': 5, 'is_frozen': False, 'is_freezing': False, 'pins': {'ptrwaiter': 0, 'child': 1, 'frozen': 0, 'subtree': 1, 'importing': 0, 'replicated': 1, 'dirty': 1, 'authpin': 1}, 'nref': 5}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x10000000271.010*', 'snapid_first': 2, 'projected_version': '258', 'version': '236', 'committing_version': '205', 'committed_version': '205', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 11, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'waiter': 0, 'authpin': 1}, 'nref': 4}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x10000000271.111*', 'snapid_first': 2, 'projected_version': '228', 'version': '218', 'committing_version': '205', 'committed_version': '205', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 5, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'authpin': 1}, 'nref': 4}}, {'is_auth': True, 'auth_first': 0, 'auth_second': -2, 'export_pin': -1, 'distributed_ephemeral_pin': True, 'random_ephemeral_pin': False, 'export_pin_target': 0, 'dir': {'path': '/tree', 'dirfrag': '0x10000000271.100*', 'snapid_first': 2, 'projected_version': '258', 'version': '242', 'committing_version': '205', 'committed_version': '205', 'is_rep': False, 'dir_auth': '0', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 8, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'frozen': 0, 'subtree': 1, 'dirty': 1, 'waiter': 0, 'authpin': 1}, 'nref': 4}}] 2023-11-28T18:31:24.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:31:24.174 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 tree) 2023-11-28T18:31:32.736 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:31:32.740 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:31:39.189 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-11-28T18:31:39.356 INFO:teuthology.orchestra.run.smithi105.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"967","version":"967","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":3,"2":2}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"253","version":"253","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000271.011*","snapid_first":2,"projected_version":"259","version":"259","committing_version":"205","committed_version":"205","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"waiter":0,"authpin":0,"tempexporting":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000271.010*","snapid_first":2,"projected_version":"259","version":"259","committing_version":"205","committed_version":"205","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000271.101*","snapid_first":2,"projected_version":"251","version":"251","committing_version":"205","committed_version":"205","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"waiter":0,"authpin":0,"tempexporting":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000271.111*","snapid_first":2,"projected_version":"229","version":"229","committing_version":"205","committed_version":"205","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000271.100*","snapid_first":2,"projected_version":"259","version":"259","committing_version":"205","committed_version":"205","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":0},"nref":3}}] 2023-11-28T18:31:39.357 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 3, "2": 2 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "967", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "967" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "253", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "253" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "2": 1 } }, "committed_version": "205", "committing_version": "205", "dir_auth": "0", "dirfrag": "0x10000000271.011*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "259", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "259" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "205", "committing_version": "205", "dir_auth": "0", "dirfrag": "0x10000000271.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "259", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "259" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "2": 1 } }, "committed_version": "205", "committing_version": "205", "dir_auth": "0", "dirfrag": "0x10000000271.101*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "251", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "251" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "205", "committing_version": "205", "dir_auth": "0", "dirfrag": "0x10000000271.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1 }, "projected_version": "229", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "229" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "205", "committing_version": "205", "dir_auth": "0", "dirfrag": "0x10000000271.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "259", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "259" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-28T18:31:39.357 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T18:31:39.357 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:31:39.461 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:31:39.538 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T18:31:39.538 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:31:39.539 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:31:39.548 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:31:39.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.724+0000 7fd2d8c32640 1 Processor -- start 2023-11-28T18:31:39.727 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.728+0000 7fd2d8c32640 1 -- start start 2023-11-28T18:31:39.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.728+0000 7fd2d8c32640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd2d4159f40 0x7fd2d40caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:39.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.728+0000 7fd2d8c32640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd2d40cc760 0x7fd2d40ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:39.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.728+0000 7fd2d8c32640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d406bf50 con 0x7fd2d40ad680 2023-11-28T18:31:39.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.728+0000 7fd2d8c32640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d40ccbc0 con 0x7fd2d40cc760 2023-11-28T18:31:39.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.728+0000 7fd2d8c32640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d415a320 con 0x7fd2d4159f40 2023-11-28T18:31:39.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.728+0000 7fd2d2d9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd2d40ad680 0x7fd2d415da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35136/0 (socket says 172.21.15.105:35136) 2023-11-28T18:31:39.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.728+0000 7fd2d2d9d640 1 -- 172.21.15.105:0/2522056861 learned_addr learned my addr 172.21.15.105:0/2522056861 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:39.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.728+0000 7fd2cbfff640 1 --2- 172.21.15.105:0/2522056861 >> v2:172.21.15.105:3300/0 conn(0x7fd2d40cc760 0x7fd2d40ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:39.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.728+0000 7fd2d259c640 1 --2- 172.21.15.105:0/2522056861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd2d4159f40 0x7fd2d40caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:39.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.728+0000 7fd2d1d9b640 1 -- 172.21.15.105:0/2522056861 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085092568 0 0) 0x7fd2d406bf50 con 0x7fd2d40ad680 2023-11-28T18:31:39.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.728+0000 7fd2d1d9b640 1 -- 172.21.15.105:0/2522056861 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2c0003530 con 0x7fd2d40ad680 2023-11-28T18:31:39.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.728+0000 7fd2cbfff640 1 -- 172.21.15.105:0/2522056861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd2d4159f40 msgr2=0x7fd2d40caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:39.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.728+0000 7fd2cbfff640 1 --2- 172.21.15.105:0/2522056861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd2d4159f40 0x7fd2d40caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:39.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.728+0000 7fd2cbfff640 1 -- 172.21.15.105:0/2522056861 >> v1:172.21.15.105:6789/0 conn(0x7fd2d40ad680 legacy=0x7fd2d415da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:39.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.728+0000 7fd2cbfff640 1 -- 172.21.15.105:0/2522056861 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d4071380 con 0x7fd2d40cc760 2023-11-28T18:31:39.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.732+0000 7fd2cbfff640 1 --2- 172.21.15.105:0/2522056861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2d40cc760 0x7fd2d40ad140 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0097c0 tx=0x7fd2bc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fcdcbd180cbd02b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:39.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.732+0000 7fd2d1d9b640 1 -- 172.21.15.105:0/2522056861 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0ac620 con 0x7fd2d40cc760 2023-11-28T18:31:39.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.732+0000 7fd2d1d9b640 1 -- 172.21.15.105:0/2522056861 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2bc0ac780 con 0x7fd2d40cc760 2023-11-28T18:31:39.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.732+0000 7fd2d1d9b640 1 -- 172.21.15.105:0/2522056861 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0acb10 con 0x7fd2d40cc760 2023-11-28T18:31:39.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.732+0000 7fd2d8c32640 1 -- 172.21.15.105:0/2522056861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2d40cc760 msgr2=0x7fd2d40ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:39.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.732+0000 7fd2d8c32640 1 --2- 172.21.15.105:0/2522056861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2d40cc760 0x7fd2d40ad140 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0097c0 tx=0x7fd2bc01c3c0 comp rx=0 tx=0).stop 2023-11-28T18:31:39.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.732+0000 7fd2d8c32640 1 -- 172.21.15.105:0/2522056861 shutdown_connections 2023-11-28T18:31:39.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.732+0000 7fd2d8c32640 1 --2- 172.21.15.105:0/2522056861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2d40cc760 0x7fd2d40ad140 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:39.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.732+0000 7fd2d8c32640 1 --2- 172.21.15.105:0/2522056861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd2d4159f40 0x7fd2d40caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:39.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.732+0000 7fd2d8c32640 1 -- 172.21.15.105:0/2522056861 >> 172.21.15.105:0/2522056861 conn(0x7fd2d40ceb90 msgr2=0x7fd2d40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:39.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.732+0000 7fd2d8c32640 1 -- 172.21.15.105:0/2522056861 shutdown_connections 2023-11-28T18:31:39.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.732+0000 7fd2d8c32640 1 -- 172.21.15.105:0/2522056861 wait complete. 2023-11-28T18:31:39.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.732+0000 7fd2d8c32640 1 Processor -- start 2023-11-28T18:31:39.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2d8c32640 1 -- start start 2023-11-28T18:31:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2d8c32640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2d40ad680 0x7fd2d41395d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2d8c32640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2d40cc760 0x7fd2d414ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2d8c32640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd2d4159f40 0x7fd2d415ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2d8c32640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d414bbe0 con 0x7fd2d40cc760 2023-11-28T18:31:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2d8c32640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d4139b10 con 0x7fd2d40ad680 2023-11-28T18:31:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2d8c32640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d414dd80 con 0x7fd2d4159f40 2023-11-28T18:31:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2cbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2d40cc760 0x7fd2d414ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2cbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2d40cc760 0x7fd2d414ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36442/0 (socket says 172.21.15.105:36442) 2023-11-28T18:31:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2cbfff640 1 -- 172.21.15.105:0/3685123383 learned_addr learned my addr 172.21.15.105:0/3685123383 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2d2d9d640 1 --2- 172.21.15.105:0/3685123383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd2d4159f40 0x7fd2d415ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2d259c640 1 --2- 172.21.15.105:0/3685123383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2d40ad680 0x7fd2d41395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2cbfff640 1 -- 172.21.15.105:0/3685123383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd2d4159f40 msgr2=0x7fd2d415ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2cbfff640 1 --2- 172.21.15.105:0/3685123383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd2d4159f40 0x7fd2d415ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2cbfff640 1 -- 172.21.15.105:0/3685123383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2d40ad680 msgr2=0x7fd2d41395d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2cbfff640 1 --2- 172.21.15.105:0/3685123383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2d40ad680 0x7fd2d41395d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2cbfff640 1 -- 172.21.15.105:0/3685123383 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2bc009420 con 0x7fd2d40cc760 2023-11-28T18:31:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2cbfff640 1 --2- 172.21.15.105:0/3685123383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2d40cc760 0x7fd2d414ac40 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0af040 tx=0x7fd2bc019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2caffd640 1 -- 172.21.15.105:0/3685123383 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0b5410 con 0x7fd2d40cc760 2023-11-28T18:31:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2caffd640 1 -- 172.21.15.105:0/3685123383 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2bc0b5590 con 0x7fd2d40cc760 2023-11-28T18:31:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2d8c32640 1 -- 172.21.15.105:0/3685123383 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d414b620 con 0x7fd2d40cc760 2023-11-28T18:31:39.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2caffd640 1 -- 172.21.15.105:0/3685123383 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0b5940 con 0x7fd2d40cc760 2023-11-28T18:31:39.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2d8c32640 1 -- 172.21.15.105:0/3685123383 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d414da10 con 0x7fd2d40cc760 2023-11-28T18:31:39.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.736+0000 7fd2d8c32640 1 -- 172.21.15.105:0/3685123383 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d414d870 con 0x7fd2d40cc760 2023-11-28T18:31:39.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.740+0000 7fd2caffd640 1 -- 172.21.15.105:0/3685123383 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd2bc0b3040 con 0x7fd2d40cc760 2023-11-28T18:31:39.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.740+0000 7fd2caffd640 1 --2- 172.21.15.105:0/3685123383 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd2b4066900 0x7fd2b4068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:39.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.740+0000 7fd2caffd640 1 -- 172.21.15.105:0/3685123383 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd2bc11e710 con 0x7fd2d40cc760 2023-11-28T18:31:39.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.740+0000 7fd2d259c640 1 --2- 172.21.15.105:0/3685123383 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd2b4066900 0x7fd2b4068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:39.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.740+0000 7fd2d259c640 1 --2- 172.21.15.105:0/3685123383 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd2b4066900 0x7fd2b4068dc0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400b930 tx=0x7fd2c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:39.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.744+0000 7fd2caffd640 1 -- 172.21.15.105:0/3685123383 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd2d414d870 con 0x7fd2d40cc760 2023-11-28T18:31:39.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.892+0000 7fd2d8c32640 1 -- 172.21.15.105:0/3685123383 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2d414d870 con 0x7fd2d40cc760 2023-11-28T18:31:39.894 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.892+0000 7fd2caffd640 1 -- 172.21.15.105:0/3685123383 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16844 (secure 0 0 0) 0x7fd2d414d870 con 0x7fd2d40cc760 2023-11-28T18:31:39.894 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:39.895 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":117,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:31:04.144749+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:31:00.777530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-28T18:31:02.057361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:31:39.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.896+0000 7fd2d8c32640 1 -- 172.21.15.105:0/3685123383 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd2b4066900 msgr2=0x7fd2b4068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:39.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.896+0000 7fd2d8c32640 1 --2- 172.21.15.105:0/3685123383 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd2b4066900 0x7fd2b4068dc0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400b930 tx=0x7fd2c4008040 comp rx=0 tx=0).stop 2023-11-28T18:31:39.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.896+0000 7fd2d8c32640 1 -- 172.21.15.105:0/3685123383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2d40cc760 msgr2=0x7fd2d414ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:39.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.896+0000 7fd2d8c32640 1 --2- 172.21.15.105:0/3685123383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2d40cc760 0x7fd2d414ac40 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0af040 tx=0x7fd2bc019160 comp rx=0 tx=0).stop 2023-11-28T18:31:39.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.896+0000 7fd2d8c32640 1 -- 172.21.15.105:0/3685123383 shutdown_connections 2023-11-28T18:31:39.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.896+0000 7fd2d8c32640 1 --2- 172.21.15.105:0/3685123383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd2d4159f40 0x7fd2d415ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:39.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.896+0000 7fd2d8c32640 1 --2- 172.21.15.105:0/3685123383 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd2b4066900 0x7fd2b4068dc0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:39.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.896+0000 7fd2d8c32640 1 --2- 172.21.15.105:0/3685123383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2d40cc760 0x7fd2d414ac40 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:39.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.896+0000 7fd2d8c32640 1 --2- 172.21.15.105:0/3685123383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2d40ad680 0x7fd2d41395d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:39.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.896+0000 7fd2d8c32640 1 -- 172.21.15.105:0/3685123383 >> 172.21.15.105:0/3685123383 conn(0x7fd2d40ceb90 msgr2=0x7fd2d415d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:39.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.896+0000 7fd2d8c32640 1 -- 172.21.15.105:0/3685123383 shutdown_connections 2023-11-28T18:31:39.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:39.896+0000 7fd2d8c32640 1 -- 172.21.15.105:0/3685123383 wait complete. 2023-11-28T18:31:39.915 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:31:00.777530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:31:02.057361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:31:39.916 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:31:40.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.056+0000 7f708d72a640 1 Processor -- start 2023-11-28T18:31:40.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.056+0000 7f708d72a640 1 -- start start 2023-11-28T18:31:40.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.056+0000 7f708d72a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70880d1650 0x7f70880d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:40.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.056+0000 7f708d72a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f70880c7510 0x7f70880c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:40.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.056+0000 7f708d72a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70880d0c60 con 0x7f708815d300 2023-11-28T18:31:40.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.056+0000 7f708d72a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70880d1470 con 0x7f70880c7510 2023-11-28T18:31:40.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.056+0000 7f708d72a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70880cefa0 con 0x7f70880d1650 2023-11-28T18:31:40.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.056+0000 7f70877fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f70880c7510 0x7f70880c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.056+0000 7f70867fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70880d1650 0x7f70880d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.056+0000 7f70867fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70880d1650 0x7f70880d1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:42912/0 (socket says 172.21.15.105:42912) 2023-11-28T18:31:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.056+0000 7f7086ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f708815d300 0x7f70880d37a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:42064/0 (socket says 172.21.15.105:42064) 2023-11-28T18:31:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.056+0000 7f70867fc640 1 -- 172.21.15.105:0/964364461 learned_addr learned my addr 172.21.15.105:0/964364461 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.060+0000 7f70877fe640 1 -- 172.21.15.105:0/964364461 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70880d1650 msgr2=0x7f70880d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.060+0000 7f70877fe640 1 --2- 172.21.15.105:0/964364461 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70880d1650 0x7f70880d1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.060+0000 7f70877fe640 1 -- 172.21.15.105:0/964364461 >> v1:172.21.15.148:6789/0 conn(0x7f708815d300 legacy=0x7f70880d37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.060+0000 7f70877fe640 1 -- 172.21.15.105:0/964364461 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70880d0eb0 con 0x7f70880c7510 2023-11-28T18:31:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.060+0000 7f70867fc640 1 --2- 172.21.15.105:0/964364461 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70880d1650 0x7f70880d1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:31:40.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.060+0000 7f70877fe640 1 --2- 172.21.15.105:0/964364461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70880c7510 0x7f70880c79a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f70800097c0 tx=0x7f708001c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fff4f044d3c49ba1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:40.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.060+0000 7f7085ffb640 1 -- 172.21.15.105:0/964364461 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70800ab600 con 0x7f70880c7510 2023-11-28T18:31:40.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.060+0000 7f7085ffb640 1 -- 172.21.15.105:0/964364461 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70800a7050 con 0x7f70880c7510 2023-11-28T18:31:40.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.060+0000 7f7085ffb640 1 -- 172.21.15.105:0/964364461 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70800ab980 con 0x7f70880c7510 2023-11-28T18:31:40.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f708d72a640 1 -- 172.21.15.105:0/964364461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70880c7510 msgr2=0x7f70880c79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:40.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f708d72a640 1 --2- 172.21.15.105:0/964364461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70880c7510 0x7f70880c79a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f70800097c0 tx=0x7f708001c3f0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f708d72a640 1 -- 172.21.15.105:0/964364461 shutdown_connections 2023-11-28T18:31:40.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f708d72a640 1 --2- 172.21.15.105:0/964364461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70880c7510 0x7f70880c79a0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f708d72a640 1 --2- 172.21.15.105:0/964364461 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70880d1650 0x7f70880d1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f708d72a640 1 -- 172.21.15.105:0/964364461 >> 172.21.15.105:0/964364461 conn(0x7f708809c360 msgr2=0x7f70880d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:40.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f708d72a640 1 -- 172.21.15.105:0/964364461 shutdown_connections 2023-11-28T18:31:40.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f708d72a640 1 -- 172.21.15.105:0/964364461 wait complete. 2023-11-28T18:31:40.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f708d72a640 1 Processor -- start 2023-11-28T18:31:40.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f708d72a640 1 -- start start 2023-11-28T18:31:40.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f708d72a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70880c7510 0x7f708814d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:40.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f708d72a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70880d1650 0x7f70880b7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:40.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f708d72a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f708815d300 0x7f7088159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:40.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f708d72a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708814eef0 con 0x7f70880c7510 2023-11-28T18:31:40.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f708d72a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708814f380 con 0x7f70880d1650 2023-11-28T18:31:40.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f708d72a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708814dd70 con 0x7f708815d300 2023-11-28T18:31:40.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f7086ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70880c7510 0x7f708814d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:40.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f7086ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70880c7510 0x7f708814d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36450/0 (socket says 172.21.15.105:36450) 2023-11-28T18:31:40.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f7086ffd640 1 -- 172.21.15.105:0/3101987243 learned_addr learned my addr 172.21.15.105:0/3101987243 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:40.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f70867fc640 1 --2- 172.21.15.105:0/3101987243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70880d1650 0x7f70880b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:40.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f70877fe640 1 --2- 172.21.15.105:0/3101987243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f708815d300 0x7f7088159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:40.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f7086ffd640 1 -- 172.21.15.105:0/3101987243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f708815d300 msgr2=0x7f7088159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:40.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f7086ffd640 1 --2- 172.21.15.105:0/3101987243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f708815d300 0x7f7088159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f7086ffd640 1 -- 172.21.15.105:0/3101987243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70880d1650 msgr2=0x7f70880b7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:40.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f7086ffd640 1 --2- 172.21.15.105:0/3101987243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70880d1650 0x7f70880b7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f7086ffd640 1 -- 172.21.15.105:0/3101987243 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c0094d0 con 0x7f70880c7510 2023-11-28T18:31:40.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.064+0000 7f7086ffd640 1 --2- 172.21.15.105:0/3101987243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70880c7510 0x7f708814d830 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f707800c870 tx=0x7f707800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:40.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.068+0000 7f706f7fe640 1 -- 172.21.15.105:0/3101987243 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707800a640 con 0x7f70880c7510 2023-11-28T18:31:40.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.068+0000 7f706f7fe640 1 -- 172.21.15.105:0/3101987243 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707800a7a0 con 0x7f70880c7510 2023-11-28T18:31:40.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.068+0000 7f706f7fe640 1 -- 172.21.15.105:0/3101987243 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707800ab10 con 0x7f70880c7510 2023-11-28T18:31:40.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.068+0000 7f708d72a640 1 -- 172.21.15.105:0/3101987243 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7080009420 con 0x7f70880c7510 2023-11-28T18:31:40.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.068+0000 7f708d72a640 1 -- 172.21.15.105:0/3101987243 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708814e5c0 con 0x7f70880c7510 2023-11-28T18:31:40.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.068+0000 7f708d72a640 1 -- 172.21.15.105:0/3101987243 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7088067570 con 0x7f70880c7510 2023-11-28T18:31:40.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.068+0000 7f706f7fe640 1 -- 172.21.15.105:0/3101987243 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f707800ac70 con 0x7f70880c7510 2023-11-28T18:31:40.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.068+0000 7f706f7fe640 1 --2- 172.21.15.105:0/3101987243 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f70740668d0 0x7f7074068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:40.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.068+0000 7f706f7fe640 1 -- 172.21.15.105:0/3101987243 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7078083da0 con 0x7f70880c7510 2023-11-28T18:31:40.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.068+0000 7f70867fc640 1 --2- 172.21.15.105:0/3101987243 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f70740668d0 0x7f7074068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:40.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.068+0000 7f70867fc640 1 --2- 172.21.15.105:0/3101987243 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f70740668d0 0x7f7074068d90 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f70880a0fa0 tx=0x7f707c009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:40.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.072+0000 7f706f7fe640 1 -- 172.21.15.105:0/3101987243 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7078050f40 con 0x7f70880c7510 2023-11-28T18:31:40.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.224+0000 7f708d72a640 1 -- 172.21.15.105:0/3101987243 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f708806c9d0 con 0x7f70880c7510 2023-11-28T18:31:40.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.224+0000 7f706f7fe640 1 -- 172.21.15.105:0/3101987243 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7078017020 con 0x7f70880c7510 2023-11-28T18:31:40.223 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:31:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.224+0000 7f708d72a640 1 -- 172.21.15.105:0/3101987243 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f70740668d0 msgr2=0x7f7074068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.224+0000 7f708d72a640 1 --2- 172.21.15.105:0/3101987243 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f70740668d0 0x7f7074068d90 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f70880a0fa0 tx=0x7f707c009240 comp rx=0 tx=0).stop 2023-11-28T18:31:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.224+0000 7f708d72a640 1 -- 172.21.15.105:0/3101987243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70880c7510 msgr2=0x7f708814d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.224+0000 7f708d72a640 1 --2- 172.21.15.105:0/3101987243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70880c7510 0x7f708814d830 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f707800c870 tx=0x7f707800cd30 comp rx=0 tx=0).stop 2023-11-28T18:31:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.224+0000 7f708d72a640 1 -- 172.21.15.105:0/3101987243 shutdown_connections 2023-11-28T18:31:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.224+0000 7f708d72a640 1 --2- 172.21.15.105:0/3101987243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f708815d300 0x7f7088159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.224+0000 7f708d72a640 1 --2- 172.21.15.105:0/3101987243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70880d1650 0x7f70880b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.224+0000 7f708d72a640 1 --2- 172.21.15.105:0/3101987243 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f70740668d0 0x7f7074068d90 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.224+0000 7f708d72a640 1 --2- 172.21.15.105:0/3101987243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70880c7510 0x7f708814d830 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.224+0000 7f708d72a640 1 -- 172.21.15.105:0/3101987243 >> 172.21.15.105:0/3101987243 conn(0x7f708809c360 msgr2=0x7f70880d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.224+0000 7f708d72a640 1 -- 172.21.15.105:0/3101987243 shutdown_connections 2023-11-28T18:31:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.224+0000 7f708d72a640 1 -- 172.21.15.105:0/3101987243 wait complete. 2023-11-28T18:31:40.244 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:31:40.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.384+0000 7f2a52479640 1 Processor -- start 2023-11-28T18:31:40.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.384+0000 7f2a52479640 1 -- start start 2023-11-28T18:31:40.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.384+0000 7f2a52479640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2a4c071960 0x7f2a4c15f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:40.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.388+0000 7f2a52479640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a4c0701d0 con 0x7f2a4c15b030 2023-11-28T18:31:40.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.388+0000 7f2a52479640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a4c06df80 con 0x7f2a4c0b4290 2023-11-28T18:31:40.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.388+0000 7f2a52479640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a4c071df0 con 0x7f2a4c071960 2023-11-28T18:31:40.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.388+0000 7f2a50a16640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f2a4c15b030 0x7f2a4c0709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35142/0 (socket says 172.21.15.105:35142) 2023-11-28T18:31:40.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.388+0000 7f2a4b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2a4c071960 0x7f2a4c15f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:40.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.388+0000 7f2a50a16640 1 -- 172.21.15.105:0/2359067920 learned_addr learned my addr 172.21.15.105:0/2359067920 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:40.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.388+0000 7f2a4b7fe640 1 -- 172.21.15.105:0/2359067920 >> v1:172.21.15.148:6789/0 conn(0x7f2a4c0b4290 legacy=0x7f2a4c0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:40.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.388+0000 7f2a4b7fe640 1 -- 172.21.15.105:0/2359067920 >> v1:172.21.15.105:6789/0 conn(0x7f2a4c15b030 legacy=0x7f2a4c0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:40.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.388+0000 7f2a4affd640 1 -- 172.21.15.105:0/2359067920 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268596194 0 0) 0x7f2a4c0701d0 con 0x7f2a4c15b030 2023-11-28T18:31:40.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.388+0000 7f2a4b7fe640 1 -- 172.21.15.105:0/2359067920 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4c0720d0 con 0x7f2a4c071960 2023-11-28T18:31:40.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.388+0000 7f2a4b7fe640 1 --2- 172.21.15.105:0/2359067920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a4c071960 0x7f2a4c15f3b0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f2a400097c0 tx=0x7f2a4001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48583d7c1130eec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:40.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.388+0000 7f2a4affd640 1 -- 172.21.15.105:0/2359067920 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a400ab650 con 0x7f2a4c071960 2023-11-28T18:31:40.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.388+0000 7f2a4affd640 1 -- 172.21.15.105:0/2359067920 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a400ab7b0 con 0x7f2a4c071960 2023-11-28T18:31:40.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.388+0000 7f2a4affd640 1 -- 172.21.15.105:0/2359067920 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a400abb40 con 0x7f2a4c071960 2023-11-28T18:31:40.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.392+0000 7f2a52479640 1 -- 172.21.15.105:0/2359067920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a4c071960 msgr2=0x7f2a4c15f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:40.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.392+0000 7f2a52479640 1 --2- 172.21.15.105:0/2359067920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a4c071960 0x7f2a4c15f3b0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f2a400097c0 tx=0x7f2a4001c450 comp rx=0 tx=0).stop 2023-11-28T18:31:40.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.392+0000 7f2a52479640 1 -- 172.21.15.105:0/2359067920 shutdown_connections 2023-11-28T18:31:40.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.392+0000 7f2a52479640 1 --2- 172.21.15.105:0/2359067920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a4c071960 0x7f2a4c15f3b0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.392+0000 7f2a52479640 1 -- 172.21.15.105:0/2359067920 >> 172.21.15.105:0/2359067920 conn(0x7f2a4c0ceb90 msgr2=0x7f2a4c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:40.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.392+0000 7f2a52479640 1 -- 172.21.15.105:0/2359067920 shutdown_connections 2023-11-28T18:31:40.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.392+0000 7f2a52479640 1 -- 172.21.15.105:0/2359067920 wait complete. 2023-11-28T18:31:40.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.392+0000 7f2a52479640 1 Processor -- start 2023-11-28T18:31:40.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.392+0000 7f2a52479640 1 -- start start 2023-11-28T18:31:40.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.392+0000 7f2a52479640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2a4c0b4290 0x7f2a4c164b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:40.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.392+0000 7f2a52479640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a4c15b030 0x7f2a4c0b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:40.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.392+0000 7f2a52479640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a4c0b3c40 0x7f2a4c1535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:40.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.392+0000 7f2a52479640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c0b2680 con 0x7f2a4c0b3c40 2023-11-28T18:31:40.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.392+0000 7f2a52479640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c0b2960 con 0x7f2a4c15b030 2023-11-28T18:31:40.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.392+0000 7f2a52479640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c0b2c40 con 0x7f2a4c0b4290 2023-11-28T18:31:40.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.392+0000 7f2a50a16640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a4c0b3c40 0x7f2a4c1535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:40.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.392+0000 7f2a50a16640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a4c0b3c40 0x7f2a4c1535b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36464/0 (socket says 172.21.15.105:36464) 2023-11-28T18:31:40.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.392+0000 7f2a50a16640 1 -- 172.21.15.105:0/2608412713 learned_addr learned my addr 172.21.15.105:0/2608412713 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:40.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.392+0000 7f2a4bfff640 1 --2- 172.21.15.105:0/2608412713 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2a4c0b4290 0x7f2a4c164b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:40.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.392+0000 7f2a4b7fe640 1 --2- 172.21.15.105:0/2608412713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a4c15b030 0x7f2a4c0b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:40.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.396+0000 7f2a50a16640 1 -- 172.21.15.105:0/2608412713 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2a4c0b4290 msgr2=0x7f2a4c164b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:40.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.396+0000 7f2a50a16640 1 --2- 172.21.15.105:0/2608412713 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2a4c0b4290 0x7f2a4c164b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.396+0000 7f2a50a16640 1 -- 172.21.15.105:0/2608412713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a4c15b030 msgr2=0x7f2a4c0b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:40.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.396+0000 7f2a50a16640 1 --2- 172.21.15.105:0/2608412713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a4c15b030 0x7f2a4c0b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.396+0000 7f2a50a16640 1 -- 172.21.15.105:0/2608412713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a40009420 con 0x7f2a4c0b3c40 2023-11-28T18:31:40.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.396+0000 7f2a50a16640 1 --2- 172.21.15.105:0/2608412713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a4c0b3c40 0x7f2a4c1535b0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400c780 tx=0x7f2a4400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:40.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.396+0000 7f2a48ff9640 1 -- 172.21.15.105:0/2608412713 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4400ce30 con 0x7f2a4c0b3c40 2023-11-28T18:31:40.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.396+0000 7f2a48ff9640 1 -- 172.21.15.105:0/2608412713 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4400a640 con 0x7f2a4c0b3c40 2023-11-28T18:31:40.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.396+0000 7f2a48ff9640 1 -- 172.21.15.105:0/2608412713 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4400a9f0 con 0x7f2a4c0b3c40 2023-11-28T18:31:40.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.396+0000 7f2a52479640 1 -- 172.21.15.105:0/2608412713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4c0b2f20 con 0x7f2a4c0b3c40 2023-11-28T18:31:40.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.396+0000 7f2a52479640 1 -- 172.21.15.105:0/2608412713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4c0b2310 con 0x7f2a4c0b3c40 2023-11-28T18:31:40.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.396+0000 7f2a52479640 1 -- 172.21.15.105:0/2608412713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a4c067570 con 0x7f2a4c0b3c40 2023-11-28T18:31:40.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.396+0000 7f2a48ff9640 1 -- 172.21.15.105:0/2608412713 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f2a4400ab50 con 0x7f2a4c0b3c40 2023-11-28T18:31:40.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.396+0000 7f2a48ff9640 1 --2- 172.21.15.105:0/2608412713 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2a380668d0 0x7f2a38068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:40.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.396+0000 7f2a48ff9640 1 -- 172.21.15.105:0/2608412713 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2a44083900 con 0x7f2a4c0b3c40 2023-11-28T18:31:40.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.396+0000 7f2a4bfff640 1 --2- 172.21.15.105:0/2608412713 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2a380668d0 0x7f2a38068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:40.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.400+0000 7f2a4bfff640 1 --2- 172.21.15.105:0/2608412713 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2a380668d0 0x7f2a38068d90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c000f70 tx=0x7f2a3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:40.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.400+0000 7f2a48ff9640 1 -- 172.21.15.105:0/2608412713 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2a4404b280 con 0x7f2a4c0b3c40 2023-11-28T18:31:40.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.548+0000 7f2a52479640 1 -- 172.21.15.105:0/2608412713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2a4c0b4070 con 0x7f2a4c0b3c40 2023-11-28T18:31:40.550 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.552+0000 7f2a48ff9640 1 -- 172.21.15.105:0/2608412713 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v117) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2a44018020 con 0x7f2a4c0b3c40 2023-11-28T18:31:40.551 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:31:40.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.552+0000 7f2a52479640 1 -- 172.21.15.105:0/2608412713 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2a380668d0 msgr2=0x7f2a38068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:40.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.552+0000 7f2a52479640 1 --2- 172.21.15.105:0/2608412713 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2a380668d0 0x7f2a38068d90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c000f70 tx=0x7f2a3c009040 comp rx=0 tx=0).stop 2023-11-28T18:31:40.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.552+0000 7f2a52479640 1 -- 172.21.15.105:0/2608412713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a4c0b3c40 msgr2=0x7f2a4c1535b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:40.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.552+0000 7f2a52479640 1 --2- 172.21.15.105:0/2608412713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a4c0b3c40 0x7f2a4c1535b0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400c780 tx=0x7f2a4400cc40 comp rx=0 tx=0).stop 2023-11-28T18:31:40.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.552+0000 7f2a52479640 1 -- 172.21.15.105:0/2608412713 shutdown_connections 2023-11-28T18:31:40.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.552+0000 7f2a52479640 1 --2- 172.21.15.105:0/2608412713 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2a380668d0 0x7f2a38068d90 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.552+0000 7f2a52479640 1 --2- 172.21.15.105:0/2608412713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a4c0b3c40 0x7f2a4c1535b0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.552+0000 7f2a52479640 1 --2- 172.21.15.105:0/2608412713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a4c15b030 0x7f2a4c0b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.552+0000 7f2a52479640 1 --2- 172.21.15.105:0/2608412713 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2a4c0b4290 0x7f2a4c164b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.552+0000 7f2a52479640 1 -- 172.21.15.105:0/2608412713 >> 172.21.15.105:0/2608412713 conn(0x7f2a4c0ceb90 msgr2=0x7f2a4c15e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:40.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.552+0000 7f2a52479640 1 -- 172.21.15.105:0/2608412713 shutdown_connections 2023-11-28T18:31:40.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.552+0000 7f2a52479640 1 -- 172.21.15.105:0/2608412713 wait complete. 2023-11-28T18:31:40.572 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:31:40.713 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.712+0000 7fdf5952a640 1 Processor -- start 2023-11-28T18:31:40.714 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.712+0000 7fdf5952a640 1 -- start start 2023-11-28T18:31:40.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.712+0000 7fdf5952a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdf540cdb50 0x7fdf540ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:40.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.712+0000 7fdf5952a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf540d3e60 con 0x7fdf540d00b0 2023-11-28T18:31:40.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.712+0000 7fdf5952a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf540cf080 con 0x7fdf540b8a20 2023-11-28T18:31:40.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.712+0000 7fdf5952a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf540cdfc0 con 0x7fdf540cdb50 2023-11-28T18:31:40.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.716+0000 7fdf52ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fdf540b8a20 0x7fdf540cf9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:42094/0 (socket says 172.21.15.105:42094) 2023-11-28T18:31:40.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.716+0000 7fdf4a7fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fdf540d00b0 0x7fdf540b6da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35158/0 (socket says 172.21.15.105:35158) 2023-11-28T18:31:40.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.716+0000 7fdf52ffd640 1 -- 172.21.15.105:0/3022159310 learned_addr learned my addr 172.21.15.105:0/3022159310 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:40.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.716+0000 7fdf537fe640 1 --2- 172.21.15.105:0/3022159310 >> v2:172.21.15.105:3300/0 conn(0x7fdf540cdb50 0x7fdf540ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:40.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.716+0000 7fdf537fe640 1 -- 172.21.15.105:0/3022159310 >> v1:172.21.15.148:6789/0 conn(0x7fdf540b8a20 legacy=0x7fdf540cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:40.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.716+0000 7fdf537fe640 1 -- 172.21.15.105:0/3022159310 >> v1:172.21.15.105:6789/0 conn(0x7fdf540d00b0 legacy=0x7fdf540b6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:40.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.716+0000 7fdf537fe640 1 -- 172.21.15.105:0/3022159310 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf540b5fe0 con 0x7fdf540cdb50 2023-11-28T18:31:40.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.716+0000 7fdf537fe640 1 --2- 172.21.15.105:0/3022159310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf540cdb50 0x7fdf540ceb40 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0097c0 tx=0x7fdf4c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935f3420d1345740 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:40.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.716+0000 7fdf527fc640 1 -- 172.21.15.105:0/3022159310 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0ab650 con 0x7fdf540cdb50 2023-11-28T18:31:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.716+0000 7fdf527fc640 1 -- 172.21.15.105:0/3022159310 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4c0ab7b0 con 0x7fdf540cdb50 2023-11-28T18:31:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.716+0000 7fdf527fc640 1 -- 172.21.15.105:0/3022159310 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0abb40 con 0x7fdf540cdb50 2023-11-28T18:31:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf5952a640 1 -- 172.21.15.105:0/3022159310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf540cdb50 msgr2=0x7fdf540ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf5952a640 1 --2- 172.21.15.105:0/3022159310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf540cdb50 0x7fdf540ceb40 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0097c0 tx=0x7fdf4c01c450 comp rx=0 tx=0).stop 2023-11-28T18:31:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf5952a640 1 -- 172.21.15.105:0/3022159310 shutdown_connections 2023-11-28T18:31:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf5952a640 1 --2- 172.21.15.105:0/3022159310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf540cdb50 0x7fdf540ceb40 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf5952a640 1 -- 172.21.15.105:0/3022159310 >> 172.21.15.105:0/3022159310 conn(0x7fdf5409c340 msgr2=0x7fdf540d2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf5952a640 1 -- 172.21.15.105:0/3022159310 shutdown_connections 2023-11-28T18:31:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf5952a640 1 -- 172.21.15.105:0/3022159310 wait complete. 2023-11-28T18:31:40.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf5952a640 1 Processor -- start 2023-11-28T18:31:40.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf5952a640 1 -- start start 2023-11-28T18:31:40.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf5952a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf540b8a20 0x7fdf540b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:40.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf5952a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf540cdb50 0x7fdf540d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:40.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf5952a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf540d00b0 0x7fdf540d1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:40.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf5952a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf540cc310 con 0x7fdf540d00b0 2023-11-28T18:31:40.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf5952a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf540ccb20 con 0x7fdf540cdb50 2023-11-28T18:31:40.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf5952a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf540b3970 con 0x7fdf540b8a20 2023-11-28T18:31:40.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf540d00b0 0x7fdf540d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf540d00b0 0x7fdf540d1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36482/0 (socket says 172.21.15.105:36482) 2023-11-28T18:31:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf537fe640 1 -- 172.21.15.105:0/1339251056 learned_addr learned my addr 172.21.15.105:0/1339251056 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf4a7fc640 1 --2- 172.21.15.105:0/1339251056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf540cdb50 0x7fdf540d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf52ffd640 1 --2- 172.21.15.105:0/1339251056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf540b8a20 0x7fdf540b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf537fe640 1 -- 172.21.15.105:0/1339251056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf540b8a20 msgr2=0x7fdf540b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf537fe640 1 --2- 172.21.15.105:0/1339251056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf540b8a20 0x7fdf540b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf537fe640 1 -- 172.21.15.105:0/1339251056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf540cdb50 msgr2=0x7fdf540d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf537fe640 1 --2- 172.21.15.105:0/1339251056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf540cdb50 0x7fdf540d0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf537fe640 1 -- 172.21.15.105:0/1339251056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4c009420 con 0x7fdf540d00b0 2023-11-28T18:31:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf52ffd640 1 --2- 172.21.15.105:0/1339251056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf540b8a20 0x7fdf540b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:31:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.720+0000 7fdf537fe640 1 --2- 172.21.15.105:0/1339251056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf540d00b0 0x7fdf540d1430 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c019060 tx=0x7fdf4c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:40.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.724+0000 7fdf4bfff640 1 -- 172.21.15.105:0/1339251056 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0b5810 con 0x7fdf540d00b0 2023-11-28T18:31:40.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.724+0000 7fdf4bfff640 1 -- 172.21.15.105:0/1339251056 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4c0b5970 con 0x7fdf540d00b0 2023-11-28T18:31:40.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.724+0000 7fdf5952a640 1 -- 172.21.15.105:0/1339251056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf540cc560 con 0x7fdf540d00b0 2023-11-28T18:31:40.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.724+0000 7fdf4bfff640 1 -- 172.21.15.105:0/1339251056 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0b5d20 con 0x7fdf540d00b0 2023-11-28T18:31:40.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.724+0000 7fdf5952a640 1 -- 172.21.15.105:0/1339251056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5415cbf0 con 0x7fdf540d00b0 2023-11-28T18:31:40.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.724+0000 7fdf5952a640 1 -- 172.21.15.105:0/1339251056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf540afd30 con 0x7fdf540d00b0 2023-11-28T18:31:40.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.724+0000 7fdf4bfff640 1 -- 172.21.15.105:0/1339251056 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fdf4c0bf410 con 0x7fdf540d00b0 2023-11-28T18:31:40.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.724+0000 7fdf4bfff640 1 --2- 172.21.15.105:0/1339251056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdf340668d0 0x7fdf34068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:40.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.724+0000 7fdf4bfff640 1 -- 172.21.15.105:0/1339251056 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdf4c11f630 con 0x7fdf540d00b0 2023-11-28T18:31:40.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.724+0000 7fdf52ffd640 1 --2- 172.21.15.105:0/1339251056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdf340668d0 0x7fdf34068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:40.727 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.728+0000 7fdf52ffd640 1 --2- 172.21.15.105:0/1339251056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdf340668d0 0x7fdf34068d90 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fdf44005890 tx=0x7fdf44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:40.730 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.728+0000 7fdf4bfff640 1 -- 172.21.15.105:0/1339251056 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdf540afd30 con 0x7fdf540d00b0 2023-11-28T18:31:40.877 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.876+0000 7fdf5952a640 1 -- 172.21.15.105:0/1339251056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdf540afd30 con 0x7fdf540d00b0 2023-11-28T18:31:40.878 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.876+0000 7fdf4bfff640 1 -- 172.21.15.105:0/1339251056 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v117) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdf540afd30 con 0x7fdf540d00b0 2023-11-28T18:31:40.878 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:31:40.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.880+0000 7fdf5952a640 1 -- 172.21.15.105:0/1339251056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdf340668d0 msgr2=0x7fdf34068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:40.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.880+0000 7fdf5952a640 1 --2- 172.21.15.105:0/1339251056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdf340668d0 0x7fdf34068d90 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fdf44005890 tx=0x7fdf44009040 comp rx=0 tx=0).stop 2023-11-28T18:31:40.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.880+0000 7fdf5952a640 1 -- 172.21.15.105:0/1339251056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf540d00b0 msgr2=0x7fdf540d1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:40.880 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.880+0000 7fdf5952a640 1 --2- 172.21.15.105:0/1339251056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf540d00b0 0x7fdf540d1430 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c019060 tx=0x7fdf4c01ac90 comp rx=0 tx=0).stop 2023-11-28T18:31:40.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.880+0000 7fdf5952a640 1 -- 172.21.15.105:0/1339251056 shutdown_connections 2023-11-28T18:31:40.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.880+0000 7fdf5952a640 1 --2- 172.21.15.105:0/1339251056 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fdf340668d0 0x7fdf34068d90 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.880+0000 7fdf5952a640 1 --2- 172.21.15.105:0/1339251056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf540d00b0 0x7fdf540d1430 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.880+0000 7fdf5952a640 1 --2- 172.21.15.105:0/1339251056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf540cdb50 0x7fdf540d0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.880+0000 7fdf5952a640 1 --2- 172.21.15.105:0/1339251056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf540b8a20 0x7fdf540b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:40.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.880+0000 7fdf5952a640 1 -- 172.21.15.105:0/1339251056 >> 172.21.15.105:0/1339251056 conn(0x7fdf5409c340 msgr2=0x7fdf5406c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:40.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.880+0000 7fdf5952a640 1 -- 172.21.15.105:0/1339251056 shutdown_connections 2023-11-28T18:31:40.881 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:40.880+0000 7fdf5952a640 1 -- 172.21.15.105:0/1339251056 wait complete. 2023-11-28T18:31:40.899 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T18:31:41.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.036+0000 7fed3fb57640 1 Processor -- start 2023-11-28T18:31:41.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.036+0000 7fed3fb57640 1 -- start start 2023-11-28T18:31:41.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.040+0000 7fed3fb57640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fed38071960 0x7fed3815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.040+0000 7fed3fb57640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed3815b030 0x7fed380709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.040+0000 7fed3fb57640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed380701d0 con 0x7fed380b4290 2023-11-28T18:31:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.040+0000 7fed3fb57640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed3806df80 con 0x7fed38071960 2023-11-28T18:31:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.040+0000 7fed3fb57640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed38071df0 con 0x7fed3815b030 2023-11-28T18:31:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.040+0000 7fed3e0f4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed3815b030 0x7fed380709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.040+0000 7fed3e0f4640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed3815b030 0x7fed380709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:46450/0 (socket says 172.21.15.105:46450) 2023-11-28T18:31:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.040+0000 7fed3e0f4640 1 -- 172.21.15.105:0/365164528 learned_addr learned my addr 172.21.15.105:0/365164528 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.040+0000 7fed3d0f2640 1 --2- 172.21.15.105:0/365164528 >> v2:172.21.15.148:3300/0 conn(0x7fed38071960 0x7fed3815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.040+0000 7fed3e0f4640 1 -- 172.21.15.105:0/365164528 >> v2:172.21.15.148:3300/0 conn(0x7fed38071960 msgr2=0x7fed3815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.040+0000 7fed3e0f4640 1 --2- 172.21.15.105:0/365164528 >> v2:172.21.15.148:3300/0 conn(0x7fed38071960 0x7fed3815f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.040+0000 7fed3e0f4640 1 -- 172.21.15.105:0/365164528 >> v1:172.21.15.148:6789/0 conn(0x7fed380b4290 legacy=0x7fed380b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.040+0000 7fed3e0f4640 1 -- 172.21.15.105:0/365164528 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed380720d0 con 0x7fed3815b030 2023-11-28T18:31:41.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.040+0000 7fed3e0f4640 1 --2- 172.21.15.105:0/365164528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed3815b030 0x7fed380709b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fed30009e50 tx=0x7fed3001c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aace1c7dd82b72d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.044+0000 7fed3c8f1640 1 -- 172.21.15.105:0/365164528 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed300a57c0 con 0x7fed3815b030 2023-11-28T18:31:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.044+0000 7fed3c8f1640 1 -- 172.21.15.105:0/365164528 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed300a5920 con 0x7fed3815b030 2023-11-28T18:31:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.044+0000 7fed3c8f1640 1 -- 172.21.15.105:0/365164528 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed300a5cb0 con 0x7fed3815b030 2023-11-28T18:31:41.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.044+0000 7fed3fb57640 1 -- 172.21.15.105:0/365164528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed3815b030 msgr2=0x7fed380709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:41.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.044+0000 7fed3fb57640 1 --2- 172.21.15.105:0/365164528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed3815b030 0x7fed380709b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fed30009e50 tx=0x7fed3001c9e0 comp rx=0 tx=0).stop 2023-11-28T18:31:41.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.044+0000 7fed3fb57640 1 -- 172.21.15.105:0/365164528 shutdown_connections 2023-11-28T18:31:41.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.044+0000 7fed3fb57640 1 --2- 172.21.15.105:0/365164528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed3815b030 0x7fed380709b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:41.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.044+0000 7fed3fb57640 1 --2- 172.21.15.105:0/365164528 >> v2:172.21.15.148:3300/0 conn(0x7fed38071960 0x7fed3815f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:41.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.044+0000 7fed3fb57640 1 -- 172.21.15.105:0/365164528 >> 172.21.15.105:0/365164528 conn(0x7fed380ceb90 msgr2=0x7fed380cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:41.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.044+0000 7fed3fb57640 1 -- 172.21.15.105:0/365164528 shutdown_connections 2023-11-28T18:31:41.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.044+0000 7fed3fb57640 1 -- 172.21.15.105:0/365164528 wait complete. 2023-11-28T18:31:41.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.044+0000 7fed3fb57640 1 Processor -- start 2023-11-28T18:31:41.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.044+0000 7fed3fb57640 1 -- start start 2023-11-28T18:31:41.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.044+0000 7fed3fb57640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed380b4290 0x7fed3814f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:41.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.044+0000 7fed3fb57640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed3815b030 0x7fed3814e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:41.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.044+0000 7fed3fb57640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed3814e680 0x7fed38158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:41.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.048+0000 7fed3fb57640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3814cc70 con 0x7fed3814e680 2023-11-28T18:31:41.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.048+0000 7fed3fb57640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3814b580 con 0x7fed3815b030 2023-11-28T18:31:41.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.048+0000 7fed3fb57640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3814b860 con 0x7fed380b4290 2023-11-28T18:31:41.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.048+0000 7fed3e0f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed3814e680 0x7fed38158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:41.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.048+0000 7fed3e0f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed3814e680 0x7fed38158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49972/0 (socket says 172.21.15.105:49972) 2023-11-28T18:31:41.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.048+0000 7fed3e0f4640 1 -- 172.21.15.105:0/644256477 learned_addr learned my addr 172.21.15.105:0/644256477 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:41.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.048+0000 7fed3d8f3640 1 --2- 172.21.15.105:0/644256477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed380b4290 0x7fed3814f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:41.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.048+0000 7fed3d0f2640 1 --2- 172.21.15.105:0/644256477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed3815b030 0x7fed3814e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:41.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.048+0000 7fed3e0f4640 1 -- 172.21.15.105:0/644256477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed380b4290 msgr2=0x7fed3814f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:41.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.048+0000 7fed3e0f4640 1 --2- 172.21.15.105:0/644256477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed380b4290 0x7fed3814f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:41.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.048+0000 7fed3e0f4640 1 -- 172.21.15.105:0/644256477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed3815b030 msgr2=0x7fed3814e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:41.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.048+0000 7fed3e0f4640 1 --2- 172.21.15.105:0/644256477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed3815b030 0x7fed3814e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:41.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.048+0000 7fed3e0f4640 1 -- 172.21.15.105:0/644256477 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed30009b30 con 0x7fed3814e680 2023-11-28T18:31:41.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.048+0000 7fed3e0f4640 1 --2- 172.21.15.105:0/644256477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed3814e680 0x7fed38158630 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fed300af040 tx=0x7fed30019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:41.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.048+0000 7fed2e7fc640 1 -- 172.21.15.105:0/644256477 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed300be070 con 0x7fed3814e680 2023-11-28T18:31:41.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.048+0000 7fed2e7fc640 1 -- 172.21.15.105:0/644256477 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed300b8440 con 0x7fed3814e680 2023-11-28T18:31:41.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.048+0000 7fed2e7fc640 1 -- 172.21.15.105:0/644256477 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed300b87d0 con 0x7fed3814e680 2023-11-28T18:31:41.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.048+0000 7fed3fb57640 1 -- 172.21.15.105:0/644256477 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3814bb40 con 0x7fed3814e680 2023-11-28T18:31:41.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.048+0000 7fed3fb57640 1 -- 172.21.15.105:0/644256477 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3814c100 con 0x7fed3814e680 2023-11-28T18:31:41.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.048+0000 7fed3fb57640 1 -- 172.21.15.105:0/644256477 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed380cb410 con 0x7fed3814e680 2023-11-28T18:31:41.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.048+0000 7fed2e7fc640 1 -- 172.21.15.105:0/644256477 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fed300b8930 con 0x7fed3814e680 2023-11-28T18:31:41.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.052+0000 7fed2e7fc640 1 --2- 172.21.15.105:0/644256477 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fed1c0668d0 0x7fed1c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:41.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.052+0000 7fed2e7fc640 1 -- 172.21.15.105:0/644256477 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fed3011ee20 con 0x7fed3814e680 2023-11-28T18:31:41.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.052+0000 7fed3d8f3640 1 --2- 172.21.15.105:0/644256477 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fed1c0668d0 0x7fed1c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:41.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.052+0000 7fed3d8f3640 1 --2- 172.21.15.105:0/644256477 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fed1c0668d0 0x7fed1c068d90 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fed34000f70 tx=0x7fed34009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:41.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.056+0000 7fed2e7fc640 1 -- 172.21.15.105:0/644256477 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fed300ebe80 con 0x7fed3814e680 2023-11-28T18:31:41.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.216+0000 7fed3fb57640 1 -- 172.21.15.105:0/644256477 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fed38067570 con 0x7fed3814e680 2023-11-28T18:31:41.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.216+0000 7fed2e7fc640 1 -- 172.21.15.105:0/644256477 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v117) v1 ==== 80+0+0 (secure 0 0 0) 0x7fed300ef4a0 con 0x7fed3814e680 2023-11-28T18:31:41.216 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2023-11-28T18:31:41.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.216+0000 7fed3fb57640 1 -- 172.21.15.105:0/644256477 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fed1c0668d0 msgr2=0x7fed1c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:41.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.216+0000 7fed3fb57640 1 --2- 172.21.15.105:0/644256477 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fed1c0668d0 0x7fed1c068d90 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fed34000f70 tx=0x7fed34009040 comp rx=0 tx=0).stop 2023-11-28T18:31:41.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.216+0000 7fed3fb57640 1 -- 172.21.15.105:0/644256477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed3814e680 msgr2=0x7fed38158630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:41.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.216+0000 7fed3fb57640 1 --2- 172.21.15.105:0/644256477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed3814e680 0x7fed38158630 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fed300af040 tx=0x7fed30019780 comp rx=0 tx=0).stop 2023-11-28T18:31:41.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.216+0000 7fed3fb57640 1 -- 172.21.15.105:0/644256477 shutdown_connections 2023-11-28T18:31:41.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.216+0000 7fed3fb57640 1 --2- 172.21.15.105:0/644256477 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fed1c0668d0 0x7fed1c068d90 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:41.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.216+0000 7fed3fb57640 1 --2- 172.21.15.105:0/644256477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed3814e680 0x7fed38158630 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:41.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.216+0000 7fed3fb57640 1 --2- 172.21.15.105:0/644256477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed3815b030 0x7fed3814e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:41.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.216+0000 7fed3fb57640 1 --2- 172.21.15.105:0/644256477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed380b4290 0x7fed3814f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:41.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.216+0000 7fed3fb57640 1 -- 172.21.15.105:0/644256477 >> 172.21.15.105:0/644256477 conn(0x7fed380ceb90 msgr2=0x7fed380b4670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:41.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.216+0000 7fed3fb57640 1 -- 172.21.15.105:0/644256477 shutdown_connections 2023-11-28T18:31:41.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.216+0000 7fed3fb57640 1 -- 172.21.15.105:0/644256477 wait complete. 2023-11-28T18:31:41.236 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2023-11-28T18:31:41.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:31:41.237 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-28T18:31:41.255 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[65733]: fuse finished with error 0 and tester_r 0 2023-11-28T18:31:41.260 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:31:41.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:31:41.262 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T18:31:41.443 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:31:41.443 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:31:41.453 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:31:41.453 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:31:41.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:31:41.454 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T18:31:41.503 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-28T18:31:41.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:31:41.504 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:31:41.504 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:31:41.514 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:31:41.514 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:31:41.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:31:41.515 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T18:31:41.566 INFO:teuthology.orchestra.run.smithi105.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-28T18:31:41.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:31:41.567 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:31:41.568 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:31:41.577 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:31:41.577 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:31:41.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:31:41.577 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T18:31:41.627 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-28T18:31:41.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:31:41.628 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:31:41.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.764+0000 7f9db9157640 1 Processor -- start 2023-11-28T18:31:41.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.764+0000 7f9db9157640 1 -- start start 2023-11-28T18:31:41.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.768+0000 7f9db9157640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9db4159610 0x7f9db40ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:41.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.768+0000 7f9db9157640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db40cde30 con 0x7f9db40cc760 2023-11-28T18:31:41.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.768+0000 7f9db9157640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db41599f0 con 0x7f9db406cef0 2023-11-28T18:31:41.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.768+0000 7f9db9157640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db406de20 con 0x7f9db4159610 2023-11-28T18:31:41.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.768+0000 7f9db2d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9db4159610 0x7f9db40ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:41.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.768+0000 7f9daa59c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9db40cc760 0x7f9db406c7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49746/0 (socket says 172.21.15.105:49746) 2023-11-28T18:31:41.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.768+0000 7f9db2d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9db4159610 0x7f9db40ce2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49978/0 (socket says 172.21.15.105:49978) 2023-11-28T18:31:41.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.768+0000 7f9daa59c640 1 -- 172.21.15.105:0/3018256995 learned_addr learned my addr 172.21.15.105:0/3018256995 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:41.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.768+0000 7f9db2d9d640 1 -- 172.21.15.105:0/3018256995 >> v1:172.21.15.148:6789/0 conn(0x7f9db406cef0 legacy=0x7f9db40abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:41.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.768+0000 7f9db2d9d640 1 -- 172.21.15.105:0/3018256995 >> v1:172.21.15.105:6789/0 conn(0x7f9db40cc760 legacy=0x7f9db406c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:41.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.768+0000 7f9db2d9d640 1 -- 172.21.15.105:0/3018256995 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db40cd220 con 0x7f9db4159610 2023-11-28T18:31:41.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.772+0000 7f9db2d9d640 1 --2- 172.21.15.105:0/3018256995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9db4159610 0x7f9db40ce2c0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f9da40097c0 tx=0x7f9db40acca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923f06100654522e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:41.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.772+0000 7f9db259c640 1 -- 172.21.15.105:0/3018256995 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da40ab600 con 0x7f9db4159610 2023-11-28T18:31:41.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.772+0000 7f9db259c640 1 -- 172.21.15.105:0/3018256995 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da40a7050 con 0x7f9db4159610 2023-11-28T18:31:41.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.772+0000 7f9db259c640 1 -- 172.21.15.105:0/3018256995 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da40ab980 con 0x7f9db4159610 2023-11-28T18:31:41.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.772+0000 7f9db9157640 1 -- 172.21.15.105:0/3018256995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9db4159610 msgr2=0x7f9db40ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:41.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.772+0000 7f9db9157640 1 --2- 172.21.15.105:0/3018256995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9db4159610 0x7f9db40ce2c0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f9da40097c0 tx=0x7f9db40acca0 comp rx=0 tx=0).stop 2023-11-28T18:31:41.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.772+0000 7f9db9157640 1 -- 172.21.15.105:0/3018256995 shutdown_connections 2023-11-28T18:31:41.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.772+0000 7f9db9157640 1 --2- 172.21.15.105:0/3018256995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9db4159610 0x7f9db40ce2c0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:41.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.772+0000 7f9db9157640 1 -- 172.21.15.105:0/3018256995 >> 172.21.15.105:0/3018256995 conn(0x7f9db40ceb90 msgr2=0x7f9db40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:41.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.772+0000 7f9db9157640 1 -- 172.21.15.105:0/3018256995 shutdown_connections 2023-11-28T18:31:41.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.772+0000 7f9db9157640 1 -- 172.21.15.105:0/3018256995 wait complete. 2023-11-28T18:31:41.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.772+0000 7f9db9157640 1 Processor -- start 2023-11-28T18:31:41.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.772+0000 7f9db9157640 1 -- start start 2023-11-28T18:31:41.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9db9157640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9db406cef0 0x7f9db4136730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:41.782 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9db9157640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9db40cc760 0x7f9db415bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:41.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9db9157640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9db4159610 0x7f9db414d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:41.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9db9157640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db4147cb0 con 0x7f9db40cc760 2023-11-28T18:31:41.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9db9157640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db4148550 con 0x7f9db406cef0 2023-11-28T18:31:41.783 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9db9157640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db415c990 con 0x7f9db4159610 2023-11-28T18:31:41.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9daa59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9db40cc760 0x7f9db415bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:41.785 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9daa59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9db40cc760 0x7f9db415bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49992/0 (socket says 172.21.15.105:49992) 2023-11-28T18:31:41.785 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9daa59c640 1 -- 172.21.15.105:0/1948995665 learned_addr learned my addr 172.21.15.105:0/1948995665 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:41.785 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9db2d9d640 1 --2- 172.21.15.105:0/1948995665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9db406cef0 0x7f9db4136730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:41.785 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9db359e640 1 --2- 172.21.15.105:0/1948995665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9db4159610 0x7f9db414d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:41.785 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9daa59c640 1 -- 172.21.15.105:0/1948995665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9db4159610 msgr2=0x7f9db414d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:41.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9daa59c640 1 --2- 172.21.15.105:0/1948995665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9db4159610 0x7f9db414d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:41.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9daa59c640 1 -- 172.21.15.105:0/1948995665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9db406cef0 msgr2=0x7f9db4136730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:41.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9daa59c640 1 --2- 172.21.15.105:0/1948995665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9db406cef0 0x7f9db4136730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:41.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9daa59c640 1 -- 172.21.15.105:0/1948995665 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da4009420 con 0x7f9db40cc760 2023-11-28T18:31:41.786 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9daa59c640 1 --2- 172.21.15.105:0/1948995665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9db40cc760 0x7f9db415bfd0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00c530 tx=0x7f9d9c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:41.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9dabfff640 1 -- 172.21.15.105:0/1948995665 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c005cf0 con 0x7f9db40cc760 2023-11-28T18:31:41.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9dabfff640 1 -- 172.21.15.105:0/1948995665 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9c005e50 con 0x7f9db40cc760 2023-11-28T18:31:41.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9db9157640 1 -- 172.21.15.105:0/1948995665 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db4147f00 con 0x7f9db40cc760 2023-11-28T18:31:41.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9dabfff640 1 -- 172.21.15.105:0/1948995665 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c007270 con 0x7f9db40cc760 2023-11-28T18:31:41.787 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9db9157640 1 -- 172.21.15.105:0/1948995665 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db415c510 con 0x7f9db40cc760 2023-11-28T18:31:41.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.776+0000 7f9db9157640 1 -- 172.21.15.105:0/1948995665 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db40af390 con 0x7f9db40cc760 2023-11-28T18:31:41.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.780+0000 7f9dabfff640 1 -- 172.21.15.105:0/1948995665 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f9d9c0073d0 con 0x7f9db40cc760 2023-11-28T18:31:41.788 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.780+0000 7f9dabfff640 1 --2- 172.21.15.105:0/1948995665 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d940668d0 0x7f9d94068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:41.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.780+0000 7f9dabfff640 1 -- 172.21.15.105:0/1948995665 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9d9c0838b0 con 0x7f9db40cc760 2023-11-28T18:31:41.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.780+0000 7f9db2d9d640 1 --2- 172.21.15.105:0/1948995665 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d940668d0 0x7f9d94068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:41.789 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.780+0000 7f9db2d9d640 1 --2- 172.21.15.105:0/1948995665 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d940668d0 0x7f9d94068d90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9da40ad000 tx=0x7f9da4004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:41.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.784+0000 7f9dabfff640 1 -- 172.21.15.105:0/1948995665 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9d9c050af0 con 0x7f9db40cc760 2023-11-28T18:31:41.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.964+0000 7f9db9157640 1 -- 172.21.15.105:0/1948995665 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9db4067570 con 0x7f9db40cc760 2023-11-28T18:31:41.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.964+0000 7f9dabfff640 1 -- 172.21.15.105:0/1948995665 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+4960 (secure 0 0 0) 0x7f9d9c017020 con 0x7f9db40cc760 2023-11-28T18:31:41.965 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:41.965 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":126,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/2073622668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2073622668},{"type":"v1","addr":"172.21.15.148:6834","nonce":2073622668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:31:03.120082+0000","modified":"2023-11-28T18:31:32.799694+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8643,"mds_1":8640,"mds_2":7085},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7085":{"gid":7085,"name":"d","rank":2,"incarnation":123,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6835/270170970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":270170970},{"type":"v1","addr":"172.21.15.148:6835","nonce":270170970}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8640":{"gid":8640,"name":"c","rank":1,"incarnation":121,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6836/3373504883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3373504883},{"type":"v1","addr":"172.21.15.105:6836","nonce":3373504883}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8643":{"gid":8643,"name":"a","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/764864050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":764864050},{"type":"v1","addr":"172.21.15.105:6837","nonce":764864050}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-28T18:31:41.965 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 126 2023-11-28T18:31:41.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.964+0000 7f9db9157640 1 -- 172.21.15.105:0/1948995665 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d940668d0 msgr2=0x7f9d94068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:41.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.964+0000 7f9db9157640 1 --2- 172.21.15.105:0/1948995665 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d940668d0 0x7f9d94068d90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9da40ad000 tx=0x7f9da4004740 comp rx=0 tx=0).stop 2023-11-28T18:31:41.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.964+0000 7f9db9157640 1 -- 172.21.15.105:0/1948995665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9db40cc760 msgr2=0x7f9db415bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:41.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.964+0000 7f9db9157640 1 --2- 172.21.15.105:0/1948995665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9db40cc760 0x7f9db415bfd0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00c530 tx=0x7f9d9c00c9f0 comp rx=0 tx=0).stop 2023-11-28T18:31:41.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.964+0000 7f9db9157640 1 -- 172.21.15.105:0/1948995665 shutdown_connections 2023-11-28T18:31:41.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.964+0000 7f9db9157640 1 --2- 172.21.15.105:0/1948995665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9db4159610 0x7f9db414d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:41.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.964+0000 7f9db9157640 1 --2- 172.21.15.105:0/1948995665 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d940668d0 0x7f9d94068d90 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:41.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.964+0000 7f9db9157640 1 --2- 172.21.15.105:0/1948995665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9db40cc760 0x7f9db415bfd0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:41.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.964+0000 7f9db9157640 1 --2- 172.21.15.105:0/1948995665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9db406cef0 0x7f9db4136730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:41.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.964+0000 7f9db9157640 1 -- 172.21.15.105:0/1948995665 >> 172.21.15.105:0/1948995665 conn(0x7f9db40ceb90 msgr2=0x7f9db40aec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:41.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.968+0000 7f9db9157640 1 -- 172.21.15.105:0/1948995665 shutdown_connections 2023-11-28T18:31:41.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:41.968+0000 7f9db9157640 1 -- 172.21.15.105:0/1948995665 wait complete. 2023-11-28T18:31:41.986 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:31:42.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.124+0000 7f0b7cbf8640 1 Processor -- start 2023-11-28T18:31:42.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.124+0000 7f0b7cbf8640 1 -- start start 2023-11-28T18:31:42.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.124+0000 7f0b7cbf8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0b7806d730 0x7f0b7806db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:42.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.124+0000 7f0b7cbf8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0b7815a1c0 0x7f0b780cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:42.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.124+0000 7f0b7cbf8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b780c9e30 con 0x7f0b78071e10 2023-11-28T18:31:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.124+0000 7f0b7cbf8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b780aea90 con 0x7f0b7815a1c0 2023-11-28T18:31:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.124+0000 7f0b7cbf8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b780cacf0 con 0x7f0b7806d730 2023-11-28T18:31:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.124+0000 7f0b76d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0b7815a1c0 0x7f0b780cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.124+0000 7f0b76d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0b7815a1c0 0x7f0b780cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50000/0 (socket says 172.21.15.105:50000) 2023-11-28T18:31:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.124+0000 7f0b76d9d640 1 -- 172.21.15.105:0/40952165 learned_addr learned my addr 172.21.15.105:0/40952165 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.124+0000 7f0b7659c640 1 --2- 172.21.15.105:0/40952165 >> v2:172.21.15.148:3300/0 conn(0x7f0b7806d730 0x7f0b7806db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.124+0000 7f0b76d9d640 1 -- 172.21.15.105:0/40952165 >> v2:172.21.15.148:3300/0 conn(0x7f0b7806d730 msgr2=0x7f0b7806db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.124+0000 7f0b76d9d640 1 --2- 172.21.15.105:0/40952165 >> v2:172.21.15.148:3300/0 conn(0x7f0b7806d730 0x7f0b7806db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.124+0000 7f0b76d9d640 1 -- 172.21.15.105:0/40952165 >> v1:172.21.15.148:6789/0 conn(0x7f0b78071e10 legacy=0x7f0b780c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.124+0000 7f0b76d9d640 1 -- 172.21.15.105:0/40952165 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b780ca0e0 con 0x7f0b7815a1c0 2023-11-28T18:31:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.128+0000 7f0b76d9d640 1 --2- 172.21.15.105:0/40952165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b7815a1c0 0x7f0b780cd300 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f0b780cced0 tx=0x7f0b7001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2146f6696e9b9ac8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.128+0000 7f0b75d9b640 1 -- 172.21.15.105:0/40952165 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b700aa5f0 con 0x7f0b7815a1c0 2023-11-28T18:31:42.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.128+0000 7f0b75d9b640 1 -- 172.21.15.105:0/40952165 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b700aa750 con 0x7f0b7815a1c0 2023-11-28T18:31:42.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.128+0000 7f0b75d9b640 1 -- 172.21.15.105:0/40952165 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b700aaae0 con 0x7f0b7815a1c0 2023-11-28T18:31:42.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.128+0000 7f0b7cbf8640 1 -- 172.21.15.105:0/40952165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b7815a1c0 msgr2=0x7f0b780cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:42.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.128+0000 7f0b7cbf8640 1 --2- 172.21.15.105:0/40952165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b7815a1c0 0x7f0b780cd300 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f0b780cced0 tx=0x7f0b7001c490 comp rx=0 tx=0).stop 2023-11-28T18:31:42.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.128+0000 7f0b7cbf8640 1 -- 172.21.15.105:0/40952165 shutdown_connections 2023-11-28T18:31:42.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.128+0000 7f0b7cbf8640 1 --2- 172.21.15.105:0/40952165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b7815a1c0 0x7f0b780cd300 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.128+0000 7f0b7cbf8640 1 --2- 172.21.15.105:0/40952165 >> v2:172.21.15.148:3300/0 conn(0x7f0b7806d730 0x7f0b7806db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.128+0000 7f0b7cbf8640 1 -- 172.21.15.105:0/40952165 >> 172.21.15.105:0/40952165 conn(0x7f0b780b0c60 msgr2=0x7f0b780b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:42.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.128+0000 7f0b7cbf8640 1 -- 172.21.15.105:0/40952165 shutdown_connections 2023-11-28T18:31:42.133 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.128+0000 7f0b7cbf8640 1 -- 172.21.15.105:0/40952165 wait complete. 2023-11-28T18:31:42.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b7cbf8640 1 Processor -- start 2023-11-28T18:31:42.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b7cbf8640 1 -- start start 2023-11-28T18:31:42.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b7cbf8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b7806d730 0x7f0b780c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:42.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b7cbf8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b78071e10 0x7f0b78153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:42.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b7cbf8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0b7815a1c0 0x7f0b7815a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:42.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b7cbf8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b780c5930 con 0x7f0b78071e10 2023-11-28T18:31:42.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b7cbf8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b78156390 con 0x7f0b7806d730 2023-11-28T18:31:42.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b7cbf8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b78156670 con 0x7f0b7815a1c0 2023-11-28T18:31:42.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b6dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b78071e10 0x7f0b78153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:42.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b6dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b78071e10 0x7f0b78153ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50014/0 (socket says 172.21.15.105:50014) 2023-11-28T18:31:42.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b6dd9b640 1 -- 172.21.15.105:0/62193845 learned_addr learned my addr 172.21.15.105:0/62193845 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:42.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b7659c640 1 --2- 172.21.15.105:0/62193845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b7806d730 0x7f0b780c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:42.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b76d9d640 1 --2- 172.21.15.105:0/62193845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0b7815a1c0 0x7f0b7815a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:42.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b6dd9b640 1 -- 172.21.15.105:0/62193845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0b7815a1c0 msgr2=0x7f0b7815a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:42.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b6dd9b640 1 --2- 172.21.15.105:0/62193845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0b7815a1c0 0x7f0b7815a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b6dd9b640 1 -- 172.21.15.105:0/62193845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b7806d730 msgr2=0x7f0b780c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:42.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b6dd9b640 1 --2- 172.21.15.105:0/62193845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b7806d730 0x7f0b780c53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b6dd9b640 1 -- 172.21.15.105:0/62193845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b700094d0 con 0x7f0b78071e10 2023-11-28T18:31:42.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b76d9d640 1 --2- 172.21.15.105:0/62193845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0b7815a1c0 0x7f0b7815a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:42.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b6dd9b640 1 --2- 172.21.15.105:0/62193845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b78071e10 0x7f0b78153ad0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000c450 tx=0x7f0b6000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:42.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b6f7fe640 1 -- 172.21.15.105:0/62193845 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60012830 con 0x7f0b78071e10 2023-11-28T18:31:42.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b6f7fe640 1 -- 172.21.15.105:0/62193845 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b600129b0 con 0x7f0b78071e10 2023-11-28T18:31:42.138 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b7cbf8640 1 -- 172.21.15.105:0/62193845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b78156950 con 0x7f0b78071e10 2023-11-28T18:31:42.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b6f7fe640 1 -- 172.21.15.105:0/62193845 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60012d60 con 0x7f0b78071e10 2023-11-28T18:31:42.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.132+0000 7f0b7cbf8640 1 -- 172.21.15.105:0/62193845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b78154010 con 0x7f0b78071e10 2023-11-28T18:31:42.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.136+0000 7f0b7cbf8640 1 -- 172.21.15.105:0/62193845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b78067570 con 0x7f0b78071e10 2023-11-28T18:31:42.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.136+0000 7f0b6f7fe640 1 -- 172.21.15.105:0/62193845 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f0b6001c460 con 0x7f0b78071e10 2023-11-28T18:31:42.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.136+0000 7f0b6f7fe640 1 --2- 172.21.15.105:0/62193845 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0b580668d0 0x7f0b58068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:42.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.136+0000 7f0b6f7fe640 1 -- 172.21.15.105:0/62193845 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0b60084450 con 0x7f0b78071e10 2023-11-28T18:31:42.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.136+0000 7f0b7659c640 1 --2- 172.21.15.105:0/62193845 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0b580668d0 0x7f0b58068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:42.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.136+0000 7f0b7659c640 1 --2- 172.21.15.105:0/62193845 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0b580668d0 0x7f0b58068d90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0b68003b90 tx=0x7f0b680098d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:42.141 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.140+0000 7f0b6f7fe640 1 -- 172.21.15.105:0/62193845 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0b60051420 con 0x7f0b78071e10 2023-11-28T18:31:42.292 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.292+0000 7f0b7cbf8640 1 -- 172.21.15.105:0/62193845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b78158060 con 0x7f0b78071e10 2023-11-28T18:31:42.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.292+0000 7f0b6f7fe640 1 -- 172.21.15.105:0/62193845 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16844 (secure 0 0 0) 0x7f0b60054a40 con 0x7f0b78071e10 2023-11-28T18:31:42.294 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:42.294 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":117,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:31:04.144749+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:31:00.777530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-28T18:31:02.057361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:31:42.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.296+0000 7f0b7cbf8640 1 -- 172.21.15.105:0/62193845 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0b580668d0 msgr2=0x7f0b58068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:42.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.296+0000 7f0b7cbf8640 1 --2- 172.21.15.105:0/62193845 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0b580668d0 0x7f0b58068d90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0b68003b90 tx=0x7f0b680098d0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.296+0000 7f0b7cbf8640 1 -- 172.21.15.105:0/62193845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b78071e10 msgr2=0x7f0b78153ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:42.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.296+0000 7f0b7cbf8640 1 --2- 172.21.15.105:0/62193845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b78071e10 0x7f0b78153ad0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000c450 tx=0x7f0b6000c910 comp rx=0 tx=0).stop 2023-11-28T18:31:42.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.296+0000 7f0b7cbf8640 1 -- 172.21.15.105:0/62193845 shutdown_connections 2023-11-28T18:31:42.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.296+0000 7f0b7cbf8640 1 --2- 172.21.15.105:0/62193845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0b7815a1c0 0x7f0b7815a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.296+0000 7f0b7cbf8640 1 --2- 172.21.15.105:0/62193845 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0b580668d0 0x7f0b58068d90 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.296+0000 7f0b7cbf8640 1 --2- 172.21.15.105:0/62193845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b78071e10 0x7f0b78153ad0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.296+0000 7f0b7cbf8640 1 --2- 172.21.15.105:0/62193845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b7806d730 0x7f0b780c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.296+0000 7f0b7cbf8640 1 -- 172.21.15.105:0/62193845 >> 172.21.15.105:0/62193845 conn(0x7f0b780b0c60 msgr2=0x7f0b780af680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:42.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.296+0000 7f0b7cbf8640 1 -- 172.21.15.105:0/62193845 shutdown_connections 2023-11-28T18:31:42.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.296+0000 7f0b7cbf8640 1 -- 172.21.15.105:0/62193845 wait complete. 2023-11-28T18:31:42.315 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:31:00.777530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:31:02.057361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:31:42.315 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:31:42.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88b078d640 1 Processor -- start 2023-11-28T18:31:42.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88b078d640 1 -- start start 2023-11-28T18:31:42.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88b078d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f88a815d4f0 0x7f88a80d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:42.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88b078d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a80b25b0 con 0x7f88a80d3c30 2023-11-28T18:31:42.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88b078d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a80cea30 con 0x7f88a80b2d20 2023-11-28T18:31:42.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88b078d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a80b2900 con 0x7f88a815d4f0 2023-11-28T18:31:42.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88add28640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f88a80d3c30 0x7f88a80d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49756/0 (socket says 172.21.15.105:49756) 2023-11-28T18:31:42.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88add28640 1 -- 172.21.15.105:0/736907072 learned_addr learned my addr 172.21.15.105:0/736907072 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:42.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88ae529640 1 --2- 172.21.15.105:0/736907072 >> v2:172.21.15.148:3300/0 conn(0x7f88a815d4f0 0x7f88a80d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:42.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88ad527640 1 -- 172.21.15.105:0/736907072 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1580952052 0 0) 0x7f88a80b25b0 con 0x7f88a80d3c30 2023-11-28T18:31:42.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88ad527640 1 -- 172.21.15.105:0/736907072 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8894003530 con 0x7f88a80d3c30 2023-11-28T18:31:42.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88ad527640 1 -- 172.21.15.105:0/736907072 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 434107920 0 0) 0x7f8894003530 con 0x7f88a80d3c30 2023-11-28T18:31:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88ad527640 1 -- 172.21.15.105:0/736907072 >> v2:172.21.15.148:3300/0 conn(0x7f88a815d4f0 msgr2=0x7f88a80d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88ad527640 1 --2- 172.21.15.105:0/736907072 >> v2:172.21.15.148:3300/0 conn(0x7f88a815d4f0 0x7f88a80d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88ad527640 1 -- 172.21.15.105:0/736907072 >> v1:172.21.15.148:6789/0 conn(0x7f88a80b2d20 legacy=0x7f88a80b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88ad527640 1 -- 172.21.15.105:0/736907072 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a80ced10 con 0x7f88a80d3c30 2023-11-28T18:31:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88ad527640 1 -- 172.21.15.105:0/736907072 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f8898001680 con 0x7f88a80d3c30 2023-11-28T18:31:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88ad527640 1 -- 172.21.15.105:0/736907072 >> v1:172.21.15.105:6789/0 conn(0x7f88a80d3c30 legacy=0x7f88a80d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88ad527640 1 --2- 172.21.15.105:0/736907072 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88940039d0 0x7f8894003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88ad527640 1 --2- 172.21.15.105:0/736907072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88940047d0 0x7f8894004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88ad527640 1 --2- 172.21.15.105:0/736907072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88940051e0 0x7f889400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88ad527640 1 -- 172.21.15.105:0/736907072 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a80b2900 con 0x7f88940047d0 2023-11-28T18:31:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88ad527640 1 -- 172.21.15.105:0/736907072 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889400c130 con 0x7f88940051e0 2023-11-28T18:31:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88ad527640 1 -- 172.21.15.105:0/736907072 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889400c2f0 con 0x7f88940039d0 2023-11-28T18:31:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.456+0000 7f88add28640 1 --2- 172.21.15.105:0/736907072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88940047d0 0x7f8894004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:42.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.460+0000 7f88ae529640 1 --2- 172.21.15.105:0/736907072 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88940039d0 0x7f8894003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:42.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.460+0000 7f88aed2a640 1 --2- 172.21.15.105:0/736907072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88940051e0 0x7f889400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:42.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.460+0000 7f88add28640 1 -- 172.21.15.105:0/736907072 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88940039d0 msgr2=0x7f8894003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:42.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.460+0000 7f88add28640 1 --2- 172.21.15.105:0/736907072 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88940039d0 0x7f8894003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.460+0000 7f88add28640 1 -- 172.21.15.105:0/736907072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88940051e0 msgr2=0x7f889400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:42.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.460+0000 7f88add28640 1 --2- 172.21.15.105:0/736907072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88940051e0 0x7f889400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.460+0000 7f88aed2a640 1 -- 172.21.15.105:0/736907072 reap_dead start 2023-11-28T18:31:42.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.460+0000 7f88add28640 1 -- 172.21.15.105:0/736907072 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8894003530 con 0x7f88940047d0 2023-11-28T18:31:42.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.460+0000 7f88add28640 1 --2- 172.21.15.105:0/736907072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88940047d0 0x7f8894004c50 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f8898001ec0 tx=0x7f889801cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc22c8ac2ee1a6f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:42.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.460+0000 7f88ad527640 1 -- 172.21.15.105:0/736907072 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8898005ae0 con 0x7f88940047d0 2023-11-28T18:31:42.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.460+0000 7f88ad527640 1 -- 172.21.15.105:0/736907072 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8898005c40 con 0x7f88940047d0 2023-11-28T18:31:42.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.460+0000 7f88ad527640 1 -- 172.21.15.105:0/736907072 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88980072b0 con 0x7f88940047d0 2023-11-28T18:31:42.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.460+0000 7f88b078d640 1 -- 172.21.15.105:0/736907072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88940047d0 msgr2=0x7f8894004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:42.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.460+0000 7f88b078d640 1 --2- 172.21.15.105:0/736907072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88940047d0 0x7f8894004c50 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f8898001ec0 tx=0x7f889801cb10 comp rx=0 tx=0).stop 2023-11-28T18:31:42.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88b078d640 1 -- 172.21.15.105:0/736907072 shutdown_connections 2023-11-28T18:31:42.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88b078d640 1 --2- 172.21.15.105:0/736907072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88940047d0 0x7f8894004c50 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88b078d640 1 -- 172.21.15.105:0/736907072 >> 172.21.15.105:0/736907072 conn(0x7f88a80d3030 msgr2=0x7f88a80d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:42.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88b078d640 1 -- 172.21.15.105:0/736907072 shutdown_connections 2023-11-28T18:31:42.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88b078d640 1 -- 172.21.15.105:0/736907072 wait complete. 2023-11-28T18:31:42.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88b078d640 1 Processor -- start 2023-11-28T18:31:42.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88b078d640 1 -- start start 2023-11-28T18:31:42.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88b078d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88940047d0 0x7f88a814eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:42.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88b078d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88a8151960 0x7f88a80cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:42.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88b078d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88a80b1500 0x7f88a80b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:42.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88b078d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a80d0a20 con 0x7f88a8151960 2023-11-28T18:31:42.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88b078d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a80b1ef0 con 0x7f88940047d0 2023-11-28T18:31:42.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88b078d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a814f0e0 con 0x7f88a80b1500 2023-11-28T18:31:42.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88add28640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88a8151960 0x7f88a80cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:42.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88add28640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88a8151960 0x7f88a80cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50038/0 (socket says 172.21.15.105:50038) 2023-11-28T18:31:42.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88add28640 1 -- 172.21.15.105:0/227016947 learned_addr learned my addr 172.21.15.105:0/227016947 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:42.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88aed2a640 1 --2- 172.21.15.105:0/227016947 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88a80b1500 0x7f88a80b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:42.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88ae529640 1 --2- 172.21.15.105:0/227016947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88940047d0 0x7f88a814eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:42.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88add28640 1 -- 172.21.15.105:0/227016947 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88a80b1500 msgr2=0x7f88a80b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:42.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88add28640 1 --2- 172.21.15.105:0/227016947 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88a80b1500 0x7f88a80b19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88add28640 1 -- 172.21.15.105:0/227016947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88940047d0 msgr2=0x7f88a814eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:42.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88add28640 1 --2- 172.21.15.105:0/227016947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88940047d0 0x7f88a814eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88add28640 1 -- 172.21.15.105:0/227016947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a80cb1a0 con 0x7f88a8151960 2023-11-28T18:31:42.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88ae529640 1 --2- 172.21.15.105:0/227016947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88940047d0 0x7f88a814eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:42.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.464+0000 7f88add28640 1 --2- 172.21.15.105:0/227016947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88a8151960 0x7f88a80cbda0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f889801cb50 tx=0x7f8894005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:42.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.468+0000 7f889effd640 1 -- 172.21.15.105:0/227016947 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88980b4070 con 0x7f88a8151960 2023-11-28T18:31:42.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.468+0000 7f889effd640 1 -- 172.21.15.105:0/227016947 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8898001500 con 0x7f88a8151960 2023-11-28T18:31:42.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.468+0000 7f889effd640 1 -- 172.21.15.105:0/227016947 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88980072b0 con 0x7f88a8151960 2023-11-28T18:31:42.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.468+0000 7f88b078d640 1 -- 172.21.15.105:0/227016947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a80ca100 con 0x7f88a8151960 2023-11-28T18:31:42.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.468+0000 7f88b078d640 1 -- 172.21.15.105:0/227016947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a80ca6c0 con 0x7f88a8151960 2023-11-28T18:31:42.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.468+0000 7f88b078d640 1 -- 172.21.15.105:0/227016947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88a8067570 con 0x7f88a8151960 2023-11-28T18:31:42.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.468+0000 7f889effd640 1 -- 172.21.15.105:0/227016947 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8898007410 con 0x7f88a8151960 2023-11-28T18:31:42.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.468+0000 7f889effd640 1 --2- 172.21.15.105:0/227016947 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f888c0668d0 0x7f888c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:42.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.468+0000 7f889effd640 1 -- 172.21.15.105:0/227016947 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f889811dc40 con 0x7f88a8151960 2023-11-28T18:31:42.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.468+0000 7f88ae529640 1 --2- 172.21.15.105:0/227016947 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f888c0668d0 0x7f888c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:42.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.468+0000 7f88ae529640 1 --2- 172.21.15.105:0/227016947 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f888c0668d0 0x7f888c068d90 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009c60 tx=0x7f88a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:42.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.472+0000 7f889effd640 1 -- 172.21.15.105:0/227016947 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f88980eaca0 con 0x7f88a8151960 2023-11-28T18:31:42.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.624+0000 7f88b078d640 1 -- 172.21.15.105:0/227016947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f88a80b36a0 con 0x7f88a8151960 2023-11-28T18:31:42.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.624+0000 7f889effd640 1 -- 172.21.15.105:0/227016947 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) v1 ==== 93+0+10 (secure 0 0 0) 0x7f88980ee2c0 con 0x7f88a8151960 2023-11-28T18:31:42.625 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:31:42.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.624+0000 7f88b078d640 1 -- 172.21.15.105:0/227016947 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f888c0668d0 msgr2=0x7f888c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:42.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.624+0000 7f88b078d640 1 --2- 172.21.15.105:0/227016947 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f888c0668d0 0x7f888c068d90 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009c60 tx=0x7f88a4008040 comp rx=0 tx=0).stop 2023-11-28T18:31:42.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.624+0000 7f88b078d640 1 -- 172.21.15.105:0/227016947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88a8151960 msgr2=0x7f88a80cbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:42.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.624+0000 7f88b078d640 1 --2- 172.21.15.105:0/227016947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88a8151960 0x7f88a80cbda0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f889801cb50 tx=0x7f8894005800 comp rx=0 tx=0).stop 2023-11-28T18:31:42.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.624+0000 7f88b078d640 1 -- 172.21.15.105:0/227016947 shutdown_connections 2023-11-28T18:31:42.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.624+0000 7f88b078d640 1 --2- 172.21.15.105:0/227016947 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f88a80b1500 0x7f88a80b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.624+0000 7f88b078d640 1 --2- 172.21.15.105:0/227016947 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f888c0668d0 0x7f888c068d90 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.624+0000 7f88b078d640 1 --2- 172.21.15.105:0/227016947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88a8151960 0x7f88a80cbda0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.624+0000 7f88b078d640 1 --2- 172.21.15.105:0/227016947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88940047d0 0x7f88a814eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.624+0000 7f88b078d640 1 -- 172.21.15.105:0/227016947 >> 172.21.15.105:0/227016947 conn(0x7f88a80d3030 msgr2=0x7f88a80affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:42.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.624+0000 7f88b078d640 1 -- 172.21.15.105:0/227016947 shutdown_connections 2023-11-28T18:31:42.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.624+0000 7f88b078d640 1 -- 172.21.15.105:0/227016947 wait complete. 2023-11-28T18:31:42.646 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:31:42.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.788+0000 7f822129a640 1 Processor -- start 2023-11-28T18:31:42.790 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.788+0000 7f822129a640 1 -- start start 2023-11-28T18:31:42.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.792+0000 7f822129a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f821c06e4c0 0x7f821c06eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:42.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.792+0000 7f822129a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f821c06c0a0 con 0x7f821c0affb0 2023-11-28T18:31:42.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.792+0000 7f822129a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f821c06c2f0 con 0x7f821c15d390 2023-11-28T18:31:42.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.792+0000 7f822129a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f821c06cc90 con 0x7f821c06e4c0 2023-11-28T18:31:42.791 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.792+0000 7f82127fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f821c06e4c0 0x7f821c06eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:42.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.792+0000 7f82127fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f821c06e4c0 0x7f821c06eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50052/0 (socket says 172.21.15.105:50052) 2023-11-28T18:31:42.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.792+0000 7f82127fc640 1 -- 172.21.15.105:0/2185225376 learned_addr learned my addr 172.21.15.105:0/2185225376 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:42.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.792+0000 7f82127fc640 1 -- 172.21.15.105:0/2185225376 >> v1:172.21.15.148:6789/0 conn(0x7f821c15d390 legacy=0x7f821c06d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:42.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.792+0000 7f82127fc640 1 -- 172.21.15.105:0/2185225376 >> v1:172.21.15.105:6789/0 conn(0x7f821c0affb0 legacy=0x7f821c0adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:42.792 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.792+0000 7f82127fc640 1 -- 172.21.15.105:0/2185225376 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c06c5d0 con 0x7f821c06e4c0 2023-11-28T18:31:42.793 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.792+0000 7f821a7fc640 1 -- 172.21.15.105:0/2185225376 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587060711 0 0) 0x7f821c06c0a0 con 0x7f821c0affb0 2023-11-28T18:31:42.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.796+0000 7f82127fc640 1 --2- 172.21.15.105:0/2185225376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f821c06e4c0 0x7f821c06eb10 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f8204009e50 tx=0x7f820401c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5114be850d7efe9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:42.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.796+0000 7f821a7fc640 1 -- 172.21.15.105:0/2185225376 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82040a57d0 con 0x7f821c06e4c0 2023-11-28T18:31:42.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.796+0000 7f821a7fc640 1 -- 172.21.15.105:0/2185225376 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82040a5930 con 0x7f821c06e4c0 2023-11-28T18:31:42.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.796+0000 7f821a7fc640 1 -- 172.21.15.105:0/2185225376 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82040a5cc0 con 0x7f821c06e4c0 2023-11-28T18:31:42.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.796+0000 7f822129a640 1 -- 172.21.15.105:0/2185225376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f821c06e4c0 msgr2=0x7f821c06eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:42.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.796+0000 7f822129a640 1 --2- 172.21.15.105:0/2185225376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f821c06e4c0 0x7f821c06eb10 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f8204009e50 tx=0x7f820401c8f0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.796+0000 7f822129a640 1 -- 172.21.15.105:0/2185225376 shutdown_connections 2023-11-28T18:31:42.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.796+0000 7f822129a640 1 --2- 172.21.15.105:0/2185225376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f821c06e4c0 0x7f821c06eb10 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.796+0000 7f822129a640 1 -- 172.21.15.105:0/2185225376 >> 172.21.15.105:0/2185225376 conn(0x7f821c0b28d0 msgr2=0x7f821c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:42.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.796+0000 7f822129a640 1 -- 172.21.15.105:0/2185225376 shutdown_connections 2023-11-28T18:31:42.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.796+0000 7f822129a640 1 -- 172.21.15.105:0/2185225376 wait complete. 2023-11-28T18:31:42.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.796+0000 7f822129a640 1 Processor -- start 2023-11-28T18:31:42.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.796+0000 7f822129a640 1 -- start start 2023-11-28T18:31:42.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f822129a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f821c06e4c0 0x7f821c151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:42.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f822129a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f821c0affb0 0x7f821c14c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:42.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f822129a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f821c15d390 0x7f821c152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:42.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f822129a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c154d50 con 0x7f821c0affb0 2023-11-28T18:31:42.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f822129a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c14cac0 con 0x7f821c06e4c0 2023-11-28T18:31:42.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f822129a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821c154a00 con 0x7f821c15d390 2023-11-28T18:31:42.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f82127fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f821c0affb0 0x7f821c14c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:42.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f82127fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f821c0affb0 0x7f821c14c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50054/0 (socket says 172.21.15.105:50054) 2023-11-28T18:31:42.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f82127fc640 1 -- 172.21.15.105:0/3498152687 learned_addr learned my addr 172.21.15.105:0/3498152687 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:42.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f821affd640 1 --2- 172.21.15.105:0/3498152687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f821c06e4c0 0x7f821c151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:42.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f821b7fe640 1 --2- 172.21.15.105:0/3498152687 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f821c15d390 0x7f821c152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:42.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f82127fc640 1 -- 172.21.15.105:0/3498152687 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f821c15d390 msgr2=0x7f821c152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:42.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f82127fc640 1 --2- 172.21.15.105:0/3498152687 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f821c15d390 0x7f821c152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f82127fc640 1 -- 172.21.15.105:0/3498152687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f821c06e4c0 msgr2=0x7f821c151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:42.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f82127fc640 1 --2- 172.21.15.105:0/3498152687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f821c06e4c0 0x7f821c151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f82127fc640 1 -- 172.21.15.105:0/3498152687 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8204009b30 con 0x7f821c0affb0 2023-11-28T18:31:42.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f821affd640 1 --2- 172.21.15.105:0/3498152687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f821c06e4c0 0x7f821c151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:42.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f82127fc640 1 --2- 172.21.15.105:0/3498152687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f821c0affb0 0x7f821c14c580 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f8204019500 tx=0x7f8204019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:42.801 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f8213fff640 1 -- 172.21.15.105:0/3498152687 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82040b8670 con 0x7f821c0affb0 2023-11-28T18:31:42.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f8213fff640 1 -- 172.21.15.105:0/3498152687 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82040b87d0 con 0x7f821c0affb0 2023-11-28T18:31:42.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f8213fff640 1 -- 172.21.15.105:0/3498152687 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82040b8b60 con 0x7f821c0affb0 2023-11-28T18:31:42.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f822129a640 1 -- 172.21.15.105:0/3498152687 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f821c14cc50 con 0x7f821c0affb0 2023-11-28T18:31:42.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f822129a640 1 -- 172.21.15.105:0/3498152687 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821c14d210 con 0x7f821c0affb0 2023-11-28T18:31:42.802 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f822129a640 1 -- 172.21.15.105:0/3498152687 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f821c074580 con 0x7f821c0affb0 2023-11-28T18:31:42.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.800+0000 7f8213fff640 1 -- 172.21.15.105:0/3498152687 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f82040b8cc0 con 0x7f821c0affb0 2023-11-28T18:31:42.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.804+0000 7f8213fff640 1 --2- 172.21.15.105:0/3498152687 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f81fc0668d0 0x7f81fc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:42.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.804+0000 7f8213fff640 1 -- 172.21.15.105:0/3498152687 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f820411fa30 con 0x7f821c0affb0 2023-11-28T18:31:42.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.804+0000 7f821affd640 1 --2- 172.21.15.105:0/3498152687 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f81fc0668d0 0x7f81fc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:42.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.804+0000 7f821affd640 1 --2- 172.21.15.105:0/3498152687 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f81fc0668d0 0x7f81fc068d90 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f820c000f70 tx=0x7f820c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:42.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.808+0000 7f8213fff640 1 -- 172.21.15.105:0/3498152687 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f82040ecc40 con 0x7f821c0affb0 2023-11-28T18:31:42.955 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.956+0000 7f822129a640 1 -- 172.21.15.105:0/3498152687 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f821c14d3a0 con 0x7f821c0affb0 2023-11-28T18:31:42.956 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.956+0000 7f8213fff640 1 -- 172.21.15.105:0/3498152687 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v117) v1 ==== 104+0+11 (secure 0 0 0) 0x7f82040f0260 con 0x7f821c0affb0 2023-11-28T18:31:42.956 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:31:42.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.956+0000 7f822129a640 1 -- 172.21.15.105:0/3498152687 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f81fc0668d0 msgr2=0x7f81fc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:42.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.956+0000 7f822129a640 1 --2- 172.21.15.105:0/3498152687 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f81fc0668d0 0x7f81fc068d90 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f820c000f70 tx=0x7f820c009040 comp rx=0 tx=0).stop 2023-11-28T18:31:42.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.956+0000 7f822129a640 1 -- 172.21.15.105:0/3498152687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f821c0affb0 msgr2=0x7f821c14c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:42.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.956+0000 7f822129a640 1 --2- 172.21.15.105:0/3498152687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f821c0affb0 0x7f821c14c580 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f8204019500 tx=0x7f8204019690 comp rx=0 tx=0).stop 2023-11-28T18:31:42.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.956+0000 7f822129a640 1 -- 172.21.15.105:0/3498152687 shutdown_connections 2023-11-28T18:31:42.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.956+0000 7f822129a640 1 --2- 172.21.15.105:0/3498152687 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f821c15d390 0x7f821c152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.956+0000 7f822129a640 1 --2- 172.21.15.105:0/3498152687 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f81fc0668d0 0x7f81fc068d90 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.956+0000 7f822129a640 1 --2- 172.21.15.105:0/3498152687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f821c0affb0 0x7f821c14c580 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.956+0000 7f822129a640 1 --2- 172.21.15.105:0/3498152687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f821c06e4c0 0x7f821c151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:42.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.956+0000 7f822129a640 1 -- 172.21.15.105:0/3498152687 >> 172.21.15.105:0/3498152687 conn(0x7f821c0b28d0 msgr2=0x7f821c156a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:42.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.956+0000 7f822129a640 1 -- 172.21.15.105:0/3498152687 shutdown_connections 2023-11-28T18:31:42.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:42.956+0000 7f822129a640 1 -- 172.21.15.105:0/3498152687 wait complete. 2023-11-28T18:31:42.976 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:31:43.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.116+0000 7f55c4a9c640 1 Processor -- start 2023-11-28T18:31:43.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.116+0000 7f55c4a9c640 1 -- start start 2023-11-28T18:31:43.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.120+0000 7f55c4a9c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55c00b4290 0x7f55c00b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:43.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.120+0000 7f55c4a9c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f55c0071960 0x7f55c015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:43.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.120+0000 7f55c4a9c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f55c015b030 0x7f55c00709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:43.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.120+0000 7f55c4a9c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c00b4bb0 con 0x7f55c015b030 2023-11-28T18:31:43.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.120+0000 7f55c4a9c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c00701d0 con 0x7f55c0071960 2023-11-28T18:31:43.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.120+0000 7f55c4a9c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c006df80 con 0x7f55c00b4290 2023-11-28T18:31:43.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.120+0000 7f55bed9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f55c015b030 0x7f55c00709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:43.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.120+0000 7f55bed9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f55c015b030 0x7f55c00709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50056/0 (socket says 172.21.15.105:50056) 2023-11-28T18:31:43.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.120+0000 7f55bed9d640 1 -- 172.21.15.105:0/4005090281 learned_addr learned my addr 172.21.15.105:0/4005090281 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:43.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.120+0000 7f55be59c640 1 --2- 172.21.15.105:0/4005090281 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55c00b4290 0x7f55c00b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:43.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.120+0000 7f55bdd9b640 1 --2- 172.21.15.105:0/4005090281 >> v2:172.21.15.148:3300/0 conn(0x7f55c0071960 0x7f55c015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:43.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.120+0000 7f55bed9d640 1 -- 172.21.15.105:0/4005090281 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55c00b4290 msgr2=0x7f55c00b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:43.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.120+0000 7f55bed9d640 1 --2- 172.21.15.105:0/4005090281 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55c00b4290 0x7f55c00b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.120+0000 7f55bed9d640 1 -- 172.21.15.105:0/4005090281 >> v2:172.21.15.148:3300/0 conn(0x7f55c0071960 msgr2=0x7f55c015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:43.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.120+0000 7f55bed9d640 1 --2- 172.21.15.105:0/4005090281 >> v2:172.21.15.148:3300/0 conn(0x7f55c0071960 0x7f55c015f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.120+0000 7f55bed9d640 1 -- 172.21.15.105:0/4005090281 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c0071df0 con 0x7f55c015b030 2023-11-28T18:31:43.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.120+0000 7f55be59c640 1 --2- 172.21.15.105:0/4005090281 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55c00b4290 0x7f55c00b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:43.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.124+0000 7f55bed9d640 1 --2- 172.21.15.105:0/4005090281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55c015b030 0x7f55c00709b0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f55c00cc710 tx=0x7f55b801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e568dcd7f9561be4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:43.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.124+0000 7f55bd59a640 1 -- 172.21.15.105:0/4005090281 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b80a5730 con 0x7f55c015b030 2023-11-28T18:31:43.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.124+0000 7f55bd59a640 1 -- 172.21.15.105:0/4005090281 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b80a5890 con 0x7f55c015b030 2023-11-28T18:31:43.123 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.124+0000 7f55bd59a640 1 -- 172.21.15.105:0/4005090281 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b80a5c20 con 0x7f55c015b030 2023-11-28T18:31:43.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.124+0000 7f55c4a9c640 1 -- 172.21.15.105:0/4005090281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55c015b030 msgr2=0x7f55c00709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:43.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.124+0000 7f55c4a9c640 1 --2- 172.21.15.105:0/4005090281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55c015b030 0x7f55c00709b0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f55c00cc710 tx=0x7f55b801c9a0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.124+0000 7f55c4a9c640 1 -- 172.21.15.105:0/4005090281 shutdown_connections 2023-11-28T18:31:43.124 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.124+0000 7f55c4a9c640 1 --2- 172.21.15.105:0/4005090281 >> v2:172.21.15.148:3300/0 conn(0x7f55c0071960 0x7f55c015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.124+0000 7f55c4a9c640 1 --2- 172.21.15.105:0/4005090281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55c015b030 0x7f55c00709b0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.124+0000 7f55c4a9c640 1 --2- 172.21.15.105:0/4005090281 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55c00b4290 0x7f55c00b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.124+0000 7f55c4a9c640 1 -- 172.21.15.105:0/4005090281 >> 172.21.15.105:0/4005090281 conn(0x7f55c00ceb90 msgr2=0x7f55c00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:43.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.124+0000 7f55c4a9c640 1 -- 172.21.15.105:0/4005090281 shutdown_connections 2023-11-28T18:31:43.125 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.124+0000 7f55c4a9c640 1 -- 172.21.15.105:0/4005090281 wait complete. 2023-11-28T18:31:43.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.124+0000 7f55c4a9c640 1 Processor -- start 2023-11-28T18:31:43.126 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.124+0000 7f55c4a9c640 1 -- start start 2023-11-28T18:31:43.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55c4a9c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55c00b4290 0x7f55c01489e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:43.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55c4a9c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55c015b030 0x7f55c0099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:43.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55c4a9c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55c00b2d90 0x7f55c014a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:43.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55c4a9c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c014d380 con 0x7f55c00b2d90 2023-11-28T18:31:43.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55c4a9c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c014d080 con 0x7f55c015b030 2023-11-28T18:31:43.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55c4a9c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c0148f20 con 0x7f55c00b4290 2023-11-28T18:31:43.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55bed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55c00b2d90 0x7f55c014a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:43.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55bed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55c00b2d90 0x7f55c014a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50062/0 (socket says 172.21.15.105:50062) 2023-11-28T18:31:43.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55bed9d640 1 -- 172.21.15.105:0/1576809035 learned_addr learned my addr 172.21.15.105:0/1576809035 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:43.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55be59c640 1 --2- 172.21.15.105:0/1576809035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55c00b4290 0x7f55c01489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:43.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55bdd9b640 1 --2- 172.21.15.105:0/1576809035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55c015b030 0x7f55c0099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:43.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55bed9d640 1 -- 172.21.15.105:0/1576809035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55c00b4290 msgr2=0x7f55c01489e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:43.129 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55bed9d640 1 --2- 172.21.15.105:0/1576809035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55c00b4290 0x7f55c01489e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55bed9d640 1 -- 172.21.15.105:0/1576809035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55c015b030 msgr2=0x7f55c0099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:43.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55bed9d640 1 --2- 172.21.15.105:0/1576809035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55c015b030 0x7f55c0099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55bed9d640 1 -- 172.21.15.105:0/1576809035 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b8009b30 con 0x7f55c00b2d90 2023-11-28T18:31:43.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55bdd9b640 1 --2- 172.21.15.105:0/1576809035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55c015b030 0x7f55c0099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:43.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55bed9d640 1 --2- 172.21.15.105:0/1576809035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55c00b2d90 0x7f55c014a270 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f55b801b050 tx=0x7f55b8019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:43.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55aeffd640 1 -- 172.21.15.105:0/1576809035 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b80b8670 con 0x7f55c00b2d90 2023-11-28T18:31:43.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55aeffd640 1 -- 172.21.15.105:0/1576809035 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b80b87d0 con 0x7f55c00b2d90 2023-11-28T18:31:43.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55aeffd640 1 -- 172.21.15.105:0/1576809035 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b80b8b60 con 0x7f55c00b2d90 2023-11-28T18:31:43.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55c4a9c640 1 -- 172.21.15.105:0/1576809035 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c00b3290 con 0x7f55c00b2d90 2023-11-28T18:31:43.130 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55c4a9c640 1 -- 172.21.15.105:0/1576809035 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c0099f90 con 0x7f55c00b2d90 2023-11-28T18:31:43.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55c4a9c640 1 -- 172.21.15.105:0/1576809035 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c014cee0 con 0x7f55c00b2d90 2023-11-28T18:31:43.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.128+0000 7f55aeffd640 1 -- 172.21.15.105:0/1576809035 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f55b80b8cc0 con 0x7f55c00b2d90 2023-11-28T18:31:43.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.132+0000 7f55aeffd640 1 --2- 172.21.15.105:0/1576809035 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f55a0066900 0x7f55a0068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:43.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.132+0000 7f55aeffd640 1 -- 172.21.15.105:0/1576809035 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f55b811f5c0 con 0x7f55c00b2d90 2023-11-28T18:31:43.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.132+0000 7f55be59c640 1 --2- 172.21.15.105:0/1576809035 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f55a0066900 0x7f55a0068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:43.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.132+0000 7f55be59c640 1 --2- 172.21.15.105:0/1576809035 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f55a0066900 0x7f55a0068dc0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f55c0151650 tx=0x7f55b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:43.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.136+0000 7f55aeffd640 1 -- 172.21.15.105:0/1576809035 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f55c014cee0 con 0x7f55c00b2d90 2023-11-28T18:31:43.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.288+0000 7f55c4a9c640 1 -- 172.21.15.105:0/1576809035 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f55c014cee0 con 0x7f55c00b2d90 2023-11-28T18:31:43.290 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.288+0000 7f55aeffd640 1 -- 172.21.15.105:0/1576809035 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v117) v1 ==== 100+0+11 (secure 0 0 0) 0x7f55c014cee0 con 0x7f55c00b2d90 2023-11-28T18:31:43.291 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:31:43.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.292+0000 7f55c4a9c640 1 -- 172.21.15.105:0/1576809035 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f55a0066900 msgr2=0x7f55a0068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:43.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.292+0000 7f55c4a9c640 1 --2- 172.21.15.105:0/1576809035 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f55a0066900 0x7f55a0068dc0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f55c0151650 tx=0x7f55b0008040 comp rx=0 tx=0).stop 2023-11-28T18:31:43.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.292+0000 7f55c4a9c640 1 -- 172.21.15.105:0/1576809035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55c00b2d90 msgr2=0x7f55c014a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:43.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.292+0000 7f55c4a9c640 1 --2- 172.21.15.105:0/1576809035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55c00b2d90 0x7f55c014a270 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f55b801b050 tx=0x7f55b8019740 comp rx=0 tx=0).stop 2023-11-28T18:31:43.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.292+0000 7f55c4a9c640 1 -- 172.21.15.105:0/1576809035 shutdown_connections 2023-11-28T18:31:43.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.292+0000 7f55c4a9c640 1 --2- 172.21.15.105:0/1576809035 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f55a0066900 0x7f55a0068dc0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.292+0000 7f55c4a9c640 1 --2- 172.21.15.105:0/1576809035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55c00b2d90 0x7f55c014a270 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.292+0000 7f55c4a9c640 1 --2- 172.21.15.105:0/1576809035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55c015b030 0x7f55c0099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.292+0000 7f55c4a9c640 1 --2- 172.21.15.105:0/1576809035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55c00b4290 0x7f55c01489e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.293 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.292+0000 7f55c4a9c640 1 -- 172.21.15.105:0/1576809035 >> 172.21.15.105:0/1576809035 conn(0x7f55c00ceb90 msgr2=0x7f55c00ae6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:43.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.292+0000 7f55c4a9c640 1 -- 172.21.15.105:0/1576809035 shutdown_connections 2023-11-28T18:31:43.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.292+0000 7f55c4a9c640 1 -- 172.21.15.105:0/1576809035 wait complete. 2023-11-28T18:31:43.312 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:31:43.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.448+0000 7fb318772640 1 Processor -- start 2023-11-28T18:31:43.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.448+0000 7fb318772640 1 -- start start 2023-11-28T18:31:43.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.452+0000 7fb318772640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb310159f40 0x7fb3100caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:43.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.452+0000 7fb318772640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb3100ad850 0x7fb3100701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:43.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.452+0000 7fb318772640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb31006bf50 con 0x7fb3100cc760 2023-11-28T18:31:43.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.452+0000 7fb318772640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3100ccbc0 con 0x7fb3100ad850 2023-11-28T18:31:43.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.452+0000 7fb318772640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb31015a320 con 0x7fb310159f40 2023-11-28T18:31:43.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.452+0000 7fb316d0f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb3100ad850 0x7fb3100701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:43.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.452+0000 7fb316d0f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb3100ad850 0x7fb3100701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50068/0 (socket says 172.21.15.105:50068) 2023-11-28T18:31:43.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.452+0000 7fb316d0f640 1 -- 172.21.15.105:0/1669146715 learned_addr learned my addr 172.21.15.105:0/1669146715 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:43.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.452+0000 7fb31650e640 1 --2- 172.21.15.105:0/1669146715 >> v2:172.21.15.148:3300/0 conn(0x7fb310159f40 0x7fb3100caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:43.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.452+0000 7fb316d0f640 1 -- 172.21.15.105:0/1669146715 >> v2:172.21.15.148:3300/0 conn(0x7fb310159f40 msgr2=0x7fb3100caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:43.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.452+0000 7fb316d0f640 1 --2- 172.21.15.105:0/1669146715 >> v2:172.21.15.148:3300/0 conn(0x7fb310159f40 0x7fb3100caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.452+0000 7fb316d0f640 1 -- 172.21.15.105:0/1669146715 >> v1:172.21.15.105:6789/0 conn(0x7fb3100cc760 legacy=0x7fb3100ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:43.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.452+0000 7fb316d0f640 1 -- 172.21.15.105:0/1669146715 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb310071380 con 0x7fb3100ad850 2023-11-28T18:31:43.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.452+0000 7fb316d0f640 1 --2- 172.21.15.105:0/1669146715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3100ad850 0x7fb3100701d0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fb31006e500 tx=0x7fb30801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f7f49779d998ed server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:43.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.456+0000 7fb31550c640 1 -- 172.21.15.105:0/1669146715 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3080a57d0 con 0x7fb3100ad850 2023-11-28T18:31:43.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.456+0000 7fb31550c640 1 -- 172.21.15.105:0/1669146715 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3080a5930 con 0x7fb3100ad850 2023-11-28T18:31:43.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.456+0000 7fb31550c640 1 -- 172.21.15.105:0/1669146715 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3080a5cc0 con 0x7fb3100ad850 2023-11-28T18:31:43.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.456+0000 7fb318772640 1 -- 172.21.15.105:0/1669146715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3100ad850 msgr2=0x7fb3100701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:43.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.456+0000 7fb318772640 1 --2- 172.21.15.105:0/1669146715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3100ad850 0x7fb3100701d0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fb31006e500 tx=0x7fb30801c910 comp rx=0 tx=0).stop 2023-11-28T18:31:43.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.456+0000 7fb318772640 1 -- 172.21.15.105:0/1669146715 shutdown_connections 2023-11-28T18:31:43.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.456+0000 7fb318772640 1 --2- 172.21.15.105:0/1669146715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3100ad850 0x7fb3100701d0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.456+0000 7fb318772640 1 --2- 172.21.15.105:0/1669146715 >> v2:172.21.15.148:3300/0 conn(0x7fb310159f40 0x7fb3100caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.456+0000 7fb318772640 1 -- 172.21.15.105:0/1669146715 >> 172.21.15.105:0/1669146715 conn(0x7fb3100ceb90 msgr2=0x7fb3100cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:43.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.456+0000 7fb318772640 1 -- 172.21.15.105:0/1669146715 shutdown_connections 2023-11-28T18:31:43.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.456+0000 7fb318772640 1 -- 172.21.15.105:0/1669146715 wait complete. 2023-11-28T18:31:43.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.456+0000 7fb318772640 1 Processor -- start 2023-11-28T18:31:43.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.456+0000 7fb318772640 1 -- start start 2023-11-28T18:31:43.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.456+0000 7fb318772640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3100ad850 0x7fb310148d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:43.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.456+0000 7fb318772640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3100cc760 0x7fb3101494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:43.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.456+0000 7fb318772640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb310159f40 0x7fb3101499f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:43.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.456+0000 7fb318772640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3101492a0 con 0x7fb310159f40 2023-11-28T18:31:43.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.456+0000 7fb318772640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31014be80 con 0x7fb3100cc760 2023-11-28T18:31:43.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.456+0000 7fb318772640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31014a000 con 0x7fb3100ad850 2023-11-28T18:31:43.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.460+0000 7fb316d0f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb310159f40 0x7fb3101499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:43.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.460+0000 7fb316d0f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb310159f40 0x7fb3101499f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50076/0 (socket says 172.21.15.105:50076) 2023-11-28T18:31:43.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.460+0000 7fb31650e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3100ad850 0x7fb310148d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:43.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.460+0000 7fb316d0f640 1 -- 172.21.15.105:0/191381701 learned_addr learned my addr 172.21.15.105:0/191381701 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:43.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.460+0000 7fb315d0d640 1 --2- 172.21.15.105:0/191381701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3100cc760 0x7fb3101494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:43.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.460+0000 7fb316d0f640 1 -- 172.21.15.105:0/191381701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3100ad850 msgr2=0x7fb310148d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:43.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.460+0000 7fb316d0f640 1 --2- 172.21.15.105:0/191381701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3100ad850 0x7fb310148d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.460+0000 7fb316d0f640 1 -- 172.21.15.105:0/191381701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3100cc760 msgr2=0x7fb3101494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:43.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.460+0000 7fb316d0f640 1 --2- 172.21.15.105:0/191381701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3100cc760 0x7fb3101494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.460+0000 7fb316d0f640 1 -- 172.21.15.105:0/191381701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb308009b30 con 0x7fb310159f40 2023-11-28T18:31:43.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.460+0000 7fb316d0f640 1 --2- 172.21.15.105:0/191381701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb310159f40 0x7fb3101499f0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fb3080af040 tx=0x7fb3080196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:43.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.460+0000 7fb306ffd640 1 -- 172.21.15.105:0/191381701 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3080b8670 con 0x7fb310159f40 2023-11-28T18:31:43.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.460+0000 7fb306ffd640 1 -- 172.21.15.105:0/191381701 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3080b87d0 con 0x7fb310159f40 2023-11-28T18:31:43.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.460+0000 7fb306ffd640 1 -- 172.21.15.105:0/191381701 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3080b8b60 con 0x7fb310159f40 2023-11-28T18:31:43.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.460+0000 7fb318772640 1 -- 172.21.15.105:0/191381701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31014a2e0 con 0x7fb310159f40 2023-11-28T18:31:43.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.460+0000 7fb318772640 1 -- 172.21.15.105:0/191381701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb310165cd0 con 0x7fb310159f40 2023-11-28T18:31:43.462 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.460+0000 7fb318772640 1 -- 172.21.15.105:0/191381701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb31014a800 con 0x7fb310159f40 2023-11-28T18:31:43.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.460+0000 7fb306ffd640 1 -- 172.21.15.105:0/191381701 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fb3080b8cc0 con 0x7fb310159f40 2023-11-28T18:31:43.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.464+0000 7fb306ffd640 1 --2- 172.21.15.105:0/191381701 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb2f4066900 0x7fb2f4068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:43.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.464+0000 7fb306ffd640 1 -- 172.21.15.105:0/191381701 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb30811fa30 con 0x7fb310159f40 2023-11-28T18:31:43.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.464+0000 7fb31650e640 1 --2- 172.21.15.105:0/191381701 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb2f4066900 0x7fb2f4068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:43.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.464+0000 7fb31650e640 1 --2- 172.21.15.105:0/191381701 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb2f4066900 0x7fb2f4068dc0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb30c0079c0 tx=0x7fb30c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:43.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.468+0000 7fb306ffd640 1 -- 172.21.15.105:0/191381701 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb31014a800 con 0x7fb310159f40 2023-11-28T18:31:43.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.648+0000 7fb318772640 1 -- 172.21.15.105:0/191381701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb310165b30 con 0x7fb310159f40 2023-11-28T18:31:43.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.648+0000 7fb306ffd640 1 -- 172.21.15.105:0/191381701 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+4960 (secure 0 0 0) 0x7fb310165b30 con 0x7fb310159f40 2023-11-28T18:31:43.649 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:43.649 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":126,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/2073622668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2073622668},{"type":"v1","addr":"172.21.15.148:6834","nonce":2073622668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:31:03.120082+0000","modified":"2023-11-28T18:31:32.799694+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8643,"mds_1":8640,"mds_2":7085},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7085":{"gid":7085,"name":"d","rank":2,"incarnation":123,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6835/270170970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":270170970},{"type":"v1","addr":"172.21.15.148:6835","nonce":270170970}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8640":{"gid":8640,"name":"c","rank":1,"incarnation":121,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6836/3373504883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3373504883},{"type":"v1","addr":"172.21.15.105:6836","nonce":3373504883}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8643":{"gid":8643,"name":"a","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/764864050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":764864050},{"type":"v1","addr":"172.21.15.105:6837","nonce":764864050}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-28T18:31:43.649 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 126 2023-11-28T18:31:43.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.648+0000 7fb318772640 1 -- 172.21.15.105:0/191381701 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb2f4066900 msgr2=0x7fb2f4068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:43.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.648+0000 7fb318772640 1 --2- 172.21.15.105:0/191381701 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb2f4066900 0x7fb2f4068dc0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb30c0079c0 tx=0x7fb30c008040 comp rx=0 tx=0).stop 2023-11-28T18:31:43.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.648+0000 7fb318772640 1 -- 172.21.15.105:0/191381701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb310159f40 msgr2=0x7fb3101499f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:43.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.648+0000 7fb318772640 1 --2- 172.21.15.105:0/191381701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb310159f40 0x7fb3101499f0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fb3080af040 tx=0x7fb3080196b0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.652+0000 7fb318772640 1 -- 172.21.15.105:0/191381701 shutdown_connections 2023-11-28T18:31:43.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.652+0000 7fb318772640 1 --2- 172.21.15.105:0/191381701 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fb2f4066900 0x7fb2f4068dc0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.652+0000 7fb318772640 1 --2- 172.21.15.105:0/191381701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb310159f40 0x7fb3101499f0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.652+0000 7fb318772640 1 --2- 172.21.15.105:0/191381701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3100cc760 0x7fb3101494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.652+0000 7fb318772640 1 --2- 172.21.15.105:0/191381701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3100ad850 0x7fb310148d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.651 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.652+0000 7fb318772640 1 -- 172.21.15.105:0/191381701 >> 172.21.15.105:0/191381701 conn(0x7fb3100ceb90 msgr2=0x7fb3100ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:43.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.652+0000 7fb318772640 1 -- 172.21.15.105:0/191381701 shutdown_connections 2023-11-28T18:31:43.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.652+0000 7fb318772640 1 -- 172.21.15.105:0/191381701 wait complete. 2023-11-28T18:31:43.670 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:31:43.808 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.808+0000 7f1091b2a640 1 Processor -- start 2023-11-28T18:31:43.809 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.808+0000 7f1091b2a640 1 -- start start 2023-11-28T18:31:43.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.808+0000 7f1091b2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f108c15b030 0x7f108c0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:43.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.808+0000 7f1091b2a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108c0701d0 con 0x7f108c071960 2023-11-28T18:31:43.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.808+0000 7f1091b2a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108c06df80 con 0x7f108c0b4290 2023-11-28T18:31:43.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.808+0000 7f1091b2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108c071df0 con 0x7f108c15b030 2023-11-28T18:31:43.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.808+0000 7f108affd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f108c071960 0x7f108c15f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49784/0 (socket says 172.21.15.105:49784) 2023-11-28T18:31:43.810 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.808+0000 7f108affd640 1 -- 172.21.15.105:0/1790405460 learned_addr learned my addr 172.21.15.105:0/1790405460 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:43.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.808+0000 7f108bfff640 1 --2- 172.21.15.105:0/1790405460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f108c15b030 0x7f108c0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:43.811 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.808+0000 7f108a7fc640 1 -- 172.21.15.105:0/1790405460 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420203670 0 0) 0x7f108c0701d0 con 0x7f108c071960 2023-11-28T18:31:43.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.808+0000 7f108a7fc640 1 -- 172.21.15.105:0/1790405460 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1074003530 con 0x7f108c071960 2023-11-28T18:31:43.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.812+0000 7f108bfff640 1 -- 172.21.15.105:0/1790405460 >> v1:172.21.15.148:6789/0 conn(0x7f108c0b4290 legacy=0x7f108c0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:43.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.812+0000 7f108a7fc640 1 -- 172.21.15.105:0/1790405460 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 863744476 0 0) 0x7f1074003530 con 0x7f108c071960 2023-11-28T18:31:43.812 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.812+0000 7f108bfff640 1 -- 172.21.15.105:0/1790405460 >> v1:172.21.15.105:6789/0 conn(0x7f108c071960 legacy=0x7f108c15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:43.813 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.812+0000 7f108bfff640 1 -- 172.21.15.105:0/1790405460 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c0720d0 con 0x7f108c15b030 2023-11-28T18:31:43.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.812+0000 7f108bfff640 1 --2- 172.21.15.105:0/1790405460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f108c15b030 0x7f108c0709b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1084009dc0 tx=0x7f108401c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=183adda3206f1645 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:43.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.812+0000 7f108a7fc640 1 -- 172.21.15.105:0/1790405460 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1080001680 con 0x7f108c15b030 2023-11-28T18:31:43.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.812+0000 7f108a7fc640 1 -- 172.21.15.105:0/1790405460 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10840a8450 con 0x7f108c15b030 2023-11-28T18:31:43.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.812+0000 7f108a7fc640 1 -- 172.21.15.105:0/1790405460 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10840a87c0 con 0x7f108c15b030 2023-11-28T18:31:43.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.812+0000 7f1091b2a640 1 -- 172.21.15.105:0/1790405460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f108c15b030 msgr2=0x7f108c0709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:43.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.812+0000 7f1091b2a640 1 --2- 172.21.15.105:0/1790405460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f108c15b030 0x7f108c0709b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1084009dc0 tx=0x7f108401c710 comp rx=0 tx=0).stop 2023-11-28T18:31:43.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.812+0000 7f1091b2a640 1 -- 172.21.15.105:0/1790405460 shutdown_connections 2023-11-28T18:31:43.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f1091b2a640 1 --2- 172.21.15.105:0/1790405460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f108c15b030 0x7f108c0709b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f1091b2a640 1 -- 172.21.15.105:0/1790405460 >> 172.21.15.105:0/1790405460 conn(0x7f108c0ceb90 msgr2=0x7f108c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:43.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f1091b2a640 1 -- 172.21.15.105:0/1790405460 shutdown_connections 2023-11-28T18:31:43.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f1091b2a640 1 -- 172.21.15.105:0/1790405460 wait complete. 2023-11-28T18:31:43.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f1091b2a640 1 Processor -- start 2023-11-28T18:31:43.816 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f1091b2a640 1 -- start start 2023-11-28T18:31:43.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f1091b2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f108c0b4290 0x7f108c165820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f1091b2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f108c15b030 0x7f108c1591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f1091b2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f108c159700 0x7f108c09a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f1091b2a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c09bc50 con 0x7f108c15b030 2023-11-28T18:31:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f1091b2a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c1548d0 con 0x7f108c159700 2023-11-28T18:31:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f1091b2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108c09c070 con 0x7f108c0b4290 2023-11-28T18:31:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f108affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f108c15b030 0x7f108c1591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f108affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f108c15b030 0x7f108c1591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50092/0 (socket says 172.21.15.105:50092) 2023-11-28T18:31:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f108affd640 1 -- 172.21.15.105:0/672425925 learned_addr learned my addr 172.21.15.105:0/672425925 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f108bfff640 1 --2- 172.21.15.105:0/672425925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f108c159700 0x7f108c09a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f108b7fe640 1 --2- 172.21.15.105:0/672425925 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f108c0b4290 0x7f108c165820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f108affd640 1 -- 172.21.15.105:0/672425925 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f108c0b4290 msgr2=0x7f108c165820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:43.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f108affd640 1 --2- 172.21.15.105:0/672425925 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f108c0b4290 0x7f108c165820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f108affd640 1 -- 172.21.15.105:0/672425925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f108c159700 msgr2=0x7f108c09a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:43.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f108affd640 1 --2- 172.21.15.105:0/672425925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f108c159700 0x7f108c09a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f108affd640 1 -- 172.21.15.105:0/672425925 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1084009a70 con 0x7f108c15b030 2023-11-28T18:31:43.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.816+0000 7f108affd640 1 --2- 172.21.15.105:0/672425925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f108c15b030 0x7f108c1591c0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f108000c730 tx=0x7f108000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:43.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.820+0000 7f1073fff640 1 -- 172.21.15.105:0/672425925 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10800072d0 con 0x7f108c15b030 2023-11-28T18:31:43.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.820+0000 7f1073fff640 1 -- 172.21.15.105:0/672425925 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1080007430 con 0x7f108c15b030 2023-11-28T18:31:43.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.820+0000 7f1091b2a640 1 -- 172.21.15.105:0/672425925 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f108c15a0a0 con 0x7f108c15b030 2023-11-28T18:31:43.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.820+0000 7f1073fff640 1 -- 172.21.15.105:0/672425925 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10800047f0 con 0x7f108c15b030 2023-11-28T18:31:43.820 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.820+0000 7f1091b2a640 1 -- 172.21.15.105:0/672425925 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f108c09b870 con 0x7f108c15b030 2023-11-28T18:31:43.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.820+0000 7f1091b2a640 1 -- 172.21.15.105:0/672425925 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f108c09bd90 con 0x7f108c15b030 2023-11-28T18:31:43.821 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.820+0000 7f1073fff640 1 -- 172.21.15.105:0/672425925 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f1080004950 con 0x7f108c15b030 2023-11-28T18:31:43.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.820+0000 7f1073fff640 1 --2- 172.21.15.105:0/672425925 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10780668d0 0x7f1078068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:43.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.820+0000 7f1073fff640 1 -- 172.21.15.105:0/672425925 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f10800837d0 con 0x7f108c15b030 2023-11-28T18:31:43.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.820+0000 7f108b7fe640 1 --2- 172.21.15.105:0/672425925 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10780668d0 0x7f1078068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:43.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.824+0000 7f108b7fe640 1 --2- 172.21.15.105:0/672425925 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10780668d0 0x7f1078068d90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f107c007fd0 tx=0x7f107c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:43.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.824+0000 7f1073fff640 1 -- 172.21.15.105:0/672425925 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f108c09bd90 con 0x7f108c15b030 2023-11-28T18:31:43.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.976+0000 7f1091b2a640 1 -- 172.21.15.105:0/672425925 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f108c09bd90 con 0x7f108c15b030 2023-11-28T18:31:43.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.976+0000 7f1073fff640 1 -- 172.21.15.105:0/672425925 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16844 (secure 0 0 0) 0x7f108c09bd90 con 0x7f108c15b030 2023-11-28T18:31:43.977 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:43.977 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":117,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:31:04.144749+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:31:00.777530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-28T18:31:02.057361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:31:43.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.976+0000 7f1091b2a640 1 -- 172.21.15.105:0/672425925 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10780668d0 msgr2=0x7f1078068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:43.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.976+0000 7f1091b2a640 1 --2- 172.21.15.105:0/672425925 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10780668d0 0x7f1078068d90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f107c007fd0 tx=0x7f107c009040 comp rx=0 tx=0).stop 2023-11-28T18:31:43.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.976+0000 7f1091b2a640 1 -- 172.21.15.105:0/672425925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f108c15b030 msgr2=0x7f108c1591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:43.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.980+0000 7f1091b2a640 1 --2- 172.21.15.105:0/672425925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f108c15b030 0x7f108c1591c0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f108000c730 tx=0x7f108000cbf0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.980+0000 7f1091b2a640 1 -- 172.21.15.105:0/672425925 shutdown_connections 2023-11-28T18:31:43.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.980+0000 7f1091b2a640 1 --2- 172.21.15.105:0/672425925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f108c159700 0x7f108c09a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.980+0000 7f1091b2a640 1 --2- 172.21.15.105:0/672425925 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f10780668d0 0x7f1078068d90 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.980+0000 7f1091b2a640 1 --2- 172.21.15.105:0/672425925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f108c15b030 0x7f108c1591c0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.980+0000 7f1091b2a640 1 --2- 172.21.15.105:0/672425925 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f108c0b4290 0x7f108c165820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:43.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.980+0000 7f1091b2a640 1 -- 172.21.15.105:0/672425925 >> 172.21.15.105:0/672425925 conn(0x7f108c0ceb90 msgr2=0x7f108c0af0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:43.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.980+0000 7f1091b2a640 1 -- 172.21.15.105:0/672425925 shutdown_connections 2023-11-28T18:31:43.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:43.980+0000 7f1091b2a640 1 -- 172.21.15.105:0/672425925 wait complete. 2023-11-28T18:31:43.998 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T18:31:43.998 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:31:44.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.136+0000 7fc28de38640 1 Processor -- start 2023-11-28T18:31:44.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.136+0000 7fc28de38640 1 -- start start 2023-11-28T18:31:44.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.136+0000 7fc28de38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc288071230 0x7fc2880aef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:44.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.136+0000 7fc28de38640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28806e700 con 0x7fc28806cff0 2023-11-28T18:31:44.139 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.136+0000 7fc28de38640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28806e950 con 0x7fc2880cb9c0 2023-11-28T18:31:44.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.136+0000 7fc28de38640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc28806ef10 con 0x7fc288071230 2023-11-28T18:31:44.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.136+0000 7fc28d637640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc28806cff0 0x7fc28806a580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49796/0 (socket says 172.21.15.105:49796) 2023-11-28T18:31:44.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.136+0000 7fc28d637640 1 -- 172.21.15.105:0/2314478545 learned_addr learned my addr 172.21.15.105:0/2314478545 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:44.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.136+0000 7fc27ffff640 1 --2- 172.21.15.105:0/2314478545 >> v2:172.21.15.148:3300/0 conn(0x7fc288071230 0x7fc2880aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:44.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.136+0000 7fc27f7fe640 1 -- 172.21.15.105:0/2314478545 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1391434515 0 0) 0x7fc28806e700 con 0x7fc28806cff0 2023-11-28T18:31:44.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.136+0000 7fc27f7fe640 1 -- 172.21.15.105:0/2314478545 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc270003530 con 0x7fc28806cff0 2023-11-28T18:31:44.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.136+0000 7fc27ffff640 1 -- 172.21.15.105:0/2314478545 >> v1:172.21.15.148:6789/0 conn(0x7fc2880cb9c0 legacy=0x7fc288161170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:44.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.136+0000 7fc27ffff640 1 -- 172.21.15.105:0/2314478545 >> v1:172.21.15.105:6789/0 conn(0x7fc28806cff0 legacy=0x7fc28806a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:44.140 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.136+0000 7fc27ffff640 1 -- 172.21.15.105:0/2314478545 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28806ec30 con 0x7fc288071230 2023-11-28T18:31:44.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.140+0000 7fc27ffff640 1 --2- 172.21.15.105:0/2314478545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc288071230 0x7fc2880aef60 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc2780097c0 tx=0x7fc27801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37e2f82ccc9b80a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:44.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.140+0000 7fc27f7fe640 1 -- 172.21.15.105:0/2314478545 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2780ac640 con 0x7fc288071230 2023-11-28T18:31:44.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.140+0000 7fc27f7fe640 1 -- 172.21.15.105:0/2314478545 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2780ac7a0 con 0x7fc288071230 2023-11-28T18:31:44.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.140+0000 7fc27f7fe640 1 -- 172.21.15.105:0/2314478545 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2780acb30 con 0x7fc288071230 2023-11-28T18:31:44.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.140+0000 7fc28de38640 1 -- 172.21.15.105:0/2314478545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc288071230 msgr2=0x7fc2880aef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:44.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.140+0000 7fc28de38640 1 --2- 172.21.15.105:0/2314478545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc288071230 0x7fc2880aef60 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc2780097c0 tx=0x7fc27801c3c0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.140+0000 7fc28de38640 1 -- 172.21.15.105:0/2314478545 shutdown_connections 2023-11-28T18:31:44.142 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.140+0000 7fc28de38640 1 --2- 172.21.15.105:0/2314478545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc288071230 0x7fc2880aef60 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.140+0000 7fc28de38640 1 -- 172.21.15.105:0/2314478545 >> 172.21.15.105:0/2314478545 conn(0x7fc2880cc500 msgr2=0x7fc2880cc910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:44.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.140+0000 7fc28de38640 1 -- 172.21.15.105:0/2314478545 shutdown_connections 2023-11-28T18:31:44.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.140+0000 7fc28de38640 1 -- 172.21.15.105:0/2314478545 wait complete. 2023-11-28T18:31:44.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc28de38640 1 Processor -- start 2023-11-28T18:31:44.143 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc28de38640 1 -- start start 2023-11-28T18:31:44.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc28de38640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc28806cff0 0x7fc2880c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:44.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc28de38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc288071230 0x7fc2880c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:44.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc28de38640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2880cb9c0 0x7fc2880c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:44.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc28de38640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28815f6a0 con 0x7fc2880cb9c0 2023-11-28T18:31:44.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc28de38640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880c7e00 con 0x7fc28806cff0 2023-11-28T18:31:44.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc28de38640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc28815fba0 con 0x7fc288071230 2023-11-28T18:31:44.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc28d637640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2880cb9c0 0x7fc2880c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:44.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc28d637640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2880cb9c0 0x7fc2880c82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50104/0 (socket says 172.21.15.105:50104) 2023-11-28T18:31:44.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc28d637640 1 -- 172.21.15.105:0/2522401614 learned_addr learned my addr 172.21.15.105:0/2522401614 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:44.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc28ce36640 1 --2- 172.21.15.105:0/2522401614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc28806cff0 0x7fc2880c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:44.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc27ffff640 1 --2- 172.21.15.105:0/2522401614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc288071230 0x7fc2880c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:44.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc28d637640 1 -- 172.21.15.105:0/2522401614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc288071230 msgr2=0x7fc2880c35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:44.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc28d637640 1 --2- 172.21.15.105:0/2522401614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc288071230 0x7fc2880c35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc28d637640 1 -- 172.21.15.105:0/2522401614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc28806cff0 msgr2=0x7fc2880c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:44.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc28d637640 1 --2- 172.21.15.105:0/2522401614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc28806cff0 0x7fc2880c5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc28d637640 1 -- 172.21.15.105:0/2522401614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc278009420 con 0x7fc2880cb9c0 2023-11-28T18:31:44.146 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc27ffff640 1 --2- 172.21.15.105:0/2522401614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc288071230 0x7fc2880c35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:44.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc28d637640 1 --2- 172.21.15.105:0/2522401614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2880cb9c0 0x7fc2880c82e0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fc28000cb40 tx=0x7fc28000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:44.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc27d7fa640 1 -- 172.21.15.105:0/2522401614 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28000aca0 con 0x7fc2880cb9c0 2023-11-28T18:31:44.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc27d7fa640 1 -- 172.21.15.105:0/2522401614 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28000ae00 con 0x7fc2880cb9c0 2023-11-28T18:31:44.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc28de38640 1 -- 172.21.15.105:0/2522401614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc28815fe80 con 0x7fc2880cb9c0 2023-11-28T18:31:44.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc27d7fa640 1 -- 172.21.15.105:0/2522401614 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc280012620 con 0x7fc2880cb9c0 2023-11-28T18:31:44.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc28de38640 1 -- 172.21.15.105:0/2522401614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2880c6030 con 0x7fc2880cb9c0 2023-11-28T18:31:44.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.144+0000 7fc28de38640 1 -- 172.21.15.105:0/2522401614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc288074580 con 0x7fc2880cb9c0 2023-11-28T18:31:44.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.148+0000 7fc27d7fa640 1 -- 172.21.15.105:0/2522401614 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc280007070 con 0x7fc2880cb9c0 2023-11-28T18:31:44.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.148+0000 7fc27d7fa640 1 --2- 172.21.15.105:0/2522401614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc2740668d0 0x7fc274068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:44.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.148+0000 7fc27d7fa640 1 -- 172.21.15.105:0/2522401614 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc280084560 con 0x7fc2880cb9c0 2023-11-28T18:31:44.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.148+0000 7fc28ce36640 1 --2- 172.21.15.105:0/2522401614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc2740668d0 0x7fc274068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:44.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.148+0000 7fc28ce36640 1 --2- 172.21.15.105:0/2522401614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc2740668d0 0x7fc274068d90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc288072280 tx=0x7fc284009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:44.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.152+0000 7fc27d7fa640 1 -- 172.21.15.105:0/2522401614 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc280051700 con 0x7fc2880cb9c0 2023-11-28T18:31:44.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.332+0000 7fc28de38640 1 -- 172.21.15.105:0/2522401614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc288073100 con 0x7fc2880cb9c0 2023-11-28T18:31:44.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.332+0000 7fc27d7fa640 1 -- 172.21.15.105:0/2522401614 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+4960 (secure 0 0 0) 0x7fc280054d20 con 0x7fc2880cb9c0 2023-11-28T18:31:44.334 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:44.335 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":126,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/2073622668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2073622668},{"type":"v1","addr":"172.21.15.148:6834","nonce":2073622668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:31:03.120082+0000","modified":"2023-11-28T18:31:32.799694+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8643,"mds_1":8640,"mds_2":7085},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7085":{"gid":7085,"name":"d","rank":2,"incarnation":123,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6835/270170970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":270170970},{"type":"v1","addr":"172.21.15.148:6835","nonce":270170970}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8640":{"gid":8640,"name":"c","rank":1,"incarnation":121,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6836/3373504883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3373504883},{"type":"v1","addr":"172.21.15.105:6836","nonce":3373504883}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8643":{"gid":8643,"name":"a","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/764864050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":764864050},{"type":"v1","addr":"172.21.15.105:6837","nonce":764864050}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-28T18:31:44.335 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 126 2023-11-28T18:31:44.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.336+0000 7fc28de38640 1 -- 172.21.15.105:0/2522401614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc2740668d0 msgr2=0x7fc274068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:44.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.336+0000 7fc28de38640 1 --2- 172.21.15.105:0/2522401614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc2740668d0 0x7fc274068d90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc288072280 tx=0x7fc284009040 comp rx=0 tx=0).stop 2023-11-28T18:31:44.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.336+0000 7fc28de38640 1 -- 172.21.15.105:0/2522401614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2880cb9c0 msgr2=0x7fc2880c82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:44.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.336+0000 7fc28de38640 1 --2- 172.21.15.105:0/2522401614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2880cb9c0 0x7fc2880c82e0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fc28000cb40 tx=0x7fc28000a060 comp rx=0 tx=0).stop 2023-11-28T18:31:44.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.336+0000 7fc28de38640 1 -- 172.21.15.105:0/2522401614 shutdown_connections 2023-11-28T18:31:44.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.336+0000 7fc28de38640 1 --2- 172.21.15.105:0/2522401614 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc2740668d0 0x7fc274068d90 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.336+0000 7fc28de38640 1 --2- 172.21.15.105:0/2522401614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2880cb9c0 0x7fc2880c82e0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.336+0000 7fc28de38640 1 --2- 172.21.15.105:0/2522401614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc288071230 0x7fc2880c35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.336+0000 7fc28de38640 1 --2- 172.21.15.105:0/2522401614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc28806cff0 0x7fc2880c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.336+0000 7fc28de38640 1 -- 172.21.15.105:0/2522401614 >> 172.21.15.105:0/2522401614 conn(0x7fc2880cc500 msgr2=0x7fc2880ad2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:44.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.336+0000 7fc28de38640 1 -- 172.21.15.105:0/2522401614 shutdown_connections 2023-11-28T18:31:44.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.336+0000 7fc28de38640 1 -- 172.21.15.105:0/2522401614 wait complete. 2023-11-28T18:31:44.355 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:31:44.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.492+0000 7efe3e2c9640 1 Processor -- start 2023-11-28T18:31:44.493 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.492+0000 7efe3e2c9640 1 -- start start 2023-11-28T18:31:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.492+0000 7efe3e2c9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efe380caf60 0x7efe380cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.492+0000 7efe3e2c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe380ad850 0x7efe380701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.492+0000 7efe3e2c9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3806bf50 con 0x7efe38159f40 2023-11-28T18:31:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.492+0000 7efe3e2c9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe380cb3c0 con 0x7efe380caf60 2023-11-28T18:31:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.492+0000 7efe3e2c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3815a320 con 0x7efe380ad850 2023-11-28T18:31:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.492+0000 7efe377fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efe380caf60 0x7efe380cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.496+0000 7efe377fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efe380caf60 0x7efe380cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50110/0 (socket says 172.21.15.105:50110) 2023-11-28T18:31:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.496+0000 7efe377fe640 1 -- 172.21.15.105:0/3307736067 learned_addr learned my addr 172.21.15.105:0/3307736067 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.496+0000 7efe377fe640 1 -- 172.21.15.105:0/3307736067 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe380ad850 msgr2=0x7efe380701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.496+0000 7efe377fe640 1 --2- 172.21.15.105:0/3307736067 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe380ad850 0x7efe380701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.495 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.496+0000 7efe377fe640 1 -- 172.21.15.105:0/3307736067 >> v1:172.21.15.105:6789/0 conn(0x7efe38159f40 legacy=0x7efe380ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:44.496 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.496+0000 7efe377fe640 1 -- 172.21.15.105:0/3307736067 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe38071380 con 0x7efe380caf60 2023-11-28T18:31:44.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.496+0000 7efe377fe640 1 --2- 172.21.15.105:0/3307736067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe380caf60 0x7efe380cc760 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009870 tx=0x7efe2c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795be79d1c1850d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:44.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.496+0000 7efe36ffd640 1 -- 172.21.15.105:0/3307736067 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c0ab680 con 0x7efe380caf60 2023-11-28T18:31:44.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.496+0000 7efe36ffd640 1 -- 172.21.15.105:0/3307736067 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c0ab7e0 con 0x7efe380caf60 2023-11-28T18:31:44.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.496+0000 7efe36ffd640 1 -- 172.21.15.105:0/3307736067 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c0abb70 con 0x7efe380caf60 2023-11-28T18:31:44.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe3e2c9640 1 -- 172.21.15.105:0/3307736067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe380caf60 msgr2=0x7efe380cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:44.498 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe3e2c9640 1 --2- 172.21.15.105:0/3307736067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe380caf60 0x7efe380cc760 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009870 tx=0x7efe2c01c480 comp rx=0 tx=0).stop 2023-11-28T18:31:44.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe3e2c9640 1 -- 172.21.15.105:0/3307736067 shutdown_connections 2023-11-28T18:31:44.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe3e2c9640 1 --2- 172.21.15.105:0/3307736067 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe380ad850 0x7efe380701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe3e2c9640 1 --2- 172.21.15.105:0/3307736067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe380caf60 0x7efe380cc760 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe3e2c9640 1 -- 172.21.15.105:0/3307736067 >> 172.21.15.105:0/3307736067 conn(0x7efe380ceb90 msgr2=0x7efe380cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:44.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe3e2c9640 1 -- 172.21.15.105:0/3307736067 shutdown_connections 2023-11-28T18:31:44.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe3e2c9640 1 -- 172.21.15.105:0/3307736067 wait complete. 2023-11-28T18:31:44.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe3e2c9640 1 Processor -- start 2023-11-28T18:31:44.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe3e2c9640 1 -- start start 2023-11-28T18:31:44.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe3e2c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe380ad850 0x7efe3809b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:44.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe3e2c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe380caf60 0x7efe38139f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:44.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe3e2c9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe38159f40 0x7efe38136c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:44.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe3e2c9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3809c390 con 0x7efe380caf60 2023-11-28T18:31:44.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe3e2c9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3814b9a0 con 0x7efe380ad850 2023-11-28T18:31:44.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe3e2c9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3813a480 con 0x7efe38159f40 2023-11-28T18:31:44.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe377fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe380caf60 0x7efe38139f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:44.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe377fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe380caf60 0x7efe38139f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50112/0 (socket says 172.21.15.105:50112) 2023-11-28T18:31:44.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe377fe640 1 -- 172.21.15.105:0/1329820073 learned_addr learned my addr 172.21.15.105:0/1329820073 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:44.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe3c866640 1 --2- 172.21.15.105:0/1329820073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe38159f40 0x7efe38136c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:44.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe37fff640 1 --2- 172.21.15.105:0/1329820073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe380ad850 0x7efe3809b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:44.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe377fe640 1 -- 172.21.15.105:0/1329820073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe38159f40 msgr2=0x7efe38136c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:44.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe377fe640 1 --2- 172.21.15.105:0/1329820073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe38159f40 0x7efe38136c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe377fe640 1 -- 172.21.15.105:0/1329820073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe380ad850 msgr2=0x7efe3809b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:44.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe377fe640 1 --2- 172.21.15.105:0/1329820073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe380ad850 0x7efe3809b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.500+0000 7efe377fe640 1 -- 172.21.15.105:0/1329820073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2c0094d0 con 0x7efe380caf60 2023-11-28T18:31:44.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.504+0000 7efe377fe640 1 --2- 172.21.15.105:0/1329820073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe380caf60 0x7efe38139f40 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7efe2c01a0b0 tx=0x7efe2c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:44.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.504+0000 7efe34ff9640 1 -- 172.21.15.105:0/1329820073 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c0b4800 con 0x7efe380caf60 2023-11-28T18:31:44.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.504+0000 7efe34ff9640 1 -- 172.21.15.105:0/1329820073 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c0b4980 con 0x7efe380caf60 2023-11-28T18:31:44.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.504+0000 7efe34ff9640 1 -- 172.21.15.105:0/1329820073 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c0b4d30 con 0x7efe380caf60 2023-11-28T18:31:44.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.504+0000 7efe3e2c9640 1 -- 172.21.15.105:0/1329820073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe381371c0 con 0x7efe380caf60 2023-11-28T18:31:44.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.504+0000 7efe3e2c9640 1 -- 172.21.15.105:0/1329820073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3809b9f0 con 0x7efe380caf60 2023-11-28T18:31:44.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.504+0000 7efe3e2c9640 1 -- 172.21.15.105:0/1329820073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe38067570 con 0x7efe380caf60 2023-11-28T18:31:44.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.504+0000 7efe34ff9640 1 -- 172.21.15.105:0/1329820073 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7efe2c0c7440 con 0x7efe380caf60 2023-11-28T18:31:44.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.504+0000 7efe34ff9640 1 --2- 172.21.15.105:0/1329820073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe240668d0 0x7efe24068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:44.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.504+0000 7efe34ff9640 1 -- 172.21.15.105:0/1329820073 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efe2c11f400 con 0x7efe380caf60 2023-11-28T18:31:44.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.504+0000 7efe37fff640 1 --2- 172.21.15.105:0/1329820073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe240668d0 0x7efe24068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:44.506 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.508+0000 7efe37fff640 1 --2- 172.21.15.105:0/1329820073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe240668d0 0x7efe24068d90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7efe28007fd0 tx=0x7efe28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:44.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.508+0000 7efe34ff9640 1 -- 172.21.15.105:0/1329820073 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efe2c0ec5a0 con 0x7efe380caf60 2023-11-28T18:31:44.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.688+0000 7efe3e2c9640 1 -- 172.21.15.105:0/1329820073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe380cde40 con 0x7efe380caf60 2023-11-28T18:31:44.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.688+0000 7efe34ff9640 1 -- 172.21.15.105:0/1329820073 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+4960 (secure 0 0 0) 0x7efe2c0efbc0 con 0x7efe380caf60 2023-11-28T18:31:44.689 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:44.689 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":126,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/2073622668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2073622668},{"type":"v1","addr":"172.21.15.148:6834","nonce":2073622668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:31:03.120082+0000","modified":"2023-11-28T18:31:32.799694+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8643,"mds_1":8640,"mds_2":7085},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7085":{"gid":7085,"name":"d","rank":2,"incarnation":123,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6835/270170970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":270170970},{"type":"v1","addr":"172.21.15.148:6835","nonce":270170970}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8640":{"gid":8640,"name":"c","rank":1,"incarnation":121,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6836/3373504883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3373504883},{"type":"v1","addr":"172.21.15.105:6836","nonce":3373504883}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8643":{"gid":8643,"name":"a","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/764864050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":764864050},{"type":"v1","addr":"172.21.15.105:6837","nonce":764864050}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-28T18:31:44.690 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 126 2023-11-28T18:31:44.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.692+0000 7efe3e2c9640 1 -- 172.21.15.105:0/1329820073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe240668d0 msgr2=0x7efe24068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:44.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.692+0000 7efe3e2c9640 1 --2- 172.21.15.105:0/1329820073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe240668d0 0x7efe24068d90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7efe28007fd0 tx=0x7efe28009040 comp rx=0 tx=0).stop 2023-11-28T18:31:44.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.692+0000 7efe3e2c9640 1 -- 172.21.15.105:0/1329820073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe380caf60 msgr2=0x7efe38139f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:44.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.692+0000 7efe3e2c9640 1 --2- 172.21.15.105:0/1329820073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe380caf60 0x7efe38139f40 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7efe2c01a0b0 tx=0x7efe2c01acc0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.692+0000 7efe3e2c9640 1 -- 172.21.15.105:0/1329820073 shutdown_connections 2023-11-28T18:31:44.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.692+0000 7efe3e2c9640 1 --2- 172.21.15.105:0/1329820073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe38159f40 0x7efe38136c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.692+0000 7efe3e2c9640 1 --2- 172.21.15.105:0/1329820073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe240668d0 0x7efe24068d90 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.692+0000 7efe3e2c9640 1 --2- 172.21.15.105:0/1329820073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe380caf60 0x7efe38139f40 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.692+0000 7efe3e2c9640 1 --2- 172.21.15.105:0/1329820073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe380ad850 0x7efe3809b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.692+0000 7efe3e2c9640 1 -- 172.21.15.105:0/1329820073 >> 172.21.15.105:0/1329820073 conn(0x7efe380ceb90 msgr2=0x7efe380cc710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:44.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.692+0000 7efe3e2c9640 1 -- 172.21.15.105:0/1329820073 shutdown_connections 2023-11-28T18:31:44.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.692+0000 7efe3e2c9640 1 -- 172.21.15.105:0/1329820073 wait complete. 2023-11-28T18:31:44.711 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:31:44.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.848+0000 7f9a6311d640 1 Processor -- start 2023-11-28T18:31:44.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.848+0000 7f9a6311d640 1 -- start start 2023-11-28T18:31:44.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.848+0000 7f9a6311d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9a5c159f40 0x7f9a5c0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:44.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.848+0000 7f9a6311d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a5c0ad850 0x7f9a5c0701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:44.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.848+0000 7f9a6311d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a5c06bf50 con 0x7f9a5c0cc760 2023-11-28T18:31:44.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.848+0000 7f9a6311d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a5c0ccbc0 con 0x7f9a5c159f40 2023-11-28T18:31:44.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.848+0000 7f9a6311d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c15a320 con 0x7f9a5c0ad850 2023-11-28T18:31:44.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.848+0000 7f9a60eb9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9a5c159f40 0x7f9a5c0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:44.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.848+0000 7f9a53fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9a5c0cc760 0x7f9a5c0ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49814/0 (socket says 172.21.15.105:49814) 2023-11-28T18:31:44.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.848+0000 7f9a53fff640 1 -- 172.21.15.105:0/204802523 learned_addr learned my addr 172.21.15.105:0/204802523 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:44.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.852+0000 7f9a616ba640 1 --2- 172.21.15.105:0/204802523 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a5c0ad850 0x7f9a5c0701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:44.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.852+0000 7f9a60eb9640 1 -- 172.21.15.105:0/204802523 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a5c0ad850 msgr2=0x7f9a5c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:44.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.852+0000 7f9a60eb9640 1 --2- 172.21.15.105:0/204802523 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a5c0ad850 0x7f9a5c0701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.852+0000 7f9a60eb9640 1 -- 172.21.15.105:0/204802523 >> v1:172.21.15.105:6789/0 conn(0x7f9a5c0cc760 legacy=0x7f9a5c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:44.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.852+0000 7f9a60eb9640 1 -- 172.21.15.105:0/204802523 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5c071380 con 0x7f9a5c159f40 2023-11-28T18:31:44.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.852+0000 7f9a537fe640 1 -- 172.21.15.105:0/204802523 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3943358013 0 0) 0x7f9a5c06bf50 con 0x7f9a5c0cc760 2023-11-28T18:31:44.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.852+0000 7f9a616ba640 1 --2- 172.21.15.105:0/204802523 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a5c0ad850 0x7f9a5c0701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:31:44.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.852+0000 7f9a60eb9640 1 --2- 172.21.15.105:0/204802523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a5c159f40 0x7f9a5c0caf60 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f9a580097c0 tx=0x7f9a5801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea3fcd9e5cc5ec0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:44.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.852+0000 7f9a537fe640 1 -- 172.21.15.105:0/204802523 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a580ac620 con 0x7f9a5c159f40 2023-11-28T18:31:44.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.852+0000 7f9a537fe640 1 -- 172.21.15.105:0/204802523 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a580ac780 con 0x7f9a5c159f40 2023-11-28T18:31:44.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.852+0000 7f9a537fe640 1 -- 172.21.15.105:0/204802523 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a580acb10 con 0x7f9a5c159f40 2023-11-28T18:31:44.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.852+0000 7f9a6311d640 1 -- 172.21.15.105:0/204802523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a5c159f40 msgr2=0x7f9a5c0caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:44.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.852+0000 7f9a6311d640 1 --2- 172.21.15.105:0/204802523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a5c159f40 0x7f9a5c0caf60 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f9a580097c0 tx=0x7f9a5801c3c0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.852+0000 7f9a6311d640 1 -- 172.21.15.105:0/204802523 shutdown_connections 2023-11-28T18:31:44.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.852+0000 7f9a6311d640 1 --2- 172.21.15.105:0/204802523 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a5c0ad850 0x7f9a5c0701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.852+0000 7f9a6311d640 1 --2- 172.21.15.105:0/204802523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a5c159f40 0x7f9a5c0caf60 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.852+0000 7f9a6311d640 1 -- 172.21.15.105:0/204802523 >> 172.21.15.105:0/204802523 conn(0x7f9a5c0ceb90 msgr2=0x7f9a5c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:44.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a6311d640 1 -- 172.21.15.105:0/204802523 shutdown_connections 2023-11-28T18:31:44.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a6311d640 1 -- 172.21.15.105:0/204802523 wait complete. 2023-11-28T18:31:44.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a6311d640 1 Processor -- start 2023-11-28T18:31:44.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a6311d640 1 -- start start 2023-11-28T18:31:44.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a6311d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a5c0ad850 0x7f9a5c166ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:44.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a6311d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a5c0cc760 0x7f9a5c15cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:44.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a6311d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a5c159f40 0x7f9a5c14a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:44.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a6311d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c165f20 con 0x7f9a5c0ad850 2023-11-28T18:31:44.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a6311d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c14c430 con 0x7f9a5c0cc760 2023-11-28T18:31:44.861 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a6311d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c15d230 con 0x7f9a5c159f40 2023-11-28T18:31:44.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a60eb9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a5c0ad850 0x7f9a5c166ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:44.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a60eb9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a5c0ad850 0x7f9a5c166ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50128/0 (socket says 172.21.15.105:50128) 2023-11-28T18:31:44.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a60eb9640 1 -- 172.21.15.105:0/3433498039 learned_addr learned my addr 172.21.15.105:0/3433498039 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:44.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a616ba640 1 --2- 172.21.15.105:0/3433498039 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a5c159f40 0x7f9a5c14a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:44.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a53fff640 1 --2- 172.21.15.105:0/3433498039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a5c0cc760 0x7f9a5c15cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:44.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a60eb9640 1 -- 172.21.15.105:0/3433498039 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a5c159f40 msgr2=0x7f9a5c14a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:44.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a60eb9640 1 --2- 172.21.15.105:0/3433498039 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a5c159f40 0x7f9a5c14a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a60eb9640 1 -- 172.21.15.105:0/3433498039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a5c0cc760 msgr2=0x7f9a5c15cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:44.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a60eb9640 1 --2- 172.21.15.105:0/3433498039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a5c0cc760 0x7f9a5c15cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:44.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a60eb9640 1 -- 172.21.15.105:0/3433498039 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a58009420 con 0x7f9a5c0ad850 2023-11-28T18:31:44.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a53fff640 1 --2- 172.21.15.105:0/3433498039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a5c0cc760 0x7f9a5c15cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:31:44.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a616ba640 1 --2- 172.21.15.105:0/3433498039 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a5c159f40 0x7f9a5c14a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:31:44.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.856+0000 7f9a60eb9640 1 --2- 172.21.15.105:0/3433498039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a5c0ad850 0x7f9a5c166ca0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f9a58019ff0 tx=0x7f9a58019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:44.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.860+0000 7f9a517fa640 1 -- 172.21.15.105:0/3433498039 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a580b5610 con 0x7f9a5c0ad850 2023-11-28T18:31:44.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.860+0000 7f9a517fa640 1 -- 172.21.15.105:0/3433498039 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a580b5790 con 0x7f9a5c0ad850 2023-11-28T18:31:44.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.860+0000 7f9a517fa640 1 -- 172.21.15.105:0/3433498039 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a580b5b40 con 0x7f9a5c0ad850 2023-11-28T18:31:44.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.860+0000 7f9a6311d640 1 -- 172.21.15.105:0/3433498039 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a5c1661c0 con 0x7f9a5c0ad850 2023-11-28T18:31:44.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.860+0000 7f9a6311d640 1 -- 172.21.15.105:0/3433498039 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5c166780 con 0x7f9a5c0ad850 2023-11-28T18:31:44.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.860+0000 7f9a6311d640 1 -- 172.21.15.105:0/3433498039 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a5c166970 con 0x7f9a5c0ad850 2023-11-28T18:31:44.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.860+0000 7f9a517fa640 1 -- 172.21.15.105:0/3433498039 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f9a580b3040 con 0x7f9a5c0ad850 2023-11-28T18:31:44.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.860+0000 7f9a517fa640 1 --2- 172.21.15.105:0/3433498039 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9a48066900 0x7f9a48068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:44.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.860+0000 7f9a517fa640 1 -- 172.21.15.105:0/3433498039 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9a5811e2a0 con 0x7f9a5c0ad850 2023-11-28T18:31:44.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.860+0000 7f9a53fff640 1 --2- 172.21.15.105:0/3433498039 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9a48066900 0x7f9a48068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:44.864 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.864+0000 7f9a53fff640 1 --2- 172.21.15.105:0/3433498039 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9a48066900 0x7f9a48068dc0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c0ac6c0 tx=0x7f9a4c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:44.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:44.864+0000 7f9a517fa640 1 -- 172.21.15.105:0/3433498039 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9a580eb430 con 0x7f9a5c0ad850 2023-11-28T18:31:45.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.016+0000 7f9a6311d640 1 -- 172.21.15.105:0/3433498039 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a5c067570 con 0x7f9a5c0ad850 2023-11-28T18:31:45.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.016+0000 7f9a517fa640 1 -- 172.21.15.105:0/3433498039 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16844 (secure 0 0 0) 0x7f9a580ac5e0 con 0x7f9a5c0ad850 2023-11-28T18:31:45.017 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:45.017 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":117,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:31:04.144749+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:31:00.777530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-28T18:31:02.057361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:31:45.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.016+0000 7f9a6311d640 1 -- 172.21.15.105:0/3433498039 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9a48066900 msgr2=0x7f9a48068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:45.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.016+0000 7f9a6311d640 1 --2- 172.21.15.105:0/3433498039 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9a48066900 0x7f9a48068dc0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c0ac6c0 tx=0x7f9a4c009040 comp rx=0 tx=0).stop 2023-11-28T18:31:45.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.016+0000 7f9a6311d640 1 -- 172.21.15.105:0/3433498039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a5c0ad850 msgr2=0x7f9a5c166ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:45.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.016+0000 7f9a6311d640 1 --2- 172.21.15.105:0/3433498039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a5c0ad850 0x7f9a5c166ca0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f9a58019ff0 tx=0x7f9a58019160 comp rx=0 tx=0).stop 2023-11-28T18:31:45.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.020+0000 7f9a6311d640 1 -- 172.21.15.105:0/3433498039 shutdown_connections 2023-11-28T18:31:45.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.020+0000 7f9a6311d640 1 --2- 172.21.15.105:0/3433498039 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a5c159f40 0x7f9a5c14a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:45.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.020+0000 7f9a6311d640 1 --2- 172.21.15.105:0/3433498039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a5c0cc760 0x7f9a5c15cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:45.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.020+0000 7f9a6311d640 1 --2- 172.21.15.105:0/3433498039 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9a48066900 0x7f9a48068dc0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:45.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.020+0000 7f9a6311d640 1 --2- 172.21.15.105:0/3433498039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a5c0ad850 0x7f9a5c166ca0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:45.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.020+0000 7f9a6311d640 1 -- 172.21.15.105:0/3433498039 >> 172.21.15.105:0/3433498039 conn(0x7f9a5c0ceb90 msgr2=0x7f9a5c071d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:45.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.020+0000 7f9a6311d640 1 -- 172.21.15.105:0/3433498039 shutdown_connections 2023-11-28T18:31:45.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.020+0000 7f9a6311d640 1 -- 172.21.15.105:0/3433498039 wait complete. 2023-11-28T18:31:45.038 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T18:31:45.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.176+0000 7f4eb345a640 1 Processor -- start 2023-11-28T18:31:45.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.176+0000 7f4eb345a640 1 -- start start 2023-11-28T18:31:45.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.176+0000 7f4eb345a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4eac159f40 0x7f4eac0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:45.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.176+0000 7f4eb345a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4eac0ad850 0x7f4eac0701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:45.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.176+0000 7f4eb345a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eac06bf50 con 0x7f4eac0cc760 2023-11-28T18:31:45.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.176+0000 7f4eb345a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eac0ccbc0 con 0x7f4eac159f40 2023-11-28T18:31:45.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.176+0000 7f4eb345a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac15a320 con 0x7f4eac0ad850 2023-11-28T18:31:45.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.176+0000 7f4eb11f6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4eac159f40 0x7f4eac0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:45.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.176+0000 7f4eb11f6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4eac159f40 0x7f4eac0caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50130/0 (socket says 172.21.15.105:50130) 2023-11-28T18:31:45.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.176+0000 7f4eb09f5640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4eac0cc760 0x7f4eac0ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:58658/0 (socket says 172.21.15.105:58658) 2023-11-28T18:31:45.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.176+0000 7f4eb11f6640 1 -- 172.21.15.105:0/1280967948 learned_addr learned my addr 172.21.15.105:0/1280967948 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:45.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.176+0000 7f4eb19f7640 1 --2- 172.21.15.105:0/1280967948 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4eac0ad850 0x7f4eac0701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:45.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.180+0000 7f4eb11f6640 1 -- 172.21.15.105:0/1280967948 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4eac0ad850 msgr2=0x7f4eac0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:45.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.180+0000 7f4eb11f6640 1 --2- 172.21.15.105:0/1280967948 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4eac0ad850 0x7f4eac0701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:45.179 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.180+0000 7f4eb11f6640 1 -- 172.21.15.105:0/1280967948 >> v1:172.21.15.148:6789/0 conn(0x7f4eac0cc760 legacy=0x7f4eac0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:45.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.180+0000 7f4eb11f6640 1 -- 172.21.15.105:0/1280967948 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eac071380 con 0x7f4eac159f40 2023-11-28T18:31:45.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.180+0000 7f4eb11f6640 1 --2- 172.21.15.105:0/1280967948 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eac159f40 0x7f4eac0caf60 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009870 tx=0x7f4ea801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee38881649fea07f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:45.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.180+0000 7f4ea3fff640 1 -- 172.21.15.105:0/1280967948 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea80ab680 con 0x7f4eac159f40 2023-11-28T18:31:45.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.180+0000 7f4ea3fff640 1 -- 172.21.15.105:0/1280967948 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea80ab7e0 con 0x7f4eac159f40 2023-11-28T18:31:45.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.180+0000 7f4ea3fff640 1 -- 172.21.15.105:0/1280967948 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea80abb70 con 0x7f4eac159f40 2023-11-28T18:31:45.183 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.180+0000 7f4eb345a640 1 -- 172.21.15.105:0/1280967948 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eac159f40 msgr2=0x7f4eac0caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.180+0000 7f4eb345a640 1 --2- 172.21.15.105:0/1280967948 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eac159f40 0x7f4eac0caf60 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009870 tx=0x7f4ea801c480 comp rx=0 tx=0).stop 2023-11-28T18:31:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb345a640 1 -- 172.21.15.105:0/1280967948 shutdown_connections 2023-11-28T18:31:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb345a640 1 --2- 172.21.15.105:0/1280967948 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4eac0ad850 0x7f4eac0701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb345a640 1 --2- 172.21.15.105:0/1280967948 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eac159f40 0x7f4eac0caf60 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb345a640 1 -- 172.21.15.105:0/1280967948 >> 172.21.15.105:0/1280967948 conn(0x7f4eac0ceb90 msgr2=0x7f4eac0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb345a640 1 -- 172.21.15.105:0/1280967948 shutdown_connections 2023-11-28T18:31:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb345a640 1 -- 172.21.15.105:0/1280967948 wait complete. 2023-11-28T18:31:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb345a640 1 Processor -- start 2023-11-28T18:31:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb345a640 1 -- start start 2023-11-28T18:31:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb345a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4eac0ad850 0x7f4eac158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb345a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eac0cc760 0x7f4eac152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb345a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eac159f40 0x7f4eac152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb345a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac15ad10 con 0x7f4eac0cc760 2023-11-28T18:31:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb345a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac15a9e0 con 0x7f4eac159f40 2023-11-28T18:31:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb345a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac159170 con 0x7f4eac0ad850 2023-11-28T18:31:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb09f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eac0cc760 0x7f4eac152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb09f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eac0cc760 0x7f4eac152560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50142/0 (socket says 172.21.15.105:50142) 2023-11-28T18:31:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb09f5640 1 -- 172.21.15.105:0/882087324 learned_addr learned my addr 172.21.15.105:0/882087324 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb11f6640 1 --2- 172.21.15.105:0/882087324 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4eac0ad850 0x7f4eac158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb19f7640 1 --2- 172.21.15.105:0/882087324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eac159f40 0x7f4eac152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb09f5640 1 -- 172.21.15.105:0/882087324 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4eac0ad850 msgr2=0x7f4eac158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb09f5640 1 --2- 172.21.15.105:0/882087324 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4eac0ad850 0x7f4eac158a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb09f5640 1 -- 172.21.15.105:0/882087324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eac159f40 msgr2=0x7f4eac152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb09f5640 1 --2- 172.21.15.105:0/882087324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eac159f40 0x7f4eac152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb09f5640 1 -- 172.21.15.105:0/882087324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea80094d0 con 0x7f4eac0cc760 2023-11-28T18:31:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb19f7640 1 --2- 172.21.15.105:0/882087324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eac159f40 0x7f4eac152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4eb09f5640 1 --2- 172.21.15.105:0/882087324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eac0cc760 0x7f4eac152560 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00c490 tx=0x7f4e9c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4ea1ffb640 1 -- 172.21.15.105:0/882087324 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c007480 con 0x7f4eac0cc760 2023-11-28T18:31:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4ea1ffb640 1 -- 172.21.15.105:0/882087324 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c0075e0 con 0x7f4eac0cc760 2023-11-28T18:31:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.184+0000 7f4ea1ffb640 1 -- 172.21.15.105:0/882087324 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c004a10 con 0x7f4eac0cc760 2023-11-28T18:31:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.188+0000 7f4eb345a640 1 -- 172.21.15.105:0/882087324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eac159450 con 0x7f4eac0cc760 2023-11-28T18:31:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.188+0000 7f4eb345a640 1 -- 172.21.15.105:0/882087324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eac15a5f0 con 0x7f4eac0cc760 2023-11-28T18:31:45.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.188+0000 7f4eb345a640 1 -- 172.21.15.105:0/882087324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eac15aec0 con 0x7f4eac0cc760 2023-11-28T18:31:45.189 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.188+0000 7f4ea1ffb640 1 -- 172.21.15.105:0/882087324 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f4e9c004b70 con 0x7f4eac0cc760 2023-11-28T18:31:45.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.188+0000 7f4ea1ffb640 1 --2- 172.21.15.105:0/882087324 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4e90066900 0x7f4e90068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:45.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.188+0000 7f4ea1ffb640 1 -- 172.21.15.105:0/882087324 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4e9c083700 con 0x7f4eac0cc760 2023-11-28T18:31:45.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.188+0000 7f4eb11f6640 1 --2- 172.21.15.105:0/882087324 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4e90066900 0x7f4e90068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:45.190 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.192+0000 7f4eb11f6640 1 --2- 172.21.15.105:0/882087324 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4e90066900 0x7f4e90068dc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80af000 tx=0x7f4ea80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:45.193 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.192+0000 7f4ea1ffb640 1 -- 172.21.15.105:0/882087324 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4eac15aec0 con 0x7f4eac0cc760 2023-11-28T18:31:45.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.376+0000 7f4eb345a640 1 -- 172.21.15.105:0/882087324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4eac15aec0 con 0x7f4eac0cc760 2023-11-28T18:31:45.378 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-11-28T18:31:45.376+0000 7fd89b32a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T18:31:45.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.380+0000 7f4ea1ffb640 1 -- 172.21.15.105:0/882087324 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v127) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4eac15aec0 con 0x7f4eac0cc760 2023-11-28T18:31:45.380 INFO:teuthology.orchestra.run.smithi105.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T18:31:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.384+0000 7f4eb345a640 1 -- 172.21.15.105:0/882087324 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4e90066900 msgr2=0x7f4e90068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.384+0000 7f4eb345a640 1 --2- 172.21.15.105:0/882087324 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4e90066900 0x7f4e90068dc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80af000 tx=0x7f4ea80a7000 comp rx=0 tx=0).stop 2023-11-28T18:31:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.384+0000 7f4eb345a640 1 -- 172.21.15.105:0/882087324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eac0cc760 msgr2=0x7f4eac152560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.384+0000 7f4eb345a640 1 --2- 172.21.15.105:0/882087324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eac0cc760 0x7f4eac152560 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00c490 tx=0x7f4e9c00c950 comp rx=0 tx=0).stop 2023-11-28T18:31:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.384+0000 7f4eb345a640 1 -- 172.21.15.105:0/882087324 shutdown_connections 2023-11-28T18:31:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.384+0000 7f4eb345a640 1 --2- 172.21.15.105:0/882087324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eac159f40 0x7f4eac152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.384+0000 7f4eb345a640 1 --2- 172.21.15.105:0/882087324 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4e90066900 0x7f4e90068dc0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.384+0000 7f4eb345a640 1 --2- 172.21.15.105:0/882087324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eac0cc760 0x7f4eac152560 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.384+0000 7f4eb345a640 1 --2- 172.21.15.105:0/882087324 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4eac0ad850 0x7f4eac158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.384+0000 7f4eb345a640 1 -- 172.21.15.105:0/882087324 >> 172.21.15.105:0/882087324 conn(0x7f4eac0ceb90 msgr2=0x7f4eac0af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:45.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.384+0000 7f4eb345a640 1 -- 172.21.15.105:0/882087324 shutdown_connections 2023-11-28T18:31:45.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.384+0000 7f4eb345a640 1 -- 172.21.15.105:0/882087324 wait complete. 2023-11-28T18:31:45.407 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T18:31:45.450 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2023-11-28T18:31:45.453 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2023-11-28T18:31:45.453 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2023-11-28T18:31:45.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.552+0000 7f91cb39d640 1 Processor -- start 2023-11-28T18:31:45.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.552+0000 7f91cb39d640 1 -- start start 2023-11-28T18:31:45.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.552+0000 7f91cb39d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c4071960 0x7f91c415f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:45.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.552+0000 7f91cb39d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c40701d0 con 0x7f91c40b4290 2023-11-28T18:31:45.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.552+0000 7f91cb39d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c406df80 con 0x7f91c415b030 2023-11-28T18:31:45.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.552+0000 7f91cb39d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c4071df0 con 0x7f91c4071960 2023-11-28T18:31:45.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.552+0000 7f91c9139640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f91c40b4290 0x7f91c40b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49834/0 (socket says 172.21.15.105:49834) 2023-11-28T18:31:45.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.552+0000 7f91c9139640 1 -- 172.21.15.105:0/241586363 learned_addr learned my addr 172.21.15.105:0/241586363 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:45.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.552+0000 7f91c8938640 1 --2- 172.21.15.105:0/241586363 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c4071960 0x7f91c415f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:45.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.552+0000 7f91bbfff640 1 -- 172.21.15.105:0/241586363 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922195271 0 0) 0x7f91c40701d0 con 0x7f91c40b4290 2023-11-28T18:31:45.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.556+0000 7f91bbfff640 1 -- 172.21.15.105:0/241586363 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91b0003530 con 0x7f91c40b4290 2023-11-28T18:31:45.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.556+0000 7f91bbfff640 1 -- 172.21.15.105:0/241586363 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225631921 0 0) 0x7f91c406df80 con 0x7f91c415b030 2023-11-28T18:31:45.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.556+0000 7f91bbfff640 1 -- 172.21.15.105:0/241586363 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91c40701d0 con 0x7f91c415b030 2023-11-28T18:31:45.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.556+0000 7f91c8938640 1 -- 172.21.15.105:0/241586363 >> v1:172.21.15.148:6789/0 conn(0x7f91c415b030 legacy=0x7f91c40709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:45.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.556+0000 7f91c8938640 1 -- 172.21.15.105:0/241586363 >> v1:172.21.15.105:6789/0 conn(0x7f91c40b4290 legacy=0x7f91c40b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:45.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.556+0000 7f91c8938640 1 -- 172.21.15.105:0/241586363 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c40720d0 con 0x7f91c4071960 2023-11-28T18:31:45.557 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.556+0000 7f91c8938640 1 --2- 172.21.15.105:0/241586363 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c4071960 0x7f91c415f3b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f91b40097f0 tx=0x7f91b401c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a1be27290f1d218 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:45.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.556+0000 7f91bbfff640 1 -- 172.21.15.105:0/241586363 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b40aa5f0 con 0x7f91c4071960 2023-11-28T18:31:45.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.556+0000 7f91bbfff640 1 -- 172.21.15.105:0/241586363 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b40aa750 con 0x7f91c4071960 2023-11-28T18:31:45.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.556+0000 7f91bbfff640 1 -- 172.21.15.105:0/241586363 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b40aaae0 con 0x7f91c4071960 2023-11-28T18:31:45.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.556+0000 7f91cb39d640 1 -- 172.21.15.105:0/241586363 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c4071960 msgr2=0x7f91c415f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:45.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.556+0000 7f91cb39d640 1 --2- 172.21.15.105:0/241586363 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c4071960 0x7f91c415f3b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f91b40097f0 tx=0x7f91b401c490 comp rx=0 tx=0).stop 2023-11-28T18:31:45.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91cb39d640 1 -- 172.21.15.105:0/241586363 shutdown_connections 2023-11-28T18:31:45.559 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91cb39d640 1 --2- 172.21.15.105:0/241586363 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c4071960 0x7f91c415f3b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:45.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91cb39d640 1 -- 172.21.15.105:0/241586363 >> 172.21.15.105:0/241586363 conn(0x7f91c40ceb90 msgr2=0x7f91c40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:45.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91cb39d640 1 -- 172.21.15.105:0/241586363 shutdown_connections 2023-11-28T18:31:45.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91cb39d640 1 -- 172.21.15.105:0/241586363 wait complete. 2023-11-28T18:31:45.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91cb39d640 1 Processor -- start 2023-11-28T18:31:45.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91cb39d640 1 -- start start 2023-11-28T18:31:45.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91cb39d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c40b4290 0x7f91c40b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:45.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91cb39d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91c415b030 0x7f91c4152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:45.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91cb39d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c41534c0 0x7f91c40b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:45.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91cb39d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c4154490 con 0x7f91c415b030 2023-11-28T18:31:45.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91cb39d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c409c3c0 con 0x7f91c41534c0 2023-11-28T18:31:45.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91cb39d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c409c850 con 0x7f91c40b4290 2023-11-28T18:31:45.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91c8938640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91c415b030 0x7f91c4152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91c8938640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91c415b030 0x7f91c4152f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50172/0 (socket says 172.21.15.105:50172) 2023-11-28T18:31:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91c8938640 1 -- 172.21.15.105:0/1829856558 learned_addr learned my addr 172.21.15.105:0/1829856558 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91c9139640 1 --2- 172.21.15.105:0/1829856558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c40b4290 0x7f91c40b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91c993a640 1 --2- 172.21.15.105:0/1829856558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c41534c0 0x7f91c40b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91c8938640 1 -- 172.21.15.105:0/1829856558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c40b4290 msgr2=0x7f91c40b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91c8938640 1 --2- 172.21.15.105:0/1829856558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c40b4290 0x7f91c40b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91c8938640 1 -- 172.21.15.105:0/1829856558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c41534c0 msgr2=0x7f91c40b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91c8938640 1 --2- 172.21.15.105:0/1829856558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c41534c0 0x7f91c40b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91c8938640 1 -- 172.21.15.105:0/1829856558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b4009420 con 0x7f91c415b030 2023-11-28T18:31:45.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91c993a640 1 --2- 172.21.15.105:0/1829856558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c41534c0 0x7f91c40b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:45.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.560+0000 7f91c8938640 1 --2- 172.21.15.105:0/1829856558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91c415b030 0x7f91c4152f80 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f91b40190a0 tx=0x7f91b401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:45.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.564+0000 7f91b9ffb640 1 -- 172.21.15.105:0/1829856558 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b40be610 con 0x7f91c415b030 2023-11-28T18:31:45.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.564+0000 7f91b9ffb640 1 -- 172.21.15.105:0/1829856558 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b40be790 con 0x7f91c415b030 2023-11-28T18:31:45.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.564+0000 7f91b9ffb640 1 -- 172.21.15.105:0/1829856558 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b40beb40 con 0x7f91c415b030 2023-11-28T18:31:45.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.564+0000 7f91cb39d640 1 -- 172.21.15.105:0/1829856558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c41541b0 con 0x7f91c415b030 2023-11-28T18:31:45.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.564+0000 7f91cb39d640 1 -- 172.21.15.105:0/1829856558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c4153950 con 0x7f91c415b030 2023-11-28T18:31:45.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.564+0000 7f91cb39d640 1 -- 172.21.15.105:0/1829856558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c409c170 con 0x7f91c415b030 2023-11-28T18:31:45.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.564+0000 7f91b9ffb640 1 -- 172.21.15.105:0/1829856558 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f91b40b4400 con 0x7f91c415b030 2023-11-28T18:31:45.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.564+0000 7f91b9ffb640 1 --2- 172.21.15.105:0/1829856558 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f91a8066900 0x7f91a8068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:45.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.564+0000 7f91b9ffb640 1 -- 172.21.15.105:0/1829856558 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f91b40a7070 con 0x7f91c415b030 2023-11-28T18:31:45.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.564+0000 7f91c9139640 1 --2- 172.21.15.105:0/1829856558 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f91a8066900 0x7f91a8068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:45.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.564+0000 7f91c9139640 1 --2- 172.21.15.105:0/1829856558 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f91a8066900 0x7f91a8068dc0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f91c0007fd0 tx=0x7f91c0007d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:45.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.568+0000 7f91b9ffb640 1 -- 172.21.15.105:0/1829856558 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f91b40ebf50 con 0x7f91c415b030 2023-11-28T18:31:45.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:45.748+0000 7f91cb39d640 1 -- 172.21.15.105:0/1829856558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f91c409c530 con 0x7f91c415b030 2023-11-28T18:31:46.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.380+0000 7f91b9ffb640 1 -- 172.21.15.105:0/1829856558 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v128) v1 ==== 104+0+0 (secure 0 0 0) 0x7f91b40ef570 con 0x7f91c415b030 2023-11-28T18:31:46.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.384+0000 7f91cb39d640 1 -- 172.21.15.105:0/1829856558 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f91a8066900 msgr2=0x7f91a8068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:46.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.384+0000 7f91cb39d640 1 --2- 172.21.15.105:0/1829856558 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f91a8066900 0x7f91a8068dc0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f91c0007fd0 tx=0x7f91c0007d80 comp rx=0 tx=0).stop 2023-11-28T18:31:46.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.384+0000 7f91cb39d640 1 -- 172.21.15.105:0/1829856558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91c415b030 msgr2=0x7f91c4152f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:46.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.384+0000 7f91cb39d640 1 --2- 172.21.15.105:0/1829856558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91c415b030 0x7f91c4152f80 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f91b40190a0 tx=0x7f91b401acd0 comp rx=0 tx=0).stop 2023-11-28T18:31:46.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.384+0000 7f91cb39d640 1 -- 172.21.15.105:0/1829856558 shutdown_connections 2023-11-28T18:31:46.386 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.384+0000 7f91cb39d640 1 --2- 172.21.15.105:0/1829856558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c41534c0 0x7f91c40b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:46.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.384+0000 7f91cb39d640 1 --2- 172.21.15.105:0/1829856558 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f91a8066900 0x7f91a8068dc0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:46.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.384+0000 7f91cb39d640 1 --2- 172.21.15.105:0/1829856558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91c415b030 0x7f91c4152f80 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:46.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.384+0000 7f91cb39d640 1 --2- 172.21.15.105:0/1829856558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91c40b4290 0x7f91c40b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:46.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.384+0000 7f91cb39d640 1 -- 172.21.15.105:0/1829856558 >> 172.21.15.105:0/1829856558 conn(0x7f91c40ceb90 msgr2=0x7f91c40adf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:46.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.384+0000 7f91cb39d640 1 -- 172.21.15.105:0/1829856558 shutdown_connections 2023-11-28T18:31:46.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.384+0000 7f91cb39d640 1 -- 172.21.15.105:0/1829856558 wait complete. 2023-11-28T18:31:46.408 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T18:31:46.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.592+0000 7fadb9a5f640 1 Processor -- start 2023-11-28T18:31:46.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.592+0000 7fadb9a5f640 1 -- start start 2023-11-28T18:31:46.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.592+0000 7fadb9a5f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadb40ce2c0 0x7fadb406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:46.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.592+0000 7fadb9a5f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fadb406cd20 0x7fadb40abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.592+0000 7fadb9a5f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadb40cde30 con 0x7fadb4159610 2023-11-28T18:31:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.592+0000 7fadb9a5f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb406d1b0 con 0x7fadb406cd20 2023-11-28T18:31:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.592+0000 7fadb9a5f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb41599f0 con 0x7fadb40ce2c0 2023-11-28T18:31:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.592+0000 7fadb27fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadb40ce2c0 0x7fadb406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.592+0000 7fadb37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fadb406cd20 0x7fadb40abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.592+0000 7fadb2ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fadb4159610 0x7fadb40cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:58684/0 (socket says 172.21.15.105:58684) 2023-11-28T18:31:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.592+0000 7fadb2ffd640 1 -- 172.21.15.105:0/2535543412 learned_addr learned my addr 172.21.15.105:0/2535543412 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.596+0000 7fadb27fc640 1 -- 172.21.15.105:0/2535543412 >> v2:172.21.15.148:3300/0 conn(0x7fadb406cd20 msgr2=0x7fadb40abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.596+0000 7fadb27fc640 1 --2- 172.21.15.105:0/2535543412 >> v2:172.21.15.148:3300/0 conn(0x7fadb406cd20 0x7fadb40abd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.596+0000 7fadb27fc640 1 -- 172.21.15.105:0/2535543412 >> v1:172.21.15.148:6789/0 conn(0x7fadb4159610 legacy=0x7fadb40cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:46.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.596+0000 7fadb27fc640 1 -- 172.21.15.105:0/2535543412 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb406de20 con 0x7fadb40ce2c0 2023-11-28T18:31:46.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.596+0000 7fadb37fe640 1 --2- 172.21.15.105:0/2535543412 >> v2:172.21.15.148:3300/0 conn(0x7fadb406cd20 0x7fadb40abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:31:46.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.596+0000 7fadb27fc640 1 --2- 172.21.15.105:0/2535543412 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadb40ce2c0 0x7fadb406c7e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fada80097c0 tx=0x7fada801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31a250b2e837a2f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:46.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.596+0000 7fadb1ffb640 1 -- 172.21.15.105:0/2535543412 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada80ab600 con 0x7fadb40ce2c0 2023-11-28T18:31:46.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.596+0000 7fadb1ffb640 1 -- 172.21.15.105:0/2535543412 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada80a7050 con 0x7fadb40ce2c0 2023-11-28T18:31:46.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.596+0000 7fadb1ffb640 1 -- 172.21.15.105:0/2535543412 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada80ab980 con 0x7fadb40ce2c0 2023-11-28T18:31:46.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb9a5f640 1 -- 172.21.15.105:0/2535543412 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadb40ce2c0 msgr2=0x7fadb406c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:46.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb9a5f640 1 --2- 172.21.15.105:0/2535543412 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadb40ce2c0 0x7fadb406c7e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fada80097c0 tx=0x7fada801c450 comp rx=0 tx=0).stop 2023-11-28T18:31:46.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb9a5f640 1 -- 172.21.15.105:0/2535543412 shutdown_connections 2023-11-28T18:31:46.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb9a5f640 1 --2- 172.21.15.105:0/2535543412 >> v2:172.21.15.148:3300/0 conn(0x7fadb406cd20 0x7fadb40abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:46.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb9a5f640 1 --2- 172.21.15.105:0/2535543412 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadb40ce2c0 0x7fadb406c7e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:46.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb9a5f640 1 -- 172.21.15.105:0/2535543412 >> 172.21.15.105:0/2535543412 conn(0x7fadb40ceb90 msgr2=0x7fadb40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:46.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb9a5f640 1 -- 172.21.15.105:0/2535543412 shutdown_connections 2023-11-28T18:31:46.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb9a5f640 1 -- 172.21.15.105:0/2535543412 wait complete. 2023-11-28T18:31:46.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb9a5f640 1 Processor -- start 2023-11-28T18:31:46.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb9a5f640 1 -- start start 2023-11-28T18:31:46.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb9a5f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb406cd20 0x7fadb4158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:46.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb9a5f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadb40ce2c0 0x7fadb4159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:46.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb9a5f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadb4159610 0x7fadb415a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:46.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb9a5f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb40b47f0 con 0x7fadb40ce2c0 2023-11-28T18:31:46.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb9a5f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb40b2240 con 0x7fadb406cd20 2023-11-28T18:31:46.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb9a5f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb40b24b0 con 0x7fadb4159610 2023-11-28T18:31:46.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadb40ce2c0 0x7fadb4159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:46.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadb40ce2c0 0x7fadb4159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50174/0 (socket says 172.21.15.105:50174) 2023-11-28T18:31:46.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb27fc640 1 -- 172.21.15.105:0/473509279 learned_addr learned my addr 172.21.15.105:0/473509279 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:46.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb2ffd640 1 --2- 172.21.15.105:0/473509279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb406cd20 0x7fadb4158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:46.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb37fe640 1 --2- 172.21.15.105:0/473509279 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadb4159610 0x7fadb415a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:46.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb27fc640 1 -- 172.21.15.105:0/473509279 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadb4159610 msgr2=0x7fadb415a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:46.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb27fc640 1 --2- 172.21.15.105:0/473509279 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadb4159610 0x7fadb415a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:46.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb27fc640 1 -- 172.21.15.105:0/473509279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb406cd20 msgr2=0x7fadb4158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:46.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb27fc640 1 --2- 172.21.15.105:0/473509279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb406cd20 0x7fadb4158500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:46.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb27fc640 1 -- 172.21.15.105:0/473509279 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadac009a70 con 0x7fadb40ce2c0 2023-11-28T18:31:46.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb37fe640 1 --2- 172.21.15.105:0/473509279 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadb4159610 0x7fadb415a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:31:46.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.600+0000 7fadb27fc640 1 --2- 172.21.15.105:0/473509279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadb40ce2c0 0x7fadb4159b30 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fada801a080 tx=0x7fada80191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:46.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.604+0000 7fad9bfff640 1 -- 172.21.15.105:0/473509279 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada80b4410 con 0x7fadb40ce2c0 2023-11-28T18:31:46.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.604+0000 7fad9bfff640 1 -- 172.21.15.105:0/473509279 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada80b4590 con 0x7fadb40ce2c0 2023-11-28T18:31:46.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.604+0000 7fad9bfff640 1 -- 172.21.15.105:0/473509279 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada80b4940 con 0x7fadb40ce2c0 2023-11-28T18:31:46.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.604+0000 7fadb9a5f640 1 -- 172.21.15.105:0/473509279 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada8009420 con 0x7fadb40ce2c0 2023-11-28T18:31:46.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.604+0000 7fadb9a5f640 1 -- 172.21.15.105:0/473509279 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb40ccff0 con 0x7fadb40ce2c0 2023-11-28T18:31:46.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.604+0000 7fadb9a5f640 1 -- 172.21.15.105:0/473509279 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb4067570 con 0x7fadb40ce2c0 2023-11-28T18:31:46.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.604+0000 7fad9bfff640 1 -- 172.21.15.105:0/473509279 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fada80b2040 con 0x7fadb40ce2c0 2023-11-28T18:31:46.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.604+0000 7fad9bfff640 1 --2- 172.21.15.105:0/473509279 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fada00668d0 0x7fada0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:46.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.604+0000 7fad9bfff640 1 -- 172.21.15.105:0/473509279 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6287+0+0 (secure 0 0 0) 0x7fada811e4d0 con 0x7fadb40ce2c0 2023-11-28T18:31:46.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.604+0000 7fadb2ffd640 1 --2- 172.21.15.105:0/473509279 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fada00668d0 0x7fada0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:46.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.604+0000 7fadb2ffd640 1 --2- 172.21.15.105:0/473509279 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fada00668d0 0x7fada0068d90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fada4005890 tx=0x7fada4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:46.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.608+0000 7fad9bfff640 1 -- 172.21.15.105:0/473509279 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fada80eb5f0 con 0x7fadb40ce2c0 2023-11-28T18:31:46.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:46.760+0000 7fadb9a5f640 1 -- 172.21.15.105:0/473509279 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fadb40cf950 con 0x7fadb40ce2c0 2023-11-28T18:31:47.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.384+0000 7fad9bfff640 1 -- 172.21.15.105:0/473509279 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v119) v1 ==== 181+0+0 (secure 0 0 0) 0x7fada80b00d0 con 0x7fadb40ce2c0 2023-11-28T18:31:47.384 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' removed 2023-11-28T18:31:47.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.384+0000 7fadb9a5f640 1 -- 172.21.15.105:0/473509279 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fada00668d0 msgr2=0x7fada0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:47.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.384+0000 7fadb9a5f640 1 --2- 172.21.15.105:0/473509279 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fada00668d0 0x7fada0068d90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fada4005890 tx=0x7fada4009040 comp rx=0 tx=0).stop 2023-11-28T18:31:47.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.384+0000 7fadb9a5f640 1 -- 172.21.15.105:0/473509279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadb40ce2c0 msgr2=0x7fadb4159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:47.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.384+0000 7fadb9a5f640 1 --2- 172.21.15.105:0/473509279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadb40ce2c0 0x7fadb4159b30 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fada801a080 tx=0x7fada80191f0 comp rx=0 tx=0).stop 2023-11-28T18:31:47.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.384+0000 7fadb9a5f640 1 -- 172.21.15.105:0/473509279 shutdown_connections 2023-11-28T18:31:47.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.384+0000 7fadb9a5f640 1 --2- 172.21.15.105:0/473509279 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadb4159610 0x7fadb415a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:47.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.384+0000 7fadb9a5f640 1 --2- 172.21.15.105:0/473509279 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fada00668d0 0x7fada0068d90 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:47.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.384+0000 7fadb9a5f640 1 --2- 172.21.15.105:0/473509279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadb40ce2c0 0x7fadb4159b30 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:47.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.384+0000 7fadb9a5f640 1 --2- 172.21.15.105:0/473509279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb406cd20 0x7fadb4158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:47.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.384+0000 7fadb9a5f640 1 -- 172.21.15.105:0/473509279 >> 172.21.15.105:0/473509279 conn(0x7fadb40ceb90 msgr2=0x7fadb40abe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:47.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.388+0000 7fadb9a5f640 1 -- 172.21.15.105:0/473509279 shutdown_connections 2023-11-28T18:31:47.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.388+0000 7fadb9a5f640 1 -- 172.21.15.105:0/473509279 wait complete. 2023-11-28T18:31:47.415 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T18:31:47.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.556+0000 7ff064fe6640 1 Processor -- start 2023-11-28T18:31:47.558 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.556+0000 7ff064fe6640 1 -- start start 2023-11-28T18:31:47.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.560+0000 7ff064fe6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff060159f40 0x7ff0600caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:47.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.560+0000 7ff064fe6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff0600ad850 0x7ff0600701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:47.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.560+0000 7ff064fe6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff06006bf50 con 0x7ff0600cc760 2023-11-28T18:31:47.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.560+0000 7ff064fe6640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0600ccbc0 con 0x7ff0600ad850 2023-11-28T18:31:47.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.560+0000 7ff064fe6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06015a320 con 0x7ff060159f40 2023-11-28T18:31:47.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.560+0000 7ff05ed9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff0600ad850 0x7ff0600701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:47.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.560+0000 7ff057fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff0600cc760 0x7ff0600ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:58696/0 (socket says 172.21.15.105:58696) 2023-11-28T18:31:47.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.560+0000 7ff057fff640 1 -- 172.21.15.105:0/987255790 learned_addr learned my addr 172.21.15.105:0/987255790 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:47.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.560+0000 7ff05ed9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff0600ad850 0x7ff0600701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50182/0 (socket says 172.21.15.105:50182) 2023-11-28T18:31:47.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.560+0000 7ff05e59c640 1 --2- 172.21.15.105:0/987255790 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff060159f40 0x7ff0600caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:47.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.560+0000 7ff05ed9d640 1 -- 172.21.15.105:0/987255790 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff060159f40 msgr2=0x7ff0600caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:47.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.560+0000 7ff05ed9d640 1 --2- 172.21.15.105:0/987255790 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff060159f40 0x7ff0600caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:47.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.560+0000 7ff05ed9d640 1 -- 172.21.15.105:0/987255790 >> v1:172.21.15.148:6789/0 conn(0x7ff0600cc760 legacy=0x7ff0600ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:47.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.560+0000 7ff05ed9d640 1 -- 172.21.15.105:0/987255790 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff060071380 con 0x7ff0600ad850 2023-11-28T18:31:47.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.564+0000 7ff05ed9d640 1 --2- 172.21.15.105:0/987255790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0600ad850 0x7ff0600701d0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7ff058009d90 tx=0x7ff05801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29bae9823585b3ae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:47.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.564+0000 7ff05dd9b640 1 -- 172.21.15.105:0/987255790 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0580a5950 con 0x7ff0600ad850 2023-11-28T18:31:47.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.564+0000 7ff05dd9b640 1 -- 172.21.15.105:0/987255790 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0580a5ab0 con 0x7ff0600ad850 2023-11-28T18:31:47.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.564+0000 7ff05dd9b640 1 -- 172.21.15.105:0/987255790 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0580a5e40 con 0x7ff0600ad850 2023-11-28T18:31:47.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.564+0000 7ff064fe6640 1 -- 172.21.15.105:0/987255790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0600ad850 msgr2=0x7ff0600701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:47.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.564+0000 7ff064fe6640 1 --2- 172.21.15.105:0/987255790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0600ad850 0x7ff0600701d0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7ff058009d90 tx=0x7ff05801c890 comp rx=0 tx=0).stop 2023-11-28T18:31:47.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.564+0000 7ff064fe6640 1 -- 172.21.15.105:0/987255790 shutdown_connections 2023-11-28T18:31:47.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.564+0000 7ff064fe6640 1 --2- 172.21.15.105:0/987255790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0600ad850 0x7ff0600701d0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:47.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.564+0000 7ff064fe6640 1 --2- 172.21.15.105:0/987255790 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff060159f40 0x7ff0600caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:47.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.564+0000 7ff064fe6640 1 -- 172.21.15.105:0/987255790 >> 172.21.15.105:0/987255790 conn(0x7ff0600ceb90 msgr2=0x7ff0600cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:47.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.564+0000 7ff064fe6640 1 -- 172.21.15.105:0/987255790 shutdown_connections 2023-11-28T18:31:47.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.564+0000 7ff064fe6640 1 -- 172.21.15.105:0/987255790 wait complete. 2023-11-28T18:31:47.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.564+0000 7ff064fe6640 1 Processor -- start 2023-11-28T18:31:47.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.564+0000 7ff064fe6640 1 -- start start 2023-11-28T18:31:47.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff064fe6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0600ad850 0x7ff0600b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:47.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff064fe6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0600cc760 0x7ff06014d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:47.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff064fe6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff060159f40 0x7ff0600af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:47.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff064fe6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0600b49b0 con 0x7ff0600ad850 2023-11-28T18:31:47.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff064fe6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0600b2530 con 0x7ff060159f40 2023-11-28T18:31:47.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff064fe6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0600b27d0 con 0x7ff0600cc760 2023-11-28T18:31:47.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff05e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0600ad850 0x7ff0600b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff05e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0600ad850 0x7ff0600b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50188/0 (socket says 172.21.15.105:50188) 2023-11-28T18:31:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff05e59c640 1 -- 172.21.15.105:0/5636732 learned_addr learned my addr 172.21.15.105:0/5636732 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff05ed9d640 1 --2- 172.21.15.105:0/5636732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff060159f40 0x7ff0600af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff057fff640 1 --2- 172.21.15.105:0/5636732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0600cc760 0x7ff06014d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff05e59c640 1 -- 172.21.15.105:0/5636732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0600cc760 msgr2=0x7ff06014d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff05e59c640 1 --2- 172.21.15.105:0/5636732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0600cc760 0x7ff06014d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff05e59c640 1 -- 172.21.15.105:0/5636732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff060159f40 msgr2=0x7ff0600af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff05e59c640 1 --2- 172.21.15.105:0/5636732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff060159f40 0x7ff0600af1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff05e59c640 1 -- 172.21.15.105:0/5636732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff058009a70 con 0x7ff0600ad850 2023-11-28T18:31:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff05e59c640 1 --2- 172.21.15.105:0/5636732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0600ad850 0x7ff0600b3590 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7ff05000e870 tx=0x7ff05000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff056ffd640 1 -- 172.21.15.105:0/5636732 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050009510 con 0x7ff0600ad850 2023-11-28T18:31:47.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff056ffd640 1 -- 172.21.15.105:0/5636732 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05000eec0 con 0x7ff0600ad850 2023-11-28T18:31:47.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff056ffd640 1 -- 172.21.15.105:0/5636732 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05000cb40 con 0x7ff0600ad850 2023-11-28T18:31:47.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff064fe6640 1 -- 172.21.15.105:0/5636732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0600b2ab0 con 0x7ff0600ad850 2023-11-28T18:31:47.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff064fe6640 1 -- 172.21.15.105:0/5636732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0600b3070 con 0x7ff0600ad850 2023-11-28T18:31:47.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.568+0000 7ff064fe6640 1 -- 172.21.15.105:0/5636732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff060067570 con 0x7ff0600ad850 2023-11-28T18:31:47.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.572+0000 7ff056ffd640 1 -- 172.21.15.105:0/5636732 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff05000ccc0 con 0x7ff0600ad850 2023-11-28T18:31:47.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.572+0000 7ff056ffd640 1 --2- 172.21.15.105:0/5636732 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff0400668d0 0x7ff040068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:47.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.572+0000 7ff056ffd640 1 -- 172.21.15.105:0/5636732 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5849+0+0 (secure 0 0 0) 0x7ff050084600 con 0x7ff0600ad850 2023-11-28T18:31:47.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.572+0000 7ff057fff640 1 --2- 172.21.15.105:0/5636732 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff0400668d0 0x7ff040068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:47.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.572+0000 7ff057fff640 1 --2- 172.21.15.105:0/5636732 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff0400668d0 0x7ff040068d90 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff0601f7760 tx=0x7ff048009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:47.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.576+0000 7ff056ffd640 1 -- 172.21.15.105:0/5636732 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff0500517c0 con 0x7ff0600ad850 2023-11-28T18:31:47.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:47.724+0000 7ff064fe6640 1 -- 172.21.15.105:0/5636732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff0600cb280 con 0x7ff0600ad850 2023-11-28T18:31:48.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.392+0000 7ff056ffd640 1 -- 172.21.15.105:0/5636732 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v120) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff050054de0 con 0x7ff0600ad850 2023-11-28T18:31:48.394 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' removed 2023-11-28T18:31:48.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.400+0000 7ff064fe6640 1 -- 172.21.15.105:0/5636732 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff0400668d0 msgr2=0x7ff040068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:48.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.400+0000 7ff064fe6640 1 --2- 172.21.15.105:0/5636732 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff0400668d0 0x7ff040068d90 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff0601f7760 tx=0x7ff048009040 comp rx=0 tx=0).stop 2023-11-28T18:31:48.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.400+0000 7ff064fe6640 1 -- 172.21.15.105:0/5636732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0600ad850 msgr2=0x7ff0600b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:48.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.400+0000 7ff064fe6640 1 --2- 172.21.15.105:0/5636732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0600ad850 0x7ff0600b3590 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7ff05000e870 tx=0x7ff05000ec70 comp rx=0 tx=0).stop 2023-11-28T18:31:48.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.400+0000 7ff064fe6640 1 -- 172.21.15.105:0/5636732 shutdown_connections 2023-11-28T18:31:48.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.400+0000 7ff064fe6640 1 --2- 172.21.15.105:0/5636732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff060159f40 0x7ff0600af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:48.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.400+0000 7ff064fe6640 1 --2- 172.21.15.105:0/5636732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0600cc760 0x7ff06014d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:48.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.400+0000 7ff064fe6640 1 --2- 172.21.15.105:0/5636732 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff0400668d0 0x7ff040068d90 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:48.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.400+0000 7ff064fe6640 1 --2- 172.21.15.105:0/5636732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0600ad850 0x7ff0600b3590 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:48.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.400+0000 7ff064fe6640 1 -- 172.21.15.105:0/5636732 >> 172.21.15.105:0/5636732 conn(0x7ff0600ceb90 msgr2=0x7ff060072eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:48.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.400+0000 7ff064fe6640 1 -- 172.21.15.105:0/5636732 shutdown_connections 2023-11-28T18:31:48.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.400+0000 7ff064fe6640 1 -- 172.21.15.105:0/5636732 wait complete. 2023-11-28T18:31:48.425 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random 2023-11-28T18:31:48.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.564+0000 7f377ecc9640 1 Processor -- start 2023-11-28T18:31:48.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.564+0000 7f377ecc9640 1 -- start start 2023-11-28T18:31:48.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.564+0000 7f377ecc9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f377806bf50 0x7f377815d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:48.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.564+0000 7f377ecc9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f37780adb80 0x7f37780adff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:48.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.564+0000 7f377ecc9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37780cc7f0 con 0x7f3778154070 2023-11-28T18:31:48.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.564+0000 7f377ecc9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3778072540 con 0x7f37780adb80 2023-11-28T18:31:48.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.564+0000 7f377ecc9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37780ae530 con 0x7f377806bf50 2023-11-28T18:31:48.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.568+0000 7f376ffff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f377806bf50 0x7f377815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:48.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.568+0000 7f377d266640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f37780adb80 0x7f37780adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:48.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.568+0000 7f377ca65640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3778154070 0x7f37780ce310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:58712/0 (socket says 172.21.15.105:58712) 2023-11-28T18:31:48.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.568+0000 7f377ca65640 1 -- 172.21.15.105:0/2230736091 learned_addr learned my addr 172.21.15.105:0/2230736091 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:48.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.568+0000 7f377d266640 1 -- 172.21.15.105:0/2230736091 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f377806bf50 msgr2=0x7f377815d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:48.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.568+0000 7f377d266640 1 --2- 172.21.15.105:0/2230736091 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f377806bf50 0x7f377815d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:48.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.568+0000 7f377d266640 1 -- 172.21.15.105:0/2230736091 >> v1:172.21.15.148:6789/0 conn(0x7f3778154070 legacy=0x7f37780ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:48.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.568+0000 7f377d266640 1 -- 172.21.15.105:0/2230736091 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377815dd10 con 0x7f37780adb80 2023-11-28T18:31:48.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.568+0000 7f376ffff640 1 --2- 172.21.15.105:0/2230736091 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f377806bf50 0x7f377815d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:31:48.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.568+0000 7f377d266640 1 --2- 172.21.15.105:0/2230736091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37780adb80 0x7f37780adff0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3770009d90 tx=0x7f377001c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14405bdec99e01e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:48.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.568+0000 7f376f7fe640 1 -- 172.21.15.105:0/2230736091 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37700a6620 con 0x7f37780adb80 2023-11-28T18:31:48.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.568+0000 7f376f7fe640 1 -- 172.21.15.105:0/2230736091 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37700a6780 con 0x7f37780adb80 2023-11-28T18:31:48.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.568+0000 7f376f7fe640 1 -- 172.21.15.105:0/2230736091 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37700a6b10 con 0x7f37780adb80 2023-11-28T18:31:48.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.572+0000 7f377ecc9640 1 -- 172.21.15.105:0/2230736091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37780adb80 msgr2=0x7f37780adff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:48.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.572+0000 7f377ecc9640 1 --2- 172.21.15.105:0/2230736091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37780adb80 0x7f37780adff0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3770009d90 tx=0x7f377001c920 comp rx=0 tx=0).stop 2023-11-28T18:31:48.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.572+0000 7f377ecc9640 1 -- 172.21.15.105:0/2230736091 shutdown_connections 2023-11-28T18:31:48.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.572+0000 7f377ecc9640 1 --2- 172.21.15.105:0/2230736091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37780adb80 0x7f37780adff0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:48.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.572+0000 7f377ecc9640 1 --2- 172.21.15.105:0/2230736091 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f377806bf50 0x7f377815d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:48.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.572+0000 7f377ecc9640 1 -- 172.21.15.105:0/2230736091 >> 172.21.15.105:0/2230736091 conn(0x7f37780ceb90 msgr2=0x7f37780cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:48.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.572+0000 7f377ecc9640 1 -- 172.21.15.105:0/2230736091 shutdown_connections 2023-11-28T18:31:48.574 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.572+0000 7f377ecc9640 1 -- 172.21.15.105:0/2230736091 wait complete. 2023-11-28T18:31:48.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.572+0000 7f377ecc9640 1 Processor -- start 2023-11-28T18:31:48.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.572+0000 7f377ecc9640 1 -- start start 2023-11-28T18:31:48.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.572+0000 7f377ecc9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f377806bf50 0x7f377814dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:48.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.572+0000 7f377ecc9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37780adb80 0x7f377814b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:48.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.572+0000 7f377ecc9640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3778154070 0x7f377814b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:48.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.572+0000 7f377ecc9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377814b070 con 0x7f37780adb80 2023-11-28T18:31:48.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.572+0000 7f377ecc9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377814beb0 con 0x7f377806bf50 2023-11-28T18:31:48.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.572+0000 7f377ecc9640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f377814c190 con 0x7f3778154070 2023-11-28T18:31:48.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.572+0000 7f376ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37780adb80 0x7f377814b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:48.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.572+0000 7f376ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37780adb80 0x7f377814b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50192/0 (socket says 172.21.15.105:50192) 2023-11-28T18:31:48.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.572+0000 7f376ffff640 1 -- 172.21.15.105:0/3136368561 learned_addr learned my addr 172.21.15.105:0/3136368561 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:48.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.572+0000 7f377ca65640 1 --2- 172.21.15.105:0/3136368561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f377806bf50 0x7f377814dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:48.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.572+0000 7f377d266640 1 --2- 172.21.15.105:0/3136368561 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3778154070 0x7f377814b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:48.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.576+0000 7f376ffff640 1 -- 172.21.15.105:0/3136368561 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3778154070 msgr2=0x7f377814b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:48.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.576+0000 7f376ffff640 1 --2- 172.21.15.105:0/3136368561 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3778154070 0x7f377814b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:48.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.576+0000 7f376ffff640 1 -- 172.21.15.105:0/3136368561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f377806bf50 msgr2=0x7f377814dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:48.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.576+0000 7f376ffff640 1 --2- 172.21.15.105:0/3136368561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f377806bf50 0x7f377814dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:48.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.576+0000 7f376ffff640 1 -- 172.21.15.105:0/3136368561 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3768009420 con 0x7f37780adb80 2023-11-28T18:31:48.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.576+0000 7f376ffff640 1 --2- 172.21.15.105:0/3136368561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37780adb80 0x7f377814b340 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f376800e9b0 tx=0x7f376800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:48.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.576+0000 7f376d7fa640 1 -- 172.21.15.105:0/3136368561 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376800ca00 con 0x7f37780adb80 2023-11-28T18:31:48.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.576+0000 7f376d7fa640 1 -- 172.21.15.105:0/3136368561 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376800cb60 con 0x7f37780adb80 2023-11-28T18:31:48.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.576+0000 7f377ecc9640 1 -- 172.21.15.105:0/3136368561 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3770009a70 con 0x7f37780adb80 2023-11-28T18:31:48.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.576+0000 7f376d7fa640 1 -- 172.21.15.105:0/3136368561 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3768013470 con 0x7f37780adb80 2023-11-28T18:31:48.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.576+0000 7f377ecc9640 1 -- 172.21.15.105:0/3136368561 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f377814c750 con 0x7f37780adb80 2023-11-28T18:31:48.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.576+0000 7f377ecc9640 1 -- 172.21.15.105:0/3136368561 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3778067570 con 0x7f37780adb80 2023-11-28T18:31:48.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.576+0000 7f376d7fa640 1 -- 172.21.15.105:0/3136368561 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f37680135d0 con 0x7f37780adb80 2023-11-28T18:31:48.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.576+0000 7f376d7fa640 1 --2- 172.21.15.105:0/3136368561 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f37640668d0 0x7f3764068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:48.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.576+0000 7f376d7fa640 1 -- 172.21.15.105:0/3136368561 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f37680845d0 con 0x7f37780adb80 2023-11-28T18:31:48.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.576+0000 7f377ca65640 1 --2- 172.21.15.105:0/3136368561 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f37640668d0 0x7f3764068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:48.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.580+0000 7f377ca65640 1 --2- 172.21.15.105:0/3136368561 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f37640668d0 0x7f3764068d90 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3774005890 tx=0x7f3774009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:48.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.580+0000 7f376d7fa640 1 -- 172.21.15.105:0/3136368561 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3768051950 con 0x7f37780adb80 2023-11-28T18:31:48.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.732+0000 7f377ecc9640 1 -- 172.21.15.105:0/3136368561 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"} v 0) v1 -- 0x7f37780cf950 con 0x7f37780adb80 2023-11-28T18:31:48.737 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.736+0000 7f376d7fa640 1 -- 172.21.15.105:0/3136368561 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"}]=0 v63) v1 ==== 110+0+0 (secure 0 0 0) 0x7f3768054f70 con 0x7f37780adb80 2023-11-28T18:31:48.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.736+0000 7f377ecc9640 1 -- 172.21.15.105:0/3136368561 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f37640668d0 msgr2=0x7f3764068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:48.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.736+0000 7f377ecc9640 1 --2- 172.21.15.105:0/3136368561 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f37640668d0 0x7f3764068d90 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3774005890 tx=0x7f3774009040 comp rx=0 tx=0).stop 2023-11-28T18:31:48.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.736+0000 7f377ecc9640 1 -- 172.21.15.105:0/3136368561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37780adb80 msgr2=0x7f377814b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:48.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.736+0000 7f377ecc9640 1 --2- 172.21.15.105:0/3136368561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37780adb80 0x7f377814b340 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f376800e9b0 tx=0x7f376800ee70 comp rx=0 tx=0).stop 2023-11-28T18:31:48.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.740+0000 7f377ecc9640 1 -- 172.21.15.105:0/3136368561 shutdown_connections 2023-11-28T18:31:48.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.740+0000 7f377ecc9640 1 --2- 172.21.15.105:0/3136368561 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3778154070 0x7f377814b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:48.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.740+0000 7f377ecc9640 1 --2- 172.21.15.105:0/3136368561 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f37640668d0 0x7f3764068d90 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:48.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.740+0000 7f377ecc9640 1 --2- 172.21.15.105:0/3136368561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37780adb80 0x7f377814b340 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:48.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.740+0000 7f377ecc9640 1 --2- 172.21.15.105:0/3136368561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f377806bf50 0x7f377814dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:48.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.740+0000 7f377ecc9640 1 -- 172.21.15.105:0/3136368561 >> 172.21.15.105:0/3136368561 conn(0x7f37780ceb90 msgr2=0x7f37780cdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:48.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.740+0000 7f377ecc9640 1 -- 172.21.15.105:0/3136368561 shutdown_connections 2023-11-28T18:31:48.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.740+0000 7f377ecc9640 1 -- 172.21.15.105:0/3136368561 wait complete. 2023-11-28T18:31:48.758 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T18:31:48.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.904+0000 7f5dc7279640 1 Processor -- start 2023-11-28T18:31:48.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.904+0000 7f5dc7279640 1 -- start start 2023-11-28T18:31:48.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.904+0000 7f5dc7279640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5dc00ad9f0 0x7f5dc00ade80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:48.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.904+0000 7f5dc7279640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc0162820 con 0x7f5dc00cc7d0 2023-11-28T18:31:48.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.904+0000 7f5dc7279640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc0161c10 con 0x7f5dc006d610 2023-11-28T18:31:48.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.904+0000 7f5dc7279640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc00ae3c0 con 0x7f5dc00ad9f0 2023-11-28T18:31:48.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.904+0000 7f5dc4814640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5dc00ad9f0 0x7f5dc00ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:48.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.904+0000 7f5dc5816640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5dc00cc7d0 0x7f5dc00ccbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49848/0 (socket says 172.21.15.105:49848) 2023-11-28T18:31:48.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.904+0000 7f5dc4814640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5dc00ad9f0 0x7f5dc00ade80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50204/0 (socket says 172.21.15.105:50204) 2023-11-28T18:31:48.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.904+0000 7f5dc5816640 1 -- 172.21.15.105:0/3686236795 learned_addr learned my addr 172.21.15.105:0/3686236795 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:48.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.904+0000 7f5dc4814640 1 -- 172.21.15.105:0/3686236795 >> v1:172.21.15.148:6789/0 conn(0x7f5dc006d610 legacy=0x7f5dc00cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:48.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.904+0000 7f5dc4814640 1 -- 172.21.15.105:0/3686236795 >> v1:172.21.15.105:6789/0 conn(0x7f5dc00cc7d0 legacy=0x7f5dc00ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:48.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.904+0000 7f5dc4814640 1 -- 172.21.15.105:0/3686236795 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc0161ef0 con 0x7f5dc00ad9f0 2023-11-28T18:31:48.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.908+0000 7f5dc4814640 1 --2- 172.21.15.105:0/3686236795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc00ad9f0 0x7f5dc00ade80 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f5db00097c0 tx=0x7f5db001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1de1f2014586ca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:48.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.908+0000 7f5db7fff640 1 -- 172.21.15.105:0/3686236795 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db00ab650 con 0x7f5dc00ad9f0 2023-11-28T18:31:48.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.908+0000 7f5db7fff640 1 -- 172.21.15.105:0/3686236795 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db00ab7b0 con 0x7f5dc00ad9f0 2023-11-28T18:31:48.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.908+0000 7f5db7fff640 1 -- 172.21.15.105:0/3686236795 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db00abb40 con 0x7f5dc00ad9f0 2023-11-28T18:31:48.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.908+0000 7f5dc7279640 1 -- 172.21.15.105:0/3686236795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc00ad9f0 msgr2=0x7f5dc00ade80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:48.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.908+0000 7f5dc7279640 1 --2- 172.21.15.105:0/3686236795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc00ad9f0 0x7f5dc00ade80 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f5db00097c0 tx=0x7f5db001c450 comp rx=0 tx=0).stop 2023-11-28T18:31:48.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.908+0000 7f5dc7279640 1 -- 172.21.15.105:0/3686236795 shutdown_connections 2023-11-28T18:31:48.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.908+0000 7f5dc7279640 1 --2- 172.21.15.105:0/3686236795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc00ad9f0 0x7f5dc00ade80 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:48.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.908+0000 7f5dc7279640 1 -- 172.21.15.105:0/3686236795 >> 172.21.15.105:0/3686236795 conn(0x7f5dc00ceb90 msgr2=0x7f5dc00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:48.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.908+0000 7f5dc7279640 1 -- 172.21.15.105:0/3686236795 shutdown_connections 2023-11-28T18:31:48.910 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.908+0000 7f5dc7279640 1 -- 172.21.15.105:0/3686236795 wait complete. 2023-11-28T18:31:48.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc7279640 1 Processor -- start 2023-11-28T18:31:48.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc7279640 1 -- start start 2023-11-28T18:31:48.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc7279640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dc006d610 0x7f5dc0154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:48.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc7279640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc00ad9f0 0x7f5dc006c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:48.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc7279640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5dc00cc7d0 0x7f5dc0160b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:48.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc7279640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc01fe160 con 0x7f5dc00ad9f0 2023-11-28T18:31:48.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc7279640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc01fd800 con 0x7f5dc006d610 2023-11-28T18:31:48.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc7279640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc01fe580 con 0x7f5dc00cc7d0 2023-11-28T18:31:48.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc4814640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc00ad9f0 0x7f5dc006c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:48.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc4814640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc00ad9f0 0x7f5dc006c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50216/0 (socket says 172.21.15.105:50216) 2023-11-28T18:31:48.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc4814640 1 -- 172.21.15.105:0/2471196118 learned_addr learned my addr 172.21.15.105:0/2471196118 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:48.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc5816640 1 --2- 172.21.15.105:0/2471196118 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5dc00cc7d0 0x7f5dc0160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:48.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc4814640 1 -- 172.21.15.105:0/2471196118 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5dc00cc7d0 msgr2=0x7f5dc0160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:48.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc5015640 1 --2- 172.21.15.105:0/2471196118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dc006d610 0x7f5dc0154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:48.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc4814640 1 --2- 172.21.15.105:0/2471196118 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5dc00cc7d0 0x7f5dc0160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:48.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc4814640 1 -- 172.21.15.105:0/2471196118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dc006d610 msgr2=0x7f5dc0154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:48.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc4814640 1 --2- 172.21.15.105:0/2471196118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dc006d610 0x7f5dc0154df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:48.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc4814640 1 -- 172.21.15.105:0/2471196118 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db0009420 con 0x7f5dc00ad9f0 2023-11-28T18:31:48.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc5816640 1 --2- 172.21.15.105:0/2471196118 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5dc00cc7d0 0x7f5dc0160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:31:48.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc5015640 1 --2- 172.21.15.105:0/2471196118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dc006d610 0x7f5dc0154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:31:48.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc4814640 1 --2- 172.21.15.105:0/2471196118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc00ad9f0 0x7f5dc006c1e0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f5db00ad040 tx=0x7f5db001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:48.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5db5ffb640 1 -- 172.21.15.105:0/2471196118 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db00b5810 con 0x7f5dc00ad9f0 2023-11-28T18:31:48.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5db5ffb640 1 -- 172.21.15.105:0/2471196118 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db00b5970 con 0x7f5dc00ad9f0 2023-11-28T18:31:48.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc7279640 1 -- 172.21.15.105:0/2471196118 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc01fdaa0 con 0x7f5dc00ad9f0 2023-11-28T18:31:48.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5db5ffb640 1 -- 172.21.15.105:0/2471196118 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db00b5d20 con 0x7f5dc00ad9f0 2023-11-28T18:31:48.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.912+0000 7f5dc7279640 1 -- 172.21.15.105:0/2471196118 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc006cbf0 con 0x7f5dc00ad9f0 2023-11-28T18:31:48.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.916+0000 7f5dc7279640 1 -- 172.21.15.105:0/2471196118 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc01fe2a0 con 0x7f5dc00ad9f0 2023-11-28T18:31:48.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.916+0000 7f5db5ffb640 1 -- 172.21.15.105:0/2471196118 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5db00bf410 con 0x7f5dc00ad9f0 2023-11-28T18:31:48.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.916+0000 7f5db5ffb640 1 --2- 172.21.15.105:0/2471196118 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5da40668d0 0x7f5da4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:48.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.916+0000 7f5db5ffb640 1 -- 172.21.15.105:0/2471196118 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f5db011f180 con 0x7f5dc00ad9f0 2023-11-28T18:31:48.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.916+0000 7f5dc5015640 1 --2- 172.21.15.105:0/2471196118 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5da40668d0 0x7f5da4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:48.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.916+0000 7f5dc5015640 1 --2- 172.21.15.105:0/2471196118 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5da40668d0 0x7f5da4068d90 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00b4c30 tx=0x7f5dbc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:48.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:48.920+0000 7f5db5ffb640 1 -- 172.21.15.105:0/2471196118 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5dc01fe2a0 con 0x7f5dc00ad9f0 2023-11-28T18:31:49.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.068+0000 7f5dc7279640 1 -- 172.21.15.105:0/2471196118 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5dc01fe2a0 con 0x7f5dc00ad9f0 2023-11-28T18:31:49.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.072+0000 7f5db5ffb640 1 -- 172.21.15.105:0/2471196118 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v64) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5dc01fe2a0 con 0x7f5dc00ad9f0 2023-11-28T18:31:49.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.076+0000 7f5dc7279640 1 -- 172.21.15.105:0/2471196118 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5da40668d0 msgr2=0x7f5da4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:49.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.076+0000 7f5dc7279640 1 --2- 172.21.15.105:0/2471196118 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5da40668d0 0x7f5da4068d90 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00b4c30 tx=0x7f5dbc009040 comp rx=0 tx=0).stop 2023-11-28T18:31:49.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.076+0000 7f5dc7279640 1 -- 172.21.15.105:0/2471196118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc00ad9f0 msgr2=0x7f5dc006c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:49.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.076+0000 7f5dc7279640 1 --2- 172.21.15.105:0/2471196118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc00ad9f0 0x7f5dc006c1e0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f5db00ad040 tx=0x7f5db001ac90 comp rx=0 tx=0).stop 2023-11-28T18:31:49.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.076+0000 7f5dc7279640 1 -- 172.21.15.105:0/2471196118 shutdown_connections 2023-11-28T18:31:49.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.076+0000 7f5dc7279640 1 --2- 172.21.15.105:0/2471196118 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5dc00cc7d0 0x7f5dc0160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.076+0000 7f5dc7279640 1 --2- 172.21.15.105:0/2471196118 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5da40668d0 0x7f5da4068d90 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.076+0000 7f5dc7279640 1 --2- 172.21.15.105:0/2471196118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc00ad9f0 0x7f5dc006c1e0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.076+0000 7f5dc7279640 1 --2- 172.21.15.105:0/2471196118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dc006d610 0x7f5dc0154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.076+0000 7f5dc7279640 1 -- 172.21.15.105:0/2471196118 >> 172.21.15.105:0/2471196118 conn(0x7f5dc00ceb90 msgr2=0x7f5dc00ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:49.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.076+0000 7f5dc7279640 1 -- 172.21.15.105:0/2471196118 shutdown_connections 2023-11-28T18:31:49.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.076+0000 7f5dc7279640 1 -- 172.21.15.105:0/2471196118 wait complete. 2023-11-28T18:31:49.095 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_distributed 2023-11-28T18:31:49.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.232+0000 7eff8d1f8640 1 Processor -- start 2023-11-28T18:31:49.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.232+0000 7eff8d1f8640 1 -- start start 2023-11-28T18:31:49.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.236+0000 7eff8d1f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7eff880caf60 0x7eff880cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:49.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.236+0000 7eff8d1f8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7eff880ad850 0x7eff880701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:49.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.236+0000 7eff8d1f8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff8806bf50 con 0x7eff88159f40 2023-11-28T18:31:49.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.236+0000 7eff8d1f8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff880cb3c0 con 0x7eff880ad850 2023-11-28T18:31:49.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.236+0000 7eff8d1f8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff8815a320 con 0x7eff880caf60 2023-11-28T18:31:49.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.236+0000 7eff8759e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7eff880ad850 0x7eff880701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:49.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.236+0000 7eff8759e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7eff880ad850 0x7eff880701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50228/0 (socket says 172.21.15.105:50228) 2023-11-28T18:31:49.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.236+0000 7eff8759e640 1 -- 172.21.15.105:0/1508014221 learned_addr learned my addr 172.21.15.105:0/1508014221 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:49.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.236+0000 7eff7ffff640 1 --2- 172.21.15.105:0/1508014221 >> v2:172.21.15.148:3300/0 conn(0x7eff880caf60 0x7eff880cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:49.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.236+0000 7eff8759e640 1 -- 172.21.15.105:0/1508014221 >> v2:172.21.15.148:3300/0 conn(0x7eff880caf60 msgr2=0x7eff880cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:49.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.236+0000 7eff8759e640 1 --2- 172.21.15.105:0/1508014221 >> v2:172.21.15.148:3300/0 conn(0x7eff880caf60 0x7eff880cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.236+0000 7eff8759e640 1 -- 172.21.15.105:0/1508014221 >> v1:172.21.15.148:6789/0 conn(0x7eff88159f40 legacy=0x7eff880ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:49.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.236+0000 7eff8759e640 1 -- 172.21.15.105:0/1508014221 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff88071380 con 0x7eff880ad850 2023-11-28T18:31:49.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.236+0000 7eff7ffff640 1 --2- 172.21.15.105:0/1508014221 >> v2:172.21.15.148:3300/0 conn(0x7eff880caf60 0x7eff880cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:49.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.240+0000 7eff8759e640 1 --2- 172.21.15.105:0/1508014221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff880ad850 0x7eff880701d0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7eff8809d0d0 tx=0x7eff8001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420925e7450b741b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:49.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.240+0000 7eff8659c640 1 -- 172.21.15.105:0/1508014221 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff800a57d0 con 0x7eff880ad850 2023-11-28T18:31:49.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.240+0000 7eff8659c640 1 -- 172.21.15.105:0/1508014221 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff800a5930 con 0x7eff880ad850 2023-11-28T18:31:49.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.240+0000 7eff8659c640 1 -- 172.21.15.105:0/1508014221 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff800a5cc0 con 0x7eff880ad850 2023-11-28T18:31:49.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.240+0000 7eff8d1f8640 1 -- 172.21.15.105:0/1508014221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff880ad850 msgr2=0x7eff880701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:49.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.240+0000 7eff8d1f8640 1 --2- 172.21.15.105:0/1508014221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff880ad850 0x7eff880701d0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7eff8809d0d0 tx=0x7eff8001c910 comp rx=0 tx=0).stop 2023-11-28T18:31:49.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.240+0000 7eff8d1f8640 1 -- 172.21.15.105:0/1508014221 shutdown_connections 2023-11-28T18:31:49.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.240+0000 7eff8d1f8640 1 --2- 172.21.15.105:0/1508014221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff880ad850 0x7eff880701d0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.240+0000 7eff8d1f8640 1 --2- 172.21.15.105:0/1508014221 >> v2:172.21.15.148:3300/0 conn(0x7eff880caf60 0x7eff880cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.240+0000 7eff8d1f8640 1 -- 172.21.15.105:0/1508014221 >> 172.21.15.105:0/1508014221 conn(0x7eff880ceb90 msgr2=0x7eff880cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:49.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.240+0000 7eff8d1f8640 1 -- 172.21.15.105:0/1508014221 shutdown_connections 2023-11-28T18:31:49.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.240+0000 7eff8d1f8640 1 -- 172.21.15.105:0/1508014221 wait complete. 2023-11-28T18:31:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.240+0000 7eff8d1f8640 1 Processor -- start 2023-11-28T18:31:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.240+0000 7eff8d1f8640 1 -- start start 2023-11-28T18:31:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff8d1f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7eff880ad850 0x7eff8813c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff8d1f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff880caf60 0x7eff880aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff8d1f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff88159f40 0x7eff880af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff8d1f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8813cb20 con 0x7eff880caf60 2023-11-28T18:31:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff8d1f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff88154d90 con 0x7eff88159f40 2023-11-28T18:31:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff8d1f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff88155070 con 0x7eff880ad850 2023-11-28T18:31:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff7ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff880caf60 0x7eff880aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff7ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff880caf60 0x7eff880aed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50230/0 (socket says 172.21.15.105:50230) 2023-11-28T18:31:49.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff7ffff640 1 -- 172.21.15.105:0/2799955752 learned_addr learned my addr 172.21.15.105:0/2799955752 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff86d9d640 1 --2- 172.21.15.105:0/2799955752 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7eff880ad850 0x7eff8813c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff8759e640 1 --2- 172.21.15.105:0/2799955752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff88159f40 0x7eff880af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff7ffff640 1 -- 172.21.15.105:0/2799955752 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7eff880ad850 msgr2=0x7eff8813c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff7ffff640 1 --2- 172.21.15.105:0/2799955752 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7eff880ad850 0x7eff8813c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff7ffff640 1 -- 172.21.15.105:0/2799955752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff88159f40 msgr2=0x7eff880af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff7ffff640 1 --2- 172.21.15.105:0/2799955752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff88159f40 0x7eff880af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff7ffff640 1 -- 172.21.15.105:0/2799955752 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff80009b30 con 0x7eff880caf60 2023-11-28T18:31:49.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff8759e640 1 --2- 172.21.15.105:0/2799955752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff88159f40 0x7eff880af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:31:49.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff7ffff640 1 --2- 172.21.15.105:0/2799955752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff880caf60 0x7eff880aed80 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7eff7000d800 tx=0x7eff7000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:49.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff7f7fe640 1 -- 172.21.15.105:0/2799955752 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff700096c0 con 0x7eff880caf60 2023-11-28T18:31:49.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff7f7fe640 1 -- 172.21.15.105:0/2799955752 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff70009820 con 0x7eff880caf60 2023-11-28T18:31:49.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff8d1f8640 1 -- 172.21.15.105:0/2799955752 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff88155350 con 0x7eff880caf60 2023-11-28T18:31:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff7f7fe640 1 -- 172.21.15.105:0/2799955752 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff70009bd0 con 0x7eff880caf60 2023-11-28T18:31:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff8d1f8640 1 -- 172.21.15.105:0/2799955752 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff88155910 con 0x7eff880caf60 2023-11-28T18:31:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff8d1f8640 1 -- 172.21.15.105:0/2799955752 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff88067570 con 0x7eff880caf60 2023-11-28T18:31:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.244+0000 7eff7f7fe640 1 -- 172.21.15.105:0/2799955752 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7eff70009d30 con 0x7eff880caf60 2023-11-28T18:31:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.248+0000 7eff7f7fe640 1 --2- 172.21.15.105:0/2799955752 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7eff680668d0 0x7eff68068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.248+0000 7eff7f7fe640 1 -- 172.21.15.105:0/2799955752 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5435+0+0 (secure 0 0 0) 0x7eff70084ad0 con 0x7eff880caf60 2023-11-28T18:31:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.248+0000 7eff86d9d640 1 --2- 172.21.15.105:0/2799955752 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7eff680668d0 0x7eff68068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.248+0000 7eff86d9d640 1 --2- 172.21.15.105:0/2799955752 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7eff680668d0 0x7eff68068d90 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7eff78000f70 tx=0x7eff78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.252+0000 7eff7f7fe640 1 -- 172.21.15.105:0/2799955752 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7eff70051d80 con 0x7eff880caf60 2023-11-28T18:31:49.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.416+0000 7eff8d1f8640 1 -- 172.21.15.105:0/2799955752 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"} v 0) v1 -- 0x7eff880cf950 con 0x7eff880caf60 2023-11-28T18:31:49.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.420+0000 7eff7f7fe640 1 -- 172.21.15.105:0/2799955752 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"}]=0 v65) v1 ==== 115+0+0 (secure 0 0 0) 0x7eff700553a0 con 0x7eff880caf60 2023-11-28T18:31:49.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.420+0000 7eff8d1f8640 1 -- 172.21.15.105:0/2799955752 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7eff680668d0 msgr2=0x7eff68068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:49.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.420+0000 7eff8d1f8640 1 --2- 172.21.15.105:0/2799955752 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7eff680668d0 0x7eff68068d90 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7eff78000f70 tx=0x7eff78009040 comp rx=0 tx=0).stop 2023-11-28T18:31:49.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.420+0000 7eff8d1f8640 1 -- 172.21.15.105:0/2799955752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff880caf60 msgr2=0x7eff880aed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:49.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.420+0000 7eff8d1f8640 1 --2- 172.21.15.105:0/2799955752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff880caf60 0x7eff880aed80 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7eff7000d800 tx=0x7eff7000dcc0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.420+0000 7eff8d1f8640 1 -- 172.21.15.105:0/2799955752 shutdown_connections 2023-11-28T18:31:49.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.420+0000 7eff8d1f8640 1 --2- 172.21.15.105:0/2799955752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff88159f40 0x7eff880af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.420+0000 7eff8d1f8640 1 --2- 172.21.15.105:0/2799955752 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7eff680668d0 0x7eff68068d90 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.420+0000 7eff8d1f8640 1 --2- 172.21.15.105:0/2799955752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff880caf60 0x7eff880aed80 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.420+0000 7eff8d1f8640 1 --2- 172.21.15.105:0/2799955752 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7eff880ad850 0x7eff8813c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.420+0000 7eff8d1f8640 1 -- 172.21.15.105:0/2799955752 >> 172.21.15.105:0/2799955752 conn(0x7eff880ceb90 msgr2=0x7eff880ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:49.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.420+0000 7eff8d1f8640 1 -- 172.21.15.105:0/2799955752 shutdown_connections 2023-11-28T18:31:49.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.420+0000 7eff8d1f8640 1 -- 172.21.15.105:0/2799955752 wait complete. 2023-11-28T18:31:49.442 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random_max 2023-11-28T18:31:49.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.580+0000 7f8c19c5a640 1 Processor -- start 2023-11-28T18:31:49.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.580+0000 7f8c19c5a640 1 -- start start 2023-11-28T18:31:49.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.584+0000 7f8c19c5a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c14071960 0x7f8c1415f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:49.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.584+0000 7f8c19c5a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8c1415b030 0x7f8c140709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:49.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.584+0000 7f8c19c5a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c140701d0 con 0x7f8c140b4290 2023-11-28T18:31:49.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.584+0000 7f8c19c5a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c1406df80 con 0x7f8c1415b030 2023-11-28T18:31:49.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.584+0000 7f8c19c5a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c14071df0 con 0x7f8c14071960 2023-11-28T18:31:49.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.584+0000 7f8c137fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8c140b4290 0x7f8c140b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49862/0 (socket says 172.21.15.105:49862) 2023-11-28T18:31:49.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.584+0000 7f8c137fe640 1 -- 172.21.15.105:0/1437118399 learned_addr learned my addr 172.21.15.105:0/1437118399 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:49.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.584+0000 7f8c13fff640 1 --2- 172.21.15.105:0/1437118399 >> v2:172.21.15.148:3300/0 conn(0x7f8c1415b030 0x7f8c140709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:49.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.584+0000 7f8c0bfff640 1 --2- 172.21.15.105:0/1437118399 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c14071960 0x7f8c1415f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:49.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.584+0000 7f8c12ffd640 1 -- 172.21.15.105:0/1437118399 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3397927313 0 0) 0x7f8c140701d0 con 0x7f8c140b4290 2023-11-28T18:31:49.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.584+0000 7f8c12ffd640 1 -- 172.21.15.105:0/1437118399 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c00003530 con 0x7f8c140b4290 2023-11-28T18:31:49.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.584+0000 7f8c12ffd640 1 -- 172.21.15.105:0/1437118399 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1544058474 0 0) 0x7f8c00003530 con 0x7f8c140b4290 2023-11-28T18:31:49.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.584+0000 7f8c13fff640 1 -- 172.21.15.105:0/1437118399 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c14071960 msgr2=0x7f8c1415f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:49.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.584+0000 7f8c13fff640 1 --2- 172.21.15.105:0/1437118399 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c14071960 0x7f8c1415f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.584+0000 7f8c13fff640 1 -- 172.21.15.105:0/1437118399 >> v1:172.21.15.105:6789/0 conn(0x7f8c140b4290 legacy=0x7f8c140b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:49.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.584+0000 7f8c13fff640 1 -- 172.21.15.105:0/1437118399 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c140720d0 con 0x7f8c1415b030 2023-11-28T18:31:49.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.584+0000 7f8c0bfff640 1 --2- 172.21.15.105:0/1437118399 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c14071960 0x7f8c1415f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:31:49.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.588+0000 7f8c13fff640 1 --2- 172.21.15.105:0/1437118399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c1415b030 0x7f8c140709b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009d90 tx=0x7f8c0c01c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca645e64dd54584a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:49.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.588+0000 7f8c12ffd640 1 -- 172.21.15.105:0/1437118399 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c04001680 con 0x7f8c1415b030 2023-11-28T18:31:49.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.588+0000 7f8c12ffd640 1 -- 172.21.15.105:0/1437118399 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0c0a5550 con 0x7f8c1415b030 2023-11-28T18:31:49.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.588+0000 7f8c12ffd640 1 -- 172.21.15.105:0/1437118399 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c0a58e0 con 0x7f8c1415b030 2023-11-28T18:31:49.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.588+0000 7f8c19c5a640 1 -- 172.21.15.105:0/1437118399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c1415b030 msgr2=0x7f8c140709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:49.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.588+0000 7f8c19c5a640 1 --2- 172.21.15.105:0/1437118399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c1415b030 0x7f8c140709b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009d90 tx=0x7f8c0c01c840 comp rx=0 tx=0).stop 2023-11-28T18:31:49.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.588+0000 7f8c19c5a640 1 -- 172.21.15.105:0/1437118399 shutdown_connections 2023-11-28T18:31:49.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.588+0000 7f8c19c5a640 1 --2- 172.21.15.105:0/1437118399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c1415b030 0x7f8c140709b0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.588+0000 7f8c19c5a640 1 --2- 172.21.15.105:0/1437118399 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c14071960 0x7f8c1415f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.588+0000 7f8c19c5a640 1 -- 172.21.15.105:0/1437118399 >> 172.21.15.105:0/1437118399 conn(0x7f8c140ceb90 msgr2=0x7f8c140cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:49.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.588+0000 7f8c19c5a640 1 -- 172.21.15.105:0/1437118399 shutdown_connections 2023-11-28T18:31:49.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.588+0000 7f8c19c5a640 1 -- 172.21.15.105:0/1437118399 wait complete. 2023-11-28T18:31:49.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.588+0000 7f8c19c5a640 1 Processor -- start 2023-11-28T18:31:49.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.588+0000 7f8c19c5a640 1 -- start start 2023-11-28T18:31:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c19c5a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c140b4290 0x7f8c1414c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c19c5a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c1415b030 0x7f8c14152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c19c5a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c140b19b0 0x7f8c14153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c19c5a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1414e570 con 0x7f8c1415b030 2023-11-28T18:31:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c19c5a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1414e850 con 0x7f8c140b19b0 2023-11-28T18:31:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c19c5a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1414eb30 con 0x7f8c140b4290 2023-11-28T18:31:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c0bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c1415b030 0x7f8c14152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c0bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c1415b030 0x7f8c14152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50242/0 (socket says 172.21.15.105:50242) 2023-11-28T18:31:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c0bfff640 1 -- 172.21.15.105:0/805678720 learned_addr learned my addr 172.21.15.105:0/805678720 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c137fe640 1 --2- 172.21.15.105:0/805678720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c140b4290 0x7f8c1414c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c13fff640 1 --2- 172.21.15.105:0/805678720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c140b19b0 0x7f8c14153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c0bfff640 1 -- 172.21.15.105:0/805678720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c140b4290 msgr2=0x7f8c1414c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c0bfff640 1 --2- 172.21.15.105:0/805678720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c140b4290 0x7f8c1414c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c0bfff640 1 -- 172.21.15.105:0/805678720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c140b19b0 msgr2=0x7f8c14153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c0bfff640 1 --2- 172.21.15.105:0/805678720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c140b19b0 0x7f8c14153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c0bfff640 1 -- 172.21.15.105:0/805678720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc009420 con 0x7f8c1415b030 2023-11-28T18:31:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c13fff640 1 --2- 172.21.15.105:0/805678720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c140b19b0 0x7f8c14153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:31:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c137fe640 1 --2- 172.21.15.105:0/805678720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c140b4290 0x7f8c1414c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:31:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c0bfff640 1 --2- 172.21.15.105:0/805678720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c1415b030 0x7f8c14152b50 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00ebe0 tx=0x7f8bfc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c10ff9640 1 -- 172.21.15.105:0/805678720 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc00eee0 con 0x7f8c1415b030 2023-11-28T18:31:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c10ff9640 1 -- 172.21.15.105:0/805678720 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bfc00cc30 con 0x7f8c1415b030 2023-11-28T18:31:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c10ff9640 1 -- 172.21.15.105:0/805678720 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc013470 con 0x7f8c1415b030 2023-11-28T18:31:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c19c5a640 1 -- 172.21.15.105:0/805678720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0c009a70 con 0x7f8c1415b030 2023-11-28T18:31:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c19c5a640 1 -- 172.21.15.105:0/805678720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1414f0f0 con 0x7f8c1415b030 2023-11-28T18:31:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.592+0000 7f8c19c5a640 1 -- 172.21.15.105:0/805678720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1414f2b0 con 0x7f8c1415b030 2023-11-28T18:31:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.596+0000 7f8c10ff9640 1 -- 172.21.15.105:0/805678720 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8bfc0135d0 con 0x7f8c1415b030 2023-11-28T18:31:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.596+0000 7f8c10ff9640 1 --2- 172.21.15.105:0/805678720 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8bf40668d0 0x7f8bf4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.596+0000 7f8c10ff9640 1 -- 172.21.15.105:0/805678720 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f8bfc0846d0 con 0x7f8c1415b030 2023-11-28T18:31:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.596+0000 7f8c137fe640 1 --2- 172.21.15.105:0/805678720 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8bf40668d0 0x7f8bf4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.596+0000 7f8c137fe640 1 --2- 172.21.15.105:0/805678720 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8bf40668d0 0x7f8bf4068d90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8c040011f0 tx=0x7f8c04009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:49.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.600+0000 7f8c10ff9640 1 -- 172.21.15.105:0/805678720 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8bfc051980 con 0x7f8c1415b030 2023-11-28T18:31:49.748 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.748+0000 7f8c19c5a640 1 -- 172.21.15.105:0/805678720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"} v 0) v1 -- 0x7f8c14067570 con 0x7f8c1415b030 2023-11-28T18:31:49.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.752+0000 7f8c10ff9640 1 -- 172.21.15.105:0/805678720 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"}]=0 v66) v1 ==== 114+0+0 (secure 0 0 0) 0x7f8bfc054fa0 con 0x7f8c1415b030 2023-11-28T18:31:49.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.756+0000 7f8c19c5a640 1 -- 172.21.15.105:0/805678720 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8bf40668d0 msgr2=0x7f8bf4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:49.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.756+0000 7f8c19c5a640 1 --2- 172.21.15.105:0/805678720 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8bf40668d0 0x7f8bf4068d90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8c040011f0 tx=0x7f8c04009040 comp rx=0 tx=0).stop 2023-11-28T18:31:49.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.756+0000 7f8c19c5a640 1 -- 172.21.15.105:0/805678720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c1415b030 msgr2=0x7f8c14152b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:49.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.756+0000 7f8c19c5a640 1 --2- 172.21.15.105:0/805678720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c1415b030 0x7f8c14152b50 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00ebe0 tx=0x7f8bfc00c520 comp rx=0 tx=0).stop 2023-11-28T18:31:49.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.756+0000 7f8c19c5a640 1 -- 172.21.15.105:0/805678720 shutdown_connections 2023-11-28T18:31:49.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.756+0000 7f8c19c5a640 1 --2- 172.21.15.105:0/805678720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c140b19b0 0x7f8c14153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.756+0000 7f8c19c5a640 1 --2- 172.21.15.105:0/805678720 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8bf40668d0 0x7f8bf4068d90 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.756+0000 7f8c19c5a640 1 --2- 172.21.15.105:0/805678720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c1415b030 0x7f8c14152b50 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.756+0000 7f8c19c5a640 1 --2- 172.21.15.105:0/805678720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c140b4290 0x7f8c1414c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.756+0000 7f8c19c5a640 1 -- 172.21.15.105:0/805678720 >> 172.21.15.105:0/805678720 conn(0x7f8c140ceb90 msgr2=0x7f8c14158630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:49.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.756+0000 7f8c19c5a640 1 -- 172.21.15.105:0/805678720 shutdown_connections 2023-11-28T18:31:49.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.756+0000 7f8c19c5a640 1 -- 172.21.15.105:0/805678720 wait complete. 2023-11-28T18:31:49.775 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_pin' 2023-11-28T18:31:49.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.916+0000 7f2c7dd72640 1 Processor -- start 2023-11-28T18:31:49.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.916+0000 7f2c7dd72640 1 -- start start 2023-11-28T18:31:49.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.916+0000 7f2c7dd72640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2c780caf60 0x7f2c780cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:49.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.916+0000 7f2c7dd72640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c7806bf50 con 0x7f2c78159f40 2023-11-28T18:31:49.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.916+0000 7f2c7dd72640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c7815a320 con 0x7f2c780ad850 2023-11-28T18:31:49.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.916+0000 7f2c7dd72640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c780cb3c0 con 0x7f2c780caf60 2023-11-28T18:31:49.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.916+0000 7f2c6ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2c780caf60 0x7f2c780cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:49.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.916+0000 7f2c777fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f2c78159f40 0x7f2c780ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49872/0 (socket says 172.21.15.105:49872) 2023-11-28T18:31:49.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.916+0000 7f2c777fe640 1 -- 172.21.15.105:0/3673243317 learned_addr learned my addr 172.21.15.105:0/3673243317 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:49.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.916+0000 7f2c6ffff640 1 -- 172.21.15.105:0/3673243317 >> v1:172.21.15.148:6789/0 conn(0x7f2c780ad850 legacy=0x7f2c780701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:49.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.916+0000 7f2c6ffff640 1 -- 172.21.15.105:0/3673243317 >> v1:172.21.15.105:6789/0 conn(0x7f2c78159f40 legacy=0x7f2c780ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:49.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.920+0000 7f2c6ffff640 1 -- 172.21.15.105:0/3673243317 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c78071380 con 0x7f2c780caf60 2023-11-28T18:31:49.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.920+0000 7f2c76ffd640 1 -- 172.21.15.105:0/3673243317 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549751767 0 0) 0x7f2c7806bf50 con 0x7f2c78159f40 2023-11-28T18:31:49.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.920+0000 7f2c6ffff640 1 --2- 172.21.15.105:0/3673243317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c780caf60 0x7f2c780cc760 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f2c600097c0 tx=0x7f2c6001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92b2603faefc3df server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:49.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.920+0000 7f2c76ffd640 1 -- 172.21.15.105:0/3673243317 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c600ac620 con 0x7f2c780caf60 2023-11-28T18:31:49.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.920+0000 7f2c76ffd640 1 -- 172.21.15.105:0/3673243317 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c600ac780 con 0x7f2c780caf60 2023-11-28T18:31:49.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.920+0000 7f2c76ffd640 1 -- 172.21.15.105:0/3673243317 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c600acb10 con 0x7f2c780caf60 2023-11-28T18:31:49.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.920+0000 7f2c7dd72640 1 -- 172.21.15.105:0/3673243317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c780caf60 msgr2=0x7f2c780cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:49.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.920+0000 7f2c7dd72640 1 --2- 172.21.15.105:0/3673243317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c780caf60 0x7f2c780cc760 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f2c600097c0 tx=0x7f2c6001c3c0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.920+0000 7f2c7dd72640 1 -- 172.21.15.105:0/3673243317 shutdown_connections 2023-11-28T18:31:49.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.920+0000 7f2c7dd72640 1 --2- 172.21.15.105:0/3673243317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c780caf60 0x7f2c780cc760 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.920+0000 7f2c7dd72640 1 -- 172.21.15.105:0/3673243317 >> 172.21.15.105:0/3673243317 conn(0x7f2c780ceb90 msgr2=0x7f2c780cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:49.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.920+0000 7f2c7dd72640 1 -- 172.21.15.105:0/3673243317 shutdown_connections 2023-11-28T18:31:49.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.920+0000 7f2c7dd72640 1 -- 172.21.15.105:0/3673243317 wait complete. 2023-11-28T18:31:49.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c7dd72640 1 Processor -- start 2023-11-28T18:31:49.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c7dd72640 1 -- start start 2023-11-28T18:31:49.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c7dd72640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c780ad850 0x7f2c780c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:49.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c7dd72640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c780caf60 0x7f2c7806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:49.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c7dd72640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c78159f40 0x7f2c7806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:49.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c7dd72640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c78157b60 con 0x7f2c78159f40 2023-11-28T18:31:49.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c7dd72640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c78157e40 con 0x7f2c780ad850 2023-11-28T18:31:49.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c7dd72640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c78158120 con 0x7f2c780caf60 2023-11-28T18:31:49.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c77fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c78159f40 0x7f2c7806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:49.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c77fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c78159f40 0x7f2c7806cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50268/0 (socket says 172.21.15.105:50268) 2023-11-28T18:31:49.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c77fff640 1 -- 172.21.15.105:0/3668885073 learned_addr learned my addr 172.21.15.105:0/3668885073 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:49.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c777fe640 1 --2- 172.21.15.105:0/3668885073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c780ad850 0x7f2c780c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:49.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c6ffff640 1 --2- 172.21.15.105:0/3668885073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c780caf60 0x7f2c7806c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:49.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c77fff640 1 -- 172.21.15.105:0/3668885073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c780caf60 msgr2=0x7f2c7806c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:49.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c77fff640 1 --2- 172.21.15.105:0/3668885073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c780caf60 0x7f2c7806c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c77fff640 1 -- 172.21.15.105:0/3668885073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c780ad850 msgr2=0x7f2c780c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:49.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c77fff640 1 --2- 172.21.15.105:0/3668885073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c780ad850 0x7f2c780c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:49.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c77fff640 1 -- 172.21.15.105:0/3668885073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c60009420 con 0x7f2c78159f40 2023-11-28T18:31:49.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c6ffff640 1 --2- 172.21.15.105:0/3668885073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c780caf60 0x7f2c7806c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:49.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c77fff640 1 --2- 172.21.15.105:0/3668885073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c78159f40 0x7f2c7806cd20 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000c730 tx=0x7f2c7000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:49.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c74ff9640 1 -- 172.21.15.105:0/3668885073 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70003b80 con 0x7f2c78159f40 2023-11-28T18:31:49.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c74ff9640 1 -- 172.21.15.105:0/3668885073 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c70003ce0 con 0x7f2c78159f40 2023-11-28T18:31:49.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c74ff9640 1 -- 172.21.15.105:0/3668885073 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7000aa30 con 0x7f2c78159f40 2023-11-28T18:31:49.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.924+0000 7f2c7dd72640 1 -- 172.21.15.105:0/3668885073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c78158400 con 0x7f2c78159f40 2023-11-28T18:31:49.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.928+0000 7f2c7dd72640 1 -- 172.21.15.105:0/3668885073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c78158c00 con 0x7f2c78159f40 2023-11-28T18:31:49.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.928+0000 7f2c7dd72640 1 -- 172.21.15.105:0/3668885073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c78067570 con 0x7f2c78159f40 2023-11-28T18:31:49.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.928+0000 7f2c74ff9640 1 -- 172.21.15.105:0/3668885073 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f2c7000ab90 con 0x7f2c78159f40 2023-11-28T18:31:49.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.928+0000 7f2c74ff9640 1 --2- 172.21.15.105:0/3668885073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2c58066900 0x7f2c58068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:49.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.928+0000 7f2c74ff9640 1 -- 172.21.15.105:0/3668885073 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f2c70083f80 con 0x7f2c78159f40 2023-11-28T18:31:49.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.928+0000 7f2c777fe640 1 --2- 172.21.15.105:0/3668885073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2c58066900 0x7f2c58068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:49.930 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.928+0000 7f2c777fe640 1 --2- 172.21.15.105:0/3668885073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2c58066900 0x7f2c58068dc0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f2c68007a90 tx=0x7f2c680079e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:49.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:49.932+0000 7f2c74ff9640 1 -- 172.21.15.105:0/3668885073 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2c700b5280 con 0x7f2c78159f40 2023-11-28T18:31:50.135 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.136+0000 7f2c7dd72640 1 -- 172.21.15.105:0/3668885073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_pin"]} v 0) v1 -- 0x7f2c78159710 con 0x7f2c78159f40 2023-11-28T18:31:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.396+0000 7f2c74ff9640 1 -- 172.21.15.105:0/3668885073 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_pin"]}]=0 v430) v1 ==== 159+0+0 (secure 0 0 0) 0x7f2c70054df0 con 0x7f2c78159f40 2023-11-28T18:31:50.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.400+0000 7f2c7dd72640 1 -- 172.21.15.105:0/3668885073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2c58066900 msgr2=0x7f2c58068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:50.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.400+0000 7f2c7dd72640 1 --2- 172.21.15.105:0/3668885073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2c58066900 0x7f2c58068dc0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f2c68007a90 tx=0x7f2c680079e0 comp rx=0 tx=0).stop 2023-11-28T18:31:50.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.400+0000 7f2c7dd72640 1 -- 172.21.15.105:0/3668885073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c78159f40 msgr2=0x7f2c7806cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:50.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.400+0000 7f2c7dd72640 1 --2- 172.21.15.105:0/3668885073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c78159f40 0x7f2c7806cd20 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000c730 tx=0x7f2c7000cbf0 comp rx=0 tx=0).stop 2023-11-28T18:31:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.400+0000 7f2c7dd72640 1 -- 172.21.15.105:0/3668885073 shutdown_connections 2023-11-28T18:31:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.400+0000 7f2c7dd72640 1 --2- 172.21.15.105:0/3668885073 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2c58066900 0x7f2c58068dc0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.400+0000 7f2c7dd72640 1 --2- 172.21.15.105:0/3668885073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c78159f40 0x7f2c7806cd20 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.400+0000 7f2c7dd72640 1 --2- 172.21.15.105:0/3668885073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c780caf60 0x7f2c7806c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.400+0000 7f2c7dd72640 1 --2- 172.21.15.105:0/3668885073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c780ad850 0x7f2c780c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.400+0000 7f2c7dd72640 1 -- 172.21.15.105:0/3668885073 >> 172.21.15.105:0/3668885073 conn(0x7f2c780ceb90 msgr2=0x7f2c780ae790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.400+0000 7f2c7dd72640 1 -- 172.21.15.105:0/3668885073 shutdown_connections 2023-11-28T18:31:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.400+0000 7f2c7dd72640 1 -- 172.21.15.105:0/3668885073 wait complete. 2023-11-28T18:31:50.421 INFO:tasks.cephfs_test_runner:That an export pin overrides an ephemerally pinned directory. ... ok 2023-11-28T18:31:50.421 INFO:tasks.cephfs_test_runner:Starting test: test_ephemeral_pin_distribution (tasks.cephfs.test_exports.TestEphemeralPins) That ephemerally pinned subtrees are somewhat evenly distributed. 2023-11-28T18:31:50.421 INFO:tasks.cephfs_test_runner:test_ephemeral_pin_distribution (tasks.cephfs.test_exports.TestEphemeralPins) 2023-11-28T18:31:50.422 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:31:50.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.560+0000 7f3ce913e640 1 Processor -- start 2023-11-28T18:31:50.560 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.560+0000 7f3ce913e640 1 -- start start 2023-11-28T18:31:50.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.560+0000 7f3ce913e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3ce415d4f0 0x7f3ce40d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:50.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.560+0000 7f3ce913e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ce40b2d20 0x7f3ce40b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:50.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.560+0000 7f3ce913e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce40b25b0 con 0x7f3ce40d3c30 2023-11-28T18:31:50.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.560+0000 7f3ce913e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce40b2900 con 0x7f3ce415d4f0 2023-11-28T18:31:50.561 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.560+0000 7f3ce913e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce40d0a20 con 0x7f3ce40b2d20 2023-11-28T18:31:50.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.560+0000 7f3ce2d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3ce415d4f0 0x7f3ce40d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:50.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.560+0000 7f3cda59c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3ce40d3c30 0x7f3ce40d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.105:58734/0 (socket says 172.21.15.105:58734) 2023-11-28T18:31:50.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.560+0000 7f3ce359e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ce40b2d20 0x7f3ce40b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:50.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.560+0000 7f3cda59c640 1 -- 172.21.15.105:0/2207226065 learned_addr learned my addr 172.21.15.105:0/2207226065 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:50.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.560+0000 7f3ce359e640 1 -- 172.21.15.105:0/2207226065 >> v2:172.21.15.148:3300/0 conn(0x7f3ce415d4f0 msgr2=0x7f3ce40d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:50.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.560+0000 7f3ce359e640 1 --2- 172.21.15.105:0/2207226065 >> v2:172.21.15.148:3300/0 conn(0x7f3ce415d4f0 0x7f3ce40d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:50.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.560+0000 7f3ce259c640 1 -- 172.21.15.105:0/2207226065 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3620782349 0 0) 0x7f3ce40b25b0 con 0x7f3ce40d3c30 2023-11-28T18:31:50.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.560+0000 7f3ce359e640 1 -- 172.21.15.105:0/2207226065 >> v1:172.21.15.148:6789/0 conn(0x7f3ce40d3c30 legacy=0x7f3ce40d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:50.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.560+0000 7f3ce359e640 1 -- 172.21.15.105:0/2207226065 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce40cea30 con 0x7f3ce40b2d20 2023-11-28T18:31:50.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.564+0000 7f3ce359e640 1 --2- 172.21.15.105:0/2207226065 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ce40b2d20 0x7f3ce40b3190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009d90 tx=0x7f3cdc01c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eb1f7f17153a80a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:50.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.564+0000 7f3ce259c640 1 -- 172.21.15.105:0/2207226065 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc0a5950 con 0x7f3ce40b2d20 2023-11-28T18:31:50.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.564+0000 7f3ce259c640 1 -- 172.21.15.105:0/2207226065 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cdc0a5ab0 con 0x7f3ce40b2d20 2023-11-28T18:31:50.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.564+0000 7f3ce259c640 1 -- 172.21.15.105:0/2207226065 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc0a5e40 con 0x7f3ce40b2d20 2023-11-28T18:31:50.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.564+0000 7f3ce913e640 1 -- 172.21.15.105:0/2207226065 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ce40b2d20 msgr2=0x7f3ce40b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:50.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.564+0000 7f3ce913e640 1 --2- 172.21.15.105:0/2207226065 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ce40b2d20 0x7f3ce40b3190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009d90 tx=0x7f3cdc01c890 comp rx=0 tx=0).stop 2023-11-28T18:31:50.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.564+0000 7f3ce913e640 1 -- 172.21.15.105:0/2207226065 shutdown_connections 2023-11-28T18:31:50.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.564+0000 7f3ce913e640 1 --2- 172.21.15.105:0/2207226065 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ce40b2d20 0x7f3ce40b3190 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:50.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.564+0000 7f3ce913e640 1 --2- 172.21.15.105:0/2207226065 >> v2:172.21.15.148:3300/0 conn(0x7f3ce415d4f0 0x7f3ce40d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:50.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.564+0000 7f3ce913e640 1 -- 172.21.15.105:0/2207226065 >> 172.21.15.105:0/2207226065 conn(0x7f3ce40d3030 msgr2=0x7f3ce40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:50.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.564+0000 7f3ce913e640 1 -- 172.21.15.105:0/2207226065 shutdown_connections 2023-11-28T18:31:50.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.564+0000 7f3ce913e640 1 -- 172.21.15.105:0/2207226065 wait complete. 2023-11-28T18:31:50.567 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3ce913e640 1 Processor -- start 2023-11-28T18:31:50.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3ce913e640 1 -- start start 2023-11-28T18:31:50.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3ce913e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ce40b2d20 0x7f3ce41605e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:50.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3ce913e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce40d3c30 0x7f3ce41555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:50.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3ce913e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ce415d4f0 0x7f3ce40cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:50.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3ce913e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce40cc500 con 0x7f3ce415d4f0 2023-11-28T18:31:50.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3ce913e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce4157d60 con 0x7f3ce40d3c30 2023-11-28T18:31:50.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3ce913e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce4158040 con 0x7f3ce40b2d20 2023-11-28T18:31:50.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3ce359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ce415d4f0 0x7f3ce40cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:50.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3ce359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ce415d4f0 0x7f3ce40cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50272/0 (socket says 172.21.15.105:50272) 2023-11-28T18:31:50.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3ce359e640 1 -- 172.21.15.105:0/1481638765 learned_addr learned my addr 172.21.15.105:0/1481638765 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:50.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3cda59c640 1 --2- 172.21.15.105:0/1481638765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce40d3c30 0x7f3ce41555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:50.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3ce2d9d640 1 --2- 172.21.15.105:0/1481638765 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ce40b2d20 0x7f3ce41605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:50.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3ce359e640 1 -- 172.21.15.105:0/1481638765 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ce40b2d20 msgr2=0x7f3ce41605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:50.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3ce359e640 1 --2- 172.21.15.105:0/1481638765 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ce40b2d20 0x7f3ce41605e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:50.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3ce359e640 1 -- 172.21.15.105:0/1481638765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce40d3c30 msgr2=0x7f3ce41555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:50.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3ce359e640 1 --2- 172.21.15.105:0/1481638765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce40d3c30 0x7f3ce41555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:50.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3ce359e640 1 -- 172.21.15.105:0/1481638765 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cdc009a70 con 0x7f3ce415d4f0 2023-11-28T18:31:50.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3ce359e640 1 --2- 172.21.15.105:0/1481638765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ce415d4f0 0x7f3ce40cc920 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0b0040 tx=0x7f3cdc019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:50.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3cdbfff640 1 -- 172.21.15.105:0/1481638765 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc0add90 con 0x7f3ce415d4f0 2023-11-28T18:31:50.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3cdbfff640 1 -- 172.21.15.105:0/1481638765 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cdc009450 con 0x7f3ce415d4f0 2023-11-28T18:31:50.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3ce913e640 1 -- 172.21.15.105:0/1481638765 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce4158320 con 0x7f3ce415d4f0 2023-11-28T18:31:50.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3cdbfff640 1 -- 172.21.15.105:0/1481638765 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc0b9690 con 0x7f3ce415d4f0 2023-11-28T18:31:50.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.568+0000 7f3ce913e640 1 -- 172.21.15.105:0/1481638765 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce40cce60 con 0x7f3ce415d4f0 2023-11-28T18:31:50.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.572+0000 7f3ce913e640 1 -- 172.21.15.105:0/1481638765 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce40cc640 con 0x7f3ce415d4f0 2023-11-28T18:31:50.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.572+0000 7f3cdbfff640 1 -- 172.21.15.105:0/1481638765 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3cdc0b97f0 con 0x7f3ce415d4f0 2023-11-28T18:31:50.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.572+0000 7f3cdbfff640 1 --2- 172.21.15.105:0/1481638765 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3cc40668d0 0x7f3cc4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:50.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.572+0000 7f3cdbfff640 1 -- 172.21.15.105:0/1481638765 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f3cdc11f940 con 0x7f3ce415d4f0 2023-11-28T18:31:50.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.572+0000 7f3ce2d9d640 1 --2- 172.21.15.105:0/1481638765 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3cc40668d0 0x7f3cc4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:50.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.572+0000 7f3ce2d9d640 1 --2- 172.21.15.105:0/1481638765 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3cc40668d0 0x7f3cc4068d90 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4009b60 tx=0x7f3cd4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:50.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.576+0000 7f3cdbfff640 1 -- 172.21.15.105:0/1481638765 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3ce40cc640 con 0x7f3ce415d4f0 2023-11-28T18:31:50.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.728+0000 7f3ce913e640 1 -- 172.21.15.105:0/1481638765 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ce40cc640 con 0x7f3ce415d4f0 2023-11-28T18:31:50.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.728+0000 7f3cdbfff640 1 -- 172.21.15.105:0/1481638765 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+13523 (secure 0 0 0) 0x7f3ce40cc640 con 0x7f3ce415d4f0 2023-11-28T18:31:50.730 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:50.730 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":120,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:31:48.387447+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:6836/3373504883":"2023-11-29T18:31:45.380815+0000","172.21.15.105:6834/3373504883":"2023-11-29T18:31:45.380815+0000","172.21.15.148:6835/270170970":"2023-11-29T18:31:45.380805+0000","172.21.15.148:6833/270170970":"2023-11-29T18:31:45.380805+0000","172.21.15.105:6837/764864050":"2023-11-29T18:31:45.380820+0000","172.21.15.105:6835/764864050":"2023-11-29T18:31:45.380820+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:31:50.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.732+0000 7f3ce913e640 1 -- 172.21.15.105:0/1481638765 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3cc40668d0 msgr2=0x7f3cc4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:50.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.732+0000 7f3ce913e640 1 --2- 172.21.15.105:0/1481638765 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3cc40668d0 0x7f3cc4068d90 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4009b60 tx=0x7f3cd4009190 comp rx=0 tx=0).stop 2023-11-28T18:31:50.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.732+0000 7f3ce913e640 1 -- 172.21.15.105:0/1481638765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ce415d4f0 msgr2=0x7f3ce40cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:50.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.732+0000 7f3ce913e640 1 --2- 172.21.15.105:0/1481638765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ce415d4f0 0x7f3ce40cc920 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0b0040 tx=0x7f3cdc019630 comp rx=0 tx=0).stop 2023-11-28T18:31:50.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.732+0000 7f3ce913e640 1 -- 172.21.15.105:0/1481638765 shutdown_connections 2023-11-28T18:31:50.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.732+0000 7f3ce913e640 1 --2- 172.21.15.105:0/1481638765 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3cc40668d0 0x7f3cc4068d90 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:50.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.732+0000 7f3ce913e640 1 --2- 172.21.15.105:0/1481638765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ce415d4f0 0x7f3ce40cc920 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:50.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.732+0000 7f3ce913e640 1 --2- 172.21.15.105:0/1481638765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce40d3c30 0x7f3ce41555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:50.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.732+0000 7f3ce913e640 1 --2- 172.21.15.105:0/1481638765 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ce40b2d20 0x7f3ce41605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:50.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.732+0000 7f3ce913e640 1 -- 172.21.15.105:0/1481638765 >> 172.21.15.105:0/1481638765 conn(0x7f3ce40d3030 msgr2=0x7f3ce40b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:50.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.732+0000 7f3ce913e640 1 -- 172.21.15.105:0/1481638765 shutdown_connections 2023-11-28T18:31:50.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.732+0000 7f3ce913e640 1 -- 172.21.15.105:0/1481638765 wait complete. 2023-11-28T18:31:50.751 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:31:50.752 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:31:50.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.892+0000 7f00ba9f8640 1 Processor -- start 2023-11-28T18:31:50.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.892+0000 7f00ba9f8640 1 -- start start 2023-11-28T18:31:50.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.892+0000 7f00ba9f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00b4159f40 0x7f00b40caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:50.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.892+0000 7f00ba9f8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b406bf50 con 0x7f00b40cc760 2023-11-28T18:31:50.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.892+0000 7f00ba9f8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b40ccbc0 con 0x7f00b40ad850 2023-11-28T18:31:50.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.892+0000 7f00ba9f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b415a320 con 0x7f00b4159f40 2023-11-28T18:31:50.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.892+0000 7f00b37fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f00b40cc760 0x7f00b40ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49878/0 (socket says 172.21.15.105:49878) 2023-11-28T18:31:50.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.892+0000 7f00b37fe640 1 -- 172.21.15.105:0/3797661262 learned_addr learned my addr 172.21.15.105:0/3797661262 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:50.895 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.892+0000 7f00b3fff640 1 --2- 172.21.15.105:0/3797661262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00b4159f40 0x7f00b40caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:50.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.892+0000 7f00b2ffd640 1 -- 172.21.15.105:0/3797661262 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1209863382 0 0) 0x7f00b406bf50 con 0x7f00b40cc760 2023-11-28T18:31:50.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.892+0000 7f00b2ffd640 1 -- 172.21.15.105:0/3797661262 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f009c005740 con 0x7f00b40cc760 2023-11-28T18:31:50.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.892+0000 7f00b2ffd640 1 -- 172.21.15.105:0/3797661262 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 288340064 0 0) 0x7f00b40ccbc0 con 0x7f00b40ad850 2023-11-28T18:31:50.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.892+0000 7f00b2ffd640 1 -- 172.21.15.105:0/3797661262 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00b406bf50 con 0x7f00b40ad850 2023-11-28T18:31:50.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.892+0000 7f00b2ffd640 1 -- 172.21.15.105:0/3797661262 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3461149519 0 0) 0x7f009c005740 con 0x7f00b40cc760 2023-11-28T18:31:50.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.892+0000 7f00b3fff640 1 -- 172.21.15.105:0/3797661262 >> v1:172.21.15.148:6789/0 conn(0x7f00b40ad850 legacy=0x7f00b40701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:50.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.892+0000 7f00b3fff640 1 -- 172.21.15.105:0/3797661262 >> v1:172.21.15.105:6789/0 conn(0x7f00b40cc760 legacy=0x7f00b40ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:50.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.892+0000 7f00b3fff640 1 -- 172.21.15.105:0/3797661262 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b4071380 con 0x7f00b4159f40 2023-11-28T18:31:50.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.896+0000 7f00b3fff640 1 --2- 172.21.15.105:0/3797661262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00b4159f40 0x7f00b40caf60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f00a40097f0 tx=0x7f00a401c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b24e9b5c443c0254 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:50.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.896+0000 7f00b2ffd640 1 -- 172.21.15.105:0/3797661262 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a8001680 con 0x7f00b4159f40 2023-11-28T18:31:50.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.896+0000 7f00b2ffd640 1 -- 172.21.15.105:0/3797661262 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a40a7050 con 0x7f00b4159f40 2023-11-28T18:31:50.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.896+0000 7f00b2ffd640 1 -- 172.21.15.105:0/3797661262 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a40ab620 con 0x7f00b4159f40 2023-11-28T18:31:50.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.896+0000 7f00ba9f8640 1 -- 172.21.15.105:0/3797661262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00b4159f40 msgr2=0x7f00b40caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:50.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.896+0000 7f00ba9f8640 1 --2- 172.21.15.105:0/3797661262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00b4159f40 0x7f00b40caf60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f00a40097f0 tx=0x7f00a401c320 comp rx=0 tx=0).stop 2023-11-28T18:31:50.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.896+0000 7f00ba9f8640 1 -- 172.21.15.105:0/3797661262 shutdown_connections 2023-11-28T18:31:50.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.896+0000 7f00ba9f8640 1 --2- 172.21.15.105:0/3797661262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00b4159f40 0x7f00b40caf60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:50.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.896+0000 7f00ba9f8640 1 -- 172.21.15.105:0/3797661262 >> 172.21.15.105:0/3797661262 conn(0x7f00b40ceb90 msgr2=0x7f00b40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:50.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.896+0000 7f00ba9f8640 1 -- 172.21.15.105:0/3797661262 shutdown_connections 2023-11-28T18:31:50.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.896+0000 7f00ba9f8640 1 -- 172.21.15.105:0/3797661262 wait complete. 2023-11-28T18:31:50.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00ba9f8640 1 Processor -- start 2023-11-28T18:31:50.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00ba9f8640 1 -- start start 2023-11-28T18:31:50.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00ba9f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00b40ad850 0x7f00b413d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:50.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00ba9f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00b40cc760 0x7f00b41584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:50.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00ba9f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00b4159f40 0x7f00b415ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:50.902 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00ba9f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b415d380 con 0x7f00b40cc760 2023-11-28T18:31:50.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00ba9f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b415c9b0 con 0x7f00b40ad850 2023-11-28T18:31:50.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00ba9f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b414ad50 con 0x7f00b4159f40 2023-11-28T18:31:50.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00b37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00b40cc760 0x7f00b41584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:50.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00b37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00b40cc760 0x7f00b41584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50278/0 (socket says 172.21.15.105:50278) 2023-11-28T18:31:50.903 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00b37fe640 1 -- 172.21.15.105:0/845770607 learned_addr learned my addr 172.21.15.105:0/845770607 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:50.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00b3fff640 1 --2- 172.21.15.105:0/845770607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00b40ad850 0x7f00b413d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:50.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00b8f95640 1 --2- 172.21.15.105:0/845770607 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00b4159f40 0x7f00b415ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:50.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00b37fe640 1 -- 172.21.15.105:0/845770607 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00b4159f40 msgr2=0x7f00b415ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:50.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00b37fe640 1 --2- 172.21.15.105:0/845770607 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00b4159f40 0x7f00b415ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:50.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00b37fe640 1 -- 172.21.15.105:0/845770607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00b40ad850 msgr2=0x7f00b413d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:50.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00b37fe640 1 --2- 172.21.15.105:0/845770607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00b40ad850 0x7f00b413d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:50.904 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00b37fe640 1 -- 172.21.15.105:0/845770607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a4009420 con 0x7f00b40cc760 2023-11-28T18:31:50.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00b3fff640 1 --2- 172.21.15.105:0/845770607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00b40ad850 0x7f00b413d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:50.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00b37fe640 1 --2- 172.21.15.105:0/845770607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00b40cc760 0x7f00b41584f0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f00a800c750 tx=0x7f00a800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:50.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00b0ff9640 1 -- 172.21.15.105:0/845770607 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a8005af0 con 0x7f00b40cc760 2023-11-28T18:31:50.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00b0ff9640 1 -- 172.21.15.105:0/845770607 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a8005c70 con 0x7f00b40cc760 2023-11-28T18:31:50.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00b0ff9640 1 -- 172.21.15.105:0/845770607 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a8007070 con 0x7f00b40cc760 2023-11-28T18:31:50.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00ba9f8640 1 -- 172.21.15.105:0/845770607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b414c820 con 0x7f00b40cc760 2023-11-28T18:31:50.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.900+0000 7f00ba9f8640 1 -- 172.21.15.105:0/845770607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b415c640 con 0x7f00b40cc760 2023-11-28T18:31:50.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.904+0000 7f00ba9f8640 1 -- 172.21.15.105:0/845770607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b414cd40 con 0x7f00b40cc760 2023-11-28T18:31:50.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.904+0000 7f00b0ff9640 1 -- 172.21.15.105:0/845770607 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f00a80071d0 con 0x7f00b40cc760 2023-11-28T18:31:50.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.904+0000 7f00b0ff9640 1 --2- 172.21.15.105:0/845770607 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00a00668d0 0x7f00a0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:50.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.904+0000 7f00b0ff9640 1 -- 172.21.15.105:0/845770607 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f00a8084240 con 0x7f00b40cc760 2023-11-28T18:31:50.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.904+0000 7f00b3fff640 1 --2- 172.21.15.105:0/845770607 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00a00668d0 0x7f00a0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:50.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.904+0000 7f00b3fff640 1 --2- 172.21.15.105:0/845770607 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00a00668d0 0x7f00a0068d90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f00a40ad000 tx=0x7f00a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:50.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:50.908+0000 7f00b0ff9640 1 -- 172.21.15.105:0/845770607 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f00a8051470 con 0x7f00b40cc760 2023-11-28T18:31:51.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.056+0000 7f00ba9f8640 1 -- 172.21.15.105:0/845770607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f00b415cb20 con 0x7f00b40cc760 2023-11-28T18:31:51.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.060+0000 7f00b0ff9640 1 -- 172.21.15.105:0/845770607 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) v1 ==== 93+0+10 (secure 0 0 0) 0x7f00a8054a90 con 0x7f00b40cc760 2023-11-28T18:31:51.059 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:31:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.060+0000 7f00ba9f8640 1 -- 172.21.15.105:0/845770607 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00a00668d0 msgr2=0x7f00a0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.060+0000 7f00ba9f8640 1 --2- 172.21.15.105:0/845770607 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00a00668d0 0x7f00a0068d90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f00a40ad000 tx=0x7f00a4008040 comp rx=0 tx=0).stop 2023-11-28T18:31:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.060+0000 7f00ba9f8640 1 -- 172.21.15.105:0/845770607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00b40cc760 msgr2=0x7f00b41584f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.060+0000 7f00ba9f8640 1 --2- 172.21.15.105:0/845770607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00b40cc760 0x7f00b41584f0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f00a800c750 tx=0x7f00a800cc10 comp rx=0 tx=0).stop 2023-11-28T18:31:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.060+0000 7f00ba9f8640 1 -- 172.21.15.105:0/845770607 shutdown_connections 2023-11-28T18:31:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.060+0000 7f00ba9f8640 1 --2- 172.21.15.105:0/845770607 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00b4159f40 0x7f00b415ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.060+0000 7f00ba9f8640 1 --2- 172.21.15.105:0/845770607 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00a00668d0 0x7f00a0068d90 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.060+0000 7f00ba9f8640 1 --2- 172.21.15.105:0/845770607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00b40cc760 0x7f00b41584f0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:51.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.060+0000 7f00ba9f8640 1 --2- 172.21.15.105:0/845770607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00b40ad850 0x7f00b413d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:51.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.060+0000 7f00ba9f8640 1 -- 172.21.15.105:0/845770607 >> 172.21.15.105:0/845770607 conn(0x7f00b40ceb90 msgr2=0x7f00b414d380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:51.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.060+0000 7f00ba9f8640 1 -- 172.21.15.105:0/845770607 shutdown_connections 2023-11-28T18:31:51.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.060+0000 7f00ba9f8640 1 -- 172.21.15.105:0/845770607 wait complete. 2023-11-28T18:31:51.080 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_distribution' 2023-11-28T18:31:51.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.216+0000 7f8a668dd640 1 Processor -- start 2023-11-28T18:31:51.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.216+0000 7f8a668dd640 1 -- start start 2023-11-28T18:31:51.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.220+0000 7f8a668dd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a6006bf50 0x7f8a6015d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:51.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.220+0000 7f8a668dd640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a600cc7f0 con 0x7f8a600adb80 2023-11-28T18:31:51.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.220+0000 7f8a668dd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a6015dd10 con 0x7f8a60154070 2023-11-28T18:31:51.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.220+0000 7f8a668dd640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a60072540 con 0x7f8a6006bf50 2023-11-28T18:31:51.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.220+0000 7f8a64e7a640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8a600adb80 0x7f8a600acd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:52626/0 (socket says 172.21.15.105:52626) 2023-11-28T18:31:51.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.220+0000 7f8a64e7a640 1 -- 172.21.15.105:0/3721365844 learned_addr learned my addr 172.21.15.105:0/3721365844 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:51.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.220+0000 7f8a5f7fe640 1 --2- 172.21.15.105:0/3721365844 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a6006bf50 0x7f8a6015d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:51.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.220+0000 7f8a5effd640 1 -- 172.21.15.105:0/3721365844 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 487663281 0 0) 0x7f8a600cc7f0 con 0x7f8a600adb80 2023-11-28T18:31:51.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.220+0000 7f8a5effd640 1 -- 172.21.15.105:0/3721365844 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a48003530 con 0x7f8a600adb80 2023-11-28T18:31:51.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.220+0000 7f8a5effd640 1 -- 172.21.15.105:0/3721365844 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 643747574 0 0) 0x7f8a6015dd10 con 0x7f8a60154070 2023-11-28T18:31:51.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.220+0000 7f8a5effd640 1 -- 172.21.15.105:0/3721365844 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a600cc7f0 con 0x7f8a60154070 2023-11-28T18:31:51.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.220+0000 7f8a5f7fe640 1 -- 172.21.15.105:0/3721365844 >> v1:172.21.15.148:6789/0 conn(0x7f8a60154070 legacy=0x7f8a600ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:51.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.220+0000 7f8a5f7fe640 1 -- 172.21.15.105:0/3721365844 >> v1:172.21.15.105:6789/0 conn(0x7f8a600adb80 legacy=0x7f8a600acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:51.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.220+0000 7f8a5f7fe640 1 -- 172.21.15.105:0/3721365844 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6015dff0 con 0x7f8a6006bf50 2023-11-28T18:31:51.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.224+0000 7f8a5f7fe640 1 --2- 172.21.15.105:0/3721365844 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a6006bf50 0x7f8a6015d700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8a540097f0 tx=0x7f8a5401c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fb24270818880d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:51.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.224+0000 7f8a5effd640 1 -- 172.21.15.105:0/3721365844 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a540aa5f0 con 0x7f8a6006bf50 2023-11-28T18:31:51.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.224+0000 7f8a5effd640 1 -- 172.21.15.105:0/3721365844 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a540aa750 con 0x7f8a6006bf50 2023-11-28T18:31:51.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.224+0000 7f8a5effd640 1 -- 172.21.15.105:0/3721365844 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a540aaae0 con 0x7f8a6006bf50 2023-11-28T18:31:51.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.224+0000 7f8a668dd640 1 -- 172.21.15.105:0/3721365844 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a6006bf50 msgr2=0x7f8a6015d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:51.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.224+0000 7f8a668dd640 1 --2- 172.21.15.105:0/3721365844 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a6006bf50 0x7f8a6015d700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8a540097f0 tx=0x7f8a5401c490 comp rx=0 tx=0).stop 2023-11-28T18:31:51.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.224+0000 7f8a668dd640 1 -- 172.21.15.105:0/3721365844 shutdown_connections 2023-11-28T18:31:51.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.224+0000 7f8a668dd640 1 --2- 172.21.15.105:0/3721365844 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a6006bf50 0x7f8a6015d700 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:51.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.224+0000 7f8a668dd640 1 -- 172.21.15.105:0/3721365844 >> 172.21.15.105:0/3721365844 conn(0x7f8a600ceb90 msgr2=0x7f8a600cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:51.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.224+0000 7f8a668dd640 1 -- 172.21.15.105:0/3721365844 shutdown_connections 2023-11-28T18:31:51.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.224+0000 7f8a668dd640 1 -- 172.21.15.105:0/3721365844 wait complete. 2023-11-28T18:31:51.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.224+0000 7f8a668dd640 1 Processor -- start 2023-11-28T18:31:51.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.224+0000 7f8a668dd640 1 -- start start 2023-11-28T18:31:51.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a668dd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a6006bf50 0x7f8a6014e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:51.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a668dd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a600adb80 0x7f8a60150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:51.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a668dd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a60154070 0x7f8a601506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:51.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a668dd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6009a080 con 0x7f8a60154070 2023-11-28T18:31:51.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a668dd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a60099a50 con 0x7f8a600adb80 2023-11-28T18:31:51.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a668dd640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6014f250 con 0x7f8a6006bf50 2023-11-28T18:31:51.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a64e7a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a60154070 0x7f8a601506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:51.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a64e7a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a60154070 0x7f8a601506d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48560/0 (socket says 172.21.15.105:48560) 2023-11-28T18:31:51.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a64e7a640 1 -- 172.21.15.105:0/632210150 learned_addr learned my addr 172.21.15.105:0/632210150 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:51.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a5f7fe640 1 --2- 172.21.15.105:0/632210150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a600adb80 0x7f8a60150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:51.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a5ffff640 1 --2- 172.21.15.105:0/632210150 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a6006bf50 0x7f8a6014e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:51.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a64e7a640 1 -- 172.21.15.105:0/632210150 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a6006bf50 msgr2=0x7f8a6014e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:51.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a64e7a640 1 --2- 172.21.15.105:0/632210150 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a6006bf50 0x7f8a6014e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:51.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a64e7a640 1 -- 172.21.15.105:0/632210150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a600adb80 msgr2=0x7f8a60150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:51.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a64e7a640 1 --2- 172.21.15.105:0/632210150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a600adb80 0x7f8a60150190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:51.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a64e7a640 1 -- 172.21.15.105:0/632210150 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a54009420 con 0x7f8a60154070 2023-11-28T18:31:51.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a64e7a640 1 --2- 172.21.15.105:0/632210150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a60154070 0x7f8a601506d0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800cbc0 tx=0x7f8a5800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:51.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a5cff9640 1 -- 172.21.15.105:0/632210150 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5800ad40 con 0x7f8a60154070 2023-11-28T18:31:51.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a5cff9640 1 -- 172.21.15.105:0/632210150 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a58012400 con 0x7f8a60154070 2023-11-28T18:31:51.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a668dd640 1 -- 172.21.15.105:0/632210150 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a60150d50 con 0x7f8a60154070 2023-11-28T18:31:51.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a5cff9640 1 -- 172.21.15.105:0/632210150 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5800aea0 con 0x7f8a60154070 2023-11-28T18:31:51.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a668dd640 1 -- 172.21.15.105:0/632210150 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a60099ca0 con 0x7f8a60154070 2023-11-28T18:31:51.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.228+0000 7f8a668dd640 1 -- 172.21.15.105:0/632210150 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a6009a1c0 con 0x7f8a60154070 2023-11-28T18:31:51.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.232+0000 7f8a5cff9640 1 -- 172.21.15.105:0/632210150 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8a58007070 con 0x7f8a60154070 2023-11-28T18:31:51.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.232+0000 7f8a5cff9640 1 --2- 172.21.15.105:0/632210150 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a4c0668d0 0x7f8a4c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:51.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.232+0000 7f8a5cff9640 1 -- 172.21.15.105:0/632210150 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f8a58084320 con 0x7f8a60154070 2023-11-28T18:31:51.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.232+0000 7f8a5ffff640 1 --2- 172.21.15.105:0/632210150 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a4c0668d0 0x7f8a4c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:51.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.232+0000 7f8a5ffff640 1 --2- 172.21.15.105:0/632210150 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a4c0668d0 0x7f8a4c068d90 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8a50007fd0 tx=0x7f8a50009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:51.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.236+0000 7f8a5cff9640 1 -- 172.21.15.105:0/632210150 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8a6009a1c0 con 0x7f8a60154070 2023-11-28T18:31:51.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:51.440+0000 7f8a668dd640 1 -- 172.21.15.105:0/632210150 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_distribution"]} v 0) v1 -- 0x7f8a6009a1c0 con 0x7f8a60154070 2023-11-28T18:31:52.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.408+0000 7f8a5cff9640 1 -- 172.21.15.105:0/632210150 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_distribution"]}]=0 v432) v1 ==== 157+0+0 (secure 0 0 0) 0x7f8a6009a1c0 con 0x7f8a60154070 2023-11-28T18:31:52.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.412+0000 7f8a668dd640 1 -- 172.21.15.105:0/632210150 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a4c0668d0 msgr2=0x7f8a4c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:52.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.412+0000 7f8a668dd640 1 --2- 172.21.15.105:0/632210150 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a4c0668d0 0x7f8a4c068d90 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8a50007fd0 tx=0x7f8a50009040 comp rx=0 tx=0).stop 2023-11-28T18:31:52.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.412+0000 7f8a668dd640 1 -- 172.21.15.105:0/632210150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a60154070 msgr2=0x7f8a601506d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:52.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.412+0000 7f8a668dd640 1 --2- 172.21.15.105:0/632210150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a60154070 0x7f8a601506d0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800cbc0 tx=0x7f8a5800a430 comp rx=0 tx=0).stop 2023-11-28T18:31:52.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.412+0000 7f8a668dd640 1 -- 172.21.15.105:0/632210150 shutdown_connections 2023-11-28T18:31:52.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.412+0000 7f8a668dd640 1 --2- 172.21.15.105:0/632210150 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8a4c0668d0 0x7f8a4c068d90 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:52.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.412+0000 7f8a668dd640 1 --2- 172.21.15.105:0/632210150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a60154070 0x7f8a601506d0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:52.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.412+0000 7f8a668dd640 1 --2- 172.21.15.105:0/632210150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a600adb80 0x7f8a60150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:52.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.412+0000 7f8a668dd640 1 --2- 172.21.15.105:0/632210150 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a6006bf50 0x7f8a6014e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:52.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.412+0000 7f8a668dd640 1 -- 172.21.15.105:0/632210150 >> 172.21.15.105:0/632210150 conn(0x7f8a600ceb90 msgr2=0x7f8a600afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:52.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.412+0000 7f8a668dd640 1 -- 172.21.15.105:0/632210150 shutdown_connections 2023-11-28T18:31:52.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.412+0000 7f8a668dd640 1 -- 172.21.15.105:0/632210150 wait complete. 2023-11-28T18:31:52.435 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T18:31:52.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.576+0000 7f997b7f8640 1 Processor -- start 2023-11-28T18:31:52.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.576+0000 7f997b7f8640 1 -- start start 2023-11-28T18:31:52.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.580+0000 7f997b7f8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9974159f40 0x7f99740caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:52.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.580+0000 7f997b7f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f99740ad850 0x7f99740701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:52.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.580+0000 7f997b7f8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997406bf50 con 0x7f99740cc760 2023-11-28T18:31:52.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.580+0000 7f997b7f8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99740ccbc0 con 0x7f9974159f40 2023-11-28T18:31:52.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.580+0000 7f997b7f8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f997415a320 con 0x7f99740ad850 2023-11-28T18:31:52.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.580+0000 7f9979594640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9974159f40 0x7f99740caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:52.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.580+0000 7f9979594640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9974159f40 0x7f99740caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48572/0 (socket says 172.21.15.105:48572) 2023-11-28T18:31:52.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.580+0000 7f9979594640 1 -- 172.21.15.105:0/3141858471 learned_addr learned my addr 172.21.15.105:0/3141858471 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:52.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.580+0000 7f9978d93640 1 --1- 172.21.15.105:0/3141858471 >> v1:172.21.15.105:6789/0 conn(0x7f99740cc760 0x7f99740ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:52632/0 (socket says 172.21.15.105:52632) 2023-11-28T18:31:52.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.580+0000 7f9979d95640 1 --2- 172.21.15.105:0/3141858471 >> v2:172.21.15.148:3300/0 conn(0x7f99740ad850 0x7f99740701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:52.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.580+0000 7f9979594640 1 -- 172.21.15.105:0/3141858471 >> v2:172.21.15.148:3300/0 conn(0x7f99740ad850 msgr2=0x7f99740701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:52.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.580+0000 7f9979594640 1 --2- 172.21.15.105:0/3141858471 >> v2:172.21.15.148:3300/0 conn(0x7f99740ad850 0x7f99740701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:52.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.580+0000 7f9979594640 1 -- 172.21.15.105:0/3141858471 >> v1:172.21.15.105:6789/0 conn(0x7f99740cc760 legacy=0x7f99740ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:52.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.580+0000 7f9979594640 1 -- 172.21.15.105:0/3141858471 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9974071380 con 0x7f9974159f40 2023-11-28T18:31:52.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.580+0000 7f9979d95640 1 --2- 172.21.15.105:0/3141858471 >> v2:172.21.15.148:3300/0 conn(0x7f99740ad850 0x7f99740701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:31:52.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.584+0000 7f9979594640 1 --2- 172.21.15.105:0/3141858471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9974159f40 0x7f99740caf60 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f99740b0850 tx=0x7f997001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649c63c29c430d2b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:52.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.584+0000 7f996bfff640 1 -- 172.21.15.105:0/3141858471 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99700ab640 con 0x7f9974159f40 2023-11-28T18:31:52.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.584+0000 7f996bfff640 1 -- 172.21.15.105:0/3141858471 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99700a7070 con 0x7f9974159f40 2023-11-28T18:31:52.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.584+0000 7f996bfff640 1 -- 172.21.15.105:0/3141858471 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99700ab9c0 con 0x7f9974159f40 2023-11-28T18:31:52.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.584+0000 7f997b7f8640 1 -- 172.21.15.105:0/3141858471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9974159f40 msgr2=0x7f99740caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:52.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.584+0000 7f997b7f8640 1 --2- 172.21.15.105:0/3141858471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9974159f40 0x7f99740caf60 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f99740b0850 tx=0x7f997001c440 comp rx=0 tx=0).stop 2023-11-28T18:31:52.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.584+0000 7f997b7f8640 1 -- 172.21.15.105:0/3141858471 shutdown_connections 2023-11-28T18:31:52.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.584+0000 7f997b7f8640 1 --2- 172.21.15.105:0/3141858471 >> v2:172.21.15.148:3300/0 conn(0x7f99740ad850 0x7f99740701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:52.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.584+0000 7f997b7f8640 1 --2- 172.21.15.105:0/3141858471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9974159f40 0x7f99740caf60 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:52.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.584+0000 7f997b7f8640 1 -- 172.21.15.105:0/3141858471 >> 172.21.15.105:0/3141858471 conn(0x7f99740ceb90 msgr2=0x7f99740cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:52.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.584+0000 7f997b7f8640 1 -- 172.21.15.105:0/3141858471 shutdown_connections 2023-11-28T18:31:52.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.584+0000 7f997b7f8640 1 -- 172.21.15.105:0/3141858471 wait complete. 2023-11-28T18:31:52.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.584+0000 7f997b7f8640 1 Processor -- start 2023-11-28T18:31:52.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.584+0000 7f997b7f8640 1 -- start start 2023-11-28T18:31:52.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f997b7f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99740ad850 0x7f9974158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:52.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f997b7f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99740cc760 0x7f9974155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:52.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f997b7f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9974159f40 0x7f9974152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:52.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f997b7f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99741590c0 con 0x7f9974159f40 2023-11-28T18:31:52.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f997b7f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9974157ae0 con 0x7f99740cc760 2023-11-28T18:31:52.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f997b7f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9974157dc0 con 0x7f99740ad850 2023-11-28T18:31:52.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f9979d95640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9974159f40 0x7f9974152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:52.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f9979d95640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9974159f40 0x7f9974152090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48586/0 (socket says 172.21.15.105:48586) 2023-11-28T18:31:52.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f9979d95640 1 -- 172.21.15.105:0/437840990 learned_addr learned my addr 172.21.15.105:0/437840990 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:52.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f9978d93640 1 --2- 172.21.15.105:0/437840990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99740cc760 0x7f9974155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:52.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f9979d95640 1 -- 172.21.15.105:0/437840990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99740ad850 msgr2=0x7f9974158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:52.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f9979594640 1 --2- 172.21.15.105:0/437840990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99740ad850 0x7f9974158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:52.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f9979d95640 1 --2- 172.21.15.105:0/437840990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99740ad850 0x7f9974158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:52.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f9979d95640 1 -- 172.21.15.105:0/437840990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99740cc760 msgr2=0x7f9974155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:52.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f9979d95640 1 --2- 172.21.15.105:0/437840990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99740cc760 0x7f9974155ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:52.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f9979d95640 1 -- 172.21.15.105:0/437840990 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99700094d0 con 0x7f9974159f40 2023-11-28T18:31:52.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f9978d93640 1 --2- 172.21.15.105:0/437840990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99740cc760 0x7f9974155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:31:52.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f9979594640 1 --2- 172.21.15.105:0/437840990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99740ad850 0x7f9974158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:31:52.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f9979d95640 1 --2- 172.21.15.105:0/437840990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9974159f40 0x7f9974152090 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f996c00cef0 tx=0x7f996c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:52.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f9969ffb640 1 -- 172.21.15.105:0/437840990 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c00d840 con 0x7f9974159f40 2023-11-28T18:31:52.590 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f9969ffb640 1 -- 172.21.15.105:0/437840990 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c00d9c0 con 0x7f9974159f40 2023-11-28T18:31:52.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f9969ffb640 1 -- 172.21.15.105:0/437840990 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c00dd70 con 0x7f9974159f40 2023-11-28T18:31:52.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f997b7f8640 1 -- 172.21.15.105:0/437840990 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99741580a0 con 0x7f9974159f40 2023-11-28T18:31:52.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f997b7f8640 1 -- 172.21.15.105:0/437840990 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9974158660 con 0x7f9974159f40 2023-11-28T18:31:52.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f997b7f8640 1 -- 172.21.15.105:0/437840990 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9974067570 con 0x7f9974159f40 2023-11-28T18:31:52.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.588+0000 7f9969ffb640 1 -- 172.21.15.105:0/437840990 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f996c01e3b0 con 0x7f9974159f40 2023-11-28T18:31:52.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.592+0000 7f9969ffb640 1 --2- 172.21.15.105:0/437840990 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99580668d0 0x7f9958068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:52.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.592+0000 7f9969ffb640 1 -- 172.21.15.105:0/437840990 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f996c084dc0 con 0x7f9974159f40 2023-11-28T18:31:52.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.592+0000 7f9979594640 1 --2- 172.21.15.105:0/437840990 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99580668d0 0x7f9958068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:52.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.592+0000 7f9979594640 1 --2- 172.21.15.105:0/437840990 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99580668d0 0x7f9958068d90 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f99700af000 tx=0x7f9970008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:52.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.596+0000 7f9969ffb640 1 -- 172.21.15.105:0/437840990 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f996c0521b0 con 0x7f9974159f40 2023-11-28T18:31:52.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.744+0000 7f997b7f8640 1 -- 172.21.15.105:0/437840990 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f99740cf950 con 0x7f9974159f40 2023-11-28T18:31:52.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.748+0000 7f9969ffb640 1 -- 172.21.15.105:0/437840990 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v67)=0 v67) v1 ==== 122+0+0 (secure 0 0 0) 0x7f996c01a0c0 con 0x7f9974159f40 2023-11-28T18:31:52.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.752+0000 7f997b7f8640 1 -- 172.21.15.105:0/437840990 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99580668d0 msgr2=0x7f9958068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.752+0000 7f997b7f8640 1 --2- 172.21.15.105:0/437840990 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99580668d0 0x7f9958068d90 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f99700af000 tx=0x7f9970008040 comp rx=0 tx=0).stop 2023-11-28T18:31:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.752+0000 7f997b7f8640 1 -- 172.21.15.105:0/437840990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9974159f40 msgr2=0x7f9974152090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.752+0000 7f997b7f8640 1 --2- 172.21.15.105:0/437840990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9974159f40 0x7f9974152090 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f996c00cef0 tx=0x7f996c00a710 comp rx=0 tx=0).stop 2023-11-28T18:31:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.752+0000 7f997b7f8640 1 -- 172.21.15.105:0/437840990 shutdown_connections 2023-11-28T18:31:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.752+0000 7f997b7f8640 1 --2- 172.21.15.105:0/437840990 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99580668d0 0x7f9958068d90 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.752+0000 7f997b7f8640 1 --2- 172.21.15.105:0/437840990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9974159f40 0x7f9974152090 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.752+0000 7f997b7f8640 1 --2- 172.21.15.105:0/437840990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99740cc760 0x7f9974155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.752+0000 7f997b7f8640 1 --2- 172.21.15.105:0/437840990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99740ad850 0x7f9974158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.752+0000 7f997b7f8640 1 -- 172.21.15.105:0/437840990 >> 172.21.15.105:0/437840990 conn(0x7f99740ceb90 msgr2=0x7f99740ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:52.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.752+0000 7f997b7f8640 1 -- 172.21.15.105:0/437840990 shutdown_connections 2023-11-28T18:31:52.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:52.752+0000 7f997b7f8640 1 -- 172.21.15.105:0/437840990 wait complete. 2023-11-28T18:31:52.771 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T18:31:52.772 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:31:52.889 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:31:52.993 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T18:31:52.993 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:31:52.994 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:31:53.003 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:31:53.003 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:31:53.003 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:31:53.014 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:31:53.014 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:31:53.014 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:31:53.055 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:31:53.055 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:31:53.055 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:31:53.071 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:31:53.072 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T18:31:53.072 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:31:53.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f40429df640 1 Processor -- start 2023-11-28T18:31:53.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f40429df640 1 -- start start 2023-11-28T18:31:53.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f40429df640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f403c071960 0x7f403c15f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:53.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f40429df640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403c0701d0 con 0x7f403c0b4290 2023-11-28T18:31:53.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f40429df640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403c06df80 con 0x7f403c15b030 2023-11-28T18:31:53.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f40429df640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f403c071df0 con 0x7f403c071960 2023-11-28T18:31:53.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403bfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f403c0b4290 0x7f403c0b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:52634/0 (socket says 172.21.15.105:52634) 2023-11-28T18:31:53.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403bfff640 1 -- 172.21.15.105:0/2903414040 learned_addr learned my addr 172.21.15.105:0/2903414040 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:53.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403affd640 1 -- 172.21.15.105:0/2903414040 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929965949 0 0) 0x7f403c0701d0 con 0x7f403c0b4290 2023-11-28T18:31:53.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403affd640 1 -- 172.21.15.105:0/2903414040 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4024003530 con 0x7f403c0b4290 2023-11-28T18:31:53.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403b7fe640 1 --2- 172.21.15.105:0/2903414040 >> v2:172.21.15.148:3300/0 conn(0x7f403c071960 0x7f403c15f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:53.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403affd640 1 -- 172.21.15.105:0/2903414040 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1814955142 0 0) 0x7f4024003530 con 0x7f403c0b4290 2023-11-28T18:31:53.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403affd640 1 -- 172.21.15.105:0/2903414040 >> v2:172.21.15.148:3300/0 conn(0x7f403c071960 msgr2=0x7f403c15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:53.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403affd640 1 --2- 172.21.15.105:0/2903414040 >> v2:172.21.15.148:3300/0 conn(0x7f403c071960 0x7f403c15f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:53.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403affd640 1 -- 172.21.15.105:0/2903414040 >> v1:172.21.15.148:6789/0 conn(0x7f403c15b030 legacy=0x7f403c0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:53.253 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403affd640 1 -- 172.21.15.105:0/2903414040 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c0720d0 con 0x7f403c0b4290 2023-11-28T18:31:53.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403affd640 1 -- 172.21.15.105:0/2903414040 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f402c001680 con 0x7f403c0b4290 2023-11-28T18:31:53.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403affd640 1 -- 172.21.15.105:0/2903414040 >> v1:172.21.15.105:6789/0 conn(0x7f403c0b4290 legacy=0x7f403c0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:53.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403affd640 1 --2- 172.21.15.105:0/2903414040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4024003980 0x7f4024003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:53.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403affd640 1 --2- 172.21.15.105:0/2903414040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40240047d0 0x7f4024004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:53.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403affd640 1 --2- 172.21.15.105:0/2903414040 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4024005190 0x7f402400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:53.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403affd640 1 -- 172.21.15.105:0/2903414040 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c071df0 con 0x7f4024003980 2023-11-28T18:31:53.254 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403affd640 1 -- 172.21.15.105:0/2903414040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402400c0e0 con 0x7f40240047d0 2023-11-28T18:31:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403affd640 1 -- 172.21.15.105:0/2903414040 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402400c2a0 con 0x7f4024005190 2023-11-28T18:31:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403bfff640 1 --2- 172.21.15.105:0/2903414040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4024003980 0x7f4024003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403b7fe640 1 --2- 172.21.15.105:0/2903414040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40240047d0 0x7f4024004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f4040f7c640 1 --2- 172.21.15.105:0/2903414040 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4024005190 0x7f402400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403bfff640 1 -- 172.21.15.105:0/2903414040 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4024005190 msgr2=0x7f402400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403bfff640 1 --2- 172.21.15.105:0/2903414040 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4024005190 0x7f402400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403bfff640 1 -- 172.21.15.105:0/2903414040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40240047d0 msgr2=0x7f4024004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403bfff640 1 --2- 172.21.15.105:0/2903414040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40240047d0 0x7f4024004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.248+0000 7f403bfff640 1 -- 172.21.15.105:0/2903414040 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4024003530 con 0x7f4024003980 2023-11-28T18:31:53.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.252+0000 7f4040f7c640 1 -- 172.21.15.105:0/2903414040 reap_dead start 2023-11-28T18:31:53.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.252+0000 7f403bfff640 1 --2- 172.21.15.105:0/2903414040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4024003980 0x7f4024003da0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f402c00c240 tx=0x7f402c01cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495689bb7b801486 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:53.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.252+0000 7f403affd640 1 -- 172.21.15.105:0/2903414040 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c0a5b60 con 0x7f4024003980 2023-11-28T18:31:53.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.252+0000 7f403affd640 1 -- 172.21.15.105:0/2903414040 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c0a5cc0 con 0x7f4024003980 2023-11-28T18:31:53.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.252+0000 7f403affd640 1 -- 172.21.15.105:0/2903414040 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c005b20 con 0x7f4024003980 2023-11-28T18:31:53.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.252+0000 7f40429df640 1 -- 172.21.15.105:0/2903414040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4024003980 msgr2=0x7f4024003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:53.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.252+0000 7f40429df640 1 --2- 172.21.15.105:0/2903414040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4024003980 0x7f4024003da0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f402c00c240 tx=0x7f402c01cbd0 comp rx=0 tx=0).stop 2023-11-28T18:31:53.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.252+0000 7f40429df640 1 -- 172.21.15.105:0/2903414040 shutdown_connections 2023-11-28T18:31:53.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.252+0000 7f40429df640 1 --2- 172.21.15.105:0/2903414040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4024003980 0x7f4024003da0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:53.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.252+0000 7f40429df640 1 -- 172.21.15.105:0/2903414040 >> 172.21.15.105:0/2903414040 conn(0x7f403c0ceb90 msgr2=0x7f403c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:53.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.252+0000 7f40429df640 1 -- 172.21.15.105:0/2903414040 shutdown_connections 2023-11-28T18:31:53.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.252+0000 7f40429df640 1 -- 172.21.15.105:0/2903414040 wait complete. 2023-11-28T18:31:53.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f40429df640 1 Processor -- start 2023-11-28T18:31:53.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f40429df640 1 -- start start 2023-11-28T18:31:53.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f40429df640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4024003980 0x7f403c155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:53.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f40429df640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f403c1f65c0 0x7f403c099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:53.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f40429df640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f403c09bc70 0x7f403c09c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:53.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f40429df640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c09c660 con 0x7f4024003980 2023-11-28T18:31:53.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f40429df640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c09a3c0 con 0x7f403c09bc70 2023-11-28T18:31:53.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f40429df640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403c09a6a0 con 0x7f403c1f65c0 2023-11-28T18:31:53.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f403bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4024003980 0x7f403c155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:53.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f403bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4024003980 0x7f403c155650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48594/0 (socket says 172.21.15.105:48594) 2023-11-28T18:31:53.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f403bfff640 1 -- 172.21.15.105:0/1248350682 learned_addr learned my addr 172.21.15.105:0/1248350682 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:53.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f403b7fe640 1 --2- 172.21.15.105:0/1248350682 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f403c1f65c0 0x7f403c099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:53.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f4040f7c640 1 --2- 172.21.15.105:0/1248350682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f403c09bc70 0x7f403c09c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f403bfff640 1 -- 172.21.15.105:0/1248350682 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f403c1f65c0 msgr2=0x7f403c099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f403bfff640 1 --2- 172.21.15.105:0/1248350682 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f403c1f65c0 0x7f403c099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f403bfff640 1 -- 172.21.15.105:0/1248350682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f403c09bc70 msgr2=0x7f403c09c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f403bfff640 1 --2- 172.21.15.105:0/1248350682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f403c09bc70 0x7f403c09c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f403bfff640 1 -- 172.21.15.105:0/1248350682 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c09a980 con 0x7f4024003980 2023-11-28T18:31:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f4040f7c640 1 --2- 172.21.15.105:0/1248350682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f403c09bc70 0x7f403c09c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f403b7fe640 1 --2- 172.21.15.105:0/1248350682 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f403c1f65c0 0x7f403c099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f403bfff640 1 --2- 172.21.15.105:0/1248350682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4024003980 0x7f403c155650 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f402c00c240 tx=0x7f4024005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f4038ff9640 1 -- 172.21.15.105:0/1248350682 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c0b3070 con 0x7f4024003980 2023-11-28T18:31:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f4038ff9640 1 -- 172.21.15.105:0/1248350682 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c004e90 con 0x7f4024003980 2023-11-28T18:31:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f40429df640 1 -- 172.21.15.105:0/1248350682 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403c09ac60 con 0x7f4024003980 2023-11-28T18:31:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f4038ff9640 1 -- 172.21.15.105:0/1248350682 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c000ee0 con 0x7f4024003980 2023-11-28T18:31:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.256+0000 7f40429df640 1 -- 172.21.15.105:0/1248350682 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403c155b90 con 0x7f4024003980 2023-11-28T18:31:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.260+0000 7f40429df640 1 -- 172.21.15.105:0/1248350682 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f403c158b30 con 0x7f4024003980 2023-11-28T18:31:53.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.260+0000 7f4038ff9640 1 -- 172.21.15.105:0/1248350682 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f402c007140 con 0x7f4024003980 2023-11-28T18:31:53.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.260+0000 7f4038ff9640 1 --2- 172.21.15.105:0/1248350682 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f40280668d0 0x7f4028068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:53.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.260+0000 7f4038ff9640 1 -- 172.21.15.105:0/1248350682 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f402c11e410 con 0x7f4024003980 2023-11-28T18:31:53.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.260+0000 7f403b7fe640 1 --2- 172.21.15.105:0/1248350682 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f40280668d0 0x7f4028068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:53.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.260+0000 7f403b7fe640 1 --2- 172.21.15.105:0/1248350682 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f40280668d0 0x7f4028068d90 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f403c0af540 tx=0x7f4030008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:53.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.264+0000 7f4038ff9640 1 -- 172.21.15.105:0/1248350682 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f402c0eb750 con 0x7f4024003980 2023-11-28T18:31:53.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.448+0000 7f40429df640 1 -- 172.21.15.105:0/1248350682 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f403c067570 con 0x7f4024003980 2023-11-28T18:31:53.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.448+0000 7f4038ff9640 1 -- 172.21.15.105:0/1248350682 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+3713 (secure 0 0 0) 0x7f402c0eed70 con 0x7f4024003980 2023-11-28T18:31:53.450 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:53.450 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":128,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/2073622668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2073622668},{"type":"v1","addr":"172.21.15.148:6834","nonce":2073622668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":8428,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/288931369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":288931369},{"type":"v1","addr":"172.21.15.148:6835","nonce":288931369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":128},{"gid":9018,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/2679260406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2679260406},{"type":"v1","addr":"172.21.15.105:6836","nonce":2679260406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":128},{"gid":9021,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/4133610275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":4133610275},{"type":"v1","addr":"172.21.15.105:6837","nonce":4133610275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":128}],"filesystems":[]} 2023-11-28T18:31:53.451 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 128 2023-11-28T18:31:53.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.452+0000 7f40429df640 1 -- 172.21.15.105:0/1248350682 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f40280668d0 msgr2=0x7f4028068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:53.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.452+0000 7f40429df640 1 --2- 172.21.15.105:0/1248350682 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f40280668d0 0x7f4028068d90 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f403c0af540 tx=0x7f4030008040 comp rx=0 tx=0).stop 2023-11-28T18:31:53.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.452+0000 7f40429df640 1 -- 172.21.15.105:0/1248350682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4024003980 msgr2=0x7f403c155650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:53.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.452+0000 7f40429df640 1 --2- 172.21.15.105:0/1248350682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4024003980 0x7f403c155650 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f402c00c240 tx=0x7f4024005800 comp rx=0 tx=0).stop 2023-11-28T18:31:53.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.452+0000 7f40429df640 1 -- 172.21.15.105:0/1248350682 shutdown_connections 2023-11-28T18:31:53.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.452+0000 7f40429df640 1 --2- 172.21.15.105:0/1248350682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f403c09bc70 0x7f403c09c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:53.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.452+0000 7f40429df640 1 --2- 172.21.15.105:0/1248350682 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f403c1f65c0 0x7f403c099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:53.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.452+0000 7f40429df640 1 --2- 172.21.15.105:0/1248350682 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f40280668d0 0x7f4028068d90 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:53.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.452+0000 7f40429df640 1 --2- 172.21.15.105:0/1248350682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4024003980 0x7f403c155650 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:53.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.452+0000 7f40429df640 1 -- 172.21.15.105:0/1248350682 >> 172.21.15.105:0/1248350682 conn(0x7f403c0ceb90 msgr2=0x7f403c0ca5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:53.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.452+0000 7f40429df640 1 -- 172.21.15.105:0/1248350682 shutdown_connections 2023-11-28T18:31:53.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.452+0000 7f40429df640 1 -- 172.21.15.105:0/1248350682 wait complete. 2023-11-28T18:31:53.471 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T18:31:53.472 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T18:31:53.472 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T18:31:53.473 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:31:53.473 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T18:31:53.473 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T18:31:53.474 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T18:31:53.474 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:31:53.475 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T18:31:53.475 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T18:31:53.475 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T18:31:53.476 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:31:53.476 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T18:31:53.476 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T18:31:53.477 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T18:31:53.477 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:31:53.484 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T18:31:53.484 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T18:31:53.486 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T18:31:53.487 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T18:31:53.489 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T18:31:53.490 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T18:31:53.492 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T18:31:53.493 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T18:31:53.536 INFO:tasks.ceph.mds.a:Started 2023-11-28T18:31:53.537 DEBUG:teuthology.parallel:result is None 2023-11-28T18:31:53.538 INFO:tasks.ceph.mds.b:Started 2023-11-28T18:31:53.538 DEBUG:teuthology.parallel:result is None 2023-11-28T18:31:53.539 INFO:tasks.ceph.mds.c:Started 2023-11-28T18:31:53.540 DEBUG:teuthology.parallel:result is None 2023-11-28T18:31:53.540 INFO:tasks.ceph.mds.d:Started 2023-11-28T18:31:53.541 DEBUG:teuthology.parallel:result is None 2023-11-28T18:31:53.541 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T18:31:53.624 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2023-11-28T18:31:53.626 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2023-11-28T18:31:53.626 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2023-11-28T18:31:53.629 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2023-11-28T18:31:53.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.684+0000 7fe2b153b640 1 Processor -- start 2023-11-28T18:31:53.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.684+0000 7fe2b153b640 1 -- start start 2023-11-28T18:31:53.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.688+0000 7fe2b153b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2ac15cef0 0x7fe2ac0d1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:53.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.688+0000 7fe2b153b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ac0ce840 con 0x7fe2ac0cfb30 2023-11-28T18:31:53.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.688+0000 7fe2b153b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ac0cd990 con 0x7fe2ac0d1b40 2023-11-28T18:31:53.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.688+0000 7fe2b153b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac0b4780 con 0x7fe2ac15cef0 2023-11-28T18:31:53.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.688+0000 7fe2b0d3a640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe2ac0cfb30 0x7fe2ac166fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:52660/0 (socket says 172.21.15.105:52660) 2023-11-28T18:31:53.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.688+0000 7fe2b0d3a640 1 -- 172.21.15.105:0/3350986665 learned_addr learned my addr 172.21.15.105:0/3350986665 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:53.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.688+0000 7fe2abfff640 1 --2- 172.21.15.105:0/3350986665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2ac15cef0 0x7fe2ac0d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:53.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.688+0000 7fe2aaffd640 1 -- 172.21.15.105:0/3350986665 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3551893545 0 0) 0x7fe2ac0ce840 con 0x7fe2ac0cfb30 2023-11-28T18:31:53.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.688+0000 7fe2aaffd640 1 -- 172.21.15.105:0/3350986665 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe294003530 con 0x7fe2ac0cfb30 2023-11-28T18:31:53.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.688+0000 7fe2aaffd640 1 -- 172.21.15.105:0/3350986665 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2729118260 0 0) 0x7fe2ac0cd990 con 0x7fe2ac0d1b40 2023-11-28T18:31:53.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.688+0000 7fe2aaffd640 1 -- 172.21.15.105:0/3350986665 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2ac0ce840 con 0x7fe2ac0d1b40 2023-11-28T18:31:53.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.688+0000 7fe2abfff640 1 -- 172.21.15.105:0/3350986665 >> v1:172.21.15.148:6789/0 conn(0x7fe2ac0d1b40 legacy=0x7fe2ac0cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:53.689 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.688+0000 7fe2abfff640 1 -- 172.21.15.105:0/3350986665 >> v1:172.21.15.105:6789/0 conn(0x7fe2ac0cfb30 legacy=0x7fe2ac166fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:53.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.688+0000 7fe2abfff640 1 -- 172.21.15.105:0/3350986665 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac0cecd0 con 0x7fe2ac15cef0 2023-11-28T18:31:53.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.692+0000 7fe2abfff640 1 --2- 172.21.15.105:0/3350986665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2ac15cef0 0x7fe2ac0d1600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0097f0 tx=0x7fe29c01c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a85ef52f4306cea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:53.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.692+0000 7fe2aaffd640 1 -- 172.21.15.105:0/3350986665 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c0ab600 con 0x7fe2ac15cef0 2023-11-28T18:31:53.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.692+0000 7fe2aaffd640 1 -- 172.21.15.105:0/3350986665 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29c0ab760 con 0x7fe2ac15cef0 2023-11-28T18:31:53.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.692+0000 7fe2aaffd640 1 -- 172.21.15.105:0/3350986665 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c0abaf0 con 0x7fe2ac15cef0 2023-11-28T18:31:53.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.692+0000 7fe2b153b640 1 -- 172.21.15.105:0/3350986665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2ac15cef0 msgr2=0x7fe2ac0d1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:53.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.692+0000 7fe2b153b640 1 --2- 172.21.15.105:0/3350986665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2ac15cef0 0x7fe2ac0d1600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0097f0 tx=0x7fe29c01c400 comp rx=0 tx=0).stop 2023-11-28T18:31:53.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.692+0000 7fe2b153b640 1 -- 172.21.15.105:0/3350986665 shutdown_connections 2023-11-28T18:31:53.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.692+0000 7fe2b153b640 1 --2- 172.21.15.105:0/3350986665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2ac15cef0 0x7fe2ac0d1600 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:53.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.692+0000 7fe2b153b640 1 -- 172.21.15.105:0/3350986665 >> 172.21.15.105:0/3350986665 conn(0x7fe2ac0d2e10 msgr2=0x7fe2ac0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:53.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.692+0000 7fe2b153b640 1 -- 172.21.15.105:0/3350986665 shutdown_connections 2023-11-28T18:31:53.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.692+0000 7fe2b153b640 1 -- 172.21.15.105:0/3350986665 wait complete. 2023-11-28T18:31:53.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.692+0000 7fe2b153b640 1 Processor -- start 2023-11-28T18:31:53.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.692+0000 7fe2b153b640 1 -- start start 2023-11-28T18:31:53.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2b153b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2ac0cfb30 0x7fe2ac150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:53.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2b153b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2ac0d1b40 0x7fe2ac154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:53.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2b153b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac15cef0 0x7fe2ac1582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:53.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2b153b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac15daf0 con 0x7fe2ac15cef0 2023-11-28T18:31:53.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2b153b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac1553d0 con 0x7fe2ac0cfb30 2023-11-28T18:31:53.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2b153b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac1556b0 con 0x7fe2ac0d1b40 2023-11-28T18:31:53.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2b0d3a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac15cef0 0x7fe2ac1582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:53.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2b0d3a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac15cef0 0x7fe2ac1582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48620/0 (socket says 172.21.15.105:48620) 2023-11-28T18:31:53.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2b0d3a640 1 -- 172.21.15.105:0/836117657 learned_addr learned my addr 172.21.15.105:0/836117657 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:53.698 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2abfff640 1 --2- 172.21.15.105:0/836117657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2ac0cfb30 0x7fe2ac150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:53.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2ab7fe640 1 --2- 172.21.15.105:0/836117657 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2ac0d1b40 0x7fe2ac154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:53.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2b0d3a640 1 -- 172.21.15.105:0/836117657 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2ac0d1b40 msgr2=0x7fe2ac154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:53.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2b0d3a640 1 --2- 172.21.15.105:0/836117657 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2ac0d1b40 0x7fe2ac154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:53.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2b0d3a640 1 -- 172.21.15.105:0/836117657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2ac0cfb30 msgr2=0x7fe2ac150590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:53.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2b0d3a640 1 --2- 172.21.15.105:0/836117657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2ac0cfb30 0x7fe2ac150590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:53.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2b0d3a640 1 -- 172.21.15.105:0/836117657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c009420 con 0x7fe2ac15cef0 2023-11-28T18:31:53.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2b0d3a640 1 --2- 172.21.15.105:0/836117657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac15cef0 0x7fe2ac1582c0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400cae0 tx=0x7fe2a400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:53.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2a8ff9640 1 -- 172.21.15.105:0/836117657 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a400ac40 con 0x7fe2ac15cef0 2023-11-28T18:31:53.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2a8ff9640 1 -- 172.21.15.105:0/836117657 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a400ada0 con 0x7fe2ac15cef0 2023-11-28T18:31:53.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2a8ff9640 1 -- 172.21.15.105:0/836117657 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4012600 con 0x7fe2ac15cef0 2023-11-28T18:31:53.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2b153b640 1 -- 172.21.15.105:0/836117657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ac153e70 con 0x7fe2ac15cef0 2023-11-28T18:31:53.701 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2b153b640 1 -- 172.21.15.105:0/836117657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ac154430 con 0x7fe2ac15cef0 2023-11-28T18:31:53.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2b153b640 1 -- 172.21.15.105:0/836117657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2ac0d03c0 con 0x7fe2ac15cef0 2023-11-28T18:31:53.702 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.696+0000 7fe2a8ff9640 1 -- 172.21.15.105:0/836117657 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fe2a4007070 con 0x7fe2ac15cef0 2023-11-28T18:31:53.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.700+0000 7fe2a8ff9640 1 --2- 172.21.15.105:0/836117657 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe2980668d0 0x7fe298068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:53.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.700+0000 7fe2a8ff9640 1 -- 172.21.15.105:0/836117657 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fe2a40841a0 con 0x7fe2ac15cef0 2023-11-28T18:31:53.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.700+0000 7fe2abfff640 1 --2- 172.21.15.105:0/836117657 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe2980668d0 0x7fe298068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:53.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.700+0000 7fe2abfff640 1 --2- 172.21.15.105:0/836117657 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe2980668d0 0x7fe298068d90 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0af000 tx=0x7fe29c0b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:53.703 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.704+0000 7fe2a8ff9640 1 -- 172.21.15.105:0/836117657 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe2a4051590 con 0x7fe2ac15cef0 2023-11-28T18:31:53.863 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:53.864+0000 7fe2b153b640 1 -- 172.21.15.105:0/836117657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe2ac1545f0 con 0x7fe2ac15cef0 2023-11-28T18:31:55.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.208+0000 7fe2a8ff9640 1 -- 172.21.15.105:0/836117657 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v125) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe2a4054bb0 con 0x7fe2ac15cef0 2023-11-28T18:31:55.208 INFO:teuthology.orchestra.run.smithi105.stderr: removed all blocklist entries 2023-11-28T18:31:55.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.208+0000 7fe2b153b640 1 -- 172.21.15.105:0/836117657 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe2980668d0 msgr2=0x7fe298068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:55.210 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.208+0000 7fe2b153b640 1 --2- 172.21.15.105:0/836117657 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe2980668d0 0x7fe298068d90 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0af000 tx=0x7fe29c0b6040 comp rx=0 tx=0).stop 2023-11-28T18:31:55.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.208+0000 7fe2b153b640 1 -- 172.21.15.105:0/836117657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac15cef0 msgr2=0x7fe2ac1582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:55.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.208+0000 7fe2b153b640 1 --2- 172.21.15.105:0/836117657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac15cef0 0x7fe2ac1582c0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400cae0 tx=0x7fe2a400a060 comp rx=0 tx=0).stop 2023-11-28T18:31:55.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.208+0000 7fe2b153b640 1 -- 172.21.15.105:0/836117657 shutdown_connections 2023-11-28T18:31:55.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.208+0000 7fe2b153b640 1 --2- 172.21.15.105:0/836117657 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe2980668d0 0x7fe298068d90 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.208+0000 7fe2b153b640 1 --2- 172.21.15.105:0/836117657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac15cef0 0x7fe2ac1582c0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.208+0000 7fe2b153b640 1 --2- 172.21.15.105:0/836117657 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2ac0d1b40 0x7fe2ac154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.208+0000 7fe2b153b640 1 --2- 172.21.15.105:0/836117657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2ac0cfb30 0x7fe2ac150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.208+0000 7fe2b153b640 1 -- 172.21.15.105:0/836117657 >> 172.21.15.105:0/836117657 conn(0x7fe2ac0d2e10 msgr2=0x7fe2ac0b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:55.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.212+0000 7fe2b153b640 1 -- 172.21.15.105:0/836117657 shutdown_connections 2023-11-28T18:31:55.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.212+0000 7fe2b153b640 1 -- 172.21.15.105:0/836117657 wait complete. 2023-11-28T18:31:55.234 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T18:31:55.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.376+0000 7fa21cd80640 1 Processor -- start 2023-11-28T18:31:55.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.376+0000 7fa21cd80640 1 -- start start 2023-11-28T18:31:55.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.376+0000 7fa21cd80640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa218071a10 0x7fa2180ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:55.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.376+0000 7fa21cd80640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa218070a10 0x7fa218070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:55.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.376+0000 7fa21cd80640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2180cefa0 con 0x7fa21815d3c0 2023-11-28T18:31:55.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.376+0000 7fa21cd80640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa218071360 con 0x7fa218071a10 2023-11-28T18:31:55.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.376+0000 7fa21cd80640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21815d7a0 con 0x7fa218070a10 2023-11-28T18:31:55.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.376+0000 7fa215d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa218071a10 0x7fa2180ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:55.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.376+0000 7fa21659c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa21815d3c0 0x7fa218161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:52662/0 (socket says 172.21.15.105:52662) 2023-11-28T18:31:55.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.376+0000 7fa21659c640 1 -- 172.21.15.105:0/2723257272 learned_addr learned my addr 172.21.15.105:0/2723257272 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:55.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.376+0000 7fa215d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa218071a10 0x7fa2180ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48622/0 (socket says 172.21.15.105:48622) 2023-11-28T18:31:55.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.376+0000 7fa216d9d640 1 --2- 172.21.15.105:0/2723257272 >> v2:172.21.15.148:3300/0 conn(0x7fa218070a10 0x7fa218070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:55.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.376+0000 7fa215d9b640 1 -- 172.21.15.105:0/2723257272 >> v2:172.21.15.148:3300/0 conn(0x7fa218070a10 msgr2=0x7fa218070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:55.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.376+0000 7fa215d9b640 1 --2- 172.21.15.105:0/2723257272 >> v2:172.21.15.148:3300/0 conn(0x7fa218070a10 0x7fa218070e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.376+0000 7fa215d9b640 1 -- 172.21.15.105:0/2723257272 >> v1:172.21.15.105:6789/0 conn(0x7fa21815d3c0 legacy=0x7fa218161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:55.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.376+0000 7fa215d9b640 1 -- 172.21.15.105:0/2723257272 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2180ce9e0 con 0x7fa218071a10 2023-11-28T18:31:55.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.380+0000 7fa215d9b640 1 --2- 172.21.15.105:0/2723257272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa218071a10 0x7fa2180ce190 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fa21806ba90 tx=0x7fa20c01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ddd1544c3f3181 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:55.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.380+0000 7fa21559a640 1 -- 172.21.15.105:0/2723257272 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c0ac600 con 0x7fa218071a10 2023-11-28T18:31:55.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.380+0000 7fa21559a640 1 -- 172.21.15.105:0/2723257272 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20c0a8050 con 0x7fa218071a10 2023-11-28T18:31:55.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.380+0000 7fa21559a640 1 -- 172.21.15.105:0/2723257272 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c0ac980 con 0x7fa218071a10 2023-11-28T18:31:55.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.380+0000 7fa21cd80640 1 -- 172.21.15.105:0/2723257272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa218071a10 msgr2=0x7fa2180ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:55.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.380+0000 7fa21cd80640 1 --2- 172.21.15.105:0/2723257272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa218071a10 0x7fa2180ce190 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fa21806ba90 tx=0x7fa20c01c3a0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.380+0000 7fa21cd80640 1 -- 172.21.15.105:0/2723257272 shutdown_connections 2023-11-28T18:31:55.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.380+0000 7fa21cd80640 1 --2- 172.21.15.105:0/2723257272 >> v2:172.21.15.148:3300/0 conn(0x7fa218070a10 0x7fa218070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.380+0000 7fa21cd80640 1 --2- 172.21.15.105:0/2723257272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa218071a10 0x7fa2180ce190 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.380+0000 7fa21cd80640 1 -- 172.21.15.105:0/2723257272 >> 172.21.15.105:0/2723257272 conn(0x7fa2180cf540 msgr2=0x7fa2180cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:55.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.380+0000 7fa21cd80640 1 -- 172.21.15.105:0/2723257272 shutdown_connections 2023-11-28T18:31:55.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.380+0000 7fa21cd80640 1 -- 172.21.15.105:0/2723257272 wait complete. 2023-11-28T18:31:55.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa21cd80640 1 Processor -- start 2023-11-28T18:31:55.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa21cd80640 1 -- start start 2023-11-28T18:31:55.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa21cd80640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa218070a10 0x7fa21814b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:55.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa21cd80640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa218071a10 0x7fa21814b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:55.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa21cd80640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa21815d3c0 0x7fa2180ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:55.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa21cd80640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa218159f80 con 0x7fa21815d3c0 2023-11-28T18:31:55.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa21cd80640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21815a1d0 con 0x7fa218071a10 2023-11-28T18:31:55.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa21cd80640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21815a4b0 con 0x7fa218070a10 2023-11-28T18:31:55.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa216d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa21815d3c0 0x7fa2180ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:55.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa216d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa21815d3c0 0x7fa2180ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48638/0 (socket says 172.21.15.105:48638) 2023-11-28T18:31:55.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa216d9d640 1 -- 172.21.15.105:0/2496903449 learned_addr learned my addr 172.21.15.105:0/2496903449 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:55.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa215d9b640 1 --2- 172.21.15.105:0/2496903449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa218071a10 0x7fa21814b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:55.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa21659c640 1 --2- 172.21.15.105:0/2496903449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa218070a10 0x7fa21814b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:55.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa216d9d640 1 -- 172.21.15.105:0/2496903449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa218070a10 msgr2=0x7fa21814b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:55.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa216d9d640 1 --2- 172.21.15.105:0/2496903449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa218070a10 0x7fa21814b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa216d9d640 1 -- 172.21.15.105:0/2496903449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa218071a10 msgr2=0x7fa21814b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:55.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa216d9d640 1 --2- 172.21.15.105:0/2496903449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa218071a10 0x7fa21814b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa216d9d640 1 -- 172.21.15.105:0/2496903449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c009420 con 0x7fa21815d3c0 2023-11-28T18:31:55.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa215d9b640 1 --2- 172.21.15.105:0/2496903449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa218071a10 0x7fa21814b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:55.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa216d9d640 1 --2- 172.21.15.105:0/2496903449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa21815d3c0 0x7fa2180ac8c0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fa210007e40 tx=0x7fa21000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:55.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa206ffd640 1 -- 172.21.15.105:0/2496903449 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210014620 con 0x7fa21815d3c0 2023-11-28T18:31:55.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa206ffd640 1 -- 172.21.15.105:0/2496903449 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21000ee90 con 0x7fa21815d3c0 2023-11-28T18:31:55.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa21cd80640 1 -- 172.21.15.105:0/2496903449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21815ab60 con 0x7fa21815d3c0 2023-11-28T18:31:55.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa206ffd640 1 -- 172.21.15.105:0/2496903449 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2100149e0 con 0x7fa21815d3c0 2023-11-28T18:31:55.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.384+0000 7fa21cd80640 1 -- 172.21.15.105:0/2496903449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21814c2a0 con 0x7fa21815d3c0 2023-11-28T18:31:55.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.388+0000 7fa21cd80640 1 -- 172.21.15.105:0/2496903449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa21815a9c0 con 0x7fa21815d3c0 2023-11-28T18:31:55.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.388+0000 7fa206ffd640 1 -- 172.21.15.105:0/2496903449 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa210014b40 con 0x7fa21815d3c0 2023-11-28T18:31:55.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.388+0000 7fa206ffd640 1 --2- 172.21.15.105:0/2496903449 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1f80668d0 0x7fa1f8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:55.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.388+0000 7fa206ffd640 1 -- 172.21.15.105:0/2496903449 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fa210084ce0 con 0x7fa21815d3c0 2023-11-28T18:31:55.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.388+0000 7fa21659c640 1 --2- 172.21.15.105:0/2496903449 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1f80668d0 0x7fa1f8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:55.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.388+0000 7fa21659c640 1 --2- 172.21.15.105:0/2496903449 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1f80668d0 0x7fa1f8068d90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa208007fd0 tx=0x7fa208009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:55.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.392+0000 7fa206ffd640 1 -- 172.21.15.105:0/2496903449 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa21815a9c0 con 0x7fa21815d3c0 2023-11-28T18:31:55.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.596+0000 7fa21cd80640 1 -- 172.21.15.105:0/2496903449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa21815a9c0 con 0x7fa21815d3c0 2023-11-28T18:31:55.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.596+0000 7fa206ffd640 1 -- 172.21.15.105:0/2496903449 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa21815a9c0 con 0x7fa21815d3c0 2023-11-28T18:31:55.599 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:55.599 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:31:55.600 INFO:teuthology.orchestra.run.smithi105.stdout: "auth_dump": [ 2023-11-28T18:31:55.600 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:31:55.601 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.a", 2023-11-28T18:31:55.601 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlOXSTGxAAU5kGNB3tznYsDNG51vCIZA==", 2023-11-28T18:31:55.602 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:31:55.603 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:31:55.603 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:31:55.603 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:31:55.604 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:31:55.604 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:55.604 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:31:55.604 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:31:55.605 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.b", 2023-11-28T18:31:55.605 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlrnbWJxAAKefl0YI2MhobnAOIj6WeaQ==", 2023-11-28T18:31:55.605 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:31:55.605 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:31:55.605 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:31:55.605 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:31:55.605 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:31:55.605 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:55.606 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:31:55.606 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:31:55.606 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.c", 2023-11-28T18:31:55.606 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2Zl5Y7GIhAA5OvgkNNFaI7LnzSsQVdFsQ==", 2023-11-28T18:31:55.606 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:31:55.606 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:31:55.607 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:31:55.607 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:31:55.607 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:31:55.607 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:55.607 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:31:55.607 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:31:55.608 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.d", 2023-11-28T18:31:55.608 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlmfYdLxAAreI7UE1YCjzkj6yl3s5Fwg==", 2023-11-28T18:31:55.608 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:31:55.608 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:31:55.609 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:31:55.609 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:31:55.609 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:31:55.609 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:55.610 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:31:55.610 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:31:55.610 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.0", 2023-11-28T18:31:55.610 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDtL2ZlphxQEBAAQ280OXh1zeaotIdVU8oZng==", 2023-11-28T18:31:55.610 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:31:55.610 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:31:55.610 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:31:55.611 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:31:55.611 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:55.611 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:31:55.611 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:31:55.611 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.1", 2023-11-28T18:31:55.611 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDwL2ZlezmOFxAARRBIOTkVJYfquIdcpW7sQQ==", 2023-11-28T18:31:55.611 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:31:55.611 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:31:55.611 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:31:55.612 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:31:55.612 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:55.612 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:31:55.612 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:31:55.612 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.2", 2023-11-28T18:31:55.612 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDzL2Zl4YcSHxAA2ww2F4TAD/0H/ZJ+aVMGEQ==", 2023-11-28T18:31:55.612 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:31:55.612 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:31:55.612 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:31:55.612 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:31:55.613 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:55.613 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:31:55.613 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:31:55.613 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.3", 2023-11-28T18:31:55.613 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD2L2ZlNxWxJRAAImp7TjTF18KwFTZ14DM6EA==", 2023-11-28T18:31:55.613 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:31:55.613 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:31:55.613 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:31:55.614 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:31:55.614 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:55.614 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:31:55.614 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:31:55.614 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.4", 2023-11-28T18:31:55.614 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD8L2ZlOAieLxAAz8g+anH8uJRfsttUS41GGQ==", 2023-11-28T18:31:55.614 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:31:55.614 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:31:55.614 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:31:55.614 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:31:55.615 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:55.615 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:31:55.615 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:31:55.615 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.5", 2023-11-28T18:31:55.615 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD/L2Zl5PKTNxAAPk+uIodLPjSd0EEMVT6iEw==", 2023-11-28T18:31:55.615 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:31:55.615 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:31:55.615 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:31:55.615 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:31:55.615 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:55.616 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:31:55.616 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:31:55.616 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.6", 2023-11-28T18:31:55.616 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQADMGZll+FUAxAAHlym5Z8rd/OF1911ZE2OUg==", 2023-11-28T18:31:55.616 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:31:55.616 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:31:55.616 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:31:55.616 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:31:55.616 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:55.617 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:31:55.617 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:31:55.617 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.7", 2023-11-28T18:31:55.617 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQAGMGZl8GO4ChAAVn3N6DnxUcs+ew09QvGo2w==", 2023-11-28T18:31:55.617 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:31:55.617 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2023-11-28T18:31:55.617 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2023-11-28T18:31:55.617 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:31:55.617 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:55.618 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:31:55.618 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:31:55.618 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.0", 2023-11-28T18:31:55.618 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlSt0VMxAAQ8+j1BgBPMJ2cuiZj8aG0w==", 2023-11-28T18:31:55.618 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:31:55.618 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:31:55.619 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow r", 2023-11-28T18:31:55.619 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-28T18:31:55.619 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:55.619 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:31:55.619 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:31:55.619 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.1", 2023-11-28T18:31:55.620 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDqL2ZlfZ1/ARAAhnJB6oj4Cc6p/oCjM6hTCw==", 2023-11-28T18:31:55.620 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:31:55.620 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:31:55.620 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:31:55.620 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:31:55.620 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:31:55.620 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:55.620 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:31:55.620 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:31:55.620 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.2", 2023-11-28T18:31:55.620 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlPLtyORAAU2GST6TJzlTfh6YqEb3U3w==", 2023-11-28T18:31:55.620 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:31:55.621 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:31:55.621 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:31:55.621 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:31:55.621 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:31:55.621 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:55.621 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:31:55.621 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:31:55.621 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.3", 2023-11-28T18:31:55.621 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDqL2ZlhiFuBxAAKQRxCdtqWpAHy1AVvdj9Sg==", 2023-11-28T18:31:55.621 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:31:55.621 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2023-11-28T18:31:55.621 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2023-11-28T18:31:55.621 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2023-11-28T18:31:55.622 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2023-11-28T18:31:55.622 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:55.622 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:31:55.622 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:31:55.622 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.admin", 2023-11-28T18:31:55.622 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDoL2ZlkuP2OhAAxsdZEF9122vkH53EwB8Qug==", 2023-11-28T18:31:55.622 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:31:55.622 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:31:55.623 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2023-11-28T18:31:55.623 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2023-11-28T18:31:55.623 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:31:55.623 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:55.623 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:31:55.623 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:31:55.623 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.x", 2023-11-28T18:31:55.623 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlEG49FxAAespsG+QK1WTpbmpV+DIwiw==", 2023-11-28T18:31:55.623 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:31:55.623 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:31:55.624 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2023-11-28T18:31:55.624 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:31:55.624 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:55.624 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:31:55.624 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:31:55.624 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.y", 2023-11-28T18:31:55.624 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDpL2ZlU89RExAAwMJp+8zZ6aFEt0KbiB6toA==", 2023-11-28T18:31:55.624 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2023-11-28T18:31:55.624 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2023-11-28T18:31:55.624 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2023-11-28T18:31:55.624 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2023-11-28T18:31:55.624 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:55.625 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:55.625 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2023-11-28T18:31:55.625 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:31:55.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.600+0000 7fa21cd80640 1 -- 172.21.15.105:0/2496903449 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1f80668d0 msgr2=0x7fa1f8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:55.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.600+0000 7fa21cd80640 1 --2- 172.21.15.105:0/2496903449 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1f80668d0 0x7fa1f8068d90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa208007fd0 tx=0x7fa208009040 comp rx=0 tx=0).stop 2023-11-28T18:31:55.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.600+0000 7fa21cd80640 1 -- 172.21.15.105:0/2496903449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa21815d3c0 msgr2=0x7fa2180ac8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:55.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.600+0000 7fa21cd80640 1 --2- 172.21.15.105:0/2496903449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa21815d3c0 0x7fa2180ac8c0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fa210007e40 tx=0x7fa21000e4b0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.600+0000 7fa21cd80640 1 -- 172.21.15.105:0/2496903449 shutdown_connections 2023-11-28T18:31:55.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.600+0000 7fa21cd80640 1 --2- 172.21.15.105:0/2496903449 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa1f80668d0 0x7fa1f8068d90 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.600+0000 7fa21cd80640 1 --2- 172.21.15.105:0/2496903449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa21815d3c0 0x7fa2180ac8c0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.600+0000 7fa21cd80640 1 --2- 172.21.15.105:0/2496903449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa218071a10 0x7fa21814b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.600+0000 7fa21cd80640 1 --2- 172.21.15.105:0/2496903449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa218070a10 0x7fa21814b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.600+0000 7fa21cd80640 1 -- 172.21.15.105:0/2496903449 >> 172.21.15.105:0/2496903449 conn(0x7fa2180cf540 msgr2=0x7fa218072eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:55.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.600+0000 7fa21cd80640 1 -- 172.21.15.105:0/2496903449 shutdown_connections 2023-11-28T18:31:55.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.600+0000 7fa21cd80640 1 -- 172.21.15.105:0/2496903449 wait complete. 2023-11-28T18:31:55.628 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:31:55.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.768+0000 7f8bf1afe640 1 Processor -- start 2023-11-28T18:31:55.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.768+0000 7f8bf1afe640 1 -- start start 2023-11-28T18:31:55.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.768+0000 7f8bf1afe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8bec0b4290 0x7f8bec0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:55.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.768+0000 7f8bf1afe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8bec071960 0x7f8bec15f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:55.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.768+0000 7f8bf1afe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8bec15b030 0x7f8bec0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:55.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.768+0000 7f8bf1afe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bec0b4bb0 con 0x7f8bec071960 2023-11-28T18:31:55.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.768+0000 7f8bf1afe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bec0701d0 con 0x7f8bec0b4290 2023-11-28T18:31:55.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.768+0000 7f8bf1afe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec06df80 con 0x7f8bec15b030 2023-11-28T18:31:55.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.768+0000 7f8beaffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8bec071960 0x7f8bec15f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:55.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.768+0000 7f8beaffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8bec071960 0x7f8bec15f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48646/0 (socket says 172.21.15.105:48646) 2023-11-28T18:31:55.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.768+0000 7f8beaffd640 1 -- 172.21.15.105:0/4028922912 learned_addr learned my addr 172.21.15.105:0/4028922912 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:55.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.768+0000 7f8beb7fe640 1 --2- 172.21.15.105:0/4028922912 >> v2:172.21.15.148:3300/0 conn(0x7f8bec0b4290 0x7f8bec0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:55.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.768+0000 7f8bebfff640 1 --2- 172.21.15.105:0/4028922912 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8bec15b030 0x7f8bec0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:55.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.768+0000 7f8beaffd640 1 -- 172.21.15.105:0/4028922912 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8bec15b030 msgr2=0x7f8bec0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:55.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.768+0000 7f8beaffd640 1 --2- 172.21.15.105:0/4028922912 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8bec15b030 0x7f8bec0709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.768+0000 7f8beaffd640 1 -- 172.21.15.105:0/4028922912 >> v2:172.21.15.148:3300/0 conn(0x7f8bec0b4290 msgr2=0x7f8bec0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:55.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.768+0000 7f8beaffd640 1 --2- 172.21.15.105:0/4028922912 >> v2:172.21.15.148:3300/0 conn(0x7f8bec0b4290 0x7f8bec0b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.768+0000 7f8beaffd640 1 -- 172.21.15.105:0/4028922912 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bec071df0 con 0x7f8bec071960 2023-11-28T18:31:55.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.772+0000 7f8beaffd640 1 --2- 172.21.15.105:0/4028922912 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bec071960 0x7f8bec15f3b0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f8bec073a70 tx=0x7f8be001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58ff95488380d89 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:55.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.772+0000 7f8bea7fc640 1 -- 172.21.15.105:0/4028922912 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be00ac600 con 0x7f8bec071960 2023-11-28T18:31:55.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.772+0000 7f8bea7fc640 1 -- 172.21.15.105:0/4028922912 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be00a8050 con 0x7f8bec071960 2023-11-28T18:31:55.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.772+0000 7f8bea7fc640 1 -- 172.21.15.105:0/4028922912 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be00ac980 con 0x7f8bec071960 2023-11-28T18:31:55.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.772+0000 7f8bf1afe640 1 -- 172.21.15.105:0/4028922912 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bec071960 msgr2=0x7f8bec15f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:55.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.772+0000 7f8bf1afe640 1 --2- 172.21.15.105:0/4028922912 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bec071960 0x7f8bec15f3b0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f8bec073a70 tx=0x7f8be001c4d0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.772+0000 7f8bf1afe640 1 -- 172.21.15.105:0/4028922912 shutdown_connections 2023-11-28T18:31:55.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.772+0000 7f8bf1afe640 1 --2- 172.21.15.105:0/4028922912 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8bec15b030 0x7f8bec0709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.772+0000 7f8bf1afe640 1 --2- 172.21.15.105:0/4028922912 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bec071960 0x7f8bec15f3b0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.772+0000 7f8bf1afe640 1 --2- 172.21.15.105:0/4028922912 >> v2:172.21.15.148:3300/0 conn(0x7f8bec0b4290 0x7f8bec0b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.772+0000 7f8bf1afe640 1 -- 172.21.15.105:0/4028922912 >> 172.21.15.105:0/4028922912 conn(0x7f8bec0ceb90 msgr2=0x7f8bec0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:55.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.772+0000 7f8bf1afe640 1 -- 172.21.15.105:0/4028922912 shutdown_connections 2023-11-28T18:31:55.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.772+0000 7f8bf1afe640 1 -- 172.21.15.105:0/4028922912 wait complete. 2023-11-28T18:31:55.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8bf1afe640 1 Processor -- start 2023-11-28T18:31:55.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8bf1afe640 1 -- start start 2023-11-28T18:31:55.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8bf1afe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bec0b4290 0x7f8bec1655a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:55.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8bf1afe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bec15b030 0x7f8bec14b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:55.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8bf1afe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8bec151e90 0x7f8bec152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:55.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8bf1afe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec152880 con 0x7f8bec15b030 2023-11-28T18:31:55.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8bf1afe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec0ce170 con 0x7f8bec0b4290 2023-11-28T18:31:55.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8bf1afe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec158980 con 0x7f8bec151e90 2023-11-28T18:31:55.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8beaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bec15b030 0x7f8bec14b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:55.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8beaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bec15b030 0x7f8bec14b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48648/0 (socket says 172.21.15.105:48648) 2023-11-28T18:31:55.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8beaffd640 1 -- 172.21.15.105:0/1474530286 learned_addr learned my addr 172.21.15.105:0/1474530286 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:55.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8beb7fe640 1 --2- 172.21.15.105:0/1474530286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bec0b4290 0x7f8bec1655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:55.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8bebfff640 1 --2- 172.21.15.105:0/1474530286 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8bec151e90 0x7f8bec152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:55.779 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8beaffd640 1 -- 172.21.15.105:0/1474530286 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8bec151e90 msgr2=0x7f8bec152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:55.779 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8beaffd640 1 --2- 172.21.15.105:0/1474530286 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8bec151e90 0x7f8bec152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.779 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8beaffd640 1 -- 172.21.15.105:0/1474530286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bec0b4290 msgr2=0x7f8bec1655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:55.779 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8beaffd640 1 --2- 172.21.15.105:0/1474530286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bec0b4290 0x7f8bec1655a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.779 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8beaffd640 1 -- 172.21.15.105:0/1474530286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be00094d0 con 0x7f8bec15b030 2023-11-28T18:31:55.779 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8beb7fe640 1 --2- 172.21.15.105:0/1474530286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bec0b4290 0x7f8bec1655a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:55.779 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8beaffd640 1 --2- 172.21.15.105:0/1474530286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bec15b030 0x7f8bec14b6d0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f8be001cd70 tx=0x7f8be001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:55.779 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8bd3fff640 1 -- 172.21.15.105:0/1474530286 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be00b7450 con 0x7f8bec15b030 2023-11-28T18:31:55.779 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8bd3fff640 1 -- 172.21.15.105:0/1474530286 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be00b75b0 con 0x7f8bec15b030 2023-11-28T18:31:55.779 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8bd3fff640 1 -- 172.21.15.105:0/1474530286 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be00b7920 con 0x7f8bec15b030 2023-11-28T18:31:55.779 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8bf1afe640 1 -- 172.21.15.105:0/1474530286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bec14bc10 con 0x7f8bec15b030 2023-11-28T18:31:55.780 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8bf1afe640 1 -- 172.21.15.105:0/1474530286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bec158630 con 0x7f8bec15b030 2023-11-28T18:31:55.781 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.776+0000 7f8bf1afe640 1 -- 172.21.15.105:0/1474530286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bec067570 con 0x7f8bec15b030 2023-11-28T18:31:55.781 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.780+0000 7f8bd3fff640 1 -- 172.21.15.105:0/1474530286 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8be00b3070 con 0x7f8bec15b030 2023-11-28T18:31:55.781 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.780+0000 7f8bd3fff640 1 --2- 172.21.15.105:0/1474530286 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8bd8066900 0x7f8bd8068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:55.781 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.780+0000 7f8bd3fff640 1 -- 172.21.15.105:0/1474530286 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f8be011e1a0 con 0x7f8bec15b030 2023-11-28T18:31:55.781 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.780+0000 7f8beb7fe640 1 --2- 172.21.15.105:0/1474530286 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8bd8066900 0x7f8bd8068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:55.781 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.780+0000 7f8beb7fe640 1 --2- 172.21.15.105:0/1474530286 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8bd8066900 0x7f8bd8068dc0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc009bb0 tx=0x7f8bdc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:55.784 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.784+0000 7f8bd3fff640 1 -- 172.21.15.105:0/1474530286 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8be00eb550 con 0x7f8bec15b030 2023-11-28T18:31:55.932 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.932+0000 7f8bf1afe640 1 -- 172.21.15.105:0/1474530286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bec158ac0 con 0x7f8bec15b030 2023-11-28T18:31:55.933 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.932+0000 7f8bd3fff640 1 -- 172.21.15.105:0/1474530286 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+13132 (secure 0 0 0) 0x7f8be00eeb70 con 0x7f8bec15b030 2023-11-28T18:31:55.934 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:55.934 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":125,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:31:55.206896+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:31:55.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.936+0000 7f8bf1afe640 1 -- 172.21.15.105:0/1474530286 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8bd8066900 msgr2=0x7f8bd8068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:55.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.936+0000 7f8bf1afe640 1 --2- 172.21.15.105:0/1474530286 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8bd8066900 0x7f8bd8068dc0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc009bb0 tx=0x7f8bdc008040 comp rx=0 tx=0).stop 2023-11-28T18:31:55.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.936+0000 7f8bf1afe640 1 -- 172.21.15.105:0/1474530286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bec15b030 msgr2=0x7f8bec14b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:55.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.936+0000 7f8bf1afe640 1 --2- 172.21.15.105:0/1474530286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bec15b030 0x7f8bec14b6d0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f8be001cd70 tx=0x7f8be001ad10 comp rx=0 tx=0).stop 2023-11-28T18:31:55.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.936+0000 7f8bf1afe640 1 -- 172.21.15.105:0/1474530286 shutdown_connections 2023-11-28T18:31:55.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.936+0000 7f8bf1afe640 1 --2- 172.21.15.105:0/1474530286 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8bec151e90 0x7f8bec152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.936+0000 7f8bf1afe640 1 --2- 172.21.15.105:0/1474530286 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8bd8066900 0x7f8bd8068dc0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.936+0000 7f8bf1afe640 1 --2- 172.21.15.105:0/1474530286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bec15b030 0x7f8bec14b6d0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.936+0000 7f8bf1afe640 1 --2- 172.21.15.105:0/1474530286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bec0b4290 0x7f8bec1655a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:55.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.936+0000 7f8bf1afe640 1 -- 172.21.15.105:0/1474530286 >> 172.21.15.105:0/1474530286 conn(0x7f8bec0ceb90 msgr2=0x7f8bec0af7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:55.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.936+0000 7f8bf1afe640 1 -- 172.21.15.105:0/1474530286 shutdown_connections 2023-11-28T18:31:55.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:55.936+0000 7f8bf1afe640 1 -- 172.21.15.105:0/1474530286 wait complete. 2023-11-28T18:31:55.955 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:31:55.955 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:31:56.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.096+0000 7f01114ea640 1 Processor -- start 2023-11-28T18:31:56.096 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.096+0000 7f01114ea640 1 -- start start 2023-11-28T18:31:56.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.096+0000 7f01114ea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f010c06cef0 0x7f010c0abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:56.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.096+0000 7f01114ea640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010c0cde30 con 0x7f010c159610 2023-11-28T18:31:56.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.096+0000 7f01114ea640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010c1599f0 con 0x7f010c0ce2c0 2023-11-28T18:31:56.097 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.096+0000 7f01114ea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010c06de20 con 0x7f010c06cef0 2023-11-28T18:31:56.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.096+0000 7f010affd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f010c159610 0x7f010c0cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:52676/0 (socket says 172.21.15.105:52676) 2023-11-28T18:31:56.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.096+0000 7f010affd640 1 -- 172.21.15.105:0/1840766359 learned_addr learned my addr 172.21.15.105:0/1840766359 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:56.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.096+0000 7f010b7fe640 1 --2- 172.21.15.105:0/1840766359 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f010c06cef0 0x7f010c0abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:56.098 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.096+0000 7f010a7fc640 1 -- 172.21.15.105:0/1840766359 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355402354 0 0) 0x7f010c0cde30 con 0x7f010c159610 2023-11-28T18:31:56.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.096+0000 7f010a7fc640 1 -- 172.21.15.105:0/1840766359 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00f8003530 con 0x7f010c159610 2023-11-28T18:31:56.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.096+0000 7f010a7fc640 1 -- 172.21.15.105:0/1840766359 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1485699023 0 0) 0x7f010c1599f0 con 0x7f010c0ce2c0 2023-11-28T18:31:56.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.096+0000 7f010a7fc640 1 -- 172.21.15.105:0/1840766359 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f010c0cde30 con 0x7f010c0ce2c0 2023-11-28T18:31:56.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.096+0000 7f010b7fe640 1 -- 172.21.15.105:0/1840766359 >> v1:172.21.15.148:6789/0 conn(0x7f010c0ce2c0 legacy=0x7f010c06c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:56.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.096+0000 7f010b7fe640 1 -- 172.21.15.105:0/1840766359 >> v1:172.21.15.105:6789/0 conn(0x7f010c159610 legacy=0x7f010c0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:56.099 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.096+0000 7f010b7fe640 1 -- 172.21.15.105:0/1840766359 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010c0cd220 con 0x7f010c06cef0 2023-11-28T18:31:56.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.100+0000 7f010b7fe640 1 --2- 172.21.15.105:0/1840766359 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f010c06cef0 0x7f010c0abd80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f010400aed0 tx=0x7f010401c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7668962dbf3d6df server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:56.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.100+0000 7f010a7fc640 1 -- 172.21.15.105:0/1840766359 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010400a2d0 con 0x7f010c06cef0 2023-11-28T18:31:56.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.100+0000 7f010a7fc640 1 -- 172.21.15.105:0/1840766359 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01040a5690 con 0x7f010c06cef0 2023-11-28T18:31:56.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.100+0000 7f010a7fc640 1 -- 172.21.15.105:0/1840766359 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01040a5a20 con 0x7f010c06cef0 2023-11-28T18:31:56.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.100+0000 7f01114ea640 1 -- 172.21.15.105:0/1840766359 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f010c06cef0 msgr2=0x7f010c0abd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:56.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.100+0000 7f01114ea640 1 --2- 172.21.15.105:0/1840766359 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f010c06cef0 0x7f010c0abd80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f010400aed0 tx=0x7f010401c960 comp rx=0 tx=0).stop 2023-11-28T18:31:56.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.100+0000 7f01114ea640 1 -- 172.21.15.105:0/1840766359 shutdown_connections 2023-11-28T18:31:56.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.100+0000 7f01114ea640 1 --2- 172.21.15.105:0/1840766359 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f010c06cef0 0x7f010c0abd80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:56.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.100+0000 7f01114ea640 1 -- 172.21.15.105:0/1840766359 >> 172.21.15.105:0/1840766359 conn(0x7f010c0ceb90 msgr2=0x7f010c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:56.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.100+0000 7f01114ea640 1 -- 172.21.15.105:0/1840766359 shutdown_connections 2023-11-28T18:31:56.103 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.100+0000 7f01114ea640 1 -- 172.21.15.105:0/1840766359 wait complete. 2023-11-28T18:31:56.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f01114ea640 1 Processor -- start 2023-11-28T18:31:56.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f01114ea640 1 -- start start 2023-11-28T18:31:56.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f01114ea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f010c06cef0 0x7f010c148da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:56.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f01114ea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f010c0ce2c0 0x7f010c15bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:56.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f01114ea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f010c159610 0x7f010c14a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:56.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f01114ea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c0b2ef0 con 0x7f010c159610 2023-11-28T18:31:56.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f01114ea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c0b22e0 con 0x7f010c0ce2c0 2023-11-28T18:31:56.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f01114ea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010c0b25c0 con 0x7f010c06cef0 2023-11-28T18:31:56.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f010b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f010c159610 0x7f010c14a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:56.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f010b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f010c159610 0x7f010c14a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48660/0 (socket says 172.21.15.105:48660) 2023-11-28T18:31:56.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f010b7fe640 1 -- 172.21.15.105:0/445357204 learned_addr learned my addr 172.21.15.105:0/445357204 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:56.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f0103fff640 1 --2- 172.21.15.105:0/445357204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f010c0ce2c0 0x7f010c15bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:56.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f010affd640 1 --2- 172.21.15.105:0/445357204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f010c06cef0 0x7f010c148da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:56.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f010b7fe640 1 -- 172.21.15.105:0/445357204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f010c06cef0 msgr2=0x7f010c148da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:56.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f010b7fe640 1 --2- 172.21.15.105:0/445357204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f010c06cef0 0x7f010c148da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:56.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f010b7fe640 1 -- 172.21.15.105:0/445357204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f010c0ce2c0 msgr2=0x7f010c15bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:56.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f010b7fe640 1 --2- 172.21.15.105:0/445357204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f010c0ce2c0 0x7f010c15bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:56.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f010b7fe640 1 -- 172.21.15.105:0/445357204 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010400ab80 con 0x7f010c159610 2023-11-28T18:31:56.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f010affd640 1 --2- 172.21.15.105:0/445357204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f010c06cef0 0x7f010c148da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:56.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f010b7fe640 1 --2- 172.21.15.105:0/445357204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f010c159610 0x7f010c14a7c0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f010401a590 tx=0x7f0104019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:56.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f01037fe640 1 -- 172.21.15.105:0/445357204 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01040b8670 con 0x7f010c159610 2023-11-28T18:31:56.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f01037fe640 1 -- 172.21.15.105:0/445357204 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01040b87d0 con 0x7f010c159610 2023-11-28T18:31:56.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f01037fe640 1 -- 172.21.15.105:0/445357204 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01040b8b80 con 0x7f010c159610 2023-11-28T18:31:56.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.104+0000 7f01114ea640 1 -- 172.21.15.105:0/445357204 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010c0ccff0 con 0x7f010c159610 2023-11-28T18:31:56.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.108+0000 7f01114ea640 1 -- 172.21.15.105:0/445357204 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010c0b2b80 con 0x7f010c159610 2023-11-28T18:31:56.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.108+0000 7f01114ea640 1 -- 172.21.15.105:0/445357204 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f010c0b30a0 con 0x7f010c159610 2023-11-28T18:31:56.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.108+0000 7f01037fe640 1 -- 172.21.15.105:0/445357204 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f01040b8ce0 con 0x7f010c159610 2023-11-28T18:31:56.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.108+0000 7f01037fe640 1 --2- 172.21.15.105:0/445357204 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00ec0668d0 0x7f00ec068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:56.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.108+0000 7f01037fe640 1 -- 172.21.15.105:0/445357204 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f010411f550 con 0x7f010c159610 2023-11-28T18:31:56.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.108+0000 7f010affd640 1 --2- 172.21.15.105:0/445357204 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00ec0668d0 0x7f00ec068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:56.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.108+0000 7f010affd640 1 --2- 172.21.15.105:0/445357204 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00ec0668d0 0x7f00ec068d90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f00fc000f70 tx=0x7f00fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:56.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.112+0000 7f01037fe640 1 -- 172.21.15.105:0/445357204 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f010c0b30a0 con 0x7f010c159610 2023-11-28T18:31:56.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.264+0000 7f01114ea640 1 -- 172.21.15.105:0/445357204 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f010c067570 con 0x7f010c159610 2023-11-28T18:31:56.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.264+0000 7f01037fe640 1 -- 172.21.15.105:0/445357204 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7f01040ec900 con 0x7f010c159610 2023-11-28T18:31:56.266 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:31:56.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.268+0000 7f01114ea640 1 -- 172.21.15.105:0/445357204 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00ec0668d0 msgr2=0x7f00ec068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:56.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.268+0000 7f01114ea640 1 --2- 172.21.15.105:0/445357204 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00ec0668d0 0x7f00ec068d90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f00fc000f70 tx=0x7f00fc009040 comp rx=0 tx=0).stop 2023-11-28T18:31:56.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.268+0000 7f01114ea640 1 -- 172.21.15.105:0/445357204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f010c159610 msgr2=0x7f010c14a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:56.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.268+0000 7f01114ea640 1 --2- 172.21.15.105:0/445357204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f010c159610 0x7f010c14a7c0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f010401a590 tx=0x7f0104019700 comp rx=0 tx=0).stop 2023-11-28T18:31:56.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.268+0000 7f01114ea640 1 -- 172.21.15.105:0/445357204 shutdown_connections 2023-11-28T18:31:56.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.268+0000 7f01114ea640 1 --2- 172.21.15.105:0/445357204 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f00ec0668d0 0x7f00ec068d90 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:56.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.268+0000 7f01114ea640 1 --2- 172.21.15.105:0/445357204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f010c159610 0x7f010c14a7c0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:56.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.268+0000 7f01114ea640 1 --2- 172.21.15.105:0/445357204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f010c0ce2c0 0x7f010c15bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:56.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.268+0000 7f01114ea640 1 --2- 172.21.15.105:0/445357204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f010c06cef0 0x7f010c148da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:56.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.268+0000 7f01114ea640 1 -- 172.21.15.105:0/445357204 >> 172.21.15.105:0/445357204 conn(0x7f010c0ceb90 msgr2=0x7f010c0cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:56.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.268+0000 7f01114ea640 1 -- 172.21.15.105:0/445357204 shutdown_connections 2023-11-28T18:31:56.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.268+0000 7f01114ea640 1 -- 172.21.15.105:0/445357204 wait complete. 2023-11-28T18:31:56.287 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T18:31:56.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.424+0000 7f46529e6640 1 Processor -- start 2023-11-28T18:31:56.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.424+0000 7f46529e6640 1 -- start start 2023-11-28T18:31:56.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.428+0000 7f46529e6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f464c071960 0x7f464c15f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:56.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.428+0000 7f46529e6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f464c15b030 0x7f464c0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:56.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.428+0000 7f46529e6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f464c0701d0 con 0x7f464c0b4290 2023-11-28T18:31:56.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.428+0000 7f46529e6640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f464c06df80 con 0x7f464c071960 2023-11-28T18:31:56.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.428+0000 7f46529e6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f464c071df0 con 0x7f464c15b030 2023-11-28T18:31:56.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.428+0000 7f464b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f464c071960 0x7f464c15f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:56.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.428+0000 7f464bfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f464c0b4290 0x7f464c0b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:52692/0 (socket says 172.21.15.105:52692) 2023-11-28T18:31:56.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.428+0000 7f464bfff640 1 -- 172.21.15.105:0/1475440452 learned_addr learned my addr 172.21.15.105:0/1475440452 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:56.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.428+0000 7f4650f83640 1 --2- 172.21.15.105:0/1475440452 >> v2:172.21.15.148:3300/0 conn(0x7f464c15b030 0x7f464c0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:56.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.428+0000 7f464b7fe640 1 -- 172.21.15.105:0/1475440452 >> v2:172.21.15.148:3300/0 conn(0x7f464c15b030 msgr2=0x7f464c0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:56.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.428+0000 7f464b7fe640 1 --2- 172.21.15.105:0/1475440452 >> v2:172.21.15.148:3300/0 conn(0x7f464c15b030 0x7f464c0709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:56.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.428+0000 7f464b7fe640 1 -- 172.21.15.105:0/1475440452 >> v1:172.21.15.105:6789/0 conn(0x7f464c0b4290 legacy=0x7f464c0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:56.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.428+0000 7f464b7fe640 1 -- 172.21.15.105:0/1475440452 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f464c0720d0 con 0x7f464c071960 2023-11-28T18:31:56.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.432+0000 7f464b7fe640 1 --2- 172.21.15.105:0/1475440452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f464c071960 0x7f464c15f3b0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f464c067c50 tx=0x7f464001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab282ba987f71840 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:56.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.432+0000 7f464affd640 1 -- 172.21.15.105:0/1475440452 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46400ab600 con 0x7f464c071960 2023-11-28T18:31:56.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.432+0000 7f464affd640 1 -- 172.21.15.105:0/1475440452 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46400a7050 con 0x7f464c071960 2023-11-28T18:31:56.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.432+0000 7f464affd640 1 -- 172.21.15.105:0/1475440452 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46400ab980 con 0x7f464c071960 2023-11-28T18:31:56.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.432+0000 7f46529e6640 1 -- 172.21.15.105:0/1475440452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f464c071960 msgr2=0x7f464c15f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:56.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.432+0000 7f46529e6640 1 --2- 172.21.15.105:0/1475440452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f464c071960 0x7f464c15f3b0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f464c067c50 tx=0x7f464001c430 comp rx=0 tx=0).stop 2023-11-28T18:31:56.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.432+0000 7f46529e6640 1 -- 172.21.15.105:0/1475440452 shutdown_connections 2023-11-28T18:31:56.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.432+0000 7f46529e6640 1 --2- 172.21.15.105:0/1475440452 >> v2:172.21.15.148:3300/0 conn(0x7f464c15b030 0x7f464c0709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:56.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.432+0000 7f46529e6640 1 --2- 172.21.15.105:0/1475440452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f464c071960 0x7f464c15f3b0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:56.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.432+0000 7f46529e6640 1 -- 172.21.15.105:0/1475440452 >> 172.21.15.105:0/1475440452 conn(0x7f464c0ceb90 msgr2=0x7f464c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:56.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.432+0000 7f46529e6640 1 -- 172.21.15.105:0/1475440452 shutdown_connections 2023-11-28T18:31:56.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.432+0000 7f46529e6640 1 -- 172.21.15.105:0/1475440452 wait complete. 2023-11-28T18:31:56.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.432+0000 7f46529e6640 1 Processor -- start 2023-11-28T18:31:56.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.432+0000 7f46529e6640 1 -- start start 2023-11-28T18:31:56.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f46529e6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f464c0b4290 0x7f464c158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:56.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f46529e6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f464c15b030 0x7f464c153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:56.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f46529e6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f464c1539c0 0x7f464c154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:56.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f46529e6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c15a530 con 0x7f464c0b4290 2023-11-28T18:31:56.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f46529e6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c09ad30 con 0x7f464c15b030 2023-11-28T18:31:56.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f46529e6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f464c09b010 con 0x7f464c1539c0 2023-11-28T18:31:56.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f464bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f464c0b4290 0x7f464c158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:56.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f464bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f464c0b4290 0x7f464c158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48682/0 (socket says 172.21.15.105:48682) 2023-11-28T18:31:56.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f464bfff640 1 -- 172.21.15.105:0/1196131718 learned_addr learned my addr 172.21.15.105:0/1196131718 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:56.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f4650f83640 1 --2- 172.21.15.105:0/1196131718 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f464c1539c0 0x7f464c154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:56.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f464b7fe640 1 --2- 172.21.15.105:0/1196131718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f464c15b030 0x7f464c153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:56.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f464bfff640 1 -- 172.21.15.105:0/1196131718 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f464c1539c0 msgr2=0x7f464c154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:56.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f464bfff640 1 --2- 172.21.15.105:0/1196131718 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f464c1539c0 0x7f464c154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:56.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f464bfff640 1 -- 172.21.15.105:0/1196131718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f464c15b030 msgr2=0x7f464c153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:56.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f464bfff640 1 --2- 172.21.15.105:0/1196131718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f464c15b030 0x7f464c153f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:56.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f464bfff640 1 -- 172.21.15.105:0/1196131718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4640009420 con 0x7f464c0b4290 2023-11-28T18:31:56.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f464bfff640 1 --2- 172.21.15.105:0/1196131718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f464c0b4290 0x7f464c158630 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f463c00c500 tx=0x7f463c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:56.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f4648ff9640 1 -- 172.21.15.105:0/1196131718 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c005cf0 con 0x7f464c0b4290 2023-11-28T18:31:56.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f4648ff9640 1 -- 172.21.15.105:0/1196131718 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463c005e50 con 0x7f464c0b4290 2023-11-28T18:31:56.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f46529e6640 1 -- 172.21.15.105:0/1196131718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464c09b2f0 con 0x7f464c0b4290 2023-11-28T18:31:56.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f4648ff9640 1 -- 172.21.15.105:0/1196131718 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c0072f0 con 0x7f464c0b4290 2023-11-28T18:31:56.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f46529e6640 1 -- 172.21.15.105:0/1196131718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f464c0b2bc0 con 0x7f464c0b4290 2023-11-28T18:31:56.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.436+0000 7f46529e6640 1 -- 172.21.15.105:0/1196131718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f464c09ab90 con 0x7f464c0b4290 2023-11-28T18:31:56.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.440+0000 7f4648ff9640 1 -- 172.21.15.105:0/1196131718 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f463c007450 con 0x7f464c0b4290 2023-11-28T18:31:56.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.440+0000 7f4648ff9640 1 --2- 172.21.15.105:0/1196131718 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46380668d0 0x7f4638068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:56.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.440+0000 7f4648ff9640 1 -- 172.21.15.105:0/1196131718 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f463c0839c0 con 0x7f464c0b4290 2023-11-28T18:31:56.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.440+0000 7f464b7fe640 1 --2- 172.21.15.105:0/1196131718 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46380668d0 0x7f4638068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:56.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.440+0000 7f464b7fe640 1 --2- 172.21.15.105:0/1196131718 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46380668d0 0x7f4638068d90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f464c073a70 tx=0x7f4640008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:56.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.440+0000 7f4648ff9640 1 -- 172.21.15.105:0/1196131718 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f463c050f30 con 0x7f464c0b4290 2023-11-28T18:31:56.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.592+0000 7f46529e6640 1 -- 172.21.15.105:0/1196131718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f464c09ab90 con 0x7f464c0b4290 2023-11-28T18:31:56.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.592+0000 7f4648ff9640 1 -- 172.21.15.105:0/1196131718 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v125) v1 ==== 84+0+65 (secure 0 0 0) 0x7f463c021060 con 0x7f464c0b4290 2023-11-28T18:31:56.594 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:56.594 INFO:teuthology.orchestra.run.smithi105.stdout:[ 2023-11-28T18:31:56.594 INFO:teuthology.orchestra.run.smithi105.stdout: { 2023-11-28T18:31:56.594 INFO:teuthology.orchestra.run.smithi105.stdout: "poolnum": 1, 2023-11-28T18:31:56.594 INFO:teuthology.orchestra.run.smithi105.stdout: "poolname": ".mgr" 2023-11-28T18:31:56.594 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:31:56.595 INFO:teuthology.orchestra.run.smithi105.stdout:] 2023-11-28T18:31:56.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.596+0000 7f46529e6640 1 -- 172.21.15.105:0/1196131718 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46380668d0 msgr2=0x7f4638068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:56.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.596+0000 7f46529e6640 1 --2- 172.21.15.105:0/1196131718 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46380668d0 0x7f4638068d90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f464c073a70 tx=0x7f4640008040 comp rx=0 tx=0).stop 2023-11-28T18:31:56.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.596+0000 7f46529e6640 1 -- 172.21.15.105:0/1196131718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f464c0b4290 msgr2=0x7f464c158630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:56.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.596+0000 7f46529e6640 1 --2- 172.21.15.105:0/1196131718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f464c0b4290 0x7f464c158630 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f463c00c500 tx=0x7f463c00c9c0 comp rx=0 tx=0).stop 2023-11-28T18:31:56.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.596+0000 7f46529e6640 1 -- 172.21.15.105:0/1196131718 shutdown_connections 2023-11-28T18:31:56.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.596+0000 7f46529e6640 1 --2- 172.21.15.105:0/1196131718 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f464c1539c0 0x7f464c154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:56.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.596+0000 7f46529e6640 1 --2- 172.21.15.105:0/1196131718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f464c15b030 0x7f464c153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:56.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.596+0000 7f46529e6640 1 --2- 172.21.15.105:0/1196131718 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f46380668d0 0x7f4638068d90 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:56.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.596+0000 7f46529e6640 1 --2- 172.21.15.105:0/1196131718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f464c0b4290 0x7f464c158630 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:56.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.596+0000 7f46529e6640 1 -- 172.21.15.105:0/1196131718 >> 172.21.15.105:0/1196131718 conn(0x7f464c0ceb90 msgr2=0x7f464c15f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:56.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.596+0000 7f46529e6640 1 -- 172.21.15.105:0/1196131718 shutdown_connections 2023-11-28T18:31:56.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.596+0000 7f46529e6640 1 -- 172.21.15.105:0/1196131718 wait complete. 2023-11-28T18:31:56.614 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T18:31:56.615 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T18:31:56.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.756+0000 7feec550f640 1 Processor -- start 2023-11-28T18:31:56.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.756+0000 7feec550f640 1 -- start start 2023-11-28T18:31:56.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.756+0000 7feec550f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feec015b030 0x7feec00709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:56.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.756+0000 7feec550f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec00701d0 con 0x7feec0071960 2023-11-28T18:31:56.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.756+0000 7feec550f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec006df80 con 0x7feec00b4290 2023-11-28T18:31:56.757 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.756+0000 7feec550f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec0071df0 con 0x7feec015b030 2023-11-28T18:31:56.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.756+0000 7feebe7fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7feec0071960 0x7feec015f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:52706/0 (socket says 172.21.15.105:52706) 2023-11-28T18:31:56.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.756+0000 7feebe7fc640 1 -- 172.21.15.105:0/4197288430 learned_addr learned my addr 172.21.15.105:0/4197288430 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:56.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.756+0000 7feebf7fe640 1 --2- 172.21.15.105:0/4197288430 >> v2:172.21.15.148:3300/0 conn(0x7feec015b030 0x7feec00709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:56.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.756+0000 7feebdffb640 1 -- 172.21.15.105:0/4197288430 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2242848038 0 0) 0x7feec00701d0 con 0x7feec0071960 2023-11-28T18:31:56.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.756+0000 7feebdffb640 1 -- 172.21.15.105:0/4197288430 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feea8003530 con 0x7feec0071960 2023-11-28T18:31:56.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.756+0000 7feebdffb640 1 -- 172.21.15.105:0/4197288430 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3036032244 0 0) 0x7feec006df80 con 0x7feec00b4290 2023-11-28T18:31:56.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.756+0000 7feebdffb640 1 -- 172.21.15.105:0/4197288430 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feec00701d0 con 0x7feec00b4290 2023-11-28T18:31:56.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.756+0000 7feebf7fe640 1 -- 172.21.15.105:0/4197288430 >> v1:172.21.15.148:6789/0 conn(0x7feec00b4290 legacy=0x7feec00b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:56.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.756+0000 7feebf7fe640 1 -- 172.21.15.105:0/4197288430 >> v1:172.21.15.105:6789/0 conn(0x7feec0071960 legacy=0x7feec015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.756+0000 7feebdffb640 1 -- 172.21.15.105:0/4197288430 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 935735107 0 0) 0x7feea8003530 con 0x7feec0071960 2023-11-28T18:31:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.756+0000 7feebf7fe640 1 -- 172.21.15.105:0/4197288430 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec00720d0 con 0x7feec015b030 2023-11-28T18:31:56.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.760+0000 7feebf7fe640 1 --2- 172.21.15.105:0/4197288430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feec015b030 0x7feec00709b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009d90 tx=0x7feeb801c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3848ebc015b82288 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:56.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.760+0000 7feebdffb640 1 -- 172.21.15.105:0/4197288430 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb4001680 con 0x7feec015b030 2023-11-28T18:31:56.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.760+0000 7feebdffb640 1 -- 172.21.15.105:0/4197288430 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb80a5590 con 0x7feec015b030 2023-11-28T18:31:56.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.760+0000 7feebdffb640 1 -- 172.21.15.105:0/4197288430 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb80a5920 con 0x7feec015b030 2023-11-28T18:31:56.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.760+0000 7feec550f640 1 -- 172.21.15.105:0/4197288430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feec015b030 msgr2=0x7feec00709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:56.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.760+0000 7feec550f640 1 --2- 172.21.15.105:0/4197288430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feec015b030 0x7feec00709b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009d90 tx=0x7feeb801c840 comp rx=0 tx=0).stop 2023-11-28T18:31:56.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.760+0000 7feec550f640 1 -- 172.21.15.105:0/4197288430 shutdown_connections 2023-11-28T18:31:56.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.760+0000 7feec550f640 1 --2- 172.21.15.105:0/4197288430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feec015b030 0x7feec00709b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:56.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.760+0000 7feec550f640 1 -- 172.21.15.105:0/4197288430 >> 172.21.15.105:0/4197288430 conn(0x7feec00ceb90 msgr2=0x7feec00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:56.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.760+0000 7feec550f640 1 -- 172.21.15.105:0/4197288430 shutdown_connections 2023-11-28T18:31:56.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.760+0000 7feec550f640 1 -- 172.21.15.105:0/4197288430 wait complete. 2023-11-28T18:31:56.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.764+0000 7feec550f640 1 Processor -- start 2023-11-28T18:31:56.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.764+0000 7feec550f640 1 -- start start 2023-11-28T18:31:56.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.764+0000 7feec550f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feec00b4290 0x7feec014d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:56.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.764+0000 7feec550f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feec015b030 0x7feec0099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:56.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.764+0000 7feec550f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feec0099f90 0x7feec0153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:56.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.764+0000 7feec550f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec009a440 con 0x7feec0099f90 2023-11-28T18:31:56.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.764+0000 7feec550f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec015c180 con 0x7feec00b4290 2023-11-28T18:31:56.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.764+0000 7feec550f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec015b570 con 0x7feec015b030 2023-11-28T18:31:56.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.764+0000 7feebf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feec0099f90 0x7feec0153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:56.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.764+0000 7feebf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feec0099f90 0x7feec0153a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48696/0 (socket says 172.21.15.105:48696) 2023-11-28T18:31:56.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.764+0000 7feebf7fe640 1 -- 172.21.15.105:0/2494421090 learned_addr learned my addr 172.21.15.105:0/2494421090 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:56.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.764+0000 7feebeffd640 1 --2- 172.21.15.105:0/2494421090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feec00b4290 0x7feec014d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:56.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.764+0000 7feebe7fc640 1 --2- 172.21.15.105:0/2494421090 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feec015b030 0x7feec0099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:56.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.764+0000 7feebf7fe640 1 -- 172.21.15.105:0/2494421090 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feec015b030 msgr2=0x7feec0099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:56.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.764+0000 7feebf7fe640 1 --2- 172.21.15.105:0/2494421090 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feec015b030 0x7feec0099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:56.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.764+0000 7feebf7fe640 1 -- 172.21.15.105:0/2494421090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feec00b4290 msgr2=0x7feec014d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:56.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.764+0000 7feebf7fe640 1 --2- 172.21.15.105:0/2494421090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feec00b4290 0x7feec014d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:56.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.764+0000 7feebf7fe640 1 -- 172.21.15.105:0/2494421090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb8009a70 con 0x7feec0099f90 2023-11-28T18:31:56.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.764+0000 7feebf7fe640 1 --2- 172.21.15.105:0/2494421090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feec0099f90 0x7feec0153a40 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7feeb80b0040 tx=0x7feeb80195e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:56.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.764+0000 7feea77fe640 1 -- 172.21.15.105:0/2494421090 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8003ba0 con 0x7feec0099f90 2023-11-28T18:31:56.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.768+0000 7feea77fe640 1 -- 172.21.15.105:0/2494421090 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb8003d00 con 0x7feec0099f90 2023-11-28T18:31:56.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.768+0000 7feec550f640 1 -- 172.21.15.105:0/2494421090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec015b850 con 0x7feec0099f90 2023-11-28T18:31:56.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.768+0000 7feea77fe640 1 -- 172.21.15.105:0/2494421090 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8004270 con 0x7feec0099f90 2023-11-28T18:31:56.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.768+0000 7feec550f640 1 -- 172.21.15.105:0/2494421090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec015be10 con 0x7feec0099f90 2023-11-28T18:31:56.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.768+0000 7feec550f640 1 -- 172.21.15.105:0/2494421090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec015bfd0 con 0x7feec0099f90 2023-11-28T18:31:56.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.768+0000 7feea77fe640 1 -- 172.21.15.105:0/2494421090 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7feeb80043d0 con 0x7feec0099f90 2023-11-28T18:31:56.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.768+0000 7feea77fe640 1 --2- 172.21.15.105:0/2494421090 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feeac0668d0 0x7feeac068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:56.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.768+0000 7feea77fe640 1 -- 172.21.15.105:0/2494421090 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5177+0+0 (secure 0 0 0) 0x7feeb811f220 con 0x7feec0099f90 2023-11-28T18:31:56.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.768+0000 7feebeffd640 1 --2- 172.21.15.105:0/2494421090 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feeac0668d0 0x7feeac068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:56.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.768+0000 7feebeffd640 1 --2- 172.21.15.105:0/2494421090 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feeac0668d0 0x7feeac068d90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7feeb00011f0 tx=0x7feeb0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:56.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.772+0000 7feea77fe640 1 -- 172.21.15.105:0/2494421090 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7feeb80ec860 con 0x7feec0099f90 2023-11-28T18:31:56.924 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:56.924+0000 7feec550f640 1 -- 172.21.15.105:0/2494421090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7feec015bfd0 con 0x7feec0099f90 2023-11-28T18:31:57.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.432+0000 7feea77fe640 1 -- 172.21.15.105:0/2494421090 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v126) v1 ==== 138+0+0 (secure 0 0 0) 0x7feeb8009450 con 0x7feec0099f90 2023-11-28T18:31:57.514 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' created 2023-11-28T18:31:57.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.436+0000 7feec550f640 1 -- 172.21.15.105:0/2494421090 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feeac0668d0 msgr2=0x7feeac068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:57.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.436+0000 7feec550f640 1 --2- 172.21.15.105:0/2494421090 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feeac0668d0 0x7feeac068d90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7feeb00011f0 tx=0x7feeb0009040 comp rx=0 tx=0).stop 2023-11-28T18:31:57.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.436+0000 7feec550f640 1 -- 172.21.15.105:0/2494421090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feec0099f90 msgr2=0x7feec0153a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:57.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.436+0000 7feec550f640 1 --2- 172.21.15.105:0/2494421090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feec0099f90 0x7feec0153a40 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7feeb80b0040 tx=0x7feeb80195e0 comp rx=0 tx=0).stop 2023-11-28T18:31:57.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.436+0000 7feec550f640 1 -- 172.21.15.105:0/2494421090 shutdown_connections 2023-11-28T18:31:57.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.436+0000 7feec550f640 1 --2- 172.21.15.105:0/2494421090 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7feeac0668d0 0x7feeac068d90 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:57.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.436+0000 7feec550f640 1 --2- 172.21.15.105:0/2494421090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feec0099f90 0x7feec0153a40 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:57.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.436+0000 7feec550f640 1 --2- 172.21.15.105:0/2494421090 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feec015b030 0x7feec0099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:57.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.436+0000 7feec550f640 1 --2- 172.21.15.105:0/2494421090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feec00b4290 0x7feec014d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:57.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.436+0000 7feec550f640 1 -- 172.21.15.105:0/2494421090 >> 172.21.15.105:0/2494421090 conn(0x7feec00ceb90 msgr2=0x7feec00aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:57.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.436+0000 7feec550f640 1 -- 172.21.15.105:0/2494421090 shutdown_connections 2023-11-28T18:31:57.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.436+0000 7feec550f640 1 -- 172.21.15.105:0/2494421090 wait complete. 2023-11-28T18:31:57.517 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T18:31:57.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.664+0000 7fa98469a640 1 Processor -- start 2023-11-28T18:31:57.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.668+0000 7fa98469a640 1 -- start start 2023-11-28T18:31:57.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.668+0000 7fa98469a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa97c0caf60 0x7fa97c0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:57.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.668+0000 7fa98469a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa97c0ad850 0x7fa97c0701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:57.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.668+0000 7fa98469a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa97c06bf50 con 0x7fa97c159f40 2023-11-28T18:31:57.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.668+0000 7fa98469a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa97c0cb3c0 con 0x7fa97c0caf60 2023-11-28T18:31:57.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.668+0000 7fa98469a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c15a320 con 0x7fa97c0ad850 2023-11-28T18:31:57.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.668+0000 7fa981c35640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa97c0caf60 0x7fa97c0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:57.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.668+0000 7fa981c35640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa97c0caf60 0x7fa97c0cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48712/0 (socket says 172.21.15.105:48712) 2023-11-28T18:31:57.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.668+0000 7fa981c35640 1 -- 172.21.15.105:0/2192264168 learned_addr learned my addr 172.21.15.105:0/2192264168 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:57.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.668+0000 7fa982c37640 1 --2- 172.21.15.105:0/2192264168 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa97c0ad850 0x7fa97c0701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:57.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.668+0000 7fa981c35640 1 -- 172.21.15.105:0/2192264168 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa97c0ad850 msgr2=0x7fa97c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:57.670 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.668+0000 7fa981c35640 1 --2- 172.21.15.105:0/2192264168 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa97c0ad850 0x7fa97c0701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:57.670 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.668+0000 7fa981c35640 1 -- 172.21.15.105:0/2192264168 >> v1:172.21.15.148:6789/0 conn(0x7fa97c159f40 legacy=0x7fa97c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:57.670 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.668+0000 7fa981c35640 1 -- 172.21.15.105:0/2192264168 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97c071380 con 0x7fa97c0caf60 2023-11-28T18:31:57.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.672+0000 7fa981c35640 1 --2- 172.21.15.105:0/2192264168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa97c0caf60 0x7fa97c0cc760 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fa96c009870 tx=0x7fa96c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8324d41fc133c1ab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:57.673 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.672+0000 7fa981434640 1 -- 172.21.15.105:0/2192264168 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c0ab680 con 0x7fa97c0caf60 2023-11-28T18:31:57.673 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.672+0000 7fa981434640 1 -- 172.21.15.105:0/2192264168 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96c0ab7e0 con 0x7fa97c0caf60 2023-11-28T18:31:57.673 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.672+0000 7fa981434640 1 -- 172.21.15.105:0/2192264168 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c0abb70 con 0x7fa97c0caf60 2023-11-28T18:31:57.673 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.672+0000 7fa98469a640 1 -- 172.21.15.105:0/2192264168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa97c0caf60 msgr2=0x7fa97c0cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:57.673 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.672+0000 7fa98469a640 1 --2- 172.21.15.105:0/2192264168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa97c0caf60 0x7fa97c0cc760 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fa96c009870 tx=0x7fa96c01c480 comp rx=0 tx=0).stop 2023-11-28T18:31:57.674 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.672+0000 7fa98469a640 1 -- 172.21.15.105:0/2192264168 shutdown_connections 2023-11-28T18:31:57.674 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.672+0000 7fa98469a640 1 --2- 172.21.15.105:0/2192264168 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa97c0ad850 0x7fa97c0701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.672+0000 7fa98469a640 1 --2- 172.21.15.105:0/2192264168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa97c0caf60 0x7fa97c0cc760 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.672+0000 7fa98469a640 1 -- 172.21.15.105:0/2192264168 >> 172.21.15.105:0/2192264168 conn(0x7fa97c0ceb90 msgr2=0x7fa97c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.672+0000 7fa98469a640 1 -- 172.21.15.105:0/2192264168 shutdown_connections 2023-11-28T18:31:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.672+0000 7fa98469a640 1 -- 172.21.15.105:0/2192264168 wait complete. 2023-11-28T18:31:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.672+0000 7fa98469a640 1 Processor -- start 2023-11-28T18:31:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa98469a640 1 -- start start 2023-11-28T18:31:57.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa98469a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa97c0ad850 0x7fa97c15ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:57.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa98469a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa97c0caf60 0x7fa97c09ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:57.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa98469a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa97c159f40 0x7fa97c0cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:57.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa98469a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c09c7c0 con 0x7fa97c0caf60 2023-11-28T18:31:57.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa98469a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c15d360 con 0x7fa97c0ad850 2023-11-28T18:31:57.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa98469a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c09c4e0 con 0x7fa97c159f40 2023-11-28T18:31:57.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa981c35640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa97c0caf60 0x7fa97c09ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:57.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa981c35640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa97c0caf60 0x7fa97c09ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48726/0 (socket says 172.21.15.105:48726) 2023-11-28T18:31:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa981c35640 1 -- 172.21.15.105:0/1474479260 learned_addr learned my addr 172.21.15.105:0/1474479260 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa982436640 1 --2- 172.21.15.105:0/1474479260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa97c0ad850 0x7fa97c15ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa982c37640 1 --2- 172.21.15.105:0/1474479260 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa97c159f40 0x7fa97c0cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa981c35640 1 -- 172.21.15.105:0/1474479260 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa97c159f40 msgr2=0x7fa97c0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa981c35640 1 --2- 172.21.15.105:0/1474479260 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa97c159f40 0x7fa97c0cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa981c35640 1 -- 172.21.15.105:0/1474479260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa97c0ad850 msgr2=0x7fa97c15ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa981c35640 1 --2- 172.21.15.105:0/1474479260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa97c0ad850 0x7fa97c15ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa981c35640 1 -- 172.21.15.105:0/1474479260 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c0094d0 con 0x7fa97c0caf60 2023-11-28T18:31:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa982c37640 1 --2- 172.21.15.105:0/1474479260 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa97c159f40 0x7fa97c0cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:57.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa981c35640 1 --2- 172.21.15.105:0/1474479260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa97c0caf60 0x7fa97c09ba10 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fa96c01a0b0 tx=0x7fa96c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:57.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa972ffd640 1 -- 172.21.15.105:0/1474479260 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c0b4800 con 0x7fa97c0caf60 2023-11-28T18:31:57.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa972ffd640 1 -- 172.21.15.105:0/1474479260 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96c0b4980 con 0x7fa97c0caf60 2023-11-28T18:31:57.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa98469a640 1 -- 172.21.15.105:0/1474479260 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa97c15a6c0 con 0x7fa97c0caf60 2023-11-28T18:31:57.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa972ffd640 1 -- 172.21.15.105:0/1474479260 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c0b4d30 con 0x7fa97c0caf60 2023-11-28T18:31:57.680 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa98469a640 1 -- 172.21.15.105:0/1474479260 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa97c15ac80 con 0x7fa97c0caf60 2023-11-28T18:31:57.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.676+0000 7fa98469a640 1 -- 172.21.15.105:0/1474479260 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa97c15a520 con 0x7fa97c0caf60 2023-11-28T18:31:57.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.680+0000 7fa972ffd640 1 -- 172.21.15.105:0/1474479260 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fa96c0b3380 con 0x7fa97c0caf60 2023-11-28T18:31:57.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.680+0000 7fa972ffd640 1 --2- 172.21.15.105:0/1474479260 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa9600668d0 0x7fa960068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:57.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.680+0000 7fa972ffd640 1 -- 172.21.15.105:0/1474479260 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5547+0+0 (secure 0 0 0) 0x7fa96c11ed40 con 0x7fa97c0caf60 2023-11-28T18:31:57.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.680+0000 7fa982436640 1 --2- 172.21.15.105:0/1474479260 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa9600668d0 0x7fa960068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:57.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.680+0000 7fa982436640 1 --2- 172.21.15.105:0/1474479260 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa9600668d0 0x7fa960068d90 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa978007fd0 tx=0x7fa978009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:57.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.684+0000 7fa972ffd640 1 -- 172.21.15.105:0/1474479260 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa97c15a520 con 0x7fa97c0caf60 2023-11-28T18:31:57.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:57.836+0000 7fa98469a640 1 -- 172.21.15.105:0/1474479260 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa97c15a520 con 0x7fa97c0caf60 2023-11-28T18:31:58.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.436+0000 7fa972ffd640 1 -- 172.21.15.105:0/1474479260 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v127) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa97c15a520 con 0x7fa97c0caf60 2023-11-28T18:31:58.436 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' created 2023-11-28T18:31:58.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.436+0000 7fa98469a640 1 -- 172.21.15.105:0/1474479260 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa9600668d0 msgr2=0x7fa960068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:58.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.436+0000 7fa98469a640 1 --2- 172.21.15.105:0/1474479260 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa9600668d0 0x7fa960068d90 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa978007fd0 tx=0x7fa978009040 comp rx=0 tx=0).stop 2023-11-28T18:31:58.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.436+0000 7fa98469a640 1 -- 172.21.15.105:0/1474479260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa97c0caf60 msgr2=0x7fa97c09ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:58.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.436+0000 7fa98469a640 1 --2- 172.21.15.105:0/1474479260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa97c0caf60 0x7fa97c09ba10 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fa96c01a0b0 tx=0x7fa96c01acc0 comp rx=0 tx=0).stop 2023-11-28T18:31:58.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.440+0000 7fa98469a640 1 -- 172.21.15.105:0/1474479260 shutdown_connections 2023-11-28T18:31:58.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.440+0000 7fa98469a640 1 --2- 172.21.15.105:0/1474479260 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa97c159f40 0x7fa97c0cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:58.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.440+0000 7fa98469a640 1 --2- 172.21.15.105:0/1474479260 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fa9600668d0 0x7fa960068d90 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:58.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.440+0000 7fa98469a640 1 --2- 172.21.15.105:0/1474479260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa97c0caf60 0x7fa97c09ba10 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:58.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.440+0000 7fa98469a640 1 --2- 172.21.15.105:0/1474479260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa97c0ad850 0x7fa97c15ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:58.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.440+0000 7fa98469a640 1 -- 172.21.15.105:0/1474479260 >> 172.21.15.105:0/1474479260 conn(0x7fa97c0ceb90 msgr2=0x7fa97c06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:58.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.440+0000 7fa98469a640 1 -- 172.21.15.105:0/1474479260 shutdown_connections 2023-11-28T18:31:58.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.440+0000 7fa98469a640 1 -- 172.21.15.105:0/1474479260 wait complete. 2023-11-28T18:31:58.475 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T18:31:58.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.616+0000 7f5b57a9c640 1 Processor -- start 2023-11-28T18:31:58.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.616+0000 7f5b57a9c640 1 -- start start 2023-11-28T18:31:58.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.616+0000 7f5b57a9c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b50159f40 0x7f5b500caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:58.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.616+0000 7f5b57a9c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b5006bf50 con 0x7f5b500cc760 2023-11-28T18:31:58.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.616+0000 7f5b57a9c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b500ccbc0 con 0x7f5b500ad850 2023-11-28T18:31:58.617 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.616+0000 7f5b57a9c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5015a320 con 0x7f5b50159f40 2023-11-28T18:31:58.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.616+0000 7f5b55037640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5b500cc760 0x7f5b500ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:52720/0 (socket says 172.21.15.105:52720) 2023-11-28T18:31:58.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.616+0000 7f5b55037640 1 -- 172.21.15.105:0/1944436271 learned_addr learned my addr 172.21.15.105:0/1944436271 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:58.618 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.616+0000 7f5b55838640 1 --2- 172.21.15.105:0/1944436271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b50159f40 0x7f5b500caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:58.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.616+0000 7f5b54836640 1 -- 172.21.15.105:0/1944436271 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1201801963 0 0) 0x7f5b5006bf50 con 0x7f5b500cc760 2023-11-28T18:31:58.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.616+0000 7f5b54836640 1 -- 172.21.15.105:0/1944436271 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b3c003530 con 0x7f5b500cc760 2023-11-28T18:31:58.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.616+0000 7f5b54836640 1 -- 172.21.15.105:0/1944436271 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4160785487 0 0) 0x7f5b500ccbc0 con 0x7f5b500ad850 2023-11-28T18:31:58.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.616+0000 7f5b54836640 1 -- 172.21.15.105:0/1944436271 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b5006bf50 con 0x7f5b500ad850 2023-11-28T18:31:58.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.616+0000 7f5b55838640 1 -- 172.21.15.105:0/1944436271 >> v1:172.21.15.148:6789/0 conn(0x7f5b500ad850 legacy=0x7f5b500701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:58.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.616+0000 7f5b55838640 1 -- 172.21.15.105:0/1944436271 >> v1:172.21.15.105:6789/0 conn(0x7f5b500cc760 legacy=0x7f5b500ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:58.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.616+0000 7f5b55838640 1 -- 172.21.15.105:0/1944436271 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b50071380 con 0x7f5b50159f40 2023-11-28T18:31:58.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.620+0000 7f5b55838640 1 --2- 172.21.15.105:0/1944436271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b50159f40 0x7f5b500caf60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0097f0 tx=0x7f5b4c01c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7246d0ad4542d81e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:58.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.620+0000 7f5b54836640 1 -- 172.21.15.105:0/1944436271 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c0ab600 con 0x7f5b50159f40 2023-11-28T18:31:58.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.620+0000 7f5b54836640 1 -- 172.21.15.105:0/1944436271 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4c0ab760 con 0x7f5b50159f40 2023-11-28T18:31:58.621 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.620+0000 7f5b54836640 1 -- 172.21.15.105:0/1944436271 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c0abaf0 con 0x7f5b50159f40 2023-11-28T18:31:58.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.620+0000 7f5b57a9c640 1 -- 172.21.15.105:0/1944436271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b50159f40 msgr2=0x7f5b500caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:58.622 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.620+0000 7f5b57a9c640 1 --2- 172.21.15.105:0/1944436271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b50159f40 0x7f5b500caf60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0097f0 tx=0x7f5b4c01c400 comp rx=0 tx=0).stop 2023-11-28T18:31:58.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.620+0000 7f5b57a9c640 1 -- 172.21.15.105:0/1944436271 shutdown_connections 2023-11-28T18:31:58.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.620+0000 7f5b57a9c640 1 --2- 172.21.15.105:0/1944436271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b50159f40 0x7f5b500caf60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:58.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.620+0000 7f5b57a9c640 1 -- 172.21.15.105:0/1944436271 >> 172.21.15.105:0/1944436271 conn(0x7f5b500ceb90 msgr2=0x7f5b500cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:58.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.620+0000 7f5b57a9c640 1 -- 172.21.15.105:0/1944436271 shutdown_connections 2023-11-28T18:31:58.623 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.620+0000 7f5b57a9c640 1 -- 172.21.15.105:0/1944436271 wait complete. 2023-11-28T18:31:58.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b57a9c640 1 Processor -- start 2023-11-28T18:31:58.624 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b57a9c640 1 -- start start 2023-11-28T18:31:58.625 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b57a9c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b500ad850 0x7f5b5014e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:58.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b57a9c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b500cc760 0x7f5b5014f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:58.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b57a9c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b50159f40 0x7f5b5014e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:58.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b57a9c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b50151d30 con 0x7f5b50159f40 2023-11-28T18:31:58.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b57a9c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b50151120 con 0x7f5b500cc760 2023-11-28T18:31:58.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b57a9c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b50151400 con 0x7f5b500ad850 2023-11-28T18:31:58.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b56039640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b50159f40 0x7f5b5014e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:58.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b56039640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b50159f40 0x7f5b5014e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48740/0 (socket says 172.21.15.105:48740) 2023-11-28T18:31:58.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b56039640 1 -- 172.21.15.105:0/3704420889 learned_addr learned my addr 172.21.15.105:0/3704420889 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:58.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b55838640 1 --2- 172.21.15.105:0/3704420889 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b500ad850 0x7f5b5014e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:58.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b55037640 1 --2- 172.21.15.105:0/3704420889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b500cc760 0x7f5b5014f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:58.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b56039640 1 -- 172.21.15.105:0/3704420889 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b500ad850 msgr2=0x7f5b5014e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:58.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b56039640 1 --2- 172.21.15.105:0/3704420889 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b500ad850 0x7f5b5014e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:58.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b56039640 1 -- 172.21.15.105:0/3704420889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b500cc760 msgr2=0x7f5b5014f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:58.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b56039640 1 --2- 172.21.15.105:0/3704420889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b500cc760 0x7f5b5014f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:58.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b56039640 1 -- 172.21.15.105:0/3704420889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c009420 con 0x7f5b50159f40 2023-11-28T18:31:58.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b56039640 1 --2- 172.21.15.105:0/3704420889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b50159f40 0x7f5b5014e170 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800c730 tx=0x7f5b4800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:58.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b467fc640 1 -- 172.21.15.105:0/3704420889 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b48003b80 con 0x7f5b50159f40 2023-11-28T18:31:58.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b467fc640 1 -- 172.21.15.105:0/3704420889 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b48003ce0 con 0x7f5b50159f40 2023-11-28T18:31:58.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b57a9c640 1 -- 172.21.15.105:0/3704420889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b501516e0 con 0x7f5b50159f40 2023-11-28T18:31:58.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b467fc640 1 -- 172.21.15.105:0/3704420889 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4800aa90 con 0x7f5b50159f40 2023-11-28T18:31:58.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.624+0000 7f5b57a9c640 1 -- 172.21.15.105:0/3704420889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b50150e10 con 0x7f5b50159f40 2023-11-28T18:31:58.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.628+0000 7f5b57a9c640 1 -- 172.21.15.105:0/3704420889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b50151ee0 con 0x7f5b50159f40 2023-11-28T18:31:58.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.628+0000 7f5b467fc640 1 -- 172.21.15.105:0/3704420889 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5b4800abf0 con 0x7f5b50159f40 2023-11-28T18:31:58.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.628+0000 7f5b467fc640 1 --2- 172.21.15.105:0/3704420889 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5b34066900 0x7f5b34068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:58.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.628+0000 7f5b467fc640 1 -- 172.21.15.105:0/3704420889 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f5b48084a00 con 0x7f5b50159f40 2023-11-28T18:31:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.628+0000 7f5b55838640 1 --2- 172.21.15.105:0/3704420889 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5b34066900 0x7f5b34068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.632+0000 7f5b55838640 1 --2- 172.21.15.105:0/3704420889 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5b34066900 0x7f5b34068dc0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0af000 tx=0x7f5b4c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:58.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.632+0000 7f5b467fc640 1 -- 172.21.15.105:0/3704420889 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5b48051d80 con 0x7f5b50159f40 2023-11-28T18:31:58.823 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:58.824+0000 7f5b57a9c640 1 -- 172.21.15.105:0/3704420889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5b50067570 con 0x7f5b50159f40 2023-11-28T18:31:59.466 INFO:teuthology.orchestra.run.smithi105.stderr:new fs with metadata pool 19 and data pool 20 2023-11-28T18:31:59.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.464+0000 7f5b467fc640 1 -- 172.21.15.105:0/3704420889 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v130) v1 ==== 174+0+0 (secure 0 0 0) 0x7f5b480553a0 con 0x7f5b50159f40 2023-11-28T18:31:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.468+0000 7f5b57a9c640 1 -- 172.21.15.105:0/3704420889 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5b34066900 msgr2=0x7f5b34068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.468+0000 7f5b57a9c640 1 --2- 172.21.15.105:0/3704420889 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5b34066900 0x7f5b34068dc0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0af000 tx=0x7f5b4c0a7000 comp rx=0 tx=0).stop 2023-11-28T18:31:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.468+0000 7f5b57a9c640 1 -- 172.21.15.105:0/3704420889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b50159f40 msgr2=0x7f5b5014e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.468+0000 7f5b57a9c640 1 --2- 172.21.15.105:0/3704420889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b50159f40 0x7f5b5014e170 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800c730 tx=0x7f5b4800cbf0 comp rx=0 tx=0).stop 2023-11-28T18:31:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.468+0000 7f5b57a9c640 1 -- 172.21.15.105:0/3704420889 shutdown_connections 2023-11-28T18:31:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.468+0000 7f5b57a9c640 1 --2- 172.21.15.105:0/3704420889 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5b34066900 0x7f5b34068dc0 secure :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0af000 tx=0x7f5b4c0a7000 comp rx=0 tx=0).stop 2023-11-28T18:31:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.468+0000 7f5b57a9c640 1 --2- 172.21.15.105:0/3704420889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b50159f40 0x7f5b5014e170 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:59.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.468+0000 7f5b57a9c640 1 --2- 172.21.15.105:0/3704420889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b500cc760 0x7f5b5014f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:59.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.468+0000 7f5b57a9c640 1 --2- 172.21.15.105:0/3704420889 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b500ad850 0x7f5b5014e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:59.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.468+0000 7f5b57a9c640 1 -- 172.21.15.105:0/3704420889 >> 172.21.15.105:0/3704420889 conn(0x7f5b500ceb90 msgr2=0x7f5b500ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:59.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.476+0000 7f5b57a9c640 1 -- 172.21.15.105:0/3704420889 shutdown_connections 2023-11-28T18:31:59.477 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.476+0000 7f5b57a9c640 1 -- 172.21.15.105:0/3704420889 wait complete. 2023-11-28T18:31:59.508 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:31:59.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.644+0000 7efe9fadd640 1 Processor -- start 2023-11-28T18:31:59.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.644+0000 7efe9fadd640 1 -- start start 2023-11-28T18:31:59.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.648+0000 7efe9fadd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efe98154070 0x7efe9815d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:59.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.648+0000 7efe9fadd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe9806bf50 0x7efe980ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:59.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.648+0000 7efe9fadd640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe980cc7f0 con 0x7efe980adb80 2023-11-28T18:31:59.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.648+0000 7efe9fadd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe98072540 con 0x7efe98154070 2023-11-28T18:31:59.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.648+0000 7efe9fadd640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9815dd10 con 0x7efe9806bf50 2023-11-28T18:31:59.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.648+0000 7efe9e07a640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7efe980adb80 0x7efe980acd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:52728/0 (socket says 172.21.15.105:52728) 2023-11-28T18:31:59.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.648+0000 7efe9e07a640 1 -- 172.21.15.105:0/3960146059 learned_addr learned my addr 172.21.15.105:0/3960146059 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:59.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.648+0000 7efe9d879640 1 --2- 172.21.15.105:0/3960146059 >> v2:172.21.15.148:3300/0 conn(0x7efe98154070 0x7efe9815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:59.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.648+0000 7efe9d078640 1 --2- 172.21.15.105:0/3960146059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe9806bf50 0x7efe980ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:59.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.648+0000 7efe9c877640 1 -- 172.21.15.105:0/3960146059 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 876380687 0 0) 0x7efe980cc7f0 con 0x7efe980adb80 2023-11-28T18:31:59.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.648+0000 7efe9c877640 1 -- 172.21.15.105:0/3960146059 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe84003530 con 0x7efe980adb80 2023-11-28T18:31:59.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.648+0000 7efe9d078640 1 -- 172.21.15.105:0/3960146059 >> v2:172.21.15.148:3300/0 conn(0x7efe98154070 msgr2=0x7efe9815d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:59.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.648+0000 7efe9d078640 1 --2- 172.21.15.105:0/3960146059 >> v2:172.21.15.148:3300/0 conn(0x7efe98154070 0x7efe9815d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:59.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.648+0000 7efe9d078640 1 -- 172.21.15.105:0/3960146059 >> v1:172.21.15.105:6789/0 conn(0x7efe980adb80 legacy=0x7efe980acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:59.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.648+0000 7efe9d078640 1 -- 172.21.15.105:0/3960146059 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9815dff0 con 0x7efe9806bf50 2023-11-28T18:31:59.650 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.648+0000 7efe9d879640 1 --2- 172.21.15.105:0/3960146059 >> v2:172.21.15.148:3300/0 conn(0x7efe98154070 0x7efe9815d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:31:59.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.652+0000 7efe9d078640 1 --2- 172.21.15.105:0/3960146059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe9806bf50 0x7efe980ce310 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efe880097c0 tx=0x7efe8801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9345b7c73f37295 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:59.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.652+0000 7efe9c877640 1 -- 172.21.15.105:0/3960146059 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe880ab650 con 0x7efe9806bf50 2023-11-28T18:31:59.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.652+0000 7efe9c877640 1 -- 172.21.15.105:0/3960146059 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe880ab7b0 con 0x7efe9806bf50 2023-11-28T18:31:59.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.652+0000 7efe9c877640 1 -- 172.21.15.105:0/3960146059 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe880abb40 con 0x7efe9806bf50 2023-11-28T18:31:59.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.652+0000 7efe9fadd640 1 -- 172.21.15.105:0/3960146059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe9806bf50 msgr2=0x7efe980ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:59.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.652+0000 7efe9fadd640 1 --2- 172.21.15.105:0/3960146059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe9806bf50 0x7efe980ce310 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efe880097c0 tx=0x7efe8801c450 comp rx=0 tx=0).stop 2023-11-28T18:31:59.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.652+0000 7efe9fadd640 1 -- 172.21.15.105:0/3960146059 shutdown_connections 2023-11-28T18:31:59.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.652+0000 7efe9fadd640 1 --2- 172.21.15.105:0/3960146059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe9806bf50 0x7efe980ce310 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:59.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.652+0000 7efe9fadd640 1 --2- 172.21.15.105:0/3960146059 >> v2:172.21.15.148:3300/0 conn(0x7efe98154070 0x7efe9815d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:59.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.652+0000 7efe9fadd640 1 -- 172.21.15.105:0/3960146059 >> 172.21.15.105:0/3960146059 conn(0x7efe980ceb90 msgr2=0x7efe980cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:59.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.652+0000 7efe9fadd640 1 -- 172.21.15.105:0/3960146059 shutdown_connections 2023-11-28T18:31:59.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.652+0000 7efe9fadd640 1 -- 172.21.15.105:0/3960146059 wait complete. 2023-11-28T18:31:59.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe9fadd640 1 Processor -- start 2023-11-28T18:31:59.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe9fadd640 1 -- start start 2023-11-28T18:31:59.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe9fadd640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe9806bf50 0x7efe98145190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:59.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe9fadd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe980adb80 0x7efe980b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:59.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe9fadd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe98154070 0x7efe9814d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:59.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe9fadd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe980b46a0 con 0x7efe98154070 2023-11-28T18:31:59.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe9fadd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe980b43c0 con 0x7efe980adb80 2023-11-28T18:31:59.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe9fadd640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9814b700 con 0x7efe9806bf50 2023-11-28T18:31:59.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe9e07a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe98154070 0x7efe9814d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:59.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe9e07a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe98154070 0x7efe9814d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48748/0 (socket says 172.21.15.105:48748) 2023-11-28T18:31:59.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe9e07a640 1 -- 172.21.15.105:0/3924880232 learned_addr learned my addr 172.21.15.105:0/3924880232 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:59.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe9d879640 1 --2- 172.21.15.105:0/3924880232 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe9806bf50 0x7efe98145190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:59.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe9d078640 1 --2- 172.21.15.105:0/3924880232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe980adb80 0x7efe980b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:59.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe9e07a640 1 -- 172.21.15.105:0/3924880232 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe9806bf50 msgr2=0x7efe98145190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:59.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe9e07a640 1 --2- 172.21.15.105:0/3924880232 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe9806bf50 0x7efe98145190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:59.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe9e07a640 1 -- 172.21.15.105:0/3924880232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe980adb80 msgr2=0x7efe980b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:59.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe9e07a640 1 --2- 172.21.15.105:0/3924880232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe980adb80 0x7efe980b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:59.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe9e07a640 1 -- 172.21.15.105:0/3924880232 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe94009420 con 0x7efe98154070 2023-11-28T18:31:59.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe9e07a640 1 --2- 172.21.15.105:0/3924880232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe98154070 0x7efe9814d9d0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7efe9000cb40 tx=0x7efe9000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:59.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe8e7fc640 1 -- 172.21.15.105:0/3924880232 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9000aca0 con 0x7efe98154070 2023-11-28T18:31:59.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe8e7fc640 1 -- 172.21.15.105:0/3924880232 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9000ae00 con 0x7efe98154070 2023-11-28T18:31:59.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe9fadd640 1 -- 172.21.15.105:0/3924880232 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe88009420 con 0x7efe98154070 2023-11-28T18:31:59.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe8e7fc640 1 -- 172.21.15.105:0/3924880232 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe90012620 con 0x7efe98154070 2023-11-28T18:31:59.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe9fadd640 1 -- 172.21.15.105:0/3924880232 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe980ae520 con 0x7efe98154070 2023-11-28T18:31:59.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.656+0000 7efe9fadd640 1 -- 172.21.15.105:0/3924880232 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe9809cd20 con 0x7efe98154070 2023-11-28T18:31:59.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.660+0000 7efe8e7fc640 1 -- 172.21.15.105:0/3924880232 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7efe90007070 con 0x7efe98154070 2023-11-28T18:31:59.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.660+0000 7efe8e7fc640 1 --2- 172.21.15.105:0/3924880232 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe7c0668d0 0x7efe7c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:59.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.660+0000 7efe8e7fc640 1 -- 172.21.15.105:0/3924880232 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efe90084660 con 0x7efe98154070 2023-11-28T18:31:59.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.660+0000 7efe9d879640 1 --2- 172.21.15.105:0/3924880232 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe7c0668d0 0x7efe7c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:59.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.660+0000 7efe9d879640 1 --2- 172.21.15.105:0/3924880232 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe7c0668d0 0x7efe7c068d90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7efe94009c50 tx=0x7efe94009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:59.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.664+0000 7efe8e7fc640 1 -- 172.21.15.105:0/3924880232 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efe900516f0 con 0x7efe98154070 2023-11-28T18:31:59.814 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.812+0000 7efe9fadd640 1 -- 172.21.15.105:0/3924880232 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe98160a90 con 0x7efe98154070 2023-11-28T18:31:59.815 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.816+0000 7efe8e7fc640 1 -- 172.21.15.105:0/3924880232 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16805 (secure 0 0 0) 0x7efe90054d10 con 0x7efe98154070 2023-11-28T18:31:59.815 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:31:59.815 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":128,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:31:58.827091+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:31:56.971692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-28T18:31:57.881657+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:31:59.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.816+0000 7efe9fadd640 1 -- 172.21.15.105:0/3924880232 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe7c0668d0 msgr2=0x7efe7c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:59.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.816+0000 7efe9fadd640 1 --2- 172.21.15.105:0/3924880232 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe7c0668d0 0x7efe7c068d90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7efe94009c50 tx=0x7efe94009190 comp rx=0 tx=0).stop 2023-11-28T18:31:59.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.816+0000 7efe9fadd640 1 -- 172.21.15.105:0/3924880232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe98154070 msgr2=0x7efe9814d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:59.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.816+0000 7efe9fadd640 1 --2- 172.21.15.105:0/3924880232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe98154070 0x7efe9814d9d0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7efe9000cb40 tx=0x7efe9000a060 comp rx=0 tx=0).stop 2023-11-28T18:31:59.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.816+0000 7efe9fadd640 1 -- 172.21.15.105:0/3924880232 shutdown_connections 2023-11-28T18:31:59.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.816+0000 7efe9fadd640 1 --2- 172.21.15.105:0/3924880232 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7efe7c0668d0 0x7efe7c068d90 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:59.817 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.816+0000 7efe9fadd640 1 --2- 172.21.15.105:0/3924880232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe98154070 0x7efe9814d9d0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:59.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.816+0000 7efe9fadd640 1 --2- 172.21.15.105:0/3924880232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe980adb80 0x7efe980b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:59.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.816+0000 7efe9fadd640 1 --2- 172.21.15.105:0/3924880232 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe9806bf50 0x7efe98145190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:59.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.816+0000 7efe9fadd640 1 -- 172.21.15.105:0/3924880232 >> 172.21.15.105:0/3924880232 conn(0x7efe980ceb90 msgr2=0x7efe980afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:59.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.816+0000 7efe9fadd640 1 -- 172.21.15.105:0/3924880232 shutdown_connections 2023-11-28T18:31:59.819 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.816+0000 7efe9fadd640 1 -- 172.21.15.105:0/3924880232 wait complete. 2023-11-28T18:31:59.836 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:31:59.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fceabd1d640 1 Processor -- start 2023-11-28T18:31:59.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fceabd1d640 1 -- start start 2023-11-28T18:31:59.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fceabd1d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcea415d4f0 0x7fcea40d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:59.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fceabd1d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea40b25b0 con 0x7fcea40b2d20 2023-11-28T18:31:59.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fceabd1d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea40cea30 con 0x7fcea40d3c30 2023-11-28T18:31:59.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fceabd1d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea40b2900 con 0x7fcea415d4f0 2023-11-28T18:31:59.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fceaa2ba640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fcea40b2d20 0x7fcea40b31d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:52738/0 (socket says 172.21.15.105:52738) 2023-11-28T18:31:59.976 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fceaa2ba640 1 -- 172.21.15.105:0/4125374402 learned_addr learned my addr 172.21.15.105:0/4125374402 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:31:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea8ab7640 1 -- 172.21.15.105:0/4125374402 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2927334350 0 0) 0x7fcea40b25b0 con 0x7fcea40b2d20 2023-11-28T18:31:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea9ab9640 1 --2- 172.21.15.105:0/4125374402 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcea415d4f0 0x7fcea40d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea8ab7640 1 -- 172.21.15.105:0/4125374402 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce90003530 con 0x7fcea40b2d20 2023-11-28T18:31:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea8ab7640 1 -- 172.21.15.105:0/4125374402 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2734249018 0 0) 0x7fce90003530 con 0x7fcea40b2d20 2023-11-28T18:31:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea8ab7640 1 -- 172.21.15.105:0/4125374402 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcea415d4f0 msgr2=0x7fcea40d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea8ab7640 1 --2- 172.21.15.105:0/4125374402 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcea415d4f0 0x7fcea40d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea8ab7640 1 -- 172.21.15.105:0/4125374402 >> v1:172.21.15.148:6789/0 conn(0x7fcea40d3c30 legacy=0x7fcea40d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea8ab7640 1 -- 172.21.15.105:0/4125374402 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea40ced10 con 0x7fcea40b2d20 2023-11-28T18:31:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea8ab7640 1 -- 172.21.15.105:0/4125374402 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fce9c001a00 con 0x7fcea40b2d20 2023-11-28T18:31:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea8ab7640 1 -- 172.21.15.105:0/4125374402 >> v1:172.21.15.105:6789/0 conn(0x7fcea40b2d20 legacy=0x7fcea40b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea8ab7640 1 --2- 172.21.15.105:0/4125374402 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcea415d4f0 0x7fce900039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea8ab7640 1 --2- 172.21.15.105:0/4125374402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce90003f80 0x7fce900047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea8ab7640 1 --2- 172.21.15.105:0/4125374402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce90004d10 0x7fce90005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea8ab7640 1 -- 172.21.15.105:0/4125374402 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea40b2900 con 0x7fce90003f80 2023-11-28T18:31:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea8ab7640 1 -- 172.21.15.105:0/4125374402 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9000bb80 con 0x7fce90004d10 2023-11-28T18:31:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea8ab7640 1 -- 172.21.15.105:0/4125374402 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9000bd40 con 0x7fcea415d4f0 2023-11-28T18:31:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea92b8640 1 --2- 172.21.15.105:0/4125374402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce90003f80 0x7fce900047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea9ab9640 1 --2- 172.21.15.105:0/4125374402 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcea415d4f0 0x7fce900039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fceaa2ba640 1 --2- 172.21.15.105:0/4125374402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce90004d10 0x7fce90005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea92b8640 1 -- 172.21.15.105:0/4125374402 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcea415d4f0 msgr2=0x7fce900039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea92b8640 1 --2- 172.21.15.105:0/4125374402 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcea415d4f0 0x7fce900039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea92b8640 1 -- 172.21.15.105:0/4125374402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce90004d10 msgr2=0x7fce90005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:31:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea92b8640 1 --2- 172.21.15.105:0/4125374402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce90004d10 0x7fce90005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.976+0000 7fcea92b8640 1 -- 172.21.15.105:0/4125374402 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce90003530 con 0x7fce90003f80 2023-11-28T18:31:59.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.980+0000 7fcea92b8640 1 --2- 172.21.15.105:0/4125374402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce90003f80 0x7fce900047d0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fce94001780 tx=0x7fce9401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656c676bca4397da server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:59.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.980+0000 7fcea8ab7640 1 -- 172.21.15.105:0/4125374402 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce940052d0 con 0x7fce90003f80 2023-11-28T18:31:59.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.980+0000 7fcea8ab7640 1 -- 172.21.15.105:0/4125374402 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce940a7050 con 0x7fce90003f80 2023-11-28T18:31:59.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.980+0000 7fcea8ab7640 1 -- 172.21.15.105:0/4125374402 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94003600 con 0x7fce90003f80 2023-11-28T18:31:59.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.980+0000 7fceabd1d640 1 -- 172.21.15.105:0/4125374402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce90003f80 msgr2=0x7fce900047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:59.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.980+0000 7fceabd1d640 1 --2- 172.21.15.105:0/4125374402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce90003f80 0x7fce900047d0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fce94001780 tx=0x7fce9401c3d0 comp rx=0 tx=0).stop 2023-11-28T18:31:59.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.980+0000 7fceaa2ba640 1 -- 172.21.15.105:0/4125374402 reap_dead start 2023-11-28T18:31:59.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.980+0000 7fceabd1d640 1 -- 172.21.15.105:0/4125374402 shutdown_connections 2023-11-28T18:31:59.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.980+0000 7fceabd1d640 1 -- 172.21.15.105:0/4125374402 >> 172.21.15.105:0/4125374402 conn(0x7fcea40d3030 msgr2=0x7fcea40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:31:59.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.980+0000 7fceabd1d640 1 -- 172.21.15.105:0/4125374402 shutdown_connections 2023-11-28T18:31:59.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.980+0000 7fceabd1d640 1 -- 172.21.15.105:0/4125374402 wait complete. 2023-11-28T18:31:59.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fceabd1d640 1 Processor -- start 2023-11-28T18:31:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fceabd1d640 1 -- start start 2023-11-28T18:31:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fceabd1d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce90003f80 0x7fcea40cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fceabd1d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcea40b8770 0x7fcea415eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fceabd1d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcea415f030 0x7fcea41f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fceabd1d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea40d0a20 con 0x7fcea415f030 2023-11-28T18:31:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fceabd1d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea415dae0 con 0x7fcea40b8770 2023-11-28T18:31:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fceabd1d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea415dd30 con 0x7fce90003f80 2023-11-28T18:31:59.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fceaa2ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcea415f030 0x7fcea41f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:59.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fceaa2ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcea415f030 0x7fcea41f5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48754/0 (socket says 172.21.15.105:48754) 2023-11-28T18:31:59.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fceaa2ba640 1 -- 172.21.15.105:0/2525293107 learned_addr learned my addr 172.21.15.105:0/2525293107 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:31:59.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fcea92b8640 1 --2- 172.21.15.105:0/2525293107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcea40b8770 0x7fcea415eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:59.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fcea9ab9640 1 --2- 172.21.15.105:0/2525293107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce90003f80 0x7fcea40cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:59.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fceaa2ba640 1 -- 172.21.15.105:0/2525293107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce90003f80 msgr2=0x7fcea40cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:59.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fceaa2ba640 1 --2- 172.21.15.105:0/2525293107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce90003f80 0x7fcea40cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:59.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fceaa2ba640 1 -- 172.21.15.105:0/2525293107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcea40b8770 msgr2=0x7fcea415eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:31:59.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fceaa2ba640 1 --2- 172.21.15.105:0/2525293107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcea40b8770 0x7fcea415eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:31:59.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fceaa2ba640 1 -- 172.21.15.105:0/2525293107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea415e010 con 0x7fcea415f030 2023-11-28T18:31:59.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fcea9ab9640 1 --2- 172.21.15.105:0/2525293107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce90003f80 0x7fcea40cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:31:59.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fceaa2ba640 1 --2- 172.21.15.105:0/2525293107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcea415f030 0x7fcea41f5fc0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fcea40cf440 tx=0x7fce9c007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:59.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fce9a7fc640 1 -- 172.21.15.105:0/2525293107 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c00ab90 con 0x7fcea415f030 2023-11-28T18:31:59.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fce9a7fc640 1 -- 172.21.15.105:0/2525293107 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9c00ad30 con 0x7fcea415f030 2023-11-28T18:31:59.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fceabd1d640 1 -- 172.21.15.105:0/2525293107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea415e2f0 con 0x7fcea415f030 2023-11-28T18:31:59.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fce9a7fc640 1 -- 172.21.15.105:0/2525293107 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c00aea0 con 0x7fcea415f030 2023-11-28T18:31:59.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.984+0000 7fceabd1d640 1 -- 172.21.15.105:0/2525293107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea415c9e0 con 0x7fcea415f030 2023-11-28T18:31:59.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.988+0000 7fceabd1d640 1 -- 172.21.15.105:0/2525293107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea4067570 con 0x7fcea415f030 2023-11-28T18:31:59.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.988+0000 7fce9a7fc640 1 -- 172.21.15.105:0/2525293107 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fce9c013630 con 0x7fcea415f030 2023-11-28T18:31:59.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.988+0000 7fce9a7fc640 1 --2- 172.21.15.105:0/2525293107 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fce880668d0 0x7fce88068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:31:59.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.988+0000 7fce9a7fc640 1 -- 172.21.15.105:0/2525293107 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fce9c084bb0 con 0x7fcea415f030 2023-11-28T18:31:59.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.988+0000 7fcea9ab9640 1 --2- 172.21.15.105:0/2525293107 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fce880668d0 0x7fce88068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:31:59.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.988+0000 7fcea9ab9640 1 --2- 172.21.15.105:0/2525293107 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fce880668d0 0x7fce88068d90 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fcea0009be0 tx=0x7fcea0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:31:59.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:31:59.992+0000 7fce9a7fc640 1 -- 172.21.15.105:0/2525293107 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fce9c051c10 con 0x7fcea415f030 2023-11-28T18:32:00.144 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.144+0000 7fceabd1d640 1 -- 172.21.15.105:0/2525293107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcea415e7a0 con 0x7fcea415f030 2023-11-28T18:32:00.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.144+0000 7fce9a7fc640 1 -- 172.21.15.105:0/2525293107 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16805 (secure 0 0 0) 0x7fce9c055230 con 0x7fcea415f030 2023-11-28T18:32:00.146 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:32:00.146 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":128,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:31:58.827091+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:31:56.971692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-28T18:31:57.881657+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:32:00.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.148+0000 7fceabd1d640 1 -- 172.21.15.105:0/2525293107 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fce880668d0 msgr2=0x7fce88068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:00.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.148+0000 7fceabd1d640 1 --2- 172.21.15.105:0/2525293107 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fce880668d0 0x7fce88068d90 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fcea0009be0 tx=0x7fcea0008040 comp rx=0 tx=0).stop 2023-11-28T18:32:00.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.148+0000 7fceabd1d640 1 -- 172.21.15.105:0/2525293107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcea415f030 msgr2=0x7fcea41f5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:00.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.148+0000 7fceabd1d640 1 --2- 172.21.15.105:0/2525293107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcea415f030 0x7fcea41f5fc0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fcea40cf440 tx=0x7fce9c007bd0 comp rx=0 tx=0).stop 2023-11-28T18:32:00.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.148+0000 7fceabd1d640 1 -- 172.21.15.105:0/2525293107 shutdown_connections 2023-11-28T18:32:00.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.148+0000 7fceabd1d640 1 --2- 172.21.15.105:0/2525293107 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fce880668d0 0x7fce88068d90 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:00.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.148+0000 7fceabd1d640 1 --2- 172.21.15.105:0/2525293107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcea415f030 0x7fcea41f5fc0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:00.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.148+0000 7fceabd1d640 1 --2- 172.21.15.105:0/2525293107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcea40b8770 0x7fcea415eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:00.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.148+0000 7fceabd1d640 1 --2- 172.21.15.105:0/2525293107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce90003f80 0x7fcea40cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:00.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.148+0000 7fceabd1d640 1 -- 172.21.15.105:0/2525293107 >> 172.21.15.105:0/2525293107 conn(0x7fcea40d3030 msgr2=0x7fcea40b0150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:00.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.148+0000 7fceabd1d640 1 -- 172.21.15.105:0/2525293107 shutdown_connections 2023-11-28T18:32:00.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.148+0000 7fceabd1d640 1 -- 172.21.15.105:0/2525293107 wait complete. 2023-11-28T18:32:00.168 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T18:32:00.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.308+0000 7f28b445a640 1 Processor -- start 2023-11-28T18:32:00.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.308+0000 7f28b445a640 1 -- start start 2023-11-28T18:32:00.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.308+0000 7f28b445a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f28ac0d3c30 0x7f28ac0d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:00.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.308+0000 7f28b445a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28ac0b25b0 con 0x7f28ac15d4f0 2023-11-28T18:32:00.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.308+0000 7f28b445a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28ac0cea30 con 0x7f28ac0b2d20 2023-11-28T18:32:00.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.308+0000 7f28b445a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28ac0b2900 con 0x7f28ac0d3c30 2023-11-28T18:32:00.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.308+0000 7f28b19f5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f28ac0d3c30 0x7f28ac0d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:00.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.308+0000 7f28b21f6640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f28ac15d4f0 0x7f28ac0d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:52750/0 (socket says 172.21.15.105:52750) 2023-11-28T18:32:00.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.308+0000 7f28b19f5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f28ac0d3c30 0x7f28ac0d1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48768/0 (socket says 172.21.15.105:48768) 2023-11-28T18:32:00.313 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.308+0000 7f28b21f6640 1 -- 172.21.15.105:0/1278647156 learned_addr learned my addr 172.21.15.105:0/1278647156 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:32:00.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.308+0000 7f28b19f5640 1 -- 172.21.15.105:0/1278647156 >> v1:172.21.15.148:6789/0 conn(0x7f28ac0b2d20 legacy=0x7f28ac0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:00.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.308+0000 7f28b19f5640 1 -- 172.21.15.105:0/1278647156 >> v1:172.21.15.105:6789/0 conn(0x7f28ac15d4f0 legacy=0x7f28ac0d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:00.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.308+0000 7f28b19f5640 1 -- 172.21.15.105:0/1278647156 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ac0ced10 con 0x7f28ac0d3c30 2023-11-28T18:32:00.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.308+0000 7f28b11f4640 1 -- 172.21.15.105:0/1278647156 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92748820 0 0) 0x7f28ac0b25b0 con 0x7f28ac15d4f0 2023-11-28T18:32:00.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.312+0000 7f28b19f5640 1 --2- 172.21.15.105:0/1278647156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28ac0d3c30 0x7f28ac0d1520 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f289c0097c0 tx=0x7f289c01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fefea68a04ef6c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:00.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.312+0000 7f28b11f4640 1 -- 172.21.15.105:0/1278647156 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c0ab600 con 0x7f28ac0d3c30 2023-11-28T18:32:00.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.312+0000 7f28b11f4640 1 -- 172.21.15.105:0/1278647156 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289c0a7050 con 0x7f28ac0d3c30 2023-11-28T18:32:00.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.312+0000 7f28b11f4640 1 -- 172.21.15.105:0/1278647156 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c0ab980 con 0x7f28ac0d3c30 2023-11-28T18:32:00.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.312+0000 7f28b445a640 1 -- 172.21.15.105:0/1278647156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28ac0d3c30 msgr2=0x7f28ac0d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:00.315 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.312+0000 7f28b445a640 1 --2- 172.21.15.105:0/1278647156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28ac0d3c30 0x7f28ac0d1520 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f289c0097c0 tx=0x7f289c01c380 comp rx=0 tx=0).stop 2023-11-28T18:32:00.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.312+0000 7f28b445a640 1 -- 172.21.15.105:0/1278647156 shutdown_connections 2023-11-28T18:32:00.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.312+0000 7f28b445a640 1 --2- 172.21.15.105:0/1278647156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28ac0d3c30 0x7f28ac0d1520 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:00.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.312+0000 7f28b445a640 1 -- 172.21.15.105:0/1278647156 >> 172.21.15.105:0/1278647156 conn(0x7f28ac0d3030 msgr2=0x7f28ac0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:00.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.312+0000 7f28b445a640 1 -- 172.21.15.105:0/1278647156 shutdown_connections 2023-11-28T18:32:00.316 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.312+0000 7f28b445a640 1 -- 172.21.15.105:0/1278647156 wait complete. 2023-11-28T18:32:00.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b445a640 1 Processor -- start 2023-11-28T18:32:00.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b445a640 1 -- start start 2023-11-28T18:32:00.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b445a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28ac0b2d20 0x7f28ac151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:00.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b445a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28ac0d3c30 0x7f28ac0c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:00.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b445a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ac15d4f0 0x7f28ac06b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:00.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b445a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac0d0a20 con 0x7f28ac0b2d20 2023-11-28T18:32:00.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b445a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac06ce10 con 0x7f28ac15d4f0 2023-11-28T18:32:00.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b445a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac1504c0 con 0x7f28ac0d3c30 2023-11-28T18:32:00.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b21f6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28ac0b2d20 0x7f28ac151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:00.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b21f6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28ac0b2d20 0x7f28ac151280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48770/0 (socket says 172.21.15.105:48770) 2023-11-28T18:32:00.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b21f6640 1 -- 172.21.15.105:0/3757046235 learned_addr learned my addr 172.21.15.105:0/3757046235 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:00.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b19f5640 1 --2- 172.21.15.105:0/3757046235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28ac0d3c30 0x7f28ac0c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:00.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b29f7640 1 --2- 172.21.15.105:0/3757046235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ac15d4f0 0x7f28ac06b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:00.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b21f6640 1 -- 172.21.15.105:0/3757046235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28ac0d3c30 msgr2=0x7f28ac0c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:00.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b21f6640 1 --2- 172.21.15.105:0/3757046235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28ac0d3c30 0x7f28ac0c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:00.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b21f6640 1 -- 172.21.15.105:0/3757046235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ac15d4f0 msgr2=0x7f28ac06b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:00.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b21f6640 1 --2- 172.21.15.105:0/3757046235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ac15d4f0 0x7f28ac06b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:00.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b21f6640 1 -- 172.21.15.105:0/3757046235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289c009420 con 0x7f28ac0b2d20 2023-11-28T18:32:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b19f5640 1 --2- 172.21.15.105:0/3757046235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28ac0d3c30 0x7f28ac0c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:32:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b29f7640 1 --2- 172.21.15.105:0/3757046235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ac15d4f0 0x7f28ac06b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:32:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b21f6640 1 --2- 172.21.15.105:0/3757046235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28ac0b2d20 0x7f28ac151280 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f28a800c480 tx=0x7f28a800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28a2ffd640 1 -- 172.21.15.105:0/3757046235 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a80074e0 con 0x7f28ac0b2d20 2023-11-28T18:32:00.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28a2ffd640 1 -- 172.21.15.105:0/3757046235 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a8007660 con 0x7f28ac0b2d20 2023-11-28T18:32:00.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28a2ffd640 1 -- 172.21.15.105:0/3757046235 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a8004a30 con 0x7f28ac0b2d20 2023-11-28T18:32:00.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b445a640 1 -- 172.21.15.105:0/3757046235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ac1507a0 con 0x7f28ac0b2d20 2023-11-28T18:32:00.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.316+0000 7f28b445a640 1 -- 172.21.15.105:0/3757046235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ac150d60 con 0x7f28ac0b2d20 2023-11-28T18:32:00.323 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.320+0000 7f28b445a640 1 -- 172.21.15.105:0/3757046235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28ac067570 con 0x7f28ac0b2d20 2023-11-28T18:32:00.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.320+0000 7f28a2ffd640 1 -- 172.21.15.105:0/3757046235 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f28a8004b90 con 0x7f28ac0b2d20 2023-11-28T18:32:00.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.320+0000 7f28a2ffd640 1 --2- 172.21.15.105:0/3757046235 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f28900668d0 0x7f2890068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:00.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.320+0000 7f28a2ffd640 1 -- 172.21.15.105:0/3757046235 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f28a80837a0 con 0x7f28ac0b2d20 2023-11-28T18:32:00.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.320+0000 7f28b19f5640 1 --2- 172.21.15.105:0/3757046235 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f28900668d0 0x7f2890068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:00.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.320+0000 7f28b19f5640 1 --2- 172.21.15.105:0/3757046235 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f28900668d0 0x7f2890068d90 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f289c0ad000 tx=0x7f289c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:00.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.324+0000 7f28a2ffd640 1 -- 172.21.15.105:0/3757046235 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f28a8054320 con 0x7f28ac0b2d20 2023-11-28T18:32:00.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:00.508+0000 7f28b445a640 1 -- 172.21.15.105:0/3757046235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f28ac06b210 con 0x7f28ac0b2d20 2023-11-28T18:32:01.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.480+0000 7f28a2ffd640 1 -- 172.21.15.105:0/3757046235 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v133) v1 ==== 118+0+0 (secure 0 0 0) 0x7f28a8053eb0 con 0x7f28ac0b2d20 2023-11-28T18:32:01.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.484+0000 7f28b445a640 1 -- 172.21.15.105:0/3757046235 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f28900668d0 msgr2=0x7f2890068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:01.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.484+0000 7f28b445a640 1 --2- 172.21.15.105:0/3757046235 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f28900668d0 0x7f2890068d90 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f289c0ad000 tx=0x7f289c008040 comp rx=0 tx=0).stop 2023-11-28T18:32:01.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.484+0000 7f28b445a640 1 -- 172.21.15.105:0/3757046235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28ac0b2d20 msgr2=0x7f28ac151280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:01.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.484+0000 7f28b445a640 1 --2- 172.21.15.105:0/3757046235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28ac0b2d20 0x7f28ac151280 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f28a800c480 tx=0x7f28a800c940 comp rx=0 tx=0).stop 2023-11-28T18:32:01.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.484+0000 7f28b445a640 1 -- 172.21.15.105:0/3757046235 shutdown_connections 2023-11-28T18:32:01.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.484+0000 7f28b445a640 1 --2- 172.21.15.105:0/3757046235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ac15d4f0 0x7f28ac06b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:01.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.484+0000 7f28b445a640 1 --2- 172.21.15.105:0/3757046235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28ac0d3c30 0x7f28ac0c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:01.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.484+0000 7f28b445a640 1 --2- 172.21.15.105:0/3757046235 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f28900668d0 0x7f2890068d90 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:01.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.484+0000 7f28b445a640 1 --2- 172.21.15.105:0/3757046235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28ac0b2d20 0x7f28ac151280 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:01.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.484+0000 7f28b445a640 1 -- 172.21.15.105:0/3757046235 >> 172.21.15.105:0/3757046235 conn(0x7f28ac0d3030 msgr2=0x7f28ac0d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:01.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.484+0000 7f28b445a640 1 -- 172.21.15.105:0/3757046235 shutdown_connections 2023-11-28T18:32:01.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.484+0000 7f28b445a640 1 -- 172.21.15.105:0/3757046235 wait complete. 2023-11-28T18:32:01.509 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T18:32:01.509 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T18:32:01.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.652+0000 7ffbd98fe640 1 Processor -- start 2023-11-28T18:32:01.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.652+0000 7ffbd98fe640 1 -- start start 2023-11-28T18:32:01.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.652+0000 7ffbd98fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffbd40d1650 0x7ffbd40d1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:01.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.652+0000 7ffbd98fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffbd40c7510 0x7ffbd40c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:01.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.652+0000 7ffbd98fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd40d0c60 con 0x7ffbd415d300 2023-11-28T18:32:01.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.652+0000 7ffbd98fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd40d1470 con 0x7ffbd40d1650 2023-11-28T18:32:01.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.652+0000 7ffbd98fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd40d2040 con 0x7ffbd40c7510 2023-11-28T18:32:01.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.652+0000 7ffbca7fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffbd40d1650 0x7ffbd40d1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:01.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.652+0000 7ffbca7fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffbd40d1650 0x7ffbd40d1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58462/0 (socket says 172.21.15.105:58462) 2023-11-28T18:32:01.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.652+0000 7ffbca7fc640 1 -- 172.21.15.105:0/1733334603 learned_addr learned my addr 172.21.15.105:0/1733334603 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:01.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.652+0000 7ffbd37fe640 1 --2- 172.21.15.105:0/1733334603 >> v2:172.21.15.148:3300/0 conn(0x7ffbd40c7510 0x7ffbd40c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:01.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.652+0000 7ffbca7fc640 1 -- 172.21.15.105:0/1733334603 >> v2:172.21.15.148:3300/0 conn(0x7ffbd40c7510 msgr2=0x7ffbd40c79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:01.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.652+0000 7ffbca7fc640 1 --2- 172.21.15.105:0/1733334603 >> v2:172.21.15.148:3300/0 conn(0x7ffbd40c7510 0x7ffbd40c79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:01.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.652+0000 7ffbca7fc640 1 -- 172.21.15.105:0/1733334603 >> v1:172.21.15.148:6789/0 conn(0x7ffbd415d300 legacy=0x7ffbd40d37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:01.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.652+0000 7ffbca7fc640 1 -- 172.21.15.105:0/1733334603 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd40d0eb0 con 0x7ffbd40d1650 2023-11-28T18:32:01.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.652+0000 7ffbd37fe640 1 --2- 172.21.15.105:0/1733334603 >> v2:172.21.15.148:3300/0 conn(0x7ffbd40c7510 0x7ffbd40c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:32:01.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.656+0000 7ffbca7fc640 1 --2- 172.21.15.105:0/1733334603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbd40d1650 0x7ffbd40d1b00 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7ffbd40cf750 tx=0x7ffbbc01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876d06566fc46ef6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:01.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.656+0000 7ffbd27fc640 1 -- 172.21.15.105:0/1733334603 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0ab600 con 0x7ffbd40d1650 2023-11-28T18:32:01.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.656+0000 7ffbd27fc640 1 -- 172.21.15.105:0/1733334603 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbbc0a7050 con 0x7ffbd40d1650 2023-11-28T18:32:01.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.656+0000 7ffbd27fc640 1 -- 172.21.15.105:0/1733334603 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0ab980 con 0x7ffbd40d1650 2023-11-28T18:32:01.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.656+0000 7ffbd98fe640 1 -- 172.21.15.105:0/1733334603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbd40d1650 msgr2=0x7ffbd40d1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:01.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.656+0000 7ffbd98fe640 1 --2- 172.21.15.105:0/1733334603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbd40d1650 0x7ffbd40d1b00 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7ffbd40cf750 tx=0x7ffbbc01c430 comp rx=0 tx=0).stop 2023-11-28T18:32:01.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.656+0000 7ffbd98fe640 1 -- 172.21.15.105:0/1733334603 shutdown_connections 2023-11-28T18:32:01.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.656+0000 7ffbd98fe640 1 --2- 172.21.15.105:0/1733334603 >> v2:172.21.15.148:3300/0 conn(0x7ffbd40c7510 0x7ffbd40c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:01.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.656+0000 7ffbd98fe640 1 --2- 172.21.15.105:0/1733334603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbd40d1650 0x7ffbd40d1b00 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:01.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.656+0000 7ffbd98fe640 1 -- 172.21.15.105:0/1733334603 >> 172.21.15.105:0/1733334603 conn(0x7ffbd409c360 msgr2=0x7ffbd40d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:01.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.656+0000 7ffbd98fe640 1 -- 172.21.15.105:0/1733334603 shutdown_connections 2023-11-28T18:32:01.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.656+0000 7ffbd98fe640 1 -- 172.21.15.105:0/1733334603 wait complete. 2023-11-28T18:32:01.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd98fe640 1 Processor -- start 2023-11-28T18:32:01.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd98fe640 1 -- start start 2023-11-28T18:32:01.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd98fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbd40c7510 0x7ffbd4153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:01.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd98fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd40d1650 0x7ffbd40b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:01.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd98fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbd415d300 0x7ffbd40b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:01.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd98fe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd4153b40 con 0x7ffbd40c7510 2023-11-28T18:32:01.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd98fe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd40b6010 con 0x7ffbd40d1650 2023-11-28T18:32:01.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd98fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd40b6260 con 0x7ffbd415d300 2023-11-28T18:32:01.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbd40c7510 0x7ffbd4153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:01.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbd40c7510 0x7ffbd4153600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58466/0 (socket says 172.21.15.105:58466) 2023-11-28T18:32:01.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd2ffd640 1 -- 172.21.15.105:0/1321908878 learned_addr learned my addr 172.21.15.105:0/1321908878 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:01.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbca7fc640 1 --2- 172.21.15.105:0/1321908878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd40d1650 0x7ffbd40b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:01.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd37fe640 1 --2- 172.21.15.105:0/1321908878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbd415d300 0x7ffbd40b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:01.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd2ffd640 1 -- 172.21.15.105:0/1321908878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbd415d300 msgr2=0x7ffbd40b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:01.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd2ffd640 1 --2- 172.21.15.105:0/1321908878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbd415d300 0x7ffbd40b4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:01.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd2ffd640 1 -- 172.21.15.105:0/1321908878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd40d1650 msgr2=0x7ffbd40b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:01.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd2ffd640 1 --2- 172.21.15.105:0/1321908878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd40d1650 0x7ffbd40b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:01.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd2ffd640 1 -- 172.21.15.105:0/1321908878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc0094d0 con 0x7ffbd40c7510 2023-11-28T18:32:01.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd37fe640 1 --2- 172.21.15.105:0/1321908878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbd415d300 0x7ffbd40b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:32:01.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd2ffd640 1 --2- 172.21.15.105:0/1321908878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbd40c7510 0x7ffbd4153600 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7ffbc400ba50 tx=0x7ffbc400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:01.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd0ff9640 1 -- 172.21.15.105:0/1321908878 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc400cbe0 con 0x7ffbd40c7510 2023-11-28T18:32:01.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd0ff9640 1 -- 172.21.15.105:0/1321908878 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc400cd80 con 0x7ffbd40c7510 2023-11-28T18:32:01.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd98fe640 1 -- 172.21.15.105:0/1321908878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd40b6800 con 0x7ffbd40c7510 2023-11-28T18:32:01.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd0ff9640 1 -- 172.21.15.105:0/1321908878 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4014680 con 0x7ffbd40c7510 2023-11-28T18:32:01.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.660+0000 7ffbd98fe640 1 -- 172.21.15.105:0/1321908878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd40b6ae0 con 0x7ffbd40c7510 2023-11-28T18:32:01.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.664+0000 7ffbd98fe640 1 -- 172.21.15.105:0/1321908878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd40b6660 con 0x7ffbd40c7510 2023-11-28T18:32:01.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.664+0000 7ffbd0ff9640 1 -- 172.21.15.105:0/1321908878 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ffbc40147e0 con 0x7ffbd40c7510 2023-11-28T18:32:01.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.664+0000 7ffbd0ff9640 1 --2- 172.21.15.105:0/1321908878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffbb40668d0 0x7ffbb4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:01.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.664+0000 7ffbd0ff9640 1 -- 172.21.15.105:0/1321908878 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ffbc40844a0 con 0x7ffbd40c7510 2023-11-28T18:32:01.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.664+0000 7ffbca7fc640 1 --2- 172.21.15.105:0/1321908878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffbb40668d0 0x7ffbb4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:01.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.664+0000 7ffbca7fc640 1 --2- 172.21.15.105:0/1321908878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffbb40668d0 0x7ffbb4068d90 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc0ad000 tx=0x7ffbbc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:01.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.668+0000 7ffbd0ff9640 1 -- 172.21.15.105:0/1321908878 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ffbd40b6660 con 0x7ffbd40c7510 2023-11-28T18:32:01.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:01.848+0000 7ffbd98fe640 1 -- 172.21.15.105:0/1321908878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ffbd40b6660 con 0x7ffbd40c7510 2023-11-28T18:32:02.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.484+0000 7ffbd0ff9640 1 -- 172.21.15.105:0/1321908878 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v134) v1 ==== 122+0+0 (secure 0 0 0) 0x7ffbd40b6660 con 0x7ffbd40c7510 2023-11-28T18:32:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.488+0000 7ffbd98fe640 1 -- 172.21.15.105:0/1321908878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffbb40668d0 msgr2=0x7ffbb4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:02.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.488+0000 7ffbd98fe640 1 --2- 172.21.15.105:0/1321908878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffbb40668d0 0x7ffbb4068d90 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc0ad000 tx=0x7ffbbc008040 comp rx=0 tx=0).stop 2023-11-28T18:32:02.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.488+0000 7ffbd98fe640 1 -- 172.21.15.105:0/1321908878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbd40c7510 msgr2=0x7ffbd4153600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:02.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.488+0000 7ffbd98fe640 1 --2- 172.21.15.105:0/1321908878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbd40c7510 0x7ffbd4153600 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7ffbc400ba50 tx=0x7ffbc400bf10 comp rx=0 tx=0).stop 2023-11-28T18:32:02.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.488+0000 7ffbd98fe640 1 -- 172.21.15.105:0/1321908878 shutdown_connections 2023-11-28T18:32:02.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.488+0000 7ffbd98fe640 1 --2- 172.21.15.105:0/1321908878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffbd415d300 0x7ffbd40b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:02.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.488+0000 7ffbd98fe640 1 --2- 172.21.15.105:0/1321908878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd40d1650 0x7ffbd40b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:02.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.488+0000 7ffbd98fe640 1 --2- 172.21.15.105:0/1321908878 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ffbb40668d0 0x7ffbb4068d90 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:02.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.488+0000 7ffbd98fe640 1 --2- 172.21.15.105:0/1321908878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbd40c7510 0x7ffbd4153600 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:02.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.488+0000 7ffbd98fe640 1 -- 172.21.15.105:0/1321908878 >> 172.21.15.105:0/1321908878 conn(0x7ffbd409c360 msgr2=0x7ffbd41566f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:02.488 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.488+0000 7ffbd98fe640 1 -- 172.21.15.105:0/1321908878 shutdown_connections 2023-11-28T18:32:02.489 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.488+0000 7ffbd98fe640 1 -- 172.21.15.105:0/1321908878 wait complete. 2023-11-28T18:32:02.508 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T18:32:02.509 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:32:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.648+0000 7f45cced6640 1 Processor -- start 2023-11-28T18:32:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.648+0000 7f45cced6640 1 -- start start 2023-11-28T18:32:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45cced6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f45c80cc760 0x7f45c8164d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45cced6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c80ccc10 con 0x7f45c80caf60 2023-11-28T18:32:02.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45cced6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c806bf50 con 0x7f45c8159f40 2023-11-28T18:32:02.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45cced6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c80cb3c0 con 0x7f45c80cc760 2023-11-28T18:32:02.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45bdd9b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f45c80caf60 0x7f45c80ad850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41756/0 (socket says 172.21.15.105:41756) 2023-11-28T18:32:02.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45bdd9b640 1 -- 172.21.15.105:0/2605116361 learned_addr learned my addr 172.21.15.105:0/2605116361 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:32:02.655 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45c6d9d640 1 --2- 172.21.15.105:0/2605116361 >> v2:172.21.15.148:3300/0 conn(0x7f45c80cc760 0x7f45c8164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:02.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45c5d9b640 1 -- 172.21.15.105:0/2605116361 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 652854734 0 0) 0x7f45c80ccc10 con 0x7f45c80caf60 2023-11-28T18:32:02.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45c5d9b640 1 -- 172.21.15.105:0/2605116361 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45b4003530 con 0x7f45c80caf60 2023-11-28T18:32:02.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45c5d9b640 1 -- 172.21.15.105:0/2605116361 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 772499697 0 0) 0x7f45b4003530 con 0x7f45c80caf60 2023-11-28T18:32:02.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45c5d9b640 1 -- 172.21.15.105:0/2605116361 >> v2:172.21.15.148:3300/0 conn(0x7f45c80cc760 msgr2=0x7f45c8164d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:02.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45c5d9b640 1 --2- 172.21.15.105:0/2605116361 >> v2:172.21.15.148:3300/0 conn(0x7f45c80cc760 0x7f45c8164d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:02.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45c5d9b640 1 -- 172.21.15.105:0/2605116361 >> v1:172.21.15.148:6789/0 conn(0x7f45c8159f40 legacy=0x7f45c80ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:02.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45c5d9b640 1 -- 172.21.15.105:0/2605116361 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c815a320 con 0x7f45c80caf60 2023-11-28T18:32:02.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45c5d9b640 1 -- 172.21.15.105:0/2605116361 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f45b0001a00 con 0x7f45c80caf60 2023-11-28T18:32:02.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45c5d9b640 1 -- 172.21.15.105:0/2605116361 >> v1:172.21.15.105:6789/0 conn(0x7f45c80caf60 legacy=0x7f45c80ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:02.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45c5d9b640 1 --2- 172.21.15.105:0/2605116361 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45b40039d0 0x7f45b4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:02.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45c5d9b640 1 --2- 172.21.15.105:0/2605116361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45b40047d0 0x7f45b4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:02.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45c5d9b640 1 --2- 172.21.15.105:0/2605116361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b40051e0 0x7f45b400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:02.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45c5d9b640 1 -- 172.21.15.105:0/2605116361 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c80cb3c0 con 0x7f45b40047d0 2023-11-28T18:32:02.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45c5d9b640 1 -- 172.21.15.105:0/2605116361 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b400c130 con 0x7f45b40051e0 2023-11-28T18:32:02.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45c5d9b640 1 -- 172.21.15.105:0/2605116361 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b400e500 con 0x7f45b40039d0 2023-11-28T18:32:02.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45bdd9b640 1 --2- 172.21.15.105:0/2605116361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45b40047d0 0x7f45b4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:02.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45c6d9d640 1 --2- 172.21.15.105:0/2605116361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b40051e0 0x7f45b400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:02.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45c659c640 1 --2- 172.21.15.105:0/2605116361 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45b40039d0 0x7f45b4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:02.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45bdd9b640 1 -- 172.21.15.105:0/2605116361 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45b40039d0 msgr2=0x7f45b4003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:02.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45bdd9b640 1 --2- 172.21.15.105:0/2605116361 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45b40039d0 0x7f45b4003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:02.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45bdd9b640 1 -- 172.21.15.105:0/2605116361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b40051e0 msgr2=0x7f45b400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:02.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45bdd9b640 1 --2- 172.21.15.105:0/2605116361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b40051e0 0x7f45b400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:02.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45bdd9b640 1 -- 172.21.15.105:0/2605116361 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b4003530 con 0x7f45b40047d0 2023-11-28T18:32:02.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45c6d9d640 1 -- 172.21.15.105:0/2605116361 reap_dead start 2023-11-28T18:32:02.660 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.652+0000 7f45c659c640 1 --2- 172.21.15.105:0/2605116361 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45b40039d0 0x7f45b4003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:32:02.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.656+0000 7f45bdd9b640 1 --2- 172.21.15.105:0/2605116361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45b40047d0 0x7f45b4004c50 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f45b0002240 tx=0x7f45b001d190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769e6eaadc6f4059 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:02.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.656+0000 7f45c5d9b640 1 -- 172.21.15.105:0/2605116361 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b00ad630 con 0x7f45b40047d0 2023-11-28T18:32:02.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.656+0000 7f45c5d9b640 1 -- 172.21.15.105:0/2605116361 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b00a8070 con 0x7f45b40047d0 2023-11-28T18:32:02.667 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.656+0000 7f45c5d9b640 1 -- 172.21.15.105:0/2605116361 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b00ada50 con 0x7f45b40047d0 2023-11-28T18:32:02.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.656+0000 7f45cced6640 1 -- 172.21.15.105:0/2605116361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45b40047d0 msgr2=0x7f45b4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:02.668 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.656+0000 7f45cced6640 1 --2- 172.21.15.105:0/2605116361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45b40047d0 0x7f45b4004c50 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f45b0002240 tx=0x7f45b001d190 comp rx=0 tx=0).stop 2023-11-28T18:32:02.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.656+0000 7f45cced6640 1 -- 172.21.15.105:0/2605116361 shutdown_connections 2023-11-28T18:32:02.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.656+0000 7f45cced6640 1 --2- 172.21.15.105:0/2605116361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45b40047d0 0x7f45b4004c50 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:02.669 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.656+0000 7f45cced6640 1 -- 172.21.15.105:0/2605116361 >> 172.21.15.105:0/2605116361 conn(0x7f45c80ceb90 msgr2=0x7f45c80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:02.670 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.656+0000 7f45cced6640 1 -- 172.21.15.105:0/2605116361 shutdown_connections 2023-11-28T18:32:02.670 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.656+0000 7f45cced6640 1 -- 172.21.15.105:0/2605116361 wait complete. 2023-11-28T18:32:02.670 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.656+0000 7f45cced6640 1 Processor -- start 2023-11-28T18:32:02.670 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.656+0000 7f45cced6640 1 -- start start 2023-11-28T18:32:02.670 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45cced6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b40047d0 0x7f45c80ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:02.670 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45cced6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45c809ad20 0x7f45c809b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:02.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45cced6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45c809ba40 0x7f45c809bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:02.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45cced6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c815c1f0 con 0x7f45c809ba40 2023-11-28T18:32:02.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45cced6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c815b2f0 con 0x7f45b40047d0 2023-11-28T18:32:02.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45cced6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c815b570 con 0x7f45c809ad20 2023-11-28T18:32:02.671 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45c6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45c809ba40 0x7f45c809bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:02.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45c6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45c809ba40 0x7f45c809bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58474/0 (socket says 172.21.15.105:58474) 2023-11-28T18:32:02.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45c659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b40047d0 0x7f45c80ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:02.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45c6d9d640 1 -- 172.21.15.105:0/2237981892 learned_addr learned my addr 172.21.15.105:0/2237981892 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:02.672 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45bdd9b640 1 --2- 172.21.15.105:0/2237981892 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45c809ad20 0x7f45c809b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:02.673 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45c6d9d640 1 -- 172.21.15.105:0/2237981892 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45c809ad20 msgr2=0x7f45c809b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:02.673 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45c6d9d640 1 --2- 172.21.15.105:0/2237981892 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45c809ad20 0x7f45c809b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:02.674 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45c6d9d640 1 -- 172.21.15.105:0/2237981892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b40047d0 msgr2=0x7f45c80ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:02.674 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45c6d9d640 1 --2- 172.21.15.105:0/2237981892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b40047d0 0x7f45c80ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:02.674 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45c6d9d640 1 -- 172.21.15.105:0/2237981892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c815b850 con 0x7f45c809ba40 2023-11-28T18:32:02.674 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45c6d9d640 1 --2- 172.21.15.105:0/2237981892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45c809ba40 0x7f45c809bec0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f45c80b4e80 tx=0x7f45c0011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:02.674 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45bf7fe640 1 -- 172.21.15.105:0/2237981892 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c0004240 con 0x7f45c809ba40 2023-11-28T18:32:02.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45bf7fe640 1 -- 172.21.15.105:0/2237981892 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c00043a0 con 0x7f45c809ba40 2023-11-28T18:32:02.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45bf7fe640 1 -- 172.21.15.105:0/2237981892 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c000f8d0 con 0x7f45c809ba40 2023-11-28T18:32:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45cced6640 1 -- 172.21.15.105:0/2237981892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c815bb30 con 0x7f45c809ba40 2023-11-28T18:32:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45cced6640 1 -- 172.21.15.105:0/2237981892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c809b6e0 con 0x7f45c809ba40 2023-11-28T18:32:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.660+0000 7f45cced6640 1 -- 172.21.15.105:0/2237981892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c815c330 con 0x7f45c809ba40 2023-11-28T18:32:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.664+0000 7f45bf7fe640 1 -- 172.21.15.105:0/2237981892 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f45c000fa30 con 0x7f45c809ba40 2023-11-28T18:32:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.664+0000 7f45bf7fe640 1 --2- 172.21.15.105:0/2237981892 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f45a80668d0 0x7f45a8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.664+0000 7f45bf7fe640 1 -- 172.21.15.105:0/2237981892 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f45c0084950 con 0x7f45c809ba40 2023-11-28T18:32:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.664+0000 7f45c659c640 1 --2- 172.21.15.105:0/2237981892 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f45a80668d0 0x7f45a8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.664+0000 7f45c659c640 1 --2- 172.21.15.105:0/2237981892 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f45a80668d0 0x7f45a8068d90 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f45b8007fd0 tx=0x7f45b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.668+0000 7f45bf7fe640 1 -- 172.21.15.105:0/2237981892 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f45c815c330 con 0x7f45c809ba40 2023-11-28T18:32:02.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.848+0000 7f45cced6640 1 -- 172.21.15.105:0/2237981892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45c815c330 con 0x7f45c809ba40 2023-11-28T18:32:02.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.848+0000 7f45bf7fe640 1 -- 172.21.15.105:0/2237981892 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) v1 ==== 95+0+4933 (secure 0 0 0) 0x7f45c815c330 con 0x7f45c809ba40 2023-11-28T18:32:02.848 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:32:02.849 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":134,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3161918450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3161918450},{"type":"v1","addr":"172.21.15.105:6835","nonce":3161918450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":129},{"gid":9120,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/3958106803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3958106803},{"type":"v1","addr":"172.21.15.148:6834","nonce":3958106803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":129},{"gid":9123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3706038483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":3706038483},{"type":"v1","addr":"172.21.15.148:6835","nonce":3706038483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":129}],"filesystems":[{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:31:58.827408+0000","modified":"2023-11-28T18:32:01.852953+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9126},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9126":{"gid":9126,"name":"c","rank":0,"incarnation":130,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/1978722810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1978722810},{"type":"v1","addr":"172.21.15.105:6837","nonce":1978722810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-28T18:32:02.849 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 134 2023-11-28T18:32:02.850 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.848+0000 7f45cced6640 1 -- 172.21.15.105:0/2237981892 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f45a80668d0 msgr2=0x7f45a8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:02.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.848+0000 7f45cced6640 1 --2- 172.21.15.105:0/2237981892 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f45a80668d0 0x7f45a8068d90 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f45b8007fd0 tx=0x7f45b8009040 comp rx=0 tx=0).stop 2023-11-28T18:32:02.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.848+0000 7f45cced6640 1 -- 172.21.15.105:0/2237981892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45c809ba40 msgr2=0x7f45c809bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:02.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.848+0000 7f45cced6640 1 --2- 172.21.15.105:0/2237981892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45c809ba40 0x7f45c809bec0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f45c80b4e80 tx=0x7f45c0011cf0 comp rx=0 tx=0).stop 2023-11-28T18:32:02.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.848+0000 7f45cced6640 1 -- 172.21.15.105:0/2237981892 shutdown_connections 2023-11-28T18:32:02.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.848+0000 7f45cced6640 1 --2- 172.21.15.105:0/2237981892 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f45a80668d0 0x7f45a8068d90 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:02.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.848+0000 7f45cced6640 1 --2- 172.21.15.105:0/2237981892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45c809ba40 0x7f45c809bec0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:02.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.848+0000 7f45cced6640 1 --2- 172.21.15.105:0/2237981892 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45c809ad20 0x7f45c809b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:02.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.848+0000 7f45cced6640 1 --2- 172.21.15.105:0/2237981892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b40047d0 0x7f45c80ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:02.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.848+0000 7f45cced6640 1 -- 172.21.15.105:0/2237981892 >> 172.21.15.105:0/2237981892 conn(0x7f45c80ceb90 msgr2=0x7f45c815e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:02.851 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.852+0000 7f45cced6640 1 -- 172.21.15.105:0/2237981892 shutdown_connections 2023-11-28T18:32:02.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:02.852+0000 7f45cced6640 1 -- 172.21.15.105:0/2237981892 wait complete. 2023-11-28T18:32:02.870 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:32:03.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.008+0000 7f2c514fe640 1 Processor -- start 2023-11-28T18:32:03.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.008+0000 7f2c514fe640 1 -- start start 2023-11-28T18:32:03.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.008+0000 7f2c514fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2c4c154070 0x7f2c4c15d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:03.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.008+0000 7f2c514fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c4c0cc7f0 con 0x7f2c4c06bf50 2023-11-28T18:32:03.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.008+0000 7f2c514fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c4c15dd10 con 0x7f2c4c0adb80 2023-11-28T18:32:03.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.008+0000 7f2c514fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c4c072540 con 0x7f2c4c154070 2023-11-28T18:32:03.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.008+0000 7f2c4affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2c4c154070 0x7f2c4c15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:03.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.008+0000 7f2c4affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2c4c154070 0x7f2c4c15d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58480/0 (socket says 172.21.15.105:58480) 2023-11-28T18:32:03.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.008+0000 7f2c427fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f2c4c06bf50 0x7f2c4c0ce310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41762/0 (socket says 172.21.15.105:41762) 2023-11-28T18:32:03.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.008+0000 7f2c4affd640 1 -- 172.21.15.105:0/3008422379 learned_addr learned my addr 172.21.15.105:0/3008422379 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:03.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.008+0000 7f2c4affd640 1 -- 172.21.15.105:0/3008422379 >> v1:172.21.15.148:6789/0 conn(0x7f2c4c0adb80 legacy=0x7f2c4c0acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:03.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.008+0000 7f2c4affd640 1 -- 172.21.15.105:0/3008422379 >> v1:172.21.15.105:6789/0 conn(0x7f2c4c06bf50 legacy=0x7f2c4c0ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:03.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.012+0000 7f2c4affd640 1 -- 172.21.15.105:0/3008422379 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4c15dff0 con 0x7f2c4c154070 2023-11-28T18:32:03.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.008+0000 7f2c4a7fc640 1 -- 172.21.15.105:0/3008422379 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2166961997 0 0) 0x7f2c4c0cc7f0 con 0x7f2c4c06bf50 2023-11-28T18:32:03.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.012+0000 7f2c4affd640 1 --2- 172.21.15.105:0/3008422379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c4c154070 0x7f2c4c15d700 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009870 tx=0x7f2c3c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f53ce4cd0d82d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:03.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.012+0000 7f2c4a7fc640 1 -- 172.21.15.105:0/3008422379 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c0aa670 con 0x7f2c4c154070 2023-11-28T18:32:03.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.012+0000 7f2c4a7fc640 1 -- 172.21.15.105:0/3008422379 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3c0aa7d0 con 0x7f2c4c154070 2023-11-28T18:32:03.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.012+0000 7f2c4a7fc640 1 -- 172.21.15.105:0/3008422379 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c0aab60 con 0x7f2c4c154070 2023-11-28T18:32:03.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.012+0000 7f2c514fe640 1 -- 172.21.15.105:0/3008422379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c4c154070 msgr2=0x7f2c4c15d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:03.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.012+0000 7f2c514fe640 1 --2- 172.21.15.105:0/3008422379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c4c154070 0x7f2c4c15d700 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009870 tx=0x7f2c3c01c510 comp rx=0 tx=0).stop 2023-11-28T18:32:03.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.012+0000 7f2c514fe640 1 -- 172.21.15.105:0/3008422379 shutdown_connections 2023-11-28T18:32:03.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.012+0000 7f2c514fe640 1 --2- 172.21.15.105:0/3008422379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c4c154070 0x7f2c4c15d700 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.012+0000 7f2c514fe640 1 -- 172.21.15.105:0/3008422379 >> 172.21.15.105:0/3008422379 conn(0x7f2c4c0ceb90 msgr2=0x7f2c4c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:03.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.012+0000 7f2c514fe640 1 -- 172.21.15.105:0/3008422379 shutdown_connections 2023-11-28T18:32:03.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.012+0000 7f2c514fe640 1 -- 172.21.15.105:0/3008422379 wait complete. 2023-11-28T18:32:03.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.016+0000 7f2c514fe640 1 Processor -- start 2023-11-28T18:32:03.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.016+0000 7f2c514fe640 1 -- start start 2023-11-28T18:32:03.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.016+0000 7f2c514fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c4c06bf50 0x7f2c4c1474f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:03.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.016+0000 7f2c514fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c4c0adb80 0x7f2c4c1f7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:03.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.016+0000 7f2c514fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c4c154070 0x7f2c4c14af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:03.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.016+0000 7f2c514fe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c1f6ac0 con 0x7f2c4c154070 2023-11-28T18:32:03.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.016+0000 7f2c514fe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c147a30 con 0x7f2c4c0adb80 2023-11-28T18:32:03.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.016+0000 7f2c514fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c0c83b0 con 0x7f2c4c06bf50 2023-11-28T18:32:03.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.016+0000 7f2c4b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c4c154070 0x7f2c4c14af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:03.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.016+0000 7f2c4b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c4c154070 0x7f2c4c14af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58496/0 (socket says 172.21.15.105:58496) 2023-11-28T18:32:03.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.016+0000 7f2c4b7fe640 1 -- 172.21.15.105:0/4065955798 learned_addr learned my addr 172.21.15.105:0/4065955798 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:03.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.016+0000 7f2c4affd640 1 --2- 172.21.15.105:0/4065955798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c4c06bf50 0x7f2c4c1474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:03.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.016+0000 7f2c427fc640 1 --2- 172.21.15.105:0/4065955798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c4c0adb80 0x7f2c4c1f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:03.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.016+0000 7f2c4b7fe640 1 -- 172.21.15.105:0/4065955798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c4c06bf50 msgr2=0x7f2c4c1474f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:03.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.016+0000 7f2c4b7fe640 1 --2- 172.21.15.105:0/4065955798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c4c06bf50 0x7f2c4c1474f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.016+0000 7f2c4b7fe640 1 -- 172.21.15.105:0/4065955798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c4c0adb80 msgr2=0x7f2c4c1f7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:03.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.016+0000 7f2c4b7fe640 1 --2- 172.21.15.105:0/4065955798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c4c0adb80 0x7f2c4c1f7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.016+0000 7f2c4b7fe640 1 -- 172.21.15.105:0/4065955798 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c0094d0 con 0x7f2c4c154070 2023-11-28T18:32:03.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.016+0000 7f2c4b7fe640 1 --2- 172.21.15.105:0/4065955798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c4c154070 0x7f2c4c14af50 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400c430 tx=0x7f2c4400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:03.020 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.016+0000 7f2c43fff640 1 -- 172.21.15.105:0/4065955798 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c44005220 con 0x7f2c4c154070 2023-11-28T18:32:03.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.016+0000 7f2c43fff640 1 -- 172.21.15.105:0/4065955798 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c44005380 con 0x7f2c4c154070 2023-11-28T18:32:03.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.020+0000 7f2c43fff640 1 -- 172.21.15.105:0/4065955798 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c44007270 con 0x7f2c4c154070 2023-11-28T18:32:03.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.020+0000 7f2c514fe640 1 -- 172.21.15.105:0/4065955798 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4c1f6d30 con 0x7f2c4c154070 2023-11-28T18:32:03.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.020+0000 7f2c514fe640 1 -- 172.21.15.105:0/4065955798 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c4c1f7a70 con 0x7f2c4c154070 2023-11-28T18:32:03.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.020+0000 7f2c514fe640 1 -- 172.21.15.105:0/4065955798 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4c160af0 con 0x7f2c4c154070 2023-11-28T18:32:03.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.020+0000 7f2c43fff640 1 -- 172.21.15.105:0/4065955798 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f2c440073d0 con 0x7f2c4c154070 2023-11-28T18:32:03.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.020+0000 7f2c43fff640 1 --2- 172.21.15.105:0/4065955798 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2c2c0668d0 0x7f2c2c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:03.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.020+0000 7f2c43fff640 1 -- 172.21.15.105:0/4065955798 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2c44083a80 con 0x7f2c4c154070 2023-11-28T18:32:03.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.020+0000 7f2c4affd640 1 --2- 172.21.15.105:0/4065955798 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2c2c0668d0 0x7f2c2c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:03.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.020+0000 7f2c4affd640 1 --2- 172.21.15.105:0/4065955798 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2c2c0668d0 0x7f2c2c068d90 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0ae000 tx=0x7f2c3c0b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:03.025 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.024+0000 7f2c43fff640 1 -- 172.21.15.105:0/4065955798 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2c44050ae0 con 0x7f2c4c154070 2023-11-28T18:32:03.093 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:32:03.097 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:32:03.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.176+0000 7f2c514fe640 1 -- 172.21.15.105:0/4065955798 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c4c067570 con 0x7f2c4c154070 2023-11-28T18:32:03.178 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.176+0000 7f2c43fff640 1 -- 172.21.15.105:0/4065955798 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16792 (secure 0 0 0) 0x7f2c44054100 con 0x7f2c4c154070 2023-11-28T18:32:03.178 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:32:03.179 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":129,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:32:00.453226+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:31:56.971692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-28T18:31:57.881657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:32:03.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.180+0000 7f2c514fe640 1 -- 172.21.15.105:0/4065955798 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2c2c0668d0 msgr2=0x7f2c2c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:03.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.180+0000 7f2c514fe640 1 --2- 172.21.15.105:0/4065955798 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2c2c0668d0 0x7f2c2c068d90 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0ae000 tx=0x7f2c3c0b5040 comp rx=0 tx=0).stop 2023-11-28T18:32:03.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.180+0000 7f2c514fe640 1 -- 172.21.15.105:0/4065955798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c4c154070 msgr2=0x7f2c4c14af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:03.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.180+0000 7f2c514fe640 1 --2- 172.21.15.105:0/4065955798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c4c154070 0x7f2c4c14af50 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400c430 tx=0x7f2c4400c830 comp rx=0 tx=0).stop 2023-11-28T18:32:03.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.180+0000 7f2c514fe640 1 -- 172.21.15.105:0/4065955798 shutdown_connections 2023-11-28T18:32:03.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.180+0000 7f2c514fe640 1 --2- 172.21.15.105:0/4065955798 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2c2c0668d0 0x7f2c2c068d90 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.180+0000 7f2c514fe640 1 --2- 172.21.15.105:0/4065955798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c4c154070 0x7f2c4c14af50 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.180+0000 7f2c514fe640 1 --2- 172.21.15.105:0/4065955798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c4c0adb80 0x7f2c4c1f7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.180+0000 7f2c514fe640 1 --2- 172.21.15.105:0/4065955798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2c4c06bf50 0x7f2c4c1474f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.180+0000 7f2c514fe640 1 -- 172.21.15.105:0/4065955798 >> 172.21.15.105:0/4065955798 conn(0x7f2c4c0ceb90 msgr2=0x7f2c4c0ac5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:03.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.180+0000 7f2c514fe640 1 -- 172.21.15.105:0/4065955798 shutdown_connections 2023-11-28T18:32:03.182 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.180+0000 7f2c514fe640 1 -- 172.21.15.105:0/4065955798 wait complete. 2023-11-28T18:32:03.200 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T18:32:03.200 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:32:03.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12e4d0f640 1 Processor -- start 2023-11-28T18:32:03.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12e4d0f640 1 -- start start 2023-11-28T18:32:03.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12e4d0f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f12e015d4f0 0x7f12e00d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:03.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12e4d0f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12e00d3c30 0x7f12e00d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:03.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12e4d0f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12e00b25b0 con 0x7f12e00d1890 2023-11-28T18:32:03.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12e4d0f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e00b2900 con 0x7f12e015d4f0 2023-11-28T18:32:03.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12e4d0f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e00cea30 con 0x7f12e00d3c30 2023-11-28T18:32:03.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12ded9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f12e00d1890 0x7f12e00cfa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41768/0 (socket says 172.21.15.105:41768) 2023-11-28T18:32:03.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12ded9d640 1 -- 172.21.15.105:0/1935976804 learned_addr learned my addr 172.21.15.105:0/1935976804 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:32:03.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12de59c640 1 --2- 172.21.15.105:0/1935976804 >> v2:172.21.15.148:3300/0 conn(0x7f12e015d4f0 0x7f12e00d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:03.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12d5d9b640 1 --2- 172.21.15.105:0/1935976804 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12e00d3c30 0x7f12e00d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:03.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12ddd9b640 1 -- 172.21.15.105:0/1935976804 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 238300878 0 0) 0x7f12e00b25b0 con 0x7f12e00d1890 2023-11-28T18:32:03.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12ddd9b640 1 -- 172.21.15.105:0/1935976804 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12c0003530 con 0x7f12e00d1890 2023-11-28T18:32:03.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12ddd9b640 1 -- 172.21.15.105:0/1935976804 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3412816370 0 0) 0x7f12c0003530 con 0x7f12e00d1890 2023-11-28T18:32:03.387 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12ddd9b640 1 -- 172.21.15.105:0/1935976804 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12e00d3c30 msgr2=0x7f12e00d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:03.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12ddd9b640 1 --2- 172.21.15.105:0/1935976804 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12e00d3c30 0x7f12e00d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12ddd9b640 1 -- 172.21.15.105:0/1935976804 >> v2:172.21.15.148:3300/0 conn(0x7f12e015d4f0 msgr2=0x7f12e00d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:03.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12ddd9b640 1 --2- 172.21.15.105:0/1935976804 >> v2:172.21.15.148:3300/0 conn(0x7f12e015d4f0 0x7f12e00d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12ddd9b640 1 -- 172.21.15.105:0/1935976804 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e00ced10 con 0x7f12e00d1890 2023-11-28T18:32:03.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12ddd9b640 1 -- 172.21.15.105:0/1935976804 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f12d8001a00 con 0x7f12e00d1890 2023-11-28T18:32:03.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12de59c640 1 --2- 172.21.15.105:0/1935976804 >> v2:172.21.15.148:3300/0 conn(0x7f12e015d4f0 0x7f12e00d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:32:03.389 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12d5d9b640 1 --2- 172.21.15.105:0/1935976804 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12e00d3c30 0x7f12e00d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:32:03.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12ddd9b640 1 -- 172.21.15.105:0/1935976804 >> v1:172.21.15.105:6789/0 conn(0x7f12e00d1890 legacy=0x7f12e00cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:03.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12ddd9b640 1 --2- 172.21.15.105:0/1935976804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12c0003980 0x7f12c0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:03.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12ddd9b640 1 --2- 172.21.15.105:0/1935976804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c00047d0 0x7f12c0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:03.390 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12ddd9b640 1 --2- 172.21.15.105:0/1935976804 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12e00d3c30 0x7f12c0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:03.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12ddd9b640 1 -- 172.21.15.105:0/1935976804 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e00b2900 con 0x7f12c0003980 2023-11-28T18:32:03.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12ddd9b640 1 -- 172.21.15.105:0/1935976804 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e00cea30 con 0x7f12c00047d0 2023-11-28T18:32:03.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12ddd9b640 1 -- 172.21.15.105:0/1935976804 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c000b990 con 0x7f12e00d3c30 2023-11-28T18:32:03.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12de59c640 1 --2- 172.21.15.105:0/1935976804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12c0003980 0x7f12c0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:03.391 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12d5d9b640 1 --2- 172.21.15.105:0/1935976804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c00047d0 0x7f12c0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:03.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.380+0000 7f12ded9d640 1 --2- 172.21.15.105:0/1935976804 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12e00d3c30 0x7f12c0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:03.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.384+0000 7f12de59c640 1 -- 172.21.15.105:0/1935976804 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12e00d3c30 msgr2=0x7f12c0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:03.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.384+0000 7f12de59c640 1 --2- 172.21.15.105:0/1935976804 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12e00d3c30 0x7f12c0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.384+0000 7f12de59c640 1 -- 172.21.15.105:0/1935976804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c00047d0 msgr2=0x7f12c0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:03.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.384+0000 7f12de59c640 1 --2- 172.21.15.105:0/1935976804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c00047d0 0x7f12c0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.384+0000 7f12de59c640 1 -- 172.21.15.105:0/1935976804 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c8009420 con 0x7f12c0003980 2023-11-28T18:32:03.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.384+0000 7f12d5d9b640 1 --2- 172.21.15.105:0/1935976804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c00047d0 0x7f12c0004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:32:03.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.384+0000 7f12de59c640 1 --2- 172.21.15.105:0/1935976804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12c0003980 0x7f12c0003da0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009cb0 tx=0x7f12d001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5abd4a17189738 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:03.392 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.384+0000 7f12ddd9b640 1 -- 172.21.15.105:0/1935976804 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d00a5710 con 0x7f12c0003980 2023-11-28T18:32:03.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.384+0000 7f12ddd9b640 1 -- 172.21.15.105:0/1935976804 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d00a5870 con 0x7f12c0003980 2023-11-28T18:32:03.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.384+0000 7f12ddd9b640 1 -- 172.21.15.105:0/1935976804 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d00a5be0 con 0x7f12c0003980 2023-11-28T18:32:03.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.384+0000 7f12e4d0f640 1 -- 172.21.15.105:0/1935976804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12c0003980 msgr2=0x7f12c0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:03.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.384+0000 7f12e4d0f640 1 --2- 172.21.15.105:0/1935976804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12c0003980 0x7f12c0003da0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009cb0 tx=0x7f12d001cc40 comp rx=0 tx=0).stop 2023-11-28T18:32:03.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.384+0000 7f12ded9d640 1 -- 172.21.15.105:0/1935976804 reap_dead start 2023-11-28T18:32:03.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.384+0000 7f12e4d0f640 1 -- 172.21.15.105:0/1935976804 shutdown_connections 2023-11-28T18:32:03.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.384+0000 7f12e4d0f640 1 -- 172.21.15.105:0/1935976804 >> 172.21.15.105:0/1935976804 conn(0x7f12e00d3030 msgr2=0x7f12e00d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:03.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.384+0000 7f12e4d0f640 1 -- 172.21.15.105:0/1935976804 shutdown_connections 2023-11-28T18:32:03.393 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.388+0000 7f12e4d0f640 1 -- 172.21.15.105:0/1935976804 wait complete. 2023-11-28T18:32:03.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.388+0000 7f12e4d0f640 1 Processor -- start 2023-11-28T18:32:03.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.388+0000 7f12e4d0f640 1 -- start start 2023-11-28T18:32:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.388+0000 7f12e4d0f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12c0003980 0x7f12e00b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.388+0000 7f12e4d0f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12e014e4e0 0x7f12e00c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.388+0000 7f12e4d0f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12e00c9a20 0x7f12e0159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.388+0000 7f12e4d0f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e00c9ea0 con 0x7f12e014e4e0 2023-11-28T18:32:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.388+0000 7f12e4d0f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e01538b0 con 0x7f12e00c9a20 2023-11-28T18:32:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.388+0000 7f12e4d0f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e0153b90 con 0x7f12c0003980 2023-11-28T18:32:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.388+0000 7f12d5d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12e014e4e0 0x7f12e00c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.388+0000 7f12d5d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12e014e4e0 0x7f12e00c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58512/0 (socket says 172.21.15.105:58512) 2023-11-28T18:32:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.388+0000 7f12d5d9b640 1 -- 172.21.15.105:0/461491576 learned_addr learned my addr 172.21.15.105:0/461491576 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.388+0000 7f12de59c640 1 --2- 172.21.15.105:0/461491576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12c0003980 0x7f12e00b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.388+0000 7f12ded9d640 1 --2- 172.21.15.105:0/461491576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12e00c9a20 0x7f12e0159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.388+0000 7f12d5d9b640 1 -- 172.21.15.105:0/461491576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12c0003980 msgr2=0x7f12e00b5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.388+0000 7f12d5d9b640 1 --2- 172.21.15.105:0/461491576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12c0003980 0x7f12e00b5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.388+0000 7f12d5d9b640 1 -- 172.21.15.105:0/461491576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12e00c9a20 msgr2=0x7f12e0159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.388+0000 7f12d5d9b640 1 --2- 172.21.15.105:0/461491576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12e00c9a20 0x7f12e0159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.388+0000 7f12d5d9b640 1 -- 172.21.15.105:0/461491576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e0153e70 con 0x7f12e014e4e0 2023-11-28T18:32:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.388+0000 7f12d5d9b640 1 --2- 172.21.15.105:0/461491576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12e014e4e0 0x7f12e00c94e0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f12c80118d0 tx=0x7f12c8011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.392+0000 7f12d77fe640 1 -- 172.21.15.105:0/461491576 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c800fb50 con 0x7f12e014e4e0 2023-11-28T18:32:03.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.392+0000 7f12d77fe640 1 -- 172.21.15.105:0/461491576 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c800fcb0 con 0x7f12e014e4e0 2023-11-28T18:32:03.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.392+0000 7f12d77fe640 1 -- 172.21.15.105:0/461491576 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c8011ec0 con 0x7f12e014e4e0 2023-11-28T18:32:03.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.392+0000 7f12e4d0f640 1 -- 172.21.15.105:0/461491576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e0154150 con 0x7f12e014e4e0 2023-11-28T18:32:03.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.392+0000 7f12e4d0f640 1 -- 172.21.15.105:0/461491576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e014e960 con 0x7f12e014e4e0 2023-11-28T18:32:03.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.392+0000 7f12e4d0f640 1 -- 172.21.15.105:0/461491576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e0067570 con 0x7f12e014e4e0 2023-11-28T18:32:03.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.392+0000 7f12d77fe640 1 -- 172.21.15.105:0/461491576 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f12c8015460 con 0x7f12e014e4e0 2023-11-28T18:32:03.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.392+0000 7f12d77fe640 1 --2- 172.21.15.105:0/461491576 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f12c4066900 0x7f12c4068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:03.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.392+0000 7f12d77fe640 1 -- 172.21.15.105:0/461491576 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f12c8084de0 con 0x7f12e014e4e0 2023-11-28T18:32:03.399 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.392+0000 7f12de59c640 1 --2- 172.21.15.105:0/461491576 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f12c4066900 0x7f12c4068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:03.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.396+0000 7f12de59c640 1 --2- 172.21.15.105:0/461491576 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f12c4066900 0x7f12c4068dc0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f12d00af000 tx=0x7f12d00046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:03.400 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.396+0000 7f12d77fe640 1 -- 172.21.15.105:0/461491576 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f12c8051ee0 con 0x7f12e014e4e0 2023-11-28T18:32:03.548 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.548+0000 7f12e4d0f640 1 -- 172.21.15.105:0/461491576 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f12e0153d00 con 0x7f12c4066900 2023-11-28T18:32:03.556 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.556+0000 7f12d77fe640 1 -- 172.21.15.105:0/461491576 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289657 (secure 0 0 0) 0x7f12e0153d00 con 0x7f12c4066900 2023-11-28T18:32:03.559 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:32:03.561 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:32:03.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.560+0000 7f12e4d0f640 1 -- 172.21.15.105:0/461491576 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f12c4066900 msgr2=0x7f12c4068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:03.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.560+0000 7f12e4d0f640 1 --2- 172.21.15.105:0/461491576 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f12c4066900 0x7f12c4068dc0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f12d00af000 tx=0x7f12d00046d0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.562 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.560+0000 7f12e4d0f640 1 -- 172.21.15.105:0/461491576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12e014e4e0 msgr2=0x7f12e00c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:03.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.560+0000 7f12e4d0f640 1 --2- 172.21.15.105:0/461491576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12e014e4e0 0x7f12e00c94e0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f12c80118d0 tx=0x7f12c8011cd0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.560+0000 7f12e4d0f640 1 -- 172.21.15.105:0/461491576 shutdown_connections 2023-11-28T18:32:03.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.560+0000 7f12e4d0f640 1 --2- 172.21.15.105:0/461491576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12e00c9a20 0x7f12e0159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.560+0000 7f12e4d0f640 1 --2- 172.21.15.105:0/461491576 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f12c4066900 0x7f12c4068dc0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.563 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.560+0000 7f12e4d0f640 1 --2- 172.21.15.105:0/461491576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12e014e4e0 0x7f12e00c94e0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.560+0000 7f12e4d0f640 1 --2- 172.21.15.105:0/461491576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12c0003980 0x7f12e00b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.560+0000 7f12e4d0f640 1 -- 172.21.15.105:0/461491576 >> 172.21.15.105:0/461491576 conn(0x7f12e00d3030 msgr2=0x7f12e015c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:03.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.560+0000 7f12e4d0f640 1 -- 172.21.15.105:0/461491576 shutdown_connections 2023-11-28T18:32:03.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.560+0000 7f12e4d0f640 1 -- 172.21.15.105:0/461491576 wait complete. 2023-11-28T18:32:03.579 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2023-11-28T18:32:03.379889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":410372,"kb_used_data":13772,"kb_used_omap":12,"kb_used_meta":396467,"kb_avail":754564348,"statfs":{"total":773094113280,"available":772673892352,"internally_reserved":0,"allocated":14102528,"data_stored":7173360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":405982770},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.940961"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.875396+0000","last_change":"2023-11-28T18:31:59.482608+0000","last_active":"2023-11-28T18:32:00.875396+0000","last_peered":"2023-11-28T18:32:00.875396+0000","last_clean":"2023-11-28T18:32:00.875396+0000","last_became_active":"2023-11-28T18:31:59.482352+0000","last_became_peered":"2023-11-28T18:31:59.482352+0000","last_unstale":"2023-11-28T18:32:00.875396+0000","last_undegraded":"2023-11-28T18:32:00.875396+0000","last_fullsized":"2023-11-28T18:32:00.875396+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:23:03.934139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877742+0000","last_change":"2023-11-28T18:31:59.487336+0000","last_active":"2023-11-28T18:32:00.877742+0000","last_peered":"2023-11-28T18:32:00.877742+0000","last_clean":"2023-11-28T18:32:00.877742+0000","last_became_active":"2023-11-28T18:31:59.486133+0000","last_became_peered":"2023-11-28T18:31:59.486133+0000","last_unstale":"2023-11-28T18:32:00.877742+0000","last_undegraded":"2023-11-28T18:32:00.877742+0000","last_fullsized":"2023-11-28T18:32:00.877742+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:17.025975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460893+0000","last_change":"2023-11-28T18:31:59.467470+0000","last_active":"2023-11-28T18:32:00.460893+0000","last_peered":"2023-11-28T18:32:00.460893+0000","last_clean":"2023-11-28T18:32:00.460893+0000","last_became_active":"2023-11-28T18:31:59.467185+0000","last_became_peered":"2023-11-28T18:31:59.467185+0000","last_unstale":"2023-11-28T18:32:00.460893+0000","last_undegraded":"2023-11-28T18:32:00.460893+0000","last_fullsized":"2023-11-28T18:32:00.460893+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:18:53.760883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877500+0000","last_change":"2023-11-28T18:31:59.489101+0000","last_active":"2023-11-28T18:32:00.877500+0000","last_peered":"2023-11-28T18:32:00.877500+0000","last_clean":"2023-11-28T18:32:00.877500+0000","last_became_active":"2023-11-28T18:31:59.488768+0000","last_became_peered":"2023-11-28T18:31:59.488768+0000","last_unstale":"2023-11-28T18:32:00.877500+0000","last_undegraded":"2023-11-28T18:32:00.877500+0000","last_fullsized":"2023-11-28T18:32:00.877500+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:06:15.465659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.457918+0000","last_change":"2023-11-28T18:31:59.461449+0000","last_active":"2023-11-28T18:32:00.457918+0000","last_peered":"2023-11-28T18:32:00.457918+0000","last_clean":"2023-11-28T18:32:00.457918+0000","last_became_active":"2023-11-28T18:31:59.461161+0000","last_became_peered":"2023-11-28T18:31:59.461161+0000","last_unstale":"2023-11-28T18:32:00.457918+0000","last_undegraded":"2023-11-28T18:32:00.457918+0000","last_fullsized":"2023-11-28T18:32:00.457918+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:02.710984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459584+0000","last_change":"2023-11-28T18:31:59.474620+0000","last_active":"2023-11-28T18:32:00.459584+0000","last_peered":"2023-11-28T18:32:00.459584+0000","last_clean":"2023-11-28T18:32:00.459584+0000","last_became_active":"2023-11-28T18:31:59.474317+0000","last_became_peered":"2023-11-28T18:31:59.474317+0000","last_unstale":"2023-11-28T18:32:00.459584+0000","last_undegraded":"2023-11-28T18:32:00.459584+0000","last_fullsized":"2023-11-28T18:32:00.459584+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:50:40.320398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876129+0000","last_change":"2023-11-28T18:31:59.497301+0000","last_active":"2023-11-28T18:32:00.876129+0000","last_peered":"2023-11-28T18:32:00.876129+0000","last_clean":"2023-11-28T18:32:00.876129+0000","last_became_active":"2023-11-28T18:31:59.496974+0000","last_became_peered":"2023-11-28T18:31:59.496974+0000","last_unstale":"2023-11-28T18:32:00.876129+0000","last_undegraded":"2023-11-28T18:32:00.876129+0000","last_fullsized":"2023-11-28T18:32:00.876129+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:14:51.587300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.875993+0000","last_change":"2023-11-28T18:31:59.483580+0000","last_active":"2023-11-28T18:32:00.875993+0000","last_peered":"2023-11-28T18:32:00.875993+0000","last_clean":"2023-11-28T18:32:00.875993+0000","last_became_active":"2023-11-28T18:31:59.483263+0000","last_became_peered":"2023-11-28T18:31:59.483263+0000","last_unstale":"2023-11-28T18:32:00.875993+0000","last_undegraded":"2023-11-28T18:32:00.875993+0000","last_fullsized":"2023-11-28T18:32:00.875993+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:35:25.019756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459930+0000","last_change":"2023-11-28T18:31:59.477152+0000","last_active":"2023-11-28T18:32:00.459930+0000","last_peered":"2023-11-28T18:32:00.459930+0000","last_clean":"2023-11-28T18:32:00.459930+0000","last_became_active":"2023-11-28T18:31:59.476464+0000","last_became_peered":"2023-11-28T18:31:59.476464+0000","last_unstale":"2023-11-28T18:32:00.459930+0000","last_undegraded":"2023-11-28T18:32:00.459930+0000","last_fullsized":"2023-11-28T18:32:00.459930+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:03.036695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.875086+0000","last_change":"2023-11-28T18:31:59.479082+0000","last_active":"2023-11-28T18:32:00.875086+0000","last_peered":"2023-11-28T18:32:00.875086+0000","last_clean":"2023-11-28T18:32:00.875086+0000","last_became_active":"2023-11-28T18:31:59.478786+0000","last_became_peered":"2023-11-28T18:31:59.478786+0000","last_unstale":"2023-11-28T18:32:00.875086+0000","last_undegraded":"2023-11-28T18:32:00.875086+0000","last_fullsized":"2023-11-28T18:32:00.875086+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:24:27.946466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877831+0000","last_change":"2023-11-28T18:31:59.489605+0000","last_active":"2023-11-28T18:32:00.877831+0000","last_peered":"2023-11-28T18:32:00.877831+0000","last_clean":"2023-11-28T18:32:00.877831+0000","last_became_active":"2023-11-28T18:31:59.489340+0000","last_became_peered":"2023-11-28T18:31:59.489340+0000","last_unstale":"2023-11-28T18:32:00.877831+0000","last_undegraded":"2023-11-28T18:32:00.877831+0000","last_fullsized":"2023-11-28T18:32:00.877831+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:29:27.272539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.875294+0000","last_change":"2023-11-28T18:31:59.476827+0000","last_active":"2023-11-28T18:32:00.875294+0000","last_peered":"2023-11-28T18:32:00.875294+0000","last_clean":"2023-11-28T18:32:00.875294+0000","last_became_active":"2023-11-28T18:31:59.475744+0000","last_became_peered":"2023-11-28T18:31:59.475744+0000","last_unstale":"2023-11-28T18:32:00.875294+0000","last_undegraded":"2023-11-28T18:32:00.875294+0000","last_fullsized":"2023-11-28T18:32:00.875294+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:05.071170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459763+0000","last_change":"2023-11-28T18:31:59.474829+0000","last_active":"2023-11-28T18:32:00.459763+0000","last_peered":"2023-11-28T18:32:00.459763+0000","last_clean":"2023-11-28T18:32:00.459763+0000","last_became_active":"2023-11-28T18:31:59.474547+0000","last_became_peered":"2023-11-28T18:31:59.474547+0000","last_unstale":"2023-11-28T18:32:00.459763+0000","last_undegraded":"2023-11-28T18:32:00.459763+0000","last_fullsized":"2023-11-28T18:32:00.459763+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:34:42.082662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.874837+0000","last_change":"2023-11-28T18:31:59.485274+0000","last_active":"2023-11-28T18:32:00.874837+0000","last_peered":"2023-11-28T18:32:00.874837+0000","last_clean":"2023-11-28T18:32:00.874837+0000","last_became_active":"2023-11-28T18:31:59.484477+0000","last_became_peered":"2023-11-28T18:31:59.484477+0000","last_unstale":"2023-11-28T18:32:00.874837+0000","last_undegraded":"2023-11-28T18:32:00.874837+0000","last_fullsized":"2023-11-28T18:32:00.874837+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:08:01.303585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.874566+0000","last_change":"2023-11-28T18:31:59.482115+0000","last_active":"2023-11-28T18:32:00.874566+0000","last_peered":"2023-11-28T18:32:00.874566+0000","last_clean":"2023-11-28T18:32:00.874566+0000","last_became_active":"2023-11-28T18:31:59.481654+0000","last_became_peered":"2023-11-28T18:31:59.481654+0000","last_unstale":"2023-11-28T18:32:00.874566+0000","last_undegraded":"2023-11-28T18:32:00.874566+0000","last_fullsized":"2023-11-28T18:32:00.874566+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:52:02.359432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459421+0000","last_change":"2023-11-28T18:31:59.468289+0000","last_active":"2023-11-28T18:32:00.459421+0000","last_peered":"2023-11-28T18:32:00.459421+0000","last_clean":"2023-11-28T18:32:00.459421+0000","last_became_active":"2023-11-28T18:31:59.467344+0000","last_became_peered":"2023-11-28T18:31:59.467344+0000","last_unstale":"2023-11-28T18:32:00.459421+0000","last_undegraded":"2023-11-28T18:32:00.459421+0000","last_fullsized":"2023-11-28T18:32:00.459421+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:17.025975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459936+0000","last_change":"2023-11-28T18:31:59.464786+0000","last_active":"2023-11-28T18:32:00.459936+0000","last_peered":"2023-11-28T18:32:00.459936+0000","last_clean":"2023-11-28T18:32:00.459936+0000","last_became_active":"2023-11-28T18:31:59.464533+0000","last_became_peered":"2023-11-28T18:31:59.464533+0000","last_unstale":"2023-11-28T18:32:00.459936+0000","last_undegraded":"2023-11-28T18:32:00.459936+0000","last_fullsized":"2023-11-28T18:32:00.459936+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:12:03.671740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.458141+0000","last_change":"2023-11-28T18:31:59.461521+0000","last_active":"2023-11-28T18:32:00.458141+0000","last_peered":"2023-11-28T18:32:00.458141+0000","last_clean":"2023-11-28T18:32:00.458141+0000","last_became_active":"2023-11-28T18:31:59.461150+0000","last_became_peered":"2023-11-28T18:31:59.461150+0000","last_unstale":"2023-11-28T18:32:00.458141+0000","last_undegraded":"2023-11-28T18:32:00.458141+0000","last_fullsized":"2023-11-28T18:32:00.458141+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:30.445631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877461+0000","last_change":"2023-11-28T18:31:59.484931+0000","last_active":"2023-11-28T18:32:00.877461+0000","last_peered":"2023-11-28T18:32:00.877461+0000","last_clean":"2023-11-28T18:32:00.877461+0000","last_became_active":"2023-11-28T18:31:59.484652+0000","last_became_peered":"2023-11-28T18:31:59.484652+0000","last_unstale":"2023-11-28T18:32:00.877461+0000","last_undegraded":"2023-11-28T18:32:00.877461+0000","last_fullsized":"2023-11-28T18:32:00.877461+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:31:42.639872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.457911+0000","last_change":"2023-11-28T18:31:59.469465+0000","last_active":"2023-11-28T18:32:00.457911+0000","last_peered":"2023-11-28T18:32:00.457911+0000","last_clean":"2023-11-28T18:32:00.457911+0000","last_became_active":"2023-11-28T18:31:59.469206+0000","last_became_peered":"2023-11-28T18:31:59.469206+0000","last_unstale":"2023-11-28T18:32:00.457911+0000","last_undegraded":"2023-11-28T18:32:00.457911+0000","last_fullsized":"2023-11-28T18:32:00.457911+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:12:03.671740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877252+0000","last_change":"2023-11-28T18:31:59.487946+0000","last_active":"2023-11-28T18:32:00.877252+0000","last_peered":"2023-11-28T18:32:00.877252+0000","last_clean":"2023-11-28T18:32:00.877252+0000","last_became_active":"2023-11-28T18:31:59.487660+0000","last_became_peered":"2023-11-28T18:31:59.487660+0000","last_unstale":"2023-11-28T18:32:00.877252+0000","last_undegraded":"2023-11-28T18:32:00.877252+0000","last_fullsized":"2023-11-28T18:32:00.877252+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:07:10.812548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.874627+0000","last_change":"2023-11-28T18:31:59.477083+0000","last_active":"2023-11-28T18:32:00.874627+0000","last_peered":"2023-11-28T18:32:00.874627+0000","last_clean":"2023-11-28T18:32:00.874627+0000","last_became_active":"2023-11-28T18:31:59.476782+0000","last_became_peered":"2023-11-28T18:31:59.476782+0000","last_unstale":"2023-11-28T18:32:00.874627+0000","last_undegraded":"2023-11-28T18:32:00.874627+0000","last_fullsized":"2023-11-28T18:32:00.874627+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:47:03.620849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876803+0000","last_change":"2023-11-28T18:31:59.487498+0000","last_active":"2023-11-28T18:32:00.876803+0000","last_peered":"2023-11-28T18:32:00.876803+0000","last_clean":"2023-11-28T18:32:00.876803+0000","last_became_active":"2023-11-28T18:31:59.487193+0000","last_became_peered":"2023-11-28T18:31:59.487193+0000","last_unstale":"2023-11-28T18:32:00.876803+0000","last_undegraded":"2023-11-28T18:32:00.876803+0000","last_fullsized":"2023-11-28T18:32:00.876803+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:15:07.840277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459553+0000","last_change":"2023-11-28T18:31:59.475913+0000","last_active":"2023-11-28T18:32:00.459553+0000","last_peered":"2023-11-28T18:32:00.459553+0000","last_clean":"2023-11-28T18:32:00.459553+0000","last_became_active":"2023-11-28T18:31:59.475635+0000","last_became_peered":"2023-11-28T18:31:59.475635+0000","last_unstale":"2023-11-28T18:32:00.459553+0000","last_undegraded":"2023-11-28T18:32:00.459553+0000","last_fullsized":"2023-11-28T18:32:00.459553+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:29:27.272539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460844+0000","last_change":"2023-11-28T18:31:59.474749+0000","last_active":"2023-11-28T18:32:00.460844+0000","last_peered":"2023-11-28T18:32:00.460844+0000","last_clean":"2023-11-28T18:32:00.460844+0000","last_became_active":"2023-11-28T18:31:59.474471+0000","last_became_peered":"2023-11-28T18:31:59.474471+0000","last_unstale":"2023-11-28T18:32:00.460844+0000","last_undegraded":"2023-11-28T18:32:00.460844+0000","last_fullsized":"2023-11-28T18:32:00.460844+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:15:07.840277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460930+0000","last_change":"2023-11-28T18:31:59.480362+0000","last_active":"2023-11-28T18:32:00.460930+0000","last_peered":"2023-11-28T18:32:00.460930+0000","last_clean":"2023-11-28T18:32:00.460930+0000","last_became_active":"2023-11-28T18:31:59.480066+0000","last_became_peered":"2023-11-28T18:31:59.480066+0000","last_unstale":"2023-11-28T18:32:00.460930+0000","last_undegraded":"2023-11-28T18:32:00.460930+0000","last_fullsized":"2023-11-28T18:32:00.460930+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:06:15.465659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877478+0000","last_change":"2023-11-28T18:31:59.484372+0000","last_active":"2023-11-28T18:32:00.877478+0000","last_peered":"2023-11-28T18:32:00.877478+0000","last_clean":"2023-11-28T18:32:00.877478+0000","last_became_active":"2023-11-28T18:31:59.484084+0000","last_became_peered":"2023-11-28T18:31:59.484084+0000","last_unstale":"2023-11-28T18:32:00.877478+0000","last_undegraded":"2023-11-28T18:32:00.877478+0000","last_fullsized":"2023-11-28T18:32:00.877478+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:50:28.561689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.874869+0000","last_change":"2023-11-28T18:31:59.473971+0000","last_active":"2023-11-28T18:32:00.874869+0000","last_peered":"2023-11-28T18:32:00.874869+0000","last_clean":"2023-11-28T18:32:00.874869+0000","last_became_active":"2023-11-28T18:31:59.473711+0000","last_became_peered":"2023-11-28T18:31:59.473711+0000","last_unstale":"2023-11-28T18:32:00.874869+0000","last_undegraded":"2023-11-28T18:32:00.874869+0000","last_fullsized":"2023-11-28T18:32:00.874869+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:59:50.018736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460520+0000","last_change":"2023-11-28T18:31:59.472923+0000","last_active":"2023-11-28T18:32:00.460520+0000","last_peered":"2023-11-28T18:32:00.460520+0000","last_clean":"2023-11-28T18:32:00.460520+0000","last_became_active":"2023-11-28T18:31:59.472516+0000","last_became_peered":"2023-11-28T18:31:59.472516+0000","last_unstale":"2023-11-28T18:32:00.460520+0000","last_undegraded":"2023-11-28T18:32:00.460520+0000","last_fullsized":"2023-11-28T18:32:00.460520+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:01:47.297968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460787+0000","last_change":"2023-11-28T18:31:59.476734+0000","last_active":"2023-11-28T18:32:00.460787+0000","last_peered":"2023-11-28T18:32:00.460787+0000","last_clean":"2023-11-28T18:32:00.460787+0000","last_became_active":"2023-11-28T18:31:59.476120+0000","last_became_peered":"2023-11-28T18:31:59.476120+0000","last_unstale":"2023-11-28T18:32:00.460787+0000","last_undegraded":"2023-11-28T18:32:00.460787+0000","last_fullsized":"2023-11-28T18:32:00.460787+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:21:06.605370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876107+0000","last_change":"2023-11-28T18:31:59.467896+0000","last_active":"2023-11-28T18:32:00.876107+0000","last_peered":"2023-11-28T18:32:00.876107+0000","last_clean":"2023-11-28T18:32:00.876107+0000","last_became_active":"2023-11-28T18:31:59.467567+0000","last_became_peered":"2023-11-28T18:31:59.467567+0000","last_unstale":"2023-11-28T18:32:00.876107+0000","last_undegraded":"2023-11-28T18:32:00.876107+0000","last_fullsized":"2023-11-28T18:32:00.876107+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:51:51.574894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.874430+0000","last_change":"2023-11-28T18:31:59.485125+0000","last_active":"2023-11-28T18:32:00.874430+0000","last_peered":"2023-11-28T18:32:00.874430+0000","last_clean":"2023-11-28T18:32:00.874430+0000","last_became_active":"2023-11-28T18:31:59.484787+0000","last_became_peered":"2023-11-28T18:31:59.484787+0000","last_unstale":"2023-11-28T18:32:00.874430+0000","last_undegraded":"2023-11-28T18:32:00.874430+0000","last_fullsized":"2023-11-28T18:32:00.874430+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:19:15.699737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876236+0000","last_change":"2023-11-28T18:31:59.476809+0000","last_active":"2023-11-28T18:32:00.876236+0000","last_peered":"2023-11-28T18:32:00.876236+0000","last_clean":"2023-11-28T18:32:00.876236+0000","last_became_active":"2023-11-28T18:31:59.476498+0000","last_became_peered":"2023-11-28T18:31:59.476498+0000","last_unstale":"2023-11-28T18:32:00.876236+0000","last_undegraded":"2023-11-28T18:32:00.876236+0000","last_fullsized":"2023-11-28T18:32:00.876236+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:37:13.925200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"128'4","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460805+0000","last_change":"2023-11-28T18:31:58.463033+0000","last_active":"2023-11-28T18:32:00.460805+0000","last_peered":"2023-11-28T18:32:00.460805+0000","last_clean":"2023-11-28T18:32:00.460805+0000","last_became_active":"2023-11-28T18:31:58.462622+0000","last_became_peered":"2023-11-28T18:31:58.462622+0000","last_unstale":"2023-11-28T18:32:00.460805+0000","last_undegraded":"2023-11-28T18:32:00.460805+0000","last_fullsized":"2023-11-28T18:32:00.460805+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:55:33.705881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.874787+0000","last_change":"2023-11-28T18:31:59.479323+0000","last_active":"2023-11-28T18:32:00.874787+0000","last_peered":"2023-11-28T18:32:00.874787+0000","last_clean":"2023-11-28T18:32:00.874787+0000","last_became_active":"2023-11-28T18:31:59.479107+0000","last_became_peered":"2023-11-28T18:31:59.479107+0000","last_unstale":"2023-11-28T18:32:00.874787+0000","last_undegraded":"2023-11-28T18:32:00.874787+0000","last_fullsized":"2023-11-28T18:32:00.874787+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:18:44.496924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"128'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.458252+0000","last_change":"2023-11-28T18:31:58.458546+0000","last_active":"2023-11-28T18:32:00.458252+0000","last_peered":"2023-11-28T18:32:00.458252+0000","last_clean":"2023-11-28T18:32:00.458252+0000","last_became_active":"2023-11-28T18:31:58.458067+0000","last_became_peered":"2023-11-28T18:31:58.458067+0000","last_unstale":"2023-11-28T18:32:00.458252+0000","last_undegraded":"2023-11-28T18:32:00.458252+0000","last_fullsized":"2023-11-28T18:32:00.458252+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:06:09.810035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877271+0000","last_change":"2023-11-28T18:31:59.487976+0000","last_active":"2023-11-28T18:32:00.877271+0000","last_peered":"2023-11-28T18:32:00.877271+0000","last_clean":"2023-11-28T18:32:00.877271+0000","last_became_active":"2023-11-28T18:31:59.487689+0000","last_became_peered":"2023-11-28T18:31:59.487689+0000","last_unstale":"2023-11-28T18:32:00.877271+0000","last_undegraded":"2023-11-28T18:32:00.877271+0000","last_fullsized":"2023-11-28T18:32:00.877271+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:18:53.760883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"128'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877585+0000","last_change":"2023-11-28T18:31:58.468883+0000","last_active":"2023-11-28T18:32:00.877585+0000","last_peered":"2023-11-28T18:32:00.877585+0000","last_clean":"2023-11-28T18:32:00.877585+0000","last_became_active":"2023-11-28T18:31:58.468584+0000","last_became_peered":"2023-11-28T18:31:58.468584+0000","last_unstale":"2023-11-28T18:32:00.877585+0000","last_undegraded":"2023-11-28T18:32:00.877585+0000","last_fullsized":"2023-11-28T18:32:00.877585+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:54:57.441652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.874847+0000","last_change":"2023-11-28T18:31:59.488944+0000","last_active":"2023-11-28T18:32:00.874847+0000","last_peered":"2023-11-28T18:32:00.874847+0000","last_clean":"2023-11-28T18:32:00.874847+0000","last_became_active":"2023-11-28T18:31:59.488661+0000","last_became_peered":"2023-11-28T18:31:59.488661+0000","last_unstale":"2023-11-28T18:32:00.874847+0000","last_undegraded":"2023-11-28T18:32:00.874847+0000","last_fullsized":"2023-11-28T18:32:00.874847+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:49:19.043247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"128'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877198+0000","last_change":"2023-11-28T18:31:58.469430+0000","last_active":"2023-11-28T18:32:00.877198+0000","last_peered":"2023-11-28T18:32:00.877198+0000","last_clean":"2023-11-28T18:32:00.877198+0000","last_became_active":"2023-11-28T18:31:58.469142+0000","last_became_peered":"2023-11-28T18:31:58.469142+0000","last_unstale":"2023-11-28T18:32:00.877198+0000","last_undegraded":"2023-11-28T18:32:00.877198+0000","last_fullsized":"2023-11-28T18:32:00.877198+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:34:41.075893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876755+0000","last_change":"2023-11-28T18:31:59.467964+0000","last_active":"2023-11-28T18:32:00.876755+0000","last_peered":"2023-11-28T18:32:00.876755+0000","last_clean":"2023-11-28T18:32:00.876755+0000","last_became_active":"2023-11-28T18:31:59.467582+0000","last_became_peered":"2023-11-28T18:31:59.467582+0000","last_unstale":"2023-11-28T18:32:00.876755+0000","last_undegraded":"2023-11-28T18:32:00.876755+0000","last_fullsized":"2023-11-28T18:32:00.876755+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:01:47.297968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"128'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876552+0000","last_change":"2023-11-28T18:31:58.466580+0000","last_active":"2023-11-28T18:32:00.876552+0000","last_peered":"2023-11-28T18:32:00.876552+0000","last_clean":"2023-11-28T18:32:00.876552+0000","last_became_active":"2023-11-28T18:31:58.466314+0000","last_became_peered":"2023-11-28T18:31:58.466314+0000","last_unstale":"2023-11-28T18:32:00.876552+0000","last_undegraded":"2023-11-28T18:32:00.876552+0000","last_fullsized":"2023-11-28T18:32:00.876552+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:20.480341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876846+0000","last_change":"2023-11-28T18:31:59.485194+0000","last_active":"2023-11-28T18:32:00.876846+0000","last_peered":"2023-11-28T18:32:00.876846+0000","last_clean":"2023-11-28T18:32:00.876846+0000","last_became_active":"2023-11-28T18:31:59.484777+0000","last_became_peered":"2023-11-28T18:31:59.484777+0000","last_unstale":"2023-11-28T18:32:00.876846+0000","last_undegraded":"2023-11-28T18:32:00.876846+0000","last_fullsized":"2023-11-28T18:32:00.876846+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:03.287701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"128'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.457720+0000","last_change":"2023-11-28T18:31:58.462756+0000","last_active":"2023-11-28T18:32:00.457720+0000","last_peered":"2023-11-28T18:32:00.457720+0000","last_clean":"2023-11-28T18:32:00.457720+0000","last_became_active":"2023-11-28T18:31:58.462493+0000","last_became_peered":"2023-11-28T18:31:58.462493+0000","last_unstale":"2023-11-28T18:32:00.457720+0000","last_undegraded":"2023-11-28T18:32:00.457720+0000","last_fullsized":"2023-11-28T18:32:00.457720+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:21:05.598600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.458185+0000","last_change":"2023-11-28T18:31:59.475931+0000","last_active":"2023-11-28T18:32:00.458185+0000","last_peered":"2023-11-28T18:32:00.458185+0000","last_clean":"2023-11-28T18:32:00.458185+0000","last_became_active":"2023-11-28T18:31:59.475635+0000","last_became_peered":"2023-11-28T18:31:59.475635+0000","last_unstale":"2023-11-28T18:32:00.458185+0000","last_undegraded":"2023-11-28T18:32:00.458185+0000","last_fullsized":"2023-11-28T18:32:00.458185+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:32:47.473720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459438+0000","last_change":"2023-11-28T18:31:58.456222+0000","last_active":"2023-11-28T18:32:00.459438+0000","last_peered":"2023-11-28T18:32:00.459438+0000","last_clean":"2023-11-28T18:32:00.459438+0000","last_became_active":"2023-11-28T18:31:58.455955+0000","last_became_peered":"2023-11-28T18:31:58.455955+0000","last_unstale":"2023-11-28T18:32:00.459438+0000","last_undegraded":"2023-11-28T18:32:00.459438+0000","last_fullsized":"2023-11-28T18:32:00.459438+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:02.280932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877341+0000","last_change":"2023-11-28T18:31:59.488135+0000","last_active":"2023-11-28T18:32:00.877341+0000","last_peered":"2023-11-28T18:32:00.877341+0000","last_clean":"2023-11-28T18:32:00.877341+0000","last_became_active":"2023-11-28T18:31:59.487860+0000","last_became_peered":"2023-11-28T18:31:59.487860+0000","last_unstale":"2023-11-28T18:32:00.877341+0000","last_undegraded":"2023-11-28T18:32:00.877341+0000","last_fullsized":"2023-11-28T18:32:00.877341+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:38:56.438986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460461+0000","last_change":"2023-11-28T18:31:58.452319+0000","last_active":"2023-11-28T18:32:00.460461+0000","last_peered":"2023-11-28T18:32:00.460461+0000","last_clean":"2023-11-28T18:32:00.460461+0000","last_became_active":"2023-11-28T18:31:58.452038+0000","last_became_peered":"2023-11-28T18:31:58.452038+0000","last_unstale":"2023-11-28T18:32:00.460461+0000","last_undegraded":"2023-11-28T18:32:00.460461+0000","last_fullsized":"2023-11-28T18:32:00.460461+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:08.297316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.874654+0000","last_change":"2023-11-28T18:31:59.482080+0000","last_active":"2023-11-28T18:32:00.874654+0000","last_peered":"2023-11-28T18:32:00.874654+0000","last_clean":"2023-11-28T18:32:00.874654+0000","last_became_active":"2023-11-28T18:31:59.481289+0000","last_became_peered":"2023-11-28T18:31:59.481289+0000","last_unstale":"2023-11-28T18:32:00.874654+0000","last_undegraded":"2023-11-28T18:32:00.874654+0000","last_fullsized":"2023-11-28T18:32:00.874654+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:57:32.556863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877793+0000","last_change":"2023-11-28T18:31:58.463931+0000","last_active":"2023-11-28T18:32:00.877793+0000","last_peered":"2023-11-28T18:32:00.877793+0000","last_clean":"2023-11-28T18:32:00.877793+0000","last_became_active":"2023-11-28T18:31:58.463414+0000","last_became_peered":"2023-11-28T18:31:58.463414+0000","last_unstale":"2023-11-28T18:32:00.877793+0000","last_undegraded":"2023-11-28T18:32:00.877793+0000","last_fullsized":"2023-11-28T18:32:00.877793+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:01:46.291198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459608+0000","last_change":"2023-11-28T18:31:59.486109+0000","last_active":"2023-11-28T18:32:00.459608+0000","last_peered":"2023-11-28T18:32:00.459608+0000","last_clean":"2023-11-28T18:32:00.459608+0000","last_became_active":"2023-11-28T18:31:59.484511+0000","last_became_peered":"2023-11-28T18:31:59.484511+0000","last_unstale":"2023-11-28T18:32:00.459608+0000","last_undegraded":"2023-11-28T18:32:00.459608+0000","last_fullsized":"2023-11-28T18:32:00.459608+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:08:59.784543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.875241+0000","last_change":"2023-11-28T18:31:58.465300+0000","last_active":"2023-11-28T18:32:00.875241+0000","last_peered":"2023-11-28T18:32:00.875241+0000","last_clean":"2023-11-28T18:32:00.875241+0000","last_became_active":"2023-11-28T18:31:58.464788+0000","last_became_peered":"2023-11-28T18:31:58.464788+0000","last_unstale":"2023-11-28T18:32:00.875241+0000","last_undegraded":"2023-11-28T18:32:00.875241+0000","last_fullsized":"2023-11-28T18:32:00.875241+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:38:56.117848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460854+0000","last_change":"2023-11-28T18:31:59.480620+0000","last_active":"2023-11-28T18:32:00.460854+0000","last_peered":"2023-11-28T18:32:00.460854+0000","last_clean":"2023-11-28T18:32:00.460854+0000","last_became_active":"2023-11-28T18:31:59.480298+0000","last_became_peered":"2023-11-28T18:31:59.480298+0000","last_unstale":"2023-11-28T18:32:00.460854+0000","last_undegraded":"2023-11-28T18:32:00.460854+0000","last_fullsized":"2023-11-28T18:32:00.460854+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:07:10.812548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"128'3","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877588+0000","last_change":"2023-11-28T18:31:58.466863+0000","last_active":"2023-11-28T18:32:00.877588+0000","last_peered":"2023-11-28T18:32:00.877588+0000","last_clean":"2023-11-28T18:32:00.877588+0000","last_became_active":"2023-11-28T18:31:58.466545+0000","last_became_peered":"2023-11-28T18:31:58.466545+0000","last_unstale":"2023-11-28T18:32:00.877588+0000","last_undegraded":"2023-11-28T18:32:00.877588+0000","last_fullsized":"2023-11-28T18:32:00.877588+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:34:02.704967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460670+0000","last_change":"2023-11-28T18:31:59.485179+0000","last_active":"2023-11-28T18:32:00.460670+0000","last_peered":"2023-11-28T18:32:00.460670+0000","last_clean":"2023-11-28T18:32:00.460670+0000","last_became_active":"2023-11-28T18:31:59.484851+0000","last_became_peered":"2023-11-28T18:31:59.484851+0000","last_unstale":"2023-11-28T18:32:00.460670+0000","last_undegraded":"2023-11-28T18:32:00.460670+0000","last_fullsized":"2023-11-28T18:32:00.460670+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:03.287701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"128'6","reported_seq":18,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877303+0000","last_change":"2023-11-28T18:31:58.470568+0000","last_active":"2023-11-28T18:32:00.877303+0000","last_peered":"2023-11-28T18:32:00.877303+0000","last_clean":"2023-11-28T18:32:00.877303+0000","last_became_active":"2023-11-28T18:31:58.470239+0000","last_became_peered":"2023-11-28T18:31:58.470239+0000","last_unstale":"2023-11-28T18:32:00.877303+0000","last_undegraded":"2023-11-28T18:32:00.877303+0000","last_fullsized":"2023-11-28T18:32:00.877303+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:02.029926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.875007+0000","last_change":"2023-11-28T18:31:59.479834+0000","last_active":"2023-11-28T18:32:00.875007+0000","last_peered":"2023-11-28T18:32:00.875007+0000","last_clean":"2023-11-28T18:32:00.875007+0000","last_became_active":"2023-11-28T18:31:59.479534+0000","last_became_peered":"2023-11-28T18:31:59.479534+0000","last_unstale":"2023-11-28T18:32:00.875007+0000","last_undegraded":"2023-11-28T18:32:00.875007+0000","last_fullsized":"2023-11-28T18:32:00.875007+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:10:44.436247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"128'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.875081+0000","last_change":"2023-11-28T18:31:58.465789+0000","last_active":"2023-11-28T18:32:00.875081+0000","last_peered":"2023-11-28T18:32:00.875081+0000","last_clean":"2023-11-28T18:32:00.875081+0000","last_became_active":"2023-11-28T18:31:58.465511+0000","last_became_peered":"2023-11-28T18:31:58.465511+0000","last_unstale":"2023-11-28T18:32:00.875081+0000","last_undegraded":"2023-11-28T18:32:00.875081+0000","last_fullsized":"2023-11-28T18:32:00.875081+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:25:17.256080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877871+0000","last_change":"2023-11-28T18:31:59.483918+0000","last_active":"2023-11-28T18:32:00.877871+0000","last_peered":"2023-11-28T18:32:00.877871+0000","last_clean":"2023-11-28T18:32:00.877871+0000","last_became_active":"2023-11-28T18:31:59.482610+0000","last_became_peered":"2023-11-28T18:31:59.482610+0000","last_unstale":"2023-11-28T18:32:00.877871+0000","last_undegraded":"2023-11-28T18:32:00.877871+0000","last_fullsized":"2023-11-28T18:32:00.877871+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:18:37.964759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460222+0000","last_change":"2023-11-28T18:31:58.459955+0000","last_active":"2023-11-28T18:32:00.460222+0000","last_peered":"2023-11-28T18:32:00.460222+0000","last_clean":"2023-11-28T18:32:00.460222+0000","last_became_active":"2023-11-28T18:31:58.457155+0000","last_became_peered":"2023-11-28T18:31:58.457155+0000","last_unstale":"2023-11-28T18:32:00.460222+0000","last_undegraded":"2023-11-28T18:32:00.460222+0000","last_fullsized":"2023-11-28T18:32:00.460222+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:42.191962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.458273+0000","last_change":"2023-11-28T18:31:59.470228+0000","last_active":"2023-11-28T18:32:00.458273+0000","last_peered":"2023-11-28T18:32:00.458273+0000","last_clean":"2023-11-28T18:32:00.458273+0000","last_became_active":"2023-11-28T18:31:59.469975+0000","last_became_peered":"2023-11-28T18:31:59.469975+0000","last_unstale":"2023-11-28T18:32:00.458273+0000","last_undegraded":"2023-11-28T18:32:00.458273+0000","last_fullsized":"2023-11-28T18:32:00.458273+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:41.384216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"128'8","reported_seq":20,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459529+0000","last_change":"2023-11-28T18:31:58.454212+0000","last_active":"2023-11-28T18:32:00.459529+0000","last_peered":"2023-11-28T18:32:00.459529+0000","last_clean":"2023-11-28T18:32:00.459529+0000","last_became_active":"2023-11-28T18:31:58.453946+0000","last_became_peered":"2023-11-28T18:31:58.453946+0000","last_unstale":"2023-11-28T18:32:00.459529+0000","last_undegraded":"2023-11-28T18:32:00.459529+0000","last_fullsized":"2023-11-28T18:32:00.459529+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:21:05.598600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460226+0000","last_change":"2023-11-28T18:31:59.475375+0000","last_active":"2023-11-28T18:32:00.460226+0000","last_peered":"2023-11-28T18:32:00.460226+0000","last_clean":"2023-11-28T18:32:00.460226+0000","last_became_active":"2023-11-28T18:31:59.475085+0000","last_became_peered":"2023-11-28T18:31:59.475085+0000","last_unstale":"2023-11-28T18:32:00.460226+0000","last_undegraded":"2023-11-28T18:32:00.460226+0000","last_fullsized":"2023-11-28T18:32:00.460226+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:37:13.925200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"128'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877867+0000","last_change":"2023-11-28T18:31:58.466101+0000","last_active":"2023-11-28T18:32:00.877867+0000","last_peered":"2023-11-28T18:32:00.877867+0000","last_clean":"2023-11-28T18:32:00.877867+0000","last_became_active":"2023-11-28T18:31:58.465528+0000","last_became_peered":"2023-11-28T18:31:58.465528+0000","last_unstale":"2023-11-28T18:32:00.877867+0000","last_undegraded":"2023-11-28T18:32:00.877867+0000","last_fullsized":"2023-11-28T18:32:00.877867+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:02.280932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876251+0000","last_change":"2023-11-28T18:31:59.481710+0000","last_active":"2023-11-28T18:32:00.876251+0000","last_peered":"2023-11-28T18:32:00.876251+0000","last_clean":"2023-11-28T18:32:00.876251+0000","last_became_active":"2023-11-28T18:31:59.481363+0000","last_became_peered":"2023-11-28T18:31:59.481363+0000","last_unstale":"2023-11-28T18:32:00.876251+0000","last_undegraded":"2023-11-28T18:32:00.876251+0000","last_fullsized":"2023-11-28T18:32:00.876251+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:42:34.470538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"128'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460525+0000","last_change":"2023-11-28T18:31:58.461973+0000","last_active":"2023-11-28T18:32:00.460525+0000","last_peered":"2023-11-28T18:32:00.460525+0000","last_clean":"2023-11-28T18:32:00.460525+0000","last_became_active":"2023-11-28T18:31:58.461697+0000","last_became_peered":"2023-11-28T18:31:58.461697+0000","last_unstale":"2023-11-28T18:32:00.460525+0000","last_undegraded":"2023-11-28T18:32:00.460525+0000","last_fullsized":"2023-11-28T18:32:00.460525+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:35:24.012986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876566+0000","last_change":"2023-11-28T18:31:59.483760+0000","last_active":"2023-11-28T18:32:00.876566+0000","last_peered":"2023-11-28T18:32:00.876566+0000","last_clean":"2023-11-28T18:32:00.876566+0000","last_became_active":"2023-11-28T18:31:59.483403+0000","last_became_peered":"2023-11-28T18:31:59.483403+0000","last_unstale":"2023-11-28T18:32:00.876566+0000","last_undegraded":"2023-11-28T18:32:00.876566+0000","last_fullsized":"2023-11-28T18:32:00.876566+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:07:19.157448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"128'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459900+0000","last_change":"2023-11-28T18:31:58.464128+0000","last_active":"2023-11-28T18:32:00.459900+0000","last_peered":"2023-11-28T18:32:00.459900+0000","last_clean":"2023-11-28T18:32:00.459900+0000","last_became_active":"2023-11-28T18:31:58.463478+0000","last_became_peered":"2023-11-28T18:31:58.463478+0000","last_unstale":"2023-11-28T18:32:00.459900+0000","last_undegraded":"2023-11-28T18:32:00.459900+0000","last_fullsized":"2023-11-28T18:32:00.459900+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:06:14.458890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.874831+0000","last_change":"2023-11-28T18:31:59.483751+0000","last_active":"2023-11-28T18:32:00.874831+0000","last_peered":"2023-11-28T18:32:00.874831+0000","last_clean":"2023-11-28T18:32:00.874831+0000","last_became_active":"2023-11-28T18:31:59.483475+0000","last_became_peered":"2023-11-28T18:31:59.483475+0000","last_unstale":"2023-11-28T18:32:00.874831+0000","last_undegraded":"2023-11-28T18:32:00.874831+0000","last_fullsized":"2023-11-28T18:32:00.874831+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:43:07.474936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.458005+0000","last_change":"2023-11-28T18:31:58.460778+0000","last_active":"2023-11-28T18:32:00.458005+0000","last_peered":"2023-11-28T18:32:00.458005+0000","last_clean":"2023-11-28T18:32:00.458005+0000","last_became_active":"2023-11-28T18:31:58.460302+0000","last_became_peered":"2023-11-28T18:31:58.460302+0000","last_unstale":"2023-11-28T18:32:00.458005+0000","last_undegraded":"2023-11-28T18:32:00.458005+0000","last_fullsized":"2023-11-28T18:32:00.458005+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:22:51.062346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.457924+0000","last_change":"2023-11-28T18:31:59.478768+0000","last_active":"2023-11-28T18:32:00.457924+0000","last_peered":"2023-11-28T18:32:00.457924+0000","last_clean":"2023-11-28T18:32:00.457924+0000","last_became_active":"2023-11-28T18:31:59.478507+0000","last_became_peered":"2023-11-28T18:31:59.478507+0000","last_unstale":"2023-11-28T18:32:00.457924+0000","last_undegraded":"2023-11-28T18:32:00.457924+0000","last_fullsized":"2023-11-28T18:32:00.457924+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:33:59.180002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"128'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876662+0000","last_change":"2023-11-28T18:31:58.466034+0000","last_active":"2023-11-28T18:32:00.876662+0000","last_peered":"2023-11-28T18:32:00.876662+0000","last_clean":"2023-11-28T18:32:00.876662+0000","last_became_active":"2023-11-28T18:31:58.465721+0000","last_became_peered":"2023-11-28T18:31:58.465721+0000","last_unstale":"2023-11-28T18:32:00.876662+0000","last_undegraded":"2023-11-28T18:32:00.876662+0000","last_fullsized":"2023-11-28T18:32:00.876662+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:40.046834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.875288+0000","last_change":"2023-11-28T18:31:59.485034+0000","last_active":"2023-11-28T18:32:00.875288+0000","last_peered":"2023-11-28T18:32:00.875288+0000","last_clean":"2023-11-28T18:32:00.875288+0000","last_became_active":"2023-11-28T18:31:59.484739+0000","last_became_peered":"2023-11-28T18:31:59.484739+0000","last_unstale":"2023-11-28T18:32:00.875288+0000","last_undegraded":"2023-11-28T18:32:00.875288+0000","last_fullsized":"2023-11-28T18:32:00.875288+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:36:10.244933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.874895+0000","last_change":"2023-11-28T18:31:58.466656+0000","last_active":"2023-11-28T18:32:00.874895+0000","last_peered":"2023-11-28T18:32:00.874895+0000","last_clean":"2023-11-28T18:32:00.874895+0000","last_became_active":"2023-11-28T18:31:58.466407+0000","last_became_peered":"2023-11-28T18:31:58.466407+0000","last_unstale":"2023-11-28T18:32:00.874895+0000","last_undegraded":"2023-11-28T18:32:00.874895+0000","last_fullsized":"2023-11-28T18:32:00.874895+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:31:34.336753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.458273+0000","last_change":"2023-11-28T18:31:59.475681+0000","last_active":"2023-11-28T18:32:00.458273+0000","last_peered":"2023-11-28T18:32:00.458273+0000","last_clean":"2023-11-28T18:32:00.458273+0000","last_became_active":"2023-11-28T18:31:59.475211+0000","last_became_peered":"2023-11-28T18:31:59.475211+0000","last_unstale":"2023-11-28T18:32:00.458273+0000","last_undegraded":"2023-11-28T18:32:00.458273+0000","last_fullsized":"2023-11-28T18:32:00.458273+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:33:00.087111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876289+0000","last_change":"2023-11-28T18:31:58.465083+0000","last_active":"2023-11-28T18:32:00.876289+0000","last_peered":"2023-11-28T18:32:00.876289+0000","last_clean":"2023-11-28T18:32:00.876289+0000","last_became_active":"2023-11-28T18:31:58.464760+0000","last_became_peered":"2023-11-28T18:31:58.464760+0000","last_unstale":"2023-11-28T18:32:00.876289+0000","last_undegraded":"2023-11-28T18:32:00.876289+0000","last_fullsized":"2023-11-28T18:32:00.876289+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:33:58.173232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877845+0000","last_change":"2023-11-28T18:31:59.490489+0000","last_active":"2023-11-28T18:32:00.877845+0000","last_peered":"2023-11-28T18:32:00.877845+0000","last_clean":"2023-11-28T18:32:00.877845+0000","last_became_active":"2023-11-28T18:31:59.490218+0000","last_became_peered":"2023-11-28T18:31:59.490218+0000","last_unstale":"2023-11-28T18:32:00.877845+0000","last_undegraded":"2023-11-28T18:32:00.877845+0000","last_fullsized":"2023-11-28T18:32:00.877845+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:50:40.320398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459609+0000","last_change":"2023-11-28T18:31:58.456766+0000","last_active":"2023-11-28T18:32:00.459609+0000","last_peered":"2023-11-28T18:32:00.459609+0000","last_clean":"2023-11-28T18:32:00.459609+0000","last_became_active":"2023-11-28T18:31:58.456515+0000","last_became_peered":"2023-11-28T18:31:58.456515+0000","last_unstale":"2023-11-28T18:32:00.459609+0000","last_undegraded":"2023-11-28T18:32:00.459609+0000","last_fullsized":"2023-11-28T18:32:00.459609+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:18:52.754113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.457843+0000","last_change":"2023-11-28T18:31:59.475363+0000","last_active":"2023-11-28T18:32:00.457843+0000","last_peered":"2023-11-28T18:32:00.457843+0000","last_clean":"2023-11-28T18:32:00.457843+0000","last_became_active":"2023-11-28T18:31:59.475049+0000","last_became_peered":"2023-11-28T18:31:59.475049+0000","last_unstale":"2023-11-28T18:32:00.457843+0000","last_undegraded":"2023-11-28T18:32:00.457843+0000","last_fullsized":"2023-11-28T18:32:00.457843+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:03.036695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"128'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.875436+0000","last_change":"2023-11-28T18:31:58.464128+0000","last_active":"2023-11-28T18:32:00.875436+0000","last_peered":"2023-11-28T18:32:00.875436+0000","last_clean":"2023-11-28T18:32:00.875436+0000","last_became_active":"2023-11-28T18:31:58.463835+0000","last_became_peered":"2023-11-28T18:31:58.463835+0000","last_unstale":"2023-11-28T18:32:00.875436+0000","last_undegraded":"2023-11-28T18:32:00.875436+0000","last_fullsized":"2023-11-28T18:32:00.875436+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:45:51.904590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460158+0000","last_change":"2023-11-28T18:31:59.479617+0000","last_active":"2023-11-28T18:32:00.460158+0000","last_peered":"2023-11-28T18:32:00.460158+0000","last_clean":"2023-11-28T18:32:00.460158+0000","last_became_active":"2023-11-28T18:31:59.479323+0000","last_became_peered":"2023-11-28T18:31:59.479323+0000","last_unstale":"2023-11-28T18:32:00.460158+0000","last_undegraded":"2023-11-28T18:32:00.460158+0000","last_fullsized":"2023-11-28T18:32:00.460158+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:51:51.574894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877023+0000","last_change":"2023-11-28T18:31:58.466768+0000","last_active":"2023-11-28T18:32:00.877023+0000","last_peered":"2023-11-28T18:32:00.877023+0000","last_clean":"2023-11-28T18:32:00.877023+0000","last_became_active":"2023-11-28T18:31:58.466495+0000","last_became_peered":"2023-11-28T18:31:58.466495+0000","last_unstale":"2023-11-28T18:32:00.877023+0000","last_undegraded":"2023-11-28T18:32:00.877023+0000","last_fullsized":"2023-11-28T18:32:00.877023+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:08.297316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876111+0000","last_change":"2023-11-28T18:31:59.497301+0000","last_active":"2023-11-28T18:32:00.876111+0000","last_peered":"2023-11-28T18:32:00.876111+0000","last_clean":"2023-11-28T18:32:00.876111+0000","last_became_active":"2023-11-28T18:31:59.497008+0000","last_became_peered":"2023-11-28T18:31:59.497008+0000","last_unstale":"2023-11-28T18:32:00.876111+0000","last_undegraded":"2023-11-28T18:32:00.876111+0000","last_fullsized":"2023-11-28T18:32:00.876111+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:30.295476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876942+0000","last_change":"2023-11-28T18:31:58.464865+0000","last_active":"2023-11-28T18:32:00.876942+0000","last_peered":"2023-11-28T18:32:00.876942+0000","last_clean":"2023-11-28T18:32:00.876942+0000","last_became_active":"2023-11-28T18:31:58.464575+0000","last_became_peered":"2023-11-28T18:31:58.464575+0000","last_unstale":"2023-11-28T18:32:00.876942+0000","last_undegraded":"2023-11-28T18:32:00.876942+0000","last_fullsized":"2023-11-28T18:32:00.876942+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:55:33.705881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.874925+0000","last_change":"2023-11-28T18:31:59.483192+0000","last_active":"2023-11-28T18:32:00.874925+0000","last_peered":"2023-11-28T18:32:00.874925+0000","last_clean":"2023-11-28T18:32:00.874925+0000","last_became_active":"2023-11-28T18:31:59.480298+0000","last_became_peered":"2023-11-28T18:31:59.480298+0000","last_unstale":"2023-11-28T18:32:00.874925+0000","last_undegraded":"2023-11-28T18:32:00.874925+0000","last_fullsized":"2023-11-28T18:32:00.874925+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:55:10.107387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459728+0000","last_change":"2023-11-28T18:31:58.460507+0000","last_active":"2023-11-28T18:32:00.459728+0000","last_peered":"2023-11-28T18:32:00.459728+0000","last_clean":"2023-11-28T18:32:00.459728+0000","last_became_active":"2023-11-28T18:31:58.460248+0000","last_became_peered":"2023-11-28T18:31:58.460248+0000","last_unstale":"2023-11-28T18:32:00.459728+0000","last_undegraded":"2023-11-28T18:32:00.459728+0000","last_fullsized":"2023-11-28T18:32:00.459728+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:15:06.833507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459746+0000","last_change":"2023-11-28T18:31:59.478535+0000","last_active":"2023-11-28T18:32:00.459746+0000","last_peered":"2023-11-28T18:32:00.459746+0000","last_clean":"2023-11-28T18:32:00.459746+0000","last_became_active":"2023-11-28T18:31:59.477929+0000","last_became_peered":"2023-11-28T18:31:59.477929+0000","last_unstale":"2023-11-28T18:32:00.459746+0000","last_undegraded":"2023-11-28T18:32:00.459746+0000","last_fullsized":"2023-11-28T18:32:00.459746+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:06:10.816804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877470+0000","last_change":"2023-11-28T18:31:58.463918+0000","last_active":"2023-11-28T18:32:00.877470+0000","last_peered":"2023-11-28T18:32:00.877470+0000","last_clean":"2023-11-28T18:32:00.877470+0000","last_became_active":"2023-11-28T18:31:58.463458+0000","last_became_peered":"2023-11-28T18:31:58.463458+0000","last_unstale":"2023-11-28T18:32:00.877470+0000","last_undegraded":"2023-11-28T18:32:00.877470+0000","last_fullsized":"2023-11-28T18:32:00.877470+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:37:12.918431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459901+0000","last_change":"2023-11-28T18:31:59.479415+0000","last_active":"2023-11-28T18:32:00.459901+0000","last_peered":"2023-11-28T18:32:00.459901+0000","last_clean":"2023-11-28T18:32:00.459901+0000","last_became_active":"2023-11-28T18:31:59.479144+0000","last_became_peered":"2023-11-28T18:31:59.479144+0000","last_unstale":"2023-11-28T18:32:00.459901+0000","last_undegraded":"2023-11-28T18:32:00.459901+0000","last_fullsized":"2023-11-28T18:32:00.459901+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:22:52.069116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876718+0000","last_change":"2023-11-28T18:31:58.467928+0000","last_active":"2023-11-28T18:32:00.876718+0000","last_peered":"2023-11-28T18:32:00.876718+0000","last_clean":"2023-11-28T18:32:00.876718+0000","last_became_active":"2023-11-28T18:31:58.467636+0000","last_became_peered":"2023-11-28T18:31:58.467636+0000","last_unstale":"2023-11-28T18:32:00.876718+0000","last_undegraded":"2023-11-28T18:32:00.876718+0000","last_fullsized":"2023-11-28T18:32:00.876718+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:29.438862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460049+0000","last_change":"2023-11-28T18:31:59.479948+0000","last_active":"2023-11-28T18:32:00.460049+0000","last_peered":"2023-11-28T18:32:00.460049+0000","last_clean":"2023-11-28T18:32:00.460049+0000","last_became_active":"2023-11-28T18:31:59.479688+0000","last_became_peered":"2023-11-28T18:31:59.479688+0000","last_unstale":"2023-11-28T18:32:00.460049+0000","last_undegraded":"2023-11-28T18:32:00.460049+0000","last_fullsized":"2023-11-28T18:32:00.460049+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:02.710984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459564+0000","last_change":"2023-11-28T18:31:58.460922+0000","last_active":"2023-11-28T18:32:00.459564+0000","last_peered":"2023-11-28T18:32:00.459564+0000","last_clean":"2023-11-28T18:32:00.459564+0000","last_became_active":"2023-11-28T18:31:58.460668+0000","last_became_peered":"2023-11-28T18:31:58.460668+0000","last_unstale":"2023-11-28T18:32:00.459564+0000","last_undegraded":"2023-11-28T18:32:00.459564+0000","last_fullsized":"2023-11-28T18:32:00.459564+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:07:09.805778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877615+0000","last_change":"2023-11-28T18:31:59.497732+0000","last_active":"2023-11-28T18:32:00.877615+0000","last_peered":"2023-11-28T18:32:00.877615+0000","last_clean":"2023-11-28T18:32:00.877615+0000","last_became_active":"2023-11-28T18:31:59.497224+0000","last_became_peered":"2023-11-28T18:31:59.497224+0000","last_unstale":"2023-11-28T18:32:00.877615+0000","last_undegraded":"2023-11-28T18:32:00.877615+0000","last_fullsized":"2023-11-28T18:32:00.877615+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:08.815498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.457917+0000","last_change":"2023-11-28T18:31:58.449122+0000","last_active":"2023-11-28T18:32:00.457917+0000","last_peered":"2023-11-28T18:32:00.457917+0000","last_clean":"2023-11-28T18:32:00.457917+0000","last_became_active":"2023-11-28T18:31:58.448939+0000","last_became_peered":"2023-11-28T18:31:58.448939+0000","last_unstale":"2023-11-28T18:32:00.457917+0000","last_undegraded":"2023-11-28T18:32:00.457917+0000","last_fullsized":"2023-11-28T18:32:00.457917+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:15:06.833507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.461038+0000","last_change":"2023-11-28T18:31:59.481570+0000","last_active":"2023-11-28T18:32:00.461038+0000","last_peered":"2023-11-28T18:32:00.461038+0000","last_clean":"2023-11-28T18:32:00.461038+0000","last_became_active":"2023-11-28T18:31:59.481274+0000","last_became_peered":"2023-11-28T18:31:59.481274+0000","last_unstale":"2023-11-28T18:32:00.461038+0000","last_undegraded":"2023-11-28T18:32:00.461038+0000","last_fullsized":"2023-11-28T18:32:00.461038+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:17.025975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876388+0000","last_change":"2023-11-28T18:31:58.468957+0000","last_active":"2023-11-28T18:32:00.876388+0000","last_peered":"2023-11-28T18:32:00.876388+0000","last_clean":"2023-11-28T18:32:00.876388+0000","last_became_active":"2023-11-28T18:31:58.468642+0000","last_became_peered":"2023-11-28T18:31:58.468642+0000","last_unstale":"2023-11-28T18:32:00.876388+0000","last_undegraded":"2023-11-28T18:32:00.876388+0000","last_fullsized":"2023-11-28T18:32:00.876388+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:01.704215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":258,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-11-28T18:31:59.468640+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:31:59.468640+0000","last_peered":"2023-11-28T18:31:59.468640+0000","last_clean":"2023-11-28T18:31:59.468640+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:31:59.468640+0000","last_undegraded":"2023-11-28T18:31:59.468640+0000","last_fullsized":"2023-11-28T18:31:59.468640+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574956,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51388,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320452,"statfs":{"total":96636764160,"available":96584142848,"internally_reserved":0,"allocated":2211840,"data_stored":1338063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65200000000000002,"15min":0.69699999999999995},"min":{"1min":0.48899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.90400000000000003,"5min":0.96999999999999997,"15min":2.427},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.64600000000000002,"15min":0.69699999999999995},"min":{"1min":0.372,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":2.2509999999999999},"last":0.54000000000000004}]},{"osd":1,"last update":"Tue Nov 28 18:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66800000000000004,"15min":0.70199999999999996},"min":{"1min":0.41299999999999998,"5min":0.38900000000000001,"15min":0.34699999999999998},"max":{"1min":0.85799999999999998,"5min":0.96699999999999997,"15min":2.3849999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66200000000000003,"15min":0.72599999999999998},"min":{"1min":0.44400000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.95399999999999996,"5min":1.079,"15min":2.3130000000000002},"last":0.59099999999999997}]},{"osd":2,"last update":"Tue Nov 28 18:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.68600000000000005,"15min":0.747},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94599999999999995,"5min":0.998,"15min":2.4590000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.69599999999999995,"15min":0.72699999999999998},"min":{"1min":0.56799999999999995,"5min":0.41299999999999998,"15min":0.39300000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":2.411},"last":0.747}]},{"osd":3,"last update":"Tue Nov 28 18:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.67300000000000004,"15min":0.73399999999999999},"min":{"1min":0.45100000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":2.2730000000000001},"last":0.628},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.72799999999999998,"15min":0.77900000000000003},"min":{"1min":0.59299999999999997,"5min":0.432,"15min":0.432},"max":{"1min":1.008,"5min":1.0389999999999999,"15min":2.3650000000000002},"last":0.67600000000000005}]},{"osd":4,"last update":"Tue Nov 28 18:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.61699999999999999,"15min":0.60599999999999998},"min":{"1min":0.434,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60699999999999998,"15min":0.63},"min":{"1min":0.40699999999999997,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.86399999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.69199999999999995}]},{"osd":5,"last update":"Tue Nov 28 18:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64400000000000002,"15min":0.66500000000000004},"min":{"1min":0.45700000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0589999999999999,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.67400000000000004,"15min":0.69199999999999995},"min":{"1min":0.52800000000000002,"5min":0.39900000000000002,"15min":0.39300000000000002},"max":{"1min":0.996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.65500000000000003}]},{"osd":6,"last update":"Tue Nov 28 18:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.72499999999999998,"15min":0.73699999999999999},"min":{"1min":0.495,"5min":0.375,"15min":0.375},"max":{"1min":1.036,"5min":1.131,"15min":1.131},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.66600000000000004,"15min":0.71099999999999997},"min":{"1min":0.46000000000000002,"5min":0.36699999999999999,"15min":0.35899999999999999},"max":{"1min":1.02,"5min":1.081,"15min":1.081},"last":0.57099999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73016,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298824,"statfs":{"total":96636764160,"available":96561995776,"internally_reserved":0,"allocated":1609728,"data_stored":746665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.68600000000000005,"15min":0.65300000000000002},"min":{"1min":0.51300000000000001,"5min":0.35499999999999998,"15min":0.27300000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.56999999999999995,"5min":0.40699999999999997,"15min":0.35399999999999998},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":0.75700000000000001}]},{"osd":1,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.67300000000000004,"15min":0.76500000000000001},"min":{"1min":0.45100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":4.2389999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.69899999999999995,"15min":0.76300000000000001},"min":{"1min":0.53000000000000003,"5min":0.38200000000000001,"15min":0.307},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":4.218},"last":0.71599999999999997}]},{"osd":2,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.67900000000000005,"15min":0.70399999999999996},"min":{"1min":0.44400000000000001,"5min":0.373,"15min":0.36299999999999999},"max":{"1min":2.7599999999999998,"5min":2.7599999999999998,"15min":2.7599999999999998},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.71299999999999997,"15min":0.70799999999999996},"min":{"1min":0.54700000000000004,"5min":0.42999999999999999,"15min":0.40100000000000002},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.73699999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70499999999999996,"15min":0.71899999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.39800000000000002},"max":{"1min":0.99299999999999999,"5min":1.016,"15min":1.016},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.52600000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.8090000000000002,"5min":2.8090000000000002,"15min":2.8090000000000002},"last":0.77100000000000002}]},{"osd":4,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60399999999999998,"15min":0.65000000000000002},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64000000000000001,"15min":0.67100000000000004},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.64100000000000001,"15min":0.66900000000000004},"min":{"1min":0.39000000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.67900000000000005,"15min":0.73599999999999999},"min":{"1min":0.46800000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.66000000000000003}]},{"osd":7,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69199999999999995,"15min":0.747},"min":{"1min":0.44500000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.70799999999999996,"15min":0.73799999999999999},"min":{"1min":0.495,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.8490000000000002,"5min":2.8490000000000002,"15min":2.8490000000000002},"last":0.80800000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36920,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334920,"statfs":{"total":96636764160,"available":96598958080,"internally_reserved":0,"allocated":1617920,"data_stored":752635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67100000000000004,"15min":0.70199999999999996},"min":{"1min":0.432,"5min":0.31,"15min":0.31},"max":{"1min":0.86299999999999999,"5min":2.1299999999999999,"15min":2.8929999999999998},"last":0.745},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70799999999999996,"15min":0.73899999999999999},"min":{"1min":0.52300000000000002,"5min":0.39200000000000002,"15min":0.33900000000000002},"max":{"1min":1.0169999999999999,"5min":1.0249999999999999,"15min":2.9140000000000001},"last":0.83899999999999997}]},{"osd":1,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.68500000000000005,"15min":0.72499999999999998},"min":{"1min":0.52800000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":3.4809999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.72799999999999998,"15min":0.77800000000000002},"min":{"1min":0.64500000000000002,"5min":0.40200000000000002,"15min":0.39100000000000001},"max":{"1min":1.0509999999999999,"5min":2.2109999999999999,"15min":3.4580000000000002},"last":0.78200000000000003}]},{"osd":2,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.67100000000000004,"15min":0.73199999999999998},"min":{"1min":0.54300000000000004,"5min":0.378,"15min":0.378},"max":{"1min":0.96399999999999997,"5min":1.0309999999999999,"15min":3.9510000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66900000000000004,"15min":0.74199999999999999},"min":{"1min":0.46400000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91300000000000003,"5min":1.014,"15min":4.0149999999999997},"last":0.69799999999999995}]},{"osd":3,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69799999999999995,"15min":0.78900000000000003},"min":{"1min":0.58999999999999997,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85999999999999999,"5min":2.254,"15min":4.0869999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68799999999999994,"15min":0.78100000000000003},"min":{"1min":0.46800000000000003,"5min":0.46300000000000002,"15min":0.41299999999999998},"max":{"1min":0.92900000000000005,"5min":0.98599999999999999,"15min":3.9900000000000002},"last":0.72399999999999998}]},{"osd":4,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.44400000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82199999999999995,"5min":1.696,"15min":1.696},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.65900000000000003,"15min":0.69399999999999995},"min":{"1min":0.39200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86699999999999999,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.52300000000000002}]},{"osd":6,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64300000000000002,"15min":0.66900000000000004},"min":{"1min":0.41399999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79500000000000004,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68300000000000005,"15min":0.70699999999999996},"min":{"1min":0.51200000000000001,"5min":0.438,"15min":0.42399999999999999},"max":{"1min":0.877,"5min":1.262,"15min":1.756},"last":0.82199999999999995}]},{"osd":7,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71899999999999997,"15min":0.74399999999999999},"min":{"1min":0.623,"5min":0.42399999999999999,"15min":0.41699999999999998},"max":{"1min":0.89800000000000002,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.72799999999999998,"15min":0.747},"min":{"1min":0.54100000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94499999999999995,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.65100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574956,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73004,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298836,"statfs":{"total":96636764160,"available":96562008064,"internally_reserved":0,"allocated":1601536,"data_stored":745993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.64900000000000002,"15min":0.67500000000000004},"min":{"1min":0.35099999999999998,"5min":0.32600000000000001,"15min":0.315},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":2.363},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.65300000000000002,"15min":0.64800000000000002},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.997}]},{"osd":1,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.65300000000000002,"15min":0.63400000000000001},"min":{"1min":0.501,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93500000000000005,"5min":1.0029999999999999,"15min":1.0629999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.69299999999999995,"15min":0.66500000000000004},"min":{"1min":0.46899999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96499999999999997,"5min":1.02,"15min":1.02},"last":0.84599999999999997}]},{"osd":2,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72099999999999997,"15min":0.70399999999999996},"min":{"1min":0.56799999999999995,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.95899999999999996,"5min":1.056,"15min":1.056},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73899999999999999,"15min":0.73099999999999998},"min":{"1min":0.46800000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.91500000000000004,"5min":1.091,"15min":1.871},"last":0.98299999999999998}]},{"osd":3,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.67900000000000005,"15min":0.65800000000000003},"min":{"1min":0.48699999999999999,"5min":0.45700000000000002,"15min":0.40899999999999997},"max":{"1min":0.93200000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70199999999999996,"15min":0.68799999999999994},"min":{"1min":0.53500000000000003,"5min":0.442,"15min":0.442},"max":{"1min":0.92200000000000004,"5min":1.014,"15min":1.014},"last":1.0469999999999999}]},{"osd":5,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60299999999999998,"15min":0.59199999999999997},"min":{"1min":0.38200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.89800000000000002,"5min":1.03,"15min":1.1799999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.60299999999999998,"15min":0.627},"min":{"1min":0.33500000000000002,"5min":0.33400000000000002,"15min":0.32300000000000001},"max":{"1min":0.878,"5min":1.05,"15min":1.79},"last":0.78400000000000003}]},{"osd":6,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61599999999999999,"15min":0.63600000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85199999999999998,"5min":1.0680000000000001,"15min":1.123},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.69099999999999995,"15min":0.70199999999999996},"min":{"1min":0.433,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94699999999999995,"5min":1.1970000000000001,"15min":1.925},"last":0.75900000000000001}]},{"osd":7,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68300000000000005,"15min":0.70599999999999996},"min":{"1min":0.45600000000000002,"5min":0.41999999999999998,"15min":0.36399999999999999},"max":{"1min":0.94099999999999995,"5min":1.1040000000000001,"15min":1.843},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.75900000000000001,"15min":0.77000000000000002},"min":{"1min":0.55000000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0149999999999999,"5min":1.1299999999999999,"15min":1.593},"last":1.101}]}]},{"osd":3,"up_from":13,"seq":55834574955,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50812,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321028,"statfs":{"total":96636764160,"available":96584732672,"internally_reserved":0,"allocated":1617920,"data_stored":747675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.441,"15min":0.42199999999999999},"min":{"1min":0.26000000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.64200000000000002,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.499,"15min":0.47599999999999998},"min":{"1min":0.32500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.65300000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.60699999999999998}]},{"osd":1,"last update":"Tue Nov 28 18:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.44600000000000001,"15min":0.45900000000000002},"min":{"1min":0.308,"5min":0.28799999999999998,"15min":0.27300000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.96099999999999997},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55600000000000005,"15min":0.51900000000000002},"min":{"1min":0.36799999999999999,"5min":0.249,"15min":0.247},"max":{"1min":0.69499999999999995,"5min":4.5599999999999996,"15min":4.5599999999999996},"last":0.52300000000000002}]},{"osd":2,"last update":"Tue Nov 28 18:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55300000000000005,"15min":0.52900000000000003},"min":{"1min":0.33600000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71499999999999997,"5min":2.48,"15min":2.48},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56100000000000005,"15min":0.55900000000000005},"min":{"1min":0.40500000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71099999999999997,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.55800000000000005}]},{"osd":4,"last update":"Tue Nov 28 18:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.88500000000000001,"15min":0.80700000000000005},"min":{"1min":0.48199999999999998,"5min":0.441,"15min":0.39800000000000002},"max":{"1min":0.95699999999999996,"5min":12.695,"15min":12.695},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.79400000000000004,"15min":0.77700000000000002},"min":{"1min":0.503,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.03,"5min":6.3250000000000002,"15min":6.3250000000000002},"last":0.77100000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.76800000000000002,"15min":0.81100000000000005},"min":{"1min":0.52400000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.5860000000000001,"5min":1.9430000000000001,"15min":2.5019999999999998},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79400000000000004,"15min":0.78100000000000003},"min":{"1min":0.58999999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.109,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.034}]},{"osd":6,"last update":"Tue Nov 28 18:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.81200000000000006,"15min":0.80400000000000005},"min":{"1min":0.46600000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1299999999999999,"5min":6.1790000000000003,"15min":6.1790000000000003},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78200000000000003,"15min":0.79500000000000004},"min":{"1min":0.54200000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0800000000000001,"5min":1.984,"15min":1.984},"last":0.90100000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.78800000000000003,"15min":0.80400000000000005},"min":{"1min":0.54500000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.997,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.83599999999999997,"15min":0.81699999999999995},"min":{"1min":0.55100000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.5449999999999999,"5min":6.2249999999999996,"15min":6.2249999999999996},"last":0.87}]}]},{"osd":2,"up_from":13,"seq":55834574956,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36928,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334912,"statfs":{"total":96636764160,"available":96598949888,"internally_reserved":0,"allocated":1622016,"data_stored":753229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.434,"15min":0.47799999999999998},"min":{"1min":0.26000000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.66500000000000004,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.46400000000000002,"15min":0.48899999999999999},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26500000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":2.0150000000000001},"last":0.378}]},{"osd":1,"last update":"Tue Nov 28 18:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47799999999999998,"15min":0.50900000000000001},"min":{"1min":0.30399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66600000000000004,"5min":0.70999999999999996,"15min":1.48},"last":0.434},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56699999999999995,"15min":0.53100000000000003},"min":{"1min":0.35899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78800000000000003,"5min":8.3539999999999992,"15min":8.3539999999999992},"last":0.51500000000000001}]},{"osd":3,"last update":"Tue Nov 28 18:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.496,"15min":0.498},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69499999999999995,"5min":3.1549999999999998,"15min":3.1549999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.56100000000000005},"min":{"1min":0.38900000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71999999999999997,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":0.53200000000000003}]},{"osd":4,"last update":"Tue Nov 28 18:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.751,"15min":0.76600000000000001},"min":{"1min":0.50700000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.337,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.76100000000000001,"15min":0.77000000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.137,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue Nov 28 18:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76000000000000001,"15min":0.76200000000000001},"min":{"1min":0.438,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.198,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.73499999999999999,"15min":0.90800000000000003},"min":{"1min":0.5,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.298,"5min":1.298,"15min":7.4820000000000002},"last":0.75}]},{"osd":6,"last update":"Tue Nov 28 18:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.84799999999999998,"15min":0.81299999999999994},"min":{"1min":0.51600000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.0429999999999999,"5min":12.019,"15min":12.019},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.81999999999999995,"15min":0.84599999999999997},"min":{"1min":0.55600000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0860000000000001,"5min":8.7400000000000002,"15min":8.7400000000000002},"last":0.73199999999999998}]},{"osd":7,"last update":"Tue Nov 28 18:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.82799999999999996,"15min":0.83399999999999996},"min":{"1min":0.54500000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2430000000000001,"5min":8.327,"15min":8.327},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.82699999999999996,"15min":0.88},"min":{"1min":0.54600000000000004,"5min":0.441,"15min":0.42199999999999999},"max":{"1min":1.1399999999999999,"5min":8.609,"15min":8.609},"last":0.82199999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41084,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330756,"statfs":{"total":96636764160,"available":96594694144,"internally_reserved":0,"allocated":1617920,"data_stored":752683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41599999999999998,"15min":0.42199999999999999},"min":{"1min":0.25600000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.70999999999999996},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.47199999999999998,"15min":0.45600000000000002},"min":{"1min":0.28100000000000003,"5min":0.27300000000000002,"15min":0.26600000000000001},"max":{"1min":0.60599999999999998,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.56799999999999995}]},{"osd":2,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44900000000000001,"15min":0.44700000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.58299999999999996,"5min":0.69499999999999995,"15min":0.75700000000000001},"last":0.505},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.49199999999999999,"15min":0.48299999999999998},"min":{"1min":0.31900000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.59999999999999998,"5min":0.73399999999999999,"15min":0.79200000000000004},"last":0.53500000000000003}]},{"osd":3,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.36699999999999999},"max":{"1min":0.77200000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.621},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.50700000000000001,"15min":0.496},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.32200000000000001},"max":{"1min":0.67100000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.60599999999999998}]},{"osd":4,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71599999999999997,"15min":0.73299999999999998},"min":{"1min":0.53000000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.876,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.755,"15min":0.76200000000000001},"min":{"1min":0.54400000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91100000000000003,"5min":1.0780000000000001,"15min":1.155},"last":1.0349999999999999}]},{"osd":5,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.69599999999999995,"15min":0.72799999999999998},"min":{"1min":0.47499999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85199999999999998,"5min":0.997,"15min":1.3799999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70399999999999996,"15min":0.72899999999999998},"min":{"1min":0.48799999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.82199999999999995,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.1779999999999999}]},{"osd":6,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.73699999999999999,"15min":0.76000000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89900000000000002,"5min":1.1279999999999999,"15min":1.2969999999999999},"last":1},{"interface":"front","average":{"1min":0.752,"5min":0.76300000000000001,"15min":0.78900000000000003},"min":{"1min":0.53400000000000003,"5min":0.495,"15min":0.495},"max":{"1min":0.89500000000000002,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.1279999999999999}]},{"osd":7,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73199999999999998,"15min":0.74299999999999999},"min":{"1min":0.54600000000000004,"5min":0.503,"15min":0.47899999999999998},"max":{"1min":0.89000000000000001,"5min":1.0669999999999999,"15min":1.2410000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73599999999999999,"15min":0.76000000000000001},"min":{"1min":0.55100000000000005,"5min":0.52300000000000002,"15min":0.48499999999999999},"max":{"1min":0.91000000000000003,"5min":1.0109999999999999,"15min":1.4239999999999999},"last":0.93400000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47220,"kb_used_data":2152,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324620,"statfs":{"total":96636764160,"available":96588410880,"internally_reserved":0,"allocated":2203648,"data_stored":1336417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.41399999999999998,"15min":0.434},"min":{"1min":0.27700000000000002,"5min":0.248,"15min":0.248},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.437},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.30199999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.71399999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.36799999999999999}]},{"osd":2,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.47799999999999998,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.318,"15min":0.30499999999999999},"max":{"1min":0.67400000000000004,"5min":0.72899999999999998,"15min":0.92900000000000005},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.48999999999999999,"15min":0.50600000000000001},"min":{"1min":0.39300000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69999999999999996,"5min":0.754,"15min":0.91200000000000003},"last":0.45700000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47399999999999998,"15min":0.48499999999999999},"min":{"1min":0.33000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70099999999999996,"5min":0.78600000000000003,"15min":0.86499999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53300000000000003,"15min":0.54600000000000004},"min":{"1min":0.36499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72999999999999998,"5min":0.78500000000000003,"15min":1.087},"last":0.58099999999999996}]},{"osd":4,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.131},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.71099999999999997,"15min":0.71799999999999997},"min":{"1min":0.55500000000000005,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94299999999999995,"5min":0.995,"15min":1.244},"last":0.72299999999999998}]},{"osd":5,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70899999999999996,"15min":0.72399999999999998},"min":{"1min":0.46999999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.92900000000000005,"5min":0.93600000000000005,"15min":1.121},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.45000000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":1.075},"last":0.60299999999999998}]},{"osd":6,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72999999999999998,"15min":0.73399999999999999},"min":{"1min":0.54700000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.407},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71799999999999997,"15min":0.73399999999999999},"min":{"1min":0.57199999999999995,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.877,"5min":0.96099999999999997,"15min":1.089},"last":0.64900000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.754,"15min":0.75600000000000001},"min":{"1min":0.60999999999999999,"5min":0.504,"15min":0.504},"max":{"1min":0.96699999999999997,"5min":1.054,"15min":1.212},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72199999999999998,"15min":0.74199999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91600000000000004,"5min":0.97699999999999998,"15min":1.0960000000000001},"last":0.74199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:32:03.587 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:32:03.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.760+0000 7f2332ddf640 1 Processor -- start 2023-11-28T18:32:03.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.760+0000 7f2332ddf640 1 -- start start 2023-11-28T18:32:03.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.760+0000 7f2332ddf640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f232c0cc760 0x7f232c164d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:03.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.760+0000 7f2332ddf640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f232c0ccc10 con 0x7f232c0caf60 2023-11-28T18:32:03.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.760+0000 7f2332ddf640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f232c06bf50 con 0x7f232c159f40 2023-11-28T18:32:03.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.760+0000 7f2332ddf640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f232c0cb3c0 con 0x7f232c0cc760 2023-11-28T18:32:03.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.760+0000 7f2323fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f232c0caf60 0x7f232c0ad850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41778/0 (socket says 172.21.15.105:41778) 2023-11-28T18:32:03.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.760+0000 7f2323fff640 1 -- 172.21.15.105:0/4179498575 learned_addr learned my addr 172.21.15.105:0/4179498575 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:32:03.762 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.760+0000 7f233137c640 1 --2- 172.21.15.105:0/4179498575 >> v2:172.21.15.148:3300/0 conn(0x7f232c0cc760 0x7f232c164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:03.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f23237fe640 1 -- 172.21.15.105:0/4179498575 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214287925 0 0) 0x7f232c0ccc10 con 0x7f232c0caf60 2023-11-28T18:32:03.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f23237fe640 1 -- 172.21.15.105:0/4179498575 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2314003530 con 0x7f232c0caf60 2023-11-28T18:32:03.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f23237fe640 1 -- 172.21.15.105:0/4179498575 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1801997786 0 0) 0x7f2314003530 con 0x7f232c0caf60 2023-11-28T18:32:03.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f23237fe640 1 -- 172.21.15.105:0/4179498575 >> v2:172.21.15.148:3300/0 conn(0x7f232c0cc760 msgr2=0x7f232c164d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:03.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f23237fe640 1 --2- 172.21.15.105:0/4179498575 >> v2:172.21.15.148:3300/0 conn(0x7f232c0cc760 0x7f232c164d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f23237fe640 1 -- 172.21.15.105:0/4179498575 >> v1:172.21.15.148:6789/0 conn(0x7f232c159f40 legacy=0x7f232c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:03.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f23237fe640 1 -- 172.21.15.105:0/4179498575 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232c15a320 con 0x7f232c0caf60 2023-11-28T18:32:03.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f23237fe640 1 -- 172.21.15.105:0/4179498575 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f231c001680 con 0x7f232c0caf60 2023-11-28T18:32:03.766 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f233137c640 1 --2- 172.21.15.105:0/4179498575 >> v2:172.21.15.148:3300/0 conn(0x7f232c0cc760 0x7f232c164d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:32:03.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f23237fe640 1 -- 172.21.15.105:0/4179498575 >> v1:172.21.15.105:6789/0 conn(0x7f232c0caf60 legacy=0x7f232c0ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:03.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f23237fe640 1 --2- 172.21.15.105:0/4179498575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2314003980 0x7f2314003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:03.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f23237fe640 1 --2- 172.21.15.105:0/4179498575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23140047d0 0x7f2314004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:03.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f23237fe640 1 --2- 172.21.15.105:0/4179498575 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2314005190 0x7f231400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:03.767 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f23237fe640 1 -- 172.21.15.105:0/4179498575 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232c0cb3c0 con 0x7f2314003980 2023-11-28T18:32:03.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f23237fe640 1 -- 172.21.15.105:0/4179498575 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231400c130 con 0x7f23140047d0 2023-11-28T18:32:03.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f23237fe640 1 -- 172.21.15.105:0/4179498575 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f231400c2f0 con 0x7f2314005190 2023-11-28T18:32:03.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f2330b7b640 1 --2- 172.21.15.105:0/4179498575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2314003980 0x7f2314003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:03.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f2323fff640 1 --2- 172.21.15.105:0/4179498575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23140047d0 0x7f2314004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:03.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f233137c640 1 --2- 172.21.15.105:0/4179498575 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2314005190 0x7f231400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:03.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f2330b7b640 1 -- 172.21.15.105:0/4179498575 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2314005190 msgr2=0x7f231400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:03.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f2330b7b640 1 --2- 172.21.15.105:0/4179498575 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2314005190 0x7f231400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f2330b7b640 1 -- 172.21.15.105:0/4179498575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23140047d0 msgr2=0x7f2314004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:03.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f2330b7b640 1 --2- 172.21.15.105:0/4179498575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23140047d0 0x7f2314004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f2330b7b640 1 -- 172.21.15.105:0/4179498575 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2314003530 con 0x7f2314003980 2023-11-28T18:32:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.764+0000 7f233137c640 1 -- 172.21.15.105:0/4179498575 reap_dead start 2023-11-28T18:32:03.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.768+0000 7f2330b7b640 1 --2- 172.21.15.105:0/4179498575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2314003980 0x7f2314003da0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f2328001780 tx=0x7f232801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d14695399cd303c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:03.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.768+0000 7f23237fe640 1 -- 172.21.15.105:0/4179498575 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23280052d0 con 0x7f2314003980 2023-11-28T18:32:03.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.768+0000 7f23237fe640 1 -- 172.21.15.105:0/4179498575 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23280a7050 con 0x7f2314003980 2023-11-28T18:32:03.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.768+0000 7f23237fe640 1 -- 172.21.15.105:0/4179498575 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328003600 con 0x7f2314003980 2023-11-28T18:32:03.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.768+0000 7f2332ddf640 1 -- 172.21.15.105:0/4179498575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2314003980 msgr2=0x7f2314003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:03.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.768+0000 7f2332ddf640 1 --2- 172.21.15.105:0/4179498575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2314003980 0x7f2314003da0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f2328001780 tx=0x7f232801c3d0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.768+0000 7f2332ddf640 1 -- 172.21.15.105:0/4179498575 shutdown_connections 2023-11-28T18:32:03.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.768+0000 7f2332ddf640 1 --2- 172.21.15.105:0/4179498575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2314003980 0x7f2314003da0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.768+0000 7f2332ddf640 1 -- 172.21.15.105:0/4179498575 >> 172.21.15.105:0/4179498575 conn(0x7f232c0ceb90 msgr2=0x7f232c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:03.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.768+0000 7f2332ddf640 1 -- 172.21.15.105:0/4179498575 shutdown_connections 2023-11-28T18:32:03.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.768+0000 7f2332ddf640 1 -- 172.21.15.105:0/4179498575 wait complete. 2023-11-28T18:32:03.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.768+0000 7f2332ddf640 1 Processor -- start 2023-11-28T18:32:03.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.768+0000 7f2332ddf640 1 -- start start 2023-11-28T18:32:03.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f2332ddf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2314003980 0x7f232c152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:03.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f2332ddf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f232c1f4f50 0x7f232c15ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:03.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f2332ddf640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f232c1f53d0 0x7f232c153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:03.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f2332ddf640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232c15b6c0 con 0x7f2314003980 2023-11-28T18:32:03.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f2332ddf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232c099dc0 con 0x7f232c1f4f50 2023-11-28T18:32:03.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f2332ddf640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f232c09a0a0 con 0x7f232c1f53d0 2023-11-28T18:32:03.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f2330b7b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2314003980 0x7f232c152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:03.776 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f2330b7b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2314003980 0x7f232c152510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58526/0 (socket says 172.21.15.105:58526) 2023-11-28T18:32:03.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f2330b7b640 1 -- 172.21.15.105:0/139368901 learned_addr learned my addr 172.21.15.105:0/139368901 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:03.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f233137c640 1 --2- 172.21.15.105:0/139368901 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f232c1f53d0 0x7f232c153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:03.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f2323fff640 1 --2- 172.21.15.105:0/139368901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f232c1f4f50 0x7f232c15ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:03.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f2330b7b640 1 -- 172.21.15.105:0/139368901 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f232c1f53d0 msgr2=0x7f232c153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:03.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f2330b7b640 1 --2- 172.21.15.105:0/139368901 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f232c1f53d0 0x7f232c153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f2330b7b640 1 -- 172.21.15.105:0/139368901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f232c1f4f50 msgr2=0x7f232c15ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:03.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f2330b7b640 1 --2- 172.21.15.105:0/139368901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f232c1f4f50 0x7f232c15ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f2330b7b640 1 -- 172.21.15.105:0/139368901 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232c09a380 con 0x7f2314003980 2023-11-28T18:32:03.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f2330b7b640 1 --2- 172.21.15.105:0/139368901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2314003980 0x7f232c152510 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f232801c200 tx=0x7f2314005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:03.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f23217fa640 1 -- 172.21.15.105:0/139368901 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23280ba070 con 0x7f2314003980 2023-11-28T18:32:03.777 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f23217fa640 1 -- 172.21.15.105:0/139368901 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23280a4e90 con 0x7f2314003980 2023-11-28T18:32:03.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f2332ddf640 1 -- 172.21.15.105:0/139368901 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f232c09a660 con 0x7f2314003980 2023-11-28T18:32:03.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f23217fa640 1 -- 172.21.15.105:0/139368901 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328007280 con 0x7f2314003980 2023-11-28T18:32:03.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f2332ddf640 1 -- 172.21.15.105:0/139368901 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f232c152a50 con 0x7f2314003980 2023-11-28T18:32:03.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f2332ddf640 1 -- 172.21.15.105:0/139368901 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f232c09a8a0 con 0x7f2314003980 2023-11-28T18:32:03.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f23217fa640 1 -- 172.21.15.105:0/139368901 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f23280073e0 con 0x7f2314003980 2023-11-28T18:32:03.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.772+0000 7f23217fa640 1 --2- 172.21.15.105:0/139368901 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2318066900 0x7f2318068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:03.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.776+0000 7f23217fa640 1 -- 172.21.15.105:0/139368901 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f232811ddb0 con 0x7f2314003980 2023-11-28T18:32:03.778 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.776+0000 7f2323fff640 1 --2- 172.21.15.105:0/139368901 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2318066900 0x7f2318068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:03.779 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.776+0000 7f2323fff640 1 --2- 172.21.15.105:0/139368901 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2318066900 0x7f2318068dc0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f232c074820 tx=0x7f231c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:03.779 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.776+0000 7f23217fa640 1 -- 172.21.15.105:0/139368901 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f23280ee8a0 con 0x7f2314003980 2023-11-28T18:32:03.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.928+0000 7f2332ddf640 1 -- 172.21.15.105:0/139368901 --> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f232c15d810 con 0x7f2318066900 2023-11-28T18:32:03.937 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.936+0000 7f23217fa640 1 -- 172.21.15.105:0/139368901 <== mgr.4104 v2:172.21.15.105:6832/1892510418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289657 (secure 0 0 0) 0x7f232c15d810 con 0x7f2318066900 2023-11-28T18:32:03.941 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:32:03.943 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-11-28T18:32:03.944 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.940+0000 7f2332ddf640 1 -- 172.21.15.105:0/139368901 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2318066900 msgr2=0x7f2318068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.940+0000 7f2332ddf640 1 --2- 172.21.15.105:0/139368901 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2318066900 0x7f2318068dc0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f232c074820 tx=0x7f231c009040 comp rx=0 tx=0).stop 2023-11-28T18:32:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.940+0000 7f2332ddf640 1 -- 172.21.15.105:0/139368901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2314003980 msgr2=0x7f232c152510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.940+0000 7f2332ddf640 1 --2- 172.21.15.105:0/139368901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2314003980 0x7f232c152510 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f232801c200 tx=0x7f2314005800 comp rx=0 tx=0).stop 2023-11-28T18:32:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.940+0000 7f2332ddf640 1 -- 172.21.15.105:0/139368901 shutdown_connections 2023-11-28T18:32:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.940+0000 7f2332ddf640 1 --2- 172.21.15.105:0/139368901 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f232c1f53d0 0x7f232c153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.940+0000 7f2332ddf640 1 --2- 172.21.15.105:0/139368901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f232c1f4f50 0x7f232c15ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.940+0000 7f2332ddf640 1 --2- 172.21.15.105:0/139368901 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f2318066900 0x7f2318068dc0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.940+0000 7f2332ddf640 1 --2- 172.21.15.105:0/139368901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2314003980 0x7f232c152510 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:03.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.940+0000 7f2332ddf640 1 -- 172.21.15.105:0/139368901 >> 172.21.15.105:0/139368901 conn(0x7f232c0ceb90 msgr2=0x7f232c0cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:03.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.940+0000 7f2332ddf640 1 -- 172.21.15.105:0/139368901 shutdown_connections 2023-11-28T18:32:03.946 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:03.940+0000 7f2332ddf640 1 -- 172.21.15.105:0/139368901 wait complete. 2023-11-28T18:32:03.960 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2023-11-28T18:32:03.379889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":410372,"kb_used_data":13772,"kb_used_omap":12,"kb_used_meta":396467,"kb_avail":754564348,"statfs":{"total":773094113280,"available":772673892352,"internally_reserved":0,"allocated":14102528,"data_stored":7173360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":405982770},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.940961"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.875396+0000","last_change":"2023-11-28T18:31:59.482608+0000","last_active":"2023-11-28T18:32:00.875396+0000","last_peered":"2023-11-28T18:32:00.875396+0000","last_clean":"2023-11-28T18:32:00.875396+0000","last_became_active":"2023-11-28T18:31:59.482352+0000","last_became_peered":"2023-11-28T18:31:59.482352+0000","last_unstale":"2023-11-28T18:32:00.875396+0000","last_undegraded":"2023-11-28T18:32:00.875396+0000","last_fullsized":"2023-11-28T18:32:00.875396+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:23:03.934139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877742+0000","last_change":"2023-11-28T18:31:59.487336+0000","last_active":"2023-11-28T18:32:00.877742+0000","last_peered":"2023-11-28T18:32:00.877742+0000","last_clean":"2023-11-28T18:32:00.877742+0000","last_became_active":"2023-11-28T18:31:59.486133+0000","last_became_peered":"2023-11-28T18:31:59.486133+0000","last_unstale":"2023-11-28T18:32:00.877742+0000","last_undegraded":"2023-11-28T18:32:00.877742+0000","last_fullsized":"2023-11-28T18:32:00.877742+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:17.025975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460893+0000","last_change":"2023-11-28T18:31:59.467470+0000","last_active":"2023-11-28T18:32:00.460893+0000","last_peered":"2023-11-28T18:32:00.460893+0000","last_clean":"2023-11-28T18:32:00.460893+0000","last_became_active":"2023-11-28T18:31:59.467185+0000","last_became_peered":"2023-11-28T18:31:59.467185+0000","last_unstale":"2023-11-28T18:32:00.460893+0000","last_undegraded":"2023-11-28T18:32:00.460893+0000","last_fullsized":"2023-11-28T18:32:00.460893+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:18:53.760883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877500+0000","last_change":"2023-11-28T18:31:59.489101+0000","last_active":"2023-11-28T18:32:00.877500+0000","last_peered":"2023-11-28T18:32:00.877500+0000","last_clean":"2023-11-28T18:32:00.877500+0000","last_became_active":"2023-11-28T18:31:59.488768+0000","last_became_peered":"2023-11-28T18:31:59.488768+0000","last_unstale":"2023-11-28T18:32:00.877500+0000","last_undegraded":"2023-11-28T18:32:00.877500+0000","last_fullsized":"2023-11-28T18:32:00.877500+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:06:15.465659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.457918+0000","last_change":"2023-11-28T18:31:59.461449+0000","last_active":"2023-11-28T18:32:00.457918+0000","last_peered":"2023-11-28T18:32:00.457918+0000","last_clean":"2023-11-28T18:32:00.457918+0000","last_became_active":"2023-11-28T18:31:59.461161+0000","last_became_peered":"2023-11-28T18:31:59.461161+0000","last_unstale":"2023-11-28T18:32:00.457918+0000","last_undegraded":"2023-11-28T18:32:00.457918+0000","last_fullsized":"2023-11-28T18:32:00.457918+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:02.710984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459584+0000","last_change":"2023-11-28T18:31:59.474620+0000","last_active":"2023-11-28T18:32:00.459584+0000","last_peered":"2023-11-28T18:32:00.459584+0000","last_clean":"2023-11-28T18:32:00.459584+0000","last_became_active":"2023-11-28T18:31:59.474317+0000","last_became_peered":"2023-11-28T18:31:59.474317+0000","last_unstale":"2023-11-28T18:32:00.459584+0000","last_undegraded":"2023-11-28T18:32:00.459584+0000","last_fullsized":"2023-11-28T18:32:00.459584+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:50:40.320398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876129+0000","last_change":"2023-11-28T18:31:59.497301+0000","last_active":"2023-11-28T18:32:00.876129+0000","last_peered":"2023-11-28T18:32:00.876129+0000","last_clean":"2023-11-28T18:32:00.876129+0000","last_became_active":"2023-11-28T18:31:59.496974+0000","last_became_peered":"2023-11-28T18:31:59.496974+0000","last_unstale":"2023-11-28T18:32:00.876129+0000","last_undegraded":"2023-11-28T18:32:00.876129+0000","last_fullsized":"2023-11-28T18:32:00.876129+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:14:51.587300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.875993+0000","last_change":"2023-11-28T18:31:59.483580+0000","last_active":"2023-11-28T18:32:00.875993+0000","last_peered":"2023-11-28T18:32:00.875993+0000","last_clean":"2023-11-28T18:32:00.875993+0000","last_became_active":"2023-11-28T18:31:59.483263+0000","last_became_peered":"2023-11-28T18:31:59.483263+0000","last_unstale":"2023-11-28T18:32:00.875993+0000","last_undegraded":"2023-11-28T18:32:00.875993+0000","last_fullsized":"2023-11-28T18:32:00.875993+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:35:25.019756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459930+0000","last_change":"2023-11-28T18:31:59.477152+0000","last_active":"2023-11-28T18:32:00.459930+0000","last_peered":"2023-11-28T18:32:00.459930+0000","last_clean":"2023-11-28T18:32:00.459930+0000","last_became_active":"2023-11-28T18:31:59.476464+0000","last_became_peered":"2023-11-28T18:31:59.476464+0000","last_unstale":"2023-11-28T18:32:00.459930+0000","last_undegraded":"2023-11-28T18:32:00.459930+0000","last_fullsized":"2023-11-28T18:32:00.459930+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:03.036695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.875086+0000","last_change":"2023-11-28T18:31:59.479082+0000","last_active":"2023-11-28T18:32:00.875086+0000","last_peered":"2023-11-28T18:32:00.875086+0000","last_clean":"2023-11-28T18:32:00.875086+0000","last_became_active":"2023-11-28T18:31:59.478786+0000","last_became_peered":"2023-11-28T18:31:59.478786+0000","last_unstale":"2023-11-28T18:32:00.875086+0000","last_undegraded":"2023-11-28T18:32:00.875086+0000","last_fullsized":"2023-11-28T18:32:00.875086+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:24:27.946466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877831+0000","last_change":"2023-11-28T18:31:59.489605+0000","last_active":"2023-11-28T18:32:00.877831+0000","last_peered":"2023-11-28T18:32:00.877831+0000","last_clean":"2023-11-28T18:32:00.877831+0000","last_became_active":"2023-11-28T18:31:59.489340+0000","last_became_peered":"2023-11-28T18:31:59.489340+0000","last_unstale":"2023-11-28T18:32:00.877831+0000","last_undegraded":"2023-11-28T18:32:00.877831+0000","last_fullsized":"2023-11-28T18:32:00.877831+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:29:27.272539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.875294+0000","last_change":"2023-11-28T18:31:59.476827+0000","last_active":"2023-11-28T18:32:00.875294+0000","last_peered":"2023-11-28T18:32:00.875294+0000","last_clean":"2023-11-28T18:32:00.875294+0000","last_became_active":"2023-11-28T18:31:59.475744+0000","last_became_peered":"2023-11-28T18:31:59.475744+0000","last_unstale":"2023-11-28T18:32:00.875294+0000","last_undegraded":"2023-11-28T18:32:00.875294+0000","last_fullsized":"2023-11-28T18:32:00.875294+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:05.071170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459763+0000","last_change":"2023-11-28T18:31:59.474829+0000","last_active":"2023-11-28T18:32:00.459763+0000","last_peered":"2023-11-28T18:32:00.459763+0000","last_clean":"2023-11-28T18:32:00.459763+0000","last_became_active":"2023-11-28T18:31:59.474547+0000","last_became_peered":"2023-11-28T18:31:59.474547+0000","last_unstale":"2023-11-28T18:32:00.459763+0000","last_undegraded":"2023-11-28T18:32:00.459763+0000","last_fullsized":"2023-11-28T18:32:00.459763+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:34:42.082662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.874837+0000","last_change":"2023-11-28T18:31:59.485274+0000","last_active":"2023-11-28T18:32:00.874837+0000","last_peered":"2023-11-28T18:32:00.874837+0000","last_clean":"2023-11-28T18:32:00.874837+0000","last_became_active":"2023-11-28T18:31:59.484477+0000","last_became_peered":"2023-11-28T18:31:59.484477+0000","last_unstale":"2023-11-28T18:32:00.874837+0000","last_undegraded":"2023-11-28T18:32:00.874837+0000","last_fullsized":"2023-11-28T18:32:00.874837+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:08:01.303585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.874566+0000","last_change":"2023-11-28T18:31:59.482115+0000","last_active":"2023-11-28T18:32:00.874566+0000","last_peered":"2023-11-28T18:32:00.874566+0000","last_clean":"2023-11-28T18:32:00.874566+0000","last_became_active":"2023-11-28T18:31:59.481654+0000","last_became_peered":"2023-11-28T18:31:59.481654+0000","last_unstale":"2023-11-28T18:32:00.874566+0000","last_undegraded":"2023-11-28T18:32:00.874566+0000","last_fullsized":"2023-11-28T18:32:00.874566+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:52:02.359432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459421+0000","last_change":"2023-11-28T18:31:59.468289+0000","last_active":"2023-11-28T18:32:00.459421+0000","last_peered":"2023-11-28T18:32:00.459421+0000","last_clean":"2023-11-28T18:32:00.459421+0000","last_became_active":"2023-11-28T18:31:59.467344+0000","last_became_peered":"2023-11-28T18:31:59.467344+0000","last_unstale":"2023-11-28T18:32:00.459421+0000","last_undegraded":"2023-11-28T18:32:00.459421+0000","last_fullsized":"2023-11-28T18:32:00.459421+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:17.025975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459936+0000","last_change":"2023-11-28T18:31:59.464786+0000","last_active":"2023-11-28T18:32:00.459936+0000","last_peered":"2023-11-28T18:32:00.459936+0000","last_clean":"2023-11-28T18:32:00.459936+0000","last_became_active":"2023-11-28T18:31:59.464533+0000","last_became_peered":"2023-11-28T18:31:59.464533+0000","last_unstale":"2023-11-28T18:32:00.459936+0000","last_undegraded":"2023-11-28T18:32:00.459936+0000","last_fullsized":"2023-11-28T18:32:00.459936+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:12:03.671740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.458141+0000","last_change":"2023-11-28T18:31:59.461521+0000","last_active":"2023-11-28T18:32:00.458141+0000","last_peered":"2023-11-28T18:32:00.458141+0000","last_clean":"2023-11-28T18:32:00.458141+0000","last_became_active":"2023-11-28T18:31:59.461150+0000","last_became_peered":"2023-11-28T18:31:59.461150+0000","last_unstale":"2023-11-28T18:32:00.458141+0000","last_undegraded":"2023-11-28T18:32:00.458141+0000","last_fullsized":"2023-11-28T18:32:00.458141+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:30.445631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877461+0000","last_change":"2023-11-28T18:31:59.484931+0000","last_active":"2023-11-28T18:32:00.877461+0000","last_peered":"2023-11-28T18:32:00.877461+0000","last_clean":"2023-11-28T18:32:00.877461+0000","last_became_active":"2023-11-28T18:31:59.484652+0000","last_became_peered":"2023-11-28T18:31:59.484652+0000","last_unstale":"2023-11-28T18:32:00.877461+0000","last_undegraded":"2023-11-28T18:32:00.877461+0000","last_fullsized":"2023-11-28T18:32:00.877461+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:31:42.639872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.457911+0000","last_change":"2023-11-28T18:31:59.469465+0000","last_active":"2023-11-28T18:32:00.457911+0000","last_peered":"2023-11-28T18:32:00.457911+0000","last_clean":"2023-11-28T18:32:00.457911+0000","last_became_active":"2023-11-28T18:31:59.469206+0000","last_became_peered":"2023-11-28T18:31:59.469206+0000","last_unstale":"2023-11-28T18:32:00.457911+0000","last_undegraded":"2023-11-28T18:32:00.457911+0000","last_fullsized":"2023-11-28T18:32:00.457911+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:12:03.671740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877252+0000","last_change":"2023-11-28T18:31:59.487946+0000","last_active":"2023-11-28T18:32:00.877252+0000","last_peered":"2023-11-28T18:32:00.877252+0000","last_clean":"2023-11-28T18:32:00.877252+0000","last_became_active":"2023-11-28T18:31:59.487660+0000","last_became_peered":"2023-11-28T18:31:59.487660+0000","last_unstale":"2023-11-28T18:32:00.877252+0000","last_undegraded":"2023-11-28T18:32:00.877252+0000","last_fullsized":"2023-11-28T18:32:00.877252+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:07:10.812548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.874627+0000","last_change":"2023-11-28T18:31:59.477083+0000","last_active":"2023-11-28T18:32:00.874627+0000","last_peered":"2023-11-28T18:32:00.874627+0000","last_clean":"2023-11-28T18:32:00.874627+0000","last_became_active":"2023-11-28T18:31:59.476782+0000","last_became_peered":"2023-11-28T18:31:59.476782+0000","last_unstale":"2023-11-28T18:32:00.874627+0000","last_undegraded":"2023-11-28T18:32:00.874627+0000","last_fullsized":"2023-11-28T18:32:00.874627+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:47:03.620849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876803+0000","last_change":"2023-11-28T18:31:59.487498+0000","last_active":"2023-11-28T18:32:00.876803+0000","last_peered":"2023-11-28T18:32:00.876803+0000","last_clean":"2023-11-28T18:32:00.876803+0000","last_became_active":"2023-11-28T18:31:59.487193+0000","last_became_peered":"2023-11-28T18:31:59.487193+0000","last_unstale":"2023-11-28T18:32:00.876803+0000","last_undegraded":"2023-11-28T18:32:00.876803+0000","last_fullsized":"2023-11-28T18:32:00.876803+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:15:07.840277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459553+0000","last_change":"2023-11-28T18:31:59.475913+0000","last_active":"2023-11-28T18:32:00.459553+0000","last_peered":"2023-11-28T18:32:00.459553+0000","last_clean":"2023-11-28T18:32:00.459553+0000","last_became_active":"2023-11-28T18:31:59.475635+0000","last_became_peered":"2023-11-28T18:31:59.475635+0000","last_unstale":"2023-11-28T18:32:00.459553+0000","last_undegraded":"2023-11-28T18:32:00.459553+0000","last_fullsized":"2023-11-28T18:32:00.459553+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:29:27.272539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460844+0000","last_change":"2023-11-28T18:31:59.474749+0000","last_active":"2023-11-28T18:32:00.460844+0000","last_peered":"2023-11-28T18:32:00.460844+0000","last_clean":"2023-11-28T18:32:00.460844+0000","last_became_active":"2023-11-28T18:31:59.474471+0000","last_became_peered":"2023-11-28T18:31:59.474471+0000","last_unstale":"2023-11-28T18:32:00.460844+0000","last_undegraded":"2023-11-28T18:32:00.460844+0000","last_fullsized":"2023-11-28T18:32:00.460844+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:15:07.840277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460930+0000","last_change":"2023-11-28T18:31:59.480362+0000","last_active":"2023-11-28T18:32:00.460930+0000","last_peered":"2023-11-28T18:32:00.460930+0000","last_clean":"2023-11-28T18:32:00.460930+0000","last_became_active":"2023-11-28T18:31:59.480066+0000","last_became_peered":"2023-11-28T18:31:59.480066+0000","last_unstale":"2023-11-28T18:32:00.460930+0000","last_undegraded":"2023-11-28T18:32:00.460930+0000","last_fullsized":"2023-11-28T18:32:00.460930+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:06:15.465659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877478+0000","last_change":"2023-11-28T18:31:59.484372+0000","last_active":"2023-11-28T18:32:00.877478+0000","last_peered":"2023-11-28T18:32:00.877478+0000","last_clean":"2023-11-28T18:32:00.877478+0000","last_became_active":"2023-11-28T18:31:59.484084+0000","last_became_peered":"2023-11-28T18:31:59.484084+0000","last_unstale":"2023-11-28T18:32:00.877478+0000","last_undegraded":"2023-11-28T18:32:00.877478+0000","last_fullsized":"2023-11-28T18:32:00.877478+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:50:28.561689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.874869+0000","last_change":"2023-11-28T18:31:59.473971+0000","last_active":"2023-11-28T18:32:00.874869+0000","last_peered":"2023-11-28T18:32:00.874869+0000","last_clean":"2023-11-28T18:32:00.874869+0000","last_became_active":"2023-11-28T18:31:59.473711+0000","last_became_peered":"2023-11-28T18:31:59.473711+0000","last_unstale":"2023-11-28T18:32:00.874869+0000","last_undegraded":"2023-11-28T18:32:00.874869+0000","last_fullsized":"2023-11-28T18:32:00.874869+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:59:50.018736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460520+0000","last_change":"2023-11-28T18:31:59.472923+0000","last_active":"2023-11-28T18:32:00.460520+0000","last_peered":"2023-11-28T18:32:00.460520+0000","last_clean":"2023-11-28T18:32:00.460520+0000","last_became_active":"2023-11-28T18:31:59.472516+0000","last_became_peered":"2023-11-28T18:31:59.472516+0000","last_unstale":"2023-11-28T18:32:00.460520+0000","last_undegraded":"2023-11-28T18:32:00.460520+0000","last_fullsized":"2023-11-28T18:32:00.460520+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:01:47.297968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460787+0000","last_change":"2023-11-28T18:31:59.476734+0000","last_active":"2023-11-28T18:32:00.460787+0000","last_peered":"2023-11-28T18:32:00.460787+0000","last_clean":"2023-11-28T18:32:00.460787+0000","last_became_active":"2023-11-28T18:31:59.476120+0000","last_became_peered":"2023-11-28T18:31:59.476120+0000","last_unstale":"2023-11-28T18:32:00.460787+0000","last_undegraded":"2023-11-28T18:32:00.460787+0000","last_fullsized":"2023-11-28T18:32:00.460787+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:21:06.605370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876107+0000","last_change":"2023-11-28T18:31:59.467896+0000","last_active":"2023-11-28T18:32:00.876107+0000","last_peered":"2023-11-28T18:32:00.876107+0000","last_clean":"2023-11-28T18:32:00.876107+0000","last_became_active":"2023-11-28T18:31:59.467567+0000","last_became_peered":"2023-11-28T18:31:59.467567+0000","last_unstale":"2023-11-28T18:32:00.876107+0000","last_undegraded":"2023-11-28T18:32:00.876107+0000","last_fullsized":"2023-11-28T18:32:00.876107+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:51:51.574894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.874430+0000","last_change":"2023-11-28T18:31:59.485125+0000","last_active":"2023-11-28T18:32:00.874430+0000","last_peered":"2023-11-28T18:32:00.874430+0000","last_clean":"2023-11-28T18:32:00.874430+0000","last_became_active":"2023-11-28T18:31:59.484787+0000","last_became_peered":"2023-11-28T18:31:59.484787+0000","last_unstale":"2023-11-28T18:32:00.874430+0000","last_undegraded":"2023-11-28T18:32:00.874430+0000","last_fullsized":"2023-11-28T18:32:00.874430+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:19:15.699737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876236+0000","last_change":"2023-11-28T18:31:59.476809+0000","last_active":"2023-11-28T18:32:00.876236+0000","last_peered":"2023-11-28T18:32:00.876236+0000","last_clean":"2023-11-28T18:32:00.876236+0000","last_became_active":"2023-11-28T18:31:59.476498+0000","last_became_peered":"2023-11-28T18:31:59.476498+0000","last_unstale":"2023-11-28T18:32:00.876236+0000","last_undegraded":"2023-11-28T18:32:00.876236+0000","last_fullsized":"2023-11-28T18:32:00.876236+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:37:13.925200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"128'4","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460805+0000","last_change":"2023-11-28T18:31:58.463033+0000","last_active":"2023-11-28T18:32:00.460805+0000","last_peered":"2023-11-28T18:32:00.460805+0000","last_clean":"2023-11-28T18:32:00.460805+0000","last_became_active":"2023-11-28T18:31:58.462622+0000","last_became_peered":"2023-11-28T18:31:58.462622+0000","last_unstale":"2023-11-28T18:32:00.460805+0000","last_undegraded":"2023-11-28T18:32:00.460805+0000","last_fullsized":"2023-11-28T18:32:00.460805+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:55:33.705881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.874787+0000","last_change":"2023-11-28T18:31:59.479323+0000","last_active":"2023-11-28T18:32:00.874787+0000","last_peered":"2023-11-28T18:32:00.874787+0000","last_clean":"2023-11-28T18:32:00.874787+0000","last_became_active":"2023-11-28T18:31:59.479107+0000","last_became_peered":"2023-11-28T18:31:59.479107+0000","last_unstale":"2023-11-28T18:32:00.874787+0000","last_undegraded":"2023-11-28T18:32:00.874787+0000","last_fullsized":"2023-11-28T18:32:00.874787+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:18:44.496924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"128'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.458252+0000","last_change":"2023-11-28T18:31:58.458546+0000","last_active":"2023-11-28T18:32:00.458252+0000","last_peered":"2023-11-28T18:32:00.458252+0000","last_clean":"2023-11-28T18:32:00.458252+0000","last_became_active":"2023-11-28T18:31:58.458067+0000","last_became_peered":"2023-11-28T18:31:58.458067+0000","last_unstale":"2023-11-28T18:32:00.458252+0000","last_undegraded":"2023-11-28T18:32:00.458252+0000","last_fullsized":"2023-11-28T18:32:00.458252+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:06:09.810035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877271+0000","last_change":"2023-11-28T18:31:59.487976+0000","last_active":"2023-11-28T18:32:00.877271+0000","last_peered":"2023-11-28T18:32:00.877271+0000","last_clean":"2023-11-28T18:32:00.877271+0000","last_became_active":"2023-11-28T18:31:59.487689+0000","last_became_peered":"2023-11-28T18:31:59.487689+0000","last_unstale":"2023-11-28T18:32:00.877271+0000","last_undegraded":"2023-11-28T18:32:00.877271+0000","last_fullsized":"2023-11-28T18:32:00.877271+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:18:53.760883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"128'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877585+0000","last_change":"2023-11-28T18:31:58.468883+0000","last_active":"2023-11-28T18:32:00.877585+0000","last_peered":"2023-11-28T18:32:00.877585+0000","last_clean":"2023-11-28T18:32:00.877585+0000","last_became_active":"2023-11-28T18:31:58.468584+0000","last_became_peered":"2023-11-28T18:31:58.468584+0000","last_unstale":"2023-11-28T18:32:00.877585+0000","last_undegraded":"2023-11-28T18:32:00.877585+0000","last_fullsized":"2023-11-28T18:32:00.877585+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:54:57.441652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.874847+0000","last_change":"2023-11-28T18:31:59.488944+0000","last_active":"2023-11-28T18:32:00.874847+0000","last_peered":"2023-11-28T18:32:00.874847+0000","last_clean":"2023-11-28T18:32:00.874847+0000","last_became_active":"2023-11-28T18:31:59.488661+0000","last_became_peered":"2023-11-28T18:31:59.488661+0000","last_unstale":"2023-11-28T18:32:00.874847+0000","last_undegraded":"2023-11-28T18:32:00.874847+0000","last_fullsized":"2023-11-28T18:32:00.874847+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:49:19.043247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"128'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877198+0000","last_change":"2023-11-28T18:31:58.469430+0000","last_active":"2023-11-28T18:32:00.877198+0000","last_peered":"2023-11-28T18:32:00.877198+0000","last_clean":"2023-11-28T18:32:00.877198+0000","last_became_active":"2023-11-28T18:31:58.469142+0000","last_became_peered":"2023-11-28T18:31:58.469142+0000","last_unstale":"2023-11-28T18:32:00.877198+0000","last_undegraded":"2023-11-28T18:32:00.877198+0000","last_fullsized":"2023-11-28T18:32:00.877198+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:34:41.075893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876755+0000","last_change":"2023-11-28T18:31:59.467964+0000","last_active":"2023-11-28T18:32:00.876755+0000","last_peered":"2023-11-28T18:32:00.876755+0000","last_clean":"2023-11-28T18:32:00.876755+0000","last_became_active":"2023-11-28T18:31:59.467582+0000","last_became_peered":"2023-11-28T18:31:59.467582+0000","last_unstale":"2023-11-28T18:32:00.876755+0000","last_undegraded":"2023-11-28T18:32:00.876755+0000","last_fullsized":"2023-11-28T18:32:00.876755+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:01:47.297968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"128'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876552+0000","last_change":"2023-11-28T18:31:58.466580+0000","last_active":"2023-11-28T18:32:00.876552+0000","last_peered":"2023-11-28T18:32:00.876552+0000","last_clean":"2023-11-28T18:32:00.876552+0000","last_became_active":"2023-11-28T18:31:58.466314+0000","last_became_peered":"2023-11-28T18:31:58.466314+0000","last_unstale":"2023-11-28T18:32:00.876552+0000","last_undegraded":"2023-11-28T18:32:00.876552+0000","last_fullsized":"2023-11-28T18:32:00.876552+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:20.480341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876846+0000","last_change":"2023-11-28T18:31:59.485194+0000","last_active":"2023-11-28T18:32:00.876846+0000","last_peered":"2023-11-28T18:32:00.876846+0000","last_clean":"2023-11-28T18:32:00.876846+0000","last_became_active":"2023-11-28T18:31:59.484777+0000","last_became_peered":"2023-11-28T18:31:59.484777+0000","last_unstale":"2023-11-28T18:32:00.876846+0000","last_undegraded":"2023-11-28T18:32:00.876846+0000","last_fullsized":"2023-11-28T18:32:00.876846+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:03.287701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"128'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.457720+0000","last_change":"2023-11-28T18:31:58.462756+0000","last_active":"2023-11-28T18:32:00.457720+0000","last_peered":"2023-11-28T18:32:00.457720+0000","last_clean":"2023-11-28T18:32:00.457720+0000","last_became_active":"2023-11-28T18:31:58.462493+0000","last_became_peered":"2023-11-28T18:31:58.462493+0000","last_unstale":"2023-11-28T18:32:00.457720+0000","last_undegraded":"2023-11-28T18:32:00.457720+0000","last_fullsized":"2023-11-28T18:32:00.457720+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:21:05.598600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.458185+0000","last_change":"2023-11-28T18:31:59.475931+0000","last_active":"2023-11-28T18:32:00.458185+0000","last_peered":"2023-11-28T18:32:00.458185+0000","last_clean":"2023-11-28T18:32:00.458185+0000","last_became_active":"2023-11-28T18:31:59.475635+0000","last_became_peered":"2023-11-28T18:31:59.475635+0000","last_unstale":"2023-11-28T18:32:00.458185+0000","last_undegraded":"2023-11-28T18:32:00.458185+0000","last_fullsized":"2023-11-28T18:32:00.458185+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:32:47.473720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459438+0000","last_change":"2023-11-28T18:31:58.456222+0000","last_active":"2023-11-28T18:32:00.459438+0000","last_peered":"2023-11-28T18:32:00.459438+0000","last_clean":"2023-11-28T18:32:00.459438+0000","last_became_active":"2023-11-28T18:31:58.455955+0000","last_became_peered":"2023-11-28T18:31:58.455955+0000","last_unstale":"2023-11-28T18:32:00.459438+0000","last_undegraded":"2023-11-28T18:32:00.459438+0000","last_fullsized":"2023-11-28T18:32:00.459438+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:02.280932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877341+0000","last_change":"2023-11-28T18:31:59.488135+0000","last_active":"2023-11-28T18:32:00.877341+0000","last_peered":"2023-11-28T18:32:00.877341+0000","last_clean":"2023-11-28T18:32:00.877341+0000","last_became_active":"2023-11-28T18:31:59.487860+0000","last_became_peered":"2023-11-28T18:31:59.487860+0000","last_unstale":"2023-11-28T18:32:00.877341+0000","last_undegraded":"2023-11-28T18:32:00.877341+0000","last_fullsized":"2023-11-28T18:32:00.877341+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:38:56.438986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460461+0000","last_change":"2023-11-28T18:31:58.452319+0000","last_active":"2023-11-28T18:32:00.460461+0000","last_peered":"2023-11-28T18:32:00.460461+0000","last_clean":"2023-11-28T18:32:00.460461+0000","last_became_active":"2023-11-28T18:31:58.452038+0000","last_became_peered":"2023-11-28T18:31:58.452038+0000","last_unstale":"2023-11-28T18:32:00.460461+0000","last_undegraded":"2023-11-28T18:32:00.460461+0000","last_fullsized":"2023-11-28T18:32:00.460461+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:08.297316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.874654+0000","last_change":"2023-11-28T18:31:59.482080+0000","last_active":"2023-11-28T18:32:00.874654+0000","last_peered":"2023-11-28T18:32:00.874654+0000","last_clean":"2023-11-28T18:32:00.874654+0000","last_became_active":"2023-11-28T18:31:59.481289+0000","last_became_peered":"2023-11-28T18:31:59.481289+0000","last_unstale":"2023-11-28T18:32:00.874654+0000","last_undegraded":"2023-11-28T18:32:00.874654+0000","last_fullsized":"2023-11-28T18:32:00.874654+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:57:32.556863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877793+0000","last_change":"2023-11-28T18:31:58.463931+0000","last_active":"2023-11-28T18:32:00.877793+0000","last_peered":"2023-11-28T18:32:00.877793+0000","last_clean":"2023-11-28T18:32:00.877793+0000","last_became_active":"2023-11-28T18:31:58.463414+0000","last_became_peered":"2023-11-28T18:31:58.463414+0000","last_unstale":"2023-11-28T18:32:00.877793+0000","last_undegraded":"2023-11-28T18:32:00.877793+0000","last_fullsized":"2023-11-28T18:32:00.877793+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:01:46.291198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459608+0000","last_change":"2023-11-28T18:31:59.486109+0000","last_active":"2023-11-28T18:32:00.459608+0000","last_peered":"2023-11-28T18:32:00.459608+0000","last_clean":"2023-11-28T18:32:00.459608+0000","last_became_active":"2023-11-28T18:31:59.484511+0000","last_became_peered":"2023-11-28T18:31:59.484511+0000","last_unstale":"2023-11-28T18:32:00.459608+0000","last_undegraded":"2023-11-28T18:32:00.459608+0000","last_fullsized":"2023-11-28T18:32:00.459608+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:08:59.784543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.875241+0000","last_change":"2023-11-28T18:31:58.465300+0000","last_active":"2023-11-28T18:32:00.875241+0000","last_peered":"2023-11-28T18:32:00.875241+0000","last_clean":"2023-11-28T18:32:00.875241+0000","last_became_active":"2023-11-28T18:31:58.464788+0000","last_became_peered":"2023-11-28T18:31:58.464788+0000","last_unstale":"2023-11-28T18:32:00.875241+0000","last_undegraded":"2023-11-28T18:32:00.875241+0000","last_fullsized":"2023-11-28T18:32:00.875241+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:38:56.117848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460854+0000","last_change":"2023-11-28T18:31:59.480620+0000","last_active":"2023-11-28T18:32:00.460854+0000","last_peered":"2023-11-28T18:32:00.460854+0000","last_clean":"2023-11-28T18:32:00.460854+0000","last_became_active":"2023-11-28T18:31:59.480298+0000","last_became_peered":"2023-11-28T18:31:59.480298+0000","last_unstale":"2023-11-28T18:32:00.460854+0000","last_undegraded":"2023-11-28T18:32:00.460854+0000","last_fullsized":"2023-11-28T18:32:00.460854+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:07:10.812548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"128'3","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877588+0000","last_change":"2023-11-28T18:31:58.466863+0000","last_active":"2023-11-28T18:32:00.877588+0000","last_peered":"2023-11-28T18:32:00.877588+0000","last_clean":"2023-11-28T18:32:00.877588+0000","last_became_active":"2023-11-28T18:31:58.466545+0000","last_became_peered":"2023-11-28T18:31:58.466545+0000","last_unstale":"2023-11-28T18:32:00.877588+0000","last_undegraded":"2023-11-28T18:32:00.877588+0000","last_fullsized":"2023-11-28T18:32:00.877588+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:34:02.704967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460670+0000","last_change":"2023-11-28T18:31:59.485179+0000","last_active":"2023-11-28T18:32:00.460670+0000","last_peered":"2023-11-28T18:32:00.460670+0000","last_clean":"2023-11-28T18:32:00.460670+0000","last_became_active":"2023-11-28T18:31:59.484851+0000","last_became_peered":"2023-11-28T18:31:59.484851+0000","last_unstale":"2023-11-28T18:32:00.460670+0000","last_undegraded":"2023-11-28T18:32:00.460670+0000","last_fullsized":"2023-11-28T18:32:00.460670+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:03.287701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"128'6","reported_seq":18,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877303+0000","last_change":"2023-11-28T18:31:58.470568+0000","last_active":"2023-11-28T18:32:00.877303+0000","last_peered":"2023-11-28T18:32:00.877303+0000","last_clean":"2023-11-28T18:32:00.877303+0000","last_became_active":"2023-11-28T18:31:58.470239+0000","last_became_peered":"2023-11-28T18:31:58.470239+0000","last_unstale":"2023-11-28T18:32:00.877303+0000","last_undegraded":"2023-11-28T18:32:00.877303+0000","last_fullsized":"2023-11-28T18:32:00.877303+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:02.029926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.875007+0000","last_change":"2023-11-28T18:31:59.479834+0000","last_active":"2023-11-28T18:32:00.875007+0000","last_peered":"2023-11-28T18:32:00.875007+0000","last_clean":"2023-11-28T18:32:00.875007+0000","last_became_active":"2023-11-28T18:31:59.479534+0000","last_became_peered":"2023-11-28T18:31:59.479534+0000","last_unstale":"2023-11-28T18:32:00.875007+0000","last_undegraded":"2023-11-28T18:32:00.875007+0000","last_fullsized":"2023-11-28T18:32:00.875007+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:10:44.436247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"128'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.875081+0000","last_change":"2023-11-28T18:31:58.465789+0000","last_active":"2023-11-28T18:32:00.875081+0000","last_peered":"2023-11-28T18:32:00.875081+0000","last_clean":"2023-11-28T18:32:00.875081+0000","last_became_active":"2023-11-28T18:31:58.465511+0000","last_became_peered":"2023-11-28T18:31:58.465511+0000","last_unstale":"2023-11-28T18:32:00.875081+0000","last_undegraded":"2023-11-28T18:32:00.875081+0000","last_fullsized":"2023-11-28T18:32:00.875081+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:25:17.256080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877871+0000","last_change":"2023-11-28T18:31:59.483918+0000","last_active":"2023-11-28T18:32:00.877871+0000","last_peered":"2023-11-28T18:32:00.877871+0000","last_clean":"2023-11-28T18:32:00.877871+0000","last_became_active":"2023-11-28T18:31:59.482610+0000","last_became_peered":"2023-11-28T18:31:59.482610+0000","last_unstale":"2023-11-28T18:32:00.877871+0000","last_undegraded":"2023-11-28T18:32:00.877871+0000","last_fullsized":"2023-11-28T18:32:00.877871+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:18:37.964759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460222+0000","last_change":"2023-11-28T18:31:58.459955+0000","last_active":"2023-11-28T18:32:00.460222+0000","last_peered":"2023-11-28T18:32:00.460222+0000","last_clean":"2023-11-28T18:32:00.460222+0000","last_became_active":"2023-11-28T18:31:58.457155+0000","last_became_peered":"2023-11-28T18:31:58.457155+0000","last_unstale":"2023-11-28T18:32:00.460222+0000","last_undegraded":"2023-11-28T18:32:00.460222+0000","last_fullsized":"2023-11-28T18:32:00.460222+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:42.191962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.458273+0000","last_change":"2023-11-28T18:31:59.470228+0000","last_active":"2023-11-28T18:32:00.458273+0000","last_peered":"2023-11-28T18:32:00.458273+0000","last_clean":"2023-11-28T18:32:00.458273+0000","last_became_active":"2023-11-28T18:31:59.469975+0000","last_became_peered":"2023-11-28T18:31:59.469975+0000","last_unstale":"2023-11-28T18:32:00.458273+0000","last_undegraded":"2023-11-28T18:32:00.458273+0000","last_fullsized":"2023-11-28T18:32:00.458273+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:41.384216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"128'8","reported_seq":20,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459529+0000","last_change":"2023-11-28T18:31:58.454212+0000","last_active":"2023-11-28T18:32:00.459529+0000","last_peered":"2023-11-28T18:32:00.459529+0000","last_clean":"2023-11-28T18:32:00.459529+0000","last_became_active":"2023-11-28T18:31:58.453946+0000","last_became_peered":"2023-11-28T18:31:58.453946+0000","last_unstale":"2023-11-28T18:32:00.459529+0000","last_undegraded":"2023-11-28T18:32:00.459529+0000","last_fullsized":"2023-11-28T18:32:00.459529+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:21:05.598600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460226+0000","last_change":"2023-11-28T18:31:59.475375+0000","last_active":"2023-11-28T18:32:00.460226+0000","last_peered":"2023-11-28T18:32:00.460226+0000","last_clean":"2023-11-28T18:32:00.460226+0000","last_became_active":"2023-11-28T18:31:59.475085+0000","last_became_peered":"2023-11-28T18:31:59.475085+0000","last_unstale":"2023-11-28T18:32:00.460226+0000","last_undegraded":"2023-11-28T18:32:00.460226+0000","last_fullsized":"2023-11-28T18:32:00.460226+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:37:13.925200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"128'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877867+0000","last_change":"2023-11-28T18:31:58.466101+0000","last_active":"2023-11-28T18:32:00.877867+0000","last_peered":"2023-11-28T18:32:00.877867+0000","last_clean":"2023-11-28T18:32:00.877867+0000","last_became_active":"2023-11-28T18:31:58.465528+0000","last_became_peered":"2023-11-28T18:31:58.465528+0000","last_unstale":"2023-11-28T18:32:00.877867+0000","last_undegraded":"2023-11-28T18:32:00.877867+0000","last_fullsized":"2023-11-28T18:32:00.877867+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:02.280932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876251+0000","last_change":"2023-11-28T18:31:59.481710+0000","last_active":"2023-11-28T18:32:00.876251+0000","last_peered":"2023-11-28T18:32:00.876251+0000","last_clean":"2023-11-28T18:32:00.876251+0000","last_became_active":"2023-11-28T18:31:59.481363+0000","last_became_peered":"2023-11-28T18:31:59.481363+0000","last_unstale":"2023-11-28T18:32:00.876251+0000","last_undegraded":"2023-11-28T18:32:00.876251+0000","last_fullsized":"2023-11-28T18:32:00.876251+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:42:34.470538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"128'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460525+0000","last_change":"2023-11-28T18:31:58.461973+0000","last_active":"2023-11-28T18:32:00.460525+0000","last_peered":"2023-11-28T18:32:00.460525+0000","last_clean":"2023-11-28T18:32:00.460525+0000","last_became_active":"2023-11-28T18:31:58.461697+0000","last_became_peered":"2023-11-28T18:31:58.461697+0000","last_unstale":"2023-11-28T18:32:00.460525+0000","last_undegraded":"2023-11-28T18:32:00.460525+0000","last_fullsized":"2023-11-28T18:32:00.460525+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:35:24.012986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876566+0000","last_change":"2023-11-28T18:31:59.483760+0000","last_active":"2023-11-28T18:32:00.876566+0000","last_peered":"2023-11-28T18:32:00.876566+0000","last_clean":"2023-11-28T18:32:00.876566+0000","last_became_active":"2023-11-28T18:31:59.483403+0000","last_became_peered":"2023-11-28T18:31:59.483403+0000","last_unstale":"2023-11-28T18:32:00.876566+0000","last_undegraded":"2023-11-28T18:32:00.876566+0000","last_fullsized":"2023-11-28T18:32:00.876566+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:07:19.157448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"128'1","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459900+0000","last_change":"2023-11-28T18:31:58.464128+0000","last_active":"2023-11-28T18:32:00.459900+0000","last_peered":"2023-11-28T18:32:00.459900+0000","last_clean":"2023-11-28T18:32:00.459900+0000","last_became_active":"2023-11-28T18:31:58.463478+0000","last_became_peered":"2023-11-28T18:31:58.463478+0000","last_unstale":"2023-11-28T18:32:00.459900+0000","last_undegraded":"2023-11-28T18:32:00.459900+0000","last_fullsized":"2023-11-28T18:32:00.459900+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:06:14.458890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.874831+0000","last_change":"2023-11-28T18:31:59.483751+0000","last_active":"2023-11-28T18:32:00.874831+0000","last_peered":"2023-11-28T18:32:00.874831+0000","last_clean":"2023-11-28T18:32:00.874831+0000","last_became_active":"2023-11-28T18:31:59.483475+0000","last_became_peered":"2023-11-28T18:31:59.483475+0000","last_unstale":"2023-11-28T18:32:00.874831+0000","last_undegraded":"2023-11-28T18:32:00.874831+0000","last_fullsized":"2023-11-28T18:32:00.874831+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:43:07.474936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.458005+0000","last_change":"2023-11-28T18:31:58.460778+0000","last_active":"2023-11-28T18:32:00.458005+0000","last_peered":"2023-11-28T18:32:00.458005+0000","last_clean":"2023-11-28T18:32:00.458005+0000","last_became_active":"2023-11-28T18:31:58.460302+0000","last_became_peered":"2023-11-28T18:31:58.460302+0000","last_unstale":"2023-11-28T18:32:00.458005+0000","last_undegraded":"2023-11-28T18:32:00.458005+0000","last_fullsized":"2023-11-28T18:32:00.458005+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:22:51.062346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.457924+0000","last_change":"2023-11-28T18:31:59.478768+0000","last_active":"2023-11-28T18:32:00.457924+0000","last_peered":"2023-11-28T18:32:00.457924+0000","last_clean":"2023-11-28T18:32:00.457924+0000","last_became_active":"2023-11-28T18:31:59.478507+0000","last_became_peered":"2023-11-28T18:31:59.478507+0000","last_unstale":"2023-11-28T18:32:00.457924+0000","last_undegraded":"2023-11-28T18:32:00.457924+0000","last_fullsized":"2023-11-28T18:32:00.457924+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:33:59.180002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"128'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876662+0000","last_change":"2023-11-28T18:31:58.466034+0000","last_active":"2023-11-28T18:32:00.876662+0000","last_peered":"2023-11-28T18:32:00.876662+0000","last_clean":"2023-11-28T18:32:00.876662+0000","last_became_active":"2023-11-28T18:31:58.465721+0000","last_became_peered":"2023-11-28T18:31:58.465721+0000","last_unstale":"2023-11-28T18:32:00.876662+0000","last_undegraded":"2023-11-28T18:32:00.876662+0000","last_fullsized":"2023-11-28T18:32:00.876662+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:40.046834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.875288+0000","last_change":"2023-11-28T18:31:59.485034+0000","last_active":"2023-11-28T18:32:00.875288+0000","last_peered":"2023-11-28T18:32:00.875288+0000","last_clean":"2023-11-28T18:32:00.875288+0000","last_became_active":"2023-11-28T18:31:59.484739+0000","last_became_peered":"2023-11-28T18:31:59.484739+0000","last_unstale":"2023-11-28T18:32:00.875288+0000","last_undegraded":"2023-11-28T18:32:00.875288+0000","last_fullsized":"2023-11-28T18:32:00.875288+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:36:10.244933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.874895+0000","last_change":"2023-11-28T18:31:58.466656+0000","last_active":"2023-11-28T18:32:00.874895+0000","last_peered":"2023-11-28T18:32:00.874895+0000","last_clean":"2023-11-28T18:32:00.874895+0000","last_became_active":"2023-11-28T18:31:58.466407+0000","last_became_peered":"2023-11-28T18:31:58.466407+0000","last_unstale":"2023-11-28T18:32:00.874895+0000","last_undegraded":"2023-11-28T18:32:00.874895+0000","last_fullsized":"2023-11-28T18:32:00.874895+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:31:34.336753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.458273+0000","last_change":"2023-11-28T18:31:59.475681+0000","last_active":"2023-11-28T18:32:00.458273+0000","last_peered":"2023-11-28T18:32:00.458273+0000","last_clean":"2023-11-28T18:32:00.458273+0000","last_became_active":"2023-11-28T18:31:59.475211+0000","last_became_peered":"2023-11-28T18:31:59.475211+0000","last_unstale":"2023-11-28T18:32:00.458273+0000","last_undegraded":"2023-11-28T18:32:00.458273+0000","last_fullsized":"2023-11-28T18:32:00.458273+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:33:00.087111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876289+0000","last_change":"2023-11-28T18:31:58.465083+0000","last_active":"2023-11-28T18:32:00.876289+0000","last_peered":"2023-11-28T18:32:00.876289+0000","last_clean":"2023-11-28T18:32:00.876289+0000","last_became_active":"2023-11-28T18:31:58.464760+0000","last_became_peered":"2023-11-28T18:31:58.464760+0000","last_unstale":"2023-11-28T18:32:00.876289+0000","last_undegraded":"2023-11-28T18:32:00.876289+0000","last_fullsized":"2023-11-28T18:32:00.876289+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:33:58.173232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877845+0000","last_change":"2023-11-28T18:31:59.490489+0000","last_active":"2023-11-28T18:32:00.877845+0000","last_peered":"2023-11-28T18:32:00.877845+0000","last_clean":"2023-11-28T18:32:00.877845+0000","last_became_active":"2023-11-28T18:31:59.490218+0000","last_became_peered":"2023-11-28T18:31:59.490218+0000","last_unstale":"2023-11-28T18:32:00.877845+0000","last_undegraded":"2023-11-28T18:32:00.877845+0000","last_fullsized":"2023-11-28T18:32:00.877845+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:50:40.320398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459609+0000","last_change":"2023-11-28T18:31:58.456766+0000","last_active":"2023-11-28T18:32:00.459609+0000","last_peered":"2023-11-28T18:32:00.459609+0000","last_clean":"2023-11-28T18:32:00.459609+0000","last_became_active":"2023-11-28T18:31:58.456515+0000","last_became_peered":"2023-11-28T18:31:58.456515+0000","last_unstale":"2023-11-28T18:32:00.459609+0000","last_undegraded":"2023-11-28T18:32:00.459609+0000","last_fullsized":"2023-11-28T18:32:00.459609+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:18:52.754113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.457843+0000","last_change":"2023-11-28T18:31:59.475363+0000","last_active":"2023-11-28T18:32:00.457843+0000","last_peered":"2023-11-28T18:32:00.457843+0000","last_clean":"2023-11-28T18:32:00.457843+0000","last_became_active":"2023-11-28T18:31:59.475049+0000","last_became_peered":"2023-11-28T18:31:59.475049+0000","last_unstale":"2023-11-28T18:32:00.457843+0000","last_undegraded":"2023-11-28T18:32:00.457843+0000","last_fullsized":"2023-11-28T18:32:00.457843+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:03.036695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"128'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.875436+0000","last_change":"2023-11-28T18:31:58.464128+0000","last_active":"2023-11-28T18:32:00.875436+0000","last_peered":"2023-11-28T18:32:00.875436+0000","last_clean":"2023-11-28T18:32:00.875436+0000","last_became_active":"2023-11-28T18:31:58.463835+0000","last_became_peered":"2023-11-28T18:31:58.463835+0000","last_unstale":"2023-11-28T18:32:00.875436+0000","last_undegraded":"2023-11-28T18:32:00.875436+0000","last_fullsized":"2023-11-28T18:32:00.875436+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:45:51.904590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460158+0000","last_change":"2023-11-28T18:31:59.479617+0000","last_active":"2023-11-28T18:32:00.460158+0000","last_peered":"2023-11-28T18:32:00.460158+0000","last_clean":"2023-11-28T18:32:00.460158+0000","last_became_active":"2023-11-28T18:31:59.479323+0000","last_became_peered":"2023-11-28T18:31:59.479323+0000","last_unstale":"2023-11-28T18:32:00.460158+0000","last_undegraded":"2023-11-28T18:32:00.460158+0000","last_fullsized":"2023-11-28T18:32:00.460158+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:51:51.574894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877023+0000","last_change":"2023-11-28T18:31:58.466768+0000","last_active":"2023-11-28T18:32:00.877023+0000","last_peered":"2023-11-28T18:32:00.877023+0000","last_clean":"2023-11-28T18:32:00.877023+0000","last_became_active":"2023-11-28T18:31:58.466495+0000","last_became_peered":"2023-11-28T18:31:58.466495+0000","last_unstale":"2023-11-28T18:32:00.877023+0000","last_undegraded":"2023-11-28T18:32:00.877023+0000","last_fullsized":"2023-11-28T18:32:00.877023+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:08.297316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876111+0000","last_change":"2023-11-28T18:31:59.497301+0000","last_active":"2023-11-28T18:32:00.876111+0000","last_peered":"2023-11-28T18:32:00.876111+0000","last_clean":"2023-11-28T18:32:00.876111+0000","last_became_active":"2023-11-28T18:31:59.497008+0000","last_became_peered":"2023-11-28T18:31:59.497008+0000","last_unstale":"2023-11-28T18:32:00.876111+0000","last_undegraded":"2023-11-28T18:32:00.876111+0000","last_fullsized":"2023-11-28T18:32:00.876111+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:30.295476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876942+0000","last_change":"2023-11-28T18:31:58.464865+0000","last_active":"2023-11-28T18:32:00.876942+0000","last_peered":"2023-11-28T18:32:00.876942+0000","last_clean":"2023-11-28T18:32:00.876942+0000","last_became_active":"2023-11-28T18:31:58.464575+0000","last_became_peered":"2023-11-28T18:31:58.464575+0000","last_unstale":"2023-11-28T18:32:00.876942+0000","last_undegraded":"2023-11-28T18:32:00.876942+0000","last_fullsized":"2023-11-28T18:32:00.876942+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:55:33.705881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.874925+0000","last_change":"2023-11-28T18:31:59.483192+0000","last_active":"2023-11-28T18:32:00.874925+0000","last_peered":"2023-11-28T18:32:00.874925+0000","last_clean":"2023-11-28T18:32:00.874925+0000","last_became_active":"2023-11-28T18:31:59.480298+0000","last_became_peered":"2023-11-28T18:31:59.480298+0000","last_unstale":"2023-11-28T18:32:00.874925+0000","last_undegraded":"2023-11-28T18:32:00.874925+0000","last_fullsized":"2023-11-28T18:32:00.874925+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:55:10.107387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459728+0000","last_change":"2023-11-28T18:31:58.460507+0000","last_active":"2023-11-28T18:32:00.459728+0000","last_peered":"2023-11-28T18:32:00.459728+0000","last_clean":"2023-11-28T18:32:00.459728+0000","last_became_active":"2023-11-28T18:31:58.460248+0000","last_became_peered":"2023-11-28T18:31:58.460248+0000","last_unstale":"2023-11-28T18:32:00.459728+0000","last_undegraded":"2023-11-28T18:32:00.459728+0000","last_fullsized":"2023-11-28T18:32:00.459728+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:15:06.833507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459746+0000","last_change":"2023-11-28T18:31:59.478535+0000","last_active":"2023-11-28T18:32:00.459746+0000","last_peered":"2023-11-28T18:32:00.459746+0000","last_clean":"2023-11-28T18:32:00.459746+0000","last_became_active":"2023-11-28T18:31:59.477929+0000","last_became_peered":"2023-11-28T18:31:59.477929+0000","last_unstale":"2023-11-28T18:32:00.459746+0000","last_undegraded":"2023-11-28T18:32:00.459746+0000","last_fullsized":"2023-11-28T18:32:00.459746+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:06:10.816804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877470+0000","last_change":"2023-11-28T18:31:58.463918+0000","last_active":"2023-11-28T18:32:00.877470+0000","last_peered":"2023-11-28T18:32:00.877470+0000","last_clean":"2023-11-28T18:32:00.877470+0000","last_became_active":"2023-11-28T18:31:58.463458+0000","last_became_peered":"2023-11-28T18:31:58.463458+0000","last_unstale":"2023-11-28T18:32:00.877470+0000","last_undegraded":"2023-11-28T18:32:00.877470+0000","last_fullsized":"2023-11-28T18:32:00.877470+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:37:12.918431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459901+0000","last_change":"2023-11-28T18:31:59.479415+0000","last_active":"2023-11-28T18:32:00.459901+0000","last_peered":"2023-11-28T18:32:00.459901+0000","last_clean":"2023-11-28T18:32:00.459901+0000","last_became_active":"2023-11-28T18:31:59.479144+0000","last_became_peered":"2023-11-28T18:31:59.479144+0000","last_unstale":"2023-11-28T18:32:00.459901+0000","last_undegraded":"2023-11-28T18:32:00.459901+0000","last_fullsized":"2023-11-28T18:32:00.459901+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:22:52.069116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876718+0000","last_change":"2023-11-28T18:31:58.467928+0000","last_active":"2023-11-28T18:32:00.876718+0000","last_peered":"2023-11-28T18:32:00.876718+0000","last_clean":"2023-11-28T18:32:00.876718+0000","last_became_active":"2023-11-28T18:31:58.467636+0000","last_became_peered":"2023-11-28T18:31:58.467636+0000","last_unstale":"2023-11-28T18:32:00.876718+0000","last_undegraded":"2023-11-28T18:32:00.876718+0000","last_fullsized":"2023-11-28T18:32:00.876718+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:29.438862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.460049+0000","last_change":"2023-11-28T18:31:59.479948+0000","last_active":"2023-11-28T18:32:00.460049+0000","last_peered":"2023-11-28T18:32:00.460049+0000","last_clean":"2023-11-28T18:32:00.460049+0000","last_became_active":"2023-11-28T18:31:59.479688+0000","last_became_peered":"2023-11-28T18:31:59.479688+0000","last_unstale":"2023-11-28T18:32:00.460049+0000","last_undegraded":"2023-11-28T18:32:00.460049+0000","last_fullsized":"2023-11-28T18:32:00.460049+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:02.710984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.459564+0000","last_change":"2023-11-28T18:31:58.460922+0000","last_active":"2023-11-28T18:32:00.459564+0000","last_peered":"2023-11-28T18:32:00.459564+0000","last_clean":"2023-11-28T18:32:00.459564+0000","last_became_active":"2023-11-28T18:31:58.460668+0000","last_became_peered":"2023-11-28T18:31:58.460668+0000","last_unstale":"2023-11-28T18:32:00.459564+0000","last_undegraded":"2023-11-28T18:32:00.459564+0000","last_fullsized":"2023-11-28T18:32:00.459564+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:07:09.805778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.877615+0000","last_change":"2023-11-28T18:31:59.497732+0000","last_active":"2023-11-28T18:32:00.877615+0000","last_peered":"2023-11-28T18:32:00.877615+0000","last_clean":"2023-11-28T18:32:00.877615+0000","last_became_active":"2023-11-28T18:31:59.497224+0000","last_became_peered":"2023-11-28T18:31:59.497224+0000","last_unstale":"2023-11-28T18:32:00.877615+0000","last_undegraded":"2023-11-28T18:32:00.877615+0000","last_fullsized":"2023-11-28T18:32:00.877615+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:08.815498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.457917+0000","last_change":"2023-11-28T18:31:58.449122+0000","last_active":"2023-11-28T18:32:00.457917+0000","last_peered":"2023-11-28T18:32:00.457917+0000","last_clean":"2023-11-28T18:32:00.457917+0000","last_became_active":"2023-11-28T18:31:58.448939+0000","last_became_peered":"2023-11-28T18:31:58.448939+0000","last_unstale":"2023-11-28T18:32:00.457917+0000","last_undegraded":"2023-11-28T18:32:00.457917+0000","last_fullsized":"2023-11-28T18:32:00.457917+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:15:06.833507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.461038+0000","last_change":"2023-11-28T18:31:59.481570+0000","last_active":"2023-11-28T18:32:00.461038+0000","last_peered":"2023-11-28T18:32:00.461038+0000","last_clean":"2023-11-28T18:32:00.461038+0000","last_became_active":"2023-11-28T18:31:59.481274+0000","last_became_peered":"2023-11-28T18:31:59.481274+0000","last_unstale":"2023-11-28T18:32:00.461038+0000","last_undegraded":"2023-11-28T18:32:00.461038+0000","last_fullsized":"2023-11-28T18:32:00.461038+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:58.436016+0000","last_clean_scrub_stamp":"2023-11-28T18:31:58.436016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:25:17.025975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-11-28T18:32:00.876388+0000","last_change":"2023-11-28T18:31:58.468957+0000","last_active":"2023-11-28T18:32:00.876388+0000","last_peered":"2023-11-28T18:32:00.876388+0000","last_clean":"2023-11-28T18:32:00.876388+0000","last_became_active":"2023-11-28T18:31:58.468642+0000","last_became_peered":"2023-11-28T18:31:58.468642+0000","last_unstale":"2023-11-28T18:32:00.876388+0000","last_undegraded":"2023-11-28T18:32:00.876388+0000","last_fullsized":"2023-11-28T18:32:00.876388+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:31:57.429246+0000","last_clean_scrub_stamp":"2023-11-28T18:31:57.429246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:15:01.704215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":258,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-11-28T18:31:59.468640+0000","last_change":"2023-11-28T18:23:23.497372+0000","last_active":"2023-11-28T18:31:59.468640+0000","last_peered":"2023-11-28T18:31:59.468640+0000","last_clean":"2023-11-28T18:31:59.468640+0000","last_became_active":"2023-11-28T18:23:23.496366+0000","last_became_peered":"2023-11-28T18:23:23.496366+0000","last_unstale":"2023-11-28T18:31:59.468640+0000","last_undegraded":"2023-11-28T18:31:59.468640+0000","last_fullsized":"2023-11-28T18:31:59.468640+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T18:23:21.757145+0000","last_clean_scrub_stamp":"2023-11-28T18:23:21.757145+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:56.716238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574956,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51388,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320452,"statfs":{"total":96636764160,"available":96584142848,"internally_reserved":0,"allocated":2211840,"data_stored":1338063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65200000000000002,"15min":0.69699999999999995},"min":{"1min":0.48899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.90400000000000003,"5min":0.96999999999999997,"15min":2.427},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.64600000000000002,"15min":0.69699999999999995},"min":{"1min":0.372,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":2.2509999999999999},"last":0.54000000000000004}]},{"osd":1,"last update":"Tue Nov 28 18:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66800000000000004,"15min":0.70199999999999996},"min":{"1min":0.41299999999999998,"5min":0.38900000000000001,"15min":0.34699999999999998},"max":{"1min":0.85799999999999998,"5min":0.96699999999999997,"15min":2.3849999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66200000000000003,"15min":0.72599999999999998},"min":{"1min":0.44400000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.95399999999999996,"5min":1.079,"15min":2.3130000000000002},"last":0.59099999999999997}]},{"osd":2,"last update":"Tue Nov 28 18:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.68600000000000005,"15min":0.747},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94599999999999995,"5min":0.998,"15min":2.4590000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.69599999999999995,"15min":0.72699999999999998},"min":{"1min":0.56799999999999995,"5min":0.41299999999999998,"15min":0.39300000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":2.411},"last":0.747}]},{"osd":3,"last update":"Tue Nov 28 18:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.67300000000000004,"15min":0.73399999999999999},"min":{"1min":0.45100000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":2.2730000000000001},"last":0.628},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.72799999999999998,"15min":0.77900000000000003},"min":{"1min":0.59299999999999997,"5min":0.432,"15min":0.432},"max":{"1min":1.008,"5min":1.0389999999999999,"15min":2.3650000000000002},"last":0.67600000000000005}]},{"osd":4,"last update":"Tue Nov 28 18:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.61699999999999999,"15min":0.60599999999999998},"min":{"1min":0.434,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60699999999999998,"15min":0.63},"min":{"1min":0.40699999999999997,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.86399999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.69199999999999995}]},{"osd":5,"last update":"Tue Nov 28 18:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64400000000000002,"15min":0.66500000000000004},"min":{"1min":0.45700000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0589999999999999,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.67400000000000004,"15min":0.69199999999999995},"min":{"1min":0.52800000000000002,"5min":0.39900000000000002,"15min":0.39300000000000002},"max":{"1min":0.996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.65500000000000003}]},{"osd":6,"last update":"Tue Nov 28 18:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.72499999999999998,"15min":0.73699999999999999},"min":{"1min":0.495,"5min":0.375,"15min":0.375},"max":{"1min":1.036,"5min":1.131,"15min":1.131},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.66600000000000004,"15min":0.71099999999999997},"min":{"1min":0.46000000000000002,"5min":0.36699999999999999,"15min":0.35899999999999999},"max":{"1min":1.02,"5min":1.081,"15min":1.081},"last":0.57099999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73016,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298824,"statfs":{"total":96636764160,"available":96561995776,"internally_reserved":0,"allocated":1609728,"data_stored":746665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.68600000000000005,"15min":0.65300000000000002},"min":{"1min":0.51300000000000001,"5min":0.35499999999999998,"15min":0.27300000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.56999999999999995,"5min":0.40699999999999997,"15min":0.35399999999999998},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":0.75700000000000001}]},{"osd":1,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.67300000000000004,"15min":0.76500000000000001},"min":{"1min":0.45100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":4.2389999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.69899999999999995,"15min":0.76300000000000001},"min":{"1min":0.53000000000000003,"5min":0.38200000000000001,"15min":0.307},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":4.218},"last":0.71599999999999997}]},{"osd":2,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.67900000000000005,"15min":0.70399999999999996},"min":{"1min":0.44400000000000001,"5min":0.373,"15min":0.36299999999999999},"max":{"1min":2.7599999999999998,"5min":2.7599999999999998,"15min":2.7599999999999998},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.71299999999999997,"15min":0.70799999999999996},"min":{"1min":0.54700000000000004,"5min":0.42999999999999999,"15min":0.40100000000000002},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.73699999999999999}]},{"osd":3,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70499999999999996,"15min":0.71899999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.39800000000000002},"max":{"1min":0.99299999999999999,"5min":1.016,"15min":1.016},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.52600000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.8090000000000002,"5min":2.8090000000000002,"15min":2.8090000000000002},"last":0.77100000000000002}]},{"osd":4,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60399999999999998,"15min":0.65000000000000002},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64000000000000001,"15min":0.67100000000000004},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.64100000000000001,"15min":0.66900000000000004},"min":{"1min":0.39000000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.67900000000000005,"15min":0.73599999999999999},"min":{"1min":0.46800000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.66000000000000003}]},{"osd":7,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69199999999999995,"15min":0.747},"min":{"1min":0.44500000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.70799999999999996,"15min":0.73799999999999999},"min":{"1min":0.495,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.8490000000000002,"5min":2.8490000000000002,"15min":2.8490000000000002},"last":0.80800000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36920,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334920,"statfs":{"total":96636764160,"available":96598958080,"internally_reserved":0,"allocated":1617920,"data_stored":752635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67100000000000004,"15min":0.70199999999999996},"min":{"1min":0.432,"5min":0.31,"15min":0.31},"max":{"1min":0.86299999999999999,"5min":2.1299999999999999,"15min":2.8929999999999998},"last":0.745},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70799999999999996,"15min":0.73899999999999999},"min":{"1min":0.52300000000000002,"5min":0.39200000000000002,"15min":0.33900000000000002},"max":{"1min":1.0169999999999999,"5min":1.0249999999999999,"15min":2.9140000000000001},"last":0.83899999999999997}]},{"osd":1,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.68500000000000005,"15min":0.72499999999999998},"min":{"1min":0.52800000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":3.4809999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.72799999999999998,"15min":0.77800000000000002},"min":{"1min":0.64500000000000002,"5min":0.40200000000000002,"15min":0.39100000000000001},"max":{"1min":1.0509999999999999,"5min":2.2109999999999999,"15min":3.4580000000000002},"last":0.78200000000000003}]},{"osd":2,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.67100000000000004,"15min":0.73199999999999998},"min":{"1min":0.54300000000000004,"5min":0.378,"15min":0.378},"max":{"1min":0.96399999999999997,"5min":1.0309999999999999,"15min":3.9510000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66900000000000004,"15min":0.74199999999999999},"min":{"1min":0.46400000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91300000000000003,"5min":1.014,"15min":4.0149999999999997},"last":0.69799999999999995}]},{"osd":3,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69799999999999995,"15min":0.78900000000000003},"min":{"1min":0.58999999999999997,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85999999999999999,"5min":2.254,"15min":4.0869999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68799999999999994,"15min":0.78100000000000003},"min":{"1min":0.46800000000000003,"5min":0.46300000000000002,"15min":0.41299999999999998},"max":{"1min":0.92900000000000005,"5min":0.98599999999999999,"15min":3.9900000000000002},"last":0.72399999999999998}]},{"osd":4,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.44400000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82199999999999995,"5min":1.696,"15min":1.696},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.65900000000000003,"15min":0.69399999999999995},"min":{"1min":0.39200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86699999999999999,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.52300000000000002}]},{"osd":6,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64300000000000002,"15min":0.66900000000000004},"min":{"1min":0.41399999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79500000000000004,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68300000000000005,"15min":0.70699999999999996},"min":{"1min":0.51200000000000001,"5min":0.438,"15min":0.42399999999999999},"max":{"1min":0.877,"5min":1.262,"15min":1.756},"last":0.82199999999999995}]},{"osd":7,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71899999999999997,"15min":0.74399999999999999},"min":{"1min":0.623,"5min":0.42399999999999999,"15min":0.41699999999999998},"max":{"1min":0.89800000000000002,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.72799999999999998,"15min":0.747},"min":{"1min":0.54100000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94499999999999995,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.65100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574956,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73004,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298836,"statfs":{"total":96636764160,"available":96562008064,"internally_reserved":0,"allocated":1601536,"data_stored":745993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.64900000000000002,"15min":0.67500000000000004},"min":{"1min":0.35099999999999998,"5min":0.32600000000000001,"15min":0.315},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":2.363},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.65300000000000002,"15min":0.64800000000000002},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.997}]},{"osd":1,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.65300000000000002,"15min":0.63400000000000001},"min":{"1min":0.501,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93500000000000005,"5min":1.0029999999999999,"15min":1.0629999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.69299999999999995,"15min":0.66500000000000004},"min":{"1min":0.46899999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96499999999999997,"5min":1.02,"15min":1.02},"last":0.84599999999999997}]},{"osd":2,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72099999999999997,"15min":0.70399999999999996},"min":{"1min":0.56799999999999995,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.95899999999999996,"5min":1.056,"15min":1.056},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73899999999999999,"15min":0.73099999999999998},"min":{"1min":0.46800000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.91500000000000004,"5min":1.091,"15min":1.871},"last":0.98299999999999998}]},{"osd":3,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.67900000000000005,"15min":0.65800000000000003},"min":{"1min":0.48699999999999999,"5min":0.45700000000000002,"15min":0.40899999999999997},"max":{"1min":0.93200000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70199999999999996,"15min":0.68799999999999994},"min":{"1min":0.53500000000000003,"5min":0.442,"15min":0.442},"max":{"1min":0.92200000000000004,"5min":1.014,"15min":1.014},"last":1.0469999999999999}]},{"osd":5,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60299999999999998,"15min":0.59199999999999997},"min":{"1min":0.38200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.89800000000000002,"5min":1.03,"15min":1.1799999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.60299999999999998,"15min":0.627},"min":{"1min":0.33500000000000002,"5min":0.33400000000000002,"15min":0.32300000000000001},"max":{"1min":0.878,"5min":1.05,"15min":1.79},"last":0.78400000000000003}]},{"osd":6,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61599999999999999,"15min":0.63600000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85199999999999998,"5min":1.0680000000000001,"15min":1.123},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.69099999999999995,"15min":0.70199999999999996},"min":{"1min":0.433,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94699999999999995,"5min":1.1970000000000001,"15min":1.925},"last":0.75900000000000001}]},{"osd":7,"last update":"Tue Nov 28 18:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68300000000000005,"15min":0.70599999999999996},"min":{"1min":0.45600000000000002,"5min":0.41999999999999998,"15min":0.36399999999999999},"max":{"1min":0.94099999999999995,"5min":1.1040000000000001,"15min":1.843},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.75900000000000001,"15min":0.77000000000000002},"min":{"1min":0.55000000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0149999999999999,"5min":1.1299999999999999,"15min":1.593},"last":1.101}]}]},{"osd":3,"up_from":13,"seq":55834574955,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50812,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321028,"statfs":{"total":96636764160,"available":96584732672,"internally_reserved":0,"allocated":1617920,"data_stored":747675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.441,"15min":0.42199999999999999},"min":{"1min":0.26000000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.64200000000000002,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.499,"15min":0.47599999999999998},"min":{"1min":0.32500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.65300000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.60699999999999998}]},{"osd":1,"last update":"Tue Nov 28 18:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.44600000000000001,"15min":0.45900000000000002},"min":{"1min":0.308,"5min":0.28799999999999998,"15min":0.27300000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.96099999999999997},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55600000000000005,"15min":0.51900000000000002},"min":{"1min":0.36799999999999999,"5min":0.249,"15min":0.247},"max":{"1min":0.69499999999999995,"5min":4.5599999999999996,"15min":4.5599999999999996},"last":0.52300000000000002}]},{"osd":2,"last update":"Tue Nov 28 18:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55300000000000005,"15min":0.52900000000000003},"min":{"1min":0.33600000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71499999999999997,"5min":2.48,"15min":2.48},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56100000000000005,"15min":0.55900000000000005},"min":{"1min":0.40500000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71099999999999997,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.55800000000000005}]},{"osd":4,"last update":"Tue Nov 28 18:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.88500000000000001,"15min":0.80700000000000005},"min":{"1min":0.48199999999999998,"5min":0.441,"15min":0.39800000000000002},"max":{"1min":0.95699999999999996,"5min":12.695,"15min":12.695},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.79400000000000004,"15min":0.77700000000000002},"min":{"1min":0.503,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.03,"5min":6.3250000000000002,"15min":6.3250000000000002},"last":0.77100000000000002}]},{"osd":5,"last update":"Tue Nov 28 18:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.76800000000000002,"15min":0.81100000000000005},"min":{"1min":0.52400000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.5860000000000001,"5min":1.9430000000000001,"15min":2.5019999999999998},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79400000000000004,"15min":0.78100000000000003},"min":{"1min":0.58999999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.109,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.034}]},{"osd":6,"last update":"Tue Nov 28 18:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.81200000000000006,"15min":0.80400000000000005},"min":{"1min":0.46600000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1299999999999999,"5min":6.1790000000000003,"15min":6.1790000000000003},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78200000000000003,"15min":0.79500000000000004},"min":{"1min":0.54200000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0800000000000001,"5min":1.984,"15min":1.984},"last":0.90100000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.78800000000000003,"15min":0.80400000000000005},"min":{"1min":0.54500000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.997,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.83599999999999997,"15min":0.81699999999999995},"min":{"1min":0.55100000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.5449999999999999,"5min":6.2249999999999996,"15min":6.2249999999999996},"last":0.87}]}]},{"osd":2,"up_from":13,"seq":55834574956,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36928,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334912,"statfs":{"total":96636764160,"available":96598949888,"internally_reserved":0,"allocated":1622016,"data_stored":753229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.434,"15min":0.47799999999999998},"min":{"1min":0.26000000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.66500000000000004,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.46400000000000002,"15min":0.48899999999999999},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26500000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":2.0150000000000001},"last":0.378}]},{"osd":1,"last update":"Tue Nov 28 18:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47799999999999998,"15min":0.50900000000000001},"min":{"1min":0.30399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66600000000000004,"5min":0.70999999999999996,"15min":1.48},"last":0.434},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56699999999999995,"15min":0.53100000000000003},"min":{"1min":0.35899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78800000000000003,"5min":8.3539999999999992,"15min":8.3539999999999992},"last":0.51500000000000001}]},{"osd":3,"last update":"Tue Nov 28 18:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.496,"15min":0.498},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69499999999999995,"5min":3.1549999999999998,"15min":3.1549999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.56100000000000005},"min":{"1min":0.38900000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71999999999999997,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":0.53200000000000003}]},{"osd":4,"last update":"Tue Nov 28 18:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.751,"15min":0.76600000000000001},"min":{"1min":0.50700000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.337,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.76100000000000001,"15min":0.77000000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.137,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue Nov 28 18:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76000000000000001,"15min":0.76200000000000001},"min":{"1min":0.438,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.198,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.73499999999999999,"15min":0.90800000000000003},"min":{"1min":0.5,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.298,"5min":1.298,"15min":7.4820000000000002},"last":0.75}]},{"osd":6,"last update":"Tue Nov 28 18:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.84799999999999998,"15min":0.81299999999999994},"min":{"1min":0.51600000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.0429999999999999,"5min":12.019,"15min":12.019},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.81999999999999995,"15min":0.84599999999999997},"min":{"1min":0.55600000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0860000000000001,"5min":8.7400000000000002,"15min":8.7400000000000002},"last":0.73199999999999998}]},{"osd":7,"last update":"Tue Nov 28 18:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.82799999999999996,"15min":0.83399999999999996},"min":{"1min":0.54500000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2430000000000001,"5min":8.327,"15min":8.327},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.82699999999999996,"15min":0.88},"min":{"1min":0.54600000000000004,"5min":0.441,"15min":0.42199999999999999},"max":{"1min":1.1399999999999999,"5min":8.609,"15min":8.609},"last":0.82199999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41084,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330756,"statfs":{"total":96636764160,"available":96594694144,"internally_reserved":0,"allocated":1617920,"data_stored":752683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41599999999999998,"15min":0.42199999999999999},"min":{"1min":0.25600000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.70999999999999996},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.47199999999999998,"15min":0.45600000000000002},"min":{"1min":0.28100000000000003,"5min":0.27300000000000002,"15min":0.26600000000000001},"max":{"1min":0.60599999999999998,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.56799999999999995}]},{"osd":2,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44900000000000001,"15min":0.44700000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.58299999999999996,"5min":0.69499999999999995,"15min":0.75700000000000001},"last":0.505},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.49199999999999999,"15min":0.48299999999999998},"min":{"1min":0.31900000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.59999999999999998,"5min":0.73399999999999999,"15min":0.79200000000000004},"last":0.53500000000000003}]},{"osd":3,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.36699999999999999},"max":{"1min":0.77200000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.621},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.50700000000000001,"15min":0.496},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.32200000000000001},"max":{"1min":0.67100000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.60599999999999998}]},{"osd":4,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71599999999999997,"15min":0.73299999999999998},"min":{"1min":0.53000000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.876,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.755,"15min":0.76200000000000001},"min":{"1min":0.54400000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91100000000000003,"5min":1.0780000000000001,"15min":1.155},"last":1.0349999999999999}]},{"osd":5,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.69599999999999995,"15min":0.72799999999999998},"min":{"1min":0.47499999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85199999999999998,"5min":0.997,"15min":1.3799999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70399999999999996,"15min":0.72899999999999998},"min":{"1min":0.48799999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.82199999999999995,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.1779999999999999}]},{"osd":6,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.73699999999999999,"15min":0.76000000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89900000000000002,"5min":1.1279999999999999,"15min":1.2969999999999999},"last":1},{"interface":"front","average":{"1min":0.752,"5min":0.76300000000000001,"15min":0.78900000000000003},"min":{"1min":0.53400000000000003,"5min":0.495,"15min":0.495},"max":{"1min":0.89500000000000002,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.1279999999999999}]},{"osd":7,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73199999999999998,"15min":0.74299999999999999},"min":{"1min":0.54600000000000004,"5min":0.503,"15min":0.47899999999999998},"max":{"1min":0.89000000000000001,"5min":1.0669999999999999,"15min":1.2410000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73599999999999999,"15min":0.76000000000000001},"min":{"1min":0.55100000000000005,"5min":0.52300000000000002,"15min":0.48499999999999999},"max":{"1min":0.91000000000000003,"5min":1.0109999999999999,"15min":1.4239999999999999},"last":0.93400000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47220,"kb_used_data":2152,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324620,"statfs":{"total":96636764160,"available":96588410880,"internally_reserved":0,"allocated":2203648,"data_stored":1336417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.41399999999999998,"15min":0.434},"min":{"1min":0.27700000000000002,"5min":0.248,"15min":0.248},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.437},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.30199999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.71399999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.36799999999999999}]},{"osd":2,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.47799999999999998,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.318,"15min":0.30499999999999999},"max":{"1min":0.67400000000000004,"5min":0.72899999999999998,"15min":0.92900000000000005},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.48999999999999999,"15min":0.50600000000000001},"min":{"1min":0.39300000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69999999999999996,"5min":0.754,"15min":0.91200000000000003},"last":0.45700000000000002}]},{"osd":3,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47399999999999998,"15min":0.48499999999999999},"min":{"1min":0.33000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70099999999999996,"5min":0.78600000000000003,"15min":0.86499999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53300000000000003,"15min":0.54600000000000004},"min":{"1min":0.36499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72999999999999998,"5min":0.78500000000000003,"15min":1.087},"last":0.58099999999999996}]},{"osd":4,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.131},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.71099999999999997,"15min":0.71799999999999997},"min":{"1min":0.55500000000000005,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94299999999999995,"5min":0.995,"15min":1.244},"last":0.72299999999999998}]},{"osd":5,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70899999999999996,"15min":0.72399999999999998},"min":{"1min":0.46999999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.92900000000000005,"5min":0.93600000000000005,"15min":1.121},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.45000000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":1.075},"last":0.60299999999999998}]},{"osd":6,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72999999999999998,"15min":0.73399999999999999},"min":{"1min":0.54700000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.407},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71799999999999997,"15min":0.73399999999999999},"min":{"1min":0.57199999999999995,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.877,"5min":0.96099999999999997,"15min":1.089},"last":0.64900000000000002}]},{"osd":7,"last update":"Tue Nov 28 18:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.754,"15min":0.75600000000000001},"min":{"1min":0.60999999999999999,"5min":0.504,"15min":0.504},"max":{"1min":0.96699999999999997,"5min":1.054,"15min":1.212},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72199999999999998,"15min":0.74199999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91600000000000004,"5min":0.97699999999999998,"15min":1.0960000000000001},"last":0.74199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T18:32:03.967 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T18:32:03.967 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-28T18:32:04.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.144+0000 7fbc066bb640 1 Processor -- start 2023-11-28T18:32:04.145 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.144+0000 7fbc066bb640 1 -- start start 2023-11-28T18:32:04.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.148+0000 7fbc066bb640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc00159610 0x7fbc000ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:04.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.148+0000 7fbc066bb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbc0006cef0 0x7fbc000ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:04.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.148+0000 7fbc066bb640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc000cde30 con 0x7fbc000cc760 2023-11-28T18:32:04.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.148+0000 7fbc066bb640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc001599f0 con 0x7fbc0006cef0 2023-11-28T18:32:04.147 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.148+0000 7fbc066bb640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0006de20 con 0x7fbc00159610 2023-11-28T18:32:04.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.148+0000 7fbc04c58640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbc0006cef0 0x7fbc000ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:04.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.148+0000 7fbc04c58640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbc0006cef0 0x7fbc000ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58540/0 (socket says 172.21.15.105:58540) 2023-11-28T18:32:04.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.148+0000 7fbbff7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fbc000cc760 0x7fbc0006c7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41786/0 (socket says 172.21.15.105:41786) 2023-11-28T18:32:04.148 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.148+0000 7fbc04c58640 1 -- 172.21.15.105:0/1633350942 learned_addr learned my addr 172.21.15.105:0/1633350942 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:04.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.148+0000 7fbc04c58640 1 -- 172.21.15.105:0/1633350942 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc00159610 msgr2=0x7fbc000ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:04.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.148+0000 7fbc04c58640 1 --2- 172.21.15.105:0/1633350942 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc00159610 0x7fbc000ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:04.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.148+0000 7fbc04c58640 1 -- 172.21.15.105:0/1633350942 >> v1:172.21.15.105:6789/0 conn(0x7fbc000cc760 legacy=0x7fbc0006c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:04.149 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.148+0000 7fbc04c58640 1 -- 172.21.15.105:0/1633350942 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc000cd220 con 0x7fbc0006cef0 2023-11-28T18:32:04.150 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.148+0000 7fbbfeffd640 1 -- 172.21.15.105:0/1633350942 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 369683446 0 0) 0x7fbc000cde30 con 0x7fbc000cc760 2023-11-28T18:32:04.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.148+0000 7fbc04c58640 1 --2- 172.21.15.105:0/1633350942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc0006cef0 0x7fbc000ccbc0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009e50 tx=0x7fbbf801c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8943ba75d6005ece server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:04.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.148+0000 7fbbfeffd640 1 -- 172.21.15.105:0/1633350942 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf80a5710 con 0x7fbc0006cef0 2023-11-28T18:32:04.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbbfeffd640 1 -- 172.21.15.105:0/1633350942 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf80a5870 con 0x7fbc0006cef0 2023-11-28T18:32:04.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbbfeffd640 1 -- 172.21.15.105:0/1633350942 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf80a5c00 con 0x7fbc0006cef0 2023-11-28T18:32:04.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbc066bb640 1 -- 172.21.15.105:0/1633350942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc0006cef0 msgr2=0x7fbc000ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:04.152 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbc066bb640 1 --2- 172.21.15.105:0/1633350942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc0006cef0 0x7fbc000ccbc0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009e50 tx=0x7fbbf801c9e0 comp rx=0 tx=0).stop 2023-11-28T18:32:04.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbc066bb640 1 -- 172.21.15.105:0/1633350942 shutdown_connections 2023-11-28T18:32:04.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbc066bb640 1 --2- 172.21.15.105:0/1633350942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc0006cef0 0x7fbc000ccbc0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:04.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbc066bb640 1 --2- 172.21.15.105:0/1633350942 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc00159610 0x7fbc000ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:04.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbc066bb640 1 -- 172.21.15.105:0/1633350942 >> 172.21.15.105:0/1633350942 conn(0x7fbc000ceb90 msgr2=0x7fbc000cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:04.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbc066bb640 1 -- 172.21.15.105:0/1633350942 shutdown_connections 2023-11-28T18:32:04.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbc066bb640 1 -- 172.21.15.105:0/1633350942 wait complete. 2023-11-28T18:32:04.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbc066bb640 1 Processor -- start 2023-11-28T18:32:04.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbc066bb640 1 -- start start 2023-11-28T18:32:04.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbc066bb640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc0006cef0 0x7fbc0015c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:04.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbc066bb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc000cc760 0x7fbc0015c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:04.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbc066bb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc00159610 0x7fbc00159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:04.155 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbc066bb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc00149090 con 0x7fbc00159610 2023-11-28T18:32:04.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbc066bb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc001484a0 con 0x7fbc000cc760 2023-11-28T18:32:04.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbc066bb640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc001486f0 con 0x7fbc0006cef0 2023-11-28T18:32:04.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbc04c58640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc00159610 0x7fbc00159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:04.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbc04c58640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc00159610 0x7fbc00159be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58548/0 (socket says 172.21.15.105:58548) 2023-11-28T18:32:04.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbc04c58640 1 -- 172.21.15.105:0/3169283190 learned_addr learned my addr 172.21.15.105:0/3169283190 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:04.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbbfffff640 1 --2- 172.21.15.105:0/3169283190 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc0006cef0 0x7fbc0015c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:04.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.152+0000 7fbbff7fe640 1 --2- 172.21.15.105:0/3169283190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc000cc760 0x7fbc0015c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:04.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.156+0000 7fbc04c58640 1 -- 172.21.15.105:0/3169283190 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc0006cef0 msgr2=0x7fbc0015c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:04.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.156+0000 7fbc04c58640 1 --2- 172.21.15.105:0/3169283190 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc0006cef0 0x7fbc0015c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:04.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.156+0000 7fbc04c58640 1 -- 172.21.15.105:0/3169283190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc000cc760 msgr2=0x7fbc0015c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:04.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.156+0000 7fbc04c58640 1 --2- 172.21.15.105:0/3169283190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc000cc760 0x7fbc0015c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:04.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.156+0000 7fbc04c58640 1 -- 172.21.15.105:0/3169283190 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf8009b30 con 0x7fbc00159610 2023-11-28T18:32:04.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.156+0000 7fbbff7fe640 1 --2- 172.21.15.105:0/3169283190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc000cc760 0x7fbc0015c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:32:04.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.156+0000 7fbc04c58640 1 --2- 172.21.15.105:0/3169283190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc00159610 0x7fbc00159be0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009e50 tx=0x7fbbf8019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:04.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.156+0000 7fbbfcff9640 1 -- 172.21.15.105:0/3169283190 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf80b8470 con 0x7fbc00159610 2023-11-28T18:32:04.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.156+0000 7fbbfcff9640 1 -- 172.21.15.105:0/3169283190 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf80b85d0 con 0x7fbc00159610 2023-11-28T18:32:04.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.156+0000 7fbbfcff9640 1 -- 172.21.15.105:0/3169283190 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf80b8940 con 0x7fbc00159610 2023-11-28T18:32:04.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.156+0000 7fbc066bb640 1 -- 172.21.15.105:0/3169283190 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc001489d0 con 0x7fbc00159610 2023-11-28T18:32:04.157 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.156+0000 7fbc066bb640 1 -- 172.21.15.105:0/3169283190 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc0015a120 con 0x7fbc00159610 2023-11-28T18:32:04.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.156+0000 7fbc066bb640 1 -- 172.21.15.105:0/3169283190 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc001491d0 con 0x7fbc00159610 2023-11-28T18:32:04.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.156+0000 7fbbfcff9640 1 -- 172.21.15.105:0/3169283190 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fbbf80b8aa0 con 0x7fbc00159610 2023-11-28T18:32:04.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.156+0000 7fbbfcff9640 1 --2- 172.21.15.105:0/3169283190 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbbec0668d0 0x7fbbec068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:04.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.160+0000 7fbbfcff9640 1 -- 172.21.15.105:0/3169283190 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbbf811fa60 con 0x7fbc00159610 2023-11-28T18:32:04.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.160+0000 7fbbfffff640 1 --2- 172.21.15.105:0/3169283190 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbbec0668d0 0x7fbbec068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:04.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.160+0000 7fbbfffff640 1 --2- 172.21.15.105:0/3169283190 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbbec0668d0 0x7fbbec068d90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00098f0 tx=0x7fbbf0006f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:04.162 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.160+0000 7fbbfcff9640 1 -- 172.21.15.105:0/3169283190 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbc001491d0 con 0x7fbc00159610 2023-11-28T18:32:04.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.364+0000 7fbc066bb640 1 -- 172.21.15.105:0/3169283190 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fbc001491d0 con 0x7fbc00159610 2023-11-28T18:32:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.368+0000 7fbbfcff9640 1 -- 172.21.15.105:0/3169283190 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+147 (secure 0 0 0) 0x7fbbf80bc0b0 con 0x7fbc00159610 2023-11-28T18:32:04.368 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2023-11-28T18:32:04.368 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDpL2ZlSt0VMxAAQ8+j1BgBPMJ2cuiZj8aG0w== 2023-11-28T18:32:04.369 INFO:teuthology.orchestra.run.smithi105.stdout: caps mds = "allow" 2023-11-28T18:32:04.369 INFO:teuthology.orchestra.run.smithi105.stdout: caps mon = "allow r" 2023-11-28T18:32:04.369 INFO:teuthology.orchestra.run.smithi105.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-28T18:32:04.369 INFO:teuthology.orchestra.run.smithi105.stderr:updated caps for client.0 2023-11-28T18:32:04.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.368+0000 7fbc066bb640 1 -- 172.21.15.105:0/3169283190 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbbec0668d0 msgr2=0x7fbbec068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.368+0000 7fbc066bb640 1 --2- 172.21.15.105:0/3169283190 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbbec0668d0 0x7fbbec068d90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00098f0 tx=0x7fbbf0006f90 comp rx=0 tx=0).stop 2023-11-28T18:32:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.368+0000 7fbc066bb640 1 -- 172.21.15.105:0/3169283190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc00159610 msgr2=0x7fbc00159be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.368+0000 7fbc066bb640 1 --2- 172.21.15.105:0/3169283190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc00159610 0x7fbc00159be0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009e50 tx=0x7fbbf8019780 comp rx=0 tx=0).stop 2023-11-28T18:32:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.372+0000 7fbc066bb640 1 -- 172.21.15.105:0/3169283190 shutdown_connections 2023-11-28T18:32:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.372+0000 7fbc066bb640 1 --2- 172.21.15.105:0/3169283190 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbbec0668d0 0x7fbbec068d90 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.372+0000 7fbc066bb640 1 --2- 172.21.15.105:0/3169283190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc00159610 0x7fbc00159be0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.372+0000 7fbc066bb640 1 --2- 172.21.15.105:0/3169283190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc000cc760 0x7fbc0015c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.372+0000 7fbc066bb640 1 --2- 172.21.15.105:0/3169283190 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc0006cef0 0x7fbc0015c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.372+0000 7fbc066bb640 1 -- 172.21.15.105:0/3169283190 >> 172.21.15.105:0/3169283190 conn(0x7fbc000ceb90 msgr2=0x7fbc000ae810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:04.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.372+0000 7fbc066bb640 1 -- 172.21.15.105:0/3169283190 shutdown_connections 2023-11-28T18:32:04.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.372+0000 7fbc066bb640 1 -- 172.21.15.105:0/3169283190 wait complete. 2023-11-28T18:32:04.390 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:32:04.428 INFO:tasks.ceph.mds.c.smithi105.stderr:2023-11-28T18:32:04.428+0000 7ff0bb051640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T18:32:04.527 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.528+0000 7f3e5c5f8640 1 Processor -- start 2023-11-28T18:32:04.528 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.528+0000 7f3e5c5f8640 1 -- start start 2023-11-28T18:32:04.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.528+0000 7f3e5c5f8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3e540d3c30 0x7f3e540d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:04.529 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.528+0000 7f3e5c5f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e540b2d20 0x7f3e540b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:04.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.528+0000 7f3e5c5f8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e540b25b0 con 0x7f3e5415d4f0 2023-11-28T18:32:04.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.528+0000 7f3e5c5f8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e540b2900 con 0x7f3e540d3c30 2023-11-28T18:32:04.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.528+0000 7f3e5c5f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e540d0a20 con 0x7f3e540b2d20 2023-11-28T18:32:04.530 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.528+0000 7f3e59b93640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3e540d3c30 0x7f3e540d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:04.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.528+0000 7f3e59b93640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3e540d3c30 0x7f3e540d1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58556/0 (socket says 172.21.15.105:58556) 2023-11-28T18:32:04.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.528+0000 7f3e59b93640 1 -- 172.21.15.105:0/1646781600 learned_addr learned my addr 172.21.15.105:0/1646781600 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:04.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.528+0000 7f3e5ab95640 1 --2- 172.21.15.105:0/1646781600 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e540b2d20 0x7f3e540b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:04.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.528+0000 7f3e59b93640 1 -- 172.21.15.105:0/1646781600 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e540b2d20 msgr2=0x7f3e540b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:04.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.528+0000 7f3e59b93640 1 --2- 172.21.15.105:0/1646781600 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e540b2d20 0x7f3e540b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:04.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.528+0000 7f3e59b93640 1 -- 172.21.15.105:0/1646781600 >> v1:172.21.15.148:6789/0 conn(0x7f3e5415d4f0 legacy=0x7f3e540d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:04.531 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.528+0000 7f3e59b93640 1 -- 172.21.15.105:0/1646781600 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e540cea30 con 0x7f3e540d3c30 2023-11-28T18:32:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.532+0000 7f3e59b93640 1 --2- 172.21.15.105:0/1646781600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e540d3c30 0x7f3e540d1520 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f3e44009870 tx=0x7f3e4401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eef35a6999abb21 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.532+0000 7f3e59392640 1 -- 172.21.15.105:0/1646781600 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e440ab680 con 0x7f3e540d3c30 2023-11-28T18:32:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.532+0000 7f3e59392640 1 -- 172.21.15.105:0/1646781600 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e440ab7e0 con 0x7f3e540d3c30 2023-11-28T18:32:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.532+0000 7f3e59392640 1 -- 172.21.15.105:0/1646781600 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e440abb70 con 0x7f3e540d3c30 2023-11-28T18:32:04.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.532+0000 7f3e5c5f8640 1 -- 172.21.15.105:0/1646781600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e540d3c30 msgr2=0x7f3e540d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:04.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.532+0000 7f3e5c5f8640 1 --2- 172.21.15.105:0/1646781600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e540d3c30 0x7f3e540d1520 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f3e44009870 tx=0x7f3e4401c480 comp rx=0 tx=0).stop 2023-11-28T18:32:04.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.532+0000 7f3e5c5f8640 1 -- 172.21.15.105:0/1646781600 shutdown_connections 2023-11-28T18:32:04.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.532+0000 7f3e5c5f8640 1 --2- 172.21.15.105:0/1646781600 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e540b2d20 0x7f3e540b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:04.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.532+0000 7f3e5c5f8640 1 --2- 172.21.15.105:0/1646781600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e540d3c30 0x7f3e540d1520 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:04.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.532+0000 7f3e5c5f8640 1 -- 172.21.15.105:0/1646781600 >> 172.21.15.105:0/1646781600 conn(0x7f3e540d3030 msgr2=0x7f3e540d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:04.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.532+0000 7f3e5c5f8640 1 -- 172.21.15.105:0/1646781600 shutdown_connections 2023-11-28T18:32:04.534 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.532+0000 7f3e5c5f8640 1 -- 172.21.15.105:0/1646781600 wait complete. 2023-11-28T18:32:04.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e5c5f8640 1 Processor -- start 2023-11-28T18:32:04.535 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e5c5f8640 1 -- start start 2023-11-28T18:32:04.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e5c5f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e540b2d20 0x7f3e54142c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:04.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e5c5f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e540d3c30 0x7f3e54158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:04.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e5c5f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e5415d4f0 0x7f3e540d1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:04.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e5c5f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e54143170 con 0x7f3e540d3c30 2023-11-28T18:32:04.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e5c5f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e54159180 con 0x7f3e540b2d20 2023-11-28T18:32:04.537 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e5c5f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e54159440 con 0x7f3e5415d4f0 2023-11-28T18:32:04.538 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e59b93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e540d3c30 0x7f3e54158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:04.538 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e59b93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e540d3c30 0x7f3e54158c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58572/0 (socket says 172.21.15.105:58572) 2023-11-28T18:32:04.538 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e59b93640 1 -- 172.21.15.105:0/4121494428 learned_addr learned my addr 172.21.15.105:0/4121494428 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:04.538 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e5a394640 1 --2- 172.21.15.105:0/4121494428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e540b2d20 0x7f3e54142c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:04.538 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e5ab95640 1 --2- 172.21.15.105:0/4121494428 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e5415d4f0 0x7f3e540d1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:04.538 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e59b93640 1 -- 172.21.15.105:0/4121494428 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e5415d4f0 msgr2=0x7f3e540d1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:04.538 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e59b93640 1 --2- 172.21.15.105:0/4121494428 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e5415d4f0 0x7f3e540d1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:04.538 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e59b93640 1 -- 172.21.15.105:0/4121494428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e540b2d20 msgr2=0x7f3e54142c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:04.539 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e59b93640 1 --2- 172.21.15.105:0/4121494428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e540b2d20 0x7f3e54142c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:04.539 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e59b93640 1 -- 172.21.15.105:0/4121494428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e440094d0 con 0x7f3e540d3c30 2023-11-28T18:32:04.539 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e59b93640 1 --2- 172.21.15.105:0/4121494428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e540d3c30 0x7f3e54158c40 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f3e4401a0b0 tx=0x7f3e4401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:04.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e4affd640 1 -- 172.21.15.105:0/4121494428 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e440b4800 con 0x7f3e540d3c30 2023-11-28T18:32:04.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e4affd640 1 -- 172.21.15.105:0/4121494428 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e440b4980 con 0x7f3e540d3c30 2023-11-28T18:32:04.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e4affd640 1 -- 172.21.15.105:0/4121494428 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e440b4d30 con 0x7f3e540d3c30 2023-11-28T18:32:04.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e5c5f8640 1 -- 172.21.15.105:0/4121494428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e54159720 con 0x7f3e540d3c30 2023-11-28T18:32:04.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.536+0000 7f3e5c5f8640 1 -- 172.21.15.105:0/4121494428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e54159ce0 con 0x7f3e540d3c30 2023-11-28T18:32:04.540 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.540+0000 7f3e5c5f8640 1 -- 172.21.15.105:0/4121494428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e54067570 con 0x7f3e540d3c30 2023-11-28T18:32:04.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.540+0000 7f3e4affd640 1 -- 172.21.15.105:0/4121494428 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3e440b3380 con 0x7f3e540d3c30 2023-11-28T18:32:04.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.540+0000 7f3e4affd640 1 --2- 172.21.15.105:0/4121494428 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3e380668d0 0x7f3e38068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:04.541 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.540+0000 7f3e4affd640 1 -- 172.21.15.105:0/4121494428 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3e4411eec0 con 0x7f3e540d3c30 2023-11-28T18:32:04.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.540+0000 7f3e5a394640 1 --2- 172.21.15.105:0/4121494428 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3e380668d0 0x7f3e38068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:04.542 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.540+0000 7f3e5a394640 1 --2- 172.21.15.105:0/4121494428 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3e380668d0 0x7f3e38068d90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3e50007fd0 tx=0x7f3e50009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:04.545 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.544+0000 7f3e4affd640 1 -- 172.21.15.105:0/4121494428 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3e440ec050 con 0x7f3e540d3c30 2023-11-28T18:32:04.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.728+0000 7f3e5c5f8640 1 -- 172.21.15.105:0/4121494428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e540d00f0 con 0x7f3e540d3c30 2023-11-28T18:32:04.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.728+0000 7f3e4affd640 1 -- 172.21.15.105:0/4121494428 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) v1 ==== 95+0+4933 (secure 0 0 0) 0x7f3e440c4020 con 0x7f3e540d3c30 2023-11-28T18:32:04.729 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:32:04.729 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":134,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3161918450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3161918450},{"type":"v1","addr":"172.21.15.105:6835","nonce":3161918450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":129},{"gid":9120,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/3958106803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3958106803},{"type":"v1","addr":"172.21.15.148:6834","nonce":3958106803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":129},{"gid":9123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3706038483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":3706038483},{"type":"v1","addr":"172.21.15.148:6835","nonce":3706038483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":129}],"filesystems":[{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:31:58.827408+0000","modified":"2023-11-28T18:32:01.852953+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9126},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9126":{"gid":9126,"name":"c","rank":0,"incarnation":130,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/1978722810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1978722810},{"type":"v1","addr":"172.21.15.105:6837","nonce":1978722810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-28T18:32:04.729 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 134 2023-11-28T18:32:04.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.732+0000 7f3e5c5f8640 1 -- 172.21.15.105:0/4121494428 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3e380668d0 msgr2=0x7f3e38068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:04.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.732+0000 7f3e5c5f8640 1 --2- 172.21.15.105:0/4121494428 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3e380668d0 0x7f3e38068d90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3e50007fd0 tx=0x7f3e50009040 comp rx=0 tx=0).stop 2023-11-28T18:32:04.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.732+0000 7f3e5c5f8640 1 -- 172.21.15.105:0/4121494428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e540d3c30 msgr2=0x7f3e54158c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:04.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.732+0000 7f3e5c5f8640 1 --2- 172.21.15.105:0/4121494428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e540d3c30 0x7f3e54158c40 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f3e4401a0b0 tx=0x7f3e4401acc0 comp rx=0 tx=0).stop 2023-11-28T18:32:04.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.732+0000 7f3e5c5f8640 1 -- 172.21.15.105:0/4121494428 shutdown_connections 2023-11-28T18:32:04.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.732+0000 7f3e5c5f8640 1 --2- 172.21.15.105:0/4121494428 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e5415d4f0 0x7f3e540d1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:04.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.732+0000 7f3e5c5f8640 1 --2- 172.21.15.105:0/4121494428 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3e380668d0 0x7f3e38068d90 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:04.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.732+0000 7f3e5c5f8640 1 --2- 172.21.15.105:0/4121494428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e540d3c30 0x7f3e54158c40 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:04.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.732+0000 7f3e5c5f8640 1 --2- 172.21.15.105:0/4121494428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e540b2d20 0x7f3e54142c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:04.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.732+0000 7f3e5c5f8640 1 -- 172.21.15.105:0/4121494428 >> 172.21.15.105:0/4121494428 conn(0x7f3e540d3030 msgr2=0x7f3e540afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:04.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.732+0000 7f3e5c5f8640 1 -- 172.21.15.105:0/4121494428 shutdown_connections 2023-11-28T18:32:04.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:04.732+0000 7f3e5c5f8640 1 -- 172.21.15.105:0/4121494428 wait complete. 2023-11-28T18:32:05.152 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 134, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:31:58.827408+0000', 'modified': '2023-11-28T18:32:01.852953+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9126}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9126': {'gid': 9126, 'name': 'c', 'rank': 0, 'incarnation': 130, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6837/1978722810', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 1978722810}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 1978722810}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:32:05.152 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T18:32:05.152 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T18:32:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.292+0000 7f277e8e3640 1 Processor -- start 2023-11-28T18:32:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.292+0000 7f277e8e3640 1 -- start start 2023-11-28T18:32:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.296+0000 7f277e8e3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2778159f40 0x7f27780caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.296+0000 7f277e8e3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f27780ad850 0x7f27780701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.296+0000 7f277e8e3640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f277806bf50 con 0x7f27780cc760 2023-11-28T18:32:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.296+0000 7f277e8e3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27780ccbc0 con 0x7f27780ad850 2023-11-28T18:32:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.296+0000 7f277e8e3640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277815a320 con 0x7f2778159f40 2023-11-28T18:32:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.296+0000 7f27777fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f27780cc760 0x7f27780ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41800/0 (socket says 172.21.15.105:41800) 2023-11-28T18:32:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.296+0000 7f27777fe640 1 -- 172.21.15.105:0/1364607204 learned_addr learned my addr 172.21.15.105:0/1364607204 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:32:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.296+0000 7f277ce80640 1 --2- 172.21.15.105:0/1364607204 >> v2:172.21.15.148:3300/0 conn(0x7f27780ad850 0x7f27780701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.296+0000 7f2777fff640 1 --2- 172.21.15.105:0/1364607204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2778159f40 0x7f27780caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.296+0000 7f2776ffd640 1 -- 172.21.15.105:0/1364607204 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 300699493 0 0) 0x7f277806bf50 con 0x7f27780cc760 2023-11-28T18:32:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.296+0000 7f2776ffd640 1 -- 172.21.15.105:0/1364607204 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2760003530 con 0x7f27780cc760 2023-11-28T18:32:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.296+0000 7f2776ffd640 1 -- 172.21.15.105:0/1364607204 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4219481954 0 0) 0x7f2760003530 con 0x7f27780cc760 2023-11-28T18:32:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.296+0000 7f277ce80640 1 -- 172.21.15.105:0/1364607204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2778159f40 msgr2=0x7f27780caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.296+0000 7f277ce80640 1 --2- 172.21.15.105:0/1364607204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2778159f40 0x7f27780caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.296+0000 7f277ce80640 1 -- 172.21.15.105:0/1364607204 >> v1:172.21.15.105:6789/0 conn(0x7f27780cc760 legacy=0x7f27780ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.296+0000 7f277ce80640 1 -- 172.21.15.105:0/1364607204 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2778071380 con 0x7f27780ad850 2023-11-28T18:32:05.299 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.296+0000 7f2777fff640 1 --2- 172.21.15.105:0/1364607204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2778159f40 0x7f27780caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:32:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.300+0000 7f277ce80640 1 --2- 172.21.15.105:0/1364607204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27780ad850 0x7f27780701d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2770009d90 tx=0x7f277001c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89934b79243514a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.300+0000 7f2776ffd640 1 -- 172.21.15.105:0/1364607204 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c001680 con 0x7f27780ad850 2023-11-28T18:32:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.300+0000 7f2776ffd640 1 -- 172.21.15.105:0/1364607204 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27700a55b0 con 0x7f27780ad850 2023-11-28T18:32:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.300+0000 7f2776ffd640 1 -- 172.21.15.105:0/1364607204 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27700a5940 con 0x7f27780ad850 2023-11-28T18:32:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.300+0000 7f277e8e3640 1 -- 172.21.15.105:0/1364607204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27780ad850 msgr2=0x7f27780701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.300+0000 7f277e8e3640 1 --2- 172.21.15.105:0/1364607204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27780ad850 0x7f27780701d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2770009d90 tx=0x7f277001c7b0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.300+0000 7f277e8e3640 1 -- 172.21.15.105:0/1364607204 shutdown_connections 2023-11-28T18:32:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.300+0000 7f277e8e3640 1 --2- 172.21.15.105:0/1364607204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27780ad850 0x7f27780701d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.300+0000 7f277e8e3640 1 --2- 172.21.15.105:0/1364607204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2778159f40 0x7f27780caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.300+0000 7f277e8e3640 1 -- 172.21.15.105:0/1364607204 >> 172.21.15.105:0/1364607204 conn(0x7f27780ceb90 msgr2=0x7f27780cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:05.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.300+0000 7f277e8e3640 1 -- 172.21.15.105:0/1364607204 shutdown_connections 2023-11-28T18:32:05.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.300+0000 7f277e8e3640 1 -- 172.21.15.105:0/1364607204 wait complete. 2023-11-28T18:32:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.300+0000 7f277e8e3640 1 Processor -- start 2023-11-28T18:32:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.300+0000 7f277e8e3640 1 -- start start 2023-11-28T18:32:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f277e8e3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27780ad850 0x7f277814eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f277e8e3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27780cc760 0x7f27781591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f277e8e3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2778159f40 0x7f277814f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f277e8e3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277814c8b0 con 0x7f2778159f40 2023-11-28T18:32:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f277e8e3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2778157920 con 0x7f27780ad850 2023-11-28T18:32:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f277e8e3640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2778157c00 con 0x7f27780cc760 2023-11-28T18:32:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f277ce80640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2778159f40 0x7f277814f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f277ce80640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2778159f40 0x7f277814f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58574/0 (socket says 172.21.15.105:58574) 2023-11-28T18:32:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f277ce80640 1 -- 172.21.15.105:0/3252259845 learned_addr learned my addr 172.21.15.105:0/3252259845 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f2777fff640 1 --2- 172.21.15.105:0/3252259845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27780ad850 0x7f277814eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f27777fe640 1 --2- 172.21.15.105:0/3252259845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27780cc760 0x7f27781591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f277ce80640 1 -- 172.21.15.105:0/3252259845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27780cc760 msgr2=0x7f27781591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:05.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f277ce80640 1 --2- 172.21.15.105:0/3252259845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27780cc760 0x7f27781591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f277ce80640 1 -- 172.21.15.105:0/3252259845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27780ad850 msgr2=0x7f277814eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:05.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f277ce80640 1 --2- 172.21.15.105:0/3252259845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27780ad850 0x7f277814eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f277ce80640 1 -- 172.21.15.105:0/3252259845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2768009420 con 0x7f2778159f40 2023-11-28T18:32:05.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f27777fe640 1 --2- 172.21.15.105:0/3252259845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27780cc760 0x7f27781591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:32:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f277ce80640 1 --2- 172.21.15.105:0/3252259845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2778159f40 0x7f277814f6d0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f27700b0040 tx=0x7f2770019550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f2774ff9640 1 -- 172.21.15.105:0/3252259845 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27700b9650 con 0x7f2778159f40 2023-11-28T18:32:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f2774ff9640 1 -- 172.21.15.105:0/3252259845 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27700b97b0 con 0x7f2778159f40 2023-11-28T18:32:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f2774ff9640 1 -- 172.21.15.105:0/3252259845 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27700b9b60 con 0x7f2778159f40 2023-11-28T18:32:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f277e8e3640 1 -- 172.21.15.105:0/3252259845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2770009a70 con 0x7f2778159f40 2023-11-28T18:32:05.310 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.304+0000 7f277e8e3640 1 -- 172.21.15.105:0/3252259845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f277814c4d0 con 0x7f2778159f40 2023-11-28T18:32:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f2774ff9640 1 -- 172.21.15.105:0/3252259845 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f27700b9cc0 con 0x7f2778159f40 2023-11-28T18:32:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f2774ff9640 1 --2- 172.21.15.105:0/3252259845 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27640668d0 0x7f2764068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f2774ff9640 1 -- 172.21.15.105:0/3252259845 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f277011f820 con 0x7f2778159f40 2023-11-28T18:32:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f2777fff640 1 --2- 172.21.15.105:0/3252259845 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27640668d0 0x7f2764068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f2781c3f000 1 Processor -- start 2023-11-28T18:32:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f2781c3f000 1 -- start start 2023-11-28T18:32:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f2777fff640 1 --2- 172.21.15.105:0/3252259845 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27640668d0 0x7f2764068d90 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2768009c50 tx=0x7f2768009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f2781c3f000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a912d7fd30 0x55a912d80110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f2781c3f000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d80650 0x55a912d32fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f2781c3f000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912d33500 0x55a912d338e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f2781c3f000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a912bfc110 con 0x55a912d80650 2023-11-28T18:32:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f2781c3f000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a912b8c9a0 con 0x55a912d33500 2023-11-28T18:32:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f2781c3f000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a912b89ed0 con 0x55a912d7fd30 2023-11-28T18:32:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f27777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d80650 0x55a912d32fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f27777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d80650 0x55a912d32fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58578/0 (socket says 172.21.15.105:58578) 2023-11-28T18:32:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f27777fe640 1 -- 172.21.15.105:0/968062032 learned_addr learned my addr 172.21.15.105:0/968062032 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f2777fff640 1 --2- 172.21.15.105:0/968062032 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a912d7fd30 0x55a912d80110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f277ce80640 1 --2- 172.21.15.105:0/968062032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912d33500 0x55a912d338e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f27777fe640 1 -- 172.21.15.105:0/968062032 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a912d7fd30 msgr2=0x55a912d80110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f27777fe640 1 --2- 172.21.15.105:0/968062032 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a912d7fd30 0x55a912d80110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f27777fe640 1 -- 172.21.15.105:0/968062032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912d33500 msgr2=0x55a912d338e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f27777fe640 1 --2- 172.21.15.105:0/968062032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912d33500 0x55a912d338e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f27777fe640 1 -- 172.21.15.105:0/968062032 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a91287d890 con 0x55a912d80650 2023-11-28T18:32:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f27777fe640 1 --2- 172.21.15.105:0/968062032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d80650 0x55a912d32fc0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f276c001390 tx=0x7f276c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b60ca185189edb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f275dffb640 1 -- 172.21.15.105:0/968062032 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c005af0 con 0x55a912d80650 2023-11-28T18:32:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f275dffb640 1 -- 172.21.15.105:0/968062032 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276c005c50 con 0x55a912d80650 2023-11-28T18:32:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.308+0000 7f275dffb640 1 -- 172.21.15.105:0/968062032 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c00e800 con 0x55a912d80650 2023-11-28T18:32:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2781c3f000 1 -- 172.21.15.105:0/968062032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d80650 msgr2=0x55a912d32fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2781c3f000 1 --2- 172.21.15.105:0/968062032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d80650 0x55a912d32fc0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f276c001390 tx=0x7f276c007e10 comp rx=0 tx=0).stop 2023-11-28T18:32:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2781c3f000 1 -- 172.21.15.105:0/968062032 shutdown_connections 2023-11-28T18:32:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2781c3f000 1 --2- 172.21.15.105:0/968062032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912d33500 0x55a912d338e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2781c3f000 1 --2- 172.21.15.105:0/968062032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d80650 0x55a912d32fc0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2781c3f000 1 --2- 172.21.15.105:0/968062032 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a912d7fd30 0x55a912d80110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2781c3f000 1 -- 172.21.15.105:0/968062032 >> 172.21.15.105:0/968062032 conn(0x55a912d608c0 msgr2=0x55a912d62cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2781c3f000 1 -- 172.21.15.105:0/968062032 shutdown_connections 2023-11-28T18:32:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2781c3f000 1 -- 172.21.15.105:0/968062032 wait complete. 2023-11-28T18:32:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2781c3f000 1 -- start start 2023-11-28T18:32:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2781c3f000 1 Processor -- start 2023-11-28T18:32:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2781c3f000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a912d33500 0x55a912d394e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2781c3f000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d7fd30 0x55a912d39a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2781c3f000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912d80650 0x55a912d39f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2781c3f000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a912b857c0 con 0x55a912d7fd30 2023-11-28T18:32:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2781c3f000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a912b3dbf0 con 0x55a912d80650 2023-11-28T18:32:05.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2781c3f000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a912b43a90 con 0x55a912d33500 2023-11-28T18:32:05.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2781c3f000 10 client.0 _ll_register_callbacks cb 0x55a912d20e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:32:05.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f27777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d7fd30 0x55a912d39a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.322 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f27777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d7fd30 0x55a912d39a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58584/0 (socket says 172.21.15.105:58584) 2023-11-28T18:32:05.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f27777fe640 1 -- 172.21.15.105:0/1656288386 learned_addr learned my addr 172.21.15.105:0/1656288386 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:05.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2777fff640 1 --2- 172.21.15.105:0/1656288386 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a912d33500 0x55a912d394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f277ce80640 1 --2- 172.21.15.105:0/1656288386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912d80650 0x55a912d39f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f27777fe640 1 -- 172.21.15.105:0/1656288386 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a912d33500 msgr2=0x55a912d394e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:05.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f27777fe640 1 --2- 172.21.15.105:0/1656288386 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a912d33500 0x55a912d394e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f27777fe640 1 -- 172.21.15.105:0/1656288386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912d80650 msgr2=0x55a912d39f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:05.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f27777fe640 1 --2- 172.21.15.105:0/1656288386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912d80650 0x55a912d39f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.324 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f27777fe640 1 -- 172.21.15.105:0/1656288386 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f276c00f040 con 0x55a912d7fd30 2023-11-28T18:32:05.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f27777fe640 1 --2- 172.21.15.105:0/1656288386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d7fd30 0x55a912d39a20 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f276c017040 tx=0x7f276c0015a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69eb5107087db5ea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:05.325 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2756ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:32:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2756ffd640 1 -- 172.21.15.105:0/1656288386 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c005e20 con 0x55a912d7fd30 2023-11-28T18:32:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2781c3f000 1 -- 172.21.15.105:0/1656288386 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a912b47470 con 0x55a912d7fd30 2023-11-28T18:32:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2756ffd640 1 -- 172.21.15.105:0/1656288386 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f276c0072e0 con 0x55a912d7fd30 2023-11-28T18:32:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2756ffd640 1 -- 172.21.15.105:0/1656288386 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=134) v2 ==== 24+0+0 (secure 0 0 0) 0x7f276c0074a0 con 0x55a912d7fd30 2023-11-28T18:32:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2781c3f000 10 client.9258 fetch_fsmap learned FSMap version 134 2023-11-28T18:32:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2781c3f000 1 -- 172.21.15.105:0/1656288386 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=134,monmap=2+}) v3 -- 0x55a912b46190 con 0x55a912d7fd30 2023-11-28T18:32:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.312+0000 7f2756ffd640 1 -- 172.21.15.105:0/1656288386 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a912b47470 con 0x55a912d7fd30 2023-11-28T18:32:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2756ffd640 1 -- 172.21.15.105:0/1656288386 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 134) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f276c007600 con 0x55a912d7fd30 2023-11-28T18:32:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2781c3f000 10 client.9258 fetch_fsmap finished waiting for FSMap version 134 2023-11-28T18:32:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2781c3f000 10 client.9258 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:32:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2781c3f000 20 client.9258 populate_metadata read hostname 'smithi105' 2023-11-28T18:32:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2781c3f000 1 --2- 172.21.15.105:0/1656288386 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55a912d3a970 0x55a912d91ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2781c3f000 4 client.9258 mds_command: new command op to 9126 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:32:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2781c3f000 1 -- 172.21.15.105:0/1656288386 --> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a912b4ab50 con 0x55a912d3a970 2023-11-28T18:32:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2777fff640 1 --2- 172.21.15.105:0/1656288386 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55a912d3a970 0x55a912d91ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2777fff640 1 --2- 172.21.15.105:0/1656288386 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55a912d3a970 0x55a912d91ff0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8f9f905a2b37cca server_cookie=818485a1c0b29594 in_seq=0 out_seq=0 2023-11-28T18:32:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2756ffd640 10 client.9258 ms_handle_connect on v2:172.21.15.105:6836/1978722810 2023-11-28T18:32:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2756ffd640 1 -- 172.21.15.105:0/1656288386 <== mds.0 v2:172.21.15.105:6836/1978722810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a912b4ab50 con 0x55a912d3a970 2023-11-28T18:32:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2756ffd640 10 client.9258 handle_command_reply: tid=0 2023-11-28T18:32:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2781c3f000 1 client.9258 shutdown 2023-11-28T18:32:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2781c3f000 1 -- 172.21.15.105:0/1656288386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d7fd30 msgr2=0x55a912d39a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2781c3f000 1 --2- 172.21.15.105:0/1656288386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d7fd30 0x55a912d39a20 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f276c017040 tx=0x7f276c0015a0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2781c3f000 1 -- 172.21.15.105:0/1656288386 shutdown_connections 2023-11-28T18:32:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2781c3f000 1 --2- 172.21.15.105:0/1656288386 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55a912d3a970 0x55a912d91ff0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2781c3f000 1 --2- 172.21.15.105:0/1656288386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912d80650 0x55a912d39f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2781c3f000 1 --2- 172.21.15.105:0/1656288386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d7fd30 0x55a912d39a20 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2756ffd640 0 client.9258 ms_handle_reset on v2:172.21.15.105:6836/1978722810 2023-11-28T18:32:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2781c3f000 1 --2- 172.21.15.105:0/1656288386 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a912d33500 0x55a912d394e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2781c3f000 1 -- 172.21.15.105:0/1656288386 >> 172.21.15.105:0/1656288386 conn(0x55a912d608c0 msgr2=0x55a912d326b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2781c3f000 1 -- 172.21.15.105:0/1656288386 shutdown_connections 2023-11-28T18:32:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2781c3f000 1 -- 172.21.15.105:0/1656288386 wait complete. 2023-11-28T18:32:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.316+0000 7f2781c3f000 20 client.9258 trim_cache size 0 max 16384 2023-11-28T18:32:05.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f2781c3f000 1 Processor -- start 2023-11-28T18:32:05.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f2781c3f000 1 -- start start 2023-11-28T18:32:05.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f2781c3f000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912d33500 0x55a912d5f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f2781c3f000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a912d7fd30 0x55a912d5f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f2781c3f000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d80650 0x55a912d91ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f2781c3f000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91283e940 con 0x55a912d80650 2023-11-28T18:32:05.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f2781c3f000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a912b208a0 con 0x55a912d33500 2023-11-28T18:32:05.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f2781c3f000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a912c0c570 con 0x55a912d7fd30 2023-11-28T18:32:05.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f277ce80640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d80650 0x55a912d91ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f277ce80640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d80650 0x55a912d91ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58590/0 (socket says 172.21.15.105:58590) 2023-11-28T18:32:05.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f277ce80640 1 -- 172.21.15.105:0/4285020894 learned_addr learned my addr 172.21.15.105:0/4285020894 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:05.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f2777fff640 1 --2- 172.21.15.105:0/4285020894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912d33500 0x55a912d5f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f27777fe640 1 --2- 172.21.15.105:0/4285020894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a912d7fd30 0x55a912d5f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f277ce80640 1 -- 172.21.15.105:0/4285020894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a912d7fd30 msgr2=0x55a912d5f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f277ce80640 1 --2- 172.21.15.105:0/4285020894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a912d7fd30 0x55a912d5f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f277ce80640 1 -- 172.21.15.105:0/4285020894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912d33500 msgr2=0x55a912d5f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f277ce80640 1 --2- 172.21.15.105:0/4285020894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912d33500 0x55a912d5f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f277ce80640 1 -- 172.21.15.105:0/4285020894 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a912d33370 con 0x55a912d80650 2023-11-28T18:32:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f27777fe640 1 --2- 172.21.15.105:0/4285020894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a912d7fd30 0x55a912d5f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:32:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f277ce80640 1 --2- 172.21.15.105:0/4285020894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d80650 0x55a912d91ff0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f2770009d90 tx=0x7f277011f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b3e921a24fdad2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f275e7fc640 1 -- 172.21.15.105:0/4285020894 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27700f4840 con 0x55a912d80650 2023-11-28T18:32:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f275e7fc640 1 -- 172.21.15.105:0/4285020894 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27700f89f0 con 0x55a912d80650 2023-11-28T18:32:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.332+0000 7f275e7fc640 1 -- 172.21.15.105:0/4285020894 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27700ece00 con 0x55a912d80650 2023-11-28T18:32:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2781c3f000 1 -- 172.21.15.105:0/4285020894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d80650 msgr2=0x55a912d91ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:05.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2781c3f000 1 --2- 172.21.15.105:0/4285020894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d80650 0x55a912d91ff0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f2770009d90 tx=0x7f277011f040 comp rx=0 tx=0).stop 2023-11-28T18:32:05.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2781c3f000 1 -- 172.21.15.105:0/4285020894 shutdown_connections 2023-11-28T18:32:05.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2781c3f000 1 --2- 172.21.15.105:0/4285020894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d80650 0x55a912d91ff0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2781c3f000 1 --2- 172.21.15.105:0/4285020894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a912d7fd30 0x55a912d5f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2781c3f000 1 --2- 172.21.15.105:0/4285020894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912d33500 0x55a912d5f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2781c3f000 1 -- 172.21.15.105:0/4285020894 >> 172.21.15.105:0/4285020894 conn(0x55a912d608c0 msgr2=0x55a912d31720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:05.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2781c3f000 1 -- 172.21.15.105:0/4285020894 shutdown_connections 2023-11-28T18:32:05.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2781c3f000 1 -- 172.21.15.105:0/4285020894 wait complete. 2023-11-28T18:32:05.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2781c3f000 1 -- start start 2023-11-28T18:32:05.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2781c3f000 1 Processor -- start 2023-11-28T18:32:05.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2781c3f000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d33500 0x55a912de9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2781c3f000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a912d7fd30 0x55a912de9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2781c3f000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912d80650 0x55a912da4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2781c3f000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a912b4ab50 con 0x55a912d33500 2023-11-28T18:32:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2781c3f000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a912b4f2e0 con 0x55a912d80650 2023-11-28T18:32:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2781c3f000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a912d96cd0 con 0x55a912d7fd30 2023-11-28T18:32:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2781c3f000 10 client.0 _ll_register_callbacks cb 0x55a912bfc060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:32:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2777fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d33500 0x55a912de9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2777fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d33500 0x55a912de9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58602/0 (socket says 172.21.15.105:58602) 2023-11-28T18:32:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2777fff640 1 -- 172.21.15.105:0/3600407469 learned_addr learned my addr 172.21.15.105:0/3600407469 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f27777fe640 1 --2- 172.21.15.105:0/3600407469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a912d7fd30 0x55a912de9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f277ce80640 1 --2- 172.21.15.105:0/3600407469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912d80650 0x55a912da4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2777fff640 1 -- 172.21.15.105:0/3600407469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a912d7fd30 msgr2=0x55a912de9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2777fff640 1 --2- 172.21.15.105:0/3600407469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a912d7fd30 0x55a912de9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2777fff640 1 -- 172.21.15.105:0/3600407469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912d80650 msgr2=0x55a912da4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2777fff640 1 --2- 172.21.15.105:0/3600407469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912d80650 0x55a912da4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2777fff640 1 -- 172.21.15.105:0/3600407469 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f277814c4d0 con 0x55a912d33500 2023-11-28T18:32:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f27777fe640 1 --2- 172.21.15.105:0/3600407469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a912d7fd30 0x55a912de9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:32:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f277ce80640 1 --2- 172.21.15.105:0/3600407469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912d80650 0x55a912da4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:32:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2777fff640 1 --2- 172.21.15.105:0/3600407469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d33500 0x55a912de9020 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f2768011b20 tx=0x7f2768004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fa60dcd7f37f58 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2756ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:32:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2756ffd640 1 -- 172.21.15.105:0/3600407469 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276802a550 con 0x55a912d33500 2023-11-28T18:32:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2756ffd640 1 -- 172.21.15.105:0/3600407469 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2768028070 con 0x55a912d33500 2023-11-28T18:32:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2781c3f000 1 -- 172.21.15.105:0/3600407469 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a912b84110 con 0x55a912d33500 2023-11-28T18:32:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2756ffd640 1 -- 172.21.15.105:0/3600407469 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=134) v2 ==== 24+0+0 (secure 0 0 0) 0x7f276802a6b0 con 0x55a912d33500 2023-11-28T18:32:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2781c3f000 10 client.9264 fetch_fsmap learned FSMap version 134 2023-11-28T18:32:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.336+0000 7f2781c3f000 1 -- 172.21.15.105:0/3600407469 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=134,monmap=2+}) v3 -- 0x55a912b2c490 con 0x55a912d33500 2023-11-28T18:32:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2756ffd640 1 -- 172.21.15.105:0/3600407469 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a912b84110 con 0x55a912d33500 2023-11-28T18:32:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2756ffd640 1 -- 172.21.15.105:0/3600407469 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 134) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2768003620 con 0x55a912d33500 2023-11-28T18:32:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2781c3f000 10 client.9264 fetch_fsmap finished waiting for FSMap version 134 2023-11-28T18:32:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2781c3f000 10 client.9264 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:32:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2781c3f000 20 client.9264 populate_metadata read hostname 'smithi105' 2023-11-28T18:32:05.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2781c3f000 1 --2- 172.21.15.105:0/3600407469 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55a912defe80 0x55a912df2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2781c3f000 4 client.9264 mds_command: new command op to 9126 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:32:05.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2781c3f000 1 -- 172.21.15.105:0/3600407469 --> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a912bb2650 con 0x55a912defe80 2023-11-28T18:32:05.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f27777fe640 1 --2- 172.21.15.105:0/3600407469 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55a912defe80 0x55a912df2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2756ffd640 10 client.9264 ms_handle_connect on v2:172.21.15.105:6836/1978722810 2023-11-28T18:32:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f27777fe640 1 --2- 172.21.15.105:0/3600407469 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55a912defe80 0x55a912df2270 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0bd0d4354732916 server_cookie=5df063732a6240d8 in_seq=0 out_seq=0 2023-11-28T18:32:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2756ffd640 1 -- 172.21.15.105:0/3600407469 <== mds.0 v2:172.21.15.105:6836/1978722810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x55a912bb2650 con 0x55a912defe80 2023-11-28T18:32:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2756ffd640 10 client.9264 handle_command_reply: tid=0 2023-11-28T18:32:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2781c3f000 1 client.9264 shutdown 2023-11-28T18:32:05.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2781c3f000 1 -- 172.21.15.105:0/3600407469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d33500 msgr2=0x55a912de9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:05.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2781c3f000 1 --2- 172.21.15.105:0/3600407469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d33500 0x55a912de9020 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f2768011b20 tx=0x7f2768004740 comp rx=0 tx=0).stop 2023-11-28T18:32:05.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2781c3f000 1 -- 172.21.15.105:0/3600407469 shutdown_connections 2023-11-28T18:32:05.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2781c3f000 1 --2- 172.21.15.105:0/3600407469 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55a912defe80 0x55a912df2270 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.350 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2781c3f000 1 --2- 172.21.15.105:0/3600407469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a912d80650 0x55a912da4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2781c3f000 1 --2- 172.21.15.105:0/3600407469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a912d7fd30 0x55a912de9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2756ffd640 0 client.9264 ms_handle_reset on v2:172.21.15.105:6836/1978722810 2023-11-28T18:32:05.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2781c3f000 1 --2- 172.21.15.105:0/3600407469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a912d33500 0x55a912de9020 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2781c3f000 1 -- 172.21.15.105:0/3600407469 >> 172.21.15.105:0/3600407469 conn(0x55a912d608c0 msgr2=0x55a912d31fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:05.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2781c3f000 1 -- 172.21.15.105:0/3600407469 shutdown_connections 2023-11-28T18:32:05.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2781c3f000 1 -- 172.21.15.105:0/3600407469 wait complete. 2023-11-28T18:32:05.351 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f2781c3f000 20 client.9264 trim_cache size 0 max 16384 2023-11-28T18:32:05.352 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:32:05.352 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:32:05.352 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2023-11-28T18:32:05.352 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 9126, 2023-11-28T18:32:05.352 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:32:05.352 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:32:05.352 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:32:05.352 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 5.8760421100000002, 2023-11-28T18:32:05.353 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 134, 2023-11-28T18:32:05.353 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 129, 2023-11-28T18:32:05.353 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 129, 2023-11-28T18:32:05.353 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 11.705775954 2023-11-28T18:32:05.353 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:32:05.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f277e8e3640 1 -- 172.21.15.105:0/3252259845 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27640668d0 msgr2=0x7f2764068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:05.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f277e8e3640 1 --2- 172.21.15.105:0/3252259845 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27640668d0 0x7f2764068d90 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2768009c50 tx=0x7f2768009190 comp rx=0 tx=0).stop 2023-11-28T18:32:05.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f277e8e3640 1 -- 172.21.15.105:0/3252259845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2778159f40 msgr2=0x7f277814f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:05.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f277e8e3640 1 --2- 172.21.15.105:0/3252259845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2778159f40 0x7f277814f6d0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f27700b0040 tx=0x7f2770019550 comp rx=0 tx=0).stop 2023-11-28T18:32:05.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f277e8e3640 1 -- 172.21.15.105:0/3252259845 shutdown_connections 2023-11-28T18:32:05.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f277e8e3640 1 --2- 172.21.15.105:0/3252259845 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f27640668d0 0x7f2764068d90 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f277e8e3640 1 --2- 172.21.15.105:0/3252259845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2778159f40 0x7f277814f6d0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f277e8e3640 1 --2- 172.21.15.105:0/3252259845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27780cc760 0x7f27781591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.354 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f277e8e3640 1 --2- 172.21.15.105:0/3252259845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27780ad850 0x7f277814eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f277e8e3640 1 -- 172.21.15.105:0/3252259845 >> 172.21.15.105:0/3252259845 conn(0x7f27780ceb90 msgr2=0x7f2778071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f277e8e3640 1 -- 172.21.15.105:0/3252259845 shutdown_connections 2023-11-28T18:32:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.340+0000 7f277e8e3640 1 -- 172.21.15.105:0/3252259845 wait complete. 2023-11-28T18:32:05.360 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T18:32:05.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.496+0000 7f1ed6a9c640 1 Processor -- start 2023-11-28T18:32:05.497 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.496+0000 7f1ed6a9c640 1 -- start start 2023-11-28T18:32:05.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.500+0000 7f1ed6a9c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1ed01575b0 0x7f1ed015ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.500+0000 7f1ed6a9c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1ed015d7b0 0x7f1ed006cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.499 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.500+0000 7f1ed6a9c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ed015e0b0 con 0x7f1ed0157990 2023-11-28T18:32:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.500+0000 7f1ed6a9c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed00ca660 con 0x7f1ed01575b0 2023-11-28T18:32:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.500+0000 7f1ed6a9c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed015e390 con 0x7f1ed015d7b0 2023-11-28T18:32:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.500+0000 7f1ed5a9a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1ed01575b0 0x7f1ed015ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.500+0000 7f1ed5299640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f1ed0157990 0x7f1ed00c9f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41808/0 (socket says 172.21.15.105:41808) 2023-11-28T18:32:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.500+0000 7f1ed5299640 1 -- 172.21.15.105:0/179602824 learned_addr learned my addr 172.21.15.105:0/179602824 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:32:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.500+0000 7f1ed5a9a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1ed01575b0 0x7f1ed015ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58610/0 (socket says 172.21.15.105:58610) 2023-11-28T18:32:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.500+0000 7f1ed629b640 1 --2- 172.21.15.105:0/179602824 >> v2:172.21.15.148:3300/0 conn(0x7f1ed015d7b0 0x7f1ed006cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.500+0000 7f1ed5a9a640 1 -- 172.21.15.105:0/179602824 >> v2:172.21.15.148:3300/0 conn(0x7f1ed015d7b0 msgr2=0x7f1ed006cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.500+0000 7f1ed5a9a640 1 --2- 172.21.15.105:0/179602824 >> v2:172.21.15.148:3300/0 conn(0x7f1ed015d7b0 0x7f1ed006cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.500+0000 7f1ed5a9a640 1 -- 172.21.15.105:0/179602824 >> v1:172.21.15.105:6789/0 conn(0x7f1ed0157990 legacy=0x7f1ed00c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.500+0000 7f1ed5a9a640 1 -- 172.21.15.105:0/179602824 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed015e670 con 0x7f1ed01575b0 2023-11-28T18:32:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.500+0000 7f1ed629b640 1 --2- 172.21.15.105:0/179602824 >> v2:172.21.15.148:3300/0 conn(0x7f1ed015d7b0 0x7f1ed006cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:32:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.500+0000 7f1ed5a9a640 1 --2- 172.21.15.105:0/179602824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ed01575b0 0x7f1ed015ada0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00b1290 tx=0x7f1ecc01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a5771af5c9978a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.504+0000 7f1ed4a98640 1 -- 172.21.15.105:0/179602824 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc0a7070 con 0x7f1ed01575b0 2023-11-28T18:32:05.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.504+0000 7f1ed4a98640 1 -- 172.21.15.105:0/179602824 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ecc0ab430 con 0x7f1ed01575b0 2023-11-28T18:32:05.503 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.504+0000 7f1ed4a98640 1 -- 172.21.15.105:0/179602824 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc0ab7c0 con 0x7f1ed01575b0 2023-11-28T18:32:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.504+0000 7f1ed6a9c640 1 -- 172.21.15.105:0/179602824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ed01575b0 msgr2=0x7f1ed015ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.504+0000 7f1ed6a9c640 1 --2- 172.21.15.105:0/179602824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ed01575b0 0x7f1ed015ada0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00b1290 tx=0x7f1ecc01c430 comp rx=0 tx=0).stop 2023-11-28T18:32:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.504+0000 7f1ed6a9c640 1 -- 172.21.15.105:0/179602824 shutdown_connections 2023-11-28T18:32:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.504+0000 7f1ed6a9c640 1 --2- 172.21.15.105:0/179602824 >> v2:172.21.15.148:3300/0 conn(0x7f1ed015d7b0 0x7f1ed006cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.504+0000 7f1ed6a9c640 1 --2- 172.21.15.105:0/179602824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ed01575b0 0x7f1ed015ada0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.504+0000 7f1ed6a9c640 1 -- 172.21.15.105:0/179602824 >> 172.21.15.105:0/179602824 conn(0x7f1ed00b0480 msgr2=0x7f1ed00b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.504+0000 7f1ed6a9c640 1 -- 172.21.15.105:0/179602824 shutdown_connections 2023-11-28T18:32:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.504+0000 7f1ed6a9c640 1 -- 172.21.15.105:0/179602824 wait complete. 2023-11-28T18:32:05.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.504+0000 7f1ed6a9c640 1 Processor -- start 2023-11-28T18:32:05.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.504+0000 7f1ed6a9c640 1 -- start start 2023-11-28T18:32:05.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.504+0000 7f1ed6a9c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1ed01575b0 0x7f1ed00c2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.504+0000 7f1ed6a9c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ed0157990 0x7f1ed0155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.504+0000 7f1ed6a9c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ed015d7b0 0x7f1ed00c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.504+0000 7f1ed6a9c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed00c3790 con 0x7f1ed0157990 2023-11-28T18:32:05.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.504+0000 7f1ed6a9c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed01570b0 con 0x7f1ed015d7b0 2023-11-28T18:32:05.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.504+0000 7f1ed6a9c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed00c2b80 con 0x7f1ed01575b0 2023-11-28T18:32:05.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.504+0000 7f1ed5299640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ed0157990 0x7f1ed0155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.508+0000 7f1ed5299640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ed0157990 0x7f1ed0155f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58622/0 (socket says 172.21.15.105:58622) 2023-11-28T18:32:05.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.508+0000 7f1ed5299640 1 -- 172.21.15.105:0/4069932370 learned_addr learned my addr 172.21.15.105:0/4069932370 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:05.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.508+0000 7f1ed5a9a640 1 --2- 172.21.15.105:0/4069932370 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1ed01575b0 0x7f1ed00c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.508+0000 7f1ed629b640 1 --2- 172.21.15.105:0/4069932370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ed015d7b0 0x7f1ed00c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.508+0000 7f1ed5299640 1 -- 172.21.15.105:0/4069932370 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1ed01575b0 msgr2=0x7f1ed00c2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:05.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.508+0000 7f1ed5299640 1 --2- 172.21.15.105:0/4069932370 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1ed01575b0 0x7f1ed00c2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.508+0000 7f1ed5299640 1 -- 172.21.15.105:0/4069932370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ed015d7b0 msgr2=0x7f1ed00c3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:05.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.508+0000 7f1ed5299640 1 --2- 172.21.15.105:0/4069932370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ed015d7b0 0x7f1ed00c3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.508+0000 7f1ed5299640 1 -- 172.21.15.105:0/4069932370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ecc009420 con 0x7f1ed0157990 2023-11-28T18:32:05.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.508+0000 7f1ed5299640 1 --2- 172.21.15.105:0/4069932370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ed0157990 0x7f1ed0155f70 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000c530 tx=0x7f1ec000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:05.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.508+0000 7f1ec67fc640 1 -- 172.21.15.105:0/4069932370 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0005cf0 con 0x7f1ed0157990 2023-11-28T18:32:05.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.508+0000 7f1ec67fc640 1 -- 172.21.15.105:0/4069932370 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec0005e50 con 0x7f1ed0157990 2023-11-28T18:32:05.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.508+0000 7f1ed6a9c640 1 -- 172.21.15.105:0/4069932370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed00c2e60 con 0x7f1ed0157990 2023-11-28T18:32:05.509 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.508+0000 7f1ec67fc640 1 -- 172.21.15.105:0/4069932370 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0007270 con 0x7f1ed0157990 2023-11-28T18:32:05.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.508+0000 7f1ed6a9c640 1 -- 172.21.15.105:0/4069932370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed00c3420 con 0x7f1ed0157990 2023-11-28T18:32:05.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.508+0000 7f1ed6a9c640 1 -- 172.21.15.105:0/4069932370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed0074580 con 0x7f1ed0157990 2023-11-28T18:32:05.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.508+0000 7f1ec67fc640 1 -- 172.21.15.105:0/4069932370 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f1ec00073d0 con 0x7f1ed0157990 2023-11-28T18:32:05.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.508+0000 7f1ec67fc640 1 --2- 172.21.15.105:0/4069932370 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1eb40668d0 0x7f1eb4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:05.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.508+0000 7f1ec67fc640 1 -- 172.21.15.105:0/4069932370 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1ec0083990 con 0x7f1ed0157990 2023-11-28T18:32:05.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.512+0000 7f1ed5a9a640 1 --2- 172.21.15.105:0/4069932370 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1eb40668d0 0x7f1eb4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:05.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.512+0000 7f1ed5a9a640 1 --2- 172.21.15.105:0/4069932370 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1eb40668d0 0x7f1eb4068d90 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc0af000 tx=0x7f1ecc0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:05.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.516+0000 7f1ec67fc640 1 -- 172.21.15.105:0/4069932370 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1ec0050b30 con 0x7f1ed0157990 2023-11-28T18:32:05.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.696+0000 7f1ed6a9c640 1 -- 172.21.15.105:0/4069932370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1ed015cf40 con 0x7f1ed0157990 2023-11-28T18:32:05.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.696+0000 7f1ec67fc640 1 -- 172.21.15.105:0/4069932370 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v134) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1ec0017020 con 0x7f1ed0157990 2023-11-28T18:32:05.697 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T18:32:05.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.700+0000 7f1ed6a9c640 1 -- 172.21.15.105:0/4069932370 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1eb40668d0 msgr2=0x7f1eb4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:05.699 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.700+0000 7f1ed6a9c640 1 --2- 172.21.15.105:0/4069932370 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1eb40668d0 0x7f1eb4068d90 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc0af000 tx=0x7f1ecc0a7000 comp rx=0 tx=0).stop 2023-11-28T18:32:05.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.700+0000 7f1ed6a9c640 1 -- 172.21.15.105:0/4069932370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ed0157990 msgr2=0x7f1ed0155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:05.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.700+0000 7f1ed6a9c640 1 --2- 172.21.15.105:0/4069932370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ed0157990 0x7f1ed0155f70 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000c530 tx=0x7f1ec000c9f0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.700+0000 7f1ed6a9c640 1 -- 172.21.15.105:0/4069932370 shutdown_connections 2023-11-28T18:32:05.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.700+0000 7f1ed6a9c640 1 --2- 172.21.15.105:0/4069932370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ed015d7b0 0x7f1ed00c3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.700+0000 7f1ed6a9c640 1 --2- 172.21.15.105:0/4069932370 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f1eb40668d0 0x7f1eb4068d90 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.700+0000 7f1ed6a9c640 1 --2- 172.21.15.105:0/4069932370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ed0157990 0x7f1ed0155f70 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.700+0000 7f1ed6a9c640 1 --2- 172.21.15.105:0/4069932370 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1ed01575b0 0x7f1ed00c2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:05.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.700+0000 7f1ed6a9c640 1 -- 172.21.15.105:0/4069932370 >> 172.21.15.105:0/4069932370 conn(0x7f1ed00b0480 msgr2=0x7f1ed00ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:05.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.700+0000 7f1ed6a9c640 1 -- 172.21.15.105:0/4069932370 shutdown_connections 2023-11-28T18:32:05.700 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:05.700+0000 7f1ed6a9c640 1 -- 172.21.15.105:0/4069932370 wait complete. 2023-11-28T18:32:05.717 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T18:32:05.717 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T18:32:05.718 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2023-11-28T18:32:05.718 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T18:32:05.718 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T18:32:05.718 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T18:32:05.718 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T18:32:05.718 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T18:32:05.718 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:32:05.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:32:05.719 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2023-11-28T18:32:05.728 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T18:32:05.728 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T18:32:05.729 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T18:32:05.729 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:32:05.729 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-11-28T18:32:05.729 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:32:05.729 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:32:05.729 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:32:05.729 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2023-11-28T18:32:05.729 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:32:05.729 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:32:05.729 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2023-11-28T18:32:05.729 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T18:32:05.730 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:32:05.730 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T18:32:05.730 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:32:05.730 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-28T18:32:05.730 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 41896sec preferred_lft 41896sec 2023-11-28T18:32:05.730 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2023-11-28T18:32:05.730 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:32:05.730 INFO:teuthology.orchestra.run.smithi105.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-28T18:32:05.730 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 02:42:cc:d4:87:d2 brd ff:ff:ff:ff:ff:ff 2023-11-28T18:32:05.730 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-28T18:32:05.730 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:32:05.731 INFO:teuthology.orchestra.run.smithi105.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T18:32:05.731 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-28T18:32:05.731 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T18:32:05.731 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:32:05.731 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-28T18:32:05.731 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:32:05.731 INFO:teuthology.orchestra.run.smithi105.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T18:32:05.731 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:32:05.731 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-28T18:32:05.731 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:32:05.731 INFO:teuthology.orchestra.run.smithi105.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T18:32:05.731 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:32:05.732 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-28T18:32:05.732 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-11-28T18:32:05.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:32:05.732 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T18:32:05.793 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-11-28T18:32:05.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:32:05.795 DEBUG:teuthology.orchestra.run.smithi105:> route 2023-11-28T18:32:05.852 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2023-11-28T18:32:05.853 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T18:32:05.853 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-28T18:32:05.853 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T18:32:05.853 INFO:teuthology.orchestra.run.smithi105.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-28T18:32:05.853 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T18:32:05.853 INFO:teuthology.orchestra.run.smithi105.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:32:05.853 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:32:05.854 INFO:teuthology.orchestra.run.smithi105.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-28T18:32:05.854 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T18:32:05.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:32:05.855 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-11-28T18:32:05.855 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:32:05.855 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T18:32:05.855 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T18:32:05.855 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T18:32:05.856 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:32:05.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:32:05.949 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-11-28T18:32:05.958 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T18:32:05.958 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T18:32:05.959 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:32:05.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:32:05.959 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2023-11-28T18:32:06.026 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-28T18:32:06.027 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:32:06.027 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T18:32:06.082 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T18:32:06.083 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T18:32:06.083 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T18:32:06.138 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:32:06.208 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:32:06.264 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:32:06.264 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:32:06.265 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:32:06.265 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:32:06.265 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:32:06.265 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:32:06.265 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:32:06.265 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:32:06.265 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:32:06.266 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:32:06.266 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:32:06.266 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:32:06.266 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:32:06.266 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:32:06.266 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:32:06.267 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:32:06.267 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:32:06.267 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:32:06.267 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:32:06.268 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:32:06.268 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:32:06.268 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:32:06.268 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:32:06.268 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:32:06.268 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:32:06.268 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:32:06.269 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:32:06.269 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:32:06.269 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:32:06.269 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:32:06.269 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:32:06.269 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:32:06.269 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:32:06.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:32:06.270 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:32:06.327 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T18:32:06.327 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-28T18:32:06.373 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-11-28T18:32:06.396 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T18:32:06.450 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2023-11-28T18:32:06.450 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2023-11-28T18:32:06.450 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2023-11-28T18:32:06.450 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2023-11-28T18:32:06.451 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2023-11-28T18:32:06.451 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2023-11-28T18:32:06.451 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2023-11-28T18:32:06.451 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2023-11-28T18:32:06.452 INFO:teuthology.orchestra.run.smithi105.stdout:/run/lock 2023-11-28T18:32:06.452 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2023-11-28T18:32:06.452 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2023-11-28T18:32:06.452 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2023-11-28T18:32:06.453 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T18:32:06.453 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2023-11-28T18:32:06.453 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2023-11-28T18:32:06.453 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2023-11-28T18:32:06.453 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2023-11-28T18:32:06.454 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/fs/nfsd 2023-11-28T18:32:06.454 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2023-11-28T18:32:06.454 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2023-11-28T18:32:06.454 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T18:32:06.455 INFO:teuthology.orchestra.run.smithi105.stdout:/run/rpc_pipefs 2023-11-28T18:32:06.455 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2023-11-28T18:32:06.455 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2023-11-28T18:32:06.455 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T18:32:06.455 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T18:32:06.455 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T18:32:06.456 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T18:32:06.456 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2023-11-28T18:32:06.456 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:32:06.456 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:32:06.456 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:32:06.456 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:32:06.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:32:06.457 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-11-28T18:32:06.471 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:2023-11-28T18:32:06.472+0000 7f930351e580 -1 init, newargv = 0x558cda4d1fd0 newargc=15 2023-11-28T18:32:06.472 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[67801]: starting ceph client 2023-11-28T18:32:06.478 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[67801]: starting fuse 2023-11-28T18:32:06.481 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-11-28T18:32:06.481 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-28T18:32:06.481 DEBUG:teuthology.orchestra.run.smithi105:> sudo stdin-killer -- python3 -c ' 2023-11-28T18:32:06.481 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2023-11-28T18:32:06.482 DEBUG:teuthology.orchestra.run.smithi105:> import re 2023-11-28T18:32:06.482 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-11-28T18:32:06.482 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2023-11-28T18:32:06.482 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:32:06.482 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2023-11-28T18:32:06.482 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-28T18:32:06.482 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2023-11-28T18:32:06.482 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-28T18:32:06.482 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:32:06.482 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2023-11-28T18:32:06.482 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2023-11-28T18:32:06.483 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2023-11-28T18:32:06.483 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2023-11-28T18:32:06.483 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:32:06.483 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2023-11-28T18:32:06.483 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T18:32:06.483 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T18:32:06.483 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T18:32:06.483 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2023-11-28T18:32:06.483 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2023-11-28T18:32:06.483 DEBUG:teuthology.orchestra.run.smithi105:> return f 2023-11-28T18:32:06.483 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T18:32:06.483 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:32:06.484 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.0")) 2023-11-28T18:32:06.484 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-11-28T18:32:06.603 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.0.67801.asok 2023-11-28T18:32:06.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-28T18:32:06.614 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.67801.asok 2023-11-28T18:32:06.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:32:06.614 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.67801.asok status 2023-11-28T18:32:06.786 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:32:06.786 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2023-11-28T18:32:06.786 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "47c1c547372ed5bff7984647dc200bac0e019caf", 2023-11-28T18:32:06.786 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev)", 2023-11-28T18:32:06.786 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "0", 2023-11-28T18:32:06.786 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2023-11-28T18:32:06.786 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-28T18:32:06.786 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "67801", 2023-11-28T18:32:06.786 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2023-11-28T18:32:06.787 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:32:06.787 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2023-11-28T18:32:06.787 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2023-11-28T18:32:06.787 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 9279, 2023-11-28T18:32:06.787 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2023-11-28T18:32:06.787 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2023-11-28T18:32:06.787 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2023-11-28T18:32:06.787 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 9279 2023-11-28T18:32:06.787 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:32:06.787 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-11-28T18:32:06.787 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-11-28T18:32:06.787 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-11-28T18:32:06.788 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 1819386720 2023-11-28T18:32:06.788 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-11-28T18:32:06.789 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:32:06.789 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-11-28T18:32:06.789 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-11-28T18:32:06.789 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-11-28T18:32:06.789 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 1819386720 2023-11-28T18:32:06.789 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-11-28T18:32:06.789 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.9279 192.168.0.1:0/1819386720", 2023-11-28T18:32:06.790 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/1819386720", 2023-11-28T18:32:06.790 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2023-11-28T18:32:06.790 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 134, 2023-11-28T18:32:06.790 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 129, 2023-11-28T18:32:06.790 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2023-11-28T18:32:06.790 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2023-11-28T18:32:06.790 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2023-11-28T18:32:06.790 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:32:06.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:32:06.802 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2023-11-28T18:32:06.803 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-28T18:32:06.814 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2023-11-28T18:32:06.814 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-28T18:32:06.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:32:06.815 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T18:32:06.880 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random True 2023-11-28T18:32:07.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.064+0000 7fe18e85f640 1 Processor -- start 2023-11-28T18:32:07.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.064+0000 7fe18e85f640 1 -- start start 2023-11-28T18:32:07.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.068+0000 7fe18e85f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1880b4290 0x7fe1880b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:07.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.068+0000 7fe18e85f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe188071960 0x7fe18815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:07.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.068+0000 7fe18e85f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1880b4bb0 con 0x7fe18815b030 2023-11-28T18:32:07.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.068+0000 7fe18e85f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1880701d0 con 0x7fe188071960 2023-11-28T18:32:07.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.068+0000 7fe18e85f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18806df80 con 0x7fe1880b4290 2023-11-28T18:32:07.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.068+0000 7fe1877fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe188071960 0x7fe18815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:07.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.068+0000 7fe1877fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe188071960 0x7fe18815f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58634/0 (socket says 172.21.15.105:58634) 2023-11-28T18:32:07.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.068+0000 7fe1877fe640 1 -- 172.21.15.105:0/1417937018 learned_addr learned my addr 172.21.15.105:0/1417937018 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:07.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.068+0000 7fe187fff640 1 --2- 172.21.15.105:0/1417937018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1880b4290 0x7fe1880b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:07.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.068+0000 7fe1877fe640 1 -- 172.21.15.105:0/1417937018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1880b4290 msgr2=0x7fe1880b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:07.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.068+0000 7fe1877fe640 1 --2- 172.21.15.105:0/1417937018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1880b4290 0x7fe1880b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.068+0000 7fe1877fe640 1 -- 172.21.15.105:0/1417937018 >> v1:172.21.15.105:6789/0 conn(0x7fe18815b030 legacy=0x7fe1880709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:07.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.068+0000 7fe1877fe640 1 -- 172.21.15.105:0/1417937018 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe188071df0 con 0x7fe188071960 2023-11-28T18:32:07.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.072+0000 7fe1877fe640 1 --2- 172.21.15.105:0/1417937018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe188071960 0x7fe18815f3b0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009870 tx=0x7fe17c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d2aec16c57d4f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:07.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.072+0000 7fe186ffd640 1 -- 172.21.15.105:0/1417937018 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c0aa670 con 0x7fe188071960 2023-11-28T18:32:07.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.072+0000 7fe186ffd640 1 -- 172.21.15.105:0/1417937018 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17c0aa7d0 con 0x7fe188071960 2023-11-28T18:32:07.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.072+0000 7fe186ffd640 1 -- 172.21.15.105:0/1417937018 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c0aab60 con 0x7fe188071960 2023-11-28T18:32:07.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.072+0000 7fe18e85f640 1 -- 172.21.15.105:0/1417937018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe188071960 msgr2=0x7fe18815f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:07.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.072+0000 7fe18e85f640 1 --2- 172.21.15.105:0/1417937018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe188071960 0x7fe18815f3b0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009870 tx=0x7fe17c01c510 comp rx=0 tx=0).stop 2023-11-28T18:32:07.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.072+0000 7fe18e85f640 1 -- 172.21.15.105:0/1417937018 shutdown_connections 2023-11-28T18:32:07.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.072+0000 7fe18e85f640 1 --2- 172.21.15.105:0/1417937018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe188071960 0x7fe18815f3b0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.072+0000 7fe18e85f640 1 --2- 172.21.15.105:0/1417937018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1880b4290 0x7fe1880b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.074 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.072+0000 7fe18e85f640 1 -- 172.21.15.105:0/1417937018 >> 172.21.15.105:0/1417937018 conn(0x7fe1880ceb90 msgr2=0x7fe1880cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:07.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.072+0000 7fe18e85f640 1 -- 172.21.15.105:0/1417937018 shutdown_connections 2023-11-28T18:32:07.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.072+0000 7fe18e85f640 1 -- 172.21.15.105:0/1417937018 wait complete. 2023-11-28T18:32:07.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.072+0000 7fe18e85f640 1 Processor -- start 2023-11-28T18:32:07.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.072+0000 7fe18e85f640 1 -- start start 2023-11-28T18:32:07.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe18e85f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1880b4290 0x7fe1880b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:07.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe18e85f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe18815b030 0x7fe18814cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:07.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe18e85f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe18814d170 0x7fe1880af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:07.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe18e85f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1880b40f0 con 0x7fe18814d170 2023-11-28T18:32:07.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe18e85f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1880b03d0 con 0x7fe18815b030 2023-11-28T18:32:07.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe18e85f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1880b3bc0 con 0x7fe1880b4290 2023-11-28T18:32:07.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe18cdfc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe18814d170 0x7fe1880af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:07.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe18cdfc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe18814d170 0x7fe1880af0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58636/0 (socket says 172.21.15.105:58636) 2023-11-28T18:32:07.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe18cdfc640 1 -- 172.21.15.105:0/3723808960 learned_addr learned my addr 172.21.15.105:0/3723808960 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:07.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe1877fe640 1 --2- 172.21.15.105:0/3723808960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe18815b030 0x7fe18814cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:07.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe187fff640 1 --2- 172.21.15.105:0/3723808960 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1880b4290 0x7fe1880b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:07.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe18cdfc640 1 -- 172.21.15.105:0/3723808960 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1880b4290 msgr2=0x7fe1880b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:07.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe18cdfc640 1 --2- 172.21.15.105:0/3723808960 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1880b4290 0x7fe1880b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe18cdfc640 1 -- 172.21.15.105:0/3723808960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe18815b030 msgr2=0x7fe18814cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:07.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe18cdfc640 1 --2- 172.21.15.105:0/3723808960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe18815b030 0x7fe18814cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe18cdfc640 1 -- 172.21.15.105:0/3723808960 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c0094d0 con 0x7fe18814d170 2023-11-28T18:32:07.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe18cdfc640 1 --2- 172.21.15.105:0/3723808960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe18814d170 0x7fe1880af0a0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fe18000c760 tx=0x7fe18000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:07.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe184ff9640 1 -- 172.21.15.105:0/3723808960 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe180003b80 con 0x7fe18814d170 2023-11-28T18:32:07.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe184ff9640 1 -- 172.21.15.105:0/3723808960 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe180003ce0 con 0x7fe18814d170 2023-11-28T18:32:07.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe18e85f640 1 -- 172.21.15.105:0/3723808960 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1880b2bb0 con 0x7fe18814d170 2023-11-28T18:32:07.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe184ff9640 1 -- 172.21.15.105:0/3723808960 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18000aa90 con 0x7fe18814d170 2023-11-28T18:32:07.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe18e85f640 1 -- 172.21.15.105:0/3723808960 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1880b2560 con 0x7fe18814d170 2023-11-28T18:32:07.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe18e85f640 1 -- 172.21.15.105:0/3723808960 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1880b2d40 con 0x7fe18814d170 2023-11-28T18:32:07.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.076+0000 7fe184ff9640 1 -- 172.21.15.105:0/3723808960 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fe18000abf0 con 0x7fe18814d170 2023-11-28T18:32:07.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.080+0000 7fe184ff9640 1 --2- 172.21.15.105:0/3723808960 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe1740668d0 0x7fe174068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:07.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.080+0000 7fe184ff9640 1 -- 172.21.15.105:0/3723808960 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe1800841f0 con 0x7fe18814d170 2023-11-28T18:32:07.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.080+0000 7fe187fff640 1 --2- 172.21.15.105:0/3723808960 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe1740668d0 0x7fe174068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:07.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.080+0000 7fe187fff640 1 --2- 172.21.15.105:0/3723808960 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe1740668d0 0x7fe174068d90 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe1880caf30 tx=0x7fe178008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:07.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.084+0000 7fe184ff9640 1 -- 172.21.15.105:0/3723808960 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe1800513a0 con 0x7fe18814d170 2023-11-28T18:32:07.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.232+0000 7fe18e85f640 1 -- 172.21.15.105:0/3723808960 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random}] v 0) v1 -- 0x7fe188067570 con 0x7fe18814d170 2023-11-28T18:32:07.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.236+0000 7fe184ff9640 1 -- 172.21.15.105:0/3723808960 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random}]=0 v68)=0 v68) v1 ==== 128+0+0 (secure 0 0 0) 0x7fe1800549c0 con 0x7fe18814d170 2023-11-28T18:32:07.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.240+0000 7fe18e85f640 1 -- 172.21.15.105:0/3723808960 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe1740668d0 msgr2=0x7fe174068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:07.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.240+0000 7fe18e85f640 1 --2- 172.21.15.105:0/3723808960 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe1740668d0 0x7fe174068d90 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe1880caf30 tx=0x7fe178008040 comp rx=0 tx=0).stop 2023-11-28T18:32:07.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.240+0000 7fe18e85f640 1 -- 172.21.15.105:0/3723808960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe18814d170 msgr2=0x7fe1880af0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:07.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.240+0000 7fe18e85f640 1 --2- 172.21.15.105:0/3723808960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe18814d170 0x7fe1880af0a0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fe18000c760 tx=0x7fe18000cc20 comp rx=0 tx=0).stop 2023-11-28T18:32:07.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.240+0000 7fe18e85f640 1 -- 172.21.15.105:0/3723808960 shutdown_connections 2023-11-28T18:32:07.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.240+0000 7fe18e85f640 1 --2- 172.21.15.105:0/3723808960 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fe1740668d0 0x7fe174068d90 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.240+0000 7fe18e85f640 1 --2- 172.21.15.105:0/3723808960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe18814d170 0x7fe1880af0a0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.240+0000 7fe18e85f640 1 --2- 172.21.15.105:0/3723808960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe18815b030 0x7fe18814cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.240+0000 7fe18e85f640 1 --2- 172.21.15.105:0/3723808960 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1880b4290 0x7fe1880b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.240+0000 7fe18e85f640 1 -- 172.21.15.105:0/3723808960 >> 172.21.15.105:0/3723808960 conn(0x7fe1880ceb90 msgr2=0x7fe188150490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:07.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.240+0000 7fe18e85f640 1 -- 172.21.15.105:0/3723808960 shutdown_connections 2023-11-28T18:32:07.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.240+0000 7fe18e85f640 1 -- 172.21.15.105:0/3723808960 wait complete. 2023-11-28T18:32:07.259 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_distributed True 2023-11-28T18:32:07.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.400+0000 7f2776dc3640 1 Processor -- start 2023-11-28T18:32:07.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.400+0000 7f2776dc3640 1 -- start start 2023-11-28T18:32:07.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.400+0000 7f2776dc3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f27700b4290 0x7f27700b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:07.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.400+0000 7f2776dc3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2770071960 0x7f277015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:07.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.400+0000 7f2776dc3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f277015b030 0x7f27700709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:07.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.400+0000 7f2776dc3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27700b4bb0 con 0x7f2770071960 2023-11-28T18:32:07.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.400+0000 7f2776dc3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27700701d0 con 0x7f27700b4290 2023-11-28T18:32:07.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.400+0000 7f2776dc3640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277006df80 con 0x7f277015b030 2023-11-28T18:32:07.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.400+0000 7f2767fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2770071960 0x7f277015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:07.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.400+0000 7f2767fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2770071960 0x7f277015f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58638/0 (socket says 172.21.15.105:58638) 2023-11-28T18:32:07.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.404+0000 7f2767fff640 1 -- 172.21.15.105:0/2536538835 learned_addr learned my addr 172.21.15.105:0/2536538835 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:07.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.404+0000 7f2774b5f640 1 --2- 172.21.15.105:0/2536538835 >> v2:172.21.15.148:3300/0 conn(0x7f27700b4290 0x7f27700b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:07.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.404+0000 7f2775360640 1 --2- 172.21.15.105:0/2536538835 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f277015b030 0x7f27700709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:07.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.404+0000 7f2767fff640 1 -- 172.21.15.105:0/2536538835 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f277015b030 msgr2=0x7f27700709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:07.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.404+0000 7f2767fff640 1 --2- 172.21.15.105:0/2536538835 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f277015b030 0x7f27700709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.404+0000 7f2767fff640 1 -- 172.21.15.105:0/2536538835 >> v2:172.21.15.148:3300/0 conn(0x7f27700b4290 msgr2=0x7f27700b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:07.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.404+0000 7f2767fff640 1 --2- 172.21.15.105:0/2536538835 >> v2:172.21.15.148:3300/0 conn(0x7f27700b4290 0x7f27700b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.404+0000 7f2767fff640 1 -- 172.21.15.105:0/2536538835 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2770071df0 con 0x7f2770071960 2023-11-28T18:32:07.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.404+0000 7f2767fff640 1 --2- 172.21.15.105:0/2536538835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2770071960 0x7f277015f3b0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f2770073a70 tx=0x7f276001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db4f50c2fb6253 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:07.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.404+0000 7f27677fe640 1 -- 172.21.15.105:0/2536538835 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27600ac600 con 0x7f2770071960 2023-11-28T18:32:07.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.404+0000 7f27677fe640 1 -- 172.21.15.105:0/2536538835 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27600a8050 con 0x7f2770071960 2023-11-28T18:32:07.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.404+0000 7f27677fe640 1 -- 172.21.15.105:0/2536538835 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27600ac980 con 0x7f2770071960 2023-11-28T18:32:07.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2776dc3640 1 -- 172.21.15.105:0/2536538835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2770071960 msgr2=0x7f277015f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:07.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2776dc3640 1 --2- 172.21.15.105:0/2536538835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2770071960 0x7f277015f3b0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f2770073a70 tx=0x7f276001c4d0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2776dc3640 1 -- 172.21.15.105:0/2536538835 shutdown_connections 2023-11-28T18:32:07.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2776dc3640 1 --2- 172.21.15.105:0/2536538835 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f277015b030 0x7f27700709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2776dc3640 1 --2- 172.21.15.105:0/2536538835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2770071960 0x7f277015f3b0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2776dc3640 1 --2- 172.21.15.105:0/2536538835 >> v2:172.21.15.148:3300/0 conn(0x7f27700b4290 0x7f27700b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2776dc3640 1 -- 172.21.15.105:0/2536538835 >> 172.21.15.105:0/2536538835 conn(0x7f27700ceb90 msgr2=0x7f27700cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:07.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2776dc3640 1 -- 172.21.15.105:0/2536538835 shutdown_connections 2023-11-28T18:32:07.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2776dc3640 1 -- 172.21.15.105:0/2536538835 wait complete. 2023-11-28T18:32:07.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2776dc3640 1 Processor -- start 2023-11-28T18:32:07.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2776dc3640 1 -- start start 2023-11-28T18:32:07.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2776dc3640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27700b4290 0x7f27701546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:07.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2776dc3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f277015b030 0x7f27701567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:07.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2776dc3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2770156d20 0x7f2770159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:07.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2776dc3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2770099a50 con 0x7f2770156d20 2023-11-28T18:32:07.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2776dc3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277009a5d0 con 0x7f277015b030 2023-11-28T18:32:07.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2776dc3640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27700ce760 con 0x7f27700b4290 2023-11-28T18:32:07.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2775360640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2770156d20 0x7f2770159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:07.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2775360640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2770156d20 0x7f2770159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58654/0 (socket says 172.21.15.105:58654) 2023-11-28T18:32:07.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2775360640 1 -- 172.21.15.105:0/743693769 learned_addr learned my addr 172.21.15.105:0/743693769 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:07.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2774b5f640 1 --2- 172.21.15.105:0/743693769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27700b4290 0x7f27701546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:07.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2767fff640 1 --2- 172.21.15.105:0/743693769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f277015b030 0x7f27701567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:07.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2775360640 1 -- 172.21.15.105:0/743693769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27700b4290 msgr2=0x7f27701546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:07.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.408+0000 7f2775360640 1 --2- 172.21.15.105:0/743693769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27700b4290 0x7f27701546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.412+0000 7f2775360640 1 -- 172.21.15.105:0/743693769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f277015b030 msgr2=0x7f27701567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:07.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.412+0000 7f2775360640 1 --2- 172.21.15.105:0/743693769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f277015b030 0x7f27701567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.412+0000 7f2775360640 1 -- 172.21.15.105:0/743693769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27600094d0 con 0x7f2770156d20 2023-11-28T18:32:07.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.412+0000 7f2775360640 1 --2- 172.21.15.105:0/743693769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2770156d20 0x7f2770159680 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f2768007d80 tx=0x7f276800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:07.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.412+0000 7f27657fa640 1 -- 172.21.15.105:0/743693769 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768014420 con 0x7f2770156d20 2023-11-28T18:32:07.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.412+0000 7f27657fa640 1 -- 172.21.15.105:0/743693769 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2768014580 con 0x7f2770156d20 2023-11-28T18:32:07.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.412+0000 7f2776dc3640 1 -- 172.21.15.105:0/743693769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f277015b470 con 0x7f2770156d20 2023-11-28T18:32:07.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.412+0000 7f27657fa640 1 -- 172.21.15.105:0/743693769 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768014930 con 0x7f2770156d20 2023-11-28T18:32:07.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.412+0000 7f2776dc3640 1 -- 172.21.15.105:0/743693769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2770099f80 con 0x7f2770156d20 2023-11-28T18:32:07.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.412+0000 7f2776dc3640 1 -- 172.21.15.105:0/743693769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f277009a760 con 0x7f2770156d20 2023-11-28T18:32:07.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.412+0000 7f27657fa640 1 -- 172.21.15.105:0/743693769 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f2768014a90 con 0x7f2770156d20 2023-11-28T18:32:07.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.412+0000 7f27657fa640 1 --2- 172.21.15.105:0/743693769 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f275c0668d0 0x7f275c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:07.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.412+0000 7f27657fa640 1 -- 172.21.15.105:0/743693769 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2768084c10 con 0x7f2770156d20 2023-11-28T18:32:07.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.412+0000 7f2774b5f640 1 --2- 172.21.15.105:0/743693769 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f275c0668d0 0x7f275c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:07.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.416+0000 7f2774b5f640 1 --2- 172.21.15.105:0/743693769 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f275c0668d0 0x7f275c068d90 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f277015e150 tx=0x7f276c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:07.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.416+0000 7f27657fa640 1 -- 172.21.15.105:0/743693769 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2768051e30 con 0x7f2770156d20 2023-11-28T18:32:07.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.572+0000 7f2776dc3640 1 -- 172.21.15.105:0/743693769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_distributed}] v 0) v1 -- 0x7f2770067570 con 0x7f2770156d20 2023-11-28T18:32:07.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.576+0000 7f27657fa640 1 -- 172.21.15.105:0/743693769 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_distributed}]=0 v69)=0 v69) v1 ==== 133+0+0 (secure 0 0 0) 0x7f2768055450 con 0x7f2770156d20 2023-11-28T18:32:07.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.580+0000 7f2776dc3640 1 -- 172.21.15.105:0/743693769 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f275c0668d0 msgr2=0x7f275c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:07.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.580+0000 7f2776dc3640 1 --2- 172.21.15.105:0/743693769 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f275c0668d0 0x7f275c068d90 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f277015e150 tx=0x7f276c008040 comp rx=0 tx=0).stop 2023-11-28T18:32:07.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.580+0000 7f2776dc3640 1 -- 172.21.15.105:0/743693769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2770156d20 msgr2=0x7f2770159680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:07.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.580+0000 7f2776dc3640 1 --2- 172.21.15.105:0/743693769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2770156d20 0x7f2770159680 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f2768007d80 tx=0x7f276800e4b0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.580+0000 7f2776dc3640 1 -- 172.21.15.105:0/743693769 shutdown_connections 2023-11-28T18:32:07.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.580+0000 7f2776dc3640 1 --2- 172.21.15.105:0/743693769 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f275c0668d0 0x7f275c068d90 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.580+0000 7f2776dc3640 1 --2- 172.21.15.105:0/743693769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2770156d20 0x7f2770159680 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.580+0000 7f2776dc3640 1 --2- 172.21.15.105:0/743693769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f277015b030 0x7f27701567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.580+0000 7f2776dc3640 1 --2- 172.21.15.105:0/743693769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27700b4290 0x7f27701546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.580+0000 7f2776dc3640 1 -- 172.21.15.105:0/743693769 >> 172.21.15.105:0/743693769 conn(0x7f27700ceb90 msgr2=0x7f27700ada30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:07.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.580+0000 7f2776dc3640 1 -- 172.21.15.105:0/743693769 shutdown_connections 2023-11-28T18:32:07.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.580+0000 7f2776dc3640 1 -- 172.21.15.105:0/743693769 wait complete. 2023-11-28T18:32:07.599 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random_max 1.0 2023-11-28T18:32:07.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.740+0000 7f02f45d1640 1 Processor -- start 2023-11-28T18:32:07.739 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.740+0000 7f02f45d1640 1 -- start start 2023-11-28T18:32:07.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.740+0000 7f02f45d1640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02ec154070 0x7f02ec15d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:07.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.740+0000 7f02f45d1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f02ec06bf50 0x7f02ec0ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:07.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.740+0000 7f02f45d1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f02ec0adb80 0x7f02ec0adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:07.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.740+0000 7f02f45d1640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02ec072540 con 0x7f02ec0adb80 2023-11-28T18:32:07.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.740+0000 7f02f45d1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02ec0ae510 con 0x7f02ec06bf50 2023-11-28T18:32:07.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.740+0000 7f02f45d1640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec0cc7f0 con 0x7f02ec154070 2023-11-28T18:32:07.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.740+0000 7f02f2b6e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f02ec0adb80 0x7f02ec0adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:07.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.740+0000 7f02f2b6e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f02ec0adb80 0x7f02ec0adfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58656/0 (socket says 172.21.15.105:58656) 2023-11-28T18:32:07.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.740+0000 7f02f2b6e640 1 -- 172.21.15.105:0/507732184 learned_addr learned my addr 172.21.15.105:0/507732184 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:07.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.740+0000 7f02f236d640 1 --2- 172.21.15.105:0/507732184 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02ec154070 0x7f02ec15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:07.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.740+0000 7f02f1b6c640 1 --2- 172.21.15.105:0/507732184 >> v2:172.21.15.148:3300/0 conn(0x7f02ec06bf50 0x7f02ec0ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:07.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.740+0000 7f02f2b6e640 1 -- 172.21.15.105:0/507732184 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02ec154070 msgr2=0x7f02ec15d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:07.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.740+0000 7f02f2b6e640 1 --2- 172.21.15.105:0/507732184 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02ec154070 0x7f02ec15d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.740+0000 7f02f2b6e640 1 -- 172.21.15.105:0/507732184 >> v2:172.21.15.148:3300/0 conn(0x7f02ec06bf50 msgr2=0x7f02ec0ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:07.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.740+0000 7f02f2b6e640 1 --2- 172.21.15.105:0/507732184 >> v2:172.21.15.148:3300/0 conn(0x7f02ec06bf50 0x7f02ec0ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.740+0000 7f02f2b6e640 1 -- 172.21.15.105:0/507732184 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ec15dd10 con 0x7f02ec0adb80 2023-11-28T18:32:07.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.744+0000 7f02f2b6e640 1 --2- 172.21.15.105:0/507732184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02ec0adb80 0x7f02ec0adfd0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f02ec0cc710 tx=0x7f02e401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0240f321e366071 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:07.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.744+0000 7f02f136b640 1 -- 172.21.15.105:0/507732184 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e40a6460 con 0x7f02ec0adb80 2023-11-28T18:32:07.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.744+0000 7f02f136b640 1 -- 172.21.15.105:0/507732184 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e40a65c0 con 0x7f02ec0adb80 2023-11-28T18:32:07.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.744+0000 7f02f136b640 1 -- 172.21.15.105:0/507732184 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e40a6950 con 0x7f02ec0adb80 2023-11-28T18:32:07.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.744+0000 7f02f45d1640 1 -- 172.21.15.105:0/507732184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02ec0adb80 msgr2=0x7f02ec0adfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:07.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.744+0000 7f02f45d1640 1 --2- 172.21.15.105:0/507732184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02ec0adb80 0x7f02ec0adfd0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f02ec0cc710 tx=0x7f02e401c9a0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.744+0000 7f02f45d1640 1 -- 172.21.15.105:0/507732184 shutdown_connections 2023-11-28T18:32:07.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.744+0000 7f02f45d1640 1 --2- 172.21.15.105:0/507732184 >> v2:172.21.15.148:3300/0 conn(0x7f02ec06bf50 0x7f02ec0ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.744+0000 7f02f45d1640 1 --2- 172.21.15.105:0/507732184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02ec0adb80 0x7f02ec0adfd0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.744+0000 7f02f45d1640 1 --2- 172.21.15.105:0/507732184 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02ec154070 0x7f02ec15d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.744+0000 7f02f45d1640 1 -- 172.21.15.105:0/507732184 >> 172.21.15.105:0/507732184 conn(0x7f02ec0ceb90 msgr2=0x7f02ec0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:07.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.744+0000 7f02f45d1640 1 -- 172.21.15.105:0/507732184 shutdown_connections 2023-11-28T18:32:07.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.744+0000 7f02f45d1640 1 -- 172.21.15.105:0/507732184 wait complete. 2023-11-28T18:32:07.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02f45d1640 1 Processor -- start 2023-11-28T18:32:07.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02f45d1640 1 -- start start 2023-11-28T18:32:07.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02f45d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02ec06bf50 0x7f02ec153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:07.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02f45d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02ec0adb80 0x7f02ec14ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:07.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02f45d1640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02ec154070 0x7f02ec1f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:07.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02f45d1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec1537a0 con 0x7f02ec06bf50 2023-11-28T18:32:07.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02f45d1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec1521c0 con 0x7f02ec0adb80 2023-11-28T18:32:07.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02f45d1640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec1524a0 con 0x7f02ec154070 2023-11-28T18:32:07.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02f236d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02ec06bf50 0x7f02ec153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:07.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02f236d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02ec06bf50 0x7f02ec153260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58664/0 (socket says 172.21.15.105:58664) 2023-11-28T18:32:07.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02f236d640 1 -- 172.21.15.105:0/4131875961 learned_addr learned my addr 172.21.15.105:0/4131875961 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:07.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02f1b6c640 1 --2- 172.21.15.105:0/4131875961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02ec0adb80 0x7f02ec14ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:07.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02f2b6e640 1 --2- 172.21.15.105:0/4131875961 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02ec154070 0x7f02ec1f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:07.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02f236d640 1 -- 172.21.15.105:0/4131875961 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02ec154070 msgr2=0x7f02ec1f5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:07.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02f236d640 1 --2- 172.21.15.105:0/4131875961 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02ec154070 0x7f02ec1f5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02f236d640 1 -- 172.21.15.105:0/4131875961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02ec0adb80 msgr2=0x7f02ec14ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:07.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02f236d640 1 --2- 172.21.15.105:0/4131875961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02ec0adb80 0x7f02ec14ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02f236d640 1 -- 172.21.15.105:0/4131875961 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e4009b30 con 0x7f02ec06bf50 2023-11-28T18:32:07.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02f236d640 1 --2- 172.21.15.105:0/4131875961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02ec06bf50 0x7f02ec153260 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f02e800d5f0 tx=0x7f02e800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:07.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02e2ffd640 1 -- 172.21.15.105:0/4131875961 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e80125f0 con 0x7f02ec06bf50 2023-11-28T18:32:07.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02e2ffd640 1 -- 172.21.15.105:0/4131875961 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e8012750 con 0x7f02ec06bf50 2023-11-28T18:32:07.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02e2ffd640 1 -- 172.21.15.105:0/4131875961 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e8012ae0 con 0x7f02ec06bf50 2023-11-28T18:32:07.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02f45d1640 1 -- 172.21.15.105:0/4131875961 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ec152780 con 0x7f02ec06bf50 2023-11-28T18:32:07.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02f45d1640 1 -- 172.21.15.105:0/4131875961 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ec152d40 con 0x7f02ec06bf50 2023-11-28T18:32:07.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.748+0000 7f02f45d1640 1 -- 172.21.15.105:0/4131875961 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ec152ed0 con 0x7f02ec06bf50 2023-11-28T18:32:07.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.752+0000 7f02e2ffd640 1 -- 172.21.15.105:0/4131875961 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f02e8012c40 con 0x7f02ec06bf50 2023-11-28T18:32:07.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.752+0000 7f02e2ffd640 1 --2- 172.21.15.105:0/4131875961 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f02d00668d0 0x7f02d0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:07.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.752+0000 7f02e2ffd640 1 -- 172.21.15.105:0/4131875961 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f02e8084950 con 0x7f02ec06bf50 2023-11-28T18:32:07.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.752+0000 7f02f1b6c640 1 --2- 172.21.15.105:0/4131875961 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f02d00668d0 0x7f02d0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:07.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.752+0000 7f02f1b6c640 1 --2- 172.21.15.105:0/4131875961 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f02d00668d0 0x7f02d0068d90 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f02ec0c8b20 tx=0x7f02dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:07.755 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.756+0000 7f02e2ffd640 1 -- 172.21.15.105:0/4131875961 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f02e8051a60 con 0x7f02ec06bf50 2023-11-28T18:32:07.907 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.908+0000 7f02f45d1640 1 -- 172.21.15.105:0/4131875961 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random_max}] v 0) v1 -- 0x7f02ec067570 con 0x7f02ec06bf50 2023-11-28T18:32:07.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.912+0000 7f02e2ffd640 1 -- 172.21.15.105:0/4131875961 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random_max}]=0 v70)=0 v70) v1 ==== 131+0+0 (secure 0 0 0) 0x7f02e8055080 con 0x7f02ec06bf50 2023-11-28T18:32:07.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.912+0000 7f02f45d1640 1 -- 172.21.15.105:0/4131875961 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f02d00668d0 msgr2=0x7f02d0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:07.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.912+0000 7f02f45d1640 1 --2- 172.21.15.105:0/4131875961 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f02d00668d0 0x7f02d0068d90 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f02ec0c8b20 tx=0x7f02dc008040 comp rx=0 tx=0).stop 2023-11-28T18:32:07.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.912+0000 7f02f45d1640 1 -- 172.21.15.105:0/4131875961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02ec06bf50 msgr2=0x7f02ec153260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:07.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.912+0000 7f02f45d1640 1 --2- 172.21.15.105:0/4131875961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02ec06bf50 0x7f02ec153260 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f02e800d5f0 tx=0x7f02e800dab0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.912+0000 7f02f45d1640 1 -- 172.21.15.105:0/4131875961 shutdown_connections 2023-11-28T18:32:07.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.912+0000 7f02f45d1640 1 --2- 172.21.15.105:0/4131875961 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02ec154070 0x7f02ec1f5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.912+0000 7f02f45d1640 1 --2- 172.21.15.105:0/4131875961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02ec0adb80 0x7f02ec14ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.912+0000 7f02f45d1640 1 --2- 172.21.15.105:0/4131875961 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f02d00668d0 0x7f02d0068d90 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.912+0000 7f02f45d1640 1 --2- 172.21.15.105:0/4131875961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02ec06bf50 0x7f02ec153260 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:07.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.912+0000 7f02f45d1640 1 -- 172.21.15.105:0/4131875961 >> 172.21.15.105:0/4131875961 conn(0x7f02ec0ceb90 msgr2=0x7f02ec06ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:07.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.912+0000 7f02f45d1640 1 -- 172.21.15.105:0/4131875961 shutdown_connections 2023-11-28T18:32:07.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:07.912+0000 7f02f45d1640 1 -- 172.21.15.105:0/4131875961 wait complete. 2023-11-28T18:32:07.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:32:07.934 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-28T18:32:07.934 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-11-28T18:32:07.934 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:32:07.935 DEBUG:teuthology.orchestra.run.smithi105:> # Use up a random number of inode numbers so the ephemeral pinning is not the same every test. 2023-11-28T18:32:07.935 DEBUG:teuthology.orchestra.run.smithi105:> mkdir .inode_number_thrash 2023-11-28T18:32:07.935 DEBUG:teuthology.orchestra.run.smithi105:> count=$((RANDOM % 1024)) 2023-11-28T18:32:07.935 DEBUG:teuthology.orchestra.run.smithi105:> for ((i = 0; i < count; i++)); do touch .inode_number_thrash/$i; done 2023-11-28T18:32:07.935 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf .inode_number_thrash 2023-11-28T18:32:07.935 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:32:15.251 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-11-28T18:32:16.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.708+0000 7fc340eea640 1 Processor -- start 2023-11-28T18:32:16.707 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.708+0000 7fc340eea640 1 -- start start 2023-11-28T18:32:16.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.708+0000 7fc340eea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc33c159f40 0x7fc33c0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:16.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.708+0000 7fc340eea640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc33c0ad850 0x7fc33c0701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:16.708 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.708+0000 7fc340eea640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc33c06bf50 con 0x7fc33c0cc760 2023-11-28T18:32:16.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.708+0000 7fc340eea640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc33c0ccbc0 con 0x7fc33c0ad850 2023-11-28T18:32:16.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.708+0000 7fc340eea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c15a320 con 0x7fc33c159f40 2023-11-28T18:32:16.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.708+0000 7fc33a59c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc33c159f40 0x7fc33c0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:16.709 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.708+0000 7fc33a59c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc33c159f40 0x7fc33c0caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:50858/0 (socket says 172.21.15.105:50858) 2023-11-28T18:32:16.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.708+0000 7fc33a59c640 1 -- 172.21.15.105:0/209371967 learned_addr learned my addr 172.21.15.105:0/209371967 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:16.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.708+0000 7fc33ad9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc33c0ad850 0x7fc33c0701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:16.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.708+0000 7fc33a59c640 1 -- 172.21.15.105:0/209371967 >> v2:172.21.15.148:3300/0 conn(0x7fc33c0ad850 msgr2=0x7fc33c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:16.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.708+0000 7fc33a59c640 1 --2- 172.21.15.105:0/209371967 >> v2:172.21.15.148:3300/0 conn(0x7fc33c0ad850 0x7fc33c0701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:16.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.708+0000 7fc33a59c640 1 -- 172.21.15.105:0/209371967 >> v1:172.21.15.148:6789/0 conn(0x7fc33c0cc760 legacy=0x7fc33c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:16.710 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.708+0000 7fc33a59c640 1 -- 172.21.15.105:0/209371967 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33c071380 con 0x7fc33c159f40 2023-11-28T18:32:16.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.712+0000 7fc33a59c640 1 --2- 172.21.15.105:0/209371967 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc33c159f40 0x7fc33c0caf60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009870 tx=0x7fc32c01c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aef9b4165b7cfe7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:16.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.712+0000 7fc33959a640 1 -- 172.21.15.105:0/209371967 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0ab680 con 0x7fc33c159f40 2023-11-28T18:32:16.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.712+0000 7fc33959a640 1 -- 172.21.15.105:0/209371967 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32c0ab7e0 con 0x7fc33c159f40 2023-11-28T18:32:16.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.712+0000 7fc33959a640 1 -- 172.21.15.105:0/209371967 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0abb70 con 0x7fc33c159f40 2023-11-28T18:32:16.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.712+0000 7fc340eea640 1 -- 172.21.15.105:0/209371967 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc33c159f40 msgr2=0x7fc33c0caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:16.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.712+0000 7fc340eea640 1 --2- 172.21.15.105:0/209371967 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc33c159f40 0x7fc33c0caf60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009870 tx=0x7fc32c01c480 comp rx=0 tx=0).stop 2023-11-28T18:32:16.715 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.712+0000 7fc340eea640 1 -- 172.21.15.105:0/209371967 shutdown_connections 2023-11-28T18:32:16.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.712+0000 7fc340eea640 1 --2- 172.21.15.105:0/209371967 >> v2:172.21.15.148:3300/0 conn(0x7fc33c0ad850 0x7fc33c0701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:16.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.712+0000 7fc340eea640 1 --2- 172.21.15.105:0/209371967 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc33c159f40 0x7fc33c0caf60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:16.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.712+0000 7fc340eea640 1 -- 172.21.15.105:0/209371967 >> 172.21.15.105:0/209371967 conn(0x7fc33c0ceb90 msgr2=0x7fc33c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:16.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.712+0000 7fc340eea640 1 -- 172.21.15.105:0/209371967 shutdown_connections 2023-11-28T18:32:16.716 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.712+0000 7fc340eea640 1 -- 172.21.15.105:0/209371967 wait complete. 2023-11-28T18:32:16.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc340eea640 1 Processor -- start 2023-11-28T18:32:16.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc340eea640 1 -- start start 2023-11-28T18:32:16.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc340eea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc33c0ad850 0x7fc33c149510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:16.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc340eea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc33c0cc760 0x7fc33c151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:16.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc340eea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc33c159f40 0x7fc33c152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:16.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc340eea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c1556f0 con 0x7fc33c0ad850 2023-11-28T18:32:16.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc340eea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c149a50 con 0x7fc33c159f40 2023-11-28T18:32:16.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc340eea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c152830 con 0x7fc33c0cc760 2023-11-28T18:32:16.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc33a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc33c0ad850 0x7fc33c149510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:16.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc33a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc33c0ad850 0x7fc33c149510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50550/0 (socket says 172.21.15.105:50550) 2023-11-28T18:32:16.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc33a59c640 1 -- 172.21.15.105:0/107397670 learned_addr learned my addr 172.21.15.105:0/107397670 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:16.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc33ad9d640 1 --2- 172.21.15.105:0/107397670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc33c159f40 0x7fc33c152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:16.718 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc339d9b640 1 --2- 172.21.15.105:0/107397670 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc33c0cc760 0x7fc33c151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:16.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc33a59c640 1 -- 172.21.15.105:0/107397670 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc33c0cc760 msgr2=0x7fc33c151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:16.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc33a59c640 1 --2- 172.21.15.105:0/107397670 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc33c0cc760 0x7fc33c151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:16.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc33a59c640 1 -- 172.21.15.105:0/107397670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc33c159f40 msgr2=0x7fc33c152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:16.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc33a59c640 1 --2- 172.21.15.105:0/107397670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc33c159f40 0x7fc33c152f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:16.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc33a59c640 1 -- 172.21.15.105:0/107397670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32c0094d0 con 0x7fc33c0ad850 2023-11-28T18:32:16.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc33a59c640 1 --2- 172.21.15.105:0/107397670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc33c0ad850 0x7fc33c149510 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fc32c01a0b0 tx=0x7fc32c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:16.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc32affd640 1 -- 172.21.15.105:0/107397670 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0b4800 con 0x7fc33c0ad850 2023-11-28T18:32:16.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc32affd640 1 -- 172.21.15.105:0/107397670 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32c0b4980 con 0x7fc33c0ad850 2023-11-28T18:32:16.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc340eea640 1 -- 172.21.15.105:0/107397670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc33c152b10 con 0x7fc33c0ad850 2023-11-28T18:32:16.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.716+0000 7fc32affd640 1 -- 172.21.15.105:0/107397670 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0b4d30 con 0x7fc33c0ad850 2023-11-28T18:32:16.721 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.720+0000 7fc340eea640 1 -- 172.21.15.105:0/107397670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc33c155380 con 0x7fc33c0ad850 2023-11-28T18:32:16.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.720+0000 7fc340eea640 1 -- 172.21.15.105:0/107397670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc33c067570 con 0x7fc33c0ad850 2023-11-28T18:32:16.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.720+0000 7fc32affd640 1 -- 172.21.15.105:0/107397670 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc32c0c7440 con 0x7fc33c0ad850 2023-11-28T18:32:16.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.720+0000 7fc32affd640 1 --2- 172.21.15.105:0/107397670 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc31c0668d0 0x7fc31c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:16.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.720+0000 7fc32affd640 1 -- 172.21.15.105:0/107397670 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc32c11f290 con 0x7fc33c0ad850 2023-11-28T18:32:16.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.720+0000 7fc339d9b640 1 --2- 172.21.15.105:0/107397670 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc31c0668d0 0x7fc31c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:16.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.720+0000 7fc339d9b640 1 --2- 172.21.15.105:0/107397670 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc31c0668d0 0x7fc31c068d90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc33c153e30 tx=0x7fc330009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:16.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.724+0000 7fc32affd640 1 -- 172.21.15.105:0/107397670 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc32c0ec440 con 0x7fc33c0ad850 2023-11-28T18:32:16.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:16.908+0000 7fc340eea640 1 -- 172.21.15.105:0/107397670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7fc33c0aef60 con 0x7fc33c0ad850 2023-11-28T18:32:17.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.260+0000 7fc32affd640 1 -- 172.21.15.105:0/107397670 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v135) v1 ==== 105+0+0 (secure 0 0 0) 0x7fc32c0efa20 con 0x7fc33c0ad850 2023-11-28T18:32:17.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.260+0000 7fc340eea640 1 -- 172.21.15.105:0/107397670 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc31c0668d0 msgr2=0x7fc31c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:17.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.260+0000 7fc340eea640 1 --2- 172.21.15.105:0/107397670 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc31c0668d0 0x7fc31c068d90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc33c153e30 tx=0x7fc330009040 comp rx=0 tx=0).stop 2023-11-28T18:32:17.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.260+0000 7fc340eea640 1 -- 172.21.15.105:0/107397670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc33c0ad850 msgr2=0x7fc33c149510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:17.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.260+0000 7fc340eea640 1 --2- 172.21.15.105:0/107397670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc33c0ad850 0x7fc33c149510 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fc32c01a0b0 tx=0x7fc32c01acc0 comp rx=0 tx=0).stop 2023-11-28T18:32:17.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.260+0000 7fc340eea640 1 -- 172.21.15.105:0/107397670 shutdown_connections 2023-11-28T18:32:17.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.260+0000 7fc340eea640 1 --2- 172.21.15.105:0/107397670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc33c159f40 0x7fc33c152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:17.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.260+0000 7fc340eea640 1 --2- 172.21.15.105:0/107397670 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc33c0cc760 0x7fc33c151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:17.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.260+0000 7fc340eea640 1 --2- 172.21.15.105:0/107397670 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc31c0668d0 0x7fc31c068d90 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:17.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.260+0000 7fc340eea640 1 --2- 172.21.15.105:0/107397670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc33c0ad850 0x7fc33c149510 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:17.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.260+0000 7fc340eea640 1 -- 172.21.15.105:0/107397670 >> 172.21.15.105:0/107397670 conn(0x7fc33c0ceb90 msgr2=0x7fc33c0af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:17.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.260+0000 7fc340eea640 1 -- 172.21.15.105:0/107397670 shutdown_connections 2023-11-28T18:32:17.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.260+0000 7fc340eea640 1 -- 172.21.15.105:0/107397670 wait complete. 2023-11-28T18:32:17.287 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:32:17.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.428+0000 7fbcb9e6b640 1 Processor -- start 2023-11-28T18:32:17.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.428+0000 7fbcb9e6b640 1 -- start start 2023-11-28T18:32:17.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.428+0000 7fbcb9e6b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcb415d4f0 0x7fbcb40d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:17.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.428+0000 7fbcb9e6b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbcb40d3c30 0x7fbcb40d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:17.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.428+0000 7fbcb9e6b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcb40b25b0 con 0x7fbcb40d1890 2023-11-28T18:32:17.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.428+0000 7fbcb9e6b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcb40b2900 con 0x7fbcb40d3c30 2023-11-28T18:32:17.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.428+0000 7fbcb9e6b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb40cea30 con 0x7fbcb415d4f0 2023-11-28T18:32:17.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.428+0000 7fbcb2ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbcb40d3c30 0x7fbcb40d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:17.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.428+0000 7fbcb2ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbcb40d3c30 0x7fbcb40d1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50562/0 (socket says 172.21.15.105:50562) 2023-11-28T18:32:17.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.428+0000 7fbcb2ffd640 1 -- 172.21.15.105:0/598536532 learned_addr learned my addr 172.21.15.105:0/598536532 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:17.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.428+0000 7fbcb37fe640 1 --2- 172.21.15.105:0/598536532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcb415d4f0 0x7fbcb40d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:17.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.428+0000 7fbcb2ffd640 1 -- 172.21.15.105:0/598536532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcb415d4f0 msgr2=0x7fbcb40d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:17.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.428+0000 7fbcb2ffd640 1 --2- 172.21.15.105:0/598536532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcb415d4f0 0x7fbcb40d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:17.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.428+0000 7fbcb2ffd640 1 -- 172.21.15.105:0/598536532 >> v1:172.21.15.148:6789/0 conn(0x7fbcb40d1890 legacy=0x7fbcb40cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:17.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.428+0000 7fbcb2ffd640 1 -- 172.21.15.105:0/598536532 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb40ced10 con 0x7fbcb40d3c30 2023-11-28T18:32:17.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.432+0000 7fbcb2ffd640 1 --2- 172.21.15.105:0/598536532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcb40d3c30 0x7fbcb40d1350 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009870 tx=0x7fbca801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7248604212698cc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:17.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.432+0000 7fbcb27fc640 1 -- 172.21.15.105:0/598536532 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca80ad640 con 0x7fbcb40d3c30 2023-11-28T18:32:17.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.432+0000 7fbcb27fc640 1 -- 172.21.15.105:0/598536532 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca80ad7a0 con 0x7fbcb40d3c30 2023-11-28T18:32:17.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.432+0000 7fbcb27fc640 1 -- 172.21.15.105:0/598536532 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca80adb30 con 0x7fbcb40d3c30 2023-11-28T18:32:17.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.432+0000 7fbcb9e6b640 1 -- 172.21.15.105:0/598536532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcb40d3c30 msgr2=0x7fbcb40d1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:17.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.432+0000 7fbcb9e6b640 1 --2- 172.21.15.105:0/598536532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcb40d3c30 0x7fbcb40d1350 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009870 tx=0x7fbca801c480 comp rx=0 tx=0).stop 2023-11-28T18:32:17.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.432+0000 7fbcb9e6b640 1 -- 172.21.15.105:0/598536532 shutdown_connections 2023-11-28T18:32:17.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.432+0000 7fbcb9e6b640 1 --2- 172.21.15.105:0/598536532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcb40d3c30 0x7fbcb40d1350 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:17.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.432+0000 7fbcb9e6b640 1 --2- 172.21.15.105:0/598536532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcb415d4f0 0x7fbcb40d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:17.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.432+0000 7fbcb9e6b640 1 -- 172.21.15.105:0/598536532 >> 172.21.15.105:0/598536532 conn(0x7fbcb40d3030 msgr2=0x7fbcb40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:17.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.432+0000 7fbcb9e6b640 1 -- 172.21.15.105:0/598536532 shutdown_connections 2023-11-28T18:32:17.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.432+0000 7fbcb9e6b640 1 -- 172.21.15.105:0/598536532 wait complete. 2023-11-28T18:32:17.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb9e6b640 1 Processor -- start 2023-11-28T18:32:17.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb9e6b640 1 -- start start 2023-11-28T18:32:17.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb9e6b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcb40d1890 0x7fbcb40c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:17.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb9e6b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcb40d3c30 0x7fbcb40ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:17.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb9e6b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcb415d4f0 0x7fbcb40ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:17.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb9e6b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb415ba40 con 0x7fbcb40d1890 2023-11-28T18:32:17.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb9e6b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb415b130 con 0x7fbcb415d4f0 2023-11-28T18:32:17.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb9e6b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb40c9ec0 con 0x7fbcb40d3c30 2023-11-28T18:32:17.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcb40d1890 0x7fbcb40c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:17.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcb40d1890 0x7fbcb40c9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50576/0 (socket says 172.21.15.105:50576) 2023-11-28T18:32:17.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb37fe640 1 -- 172.21.15.105:0/2758418332 learned_addr learned my addr 172.21.15.105:0/2758418332 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:17.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb2ffd640 1 --2- 172.21.15.105:0/2758418332 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcb40d3c30 0x7fbcb40ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:17.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb3fff640 1 --2- 172.21.15.105:0/2758418332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcb415d4f0 0x7fbcb40ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:17.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb37fe640 1 -- 172.21.15.105:0/2758418332 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcb40d3c30 msgr2=0x7fbcb40ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:17.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb37fe640 1 --2- 172.21.15.105:0/2758418332 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcb40d3c30 0x7fbcb40ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:17.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb37fe640 1 -- 172.21.15.105:0/2758418332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcb415d4f0 msgr2=0x7fbcb40ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:17.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb37fe640 1 --2- 172.21.15.105:0/2758418332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcb415d4f0 0x7fbcb40ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:17.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb37fe640 1 -- 172.21.15.105:0/2758418332 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca80094d0 con 0x7fbcb40d1890 2023-11-28T18:32:17.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb2ffd640 1 --2- 172.21.15.105:0/2758418332 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcb40d3c30 0x7fbcb40ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:32:17.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb37fe640 1 --2- 172.21.15.105:0/2758418332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcb40d1890 0x7fbcb40c9980 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fbca400d5f0 tx=0x7fbca400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:17.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbc9bfff640 1 -- 172.21.15.105:0/2758418332 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca40125f0 con 0x7fbcb40d1890 2023-11-28T18:32:17.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbc9bfff640 1 -- 172.21.15.105:0/2758418332 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca4012750 con 0x7fbcb40d1890 2023-11-28T18:32:17.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb9e6b640 1 -- 172.21.15.105:0/2758418332 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb40cb460 con 0x7fbcb40d1890 2023-11-28T18:32:17.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbc9bfff640 1 -- 172.21.15.105:0/2758418332 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca4012b00 con 0x7fbcb40d1890 2023-11-28T18:32:17.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb9e6b640 1 -- 172.21.15.105:0/2758418332 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb415b660 con 0x7fbcb40d1890 2023-11-28T18:32:17.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.436+0000 7fbcb9e6b640 1 -- 172.21.15.105:0/2758418332 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb4067570 con 0x7fbcb40d1890 2023-11-28T18:32:17.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.440+0000 7fbc9bfff640 1 -- 172.21.15.105:0/2758418332 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fbca4012c60 con 0x7fbcb40d1890 2023-11-28T18:32:17.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.440+0000 7fbc9bfff640 1 --2- 172.21.15.105:0/2758418332 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbca00668d0 0x7fbca0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:17.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.440+0000 7fbc9bfff640 1 -- 172.21.15.105:0/2758418332 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbca4084b90 con 0x7fbcb40d1890 2023-11-28T18:32:17.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.440+0000 7fbcb2ffd640 1 --2- 172.21.15.105:0/2758418332 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbca00668d0 0x7fbca0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:17.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.440+0000 7fbcb2ffd640 1 --2- 172.21.15.105:0/2758418332 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbca00668d0 0x7fbca0068d90 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fbca80af000 tx=0x7fbca8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:17.443 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.444+0000 7fbc9bfff640 1 -- 172.21.15.105:0/2758418332 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbca4051bf0 con 0x7fbcb40d1890 2023-11-28T18:32:17.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.652+0000 7fbcb9e6b640 1 -- 172.21.15.105:0/2758418332 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbcb40b0ca0 con 0x7fbcb40d1890 2023-11-28T18:32:17.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.652+0000 7fbc9bfff640 1 -- 172.21.15.105:0/2758418332 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+4949 (secure 0 0 0) 0x7fbca4055210 con 0x7fbcb40d1890 2023-11-28T18:32:17.654 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:32:17.654 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":136,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3161918450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3161918450},{"type":"v1","addr":"172.21.15.105:6835","nonce":3161918450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":129},{"gid":9120,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6834/3958106803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3958106803},{"type":"v1","addr":"172.21.15.148:6834","nonce":3958106803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":129}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:31:58.827408+0000","modified":"2023-11-28T18:32:17.265737+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1],"up":{"mds_0":9126,"mds_1":9123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9123":{"gid":9123,"name":"b","rank":1,"incarnation":136,"state":"up:creating","state_seq":1,"addr":"172.21.15.148:6835/3706038483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":3706038483},{"type":"v1","addr":"172.21.15.148:6835","nonce":3706038483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9126":{"gid":9126,"name":"c","rank":0,"incarnation":130,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/1978722810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1978722810},{"type":"v1","addr":"172.21.15.105:6837","nonce":1978722810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-28T18:32:17.655 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 136 2023-11-28T18:32:17.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.656+0000 7fbcb9e6b640 1 -- 172.21.15.105:0/2758418332 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbca00668d0 msgr2=0x7fbca0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:17.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.656+0000 7fbcb9e6b640 1 --2- 172.21.15.105:0/2758418332 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbca00668d0 0x7fbca0068d90 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fbca80af000 tx=0x7fbca8008040 comp rx=0 tx=0).stop 2023-11-28T18:32:17.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.656+0000 7fbcb9e6b640 1 -- 172.21.15.105:0/2758418332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcb40d1890 msgr2=0x7fbcb40c9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:17.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.656+0000 7fbcb9e6b640 1 --2- 172.21.15.105:0/2758418332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcb40d1890 0x7fbcb40c9980 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fbca400d5f0 tx=0x7fbca400dab0 comp rx=0 tx=0).stop 2023-11-28T18:32:17.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.656+0000 7fbcb9e6b640 1 -- 172.21.15.105:0/2758418332 shutdown_connections 2023-11-28T18:32:17.656 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.656+0000 7fbcb9e6b640 1 --2- 172.21.15.105:0/2758418332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcb415d4f0 0x7fbcb40ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:17.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.656+0000 7fbcb9e6b640 1 --2- 172.21.15.105:0/2758418332 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcb40d3c30 0x7fbcb40ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:17.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.656+0000 7fbcb9e6b640 1 --2- 172.21.15.105:0/2758418332 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbca00668d0 0x7fbca0068d90 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:17.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.656+0000 7fbcb9e6b640 1 --2- 172.21.15.105:0/2758418332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcb40d1890 0x7fbcb40c9980 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:17.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.656+0000 7fbcb9e6b640 1 -- 172.21.15.105:0/2758418332 >> 172.21.15.105:0/2758418332 conn(0x7fbcb40d3030 msgr2=0x7fbcb415c760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:17.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.656+0000 7fbcb9e6b640 1 -- 172.21.15.105:0/2758418332 shutdown_connections 2023-11-28T18:32:17.657 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:17.656+0000 7fbcb9e6b640 1 -- 172.21.15.105:0/2758418332 wait complete. 2023-11-28T18:32:18.078 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 136, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:31:58.827408+0000', 'modified': '2023-11-28T18:32:17.265737+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 9126, 'mds_1': 9123}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9123': {'gid': 9123, 'name': 'b', 'rank': 1, 'incarnation': 136, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.148:6835/3706038483', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6833', 'nonce': 3706038483}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 3706038483}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_9126': {'gid': 9126, 'name': 'c', 'rank': 0, 'incarnation': 130, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6837/1978722810', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 1978722810}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 1978722810}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:32:18.078 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9123:up:creating 2023-11-28T18:32:19.079 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:32:19.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.228+0000 7f78dd1aa640 1 Processor -- start 2023-11-28T18:32:19.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.228+0000 7f78dd1aa640 1 -- start start 2023-11-28T18:32:19.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.228+0000 7f78dd1aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78d8071a10 0x7f78d80ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:19.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.228+0000 7f78dd1aa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f78d8070a10 0x7f78d8070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:19.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.228+0000 7f78dd1aa640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d80cefa0 con 0x7f78d815d3c0 2023-11-28T18:32:19.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.228+0000 7f78dd1aa640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78d80713c0 con 0x7f78d8070a10 2023-11-28T18:32:19.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.228+0000 7f78dd1aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d815d7a0 con 0x7f78d8071a10 2023-11-28T18:32:19.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.228+0000 7f78d759e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f78d8070a10 0x7f78d8070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:19.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.228+0000 7f78d6d9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f78d815d3c0 0x7f78d8161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37532/0 (socket says 172.21.15.105:37532) 2023-11-28T18:32:19.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.228+0000 7f78d6d9d640 1 -- 172.21.15.105:0/1672502278 learned_addr learned my addr 172.21.15.105:0/1672502278 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:32:19.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.228+0000 7f78ce59c640 1 --2- 172.21.15.105:0/1672502278 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78d8071a10 0x7f78d80ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:19.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.228+0000 7f78d759e640 1 -- 172.21.15.105:0/1672502278 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78d8071a10 msgr2=0x7f78d80ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:19.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.228+0000 7f78d759e640 1 --2- 172.21.15.105:0/1672502278 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78d8071a10 0x7f78d80ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:19.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.228+0000 7f78d759e640 1 -- 172.21.15.105:0/1672502278 >> v1:172.21.15.105:6789/0 conn(0x7f78d815d3c0 legacy=0x7f78d8161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:19.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.228+0000 7f78d759e640 1 -- 172.21.15.105:0/1672502278 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d80ce9e0 con 0x7f78d8070a10 2023-11-28T18:32:19.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.228+0000 7f78d659c640 1 -- 172.21.15.105:0/1672502278 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2606209328 0 0) 0x7f78d80cefa0 con 0x7f78d815d3c0 2023-11-28T18:32:19.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.228+0000 7f78ce59c640 1 --2- 172.21.15.105:0/1672502278 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78d8071a10 0x7f78d80ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:32:19.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.232+0000 7f78d759e640 1 --2- 172.21.15.105:0/1672502278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78d8070a10 0x7f78d8070e80 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f78d00097c0 tx=0x7f78d001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56f803ce0ddc477 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:19.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.232+0000 7f78d659c640 1 -- 172.21.15.105:0/1672502278 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d00ab600 con 0x7f78d8070a10 2023-11-28T18:32:19.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.232+0000 7f78d659c640 1 -- 172.21.15.105:0/1672502278 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d00a7050 con 0x7f78d8070a10 2023-11-28T18:32:19.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.232+0000 7f78d659c640 1 -- 172.21.15.105:0/1672502278 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d00ab980 con 0x7f78d8070a10 2023-11-28T18:32:19.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.232+0000 7f78dd1aa640 1 -- 172.21.15.105:0/1672502278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78d8070a10 msgr2=0x7f78d8070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:19.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.232+0000 7f78dd1aa640 1 --2- 172.21.15.105:0/1672502278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78d8070a10 0x7f78d8070e80 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f78d00097c0 tx=0x7f78d001c3c0 comp rx=0 tx=0).stop 2023-11-28T18:32:19.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.232+0000 7f78dd1aa640 1 -- 172.21.15.105:0/1672502278 shutdown_connections 2023-11-28T18:32:19.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.232+0000 7f78dd1aa640 1 --2- 172.21.15.105:0/1672502278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78d8070a10 0x7f78d8070e80 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:19.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.232+0000 7f78dd1aa640 1 --2- 172.21.15.105:0/1672502278 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78d8071a10 0x7f78d80ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:19.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.232+0000 7f78dd1aa640 1 -- 172.21.15.105:0/1672502278 >> 172.21.15.105:0/1672502278 conn(0x7f78d80cf540 msgr2=0x7f78d80cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:19.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.232+0000 7f78dd1aa640 1 -- 172.21.15.105:0/1672502278 shutdown_connections 2023-11-28T18:32:19.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.232+0000 7f78dd1aa640 1 -- 172.21.15.105:0/1672502278 wait complete. 2023-11-28T18:32:19.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78dd1aa640 1 Processor -- start 2023-11-28T18:32:19.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78dd1aa640 1 -- start start 2023-11-28T18:32:19.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78dd1aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78d8070a10 0x7f78d815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:19.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78dd1aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78d8071a10 0x7f78d8155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:19.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78dd1aa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78d815d3c0 0x7f78d8142e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:19.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78dd1aa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d8156260 con 0x7f78d8071a10 2023-11-28T18:32:19.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78dd1aa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d815f600 con 0x7f78d8070a10 2023-11-28T18:32:19.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78dd1aa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d815fa90 con 0x7f78d815d3c0 2023-11-28T18:32:19.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78ce59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78d8071a10 0x7f78d8155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:19.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78ce59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78d8071a10 0x7f78d8155d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50608/0 (socket says 172.21.15.105:50608) 2023-11-28T18:32:19.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78ce59c640 1 -- 172.21.15.105:0/1851325094 learned_addr learned my addr 172.21.15.105:0/1851325094 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:19.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78ce59c640 1 -- 172.21.15.105:0/1851325094 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78d815d3c0 msgr2=0x7f78d8142e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:19.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78ce59c640 1 --2- 172.21.15.105:0/1851325094 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78d815d3c0 0x7f78d8142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:19.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78d6d9d640 1 --2- 172.21.15.105:0/1851325094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78d8070a10 0x7f78d815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:19.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78ce59c640 1 -- 172.21.15.105:0/1851325094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78d8070a10 msgr2=0x7f78d815e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:19.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78ce59c640 1 --2- 172.21.15.105:0/1851325094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78d8070a10 0x7f78d815e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:19.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78ce59c640 1 -- 172.21.15.105:0/1851325094 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d0009420 con 0x7f78d8071a10 2023-11-28T18:32:19.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78d6d9d640 1 --2- 172.21.15.105:0/1851325094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78d8070a10 0x7f78d815e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:32:19.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78ce59c640 1 --2- 172.21.15.105:0/1851325094 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78d8071a10 0x7f78d8155d20 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f78c0009fc0 tx=0x7f78c000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:19.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78cffff640 1 -- 172.21.15.105:0/1851325094 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c0003ba0 con 0x7f78d8071a10 2023-11-28T18:32:19.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78cffff640 1 -- 172.21.15.105:0/1851325094 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c000f040 con 0x7f78d8071a10 2023-11-28T18:32:19.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78cffff640 1 -- 172.21.15.105:0/1851325094 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c0004030 con 0x7f78d8071a10 2023-11-28T18:32:19.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78dd1aa640 1 -- 172.21.15.105:0/1851325094 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d815e9f0 con 0x7f78d8071a10 2023-11-28T18:32:19.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.236+0000 7f78dd1aa640 1 -- 172.21.15.105:0/1851325094 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d815efb0 con 0x7f78d8071a10 2023-11-28T18:32:19.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.240+0000 7f78dd1aa640 1 -- 172.21.15.105:0/1851325094 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d8067570 con 0x7f78d8071a10 2023-11-28T18:32:19.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.240+0000 7f78cffff640 1 -- 172.21.15.105:0/1851325094 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f78c0004190 con 0x7f78d8071a10 2023-11-28T18:32:19.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.240+0000 7f78cffff640 1 --2- 172.21.15.105:0/1851325094 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78b8066900 0x7f78b8068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:19.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.240+0000 7f78cffff640 1 -- 172.21.15.105:0/1851325094 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f78c0084990 con 0x7f78d8071a10 2023-11-28T18:32:19.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.240+0000 7f78d6d9d640 1 --2- 172.21.15.105:0/1851325094 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78b8066900 0x7f78b8068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:19.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.240+0000 7f78d6d9d640 1 --2- 172.21.15.105:0/1851325094 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78b8066900 0x7f78b8068dc0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f78c8007fd0 tx=0x7f78c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:19.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.244+0000 7f78cffff640 1 -- 172.21.15.105:0/1851325094 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f78c0051a10 con 0x7f78d8071a10 2023-11-28T18:32:19.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.428+0000 7f78dd1aa640 1 -- 172.21.15.105:0/1851325094 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78d8066960 con 0x7f78d8071a10 2023-11-28T18:32:19.429 INFO:tasks.ceph.mds.c.smithi105.stderr:2023-11-28T18:32:19.428+0000 7ff0bb051640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-28T18:32:19.429 INFO:tasks.ceph.mds.c.smithi105.stderr:2023-11-28T18:32:19.428+0000 7ff0bb051640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-11-28T18:32:19.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.428+0000 7f78cffff640 1 -- 172.21.15.105:0/1851325094 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+4961 (secure 0 0 0) 0x7f78c001a070 con 0x7f78d8071a10 2023-11-28T18:32:19.430 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:32:19.430 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":139,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3161918450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3161918450},{"type":"v1","addr":"172.21.15.105:6835","nonce":3161918450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":129}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:31:58.827408+0000","modified":"2023-11-28T18:32:18.299518+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9126,"mds_1":9123,"mds_2":9120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9120":{"gid":9120,"name":"d","rank":2,"incarnation":138,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6834/3958106803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3958106803},{"type":"v1","addr":"172.21.15.148:6834","nonce":3958106803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9123":{"gid":9123,"name":"b","rank":1,"incarnation":136,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6835/3706038483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":3706038483},{"type":"v1","addr":"172.21.15.148:6835","nonce":3706038483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9126":{"gid":9126,"name":"c","rank":0,"incarnation":130,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/1978722810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1978722810},{"type":"v1","addr":"172.21.15.105:6837","nonce":1978722810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-28T18:32:19.430 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 139 2023-11-28T18:32:19.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.432+0000 7f78dd1aa640 1 -- 172.21.15.105:0/1851325094 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78b8066900 msgr2=0x7f78b8068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:19.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.432+0000 7f78dd1aa640 1 --2- 172.21.15.105:0/1851325094 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78b8066900 0x7f78b8068dc0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f78c8007fd0 tx=0x7f78c8009040 comp rx=0 tx=0).stop 2023-11-28T18:32:19.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.432+0000 7f78dd1aa640 1 -- 172.21.15.105:0/1851325094 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78d8071a10 msgr2=0x7f78d8155d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:19.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.432+0000 7f78dd1aa640 1 --2- 172.21.15.105:0/1851325094 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78d8071a10 0x7f78d8155d20 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f78c0009fc0 tx=0x7f78c000d560 comp rx=0 tx=0).stop 2023-11-28T18:32:19.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.432+0000 7f78dd1aa640 1 -- 172.21.15.105:0/1851325094 shutdown_connections 2023-11-28T18:32:19.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.432+0000 7f78dd1aa640 1 --2- 172.21.15.105:0/1851325094 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78d815d3c0 0x7f78d8142e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:19.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.432+0000 7f78dd1aa640 1 --2- 172.21.15.105:0/1851325094 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f78b8066900 0x7f78b8068dc0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:19.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.432+0000 7f78dd1aa640 1 --2- 172.21.15.105:0/1851325094 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78d8071a10 0x7f78d8155d20 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:19.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.432+0000 7f78dd1aa640 1 --2- 172.21.15.105:0/1851325094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78d8070a10 0x7f78d815e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:19.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.432+0000 7f78dd1aa640 1 -- 172.21.15.105:0/1851325094 >> 172.21.15.105:0/1851325094 conn(0x7f78d80cf540 msgr2=0x7f78d8161870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:19.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.432+0000 7f78dd1aa640 1 -- 172.21.15.105:0/1851325094 shutdown_connections 2023-11-28T18:32:19.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:19.432+0000 7f78dd1aa640 1 -- 172.21.15.105:0/1851325094 wait complete. 2023-11-28T18:32:19.853 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 139, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:31:58.827408+0000', 'modified': '2023-11-28T18:32:18.299518+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9126, 'mds_1': 9123, 'mds_2': 9120}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9120': {'gid': 9120, 'name': 'd', 'rank': 2, 'incarnation': 138, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.148:6834/3958106803', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6832', 'nonce': 3958106803}, {'type': 'v1', 'addr': '172.21.15.148:6834', 'nonce': 3958106803}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_9123': {'gid': 9123, 'name': 'b', 'rank': 1, 'incarnation': 136, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.148:6835/3706038483', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6833', 'nonce': 3706038483}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 3706038483}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_9126': {'gid': 9126, 'name': 'c', 'rank': 0, 'incarnation': 130, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6837/1978722810', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 1978722810}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 1978722810}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:32:19.853 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-11-28T18:32:19.853 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-28T18:32:20.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.004+0000 7f72af59e640 1 Processor -- start 2023-11-28T18:32:20.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.004+0000 7f72af59e640 1 -- start start 2023-11-28T18:32:20.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.008+0000 7f72af59e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f72b0157990 0x7f72b015ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.008+0000 7f72af59e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72b015d7b0 0x7f72b006cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.008+0000 7f72af59e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b015e0b0 con 0x7f72b01575b0 2023-11-28T18:32:20.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.008+0000 7f72af59e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b00ca660 con 0x7f72b0157990 2023-11-28T18:32:20.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.008+0000 7f72af59e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b015e390 con 0x7f72b015d7b0 2023-11-28T18:32:20.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.008+0000 7f72a7fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f72b0157990 0x7f72b015ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.008+0000 7f72a7fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f72b0157990 0x7f72b015ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50614/0 (socket says 172.21.15.105:50614) 2023-11-28T18:32:20.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.008+0000 7f72a7fff640 1 -- 172.21.15.105:0/1684372405 learned_addr learned my addr 172.21.15.105:0/1684372405 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:20.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.008+0000 7f72aed9d640 1 --2- 172.21.15.105:0/1684372405 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72b015d7b0 0x7f72b006cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.008+0000 7f72a7fff640 1 -- 172.21.15.105:0/1684372405 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72b015d7b0 msgr2=0x7f72b006cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.008+0000 7f72a7fff640 1 --2- 172.21.15.105:0/1684372405 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72b015d7b0 0x7f72b006cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.008+0000 7f72a7fff640 1 -- 172.21.15.105:0/1684372405 >> v1:172.21.15.148:6789/0 conn(0x7f72b01575b0 legacy=0x7f72b00c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.008+0000 7f72a7fff640 1 -- 172.21.15.105:0/1684372405 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b015e670 con 0x7f72b0157990 2023-11-28T18:32:20.010 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.012+0000 7f72a7fff640 1 --2- 172.21.15.105:0/1684372405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72b0157990 0x7f72b015ada0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f729800bb10 tx=0x7f729801ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f88d02ec030a0b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.012+0000 7f72add9b640 1 -- 172.21.15.105:0/1684372405 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72980a7cb0 con 0x7f72b0157990 2023-11-28T18:32:20.011 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.012+0000 7f72add9b640 1 -- 172.21.15.105:0/1684372405 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72980a7e10 con 0x7f72b0157990 2023-11-28T18:32:20.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.012+0000 7f72add9b640 1 -- 172.21.15.105:0/1684372405 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72980af660 con 0x7f72b0157990 2023-11-28T18:32:20.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.012+0000 7f72af59e640 1 -- 172.21.15.105:0/1684372405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72b0157990 msgr2=0x7f72b015ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.012+0000 7f72af59e640 1 --2- 172.21.15.105:0/1684372405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72b0157990 0x7f72b015ada0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f729800bb10 tx=0x7f729801ebe0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.012+0000 7f72af59e640 1 -- 172.21.15.105:0/1684372405 shutdown_connections 2023-11-28T18:32:20.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.012+0000 7f72af59e640 1 --2- 172.21.15.105:0/1684372405 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72b015d7b0 0x7f72b006cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.012+0000 7f72af59e640 1 --2- 172.21.15.105:0/1684372405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72b0157990 0x7f72b015ada0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.012+0000 7f72af59e640 1 -- 172.21.15.105:0/1684372405 >> 172.21.15.105:0/1684372405 conn(0x7f72b00b0480 msgr2=0x7f72b00b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:20.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.012+0000 7f72af59e640 1 -- 172.21.15.105:0/1684372405 shutdown_connections 2023-11-28T18:32:20.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.012+0000 7f72af59e640 1 -- 172.21.15.105:0/1684372405 wait complete. 2023-11-28T18:32:20.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.012+0000 7f72af59e640 1 Processor -- start 2023-11-28T18:32:20.013 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.012+0000 7f72af59e640 1 -- start start 2023-11-28T18:32:20.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72af59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72b01575b0 0x7f72b00c0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72af59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72b0157990 0x7f72b0156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72af59e640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72b015d7b0 0x7f72b01560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72af59e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b0154d00 con 0x7f72b0157990 2023-11-28T18:32:20.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72af59e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b00c5c50 con 0x7f72b01575b0 2023-11-28T18:32:20.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72af59e640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b00c64b0 con 0x7f72b015d7b0 2023-11-28T18:32:20.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72a7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72b0157990 0x7f72b0156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72a7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72b0157990 0x7f72b0156d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50622/0 (socket says 172.21.15.105:50622) 2023-11-28T18:32:20.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72a7fff640 1 -- 172.21.15.105:0/3193760757 learned_addr learned my addr 172.21.15.105:0/3193760757 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:20.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72aed9d640 1 --2- 172.21.15.105:0/3193760757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72b015d7b0 0x7f72b01560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72ae59c640 1 --2- 172.21.15.105:0/3193760757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72b01575b0 0x7f72b00c0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72a7fff640 1 -- 172.21.15.105:0/3193760757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72b015d7b0 msgr2=0x7f72b01560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72a7fff640 1 --2- 172.21.15.105:0/3193760757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72b015d7b0 0x7f72b01560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72a7fff640 1 -- 172.21.15.105:0/3193760757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72b01575b0 msgr2=0x7f72b00c0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72a7fff640 1 --2- 172.21.15.105:0/3193760757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72b01575b0 0x7f72b00c0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72a7fff640 1 -- 172.21.15.105:0/3193760757 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729800b770 con 0x7f72b0157990 2023-11-28T18:32:20.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72ae59c640 1 --2- 172.21.15.105:0/3193760757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72b01575b0 0x7f72b00c0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:32:20.016 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72a7fff640 1 --2- 172.21.15.105:0/3193760757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72b0157990 0x7f72b0156d90 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f729800b2c0 tx=0x7f729801b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72a6ffd640 1 -- 172.21.15.105:0/3193760757 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298006ee0 con 0x7f72b0157990 2023-11-28T18:32:20.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72a6ffd640 1 -- 172.21.15.105:0/3193760757 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72980afdb0 con 0x7f72b0157990 2023-11-28T18:32:20.017 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72af59e640 1 -- 172.21.15.105:0/3193760757 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b0154a20 con 0x7f72b0157990 2023-11-28T18:32:20.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72a6ffd640 1 -- 172.21.15.105:0/3193760757 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72980ba660 con 0x7f72b0157990 2023-11-28T18:32:20.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72af59e640 1 -- 172.21.15.105:0/3193760757 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b00c6140 con 0x7f72b0157990 2023-11-28T18:32:20.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.016+0000 7f72a6ffd640 1 -- 172.21.15.105:0/3193760757 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f72980b6040 con 0x7f72b0157990 2023-11-28T18:32:20.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72a6ffd640 1 --2- 172.21.15.105:0/3193760757 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7290066900 0x7f7290068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.018 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72a6ffd640 1 -- 172.21.15.105:0/3193760757 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7298121780 con 0x7f72b0157990 2023-11-28T18:32:20.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72ae59c640 1 --2- 172.21.15.105:0/3193760757 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7290066900 0x7f7290068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72b7dec000 1 Processor -- start 2023-11-28T18:32:20.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72ae59c640 1 --2- 172.21.15.105:0/3193760757 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7290066900 0x7f7290068dc0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f72a0000f70 tx=0x7f72a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.019 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72b7dec000 1 -- start start 2023-11-28T18:32:20.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72b7dec000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d26d3d4d30 0x55d26d3d5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72b7dec000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d26d3d5650 0x55d26d387fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.021 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72b7dec000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d388500 0x55d26d3888e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72b7dec000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26d251110 con 0x55d26d388500 2023-11-28T18:32:20.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72b7dec000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26d1e19a0 con 0x55d26d3d4d30 2023-11-28T18:32:20.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72b7dec000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d26d1deed0 con 0x55d26d3d5650 2023-11-28T18:32:20.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72aed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d388500 0x55d26d3888e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72aed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d388500 0x55d26d3888e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50624/0 (socket says 172.21.15.105:50624) 2023-11-28T18:32:20.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72aed9d640 1 -- 172.21.15.105:0/494437495 learned_addr learned my addr 172.21.15.105:0/494437495 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:20.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72ae59c640 1 --2- 172.21.15.105:0/494437495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d26d3d4d30 0x55d26d3d5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72a7fff640 1 --2- 172.21.15.105:0/494437495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d26d3d5650 0x55d26d387fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72aed9d640 1 -- 172.21.15.105:0/494437495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d26d3d5650 msgr2=0x55d26d387fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72aed9d640 1 --2- 172.21.15.105:0/494437495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d26d3d5650 0x55d26d387fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.022 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72aed9d640 1 -- 172.21.15.105:0/494437495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d26d3d4d30 msgr2=0x55d26d3d5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72aed9d640 1 --2- 172.21.15.105:0/494437495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d26d3d4d30 0x55d26d3d5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72aed9d640 1 -- 172.21.15.105:0/494437495 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d26ced2890 con 0x55d26d388500 2023-11-28T18:32:20.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f72aed9d640 1 --2- 172.21.15.105:0/494437495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d388500 0x55d26d3888e0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f72a8011550 tx=0x7f72a8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3d8c9cf2c0629f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f728ffff640 1 -- 172.21.15.105:0/494437495 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a800e040 con 0x55d26d388500 2023-11-28T18:32:20.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f728ffff640 1 -- 172.21.15.105:0/494437495 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a800f040 con 0x55d26d388500 2023-11-28T18:32:20.023 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.020+0000 7f728ffff640 1 -- 172.21.15.105:0/494437495 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a8015a40 con 0x55d26d388500 2023-11-28T18:32:20.026 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72b7dec000 1 -- 172.21.15.105:0/494437495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d388500 msgr2=0x55d26d3888e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72b7dec000 1 --2- 172.21.15.105:0/494437495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d388500 0x55d26d3888e0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f72a8011550 tx=0x7f72a8009750 comp rx=0 tx=0).stop 2023-11-28T18:32:20.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72b7dec000 1 -- 172.21.15.105:0/494437495 shutdown_connections 2023-11-28T18:32:20.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72b7dec000 1 --2- 172.21.15.105:0/494437495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d388500 0x55d26d3888e0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72b7dec000 1 --2- 172.21.15.105:0/494437495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d26d3d5650 0x55d26d387fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72b7dec000 1 --2- 172.21.15.105:0/494437495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d26d3d4d30 0x55d26d3d5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72b7dec000 1 -- 172.21.15.105:0/494437495 >> 172.21.15.105:0/494437495 conn(0x55d26d3b58c0 msgr2=0x55d26d3b7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:20.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72b7dec000 1 -- 172.21.15.105:0/494437495 shutdown_connections 2023-11-28T18:32:20.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72b7dec000 1 -- 172.21.15.105:0/494437495 wait complete. 2023-11-28T18:32:20.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72b7dec000 1 -- start start 2023-11-28T18:32:20.027 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72b7dec000 1 Processor -- start 2023-11-28T18:32:20.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72b7dec000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d26d388500 0x55d26d38e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72b7dec000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d3d4d30 0x55d26d38ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72b7dec000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d26d3d5650 0x55d26d38ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72b7dec000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26d1da7c0 con 0x55d26d3d4d30 2023-11-28T18:32:20.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72b7dec000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26d192bf0 con 0x55d26d388500 2023-11-28T18:32:20.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72b7dec000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d26d198a90 con 0x55d26d3d5650 2023-11-28T18:32:20.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72b7dec000 10 client.0 _ll_register_callbacks cb 0x55d26d375e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:32:20.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72a7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d3d4d30 0x55d26d38ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72a7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d3d4d30 0x55d26d38ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50638/0 (socket says 172.21.15.105:50638) 2023-11-28T18:32:20.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72a7fff640 1 -- 172.21.15.105:0/3196867995 learned_addr learned my addr 172.21.15.105:0/3196867995 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:20.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72ae59c640 1 --2- 172.21.15.105:0/3196867995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d26d388500 0x55d26d38e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.028 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72aed9d640 1 --2- 172.21.15.105:0/3196867995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d26d3d5650 0x55d26d38ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72a7fff640 1 -- 172.21.15.105:0/3196867995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d26d3d5650 msgr2=0x55d26d38ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72a7fff640 1 --2- 172.21.15.105:0/3196867995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d26d3d5650 0x55d26d38ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72a7fff640 1 -- 172.21.15.105:0/3196867995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d26d388500 msgr2=0x55d26d38e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72a7fff640 1 --2- 172.21.15.105:0/3196867995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d26d388500 0x55d26d38e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72a7fff640 1 -- 172.21.15.105:0/3196867995 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a8011230 con 0x55d26d3d4d30 2023-11-28T18:32:20.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72a7fff640 1 --2- 172.21.15.105:0/3196867995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d3d4d30 0x55d26d38ea20 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f729811fb60 tx=0x7f72980f2060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f5dc7d6ae96e12 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f728d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:32:20.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f728d7fa640 1 -- 172.21.15.105:0/3196867995 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72980f2b60 con 0x55d26d3d4d30 2023-11-28T18:32:20.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f728d7fa640 1 -- 172.21.15.105:0/3196867995 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f72980fa950 con 0x55d26d3d4d30 2023-11-28T18:32:20.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72b7dec000 1 -- 172.21.15.105:0/3196867995 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d26d19c470 con 0x55d26d3d4d30 2023-11-28T18:32:20.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f728d7fa640 1 -- 172.21.15.105:0/3196867995 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=139) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72980f1bf0 con 0x55d26d3d4d30 2023-11-28T18:32:20.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72b7dec000 10 client.9330 fetch_fsmap learned FSMap version 139 2023-11-28T18:32:20.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.024+0000 7f72b7dec000 1 -- 172.21.15.105:0/3196867995 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=139,monmap=2+}) v3 -- 0x55d26d19b190 con 0x55d26d3d4d30 2023-11-28T18:32:20.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f728d7fa640 1 -- 172.21.15.105:0/3196867995 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d26d19c470 con 0x55d26d3d4d30 2023-11-28T18:32:20.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f728d7fa640 1 -- 172.21.15.105:0/3196867995 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 139) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f72980fa4c0 con 0x55d26d3d4d30 2023-11-28T18:32:20.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f72b7dec000 10 client.9330 fetch_fsmap finished waiting for FSMap version 139 2023-11-28T18:32:20.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f72b7dec000 10 client.9330 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T18:32:20.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f72b7dec000 20 client.9330 populate_metadata read hostname 'smithi105' 2023-11-28T18:32:20.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f72b7dec000 1 --2- 172.21.15.105:0/3196867995 >> [v2:172.21.15.148:6832/3958106803,v1:172.21.15.148:6834/3958106803] conn(0x55d26d38f970 0x55d26d3e6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f72b7dec000 4 client.9330 mds_command: new command op to 9120 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:32:20.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f72b7dec000 1 -- 172.21.15.105:0/3196867995 --> [v2:172.21.15.148:6832/3958106803,v1:172.21.15.148:6834/3958106803] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d26d19fb50 con 0x55d26d38f970 2023-11-28T18:32:20.030 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f72ae59c640 1 --2- 172.21.15.105:0/3196867995 >> [v2:172.21.15.148:6832/3958106803,v1:172.21.15.148:6834/3958106803] conn(0x55d26d38f970 0x55d26d3e6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f72ae59c640 1 --2- 172.21.15.105:0/3196867995 >> [v2:172.21.15.148:6832/3958106803,v1:172.21.15.148:6834/3958106803] conn(0x55d26d38f970 0x55d26d3e6ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b3fd14a588bb6e27 server_cookie=826a0aa1f61b927 in_seq=0 out_seq=0 2023-11-28T18:32:20.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f728d7fa640 10 client.9330 ms_handle_connect on v2:172.21.15.148:6832/3958106803 2023-11-28T18:32:20.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f728d7fa640 1 -- 172.21.15.105:0/3196867995 <== mds.2 v2:172.21.15.148:6832/3958106803 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d26d19fb50 con 0x55d26d38f970 2023-11-28T18:32:20.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f728d7fa640 10 client.9330 handle_command_reply: tid=0 2023-11-28T18:32:20.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f72b7dec000 1 client.9330 shutdown 2023-11-28T18:32:20.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f72b7dec000 1 -- 172.21.15.105:0/3196867995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d3d4d30 msgr2=0x55d26d38ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f72b7dec000 1 --2- 172.21.15.105:0/3196867995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d3d4d30 0x55d26d38ea20 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f729811fb60 tx=0x7f72980f2060 comp rx=0 tx=0).stop 2023-11-28T18:32:20.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f72b7dec000 1 -- 172.21.15.105:0/3196867995 shutdown_connections 2023-11-28T18:32:20.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f72b7dec000 1 --2- 172.21.15.105:0/3196867995 >> [v2:172.21.15.148:6832/3958106803,v1:172.21.15.148:6834/3958106803] conn(0x55d26d38f970 0x55d26d3e6ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f72b7dec000 1 --2- 172.21.15.105:0/3196867995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d26d3d5650 0x55d26d38ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f72b7dec000 1 --2- 172.21.15.105:0/3196867995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d3d4d30 0x55d26d38ea20 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f72b7dec000 1 --2- 172.21.15.105:0/3196867995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d26d388500 0x55d26d38e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f72b7dec000 1 -- 172.21.15.105:0/3196867995 >> 172.21.15.105:0/3196867995 conn(0x55d26d3b58c0 msgr2=0x55d26d3876b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:20.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f728d7fa640 0 client.9330 ms_handle_reset on v2:172.21.15.148:6832/3958106803 2023-11-28T18:32:20.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f72b7dec000 1 -- 172.21.15.105:0/3196867995 shutdown_connections 2023-11-28T18:32:20.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f72b7dec000 1 -- 172.21.15.105:0/3196867995 wait complete. 2023-11-28T18:32:20.033 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.028+0000 7f72b7dec000 20 client.9330 trim_cache size 0 max 16384 2023-11-28T18:32:20.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.044+0000 7f72b7dec000 1 Processor -- start 2023-11-28T18:32:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.044+0000 7f72b7dec000 1 -- start start 2023-11-28T18:32:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.044+0000 7f72b7dec000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d26d388500 0x55d26d3b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.044+0000 7f72b7dec000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d26d3d4d30 0x55d26d3b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.044+0000 7f72b7dec000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d3d5650 0x55d26d3e6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.044+0000 7f72b7dec000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26ce93940 con 0x55d26d3d5650 2023-11-28T18:32:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.044+0000 7f72b7dec000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26d1d7bc0 con 0x55d26d388500 2023-11-28T18:32:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.044+0000 7f72b7dec000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d26d38bdd0 con 0x55d26d3d4d30 2023-11-28T18:32:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.044+0000 7f72aed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d3d5650 0x55d26d3e6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.044+0000 7f72aed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d3d5650 0x55d26d3e6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50650/0 (socket says 172.21.15.105:50650) 2023-11-28T18:32:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.044+0000 7f72aed9d640 1 -- 172.21.15.105:0/92094934 learned_addr learned my addr 172.21.15.105:0/92094934 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.044+0000 7f72ae59c640 1 --2- 172.21.15.105:0/92094934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d26d388500 0x55d26d3b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.044+0000 7f72a7fff640 1 --2- 172.21.15.105:0/92094934 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d26d3d4d30 0x55d26d3b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.044+0000 7f72aed9d640 1 -- 172.21.15.105:0/92094934 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d26d3d4d30 msgr2=0x55d26d3b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.044+0000 7f72aed9d640 1 --2- 172.21.15.105:0/92094934 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d26d3d4d30 0x55d26d3b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.044+0000 7f72aed9d640 1 -- 172.21.15.105:0/92094934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d26d388500 msgr2=0x55d26d3b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.044+0000 7f72aed9d640 1 --2- 172.21.15.105:0/92094934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d26d388500 0x55d26d3b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.044+0000 7f72aed9d640 1 -- 172.21.15.105:0/92094934 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d26d388370 con 0x55d26d3d5650 2023-11-28T18:32:20.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.044+0000 7f72a7fff640 1 --2- 172.21.15.105:0/92094934 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d26d3d4d30 0x55d26d3b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:32:20.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.044+0000 7f72aed9d640 1 --2- 172.21.15.105:0/92094934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d3d5650 0x55d26d3e6ff0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f72a8019000 tx=0x7f72a8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98307e59ceaefbab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72a4ff9640 1 -- 172.21.15.105:0/92094934 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a800e040 con 0x55d26d3d5650 2023-11-28T18:32:20.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72a4ff9640 1 -- 172.21.15.105:0/92094934 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a8003bc0 con 0x55d26d3d5650 2023-11-28T18:32:20.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72a4ff9640 1 -- 172.21.15.105:0/92094934 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a8009cc0 con 0x55d26d3d5650 2023-11-28T18:32:20.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72b7dec000 1 -- 172.21.15.105:0/92094934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d3d5650 msgr2=0x55d26d3e6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72b7dec000 1 --2- 172.21.15.105:0/92094934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d3d5650 0x55d26d3e6ff0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f72a8019000 tx=0x7f72a8016000 comp rx=0 tx=0).stop 2023-11-28T18:32:20.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72b7dec000 1 -- 172.21.15.105:0/92094934 shutdown_connections 2023-11-28T18:32:20.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72b7dec000 1 --2- 172.21.15.105:0/92094934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d3d5650 0x55d26d3e6ff0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72b7dec000 1 --2- 172.21.15.105:0/92094934 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d26d3d4d30 0x55d26d3b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72b7dec000 1 --2- 172.21.15.105:0/92094934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d26d388500 0x55d26d3b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72b7dec000 1 -- 172.21.15.105:0/92094934 >> 172.21.15.105:0/92094934 conn(0x55d26d3b58c0 msgr2=0x55d26d386720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:20.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72b7dec000 1 -- 172.21.15.105:0/92094934 shutdown_connections 2023-11-28T18:32:20.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72b7dec000 1 -- 172.21.15.105:0/92094934 wait complete. 2023-11-28T18:32:20.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72b7dec000 1 -- start start 2023-11-28T18:32:20.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72b7dec000 1 Processor -- start 2023-11-28T18:32:20.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72b7dec000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d26d388500 0x55d26d435810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72b7dec000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d3d4d30 0x55d26d435d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72b7dec000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d26d3d5650 0x55d26d3f0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72b7dec000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26d1da7c0 con 0x55d26d3d4d30 2023-11-28T18:32:20.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72b7dec000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26d18d0a0 con 0x55d26d388500 2023-11-28T18:32:20.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72b7dec000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d26d1a42e0 con 0x55d26d3d5650 2023-11-28T18:32:20.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72b7dec000 10 client.0 _ll_register_callbacks cb 0x55d26d251060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:32:20.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72a7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d3d4d30 0x55d26d435d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72a7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d3d4d30 0x55d26d435d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50656/0 (socket says 172.21.15.105:50656) 2023-11-28T18:32:20.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72a7fff640 1 -- 172.21.15.105:0/1501361249 learned_addr learned my addr 172.21.15.105:0/1501361249 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:20.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72aed9d640 1 --2- 172.21.15.105:0/1501361249 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d26d3d5650 0x55d26d3f0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72ae59c640 1 --2- 172.21.15.105:0/1501361249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d26d388500 0x55d26d435810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72a7fff640 1 -- 172.21.15.105:0/1501361249 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d26d3d5650 msgr2=0x55d26d3f0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72a7fff640 1 --2- 172.21.15.105:0/1501361249 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d26d3d5650 0x55d26d3f0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.054 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72a7fff640 1 -- 172.21.15.105:0/1501361249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d26d388500 msgr2=0x55d26d435810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72a7fff640 1 --2- 172.21.15.105:0/1501361249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d26d388500 0x55d26d435810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72a7fff640 1 -- 172.21.15.105:0/1501361249 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d26ced2890 con 0x55d26d3d4d30 2023-11-28T18:32:20.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72aed9d640 1 --2- 172.21.15.105:0/1501361249 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d26d3d5650 0x55d26d3f0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:32:20.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72ae59c640 1 --2- 172.21.15.105:0/1501361249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d26d388500 0x55d26d435810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:32:20.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72a7fff640 1 --2- 172.21.15.105:0/1501361249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d3d4d30 0x55d26d435d50 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f72980f0a50 tx=0x7f729811fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6812db77bbc65de server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f728d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:32:20.055 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f728d7fa640 1 -- 172.21.15.105:0/1501361249 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72980f2700 con 0x55d26d3d4d30 2023-11-28T18:32:20.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f728d7fa640 1 -- 172.21.15.105:0/1501361249 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f72980fa950 con 0x55d26d3d4d30 2023-11-28T18:32:20.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.048+0000 7f72b7dec000 1 -- 172.21.15.105:0/1501361249 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d26d1d9110 con 0x55d26d3d4d30 2023-11-28T18:32:20.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f728d7fa640 1 -- 172.21.15.105:0/1501361249 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=139) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72980faba0 con 0x55d26d3d4d30 2023-11-28T18:32:20.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72b7dec000 10 client.9336 fetch_fsmap learned FSMap version 139 2023-11-28T18:32:20.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72b7dec000 1 -- 172.21.15.105:0/1501361249 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=139,monmap=2+}) v3 -- 0x55d26d181490 con 0x55d26d3d4d30 2023-11-28T18:32:20.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f728d7fa640 1 -- 172.21.15.105:0/1501361249 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d26d1d9110 con 0x55d26d3d4d30 2023-11-28T18:32:20.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f728d7fa640 1 -- 172.21.15.105:0/1501361249 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 139) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f7298102050 con 0x55d26d3d4d30 2023-11-28T18:32:20.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72b7dec000 10 client.9336 fetch_fsmap finished waiting for FSMap version 139 2023-11-28T18:32:20.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72b7dec000 10 client.9336 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T18:32:20.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72b7dec000 20 client.9336 populate_metadata read hostname 'smithi105' 2023-11-28T18:32:20.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72b7dec000 1 --2- 172.21.15.105:0/1501361249 >> [v2:172.21.15.148:6832/3958106803,v1:172.21.15.148:6834/3958106803] conn(0x55d26d3f1430 0x55d26d3f1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72b7dec000 4 client.9336 mds_command: new command op to 9120 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:32:20.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72b7dec000 1 -- 172.21.15.105:0/1501361249 --> [v2:172.21.15.148:6832/3958106803,v1:172.21.15.148:6834/3958106803] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d26d207650 con 0x55d26d3f1430 2023-11-28T18:32:20.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72ae59c640 1 --2- 172.21.15.105:0/1501361249 >> [v2:172.21.15.148:6832/3958106803,v1:172.21.15.148:6834/3958106803] conn(0x55d26d3f1430 0x55d26d3f1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72ae59c640 1 --2- 172.21.15.105:0/1501361249 >> [v2:172.21.15.148:6832/3958106803,v1:172.21.15.148:6834/3958106803] conn(0x55d26d3f1430 0x55d26d3f1810 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=86c3c87ca431ff44 server_cookie=acbae93c93641f5b in_seq=0 out_seq=0 2023-11-28T18:32:20.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f728d7fa640 10 client.9336 ms_handle_connect on v2:172.21.15.148:6832/3958106803 2023-11-28T18:32:20.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f728d7fa640 1 -- 172.21.15.105:0/1501361249 <== mds.2 v2:172.21.15.148:6832/3958106803 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x55d26d207650 con 0x55d26d3f1430 2023-11-28T18:32:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f728d7fa640 10 client.9336 handle_command_reply: tid=0 2023-11-28T18:32:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72b7dec000 1 client.9336 shutdown 2023-11-28T18:32:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72b7dec000 1 -- 172.21.15.105:0/1501361249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d3d4d30 msgr2=0x55d26d435d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72b7dec000 1 --2- 172.21.15.105:0/1501361249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d3d4d30 0x55d26d435d50 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f72980f0a50 tx=0x7f729811fa30 comp rx=0 tx=0).stop 2023-11-28T18:32:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72b7dec000 1 -- 172.21.15.105:0/1501361249 shutdown_connections 2023-11-28T18:32:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72b7dec000 1 --2- 172.21.15.105:0/1501361249 >> [v2:172.21.15.148:6832/3958106803,v1:172.21.15.148:6834/3958106803] conn(0x55d26d3f1430 0x55d26d3f1810 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72b7dec000 1 --2- 172.21.15.105:0/1501361249 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d26d3d5650 0x55d26d3f0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72b7dec000 1 --2- 172.21.15.105:0/1501361249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d26d3d4d30 0x55d26d435d50 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72b7dec000 1 --2- 172.21.15.105:0/1501361249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d26d388500 0x55d26d435810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f728d7fa640 0 client.9336 ms_handle_reset on v2:172.21.15.148:6832/3958106803 2023-11-28T18:32:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72b7dec000 1 -- 172.21.15.105:0/1501361249 >> 172.21.15.105:0/1501361249 conn(0x55d26d3b58c0 msgr2=0x55d26d386fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72b7dec000 1 -- 172.21.15.105:0/1501361249 shutdown_connections 2023-11-28T18:32:20.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72b7dec000 1 -- 172.21.15.105:0/1501361249 wait complete. 2023-11-28T18:32:20.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72b7dec000 20 client.9336 trim_cache size 0 max 16384 2023-11-28T18:32:20.061 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:32:20.061 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:32:20.061 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 2, 2023-11-28T18:32:20.061 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 9120, 2023-11-28T18:32:20.061 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:32:20.061 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:32:20.062 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:32:20.062 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 1.768906037, 2023-11-28T18:32:20.062 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 139, 2023-11-28T18:32:20.062 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 129, 2023-11-28T18:32:20.062 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 129, 2023-11-28T18:32:20.062 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 26.423146482 2023-11-28T18:32:20.062 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:32:20.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72af59e640 1 -- 172.21.15.105:0/3193760757 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7290066900 msgr2=0x7f7290068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72af59e640 1 --2- 172.21.15.105:0/3193760757 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7290066900 0x7f7290068dc0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f72a0000f70 tx=0x7f72a0009040 comp rx=0 tx=0).stop 2023-11-28T18:32:20.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72af59e640 1 -- 172.21.15.105:0/3193760757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72b0157990 msgr2=0x7f72b0156d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72af59e640 1 --2- 172.21.15.105:0/3193760757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72b0157990 0x7f72b0156d90 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f729800b2c0 tx=0x7f729801b980 comp rx=0 tx=0).stop 2023-11-28T18:32:20.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72af59e640 1 -- 172.21.15.105:0/3193760757 shutdown_connections 2023-11-28T18:32:20.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72af59e640 1 --2- 172.21.15.105:0/3193760757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72b015d7b0 0x7f72b01560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72af59e640 1 --2- 172.21.15.105:0/3193760757 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7290066900 0x7f7290068dc0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72af59e640 1 --2- 172.21.15.105:0/3193760757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72b0157990 0x7f72b0156d90 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72af59e640 1 --2- 172.21.15.105:0/3193760757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72b01575b0 0x7f72b00c0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72af59e640 1 -- 172.21.15.105:0/3193760757 >> 172.21.15.105:0/3193760757 conn(0x7f72b00b0480 msgr2=0x7f72b00cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:20.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72af59e640 1 -- 172.21.15.105:0/3193760757 shutdown_connections 2023-11-28T18:32:20.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.052+0000 7f72af59e640 1 -- 172.21.15.105:0/3193760757 wait complete. 2023-11-28T18:32:20.072 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-28T18:32:20.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.212+0000 7f900e4ea640 1 Processor -- start 2023-11-28T18:32:20.211 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.212+0000 7f900e4ea640 1 -- start start 2023-11-28T18:32:20.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.212+0000 7f900e4ea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9008071960 0x7f900815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.212+0000 7f900e4ea640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f900815b030 0x7f90080709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.212+0000 7f900e4ea640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90080701d0 con 0x7f90080b4290 2023-11-28T18:32:20.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.212+0000 7f900e4ea640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f900806df80 con 0x7f900815b030 2023-11-28T18:32:20.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.212+0000 7f900e4ea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9008071df0 con 0x7f9008071960 2023-11-28T18:32:20.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.212+0000 7f900ca87640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f900815b030 0x7f90080709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.212+0000 7f900ca87640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f900815b030 0x7f90080709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50658/0 (socket says 172.21.15.105:50658) 2023-11-28T18:32:20.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.212+0000 7f900ca87640 1 -- 172.21.15.105:0/1019196196 learned_addr learned my addr 172.21.15.105:0/1019196196 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:20.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.212+0000 7f90077fe640 1 --2- 172.21.15.105:0/1019196196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9008071960 0x7f900815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.212+0000 7f900ca87640 1 -- 172.21.15.105:0/1019196196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9008071960 msgr2=0x7f900815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.212+0000 7f900ca87640 1 --2- 172.21.15.105:0/1019196196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9008071960 0x7f900815f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.212+0000 7f900ca87640 1 -- 172.21.15.105:0/1019196196 >> v1:172.21.15.148:6789/0 conn(0x7f90080b4290 legacy=0x7f90080b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.212+0000 7f900ca87640 1 -- 172.21.15.105:0/1019196196 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90080720d0 con 0x7f900815b030 2023-11-28T18:32:20.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.216+0000 7f900ca87640 1 --2- 172.21.15.105:0/1019196196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900815b030 0x7f90080709b0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f9000009e50 tx=0x7f900001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e12f298a3a034cb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.216+0000 7f9006ffd640 1 -- 172.21.15.105:0/1019196196 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90000a57c0 con 0x7f900815b030 2023-11-28T18:32:20.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.216+0000 7f9006ffd640 1 -- 172.21.15.105:0/1019196196 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90000a5920 con 0x7f900815b030 2023-11-28T18:32:20.216 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.216+0000 7f9006ffd640 1 -- 172.21.15.105:0/1019196196 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90000a5cb0 con 0x7f900815b030 2023-11-28T18:32:20.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.216+0000 7f900e4ea640 1 -- 172.21.15.105:0/1019196196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900815b030 msgr2=0x7f90080709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.216+0000 7f900e4ea640 1 --2- 172.21.15.105:0/1019196196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900815b030 0x7f90080709b0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f9000009e50 tx=0x7f900001c9e0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.216+0000 7f900e4ea640 1 -- 172.21.15.105:0/1019196196 shutdown_connections 2023-11-28T18:32:20.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.216+0000 7f900e4ea640 1 --2- 172.21.15.105:0/1019196196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900815b030 0x7f90080709b0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.216+0000 7f900e4ea640 1 --2- 172.21.15.105:0/1019196196 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9008071960 0x7f900815f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.216+0000 7f900e4ea640 1 -- 172.21.15.105:0/1019196196 >> 172.21.15.105:0/1019196196 conn(0x7f90080ceb90 msgr2=0x7f90080cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:20.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.216+0000 7f900e4ea640 1 -- 172.21.15.105:0/1019196196 shutdown_connections 2023-11-28T18:32:20.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.216+0000 7f900e4ea640 1 -- 172.21.15.105:0/1019196196 wait complete. 2023-11-28T18:32:20.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f900e4ea640 1 Processor -- start 2023-11-28T18:32:20.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f900e4ea640 1 -- start start 2023-11-28T18:32:20.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f900e4ea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90080b4290 0x7f9008099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f900e4ea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900815b030 0x7f90081f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f900e4ea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f900809b560 0x7f90081f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f900e4ea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900809c6d0 con 0x7f900815b030 2023-11-28T18:32:20.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f900e4ea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900809a030 con 0x7f900809b560 2023-11-28T18:32:20.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f900e4ea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900809a310 con 0x7f90080b4290 2023-11-28T18:32:20.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f90077fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900815b030 0x7f90081f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f90077fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900815b030 0x7f90081f5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50660/0 (socket says 172.21.15.105:50660) 2023-11-28T18:32:20.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f90077fe640 1 -- 172.21.15.105:0/3847681399 learned_addr learned my addr 172.21.15.105:0/3847681399 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:20.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f9007fff640 1 --2- 172.21.15.105:0/3847681399 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90080b4290 0x7f9008099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f900ca87640 1 --2- 172.21.15.105:0/3847681399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f900809b560 0x7f90081f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f90077fe640 1 -- 172.21.15.105:0/3847681399 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90080b4290 msgr2=0x7f9008099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f90077fe640 1 --2- 172.21.15.105:0/3847681399 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90080b4290 0x7f9008099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f90077fe640 1 -- 172.21.15.105:0/3847681399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f900809b560 msgr2=0x7f90081f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f90077fe640 1 --2- 172.21.15.105:0/3847681399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f900809b560 0x7f90081f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f90077fe640 1 -- 172.21.15.105:0/3847681399 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9000009b30 con 0x7f900815b030 2023-11-28T18:32:20.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f90077fe640 1 --2- 172.21.15.105:0/3847681399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900815b030 0x7f90081f5d00 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00e930 tx=0x7f8ffc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f9004ff9640 1 -- 172.21.15.105:0/3847681399 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc009690 con 0x7f900815b030 2023-11-28T18:32:20.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f9004ff9640 1 -- 172.21.15.105:0/3847681399 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ffc00ca00 con 0x7f900815b030 2023-11-28T18:32:20.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f9004ff9640 1 -- 172.21.15.105:0/3847681399 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc00cd70 con 0x7f900815b030 2023-11-28T18:32:20.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f900e4ea640 1 -- 172.21.15.105:0/3847681399 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900809a5f0 con 0x7f900815b030 2023-11-28T18:32:20.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.220+0000 7f900e4ea640 1 -- 172.21.15.105:0/3847681399 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900809abb0 con 0x7f900815b030 2023-11-28T18:32:20.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.224+0000 7f9004ff9640 1 -- 172.21.15.105:0/3847681399 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f8ffc013470 con 0x7f900815b030 2023-11-28T18:32:20.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.224+0000 7f9004ff9640 1 --2- 172.21.15.105:0/3847681399 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8ff40668d0 0x7f8ff4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.224+0000 7f9004ff9640 1 -- 172.21.15.105:0/3847681399 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8ffc084710 con 0x7f900815b030 2023-11-28T18:32:20.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.224+0000 7f9007fff640 1 --2- 172.21.15.105:0/3847681399 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8ff40668d0 0x7f8ff4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.224+0000 7f9011851000 1 Processor -- start 2023-11-28T18:32:20.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.224+0000 7f9007fff640 1 --2- 172.21.15.105:0/3847681399 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8ff40668d0 0x7f8ff4068d90 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8007fd0 tx=0x7f8ff8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.224+0000 7f9011851000 1 -- start start 2023-11-28T18:32:20.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f73a4cbd30 0x55f73a4cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a4cc650 0x55f73a47efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f73a47f500 0x55f73a47f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73a348110 con 0x55f73a4cc650 2023-11-28T18:32:20.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73a2d89a0 con 0x55f73a47f500 2023-11-28T18:32:20.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f73a2d5ed0 con 0x55f73a4cbd30 2023-11-28T18:32:20.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f90077fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a4cc650 0x55f73a47efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f90077fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a4cc650 0x55f73a47efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50666/0 (socket says 172.21.15.105:50666) 2023-11-28T18:32:20.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f90077fe640 1 -- 172.21.15.105:0/2983535512 learned_addr learned my addr 172.21.15.105:0/2983535512 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:20.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9007fff640 1 --2- 172.21.15.105:0/2983535512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f73a4cbd30 0x55f73a4cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f900ca87640 1 --2- 172.21.15.105:0/2983535512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f73a47f500 0x55f73a47f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f90077fe640 1 -- 172.21.15.105:0/2983535512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f73a4cbd30 msgr2=0x55f73a4cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f90077fe640 1 --2- 172.21.15.105:0/2983535512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f73a4cbd30 0x55f73a4cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f90077fe640 1 -- 172.21.15.105:0/2983535512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f73a47f500 msgr2=0x55f73a47f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f90077fe640 1 --2- 172.21.15.105:0/2983535512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f73a47f500 0x55f73a47f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f90077fe640 1 -- 172.21.15.105:0/2983535512 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f739fc9890 con 0x55f73a4cc650 2023-11-28T18:32:20.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f900ca87640 1 --2- 172.21.15.105:0/2983535512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f73a47f500 0x55f73a47f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:32:20.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f90077fe640 1 --2- 172.21.15.105:0/2983535512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a4cc650 0x55f73a47efc0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc004780 tx=0x7f8ffc055810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74707837360bfa1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f8fee7fc640 1 -- 172.21.15.105:0/2983535512 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc053a70 con 0x55f73a4cc650 2023-11-28T18:32:20.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f8fee7fc640 1 -- 172.21.15.105:0/2983535512 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ffc051de0 con 0x55f73a4cc650 2023-11-28T18:32:20.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f8fee7fc640 1 -- 172.21.15.105:0/2983535512 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc05d6b0 con 0x55f73a4cc650 2023-11-28T18:32:20.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 -- 172.21.15.105:0/2983535512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a4cc650 msgr2=0x55f73a47efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 --2- 172.21.15.105:0/2983535512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a4cc650 0x55f73a47efc0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc004780 tx=0x7f8ffc055810 comp rx=0 tx=0).stop 2023-11-28T18:32:20.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 -- 172.21.15.105:0/2983535512 shutdown_connections 2023-11-28T18:32:20.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 --2- 172.21.15.105:0/2983535512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f73a47f500 0x55f73a47f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 --2- 172.21.15.105:0/2983535512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a4cc650 0x55f73a47efc0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 --2- 172.21.15.105:0/2983535512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f73a4cbd30 0x55f73a4cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 -- 172.21.15.105:0/2983535512 >> 172.21.15.105:0/2983535512 conn(0x55f73a4ac8c0 msgr2=0x55f73a4aecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:20.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 -- 172.21.15.105:0/2983535512 shutdown_connections 2023-11-28T18:32:20.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 -- 172.21.15.105:0/2983535512 wait complete. 2023-11-28T18:32:20.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 -- start start 2023-11-28T18:32:20.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 Processor -- start 2023-11-28T18:32:20.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a47f500 0x55f73a4854e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f73a4cbd30 0x55f73a485a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f73a4cc650 0x55f73a485f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73a2d17c0 con 0x55f73a47f500 2023-11-28T18:32:20.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73a289bf0 con 0x55f73a4cbd30 2023-11-28T18:32:20.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f73a28fa90 con 0x55f73a4cc650 2023-11-28T18:32:20.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.228+0000 7f9011851000 10 client.0 _ll_register_callbacks cb 0x55f73a46ce20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:32:20.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9007fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a47f500 0x55f73a4854e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9007fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a47f500 0x55f73a4854e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50670/0 (socket says 172.21.15.105:50670) 2023-11-28T18:32:20.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9007fff640 1 -- 172.21.15.105:0/2773370959 learned_addr learned my addr 172.21.15.105:0/2773370959 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:20.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f90077fe640 1 --2- 172.21.15.105:0/2773370959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f73a4cbd30 0x55f73a485a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f900ca87640 1 --2- 172.21.15.105:0/2773370959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f73a4cc650 0x55f73a485f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9007fff640 1 -- 172.21.15.105:0/2773370959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f73a4cc650 msgr2=0x55f73a485f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9007fff640 1 --2- 172.21.15.105:0/2773370959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f73a4cc650 0x55f73a485f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9007fff640 1 -- 172.21.15.105:0/2773370959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f73a4cbd30 msgr2=0x55f73a485a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9007fff640 1 --2- 172.21.15.105:0/2773370959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f73a4cbd30 0x55f73a485a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9007fff640 1 -- 172.21.15.105:0/2773370959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900809abb0 con 0x55f73a47f500 2023-11-28T18:32:20.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f900ca87640 1 --2- 172.21.15.105:0/2773370959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f73a4cc650 0x55f73a485f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:32:20.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9007fff640 1 --2- 172.21.15.105:0/2773370959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a47f500 0x55f73a4854e0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f8ff80013a0 tx=0x7f8ff8001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9560fb0d4e82df2c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f8fe6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:32:20.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f8fe6ffd640 1 -- 172.21.15.105:0/2773370959 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8013680 con 0x55f73a47f500 2023-11-28T18:32:20.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f8fe6ffd640 1 -- 172.21.15.105:0/2773370959 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8ff801c040 con 0x55f73a47f500 2023-11-28T18:32:20.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9011851000 1 -- 172.21.15.105:0/2773370959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f73a293470 con 0x55f73a47f500 2023-11-28T18:32:20.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f8fe6ffd640 1 -- 172.21.15.105:0/2773370959 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=139) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ff80137e0 con 0x55f73a47f500 2023-11-28T18:32:20.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9011851000 10 client.9348 fetch_fsmap learned FSMap version 139 2023-11-28T18:32:20.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9011851000 1 -- 172.21.15.105:0/2773370959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=139,monmap=2+}) v3 -- 0x55f73a292190 con 0x55f73a47f500 2023-11-28T18:32:20.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f8fe6ffd640 1 -- 172.21.15.105:0/2773370959 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f73a293470 con 0x55f73a47f500 2023-11-28T18:32:20.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f8fe6ffd640 1 -- 172.21.15.105:0/2773370959 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 139) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f8ff8012070 con 0x55f73a47f500 2023-11-28T18:32:20.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9011851000 10 client.9348 fetch_fsmap finished waiting for FSMap version 139 2023-11-28T18:32:20.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9011851000 10 client.9348 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T18:32:20.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9011851000 20 client.9348 populate_metadata read hostname 'smithi105' 2023-11-28T18:32:20.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9011851000 1 --2- 172.21.15.105:0/2773370959 >> [v2:172.21.15.148:6833/3706038483,v1:172.21.15.148:6835/3706038483] conn(0x55f73a486970 0x55f73a4ddff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9011851000 4 client.9348 mds_command: new command op to 9123 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:32:20.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9011851000 1 -- 172.21.15.105:0/2773370959 --> [v2:172.21.15.148:6833/3706038483,v1:172.21.15.148:6835/3706038483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f73a296b50 con 0x55f73a486970 2023-11-28T18:32:20.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f90077fe640 1 --2- 172.21.15.105:0/2773370959 >> [v2:172.21.15.148:6833/3706038483,v1:172.21.15.148:6835/3706038483] conn(0x55f73a486970 0x55f73a4ddff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f90077fe640 1 --2- 172.21.15.105:0/2773370959 >> [v2:172.21.15.148:6833/3706038483,v1:172.21.15.148:6835/3706038483] conn(0x55f73a486970 0x55f73a4ddff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dc1d7a92b64219d server_cookie=9a707d2cb4949ff8 in_seq=0 out_seq=0 2023-11-28T18:32:20.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f8fe6ffd640 10 client.9348 ms_handle_connect on v2:172.21.15.148:6833/3706038483 2023-11-28T18:32:20.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f8fe6ffd640 1 -- 172.21.15.105:0/2773370959 <== mds.1 v2:172.21.15.148:6833/3706038483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f73a296b50 con 0x55f73a486970 2023-11-28T18:32:20.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f8fe6ffd640 10 client.9348 handle_command_reply: tid=0 2023-11-28T18:32:20.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9011851000 1 client.9348 shutdown 2023-11-28T18:32:20.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9011851000 1 -- 172.21.15.105:0/2773370959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a47f500 msgr2=0x55f73a4854e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9011851000 1 --2- 172.21.15.105:0/2773370959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a47f500 0x55f73a4854e0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f8ff80013a0 tx=0x7f8ff8001130 comp rx=0 tx=0).stop 2023-11-28T18:32:20.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9011851000 1 -- 172.21.15.105:0/2773370959 shutdown_connections 2023-11-28T18:32:20.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9011851000 1 --2- 172.21.15.105:0/2773370959 >> [v2:172.21.15.148:6833/3706038483,v1:172.21.15.148:6835/3706038483] conn(0x55f73a486970 0x55f73a4ddff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9011851000 1 --2- 172.21.15.105:0/2773370959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f73a4cc650 0x55f73a485f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9011851000 1 --2- 172.21.15.105:0/2773370959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f73a4cbd30 0x55f73a485a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f8fe6ffd640 0 client.9348 ms_handle_reset on v2:172.21.15.148:6833/3706038483 2023-11-28T18:32:20.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9011851000 1 --2- 172.21.15.105:0/2773370959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a47f500 0x55f73a4854e0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9011851000 1 -- 172.21.15.105:0/2773370959 >> 172.21.15.105:0/2773370959 conn(0x55f73a4ac8c0 msgr2=0x55f73a47e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:20.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9011851000 1 -- 172.21.15.105:0/2773370959 shutdown_connections 2023-11-28T18:32:20.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.232+0000 7f9011851000 1 -- 172.21.15.105:0/2773370959 wait complete. 2023-11-28T18:32:20.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.236+0000 7f9011851000 20 client.9348 trim_cache size 0 max 16384 2023-11-28T18:32:20.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.248+0000 7f9011851000 1 Processor -- start 2023-11-28T18:32:20.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.248+0000 7f9011851000 1 -- start start 2023-11-28T18:32:20.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.248+0000 7f9011851000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f73a47f500 0x55f73a4ab3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.248+0000 7f9011851000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f73a4cbd30 0x55f73a4ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9011851000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a4cc650 0x55f73a4ddff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9011851000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f739f8a940 con 0x55f73a4cc650 2023-11-28T18:32:20.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9011851000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73a479430 con 0x55f73a4cbd30 2023-11-28T18:32:20.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9011851000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f73a288a10 con 0x55f73a47f500 2023-11-28T18:32:20.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f900ca87640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a4cc650 0x55f73a4ddff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f900ca87640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a4cc650 0x55f73a4ddff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50684/0 (socket says 172.21.15.105:50684) 2023-11-28T18:32:20.251 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f900ca87640 1 -- 172.21.15.105:0/1663714264 learned_addr learned my addr 172.21.15.105:0/1663714264 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:20.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9007fff640 1 --2- 172.21.15.105:0/1663714264 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f73a47f500 0x55f73a4ab3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f90077fe640 1 --2- 172.21.15.105:0/1663714264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f73a4cbd30 0x55f73a4ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f900ca87640 1 -- 172.21.15.105:0/1663714264 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f73a47f500 msgr2=0x55f73a4ab3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f900ca87640 1 --2- 172.21.15.105:0/1663714264 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f73a47f500 0x55f73a4ab3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f900ca87640 1 -- 172.21.15.105:0/1663714264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f73a4cbd30 msgr2=0x55f73a4ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f900ca87640 1 --2- 172.21.15.105:0/1663714264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f73a4cbd30 0x55f73a4ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f900ca87640 1 -- 172.21.15.105:0/1663714264 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f73a47f370 con 0x55f73a4cc650 2023-11-28T18:32:20.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f90077fe640 1 --2- 172.21.15.105:0/1663714264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f73a4cbd30 0x55f73a4ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:32:20.252 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f900ca87640 1 --2- 172.21.15.105:0/1663714264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a4cc650 0x55f73a4ddff0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f9000009e50 tx=0x7f90000a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55beccb66be9eece server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f8feeffd640 1 -- 172.21.15.105:0/1663714264 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90000a5a90 con 0x55f73a4cc650 2023-11-28T18:32:20.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f8feeffd640 1 -- 172.21.15.105:0/1663714264 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90000a5bf0 con 0x55f73a4cc650 2023-11-28T18:32:20.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f8feeffd640 1 -- 172.21.15.105:0/1663714264 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90000c0400 con 0x55f73a4cc650 2023-11-28T18:32:20.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9011851000 1 -- 172.21.15.105:0/1663714264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a4cc650 msgr2=0x55f73a4ddff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9011851000 1 --2- 172.21.15.105:0/1663714264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a4cc650 0x55f73a4ddff0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f9000009e50 tx=0x7f90000a7070 comp rx=0 tx=0).stop 2023-11-28T18:32:20.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9011851000 1 -- 172.21.15.105:0/1663714264 shutdown_connections 2023-11-28T18:32:20.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9011851000 1 --2- 172.21.15.105:0/1663714264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a4cc650 0x55f73a4ddff0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.255 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9011851000 1 --2- 172.21.15.105:0/1663714264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f73a4cbd30 0x55f73a4ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9011851000 1 --2- 172.21.15.105:0/1663714264 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f73a47f500 0x55f73a4ab3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9011851000 1 -- 172.21.15.105:0/1663714264 >> 172.21.15.105:0/1663714264 conn(0x55f73a4ac8c0 msgr2=0x55f73a47d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:20.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9011851000 1 -- 172.21.15.105:0/1663714264 shutdown_connections 2023-11-28T18:32:20.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9011851000 1 -- 172.21.15.105:0/1663714264 wait complete. 2023-11-28T18:32:20.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9011851000 1 -- start start 2023-11-28T18:32:20.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9011851000 1 Processor -- start 2023-11-28T18:32:20.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9011851000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f73a47f500 0x55f73a52c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9011851000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f73a4cbd30 0x55f73a52cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9011851000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a4cc650 0x55f73a4e7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9011851000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73a2cefd0 con 0x55f73a4cc650 2023-11-28T18:32:20.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9011851000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f739fca1b0 con 0x55f73a4cbd30 2023-11-28T18:32:20.256 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9011851000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f73a29b6c0 con 0x55f73a47f500 2023-11-28T18:32:20.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f9011851000 10 client.0 _ll_register_callbacks cb 0x55f73a348060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:32:20.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.252+0000 7f900ca87640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a4cc650 0x55f73a4e7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f90077fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f73a4cbd30 0x55f73a52cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f900ca87640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a4cc650 0x55f73a4e7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50686/0 (socket says 172.21.15.105:50686) 2023-11-28T18:32:20.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f900ca87640 1 -- 172.21.15.105:0/1957899638 learned_addr learned my addr 172.21.15.105:0/1957899638 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:20.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9007fff640 1 --2- 172.21.15.105:0/1957899638 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f73a47f500 0x55f73a52c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f900ca87640 1 -- 172.21.15.105:0/1957899638 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f73a47f500 msgr2=0x55f73a52c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f900ca87640 1 --2- 172.21.15.105:0/1957899638 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f73a47f500 0x55f73a52c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f900ca87640 1 -- 172.21.15.105:0/1957899638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f73a4cbd30 msgr2=0x55f73a52cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f900ca87640 1 --2- 172.21.15.105:0/1957899638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f73a4cbd30 0x55f73a52cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f900ca87640 1 -- 172.21.15.105:0/1957899638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90080720d0 con 0x55f73a4cc650 2023-11-28T18:32:20.257 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f900ca87640 1 --2- 172.21.15.105:0/1957899638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a4cc650 0x55f73a4e7a70 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f900001c450 tx=0x7f900001b220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447e616a7862f524 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f8fe6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:32:20.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f8fe6ffd640 1 -- 172.21.15.105:0/1957899638 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90000cc070 con 0x55f73a4cc650 2023-11-28T18:32:20.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f8fe6ffd640 1 -- 172.21.15.105:0/1957899638 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f90000bb070 con 0x55f73a4cc650 2023-11-28T18:32:20.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9011851000 1 -- 172.21.15.105:0/1957899638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f73a2d0110 con 0x55f73a4cc650 2023-11-28T18:32:20.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f8fe6ffd640 1 -- 172.21.15.105:0/1957899638 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=139) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90000091d0 con 0x55f73a4cc650 2023-11-28T18:32:20.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9011851000 10 client.9354 fetch_fsmap learned FSMap version 139 2023-11-28T18:32:20.258 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9011851000 1 -- 172.21.15.105:0/1957899638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=139,monmap=2+}) v3 -- 0x55f73a278490 con 0x55f73a4cc650 2023-11-28T18:32:20.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f8fe6ffd640 1 -- 172.21.15.105:0/1957899638 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f73a2d0110 con 0x55f73a4cc650 2023-11-28T18:32:20.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f8fe6ffd640 1 -- 172.21.15.105:0/1957899638 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 139) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f90000cf020 con 0x55f73a4cc650 2023-11-28T18:32:20.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9011851000 10 client.9354 fetch_fsmap finished waiting for FSMap version 139 2023-11-28T18:32:20.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9011851000 10 client.9354 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T18:32:20.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9011851000 20 client.9354 populate_metadata read hostname 'smithi105' 2023-11-28T18:32:20.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9011851000 1 --2- 172.21.15.105:0/1957899638 >> [v2:172.21.15.148:6833/3706038483,v1:172.21.15.148:6835/3706038483] conn(0x55f73a4e91d0 0x55f73a535680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9011851000 4 client.9354 mds_command: new command op to 9123 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:32:20.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9011851000 1 -- 172.21.15.105:0/1957899638 --> [v2:172.21.15.148:6833/3706038483,v1:172.21.15.148:6835/3706038483] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f73a2fe650 con 0x55f73a4e91d0 2023-11-28T18:32:20.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9007fff640 1 --2- 172.21.15.105:0/1957899638 >> [v2:172.21.15.148:6833/3706038483,v1:172.21.15.148:6835/3706038483] conn(0x55f73a4e91d0 0x55f73a535680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9007fff640 1 --2- 172.21.15.105:0/1957899638 >> [v2:172.21.15.148:6833/3706038483,v1:172.21.15.148:6835/3706038483] conn(0x55f73a4e91d0 0x55f73a535680 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5315ca652adf8c04 server_cookie=9fd42f5cc794b75f in_seq=0 out_seq=0 2023-11-28T18:32:20.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f8fe6ffd640 10 client.9354 ms_handle_connect on v2:172.21.15.148:6833/3706038483 2023-11-28T18:32:20.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f8fe6ffd640 1 -- 172.21.15.105:0/1957899638 <== mds.1 v2:172.21.15.148:6833/3706038483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55f73a2fe650 con 0x55f73a4e91d0 2023-11-28T18:32:20.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f8fe6ffd640 10 client.9354 handle_command_reply: tid=0 2023-11-28T18:32:20.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9011851000 1 client.9354 shutdown 2023-11-28T18:32:20.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9011851000 1 -- 172.21.15.105:0/1957899638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a4cc650 msgr2=0x55f73a4e7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9011851000 1 --2- 172.21.15.105:0/1957899638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a4cc650 0x55f73a4e7a70 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f900001c450 tx=0x7f900001b220 comp rx=0 tx=0).stop 2023-11-28T18:32:20.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9011851000 1 -- 172.21.15.105:0/1957899638 shutdown_connections 2023-11-28T18:32:20.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9011851000 1 --2- 172.21.15.105:0/1957899638 >> [v2:172.21.15.148:6833/3706038483,v1:172.21.15.148:6835/3706038483] conn(0x55f73a4e91d0 0x55f73a535680 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9011851000 1 --2- 172.21.15.105:0/1957899638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f73a4cc650 0x55f73a4e7a70 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9011851000 1 --2- 172.21.15.105:0/1957899638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f73a4cbd30 0x55f73a52cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f8fe6ffd640 0 client.9354 ms_handle_reset on v2:172.21.15.148:6833/3706038483 2023-11-28T18:32:20.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9011851000 1 --2- 172.21.15.105:0/1957899638 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f73a47f500 0x55f73a52c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9011851000 1 -- 172.21.15.105:0/1957899638 >> 172.21.15.105:0/1957899638 conn(0x55f73a4ac8c0 msgr2=0x55f73a47dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:20.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9011851000 1 -- 172.21.15.105:0/1957899638 shutdown_connections 2023-11-28T18:32:20.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9011851000 1 -- 172.21.15.105:0/1957899638 wait complete. 2023-11-28T18:32:20.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.256+0000 7f9011851000 20 client.9354 trim_cache size 0 max 16384 2023-11-28T18:32:20.262 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:32:20.263 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:32:20.263 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 1, 2023-11-28T18:32:20.263 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 9123, 2023-11-28T18:32:20.263 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:32:20.263 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:32:20.263 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:32:20.263 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 2.9900930140000002, 2023-11-28T18:32:20.263 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 139, 2023-11-28T18:32:20.263 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 129, 2023-11-28T18:32:20.263 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 129, 2023-11-28T18:32:20.263 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 26.627768838000001 2023-11-28T18:32:20.263 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:32:20.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.260+0000 7f900e4ea640 1 -- 172.21.15.105:0/3847681399 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8ff40668d0 msgr2=0x7f8ff4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.260+0000 7f900e4ea640 1 --2- 172.21.15.105:0/3847681399 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8ff40668d0 0x7f8ff4068d90 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8007fd0 tx=0x7f8ff8009040 comp rx=0 tx=0).stop 2023-11-28T18:32:20.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.260+0000 7f900e4ea640 1 -- 172.21.15.105:0/3847681399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900815b030 msgr2=0x7f90081f5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.264 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.260+0000 7f900e4ea640 1 --2- 172.21.15.105:0/3847681399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900815b030 0x7f90081f5d00 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00e930 tx=0x7f8ffc00edf0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.260+0000 7f900e4ea640 1 -- 172.21.15.105:0/3847681399 shutdown_connections 2023-11-28T18:32:20.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.260+0000 7f900e4ea640 1 --2- 172.21.15.105:0/3847681399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f900809b560 0x7f90081f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.260+0000 7f900e4ea640 1 --2- 172.21.15.105:0/3847681399 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f8ff40668d0 0x7f8ff4068d90 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.260+0000 7f900e4ea640 1 --2- 172.21.15.105:0/3847681399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900815b030 0x7f90081f5d00 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.260+0000 7f900e4ea640 1 --2- 172.21.15.105:0/3847681399 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90080b4290 0x7f9008099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.260+0000 7f900e4ea640 1 -- 172.21.15.105:0/3847681399 >> 172.21.15.105:0/3847681399 conn(0x7f90080ceb90 msgr2=0x7f90080cc760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:20.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.260+0000 7f900e4ea640 1 -- 172.21.15.105:0/3847681399 shutdown_connections 2023-11-28T18:32:20.265 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.260+0000 7f900e4ea640 1 -- 172.21.15.105:0/3847681399 wait complete. 2023-11-28T18:32:20.277 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T18:32:20.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.416+0000 7f3138e79640 1 Processor -- start 2023-11-28T18:32:20.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.416+0000 7f3138e79640 1 -- start start 2023-11-28T18:32:20.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.420+0000 7f3138e79640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f31340cc760 0x7f3134164d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.420+0000 7f3138e79640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31340ccc10 con 0x7f31340caf60 2023-11-28T18:32:20.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.420+0000 7f3138e79640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f313406bf50 con 0x7f3134159f40 2023-11-28T18:32:20.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.420+0000 7f3138e79640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31340cb3c0 con 0x7f31340cc760 2023-11-28T18:32:20.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.420+0000 7f312bfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f31340caf60 0x7f31340ad850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37544/0 (socket says 172.21.15.105:37544) 2023-11-28T18:32:20.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.420+0000 7f312bfff640 1 -- 172.21.15.105:0/1025231586 learned_addr learned my addr 172.21.15.105:0/1025231586 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:32:20.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.420+0000 7f3132d9d640 1 --2- 172.21.15.105:0/1025231586 >> v2:172.21.15.105:3300/0 conn(0x7f31340cc760 0x7f3134164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.420+0000 7f3131d9b640 1 -- 172.21.15.105:0/1025231586 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3822676378 0 0) 0x7f31340ccc10 con 0x7f31340caf60 2023-11-28T18:32:20.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.420+0000 7f3131d9b640 1 -- 172.21.15.105:0/1025231586 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3120003530 con 0x7f31340caf60 2023-11-28T18:32:20.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.420+0000 7f3132d9d640 1 -- 172.21.15.105:0/1025231586 >> v1:172.21.15.148:6789/0 conn(0x7f3134159f40 legacy=0x7f31340ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.420+0000 7f3132d9d640 1 -- 172.21.15.105:0/1025231586 >> v1:172.21.15.105:6789/0 conn(0x7f31340caf60 legacy=0x7f31340ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.420+0000 7f3132d9d640 1 -- 172.21.15.105:0/1025231586 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313415a320 con 0x7f31340cc760 2023-11-28T18:32:20.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.424+0000 7f3132d9d640 1 --2- 172.21.15.105:0/1025231586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31340cc760 0x7f3134164d90 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f312c009d90 tx=0x7f312c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7576a4deaaab8632 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.424+0000 7f3131d9b640 1 -- 172.21.15.105:0/1025231586 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c0a5950 con 0x7f31340cc760 2023-11-28T18:32:20.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.424+0000 7f3131d9b640 1 -- 172.21.15.105:0/1025231586 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312c0a5ab0 con 0x7f31340cc760 2023-11-28T18:32:20.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.424+0000 7f3131d9b640 1 -- 172.21.15.105:0/1025231586 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c0a5e20 con 0x7f31340cc760 2023-11-28T18:32:20.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.424+0000 7f3138e79640 1 -- 172.21.15.105:0/1025231586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31340cc760 msgr2=0x7f3134164d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.424+0000 7f3138e79640 1 --2- 172.21.15.105:0/1025231586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31340cc760 0x7f3134164d90 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f312c009d90 tx=0x7f312c01c890 comp rx=0 tx=0).stop 2023-11-28T18:32:20.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.424+0000 7f3138e79640 1 -- 172.21.15.105:0/1025231586 shutdown_connections 2023-11-28T18:32:20.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.424+0000 7f3138e79640 1 --2- 172.21.15.105:0/1025231586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31340cc760 0x7f3134164d90 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.424+0000 7f3138e79640 1 -- 172.21.15.105:0/1025231586 >> 172.21.15.105:0/1025231586 conn(0x7f31340ceb90 msgr2=0x7f31340cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:20.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.424+0000 7f3138e79640 1 -- 172.21.15.105:0/1025231586 shutdown_connections 2023-11-28T18:32:20.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.424+0000 7f3138e79640 1 -- 172.21.15.105:0/1025231586 wait complete. 2023-11-28T18:32:20.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.424+0000 7f3138e79640 1 Processor -- start 2023-11-28T18:32:20.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.424+0000 7f3138e79640 1 -- start start 2023-11-28T18:32:20.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.428+0000 7f3138e79640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31340caf60 0x7f3134143d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.428+0000 7f3138e79640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31340cc760 0x7f313414b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.428+0000 7f3138e79640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3134159f40 0x7f313414aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.428+0000 7f3138e79640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3134149150 con 0x7f31340caf60 2023-11-28T18:32:20.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.428+0000 7f3138e79640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3134161f30 con 0x7f31340cc760 2023-11-28T18:32:20.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.428+0000 7f3138e79640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3134162210 con 0x7f3134159f40 2023-11-28T18:32:20.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.428+0000 7f313259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31340caf60 0x7f3134143d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.428+0000 7f313259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31340caf60 0x7f3134143d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50696/0 (socket says 172.21.15.105:50696) 2023-11-28T18:32:20.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.428+0000 7f313259c640 1 -- 172.21.15.105:0/1695241263 learned_addr learned my addr 172.21.15.105:0/1695241263 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:20.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.428+0000 7f312bfff640 1 --2- 172.21.15.105:0/1695241263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31340cc760 0x7f313414b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.428+0000 7f3132d9d640 1 --2- 172.21.15.105:0/1695241263 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3134159f40 0x7f313414aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.428+0000 7f313259c640 1 -- 172.21.15.105:0/1695241263 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3134159f40 msgr2=0x7f313414aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.428+0000 7f313259c640 1 --2- 172.21.15.105:0/1695241263 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3134159f40 0x7f313414aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.428+0000 7f313259c640 1 -- 172.21.15.105:0/1695241263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31340cc760 msgr2=0x7f313414b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.428+0000 7f313259c640 1 --2- 172.21.15.105:0/1695241263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31340cc760 0x7f313414b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.428+0000 7f313259c640 1 -- 172.21.15.105:0/1695241263 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c009a70 con 0x7f31340caf60 2023-11-28T18:32:20.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.428+0000 7f313259c640 1 --2- 172.21.15.105:0/1695241263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31340caf60 0x7f3134143d10 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f312400c500 tx=0x7f312400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.428+0000 7f312affd640 1 -- 172.21.15.105:0/1695241263 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3124005220 con 0x7f31340caf60 2023-11-28T18:32:20.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.428+0000 7f312affd640 1 -- 172.21.15.105:0/1695241263 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3124005380 con 0x7f31340caf60 2023-11-28T18:32:20.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.428+0000 7f3138e79640 1 -- 172.21.15.105:0/1695241263 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31341624f0 con 0x7f31340caf60 2023-11-28T18:32:20.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.428+0000 7f312affd640 1 -- 172.21.15.105:0/1695241263 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3124007270 con 0x7f31340caf60 2023-11-28T18:32:20.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.428+0000 7f3138e79640 1 -- 172.21.15.105:0/1695241263 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3134162ab0 con 0x7f31340caf60 2023-11-28T18:32:20.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f312affd640 1 -- 172.21.15.105:0/1695241263 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f31240073d0 con 0x7f31340caf60 2023-11-28T18:32:20.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f312affd640 1 --2- 172.21.15.105:0/1695241263 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3114066900 0x7f3114068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f312affd640 1 -- 172.21.15.105:0/1695241263 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3124083d80 con 0x7f31340caf60 2023-11-28T18:32:20.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f312bfff640 1 --2- 172.21.15.105:0/1695241263 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3114066900 0x7f3114068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f313c1e4000 1 Processor -- start 2023-11-28T18:32:20.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f313c1e4000 1 -- start start 2023-11-28T18:32:20.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f312bfff640 1 --2- 172.21.15.105:0/1695241263 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3114066900 0x7f3114068dc0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3134150d40 tx=0x7f311c007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f313c1e4000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c46c895d30 0x55c46c896110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f313c1e4000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c46c896650 0x55c46c848fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f313c1e4000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c849500 0x55c46c8498e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f313c1e4000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46c712110 con 0x55c46c849500 2023-11-28T18:32:20.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f313c1e4000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46c6a29a0 con 0x55c46c896650 2023-11-28T18:32:20.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f313c1e4000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c46c69fed0 con 0x55c46c895d30 2023-11-28T18:32:20.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f3132d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c849500 0x55c46c8498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f3132d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c849500 0x55c46c8498e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50710/0 (socket says 172.21.15.105:50710) 2023-11-28T18:32:20.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f3132d9d640 1 -- 172.21.15.105:0/4223572069 learned_addr learned my addr 172.21.15.105:0/4223572069 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:20.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f313259c640 1 --2- 172.21.15.105:0/4223572069 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c46c895d30 0x55c46c896110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f312bfff640 1 --2- 172.21.15.105:0/4223572069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c46c896650 0x55c46c848fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f3132d9d640 1 -- 172.21.15.105:0/4223572069 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c46c895d30 msgr2=0x55c46c896110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f3132d9d640 1 --2- 172.21.15.105:0/4223572069 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c46c895d30 0x55c46c896110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f3132d9d640 1 -- 172.21.15.105:0/4223572069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c46c896650 msgr2=0x55c46c848fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f3132d9d640 1 --2- 172.21.15.105:0/4223572069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c46c896650 0x55c46c848fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f3132d9d640 1 -- 172.21.15.105:0/4223572069 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c46c393890 con 0x55c46c849500 2023-11-28T18:32:20.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.432+0000 7f3132d9d640 1 --2- 172.21.15.105:0/4223572069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c849500 0x55c46c8498e0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f312c0b0040 tx=0x7f312c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1929eee7bb602287 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.436 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f3113fff640 1 -- 172.21.15.105:0/4223572069 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c0ad640 con 0x55c46c849500 2023-11-28T18:32:20.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f3113fff640 1 -- 172.21.15.105:0/4223572069 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312c0ad7a0 con 0x55c46c849500 2023-11-28T18:32:20.437 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f3113fff640 1 -- 172.21.15.105:0/4223572069 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c0adb50 con 0x55c46c849500 2023-11-28T18:32:20.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f313c1e4000 1 -- 172.21.15.105:0/4223572069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c849500 msgr2=0x55c46c8498e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f313c1e4000 1 --2- 172.21.15.105:0/4223572069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c849500 0x55c46c8498e0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f312c0b0040 tx=0x7f312c004ac0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f313c1e4000 1 -- 172.21.15.105:0/4223572069 shutdown_connections 2023-11-28T18:32:20.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f313c1e4000 1 --2- 172.21.15.105:0/4223572069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c849500 0x55c46c8498e0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f313c1e4000 1 --2- 172.21.15.105:0/4223572069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c46c896650 0x55c46c848fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f313c1e4000 1 --2- 172.21.15.105:0/4223572069 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c46c895d30 0x55c46c896110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f313c1e4000 1 -- 172.21.15.105:0/4223572069 >> 172.21.15.105:0/4223572069 conn(0x55c46c8768c0 msgr2=0x55c46c878cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:20.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f313c1e4000 1 -- 172.21.15.105:0/4223572069 shutdown_connections 2023-11-28T18:32:20.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f313c1e4000 1 -- 172.21.15.105:0/4223572069 wait complete. 2023-11-28T18:32:20.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f313c1e4000 1 -- start start 2023-11-28T18:32:20.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f313c1e4000 1 Processor -- start 2023-11-28T18:32:20.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f313c1e4000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c46c849500 0x55c46c84f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.443 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f313c1e4000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c895d30 0x55c46c84fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.443 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f313c1e4000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c46c896650 0x55c46c84ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.443 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f313c1e4000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46c69b7c0 con 0x55c46c895d30 2023-11-28T18:32:20.443 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f313c1e4000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46c653bf0 con 0x55c46c849500 2023-11-28T18:32:20.443 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f313c1e4000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c46c659a90 con 0x55c46c896650 2023-11-28T18:32:20.443 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f313c1e4000 10 client.0 _ll_register_callbacks cb 0x55c46c836e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:32:20.443 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f312bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c895d30 0x55c46c84fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.443 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f312bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c895d30 0x55c46c84fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50722/0 (socket says 172.21.15.105:50722) 2023-11-28T18:32:20.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f312bfff640 1 -- 172.21.15.105:0/3744936923 learned_addr learned my addr 172.21.15.105:0/3744936923 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:20.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f3132d9d640 1 --2- 172.21.15.105:0/3744936923 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c46c896650 0x55c46c84ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f313259c640 1 --2- 172.21.15.105:0/3744936923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c46c849500 0x55c46c84f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f312bfff640 1 -- 172.21.15.105:0/3744936923 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c46c896650 msgr2=0x55c46c84ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f312bfff640 1 --2- 172.21.15.105:0/3744936923 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c46c896650 0x55c46c84ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f312bfff640 1 -- 172.21.15.105:0/3744936923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c46c849500 msgr2=0x55c46c84f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f312bfff640 1 --2- 172.21.15.105:0/3744936923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c46c849500 0x55c46c84f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f312bfff640 1 -- 172.21.15.105:0/3744936923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f313415a320 con 0x55c46c895d30 2023-11-28T18:32:20.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.436+0000 7f313259c640 1 --2- 172.21.15.105:0/3744936923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c46c849500 0x55c46c84f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:32:20.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f312bfff640 1 --2- 172.21.15.105:0/3744936923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c895d30 0x55c46c84fa60 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f311c0065b0 tx=0x7f311c001160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76c23e31ba2f613 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f31117fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:32:20.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f31117fa640 1 -- 172.21.15.105:0/3744936923 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c026070 con 0x55c46c895d30 2023-11-28T18:32:20.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f31117fa640 1 -- 172.21.15.105:0/3744936923 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f311c01b070 con 0x55c46c895d30 2023-11-28T18:32:20.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313c1e4000 1 -- 172.21.15.105:0/3744936923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c46c65d470 con 0x55c46c895d30 2023-11-28T18:32:20.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f31117fa640 1 -- 172.21.15.105:0/3744936923 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=139) v2 ==== 24+0+0 (secure 0 0 0) 0x7f311c005e10 con 0x55c46c895d30 2023-11-28T18:32:20.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313c1e4000 10 client.9369 fetch_fsmap learned FSMap version 139 2023-11-28T18:32:20.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313c1e4000 1 -- 172.21.15.105:0/3744936923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=139,monmap=2+}) v3 -- 0x55c46c65c190 con 0x55c46c895d30 2023-11-28T18:32:20.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f31117fa640 1 -- 172.21.15.105:0/3744936923 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c030020 con 0x55c46c895d30 2023-11-28T18:32:20.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f31117fa640 1 -- 172.21.15.105:0/3744936923 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 139) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f311c029020 con 0x55c46c895d30 2023-11-28T18:32:20.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313c1e4000 10 client.9369 fetch_fsmap finished waiting for FSMap version 139 2023-11-28T18:32:20.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313c1e4000 10 client.9369 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:32:20.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313c1e4000 20 client.9369 populate_metadata read hostname 'smithi105' 2023-11-28T18:32:20.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313c1e4000 1 --2- 172.21.15.105:0/3744936923 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55c46c8509b0 0x55c46c8a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313c1e4000 4 client.9369 mds_command: new command op to 9126 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:32:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313c1e4000 1 -- 172.21.15.105:0/3744936923 --> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c46c660b50 con 0x55c46c8509b0 2023-11-28T18:32:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313259c640 1 --2- 172.21.15.105:0/3744936923 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55c46c8509b0 0x55c46c8a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313259c640 1 --2- 172.21.15.105:0/3744936923 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55c46c8509b0 0x55c46c8a7ff0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50c036293347fc47 server_cookie=e33042ec5e35d764 in_seq=0 out_seq=0 2023-11-28T18:32:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f31117fa640 10 client.9369 ms_handle_connect on v2:172.21.15.105:6836/1978722810 2023-11-28T18:32:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f31117fa640 1 -- 172.21.15.105:0/3744936923 <== mds.0 v2:172.21.15.105:6836/1978722810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c46c660b50 con 0x55c46c8509b0 2023-11-28T18:32:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f31117fa640 10 client.9369 handle_command_reply: tid=0 2023-11-28T18:32:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313c1e4000 1 client.9369 shutdown 2023-11-28T18:32:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313c1e4000 1 -- 172.21.15.105:0/3744936923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c895d30 msgr2=0x55c46c84fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313c1e4000 1 --2- 172.21.15.105:0/3744936923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c895d30 0x55c46c84fa60 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f311c0065b0 tx=0x7f311c001160 comp rx=0 tx=0).stop 2023-11-28T18:32:20.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313c1e4000 1 -- 172.21.15.105:0/3744936923 shutdown_connections 2023-11-28T18:32:20.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313c1e4000 1 --2- 172.21.15.105:0/3744936923 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55c46c8509b0 0x55c46c8a7ff0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313c1e4000 1 --2- 172.21.15.105:0/3744936923 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c46c896650 0x55c46c84ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313c1e4000 1 --2- 172.21.15.105:0/3744936923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c895d30 0x55c46c84fa60 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313c1e4000 1 --2- 172.21.15.105:0/3744936923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c46c849500 0x55c46c84f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f31117fa640 0 client.9369 ms_handle_reset on v2:172.21.15.105:6836/1978722810 2023-11-28T18:32:20.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313c1e4000 1 -- 172.21.15.105:0/3744936923 >> 172.21.15.105:0/3744936923 conn(0x55c46c8768c0 msgr2=0x55c46c8486b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:20.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313c1e4000 1 -- 172.21.15.105:0/3744936923 shutdown_connections 2023-11-28T18:32:20.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313c1e4000 1 -- 172.21.15.105:0/3744936923 wait complete. 2023-11-28T18:32:20.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.440+0000 7f313c1e4000 20 client.9369 trim_cache size 0 max 16384 2023-11-28T18:32:20.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.456+0000 7f313c1e4000 1 Processor -- start 2023-11-28T18:32:20.456 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.456+0000 7f313c1e4000 1 -- start start 2023-11-28T18:32:20.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.456+0000 7f313c1e4000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c849500 0x55c46c8aa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.456+0000 7f313c1e4000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c46c895d30 0x55c46c8753a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.456+0000 7f313c1e4000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c46c896650 0x55c46c8758e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.456+0000 7f313c1e4000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46c354940 con 0x55c46c849500 2023-11-28T18:32:20.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.456+0000 7f313c1e4000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46c65a8f0 con 0x55c46c895d30 2023-11-28T18:32:20.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.456+0000 7f313c1e4000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c46c84ce10 con 0x55c46c896650 2023-11-28T18:32:20.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.456+0000 7f313259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c849500 0x55c46c8aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.456+0000 7f313259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c849500 0x55c46c8aa430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50734/0 (socket says 172.21.15.105:50734) 2023-11-28T18:32:20.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.456+0000 7f313259c640 1 -- 172.21.15.105:0/4229279017 learned_addr learned my addr 172.21.15.105:0/4229279017 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:20.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.456+0000 7f3132d9d640 1 --2- 172.21.15.105:0/4229279017 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c46c896650 0x55c46c8758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.456+0000 7f312bfff640 1 --2- 172.21.15.105:0/4229279017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c46c895d30 0x55c46c8753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.456+0000 7f313259c640 1 -- 172.21.15.105:0/4229279017 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c46c896650 msgr2=0x55c46c8758e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.456+0000 7f313259c640 1 --2- 172.21.15.105:0/4229279017 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c46c896650 0x55c46c8758e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.459 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.456+0000 7f313259c640 1 -- 172.21.15.105:0/4229279017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c46c895d30 msgr2=0x55c46c8753a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.456+0000 7f313259c640 1 --2- 172.21.15.105:0/4229279017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c46c895d30 0x55c46c8753a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.456+0000 7f313259c640 1 -- 172.21.15.105:0/4229279017 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c46c642490 con 0x55c46c849500 2023-11-28T18:32:20.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.456+0000 7f312bfff640 1 --2- 172.21.15.105:0/4229279017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c46c895d30 0x55c46c8753a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:32:20.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.456+0000 7f313259c640 1 --2- 172.21.15.105:0/4229279017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c849500 0x55c46c8aa430 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f3124000fc0 tx=0x7f3124001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efb4e74edcceb07 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f3128ff9640 1 -- 172.21.15.105:0/4229279017 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31240821e0 con 0x55c46c849500 2023-11-28T18:32:20.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f3128ff9640 1 -- 172.21.15.105:0/4229279017 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3124052f00 con 0x55c46c849500 2023-11-28T18:32:20.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f3128ff9640 1 -- 172.21.15.105:0/4229279017 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3124051270 con 0x55c46c849500 2023-11-28T18:32:20.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f313c1e4000 1 -- 172.21.15.105:0/4229279017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c849500 msgr2=0x55c46c8aa430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f313c1e4000 1 --2- 172.21.15.105:0/4229279017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c849500 0x55c46c8aa430 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f3124000fc0 tx=0x7f3124001130 comp rx=0 tx=0).stop 2023-11-28T18:32:20.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f313c1e4000 1 -- 172.21.15.105:0/4229279017 shutdown_connections 2023-11-28T18:32:20.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f313c1e4000 1 --2- 172.21.15.105:0/4229279017 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c46c896650 0x55c46c8758e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f313c1e4000 1 --2- 172.21.15.105:0/4229279017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c46c895d30 0x55c46c8753a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f313c1e4000 1 --2- 172.21.15.105:0/4229279017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c849500 0x55c46c8aa430 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f313c1e4000 1 -- 172.21.15.105:0/4229279017 >> 172.21.15.105:0/4229279017 conn(0x55c46c8768c0 msgr2=0x55c46c847720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:20.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f313c1e4000 1 -- 172.21.15.105:0/4229279017 shutdown_connections 2023-11-28T18:32:20.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f313c1e4000 1 -- 172.21.15.105:0/4229279017 wait complete. 2023-11-28T18:32:20.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f313c1e4000 1 -- start start 2023-11-28T18:32:20.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f313c1e4000 1 Processor -- start 2023-11-28T18:32:20.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f313c1e4000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c46c849500 0x55c46c8f6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f313c1e4000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c46c895d30 0x55c46c8f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f313c1e4000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c896650 0x55c46c8b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f313c1e4000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46c664910 con 0x55c46c896650 2023-11-28T18:32:20.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f313c1e4000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46c850fc0 con 0x55c46c849500 2023-11-28T18:32:20.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f313c1e4000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c46c6975c0 con 0x55c46c895d30 2023-11-28T18:32:20.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f313c1e4000 10 client.0 _ll_register_callbacks cb 0x55c46c712060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:32:20.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f3132d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c896650 0x55c46c8b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f3132d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c896650 0x55c46c8b1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50740/0 (socket says 172.21.15.105:50740) 2023-11-28T18:32:20.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f3132d9d640 1 -- 172.21.15.105:0/2632201951 learned_addr learned my addr 172.21.15.105:0/2632201951 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:20.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f313259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c46c849500 0x55c46c8f6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f312bfff640 1 --2- 172.21.15.105:0/2632201951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c46c895d30 0x55c46c8f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f3132d9d640 1 -- 172.21.15.105:0/2632201951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c46c895d30 msgr2=0x55c46c8f6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f3132d9d640 1 --2- 172.21.15.105:0/2632201951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c46c895d30 0x55c46c8f6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f3132d9d640 1 -- 172.21.15.105:0/2632201951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c46c849500 msgr2=0x55c46c8f6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f3132d9d640 1 --2- 172.21.15.105:0/2632201951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c46c849500 0x55c46c8f6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f3132d9d640 1 -- 172.21.15.105:0/2632201951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3134162ab0 con 0x55c46c896650 2023-11-28T18:32:20.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f3132d9d640 1 --2- 172.21.15.105:0/2632201951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c896650 0x55c46c8b1a70 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f312c0095c0 tx=0x7f312c01b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59adcd71dc998cd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.460+0000 7f31117fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:32:20.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f31117fa640 1 -- 172.21.15.105:0/2632201951 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c004240 con 0x55c46c896650 2023-11-28T18:32:20.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f31117fa640 1 -- 172.21.15.105:0/2632201951 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f312c0c9040 con 0x55c46c896650 2023-11-28T18:32:20.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313c1e4000 1 -- 172.21.15.105:0/2632201951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c46c69a110 con 0x55c46c896650 2023-11-28T18:32:20.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f31117fa640 1 -- 172.21.15.105:0/2632201951 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=139) v2 ==== 24+0+0 (secure 0 0 0) 0x7f312c0a58d0 con 0x55c46c896650 2023-11-28T18:32:20.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313c1e4000 10 client.9375 fetch_fsmap learned FSMap version 139 2023-11-28T18:32:20.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313c1e4000 1 -- 172.21.15.105:0/2632201951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=139,monmap=2+}) v3 -- 0x55c46c393cc0 con 0x55c46c896650 2023-11-28T18:32:20.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f31117fa640 1 -- 172.21.15.105:0/2632201951 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c46c69a110 con 0x55c46c896650 2023-11-28T18:32:20.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f31117fa640 1 -- 172.21.15.105:0/2632201951 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 139) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f312c0cd070 con 0x55c46c896650 2023-11-28T18:32:20.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313c1e4000 10 client.9375 fetch_fsmap finished waiting for FSMap version 139 2023-11-28T18:32:20.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313c1e4000 10 client.9375 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:32:20.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313c1e4000 20 client.9375 populate_metadata read hostname 'smithi105' 2023-11-28T18:32:20.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313c1e4000 1 --2- 172.21.15.105:0/2632201951 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55c46c8fd460 0x55c46c8ff850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313c1e4000 4 client.9375 mds_command: new command op to 9126 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:32:20.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313c1e4000 1 -- 172.21.15.105:0/2632201951 --> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c46c6c8650 con 0x55c46c8fd460 2023-11-28T18:32:20.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313259c640 1 --2- 172.21.15.105:0/2632201951 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55c46c8fd460 0x55c46c8ff850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313259c640 1 --2- 172.21.15.105:0/2632201951 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55c46c8fd460 0x55c46c8ff850 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0f549864b8a2926 server_cookie=3842115191f67310 in_seq=0 out_seq=0 2023-11-28T18:32:20.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f31117fa640 10 client.9375 ms_handle_connect on v2:172.21.15.105:6836/1978722810 2023-11-28T18:32:20.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f31117fa640 1 -- 172.21.15.105:0/2632201951 <== mds.0 v2:172.21.15.105:6836/1978722810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x55c46c6c8650 con 0x55c46c8fd460 2023-11-28T18:32:20.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f31117fa640 10 client.9375 handle_command_reply: tid=0 2023-11-28T18:32:20.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313c1e4000 1 client.9375 shutdown 2023-11-28T18:32:20.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313c1e4000 1 -- 172.21.15.105:0/2632201951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c896650 msgr2=0x55c46c8b1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313c1e4000 1 --2- 172.21.15.105:0/2632201951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c896650 0x55c46c8b1a70 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f312c0095c0 tx=0x7f312c01b0d0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313c1e4000 1 -- 172.21.15.105:0/2632201951 shutdown_connections 2023-11-28T18:32:20.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313c1e4000 1 --2- 172.21.15.105:0/2632201951 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55c46c8fd460 0x55c46c8ff850 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313c1e4000 1 --2- 172.21.15.105:0/2632201951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c46c896650 0x55c46c8b1a70 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313c1e4000 1 --2- 172.21.15.105:0/2632201951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c46c895d30 0x55c46c8f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313c1e4000 1 --2- 172.21.15.105:0/2632201951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c46c849500 0x55c46c8f6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f31117fa640 0 client.9375 ms_handle_reset on v2:172.21.15.105:6836/1978722810 2023-11-28T18:32:20.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313c1e4000 1 -- 172.21.15.105:0/2632201951 >> 172.21.15.105:0/2632201951 conn(0x55c46c8768c0 msgr2=0x55c46c847fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:20.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313c1e4000 1 -- 172.21.15.105:0/2632201951 shutdown_connections 2023-11-28T18:32:20.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313c1e4000 1 -- 172.21.15.105:0/2632201951 wait complete. 2023-11-28T18:32:20.470 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f313c1e4000 20 client.9375 trim_cache size 0 max 16384 2023-11-28T18:32:20.470 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:32:20.470 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:32:20.471 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2023-11-28T18:32:20.471 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 9126, 2023-11-28T18:32:20.471 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:32:20.471 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:32:20.471 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:32:20.471 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 21.000308833999998, 2023-11-28T18:32:20.471 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 139, 2023-11-28T18:32:20.471 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 129, 2023-11-28T18:32:20.471 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 129, 2023-11-28T18:32:20.471 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 26.830040549 2023-11-28T18:32:20.471 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:32:20.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f3138e79640 1 -- 172.21.15.105:0/1695241263 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3114066900 msgr2=0x7f3114068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f3138e79640 1 --2- 172.21.15.105:0/1695241263 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3114066900 0x7f3114068dc0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3134150d40 tx=0x7f311c007d40 comp rx=0 tx=0).stop 2023-11-28T18:32:20.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f3138e79640 1 -- 172.21.15.105:0/1695241263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31340caf60 msgr2=0x7f3134143d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f3138e79640 1 --2- 172.21.15.105:0/1695241263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31340caf60 0x7f3134143d10 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f312400c500 tx=0x7f312400c9c0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f3138e79640 1 -- 172.21.15.105:0/1695241263 shutdown_connections 2023-11-28T18:32:20.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f3138e79640 1 --2- 172.21.15.105:0/1695241263 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3134159f40 0x7f313414aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f3138e79640 1 --2- 172.21.15.105:0/1695241263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31340cc760 0x7f313414b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f3138e79640 1 --2- 172.21.15.105:0/1695241263 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3114066900 0x7f3114068dc0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f3138e79640 1 --2- 172.21.15.105:0/1695241263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31340caf60 0x7f3134143d10 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f3138e79640 1 -- 172.21.15.105:0/1695241263 >> 172.21.15.105:0/1695241263 conn(0x7f31340ceb90 msgr2=0x7f31340cdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:20.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f3138e79640 1 -- 172.21.15.105:0/1695241263 shutdown_connections 2023-11-28T18:32:20.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.464+0000 7f3138e79640 1 -- 172.21.15.105:0/1695241263 wait complete. 2023-11-28T18:32:20.484 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-11-28T18:32:20.626 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.628+0000 7f4d97c46640 1 Processor -- start 2023-11-28T18:32:20.627 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.628+0000 7f4d97c46640 1 -- start start 2023-11-28T18:32:20.628 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.628+0000 7f4d97c46640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4d90159f40 0x7f4d900caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.628+0000 7f4d97c46640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4d900cc760 0x7f4d900ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.628+0000 7f4d97c46640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d900ad680 0x7f4d9015da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.628+0000 7f4d97c46640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d9006bf50 con 0x7f4d90159f40 2023-11-28T18:32:20.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.628+0000 7f4d97c46640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d900ccbc0 con 0x7f4d900cc760 2023-11-28T18:32:20.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.628+0000 7f4d97c46640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9015a320 con 0x7f4d900ad680 2023-11-28T18:32:20.629 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.628+0000 7f4d959e2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4d90159f40 0x7f4d900caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.628+0000 7f4d959e2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4d90159f40 0x7f4d900caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50754/0 (socket says 172.21.15.105:50754) 2023-11-28T18:32:20.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.628+0000 7f4d959e2640 1 -- 172.21.15.105:0/2218836774 learned_addr learned my addr 172.21.15.105:0/2218836774 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:20.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.628+0000 7f4d961e3640 1 --2- 172.21.15.105:0/2218836774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d900ad680 0x7f4d9015da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.630 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.628+0000 7f4d951e1640 1 --2- 172.21.15.105:0/2218836774 >> v2:172.21.15.148:3300/0 conn(0x7f4d900cc760 0x7f4d900ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.628+0000 7f4d959e2640 1 -- 172.21.15.105:0/2218836774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d900ad680 msgr2=0x7f4d9015da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.632 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.628+0000 7f4d959e2640 1 --2- 172.21.15.105:0/2218836774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d900ad680 0x7f4d9015da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.628+0000 7f4d959e2640 1 -- 172.21.15.105:0/2218836774 >> v2:172.21.15.148:3300/0 conn(0x7f4d900cc760 msgr2=0x7f4d900ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:20.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.628+0000 7f4d959e2640 1 --2- 172.21.15.105:0/2218836774 >> v2:172.21.15.148:3300/0 conn(0x7f4d900cc760 0x7f4d900ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.633 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.628+0000 7f4d959e2640 1 -- 172.21.15.105:0/2218836774 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d90071380 con 0x7f4d90159f40 2023-11-28T18:32:20.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.632+0000 7f4d959e2640 1 --2- 172.21.15.105:0/2218836774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d90159f40 0x7f4d900caf60 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f4d9009d0d0 tx=0x7f4d8c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159b64e9438effd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.632+0000 7f4d949e0640 1 -- 172.21.15.105:0/2218836774 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0ab600 con 0x7f4d90159f40 2023-11-28T18:32:20.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.632+0000 7f4d949e0640 1 -- 172.21.15.105:0/2218836774 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8c0ab760 con 0x7f4d90159f40 2023-11-28T18:32:20.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.632+0000 7f4d949e0640 1 -- 172.21.15.105:0/2218836774 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0abaf0 con 0x7f4d90159f40 2023-11-28T18:32:20.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.632+0000 7f4d97c46640 1 -- 172.21.15.105:0/2218836774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d90159f40 msgr2=0x7f4d900caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.632+0000 7f4d97c46640 1 --2- 172.21.15.105:0/2218836774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d90159f40 0x7f4d900caf60 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f4d9009d0d0 tx=0x7f4d8c01c440 comp rx=0 tx=0).stop 2023-11-28T18:32:20.637 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.632+0000 7f4d97c46640 1 -- 172.21.15.105:0/2218836774 shutdown_connections 2023-11-28T18:32:20.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.632+0000 7f4d97c46640 1 --2- 172.21.15.105:0/2218836774 >> v2:172.21.15.148:3300/0 conn(0x7f4d900cc760 0x7f4d900ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.632+0000 7f4d97c46640 1 --2- 172.21.15.105:0/2218836774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d900ad680 0x7f4d9015da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.632+0000 7f4d97c46640 1 --2- 172.21.15.105:0/2218836774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d90159f40 0x7f4d900caf60 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.632+0000 7f4d97c46640 1 -- 172.21.15.105:0/2218836774 >> 172.21.15.105:0/2218836774 conn(0x7f4d900ceb90 msgr2=0x7f4d900cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:20.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.632+0000 7f4d97c46640 1 -- 172.21.15.105:0/2218836774 shutdown_connections 2023-11-28T18:32:20.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.632+0000 7f4d97c46640 1 -- 172.21.15.105:0/2218836774 wait complete. 2023-11-28T18:32:20.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d97c46640 1 Processor -- start 2023-11-28T18:32:20.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d97c46640 1 -- start start 2023-11-28T18:32:20.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d97c46640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d900ad680 0x7f4d90151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d97c46640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d900cc760 0x7f4d90163ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d97c46640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d90159f40 0x7f4d901643e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d97c46640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9015c580 con 0x7f4d90159f40 2023-11-28T18:32:20.644 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d97c46640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9015c860 con 0x7f4d900ad680 2023-11-28T18:32:20.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d97c46640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9015cb40 con 0x7f4d900cc760 2023-11-28T18:32:20.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d961e3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d90159f40 0x7f4d901643e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d961e3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d90159f40 0x7f4d901643e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50756/0 (socket says 172.21.15.105:50756) 2023-11-28T18:32:20.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d961e3640 1 -- 172.21.15.105:0/2763758183 learned_addr learned my addr 172.21.15.105:0/2763758183 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:20.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d951e1640 1 --2- 172.21.15.105:0/2763758183 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d900cc760 0x7f4d90163ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d959e2640 1 --2- 172.21.15.105:0/2763758183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d900ad680 0x7f4d90151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d961e3640 1 -- 172.21.15.105:0/2763758183 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d900cc760 msgr2=0x7f4d90163ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d961e3640 1 --2- 172.21.15.105:0/2763758183 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d900cc760 0x7f4d90163ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d961e3640 1 -- 172.21.15.105:0/2763758183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d900ad680 msgr2=0x7f4d90151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:20.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d961e3640 1 --2- 172.21.15.105:0/2763758183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d900ad680 0x7f4d90151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:20.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d961e3640 1 -- 172.21.15.105:0/2763758183 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8c0094d0 con 0x7f4d90159f40 2023-11-28T18:32:20.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d959e2640 1 --2- 172.21.15.105:0/2763758183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d900ad680 0x7f4d90151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:32:20.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d961e3640 1 --2- 172.21.15.105:0/2763758183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d90159f40 0x7f4d901643e0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800eed0 tx=0x7f4d8800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d867fc640 1 -- 172.21.15.105:0/2763758183 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d88010040 con 0x7f4d90159f40 2023-11-28T18:32:20.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d867fc640 1 -- 172.21.15.105:0/2763758183 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d88014400 con 0x7f4d90159f40 2023-11-28T18:32:20.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d97c46640 1 -- 172.21.15.105:0/2763758183 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9015ce20 con 0x7f4d90159f40 2023-11-28T18:32:20.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d867fc640 1 -- 172.21.15.105:0/2763758183 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d88009ea0 con 0x7f4d90159f40 2023-11-28T18:32:20.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d97c46640 1 -- 172.21.15.105:0/2763758183 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d90151580 con 0x7f4d90159f40 2023-11-28T18:32:20.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.636+0000 7f4d97c46640 1 -- 172.21.15.105:0/2763758183 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d90067570 con 0x7f4d90159f40 2023-11-28T18:32:20.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.640+0000 7f4d867fc640 1 -- 172.21.15.105:0/2763758183 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f4d880147b0 con 0x7f4d90159f40 2023-11-28T18:32:20.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.640+0000 7f4d867fc640 1 --2- 172.21.15.105:0/2763758183 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d740668d0 0x7f4d74068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:20.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.640+0000 7f4d867fc640 1 -- 172.21.15.105:0/2763758183 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4d88084ee0 con 0x7f4d90159f40 2023-11-28T18:32:20.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.640+0000 7f4d959e2640 1 --2- 172.21.15.105:0/2763758183 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d740668d0 0x7f4d74068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:20.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.640+0000 7f4d959e2640 1 --2- 172.21.15.105:0/2763758183 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d740668d0 0x7f4d74068d90 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0af000 tx=0x7f4d8c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:20.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.644+0000 7f4d867fc640 1 -- 172.21.15.105:0/2763758183 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4d88051fd0 con 0x7f4d90159f40 2023-11-28T18:32:20.826 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:20.824+0000 7f4d97c46640 1 -- 172.21.15.105:0/2763758183 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f4d9015d2d0 con 0x7f4d90159f40 2023-11-28T18:32:21.300 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.300+0000 7f4d867fc640 1 -- 172.21.15.105:0/2763758183 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v140) v1 ==== 105+0+0 (secure 0 0 0) 0x7f4d880180b0 con 0x7f4d90159f40 2023-11-28T18:32:21.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.300+0000 7f4d97c46640 1 -- 172.21.15.105:0/2763758183 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d740668d0 msgr2=0x7f4d74068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:21.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.300+0000 7f4d97c46640 1 --2- 172.21.15.105:0/2763758183 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d740668d0 0x7f4d74068d90 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0af000 tx=0x7f4d8c0a7000 comp rx=0 tx=0).stop 2023-11-28T18:32:21.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.300+0000 7f4d97c46640 1 -- 172.21.15.105:0/2763758183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d90159f40 msgr2=0x7f4d901643e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:21.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.300+0000 7f4d97c46640 1 --2- 172.21.15.105:0/2763758183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d90159f40 0x7f4d901643e0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800eed0 tx=0x7f4d8800b480 comp rx=0 tx=0).stop 2023-11-28T18:32:21.302 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.300+0000 7f4d97c46640 1 -- 172.21.15.105:0/2763758183 shutdown_connections 2023-11-28T18:32:21.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.300+0000 7f4d97c46640 1 --2- 172.21.15.105:0/2763758183 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d740668d0 0x7f4d74068d90 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:21.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.300+0000 7f4d97c46640 1 --2- 172.21.15.105:0/2763758183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d90159f40 0x7f4d901643e0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:21.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.300+0000 7f4d97c46640 1 --2- 172.21.15.105:0/2763758183 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d900cc760 0x7f4d90163ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:21.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.300+0000 7f4d97c46640 1 --2- 172.21.15.105:0/2763758183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d900ad680 0x7f4d90151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:21.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.300+0000 7f4d97c46640 1 -- 172.21.15.105:0/2763758183 >> 172.21.15.105:0/2763758183 conn(0x7f4d900ceb90 msgr2=0x7f4d90163020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:21.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.304+0000 7f4d97c46640 1 -- 172.21.15.105:0/2763758183 shutdown_connections 2023-11-28T18:32:21.303 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.304+0000 7f4d97c46640 1 -- 172.21.15.105:0/2763758183 wait complete. 2023-11-28T18:32:21.321 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:32:21.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.460+0000 7f86deefe640 1 Processor -- start 2023-11-28T18:32:21.461 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.460+0000 7f86deefe640 1 -- start start 2023-11-28T18:32:21.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.460+0000 7f86deefe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86d80caf60 0x7f86d80cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:21.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.460+0000 7f86deefe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86d80ad850 0x7f86d80701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:21.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.460+0000 7f86deefe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d806bf50 con 0x7f86d8159f40 2023-11-28T18:32:21.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.460+0000 7f86deefe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86d80cb3c0 con 0x7f86d80caf60 2023-11-28T18:32:21.463 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.460+0000 7f86deefe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d815a320 con 0x7f86d80ad850 2023-11-28T18:32:21.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.464+0000 7f86dd49b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86d80ad850 0x7f86d80701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:21.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.464+0000 7f86dd49b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86d80ad850 0x7f86d80701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.105:58370/0 (socket says 172.21.15.105:58370) 2023-11-28T18:32:21.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.464+0000 7f86dd49b640 1 -- 172.21.15.105:0/949066976 learned_addr learned my addr 172.21.15.105:0/949066976 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:21.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.464+0000 7f86cffff640 1 --2- 172.21.15.105:0/949066976 >> v2:172.21.15.148:3300/0 conn(0x7f86d80caf60 0x7f86d80cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:21.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.464+0000 7f86dd49b640 1 -- 172.21.15.105:0/949066976 >> v2:172.21.15.148:3300/0 conn(0x7f86d80caf60 msgr2=0x7f86d80cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:21.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.464+0000 7f86dd49b640 1 --2- 172.21.15.105:0/949066976 >> v2:172.21.15.148:3300/0 conn(0x7f86d80caf60 0x7f86d80cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:21.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.464+0000 7f86dd49b640 1 -- 172.21.15.105:0/949066976 >> v1:172.21.15.148:6789/0 conn(0x7f86d8159f40 legacy=0x7f86d80ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:21.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.464+0000 7f86dd49b640 1 -- 172.21.15.105:0/949066976 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d8071380 con 0x7f86d80ad850 2023-11-28T18:32:21.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.464+0000 7f86dd49b640 1 --2- 172.21.15.105:0/949066976 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86d80ad850 0x7f86d80701d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f86d0009870 tx=0x7f86d001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa2f8b878629b65 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:21.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.464+0000 7f86cf7fe640 1 -- 172.21.15.105:0/949066976 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d00ab680 con 0x7f86d80ad850 2023-11-28T18:32:21.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.468+0000 7f86cf7fe640 1 -- 172.21.15.105:0/949066976 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d00ab7e0 con 0x7f86d80ad850 2023-11-28T18:32:21.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.468+0000 7f86cf7fe640 1 -- 172.21.15.105:0/949066976 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d00abb70 con 0x7f86d80ad850 2023-11-28T18:32:21.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.468+0000 7f86deefe640 1 -- 172.21.15.105:0/949066976 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86d80ad850 msgr2=0x7f86d80701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:21.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.468+0000 7f86deefe640 1 --2- 172.21.15.105:0/949066976 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86d80ad850 0x7f86d80701d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f86d0009870 tx=0x7f86d001c480 comp rx=0 tx=0).stop 2023-11-28T18:32:21.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.468+0000 7f86deefe640 1 -- 172.21.15.105:0/949066976 shutdown_connections 2023-11-28T18:32:21.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.468+0000 7f86deefe640 1 --2- 172.21.15.105:0/949066976 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86d80ad850 0x7f86d80701d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:21.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.468+0000 7f86deefe640 1 --2- 172.21.15.105:0/949066976 >> v2:172.21.15.148:3300/0 conn(0x7f86d80caf60 0x7f86d80cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:21.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.468+0000 7f86deefe640 1 -- 172.21.15.105:0/949066976 >> 172.21.15.105:0/949066976 conn(0x7f86d80ceb90 msgr2=0x7f86d80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:21.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.468+0000 7f86deefe640 1 -- 172.21.15.105:0/949066976 shutdown_connections 2023-11-28T18:32:21.467 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.468+0000 7f86deefe640 1 -- 172.21.15.105:0/949066976 wait complete. 2023-11-28T18:32:21.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.468+0000 7f86deefe640 1 Processor -- start 2023-11-28T18:32:21.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.468+0000 7f86deefe640 1 -- start start 2023-11-28T18:32:21.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.468+0000 7f86deefe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d80ad850 0x7f86d8148030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:21.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.468+0000 7f86deefe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86d80caf60 0x7f86d8147390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:21.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.468+0000 7f86deefe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86d8159f40 0x7f86d80ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:21.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.468+0000 7f86deefe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d814a180 con 0x7f86d8159f40 2023-11-28T18:32:21.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.468+0000 7f86deefe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d80c9d30 con 0x7f86d80ad850 2023-11-28T18:32:21.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.468+0000 7f86deefe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d8149bc0 con 0x7f86d80caf60 2023-11-28T18:32:21.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.468+0000 7f86dd49b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86d8159f40 0x7f86d80ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:21.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.472+0000 7f86dd49b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86d8159f40 0x7f86d80ca150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56940/0 (socket says 172.21.15.105:56940) 2023-11-28T18:32:21.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.472+0000 7f86dd49b640 1 -- 172.21.15.105:0/1399115521 learned_addr learned my addr 172.21.15.105:0/1399115521 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:21.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.472+0000 7f86cffff640 1 --2- 172.21.15.105:0/1399115521 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86d80caf60 0x7f86d8147390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:21.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.472+0000 7f86dcc9a640 1 --2- 172.21.15.105:0/1399115521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d80ad850 0x7f86d8148030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:21.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.472+0000 7f86dd49b640 1 -- 172.21.15.105:0/1399115521 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86d80caf60 msgr2=0x7f86d8147390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:21.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.472+0000 7f86dd49b640 1 --2- 172.21.15.105:0/1399115521 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86d80caf60 0x7f86d8147390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:21.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.472+0000 7f86dd49b640 1 -- 172.21.15.105:0/1399115521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d80ad850 msgr2=0x7f86d8148030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:21.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.472+0000 7f86dd49b640 1 --2- 172.21.15.105:0/1399115521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d80ad850 0x7f86d8148030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:21.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.472+0000 7f86dd49b640 1 -- 172.21.15.105:0/1399115521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d00094d0 con 0x7f86d8159f40 2023-11-28T18:32:21.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.472+0000 7f86dd49b640 1 --2- 172.21.15.105:0/1399115521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86d8159f40 0x7f86d80ca150 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f86d00ad040 tx=0x7f86d001acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:21.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.472+0000 7f86cd7fa640 1 -- 172.21.15.105:0/1399115521 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d00b4800 con 0x7f86d8159f40 2023-11-28T18:32:21.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.472+0000 7f86cd7fa640 1 -- 172.21.15.105:0/1399115521 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d00b4980 con 0x7f86d8159f40 2023-11-28T18:32:21.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.472+0000 7f86deefe640 1 -- 172.21.15.105:0/1399115521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d8149ea0 con 0x7f86d8159f40 2023-11-28T18:32:21.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.472+0000 7f86cd7fa640 1 -- 172.21.15.105:0/1399115521 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d00b4d30 con 0x7f86d8159f40 2023-11-28T18:32:21.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.472+0000 7f86deefe640 1 -- 172.21.15.105:0/1399115521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d8148570 con 0x7f86d8159f40 2023-11-28T18:32:21.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.472+0000 7f86deefe640 1 -- 172.21.15.105:0/1399115521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d80c9e70 con 0x7f86d8159f40 2023-11-28T18:32:21.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.472+0000 7f86cd7fa640 1 -- 172.21.15.105:0/1399115521 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f86d00c7440 con 0x7f86d8159f40 2023-11-28T18:32:21.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.476+0000 7f86cd7fa640 1 --2- 172.21.15.105:0/1399115521 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f86c4066900 0x7f86c4068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:21.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.476+0000 7f86cd7fa640 1 -- 172.21.15.105:0/1399115521 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f86d011f120 con 0x7f86d8159f40 2023-11-28T18:32:21.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.476+0000 7f86dcc9a640 1 --2- 172.21.15.105:0/1399115521 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f86c4066900 0x7f86c4068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:21.476 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.476+0000 7f86dcc9a640 1 --2- 172.21.15.105:0/1399115521 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f86c4066900 0x7f86c4068dc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f86d80cea90 tx=0x7f86d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:21.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.480+0000 7f86cd7fa640 1 -- 172.21.15.105:0/1399115521 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f86d80c9e70 con 0x7f86d8159f40 2023-11-28T18:32:21.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.660+0000 7f86deefe640 1 -- 172.21.15.105:0/1399115521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86d80c9e70 con 0x7f86d8159f40 2023-11-28T18:32:21.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.660+0000 7f86cd7fa640 1 -- 172.21.15.105:0/1399115521 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+4961 (secure 0 0 0) 0x7f86d80c9e70 con 0x7f86d8159f40 2023-11-28T18:32:21.660 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:32:21.660 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":140,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3161918450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3161918450},{"type":"v1","addr":"172.21.15.105:6835","nonce":3161918450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":129}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:31:58.827408+0000","modified":"2023-11-28T18:32:20.829600+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9126,"mds_1":9123,"mds_2":9120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9120":{"gid":9120,"name":"d","rank":2,"incarnation":138,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6834/3958106803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3958106803},{"type":"v1","addr":"172.21.15.148:6834","nonce":3958106803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9123":{"gid":9123,"name":"b","rank":1,"incarnation":136,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6835/3706038483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6833","nonce":3706038483},{"type":"v1","addr":"172.21.15.148:6835","nonce":3706038483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9126":{"gid":9126,"name":"c","rank":0,"incarnation":130,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/1978722810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1978722810},{"type":"v1","addr":"172.21.15.105:6837","nonce":1978722810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-28T18:32:21.661 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 140 2023-11-28T18:32:21.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.660+0000 7f86deefe640 1 -- 172.21.15.105:0/1399115521 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f86c4066900 msgr2=0x7f86c4068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:21.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.660+0000 7f86deefe640 1 --2- 172.21.15.105:0/1399115521 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f86c4066900 0x7f86c4068dc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f86d80cea90 tx=0x7f86d4009040 comp rx=0 tx=0).stop 2023-11-28T18:32:21.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.660+0000 7f86deefe640 1 -- 172.21.15.105:0/1399115521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86d8159f40 msgr2=0x7f86d80ca150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:21.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.660+0000 7f86deefe640 1 --2- 172.21.15.105:0/1399115521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86d8159f40 0x7f86d80ca150 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f86d00ad040 tx=0x7f86d001acc0 comp rx=0 tx=0).stop 2023-11-28T18:32:21.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.660+0000 7f86deefe640 1 -- 172.21.15.105:0/1399115521 shutdown_connections 2023-11-28T18:32:21.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.660+0000 7f86deefe640 1 --2- 172.21.15.105:0/1399115521 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f86c4066900 0x7f86c4068dc0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:21.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.660+0000 7f86deefe640 1 --2- 172.21.15.105:0/1399115521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86d8159f40 0x7f86d80ca150 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:21.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.660+0000 7f86deefe640 1 --2- 172.21.15.105:0/1399115521 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86d80caf60 0x7f86d8147390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:21.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.660+0000 7f86deefe640 1 --2- 172.21.15.105:0/1399115521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d80ad850 0x7f86d8148030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:21.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.660+0000 7f86deefe640 1 -- 172.21.15.105:0/1399115521 >> 172.21.15.105:0/1399115521 conn(0x7f86d80ceb90 msgr2=0x7f86d8152820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:21.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.660+0000 7f86deefe640 1 -- 172.21.15.105:0/1399115521 shutdown_connections 2023-11-28T18:32:21.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:21.660+0000 7f86deefe640 1 -- 172.21.15.105:0/1399115521 wait complete. 2023-11-28T18:32:22.082 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 140, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T18:31:58.827408+0000', 'modified': '2023-11-28T18:32:20.829600+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9126, 'mds_1': 9123, 'mds_2': 9120}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9120': {'gid': 9120, 'name': 'd', 'rank': 2, 'incarnation': 138, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.148:6834/3958106803', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6832', 'nonce': 3958106803}, {'type': 'v1', 'addr': '172.21.15.148:6834', 'nonce': 3958106803}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_9123': {'gid': 9123, 'name': 'b', 'rank': 1, 'incarnation': 136, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.148:6835/3706038483', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6833', 'nonce': 3706038483}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 3706038483}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_9126': {'gid': 9126, 'name': 'c', 'rank': 0, 'incarnation': 130, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6837/1978722810', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 1978722810}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 1978722810}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T18:32:22.082 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-11-28T18:32:22.083 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-28T18:32:22.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.220+0000 7f981e92a640 1 Processor -- start 2023-11-28T18:32:22.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.220+0000 7f981e92a640 1 -- start start 2023-11-28T18:32:22.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.224+0000 7f981e92a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f981815d4f0 0x7f98180d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.224+0000 7f981e92a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f98180d3c30 0x7f98180d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.224+0000 7f981e92a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98180b25b0 con 0x7f98180d1890 2023-11-28T18:32:22.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.224+0000 7f981e92a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98180b2900 con 0x7f98180d3c30 2023-11-28T18:32:22.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.224+0000 7f981e92a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98180d1cc0 con 0x7f981815d4f0 2023-11-28T18:32:22.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.224+0000 7f98177fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f98180d3c30 0x7f98180d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.224+0000 7f98177fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f98180d3c30 0x7f98180d1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56948/0 (socket says 172.21.15.105:56948) 2023-11-28T18:32:22.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.224+0000 7f98177fe640 1 -- 172.21.15.105:0/1187925085 learned_addr learned my addr 172.21.15.105:0/1187925085 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:22.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.224+0000 7f98177fe640 1 -- 172.21.15.105:0/1187925085 >> v2:172.21.15.148:3300/0 conn(0x7f981815d4f0 msgr2=0x7f98180d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.224+0000 7f98177fe640 1 --2- 172.21.15.105:0/1187925085 >> v2:172.21.15.148:3300/0 conn(0x7f981815d4f0 0x7f98180d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.224 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.224+0000 7f98177fe640 1 -- 172.21.15.105:0/1187925085 >> v1:172.21.15.148:6789/0 conn(0x7f98180d1890 legacy=0x7f98180cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.224+0000 7f98177fe640 1 -- 172.21.15.105:0/1187925085 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98180cea30 con 0x7f98180d3c30 2023-11-28T18:32:22.225 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.224+0000 7f98177fe640 1 --2- 172.21.15.105:0/1187925085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98180d3c30 0x7f98180d1350 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f981808bbf0 tx=0x7f980c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6d20454f12b9a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.224+0000 7f9816ffd640 1 -- 172.21.15.105:0/1187925085 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c0ab600 con 0x7f98180d3c30 2023-11-28T18:32:22.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.224+0000 7f9816ffd640 1 -- 172.21.15.105:0/1187925085 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980c0ab760 con 0x7f98180d3c30 2023-11-28T18:32:22.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.224+0000 7f9816ffd640 1 -- 172.21.15.105:0/1187925085 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c0abaf0 con 0x7f98180d3c30 2023-11-28T18:32:22.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.228+0000 7f981e92a640 1 -- 172.21.15.105:0/1187925085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98180d3c30 msgr2=0x7f98180d1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.228+0000 7f981e92a640 1 --2- 172.21.15.105:0/1187925085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98180d3c30 0x7f98180d1350 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f981808bbf0 tx=0x7f980c01c400 comp rx=0 tx=0).stop 2023-11-28T18:32:22.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.228+0000 7f981e92a640 1 -- 172.21.15.105:0/1187925085 shutdown_connections 2023-11-28T18:32:22.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.228+0000 7f981e92a640 1 --2- 172.21.15.105:0/1187925085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98180d3c30 0x7f98180d1350 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.228+0000 7f981e92a640 1 --2- 172.21.15.105:0/1187925085 >> v2:172.21.15.148:3300/0 conn(0x7f981815d4f0 0x7f98180d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.228+0000 7f981e92a640 1 -- 172.21.15.105:0/1187925085 >> 172.21.15.105:0/1187925085 conn(0x7f98180d3030 msgr2=0x7f98180d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:22.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.228+0000 7f981e92a640 1 -- 172.21.15.105:0/1187925085 shutdown_connections 2023-11-28T18:32:22.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.228+0000 7f981e92a640 1 -- 172.21.15.105:0/1187925085 wait complete. 2023-11-28T18:32:22.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.228+0000 7f981e92a640 1 Processor -- start 2023-11-28T18:32:22.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.228+0000 7f981e92a640 1 -- start start 2023-11-28T18:32:22.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.228+0000 7f981e92a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98180d1890 0x7f98181559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.228+0000 7f981e92a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98180d3c30 0x7f98180cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.228+0000 7f981e92a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f981815d4f0 0x7f98181461d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.228+0000 7f981e92a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98180cca10 con 0x7f981815d4f0 2023-11-28T18:32:22.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.228+0000 7f981e92a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98180ccea0 con 0x7f98180d3c30 2023-11-28T18:32:22.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.228+0000 7f981e92a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98180cbe00 con 0x7f98180d1890 2023-11-28T18:32:22.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.228+0000 7f981cec7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f981815d4f0 0x7f98181461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.228+0000 7f98177fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98180d3c30 0x7f98180cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.228+0000 7f981cec7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f981815d4f0 0x7f98181461d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56954/0 (socket says 172.21.15.105:56954) 2023-11-28T18:32:22.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.228+0000 7f981cec7640 1 -- 172.21.15.105:0/2279243165 learned_addr learned my addr 172.21.15.105:0/2279243165 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:22.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.228+0000 7f9817fff640 1 --2- 172.21.15.105:0/2279243165 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98180d1890 0x7f98181559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.232+0000 7f981cec7640 1 -- 172.21.15.105:0/2279243165 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98180d1890 msgr2=0x7f98181559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.232+0000 7f981cec7640 1 --2- 172.21.15.105:0/2279243165 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98180d1890 0x7f98181559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.232+0000 7f981cec7640 1 -- 172.21.15.105:0/2279243165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98180d3c30 msgr2=0x7f98180cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.232+0000 7f981cec7640 1 --2- 172.21.15.105:0/2279243165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98180d3c30 0x7f98180cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.231 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.232+0000 7f981cec7640 1 -- 172.21.15.105:0/2279243165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980c0094d0 con 0x7f981815d4f0 2023-11-28T18:32:22.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.232+0000 7f981cec7640 1 --2- 172.21.15.105:0/2279243165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f981815d4f0 0x7f98181461d0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f981000b990 tx=0x7f981000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.232+0000 7f9814ff9640 1 -- 172.21.15.105:0/2279243165 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981000cba0 con 0x7f981815d4f0 2023-11-28T18:32:22.232 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.232+0000 7f9814ff9640 1 -- 172.21.15.105:0/2279243165 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981000cd00 con 0x7f981815d4f0 2023-11-28T18:32:22.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.232+0000 7f981e92a640 1 -- 172.21.15.105:0/2279243165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98180cc0e0 con 0x7f981815d4f0 2023-11-28T18:32:22.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.232+0000 7f9814ff9640 1 -- 172.21.15.105:0/2279243165 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9810014680 con 0x7f981815d4f0 2023-11-28T18:32:22.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.232+0000 7f981e92a640 1 -- 172.21.15.105:0/2279243165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98180cc6a0 con 0x7f981815d4f0 2023-11-28T18:32:22.233 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.232+0000 7f9814ff9640 1 -- 172.21.15.105:0/2279243165 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f98100147e0 con 0x7f981815d4f0 2023-11-28T18:32:22.234 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f9814ff9640 1 --2- 172.21.15.105:0/2279243165 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f98040668d0 0x7f9804068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f9814ff9640 1 -- 172.21.15.105:0/2279243165 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f98100844e0 con 0x7f981815d4f0 2023-11-28T18:32:22.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f9817fff640 1 --2- 172.21.15.105:0/2279243165 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f98040668d0 0x7f9804068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f9821ca4000 1 Processor -- start 2023-11-28T18:32:22.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f9821ca4000 1 -- start start 2023-11-28T18:32:22.235 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f9817fff640 1 --2- 172.21.15.105:0/2279243165 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f98040668d0 0x7f9804068d90 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f98080098f0 tx=0x7f9808006f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f9821ca4000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555bc0d5cd30 0x555bc0d5d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f9821ca4000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d5d650 0x555bc0d0ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f9821ca4000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bc0d10500 0x555bc0d108e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f9821ca4000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bc0bd9110 con 0x555bc0d5d650 2023-11-28T18:32:22.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f9821ca4000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bc0b699a0 con 0x555bc0d10500 2023-11-28T18:32:22.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f9821ca4000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bc0b66ed0 con 0x555bc0d5cd30 2023-11-28T18:32:22.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f98177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d5d650 0x555bc0d0ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f98177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d5d650 0x555bc0d0ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56960/0 (socket says 172.21.15.105:56960) 2023-11-28T18:32:22.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f98177fe640 1 -- 172.21.15.105:0/2189027055 learned_addr learned my addr 172.21.15.105:0/2189027055 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:22.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f9817fff640 1 --2- 172.21.15.105:0/2189027055 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555bc0d5cd30 0x555bc0d5d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f981cec7640 1 --2- 172.21.15.105:0/2189027055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bc0d10500 0x555bc0d108e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f98177fe640 1 -- 172.21.15.105:0/2189027055 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555bc0d5cd30 msgr2=0x555bc0d5d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f98177fe640 1 --2- 172.21.15.105:0/2189027055 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555bc0d5cd30 0x555bc0d5d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f98177fe640 1 -- 172.21.15.105:0/2189027055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bc0d10500 msgr2=0x555bc0d108e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f98177fe640 1 --2- 172.21.15.105:0/2189027055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bc0d10500 0x555bc0d108e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f98177fe640 1 -- 172.21.15.105:0/2189027055 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bc085a890 con 0x555bc0d5d650 2023-11-28T18:32:22.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f98177fe640 1 --2- 172.21.15.105:0/2189027055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d5d650 0x555bc0d0ffc0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f980c0097f0 tx=0x7f980c0a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e271eb33c051af8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f97fdffb640 1 -- 172.21.15.105:0/2189027055 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c0aba90 con 0x555bc0d5d650 2023-11-28T18:32:22.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f97fdffb640 1 -- 172.21.15.105:0/2189027055 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980c0abc10 con 0x555bc0d5d650 2023-11-28T18:32:22.240 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f97fdffb640 1 -- 172.21.15.105:0/2189027055 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c0bf420 con 0x555bc0d5d650 2023-11-28T18:32:22.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f9821ca4000 1 -- 172.21.15.105:0/2189027055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d5d650 msgr2=0x555bc0d0ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f9821ca4000 1 --2- 172.21.15.105:0/2189027055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d5d650 0x555bc0d0ffc0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f980c0097f0 tx=0x7f980c0a7000 comp rx=0 tx=0).stop 2023-11-28T18:32:22.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f9821ca4000 1 -- 172.21.15.105:0/2189027055 shutdown_connections 2023-11-28T18:32:22.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f9821ca4000 1 --2- 172.21.15.105:0/2189027055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bc0d10500 0x555bc0d108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f9821ca4000 1 --2- 172.21.15.105:0/2189027055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d5d650 0x555bc0d0ffc0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f9821ca4000 1 --2- 172.21.15.105:0/2189027055 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555bc0d5cd30 0x555bc0d5d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f9821ca4000 1 -- 172.21.15.105:0/2189027055 >> 172.21.15.105:0/2189027055 conn(0x555bc0d3d8c0 msgr2=0x555bc0d3fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:22.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f9821ca4000 1 -- 172.21.15.105:0/2189027055 shutdown_connections 2023-11-28T18:32:22.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.236+0000 7f9821ca4000 1 -- 172.21.15.105:0/2189027055 wait complete. 2023-11-28T18:32:22.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f9821ca4000 1 -- start start 2023-11-28T18:32:22.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f9821ca4000 1 Processor -- start 2023-11-28T18:32:22.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f9821ca4000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555bc0d10500 0x555bc0d16520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f9821ca4000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d5cd30 0x555bc0d16a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f9821ca4000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bc0d5d650 0x555bc0d16fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f9821ca4000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bc0b627c0 con 0x555bc0d5cd30 2023-11-28T18:32:22.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f9821ca4000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bc0b1abf0 con 0x555bc0d5d650 2023-11-28T18:32:22.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f9821ca4000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bc0b20a90 con 0x555bc0d10500 2023-11-28T18:32:22.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f9821ca4000 10 client.0 _ll_register_callbacks cb 0x555bc0cfde20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:32:22.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f98177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d5cd30 0x555bc0d16a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f98177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d5cd30 0x555bc0d16a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56968/0 (socket says 172.21.15.105:56968) 2023-11-28T18:32:22.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f98177fe640 1 -- 172.21.15.105:0/545686954 learned_addr learned my addr 172.21.15.105:0/545686954 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:22.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f9817fff640 1 --2- 172.21.15.105:0/545686954 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555bc0d10500 0x555bc0d16520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f981cec7640 1 --2- 172.21.15.105:0/545686954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bc0d5d650 0x555bc0d16fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f98177fe640 1 -- 172.21.15.105:0/545686954 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555bc0d10500 msgr2=0x555bc0d16520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f98177fe640 1 --2- 172.21.15.105:0/545686954 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555bc0d10500 0x555bc0d16520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f98177fe640 1 -- 172.21.15.105:0/545686954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bc0d5d650 msgr2=0x555bc0d16fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f98177fe640 1 --2- 172.21.15.105:0/545686954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bc0d5d650 0x555bc0d16fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f98177fe640 1 -- 172.21.15.105:0/545686954 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98180cea30 con 0x555bc0d5cd30 2023-11-28T18:32:22.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f98177fe640 1 --2- 172.21.15.105:0/545686954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d5cd30 0x555bc0d16a60 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f980c01aab0 tx=0x7f980c01ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8770b637d40623 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f97f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:32:22.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f97f6ffd640 1 -- 172.21.15.105:0/545686954 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c004240 con 0x555bc0d5cd30 2023-11-28T18:32:22.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f97f6ffd640 1 -- 172.21.15.105:0/545686954 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f980c0043a0 con 0x555bc0d5cd30 2023-11-28T18:32:22.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f9821ca4000 1 -- 172.21.15.105:0/545686954 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555bc0b24470 con 0x555bc0d5cd30 2023-11-28T18:32:22.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f97f6ffd640 1 -- 172.21.15.105:0/545686954 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7f980c003620 con 0x555bc0d5cd30 2023-11-28T18:32:22.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f9821ca4000 10 client.9396 fetch_fsmap learned FSMap version 140 2023-11-28T18:32:22.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f9821ca4000 1 -- 172.21.15.105:0/545686954 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=2+}) v3 -- 0x555bc0b23190 con 0x555bc0d5cd30 2023-11-28T18:32:22.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f97f6ffd640 1 -- 172.21.15.105:0/545686954 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555bc0b24470 con 0x555bc0d5cd30 2023-11-28T18:32:22.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f97f6ffd640 1 -- 172.21.15.105:0/545686954 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 140) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f980c0c7020 con 0x555bc0d5cd30 2023-11-28T18:32:22.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f9821ca4000 10 client.9396 fetch_fsmap finished waiting for FSMap version 140 2023-11-28T18:32:22.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f9821ca4000 10 client.9396 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T18:32:22.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f9821ca4000 20 client.9396 populate_metadata read hostname 'smithi105' 2023-11-28T18:32:22.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f9821ca4000 1 --2- 172.21.15.105:0/545686954 >> [v2:172.21.15.148:6832/3958106803,v1:172.21.15.148:6834/3958106803] conn(0x555bc0d179b0 0x555bc0d6eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f9821ca4000 4 client.9396 mds_command: new command op to 9120 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:32:22.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.240+0000 7f9821ca4000 1 -- 172.21.15.105:0/545686954 --> [v2:172.21.15.148:6832/3958106803,v1:172.21.15.148:6834/3958106803] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bc0b27b50 con 0x555bc0d179b0 2023-11-28T18:32:22.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.244+0000 7f9817fff640 1 --2- 172.21.15.105:0/545686954 >> [v2:172.21.15.148:6832/3958106803,v1:172.21.15.148:6834/3958106803] conn(0x555bc0d179b0 0x555bc0d6eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.247 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.244+0000 7f97f6ffd640 10 client.9396 ms_handle_connect on v2:172.21.15.148:6832/3958106803 2023-11-28T18:32:22.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.244+0000 7f9817fff640 1 --2- 172.21.15.105:0/545686954 >> [v2:172.21.15.148:6832/3958106803,v1:172.21.15.148:6834/3958106803] conn(0x555bc0d179b0 0x555bc0d6eff0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5741309c94b982d6 server_cookie=6ec6991425bf35a7 in_seq=0 out_seq=0 2023-11-28T18:32:22.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.244+0000 7f97f6ffd640 1 -- 172.21.15.105:0/545686954 <== mds.2 v2:172.21.15.148:6832/3958106803 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555bc0b27b50 con 0x555bc0d179b0 2023-11-28T18:32:22.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.244+0000 7f97f6ffd640 10 client.9396 handle_command_reply: tid=0 2023-11-28T18:32:22.248 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.244+0000 7f9821ca4000 1 client.9396 shutdown 2023-11-28T18:32:22.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.244+0000 7f9821ca4000 1 -- 172.21.15.105:0/545686954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d5cd30 msgr2=0x555bc0d16a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.244+0000 7f9821ca4000 1 --2- 172.21.15.105:0/545686954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d5cd30 0x555bc0d16a60 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f980c01aab0 tx=0x7f980c01ac40 comp rx=0 tx=0).stop 2023-11-28T18:32:22.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.244+0000 7f9821ca4000 1 -- 172.21.15.105:0/545686954 shutdown_connections 2023-11-28T18:32:22.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.244+0000 7f9821ca4000 1 --2- 172.21.15.105:0/545686954 >> [v2:172.21.15.148:6832/3958106803,v1:172.21.15.148:6834/3958106803] conn(0x555bc0d179b0 0x555bc0d6eff0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.244+0000 7f9821ca4000 1 --2- 172.21.15.105:0/545686954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bc0d5d650 0x555bc0d16fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.244+0000 7f9821ca4000 1 --2- 172.21.15.105:0/545686954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d5cd30 0x555bc0d16a60 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.244+0000 7f9821ca4000 1 --2- 172.21.15.105:0/545686954 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555bc0d10500 0x555bc0d16520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.244+0000 7f97f6ffd640 0 client.9396 ms_handle_reset on v2:172.21.15.148:6832/3958106803 2023-11-28T18:32:22.249 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.244+0000 7f9821ca4000 1 -- 172.21.15.105:0/545686954 >> 172.21.15.105:0/545686954 conn(0x555bc0d3d8c0 msgr2=0x555bc0d0f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:22.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.244+0000 7f9821ca4000 1 -- 172.21.15.105:0/545686954 shutdown_connections 2023-11-28T18:32:22.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.244+0000 7f9821ca4000 1 -- 172.21.15.105:0/545686954 wait complete. 2023-11-28T18:32:22.250 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.244+0000 7f9821ca4000 20 client.9396 trim_cache size 0 max 16384 2023-11-28T18:32:22.259 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f9821ca4000 1 Processor -- start 2023-11-28T18:32:22.260 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f9821ca4000 1 -- start start 2023-11-28T18:32:22.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f9821ca4000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d10500 0x555bc0d3c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.261 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f9821ca4000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bc0d5cd30 0x555bc0d3c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f9821ca4000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555bc0d5d650 0x555bc0d6eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f9821ca4000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bc081b940 con 0x555bc0d10500 2023-11-28T18:32:22.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f9821ca4000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bc085b1b0 con 0x555bc0d5cd30 2023-11-28T18:32:22.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f9821ca4000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bc0d73cd0 con 0x555bc0d5d650 2023-11-28T18:32:22.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f9817fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d10500 0x555bc0d3c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f9817fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d10500 0x555bc0d3c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56972/0 (socket says 172.21.15.105:56972) 2023-11-28T18:32:22.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f9817fff640 1 -- 172.21.15.105:0/2673769203 learned_addr learned my addr 172.21.15.105:0/2673769203 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:22.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f981cec7640 1 --2- 172.21.15.105:0/2673769203 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555bc0d5d650 0x555bc0d6eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f9817fff640 1 -- 172.21.15.105:0/2673769203 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555bc0d5d650 msgr2=0x555bc0d6eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.262 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f9817fff640 1 --2- 172.21.15.105:0/2673769203 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555bc0d5d650 0x555bc0d6eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f98177fe640 1 --2- 172.21.15.105:0/2673769203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bc0d5cd30 0x555bc0d3c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f9817fff640 1 -- 172.21.15.105:0/2673769203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bc0d5cd30 msgr2=0x555bc0d3c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f9817fff640 1 --2- 172.21.15.105:0/2673769203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bc0d5cd30 0x555bc0d3c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f9817fff640 1 -- 172.21.15.105:0/2673769203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bc0d10370 con 0x555bc0d10500 2023-11-28T18:32:22.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f981cec7640 1 --2- 172.21.15.105:0/2673769203 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555bc0d5d650 0x555bc0d6eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:32:22.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f9817fff640 1 --2- 172.21.15.105:0/2673769203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d10500 0x555bc0d3c3a0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f980800ea60 tx=0x7f9808016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f6834e32769e9e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f97fe7fc640 1 -- 172.21.15.105:0/2673769203 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9808027040 con 0x555bc0d10500 2023-11-28T18:32:22.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f97fe7fc640 1 -- 172.21.15.105:0/2673769203 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9808015d50 con 0x555bc0d10500 2023-11-28T18:32:22.263 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.260+0000 7f97fe7fc640 1 -- 172.21.15.105:0/2673769203 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9808028b00 con 0x555bc0d10500 2023-11-28T18:32:22.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9821ca4000 1 -- 172.21.15.105:0/2673769203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d10500 msgr2=0x555bc0d3c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9821ca4000 1 --2- 172.21.15.105:0/2673769203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d10500 0x555bc0d3c3a0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f980800ea60 tx=0x7f9808016000 comp rx=0 tx=0).stop 2023-11-28T18:32:22.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9821ca4000 1 -- 172.21.15.105:0/2673769203 shutdown_connections 2023-11-28T18:32:22.266 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9821ca4000 1 --2- 172.21.15.105:0/2673769203 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555bc0d5d650 0x555bc0d6eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9821ca4000 1 --2- 172.21.15.105:0/2673769203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bc0d5cd30 0x555bc0d3c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9821ca4000 1 --2- 172.21.15.105:0/2673769203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d10500 0x555bc0d3c3a0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9821ca4000 1 -- 172.21.15.105:0/2673769203 >> 172.21.15.105:0/2673769203 conn(0x555bc0d3d8c0 msgr2=0x555bc0d0e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:22.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9821ca4000 1 -- 172.21.15.105:0/2673769203 shutdown_connections 2023-11-28T18:32:22.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9821ca4000 1 -- 172.21.15.105:0/2673769203 wait complete. 2023-11-28T18:32:22.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9821ca4000 1 -- start start 2023-11-28T18:32:22.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9821ca4000 1 Processor -- start 2023-11-28T18:32:22.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9821ca4000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d10500 0x555bc0dbd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9821ca4000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bc0d5cd30 0x555bc0dbdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9821ca4000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555bc0d5d650 0x555bc0d78a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.267 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9821ca4000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bc0ba3440 con 0x555bc0d10500 2023-11-28T18:32:22.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9821ca4000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bc0d1a510 con 0x555bc0d5cd30 2023-11-28T18:32:22.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9821ca4000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bc0b37b70 con 0x555bc0d5d650 2023-11-28T18:32:22.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9821ca4000 10 client.0 _ll_register_callbacks cb 0x555bc0bd9060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:32:22.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9817fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d10500 0x555bc0dbd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9817fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d10500 0x555bc0dbd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56974/0 (socket says 172.21.15.105:56974) 2023-11-28T18:32:22.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9817fff640 1 -- 172.21.15.105:0/1661921222 learned_addr learned my addr 172.21.15.105:0/1661921222 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:22.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f981cec7640 1 --2- 172.21.15.105:0/1661921222 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555bc0d5d650 0x555bc0d78a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f98177fe640 1 --2- 172.21.15.105:0/1661921222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bc0d5cd30 0x555bc0dbdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9817fff640 1 -- 172.21.15.105:0/1661921222 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555bc0d5d650 msgr2=0x555bc0d78a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9817fff640 1 --2- 172.21.15.105:0/1661921222 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555bc0d5d650 0x555bc0d78a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9817fff640 1 -- 172.21.15.105:0/1661921222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bc0d5cd30 msgr2=0x555bc0dbdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.268 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9817fff640 1 --2- 172.21.15.105:0/1661921222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bc0d5cd30 0x555bc0dbdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9817fff640 1 -- 172.21.15.105:0/1661921222 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980801a040 con 0x555bc0d10500 2023-11-28T18:32:22.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f98177fe640 1 --2- 172.21.15.105:0/1661921222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bc0d5cd30 0x555bc0dbdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:32:22.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9817fff640 1 --2- 172.21.15.105:0/1661921222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d10500 0x555bc0dbd810 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f9808015c20 tx=0x7f98080046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2b0cfc56184c3e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f97f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:32:22.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f97f6ffd640 1 -- 172.21.15.105:0/1661921222 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9808028240 con 0x555bc0d10500 2023-11-28T18:32:22.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f97f6ffd640 1 -- 172.21.15.105:0/1661921222 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f98080283a0 con 0x555bc0d10500 2023-11-28T18:32:22.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9821ca4000 1 -- 172.21.15.105:0/1661921222 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555bc0b61110 con 0x555bc0d10500 2023-11-28T18:32:22.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f97f6ffd640 1 -- 172.21.15.105:0/1661921222 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98080285a0 con 0x555bc0d10500 2023-11-28T18:32:22.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9821ca4000 10 client.9402 fetch_fsmap learned FSMap version 140 2023-11-28T18:32:22.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f9821ca4000 1 -- 172.21.15.105:0/1661921222 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=2+}) v3 -- 0x555bc0b09490 con 0x555bc0d10500 2023-11-28T18:32:22.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.264+0000 7f97f6ffd640 1 -- 172.21.15.105:0/1661921222 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555bc0b61110 con 0x555bc0d10500 2023-11-28T18:32:22.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f97f6ffd640 1 -- 172.21.15.105:0/1661921222 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 140) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f9808011070 con 0x555bc0d10500 2023-11-28T18:32:22.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f9821ca4000 10 client.9402 fetch_fsmap finished waiting for FSMap version 140 2023-11-28T18:32:22.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f9821ca4000 10 client.9402 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T18:32:22.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f9821ca4000 20 client.9402 populate_metadata read hostname 'smithi105' 2023-11-28T18:32:22.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f9821ca4000 1 --2- 172.21.15.105:0/1661921222 >> [v2:172.21.15.148:6832/3958106803,v1:172.21.15.148:6834/3958106803] conn(0x555bc0dc4670 0x555bc0dc6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.270 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f9821ca4000 4 client.9402 mds_command: new command op to 9120 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:32:22.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f9821ca4000 1 -- 172.21.15.105:0/1661921222 --> [v2:172.21.15.148:6832/3958106803,v1:172.21.15.148:6834/3958106803] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555bc0b8f650 con 0x555bc0dc4670 2023-11-28T18:32:22.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f98177fe640 1 --2- 172.21.15.105:0/1661921222 >> [v2:172.21.15.148:6832/3958106803,v1:172.21.15.148:6834/3958106803] conn(0x555bc0dc4670 0x555bc0dc6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f97f6ffd640 10 client.9402 ms_handle_connect on v2:172.21.15.148:6832/3958106803 2023-11-28T18:32:22.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f98177fe640 1 --2- 172.21.15.105:0/1661921222 >> [v2:172.21.15.148:6832/3958106803,v1:172.21.15.148:6834/3958106803] conn(0x555bc0dc4670 0x555bc0dc6a60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4437a721329d7f1c server_cookie=3a9bdb16b22ad911 in_seq=0 out_seq=0 2023-11-28T18:32:22.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f97f6ffd640 1 -- 172.21.15.105:0/1661921222 <== mds.2 v2:172.21.15.148:6832/3958106803 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x555bc0b8f650 con 0x555bc0dc4670 2023-11-28T18:32:22.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f97f6ffd640 10 client.9402 handle_command_reply: tid=0 2023-11-28T18:32:22.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f9821ca4000 1 client.9402 shutdown 2023-11-28T18:32:22.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f9821ca4000 1 -- 172.21.15.105:0/1661921222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d10500 msgr2=0x555bc0dbd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f9821ca4000 1 --2- 172.21.15.105:0/1661921222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d10500 0x555bc0dbd810 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f9808015c20 tx=0x7f98080046d0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f9821ca4000 1 -- 172.21.15.105:0/1661921222 shutdown_connections 2023-11-28T18:32:22.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f9821ca4000 1 --2- 172.21.15.105:0/1661921222 >> [v2:172.21.15.148:6832/3958106803,v1:172.21.15.148:6834/3958106803] conn(0x555bc0dc4670 0x555bc0dc6a60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f9821ca4000 1 --2- 172.21.15.105:0/1661921222 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555bc0d5d650 0x555bc0d78a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f9821ca4000 1 --2- 172.21.15.105:0/1661921222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bc0d5cd30 0x555bc0dbdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f9821ca4000 1 --2- 172.21.15.105:0/1661921222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bc0d10500 0x555bc0dbd810 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f9821ca4000 1 -- 172.21.15.105:0/1661921222 >> 172.21.15.105:0/1661921222 conn(0x555bc0d3d8c0 msgr2=0x555bc0d0efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:22.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f97f6ffd640 0 client.9402 ms_handle_reset on v2:172.21.15.148:6832/3958106803 2023-11-28T18:32:22.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f9821ca4000 1 -- 172.21.15.105:0/1661921222 shutdown_connections 2023-11-28T18:32:22.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f9821ca4000 1 -- 172.21.15.105:0/1661921222 wait complete. 2023-11-28T18:32:22.273 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f9821ca4000 20 client.9402 trim_cache size 0 max 16384 2023-11-28T18:32:22.274 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:32:22.274 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:32:22.274 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 2, 2023-11-28T18:32:22.274 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 9120, 2023-11-28T18:32:22.274 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:32:22.274 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:32:22.274 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:32:22.274 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 3.9845097049999998, 2023-11-28T18:32:22.274 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 140, 2023-11-28T18:32:22.274 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 129, 2023-11-28T18:32:22.275 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 129, 2023-11-28T18:32:22.275 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 28.638751136 2023-11-28T18:32:22.275 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:32:22.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f981e92a640 1 -- 172.21.15.105:0/2279243165 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f98040668d0 msgr2=0x7f9804068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f981e92a640 1 --2- 172.21.15.105:0/2279243165 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f98040668d0 0x7f9804068d90 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f98080098f0 tx=0x7f9808006f90 comp rx=0 tx=0).stop 2023-11-28T18:32:22.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f981e92a640 1 -- 172.21.15.105:0/2279243165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f981815d4f0 msgr2=0x7f98181461d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f981e92a640 1 --2- 172.21.15.105:0/2279243165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f981815d4f0 0x7f98181461d0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f981000b990 tx=0x7f981000be50 comp rx=0 tx=0).stop 2023-11-28T18:32:22.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f981e92a640 1 -- 172.21.15.105:0/2279243165 shutdown_connections 2023-11-28T18:32:22.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f981e92a640 1 --2- 172.21.15.105:0/2279243165 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f98040668d0 0x7f9804068d90 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f981e92a640 1 --2- 172.21.15.105:0/2279243165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f981815d4f0 0x7f98181461d0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f981e92a640 1 --2- 172.21.15.105:0/2279243165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98180d3c30 0x7f98180cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f981e92a640 1 --2- 172.21.15.105:0/2279243165 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98180d1890 0x7f98181559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f981e92a640 1 -- 172.21.15.105:0/2279243165 >> 172.21.15.105:0/2279243165 conn(0x7f98180d3030 msgr2=0x7f98180afcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:22.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f981e92a640 1 -- 172.21.15.105:0/2279243165 shutdown_connections 2023-11-28T18:32:22.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.268+0000 7f981e92a640 1 -- 172.21.15.105:0/2279243165 wait complete. 2023-11-28T18:32:22.287 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-28T18:32:22.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.428+0000 7fdeb33f8640 1 Processor -- start 2023-11-28T18:32:22.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.428+0000 7fdeb33f8640 1 -- start start 2023-11-28T18:32:22.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.428+0000 7fdeb33f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeac0cfb30 0x7fdeac167fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.428+0000 7fdeb33f8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeac0ce840 con 0x7fdeac15d2d0 2023-11-28T18:32:22.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.428+0000 7fdeb33f8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeac0cd990 con 0x7fdeac15cef0 2023-11-28T18:32:22.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.428+0000 7fdeb33f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac0b4780 con 0x7fdeac0cfb30 2023-11-28T18:32:22.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.428+0000 7fdeb1bf5640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fdeac15d2d0 0x7fdeac0cf420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:53906/0 (socket says 172.21.15.105:53906) 2023-11-28T18:32:22.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.428+0000 7fdeb1bf5640 1 -- 172.21.15.105:0/1592332114 learned_addr learned my addr 172.21.15.105:0/1592332114 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:32:22.429 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.428+0000 7fdeb2bf7640 1 --2- 172.21.15.105:0/1592332114 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeac0cfb30 0x7fdeac167fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.428+0000 7fdeb13f4640 1 -- 172.21.15.105:0/1592332114 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422509604 0 0) 0x7fdeac0ce840 con 0x7fdeac15d2d0 2023-11-28T18:32:22.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.428+0000 7fdeb13f4640 1 -- 172.21.15.105:0/1592332114 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde98003530 con 0x7fdeac15d2d0 2023-11-28T18:32:22.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.428+0000 7fdeb2bf7640 1 -- 172.21.15.105:0/1592332114 >> v1:172.21.15.148:6789/0 conn(0x7fdeac15cef0 legacy=0x7fdeac0d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.428+0000 7fdeb13f4640 1 -- 172.21.15.105:0/1592332114 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 887460953 0 0) 0x7fdeac0cd990 con 0x7fdeac15cef0 2023-11-28T18:32:22.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.428+0000 7fdeb2bf7640 1 -- 172.21.15.105:0/1592332114 >> v1:172.21.15.105:6789/0 conn(0x7fdeac15d2d0 legacy=0x7fdeac0cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.428+0000 7fdeb2bf7640 1 -- 172.21.15.105:0/1592332114 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac0cecd0 con 0x7fdeac0cfb30 2023-11-28T18:32:22.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.432+0000 7fdeb2bf7640 1 --2- 172.21.15.105:0/1592332114 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeac0cfb30 0x7fdeac167fe0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009dc0 tx=0x7fdea401c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acf297c021c9dc17 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.432+0000 7fdeb13f4640 1 -- 172.21.15.105:0/1592332114 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea40a5970 con 0x7fdeac0cfb30 2023-11-28T18:32:22.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.432+0000 7fdeb13f4640 1 -- 172.21.15.105:0/1592332114 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea40a5ad0 con 0x7fdeac0cfb30 2023-11-28T18:32:22.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.432+0000 7fdeb13f4640 1 -- 172.21.15.105:0/1592332114 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea40a5e60 con 0x7fdeac0cfb30 2023-11-28T18:32:22.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.432+0000 7fdeb33f8640 1 -- 172.21.15.105:0/1592332114 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeac0cfb30 msgr2=0x7fdeac167fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.432+0000 7fdeb33f8640 1 --2- 172.21.15.105:0/1592332114 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeac0cfb30 0x7fdeac167fe0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009dc0 tx=0x7fdea401c8b0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.432+0000 7fdeb33f8640 1 -- 172.21.15.105:0/1592332114 shutdown_connections 2023-11-28T18:32:22.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.432+0000 7fdeb33f8640 1 --2- 172.21.15.105:0/1592332114 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeac0cfb30 0x7fdeac167fe0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.432+0000 7fdeb33f8640 1 -- 172.21.15.105:0/1592332114 >> 172.21.15.105:0/1592332114 conn(0x7fdeac0d2e10 msgr2=0x7fdeac0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:22.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.432+0000 7fdeb33f8640 1 -- 172.21.15.105:0/1592332114 shutdown_connections 2023-11-28T18:32:22.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.432+0000 7fdeb33f8640 1 -- 172.21.15.105:0/1592332114 wait complete. 2023-11-28T18:32:22.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb33f8640 1 Processor -- start 2023-11-28T18:32:22.435 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb33f8640 1 -- start start 2023-11-28T18:32:22.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb33f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdeac0cfb30 0x7fdeac0cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb33f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeac15cef0 0x7fdeac0c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb33f8640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeac15d2d0 0x7fdeac0c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb33f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac0c8240 con 0x7fdeac0cfb30 2023-11-28T18:32:22.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb33f8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac0c73c0 con 0x7fdeac15cef0 2023-11-28T18:32:22.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb33f8640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac0c75c0 con 0x7fdeac15d2d0 2023-11-28T18:32:22.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb23f6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdeac0cfb30 0x7fdeac0cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb23f6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdeac0cfb30 0x7fdeac0cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56986/0 (socket says 172.21.15.105:56986) 2023-11-28T18:32:22.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb23f6640 1 -- 172.21.15.105:0/1565107208 learned_addr learned my addr 172.21.15.105:0/1565107208 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:22.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb2bf7640 1 --2- 172.21.15.105:0/1565107208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeac15d2d0 0x7fdeac0c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb1bf5640 1 --2- 172.21.15.105:0/1565107208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeac15cef0 0x7fdeac0c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb23f6640 1 -- 172.21.15.105:0/1565107208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeac15d2d0 msgr2=0x7fdeac0c6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb23f6640 1 --2- 172.21.15.105:0/1565107208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeac15d2d0 0x7fdeac0c6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb23f6640 1 -- 172.21.15.105:0/1565107208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeac15cef0 msgr2=0x7fdeac0c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb23f6640 1 --2- 172.21.15.105:0/1565107208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeac15cef0 0x7fdeac0c6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb23f6640 1 -- 172.21.15.105:0/1565107208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea4009a70 con 0x7fdeac0cfb30 2023-11-28T18:32:22.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb2bf7640 1 --2- 172.21.15.105:0/1565107208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeac15d2d0 0x7fdeac0c6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:32:22.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb1bf5640 1 --2- 172.21.15.105:0/1565107208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeac15cef0 0x7fdeac0c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:32:22.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb23f6640 1 --2- 172.21.15.105:0/1565107208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdeac0cfb30 0x7fdeac0cbd00 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fdea800c4d0 tx=0x7fdea800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdea2ffd640 1 -- 172.21.15.105:0/1565107208 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8005480 con 0x7fdeac0cfb30 2023-11-28T18:32:22.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdea2ffd640 1 -- 172.21.15.105:0/1565107208 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea8003600 con 0x7fdeac0cfb30 2023-11-28T18:32:22.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdea2ffd640 1 -- 172.21.15.105:0/1565107208 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea80039b0 con 0x7fdeac0cfb30 2023-11-28T18:32:22.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb33f8640 1 -- 172.21.15.105:0/1565107208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeac0c78a0 con 0x7fdeac0cfb30 2023-11-28T18:32:22.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.436+0000 7fdeb33f8640 1 -- 172.21.15.105:0/1565107208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeac0c7e60 con 0x7fdeac0cfb30 2023-11-28T18:32:22.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.440+0000 7fdea2ffd640 1 -- 172.21.15.105:0/1565107208 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fdea8003b10 con 0x7fdeac0cfb30 2023-11-28T18:32:22.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.440+0000 7fdea2ffd640 1 --2- 172.21.15.105:0/1565107208 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fde900668d0 0x7fde90068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.440+0000 7fdea2ffd640 1 -- 172.21.15.105:0/1565107208 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdea8083b70 con 0x7fdeac0cfb30 2023-11-28T18:32:22.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.440+0000 7fdeb1bf5640 1 --2- 172.21.15.105:0/1565107208 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fde900668d0 0x7fde90068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.440+0000 7fdeb79d2000 1 Processor -- start 2023-11-28T18:32:22.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.440+0000 7fdeb1bf5640 1 --2- 172.21.15.105:0/1565107208 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fde900668d0 0x7fde90068d90 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0d0880 tx=0x7fde9c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.440+0000 7fdeb79d2000 1 -- start start 2023-11-28T18:32:22.443 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb79d2000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56413ee40d30 0x56413ee41110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.443 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb79d2000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413ee41650 0x56413edf3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.443 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb79d2000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413edf4500 0x56413edf48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb79d2000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413ecbd110 con 0x56413edf4500 2023-11-28T18:32:22.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb79d2000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413ec4d9a0 con 0x56413ee41650 2023-11-28T18:32:22.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb79d2000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x56413ec4aed0 con 0x56413ee40d30 2023-11-28T18:32:22.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb2bf7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413edf4500 0x56413edf48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb2bf7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413edf4500 0x56413edf48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56996/0 (socket says 172.21.15.105:56996) 2023-11-28T18:32:22.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb2bf7640 1 -- 172.21.15.105:0/1550043114 learned_addr learned my addr 172.21.15.105:0/1550043114 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:22.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb23f6640 1 --2- 172.21.15.105:0/1550043114 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56413ee40d30 0x56413ee41110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.444 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb1bf5640 1 --2- 172.21.15.105:0/1550043114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413ee41650 0x56413edf3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb2bf7640 1 -- 172.21.15.105:0/1550043114 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56413ee40d30 msgr2=0x56413ee41110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb2bf7640 1 --2- 172.21.15.105:0/1550043114 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56413ee40d30 0x56413ee41110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb2bf7640 1 -- 172.21.15.105:0/1550043114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413ee41650 msgr2=0x56413edf3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb2bf7640 1 --2- 172.21.15.105:0/1550043114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413ee41650 0x56413edf3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb2bf7640 1 -- 172.21.15.105:0/1550043114 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56413e93e890 con 0x56413edf4500 2023-11-28T18:32:22.445 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb2bf7640 1 --2- 172.21.15.105:0/1550043114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413edf4500 0x56413edf48e0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fdea40b0040 tx=0x7fdea4004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc4edf114b0d58e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fde8ffff640 1 -- 172.21.15.105:0/1550043114 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea40b8650 con 0x56413edf4500 2023-11-28T18:32:22.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fde8ffff640 1 -- 172.21.15.105:0/1550043114 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea40b87b0 con 0x56413edf4500 2023-11-28T18:32:22.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fde8ffff640 1 -- 172.21.15.105:0/1550043114 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea40b8b60 con 0x56413edf4500 2023-11-28T18:32:22.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb79d2000 1 -- 172.21.15.105:0/1550043114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413edf4500 msgr2=0x56413edf48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb79d2000 1 --2- 172.21.15.105:0/1550043114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413edf4500 0x56413edf48e0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fdea40b0040 tx=0x7fdea4004ac0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb79d2000 1 -- 172.21.15.105:0/1550043114 shutdown_connections 2023-11-28T18:32:22.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb79d2000 1 --2- 172.21.15.105:0/1550043114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413edf4500 0x56413edf48e0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb79d2000 1 --2- 172.21.15.105:0/1550043114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413ee41650 0x56413edf3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb79d2000 1 --2- 172.21.15.105:0/1550043114 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56413ee40d30 0x56413ee41110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb79d2000 1 -- 172.21.15.105:0/1550043114 >> 172.21.15.105:0/1550043114 conn(0x56413ee218c0 msgr2=0x56413ee23cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:22.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb79d2000 1 -- 172.21.15.105:0/1550043114 shutdown_connections 2023-11-28T18:32:22.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb79d2000 1 -- 172.21.15.105:0/1550043114 wait complete. 2023-11-28T18:32:22.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb79d2000 1 -- start start 2023-11-28T18:32:22.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.444+0000 7fdeb79d2000 1 Processor -- start 2023-11-28T18:32:22.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb79d2000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413edf4500 0x56413edfa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb79d2000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413ee40d30 0x56413edfaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb79d2000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56413ee41650 0x56413edfafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb79d2000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413ec467c0 con 0x56413edf4500 2023-11-28T18:32:22.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb79d2000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413ebfebf0 con 0x56413ee40d30 2023-11-28T18:32:22.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb79d2000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x56413ec04a90 con 0x56413ee41650 2023-11-28T18:32:22.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb79d2000 10 client.0 _ll_register_callbacks cb 0x56413ede1e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:32:22.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb23f6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413edf4500 0x56413edfa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb23f6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413edf4500 0x56413edfa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57004/0 (socket says 172.21.15.105:57004) 2023-11-28T18:32:22.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb23f6640 1 -- 172.21.15.105:0/2798915625 learned_addr learned my addr 172.21.15.105:0/2798915625 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:22.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb1bf5640 1 --2- 172.21.15.105:0/2798915625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413ee40d30 0x56413edfaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb2bf7640 1 --2- 172.21.15.105:0/2798915625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56413ee41650 0x56413edfafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb23f6640 1 -- 172.21.15.105:0/2798915625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56413ee41650 msgr2=0x56413edfafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb23f6640 1 --2- 172.21.15.105:0/2798915625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56413ee41650 0x56413edfafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb23f6640 1 -- 172.21.15.105:0/2798915625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413ee40d30 msgr2=0x56413edfaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb23f6640 1 --2- 172.21.15.105:0/2798915625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413ee40d30 0x56413edfaa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb23f6640 1 -- 172.21.15.105:0/2798915625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeac0cecd0 con 0x56413edf4500 2023-11-28T18:32:22.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb23f6640 1 --2- 172.21.15.105:0/2798915625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413edf4500 0x56413edfa520 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fdea8007fd0 tx=0x7fdea805ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bedd13c04f34e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.450 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fde8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:32:22.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fde8d7fa640 1 -- 172.21.15.105:0/2798915625 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8052e40 con 0x56413edf4500 2023-11-28T18:32:22.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fde8d7fa640 1 -- 172.21.15.105:0/2798915625 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdea806dc30 con 0x56413edf4500 2023-11-28T18:32:22.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb79d2000 1 -- 172.21.15.105:0/2798915625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56413ec08470 con 0x56413edf4500 2023-11-28T18:32:22.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fde8d7fa640 1 -- 172.21.15.105:0/2798915625 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdea806ddf0 con 0x56413edf4500 2023-11-28T18:32:22.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb79d2000 10 client.9414 fetch_fsmap learned FSMap version 140 2023-11-28T18:32:22.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb79d2000 1 -- 172.21.15.105:0/2798915625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=2+}) v3 -- 0x56413ec07190 con 0x56413edf4500 2023-11-28T18:32:22.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fde8d7fa640 1 -- 172.21.15.105:0/2798915625 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56413ec08470 con 0x56413edf4500 2023-11-28T18:32:22.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fde8d7fa640 1 -- 172.21.15.105:0/2798915625 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 140) v1 ==== 3318+0+0 (secure 0 0 0) 0x7fdea805c850 con 0x56413edf4500 2023-11-28T18:32:22.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb79d2000 10 client.9414 fetch_fsmap finished waiting for FSMap version 140 2023-11-28T18:32:22.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb79d2000 10 client.9414 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T18:32:22.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb79d2000 20 client.9414 populate_metadata read hostname 'smithi105' 2023-11-28T18:32:22.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb79d2000 1 --2- 172.21.15.105:0/2798915625 >> [v2:172.21.15.148:6833/3706038483,v1:172.21.15.148:6835/3706038483] conn(0x56413edfb9b0 0x56413ee52ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb79d2000 4 client.9414 mds_command: new command op to 9123 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:32:22.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb79d2000 1 -- 172.21.15.105:0/2798915625 --> [v2:172.21.15.148:6833/3706038483,v1:172.21.15.148:6835/3706038483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56413ec0bb50 con 0x56413edfb9b0 2023-11-28T18:32:22.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb1bf5640 1 --2- 172.21.15.105:0/2798915625 >> [v2:172.21.15.148:6833/3706038483,v1:172.21.15.148:6835/3706038483] conn(0x56413edfb9b0 0x56413ee52ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fdeb1bf5640 1 --2- 172.21.15.105:0/2798915625 >> [v2:172.21.15.148:6833/3706038483,v1:172.21.15.148:6835/3706038483] conn(0x56413edfb9b0 0x56413ee52ff0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f9cfb0577d6cf5b server_cookie=1170fb84db5e16fd in_seq=0 out_seq=0 2023-11-28T18:32:22.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.448+0000 7fde8d7fa640 10 client.9414 ms_handle_connect on v2:172.21.15.148:6833/3706038483 2023-11-28T18:32:22.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.452+0000 7fde8d7fa640 1 -- 172.21.15.105:0/2798915625 <== mds.1 v2:172.21.15.148:6833/3706038483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56413ec0bb50 con 0x56413edfb9b0 2023-11-28T18:32:22.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.452+0000 7fde8d7fa640 10 client.9414 handle_command_reply: tid=0 2023-11-28T18:32:22.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.452+0000 7fdeb79d2000 1 client.9414 shutdown 2023-11-28T18:32:22.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.452+0000 7fdeb79d2000 1 -- 172.21.15.105:0/2798915625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413edf4500 msgr2=0x56413edfa520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.452+0000 7fdeb79d2000 1 --2- 172.21.15.105:0/2798915625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413edf4500 0x56413edfa520 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fdea8007fd0 tx=0x7fdea805ce20 comp rx=0 tx=0).stop 2023-11-28T18:32:22.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.452+0000 7fdeb79d2000 1 -- 172.21.15.105:0/2798915625 shutdown_connections 2023-11-28T18:32:22.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.452+0000 7fdeb79d2000 1 --2- 172.21.15.105:0/2798915625 >> [v2:172.21.15.148:6833/3706038483,v1:172.21.15.148:6835/3706038483] conn(0x56413edfb9b0 0x56413ee52ff0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.452+0000 7fdeb79d2000 1 --2- 172.21.15.105:0/2798915625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56413ee41650 0x56413edfafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.452+0000 7fdeb79d2000 1 --2- 172.21.15.105:0/2798915625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413ee40d30 0x56413edfaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.452+0000 7fde8d7fa640 0 client.9414 ms_handle_reset on v2:172.21.15.148:6833/3706038483 2023-11-28T18:32:22.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.452+0000 7fdeb79d2000 1 --2- 172.21.15.105:0/2798915625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413edf4500 0x56413edfa520 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.452+0000 7fdeb79d2000 1 -- 172.21.15.105:0/2798915625 >> 172.21.15.105:0/2798915625 conn(0x56413ee218c0 msgr2=0x56413edf36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:22.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.452+0000 7fdeb79d2000 1 -- 172.21.15.105:0/2798915625 shutdown_connections 2023-11-28T18:32:22.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.452+0000 7fdeb79d2000 1 -- 172.21.15.105:0/2798915625 wait complete. 2023-11-28T18:32:22.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.452+0000 7fdeb79d2000 20 client.9414 trim_cache size 0 max 16384 2023-11-28T18:32:22.468 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.468+0000 7fdeb79d2000 1 Processor -- start 2023-11-28T18:32:22.469 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.468+0000 7fdeb79d2000 1 -- start start 2023-11-28T18:32:22.471 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.468+0000 7fdeb79d2000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56413edf4500 0x56413ee203a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.468+0000 7fdeb79d2000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413ee40d30 0x56413ee208e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.468+0000 7fdeb79d2000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413ee41650 0x56413ee52ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.468+0000 7fdeb79d2000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413e8ff940 con 0x56413ee40d30 2023-11-28T18:32:22.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.468+0000 7fdeb79d2000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413edfbfc0 con 0x56413ee41650 2023-11-28T18:32:22.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.468+0000 7fdeb79d2000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x56413ec1bb70 con 0x56413edf4500 2023-11-28T18:32:22.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.468+0000 7fdeb1bf5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413ee40d30 0x56413ee208e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.468+0000 7fdeb1bf5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413ee40d30 0x56413ee208e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57012/0 (socket says 172.21.15.105:57012) 2023-11-28T18:32:22.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.468+0000 7fdeb1bf5640 1 -- 172.21.15.105:0/768846035 learned_addr learned my addr 172.21.15.105:0/768846035 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:22.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.468+0000 7fdeb23f6640 1 --2- 172.21.15.105:0/768846035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56413edf4500 0x56413ee203a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.472 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.468+0000 7fdeb2bf7640 1 --2- 172.21.15.105:0/768846035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413ee41650 0x56413ee52ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.468+0000 7fdeb1bf5640 1 -- 172.21.15.105:0/768846035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56413edf4500 msgr2=0x56413ee203a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.468+0000 7fdeb1bf5640 1 --2- 172.21.15.105:0/768846035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56413edf4500 0x56413ee203a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.468+0000 7fdeb1bf5640 1 -- 172.21.15.105:0/768846035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413ee41650 msgr2=0x56413ee52ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.468+0000 7fdeb1bf5640 1 --2- 172.21.15.105:0/768846035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413ee41650 0x56413ee52ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.468+0000 7fdeb1bf5640 1 -- 172.21.15.105:0/768846035 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56413ebed490 con 0x56413ee40d30 2023-11-28T18:32:22.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.468+0000 7fdeb2bf7640 1 --2- 172.21.15.105:0/768846035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413ee41650 0x56413ee52ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:32:22.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.468+0000 7fdeb23f6640 1 --2- 172.21.15.105:0/768846035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56413edf4500 0x56413ee203a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:32:22.473 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.468+0000 7fdeb1bf5640 1 --2- 172.21.15.105:0/768846035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413ee40d30 0x56413ee208e0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fde9c007fd0 tx=0x7fde9c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2192ac1dc2797ec3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdea0ff9640 1 -- 172.21.15.105:0/768846035 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c0258d0 con 0x56413ee40d30 2023-11-28T18:32:22.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdea0ff9640 1 -- 172.21.15.105:0/768846035 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9c029070 con 0x56413ee40d30 2023-11-28T18:32:22.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdea0ff9640 1 -- 172.21.15.105:0/768846035 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c0075a0 con 0x56413ee40d30 2023-11-28T18:32:22.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb79d2000 1 -- 172.21.15.105:0/768846035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413ee40d30 msgr2=0x56413ee208e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb79d2000 1 --2- 172.21.15.105:0/768846035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413ee40d30 0x56413ee208e0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fde9c007fd0 tx=0x7fde9c011000 comp rx=0 tx=0).stop 2023-11-28T18:32:22.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb79d2000 1 -- 172.21.15.105:0/768846035 shutdown_connections 2023-11-28T18:32:22.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb79d2000 1 --2- 172.21.15.105:0/768846035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413ee41650 0x56413ee52ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb79d2000 1 --2- 172.21.15.105:0/768846035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413ee40d30 0x56413ee208e0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb79d2000 1 --2- 172.21.15.105:0/768846035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56413edf4500 0x56413ee203a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb79d2000 1 -- 172.21.15.105:0/768846035 >> 172.21.15.105:0/768846035 conn(0x56413ee218c0 msgr2=0x56413edf2720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:22.474 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb79d2000 1 -- 172.21.15.105:0/768846035 shutdown_connections 2023-11-28T18:32:22.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb79d2000 1 -- 172.21.15.105:0/768846035 wait complete. 2023-11-28T18:32:22.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb79d2000 1 -- start start 2023-11-28T18:32:22.475 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb79d2000 1 Processor -- start 2023-11-28T18:32:22.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb79d2000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413edf4500 0x56413eea1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb79d2000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56413ee40d30 0x56413eea1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb79d2000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413ee41650 0x56413ee5ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb79d2000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413ec05380 con 0x56413ee41650 2023-11-28T18:32:22.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb79d2000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413ec039b0 con 0x56413edf4500 2023-11-28T18:32:22.478 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb79d2000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x56413ec106c0 con 0x56413ee40d30 2023-11-28T18:32:22.479 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb79d2000 10 client.0 _ll_register_callbacks cb 0x56413ecbd060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:32:22.479 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb2bf7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413ee41650 0x56413ee5ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.479 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb23f6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413edf4500 0x56413eea1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.479 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb23f6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413edf4500 0x56413eea1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:43456/0 (socket says 172.21.15.105:43456) 2023-11-28T18:32:22.479 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb23f6640 1 -- 172.21.15.105:0/475029451 learned_addr learned my addr 172.21.15.105:0/475029451 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:22.479 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb1bf5640 1 --2- 172.21.15.105:0/475029451 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56413ee40d30 0x56413eea1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb2bf7640 1 -- 172.21.15.105:0/475029451 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56413ee40d30 msgr2=0x56413eea1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb2bf7640 1 --2- 172.21.15.105:0/475029451 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56413ee40d30 0x56413eea1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb2bf7640 1 -- 172.21.15.105:0/475029451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413edf4500 msgr2=0x56413eea1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb2bf7640 1 --2- 172.21.15.105:0/475029451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413edf4500 0x56413eea1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb2bf7640 1 -- 172.21.15.105:0/475029451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9c01c040 con 0x56413ee41650 2023-11-28T18:32:22.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.472+0000 7fdeb23f6640 1 --2- 172.21.15.105:0/475029451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413edf4500 0x56413eea1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:32:22.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb2bf7640 1 --2- 172.21.15.105:0/475029451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413ee41650 0x56413ee5ca70 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fdea401a4e0 tx=0x7fdea4004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c31a7e48f4c025 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fde8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:32:22.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fde8d7fa640 1 -- 172.21.15.105:0/475029451 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea40c8040 con 0x56413ee41650 2023-11-28T18:32:22.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fde8d7fa640 1 -- 172.21.15.105:0/475029451 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdea40cfc20 con 0x56413ee41650 2023-11-28T18:32:22.480 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb79d2000 1 -- 172.21.15.105:0/475029451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56413ec45110 con 0x56413ee41650 2023-11-28T18:32:22.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fde8d7fa640 1 -- 172.21.15.105:0/475029451 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdea4007000 con 0x56413ee41650 2023-11-28T18:32:22.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb79d2000 10 client.9420 fetch_fsmap learned FSMap version 140 2023-11-28T18:32:22.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb79d2000 1 -- 172.21.15.105:0/475029451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=2+}) v3 -- 0x56413e93ecc0 con 0x56413ee41650 2023-11-28T18:32:22.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fde8d7fa640 1 -- 172.21.15.105:0/475029451 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56413ec45110 con 0x56413ee41650 2023-11-28T18:32:22.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fde8d7fa640 1 -- 172.21.15.105:0/475029451 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 140) v1 ==== 3318+0+0 (secure 0 0 0) 0x7fdea40b8a60 con 0x56413ee41650 2023-11-28T18:32:22.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb79d2000 10 client.9420 fetch_fsmap finished waiting for FSMap version 140 2023-11-28T18:32:22.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb79d2000 10 client.9420 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T18:32:22.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb79d2000 20 client.9420 populate_metadata read hostname 'smithi105' 2023-11-28T18:32:22.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb79d2000 1 --2- 172.21.15.105:0/475029451 >> [v2:172.21.15.148:6833/3706038483,v1:172.21.15.148:6835/3706038483] conn(0x56413ee5d430 0x56413ee5d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb79d2000 4 client.9420 mds_command: new command op to 9123 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:32:22.481 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb79d2000 1 -- 172.21.15.105:0/475029451 --> [v2:172.21.15.148:6833/3706038483,v1:172.21.15.148:6835/3706038483] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56413ec73650 con 0x56413ee5d430 2023-11-28T18:32:22.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb23f6640 1 --2- 172.21.15.105:0/475029451 >> [v2:172.21.15.148:6833/3706038483,v1:172.21.15.148:6835/3706038483] conn(0x56413ee5d430 0x56413ee5d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb23f6640 1 --2- 172.21.15.105:0/475029451 >> [v2:172.21.15.148:6833/3706038483,v1:172.21.15.148:6835/3706038483] conn(0x56413ee5d430 0x56413ee5d810 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b44c51e22bf91775 server_cookie=9b5f29dd9bf11f17 in_seq=0 out_seq=0 2023-11-28T18:32:22.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fde8d7fa640 10 client.9420 ms_handle_connect on v2:172.21.15.148:6833/3706038483 2023-11-28T18:32:22.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fde8d7fa640 1 -- 172.21.15.105:0/475029451 <== mds.1 v2:172.21.15.148:6833/3706038483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x56413ec73650 con 0x56413ee5d430 2023-11-28T18:32:22.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fde8d7fa640 10 client.9420 handle_command_reply: tid=0 2023-11-28T18:32:22.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb79d2000 1 client.9420 shutdown 2023-11-28T18:32:22.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb79d2000 1 -- 172.21.15.105:0/475029451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413ee41650 msgr2=0x56413ee5ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb79d2000 1 --2- 172.21.15.105:0/475029451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413ee41650 0x56413ee5ca70 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fdea401a4e0 tx=0x7fdea4004a50 comp rx=0 tx=0).stop 2023-11-28T18:32:22.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb79d2000 1 -- 172.21.15.105:0/475029451 shutdown_connections 2023-11-28T18:32:22.482 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb79d2000 1 --2- 172.21.15.105:0/475029451 >> [v2:172.21.15.148:6833/3706038483,v1:172.21.15.148:6835/3706038483] conn(0x56413ee5d430 0x56413ee5d810 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb79d2000 1 --2- 172.21.15.105:0/475029451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56413ee41650 0x56413ee5ca70 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb79d2000 1 --2- 172.21.15.105:0/475029451 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56413ee40d30 0x56413eea1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb79d2000 1 --2- 172.21.15.105:0/475029451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56413edf4500 0x56413eea1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fde8d7fa640 0 client.9420 ms_handle_reset on v2:172.21.15.148:6833/3706038483 2023-11-28T18:32:22.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb79d2000 1 -- 172.21.15.105:0/475029451 >> 172.21.15.105:0/475029451 conn(0x56413ee218c0 msgr2=0x56413edf2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:22.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb79d2000 1 -- 172.21.15.105:0/475029451 shutdown_connections 2023-11-28T18:32:22.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb79d2000 1 -- 172.21.15.105:0/475029451 wait complete. 2023-11-28T18:32:22.483 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb79d2000 20 client.9420 trim_cache size 0 max 16384 2023-11-28T18:32:22.483 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:32:22.484 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:32:22.484 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 1, 2023-11-28T18:32:22.484 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 9123, 2023-11-28T18:32:22.484 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:32:22.484 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:32:22.484 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:32:22.484 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 5.2097504609999996, 2023-11-28T18:32:22.484 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 140, 2023-11-28T18:32:22.484 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 129, 2023-11-28T18:32:22.484 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 129, 2023-11-28T18:32:22.485 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 28.847428094000001 2023-11-28T18:32:22.485 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:32:22.485 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb33f8640 1 -- 172.21.15.105:0/1565107208 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fde900668d0 msgr2=0x7fde90068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb33f8640 1 --2- 172.21.15.105:0/1565107208 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fde900668d0 0x7fde90068d90 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0d0880 tx=0x7fde9c009040 comp rx=0 tx=0).stop 2023-11-28T18:32:22.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb33f8640 1 -- 172.21.15.105:0/1565107208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdeac0cfb30 msgr2=0x7fdeac0cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb33f8640 1 --2- 172.21.15.105:0/1565107208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdeac0cfb30 0x7fdeac0cbd00 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fdea800c4d0 tx=0x7fdea800c990 comp rx=0 tx=0).stop 2023-11-28T18:32:22.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb33f8640 1 -- 172.21.15.105:0/1565107208 shutdown_connections 2023-11-28T18:32:22.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb33f8640 1 --2- 172.21.15.105:0/1565107208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeac15d2d0 0x7fdeac0c6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb33f8640 1 --2- 172.21.15.105:0/1565107208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeac15cef0 0x7fdeac0c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb33f8640 1 --2- 172.21.15.105:0/1565107208 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fde900668d0 0x7fde90068d90 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb33f8640 1 --2- 172.21.15.105:0/1565107208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdeac0cfb30 0x7fdeac0cbd00 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.476+0000 7fdeb33f8640 1 -- 172.21.15.105:0/1565107208 >> 172.21.15.105:0/1565107208 conn(0x7fdeac0d2e10 msgr2=0x7fdeac0d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:22.486 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.480+0000 7fdeb33f8640 1 -- 172.21.15.105:0/1565107208 shutdown_connections 2023-11-28T18:32:22.487 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.480+0000 7fdeb33f8640 1 -- 172.21.15.105:0/1565107208 wait complete. 2023-11-28T18:32:22.496 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T18:32:22.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f6043c5f640 1 Processor -- start 2023-11-28T18:32:22.634 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f6043c5f640 1 -- start start 2023-11-28T18:32:22.635 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f6043c5f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f603c0d00b0 0x7f603c0cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f6043c5f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f603c0d3e60 con 0x7f603c0ceb40 2023-11-28T18:32:22.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f6043c5f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f603c0b5fe0 con 0x7f603c0b8a20 2023-11-28T18:32:22.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f6043c5f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f603c0b62c0 con 0x7f603c0d00b0 2023-11-28T18:32:22.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60421fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f603c0ceb40 0x7f603c0b6da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:53908/0 (socket says 172.21.15.105:53908) 2023-11-28T18:32:22.636 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60421fc640 1 -- 172.21.15.105:0/1130574859 learned_addr learned my addr 172.21.15.105:0/1130574859 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:32:22.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60411fa640 1 --2- 172.21.15.105:0/1130574859 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f603c0d00b0 0x7f603c0cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60409f9640 1 -- 172.21.15.105:0/1130574859 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 149848023 0 0) 0x7f603c0d3e60 con 0x7f603c0ceb40 2023-11-28T18:32:22.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60409f9640 1 -- 172.21.15.105:0/1130574859 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6028003530 con 0x7f603c0ceb40 2023-11-28T18:32:22.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60409f9640 1 -- 172.21.15.105:0/1130574859 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3627313537 0 0) 0x7f6028003530 con 0x7f603c0ceb40 2023-11-28T18:32:22.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60409f9640 1 -- 172.21.15.105:0/1130574859 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f603c0d00b0 msgr2=0x7f603c0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60409f9640 1 --2- 172.21.15.105:0/1130574859 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f603c0d00b0 0x7f603c0cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.638 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60409f9640 1 -- 172.21.15.105:0/1130574859 >> v1:172.21.15.148:6789/0 conn(0x7f603c0b8a20 legacy=0x7f603c0cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60409f9640 1 -- 172.21.15.105:0/1130574859 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c0b65a0 con 0x7f603c0ceb40 2023-11-28T18:32:22.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60409f9640 1 -- 172.21.15.105:0/1130574859 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f6034001a00 con 0x7f603c0ceb40 2023-11-28T18:32:22.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60409f9640 1 -- 172.21.15.105:0/1130574859 >> v1:172.21.15.105:6789/0 conn(0x7f603c0ceb40 legacy=0x7f603c0b6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60409f9640 1 --2- 172.21.15.105:0/1130574859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60280039b0 0x7f6028003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60409f9640 1 --2- 172.21.15.105:0/1130574859 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f603c0d00b0 0x7f60280047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60409f9640 1 --2- 172.21.15.105:0/1130574859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6028004d10 0x7f6028005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60409f9640 1 -- 172.21.15.105:0/1130574859 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c0b62c0 con 0x7f6028004d10 2023-11-28T18:32:22.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60409f9640 1 -- 172.21.15.105:0/1130574859 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602800ba40 con 0x7f60280039b0 2023-11-28T18:32:22.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60409f9640 1 -- 172.21.15.105:0/1130574859 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602800bbb0 con 0x7f603c0d00b0 2023-11-28T18:32:22.639 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60421fc640 1 --2- 172.21.15.105:0/1130574859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6028004d10 0x7f6028005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.640 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60419fb640 1 --2- 172.21.15.105:0/1130574859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60280039b0 0x7f6028003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.640 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60411fa640 1 --2- 172.21.15.105:0/1130574859 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f603c0d00b0 0x7f60280047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.640 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60421fc640 1 -- 172.21.15.105:0/1130574859 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f603c0d00b0 msgr2=0x7f60280047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.640 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60421fc640 1 --2- 172.21.15.105:0/1130574859 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f603c0d00b0 0x7f60280047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.640 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60421fc640 1 -- 172.21.15.105:0/1130574859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60280039b0 msgr2=0x7f6028003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.640 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60421fc640 1 --2- 172.21.15.105:0/1130574859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60280039b0 0x7f6028003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.640 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.636+0000 7f60421fc640 1 -- 172.21.15.105:0/1130574859 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6028003530 con 0x7f6028004d10 2023-11-28T18:32:22.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.640+0000 7f60421fc640 1 --2- 172.21.15.105:0/1130574859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6028004d10 0x7f6028005190 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f603400a810 tx=0x7f603401fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03462efddc87d5f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.640+0000 7f60409f9640 1 -- 172.21.15.105:0/1130574859 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60340ac680 con 0x7f6028004d10 2023-11-28T18:32:22.641 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.640+0000 7f60409f9640 1 -- 172.21.15.105:0/1130574859 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60340ac7e0 con 0x7f6028004d10 2023-11-28T18:32:22.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.640+0000 7f60409f9640 1 -- 172.21.15.105:0/1130574859 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60340acb90 con 0x7f6028004d10 2023-11-28T18:32:22.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.640+0000 7f6043c5f640 1 -- 172.21.15.105:0/1130574859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6028004d10 msgr2=0x7f6028005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.640+0000 7f6043c5f640 1 --2- 172.21.15.105:0/1130574859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6028004d10 0x7f6028005190 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f603400a810 tx=0x7f603401fa60 comp rx=0 tx=0).stop 2023-11-28T18:32:22.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.640+0000 7f60421fc640 1 -- 172.21.15.105:0/1130574859 reap_dead start 2023-11-28T18:32:22.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.640+0000 7f6043c5f640 1 -- 172.21.15.105:0/1130574859 shutdown_connections 2023-11-28T18:32:22.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.640+0000 7f6043c5f640 1 -- 172.21.15.105:0/1130574859 >> 172.21.15.105:0/1130574859 conn(0x7f603c09c340 msgr2=0x7f603c0d2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:22.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.640+0000 7f6043c5f640 1 -- 172.21.15.105:0/1130574859 shutdown_connections 2023-11-28T18:32:22.642 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.640+0000 7f6043c5f640 1 -- 172.21.15.105:0/1130574859 wait complete. 2023-11-28T18:32:22.643 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f6043c5f640 1 Processor -- start 2023-11-28T18:32:22.643 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f6043c5f640 1 -- start start 2023-11-28T18:32:22.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f6043c5f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6028004d10 0x7f603c1607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f6043c5f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f603c1f6d10 0x7f603c1559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f6043c5f640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f603c15db30 0x7f603c1f6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f6043c5f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c151d40 con 0x7f603c1f6d10 2023-11-28T18:32:22.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f6043c5f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c155f00 con 0x7f6028004d10 2023-11-28T18:32:22.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f6043c5f640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f603c156120 con 0x7f603c15db30 2023-11-28T18:32:22.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f60419fb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6028004d10 0x7f603c1607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.645 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f60411fa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f603c1f6d10 0x7f603c1559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f60421fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f603c15db30 0x7f603c1f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f60411fa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f603c1f6d10 0x7f603c1559c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57034/0 (socket says 172.21.15.105:57034) 2023-11-28T18:32:22.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f60411fa640 1 -- 172.21.15.105:0/95017652 learned_addr learned my addr 172.21.15.105:0/95017652 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:22.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f60419fb640 1 --2- 172.21.15.105:0/95017652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6028004d10 0x7f603c1607e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:43480/0 (socket says 172.21.15.105:43480) 2023-11-28T18:32:22.646 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f60411fa640 1 -- 172.21.15.105:0/95017652 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f603c15db30 msgr2=0x7f603c1f6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f60411fa640 1 --2- 172.21.15.105:0/95017652 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f603c15db30 0x7f603c1f6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f60411fa640 1 -- 172.21.15.105:0/95017652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6028004d10 msgr2=0x7f603c1607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f60411fa640 1 --2- 172.21.15.105:0/95017652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6028004d10 0x7f603c1607e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f60411fa640 1 -- 172.21.15.105:0/95017652 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c156400 con 0x7f603c1f6d10 2023-11-28T18:32:22.647 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f60411fa640 1 --2- 172.21.15.105:0/95017652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f603c1f6d10 0x7f603c1559c0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f602c011610 tx=0x7f602c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f60327fc640 1 -- 172.21.15.105:0/95017652 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c015870 con 0x7f603c1f6d10 2023-11-28T18:32:22.648 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f60327fc640 1 -- 172.21.15.105:0/95017652 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602c0159d0 con 0x7f603c1f6d10 2023-11-28T18:32:22.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f60327fc640 1 -- 172.21.15.105:0/95017652 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c015d40 con 0x7f603c1f6d10 2023-11-28T18:32:22.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f6043c5f640 1 -- 172.21.15.105:0/95017652 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f603c1566e0 con 0x7f603c1f6d10 2023-11-28T18:32:22.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.644+0000 7f6043c5f640 1 -- 172.21.15.105:0/95017652 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603c158e30 con 0x7f603c1f6d10 2023-11-28T18:32:22.649 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.648+0000 7f60327fc640 1 -- 172.21.15.105:0/95017652 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f602c00f480 con 0x7f603c1f6d10 2023-11-28T18:32:22.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.648+0000 7f60327fc640 1 --2- 172.21.15.105:0/95017652 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f60200668d0 0x7f6020068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.648+0000 7f60327fc640 1 -- 172.21.15.105:0/95017652 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f602c084950 con 0x7f603c1f6d10 2023-11-28T18:32:22.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.648+0000 7f60419fb640 1 --2- 172.21.15.105:0/95017652 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f60200668d0 0x7f6020068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.648+0000 7f6046fca000 1 Processor -- start 2023-11-28T18:32:22.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.648+0000 7f6046fca000 1 -- start start 2023-11-28T18:32:22.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.648+0000 7f60419fb640 1 --2- 172.21.15.105:0/95017652 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f60200668d0 0x7f6020068d90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6038000c10 tx=0x7f6038009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f6046fca000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2671add30 0x55e2671ae110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f6046fca000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2671ae650 0x55e267160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.652 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f6046fca000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e267161500 0x55e2671618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f6046fca000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26702a110 con 0x55e2671ae650 2023-11-28T18:32:22.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f6046fca000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e266fba9a0 con 0x55e2671add30 2023-11-28T18:32:22.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f6046fca000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e266fb7ed0 con 0x55e267161500 2023-11-28T18:32:22.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f60411fa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2671ae650 0x55e267160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f60411fa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2671ae650 0x55e267160fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57040/0 (socket says 172.21.15.105:57040) 2023-11-28T18:32:22.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f60411fa640 1 -- 172.21.15.105:0/2098536531 learned_addr learned my addr 172.21.15.105:0/2098536531 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:22.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f60421fc640 1 --2- 172.21.15.105:0/2098536531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e267161500 0x55e2671618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f60411fa640 1 -- 172.21.15.105:0/2098536531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e267161500 msgr2=0x55e2671618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f60411fa640 1 --2- 172.21.15.105:0/2098536531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e267161500 0x55e2671618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f60411fa640 1 -- 172.21.15.105:0/2098536531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2671add30 msgr2=0x55e2671ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.653 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f60419fb640 1 --2- 172.21.15.105:0/2098536531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2671add30 0x55e2671ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f60411fa640 1 --2- 172.21.15.105:0/2098536531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2671add30 0x55e2671ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f60411fa640 1 -- 172.21.15.105:0/2098536531 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e266cab890 con 0x55e2671ae650 2023-11-28T18:32:22.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f60421fc640 1 --2- 172.21.15.105:0/2098536531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e267161500 0x55e2671618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:32:22.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f60419fb640 1 --2- 172.21.15.105:0/2098536531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2671add30 0x55e2671ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:32:22.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f60411fa640 1 --2- 172.21.15.105:0/2098536531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2671ae650 0x55e267160fc0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f602c05d440 tx=0x7f602c0557e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2209ceb6dc955 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f601f7fe640 1 -- 172.21.15.105:0/2098536531 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c053ac0 con 0x55e2671ae650 2023-11-28T18:32:22.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f601f7fe640 1 -- 172.21.15.105:0/2098536531 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602c051e30 con 0x55e2671ae650 2023-11-28T18:32:22.654 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f601f7fe640 1 -- 172.21.15.105:0/2098536531 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c05d720 con 0x55e2671ae650 2023-11-28T18:32:22.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f6046fca000 1 -- 172.21.15.105:0/2098536531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2671ae650 msgr2=0x55e267160fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f6046fca000 1 --2- 172.21.15.105:0/2098536531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2671ae650 0x55e267160fc0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f602c05d440 tx=0x7f602c0557e0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.658 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f6046fca000 1 -- 172.21.15.105:0/2098536531 shutdown_connections 2023-11-28T18:32:22.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f6046fca000 1 --2- 172.21.15.105:0/2098536531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e267161500 0x55e2671618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f6046fca000 1 --2- 172.21.15.105:0/2098536531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2671ae650 0x55e267160fc0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f6046fca000 1 --2- 172.21.15.105:0/2098536531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2671add30 0x55e2671ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f6046fca000 1 -- 172.21.15.105:0/2098536531 >> 172.21.15.105:0/2098536531 conn(0x55e26718e8c0 msgr2=0x55e267190cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:22.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f6046fca000 1 -- 172.21.15.105:0/2098536531 shutdown_connections 2023-11-28T18:32:22.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f6046fca000 1 -- 172.21.15.105:0/2098536531 wait complete. 2023-11-28T18:32:22.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f6046fca000 1 -- start start 2023-11-28T18:32:22.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.652+0000 7f6046fca000 1 Processor -- start 2023-11-28T18:32:22.659 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f6046fca000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e267161500 0x55e267167520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f6046fca000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e2671add30 0x55e267167a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f6046fca000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2671ae650 0x55e267167fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f6046fca000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e266fb37c0 con 0x55e2671ae650 2023-11-28T18:32:22.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f6046fca000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e266f6bbf0 con 0x55e267161500 2023-11-28T18:32:22.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f6046fca000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e266f71a90 con 0x55e2671add30 2023-11-28T18:32:22.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f6046fca000 10 client.0 _ll_register_callbacks cb 0x55e26714ee20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:32:22.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f60421fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2671ae650 0x55e267167fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f60421fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2671ae650 0x55e267167fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57046/0 (socket says 172.21.15.105:57046) 2023-11-28T18:32:22.661 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f60421fc640 1 -- 172.21.15.105:0/714347108 learned_addr learned my addr 172.21.15.105:0/714347108 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:22.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f60419fb640 1 --2- 172.21.15.105:0/714347108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e267161500 0x55e267167520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f60411fa640 1 --2- 172.21.15.105:0/714347108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e2671add30 0x55e267167a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f60421fc640 1 -- 172.21.15.105:0/714347108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e2671add30 msgr2=0x55e267167a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f60421fc640 1 --2- 172.21.15.105:0/714347108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e2671add30 0x55e267167a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f60421fc640 1 -- 172.21.15.105:0/714347108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e267161500 msgr2=0x55e267167520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f60421fc640 1 --2- 172.21.15.105:0/714347108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e267161500 0x55e267167520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f60421fc640 1 -- 172.21.15.105:0/714347108 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603c158e30 con 0x55e2671ae650 2023-11-28T18:32:22.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f60411fa640 1 --2- 172.21.15.105:0/714347108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e2671add30 0x55e267167a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:32:22.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f60419fb640 1 --2- 172.21.15.105:0/714347108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e267161500 0x55e267167520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:32:22.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f60421fc640 1 --2- 172.21.15.105:0/714347108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2671ae650 0x55e267167fa0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f60340ae040 tx=0x7f603401c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9114e92d235fce5e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.662 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f601cff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:32:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f601cff9640 1 -- 172.21.15.105:0/714347108 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60340bd070 con 0x55e2671ae650 2023-11-28T18:32:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f6046fca000 1 -- 172.21.15.105:0/714347108 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e266f75470 con 0x55e2671ae650 2023-11-28T18:32:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f601cff9640 1 -- 172.21.15.105:0/714347108 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f60340ac950 con 0x55e2671ae650 2023-11-28T18:32:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f601cff9640 1 -- 172.21.15.105:0/714347108 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6034001740 con 0x55e2671ae650 2023-11-28T18:32:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f6046fca000 10 client.9432 fetch_fsmap learned FSMap version 140 2023-11-28T18:32:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f6046fca000 1 -- 172.21.15.105:0/714347108 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=2+}) v3 -- 0x55e266f74190 con 0x55e2671ae650 2023-11-28T18:32:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f601cff9640 1 -- 172.21.15.105:0/714347108 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e266f75470 con 0x55e2671ae650 2023-11-28T18:32:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f601cff9640 1 -- 172.21.15.105:0/714347108 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 140) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f6034004b70 con 0x55e2671ae650 2023-11-28T18:32:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f6046fca000 10 client.9432 fetch_fsmap finished waiting for FSMap version 140 2023-11-28T18:32:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f6046fca000 10 client.9432 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:32:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f6046fca000 20 client.9432 populate_metadata read hostname 'smithi105' 2023-11-28T18:32:22.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f6046fca000 1 --2- 172.21.15.105:0/714347108 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55e2671689b0 0x55e2671bfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f6046fca000 4 client.9432 mds_command: new command op to 9126 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T18:32:22.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f6046fca000 1 -- 172.21.15.105:0/714347108 --> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e266f78b50 con 0x55e2671689b0 2023-11-28T18:32:22.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f60419fb640 1 --2- 172.21.15.105:0/714347108 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55e2671689b0 0x55e2671bfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f601cff9640 10 client.9432 ms_handle_connect on v2:172.21.15.105:6836/1978722810 2023-11-28T18:32:22.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f60419fb640 1 --2- 172.21.15.105:0/714347108 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55e2671689b0 0x55e2671bfff0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffae3a4e9037203d server_cookie=501fe456fb11deef in_seq=0 out_seq=0 2023-11-28T18:32:22.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f601cff9640 1 -- 172.21.15.105:0/714347108 <== mds.0 v2:172.21.15.105:6836/1978722810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e266f78b50 con 0x55e2671689b0 2023-11-28T18:32:22.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.656+0000 7f601cff9640 10 client.9432 handle_command_reply: tid=0 2023-11-28T18:32:22.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.660+0000 7f6046fca000 1 client.9432 shutdown 2023-11-28T18:32:22.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.660+0000 7f6046fca000 1 -- 172.21.15.105:0/714347108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2671ae650 msgr2=0x55e267167fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.660+0000 7f6046fca000 1 --2- 172.21.15.105:0/714347108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2671ae650 0x55e267167fa0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f60340ae040 tx=0x7f603401c800 comp rx=0 tx=0).stop 2023-11-28T18:32:22.664 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.660+0000 7f6046fca000 1 -- 172.21.15.105:0/714347108 shutdown_connections 2023-11-28T18:32:22.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.660+0000 7f6046fca000 1 --2- 172.21.15.105:0/714347108 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55e2671689b0 0x55e2671bfff0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.660+0000 7f6046fca000 1 --2- 172.21.15.105:0/714347108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2671ae650 0x55e267167fa0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.660+0000 7f6046fca000 1 --2- 172.21.15.105:0/714347108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e2671add30 0x55e267167a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.660+0000 7f6046fca000 1 --2- 172.21.15.105:0/714347108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e267161500 0x55e267167520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.660+0000 7f6046fca000 1 -- 172.21.15.105:0/714347108 >> 172.21.15.105:0/714347108 conn(0x55e26718e8c0 msgr2=0x55e2671606b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:22.665 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.660+0000 7f601cff9640 0 client.9432 ms_handle_reset on v2:172.21.15.105:6836/1978722810 2023-11-28T18:32:22.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.660+0000 7f6046fca000 1 -- 172.21.15.105:0/714347108 shutdown_connections 2023-11-28T18:32:22.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.660+0000 7f6046fca000 1 -- 172.21.15.105:0/714347108 wait complete. 2023-11-28T18:32:22.666 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.660+0000 7f6046fca000 20 client.9432 trim_cache size 0 max 16384 2023-11-28T18:32:22.674 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.672+0000 7f6046fca000 1 Processor -- start 2023-11-28T18:32:22.674 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f6046fca000 1 -- start start 2023-11-28T18:32:22.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f6046fca000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e267161500 0x55e26718d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f6046fca000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2671add30 0x55e26718d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f6046fca000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2671ae650 0x55e2671bfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.675 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f6046fca000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e266c6c940 con 0x55e2671add30 2023-11-28T18:32:22.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f6046fca000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2671c4ce0 con 0x55e2671ae650 2023-11-28T18:32:22.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f6046fca000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e26716b510 con 0x55e267161500 2023-11-28T18:32:22.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f60411fa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2671add30 0x55e26718d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f60411fa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2671add30 0x55e26718d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57054/0 (socket says 172.21.15.105:57054) 2023-11-28T18:32:22.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f60411fa640 1 -- 172.21.15.105:0/4078033435 learned_addr learned my addr 172.21.15.105:0/4078033435 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:22.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f60411fa640 1 -- 172.21.15.105:0/4078033435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e267161500 msgr2=0x55e26718d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f60411fa640 1 --2- 172.21.15.105:0/4078033435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e267161500 0x55e26718d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f60411fa640 1 -- 172.21.15.105:0/4078033435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2671ae650 msgr2=0x55e2671bfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f60411fa640 1 --2- 172.21.15.105:0/4078033435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2671ae650 0x55e2671bfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.676 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f60411fa640 1 -- 172.21.15.105:0/4078033435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e267161370 con 0x55e2671add30 2023-11-28T18:32:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f60411fa640 1 --2- 172.21.15.105:0/4078033435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2671add30 0x55e26718d8e0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f602c05d440 tx=0x7f602c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d958aef7352ea97a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f601ffff640 1 -- 172.21.15.105:0/4078033435 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c051a60 con 0x55e2671add30 2023-11-28T18:32:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f601ffff640 1 -- 172.21.15.105:0/4078033435 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602c055260 con 0x55e2671add30 2023-11-28T18:32:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f601ffff640 1 -- 172.21.15.105:0/4078033435 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c051e30 con 0x55e2671add30 2023-11-28T18:32:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f6046fca000 1 -- 172.21.15.105:0/4078033435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2671add30 msgr2=0x55e26718d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f6046fca000 1 --2- 172.21.15.105:0/4078033435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2671add30 0x55e26718d8e0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f602c05d440 tx=0x7f602c01f000 comp rx=0 tx=0).stop 2023-11-28T18:32:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f6046fca000 1 -- 172.21.15.105:0/4078033435 shutdown_connections 2023-11-28T18:32:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f6046fca000 1 --2- 172.21.15.105:0/4078033435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2671ae650 0x55e2671bfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f6046fca000 1 --2- 172.21.15.105:0/4078033435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2671add30 0x55e26718d8e0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f6046fca000 1 --2- 172.21.15.105:0/4078033435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e267161500 0x55e26718d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f6046fca000 1 -- 172.21.15.105:0/4078033435 >> 172.21.15.105:0/4078033435 conn(0x55e26718e8c0 msgr2=0x55e26715f720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f6046fca000 1 -- 172.21.15.105:0/4078033435 shutdown_connections 2023-11-28T18:32:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f6046fca000 1 -- 172.21.15.105:0/4078033435 wait complete. 2023-11-28T18:32:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f6046fca000 1 -- start start 2023-11-28T18:32:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.676+0000 7f6046fca000 1 Processor -- start 2023-11-28T18:32:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e267161500 0x55e267217020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2671add30 0x55e267217560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e2671ae650 0x55e2671d2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e266f4e8a0 con 0x55e267161500 2023-11-28T18:32:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e266fb9f30 con 0x55e2671add30 2023-11-28T18:32:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e266f7d6c0 con 0x55e2671ae650 2023-11-28T18:32:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 10 client.0 _ll_register_callbacks cb 0x55e26702a060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T18:32:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f60419fb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e267161500 0x55e267217020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f60419fb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e267161500 0x55e267217020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57064/0 (socket says 172.21.15.105:57064) 2023-11-28T18:32:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f60419fb640 1 -- 172.21.15.105:0/2380142049 learned_addr learned my addr 172.21.15.105:0/2380142049 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f60421fc640 1 --2- 172.21.15.105:0/2380142049 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e2671ae650 0x55e2671d2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f60419fb640 1 -- 172.21.15.105:0/2380142049 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e2671ae650 msgr2=0x55e2671d2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f60419fb640 1 --2- 172.21.15.105:0/2380142049 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e2671ae650 0x55e2671d2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f60411fa640 1 --2- 172.21.15.105:0/2380142049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2671add30 0x55e267217560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f60419fb640 1 -- 172.21.15.105:0/2380142049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2671add30 msgr2=0x55e267217560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f60419fb640 1 --2- 172.21.15.105:0/2380142049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2671add30 0x55e267217560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f60419fb640 1 -- 172.21.15.105:0/2380142049 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e266cab890 con 0x55e267161500 2023-11-28T18:32:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f60421fc640 1 --2- 172.21.15.105:0/2380142049 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e2671ae650 0x55e2671d2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:32:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f60419fb640 1 --2- 172.21.15.105:0/2380142049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e267161500 0x55e267217020 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f6038012f90 tx=0x7f6038007cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c054f44d126282 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f601cff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2023-11-28T18:32:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f601cff9640 1 -- 172.21.15.105:0/2380142049 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6038028b00 con 0x55e267161500 2023-11-28T18:32:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f601cff9640 1 -- 172.21.15.105:0/2380142049 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6038028c60 con 0x55e267161500 2023-11-28T18:32:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 1 -- 172.21.15.105:0/2380142049 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e266fb2110 con 0x55e267161500 2023-11-28T18:32:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f601cff9640 1 -- 172.21.15.105:0/2380142049 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6038028e20 con 0x55e267161500 2023-11-28T18:32:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 10 client.9438 fetch_fsmap learned FSMap version 140 2023-11-28T18:32:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 1 -- 172.21.15.105:0/2380142049 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=2+}) v3 -- 0x55e266f5a490 con 0x55e267161500 2023-11-28T18:32:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f601cff9640 1 -- 172.21.15.105:0/2380142049 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e266fb2110 con 0x55e267161500 2023-11-28T18:32:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f601cff9640 1 -- 172.21.15.105:0/2380142049 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 140) v1 ==== 3318+0+0 (secure 0 0 0) 0x7f6038022450 con 0x55e267161500 2023-11-28T18:32:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 10 client.9438 fetch_fsmap finished waiting for FSMap version 140 2023-11-28T18:32:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 10 client.9438 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T18:32:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 20 client.9438 populate_metadata read hostname 'smithi105' 2023-11-28T18:32:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 1 --2- 172.21.15.105:0/2380142049 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55e26721de80 0x55e267220270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 4 client.9438 mds_command: new command op to 9126 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T18:32:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 1 -- 172.21.15.105:0/2380142049 --> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e266fe0650 con 0x55e26721de80 2023-11-28T18:32:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f60411fa640 1 --2- 172.21.15.105:0/2380142049 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55e26721de80 0x55e267220270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f601cff9640 10 client.9438 ms_handle_connect on v2:172.21.15.105:6836/1978722810 2023-11-28T18:32:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f60411fa640 1 --2- 172.21.15.105:0/2380142049 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55e26721de80 0x55e267220270 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc77df8b520dcf56 server_cookie=b09e407a08bfd39f in_seq=0 out_seq=0 2023-11-28T18:32:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f601cff9640 1 -- 172.21.15.105:0/2380142049 <== mds.0 v2:172.21.15.105:6836/1978722810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55e266fe0650 con 0x55e26721de80 2023-11-28T18:32:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f601cff9640 10 client.9438 handle_command_reply: tid=0 2023-11-28T18:32:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 1 client.9438 shutdown 2023-11-28T18:32:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 1 -- 172.21.15.105:0/2380142049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e267161500 msgr2=0x55e267217020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 1 --2- 172.21.15.105:0/2380142049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e267161500 0x55e267217020 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f6038012f90 tx=0x7f6038007cb0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 1 -- 172.21.15.105:0/2380142049 shutdown_connections 2023-11-28T18:32:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 1 --2- 172.21.15.105:0/2380142049 >> [v2:172.21.15.105:6836/1978722810,v1:172.21.15.105:6837/1978722810] conn(0x55e26721de80 0x55e267220270 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 1 --2- 172.21.15.105:0/2380142049 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e2671ae650 0x55e2671d2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 1 --2- 172.21.15.105:0/2380142049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2671add30 0x55e267217560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f601cff9640 0 client.9438 ms_handle_reset on v2:172.21.15.105:6836/1978722810 2023-11-28T18:32:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 1 --2- 172.21.15.105:0/2380142049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e267161500 0x55e267217020 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 1 -- 172.21.15.105:0/2380142049 >> 172.21.15.105:0/2380142049 conn(0x55e26718e8c0 msgr2=0x55e26715ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 1 -- 172.21.15.105:0/2380142049 shutdown_connections 2023-11-28T18:32:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 1 -- 172.21.15.105:0/2380142049 wait complete. 2023-11-28T18:32:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.680+0000 7f6046fca000 20 client.9438 trim_cache size 0 max 16384 2023-11-28T18:32:22.688 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-11-28T18:32:22.688 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "b302bec5-10e5-4c91-9026-ecb69250e81c", 2023-11-28T18:32:22.688 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2023-11-28T18:32:22.688 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 9126, 2023-11-28T18:32:22.688 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2023-11-28T18:32:22.688 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2023-11-28T18:32:22.688 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2023-11-28T18:32:22.688 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 23.217904396000002, 2023-11-28T18:32:22.688 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 140, 2023-11-28T18:32:22.689 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 129, 2023-11-28T18:32:22.689 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 129, 2023-11-28T18:32:22.689 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 29.047636316999998 2023-11-28T18:32:22.689 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-11-28T18:32:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.684+0000 7f6043c5f640 1 -- 172.21.15.105:0/95017652 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f60200668d0 msgr2=0x7f6020068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.684+0000 7f6043c5f640 1 --2- 172.21.15.105:0/95017652 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f60200668d0 0x7f6020068d90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6038000c10 tx=0x7f6038009040 comp rx=0 tx=0).stop 2023-11-28T18:32:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.684+0000 7f6043c5f640 1 -- 172.21.15.105:0/95017652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f603c1f6d10 msgr2=0x7f603c1559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.684+0000 7f6043c5f640 1 --2- 172.21.15.105:0/95017652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f603c1f6d10 0x7f603c1559c0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f602c011610 tx=0x7f602c011ad0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.684+0000 7f6043c5f640 1 -- 172.21.15.105:0/95017652 shutdown_connections 2023-11-28T18:32:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.684+0000 7f6043c5f640 1 --2- 172.21.15.105:0/95017652 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f603c15db30 0x7f603c1f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.684+0000 7f6043c5f640 1 --2- 172.21.15.105:0/95017652 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f60200668d0 0x7f6020068d90 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.684+0000 7f6043c5f640 1 --2- 172.21.15.105:0/95017652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f603c1f6d10 0x7f603c1559c0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.684+0000 7f6043c5f640 1 --2- 172.21.15.105:0/95017652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6028004d10 0x7f603c1607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.684+0000 7f6043c5f640 1 -- 172.21.15.105:0/95017652 >> 172.21.15.105:0/95017652 conn(0x7f603c09c340 msgr2=0x7f603c0d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.684+0000 7f6043c5f640 1 -- 172.21.15.105:0/95017652 shutdown_connections 2023-11-28T18:32:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.684+0000 7f6043c5f640 1 -- 172.21.15.105:0/95017652 wait complete. 2023-11-28T18:32:22.701 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_distributed_factor 42.333333333333336 2023-11-28T18:32:22.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.840+0000 7f3d49279640 1 Processor -- start 2023-11-28T18:32:22.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.840+0000 7f3d49279640 1 -- start start 2023-11-28T18:32:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.844+0000 7f3d49279640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3d4415d510 0x7f3d440d3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.844+0000 7f3d49279640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d440cf960 0x7f3d440cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.844+0000 7f3d49279640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3d440b30d0 0x7f3d440b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.844+0000 7f3d49279640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d4415d8f0 con 0x7f3d4415d510 2023-11-28T18:32:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.844+0000 7f3d49279640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d440d0300 con 0x7f3d440b30d0 2023-11-28T18:32:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.844+0000 7f3d49279640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d440cf3e0 con 0x7f3d440cf960 2023-11-28T18:32:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.844+0000 7f3d42ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3d4415d510 0x7f3d440d3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.844+0000 7f3d42ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3d4415d510 0x7f3d440d3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57068/0 (socket says 172.21.15.105:57068) 2023-11-28T18:32:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.844+0000 7f3d42ffd640 1 -- 172.21.15.105:0/1349779543 learned_addr learned my addr 172.21.15.105:0/1349779543 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.844+0000 7f3d42ffd640 1 -- 172.21.15.105:0/1349779543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d440cf960 msgr2=0x7f3d440cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.844+0000 7f3d3a7fc640 1 --2- 172.21.15.105:0/1349779543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d440cf960 0x7f3d440cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.844+0000 7f3d437fe640 1 --2- 172.21.15.105:0/1349779543 >> v2:172.21.15.148:3300/0 conn(0x7f3d440b30d0 0x7f3d440b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.844+0000 7f3d42ffd640 1 --2- 172.21.15.105:0/1349779543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d440cf960 0x7f3d440cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.844+0000 7f3d42ffd640 1 -- 172.21.15.105:0/1349779543 >> v2:172.21.15.148:3300/0 conn(0x7f3d440b30d0 msgr2=0x7f3d440b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:32:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.844+0000 7f3d42ffd640 1 --2- 172.21.15.105:0/1349779543 >> v2:172.21.15.148:3300/0 conn(0x7f3d440b30d0 0x7f3d440b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.844+0000 7f3d42ffd640 1 -- 172.21.15.105:0/1349779543 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d440b25f0 con 0x7f3d4415d510 2023-11-28T18:32:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.844+0000 7f3d3a7fc640 1 --2- 172.21.15.105:0/1349779543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d440cf960 0x7f3d440cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:32:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.844+0000 7f3d437fe640 1 --2- 172.21.15.105:0/1349779543 >> v2:172.21.15.148:3300/0 conn(0x7f3d440b30d0 0x7f3d440b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:32:22.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.848+0000 7f3d42ffd640 1 --2- 172.21.15.105:0/1349779543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4415d510 0x7f3d440d3b40 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f3d440ca0b0 tx=0x7f3d3401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80133558d9dc03ff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.848+0000 7f3d427fc640 1 -- 172.21.15.105:0/1349779543 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d340aa5f0 con 0x7f3d4415d510 2023-11-28T18:32:22.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.848+0000 7f3d427fc640 1 -- 172.21.15.105:0/1349779543 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d340aa750 con 0x7f3d4415d510 2023-11-28T18:32:22.847 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.848+0000 7f3d427fc640 1 -- 172.21.15.105:0/1349779543 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d340aaae0 con 0x7f3d4415d510 2023-11-28T18:32:22.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.848+0000 7f3d49279640 1 -- 172.21.15.105:0/1349779543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4415d510 msgr2=0x7f3d440d3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.848+0000 7f3d49279640 1 --2- 172.21.15.105:0/1349779543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4415d510 0x7f3d440d3b40 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f3d440ca0b0 tx=0x7f3d3401c4d0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.848+0000 7f3d49279640 1 -- 172.21.15.105:0/1349779543 shutdown_connections 2023-11-28T18:32:22.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.848+0000 7f3d49279640 1 --2- 172.21.15.105:0/1349779543 >> v2:172.21.15.148:3300/0 conn(0x7f3d440b30d0 0x7f3d440b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.848+0000 7f3d49279640 1 --2- 172.21.15.105:0/1349779543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d440cf960 0x7f3d440cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.848+0000 7f3d49279640 1 --2- 172.21.15.105:0/1349779543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4415d510 0x7f3d440d3b40 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.848+0000 7f3d49279640 1 -- 172.21.15.105:0/1349779543 >> 172.21.15.105:0/1349779543 conn(0x7f3d440d3050 msgr2=0x7f3d440d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:22.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.848+0000 7f3d49279640 1 -- 172.21.15.105:0/1349779543 shutdown_connections 2023-11-28T18:32:22.848 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.848+0000 7f3d49279640 1 -- 172.21.15.105:0/1349779543 wait complete. 2023-11-28T18:32:22.849 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.848+0000 7f3d49279640 1 Processor -- start 2023-11-28T18:32:22.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.848+0000 7f3d49279640 1 -- start start 2023-11-28T18:32:22.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d49279640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d440b30d0 0x7f3d4406c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d49279640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d440cf960 0x7f3d4406c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d49279640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4415d510 0x7f3d44157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.852 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d49279640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4415f8a0 con 0x7f3d4415d510 2023-11-28T18:32:22.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d49279640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4415ec20 con 0x7f3d440b30d0 2023-11-28T18:32:22.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d49279640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4415ef00 con 0x7f3d440cf960 2023-11-28T18:32:22.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d437fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4415d510 0x7f3d44157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.853 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d437fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4415d510 0x7f3d44157ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57070/0 (socket says 172.21.15.105:57070) 2023-11-28T18:32:22.854 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d437fe640 1 -- 172.21.15.105:0/1509150794 learned_addr learned my addr 172.21.15.105:0/1509150794 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:32:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d42ffd640 1 --2- 172.21.15.105:0/1509150794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d440b30d0 0x7f3d4406c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d3a7fc640 1 --2- 172.21.15.105:0/1509150794 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d440cf960 0x7f3d4406c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d437fe640 1 -- 172.21.15.105:0/1509150794 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d440cf960 msgr2=0x7f3d4406c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d437fe640 1 --2- 172.21.15.105:0/1509150794 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d440cf960 0x7f3d4406c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d437fe640 1 -- 172.21.15.105:0/1509150794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d440b30d0 msgr2=0x7f3d4406c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d437fe640 1 --2- 172.21.15.105:0/1509150794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d440b30d0 0x7f3d4406c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d437fe640 1 -- 172.21.15.105:0/1509150794 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d340094d0 con 0x7f3d4415d510 2023-11-28T18:32:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d437fe640 1 --2- 172.21.15.105:0/1509150794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4415d510 0x7f3d44157ff0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00cef0 tx=0x7f3d3c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d3bfff640 1 -- 172.21.15.105:0/1509150794 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c00d840 con 0x7f3d4415d510 2023-11-28T18:32:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d3bfff640 1 -- 172.21.15.105:0/1509150794 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3c00d9a0 con 0x7f3d4415d510 2023-11-28T18:32:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d3bfff640 1 -- 172.21.15.105:0/1509150794 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c00dd50 con 0x7f3d4415d510 2023-11-28T18:32:22.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d49279640 1 -- 172.21.15.105:0/1509150794 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d4415f1e0 con 0x7f3d4415d510 2023-11-28T18:32:22.858 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d49279640 1 -- 172.21.15.105:0/1509150794 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d44152ab0 con 0x7f3d4415d510 2023-11-28T18:32:22.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d49279640 1 -- 172.21.15.105:0/1509150794 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d4415f9e0 con 0x7f3d4415d510 2023-11-28T18:32:22.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.852+0000 7f3d3bfff640 1 -- 172.21.15.105:0/1509150794 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3d3c01e3b0 con 0x7f3d4415d510 2023-11-28T18:32:22.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.856+0000 7f3d3bfff640 1 --2- 172.21.15.105:0/1509150794 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d24066900 0x7f3d24068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:32:22.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.856+0000 7f3d3bfff640 1 -- 172.21.15.105:0/1509150794 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3d3c084fb0 con 0x7f3d4415d510 2023-11-28T18:32:22.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.856+0000 7f3d42ffd640 1 --2- 172.21.15.105:0/1509150794 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d24066900 0x7f3d24068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:32:22.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.856+0000 7f3d42ffd640 1 --2- 172.21.15.105:0/1509150794 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d24066900 0x7f3d24068dc0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3d340ae000 tx=0x7f3d340b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:32:22.859 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:22.860+0000 7f3d3bfff640 1 -- 172.21.15.105:0/1509150794 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3d3c0b6280 con 0x7f3d4415d510 2023-11-28T18:32:23.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:23.008+0000 7f3d49279640 1 -- 172.21.15.105:0/1509150794 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_distributed_factor}] v 0) v1 -- 0x7f3d44067570 con 0x7f3d4415d510 2023-11-28T18:32:23.012 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:23.012+0000 7f3d3bfff640 1 -- 172.21.15.105:0/1509150794 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_distributed_factor}]=0 v71)=0 v71) v1 ==== 154+0+0 (secure 0 0 0) 0x7f3d3c01a020 con 0x7f3d4415d510 2023-11-28T18:32:23.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:23.012+0000 7f3d49279640 1 -- 172.21.15.105:0/1509150794 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d24066900 msgr2=0x7f3d24068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:23.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:23.012+0000 7f3d49279640 1 --2- 172.21.15.105:0/1509150794 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d24066900 0x7f3d24068dc0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3d340ae000 tx=0x7f3d340b5040 comp rx=0 tx=0).stop 2023-11-28T18:32:23.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:23.012+0000 7f3d49279640 1 -- 172.21.15.105:0/1509150794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4415d510 msgr2=0x7f3d44157ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:32:23.014 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:23.012+0000 7f3d49279640 1 --2- 172.21.15.105:0/1509150794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4415d510 0x7f3d44157ff0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00cef0 tx=0x7f3d3c00a710 comp rx=0 tx=0).stop 2023-11-28T18:32:23.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:23.012+0000 7f3d49279640 1 -- 172.21.15.105:0/1509150794 shutdown_connections 2023-11-28T18:32:23.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:23.012+0000 7f3d49279640 1 --2- 172.21.15.105:0/1509150794 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d24066900 0x7f3d24068dc0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:23.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:23.012+0000 7f3d49279640 1 --2- 172.21.15.105:0/1509150794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4415d510 0x7f3d44157ff0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:23.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:23.012+0000 7f3d49279640 1 --2- 172.21.15.105:0/1509150794 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d440cf960 0x7f3d4406c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:23.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:23.012+0000 7f3d49279640 1 --2- 172.21.15.105:0/1509150794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d440b30d0 0x7f3d4406c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:32:23.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:23.012+0000 7f3d49279640 1 -- 172.21.15.105:0/1509150794 >> 172.21.15.105:0/1509150794 conn(0x7f3d440d3050 msgr2=0x7f3d440d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:32:23.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:23.016+0000 7f3d49279640 1 -- 172.21.15.105:0/1509150794 shutdown_connections 2023-11-28T18:32:23.015 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:32:23.016+0000 7f3d49279640 1 -- 172.21.15.105:0/1509150794 wait complete. 2023-11-28T18:32:23.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:32:23.033 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-28T18:32:23.033 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:32:23.034 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p tree 2023-11-28T18:32:23.034 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:32:23.034 DEBUG:teuthology.orchestra.run.smithi105:> setfattr -n ceph.dir.pin.distributed -v 1 tree 2023-11-28T18:32:23.034 DEBUG:teuthology.orchestra.run.smithi105:> 2023-11-28T18:32:23.034 DEBUG:teuthology.orchestra.run.smithi105:> for ((i = 0; i < 1000; i++)); do 2023-11-28T18:32:23.034 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p "tree/$i" 2023-11-28T18:32:23.034 DEBUG:teuthology.orchestra.run.smithi105:> echo file > "tree/$i/file" 2023-11-28T18:32:23.034 DEBUG:teuthology.orchestra.run.smithi105:> done 2023-11-28T18:32:23.034 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-11-28T18:32:23.045 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree 2023-11-28T18:32:23.050 INFO:teuthology.orchestra.run.smithi105.stderr:+ setfattr -n ceph.dir.pin.distributed -v 1 tree 2023-11-28T18:32:23.054 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i = 0 )) 2023-11-28T18:32:23.054 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.054 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/0 2023-11-28T18:32:23.060 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.066 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.066 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.066 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/1 2023-11-28T18:32:23.074 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.076 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.076 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.077 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/2 2023-11-28T18:32:23.083 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.085 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.085 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.085 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/3 2023-11-28T18:32:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/4 2023-11-28T18:32:23.100 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.102 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.102 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.102 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/5 2023-11-28T18:32:23.108 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.110 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/6 2023-11-28T18:32:23.116 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.118 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.119 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.119 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/7 2023-11-28T18:32:23.125 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.127 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.127 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.128 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/8 2023-11-28T18:32:23.134 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.136 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.136 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.136 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/9 2023-11-28T18:32:23.144 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.144 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.144 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.144 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/10 2023-11-28T18:32:23.151 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.153 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.153 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.153 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/11 2023-11-28T18:32:23.159 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.161 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.161 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.161 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/12 2023-11-28T18:32:23.168 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.170 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.170 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.170 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/13 2023-11-28T18:32:23.177 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.180 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/14 2023-11-28T18:32:23.186 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.188 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.188 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.188 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/15 2023-11-28T18:32:23.195 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.197 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.197 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.197 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/16 2023-11-28T18:32:23.203 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.205 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.206 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.206 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/17 2023-11-28T18:32:23.212 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.214 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.214 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.214 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/18 2023-11-28T18:32:23.221 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/19 2023-11-28T18:32:23.229 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.231 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.232 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.232 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/20 2023-11-28T18:32:23.239 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/21 2023-11-28T18:32:23.247 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.249 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.249 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.249 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/22 2023-11-28T18:32:23.255 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.257 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.257 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.258 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/23 2023-11-28T18:32:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/24 2023-11-28T18:32:23.272 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.277 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/25 2023-11-28T18:32:23.280 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.282 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.282 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.283 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/26 2023-11-28T18:32:23.288 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.290 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.290 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.291 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/27 2023-11-28T18:32:23.296 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.298 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.298 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.298 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/28 2023-11-28T18:32:23.304 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.306 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.306 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.306 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/29 2023-11-28T18:32:23.313 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.315 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.315 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.315 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/30 2023-11-28T18:32:23.321 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.324 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.324 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.324 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/31 2023-11-28T18:32:23.330 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.333 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.333 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.333 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/32 2023-11-28T18:32:23.339 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.342 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/33 2023-11-28T18:32:23.348 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.350 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.350 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.350 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/34 2023-11-28T18:32:23.356 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.359 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.359 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.359 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/35 2023-11-28T18:32:23.366 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.367 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.367 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.367 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/36 2023-11-28T18:32:23.374 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.376 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.377 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.377 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/37 2023-11-28T18:32:23.383 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.385 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.385 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.385 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/38 2023-11-28T18:32:23.391 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.394 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.394 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.394 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/39 2023-11-28T18:32:23.400 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.402 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.402 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.402 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/40 2023-11-28T18:32:23.409 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.411 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.412 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.412 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/41 2023-11-28T18:32:23.418 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.420 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.420 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.420 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/42 2023-11-28T18:32:23.427 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.429 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.429 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.429 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/43 2023-11-28T18:32:23.435 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.438 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/44 2023-11-28T18:32:23.444 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.445 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.446 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.446 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/45 2023-11-28T18:32:23.452 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/46 2023-11-28T18:32:23.460 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.463 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.463 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.463 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/47 2023-11-28T18:32:23.469 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.471 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.472 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.472 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/48 2023-11-28T18:32:23.483 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.483 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.483 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.483 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/49 2023-11-28T18:32:23.488 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.490 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.490 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.490 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/50 2023-11-28T18:32:23.496 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.498 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.498 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.498 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/51 2023-11-28T18:32:23.505 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.507 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.507 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.507 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/52 2023-11-28T18:32:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.516 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.516 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.516 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/53 2023-11-28T18:32:23.522 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.525 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.525 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.525 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/54 2023-11-28T18:32:23.531 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.533 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.534 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.534 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/55 2023-11-28T18:32:23.540 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.542 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.542 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.542 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/56 2023-11-28T18:32:23.548 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.550 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.550 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.550 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/57 2023-11-28T18:32:23.556 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.558 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.559 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.559 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/58 2023-11-28T18:32:23.565 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.567 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.567 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.567 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/59 2023-11-28T18:32:23.572 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.574 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.575 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.575 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/60 2023-11-28T18:32:23.581 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.583 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.583 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.583 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/61 2023-11-28T18:32:23.589 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.591 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.592 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.592 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/62 2023-11-28T18:32:23.597 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.599 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.600 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.600 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/63 2023-11-28T18:32:23.606 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/64 2023-11-28T18:32:23.614 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.616 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.616 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.616 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/65 2023-11-28T18:32:23.623 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.625 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.625 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.625 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/66 2023-11-28T18:32:23.631 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.633 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.634 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.634 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/67 2023-11-28T18:32:23.641 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:23.644 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:23.644 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:23.644 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/68 2023-11-28T18:32:28.661 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.664 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.664 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.664 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/69 2023-11-28T18:32:28.671 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.674 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.674 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.674 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/70 2023-11-28T18:32:28.681 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.683 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.684 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.684 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/71 2023-11-28T18:32:28.690 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.692 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.692 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.692 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/72 2023-11-28T18:32:28.699 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.701 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.702 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.702 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/73 2023-11-28T18:32:28.708 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.711 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.711 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.711 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/74 2023-11-28T18:32:28.717 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.719 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.720 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.720 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/75 2023-11-28T18:32:28.727 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.728 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.728 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.729 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/76 2023-11-28T18:32:28.735 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.737 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.737 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.738 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/77 2023-11-28T18:32:28.744 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.747 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.747 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.747 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/78 2023-11-28T18:32:28.753 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.755 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.755 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.755 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/79 2023-11-28T18:32:28.762 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.764 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.764 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.764 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/80 2023-11-28T18:32:28.770 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.773 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.773 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.773 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/81 2023-11-28T18:32:28.779 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.782 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.782 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.782 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/82 2023-11-28T18:32:28.789 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.791 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.791 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.792 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/83 2023-11-28T18:32:28.798 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.800 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.800 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.800 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/84 2023-11-28T18:32:28.807 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.810 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.810 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.810 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/85 2023-11-28T18:32:28.816 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.818 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.818 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.819 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/86 2023-11-28T18:32:28.824 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.827 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.827 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.827 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/87 2023-11-28T18:32:28.833 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.836 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.836 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.836 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/88 2023-11-28T18:32:28.842 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.844 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.845 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.845 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/89 2023-11-28T18:32:28.851 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.853 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.853 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.853 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/90 2023-11-28T18:32:28.859 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.862 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.862 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.862 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/91 2023-11-28T18:32:28.868 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.870 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.870 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.870 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/92 2023-11-28T18:32:28.876 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.878 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.878 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.879 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/93 2023-11-28T18:32:28.884 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.887 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.887 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.887 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/94 2023-11-28T18:32:28.893 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.895 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.895 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.895 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/95 2023-11-28T18:32:28.901 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.903 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.903 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.904 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/96 2023-11-28T18:32:28.911 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.913 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.913 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.913 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/97 2023-11-28T18:32:28.919 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.921 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.921 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.921 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/98 2023-11-28T18:32:28.928 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.930 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.930 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.930 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/99 2023-11-28T18:32:28.937 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.939 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.939 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.939 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/100 2023-11-28T18:32:28.945 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.947 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.948 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.948 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/101 2023-11-28T18:32:28.954 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.956 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.956 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.956 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/102 2023-11-28T18:32:28.962 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.964 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.964 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.964 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/103 2023-11-28T18:32:28.970 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.972 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.973 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.973 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/104 2023-11-28T18:32:28.979 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.981 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.981 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.981 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/105 2023-11-28T18:32:28.988 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.989 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.989 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.989 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/106 2023-11-28T18:32:28.995 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:28.997 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:28.997 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:28.998 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/107 2023-11-28T18:32:29.004 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.006 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.006 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.006 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/108 2023-11-28T18:32:29.012 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.014 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.014 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.014 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/109 2023-11-28T18:32:29.020 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.022 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.022 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.023 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/110 2023-11-28T18:32:29.029 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.031 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.031 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.031 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/111 2023-11-28T18:32:29.037 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.039 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.039 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.040 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/112 2023-11-28T18:32:29.045 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.047 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.048 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.048 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/113 2023-11-28T18:32:29.054 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.056 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.056 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.057 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/114 2023-11-28T18:32:29.063 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.065 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.065 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.065 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/115 2023-11-28T18:32:29.071 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.074 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.075 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.075 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/116 2023-11-28T18:32:29.080 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.082 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.082 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.082 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/117 2023-11-28T18:32:29.088 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.090 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.090 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.090 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/118 2023-11-28T18:32:29.096 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.099 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.099 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.099 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/119 2023-11-28T18:32:29.105 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.106 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.107 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.107 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/120 2023-11-28T18:32:29.112 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.115 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.115 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.116 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/121 2023-11-28T18:32:29.120 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.123 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.123 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.123 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/122 2023-11-28T18:32:29.129 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.131 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.131 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.131 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/123 2023-11-28T18:32:29.137 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.139 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.140 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.140 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/124 2023-11-28T18:32:29.145 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.147 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.147 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.147 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/125 2023-11-28T18:32:29.153 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.155 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.156 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.156 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/126 2023-11-28T18:32:29.162 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.164 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.164 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.164 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/127 2023-11-28T18:32:29.170 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.172 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.173 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.173 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/128 2023-11-28T18:32:29.179 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.181 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.181 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.181 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/129 2023-11-28T18:32:29.188 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.189 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.189 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.189 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/130 2023-11-28T18:32:29.195 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.197 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.197 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.197 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/131 2023-11-28T18:32:29.203 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.205 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.205 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.205 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/132 2023-11-28T18:32:29.211 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/133 2023-11-28T18:32:29.220 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.221 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.222 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.222 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/134 2023-11-28T18:32:29.228 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.230 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.230 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.230 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/135 2023-11-28T18:32:29.236 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.238 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.238 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.238 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/136 2023-11-28T18:32:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.246 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.246 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.246 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/137 2023-11-28T18:32:29.253 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.255 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.255 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.255 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/138 2023-11-28T18:32:29.261 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.263 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.263 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.263 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/139 2023-11-28T18:32:29.269 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.271 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.271 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.271 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/140 2023-11-28T18:32:29.277 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.280 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.280 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.280 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/141 2023-11-28T18:32:29.285 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.288 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.288 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.288 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/142 2023-11-28T18:32:29.294 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.296 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.296 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.296 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/143 2023-11-28T18:32:29.302 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.304 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.304 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.304 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/144 2023-11-28T18:32:29.310 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.312 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.312 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.312 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/145 2023-11-28T18:32:29.318 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.321 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.321 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.321 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/146 2023-11-28T18:32:29.327 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.329 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.329 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.330 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/147 2023-11-28T18:32:29.335 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.338 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.338 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.338 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/148 2023-11-28T18:32:29.343 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.346 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.346 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.347 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/149 2023-11-28T18:32:29.352 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.354 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.354 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.355 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/150 2023-11-28T18:32:29.361 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.364 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.364 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.364 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/151 2023-11-28T18:32:29.370 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.373 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.373 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.373 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/152 2023-11-28T18:32:29.379 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.382 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.382 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.382 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/153 2023-11-28T18:32:29.388 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.390 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.391 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.391 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/154 2023-11-28T18:32:29.397 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.399 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.399 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.399 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/155 2023-11-28T18:32:29.404 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.407 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.407 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.407 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/156 2023-11-28T18:32:29.413 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.415 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.415 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.415 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/157 2023-11-28T18:32:29.420 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.423 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.423 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.423 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/158 2023-11-28T18:32:29.429 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.431 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.431 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.432 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/159 2023-11-28T18:32:29.437 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.440 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.440 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.440 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/160 2023-11-28T18:32:29.446 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.448 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.448 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.449 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/161 2023-11-28T18:32:29.454 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.456 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.457 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.457 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/162 2023-11-28T18:32:29.463 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.465 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.465 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.466 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/163 2023-11-28T18:32:29.472 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.474 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.474 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.474 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/164 2023-11-28T18:32:29.481 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.483 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.483 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.483 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/165 2023-11-28T18:32:29.490 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.493 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.493 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.493 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/166 2023-11-28T18:32:29.499 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.501 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.501 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.501 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/167 2023-11-28T18:32:29.514 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.516 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.516 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.516 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/168 2023-11-28T18:32:29.524 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.526 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.526 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.526 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/169 2023-11-28T18:32:29.533 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.535 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.535 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.536 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/170 2023-11-28T18:32:29.542 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.544 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.544 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.544 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/171 2023-11-28T18:32:29.551 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.552 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.553 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.553 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/172 2023-11-28T18:32:29.559 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.561 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.561 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.561 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/173 2023-11-28T18:32:29.567 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.569 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.569 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.569 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/174 2023-11-28T18:32:29.576 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.578 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.578 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.578 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/175 2023-11-28T18:32:29.584 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.586 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.586 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.587 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/176 2023-11-28T18:32:29.592 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.594 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.595 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.595 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/177 2023-11-28T18:32:29.600 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.603 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.603 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.603 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/178 2023-11-28T18:32:29.609 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.611 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.611 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.611 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/179 2023-11-28T18:32:29.617 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.619 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.619 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.620 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/180 2023-11-28T18:32:29.625 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.627 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.628 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.628 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/181 2023-11-28T18:32:29.633 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.636 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.636 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.636 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/182 2023-11-28T18:32:29.642 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.644 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.644 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.644 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/183 2023-11-28T18:32:29.650 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.652 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.653 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.653 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/184 2023-11-28T18:32:29.658 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.661 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.661 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.661 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/185 2023-11-28T18:32:29.667 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.669 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.669 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.669 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/186 2023-11-28T18:32:29.675 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.677 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.677 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.678 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/187 2023-11-28T18:32:29.684 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.686 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.686 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.686 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/188 2023-11-28T18:32:29.692 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.694 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.695 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.695 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/189 2023-11-28T18:32:29.700 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.702 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.703 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.703 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/190 2023-11-28T18:32:29.708 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.711 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.711 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.711 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/191 2023-11-28T18:32:29.716 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.719 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.719 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.719 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/192 2023-11-28T18:32:29.725 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.727 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.728 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.728 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/193 2023-11-28T18:32:29.733 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.735 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.735 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.735 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/194 2023-11-28T18:32:29.742 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.744 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.744 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.745 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/195 2023-11-28T18:32:29.750 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.753 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.753 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.753 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/196 2023-11-28T18:32:29.759 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.761 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.761 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.761 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/197 2023-11-28T18:32:29.767 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.769 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.770 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.770 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/198 2023-11-28T18:32:29.775 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.777 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.777 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.778 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/199 2023-11-28T18:32:29.783 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.785 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.786 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.786 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/200 2023-11-28T18:32:29.791 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.793 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.793 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.794 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/201 2023-11-28T18:32:29.799 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.801 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.802 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.802 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/202 2023-11-28T18:32:29.808 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.809 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.809 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.810 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/203 2023-11-28T18:32:29.816 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.818 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.818 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.818 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/204 2023-11-28T18:32:29.824 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.826 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.826 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.826 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/205 2023-11-28T18:32:29.832 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.834 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.834 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.834 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/206 2023-11-28T18:32:29.840 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.842 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.843 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.843 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/207 2023-11-28T18:32:29.849 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.851 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.852 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.852 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/208 2023-11-28T18:32:29.857 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.860 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.860 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.860 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/209 2023-11-28T18:32:29.866 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.868 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.869 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.869 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/210 2023-11-28T18:32:29.874 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.877 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.877 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.877 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/211 2023-11-28T18:32:29.883 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.885 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.885 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.885 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/212 2023-11-28T18:32:29.892 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.893 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.893 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.893 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/213 2023-11-28T18:32:29.899 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.901 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.901 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.901 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/214 2023-11-28T18:32:29.907 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.909 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.910 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.910 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/215 2023-11-28T18:32:29.916 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.918 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.918 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.919 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/216 2023-11-28T18:32:29.924 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.926 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.927 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.927 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/217 2023-11-28T18:32:29.932 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/218 2023-11-28T18:32:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.943 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.943 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.943 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/219 2023-11-28T18:32:29.950 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.952 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.952 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.952 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/220 2023-11-28T18:32:29.958 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.961 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.961 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.961 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/221 2023-11-28T18:32:29.968 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.969 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.970 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.970 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/222 2023-11-28T18:32:29.977 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.980 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.980 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.980 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/223 2023-11-28T18:32:29.986 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.988 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.989 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.989 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/224 2023-11-28T18:32:29.995 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:29.997 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:29.997 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:29.997 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/225 2023-11-28T18:32:30.004 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.006 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.006 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.006 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/226 2023-11-28T18:32:30.012 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.014 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.015 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.015 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/227 2023-11-28T18:32:30.022 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.024 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.024 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.024 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/228 2023-11-28T18:32:30.030 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.033 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.033 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.033 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/229 2023-11-28T18:32:30.042 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.042 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.042 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.042 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/230 2023-11-28T18:32:30.048 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.050 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.050 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.050 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/231 2023-11-28T18:32:30.056 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.059 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.059 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.059 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/232 2023-11-28T18:32:30.065 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.068 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.068 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.068 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/233 2023-11-28T18:32:30.075 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.077 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.077 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.077 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/234 2023-11-28T18:32:30.084 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.086 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.086 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.086 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/235 2023-11-28T18:32:30.092 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.094 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.094 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.094 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/236 2023-11-28T18:32:30.100 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.103 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.103 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.103 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/237 2023-11-28T18:32:30.109 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.111 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.111 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.112 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/238 2023-11-28T18:32:30.117 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.119 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.120 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.120 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/239 2023-11-28T18:32:30.127 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.128 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.128 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.128 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/240 2023-11-28T18:32:30.134 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.136 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.136 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.136 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/241 2023-11-28T18:32:30.142 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.145 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.145 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.145 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/242 2023-11-28T18:32:30.151 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.154 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.154 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.154 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/243 2023-11-28T18:32:30.160 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.163 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.163 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.163 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/244 2023-11-28T18:32:30.168 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.171 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.171 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.171 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/245 2023-11-28T18:32:30.177 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.179 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.179 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.179 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/246 2023-11-28T18:32:30.185 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.187 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.187 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.188 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/247 2023-11-28T18:32:30.194 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.196 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.196 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.196 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/248 2023-11-28T18:32:30.201 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.204 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.204 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.204 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/249 2023-11-28T18:32:30.209 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.211 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.212 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.212 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/250 2023-11-28T18:32:30.218 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.220 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.220 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.220 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/251 2023-11-28T18:32:30.226 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.229 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.229 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.229 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/252 2023-11-28T18:32:30.237 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.238 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.238 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.238 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/253 2023-11-28T18:32:30.245 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.246 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.246 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.246 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/254 2023-11-28T18:32:30.252 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.254 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.254 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.254 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/255 2023-11-28T18:32:30.260 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.262 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.262 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.262 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/256 2023-11-28T18:32:30.269 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.271 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.272 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/257 2023-11-28T18:32:30.278 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/258 2023-11-28T18:32:30.287 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.288 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.288 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.288 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/259 2023-11-28T18:32:30.294 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.298 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.298 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.298 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/260 2023-11-28T18:32:30.303 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.306 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.306 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.306 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/261 2023-11-28T18:32:30.312 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.315 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.315 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.315 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/262 2023-11-28T18:32:30.321 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.324 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.324 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.324 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/263 2023-11-28T18:32:30.331 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.333 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.333 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.333 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/264 2023-11-28T18:32:30.339 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.342 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.342 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.342 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/265 2023-11-28T18:32:30.348 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.350 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.350 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.350 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/266 2023-11-28T18:32:30.356 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.358 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.358 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.358 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/267 2023-11-28T18:32:30.364 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.366 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.366 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.366 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/268 2023-11-28T18:32:30.372 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.375 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.375 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.375 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/269 2023-11-28T18:32:30.380 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.383 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.383 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.383 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/270 2023-11-28T18:32:30.388 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.391 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.391 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.391 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/271 2023-11-28T18:32:30.397 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.399 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.399 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.399 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/272 2023-11-28T18:32:30.405 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.408 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.408 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.408 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/273 2023-11-28T18:32:30.414 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.416 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.416 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.416 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/274 2023-11-28T18:32:30.423 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.425 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.425 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.425 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/275 2023-11-28T18:32:30.431 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.433 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.433 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.433 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/276 2023-11-28T18:32:30.439 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.441 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.442 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.442 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/277 2023-11-28T18:32:30.450 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.451 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.451 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.451 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/278 2023-11-28T18:32:30.456 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.459 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.459 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.459 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/279 2023-11-28T18:32:30.465 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.468 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.468 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.468 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/280 2023-11-28T18:32:30.474 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.476 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.476 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.476 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/281 2023-11-28T18:32:30.482 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.484 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.484 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.484 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/282 2023-11-28T18:32:30.490 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.492 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.492 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.493 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/283 2023-11-28T18:32:30.498 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.500 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.500 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.500 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/284 2023-11-28T18:32:30.508 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.511 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.511 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.511 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/285 2023-11-28T18:32:30.517 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.520 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.520 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.520 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/286 2023-11-28T18:32:30.526 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.528 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.528 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.528 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/287 2023-11-28T18:32:30.534 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.536 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.536 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.537 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/288 2023-11-28T18:32:30.542 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.544 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.544 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.544 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/289 2023-11-28T18:32:30.550 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.552 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.553 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.553 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/290 2023-11-28T18:32:30.558 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.561 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.561 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.561 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/291 2023-11-28T18:32:30.567 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.569 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.569 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.570 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/292 2023-11-28T18:32:30.575 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.578 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.578 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.579 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/293 2023-11-28T18:32:30.584 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.586 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.586 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.586 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/294 2023-11-28T18:32:30.593 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.595 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.595 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.595 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/295 2023-11-28T18:32:30.601 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.603 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.603 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.604 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/296 2023-11-28T18:32:30.609 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.611 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.612 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.612 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/297 2023-11-28T18:32:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.620 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.620 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.621 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/298 2023-11-28T18:32:30.626 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.628 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.628 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.628 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/299 2023-11-28T18:32:30.634 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/300 2023-11-28T18:32:30.642 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.644 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.645 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.645 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/301 2023-11-28T18:32:30.650 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.655 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.655 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.655 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/302 2023-11-28T18:32:30.659 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.662 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.662 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.662 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/303 2023-11-28T18:32:30.667 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.669 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.670 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.670 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/304 2023-11-28T18:32:30.676 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.678 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.678 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.679 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/305 2023-11-28T18:32:30.684 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.686 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.687 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.687 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/306 2023-11-28T18:32:30.693 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.695 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.696 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.696 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/307 2023-11-28T18:32:30.701 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.703 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.704 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.704 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/308 2023-11-28T18:32:30.709 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.712 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.712 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.713 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/309 2023-11-28T18:32:30.718 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.721 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.721 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.721 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/310 2023-11-28T18:32:30.727 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.729 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.729 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.729 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/311 2023-11-28T18:32:30.735 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.737 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.738 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.738 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/312 2023-11-28T18:32:30.744 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.746 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.746 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.746 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/313 2023-11-28T18:32:30.752 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.755 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.755 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.755 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/314 2023-11-28T18:32:30.760 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/315 2023-11-28T18:32:30.768 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.770 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.770 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.770 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/316 2023-11-28T18:32:30.776 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.778 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.778 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.778 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/317 2023-11-28T18:32:30.784 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.786 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.786 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.786 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/318 2023-11-28T18:32:30.792 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.794 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.794 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.795 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/319 2023-11-28T18:32:30.801 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.803 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.803 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.803 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/320 2023-11-28T18:32:30.809 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.812 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.812 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.812 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/321 2023-11-28T18:32:30.818 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.820 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.820 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.820 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/322 2023-11-28T18:32:30.826 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.829 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.829 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.830 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/323 2023-11-28T18:32:30.834 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.836 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.837 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.837 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/324 2023-11-28T18:32:30.842 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.844 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.845 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.845 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/325 2023-11-28T18:32:30.851 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.853 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.853 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.853 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/326 2023-11-28T18:32:30.860 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.862 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.864 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.865 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/327 2023-11-28T18:32:30.868 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.870 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.871 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.871 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/328 2023-11-28T18:32:30.877 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.879 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.879 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.880 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/329 2023-11-28T18:32:30.885 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.888 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.888 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.888 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/330 2023-11-28T18:32:30.894 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.896 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.897 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.897 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/331 2023-11-28T18:32:30.903 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.905 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.905 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.906 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/332 2023-11-28T18:32:30.911 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.914 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.914 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.914 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/333 2023-11-28T18:32:30.920 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.922 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.922 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.923 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/334 2023-11-28T18:32:30.928 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.931 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.931 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.931 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/335 2023-11-28T18:32:30.936 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.938 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.939 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.939 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/336 2023-11-28T18:32:30.945 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.948 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/337 2023-11-28T18:32:30.953 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.956 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/338 2023-11-28T18:32:30.961 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.963 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.963 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.963 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/339 2023-11-28T18:32:30.969 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.972 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.972 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.972 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/340 2023-11-28T18:32:30.978 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.980 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.980 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.980 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/341 2023-11-28T18:32:30.987 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.988 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.988 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.988 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/342 2023-11-28T18:32:30.994 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:30.996 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:30.996 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:30.997 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/343 2023-11-28T18:32:31.002 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/344 2023-11-28T18:32:31.010 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.012 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.012 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.013 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/345 2023-11-28T18:32:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/346 2023-11-28T18:32:31.027 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.029 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.029 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.029 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/347 2023-11-28T18:32:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.038 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.038 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.038 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/348 2023-11-28T18:32:31.044 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.046 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.046 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.046 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/349 2023-11-28T18:32:31.052 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.054 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.055 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.055 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/350 2023-11-28T18:32:31.062 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.063 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.063 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.063 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/351 2023-11-28T18:32:31.069 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.071 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.071 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.071 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/352 2023-11-28T18:32:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.079 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.079 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.079 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/353 2023-11-28T18:32:31.085 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.087 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.087 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.087 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/354 2023-11-28T18:32:31.094 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.096 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.096 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.096 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/355 2023-11-28T18:32:31.102 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.104 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.104 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.104 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/356 2023-11-28T18:32:31.110 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.112 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.113 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.113 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/357 2023-11-28T18:32:31.119 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.121 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.121 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.121 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/358 2023-11-28T18:32:31.128 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.130 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.130 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.130 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/359 2023-11-28T18:32:31.137 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.138 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.139 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.139 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/360 2023-11-28T18:32:31.144 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.147 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.147 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.147 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/361 2023-11-28T18:32:31.153 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.155 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.156 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.156 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/362 2023-11-28T18:32:31.161 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.164 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.164 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.164 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/363 2023-11-28T18:32:31.170 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.172 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.172 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.172 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/364 2023-11-28T18:32:31.179 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.181 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.181 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.181 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/365 2023-11-28T18:32:31.187 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.189 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.190 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.190 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/366 2023-11-28T18:32:31.196 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.198 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.198 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.198 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/367 2023-11-28T18:32:31.204 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.207 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.207 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.207 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/368 2023-11-28T18:32:31.213 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.215 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.215 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.215 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/369 2023-11-28T18:32:31.221 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.224 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.224 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.224 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/370 2023-11-28T18:32:31.229 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.232 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.232 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.232 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/371 2023-11-28T18:32:31.238 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.240 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.240 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.240 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/372 2023-11-28T18:32:31.246 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.248 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.248 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.249 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/373 2023-11-28T18:32:31.254 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.257 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.257 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.257 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/374 2023-11-28T18:32:31.266 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.266 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.267 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.267 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/375 2023-11-28T18:32:31.271 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.273 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.273 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.273 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/376 2023-11-28T18:32:31.279 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.281 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.281 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.281 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/377 2023-11-28T18:32:31.287 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.290 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.290 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.290 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/378 2023-11-28T18:32:31.296 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.298 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.298 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.298 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/379 2023-11-28T18:32:31.304 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.306 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.307 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.307 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/380 2023-11-28T18:32:31.312 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.314 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.315 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.315 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/381 2023-11-28T18:32:31.321 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.323 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.323 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.323 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/382 2023-11-28T18:32:31.330 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.332 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.332 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.332 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/383 2023-11-28T18:32:31.338 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.340 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.340 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.341 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/384 2023-11-28T18:32:31.347 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.349 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.349 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.349 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/385 2023-11-28T18:32:31.356 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/386 2023-11-28T18:32:31.365 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.367 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.367 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/387 2023-11-28T18:32:31.373 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.375 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.375 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.375 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/388 2023-11-28T18:32:31.382 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.384 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.384 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.384 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/389 2023-11-28T18:32:31.390 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.392 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.392 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.392 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/390 2023-11-28T18:32:31.398 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.400 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.401 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.401 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/391 2023-11-28T18:32:31.406 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.409 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.409 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.409 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/392 2023-11-28T18:32:31.415 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.418 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.418 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.419 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/393 2023-11-28T18:32:31.424 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.426 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.426 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.427 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/394 2023-11-28T18:32:31.433 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.435 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.435 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.435 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/395 2023-11-28T18:32:31.441 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.443 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.443 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.443 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/396 2023-11-28T18:32:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.452 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.452 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.452 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/397 2023-11-28T18:32:31.458 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.460 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.460 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.460 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/398 2023-11-28T18:32:31.466 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.469 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.470 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.470 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/399 2023-11-28T18:32:31.475 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.478 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.478 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.478 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/400 2023-11-28T18:32:31.485 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.487 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.488 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.488 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/401 2023-11-28T18:32:31.494 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.497 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.497 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.498 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/402 2023-11-28T18:32:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.506 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.507 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.507 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/403 2023-11-28T18:32:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/404 2023-11-28T18:32:31.522 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.525 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.525 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.525 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/405 2023-11-28T18:32:31.530 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.533 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.534 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.534 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/406 2023-11-28T18:32:31.540 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.542 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.542 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.542 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/407 2023-11-28T18:32:31.548 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.551 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.551 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.551 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/408 2023-11-28T18:32:31.558 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.560 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.560 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.560 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/409 2023-11-28T18:32:31.567 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.569 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.569 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.569 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/410 2023-11-28T18:32:31.575 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.578 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.578 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.578 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/411 2023-11-28T18:32:31.584 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.587 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.587 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.587 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/412 2023-11-28T18:32:31.594 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.597 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.597 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.597 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/413 2023-11-28T18:32:31.603 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.605 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.605 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.605 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/414 2023-11-28T18:32:31.612 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.615 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.615 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.615 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/415 2023-11-28T18:32:31.622 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.624 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.624 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.624 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/416 2023-11-28T18:32:31.631 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.633 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.633 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.633 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/417 2023-11-28T18:32:31.640 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.642 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.642 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.642 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/418 2023-11-28T18:32:31.649 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.651 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.651 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.652 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/419 2023-11-28T18:32:31.658 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.660 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.660 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.660 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/420 2023-11-28T18:32:31.667 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.669 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.669 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.669 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/421 2023-11-28T18:32:31.677 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.678 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.678 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/422 2023-11-28T18:32:31.685 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.687 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/423 2023-11-28T18:32:31.694 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.697 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.697 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.697 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/424 2023-11-28T18:32:31.703 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.705 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.706 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.706 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/425 2023-11-28T18:32:31.712 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.715 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.715 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.715 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/426 2023-11-28T18:32:31.721 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.724 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.724 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.724 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/427 2023-11-28T18:32:31.730 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.732 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.732 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.732 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/428 2023-11-28T18:32:31.738 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.741 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.741 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.741 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/429 2023-11-28T18:32:31.747 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.750 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.750 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.750 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/430 2023-11-28T18:32:31.756 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.759 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.759 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.759 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/431 2023-11-28T18:32:31.766 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.768 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.768 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.769 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/432 2023-11-28T18:32:31.774 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.777 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.777 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.777 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/433 2023-11-28T18:32:31.783 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.785 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.785 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.785 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/434 2023-11-28T18:32:31.791 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/435 2023-11-28T18:32:31.799 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.801 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.802 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.802 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/436 2023-11-28T18:32:31.808 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.810 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.810 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.810 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/437 2023-11-28T18:32:31.816 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.818 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.818 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.818 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/438 2023-11-28T18:32:31.825 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.828 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/439 2023-11-28T18:32:31.833 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.835 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.835 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.836 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/440 2023-11-28T18:32:31.841 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.844 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.844 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.844 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/441 2023-11-28T18:32:31.850 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.852 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.852 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.852 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/442 2023-11-28T18:32:31.858 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.860 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.860 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.860 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/443 2023-11-28T18:32:31.866 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.868 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.869 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.869 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/444 2023-11-28T18:32:31.875 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.877 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.877 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.878 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/445 2023-11-28T18:32:31.886 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.886 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.886 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.886 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/446 2023-11-28T18:32:31.893 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.895 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.895 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.895 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/447 2023-11-28T18:32:31.901 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.904 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.904 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.904 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/448 2023-11-28T18:32:31.910 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.912 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.912 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.912 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/449 2023-11-28T18:32:31.918 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.921 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.921 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.921 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/450 2023-11-28T18:32:31.930 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.932 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.932 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.932 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/451 2023-11-28T18:32:31.938 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.940 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.940 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/452 2023-11-28T18:32:31.947 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.949 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.949 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.949 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/453 2023-11-28T18:32:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/454 2023-11-28T18:32:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.966 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.966 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.966 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/455 2023-11-28T18:32:31.975 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.975 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.975 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.975 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/456 2023-11-28T18:32:31.980 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.982 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.982 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.982 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/457 2023-11-28T18:32:31.989 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.991 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:31.991 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:31.991 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/458 2023-11-28T18:32:31.997 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:31.999 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.000 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.000 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/459 2023-11-28T18:32:32.006 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/460 2023-11-28T18:32:32.014 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.016 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.016 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.017 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/461 2023-11-28T18:32:32.023 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.025 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.025 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.025 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/462 2023-11-28T18:32:32.031 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.033 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.033 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.033 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/463 2023-11-28T18:32:32.040 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.042 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.042 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.042 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/464 2023-11-28T18:32:32.048 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.050 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.050 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.050 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/465 2023-11-28T18:32:32.056 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.059 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.059 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.059 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/466 2023-11-28T18:32:32.065 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.067 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.067 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.067 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/467 2023-11-28T18:32:32.073 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.076 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.076 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.076 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/468 2023-11-28T18:32:32.082 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.084 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.084 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.084 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/469 2023-11-28T18:32:32.090 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/470 2023-11-28T18:32:32.098 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.100 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.101 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.101 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/471 2023-11-28T18:32:32.106 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.109 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.109 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.109 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/472 2023-11-28T18:32:32.115 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.117 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.118 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.118 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/473 2023-11-28T18:32:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.125 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.126 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.126 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/474 2023-11-28T18:32:32.131 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.133 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.134 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.134 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/475 2023-11-28T18:32:32.140 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.142 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.142 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.143 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/476 2023-11-28T18:32:32.149 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.151 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.151 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.151 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/477 2023-11-28T18:32:32.157 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.160 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.160 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.160 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/478 2023-11-28T18:32:32.166 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.168 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.169 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.169 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/479 2023-11-28T18:32:32.175 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.177 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.177 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.178 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/480 2023-11-28T18:32:32.185 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.187 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.187 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.188 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/481 2023-11-28T18:32:32.194 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.196 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.196 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.196 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/482 2023-11-28T18:32:32.203 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.206 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.206 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.206 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/483 2023-11-28T18:32:32.213 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.215 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.215 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.215 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/484 2023-11-28T18:32:32.223 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.224 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.224 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.225 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/485 2023-11-28T18:32:32.231 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.234 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.234 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.234 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/486 2023-11-28T18:32:32.240 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.243 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.243 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.244 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/487 2023-11-28T18:32:32.250 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.252 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.252 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.253 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/488 2023-11-28T18:32:32.259 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.261 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.262 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.262 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/489 2023-11-28T18:32:32.268 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.271 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.271 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.271 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/490 2023-11-28T18:32:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/491 2023-11-28T18:32:32.286 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.289 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.290 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.290 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/492 2023-11-28T18:32:32.300 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.300 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.300 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.300 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/493 2023-11-28T18:32:32.304 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.306 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.306 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.306 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/494 2023-11-28T18:32:32.312 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.315 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.315 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.315 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/495 2023-11-28T18:32:32.321 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.323 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.323 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.323 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/496 2023-11-28T18:32:32.330 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.332 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.332 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.332 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/497 2023-11-28T18:32:32.339 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.341 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.341 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.341 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/498 2023-11-28T18:32:32.348 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.350 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.350 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.350 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/499 2023-11-28T18:32:32.356 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.358 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.358 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.358 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/500 2023-11-28T18:32:32.365 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.367 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.367 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.368 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/501 2023-11-28T18:32:32.374 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.376 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.376 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.376 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/502 2023-11-28T18:32:32.382 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.384 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.384 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.385 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/503 2023-11-28T18:32:32.390 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.392 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.392 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.392 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/504 2023-11-28T18:32:32.399 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.401 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.401 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.401 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/505 2023-11-28T18:32:32.407 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.409 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.409 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.410 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/506 2023-11-28T18:32:32.415 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.418 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.418 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.418 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/507 2023-11-28T18:32:32.424 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.426 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.426 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.426 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/508 2023-11-28T18:32:32.432 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.435 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.435 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.435 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/509 2023-11-28T18:32:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/510 2023-11-28T18:32:32.450 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.452 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.452 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.452 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/511 2023-11-28T18:32:32.459 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.461 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.461 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.461 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/512 2023-11-28T18:32:32.467 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.469 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.469 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.469 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/513 2023-11-28T18:32:32.475 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.477 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.478 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.478 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/514 2023-11-28T18:32:32.487 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.489 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.490 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.490 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/515 2023-11-28T18:32:32.496 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.499 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.499 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.499 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/516 2023-11-28T18:32:32.505 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.508 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.508 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.508 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/517 2023-11-28T18:32:32.515 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.517 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.517 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.518 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/518 2023-11-28T18:32:32.525 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.527 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.527 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.527 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/519 2023-11-28T18:32:32.533 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.535 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.535 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.535 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/520 2023-11-28T18:32:32.541 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.543 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.543 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.544 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/521 2023-11-28T18:32:32.549 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.552 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.552 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.552 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/522 2023-11-28T18:32:32.558 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.561 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.561 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.561 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/523 2023-11-28T18:32:32.567 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.569 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.569 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.569 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/524 2023-11-28T18:32:32.575 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.578 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.578 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.578 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/525 2023-11-28T18:32:32.584 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.587 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.587 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.587 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/526 2023-11-28T18:32:32.593 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.596 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.596 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.596 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/527 2023-11-28T18:32:32.602 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.605 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.605 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.605 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/528 2023-11-28T18:32:32.612 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.614 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.614 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.615 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/529 2023-11-28T18:32:32.621 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.623 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.623 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.623 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/530 2023-11-28T18:32:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.632 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.632 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.632 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/531 2023-11-28T18:32:32.639 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.641 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.641 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.641 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/532 2023-11-28T18:32:32.647 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/533 2023-11-28T18:32:32.655 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/534 2023-11-28T18:32:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.666 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.666 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.666 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/535 2023-11-28T18:32:32.672 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.675 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.675 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.675 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/536 2023-11-28T18:32:32.681 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.683 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.683 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.683 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/537 2023-11-28T18:32:32.690 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.692 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.692 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.692 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/538 2023-11-28T18:32:32.699 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.701 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.701 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.701 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/539 2023-11-28T18:32:32.707 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.709 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.709 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.709 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/540 2023-11-28T18:32:32.717 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.718 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.718 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.719 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/541 2023-11-28T18:32:32.724 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.727 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.727 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.727 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/542 2023-11-28T18:32:32.733 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.735 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.736 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.736 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/543 2023-11-28T18:32:32.741 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.744 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.744 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.744 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/544 2023-11-28T18:32:32.750 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.752 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.753 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.753 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/545 2023-11-28T18:32:32.759 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.760 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.761 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.761 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/546 2023-11-28T18:32:32.767 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.769 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.769 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.769 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/547 2023-11-28T18:32:32.775 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.777 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.778 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.778 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/548 2023-11-28T18:32:32.785 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.786 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.786 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.786 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/549 2023-11-28T18:32:32.792 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.795 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.796 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.796 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/550 2023-11-28T18:32:32.801 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.803 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.803 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.803 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/551 2023-11-28T18:32:32.809 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.811 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.811 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.812 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/552 2023-11-28T18:32:32.818 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.820 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.820 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.820 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/553 2023-11-28T18:32:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/554 2023-11-28T18:32:32.834 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.837 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.837 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.837 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/555 2023-11-28T18:32:32.842 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.845 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.845 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.845 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/556 2023-11-28T18:32:32.852 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.854 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.854 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.854 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/557 2023-11-28T18:32:32.861 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.863 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.863 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.863 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/558 2023-11-28T18:32:32.869 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.872 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.872 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.872 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/559 2023-11-28T18:32:32.879 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.881 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.881 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.881 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/560 2023-11-28T18:32:32.887 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.890 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.890 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.890 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/561 2023-11-28T18:32:32.896 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.898 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.898 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.898 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/562 2023-11-28T18:32:32.905 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.907 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.907 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.907 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/563 2023-11-28T18:32:32.913 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.917 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.917 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.918 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/564 2023-11-28T18:32:32.922 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.924 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.924 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.924 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/565 2023-11-28T18:32:32.930 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.932 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.932 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.932 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/566 2023-11-28T18:32:32.938 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.941 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.941 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.941 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/567 2023-11-28T18:32:32.947 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.949 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.949 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.949 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/568 2023-11-28T18:32:32.955 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.958 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.958 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.958 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/569 2023-11-28T18:32:32.964 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.966 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.966 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.967 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/570 2023-11-28T18:32:32.972 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.974 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.975 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.975 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/571 2023-11-28T18:32:32.980 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.982 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.983 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.983 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/572 2023-11-28T18:32:32.989 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.991 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.991 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:32.991 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/573 2023-11-28T18:32:32.997 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:32.999 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:32.999 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/574 2023-11-28T18:32:33.006 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.008 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.008 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.009 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/575 2023-11-28T18:32:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.017 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.017 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.017 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/576 2023-11-28T18:32:33.023 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.025 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.025 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.025 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/577 2023-11-28T18:32:33.032 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.034 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.035 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.035 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/578 2023-11-28T18:32:33.041 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.043 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.043 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.043 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/579 2023-11-28T18:32:33.049 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.052 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.052 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.052 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/580 2023-11-28T18:32:33.058 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.061 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.061 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.061 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/581 2023-11-28T18:32:33.067 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.070 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.070 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.070 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/582 2023-11-28T18:32:33.076 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/583 2023-11-28T18:32:33.085 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.087 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.087 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.088 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/584 2023-11-28T18:32:33.093 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.096 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.096 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.096 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/585 2023-11-28T18:32:33.102 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.105 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.105 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.105 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/586 2023-11-28T18:32:33.110 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.113 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.113 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.113 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/587 2023-11-28T18:32:33.118 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.120 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.121 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.121 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/588 2023-11-28T18:32:33.127 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.129 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.129 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.129 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/589 2023-11-28T18:32:33.135 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.137 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.137 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.138 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/590 2023-11-28T18:32:33.143 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.145 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.146 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.146 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/591 2023-11-28T18:32:33.151 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.153 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.153 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.153 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/592 2023-11-28T18:32:33.159 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:32:33.160 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.161 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.162 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.162 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/593 2023-11-28T18:32:33.169 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:32:33.169 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.172 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.172 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.172 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/594 2023-11-28T18:32:33.178 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.180 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.181 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.181 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/595 2023-11-28T18:32:33.189 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.192 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.192 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.192 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/596 2023-11-28T18:32:33.196 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.199 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.199 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.199 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/597 2023-11-28T18:32:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.207 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.207 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.207 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/598 2023-11-28T18:32:33.213 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.215 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.215 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.215 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/599 2023-11-28T18:32:33.221 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.224 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.224 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.225 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/600 2023-11-28T18:32:33.230 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.232 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.232 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.233 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/601 2023-11-28T18:32:33.238 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.240 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.241 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.241 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/602 2023-11-28T18:32:33.247 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.249 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.249 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.250 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/603 2023-11-28T18:32:33.256 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.258 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.258 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.258 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/604 2023-11-28T18:32:33.264 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.267 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.268 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.268 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/605 2023-11-28T18:32:33.272 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.274 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.274 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.274 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/606 2023-11-28T18:32:33.282 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.283 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.284 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.284 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/607 2023-11-28T18:32:33.289 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.291 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.291 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.292 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/608 2023-11-28T18:32:33.298 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.300 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.300 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.300 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/609 2023-11-28T18:32:33.306 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.308 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.308 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.309 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/610 2023-11-28T18:32:33.314 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.316 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.316 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.317 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/611 2023-11-28T18:32:33.323 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.325 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.325 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.325 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/612 2023-11-28T18:32:33.332 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.334 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.334 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.334 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/613 2023-11-28T18:32:33.340 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.342 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.343 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.343 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/614 2023-11-28T18:32:33.349 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.351 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.353 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.353 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/615 2023-11-28T18:32:33.358 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.361 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/616 2023-11-28T18:32:33.367 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/617 2023-11-28T18:32:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/618 2023-11-28T18:32:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/619 2023-11-28T18:32:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.396 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.396 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.396 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/620 2023-11-28T18:32:33.402 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.405 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.405 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.405 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/621 2023-11-28T18:32:33.411 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.414 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.414 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.414 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/622 2023-11-28T18:32:33.420 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.422 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.422 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.423 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/623 2023-11-28T18:32:33.428 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.431 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.431 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.431 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/624 2023-11-28T18:32:33.437 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.439 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.439 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.439 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/625 2023-11-28T18:32:33.446 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.448 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.448 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.448 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/626 2023-11-28T18:32:33.454 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.456 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.456 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.456 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/627 2023-11-28T18:32:33.463 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.465 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.465 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.465 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/628 2023-11-28T18:32:33.471 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.473 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.473 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.473 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/629 2023-11-28T18:32:33.479 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.481 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.481 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.482 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/630 2023-11-28T18:32:33.487 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.489 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.489 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.489 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/631 2023-11-28T18:32:33.495 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.497 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.497 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.497 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/632 2023-11-28T18:32:33.504 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.506 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.506 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.506 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/633 2023-11-28T18:32:33.512 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.515 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.515 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.515 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/634 2023-11-28T18:32:33.522 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.524 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.524 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.525 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/635 2023-11-28T18:32:33.533 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.535 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.535 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.535 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/636 2023-11-28T18:32:33.541 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.544 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.544 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.544 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/637 2023-11-28T18:32:33.550 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.553 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.553 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.553 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/638 2023-11-28T18:32:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.562 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.562 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.562 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/639 2023-11-28T18:32:33.568 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.571 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.571 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.571 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/640 2023-11-28T18:32:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/641 2023-11-28T18:32:33.586 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/642 2023-11-28T18:32:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.598 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.598 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.598 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/643 2023-11-28T18:32:33.604 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.606 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.606 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.606 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/644 2023-11-28T18:32:33.612 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.614 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.614 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.614 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/645 2023-11-28T18:32:33.620 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.622 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.622 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.623 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/646 2023-11-28T18:32:33.628 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.631 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.631 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.631 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/647 2023-11-28T18:32:33.637 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.665 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.665 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.666 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/648 2023-11-28T18:32:33.673 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.678 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.678 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.678 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/649 2023-11-28T18:32:33.686 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.689 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.689 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.689 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/650 2023-11-28T18:32:33.696 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.700 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.700 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.700 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/651 2023-11-28T18:32:33.707 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.710 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.710 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.710 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/652 2023-11-28T18:32:33.718 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.721 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.721 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.721 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/653 2023-11-28T18:32:33.728 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.732 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.732 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.732 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/654 2023-11-28T18:32:33.739 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/655 2023-11-28T18:32:33.751 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.753 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.753 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.754 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/656 2023-11-28T18:32:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/657 2023-11-28T18:32:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.776 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.776 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.776 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/658 2023-11-28T18:32:33.784 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.788 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.788 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.789 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/659 2023-11-28T18:32:33.796 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.800 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.800 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.800 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/660 2023-11-28T18:32:33.808 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.812 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.812 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.813 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/661 2023-11-28T18:32:33.821 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.825 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.825 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.825 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/662 2023-11-28T18:32:33.832 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.836 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.836 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.836 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/663 2023-11-28T18:32:33.844 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.847 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.847 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.847 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/664 2023-11-28T18:32:33.854 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.858 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.858 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.858 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/665 2023-11-28T18:32:33.865 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.868 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.868 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.868 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/666 2023-11-28T18:32:33.875 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.878 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.879 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.879 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/667 2023-11-28T18:32:33.886 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.889 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.889 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.889 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/668 2023-11-28T18:32:33.896 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.899 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.899 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.899 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/669 2023-11-28T18:32:33.907 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.910 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.910 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.910 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/670 2023-11-28T18:32:33.918 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.921 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.921 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.922 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/671 2023-11-28T18:32:33.928 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.932 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.932 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/672 2023-11-28T18:32:33.939 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.942 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.942 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.942 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/673 2023-11-28T18:32:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.952 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.953 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.953 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/674 2023-11-28T18:32:33.960 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.964 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.965 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.965 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/675 2023-11-28T18:32:33.973 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.976 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.976 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.976 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/676 2023-11-28T18:32:33.983 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.986 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.986 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.986 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/677 2023-11-28T18:32:33.993 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:33.996 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:33.996 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:33.997 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/678 2023-11-28T18:32:34.004 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.007 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.007 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.008 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/679 2023-11-28T18:32:34.016 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.018 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.018 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.018 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/680 2023-11-28T18:32:34.026 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.029 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.029 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.029 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/681 2023-11-28T18:32:34.036 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.039 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.039 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.040 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/682 2023-11-28T18:32:34.046 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.050 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.050 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.050 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/683 2023-11-28T18:32:34.057 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.061 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.061 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.061 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/684 2023-11-28T18:32:34.068 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.071 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.072 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.072 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/685 2023-11-28T18:32:34.079 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.083 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.083 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.083 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/686 2023-11-28T18:32:34.091 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.094 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.095 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.095 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/687 2023-11-28T18:32:34.101 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.104 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.104 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.104 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/688 2023-11-28T18:32:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.115 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.115 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.115 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/689 2023-11-28T18:32:34.122 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.125 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.125 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.125 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/690 2023-11-28T18:32:34.132 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.135 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.135 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.136 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/691 2023-11-28T18:32:34.142 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.146 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.149 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.149 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/692 2023-11-28T18:32:34.153 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.156 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.156 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.157 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/693 2023-11-28T18:32:34.163 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.166 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.167 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.167 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/694 2023-11-28T18:32:34.174 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.177 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.177 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.177 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/695 2023-11-28T18:32:34.184 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.188 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.188 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.188 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/696 2023-11-28T18:32:34.195 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.198 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.198 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.198 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/697 2023-11-28T18:32:34.206 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.209 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.209 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.209 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/698 2023-11-28T18:32:34.216 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.220 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.220 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.220 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/699 2023-11-28T18:32:34.227 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.231 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.231 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.231 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/700 2023-11-28T18:32:34.238 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.241 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.241 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.242 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/701 2023-11-28T18:32:34.249 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.252 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.252 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.252 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/702 2023-11-28T18:32:34.257 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.261 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.261 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.261 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/703 2023-11-28T18:32:34.268 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.271 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.271 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.271 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/704 2023-11-28T18:32:34.278 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.281 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.282 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.282 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/705 2023-11-28T18:32:34.289 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/706 2023-11-28T18:32:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/707 2023-11-28T18:32:34.309 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.312 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.312 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.312 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/708 2023-11-28T18:32:34.319 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.323 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.323 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.323 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/709 2023-11-28T18:32:34.330 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:34.333 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:34.333 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:34.334 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/710 2023-11-28T18:32:34.349 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:38.805 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:38.806 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:38.806 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/711 2023-11-28T18:32:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:38.818 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:38.818 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/712 2023-11-28T18:32:43.714 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:43.720 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:43.720 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:43.720 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/713 2023-11-28T18:32:43.750 INFO:teuthology.orchestra.run.smithi105.stderr:+ echo file 2023-11-28T18:32:43.759 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i++ )) 2023-11-28T18:32:43.760 INFO:teuthology.orchestra.run.smithi105.stderr:+ (( i < 1000 )) 2023-11-28T18:32:43.760 INFO:teuthology.orchestra.run.smithi105.stderr:+ mkdir -p tree/714 2023-11-28T18:32:43.793 INFO:tasks.ceph.mds.b.smithi148.stderr:./src/mds/Locker.cc: In function 'bool Locker::xlock_start(SimpleLock*, MDRequestRef&)' thread 7fc503617640 time 2023-11-28T18:32:43.798672+0000 2023-11-28T18:32:43.794 INFO:tasks.ceph.mds.b.smithi148.stderr:./src/mds/Locker.cc: 1963: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2023-11-28T18:32:43.794 INFO:tasks.ceph.mds.b.smithi148.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-11-28T18:32:43.794 INFO:tasks.ceph.mds.b.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fc50956fe96] 2023-11-28T18:32:43.795 INFO:tasks.ceph.mds.b.smithi148.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x17004a) [0x7fc50957004a] 2023-11-28T18:32:43.795 INFO:tasks.ceph.mds.b.smithi148.stderr: 3: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x8d8) [0x55cda60c03e8] 2023-11-28T18:32:43.795 INFO:tasks.ceph.mds.b.smithi148.stderr: 4: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20b4) [0x55cda60b5174] 2023-11-28T18:32:43.795 INFO:tasks.ceph.mds.b.smithi148.stderr: 5: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x8d3) [0x55cda6066033] 2023-11-28T18:32:43.795 INFO:tasks.ceph.mds.b.smithi148.stderr: 6: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x21c) [0x55cda5fa0dec] 2023-11-28T18:32:43.795 INFO:tasks.ceph.mds.b.smithi148.stderr: 7: (Server::handle_client_mkdir(boost::intrusive_ptr&)+0x60) [0x55cda5fbc730] 2023-11-28T18:32:43.795 INFO:tasks.ceph.mds.b.smithi148.stderr: 8: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x801) [0x55cda5f9da91] 2023-11-28T18:32:43.795 INFO:tasks.ceph.mds.b.smithi148.stderr: 9: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6ac) [0x55cda5f9e34c] 2023-11-28T18:32:43.795 INFO:tasks.ceph.mds.b.smithi148.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cda5f8a837] 2023-11-28T18:32:43.795 INFO:tasks.ceph.mds.b.smithi148.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4cb) [0x55cda5f0e5fb] 2023-11-28T18:32:43.795 INFO:tasks.ceph.mds.b.smithi148.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cda5f0eacc] 2023-11-28T18:32:43.796 INFO:tasks.ceph.mds.b.smithi148.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55cda5ef98e5] 2023-11-28T18:32:43.796 INFO:tasks.ceph.mds.b.smithi148.stderr: 14: (DispatchQueue::entry()+0x5ea) [0x7fc50975c11a] 2023-11-28T18:32:43.796 INFO:tasks.ceph.mds.b.smithi148.stderr: 15: /usr/lib/ceph/libceph-common.so.2(+0x3ee991) [0x7fc5097ee991] 2023-11-28T18:32:43.796 INFO:tasks.ceph.mds.b.smithi148.stderr: 16: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7fc508c94b43] 2023-11-28T18:32:43.796 INFO:tasks.ceph.mds.b.smithi148.stderr: 17: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7fc508d26a00] 2023-11-28T18:32:43.796 INFO:tasks.ceph.mds.b.smithi148.stderr:2023-11-28T18:32:43.795+0000 7fc503617640 -1 ./src/mds/Locker.cc: In function 'bool Locker::xlock_start(SimpleLock*, MDRequestRef&)' thread 7fc503617640 time 2023-11-28T18:32:43.798672+0000 2023-11-28T18:32:43.796 INFO:tasks.ceph.mds.b.smithi148.stderr:./src/mds/Locker.cc: 1963: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2023-11-28T18:32:43.796 INFO:tasks.ceph.mds.b.smithi148.stderr: 2023-11-28T18:32:43.796 INFO:tasks.ceph.mds.b.smithi148.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-11-28T18:32:43.796 INFO:tasks.ceph.mds.b.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fc50956fe96] 2023-11-28T18:32:43.796 INFO:tasks.ceph.mds.b.smithi148.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x17004a) [0x7fc50957004a] 2023-11-28T18:32:43.796 INFO:tasks.ceph.mds.b.smithi148.stderr: 3: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x8d8) [0x55cda60c03e8] 2023-11-28T18:32:43.797 INFO:tasks.ceph.mds.b.smithi148.stderr: 4: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20b4) [0x55cda60b5174] 2023-11-28T18:32:43.797 INFO:tasks.ceph.mds.b.smithi148.stderr: 5: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x8d3) [0x55cda6066033] 2023-11-28T18:32:43.797 INFO:tasks.ceph.mds.b.smithi148.stderr: 6: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x21c) [0x55cda5fa0dec] 2023-11-28T18:32:43.797 INFO:tasks.ceph.mds.b.smithi148.stderr: 7: (Server::handle_client_mkdir(boost::intrusive_ptr&)+0x60) [0x55cda5fbc730] 2023-11-28T18:32:43.797 INFO:tasks.ceph.mds.b.smithi148.stderr: 8: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x801) [0x55cda5f9da91] 2023-11-28T18:32:43.797 INFO:tasks.ceph.mds.b.smithi148.stderr: 9: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6ac) [0x55cda5f9e34c] 2023-11-28T18:32:43.797 INFO:tasks.ceph.mds.b.smithi148.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cda5f8a837] 2023-11-28T18:32:43.797 INFO:tasks.ceph.mds.b.smithi148.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4cb) [0x55cda5f0e5fb] 2023-11-28T18:32:43.797 INFO:tasks.ceph.mds.b.smithi148.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cda5f0eacc] 2023-11-28T18:32:43.797 INFO:tasks.ceph.mds.b.smithi148.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55cda5ef98e5] 2023-11-28T18:32:43.797 INFO:tasks.ceph.mds.b.smithi148.stderr: 14: (DispatchQueue::entry()+0x5ea) [0x7fc50975c11a] 2023-11-28T18:32:43.797 INFO:tasks.ceph.mds.b.smithi148.stderr: 15: /usr/lib/ceph/libceph-common.so.2(+0x3ee991) [0x7fc5097ee991] 2023-11-28T18:32:43.798 INFO:tasks.ceph.mds.b.smithi148.stderr: 16: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7fc508c94b43] 2023-11-28T18:32:43.798 INFO:tasks.ceph.mds.b.smithi148.stderr: 17: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7fc508d26a00] 2023-11-28T18:32:43.798 INFO:tasks.ceph.mds.b.smithi148.stderr: 2023-11-28T18:32:43.798 INFO:tasks.ceph.mds.b.smithi148.stderr:*** Caught signal (Aborted) ** 2023-11-28T18:32:43.798 INFO:tasks.ceph.mds.b.smithi148.stderr: in thread 7fc503617640 thread_name:ms_dispatch 2023-11-28T18:32:43.799 INFO:tasks.ceph.mds.b.smithi148.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-11-28T18:32:43.799 INFO:tasks.ceph.mds.b.smithi148.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fc508c42520] 2023-11-28T18:32:43.799 INFO:tasks.ceph.mds.b.smithi148.stderr: 2: pthread_kill() 2023-11-28T18:32:43.799 INFO:tasks.ceph.mds.b.smithi148.stderr: 3: raise() 2023-11-28T18:32:43.799 INFO:tasks.ceph.mds.b.smithi148.stderr: 4: abort() 2023-11-28T18:32:43.799 INFO:tasks.ceph.mds.b.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fc50956fef0] 2023-11-28T18:32:43.799 INFO:tasks.ceph.mds.b.smithi148.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x17004a) [0x7fc50957004a] 2023-11-28T18:32:43.799 INFO:tasks.ceph.mds.b.smithi148.stderr: 7: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x8d8) [0x55cda60c03e8] 2023-11-28T18:32:43.800 INFO:tasks.ceph.mds.b.smithi148.stderr: 8: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20b4) [0x55cda60b5174] 2023-11-28T18:32:43.800 INFO:tasks.ceph.mds.b.smithi148.stderr: 9: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x8d3) [0x55cda6066033] 2023-11-28T18:32:43.800 INFO:tasks.ceph.mds.b.smithi148.stderr: 10: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x21c) [0x55cda5fa0dec] 2023-11-28T18:32:43.800 INFO:tasks.ceph.mds.b.smithi148.stderr: 11: (Server::handle_client_mkdir(boost::intrusive_ptr&)+0x60) [0x55cda5fbc730] 2023-11-28T18:32:43.800 INFO:tasks.ceph.mds.b.smithi148.stderr: 12: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x801) [0x55cda5f9da91] 2023-11-28T18:32:43.800 INFO:tasks.ceph.mds.b.smithi148.stderr: 13: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6ac) [0x55cda5f9e34c] 2023-11-28T18:32:43.800 INFO:tasks.ceph.mds.b.smithi148.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cda5f8a837] 2023-11-28T18:32:43.800 INFO:tasks.ceph.mds.b.smithi148.stderr: 15: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4cb) [0x55cda5f0e5fb] 2023-11-28T18:32:43.800 INFO:tasks.ceph.mds.b.smithi148.stderr: 16: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cda5f0eacc] 2023-11-28T18:32:43.800 INFO:tasks.ceph.mds.b.smithi148.stderr: 17: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55cda5ef98e5] 2023-11-28T18:32:43.800 INFO:tasks.ceph.mds.b.smithi148.stderr: 18: (DispatchQueue::entry()+0x5ea) [0x7fc50975c11a] 2023-11-28T18:32:43.800 INFO:tasks.ceph.mds.b.smithi148.stderr: 19: /usr/lib/ceph/libceph-common.so.2(+0x3ee991) [0x7fc5097ee991] 2023-11-28T18:32:43.801 INFO:tasks.ceph.mds.b.smithi148.stderr: 20: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7fc508c94b43] 2023-11-28T18:32:43.801 INFO:tasks.ceph.mds.b.smithi148.stderr: 21: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7fc508d26a00] 2023-11-28T18:32:43.801 INFO:tasks.ceph.mds.b.smithi148.stderr:2023-11-28T18:32:43.795+0000 7fc503617640 -1 *** Caught signal (Aborted) ** 2023-11-28T18:32:43.801 INFO:tasks.ceph.mds.b.smithi148.stderr: in thread 7fc503617640 thread_name:ms_dispatch 2023-11-28T18:32:43.801 INFO:tasks.ceph.mds.b.smithi148.stderr: 2023-11-28T18:32:43.801 INFO:tasks.ceph.mds.b.smithi148.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-11-28T18:32:43.801 INFO:tasks.ceph.mds.b.smithi148.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fc508c42520] 2023-11-28T18:32:43.801 INFO:tasks.ceph.mds.b.smithi148.stderr: 2: pthread_kill() 2023-11-28T18:32:43.801 INFO:tasks.ceph.mds.b.smithi148.stderr: 3: raise() 2023-11-28T18:32:43.801 INFO:tasks.ceph.mds.b.smithi148.stderr: 4: abort() 2023-11-28T18:32:43.801 INFO:tasks.ceph.mds.b.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fc50956fef0] 2023-11-28T18:32:43.801 INFO:tasks.ceph.mds.b.smithi148.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x17004a) [0x7fc50957004a] 2023-11-28T18:32:43.802 INFO:tasks.ceph.mds.b.smithi148.stderr: 7: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x8d8) [0x55cda60c03e8] 2023-11-28T18:32:43.802 INFO:tasks.ceph.mds.b.smithi148.stderr: 8: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20b4) [0x55cda60b5174] 2023-11-28T18:32:43.802 INFO:tasks.ceph.mds.b.smithi148.stderr: 9: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x8d3) [0x55cda6066033] 2023-11-28T18:32:43.802 INFO:tasks.ceph.mds.b.smithi148.stderr: 10: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x21c) [0x55cda5fa0dec] 2023-11-28T18:32:43.802 INFO:tasks.ceph.mds.b.smithi148.stderr: 11: (Server::handle_client_mkdir(boost::intrusive_ptr&)+0x60) [0x55cda5fbc730] 2023-11-28T18:32:43.802 INFO:tasks.ceph.mds.b.smithi148.stderr: 12: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x801) [0x55cda5f9da91] 2023-11-28T18:32:43.802 INFO:tasks.ceph.mds.b.smithi148.stderr: 13: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6ac) [0x55cda5f9e34c] 2023-11-28T18:32:43.802 INFO:tasks.ceph.mds.b.smithi148.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cda5f8a837] 2023-11-28T18:32:43.802 INFO:tasks.ceph.mds.b.smithi148.stderr: 15: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4cb) [0x55cda5f0e5fb] 2023-11-28T18:32:43.802 INFO:tasks.ceph.mds.b.smithi148.stderr: 16: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cda5f0eacc] 2023-11-28T18:32:43.802 INFO:tasks.ceph.mds.b.smithi148.stderr: 17: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55cda5ef98e5] 2023-11-28T18:32:43.802 INFO:tasks.ceph.mds.b.smithi148.stderr: 18: (DispatchQueue::entry()+0x5ea) [0x7fc50975c11a] 2023-11-28T18:32:43.803 INFO:tasks.ceph.mds.b.smithi148.stderr: 19: /usr/lib/ceph/libceph-common.so.2(+0x3ee991) [0x7fc5097ee991] 2023-11-28T18:32:43.803 INFO:tasks.ceph.mds.b.smithi148.stderr: 20: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7fc508c94b43] 2023-11-28T18:32:43.803 INFO:tasks.ceph.mds.b.smithi148.stderr: 21: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7fc508d26a00] 2023-11-28T18:32:43.803 INFO:tasks.ceph.mds.b.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-28T18:32:43.803 INFO:tasks.ceph.mds.b.smithi148.stderr: 2023-11-28T18:32:43.834 INFO:tasks.ceph.mds.b.smithi148.stderr: -3> 2023-11-28T18:32:43.795+0000 7fc503617640 -1 ./src/mds/Locker.cc: In function 'bool Locker::xlock_start(SimpleLock*, MDRequestRef&)' thread 7fc503617640 time 2023-11-28T18:32:43.798672+0000 2023-11-28T18:32:43.834 INFO:tasks.ceph.mds.b.smithi148.stderr:./src/mds/Locker.cc: 1963: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2023-11-28T18:32:43.834 INFO:tasks.ceph.mds.b.smithi148.stderr: 2023-11-28T18:32:43.834 INFO:tasks.ceph.mds.b.smithi148.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-11-28T18:32:43.834 INFO:tasks.ceph.mds.b.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fc50956fe96] 2023-11-28T18:32:43.834 INFO:tasks.ceph.mds.b.smithi148.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x17004a) [0x7fc50957004a] 2023-11-28T18:32:43.834 INFO:tasks.ceph.mds.b.smithi148.stderr: 3: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x8d8) [0x55cda60c03e8] 2023-11-28T18:32:43.835 INFO:tasks.ceph.mds.b.smithi148.stderr: 4: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20b4) [0x55cda60b5174] 2023-11-28T18:32:43.835 INFO:tasks.ceph.mds.b.smithi148.stderr: 5: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x8d3) [0x55cda6066033] 2023-11-28T18:32:43.835 INFO:tasks.ceph.mds.b.smithi148.stderr: 6: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x21c) [0x55cda5fa0dec] 2023-11-28T18:32:43.835 INFO:tasks.ceph.mds.b.smithi148.stderr: 7: (Server::handle_client_mkdir(boost::intrusive_ptr&)+0x60) [0x55cda5fbc730] 2023-11-28T18:32:43.835 INFO:tasks.ceph.mds.b.smithi148.stderr: 8: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x801) [0x55cda5f9da91] 2023-11-28T18:32:43.835 INFO:tasks.ceph.mds.b.smithi148.stderr: 9: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6ac) [0x55cda5f9e34c] 2023-11-28T18:32:43.835 INFO:tasks.ceph.mds.b.smithi148.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cda5f8a837] 2023-11-28T18:32:43.835 INFO:tasks.ceph.mds.b.smithi148.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4cb) [0x55cda5f0e5fb] 2023-11-28T18:32:43.835 INFO:tasks.ceph.mds.b.smithi148.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cda5f0eacc] 2023-11-28T18:32:43.835 INFO:tasks.ceph.mds.b.smithi148.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55cda5ef98e5] 2023-11-28T18:32:43.835 INFO:tasks.ceph.mds.b.smithi148.stderr: 14: (DispatchQueue::entry()+0x5ea) [0x7fc50975c11a] 2023-11-28T18:32:43.835 INFO:tasks.ceph.mds.b.smithi148.stderr: 15: /usr/lib/ceph/libceph-common.so.2(+0x3ee991) [0x7fc5097ee991] 2023-11-28T18:32:43.836 INFO:tasks.ceph.mds.b.smithi148.stderr: 16: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7fc508c94b43] 2023-11-28T18:32:43.836 INFO:tasks.ceph.mds.b.smithi148.stderr: 17: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7fc508d26a00] 2023-11-28T18:32:43.836 INFO:tasks.ceph.mds.b.smithi148.stderr: 2023-11-28T18:32:43.836 INFO:tasks.ceph.mds.b.smithi148.stderr: 0> 2023-11-28T18:32:43.795+0000 7fc503617640 -1 *** Caught signal (Aborted) ** 2023-11-28T18:32:43.836 INFO:tasks.ceph.mds.b.smithi148.stderr: in thread 7fc503617640 thread_name:ms_dispatch 2023-11-28T18:32:43.836 INFO:tasks.ceph.mds.b.smithi148.stderr: 2023-11-28T18:32:43.836 INFO:tasks.ceph.mds.b.smithi148.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-11-28T18:32:43.836 INFO:tasks.ceph.mds.b.smithi148.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fc508c42520] 2023-11-28T18:32:43.836 INFO:tasks.ceph.mds.b.smithi148.stderr: 2: pthread_kill() 2023-11-28T18:32:43.836 INFO:tasks.ceph.mds.b.smithi148.stderr: 3: raise() 2023-11-28T18:32:43.836 INFO:tasks.ceph.mds.b.smithi148.stderr: 4: abort() 2023-11-28T18:32:43.836 INFO:tasks.ceph.mds.b.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fc50956fef0] 2023-11-28T18:32:43.837 INFO:tasks.ceph.mds.b.smithi148.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x17004a) [0x7fc50957004a] 2023-11-28T18:32:43.837 INFO:tasks.ceph.mds.b.smithi148.stderr: 7: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x8d8) [0x55cda60c03e8] 2023-11-28T18:32:43.837 INFO:tasks.ceph.mds.b.smithi148.stderr: 8: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20b4) [0x55cda60b5174] 2023-11-28T18:32:43.837 INFO:tasks.ceph.mds.b.smithi148.stderr: 9: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x8d3) [0x55cda6066033] 2023-11-28T18:32:43.837 INFO:tasks.ceph.mds.b.smithi148.stderr: 10: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x21c) [0x55cda5fa0dec] 2023-11-28T18:32:43.837 INFO:tasks.ceph.mds.b.smithi148.stderr: 11: (Server::handle_client_mkdir(boost::intrusive_ptr&)+0x60) [0x55cda5fbc730] 2023-11-28T18:32:43.837 INFO:tasks.ceph.mds.b.smithi148.stderr: 12: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x801) [0x55cda5f9da91] 2023-11-28T18:32:43.837 INFO:tasks.ceph.mds.b.smithi148.stderr: 13: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6ac) [0x55cda5f9e34c] 2023-11-28T18:32:43.837 INFO:tasks.ceph.mds.b.smithi148.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cda5f8a837] 2023-11-28T18:32:43.837 INFO:tasks.ceph.mds.b.smithi148.stderr: 15: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4cb) [0x55cda5f0e5fb] 2023-11-28T18:32:43.837 INFO:tasks.ceph.mds.b.smithi148.stderr: 16: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cda5f0eacc] 2023-11-28T18:32:43.837 INFO:tasks.ceph.mds.b.smithi148.stderr: 17: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55cda5ef98e5] 2023-11-28T18:32:43.838 INFO:tasks.ceph.mds.b.smithi148.stderr: 18: (DispatchQueue::entry()+0x5ea) [0x7fc50975c11a] 2023-11-28T18:32:43.838 INFO:tasks.ceph.mds.b.smithi148.stderr: 19: /usr/lib/ceph/libceph-common.so.2(+0x3ee991) [0x7fc5097ee991] 2023-11-28T18:32:43.838 INFO:tasks.ceph.mds.b.smithi148.stderr: 20: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7fc508c94b43] 2023-11-28T18:32:43.838 INFO:tasks.ceph.mds.b.smithi148.stderr: 21: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7fc508d26a00] 2023-11-28T18:32:43.838 INFO:tasks.ceph.mds.b.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-28T18:32:43.838 INFO:tasks.ceph.mds.b.smithi148.stderr: 2023-11-28T18:32:43.845 INFO:tasks.ceph.mds.b.smithi148.stderr: -3> 2023-11-28T18:32:43.795+0000 7fc503617640 -1 ./src/mds/Locker.cc: In function 'bool Locker::xlock_start(SimpleLock*, MDRequestRef&)' thread 7fc503617640 time 2023-11-28T18:32:43.798672+0000 2023-11-28T18:32:43.846 INFO:tasks.ceph.mds.b.smithi148.stderr:./src/mds/Locker.cc: 1963: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2023-11-28T18:32:43.846 INFO:tasks.ceph.mds.b.smithi148.stderr: 2023-11-28T18:32:43.846 INFO:tasks.ceph.mds.b.smithi148.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-11-28T18:32:43.846 INFO:tasks.ceph.mds.b.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fc50956fe96] 2023-11-28T18:32:43.846 INFO:tasks.ceph.mds.b.smithi148.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x17004a) [0x7fc50957004a] 2023-11-28T18:32:43.846 INFO:tasks.ceph.mds.b.smithi148.stderr: 3: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x8d8) [0x55cda60c03e8] 2023-11-28T18:32:43.846 INFO:tasks.ceph.mds.b.smithi148.stderr: 4: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20b4) [0x55cda60b5174] 2023-11-28T18:32:43.846 INFO:tasks.ceph.mds.b.smithi148.stderr: 5: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x8d3) [0x55cda6066033] 2023-11-28T18:32:43.846 INFO:tasks.ceph.mds.b.smithi148.stderr: 6: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x21c) [0x55cda5fa0dec] 2023-11-28T18:32:43.846 INFO:tasks.ceph.mds.b.smithi148.stderr: 7: (Server::handle_client_mkdir(boost::intrusive_ptr&)+0x60) [0x55cda5fbc730] 2023-11-28T18:32:43.846 INFO:tasks.ceph.mds.b.smithi148.stderr: 8: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x801) [0x55cda5f9da91] 2023-11-28T18:32:43.847 INFO:tasks.ceph.mds.b.smithi148.stderr: 9: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6ac) [0x55cda5f9e34c] 2023-11-28T18:32:43.847 INFO:tasks.ceph.mds.b.smithi148.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cda5f8a837] 2023-11-28T18:32:43.847 INFO:tasks.ceph.mds.b.smithi148.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4cb) [0x55cda5f0e5fb] 2023-11-28T18:32:43.847 INFO:tasks.ceph.mds.b.smithi148.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cda5f0eacc] 2023-11-28T18:32:43.847 INFO:tasks.ceph.mds.b.smithi148.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55cda5ef98e5] 2023-11-28T18:32:43.847 INFO:tasks.ceph.mds.b.smithi148.stderr: 14: (DispatchQueue::entry()+0x5ea) [0x7fc50975c11a] 2023-11-28T18:32:43.847 INFO:tasks.ceph.mds.b.smithi148.stderr: 15: /usr/lib/ceph/libceph-common.so.2(+0x3ee991) [0x7fc5097ee991] 2023-11-28T18:32:43.847 INFO:tasks.ceph.mds.b.smithi148.stderr: 16: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7fc508c94b43] 2023-11-28T18:32:43.847 INFO:tasks.ceph.mds.b.smithi148.stderr: 17: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7fc508d26a00] 2023-11-28T18:32:43.847 INFO:tasks.ceph.mds.b.smithi148.stderr: 2023-11-28T18:32:43.847 INFO:tasks.ceph.mds.b.smithi148.stderr: 0> 2023-11-28T18:32:43.795+0000 7fc503617640 -1 *** Caught signal (Aborted) ** 2023-11-28T18:32:43.847 INFO:tasks.ceph.mds.b.smithi148.stderr: in thread 7fc503617640 thread_name:ms_dispatch 2023-11-28T18:32:43.848 INFO:tasks.ceph.mds.b.smithi148.stderr: 2023-11-28T18:32:43.848 INFO:tasks.ceph.mds.b.smithi148.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-11-28T18:32:43.848 INFO:tasks.ceph.mds.b.smithi148.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fc508c42520] 2023-11-28T18:32:43.848 INFO:tasks.ceph.mds.b.smithi148.stderr: 2: pthread_kill() 2023-11-28T18:32:43.848 INFO:tasks.ceph.mds.b.smithi148.stderr: 3: raise() 2023-11-28T18:32:43.848 INFO:tasks.ceph.mds.b.smithi148.stderr: 4: abort() 2023-11-28T18:32:43.848 INFO:tasks.ceph.mds.b.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fc50956fef0] 2023-11-28T18:32:43.848 INFO:tasks.ceph.mds.b.smithi148.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x17004a) [0x7fc50957004a] 2023-11-28T18:32:43.848 INFO:tasks.ceph.mds.b.smithi148.stderr: 7: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x8d8) [0x55cda60c03e8] 2023-11-28T18:32:43.848 INFO:tasks.ceph.mds.b.smithi148.stderr: 8: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20b4) [0x55cda60b5174] 2023-11-28T18:32:43.848 INFO:tasks.ceph.mds.b.smithi148.stderr: 9: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x8d3) [0x55cda6066033] 2023-11-28T18:32:43.848 INFO:tasks.ceph.mds.b.smithi148.stderr: 10: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x21c) [0x55cda5fa0dec] 2023-11-28T18:32:43.849 INFO:tasks.ceph.mds.b.smithi148.stderr: 11: (Server::handle_client_mkdir(boost::intrusive_ptr&)+0x60) [0x55cda5fbc730] 2023-11-28T18:32:43.849 INFO:tasks.ceph.mds.b.smithi148.stderr: 12: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x801) [0x55cda5f9da91] 2023-11-28T18:32:43.849 INFO:tasks.ceph.mds.b.smithi148.stderr: 13: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6ac) [0x55cda5f9e34c] 2023-11-28T18:32:43.849 INFO:tasks.ceph.mds.b.smithi148.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55cda5f8a837] 2023-11-28T18:32:43.849 INFO:tasks.ceph.mds.b.smithi148.stderr: 15: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4cb) [0x55cda5f0e5fb] 2023-11-28T18:32:43.849 INFO:tasks.ceph.mds.b.smithi148.stderr: 16: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cda5f0eacc] 2023-11-28T18:32:43.849 INFO:tasks.ceph.mds.b.smithi148.stderr: 17: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55cda5ef98e5] 2023-11-28T18:32:43.849 INFO:tasks.ceph.mds.b.smithi148.stderr: 18: (DispatchQueue::entry()+0x5ea) [0x7fc50975c11a] 2023-11-28T18:32:43.849 INFO:tasks.ceph.mds.b.smithi148.stderr: 19: /usr/lib/ceph/libceph-common.so.2(+0x3ee991) [0x7fc5097ee991] 2023-11-28T18:32:43.849 INFO:tasks.ceph.mds.b.smithi148.stderr: 20: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7fc508c94b43] 2023-11-28T18:32:43.849 INFO:tasks.ceph.mds.b.smithi148.stderr: 21: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7fc508d26a00] 2023-11-28T18:32:43.849 INFO:tasks.ceph.mds.b.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-28T18:32:43.850 INFO:tasks.ceph.mds.b.smithi148.stderr: 2023-11-28T18:32:44.064 INFO:tasks.ceph.mds.b.smithi148.stderr:daemon-helper: command crashed with signal 6 2023-11-28T18:32:45.213 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-11-28T18:32:51.827 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~7s 2023-11-28T18:32:58.440 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~13s 2023-11-28T18:33:02.232 INFO:tasks.ceph.mds.a.smithi105.stderr:./src/mds/Locker.cc: In function 'bool Locker::xlock_start(SimpleLock*, MDRequestRef&)' thread 7f5d8032a640 time 2023-11-28T18:33:02.236328+0000 2023-11-28T18:33:02.233 INFO:tasks.ceph.mds.a.smithi105.stderr:./src/mds/Locker.cc: 1963: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2023-11-28T18:33:02.233 INFO:tasks.ceph.mds.a.smithi105.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-11-28T18:33:02.233 INFO:tasks.ceph.mds.a.smithi105.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f5d8636fe96] 2023-11-28T18:33:02.233 INFO:tasks.ceph.mds.a.smithi105.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x17004a) [0x7f5d8637004a] 2023-11-28T18:33:02.233 INFO:tasks.ceph.mds.a.smithi105.stderr: 3: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x8d8) [0x564edaee33e8] 2023-11-28T18:33:02.234 INFO:tasks.ceph.mds.a.smithi105.stderr: 4: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20b4) [0x564edaed8174] 2023-11-28T18:33:02.234 INFO:tasks.ceph.mds.a.smithi105.stderr: 5: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x8d3) [0x564edae89033] 2023-11-28T18:33:02.234 INFO:tasks.ceph.mds.a.smithi105.stderr: 6: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x21c) [0x564edadc3dec] 2023-11-28T18:33:02.234 INFO:tasks.ceph.mds.a.smithi105.stderr: 7: (Server::handle_client_mkdir(boost::intrusive_ptr&)+0x60) [0x564edaddf730] 2023-11-28T18:33:02.234 INFO:tasks.ceph.mds.a.smithi105.stderr: 8: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x801) [0x564edadc0a91] 2023-11-28T18:33:02.234 INFO:tasks.ceph.mds.a.smithi105.stderr: 9: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6ac) [0x564edadc134c] 2023-11-28T18:33:02.234 INFO:tasks.ceph.mds.a.smithi105.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564edadad837] 2023-11-28T18:33:02.234 INFO:tasks.ceph.mds.a.smithi105.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4cb) [0x564edad315fb] 2023-11-28T18:33:02.234 INFO:tasks.ceph.mds.a.smithi105.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564edad31acc] 2023-11-28T18:33:02.234 INFO:tasks.ceph.mds.a.smithi105.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x564edad1c8e5] 2023-11-28T18:33:02.234 INFO:tasks.ceph.mds.a.smithi105.stderr: 14: (DispatchQueue::entry()+0x5ea) [0x7f5d8655c11a] 2023-11-28T18:33:02.235 INFO:tasks.ceph.mds.a.smithi105.stderr: 15: /usr/lib/ceph/libceph-common.so.2(+0x3ee991) [0x7f5d865ee991] 2023-11-28T18:33:02.235 INFO:tasks.ceph.mds.a.smithi105.stderr: 16: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f5d85a94b43] 2023-11-28T18:33:02.235 INFO:tasks.ceph.mds.a.smithi105.stderr: 17: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f5d85b26a00] 2023-11-28T18:33:02.235 INFO:tasks.ceph.mds.a.smithi105.stderr:*** Caught signal (Aborted) ** 2023-11-28T18:33:02.235 INFO:tasks.ceph.mds.a.smithi105.stderr: in thread 7f5d8032a640 thread_name:ms_dispatch 2023-11-28T18:33:02.235 INFO:tasks.ceph.mds.a.smithi105.stderr:2023-11-28T18:33:02.231+0000 7f5d8032a640 -1 ./src/mds/Locker.cc: In function 'bool Locker::xlock_start(SimpleLock*, MDRequestRef&)' thread 7f5d8032a640 time 2023-11-28T18:33:02.236328+0000 2023-11-28T18:33:02.235 INFO:tasks.ceph.mds.a.smithi105.stderr:./src/mds/Locker.cc: 1963: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2023-11-28T18:33:02.235 INFO:tasks.ceph.mds.a.smithi105.stderr: 2023-11-28T18:33:02.235 INFO:tasks.ceph.mds.a.smithi105.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-11-28T18:33:02.235 INFO:tasks.ceph.mds.a.smithi105.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f5d8636fe96] 2023-11-28T18:33:02.235 INFO:tasks.ceph.mds.a.smithi105.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x17004a) [0x7f5d8637004a] 2023-11-28T18:33:02.235 INFO:tasks.ceph.mds.a.smithi105.stderr: 3: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x8d8) [0x564edaee33e8] 2023-11-28T18:33:02.236 INFO:tasks.ceph.mds.a.smithi105.stderr: 4: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20b4) [0x564edaed8174] 2023-11-28T18:33:02.236 INFO:tasks.ceph.mds.a.smithi105.stderr: 5: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x8d3) [0x564edae89033] 2023-11-28T18:33:02.236 INFO:tasks.ceph.mds.a.smithi105.stderr: 6: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x21c) [0x564edadc3dec] 2023-11-28T18:33:02.236 INFO:tasks.ceph.mds.a.smithi105.stderr: 7: (Server::handle_client_mkdir(boost::intrusive_ptr&)+0x60) [0x564edaddf730] 2023-11-28T18:33:02.236 INFO:tasks.ceph.mds.a.smithi105.stderr: 8: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x801) [0x564edadc0a91] 2023-11-28T18:33:02.236 INFO:tasks.ceph.mds.a.smithi105.stderr: 9: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6ac) [0x564edadc134c] 2023-11-28T18:33:02.236 INFO:tasks.ceph.mds.a.smithi105.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564edadad837] 2023-11-28T18:33:02.236 INFO:tasks.ceph.mds.a.smithi105.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4cb) [0x564edad315fb] 2023-11-28T18:33:02.236 INFO:tasks.ceph.mds.a.smithi105.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564edad31acc] 2023-11-28T18:33:02.236 INFO:tasks.ceph.mds.a.smithi105.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x564edad1c8e5] 2023-11-28T18:33:02.236 INFO:tasks.ceph.mds.a.smithi105.stderr: 14: (DispatchQueue::entry()+0x5ea) [0x7f5d8655c11a] 2023-11-28T18:33:02.236 INFO:tasks.ceph.mds.a.smithi105.stderr: 15: /usr/lib/ceph/libceph-common.so.2(+0x3ee991) [0x7f5d865ee991] 2023-11-28T18:33:02.237 INFO:tasks.ceph.mds.a.smithi105.stderr: 16: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f5d85a94b43] 2023-11-28T18:33:02.237 INFO:tasks.ceph.mds.a.smithi105.stderr: 17: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f5d85b26a00] 2023-11-28T18:33:02.237 INFO:tasks.ceph.mds.a.smithi105.stderr: 2023-11-28T18:33:02.237 INFO:tasks.ceph.mds.a.smithi105.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-11-28T18:33:02.237 INFO:tasks.ceph.mds.a.smithi105.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f5d85a42520] 2023-11-28T18:33:02.237 INFO:tasks.ceph.mds.a.smithi105.stderr: 2: pthread_kill() 2023-11-28T18:33:02.238 INFO:tasks.ceph.mds.a.smithi105.stderr: 3: raise() 2023-11-28T18:33:02.238 INFO:tasks.ceph.mds.a.smithi105.stderr: 4: abort() 2023-11-28T18:33:02.238 INFO:tasks.ceph.mds.a.smithi105.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f5d8636fef0] 2023-11-28T18:33:02.238 INFO:tasks.ceph.mds.a.smithi105.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x17004a) [0x7f5d8637004a] 2023-11-28T18:33:02.238 INFO:tasks.ceph.mds.a.smithi105.stderr: 7: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x8d8) [0x564edaee33e8] 2023-11-28T18:33:02.238 INFO:tasks.ceph.mds.a.smithi105.stderr: 8: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20b4) [0x564edaed8174] 2023-11-28T18:33:02.238 INFO:tasks.ceph.mds.a.smithi105.stderr: 9: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x8d3) [0x564edae89033] 2023-11-28T18:33:02.238 INFO:tasks.ceph.mds.a.smithi105.stderr: 10: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x21c) [0x564edadc3dec] 2023-11-28T18:33:02.238 INFO:tasks.ceph.mds.a.smithi105.stderr: 11: (Server::handle_client_mkdir(boost::intrusive_ptr&)+0x60) [0x564edaddf730] 2023-11-28T18:33:02.238 INFO:tasks.ceph.mds.a.smithi105.stderr: 12: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x801) [0x564edadc0a91] 2023-11-28T18:33:02.238 INFO:tasks.ceph.mds.a.smithi105.stderr: 13: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6ac) [0x564edadc134c] 2023-11-28T18:33:02.239 INFO:tasks.ceph.mds.a.smithi105.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564edadad837] 2023-11-28T18:33:02.239 INFO:tasks.ceph.mds.a.smithi105.stderr: 15: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4cb) [0x564edad315fb] 2023-11-28T18:33:02.239 INFO:tasks.ceph.mds.a.smithi105.stderr: 16: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564edad31acc] 2023-11-28T18:33:02.239 INFO:tasks.ceph.mds.a.smithi105.stderr: 17: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x564edad1c8e5] 2023-11-28T18:33:02.239 INFO:tasks.ceph.mds.a.smithi105.stderr: 18: (DispatchQueue::entry()+0x5ea) [0x7f5d8655c11a] 2023-11-28T18:33:02.239 INFO:tasks.ceph.mds.a.smithi105.stderr: 19: /usr/lib/ceph/libceph-common.so.2(+0x3ee991) [0x7f5d865ee991] 2023-11-28T18:33:02.239 INFO:tasks.ceph.mds.a.smithi105.stderr: 20: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f5d85a94b43] 2023-11-28T18:33:02.239 INFO:tasks.ceph.mds.a.smithi105.stderr: 21: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f5d85b26a00] 2023-11-28T18:33:02.239 INFO:tasks.ceph.mds.a.smithi105.stderr:2023-11-28T18:33:02.235+0000 7f5d8032a640 -1 *** Caught signal (Aborted) ** 2023-11-28T18:33:02.240 INFO:tasks.ceph.mds.a.smithi105.stderr: in thread 7f5d8032a640 thread_name:ms_dispatch 2023-11-28T18:33:02.240 INFO:tasks.ceph.mds.a.smithi105.stderr: 2023-11-28T18:33:02.240 INFO:tasks.ceph.mds.a.smithi105.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-11-28T18:33:02.240 INFO:tasks.ceph.mds.a.smithi105.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f5d85a42520] 2023-11-28T18:33:02.240 INFO:tasks.ceph.mds.a.smithi105.stderr: 2: pthread_kill() 2023-11-28T18:33:02.240 INFO:tasks.ceph.mds.a.smithi105.stderr: 3: raise() 2023-11-28T18:33:02.240 INFO:tasks.ceph.mds.a.smithi105.stderr: 4: abort() 2023-11-28T18:33:02.240 INFO:tasks.ceph.mds.a.smithi105.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f5d8636fef0] 2023-11-28T18:33:02.240 INFO:tasks.ceph.mds.a.smithi105.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x17004a) [0x7f5d8637004a] 2023-11-28T18:33:02.240 INFO:tasks.ceph.mds.a.smithi105.stderr: 7: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x8d8) [0x564edaee33e8] 2023-11-28T18:33:02.240 INFO:tasks.ceph.mds.a.smithi105.stderr: 8: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20b4) [0x564edaed8174] 2023-11-28T18:33:02.240 INFO:tasks.ceph.mds.a.smithi105.stderr: 9: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x8d3) [0x564edae89033] 2023-11-28T18:33:02.241 INFO:tasks.ceph.mds.a.smithi105.stderr: 10: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x21c) [0x564edadc3dec] 2023-11-28T18:33:02.241 INFO:tasks.ceph.mds.a.smithi105.stderr: 11: (Server::handle_client_mkdir(boost::intrusive_ptr&)+0x60) [0x564edaddf730] 2023-11-28T18:33:02.241 INFO:tasks.ceph.mds.a.smithi105.stderr: 12: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x801) [0x564edadc0a91] 2023-11-28T18:33:02.241 INFO:tasks.ceph.mds.a.smithi105.stderr: 13: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6ac) [0x564edadc134c] 2023-11-28T18:33:02.241 INFO:tasks.ceph.mds.a.smithi105.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564edadad837] 2023-11-28T18:33:02.241 INFO:tasks.ceph.mds.a.smithi105.stderr: 15: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4cb) [0x564edad315fb] 2023-11-28T18:33:02.241 INFO:tasks.ceph.mds.a.smithi105.stderr: 16: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564edad31acc] 2023-11-28T18:33:02.241 INFO:tasks.ceph.mds.a.smithi105.stderr: 17: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x564edad1c8e5] 2023-11-28T18:33:02.241 INFO:tasks.ceph.mds.a.smithi105.stderr: 18: (DispatchQueue::entry()+0x5ea) [0x7f5d8655c11a] 2023-11-28T18:33:02.241 INFO:tasks.ceph.mds.a.smithi105.stderr: 19: /usr/lib/ceph/libceph-common.so.2(+0x3ee991) [0x7f5d865ee991] 2023-11-28T18:33:02.241 INFO:tasks.ceph.mds.a.smithi105.stderr: 20: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f5d85a94b43] 2023-11-28T18:33:02.241 INFO:tasks.ceph.mds.a.smithi105.stderr: 21: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f5d85b26a00] 2023-11-28T18:33:02.242 INFO:tasks.ceph.mds.a.smithi105.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-28T18:33:02.242 INFO:tasks.ceph.mds.a.smithi105.stderr: 2023-11-28T18:33:02.254 INFO:tasks.ceph.mds.a.smithi105.stderr: -1> 2023-11-28T18:33:02.231+0000 7f5d8032a640 -1 ./src/mds/Locker.cc: In function 'bool Locker::xlock_start(SimpleLock*, MDRequestRef&)' thread 7f5d8032a640 time 2023-11-28T18:33:02.236328+0000 2023-11-28T18:33:02.255 INFO:tasks.ceph.mds.a.smithi105.stderr:./src/mds/Locker.cc: 1963: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2023-11-28T18:33:02.255 INFO:tasks.ceph.mds.a.smithi105.stderr: 2023-11-28T18:33:02.255 INFO:tasks.ceph.mds.a.smithi105.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-11-28T18:33:02.255 INFO:tasks.ceph.mds.a.smithi105.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f5d8636fe96] 2023-11-28T18:33:02.255 INFO:tasks.ceph.mds.a.smithi105.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x17004a) [0x7f5d8637004a] 2023-11-28T18:33:02.255 INFO:tasks.ceph.mds.a.smithi105.stderr: 3: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x8d8) [0x564edaee33e8] 2023-11-28T18:33:02.255 INFO:tasks.ceph.mds.a.smithi105.stderr: 4: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20b4) [0x564edaed8174] 2023-11-28T18:33:02.255 INFO:tasks.ceph.mds.a.smithi105.stderr: 5: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x8d3) [0x564edae89033] 2023-11-28T18:33:02.255 INFO:tasks.ceph.mds.a.smithi105.stderr: 6: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x21c) [0x564edadc3dec] 2023-11-28T18:33:02.255 INFO:tasks.ceph.mds.a.smithi105.stderr: 7: (Server::handle_client_mkdir(boost::intrusive_ptr&)+0x60) [0x564edaddf730] 2023-11-28T18:33:02.255 INFO:tasks.ceph.mds.a.smithi105.stderr: 8: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x801) [0x564edadc0a91] 2023-11-28T18:33:02.256 INFO:tasks.ceph.mds.a.smithi105.stderr: 9: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6ac) [0x564edadc134c] 2023-11-28T18:33:02.256 INFO:tasks.ceph.mds.a.smithi105.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564edadad837] 2023-11-28T18:33:02.256 INFO:tasks.ceph.mds.a.smithi105.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4cb) [0x564edad315fb] 2023-11-28T18:33:02.256 INFO:tasks.ceph.mds.a.smithi105.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564edad31acc] 2023-11-28T18:33:02.256 INFO:tasks.ceph.mds.a.smithi105.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x564edad1c8e5] 2023-11-28T18:33:02.256 INFO:tasks.ceph.mds.a.smithi105.stderr: 14: (DispatchQueue::entry()+0x5ea) [0x7f5d8655c11a] 2023-11-28T18:33:02.256 INFO:tasks.ceph.mds.a.smithi105.stderr: 15: /usr/lib/ceph/libceph-common.so.2(+0x3ee991) [0x7f5d865ee991] 2023-11-28T18:33:02.256 INFO:tasks.ceph.mds.a.smithi105.stderr: 16: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f5d85a94b43] 2023-11-28T18:33:02.256 INFO:tasks.ceph.mds.a.smithi105.stderr: 17: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f5d85b26a00] 2023-11-28T18:33:02.256 INFO:tasks.ceph.mds.a.smithi105.stderr: 2023-11-28T18:33:02.256 INFO:tasks.ceph.mds.a.smithi105.stderr: 0> 2023-11-28T18:33:02.235+0000 7f5d8032a640 -1 *** Caught signal (Aborted) ** 2023-11-28T18:33:02.256 INFO:tasks.ceph.mds.a.smithi105.stderr: in thread 7f5d8032a640 thread_name:ms_dispatch 2023-11-28T18:33:02.257 INFO:tasks.ceph.mds.a.smithi105.stderr: 2023-11-28T18:33:02.257 INFO:tasks.ceph.mds.a.smithi105.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-11-28T18:33:02.257 INFO:tasks.ceph.mds.a.smithi105.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f5d85a42520] 2023-11-28T18:33:02.257 INFO:tasks.ceph.mds.a.smithi105.stderr: 2: pthread_kill() 2023-11-28T18:33:02.257 INFO:tasks.ceph.mds.a.smithi105.stderr: 3: raise() 2023-11-28T18:33:02.257 INFO:tasks.ceph.mds.a.smithi105.stderr: 4: abort() 2023-11-28T18:33:02.257 INFO:tasks.ceph.mds.a.smithi105.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f5d8636fef0] 2023-11-28T18:33:02.257 INFO:tasks.ceph.mds.a.smithi105.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x17004a) [0x7f5d8637004a] 2023-11-28T18:33:02.257 INFO:tasks.ceph.mds.a.smithi105.stderr: 7: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x8d8) [0x564edaee33e8] 2023-11-28T18:33:02.257 INFO:tasks.ceph.mds.a.smithi105.stderr: 8: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20b4) [0x564edaed8174] 2023-11-28T18:33:02.257 INFO:tasks.ceph.mds.a.smithi105.stderr: 9: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x8d3) [0x564edae89033] 2023-11-28T18:33:02.258 INFO:tasks.ceph.mds.a.smithi105.stderr: 10: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x21c) [0x564edadc3dec] 2023-11-28T18:33:02.258 INFO:tasks.ceph.mds.a.smithi105.stderr: 11: (Server::handle_client_mkdir(boost::intrusive_ptr&)+0x60) [0x564edaddf730] 2023-11-28T18:33:02.258 INFO:tasks.ceph.mds.a.smithi105.stderr: 12: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x801) [0x564edadc0a91] 2023-11-28T18:33:02.258 INFO:tasks.ceph.mds.a.smithi105.stderr: 13: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6ac) [0x564edadc134c] 2023-11-28T18:33:02.258 INFO:tasks.ceph.mds.a.smithi105.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564edadad837] 2023-11-28T18:33:02.258 INFO:tasks.ceph.mds.a.smithi105.stderr: 15: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4cb) [0x564edad315fb] 2023-11-28T18:33:02.258 INFO:tasks.ceph.mds.a.smithi105.stderr: 16: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564edad31acc] 2023-11-28T18:33:02.258 INFO:tasks.ceph.mds.a.smithi105.stderr: 17: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x564edad1c8e5] 2023-11-28T18:33:02.258 INFO:tasks.ceph.mds.a.smithi105.stderr: 18: (DispatchQueue::entry()+0x5ea) [0x7f5d8655c11a] 2023-11-28T18:33:02.258 INFO:tasks.ceph.mds.a.smithi105.stderr: 19: /usr/lib/ceph/libceph-common.so.2(+0x3ee991) [0x7f5d865ee991] 2023-11-28T18:33:02.258 INFO:tasks.ceph.mds.a.smithi105.stderr: 20: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f5d85a94b43] 2023-11-28T18:33:02.258 INFO:tasks.ceph.mds.a.smithi105.stderr: 21: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f5d85b26a00] 2023-11-28T18:33:02.261 INFO:tasks.ceph.mds.a.smithi105.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-28T18:33:02.261 INFO:tasks.ceph.mds.a.smithi105.stderr: 2023-11-28T18:33:02.261 INFO:tasks.ceph.mds.a.smithi105.stderr: -1> 2023-11-28T18:33:02.231+0000 7f5d8032a640 -1 ./src/mds/Locker.cc: In function 'bool Locker::xlock_start(SimpleLock*, MDRequestRef&)' thread 7f5d8032a640 time 2023-11-28T18:33:02.236328+0000 2023-11-28T18:33:02.262 INFO:tasks.ceph.mds.a.smithi105.stderr:./src/mds/Locker.cc: 1963: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2023-11-28T18:33:02.262 INFO:tasks.ceph.mds.a.smithi105.stderr: 2023-11-28T18:33:02.262 INFO:tasks.ceph.mds.a.smithi105.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-11-28T18:33:02.262 INFO:tasks.ceph.mds.a.smithi105.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f5d8636fe96] 2023-11-28T18:33:02.262 INFO:tasks.ceph.mds.a.smithi105.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x17004a) [0x7f5d8637004a] 2023-11-28T18:33:02.262 INFO:tasks.ceph.mds.a.smithi105.stderr: 3: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x8d8) [0x564edaee33e8] 2023-11-28T18:33:02.263 INFO:tasks.ceph.mds.a.smithi105.stderr: 4: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20b4) [0x564edaed8174] 2023-11-28T18:33:02.263 INFO:tasks.ceph.mds.a.smithi105.stderr: 5: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x8d3) [0x564edae89033] 2023-11-28T18:33:02.263 INFO:tasks.ceph.mds.a.smithi105.stderr: 6: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x21c) [0x564edadc3dec] 2023-11-28T18:33:02.263 INFO:tasks.ceph.mds.a.smithi105.stderr: 7: (Server::handle_client_mkdir(boost::intrusive_ptr&)+0x60) [0x564edaddf730] 2023-11-28T18:33:02.263 INFO:tasks.ceph.mds.a.smithi105.stderr: 8: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x801) [0x564edadc0a91] 2023-11-28T18:33:02.263 INFO:tasks.ceph.mds.a.smithi105.stderr: 9: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6ac) [0x564edadc134c] 2023-11-28T18:33:02.263 INFO:tasks.ceph.mds.a.smithi105.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564edadad837] 2023-11-28T18:33:02.264 INFO:tasks.ceph.mds.a.smithi105.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4cb) [0x564edad315fb] 2023-11-28T18:33:02.264 INFO:tasks.ceph.mds.a.smithi105.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564edad31acc] 2023-11-28T18:33:02.264 INFO:tasks.ceph.mds.a.smithi105.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x564edad1c8e5] 2023-11-28T18:33:02.264 INFO:tasks.ceph.mds.a.smithi105.stderr: 14: (DispatchQueue::entry()+0x5ea) [0x7f5d8655c11a] 2023-11-28T18:33:02.264 INFO:tasks.ceph.mds.a.smithi105.stderr: 15: /usr/lib/ceph/libceph-common.so.2(+0x3ee991) [0x7f5d865ee991] 2023-11-28T18:33:02.264 INFO:tasks.ceph.mds.a.smithi105.stderr: 16: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f5d85a94b43] 2023-11-28T18:33:02.264 INFO:tasks.ceph.mds.a.smithi105.stderr: 17: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f5d85b26a00] 2023-11-28T18:33:02.264 INFO:tasks.ceph.mds.a.smithi105.stderr: 2023-11-28T18:33:02.264 INFO:tasks.ceph.mds.a.smithi105.stderr: 0> 2023-11-28T18:33:02.235+0000 7f5d8032a640 -1 *** Caught signal (Aborted) ** 2023-11-28T18:33:02.264 INFO:tasks.ceph.mds.a.smithi105.stderr: in thread 7f5d8032a640 thread_name:ms_dispatch 2023-11-28T18:33:02.264 INFO:tasks.ceph.mds.a.smithi105.stderr: 2023-11-28T18:33:02.264 INFO:tasks.ceph.mds.a.smithi105.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-11-28T18:33:02.265 INFO:tasks.ceph.mds.a.smithi105.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f5d85a42520] 2023-11-28T18:33:02.265 INFO:tasks.ceph.mds.a.smithi105.stderr: 2: pthread_kill() 2023-11-28T18:33:02.265 INFO:tasks.ceph.mds.a.smithi105.stderr: 3: raise() 2023-11-28T18:33:02.265 INFO:tasks.ceph.mds.a.smithi105.stderr: 4: abort() 2023-11-28T18:33:02.265 INFO:tasks.ceph.mds.a.smithi105.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f5d8636fef0] 2023-11-28T18:33:02.265 INFO:tasks.ceph.mds.a.smithi105.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x17004a) [0x7f5d8637004a] 2023-11-28T18:33:02.265 INFO:tasks.ceph.mds.a.smithi105.stderr: 7: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x8d8) [0x564edaee33e8] 2023-11-28T18:33:02.265 INFO:tasks.ceph.mds.a.smithi105.stderr: 8: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20b4) [0x564edaed8174] 2023-11-28T18:33:02.265 INFO:tasks.ceph.mds.a.smithi105.stderr: 9: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x8d3) [0x564edae89033] 2023-11-28T18:33:02.265 INFO:tasks.ceph.mds.a.smithi105.stderr: 10: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x21c) [0x564edadc3dec] 2023-11-28T18:33:02.265 INFO:tasks.ceph.mds.a.smithi105.stderr: 11: (Server::handle_client_mkdir(boost::intrusive_ptr&)+0x60) [0x564edaddf730] 2023-11-28T18:33:02.266 INFO:tasks.ceph.mds.a.smithi105.stderr: 12: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x801) [0x564edadc0a91] 2023-11-28T18:33:02.266 INFO:tasks.ceph.mds.a.smithi105.stderr: 13: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6ac) [0x564edadc134c] 2023-11-28T18:33:02.266 INFO:tasks.ceph.mds.a.smithi105.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564edadad837] 2023-11-28T18:33:02.266 INFO:tasks.ceph.mds.a.smithi105.stderr: 15: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4cb) [0x564edad315fb] 2023-11-28T18:33:02.266 INFO:tasks.ceph.mds.a.smithi105.stderr: 16: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564edad31acc] 2023-11-28T18:33:02.266 INFO:tasks.ceph.mds.a.smithi105.stderr: 17: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x564edad1c8e5] 2023-11-28T18:33:02.266 INFO:tasks.ceph.mds.a.smithi105.stderr: 18: (DispatchQueue::entry()+0x5ea) [0x7f5d8655c11a] 2023-11-28T18:33:02.266 INFO:tasks.ceph.mds.a.smithi105.stderr: 19: /usr/lib/ceph/libceph-common.so.2(+0x3ee991) [0x7f5d865ee991] 2023-11-28T18:33:02.266 INFO:tasks.ceph.mds.a.smithi105.stderr: 20: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f5d85a94b43] 2023-11-28T18:33:02.266 INFO:tasks.ceph.mds.a.smithi105.stderr: 21: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f5d85b26a00] 2023-11-28T18:33:02.266 INFO:tasks.ceph.mds.a.smithi105.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-28T18:33:02.266 INFO:tasks.ceph.mds.a.smithi105.stderr: 2023-11-28T18:33:02.487 INFO:tasks.ceph.mds.a.smithi105.stderr:daemon-helper: command crashed with signal 6 2023-11-28T18:33:03.509 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:33:03.515 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:33:04.431 INFO:tasks.ceph.mds.c.smithi105.stderr:2023-11-28T18:33:04.431+0000 7ff0bb051640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-28T18:33:04.953 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-11-28T18:33:04.953 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~20s 2023-11-28T18:33:11.464 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2023-11-28T18:33:11.464 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~26s 2023-11-28T18:33:17.977 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~13s 2023-11-28T18:33:17.977 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~33s 2023-11-28T18:33:24.491 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~20s 2023-11-28T18:33:24.492 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~39s 2023-11-28T18:33:31.006 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~26s 2023-11-28T18:33:31.006 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~46s 2023-11-28T18:33:33.565 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:33:33.570 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:33:37.518 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~33s 2023-11-28T18:33:37.518 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~52s 2023-11-28T18:33:44.032 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~39s 2023-11-28T18:33:44.032 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~59s 2023-11-28T18:33:50.543 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~46s 2023-11-28T18:33:50.543 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~65s 2023-11-28T18:33:57.056 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~52s 2023-11-28T18:33:57.056 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~72s 2023-11-28T18:34:03.569 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~59s 2023-11-28T18:34:03.569 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~78s 2023-11-28T18:34:03.637 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:34:03.641 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:34:10.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~65s 2023-11-28T18:34:10.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~85s 2023-11-28T18:34:16.591 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~72s 2023-11-28T18:34:16.592 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~91s 2023-11-28T18:34:23.105 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~78s 2023-11-28T18:34:23.105 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~98s 2023-11-28T18:34:29.618 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~85s 2023-11-28T18:34:29.618 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~105s 2023-11-28T18:34:33.708 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:34:33.713 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:34:36.130 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~91s 2023-11-28T18:34:36.130 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~111s 2023-11-28T18:34:42.641 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~98s 2023-11-28T18:34:42.641 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~118s 2023-11-28T18:34:49.151 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~104s 2023-11-28T18:34:49.152 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~124s 2023-11-28T18:34:55.661 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~111s 2023-11-28T18:34:55.661 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~131s 2023-11-28T18:35:02.171 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~117s 2023-11-28T18:35:02.171 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~137s 2023-11-28T18:35:03.767 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:35:03.772 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:35:08.683 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~124s 2023-11-28T18:35:08.684 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~144s 2023-11-28T18:35:15.195 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~130s 2023-11-28T18:35:15.196 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~150s 2023-11-28T18:35:21.708 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~137s 2023-11-28T18:35:21.708 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~157s 2023-11-28T18:35:28.221 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~143s 2023-11-28T18:35:28.221 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~163s 2023-11-28T18:35:34.045 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:35:34.050 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:35:34.733 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~150s 2023-11-28T18:35:34.733 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~170s 2023-11-28T18:35:41.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~156s 2023-11-28T18:35:41.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~176s 2023-11-28T18:35:47.758 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~163s 2023-11-28T18:35:47.758 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~183s 2023-11-28T18:35:54.270 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~169s 2023-11-28T18:35:54.270 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~189s 2023-11-28T18:36:00.781 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~176s 2023-11-28T18:36:00.782 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~196s 2023-11-28T18:36:04.143 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:36:04.147 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:36:07.293 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~182s 2023-11-28T18:36:07.294 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~202s 2023-11-28T18:36:13.807 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~189s 2023-11-28T18:36:13.835 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~209s 2023-11-28T18:36:20.347 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~195s 2023-11-28T18:36:20.348 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~215s 2023-11-28T18:36:26.860 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~202s 2023-11-28T18:36:26.861 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~222s 2023-11-28T18:36:33.374 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~208s 2023-11-28T18:36:33.374 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~228s 2023-11-28T18:36:34.207 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:36:34.211 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:36:39.885 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~215s 2023-11-28T18:36:39.885 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~235s 2023-11-28T18:36:46.398 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~221s 2023-11-28T18:36:46.398 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~241s 2023-11-28T18:36:52.910 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~228s 2023-11-28T18:36:52.910 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~248s 2023-11-28T18:36:59.422 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~234s 2023-11-28T18:36:59.422 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~254s 2023-11-28T18:37:04.270 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:37:04.274 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:37:05.934 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~241s 2023-11-28T18:37:05.935 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~261s 2023-11-28T18:37:12.446 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~247s 2023-11-28T18:37:12.447 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~267s 2023-11-28T18:37:18.960 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~254s 2023-11-28T18:37:18.960 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~274s 2023-11-28T18:37:23.072 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2023-11-28T18:37:25.473 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~261s 2023-11-28T18:37:25.473 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~280s 2023-11-28T18:37:31.985 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~267s 2023-11-28T18:37:31.985 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~287s 2023-11-28T18:37:34.586 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:37:34.590 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:37:38.496 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~274s 2023-11-28T18:37:38.496 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~293s 2023-11-28T18:37:43.761 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2023-11-28T18:37:45.008 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~280s 2023-11-28T18:37:45.008 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~300s 2023-11-28T18:37:51.519 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~287s 2023-11-28T18:37:51.520 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~306s 2023-11-28T18:37:51.520 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2023-11-28T18:37:51.520 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:37:51.520 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:37:51.531 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:37:51.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:37:51.581 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T18:37:51.581 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:37:51.683 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T18:37:51.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.720+0000 7fbf8f26b640 1 Processor -- start 2023-11-28T18:37:51.717 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.720+0000 7fbf8f26b640 1 -- start start 2023-11-28T18:37:51.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.720+0000 7fbf8f26b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbf880ad9f0 0x7fbf880ade80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:51.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.720+0000 7fbf8f26b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf88162820 con 0x7fbf880cc7d0 2023-11-28T18:37:51.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.720+0000 7fbf8f26b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf88161c10 con 0x7fbf8806d610 2023-11-28T18:37:51.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.720+0000 7fbf8f26b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf880ae3c0 con 0x7fbf880ad9f0 2023-11-28T18:37:51.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.720+0000 7fbf8c806640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbf880ad9f0 0x7fbf880ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:51.719 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.720+0000 7fbf8d808640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fbf880cc7d0 0x7fbf880ccbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51292/0 (socket says 172.21.15.105:51292) 2023-11-28T18:37:51.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.720+0000 7fbf8c806640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbf880ad9f0 0x7fbf880ade80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54164/0 (socket says 172.21.15.105:54164) 2023-11-28T18:37:51.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.720+0000 7fbf8d808640 1 -- 172.21.15.105:0/2624781873 learned_addr learned my addr 172.21.15.105:0/2624781873 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:37:51.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.720+0000 7fbf8c806640 1 -- 172.21.15.105:0/2624781873 >> v1:172.21.15.148:6789/0 conn(0x7fbf8806d610 legacy=0x7fbf880cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:51.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.720+0000 7fbf8c806640 1 -- 172.21.15.105:0/2624781873 >> v1:172.21.15.105:6789/0 conn(0x7fbf880cc7d0 legacy=0x7fbf880ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:51.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.720+0000 7fbf7ffff640 1 -- 172.21.15.105:0/2624781873 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3171549134 0 0) 0x7fbf88162820 con 0x7fbf880cc7d0 2023-11-28T18:37:51.720 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.720+0000 7fbf8c806640 1 -- 172.21.15.105:0/2624781873 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf88161ef0 con 0x7fbf880ad9f0 2023-11-28T18:37:51.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.724+0000 7fbf8c806640 1 --2- 172.21.15.105:0/2624781873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf880ad9f0 0x7fbf880ade80 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fbf780097c0 tx=0x7fbf7801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53fd39e45bd4064d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:51.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.724+0000 7fbf7ffff640 1 -- 172.21.15.105:0/2624781873 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf780ab650 con 0x7fbf880ad9f0 2023-11-28T18:37:51.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.724+0000 7fbf7ffff640 1 -- 172.21.15.105:0/2624781873 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf780ab7b0 con 0x7fbf880ad9f0 2023-11-28T18:37:51.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.724+0000 7fbf7ffff640 1 -- 172.21.15.105:0/2624781873 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf780abb40 con 0x7fbf880ad9f0 2023-11-28T18:37:51.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.724+0000 7fbf8f26b640 1 -- 172.21.15.105:0/2624781873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf880ad9f0 msgr2=0x7fbf880ade80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:51.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.724+0000 7fbf8f26b640 1 --2- 172.21.15.105:0/2624781873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf880ad9f0 0x7fbf880ade80 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fbf780097c0 tx=0x7fbf7801c450 comp rx=0 tx=0).stop 2023-11-28T18:37:51.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.724+0000 7fbf8f26b640 1 -- 172.21.15.105:0/2624781873 shutdown_connections 2023-11-28T18:37:51.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.724+0000 7fbf8f26b640 1 --2- 172.21.15.105:0/2624781873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf880ad9f0 0x7fbf880ade80 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:51.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.724+0000 7fbf8f26b640 1 -- 172.21.15.105:0/2624781873 >> 172.21.15.105:0/2624781873 conn(0x7fbf880ceb90 msgr2=0x7fbf880cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:51.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.724+0000 7fbf8f26b640 1 -- 172.21.15.105:0/2624781873 shutdown_connections 2023-11-28T18:37:51.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.724+0000 7fbf8f26b640 1 -- 172.21.15.105:0/2624781873 wait complete. 2023-11-28T18:37:51.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf8f26b640 1 Processor -- start 2023-11-28T18:37:51.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf8f26b640 1 -- start start 2023-11-28T18:37:51.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf8f26b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf8806d610 0x7fbf8815c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:51.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf8f26b640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbf880ad9f0 0x7fbf88162f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:51.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf8f26b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf880cc7d0 0x7fbf8815fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:51.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf8f26b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf88164190 con 0x7fbf8806d610 2023-11-28T18:37:51.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf8f26b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf881638f0 con 0x7fbf880cc7d0 2023-11-28T18:37:51.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf8f26b640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf88163bd0 con 0x7fbf880ad9f0 2023-11-28T18:37:51.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf8d007640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf8806d610 0x7fbf8815c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:51.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf8d007640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf8806d610 0x7fbf8815c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54178/0 (socket says 172.21.15.105:54178) 2023-11-28T18:37:51.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf8d007640 1 -- 172.21.15.105:0/1230576463 learned_addr learned my addr 172.21.15.105:0/1230576463 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:51.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf8d808640 1 --2- 172.21.15.105:0/1230576463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf880cc7d0 0x7fbf8815fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:51.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf8c806640 1 --2- 172.21.15.105:0/1230576463 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbf880ad9f0 0x7fbf88162f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:51.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf8d007640 1 -- 172.21.15.105:0/1230576463 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbf880ad9f0 msgr2=0x7fbf88162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:51.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf8d007640 1 --2- 172.21.15.105:0/1230576463 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbf880ad9f0 0x7fbf88162f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:51.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf8d007640 1 -- 172.21.15.105:0/1230576463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf880cc7d0 msgr2=0x7fbf8815fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:51.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf8d007640 1 --2- 172.21.15.105:0/1230576463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf880cc7d0 0x7fbf8815fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:51.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf8d007640 1 -- 172.21.15.105:0/1230576463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf78009420 con 0x7fbf8806d610 2023-11-28T18:37:51.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf8c806640 1 --2- 172.21.15.105:0/1230576463 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbf880ad9f0 0x7fbf88162f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:37:51.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf8d007640 1 --2- 172.21.15.105:0/1230576463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf8806d610 0x7fbf8815c2f0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400c3c0 tx=0x7fbf8400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:51.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf7dffb640 1 -- 172.21.15.105:0/1230576463 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf840052d0 con 0x7fbf8806d610 2023-11-28T18:37:51.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf7dffb640 1 -- 172.21.15.105:0/1230576463 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf84005430 con 0x7fbf8806d610 2023-11-28T18:37:51.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf7dffb640 1 -- 172.21.15.105:0/1230576463 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf84003840 con 0x7fbf8806d610 2023-11-28T18:37:51.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf8f26b640 1 -- 172.21.15.105:0/1230576463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf88163eb0 con 0x7fbf8806d610 2023-11-28T18:37:51.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.728+0000 7fbf8f26b640 1 -- 172.21.15.105:0/1230576463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf8815c830 con 0x7fbf8806d610 2023-11-28T18:37:51.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.732+0000 7fbf8f26b640 1 -- 172.21.15.105:0/1230576463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf88067570 con 0x7fbf8806d610 2023-11-28T18:37:51.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.732+0000 7fbf7dffb640 1 -- 172.21.15.105:0/1230576463 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fbf840039a0 con 0x7fbf8806d610 2023-11-28T18:37:51.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.732+0000 7fbf7dffb640 1 --2- 172.21.15.105:0/1230576463 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbf6c0668d0 0x7fbf6c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:51.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.732+0000 7fbf7dffb640 1 -- 172.21.15.105:0/1230576463 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fbf84083a90 con 0x7fbf8806d610 2023-11-28T18:37:51.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.732+0000 7fbf8c806640 1 --2- 172.21.15.105:0/1230576463 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbf6c0668d0 0x7fbf6c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:51.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.732+0000 7fbf8c806640 1 --2- 172.21.15.105:0/1230576463 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbf6c0668d0 0x7fbf6c068d90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fbf780af000 tx=0x7fbf780a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:51.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.736+0000 7fbf7dffb640 1 -- 172.21.15.105:0/1230576463 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbf84050bd0 con 0x7fbf8806d610 2023-11-28T18:37:51.782 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T18:37:51.782 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:37:51.782 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:37:51.791 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:37:51.887 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.888+0000 7fbf8f26b640 1 -- 172.21.15.105:0/1230576463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf880b4a40 con 0x7fbf8806d610 2023-11-28T18:37:51.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.892+0000 7fbf7dffb640 1 -- 172.21.15.105:0/1230576463 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+16923 (secure 0 0 0) 0x7fbf84017090 con 0x7fbf8806d610 2023-11-28T18:37:51.889 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:37:51.889 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":130,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:32:58.181121+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:31:56.971692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-28T18:31:57.881657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6835/3706038483":"2023-11-29T18:32:58.181068+0000","172.21.15.148:6833/3706038483":"2023-11-29T18:32:58.181068+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:37:51.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.892+0000 7fbf8f26b640 1 -- 172.21.15.105:0/1230576463 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbf6c0668d0 msgr2=0x7fbf6c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:51.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.892+0000 7fbf8f26b640 1 --2- 172.21.15.105:0/1230576463 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbf6c0668d0 0x7fbf6c068d90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fbf780af000 tx=0x7fbf780a7000 comp rx=0 tx=0).stop 2023-11-28T18:37:51.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.892+0000 7fbf8f26b640 1 -- 172.21.15.105:0/1230576463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf8806d610 msgr2=0x7fbf8815c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:51.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.892+0000 7fbf8f26b640 1 --2- 172.21.15.105:0/1230576463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf8806d610 0x7fbf8815c2f0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400c3c0 tx=0x7fbf8400c880 comp rx=0 tx=0).stop 2023-11-28T18:37:51.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.892+0000 7fbf8f26b640 1 -- 172.21.15.105:0/1230576463 shutdown_connections 2023-11-28T18:37:51.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.892+0000 7fbf8f26b640 1 --2- 172.21.15.105:0/1230576463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf880cc7d0 0x7fbf8815fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:51.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.892+0000 7fbf8f26b640 1 --2- 172.21.15.105:0/1230576463 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbf880ad9f0 0x7fbf88162f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:51.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.892+0000 7fbf8f26b640 1 --2- 172.21.15.105:0/1230576463 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fbf6c0668d0 0x7fbf6c068d90 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:51.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.892+0000 7fbf8f26b640 1 --2- 172.21.15.105:0/1230576463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf8806d610 0x7fbf8815c2f0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:51.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.892+0000 7fbf8f26b640 1 -- 172.21.15.105:0/1230576463 >> 172.21.15.105:0/1230576463 conn(0x7fbf880ceb90 msgr2=0x7fbf880cdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:51.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.892+0000 7fbf8f26b640 1 -- 172.21.15.105:0/1230576463 shutdown_connections 2023-11-28T18:37:51.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.892+0000 7fbf8f26b640 1 -- 172.21.15.105:0/1230576463 wait complete. 2023-11-28T18:37:51.911 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:31:56.971692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:31:57.881657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:37:51.912 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:37:51.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.980+0000 7f905b05a640 1 Processor -- start 2023-11-28T18:37:51.977 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.980+0000 7f905b05a640 1 -- start start 2023-11-28T18:37:51.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.980+0000 7f905b05a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f90540d3c30 0x7f90540d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:51.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.980+0000 7f905b05a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90540b25b0 con 0x7f90540b2d20 2023-11-28T18:37:51.978 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.980+0000 7f905b05a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90540cea30 con 0x7f905415d4f0 2023-11-28T18:37:51.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.980+0000 7f905b05a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90540b2900 con 0x7f90540d3c30 2023-11-28T18:37:51.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.980+0000 7f9053fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f90540d3c30 0x7f90540d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:51.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.980+0000 7f90595f7640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f90540b2d20 0x7f90540b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51298/0 (socket says 172.21.15.105:51298) 2023-11-28T18:37:51.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.980+0000 7f9053fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f90540d3c30 0x7f90540d1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54180/0 (socket says 172.21.15.105:54180) 2023-11-28T18:37:51.979 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.980+0000 7f90595f7640 1 -- 172.21.15.105:0/1254885735 learned_addr learned my addr 172.21.15.105:0/1254885735 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:37:51.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.980+0000 7f9053fff640 1 -- 172.21.15.105:0/1254885735 >> v1:172.21.15.148:6789/0 conn(0x7f905415d4f0 legacy=0x7f90540d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:51.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.980+0000 7f9053fff640 1 -- 172.21.15.105:0/1254885735 >> v1:172.21.15.105:6789/0 conn(0x7f90540b2d20 legacy=0x7f90540b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:51.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.980+0000 7f9053fff640 1 -- 172.21.15.105:0/1254885735 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90540ced10 con 0x7f90540d3c30 2023-11-28T18:37:51.980 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.980+0000 7f90537fe640 1 -- 172.21.15.105:0/1254885735 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4177444987 0 0) 0x7f90540b25b0 con 0x7f90540b2d20 2023-11-28T18:37:51.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.984+0000 7f9053fff640 1 --2- 172.21.15.105:0/1254885735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90540d3c30 0x7f90540d1520 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f90400097c0 tx=0x7f904001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76cab2fa1059391 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:51.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.984+0000 7f90537fe640 1 -- 172.21.15.105:0/1254885735 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90400ad660 con 0x7f90540d3c30 2023-11-28T18:37:51.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.984+0000 7f90537fe640 1 -- 172.21.15.105:0/1254885735 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90400ad7c0 con 0x7f90540d3c30 2023-11-28T18:37:51.982 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.984+0000 7f90537fe640 1 -- 172.21.15.105:0/1254885735 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90400adb50 con 0x7f90540d3c30 2023-11-28T18:37:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.984+0000 7f905b05a640 1 -- 172.21.15.105:0/1254885735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90540d3c30 msgr2=0x7f90540d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.984+0000 7f905b05a640 1 --2- 172.21.15.105:0/1254885735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90540d3c30 0x7f90540d1520 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f90400097c0 tx=0x7f904001c3c0 comp rx=0 tx=0).stop 2023-11-28T18:37:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.984+0000 7f905b05a640 1 -- 172.21.15.105:0/1254885735 shutdown_connections 2023-11-28T18:37:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.984+0000 7f905b05a640 1 --2- 172.21.15.105:0/1254885735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90540d3c30 0x7f90540d1520 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.984+0000 7f905b05a640 1 -- 172.21.15.105:0/1254885735 >> 172.21.15.105:0/1254885735 conn(0x7f90540d3030 msgr2=0x7f90540d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.984+0000 7f905b05a640 1 -- 172.21.15.105:0/1254885735 shutdown_connections 2023-11-28T18:37:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.984+0000 7f905b05a640 1 -- 172.21.15.105:0/1254885735 wait complete. 2023-11-28T18:37:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f905b05a640 1 Processor -- start 2023-11-28T18:37:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f905b05a640 1 -- start start 2023-11-28T18:37:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f905b05a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90540b2d20 0x7f90540b60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f905b05a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90540d3c30 0x7f90541538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f905b05a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f905415d4f0 0x7f905406b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f905b05a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90540d0a20 con 0x7f905415d4f0 2023-11-28T18:37:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f905b05a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90540cca20 con 0x7f90540b2d20 2023-11-28T18:37:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f905b05a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90540ccd00 con 0x7f90540d3c30 2023-11-28T18:37:51.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f90595f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f905415d4f0 0x7f905406b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:51.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f90595f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f905415d4f0 0x7f905406b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54194/0 (socket says 172.21.15.105:54194) 2023-11-28T18:37:51.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f90595f7640 1 -- 172.21.15.105:0/1204491923 learned_addr learned my addr 172.21.15.105:0/1204491923 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:51.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f9058df6640 1 --2- 172.21.15.105:0/1204491923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90540b2d20 0x7f90540b60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:51.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f9053fff640 1 --2- 172.21.15.105:0/1204491923 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90540d3c30 0x7f90541538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:51.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f90595f7640 1 -- 172.21.15.105:0/1204491923 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90540d3c30 msgr2=0x7f90541538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:51.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f90595f7640 1 --2- 172.21.15.105:0/1204491923 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90540d3c30 0x7f90541538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:51.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f90595f7640 1 -- 172.21.15.105:0/1204491923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90540b2d20 msgr2=0x7f90540b60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:51.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f90595f7640 1 --2- 172.21.15.105:0/1204491923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90540b2d20 0x7f90540b60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:51.989 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f90595f7640 1 -- 172.21.15.105:0/1204491923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9040009420 con 0x7f905415d4f0 2023-11-28T18:37:51.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f90595f7640 1 --2- 172.21.15.105:0/1204491923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f905415d4f0 0x7f905406b0e0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f904c00c780 tx=0x7f904c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:51.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f90517fa640 1 -- 172.21.15.105:0/1204491923 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c00ce90 con 0x7f905415d4f0 2023-11-28T18:37:51.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f90517fa640 1 -- 172.21.15.105:0/1204491923 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c00a640 con 0x7f905415d4f0 2023-11-28T18:37:51.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f905b05a640 1 -- 172.21.15.105:0/1204491923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f905415e710 con 0x7f905415d4f0 2023-11-28T18:37:51.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f90517fa640 1 -- 172.21.15.105:0/1204491923 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c00a9d0 con 0x7f905415d4f0 2023-11-28T18:37:51.990 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.988+0000 7f905b05a640 1 -- 172.21.15.105:0/1204491923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90540d17c0 con 0x7f905415d4f0 2023-11-28T18:37:51.991 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.992+0000 7f905b05a640 1 -- 172.21.15.105:0/1204491923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f905415e570 con 0x7f905415d4f0 2023-11-28T18:37:51.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.992+0000 7f90517fa640 1 -- 172.21.15.105:0/1204491923 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f904c00ab30 con 0x7f905415d4f0 2023-11-28T18:37:51.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.992+0000 7f90517fa640 1 --2- 172.21.15.105:0/1204491923 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f90380668d0 0x7f9038068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:51.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.992+0000 7f90517fa640 1 -- 172.21.15.105:0/1204491923 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f904c083ce0 con 0x7f905415d4f0 2023-11-28T18:37:51.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.992+0000 7f9058df6640 1 --2- 172.21.15.105:0/1204491923 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f90380668d0 0x7f9038068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:51.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.992+0000 7f9058df6640 1 --2- 172.21.15.105:0/1204491923 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f90380668d0 0x7f9038068d90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9054160b60 tx=0x7f9048009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:51.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:51.996+0000 7f90517fa640 1 -- 172.21.15.105:0/1204491923 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f905415e570 con 0x7f905415d4f0 2023-11-28T18:37:52.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.060+0000 7fc5aeb1d640 1 Processor -- start 2023-11-28T18:37:52.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.060+0000 7fc5aeb1d640 1 -- start start 2023-11-28T18:37:52.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5aeb1d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5a815d4f0 0x7fc5a80d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5aeb1d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a80b25b0 con 0x7fc5a80b2d20 2023-11-28T18:37:52.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5aeb1d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a80cea30 con 0x7fc5a80d3c30 2023-11-28T18:37:52.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5aeb1d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a80b2900 con 0x7fc5a815d4f0 2023-11-28T18:37:52.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5ad0ba640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc5a80b2d20 0x7fc5a80b31d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51312/0 (socket says 172.21.15.105:51312) 2023-11-28T18:37:52.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5ad0ba640 1 -- 172.21.15.105:0/681956191 learned_addr learned my addr 172.21.15.105:0/681956191 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:37:52.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5ac8b9640 1 --2- 172.21.15.105:0/681956191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5a815d4f0 0x7fc5a80d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a77fe640 1 -- 172.21.15.105:0/681956191 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 286450982 0 0) 0x7fc5a80b25b0 con 0x7fc5a80b2d20 2023-11-28T18:37:52.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a77fe640 1 -- 172.21.15.105:0/681956191 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc598003530 con 0x7fc5a80b2d20 2023-11-28T18:37:52.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a77fe640 1 -- 172.21.15.105:0/681956191 <== mon.1 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3335127221 0 0) 0x7fc5a80cea30 con 0x7fc5a80d3c30 2023-11-28T18:37:52.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a77fe640 1 -- 172.21.15.105:0/681956191 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5a80b25b0 con 0x7fc5a80d3c30 2023-11-28T18:37:52.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a77fe640 1 -- 172.21.15.105:0/681956191 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 990534221 0 0) 0x7fc598003530 con 0x7fc5a80b2d20 2023-11-28T18:37:52.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a77fe640 1 -- 172.21.15.105:0/681956191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5a815d4f0 msgr2=0x7fc5a80d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:52.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a77fe640 1 --2- 172.21.15.105:0/681956191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5a815d4f0 0x7fc5a80d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a77fe640 1 -- 172.21.15.105:0/681956191 >> v1:172.21.15.148:6789/0 conn(0x7fc5a80d3c30 legacy=0x7fc5a80d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a77fe640 1 -- 172.21.15.105:0/681956191 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a80ced10 con 0x7fc5a80b2d20 2023-11-28T18:37:52.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a77fe640 1 -- 172.21.15.105:0/681956191 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7fc5a0001a00 con 0x7fc5a80b2d20 2023-11-28T18:37:52.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5ac8b9640 1 --2- 172.21.15.105:0/681956191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5a815d4f0 0x7fc5a80d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:37:52.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a77fe640 1 -- 172.21.15.105:0/681956191 >> v1:172.21.15.105:6789/0 conn(0x7fc5a80b2d20 legacy=0x7fc5a80b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a77fe640 1 --2- 172.21.15.105:0/681956191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc598003b40 0x7fc598003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a77fe640 1 --2- 172.21.15.105:0/681956191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5980047d0 0x7fc598004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a77fe640 1 --2- 172.21.15.105:0/681956191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5a815d4f0 0x7fc598005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a77fe640 1 -- 172.21.15.105:0/681956191 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a80b2900 con 0x7fc5980047d0 2023-11-28T18:37:52.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a77fe640 1 -- 172.21.15.105:0/681956191 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59800ba70 con 0x7fc598003b40 2023-11-28T18:37:52.068 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a77fe640 1 -- 172.21.15.105:0/681956191 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59800bc30 con 0x7fc5a815d4f0 2023-11-28T18:37:52.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a7fff640 1 --2- 172.21.15.105:0/681956191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5980047d0 0x7fc598004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5ac8b9640 1 --2- 172.21.15.105:0/681956191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc598003b40 0x7fc598003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.069 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5ad0ba640 1 --2- 172.21.15.105:0/681956191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5a815d4f0 0x7fc598005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a7fff640 1 -- 172.21.15.105:0/681956191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5a815d4f0 msgr2=0x7fc598005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:52.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a7fff640 1 --2- 172.21.15.105:0/681956191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5a815d4f0 0x7fc598005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a7fff640 1 -- 172.21.15.105:0/681956191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc598003b40 msgr2=0x7fc598003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:52.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a7fff640 1 --2- 172.21.15.105:0/681956191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc598003b40 0x7fc598003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5a7fff640 1 -- 172.21.15.105:0/681956191 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59c009420 con 0x7fc5980047d0 2023-11-28T18:37:52.071 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.064+0000 7fc5ac8b9640 1 --2- 172.21.15.105:0/681956191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc598003b40 0x7fc598003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:37:52.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.068+0000 7fc5a7fff640 1 --2- 172.21.15.105:0/681956191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5980047d0 0x7fc598004c50 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fc594007fd0 tx=0x7fc59401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f4c74cc8c073e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:52.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.068+0000 7fc5a77fe640 1 -- 172.21.15.105:0/681956191 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5940052d0 con 0x7fc5980047d0 2023-11-28T18:37:52.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.068+0000 7fc5a77fe640 1 -- 172.21.15.105:0/681956191 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5940a7050 con 0x7fc5980047d0 2023-11-28T18:37:52.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.068+0000 7fc5a77fe640 1 -- 172.21.15.105:0/681956191 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc594003600 con 0x7fc5980047d0 2023-11-28T18:37:52.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.068+0000 7fc5aeb1d640 1 -- 172.21.15.105:0/681956191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5980047d0 msgr2=0x7fc598004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.068+0000 7fc5aeb1d640 1 --2- 172.21.15.105:0/681956191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5980047d0 0x7fc598004c50 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fc594007fd0 tx=0x7fc59401c4f0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.068+0000 7fc5ad0ba640 1 -- 172.21.15.105:0/681956191 reap_dead start 2023-11-28T18:37:52.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.068+0000 7fc5aeb1d640 1 -- 172.21.15.105:0/681956191 shutdown_connections 2023-11-28T18:37:52.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.068+0000 7fc5aeb1d640 1 -- 172.21.15.105:0/681956191 >> 172.21.15.105:0/681956191 conn(0x7fc5a80d3030 msgr2=0x7fc5a80d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:52.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.068+0000 7fc5aeb1d640 1 -- 172.21.15.105:0/681956191 shutdown_connections 2023-11-28T18:37:52.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.068+0000 7fc5aeb1d640 1 -- 172.21.15.105:0/681956191 wait complete. 2023-11-28T18:37:52.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.068+0000 7fc5aeb1d640 1 Processor -- start 2023-11-28T18:37:52.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5aeb1d640 1 -- start start 2023-11-28T18:37:52.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5aeb1d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5980047d0 0x7fc5a8155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5aeb1d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5a80b8750 0x7fc5a80b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5aeb1d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5a8156360 0x7fc5a80b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5aeb1d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a80d0a20 con 0x7fc5a8156360 2023-11-28T18:37:52.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5aeb1d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a80b8be0 con 0x7fc5a80b8750 2023-11-28T18:37:52.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5aeb1d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a80b66c0 con 0x7fc5980047d0 2023-11-28T18:37:52.082 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5ad0ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5a8156360 0x7fc5a80b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5ad0ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5a8156360 0x7fc5a80b7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54216/0 (socket says 172.21.15.105:54216) 2023-11-28T18:37:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5ad0ba640 1 -- 172.21.15.105:0/3533525452 learned_addr learned my addr 172.21.15.105:0/3533525452 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5ac8b9640 1 --2- 172.21.15.105:0/3533525452 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5980047d0 0x7fc5a8155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5a7fff640 1 --2- 172.21.15.105:0/3533525452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5a80b8750 0x7fc5a80b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5ad0ba640 1 -- 172.21.15.105:0/3533525452 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5980047d0 msgr2=0x7fc5a8155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5ad0ba640 1 --2- 172.21.15.105:0/3533525452 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5980047d0 0x7fc5a8155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5ad0ba640 1 -- 172.21.15.105:0/3533525452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5a80b8750 msgr2=0x7fc5a80b6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5ad0ba640 1 --2- 172.21.15.105:0/3533525452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5a80b8750 0x7fc5a80b6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5ad0ba640 1 -- 172.21.15.105:0/3533525452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a8155b90 con 0x7fc5a8156360 2023-11-28T18:37:52.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5ad0ba640 1 --2- 172.21.15.105:0/3533525452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5a8156360 0x7fc5a80b7090 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80cf440 tx=0x7fc5a0001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:52.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5a57fa640 1 -- 172.21.15.105:0/3533525452 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a000eb00 con 0x7fc5a8156360 2023-11-28T18:37:52.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5a57fa640 1 -- 172.21.15.105:0/3533525452 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a000eca0 con 0x7fc5a8156360 2023-11-28T18:37:52.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5a57fa640 1 -- 172.21.15.105:0/3533525452 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0014430 con 0x7fc5a8156360 2023-11-28T18:37:52.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5aeb1d640 1 -- 172.21.15.105:0/3533525452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a80b5dd0 con 0x7fc5a8156360 2023-11-28T18:37:52.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5aeb1d640 1 -- 172.21.15.105:0/3533525452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a80b6350 con 0x7fc5a8156360 2023-11-28T18:37:52.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.072+0000 7fc5aeb1d640 1 -- 172.21.15.105:0/3533525452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a80b6830 con 0x7fc5a8156360 2023-11-28T18:37:52.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.076+0000 7fc5a57fa640 1 -- 172.21.15.105:0/3533525452 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc5a0014590 con 0x7fc5a8156360 2023-11-28T18:37:52.086 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.076+0000 7fc5a57fa640 1 --2- 172.21.15.105:0/3533525452 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc58c066900 0x7fc58c068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.076+0000 7fc5a57fa640 1 -- 172.21.15.105:0/3533525452 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fc5a00840f0 con 0x7fc5a8156360 2023-11-28T18:37:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.076+0000 7fc5ac8b9640 1 --2- 172.21.15.105:0/3533525452 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc58c066900 0x7fc58c068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.076+0000 7fc5ac8b9640 1 --2- 172.21.15.105:0/3533525452 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc58c066900 0x7fc58c068dc0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009c80 tx=0x7fc59c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.080+0000 7fc5a57fa640 1 -- 172.21.15.105:0/3533525452 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc5a00511a0 con 0x7fc5a8156360 2023-11-28T18:37:52.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.164+0000 7f905b05a640 1 -- 172.21.15.105:0/1204491923 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f905415e950 con 0x7f905415d4f0 2023-11-28T18:37:52.162 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.164+0000 7f90517fa640 1 -- 172.21.15.105:0/1204491923 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+16923 (secure 0 0 0) 0x7f905415e950 con 0x7f905415d4f0 2023-11-28T18:37:52.162 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:37:52.163 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":130,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:32:58.181121+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:31:56.971692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-28T18:31:57.881657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6835/3706038483":"2023-11-29T18:32:58.181068+0000","172.21.15.148:6833/3706038483":"2023-11-29T18:32:58.181068+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:37:52.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.168+0000 7f905b05a640 1 -- 172.21.15.105:0/1204491923 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f90380668d0 msgr2=0x7f9038068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.168+0000 7f905b05a640 1 --2- 172.21.15.105:0/1204491923 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f90380668d0 0x7f9038068d90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9054160b60 tx=0x7f9048009040 comp rx=0 tx=0).stop 2023-11-28T18:37:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.168+0000 7f905b05a640 1 -- 172.21.15.105:0/1204491923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f905415d4f0 msgr2=0x7f905406b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.168+0000 7f905b05a640 1 --2- 172.21.15.105:0/1204491923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f905415d4f0 0x7f905406b0e0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f904c00c780 tx=0x7f904c00cc40 comp rx=0 tx=0).stop 2023-11-28T18:37:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.168+0000 7f905b05a640 1 -- 172.21.15.105:0/1204491923 shutdown_connections 2023-11-28T18:37:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.168+0000 7f905b05a640 1 --2- 172.21.15.105:0/1204491923 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f90380668d0 0x7f9038068d90 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.168+0000 7f905b05a640 1 --2- 172.21.15.105:0/1204491923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f905415d4f0 0x7f905406b0e0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.168+0000 7f905b05a640 1 --2- 172.21.15.105:0/1204491923 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90540d3c30 0x7f90541538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.168+0000 7f905b05a640 1 --2- 172.21.15.105:0/1204491923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90540b2d20 0x7f90540b60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.168+0000 7f905b05a640 1 -- 172.21.15.105:0/1204491923 >> 172.21.15.105:0/1204491923 conn(0x7f90540d3030 msgr2=0x7f905406c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:52.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.168+0000 7f905b05a640 1 -- 172.21.15.105:0/1204491923 shutdown_connections 2023-11-28T18:37:52.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.168+0000 7f905b05a640 1 -- 172.21.15.105:0/1204491923 wait complete. 2023-11-28T18:37:52.185 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:31:56.971692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:31:57.881657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:37:52.186 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:37:52.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.244+0000 7fc5aeb1d640 1 -- 172.21.15.105:0/3533525452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc5a8067570 con 0x7fc5a8156360 2023-11-28T18:37:52.241 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.244+0000 7fc5a57fa640 1 -- 172.21.15.105:0/3533525452 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc5a0001920 con 0x7fc5a8156360 2023-11-28T18:37:52.242 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:37:52.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.244+0000 7fc5aeb1d640 1 -- 172.21.15.105:0/3533525452 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc58c066900 msgr2=0x7fc58c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.244+0000 7fc5aeb1d640 1 --2- 172.21.15.105:0/3533525452 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc58c066900 0x7fc58c068dc0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009c80 tx=0x7fc59c009190 comp rx=0 tx=0).stop 2023-11-28T18:37:52.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.244+0000 7fc5aeb1d640 1 -- 172.21.15.105:0/3533525452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5a8156360 msgr2=0x7fc5a80b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.244+0000 7fc5aeb1d640 1 --2- 172.21.15.105:0/3533525452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5a8156360 0x7fc5a80b7090 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80cf440 tx=0x7fc5a0001ac0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.244+0000 7fc5aeb1d640 1 -- 172.21.15.105:0/3533525452 shutdown_connections 2023-11-28T18:37:52.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.244+0000 7fc5aeb1d640 1 --2- 172.21.15.105:0/3533525452 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc58c066900 0x7fc58c068dc0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.244+0000 7fc5aeb1d640 1 --2- 172.21.15.105:0/3533525452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5a8156360 0x7fc5a80b7090 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.244+0000 7fc5aeb1d640 1 --2- 172.21.15.105:0/3533525452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5a80b8750 0x7fc5a80b6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.244+0000 7fc5aeb1d640 1 --2- 172.21.15.105:0/3533525452 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc5980047d0 0x7fc5a8155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.244+0000 7fc5aeb1d640 1 -- 172.21.15.105:0/3533525452 >> 172.21.15.105:0/3533525452 conn(0x7fc5a80d3030 msgr2=0x7fc5a80b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:52.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.244+0000 7fc5aeb1d640 1 -- 172.21.15.105:0/3533525452 shutdown_connections 2023-11-28T18:37:52.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.244+0000 7fc5aeb1d640 1 -- 172.21.15.105:0/3533525452 wait complete. 2023-11-28T18:37:52.264 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:37:52.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.332+0000 7f9a134dc640 1 Processor -- start 2023-11-28T18:37:52.331 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.332+0000 7f9a134dc640 1 -- start start 2023-11-28T18:37:52.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.336+0000 7f9a134dc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9a0c0d1b00 0x7f9a0c168090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.336+0000 7f9a134dc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0c0cf440 con 0x7f9a0c0d0d40 2023-11-28T18:37:52.332 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.336+0000 7f9a134dc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0c0cffc0 con 0x7f9a0c15d1a0 2023-11-28T18:37:52.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.336+0000 7f9a134dc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0c0ceb40 con 0x7f9a0c0d1b00 2023-11-28T18:37:52.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.336+0000 7f9a11cd9640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9a0c0d0d40 0x7f9a0c0d13f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51324/0 (socket says 172.21.15.105:51324) 2023-11-28T18:37:52.333 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.336+0000 7f9a11cd9640 1 -- 172.21.15.105:0/1094482079 learned_addr learned my addr 172.21.15.105:0/1094482079 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:37:52.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.336+0000 7f9a12cdb640 1 --2- 172.21.15.105:0/1094482079 >> v2:172.21.15.105:3300/0 conn(0x7f9a0c0d1b00 0x7f9a0c168090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.336+0000 7f9a114d8640 1 -- 172.21.15.105:0/1094482079 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1170190375 0 0) 0x7f9a0c0cf440 con 0x7f9a0c0d0d40 2023-11-28T18:37:52.334 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.336+0000 7f9a114d8640 1 -- 172.21.15.105:0/1094482079 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99f8003530 con 0x7f9a0c0d0d40 2023-11-28T18:37:52.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.336+0000 7f9a12cdb640 1 -- 172.21.15.105:0/1094482079 >> v1:172.21.15.148:6789/0 conn(0x7f9a0c15d1a0 legacy=0x7f9a0c0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.336+0000 7f9a12cdb640 1 -- 172.21.15.105:0/1094482079 >> v1:172.21.15.105:6789/0 conn(0x7f9a0c0d0d40 legacy=0x7f9a0c0d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.336+0000 7f9a12cdb640 1 -- 172.21.15.105:0/1094482079 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c0cf690 con 0x7f9a0c0d1b00 2023-11-28T18:37:52.335 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.340+0000 7f9a12cdb640 1 --2- 172.21.15.105:0/1094482079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c0d1b00 0x7f9a0c168090 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009d90 tx=0x7f9a0401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395a4b15488672d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:52.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.340+0000 7f9a114d8640 1 -- 172.21.15.105:0/1094482079 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a040a5950 con 0x7f9a0c0d1b00 2023-11-28T18:37:52.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.340+0000 7f9a114d8640 1 -- 172.21.15.105:0/1094482079 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a040a5ab0 con 0x7f9a0c0d1b00 2023-11-28T18:37:52.336 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.340+0000 7f9a114d8640 1 -- 172.21.15.105:0/1094482079 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a040a5e40 con 0x7f9a0c0d1b00 2023-11-28T18:37:52.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.340+0000 7f9a134dc640 1 -- 172.21.15.105:0/1094482079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c0d1b00 msgr2=0x7f9a0c168090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.340+0000 7f9a134dc640 1 --2- 172.21.15.105:0/1094482079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c0d1b00 0x7f9a0c168090 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009d90 tx=0x7f9a0401c890 comp rx=0 tx=0).stop 2023-11-28T18:37:52.337 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.340+0000 7f9a134dc640 1 -- 172.21.15.105:0/1094482079 shutdown_connections 2023-11-28T18:37:52.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.340+0000 7f9a134dc640 1 --2- 172.21.15.105:0/1094482079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c0d1b00 0x7f9a0c168090 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.340+0000 7f9a134dc640 1 -- 172.21.15.105:0/1094482079 >> 172.21.15.105:0/1094482079 conn(0x7f9a0c0d2e10 msgr2=0x7f9a0c0d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:52.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.340+0000 7f9a134dc640 1 -- 172.21.15.105:0/1094482079 shutdown_connections 2023-11-28T18:37:52.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.340+0000 7f9a134dc640 1 -- 172.21.15.105:0/1094482079 wait complete. 2023-11-28T18:37:52.338 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.340+0000 7f9a134dc640 1 Processor -- start 2023-11-28T18:37:52.339 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.340+0000 7f9a134dc640 1 -- start start 2023-11-28T18:37:52.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.340+0000 7f9a134dc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c0d0d40 0x7f9a0c0c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.340+0000 7f9a134dc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a0c0d1b00 0x7f9a0c0b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.340 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.340+0000 7f9a134dc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a0c15d1a0 0x7f9a0c1f9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.340+0000 7f9a134dc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0b8a80 con 0x7f9a0c0d0d40 2023-11-28T18:37:52.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.340+0000 7f9a134dc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0cb480 con 0x7f9a0c15d1a0 2023-11-28T18:37:52.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.340+0000 7f9a134dc640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0cb760 con 0x7f9a0c0d1b00 2023-11-28T18:37:52.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.344+0000 7f9a124da640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c0d0d40 0x7f9a0c0c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.344+0000 7f9a124da640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c0d0d40 0x7f9a0c0c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54238/0 (socket says 172.21.15.105:54238) 2023-11-28T18:37:52.341 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.344+0000 7f9a124da640 1 -- 172.21.15.105:0/3019046065 learned_addr learned my addr 172.21.15.105:0/3019046065 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:52.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.344+0000 7f9a12cdb640 1 --2- 172.21.15.105:0/3019046065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a0c15d1a0 0x7f9a0c1f9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.344+0000 7f9a11cd9640 1 --2- 172.21.15.105:0/3019046065 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a0c0d1b00 0x7f9a0c0b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.344+0000 7f9a124da640 1 -- 172.21.15.105:0/3019046065 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a0c0d1b00 msgr2=0x7f9a0c0b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.344+0000 7f9a124da640 1 --2- 172.21.15.105:0/3019046065 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a0c0d1b00 0x7f9a0c0b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.344+0000 7f9a124da640 1 -- 172.21.15.105:0/3019046065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a0c15d1a0 msgr2=0x7f9a0c1f9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.342 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.344+0000 7f9a124da640 1 --2- 172.21.15.105:0/3019046065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a0c15d1a0 0x7f9a0c1f9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.344+0000 7f9a124da640 1 -- 172.21.15.105:0/3019046065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a04009a70 con 0x7f9a0c0d0d40 2023-11-28T18:37:52.343 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.344+0000 7f9a124da640 1 --2- 172.21.15.105:0/3019046065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c0d0d40 0x7f9a0c0c9ee0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800c3c0 tx=0x7f9a0800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:52.344 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.344+0000 7f9a02ffd640 1 -- 172.21.15.105:0/3019046065 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a080052d0 con 0x7f9a0c0d0d40 2023-11-28T18:37:52.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.344+0000 7f9a02ffd640 1 -- 172.21.15.105:0/3019046065 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a08005430 con 0x7f9a0c0d0d40 2023-11-28T18:37:52.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.344+0000 7f9a134dc640 1 -- 172.21.15.105:0/3019046065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0c0cba40 con 0x7f9a0c0d0d40 2023-11-28T18:37:52.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.344+0000 7f9a02ffd640 1 -- 172.21.15.105:0/3019046065 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a08003840 con 0x7f9a0c0d0d40 2023-11-28T18:37:52.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.344+0000 7f9a134dc640 1 -- 172.21.15.105:0/3019046065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0c0cbd20 con 0x7f9a0c0d0d40 2023-11-28T18:37:52.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.344+0000 7f9a134dc640 1 -- 172.21.15.105:0/3019046065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0c074580 con 0x7f9a0c0d0d40 2023-11-28T18:37:52.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.344+0000 7f9a02ffd640 1 -- 172.21.15.105:0/3019046065 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f9a080039a0 con 0x7f9a0c0d0d40 2023-11-28T18:37:52.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.348+0000 7f9a02ffd640 1 --2- 172.21.15.105:0/3019046065 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99f00668d0 0x7f99f0068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.345 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.348+0000 7f9a02ffd640 1 -- 172.21.15.105:0/3019046065 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f9a08083a60 con 0x7f9a0c0d0d40 2023-11-28T18:37:52.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.348+0000 7f9a11cd9640 1 --2- 172.21.15.105:0/3019046065 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99f00668d0 0x7f99f0068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.346 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.348+0000 7f9a11cd9640 1 --2- 172.21.15.105:0/3019046065 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99f00668d0 0x7f99f0068d90 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0b8c70 tx=0x7f99fc007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:52.348 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.352+0000 7f9a02ffd640 1 -- 172.21.15.105:0/3019046065 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9a08050ba0 con 0x7f9a0c0d0d40 2023-11-28T18:37:52.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.416+0000 7f83f12ea640 1 Processor -- start 2023-11-28T18:37:52.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.416+0000 7f83f12ea640 1 -- start start 2023-11-28T18:37:52.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.416+0000 7f83f12ea640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f83ec0b4290 0x7f83ec0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.416+0000 7f83f12ea640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f83ec15b030 0x7f83ec0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.413 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.416+0000 7f83f12ea640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83ec0b4bb0 con 0x7f83ec071960 2023-11-28T18:37:52.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.416+0000 7f83f12ea640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ec0701d0 con 0x7f83ec15b030 2023-11-28T18:37:52.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.416+0000 7f83f12ea640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ec06df80 con 0x7f83ec0b4290 2023-11-28T18:37:52.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.416+0000 7f83eb7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f83ec15b030 0x7f83ec0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.416+0000 7f83eb7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f83ec15b030 0x7f83ec0709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54246/0 (socket says 172.21.15.105:54246) 2023-11-28T18:37:52.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.416+0000 7f83eb7fe640 1 -- 172.21.15.105:0/4102524583 learned_addr learned my addr 172.21.15.105:0/4102524583 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:52.414 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.416+0000 7f83eaffd640 1 --2- 172.21.15.105:0/4102524583 >> v2:172.21.15.148:3300/0 conn(0x7f83ec0b4290 0x7f83ec0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.416+0000 7f83eb7fe640 1 -- 172.21.15.105:0/4102524583 >> v2:172.21.15.148:3300/0 conn(0x7f83ec0b4290 msgr2=0x7f83ec0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:52.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.416+0000 7f83eb7fe640 1 --2- 172.21.15.105:0/4102524583 >> v2:172.21.15.148:3300/0 conn(0x7f83ec0b4290 0x7f83ec0b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.416+0000 7f83eb7fe640 1 -- 172.21.15.105:0/4102524583 >> v1:172.21.15.148:6789/0 conn(0x7f83ec071960 legacy=0x7f83ec15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.416+0000 7f83eb7fe640 1 -- 172.21.15.105:0/4102524583 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec071df0 con 0x7f83ec15b030 2023-11-28T18:37:52.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.420+0000 7f83eb7fe640 1 --2- 172.21.15.105:0/4102524583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83ec15b030 0x7f83ec0709b0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f83ec073a70 tx=0x7f83e401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc7b00f20d45659 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:52.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.420+0000 7f83e9ffb640 1 -- 172.21.15.105:0/4102524583 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e40a5770 con 0x7f83ec15b030 2023-11-28T18:37:52.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.420+0000 7f83e9ffb640 1 -- 172.21.15.105:0/4102524583 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e40a58d0 con 0x7f83ec15b030 2023-11-28T18:37:52.417 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.420+0000 7f83e9ffb640 1 -- 172.21.15.105:0/4102524583 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e40a5c60 con 0x7f83ec15b030 2023-11-28T18:37:52.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.420+0000 7f83f12ea640 1 -- 172.21.15.105:0/4102524583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83ec15b030 msgr2=0x7f83ec0709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.418 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.420+0000 7f83f12ea640 1 --2- 172.21.15.105:0/4102524583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83ec15b030 0x7f83ec0709b0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f83ec073a70 tx=0x7f83e401c9a0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.420+0000 7f83f12ea640 1 -- 172.21.15.105:0/4102524583 shutdown_connections 2023-11-28T18:37:52.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.420+0000 7f83f12ea640 1 --2- 172.21.15.105:0/4102524583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83ec15b030 0x7f83ec0709b0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.420+0000 7f83f12ea640 1 --2- 172.21.15.105:0/4102524583 >> v2:172.21.15.148:3300/0 conn(0x7f83ec0b4290 0x7f83ec0b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.420+0000 7f83f12ea640 1 -- 172.21.15.105:0/4102524583 >> 172.21.15.105:0/4102524583 conn(0x7f83ec0ceb90 msgr2=0x7f83ec0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:52.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.420+0000 7f83f12ea640 1 -- 172.21.15.105:0/4102524583 shutdown_connections 2023-11-28T18:37:52.419 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.420+0000 7f83f12ea640 1 -- 172.21.15.105:0/4102524583 wait complete. 2023-11-28T18:37:52.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.420+0000 7f83f12ea640 1 Processor -- start 2023-11-28T18:37:52.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83f12ea640 1 -- start start 2023-11-28T18:37:52.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83f12ea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83ec0b4290 0x7f83ec152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83f12ea640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83ec15b030 0x7f83ec14f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83f12ea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83ec150f90 0x7f83ec1403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83f12ea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec151440 con 0x7f83ec150f90 2023-11-28T18:37:52.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83f12ea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec14fef0 con 0x7f83ec0b4290 2023-11-28T18:37:52.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83f12ea640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec1501d0 con 0x7f83ec15b030 2023-11-28T18:37:52.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83eb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83ec150f90 0x7f83ec1403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83eb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83ec150f90 0x7f83ec1403f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54252/0 (socket says 172.21.15.105:54252) 2023-11-28T18:37:52.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83eb7fe640 1 -- 172.21.15.105:0/2451079440 learned_addr learned my addr 172.21.15.105:0/2451079440 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:52.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83eaffd640 1 --2- 172.21.15.105:0/2451079440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83ec0b4290 0x7f83ec152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83ea7fc640 1 --2- 172.21.15.105:0/2451079440 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83ec15b030 0x7f83ec14f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83eb7fe640 1 -- 172.21.15.105:0/2451079440 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83ec15b030 msgr2=0x7f83ec14f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83eb7fe640 1 --2- 172.21.15.105:0/2451079440 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83ec15b030 0x7f83ec14f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83eb7fe640 1 -- 172.21.15.105:0/2451079440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83ec0b4290 msgr2=0x7f83ec152820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83eb7fe640 1 --2- 172.21.15.105:0/2451079440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83ec0b4290 0x7f83ec152820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83eb7fe640 1 -- 172.21.15.105:0/2451079440 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e4009b30 con 0x7f83ec150f90 2023-11-28T18:37:52.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83eb7fe640 1 --2- 172.21.15.105:0/2451079440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83ec150f90 0x7f83ec1403f0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f83e40af040 tx=0x7f83e4019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:52.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83d37fe640 1 -- 172.21.15.105:0/2451079440 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e40b8670 con 0x7f83ec150f90 2023-11-28T18:37:52.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83d37fe640 1 -- 172.21.15.105:0/2451079440 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e40b87f0 con 0x7f83ec150f90 2023-11-28T18:37:52.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83d37fe640 1 -- 172.21.15.105:0/2451079440 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e40b8ba0 con 0x7f83ec150f90 2023-11-28T18:37:52.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83f12ea640 1 -- 172.21.15.105:0/2451079440 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ec1504b0 con 0x7f83ec150f90 2023-11-28T18:37:52.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83f12ea640 1 -- 172.21.15.105:0/2451079440 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83ec150a70 con 0x7f83ec150f90 2023-11-28T18:37:52.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.424+0000 7f83f12ea640 1 -- 172.21.15.105:0/2451079440 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83ec067570 con 0x7f83ec150f90 2023-11-28T18:37:52.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.428+0000 7f83d37fe640 1 -- 172.21.15.105:0/2451079440 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f83e40b8d00 con 0x7f83ec150f90 2023-11-28T18:37:52.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.428+0000 7f83d37fe640 1 --2- 172.21.15.105:0/2451079440 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f83d80668d0 0x7f83d8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.428+0000 7f83d37fe640 1 -- 172.21.15.105:0/2451079440 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f83e411f9d0 con 0x7f83ec150f90 2023-11-28T18:37:52.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.428+0000 7f83eaffd640 1 --2- 172.21.15.105:0/2451079440 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f83d80668d0 0x7f83d8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.428+0000 7f83eaffd640 1 --2- 172.21.15.105:0/2451079440 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f83d80668d0 0x7f83d8068d90 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009b60 tx=0x7f83dc009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:52.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.436+0000 7f83d37fe640 1 -- 172.21.15.105:0/2451079440 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f83e40eca60 con 0x7f83ec150f90 2023-11-28T18:37:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.512+0000 7f9a134dc640 1 -- 172.21.15.105:0/3019046065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9a0c1f90b0 con 0x7f9a0c0d0d40 2023-11-28T18:37:52.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.512+0000 7f9a02ffd640 1 -- 172.21.15.105:0/3019046065 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a080170a0 con 0x7f9a0c0d0d40 2023-11-28T18:37:52.511 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:37:52.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.516+0000 7f9a134dc640 1 -- 172.21.15.105:0/3019046065 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99f00668d0 msgr2=0x7f99f0068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.516+0000 7f9a134dc640 1 --2- 172.21.15.105:0/3019046065 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99f00668d0 0x7f99f0068d90 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0b8c70 tx=0x7f99fc007d40 comp rx=0 tx=0).stop 2023-11-28T18:37:52.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.516+0000 7f9a134dc640 1 -- 172.21.15.105:0/3019046065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c0d0d40 msgr2=0x7f9a0c0c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.516+0000 7f9a134dc640 1 --2- 172.21.15.105:0/3019046065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c0d0d40 0x7f9a0c0c9ee0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800c3c0 tx=0x7f9a0800c880 comp rx=0 tx=0).stop 2023-11-28T18:37:52.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.516+0000 7f9a134dc640 1 -- 172.21.15.105:0/3019046065 shutdown_connections 2023-11-28T18:37:52.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.516+0000 7f9a134dc640 1 --2- 172.21.15.105:0/3019046065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a0c15d1a0 0x7f9a0c1f9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.516+0000 7f9a134dc640 1 --2- 172.21.15.105:0/3019046065 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a0c0d1b00 0x7f9a0c0b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.516+0000 7f9a134dc640 1 --2- 172.21.15.105:0/3019046065 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f99f00668d0 0x7f99f0068d90 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.516+0000 7f9a134dc640 1 --2- 172.21.15.105:0/3019046065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c0d0d40 0x7f9a0c0c9ee0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.516+0000 7f9a134dc640 1 -- 172.21.15.105:0/3019046065 >> 172.21.15.105:0/3019046065 conn(0x7f9a0c0d2e10 msgr2=0x7f9a0c0b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:52.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.516+0000 7f9a134dc640 1 -- 172.21.15.105:0/3019046065 shutdown_connections 2023-11-28T18:37:52.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.516+0000 7f9a134dc640 1 -- 172.21.15.105:0/3019046065 wait complete. 2023-11-28T18:37:52.532 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:37:52.586 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.588+0000 7f83f12ea640 1 -- 172.21.15.105:0/2451079440 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f83ec158810 con 0x7f83ec150f90 2023-11-28T18:37:52.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.588+0000 7f83d37fe640 1 -- 172.21.15.105:0/2451079440 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v130) v1 ==== 104+0+11 (secure 0 0 0) 0x7f83e40bc020 con 0x7f83ec150f90 2023-11-28T18:37:52.590 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:37:52.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.592+0000 7f83f12ea640 1 -- 172.21.15.105:0/2451079440 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f83d80668d0 msgr2=0x7f83d8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.592+0000 7f83f12ea640 1 --2- 172.21.15.105:0/2451079440 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f83d80668d0 0x7f83d8068d90 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009b60 tx=0x7f83dc009190 comp rx=0 tx=0).stop 2023-11-28T18:37:52.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.592+0000 7f83f12ea640 1 -- 172.21.15.105:0/2451079440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83ec150f90 msgr2=0x7f83ec1403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.592+0000 7f83f12ea640 1 --2- 172.21.15.105:0/2451079440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83ec150f90 0x7f83ec1403f0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f83e40af040 tx=0x7f83e4019740 comp rx=0 tx=0).stop 2023-11-28T18:37:52.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.592+0000 7f83f12ea640 1 -- 172.21.15.105:0/2451079440 shutdown_connections 2023-11-28T18:37:52.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.592+0000 7f83f12ea640 1 --2- 172.21.15.105:0/2451079440 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f83d80668d0 0x7f83d8068d90 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.592+0000 7f83f12ea640 1 --2- 172.21.15.105:0/2451079440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83ec150f90 0x7f83ec1403f0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.591 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.592+0000 7f83f12ea640 1 --2- 172.21.15.105:0/2451079440 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83ec15b030 0x7f83ec14f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.592+0000 7f83f12ea640 1 --2- 172.21.15.105:0/2451079440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83ec0b4290 0x7f83ec152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.592+0000 7f83f12ea640 1 -- 172.21.15.105:0/2451079440 >> 172.21.15.105:0/2451079440 conn(0x7f83ec0ceb90 msgr2=0x7f83ec159b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:52.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.592+0000 7f83f12ea640 1 -- 172.21.15.105:0/2451079440 shutdown_connections 2023-11-28T18:37:52.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.592+0000 7f83f12ea640 1 -- 172.21.15.105:0/2451079440 wait complete. 2023-11-28T18:37:52.608 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:37:52.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.680+0000 7fc4227d0640 1 Processor -- start 2023-11-28T18:37:52.679 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.680+0000 7fc4227d0640 1 -- start start 2023-11-28T18:37:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.684+0000 7fc4227d0640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc41c15d000 0x7fc41c0d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.684+0000 7fc4227d0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc41c0b2a80 0x7fc41c0b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.684+0000 7fc4227d0640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc41c0ce4c0 con 0x7fc41c0cf6d0 2023-11-28T18:37:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.684+0000 7fc4227d0640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc41c0cedf0 con 0x7fc41c15d000 2023-11-28T18:37:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.684+0000 7fc4227d0640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc41c0ce7a0 con 0x7fc41c0b2a80 2023-11-28T18:37:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.684+0000 7fc4217ce640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc41c15d000 0x7fc41c0d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.684+0000 7fc420fcd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc41c0cf6d0 0x7fc41c0cfb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51334/0 (socket says 172.21.15.105:51334) 2023-11-28T18:37:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.684+0000 7fc420fcd640 1 -- 172.21.15.105:0/3334339943 learned_addr learned my addr 172.21.15.105:0/3334339943 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:37:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.684+0000 7fc421fcf640 1 --2- 172.21.15.105:0/3334339943 >> v2:172.21.15.148:3300/0 conn(0x7fc41c0b2a80 0x7fc41c0b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.684+0000 7fc4217ce640 1 -- 172.21.15.105:0/3334339943 >> v2:172.21.15.148:3300/0 conn(0x7fc41c0b2a80 msgr2=0x7fc41c0b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.684+0000 7fc4217ce640 1 --2- 172.21.15.105:0/3334339943 >> v2:172.21.15.148:3300/0 conn(0x7fc41c0b2a80 0x7fc41c0b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.684+0000 7fc4217ce640 1 -- 172.21.15.105:0/3334339943 >> v1:172.21.15.105:6789/0 conn(0x7fc41c0cf6d0 legacy=0x7fc41c0cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.684+0000 7fc4217ce640 1 -- 172.21.15.105:0/3334339943 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41c0cea80 con 0x7fc41c15d000 2023-11-28T18:37:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.688+0000 7fc4217ce640 1 --2- 172.21.15.105:0/3334339943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc41c15d000 0x7fc41c0d3920 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0b2050 tx=0x7fc41801c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ecab5ff21e0f14 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.688+0000 7fc413fff640 1 -- 172.21.15.105:0/3334339943 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4180ab600 con 0x7fc41c15d000 2023-11-28T18:37:52.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.688+0000 7fc413fff640 1 -- 172.21.15.105:0/3334339943 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4180a7050 con 0x7fc41c15d000 2023-11-28T18:37:52.685 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.688+0000 7fc413fff640 1 -- 172.21.15.105:0/3334339943 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4180ab980 con 0x7fc41c15d000 2023-11-28T18:37:52.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.688+0000 7fc4227d0640 1 -- 172.21.15.105:0/3334339943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc41c15d000 msgr2=0x7fc41c0d3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.688+0000 7fc4227d0640 1 --2- 172.21.15.105:0/3334339943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc41c15d000 0x7fc41c0d3920 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0b2050 tx=0x7fc41801c340 comp rx=0 tx=0).stop 2023-11-28T18:37:52.686 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.688+0000 7fc4227d0640 1 -- 172.21.15.105:0/3334339943 shutdown_connections 2023-11-28T18:37:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.688+0000 7fc4227d0640 1 --2- 172.21.15.105:0/3334339943 >> v2:172.21.15.148:3300/0 conn(0x7fc41c0b2a80 0x7fc41c0b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.688+0000 7fc4227d0640 1 --2- 172.21.15.105:0/3334339943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc41c15d000 0x7fc41c0d3920 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.688+0000 7fc4227d0640 1 -- 172.21.15.105:0/3334339943 >> 172.21.15.105:0/3334339943 conn(0x7fc41c0d2e00 msgr2=0x7fc41c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.688+0000 7fc4227d0640 1 -- 172.21.15.105:0/3334339943 shutdown_connections 2023-11-28T18:37:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.688+0000 7fc4227d0640 1 -- 172.21.15.105:0/3334339943 wait complete. 2023-11-28T18:37:52.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.688+0000 7fc4227d0640 1 Processor -- start 2023-11-28T18:37:52.690 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.688+0000 7fc4227d0640 1 -- start start 2023-11-28T18:37:52.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.692+0000 7fc4227d0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc41c0b2a80 0x7fc41c152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.692+0000 7fc4227d0640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc41c0cf6d0 0x7fc41c0cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.692+0000 7fc4227d0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc41c15d000 0x7fc41c0cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.692+0000 7fc4227d0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c1546b0 con 0x7fc41c0b2a80 2023-11-28T18:37:52.691 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.692+0000 7fc4227d0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c154ad0 con 0x7fc41c15d000 2023-11-28T18:37:52.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.692+0000 7fc4227d0640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c1537e0 con 0x7fc41c0cf6d0 2023-11-28T18:37:52.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.692+0000 7fc4217ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc41c0b2a80 0x7fc41c152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.692+0000 7fc4217ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc41c0b2a80 0x7fc41c152ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54266/0 (socket says 172.21.15.105:54266) 2023-11-28T18:37:52.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.692+0000 7fc4217ce640 1 -- 172.21.15.105:0/1822351559 learned_addr learned my addr 172.21.15.105:0/1822351559 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:52.692 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.692+0000 7fc4217ce640 1 -- 172.21.15.105:0/1822351559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc41c0cf6d0 msgr2=0x7fc41c0cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.692+0000 7fc4217ce640 1 --2- 172.21.15.105:0/1822351559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc41c0cf6d0 0x7fc41c0cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.692+0000 7fc4217ce640 1 -- 172.21.15.105:0/1822351559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc41c15d000 msgr2=0x7fc41c0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.692+0000 7fc4217ce640 1 --2- 172.21.15.105:0/1822351559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc41c15d000 0x7fc41c0cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.692+0000 7fc4217ce640 1 -- 172.21.15.105:0/1822351559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc418009420 con 0x7fc41c0b2a80 2023-11-28T18:37:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.692+0000 7fc4217ce640 1 --2- 172.21.15.105:0/1822351559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc41c0b2a80 0x7fc41c152ae0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fc4180ae040 tx=0x7fc4180190e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.692+0000 7fc411ffb640 1 -- 172.21.15.105:0/1822351559 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4180b4430 con 0x7fc41c0b2a80 2023-11-28T18:37:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.692+0000 7fc411ffb640 1 -- 172.21.15.105:0/1822351559 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4180b45b0 con 0x7fc41c0b2a80 2023-11-28T18:37:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.692+0000 7fc4227d0640 1 -- 172.21.15.105:0/1822351559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41c153a30 con 0x7fc41c0b2a80 2023-11-28T18:37:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.692+0000 7fc411ffb640 1 -- 172.21.15.105:0/1822351559 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4180b4960 con 0x7fc41c0b2a80 2023-11-28T18:37:52.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.692+0000 7fc4227d0640 1 -- 172.21.15.105:0/1822351559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41c153ff0 con 0x7fc41c0b2a80 2023-11-28T18:37:52.695 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.692+0000 7fc4227d0640 1 -- 172.21.15.105:0/1822351559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc41c1547f0 con 0x7fc41c0b2a80 2023-11-28T18:37:52.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.696+0000 7fc411ffb640 1 -- 172.21.15.105:0/1822351559 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fc4180b2040 con 0x7fc41c0b2a80 2023-11-28T18:37:52.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.696+0000 7fc411ffb640 1 --2- 172.21.15.105:0/1822351559 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc4000668d0 0x7fc400068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.696+0000 7fc411ffb640 1 -- 172.21.15.105:0/1822351559 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fc41811e450 con 0x7fc41c0b2a80 2023-11-28T18:37:52.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.696+0000 7fc420fcd640 1 --2- 172.21.15.105:0/1822351559 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc4000668d0 0x7fc400068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.696 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.696+0000 7fc420fcd640 1 --2- 172.21.15.105:0/1822351559 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc4000668d0 0x7fc400068d90 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fc41c167b90 tx=0x7fc40c007a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:52.697 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.700+0000 7fc411ffb640 1 -- 172.21.15.105:0/1822351559 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc41c1547f0 con 0x7fc41c0b2a80 2023-11-28T18:37:52.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.760+0000 7f3d95501640 1 Processor -- start 2023-11-28T18:37:52.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.760+0000 7f3d95501640 1 -- start start 2023-11-28T18:37:52.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.760+0000 7f3d95501640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d9015d320 0x7f3d9006c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.760+0000 7f3d95501640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3d900ce230 0x7f3d900cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.760+0000 7f3d95501640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d900b3c40 con 0x7f3d900cdbb0 2023-11-28T18:37:52.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.760+0000 7f3d95501640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d9006ca80 con 0x7f3d900ce230 2023-11-28T18:37:52.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.760+0000 7f3d95501640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9015d700 con 0x7f3d9015d320 2023-11-28T18:37:52.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.760+0000 7f3d8e7fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3d900ce230 0x7f3d900cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.760+0000 7f3d8f7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3d900cdbb0 0x7f3d900cef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51338/0 (socket says 172.21.15.105:51338) 2023-11-28T18:37:52.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.760+0000 7f3d8f7fe640 1 -- 172.21.15.105:0/1761498801 learned_addr learned my addr 172.21.15.105:0/1761498801 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:37:52.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.760+0000 7f3d8effd640 1 --2- 172.21.15.105:0/1761498801 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d9015d320 0x7f3d9006c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.760+0000 7f3d8dffb640 1 -- 172.21.15.105:0/1761498801 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 401591150 0 0) 0x7f3d900b3c40 con 0x7f3d900cdbb0 2023-11-28T18:37:52.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.760+0000 7f3d8dffb640 1 -- 172.21.15.105:0/1761498801 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d78003530 con 0x7f3d900cdbb0 2023-11-28T18:37:52.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.760+0000 7f3d8e7fc640 1 -- 172.21.15.105:0/1761498801 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d9015d320 msgr2=0x7f3d9006c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:52.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.760+0000 7f3d8e7fc640 1 --2- 172.21.15.105:0/1761498801 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d9015d320 0x7f3d9006c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.760+0000 7f3d8e7fc640 1 -- 172.21.15.105:0/1761498801 >> v1:172.21.15.105:6789/0 conn(0x7f3d900cdbb0 legacy=0x7f3d900cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.760+0000 7f3d8e7fc640 1 -- 172.21.15.105:0/1761498801 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d900cdfc0 con 0x7f3d900ce230 2023-11-28T18:37:52.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.764+0000 7f3d8e7fc640 1 --2- 172.21.15.105:0/1761498801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d900ce230 0x7f3d900cd670 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f3d840097c0 tx=0x7f3d8401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee0e9b4849c953e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:52.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.764+0000 7f3d8dffb640 1 -- 172.21.15.105:0/1761498801 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d840ab5f0 con 0x7f3d900ce230 2023-11-28T18:37:52.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.764+0000 7f3d8dffb640 1 -- 172.21.15.105:0/1761498801 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d840ab750 con 0x7f3d900ce230 2023-11-28T18:37:52.763 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.764+0000 7f3d8dffb640 1 -- 172.21.15.105:0/1761498801 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d840abae0 con 0x7f3d900ce230 2023-11-28T18:37:52.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.764+0000 7f3d95501640 1 -- 172.21.15.105:0/1761498801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d900ce230 msgr2=0x7f3d900cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.764+0000 7f3d95501640 1 --2- 172.21.15.105:0/1761498801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d900ce230 0x7f3d900cd670 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f3d840097c0 tx=0x7f3d8401c3f0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.764+0000 7f3d95501640 1 -- 172.21.15.105:0/1761498801 shutdown_connections 2023-11-28T18:37:52.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.764+0000 7f3d95501640 1 --2- 172.21.15.105:0/1761498801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d900ce230 0x7f3d900cd670 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.764 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.764+0000 7f3d95501640 1 --2- 172.21.15.105:0/1761498801 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d9015d320 0x7f3d9006c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.764+0000 7f3d95501640 1 -- 172.21.15.105:0/1761498801 >> 172.21.15.105:0/1761498801 conn(0x7f3d900d0df0 msgr2=0x7f3d900d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:52.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.764+0000 7f3d95501640 1 -- 172.21.15.105:0/1761498801 shutdown_connections 2023-11-28T18:37:52.765 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.764+0000 7f3d95501640 1 -- 172.21.15.105:0/1761498801 wait complete. 2023-11-28T18:37:52.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d95501640 1 Processor -- start 2023-11-28T18:37:52.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d95501640 1 -- start start 2023-11-28T18:37:52.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d95501640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d900cdbb0 0x7f3d9015f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d95501640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d900ce230 0x7f3d90159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d95501640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d9015d320 0x7f3d90159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d95501640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d90151e00 con 0x7f3d900cdbb0 2023-11-28T18:37:52.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d95501640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d900b4570 con 0x7f3d9015d320 2023-11-28T18:37:52.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d95501640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9015a190 con 0x7f3d900ce230 2023-11-28T18:37:52.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d8effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d900cdbb0 0x7f3d9015f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.768 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d8effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d900cdbb0 0x7f3d9015f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54288/0 (socket says 172.21.15.105:54288) 2023-11-28T18:37:52.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d8effd640 1 -- 172.21.15.105:0/1737214958 learned_addr learned my addr 172.21.15.105:0/1737214958 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:52.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d8f7fe640 1 --2- 172.21.15.105:0/1737214958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d9015d320 0x7f3d90159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d8e7fc640 1 --2- 172.21.15.105:0/1737214958 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d900ce230 0x7f3d90159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d8effd640 1 -- 172.21.15.105:0/1737214958 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d900ce230 msgr2=0x7f3d90159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d8effd640 1 --2- 172.21.15.105:0/1737214958 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d900ce230 0x7f3d90159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d8effd640 1 -- 172.21.15.105:0/1737214958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d9015d320 msgr2=0x7f3d90159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.769 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d8effd640 1 --2- 172.21.15.105:0/1737214958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d9015d320 0x7f3d90159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d8effd640 1 -- 172.21.15.105:0/1737214958 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d84009420 con 0x7f3d900cdbb0 2023-11-28T18:37:52.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d8f7fe640 1 --2- 172.21.15.105:0/1737214958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d9015d320 0x7f3d90159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:37:52.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d8e7fc640 1 --2- 172.21.15.105:0/1737214958 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d900ce230 0x7f3d90159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:37:52.770 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d8effd640 1 --2- 172.21.15.105:0/1737214958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d900cdbb0 0x7f3d9015f120 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f3d8000e870 tx=0x7f3d8000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:52.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d777fe640 1 -- 172.21.15.105:0/1737214958 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d80009510 con 0x7f3d900cdbb0 2023-11-28T18:37:52.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d777fe640 1 -- 172.21.15.105:0/1737214958 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8000eec0 con 0x7f3d900cdbb0 2023-11-28T18:37:52.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d777fe640 1 -- 172.21.15.105:0/1737214958 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8000cb40 con 0x7f3d900cdbb0 2023-11-28T18:37:52.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d95501640 1 -- 172.21.15.105:0/1737214958 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d9015fb70 con 0x7f3d900cdbb0 2023-11-28T18:37:52.771 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.768+0000 7f3d95501640 1 -- 172.21.15.105:0/1737214958 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d900b3f20 con 0x7f3d900cdbb0 2023-11-28T18:37:52.772 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.772+0000 7f3d95501640 1 -- 172.21.15.105:0/1737214958 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d90067570 con 0x7f3d900cdbb0 2023-11-28T18:37:52.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.772+0000 7f3d777fe640 1 -- 172.21.15.105:0/1737214958 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f3d8000cca0 con 0x7f3d900cdbb0 2023-11-28T18:37:52.773 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.772+0000 7f3d777fe640 1 --2- 172.21.15.105:0/1737214958 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d7c0668d0 0x7f3d7c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:52.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.772+0000 7f3d777fe640 1 -- 172.21.15.105:0/1737214958 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f3d80084800 con 0x7f3d900cdbb0 2023-11-28T18:37:52.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.772+0000 7f3d8e7fc640 1 --2- 172.21.15.105:0/1737214958 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d7c0668d0 0x7f3d7c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:52.774 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.772+0000 7f3d8e7fc640 1 --2- 172.21.15.105:0/1737214958 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d7c0668d0 0x7f3d7c068d90 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3d840af000 tx=0x7f3d84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:52.775 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.776+0000 7f3d777fe640 1 -- 172.21.15.105:0/1737214958 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3d80051800 con 0x7f3d900cdbb0 2023-11-28T18:37:52.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.864+0000 7fc4227d0640 1 -- 172.21.15.105:0/1822351559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc41c1547f0 con 0x7fc41c0b2a80 2023-11-28T18:37:52.862 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.864+0000 7fc411ffb640 1 -- 172.21.15.105:0/1822351559 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v130) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc4180eb580 con 0x7fc41c0b2a80 2023-11-28T18:37:52.863 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:37:52.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.868+0000 7fc4227d0640 1 -- 172.21.15.105:0/1822351559 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc4000668d0 msgr2=0x7fc400068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.868+0000 7fc4227d0640 1 --2- 172.21.15.105:0/1822351559 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc4000668d0 0x7fc400068d90 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fc41c167b90 tx=0x7fc40c007a50 comp rx=0 tx=0).stop 2023-11-28T18:37:52.865 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.868+0000 7fc4227d0640 1 -- 172.21.15.105:0/1822351559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc41c0b2a80 msgr2=0x7fc41c152ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.868+0000 7fc4227d0640 1 --2- 172.21.15.105:0/1822351559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc41c0b2a80 0x7fc41c152ae0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fc4180ae040 tx=0x7fc4180190e0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.868+0000 7fc4227d0640 1 -- 172.21.15.105:0/1822351559 shutdown_connections 2023-11-28T18:37:52.866 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.868+0000 7fc4227d0640 1 --2- 172.21.15.105:0/1822351559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc41c15d000 0x7fc41c0cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.868+0000 7fc4227d0640 1 --2- 172.21.15.105:0/1822351559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc41c0cf6d0 0x7fc41c0cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.868+0000 7fc4227d0640 1 --2- 172.21.15.105:0/1822351559 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fc4000668d0 0x7fc400068d90 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.868+0000 7fc4227d0640 1 --2- 172.21.15.105:0/1822351559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc41c0b2a80 0x7fc41c152ae0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.867 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.868+0000 7fc4227d0640 1 -- 172.21.15.105:0/1822351559 >> 172.21.15.105:0/1822351559 conn(0x7fc41c0d2e00 msgr2=0x7fc41c06bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:52.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.868+0000 7fc4227d0640 1 -- 172.21.15.105:0/1822351559 shutdown_connections 2023-11-28T18:37:52.868 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.868+0000 7fc4227d0640 1 -- 172.21.15.105:0/1822351559 wait complete. 2023-11-28T18:37:52.885 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:37:52.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.940+0000 7f3d95501640 1 -- 172.21.15.105:0/1737214958 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3d900d0910 con 0x7f3d900cdbb0 2023-11-28T18:37:52.936 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.940+0000 7f3d777fe640 1 -- 172.21.15.105:0/1737214958 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v130) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3d80054e20 con 0x7f3d900cdbb0 2023-11-28T18:37:52.937 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:37:52.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.940+0000 7f3d95501640 1 -- 172.21.15.105:0/1737214958 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d7c0668d0 msgr2=0x7f3d7c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.940+0000 7f3d95501640 1 --2- 172.21.15.105:0/1737214958 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d7c0668d0 0x7f3d7c068d90 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3d840af000 tx=0x7f3d84008040 comp rx=0 tx=0).stop 2023-11-28T18:37:52.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.940+0000 7f3d95501640 1 -- 172.21.15.105:0/1737214958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d900cdbb0 msgr2=0x7f3d9015f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:52.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.940+0000 7f3d95501640 1 --2- 172.21.15.105:0/1737214958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d900cdbb0 0x7f3d9015f120 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f3d8000e870 tx=0x7f3d8000ec70 comp rx=0 tx=0).stop 2023-11-28T18:37:52.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.940+0000 7f3d95501640 1 -- 172.21.15.105:0/1737214958 shutdown_connections 2023-11-28T18:37:52.939 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.940+0000 7f3d95501640 1 --2- 172.21.15.105:0/1737214958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d9015d320 0x7f3d90159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.940+0000 7f3d95501640 1 --2- 172.21.15.105:0/1737214958 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d900ce230 0x7f3d90159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.940+0000 7f3d95501640 1 --2- 172.21.15.105:0/1737214958 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f3d7c0668d0 0x7f3d7c068d90 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.940+0000 7f3d95501640 1 --2- 172.21.15.105:0/1737214958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d900cdbb0 0x7f3d9015f120 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:52.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.940+0000 7f3d95501640 1 -- 172.21.15.105:0/1737214958 >> 172.21.15.105:0/1737214958 conn(0x7f3d900d0df0 msgr2=0x7f3d90072f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:52.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.940+0000 7f3d95501640 1 -- 172.21.15.105:0/1737214958 shutdown_connections 2023-11-28T18:37:52.940 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:52.940+0000 7f3d95501640 1 -- 172.21.15.105:0/1737214958 wait complete. 2023-11-28T18:37:52.959 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T18:37:53.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.032+0000 7fd91c4d6640 1 Processor -- start 2023-11-28T18:37:53.029 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.032+0000 7fd91c4d6640 1 -- start start 2023-11-28T18:37:53.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.032+0000 7fd91c4d6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd91415d4f0 0x7fd9140d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.032+0000 7fd91c4d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9140d3c30 0x7fd9140d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.032+0000 7fd91c4d6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9140b25b0 con 0x7fd9140d1890 2023-11-28T18:37:53.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.032+0000 7fd91c4d6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9140b2900 con 0x7fd91415d4f0 2023-11-28T18:37:53.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.032+0000 7fd91c4d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9140cea30 con 0x7fd9140d3c30 2023-11-28T18:37:53.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.032+0000 7fd91a272640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd91415d4f0 0x7fd9140d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.032+0000 7fd91a272640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd91415d4f0 0x7fd9140d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.105:43216/0 (socket says 172.21.15.105:43216) 2023-11-28T18:37:53.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.032+0000 7fd91a272640 1 -- 172.21.15.105:0/620992176 learned_addr learned my addr 172.21.15.105:0/620992176 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:53.031 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.032+0000 7fd919a71640 1 --2- 172.21.15.105:0/620992176 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9140d3c30 0x7fd9140d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.032+0000 7fd91a272640 1 -- 172.21.15.105:0/620992176 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9140d3c30 msgr2=0x7fd9140d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:53.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.032+0000 7fd91a272640 1 --2- 172.21.15.105:0/620992176 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9140d3c30 0x7fd9140d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.032+0000 7fd91a272640 1 -- 172.21.15.105:0/620992176 >> v1:172.21.15.148:6789/0 conn(0x7fd9140d1890 legacy=0x7fd9140cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.032 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.032+0000 7fd91a272640 1 -- 172.21.15.105:0/620992176 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9140ced10 con 0x7fd91415d4f0 2023-11-28T18:37:53.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.036+0000 7fd91a272640 1 --2- 172.21.15.105:0/620992176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd91415d4f0 0x7fd9140d0e10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd910009870 tx=0x7fd91001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a3b1fb7f6540866 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:53.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.036+0000 7fd919270640 1 -- 172.21.15.105:0/620992176 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9100ad640 con 0x7fd91415d4f0 2023-11-28T18:37:53.034 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.036+0000 7fd919270640 1 -- 172.21.15.105:0/620992176 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9100ad7a0 con 0x7fd91415d4f0 2023-11-28T18:37:53.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.036+0000 7fd919270640 1 -- 172.21.15.105:0/620992176 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9100adb30 con 0x7fd91415d4f0 2023-11-28T18:37:53.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.036+0000 7fd91c4d6640 1 -- 172.21.15.105:0/620992176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd91415d4f0 msgr2=0x7fd9140d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.036+0000 7fd91c4d6640 1 --2- 172.21.15.105:0/620992176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd91415d4f0 0x7fd9140d0e10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd910009870 tx=0x7fd91001c480 comp rx=0 tx=0).stop 2023-11-28T18:37:53.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.036+0000 7fd91c4d6640 1 -- 172.21.15.105:0/620992176 shutdown_connections 2023-11-28T18:37:53.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.036+0000 7fd91c4d6640 1 --2- 172.21.15.105:0/620992176 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9140d3c30 0x7fd9140d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.036+0000 7fd91c4d6640 1 --2- 172.21.15.105:0/620992176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd91415d4f0 0x7fd9140d0e10 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.036+0000 7fd91c4d6640 1 -- 172.21.15.105:0/620992176 >> 172.21.15.105:0/620992176 conn(0x7fd9140d3030 msgr2=0x7fd9140d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:53.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.036+0000 7fd91c4d6640 1 -- 172.21.15.105:0/620992176 shutdown_connections 2023-11-28T18:37:53.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.036+0000 7fd91c4d6640 1 -- 172.21.15.105:0/620992176 wait complete. 2023-11-28T18:37:53.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd91c4d6640 1 Processor -- start 2023-11-28T18:37:53.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd91c4d6640 1 -- start start 2023-11-28T18:37:53.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd91c4d6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9140d1890 0x7fd91414a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd91c4d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9140d3c30 0x7fd9140c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd91c4d6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd91415d4f0 0x7fd91406b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd91c4d6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9140ccbe0 con 0x7fd91415d4f0 2023-11-28T18:37:53.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd91c4d6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9140cc8b0 con 0x7fd9140d3c30 2023-11-28T18:37:53.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd91c4d6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9140ca1a0 con 0x7fd9140d1890 2023-11-28T18:37:53.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd91aa73640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd91415d4f0 0x7fd91406b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd91aa73640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd91415d4f0 0x7fd91406b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54292/0 (socket says 172.21.15.105:54292) 2023-11-28T18:37:53.043 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd91aa73640 1 -- 172.21.15.105:0/3588491478 learned_addr learned my addr 172.21.15.105:0/3588491478 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:53.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd91a272640 1 --2- 172.21.15.105:0/3588491478 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9140d1890 0x7fd91414a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd919a71640 1 --2- 172.21.15.105:0/3588491478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9140d3c30 0x7fd9140c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd91aa73640 1 -- 172.21.15.105:0/3588491478 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9140d1890 msgr2=0x7fd91414a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd91aa73640 1 --2- 172.21.15.105:0/3588491478 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9140d1890 0x7fd91414a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd91aa73640 1 -- 172.21.15.105:0/3588491478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9140d3c30 msgr2=0x7fd9140c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.044 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd91aa73640 1 --2- 172.21.15.105:0/3588491478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9140d3c30 0x7fd9140c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd91aa73640 1 -- 172.21.15.105:0/3588491478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9100094d0 con 0x7fd91415d4f0 2023-11-28T18:37:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd91aa73640 1 --2- 172.21.15.105:0/3588491478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd91415d4f0 0x7fd91406b6a0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00c730 tx=0x7fd90c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd90affd640 1 -- 172.21.15.105:0/3588491478 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c003b80 con 0x7fd91415d4f0 2023-11-28T18:37:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd90affd640 1 -- 172.21.15.105:0/3588491478 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c003ce0 con 0x7fd91415d4f0 2023-11-28T18:37:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd91c4d6640 1 -- 172.21.15.105:0/3588491478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9140ca480 con 0x7fd91415d4f0 2023-11-28T18:37:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd90affd640 1 -- 172.21.15.105:0/3588491478 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00aa30 con 0x7fd91415d4f0 2023-11-28T18:37:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.040+0000 7fd91c4d6640 1 -- 172.21.15.105:0/3588491478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9140c9d40 con 0x7fd91415d4f0 2023-11-28T18:37:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.044+0000 7fd91c4d6640 1 -- 172.21.15.105:0/3588491478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9140ccd50 con 0x7fd91415d4f0 2023-11-28T18:37:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.044+0000 7fd90affd640 1 -- 172.21.15.105:0/3588491478 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7fd90c00ab90 con 0x7fd91415d4f0 2023-11-28T18:37:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.044+0000 7fd90affd640 1 --2- 172.21.15.105:0/3588491478 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd8f8066900 0x7fd8f8068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.044+0000 7fd90affd640 1 -- 172.21.15.105:0/3588491478 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fd90c084220 con 0x7fd91415d4f0 2023-11-28T18:37:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.044+0000 7fd91a272640 1 --2- 172.21.15.105:0/3588491478 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd8f8066900 0x7fd8f8068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.044+0000 7fd91a272640 1 --2- 172.21.15.105:0/3588491478 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd8f8066900 0x7fd8f8068dc0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd9100af000 tx=0x7fd910008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.048+0000 7fd90affd640 1 -- 172.21.15.105:0/3588491478 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd90c051360 con 0x7fd91415d4f0 2023-11-28T18:37:53.101 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.104+0000 7f81e6a17640 1 Processor -- start 2023-11-28T18:37:53.102 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.104+0000 7f81e6a17640 1 -- start start 2023-11-28T18:37:53.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.104+0000 7f81e6a17640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f81e015d4f0 0x7f81e00d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.104+0000 7f81e6a17640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81e00d3c30 0x7f81e00d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.104+0000 7f81e6a17640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e00b25b0 con 0x7f81e00d1890 2023-11-28T18:37:53.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.104+0000 7f81e6a17640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e00b2900 con 0x7f81e015d4f0 2023-11-28T18:37:53.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.104+0000 7f81e6a17640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e00cea30 con 0x7f81e00d3c30 2023-11-28T18:37:53.104 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.104+0000 7f81e4fb4640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f81e00d1890 0x7f81e00cfa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51344/0 (socket says 172.21.15.105:51344) 2023-11-28T18:37:53.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.104+0000 7f81e4fb4640 1 -- 172.21.15.105:0/568273384 learned_addr learned my addr 172.21.15.105:0/568273384 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:37:53.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.104+0000 7f81dffff640 1 --2- 172.21.15.105:0/568273384 >> v2:172.21.15.105:3300/0 conn(0x7f81e015d4f0 0x7f81e00d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.104+0000 7f81df7fe640 1 --2- 172.21.15.105:0/568273384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81e00d3c30 0x7f81e00d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.104+0000 7f81deffd640 1 -- 172.21.15.105:0/568273384 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 680621964 0 0) 0x7f81e00b25b0 con 0x7f81e00d1890 2023-11-28T18:37:53.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.108+0000 7f81deffd640 1 -- 172.21.15.105:0/568273384 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81c8003530 con 0x7f81e00d1890 2023-11-28T18:37:53.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.108+0000 7f81dffff640 1 -- 172.21.15.105:0/568273384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81e00d3c30 msgr2=0x7f81e00d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:53.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.108+0000 7f81dffff640 1 --2- 172.21.15.105:0/568273384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81e00d3c30 0x7f81e00d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.108+0000 7f81dffff640 1 -- 172.21.15.105:0/568273384 >> v1:172.21.15.105:6789/0 conn(0x7f81e00d1890 legacy=0x7f81e00cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.105 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.108+0000 7f81dffff640 1 -- 172.21.15.105:0/568273384 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e00ced10 con 0x7f81e015d4f0 2023-11-28T18:37:53.106 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.108+0000 7f81dffff640 1 --2- 172.21.15.105:0/568273384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81e015d4f0 0x7f81e00d0e10 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f81d00097c0 tx=0x7f81d001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54553d987f2a34af server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:53.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.108+0000 7f81deffd640 1 -- 172.21.15.105:0/568273384 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d00ab600 con 0x7f81e015d4f0 2023-11-28T18:37:53.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.108+0000 7f81deffd640 1 -- 172.21.15.105:0/568273384 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d00a7050 con 0x7f81e015d4f0 2023-11-28T18:37:53.107 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.108+0000 7f81deffd640 1 -- 172.21.15.105:0/568273384 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d00ab960 con 0x7f81e015d4f0 2023-11-28T18:37:53.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.108+0000 7f81e6a17640 1 -- 172.21.15.105:0/568273384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81e015d4f0 msgr2=0x7f81e00d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.108+0000 7f81e6a17640 1 --2- 172.21.15.105:0/568273384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81e015d4f0 0x7f81e00d0e10 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f81d00097c0 tx=0x7f81d001c3c0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.108+0000 7f81e6a17640 1 -- 172.21.15.105:0/568273384 shutdown_connections 2023-11-28T18:37:53.108 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.108+0000 7f81e6a17640 1 --2- 172.21.15.105:0/568273384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81e00d3c30 0x7f81e00d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.108+0000 7f81e6a17640 1 --2- 172.21.15.105:0/568273384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81e015d4f0 0x7f81e00d0e10 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.108+0000 7f81e6a17640 1 -- 172.21.15.105:0/568273384 >> 172.21.15.105:0/568273384 conn(0x7f81e00d3030 msgr2=0x7f81e00d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:53.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.112+0000 7f81e6a17640 1 -- 172.21.15.105:0/568273384 shutdown_connections 2023-11-28T18:37:53.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.112+0000 7f81e6a17640 1 -- 172.21.15.105:0/568273384 wait complete. 2023-11-28T18:37:53.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.112+0000 7f81e6a17640 1 Processor -- start 2023-11-28T18:37:53.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.112+0000 7f81e6a17640 1 -- start start 2023-11-28T18:37:53.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.112+0000 7f81e6a17640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81e00d1890 0x7f81e0157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.112+0000 7f81e6a17640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81e00d3c30 0x7f81e015f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.112+0000 7f81e6a17640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81e015d4f0 0x7f81e015fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.112+0000 7f81e6a17640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e01fa460 con 0x7f81e015d4f0 2023-11-28T18:37:53.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.112+0000 7f81e6a17640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e0159390 con 0x7f81e00d1890 2023-11-28T18:37:53.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.112+0000 7f81e6a17640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e0159c30 con 0x7f81e00d3c30 2023-11-28T18:37:53.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.112+0000 7f81e4fb4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81e015d4f0 0x7f81e015fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.112+0000 7f81e4fb4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81e015d4f0 0x7f81e015fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54310/0 (socket says 172.21.15.105:54310) 2023-11-28T18:37:53.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.112+0000 7f81e4fb4640 1 -- 172.21.15.105:0/1886671565 learned_addr learned my addr 172.21.15.105:0/1886671565 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:53.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.112+0000 7f81df7fe640 1 --2- 172.21.15.105:0/1886671565 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81e00d3c30 0x7f81e015f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.112+0000 7f81dffff640 1 --2- 172.21.15.105:0/1886671565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81e00d1890 0x7f81e0157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.112+0000 7f81e4fb4640 1 -- 172.21.15.105:0/1886671565 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81e00d3c30 msgr2=0x7f81e015f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.112+0000 7f81e4fb4640 1 --2- 172.21.15.105:0/1886671565 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81e00d3c30 0x7f81e015f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.112+0000 7f81e4fb4640 1 -- 172.21.15.105:0/1886671565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81e00d1890 msgr2=0x7f81e0157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.112+0000 7f81e4fb4640 1 --2- 172.21.15.105:0/1886671565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81e00d1890 0x7f81e0157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.112+0000 7f81e4fb4640 1 -- 172.21.15.105:0/1886671565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d0009420 con 0x7f81e015d4f0 2023-11-28T18:37:53.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.116+0000 7f81e4fb4640 1 --2- 172.21.15.105:0/1886671565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81e015d4f0 0x7f81e015fa90 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f81d800caa0 tx=0x7f81d800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:53.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.116+0000 7f81dcff9640 1 -- 172.21.15.105:0/1886671565 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d800ac00 con 0x7f81e015d4f0 2023-11-28T18:37:53.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.116+0000 7f81dcff9640 1 -- 172.21.15.105:0/1886671565 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d800ad60 con 0x7f81e015d4f0 2023-11-28T18:37:53.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.116+0000 7f81dcff9640 1 -- 172.21.15.105:0/1886671565 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8012620 con 0x7f81e015d4f0 2023-11-28T18:37:53.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.116+0000 7f81e6a17640 1 -- 172.21.15.105:0/1886671565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e01600a0 con 0x7f81e015d4f0 2023-11-28T18:37:53.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.116+0000 7f81e6a17640 1 -- 172.21.15.105:0/1886671565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e01f9e10 con 0x7f81e015d4f0 2023-11-28T18:37:53.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.116+0000 7f81e6a17640 1 -- 172.21.15.105:0/1886671565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e01fa610 con 0x7f81e015d4f0 2023-11-28T18:37:53.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.116+0000 7f81dcff9640 1 -- 172.21.15.105:0/1886671565 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f81d8007070 con 0x7f81e015d4f0 2023-11-28T18:37:53.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.116+0000 7f81dcff9640 1 --2- 172.21.15.105:0/1886671565 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f81cc0668d0 0x7f81cc068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.120+0000 7f81dcff9640 1 -- 172.21.15.105:0/1886671565 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f81d80844c0 con 0x7f81e015d4f0 2023-11-28T18:37:53.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.120+0000 7f81dffff640 1 --2- 172.21.15.105:0/1886671565 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f81cc0668d0 0x7f81cc068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.120+0000 7f81dffff640 1 --2- 172.21.15.105:0/1886671565 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f81cc0668d0 0x7f81cc068d90 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f81d00ad000 tx=0x7f81d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:53.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.124+0000 7f81dcff9640 1 -- 172.21.15.105:0/1886671565 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f81e01fa610 con 0x7f81e015d4f0 2023-11-28T18:37:53.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.200+0000 7fd91c4d6640 1 -- 172.21.15.105:0/3588491478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd914067570 con 0x7fd91415d4f0 2023-11-28T18:37:53.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.200+0000 7fd90affd640 1 -- 172.21.15.105:0/3588491478 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v130) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd90c054980 con 0x7fd91415d4f0 2023-11-28T18:37:53.200 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:37:53.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.204+0000 7fd91c4d6640 1 -- 172.21.15.105:0/3588491478 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd8f8066900 msgr2=0x7fd8f8068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.204+0000 7fd91c4d6640 1 --2- 172.21.15.105:0/3588491478 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd8f8066900 0x7fd8f8068dc0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd9100af000 tx=0x7fd910008040 comp rx=0 tx=0).stop 2023-11-28T18:37:53.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.204+0000 7fd91c4d6640 1 -- 172.21.15.105:0/3588491478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd91415d4f0 msgr2=0x7fd91406b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.204+0000 7fd91c4d6640 1 --2- 172.21.15.105:0/3588491478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd91415d4f0 0x7fd91406b6a0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00c730 tx=0x7fd90c00cbf0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.202 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.204+0000 7fd91c4d6640 1 -- 172.21.15.105:0/3588491478 shutdown_connections 2023-11-28T18:37:53.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.204+0000 7fd91c4d6640 1 --2- 172.21.15.105:0/3588491478 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7fd8f8066900 0x7fd8f8068dc0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.204+0000 7fd91c4d6640 1 --2- 172.21.15.105:0/3588491478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd91415d4f0 0x7fd91406b6a0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.204+0000 7fd91c4d6640 1 --2- 172.21.15.105:0/3588491478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9140d3c30 0x7fd9140c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.204+0000 7fd91c4d6640 1 --2- 172.21.15.105:0/3588491478 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9140d1890 0x7fd91414a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.204+0000 7fd91c4d6640 1 -- 172.21.15.105:0/3588491478 >> 172.21.15.105:0/3588491478 conn(0x7fd9140d3030 msgr2=0x7fd91406c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:53.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.204+0000 7fd91c4d6640 1 -- 172.21.15.105:0/3588491478 shutdown_connections 2023-11-28T18:37:53.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.204+0000 7fd91c4d6640 1 -- 172.21.15.105:0/3588491478 wait complete. 2023-11-28T18:37:53.221 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T18:37:53.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.288+0000 7f81e6a17640 1 -- 172.21.15.105:0/1886671565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f81e0067570 con 0x7f81e015d4f0 2023-11-28T18:37:53.285 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.288+0000 7f81dcff9640 1 -- 172.21.15.105:0/1886671565 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v130) v1 ==== 80+0+124 (secure 0 0 0) 0x7f81d8051600 con 0x7f81e015d4f0 2023-11-28T18:37:53.285 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.148:6835/3706038483 2023-11-29T18:32:58.181068+0000 2023-11-28T18:37:53.285 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.148:6833/3706038483 2023-11-29T18:32:58.181068+0000 2023-11-28T18:37:53.286 INFO:teuthology.orchestra.run.smithi105.stderr:listed 2 entries 2023-11-28T18:37:53.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.288+0000 7f81e6a17640 1 -- 172.21.15.105:0/1886671565 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f81cc0668d0 msgr2=0x7f81cc068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.288+0000 7f81e6a17640 1 --2- 172.21.15.105:0/1886671565 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f81cc0668d0 0x7f81cc068d90 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f81d00ad000 tx=0x7f81d0008040 comp rx=0 tx=0).stop 2023-11-28T18:37:53.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.288+0000 7f81e6a17640 1 -- 172.21.15.105:0/1886671565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81e015d4f0 msgr2=0x7f81e015fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.288+0000 7f81e6a17640 1 --2- 172.21.15.105:0/1886671565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81e015d4f0 0x7f81e015fa90 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f81d800caa0 tx=0x7f81d800cf60 comp rx=0 tx=0).stop 2023-11-28T18:37:53.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.288+0000 7f81e6a17640 1 -- 172.21.15.105:0/1886671565 shutdown_connections 2023-11-28T18:37:53.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.288+0000 7f81e6a17640 1 --2- 172.21.15.105:0/1886671565 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f81cc0668d0 0x7f81cc068d90 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.288+0000 7f81e6a17640 1 --2- 172.21.15.105:0/1886671565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81e015d4f0 0x7f81e015fa90 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.288+0000 7f81e6a17640 1 --2- 172.21.15.105:0/1886671565 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81e00d3c30 0x7f81e015f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.288+0000 7f81e6a17640 1 --2- 172.21.15.105:0/1886671565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81e00d1890 0x7f81e0157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.288+0000 7f81e6a17640 1 -- 172.21.15.105:0/1886671565 >> 172.21.15.105:0/1886671565 conn(0x7f81e00d3030 msgr2=0x7f81e00b4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:53.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.288+0000 7f81e6a17640 1 -- 172.21.15.105:0/1886671565 shutdown_connections 2023-11-28T18:37:53.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.288+0000 7f81e6a17640 1 -- 172.21.15.105:0/1886671565 wait complete. 2023-11-28T18:37:53.307 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:37:53.307 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:37:53.317 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:37:53.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.364+0000 7f8702879640 1 Processor -- start 2023-11-28T18:37:53.363 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.364+0000 7f8702879640 1 -- start start 2023-11-28T18:37:53.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.368+0000 7f8702879640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86fc0af340 0x7f86fc0721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.368+0000 7f8702879640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86fc06f190 con 0x7f86fc06a580 2023-11-28T18:37:53.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.368+0000 7f8702879640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86fc0b2ee0 con 0x7f86fc06ed00 2023-11-28T18:37:53.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.368+0000 7f8702879640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86fc0b34a0 con 0x7f86fc0af340 2023-11-28T18:37:53.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.368+0000 7f86fbfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86fc0af340 0x7f86fc0721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.368+0000 7f86fbfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86fc0af340 0x7f86fc0721d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54322/0 (socket says 172.21.15.105:54322) 2023-11-28T18:37:53.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.368+0000 7f86fbfff640 1 -- 172.21.15.105:0/1534044580 learned_addr learned my addr 172.21.15.105:0/1534044580 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:53.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.368+0000 7f86fbfff640 1 -- 172.21.15.105:0/1534044580 >> v1:172.21.15.148:6789/0 conn(0x7f86fc06ed00 legacy=0x7f86fc0d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.368+0000 7f86fbfff640 1 -- 172.21.15.105:0/1534044580 >> v1:172.21.15.105:6789/0 conn(0x7f86fc06a580 legacy=0x7f86fc06d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.368+0000 7f86fbfff640 1 -- 172.21.15.105:0/1534044580 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86fc0b31c0 con 0x7f86fc0af340 2023-11-28T18:37:53.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.368+0000 7f86faffd640 1 -- 172.21.15.105:0/1534044580 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3649387323 0 0) 0x7f86fc06f190 con 0x7f86fc06a580 2023-11-28T18:37:53.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.372+0000 7f86fbfff640 1 --2- 172.21.15.105:0/1534044580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86fc0af340 0x7f86fc0721d0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009870 tx=0x7f86ec01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddacc025fa4670db server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:53.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.372+0000 7f86faffd640 1 -- 172.21.15.105:0/1534044580 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec0aa670 con 0x7f86fc0af340 2023-11-28T18:37:53.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.372+0000 7f86faffd640 1 -- 172.21.15.105:0/1534044580 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ec0aa7d0 con 0x7f86fc0af340 2023-11-28T18:37:53.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.372+0000 7f86faffd640 1 -- 172.21.15.105:0/1534044580 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec0aab60 con 0x7f86fc0af340 2023-11-28T18:37:53.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.372+0000 7f8702879640 1 -- 172.21.15.105:0/1534044580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86fc0af340 msgr2=0x7f86fc0721d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.372+0000 7f8702879640 1 --2- 172.21.15.105:0/1534044580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86fc0af340 0x7f86fc0721d0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009870 tx=0x7f86ec01c510 comp rx=0 tx=0).stop 2023-11-28T18:37:53.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.372+0000 7f8702879640 1 -- 172.21.15.105:0/1534044580 shutdown_connections 2023-11-28T18:37:53.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.372+0000 7f8702879640 1 --2- 172.21.15.105:0/1534044580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86fc0af340 0x7f86fc0721d0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.372+0000 7f8702879640 1 -- 172.21.15.105:0/1534044580 >> 172.21.15.105:0/1534044580 conn(0x7f86fc06b860 msgr2=0x7f86fc06bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:53.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.372+0000 7f8702879640 1 -- 172.21.15.105:0/1534044580 shutdown_connections 2023-11-28T18:37:53.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.372+0000 7f8702879640 1 -- 172.21.15.105:0/1534044580 wait complete. 2023-11-28T18:37:53.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f8702879640 1 Processor -- start 2023-11-28T18:37:53.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f8702879640 1 -- start start 2023-11-28T18:37:53.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f8702879640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86fc06a580 0x7f86fc146ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f8702879640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86fc06ed00 0x7f86fc1600a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f8702879640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86fc0af340 0x7f86fc142790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f8702879640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc1480d0 con 0x7f86fc0af340 2023-11-28T18:37:53.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f8702879640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc142300 con 0x7f86fc06ed00 2023-11-28T18:37:53.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f8702879640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc1474c0 con 0x7f86fc06a580 2023-11-28T18:37:53.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f8700e16640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86fc0af340 0x7f86fc142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f8700e16640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86fc0af340 0x7f86fc142790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54334/0 (socket says 172.21.15.105:54334) 2023-11-28T18:37:53.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f86fbfff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86fc06a580 0x7f86fc146ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f8700e16640 1 -- 172.21.15.105:0/1762624268 learned_addr learned my addr 172.21.15.105:0/1762624268 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:53.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f86fb7fe640 1 --2- 172.21.15.105:0/1762624268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86fc06ed00 0x7f86fc1600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f8700e16640 1 -- 172.21.15.105:0/1762624268 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86fc06a580 msgr2=0x7f86fc146ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f8700e16640 1 --2- 172.21.15.105:0/1762624268 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86fc06a580 0x7f86fc146ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f8700e16640 1 -- 172.21.15.105:0/1762624268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86fc06ed00 msgr2=0x7f86fc1600a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f8700e16640 1 --2- 172.21.15.105:0/1762624268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86fc06ed00 0x7f86fc1600a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f8700e16640 1 -- 172.21.15.105:0/1762624268 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ec0094d0 con 0x7f86fc0af340 2023-11-28T18:37:53.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f8700e16640 1 --2- 172.21.15.105:0/1762624268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86fc0af340 0x7f86fc142790 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f86f4001570 tx=0x7f86f400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:53.379 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f86f8ff9640 1 -- 172.21.15.105:0/1762624268 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4003db0 con 0x7f86fc0af340 2023-11-28T18:37:53.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f86f8ff9640 1 -- 172.21.15.105:0/1762624268 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f4005070 con 0x7f86fc0af340 2023-11-28T18:37:53.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f86f8ff9640 1 -- 172.21.15.105:0/1762624268 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4005420 con 0x7f86fc0af340 2023-11-28T18:37:53.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f8702879640 1 -- 172.21.15.105:0/1762624268 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86fc1477a0 con 0x7f86fc0af340 2023-11-28T18:37:53.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.376+0000 7f8702879640 1 -- 172.21.15.105:0/1762624268 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86fc147d60 con 0x7f86fc0af340 2023-11-28T18:37:53.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.380+0000 7f8702879640 1 -- 172.21.15.105:0/1762624268 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86fc067570 con 0x7f86fc0af340 2023-11-28T18:37:53.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.380+0000 7f86f8ff9640 1 -- 172.21.15.105:0/1762624268 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f86f4005580 con 0x7f86fc0af340 2023-11-28T18:37:53.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.380+0000 7f86f8ff9640 1 --2- 172.21.15.105:0/1762624268 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f86e80668d0 0x7f86e8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.380+0000 7f86f8ff9640 1 -- 172.21.15.105:0/1762624268 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f86f4083d80 con 0x7f86fc0af340 2023-11-28T18:37:53.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.380+0000 7f86fbfff640 1 --2- 172.21.15.105:0/1762624268 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f86e80668d0 0x7f86e8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.380+0000 7f86fbfff640 1 --2- 172.21.15.105:0/1762624268 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f86e80668d0 0x7f86e8068d90 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f86ec0ae000 tx=0x7f86ec0b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:53.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.384+0000 7f86f8ff9640 1 -- 172.21.15.105:0/1762624268 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f86f4050f60 con 0x7f86fc0af340 2023-11-28T18:37:53.507 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.508+0000 7ff88499d640 1 Processor -- start 2023-11-28T18:37:53.508 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.508+0000 7ff88499d640 1 -- start start 2023-11-28T18:37:53.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.512+0000 7ff88499d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff880159f40 0x7ff8800caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.512+0000 7ff88499d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff8800ad850 0x7ff8800701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.512+0000 7ff88499d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88006bf50 con 0x7ff8800cc760 2023-11-28T18:37:53.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.512+0000 7ff88499d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8800ccbc0 con 0x7ff8800ad850 2023-11-28T18:37:53.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.512+0000 7ff88499d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff88015a320 con 0x7ff880159f40 2023-11-28T18:37:53.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.512+0000 7ff87dd9b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff8800cc760 0x7ff8800ad140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51374/0 (socket says 172.21.15.105:51374) 2023-11-28T18:37:53.510 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.512+0000 7ff87dd9b640 1 -- 172.21.15.105:0/1386352856 learned_addr learned my addr 172.21.15.105:0/1386352856 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:37:53.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.512+0000 7ff87e59c640 1 --2- 172.21.15.105:0/1386352856 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff880159f40 0x7ff8800caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.512+0000 7ff87ed9d640 1 --2- 172.21.15.105:0/1386352856 >> v2:172.21.15.148:3300/0 conn(0x7ff8800ad850 0x7ff8800701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.512+0000 7ff87d59a640 1 -- 172.21.15.105:0/1386352856 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597252159 0 0) 0x7ff88006bf50 con 0x7ff8800cc760 2023-11-28T18:37:53.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.512+0000 7ff87d59a640 1 -- 172.21.15.105:0/1386352856 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff868003530 con 0x7ff8800cc760 2023-11-28T18:37:53.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.512+0000 7ff87e59c640 1 -- 172.21.15.105:0/1386352856 >> v2:172.21.15.148:3300/0 conn(0x7ff8800ad850 msgr2=0x7ff8800701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:53.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.512+0000 7ff87e59c640 1 --2- 172.21.15.105:0/1386352856 >> v2:172.21.15.148:3300/0 conn(0x7ff8800ad850 0x7ff8800701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.512+0000 7ff87e59c640 1 -- 172.21.15.105:0/1386352856 >> v1:172.21.15.105:6789/0 conn(0x7ff8800cc760 legacy=0x7ff8800ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.511 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.512+0000 7ff87e59c640 1 -- 172.21.15.105:0/1386352856 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff880071380 con 0x7ff880159f40 2023-11-28T18:37:53.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.516+0000 7ff87e59c640 1 --2- 172.21.15.105:0/1386352856 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff880159f40 0x7ff8800caf60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff8700097c0 tx=0x7ff87001c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82880d56da296ff0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:53.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.516+0000 7ff87d59a640 1 -- 172.21.15.105:0/1386352856 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8700ac620 con 0x7ff880159f40 2023-11-28T18:37:53.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.516+0000 7ff87d59a640 1 -- 172.21.15.105:0/1386352856 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8700ac780 con 0x7ff880159f40 2023-11-28T18:37:53.513 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.516+0000 7ff87d59a640 1 -- 172.21.15.105:0/1386352856 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8700acb10 con 0x7ff880159f40 2023-11-28T18:37:53.514 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.516+0000 7ff88499d640 1 -- 172.21.15.105:0/1386352856 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff880159f40 msgr2=0x7ff8800caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.516+0000 7ff88499d640 1 --2- 172.21.15.105:0/1386352856 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff880159f40 0x7ff8800caf60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff8700097c0 tx=0x7ff87001c3c0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.516+0000 7ff88499d640 1 -- 172.21.15.105:0/1386352856 shutdown_connections 2023-11-28T18:37:53.515 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.516+0000 7ff88499d640 1 --2- 172.21.15.105:0/1386352856 >> v2:172.21.15.148:3300/0 conn(0x7ff8800ad850 0x7ff8800701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.516+0000 7ff88499d640 1 --2- 172.21.15.105:0/1386352856 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff880159f40 0x7ff8800caf60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.516+0000 7ff88499d640 1 -- 172.21.15.105:0/1386352856 >> 172.21.15.105:0/1386352856 conn(0x7ff8800ceb90 msgr2=0x7ff8800cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:53.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.516+0000 7ff88499d640 1 -- 172.21.15.105:0/1386352856 shutdown_connections 2023-11-28T18:37:53.516 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.516+0000 7ff88499d640 1 -- 172.21.15.105:0/1386352856 wait complete. 2023-11-28T18:37:53.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff88499d640 1 Processor -- start 2023-11-28T18:37:53.517 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff88499d640 1 -- start start 2023-11-28T18:37:53.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff88499d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff8800ad850 0x7ff880143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff88499d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8800cc760 0x7ff8800b3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff88499d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff880159f40 0x7ff8801448e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff88499d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff880143b90 con 0x7ff880159f40 2023-11-28T18:37:53.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff88499d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8800b4780 con 0x7ff8800cc760 2023-11-28T18:37:53.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff88499d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8800b41c0 con 0x7ff8800ad850 2023-11-28T18:37:53.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff87ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff880159f40 0x7ff8801448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff87ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff880159f40 0x7ff8801448e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54342/0 (socket says 172.21.15.105:54342) 2023-11-28T18:37:53.519 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff87ed9d640 1 -- 172.21.15.105:0/1408444114 learned_addr learned my addr 172.21.15.105:0/1408444114 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:53.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff87dd9b640 1 --2- 172.21.15.105:0/1408444114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8800cc760 0x7ff8800b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff87e59c640 1 --2- 172.21.15.105:0/1408444114 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff8800ad850 0x7ff880143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff87ed9d640 1 -- 172.21.15.105:0/1408444114 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff8800ad850 msgr2=0x7ff880143650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff87ed9d640 1 --2- 172.21.15.105:0/1408444114 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff8800ad850 0x7ff880143650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff87ed9d640 1 -- 172.21.15.105:0/1408444114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8800cc760 msgr2=0x7ff8800b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff87ed9d640 1 --2- 172.21.15.105:0/1408444114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8800cc760 0x7ff8800b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.520 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff87ed9d640 1 -- 172.21.15.105:0/1408444114 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff870009420 con 0x7ff880159f40 2023-11-28T18:37:53.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff87ed9d640 1 --2- 172.21.15.105:0/1408444114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff880159f40 0x7ff8801448e0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7ff87800efc0 tx=0x7ff87800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:53.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff86effd640 1 -- 172.21.15.105:0/1408444114 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878014480 con 0x7ff880159f40 2023-11-28T18:37:53.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff86effd640 1 -- 172.21.15.105:0/1408444114 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87800f040 con 0x7ff880159f40 2023-11-28T18:37:53.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff86effd640 1 -- 172.21.15.105:0/1408444114 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878014820 con 0x7ff880159f40 2023-11-28T18:37:53.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff88499d640 1 -- 172.21.15.105:0/1408444114 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8800b44a0 con 0x7ff880159f40 2023-11-28T18:37:53.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.520+0000 7ff88499d640 1 -- 172.21.15.105:0/1408444114 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8801443c0 con 0x7ff880159f40 2023-11-28T18:37:53.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.524+0000 7ff88499d640 1 -- 172.21.15.105:0/1408444114 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8801445e0 con 0x7ff880159f40 2023-11-28T18:37:53.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.524+0000 7ff86effd640 1 -- 172.21.15.105:0/1408444114 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7ff878014980 con 0x7ff880159f40 2023-11-28T18:37:53.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.524+0000 7ff86effd640 1 --2- 172.21.15.105:0/1408444114 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff8600668d0 0x7ff860068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.524+0000 7ff86effd640 1 -- 172.21.15.105:0/1408444114 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7ff878085400 con 0x7ff880159f40 2023-11-28T18:37:53.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.524+0000 7ff87e59c640 1 --2- 172.21.15.105:0/1408444114 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff8600668d0 0x7ff860068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.524+0000 7ff87e59c640 1 --2- 172.21.15.105:0/1408444114 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff8600668d0 0x7ff860068d90 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff8700ae000 tx=0x7ff8700a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:53.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.528+0000 7ff86effd640 1 -- 172.21.15.105:0/1408444114 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff8780524b0 con 0x7ff880159f40 2023-11-28T18:37:53.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.552+0000 7f8702879640 1 -- 172.21.15.105:0/1762624268 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f86fc148210 con 0x7f86fc0af340 2023-11-28T18:37:53.551 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.552+0000 7f86f8ff9640 1 -- 172.21.15.105:0/1762624268 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v130) v1 ==== 80+0+124 (secure 0 0 0) 0x7f86f4017070 con 0x7f86fc0af340 2023-11-28T18:37:53.552 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.148:6835/3706038483 2023-11-29T18:32:58.181068+0000 2023-11-28T18:37:53.552 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.148:6833/3706038483 2023-11-29T18:32:58.181068+0000 2023-11-28T18:37:53.552 INFO:teuthology.orchestra.run.smithi105.stderr:listed 2 entries 2023-11-28T18:37:53.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.556+0000 7f8702879640 1 -- 172.21.15.105:0/1762624268 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f86e80668d0 msgr2=0x7f86e8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.556+0000 7f8702879640 1 --2- 172.21.15.105:0/1762624268 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f86e80668d0 0x7f86e8068d90 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f86ec0ae000 tx=0x7f86ec0b5040 comp rx=0 tx=0).stop 2023-11-28T18:37:53.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.556+0000 7f8702879640 1 -- 172.21.15.105:0/1762624268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86fc0af340 msgr2=0x7f86fc142790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.556+0000 7f8702879640 1 --2- 172.21.15.105:0/1762624268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86fc0af340 0x7f86fc142790 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f86f4001570 tx=0x7f86f400cd20 comp rx=0 tx=0).stop 2023-11-28T18:37:53.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.556+0000 7f8702879640 1 -- 172.21.15.105:0/1762624268 shutdown_connections 2023-11-28T18:37:53.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.556+0000 7f8702879640 1 --2- 172.21.15.105:0/1762624268 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f86e80668d0 0x7f86e8068d90 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.556+0000 7f8702879640 1 --2- 172.21.15.105:0/1762624268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86fc0af340 0x7f86fc142790 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.554 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.556+0000 7f8702879640 1 --2- 172.21.15.105:0/1762624268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86fc06ed00 0x7f86fc1600a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.556+0000 7f8702879640 1 --2- 172.21.15.105:0/1762624268 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86fc06a580 0x7f86fc146ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.556+0000 7f8702879640 1 -- 172.21.15.105:0/1762624268 >> 172.21.15.105:0/1762624268 conn(0x7f86fc06b860 msgr2=0x7f86fc06bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:53.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.556+0000 7f8702879640 1 -- 172.21.15.105:0/1762624268 shutdown_connections 2023-11-28T18:37:53.555 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.556+0000 7f8702879640 1 -- 172.21.15.105:0/1762624268 wait complete. 2023-11-28T18:37:53.573 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2023-11-28T18:37:53.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:37:53.574 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-28T18:37:53.641 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:37:53.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:37:53.641 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T18:37:53.682 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.684+0000 7ff88499d640 1 -- 172.21.15.105:0/1408444114 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8800cf720 con 0x7ff880159f40 2023-11-28T18:37:53.683 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.684+0000 7ff86effd640 1 -- 172.21.15.105:0/1408444114 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+16923 (secure 0 0 0) 0x7ff878055ad0 con 0x7ff880159f40 2023-11-28T18:37:53.686 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:37:53.686 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":130,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:32:58.181121+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:31:56.971692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-28T18:31:57.881657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6835/3706038483":"2023-11-29T18:32:58.181068+0000","172.21.15.148:6833/3706038483":"2023-11-29T18:32:58.181068+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:37:53.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.688+0000 7ff88499d640 1 -- 172.21.15.105:0/1408444114 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff8600668d0 msgr2=0x7ff860068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.688+0000 7ff88499d640 1 --2- 172.21.15.105:0/1408444114 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff8600668d0 0x7ff860068d90 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff8700ae000 tx=0x7ff8700a7000 comp rx=0 tx=0).stop 2023-11-28T18:37:53.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.688+0000 7ff88499d640 1 -- 172.21.15.105:0/1408444114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff880159f40 msgr2=0x7ff8801448e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.688+0000 7ff88499d640 1 --2- 172.21.15.105:0/1408444114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff880159f40 0x7ff8801448e0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7ff87800efc0 tx=0x7ff87800c490 comp rx=0 tx=0).stop 2023-11-28T18:37:53.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.688+0000 7ff88499d640 1 -- 172.21.15.105:0/1408444114 shutdown_connections 2023-11-28T18:37:53.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.688+0000 7ff88499d640 1 --2- 172.21.15.105:0/1408444114 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7ff8600668d0 0x7ff860068d90 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.688+0000 7ff88499d640 1 --2- 172.21.15.105:0/1408444114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff880159f40 0x7ff8801448e0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.688+0000 7ff88499d640 1 --2- 172.21.15.105:0/1408444114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8800cc760 0x7ff8800b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.687 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.688+0000 7ff88499d640 1 --2- 172.21.15.105:0/1408444114 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff8800ad850 0x7ff880143650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.688+0000 7ff88499d640 1 -- 172.21.15.105:0/1408444114 >> 172.21.15.105:0/1408444114 conn(0x7ff8800ceb90 msgr2=0x7ff8800af8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:53.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.688+0000 7ff88499d640 1 -- 172.21.15.105:0/1408444114 shutdown_connections 2023-11-28T18:37:53.688 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.688+0000 7ff88499d640 1 -- 172.21.15.105:0/1408444114 wait complete. 2023-11-28T18:37:53.692 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:37:53.692 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:37:53.703 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:37:53.703 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:37:53.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:37:53.703 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T18:37:53.705 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:31:56.971692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:31:57.881657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:37:53.705 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:37:53.756 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-28T18:37:53.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:37:53.757 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:37:53.757 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:37:53.804 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:37:53.804 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:37:53.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:37:53.805 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T18:37:53.856 INFO:teuthology.orchestra.run.smithi105.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-28T18:37:53.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:37:53.857 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:37:53.857 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:37:53.867 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:37:53.867 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:37:53.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:37:53.867 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T18:37:53.888 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.892+0000 7f9daaa17640 1 Processor -- start 2023-11-28T18:37:53.889 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.892+0000 7f9daaa17640 1 -- start start 2023-11-28T18:37:53.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.892+0000 7f9daaa17640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9da40caf60 0x7f9da40cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.892+0000 7f9daaa17640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da406bf50 con 0x7f9da40ad850 2023-11-28T18:37:53.891 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.892+0000 7f9daaa17640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da415a320 con 0x7f9da4159f40 2023-11-28T18:37:53.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.892+0000 7f9daaa17640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da40cb3c0 con 0x7f9da40caf60 2023-11-28T18:37:53.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.892+0000 7f9da37fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9da40caf60 0x7f9da40cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.892+0000 7f9da8fb4640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9da40ad850 0x7f9da40701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51388/0 (socket says 172.21.15.105:51388) 2023-11-28T18:37:53.892 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.892+0000 7f9da8fb4640 1 -- 172.21.15.105:0/1932232882 learned_addr learned my addr 172.21.15.105:0/1932232882 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:37:53.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.892+0000 7f9da37fe640 1 -- 172.21.15.105:0/1932232882 >> v1:172.21.15.148:6789/0 conn(0x7f9da4159f40 legacy=0x7f9da40ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.892+0000 7f9da37fe640 1 -- 172.21.15.105:0/1932232882 >> v1:172.21.15.105:6789/0 conn(0x7f9da40ad850 legacy=0x7f9da40701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.892+0000 7f9da2ffd640 1 -- 172.21.15.105:0/1932232882 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 989674624 0 0) 0x7f9da406bf50 con 0x7f9da40ad850 2023-11-28T18:37:53.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.892+0000 7f9da37fe640 1 -- 172.21.15.105:0/1932232882 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da4071380 con 0x7f9da40caf60 2023-11-28T18:37:53.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.896+0000 7f9da37fe640 1 --2- 172.21.15.105:0/1932232882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da40caf60 0x7f9da40cc760 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f9d980097c0 tx=0x7f9d9801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2da3de1e0c34dfa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:53.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.896+0000 7f9da2ffd640 1 -- 172.21.15.105:0/1932232882 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d980ac620 con 0x7f9da40caf60 2023-11-28T18:37:53.896 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.896+0000 7f9da2ffd640 1 -- 172.21.15.105:0/1932232882 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d980ac780 con 0x7f9da40caf60 2023-11-28T18:37:53.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.896+0000 7f9da2ffd640 1 -- 172.21.15.105:0/1932232882 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d980acb10 con 0x7f9da40caf60 2023-11-28T18:37:53.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.896+0000 7f9daaa17640 1 -- 172.21.15.105:0/1932232882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da40caf60 msgr2=0x7f9da40cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.896+0000 7f9daaa17640 1 --2- 172.21.15.105:0/1932232882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da40caf60 0x7f9da40cc760 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f9d980097c0 tx=0x7f9d9801c3c0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.896+0000 7f9daaa17640 1 -- 172.21.15.105:0/1932232882 shutdown_connections 2023-11-28T18:37:53.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.896+0000 7f9daaa17640 1 --2- 172.21.15.105:0/1932232882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da40caf60 0x7f9da40cc760 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.896+0000 7f9daaa17640 1 -- 172.21.15.105:0/1932232882 >> 172.21.15.105:0/1932232882 conn(0x7f9da40ceb90 msgr2=0x7f9da40cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:53.898 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.896+0000 7f9daaa17640 1 -- 172.21.15.105:0/1932232882 shutdown_connections 2023-11-28T18:37:53.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.896+0000 7f9daaa17640 1 -- 172.21.15.105:0/1932232882 wait complete. 2023-11-28T18:37:53.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9daaa17640 1 Processor -- start 2023-11-28T18:37:53.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9daaa17640 1 -- start start 2023-11-28T18:37:53.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9daaa17640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da40ad850 0x7f9da40af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.905 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9daaa17640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9da40caf60 0x7f9da415b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.906 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9daaa17640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da4159f40 0x7f9da4152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9daaa17640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da41468c0 con 0x7f9da4159f40 2023-11-28T18:37:53.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9daaa17640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da40b04a0 con 0x7f9da40ad850 2023-11-28T18:37:53.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9daaa17640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da40afc00 con 0x7f9da40caf60 2023-11-28T18:37:53.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9da8fb4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da4159f40 0x7f9da4152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9da8fb4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da4159f40 0x7f9da4152370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54366/0 (socket says 172.21.15.105:54366) 2023-11-28T18:37:53.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9da8fb4640 1 -- 172.21.15.105:0/2625718861 learned_addr learned my addr 172.21.15.105:0/2625718861 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:53.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9da3fff640 1 --2- 172.21.15.105:0/2625718861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da40ad850 0x7f9da40af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9da37fe640 1 --2- 172.21.15.105:0/2625718861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9da40caf60 0x7f9da415b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9da8fb4640 1 -- 172.21.15.105:0/2625718861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9da40caf60 msgr2=0x7f9da415b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9da8fb4640 1 --2- 172.21.15.105:0/2625718861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9da40caf60 0x7f9da415b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9da8fb4640 1 -- 172.21.15.105:0/2625718861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da40ad850 msgr2=0x7f9da40af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:53.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9da8fb4640 1 --2- 172.21.15.105:0/2625718861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da40ad850 0x7f9da40af520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:53.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9da8fb4640 1 -- 172.21.15.105:0/2625718861 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d98009420 con 0x7f9da4159f40 2023-11-28T18:37:53.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9da8fb4640 1 --2- 172.21.15.105:0/2625718861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da4159f40 0x7f9da4152370 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00c700 tx=0x7f9d9c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:53.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9da0ff9640 1 -- 172.21.15.105:0/2625718861 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c00ce10 con 0x7f9da4159f40 2023-11-28T18:37:53.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9da0ff9640 1 -- 172.21.15.105:0/2625718861 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9c00a640 con 0x7f9da4159f40 2023-11-28T18:37:53.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9da0ff9640 1 -- 172.21.15.105:0/2625718861 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c00aa70 con 0x7f9da4159f40 2023-11-28T18:37:53.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9daaa17640 1 -- 172.21.15.105:0/2625718861 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da40afee0 con 0x7f9da4159f40 2023-11-28T18:37:53.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9daaa17640 1 -- 172.21.15.105:0/2625718861 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da4151e50 con 0x7f9da4159f40 2023-11-28T18:37:53.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.900+0000 7f9daaa17640 1 -- 172.21.15.105:0/2625718861 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da4152040 con 0x7f9da4159f40 2023-11-28T18:37:53.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.904+0000 7f9da0ff9640 1 -- 172.21.15.105:0/2625718861 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f9d9c00abd0 con 0x7f9da4159f40 2023-11-28T18:37:53.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.904+0000 7f9da0ff9640 1 --2- 172.21.15.105:0/2625718861 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d900668d0 0x7f9d90068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:53.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.904+0000 7f9da0ff9640 1 -- 172.21.15.105:0/2625718861 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f9d9c083a30 con 0x7f9da4159f40 2023-11-28T18:37:53.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.904+0000 7f9da3fff640 1 --2- 172.21.15.105:0/2625718861 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d900668d0 0x7f9d90068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:53.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.904+0000 7f9da3fff640 1 --2- 172.21.15.105:0/2625718861 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d900668d0 0x7f9d90068d90 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9d94007fd0 tx=0x7f9d94009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:53.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:53.908+0000 7f9da0ff9640 1 -- 172.21.15.105:0/2625718861 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9d9c050b70 con 0x7f9da4159f40 2023-11-28T18:37:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-28T18:37:53.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:37:53.929 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:37:54.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.060+0000 7f9daaa17640 1 -- 172.21.15.105:0/2625718861 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9da4067570 con 0x7f9da4159f40 2023-11-28T18:37:54.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.060+0000 7f9da0ff9640 1 -- 172.21.15.105:0/2625718861 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9d9c018070 con 0x7f9da4159f40 2023-11-28T18:37:54.060 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:37:54.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.064+0000 7f9daaa17640 1 -- 172.21.15.105:0/2625718861 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d900668d0 msgr2=0x7f9d90068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.062 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.064+0000 7f9daaa17640 1 --2- 172.21.15.105:0/2625718861 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d900668d0 0x7f9d90068d90 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9d94007fd0 tx=0x7f9d94009040 comp rx=0 tx=0).stop 2023-11-28T18:37:54.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.064+0000 7f9daaa17640 1 -- 172.21.15.105:0/2625718861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da4159f40 msgr2=0x7f9da4152370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.064+0000 7f9daaa17640 1 --2- 172.21.15.105:0/2625718861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da4159f40 0x7f9da4152370 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00c700 tx=0x7f9d9c00cbc0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.064+0000 7f9daaa17640 1 -- 172.21.15.105:0/2625718861 shutdown_connections 2023-11-28T18:37:54.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.064+0000 7f9daaa17640 1 --2- 172.21.15.105:0/2625718861 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d900668d0 0x7f9d90068d90 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.064+0000 7f9daaa17640 1 --2- 172.21.15.105:0/2625718861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da4159f40 0x7f9da4152370 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.064+0000 7f9daaa17640 1 --2- 172.21.15.105:0/2625718861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9da40caf60 0x7f9da415b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.064+0000 7f9daaa17640 1 --2- 172.21.15.105:0/2625718861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da40ad850 0x7f9da40af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.064+0000 7f9daaa17640 1 -- 172.21.15.105:0/2625718861 >> 172.21.15.105:0/2625718861 conn(0x7f9da40ceb90 msgr2=0x7f9da4156b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:54.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.064+0000 7f9daaa17640 1 -- 172.21.15.105:0/2625718861 shutdown_connections 2023-11-28T18:37:54.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.064+0000 7f9daaa17640 1 -- 172.21.15.105:0/2625718861 wait complete. 2023-11-28T18:37:54.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.072+0000 7f7d78054640 1 Processor -- start 2023-11-28T18:37:54.070 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.072+0000 7f7d78054640 1 -- start start 2023-11-28T18:37:54.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.072+0000 7f7d78054640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7d700d3c30 0x7f7d700d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.072+0000 7f7d78054640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d700b25b0 con 0x7f7d700b2d20 2023-11-28T18:37:54.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.072+0000 7f7d78054640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d700cea30 con 0x7f7d7015d4f0 2023-11-28T18:37:54.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.072+0000 7f7d78054640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d700b2900 con 0x7f7d700d3c30 2023-11-28T18:37:54.072 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.076+0000 7f7d755ef640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7d700d3c30 0x7f7d700d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.076+0000 7f7d765f1640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7d700b2d20 0x7f7d700b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51396/0 (socket says 172.21.15.105:51396) 2023-11-28T18:37:54.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.076+0000 7f7d765f1640 1 -- 172.21.15.105:0/456224775 learned_addr learned my addr 172.21.15.105:0/456224775 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:37:54.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.076+0000 7f7d755ef640 1 -- 172.21.15.105:0/456224775 >> v1:172.21.15.148:6789/0 conn(0x7f7d7015d4f0 legacy=0x7f7d700d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.076+0000 7f7d755ef640 1 -- 172.21.15.105:0/456224775 >> v1:172.21.15.105:6789/0 conn(0x7f7d700b2d20 legacy=0x7f7d700b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.073 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.076+0000 7f7d755ef640 1 -- 172.21.15.105:0/456224775 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d700ced10 con 0x7f7d700d3c30 2023-11-28T18:37:54.075 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.076+0000 7f7d755ef640 1 --2- 172.21.15.105:0/456224775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d700d3c30 0x7f7d700d1520 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f7d600097c0 tx=0x7f7d6001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a5a1d62db4a61c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:54.076 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.076+0000 7f7d74dee640 1 -- 172.21.15.105:0/456224775 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d600ad660 con 0x7f7d700d3c30 2023-11-28T18:37:54.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.076+0000 7f7d74dee640 1 -- 172.21.15.105:0/456224775 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d600ad7c0 con 0x7f7d700d3c30 2023-11-28T18:37:54.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.076+0000 7f7d74dee640 1 -- 172.21.15.105:0/456224775 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d600adb50 con 0x7f7d700d3c30 2023-11-28T18:37:54.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.080+0000 7f7d78054640 1 -- 172.21.15.105:0/456224775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d700d3c30 msgr2=0x7f7d700d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.080+0000 7f7d78054640 1 --2- 172.21.15.105:0/456224775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d700d3c30 0x7f7d700d1520 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f7d600097c0 tx=0x7f7d6001c3c0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.080+0000 7f7d78054640 1 -- 172.21.15.105:0/456224775 shutdown_connections 2023-11-28T18:37:54.077 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.080+0000 7f7d78054640 1 --2- 172.21.15.105:0/456224775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d700d3c30 0x7f7d700d1520 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.080+0000 7f7d78054640 1 -- 172.21.15.105:0/456224775 >> 172.21.15.105:0/456224775 conn(0x7f7d700d3030 msgr2=0x7f7d700d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:54.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.080+0000 7f7d78054640 1 -- 172.21.15.105:0/456224775 shutdown_connections 2023-11-28T18:37:54.078 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.080+0000 7f7d78054640 1 -- 172.21.15.105:0/456224775 wait complete. 2023-11-28T18:37:54.079 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.080+0000 7f7d78054640 1 Processor -- start 2023-11-28T18:37:54.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.080+0000 7f7d78054640 1 -- start start 2023-11-28T18:37:54.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.080+0000 7f7d78054640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d700b2d20 0x7f7d7014c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.080+0000 7f7d78054640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d700d3c30 0x7f7d700b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.080+0000 7f7d78054640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d7015d4f0 0x7f7d700b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.080+0000 7f7d78054640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d700d0a20 con 0x7f7d700d3c30 2023-11-28T18:37:54.080 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.080+0000 7f7d78054640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7014d770 con 0x7f7d700b2d20 2023-11-28T18:37:54.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.080+0000 7f7d78054640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d70151a40 con 0x7f7d7015d4f0 2023-11-28T18:37:54.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.080+0000 7f7d755ef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d700d3c30 0x7f7d700b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.080+0000 7f7d755ef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d700d3c30 0x7f7d700b0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54394/0 (socket says 172.21.15.105:54394) 2023-11-28T18:37:54.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.080+0000 7f7d755ef640 1 -- 172.21.15.105:0/2480985440 learned_addr learned my addr 172.21.15.105:0/2480985440 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:54.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.084+0000 7f7d765f1640 1 --2- 172.21.15.105:0/2480985440 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d7015d4f0 0x7f7d700b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.081 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.080+0000 7f7d75df0640 1 --2- 172.21.15.105:0/2480985440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d700b2d20 0x7f7d7014c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.084+0000 7f7d755ef640 1 -- 172.21.15.105:0/2480985440 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d7015d4f0 msgr2=0x7f7d700b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.083 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.084+0000 7f7d755ef640 1 --2- 172.21.15.105:0/2480985440 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d7015d4f0 0x7f7d700b0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.084+0000 7f7d755ef640 1 -- 172.21.15.105:0/2480985440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d700b2d20 msgr2=0x7f7d7014c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.084+0000 7f7d755ef640 1 --2- 172.21.15.105:0/2480985440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d700b2d20 0x7f7d7014c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.084+0000 7f7d755ef640 1 -- 172.21.15.105:0/2480985440 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d60009420 con 0x7f7d700d3c30 2023-11-28T18:37:54.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.084+0000 7f7d75df0640 1 --2- 172.21.15.105:0/2480985440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d700b2d20 0x7f7d7014c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:37:54.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.084+0000 7f7d765f1640 1 --2- 172.21.15.105:0/2480985440 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d7015d4f0 0x7f7d700b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:37:54.084 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.084+0000 7f7d755ef640 1 --2- 172.21.15.105:0/2480985440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d700d3c30 0x7f7d700b0640 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f7d600b0040 tx=0x7f7d60019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:54.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.084+0000 7f7d667fc640 1 -- 172.21.15.105:0/2480985440 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d600b6610 con 0x7f7d700d3c30 2023-11-28T18:37:54.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.084+0000 7f7d667fc640 1 -- 172.21.15.105:0/2480985440 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d600b67b0 con 0x7f7d700d3c30 2023-11-28T18:37:54.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.084+0000 7f7d78054640 1 -- 172.21.15.105:0/2480985440 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7014ce40 con 0x7f7d700d3c30 2023-11-28T18:37:54.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.084+0000 7f7d667fc640 1 -- 172.21.15.105:0/2480985440 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d600b6b60 con 0x7f7d700d3c30 2023-11-28T18:37:54.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.084+0000 7f7d78054640 1 -- 172.21.15.105:0/2480985440 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7014d400 con 0x7f7d700d3c30 2023-11-28T18:37:54.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.084+0000 7f7d78054640 1 -- 172.21.15.105:0/2480985440 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d7014d8e0 con 0x7f7d700d3c30 2023-11-28T18:37:54.085 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.084+0000 7f7d667fc640 1 -- 172.21.15.105:0/2480985440 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f7d600b4040 con 0x7f7d700d3c30 2023-11-28T18:37:54.087 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:37:54.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.084+0000 7f7d667fc640 1 --2- 172.21.15.105:0/2480985440 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7d540668d0 0x7f7d54068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.084+0000 7f7d667fc640 1 -- 172.21.15.105:0/2480985440 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f7d6011ea70 con 0x7f7d700d3c30 2023-11-28T18:37:54.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.088+0000 7f7d75df0640 1 --2- 172.21.15.105:0/2480985440 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7d540668d0 0x7f7d54068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.088+0000 7f7d75df0640 1 --2- 172.21.15.105:0/2480985440 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7d540668d0 0x7f7d54068d90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c000f70 tx=0x7f7d6c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:54.088 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.088+0000 7f7d667fc640 1 -- 172.21.15.105:0/2480985440 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7d600ebb10 con 0x7f7d700d3c30 2023-11-28T18:37:54.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.232+0000 7f9d7379d640 1 Processor -- start 2023-11-28T18:37:54.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.232+0000 7f9d7379d640 1 -- start start 2023-11-28T18:37:54.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.232+0000 7f9d7379d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9d6c15d4f0 0x7f9d6c0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.232+0000 7f9d7379d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d6c0d3c30 0x7f9d6c0d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.232+0000 7f9d7379d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d6c0b25b0 con 0x7f9d6c0d1890 2023-11-28T18:37:54.236 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.232+0000 7f9d7379d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d6c0b2900 con 0x7f9d6c15d4f0 2023-11-28T18:37:54.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.232+0000 7f9d7379d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c0cea30 con 0x7f9d6c0d3c30 2023-11-28T18:37:54.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.232+0000 7f9d71539640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9d6c15d4f0 0x7f9d6c0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.232+0000 7f9d71539640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9d6c15d4f0 0x7f9d6c0d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54402/0 (socket says 172.21.15.105:54402) 2023-11-28T18:37:54.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.232+0000 7f9d71539640 1 -- 172.21.15.105:0/3749889883 learned_addr learned my addr 172.21.15.105:0/3749889883 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:54.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.236+0000 7f9d70d38640 1 --2- 172.21.15.105:0/3749889883 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d6c0d3c30 0x7f9d6c0d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.236+0000 7f9d71539640 1 -- 172.21.15.105:0/3749889883 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d6c0d3c30 msgr2=0x7f9d6c0d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:54.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.236+0000 7f9d71539640 1 --2- 172.21.15.105:0/3749889883 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d6c0d3c30 0x7f9d6c0d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.237 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.236+0000 7f9d71539640 1 -- 172.21.15.105:0/3749889883 >> v1:172.21.15.148:6789/0 conn(0x7f9d6c0d1890 legacy=0x7f9d6c0cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:54.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.236+0000 7f9d71539640 1 -- 172.21.15.105:0/3749889883 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c0ced10 con 0x7f9d6c15d4f0 2023-11-28T18:37:54.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.236+0000 7f9d70d38640 1 --2- 172.21.15.105:0/3749889883 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d6c0d3c30 0x7f9d6c0d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:37:54.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.236+0000 7f9d71539640 1 --2- 172.21.15.105:0/3749889883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d6c15d4f0 0x7f9d6c0d0e10 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009870 tx=0x7f9d6801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f03b2185b34660c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:54.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.236+0000 7f9d63fff640 1 -- 172.21.15.105:0/3749889883 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d680ac600 con 0x7f9d6c15d4f0 2023-11-28T18:37:54.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.236+0000 7f9d63fff640 1 -- 172.21.15.105:0/3749889883 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d680a8050 con 0x7f9d6c15d4f0 2023-11-28T18:37:54.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.236+0000 7f9d63fff640 1 -- 172.21.15.105:0/3749889883 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d680ac980 con 0x7f9d6c15d4f0 2023-11-28T18:37:54.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.236+0000 7f9d7379d640 1 -- 172.21.15.105:0/3749889883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d6c15d4f0 msgr2=0x7f9d6c0d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.236+0000 7f9d7379d640 1 --2- 172.21.15.105:0/3749889883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d6c15d4f0 0x7f9d6c0d0e10 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009870 tx=0x7f9d6801c440 comp rx=0 tx=0).stop 2023-11-28T18:37:54.238 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d7379d640 1 -- 172.21.15.105:0/3749889883 shutdown_connections 2023-11-28T18:37:54.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d7379d640 1 --2- 172.21.15.105:0/3749889883 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d6c0d3c30 0x7f9d6c0d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d7379d640 1 --2- 172.21.15.105:0/3749889883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d6c15d4f0 0x7f9d6c0d0e10 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d7379d640 1 -- 172.21.15.105:0/3749889883 >> 172.21.15.105:0/3749889883 conn(0x7f9d6c0d3030 msgr2=0x7f9d6c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:54.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d7379d640 1 -- 172.21.15.105:0/3749889883 shutdown_connections 2023-11-28T18:37:54.239 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d7379d640 1 -- 172.21.15.105:0/3749889883 wait complete. 2023-11-28T18:37:54.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d7379d640 1 Processor -- start 2023-11-28T18:37:54.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d7379d640 1 -- start start 2023-11-28T18:37:54.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d7379d640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d6c0d1890 0x7f9d6c153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d7379d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d6c0d3c30 0x7f9d6c153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d7379d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d6c15d4f0 0x7f9d6c0ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d7379d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c0caf80 con 0x7f9d6c0d3c30 2023-11-28T18:37:54.242 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d7379d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c154210 con 0x7f9d6c15d4f0 2023-11-28T18:37:54.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d7379d640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c154fc0 con 0x7f9d6c0d1890 2023-11-28T18:37:54.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d70d38640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d6c0d3c30 0x7f9d6c153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d70d38640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d6c0d3c30 0x7f9d6c153cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54408/0 (socket says 172.21.15.105:54408) 2023-11-28T18:37:54.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d70d38640 1 -- 172.21.15.105:0/2470519513 learned_addr learned my addr 172.21.15.105:0/2470519513 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:54.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d71539640 1 --2- 172.21.15.105:0/2470519513 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d6c0d1890 0x7f9d6c153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d71d3a640 1 --2- 172.21.15.105:0/2470519513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d6c15d4f0 0x7f9d6c0ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d70d38640 1 -- 172.21.15.105:0/2470519513 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d6c0d1890 msgr2=0x7f9d6c153650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d70d38640 1 --2- 172.21.15.105:0/2470519513 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d6c0d1890 0x7f9d6c153650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d70d38640 1 -- 172.21.15.105:0/2470519513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d6c15d4f0 msgr2=0x7f9d6c0ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d70d38640 1 --2- 172.21.15.105:0/2470519513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d6c15d4f0 0x7f9d6c0ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.243 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d70d38640 1 -- 172.21.15.105:0/2470519513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d680094d0 con 0x7f9d6c0d3c30 2023-11-28T18:37:54.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d70d38640 1 --2- 172.21.15.105:0/2470519513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d6c0d3c30 0x7f9d6c153cd0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00c7f0 tx=0x7f9d5c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:54.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d61ffb640 1 -- 172.21.15.105:0/2470519513 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c007820 con 0x7f9d6c0d3c30 2023-11-28T18:37:54.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d61ffb640 1 -- 172.21.15.105:0/2470519513 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5c007980 con 0x7f9d6c0d3c30 2023-11-28T18:37:54.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d7379d640 1 -- 172.21.15.105:0/2470519513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6c154440 con 0x7f9d6c0d3c30 2023-11-28T18:37:54.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.240+0000 7f9d61ffb640 1 -- 172.21.15.105:0/2470519513 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c007d30 con 0x7f9d6c0d3c30 2023-11-28T18:37:54.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.244+0000 7f9d7379d640 1 -- 172.21.15.105:0/2470519513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6c154a00 con 0x7f9d6c0d3c30 2023-11-28T18:37:54.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.244+0000 7f9d7379d640 1 -- 172.21.15.105:0/2470519513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d6c15a3c0 con 0x7f9d6c0d3c30 2023-11-28T18:37:54.244 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.244+0000 7f9d61ffb640 1 -- 172.21.15.105:0/2470519513 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f9d5c0113f0 con 0x7f9d6c0d3c30 2023-11-28T18:37:54.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.244+0000 7f9d61ffb640 1 --2- 172.21.15.105:0/2470519513 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d500668d0 0x7f9d50068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.244+0000 7f9d61ffb640 1 -- 172.21.15.105:0/2470519513 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f9d5c084940 con 0x7f9d6c0d3c30 2023-11-28T18:37:54.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.244+0000 7f9d71539640 1 --2- 172.21.15.105:0/2470519513 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d500668d0 0x7f9d50068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.245 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.244+0000 7f9d71539640 1 --2- 172.21.15.105:0/2470519513 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d500668d0 0x7f9d50068d90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9d680ae000 tx=0x7f9d680a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:54.246 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.248+0000 7f9d61ffb640 1 -- 172.21.15.105:0/2470519513 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9d6c15a3c0 con 0x7f9d6c0d3c30 2023-11-28T18:37:54.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.276+0000 7f7d78054640 1 -- 172.21.15.105:0/2480985440 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d70067570 con 0x7f7d700d3c30 2023-11-28T18:37:54.274 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.276+0000 7f7d667fc640 1 -- 172.21.15.105:0/2480985440 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 151 v151) v1 ==== 95+0+4219 (secure 0 0 0) 0x7f7d600ef130 con 0x7f7d700d3c30 2023-11-28T18:37:54.275 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:37:54.275 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":151,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":151,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:31:58.827408+0000","modified":"2023-11-28T18:36:11.513093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9126,"mds_1":9117,"mds_2":9120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9117":{"gid":9117,"name":"a","rank":1,"incarnation":142,"state":"up:active","state_seq":21,"addr":"172.21.15.105:6835/3161918450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3161918450},{"type":"v1","addr":"172.21.15.105:6835","nonce":3161918450}]},"join_fscid":-1,"laggy_since":"2023-11-28T18:33:18.186653+0000","export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9120":{"gid":9120,"name":"d","rank":2,"incarnation":138,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6834/3958106803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3958106803},{"type":"v1","addr":"172.21.15.148:6834","nonce":3958106803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9126":{"gid":9126,"name":"c","rank":0,"incarnation":130,"state":"up:active","state_seq":66,"addr":"172.21.15.105:6837/1978722810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1978722810},{"type":"v1","addr":"172.21.15.105:6837","nonce":1978722810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-28T18:37:54.275 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 151 2023-11-28T18:37:54.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.280+0000 7f7d78054640 1 -- 172.21.15.105:0/2480985440 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7d540668d0 msgr2=0x7f7d54068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.280+0000 7f7d78054640 1 --2- 172.21.15.105:0/2480985440 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7d540668d0 0x7f7d54068d90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c000f70 tx=0x7f7d6c009040 comp rx=0 tx=0).stop 2023-11-28T18:37:54.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.280+0000 7f7d78054640 1 -- 172.21.15.105:0/2480985440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d700d3c30 msgr2=0x7f7d700b0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.280+0000 7f7d78054640 1 --2- 172.21.15.105:0/2480985440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d700d3c30 0x7f7d700b0640 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f7d600b0040 tx=0x7f7d60019160 comp rx=0 tx=0).stop 2023-11-28T18:37:54.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.280+0000 7f7d78054640 1 -- 172.21.15.105:0/2480985440 shutdown_connections 2023-11-28T18:37:54.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.280+0000 7f7d78054640 1 --2- 172.21.15.105:0/2480985440 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d7015d4f0 0x7f7d700b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.280+0000 7f7d78054640 1 --2- 172.21.15.105:0/2480985440 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f7d540668d0 0x7f7d54068d90 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.280+0000 7f7d78054640 1 --2- 172.21.15.105:0/2480985440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d700d3c30 0x7f7d700b0640 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.280+0000 7f7d78054640 1 --2- 172.21.15.105:0/2480985440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d700b2d20 0x7f7d7014c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.280+0000 7f7d78054640 1 -- 172.21.15.105:0/2480985440 >> 172.21.15.105:0/2480985440 conn(0x7f7d700d3030 msgr2=0x7f7d700cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:54.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.280+0000 7f7d78054640 1 -- 172.21.15.105:0/2480985440 shutdown_connections 2023-11-28T18:37:54.278 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.280+0000 7f7d78054640 1 -- 172.21.15.105:0/2480985440 wait complete. 2023-11-28T18:37:54.296 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:37:54.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.404+0000 7f9d7379d640 1 -- 172.21.15.105:0/2470519513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9d6c15a3c0 con 0x7f9d6c0d3c30 2023-11-28T18:37:54.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.404+0000 7f9d61ffb640 1 -- 172.21.15.105:0/2470519513 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v130) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9d6c15a3c0 con 0x7f9d6c0d3c30 2023-11-28T18:37:54.403 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:37:54.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.408+0000 7f9d7379d640 1 -- 172.21.15.105:0/2470519513 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d500668d0 msgr2=0x7f9d50068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.408+0000 7f9d7379d640 1 --2- 172.21.15.105:0/2470519513 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d500668d0 0x7f9d50068d90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9d680ae000 tx=0x7f9d680a7000 comp rx=0 tx=0).stop 2023-11-28T18:37:54.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.408+0000 7f9d7379d640 1 -- 172.21.15.105:0/2470519513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d6c0d3c30 msgr2=0x7f9d6c153cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.408+0000 7f9d7379d640 1 --2- 172.21.15.105:0/2470519513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d6c0d3c30 0x7f9d6c153cd0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00c7f0 tx=0x7f9d5c00ccb0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.408+0000 7f9d7379d640 1 -- 172.21.15.105:0/2470519513 shutdown_connections 2023-11-28T18:37:54.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.408+0000 7f9d7379d640 1 --2- 172.21.15.105:0/2470519513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d6c15d4f0 0x7f9d6c0ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.408+0000 7f9d7379d640 1 --2- 172.21.15.105:0/2470519513 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f9d500668d0 0x7f9d50068d90 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.408+0000 7f9d7379d640 1 --2- 172.21.15.105:0/2470519513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d6c0d3c30 0x7f9d6c153cd0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.408+0000 7f9d7379d640 1 --2- 172.21.15.105:0/2470519513 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d6c0d1890 0x7f9d6c153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.408+0000 7f9d7379d640 1 -- 172.21.15.105:0/2470519513 >> 172.21.15.105:0/2470519513 conn(0x7f9d6c0d3030 msgr2=0x7f9d6c0d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:54.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.408+0000 7f9d7379d640 1 -- 172.21.15.105:0/2470519513 shutdown_connections 2023-11-28T18:37:54.406 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.408+0000 7f9d7379d640 1 -- 172.21.15.105:0/2470519513 wait complete. 2023-11-28T18:37:54.424 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:37:54.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.444+0000 7f5af265a640 1 Processor -- start 2023-11-28T18:37:54.440 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.444+0000 7f5af265a640 1 -- start start 2023-11-28T18:37:54.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.444+0000 7f5af265a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5aec0b3d30 0x7f5aec1680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.444+0000 7f5af265a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aec0b25b0 con 0x7f5aec0d3c30 2023-11-28T18:37:54.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.444+0000 7f5af265a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aec0d1c30 con 0x7f5aec15d4f0 2023-11-28T18:37:54.441 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.444+0000 7f5af265a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aec0b2900 con 0x7f5aec0b3d30 2023-11-28T18:37:54.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.444+0000 7f5aeb7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5aec0d3c30 0x7f5aec0d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51406/0 (socket says 172.21.15.105:51406) 2023-11-28T18:37:54.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.444+0000 7f5aeb7fe640 1 -- 172.21.15.105:0/2833645194 learned_addr learned my addr 172.21.15.105:0/2833645194 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:37:54.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.444+0000 7f5af0bf7640 1 --2- 172.21.15.105:0/2833645194 >> v2:172.21.15.148:3300/0 conn(0x7f5aec0b3d30 0x7f5aec1680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.444+0000 7f5aeaffd640 1 -- 172.21.15.105:0/2833645194 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3682235677 0 0) 0x7f5aec0b25b0 con 0x7f5aec0d3c30 2023-11-28T18:37:54.443 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.444+0000 7f5aeaffd640 1 -- 172.21.15.105:0/2833645194 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ad4003530 con 0x7f5aec0d3c30 2023-11-28T18:37:54.443 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.444+0000 7f5af0bf7640 1 -- 172.21.15.105:0/2833645194 >> v1:172.21.15.148:6789/0 conn(0x7f5aec15d4f0 legacy=0x7f5aec0d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.443 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.444+0000 7f5af0bf7640 1 -- 172.21.15.105:0/2833645194 >> v1:172.21.15.105:6789/0 conn(0x7f5aec0d3c30 legacy=0x7f5aec0d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.443 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.444+0000 7f5af0bf7640 1 -- 172.21.15.105:0/2833645194 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec0cea30 con 0x7f5aec0b3d30 2023-11-28T18:37:54.446 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.448+0000 7f5af0bf7640 1 --2- 172.21.15.105:0/2833645194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aec0b3d30 0x7f5aec1680a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009d90 tx=0x7f5ae401c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebf5100b1c100859 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:54.446 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.448+0000 7f5aeaffd640 1 -- 172.21.15.105:0/2833645194 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae40a58c0 con 0x7f5aec0b3d30 2023-11-28T18:37:54.446 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.448+0000 7f5aeaffd640 1 -- 172.21.15.105:0/2833645194 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae40a5a20 con 0x7f5aec0b3d30 2023-11-28T18:37:54.446 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.448+0000 7f5aeaffd640 1 -- 172.21.15.105:0/2833645194 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae40a5db0 con 0x7f5aec0b3d30 2023-11-28T18:37:54.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.448+0000 7f5af265a640 1 -- 172.21.15.105:0/2833645194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aec0b3d30 msgr2=0x7f5aec1680a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.448+0000 7f5af265a640 1 --2- 172.21.15.105:0/2833645194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aec0b3d30 0x7f5aec1680a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009d90 tx=0x7f5ae401c800 comp rx=0 tx=0).stop 2023-11-28T18:37:54.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.448+0000 7f5af265a640 1 -- 172.21.15.105:0/2833645194 shutdown_connections 2023-11-28T18:37:54.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.448+0000 7f5af265a640 1 --2- 172.21.15.105:0/2833645194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aec0b3d30 0x7f5aec1680a0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.448+0000 7f5af265a640 1 -- 172.21.15.105:0/2833645194 >> 172.21.15.105:0/2833645194 conn(0x7f5aec0d3030 msgr2=0x7f5aec0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:54.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.448+0000 7f5af265a640 1 -- 172.21.15.105:0/2833645194 shutdown_connections 2023-11-28T18:37:54.447 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.448+0000 7f5af265a640 1 -- 172.21.15.105:0/2833645194 wait complete. 2023-11-28T18:37:54.448 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5af265a640 1 Processor -- start 2023-11-28T18:37:54.449 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5af265a640 1 -- start start 2023-11-28T18:37:54.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5af265a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5aec0b3d30 0x7f5aec0b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5af265a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aec0d3c30 0x7f5aec0cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5af265a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aec15d4f0 0x7f5aec153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5af265a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec0d0a20 con 0x7f5aec15d4f0 2023-11-28T18:37:54.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5af265a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec0b5cd0 con 0x7f5aec0d3c30 2023-11-28T18:37:54.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5af265a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec0b5f70 con 0x7f5aec0b3d30 2023-11-28T18:37:54.451 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5af0bf7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aec15d4f0 0x7f5aec153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5af0bf7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aec15d4f0 0x7f5aec153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54424/0 (socket says 172.21.15.105:54424) 2023-11-28T18:37:54.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5af0bf7640 1 -- 172.21.15.105:0/1667212142 learned_addr learned my addr 172.21.15.105:0/1667212142 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:54.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5aebfff640 1 --2- 172.21.15.105:0/1667212142 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5aec0b3d30 0x7f5aec0b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5aeb7fe640 1 --2- 172.21.15.105:0/1667212142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aec0d3c30 0x7f5aec0cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5af0bf7640 1 -- 172.21.15.105:0/1667212142 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5aec0b3d30 msgr2=0x7f5aec0b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5af0bf7640 1 --2- 172.21.15.105:0/1667212142 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5aec0b3d30 0x7f5aec0b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5af0bf7640 1 -- 172.21.15.105:0/1667212142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aec0d3c30 msgr2=0x7f5aec0cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5af0bf7640 1 --2- 172.21.15.105:0/1667212142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aec0d3c30 0x7f5aec0cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5af0bf7640 1 -- 172.21.15.105:0/1667212142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae4009a70 con 0x7f5aec15d4f0 2023-11-28T18:37:54.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5af0bf7640 1 --2- 172.21.15.105:0/1667212142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aec15d4f0 0x7f5aec153300 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40b0040 tx=0x7f5ae40195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:54.452 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5ae8ff9640 1 -- 172.21.15.105:0/1667212142 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae40b9860 con 0x7f5aec15d4f0 2023-11-28T18:37:54.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5ae8ff9640 1 -- 172.21.15.105:0/1667212142 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae40b99c0 con 0x7f5aec15d4f0 2023-11-28T18:37:54.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5ae8ff9640 1 -- 172.21.15.105:0/1667212142 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae40b9d50 con 0x7f5aec15d4f0 2023-11-28T18:37:54.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5af265a640 1 -- 172.21.15.105:0/1667212142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aec0b6250 con 0x7f5aec15d4f0 2023-11-28T18:37:54.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.452+0000 7f5af265a640 1 -- 172.21.15.105:0/1667212142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aec15c970 con 0x7f5aec15d4f0 2023-11-28T18:37:54.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.456+0000 7f5af265a640 1 -- 172.21.15.105:0/1667212142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aec067570 con 0x7f5aec15d4f0 2023-11-28T18:37:54.453 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.456+0000 7f5ae8ff9640 1 -- 172.21.15.105:0/1667212142 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f5ae40ad440 con 0x7f5aec15d4f0 2023-11-28T18:37:54.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.456+0000 7f5ae8ff9640 1 --2- 172.21.15.105:0/1667212142 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5ad80668d0 0x7f5ad8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.456+0000 7f5ae8ff9640 1 -- 172.21.15.105:0/1667212142 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f5ae411f950 con 0x7f5aec15d4f0 2023-11-28T18:37:54.454 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.456+0000 7f5aebfff640 1 --2- 172.21.15.105:0/1667212142 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5ad80668d0 0x7f5ad8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.455 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.456+0000 7f5aebfff640 1 --2- 172.21.15.105:0/1667212142 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5ad80668d0 0x7f5ad8068d90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5adc000f70 tx=0x7f5adc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:54.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.460+0000 7f5ae8ff9640 1 -- 172.21.15.105:0/1667212142 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5ae40ec950 con 0x7f5aec15d4f0 2023-11-28T18:37:54.570 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.572+0000 7f13c8baa640 1 Processor -- start 2023-11-28T18:37:54.571 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.572+0000 7f13c8baa640 1 -- start start 2023-11-28T18:37:54.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.572+0000 7f13c8baa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f13c415d4f0 0x7f13c40d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.576+0000 7f13c8baa640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c40b25b0 con 0x7f13c40b2d20 2023-11-28T18:37:54.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.576+0000 7f13c8baa640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c40cea30 con 0x7f13c40d3c30 2023-11-28T18:37:54.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.576+0000 7f13c8baa640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13c40b2900 con 0x7f13c415d4f0 2023-11-28T18:37:54.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.576+0000 7f13c2d9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f13c40b2d20 0x7f13c40b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51408/0 (socket says 172.21.15.105:51408) 2023-11-28T18:37:54.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.576+0000 7f13c2d9d640 1 -- 172.21.15.105:0/3300238257 learned_addr learned my addr 172.21.15.105:0/3300238257 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:37:54.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.576+0000 7f13c259c640 1 --2- 172.21.15.105:0/3300238257 >> v2:172.21.15.148:3300/0 conn(0x7f13c415d4f0 0x7f13c40d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.576+0000 7f13c159a640 1 -- 172.21.15.105:0/3300238257 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2157036454 0 0) 0x7f13c40b25b0 con 0x7f13c40b2d20 2023-11-28T18:37:54.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.576+0000 7f13c159a640 1 -- 172.21.15.105:0/3300238257 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13ac003530 con 0x7f13c40b2d20 2023-11-28T18:37:54.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.576+0000 7f13c259c640 1 -- 172.21.15.105:0/3300238257 >> v1:172.21.15.148:6789/0 conn(0x7f13c40d3c30 legacy=0x7f13c40d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.576+0000 7f13c259c640 1 -- 172.21.15.105:0/3300238257 >> v1:172.21.15.105:6789/0 conn(0x7f13c40b2d20 legacy=0x7f13c40b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.576+0000 7f13c259c640 1 -- 172.21.15.105:0/3300238257 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c40ced10 con 0x7f13c415d4f0 2023-11-28T18:37:54.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.576+0000 7f13c259c640 1 --2- 172.21.15.105:0/3300238257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13c415d4f0 0x7f13c40d0e10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f13b40097c0 tx=0x7f13b401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb54f3759f27960d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:54.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.580+0000 7f13c159a640 1 -- 172.21.15.105:0/3300238257 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b40ad660 con 0x7f13c415d4f0 2023-11-28T18:37:54.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.580+0000 7f13c159a640 1 -- 172.21.15.105:0/3300238257 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b40ad7c0 con 0x7f13c415d4f0 2023-11-28T18:37:54.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.580+0000 7f13c159a640 1 -- 172.21.15.105:0/3300238257 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b40adb50 con 0x7f13c415d4f0 2023-11-28T18:37:54.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.580+0000 7f13c8baa640 1 -- 172.21.15.105:0/3300238257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13c415d4f0 msgr2=0x7f13c40d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.580+0000 7f13c8baa640 1 --2- 172.21.15.105:0/3300238257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13c415d4f0 0x7f13c40d0e10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f13b40097c0 tx=0x7f13b401c3c0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.580+0000 7f13c8baa640 1 -- 172.21.15.105:0/3300238257 shutdown_connections 2023-11-28T18:37:54.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.580+0000 7f13c8baa640 1 --2- 172.21.15.105:0/3300238257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13c415d4f0 0x7f13c40d0e10 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.580+0000 7f13c8baa640 1 -- 172.21.15.105:0/3300238257 >> 172.21.15.105:0/3300238257 conn(0x7f13c40d3030 msgr2=0x7f13c40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:54.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.580+0000 7f13c8baa640 1 -- 172.21.15.105:0/3300238257 shutdown_connections 2023-11-28T18:37:54.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.580+0000 7f13c8baa640 1 -- 172.21.15.105:0/3300238257 wait complete. 2023-11-28T18:37:54.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.580+0000 7f13c8baa640 1 Processor -- start 2023-11-28T18:37:54.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.580+0000 7f13c8baa640 1 -- start start 2023-11-28T18:37:54.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.580+0000 7f13c8baa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13c40b2d20 0x7f13c41510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.580+0000 7f13c8baa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c40d3c30 0x7f13c4160670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.580+0000 7f13c8baa640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f13c415d4f0 0x7f13c4155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.580+0000 7f13c8baa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c40d0a20 con 0x7f13c40d3c30 2023-11-28T18:37:54.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.580+0000 7f13c8baa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c40cca90 con 0x7f13c40b2d20 2023-11-28T18:37:54.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.580+0000 7f13c8baa640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c41598c0 con 0x7f13c415d4f0 2023-11-28T18:37:54.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.580+0000 7f13c1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c40d3c30 0x7f13c4160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.580+0000 7f13c1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c40d3c30 0x7f13c4160670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54430/0 (socket says 172.21.15.105:54430) 2023-11-28T18:37:54.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.584+0000 7f13c1d9b640 1 -- 172.21.15.105:0/2131021000 learned_addr learned my addr 172.21.15.105:0/2131021000 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:54.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.584+0000 7f13c259c640 1 --2- 172.21.15.105:0/2131021000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13c40b2d20 0x7f13c41510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.584+0000 7f13c2d9d640 1 --2- 172.21.15.105:0/2131021000 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f13c415d4f0 0x7f13c4155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.584+0000 7f13c1d9b640 1 -- 172.21.15.105:0/2131021000 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f13c415d4f0 msgr2=0x7f13c4155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.584+0000 7f13c1d9b640 1 --2- 172.21.15.105:0/2131021000 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f13c415d4f0 0x7f13c4155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.584+0000 7f13c1d9b640 1 -- 172.21.15.105:0/2131021000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13c40b2d20 msgr2=0x7f13c41510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.584+0000 7f13c1d9b640 1 --2- 172.21.15.105:0/2131021000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13c40b2d20 0x7f13c41510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.584+0000 7f13c1d9b640 1 -- 172.21.15.105:0/2131021000 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b4009420 con 0x7f13c40d3c30 2023-11-28T18:37:54.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.584+0000 7f13c259c640 1 --2- 172.21.15.105:0/2131021000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13c40b2d20 0x7f13c41510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:37:54.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.584+0000 7f13c2d9d640 1 --2- 172.21.15.105:0/2131021000 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f13c415d4f0 0x7f13c4155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:37:54.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.584+0000 7f13c1d9b640 1 --2- 172.21.15.105:0/2131021000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c40d3c30 0x7f13c4160670 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f13b800c3f0 tx=0x7f13b800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:54.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.584+0000 7f13b2ffd640 1 -- 172.21.15.105:0/2131021000 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b80052d0 con 0x7f13c40d3c30 2023-11-28T18:37:54.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.584+0000 7f13b2ffd640 1 -- 172.21.15.105:0/2131021000 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b8005430 con 0x7f13c40d3c30 2023-11-28T18:37:54.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.584+0000 7f13c8baa640 1 -- 172.21.15.105:0/2131021000 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c40cc4d0 con 0x7f13c40d3c30 2023-11-28T18:37:54.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.584+0000 7f13b2ffd640 1 -- 172.21.15.105:0/2131021000 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b8003840 con 0x7f13c40d3c30 2023-11-28T18:37:54.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.584+0000 7f13c8baa640 1 -- 172.21.15.105:0/2131021000 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c4159550 con 0x7f13c40d3c30 2023-11-28T18:37:54.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.584+0000 7f13c8baa640 1 -- 172.21.15.105:0/2131021000 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c4067570 con 0x7f13c40d3c30 2023-11-28T18:37:54.583 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.584+0000 7f13b2ffd640 1 -- 172.21.15.105:0/2131021000 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f13b80039a0 con 0x7f13c40d3c30 2023-11-28T18:37:54.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.588+0000 7f13b2ffd640 1 --2- 172.21.15.105:0/2131021000 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f13a40668d0 0x7f13a4068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.588+0000 7f13b2ffd640 1 -- 172.21.15.105:0/2131021000 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f13b8083b30 con 0x7f13c40d3c30 2023-11-28T18:37:54.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.588+0000 7f13c259c640 1 --2- 172.21.15.105:0/2131021000 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f13a40668d0 0x7f13a4068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.585 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.588+0000 7f13c259c640 1 --2- 172.21.15.105:0/2131021000 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f13a40668d0 0x7f13a4068d90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f13b40af000 tx=0x7f13b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:54.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.592+0000 7f13b2ffd640 1 -- 172.21.15.105:0/2131021000 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f13b8050bc0 con 0x7f13c40d3c30 2023-11-28T18:37:54.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.616+0000 7f5af265a640 1 -- 172.21.15.105:0/1667212142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5aec158720 con 0x7f5aec15d4f0 2023-11-28T18:37:54.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.620+0000 7f5ae8ff9640 1 -- 172.21.15.105:0/1667212142 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+16923 (secure 0 0 0) 0x7f5ae40eff70 con 0x7f5aec15d4f0 2023-11-28T18:37:54.617 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:37:54.617 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":130,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:32:58.181121+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:31:56.971692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-28T18:31:57.881657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6835/3706038483":"2023-11-29T18:32:58.181068+0000","172.21.15.148:6833/3706038483":"2023-11-29T18:32:58.181068+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:37:54.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.620+0000 7f5af265a640 1 -- 172.21.15.105:0/1667212142 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5ad80668d0 msgr2=0x7f5ad8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.620+0000 7f5af265a640 1 --2- 172.21.15.105:0/1667212142 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5ad80668d0 0x7f5ad8068d90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5adc000f70 tx=0x7f5adc009040 comp rx=0 tx=0).stop 2023-11-28T18:37:54.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.620+0000 7f5af265a640 1 -- 172.21.15.105:0/1667212142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aec15d4f0 msgr2=0x7f5aec153300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.620+0000 7f5af265a640 1 --2- 172.21.15.105:0/1667212142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aec15d4f0 0x7f5aec153300 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40b0040 tx=0x7f5ae40195a0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.620+0000 7f5af265a640 1 -- 172.21.15.105:0/1667212142 shutdown_connections 2023-11-28T18:37:54.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.620+0000 7f5af265a640 1 --2- 172.21.15.105:0/1667212142 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f5ad80668d0 0x7f5ad8068d90 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.620+0000 7f5af265a640 1 --2- 172.21.15.105:0/1667212142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aec15d4f0 0x7f5aec153300 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.619 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.620+0000 7f5af265a640 1 --2- 172.21.15.105:0/1667212142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aec0d3c30 0x7f5aec0cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.620+0000 7f5af265a640 1 --2- 172.21.15.105:0/1667212142 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5aec0b3d30 0x7f5aec0b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.620+0000 7f5af265a640 1 -- 172.21.15.105:0/1667212142 >> 172.21.15.105:0/1667212142 conn(0x7f5aec0d3030 msgr2=0x7f5aec06c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:54.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.620+0000 7f5af265a640 1 -- 172.21.15.105:0/1667212142 shutdown_connections 2023-11-28T18:37:54.620 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.620+0000 7f5af265a640 1 -- 172.21.15.105:0/1667212142 wait complete. 2023-11-28T18:37:54.642 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T18:23:21.333724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T18:31:56.971692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T18:31:57.881657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T18:37:54.642 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T18:37:54.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.744+0000 7f13c8baa640 1 -- 172.21.15.105:0/2131021000 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f13c4158ea0 con 0x7f13c40d3c30 2023-11-28T18:37:54.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.744+0000 7f13b2ffd640 1 -- 172.21.15.105:0/2131021000 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v130) v1 ==== 100+0+11 (secure 0 0 0) 0x7f13b80170a0 con 0x7f13c40d3c30 2023-11-28T18:37:54.743 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:37:54.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.748+0000 7f13c8baa640 1 -- 172.21.15.105:0/2131021000 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f13a40668d0 msgr2=0x7f13a4068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.748+0000 7f13c8baa640 1 --2- 172.21.15.105:0/2131021000 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f13a40668d0 0x7f13a4068d90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f13b40af000 tx=0x7f13b4008040 comp rx=0 tx=0).stop 2023-11-28T18:37:54.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.748+0000 7f13c8baa640 1 -- 172.21.15.105:0/2131021000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c40d3c30 msgr2=0x7f13c4160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.748+0000 7f13c8baa640 1 --2- 172.21.15.105:0/2131021000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c40d3c30 0x7f13c4160670 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f13b800c3f0 tx=0x7f13b800c8b0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.748+0000 7f13c8baa640 1 -- 172.21.15.105:0/2131021000 shutdown_connections 2023-11-28T18:37:54.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.748+0000 7f13c8baa640 1 --2- 172.21.15.105:0/2131021000 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f13c415d4f0 0x7f13c4155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.748+0000 7f13c8baa640 1 --2- 172.21.15.105:0/2131021000 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f13a40668d0 0x7f13a4068d90 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.748+0000 7f13c8baa640 1 --2- 172.21.15.105:0/2131021000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c40d3c30 0x7f13c4160670 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.748+0000 7f13c8baa640 1 --2- 172.21.15.105:0/2131021000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13c40b2d20 0x7f13c41510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.748+0000 7f13c8baa640 1 -- 172.21.15.105:0/2131021000 >> 172.21.15.105:0/2131021000 conn(0x7f13c40d3030 msgr2=0x7f13c4158240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:54.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.748+0000 7f13c8baa640 1 -- 172.21.15.105:0/2131021000 shutdown_connections 2023-11-28T18:37:54.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.748+0000 7f13c8baa640 1 -- 172.21.15.105:0/2131021000 wait complete. 2023-11-28T18:37:54.765 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T18:37:54.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.828+0000 7f0dc1f2a640 1 Processor -- start 2023-11-28T18:37:54.827 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.828+0000 7f0dc1f2a640 1 -- start start 2023-11-28T18:37:54.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.832+0000 7f0dc1f2a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0dbc15d4f0 0x7f0dbc0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.832+0000 7f0dc1f2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0dbc0d3c30 0x7f0dbc0d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.828 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.832+0000 7f0dc1f2a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dbc0b25b0 con 0x7f0dbc0d1890 2023-11-28T18:37:54.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.832+0000 7f0dc1f2a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dbc0b2900 con 0x7f0dbc15d4f0 2023-11-28T18:37:54.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.832+0000 7f0dc1f2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc0cea30 con 0x7f0dbc0d3c30 2023-11-28T18:37:54.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.832+0000 7f0dbbfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0dbc0d1890 0x7f0dbc0cfa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51410/0 (socket says 172.21.15.105:51410) 2023-11-28T18:37:54.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.832+0000 7f0dbbfff640 1 -- 172.21.15.105:0/1377137747 learned_addr learned my addr 172.21.15.105:0/1377137747 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:37:54.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.832+0000 7f0dbb7fe640 1 --2- 172.21.15.105:0/1377137747 >> v2:172.21.15.148:3300/0 conn(0x7f0dbc15d4f0 0x7f0dbc0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.829 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.832+0000 7f0dbaffd640 1 --2- 172.21.15.105:0/1377137747 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0dbc0d3c30 0x7f0dbc0d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.832+0000 7f0dba7fc640 1 -- 172.21.15.105:0/1377137747 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784435137 0 0) 0x7f0dbc0b25b0 con 0x7f0dbc0d1890 2023-11-28T18:37:54.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.832+0000 7f0dba7fc640 1 -- 172.21.15.105:0/1377137747 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0da4003530 con 0x7f0dbc0d1890 2023-11-28T18:37:54.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.832+0000 7f0dbaffd640 1 -- 172.21.15.105:0/1377137747 >> v2:172.21.15.148:3300/0 conn(0x7f0dbc15d4f0 msgr2=0x7f0dbc0d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:54.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.832+0000 7f0dbaffd640 1 --2- 172.21.15.105:0/1377137747 >> v2:172.21.15.148:3300/0 conn(0x7f0dbc15d4f0 0x7f0dbc0d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.832+0000 7f0dbaffd640 1 -- 172.21.15.105:0/1377137747 >> v1:172.21.15.105:6789/0 conn(0x7f0dbc0d1890 legacy=0x7f0dbc0cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.832+0000 7f0dbaffd640 1 -- 172.21.15.105:0/1377137747 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc0ced10 con 0x7f0dbc0d3c30 2023-11-28T18:37:54.830 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.832+0000 7f0dbb7fe640 1 --2- 172.21.15.105:0/1377137747 >> v2:172.21.15.148:3300/0 conn(0x7f0dbc15d4f0 0x7f0dbc0d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:37:54.832 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.836+0000 7f0dbaffd640 1 --2- 172.21.15.105:0/1377137747 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0dbc0d3c30 0x7f0dbc0d1350 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0db00097c0 tx=0x7f0db001c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3144edeb83acfe69 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:54.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.836+0000 7f0dba7fc640 1 -- 172.21.15.105:0/1377137747 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db00ab600 con 0x7f0dbc0d3c30 2023-11-28T18:37:54.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.836+0000 7f0dba7fc640 1 -- 172.21.15.105:0/1377137747 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db00a7050 con 0x7f0dbc0d3c30 2023-11-28T18:37:54.833 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.836+0000 7f0dba7fc640 1 -- 172.21.15.105:0/1377137747 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db00ab980 con 0x7f0dbc0d3c30 2023-11-28T18:37:54.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.836+0000 7f0dc1f2a640 1 -- 172.21.15.105:0/1377137747 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0dbc0d3c30 msgr2=0x7f0dbc0d1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.836+0000 7f0dc1f2a640 1 --2- 172.21.15.105:0/1377137747 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0dbc0d3c30 0x7f0dbc0d1350 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0db00097c0 tx=0x7f0db001c3c0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.836+0000 7f0dc1f2a640 1 -- 172.21.15.105:0/1377137747 shutdown_connections 2023-11-28T18:37:54.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.836+0000 7f0dc1f2a640 1 --2- 172.21.15.105:0/1377137747 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0dbc0d3c30 0x7f0dbc0d1350 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.836+0000 7f0dc1f2a640 1 --2- 172.21.15.105:0/1377137747 >> v2:172.21.15.148:3300/0 conn(0x7f0dbc15d4f0 0x7f0dbc0d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.836+0000 7f0dc1f2a640 1 -- 172.21.15.105:0/1377137747 >> 172.21.15.105:0/1377137747 conn(0x7f0dbc0d3030 msgr2=0x7f0dbc0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:54.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.836+0000 7f0dc1f2a640 1 -- 172.21.15.105:0/1377137747 shutdown_connections 2023-11-28T18:37:54.834 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.836+0000 7f0dc1f2a640 1 -- 172.21.15.105:0/1377137747 wait complete. 2023-11-28T18:37:54.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.836+0000 7f0dc1f2a640 1 Processor -- start 2023-11-28T18:37:54.835 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.836+0000 7f0dc1f2a640 1 -- start start 2023-11-28T18:37:54.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0dc1f2a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0dbc0d1890 0x7f0dbc15db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0dc1f2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0dbc0d3c30 0x7f0dbc14d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0dc1f2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0dbc15d4f0 0x7f0dbc159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0dc1f2a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc15e1f0 con 0x7f0dbc15d4f0 2023-11-28T18:37:54.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0dc1f2a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc1602f0 con 0x7f0dbc0d3c30 2023-11-28T18:37:54.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0dc1f2a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc1605d0 con 0x7f0dbc0d1890 2023-11-28T18:37:54.837 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0dbbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0dbc15d4f0 0x7f0dbc159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0dbbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0dbc15d4f0 0x7f0dbc159260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54434/0 (socket says 172.21.15.105:54434) 2023-11-28T18:37:54.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0dbbfff640 1 -- 172.21.15.105:0/3259164024 learned_addr learned my addr 172.21.15.105:0/3259164024 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:54.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0dbaffd640 1 --2- 172.21.15.105:0/3259164024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0dbc0d3c30 0x7f0dbc14d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0dbb7fe640 1 --2- 172.21.15.105:0/3259164024 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0dbc0d1890 0x7f0dbc15db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0dbbfff640 1 -- 172.21.15.105:0/3259164024 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0dbc0d1890 msgr2=0x7f0dbc15db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0dbbfff640 1 --2- 172.21.15.105:0/3259164024 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0dbc0d1890 0x7f0dbc15db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.838 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0dbbfff640 1 -- 172.21.15.105:0/3259164024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0dbc0d3c30 msgr2=0x7f0dbc14d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0dbbfff640 1 --2- 172.21.15.105:0/3259164024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0dbc0d3c30 0x7f0dbc14d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0dbbfff640 1 -- 172.21.15.105:0/3259164024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dac009420 con 0x7f0dbc15d4f0 2023-11-28T18:37:54.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0dbb7fe640 1 --2- 172.21.15.105:0/3259164024 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0dbc0d1890 0x7f0dbc15db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:37:54.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0dbbfff640 1 --2- 172.21.15.105:0/3259164024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0dbc15d4f0 0x7f0dbc159260 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f0db400caa0 tx=0x7f0db400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:54.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0da3fff640 1 -- 172.21.15.105:0/3259164024 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db400ac00 con 0x7f0dbc15d4f0 2023-11-28T18:37:54.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0da3fff640 1 -- 172.21.15.105:0/3259164024 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db400ad60 con 0x7f0dbc15d4f0 2023-11-28T18:37:54.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0dc1f2a640 1 -- 172.21.15.105:0/3259164024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db0009420 con 0x7f0dbc15d4f0 2023-11-28T18:37:54.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0da3fff640 1 -- 172.21.15.105:0/3259164024 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db4012620 con 0x7f0dbc15d4f0 2023-11-28T18:37:54.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0dc1f2a640 1 -- 172.21.15.105:0/3259164024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dbc0ccee0 con 0x7f0dbc15d4f0 2023-11-28T18:37:54.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0dc1f2a640 1 -- 172.21.15.105:0/3259164024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dbc14e040 con 0x7f0dbc15d4f0 2023-11-28T18:37:54.840 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.840+0000 7f0da3fff640 1 -- 172.21.15.105:0/3259164024 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f0db4007070 con 0x7f0dbc15d4f0 2023-11-28T18:37:54.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.844+0000 7f0da3fff640 1 --2- 172.21.15.105:0/3259164024 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0da80668d0 0x7f0da8068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.844+0000 7f0da3fff640 1 -- 172.21.15.105:0/3259164024 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f0db40845b0 con 0x7f0dbc15d4f0 2023-11-28T18:37:54.841 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.844+0000 7f0dbb7fe640 1 --2- 172.21.15.105:0/3259164024 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0da80668d0 0x7f0da8068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.842 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.844+0000 7f0dbb7fe640 1 --2- 172.21.15.105:0/3259164024 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0da80668d0 0x7f0da8068d90 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0dac009ca0 tx=0x7f0dac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:54.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.848+0000 7f0da3fff640 1 -- 172.21.15.105:0/3259164024 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0dbc14e040 con 0x7f0dbc15d4f0 2023-11-28T18:37:54.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.908+0000 7f33a6d57640 1 Processor -- start 2023-11-28T18:37:54.908 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.908+0000 7f33a6d57640 1 -- start start 2023-11-28T18:37:54.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f33a6d57640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33a00ce2c0 0x7f33a006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f33a6d57640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a00cde30 con 0x7f33a0159610 2023-11-28T18:37:54.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f33a6d57640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a01599f0 con 0x7f33a006cd20 2023-11-28T18:37:54.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f33a6d57640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a006d1b0 con 0x7f33a00ce2c0 2023-11-28T18:37:54.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f33a4af3640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f33a0159610 0x7f33a00cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51426/0 (socket says 172.21.15.105:51426) 2023-11-28T18:37:54.909 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f33a4af3640 1 -- 172.21.15.105:0/3725811735 learned_addr learned my addr 172.21.15.105:0/3725811735 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:37:54.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f339ffff640 1 --2- 172.21.15.105:0/3725811735 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33a00ce2c0 0x7f33a006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f339f7fe640 1 -- 172.21.15.105:0/3725811735 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968615952 0 0) 0x7f33a00cde30 con 0x7f33a0159610 2023-11-28T18:37:54.911 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f339f7fe640 1 -- 172.21.15.105:0/3725811735 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3390003450 con 0x7f33a0159610 2023-11-28T18:37:54.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f339f7fe640 1 -- 172.21.15.105:0/3725811735 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2025773498 0 0) 0x7f3390003450 con 0x7f33a0159610 2023-11-28T18:37:54.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f339f7fe640 1 -- 172.21.15.105:0/3725811735 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33a00ce2c0 msgr2=0x7f33a006c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:54.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f339f7fe640 1 --2- 172.21.15.105:0/3725811735 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33a00ce2c0 0x7f33a006c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f339f7fe640 1 -- 172.21.15.105:0/3725811735 >> v1:172.21.15.148:6789/0 conn(0x7f33a006cd20 legacy=0x7f33a00abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f339f7fe640 1 -- 172.21.15.105:0/3725811735 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a006de20 con 0x7f33a0159610 2023-11-28T18:37:54.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f339f7fe640 1 -- 172.21.15.105:0/3725811735 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f3394001680 con 0x7f33a0159610 2023-11-28T18:37:54.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f339f7fe640 1 -- 172.21.15.105:0/3725811735 >> v1:172.21.15.105:6789/0 conn(0x7f33a0159610 legacy=0x7f33a00cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.912 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f339f7fe640 1 --2- 172.21.15.105:0/3725811735 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33a00ce2c0 0x7f3390003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f339f7fe640 1 --2- 172.21.15.105:0/3725811735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3390004020 0x7f339000a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f339f7fe640 1 --2- 172.21.15.105:0/3725811735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f339000afb0 0x7f339000d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f339f7fe640 1 -- 172.21.15.105:0/3725811735 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a006d1b0 con 0x7f339000afb0 2023-11-28T18:37:54.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f339f7fe640 1 -- 172.21.15.105:0/3725811735 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339000dc80 con 0x7f3390004020 2023-11-28T18:37:54.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f339f7fe640 1 -- 172.21.15.105:0/3725811735 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339000de40 con 0x7f33a00ce2c0 2023-11-28T18:37:54.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f33a52f4640 1 --2- 172.21.15.105:0/3725811735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f339000afb0 0x7f339000d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.913 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f33a4af3640 1 --2- 172.21.15.105:0/3725811735 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33a00ce2c0 0x7f3390003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f339ffff640 1 --2- 172.21.15.105:0/3725811735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3390004020 0x7f339000a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f33a52f4640 1 -- 172.21.15.105:0/3725811735 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33a00ce2c0 msgr2=0x7f3390003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:54.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f33a52f4640 1 --2- 172.21.15.105:0/3725811735 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33a00ce2c0 0x7f3390003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f33a52f4640 1 -- 172.21.15.105:0/3725811735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3390004020 msgr2=0x7f339000a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:54.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f33a52f4640 1 --2- 172.21.15.105:0/3725811735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3390004020 0x7f339000a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.912+0000 7f33a52f4640 1 -- 172.21.15.105:0/3725811735 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3390003450 con 0x7f339000afb0 2023-11-28T18:37:54.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.916+0000 7f33a52f4640 1 --2- 172.21.15.105:0/3725811735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f339000afb0 0x7f339000d440 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f3398001e70 tx=0x7f339801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4e9465cea8cfda server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:54.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.916+0000 7f339f7fe640 1 -- 172.21.15.105:0/3725811735 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398003b80 con 0x7f339000afb0 2023-11-28T18:37:54.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.916+0000 7f339f7fe640 1 -- 172.21.15.105:0/3725811735 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3398003ce0 con 0x7f339000afb0 2023-11-28T18:37:54.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.916+0000 7f339f7fe640 1 -- 172.21.15.105:0/3725811735 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33980052c0 con 0x7f339000afb0 2023-11-28T18:37:54.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.916+0000 7f33a6d57640 1 -- 172.21.15.105:0/3725811735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f339000afb0 msgr2=0x7f339000d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.916+0000 7f33a6d57640 1 --2- 172.21.15.105:0/3725811735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f339000afb0 0x7f339000d440 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f3398001e70 tx=0x7f339801c760 comp rx=0 tx=0).stop 2023-11-28T18:37:54.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.916+0000 7f33a52f4640 1 -- 172.21.15.105:0/3725811735 reap_dead start 2023-11-28T18:37:54.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.916+0000 7f33a6d57640 1 -- 172.21.15.105:0/3725811735 shutdown_connections 2023-11-28T18:37:54.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.916+0000 7f33a6d57640 1 -- 172.21.15.105:0/3725811735 >> 172.21.15.105:0/3725811735 conn(0x7f33a00ceb90 msgr2=0x7f33a00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:54.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.916+0000 7f33a6d57640 1 -- 172.21.15.105:0/3725811735 shutdown_connections 2023-11-28T18:37:54.916 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.916+0000 7f33a6d57640 1 -- 172.21.15.105:0/3725811735 wait complete. 2023-11-28T18:37:54.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f33a6d57640 1 Processor -- start 2023-11-28T18:37:54.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f33a6d57640 1 -- start start 2023-11-28T18:37:54.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f33a6d57640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f339000afb0 0x7f33a00c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f33a6d57640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33a0154f70 0x7f33a00c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f33a6d57640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33a0152600 0x7f33a0152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f33a6d57640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a0155f80 con 0x7f33a0152600 2023-11-28T18:37:54.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f33a6d57640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a0152ff0 con 0x7f339000afb0 2023-11-28T18:37:54.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f33a6d57640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a0155400 con 0x7f33a0154f70 2023-11-28T18:37:54.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f33a52f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33a0152600 0x7f33a0152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.919 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f33a52f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33a0152600 0x7f33a0152ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54462/0 (socket says 172.21.15.105:54462) 2023-11-28T18:37:54.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f33a52f4640 1 -- 172.21.15.105:0/4094515884 learned_addr learned my addr 172.21.15.105:0/4094515884 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:54.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f33a4af3640 1 --2- 172.21.15.105:0/4094515884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f339000afb0 0x7f33a00c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f339ffff640 1 --2- 172.21.15.105:0/4094515884 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33a0154f70 0x7f33a00c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f33a52f4640 1 -- 172.21.15.105:0/4094515884 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33a0154f70 msgr2=0x7f33a00c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f33a52f4640 1 --2- 172.21.15.105:0/4094515884 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33a0154f70 0x7f33a00c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f33a52f4640 1 -- 172.21.15.105:0/4094515884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f339000afb0 msgr2=0x7f33a00c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:54.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f33a52f4640 1 --2- 172.21.15.105:0/4094515884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f339000afb0 0x7f33a00c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:54.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f33a52f4640 1 -- 172.21.15.105:0/4094515884 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a01556e0 con 0x7f33a0152600 2023-11-28T18:37:54.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f339ffff640 1 --2- 172.21.15.105:0/4094515884 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33a0154f70 0x7f33a00c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:37:54.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f33a52f4640 1 --2- 172.21.15.105:0/4094515884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33a0152600 0x7f33a0152ab0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f3398098170 tx=0x7f3390003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:54.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f339d7fa640 1 -- 172.21.15.105:0/4094515884 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33980a6660 con 0x7f33a0152600 2023-11-28T18:37:54.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f339d7fa640 1 -- 172.21.15.105:0/4094515884 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33980a67c0 con 0x7f33a0152600 2023-11-28T18:37:54.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f339d7fa640 1 -- 172.21.15.105:0/4094515884 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33980a6b50 con 0x7f33a0152600 2023-11-28T18:37:54.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f33a6d57640 1 -- 172.21.15.105:0/4094515884 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a01559c0 con 0x7f33a0152600 2023-11-28T18:37:54.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.920+0000 7f33a6d57640 1 -- 172.21.15.105:0/4094515884 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a00c8e40 con 0x7f33a0152600 2023-11-28T18:37:54.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.924+0000 7f33a6d57640 1 -- 172.21.15.105:0/4094515884 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a00c9030 con 0x7f33a0152600 2023-11-28T18:37:54.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.924+0000 7f339d7fa640 1 -- 172.21.15.105:0/4094515884 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f33980a6cb0 con 0x7f33a0152600 2023-11-28T18:37:54.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.924+0000 7f339d7fa640 1 --2- 172.21.15.105:0/4094515884 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f33840668d0 0x7f3384068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:54.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.924+0000 7f339d7fa640 1 -- 172.21.15.105:0/4094515884 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f339811dfe0 con 0x7f33a0152600 2023-11-28T18:37:54.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.924+0000 7f33a4af3640 1 --2- 172.21.15.105:0/4094515884 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f33840668d0 0x7f3384068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:54.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.924+0000 7f33a4af3640 1 --2- 172.21.15.105:0/4094515884 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f33840668d0 0x7f3384068d90 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f33940017b0 tx=0x7f3394001480 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:54.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:54.928+0000 7f339d7fa640 1 -- 172.21.15.105:0/4094515884 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f33a00c9030 con 0x7f33a0152600 2023-11-28T18:37:55.038 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.040+0000 7f0dc1f2a640 1 -- 172.21.15.105:0/3259164024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0dbc14e040 con 0x7f0dbc15d4f0 2023-11-28T18:37:55.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.040+0000 7f0da3fff640 1 -- 172.21.15.105:0/3259164024 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0db4017020 con 0x7f0dbc15d4f0 2023-11-28T18:37:55.039 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2023-11-28T18:37:55.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.044+0000 7f0dc1f2a640 1 -- 172.21.15.105:0/3259164024 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0da80668d0 msgr2=0x7f0da8068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:55.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.044+0000 7f0dc1f2a640 1 --2- 172.21.15.105:0/3259164024 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0da80668d0 0x7f0da8068d90 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0dac009ca0 tx=0x7f0dac008040 comp rx=0 tx=0).stop 2023-11-28T18:37:55.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.044+0000 7f0dc1f2a640 1 -- 172.21.15.105:0/3259164024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0dbc15d4f0 msgr2=0x7f0dbc159260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:55.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.044+0000 7f0dc1f2a640 1 --2- 172.21.15.105:0/3259164024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0dbc15d4f0 0x7f0dbc159260 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f0db400caa0 tx=0x7f0db400cf60 comp rx=0 tx=0).stop 2023-11-28T18:37:55.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.044+0000 7f0dc1f2a640 1 -- 172.21.15.105:0/3259164024 shutdown_connections 2023-11-28T18:37:55.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.044+0000 7f0dc1f2a640 1 --2- 172.21.15.105:0/3259164024 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f0da80668d0 0x7f0da8068d90 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.044+0000 7f0dc1f2a640 1 --2- 172.21.15.105:0/3259164024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0dbc15d4f0 0x7f0dbc159260 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.044+0000 7f0dc1f2a640 1 --2- 172.21.15.105:0/3259164024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0dbc0d3c30 0x7f0dbc14d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.044+0000 7f0dc1f2a640 1 --2- 172.21.15.105:0/3259164024 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0dbc0d1890 0x7f0dbc15db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.044+0000 7f0dc1f2a640 1 -- 172.21.15.105:0/3259164024 >> 172.21.15.105:0/3259164024 conn(0x7f0dbc0d3030 msgr2=0x7f0dbc0cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:55.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.044+0000 7f0dc1f2a640 1 -- 172.21.15.105:0/3259164024 shutdown_connections 2023-11-28T18:37:55.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.044+0000 7f0dc1f2a640 1 -- 172.21.15.105:0/3259164024 wait complete. 2023-11-28T18:37:55.061 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T18:37:55.127 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.128+0000 7f33a6d57640 1 -- 172.21.15.105:0/4094515884 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f33a00c9030 con 0x7f33a0152600 2023-11-28T18:37:55.128 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.128+0000 7f339d7fa640 1 -- 172.21.15.105:0/4094515884 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v130) v1 ==== 80+0+124 (secure 0 0 0) 0x7f33a00c9030 con 0x7f33a0152600 2023-11-28T18:37:55.128 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.148:6835/3706038483 2023-11-29T18:32:58.181068+0000 2023-11-28T18:37:55.128 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.148:6833/3706038483 2023-11-29T18:32:58.181068+0000 2023-11-28T18:37:55.128 INFO:teuthology.orchestra.run.smithi105.stderr:listed 2 entries 2023-11-28T18:37:55.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.132+0000 7f33a6d57640 1 -- 172.21.15.105:0/4094515884 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f33840668d0 msgr2=0x7f3384068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:55.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.132+0000 7f33a6d57640 1 --2- 172.21.15.105:0/4094515884 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f33840668d0 0x7f3384068d90 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f33940017b0 tx=0x7f3394001480 comp rx=0 tx=0).stop 2023-11-28T18:37:55.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.132+0000 7f33a6d57640 1 -- 172.21.15.105:0/4094515884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33a0152600 msgr2=0x7f33a0152ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:55.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.132+0000 7f33a6d57640 1 --2- 172.21.15.105:0/4094515884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33a0152600 0x7f33a0152ab0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f3398098170 tx=0x7f3390003700 comp rx=0 tx=0).stop 2023-11-28T18:37:55.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.132+0000 7f33a6d57640 1 -- 172.21.15.105:0/4094515884 shutdown_connections 2023-11-28T18:37:55.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.132+0000 7f33a6d57640 1 --2- 172.21.15.105:0/4094515884 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f33840668d0 0x7f3384068d90 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.132+0000 7f33a6d57640 1 --2- 172.21.15.105:0/4094515884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33a0152600 0x7f33a0152ab0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.132+0000 7f33a6d57640 1 --2- 172.21.15.105:0/4094515884 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33a0154f70 0x7f33a00c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.132+0000 7f33a6d57640 1 --2- 172.21.15.105:0/4094515884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f339000afb0 0x7f33a00c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.132+0000 7f33a6d57640 1 -- 172.21.15.105:0/4094515884 >> 172.21.15.105:0/4094515884 conn(0x7f33a00ceb90 msgr2=0x7f33a006c540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:55.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.132+0000 7f33a6d57640 1 -- 172.21.15.105:0/4094515884 shutdown_connections 2023-11-28T18:37:55.132 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.132+0000 7f33a6d57640 1 -- 172.21.15.105:0/4094515884 wait complete. 2023-11-28T18:37:55.157 ERROR:tasks.daemonwatchdog.daemon_watchdog:ignoring exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/daemonwatchdog.py", line 61, in bark mount.umount_wait(force=True) File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/fuse_mount.py", line 399, in umount_wait self.umount(cleanup=False) File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/fuse_mount.py", line 320, in umount if self.is_blocked(): File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/mount.py", line 197, in is_blocked return self.addr in output TypeError: 'in ' requires string as left operand, not NoneType 2023-11-28T18:37:55.158 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:37:55.158 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:37:55.167 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:37:55.167 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.2 is not mounted at ubuntu@smithi105.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.2 2023-11-28T18:37:55.167 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:37:55.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:37:55.168 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T18:37:55.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.216+0000 7f5689de6640 1 Processor -- start 2023-11-28T18:37:55.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.216+0000 7f5689de6640 1 -- start start 2023-11-28T18:37:55.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.216+0000 7f5689de6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5684159610 0x7f56840ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:55.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.216+0000 7f5689de6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56840cc760 0x7f56840ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:55.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.216+0000 7f5689de6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56840cde30 con 0x7f568406c7e0 2023-11-28T18:37:55.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.216+0000 7f5689de6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56841599f0 con 0x7f5684159610 2023-11-28T18:37:55.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.216+0000 7f5689de6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568406de20 con 0x7f56840cc760 2023-11-28T18:37:55.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.216+0000 7f5683fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f568406c7e0 0x7f56840abd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51428/0 (socket says 172.21.15.105:51428) 2023-11-28T18:37:55.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.216+0000 7f5683fff640 1 -- 172.21.15.105:0/3481654010 learned_addr learned my addr 172.21.15.105:0/3481654010 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:37:55.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.216+0000 7f56837fe640 1 --2- 172.21.15.105:0/3481654010 >> v2:172.21.15.148:3300/0 conn(0x7f5684159610 0x7f56840ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:55.217 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.216+0000 7f5682ffd640 1 --2- 172.21.15.105:0/3481654010 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56840cc760 0x7f56840ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:55.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.216+0000 7f56827fc640 1 -- 172.21.15.105:0/3481654010 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 676739135 0 0) 0x7f56840cde30 con 0x7f568406c7e0 2023-11-28T18:37:55.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.216+0000 7f56827fc640 1 -- 172.21.15.105:0/3481654010 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f566c003530 con 0x7f568406c7e0 2023-11-28T18:37:55.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.216+0000 7f56837fe640 1 -- 172.21.15.105:0/3481654010 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56840cc760 msgr2=0x7f56840ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:55.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.216+0000 7f56837fe640 1 --2- 172.21.15.105:0/3481654010 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56840cc760 0x7f56840ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.216+0000 7f56837fe640 1 -- 172.21.15.105:0/3481654010 >> v1:172.21.15.105:6789/0 conn(0x7f568406c7e0 legacy=0x7f56840abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:55.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.216+0000 7f56837fe640 1 -- 172.21.15.105:0/3481654010 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56840cd220 con 0x7f5684159610 2023-11-28T18:37:55.218 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.216+0000 7f5682ffd640 1 --2- 172.21.15.105:0/3481654010 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56840cc760 0x7f56840ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T18:37:55.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.220+0000 7f56837fe640 1 --2- 172.21.15.105:0/3481654010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5684159610 0x7f56840ce2c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f56740097c0 tx=0x7f56840acca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b288d0cc9e67c4ef server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:55.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.220+0000 7f56827fc640 1 -- 172.21.15.105:0/3481654010 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56740ab600 con 0x7f5684159610 2023-11-28T18:37:55.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.220+0000 7f56827fc640 1 -- 172.21.15.105:0/3481654010 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56740a7050 con 0x7f5684159610 2023-11-28T18:37:55.219 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.220+0000 7f56827fc640 1 -- 172.21.15.105:0/3481654010 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56740ab980 con 0x7f5684159610 2023-11-28T18:37:55.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.220+0000 7f5689de6640 1 -- 172.21.15.105:0/3481654010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5684159610 msgr2=0x7f56840ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:55.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.220+0000 7f5689de6640 1 --2- 172.21.15.105:0/3481654010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5684159610 0x7f56840ce2c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f56740097c0 tx=0x7f56840acca0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.220+0000 7f5689de6640 1 -- 172.21.15.105:0/3481654010 shutdown_connections 2023-11-28T18:37:55.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.220+0000 7f5689de6640 1 --2- 172.21.15.105:0/3481654010 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56840cc760 0x7f56840ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.220+0000 7f5689de6640 1 --2- 172.21.15.105:0/3481654010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5684159610 0x7f56840ce2c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.220+0000 7f5689de6640 1 -- 172.21.15.105:0/3481654010 >> 172.21.15.105:0/3481654010 conn(0x7f56840ceb90 msgr2=0x7f56840cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:55.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.220+0000 7f5689de6640 1 -- 172.21.15.105:0/3481654010 shutdown_connections 2023-11-28T18:37:55.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.220+0000 7f5689de6640 1 -- 172.21.15.105:0/3481654010 wait complete. 2023-11-28T18:37:55.220 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f5689de6640 1 Processor -- start 2023-11-28T18:37:55.221 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f5689de6640 1 -- start start 2023-11-28T18:37:55.222 INFO:teuthology.orchestra.run.smithi105.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-28T18:37:55.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f5689de6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f568406c7e0 0x7f5684157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:55.222 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f5689de6640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56840cc760 0x7f5684157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:55.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:37:55.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f5689de6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5684159610 0x7f56841669c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:55.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f5689de6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5684159b30 con 0x7f568406c7e0 2023-11-28T18:37:55.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f5689de6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5684158410 con 0x7f5684159610 2023-11-28T18:37:55.223 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f5689de6640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56841586f0 con 0x7f56840cc760 2023-11-28T18:37:55.224 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:37:55.224 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:37:55.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f56837fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f568406c7e0 0x7f5684157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:55.226 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f56837fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f568406c7e0 0x7f5684157660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54468/0 (socket says 172.21.15.105:54468) 2023-11-28T18:37:55.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f56837fe640 1 -- 172.21.15.105:0/3047719378 learned_addr learned my addr 172.21.15.105:0/3047719378 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:55.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f5683fff640 1 --2- 172.21.15.105:0/3047719378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5684159610 0x7f56841669c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:55.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f5682ffd640 1 --2- 172.21.15.105:0/3047719378 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56840cc760 0x7f5684157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:55.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f56837fe640 1 -- 172.21.15.105:0/3047719378 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56840cc760 msgr2=0x7f5684157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:55.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f56837fe640 1 --2- 172.21.15.105:0/3047719378 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56840cc760 0x7f5684157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f56837fe640 1 -- 172.21.15.105:0/3047719378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5684159610 msgr2=0x7f56841669c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:55.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f56837fe640 1 --2- 172.21.15.105:0/3047719378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5684159610 0x7f56841669c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f56837fe640 1 -- 172.21.15.105:0/3047719378 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5678009420 con 0x7f568406c7e0 2023-11-28T18:37:55.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f5683fff640 1 --2- 172.21.15.105:0/3047719378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5684159610 0x7f56841669c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:37:55.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f56837fe640 1 --2- 172.21.15.105:0/3047719378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f568406c7e0 0x7f5684157660 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f567401a080 tx=0x7f567401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:55.227 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f566bfff640 1 -- 172.21.15.105:0/3047719378 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56740b4460 con 0x7f568406c7e0 2023-11-28T18:37:55.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f566bfff640 1 -- 172.21.15.105:0/3047719378 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56740b45e0 con 0x7f568406c7e0 2023-11-28T18:37:55.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f5689de6640 1 -- 172.21.15.105:0/3047719378 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5674009420 con 0x7f568406c7e0 2023-11-28T18:37:55.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.224+0000 7f566bfff640 1 -- 172.21.15.105:0/3047719378 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56740b4990 con 0x7f568406c7e0 2023-11-28T18:37:55.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.228+0000 7f5689de6640 1 -- 172.21.15.105:0/3047719378 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568406d180 con 0x7f568406c7e0 2023-11-28T18:37:55.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.228+0000 7f5689de6640 1 -- 172.21.15.105:0/3047719378 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5684067570 con 0x7f568406c7e0 2023-11-28T18:37:55.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.228+0000 7f566bfff640 1 -- 172.21.15.105:0/3047719378 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f56740b2070 con 0x7f568406c7e0 2023-11-28T18:37:55.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.228+0000 7f566bfff640 1 --2- 172.21.15.105:0/3047719378 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56700668d0 0x7f5670068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:55.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.228+0000 7f566bfff640 1 -- 172.21.15.105:0/3047719378 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f567411e410 con 0x7f568406c7e0 2023-11-28T18:37:55.228 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.228+0000 7f5682ffd640 1 --2- 172.21.15.105:0/3047719378 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56700668d0 0x7f5670068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:55.229 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.228+0000 7f5682ffd640 1 --2- 172.21.15.105:0/3047719378 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56700668d0 0x7f5670068d90 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f56840715b0 tx=0x7f5678008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:55.230 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.232+0000 7f566bfff640 1 -- 172.21.15.105:0/3047719378 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f56740eb420 con 0x7f568406c7e0 2023-11-28T18:37:55.238 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:37:55.238 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.1 is not mounted at ubuntu@smithi148.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1 2023-11-28T18:37:55.238 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:37:55.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:37:55.239 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T18:37:55.287 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-28T18:37:55.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:37:55.289 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:37:55.289 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:37:55.339 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:37:55.339 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.3 is not mounted at ubuntu@smithi148.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.3 2023-11-28T18:37:55.339 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:37:55.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:37:55.340 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T18:37:55.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.384+0000 7f5689de6640 1 -- 172.21.15.105:0/3047719378 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f568406c3c0 con 0x7f568406c7e0 2023-11-28T18:37:55.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.384+0000 7f566bfff640 1 -- 172.21.15.105:0/3047719378 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v130) v1 ==== 104+0+11 (secure 0 0 0) 0x7f56740eea40 con 0x7f568406c7e0 2023-11-28T18:37:55.382 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2023-11-28T18:37:55.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.384+0000 7f5689de6640 1 -- 172.21.15.105:0/3047719378 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56700668d0 msgr2=0x7f5670068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:55.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.384+0000 7f5689de6640 1 --2- 172.21.15.105:0/3047719378 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56700668d0 0x7f5670068d90 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f56840715b0 tx=0x7f5678008040 comp rx=0 tx=0).stop 2023-11-28T18:37:55.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.384+0000 7f5689de6640 1 -- 172.21.15.105:0/3047719378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f568406c7e0 msgr2=0x7f5684157660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:55.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.384+0000 7f5689de6640 1 --2- 172.21.15.105:0/3047719378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f568406c7e0 0x7f5684157660 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f567401a080 tx=0x7f567401ac90 comp rx=0 tx=0).stop 2023-11-28T18:37:55.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.384+0000 7f5689de6640 1 -- 172.21.15.105:0/3047719378 shutdown_connections 2023-11-28T18:37:55.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.384+0000 7f5689de6640 1 --2- 172.21.15.105:0/3047719378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5684159610 0x7f56841669c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.388+0000 7f5689de6640 1 --2- 172.21.15.105:0/3047719378 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56840cc760 0x7f5684157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.388+0000 7f5689de6640 1 --2- 172.21.15.105:0/3047719378 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f56700668d0 0x7f5670068d90 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.388+0000 7f5689de6640 1 --2- 172.21.15.105:0/3047719378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f568406c7e0 0x7f5684157660 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.388+0000 7f5689de6640 1 -- 172.21.15.105:0/3047719378 >> 172.21.15.105:0/3047719378 conn(0x7f56840ceb90 msgr2=0x7f56840af5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:55.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.388+0000 7f5689de6640 1 -- 172.21.15.105:0/3047719378 shutdown_connections 2023-11-28T18:37:55.385 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.388+0000 7f5689de6640 1 -- 172.21.15.105:0/3047719378 wait complete. 2023-11-28T18:37:55.395 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-28T18:37:55.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:37:55.403 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T18:37:55.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.587+0000 7f4d50365640 1 Processor -- start 2023-11-28T18:37:55.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d50365640 1 -- start start 2023-11-28T18:37:55.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d50365640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4d480b4290 0x7f4d480b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:55.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d50365640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d480b4bb0 con 0x7f4d48071960 2023-11-28T18:37:55.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d50365640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d480701d0 con 0x7f4d4815b030 2023-11-28T18:37:55.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d50365640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d4806df80 con 0x7f4d480b4290 2023-11-28T18:37:55.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4d900640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4d48071960 0x7f4d4815f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51440/0 (socket says 172.21.15.105:51440) 2023-11-28T18:37:55.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4d900640 1 -- 172.21.15.105:0/3788991211 learned_addr learned my addr 172.21.15.105:0/3788991211 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-11-28T18:37:55.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4e101640 1 --2- 172.21.15.105:0/3788991211 >> v2:172.21.15.148:3300/0 conn(0x7f4d480b4290 0x7f4d480b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:55.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4d0ff640 1 -- 172.21.15.105:0/3788991211 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3123333407 0 0) 0x7f4d480b4bb0 con 0x7f4d48071960 2023-11-28T18:37:55.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4d0ff640 1 -- 172.21.15.105:0/3788991211 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d34003530 con 0x7f4d48071960 2023-11-28T18:37:55.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4d0ff640 1 -- 172.21.15.105:0/3788991211 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 613041662 0 0) 0x7f4d34003530 con 0x7f4d48071960 2023-11-28T18:37:55.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4d0ff640 1 -- 172.21.15.105:0/3788991211 >> v2:172.21.15.148:3300/0 conn(0x7f4d480b4290 msgr2=0x7f4d480b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:55.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4d0ff640 1 --2- 172.21.15.105:0/3788991211 >> v2:172.21.15.148:3300/0 conn(0x7f4d480b4290 0x7f4d480b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4d0ff640 1 -- 172.21.15.105:0/3788991211 >> v1:172.21.15.148:6789/0 conn(0x7f4d4815b030 legacy=0x7f4d480709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:55.592 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4d0ff640 1 -- 172.21.15.105:0/3788991211 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d48071df0 con 0x7f4d48071960 2023-11-28T18:37:55.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4d0ff640 1 -- 172.21.15.105:0/3788991211 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793524060 0 0) 0x7f4d38001680 con 0x7f4d48071960 2023-11-28T18:37:55.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4d0ff640 1 -- 172.21.15.105:0/3788991211 >> v1:172.21.15.105:6789/0 conn(0x7f4d48071960 legacy=0x7f4d4815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:55.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4d0ff640 1 --2- 172.21.15.105:0/3788991211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d34003980 0x7f4d34003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:55.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4d0ff640 1 --2- 172.21.15.105:0/3788991211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d340047d0 0x7f4d34004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:55.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4d0ff640 1 --2- 172.21.15.105:0/3788991211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d34005190 0x7f4d3400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:55.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4d0ff640 1 -- 172.21.15.105:0/3788991211 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4806df80 con 0x7f4d34003980 2023-11-28T18:37:55.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4d0ff640 1 -- 172.21.15.105:0/3788991211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3400c030 con 0x7f4d340047d0 2023-11-28T18:37:55.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4d0ff640 1 -- 172.21.15.105:0/3788991211 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3400c1f0 con 0x7f4d34005190 2023-11-28T18:37:55.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4e101640 1 --2- 172.21.15.105:0/3788991211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d34003980 0x7f4d34003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:55.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4d900640 1 --2- 172.21.15.105:0/3788991211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d340047d0 0x7f4d34004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:55.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4e902640 1 --2- 172.21.15.105:0/3788991211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d34005190 0x7f4d3400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:55.593 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4e101640 1 -- 172.21.15.105:0/3788991211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d34005190 msgr2=0x7f4d3400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:55.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4e101640 1 --2- 172.21.15.105:0/3788991211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d34005190 0x7f4d3400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4e101640 1 -- 172.21.15.105:0/3788991211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d340047d0 msgr2=0x7f4d34004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:55.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4e101640 1 --2- 172.21.15.105:0/3788991211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d340047d0 0x7f4d34004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4e101640 1 -- 172.21.15.105:0/3788991211 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d34003530 con 0x7f4d34003980 2023-11-28T18:37:55.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4d900640 1 --2- 172.21.15.105:0/3788991211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d340047d0 0x7f4d34004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:37:55.594 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.591+0000 7f4d4e902640 1 -- 172.21.15.105:0/3788991211 reap_dead start 2023-11-28T18:37:55.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.595+0000 7f4d4e101640 1 --2- 172.21.15.105:0/3788991211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d34003980 0x7f4d34003da0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009ae0 tx=0x7f4d4401ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e61de85bd29fa16 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:55.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.595+0000 7f4d4d0ff640 1 -- 172.21.15.105:0/3788991211 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d440a56a0 con 0x7f4d34003980 2023-11-28T18:37:55.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.595+0000 7f4d4d0ff640 1 -- 172.21.15.105:0/3788991211 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d440a5800 con 0x7f4d34003980 2023-11-28T18:37:55.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.595+0000 7f4d4d0ff640 1 -- 172.21.15.105:0/3788991211 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d440a5bb0 con 0x7f4d34003980 2023-11-28T18:37:55.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.595+0000 7f4d50365640 1 -- 172.21.15.105:0/3788991211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d34003980 msgr2=0x7f4d34003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:55.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.595+0000 7f4d50365640 1 --2- 172.21.15.105:0/3788991211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d34003980 0x7f4d34003da0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009ae0 tx=0x7f4d4401ca20 comp rx=0 tx=0).stop 2023-11-28T18:37:55.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.595+0000 7f4d50365640 1 -- 172.21.15.105:0/3788991211 shutdown_connections 2023-11-28T18:37:55.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.595+0000 7f4d50365640 1 --2- 172.21.15.105:0/3788991211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d34003980 0x7f4d34003da0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.595+0000 7f4d50365640 1 -- 172.21.15.105:0/3788991211 >> 172.21.15.105:0/3788991211 conn(0x7f4d480ceb90 msgr2=0x7f4d480cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:55.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.595+0000 7f4d50365640 1 -- 172.21.15.105:0/3788991211 shutdown_connections 2023-11-28T18:37:55.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.595+0000 7f4d50365640 1 -- 172.21.15.105:0/3788991211 wait complete. 2023-11-28T18:37:55.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.599+0000 7f4d50365640 1 Processor -- start 2023-11-28T18:37:55.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.599+0000 7f4d50365640 1 -- start start 2023-11-28T18:37:55.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.599+0000 7f4d50365640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d34003980 0x7f4d4814d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:55.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.599+0000 7f4d50365640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d4814c6f0 0x7f4d4814ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:55.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.599+0000 7f4d50365640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d481fd5d0 0x7f4d481fda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:55.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.599+0000 7f4d50365640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d481fdfc0 con 0x7f4d34003980 2023-11-28T18:37:55.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.599+0000 7f4d50365640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4814db10 con 0x7f4d481fd5d0 2023-11-28T18:37:55.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.599+0000 7f4d50365640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4814f770 con 0x7f4d4814c6f0 2023-11-28T18:37:55.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.599+0000 7f4d4e101640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d34003980 0x7f4d4814d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:55.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.599+0000 7f4d4e101640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d34003980 0x7f4d4814d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54490/0 (socket says 172.21.15.105:54490) 2023-11-28T18:37:55.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.599+0000 7f4d4e101640 1 -- 172.21.15.105:0/1630762230 learned_addr learned my addr 172.21.15.105:0/1630762230 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:55.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.599+0000 7f4d4e101640 1 -- 172.21.15.105:0/1630762230 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d4814c6f0 msgr2=0x7f4d4814ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:55.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.599+0000 7f4d4e101640 1 --2- 172.21.15.105:0/1630762230 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d4814c6f0 0x7f4d4814ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.599+0000 7f4d4e101640 1 -- 172.21.15.105:0/1630762230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d481fd5d0 msgr2=0x7f4d481fda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:55.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.599+0000 7f4d4e101640 1 --2- 172.21.15.105:0/1630762230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d481fd5d0 0x7f4d481fda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.599+0000 7f4d4e101640 1 -- 172.21.15.105:0/1630762230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4814fa50 con 0x7f4d34003980 2023-11-28T18:37:55.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.599+0000 7f4d4e101640 1 --2- 172.21.15.105:0/1630762230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d34003980 0x7f4d4814d5d0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f4d4401c850 tx=0x7f4d440197c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:55.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.599+0000 7f4d3effd640 1 -- 172.21.15.105:0/1630762230 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d440bd070 con 0x7f4d34003980 2023-11-28T18:37:55.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.599+0000 7f4d3effd640 1 -- 172.21.15.105:0/1630762230 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d44004020 con 0x7f4d34003980 2023-11-28T18:37:55.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.599+0000 7f4d3effd640 1 -- 172.21.15.105:0/1630762230 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d440043f0 con 0x7f4d34003980 2023-11-28T18:37:55.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.599+0000 7f4d50365640 1 -- 172.21.15.105:0/1630762230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d4814fd30 con 0x7f4d34003980 2023-11-28T18:37:55.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.599+0000 7f4d50365640 1 -- 172.21.15.105:0/1630762230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d481502f0 con 0x7f4d34003980 2023-11-28T18:37:55.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.603+0000 7f4d50365640 1 -- 172.21.15.105:0/1630762230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d48067570 con 0x7f4d34003980 2023-11-28T18:37:55.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.603+0000 7f4d3effd640 1 -- 172.21.15.105:0/1630762230 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f4d440b4070 con 0x7f4d34003980 2023-11-28T18:37:55.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.603+0000 7f4d3effd640 1 --2- 172.21.15.105:0/1630762230 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d2c0668d0 0x7f4d2c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:55.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.603+0000 7f4d3effd640 1 -- 172.21.15.105:0/1630762230 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f4d4411e170 con 0x7f4d34003980 2023-11-28T18:37:55.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.603+0000 7f4d4d900640 1 --2- 172.21.15.105:0/1630762230 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d2c0668d0 0x7f4d2c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:55.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.603+0000 7f4d4d900640 1 --2- 172.21.15.105:0/1630762230 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d2c0668d0 0x7f4d2c068d90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f4d38000f70 tx=0x7f4d38001b30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:55.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.607+0000 7f4d3effd640 1 -- 172.21.15.105:0/1630762230 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4d440eb320 con 0x7f4d34003980 2023-11-28T18:37:55.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.755+0000 7f4d50365640 1 -- 172.21.15.105:0/1630762230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4d480cf7f0 con 0x7f4d34003980 2023-11-28T18:37:55.756 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.759+0000 7f4d3effd640 1 -- 172.21.15.105:0/1630762230 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v130) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4d440ee940 con 0x7f4d34003980 2023-11-28T18:37:55.756 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2023-11-28T18:37:55.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.759+0000 7f4d50365640 1 -- 172.21.15.105:0/1630762230 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d2c0668d0 msgr2=0x7f4d2c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:55.758 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.759+0000 7f4d50365640 1 --2- 172.21.15.105:0/1630762230 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d2c0668d0 0x7f4d2c068d90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f4d38000f70 tx=0x7f4d38001b30 comp rx=0 tx=0).stop 2023-11-28T18:37:55.759 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.759+0000 7f4d50365640 1 -- 172.21.15.105:0/1630762230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d34003980 msgr2=0x7f4d4814d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:55.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.759+0000 7f4d50365640 1 --2- 172.21.15.105:0/1630762230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d34003980 0x7f4d4814d5d0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f4d4401c850 tx=0x7f4d440197c0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.759+0000 7f4d50365640 1 -- 172.21.15.105:0/1630762230 shutdown_connections 2023-11-28T18:37:55.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.759+0000 7f4d50365640 1 --2- 172.21.15.105:0/1630762230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d481fd5d0 0x7f4d481fda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.760 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.759+0000 7f4d50365640 1 --2- 172.21.15.105:0/1630762230 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d4814c6f0 0x7f4d4814ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.759+0000 7f4d50365640 1 --2- 172.21.15.105:0/1630762230 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f4d2c0668d0 0x7f4d2c068d90 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.759+0000 7f4d50365640 1 --2- 172.21.15.105:0/1630762230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d34003980 0x7f4d4814d5d0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.759+0000 7f4d50365640 1 -- 172.21.15.105:0/1630762230 >> 172.21.15.105:0/1630762230 conn(0x7f4d480ceb90 msgr2=0x7f4d480ac270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:55.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.759+0000 7f4d50365640 1 -- 172.21.15.105:0/1630762230 shutdown_connections 2023-11-28T18:37:55.761 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.759+0000 7f4d50365640 1 -- 172.21.15.105:0/1630762230 wait complete. 2023-11-28T18:37:55.777 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:37:55.914 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.915+0000 7f197befe640 1 Processor -- start 2023-11-28T18:37:55.915 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.915+0000 7f197befe640 1 -- start start 2023-11-28T18:37:55.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.919+0000 7f197befe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f197406d610 0x7f19740cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:55.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.919+0000 7f197befe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f19740ce530 0x7f19740ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:55.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.919+0000 7f197befe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1974162820 con 0x7f19740adf30 2023-11-28T18:37:55.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.919+0000 7f197befe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19740ae340 con 0x7f19740ce530 2023-11-28T18:37:55.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.919+0000 7f197befe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1974161c10 con 0x7f197406d610 2023-11-28T18:37:55.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.919+0000 7f1979499640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f19740ce530 0x7f19740ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:55.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.919+0000 7f1979499640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f19740ce530 0x7f19740ad9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54494/0 (socket says 172.21.15.105:54494) 2023-11-28T18:37:55.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.919+0000 7f1979499640 1 -- 172.21.15.105:0/1089471074 learned_addr learned my addr 172.21.15.105:0/1089471074 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:55.917 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.919+0000 7f1979c9a640 1 --2- 172.21.15.105:0/1089471074 >> v2:172.21.15.148:3300/0 conn(0x7f197406d610 0x7f19740cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:55.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.919+0000 7f1979499640 1 -- 172.21.15.105:0/1089471074 >> v2:172.21.15.148:3300/0 conn(0x7f197406d610 msgr2=0x7f19740cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:55.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.919+0000 7f1979499640 1 --2- 172.21.15.105:0/1089471074 >> v2:172.21.15.148:3300/0 conn(0x7f197406d610 0x7f19740cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.919+0000 7f1979499640 1 -- 172.21.15.105:0/1089471074 >> v1:172.21.15.148:6789/0 conn(0x7f19740adf30 legacy=0x7f19741650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:55.918 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.919+0000 7f1979499640 1 -- 172.21.15.105:0/1089471074 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1974161ef0 con 0x7f19740ce530 2023-11-28T18:37:55.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f1979499640 1 --2- 172.21.15.105:0/1089471074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19740ce530 0x7f19740ad9f0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f197406fa90 tx=0x7f196401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a37d163644afbe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:55.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f1978c98640 1 -- 172.21.15.105:0/1089471074 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19640ab5f0 con 0x7f19740ce530 2023-11-28T18:37:55.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f1978c98640 1 -- 172.21.15.105:0/1089471074 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19640ab750 con 0x7f19740ce530 2023-11-28T18:37:55.920 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f1978c98640 1 -- 172.21.15.105:0/1089471074 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19640abae0 con 0x7f19740ce530 2023-11-28T18:37:55.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f197befe640 1 -- 172.21.15.105:0/1089471074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19740ce530 msgr2=0x7f19740ad9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:55.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f197befe640 1 --2- 172.21.15.105:0/1089471074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19740ce530 0x7f19740ad9f0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f197406fa90 tx=0x7f196401c4d0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.921 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f197befe640 1 -- 172.21.15.105:0/1089471074 shutdown_connections 2023-11-28T18:37:55.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f197befe640 1 --2- 172.21.15.105:0/1089471074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19740ce530 0x7f19740ad9f0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f197befe640 1 --2- 172.21.15.105:0/1089471074 >> v2:172.21.15.148:3300/0 conn(0x7f197406d610 0x7f19740cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f197befe640 1 -- 172.21.15.105:0/1089471074 >> 172.21.15.105:0/1089471074 conn(0x7f19740ceb90 msgr2=0x7f19740cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:55.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f197befe640 1 -- 172.21.15.105:0/1089471074 shutdown_connections 2023-11-28T18:37:55.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f197befe640 1 -- 172.21.15.105:0/1089471074 wait complete. 2023-11-28T18:37:55.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f197befe640 1 Processor -- start 2023-11-28T18:37:55.923 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f197befe640 1 -- start start 2023-11-28T18:37:55.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f197befe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f197406d610 0x7f197415adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:55.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f197befe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19740adf30 0x7f197415ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:55.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f197befe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19740ce530 0x7f1974158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:55.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f197befe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197415a290 con 0x7f197406d610 2023-11-28T18:37:55.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f197befe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1974160cd0 con 0x7f19740adf30 2023-11-28T18:37:55.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f197befe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197415a550 con 0x7f19740ce530 2023-11-28T18:37:55.925 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f1979c9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f197406d610 0x7f197415adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:55.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f1979c9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f197406d610 0x7f197415adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54496/0 (socket says 172.21.15.105:54496) 2023-11-28T18:37:55.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.923+0000 7f1979c9a640 1 -- 172.21.15.105:0/2886090082 learned_addr learned my addr 172.21.15.105:0/2886090082 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:55.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.927+0000 7f1979499640 1 --2- 172.21.15.105:0/2886090082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19740adf30 0x7f197415ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:55.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.927+0000 7f197a49b640 1 --2- 172.21.15.105:0/2886090082 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19740ce530 0x7f1974158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:55.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.927+0000 7f1979c9a640 1 -- 172.21.15.105:0/2886090082 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19740ce530 msgr2=0x7f1974158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:55.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.927+0000 7f1979c9a640 1 --2- 172.21.15.105:0/2886090082 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19740ce530 0x7f1974158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.926 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.927+0000 7f1979c9a640 1 -- 172.21.15.105:0/2886090082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19740adf30 msgr2=0x7f197415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:55.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.927+0000 7f1979c9a640 1 --2- 172.21.15.105:0/2886090082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19740adf30 0x7f197415ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:55.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.927+0000 7f1979c9a640 1 -- 172.21.15.105:0/2886090082 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19640094d0 con 0x7f197406d610 2023-11-28T18:37:55.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.927+0000 7f197a49b640 1 --2- 172.21.15.105:0/2886090082 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19740ce530 0x7f1974158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:37:55.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.927+0000 7f1979c9a640 1 --2- 172.21.15.105:0/2886090082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f197406d610 0x7f197415adc0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f197000e870 tx=0x7f197000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:55.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.927+0000 7f196a7fc640 1 -- 172.21.15.105:0/2886090082 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1970009530 con 0x7f197406d610 2023-11-28T18:37:55.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.927+0000 7f196a7fc640 1 -- 172.21.15.105:0/2886090082 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197000c940 con 0x7f197406d610 2023-11-28T18:37:55.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.927+0000 7f196a7fc640 1 -- 172.21.15.105:0/2886090082 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197000ccd0 con 0x7f197406d610 2023-11-28T18:37:55.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.927+0000 7f197befe640 1 -- 172.21.15.105:0/2886090082 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197415a830 con 0x7f197406d610 2023-11-28T18:37:55.927 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.927+0000 7f197befe640 1 -- 172.21.15.105:0/2886090082 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19741609c0 con 0x7f197406d610 2023-11-28T18:37:55.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.927+0000 7f197befe640 1 -- 172.21.15.105:0/2886090082 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1974067570 con 0x7f197406d610 2023-11-28T18:37:55.928 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.927+0000 7f196a7fc640 1 -- 172.21.15.105:0/2886090082 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f1970013470 con 0x7f197406d610 2023-11-28T18:37:55.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.927+0000 7f196a7fc640 1 --2- 172.21.15.105:0/2886090082 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f19580668d0 0x7f1958068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:55.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.927+0000 7f196a7fc640 1 -- 172.21.15.105:0/2886090082 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f19700847f0 con 0x7f197406d610 2023-11-28T18:37:55.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.931+0000 7f1979499640 1 --2- 172.21.15.105:0/2886090082 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f19580668d0 0x7f1958068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:55.929 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.931+0000 7f1979499640 1 --2- 172.21.15.105:0/2886090082 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f19580668d0 0x7f1958068d90 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f19640af000 tx=0x7f19640a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:55.931 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:55.931+0000 7f196a7fc640 1 -- 172.21.15.105:0/2886090082 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f19700517f0 con 0x7f197406d610 2023-11-28T18:37:56.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.111+0000 7f197befe640 1 -- 172.21.15.105:0/2886090082 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19740cf950 con 0x7f197406d610 2023-11-28T18:37:56.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.111+0000 7f196a7fc640 1 -- 172.21.15.105:0/2886090082 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 151 v151) v1 ==== 95+0+4219 (secure 0 0 0) 0x7f1970054e10 con 0x7f197406d610 2023-11-28T18:37:56.111 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:37:56.111 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":151,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":151,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:31:58.827408+0000","modified":"2023-11-28T18:36:11.513093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9126,"mds_1":9117,"mds_2":9120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9117":{"gid":9117,"name":"a","rank":1,"incarnation":142,"state":"up:active","state_seq":21,"addr":"172.21.15.105:6835/3161918450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3161918450},{"type":"v1","addr":"172.21.15.105:6835","nonce":3161918450}]},"join_fscid":-1,"laggy_since":"2023-11-28T18:33:18.186653+0000","export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9120":{"gid":9120,"name":"d","rank":2,"incarnation":138,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6834/3958106803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3958106803},{"type":"v1","addr":"172.21.15.148:6834","nonce":3958106803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9126":{"gid":9126,"name":"c","rank":0,"incarnation":130,"state":"up:active","state_seq":66,"addr":"172.21.15.105:6837/1978722810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1978722810},{"type":"v1","addr":"172.21.15.105:6837","nonce":1978722810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-28T18:37:56.111 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 151 2023-11-28T18:37:56.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.115+0000 7f197befe640 1 -- 172.21.15.105:0/2886090082 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f19580668d0 msgr2=0x7f1958068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:56.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.115+0000 7f197befe640 1 --2- 172.21.15.105:0/2886090082 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f19580668d0 0x7f1958068d90 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f19640af000 tx=0x7f19640a7000 comp rx=0 tx=0).stop 2023-11-28T18:37:56.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.115+0000 7f197befe640 1 -- 172.21.15.105:0/2886090082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f197406d610 msgr2=0x7f197415adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:56.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.115+0000 7f197befe640 1 --2- 172.21.15.105:0/2886090082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f197406d610 0x7f197415adc0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f197000e870 tx=0x7f197000ed30 comp rx=0 tx=0).stop 2023-11-28T18:37:56.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.115+0000 7f197befe640 1 -- 172.21.15.105:0/2886090082 shutdown_connections 2023-11-28T18:37:56.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.115+0000 7f197befe640 1 --2- 172.21.15.105:0/2886090082 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19740ce530 0x7f1974158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.115+0000 7f197befe640 1 --2- 172.21.15.105:0/2886090082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19740adf30 0x7f197415ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.115+0000 7f197befe640 1 --2- 172.21.15.105:0/2886090082 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f19580668d0 0x7f1958068d90 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.115+0000 7f197befe640 1 --2- 172.21.15.105:0/2886090082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f197406d610 0x7f197415adc0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.115+0000 7f197befe640 1 -- 172.21.15.105:0/2886090082 >> 172.21.15.105:0/2886090082 conn(0x7f19740ceb90 msgr2=0x7f1974071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:56.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.115+0000 7f197befe640 1 -- 172.21.15.105:0/2886090082 shutdown_connections 2023-11-28T18:37:56.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.115+0000 7f197befe640 1 -- 172.21.15.105:0/2886090082 wait complete. 2023-11-28T18:37:56.132 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T18:37:56.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.271+0000 7f83a6017640 1 Processor -- start 2023-11-28T18:37:56.269 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.271+0000 7f83a6017640 1 -- start start 2023-11-28T18:37:56.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.271+0000 7f83a6017640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f83a00af340 0x7f83a00721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:56.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.271+0000 7f83a6017640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f83a006a580 0x7f83a006a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:56.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.271+0000 7f83a6017640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a006f190 con 0x7f83a006ed00 2023-11-28T18:37:56.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.271+0000 7f83a6017640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83a00b34a0 con 0x7f83a006a580 2023-11-28T18:37:56.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.271+0000 7f83a6017640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83a006af30 con 0x7f83a00af340 2023-11-28T18:37:56.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.271+0000 7f839ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f83a006a580 0x7f83a006a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:56.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.271+0000 7f839ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f83a006a580 0x7f83a006a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54502/0 (socket says 172.21.15.105:54502) 2023-11-28T18:37:56.271 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.271+0000 7f839ffff640 1 -- 172.21.15.105:0/3498413223 learned_addr learned my addr 172.21.15.105:0/3498413223 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:56.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.271+0000 7f839f7fe640 1 --2- 172.21.15.105:0/3498413223 >> v2:172.21.15.148:3300/0 conn(0x7f83a00af340 0x7f83a00721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:56.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.271+0000 7f839ffff640 1 -- 172.21.15.105:0/3498413223 >> v2:172.21.15.148:3300/0 conn(0x7f83a00af340 msgr2=0x7f83a00721d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:56.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.271+0000 7f839ffff640 1 --2- 172.21.15.105:0/3498413223 >> v2:172.21.15.148:3300/0 conn(0x7f83a00af340 0x7f83a00721d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.271+0000 7f839ffff640 1 -- 172.21.15.105:0/3498413223 >> v1:172.21.15.148:6789/0 conn(0x7f83a006ed00 legacy=0x7f83a00d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:56.272 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.271+0000 7f839ffff640 1 -- 172.21.15.105:0/3498413223 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a00b2ee0 con 0x7f83a006a580 2023-11-28T18:37:56.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.275+0000 7f839ffff640 1 --2- 172.21.15.105:0/3498413223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83a006a580 0x7f83a006a9f0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f83a006b170 tx=0x7f839801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c784931a86f63d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:56.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.275+0000 7f839e7fc640 1 -- 172.21.15.105:0/3498413223 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83980a5860 con 0x7f83a006a580 2023-11-28T18:37:56.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.275+0000 7f839e7fc640 1 -- 172.21.15.105:0/3498413223 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83980a59c0 con 0x7f83a006a580 2023-11-28T18:37:56.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.275+0000 7f839e7fc640 1 -- 172.21.15.105:0/3498413223 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83980a5d50 con 0x7f83a006a580 2023-11-28T18:37:56.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.275+0000 7f83a6017640 1 -- 172.21.15.105:0/3498413223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83a006a580 msgr2=0x7f83a006a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:56.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.275+0000 7f83a6017640 1 --2- 172.21.15.105:0/3498413223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83a006a580 0x7f83a006a9f0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f83a006b170 tx=0x7f839801c9a0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.275 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.275+0000 7f83a6017640 1 -- 172.21.15.105:0/3498413223 shutdown_connections 2023-11-28T18:37:56.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.275+0000 7f83a6017640 1 --2- 172.21.15.105:0/3498413223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83a006a580 0x7f83a006a9f0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.275+0000 7f83a6017640 1 --2- 172.21.15.105:0/3498413223 >> v2:172.21.15.148:3300/0 conn(0x7f83a00af340 0x7f83a00721d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.275+0000 7f83a6017640 1 -- 172.21.15.105:0/3498413223 >> 172.21.15.105:0/3498413223 conn(0x7f83a006b860 msgr2=0x7f83a006bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:56.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.275+0000 7f83a6017640 1 -- 172.21.15.105:0/3498413223 shutdown_connections 2023-11-28T18:37:56.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.275+0000 7f83a6017640 1 -- 172.21.15.105:0/3498413223 wait complete. 2023-11-28T18:37:56.276 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f83a6017640 1 Processor -- start 2023-11-28T18:37:56.277 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f83a6017640 1 -- start start 2023-11-28T18:37:56.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f83a6017640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83a006a580 0x7f83a013e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:56.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f83a6017640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83a006ed00 0x7f83a0144540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:56.279 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f83a6017640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83a00af340 0x7f83a013c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:56.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f83a6017640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a01437b0 con 0x7f83a006a580 2023-11-28T18:37:56.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f83a6017640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a013cfd0 con 0x7f83a00af340 2023-11-28T18:37:56.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f83a6017640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a013e800 con 0x7f83a006ed00 2023-11-28T18:37:56.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f839f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83a006a580 0x7f83a013e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:56.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f839f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83a006a580 0x7f83a013e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54512/0 (socket says 172.21.15.105:54512) 2023-11-28T18:37:56.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f839f7fe640 1 -- 172.21.15.105:0/1103502388 learned_addr learned my addr 172.21.15.105:0/1103502388 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:56.280 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f839ffff640 1 --2- 172.21.15.105:0/1103502388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83a00af340 0x7f83a013c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:56.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f839effd640 1 --2- 172.21.15.105:0/1103502388 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83a006ed00 0x7f83a0144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:56.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f839f7fe640 1 -- 172.21.15.105:0/1103502388 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83a006ed00 msgr2=0x7f83a0144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:56.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f839f7fe640 1 --2- 172.21.15.105:0/1103502388 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83a006ed00 0x7f83a0144540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f839f7fe640 1 -- 172.21.15.105:0/1103502388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83a00af340 msgr2=0x7f83a013c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:56.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f839f7fe640 1 --2- 172.21.15.105:0/1103502388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83a00af340 0x7f83a013c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f839f7fe640 1 -- 172.21.15.105:0/1103502388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8398009b30 con 0x7f83a006a580 2023-11-28T18:37:56.281 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f839effd640 1 --2- 172.21.15.105:0/1103502388 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83a006ed00 0x7f83a0144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:37:56.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f839f7fe640 1 --2- 172.21.15.105:0/1103502388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83a006a580 0x7f83a013e2c0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f839000d830 tx=0x7f839000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:56.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f8387fff640 1 -- 172.21.15.105:0/1103502388 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839000b6c0 con 0x7f83a006a580 2023-11-28T18:37:56.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f8387fff640 1 -- 172.21.15.105:0/1103502388 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839000b860 con 0x7f83a006a580 2023-11-28T18:37:56.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f83a6017640 1 -- 172.21.15.105:0/1103502388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a013d220 con 0x7f83a006a580 2023-11-28T18:37:56.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f8387fff640 1 -- 172.21.15.105:0/1103502388 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839000bc10 con 0x7f83a006a580 2023-11-28T18:37:56.282 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.279+0000 7f83a6017640 1 -- 172.21.15.105:0/1103502388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a013d7e0 con 0x7f83a006a580 2023-11-28T18:37:56.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.283+0000 7f83a6017640 1 -- 172.21.15.105:0/1103502388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a0067570 con 0x7f83a006a580 2023-11-28T18:37:56.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.283+0000 7f8387fff640 1 -- 172.21.15.105:0/1103502388 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f839001e460 con 0x7f83a006a580 2023-11-28T18:37:56.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.283+0000 7f8387fff640 1 --2- 172.21.15.105:0/1103502388 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f838c0668d0 0x7f838c068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:56.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.283+0000 7f8387fff640 1 -- 172.21.15.105:0/1103502388 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f8390084da0 con 0x7f83a006a580 2023-11-28T18:37:56.287 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.283+0000 7f839effd640 1 --2- 172.21.15.105:0/1103502388 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f838c0668d0 0x7f838c068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:56.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.283+0000 7f839effd640 1 --2- 172.21.15.105:0/1103502388 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f838c0668d0 0x7f838c068d90 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f83a006e8f0 tx=0x7f8394009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:56.288 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.287+0000 7f8387fff640 1 -- 172.21.15.105:0/1103502388 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8390051da0 con 0x7f83a006a580 2023-11-28T18:37:56.438 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.439+0000 7f83a6017640 1 -- 172.21.15.105:0/1103502388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83a0141fa0 con 0x7f83a006a580 2023-11-28T18:37:56.439 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.439+0000 7f8387fff640 1 -- 172.21.15.105:0/1103502388 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+16923 (secure 0 0 0) 0x7f83900553c0 con 0x7f83a006a580 2023-11-28T18:37:56.440 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:37:56.440 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":130,"fsid":"b302bec5-10e5-4c91-9026-ecb69250e81c","created":"2023-11-28T18:23:08.251044+0000","modified":"2023-11-28T18:32:58.181121+0000","last_up_change":"2023-11-28T18:23:20.754125+0000","last_in_change":"2023-11-28T18:23:10.823816+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T18:23:21.333724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-28T18:31:56.971692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-28T18:31:57.881657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ef4e9b9f-3655-47a8-89b8-a34b1dedbe70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6825","nonce":2669077690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6827","nonce":2669077690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6831","nonce":2669077690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2669077690},{"type":"v1","addr":"172.21.15.105:6829","nonce":2669077690}]},"public_addr":"172.21.15.105:6825/2669077690","cluster_addr":"172.21.15.105:6827/2669077690","heartbeat_back_addr":"172.21.15.105:6831/2669077690","heartbeat_front_addr":"172.21.15.105:6829/2669077690","state":["exists","up"]},{"osd":1,"uuid":"2738deef-83f7-4af6-871c-3e947cdc6954","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6809","nonce":3491434256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6811","nonce":3491434256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6815","nonce":3491434256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3491434256},{"type":"v1","addr":"172.21.15.105:6813","nonce":3491434256}]},"public_addr":"172.21.15.105:6809/3491434256","cluster_addr":"172.21.15.105:6811/3491434256","heartbeat_back_addr":"172.21.15.105:6815/3491434256","heartbeat_front_addr":"172.21.15.105:6813/3491434256","state":["exists","up"]},{"osd":2,"uuid":"83d7788f-b5c7-407b-8879-b456e53016c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6817","nonce":2303306184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6819","nonce":2303306184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6823","nonce":2303306184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2303306184},{"type":"v1","addr":"172.21.15.105:6821","nonce":2303306184}]},"public_addr":"172.21.15.105:6817/2303306184","cluster_addr":"172.21.15.105:6819/2303306184","heartbeat_back_addr":"172.21.15.105:6823/2303306184","heartbeat_front_addr":"172.21.15.105:6821/2303306184","state":["exists","up"]},{"osd":3,"uuid":"2996213e-81df-4eec-94d2-6c7dd7f4f556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6801","nonce":3228549411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6803","nonce":3228549411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6807","nonce":3228549411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3228549411},{"type":"v1","addr":"172.21.15.105:6805","nonce":3228549411}]},"public_addr":"172.21.15.105:6801/3228549411","cluster_addr":"172.21.15.105:6803/3228549411","heartbeat_back_addr":"172.21.15.105:6807/3228549411","heartbeat_front_addr":"172.21.15.105:6805/3228549411","state":["exists","up"]},{"osd":4,"uuid":"4aae5372-22fc-423c-8ed2-e8ab05881be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6801","nonce":1517755666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6803","nonce":1517755666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6807","nonce":1517755666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1517755666},{"type":"v1","addr":"172.21.15.148:6805","nonce":1517755666}]},"public_addr":"172.21.15.148:6801/1517755666","cluster_addr":"172.21.15.148:6803/1517755666","heartbeat_back_addr":"172.21.15.148:6807/1517755666","heartbeat_front_addr":"172.21.15.148:6805/1517755666","state":["exists","up"]},{"osd":5,"uuid":"2b95412d-1a2a-48b3-a7be-25495361720f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6825","nonce":3981624325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6827","nonce":3981624325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6831","nonce":3981624325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":3981624325},{"type":"v1","addr":"172.21.15.148:6829","nonce":3981624325}]},"public_addr":"172.21.15.148:6825/3981624325","cluster_addr":"172.21.15.148:6827/3981624325","heartbeat_back_addr":"172.21.15.148:6831/3981624325","heartbeat_front_addr":"172.21.15.148:6829/3981624325","state":["exists","up"]},{"osd":6,"uuid":"83fe816e-9810-46fa-a430-8670f32c7940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6809","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6811","nonce":4279905987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6813","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6815","nonce":4279905987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6823","nonce":4279905987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6817","nonce":4279905987},{"type":"v1","addr":"172.21.15.148:6819","nonce":4279905987}]},"public_addr":"172.21.15.148:6811/4279905987","cluster_addr":"172.21.15.148:6815/4279905987","heartbeat_back_addr":"172.21.15.148:6823/4279905987","heartbeat_front_addr":"172.21.15.148:6819/4279905987","state":["exists","up"]},{"osd":7,"uuid":"33b8b090-d66f-4149-983a-da294c435c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6810","nonce":3280605225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6814","nonce":3280605225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6821","nonce":3280605225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3280605225},{"type":"v1","addr":"172.21.15.148:6818","nonce":3280605225}]},"public_addr":"172.21.15.148:6810/3280605225","cluster_addr":"172.21.15.148:6814/3280605225","heartbeat_back_addr":"172.21.15.148:6821/3280605225","heartbeat_front_addr":"172.21.15.148:6818/3280605225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.512792+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.525192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.493426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.392503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.331064+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.448799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.469890+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T18:23:16.559384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6835/3706038483":"2023-11-29T18:32:58.181068+0000","172.21.15.148:6833/3706038483":"2023-11-29T18:32:58.181068+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T18:37:56.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.443+0000 7f83a6017640 1 -- 172.21.15.105:0/1103502388 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f838c0668d0 msgr2=0x7f838c068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:56.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.443+0000 7f83a6017640 1 --2- 172.21.15.105:0/1103502388 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f838c0668d0 0x7f838c068d90 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f83a006e8f0 tx=0x7f8394009040 comp rx=0 tx=0).stop 2023-11-28T18:37:56.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.443+0000 7f83a6017640 1 -- 172.21.15.105:0/1103502388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83a006a580 msgr2=0x7f83a013e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:56.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.443+0000 7f83a6017640 1 --2- 172.21.15.105:0/1103502388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83a006a580 0x7f83a013e2c0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f839000d830 tx=0x7f839000dcf0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.443+0000 7f83a6017640 1 -- 172.21.15.105:0/1103502388 shutdown_connections 2023-11-28T18:37:56.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.443+0000 7f83a6017640 1 --2- 172.21.15.105:0/1103502388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83a00af340 0x7f83a013c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.443+0000 7f83a6017640 1 --2- 172.21.15.105:0/1103502388 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83a006ed00 0x7f83a0144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.442 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.443+0000 7f83a6017640 1 --2- 172.21.15.105:0/1103502388 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f838c0668d0 0x7f838c068d90 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.443 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.443+0000 7f83a6017640 1 --2- 172.21.15.105:0/1103502388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83a006a580 0x7f83a013e2c0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.443 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.443+0000 7f83a6017640 1 -- 172.21.15.105:0/1103502388 >> 172.21.15.105:0/1103502388 conn(0x7f83a006b860 msgr2=0x7f83a006bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:56.443 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.443+0000 7f83a6017640 1 -- 172.21.15.105:0/1103502388 shutdown_connections 2023-11-28T18:37:56.443 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.443+0000 7f83a6017640 1 -- 172.21.15.105:0/1103502388 wait complete. 2023-11-28T18:37:56.461 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T18:37:56.461 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:37:56.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.599+0000 7f0923155640 1 Processor -- start 2023-11-28T18:37:56.598 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.599+0000 7f0923155640 1 -- start start 2023-11-28T18:37:56.599 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.599+0000 7f0923155640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f091c159f40 0x7f091c0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:56.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.599+0000 7f0923155640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f091c0ad850 0x7f091c0701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:56.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.599+0000 7f0923155640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f091c06bf50 con 0x7f091c0cc760 2023-11-28T18:37:56.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.599+0000 7f0923155640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f091c0ccbc0 con 0x7f091c159f40 2023-11-28T18:37:56.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.599+0000 7f0923155640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091c15a320 con 0x7f091c0ad850 2023-11-28T18:37:56.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.603+0000 7f0920ef1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f091c159f40 0x7f091c0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:56.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.603+0000 7f0920ef1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f091c159f40 0x7f091c0caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54514/0 (socket says 172.21.15.105:54514) 2023-11-28T18:37:56.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.603+0000 7f0920ef1640 1 -- 172.21.15.105:0/3058052108 learned_addr learned my addr 172.21.15.105:0/3058052108 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:56.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.603+0000 7f09216f2640 1 --2- 172.21.15.105:0/3058052108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f091c0ad850 0x7f091c0701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:56.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.603+0000 7f0920ef1640 1 -- 172.21.15.105:0/3058052108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f091c0ad850 msgr2=0x7f091c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T18:37:56.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.603+0000 7f0920ef1640 1 --2- 172.21.15.105:0/3058052108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f091c0ad850 0x7f091c0701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.603+0000 7f0920ef1640 1 -- 172.21.15.105:0/3058052108 >> v1:172.21.15.148:6789/0 conn(0x7f091c0cc760 legacy=0x7f091c0ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:56.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.603+0000 7f0920ef1640 1 -- 172.21.15.105:0/3058052108 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c071380 con 0x7f091c159f40 2023-11-28T18:37:56.602 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.603+0000 7f09216f2640 1 --2- 172.21.15.105:0/3058052108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f091c0ad850 0x7f091c0701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T18:37:56.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.603+0000 7f0920ef1640 1 --2- 172.21.15.105:0/3058052108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f091c159f40 0x7f091c0caf60 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f0918009870 tx=0x7f091801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf69f36f0c353e44 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:56.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.603+0000 7f09137fe640 1 -- 172.21.15.105:0/3058052108 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09180ab680 con 0x7f091c159f40 2023-11-28T18:37:56.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.603+0000 7f09137fe640 1 -- 172.21.15.105:0/3058052108 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09180ab7e0 con 0x7f091c159f40 2023-11-28T18:37:56.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.603+0000 7f09137fe640 1 -- 172.21.15.105:0/3058052108 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09180abb70 con 0x7f091c159f40 2023-11-28T18:37:56.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0923155640 1 -- 172.21.15.105:0/3058052108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f091c159f40 msgr2=0x7f091c0caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:56.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0923155640 1 --2- 172.21.15.105:0/3058052108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f091c159f40 0x7f091c0caf60 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f0918009870 tx=0x7f091801c480 comp rx=0 tx=0).stop 2023-11-28T18:37:56.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0923155640 1 -- 172.21.15.105:0/3058052108 shutdown_connections 2023-11-28T18:37:56.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0923155640 1 --2- 172.21.15.105:0/3058052108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f091c0ad850 0x7f091c0701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0923155640 1 --2- 172.21.15.105:0/3058052108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f091c159f40 0x7f091c0caf60 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0923155640 1 -- 172.21.15.105:0/3058052108 >> 172.21.15.105:0/3058052108 conn(0x7f091c0ceb90 msgr2=0x7f091c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:56.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0923155640 1 -- 172.21.15.105:0/3058052108 shutdown_connections 2023-11-28T18:37:56.605 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0923155640 1 -- 172.21.15.105:0/3058052108 wait complete. 2023-11-28T18:37:56.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0923155640 1 Processor -- start 2023-11-28T18:37:56.606 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0923155640 1 -- start start 2023-11-28T18:37:56.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0923155640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f091c0ad850 0x7f091c158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:56.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0923155640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f091c0cc760 0x7f091c154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:56.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0923155640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f091c159f40 0x7f091c1524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:56.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0923155640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c144f40 con 0x7f091c0ad850 2023-11-28T18:37:56.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0923155640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c153db0 con 0x7f091c0cc760 2023-11-28T18:37:56.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0923155640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091c1531a0 con 0x7f091c159f40 2023-11-28T18:37:56.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0920ef1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f091c0ad850 0x7f091c158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:56.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0920ef1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f091c0ad850 0x7f091c158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54526/0 (socket says 172.21.15.105:54526) 2023-11-28T18:37:56.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0920ef1640 1 -- 172.21.15.105:0/713220690 learned_addr learned my addr 172.21.15.105:0/713220690 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-11-28T18:37:56.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0913fff640 1 --2- 172.21.15.105:0/713220690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f091c0cc760 0x7f091c154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:56.609 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f09216f2640 1 --2- 172.21.15.105:0/713220690 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f091c159f40 0x7f091c1524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:56.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0920ef1640 1 -- 172.21.15.105:0/713220690 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f091c159f40 msgr2=0x7f091c1524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:56.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0920ef1640 1 --2- 172.21.15.105:0/713220690 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f091c159f40 0x7f091c1524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0920ef1640 1 -- 172.21.15.105:0/713220690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f091c0cc760 msgr2=0x7f091c154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:56.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0920ef1640 1 --2- 172.21.15.105:0/713220690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f091c0cc760 0x7f091c154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0920ef1640 1 -- 172.21.15.105:0/713220690 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09180094d0 con 0x7f091c0ad850 2023-11-28T18:37:56.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.607+0000 7f0913fff640 1 --2- 172.21.15.105:0/713220690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f091c0cc760 0x7f091c154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T18:37:56.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.611+0000 7f0920ef1640 1 --2- 172.21.15.105:0/713220690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f091c0ad850 0x7f091c158590 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f091801a0b0 tx=0x7f091801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:56.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.611+0000 7f09117fa640 1 -- 172.21.15.105:0/713220690 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09180b4800 con 0x7f091c0ad850 2023-11-28T18:37:56.610 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.611+0000 7f09117fa640 1 -- 172.21.15.105:0/713220690 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09180b4980 con 0x7f091c0ad850 2023-11-28T18:37:56.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.611+0000 7f09117fa640 1 -- 172.21.15.105:0/713220690 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09180b4d30 con 0x7f091c0ad850 2023-11-28T18:37:56.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.611+0000 7f0923155640 1 -- 172.21.15.105:0/713220690 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091c153480 con 0x7f091c0ad850 2023-11-28T18:37:56.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.611+0000 7f0923155640 1 -- 172.21.15.105:0/713220690 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091c153a40 con 0x7f091c0ad850 2023-11-28T18:37:56.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.611+0000 7f0923155640 1 -- 172.21.15.105:0/713220690 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091c067570 con 0x7f091c0ad850 2023-11-28T18:37:56.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.611+0000 7f09117fa640 1 -- 172.21.15.105:0/713220690 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 86170+0+0 (secure 0 0 0) 0x7f09180c7440 con 0x7f091c0ad850 2023-11-28T18:37:56.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.611+0000 7f09117fa640 1 --2- 172.21.15.105:0/713220690 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f09080668d0 0x7f0908068d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:56.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.611+0000 7f09117fa640 1 -- 172.21.15.105:0/713220690 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f091811f450 con 0x7f091c0ad850 2023-11-28T18:37:56.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.611+0000 7f0913fff640 1 --2- 172.21.15.105:0/713220690 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f09080668d0 0x7f0908068d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T18:37:56.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.615+0000 7f0913fff640 1 --2- 172.21.15.105:0/713220690 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f09080668d0 0x7f0908068d90 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f091c0acea0 tx=0x7f090c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T18:37:56.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.615+0000 7f09117fa640 1 -- 172.21.15.105:0/713220690 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f09180ec590 con 0x7f091c0ad850 2023-11-28T18:37:56.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.795+0000 7f0923155640 1 -- 172.21.15.105:0/713220690 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f091c0ce1a0 con 0x7f091c0ad850 2023-11-28T18:37:56.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.795+0000 7f09117fa640 1 -- 172.21.15.105:0/713220690 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 151 v151) v1 ==== 95+0+4219 (secure 0 0 0) 0x7f09180efbb0 con 0x7f091c0ad850 2023-11-28T18:37:56.796 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-11-28T18:37:56.796 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":151,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":151,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T18:31:58.827408+0000","modified":"2023-11-28T18:36:11.513093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9126,"mds_1":9117,"mds_2":9120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9117":{"gid":9117,"name":"a","rank":1,"incarnation":142,"state":"up:active","state_seq":21,"addr":"172.21.15.105:6835/3161918450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3161918450},{"type":"v1","addr":"172.21.15.105:6835","nonce":3161918450}]},"join_fscid":-1,"laggy_since":"2023-11-28T18:33:18.186653+0000","export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9120":{"gid":9120,"name":"d","rank":2,"incarnation":138,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6834/3958106803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3958106803},{"type":"v1","addr":"172.21.15.148:6834","nonce":3958106803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9126":{"gid":9126,"name":"c","rank":0,"incarnation":130,"state":"up:active","state_seq":66,"addr":"172.21.15.105:6837/1978722810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1978722810},{"type":"v1","addr":"172.21.15.105:6837","nonce":1978722810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-28T18:37:56.796 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 151 2023-11-28T18:37:56.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.799+0000 7f0923155640 1 -- 172.21.15.105:0/713220690 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f09080668d0 msgr2=0x7f0908068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:56.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.799+0000 7f0923155640 1 --2- 172.21.15.105:0/713220690 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f09080668d0 0x7f0908068d90 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f091c0acea0 tx=0x7f090c009040 comp rx=0 tx=0).stop 2023-11-28T18:37:56.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.799+0000 7f0923155640 1 -- 172.21.15.105:0/713220690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f091c0ad850 msgr2=0x7f091c158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T18:37:56.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.799+0000 7f0923155640 1 --2- 172.21.15.105:0/713220690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f091c0ad850 0x7f091c158590 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f091801a0b0 tx=0x7f091801acc0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.799+0000 7f0923155640 1 -- 172.21.15.105:0/713220690 shutdown_connections 2023-11-28T18:37:56.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.799+0000 7f0923155640 1 --2- 172.21.15.105:0/713220690 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f091c159f40 0x7f091c1524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.799+0000 7f0923155640 1 --2- 172.21.15.105:0/713220690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f091c0cc760 0x7f091c154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.799+0000 7f0923155640 1 --2- 172.21.15.105:0/713220690 >> [v2:172.21.15.105:6832/1892510418,v1:172.21.15.105:6833/1892510418] conn(0x7f09080668d0 0x7f0908068d90 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.799+0000 7f0923155640 1 --2- 172.21.15.105:0/713220690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f091c0ad850 0x7f091c158590 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:56.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.799+0000 7f0923155640 1 -- 172.21.15.105:0/713220690 >> 172.21.15.105:0/713220690 conn(0x7f091c0ceb90 msgr2=0x7f091c0ae770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T18:37:56.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.799+0000 7f0923155640 1 -- 172.21.15.105:0/713220690 shutdown_connections 2023-11-28T18:37:56.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.799+0000 7f0923155640 1 -- 172.21.15.105:0/713220690 wait complete. 2023-11-28T18:37:56.817 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T18:37:56.913 INFO:tasks.ceph.osd.0:Sent signal 15 2023-11-28T18:37:56.914 INFO:tasks.ceph.osd.1:Sent signal 15 2023-11-28T18:37:56.914 INFO:tasks.ceph.osd.2:Sent signal 15 2023-11-28T18:37:56.914 INFO:tasks.ceph.osd.3:Sent signal 15 2023-11-28T18:37:56.914 INFO:tasks.ceph.osd.4:Sent signal 15 2023-11-28T18:37:56.915 INFO:tasks.ceph.osd.5:Sent signal 15 2023-11-28T18:37:56.915 INFO:tasks.ceph.osd.6:Sent signal 15 2023-11-28T18:37:56.915 INFO:tasks.ceph.osd.7:Sent signal 15 2023-11-28T18:37:56.915 INFO:tasks.ceph.mds.c:Sent signal 15 2023-11-28T18:37:56.915 INFO:tasks.ceph.mds.d:Sent signal 15 2023-11-28T18:37:56.916 INFO:tasks.ceph.mon.a:Sent signal 15 2023-11-28T18:37:56.916 INFO:tasks.ceph.mon.b:Sent signal 15 2023-11-28T18:37:56.916 INFO:tasks.ceph.mon.c:Sent signal 15 2023-11-28T18:37:56.916 INFO:tasks.ceph.mgr.y:Sent signal 15 2023-11-28T18:37:56.917 INFO:tasks.ceph.mgr.x:Sent signal 15 2023-11-28T18:37:56.917 INFO:tasks.ceph.osd.4.smithi148.stderr:2023-11-28T18:37:56.922+0000 7f1b35c78640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 38543) UID: 0 2023-11-28T18:37:56.917 INFO:tasks.ceph.osd.4.smithi148.stderr:2023-11-28T18:37:56.922+0000 7f1b35c78640 -1 osd.4 130 *** Got signal Terminated *** 2023-11-28T18:37:56.917 INFO:tasks.ceph.osd.4.smithi148.stderr:2023-11-28T18:37:56.922+0000 7f1b35c78640 -1 osd.4 130 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-28T18:37:56.918 INFO:tasks.ceph.osd.5.smithi148.stderr:2023-11-28T18:37:56.922+0000 7f4817feb640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 38545) UID: 0 2023-11-28T18:37:56.918 INFO:tasks.ceph.osd.5.smithi148.stderr:2023-11-28T18:37:56.922+0000 7f4817feb640 -1 osd.5 130 *** Got signal Terminated *** 2023-11-28T18:37:56.918 INFO:tasks.ceph.osd.5.smithi148.stderr:2023-11-28T18:37:56.922+0000 7f4817feb640 -1 osd.5 130 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-28T18:37:56.918 INFO:tasks.ceph.osd.6.smithi148.stderr:2023-11-28T18:37:56.922+0000 7f8782cde640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 38547) UID: 0 2023-11-28T18:37:56.918 INFO:tasks.ceph.osd.6.smithi148.stderr:2023-11-28T18:37:56.922+0000 7f8782cde640 -1 osd.6 130 *** Got signal Terminated *** 2023-11-28T18:37:56.919 INFO:tasks.ceph.osd.6.smithi148.stderr:2023-11-28T18:37:56.922+0000 7f8782cde640 -1 osd.6 130 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-28T18:37:56.919 INFO:tasks.ceph.osd.7.smithi148.stderr:2023-11-28T18:37:56.922+0000 7fb84685a640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 38549) UID: 0 2023-11-28T18:37:56.919 INFO:tasks.ceph.osd.7.smithi148.stderr:2023-11-28T18:37:56.922+0000 7fb84685a640 -1 osd.7 130 *** Got signal Terminated *** 2023-11-28T18:37:56.919 INFO:tasks.ceph.osd.7.smithi148.stderr:2023-11-28T18:37:56.922+0000 7fb84685a640 -1 osd.7 130 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-28T18:37:56.919 INFO:tasks.ceph.mds.d.smithi148.stderr:2023-11-28T18:37:56.922+0000 7f150496b640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mds -f --cluster ceph -i d (PID: 45241) UID: 0 2023-11-28T18:37:56.920 INFO:tasks.ceph.mds.d.smithi148.stderr:2023-11-28T18:37:56.922+0000 7f150496b640 -1 mds.d *** got signal Terminated *** 2023-11-28T18:37:56.920 INFO:tasks.ceph.mon.b.smithi148.stderr:2023-11-28T18:37:56.922+0000 7ff3cd4bc640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mon -f --cluster ceph -i b (PID: 38229) UID: 0 2023-11-28T18:37:56.920 INFO:tasks.ceph.mon.b.smithi148.stderr:2023-11-28T18:37:56.922+0000 7ff3cd4bc640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2023-11-28T18:37:56.920 INFO:tasks.ceph.mon.c.smithi148.stderr:2023-11-28T18:37:56.922+0000 7f053b22d640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mon -f --cluster ceph -i c (PID: 38232) UID: 0 2023-11-28T18:37:56.920 INFO:tasks.ceph.mon.c.smithi148.stderr:2023-11-28T18:37:56.922+0000 7f053b22d640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2023-11-28T18:37:56.921 INFO:tasks.ceph.osd.0.smithi105.stderr:2023-11-28T18:37:56.915+0000 7fd864d28640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 38331) UID: 0 2023-11-28T18:37:56.921 INFO:tasks.ceph.osd.0.smithi105.stderr:2023-11-28T18:37:56.915+0000 7fd864d28640 -1 osd.0 130 *** Got signal Terminated *** 2023-11-28T18:37:56.921 INFO:tasks.ceph.osd.0.smithi105.stderr:2023-11-28T18:37:56.915+0000 7fd864d28640 -1 osd.0 130 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-28T18:37:56.921 INFO:tasks.ceph.osd.1.smithi105.stderr:2023-11-28T18:37:56.915+0000 7f7d66740640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 38329) UID: 0 2023-11-28T18:37:56.921 INFO:tasks.ceph.osd.1.smithi105.stderr:2023-11-28T18:37:56.915+0000 7f7d66740640 -1 osd.1 130 *** Got signal Terminated *** 2023-11-28T18:37:56.921 INFO:tasks.ceph.osd.1.smithi105.stderr:2023-11-28T18:37:56.915+0000 7f7d66740640 -1 osd.1 130 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-28T18:37:56.922 INFO:tasks.ceph.osd.2.smithi105.stderr:2023-11-28T18:37:56.915+0000 7f74ebde4640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 38330) UID: 0 2023-11-28T18:37:56.922 INFO:tasks.ceph.osd.2.smithi105.stderr:2023-11-28T18:37:56.915+0000 7f74ebde4640 -1 osd.2 130 *** Got signal Terminated *** 2023-11-28T18:37:56.922 INFO:tasks.ceph.osd.2.smithi105.stderr:2023-11-28T18:37:56.915+0000 7f74ebde4640 -1 osd.2 130 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-28T18:37:56.922 INFO:tasks.ceph.osd.3.smithi105.stderr:2023-11-28T18:37:56.915+0000 7fb5b0cbb640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 38335) UID: 0 2023-11-28T18:37:56.922 INFO:tasks.ceph.osd.3.smithi105.stderr:2023-11-28T18:37:56.915+0000 7fb5b0cbb640 -1 osd.3 130 *** Got signal Terminated *** 2023-11-28T18:37:56.922 INFO:tasks.ceph.osd.3.smithi105.stderr:2023-11-28T18:37:56.915+0000 7fb5b0cbb640 -1 osd.3 130 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-28T18:37:56.923 INFO:tasks.ceph.mds.c.smithi105.stderr:2023-11-28T18:37:56.919+0000 7ff0bf059640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mds -f --cluster ceph -i c (PID: 67168) UID: 0 2023-11-28T18:37:56.923 INFO:tasks.ceph.mds.c.smithi105.stderr:2023-11-28T18:37:56.919+0000 7ff0bf059640 -1 mds.c *** got signal Terminated *** 2023-11-28T18:37:56.923 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-11-28T18:37:56.919+0000 7fd8a0b35640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 38217) UID: 0 2023-11-28T18:37:56.923 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-11-28T18:37:56.919+0000 7fd8a0b35640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-11-28T18:37:56.924 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:37:56.919+0000 7fe464367640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mgr -f --cluster ceph -i y (PID: 38222) UID: 0 2023-11-28T18:37:56.924 INFO:tasks.ceph.mgr.y.smithi105.stderr:2023-11-28T18:37:56.919+0000 7fe464367640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-11-28T18:37:56.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.959+0000 7f7a9ac9c640 1 Processor -- start 2023-11-28T18:37:56.957 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.959+0000 7f7a9ac9c640 1 -- start start 2023-11-28T18:37:56.958 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.959+0000 7f7a9ac9c640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a940b4290 0x7f7a940b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:56.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.959+0000 7f7a9ac9c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a9415b030 0x7f7a940709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:56.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.959+0000 7f7a9ac9c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a940b4bb0 con 0x7f7a94071960 2023-11-28T18:37:56.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.959+0000 7f7a9ac9c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a9415b030 2023-11-28T18:37:56.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.959+0000 7f7a9ac9c640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9406df80 con 0x7f7a940b4290 2023-11-28T18:37:56.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.959+0000 7f7a93fff640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a94071960 legacy=0x7f7a9415f3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:37:56.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.959+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a940b4290 msgr2=0x7f7a940b4670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:37:56.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.959+0000 7f7a99239640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a9415b030 msgr2=0x7f7a940709b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:37:56.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.959+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a940b4290 0x7f7a940b4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:37:56.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:56.959+0000 7f7a99239640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a9415b030 0x7f7a940709b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:37:57.119 INFO:tasks.ceph.mgr.x.smithi148.stderr:daemon-helper: command crashed with signal 15 2023-11-28T18:37:57.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:57.163+0000 7f7a93fff640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a94071960 legacy=0x7f7a9415f3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:37:57.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:57.163+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a940b4290 msgr2=0x7f7a940b4670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:37:57.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:57.163+0000 7f7a99239640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a9415b030 msgr2=0x7f7a940709b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:37:57.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:57.163+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a940b4290 0x7f7a940b4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:37:57.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:57.163+0000 7f7a99239640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a9415b030 0x7f7a940709b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:37:57.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:57.567+0000 7f7a93fff640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a94071960 legacy=0x7f7a9415f3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:37:57.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:57.567+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a940b4290 msgr2=0x7f7a940b4670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:37:57.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:57.567+0000 7f7a99239640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a9415b030 msgr2=0x7f7a940709b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:37:57.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:57.567+0000 7f7a99239640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a9415b030 0x7f7a940709b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:37:57.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:57.567+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a940b4290 0x7f7a940b4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:37:58.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:58.371+0000 7f7a93fff640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a94071960 legacy=0x7f7a9415f3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:37:58.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:58.371+0000 7f7a99239640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a9415b030 msgr2=0x7f7a940709b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:37:58.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:58.371+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a940b4290 msgr2=0x7f7a940b4670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:37:58.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:58.371+0000 7f7a99239640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a9415b030 0x7f7a940709b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:37:58.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:58.371+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a940b4290 0x7f7a940b4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:37:59.959 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:59.959+0000 7f7a927fc640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a940b4290 msgr2=0x7f7a940b4670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:37:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:59.959+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a940b4290 0x7f7a940b4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:59.959+0000 7f7a927fc640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a9415b030 msgr2=0x7f7a940709b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:37:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:59.959+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a9415b030 0x7f7a940709b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:37:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:59.959+0000 7f7a927fc640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a94071960 legacy=0x7f7a9415f3b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:37:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:59.959+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780036d0 0x7f7a78005b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:37:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:59.959+0000 7f7a927fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a94071df0 con 0x7f7a780061a0 2023-11-28T18:37:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:59.959+0000 7f7a927fc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a780036d0 2023-11-28T18:37:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:59.959+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780036d0 msgr2=0x7f7a78005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:37:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:59.959+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780036d0 0x7f7a78005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:37:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:37:59.959+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a780061a0 legacy=0x7f7a78008660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:00.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:00.163+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780036d0 msgr2=0x7f7a78005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:00.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:00.163+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780036d0 0x7f7a78005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:00.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:00.163+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a780061a0 legacy=0x7f7a78008660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:00.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:00.567+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780036d0 msgr2=0x7f7a78005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:00.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:00.567+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780036d0 0x7f7a78005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:00.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:00.567+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a780061a0 legacy=0x7f7a78008660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:01.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:01.371+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780036d0 msgr2=0x7f7a78005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:01.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:01.371+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780036d0 0x7f7a78005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:01.368 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:01.371+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a780061a0 legacy=0x7f7a78008660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:02.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:02.963+0000 7f7a927fc640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780036d0 msgr2=0x7f7a78005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:02.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:02.963+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780036d0 0x7f7a78005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:02.960 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:02.963+0000 7f7a927fc640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a780061a0 legacy=0x7f7a78008660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:02.963+0000 7f7a99239640 1 -- reap_dead start 2023-11-28T18:38:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:02.963+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78009540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:02.963+0000 7f7a927fc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a940b4bb0 con 0x7f7a7800aac0 2023-11-28T18:38:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:02.963+0000 7f7a927fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a94071df0 con 0x7f7a78009f00 2023-11-28T18:38:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:02.963+0000 7f7a927fc640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78005c10 2023-11-28T18:38:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:02.963+0000 7f7a93fff640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800aac0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:02.963+0000 7f7a98a38640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78009f00 legacy=0x7f7a7800a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:02.963+0000 7f7a99239640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:02.963+0000 7f7a99239640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:03.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:03.167+0000 7f7a93fff640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800aac0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:03.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:03.167+0000 7f7a99239640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:03.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:03.167+0000 7f7a98a38640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78009f00 legacy=0x7f7a7800a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:03.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:03.167+0000 7f7a99239640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:03.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:03.571+0000 7f7a93fff640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800aac0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:03.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:03.571+0000 7f7a99239640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:03.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:03.571+0000 7f7a98a38640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78009f00 legacy=0x7f7a7800a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:03.569 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:03.571+0000 7f7a99239640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:04.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:04.375+0000 7f7a93fff640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800aac0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:04.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:04.375+0000 7f7a98a38640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78009f00 legacy=0x7f7a7800a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:04.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:04.375+0000 7f7a99239640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:04.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:04.375+0000 7f7a99239640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:04.639 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:38:04.642 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:38:05.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:05.963+0000 7f7a927fc640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:05.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:05.963+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:05.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:05.963+0000 7f7a927fc640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78009f00 legacy=0x7f7a7800a2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:05.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:05.963+0000 7f7a927fc640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800aac0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:05.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:05.963+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78008890 0x7f7a780075e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:05.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:05.963+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78007b20 0x7f7a78007f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:05.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:05.963+0000 7f7a927fc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78007b20 2023-11-28T18:38:05.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:05.963+0000 7f7a927fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9406df80 con 0x7f7a78008890 2023-11-28T18:38:05.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:05.963+0000 7f7a93fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78007b20 msgr2=0x7f7a78007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:05.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:05.963+0000 7f7a93fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78007b20 0x7f7a78007f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:05.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:05.963+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78008890 msgr2=0x7f7a780075e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:05.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:05.963+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78008890 0x7f7a780075e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:06.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:06.167+0000 7f7a93fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78007b20 msgr2=0x7f7a78007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:06.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:06.167+0000 7f7a93fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78007b20 0x7f7a78007f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:06.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:06.167+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78008890 msgr2=0x7f7a780075e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:06.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:06.167+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78008890 0x7f7a780075e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:06.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:06.571+0000 7f7a93fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78007b20 msgr2=0x7f7a78007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:06.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:06.571+0000 7f7a93fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78007b20 0x7f7a78007f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:06.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:06.571+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78008890 msgr2=0x7f7a780075e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:06.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:06.571+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78008890 0x7f7a780075e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:07.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:07.375+0000 7f7a93fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78007b20 msgr2=0x7f7a78007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:07.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:07.375+0000 7f7a93fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78007b20 0x7f7a78007f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:07.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:07.375+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78008890 msgr2=0x7f7a780075e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:07.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:07.375+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78008890 0x7f7a780075e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:08.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:08.963+0000 7f7a927fc640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78008890 msgr2=0x7f7a780075e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:08.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:08.963+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78008890 0x7f7a780075e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:08.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:08.963+0000 7f7a927fc640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78007b20 msgr2=0x7f7a78007f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:08.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:08.963+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78007b20 0x7f7a78007f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:08.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:08.963+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780141b0 0x7f7a780165a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:08.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:08.963+0000 7f7a99239640 1 -- reap_dead start 2023-11-28T18:38:08.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:08.963+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78009540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:08.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:08.963+0000 7f7a927fc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a780085f0 con 0x7f7a7800aac0 2023-11-28T18:38:08.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:08.963+0000 7f7a927fc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a780141b0 2023-11-28T18:38:08.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:08.963+0000 7f7a927fc640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9406df80 con 0x7f7a78005c10 2023-11-28T18:38:08.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:08.963+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780141b0 msgr2=0x7f7a780165a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:08.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:08.963+0000 7f7a93fff640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800aac0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:08.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:08.963+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780141b0 0x7f7a780165a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:08.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:08.963+0000 7f7a99239640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:08.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:08.963+0000 7f7a99239640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:09.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:09.167+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780141b0 msgr2=0x7f7a780165a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:09.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:09.167+0000 7f7a93fff640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800aac0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:09.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:09.167+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780141b0 0x7f7a780165a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:09.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:09.167+0000 7f7a99239640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:09.165 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:09.167+0000 7f7a99239640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:09.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:09.571+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780141b0 msgr2=0x7f7a780165a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:09.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:09.571+0000 7f7a93fff640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800aac0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:09.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:09.571+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780141b0 0x7f7a780165a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:09.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:09.571+0000 7f7a99239640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:09.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:09.571+0000 7f7a99239640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:10.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:10.375+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780141b0 msgr2=0x7f7a780165a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:10.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:10.375+0000 7f7a93fff640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800aac0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:10.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:10.375+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780141b0 0x7f7a780165a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:10.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:10.375+0000 7f7a99239640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:10.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:10.375+0000 7f7a99239640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:11.961 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:11.963+0000 7f7a927fc640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:11.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:11.963+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:11.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:11.963+0000 7f7a927fc640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780141b0 msgr2=0x7f7a780165a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:11.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:11.963+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780141b0 0x7f7a780165a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:11.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:11.963+0000 7f7a927fc640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800aac0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:11.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:11.963+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78007ff0 0x7f7a78004b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:11.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:11.963+0000 7f7a927fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a940b4bb0 con 0x7f7a780076a0 2023-11-28T18:38:11.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:11.963+0000 7f7a927fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78007ff0 2023-11-28T18:38:11.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:11.963+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78007ff0 msgr2=0x7f7a78004b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:11.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:11.963+0000 7f7a98a38640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a780076a0 legacy=0x7f7a7800a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:11.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:11.963+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78007ff0 0x7f7a78004b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:12.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:12.167+0000 7f7a98a38640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a780076a0 legacy=0x7f7a7800a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:12.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:12.167+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78007ff0 msgr2=0x7f7a78004b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:12.164 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:12.167+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78007ff0 0x7f7a78004b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:12.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:12.571+0000 7f7a98a38640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a780076a0 legacy=0x7f7a7800a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:12.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:12.571+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78007ff0 msgr2=0x7f7a78004b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:12.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:12.571+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78007ff0 0x7f7a78004b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:13.375+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78007ff0 msgr2=0x7f7a78004b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:13.375+0000 7f7a98a38640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a780076a0 legacy=0x7f7a7800a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:13.375+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78007ff0 0x7f7a78004b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:14.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:14.963+0000 7f7a927fc640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78007ff0 msgr2=0x7f7a78004b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:14.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:14.963+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78007ff0 0x7f7a78004b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:14.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:14.963+0000 7f7a927fc640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a780076a0 legacy=0x7f7a7800a2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:14.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:14.963+0000 7f7a99239640 1 -- reap_dead start 2023-11-28T18:38:14.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:14.963+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780141b0 0x7f7a780165a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:14.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:14.963+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78009540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:14.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:14.963+0000 7f7a927fc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a780085f0 con 0x7f7a7800aac0 2023-11-28T18:38:14.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:14.963+0000 7f7a927fc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a780141b0 2023-11-28T18:38:14.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:14.963+0000 7f7a927fc640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9406df80 con 0x7f7a78005c10 2023-11-28T18:38:14.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:14.963+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780141b0 msgr2=0x7f7a780165a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:14.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:14.963+0000 7f7a93fff640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800aac0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:14.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:14.963+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780141b0 0x7f7a780165a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:14.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:14.963+0000 7f7a99239640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:14.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:14.963+0000 7f7a99239640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:15.162 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:15.167+0000 7f7a93fff640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800aac0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:15.162 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:15.167+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780141b0 msgr2=0x7f7a780165a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:15.162 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:15.167+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780141b0 0x7f7a780165a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:15.162 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:15.167+0000 7f7a99239640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:15.163 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:15.167+0000 7f7a99239640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:15.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:15.571+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780141b0 msgr2=0x7f7a780165a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:15.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:15.571+0000 7f7a93fff640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800aac0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:15.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:15.571+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780141b0 0x7f7a780165a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:15.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:15.571+0000 7f7a99239640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:15.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:15.571+0000 7f7a99239640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:16.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:16.375+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780141b0 msgr2=0x7f7a780165a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:16.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:16.375+0000 7f7a93fff640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800aac0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:16.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:16.375+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780141b0 0x7f7a780165a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:16.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:16.375+0000 7f7a99239640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:16.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:16.375+0000 7f7a99239640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:17.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:17.963+0000 7f7a927fc640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:17.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:17.963+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:17.962 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:17.963+0000 7f7a927fc640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780141b0 msgr2=0x7f7a780165a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:17.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:17.963+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780141b0 0x7f7a780165a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:17.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:17.963+0000 7f7a927fc640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800aac0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:17.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:17.963+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a7800a2e0 0x7f7a78005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:17.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:17.963+0000 7f7a927fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a940b4bb0 con 0x7f7a780056a0 2023-11-28T18:38:17.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:17.963+0000 7f7a927fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a7800a2e0 2023-11-28T18:38:17.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:17.963+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a7800a2e0 msgr2=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:17.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:17.963+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a780056a0 legacy=0x7f7a780110f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:17.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:17.963+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a7800a2e0 0x7f7a78005160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:18.163 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:18.167+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a7800a2e0 msgr2=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:18.163 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:18.167+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a780056a0 legacy=0x7f7a780110f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:18.163 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:18.167+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a7800a2e0 0x7f7a78005160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:18.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:18.571+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a780056a0 legacy=0x7f7a780110f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:18.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:18.571+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a7800a2e0 msgr2=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:18.568 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:18.571+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a7800a2e0 0x7f7a78005160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:19.372 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:19.375+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a780056a0 legacy=0x7f7a780110f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:19.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:19.375+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a7800a2e0 msgr2=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:19.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:19.375+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a7800a2e0 0x7f7a78005160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:20.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:20.967+0000 7f7a927fc640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a7800a2e0 msgr2=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:20.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:20.967+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a7800a2e0 0x7f7a78005160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:20.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:20.967+0000 7f7a927fc640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a780056a0 legacy=0x7f7a780110f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:20.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:20.967+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78004b50 0x7f7a78011800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:20.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:20.967+0000 7f7a99239640 1 -- reap_dead start 2023-11-28T18:38:20.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:20.967+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:20.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:20.967+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780056a0 0x7f7a78009540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:20.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:20.967+0000 7f7a927fc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a780056a0 2023-11-28T18:38:20.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:20.967+0000 7f7a927fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9406df80 con 0x7f7a78004b50 2023-11-28T18:38:20.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:20.967+0000 7f7a927fc640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7800a810 con 0x7f7a78005c10 2023-11-28T18:38:20.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:20.967+0000 7f7a99239640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780056a0 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:20.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:20.967+0000 7f7a99239640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780056a0 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:20.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:20.967+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:20.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:20.967+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78004b50 msgr2=0x7f7a78011800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:20.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:20.967+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78005160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:20.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:20.967+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78004b50 0x7f7a78011800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:21.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:21.171+0000 7f7a99239640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780056a0 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:21.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:21.171+0000 7f7a99239640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780056a0 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:21.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:21.171+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:21.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:21.171+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78004b50 msgr2=0x7f7a78011800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:21.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:21.171+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78005160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:21.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:21.171+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78004b50 0x7f7a78011800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:21.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:21.575+0000 7f7a99239640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780056a0 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:21.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:21.575+0000 7f7a99239640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780056a0 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:21.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:21.575+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78004b50 msgr2=0x7f7a78011800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:21.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:21.575+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:21.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:21.575+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78005160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:21.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:21.575+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78004b50 0x7f7a78011800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:22.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:22.379+0000 7f7a99239640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780056a0 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:22.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:22.379+0000 7f7a99239640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780056a0 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:22.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:22.379+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78004b50 msgr2=0x7f7a78011800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:22.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:22.379+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:22.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:22.379+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78005160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:22.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:22.379+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78004b50 0x7f7a78011800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:23.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:23.967+0000 7f7a927fc640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:23.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:23.967+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78005160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:23.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:23.967+0000 7f7a927fc640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78004b50 msgr2=0x7f7a78011800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:23.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:23.967+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78004b50 0x7f7a78011800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:23.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:23.967+0000 7f7a927fc640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780056a0 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:23.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:23.967+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780056a0 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:23.963 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:23.967+0000 7f7a927fc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a780085f0 con 0x7f7a7800aac0 2023-11-28T18:38:23.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:23.967+0000 7f7a927fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a940b4bb0 con 0x7f7a78013d50 2023-11-28T18:38:23.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:23.967+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800aac0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:23.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:23.967+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78013d50 legacy=0x7f7a780161c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:24.171+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800aac0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:24.171+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78013d50 legacy=0x7f7a780161c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:24.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:24.575+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800aac0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:24.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:24.575+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78013d50 legacy=0x7f7a780161c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:25.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:25.379+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800aac0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:25.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:25.379+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78013d50 legacy=0x7f7a780161c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:26.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:26.967+0000 7f7a927fc640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78013d50 legacy=0x7f7a780161c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:26.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:26.967+0000 7f7a927fc640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800aac0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:26.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:26.967+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:26.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:26.967+0000 7f7a99239640 1 -- reap_dead start 2023-11-28T18:38:26.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:26.967+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78009540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:26.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:26.967+0000 7f7a927fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a94071df0 con 0x7f7a780056a0 2023-11-28T18:38:26.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:26.967+0000 7f7a927fc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a7800aac0 2023-11-28T18:38:26.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:26.967+0000 7f7a927fc640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9406df80 con 0x7f7a78005c10 2023-11-28T18:38:26.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:26.967+0000 7f7a99239640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:26.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:26.967+0000 7f7a99239640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:26.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:26.967+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a780056a0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:26.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:26.967+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:26.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:26.967+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78005160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:27.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:27.171+0000 7f7a99239640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:27.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:27.171+0000 7f7a99239640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:27.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:27.171+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:27.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:27.171+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a780056a0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:27.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:27.171+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78005160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:27.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:27.575+0000 7f7a99239640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:27.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:27.575+0000 7f7a99239640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:27.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:27.575+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a780056a0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:27.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:27.575+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:27.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:27.575+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78005160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:28.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:28.379+0000 7f7a99239640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:28.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:28.379+0000 7f7a99239640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:28.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:28.379+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a780056a0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:28.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:28.379+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:28.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:28.379+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78005160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:29.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:29.967+0000 7f7a927fc640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 msgr2=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:29.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:29.967+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78005c10 0x7f7a78005160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:29.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:29.967+0000 7f7a927fc640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:29.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:29.967+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:29.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:29.967+0000 7f7a927fc640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a780056a0 legacy=0x7f7a78008df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:29.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:29.967+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013810 0x7f7a780161c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:29.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:29.967+0000 7f7a927fc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a780085f0 con 0x7f7a78003b00 2023-11-28T18:38:29.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:29.967+0000 7f7a927fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78013810 2023-11-28T18:38:29.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:29.967+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78003b00 legacy=0x7f7a78013100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:29.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:29.967+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013810 msgr2=0x7f7a780161c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:29.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:29.967+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013810 0x7f7a780161c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:30.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:30.171+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78003b00 legacy=0x7f7a78013100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:30.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:30.171+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013810 msgr2=0x7f7a780161c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:30.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:30.171+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013810 0x7f7a780161c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:30.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:30.575+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78003b00 legacy=0x7f7a78013100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:30.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:30.575+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013810 msgr2=0x7f7a780161c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:30.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:30.575+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013810 0x7f7a780161c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:31.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:31.379+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78003b00 legacy=0x7f7a78013100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:31.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:31.379+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013810 msgr2=0x7f7a780161c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:31.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:31.379+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013810 0x7f7a780161c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:32.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:32.967+0000 7f7a927fc640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013810 msgr2=0x7f7a780161c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:32.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:32.967+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013810 0x7f7a780161c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:32.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:32.967+0000 7f7a927fc640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78003b00 legacy=0x7f7a78013100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:32.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:32.967+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78009540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:32.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:32.967+0000 7f7a99239640 1 -- reap_dead start 2023-11-28T18:38:32.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:32.967+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78008d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:32.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:32.967+0000 7f7a927fc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a94071df0 con 0x7f7a78005c10 2023-11-28T18:38:32.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:32.967+0000 7f7a927fc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a7800aac0 2023-11-28T18:38:32.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:32.967+0000 7f7a927fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9406df80 con 0x7f7a78003b00 2023-11-28T18:38:32.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:32.967+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:32.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:32.967+0000 7f7a99239640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:32.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:32.967+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:32.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:32.967+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a78008d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:32.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:32.967+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78008d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:33.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:33.171+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:33.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:33.171+0000 7f7a99239640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:33.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:33.171+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:33.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:33.171+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a78008d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:33.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:33.171+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78008d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:33.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:33.575+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:33.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:33.575+0000 7f7a99239640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:33.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:33.575+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:33.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:33.575+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a78008d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:33.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:33.575+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78008d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:34.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:34.379+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:34.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:34.379+0000 7f7a99239640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:34.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:34.379+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:34.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:34.379+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a78008d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:34.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:34.379+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78008d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:34.702 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:38:34.707 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:38:35.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:35.967+0000 7f7a927fc640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a78008d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:35.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:35.967+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78008d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:35.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:35.967+0000 7f7a927fc640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:35.964 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:35.967+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:35.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:35.967+0000 7f7a927fc640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:35.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:35.967+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 0x7f7a780134e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:35.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:35.967+0000 7f7a927fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a780085f0 con 0x7f7a78003ee0 2023-11-28T18:38:35.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:35.967+0000 7f7a927fc640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78013100 2023-11-28T18:38:35.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:35.967+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78003ee0 legacy=0x7f7a780042c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:35.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:35.967+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 msgr2=0x7f7a780134e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:35.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:35.967+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 0x7f7a780134e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:36.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:36.171+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 msgr2=0x7f7a780134e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:36.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:36.171+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78003ee0 legacy=0x7f7a780042c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:36.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:36.171+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 0x7f7a780134e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:36.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:36.575+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 msgr2=0x7f7a780134e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:36.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:36.575+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78003ee0 legacy=0x7f7a780042c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:36.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:36.575+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 0x7f7a780134e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:37.379+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 msgr2=0x7f7a780134e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:37.379+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78003ee0 legacy=0x7f7a780042c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:37.379+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 0x7f7a780134e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:38.967+0000 7f7a927fc640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 msgr2=0x7f7a780134e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:38.967+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 0x7f7a780134e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:38.967+0000 7f7a927fc640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78003ee0 legacy=0x7f7a780042c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:38.967+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 0x7f7a780134e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:38.967+0000 7f7a99239640 1 -- reap_dead start 2023-11-28T18:38:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:38.967+0000 7f7a927fc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a94071df0 con 0x7f7a78005c10 2023-11-28T18:38:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:38.967+0000 7f7a927fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a780085f0 con 0x7f7a78003ee0 2023-11-28T18:38:38.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:38.967+0000 7f7a927fc640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78013100 2023-11-28T18:38:38.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:38.967+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:38.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:38.967+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 msgr2=0x7f7a780134e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:38.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:38.967+0000 7f7a99239640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78003ee0 legacy=0x7f7a780042c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:38.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:38.967+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 0x7f7a780134e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:39.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:39.171+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:39.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:39.171+0000 7f7a99239640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78003ee0 legacy=0x7f7a780042c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:39.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:39.171+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 msgr2=0x7f7a780134e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:39.168 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:39.171+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 0x7f7a780134e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:39.575+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:39.575+0000 7f7a99239640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78003ee0 legacy=0x7f7a780042c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:39.575+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 msgr2=0x7f7a780134e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:39.575+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 0x7f7a780134e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:40.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:40.379+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:40.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:40.379+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 msgr2=0x7f7a780134e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:40.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:40.379+0000 7f7a99239640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78003ee0 legacy=0x7f7a780042c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:40.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:40.379+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 0x7f7a780134e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:41.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:41.967+0000 7f7a927fc640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 msgr2=0x7f7a780134e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:44.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:41.967+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 0x7f7a780134e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:44.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:41.967+0000 7f7a927fc640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78003ee0 legacy=0x7f7a780042c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:44.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:41.967+0000 7f7a927fc640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:44.305 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:41.967+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78009540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:44.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:41.967+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a780049d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:44.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:41.967+0000 7f7a927fc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a7800aac0 2023-11-28T18:38:44.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:41.967+0000 7f7a927fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9406df80 con 0x7f7a78003b00 2023-11-28T18:38:44.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:41.967+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:44.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:41.967+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:44.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:41.967+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a780049d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:44.306 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:41.967+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a780049d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:44.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:42.171+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:44.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:42.171+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:44.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:42.171+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a780049d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:44.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:42.171+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a780049d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:44.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:42.575+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:44.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:42.575+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:44.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:42.575+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a780049d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:44.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:42.575+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a780049d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:44.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:43.379+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:44.308 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:43.379+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:44.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:43.379+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a780049d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:44.309 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:43.379+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a780049d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:44.965 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:44.967+0000 7f7a927fc640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a780049d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:44.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:44.967+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a780049d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:44.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:44.967+0000 7f7a927fc640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 msgr2=0x7f7a78009540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:44.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:44.967+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78009540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:44.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:44.967+0000 7f7a99239640 1 -- reap_dead start 2023-11-28T18:38:44.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:44.967+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a780049d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:44.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:44.967+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 0x7f7a780134e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:44.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:44.967+0000 7f7a927fc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a940b4bb0 con 0x7f7a78005c10 2023-11-28T18:38:44.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:44.967+0000 7f7a927fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78003b00 2023-11-28T18:38:44.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:44.967+0000 7f7a927fc640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9406df80 con 0x7f7a78013100 2023-11-28T18:38:44.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:44.967+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:44.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:44.967+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a780049d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:44.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:44.967+0000 7f7a99239640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 msgr2=0x7f7a780134e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:44.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:44.967+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a780049d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:44.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:44.967+0000 7f7a99239640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 0x7f7a780134e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:45.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:45.171+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:45.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:45.171+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a780049d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:45.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:45.171+0000 7f7a99239640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 msgr2=0x7f7a780134e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:45.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:45.171+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a780049d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:45.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:45.171+0000 7f7a99239640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 0x7f7a780134e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:45.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:45.575+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:45.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:45.575+0000 7f7a99239640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 msgr2=0x7f7a780134e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:45.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:45.575+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a780049d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:45.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:45.575+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a780049d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:45.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:45.575+0000 7f7a99239640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 0x7f7a780134e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:46.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:46.379+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:46.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:46.379+0000 7f7a99239640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 msgr2=0x7f7a780134e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:46.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:46.379+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a780049d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:46.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:46.379+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a780049d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:46.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:46.379+0000 7f7a99239640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 0x7f7a780134e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:47.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:47.971+0000 7f7a927fc640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 msgr2=0x7f7a780134e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:47.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:47.971+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78013100 0x7f7a780134e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:47.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:47.971+0000 7f7a927fc640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a780049d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:47.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:47.971+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a780049d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:47.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:47.971+0000 7f7a927fc640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:47.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:47.971+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:47.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:47.971+0000 7f7a927fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a94071df0 con 0x7f7a78004420 2023-11-28T18:38:47.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:47.971+0000 7f7a927fc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a7800aac0 2023-11-28T18:38:47.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:47.971+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 msgr2=0x7f7a78003ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:47.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:47.971+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78003ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:47.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:47.971+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78004420 legacy=0x7f7a78008d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:48.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:48.175+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 msgr2=0x7f7a78003ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:48.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:48.175+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78003ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:48.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:48.175+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78004420 legacy=0x7f7a78008d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:48.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:48.579+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 msgr2=0x7f7a78003ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:48.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:48.579+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78003ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:48.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:48.579+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78004420 legacy=0x7f7a78008d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:49.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:49.383+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 msgr2=0x7f7a78003ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:49.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:49.383+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78003ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:49.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:49.383+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78004420 legacy=0x7f7a78008d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:50.967+0000 7f7a927fc640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 msgr2=0x7f7a78003ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:50.967+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800aac0 0x7f7a78003ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:50.967+0000 7f7a927fc640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78004420 legacy=0x7f7a78008d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:50.967+0000 7f7a99239640 1 -- reap_dead start 2023-11-28T18:38:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:50.967+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78009420 0x7f7a780191a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:50.967+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:50.967+0000 7f7a927fc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a940b4bb0 con 0x7f7a78005c10 2023-11-28T18:38:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:50.967+0000 7f7a927fc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78009420 2023-11-28T18:38:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:50.967+0000 7f7a927fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9406df80 con 0x7f7a78003b00 2023-11-28T18:38:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:50.967+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78009420 msgr2=0x7f7a780191a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:50.967+0000 7f7a99239640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:50.967+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78009420 0x7f7a780191a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:50.967+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a78003ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:50.967+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78003ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:51.171+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78009420 msgr2=0x7f7a780191a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:51.171+0000 7f7a99239640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:51.171+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78009420 0x7f7a780191a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:51.171+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a78003ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:51.171+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78003ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:51.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:51.575+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78009420 msgr2=0x7f7a780191a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:51.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:51.575+0000 7f7a99239640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:51.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:51.575+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78009420 0x7f7a780191a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:51.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:51.575+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a78003ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:51.572 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:51.575+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78003ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:52.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:52.378+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78009420 msgr2=0x7f7a780191a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:52.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:52.378+0000 7f7a99239640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:52.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:52.378+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78009420 0x7f7a780191a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:52.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:52.378+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a78003ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:52.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:52.378+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78003ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:53.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:53.970+0000 7f7a927fc640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a78003ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:53.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:53.970+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78003ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:53.966 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:53.970+0000 7f7a927fc640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78009420 msgr2=0x7f7a780191a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:53.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:53.970+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78009420 0x7f7a780191a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:53.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:53.970+0000 7f7a927fc640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:53.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:53.970+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004420 0x7f7a78004970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:53.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:53.970+0000 7f7a927fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a94071df0 con 0x7f7a78008d10 2023-11-28T18:38:53.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:53.970+0000 7f7a927fc640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78004420 2023-11-28T18:38:53.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:53.970+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78008d10 legacy=0x7f7a78013100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:53.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:53.970+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004420 msgr2=0x7f7a78004970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:53.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:53.970+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004420 0x7f7a78004970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:54.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:54.174+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78008d10 legacy=0x7f7a78013100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:54.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:54.174+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004420 msgr2=0x7f7a78004970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:54.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:54.174+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004420 0x7f7a78004970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:54.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:54.578+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004420 msgr2=0x7f7a78004970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:54.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:54.578+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78008d10 legacy=0x7f7a78013100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:54.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:54.578+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004420 0x7f7a78004970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:55.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:55.382+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004420 msgr2=0x7f7a78004970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:55.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:55.382+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78008d10 legacy=0x7f7a78013100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:55.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:55.382+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004420 0x7f7a78004970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:56.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:56.970+0000 7f7a927fc640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004420 msgr2=0x7f7a78004970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:56.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:56.970+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004420 0x7f7a78004970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:56.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:56.970+0000 7f7a927fc640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78008d10 legacy=0x7f7a78013100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:56.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:56.970+0000 7f7a99239640 1 -- reap_dead start 2023-11-28T18:38:56.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:56.970+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78015180 0x7f7a78015560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:56.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:56.970+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:56.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:56.970+0000 7f7a927fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a940b4bb0 con 0x7f7a78008d10 2023-11-28T18:38:56.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:56.970+0000 7f7a927fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78003b00 2023-11-28T18:38:56.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:56.970+0000 7f7a927fc640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9406df80 con 0x7f7a78015180 2023-11-28T18:38:56.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:56.970+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78008d10 legacy=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:56.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:56.970+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78015180 msgr2=0x7f7a78015560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:56.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:56.970+0000 7f7a99239640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a78003ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:56.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:56.970+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78015180 0x7f7a78015560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:56.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:56.970+0000 7f7a99239640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78003ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:38:57.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:57.174+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78015180 msgr2=0x7f7a78015560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:57.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:57.174+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78015180 0x7f7a78015560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:57.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:57.174+0000 7f7a99239640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a78003ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:57.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:57.174+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78008d10 legacy=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:57.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:57.174+0000 7f7a99239640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78003ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:38:57.578 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:57.578+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78015180 msgr2=0x7f7a78015560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:57.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:57.578+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78008d10 legacy=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:57.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:57.578+0000 7f7a99239640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a78003ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:57.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:57.578+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78015180 0x7f7a78015560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:57.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:57.578+0000 7f7a99239640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78003ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:38:58.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:58.382+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78015180 msgr2=0x7f7a78015560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:38:58.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:58.382+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78008d10 legacy=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:38:58.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:58.382+0000 7f7a99239640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a78003ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:38:58.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:58.382+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78015180 0x7f7a78015560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:58.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:58.382+0000 7f7a99239640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78003ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:38:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:59.970+0000 7f7a927fc640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78015180 msgr2=0x7f7a78015560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:59.970+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78015180 0x7f7a78015560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:59.970+0000 7f7a927fc640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a78003ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:59.970+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78003ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:38:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:59.970+0000 7f7a927fc640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78008d10 legacy=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:38:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:59.970+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:38:59.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:59.970+0000 7f7a927fc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a94071df0 con 0x7f7a78005c10 2023-11-28T18:38:59.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:59.970+0000 7f7a927fc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78004970 2023-11-28T18:38:59.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:59.970+0000 7f7a93fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:38:59.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:59.970+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:38:59.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:38:59.970+0000 7f7a93fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:00.171 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:00.174+0000 7f7a93fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:00.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:00.174+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:00.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:00.174+0000 7f7a93fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:00.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:00.578+0000 7f7a93fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:00.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:00.578+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:00.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:00.578+0000 7f7a93fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:01.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:01.382+0000 7f7a93fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:01.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:01.382+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:01.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:01.382+0000 7f7a93fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:02.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:02.970+0000 7f7a927fc640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:02.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:02.970+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:02.970+0000 7f7a927fc640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:02.970+0000 7f7a99239640 1 -- reap_dead start 2023-11-28T18:39:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:02.970+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:02.970+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:02.970+0000 7f7a927fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a940b4bb0 con 0x7f7a78008d10 2023-11-28T18:39:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:02.970+0000 7f7a927fc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78004970 2023-11-28T18:39:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:02.970+0000 7f7a927fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9406df80 con 0x7f7a78003b00 2023-11-28T18:39:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:02.970+0000 7f7a99239640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:02.970+0000 7f7a99239640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:02.970+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a78003ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:02.970+0000 7f7a98a38640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78008d10 legacy=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:02.970+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78003ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:03.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:03.174+0000 7f7a99239640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:03.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:03.174+0000 7f7a99239640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:03.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:03.174+0000 7f7a98a38640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78008d10 legacy=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:03.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:03.174+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a78003ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:03.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:03.174+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78003ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:03.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:03.578+0000 7f7a99239640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:03.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:03.578+0000 7f7a99239640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:03.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:03.578+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a78003ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:03.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:03.578+0000 7f7a98a38640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78008d10 legacy=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:03.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:03.578+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78003ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:04.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:04.382+0000 7f7a99239640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:04.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:04.382+0000 7f7a99239640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:04.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:04.382+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a78003ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:04.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:04.382+0000 7f7a98a38640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78008d10 legacy=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:04.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:04.382+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78003ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:05.026 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:39:05.030 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:39:05.967 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:05.970+0000 7f7a927fc640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 msgr2=0x7f7a78003ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:05.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:05.970+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78003b00 0x7f7a78003ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:05.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:05.970+0000 7f7a927fc640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:05.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:05.970+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:05.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:05.970+0000 7f7a927fc640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78008d10 legacy=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:05.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:05.970+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a780090f0 0x7f7a780094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:05.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:05.970+0000 7f7a927fc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a94071df0 con 0x7f7a78005c10 2023-11-28T18:39:05.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:05.970+0000 7f7a927fc640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a780090f0 2023-11-28T18:39:05.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:05.970+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:05.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:05.970+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a780090f0 msgr2=0x7f7a780094d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:05.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:05.970+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a780090f0 0x7f7a780094d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:06.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:06.174+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:06.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:06.174+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a780090f0 msgr2=0x7f7a780094d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:06.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:06.174+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a780090f0 0x7f7a780094d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:06.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:06.578+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:06.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:06.578+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a780090f0 msgr2=0x7f7a780094d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:06.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:06.578+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a780090f0 0x7f7a780094d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:07.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:07.382+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:07.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:07.382+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a780090f0 msgr2=0x7f7a780094d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:07.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:07.382+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a780090f0 0x7f7a780094d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:08.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:08.970+0000 7f7a927fc640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a780090f0 msgr2=0x7f7a780094d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:08.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:08.970+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a780090f0 0x7f7a780094d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:08.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:08.970+0000 7f7a927fc640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78005c10 legacy=0x7f7a78005120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:08.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:08.970+0000 7f7a99239640 1 -- reap_dead start 2023-11-28T18:39:08.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:08.970+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:08.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:08.970+0000 7f7a927fc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a940b4bb0 con 0x7f7a78013060 2023-11-28T18:39:08.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:08.970+0000 7f7a927fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a94071df0 con 0x7f7a78005120 2023-11-28T18:39:08.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:08.970+0000 7f7a927fc640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78004970 2023-11-28T18:39:08.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:08.970+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013060 legacy=0x7f7a78014dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:08.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:08.970+0000 7f7a99239640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78005120 legacy=0x7f7a78003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:08.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:08.970+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78013480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:08.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:08.970+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:09.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:09.174+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013060 legacy=0x7f7a78014dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:09.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:09.174+0000 7f7a99239640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78005120 legacy=0x7f7a78003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:09.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:09.174+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78013480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:09.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:09.174+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:09.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:09.578+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013060 legacy=0x7f7a78014dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:09.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:09.578+0000 7f7a99239640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78005120 legacy=0x7f7a78003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:09.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:09.578+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78013480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:09.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:09.578+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:10.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:10.382+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013060 legacy=0x7f7a78014dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:10.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:10.382+0000 7f7a99239640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78005120 legacy=0x7f7a78003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:10.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:10.382+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78013480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:10.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:10.382+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:11.970+0000 7f7a927fc640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78013480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:11.970+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:11.970+0000 7f7a927fc640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78005120 legacy=0x7f7a78003ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:11.970+0000 7f7a927fc640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013060 legacy=0x7f7a78014dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:11.970+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78005520 0x7f7a78014170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:11.970+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a780146b0 0x7f7a78008d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:11.970+0000 7f7a927fc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78005520 2023-11-28T18:39:11.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:11.970+0000 7f7a927fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9406df80 con 0x7f7a780146b0 2023-11-28T18:39:11.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:11.970+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78005520 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:11.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:11.970+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78005520 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:11.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:11.970+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a780146b0 msgr2=0x7f7a78008d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:11.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:11.970+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a780146b0 0x7f7a78008d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:12.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:12.174+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78005520 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:12.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:12.174+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78005520 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:12.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:12.174+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a780146b0 msgr2=0x7f7a78008d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:12.172 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:12.174+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a780146b0 0x7f7a78008d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:12.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:12.578+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78005520 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:12.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:12.578+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78005520 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:12.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:12.578+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a780146b0 msgr2=0x7f7a78008d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:12.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:12.578+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a780146b0 0x7f7a78008d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:13.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:13.382+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78005520 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:13.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:13.382+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78005520 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:13.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:13.382+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a780146b0 msgr2=0x7f7a78008d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:13.378 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:13.382+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a780146b0 0x7f7a78008d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:14.968 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:14.970+0000 7f7a927fc640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a780146b0 msgr2=0x7f7a78008d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:14.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:14.970+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a780146b0 0x7f7a78008d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:14.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:14.970+0000 7f7a927fc640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78005520 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:14.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:14.970+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78005520 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:14.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:14.970+0000 7f7a99239640 1 -- reap_dead start 2023-11-28T18:39:14.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:14.970+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:14.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:14.970+0000 7f7a927fc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a780085f0 con 0x7f7a78013060 2023-11-28T18:39:14.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:14.970+0000 7f7a927fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a940b4bb0 con 0x7f7a78005120 2023-11-28T18:39:14.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:14.970+0000 7f7a927fc640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78004970 2023-11-28T18:39:14.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:14.970+0000 7f7a99239640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013060 legacy=0x7f7a78003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:14.970+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78013480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:14.970+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78005120 legacy=0x7f7a78009250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:14.970+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:15.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:15.174+0000 7f7a99239640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013060 legacy=0x7f7a78003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:15.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:15.174+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78005120 legacy=0x7f7a78009250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:15.170 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:15.174+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78013480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:15.170 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:15.174+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:15.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:15.578+0000 7f7a99239640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013060 legacy=0x7f7a78003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:15.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:15.578+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78005120 legacy=0x7f7a78009250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:15.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:15.578+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78013480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:15.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:15.578+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:16.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:16.382+0000 7f7a99239640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013060 legacy=0x7f7a78003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:16.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:16.382+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78005120 legacy=0x7f7a78009250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:16.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:16.382+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78013480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:16.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:16.382+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:17.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:17.970+0000 7f7a927fc640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78013480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:17.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:17.970+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:17.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:17.970+0000 7f7a927fc640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78005120 legacy=0x7f7a78009250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:17.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:17.970+0000 7f7a927fc640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013060 legacy=0x7f7a78003ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:17.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:17.970+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78008d10 0x7f7a78014170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:17.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:17.970+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780146b0 0x7f7a78017b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:17.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:17.970+0000 7f7a927fc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a780146b0 2023-11-28T18:39:17.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:17.970+0000 7f7a927fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9406df80 con 0x7f7a78008d10 2023-11-28T18:39:17.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:17.970+0000 7f7a93fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780146b0 msgr2=0x7f7a78017b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:17.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:17.970+0000 7f7a93fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780146b0 0x7f7a78017b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:17.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:17.970+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78008d10 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:17.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:17.970+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78008d10 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:18.169 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:18.174+0000 7f7a93fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780146b0 msgr2=0x7f7a78017b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:18.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:18.174+0000 7f7a93fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780146b0 0x7f7a78017b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:18.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:18.174+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78008d10 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:18.198 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:18.174+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78008d10 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:18.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:18.578+0000 7f7a93fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780146b0 msgr2=0x7f7a78017b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:18.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:18.578+0000 7f7a93fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780146b0 0x7f7a78017b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:18.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:18.578+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78008d10 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:18.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:18.578+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78008d10 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:19.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:19.382+0000 7f7a93fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780146b0 msgr2=0x7f7a78017b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:19.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:19.382+0000 7f7a93fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780146b0 0x7f7a78017b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:19.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:19.382+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78008d10 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:19.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:19.382+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78008d10 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:20.969 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:20.974+0000 7f7a927fc640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78008d10 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:20.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:20.974+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78008d10 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:20.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:20.974+0000 7f7a927fc640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a780146b0 msgr2=0x7f7a78017b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:20.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:20.974+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a780146b0 0x7f7a78017b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:20.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:20.974+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a780180c0 0x7f7a7801a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:20.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:20.974+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7801a9f0 0x7f7a7801add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:20.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:20.974+0000 7f7a99239640 1 -- reap_dead start 2023-11-28T18:39:20.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:20.974+0000 7f7a927fc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a94071df0 con 0x7f7a78013060 2023-11-28T18:39:20.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:20.974+0000 7f7a927fc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a7801a9f0 2023-11-28T18:39:20.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:20.974+0000 7f7a927fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9406df80 con 0x7f7a780180c0 2023-11-28T18:39:20.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:20.974+0000 7f7a99239640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7801a9f0 msgr2=0x7f7a7801add0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:20.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:20.974+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013060 legacy=0x7f7a78003ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:20.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:20.974+0000 7f7a99239640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7801a9f0 0x7f7a7801add0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:20.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:20.974+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a780180c0 msgr2=0x7f7a7801a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:20.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:20.974+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a780180c0 0x7f7a7801a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:21.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:21.178+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013060 legacy=0x7f7a78003ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:21.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:21.178+0000 7f7a99239640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7801a9f0 msgr2=0x7f7a7801add0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:21.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:21.178+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a780180c0 msgr2=0x7f7a7801a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:21.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:21.178+0000 7f7a99239640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7801a9f0 0x7f7a7801add0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:21.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:21.178+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a780180c0 0x7f7a7801a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:21.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:21.582+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013060 legacy=0x7f7a78003ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:21.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:21.582+0000 7f7a99239640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7801a9f0 msgr2=0x7f7a7801add0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:21.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:21.582+0000 7f7a99239640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7801a9f0 0x7f7a7801add0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:21.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:21.582+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a780180c0 msgr2=0x7f7a7801a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:21.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:21.582+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a780180c0 0x7f7a7801a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:22.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:22.386+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013060 legacy=0x7f7a78003ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:22.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:22.386+0000 7f7a99239640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7801a9f0 msgr2=0x7f7a7801add0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:22.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:22.386+0000 7f7a99239640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7801a9f0 0x7f7a7801add0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:22.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:22.386+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a780180c0 msgr2=0x7f7a7801a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:22.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:22.386+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a780180c0 0x7f7a7801a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:23.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:23.974+0000 7f7a927fc640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a780180c0 msgr2=0x7f7a7801a4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:23.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:23.974+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a780180c0 0x7f7a7801a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:23.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:23.974+0000 7f7a927fc640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7801a9f0 msgr2=0x7f7a7801add0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:23.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:23.974+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7801a9f0 0x7f7a7801add0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:23.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:23.974+0000 7f7a927fc640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013060 legacy=0x7f7a78003ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:23.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:23.974+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:23.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:23.974+0000 7f7a927fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a780085f0 con 0x7f7a78005120 2023-11-28T18:39:23.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:23.974+0000 7f7a927fc640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78004970 2023-11-28T18:39:23.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:23.974+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78013480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:23.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:23.974+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78005120 legacy=0x7f7a780091c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:23.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:23.974+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:24.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:24.178+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78005120 legacy=0x7f7a780091c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:24.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:24.178+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78013480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:24.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:24.178+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:24.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:24.582+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78005120 legacy=0x7f7a780091c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:24.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:24.582+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78013480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:24.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:24.582+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:25.386+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78005120 legacy=0x7f7a780091c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:25.386+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78013480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:25.386+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:26.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:26.974+0000 7f7a927fc640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78013480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:26.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:26.974+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:26.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:26.974+0000 7f7a927fc640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78005120 legacy=0x7f7a780091c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:26.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:26.974+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:26.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:26.974+0000 7f7a99239640 1 -- reap_dead start 2023-11-28T18:39:26.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:26.974+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a780180c0 0x7f7a7801a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:26.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:26.974+0000 7f7a927fc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a94071df0 con 0x7f7a78013060 2023-11-28T18:39:26.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:26.974+0000 7f7a927fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a780180c0 2023-11-28T18:39:26.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:26.974+0000 7f7a927fc640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9406df80 con 0x7f7a78004970 2023-11-28T18:39:26.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:26.974+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013060 legacy=0x7f7a78003ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:26.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:26.974+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78013480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:26.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:26.974+0000 7f7a99239640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a780180c0 msgr2=0x7f7a7801a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:26.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:26.974+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:26.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:26.974+0000 7f7a99239640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a780180c0 0x7f7a7801a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:27.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:27.178+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013060 legacy=0x7f7a78003ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:27.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:27.178+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78013480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:27.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:27.178+0000 7f7a99239640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a780180c0 msgr2=0x7f7a7801a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:27.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:27.178+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:27.177 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:27.178+0000 7f7a99239640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a780180c0 0x7f7a7801a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:27.582+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013060 legacy=0x7f7a78003ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:27.582+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78013480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:27.582+0000 7f7a99239640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a780180c0 msgr2=0x7f7a7801a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:27.582+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:27.582+0000 7f7a99239640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a780180c0 0x7f7a7801a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:28.380 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:28.386+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013060 legacy=0x7f7a78003ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:28.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:28.386+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78013480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:28.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:28.386+0000 7f7a99239640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a780180c0 msgr2=0x7f7a7801a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:28.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:28.386+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:28.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:28.386+0000 7f7a99239640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a780180c0 0x7f7a7801a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:29.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:29.974+0000 7f7a927fc640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78013480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:29.970 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:29.974+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78013480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:29.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:29.974+0000 7f7a927fc640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a780180c0 msgr2=0x7f7a7801a4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:29.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:29.974+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a780180c0 0x7f7a7801a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:29.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:29.974+0000 7f7a927fc640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013060 legacy=0x7f7a78003ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:29.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:29.974+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78005120 0x7f7a78014170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:29.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:29.974+0000 7f7a927fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a780085f0 con 0x7f7a78017b60 2023-11-28T18:39:29.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:29.974+0000 7f7a927fc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78005120 2023-11-28T18:39:29.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:29.974+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78005120 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:29.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:29.974+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78005120 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:29.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:29.974+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78017b60 legacy=0x7f7a78009190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:30.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:30.178+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78005120 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:30.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:30.178+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78005120 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:30.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:30.178+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78017b60 legacy=0x7f7a78009190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:30.579 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:30.582+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78005120 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:30.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:30.582+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78005120 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:30.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:30.582+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78017b60 legacy=0x7f7a78009190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:31.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:31.386+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78005120 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:31.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:31.386+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78005120 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:31.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:31.386+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78017b60 legacy=0x7f7a78009190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:32.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:32.974+0000 7f7a927fc640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78005120 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:32.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:32.974+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78005120 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:32.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:32.974+0000 7f7a927fc640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78017b60 legacy=0x7f7a78009190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:32.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:32.974+0000 7f7a99239640 1 -- reap_dead start 2023-11-28T18:39:32.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:32.974+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800f040 0x7f7a7800f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:32.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:32.974+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:32.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:32.974+0000 7f7a927fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a94071df0 con 0x7f7a78017b60 2023-11-28T18:39:32.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:32.974+0000 7f7a927fc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a7800f040 2023-11-28T18:39:32.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:32.974+0000 7f7a927fc640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9406df80 con 0x7f7a78004970 2023-11-28T18:39:32.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:32.974+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800f040 msgr2=0x7f7a7800f420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:32.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:32.974+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800f040 0x7f7a7800f420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:32.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:32.974+0000 7f7a99239640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78017b60 legacy=0x7f7a78009190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:32.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:32.974+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:32.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:32.974+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:33.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:33.178+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800f040 msgr2=0x7f7a7800f420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:33.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:33.178+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800f040 0x7f7a7800f420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:33.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:33.178+0000 7f7a99239640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78017b60 legacy=0x7f7a78009190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:33.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:33.178+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:33.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:33.178+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:33.582+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800f040 msgr2=0x7f7a7800f420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:33.582+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800f040 0x7f7a7800f420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:33.582+0000 7f7a99239640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78017b60 legacy=0x7f7a78009190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:33.582+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:33.582+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:34.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:34.386+0000 7f7a98a38640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800f040 msgr2=0x7f7a7800f420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:34.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:34.386+0000 7f7a98a38640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800f040 0x7f7a7800f420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:34.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:34.386+0000 7f7a93fff640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:34.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:34.386+0000 7f7a99239640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78017b60 legacy=0x7f7a78009190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:34.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:34.386+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:35.088 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:39:35.092 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:39:35.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:35.974+0000 7f7a927fc640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:35.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:35.974+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:35.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:35.974+0000 7f7a927fc640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800f040 msgr2=0x7f7a7800f420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:35.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:35.974+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a7800f040 0x7f7a7800f420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:35.971 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:35.974+0000 7f7a927fc640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78017b60 legacy=0x7f7a78009190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:35.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:35.974+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78017f70 0x7f7a7801a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:35.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:35.974+0000 7f7a927fc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a780085f0 con 0x7f7a78013480 2023-11-28T18:39:35.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:35.974+0000 7f7a927fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78017f70 2023-11-28T18:39:35.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:35.974+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013480 legacy=0x7f7a78003ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:35.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:35.974+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78017f70 msgr2=0x7f7a7801a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:35.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:35.974+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78017f70 0x7f7a7801a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:36.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:36.178+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013480 legacy=0x7f7a78003ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:36.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:36.178+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78017f70 msgr2=0x7f7a7801a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:36.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:36.178+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78017f70 0x7f7a7801a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:36.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:36.582+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013480 legacy=0x7f7a78003ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:36.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:36.582+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78017f70 msgr2=0x7f7a7801a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:36.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:36.582+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78017f70 0x7f7a7801a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:37.386+0000 7f7a98a38640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013480 legacy=0x7f7a78003ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:37.386+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78017f70 msgr2=0x7f7a7801a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:37.386+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78017f70 0x7f7a7801a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:38.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:38.974+0000 7f7a927fc640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78017f70 msgr2=0x7f7a7801a360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:38.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:38.974+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78017f70 0x7f7a7801a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:38.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:38.974+0000 7f7a927fc640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013480 legacy=0x7f7a78003ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:38.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:38.974+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a7800f960 0x7f7a78011d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:38.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:38.974+0000 7f7a99239640 1 -- reap_dead start 2023-11-28T18:39:38.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:38.974+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:38.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:38.974+0000 7f7a927fc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a94071df0 con 0x7f7a78013480 2023-11-28T18:39:38.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:38.974+0000 7f7a927fc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78004970 2023-11-28T18:39:38.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:38.974+0000 7f7a927fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9406df80 con 0x7f7a7800f960 2023-11-28T18:39:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:38.974+0000 7f7a93fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:38.974+0000 7f7a99239640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013480 legacy=0x7f7a78009190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:38.974+0000 7f7a93fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:38.974+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a7800f960 msgr2=0x7f7a78011d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:38.974+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a7800f960 0x7f7a78011d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:39.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:39.178+0000 7f7a93fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:39.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:39.178+0000 7f7a99239640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013480 legacy=0x7f7a78009190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:39.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:39.178+0000 7f7a93fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:39.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:39.178+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a7800f960 msgr2=0x7f7a78011d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:39.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:39.178+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a7800f960 0x7f7a78011d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:39.582+0000 7f7a99239640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013480 legacy=0x7f7a78009190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:39.582+0000 7f7a93fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:39.582+0000 7f7a93fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:39.582+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a7800f960 msgr2=0x7f7a78011d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:39.582+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a7800f960 0x7f7a78011d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:40.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:40.386+0000 7f7a93fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:40.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:40.386+0000 7f7a99239640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013480 legacy=0x7f7a78009190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:40.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:40.386+0000 7f7a93fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:40.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:40.386+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a7800f960 msgr2=0x7f7a78011d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:40.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:40.386+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a7800f960 0x7f7a78011d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:41.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:41.974+0000 7f7a927fc640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a7800f960 msgr2=0x7f7a78011d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:41.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:41.974+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a7800f960 0x7f7a78011d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:41.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:41.974+0000 7f7a927fc640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:41.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:41.974+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:41.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:41.974+0000 7f7a927fc640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78013480 legacy=0x7f7a78009190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:41.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:41.974+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78003ab0 0x7f7a78003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:41.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:41.974+0000 7f7a927fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a780085f0 con 0x7f7a7800f040 2023-11-28T18:39:41.972 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:41.974+0000 7f7a927fc640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78003ab0 2023-11-28T18:39:41.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:41.974+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78003ab0 msgr2=0x7f7a78003ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:41.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:41.974+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a7800f040 legacy=0x7f7a78012290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:41.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:41.974+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78003ab0 0x7f7a78003ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:42.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:42.178+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a7800f040 legacy=0x7f7a78012290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:42.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:42.178+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78003ab0 msgr2=0x7f7a78003ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:42.176 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:42.178+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78003ab0 0x7f7a78003ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:42.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:42.582+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78003ab0 msgr2=0x7f7a78003ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:42.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:42.582+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a7800f040 legacy=0x7f7a78012290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:42.577 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:42.582+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78003ab0 0x7f7a78003ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:43.383 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:43.386+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78003ab0 msgr2=0x7f7a78003ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:43.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:43.386+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a7800f040 legacy=0x7f7a78012290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:43.384 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:43.386+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78003ab0 0x7f7a78003ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:44.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:44.970+0000 7f7a927fc640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78003ab0 msgr2=0x7f7a78003ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:44.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:44.974+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78003ab0 0x7f7a78003ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:44.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:44.974+0000 7f7a927fc640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a7800f040 legacy=0x7f7a78012290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:44.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:44.974+0000 7f7a99239640 1 -- reap_dead start 2023-11-28T18:39:44.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:44.974+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78003ab0 0x7f7a78003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:44.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:44.974+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:44.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:44.974+0000 7f7a927fc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a94071df0 con 0x7f7a7800f040 2023-11-28T18:39:44.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:44.974+0000 7f7a927fc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78004970 2023-11-28T18:39:44.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:44.974+0000 7f7a927fc640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9406df80 con 0x7f7a78003ab0 2023-11-28T18:39:44.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:44.974+0000 7f7a99239640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800f040 legacy=0x7f7a780090f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:44.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:44.974+0000 7f7a93fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:44.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:44.974+0000 7f7a93fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:44.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:44.974+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78003ab0 msgr2=0x7f7a78003e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:44.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:44.974+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78003ab0 0x7f7a78003e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:45.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:45.178+0000 7f7a99239640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800f040 legacy=0x7f7a780090f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:45.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:45.178+0000 7f7a93fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:45.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:45.178+0000 7f7a93fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:45.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:45.178+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78003ab0 msgr2=0x7f7a78003e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:45.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:45.178+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78003ab0 0x7f7a78003e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:45.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:45.582+0000 7f7a99239640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800f040 legacy=0x7f7a780090f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:45.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:45.582+0000 7f7a93fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:45.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:45.582+0000 7f7a93fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:45.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:45.582+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78003ab0 msgr2=0x7f7a78003e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:45.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:45.582+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78003ab0 0x7f7a78003e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:46.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:46.386+0000 7f7a99239640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800f040 legacy=0x7f7a780090f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:46.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:46.386+0000 7f7a93fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:46.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:46.386+0000 7f7a93fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:46.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:46.386+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78003ab0 msgr2=0x7f7a78003e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:46.382 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:46.386+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78003ab0 0x7f7a78003e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:47.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:47.974+0000 7f7a927fc640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78003ab0 msgr2=0x7f7a78003e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:47.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:47.974+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78003ab0 0x7f7a78003e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:47.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:47.974+0000 7f7a927fc640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:47.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:47.974+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:47.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:47.974+0000 7f7a927fc640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a7800f040 legacy=0x7f7a780090f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:47.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:47.974+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013480 0x7f7a78016b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:47.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:47.974+0000 7f7a927fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a780085f0 con 0x7f7a78017090 2023-11-28T18:39:47.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:47.974+0000 7f7a927fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78013480 2023-11-28T18:39:47.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:47.974+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78017090 legacy=0x7f7a780174a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:47.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:47.974+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013480 msgr2=0x7f7a78016b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:47.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:47.974+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013480 0x7f7a78016b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:48.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:48.178+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013480 msgr2=0x7f7a78016b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:48.173 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:48.178+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78017090 legacy=0x7f7a780174a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:48.174 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:48.178+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013480 0x7f7a78016b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:48.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:48.582+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78017090 legacy=0x7f7a780174a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:48.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:48.582+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013480 msgr2=0x7f7a78016b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:48.582 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:48.582+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013480 0x7f7a78016b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:49.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:49.386+0000 7f7a93fff640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78017090 legacy=0x7f7a780174a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:49.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:49.386+0000 7f7a98a38640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013480 msgr2=0x7f7a78016b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:49.381 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:49.386+0000 7f7a98a38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013480 0x7f7a78016b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:50.978+0000 7f7a927fc640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013480 msgr2=0x7f7a78016b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:50.978+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013480 0x7f7a78016b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:50.978+0000 7f7a927fc640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78017090 legacy=0x7f7a780174a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:50.978+0000 7f7a99239640 1 -- reap_dead start 2023-11-28T18:39:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:50.978+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013480 0x7f7a78016b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:50.978+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:50.978+0000 7f7a927fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a94071df0 con 0x7f7a78012490 2023-11-28T18:39:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:50.978+0000 7f7a927fc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78004970 2023-11-28T18:39:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:50.978+0000 7f7a927fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9406df80 con 0x7f7a78013480 2023-11-28T18:39:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:50.978+0000 7f7a99239640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:50.978+0000 7f7a99239640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:50.978+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013480 msgr2=0x7f7a78016b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:50.978+0000 7f7a98a38640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78012490 legacy=0x7f7a78012870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:50.978+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013480 0x7f7a78016b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:51.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:51.182+0000 7f7a99239640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:51.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:51.182+0000 7f7a99239640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:51.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:51.182+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013480 msgr2=0x7f7a78016b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:51.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:51.182+0000 7f7a98a38640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78012490 legacy=0x7f7a78012870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:51.181 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:51.182+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013480 0x7f7a78016b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:51.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:51.586+0000 7f7a99239640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:51.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:51.586+0000 7f7a99239640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:51.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:51.586+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013480 msgr2=0x7f7a78016b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:51.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:51.586+0000 7f7a98a38640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78012490 legacy=0x7f7a78012870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:51.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:51.586+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013480 0x7f7a78016b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:52.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:52.390+0000 7f7a99239640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:39:52.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:52.390+0000 7f7a99239640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:52.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:52.390+0000 7f7a93fff640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013480 msgr2=0x7f7a78016b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:52.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:52.390+0000 7f7a98a38640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78012490 legacy=0x7f7a78012870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:39:52.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:52.390+0000 7f7a93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013480 0x7f7a78016b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:53.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:53.977+0000 7f7a927fc640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013480 msgr2=0x7f7a78016b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:53.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:53.977+0000 7f7a927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7a78013480 0x7f7a78016b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:53.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:53.977+0000 7f7a927fc640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 msgr2=0x7f7a78014170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:53.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:53.977+0000 7f7a927fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a78004970 0x7f7a78014170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:39:53.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:53.977+0000 7f7a927fc640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f7a78012490 legacy=0x7f7a78012870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:39:53.973 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:53.977+0000 7f7a927fc640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78012f80 0x7f7a780090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:53.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:53.977+0000 7f7a927fc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a780085f0 con 0x7f7a78003ab0 2023-11-28T18:39:53.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:53.977+0000 7f7a927fc640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a940701d0 con 0x7f7a78012f80 2023-11-28T18:39:53.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:53.977+0000 7f7a93fff640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78003ab0 legacy=0x7f7a780174a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:53.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:53.977+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78012f80 msgr2=0x7f7a780090f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:53.974 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:53.977+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78012f80 0x7f7a780090f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:54.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:54.181+0000 7f7a93fff640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78003ab0 legacy=0x7f7a780174a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:54.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:54.181+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78012f80 msgr2=0x7f7a780090f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:54.180 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:54.181+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78012f80 0x7f7a780090f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:54.580 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:54.585+0000 7f7a93fff640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78003ab0 legacy=0x7f7a780174a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:54.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:54.585+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78012f80 msgr2=0x7f7a780090f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:54.581 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:54.585+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78012f80 0x7f7a780090f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:55.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:55.389+0000 7f7a93fff640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f7a78003ab0 legacy=0x7f7a780174a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:55.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:55.389+0000 7f7a98a38640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78012f80 msgr2=0x7f7a780090f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:55.388 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:55.389+0000 7f7a98a38640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a78012f80 0x7f7a780090f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:56.846 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-11-28T18:39:56.848 INFO:tasks.cephfs_test_runner:That ephemerally pinned subtrees are somewhat evenly distributed. ... ERROR 2023-11-28T18:39:56.850 INFO:tasks.cephfs_test_runner:ERROR 2023-11-28T18:39:56.850 INFO:tasks.cephfs_test_runner: 2023-11-28T18:39:56.850 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T18:39:56.851 INFO:tasks.cephfs_test_runner:ERROR: test_ephemeral_pin_distribution (tasks.cephfs.test_exports.TestEphemeralPins) 2023-11-28T18:39:56.851 INFO:tasks.cephfs_test_runner:That ephemerally pinned subtrees are somewhat evenly distributed. 2023-11-28T18:39:56.851 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T18:39:56.851 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T18:39:56.851 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 747, in recv_stderr 2023-11-28T18:39:56.851 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2023-11-28T18:39:56.851 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/buffered_pipe.py", line 164, in read 2023-11-28T18:39:56.851 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2023-11-28T18:39:56.851 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2023-11-28T18:39:56.851 INFO:tasks.cephfs_test_runner: 2023-11-28T18:39:56.851 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2023-11-28T18:39:56.851 INFO:tasks.cephfs_test_runner: 2023-11-28T18:39:56.852 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T18:39:56.852 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/test_exports.py", line 436, in test_ephemeral_pin_distribution 2023-11-28T18:39:56.852 INFO:tasks.cephfs_test_runner: self._setup_tree(count=1000, distributed=True) 2023-11-28T18:39:56.852 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/test_exports.py", line 293, in _setup_tree 2023-11-28T18:39:56.852 INFO:tasks.cephfs_test_runner: return self.mount_a.run_shell_payload(f""" 2023-11-28T18:39:56.852 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/mount.py", line 793, in run_shell_payload 2023-11-28T18:39:56.852 INFO:tasks.cephfs_test_runner: return self.run_shell(**kwargs) 2023-11-28T18:39:56.852 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/mount.py", line 784, in run_shell 2023-11-28T18:39:56.852 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-11-28T18:39:56.852 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/remote.py", line 523, in run 2023-11-28T18:39:56.852 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-11-28T18:39:56.852 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 455, in run 2023-11-28T18:39:56.852 INFO:tasks.cephfs_test_runner: r.wait() 2023-11-28T18:39:56.852 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 148, in wait 2023-11-28T18:39:56.852 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2023-11-28T18:39:56.853 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2023-11-28T18:39:56.853 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2023-11-28T18:39:56.853 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/gevent/_compat.py", line 65, in reraise 2023-11-28T18:39:56.853 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2023-11-28T18:39:56.853 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2023-11-28T18:39:56.853 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 323, in copy_file_to 2023-11-28T18:39:56.853 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2023-11-28T18:39:56.853 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 276, in copy_to_log 2023-11-28T18:39:56.853 INFO:tasks.cephfs_test_runner: for line in f: 2023-11-28T18:39:56.853 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 125, in __next__ 2023-11-28T18:39:56.853 INFO:tasks.cephfs_test_runner: line = self.readline() 2023-11-28T18:39:56.853 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 291, in readline 2023-11-28T18:39:56.853 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2023-11-28T18:39:56.853 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 1376, in _read 2023-11-28T18:39:56.854 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2023-11-28T18:39:56.854 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 749, in recv_stderr 2023-11-28T18:39:56.854 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2023-11-28T18:39:56.854 INFO:tasks.cephfs_test_runner:socket.timeout 2023-11-28T18:39:56.854 INFO:tasks.cephfs_test_runner: 2023-11-28T18:39:56.854 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T18:39:56.854 INFO:tasks.cephfs_test_runner:ERROR: test_ephemeral_pin_distribution (tasks.cephfs.test_exports.TestEphemeralPins) 2023-11-28T18:39:56.854 INFO:tasks.cephfs_test_runner:That ephemerally pinned subtrees are somewhat evenly distributed. 2023-11-28T18:39:56.854 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T18:39:56.854 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T18:39:56.854 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/cephfs_test_case.py", line 208, in tearDown 2023-11-28T18:39:56.854 INFO:tasks.cephfs_test_runner: self.mds_cluster.delete_all_filesystems() 2023-11-28T18:39:56.855 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 494, in delete_all_filesystems 2023-11-28T18:39:56.855 INFO:tasks.cephfs_test_runner: Filesystem(ctx=self._ctx, fscid=fs['id']).destroy() 2023-11-28T18:39:56.855 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 840, in destroy 2023-11-28T18:39:56.855 INFO:tasks.cephfs_test_runner: data_pools = self.get_data_pool_names(refresh=True) 2023-11-28T18:39:56.855 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 970, in get_data_pool_names 2023-11-28T18:39:56.855 INFO:tasks.cephfs_test_runner: self.get_pool_names(refresh = True) 2023-11-28T18:39:56.855 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 939, in get_pool_names 2023-11-28T18:39:56.855 INFO:tasks.cephfs_test_runner: status = self.status() 2023-11-28T18:39:56.855 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 396, in status 2023-11-28T18:39:56.855 INFO:tasks.cephfs_test_runner: return FSStatus(self.mon_manager, epoch) 2023-11-28T18:39:56.855 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 88, in __init__ 2023-11-28T18:39:56.855 INFO:tasks.cephfs_test_runner: self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) 2023-11-28T18:39:56.855 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2023-11-28T18:39:56.855 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2023-11-28T18:39:56.856 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2023-11-28T18:39:56.856 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2023-11-28T18:39:56.856 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd 2023-11-28T18:39:56.856 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-11-28T18:39:56.856 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/remote.py", line 523, in run 2023-11-28T18:39:56.856 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-11-28T18:39:56.856 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 455, in run 2023-11-28T18:39:56.856 INFO:tasks.cephfs_test_runner: r.wait() 2023-11-28T18:39:56.856 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 161, in wait 2023-11-28T18:39:56.856 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-11-28T18:39:56.856 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-11-28T18:39:56.856 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-11-28T18:39:56.856 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' 2023-11-28T18:39:56.856 INFO:tasks.cephfs_test_runner: 2023-11-28T18:39:56.857 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T18:39:56.857 INFO:tasks.cephfs_test_runner:Ran 8 tests in 936.876s 2023-11-28T18:39:56.857 INFO:tasks.cephfs_test_runner: 2023-11-28T18:39:56.857 INFO:tasks.cephfs_test_runner:FAILED (errors=2) 2023-11-28T18:39:56.857 INFO:tasks.cephfs_test_runner: 2023-11-28T18:39:56.857 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T18:39:56.857 INFO:tasks.cephfs_test_runner:ERROR: test_ephemeral_pin_distribution (tasks.cephfs.test_exports.TestEphemeralPins) 2023-11-28T18:39:56.857 INFO:tasks.cephfs_test_runner:That ephemerally pinned subtrees are somewhat evenly distributed. 2023-11-28T18:39:56.857 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T18:39:56.857 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T18:39:56.857 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 747, in recv_stderr 2023-11-28T18:39:56.857 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2023-11-28T18:39:56.858 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/buffered_pipe.py", line 164, in read 2023-11-28T18:39:56.858 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2023-11-28T18:39:56.858 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2023-11-28T18:39:56.858 INFO:tasks.cephfs_test_runner: 2023-11-28T18:39:56.858 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2023-11-28T18:39:56.858 INFO:tasks.cephfs_test_runner: 2023-11-28T18:39:56.858 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T18:39:56.858 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/test_exports.py", line 436, in test_ephemeral_pin_distribution 2023-11-28T18:39:56.858 INFO:tasks.cephfs_test_runner: self._setup_tree(count=1000, distributed=True) 2023-11-28T18:39:56.858 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/test_exports.py", line 293, in _setup_tree 2023-11-28T18:39:56.858 INFO:tasks.cephfs_test_runner: return self.mount_a.run_shell_payload(f""" 2023-11-28T18:39:56.858 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/mount.py", line 793, in run_shell_payload 2023-11-28T18:39:56.858 INFO:tasks.cephfs_test_runner: return self.run_shell(**kwargs) 2023-11-28T18:39:56.858 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/mount.py", line 784, in run_shell 2023-11-28T18:39:56.859 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-11-28T18:39:56.859 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/remote.py", line 523, in run 2023-11-28T18:39:56.859 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-11-28T18:39:56.859 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 455, in run 2023-11-28T18:39:56.859 INFO:tasks.cephfs_test_runner: r.wait() 2023-11-28T18:39:56.859 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 148, in wait 2023-11-28T18:39:56.859 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2023-11-28T18:39:56.859 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2023-11-28T18:39:56.859 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2023-11-28T18:39:56.859 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/gevent/_compat.py", line 65, in reraise 2023-11-28T18:39:56.859 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2023-11-28T18:39:56.859 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2023-11-28T18:39:56.859 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 323, in copy_file_to 2023-11-28T18:39:56.859 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2023-11-28T18:39:56.859 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 276, in copy_to_log 2023-11-28T18:39:56.860 INFO:tasks.cephfs_test_runner: for line in f: 2023-11-28T18:39:56.860 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 125, in __next__ 2023-11-28T18:39:56.860 INFO:tasks.cephfs_test_runner: line = self.readline() 2023-11-28T18:39:56.860 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 291, in readline 2023-11-28T18:39:56.860 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2023-11-28T18:39:56.860 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 1376, in _read 2023-11-28T18:39:56.860 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2023-11-28T18:39:56.860 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 749, in recv_stderr 2023-11-28T18:39:56.860 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2023-11-28T18:39:56.860 INFO:tasks.cephfs_test_runner:socket.timeout 2023-11-28T18:39:56.860 INFO:tasks.cephfs_test_runner: 2023-11-28T18:39:56.860 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T18:39:56.860 INFO:tasks.cephfs_test_runner:ERROR: test_ephemeral_pin_distribution (tasks.cephfs.test_exports.TestEphemeralPins) 2023-11-28T18:39:56.861 INFO:tasks.cephfs_test_runner:That ephemerally pinned subtrees are somewhat evenly distributed. 2023-11-28T18:39:56.861 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T18:39:56.861 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T18:39:56.861 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/cephfs_test_case.py", line 208, in tearDown 2023-11-28T18:39:56.861 INFO:tasks.cephfs_test_runner: self.mds_cluster.delete_all_filesystems() 2023-11-28T18:39:56.861 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 494, in delete_all_filesystems 2023-11-28T18:39:56.861 INFO:tasks.cephfs_test_runner: Filesystem(ctx=self._ctx, fscid=fs['id']).destroy() 2023-11-28T18:39:56.861 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 840, in destroy 2023-11-28T18:39:56.861 INFO:tasks.cephfs_test_runner: data_pools = self.get_data_pool_names(refresh=True) 2023-11-28T18:39:56.861 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 970, in get_data_pool_names 2023-11-28T18:39:56.861 INFO:tasks.cephfs_test_runner: self.get_pool_names(refresh = True) 2023-11-28T18:39:56.861 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 939, in get_pool_names 2023-11-28T18:39:56.861 INFO:tasks.cephfs_test_runner: status = self.status() 2023-11-28T18:39:56.862 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 396, in status 2023-11-28T18:39:56.862 INFO:tasks.cephfs_test_runner: return FSStatus(self.mon_manager, epoch) 2023-11-28T18:39:56.862 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 88, in __init__ 2023-11-28T18:39:56.862 INFO:tasks.cephfs_test_runner: self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) 2023-11-28T18:39:56.862 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2023-11-28T18:39:56.862 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2023-11-28T18:39:56.862 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2023-11-28T18:39:56.862 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2023-11-28T18:39:56.862 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd 2023-11-28T18:39:56.862 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-11-28T18:39:56.862 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/remote.py", line 523, in run 2023-11-28T18:39:56.862 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-11-28T18:39:56.862 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 455, in run 2023-11-28T18:39:56.863 INFO:tasks.cephfs_test_runner: r.wait() 2023-11-28T18:39:56.863 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 161, in wait 2023-11-28T18:39:56.863 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-11-28T18:39:56.863 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-11-28T18:39:56.863 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-11-28T18:39:56.863 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' 2023-11-28T18:39:56.863 INFO:tasks.cephfs_test_runner: 2023-11-28T18:39:56.863 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_ephemeral_pin_distribution (tasks.cephfs.test_exports.TestEphemeralPins), test_ephemeral_pin_distribution (tasks.cephfs.test_exports.TestEphemeralPins) 2023-11-28T18:39:56.970 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=9d95c8cb01ac45edb20eeae91a06c2ae Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_ephemeral_pin_distribution (tasks.cephfs.test_exports.TestEphemeralPins), test_ephemeral_pin_distribution (tasks.cephfs.test_exports.TestEphemeralPins) 2023-11-28T18:39:56.972 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-28T18:39:56.981 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-11-28T18:39:56.998 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-11-28T18:39:56.998 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:39:56.998 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:39:57.012 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:39:57.012 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-11-28T18:39:57.012 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:39:57.064 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354664k,nr_inodes=4088666,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3891 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-28T18:39:57.065 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:39:57.065 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:39:57.075 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:39:57.075 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-11-28T18:39:57.075 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T18:39:57.132 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354676k,nr_inodes=4088669,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=866 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-28T18:39:57.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:39:57.132 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-11-28T18:39:57.141 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T18:39:57.142 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T18:39:57.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:39:57.143 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T18:39:57.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:39:57.213 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T18:39:57.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:39:57.311 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2023-11-28T18:39:57.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:39:57.474 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2023-11-28T18:39:57.484 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T18:39:57.484 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-28T18:39:57.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:39:57.485 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T18:39:57.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:39:57.550 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T18:39:57.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T18:39:57.618 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2023-11-28T18:39:57.834 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-28T18:39:57.844 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T18:39:57.844 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T18:39:57.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:57.993+0000 7f867ba9a640 1 Processor -- start 2023-11-28T18:39:57.992 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:57.993+0000 7f867ba9a640 1 -- start start 2023-11-28T18:39:57.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:57.993+0000 7f867ba9a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f86740d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:57.993 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:57.993+0000 7f867ba9a640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86740d3c30 0x7f86740d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:39:57.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:57.993+0000 7f867ba9a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740b25b0 con 0x7f86740d1890 2023-11-28T18:39:57.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:57.993+0000 7f867ba9a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f867415d4f0 2023-11-28T18:39:57.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:57.993+0000 7f867ba9a640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86740cea30 con 0x7f86740d3c30 2023-11-28T18:39:57.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:57.993+0000 7f867a037640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86740d1890 legacy=0x7f86740cfa10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:57.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:57.993+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86740d3c30 msgr2=0x7f86740d1350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:57.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:57.993+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 msgr2=0x7f86740d0e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:57.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:57.993+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86740d3c30 0x7f86740d1350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:57.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:57.993+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f86740d0e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:39:58.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:58.197+0000 7f867a037640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86740d1890 legacy=0x7f86740cfa10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:58.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:58.197+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 msgr2=0x7f86740d0e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:58.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:58.197+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86740d3c30 msgr2=0x7f86740d1350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:58.194 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:58.197+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f86740d0e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:58.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:58.197+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86740d3c30 0x7f86740d1350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:39:58.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:58.601+0000 7f867a037640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86740d1890 legacy=0x7f86740cfa10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:58.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:58.601+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 msgr2=0x7f86740d0e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:58.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:58.601+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86740d3c30 msgr2=0x7f86740d1350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:58.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:58.601+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f86740d0e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:58.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:58.601+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86740d3c30 0x7f86740d1350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:39:59.403 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:59.405+0000 7f867a037640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86740d1890 legacy=0x7f86740cfa10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:39:59.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:59.405+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86740d3c30 msgr2=0x7f86740d1350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:39:59.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:59.405+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 msgr2=0x7f86740d0e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:39:59.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:59.405+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86740d3c30 0x7f86740d1350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:39:59.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:39:59.405+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f86740d0e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:00.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:00.997+0000 7f866b7fe640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86740d3c30 msgr2=0x7f86740d1350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:00.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:00.997+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86740d3c30 0x7f86740d1350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:00.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:00.997+0000 7f866b7fe640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 msgr2=0x7f86740d0e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:00.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:00.997+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f86740d0e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:00.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:00.997+0000 7f866b7fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86740d1890 legacy=0x7f86740cfa10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:00.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:00.997+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658007380 0x7f8658009840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:00.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:00.997+0000 7f866b7fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740ced10 con 0x7f86580036d0 2023-11-28T18:40:00.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:00.997+0000 7f866b7fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f8658007380 2023-11-28T18:40:00.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:00.997+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658007380 msgr2=0x7f8658009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:00.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:00.997+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658007380 0x7f8658009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:00.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:00.997+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580036d0 legacy=0x7f8658005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:01.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:01.201+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658007380 msgr2=0x7f8658009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:01.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:01.201+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658007380 0x7f8658009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:01.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:01.201+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580036d0 legacy=0x7f8658005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:01.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:01.605+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658007380 msgr2=0x7f8658009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:01.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:01.605+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658007380 0x7f8658009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:01.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:01.605+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580036d0 legacy=0x7f8658005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:02.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:02.409+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658007380 msgr2=0x7f8658009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:02.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:02.409+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658007380 0x7f8658009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:02.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:02.409+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580036d0 legacy=0x7f8658005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:03.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:03.993+0000 7f866b7fe640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658007380 msgr2=0x7f8658009840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:03.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:03.993+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658007380 0x7f8658009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:03.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:03.993+0000 7f866b7fe640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580036d0 legacy=0x7f8658005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:03.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:03.993+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86740d3c30 0x7f8658009f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:03.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:03.993+0000 7f867a037640 1 -- reap_dead start 2023-11-28T18:40:03.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:03.993+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f865800a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:03.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:03.993+0000 7f866b7fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740b25b0 con 0x7f86740d1890 2023-11-28T18:40:03.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:03.993+0000 7f866b7fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f867415d4f0 2023-11-28T18:40:03.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:03.993+0000 7f866b7fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86740cea30 con 0x7f86740d3c30 2023-11-28T18:40:03.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:03.993+0000 7f867a037640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86740d1890 legacy=0x7f86580110a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:03.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:03.993+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 msgr2=0x7f865800a4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:03.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:03.993+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86740d3c30 msgr2=0x7f8658009f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:03.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:03.993+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86740d3c30 0x7f8658009f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:03.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:03.993+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f865800a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:04.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:04.197+0000 7f867a037640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86740d1890 legacy=0x7f86580110a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:04.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:04.197+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86740d3c30 msgr2=0x7f8658009f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:04.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:04.197+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 msgr2=0x7f865800a4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:04.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:04.197+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86740d3c30 0x7f8658009f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:04.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:04.197+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f865800a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:04.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:04.601+0000 7f867a037640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86740d1890 legacy=0x7f86580110a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:04.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:04.601+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86740d3c30 msgr2=0x7f8658009f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:04.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:04.601+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 msgr2=0x7f865800a4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:04.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:04.601+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86740d3c30 0x7f8658009f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:04.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:04.601+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f865800a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:05.151 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:40:05.155 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:40:05.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:05.405+0000 7f867a037640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86740d1890 legacy=0x7f86580110a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:05.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:05.405+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86740d3c30 msgr2=0x7f8658009f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:05.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:05.405+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 msgr2=0x7f865800a4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:05.401 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:05.405+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86740d3c30 0x7f8658009f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:05.402 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:05.405+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f865800a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:06.994 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:06.997+0000 7f866b7fe640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86740d3c30 msgr2=0x7f8658009f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:06.997+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86740d3c30 0x7f8658009f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:06.997+0000 7f866b7fe640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 msgr2=0x7f865800a4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:06.997+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f865800a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:06.997+0000 7f866b7fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86740d1890 legacy=0x7f86580110a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:06.997+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658006280 0x7f8658009840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:06.997+0000 7f866b7fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740ced10 con 0x7f86580066c0 2023-11-28T18:40:06.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:06.997+0000 7f866b7fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f8658006280 2023-11-28T18:40:06.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:06.997+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658006280 msgr2=0x7f8658009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:06.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:06.997+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658006280 0x7f8658009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:06.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:06.997+0000 7f8679035640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:07.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:07.201+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658006280 msgr2=0x7f8658009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:07.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:07.201+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658006280 0x7f8658009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:07.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:07.201+0000 7f8679035640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:07.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:07.605+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658006280 msgr2=0x7f8658009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:07.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:07.605+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658006280 0x7f8658009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:07.601 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:07.605+0000 7f8679035640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:08.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:08.409+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658006280 msgr2=0x7f8658009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:08.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:08.409+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658006280 0x7f8658009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:08.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:08.409+0000 7f8679035640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:09.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:09.993+0000 7f866b7fe640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658006280 msgr2=0x7f8658009840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:09.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:09.993+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658006280 0x7f8658009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:09.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:09.993+0000 7f866b7fe640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:09.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:09.993+0000 7f867a037640 1 -- reap_dead start 2023-11-28T18:40:09.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:09.993+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658005af0 0x7f86580157e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:09.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:09.993+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f865800a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:09.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:09.993+0000 7f866b7fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740b25b0 con 0x7f86580066c0 2023-11-28T18:40:09.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:09.993+0000 7f866b7fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f8658005af0 2023-11-28T18:40:09.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:09.993+0000 7f866b7fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740cea30 con 0x7f867415d4f0 2023-11-28T18:40:09.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:09.993+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658005af0 msgr2=0x7f86580157e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:09.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:09.993+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658005af0 0x7f86580157e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:09.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:09.993+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 msgr2=0x7f865800a4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:09.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:09.993+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:09.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:09.993+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f865800a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:10.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:10.197+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658005af0 msgr2=0x7f86580157e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:10.195 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:10.197+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658005af0 0x7f86580157e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:10.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:10.197+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:10.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:10.197+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 msgr2=0x7f865800a4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:10.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:10.197+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f865800a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:10.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:10.601+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658005af0 msgr2=0x7f86580157e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:10.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:10.601+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658005af0 0x7f86580157e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:10.596 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:10.601+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:10.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:10.601+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 msgr2=0x7f865800a4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:10.597 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:10.601+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f865800a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:11.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:11.405+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658005af0 msgr2=0x7f86580157e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:11.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:11.405+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658005af0 0x7f86580157e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:11.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:11.405+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:11.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:11.405+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 msgr2=0x7f865800a4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:11.404 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:11.405+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f865800a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:12.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:12.997+0000 7f866b7fe640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 msgr2=0x7f865800a4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:12.995 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:12.997+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f865800a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:12.997+0000 7f866b7fe640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658005af0 msgr2=0x7f86580157e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:12.997+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658005af0 0x7f86580157e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:12.997+0000 7f866b7fe640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:12.997+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008f50 0x7f8658009390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:12.997+0000 7f866b7fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740ced10 con 0x7f8658009f80 2023-11-28T18:40:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:12.997+0000 7f866b7fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f8658008f50 2023-11-28T18:40:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:12.997+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f8658009f80 legacy=0x7f8658007830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:12.997+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008f50 msgr2=0x7f8658009390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:12.997+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008f50 0x7f8658009390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:13.201+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f8658009f80 legacy=0x7f8658007830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:13.201+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008f50 msgr2=0x7f8658009390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:13.201+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008f50 0x7f8658009390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:13.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:13.605+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f8658009f80 legacy=0x7f8658007830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:13.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:13.605+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008f50 msgr2=0x7f8658009390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:13.600 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:13.605+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008f50 0x7f8658009390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:14.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:14.409+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f8658009f80 legacy=0x7f8658007830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:14.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:14.409+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008f50 msgr2=0x7f8658009390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:14.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:14.409+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008f50 0x7f8658009390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:15.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:15.997+0000 7f866b7fe640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008f50 msgr2=0x7f8658009390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:15.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:15.997+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008f50 0x7f8658009390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:15.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:15.997+0000 7f866b7fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f8658009f80 legacy=0x7f8658007830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:15.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:15.997+0000 7f867a037640 1 -- reap_dead start 2023-11-28T18:40:15.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:15.997+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f865800a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:15.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:15.997+0000 7f866b7fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740b25b0 con 0x7f86580098d0 2023-11-28T18:40:15.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:15.997+0000 7f866b7fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740ced10 con 0x7f86580066c0 2023-11-28T18:40:15.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:15.997+0000 7f866b7fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f867415d4f0 2023-11-28T18:40:15.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:15.997+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580098d0 legacy=0x7f86580120b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:15.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:15.997+0000 7f867a037640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 msgr2=0x7f865800a4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:15.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:15.997+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:15.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:15.997+0000 7f867a037640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f865800a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:16.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:16.201+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580098d0 legacy=0x7f86580120b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:16.196 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:16.201+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:16.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:16.201+0000 7f867a037640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 msgr2=0x7f865800a4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:16.197 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:16.201+0000 7f867a037640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f865800a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:16.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:16.605+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580098d0 legacy=0x7f86580120b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:16.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:16.605+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:16.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:16.605+0000 7f867a037640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 msgr2=0x7f865800a4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:16.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:16.605+0000 7f867a037640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f865800a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:17.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:17.409+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580098d0 legacy=0x7f86580120b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:17.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:17.409+0000 7f867a037640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 msgr2=0x7f865800a4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:17.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:17.409+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:17.405 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:17.409+0000 7f867a037640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f865800a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:18.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:19.001+0000 7f866b7fe640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 msgr2=0x7f865800a4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:18.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:19.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f867415d4f0 0x7f865800a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:18.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:19.001+0000 7f866b7fe640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:18.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:19.001+0000 7f866b7fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580098d0 legacy=0x7f86580120b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:18.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:19.001+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86580077f0 0x7f86580157e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:18.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:19.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658007c50 0x7f86580080d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:18.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:19.001+0000 7f866b7fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f8658007c50 2023-11-28T18:40:18.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:19.001+0000 7f866b7fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86740cea30 con 0x7f86580077f0 2023-11-28T18:40:18.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:19.001+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658007c50 msgr2=0x7f86580080d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:18.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:19.001+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658007c50 0x7f86580080d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:18.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:19.001+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86580077f0 msgr2=0x7f86580157e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:19.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:19.001+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86580077f0 0x7f86580157e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:19.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:19.205+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658007c50 msgr2=0x7f86580080d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:19.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:19.205+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658007c50 0x7f86580080d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:19.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:19.205+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86580077f0 msgr2=0x7f86580157e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:19.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:19.205+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86580077f0 0x7f86580157e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:19.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:19.609+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658007c50 msgr2=0x7f86580080d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:19.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:19.609+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658007c50 0x7f86580080d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:19.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:19.609+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86580077f0 msgr2=0x7f86580157e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:19.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:19.609+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86580077f0 0x7f86580157e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:20.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:20.413+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658007c50 msgr2=0x7f86580080d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:20.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:20.413+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658007c50 0x7f86580080d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:20.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:20.413+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86580077f0 msgr2=0x7f86580157e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:20.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:20.413+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86580077f0 0x7f86580157e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:21.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:22.001+0000 7f866b7fe640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86580077f0 msgr2=0x7f86580157e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:21.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:22.001+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f86580077f0 0x7f86580157e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:21.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:22.001+0000 7f866b7fe640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658007c50 msgr2=0x7f86580080d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:21.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:22.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658007c50 0x7f86580080d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:21.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:22.001+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 0x7f86580089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:21.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:22.001+0000 7f867a037640 1 -- reap_dead start 2023-11-28T18:40:21.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:22.001+0000 7f866b7fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8658009390 con 0x7f86580098d0 2023-11-28T18:40:21.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:22.001+0000 7f866b7fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740b25b0 con 0x7f86580066c0 2023-11-28T18:40:21.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:22.001+0000 7f866b7fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f8658008610 2023-11-28T18:40:21.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:22.001+0000 7f867a037640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580098d0 legacy=0x7f86580120b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:21.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:22.001+0000 7f8679035640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:21.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:22.001+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 msgr2=0x7f86580089f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:21.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:22.001+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 0x7f86580089f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:22.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:22.205+0000 7f867a037640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580098d0 legacy=0x7f86580120b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:22.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:22.205+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 msgr2=0x7f86580089f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:22.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:22.205+0000 7f8679035640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:22.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:22.205+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 0x7f86580089f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:22.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:22.609+0000 7f867a037640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580098d0 legacy=0x7f86580120b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:22.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:22.609+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 msgr2=0x7f86580089f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:22.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:22.609+0000 7f8679035640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:22.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:22.609+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 0x7f86580089f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:23.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:23.413+0000 7f867a037640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580098d0 legacy=0x7f86580120b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:23.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:23.413+0000 7f8679035640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:23.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:23.413+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 msgr2=0x7f86580089f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:23.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:23.413+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 0x7f86580089f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:24.996 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:25.001+0000 7f866b7fe640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 msgr2=0x7f86580089f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:24.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:25.001+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 0x7f86580089f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:24.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:25.001+0000 7f866b7fe640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:24.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:25.001+0000 7f866b7fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580098d0 legacy=0x7f86580120b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:24.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:25.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580080d0 0x7f86580157e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:24.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:25.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800a4c0 0x7f865800d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:24.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:25.001+0000 7f866b7fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f86580080d0 2023-11-28T18:40:24.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:25.001+0000 7f866b7fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740cea30 con 0x7f865800a4c0 2023-11-28T18:40:24.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:25.001+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580080d0 msgr2=0x7f86580157e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:24.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:25.001+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580080d0 0x7f86580157e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:24.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:25.001+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800a4c0 msgr2=0x7f865800d080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:24.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:25.001+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800a4c0 0x7f865800d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:25.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:25.205+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580080d0 msgr2=0x7f86580157e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:25.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:25.205+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580080d0 0x7f86580157e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:25.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:25.205+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800a4c0 msgr2=0x7f865800d080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:25.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:25.205+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800a4c0 0x7f865800d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:25.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:25.609+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580080d0 msgr2=0x7f86580157e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:25.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:25.609+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580080d0 0x7f86580157e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:25.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:25.609+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800a4c0 msgr2=0x7f865800d080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:25.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:25.609+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800a4c0 0x7f865800d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:26.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:26.413+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580080d0 msgr2=0x7f86580157e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:26.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:26.413+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580080d0 0x7f86580157e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:26.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:26.413+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800a4c0 msgr2=0x7f865800d080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:26.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:26.413+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800a4c0 0x7f865800d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:27.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.001+0000 7f866b7fe640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800a4c0 msgr2=0x7f865800d080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:27.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800a4c0 0x7f865800d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:27.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.001+0000 7f866b7fe640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580080d0 msgr2=0x7f86580157e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:27.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580080d0 0x7f86580157e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:27.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800d5c0 0x7f865800d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:27.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.001+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 0x7f86580089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:27.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.001+0000 7f867a037640 1 -- reap_dead start 2023-11-28T18:40:27.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.001+0000 7f866b7fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740ced10 con 0x7f86580066c0 2023-11-28T18:40:27.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.001+0000 7f866b7fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f865800d5c0 2023-11-28T18:40:27.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.001+0000 7f866b7fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86740cea30 con 0x7f8658008610 2023-11-28T18:40:27.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.001+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800d5c0 msgr2=0x7f865800d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:27.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.001+0000 7f867a037640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:27.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.001+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800d5c0 0x7f865800d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:27.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.001+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 msgr2=0x7f86580089f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:27.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.001+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 0x7f86580089f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:28.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.205+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800d5c0 msgr2=0x7f865800d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:28.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.205+0000 7f867a037640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:28.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.205+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800d5c0 0x7f865800d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:28.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.205+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 msgr2=0x7f86580089f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:28.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.205+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 0x7f86580089f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:28.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.609+0000 7f867a037640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:28.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.609+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800d5c0 msgr2=0x7f865800d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:28.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.609+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800d5c0 0x7f865800d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:28.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.609+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 msgr2=0x7f86580089f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:28.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:28.609+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 0x7f86580089f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:29.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:29.413+0000 7f867a037640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:29.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:29.413+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800d5c0 msgr2=0x7f865800d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:29.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:29.413+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800d5c0 0x7f865800d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:29.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:29.413+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 msgr2=0x7f86580089f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:29.409 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:29.413+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 0x7f86580089f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:30.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:31.001+0000 7f866b7fe640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 msgr2=0x7f86580089f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:30.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:31.001+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 0x7f86580089f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:30.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:31.001+0000 7f866b7fe640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800d5c0 msgr2=0x7f865800d9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:30.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:31.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800d5c0 0x7f865800d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:30.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:31.001+0000 7f866b7fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:30.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:31.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800a4c0 0x7f865800c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:30.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:31.001+0000 7f866b7fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8658009390 con 0x7f86580098d0 2023-11-28T18:40:30.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:31.001+0000 7f866b7fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f865800a4c0 2023-11-28T18:40:30.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:31.001+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800a4c0 msgr2=0x7f865800c080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:30.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:31.001+0000 7f8679035640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580098d0 legacy=0x7f865800c5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:30.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:31.001+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800a4c0 0x7f865800c080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:31.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:31.205+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800a4c0 msgr2=0x7f865800c080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:31.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:31.205+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800a4c0 0x7f865800c080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:31.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:31.205+0000 7f8679035640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580098d0 legacy=0x7f865800c5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:31.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:31.609+0000 7f8679035640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580098d0 legacy=0x7f865800c5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:31.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:31.609+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800a4c0 msgr2=0x7f865800c080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:31.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:31.609+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800a4c0 0x7f865800c080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:32.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:32.413+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800a4c0 msgr2=0x7f865800c080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:32.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:32.413+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800a4c0 0x7f865800c080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:32.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:32.413+0000 7f8679035640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580098d0 legacy=0x7f865800c5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:33.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.001+0000 7f866b7fe640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800a4c0 msgr2=0x7f865800c080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:33.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800a4c0 0x7f865800c080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:33.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.001+0000 7f866b7fe640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580098d0 legacy=0x7f865800c5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:34.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.001+0000 7f867a037640 1 -- reap_dead start 2023-11-28T18:40:34.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f8658016fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:34.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.001+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 0x7f86580089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:34.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.001+0000 7f866b7fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740ced10 con 0x7f86580066c0 2023-11-28T18:40:34.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.001+0000 7f866b7fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f865800ccd0 2023-11-28T18:40:34.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.001+0000 7f866b7fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86740cea30 con 0x7f8658008610 2023-11-28T18:40:34.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.001+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:34.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.001+0000 7f867a037640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 msgr2=0x7f86580089f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:34.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.001+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f8658016fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:34.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.001+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f8658016fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:34.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.001+0000 7f867a037640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 0x7f86580089f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:34.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.205+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:34.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.205+0000 7f867a037640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 msgr2=0x7f86580089f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:34.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.205+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f8658016fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:34.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.205+0000 7f867a037640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 0x7f86580089f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:34.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.205+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f8658016fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:34.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.609+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:34.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.609+0000 7f867a037640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 msgr2=0x7f86580089f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:34.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.609+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f8658016fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:34.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.609+0000 7f867a037640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 0x7f86580089f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:34.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:34.609+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f8658016fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:35.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:35.413+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:35.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:35.413+0000 7f867a037640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 msgr2=0x7f86580089f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:35.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:35.413+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f8658016fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:35.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:35.413+0000 7f867a037640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 0x7f86580089f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:35.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:35.413+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f8658016fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:35.482 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:40:35.486 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:40:36.997 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:37.001+0000 7f866b7fe640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 msgr2=0x7f86580089f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:36.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:37.001+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008610 0x7f86580089f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:36.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:37.001+0000 7f866b7fe640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f8658016fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:36.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:37.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f8658016fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:36.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:37.001+0000 7f866b7fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:36.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:37.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800c790 0x7f865800d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:36.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:37.001+0000 7f866b7fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8658009390 con 0x7f86580098d0 2023-11-28T18:40:36.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:37.001+0000 7f866b7fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f865800c790 2023-11-28T18:40:36.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:37.001+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800c790 msgr2=0x7f865800d150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:36.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:37.001+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800c790 0x7f865800d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:36.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:37.001+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580098d0 legacy=0x7f865800c080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:37.205+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800c790 msgr2=0x7f865800d150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:37.205+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800c790 0x7f865800d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:37.205+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580098d0 legacy=0x7f865800c080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:37.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:37.609+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800c790 msgr2=0x7f865800d150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:37.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:37.609+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800c790 0x7f865800d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:37.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:37.609+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580098d0 legacy=0x7f865800c080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:38.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:38.413+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800c790 msgr2=0x7f865800d150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:38.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:38.413+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800c790 0x7f865800d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:38.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:38.413+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580098d0 legacy=0x7f865800c080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:39.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:40.001+0000 7f866b7fe640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800c790 msgr2=0x7f865800d150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:39.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:40.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800c790 0x7f865800d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:39.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:40.001+0000 7f866b7fe640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580098d0 legacy=0x7f865800c080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:39.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:40.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f865800d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:39.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:40.001+0000 7f867a037640 1 -- reap_dead start 2023-11-28T18:40:39.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:40.001+0000 7f866b7fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740ced10 con 0x7f86580066c0 2023-11-28T18:40:39.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:40.001+0000 7f866b7fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8658009390 con 0x7f86580098d0 2023-11-28T18:40:39.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:40.001+0000 7f866b7fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f865800ccd0 2023-11-28T18:40:39.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:40.001+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:39.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:40.001+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580098d0 legacy=0x7f865800c080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:39.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:40.001+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:39.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:40.001+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f865800d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:40.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:40.205+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:40.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:40.205+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:40.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:40.205+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580098d0 legacy=0x7f865800c080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:40.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:40.205+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f865800d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:40.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:40.609+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:40.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:40.609+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:40.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:40.609+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580098d0 legacy=0x7f865800c080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:40.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:40.609+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f865800d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:41.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:41.413+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:41.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:41.413+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580098d0 legacy=0x7f865800c080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:41.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:41.413+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:41.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:41.413+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f865800d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:42.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:43.001+0000 7f866b7fe640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f865800d690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:42.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:43.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f865800d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:42.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:43.001+0000 7f866b7fe640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f86580098d0 legacy=0x7f865800c080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:42.998 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:43.001+0000 7f866b7fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:42.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:43.001+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800c790 0x7f865800d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:42.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:43.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580085a0 0x7f86580089b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:42.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:43.001+0000 7f866b7fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f86580085a0 2023-11-28T18:40:42.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:43.001+0000 7f866b7fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86740cea30 con 0x7f865800c790 2023-11-28T18:40:42.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:43.001+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580085a0 msgr2=0x7f86580089b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:42.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:43.001+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580085a0 0x7f86580089b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:42.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:43.001+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800c790 msgr2=0x7f865800d150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:43.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:43.001+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800c790 0x7f865800d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:43.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:43.205+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580085a0 msgr2=0x7f86580089b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:43.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:43.205+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580085a0 0x7f86580089b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:43.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:43.205+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800c790 msgr2=0x7f865800d150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:43.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:43.205+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800c790 0x7f865800d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:43.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:43.609+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580085a0 msgr2=0x7f86580089b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:43.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:43.609+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580085a0 0x7f86580089b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:43.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:43.609+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800c790 msgr2=0x7f865800d150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:43.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:43.609+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800c790 0x7f865800d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:44.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:44.413+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580085a0 msgr2=0x7f86580089b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:44.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:44.413+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580085a0 0x7f86580089b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:44.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:44.413+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800c790 msgr2=0x7f865800d150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:44.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:44.413+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800c790 0x7f865800d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:45.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.001+0000 7f866b7fe640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800c790 msgr2=0x7f865800d150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:45.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.001+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800c790 0x7f865800d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:45.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.001+0000 7f866b7fe640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580085a0 msgr2=0x7f86580089b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:46.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580085a0 0x7f86580089b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:46.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.001+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658016fd0 0x7f86580193c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:46.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.001+0000 7f867a037640 1 -- reap_dead start 2023-11-28T18:40:46.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f865800d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:46.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.001+0000 7f866b7fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740b25b0 con 0x7f86580066c0 2023-11-28T18:40:46.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.001+0000 7f866b7fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f865800ccd0 2023-11-28T18:40:46.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.001+0000 7f866b7fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86740cea30 con 0x7f8658016fd0 2023-11-28T18:40:46.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.001+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:46.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.001+0000 7f867a037640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f865800d150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:46.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.001+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658016fd0 msgr2=0x7f86580193c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:46.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.001+0000 7f867a037640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f865800d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:46.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.001+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658016fd0 0x7f86580193c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:46.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.205+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:46.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.205+0000 7f867a037640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f865800d150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:46.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.205+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658016fd0 msgr2=0x7f86580193c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:46.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.205+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658016fd0 0x7f86580193c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:46.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.205+0000 7f867a037640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f865800d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:46.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.609+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:46.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.609+0000 7f867a037640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f865800d150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:46.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.609+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658016fd0 msgr2=0x7f86580193c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:46.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.609+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658016fd0 0x7f86580193c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:46.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:46.609+0000 7f867a037640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f865800d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:47.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:47.413+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:47.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:47.413+0000 7f867a037640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f865800d150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:47.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:47.413+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658016fd0 msgr2=0x7f86580193c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:47.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:47.413+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658016fd0 0x7f86580193c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:47.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:47.413+0000 7f867a037640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f865800d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:48.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:49.001+0000 7f866b7fe640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658016fd0 msgr2=0x7f86580193c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:48.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:49.001+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658016fd0 0x7f86580193c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:48.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:49.001+0000 7f866b7fe640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f865800d150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:48.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:49.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f865800d150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:49.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:49.001+0000 7f866b7fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:49.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:49.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580085a0 0x7f86580089b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:49.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:49.001+0000 7f866b7fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740ced10 con 0x7f865800c080 2023-11-28T18:40:49.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:49.001+0000 7f866b7fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f86580085a0 2023-11-28T18:40:49.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:49.001+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580085a0 msgr2=0x7f86580089b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:49.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:49.001+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580085a0 0x7f86580089b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:49.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:49.001+0000 7f8679035640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:49.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:49.205+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580085a0 msgr2=0x7f86580089b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:49.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:49.205+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580085a0 0x7f86580089b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:49.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:49.205+0000 7f8679035640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:49.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:49.609+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580085a0 msgr2=0x7f86580089b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:49.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:49.609+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580085a0 0x7f86580089b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:49.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:49.609+0000 7f8679035640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:50.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:50.413+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580085a0 msgr2=0x7f86580089b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:50.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:50.413+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580085a0 0x7f86580089b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:50.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:50.413+0000 7f8679035640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:51.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:52.001+0000 7f866b7fe640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580085a0 msgr2=0x7f86580089b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:51.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:52.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580085a0 0x7f86580089b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:52.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:52.001+0000 7f866b7fe640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:52.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:52.001+0000 7f867a037640 1 -- reap_dead start 2023-11-28T18:40:52.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:52.001+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 0x7f8658010370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:52.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:52.001+0000 7f866b7fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740b25b0 con 0x7f86580066c0 2023-11-28T18:40:52.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:52.001+0000 7f866b7fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740ced10 con 0x7f865800c080 2023-11-28T18:40:52.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:52.001+0000 7f866b7fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f865800ff90 2023-11-28T18:40:52.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:52.001+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:52.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:52.001+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 msgr2=0x7f8658010370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:52.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:52.001+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 0x7f8658010370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:52.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:52.001+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:52.199 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:52.205+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 msgr2=0x7f8658010370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:52.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:52.205+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:52.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:52.205+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 0x7f8658010370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:52.200 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:52.205+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:52.603 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:52.609+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:52.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:52.609+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 msgr2=0x7f8658010370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:52.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:52.609+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 0x7f8658010370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:52.604 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:52.609+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:53.407 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:53.413+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:40:53.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:53.413+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 msgr2=0x7f8658010370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:53.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:53.413+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 0x7f8658010370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:53.408 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:53.413+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:54.999 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:55.005+0000 7f866b7fe640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 msgr2=0x7f8658010370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:55.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:55.005+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 0x7f8658010370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:55.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:55.005+0000 7f866b7fe640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:55.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:55.005+0000 7f866b7fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066c0 legacy=0x7f8658006b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:55.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:55.005+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580089b0 0x7f8658018fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:55.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:55.005+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658019520 0x7f865800ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:55.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:55.005+0000 7f866b7fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f86580089b0 2023-11-28T18:40:55.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:55.005+0000 7f866b7fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86740cea30 con 0x7f8658019520 2023-11-28T18:40:55.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:55.005+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580089b0 msgr2=0x7f8658018fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:55.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:55.005+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658019520 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:55.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:55.005+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658019520 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:55.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:55.005+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580089b0 0x7f8658018fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:55.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:55.209+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580089b0 msgr2=0x7f8658018fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:55.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:55.209+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658019520 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:55.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:55.209+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658019520 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:55.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:55.209+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580089b0 0x7f8658018fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:55.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:55.613+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658019520 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:55.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:55.613+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580089b0 msgr2=0x7f8658018fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:55.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:55.613+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658019520 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:55.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:55.613+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580089b0 0x7f8658018fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:56.417+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658019520 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:56.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:56.417+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580089b0 msgr2=0x7f8658018fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:40:56.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:56.417+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658019520 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:56.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:56.417+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580089b0 0x7f8658018fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:58.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.005+0000 7f866b7fe640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658019520 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:58.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.005+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658019520 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:58.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.005+0000 7f866b7fe640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580089b0 msgr2=0x7f8658018fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:40:58.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.005+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580089b0 0x7f8658018fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:40:58.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.005+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 0x7f8658010370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:58.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.005+0000 7f867a037640 1 -- reap_dead start 2023-11-28T18:40:58.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.005+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:40:58.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.005+0000 7f866b7fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8658009390 con 0x7f865800c080 2023-11-28T18:40:58.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.005+0000 7f866b7fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f865800ff90 2023-11-28T18:40:58.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.005+0000 7f866b7fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86740cea30 con 0x7f865800d210 2023-11-28T18:40:58.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.005+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 msgr2=0x7f8658010370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:58.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.005+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 0x7f8658010370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:58.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.005+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:58.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.005+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 msgr2=0x7f86580148d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:58.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.005+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:40:58.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.209+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 msgr2=0x7f8658010370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:58.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.209+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 0x7f8658010370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:58.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.209+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:58.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.209+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 msgr2=0x7f86580148d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:58.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.209+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:40:58.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.613+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 msgr2=0x7f8658010370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:58.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.613+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 0x7f8658010370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:58.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.613+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:58.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.613+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 msgr2=0x7f86580148d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:58.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:58.613+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:40:59.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:59.417+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 msgr2=0x7f8658010370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:40:59.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:59.417+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 0x7f8658010370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:40:59.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:59.417+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:40:59.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:59.417+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 msgr2=0x7f86580148d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:40:59.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:40:59.417+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:01.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:01.004+0000 7f866b7fe640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 msgr2=0x7f86580148d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:01.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:01.004+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:01.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:01.004+0000 7f866b7fe640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 msgr2=0x7f8658010370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:01.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:01.004+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 0x7f8658010370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:01.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:01.004+0000 7f866b7fe640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:01.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:01.004+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580066c0 0x7f865800ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:01.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:01.004+0000 7f866b7fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740b25b0 con 0x7f86580089b0 2023-11-28T18:41:01.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:01.004+0000 7f866b7fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f86580066c0 2023-11-28T18:41:01.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:01.004+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580089b0 legacy=0x7f8658006b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:01.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:01.004+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580066c0 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:01.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:01.004+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580066c0 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:01.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:01.208+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580089b0 legacy=0x7f8658006b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:01.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:01.208+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580066c0 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:01.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:01.208+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580066c0 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:01.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:01.612+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580089b0 legacy=0x7f8658006b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:01.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:01.612+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580066c0 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:01.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:01.612+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580066c0 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:02.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:02.416+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580089b0 legacy=0x7f8658006b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:02.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:02.416+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580066c0 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:02.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:02.416+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580066c0 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:04.000 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.004+0000 7f866b7fe640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580066c0 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:04.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.004+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580066c0 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:04.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.004+0000 7f866b7fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580089b0 legacy=0x7f8658006b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:04.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.004+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:04.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.004+0000 7f867a037640 1 -- reap_dead start 2023-11-28T18:41:04.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.004+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 0x7f8658010370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:04.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.004+0000 7f866b7fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8658009390 con 0x7f865800c080 2023-11-28T18:41:04.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.004+0000 7f866b7fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f865800ff90 2023-11-28T18:41:04.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.004+0000 7f866b7fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86740cea30 con 0x7f865800d210 2023-11-28T18:41:04.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.004+0000 7f867a037640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 msgr2=0x7f8658010370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:04.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.004+0000 7f867a037640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 0x7f8658010370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:04.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.004+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 msgr2=0x7f86580148d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:04.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.004+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:04.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.004+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:04.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.208+0000 7f867a037640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 msgr2=0x7f8658010370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:04.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.208+0000 7f867a037640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 0x7f8658010370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:04.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.208+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:04.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.208+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 msgr2=0x7f86580148d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:04.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.208+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:04.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.612+0000 7f867a037640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 msgr2=0x7f8658010370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:04.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.612+0000 7f867a037640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 0x7f8658010370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:04.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.612+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 msgr2=0x7f86580148d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:04.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.612+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:04.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:04.612+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:05.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:05.416+0000 7f867a037640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 msgr2=0x7f8658010370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:05.416+0000 7f867a037640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 0x7f8658010370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:05.416+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 msgr2=0x7f86580148d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:05.416+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:05.416+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:05.555 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:41:05.560 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:41:07.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:07.004+0000 7f866b7fe640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 msgr2=0x7f86580148d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:07.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:07.004+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:07.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:07.004+0000 7f866b7fe640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 msgr2=0x7f8658010370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:07.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:07.004+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800ff90 0x7f8658010370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:07.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:07.004+0000 7f866b7fe640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:07.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:07.004+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580089b0 0x7f865800ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:07.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:07.004+0000 7f866b7fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740b25b0 con 0x7f8658009970 2023-11-28T18:41:07.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:07.004+0000 7f866b7fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f86580089b0 2023-11-28T18:41:07.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:07.004+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f8658009970 legacy=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:07.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:07.004+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580089b0 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:07.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:07.004+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580089b0 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:07.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:07.208+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f8658009970 legacy=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:07.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:07.208+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580089b0 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:07.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:07.208+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580089b0 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:07.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:07.612+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f8658009970 legacy=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:07.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:07.612+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580089b0 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:07.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:07.612+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580089b0 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:08.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:08.416+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f8658009970 legacy=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:08.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:08.416+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580089b0 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:08.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:08.416+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580089b0 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:10.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:10.004+0000 7f866b7fe640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580089b0 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:10.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:10.004+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580089b0 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:10.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:10.004+0000 7f866b7fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f8658009970 legacy=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:10.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:10.004+0000 7f867a037640 1 -- reap_dead start 2023-11-28T18:41:10.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:10.004+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:10.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:10.004+0000 7f866b7fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8658009390 con 0x7f86580066a0 2023-11-28T18:41:10.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:10.004+0000 7f866b7fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740b25b0 con 0x7f865800c080 2023-11-28T18:41:10.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:10.004+0000 7f866b7fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f865800d210 2023-11-28T18:41:10.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:10.004+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066a0 legacy=0x7f865800ef90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:10.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:10.004+0000 7f867a037640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 msgr2=0x7f86580148d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:10.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:10.004+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:10.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:10.004+0000 7f867a037640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:10.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:10.208+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066a0 legacy=0x7f865800ef90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:10.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:10.208+0000 7f867a037640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 msgr2=0x7f86580148d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:10.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:10.208+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:10.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:10.208+0000 7f867a037640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:10.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:10.612+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066a0 legacy=0x7f865800ef90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:10.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:10.612+0000 7f867a037640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 msgr2=0x7f86580148d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:10.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:10.612+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:10.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:10.612+0000 7f867a037640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:11.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:11.416+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066a0 legacy=0x7f865800ef90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:11.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:11.416+0000 7f867a037640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 msgr2=0x7f86580148d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:11.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:11.416+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:11.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:11.416+0000 7f867a037640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:13.001 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:13.004+0000 7f866b7fe640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 msgr2=0x7f86580148d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:13.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:13.004+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:13.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:13.004+0000 7f866b7fe640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:13.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:13.004+0000 7f866b7fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066a0 legacy=0x7f865800ef90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:13.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:13.004+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8658008980 0x7f865800ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:13.004+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658009970 0x7f8658006af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:13.004+0000 7f866b7fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f8658009970 2023-11-28T18:41:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:13.004+0000 7f866b7fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740cea30 con 0x7f8658008980 2023-11-28T18:41:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:13.004+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658009970 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:13.004+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658009970 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:13.004+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f8658008980 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:13.004+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8658008980 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:13.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:13.208+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658009970 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:13.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:13.208+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658009970 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:13.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:13.208+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f8658008980 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:13.205 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:13.208+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8658008980 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:13.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:13.612+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658009970 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:13.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:13.612+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658009970 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:13.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:13.612+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f8658008980 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:13.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:13.612+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8658008980 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:14.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:14.416+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658009970 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:14.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:14.416+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658009970 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:14.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:14.416+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f8658008980 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:14.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:14.416+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8658008980 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:16.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:16.004+0000 7f866b7fe640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f8658008980 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:16.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:16.004+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8658008980 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:16.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:16.004+0000 7f866b7fe640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658009970 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:16.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:16.004+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658009970 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:16.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:16.004+0000 7f867a037640 1 -- reap_dead start 2023-11-28T18:41:16.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:16.004+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:16.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:16.004+0000 7f866b7fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740ced10 con 0x7f86580066a0 2023-11-28T18:41:16.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:16.004+0000 7f866b7fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8658009390 con 0x7f865800c080 2023-11-28T18:41:16.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:16.004+0000 7f866b7fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f865800d210 2023-11-28T18:41:16.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:16.004+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066a0 legacy=0x7f865800b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:16.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:16.004+0000 7f867a037640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 msgr2=0x7f86580148d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:16.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:16.004+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:16.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:16.004+0000 7f867a037640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:16.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:16.208+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066a0 legacy=0x7f865800b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:16.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:16.208+0000 7f867a037640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 msgr2=0x7f86580148d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:16.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:16.208+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:16.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:16.208+0000 7f867a037640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:16.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:16.612+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066a0 legacy=0x7f865800b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:16.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:16.612+0000 7f867a037640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 msgr2=0x7f86580148d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:16.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:16.612+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:16.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:16.612+0000 7f867a037640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:17.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:17.416+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066a0 legacy=0x7f865800b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:17.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:17.416+0000 7f867a037640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 msgr2=0x7f86580148d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:17.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:17.416+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:17.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:17.416+0000 7f867a037640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:19.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:19.004+0000 7f866b7fe640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 msgr2=0x7f86580148d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:19.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:19.004+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f865800d210 0x7f86580148d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:19.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:19.004+0000 7f866b7fe640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:19.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:19.004+0000 7f866b7fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066a0 legacy=0x7f865800b070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:19.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:19.004+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8658008980 0x7f865800ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:19.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:19.004+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658009970 0x7f8658018e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:19.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:19.004+0000 7f866b7fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f8658009970 2023-11-28T18:41:19.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:19.004+0000 7f866b7fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740cea30 con 0x7f8658008980 2023-11-28T18:41:19.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:19.004+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658009970 msgr2=0x7f8658018e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:19.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:19.004+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658009970 0x7f8658018e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:19.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:19.004+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f8658008980 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:19.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:19.004+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8658008980 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:19.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:19.208+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658009970 msgr2=0x7f8658018e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:19.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:19.208+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658009970 0x7f8658018e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:19.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:19.208+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f8658008980 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:19.204 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:19.208+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8658008980 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:19.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:19.612+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658009970 msgr2=0x7f8658018e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:19.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:19.612+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658009970 0x7f8658018e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:19.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:19.612+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f8658008980 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:19.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:19.612+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8658008980 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:20.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:20.416+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658009970 msgr2=0x7f8658018e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:20.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:20.416+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658009970 0x7f8658018e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:20.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:20.416+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f8658008980 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:20.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:20.416+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8658008980 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:22.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:22.004+0000 7f866b7fe640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f8658008980 msgr2=0x7f865800ccd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:22.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:22.004+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8658008980 0x7f865800ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:22.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:22.004+0000 7f866b7fe640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658009970 msgr2=0x7f8658018e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:22.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:22.004+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658009970 0x7f8658018e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:22.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:22.004+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658006af0 0x7f865800b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:22.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:22.004+0000 7f867a037640 1 -- reap_dead start 2023-11-28T18:41:22.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:22.004+0000 7f866b7fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740b25b0 con 0x7f86580066a0 2023-11-28T18:41:22.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:22.004+0000 7f866b7fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740ced10 con 0x7f865800c080 2023-11-28T18:41:22.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:22.004+0000 7f866b7fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f8658006af0 2023-11-28T18:41:22.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:22.004+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066a0 legacy=0x7f865800b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:22.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:22.004+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658006af0 msgr2=0x7f865800b780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:22.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:22.004+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658006af0 0x7f865800b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:22.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:22.004+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:22.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:22.208+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066a0 legacy=0x7f865800b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:22.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:22.208+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658006af0 msgr2=0x7f865800b780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:22.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:22.208+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:22.203 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:22.208+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658006af0 0x7f865800b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:22.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:22.612+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658006af0 msgr2=0x7f865800b780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:22.607 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:22.612+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066a0 legacy=0x7f865800b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:22.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:22.612+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658006af0 0x7f865800b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:22.608 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:22.612+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:23.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:23.416+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658006af0 msgr2=0x7f865800b780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:23.411 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:23.416+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066a0 legacy=0x7f865800b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:23.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:23.416+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:23.412 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:23.416+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658006af0 0x7f865800b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:25.002 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:25.008+0000 7f866b7fe640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658006af0 msgr2=0x7f865800b780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:25.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:25.008+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658006af0 0x7f865800b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:25.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:25.008+0000 7f866b7fe640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:25.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:25.008+0000 7f866b7fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066a0 legacy=0x7f865800b070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:25.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:25.008+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 0x7f8658018e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:25.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:25.008+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f865800d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:25.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:25.008+0000 7f866b7fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f865800ccd0 2023-11-28T18:41:25.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:25.008+0000 7f866b7fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86740cea30 con 0x7f8658008980 2023-11-28T18:41:25.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:25.008+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f865800d0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:25.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:25.008+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 msgr2=0x7f8658018e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:25.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:25.008+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f865800d0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:25.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:25.008+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 0x7f8658018e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:25.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:25.212+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 msgr2=0x7f8658018e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:25.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:25.212+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f865800d0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:25.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:25.212+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 0x7f8658018e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:25.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:25.212+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f865800d0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:25.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:25.616+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f865800d0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:25.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:25.616+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 msgr2=0x7f8658018e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:25.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:25.616+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 0x7f8658018e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:25.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:25.616+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f865800d0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:26.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:26.420+0000 7f8679836640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 msgr2=0x7f8658018e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:26.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:26.420+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f865800d0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:26.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:26.420+0000 7f8679836640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 0x7f8658018e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:26.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:26.420+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f865800d0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:28.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.008+0000 7f866b7fe640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 msgr2=0x7f8658018e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:28.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.008+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 0x7f8658018e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:28.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.008+0000 7f866b7fe640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f865800d0b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:28.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.008+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f865800d0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:28.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.008+0000 7f867a037640 1 -- reap_dead start 2023-11-28T18:41:28.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.008+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658006af0 0x7f865800b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:28.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.008+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 0x7f8658017fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:28.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.008+0000 7f866b7fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8658009390 con 0x7f865800c080 2023-11-28T18:41:28.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.008+0000 7f866b7fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f8658006af0 2023-11-28T18:41:28.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.008+0000 7f866b7fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86740cea30 con 0x7f8658008980 2023-11-28T18:41:28.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.008+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658006af0 msgr2=0x7f865800b780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:28.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.008+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658006af0 0x7f865800b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:28.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.008+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:28.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.008+0000 7f867a037640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 msgr2=0x7f8658017fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:28.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.008+0000 7f867a037640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 0x7f8658017fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:28.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.212+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658006af0 msgr2=0x7f865800b780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:28.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.212+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658006af0 0x7f865800b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:28.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.212+0000 7f867a037640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 msgr2=0x7f8658017fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:28.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.212+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:28.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.212+0000 7f867a037640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 0x7f8658017fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:28.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.616+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658006af0 msgr2=0x7f865800b780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:28.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.616+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658006af0 0x7f865800b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:28.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.616+0000 7f867a037640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 msgr2=0x7f8658017fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:28.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.616+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:28.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:28.616+0000 7f867a037640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 0x7f8658017fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:29.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:29.420+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658006af0 msgr2=0x7f865800b780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:29.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:29.420+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658006af0 0x7f865800b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:29.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:29.420+0000 7f8679836640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:29.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:29.420+0000 7f867a037640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 msgr2=0x7f8658017fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:29.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:29.420+0000 7f867a037640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 0x7f8658017fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:31.003 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:31.008+0000 7f866b7fe640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 msgr2=0x7f8658017fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:31.008+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 0x7f8658017fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:31.008+0000 7f866b7fe640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8658006af0 msgr2=0x7f865800b780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:31.008+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8658006af0 0x7f865800b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:31.008+0000 7f866b7fe640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:31.008+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f8658018520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:31.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:31.008+0000 7f866b7fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740b25b0 con 0x7f86580066a0 2023-11-28T18:41:31.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:31.008+0000 7f866b7fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f865800ccd0 2023-11-28T18:41:31.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:31.008+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066a0 legacy=0x7f865800b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:31.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:31.008+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f8658018520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:31.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:31.008+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f8658018520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:31.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:31.212+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066a0 legacy=0x7f865800b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:31.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:31.212+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f8658018520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:31.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:31.212+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f8658018520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:31.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:31.616+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066a0 legacy=0x7f865800b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:31.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:31.616+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f8658018520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:31.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:31.616+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f8658018520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:32.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:32.420+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066a0 legacy=0x7f865800b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:32.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:32.420+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f8658018520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:32.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:32.420+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f8658018520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:34.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:34.008+0000 7f866b7fe640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 msgr2=0x7f8658018520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:34.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:34.008+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865800ccd0 0x7f8658018520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:34.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:34.008+0000 7f866b7fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f86580066a0 legacy=0x7f865800b070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:34.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:34.008+0000 7f867a037640 1 -- reap_dead start 2023-11-28T18:41:34.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:34.008+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865801c920 0x7f865801ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:34.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:34.008+0000 7f866b7fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8658009390 con 0x7f865800d100 2023-11-28T18:41:34.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:34.008+0000 7f866b7fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740b25b0 con 0x7f865800c080 2023-11-28T18:41:34.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:34.008+0000 7f866b7fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f865801c920 2023-11-28T18:41:34.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:34.008+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f865800d100 legacy=0x7f8658018a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:34.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:34.008+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:34.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:34.008+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865801c920 msgr2=0x7f865801ed10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:34.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:34.008+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865801c920 0x7f865801ed10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:34.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:34.212+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f865800d100 legacy=0x7f8658018a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:34.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:34.212+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:34.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:34.212+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865801c920 msgr2=0x7f865801ed10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:34.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:34.212+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865801c920 0x7f865801ed10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:34.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:34.616+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f865800d100 legacy=0x7f8658018a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:34.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:34.616+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:34.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:34.616+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865801c920 msgr2=0x7f865801ed10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:34.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:34.616+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865801c920 0x7f865801ed10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:35.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:35.420+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f865800d100 legacy=0x7f8658018a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:35.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:35.420+0000 7f8679035640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865801c920 msgr2=0x7f865801ed10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:35.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:35.420+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:35.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:35.420+0000 7f8679035640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865801c920 0x7f865801ed10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:35.616 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:41:35.621 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:41:37.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:37.008+0000 7f866b7fe640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865801c920 msgr2=0x7f865801ed10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:37.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:37.008+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865801c920 0x7f865801ed10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:37.004 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:37.008+0000 7f866b7fe640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800c080 legacy=0x7f865800d690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:37.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:37.008+0000 7f866b7fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f865800d100 legacy=0x7f8658018a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:37.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:37.008+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:37.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:37.008+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 0x7f865800b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:37.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:37.008+0000 7f866b7fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f86580066a0 2023-11-28T18:41:37.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:37.008+0000 7f866b7fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86740cea30 con 0x7f8658008980 2023-11-28T18:41:37.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:37.008+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580066a0 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:37.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:37.008+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:37.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:37.008+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 msgr2=0x7f865800b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:37.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:37.008+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 0x7f865800b070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:37.212+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580066a0 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:37.212+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:37.212+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 msgr2=0x7f865800b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:37.212+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 0x7f865800b070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:37.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:37.616+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580066a0 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:37.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:37.616+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:37.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:37.616+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 msgr2=0x7f865800b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:37.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:37.616+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 0x7f865800b070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:38.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:38.420+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580066a0 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:38.420+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:38.420+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 msgr2=0x7f865800b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:38.420+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 0x7f865800b070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:40.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.008+0000 7f866b7fe640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 msgr2=0x7f865800b070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:40.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.008+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658008980 0x7f865800b070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:40.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.008+0000 7f866b7fe640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580066a0 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:40.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.008+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:40.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.008+0000 7f867a037640 1 -- reap_dead start 2023-11-28T18:41:40.005 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.008+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 0x7f8658015360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:40.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.008+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865801c920 0x7f865800b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:40.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.008+0000 7f866b7fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740ced10 con 0x7f865800d100 2023-11-28T18:41:40.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.008+0000 7f866b7fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f865801c920 2023-11-28T18:41:40.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.008+0000 7f866b7fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86740cea30 con 0x7f8658012f10 2023-11-28T18:41:40.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.008+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f865800d100 legacy=0x7f865800b5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:40.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.008+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 msgr2=0x7f8658015360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:40.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.008+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 0x7f8658015360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:40.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.008+0000 7f867a037640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865801c920 msgr2=0x7f865800b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:40.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.008+0000 7f867a037640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865801c920 0x7f865800b070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:40.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.212+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f865800d100 legacy=0x7f865800b5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:40.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.212+0000 7f867a037640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865801c920 msgr2=0x7f865800b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:40.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.212+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 msgr2=0x7f8658015360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:40.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.212+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 0x7f8658015360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:40.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.212+0000 7f867a037640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865801c920 0x7f865800b070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:40.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.616+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f865800d100 legacy=0x7f865800b5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:40.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.616+0000 7f867a037640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865801c920 msgr2=0x7f865800b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:40.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.616+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 msgr2=0x7f8658015360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:40.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.616+0000 7f867a037640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865801c920 0x7f865800b070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:40.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:40.616+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 0x7f8658015360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:41.415 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:41.420+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f865800d100 legacy=0x7f865800b5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:41.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:41.420+0000 7f867a037640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865801c920 msgr2=0x7f865800b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:41.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:41.420+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 msgr2=0x7f8658015360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:41.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:41.420+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 0x7f8658015360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:41.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:41.420+0000 7f867a037640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865801c920 0x7f865800b070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:43.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:43.007+0000 7f866b7fe640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 msgr2=0x7f8658015360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:43.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:43.007+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 0x7f8658015360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:43.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:43.007+0000 7f866b7fe640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f865801c920 msgr2=0x7f865800b070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:43.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:43.007+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f865801c920 0x7f865800b070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:43.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:43.007+0000 7f866b7fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f865800d100 legacy=0x7f865800b5b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:43.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:43.007+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:43.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:43.007+0000 7f866b7fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8658009390 con 0x7f865800d690 2023-11-28T18:41:43.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:43.007+0000 7f866b7fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f86580066a0 2023-11-28T18:41:43.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:43.007+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580066a0 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:43.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:43.007+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:43.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:43.007+0000 7f8679035640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800d690 legacy=0x7f865800bcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:43.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:43.211+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580066a0 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:43.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:43.211+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:43.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:43.211+0000 7f8679035640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800d690 legacy=0x7f865800bcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:43.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:43.615+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580066a0 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:43.611 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:43.615+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:43.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:43.615+0000 7f8679035640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800d690 legacy=0x7f865800bcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:44.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:44.419+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580066a0 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:44.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:44.419+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:44.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:44.419+0000 7f8679035640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800d690 legacy=0x7f865800bcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:46.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.007+0000 7f866b7fe640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580066a0 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:46.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.007+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:46.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.007+0000 7f866b7fe640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800d690 legacy=0x7f865800bcc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:46.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.007+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580158a0 0x7f8658017c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:46.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.007+0000 7f867a037640 1 -- reap_dead start 2023-11-28T18:41:46.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.007+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 0x7f8658015360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:46.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.007+0000 7f866b7fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740ced10 con 0x7f865800d690 2023-11-28T18:41:46.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.007+0000 7f866b7fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f86580158a0 2023-11-28T18:41:46.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.007+0000 7f866b7fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86740cea30 con 0x7f8658012f10 2023-11-28T18:41:46.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.007+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580158a0 msgr2=0x7f8658017c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:46.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.007+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580158a0 0x7f8658017c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:46.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.007+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800d690 legacy=0x7f865800bcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:46.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.007+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 msgr2=0x7f8658015360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:46.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.007+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 0x7f8658015360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:46.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.211+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580158a0 msgr2=0x7f8658017c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:46.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.211+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580158a0 0x7f8658017c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:46.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.211+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800d690 legacy=0x7f865800bcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:46.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.211+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 msgr2=0x7f8658015360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:46.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.211+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 0x7f8658015360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:46.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.615+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580158a0 msgr2=0x7f8658017c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:46.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.615+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580158a0 0x7f8658017c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:46.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.615+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800d690 legacy=0x7f865800bcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:46.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.615+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 msgr2=0x7f8658015360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:46.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:46.615+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 0x7f8658015360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:47.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:47.419+0000 7f8679836640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580158a0 msgr2=0x7f8658017c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:47.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:47.419+0000 7f8679836640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580158a0 0x7f8658017c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:47.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:47.419+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800d690 legacy=0x7f865800bcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:47.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:47.419+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 msgr2=0x7f8658015360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:47.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:47.419+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 0x7f8658015360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:49.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:49.007+0000 7f866b7fe640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 msgr2=0x7f8658015360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:49.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:49.007+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 0x7f8658015360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:49.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:49.007+0000 7f866b7fe640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f86580158a0 msgr2=0x7f8658017c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:49.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:49.007+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86580158a0 0x7f8658017c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:49.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:49.007+0000 7f866b7fe640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800d690 legacy=0x7f865800bcc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:49.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:49.007+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:49.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:49.007+0000 7f866b7fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8658009390 con 0x7f865800d100 2023-11-28T18:41:49.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:49.007+0000 7f866b7fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f86580066a0 2023-11-28T18:41:49.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:49.007+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f865800d100 legacy=0x7f865800b030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:49.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:49.007+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580066a0 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:49.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:49.007+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:49.207 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:49.211+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f865800d100 legacy=0x7f865800b030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:49.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:49.211+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580066a0 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:49.208 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:49.211+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:49.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:49.615+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f865800d100 legacy=0x7f865800b030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:49.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:49.615+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580066a0 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:49.612 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:49.615+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:50.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:50.419+0000 7f8679035640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f865800d100 legacy=0x7f865800b030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:50.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:50.419+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580066a0 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:50.416 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:50.419+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:52.011+0000 7f866b7fe640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580066a0 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:52.011+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:52.011+0000 7f866b7fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f865800d100 legacy=0x7f865800b030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:52.011+0000 7f867a037640 1 -- reap_dead start 2023-11-28T18:41:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:52.011+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 0x7f8658015360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:52.011+0000 7f866b7fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740ced10 con 0x7f865800b740 2023-11-28T18:41:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:52.011+0000 7f866b7fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8658009390 con 0x7f865800d690 2023-11-28T18:41:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:52.011+0000 7f866b7fe640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f8658012f10 2023-11-28T18:41:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:52.011+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f865800b740 legacy=0x7f865801b910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:52.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:52.011+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800d690 legacy=0x7f865800b030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:52.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:52.011+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 msgr2=0x7f8658015360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:52.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:52.011+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 0x7f8658015360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:52.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:52.215+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f865800b740 legacy=0x7f865801b910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:52.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:52.215+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800d690 legacy=0x7f865800b030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:52.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:52.215+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 msgr2=0x7f8658015360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:52.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:52.215+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 0x7f8658015360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:52.615 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:52.619+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f865800b740 legacy=0x7f865801b910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:52.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:52.619+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800d690 legacy=0x7f865800b030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:52.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:52.619+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 msgr2=0x7f8658015360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:52.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:52.619+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 0x7f8658015360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:53.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:53.423+0000 7f8679836640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f865800b740 legacy=0x7f865801b910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2023-11-28T18:41:53.421 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:53.423+0000 7f867a037640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800d690 legacy=0x7f865800b030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.148:6789/0 2023-11-28T18:41:53.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:53.423+0000 7f8679035640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 msgr2=0x7f8658015360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3301/0 2023-11-28T18:41:53.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:53.423+0000 7f8679035640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 0x7f8658015360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:55.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:55.011+0000 7f866b7fe640 1 -- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 msgr2=0x7f8658015360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:55.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:55.011+0000 7f866b7fe640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8658012f10 0x7f8658015360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T18:41:55.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:55.011+0000 7f866b7fe640 1 -- >> v1:172.21.15.148:6789/0 conn(0x7f865800d690 legacy=0x7f865800b030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:55.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:55.011+0000 7f866b7fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f865800b740 legacy=0x7f865801b910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T18:41:55.007 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:55.011+0000 7f866b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:55.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:55.011+0000 7f866b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800bb20 0x7f86580178b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T18:41:55.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:55.011+0000 7f866b7fe640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740b2900 con 0x7f865800bb20 2023-11-28T18:41:55.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:55.011+0000 7f866b7fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740cea30 con 0x7f86580066a0 2023-11-28T18:41:55.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:55.011+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800bb20 msgr2=0x7f86580178b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:55.008 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:55.011+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800bb20 0x7f86580178b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:55.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:55.011+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580066a0 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:55.009 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:55.011+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T18:41:55.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:55.215+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800bb20 msgr2=0x7f86580178b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:55.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:55.215+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800bb20 0x7f86580178b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:55.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:55.215+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580066a0 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:55.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:55.215+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T18:41:55.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:55.619+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800bb20 msgr2=0x7f86580178b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:55.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:55.619+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800bb20 0x7f86580178b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:55.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:55.619+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580066a0 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:55.616 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:55.619+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T18:41:56.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:56.423+0000 7f8679035640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f865800bb20 msgr2=0x7f86580178b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2023-11-28T18:41:56.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:56.423+0000 7f8679035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f865800bb20 0x7f86580178b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:56.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:56.423+0000 7f8679836640 1 -- >> v2:172.21.15.148:3300/0 conn(0x7f86580066a0 msgr2=0x7f8658006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.148:3300/0 2023-11-28T18:41:56.420 INFO:teuthology.orchestra.run.smithi105.stderr:2023-11-28T18:41:56.423+0000 7f8679836640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86580066a0 0x7f8658006af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T18:41:57.878 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-11-28T18:41:57.878 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2758, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2590, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2356, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1611, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-11-28T18:41:57.892 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-28T18:41:57.893 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T18:41:57.893 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:41:57.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:41:57.896 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2758, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2590, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2356, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1611, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2023-11-28T18:41:57.906 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T18:41:57.908 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T18:41:57.908 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:41:57.909 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T18:41:57.909 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T18:41:57.909 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:41:57.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:41:57.909 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2758, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2590, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2356, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1611, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi148 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b' 2023-11-28T18:41:57.922 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T18:41:57.922 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T18:41:57.922 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:41:57.922 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T18:41:57.922 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-28T18:41:57.923 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-28T18:41:57.923 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:41:57.923 INFO:tasks.ceph.osd.0:Stopped 2023-11-28T18:41:57.923 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-28T18:41:57.923 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:41:57.923 INFO:tasks.ceph.osd.1:Stopped 2023-11-28T18:41:57.924 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-28T18:41:57.924 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:41:57.924 INFO:tasks.ceph.osd.2:Stopped 2023-11-28T18:41:57.924 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-28T18:41:57.924 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:41:57.925 INFO:tasks.ceph.osd.3:Stopped 2023-11-28T18:41:57.925 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-28T18:41:57.925 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:41:57.925 INFO:tasks.ceph.osd.4:Stopped 2023-11-28T18:41:57.926 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-28T18:41:57.926 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:41:57.926 INFO:tasks.ceph.osd.5:Stopped 2023-11-28T18:41:57.926 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-28T18:41:57.926 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:41:57.926 INFO:tasks.ceph.osd.6:Stopped 2023-11-28T18:41:57.928 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-28T18:41:57.929 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:41:57.929 INFO:tasks.ceph.osd.7:Stopped 2023-11-28T18:41:57.929 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-28T18:41:57.929 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-28T18:41:57.929 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:41:57.930 INFO:tasks.ceph.mgr.y:Stopped 2023-11-28T18:41:57.930 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-28T18:41:57.930 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:41:57.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:41:57.930 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2758, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2590, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2356, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1611, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi148 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x' 2023-11-28T18:41:57.942 INFO:tasks.ceph.mgr.x:Stopped 2023-11-28T18:41:57.942 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-28T18:41:57.943 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-28T18:41:57.943 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:41:57.943 INFO:tasks.ceph.mon.a:Stopped 2023-11-28T18:41:57.943 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-28T18:41:57.943 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:41:57.944 INFO:tasks.ceph.mon.b:Stopped 2023-11-28T18:41:57.944 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-28T18:41:57.944 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T18:41:57.946 INFO:tasks.ceph.mon.c:Stopped 2023-11-28T18:41:57.946 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-28T18:41:57.947 DEBUG:teuthology.orchestra.run.smithi105:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Replacing daemon mds' | head -n 1 2023-11-28T18:41:57.989 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:41:57.989 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-28T18:41:58.272 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:41:58.272 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-28T18:41:58.486 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:41:58.486 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-28T18:41:58.697 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:41:58.698 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-28T18:41:58.889 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:41:58.889 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-28T18:41:59.049 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:41:59.049 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-28T18:41:59.185 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:41:59.185 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-28T18:41:59.353 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:41:59.354 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-28T18:41:59.634 INFO:tasks.ceph:Archiving mon data... 2023-11-28T18:41:59.635 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2023-11-28_06:23:42-fs-wip-vshankar-testing-20231127.102654-testing-default-smithi/7469459/data/mon.a.tgz 2023-11-28T18:41:59.635 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2023-11-28T18:41:59.645 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.hc0SgRhBek 2023-11-28T18:41:59.645 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.hc0SgRhBek 2023-11-28T18:41:59.799 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.hc0SgRhBek 2023-11-28T18:41:59.880 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.hc0SgRhBek is 449KB 2023-11-28T18:41:59.950 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.hc0SgRhBek 2023-11-28T18:41:59.959 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2023-11-28_06:23:42-fs-wip-vshankar-testing-20231127.102654-testing-default-smithi/7469459/data/mon.b.tgz 2023-11-28T18:41:59.959 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2023-11-28T18:41:59.968 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.jT0tttlpJy 2023-11-28T18:41:59.969 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.jT0tttlpJy 2023-11-28T18:42:00.130 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.jT0tttlpJy 2023-11-28T18:42:00.206 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.jT0tttlpJy is 466KB 2023-11-28T18:42:00.272 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.jT0tttlpJy 2023-11-28T18:42:00.282 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2023-11-28_06:23:42-fs-wip-vshankar-testing-20231127.102654-testing-default-smithi/7469459/data/mon.c.tgz 2023-11-28T18:42:00.282 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2023-11-28T18:42:00.334 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.G6NuByB6mF 2023-11-28T18:42:00.334 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.G6NuByB6mF 2023-11-28T18:42:00.487 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.G6NuByB6mF 2023-11-28T18:42:00.567 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.G6NuByB6mF is 465KB 2023-11-28T18:42:00.637 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.G6NuByB6mF 2023-11-28T18:42:00.646 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-28T18:42:00.646 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T18:42:00.650 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T18:42:00.713 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-28T18:42:00.713 DEBUG:teuthology.orchestra.run.smithi105:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T18:42:00.718 DEBUG:teuthology.orchestra.run.smithi148:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T18:42:00.752 INFO:teuthology.orchestra.run.smithi105.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T18:42:00.787 INFO:teuthology.orchestra.run.smithi148.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T18:42:00.794 INFO:tasks.ceph:Archiving crash dumps... 2023-11-28T18:42:00.796 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-11-28_06:23:42-fs-wip-vshankar-testing-20231127.102654-testing-default-smithi/7469459/remote/smithi105/crash 2023-11-28T18:42:00.797 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T18:42:00.847 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-11-28_06:23:42-fs-wip-vshankar-testing-20231127.102654-testing-default-smithi/7469459/remote/smithi148/crash 2023-11-28T18:42:00.848 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T18:42:00.908 INFO:tasks.ceph:Shutting down logrotate 2023-11-28T18:42:00.909 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:42:00.912 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T18:42:00.992 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T18:42:01.012 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T18:42:01.032 INFO:tasks.ceph:Compressing logs... 2023-11-28T18:42:01.032 DEBUG:teuthology.orchestra.run.smithi105:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-28T18:42:01.058 DEBUG:teuthology.orchestra.run.smithi148:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-28T18:42:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67432.log 2023-11-28T18:42:01.088 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54946.log 2023-11-28T18:42:01.089 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59599.log 2023-11-28T18:42:01.089 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67432.log.gz 2023-11-28T18:42:01.090 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53004.log 2023-11-28T18:42:01.090 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54946.log.gz 2023-11-28T18:42:01.091 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60595.log 2023-11-28T18:42:01.092 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59599.log.gz 2023-11-28T18:42:01.094 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45503.log 2023-11-28T18:42:01.094 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53004.log.gz 2023-11-28T18:42:01.094 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47123.log 2023-11-28T18:42:01.095 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60595.log.gz 2023-11-28T18:42:01.095 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49498.log 2023-11-28T18:42:01.095 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45503.log.gz 2023-11-28T18:42:01.096 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49708.log 2023-11-28T18:42:01.097 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47123.log.gz 2023-11-28T18:42:01.097 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49498.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49498.log.gz 2023-11-28T18:42:01.097 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61726.log 2023-11-28T18:42:01.098 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49708.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49708.log.gz 2023-11-28T18:42:01.098 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64883.log 2023-11-28T18:42:01.099 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57588.log 2023-11-28T18:42:01.099 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61726.log.gz 2023-11-28T18:42:01.100 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64883.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64883.log.gz 2023-11-28T18:42:01.100 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66653.log 2023-11-28T18:42:01.100 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67618.log 2023-11-28T18:42:01.100 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57588.log.gz 2023-11-28T18:42:01.101 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66653.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66653.log.gz 2023-11-28T18:42:01.101 INFO:teuthology.orchestra.run.smithi105.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68834.log 2023-11-28T18:42:01.102 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54203.log 2023-11-28T18:42:01.102 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67618.log.gz 2023-11-28T18:42:01.103 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.68834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47383.log 2023-11-28T18:42:01.103 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.68834.log.gz 2023-11-28T18:42:01.104 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67342.log 2023-11-28T18:42:01.104 INFO:teuthology.orchestra.run.smithi105.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54203.log.gz 2023-11-28T18:42:01.105 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47383.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47383.log.gz 2023-11-28T18:42:01.105 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53602.log 2023-11-28T18:42:01.106 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52270.log 2023-11-28T18:42:01.106 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67342.log.gz 2023-11-28T18:42:01.106 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38444.log 2023-11-28T18:42:01.107 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53602.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.53602.log.gz 2023-11-28T18:42:01.107 INFO:teuthology.orchestra.run.smithi105.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59893.log 2023-11-28T18:42:01.108 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38396.log 2023-11-28T18:42:01.108 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.38444.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-28T18:42:01.109 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38444.log.gz 2023-11-28T18:42:01.109 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52297.log 2023-11-28T18:42:01.110 INFO:teuthology.orchestra.run.smithi105.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52270.log.gz 2023-11-28T18:42:01.110 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59893.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59893.log.gz -5 2023-11-28T18:42:01.110 INFO:teuthology.orchestra.run.smithi105.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49982.log 2023-11-28T18:42:01.110 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52153.log 2023-11-28T18:42:01.110 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52297.log.gz 2023-11-28T18:42:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66925.log 2023-11-28T18:42:01.113 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49982.log.gz 2023-11-28T18:42:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47754.log 2023-11-28T18:42:01.113 INFO:teuthology.orchestra.run.smithi105.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52153.log.gz 2023-11-28T18:42:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66925.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66925.log.gz -5 2023-11-28T18:42:01.114 INFO:teuthology.orchestra.run.smithi105.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52088.log 2023-11-28T18:42:01.114 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.38396.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-28T18:42:01.114 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38396.log.gz 2023-11-28T18:42:01.114 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38348.log 2023-11-28T18:42:01.115 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-28T18:42:01.115 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.38348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38420.log 2023-11-28T18:42:01.115 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38348.log.gz 2023-11-28T18:42:01.115 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-28T18:42:01.115 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.38420.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38420.log.gz 2023-11-28T18:42:01.117 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-28T18:42:01.117 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.44317.log 2023-11-28T18:42:01.119 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52589.log 2023-11-28T18:42:01.119 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47754.log.gz 2023-11-28T18:42:01.119 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52088.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52088.log.gzgzip 2023-11-28T18:42:01.119 INFO:teuthology.orchestra.run.smithi105.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61324.log 2023-11-28T18:42:01.119 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64779.log 2023-11-28T18:42:01.119 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52589.log.gz 2023-11-28T18:42:01.119 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57197.log 2023-11-28T18:42:01.120 INFO:teuthology.orchestra.run.smithi105.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.61324.log.gz 2023-11-28T18:42:01.120 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64779.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64779.log.gz 2023-11-28T18:42:01.120 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47435.log 2023-11-28T18:42:01.120 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54411.log 2023-11-28T18:42:01.120 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57197.log.gz 2023-11-28T18:42:01.123 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-28T18:42:01.123 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.3.44317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44203.log 2023-11-28T18:42:01.123 INFO:teuthology.orchestra.run.smithi148.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.3.44317.log.gz/var/log/ceph/ceph-mon.c.log: 2023-11-28T18:42:01.123 INFO:teuthology.orchestra.run.smithi148.stderr: 92.7% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-28T18:42:01.124 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38468.log 2023-11-28T18:42:01.124 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.1.44203.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-28T18:42:01.124 INFO:teuthology.orchestra.run.smithi148.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.1.44203.log.gz 2023-11-28T18:42:01.125 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45904.log 2023-11-28T18:42:01.125 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47435.log.gz 2023-11-28T18:42:01.125 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54411.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.54411.log.gz 2023-11-28T18:42:01.126 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48311.log 2023-11-28T18:42:01.126 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53066.log 2023-11-28T18:42:01.126 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45904.log.gz 2023-11-28T18:42:01.126 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47702.log 2023-11-28T18:42:01.126 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48311.log.gz 2023-11-28T18:42:01.126 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57145.log 2023-11-28T18:42:01.126 INFO:teuthology.orchestra.run.smithi105.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.53066.log.gz 2023-11-28T18:42:01.127 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47702.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47702.log.gz 2023-11-28T18:42:01.127 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67172.log 2023-11-28T18:42:01.127 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54281.log 2023-11-28T18:42:01.127 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57145.log.gz 2023-11-28T18:42:01.128 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.38468.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38468.log.gz 2023-11-28T18:42:01.128 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-28T18:42:01.128 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-28T18:42:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67172.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67172.log.gz 2023-11-28T18:42:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45910.log 2023-11-28T18:42:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65418.log 2023-11-28T18:42:01.130 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54281.log.gz 2023-11-28T18:42:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45910.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45910.log.gz 2023-11-28T18:42:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59971.log 2023-11-28T18:42:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65418.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-28T18:42:01.131 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65418.log.gz 2023-11-28T18:42:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53458.log 2023-11-28T18:42:01.131 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59971.log.gz 2023-11-28T18:42:01.132 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mds.b.log: 94.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-28T18:42:01.132 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-11-28T18:42:01.132 INFO:teuthology.orchestra.run.smithi148.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-28T18:42:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47097.log 2023-11-28T18:42:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52978.log 2023-11-28T18:42:01.134 INFO:teuthology.orchestra.run.smithi105.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53458.log.gz 2023-11-28T18:42:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51686.log 2023-11-28T18:42:01.134 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47097.log.gz 2023-11-28T18:42:01.135 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38492.log 2023-11-28T18:42:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48837.log 2023-11-28T18:42:01.136 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52978.log.gz 2023-11-28T18:42:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51686.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51686.log.gz 2023-11-28T18:42:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49630.log 2023-11-28T18:42:01.136 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.38492.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38492.log.gz 2023-11-28T18:42:01.137 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38372.log 2023-11-28T18:42:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67081.log 2023-11-28T18:42:01.138 INFO:teuthology.orchestra.run.smithi105.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48837.log.gz 2023-11-28T18:42:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49298.log 2023-11-28T18:42:01.138 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49630.log.gz 2023-11-28T18:42:01.139 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.38372.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38372.log.gz 2023-11-28T18:42:01.139 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-28T18:42:01.141 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62344.log 2023-11-28T18:42:01.141 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67081.log.gz 2023-11-28T18:42:01.141 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58146.log 2023-11-28T18:42:01.141 INFO:teuthology.orchestra.run.smithi105.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49298.log.gz 2023-11-28T18:42:01.141 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62344.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62344.log.gzgzip 2023-11-28T18:42:01.142 INFO:teuthology.orchestra.run.smithi105.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60487.log 2023-11-28T18:42:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.58146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47513.log 2023-11-28T18:42:01.142 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58146.log.gz 2023-11-28T18:42:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47279.log 2023-11-28T18:42:01.143 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60487.log.gz 2023-11-28T18:42:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66523.log 2023-11-28T18:42:01.144 INFO:teuthology.orchestra.run.smithi105.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47513.log.gz 2023-11-28T18:42:01.144 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38516.log 2023-11-28T18:42:01.145 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47279.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47279.log.gz 2023-11-28T18:42:01.145 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51634.log 2023-11-28T18:42:01.146 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47175.log 2023-11-28T18:42:01.146 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66523.log.gz 2023-11-28T18:42:01.147 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51634.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51634.log.gz 2023-11-28T18:42:01.147 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51478.log 2023-11-28T18:42:01.147 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50958.log 2023-11-28T18:42:01.148 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47175.log.gz 2023-11-28T18:42:01.148 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66125.log 2023-11-28T18:42:01.148 INFO:teuthology.orchestra.run.smithi105.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51478.log.gz 2023-11-28T18:42:01.149 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50958.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.33415.log 2023-11-28T18:42:01.149 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50958.log.gz 2023-11-28T18:42:01.150 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45428.log 2023-11-28T18:42:01.150 INFO:teuthology.orchestra.run.smithi105.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.66125.log.gz 2023-11-28T18:42:01.151 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph.tmp-client.admin.33415.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.33415.log.gz 2023-11-28T18:42:01.151 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50117.log 2023-11-28T18:42:01.152 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.38516.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38516.log.gz 2023-11-28T18:42:01.152 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46908.log 2023-11-28T18:42:01.153 INFO:teuthology.orchestra.run.smithi105.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45428.log.gz 2023-11-28T18:42:01.153 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50117.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50117.log.gz 2023-11-28T18:42:01.154 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62777.log 2023-11-28T18:42:01.154 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57444.log 2023-11-28T18:42:01.155 INFO:teuthology.orchestra.run.smithi105.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46908.log.gz 2023-11-28T18:42:01.155 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56437.log 2023-11-28T18:42:01.155 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62777.log.gz 2023-11-28T18:42:01.156 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62635.log 2023-11-28T18:42:01.156 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57444.log.gz 2023-11-28T18:42:01.157 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62136.log 2023-11-28T18:42:01.157 INFO:teuthology.orchestra.run.smithi105.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.56437.log.gz 2023-11-28T18:42:01.158 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60943.log 2023-11-28T18:42:01.158 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62635.log.gz 2023-11-28T18:42:01.159 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55478.log 2023-11-28T18:42:01.159 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62136.log.gz 2023-11-28T18:42:01.160 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60943.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60943.log.gz 2023-11-28T18:42:01.160 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48020.log 2023-11-28T18:42:01.161 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55478.log: gzip -5 88.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.55478.log.gz /var/log/ceph/ceph-client.admin.64909.log 2023-11-28T18:42:01.161 INFO:teuthology.orchestra.run.smithi105.stderr: 2023-11-28T18:42:01.162 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61046.log 2023-11-28T18:42:01.162 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48020.log.gz 2023-11-28T18:42:01.163 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64035.log 2023-11-28T18:42:01.163 INFO:teuthology.orchestra.run.smithi105.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.64909.log.gz 2023-11-28T18:42:01.164 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48177.log 2023-11-28T18:42:01.164 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61046.log.gz 2023-11-28T18:42:01.165 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64035.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64035.log.gz 2023-11-28T18:42:01.165 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57642.log 2023-11-28T18:42:01.166 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48494.log 2023-11-28T18:42:01.166 INFO:teuthology.orchestra.run.smithi105.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48177.log.gz 2023-11-28T18:42:01.167 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46934.log 2023-11-28T18:42:01.167 INFO:teuthology.orchestra.run.smithi105.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57642.log.gz 2023-11-28T18:42:01.168 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48494.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48494.log.gz 2023-11-28T18:42:01.168 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48283.log 2023-11-28T18:42:01.169 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46934.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-28T18:42:01.169 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46934.log.gz 2023-11-28T18:42:01.170 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49930.log 2023-11-28T18:42:01.170 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48283.log.gz 2023-11-28T18:42:01.171 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66385.log 2023-11-28T18:42:01.172 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57785.log 2023-11-28T18:42:01.172 INFO:teuthology.orchestra.run.smithi105.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49930.log.gz 2023-11-28T18:42:01.173 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51045.log 2023-11-28T18:42:01.173 INFO:teuthology.orchestra.run.smithi105.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66385.log.gz 2023-11-28T18:42:01.173 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57785.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57785.log.gz 2023-11-28T18:42:01.174 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44954.log 2023-11-28T18:42:01.175 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66873.log 2023-11-28T18:42:01.176 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44954.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.51045.log.gz 2023-11-28T18:42:01.176 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44954.log.gz 2023-11-28T18:42:01.177 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45715.log 2023-11-28T18:42:01.178 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55000.log 2023-11-28T18:42:01.178 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66873.log.gz 2023-11-28T18:42:01.184 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45715.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45715.log.gz 2023-11-28T18:42:01.184 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51660.log 2023-11-28T18:42:01.184 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55000.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55000.log.gz 2023-11-28T18:42:01.185 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62403.log 2023-11-28T18:42:01.185 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51660.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51660.log.gz 2023-11-28T18:42:01.185 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46548.log 2023-11-28T18:42:01.185 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62403.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62403.log.gz 2023-11-28T18:42:01.185 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49097.log 2023-11-28T18:42:01.185 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56311.log 2023-11-28T18:42:01.186 INFO:teuthology.orchestra.run.smithi105.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.46548.log.gz 2023-11-28T18:42:01.186 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49097.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49097.log.gz 2023-11-28T18:42:01.188 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66575.log 2023-11-28T18:42:01.188 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56311.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56311.log.gz 2023-11-28T18:42:01.188 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45163.log 2023-11-28T18:42:01.188 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66575.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66575.log.gz 2023-11-28T18:42:01.188 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65392.log 2023-11-28T18:42:01.188 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45163.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45163.log.gz 2023-11-28T18:42:01.189 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54501.log 2023-11-28T18:42:01.190 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65392.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65392.log.gz 2023-11-28T18:42:01.190 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53332.log 2023-11-28T18:42:01.190 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54501.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54501.log.gz 2023-11-28T18:42:01.190 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54657.log 2023-11-28T18:42:01.191 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53332.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53332.log.gz 2023-11-28T18:42:01.191 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49904.log 2023-11-28T18:42:01.192 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54657.log: gzip -5 --verbose -- 89.1% /var/log/ceph/ceph-client.admin.47487.log -- replaced with /var/log/ceph/ceph-client.admin.54657.log.gz 2023-11-28T18:42:01.192 INFO:teuthology.orchestra.run.smithi105.stderr: 2023-11-28T18:42:01.193 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49904.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49904.log.gz 2023-11-28T18:42:01.193 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57941.log 2023-11-28T18:42:01.194 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47487.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47487.log.gz 2023-11-28T18:42:01.194 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54553.log 2023-11-28T18:42:01.195 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57941.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57941.log.gz 2023-11-28T18:42:01.195 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47539.log 2023-11-28T18:42:01.196 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54553.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54553.log.gz 2023-11-28T18:42:01.196 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45324.log 2023-11-28T18:42:01.197 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47539.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47539.log.gz 2023-11-28T18:42:01.197 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45896.log 2023-11-28T18:42:01.198 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45324.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45324.log.gz 2023-11-28T18:42:01.198 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50036.log 2023-11-28T18:42:01.199 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45896.log: gzip -5 88.7% --verbose -- replaced with /var/log/ceph/ceph-client.admin.45896.log.gz 2023-11-28T18:42:01.200 INFO:teuthology.orchestra.run.smithi105.stderr: -- /var/log/ceph/ceph-client.admin.65988.log 2023-11-28T18:42:01.200 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50036.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50036.log.gz 2023-11-28T18:42:01.201 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53408.log 2023-11-28T18:42:01.201 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65988.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65988.log.gz 2023-11-28T18:42:01.202 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59841.log 2023-11-28T18:42:01.202 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46182.log 2023-11-28T18:42:01.203 INFO:teuthology.orchestra.run.smithi105.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.53408.log.gz 2023-11-28T18:42:01.204 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59841.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.59841.log.gz 2023-11-28T18:42:01.204 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47253.log 2023-11-28T18:42:01.205 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46182.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46182.log.gz 2023-11-28T18:42:01.205 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59677.log 2023-11-28T18:42:01.206 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47253.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47253.log.gz 2023-11-28T18:42:01.206 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51392.log 2023-11-28T18:42:01.207 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59677.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59677.log.gz 2023-11-28T18:42:01.207 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55272.log 2023-11-28T18:42:01.208 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51392.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51392.log.gz 2023-11-28T18:42:01.208 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65196.log 2023-11-28T18:42:01.209 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55272.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55272.log.gz 2023-11-28T18:42:01.209 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62240.log 2023-11-28T18:42:01.210 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65196.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65196.log.gz 2023-11-28T18:42:01.210 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57757.log 2023-11-28T18:42:01.211 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62240.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62240.log.gz 2023-11-28T18:42:01.211 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61994.log 2023-11-28T18:42:01.212 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57757.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57757.log.gz 2023-11-28T18:42:01.213 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48967.log 2023-11-28T18:42:01.213 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61994.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61994.log.gz 2023-11-28T18:42:01.214 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64481.log 2023-11-28T18:42:01.214 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48967.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48967.log.gz 2023-11-28T18:42:01.215 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44290.log 2023-11-28T18:42:01.215 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64481.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64481.log.gz 2023-11-28T18:42:01.216 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67484.log 2023-11-28T18:42:01.217 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44290.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44290.log.gz 2023-11-28T18:42:01.217 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45699.log 2023-11-28T18:42:01.218 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67484.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67484.log.gz 2023-11-28T18:42:01.218 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66757.log 2023-11-28T18:42:01.219 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45699.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45699.log.gz 2023-11-28T18:42:01.219 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61648.log 2023-11-28T18:42:01.220 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66757.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66757.log.gz 2023-11-28T18:42:01.220 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46286.log 2023-11-28T18:42:01.221 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61648.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61648.log.gz 2023-11-28T18:42:01.221 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48572.log 2023-11-28T18:42:01.222 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46286.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46286.log.gz 2023-11-28T18:42:01.222 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62803.log 2023-11-28T18:42:01.223 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48572.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48572.log.gz 2023-11-28T18:42:01.223 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46496.log 2023-11-28T18:42:01.224 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62803.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62803.log.gz 2023-11-28T18:42:01.224 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69919.log 2023-11-28T18:42:01.225 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46496.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46496.log.gz 2023-11-28T18:42:01.225 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66411.log 2023-11-28T18:42:01.226 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.69919.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69919.log.gz 2023-11-28T18:42:01.227 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52846.log 2023-11-28T18:42:01.227 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66411.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66411.log.gz 2023-11-28T18:42:01.228 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68734.log 2023-11-28T18:42:01.228 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52846.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52846.log.gz 2023-11-28T18:42:01.229 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45006.log 2023-11-28T18:42:01.230 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.68734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67055.log 2023-11-28T18:42:01.231 INFO:teuthology.orchestra.run.smithi105.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.68734.log.gz 2023-11-28T18:42:01.231 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45006.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45006.log.gz 2023-11-28T18:42:01.232 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44696.log 2023-11-28T18:42:01.232 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67055.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67055.log.gz 2023-11-28T18:42:01.233 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59763.log 2023-11-28T18:42:01.234 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44696.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44696.log.gz 2023-11-28T18:42:01.234 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46104.log 2023-11-28T18:42:01.235 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59763.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59763.log.gz 2023-11-28T18:42:01.235 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44646.log 2023-11-28T18:42:01.236 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46104.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46104.log.gz 2023-11-28T18:42:01.236 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66437.log 2023-11-28T18:42:01.236 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64987.log 2023-11-28T18:42:01.237 INFO:teuthology.orchestra.run.smithi105.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.44646.log.gz 2023-11-28T18:42:01.237 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66437.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66437.log.gz 2023-11-28T18:42:01.238 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55375.log 2023-11-28T18:42:01.239 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64987.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64987.log.gz 2023-11-28T18:42:01.239 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49324.log 2023-11-28T18:42:01.240 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55375.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55375.log.gz 2023-11-28T18:42:01.240 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51738.log 2023-11-28T18:42:01.241 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49324.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49324.log.gz 2023-11-28T18:42:01.241 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38444.log 2023-11-28T18:42:01.242 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51738.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51738.log.gz 2023-11-28T18:42:01.242 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54683.log 2023-11-28T18:42:01.243 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.38444.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38444.log.gz 2023-11-28T18:42:01.243 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67875.log 2023-11-28T18:42:01.244 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54683.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54683.log.gzgzip 2023-11-28T18:42:01.244 INFO:teuthology.orchestra.run.smithi105.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55114.log 2023-11-28T18:42:01.245 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67875.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67875.log.gz 2023-11-28T18:42:01.245 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44159.log 2023-11-28T18:42:01.246 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55114.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55114.log.gz 2023-11-28T18:42:01.246 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54013.log 2023-11-28T18:42:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44159.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44159.log.gz 2023-11-28T18:42:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58072.log 2023-11-28T18:42:01.248 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54013.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54013.log.gz 2023-11-28T18:42:01.248 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49798.log 2023-11-28T18:42:01.249 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.0.58072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61812.log 2023-11-28T18:42:01.250 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49798.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49798.log.gz 2023-11-28T18:42:01.251 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66463.log 2023-11-28T18:42:01.252 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65678.log/var/log/ceph/ceph-client.admin.61812.log: 2023-11-28T18:42:01.253 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66463.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61812.log.gz 2023-11-28T18:42:01.253 INFO:teuthology.orchestra.run.smithi105.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66463.log.gz 2023-11-28T18:42:01.254 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55168.log 2023-11-28T18:42:01.254 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65678.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65678.log.gz 2023-11-28T18:42:01.255 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48993.log 2023-11-28T18:42:01.256 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60042.log 2023-11-28T18:42:01.256 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55168.log.gz 2023-11-28T18:42:01.257 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48993.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48993.log.gz 2023-11-28T18:42:01.257 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56767.log 2023-11-28T18:42:01.258 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60042.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60042.log.gz 2023-11-28T18:42:01.258 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69867.log 2023-11-28T18:42:01.259 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56767.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56767.log.gz 2023-11-28T18:42:01.259 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45696.log 2023-11-28T18:42:01.260 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.69867.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.69867.log.gz 2023-11-28T18:42:01.260 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45716.log 2023-11-28T18:42:01.261 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45696.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45696.log.gz 2023-11-28T18:42:01.262 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56871.log 2023-11-28T18:42:01.262 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45716.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45716.log.gz 2023-11-28T18:42:01.263 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66705.log 2023-11-28T18:42:01.264 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56337.log 2023-11-28T18:42:01.264 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56871.log: /var/log/ceph/ceph-client.admin.66705.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56871.log.gz 2023-11-28T18:42:01.265 INFO:teuthology.orchestra.run.smithi105.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66705.log.gz 2023-11-28T18:42:01.265 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46609.log 2023-11-28T18:42:01.266 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66601.log 2023-11-28T18:42:01.266 INFO:teuthology.orchestra.run.smithi105.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.56337.log.gz 2023-11-28T18:42:01.267 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46609.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46609.log.gz 2023-11-28T18:42:01.267 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48681.log 2023-11-28T18:42:01.268 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66601.log: gzip -5 --verbose 89.7% -- -- replaced with /var/log/ceph/ceph-client.admin.66601.log.gz 2023-11-28T18:42:01.268 INFO:teuthology.orchestra.run.smithi105.stderr: /var/log/ceph/ceph-client.admin.47409.log 2023-11-28T18:42:01.269 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61187.log 2023-11-28T18:42:01.270 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48681.log.gz 2023-11-28T18:42:01.271 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47409.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47409.log.gz 2023-11-28T18:42:01.271 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62318.log 2023-11-28T18:42:01.271 INFO:teuthology.orchestra.run.smithi105.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-28T18:42:01.272 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61187.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61187.log.gz -5 2023-11-28T18:42:01.272 INFO:teuthology.orchestra.run.smithi105.stderr: --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-28T18:42:01.273 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62318.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62318.log.gz 2023-11-28T18:42:01.273 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56897.log 2023-11-28T18:42:01.274 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45220.log 2023-11-28T18:42:01.275 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56897.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56897.log.gz 2023-11-28T18:42:01.276 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47806.log 2023-11-28T18:42:01.276 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45220.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45220.log.gz 2023-11-28T18:42:01.278 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65039.log 2023-11-28T18:42:01.278 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47806.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47806.log.gz 2023-11-28T18:42:01.279 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60224.log 2023-11-28T18:42:01.280 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47045.log 2023-11-28T18:42:01.281 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65039.log.gz 2023-11-28T18:42:01.281 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60224.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60224.log.gz 2023-11-28T18:42:01.282 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48203.log 2023-11-28T18:42:01.283 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47045.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47045.log.gz 2023-11-28T18:42:01.283 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60737.log 2023-11-28T18:42:01.284 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48203.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48203.log.gz 2023-11-28T18:42:01.285 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57671.log 2023-11-28T18:42:01.287 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47859.log 2023-11-28T18:42:01.288 INFO:teuthology.orchestra.run.smithi105.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60737.log.gz 2023-11-28T18:42:01.289 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57671.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57671.log.gz 2023-11-28T18:42:01.290 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60172.log 2023-11-28T18:42:01.290 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47859.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47859.log.gz 2023-11-28T18:42:01.290 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64637.log 2023-11-28T18:42:01.291 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60172.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60172.log.gz 2023-11-28T18:42:01.291 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67592.log 2023-11-28T18:42:01.292 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64637.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.64637.log.gz 2023-11-28T18:42:01.292 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64611.log 2023-11-28T18:42:01.293 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68684.log 2023-11-28T18:42:01.293 INFO:teuthology.orchestra.run.smithi105.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67592.log.gz 2023-11-28T18:42:01.294 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64611.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64611.log.gz 2023-11-28T18:42:01.294 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47149.log 2023-11-28T18:42:01.296 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.68684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49552.log 2023-11-28T18:42:01.296 INFO:teuthology.orchestra.run.smithi105.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.68684.log.gz 2023-11-28T18:42:01.297 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47149.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47149.log.gz 2023-11-28T18:42:01.297 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44490.log 2023-11-28T18:42:01.298 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62581.log 2023-11-28T18:42:01.299 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49552.log.gz 2023-11-28T18:42:01.300 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45500.log 2023-11-28T18:42:01.300 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44490.log.gz 2023-11-28T18:42:01.301 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62581.log: gzip -5 --verbose -- 88.8% /var/log/ceph/ceph-client.admin.48255.log 2023-11-28T18:42:01.302 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62581.log.gz 2023-11-28T18:42:01.303 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54800.log 2023-11-28T18:42:01.303 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45500.log.gz 2023-11-28T18:42:01.305 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44902.log 2023-11-28T18:42:01.305 INFO:teuthology.orchestra.run.smithi105.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48255.log.gz 2023-11-28T18:42:01.306 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61374.log 2023-11-28T18:42:01.307 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54800.log.gz 2023-11-28T18:42:01.308 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70156.log 2023-11-28T18:42:01.308 INFO:teuthology.orchestra.run.smithi105.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44902.log.gz 2023-11-28T18:42:01.309 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60094.log 2023-11-28T18:42:01.310 INFO:teuthology.orchestra.run.smithi105.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.61374.log.gz 2023-11-28T18:42:01.311 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.70156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50275.log 2023-11-28T18:42:01.312 INFO:teuthology.orchestra.run.smithi105.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70156.log.gz 2023-11-28T18:42:01.312 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60094.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60094.log.gz 2023-11-28T18:42:01.312 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70129.log 2023-11-28T18:42:01.314 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.0.50275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60539.log 2023-11-28T18:42:01.315 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.70129.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70129.log.gz 2023-11-28T18:42:01.315 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45915.log 2023-11-28T18:42:01.316 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62907.log 2023-11-28T18:42:01.317 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60539.log.gz 2023-11-28T18:42:01.318 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45915.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-28T18:42:01.318 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45915.log.gz 2023-11-28T18:42:01.319 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62907.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.62907.log.gz 2023-11-28T18:42:01.319 INFO:teuthology.orchestra.run.smithi105.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.62829.log 2023-11-28T18:42:01.320 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44516.log 2023-11-28T18:42:01.322 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60302.log 2023-11-28T18:42:01.322 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62829.log.gz 2023-11-28T18:42:01.323 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44516.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44516.log.gz 2023-11-28T18:42:01.323 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48441.log 2023-11-28T18:42:01.325 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60302.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60302.log.gz 2023-11-28T18:42:01.325 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48546.log 2023-11-28T18:42:01.326 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67564.log 2023-11-28T18:42:01.326 INFO:teuthology.orchestra.run.smithi105.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48441.log.gz 2023-11-28T18:42:01.328 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48546.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48546.log.gz 2023-11-28T18:42:01.329 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67564.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67564.log.gz 2023-11-28T18:42:01.329 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46208.log 2023-11-28T18:42:01.331 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48811.log 2023-11-28T18:42:01.332 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67458.log 2023-11-28T18:42:01.332 INFO:teuthology.orchestra.run.smithi105.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46208.log.gz 2023-11-28T18:42:01.334 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65733.log 2023-11-28T18:42:01.334 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48811.log.gz 2023-11-28T18:42:01.335 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49350.log 2023-11-28T18:42:01.335 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67458.log.gz 2023-11-28T18:42:01.338 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.0.65733.log: /var/log/ceph/ceph-client.admin.49350.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49350.log.gz 2023-11-28T18:42:01.339 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67538.log 2023-11-28T18:42:01.340 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59651.log 2023-11-28T18:42:01.341 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67538.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67538.log.gz 2023-11-28T18:42:01.342 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59651.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59651.log.gz 2023-11-28T18:42:01.343 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65498.log 2023-11-28T18:42:01.344 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67107.log 2023-11-28T18:42:01.346 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65498.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65498.log.gz 2023-11-28T18:42:01.347 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67107.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67107.log.gz 2023-11-28T18:42:01.348 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61864.log 2023-11-28T18:42:01.349 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56387.log 2023-11-28T18:42:01.351 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61864.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61864.log.gz 2023-11-28T18:42:01.352 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56387.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56387.log.gz 2023-11-28T18:42:01.353 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61890.log 2023-11-28T18:42:01.354 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70260.log 2023-11-28T18:42:01.356 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61890.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61890.log.gz 2023-11-28T18:42:01.357 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.70260.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70260.log.gz 2023-11-28T18:42:01.357 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61968.log 2023-11-28T18:42:01.359 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59253.log 2023-11-28T18:42:01.360 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61968.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61968.log.gz 2023-11-28T18:42:01.361 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59253.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59253.log.gz 2023-11-28T18:42:01.362 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51906.log 2023-11-28T18:42:01.364 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62292.log 2023-11-28T18:42:01.365 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51906.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51906.log.gz 2023-11-28T18:42:01.366 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62292.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62292.log.gz 2023-11-28T18:42:01.367 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52244.log 2023-11-28T18:42:01.368 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47331.log 2023-11-28T18:42:01.370 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52244.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.52244.log.gz 2023-11-28T18:42:01.371 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51802.log 2023-11-28T18:42:01.372 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47331.log.gz 2023-11-28T18:42:01.373 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49682.log 2023-11-28T18:42:01.375 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51802.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51802.log.gz 2023-11-28T18:42:01.376 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49682.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49682.log.gz 2023-11-28T18:42:01.376 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.46864.log 2023-11-28T18:42:01.378 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68632.log 2023-11-28T18:42:01.380 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.2.46864.log: 91.5% -- replaced with /var/log/ceph/ceph-client.2.46864.log.gz 2023-11-28T18:42:01.381 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.68632.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68632.log.gz 2023-11-28T18:42:01.381 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45111.log 2023-11-28T18:42:01.382 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45505.log 2023-11-28T18:42:01.384 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45111.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45111.log.gz 2023-11-28T18:42:01.385 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45505.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45505.log.gz 2023-11-28T18:42:01.386 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44980.log 2023-11-28T18:42:01.387 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61674.log 2023-11-28T18:42:01.388 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44980.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44980.log.gz 2023-11-28T18:42:01.389 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61674.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61674.log.gz 2023-11-28T18:42:01.390 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64585.log 2023-11-28T18:42:01.392 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60513.log 2023-11-28T18:42:01.393 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64585.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.64585.log.gz 2023-11-28T18:42:01.394 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60513.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60513.log.gz 2023-11-28T18:42:01.395 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56707.log 2023-11-28T18:42:01.397 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52511.log 2023-11-28T18:42:01.398 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56707.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56707.log.gz 2023-11-28T18:42:01.399 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52511.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52511.log.gz 2023-11-28T18:42:01.400 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50349.log 2023-11-28T18:42:01.401 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56949.log 2023-11-28T18:42:01.403 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50349.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50349.log.gz 2023-11-28T18:42:01.404 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56949.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56949.log.gz 2023-11-28T18:42:01.404 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62110.log 2023-11-28T18:42:01.406 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55194.log 2023-11-28T18:42:01.407 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62110.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62110.log.gz 2023-11-28T18:42:01.408 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55194.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55194.log.gz 2023-11-28T18:42:01.409 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45496.log 2023-11-28T18:42:01.411 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60276.log 2023-11-28T18:42:01.412 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45496.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45496.log.gz 2023-11-28T18:42:01.413 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60276.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60276.log.gz 2023-11-28T18:42:01.414 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54359.log 2023-11-28T18:42:01.415 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65332.log 2023-11-28T18:42:01.417 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54359.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54359.log.gz 2023-11-28T18:42:01.419 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65332.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65332.log.gz 2023-11-28T18:42:01.419 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52459.log 2023-11-28T18:42:01.420 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45059.log 2023-11-28T18:42:01.422 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52459.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52459.log.gz 2023-11-28T18:42:01.423 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45059.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45059.log.gz 2023-11-28T18:42:01.423 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65524.log 2023-11-28T18:42:01.425 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52121.log 2023-11-28T18:42:01.427 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65524.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65524.log.gz 2023-11-28T18:42:01.428 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52121.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52121.log.gz 2023-11-28T18:42:01.428 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61786.log 2023-11-28T18:42:01.430 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64455.log 2023-11-28T18:42:01.431 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61786.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61786.log.gz 2023-11-28T18:42:01.432 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64455.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.64455.log.gz 2023-11-28T18:42:01.433 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60655.log 2023-11-28T18:42:01.434 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49956.log 2023-11-28T18:42:01.436 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60655.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60655.log.gz 2023-11-28T18:42:01.437 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49956.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49956.log.gz 2023-11-28T18:42:01.437 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59815.log 2023-11-28T18:42:01.439 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51958.log 2023-11-28T18:42:01.440 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59815.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59815.log.gz 2023-11-28T18:42:01.442 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51958.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51958.log.gz 2023-11-28T18:42:01.442 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48629.log 2023-11-28T18:42:01.444 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66627.log 2023-11-28T18:42:01.446 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48629.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48629.log.gz 2023-11-28T18:42:01.447 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66627.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66627.log.gz 2023-11-28T18:42:01.447 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68936.log 2023-11-28T18:42:01.449 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68606.log 2023-11-28T18:42:01.451 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.68936.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68936.log.gz 2023-11-28T18:42:01.452 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.68606.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68606.log.gz 2023-11-28T18:42:01.452 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66899.log 2023-11-28T18:42:01.454 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60198.log 2023-11-28T18:42:01.455 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66899.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66899.log.gz 2023-11-28T18:42:01.456 INFO:teuthology.orchestra.run.smithi148.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-28T18:42:01.457 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60198.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60198.log.gz 2023-11-28T18:42:01.457 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64961.log 2023-11-28T18:42:01.458 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56629.log 2023-11-28T18:42:01.460 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64961.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64961.log.gz 2023-11-28T18:42:01.461 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56629.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56629.log.gz 2023-11-28T18:42:01.462 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45194.log 2023-11-28T18:42:01.463 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46729.log 2023-11-28T18:42:01.464 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45194.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45194.log.gz 2023-11-28T18:42:01.466 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.0.46729.log: 90.5% -- replaced with /var/log/ceph/ceph-client.0.46729.log.gz 2023-11-28T18:42:01.467 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46442.log 2023-11-28T18:42:01.469 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70017.log 2023-11-28T18:42:01.470 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46442.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46442.log.gz 2023-11-28T18:42:01.471 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.70017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56819.log 2023-11-28T18:42:01.472 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70017.log.gz 2023-11-28T18:42:01.473 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48759.log 2023-11-28T18:42:01.475 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56819.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56819.log.gz 2023-11-28T18:42:01.476 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48759.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48759.log.gz 2023-11-28T18:42:01.476 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52772.log 2023-11-28T18:42:01.478 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54177.log 2023-11-28T18:42:01.481 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.0.52772.log: /var/log/ceph/ceph-client.admin.54177.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.54177.log.gz 2023-11-28T18:42:01.482 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43994.log 2023-11-28T18:42:01.485 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43994.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43994.log.gz 2023-11-28T18:42:01.485 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60998.log 2023-11-28T18:42:01.491 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.0.60998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57535.log 2023-11-28T18:42:01.496 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60068.log 2023-11-28T18:42:01.498 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57535.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57535.log.gz 2023-11-28T18:42:01.501 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45402.log 2023-11-28T18:42:01.502 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60068.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60068.log.gz 2023-11-28T18:42:01.504 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62723.log 2023-11-28T18:42:01.505 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45402.log.gz 2023-11-28T18:42:01.509 INFO:teuthology.orchestra.run.smithi105.stderr:gzip/var/log/ceph/ceph-client.admin.62723.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48468.log 2023-11-28T18:42:01.510 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62723.log.gz 2023-11-28T18:42:01.513 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64188.log 2023-11-28T18:42:01.514 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48468.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48468.log.gz 2023-11-28T18:42:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63113.log 2023-11-28T18:42:01.518 INFO:teuthology.orchestra.run.smithi105.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.64188.log.gz 2023-11-28T18:42:01.525 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56655.log 2023-11-28T18:42:01.526 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.63113.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63113.log.gz 2023-11-28T18:42:01.532 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56655.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59279.log 2023-11-28T18:42:01.535 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56655.log.gz 2023-11-28T18:42:01.536 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49446.log 2023-11-28T18:42:01.538 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59279.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59279.log.gz 2023-11-28T18:42:01.541 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49446.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49446.log.gz 2023-11-28T18:42:01.542 INFO:teuthology.orchestra.run.smithi148.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-28T18:42:01.543 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54065.log 2023-11-28T18:42:01.548 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44542.log 2023-11-28T18:42:01.549 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54065.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54065.log.gz 2023-11-28T18:42:01.552 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51582.log 2023-11-28T18:42:01.553 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44542.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44542.log.gz 2023-11-28T18:42:01.555 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59227.log 2023-11-28T18:42:01.556 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51582.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51582.log.gz 2023-11-28T18:42:01.560 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65302.log 2023-11-28T18:42:01.562 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59227.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59227.log.gz 2023-11-28T18:42:01.563 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57275.log 2023-11-28T18:42:01.564 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65302.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65302.log.gz 2023-11-28T18:42:01.569 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57275.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57275.log.gz 2023-11-28T18:42:01.571 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57053.log 2023-11-28T18:42:01.576 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46522.log 2023-11-28T18:42:01.577 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57053.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57053.log.gz 2023-11-28T18:42:01.580 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65472.log 2023-11-28T18:42:01.581 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46522.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46522.log.gz 2023-11-28T18:42:01.583 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61942.log 2023-11-28T18:42:01.584 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65472.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65472.log.gz 2023-11-28T18:42:01.590 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52667.log 2023-11-28T18:42:01.591 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61942.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61942.log.gz 2023-11-28T18:42:01.591 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67029.log 2023-11-28T18:42:01.598 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52667.log: /var/log/ceph/ceph-client.admin.67029.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.52667.log.gz 2023-11-28T18:42:01.598 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67029.log.gz 2023-11-28T18:42:01.599 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52537.log 2023-11-28T18:42:01.601 INFO:teuthology.orchestra.run.smithi148.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-28T18:42:01.604 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46364.log 2023-11-28T18:42:01.605 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52537.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52537.log.gz 2023-11-28T18:42:01.608 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55028.log 2023-11-28T18:42:01.609 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46364.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46364.log.gz 2023-11-28T18:42:01.611 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-28T18:42:01.612 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55028.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55028.log.gz 2023-11-28T18:42:01.615 INFO:teuthology.orchestra.run.smithi105.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.0.65733.log.gz 2023-11-28T18:42:01.616 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65013.log 2023-11-28T18:42:01.619 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47201.log 2023-11-28T18:42:01.620 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65013.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65013.log.gz 2023-11-28T18:42:01.623 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47201.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47201.log.gz 2023-11-28T18:42:01.623 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45923.log 2023-11-28T18:42:01.628 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54579.log 2023-11-28T18:42:01.629 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45923.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45923.log.gz 2023-11-28T18:42:01.631 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67290.log 2023-11-28T18:42:01.632 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54579.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54579.log.gz 2023-11-28T18:42:01.640 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62933.log 2023-11-28T18:42:01.642 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67290.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67290.log.gz 2023-11-28T18:42:01.643 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45710.log 2023-11-28T18:42:01.644 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62933.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62933.log.gz 2023-11-28T18:42:01.646 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44820.log 2023-11-28T18:42:01.647 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45710.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45710.log.gz 2023-11-28T18:42:01.655 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44820.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.44820.log.gz 2023-11-28T18:42:01.655 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54091.log 2023-11-28T18:42:01.660 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70312.log 2023-11-28T18:42:01.661 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54091.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54091.log.gz 2023-11-28T18:42:01.663 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69861.log 2023-11-28T18:42:01.664 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.70312.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70312.log.gz 2023-11-28T18:42:01.672 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45490.log 2023-11-28T18:42:01.674 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.69861.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69861.log.gz 2023-11-28T18:42:01.676 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70096.log 2023-11-28T18:42:01.678 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45490.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45490.log.gz 2023-11-28T18:42:01.679 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52218.log 2023-11-28T18:42:01.683 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.70096.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70096.log.gz 2023-11-28T18:42:01.686 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52218.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52218.log.gz 2023-11-28T18:42:01.687 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69913.log 2023-11-28T18:42:01.692 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51828.log 2023-11-28T18:42:01.693 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.69913.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69913.log.gz 2023-11-28T18:42:01.696 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53831.log 2023-11-28T18:42:01.697 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51828.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51828.log.gz 2023-11-28T18:42:01.700 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48785.log 2023-11-28T18:42:01.700 INFO:teuthology.orchestra.run.smithi105.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.60998.log.gz 2023-11-28T18:42:01.701 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53831.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.53831.log.gz 2023-11-28T18:42:01.706 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45924.log 2023-11-28T18:42:01.706 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48785.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48785.log.gz 2023-11-28T18:42:01.707 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53755.log 2023-11-28T18:42:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45924.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45924.log.gz 2023-11-28T18:42:01.716 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53755.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.53755.log.gz 2023-11-28T18:42:01.716 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67003.log 2023-11-28T18:42:01.718 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67003.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67003.log.gz 2023-11-28T18:42:01.718 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62162.log 2023-11-28T18:42:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62162.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62162.log.gz 2023-11-28T18:42:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51712.log 2023-11-28T18:42:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51712.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51712.log.gz 2023-11-28T18:42:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66847.log 2023-11-28T18:42:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66847.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66847.log.gz 2023-11-28T18:42:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59405.log 2023-11-28T18:42:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59405.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.59405.log.gz 2023-11-28T18:42:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70232.log 2023-11-28T18:42:01.739 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.70232.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70232.log.gz 2023-11-28T18:42:01.739 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62607.log 2023-11-28T18:42:01.739 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62607.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62607.log.gz 2023-11-28T18:42:01.739 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61621.log 2023-11-28T18:42:01.742 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61621.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61621.log.gz 2023-11-28T18:42:01.742 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70395.log 2023-11-28T18:42:01.745 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.70395.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70395.log.gz 2023-11-28T18:42:01.746 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48655.log 2023-11-28T18:42:01.749 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48655.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48655.log.gz 2023-11-28T18:42:01.749 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62526.log 2023-11-28T18:42:01.752 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48733.log 2023-11-28T18:42:01.752 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62526.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62526.log.gz 2023-11-28T18:42:01.755 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48733.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48733.log.gz 2023-11-28T18:42:01.756 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66175.log 2023-11-28T18:42:01.756 INFO:teuthology.orchestra.run.smithi105.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.52772.log.gz 2023-11-28T18:42:01.758 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66175.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.66175.log.gz 2023-11-28T18:42:01.759 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54893.log 2023-11-28T18:42:01.761 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59737.log 2023-11-28T18:42:01.762 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54893.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.54893.log.gz 2023-11-28T18:42:01.763 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57915.log 2023-11-28T18:42:01.764 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59737.log.gz 2023-11-28T18:42:01.765 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56681.log 2023-11-28T18:42:01.766 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57915.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57915.log.gz 2023-11-28T18:42:01.768 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56845.log 2023-11-28T18:42:01.769 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56681.log.gz 2023-11-28T18:42:01.769 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47357.log 2023-11-28T18:42:01.771 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56845.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56845.log.gz 2023-11-28T18:42:01.772 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47357.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47357.log.gz 2023-11-28T18:42:01.772 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55142.log 2023-11-28T18:42:01.774 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46260.log 2023-11-28T18:42:01.775 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55142.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55142.log.gz 2023-11-28T18:42:01.777 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46260.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46260.log.gz 2023-11-28T18:42:01.777 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45708.log 2023-11-28T18:42:01.779 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47643.log 2023-11-28T18:42:01.780 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45708.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45708.log.gz 2023-11-28T18:42:01.781 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47643.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47643.log.gz 2023-11-28T18:42:01.783 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44264.log 2023-11-28T18:42:01.784 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57171.log 2023-11-28T18:42:01.785 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44264.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44264.log.gz 2023-11-28T18:42:01.787 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57171.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57171.log.gz 2023-11-28T18:42:01.787 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65366.log 2023-11-28T18:42:01.789 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65274.log 2023-11-28T18:42:01.790 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65366.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65366.log.gz 2023-11-28T18:42:01.791 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65274.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65274.log.gz 2023-11-28T18:42:01.792 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49878.log 2023-11-28T18:42:01.793 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52036.log 2023-11-28T18:42:01.795 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49878.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49878.log.gz 2023-11-28T18:42:01.796 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52036.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52036.log.gz 2023-11-28T18:42:01.796 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48151.log 2023-11-28T18:42:01.798 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67901.log 2023-11-28T18:42:01.800 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48151.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48151.log.gz 2023-11-28T18:42:01.802 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67901.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.67901.log.gz 2023-11-28T18:42:01.802 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64138.log 2023-11-28T18:42:01.803 INFO:teuthology.orchestra.run.smithi148.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-28T18:42:01.803 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62046.log 2023-11-28T18:42:01.805 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64138.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.64138.log.gz 2023-11-28T18:42:01.805 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62046.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62046.log.gz 2023-11-28T18:42:01.806 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60389.log 2023-11-28T18:42:01.808 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48229.log 2023-11-28T18:42:01.809 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60389.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60389.log.gz 2023-11-28T18:42:01.810 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48229.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48229.log.gz 2023-11-28T18:42:01.811 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64062.log 2023-11-28T18:42:01.812 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53572.log 2023-11-28T18:42:01.814 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64062.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64062.log.gz 2023-11-28T18:42:01.815 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53572.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53572.log.gz 2023-11-28T18:42:01.815 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66359.log 2023-11-28T18:42:01.817 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-28T18:42:01.819 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66359.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66359.log.gz 2023-11-28T18:42:01.823 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57327.log 2023-11-28T18:42:01.825 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62881.log 2023-11-28T18:42:01.826 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57327.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.57327.log.gz 2023-11-28T18:42:01.828 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49175.log 2023-11-28T18:42:01.830 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69832.log 2023-11-28T18:42:01.831 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49175.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49175.log.gz 2023-11-28T18:42:01.833 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.69832.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62881.log.gz 2023-11-28T18:42:01.835 INFO:teuthology.orchestra.run.smithi105.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69832.log.gz 2023-11-28T18:42:01.836 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57889.log 2023-11-28T18:42:01.838 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57731.log 2023-11-28T18:42:01.839 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57889.log.gz 2023-11-28T18:42:01.841 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67238.log 2023-11-28T18:42:01.846 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57731.log: 87.8%/var/log/ceph/ceph-client.admin.67238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44094.log 2023-11-28T18:42:01.846 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57731.log.gz 2023-11-28T18:42:01.847 INFO:teuthology.orchestra.run.smithi105.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67238.log.gz 2023-11-28T18:42:01.850 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5/var/log/ceph/ceph-client.admin.44094.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.44094.log.gz 2023-11-28T18:42:01.851 INFO:teuthology.orchestra.run.smithi105.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48337.log 2023-11-28T18:42:01.854 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48337.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48337.log.gz 2023-11-28T18:42:01.854 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44212.log 2023-11-28T18:42:01.857 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44212.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44212.log.gz 2023-11-28T18:42:01.857 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47565.log 2023-11-28T18:42:01.861 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47565.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47565.log.gz 2023-11-28T18:42:01.861 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49382.log 2023-11-28T18:42:01.863 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49382.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49382.log.gz 2023-11-28T18:42:01.864 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53546.log 2023-11-28T18:42:01.867 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53546.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53546.log.gz 2023-11-28T18:42:01.867 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65170.log 2023-11-28T18:42:01.870 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65170.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65170.log.gz 2023-11-28T18:42:01.870 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51019.log 2023-11-28T18:42:01.873 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65104.log 2023-11-28T18:42:01.873 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51019.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51019.log.gz 2023-11-28T18:42:01.877 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65104.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65104.log.gz 2023-11-28T18:42:01.877 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64088.log 2023-11-28T18:42:01.880 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64088.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.64088.log.gz 2023-11-28T18:42:01.881 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49604.log 2023-11-28T18:42:01.884 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48098.log 2023-11-28T18:42:01.884 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49604.log.gz 2023-11-28T18:42:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70266.log 2023-11-28T18:42:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48098.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48098.log.gz 2023-11-28T18:42:01.888 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54475.log 2023-11-28T18:42:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.70266.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.70266.log.gz 2023-11-28T18:42:01.892 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54475.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54475.log.gz 2023-11-28T18:42:01.893 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60815.log 2023-11-28T18:42:01.896 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60815.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60815.log.gz 2023-11-28T18:42:01.897 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55530.log 2023-11-28T18:42:01.900 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55530.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55530.log.gz 2023-11-28T18:42:01.900 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53655.log 2023-11-28T18:42:01.903 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53655.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.53655.log.gz 2023-11-28T18:42:01.904 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54307.log 2023-11-28T18:42:01.905 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47227.log 2023-11-28T18:42:01.909 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49045.log 2023-11-28T18:42:01.911 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3%/var/log/ceph/ceph-client.admin.47227.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54307.log.gz 2023-11-28T18:42:01.911 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47227.log.gz 2023-11-28T18:42:01.913 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59625.log 2023-11-28T18:42:01.918 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59625.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59625.log.gz 2023-11-28T18:42:01.918 INFO:teuthology.orchestra.run.smithi105.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49045.log.gz 2023-11-28T18:42:01.919 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44061.log 2023-11-28T18:42:01.921 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44061.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44061.log.gz 2023-11-28T18:42:01.922 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58017.log 2023-11-28T18:42:01.925 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.58017.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58017.log.gz 2023-11-28T18:42:01.925 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51984.log 2023-11-28T18:42:01.928 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51984.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51984.log.gz 2023-11-28T18:42:01.928 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53931.log 2023-11-28T18:42:01.932 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53931.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53931.log.gz 2023-11-28T18:42:01.932 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50008.log 2023-11-28T18:42:01.939 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50008.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50008.log.gz 2023-11-28T18:42:01.939 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44928.log 2023-11-28T18:42:01.939 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44928.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44928.log.gz 2023-11-28T18:42:01.939 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59355.log 2023-11-28T18:42:01.943 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59355.log: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.59355.log.gz 2023-11-28T18:42:01.943 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50169.log 2023-11-28T18:42:01.946 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50169.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.50169.log.gz 2023-11-28T18:42:01.946 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54605.log 2023-11-28T18:42:01.949 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54605.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54605.log.gz 2023-11-28T18:42:01.950 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60681.log 2023-11-28T18:42:01.952 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52405.log 2023-11-28T18:42:01.954 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60681.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60681.log.gz 2023-11-28T18:42:01.957 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52405.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52405.log.gz 2023-11-28T18:42:01.957 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62697.log 2023-11-28T18:42:01.960 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62697.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62697.log.gz 2023-11-28T18:42:01.960 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66075.log 2023-11-28T18:42:01.964 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66075.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.66075.log.gz 2023-11-28T18:42:01.965 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47676.log 2023-11-28T18:42:01.968 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47676.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47676.log.gz 2023-11-28T18:42:01.968 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55504.log 2023-11-28T18:42:01.971 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55504.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55504.log.gz 2023-11-28T18:42:01.971 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38224.log 2023-11-28T18:42:01.974 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.38224.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.38224.log.gz 2023-11-28T18:42:01.975 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58120.log 2023-11-28T18:42:01.978 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.58120.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58120.log.gz 2023-11-28T18:42:01.978 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51880.log 2023-11-28T18:42:01.981 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5/var/log/ceph/ceph-client.admin.51880.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51880.log.gz 2023-11-28T18:42:01.982 INFO:teuthology.orchestra.run.smithi105.stderr: --verbose -- /var/log/ceph/ceph-client.admin.68658.log 2023-11-28T18:42:01.985 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.68658.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68658.log.gz 2023-11-28T18:42:01.985 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51504.log 2023-11-28T18:42:01.988 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51504.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51504.log.gz 2023-11-28T18:42:01.988 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64343.log 2023-11-28T18:42:01.992 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64343.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64343.log.gz 2023-11-28T18:42:01.992 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57837.log 2023-11-28T18:42:01.995 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57837.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57837.log.gz 2023-11-28T18:42:01.995 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54527.log 2023-11-28T18:42:01.998 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54527.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54527.log.gz 2023-11-28T18:42:01.998 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47591.log 2023-11-28T18:42:02.001 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47591.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47591.log.gz 2023-11-28T18:42:02.002 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60461.log 2023-11-28T18:42:02.006 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-28T18:42:02.006 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60461.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60461.log.gz 2023-11-28T18:42:02.007 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65072.log 2023-11-28T18:42:02.011 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph.log: 86.9% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-28T18:42:02.013 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52062.log 2023-11-28T18:42:02.015 INFO:teuthology.orchestra.run.smithi105.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65072.log.gz 2023-11-28T18:42:02.018 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63065.log 2023-11-28T18:42:02.024 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.0.63065.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52062.log.gz 2023-11-28T18:42:02.025 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53705.log 2023-11-28T18:42:02.030 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53705.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.53705.log.gz 2023-11-28T18:42:02.031 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70023.log 2023-11-28T18:42:02.037 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61760.log 2023-11-28T18:42:02.038 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.70023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49019.log 2023-11-28T18:42:02.038 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70023.log.gz 2023-11-28T18:42:02.045 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61760.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61760.log.gz 2023-11-28T18:42:02.045 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55246.log 2023-11-28T18:42:02.047 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49019.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49019.log.gz 2023-11-28T18:42:02.048 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68986.log 2023-11-28T18:42:02.050 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55246.log: gzip 88.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.55246.log.gz --verbose 2023-11-28T18:42:02.050 INFO:teuthology.orchestra.run.smithi105.stderr: -- /var/log/ceph/ceph-client.admin.56793.log 2023-11-28T18:42:02.053 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.68986.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.68986.log.gz 2023-11-28T18:42:02.056 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44568.log 2023-11-28T18:42:02.058 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56793.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56793.log.gz 2023-11-28T18:42:02.060 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44568.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44568.log.gz 2023-11-28T18:42:02.060 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70069.log 2023-11-28T18:42:02.065 INFO:teuthology.orchestra.run.smithi105.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.0.50275.log.gz 2023-11-28T18:42:02.065 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.70069.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70069.log.gz 2023-11-28T18:42:02.067 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49764.log 2023-11-28T18:42:02.070 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45919.log 2023-11-28T18:42:02.071 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49764.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49764.log.gz 2023-11-28T18:42:02.078 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57027.log 2023-11-28T18:42:02.078 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45919.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45919.log.gz 2023-11-28T18:42:02.082 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59789.log 2023-11-28T18:42:02.082 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57027.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57027.log.gz 2023-11-28T18:42:02.086 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51556.log 2023-11-28T18:42:02.086 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59789.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59789.log.gz 2023-11-28T18:42:02.089 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44318.log 2023-11-28T18:42:02.089 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51556.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51556.log.gz 2023-11-28T18:42:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48046.log 2023-11-28T18:42:02.092 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44318.log.gz 2023-11-28T18:42:02.094 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52820.log 2023-11-28T18:42:02.095 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48046.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48046.log.gz 2023-11-28T18:42:02.097 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64429.log 2023-11-28T18:42:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52820.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52820.log.gz 2023-11-28T18:42:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64429.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64429.log.gz 2023-11-28T18:42:02.101 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64395.log 2023-11-28T18:42:02.104 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64395.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64395.log.gz 2023-11-28T18:42:02.105 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66783.log 2023-11-28T18:42:02.108 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66783.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.66783.log.gz 2023-11-28T18:42:02.108 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45272.log 2023-11-28T18:42:02.111 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45272.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45272.log.gz 2023-11-28T18:42:02.112 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53987.log 2023-11-28T18:42:02.115 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53987.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53987.log.gz 2023-11-28T18:42:02.115 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46390.log 2023-11-28T18:42:02.118 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46390.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46390.log.gz 2023-11-28T18:42:02.118 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70369.log 2023-11-28T18:42:02.122 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.70369.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70369.log.gz 2023-11-28T18:42:02.122 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49526.log 2023-11-28T18:42:02.125 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64001.log 2023-11-28T18:42:02.125 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49526.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49526.log.gz 2023-11-28T18:42:02.128 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64001.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64001.log.gz 2023-11-28T18:42:02.128 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61221.log 2023-11-28T18:42:02.132 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61221.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61221.log.gz 2023-11-28T18:42:02.132 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47071.log 2023-11-28T18:42:02.135 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47071.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47071.log.gz 2023-11-28T18:42:02.135 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47617.log 2023-11-28T18:42:02.138 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47617.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47617.log.gz 2023-11-28T18:42:02.139 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55298.log 2023-11-28T18:42:02.142 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55298.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55298.log.gz 2023-11-28T18:42:02.142 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57379.log 2023-11-28T18:42:02.144 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44620.log 2023-11-28T18:42:02.146 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57379.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57379.log.gz 2023-11-28T18:42:02.149 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44620.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44620.log.gz 2023-11-28T18:42:02.149 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58172.log 2023-11-28T18:42:02.152 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.58172.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58172.log.gz 2023-11-28T18:42:02.153 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49123.log 2023-11-28T18:42:02.156 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49123.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49123.log.gz 2023-11-28T18:42:02.156 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49850.log 2023-11-28T18:42:02.163 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49850.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49850.log.gz 2023-11-28T18:42:02.163 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54972.log 2023-11-28T18:42:02.163 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54972.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54972.log.gz 2023-11-28T18:42:02.164 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62749.log 2023-11-28T18:42:02.167 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62749.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62749.log.gz 2023-11-28T18:42:02.167 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69965.log 2023-11-28T18:42:02.170 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.69965.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69965.log.gz 2023-11-28T18:42:02.170 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59573.log 2023-11-28T18:42:02.173 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59573.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.59573.log.gz 2023-11-28T18:42:02.174 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59945.log 2023-11-28T18:42:02.180 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59945.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.59945.log.gz 2023-11-28T18:42:02.180 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48072.log 2023-11-28T18:42:02.180 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48072.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48072.log.gz 2023-11-28T18:42:02.181 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62855.log 2023-11-28T18:42:02.183 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62855.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62855.log.gz 2023-11-28T18:42:02.184 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52563.log 2023-11-28T18:42:02.187 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52563.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52563.log.gz 2023-11-28T18:42:02.187 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45454.log 2023-11-28T18:42:02.191 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45454.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45454.log.gz 2023-11-28T18:42:02.191 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59997.log 2023-11-28T18:42:02.194 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59997.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59997.log.gz 2023-11-28T18:42:02.194 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47833.log 2023-11-28T18:42:02.197 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47833.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47833.log.gz 2023-11-28T18:42:02.198 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54768.log 2023-11-28T18:42:02.201 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54768.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54768.log.gz 2023-11-28T18:42:02.201 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50219.log 2023-11-28T18:42:02.204 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45137.log 2023-11-28T18:42:02.205 INFO:teuthology.orchestra.run.smithi105.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50219.log.gz 2023-11-28T18:42:02.210 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62020.log 2023-11-28T18:42:02.210 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45137.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45137.log.gz 2023-11-28T18:42:02.212 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62020.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62020.log.gz 2023-11-28T18:42:02.212 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54735.log 2023-11-28T18:42:02.215 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54735.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54735.log.gz 2023-11-28T18:42:02.216 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47461.log 2023-11-28T18:42:02.219 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47461.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47461.log.gz 2023-11-28T18:42:02.219 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59711.log 2023-11-28T18:42:02.222 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59711.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59711.log.gz 2023-11-28T18:42:02.222 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62500.log 2023-11-28T18:42:02.226 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62500.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62500.log.gz 2023-11-28T18:42:02.226 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55430.log 2023-11-28T18:42:02.231 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.0.55430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63010.log 2023-11-28T18:42:02.237 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57561.log 2023-11-28T18:42:02.238 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.63010.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63010.log.gz 2023-11-28T18:42:02.241 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57561.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57561.log.gz 2023-11-28T18:42:02.243 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51444.log 2023-11-28T18:42:02.248 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51608.log 2023-11-28T18:42:02.250 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56923.log 2023-11-28T18:42:02.250 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51444.log.gz 2023-11-28T18:42:02.253 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51608.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51608.log.gz 2023-11-28T18:42:02.256 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60120.log 2023-11-28T18:42:02.258 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56923.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56923.log.gz 2023-11-28T18:42:02.259 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60120.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60120.log.gz 2023-11-28T18:42:02.260 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60146.log 2023-11-28T18:42:02.265 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60146.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60146.log.gz 2023-11-28T18:42:02.267 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70343.log 2023-11-28T18:42:02.272 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67316.log 2023-11-28T18:42:02.274 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.70343.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70343.log.gz 2023-11-28T18:42:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47780.log 2023-11-28T18:42:02.278 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67316.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67316.log.gz 2023-11-28T18:42:02.280 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64715.log 2023-11-28T18:42:02.282 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47780.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47780.log.gz 2023-11-28T18:42:02.284 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68860.log 2023-11-28T18:42:02.286 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64715.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64715.log.gz 2023-11-28T18:42:02.287 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64559.log 2023-11-28T18:42:02.288 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.68860.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68860.log.gz 2023-11-28T18:42:02.294 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64559.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64559.log.gz 2023-11-28T18:42:02.295 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67746.log 2023-11-28T18:42:02.300 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64507.log 2023-11-28T18:42:02.302 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67746.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67746.log.gz 2023-11-28T18:42:02.304 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50091.log 2023-11-28T18:42:02.306 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64507.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.64507.log.gz 2023-11-28T18:42:02.307 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57705.log 2023-11-28T18:42:02.308 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50091.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50091.log.gz 2023-11-28T18:42:02.312 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60789.log 2023-11-28T18:42:02.314 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57705.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57705.log.gz 2023-11-28T18:42:02.318 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60789.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60789.log.gz 2023-11-28T18:42:02.319 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45246.log 2023-11-28T18:42:02.324 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49472.log 2023-11-28T18:42:02.327 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45246.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45246.log.gz 2023-11-28T18:42:02.328 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65628.log 2023-11-28T18:42:02.330 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49472.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49472.log.gz 2023-11-28T18:42:02.336 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46312.log 2023-11-28T18:42:02.338 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65628.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.65628.log.gz 2023-11-28T18:42:02.340 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60629.log 2023-11-28T18:42:02.341 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46312.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46312.log.gz 2023-11-28T18:42:02.344 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51366.log 2023-11-28T18:42:02.345 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60629.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60629.log.gz 2023-11-28T18:42:02.349 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51366.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51366.log.gz 2023-11-28T18:42:02.355 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46130.log 2023-11-28T18:42:02.360 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45508.log 2023-11-28T18:42:02.362 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46130.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46130.log.gz 2023-11-28T18:42:02.364 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61248.log 2023-11-28T18:42:02.366 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45508.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45508.log.gz 2023-11-28T18:42:02.369 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48415.log 2023-11-28T18:42:02.370 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61248.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61248.log.gz 2023-11-28T18:42:02.376 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64533.log 2023-11-28T18:42:02.377 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48415.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48415.log.gz 2023-11-28T18:42:02.383 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46416.log 2023-11-28T18:42:02.383 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64533.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.64533.log.gz 2023-11-28T18:42:02.384 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67370.log 2023-11-28T18:42:02.387 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46416.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46416.log.gz 2023-11-28T18:42:02.392 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64831.log 2023-11-28T18:42:02.393 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67370.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67370.log.gz 2023-11-28T18:42:02.396 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47305.log 2023-11-28T18:42:02.398 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64831.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64831.log.gz 2023-11-28T18:42:02.400 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57967.log 2023-11-28T18:42:02.402 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47305.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47305.log.gz 2023-11-28T18:42:02.404 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50992.log 2023-11-28T18:42:02.410 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57967.log: /var/log/ceph/ceph-client.admin.50992.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50992.log.gz 2023-11-28T18:42:02.410 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55220.log 2023-11-28T18:42:02.412 INFO:teuthology.orchestra.run.smithi105.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.57967.log.gz 2023-11-28T18:42:02.418 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55220.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55220.log.gz 2023-11-28T18:42:02.419 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51418.log 2023-11-28T18:42:02.424 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69971.log 2023-11-28T18:42:02.426 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51418.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51418.log.gz 2023-11-28T18:42:02.428 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48863.log 2023-11-28T18:42:02.432 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.69971.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57117.log 2023-11-28T18:42:02.435 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48863.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69971.log.gz 2023-11-28T18:42:02.440 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57117.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48863.log.gz 2023-11-28T18:42:02.441 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64857.log 2023-11-28T18:42:02.443 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57117.log.gz 2023-11-28T18:42:02.444 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57509.log 2023-11-28T18:42:02.446 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64857.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64857.log.gz 2023-11-28T18:42:02.448 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65833.log 2023-11-28T18:42:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57509.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57509.log.gz 2023-11-28T18:42:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48941.log 2023-11-28T18:42:02.456 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65833.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52615.log 2023-11-28T18:42:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48941.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48941.log.gz 2023-11-28T18:42:02.459 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65833.log.gz 2023-11-28T18:42:02.460 INFO:teuthology.orchestra.run.smithi105.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.0.58072.log.gz 2023-11-28T18:42:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-28T18:42:02.462 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52615.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52615.log.gz 2023-11-28T18:42:02.463 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70208.log 2023-11-28T18:42:02.468 INFO:teuthology.orchestra.run.smithi105.stderr: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67140.log 2023-11-28T18:42:02.469 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.70208.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70208.log.gz 2023-11-28T18:42:02.470 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-28T18:42:02.472 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55062.log 2023-11-28T18:42:02.475 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67140.log: /var/log/ceph/ceph-client.admin.55062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48125.log 2023-11-28T18:42:02.476 INFO:teuthology.orchestra.run.smithi105.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67140.log.gz 2023-11-28T18:42:02.477 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55062.log.gz 2023-11-28T18:42:02.480 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68784.log 2023-11-28T18:42:02.484 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48125.log: gzip/var/log/ceph/ceph-client.admin.68784.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.57249.log 2023-11-28T18:42:02.484 INFO:teuthology.orchestra.run.smithi105.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68784.log.gz 2023-11-28T18:42:02.485 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48125.log.gz 2023-11-28T18:42:02.489 INFO:teuthology.orchestra.run.smithi105.stderr:gzip/var/log/ceph/ceph-client.admin.57249.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57249.log.gz 2023-11-28T18:42:02.489 INFO:teuthology.orchestra.run.smithi105.stderr: -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-28T18:42:02.491 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48389.log 2023-11-28T18:42:02.494 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48389.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48389.log.gz 2023-11-28T18:42:02.499 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45033.log 2023-11-28T18:42:02.502 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53881.log 2023-11-28T18:42:02.503 INFO:teuthology.orchestra.run.smithi105.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45033.log.gz 2023-11-28T18:42:02.504 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55325.log 2023-11-28T18:42:02.513 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53881.log: /var/log/ceph/ceph-client.admin.55325.log: 91.1% 90.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49578.log 2023-11-28T18:42:02.513 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55325.log.gz 2023-11-28T18:42:02.513 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53881.log.gz 2023-11-28T18:42:02.514 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66951.log 2023-11-28T18:42:02.520 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49578.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49578.log.gz 2023-11-28T18:42:02.521 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-28T18:42:02.522 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66951.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66951.log.gz 2023-11-28T18:42:02.527 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53166.log 2023-11-28T18:42:02.532 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47885.log 2023-11-28T18:42:02.534 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53166.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53166.log.gz 2023-11-28T18:42:02.536 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52717.log 2023-11-28T18:42:02.538 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47885.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47885.log.gz 2023-11-28T18:42:02.540 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62663.log 2023-11-28T18:42:02.541 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52717.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52717.log.gz 2023-11-28T18:42:02.548 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51145.log 2023-11-28T18:42:02.550 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62663.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62663.log.gz 2023-11-28T18:42:02.552 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49824.log 2023-11-28T18:42:02.559 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51145.log: 91.6%/var/log/ceph/ceph-client.admin.49824.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51145.log.gz 2023-11-28T18:42:02.559 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49071.log 2023-11-28T18:42:02.564 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49824.log.gz 2023-11-28T18:42:02.564 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59919.log 2023-11-28T18:42:02.566 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49071.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49071.log.gz 2023-11-28T18:42:02.567 INFO:teuthology.orchestra.run.smithi105.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-28T18:42:02.568 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50323.log 2023-11-28T18:42:02.570 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59919.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.59919.log.gz 2023-11-28T18:42:02.572 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44594.log 2023-11-28T18:42:02.574 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51932.log 2023-11-28T18:42:02.575 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44594.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44594.log.gz 2023-11-28T18:42:02.576 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50323.log.gz 2023-11-28T18:42:02.578 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49149.log 2023-11-28T18:42:02.581 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51932.log: /var/log/ceph/ceph-client.admin.49149.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51932.log.gz 2023-11-28T18:42:02.582 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49149.log.gz 2023-11-28T18:42:02.583 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68886.log 2023-11-28T18:42:02.588 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49201.log 2023-11-28T18:42:02.591 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.68886.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.68886.log.gz 2023-11-28T18:42:02.592 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57223.log 2023-11-28T18:42:02.593 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49201.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49201.log.gz 2023-11-28T18:42:02.595 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61072.log 2023-11-28T18:42:02.597 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57223.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57223.log.gz 2023-11-28T18:42:02.600 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67264.log 2023-11-28T18:42:02.602 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61072.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61072.log.gz 2023-11-28T18:42:02.606 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67264.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.67264.log.gz 2023-11-28T18:42:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66497.log 2023-11-28T18:42:02.612 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49227.log 2023-11-28T18:42:02.614 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66497.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66497.log.gz 2023-11-28T18:42:02.618 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66022.log 2023-11-28T18:42:02.618 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49227.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49227.log.gz 2023-11-28T18:42:02.621 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66022.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66022.log.gz 2023-11-28T18:42:02.623 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46156.log 2023-11-28T18:42:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45085.log 2023-11-28T18:42:02.630 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46156.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46156.log.gz 2023-11-28T18:42:02.632 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44374.log 2023-11-28T18:42:02.636 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45085.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51530.log 2023-11-28T18:42:02.637 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44374.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44374.log.gz 2023-11-28T18:42:02.639 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45085.log.gz 2023-11-28T18:42:02.640 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64805.log 2023-11-28T18:42:02.642 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51530.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51530.log.gz 2023-11-28T18:42:02.644 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64805.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64805.log.gz 2023-11-28T18:42:02.647 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54333.log 2023-11-28T18:42:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50143.log 2023-11-28T18:42:02.655 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54333.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54333.log.gz 2023-11-28T18:42:02.656 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61700.log 2023-11-28T18:42:02.657 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50143.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50143.log.gz 2023-11-28T18:42:02.659 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45491.log 2023-11-28T18:42:02.660 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61700.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61700.log.gz 2023-11-28T18:42:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57353.log 2023-11-28T18:42:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45491.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45491.log.gz 2023-11-28T18:42:02.667 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53508.log 2023-11-28T18:42:02.668 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57353.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57353.log.gz 2023-11-28T18:42:02.672 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57001.log 2023-11-28T18:42:02.674 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53508.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53508.log.gz 2023-11-28T18:42:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46234.log 2023-11-28T18:42:02.683 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57001.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57001.log.gz 2023-11-28T18:42:02.685 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46234.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46234.log.gz 2023-11-28T18:42:02.687 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67849.log 2023-11-28T18:42:02.692 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64291.log 2023-11-28T18:42:02.693 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67849.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67849.log.gz 2023-11-28T18:42:02.696 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56733.log 2023-11-28T18:42:02.699 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64291.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64291.log.gz 2023-11-28T18:42:02.700 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45704.log 2023-11-28T18:42:02.703 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56733.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.56733.log.gz 2023-11-28T18:42:02.704 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50375.log 2023-11-28T18:42:02.705 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45704.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45704.log.gz 2023-11-28T18:42:02.708 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67512.log 2023-11-28T18:42:02.711 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50375.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50375.log.gz 2023-11-28T18:42:02.718 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67512.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67512.log.gz 2023-11-28T18:42:02.719 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51298.log 2023-11-28T18:42:02.724 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67696.log 2023-11-28T18:42:02.725 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51298.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51298.log.gz 2023-11-28T18:42:02.728 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52433.log 2023-11-28T18:42:02.730 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67696.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.67696.log.gz 2023-11-28T18:42:02.732 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49736.log 2023-11-28T18:42:02.734 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52433.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52433.log.gz 2023-11-28T18:42:02.740 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53039.log 2023-11-28T18:42:02.742 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49736.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49736.log.gz 2023-11-28T18:42:02.744 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52351.log 2023-11-28T18:42:02.746 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53039.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53039.log.gz 2023-11-28T18:42:02.753 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52351.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52351.log.gz 2023-11-28T18:42:02.755 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44408.log 2023-11-28T18:42:02.760 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67801.log 2023-11-28T18:42:02.762 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44408.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44408.log.gz 2023-11-28T18:42:02.764 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48520.log 2023-11-28T18:42:02.772 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.0.67801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57811.log 2023-11-28T18:42:02.774 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48520.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48520.log.gz 2023-11-28T18:42:02.777 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54919.log 2023-11-28T18:42:02.778 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57811.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57811.log.gz 2023-11-28T18:42:02.783 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54919.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52485.log 2023-11-28T18:42:02.787 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54919.log.gz 2023-11-28T18:42:02.788 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54229.log 2023-11-28T18:42:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52485.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52485.log.gz 2023-11-28T18:42:02.792 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65222.log 2023-11-28T18:42:02.794 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54229.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54229.log.gz 2023-11-28T18:42:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45506.log 2023-11-28T18:42:02.802 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65222.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65222.log.gz 2023-11-28T18:42:02.804 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54385.log 2023-11-28T18:42:02.810 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45506.log: 88.8%/var/log/ceph/ceph-client.admin.54385.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54385.log.gz 2023-11-28T18:42:02.811 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45506.log.gz 2023-11-28T18:42:02.811 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52323.log 2023-11-28T18:42:02.817 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66049.log 2023-11-28T18:42:02.818 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52323.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52323.log.gz 2023-11-28T18:42:02.820 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70182.log 2023-11-28T18:42:02.825 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66049.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60250.log 2023-11-28T18:42:02.827 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.70182.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66049.log.gz 2023-11-28T18:42:02.827 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70182.log.gz 2023-11-28T18:42:02.828 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65602.log 2023-11-28T18:42:02.831 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60250.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60250.log.gz 2023-11-28T18:42:02.833 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47728.log 2023-11-28T18:42:02.839 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65602.log: 87.4%/var/log/ceph/ceph-client.admin.47728.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51854.log 2023-11-28T18:42:02.840 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65602.log.gz 2023-11-28T18:42:02.843 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47728.log.gz 2023-11-28T18:42:02.844 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61916.log 2023-11-28T18:42:02.846 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51854.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51854.log.gz 2023-11-28T18:42:02.848 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62552.log 2023-11-28T18:42:02.855 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61916.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61916.log.gz 2023-11-28T18:42:02.858 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62552.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62552.log.gz 2023-11-28T18:42:02.859 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47937.log 2023-11-28T18:42:02.865 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65446.log 2023-11-28T18:42:02.866 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47937.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47937.log.gz 2023-11-28T18:42:02.868 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66549.log 2023-11-28T18:42:02.871 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65446.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65446.log.gz 2023-11-28T18:42:02.873 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62435.log 2023-11-28T18:42:02.876 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66549.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45350.log 2023-11-28T18:42:02.879 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62435.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66549.log.gz 2023-11-28T18:42:02.879 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62435.log.gz 2023-11-28T18:42:02.880 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52010.log 2023-11-28T18:42:02.887 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45350.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45350.log.gz 2023-11-28T18:42:02.890 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52010.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52010.log.gz 2023-11-28T18:42:02.891 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60867.log 2023-11-28T18:42:02.897 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52379.log 2023-11-28T18:42:02.898 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60867.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60867.log.gz 2023-11-28T18:42:02.900 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47994.log 2023-11-28T18:42:02.902 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52379.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52379.log.gz 2023-11-28T18:42:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48598.log 2023-11-28T18:42:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47994.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61838.log 2023-11-28T18:42:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48598.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48598.log.gz 2023-11-28T18:42:02.911 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47994.log.gz 2023-11-28T18:42:02.916 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61838.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64935.log 2023-11-28T18:42:02.919 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61838.log.gz 2023-11-28T18:42:02.922 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64935.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64935.log.gz 2023-11-28T18:42:02.923 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54631.log 2023-11-28T18:42:02.928 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48889.log 2023-11-28T18:42:02.930 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54631.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54631.log.gz 2023-11-28T18:42:02.932 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55088.log 2023-11-28T18:42:02.937 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48889.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61098.log 2023-11-28T18:42:02.940 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.55088.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48889.log.gz 2023-11-28T18:42:02.940 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55088.log.gz 2023-11-28T18:42:02.941 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62214.log 2023-11-28T18:42:02.947 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61098.log: 88.7%gzip/var/log/ceph/ceph-client.admin.62214.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61098.log.gz 2023-11-28T18:42:02.948 INFO:teuthology.orchestra.run.smithi105.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59305.log 2023-11-28T18:42:02.951 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62214.log.gz 2023-11-28T18:42:02.952 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48363.log 2023-11-28T18:42:02.954 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59305.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.59305.log.gz 2023-11-28T18:42:02.956 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54709.log 2023-11-28T18:42:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48363.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62959.log 2023-11-28T18:42:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54709.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48363.log.gz 2023-11-28T18:42:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60893.log 2023-11-28T18:42:02.967 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62959.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.54709.log.gz 2023-11-28T18:42:02.968 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63139.log 2023-11-28T18:42:02.970 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60893.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.60893.log.gz 2023-11-28T18:42:02.971 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62959.log.gz 2023-11-28T18:42:02.976 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.63139.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64689.log 2023-11-28T18:42:02.978 INFO:teuthology.orchestra.run.smithi105.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.0.63065.log.gz 2023-11-28T18:42:02.979 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48915.log 2023-11-28T18:42:02.979 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64689.log: -- replaced with /var/log/ceph/ceph-client.admin.63139.log.gz 2023-11-28T18:42:02.981 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51095.log 2023-11-28T18:42:02.982 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48915.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48915.log.gz 2023-11-28T18:42:02.982 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64689.log.gz 2023-11-28T18:42:02.984 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44876.log 2023-11-28T18:42:02.985 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51095.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.51095.log.gz 2023-11-28T18:42:02.987 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60710.log 2023-11-28T18:42:02.989 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53805.log 2023-11-28T18:42:02.990 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60710.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44876.log.gz 2023-11-28T18:42:02.993 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3%/var/log/ceph/ceph-client.admin.53805.log: -- replaced with /var/log/ceph/ceph-client.admin.60710.log.gz 2023-11-28T18:42:02.994 INFO:teuthology.orchestra.run.smithi105.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53805.log.gz 2023-11-28T18:42:02.995 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49656.log 2023-11-28T18:42:02.998 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45298.log 2023-11-28T18:42:03.001 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45298.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45298.log.gz 2023-11-28T18:42:03.002 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57863.log 2023-11-28T18:42:03.006 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57863.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57863.log.gz 2023-11-28T18:42:03.006 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49656.log.gz 2023-11-28T18:42:03.006 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62370.log 2023-11-28T18:42:03.009 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62370.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62370.log.gz 2023-11-28T18:42:03.009 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52872.log 2023-11-28T18:42:03.013 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52872.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52872.log.gz 2023-11-28T18:42:03.013 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64317.log 2023-11-28T18:42:03.016 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64317.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64317.log.gz 2023-11-28T18:42:03.016 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46773.log 2023-11-28T18:42:03.019 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46773.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46773.log.gz 2023-11-28T18:42:03.019 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60841.log 2023-11-28T18:42:03.026 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60841.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60841.log.gz 2023-11-28T18:42:03.027 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47963.log 2023-11-28T18:42:03.029 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47963.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47963.log.gz 2023-11-28T18:42:03.029 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59867.log 2023-11-28T18:42:03.029 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.59867.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59867.log.gz 2023-11-28T18:42:03.029 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44238.log 2023-11-28T18:42:03.032 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54151.log 2023-11-28T18:42:03.033 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44238.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44238.log.gz 2023-11-28T18:42:03.037 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46993.log 2023-11-28T18:42:03.037 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54151.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54151.log.gz 2023-11-28T18:42:03.040 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57301.log 2023-11-28T18:42:03.040 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46993.log.gz 2023-11-28T18:42:03.041 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56250.log 2023-11-28T18:42:03.045 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57301.log: /var/log/ceph/ceph-client.admin.56250.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56250.log.gz 2023-11-28T18:42:03.045 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57301.log.gz 2023-11-28T18:42:03.046 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60763.log 2023-11-28T18:42:03.049 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60763.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60763.log.gz 2023-11-28T18:42:03.049 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45713.log 2023-11-28T18:42:03.052 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46967.log 2023-11-28T18:42:03.056 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46967.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46967.log.gz 2023-11-28T18:42:03.056 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45713.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45713.log.gz 2023-11-28T18:42:03.056 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65248.log 2023-11-28T18:42:03.059 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65248.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.65248.log.gz 2023-11-28T18:42:03.059 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63165.log 2023-11-28T18:42:03.063 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.63165.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63165.log.gz 2023-11-28T18:42:03.063 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67644.log 2023-11-28T18:42:03.070 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67644.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67644.log.gz 2023-11-28T18:42:03.071 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56284.log 2023-11-28T18:42:03.076 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66977.log 2023-11-28T18:42:03.078 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56284.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56284.log.gz 2023-11-28T18:42:03.081 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44346.log 2023-11-28T18:42:03.082 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66977.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66977.log.gz 2023-11-28T18:42:03.088 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51340.log 2023-11-28T18:42:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44346.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44346.log.gz 2023-11-28T18:42:03.092 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67670.log 2023-11-28T18:42:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51340.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51340.log.gz 2023-11-28T18:42:03.097 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44185.log 2023-11-28T18:42:03.099 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67670.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67670.log.gz 2023-11-28T18:42:03.100 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44185.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44185.log.gz 2023-11-28T18:42:03.103 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65550.log 2023-11-28T18:42:03.108 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47911.log 2023-11-28T18:42:03.110 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65550.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65550.log.gz 2023-11-28T18:42:03.112 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53629.log 2023-11-28T18:42:03.114 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47911.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47911.log.gz 2023-11-28T18:42:03.115 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56975.log 2023-11-28T18:42:03.120 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53629.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53629.log.gz 2023-11-28T18:42:03.120 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46470.log 2023-11-28T18:42:03.122 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.56975.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56975.log.gz 2023-11-28T18:42:03.124 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66731.log 2023-11-28T18:42:03.126 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46470.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46470.log.gz 2023-11-28T18:42:03.128 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53116.log 2023-11-28T18:42:03.130 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66731.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66731.log.gz 2023-11-28T18:42:03.138 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53116.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53116.log.gz 2023-11-28T18:42:03.139 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65781.log 2023-11-28T18:42:03.144 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70493.log 2023-11-28T18:42:03.146 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65781.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65781.log.gz 2023-11-28T18:42:03.148 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64369.log 2023-11-28T18:42:03.157 INFO:teuthology.orchestra.run.smithi148.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-28T18:42:03.158 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.70493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38342.log 2023-11-28T18:42:03.158 INFO:teuthology.orchestra.run.smithi105.stderr: 95.7% -- replaced with /var/log/ceph/ceph-client.admin.70493.log.gz 2023-11-28T18:42:03.159 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64369.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64369.log.gz 2023-11-28T18:42:03.160 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64663.log 2023-11-28T18:42:03.162 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.38342.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38342.log.gz 2023-11-28T18:42:03.169 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.64663.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64663.log.gz 2023-11-28T18:42:03.171 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52641.log 2023-11-28T18:42:03.176 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65576.log 2023-11-28T18:42:03.178 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52641.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52641.log.gz 2023-11-28T18:42:03.180 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67398.log 2023-11-28T18:42:03.182 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65576.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65576.log.gz 2023-11-28T18:42:03.188 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60435.log 2023-11-28T18:42:03.189 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.67398.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67398.log.gz 2023-11-28T18:42:03.194 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60435.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.60435.log.gz 2023-11-28T18:42:03.195 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45376.log 2023-11-28T18:42:03.200 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54125.log 2023-11-28T18:42:03.202 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45376.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45376.log.gz 2023-11-28T18:42:03.204 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57412.log 2023-11-28T18:42:03.206 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54125.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54125.log.gz 2023-11-28T18:42:03.212 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46338.log 2023-11-28T18:42:03.213 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57412.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57412.log.gz 2023-11-28T18:42:03.216 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62188.log 2023-11-28T18:42:03.218 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46338.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46338.log.gz 2023-11-28T18:42:03.219 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60335.log 2023-11-28T18:42:03.224 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62188.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.62188.log.gz 2023-11-28T18:42:03.226 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60335.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60335.log.gz 2023-11-28T18:42:03.229 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50062.log 2023-11-28T18:42:03.232 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54039.log 2023-11-28T18:42:03.234 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50062.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50062.log.gz 2023-11-28T18:42:03.236 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44464.log 2023-11-28T18:42:03.238 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54039.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54039.log.gz 2023-11-28T18:42:03.240 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70447.log 2023-11-28T18:42:03.242 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44464.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44464.log.gz 2023-11-28T18:42:03.248 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54865.log 2023-11-28T18:42:03.251 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.70447.log: 95.7% -- replaced with /var/log/ceph/ceph-client.admin.70447.log.gz 2023-11-28T18:42:03.252 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57614.log 2023-11-28T18:42:03.254 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54865.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54865.log.gz 2023-11-28T18:42:03.258 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.57614.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57614.log.gz 2023-11-28T18:42:03.259 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60567.log 2023-11-28T18:42:03.264 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66679.log 2023-11-28T18:42:03.266 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.60567.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60567.log.gz 2023-11-28T18:42:03.268 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45921.log 2023-11-28T18:42:03.270 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.66679.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66679.log.gz 2023-11-28T18:42:03.272 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49253.log 2023-11-28T18:42:03.276 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45921.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61274.log 2023-11-28T18:42:03.279 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49253.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45921.log.gz 2023-11-28T18:42:03.280 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44434.log 2023-11-28T18:42:03.282 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.61274.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.61274.log.gz 2023-11-28T18:42:03.283 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49253.log.gz 2023-11-28T18:42:03.289 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44434.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44434.log.gz 2023-11-28T18:42:03.291 INFO:teuthology.orchestra.run.smithi105.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.0.55430.log.gz 2023-11-28T18:42:03.291 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62266.log 2023-11-28T18:42:03.294 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.62266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53358.log 2023-11-28T18:42:03.296 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65807.log 2023-11-28T18:42:03.297 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53358.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.53358.log.gz 2023-11-28T18:42:03.298 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62266.log.gz 2023-11-28T18:42:03.300 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48707.log 2023-11-28T18:42:03.305 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.65807.log: /var/log/ceph/ceph-client.admin.48707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47019.log 2023-11-28T18:42:03.305 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48707.log.gz 2023-11-28T18:42:03.305 INFO:teuthology.orchestra.run.smithi105.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65807.log.gz 2023-11-28T18:42:03.306 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70421.log 2023-11-28T18:42:03.311 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47019.log: /var/log/ceph/ceph-client.admin.70421.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70421.log.gz 2023-11-28T18:42:03.311 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47019.log.gz 2023-11-28T18:42:03.311 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54255.log 2023-11-28T18:42:03.315 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54255.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54255.log.gz 2023-11-28T18:42:03.315 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69036.log 2023-11-28T18:42:03.317 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.69036.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.69036.log.gz 2023-11-28T18:42:03.840 INFO:teuthology.orchestra.run.smithi105.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-28T18:42:04.520 INFO:teuthology.orchestra.run.smithi105.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.67801.log.gz 2023-11-28T18:42:04.545 INFO:teuthology.orchestra.run.smithi148.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-28T18:42:04.642 INFO:teuthology.orchestra.run.smithi148.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-28T18:42:04.884 INFO:teuthology.orchestra.run.smithi148.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-28T18:42:04.886 INFO:teuthology.orchestra.run.smithi148.stderr: 2023-11-28T18:42:04.886 INFO:teuthology.orchestra.run.smithi148.stderr:real 0m3.805s 2023-11-28T18:42:04.886 INFO:teuthology.orchestra.run.smithi148.stderr:user 0m13.666s 2023-11-28T18:42:04.886 INFO:teuthology.orchestra.run.smithi148.stderr:sys 0m1.083s 2023-11-28T18:42:04.938 INFO:teuthology.orchestra.run.smithi105.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-28T18:42:05.110 INFO:teuthology.orchestra.run.smithi105.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-28T18:42:06.119 INFO:teuthology.orchestra.run.smithi105.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-28T18:42:09.230 INFO:teuthology.orchestra.run.smithi105.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-28T18:42:10.925 INFO:teuthology.orchestra.run.smithi105.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-28T18:42:10.927 INFO:teuthology.orchestra.run.smithi105.stderr: 2023-11-28T18:42:10.927 INFO:teuthology.orchestra.run.smithi105.stderr:real 0m9.866s 2023-11-28T18:42:10.927 INFO:teuthology.orchestra.run.smithi105.stderr:user 0m34.534s 2023-11-28T18:42:10.928 INFO:teuthology.orchestra.run.smithi105.stderr:sys 0m3.574s 2023-11-28T18:42:10.929 INFO:tasks.ceph:Archiving logs... 2023-11-28T18:42:10.929 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-11-28_06:23:42-fs-wip-vshankar-testing-20231127.102654-testing-default-smithi/7469459/remote/smithi105/log 2023-11-28T18:42:10.930 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T18:42:15.032 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-11-28_06:23:42-fs-wip-vshankar-testing-20231127.102654-testing-default-smithi/7469459/remote/smithi148/log 2023-11-28T18:42:15.033 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T18:42:16.755 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2758, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2590, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2356, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1611, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-11-28T18:42:16.766 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-28T18:42:16.780 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2758, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2590, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2356, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1611, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-11-28T18:42:16.785 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-28T18:42:16.785 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T18:42:16.793 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T18:42:16.865 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-28T18:42:16.877 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-28T18:42:16.877 DEBUG:teuthology.orchestra.run.smithi105:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T18:42:16.881 DEBUG:teuthology.orchestra.run.smithi148:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T18:42:16.893 INFO:teuthology.orchestra.run.smithi105.stdout: remote refid st t when poll reach delay offset jitter 2023-11-28T18:42:16.893 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================== 2023-11-28T18:42:16.894 INFO:teuthology.orchestra.run.smithi105.stdout:*hv01.front.sepi 44.4.53.2 2 u 54 64 377 0.117 -6.381 0.733 2023-11-28T18:42:16.894 INFO:teuthology.orchestra.run.smithi105.stdout:+hv02.front.sepi 63.231.80.2 3 u 46 64 377 0.080 -6.227 0.950 2023-11-28T18:42:16.894 INFO:teuthology.orchestra.run.smithi105.stdout:+hv03.front.sepi 74.6.168.72 3 u 47 64 377 0.089 -6.268 0.791 2023-11-28T18:42:16.898 INFO:teuthology.orchestra.run.smithi105.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2023-11-28T18:42:16.899 INFO:teuthology.orchestra.run.smithi148.stdout: remote refid st t when poll reach delay offset jitter 2023-11-28T18:42:16.900 INFO:teuthology.orchestra.run.smithi148.stdout:============================================================================== 2023-11-28T18:42:16.900 INFO:teuthology.orchestra.run.smithi148.stdout:*hv01.front.sepi 44.4.53.2 2 u 44 64 377 0.083 -7.785 0.922 2023-11-28T18:42:16.900 INFO:teuthology.orchestra.run.smithi148.stdout:+hv02.front.sepi 63.231.80.2 3 u 56 64 377 0.072 -6.167 1.408 2023-11-28T18:42:16.900 INFO:teuthology.orchestra.run.smithi148.stdout:+hv03.front.sepi 74.6.168.72 3 u 38 64 377 0.100 -8.147 1.071 2023-11-28T18:42:16.900 INFO:teuthology.orchestra.run.smithi148.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2023-11-28T18:42:16.900 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-28T18:42:16.912 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-28T18:42:16.913 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-28T18:42:16.928 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-28T18:42:16.941 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-28T18:42:16.956 INFO:teuthology.task.internal:Duration was 1897.969486 seconds 2023-11-28T18:42:16.957 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-28T18:42:16.971 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-28T18:42:16.972 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T18:42:16.975 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T18:42:17.068 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-28T18:42:17.068 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi105.front.sepia.ceph.com 2023-11-28T18:42:17.068 DEBUG:teuthology.orchestra.run.smithi105:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T18:42:17.092 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi148.front.sepia.ceph.com 2023-11-28T18:42:17.092 DEBUG:teuthology.orchestra.run.smithi148:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T18:42:17.115 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-28T18:42:17.115 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T18:42:17.144 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T18:42:17.197 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-28T18:42:17.198 DEBUG:teuthology.orchestra.run.smithi105:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T18:42:17.227 DEBUG:teuthology.orchestra.run.smithi148:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T18:42:17.310 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-28T18:42:17.323 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-28T18:42:17.323 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T18:42:17.344 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T18:42:17.366 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-28T18:42:17.380 DEBUG:teuthology.orchestra.run.smithi105:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T18:42:17.390 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T18:42:17.404 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = core 2023-11-28T18:42:17.428 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = core 2023-11-28T18:42:17.452 DEBUG:teuthology.orchestra.run.smithi105:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T18:42:17.478 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi105.front.sepia.ceph.com, flagging run as failed 2023-11-28T18:42:17.478 DEBUG:teuthology.orchestra.run.smithi148:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T18:42:17.505 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi148.front.sepia.ceph.com, flagging run as failed 2023-11-28T18:42:17.505 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-28T18:42:17.522 INFO:teuthology.task.internal:Transferring archived files... 2023-11-28T18:42:17.523 DEBUG:teuthology.misc:Transferring archived files from smithi105:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-11-28_06:23:42-fs-wip-vshankar-testing-20231127.102654-testing-default-smithi/7469459/remote/smithi105 2023-11-28T18:42:17.523 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T18:42:25.080 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2023-11-28T18:42:25.092 INFO:teuthology.task.internal: core looks like: /home/teuthworker/archive/vshankar-2023-11-28_06:23:42-fs-wip-vshankar-testing-20231127.102654-testing-default-smithi/7469459/remote/smithi105/coredump/1701196382.67173.core.gz: gzip compressed data, was "1701196382.67173.core", last modified: Tue Nov 28 18:42:17 2023, max compression, original size modulo 2^32 256630784 2023-11-28T18:42:25.093 INFO:teuthology.task.internal:core is compressed, try accessing gzip file ... 2023-11-28T18:42:26.156 INFO:teuthology.task.internal: core looks like: /tmp/tmpbp9ccath: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i a', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/usr/bin/ceph-mds', platform: 'x86_64' 2023-11-28T18:42:26.187 INFO:teuthology.task.internal: dump_program: ceph-mds -f --cluster ceph -i a 2023-11-28T18:42:26.188 DEBUG:teuthology.orchestra.run.smithi105:> which 'ceph-mds -f --cluster ceph -i a' 2023-11-28T18:42:26.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:42:26.199 ERROR:teuthology.run_tasks:Manager failed: internal.archive Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/task/internal/__init__.py", line 392, in archive yield File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2758, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2590, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2356, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1611, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/task/internal/__init__.py", line 420, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/task/internal/__init__.py", line 345, in fetch_binaries_for_coredumps remote_path = remote.sh(['which', dump_program]).rstrip() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 1: "which 'ceph-mds -f --cluster ceph -i a'" 2023-11-28T18:42:26.217 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-28T18:42:26.231 INFO:teuthology.task.internal:Not uploading archives. 2023-11-28T18:42:26.231 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-28T18:42:26.254 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-28T18:42:26.254 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T18:42:26.263 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T18:42:26.269 INFO:teuthology.orchestra.run.smithi105.stdout: 658044 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Nov 28 18:42 /home/ubuntu/cephtest 2023-11-28T18:42:26.270 INFO:teuthology.orchestra.run.smithi105.stdout: 658045 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Nov 28 18:10 /home/ubuntu/cephtest/archive 2023-11-28T18:42:26.270 INFO:teuthology.orchestra.run.smithi105.stdout: 658048 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 18:42 /home/ubuntu/cephtest/archive/syslog 2023-11-28T18:42:26.270 INFO:teuthology.orchestra.run.smithi105.stdout: 658050 4 -rw-rw-rw- 1 ubuntu ubuntu 2310 Nov 28 18:41 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-11-28T18:42:26.270 INFO:teuthology.orchestra.run.smithi105.stdout: 658049 124 -rw-rw-r-- 1 ubuntu ubuntu 126184 Nov 28 18:42 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T18:42:26.270 INFO:teuthology.orchestra.run.smithi105.stdout: 658051 120 -rw-rw-rw- 1 ubuntu ubuntu 121310 Nov 28 18:42 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-11-28T18:42:26.270 INFO:teuthology.orchestra.run.smithi105.stdout: 658046 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 18:33 /home/ubuntu/cephtest/archive/coredump 2023-11-28T18:42:26.270 INFO:teuthology.orchestra.run.smithi105.stdout: 675030 32588 -rw------- 1 root root 256630784 Nov 28 18:33 /home/ubuntu/cephtest/archive/coredump/1701196382.67173.core 2023-11-28T18:42:26.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T18:42:26.272 INFO:teuthology.orchestra.run.smithi105.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-11-28T18:42:26.272 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2758, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2590, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2356, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1611, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2023-11-28T18:42:26.286 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-28T18:42:26.301 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-28T18:42:26.315 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi105 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-11-28T18:42:26.315 INFO:teuthology.nuke:Checking targets against current locks 2023-11-28T18:42:26.324 INFO:teuthology.orchestra.run.smithi148.stdout: 655441 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Nov 28 18:42 /home/ubuntu/cephtest 2023-11-28T18:42:26.324 INFO:teuthology.orchestra.run.smithi148.stdout: 658042 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Nov 28 18:10 /home/ubuntu/cephtest/archive 2023-11-28T18:42:26.324 INFO:teuthology.orchestra.run.smithi148.stdout: 658045 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 18:42 /home/ubuntu/cephtest/archive/syslog 2023-11-28T18:42:26.325 INFO:teuthology.orchestra.run.smithi148.stdout: 658047 4 -rw-rw-rw- 1 ubuntu ubuntu 2169 Nov 28 18:41 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-11-28T18:42:26.325 INFO:teuthology.orchestra.run.smithi148.stdout: 658046 104 -rw-rw-r-- 1 ubuntu ubuntu 103716 Nov 28 18:42 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T18:42:26.325 INFO:teuthology.orchestra.run.smithi148.stdout: 658048 92 -rw-rw-rw- 1 ubuntu ubuntu 90318 Nov 28 18:42 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-11-28T18:42:26.325 INFO:teuthology.orchestra.run.smithi148.stdout: 658043 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 18:32 /home/ubuntu/cephtest/archive/coredump 2023-11-28T18:42:26.325 INFO:teuthology.orchestra.run.smithi148.stdout: 674915 37632 -rw------- 1 root root 245161984 Nov 28 18:32 /home/ubuntu/cephtest/archive/coredump/1701196363.45244.core 2023-11-28T18:42:26.326 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-11-28T18:42:26.351 DEBUG:teuthology.nuke:shortname: smithi105 2023-11-28T18:42:26.351 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T18:42:26.377 DEBUG:teuthology.nuke:shortname: smithi148 2023-11-28T18:42:26.377 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T18:42:26.381 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-11-28_06:23:42-fs-wip-vshankar-testing-20231127.102654-testing-default-smithi/7469459', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 17:58:07.576741', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDZ22c2VmsOKangy/tmCrNzMEGGlu/JeHoweWe97u2UH0HlxTHil6t5VqDGxukuHSHtFDA2TXg5f2HYYjyPSJjY='} 2023-11-28T18:42:26.404 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-11-28_06:23:42-fs-wip-vshankar-testing-20231127.102654-testing-default-smithi/7469459', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 17:58:07.578703', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGxinxnOSbx5MBENicFt1dCm1DSP36mcR2g5+fKgfl6EeU5vDQ38py62JnF5x9dPKj7SFH1W5pzPm1nMdnSnfbY='} 2023-11-28T18:42:26.437 INFO:teuthology.orchestra.console.smithi105:Power off 2023-11-28T18:42:26.437 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T18:42:26.461 DEBUG:teuthology.orchestra.console.smithi105:power off output: Chassis Power Control: Down/Off 2023-11-28T18:42:26.461 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T18:42:26.479 INFO:teuthology.orchestra.console.smithi148:Power off 2023-11-28T18:42:26.479 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T18:42:26.489 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2023-11-28T18:42:26.502 DEBUG:teuthology.orchestra.console.smithi148:power off output: Chassis Power Control: Down/Off 2023-11-28T18:42:26.502 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T18:42:26.526 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2023-11-28T18:42:30.491 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T18:42:30.527 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T18:42:30.604 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2023-11-28T18:42:30.639 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2023-11-28T18:42:34.606 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T18:42:34.640 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T18:42:34.719 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is off 2023-11-28T18:42:34.751 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is off 2023-11-28T18:42:34.820 INFO:teuthology.orchestra.console.smithi105:Power off completed 2023-11-28T18:42:34.852 INFO:teuthology.orchestra.console.smithi148:Power off completed 2023-11-28T18:42:34.958 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/exports} duration: 1897.9694864749908 failure_reason: 'Test failure: test_ephemeral_pin_distribution (tasks.cephfs.test_exports.TestEphemeralPins), test_ephemeral_pin_distribution (tasks.cephfs.test_exports.TestEphemeralPins)' flavor: default mon.a-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 mon.b-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=9d95c8cb01ac45edb20eeae91a06c2ae status: fail success: false 2023-11-28T18:42:34.958 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T18:42:35.042 INFO:teuthology.run:FAIL